2023-10-16T09:31:44.187 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-16T09:31:44.188 DEBUG:teuthology.run:Teuthology command: teuthology --owner mchangir --verbose --name mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --archive /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969 -- /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/orig.config.yaml 2023-10-16T09:31:44.237 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T09:31:44.310 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969 branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: mchangir@redhat.com first_in_suite: false job_id: '7428969' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: c7471068426370a3754c2250e7729280233d552b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c7471068426370a3754c2250e7729280233d552b kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-snap_schedule-support-subvol-and-group-arguments sha1: 7b3d12460841f3cbe9400ba1b68e3e20957ff531 owner: mchangir priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 127 sha1: c7471068426370a3754c2250e7729280233d552b sleep_before_teardown: 0 suite: fs suite_branch: mgr-snap_schedule-support-subvol-and-group-arguments suite_path: /home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa suite_relpath: qa suite_repo: https://github.com/mchangir/ceph.git suite_sha1: 7b3d12460841f3cbe9400ba1b68e3e20957ff531 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_09:20:41 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-16T09:31:44.310 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa; will attempt to use it 2023-10-16T09:31:44.311 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks 2023-10-16T09:31:44.311 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-16T09:31:44.313 INFO:teuthology.task.internal:Checking packages... 2023-10-16T09:31:44.336 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'c7471068426370a3754c2250e7729280233d552b' 2023-10-16T09:31:44.336 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-16T09:31:44.337 INFO:teuthology.packaging:ref: None 2023-10-16T09:31:44.337 INFO:teuthology.packaging:tag: None 2023-10-16T09:31:44.337 INFO:teuthology.packaging:branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments 2023-10-16T09:31:44.338 INFO:teuthology.packaging:sha1: c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:31:44.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments 2023-10-16T09:31:44.609 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6675.gc7471068 2023-10-16T09:31:44.610 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-16T09:31:44.616 INFO:teuthology.task.internal:no buildpackages task found 2023-10-16T09:31:44.616 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-16T09:31:44.638 INFO:teuthology.task.internal:Saving configuration 2023-10-16T09:31:44.655 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-16T09:31:44.660 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T09:31:44.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:53.871675', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T09:31:44.709 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:53.869712', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T09:31:44.710 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-16T09:31:44.715 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-16T09:31:44.716 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-16T09:31:44.716 INFO:teuthology.run_tasks:Running task console_log... 2023-10-16T09:31:44.792 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2490d669d0>, signals=[15]) 2023-10-16T09:31:44.793 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-16T09:31:44.800 INFO:teuthology.task.internal:Opening connections... 2023-10-16T09:31:44.800 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:31:44.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:31:44.871 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:31:44.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:31:44.938 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-16T09:31:44.944 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-10-16T09:31:44.964 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-10-16T09:31:44.964 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-10-16T09:31:45.023 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2023-10-16T09:31:45.023 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.6 (Ootpa)" 2023-10-16T09:31:45.023 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2023-10-16T09:31:45.023 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2023-10-16T09:31:45.024 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.6" 2023-10-16T09:31:45.024 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-10-16T09:31:45.024 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-16T09:31:45.024 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-10-16T09:31:45.025 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-16T09:31:45.025 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2023-10-16T09:31:45.025 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-16T09:31:45.025 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T09:31:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:31:45.026 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-16T09:31:45.026 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-16T09:31:45.026 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-16T09:31:45.026 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-16T09:31:45.027 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-10-16T09:31:45.050 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-10-16T09:31:45.071 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-10-16T09:31:45.071 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-10-16T09:31:45.130 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Red Hat Enterprise Linux" 2023-10-16T09:31:45.130 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8.6 (Ootpa)" 2023-10-16T09:31:45.130 INFO:teuthology.orchestra.run.smithi190.stdout:ID="rhel" 2023-10-16T09:31:45.131 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="fedora" 2023-10-16T09:31:45.131 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8.6" 2023-10-16T09:31:45.131 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-10-16T09:31:45.131 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-16T09:31:45.131 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-10-16T09:31:45.131 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-16T09:31:45.132 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.redhat.com/" 2023-10-16T09:31:45.132 INFO:teuthology.orchestra.run.smithi190.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-16T09:31:45.132 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T09:31:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:31:45.132 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-16T09:31:45.133 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-16T09:31:45.133 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-16T09:31:45.133 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-16T09:31:45.133 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-10-16T09:31:45.150 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-16T09:31:45.161 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-16T09:31:45.168 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-16T09:31:45.168 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-10-16T09:31:45.171 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2023-10-16T09:31:45.189 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-16T09:31:45.195 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-16T09:31:45.195 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-10-16T09:31:45.230 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2023-10-16T09:31:45.256 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-16T09:31:45.308 INFO:teuthology.run_tasks:Running task kernel... 2023-10-16T09:31:45.322 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-16T09:31:45.323 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-16T09:31:45.323 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-16T09:31:45.323 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T09:31:45.324 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T09:31:45.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:31:45.343 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-10-16T09:31:45.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:31:45.344 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2023-10-16T09:31:45.401 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:45.402 INFO:teuthology.task.kernel:Running kernel on smithi181: 4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:45.402 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2023-10-16T09:31:45.403 INFO:teuthology.orchestra.run.smithi190.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:45.403 INFO:teuthology.task.kernel:Running kernel on smithi190: 4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:45.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2023-10-16T09:31:46.591 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-16T09:31:46.591 INFO:teuthology.orchestra.run.smithi181.stdout:Unable to read consumer identity 2023-10-16T09:31:46.623 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-16T09:31:46.624 INFO:teuthology.orchestra.run.smithi190.stdout:Unable to read consumer identity 2023-10-16T09:31:48.133 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-10-16T09:31:48.255 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 8.2 kB/s | 3.5 kB 00:00 2023-10-16T09:31:49.555 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-16T09:31:50.717 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 16 MB 00:02 2023-10-16T09:31:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 469 kB/s | 24 kB 00:00 2023-10-16T09:31:53.841 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-10-16T09:31:53.902 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-16T09:31:53.903 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-10-16T09:31:53.903 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-16T09:31:54.032 DEBUG:teuthology.orchestra.run.smithi181:> echo no | sudo yum reinstall kernel || true 2023-10-16T09:31:54.225 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 453 kB/s | 24 kB 00:00 2023-10-16T09:31:54.364 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-16T09:31:54.364 INFO:teuthology.orchestra.run.smithi181.stdout:Unable to read consumer identity 2023-10-16T09:31:54.622 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:01 ago on Mon 16 Oct 2023 09:31:53 AM UTC. 2023-10-16T09:31:54.664 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-16T09:31:54.705 INFO:teuthology.orchestra.run.smithi181.stderr:Error: No packages marked for reinstall. 2023-10-16T09:31:54.751 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum reinstall -y kernel || true 2023-10-16T09:31:54.903 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-10-16T09:31:54.949 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-10-16T09:31:54.950 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2023-10-16T09:31:54.950 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-10-16T09:31:55.069 DEBUG:teuthology.orchestra.run.smithi190:> echo no | sudo yum reinstall kernel || true 2023-10-16T09:31:55.078 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-16T09:31:55.079 INFO:teuthology.orchestra.run.smithi181.stdout:Unable to read consumer identity 2023-10-16T09:31:55.336 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:02 ago on Mon 16 Oct 2023 09:31:53 AM UTC. 2023-10-16T09:31:55.377 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-16T09:31:55.397 INFO:teuthology.orchestra.run.smithi181.stderr:Error: No packages marked for reinstall. 2023-10-16T09:31:55.400 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-16T09:31:55.400 INFO:teuthology.orchestra.run.smithi190.stdout:Unable to read consumer identity 2023-10-16T09:31:55.444 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2023-10-16T09:31:55.523 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:55.523 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:55.524 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-10-16T09:31:55.524 DEBUG:teuthology.parallel:result is None 2023-10-16T09:31:55.665 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:01 ago on Mon 16 Oct 2023 09:31:54 AM UTC. 2023-10-16T09:31:55.706 INFO:teuthology.orchestra.run.smithi190.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-16T09:31:55.742 INFO:teuthology.orchestra.run.smithi190.stderr:Error: No packages marked for reinstall. 2023-10-16T09:31:55.789 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum reinstall -y kernel || true 2023-10-16T09:31:56.116 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-16T09:31:56.116 INFO:teuthology.orchestra.run.smithi190.stdout:Unable to read consumer identity 2023-10-16T09:31:56.383 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:02 ago on Mon 16 Oct 2023 09:31:54 AM UTC. 2023-10-16T09:31:56.423 INFO:teuthology.orchestra.run.smithi190.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-16T09:31:56.457 INFO:teuthology.orchestra.run.smithi190.stderr:Error: No packages marked for reinstall. 2023-10-16T09:31:56.508 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2023-10-16T09:31:56.589 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:56.590 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-10-16T09:31:56.590 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-10-16T09:31:56.590 DEBUG:teuthology.parallel:result is None 2023-10-16T09:31:56.591 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-16T09:31:56.600 INFO:teuthology.task.internal:Creating test directory... 2023-10-16T09:31:56.600 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T09:31:56.603 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T09:31:56.620 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-16T09:31:56.633 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-16T09:31:56.645 INFO:teuthology.task.internal:Creating archive directory... 2023-10-16T09:31:56.645 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T09:31:56.662 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T09:31:56.686 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-16T09:31:56.692 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-16T09:31:56.693 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T09:31:56.722 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T09:31:56.764 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:31:56.769 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:31:56.784 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:31:56.785 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T09:31:56.787 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-16T09:31:56.794 INFO:teuthology.task.internal:Configuring sudo... 2023-10-16T09:31:56.794 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T09:31:56.797 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T09:31:56.824 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-16T09:31:56.832 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-16T09:31:56.832 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T09:31:56.867 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T09:31:56.883 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:31:56.968 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:31:57.018 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:31:57.103 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:31:57.144 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:31:57.145 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T09:31:57.211 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:31:57.256 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T09:31:57.303 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:31:57.388 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T09:31:57.429 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:31:57.430 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T09:31:57.497 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-10-16T09:31:57.499 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2023-10-16T09:31:57.558 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T09:31:57.591 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T09:31:57.919 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-16T09:31:57.927 INFO:teuthology.task.internal:Starting timer... 2023-10-16T09:31:57.928 INFO:teuthology.run_tasks:Running task pcp... 2023-10-16T09:31:57.956 INFO:teuthology.run_tasks:Running task selinux... 2023-10-16T09:31:57.972 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-16T09:31:57.973 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-10-16T09:31:58.029 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2023-10-16T09:31:58.031 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2023-10-16T09:31:58.105 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2023-10-16T09:31:58.107 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-16T09:31:58.107 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-10-16T09:31:58.132 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-10-16T09:31:58.132 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2023-10-16T09:31:58.159 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2023-10-16T09:31:58.160 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi181.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2023-10-16T09:31:58.160 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T09:31:58.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:31:58.199 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T09:31:58.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:31:58.230 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-16T09:31:58.230 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-10-16T09:31:58.274 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2023-10-16T09:31:58.303 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-16T09:31:58.314 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-16T09:31:58.315 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-16T09:31:58.332 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-16T09:31:58.333 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi181.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-16T09:40:46.082 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2023-10-16T09:40:46.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-16T09:40:46.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:40:46.154 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-10-16T09:40:46.223 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-16T09:40:46.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-10-16T09:40:46.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T09:40:46.292 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-10-16T09:40:46.364 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-10-16T09:40:46.364 INFO:teuthology.run_tasks:Running task clock... 2023-10-16T09:40:46.376 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-16T09:40:46.377 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T09:40:46.377 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T09:40:46.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T09:40:46.380 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T09:40:46.408 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-16T09:40:46.423 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-16T09:40:46.436 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-16T09:40:46.450 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-16T09:40:46.490 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-10-16T09:40:46.502 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-10-16T09:40:46.517 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2023-10-16T09:40:46.518 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-16T09:40:46.527 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2023-10-16T09:40:46.529 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-16T09:40:46.540 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-16T09:40:46.554 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-16T09:40:46.600 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-10-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-10-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-10-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T09:40:46.644 INFO:teuthology.run_tasks:Running task install... 2023-10-16T09:40:46.652 DEBUG:teuthology.task.install:project ceph 2023-10-16T09:40:46.653 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}} 2023-10-16T09:40:46.653 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'} 2023-10-16T09:40:46.653 INFO:teuthology.task.install:Using flavor: default 2023-10-16T09:40:46.659 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-16T09:40:46.659 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-16T09:40:46.660 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'tag': None, 'wait_for_package': False} 2023-10-16T09:40:46.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:40:46.662 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'tag': None, 'wait_for_package': False} 2023-10-16T09:40:46.663 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:40:46.811 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/ 2023-10-16T09:40:46.812 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6675.gc7471068 2023-10-16T09:40:46.877 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/ 2023-10-16T09:40:46.877 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6675.gc7471068 2023-10-16T09:40:46.929 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-16T09:40:46.929 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:40:46.930 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-16T09:40:46.957 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-16T09:40:46.957 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c7471068426370a3754c2250e7729280233d552b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-16T09:40:47.024 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-16T09:40:47.075 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-16T09:40:47.075 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:40:47.076 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-16T09:40:47.100 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-16T09:40:47.104 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-16T09:40:47.105 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c7471068426370a3754c2250e7729280233d552b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-16T09:40:47.166 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2023-10-16T09:40:47.167 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2023-10-16T09:40:47.173 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-16T09:40:47.251 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-16T09:40:47.317 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-10-16T09:40:47.319 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-10-16T09:40:47.537 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-16T09:40:47.689 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-16T09:40:47.765 INFO:teuthology.orchestra.run.smithi190.stdout:44 files removed 2023-10-16T09:40:47.813 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-16T09:40:47.916 INFO:teuthology.orchestra.run.smithi181.stdout:44 files removed 2023-10-16T09:40:47.962 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-16T09:40:48.144 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-10-16T09:40:48.292 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-16T09:40:48.831 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 265 kB/s | 81 kB 00:00 2023-10-16T09:40:49.044 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 203 kB/s | 81 kB 00:00 2023-10-16T09:40:49.207 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-10-16T09:40:49.395 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-16T09:40:49.532 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 4.9 kB/s | 1.5 kB 00:00 2023-10-16T09:40:49.691 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-16T09:40:49.812 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-10-16T09:40:49.979 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-16T09:40:51.580 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-10-16T09:40:53.518 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 16 MB 00:03 2023-10-16T09:40:55.080 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 417 kB/s | 24 kB 00:00 2023-10-16T09:40:56.007 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-10-16T09:40:57.009 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 423 kB/s | 24 kB 00:00 2023-10-16T09:40:57.910 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-10-16T09:41:04.107 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-10-16T09:41:06.053 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-10-16T09:41:15.023 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T09:41:15.024 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T09:41:15.038 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-16T09:41:15.039 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-16T09:41:15.040 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-16T09:41:15.042 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-16T09:41:15.044 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-16T09:41:15.047 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-16T09:41:15.048 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-16T09:41:15.177 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-16T09:41:15.190 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-10-16T09:41:15.191 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-10-16T09:41:15.191 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-10-16T09:41:15.191 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-10-16T09:41:15.191 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-16T09:41:15.191 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-16T09:41:15.192 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.0.0-6675.gc7471068.el8 ceph 6.4 k 2023-10-16T09:41:15.192 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.0.0-6675.gc7471068.el8 ceph 5.4 M 2023-10-16T09:41:15.192 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.0.0-6675.gc7471068.el8 ceph 919 k 2023-10-16T09:41:15.192 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6675.gc7471068.el8 ceph 167 k 2023-10-16T09:41:15.193 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.0.0-6675.gc7471068.el8 ceph 1.6 M 2023-10-16T09:41:15.193 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 132 k 2023-10-16T09:41:15.193 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 1.8 M 2023-10-16T09:41:15.193 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 7.4 M 2023-10-16T09:41:15.193 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 47 k 2023-10-16T09:41:15.194 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.0.0-6675.gc7471068.el8 ceph 14 M 2023-10-16T09:41:15.194 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.0.0-6675.gc7471068.el8 ceph 49 M 2023-10-16T09:41:15.194 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 261 k 2023-10-16T09:41:15.194 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 192 k 2023-10-16T09:41:15.194 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:18.0.0-6675.gc7471068.el8 ceph 261 k 2023-10-16T09:41:15.195 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 18 k 2023-10-16T09:41:15.195 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-16T09:41:15.195 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-16T09:41:15.195 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-16T09:41:15.196 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.0.0-6675.gc7471068.el8 ceph 31 k 2023-10-16T09:41:15.196 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 839 k 2023-10-16T09:41:15.196 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.0.0-6675.gc7471068.el8 ceph 139 k 2023-10-16T09:41:15.196 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-16T09:41:15.196 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-16T09:41:15.197 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-16T09:41:15.197 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-16T09:41:15.197 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-16T09:41:15.197 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-16T09:41:15.197 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.0.0-6675.gc7471068.el8 ceph 197 k 2023-10-16T09:41:15.198 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.0.0-6675.gc7471068.el8 ceph 371 k 2023-10-16T09:41:15.198 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.0.0-6675.gc7471068.el8 ceph 364 k 2023-10-16T09:41:15.198 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.0.0-6675.gc7471068.el8 ceph 107 k 2023-10-16T09:41:15.198 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.0.0-6675.gc7471068.el8 ceph 94 k 2023-10-16T09:41:15.199 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.0.0-6675.gc7471068.el8 ceph 3.6 M 2023-10-16T09:41:15.199 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.0.0-6675.gc7471068.el8 ceph 180 k 2023-10-16T09:41:15.199 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-16T09:41:15.199 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-10-16T09:41:15.199 INFO:teuthology.orchestra.run.smithi181.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-16T09:41:15.200 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 3.7 M 2023-10-16T09:41:15.200 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 3.7 M 2023-10-16T09:41:15.200 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-10-16T09:41:15.200 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-16T09:41:15.200 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.0.0-6675.gc7471068.el8 ceph 23 M 2023-10-16T09:41:15.201 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 23 k 2023-10-16T09:41:15.201 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.0.0-6675.gc7471068.el8 ceph 2.3 M 2023-10-16T09:41:15.201 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 243 k 2023-10-16T09:41:15.201 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.0.0-6675.gc7471068.el8 ceph 4.6 M 2023-10-16T09:41:15.201 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.0.0-6675.gc7471068.el8 ceph 18 M 2023-10-16T09:41:15.202 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 15 k 2023-10-16T09:41:15.202 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.0.0-6675.gc7471068.el8 ceph 24 k 2023-10-16T09:41:15.202 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-16T09:41:15.202 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-16T09:41:15.202 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-16T09:41:15.202 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-16T09:41:15.203 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-16T09:41:15.203 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.0.0-6675.gc7471068.el8 ceph 184 k 2023-10-16T09:41:15.203 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-16T09:41:15.203 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-16T09:41:15.203 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-16T09:41:15.204 INFO:teuthology.orchestra.run.smithi181.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-16T09:41:15.204 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-16T09:41:15.204 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-16T09:41:15.204 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 518 k 2023-10-16T09:41:15.205 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-16T09:41:15.205 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 6.9 M 2023-10-16T09:41:15.205 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-16T09:41:15.205 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-16T09:41:15.205 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-16T09:41:15.206 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-16T09:41:15.206 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-16T09:41:15.206 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-16T09:41:15.206 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-16T09:41:15.206 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-16T09:41:15.206 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-16T09:41:15.207 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-16T09:41:15.207 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-16T09:41:15.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-16T09:41:15.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-16T09:41:15.207 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-16T09:41:15.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-16T09:41:15.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-16T09:41:15.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.0.0-6675.gc7471068.el8 ceph 45 k 2023-10-16T09:41:15.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.0.0-6675.gc7471068.el8 ceph 126 k 2023-10-16T09:41:15.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-16T09:41:15.208 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-16T09:41:15.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-16T09:41:15.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-16T09:41:15.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-16T09:41:15.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-16T09:41:15.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-16T09:41:15.209 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-16T09:41:15.210 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-16T09:41:15.210 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-16T09:41:15.210 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-16T09:41:15.210 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T09:41:15.210 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-16T09:41:15.211 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-16T09:41:15.211 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-16T09:41:15.211 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-16T09:41:15.211 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-16T09:41:15.211 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-16T09:41:15.212 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-16T09:41:15.212 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-16T09:41:15.212 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-16T09:41:15.212 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-16T09:41:15.212 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-16T09:41:15.212 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-16T09:41:15.213 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-16T09:41:15.213 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-16T09:41:15.213 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-16T09:41:15.213 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-16T09:41:15.213 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-16T09:41:15.214 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-16T09:41:15.214 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T09:41:15.214 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-16T09:41:15.214 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-16T09:41:15.214 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-16T09:41:15.215 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-16T09:41:15.215 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-16T09:41:15.215 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-16T09:41:15.215 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-16T09:41:15.215 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-16T09:41:15.216 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-16T09:41:15.216 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-16T09:41:15.216 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-16T09:41:15.216 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-16T09:41:15.216 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-16T09:41:15.217 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-16T09:41:15.217 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-16T09:41:15.217 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-16T09:41:15.217 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-16T09:41:15.217 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-16T09:41:15.218 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-10-16T09:41:15.218 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql 10 2023-10-16T09:41:15.218 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:41:15.218 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-10-16T09:41:15.218 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-10-16T09:41:15.219 INFO:teuthology.orchestra.run.smithi181.stdout:Install 128 Packages 2023-10-16T09:41:15.219 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 3 Packages 2023-10-16T09:41:15.219 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:41:15.220 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 209 M 2023-10-16T09:41:15.220 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-10-16T09:41:15.424 INFO:teuthology.orchestra.run.smithi181.stdout:(1/131): ceph-18.0.0-6675.gc7471068.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2023-10-16T09:41:15.616 INFO:teuthology.orchestra.run.smithi181.stdout:(2/131): ceph-fuse-18.0.0-6675.gc7471068.el8.x8 4.7 MB/s | 919 kB 00:00 2023-10-16T09:41:15.666 INFO:teuthology.orchestra.run.smithi181.stdout:(3/131): ceph-immutable-object-cache-18.0.0-667 3.3 MB/s | 167 kB 00:00 2023-10-16T09:41:15.875 INFO:teuthology.orchestra.run.smithi181.stdout:(4/131): ceph-mds-18.0.0-6675.gc7471068.el8.x86 11 MB/s | 2.3 MB 00:00 2023-10-16T09:41:16.009 INFO:teuthology.orchestra.run.smithi181.stdout:(5/131): ceph-mgr-18.0.0-6675.gc7471068.el8.x86 12 MB/s | 1.6 MB 00:00 2023-10-16T09:41:16.284 INFO:teuthology.orchestra.run.smithi181.stdout:(6/131): ceph-mon-18.0.0-6675.gc7471068.el8.x86 17 MB/s | 4.6 MB 00:00 2023-10-16T09:41:16.967 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T09:41:16.968 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T09:41:16.982 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-16T09:41:16.983 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-16T09:41:16.984 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-16T09:41:16.986 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-16T09:41:16.988 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-16T09:41:16.991 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-16T09:41:16.992 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-16T09:41:17.123 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-10-16T09:41:17.135 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-10-16T09:41:17.136 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2023-10-16T09:41:17.136 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-10-16T09:41:17.136 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2023-10-16T09:41:17.136 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-16T09:41:17.137 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-16T09:41:17.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:18.0.0-6675.gc7471068.el8 ceph 6.4 k 2023-10-16T09:41:17.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:18.0.0-6675.gc7471068.el8 ceph 5.4 M 2023-10-16T09:41:17.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:18.0.0-6675.gc7471068.el8 ceph 919 k 2023-10-16T09:41:17.137 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6675.gc7471068.el8 ceph 167 k 2023-10-16T09:41:17.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:18.0.0-6675.gc7471068.el8 ceph 1.6 M 2023-10-16T09:41:17.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 132 k 2023-10-16T09:41:17.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 1.8 M 2023-10-16T09:41:17.138 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 7.4 M 2023-10-16T09:41:17.139 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 47 k 2023-10-16T09:41:17.139 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:18.0.0-6675.gc7471068.el8 ceph 14 M 2023-10-16T09:41:17.139 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:18.0.0-6675.gc7471068.el8 ceph 49 M 2023-10-16T09:41:17.139 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 261 k 2023-10-16T09:41:17.140 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 192 k 2023-10-16T09:41:17.140 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:18.0.0-6675.gc7471068.el8 ceph 261 k 2023-10-16T09:41:17.140 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 18 k 2023-10-16T09:41:17.140 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-16T09:41:17.140 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-16T09:41:17.141 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-16T09:41:17.141 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:18.0.0-6675.gc7471068.el8 ceph 31 k 2023-10-16T09:41:17.141 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 839 k 2023-10-16T09:41:17.141 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:18.0.0-6675.gc7471068.el8 ceph 139 k 2023-10-16T09:41:17.142 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-16T09:41:17.142 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-16T09:41:17.142 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-16T09:41:17.142 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-16T09:41:17.142 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-16T09:41:17.143 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-16T09:41:17.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:18.0.0-6675.gc7471068.el8 ceph 197 k 2023-10-16T09:41:17.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:18.0.0-6675.gc7471068.el8 ceph 371 k 2023-10-16T09:41:17.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:18.0.0-6675.gc7471068.el8 ceph 364 k 2023-10-16T09:41:17.143 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:18.0.0-6675.gc7471068.el8 ceph 107 k 2023-10-16T09:41:17.144 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:18.0.0-6675.gc7471068.el8 ceph 94 k 2023-10-16T09:41:17.144 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:18.0.0-6675.gc7471068.el8 ceph 3.6 M 2023-10-16T09:41:17.144 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:18.0.0-6675.gc7471068.el8 ceph 180 k 2023-10-16T09:41:17.144 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-16T09:41:17.145 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2023-10-16T09:41:17.145 INFO:teuthology.orchestra.run.smithi190.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-16T09:41:17.145 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 3.7 M 2023-10-16T09:41:17.146 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 3.7 M 2023-10-16T09:41:17.146 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2023-10-16T09:41:17.146 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-16T09:41:17.146 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:18.0.0-6675.gc7471068.el8 ceph 23 M 2023-10-16T09:41:17.146 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 23 k 2023-10-16T09:41:17.147 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:18.0.0-6675.gc7471068.el8 ceph 2.3 M 2023-10-16T09:41:17.147 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 243 k 2023-10-16T09:41:17.147 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:18.0.0-6675.gc7471068.el8 ceph 4.6 M 2023-10-16T09:41:17.147 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:18.0.0-6675.gc7471068.el8 ceph 18 M 2023-10-16T09:41:17.148 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6675.gc7471068.el8 ceph-noarch 15 k 2023-10-16T09:41:17.148 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:18.0.0-6675.gc7471068.el8 ceph 24 k 2023-10-16T09:41:17.148 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-16T09:41:17.148 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-16T09:41:17.149 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-16T09:41:17.149 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-16T09:41:17.149 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-16T09:41:17.149 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:18.0.0-6675.gc7471068.el8 ceph 184 k 2023-10-16T09:41:17.150 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-16T09:41:17.150 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-16T09:41:17.150 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-16T09:41:17.150 INFO:teuthology.orchestra.run.smithi190.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-16T09:41:17.150 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-16T09:41:17.151 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-16T09:41:17.151 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 518 k 2023-10-16T09:41:17.151 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-16T09:41:17.151 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:18.0.0-6675.gc7471068.el8 ceph 6.9 M 2023-10-16T09:41:17.152 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-16T09:41:17.152 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-16T09:41:17.152 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-16T09:41:17.152 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-16T09:41:17.153 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-16T09:41:17.153 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-16T09:41:17.153 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-16T09:41:17.153 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-16T09:41:17.153 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-16T09:41:17.154 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-16T09:41:17.154 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-16T09:41:17.154 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-16T09:41:17.154 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-16T09:41:17.155 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-16T09:41:17.155 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-16T09:41:17.155 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-16T09:41:17.155 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:18.0.0-6675.gc7471068.el8 ceph 45 k 2023-10-16T09:41:17.155 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:18.0.0-6675.gc7471068.el8 ceph 126 k 2023-10-16T09:41:17.156 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-16T09:41:17.156 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-16T09:41:17.156 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-16T09:41:17.156 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-16T09:41:17.157 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-16T09:41:17.157 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-16T09:41:17.157 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-16T09:41:17.157 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-16T09:41:17.158 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-16T09:41:17.158 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-16T09:41:17.158 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-16T09:41:17.158 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T09:41:17.158 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-16T09:41:17.159 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-16T09:41:17.159 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-16T09:41:17.159 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-16T09:41:17.159 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-16T09:41:17.160 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-16T09:41:17.160 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-16T09:41:17.160 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-16T09:41:17.160 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-16T09:41:17.160 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-16T09:41:17.161 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-16T09:41:17.161 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-16T09:41:17.161 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-16T09:41:17.161 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-16T09:41:17.161 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-16T09:41:17.162 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-16T09:41:17.162 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-16T09:41:17.162 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-16T09:41:17.162 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T09:41:17.163 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-16T09:41:17.163 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-16T09:41:17.163 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-16T09:41:17.163 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-16T09:41:17.163 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-16T09:41:17.164 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-16T09:41:17.164 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-16T09:41:17.164 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-16T09:41:17.164 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-16T09:41:17.165 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-16T09:41:17.165 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-16T09:41:17.165 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-16T09:41:17.165 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-16T09:41:17.166 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-16T09:41:17.166 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-16T09:41:17.166 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-16T09:41:17.166 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-16T09:41:17.167 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-16T09:41:17.167 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling module streams: 2023-10-16T09:41:17.167 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql 10 2023-10-16T09:41:17.167 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:41:17.167 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-10-16T09:41:17.168 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-10-16T09:41:17.168 INFO:teuthology.orchestra.run.smithi190.stdout:Install 128 Packages 2023-10-16T09:41:17.168 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 3 Packages 2023-10-16T09:41:17.168 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:41:17.169 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 209 M 2023-10-16T09:41:17.169 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2023-10-16T09:41:17.293 INFO:teuthology.orchestra.run.smithi181.stdout:(7/131): ceph-osd-18.0.0-6675.gc7471068.el8.x86 18 MB/s | 18 MB 00:01 2023-10-16T09:41:17.385 INFO:teuthology.orchestra.run.smithi181.stdout:(8/131): ceph-base-18.0.0-6675.gc7471068.el8.x8 2.6 MB/s | 5.4 MB 00:02 2023-10-16T09:41:17.403 INFO:teuthology.orchestra.run.smithi190.stdout:(1/131): ceph-18.0.0-6675.gc7471068.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2023-10-16T09:41:17.485 INFO:teuthology.orchestra.run.smithi181.stdout:(9/131): ceph-selinux-18.0.0-6675.gc7471068.el8 243 kB/s | 24 kB 00:00 2023-10-16T09:41:18.119 INFO:teuthology.orchestra.run.smithi181.stdout:(10/131): ceph-radosgw-18.0.0-6675.gc7471068.el 17 MB/s | 14 MB 00:00 2023-10-16T09:41:18.185 INFO:teuthology.orchestra.run.smithi181.stdout:(11/131): cephfs-mirror-18.0.0-6675.gc7471068.e 3.8 MB/s | 261 kB 00:00 2023-10-16T09:41:18.227 INFO:teuthology.orchestra.run.smithi181.stdout:(12/131): libcephfs-devel-18.0.0-6675.gc7471068 748 kB/s | 31 kB 00:00 2023-10-16T09:41:18.311 INFO:teuthology.orchestra.run.smithi181.stdout:(13/131): libcephfs2-18.0.0-6675.gc7471068.el8. 9.9 MB/s | 839 kB 00:00 2023-10-16T09:41:18.361 INFO:teuthology.orchestra.run.smithi181.stdout:(14/131): libcephsqlite-18.0.0-6675.gc7471068.e 3.6 MB/s | 184 kB 00:00 2023-10-16T09:41:18.420 INFO:teuthology.orchestra.run.smithi181.stdout:(15/131): librados-devel-18.0.0-6675.gc7471068. 2.3 MB/s | 139 kB 00:00 2023-10-16T09:41:18.504 INFO:teuthology.orchestra.run.smithi181.stdout:(16/131): libradosstriper1-18.0.0-6675.gc747106 6.1 MB/s | 518 kB 00:00 2023-10-16T09:41:18.753 INFO:teuthology.orchestra.run.smithi190.stdout:(2/131): ceph-fuse-18.0.0-6675.gc7471068.el8.x8 681 kB/s | 919 kB 00:01 2023-10-16T09:41:19.070 INFO:teuthology.orchestra.run.smithi190.stdout:(3/131): ceph-immutable-object-cache-18.0.0-667 529 kB/s | 167 kB 00:00 2023-10-16T09:41:19.079 INFO:teuthology.orchestra.run.smithi181.stdout:(17/131): librgw2-18.0.0-6675.gc7471068.el8.x86 12 MB/s | 6.9 MB 00:00 2023-10-16T09:41:19.137 INFO:teuthology.orchestra.run.smithi181.stdout:(18/131): python3-ceph-argparse-18.0.0-6675.gc7 768 kB/s | 45 kB 00:00 2023-10-16T09:41:19.196 INFO:teuthology.orchestra.run.smithi181.stdout:(19/131): python3-ceph-common-18.0.0-6675.gc747 2.1 MB/s | 126 kB 00:00 2023-10-16T09:41:19.263 INFO:teuthology.orchestra.run.smithi181.stdout:(20/131): python3-cephfs-18.0.0-6675.gc7471068. 2.9 MB/s | 197 kB 00:00 2023-10-16T09:41:19.330 INFO:teuthology.orchestra.run.smithi181.stdout:(21/131): python3-rados-18.0.0-6675.gc7471068.e 5.4 MB/s | 371 kB 00:00 2023-10-16T09:41:19.397 INFO:teuthology.orchestra.run.smithi181.stdout:(22/131): python3-rbd-18.0.0-6675.gc7471068.el8 5.3 MB/s | 364 kB 00:00 2023-10-16T09:41:19.464 INFO:teuthology.orchestra.run.smithi181.stdout:(23/131): python3-rgw-18.0.0-6675.gc7471068.el8 1.6 MB/s | 107 kB 00:00 2023-10-16T09:41:19.523 INFO:teuthology.orchestra.run.smithi181.stdout:(24/131): rbd-fuse-18.0.0-6675.gc7471068.el8.x8 1.6 MB/s | 94 kB 00:00 2023-10-16T09:41:19.898 INFO:teuthology.orchestra.run.smithi181.stdout:(25/131): rbd-mirror-18.0.0-6675.gc7471068.el8. 9.6 MB/s | 3.6 MB 00:00 2023-10-16T09:41:19.965 INFO:teuthology.orchestra.run.smithi181.stdout:(26/131): rbd-nbd-18.0.0-6675.gc7471068.el8.x86 2.6 MB/s | 180 kB 00:00 2023-10-16T09:41:20.015 INFO:teuthology.orchestra.run.smithi181.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6675.g 463 kB/s | 23 kB 00:00 2023-10-16T09:41:20.082 INFO:teuthology.orchestra.run.smithi181.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6675.gc747106 1.9 MB/s | 132 kB 00:00 2023-10-16T09:41:20.358 INFO:teuthology.orchestra.run.smithi181.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6675.gc7471 6.4 MB/s | 1.8 MB 00:00 2023-10-16T09:41:20.454 INFO:teuthology.orchestra.run.smithi190.stdout:(4/131): ceph-base-18.0.0-6675.gc7471068.el8.x8 1.6 MB/s | 5.4 MB 00:03 2023-10-16T09:41:21.233 INFO:teuthology.orchestra.run.smithi181.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 8.5 MB/s | 7.4 MB 00:00 2023-10-16T09:41:21.300 INFO:teuthology.orchestra.run.smithi181.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6675.gc7 3.6 MB/s | 243 kB 00:00 2023-10-16T09:41:21.379 INFO:teuthology.orchestra.run.smithi190.stdout:(5/131): ceph-mgr-18.0.0-6675.gc7471068.el8.x86 1.8 MB/s | 1.6 MB 00:00 2023-10-16T09:41:21.392 INFO:teuthology.orchestra.run.smithi181.stdout:(32/131): ceph-mgr-rook-18.0.0-6675.gc7471068.e 517 kB/s | 47 kB 00:00 2023-10-16T09:41:21.421 INFO:teuthology.orchestra.run.smithi190.stdout:(6/131): ceph-mds-18.0.0-6675.gc7471068.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-10-16T09:41:21.459 INFO:teuthology.orchestra.run.smithi181.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6675.gc 218 kB/s | 15 kB 00:00 2023-10-16T09:41:21.718 INFO:teuthology.orchestra.run.smithi181.stdout:(34/131): ceph-common-18.0.0-6675.gc7471068.el8 3.5 MB/s | 23 MB 00:06 2023-10-16T09:41:21.743 INFO:teuthology.orchestra.run.smithi181.stdout:(35/131): ceph-volume-18.0.0-6675.gc7471068.el8 922 kB/s | 261 kB 00:00 2023-10-16T09:41:21.826 INFO:teuthology.orchestra.run.smithi181.stdout:(36/131): cephfs-top-18.0.0-6675.gc7471068.el8. 217 kB/s | 18 kB 00:00 2023-10-16T09:41:21.852 INFO:teuthology.orchestra.run.smithi181.stdout:(37/131): cephadm-18.0.0-6675.gc7471068.el8.noa 1.4 MB/s | 192 kB 00:00 2023-10-16T09:41:22.019 INFO:teuthology.orchestra.run.smithi181.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 353 kB/s | 59 kB 00:00 2023-10-16T09:41:22.061 INFO:teuthology.orchestra.run.smithi181.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-10-16T09:41:22.086 INFO:teuthology.orchestra.run.smithi181.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-10-16T09:41:22.136 INFO:teuthology.orchestra.run.smithi181.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 876 kB/s | 44 kB 00:00 2023-10-16T09:41:22.170 INFO:teuthology.orchestra.run.smithi181.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-10-16T09:41:22.212 INFO:teuthology.orchestra.run.smithi181.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-10-16T09:41:22.237 INFO:teuthology.orchestra.run.smithi181.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 493 kB/s | 33 kB 00:00 2023-10-16T09:41:22.262 INFO:teuthology.orchestra.run.smithi181.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-10-16T09:41:22.296 INFO:teuthology.orchestra.run.smithi190.stdout:(7/131): ceph-common-18.0.0-6675.gc7471068.el8. 4.4 MB/s | 23 MB 00:05 2023-10-16T09:41:22.304 INFO:teuthology.orchestra.run.smithi181.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-10-16T09:41:22.354 INFO:teuthology.orchestra.run.smithi181.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2023-10-16T09:41:22.380 INFO:teuthology.orchestra.run.smithi181.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-10-16T09:41:22.414 INFO:teuthology.orchestra.run.smithi181.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 189 kB/s | 11 kB 00:00 2023-10-16T09:41:22.472 INFO:teuthology.orchestra.run.smithi181.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 190 kB/s | 18 kB 00:00 2023-10-16T09:41:22.514 INFO:teuthology.orchestra.run.smithi181.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-10-16T09:41:22.556 INFO:teuthology.orchestra.run.smithi181.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-10-16T09:41:22.598 INFO:teuthology.orchestra.run.smithi181.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-10-16T09:41:22.632 INFO:teuthology.orchestra.run.smithi181.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2023-10-16T09:41:22.830 INFO:teuthology.orchestra.run.smithi190.stdout:(8/131): ceph-mon-18.0.0-6675.gc7471068.el8.x86 3.2 MB/s | 4.6 MB 00:01 2023-10-16T09:41:22.888 INFO:teuthology.orchestra.run.smithi190.stdout:(9/131): ceph-selinux-18.0.0-6675.gc7471068.el8 419 kB/s | 24 kB 00:00 2023-10-16T09:41:23.049 INFO:teuthology.orchestra.run.smithi181.stdout:(55/131): ceph-test-18.0.0-6675.gc7471068.el8.x 8.8 MB/s | 49 MB 00:05 2023-10-16T09:41:23.090 INFO:teuthology.orchestra.run.smithi181.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 34 kB/s | 16 kB 00:00 2023-10-16T09:41:23.116 INFO:teuthology.orchestra.run.smithi181.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 505 kB/s | 34 kB 00:00 2023-10-16T09:41:23.141 INFO:teuthology.orchestra.run.smithi181.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 521 kB/s | 283 kB 00:00 2023-10-16T09:41:23.166 INFO:teuthology.orchestra.run.smithi181.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-10-16T09:41:23.192 INFO:teuthology.orchestra.run.smithi181.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-10-16T09:41:23.208 INFO:teuthology.orchestra.run.smithi181.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 572 kB/s | 24 kB 00:00 2023-10-16T09:41:23.226 INFO:teuthology.orchestra.run.smithi181.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 874 kB/s | 29 kB 00:00 2023-10-16T09:41:23.251 INFO:teuthology.orchestra.run.smithi181.stdout:(63/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-10-16T09:41:23.268 INFO:teuthology.orchestra.run.smithi181.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 467 kB/s | 27 kB 00:00 2023-10-16T09:41:23.301 INFO:teuthology.orchestra.run.smithi181.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.8 MB/s | 61 kB 00:00 2023-10-16T09:41:23.335 INFO:teuthology.orchestra.run.smithi181.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-10-16T09:41:23.360 INFO:teuthology.orchestra.run.smithi181.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-10-16T09:41:23.377 INFO:teuthology.orchestra.run.smithi181.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 549 kB/s | 23 kB 00:00 2023-10-16T09:41:23.402 INFO:teuthology.orchestra.run.smithi181.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 847 kB/s | 85 kB 00:00 2023-10-16T09:41:23.436 INFO:teuthology.orchestra.run.smithi181.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-10-16T09:41:23.495 INFO:teuthology.orchestra.run.smithi181.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-10-16T09:41:23.520 INFO:teuthology.orchestra.run.smithi181.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.5 MB/s | 200 kB 00:00 2023-10-16T09:41:23.545 INFO:teuthology.orchestra.run.smithi181.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 850 kB/s | 43 kB 00:00 2023-10-16T09:41:23.570 INFO:teuthology.orchestra.run.smithi181.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-16T09:41:23.596 INFO:teuthology.orchestra.run.smithi181.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 961 kB/s | 48 kB 00:00 2023-10-16T09:41:23.629 INFO:teuthology.orchestra.run.smithi181.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-10-16T09:41:23.655 INFO:teuthology.orchestra.run.smithi181.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 729 kB/s | 43 kB 00:00 2023-10-16T09:41:23.680 INFO:teuthology.orchestra.run.smithi181.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-10-16T09:41:23.730 INFO:teuthology.orchestra.run.smithi181.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-10-16T09:41:23.764 INFO:teuthology.orchestra.run.smithi181.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.5 MB/s | 560 kB 00:00 2023-10-16T09:41:23.789 INFO:teuthology.orchestra.run.smithi190.stdout:(10/131): ceph-radosgw-18.0.0-6675.gc7471068.el 9.3 MB/s | 14 MB 00:01 2023-10-16T09:41:23.789 INFO:teuthology.orchestra.run.smithi181.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-10-16T09:41:23.814 INFO:teuthology.orchestra.run.smithi181.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-10-16T09:41:23.848 INFO:teuthology.orchestra.run.smithi181.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-10-16T09:41:23.855 INFO:teuthology.orchestra.run.smithi190.stdout:(11/131): cephfs-mirror-18.0.0-6675.gc7471068.e 3.8 MB/s | 261 kB 00:00 2023-10-16T09:41:23.873 INFO:teuthology.orchestra.run.smithi181.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-10-16T09:41:23.915 INFO:teuthology.orchestra.run.smithi181.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-10-16T09:41:23.922 INFO:teuthology.orchestra.run.smithi190.stdout:(12/131): libcephfs-devel-18.0.0-6675.gc7471068 466 kB/s | 31 kB 00:00 2023-10-16T09:41:23.949 INFO:teuthology.orchestra.run.smithi181.stdout:(86/131): thrift-0.13.0-2.el8.x86_64.rpm 2.9 MB/s | 1.7 MB 00:00 2023-10-16T09:41:23.983 INFO:teuthology.orchestra.run.smithi181.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-10-16T09:41:24.031 INFO:teuthology.orchestra.run.smithi190.stdout:(13/131): libcephfs2-18.0.0-6675.gc7471068.el8. 7.6 MB/s | 839 kB 00:00 2023-10-16T09:41:24.033 INFO:teuthology.orchestra.run.smithi181.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 868 kB/s | 72 kB 00:00 2023-10-16T09:41:24.058 INFO:teuthology.orchestra.run.smithi181.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-10-16T09:41:24.083 INFO:teuthology.orchestra.run.smithi181.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-10-16T09:41:24.098 INFO:teuthology.orchestra.run.smithi190.stdout:(14/131): libcephsqlite-18.0.0-6675.gc7471068.e 2.7 MB/s | 184 kB 00:00 2023-10-16T09:41:24.100 INFO:teuthology.orchestra.run.smithi181.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-10-16T09:41:24.126 INFO:teuthology.orchestra.run.smithi181.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2023-10-16T09:41:24.151 INFO:teuthology.orchestra.run.smithi181.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-10-16T09:41:24.196 INFO:teuthology.orchestra.run.smithi190.stdout:(15/131): librados-devel-18.0.0-6675.gc7471068. 1.4 MB/s | 139 kB 00:00 2023-10-16T09:41:24.201 INFO:teuthology.orchestra.run.smithi181.stdout:(94/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 8.1 MB/s | 2.3 MB 00:00 2023-10-16T09:41:24.218 INFO:teuthology.orchestra.run.smithi181.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-10-16T09:41:24.243 INFO:teuthology.orchestra.run.smithi181.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-10-16T09:41:24.277 INFO:teuthology.orchestra.run.smithi181.stdout:(97/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.2 MB/s | 644 kB 00:00 2023-10-16T09:41:24.302 INFO:teuthology.orchestra.run.smithi181.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-10-16T09:41:24.315 INFO:teuthology.orchestra.run.smithi190.stdout:(16/131): libradosstriper1-18.0.0-6675.gc747106 4.3 MB/s | 518 kB 00:00 2023-10-16T09:41:24.328 INFO:teuthology.orchestra.run.smithi181.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-16T09:41:24.378 INFO:teuthology.orchestra.run.smithi181.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-10-16T09:41:24.414 INFO:teuthology.orchestra.run.smithi181.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2023-10-16T09:41:24.437 INFO:teuthology.orchestra.run.smithi181.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-10-16T09:41:24.479 INFO:teuthology.orchestra.run.smithi181.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-16T09:41:24.520 INFO:teuthology.orchestra.run.smithi181.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-10-16T09:41:24.546 INFO:teuthology.orchestra.run.smithi181.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-10-16T09:41:24.571 INFO:teuthology.orchestra.run.smithi181.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-10-16T09:41:24.613 INFO:teuthology.orchestra.run.smithi181.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-16T09:41:24.639 INFO:teuthology.orchestra.run.smithi181.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 1.0 MB/s | 27 kB 00:00 2023-10-16T09:41:24.681 INFO:teuthology.orchestra.run.smithi181.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.6 MB/s | 320 kB 00:00 2023-10-16T09:41:24.706 INFO:teuthology.orchestra.run.smithi181.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-10-16T09:41:24.806 INFO:teuthology.orchestra.run.smithi181.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-10-16T09:41:24.848 INFO:teuthology.orchestra.run.smithi181.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-10-16T09:41:24.898 INFO:teuthology.orchestra.run.smithi181.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 4.4 MB/s | 225 kB 00:00 2023-10-16T09:41:24.957 INFO:teuthology.orchestra.run.smithi181.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-10-16T09:41:24.958 INFO:teuthology.orchestra.run.smithi190.stdout:(17/131): librgw2-18.0.0-6675.gc7471068.el8.x86 11 MB/s | 6.9 MB 00:00 2023-10-16T09:41:25.012 INFO:teuthology.orchestra.run.smithi181.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 562 kB/s | 64 kB 00:00 2023-10-16T09:41:25.016 INFO:teuthology.orchestra.run.smithi190.stdout:(18/131): python3-ceph-argparse-18.0.0-6675.gc7 770 kB/s | 45 kB 00:00 2023-10-16T09:41:25.032 INFO:teuthology.orchestra.run.smithi181.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.6 MB/s | 126 kB 00:00 2023-10-16T09:41:25.057 INFO:teuthology.orchestra.run.smithi181.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 446 kB/s | 20 kB 00:00 2023-10-16T09:41:25.074 INFO:teuthology.orchestra.run.smithi190.stdout:(19/131): python3-ceph-common-18.0.0-6675.gc747 2.1 MB/s | 126 kB 00:00 2023-10-16T09:41:25.083 INFO:teuthology.orchestra.run.smithi181.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-16T09:41:25.117 INFO:teuthology.orchestra.run.smithi181.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-10-16T09:41:25.141 INFO:teuthology.orchestra.run.smithi190.stdout:(20/131): python3-cephfs-18.0.0-6675.gc7471068. 2.9 MB/s | 197 kB 00:00 2023-10-16T09:41:25.158 INFO:teuthology.orchestra.run.smithi181.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-10-16T09:41:25.208 INFO:teuthology.orchestra.run.smithi190.stdout:(21/131): python3-rados-18.0.0-6675.gc7471068.e 5.4 MB/s | 371 kB 00:00 2023-10-16T09:41:25.275 INFO:teuthology.orchestra.run.smithi190.stdout:(22/131): python3-rbd-18.0.0-6675.gc7471068.el8 5.3 MB/s | 364 kB 00:00 2023-10-16T09:41:25.309 INFO:teuthology.orchestra.run.smithi181.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-10-16T09:41:25.350 INFO:teuthology.orchestra.run.smithi181.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-16T09:41:25.367 INFO:teuthology.orchestra.run.smithi190.stdout:(23/131): python3-rgw-18.0.0-6675.gc7471068.el8 1.1 MB/s | 107 kB 00:00 2023-10-16T09:41:25.393 INFO:teuthology.orchestra.run.smithi181.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-16T09:41:25.426 INFO:teuthology.orchestra.run.smithi190.stdout:(24/131): rbd-fuse-18.0.0-6675.gc7471068.el8.x8 1.6 MB/s | 94 kB 00:00 2023-10-16T09:41:25.468 INFO:teuthology.orchestra.run.smithi181.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-10-16T09:41:25.568 INFO:teuthology.orchestra.run.smithi181.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-10-16T09:41:25.593 INFO:teuthology.orchestra.run.smithi181.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-10-16T09:41:25.669 INFO:teuthology.orchestra.run.smithi181.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-10-16T09:41:25.752 INFO:teuthology.orchestra.run.smithi181.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-10-16T09:41:25.784 INFO:teuthology.orchestra.run.smithi190.stdout:(25/131): rbd-mirror-18.0.0-6675.gc7471068.el8. 10 MB/s | 3.6 MB 00:00 2023-10-16T09:41:25.803 INFO:teuthology.orchestra.run.smithi181.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-16T09:41:25.843 INFO:teuthology.orchestra.run.smithi190.stdout:(26/131): rbd-nbd-18.0.0-6675.gc7471068.el8.x86 3.0 MB/s | 180 kB 00:00 2023-10-16T09:41:25.935 INFO:teuthology.orchestra.run.smithi190.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6675.g 251 kB/s | 23 kB 00:00 2023-10-16T09:41:25.978 INFO:teuthology.orchestra.run.smithi181.stdout:(130/131): librados2-18.0.0-6675.gc7471068.el8. 9.7 MB/s | 3.7 MB 00:00 2023-10-16T09:41:26.002 INFO:teuthology.orchestra.run.smithi190.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6675.gc747106 1.9 MB/s | 132 kB 00:00 2023-10-16T09:41:26.252 INFO:teuthology.orchestra.run.smithi190.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6675.gc7471 7.1 MB/s | 1.8 MB 00:00 2023-10-16T09:41:26.253 INFO:teuthology.orchestra.run.smithi181.stdout:(131/131): librbd1-18.0.0-6675.gc7471068.el8.x8 6.4 MB/s | 3.7 MB 00:00 2023-10-16T09:41:26.256 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-10-16T09:41:26.256 INFO:teuthology.orchestra.run.smithi181.stdout:Total 19 MB/s | 209 MB 00:11 2023-10-16T09:41:27.093 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-10-16T09:41:27.213 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-10-16T09:41:27.213 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-10-16T09:41:27.238 INFO:teuthology.orchestra.run.smithi190.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 7.5 MB/s | 7.4 MB 00:00 2023-10-16T09:41:27.320 INFO:teuthology.orchestra.run.smithi190.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6675.gc7 2.9 MB/s | 243 kB 00:00 2023-10-16T09:41:27.411 INFO:teuthology.orchestra.run.smithi190.stdout:(32/131): ceph-mgr-rook-18.0.0-6675.gc7471068.e 518 kB/s | 47 kB 00:00 2023-10-16T09:41:27.478 INFO:teuthology.orchestra.run.smithi190.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6675.gc 218 kB/s | 15 kB 00:00 2023-10-16T09:41:27.554 INFO:teuthology.orchestra.run.smithi190.stdout:(34/131): ceph-volume-18.0.0-6675.gc7471068.el8 3.4 MB/s | 261 kB 00:00 2023-10-16T09:41:27.629 INFO:teuthology.orchestra.run.smithi190.stdout:(35/131): cephadm-18.0.0-6675.gc7471068.el8.noa 2.5 MB/s | 192 kB 00:00 2023-10-16T09:41:27.729 INFO:teuthology.orchestra.run.smithi190.stdout:(36/131): cephfs-top-18.0.0-6675.gc7471068.el8. 181 kB/s | 18 kB 00:00 2023-10-16T09:41:28.338 INFO:teuthology.orchestra.run.smithi190.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 503 kB/s | 306 kB 00:00 2023-10-16T09:41:28.351 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-10-16T09:41:28.355 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-10-16T09:41:28.463 INFO:teuthology.orchestra.run.smithi190.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 471 kB/s | 59 kB 00:00 2023-10-16T09:41:28.605 INFO:teuthology.orchestra.run.smithi190.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 530 kB/s | 75 kB 00:00 2023-10-16T09:41:28.830 INFO:teuthology.orchestra.run.smithi190.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-10-16T09:41:28.939 INFO:teuthology.orchestra.run.smithi190.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-10-16T09:41:29.106 INFO:teuthology.orchestra.run.smithi190.stdout:(42/131): ceph-osd-18.0.0-6675.gc7471068.el8.x8 2.3 MB/s | 18 MB 00:07 2023-10-16T09:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 966 kB/s | 185 kB 00:00 2023-10-16T09:41:29.239 INFO:teuthology.orchestra.run.smithi190.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 151 kB/s | 16 kB 00:00 2023-10-16T09:41:29.365 INFO:teuthology.orchestra.run.smithi190.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-10-16T09:41:29.399 INFO:teuthology.orchestra.run.smithi190.stdout:(46/131): python3-cachetools-3.1.1-4.el8.noarch 113 kB/s | 33 kB 00:00 2023-10-16T09:41:29.516 INFO:teuthology.orchestra.run.smithi190.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2023-10-16T09:41:29.616 INFO:teuthology.orchestra.run.smithi190.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-10-16T09:41:29.641 INFO:teuthology.orchestra.run.smithi190.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 335 kB/s | 81 kB 00:00 2023-10-16T09:41:29.716 INFO:teuthology.orchestra.run.smithi190.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-10-16T09:41:29.825 INFO:teuthology.orchestra.run.smithi190.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-10-16T09:41:29.942 INFO:teuthology.orchestra.run.smithi190.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 504 kB/s | 59 kB 00:00 2023-10-16T09:41:30.059 INFO:teuthology.orchestra.run.smithi190.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 517 kB/s | 60 kB 00:00 2023-10-16T09:41:30.092 INFO:teuthology.orchestra.run.smithi190.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 2.6 MB/s | 1.2 MB 00:00 2023-10-16T09:41:30.193 INFO:teuthology.orchestra.run.smithi190.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-10-16T09:41:30.210 INFO:teuthology.orchestra.run.smithi190.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-10-16T09:41:30.302 INFO:teuthology.orchestra.run.smithi190.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-10-16T09:41:30.335 INFO:teuthology.orchestra.run.smithi190.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-10-16T09:41:30.419 INFO:teuthology.orchestra.run.smithi190.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 550 kB/s | 64 kB 00:00 2023-10-16T09:41:30.436 INFO:teuthology.orchestra.run.smithi190.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-10-16T09:41:30.528 INFO:teuthology.orchestra.run.smithi190.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-10-16T09:41:30.545 INFO:teuthology.orchestra.run.smithi190.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-10-16T09:41:30.637 INFO:teuthology.orchestra.run.smithi190.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-10-16T09:41:30.662 INFO:teuthology.orchestra.run.smithi190.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2023-10-16T09:41:30.754 INFO:teuthology.orchestra.run.smithi190.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-10-16T09:41:30.779 INFO:teuthology.orchestra.run.smithi190.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 523 kB/s | 61 kB 00:00 2023-10-16T09:41:30.871 INFO:teuthology.orchestra.run.smithi190.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 731 kB/s | 85 kB 00:00 2023-10-16T09:41:30.896 INFO:teuthology.orchestra.run.smithi190.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-10-16T09:41:30.922 INFO:teuthology.orchestra.run.smithi190.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-16T09:41:30.980 INFO:teuthology.orchestra.run.smithi190.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2023-10-16T09:41:31.022 INFO:teuthology.orchestra.run.smithi190.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-10-16T09:41:31.064 INFO:teuthology.orchestra.run.smithi190.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-10-16T09:41:31.106 INFO:teuthology.orchestra.run.smithi190.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-10-16T09:41:31.173 INFO:teuthology.orchestra.run.smithi190.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 719 kB/s | 48 kB 00:00 2023-10-16T09:41:31.215 INFO:teuthology.orchestra.run.smithi190.stdout:(75/131): thrift-0.13.0-2.el8.x86_64.rpm 5.0 MB/s | 1.7 MB 00:00 2023-10-16T09:41:31.241 INFO:teuthology.orchestra.run.smithi190.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-10-16T09:41:31.291 INFO:teuthology.orchestra.run.smithi190.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 568 kB/s | 43 kB 00:00 2023-10-16T09:41:31.316 INFO:teuthology.orchestra.run.smithi190.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.5 MB/s | 109 kB 00:00 2023-10-16T09:41:31.341 INFO:teuthology.orchestra.run.smithi190.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-16T09:41:31.375 INFO:teuthology.orchestra.run.smithi190.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-10-16T09:41:31.400 INFO:teuthology.orchestra.run.smithi190.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-10-16T09:41:31.417 INFO:teuthology.orchestra.run.smithi190.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-10-16T09:41:31.451 INFO:teuthology.orchestra.run.smithi190.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 11 MB/s | 559 kB 00:00 2023-10-16T09:41:31.476 INFO:teuthology.orchestra.run.smithi190.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-10-16T09:41:31.510 INFO:teuthology.orchestra.run.smithi190.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-10-16T09:41:31.535 INFO:teuthology.orchestra.run.smithi190.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-16T09:41:31.577 INFO:teuthology.orchestra.run.smithi190.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-10-16T09:41:31.611 INFO:teuthology.orchestra.run.smithi190.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-10-16T09:41:31.653 INFO:teuthology.orchestra.run.smithi190.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2023-10-16T09:41:31.678 INFO:teuthology.orchestra.run.smithi190.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 896 kB/s | 60 kB 00:00 2023-10-16T09:41:31.703 INFO:teuthology.orchestra.run.smithi190.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-16T09:41:31.754 INFO:teuthology.orchestra.run.smithi190.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 762 kB/s | 58 kB 00:00 2023-10-16T09:41:31.779 INFO:teuthology.orchestra.run.smithi190.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 513 kB/s | 39 kB 00:00 2023-10-16T09:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-10-16T09:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-10-16T09:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-16T09:41:31.905 INFO:teuthology.orchestra.run.smithi190.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-10-16T09:41:31.930 INFO:teuthology.orchestra.run.smithi190.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-10-16T09:41:31.972 INFO:teuthology.orchestra.run.smithi190.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-10-16T09:41:32.014 INFO:teuthology.orchestra.run.smithi190.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-10-16T09:41:32.039 INFO:teuthology.orchestra.run.smithi190.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.5 MB/s | 39 kB 00:00 2023-10-16T09:41:32.089 INFO:teuthology.orchestra.run.smithi190.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-10-16T09:41:32.132 INFO:teuthology.orchestra.run.smithi190.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-10-16T09:41:32.165 INFO:teuthology.orchestra.run.smithi190.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-10-16T09:41:32.207 INFO:teuthology.orchestra.run.smithi190.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-10-16T09:41:32.374 INFO:teuthology.orchestra.run.smithi190.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 26 MB/s | 14 MB 00:00 2023-10-16T09:41:32.416 INFO:teuthology.orchestra.run.smithi190.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 3.3 MB/s | 713 kB 00:00 2023-10-16T09:41:32.450 INFO:teuthology.orchestra.run.smithi190.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 798 kB/s | 27 kB 00:00 2023-10-16T09:41:32.492 INFO:teuthology.orchestra.run.smithi190.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-16T09:41:32.525 INFO:teuthology.orchestra.run.smithi190.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-10-16T09:41:32.659 INFO:teuthology.orchestra.run.smithi190.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-10-16T09:41:32.701 INFO:teuthology.orchestra.run.smithi190.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-10-16T09:41:32.743 INFO:teuthology.orchestra.run.smithi190.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-10-16T09:41:32.801 INFO:teuthology.orchestra.run.smithi190.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-10-16T09:41:32.827 INFO:teuthology.orchestra.run.smithi190.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-10-16T09:41:32.852 INFO:teuthology.orchestra.run.smithi190.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-16T09:41:32.877 INFO:teuthology.orchestra.run.smithi190.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-10-16T09:41:32.902 INFO:teuthology.orchestra.run.smithi190.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-16T09:41:32.957 INFO:teuthology.orchestra.run.smithi190.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 8.7 MB/s | 709 kB 00:00 2023-10-16T09:41:33.003 INFO:teuthology.orchestra.run.smithi190.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.3 MB/s | 198 kB 00:00 2023-10-16T09:41:33.103 INFO:teuthology.orchestra.run.smithi190.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 24 MB/s | 4.9 MB 00:00 2023-10-16T09:41:33.145 INFO:teuthology.orchestra.run.smithi190.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-16T09:41:33.212 INFO:teuthology.orchestra.run.smithi190.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-10-16T09:41:33.219 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-10-16T09:41:33.237 INFO:teuthology.orchestra.run.smithi190.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2023-10-16T09:41:33.296 INFO:teuthology.orchestra.run.smithi190.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-10-16T09:41:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 23 MB/s | 1.5 MB 00:00 2023-10-16T09:41:33.419 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-16T09:41:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:(127/131): postgresql-server-10.21-2.module+el8 23 MB/s | 5.1 MB 00:00 2023-10-16T09:41:33.602 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-16T09:41:33.755 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T09:41:33.873 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T09:41:34.104 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T09:41:34.155 INFO:teuthology.orchestra.run.smithi190.stdout:(128/131): librbd1-18.0.0-6675.gc7471068.el8.x8 5.1 MB/s | 3.7 MB 00:00 2023-10-16T09:41:34.205 INFO:teuthology.orchestra.run.smithi190.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-16T09:41:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T09:41:34.770 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.0.0-6675.gc7471068.el8.x86_64 5/134 2023-10-16T09:41:34.847 INFO:teuthology.orchestra.run.smithi190.stdout:(130/131): librados2-18.0.0-6675.gc7471068.el8. 2.5 MB/s | 3.7 MB 00:01 2023-10-16T09:41:35.182 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.0.0-6675.gc7471068.el8.x86_64 5/134 2023-10-16T09:41:35.404 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 6/134 2023-10-16T09:41:35.533 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 6/134 2023-10-16T09:41:35.745 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 7/134 2023-10-16T09:41:35.840 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 7/134 2023-10-16T09:41:36.042 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.0.0-6675.gc7471068.el8.x86_64 8/134 2023-10-16T09:41:36.220 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-16T09:41:36.434 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-16T09:41:36.656 INFO:teuthology.orchestra.run.smithi190.stdout:(131/131): ceph-test-18.0.0-6675.gc7471068.el8. 3.6 MB/s | 49 MB 00:13 2023-10-16T09:41:36.658 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2023-10-16T09:41:36.659 INFO:teuthology.orchestra.run.smithi190.stdout:Total 11 MB/s | 209 MB 00:19 2023-10-16T09:41:36.673 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-16T09:41:37.078 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-16T09:41:37.386 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-16T09:41:37.491 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2023-10-16T09:41:37.561 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 14/134 2023-10-16T09:41:37.609 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2023-10-16T09:41:37.609 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2023-10-16T09:41:37.667 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 14/134 2023-10-16T09:41:37.870 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86 15/134 2023-10-16T09:41:38.016 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86 15/134 2023-10-16T09:41:38.190 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-16T09:41:38.493 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-16T09:41:38.677 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-16T09:41:38.740 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2023-10-16T09:41:38.744 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2023-10-16T09:41:38.828 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T09:41:38.962 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T09:41:39.153 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T09:41:39.755 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T09:41:39.937 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T09:41:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T09:41:40.728 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T09:41:41.428 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T09:41:41.718 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-16T09:41:41.909 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-16T09:41:42.155 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-16T09:41:42.359 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.0.0-6675.gc7471068.el8. 26/134 2023-10-16T09:41:42.505 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 27/134 2023-10-16T09:41:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.0.0-6675.gc7471068.el8.x86_6 28/134 2023-10-16T09:41:42.994 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T09:41:43.690 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2023-10-16T09:41:43.883 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-16T09:41:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-16T09:41:44.260 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T09:41:44.375 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T09:41:44.593 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T09:41:44.746 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T09:41:44.995 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T09:41:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:18.0.0-6675.gc7471068.el8.x86_64 5/134 2023-10-16T09:41:45.635 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:18.0.0-6675.gc7471068.el8.x86_64 5/134 2023-10-16T09:41:45.781 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-16T09:41:45.851 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 6/134 2023-10-16T09:41:45.994 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 6/134 2023-10-16T09:41:46.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-16T09:41:46.201 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 7/134 2023-10-16T09:41:46.301 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 7/134 2023-10-16T09:41:46.379 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-16T09:41:46.531 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:18.0.0-6675.gc7471068.el8.x86_64 8/134 2023-10-16T09:41:46.583 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-16T09:41:46.731 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-16T09:41:46.805 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-16T09:41:46.979 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-16T09:41:47.012 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-16T09:41:47.195 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-16T09:41:47.207 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.0.0-6675.gc7471068.el8.x86_64 36/134 2023-10-16T09:41:47.383 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.0.0-6675.gc7471068.el8.x86_6 37/134 2023-10-16T09:41:47.543 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-16T09:41:47.588 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-16T09:41:47.678 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-16T09:41:47.835 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T09:41:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-16T09:41:47.907 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T09:41:48.067 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 14/134 2023-10-16T09:41:48.125 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-16T09:41:48.212 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 14/134 2023-10-16T09:41:48.347 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-16T09:41:48.401 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86 15/134 2023-10-16T09:41:48.493 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T09:41:48.551 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86 15/134 2023-10-16T09:41:48.560 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T09:41:48.763 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-16T09:41:48.780 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-16T09:41:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-16T09:41:48.956 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-16T09:41:49.373 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-16T09:41:49.479 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-16T09:41:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T09:41:49.784 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T09:41:49.804 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-16T09:41:49.930 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-16T09:41:50.134 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T09:41:50.609 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T09:41:50.676 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-16T09:41:50.953 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-16T09:41:51.171 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-16T09:41:51.191 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T09:41:51.400 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T09:41:51.431 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-16T09:41:51.656 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-16T09:41:51.951 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-16T09:41:52.100 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T09:41:52.159 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-16T09:41:52.352 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-16T09:41:52.399 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-16T09:41:52.548 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-16T09:41:52.598 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-16T09:41:52.772 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-16T09:41:52.860 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-16T09:41:52.954 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-16T09:41:52.998 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:18.0.0-6675.gc7471068.el8. 26/134 2023-10-16T09:41:53.119 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 27/134 2023-10-16T09:41:53.274 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-16T09:41:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:18.0.0-6675.gc7471068.el8.x86_6 28/134 2023-10-16T09:41:53.538 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-16T09:41:53.608 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T09:41:53.780 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-16T09:41:54.027 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-16T09:41:54.175 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-16T09:41:54.362 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-16T09:41:54.669 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-16T09:41:54.899 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-16T09:41:55.046 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-16T09:41:55.237 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-16T09:41:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T09:41:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-16T09:41:55.647 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-16T09:41:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-16T09:41:56.029 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-16T09:41:56.173 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-16T09:41:56.393 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-16T09:41:56.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-16T09:41:56.629 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-16T09:41:56.704 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-16T09:41:56.831 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-16T09:41:56.993 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-16T09:41:57.081 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-16T09:41:57.198 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-16T09:41:57.309 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-16T09:41:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-16T09:41:57.475 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-16T09:41:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-16T09:41:57.693 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-16T09:41:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:18.0.0-6675.gc7471068.el8.x86_64 36/134 2023-10-16T09:41:58.047 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:18.0.0-6675.gc7471068.el8.x86_6 37/134 2023-10-16T09:41:58.215 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-16T09:41:58.392 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-16T09:41:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T09:41:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T09:41:58.775 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-16T09:41:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-16T09:41:59.053 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-16T09:41:59.102 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-16T09:41:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T09:41:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-16T09:41:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T09:41:59.576 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-16T09:41:59.578 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-16T09:41:59.762 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-16T09:41:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-16T09:41:59.945 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-16T09:42:00.099 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-16T09:42:00.116 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-16T09:42:00.301 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-16T09:42:00.435 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-16T09:42:00.662 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-16T09:42:00.893 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-16T09:42:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-16T09:42:01.133 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 92/134 2023-10-16T09:42:01.208 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 92/134 2023-10-16T09:42:01.248 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-16T09:42:01.358 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.0.0-6675.gc7471068.el8.x86_64 93/134 2023-10-16T09:42:01.542 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-16T09:42:01.768 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-16T09:42:02.052 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-16T09:42:02.270 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-16T09:42:02.606 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-16T09:42:02.831 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-16T09:42:03.022 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-16T09:42:03.279 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-16T09:42:03.456 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-16T09:42:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-16T09:42:03.934 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-16T09:42:04.201 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-16T09:42:04.461 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-16T09:42:04.699 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-16T09:42:04.914 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-16T09:42:05.083 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 94/134 2023-10-16T09:42:05.100 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-16T09:42:05.389 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-16T09:42:05.475 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 94/134 2023-10-16T09:42:05.613 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-16T09:42:05.777 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-16T09:42:05.916 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-16T09:42:06.071 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 94/134 2023-10-16T09:42:06.113 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-16T09:42:06.316 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-16T09:42:06.355 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 95/134 2023-10-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 95/134 2023-10-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-16T09:42:06.583 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 96/134 2023-10-16T09:42:06.709 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-16T09:42:06.870 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-16T09:42:07.081 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-16T09:42:07.308 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-16T09:42:07.511 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-16T09:42:07.803 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-16T09:42:08.030 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-16T09:42:08.186 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-16T09:42:08.400 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-16T09:42:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-16T09:42:09.724 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-16T09:42:09.967 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-16T09:42:10.291 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-16T09:42:10.579 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-16T09:42:10.824 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-16T09:42:11.665 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-16T09:42:11.872 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 92/134 2023-10-16T09:42:11.933 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 92/134 2023-10-16T09:42:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:18.0.0-6675.gc7471068.el8.x86_64 93/134 2023-10-16T09:42:15.789 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 94/134 2023-10-16T09:42:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 94/134 2023-10-16T09:42:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 94/134 2023-10-16T09:42:16.862 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 95/134 2023-10-16T09:42:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 95/134 2023-10-16T09:42:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 96/134 2023-10-16T09:42:25.631 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 96/134 2023-10-16T09:42:25.632 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-10-16T09:42:25.632 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-10-16T09:42:25.632 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-10-16T09:42:25.632 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2023-10-16T09:42:25.633 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-10-16T09:42:25.633 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-10-16T09:42:25.633 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-10-16T09:42:25.633 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-10-16T09:42:25.634 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-10-16T09:42:25.634 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:42:25.979 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 97/134 2023-10-16T09:42:27.563 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 97/134 2023-10-16T09:42:27.735 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 98/134 2023-10-16T09:42:27.955 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 98/134 2023-10-16T09:42:28.096 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 99/134 2023-10-16T09:42:28.550 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 99/134 2023-10-16T09:42:28.721 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 100/134 2023-10-16T09:42:28.738 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 100/134 2023-10-16T09:42:30.676 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.0.0-6675.gc7471068.el8.noarch 101/134 2023-10-16T09:42:30.879 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.0.0-6675.gc7471068.el8.noarch 101/134 2023-10-16T09:42:30.976 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 102/134 2023-10-16T09:42:31.136 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 103/134 2023-10-16T09:42:31.311 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noa 104/134 2023-10-16T09:42:31.533 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noa 104/134 2023-10-16T09:42:31.817 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.n 105/134 2023-10-16T09:42:32.777 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.n 105/134 2023-10-16T09:42:32.944 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 106/134 2023-10-16T09:42:33.036 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 106/134 2023-10-16T09:42:33.410 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 107/134 2023-10-16T09:42:33.592 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 108/134 2023-10-16T09:42:33.628 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 108/134 2023-10-16T09:42:33.811 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 109/134 2023-10-16T09:42:33.822 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 109/134 2023-10-16T09:42:35.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.0.0-6675.gc7471068.el8.x86_64 110/134 2023-10-16T09:42:35.421 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 111/134 2023-10-16T09:42:35.454 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 111/134 2023-10-16T09:42:35.595 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 112/134 2023-10-16T09:42:35.636 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6675.gc7471 112/134 2023-10-16T09:42:35.761 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 113/134 2023-10-16T09:42:35.819 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 96/134 2023-10-16T09:42:35.819 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2023-10-16T09:42:35.819 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2023-10-16T09:42:35.819 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-10-16T09:42:35.820 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /run 2023-10-16T09:42:35.820 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2023-10-16T09:42:35.820 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2023-10-16T09:42:35.820 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2023-10-16T09:42:35.820 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2023-10-16T09:42:35.821 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-10-16T09:42:35.821 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:42:36.121 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 113/134 2023-10-16T09:42:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 97/134 2023-10-16T09:42:36.269 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 114/134 2023-10-16T09:42:37.745 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 97/134 2023-10-16T09:42:37.933 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 98/134 2023-10-16T09:42:38.152 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 98/134 2023-10-16T09:42:38.335 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 99/134 2023-10-16T09:42:38.788 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 99/134 2023-10-16T09:42:39.010 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 100/134 2023-10-16T09:42:39.027 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 100/134 2023-10-16T09:42:40.717 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:18.0.0-6675.gc7471068.el8.noarch 101/134 2023-10-16T09:42:40.902 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:18.0.0-6675.gc7471068.el8.noarch 101/134 2023-10-16T09:42:40.950 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 114/134 2023-10-16T09:42:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 102/134 2023-10-16T09:42:41.225 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 103/134 2023-10-16T09:42:41.401 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noa 104/134 2023-10-16T09:42:41.621 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noa 104/134 2023-10-16T09:42:41.834 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.0.0-6675.gc7471068.el8.x86_64 115/134 2023-10-16T09:42:41.949 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.n 105/134 2023-10-16T09:42:42.181 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-16T09:42:42.404 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 117/134 2023-10-16T09:42:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-16T09:42:42.801 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T09:42:42.912 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.n 105/134 2023-10-16T09:42:42.992 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T09:42:43.076 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 106/134 2023-10-16T09:42:43.165 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 106/134 2023-10-16T09:42:43.231 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-16T09:42:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.0.0-6675.gc7471068.el8.x86_ 121/134 2023-10-16T09:42:43.524 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 107/134 2023-10-16T09:42:43.552 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-16T09:42:43.716 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 108/134 2023-10-16T09:42:43.751 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 108/134 2023-10-16T09:42:43.917 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 109/134 2023-10-16T09:42:43.928 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 109/134 2023-10-16T09:42:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:18.0.0-6675.gc7471068.el8.x86_64 110/134 2023-10-16T09:42:45.528 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 111/134 2023-10-16T09:42:45.562 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 111/134 2023-10-16T09:42:45.710 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T09:42:45.727 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 112/134 2023-10-16T09:42:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6675.gc7471 112/134 2023-10-16T09:42:45.926 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 113/134 2023-10-16T09:42:45.968 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T09:42:45.986 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T09:42:46.135 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:18.0.0-6675.gc7471068.el8.noarch 124/134 2023-10-16T09:42:46.285 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 113/134 2023-10-16T09:42:46.425 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 125/134 2023-10-16T09:42:46.467 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 114/134 2023-10-16T09:42:46.713 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.0.0-6675.gc7471068.el8.x86_64 126/134 2023-10-16T09:42:47.123 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-16T09:42:47.284 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T09:42:47.366 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T09:42:47.567 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T09:42:47.580 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T09:42:47.726 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T09:42:47.734 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T09:42:47.870 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-16T09:42:47.870 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T09:42:48.045 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T09:42:48.046 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T09:42:48.298 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T09:42:48.458 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T09:42:48.948 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T09:42:48.949 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.0.0-6675.gc7471068.el8.x86_64 1/134 2023-10-16T09:42:48.949 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 2/134 2023-10-16T09:42:48.949 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 3/134 2023-10-16T09:42:48.949 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 4/134 2023-10-16T09:42:48.950 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 5/134 2023-10-16T09:42:48.950 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 6/134 2023-10-16T09:42:48.950 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 7/134 2023-10-16T09:42:48.950 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 8/134 2023-10-16T09:42:48.950 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 9/134 2023-10-16T09:42:48.951 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 10/134 2023-10-16T09:42:48.951 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 11/134 2023-10-16T09:42:48.951 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.0.0-6675.gc7471068.el8.x86_64 12/134 2023-10-16T09:42:48.951 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 13/134 2023-10-16T09:42:48.951 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.0.0-6675.gc7471068.el8.x86_ 14/134 2023-10-16T09:42:48.952 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 15/134 2023-10-16T09:42:48.952 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 16/134 2023-10-16T09:42:48.952 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.0.0-6675.gc7471068.el8.x86_6 17/134 2023-10-16T09:42:48.952 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86 18/134 2023-10-16T09:42:48.952 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 19/134 2023-10-16T09:42:48.953 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 20/134 2023-10-16T09:42:48.953 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.0.0-6675.gc7471068.el8. 21/134 2023-10-16T09:42:48.953 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.0.0-6675.gc7471068.el8.x86_6 22/134 2023-10-16T09:42:48.953 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.0.0-6675.gc7471068.el8.x86_64 23/134 2023-10-16T09:42:48.953 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.0.0-6675.gc7471068.el8.x86_64 24/134 2023-10-16T09:42:48.954 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.0.0-6675.gc7471068.el8.x86_64 25/134 2023-10-16T09:42:48.962 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 26/134 2023-10-16T09:42:48.963 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 27/134 2023-10-16T09:42:48.963 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.0.0-6675.gc7471068.el8.x86_64 28/134 2023-10-16T09:42:48.963 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 29/134 2023-10-16T09:42:48.963 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noa 30/134 2023-10-16T09:42:48.963 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.n 31/134 2023-10-16T09:42:48.964 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 32/134 2023-10-16T09:42:48.964 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 33/134 2023-10-16T09:42:48.964 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 34/134 2023-10-16T09:42:48.964 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 35/134 2023-10-16T09:42:48.964 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 36/134 2023-10-16T09:42:48.964 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.0.0-6675.gc7471068.el8.noarch 37/134 2023-10-16T09:42:48.965 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:18.0.0-6675.gc7471068.el8.noarch 38/134 2023-10-16T09:42:48.965 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-16T09:42:48.965 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-16T09:42:48.965 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-16T09:42:48.965 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-16T09:42:48.966 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-16T09:42:48.966 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-16T09:42:48.966 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-16T09:42:48.966 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-16T09:42:48.966 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-16T09:42:48.967 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-16T09:42:48.967 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-16T09:42:48.967 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-16T09:42:48.967 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-16T09:42:48.967 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-16T09:42:48.968 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-16T09:42:48.968 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-16T09:42:48.968 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-16T09:42:48.968 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-16T09:42:48.968 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-16T09:42:48.969 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-16T09:42:48.969 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-16T09:42:48.969 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-16T09:42:48.969 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-16T09:42:48.969 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-16T09:42:48.970 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-16T09:42:48.970 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-16T09:42:48.970 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-16T09:42:48.970 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-16T09:42:48.970 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-16T09:42:48.970 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-16T09:42:48.971 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-16T09:42:48.971 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-16T09:42:48.971 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-16T09:42:48.971 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-16T09:42:48.972 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-16T09:42:48.972 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-16T09:42:48.972 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-16T09:42:48.972 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-16T09:42:48.972 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-16T09:42:48.973 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-16T09:42:48.973 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-16T09:42:48.973 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-16T09:42:48.973 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-16T09:42:48.973 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-16T09:42:48.973 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-16T09:42:48.974 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-16T09:42:48.974 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-16T09:42:48.974 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-16T09:42:48.974 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-16T09:42:48.974 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-16T09:42:48.975 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-16T09:42:48.975 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-16T09:42:48.975 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-16T09:42:48.975 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-16T09:42:48.975 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-16T09:42:48.976 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-16T09:42:48.976 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-16T09:42:48.976 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-16T09:42:48.976 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-16T09:42:48.976 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-16T09:42:48.977 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-16T09:42:48.977 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-16T09:42:48.977 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-16T09:42:48.977 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-16T09:42:48.977 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-16T09:42:48.978 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-16T09:42:48.978 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-16T09:42:48.978 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-16T09:42:48.978 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-16T09:42:48.978 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-16T09:42:48.979 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-16T09:42:48.979 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-16T09:42:48.979 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-16T09:42:48.979 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-16T09:42:48.979 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-16T09:42:48.980 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-16T09:42:48.980 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-16T09:42:48.980 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-16T09:42:48.980 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-16T09:42:48.980 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-16T09:42:48.981 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-16T09:42:48.981 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-16T09:42:48.981 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-16T09:42:48.981 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-16T09:42:48.981 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-16T09:42:48.981 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-16T09:42:48.982 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-16T09:42:48.982 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-16T09:42:48.982 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-16T09:42:48.982 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-16T09:42:48.982 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.0.0-6675.gc7471068.el8.x86_64 129/134 2023-10-16T09:42:48.983 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-16T09:42:48.983 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 131/134 2023-10-16T09:42:48.983 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T09:42:48.983 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-16T09:42:51.137 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 114/134 2023-10-16T09:42:51.643 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T09:42:51.643 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Oct 2023 09:41:03 AM UTC. 2023-10-16T09:42:51.994 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:18.0.0-6675.gc7471068.el8.x86_64 115/134 2023-10-16T09:42:52.345 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-16T09:42:52.525 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 117/134 2023-10-16T09:42:52.831 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-16T09:42:53.016 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T09:42:53.205 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T09:42:53.445 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-16T09:42:53.611 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:18.0.0-6675.gc7471068.el8.x86_ 121/134 2023-10-16T09:42:53.750 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-16T09:42:54.268 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-10-16T09:42:54.491 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:42:54.491 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-10-16T09:42:54.491 INFO:teuthology.orchestra.run.smithi181.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-16T09:42:54.492 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.492 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.492 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-10-16T09:42:54.492 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-16T09:42:54.492 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-7.el8.noarch 2023-10-16T09:42:54.493 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2023-10-16T09:42:54.493 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.493 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.493 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.493 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.494 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.494 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.494 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.494 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.495 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.495 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.495 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.495 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.495 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.496 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.496 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.496 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.496 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.496 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.497 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.497 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.497 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.497 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.497 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:42:54.498 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-16T09:42:54.498 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2023-10-16T09:42:54.498 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-16T09:42:54.498 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-16T09:42:54.498 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-3.el8.x86_64 2023-10-16T09:42:54.499 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-16T09:42:54.499 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-16T09:42:54.499 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-16T09:42:54.499 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.499 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.500 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.500 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-16T09:42:54.500 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-16T09:42:54.500 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-16T09:42:54.500 INFO:teuthology.orchestra.run.smithi181.stdout: libpq-13.5-1.el8.x86_64 2023-10-16T09:42:54.501 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-16T09:42:54.501 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-16T09:42:54.501 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.501 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.501 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-16T09:42:54.502 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.502 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-16T09:42:54.502 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-16T09:42:54.502 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-16T09:42:54.502 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-16T09:42:54.503 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-16T09:42:54.503 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-16T09:42:54.503 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-16T09:42:54.503 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-16T09:42:54.503 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-16T09:42:54.504 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-16T09:42:54.504 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-16T09:42:54.504 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-16T09:42:54.504 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-16T09:42:54.505 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-16T09:42:54.505 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T09:42:54.505 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T09:42:54.505 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T09:42:54.505 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-16T09:42:54.506 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-16T09:42:54.506 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-16T09:42:54.506 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-16T09:42:54.506 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-16T09:42:54.507 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.507 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.507 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.507 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-16T09:42:54.507 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-16T09:42:54.508 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-16T09:42:54.508 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-16T09:42:54.508 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-16T09:42:54.508 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-16T09:42:54.508 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-16T09:42:54.509 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-16T09:42:54.509 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-16T09:42:54.509 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-16T09:42:54.509 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-16T09:42:54.509 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-16T09:42:54.510 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-16T09:42:54.510 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-16T09:42:54.510 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-16T09:42:54.510 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-16T09:42:54.510 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-16T09:42:54.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-16T09:42:54.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-16T09:42:54.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-16T09:42:54.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-16T09:42:54.511 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-16T09:42:54.512 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2023-10-16T09:42:54.512 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-16T09:42:54.512 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-16T09:42:54.512 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-16T09:42:54.513 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-16T09:42:54.513 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-16T09:42:54.513 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-16T09:42:54.513 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.514 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.514 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-16T09:42:54.514 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-16T09:42:54.514 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.514 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-16T09:42:54.515 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-16T09:42:54.515 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-16T09:42:54.515 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-16T09:42:54.515 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-16T09:42:54.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-16T09:42:54.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-16T09:42:54.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-16T09:42:54.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-16T09:42:54.516 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-16T09:42:54.517 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-16T09:42:54.517 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-16T09:42:54.517 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-16T09:42:54.517 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.517 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.518 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:42:54.518 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-16T09:42:54.518 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-16T09:42:54.518 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-16T09:42:54.521 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-16T09:42:54.521 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-16T09:42:54.522 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-16T09:42:54.522 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:42:54.522 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-16T09:42:54.684 DEBUG:teuthology.parallel:result is None 2023-10-16T09:42:56.030 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T09:42:56.324 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T09:42:56.343 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T09:42:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:18.0.0-6675.gc7471068.el8.noarch 124/134 2023-10-16T09:42:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 125/134 2023-10-16T09:42:57.059 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:18.0.0-6675.gc7471068.el8.x86_64 126/134 2023-10-16T09:42:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-16T09:42:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T09:42:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T09:42:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T09:42:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T09:42:58.107 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T09:42:58.114 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T09:42:58.242 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-16T09:42:58.242 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T09:42:58.460 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T09:42:58.460 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T09:42:58.678 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T09:42:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T09:42:59.395 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T09:42:59.396 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:18.0.0-6675.gc7471068.el8.x86_64 1/134 2023-10-16T09:42:59.396 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 2/134 2023-10-16T09:42:59.396 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 3/134 2023-10-16T09:42:59.396 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 4/134 2023-10-16T09:42:59.396 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6675.gc7471 5/134 2023-10-16T09:42:59.397 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 6/134 2023-10-16T09:42:59.397 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 7/134 2023-10-16T09:42:59.397 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 8/134 2023-10-16T09:42:59.397 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 9/134 2023-10-16T09:42:59.397 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 10/134 2023-10-16T09:42:59.398 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 11/134 2023-10-16T09:42:59.398 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:18.0.0-6675.gc7471068.el8.x86_64 12/134 2023-10-16T09:42:59.398 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 13/134 2023-10-16T09:42:59.398 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:18.0.0-6675.gc7471068.el8.x86_ 14/134 2023-10-16T09:42:59.398 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 15/134 2023-10-16T09:42:59.399 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 16/134 2023-10-16T09:42:59.399 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:18.0.0-6675.gc7471068.el8.x86_6 17/134 2023-10-16T09:42:59.399 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86 18/134 2023-10-16T09:42:59.399 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 19/134 2023-10-16T09:42:59.399 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6675.gc7471068.el 20/134 2023-10-16T09:42:59.399 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:18.0.0-6675.gc7471068.el8. 21/134 2023-10-16T09:42:59.400 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:18.0.0-6675.gc7471068.el8.x86_6 22/134 2023-10-16T09:42:59.400 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:18.0.0-6675.gc7471068.el8.x86_64 23/134 2023-10-16T09:42:59.400 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:18.0.0-6675.gc7471068.el8.x86_64 24/134 2023-10-16T09:42:59.400 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:18.0.0-6675.gc7471068.el8.x86_64 25/134 2023-10-16T09:42:59.400 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 26/134 2023-10-16T09:42:59.401 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 27/134 2023-10-16T09:42:59.401 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:18.0.0-6675.gc7471068.el8.x86_64 28/134 2023-10-16T09:42:59.401 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6675.gc7471068. 29/134 2023-10-16T09:42:59.401 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noa 30/134 2023-10-16T09:42:59.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.n 31/134 2023-10-16T09:42:59.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6675.gc74 32/134 2023-10-16T09:42:59.410 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el 33/134 2023-10-16T09:42:59.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 34/134 2023-10-16T09:42:59.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.e 35/134 2023-10-16T09:42:59.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 36/134 2023-10-16T09:42:59.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:18.0.0-6675.gc7471068.el8.noarch 37/134 2023-10-16T09:42:59.411 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:18.0.0-6675.gc7471068.el8.noarch 38/134 2023-10-16T09:42:59.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-16T09:42:59.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-16T09:42:59.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-16T09:42:59.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-16T09:42:59.412 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-16T09:42:59.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-16T09:42:59.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-16T09:42:59.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-16T09:42:59.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-16T09:42:59.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-16T09:42:59.414 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-16T09:42:59.414 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-16T09:42:59.414 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-16T09:42:59.414 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-16T09:42:59.414 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-16T09:42:59.415 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-16T09:42:59.415 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-16T09:42:59.415 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-16T09:42:59.415 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-16T09:42:59.415 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-16T09:42:59.416 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-16T09:42:59.416 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-16T09:42:59.416 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-16T09:42:59.416 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-16T09:42:59.416 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-16T09:42:59.417 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-16T09:42:59.417 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-16T09:42:59.417 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-16T09:42:59.417 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-16T09:42:59.417 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-16T09:42:59.418 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-16T09:42:59.418 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-16T09:42:59.418 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-16T09:42:59.418 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-16T09:42:59.418 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-16T09:42:59.419 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-16T09:42:59.419 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-16T09:42:59.419 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-16T09:42:59.419 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-16T09:42:59.419 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-16T09:42:59.420 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-16T09:42:59.420 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-16T09:42:59.420 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-16T09:42:59.420 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-16T09:42:59.420 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-16T09:42:59.421 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-16T09:42:59.421 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-16T09:42:59.421 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-16T09:42:59.421 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-16T09:42:59.421 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-16T09:42:59.421 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-16T09:42:59.422 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-16T09:42:59.422 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-16T09:42:59.422 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-16T09:42:59.422 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-16T09:42:59.422 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-16T09:42:59.423 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-16T09:42:59.423 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-16T09:42:59.423 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-16T09:42:59.423 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-16T09:42:59.423 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-16T09:42:59.424 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-16T09:42:59.424 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-16T09:42:59.424 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-16T09:42:59.424 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-16T09:42:59.424 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-16T09:42:59.424 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-16T09:42:59.425 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-16T09:42:59.425 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-16T09:42:59.425 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-16T09:42:59.425 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-16T09:42:59.425 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-16T09:42:59.426 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-16T09:42:59.426 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-16T09:42:59.426 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-16T09:42:59.426 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-16T09:42:59.426 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-16T09:42:59.427 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-16T09:42:59.427 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-16T09:42:59.427 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-16T09:42:59.427 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-16T09:42:59.427 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-16T09:42:59.427 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-16T09:42:59.428 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-16T09:42:59.428 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-16T09:42:59.428 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-16T09:42:59.428 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-16T09:42:59.428 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-16T09:42:59.429 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-16T09:42:59.429 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-16T09:42:59.429 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:18.0.0-6675.gc7471068.el8.x86_64 129/134 2023-10-16T09:42:59.429 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-16T09:42:59.429 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 131/134 2023-10-16T09:42:59.429 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T09:42:59.430 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-16T09:43:02.110 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T09:43:02.110 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:57 ago on Mon 16 Oct 2023 09:41:05 AM UTC. 2023-10-16T09:43:04.836 INFO:teuthology.orchestra.run.smithi190.stdout:Installed products updated. 2023-10-16T09:43:05.062 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:05.062 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2023-10-16T09:43:05.062 INFO:teuthology.orchestra.run.smithi190.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-16T09:43:05.063 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.063 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.063 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2023-10-16T09:43:05.063 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-16T09:43:05.064 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.1-7.el8.noarch 2023-10-16T09:43:05.064 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.0.4-10.el8.x86_64 2023-10-16T09:43:05.064 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.064 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.064 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.065 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.065 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.065 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.065 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.065 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.066 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.066 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.066 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.066 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.066 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.067 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.067 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.067 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.067 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.067 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.067 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.068 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.068 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.068 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.068 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:18.0.0-6675.gc7471068.el8.noarch 2023-10-16T09:43:05.068 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-16T09:43:05.069 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.1-9.el8.x86_64 2023-10-16T09:43:05.069 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-16T09:43:05.069 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-16T09:43:05.069 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-3.el8.x86_64 2023-10-16T09:43:05.069 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-16T09:43:05.070 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-16T09:43:05.070 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-16T09:43:05.070 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.070 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.070 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.071 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-16T09:43:05.071 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-16T09:43:05.071 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-16T09:43:05.071 INFO:teuthology.orchestra.run.smithi190.stdout: libpq-13.5-1.el8.x86_64 2023-10-16T09:43:05.071 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-16T09:43:05.072 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-16T09:43:05.072 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.072 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.072 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-16T09:43:05.072 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.073 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-16T09:43:05.073 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-16T09:43:05.073 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-16T09:43:05.073 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-16T09:43:05.073 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-16T09:43:05.074 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-16T09:43:05.074 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-16T09:43:05.074 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-16T09:43:05.074 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-16T09:43:05.074 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-16T09:43:05.075 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-16T09:43:05.075 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-16T09:43:05.075 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-16T09:43:05.075 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-16T09:43:05.075 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T09:43:05.075 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T09:43:05.076 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T09:43:05.076 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-16T09:43:05.076 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-16T09:43:05.076 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-16T09:43:05.076 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-16T09:43:05.077 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-16T09:43:05.077 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.077 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.077 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.077 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-16T09:43:05.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-16T09:43:05.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-16T09:43:05.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-16T09:43:05.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-16T09:43:05.078 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-16T09:43:05.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-16T09:43:05.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-16T09:43:05.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-16T09:43:05.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-16T09:43:05.079 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-16T09:43:05.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-16T09:43:05.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-16T09:43:05.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-16T09:43:05.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-16T09:43:05.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-16T09:43:05.080 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-16T09:43:05.081 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-16T09:43:05.081 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-16T09:43:05.081 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-16T09:43:05.081 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-16T09:43:05.081 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-16T09:43:05.082 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-2.6-1.el8.noarch 2023-10-16T09:43:05.082 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-16T09:43:05.082 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-16T09:43:05.082 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-16T09:43:05.082 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-16T09:43:05.083 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-16T09:43:05.083 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-16T09:43:05.083 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.083 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.083 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-16T09:43:05.084 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-16T09:43:05.084 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.084 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-16T09:43:05.084 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-16T09:43:05.084 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-16T09:43:05.085 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-16T09:43:05.085 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-16T09:43:05.085 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-16T09:43:05.085 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-16T09:43:05.085 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-16T09:43:05.086 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-16T09:43:05.086 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-16T09:43:05.086 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-16T09:43:05.086 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-16T09:43:05.086 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-16T09:43:05.086 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.087 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.087 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:18.0.0-6675.gc7471068.el8.x86_64 2023-10-16T09:43:05.087 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-16T09:43:05.087 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-16T09:43:05.087 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-16T09:43:05.088 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-16T09:43:05.088 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-16T09:43:05.088 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-16T09:43:05.089 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:05.089 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-10-16T09:43:05.267 DEBUG:teuthology.parallel:result is None 2023-10-16T09:43:05.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:43:05.524 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-16T09:43:05.554 INFO:teuthology.orchestra.run.smithi181.stdout:18.0.0-6675.gc7471068.el8 2023-10-16T09:43:05.554 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6675.gc7471068.el8 2023-10-16T09:43:05.555 INFO:teuthology.task.install:The correct ceph version 18.0.0-6675.gc7471068 is installed. 2023-10-16T09:43:05.555 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T09:43:05.790 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-16T09:43:05.817 INFO:teuthology.orchestra.run.smithi190.stdout:18.0.0-6675.gc7471068.el8 2023-10-16T09:43:05.817 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6675.gc7471068.el8 2023-10-16T09:43:05.818 INFO:teuthology.task.install:The correct ceph version 18.0.0-6675.gc7471068 is installed. 2023-10-16T09:43:05.818 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-16T09:43:05.819 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:05.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T09:43:05.867 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:05.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T09:43:05.913 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-16T09:43:05.914 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:05.914 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T09:43:05.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T09:43:06.012 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:06.013 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T09:43:06.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T09:43:06.111 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-16T09:43:06.111 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:06.112 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T09:43:06.140 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T09:43:06.207 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:06.207 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T09:43:06.235 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T09:43:06.302 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-16T09:43:06.303 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:06.303 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T09:43:06.330 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T09:43:06.397 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:06.398 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T09:43:06.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T09:43:06.495 INFO:teuthology.run_tasks:Running task ceph... 2023-10-16T09:43:06.560 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-16T09:43:06.560 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2023-10-16T09:43:06.563 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2023-10-16T09:43:06.591 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-16T09:43:06.591 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T09:43:06.630 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T09:43:06.658 INFO:tasks.ceph:Creating extra log directories... 2023-10-16T09:43:06.659 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T09:43:06.697 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T09:43:06.731 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-16T09:43:06.731 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-16T09:43:06.732 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-16T09:43:06.732 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:06.732 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T09:43:06.778 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T09:43:06.907 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-16T09:43:06.933 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:06.933 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T09:43:06.950 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T09:43:07.110 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-16T09:43:07.142 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-16T09:43:07.143 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-16T09:43:07.143 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969', 'branch': 'wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7428969', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-snap_schedule-support-subvol-and-group-arguments', 'sha1': '7b3d12460841f3cbe9400ba1b68e3e20957ff531'}}, 'owner': 'mchangir', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 127, 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'mgr-snap_schedule-support-subvol-and-group-arguments', 'suite_path': '/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/mchangir/ceph.git', 'suite_sha1': '7b3d12460841f3cbe9400ba1b68e3e20957ff531', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_09:20:41', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-16T09:43:07.144 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T09:43:07.146 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T09:43:07.193 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T09:43:07.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T09:43:07.259 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:07.260 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T09:43:07.288 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T09:43:07.288 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-10-16T09:43:07.344 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T09:43:07.344 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:07.345 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 384751 Links: 1 2023-10-16T09:43:07.345 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:07.345 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:07.345 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-16 09:42:48.656617777 +0000 2023-10-16T09:43:07.345 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-16 09:40:16.829526115 +0000 2023-10-16T09:43:07.346 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-16 09:40:16.829526115 +0000 2023-10-16T09:43:07.346 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-16T09:43:07.346 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T09:43:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-16T09:43:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-16T09:43:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000233896 s, 2.2 MB/s 2023-10-16T09:43:07.411 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T09:43:07.468 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-10-16T09:43:07.525 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T09:43:07.525 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:07.526 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 389788 Links: 1 2023-10-16T09:43:07.526 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:07.526 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:07.526 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-16 09:42:48.656617777 +0000 2023-10-16T09:43:07.527 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-16 09:40:17.511512500 +0000 2023-10-16T09:43:07.527 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-16 09:40:17.511512500 +0000 2023-10-16T09:43:07.527 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-16T09:43:07.527 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T09:43:07.590 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-16T09:43:07.591 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-16T09:43:07.591 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000236764 s, 2.2 MB/s 2023-10-16T09:43:07.592 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T09:43:07.649 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-10-16T09:43:07.705 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T09:43:07.705 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:07.706 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 385946 Links: 1 2023-10-16T09:43:07.706 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:07.706 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:07.706 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-16 09:42:48.656617777 +0000 2023-10-16T09:43:07.706 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-16 09:40:18.013502478 +0000 2023-10-16T09:43:07.707 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-16 09:40:18.013502478 +0000 2023-10-16T09:43:07.707 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-16T09:43:07.707 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T09:43:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-16T09:43:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-16T09:43:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000216475 s, 2.4 MB/s 2023-10-16T09:43:07.772 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T09:43:07.829 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-10-16T09:43:07.886 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T09:43:07.886 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:07.887 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 392068 Links: 1 2023-10-16T09:43:07.887 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:07.887 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:07.887 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-16 09:42:48.656617777 +0000 2023-10-16T09:43:07.887 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-16 09:40:18.530492157 +0000 2023-10-16T09:43:07.888 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-16 09:40:18.530492157 +0000 2023-10-16T09:43:07.888 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-16T09:43:07.888 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T09:43:07.951 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-16T09:43:07.951 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-16T09:43:07.951 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000167624 s, 3.1 MB/s 2023-10-16T09:43:07.952 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T09:43:08.010 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:08.010 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:08.011 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T09:43:08.029 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T09:43:08.029 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2023-10-16T09:43:08.087 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T09:43:08.087 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:08.088 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 390835 Links: 1 2023-10-16T09:43:08.088 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:08.088 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:08.088 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-16 09:42:59.097848242 +0000 2023-10-16T09:43:08.089 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-16 09:40:16.846700137 +0000 2023-10-16T09:43:08.089 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-16 09:40:16.846700137 +0000 2023-10-16T09:43:08.089 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-16T09:43:08.089 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T09:43:08.153 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-16T09:43:08.154 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-16T09:43:08.154 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000235586 s, 2.2 MB/s 2023-10-16T09:43:08.155 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T09:43:08.213 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2023-10-16T09:43:08.270 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T09:43:08.271 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:08.271 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 390960 Links: 1 2023-10-16T09:43:08.271 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:08.271 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:08.271 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-16 09:42:59.097848242 +0000 2023-10-16T09:43:08.272 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-16 09:40:17.344685699 +0000 2023-10-16T09:43:08.272 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-16 09:40:17.344685699 +0000 2023-10-16T09:43:08.272 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-16T09:43:08.272 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T09:43:08.335 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-16T09:43:08.335 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-16T09:43:08.336 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000219534 s, 2.3 MB/s 2023-10-16T09:43:08.337 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T09:43:08.395 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2023-10-16T09:43:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T09:43:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:08.453 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 384899 Links: 1 2023-10-16T09:43:08.453 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:08.453 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:08.453 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-16 09:42:59.097848242 +0000 2023-10-16T09:43:08.454 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-16 09:40:17.860670739 +0000 2023-10-16T09:43:08.454 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-16 09:40:17.860670739 +0000 2023-10-16T09:43:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-16T09:43:08.454 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T09:43:08.517 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-16T09:43:08.517 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-16T09:43:08.518 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000242185 s, 2.1 MB/s 2023-10-16T09:43:08.519 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T09:43:08.577 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2023-10-16T09:43:08.634 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T09:43:08.634 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T09:43:08.635 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 388736 Links: 1 2023-10-16T09:43:08.635 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T09:43:08.635 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T09:43:08.635 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-10-16 09:42:59.097848242 +0000 2023-10-16T09:43:08.635 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-10-16 09:40:18.348656591 +0000 2023-10-16T09:43:08.636 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-10-16 09:40:18.348656591 +0000 2023-10-16T09:43:08.636 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-10-16T09:43:08.636 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T09:43:08.699 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-10-16T09:43:08.700 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-10-16T09:43:08.700 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000199914 s, 2.6 MB/s 2023-10-16T09:43:08.701 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T09:43:08.759 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:08.760 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T09:43:08.760 INFO:tasks.ceph:Generating config... 2023-10-16T09:43:08.762 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-16T09:43:08.762 INFO:tasks.ceph:[client] debug client = 20 2023-10-16T09:43:08.762 INFO:tasks.ceph:[client] debug ms = 1 2023-10-16T09:43:08.763 INFO:tasks.ceph:[client] pid file = 2023-10-16T09:43:08.763 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-16T09:43:08.763 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-16T09:43:08.764 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-16T09:43:08.764 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-16T09:43:08.764 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-16T09:43:08.765 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-16T09:43:08.765 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-16T09:43:08.765 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-16T09:43:08.766 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-16T09:43:08.766 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-16T09:43:08.766 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-16T09:43:08.766 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-16T09:43:08.767 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-16T09:43:08.767 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-16T09:43:08.767 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-16T09:43:08.768 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-16T09:43:08.768 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-16T09:43:08.768 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-16T09:43:08.768 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-16T09:43:08.769 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-16T09:43:08.769 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-16T09:43:08.769 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-16T09:43:08.769 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-16T09:43:08.770 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-16T09:43:08.770 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-16T09:43:08.770 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-16T09:43:08.770 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-16T09:43:08.771 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-16T09:43:08.771 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-16T09:43:08.771 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-16T09:43:08.771 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-16T09:43:08.772 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-16T09:43:08.772 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-16T09:43:08.772 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-16T09:43:08.772 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-16T09:43:08.773 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-16T09:43:08.773 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-16T09:43:08.773 INFO:tasks.ceph:Setting up mon.a... 2023-10-16T09:43:08.774 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-16T09:43:08.925 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.keyring 2023-10-16T09:43:08.928 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-16T09:43:08.983 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T09:43:09.011 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.181'), ('mon.b', '172.21.15.190'), ('mon.c', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]')] 2023-10-16T09:43:09.011 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.181,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-16T09:43:09.012 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:09.013 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T09:43:09.068 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.181 --add b 172.21.15.190 --addv c '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:09.159 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:09.159 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: generated fsid c0cf369f-f4d9-40c8-9248-54aac3970136 2023-10-16T09:43:09.160 INFO:teuthology.orchestra.run.smithi181.stdout:setting min_mon_release = pacific 2023-10-16T09:43:09.160 INFO:teuthology.orchestra.run.smithi181.stdout:epoch 0 2023-10-16T09:43:09.160 INFO:teuthology.orchestra.run.smithi181.stdout:fsid c0cf369f-f4d9-40c8-9248-54aac3970136 2023-10-16T09:43:09.160 INFO:teuthology.orchestra.run.smithi181.stdout:last_changed 2023-10-16T09:43:09.158732+0000 2023-10-16T09:43:09.160 INFO:teuthology.orchestra.run.smithi181.stdout:created 2023-10-16T09:43:09.158732+0000 2023-10-16T09:43:09.161 INFO:teuthology.orchestra.run.smithi181.stdout:min_mon_release 16 (pacific) 2023-10-16T09:43:09.161 INFO:teuthology.orchestra.run.smithi181.stdout:election_strategy: 1 2023-10-16T09:43:09.161 INFO:teuthology.orchestra.run.smithi181.stdout:0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2023-10-16T09:43:09.161 INFO:teuthology.orchestra.run.smithi181.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2023-10-16T09:43:09.161 INFO:teuthology.orchestra.run.smithi181.stdout:2: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.c 2023-10-16T09:43:09.162 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-16T09:43:09.163 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T09:43:09.180 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c0cf369f-f4d9-40c8-9248-54aac3970136... 2023-10-16T09:43:09.181 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T09:43:09.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T09:43:09.266 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2023-10-16T09:43:09.267 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2023-10-16T09:43:09.267 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T09:43:09.267 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2023-10-16T09:43:09.267 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.268 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2023-10-16T09:43:09.269 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.269 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2023-10-16T09:43:09.269 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.269 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2023-10-16T09:43:09.269 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2023-10-16T09:43:09.270 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.270 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2023-10-16T09:43:09.270 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2023-10-16T09:43:09.270 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.270 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T09:43:09.271 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2023-10-16T09:43:09.271 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.271 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T09:43:09.271 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T09:43:09.271 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.271 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2023-10-16T09:43:09.272 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.272 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2023-10-16T09:43:09.272 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2023-10-16T09:43:09.272 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2023-10-16T09:43:09.272 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2023-10-16T09:43:09.273 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2023-10-16T09:43:09.273 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2023-10-16T09:43:09.273 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2023-10-16T09:43:09.273 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T09:43:09.273 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T09:43:09.274 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2023-10-16T09:43:09.274 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.274 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T09:43:09.274 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.274 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2023-10-16T09:43:09.275 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.275 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2023-10-16T09:43:09.275 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.275 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2023-10-16T09:43:09.275 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2023-10-16T09:43:09.276 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2023-10-16T09:43:09.276 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.181,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-10-16T09:43:09.276 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on skipped message = False 2023-10-16T09:43:09.276 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = c0cf369f-f4d9-40c8-9248-54aac3970136 2023-10-16T09:43:09.276 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.276 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2023-10-16T09:43:09.277 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2023-10-16T09:43:09.277 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.277 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2023-10-16T09:43:09.277 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2023-10-16T09:43:09.277 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock profile = high_recovery_ops 2023-10-16T09:43:09.278 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.278 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2023-10-16T09:43:09.278 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2023-10-16T09:43:09.278 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.278 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2023-10-16T09:43:09.279 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2023-10-16T09:43:09.279 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2023-10-16T09:43:09.279 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.279 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2023-10-16T09:43:09.279 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2023-10-16T09:43:09.279 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.280 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2023-10-16T09:43:09.280 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.280 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2023-10-16T09:43:09.280 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.280 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2023-10-16T09:43:09.281 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.281 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2023-10-16T09:43:09.281 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2023-10-16T09:43:09.281 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.281 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2023-10-16T09:43:09.282 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2023-10-16T09:43:09.282 INFO:teuthology.orchestra.run.smithi181.stdout: bdev async discard = True 2023-10-16T09:43:09.282 INFO:teuthology.orchestra.run.smithi181.stdout: bdev enable discard = True 2023-10-16T09:43:09.282 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore allocator = bitmap 2023-10-16T09:43:09.282 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2023-10-16T09:43:09.283 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2023-10-16T09:43:09.283 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 1/20 2023-10-16T09:43:09.283 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 1/20 2023-10-16T09:43:09.283 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-16T09:43:09.283 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2023-10-16T09:43:09.284 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 4/10 2023-10-16T09:43:09.284 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T09:43:09.284 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2023-10-16T09:43:09.284 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2023-10-16T09:43:09.284 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2023-10-16T09:43:09.284 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2023-10-16T09:43:09.285 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-10-16T09:43:09.285 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.285 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2023-10-16T09:43:09.285 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-16T09:43:09.285 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2023-10-16T09:43:09.286 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-10-16T09:43:09.286 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-10-16T09:43:09.286 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:43:09.286 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:43:09.286 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2023-10-16T09:43:09.287 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2023-10-16T09:43:09.287 INFO:teuthology.orchestra.run.smithi181.stdout: debug finisher = 20 2023-10-16T09:43:09.287 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.287 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2023-10-16T09:43:09.287 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-16T09:43:09.287 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-10-16T09:43:09.288 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2023-10-16T09:43:09.288 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-10-16T09:43:09.288 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2023-10-16T09:43:09.288 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2023-10-16T09:43:09.288 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:43:09.289 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2023-10-16T09:43:09.289 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2023-10-16T09:43:09.289 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:43:09.289 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.289 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T09:43:09.290 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T09:43:09.290 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T09:43:09.290 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.290 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T09:43:09.290 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T09:43:09.290 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T09:43:09.291 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.291 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2023-10-16T09:43:09.291 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2023-10-16T09:43:09.291 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.291 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2023-10-16T09:43:09.292 INFO:teuthology.orchestra.run.smithi181.stdout: mon op complaint time = 120 2023-10-16T09:43:09.292 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:09.292 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2023-10-16T09:43:09.292 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2023-10-16T09:43:09.292 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2023-10-16T09:43:09.293 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2023-10-16T09:43:09.293 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T09:43:09.293 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T09:43:09.293 INFO:teuthology.orchestra.run.smithi181.stdout: client mount timeout = 600 2023-10-16T09:43:09.293 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2023-10-16T09:43:09.294 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-16T09:43:09.294 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = "" 2023-10-16T09:43:09.294 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-10-16T09:43:09.294 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-10-16T09:43:09.294 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2023-10-16T09:43:09.295 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2023-10-16T09:43:09.295 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2023-10-16T09:43:09.295 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-10-16T09:43:09.295 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-10-16T09:43:09.295 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-10-16T09:43:09.295 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-10-16T09:43:09.296 INFO:teuthology.orchestra.run.smithi181.stdout:[mds] 2023-10-16T09:43:09.296 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds = 20 2023-10-16T09:43:09.296 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds balancer = 20 2023-10-16T09:43:09.296 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-16T09:43:09.296 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug frag = True 2023-10-16T09:43:09.297 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug scatterstat = True 2023-10-16T09:43:09.297 INFO:teuthology.orchestra.run.smithi181.stdout: mds op complaint time = 180 2023-10-16T09:43:09.297 INFO:teuthology.orchestra.run.smithi181.stdout: mds verify scatter = True 2023-10-16T09:43:09.297 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-10-16T09:43:09.297 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-10-16T09:43:09.297 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-10-16T09:43:09.299 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-10-16T09:43:09.299 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-10-16T09:43:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T09:43:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-10-16T09:43:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-10-16T09:43:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-10-16T09:43:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-10-16T09:43:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-10-16T09:43:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-10-16T09:43:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-10-16T09:43:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T09:43:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-10-16T09:43:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T09:43:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T09:43:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-10-16T09:43:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-10-16T09:43:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-10-16T09:43:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-10-16T09:43:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-10-16T09:43:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-10-16T09:43:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-10-16T09:43:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-10-16T09:43:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T09:43:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T09:43:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-10-16T09:43:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T09:43:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-10-16T09:43:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-10-16T09:43:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-10-16T09:43:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-10-16T09:43:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-10-16T09:43:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.181,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-10-16T09:43:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on skipped message = False 2023-10-16T09:43:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = c0cf369f-f4d9-40c8-9248-54aac3970136 2023-10-16T09:43:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.308 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-10-16T09:43:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-10-16T09:43:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-10-16T09:43:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-10-16T09:43:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-10-16T09:43:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-10-16T09:43:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-10-16T09:43:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-10-16T09:43:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-10-16T09:43:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-10-16T09:43:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-10-16T09:43:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-10-16T09:43:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-10-16T09:43:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-10-16T09:43:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-10-16T09:43:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-10-16T09:43:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-10-16T09:43:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-10-16T09:43:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-10-16T09:43:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: bdev async discard = True 2023-10-16T09:43:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: bdev enable discard = True 2023-10-16T09:43:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore allocator = bitmap 2023-10-16T09:43:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-10-16T09:43:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-10-16T09:43:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 1/20 2023-10-16T09:43:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 1/20 2023-10-16T09:43:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-10-16T09:43:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-10-16T09:43:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 4/10 2023-10-16T09:43:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T09:43:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-10-16T09:43:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-10-16T09:43:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-10-16T09:43:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-10-16T09:43:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-10-16T09:43:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.317 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-10-16T09:43:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-10-16T09:43:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-10-16T09:43:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-10-16T09:43:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-10-16T09:43:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:43:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:43:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-10-16T09:43:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-10-16T09:43:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: debug finisher = 20 2023-10-16T09:43:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.319 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-10-16T09:43:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-10-16T09:43:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-10-16T09:43:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-10-16T09:43:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-10-16T09:43:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-10-16T09:43:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-10-16T09:43:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-10-16T09:43:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-10-16T09:43:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-10-16T09:43:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-10-16T09:43:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T09:43:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T09:43:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T09:43:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T09:43:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T09:43:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T09:43:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-10-16T09:43:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-10-16T09:43:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-10-16T09:43:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-10-16T09:43:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-10-16T09:43:09.324 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-10-16T09:43:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-10-16T09:43:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-10-16T09:43:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-10-16T09:43:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T09:43:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T09:43:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-10-16T09:43:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-10-16T09:43:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-10-16T09:43:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-10-16T09:43:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-10-16T09:43:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-10-16T09:43:09.326 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-10-16T09:43:09.327 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-10-16T09:43:09.327 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-10-16T09:43:09.327 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-10-16T09:43:09.327 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-10-16T09:43:09.327 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-10-16T09:43:09.328 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-10-16T09:43:09.328 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-10-16T09:43:09.328 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-10-16T09:43:09.328 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-10-16T09:43:09.328 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-10-16T09:43:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-10-16T09:43:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2023-10-16T09:43:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-10-16T09:43:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2023-10-16T09:43:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-10-16T09:43:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-10-16T09:43:09.330 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-10-16T09:43:09.330 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-16T09:43:09.330 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-16T09:43:09.608 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-16T09:43:09.608 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:09.608 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-16T09:43:09.625 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:09.625 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-16T09:43:09.681 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:43:09.681 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:09.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T09:43:09.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T09:43:09.786 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:09.787 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:09.860 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:43:09.860 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:09.860 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T09:43:09.861 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T09:43:09.901 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:09.901 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:09.959 INFO:tasks.ceph:Setting up mon nodes... 2023-10-16T09:43:09.959 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-16T09:43:09.960 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T09:43:10.014 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T09:43:10.017 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T09:43:10.099 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T09:43:10.102 INFO:tasks.ceph:Setting up mds nodes... 2023-10-16T09:43:10.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T09:43:10.156 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T09:43:10.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-16T09:43:10.184 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T09:43:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T09:43:10.280 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-16T09:43:10.306 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T09:43:10.361 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T09:43:10.363 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-16T09:43:10.390 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T09:43:10.483 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T09:43:10.486 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-16T09:43:10.511 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-16T09:43:10.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-16T09:43:10.555 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-16T09:43:10.568 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-16T09:43:10.650 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-16T09:43:10.663 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-16T09:43:10.706 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-16T09:43:10.718 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-16T09:43:10.801 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-16T09:43:10.814 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-16T09:43:10.814 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T09:43:10.815 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-16T09:43:10.841 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:10.842 INFO:tasks.ceph:role: osd.0 2023-10-16T09:43:10.842 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:43:10.843 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T09:43:10.915 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:10.915 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:10.915 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:10.916 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:10.916 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:10.916 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:10.916 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:10.916 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:10.916 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:10.917 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:11.345 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-16T09:43:11.347 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-16T09:43:11.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-16T09:43:11.519 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-16T09:43:11.553 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-16T09:43:11.620 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:11.620 INFO:tasks.ceph:role: osd.1 2023-10-16T09:43:11.620 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:43:11.621 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T09:43:11.692 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:11.693 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:11.693 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:11.693 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:11.693 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:11.694 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:11.694 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:11.694 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:11.694 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:11.695 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:12.109 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-16T09:43:12.111 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-16T09:43:12.111 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-16T09:43:12.225 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-16T09:43:12.256 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-16T09:43:12.322 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:12.322 INFO:tasks.ceph:role: osd.2 2023-10-16T09:43:12.323 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:43:12.323 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T09:43:12.396 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:12.396 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:12.396 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:12.396 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:12.397 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:12.397 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:12.397 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:12.397 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:12.397 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:12.398 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:12.846 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-16T09:43:12.848 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-16T09:43:12.848 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-16T09:43:12.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-16T09:43:13.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-16T09:43:13.099 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:13.100 INFO:tasks.ceph:role: osd.3 2023-10-16T09:43:13.100 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:43:13.100 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T09:43:13.173 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:13.174 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:13.174 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:13.174 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:13.174 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:13.174 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:13.175 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:13.175 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:13.175 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:13.175 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:13.614 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-16T09:43:13.616 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-16T09:43:13.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-16T09:43:13.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-16T09:43:13.847 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:13.953+0000 7ff682f96640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-16T09:43:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:13.953+0000 7ff682f96640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-16T09:43:13.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:13.953+0000 7ff682f96640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T09:43:13.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:13.953+0000 7ff682f96640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T09:43:13.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:13.953+0000 7ff682f96640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T09:43:13.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:13.953+0000 7ff682f96640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-16T09:43:16.814 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-16T09:43:16.846 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:16.939+0000 7f7f506ef640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-16T09:43:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:16.940+0000 7f7f506ef640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-16T09:43:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:16.940+0000 7f7f506ef640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T09:43:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:16.940+0000 7f7f506ef640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T09:43:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:16.940+0000 7f7f506ef640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T09:43:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:16.940+0000 7f7f506ef640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-16T09:43:19.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-16T09:43:19.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:19.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:19.938+0000 7efe5e1ad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-16T09:43:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:19.938+0000 7efe5e1ad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-16T09:43:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:19.938+0000 7efe5e1ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T09:43:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:19.938+0000 7efe5e1ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T09:43:19.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:19.938+0000 7efe5e1ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T09:43:19.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:19.939+0000 7efe5e1ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-16T09:43:22.806 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-16T09:43:22.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:22.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:22.929+0000 7efe470e6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-16T09:43:22.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:22.930+0000 7efe470e6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-16T09:43:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:22.930+0000 7efe470e6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T09:43:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:22.930+0000 7efe470e6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T09:43:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:22.930+0000 7efe470e6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T09:43:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:22.930+0000 7efe470e6640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-16T09:43:25.804 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-16T09:43:25.836 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-16T09:43:25.868 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:25.869 INFO:tasks.ceph:role: osd.4 2023-10-16T09:43:25.869 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:43:25.869 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T09:43:25.947 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:25.947 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:25.947 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:25.947 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:25.948 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:25.948 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:25.948 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:25.948 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:25.948 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:25.949 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:26.386 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-10-16T09:43:26.389 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-10-16T09:43:26.389 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-16T09:43:26.577 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-16T09:43:26.613 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-16T09:43:26.683 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:26.683 INFO:tasks.ceph:role: osd.5 2023-10-16T09:43:26.684 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:43:26.684 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T09:43:26.759 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:26.760 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:26.760 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:26.760 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:26.761 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:26.761 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:26.761 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:26.761 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:26.761 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:26.761 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:27.176 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-10-16T09:43:27.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-10-16T09:43:27.179 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-16T09:43:27.284 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-16T09:43:27.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-16T09:43:27.385 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:27.385 INFO:tasks.ceph:role: osd.6 2023-10-16T09:43:27.386 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:43:27.386 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T09:43:27.461 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:27.462 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:27.462 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:27.462 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:27.463 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:27.463 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:27.463 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:27.463 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:27.463 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:27.464 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:27.902 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-10-16T09:43:27.904 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-10-16T09:43:27.905 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-16T09:43:28.085 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-16T09:43:28.121 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-16T09:43:28.191 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T09:43:28.192 INFO:tasks.ceph:role: osd.7 2023-10-16T09:43:28.192 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:43:28.192 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T09:43:28.266 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T09:43:28.267 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T09:43:28.267 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T09:43:28.267 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T09:43:28.267 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T09:43:28.268 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-10-16T09:43:28.268 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T09:43:28.268 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T09:43:28.268 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T09:43:28.268 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T09:43:28.716 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-10-16T09:43:28.718 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-10-16T09:43:28.718 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-16T09:43:28.908 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-16T09:43:28.945 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:29.073 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:29.069+0000 7ff59c38d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-16T09:43:29.073 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:29.070+0000 7ff59c38d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-16T09:43:29.073 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:29.070+0000 7ff59c38d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T09:43:29.073 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:29.070+0000 7ff59c38d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T09:43:29.074 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:29.070+0000 7ff59c38d640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T09:43:29.074 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:29.070+0000 7ff59c38d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-16T09:43:31.933 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-16T09:43:31.965 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:32.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:32.054+0000 7f07c622d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-16T09:43:32.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:32.054+0000 7f07c622d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-16T09:43:32.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:32.054+0000 7f07c622d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T09:43:32.056 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:32.054+0000 7f07c622d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T09:43:32.057 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:32.054+0000 7f07c622d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T09:43:32.057 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:32.054+0000 7f07c622d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-16T09:43:34.928 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-16T09:43:34.959 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:35.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:35.047+0000 7fcbf3ce4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-16T09:43:35.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:35.048+0000 7fcbf3ce4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-16T09:43:35.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:35.048+0000 7fcbf3ce4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T09:43:35.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:35.048+0000 7fcbf3ce4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T09:43:35.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:35.048+0000 7fcbf3ce4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T09:43:35.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:35.048+0000 7fcbf3ce4640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-16T09:43:37.146 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:43:37.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:43:37.919 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-16T09:43:37.950 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:38.042 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:38.040+0000 7fbb92b90640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-16T09:43:38.042 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:38.040+0000 7fbb92b90640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-16T09:43:38.043 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:38.040+0000 7fbb92b90640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T09:43:38.043 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:38.040+0000 7fbb92b90640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T09:43:38.043 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:38.040+0000 7fbb92b90640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T09:43:38.043 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:38.041+0000 7fbb92b90640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-16T09:43:40.910 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-16T09:43:40.942 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-16T09:43:40.942 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:40.942 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-16T09:43:40.974 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:40.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-16T09:43:41.045 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:41.046 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-16T09:43:41.116 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:41.116 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-16T09:43:41.185 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:41.186 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-16T09:43:41.254 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:41.255 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-16T09:43:41.322 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:41.323 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-16T09:43:41.391 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.391 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-16T09:43:41.423 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.423 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-16T09:43:41.494 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-16T09:43:41.563 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.563 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-16T09:43:41.632 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-16T09:43:41.700 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.701 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-16T09:43:41.768 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.768 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-16T09:43:41.835 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:41.836 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-16T09:43:41.856 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:41.856 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-16T09:43:41.915 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.915 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-16T09:43:41.933 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:41.933 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-16T09:43:41.990 INFO:tasks.ceph:Adding keys to all mons... 2023-10-16T09:43:41.991 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T09:43:41.993 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T09:43:42.033 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2023-10-16T09:43:42.033 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg== 2023-10-16T09:43:42.034 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-10-16T09:43:42.034 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ== 2023-10-16T09:43:42.034 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-10-16T09:43:42.034 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ== 2023-10-16T09:43:42.035 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2023-10-16T09:43:42.035 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA== 2023-10-16T09:43:42.035 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2023-10-16T09:43:42.035 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg== 2023-10-16T09:43:42.036 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2023-10-16T09:43:42.036 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ== 2023-10-16T09:43:42.036 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2023-10-16T09:43:42.036 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw== 2023-10-16T09:43:42.036 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2023-10-16T09:43:42.037 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag== 2023-10-16T09:43:42.037 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-10-16T09:43:42.037 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A== 2023-10-16T09:43:42.037 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-10-16T09:43:42.037 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw== 2023-10-16T09:43:42.038 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2023-10-16T09:43:42.038 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w== 2023-10-16T09:43:42.038 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2023-10-16T09:43:42.038 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw== 2023-10-16T09:43:42.038 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2023-10-16T09:43:42.039 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q== 2023-10-16T09:43:42.039 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2023-10-16T09:43:42.039 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA== 2023-10-16T09:43:42.039 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T09:43:42.040 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T09:43:42.040 INFO:teuthology.orchestra.run.smithi181.stdout:[client.2] 2023-10-16T09:43:42.040 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ== 2023-10-16T09:43:42.040 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2023-10-16T09:43:42.040 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA== 2023-10-16T09:43:42.040 INFO:teuthology.orchestra.run.smithi181.stdout:[client.3] 2023-10-16T09:43:42.041 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag== 2023-10-16T09:43:42.056 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2023-10-16T09:43:42.056 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg== 2023-10-16T09:43:42.056 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-10-16T09:43:42.056 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ== 2023-10-16T09:43:42.057 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-10-16T09:43:42.057 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ== 2023-10-16T09:43:42.057 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2023-10-16T09:43:42.057 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA== 2023-10-16T09:43:42.057 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2023-10-16T09:43:42.058 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg== 2023-10-16T09:43:42.058 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2023-10-16T09:43:42.058 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ== 2023-10-16T09:43:42.058 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2023-10-16T09:43:42.058 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw== 2023-10-16T09:43:42.059 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2023-10-16T09:43:42.059 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag== 2023-10-16T09:43:42.059 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-10-16T09:43:42.059 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A== 2023-10-16T09:43:42.059 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-10-16T09:43:42.059 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw== 2023-10-16T09:43:42.060 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2023-10-16T09:43:42.060 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w== 2023-10-16T09:43:42.060 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2023-10-16T09:43:42.060 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw== 2023-10-16T09:43:42.060 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2023-10-16T09:43:42.061 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q== 2023-10-16T09:43:42.061 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2023-10-16T09:43:42.061 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA== 2023-10-16T09:43:42.061 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2023-10-16T09:43:42.061 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T09:43:42.062 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2023-10-16T09:43:42.062 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ== 2023-10-16T09:43:42.062 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2023-10-16T09:43:42.062 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA== 2023-10-16T09:43:42.062 INFO:teuthology.orchestra.run.smithi190.stdout:[client.3] 2023-10-16T09:43:42.062 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag== 2023-10-16T09:43:42.063 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:43:42.084 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:43:42.156 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.159 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.215 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.217 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.272 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.275 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.327 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.330 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.382 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.385 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.440 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.492 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:43:42.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T09:43:42.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.549 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.605 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.607 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T09:43:42.663 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.665 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.718 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.720 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.773 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.775 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T09:43:42.882 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:42.885 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:42.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:42.943 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:42.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:43.002 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:43.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:43.061 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T09:43:43.116 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-16T09:43:43.117 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-16T09:43:43.143 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T09:43:43.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-16T09:43:43.270 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-16T09:43:43.297 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T09:43:43.395 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-16T09:43:43.422 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-16T09:43:43.489 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T09:43:43.587 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-16T09:43:43.614 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:43.617 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T09:43:43.671 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-16T09:43:43.671 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-16T09:43:43.672 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-16T09:43:43.675 INFO:tasks.ceph.mon.a:Started 2023-10-16T09:43:43.675 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-16T09:43:43.675 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-16T09:43:43.715 INFO:tasks.ceph.mon.b:Started 2023-10-16T09:43:43.716 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-16T09:43:43.716 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-16T09:43:43.718 INFO:tasks.ceph.mon.c:Started 2023-10-16T09:43:43.718 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-16T09:43:43.719 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-16T09:43:43.719 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-16T09:43:43.722 INFO:tasks.ceph.mgr.y:Started 2023-10-16T09:43:43.722 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-16T09:43:43.723 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-16T09:43:43.724 INFO:tasks.ceph.mgr.x:Started 2023-10-16T09:43:43.725 INFO:tasks.ceph:Setting crush tunables to default 2023-10-16T09:43:43.725 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd crush tunables default 2023-10-16T09:43:43.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.847+0000 7fa51b880700 1 Processor -- start 2023-10-16T09:43:43.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.847+0000 7fa51b880700 1 -- start start 2023-10-16T09:43:43.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa51b880700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:43.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa51b880700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5140cf580 con 0x7fa5140ce2d0 2023-10-16T09:43:43.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa51b880700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5140ce970 con 0x7fa5140d01d0 2023-10-16T09:43:43.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa51b880700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5140d4290 con 0x7fa51415ea40 2023-10-16T09:43:43.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa519e1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:43.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa519e1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35392/0 (socket says 172.21.15.181:35392) 2023-10-16T09:43:43.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa518e1b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa5140ce2d0 0x7fa5140d1500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46468/0 (socket says 172.21.15.181:46468) 2023-10-16T09:43:43.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa519e1d700 1 -- 172.21.15.181:0/870301614 learned_addr learned my addr 172.21.15.181:0/870301614 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:43.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa519e1d700 1 -- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 msgr2=0x7fa5140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-16T09:43:43.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa519e1d700 1 -- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 msgr2=0x7fa5140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T09:43:43.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa519e1d700 1 --2- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T09:43:43.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:43.848+0000 7fa519e1d700 1 --2- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:43:44.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.049+0000 7fa519e1d700 1 --2- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.049+0000 7fa519e1d700 1 -- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 msgr2=0x7fa5140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-16T09:43:44.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.049+0000 7fa519e1d700 1 -- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 msgr2=0x7fa5140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T09:43:44.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.049+0000 7fa519e1d700 1 --2- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T09:43:44.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.049+0000 7fa519e1d700 1 --2- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:43:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.054+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748184157 0 0) 0x7fa5140cf580 con 0x7fa5140ce2d0 2023-10-16T09:43:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.054+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5000035b0 con 0x7fa5140ce2d0 2023-10-16T09:43:44.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687176377 0 0) 0x7fa5140ce970 con 0x7fa5140d01d0 2023-10-16T09:43:44.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5140cf580 con 0x7fa5140d01d0 2023-10-16T09:43:44.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1867276453 0 0) 0x7fa5140cf580 con 0x7fa5140d01d0 2023-10-16T09:43:44.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 msgr2=0x7fa5140d1c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:43:44.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 --2- 172.21.15.181:0/870301614 >> v2:172.21.15.181:3300/0 conn(0x7fa51415ea40 0x7fa5140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 >> v1:172.21.15.181:6789/0 conn(0x7fa5140ce2d0 legacy=0x7fa5140d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140cec50 con 0x7fa5140d01d0 2023-10-16T09:43:44.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa510001670 con 0x7fa5140d01d0 2023-10-16T09:43:44.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 >> v1:172.21.15.190:6789/0 conn(0x7fa5140d01d0 legacy=0x7fa5140d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa500003b90 0x7fa500003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa500004850 0x7fa500004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500005210 0x7fa50000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50000c110 con 0x7fa500003b90 2023-10-16T09:43:44.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50000c280 con 0x7fa500005210 2023-10-16T09:43:44.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50000c440 con 0x7fa500004850 2023-10-16T09:43:44.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa51961c700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa500003b90 0x7fa500003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa518e1b700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa500004850 0x7fa500004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa519e1d700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500005210 0x7fa50000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa51961c700 1 -- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa500004850 msgr2=0x7fa500004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:44.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa51961c700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa500004850 0x7fa500004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa51961c700 1 -- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500005210 msgr2=0x7fa50000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:44.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa51961c700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500005210 0x7fa50000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.055+0000 7fa51961c700 1 -- 172.21.15.181:0/870301614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140cf580 con 0x7fa500003b90 2023-10-16T09:43:44.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.056+0000 7fa519e1d700 1 -- 172.21.15.181:0/870301614 reap_dead start 2023-10-16T09:43:44.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.056+0000 7fa51961c700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa500003b90 0x7fa500003f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa510007fb0 tx=0x7fa500005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94d5b4c6617f900 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:44.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.056+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000cc10 con 0x7fa500003b90 2023-10-16T09:43:44.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.056+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51000cd70 con 0x7fa500003b90 2023-10-16T09:43:44.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.056+0000 7fa50bfff700 1 -- 172.21.15.181:0/870301614 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510012640 con 0x7fa500003b90 2023-10-16T09:43:44.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.057+0000 7fa51b880700 1 -- 172.21.15.181:0/870301614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa500003b90 msgr2=0x7fa500003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.057+0000 7fa51b880700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa500003b90 0x7fa500003f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa510007fb0 tx=0x7fa500005880 comp rx=0 tx=0).stop 2023-10-16T09:43:44.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.057+0000 7fa51b880700 1 -- 172.21.15.181:0/870301614 shutdown_connections 2023-10-16T09:43:44.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.057+0000 7fa51b880700 1 --2- 172.21.15.181:0/870301614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa500003b90 0x7fa500003f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.057+0000 7fa51b880700 1 -- 172.21.15.181:0/870301614 >> 172.21.15.181:0/870301614 conn(0x7fa51409cb60 msgr2=0x7fa5140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:44.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.057+0000 7fa51b880700 1 -- 172.21.15.181:0/870301614 shutdown_connections 2023-10-16T09:43:44.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.057+0000 7fa51b880700 1 -- 172.21.15.181:0/870301614 wait complete. 2023-10-16T09:43:44.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.059+0000 7fa51b880700 1 Processor -- start 2023-10-16T09:43:44.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.059+0000 7fa51b880700 1 -- start start 2023-10-16T09:43:44.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51b880700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500003b90 0x7fa5141f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51b880700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5140c9180 0x7fa5140c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51b880700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5140c70e0 0x7fa5140c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51b880700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5140c8a90 con 0x7fa5140c70e0 2023-10-16T09:43:44.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51b880700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5140c9b40 con 0x7fa500003b90 2023-10-16T09:43:44.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51b880700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5141f3030 con 0x7fa5140c9180 2023-10-16T09:43:44.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51961c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500003b90 0x7fa5141f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa519e1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5140c70e0 0x7fa5140c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51961c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500003b90 0x7fa5141f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:40472/0 (socket says 172.21.15.181:40472) 2023-10-16T09:43:44.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa51961c700 1 -- 172.21.15.181:0/400530353 learned_addr learned my addr 172.21.15.181:0/400530353 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:44.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa518e1b700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5140c9180 0x7fa5140c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa519e1d700 1 -- 172.21.15.181:0/400530353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5140c9180 msgr2=0x7fa5140c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa519e1d700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5140c9180 0x7fa5140c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa519e1d700 1 -- 172.21.15.181:0/400530353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500003b90 msgr2=0x7fa5141f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa519e1d700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500003b90 0x7fa5141f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa519e1d700 1 -- 172.21.15.181:0/400530353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140c7e10 con 0x7fa5140c70e0 2023-10-16T09:43:44.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa519e1d700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5140c70e0 0x7fa5140c7560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa51415ee20 tx=0x7fa50c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:44.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.060+0000 7fa509ffb700 1 -- 172.21.15.181:0/400530353 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0118e0 con 0x7fa5140c70e0 2023-10-16T09:43:44.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.061+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5140c80f0 con 0x7fa5140c70e0 2023-10-16T09:43:44.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.061+0000 7fa509ffb700 1 -- 172.21.15.181:0/400530353 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c00f050 con 0x7fa5140c70e0 2023-10-16T09:43:44.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.061+0000 7fa509ffb700 1 -- 172.21.15.181:0/400530353 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c011c80 con 0x7fa5140c70e0 2023-10-16T09:43:44.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.061+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5140c86b0 con 0x7fa5140c70e0 2023-10-16T09:43:44.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.061+0000 7fa509ffb700 1 -- 172.21.15.181:0/400530353 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa50c01f710 con 0x7fa5140c70e0 2023-10-16T09:43:44.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.061+0000 7fa509ffb700 1 -- 172.21.15.181:0/400530353 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa50c015610 con 0x7fa5140c70e0 2023-10-16T09:43:44.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.062+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5140c8840 con 0x7fa5140c70e0 2023-10-16T09:43:44.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.066+0000 7fa509ffb700 1 -- 172.21.15.181:0/400530353 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa50c01fba0 con 0x7fa5140c70e0 2023-10-16T09:43:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.142+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa51404ebb0 con 0x7fa5140c70e0 2023-10-16T09:43:44.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.143+0000 7fa509ffb700 1 -- 172.21.15.181:0/400530353 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa50c01fd40 con 0x7fa5140c70e0 2023-10-16T09:43:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.144+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5140c70e0 msgr2=0x7fa5140c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.144+0000 7fa51b880700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5140c70e0 0x7fa5140c7560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa51415ee20 tx=0x7fa50c007db0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.144+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 shutdown_connections 2023-10-16T09:43:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.144+0000 7fa51b880700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5140c70e0 0x7fa5140c7560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.144+0000 7fa51b880700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa500003b90 0x7fa5141f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.144+0000 7fa51b880700 1 --2- 172.21.15.181:0/400530353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5140c9180 0x7fa5140c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.144+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 >> 172.21.15.181:0/400530353 conn(0x7fa51409cb60 msgr2=0x7fa51406bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.145+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 shutdown_connections 2023-10-16T09:43:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:44.145+0000 7fa51b880700 1 -- 172.21.15.181:0/400530353 wait complete. 2023-10-16T09:43:44.149 INFO:teuthology.orchestra.run.smithi181.stderr:adjusted tunables profile to default 2023-10-16T09:43:44.159 INFO:tasks.ceph:check_enable_crimson: False 2023-10-16T09:43:44.159 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-16T09:43:44.159 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:44.160 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-16T09:43:44.172 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:44.170+0000 7f9e9cb4e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:43:44.179 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:44.177+0000 7f7072d4a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:43:44.185 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:44.186 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-16T09:43:44.236 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:44.235+0000 7f9e9cb4e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:43:44.250 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:44.248+0000 7f7072d4a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:43:44.251 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:44.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-16T09:43:44.313 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:44.311+0000 7f9e9cb4e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:43:44.318 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:43:44.318 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-16T09:43:44.333 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:44.331+0000 7f7072d4a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:43:44.360 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:44.360 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-16T09:43:44.388 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:44.388 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-16T09:43:44.394 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:44.393+0000 7f9e9cb4e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:43:44.418 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:44.416+0000 7f7072d4a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:43:44.419 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:44.420 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-16T09:43:44.487 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:43:44.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-16T09:43:44.543 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:44.542+0000 7f9e9cb4e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:43:44.554 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new b92cf7de-b5f2-4270-83c3-2aaf2f896a5d 0 2023-10-16T09:43:44.560 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:44.558+0000 7f7072d4a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:43:44.716 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.715+0000 7fb3ecf74700 1 Processor -- start 2023-10-16T09:43:44.717 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.715+0000 7fb3ecf74700 1 -- start start 2023-10-16T09:43:44.719 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3ecf74700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3e80b2010 0x7fb3e80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.719 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3ecf74700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3e8071be0 0x7fb3e80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.719 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3ecf74700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e806f3b0 0x7fb3e806f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.719 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3ecf74700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e80ce1f0 con 0x7fb3e80b2010 2023-10-16T09:43:44.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3ecf74700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e8072090 con 0x7fb3e8071be0 2023-10-16T09:43:44.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3ecf74700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80b2930 con 0x7fb3e806f3b0 2023-10-16T09:43:44.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3e8071be0 0x7fb3e80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e6d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e806f3b0 0x7fb3e806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.720 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e6d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e806f3b0 0x7fb3e806f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48148/0 (socket says 172.21.15.190:48148) 2023-10-16T09:43:44.721 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e5d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3e8071be0 0x7fb3e80b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34082/0 (socket says 172.21.15.190:34082) 2023-10-16T09:43:44.721 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e6d9d700 1 -- 172.21.15.190:0/1354646646 learned_addr learned my addr 172.21.15.190:0/1354646646 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:44.721 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e659c700 1 --2- 172.21.15.190:0/1354646646 >> v2:172.21.15.181:3300/0 conn(0x7fb3e80b2010 0x7fb3e80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.721 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e5d9b700 1 -- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e806f3b0 msgr2=0x7fb3e806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:44.721 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e5d9b700 1 --2- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e806f3b0 0x7fb3e806f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.722 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e5d9b700 1 -- 172.21.15.190:0/1354646646 >> v2:172.21.15.181:3300/0 conn(0x7fb3e80b2010 msgr2=0x7fb3e80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:44.722 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e5d9b700 1 --2- 172.21.15.190:0/1354646646 >> v2:172.21.15.181:3300/0 conn(0x7fb3e80b2010 0x7fb3e80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.722 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e5d9b700 1 -- 172.21.15.190:0/1354646646 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e80ccda0 con 0x7fb3e8071be0 2023-10-16T09:43:44.722 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.716+0000 7fb3e6d9d700 1 --2- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e806f3b0 0x7fb3e806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:44.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3e5d9b700 1 --2- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e8071be0 0x7fb3e80b23f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3e815e1c0 tx=0x7fb3dc009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42128cb697967d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:44.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3e559a700 1 -- 172.21.15.190:0/1354646646 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc011830 con 0x7fb3e8071be0 2023-10-16T09:43:44.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3e559a700 1 -- 172.21.15.190:0/1354646646 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc011990 con 0x7fb3e8071be0 2023-10-16T09:43:44.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3e559a700 1 -- 172.21.15.190:0/1354646646 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc011d20 con 0x7fb3e8071be0 2023-10-16T09:43:44.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 -- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e8071be0 msgr2=0x7fb3e80b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.723 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e8071be0 0x7fb3e80b23f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3e815e1c0 tx=0x7fb3dc009b20 comp rx=0 tx=0).stop 2023-10-16T09:43:44.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 -- 172.21.15.190:0/1354646646 shutdown_connections 2023-10-16T09:43:44.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/1354646646 >> v2:172.21.15.181:3300/0 conn(0x7fb3e80b2010 0x7fb3e80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e806f3b0 0x7fb3e806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/1354646646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e8071be0 0x7fb3e80b23f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.724 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 -- 172.21.15.190:0/1354646646 >> 172.21.15.190:0/1354646646 conn(0x7fb3e80cd010 msgr2=0x7fb3e80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:44.725 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 -- 172.21.15.190:0/1354646646 shutdown_connections 2023-10-16T09:43:44.725 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.717+0000 7fb3ecf74700 1 -- 172.21.15.190:0/1354646646 wait complete. 2023-10-16T09:43:44.727 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.719+0000 7fb3ecf74700 1 Processor -- start 2023-10-16T09:43:44.727 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.719+0000 7fb3ecf74700 1 -- start start 2023-10-16T09:43:44.727 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.719+0000 7fb3ecf74700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e806f3b0 0x7fb3e80afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.728 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.719+0000 7fb3ecf74700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e8071be0 0x7fb3e80b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.728 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.719+0000 7fb3ecf74700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e80b2010 0x7fb3e80c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.728 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.719+0000 7fb3ecf74700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80adb90 con 0x7fb3e8071be0 2023-10-16T09:43:44.728 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3ecf74700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80acf80 con 0x7fb3e806f3b0 2023-10-16T09:43:44.728 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3ecf74700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80ad260 con 0x7fb3e80b2010 2023-10-16T09:43:44.729 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e806f3b0 0x7fb3e80afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.729 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e806f3b0 0x7fb3e80afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34086/0 (socket says 172.21.15.190:34086) 2023-10-16T09:43:44.729 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 -- 172.21.15.190:0/4080305864 learned_addr learned my addr 172.21.15.190:0/4080305864 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:44.729 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e6d9d700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e80b2010 0x7fb3e80c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.729 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e5d9b700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e8071be0 0x7fb3e80b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.729 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 -- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e80b2010 msgr2=0x7fb3e80c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.730 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e80b2010 0x7fb3e80c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.730 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 -- 172.21.15.190:0/4080305864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e8071be0 msgr2=0x7fb3e80b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.730 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e8071be0 0x7fb3e80b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.730 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 -- 172.21.15.190:0/4080305864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e000bd40 con 0x7fb3e806f3b0 2023-10-16T09:43:44.730 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3e659c700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e806f3b0 0x7fb3e80afce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800e870 tx=0x7fb3d800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:44.731 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3d6ffd700 1 -- 172.21.15.190:0/4080305864 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8009590 con 0x7fb3e806f3b0 2023-10-16T09:43:44.731 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3d6ffd700 1 -- 172.21.15.190:0/4080305864 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d8013680 con 0x7fb3e806f3b0 2023-10-16T09:43:44.731 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3dc0094a0 con 0x7fb3e806f3b0 2023-10-16T09:43:44.731 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3d6ffd700 1 -- 172.21.15.190:0/4080305864 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80139f0 con 0x7fb3e806f3b0 2023-10-16T09:43:44.731 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e80ad820 con 0x7fb3e806f3b0 2023-10-16T09:43:44.732 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3d6ffd700 1 -- 172.21.15.190:0/4080305864 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb3d8005a20 con 0x7fb3e806f3b0 2023-10-16T09:43:44.732 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.720+0000 7fb3d6ffd700 1 -- 172.21.15.190:0/4080305864 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb3d801c790 con 0x7fb3e806f3b0 2023-10-16T09:43:44.732 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.721+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e80ade70 con 0x7fb3e806f3b0 2023-10-16T09:43:44.732 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.723+0000 7fb3d6ffd700 1 -- 172.21.15.190:0/4080305864 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb3d8021060 con 0x7fb3e806f3b0 2023-10-16T09:43:44.807 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.806+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b92cf7de-b5f2-4270-83c3-2aaf2f896a5d", "id": 0} v 0) v1 -- 0x7fb3e8072a90 con 0x7fb3e806f3b0 2023-10-16T09:43:44.813 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.811+0000 7fb3d6ffd700 1 -- 172.21.15.190:0/4080305864 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b92cf7de-b5f2-4270-83c3-2aaf2f896a5d", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb3d8017070 con 0x7fb3e806f3b0 2023-10-16T09:43:44.813 INFO:teuthology.orchestra.run.smithi190.stdout:0 2023-10-16T09:43:44.814 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e806f3b0 msgr2=0x7fb3e80afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e806f3b0 0x7fb3e80afce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800e870 tx=0x7fb3d800eb80 comp rx=0 tx=0).stop 2023-10-16T09:43:44.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 shutdown_connections 2023-10-16T09:43:44.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3e80b2010 0x7fb3e80c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3e806f3b0 0x7fb3e80afce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.815 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 --2- 172.21.15.190:0/4080305864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e8071be0 0x7fb3e80b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.816 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 >> 172.21.15.190:0/4080305864 conn(0x7fb3e80cd010 msgr2=0x7fb3e8159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:44.816 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 shutdown_connections 2023-10-16T09:43:44.816 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.812+0000 7fb3ecf74700 1 -- 172.21.15.190:0/4080305864 wait complete. 2023-10-16T09:43:44.825 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 620b29ff-a66f-46b7-bb2f-949b053468cc 1 2023-10-16T09:43:44.947 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.945+0000 7f06592c4700 1 Processor -- start 2023-10-16T09:43:44.947 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.945+0000 7f06592c4700 1 -- start start 2023-10-16T09:43:44.948 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f06592c4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb690 0x7f065406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.948 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f06592c4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0654070590 con 0x7f06540cb2b0 2023-10-16T09:43:44.948 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f06592c4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0654156330 con 0x7f06540b5fd0 2023-10-16T09:43:44.949 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f06592c4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065406fcf0 con 0x7f06540cb690 2023-10-16T09:43:44.949 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f9e9cb4e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:43:44.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f06537fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f06540b5fd0 0x7f065415f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:60668/0 (socket says 172.21.15.190:60668) 2023-10-16T09:43:44.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f06537fe700 1 -- 172.21.15.190:0/3672647912 learned_addr learned my addr 172.21.15.190:0/3672647912 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-10-16T09:43:44.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.946+0000 7f06527fc700 1 --2- 172.21.15.190:0/3672647912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb690 0x7f065406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f0651ffb700 1 -- 172.21.15.190:0/3672647912 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360030256 0 0) 0x7f0654156330 con 0x7f06540b5fd0 2023-10-16T09:43:44.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f0651ffb700 1 -- 172.21.15.190:0/3672647912 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063c0035b0 con 0x7f06540b5fd0 2023-10-16T09:43:44.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f06527fc700 1 -- 172.21.15.190:0/3672647912 >> v1:172.21.15.190:6789/0 conn(0x7f06540b5fd0 legacy=0x7f065415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f06527fc700 1 -- 172.21.15.190:0/3672647912 >> v1:172.21.15.181:6789/0 conn(0x7f06540cb2b0 legacy=0x7f06540b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f06527fc700 1 -- 172.21.15.190:0/3672647912 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065406ffd0 con 0x7f06540cb690 2023-10-16T09:43:44.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f06527fc700 1 --2- 172.21.15.190:0/3672647912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb690 0x7f065406f710 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0648009870 tx=0x7f0648009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f493f46c69384c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:44.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f0651ffb700 1 -- 172.21.15.190:0/3672647912 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648011830 con 0x7f06540cb690 2023-10-16T09:43:44.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f0651ffb700 1 -- 172.21.15.190:0/3672647912 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0648011990 con 0x7f06540cb690 2023-10-16T09:43:44.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.947+0000 7f0651ffb700 1 -- 172.21.15.190:0/3672647912 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648011d20 con 0x7f06540cb690 2023-10-16T09:43:44.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.948+0000 7f06592c4700 1 -- 172.21.15.190:0/3672647912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb690 msgr2=0x7f065406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.953 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.948+0000 7f06592c4700 1 --2- 172.21.15.190:0/3672647912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb690 0x7f065406f710 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0648009870 tx=0x7f0648009b80 comp rx=0 tx=0).stop 2023-10-16T09:43:44.954 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.948+0000 7f06592c4700 1 -- 172.21.15.190:0/3672647912 shutdown_connections 2023-10-16T09:43:44.954 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.948+0000 7f06592c4700 1 --2- 172.21.15.190:0/3672647912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb690 0x7f065406f710 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.954 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.948+0000 7f06592c4700 1 -- 172.21.15.190:0/3672647912 >> 172.21.15.190:0/3672647912 conn(0x7f06540cd010 msgr2=0x7f06540cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:44.954 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.948+0000 7f06592c4700 1 -- 172.21.15.190:0/3672647912 shutdown_connections 2023-10-16T09:43:44.954 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.948+0000 7f06592c4700 1 -- 172.21.15.190:0/3672647912 wait complete. 2023-10-16T09:43:44.955 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:44.948+0000 7f7072d4a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:43:44.957 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.950+0000 7f06592c4700 1 Processor -- start 2023-10-16T09:43:44.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.950+0000 7f06592c4700 1 -- start start 2023-10-16T09:43:44.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.951+0000 7f06592c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540b5fd0 0x7f065415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.951+0000 7f06592c4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb2b0 0x7f06540ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.951+0000 7f06592c4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06540cb690 0x7f0654154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:44.958 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.951+0000 7f06592c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06540acfc0 con 0x7f06540b5fd0 2023-10-16T09:43:44.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.951+0000 7f06592c4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06540cf5f0 con 0x7f06540cb690 2023-10-16T09:43:44.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.951+0000 7f06592c4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06540cf8d0 con 0x7f06540cb2b0 2023-10-16T09:43:44.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.951+0000 7f06527fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb2b0 0x7f06540ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb2b0 0x7f06540ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48164/0 (socket says 172.21.15.190:48164) 2023-10-16T09:43:44.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 -- 172.21.15.190:0/1789400964 learned_addr learned my addr 172.21.15.190:0/1789400964 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:44.959 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06537fe700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06540cb690 0x7f0654154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f0652ffd700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540b5fd0 0x7f065415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:44.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 -- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06540cb690 msgr2=0x7f0654154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06540cb690 0x7f0654154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 -- 172.21.15.190:0/1789400964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540b5fd0 msgr2=0x7f065415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:44.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540b5fd0 0x7f065415c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:44.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 -- 172.21.15.190:0/1789400964 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480094a0 con 0x7f06540cb2b0 2023-10-16T09:43:44.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06537fe700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06540cb690 0x7f0654154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:44.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06527fc700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb2b0 0x7f06540ceb70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06480097c0 tx=0x7f0648004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:44.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f063b7fe700 1 -- 172.21.15.190:0/1789400964 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648004590 con 0x7f06540cb2b0 2023-10-16T09:43:44.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f063b7fe700 1 -- 172.21.15.190:0/1789400964 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06480036a0 con 0x7f06540cb2b0 2023-10-16T09:43:44.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f063b7fe700 1 -- 172.21.15.190:0/1789400964 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648003a30 con 0x7f06540cb2b0 2023-10-16T09:43:44.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06540ad210 con 0x7f06540cb2b0 2023-10-16T09:43:44.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.952+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065415a2f0 con 0x7f06540cb2b0 2023-10-16T09:43:44.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.953+0000 7f063b7fe700 1 -- 172.21.15.190:0/1789400964 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f064802e450 con 0x7f06540cb2b0 2023-10-16T09:43:44.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.953+0000 7f063b7fe700 1 -- 172.21.15.190:0/1789400964 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f064801a7d0 con 0x7f06540cb2b0 2023-10-16T09:43:44.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.953+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06540ad730 con 0x7f06540cb2b0 2023-10-16T09:43:44.962 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:44.956+0000 7f063b7fe700 1 -- 172.21.15.190:0/1789400964 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f06540ad730 con 0x7f06540cb2b0 2023-10-16T09:43:45.041 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.040+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "620b29ff-a66f-46b7-bb2f-949b053468cc", "id": 1} v 0) v1 -- 0x7f06540cbf80 con 0x7f06540cb2b0 2023-10-16T09:43:45.046 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.045+0000 7f063b7fe700 1 -- 172.21.15.190:0/1789400964 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "620b29ff-a66f-46b7-bb2f-949b053468cc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0648015020 con 0x7f06540cb2b0 2023-10-16T09:43:45.047 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-10-16T09:43:45.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb2b0 msgr2=0x7f06540ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb2b0 0x7f06540ceb70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06480097c0 tx=0x7f0648004750 comp rx=0 tx=0).stop 2023-10-16T09:43:45.048 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 shutdown_connections 2023-10-16T09:43:45.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06540cb690 0x7f0654154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06540b5fd0 0x7f065415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 --2- 172.21.15.190:0/1789400964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06540cb2b0 0x7f06540ceb70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 >> 172.21.15.190:0/1789400964 conn(0x7f06540cd010 msgr2=0x7f065415e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.049 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 shutdown_connections 2023-10-16T09:43:45.050 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.046+0000 7f06592c4700 1 -- 172.21.15.190:0/1789400964 wait complete. 2023-10-16T09:43:45.060 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 151b692d-6678-4237-b515-29e7c9df96f0 2 2023-10-16T09:43:45.068 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:45.067+0000 7f7072d4a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:43:45.074 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:45.073+0000 7f9e9cb4e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:43:45.153 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:45.151+0000 7f7072d4a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:43:45.174 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:45.170+0000 7f9e9cb4e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:43:45.181 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.180+0000 7f8468f90700 1 Processor -- start 2023-10-16T09:43:45.182 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.180+0000 7f8468f90700 1 -- start start 2023-10-16T09:43:45.184 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8468f90700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8464072760 0x7f846406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.185 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8468f90700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f846406ade0 0x7f84641615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.185 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8468f90700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846406b210 con 0x7f846415da70 2023-10-16T09:43:45.185 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8468f90700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84640cef00 con 0x7f846406ade0 2023-10-16T09:43:45.185 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8468f90700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84640cec20 con 0x7f8464072760 2023-10-16T09:43:45.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8461d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8464072760 0x7f846406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8461d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8464072760 0x7f846406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34106/0 (socket says 172.21.15.190:34106) 2023-10-16T09:43:45.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8461d9b700 1 -- 172.21.15.190:0/3308805259 learned_addr learned my addr 172.21.15.190:0/3308805259 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:45.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8462d9d700 1 --2- 172.21.15.190:0/3308805259 >> v2:172.21.15.181:3300/0 conn(0x7f846406ade0 0x7f84641615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.186 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8461d9b700 1 -- 172.21.15.190:0/3308805259 >> v2:172.21.15.181:3300/0 conn(0x7f846406ade0 msgr2=0x7f84641615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:45.187 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8461d9b700 1 --2- 172.21.15.190:0/3308805259 >> v2:172.21.15.181:3300/0 conn(0x7f846406ade0 0x7f84641615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.187 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8461d9b700 1 -- 172.21.15.190:0/3308805259 >> v1:172.21.15.181:6789/0 conn(0x7f846415da70 legacy=0x7f84640ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.187 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.181+0000 7f8461d9b700 1 -- 172.21.15.190:0/3308805259 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8464071c80 con 0x7f8464072760 2023-10-16T09:43:45.187 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.182+0000 7f8462d9d700 1 --2- 172.21.15.190:0/3308805259 >> v2:172.21.15.181:3300/0 conn(0x7f846406ade0 0x7f84641615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:43:45.187 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.182+0000 7f8461d9b700 1 --2- 172.21.15.190:0/3308805259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8464072760 0x7f846406a8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f846406ef60 tx=0x7f8458009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a558caca3b7f025 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.188 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.182+0000 7f846159a700 1 -- 172.21.15.190:0/3308805259 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800b870 con 0x7f8464072760 2023-10-16T09:43:45.188 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.182+0000 7f846159a700 1 -- 172.21.15.190:0/3308805259 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845800b9d0 con 0x7f8464072760 2023-10-16T09:43:45.188 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.182+0000 7f846159a700 1 -- 172.21.15.190:0/3308805259 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800bd60 con 0x7f8464072760 2023-10-16T09:43:45.188 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.182+0000 7f8468f90700 1 -- 172.21.15.190:0/3308805259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8464072760 msgr2=0x7f846406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.189 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.182+0000 7f8468f90700 1 --2- 172.21.15.190:0/3308805259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8464072760 0x7f846406a8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f846406ef60 tx=0x7f8458009bd0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.189 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.183+0000 7f8468f90700 1 -- 172.21.15.190:0/3308805259 shutdown_connections 2023-10-16T09:43:45.189 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.183+0000 7f8468f90700 1 --2- 172.21.15.190:0/3308805259 >> v2:172.21.15.181:3300/0 conn(0x7f846406ade0 0x7f84641615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.189 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.183+0000 7f8468f90700 1 --2- 172.21.15.190:0/3308805259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8464072760 0x7f846406a8a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.189 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.183+0000 7f8468f90700 1 -- 172.21.15.190:0/3308805259 >> 172.21.15.190:0/3308805259 conn(0x7f84640cd010 msgr2=0x7f84640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.190 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.183+0000 7f8468f90700 1 -- 172.21.15.190:0/3308805259 shutdown_connections 2023-10-16T09:43:45.190 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.183+0000 7f8468f90700 1 -- 172.21.15.190:0/3308805259 wait complete. 2023-10-16T09:43:45.192 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.184+0000 7f8468f90700 1 Processor -- start 2023-10-16T09:43:45.192 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.184+0000 7f8468f90700 1 -- start start 2023-10-16T09:43:45.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8468f90700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f846406ade0 0x7f84641505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8468f90700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8464072760 0x7f84640b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8468f90700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f846415da70 0x7f84640b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8468f90700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84640c6b00 con 0x7f846415da70 2023-10-16T09:43:45.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8468f90700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846415e190 con 0x7f846406ade0 2023-10-16T09:43:45.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8468f90700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846415e470 con 0x7f8464072760 2023-10-16T09:43:45.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f846406ade0 0x7f84641505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8461d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8464072760 0x7f84640b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f846406ade0 0x7f84641505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34110/0 (socket says 172.21.15.190:34110) 2023-10-16T09:43:45.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 -- 172.21.15.190:0/1160713635 learned_addr learned my addr 172.21.15.190:0/1160713635 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:45.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8462d9d700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f846415da70 0x7f84640b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.197 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 -- 172.21.15.190:0/1160713635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8464072760 msgr2=0x7f84640b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.197 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8464072760 0x7f84640b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.197 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 -- 172.21.15.190:0/1160713635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f846415da70 msgr2=0x7f84640b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.198 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f846415da70 0x7f84640b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.198 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 -- 172.21.15.190:0/1160713635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009550 con 0x7f846406ade0 2023-10-16T09:43:45.198 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f846259c700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f846406ade0 0x7f84641505e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f845400c4e0 tx=0x7f845400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.199 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8452ffd700 1 -- 172.21.15.190:0/1160713635 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84540052d0 con 0x7f846406ade0 2023-10-16T09:43:45.199 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8452ffd700 1 -- 172.21.15.190:0/1160713635 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8454005430 con 0x7f846406ade0 2023-10-16T09:43:45.199 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846415e750 con 0x7f846406ade0 2023-10-16T09:43:45.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.185+0000 7f8452ffd700 1 -- 172.21.15.190:0/1160713635 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84540038c0 con 0x7f846406ade0 2023-10-16T09:43:45.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.186+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846415ed10 con 0x7f846406ade0 2023-10-16T09:43:45.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.186+0000 7f8452ffd700 1 -- 172.21.15.190:0/1160713635 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8454003d20 con 0x7f846406ade0 2023-10-16T09:43:45.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.186+0000 7f8452ffd700 1 -- 172.21.15.190:0/1160713635 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f8454011da0 con 0x7f846406ade0 2023-10-16T09:43:45.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.187+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846415ef00 con 0x7f846406ade0 2023-10-16T09:43:45.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.189+0000 7f8452ffd700 1 -- 172.21.15.190:0/1160713635 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f8454017070 con 0x7f846406ade0 2023-10-16T09:43:45.235 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:45.233+0000 7f7072d4a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:43:45.256 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:45.255+0000 7f9e9cb4e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:43:45.274 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.273+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "151b692d-6678-4237-b515-29e7c9df96f0", "id": 2} v 0) v1 -- 0x7f846406f6c0 con 0x7f846406ade0 2023-10-16T09:43:45.280 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8452ffd700 1 -- 172.21.15.190:0/1160713635 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "151b692d-6678-4237-b515-29e7c9df96f0", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8454011590 con 0x7f846406ade0 2023-10-16T09:43:45.280 INFO:teuthology.orchestra.run.smithi190.stdout:2 2023-10-16T09:43:45.281 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f846406ade0 msgr2=0x7f84641505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.282 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8468f90700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f846406ade0 0x7f84641505e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f845400c4e0 tx=0x7f845400c7f0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.282 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 shutdown_connections 2023-10-16T09:43:45.282 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8468f90700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f846415da70 0x7f84640b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.282 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8468f90700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f846406ade0 0x7f84641505e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.282 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8468f90700 1 --2- 172.21.15.190:0/1160713635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8464072760 0x7f84640b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.283 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.279+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 >> 172.21.15.190:0/1160713635 conn(0x7f84640cd010 msgr2=0x7f846415cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.283 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.280+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 shutdown_connections 2023-10-16T09:43:45.283 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.280+0000 7f8468f90700 1 -- 172.21.15.190:0/1160713635 wait complete. 2023-10-16T09:43:45.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new a8e30fbf-6d07-4029-91b3-a42e22ba92e5 3 2023-10-16T09:43:45.457 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.455+0000 7f57baaa5700 1 Processor -- start 2023-10-16T09:43:45.457 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.455+0000 7f57baaa5700 1 -- start start 2023-10-16T09:43:45.459 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57baaa5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57b40b2010 0x7f57b40cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.459 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57baaa5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b40ce1f0 con 0x7f57b4071be0 2023-10-16T09:43:45.460 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57baaa5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b4072090 con 0x7f57b40ce680 2023-10-16T09:43:45.460 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57baaa5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b40ccda0 con 0x7f57b40b2010 2023-10-16T09:43:45.460 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b8841700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57b40b2010 0x7f57b40cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.460 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b8841700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57b40b2010 0x7f57b40cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34116/0 (socket says 172.21.15.190:34116) 2023-10-16T09:43:45.461 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b8841700 1 -- 172.21.15.190:0/2468133581 learned_addr learned my addr 172.21.15.190:0/2468133581 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:45.461 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b9042700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f57b40ce680 0x7f57b40ceaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:60692/0 (socket says 172.21.15.190:60692) 2023-10-16T09:43:45.461 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b8841700 1 -- 172.21.15.190:0/2468133581 >> v1:172.21.15.190:6789/0 conn(0x7f57b40ce680 legacy=0x7f57b40ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.461 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b8841700 1 -- 172.21.15.190:0/2468133581 >> v1:172.21.15.181:6789/0 conn(0x7f57b4071be0 legacy=0x7f57b40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.461 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b8841700 1 -- 172.21.15.190:0/2468133581 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b40727e0 con 0x7f57b40b2010 2023-10-16T09:43:45.461 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b37fe700 1 -- 172.21.15.190:0/2468133581 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604807707 0 0) 0x7f57b4072090 con 0x7f57b40ce680 2023-10-16T09:43:45.462 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b8841700 1 --2- 172.21.15.190:0/2468133581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40b2010 0x7f57b40cc6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57a800bb90 tx=0x7f57a800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53fa4a835a7c3140 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.462 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b37fe700 1 -- 172.21.15.190:0/2468133581 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a800dd50 con 0x7f57b40b2010 2023-10-16T09:43:45.462 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b37fe700 1 -- 172.21.15.190:0/2468133581 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a8010470 con 0x7f57b40b2010 2023-10-16T09:43:45.462 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.457+0000 7f57b37fe700 1 -- 172.21.15.190:0/2468133581 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010800 con 0x7f57b40b2010 2023-10-16T09:43:45.466 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.458+0000 7f57baaa5700 1 -- 172.21.15.190:0/2468133581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40b2010 msgr2=0x7f57b40cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.466 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.458+0000 7f57baaa5700 1 --2- 172.21.15.190:0/2468133581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40b2010 0x7f57b40cc6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57a800bb90 tx=0x7f57a800bea0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.466 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.458+0000 7f57baaa5700 1 -- 172.21.15.190:0/2468133581 shutdown_connections 2023-10-16T09:43:45.467 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.458+0000 7f57baaa5700 1 --2- 172.21.15.190:0/2468133581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40b2010 0x7f57b40cc6e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.467 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.458+0000 7f57baaa5700 1 -- 172.21.15.190:0/2468133581 >> 172.21.15.190:0/2468133581 conn(0x7f57b40cd010 msgr2=0x7f57b40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.467 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.458+0000 7f57baaa5700 1 -- 172.21.15.190:0/2468133581 shutdown_connections 2023-10-16T09:43:45.467 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.458+0000 7f57baaa5700 1 -- 172.21.15.190:0/2468133581 wait complete. 2023-10-16T09:43:45.467 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.459+0000 7f57baaa5700 1 Processor -- start 2023-10-16T09:43:45.468 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57baaa5700 1 -- start start 2023-10-16T09:43:45.468 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57baaa5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b4071be0 0x7f57b40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.468 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57baaa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b40b2010 0x7f57b40acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.468 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57baaa5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40ce680 0x7f57b41f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.468 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57baaa5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b41f7ba0 con 0x7f57b40b2010 2023-10-16T09:43:45.469 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57baaa5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4153b80 con 0x7f57b40ce680 2023-10-16T09:43:45.469 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57baaa5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b41f78c0 con 0x7f57b4071be0 2023-10-16T09:43:45.469 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.460+0000 7f57b8841700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b4071be0 0x7f57b40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.469 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b9042700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40ce680 0x7f57b41f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.469 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b4071be0 0x7f57b40ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48186/0 (socket says 172.21.15.190:48186) 2023-10-16T09:43:45.470 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 -- 172.21.15.190:0/387997672 learned_addr learned my addr 172.21.15.190:0/387997672 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:45.470 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b3fff700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b40b2010 0x7f57b40acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.470 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 -- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40ce680 msgr2=0x7f57b41f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.470 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40ce680 0x7f57b41f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.470 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 -- 172.21.15.190:0/387997672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b40b2010 msgr2=0x7f57b40acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.471 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b40b2010 0x7f57b40acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.471 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 -- 172.21.15.190:0/387997672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a800b7f0 con 0x7f57b4071be0 2023-10-16T09:43:45.471 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b9042700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40ce680 0x7f57b41f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:45.471 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b3fff700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b40b2010 0x7f57b40acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:45.471 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b8841700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b4071be0 0x7f57b40ac880 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d5d0 tx=0x7f57a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.471 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b17fa700 1 -- 172.21.15.190:0/387997672 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8023640 con 0x7f57b4071be0 2023-10-16T09:43:45.472 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b17fa700 1 -- 172.21.15.190:0/387997672 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a80237a0 con 0x7f57b4071be0 2023-10-16T09:43:45.472 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b4153530 con 0x7f57b4071be0 2023-10-16T09:43:45.472 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b17fa700 1 -- 172.21.15.190:0/387997672 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8023b30 con 0x7f57b4071be0 2023-10-16T09:43:45.472 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b4146b60 con 0x7f57b4071be0 2023-10-16T09:43:45.472 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b17fa700 1 -- 172.21.15.190:0/387997672 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f57a8019400 con 0x7f57b4071be0 2023-10-16T09:43:45.473 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.461+0000 7f57b17fa700 1 -- 172.21.15.190:0/387997672 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f57a802c910 con 0x7f57b4071be0 2023-10-16T09:43:45.473 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.462+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4153390 con 0x7f57b4071be0 2023-10-16T09:43:45.473 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.464+0000 7f57b17fa700 1 -- 172.21.15.190:0/387997672 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f57b4153390 con 0x7f57b4071be0 2023-10-16T09:43:45.549 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.548+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a8e30fbf-6d07-4029-91b3-a42e22ba92e5", "id": 3} v 0) v1 -- 0x7f57b40b6510 con 0x7f57b4071be0 2023-10-16T09:43:45.555 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.554+0000 7f57b17fa700 1 -- 172.21.15.190:0/387997672 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a8e30fbf-6d07-4029-91b3-a42e22ba92e5", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f57a802c550 con 0x7f57b4071be0 2023-10-16T09:43:45.555 INFO:teuthology.orchestra.run.smithi190.stdout:3 2023-10-16T09:43:45.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.554+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b4071be0 msgr2=0x7f57b40ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.557 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.554+0000 7f57baaa5700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b4071be0 0x7f57b40ac880 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d5d0 tx=0x7f57a8004750 comp rx=0 tx=0).stop 2023-10-16T09:43:45.557 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.555+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 shutdown_connections 2023-10-16T09:43:45.557 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.555+0000 7f57baaa5700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b4071be0 0x7f57b40ac880 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.557 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.555+0000 7f57baaa5700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b40ce680 0x7f57b41f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.557 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.555+0000 7f57baaa5700 1 --2- 172.21.15.190:0/387997672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b40b2010 0x7f57b40acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.557 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.555+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 >> 172.21.15.190:0/387997672 conn(0x7f57b40cd010 msgr2=0x7f57b406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.558 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.555+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 shutdown_connections 2023-10-16T09:43:45.558 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.555+0000 7f57baaa5700 1 -- 172.21.15.190:0/387997672 wait complete. 2023-10-16T09:43:45.568 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new f35da5fd-caf3-44aa-b5c4-cc642216e1aa 4 2023-10-16T09:43:45.680 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:45.678+0000 7f7072d4a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:43:45.688 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.686+0000 7fef53a27700 1 Processor -- start 2023-10-16T09:43:45.688 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.687+0000 7fef53a27700 1 -- start start 2023-10-16T09:43:45.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.687+0000 7fef53a27700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c15b850 0x7fef4c0665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.687+0000 7fef53a27700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef4c06f1e0 0x7fef4c06f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.687+0000 7fef53a27700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c06e4b0 con 0x7fef4c15fee0 2023-10-16T09:43:45.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.687+0000 7fef53a27700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c06ed50 con 0x7fef4c06f1e0 2023-10-16T09:43:45.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.687+0000 7fef53a27700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0cb0f0 con 0x7fef4c15b850 2023-10-16T09:43:45.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.687+0000 7fef517c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c15b850 0x7fef4c0665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef517c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c15b850 0x7fef4c0665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48192/0 (socket says 172.21.15.190:48192) 2023-10-16T09:43:45.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef517c3700 1 -- 172.21.15.190:0/1121856022 learned_addr learned my addr 172.21.15.190:0/1121856022 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:45.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef50fc2700 1 --1- 172.21.15.190:0/1121856022 >> v1:172.21.15.190:6789/0 conn(0x7fef4c15fee0 0x7fef4c0c3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:60700/0 (socket says 172.21.15.190:60700) 2023-10-16T09:43:45.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef51fc4700 1 --2- 172.21.15.190:0/1121856022 >> v2:172.21.15.181:3300/0 conn(0x7fef4c06f1e0 0x7fef4c06f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef517c3700 1 -- 172.21.15.190:0/1121856022 >> v2:172.21.15.181:3300/0 conn(0x7fef4c06f1e0 msgr2=0x7fef4c06f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:45.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef517c3700 1 --2- 172.21.15.190:0/1121856022 >> v2:172.21.15.181:3300/0 conn(0x7fef4c06f1e0 0x7fef4c06f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef517c3700 1 -- 172.21.15.190:0/1121856022 >> v1:172.21.15.190:6789/0 conn(0x7fef4c15fee0 legacy=0x7fef4c0c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef517c3700 1 -- 172.21.15.190:0/1121856022 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c066b80 con 0x7fef4c15b850 2023-10-16T09:43:45.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef517c3700 1 --2- 172.21.15.190:0/1121856022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c15b850 0x7fef4c0665a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef480098f0 tx=0x7fef48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a463a001ffaffc64 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef43fff700 1 -- 172.21.15.190:0/1121856022 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48011830 con 0x7fef4c15b850 2023-10-16T09:43:45.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef43fff700 1 -- 172.21.15.190:0/1121856022 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef48011990 con 0x7fef4c15b850 2023-10-16T09:43:45.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.688+0000 7fef43fff700 1 -- 172.21.15.190:0/1121856022 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48011d20 con 0x7fef4c15b850 2023-10-16T09:43:45.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 -- 172.21.15.190:0/1121856022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c15b850 msgr2=0x7fef4c0665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 --2- 172.21.15.190:0/1121856022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c15b850 0x7fef4c0665a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef480098f0 tx=0x7fef48009c00 comp rx=0 tx=0).stop 2023-10-16T09:43:45.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 -- 172.21.15.190:0/1121856022 shutdown_connections 2023-10-16T09:43:45.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 --2- 172.21.15.190:0/1121856022 >> v2:172.21.15.181:3300/0 conn(0x7fef4c06f1e0 0x7fef4c06f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 --2- 172.21.15.190:0/1121856022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c15b850 0x7fef4c0665a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 -- 172.21.15.190:0/1121856022 >> 172.21.15.190:0/1121856022 conn(0x7fef4c0cc730 msgr2=0x7fef4c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 -- 172.21.15.190:0/1121856022 shutdown_connections 2023-10-16T09:43:45.696 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.689+0000 7fef53a27700 1 -- 172.21.15.190:0/1121856022 wait complete. 2023-10-16T09:43:45.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.691+0000 7fef53a27700 1 Processor -- start 2023-10-16T09:43:45.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.691+0000 7fef53a27700 1 -- start start 2023-10-16T09:43:45.698 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef53a27700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c06f1e0 0x7fef4c14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef53a27700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef4c15b850 0x7fef4c1f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef53a27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef4c15fee0 0x7fef4c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef53a27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1f56b0 con 0x7fef4c15fee0 2023-10-16T09:43:45.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef53a27700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1f1020 con 0x7fef4c15b850 2023-10-16T09:43:45.699 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef53a27700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c151600 con 0x7fef4c06f1e0 2023-10-16T09:43:45.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef517c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c06f1e0 0x7fef4c14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef50fc2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef4c15b850 0x7fef4c1f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef517c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c06f1e0 0x7fef4c14bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48198/0 (socket says 172.21.15.190:48198) 2023-10-16T09:43:45.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef50fc2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef4c15b850 0x7fef4c1f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34136/0 (socket says 172.21.15.190:34136) 2023-10-16T09:43:45.700 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef517c3700 1 -- 172.21.15.190:0/342094985 learned_addr learned my addr 172.21.15.190:0/342094985 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:45.701 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.692+0000 7fef51fc4700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef4c15fee0 0x7fef4c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.701 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef517c3700 1 -- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef4c15b850 msgr2=0x7fef4c1f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.701 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef517c3700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef4c15b850 0x7fef4c1f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.701 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef517c3700 1 -- 172.21.15.190:0/342094985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef4c15fee0 msgr2=0x7fef4c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.701 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef517c3700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef4c15fee0 0x7fef4c15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef517c3700 1 -- 172.21.15.190:0/342094985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48009550 con 0x7fef4c06f1e0 2023-10-16T09:43:45.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef50fc2700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef4c15b850 0x7fef4c1f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:45.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef517c3700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c06f1e0 0x7fef4c14bcc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef48000c00 tx=0x7fef48004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef41ffb700 1 -- 172.21.15.190:0/342094985 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48004480 con 0x7fef4c06f1e0 2023-10-16T09:43:45.702 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef41ffb700 1 -- 172.21.15.190:0/342094985 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4801f070 con 0x7fef4c06f1e0 2023-10-16T09:43:45.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef41ffb700 1 -- 172.21.15.190:0/342094985 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480038c0 con 0x7fef4c06f1e0 2023-10-16T09:43:45.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c14c200 con 0x7fef4c06f1e0 2023-10-16T09:43:45.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c1f53d0 con 0x7fef4c06f1e0 2023-10-16T09:43:45.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef41ffb700 1 -- 172.21.15.190:0/342094985 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fef48021030 con 0x7fef4c06f1e0 2023-10-16T09:43:45.703 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.693+0000 7fef41ffb700 1 -- 172.21.15.190:0/342094985 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fef48024430 con 0x7fef4c06f1e0 2023-10-16T09:43:45.704 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.694+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c06d810 con 0x7fef4c06f1e0 2023-10-16T09:43:45.704 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.696+0000 7fef41ffb700 1 -- 172.21.15.190:0/342094985 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fef48015020 con 0x7fef4c06f1e0 2023-10-16T09:43:45.713 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:45.712+0000 7f9e9cb4e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:43:45.779 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.778+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f35da5fd-caf3-44aa-b5c4-cc642216e1aa", "id": 4} v 0) v1 -- 0x7fef4c0cd5d0 con 0x7fef4c06f1e0 2023-10-16T09:43:45.785 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.783+0000 7fef41ffb700 1 -- 172.21.15.190:0/342094985 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f35da5fd-caf3-44aa-b5c4-cc642216e1aa", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef4801a450 con 0x7fef4c06f1e0 2023-10-16T09:43:45.785 INFO:teuthology.orchestra.run.smithi190.stdout:4 2023-10-16T09:43:45.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c06f1e0 msgr2=0x7fef4c14bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.786 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c06f1e0 0x7fef4c14bcc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef48000c00 tx=0x7fef48004750 comp rx=0 tx=0).stop 2023-10-16T09:43:45.787 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 shutdown_connections 2023-10-16T09:43:45.787 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef4c06f1e0 0x7fef4c14bcc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.787 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef4c15fee0 0x7fef4c15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.787 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 --2- 172.21.15.190:0/342094985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef4c15b850 0x7fef4c1f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.787 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 >> 172.21.15.190:0/342094985 conn(0x7fef4c0cc730 msgr2=0x7fef4c155c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.788 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 shutdown_connections 2023-10-16T09:43:45.788 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.784+0000 7fef53a27700 1 -- 172.21.15.190:0/342094985 wait complete. 2023-10-16T09:43:45.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new fa387ca5-fce1-4570-a767-5ee46549a491 5 2023-10-16T09:43:45.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.916+0000 7fa787cce700 1 Processor -- start 2023-10-16T09:43:45.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.917+0000 7fa787cce700 1 -- start start 2023-10-16T09:43:45.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.917+0000 7fa787cce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa7800d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.917+0000 7fa787cce700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7800cf580 con 0x7fa7800d01d0 2023-10-16T09:43:45.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.917+0000 7fa787cce700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7800ce970 con 0x7fa7800ce2d0 2023-10-16T09:43:45.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.917+0000 7fa787cce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800d4290 con 0x7fa78015ea40 2023-10-16T09:43:45.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa785269700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa7800ce2d0 0x7fa7800d1500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:60712/0 (socket says 172.21.15.190:60712) 2023-10-16T09:43:45.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa785269700 1 -- 172.21.15.190:0/2851981565 learned_addr learned my addr 172.21.15.190:0/2851981565 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-10-16T09:43:45.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa78626b700 1 --2- 172.21.15.190:0/2851981565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa7800d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa784a68700 1 -- 172.21.15.190:0/2851981565 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138525913 0 0) 0x7fa7800ce970 con 0x7fa7800ce2d0 2023-10-16T09:43:45.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa784a68700 1 -- 172.21.15.190:0/2851981565 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa76c0035b0 con 0x7fa7800ce2d0 2023-10-16T09:43:45.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa784a68700 1 -- 172.21.15.190:0/2851981565 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909572619 0 0) 0x7fa7800cf580 con 0x7fa7800d01d0 2023-10-16T09:43:45.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa78626b700 1 -- 172.21.15.190:0/2851981565 >> v1:172.21.15.190:6789/0 conn(0x7fa7800ce2d0 legacy=0x7fa7800d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa78626b700 1 -- 172.21.15.190:0/2851981565 >> v1:172.21.15.181:6789/0 conn(0x7fa7800d01d0 legacy=0x7fa7800d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.924 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa78626b700 1 -- 172.21.15.190:0/2851981565 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7800cec50 con 0x7fa78015ea40 2023-10-16T09:43:45.924 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa78626b700 1 --2- 172.21.15.190:0/2851981565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa7800d1c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa778009dc0 tx=0x7fa77800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fceaba2f3b0ec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.924 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa784a68700 1 -- 172.21.15.190:0/2851981565 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77800e070 con 0x7fa78015ea40 2023-10-16T09:43:45.924 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa784a68700 1 -- 172.21.15.190:0/2851981565 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77800bb20 con 0x7fa78015ea40 2023-10-16T09:43:45.924 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.918+0000 7fa784a68700 1 -- 172.21.15.190:0/2851981565 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77800bed0 con 0x7fa78015ea40 2023-10-16T09:43:45.925 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.919+0000 7fa787cce700 1 -- 172.21.15.190:0/2851981565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 msgr2=0x7fa7800d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.925 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.919+0000 7fa787cce700 1 --2- 172.21.15.190:0/2851981565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa7800d1c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa778009dc0 tx=0x7fa77800b330 comp rx=0 tx=0).stop 2023-10-16T09:43:45.925 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.919+0000 7fa787cce700 1 -- 172.21.15.190:0/2851981565 shutdown_connections 2023-10-16T09:43:45.925 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.919+0000 7fa787cce700 1 --2- 172.21.15.190:0/2851981565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa7800d1c10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.925 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.919+0000 7fa787cce700 1 -- 172.21.15.190:0/2851981565 >> 172.21.15.190:0/2851981565 conn(0x7fa78009cb60 msgr2=0x7fa7800d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:45.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.919+0000 7fa787cce700 1 -- 172.21.15.190:0/2851981565 shutdown_connections 2023-10-16T09:43:45.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.919+0000 7fa787cce700 1 -- 172.21.15.190:0/2851981565 wait complete. 2023-10-16T09:43:45.928 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.921+0000 7fa787cce700 1 Processor -- start 2023-10-16T09:43:45.929 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.921+0000 7fa787cce700 1 -- start start 2023-10-16T09:43:45.929 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.921+0000 7fa787cce700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7800ce2d0 0x7fa7800b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.929 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa787cce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7800d01d0 0x7fa7801f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.929 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa787cce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa78006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:45.929 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa787cce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800b2590 con 0x7fa7800d01d0 2023-10-16T09:43:45.930 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa787cce700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800b1cf0 con 0x7fa7800ce2d0 2023-10-16T09:43:45.930 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa787cce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800b1fd0 con 0x7fa78015ea40 2023-10-16T09:43:45.930 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7800ce2d0 0x7fa7800b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.930 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa78626b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa78006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.930 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7800ce2d0 0x7fa7800b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34146/0 (socket says 172.21.15.190:34146) 2023-10-16T09:43:45.931 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 -- 172.21.15.190:0/3876013552 learned_addr learned my addr 172.21.15.190:0/3876013552 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:45.931 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa78626b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa78006bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48214/0 (socket says 172.21.15.190:48214) 2023-10-16T09:43:45.931 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785269700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7800d01d0 0x7fa7801f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:45.931 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 -- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 msgr2=0x7fa78006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.931 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa78006bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.932 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 -- 172.21.15.190:0/3876013552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7800d01d0 msgr2=0x7fa7801f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:45.932 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7800d01d0 0x7fa7801f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:45.932 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 -- 172.21.15.190:0/3876013552 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009a70 con 0x7fa7800ce2d0 2023-10-16T09:43:45.932 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa78626b700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa78006bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:45.932 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa785a6a700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7800ce2d0 0x7fa7800b14a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0011e0 tx=0x7fa77c00ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:45.933 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa7767fc700 1 -- 172.21.15.190:0/3876013552 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0054e0 con 0x7fa7800ce2d0 2023-10-16T09:43:45.933 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa7767fc700 1 -- 172.21.15.190:0/3876013552 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c003680 con 0x7fa7800ce2d0 2023-10-16T09:43:45.933 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa7767fc700 1 -- 172.21.15.190:0/3876013552 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c003a30 con 0x7fa7800ce2d0 2023-10-16T09:43:45.933 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7800b22b0 con 0x7fa7800ce2d0 2023-10-16T09:43:45.933 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.922+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7800b19e0 con 0x7fa7800ce2d0 2023-10-16T09:43:45.934 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.923+0000 7fa7767fc700 1 -- 172.21.15.190:0/3876013552 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa77c01c440 con 0x7fa7800ce2d0 2023-10-16T09:43:45.934 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.923+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7800cae10 con 0x7fa7800ce2d0 2023-10-16T09:43:45.934 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.923+0000 7fa7767fc700 1 -- 172.21.15.190:0/3876013552 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa77c012980 con 0x7fa7800ce2d0 2023-10-16T09:43:45.934 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:45.927+0000 7fa7767fc700 1 -- 172.21.15.190:0/3876013552 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa7800cae10 con 0x7fa7800ce2d0 2023-10-16T09:43:46.012 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.010+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fa387ca5-fce1-4570-a767-5ee46549a491", "id": 5} v 0) v1 -- 0x7fa7800cb1f0 con 0x7fa7800ce2d0 2023-10-16T09:43:46.020 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.019+0000 7fa7767fc700 1 -- 172.21.15.190:0/3876013552 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa387ca5-fce1-4570-a767-5ee46549a491", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa77c017070 con 0x7fa7800ce2d0 2023-10-16T09:43:46.020 INFO:teuthology.orchestra.run.smithi190.stdout:5 2023-10-16T09:43:46.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7800ce2d0 msgr2=0x7fa7800b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7800ce2d0 0x7fa7800b14a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0011e0 tx=0x7fa77c00ca30 comp rx=0 tx=0).stop 2023-10-16T09:43:46.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 shutdown_connections 2023-10-16T09:43:46.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa78015ea40 0x7fa78006bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7800ce2d0 0x7fa7800b14a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 --2- 172.21.15.190:0/3876013552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7800d01d0 0x7fa7801f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 >> 172.21.15.190:0/3876013552 conn(0x7fa78009cb60 msgr2=0x7fa78006cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:46.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 shutdown_connections 2023-10-16T09:43:46.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.020+0000 7fa787cce700 1 -- 172.21.15.190:0/3876013552 wait complete. 2023-10-16T09:43:46.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 792bf2d5-f768-42cc-af93-fe4b434c1cbf 6 2023-10-16T09:43:46.189 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.187+0000 7f6f4ce18700 1 Processor -- start 2023-10-16T09:43:46.189 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.187+0000 7f6f4ce18700 1 -- start start 2023-10-16T09:43:46.192 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f4ce18700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f4815dbd0 0x7f6f480d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.192 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f4ce18700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f480d0610 0x7f6f480cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.192 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f4ce18700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4815dfb0 con 0x7f6f480ce2c0 2023-10-16T09:43:46.192 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f4ce18700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f480ce770 con 0x7f6f480d0610 2023-10-16T09:43:46.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f4ce18700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f480cd3a0 con 0x7f6f4815dbd0 2023-10-16T09:43:46.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f4659c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f4815dbd0 0x7f6f480d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f46d9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f6f480ce2c0 0x7f6f48161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:60726/0 (socket says 172.21.15.190:60726) 2023-10-16T09:43:46.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.188+0000 7f6f46d9d700 1 -- 172.21.15.190:0/230686169 learned_addr learned my addr 172.21.15.190:0/230686169 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-10-16T09:43:46.193 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f45d9b700 1 --2- 172.21.15.190:0/230686169 >> v2:172.21.15.181:3300/0 conn(0x7f6f480d0610 0x7f6f480cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4559a700 1 -- 172.21.15.190:0/230686169 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102768264 0 0) 0x7f6f40001110 con 0x7f6f480ce2c0 2023-10-16T09:43:46.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4559a700 1 -- 172.21.15.190:0/230686169 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f300035b0 con 0x7f6f480ce2c0 2023-10-16T09:43:46.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4659c700 1 -- 172.21.15.190:0/230686169 >> v2:172.21.15.181:3300/0 conn(0x7f6f480d0610 msgr2=0x7f6f480cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:46.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4659c700 1 --2- 172.21.15.190:0/230686169 >> v2:172.21.15.181:3300/0 conn(0x7f6f480d0610 0x7f6f480cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.194 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4659c700 1 -- 172.21.15.190:0/230686169 >> v1:172.21.15.190:6789/0 conn(0x7f6f480ce2c0 legacy=0x7f6f48161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4659c700 1 -- 172.21.15.190:0/230686169 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480cd830 con 0x7f6f4815dbd0 2023-10-16T09:43:46.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4659c700 1 --2- 172.21.15.190:0/230686169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4815dbd0 0x7f6f480d00d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f480729b0 tx=0x7f6f3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a171797dbc0fcf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:46.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4559a700 1 -- 172.21.15.190:0/230686169 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800e070 con 0x7f6f4815dbd0 2023-10-16T09:43:46.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4559a700 1 -- 172.21.15.190:0/230686169 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3800bb20 con 0x7f6f4815dbd0 2023-10-16T09:43:46.195 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.189+0000 7f6f4559a700 1 -- 172.21.15.190:0/230686169 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38012430 con 0x7f6f4815dbd0 2023-10-16T09:43:46.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 -- 172.21.15.190:0/230686169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4815dbd0 msgr2=0x7f6f480d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 --2- 172.21.15.190:0/230686169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4815dbd0 0x7f6f480d00d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f480729b0 tx=0x7f6f3800b330 comp rx=0 tx=0).stop 2023-10-16T09:43:46.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 -- 172.21.15.190:0/230686169 shutdown_connections 2023-10-16T09:43:46.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 --2- 172.21.15.190:0/230686169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f4815dbd0 0x7f6f480d00d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 --2- 172.21.15.190:0/230686169 >> v2:172.21.15.181:3300/0 conn(0x7f6f480d0610 0x7f6f480cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.196 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 -- 172.21.15.190:0/230686169 >> 172.21.15.190:0/230686169 conn(0x7f6f4809a930 msgr2=0x7f6f480d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:46.197 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 -- 172.21.15.190:0/230686169 shutdown_connections 2023-10-16T09:43:46.197 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.190+0000 7f6f4ce18700 1 -- 172.21.15.190:0/230686169 wait complete. 2023-10-16T09:43:46.199 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.191+0000 7f6f4ce18700 1 Processor -- start 2023-10-16T09:43:46.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.191+0000 7f6f4ce18700 1 -- start start 2023-10-16T09:43:46.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4ce18700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f480ce2c0 0x7f6f480c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4ce18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f480d0610 0x7f6f480b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4ce18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4815dbd0 0x7f6f48145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4ce18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c9420 con 0x7f6f480d0610 2023-10-16T09:43:46.200 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4ce18700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b7120 con 0x7f6f480ce2c0 2023-10-16T09:43:46.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4ce18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f481564c0 con 0x7f6f4815dbd0 2023-10-16T09:43:46.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f480ce2c0 0x7f6f480c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f46d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4815dbd0 0x7f6f48145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f45d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f480d0610 0x7f6f480b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f480ce2c0 0x7f6f480c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34156/0 (socket says 172.21.15.190:34156) 2023-10-16T09:43:46.201 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 -- 172.21.15.190:0/2786526614 learned_addr learned my addr 172.21.15.190:0/2786526614 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:46.202 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 -- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4815dbd0 msgr2=0x7f6f48145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.202 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4815dbd0 0x7f6f48145870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.202 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 -- 172.21.15.190:0/2786526614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f480d0610 msgr2=0x7f6f480b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.202 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f480d0610 0x7f6f480b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.202 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f4659c700 1 -- 172.21.15.190:0/2786526614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38009a70 con 0x7f6f480ce2c0 2023-10-16T09:43:46.203 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.192+0000 7f6f46d9d700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4815dbd0 0x7f6f48145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:46.203 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f4659c700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f480ce2c0 0x7f6f480c8d10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6f38014040 tx=0x7f6f38004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:46.203 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f36ffd700 1 -- 172.21.15.190:0/2786526614 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38012430 con 0x7f6f480ce2c0 2023-10-16T09:43:46.203 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f36ffd700 1 -- 172.21.15.190:0/2786526614 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38020070 con 0x7f6f480ce2c0 2023-10-16T09:43:46.203 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f36ffd700 1 -- 172.21.15.190:0/2786526614 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38004280 con 0x7f6f480ce2c0 2023-10-16T09:43:46.203 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f481567a0 con 0x7f6f480ce2c0 2023-10-16T09:43:46.204 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f48156d60 con 0x7f6f480ce2c0 2023-10-16T09:43:46.204 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f36ffd700 1 -- 172.21.15.190:0/2786526614 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6f380046e0 con 0x7f6f480ce2c0 2023-10-16T09:43:46.204 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.193+0000 7f6f36ffd700 1 -- 172.21.15.190:0/2786526614 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f6f38022070 con 0x7f6f480ce2c0 2023-10-16T09:43:46.204 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.194+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f48067d20 con 0x7f6f480ce2c0 2023-10-16T09:43:46.204 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.196+0000 7f6f36ffd700 1 -- 172.21.15.190:0/2786526614 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f6f3802c060 con 0x7f6f480ce2c0 2023-10-16T09:43:46.282 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.280+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "792bf2d5-f768-42cc-af93-fe4b434c1cbf", "id": 6} v 0) v1 -- 0x7f6f480cebb0 con 0x7f6f480ce2c0 2023-10-16T09:43:46.290 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.288+0000 7f6f36ffd700 1 -- 172.21.15.190:0/2786526614 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "792bf2d5-f768-42cc-af93-fe4b434c1cbf", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6f38012e40 con 0x7f6f480ce2c0 2023-10-16T09:43:46.290 INFO:teuthology.orchestra.run.smithi190.stdout:6 2023-10-16T09:43:46.291 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f480ce2c0 msgr2=0x7f6f480c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f480ce2c0 0x7f6f480c8d10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6f38014040 tx=0x7f6f38004ad0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 shutdown_connections 2023-10-16T09:43:46.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f4815dbd0 0x7f6f48145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f480ce2c0 0x7f6f480c8d10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.292 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 --2- 172.21.15.190:0/2786526614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f480d0610 0x7f6f480b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.293 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 >> 172.21.15.190:0/2786526614 conn(0x7f6f4809a930 msgr2=0x7f6f4806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:46.293 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 shutdown_connections 2023-10-16T09:43:46.293 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.290+0000 7f6f4ce18700 1 -- 172.21.15.190:0/2786526614 wait complete. 2023-10-16T09:43:46.303 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 13d6e7b6-bb6a-4843-9cba-8fb3454fccd3 7 2023-10-16T09:43:46.333 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:46.331+0000 7f7072d4a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:43:46.383 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:46.382+0000 7f9e9cb4e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:43:46.401 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:46.399+0000 7f7072d4a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:43:46.427 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.426+0000 7fab02fc8700 1 Processor -- start 2023-10-16T09:43:46.428 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.426+0000 7fab02fc8700 1 -- start start 2023-10-16T09:43:46.430 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab02fc8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faafc0cdd80 0x7faafc0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.430 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab02fc8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc06bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.431 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab02fc8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faafc15dfb0 con 0x7faafc15dbd0 2023-10-16T09:43:46.431 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab02fc8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faafc0ce740 con 0x7faafc0cdd80 2023-10-16T09:43:46.431 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab02fc8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0cd3a0 con 0x7faafc06b6f0 2023-10-16T09:43:46.431 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab01565700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc06bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.431 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faafc0cdd80 0x7faafc0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.432 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab00d64700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7faafc15dbd0 0x7faafc0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:60734/0 (socket says 172.21.15.190:60734) 2023-10-16T09:43:46.432 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafbfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faafc0cdd80 0x7faafc0ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34164/0 (socket says 172.21.15.190:34164) 2023-10-16T09:43:46.432 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab01565700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc06bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48230/0 (socket says 172.21.15.190:48230) 2023-10-16T09:43:46.432 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7fab00d64700 1 -- 172.21.15.190:0/1244240660 learned_addr learned my addr 172.21.15.190:0/1244240660 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-10-16T09:43:46.432 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafbfff700 1 -- 172.21.15.190:0/1244240660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 msgr2=0x7faafc06bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:46.433 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafbfff700 1 --2- 172.21.15.190:0/1244240660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc06bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.433 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafbfff700 1 -- 172.21.15.190:0/1244240660 >> v1:172.21.15.190:6789/0 conn(0x7faafc15dbd0 legacy=0x7faafc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.433 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafb7fe700 1 -- 172.21.15.190:0/1244240660 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379027772 0 0) 0x7faaf0001110 con 0x7faafc15dbd0 2023-10-16T09:43:46.433 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafbfff700 1 -- 172.21.15.190:0/1244240660 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0cd830 con 0x7faafc0cdd80 2023-10-16T09:43:46.434 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafbfff700 1 --2- 172.21.15.190:0/1244240660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc0cdd80 0x7faafc0ce200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faae8009840 tx=0x7faae8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a05271d6889126 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:46.434 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.427+0000 7faafb7fe700 1 -- 172.21.15.190:0/1244240660 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8011830 con 0x7faafc0cdd80 2023-10-16T09:43:46.434 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7faafb7fe700 1 -- 172.21.15.190:0/1244240660 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae8011990 con 0x7faafc0cdd80 2023-10-16T09:43:46.434 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7faafb7fe700 1 -- 172.21.15.190:0/1244240660 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8011d20 con 0x7faafc0cdd80 2023-10-16T09:43:46.435 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 -- 172.21.15.190:0/1244240660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc0cdd80 msgr2=0x7faafc0ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.435 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 --2- 172.21.15.190:0/1244240660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc0cdd80 0x7faafc0ce200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faae8009840 tx=0x7faae8009b50 comp rx=0 tx=0).stop 2023-10-16T09:43:46.435 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 -- 172.21.15.190:0/1244240660 shutdown_connections 2023-10-16T09:43:46.435 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 --2- 172.21.15.190:0/1244240660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc06bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.436 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 --2- 172.21.15.190:0/1244240660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc0cdd80 0x7faafc0ce200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.436 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 -- 172.21.15.190:0/1244240660 >> 172.21.15.190:0/1244240660 conn(0x7faafc09a930 msgr2=0x7faafc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:46.436 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 -- 172.21.15.190:0/1244240660 shutdown_connections 2023-10-16T09:43:46.436 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.428+0000 7fab02fc8700 1 -- 172.21.15.190:0/1244240660 wait complete. 2023-10-16T09:43:46.439 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.429+0000 7fab02fc8700 1 Processor -- start 2023-10-16T09:43:46.439 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.429+0000 7fab02fc8700 1 -- start start 2023-10-16T09:43:46.439 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab02fc8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc15c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.439 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab02fc8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faafc0cdd80 0x7faafc1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.440 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab02fc8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc15dbd0 0x7faafc0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.440 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab02fc8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc1f35f0 con 0x7faafc0cdd80 2023-10-16T09:43:46.440 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab02fc8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc15cef0 con 0x7faafc15dbd0 2023-10-16T09:43:46.440 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab02fc8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc09ed50 con 0x7faafc06b6f0 2023-10-16T09:43:46.441 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab00d64700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc15c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.441 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab00d64700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc15c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:48232/0 (socket says 172.21.15.190:48232) 2023-10-16T09:43:46.441 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab00d64700 1 -- 172.21.15.190:0/496943972 learned_addr learned my addr 172.21.15.190:0/496943972 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-10-16T09:43:46.441 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.430+0000 7fab01565700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc15dbd0 0x7faafc0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.441 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7faafbfff700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faafc0cdd80 0x7faafc1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.442 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab00d64700 1 -- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc15dbd0 msgr2=0x7faafc0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.442 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab00d64700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc15dbd0 0x7faafc0c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.442 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab00d64700 1 -- 172.21.15.190:0/496943972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faafc0cdd80 msgr2=0x7faafc1f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.442 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab00d64700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faafc0cdd80 0x7faafc1f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.442 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab00d64700 1 -- 172.21.15.190:0/496943972 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae80094a0 con 0x7faafc06b6f0 2023-10-16T09:43:46.443 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab00d64700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc15c9b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c550 tx=0x7faaf000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:46.443 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7faaf97fa700 1 -- 172.21.15.190:0/496943972 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0004c80 con 0x7faafc06b6f0 2023-10-16T09:43:46.443 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7faaf97fa700 1 -- 172.21.15.190:0/496943972 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf0004e00 con 0x7faafc06b6f0 2023-10-16T09:43:46.443 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc1f32c0 con 0x7faafc06b6f0 2023-10-16T09:43:46.443 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7faaf97fa700 1 -- 172.21.15.190:0/496943972 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0007310 con 0x7faafc06b6f0 2023-10-16T09:43:46.444 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc09f2d0 con 0x7faafc06b6f0 2023-10-16T09:43:46.444 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7faaf97fa700 1 -- 172.21.15.190:0/496943972 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faaf001d3f0 con 0x7faafc06b6f0 2023-10-16T09:43:46.444 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.431+0000 7faaf97fa700 1 -- 172.21.15.190:0/496943972 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7faaf0013a20 con 0x7faafc06b6f0 2023-10-16T09:43:46.444 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.432+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc0cb890 con 0x7faafc06b6f0 2023-10-16T09:43:46.444 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.434+0000 7faaf97fa700 1 -- 172.21.15.190:0/496943972 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7faafc15dfb0 con 0x7faafc06b6f0 2023-10-16T09:43:46.456 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:46.454+0000 7f9e9cb4e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:43:46.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.518+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "13d6e7b6-bb6a-4843-9cba-8fb3454fccd3", "id": 7} v 0) v1 -- 0x7faafc067d20 con 0x7faafc06b6f0 2023-10-16T09:43:46.527 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.525+0000 7faaf97fa700 1 -- 172.21.15.190:0/496943972 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13d6e7b6-bb6a-4843-9cba-8fb3454fccd3", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7faaf0013d50 con 0x7faafc06b6f0 2023-10-16T09:43:46.527 INFO:teuthology.orchestra.run.smithi190.stdout:7 2023-10-16T09:43:46.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 msgr2=0x7faafc15c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc15c9b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c550 tx=0x7faaf000c860 comp rx=0 tx=0).stop 2023-10-16T09:43:46.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 shutdown_connections 2023-10-16T09:43:46.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faafc06b6f0 0x7faafc15c9b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc15dbd0 0x7faafc0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 --2- 172.21.15.190:0/496943972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faafc0cdd80 0x7faafc1f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 >> 172.21.15.190:0/496943972 conn(0x7faafc09a930 msgr2=0x7faafc0cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:46.530 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 shutdown_connections 2023-10-16T09:43:46.530 INFO:teuthology.orchestra.run.smithi190.stderr:2023-10-16T09:43:46.527+0000 7fab02fc8700 1 -- 172.21.15.190:0/496943972 wait complete. 2023-10-16T09:43:46.540 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-16T09:43:46.540 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-16T09:43:46.542 INFO:tasks.ceph.osd.0:Started 2023-10-16T09:43:46.543 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-16T09:43:46.543 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-16T09:43:46.545 INFO:tasks.ceph.osd.1:Started 2023-10-16T09:43:46.545 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-16T09:43:46.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-16T09:43:46.548 INFO:tasks.ceph.osd.2:Started 2023-10-16T09:43:46.548 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-16T09:43:46.548 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-16T09:43:46.551 INFO:tasks.ceph.osd.3:Started 2023-10-16T09:43:46.551 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-16T09:43:46.551 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-16T09:43:46.554 INFO:tasks.ceph.osd.4:Started 2023-10-16T09:43:46.554 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-16T09:43:46.554 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-16T09:43:46.556 INFO:tasks.ceph.osd.5:Started 2023-10-16T09:43:46.557 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-16T09:43:46.557 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-16T09:43:46.559 INFO:tasks.ceph.osd.6:Started 2023-10-16T09:43:46.559 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-16T09:43:46.560 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-16T09:43:46.562 INFO:tasks.ceph.osd.7:Started 2023-10-16T09:43:46.562 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:43:46.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.714+0000 7f6a037ec700 1 Processor -- start 2023-10-16T09:43:46.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.714+0000 7f6a037ec700 1 -- start start 2023-10-16T09:43:46.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a037ec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc0b22a0 0x7f69fc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a037ec700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69fc06e010 0x7f69fc06e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a037ec700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc066c90 con 0x7f69fc15b8b0 2023-10-16T09:43:46.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a037ec700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc06e9c0 con 0x7f69fc06e010 2023-10-16T09:43:46.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a037ec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc066740 con 0x7f69fc0b22a0 2023-10-16T09:43:46.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a01d89700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69fc06e010 0x7f69fc06e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a01d89700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69fc06e010 0x7f69fc06e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35456/0 (socket says 172.21.15.181:35456) 2023-10-16T09:43:46.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a01d89700 1 -- 172.21.15.181:0/4211078993 learned_addr learned my addr 172.21.15.181:0/4211078993 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:46.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.715+0000 7f6a00d87700 1 --2- 172.21.15.181:0/4211078993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc0b22a0 0x7f69fc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f6a01d89700 1 -- 172.21.15.181:0/4211078993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc0b22a0 msgr2=0x7f69fc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:46.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f6a01d89700 1 --2- 172.21.15.181:0/4211078993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc0b22a0 0x7f69fc0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f6a01d89700 1 -- 172.21.15.181:0/4211078993 >> v1:172.21.15.190:6789/0 conn(0x7f69fc15b8b0 legacy=0x7f69fc15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f69f3fff700 1 -- 172.21.15.181:0/4211078993 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472856115 0 0) 0x7f69f8001110 con 0x7f69fc15b8b0 2023-10-16T09:43:46.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f6a01d89700 1 -- 172.21.15.181:0/4211078993 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0669b0 con 0x7f69fc06e010 2023-10-16T09:43:46.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f6a01d89700 1 --2- 172.21.15.181:0/4211078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc06e010 0x7f69fc06e480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009e50 tx=0x7f69f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b7a58c16cd276e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f69f3fff700 1 -- 172.21.15.181:0/4211078993 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400e070 con 0x7f69fc06e010 2023-10-16T09:43:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f69f3fff700 1 -- 172.21.15.181:0/4211078993 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f400b910 con 0x7f69fc06e010 2023-10-16T09:43:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.716+0000 7f69f3fff700 1 -- 172.21.15.181:0/4211078993 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400bcc0 con 0x7f69fc06e010 2023-10-16T09:43:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 -- 172.21.15.181:0/4211078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc06e010 msgr2=0x7f69fc06e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 --2- 172.21.15.181:0/4211078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc06e010 0x7f69fc06e480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009e50 tx=0x7f69f400b330 comp rx=0 tx=0).stop 2023-10-16T09:43:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 -- 172.21.15.181:0/4211078993 shutdown_connections 2023-10-16T09:43:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 --2- 172.21.15.181:0/4211078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc06e010 0x7f69fc06e480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 --2- 172.21.15.181:0/4211078993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc0b22a0 0x7f69fc0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 -- 172.21.15.181:0/4211078993 >> 172.21.15.181:0/4211078993 conn(0x7f69fc0b1780 msgr2=0x7f69fc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:46.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 -- 172.21.15.181:0/4211078993 shutdown_connections 2023-10-16T09:43:46.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.717+0000 7f6a037ec700 1 -- 172.21.15.181:0/4211078993 wait complete. 2023-10-16T09:43:46.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.718+0000 7f6a037ec700 1 Processor -- start 2023-10-16T09:43:46.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.718+0000 7f6a037ec700 1 -- start start 2023-10-16T09:43:46.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a037ec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc06e010 0x7f69fc09fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a037ec700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69fc0b22a0 0x7f69fc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a037ec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc15b8b0 0x7f69fc159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:46.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a037ec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0a0360 con 0x7f69fc15b8b0 2023-10-16T09:43:46.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a037ec700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0cdda0 con 0x7f69fc0b22a0 2023-10-16T09:43:46.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a037ec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc0ce920 con 0x7f69fc06e010 2023-10-16T09:43:46.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc15b8b0 0x7f69fc159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc15b8b0 0x7f69fc159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35462/0 (socket says 172.21.15.181:35462) 2023-10-16T09:43:46.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 -- 172.21.15.181:0/2110638266 learned_addr learned my addr 172.21.15.181:0/2110638266 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:46.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01588700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc06e010 0x7f69fc09fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a00d87700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69fc0b22a0 0x7f69fc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:46.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 -- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc06e010 msgr2=0x7f69fc09fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc06e010 0x7f69fc09fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 -- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69fc0b22a0 msgr2=0x7f69fc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69fc0b22a0 0x7f69fc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 -- 172.21.15.181:0/2110638266 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4009b30 con 0x7f69fc15b8b0 2023-10-16T09:43:46.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a01d89700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc15b8b0 0x7f69fc159d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69f4004270 tx=0x7f69f4004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:46.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f69f1ffb700 1 -- 172.21.15.181:0/2110638266 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4004860 con 0x7f69fc15b8b0 2023-10-16T09:43:46.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f69f1ffb700 1 -- 172.21.15.181:0/2110638266 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f40099b0 con 0x7f69fc15b8b0 2023-10-16T09:43:46.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f69f1ffb700 1 -- 172.21.15.181:0/2110638266 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f401c680 con 0x7f69fc15b8b0 2023-10-16T09:43:46.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.719+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc0cdfd0 con 0x7f69fc15b8b0 2023-10-16T09:43:46.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.720+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc0ce590 con 0x7f69fc15b8b0 2023-10-16T09:43:46.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.720+0000 7f69f1ffb700 1 -- 172.21.15.181:0/2110638266 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f69f401cae0 con 0x7f69fc15b8b0 2023-10-16T09:43:46.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.720+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc067d20 con 0x7f69fc15b8b0 2023-10-16T09:43:46.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.720+0000 7f69f1ffb700 1 -- 172.21.15.181:0/2110638266 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f69f40123f0 con 0x7f69fc15b8b0 2023-10-16T09:43:46.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.722+0000 7f69f1ffb700 1 -- 172.21.15.181:0/2110638266 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f69f4020070 con 0x7f69fc15b8b0 2023-10-16T09:43:46.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.792+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69fc0723c0 con 0x7f69fc15b8b0 2023-10-16T09:43:46.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.792+0000 7f69f1ffb700 1 -- 172.21.15.181:0/2110638266 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f69f4033020 con 0x7f69fc15b8b0 2023-10-16T09:43:46.795 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:46.795 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":10,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:43:46.520078+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:43:46.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc15b8b0 msgr2=0x7f69fc159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:46.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc15b8b0 0x7f69fc159d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69f4004270 tx=0x7f69f4004b20 comp rx=0 tx=0).stop 2023-10-16T09:43:46.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 shutdown_connections 2023-10-16T09:43:46.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69fc06e010 0x7f69fc09fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc15b8b0 0x7f69fc159d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 --2- 172.21.15.181:0/2110638266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69fc0b22a0 0x7f69fc0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:46.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 >> 172.21.15.181:0/2110638266 conn(0x7f69fc0b1780 msgr2=0x7f69fc0c5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:46.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 shutdown_connections 2023-10-16T09:43:46.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:46.793+0000 7f6a037ec700 1 -- 172.21.15.181:0/2110638266 wait complete. 2023-10-16T09:43:46.807 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-16T09:43:46.807 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-16T09:43:47.134 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:47.132+0000 7f7072d4a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:43:47.164 INFO:tasks.ceph.osd.2.smithi181.stderr:2023-10-16T09:43:47.162+0000 7f932d0e5640 -1 Falling back to public interface 2023-10-16T09:43:47.165 INFO:tasks.ceph.osd.1.smithi181.stderr:2023-10-16T09:43:47.163+0000 7f5a168fe640 -1 Falling back to public interface 2023-10-16T09:43:47.172 INFO:tasks.ceph.osd.0.smithi181.stderr:2023-10-16T09:43:47.170+0000 7f1725e1d640 -1 Falling back to public interface 2023-10-16T09:43:47.173 INFO:tasks.ceph.osd.3.smithi181.stderr:2023-10-16T09:43:47.170+0000 7f0973520640 -1 Falling back to public interface 2023-10-16T09:43:47.173 INFO:tasks.ceph.osd.3.smithi181.stderr:2023-10-16T09:43:47.171+0000 7f096a8de700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2023-10-16T09:43:47.173 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:47.171+0000 7f9e9cb4e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:43:47.174 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-10-16T09:43:47.171+0000 7f34fb159640 -1 Falling back to public interface 2023-10-16T09:43:47.178 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-10-16T09:43:47.176+0000 7ff10cf36640 -1 Falling back to public interface 2023-10-16T09:43:47.186 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-10-16T09:43:47.184+0000 7fde9630f640 -1 Falling back to public interface 2023-10-16T09:43:47.186 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-10-16T09:43:47.185+0000 7fa61070a640 -1 Falling back to public interface 2023-10-16T09:43:47.214 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:47.212+0000 7f7072d4a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:43:47.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:47.254+0000 7f9e9cb4e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:43:47.353 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:47.351+0000 7f7072d4a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:43:47.396 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:47.395+0000 7f9e9cb4e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:43:47.590 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:47.588+0000 7f7072d4a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:43:47.614 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T09:43:47.632 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:47.631+0000 7f9e9cb4e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:43:47.729 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.727+0000 7f3fd090b700 1 Processor -- start 2023-10-16T09:43:47.730 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.728+0000 7f3fd090b700 1 -- start start 2023-10-16T09:43:47.732 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.728+0000 7f3fd090b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fcc15a820 0x7f3fcc0cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:47.733 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.728+0000 7f3fd090b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc15e310 0x7f3fcc0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:47.733 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fd090b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc15ac00 con 0x7f3fcc0af6f0 2023-10-16T09:43:47.733 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fd090b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc071b90 con 0x7f3fcc15a820 2023-10-16T09:43:47.733 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fd090b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc071e70 con 0x7f3fcc15e310 2023-10-16T09:43:47.733 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fca59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fcc15a820 0x7f3fcc0cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:47.733 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fc9d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3fcc0af6f0 0x7f3fcc0ce000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46542/0 (socket says 172.21.15.181:46542) 2023-10-16T09:43:47.734 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fc9d9b700 1 -- 172.21.15.181:0/247333274 learned_addr learned my addr 172.21.15.181:0/247333274 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:43:47.734 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fcad9d700 1 --2- 172.21.15.181:0/247333274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc15e310 0x7f3fcc0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:47.734 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fca59c700 1 -- 172.21.15.181:0/247333274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc15e310 msgr2=0x7f3fcc0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:47.734 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fca59c700 1 --2- 172.21.15.181:0/247333274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc15e310 0x7f3fcc0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.734 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fca59c700 1 -- 172.21.15.181:0/247333274 >> v1:172.21.15.181:6789/0 conn(0x7f3fcc0af6f0 legacy=0x7f3fcc0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:47.735 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fc959a700 1 -- 172.21.15.181:0/247333274 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848358649 0 0) 0x7f3fcc15ac00 con 0x7f3fcc0af6f0 2023-10-16T09:43:47.735 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fca59c700 1 -- 172.21.15.181:0/247333274 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc072150 con 0x7f3fcc15a820 2023-10-16T09:43:47.735 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fca59c700 1 --2- 172.21.15.181:0/247333274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15a820 0x7f3fcc0cf220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009840 tx=0x7f3fbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43c2b7e63aa1aba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:47.735 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fc959a700 1 -- 172.21.15.181:0/247333274 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011830 con 0x7f3fcc15a820 2023-10-16T09:43:47.735 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fc959a700 1 -- 172.21.15.181:0/247333274 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc011990 con 0x7f3fcc15a820 2023-10-16T09:43:47.736 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.729+0000 7f3fc959a700 1 -- 172.21.15.181:0/247333274 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc011d20 con 0x7f3fcc15a820 2023-10-16T09:43:47.736 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 -- 172.21.15.181:0/247333274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15a820 msgr2=0x7f3fcc0cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:47.736 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 --2- 172.21.15.181:0/247333274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15a820 0x7f3fcc0cf220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009840 tx=0x7f3fbc009b50 comp rx=0 tx=0).stop 2023-10-16T09:43:47.736 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 -- 172.21.15.181:0/247333274 shutdown_connections 2023-10-16T09:43:47.736 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 --2- 172.21.15.181:0/247333274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc15e310 0x7f3fcc0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.736 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 --2- 172.21.15.181:0/247333274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15a820 0x7f3fcc0cf220 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.737 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 -- 172.21.15.181:0/247333274 >> 172.21.15.181:0/247333274 conn(0x7f3fcc06db00 msgr2=0x7f3fcc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:47.737 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 -- 172.21.15.181:0/247333274 shutdown_connections 2023-10-16T09:43:47.737 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.730+0000 7f3fd090b700 1 -- 172.21.15.181:0/247333274 wait complete. 2023-10-16T09:43:47.739 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 Processor -- start 2023-10-16T09:43:47.740 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 -- start start 2023-10-16T09:43:47.740 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc0af6f0 0x7f3fcc0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:47.740 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fcc15a820 0x7f3fcc1f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:47.740 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15e310 0x7f3fcc1f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:47.740 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0b59a0 con 0x7f3fcc15e310 2023-10-16T09:43:47.741 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0b5c80 con 0x7f3fcc15a820 2023-10-16T09:43:47.741 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fd090b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0c9e90 con 0x7f3fcc0af6f0 2023-10-16T09:43:47.741 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fcad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15e310 0x7f3fcc1f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:47.741 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fcad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15e310 0x7f3fcc1f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35472/0 (socket says 172.21.15.181:35472) 2023-10-16T09:43:47.741 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.732+0000 7f3fcad9d700 1 -- 172.21.15.181:0/1313933417 learned_addr learned my addr 172.21.15.181:0/1313933417 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:47.741 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fca59c700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc0af6f0 0x7f3fcc0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:47.742 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fc9d9b700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fcc15a820 0x7f3fcc1f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:47.742 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fcad9d700 1 -- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc0af6f0 msgr2=0x7f3fcc0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:47.742 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fcad9d700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc0af6f0 0x7f3fcc0ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.742 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fcad9d700 1 -- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fcc15a820 msgr2=0x7f3fcc1f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:47.743 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fcad9d700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fcc15a820 0x7f3fcc1f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.743 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fcad9d700 1 -- 172.21.15.181:0/1313933417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0094a0 con 0x7f3fcc15e310 2023-10-16T09:43:47.743 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fcad9d700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15e310 0x7f3fcc1f30d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400efc0 tx=0x7f3fc400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:47.743 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fbaffd700 1 -- 172.21.15.181:0/1313933417 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400cad0 con 0x7f3fcc15e310 2023-10-16T09:43:47.743 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fbaffd700 1 -- 172.21.15.181:0/1313933417 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc400f040 con 0x7f3fcc15e310 2023-10-16T09:43:47.743 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fbaffd700 1 -- 172.21.15.181:0/1313933417 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400ce70 con 0x7f3fcc15e310 2023-10-16T09:43:47.744 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc0ca170 con 0x7f3fcc15e310 2023-10-16T09:43:47.744 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc0cf040 con 0x7f3fcc15e310 2023-10-16T09:43:47.744 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fbaffd700 1 -- 172.21.15.181:0/1313933417 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3fc4014780 con 0x7f3fcc15e310 2023-10-16T09:43:47.744 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.733+0000 7f3fbaffd700 1 -- 172.21.15.181:0/1313933417 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3fc40133e0 con 0x7f3fcc15e310 2023-10-16T09:43:47.745 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.734+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc067d20 con 0x7f3fcc15e310 2023-10-16T09:43:47.745 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.737+0000 7f3fbaffd700 1 -- 172.21.15.181:0/1313933417 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3fc403bb40 con 0x7f3fcc15e310 2023-10-16T09:43:47.808 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.807+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fcc066ef0 con 0x7f3fcc15e310 2023-10-16T09:43:47.809 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.807+0000 7f3fbaffd700 1 -- 172.21.15.181:0/1313933417 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3fc402a070 con 0x7f3fcc15e310 2023-10-16T09:43:47.810 INFO:teuthology.misc.health.smithi181.stdout: 2023-10-16T09:43:47.810 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":10,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:43:46.520078+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:43:47.811 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15e310 msgr2=0x7f3fcc1f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:47.811 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15e310 0x7f3fcc1f30d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400efc0 tx=0x7f3fc400c0c0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.811 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 shutdown_connections 2023-10-16T09:43:47.812 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fcc0af6f0 0x7f3fcc0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.812 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc15e310 0x7f3fcc1f30d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.812 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 --2- 172.21.15.181:0/1313933417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fcc15a820 0x7f3fcc1f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:47.812 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 >> 172.21.15.181:0/1313933417 conn(0x7f3fcc06db00 msgr2=0x7f3fcc14fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:47.813 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 shutdown_connections 2023-10-16T09:43:47.813 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:47.808+0000 7f3fd090b700 1 -- 172.21.15.181:0/1313933417 wait complete. 2023-10-16T09:43:47.815 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:47.813+0000 7f7072d4a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:43:47.821 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-16T09:43:47.850 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:47.849+0000 7f9e9cb4e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:43:47.949 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:47.947+0000 7f7072d4a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:43:47.982 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:47.980+0000 7f9e9cb4e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:43:48.168 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:48.166+0000 7f7072d4a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:43:48.203 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:48.202+0000 7f9e9cb4e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:43:48.325 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:48.323+0000 7f7072d4a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:43:48.360 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:48.358+0000 7f9e9cb4e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:43:48.439 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:48.438+0000 7f7072d4a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:43:48.474 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:48.473+0000 7f9e9cb4e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:43:48.766 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:48.764+0000 7f7072d4a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:43:48.802 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:48.801+0000 7f9e9cb4e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:43:48.867 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:48.865+0000 7f7072d4a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:43:48.903 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:48.901+0000 7f9e9cb4e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:43:49.054 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:49.052+0000 7f7072d4a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:43:49.088 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:49.087+0000 7f9e9cb4e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:43:49.657 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:49.655+0000 7f7072d4a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:43:49.696 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:43:49.694+0000 7f9e9cb4e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:43:50.671 INFO:tasks.ceph.osd.1.smithi181.stderr:2023-10-16T09:43:50.669+0000 7f5a168fe640 -1 osd.1 0 log_to_monitors true 2023-10-16T09:43:50.687 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:50.685+0000 7f702ac75700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-16T09:43:50.687 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:43:50.685+0000 7f702ac75700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-16T09:43:50.691 INFO:tasks.ceph.osd.0.smithi181.stderr:2023-10-16T09:43:50.690+0000 7f1725e1d640 -1 osd.0 0 log_to_monitors true 2023-10-16T09:43:50.700 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-10-16T09:43:50.699+0000 7f34fb159640 -1 osd.6 0 log_to_monitors true 2023-10-16T09:43:50.701 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-10-16T09:43:50.699+0000 7ff10cf36640 -1 osd.5 0 log_to_monitors true 2023-10-16T09:43:50.709 INFO:tasks.ceph.osd.2.smithi181.stderr:2023-10-16T09:43:50.707+0000 7f932d0e5640 -1 osd.2 0 log_to_monitors true 2023-10-16T09:43:50.713 INFO:tasks.ceph.osd.3.smithi181.stderr:2023-10-16T09:43:50.711+0000 7f0973520640 -1 osd.3 0 log_to_monitors true 2023-10-16T09:43:50.741 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-10-16T09:43:50.739+0000 7fa61070a640 -1 osd.7 0 log_to_monitors true 2023-10-16T09:43:50.776 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-10-16T09:43:50.774+0000 7fde9630f640 -1 osd.4 0 log_to_monitors true 2023-10-16T09:43:53.520 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-10-16T09:43:53.519+0000 7f34eed10700 -1 osd.6 0 waiting for initial osdmap 2023-10-16T09:43:53.529 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-10-16T09:43:53.526+0000 7ff100aed700 -1 osd.5 0 waiting for initial osdmap 2023-10-16T09:43:53.533 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-10-16T09:43:53.532+0000 7f34e70eb700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:53.537 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-10-16T09:43:53.536+0000 7fa6042c1700 -1 osd.7 0 waiting for initial osdmap 2023-10-16T09:43:53.543 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-10-16T09:43:53.542+0000 7ff0f8ec8700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:53.550 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-10-16T09:43:53.549+0000 7fa5fc69c700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:53.759 INFO:tasks.ceph.osd.1.smithi181.stderr:2023-10-16T09:43:53.757+0000 7f5a0a4b5700 -1 osd.1 0 waiting for initial osdmap 2023-10-16T09:43:53.772 INFO:tasks.ceph.osd.1.smithi181.stderr:2023-10-16T09:43:53.771+0000 7f5a02890700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:53.778 INFO:tasks.ceph.osd.3.smithi181.stderr:2023-10-16T09:43:53.777+0000 7f09670d7700 -1 osd.3 0 waiting for initial osdmap 2023-10-16T09:43:53.791 INFO:tasks.ceph.osd.3.smithi181.stderr:2023-10-16T09:43:53.790+0000 7f095f4b2700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:53.812 INFO:tasks.ceph.osd.2.smithi181.stderr:2023-10-16T09:43:53.811+0000 7f9320c9c700 -1 osd.2 0 waiting for initial osdmap 2023-10-16T09:43:53.820 INFO:tasks.ceph.osd.0.smithi181.stderr:2023-10-16T09:43:53.818+0000 7f17199d4700 -1 osd.0 0 waiting for initial osdmap 2023-10-16T09:43:53.822 INFO:tasks.ceph.osd.2.smithi181.stderr:2023-10-16T09:43:53.821+0000 7f9319077700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:53.830 INFO:tasks.ceph.osd.0.smithi181.stderr:2023-10-16T09:43:53.829+0000 7f1711daf700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:53.883 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-10-16T09:43:53.882+0000 7fde89ec6700 -1 osd.4 0 waiting for initial osdmap 2023-10-16T09:43:53.895 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-10-16T09:43:53.894+0000 7fde822a1700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T09:43:54.627 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T09:43:54.751 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.749+0000 7fd5070cf700 1 Processor -- start 2023-10-16T09:43:54.751 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.749+0000 7fd5070cf700 1 -- start start 2023-10-16T09:43:54.753 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.749+0000 7fd5070cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000cc2c0 0x7fd5000b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:54.754 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.749+0000 7fd5070cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5000d0c20 0x7fd5000b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:54.754 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.749+0000 7fd5070cf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5000b43b0 con 0x7fd50015b8e0 2023-10-16T09:43:54.754 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd5070cf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50015bcc0 con 0x7fd5000d0c20 2023-10-16T09:43:54.754 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd5070cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000cfab0 con 0x7fd5000cc2c0 2023-10-16T09:43:54.754 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5000d0c20 0x7fd5000b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:54.755 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5000d0c20 0x7fd5000b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35690/0 (socket says 172.21.15.181:35690) 2023-10-16T09:43:54.755 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 -- 172.21.15.181:0/2571372806 learned_addr learned my addr 172.21.15.181:0/2571372806 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:54.755 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd4fffff700 1 --2- 172.21.15.181:0/2571372806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000cc2c0 0x7fd5000b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:54.755 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 -- 172.21.15.181:0/2571372806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000cc2c0 msgr2=0x7fd5000b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:54.755 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 --2- 172.21.15.181:0/2571372806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000cc2c0 0x7fd5000b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.756 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 -- 172.21.15.181:0/2571372806 >> v1:172.21.15.190:6789/0 conn(0x7fd50015b8e0 legacy=0x7fd5000cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:54.756 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 -- 172.21.15.181:0/2571372806 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000cfd90 con 0x7fd5000d0c20 2023-10-16T09:43:54.756 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd50566c700 1 --2- 172.21.15.181:0/2571372806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5000d0c20 0x7fd5000b4df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009e50 tx=0x7fd4f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad6ab111b98cb56 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:54.756 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd4ff7fe700 1 -- 172.21.15.181:0/2571372806 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800e070 con 0x7fd5000d0c20 2023-10-16T09:43:54.756 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd4ff7fe700 1 -- 172.21.15.181:0/2571372806 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f800b9c0 con 0x7fd5000d0c20 2023-10-16T09:43:54.757 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.750+0000 7fd4ff7fe700 1 -- 172.21.15.181:0/2571372806 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800bd70 con 0x7fd5000d0c20 2023-10-16T09:43:54.757 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 -- 172.21.15.181:0/2571372806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5000d0c20 msgr2=0x7fd5000b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:54.757 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 --2- 172.21.15.181:0/2571372806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5000d0c20 0x7fd5000b4df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009e50 tx=0x7fd4f800b330 comp rx=0 tx=0).stop 2023-10-16T09:43:54.757 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 -- 172.21.15.181:0/2571372806 shutdown_connections 2023-10-16T09:43:54.757 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 --2- 172.21.15.181:0/2571372806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5000d0c20 0x7fd5000b4df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.757 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 --2- 172.21.15.181:0/2571372806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000cc2c0 0x7fd5000b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.758 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 -- 172.21.15.181:0/2571372806 >> 172.21.15.181:0/2571372806 conn(0x7fd50006edb0 msgr2=0x7fd50006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:54.758 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 -- 172.21.15.181:0/2571372806 shutdown_connections 2023-10-16T09:43:54.758 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.751+0000 7fd5070cf700 1 -- 172.21.15.181:0/2571372806 wait complete. 2023-10-16T09:43:54.760 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.752+0000 7fd5070cf700 1 Processor -- start 2023-10-16T09:43:54.761 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.752+0000 7fd5070cf700 1 -- start start 2023-10-16T09:43:54.761 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd5070cf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5000cc2c0 0x7fd50006d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:54.761 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd5070cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000d0c20 0x7fd5001589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:54.761 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd5070cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50015b8e0 0x7fd5001e3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:54.762 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd5070cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50006d9d0 con 0x7fd50015b8e0 2023-10-16T09:43:54.762 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd5070cf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c5470 con 0x7fd5000cc2c0 2023-10-16T09:43:54.762 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd5070cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c5750 con 0x7fd5000d0c20 2023-10-16T09:43:54.762 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50015b8e0 0x7fd5001e3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:54.762 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50015b8e0 0x7fd5001e3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35700/0 (socket says 172.21.15.181:35700) 2023-10-16T09:43:54.763 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 -- 172.21.15.181:0/59151871 learned_addr learned my addr 172.21.15.181:0/59151871 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:54.763 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd4fffff700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000d0c20 0x7fd5001589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:54.763 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd504e6b700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5000cc2c0 0x7fd50006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:54.763 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 -- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000d0c20 msgr2=0x7fd5001589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:54.763 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000d0c20 0x7fd5001589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.765 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 -- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5000cc2c0 msgr2=0x7fd50006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:54.765 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5000cc2c0 0x7fd50006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.765 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 -- 172.21.15.181:0/59151871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8009b30 con 0x7fd50015b8e0 2023-10-16T09:43:54.766 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.753+0000 7fd50566c700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50015b8e0 0x7fd5001e3fa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5000cee90 tx=0x7fd4f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:54.766 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.754+0000 7fd4fd7fa700 1 -- 172.21.15.181:0/59151871 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8004800 con 0x7fd50015b8e0 2023-10-16T09:43:54.766 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.754+0000 7fd4fd7fa700 1 -- 172.21.15.181:0/59151871 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f8004960 con 0x7fd50015b8e0 2023-10-16T09:43:54.766 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.754+0000 7fd4fd7fa700 1 -- 172.21.15.181:0/59151871 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f801c680 con 0x7fd50015b8e0 2023-10-16T09:43:54.766 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.754+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5000c5a30 con 0x7fd50015b8e0 2023-10-16T09:43:54.767 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.754+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5001f6600 con 0x7fd50015b8e0 2023-10-16T09:43:54.767 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.755+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50015c980 con 0x7fd50015b8e0 2023-10-16T09:43:54.767 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.755+0000 7fd4fd7fa700 1 -- 172.21.15.181:0/59151871 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fd4f801c7e0 con 0x7fd50015b8e0 2023-10-16T09:43:54.767 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.755+0000 7fd4fd7fa700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd4e406ae70 0x7fd4e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:54.767 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.756+0000 7fd504e6b700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd4e406ae70 0x7fd4e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:54.768 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.756+0000 7fd4fd7fa700 1 -- 172.21.15.181:0/59151871 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd4f808ab10 con 0x7fd50015b8e0 2023-10-16T09:43:54.768 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.756+0000 7fd504e6b700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd4e406ae70 0x7fd4e406d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4007fd0 tx=0x7fd4f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:54.768 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.761+0000 7fd4fd7fa700 1 -- 172.21.15.181:0/59151871 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd4f8056200 con 0x7fd50015b8e0 2023-10-16T09:43:54.960 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.958+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd500067d20 con 0x7fd50015b8e0 2023-10-16T09:43:54.961 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.959+0000 7fd4fd7fa700 1 -- 172.21.15.181:0/59151871 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10944 (secure 0 0 0) 0x7fd4f8059820 con 0x7fd50015b8e0 2023-10-16T09:43:54.961 INFO:teuthology.misc.health.smithi181.stdout: 2023-10-16T09:43:54.961 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":14,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:43:54.540378+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:43:54.963 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd4e406ae70 msgr2=0x7fd4e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:54.963 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd4e406ae70 0x7fd4e406d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4007fd0 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2023-10-16T09:43:54.963 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50015b8e0 msgr2=0x7fd5001e3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:54.964 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50015b8e0 0x7fd5001e3fa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5000cee90 tx=0x7fd4f8004ad0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.964 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 shutdown_connections 2023-10-16T09:43:54.964 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd4e406ae70 0x7fd4e406d330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.964 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50015b8e0 0x7fd5001e3fa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.965 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5000cc2c0 0x7fd50006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.965 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 --2- 172.21.15.181:0/59151871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5000d0c20 0x7fd5001589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:54.965 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 >> 172.21.15.181:0/59151871 conn(0x7fd50006edb0 msgr2=0x7fd50006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:54.965 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 shutdown_connections 2023-10-16T09:43:54.965 INFO:teuthology.misc.health.smithi181.stderr:2023-10-16T09:43:54.962+0000 7fd5070cf700 1 -- 172.21.15.181:0/59151871 wait complete. 2023-10-16T09:43:54.974 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-16T09:43:54.975 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-16T09:43:54.975 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T09:43:54.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T09:43:54.978 INFO:tasks.ceph.mds.a:Started 2023-10-16T09:43:54.978 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T09:43:54.978 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T09:43:54.980 INFO:tasks.ceph.mds.c:Started 2023-10-16T09:43:54.981 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T09:43:54.981 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T09:43:54.984 INFO:tasks.ceph.mds.b:Started 2023-10-16T09:43:54.984 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T09:43:54.984 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T09:43:54.987 INFO:tasks.ceph.mds.d:Started 2023-10-16T09:43:54.987 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-16T09:43:54.987 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:43:55.057 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T09:43:55.058 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T09:43:55.071 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T09:43:55.072 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T09:43:55.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.116+0000 7f2047134700 1 Processor -- start 2023-10-16T09:43:55.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.116+0000 7f2047134700 1 -- start start 2023-10-16T09:43:55.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.116+0000 7f2047134700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f204015dfd0 0x7f20400af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2047134700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 0x7f204006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2047134700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204006f870 con 0x7f204006db80 2023-10-16T09:43:55.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2047134700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20400d1160 con 0x7f204015dfd0 2023-10-16T09:43:55.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2047134700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20400cc8e0 con 0x7f20400af450 2023-10-16T09:43:55.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f204015dfd0 0x7f20400af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f204015dfd0 0x7f20400af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35746/0 (socket says 172.21.15.181:35746) 2023-10-16T09:43:55.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 -- 172.21.15.181:0/2988744154 learned_addr learned my addr 172.21.15.181:0/2988744154 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:55.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f203ffff700 1 --2- 172.21.15.181:0/2988744154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 0x7f204006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 -- 172.21.15.181:0/2988744154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 msgr2=0x7f204006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:55.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 --2- 172.21.15.181:0/2988744154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 0x7f204006d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 -- 172.21.15.181:0/2988744154 >> v1:172.21.15.190:6789/0 conn(0x7f204006db80 legacy=0x7f20400cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 -- 172.21.15.181:0/2988744154 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400ccbc0 con 0x7f204015dfd0 2023-10-16T09:43:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f2044ed0700 1 --2- 172.21.15.181:0/2988744154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204015dfd0 0x7f20400af9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2034006fd0 tx=0x7f2034009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59177d54c62602f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f203f7fe700 1 -- 172.21.15.181:0/2988744154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034005e30 con 0x7f204015dfd0 2023-10-16T09:43:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f203f7fe700 1 -- 172.21.15.181:0/2988744154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2034006030 con 0x7f204015dfd0 2023-10-16T09:43:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.117+0000 7f203f7fe700 1 -- 172.21.15.181:0/2988744154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20340063c0 con 0x7f204015dfd0 2023-10-16T09:43:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 -- 172.21.15.181:0/2988744154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204015dfd0 msgr2=0x7f20400af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 --2- 172.21.15.181:0/2988744154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204015dfd0 0x7f20400af9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2034006fd0 tx=0x7f2034009bd0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 -- 172.21.15.181:0/2988744154 shutdown_connections 2023-10-16T09:43:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 --2- 172.21.15.181:0/2988744154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204015dfd0 0x7f20400af9a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 --2- 172.21.15.181:0/2988744154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 0x7f204006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 -- 172.21.15.181:0/2988744154 >> 172.21.15.181:0/2988744154 conn(0x7f20400d04a0 msgr2=0x7f20400d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:55.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 -- 172.21.15.181:0/2988744154 shutdown_connections 2023-10-16T09:43:55.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.118+0000 7f2047134700 1 -- 172.21.15.181:0/2988744154 wait complete. 2023-10-16T09:43:55.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.119+0000 7f2047134700 1 Processor -- start 2023-10-16T09:43:55.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.119+0000 7f2047134700 1 -- start start 2023-10-16T09:43:55.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2047134700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204006db80 0x7f20400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2047134700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 0x7f204015e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2047134700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f204015dfd0 0x7f2040157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2047134700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400b0ed0 con 0x7f204006db80 2023-10-16T09:43:55.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2047134700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400b5fc0 con 0x7f204015dfd0 2023-10-16T09:43:55.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2047134700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204015e980 con 0x7f20400af450 2023-10-16T09:43:55.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204006db80 0x7f20400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204006db80 0x7f20400b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35754/0 (socket says 172.21.15.181:35754) 2023-10-16T09:43:55.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 -- 172.21.15.181:0/231095386 learned_addr learned my addr 172.21.15.181:0/231095386 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:55.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 -- 172.21.15.181:0/231095386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 msgr2=0x7f204015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 0x7f204015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 -- 172.21.15.181:0/231095386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f204015dfd0 msgr2=0x7f2040157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f204015dfd0 0x7f2040157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 -- 172.21.15.181:0/231095386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034009500 con 0x7f204006db80 2023-10-16T09:43:55.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2044ed0700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204006db80 0x7f20400b4dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2034012000 tx=0x7f2034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:55.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f203d7fa700 1 -- 172.21.15.181:0/231095386 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034006930 con 0x7f204006db80 2023-10-16T09:43:55.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f203d7fa700 1 -- 172.21.15.181:0/231095386 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2034020070 con 0x7f204006db80 2023-10-16T09:43:55.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2040157aa0 con 0x7f204006db80 2023-10-16T09:43:55.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.120+0000 7f203d7fa700 1 -- 172.21.15.181:0/231095386 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034025650 con 0x7f204006db80 2023-10-16T09:43:55.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.121+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204015ee90 con 0x7f204006db80 2023-10-16T09:43:55.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.121+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2040067d20 con 0x7f204006db80 2023-10-16T09:43:55.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.122+0000 7f203d7fa700 1 -- 172.21.15.181:0/231095386 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f2034022030 con 0x7f204006db80 2023-10-16T09:43:55.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.122+0000 7f203d7fa700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f202406ae70 0x7f202406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.122+0000 7f203d7fa700 1 -- 172.21.15.181:0/231095386 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f203408a6c0 con 0x7f204006db80 2023-10-16T09:43:55.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.122+0000 7f203ffff700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f202406ae70 0x7f202406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.123+0000 7f203ffff700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f202406ae70 0x7f202406d330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20400cb130 tx=0x7f202c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:55.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.127+0000 7f203d7fa700 1 -- 172.21.15.181:0/231095386 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2034055d10 con 0x7f204006db80 2023-10-16T09:43:55.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.321+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2040067a70 con 0x7f204006db80 2023-10-16T09:43:55.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.322+0000 7f203d7fa700 1 -- 172.21.15.181:0/231095386 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10944 (secure 0 0 0) 0x7f2034059330 con 0x7f204006db80 2023-10-16T09:43:55.324 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:55.325 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":14,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:43:54.540378+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:43:55.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f202406ae70 msgr2=0x7f202406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f202406ae70 0x7f202406d330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20400cb130 tx=0x7f202c008040 comp rx=0 tx=0).stop 2023-10-16T09:43:55.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204006db80 msgr2=0x7f20400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204006db80 0x7f20400b4dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2034012000 tx=0x7f2034004750 comp rx=0 tx=0).stop 2023-10-16T09:43:55.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 shutdown_connections 2023-10-16T09:43:55.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f202406ae70 0x7f202406d330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f204015dfd0 0x7f2040157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204006db80 0x7f20400b4dc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 --2- 172.21.15.181:0/231095386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20400af450 0x7f204015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 >> 172.21.15.181:0/231095386 conn(0x7f20400d04a0 msgr2=0x7f20400ad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:55.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 shutdown_connections 2023-10-16T09:43:55.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.324+0000 7f2047134700 1 -- 172.21.15.181:0/231095386 wait complete. 2023-10-16T09:43:55.338 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-16T09:43:55.339 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:43:55.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.457+0000 7f31c9009700 1 Processor -- start 2023-10-16T09:43:55.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.457+0000 7f31c9009700 1 -- start start 2023-10-16T09:43:55.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c9009700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31c40cc0c0 0x7f31c415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c9009700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c40cabd0 con 0x7f31c40cdfb0 2023-10-16T09:43:55.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c9009700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c40ca5c0 con 0x7f31c406cdf0 2023-10-16T09:43:55.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c9009700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c4066740 con 0x7f31c40cc0c0 2023-10-16T09:43:55.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31c40cc0c0 0x7f31c415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31c40cc0c0 0x7f31c415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35762/0 (socket says 172.21.15.181:35762) 2023-10-16T09:43:55.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c259c700 1 -- 172.21.15.181:0/4183017229 learned_addr learned my addr 172.21.15.181:0/4183017229 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:55.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c259c700 1 -- 172.21.15.181:0/4183017229 >> v1:172.21.15.190:6789/0 conn(0x7f31c406cdf0 legacy=0x7f31c40cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:55.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c259c700 1 -- 172.21.15.181:0/4183017229 >> v1:172.21.15.181:6789/0 conn(0x7f31c40cdfb0 legacy=0x7f31c415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c1d9b700 1 -- 172.21.15.181:0/4183017229 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992298580 0 0) 0x7f31c40cabd0 con 0x7f31c40cdfb0 2023-10-16T09:43:55.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c259c700 1 -- 172.21.15.181:0/4183017229 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4066a20 con 0x7f31c40cc0c0 2023-10-16T09:43:55.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.458+0000 7f31c259c700 1 --2- 172.21.15.181:0/4183017229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c40cc0c0 0x7f31c415dfc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31b40098f0 tx=0x7f31b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14fce9fd7d10171 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:55.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c1d9b700 1 -- 172.21.15.181:0/4183017229 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4004230 con 0x7f31c40cc0c0 2023-10-16T09:43:55.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c1d9b700 1 -- 172.21.15.181:0/4183017229 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b4004390 con 0x7f31c40cc0c0 2023-10-16T09:43:55.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c1d9b700 1 -- 172.21.15.181:0/4183017229 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b40036a0 con 0x7f31c40cc0c0 2023-10-16T09:43:55.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c9009700 1 -- 172.21.15.181:0/4183017229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c40cc0c0 msgr2=0x7f31c415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c9009700 1 --2- 172.21.15.181:0/4183017229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c40cc0c0 0x7f31c415dfc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31b40098f0 tx=0x7f31b4009c00 comp rx=0 tx=0).stop 2023-10-16T09:43:55.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c9009700 1 -- 172.21.15.181:0/4183017229 shutdown_connections 2023-10-16T09:43:55.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c9009700 1 --2- 172.21.15.181:0/4183017229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c40cc0c0 0x7f31c415dfc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c9009700 1 -- 172.21.15.181:0/4183017229 >> 172.21.15.181:0/4183017229 conn(0x7f31c40b1780 msgr2=0x7f31c40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:55.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c9009700 1 -- 172.21.15.181:0/4183017229 shutdown_connections 2023-10-16T09:43:55.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.459+0000 7f31c9009700 1 -- 172.21.15.181:0/4183017229 wait complete. 2023-10-16T09:43:55.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 Processor -- start 2023-10-16T09:43:55.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 -- start start 2023-10-16T09:43:55.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c406cdf0 0x7f31c415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c40cc0c0 0x7f31c40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c40cdfb0 0x7f31c406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c40b36f0 con 0x7f31c406cdf0 2023-10-16T09:43:55.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c40c9b60 con 0x7f31c40cc0c0 2023-10-16T09:43:55.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c9009700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c40b4c50 con 0x7f31c40cdfb0 2023-10-16T09:43:55.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c406cdf0 0x7f31c415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.461+0000 7f31c2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c406cdf0 0x7f31c415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35766/0 (socket says 172.21.15.181:35766) 2023-10-16T09:43:55.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c2d9d700 1 -- 172.21.15.181:0/1466937278 learned_addr learned my addr 172.21.15.181:0/1466937278 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:55.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c259c700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c40cc0c0 0x7f31c40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c359e700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c40cdfb0 0x7f31c406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c2d9d700 1 -- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c40cdfb0 msgr2=0x7f31c406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c2d9d700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c40cdfb0 0x7f31c406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c2d9d700 1 -- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c40cc0c0 msgr2=0x7f31c40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c2d9d700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c40cc0c0 0x7f31c40c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c2d9d700 1 -- 172.21.15.181:0/1466937278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4009550 con 0x7f31c406cdf0 2023-10-16T09:43:55.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c359e700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c40cdfb0 0x7f31c406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:43:55.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c2d9d700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c406cdf0 0x7f31c415ad80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31b80011e0 tx=0x7f31b800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:55.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31ab7fe700 1 -- 172.21.15.181:0/1466937278 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b80054e0 con 0x7f31c406cdf0 2023-10-16T09:43:55.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31ab7fe700 1 -- 172.21.15.181:0/1466937278 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b8003680 con 0x7f31c406cdf0 2023-10-16T09:43:55.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c40b4f30 con 0x7f31c406cdf0 2023-10-16T09:43:55.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31ab7fe700 1 -- 172.21.15.181:0/1466937278 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8003a30 con 0x7f31c406cdf0 2023-10-16T09:43:55.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.462+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c40b54f0 con 0x7f31c406cdf0 2023-10-16T09:43:55.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.463+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c4067d20 con 0x7f31c406cdf0 2023-10-16T09:43:55.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.463+0000 7f31ab7fe700 1 -- 172.21.15.181:0/1466937278 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f31b8003b90 con 0x7f31c406cdf0 2023-10-16T09:43:55.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.464+0000 7f31ab7fe700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f31b006ae70 0x7f31b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:55.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.464+0000 7f31c259c700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f31b006ae70 0x7f31b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:55.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.464+0000 7f31ab7fe700 1 -- 172.21.15.181:0/1466937278 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f31b8088510 con 0x7f31c406cdf0 2023-10-16T09:43:55.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.464+0000 7f31c259c700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f31b006ae70 0x7f31b006d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009130 tx=0x7f31b4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:55.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.469+0000 7f31ab7fe700 1 -- 172.21.15.181:0/1466937278 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31b8053b70 con 0x7f31c406cdf0 2023-10-16T09:43:55.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.704+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31c4072510 con 0x7f31c406cdf0 2023-10-16T09:43:55.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.718+0000 7f31ab7fe700 1 -- 172.21.15.181:0/1466937278 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f31b8017020 con 0x7f31c406cdf0 2023-10-16T09:43:55.720 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:55.720 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-16T09:43:55.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f31b006ae70 msgr2=0x7f31b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f31b006ae70 0x7f31b006d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009130 tx=0x7f31b4008030 comp rx=0 tx=0).stop 2023-10-16T09:43:55.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c406cdf0 msgr2=0x7f31c415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:55.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c406cdf0 0x7f31c415ad80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31b80011e0 tx=0x7f31b800cae0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 shutdown_connections 2023-10-16T09:43:55.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f31b006ae70 0x7f31b006d330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c40cdfb0 0x7f31c406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c406cdf0 0x7f31c415ad80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 --2- 172.21.15.181:0/1466937278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c40cc0c0 0x7f31c40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:55.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 >> 172.21.15.181:0/1466937278 conn(0x7f31c40b1780 msgr2=0x7f31c415cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:55.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 shutdown_connections 2023-10-16T09:43:55.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:55.720+0000 7f31c9009700 1 -- 172.21.15.181:0/1466937278 wait complete. 2023-10-16T09:43:55.725 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 2 2023-10-16T09:43:57.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:43:57.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.864+0000 7f7e959d9700 1 Processor -- start 2023-10-16T09:43:57.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.864+0000 7f7e959d9700 1 -- start start 2023-10-16T09:43:57.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.864+0000 7f7e959d9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e9015a680 0x7f7e900cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:57.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.864+0000 7f7e959d9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900cd220 0x7f7e90164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:57.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.864+0000 7f7e959d9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900cb3b0 con 0x7f7e900d0c80 2023-10-16T09:43:57.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.864+0000 7f7e959d9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e900cbd50 con 0x7f7e9015a680 2023-10-16T09:43:57.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.864+0000 7f7e959d9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900cb690 con 0x7f7e900cd220 2023-10-16T09:43:57.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e900d0c80 0x7f7e900ccb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46920/0 (socket says 172.21.15.181:46920) 2023-10-16T09:43:57.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8e7fc700 1 -- 172.21.15.181:0/978400241 learned_addr learned my addr 172.21.15.181:0/978400241 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:43:57.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8effd700 1 --2- 172.21.15.181:0/978400241 >> v2:172.21.15.190:3300/0 conn(0x7f7e9015a680 0x7f7e900cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:57.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8f7fe700 1 --2- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900cd220 0x7f7e90164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:57.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8dffb700 1 -- 172.21.15.181:0/978400241 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229061320 0 0) 0x7f7e900cb3b0 con 0x7f7e900d0c80 2023-10-16T09:43:57.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8dffb700 1 -- 172.21.15.181:0/978400241 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e780035b0 con 0x7f7e900d0c80 2023-10-16T09:43:57.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8dffb700 1 -- 172.21.15.181:0/978400241 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 720475345 0 0) 0x7f7e780035b0 con 0x7f7e900d0c80 2023-10-16T09:43:57.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8effd700 1 -- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900cd220 msgr2=0x7f7e90164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:57.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8effd700 1 --2- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900cd220 0x7f7e90164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:57.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8effd700 1 -- 172.21.15.181:0/978400241 >> v1:172.21.15.181:6789/0 conn(0x7f7e900d0c80 legacy=0x7f7e900ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:57.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8effd700 1 -- 172.21.15.181:0/978400241 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900cb970 con 0x7f7e9015a680 2023-10-16T09:43:57.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8effd700 1 --2- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 0x7f7e900cecc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009850 tx=0x7f7e80009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2afc460184ea83a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:57.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8dffb700 1 -- 172.21.15.181:0/978400241 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84001670 con 0x7f7e9015a680 2023-10-16T09:43:57.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8dffb700 1 -- 172.21.15.181:0/978400241 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80005c30 con 0x7f7e9015a680 2023-10-16T09:43:57.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.865+0000 7f7e8dffb700 1 -- 172.21.15.181:0/978400241 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80004030 con 0x7f7e9015a680 2023-10-16T09:43:57.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 -- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 msgr2=0x7f7e900cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:57.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 --2- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 0x7f7e900cecc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009850 tx=0x7f7e80009b60 comp rx=0 tx=0).stop 2023-10-16T09:43:57.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 -- 172.21.15.181:0/978400241 shutdown_connections 2023-10-16T09:43:57.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 --2- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900cd220 0x7f7e90164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:57.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 --2- 172.21.15.181:0/978400241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 0x7f7e900cecc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:57.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 -- 172.21.15.181:0/978400241 >> 172.21.15.181:0/978400241 conn(0x7f7e90099a00 msgr2=0x7f7e900d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:57.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 -- 172.21.15.181:0/978400241 shutdown_connections 2023-10-16T09:43:57.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.866+0000 7f7e959d9700 1 -- 172.21.15.181:0/978400241 wait complete. 2023-10-16T09:43:57.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.867+0000 7f7e959d9700 1 Processor -- start 2023-10-16T09:43:57.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.867+0000 7f7e959d9700 1 -- start start 2023-10-16T09:43:57.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e959d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e900cd220 0x7f7e90155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:57.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e959d9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900d0c80 0x7f7e900c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:57.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e959d9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 0x7f7e900c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:57.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e959d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900c7980 con 0x7f7e900cd220 2023-10-16T09:43:57.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e959d9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9015b3c0 con 0x7f7e9015a680 2023-10-16T09:43:57.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e959d9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9015ab00 con 0x7f7e900d0c80 2023-10-16T09:43:57.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e900cd220 0x7f7e90155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:57.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e900cd220 0x7f7e90155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35846/0 (socket says 172.21.15.181:35846) 2023-10-16T09:43:57.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 -- 172.21.15.181:0/1247468614 learned_addr learned my addr 172.21.15.181:0/1247468614 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:57.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8f7fe700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 0x7f7e900c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:57.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8e7fc700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900d0c80 0x7f7e900c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:57.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 -- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900d0c80 msgr2=0x7f7e900c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:57.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900d0c80 0x7f7e900c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:57.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 -- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 msgr2=0x7f7e900c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:57.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 0x7f7e900c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:57.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.868+0000 7f7e8effd700 1 -- 172.21.15.181:0/1247468614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e80009430 con 0x7f7e900cd220 2023-10-16T09:43:57.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.869+0000 7f7e8effd700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e900cd220 0x7f7e90155af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e90099eb0 tx=0x7f7e80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:57.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.869+0000 7f7e77fff700 1 -- 172.21.15.181:0/1247468614 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80003ab0 con 0x7f7e900cd220 2023-10-16T09:43:57.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.869+0000 7f7e77fff700 1 -- 172.21.15.181:0/1247468614 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80003c10 con 0x7f7e900cd220 2023-10-16T09:43:57.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.869+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9015ad70 con 0x7f7e900cd220 2023-10-16T09:43:57.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.869+0000 7f7e77fff700 1 -- 172.21.15.181:0/1247468614 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8001d430 con 0x7f7e900cd220 2023-10-16T09:43:57.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.869+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e90156030 con 0x7f7e900cd220 2023-10-16T09:43:57.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.870+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e900cddc0 con 0x7f7e900cd220 2023-10-16T09:43:57.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.870+0000 7f7e77fff700 1 -- 172.21.15.181:0/1247468614 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f7e8001d590 con 0x7f7e900cd220 2023-10-16T09:43:57.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.871+0000 7f7e77fff700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e7c06ae70 0x7f7e7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:57.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.871+0000 7f7e77fff700 1 -- 172.21.15.181:0/1247468614 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7e8008a230 con 0x7f7e900cd220 2023-10-16T09:43:57.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.875+0000 7f7e77fff700 1 -- 172.21.15.181:0/1247468614 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e800557a0 con 0x7f7e900cd220 2023-10-16T09:43:57.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.881+0000 7f7e8e7fc700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e7c06ae70 0x7f7e7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:57.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:57.881+0000 7f7e8e7fc700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e7c06ae70 0x7f7e7c06d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7e900ca7b0 tx=0x7f7e84001e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.125+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e900b0530 con 0x7f7e900cd220 2023-10-16T09:43:58.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.125+0000 7f7e77fff700 1 -- 172.21.15.181:0/1247468614 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7f7e8002be90 con 0x7f7e900cd220 2023-10-16T09:43:58.127 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:58.127 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-16T09:43:58.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e7c06ae70 msgr2=0x7f7e7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e7c06ae70 0x7f7e7c06d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7e900ca7b0 tx=0x7f7e84001e40 comp rx=0 tx=0).stop 2023-10-16T09:43:58.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e900cd220 msgr2=0x7f7e90155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e900cd220 0x7f7e90155af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e90099eb0 tx=0x7f7e80004750 comp rx=0 tx=0).stop 2023-10-16T09:43:58.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 shutdown_connections 2023-10-16T09:43:58.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e7c06ae70 0x7f7e7c06d330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9015a680 0x7f7e900c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e900cd220 0x7f7e90155af0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 --2- 172.21.15.181:0/1247468614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e900d0c80 0x7f7e900c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.127+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 >> 172.21.15.181:0/1247468614 conn(0x7f7e90099a00 msgr2=0x7f7e900cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:58.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.128+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 shutdown_connections 2023-10-16T09:43:58.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.128+0000 7f7e959d9700 1 -- 172.21.15.181:0/1247468614 wait complete. 2023-10-16T09:43:58.131 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 3 2023-10-16T09:43:58.141 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:43:58.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.260+0000 7fd59b6ed700 1 Processor -- start 2023-10-16T09:43:58.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.261+0000 7fd59b6ed700 1 -- start start 2023-10-16T09:43:58.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.261+0000 7fd59b6ed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940d0370 0x7fd5940ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.261+0000 7fd59b6ed700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5940cd2d0 0x7fd5940d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.261+0000 7fd59b6ed700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5940cc510 con 0x7fd59415b990 2023-10-16T09:43:58.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.261+0000 7fd59b6ed700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5940d24f0 con 0x7fd5940cd2d0 2023-10-16T09:43:58.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.261+0000 7fd59b6ed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940cc7f0 con 0x7fd5940d0370 2023-10-16T09:43:58.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd598c88700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940d0370 0x7fd5940ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd599c8a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5940cd2d0 0x7fd5940d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd599489700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd59415b990 0x7fd5940cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59116/0 (socket says 172.21.15.181:59116) 2023-10-16T09:43:58.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd599489700 1 -- 172.21.15.181:0/2340557712 learned_addr learned my addr 172.21.15.181:0/2340557712 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:43:58.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd58bfff700 1 -- 172.21.15.181:0/2340557712 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502922915 0 0) 0x7fd5940cc510 con 0x7fd59415b990 2023-10-16T09:43:58.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd58bfff700 1 -- 172.21.15.181:0/2340557712 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5800035b0 con 0x7fd59415b990 2023-10-16T09:43:58.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd598c88700 1 -- 172.21.15.181:0/2340557712 >> v2:172.21.15.190:3300/0 conn(0x7fd5940cd2d0 msgr2=0x7fd5940d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:58.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd598c88700 1 --2- 172.21.15.181:0/2340557712 >> v2:172.21.15.190:3300/0 conn(0x7fd5940cd2d0 0x7fd5940d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd598c88700 1 -- 172.21.15.181:0/2340557712 >> v1:172.21.15.190:6789/0 conn(0x7fd59415b990 legacy=0x7fd5940cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd598c88700 1 -- 172.21.15.181:0/2340557712 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940ccad0 con 0x7fd5940d0370 2023-10-16T09:43:58.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd599c8a700 1 --2- 172.21.15.181:0/2340557712 >> v2:172.21.15.190:3300/0 conn(0x7fd5940cd2d0 0x7fd5940d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:58.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd598c88700 1 --2- 172.21.15.181:0/2340557712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940d0370 0x7fd5940ce8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd590009840 tx=0x7fd590009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3f491bf6b62e66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd58bfff700 1 -- 172.21.15.181:0/2340557712 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590011830 con 0x7fd5940d0370 2023-10-16T09:43:58.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd58bfff700 1 -- 172.21.15.181:0/2340557712 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590011990 con 0x7fd5940d0370 2023-10-16T09:43:58.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.262+0000 7fd58bfff700 1 -- 172.21.15.181:0/2340557712 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590011d00 con 0x7fd5940d0370 2023-10-16T09:43:58.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 -- 172.21.15.181:0/2340557712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940d0370 msgr2=0x7fd5940ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/2340557712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940d0370 0x7fd5940ce8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd590009840 tx=0x7fd590009b50 comp rx=0 tx=0).stop 2023-10-16T09:43:58.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 -- 172.21.15.181:0/2340557712 shutdown_connections 2023-10-16T09:43:58.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/2340557712 >> v2:172.21.15.190:3300/0 conn(0x7fd5940cd2d0 0x7fd5940d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/2340557712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940d0370 0x7fd5940ce8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 -- 172.21.15.181:0/2340557712 >> 172.21.15.181:0/2340557712 conn(0x7fd59409a9a0 msgr2=0x7fd5940d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:58.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 -- 172.21.15.181:0/2340557712 shutdown_connections 2023-10-16T09:43:58.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.263+0000 7fd59b6ed700 1 -- 172.21.15.181:0/2340557712 wait complete. 2023-10-16T09:43:58.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.264+0000 7fd59b6ed700 1 Processor -- start 2023-10-16T09:43:58.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.264+0000 7fd59b6ed700 1 -- start start 2023-10-16T09:43:58.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd59b6ed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940cd2d0 0x7fd594154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd59b6ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5940d0370 0x7fd5941f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd59b6ed700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd59415b990 0x7fd5940ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd59b6ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5941f6e90 con 0x7fd5940d0370 2023-10-16T09:43:58.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd59b6ed700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd594155190 con 0x7fd59415b990 2023-10-16T09:43:58.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd59b6ed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940cb090 con 0x7fd5940cd2d0 2023-10-16T09:43:58.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5940d0370 0x7fd5941f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5940d0370 0x7fd5941f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35872/0 (socket says 172.21.15.181:35872) 2023-10-16T09:43:58.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 -- 172.21.15.181:0/1198720768 learned_addr learned my addr 172.21.15.181:0/1198720768 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:58.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd599c8a700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd59415b990 0x7fd5940ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd599489700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940cd2d0 0x7fd594154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 -- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940cd2d0 msgr2=0x7fd594154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940cd2d0 0x7fd594154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 -- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd59415b990 msgr2=0x7fd5940ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd59415b990 0x7fd5940ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 -- 172.21.15.181:0/1198720768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c009a70 con 0x7fd5940d0370 2023-10-16T09:43:58.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd598c88700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5940d0370 0x7fd5941f7320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd590009fd0 tx=0x7fd590004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.265+0000 7fd589ffb700 1 -- 172.21.15.181:0/1198720768 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590004550 con 0x7fd5940d0370 2023-10-16T09:43:58.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.266+0000 7fd589ffb700 1 -- 172.21.15.181:0/1198720768 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59001e070 con 0x7fd5940d0370 2023-10-16T09:43:58.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.266+0000 7fd589ffb700 1 -- 172.21.15.181:0/1198720768 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900038c0 con 0x7fd5940d0370 2023-10-16T09:43:58.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.266+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5900094a0 con 0x7fd5940d0370 2023-10-16T09:43:58.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.266+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5941f6b20 con 0x7fd5940d0370 2023-10-16T09:43:58.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.267+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5940cb5b0 con 0x7fd5940d0370 2023-10-16T09:43:58.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.267+0000 7fd589ffb700 1 -- 172.21.15.181:0/1198720768 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fd590003a20 con 0x7fd5940d0370 2023-10-16T09:43:58.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.267+0000 7fd589ffb700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd57806ae70 0x7fd57806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.268+0000 7fd589ffb700 1 -- 172.21.15.181:0/1198720768 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd590089f00 con 0x7fd5940d0370 2023-10-16T09:43:58.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.268+0000 7fd599489700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd57806ae70 0x7fd57806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.268+0000 7fd599489700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd57806ae70 0x7fd57806d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd584005960 tx=0x7fd584009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.272+0000 7fd589ffb700 1 -- 172.21.15.181:0/1198720768 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5940cb5b0 con 0x7fd5940d0370 2023-10-16T09:43:58.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.473+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5941f6980 con 0x7fd5940d0370 2023-10-16T09:43:58.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.474+0000 7fd589ffb700 1 -- 172.21.15.181:0/1198720768 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12899 (secure 0 0 0) 0x7fd5941f6980 con 0x7fd5940d0370 2023-10-16T09:43:58.476 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:58.476 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":17,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:43:57.560838+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:43:58.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd57806ae70 msgr2=0x7fd57806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd57806ae70 0x7fd57806d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd584005960 tx=0x7fd584009040 comp rx=0 tx=0).stop 2023-10-16T09:43:58.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5940d0370 msgr2=0x7fd5941f7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5940d0370 0x7fd5941f7320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd590009fd0 tx=0x7fd590004750 comp rx=0 tx=0).stop 2023-10-16T09:43:58.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 shutdown_connections 2023-10-16T09:43:58.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd57806ae70 0x7fd57806d330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5940cd2d0 0x7fd594154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd59415b990 0x7fd5940ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 --2- 172.21.15.181:0/1198720768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5940d0370 0x7fd5941f7320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 >> 172.21.15.181:0/1198720768 conn(0x7fd59409a9a0 msgr2=0x7fd5940ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:58.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 shutdown_connections 2023-10-16T09:43:58.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.477+0000 7fd59b6ed700 1 -- 172.21.15.181:0/1198720768 wait complete. 2023-10-16T09:43:58.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:43:58.491 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:43:58.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.618+0000 7f8fffae4700 1 Processor -- start 2023-10-16T09:43:58.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.618+0000 7f8fffae4700 1 -- start start 2023-10-16T09:43:58.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.619+0000 7f8fffae4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80cb2b0 0x7f8ff806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.619+0000 7f8fffae4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ff80b5fd0 0x7f8ff815f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.619+0000 7f8fffae4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff8070590 con 0x7f8ff80cb690 2023-10-16T09:43:58.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.619+0000 7f8fffae4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff8156330 con 0x7f8ff80b5fd0 2023-10-16T09:43:58.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.619+0000 7f8fffae4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff806fcf0 con 0x7f8ff80cb2b0 2023-10-16T09:43:58.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.619+0000 7f8ffe081700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ff80b5fd0 0x7f8ff815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.619+0000 7f8ffd880700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80cb2b0 0x7f8ff806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffd07f700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8ff80cb690 0x7f8ff80b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59138/0 (socket says 172.21.15.181:59138) 2023-10-16T09:43:58.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffe081700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ff80b5fd0 0x7f8ff815f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:40940/0 (socket says 172.21.15.181:40940) 2023-10-16T09:43:58.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffd07f700 1 -- 172.21.15.181:0/1147954017 learned_addr learned my addr 172.21.15.181:0/1147954017 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:43:58.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffd880700 1 -- 172.21.15.181:0/1147954017 >> v2:172.21.15.190:3300/0 conn(0x7f8ff80b5fd0 msgr2=0x7f8ff815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:58.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffd880700 1 --2- 172.21.15.181:0/1147954017 >> v2:172.21.15.190:3300/0 conn(0x7f8ff80b5fd0 0x7f8ff815f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffd880700 1 -- 172.21.15.181:0/1147954017 >> v1:172.21.15.190:6789/0 conn(0x7f8ff80cb690 legacy=0x7f8ff80b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffd880700 1 -- 172.21.15.181:0/1147954017 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff806ffd0 con 0x7f8ff80cb2b0 2023-10-16T09:43:58.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffc87e700 1 -- 172.21.15.181:0/1147954017 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510689666 0 0) 0x7f8ff8070590 con 0x7f8ff80cb690 2023-10-16T09:43:58.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffe081700 1 --2- 172.21.15.181:0/1147954017 >> v2:172.21.15.190:3300/0 conn(0x7f8ff80b5fd0 0x7f8ff815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:58.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffd880700 1 --2- 172.21.15.181:0/1147954017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80cb2b0 0x7f8ff806f710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009840 tx=0x7f8fe8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2835fac63eee7bc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffc87e700 1 -- 172.21.15.181:0/1147954017 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8011830 con 0x7f8ff80cb2b0 2023-10-16T09:43:58.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffc87e700 1 -- 172.21.15.181:0/1147954017 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8011990 con 0x7f8ff80cb2b0 2023-10-16T09:43:58.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.620+0000 7f8ffc87e700 1 -- 172.21.15.181:0/1147954017 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8011d20 con 0x7f8ff80cb2b0 2023-10-16T09:43:58.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 -- 172.21.15.181:0/1147954017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80cb2b0 msgr2=0x7f8ff806f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 --2- 172.21.15.181:0/1147954017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80cb2b0 0x7f8ff806f710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009840 tx=0x7f8fe8009b50 comp rx=0 tx=0).stop 2023-10-16T09:43:58.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 -- 172.21.15.181:0/1147954017 shutdown_connections 2023-10-16T09:43:58.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 --2- 172.21.15.181:0/1147954017 >> v2:172.21.15.190:3300/0 conn(0x7f8ff80b5fd0 0x7f8ff815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 --2- 172.21.15.181:0/1147954017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80cb2b0 0x7f8ff806f710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 -- 172.21.15.181:0/1147954017 >> 172.21.15.181:0/1147954017 conn(0x7f8ff80cd010 msgr2=0x7f8ff80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:58.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 -- 172.21.15.181:0/1147954017 shutdown_connections 2023-10-16T09:43:58.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.621+0000 7f8fffae4700 1 -- 172.21.15.181:0/1147954017 wait complete. 2023-10-16T09:43:58.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.622+0000 7f8fffae4700 1 Processor -- start 2023-10-16T09:43:58.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.622+0000 7f8fffae4700 1 -- start start 2023-10-16T09:43:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8fffae4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80b5fd0 0x7f8ff8158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8fffae4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff80cb2b0 0x7f8ff80c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8fffae4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ff80cb690 0x7f8ff81564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8fffae4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80c67c0 con 0x7f8ff80cb690 2023-10-16T09:43:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8fffae4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80c5b40 con 0x7f8ff80cb2b0 2023-10-16T09:43:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8fffae4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80c5e20 con 0x7f8ff80b5fd0 2023-10-16T09:43:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ff80cb690 0x7f8ff81564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ff80cb690 0x7f8ff81564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35894/0 (socket says 172.21.15.181:35894) 2023-10-16T09:43:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 -- 172.21.15.181:0/2940242284 learned_addr learned my addr 172.21.15.181:0/2940242284 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:58.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffd880700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80b5fd0 0x7f8ff8158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffd07f700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff80cb2b0 0x7f8ff80c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 -- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80b5fd0 msgr2=0x7f8ff8158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80b5fd0 0x7f8ff8158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 -- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff80cb2b0 msgr2=0x7f8ff80c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff80cb2b0 0x7f8ff80c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 -- 172.21.15.181:0/2940242284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0009a70 con 0x7f8ff80cb690 2023-10-16T09:43:58.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8ffe081700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ff80cb690 0x7f8ff81564d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000efc0 tx=0x7f8ff000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.623+0000 7f8fee7fc700 1 -- 172.21.15.181:0/2940242284 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000ca90 con 0x7f8ff80cb690 2023-10-16T09:43:58.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.624+0000 7f8fee7fc700 1 -- 172.21.15.181:0/2940242284 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff000f040 con 0x7f8ff80cb690 2023-10-16T09:43:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.624+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe80094a0 con 0x7f8ff80cb690 2023-10-16T09:43:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.624+0000 7f8fee7fc700 1 -- 172.21.15.181:0/2940242284 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000ce10 con 0x7f8ff80cb690 2023-10-16T09:43:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.624+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff80c63e0 con 0x7f8ff80cb690 2023-10-16T09:43:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.625+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff80c6900 con 0x7f8ff80cb690 2023-10-16T09:43:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.625+0000 7f8fee7fc700 1 -- 172.21.15.181:0/2940242284 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8ff0014480 con 0x7f8ff80cb690 2023-10-16T09:43:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.625+0000 7f8fee7fc700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8fdc06af00 0x7f8fdc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.625+0000 7f8fee7fc700 1 -- 172.21.15.181:0/2940242284 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8ff00898d0 con 0x7f8ff80cb690 2023-10-16T09:43:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.625+0000 7f8ffd880700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8fdc06af00 0x7f8fdc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.626+0000 7f8ffd880700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8fdc06af00 0x7f8fdc06d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.630+0000 7f8fee7fc700 1 -- 172.21.15.181:0/2940242284 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8ff80c6900 con 0x7f8ff80cb690 2023-10-16T09:43:58.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.823+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ff80c6900 con 0x7f8ff80cb690 2023-10-16T09:43:58.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.823+0000 7f8fee7fc700 1 -- 172.21.15.181:0/2940242284 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ff80c6900 con 0x7f8ff80cb690 2023-10-16T09:43:58.825 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:43:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8fdc06af00 msgr2=0x7f8fdc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8fdc06af00 0x7f8fdc06d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8008030 comp rx=0 tx=0).stop 2023-10-16T09:43:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ff80cb690 msgr2=0x7f8ff81564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ff80cb690 0x7f8ff81564d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000efc0 tx=0x7f8ff000c3a0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 shutdown_connections 2023-10-16T09:43:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8fdc06af00 0x7f8fdc06d3c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ff80b5fd0 0x7f8ff8158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ff80cb690 0x7f8ff81564d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 --2- 172.21.15.181:0/2940242284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff80cb2b0 0x7f8ff80c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 >> 172.21.15.181:0/2940242284 conn(0x7f8ff80cd010 msgr2=0x7f8ff80d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 shutdown_connections 2023-10-16T09:43:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.826+0000 7f8fffae4700 1 -- 172.21.15.181:0/2940242284 wait complete. 2023-10-16T09:43:58.839 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T09:43:58.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.959+0000 7fd5b7698700 1 Processor -- start 2023-10-16T09:43:58.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.959+0000 7fd5b7698700 1 -- start start 2023-10-16T09:43:58.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.959+0000 7fd5b7698700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.959+0000 7fd5b7698700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5b00af450 0x7fd5b006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b7698700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b006f870 con 0x7fd5b006db80 2023-10-16T09:43:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b7698700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b00d1160 con 0x7fd5b00af450 2023-10-16T09:43:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b7698700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00cc8e0 con 0x7fd5b015dfd0 2023-10-16T09:43:58.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b5434700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b4c33700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5b00af450 0x7fd5b006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b5c35700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd5b006db80 0x7fd5b00cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59154/0 (socket says 172.21.15.181:59154) 2023-10-16T09:43:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b5c35700 1 -- 172.21.15.181:0/3691835337 learned_addr learned my addr 172.21.15.181:0/3691835337 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:43:58.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5a7fff700 1 -- 172.21.15.181:0/3691835337 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925331111 0 0) 0x7fd5b006f870 con 0x7fd5b006db80 2023-10-16T09:43:58.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5a7fff700 1 -- 172.21.15.181:0/3691835337 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd59c0035b0 con 0x7fd5b006db80 2023-10-16T09:43:58.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b4c33700 1 -- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 msgr2=0x7fd5b00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:58.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b4c33700 1 --2- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b4c33700 1 -- 172.21.15.181:0/3691835337 >> v1:172.21.15.190:6789/0 conn(0x7fd5b006db80 legacy=0x7fd5b00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b4c33700 1 -- 172.21.15.181:0/3691835337 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00ccbc0 con 0x7fd5b00af450 2023-10-16T09:43:58.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b5434700 1 --2- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:43:58.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.960+0000 7fd5b4c33700 1 --2- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 0x7fd5b006d640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009840 tx=0x7fd5ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27dd3965ba8a5d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5a7fff700 1 -- 172.21.15.181:0/3691835337 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac011830 con 0x7fd5b00af450 2023-10-16T09:43:58.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5a7fff700 1 -- 172.21.15.181:0/3691835337 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac011990 con 0x7fd5b00af450 2023-10-16T09:43:58.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5a7fff700 1 -- 172.21.15.181:0/3691835337 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac011d20 con 0x7fd5b00af450 2023-10-16T09:43:58.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 -- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 msgr2=0x7fd5b006d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 --2- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 0x7fd5b006d640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009840 tx=0x7fd5ac009b50 comp rx=0 tx=0).stop 2023-10-16T09:43:58.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 -- 172.21.15.181:0/3691835337 shutdown_connections 2023-10-16T09:43:58.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 --2- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 --2- 172.21.15.181:0/3691835337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 0x7fd5b006d640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 -- 172.21.15.181:0/3691835337 >> 172.21.15.181:0/3691835337 conn(0x7fd5b00d04a0 msgr2=0x7fd5b00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:58.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 -- 172.21.15.181:0/3691835337 shutdown_connections 2023-10-16T09:43:58.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.961+0000 7fd5b7698700 1 -- 172.21.15.181:0/3691835337 wait complete. 2023-10-16T09:43:58.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.962+0000 7fd5b7698700 1 Processor -- start 2023-10-16T09:43:58.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.962+0000 7fd5b7698700 1 -- start start 2023-10-16T09:43:58.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b7698700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b006db80 0x7fd5b00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b7698700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 0x7fd5b0156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b7698700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b7698700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0157070 con 0x7fd5b006db80 2023-10-16T09:43:58.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b7698700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00c6150 con 0x7fd5b00af450 2023-10-16T09:43:58.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b7698700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00c6430 con 0x7fd5b015dfd0 2023-10-16T09:43:58.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b5434700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b006db80 0x7fd5b00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b5c35700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b5434700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b006db80 0x7fd5b00c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35906/0 (socket says 172.21.15.181:35906) 2023-10-16T09:43:58.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b5434700 1 -- 172.21.15.181:0/4280440721 learned_addr learned my addr 172.21.15.181:0/4280440721 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:58.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b5c35700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:32936/0 (socket says 172.21.15.181:32936) 2023-10-16T09:43:58.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.963+0000 7fd5b4c33700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 0x7fd5b0156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b5434700 1 -- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 msgr2=0x7fd5b00b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b5434700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b5434700 1 -- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 msgr2=0x7fd5b0156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b5434700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 0x7fd5b0156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b5434700 1 -- 172.21.15.181:0/4280440721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a0009430 con 0x7fd5b006db80 2023-10-16T09:43:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b5434700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b006db80 0x7fd5b00c5b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000ebe0 tx=0x7fd5a000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5a5ffb700 1 -- 172.21.15.181:0/4280440721 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0009740 con 0x7fd5b006db80 2023-10-16T09:43:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5a5ffb700 1 -- 172.21.15.181:0/4280440721 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a0005a20 con 0x7fd5b006db80 2023-10-16T09:43:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac0094a0 con 0x7fd5b006db80 2023-10-16T09:43:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5a5ffb700 1 -- 172.21.15.181:0/4280440721 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0005db0 con 0x7fd5b006db80 2023-10-16T09:43:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.964+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b00c69f0 con 0x7fd5b006db80 2023-10-16T09:43:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.965+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b0157300 con 0x7fd5b006db80 2023-10-16T09:43:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.965+0000 7fd5a5ffb700 1 -- 172.21.15.181:0/4280440721 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd5a000c850 con 0x7fd5b006db80 2023-10-16T09:43:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.966+0000 7fd5a5ffb700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd59406af00 0x7fd59406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.966+0000 7fd5a5ffb700 1 -- 172.21.15.181:0/4280440721 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd5a0089480 con 0x7fd5b006db80 2023-10-16T09:43:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.966+0000 7fd5b4c33700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd59406af00 0x7fd59406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.966+0000 7fd5b4c33700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd59406af00 0x7fd59406d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:58.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:58.970+0000 7fd5a5ffb700 1 -- 172.21.15.181:0/4280440721 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5a0054880 con 0x7fd5b006db80 2023-10-16T09:43:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.166+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd5b0067d20 con 0x7fd5b006db80 2023-10-16T09:43:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.166+0000 7fd5a5ffb700 1 -- 172.21.15.181:0/4280440721 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd5a0020090 con 0x7fd5b006db80 2023-10-16T09:43:59.168 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:43:59.168 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:43:59.168 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:43:59.169 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T09:43:59.169 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T09:43:59.169 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:43:59.169 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:43:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd59406af00 msgr2=0x7fd59406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd59406af00 0x7fd59406d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac008030 comp rx=0 tx=0).stop 2023-10-16T09:43:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b006db80 msgr2=0x7fd5b00c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b006db80 0x7fd5b00c5b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000ebe0 tx=0x7fd5a000eef0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 shutdown_connections 2023-10-16T09:43:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd59406af00 0x7fd59406d3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5b015dfd0 0x7fd5b00b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b006db80 0x7fd5b00c5b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 --2- 172.21.15.181:0/4280440721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5b00af450 0x7fd5b0156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 >> 172.21.15.181:0/4280440721 conn(0x7fd5b00d04a0 msgr2=0x7fd5b00cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 shutdown_connections 2023-10-16T09:43:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.168+0000 7fd5b7698700 1 -- 172.21.15.181:0/4280440721 wait complete. 2023-10-16T09:43:59.181 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T09:43:59.182 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T09:43:59.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.307+0000 7f148c48d700 1 Processor -- start 2023-10-16T09:43:59.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.308+0000 7f148c48d700 1 -- start start 2023-10-16T09:43:59.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.308+0000 7f148c48d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840ce610 0x7f148406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148c48d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1484162650 con 0x7f14840d0fc0 2023-10-16T09:43:59.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148c48d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14840d0230 con 0x7f14840ce9f0 2023-10-16T09:43:59.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148c48d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1484162190 con 0x7f14840ce610 2023-10-16T09:43:59.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489a28700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f14840d0fc0 0x7f14840cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:46992/0 (socket says 172.21.15.181:46992) 2023-10-16T09:43:59.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489a28700 1 -- 172.21.15.181:0/2904599488 learned_addr learned my addr 172.21.15.181:0/2904599488 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:43:59.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148a229700 1 --2- 172.21.15.181:0/2904599488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840ce610 0x7f148406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489227700 1 -- 172.21.15.181:0/2904599488 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535988203 0 0) 0x7f1484162650 con 0x7f14840d0fc0 2023-10-16T09:43:59.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489227700 1 -- 172.21.15.181:0/2904599488 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14700035b0 con 0x7f14840d0fc0 2023-10-16T09:43:59.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489227700 1 -- 172.21.15.181:0/2904599488 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188306117 0 0) 0x7f14840d0230 con 0x7f14840ce9f0 2023-10-16T09:43:59.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148a229700 1 -- 172.21.15.181:0/2904599488 >> v1:172.21.15.190:6789/0 conn(0x7f14840ce9f0 legacy=0x7f148406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148a229700 1 -- 172.21.15.181:0/2904599488 >> v1:172.21.15.181:6789/0 conn(0x7f14840d0fc0 legacy=0x7f14840cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148a229700 1 -- 172.21.15.181:0/2904599488 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484162370 con 0x7f14840ce610 2023-10-16T09:43:59.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f148a229700 1 --2- 172.21.15.181:0/2904599488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840ce610 0x7f148406d660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1474009850 tx=0x7f1474009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46c97fdb92dafaf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:59.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489227700 1 -- 172.21.15.181:0/2904599488 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474005e30 con 0x7f14840ce610 2023-10-16T09:43:59.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489227700 1 -- 172.21.15.181:0/2904599488 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1474004030 con 0x7f14840ce610 2023-10-16T09:43:59.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.309+0000 7f1489227700 1 -- 172.21.15.181:0/2904599488 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14740043c0 con 0x7f14840ce610 2023-10-16T09:43:59.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.310+0000 7f148c48d700 1 -- 172.21.15.181:0/2904599488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840ce610 msgr2=0x7f148406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.310+0000 7f148c48d700 1 --2- 172.21.15.181:0/2904599488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840ce610 0x7f148406d660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1474009850 tx=0x7f1474009b60 comp rx=0 tx=0).stop 2023-10-16T09:43:59.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.310+0000 7f148c48d700 1 -- 172.21.15.181:0/2904599488 shutdown_connections 2023-10-16T09:43:59.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.310+0000 7f148c48d700 1 --2- 172.21.15.181:0/2904599488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840ce610 0x7f148406d660 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.310+0000 7f148c48d700 1 -- 172.21.15.181:0/2904599488 >> 172.21.15.181:0/2904599488 conn(0x7f14840d04a0 msgr2=0x7f14840d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:59.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.310+0000 7f148c48d700 1 -- 172.21.15.181:0/2904599488 shutdown_connections 2023-10-16T09:43:59.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.310+0000 7f148c48d700 1 -- 172.21.15.181:0/2904599488 wait complete. 2023-10-16T09:43:59.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.311+0000 7f148c48d700 1 Processor -- start 2023-10-16T09:43:59.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.311+0000 7f148c48d700 1 -- start start 2023-10-16T09:43:59.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148c48d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840ce610 0x7f14841f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148c48d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14840ce9f0 0x7f14840c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148c48d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840d0fc0 0x7f14840c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148c48d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148409c540 con 0x7f14840ce610 2023-10-16T09:43:59.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148c48d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148409b8c0 con 0x7f14840ce9f0 2023-10-16T09:43:59.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148c48d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148409bba0 con 0x7f14840d0fc0 2023-10-16T09:43:59.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840ce610 0x7f14841f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840ce610 0x7f14841f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35918/0 (socket says 172.21.15.181:35918) 2023-10-16T09:43:59.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 -- 172.21.15.181:0/3626093346 learned_addr learned my addr 172.21.15.181:0/3626093346 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:59.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f1489a28700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14840ce9f0 0x7f14840c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148aa2a700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840d0fc0 0x7f14840c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 -- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840d0fc0 msgr2=0x7f14840c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840d0fc0 0x7f14840c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 -- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14840ce9f0 msgr2=0x7f14840c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14840ce9f0 0x7f14840c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148a229700 1 -- 172.21.15.181:0/3626093346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1474009430 con 0x7f14840ce610 2023-10-16T09:43:59.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.312+0000 7f148aa2a700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840d0fc0 0x7f14840c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:43:59.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.313+0000 7f148a229700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840ce610 0x7f14841f5fa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f148406d530 tx=0x7f1474004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:59.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.313+0000 7f147affd700 1 -- 172.21.15.181:0/3626093346 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474003c70 con 0x7f14840ce610 2023-10-16T09:43:59.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.313+0000 7f147affd700 1 -- 172.21.15.181:0/3626093346 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1474003dd0 con 0x7f14840ce610 2023-10-16T09:43:59.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.313+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148409be80 con 0x7f14840ce610 2023-10-16T09:43:59.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.313+0000 7f147affd700 1 -- 172.21.15.181:0/3626093346 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147401d670 con 0x7f14840ce610 2023-10-16T09:43:59.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.313+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14841f64e0 con 0x7f14840ce610 2023-10-16T09:43:59.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.314+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14841f3350 con 0x7f14840ce610 2023-10-16T09:43:59.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.314+0000 7f147affd700 1 -- 172.21.15.181:0/3626093346 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f147401d7d0 con 0x7f14840ce610 2023-10-16T09:43:59.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.315+0000 7f147affd700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f146806af00 0x7f146806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.315+0000 7f1489a28700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f146806af00 0x7f146806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.315+0000 7f147affd700 1 -- 172.21.15.181:0/3626093346 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f147408a250 con 0x7f14840ce610 2023-10-16T09:43:59.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.315+0000 7f1489a28700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f146806af00 0x7f146806d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1480007fd0 tx=0x7f1480000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:59.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.319+0000 7f147affd700 1 -- 172.21.15.181:0/3626093346 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f14740556e0 con 0x7f14840ce610 2023-10-16T09:43:59.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.513+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1484067d20 con 0x7f14840ce610 2023-10-16T09:43:59.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.770+0000 7f147affd700 1 -- 172.21.15.181:0/3626093346 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1474006c60 con 0x7f14840ce610 2023-10-16T09:43:59.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f146806af00 msgr2=0x7f146806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f146806af00 0x7f146806d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1480007fd0 tx=0x7f1480000bc0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840ce610 msgr2=0x7f14841f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840ce610 0x7f14841f5fa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f148406d530 tx=0x7f1474004750 comp rx=0 tx=0).stop 2023-10-16T09:43:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 shutdown_connections 2023-10-16T09:43:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f146806af00 0x7f146806d3c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14840d0fc0 0x7f14840c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840ce610 0x7f14841f5fa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 --2- 172.21.15.181:0/3626093346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14840ce9f0 0x7f14840c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 >> 172.21.15.181:0/3626093346 conn(0x7f14840d04a0 msgr2=0x7f14840b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:59.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 shutdown_connections 2023-10-16T09:43:59.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.773+0000 7f148c48d700 1 -- 172.21.15.181:0/3626093346 wait complete. 2023-10-16T09:43:59.778 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T09:43:59.791 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T09:43:59.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.917+0000 7fb3994ec700 1 Processor -- start 2023-10-16T09:43:59.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.917+0000 7fb3994ec700 1 -- start start 2023-10-16T09:43:59.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3994ec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb39406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3994ec700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3940cb690 0x7fb3940b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3994ec700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb394070590 con 0x7fb3940b2ba0 2023-10-16T09:43:59.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3994ec700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb394156330 con 0x7fb3940cb690 2023-10-16T09:43:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3994ec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39406fcf0 con 0x7fb3940cb2b0 2023-10-16T09:43:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3927fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3940cb690 0x7fb3940b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3937fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb3940b2ba0 0x7fb39415f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47004/0 (socket says 172.21.15.181:47004) 2023-10-16T09:43:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.918+0000 7fb3937fe700 1 -- 172.21.15.181:0/3298109750 learned_addr learned my addr 172.21.15.181:0/3298109750 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:43:59.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb392ffd700 1 --2- 172.21.15.181:0/3298109750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb39406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3927fc700 1 -- 172.21.15.181:0/3298109750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 msgr2=0x7fb39406f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:43:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3927fc700 1 --2- 172.21.15.181:0/3298109750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb39406f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3927fc700 1 -- 172.21.15.181:0/3298109750 >> v1:172.21.15.181:6789/0 conn(0x7fb3940b2ba0 legacy=0x7fb39415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3927fc700 1 -- 172.21.15.181:0/3298109750 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39406ffd0 con 0x7fb3940cb690 2023-10-16T09:43:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb391ffb700 1 -- 172.21.15.181:0/3298109750 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865182785 0 0) 0x7fb394070590 con 0x7fb3940b2ba0 2023-10-16T09:43:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb392ffd700 1 --2- 172.21.15.181:0/3298109750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb39406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:43:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3927fc700 1 --2- 172.21.15.181:0/3298109750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940b2660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb384009840 tx=0x7fb384009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c5554b13b2cb31 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb391ffb700 1 -- 172.21.15.181:0/3298109750 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384011830 con 0x7fb3940cb690 2023-10-16T09:43:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb391ffb700 1 -- 172.21.15.181:0/3298109750 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb384011990 con 0x7fb3940cb690 2023-10-16T09:43:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb391ffb700 1 -- 172.21.15.181:0/3298109750 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384011d20 con 0x7fb3940cb690 2023-10-16T09:43:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3994ec700 1 -- 172.21.15.181:0/3298109750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 msgr2=0x7fb3940b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3994ec700 1 --2- 172.21.15.181:0/3298109750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940b2660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb384009840 tx=0x7fb384009b50 comp rx=0 tx=0).stop 2023-10-16T09:43:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3994ec700 1 -- 172.21.15.181:0/3298109750 shutdown_connections 2023-10-16T09:43:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3994ec700 1 --2- 172.21.15.181:0/3298109750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb39406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3994ec700 1 --2- 172.21.15.181:0/3298109750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940b2660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.919+0000 7fb3994ec700 1 -- 172.21.15.181:0/3298109750 >> 172.21.15.181:0/3298109750 conn(0x7fb3940cd010 msgr2=0x7fb3940cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:43:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.920+0000 7fb3994ec700 1 -- 172.21.15.181:0/3298109750 shutdown_connections 2023-10-16T09:43:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.920+0000 7fb3994ec700 1 -- 172.21.15.181:0/3298109750 wait complete. 2023-10-16T09:43:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.921+0000 7fb3994ec700 1 Processor -- start 2023-10-16T09:43:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.921+0000 7fb3994ec700 1 -- start start 2023-10-16T09:43:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3940b2ba0 0x7fb3940c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb3940c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940b4d40 con 0x7fb3940cb690 2023-10-16T09:43:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940c3720 con 0x7fb3940b2ba0 2023-10-16T09:43:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940b4130 con 0x7fb3940cb2b0 2023-10-16T09:43:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35936/0 (socket says 172.21.15.181:35936) 2023-10-16T09:43:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 -- 172.21.15.181:0/1428858918 learned_addr learned my addr 172.21.15.181:0/1428858918 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:43:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb392ffd700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3940b2ba0 0x7fb3940c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 -- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 msgr2=0x7fb3940c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3927fc700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb3940c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb3940c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 -- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3940b2ba0 msgr2=0x7fb3940c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:43:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3940b2ba0 0x7fb3940c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:43:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 -- 172.21.15.181:0/1428858918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840094a0 con 0x7fb3940cb690 2023-10-16T09:43:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb392ffd700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3940b2ba0 0x7fb3940c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:43:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3937fe700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940d0400 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00cba0 tx=0x7fb38c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb37b7fe700 1 -- 172.21.15.181:0/1428858918 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00ab80 con 0x7fb3940cb690 2023-10-16T09:43:59.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb37b7fe700 1 -- 172.21.15.181:0/1428858918 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c00ad00 con 0x7fb3940cb690 2023-10-16T09:43:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3940b4410 con 0x7fb3940cb690 2023-10-16T09:43:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb37b7fe700 1 -- 172.21.15.181:0/1428858918 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c007310 con 0x7fb3940cb690 2023-10-16T09:43:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.922+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3940b49d0 con 0x7fb3940cb690 2023-10-16T09:43:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.923+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3940b4eb0 con 0x7fb3940cb690 2023-10-16T09:43:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.924+0000 7fb37b7fe700 1 -- 172.21.15.181:0/1428858918 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb38c007470 con 0x7fb3940cb690 2023-10-16T09:43:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.924+0000 7fb37b7fe700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb38006af00 0x7fb38006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:43:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.924+0000 7fb37b7fe700 1 -- 172.21.15.181:0/1428858918 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb38c0892e0 con 0x7fb3940cb690 2023-10-16T09:43:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.924+0000 7fb392ffd700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb38006af00 0x7fb38006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:43:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.925+0000 7fb392ffd700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb38006af00 0x7fb38006d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3880089b0 tx=0x7fb388008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:43:59.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:43:59.929+0000 7fb37b7fe700 1 -- 172.21.15.181:0/1428858918 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb38c0544e0 con 0x7fb3940cb690 2023-10-16T09:44:00.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.121+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb39406f4f0 con 0x7fb3940cb690 2023-10-16T09:44:00.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.773+0000 7fb37b7fe700 1 -- 172.21.15.181:0/1428858918 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb38c0180d0 con 0x7fb3940cb690 2023-10-16T09:44:00.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb38006af00 msgr2=0x7fb38006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:00.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb38006af00 0x7fb38006d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3880089b0 tx=0x7fb388008040 comp rx=0 tx=0).stop 2023-10-16T09:44:00.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 msgr2=0x7fb3940d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:00.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940d0400 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00cba0 tx=0x7fb38c00cf60 comp rx=0 tx=0).stop 2023-10-16T09:44:00.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 shutdown_connections 2023-10-16T09:44:00.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb38006af00 0x7fb38006d3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3940cb690 0x7fb3940d0400 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3940b2ba0 0x7fb3940c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 --2- 172.21.15.181:0/1428858918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3940cb2b0 0x7fb3940c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 >> 172.21.15.181:0/1428858918 conn(0x7fb3940cd010 msgr2=0x7fb3940b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:00.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 shutdown_connections 2023-10-16T09:44:00.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.776+0000 7fb3994ec700 1 -- 172.21.15.181:0/1428858918 wait complete. 2023-10-16T09:44:00.781 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T09:44:00.800 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T09:44:00.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.923+0000 7f6e78386700 1 Processor -- start 2023-10-16T09:44:00.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.923+0000 7f6e78386700 1 -- start start 2023-10-16T09:44:00.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e78386700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:00.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e78386700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e700ce6a0 0x7f6e7006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:00.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e78386700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e700cef00 con 0x7f6e70072760 2023-10-16T09:44:00.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e78386700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e700cec20 con 0x7f6e700ce6a0 2023-10-16T09:44:00.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e78386700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70071c80 con 0x7f6e7015da70 2023-10-16T09:44:00.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e700ce6a0 0x7f6e7006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:00.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e700ce6a0 0x7f6e7006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35944/0 (socket says 172.21.15.181:35944) 2023-10-16T09:44:00.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 -- 172.21.15.181:0/4057566376 learned_addr learned my addr 172.21.15.181:0/4057566376 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:00.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e76122700 1 --2- 172.21.15.181:0/4057566376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:00.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 -- 172.21.15.181:0/4057566376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 msgr2=0x7f6e700ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:00.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 --2- 172.21.15.181:0/4057566376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 -- 172.21.15.181:0/4057566376 >> v1:172.21.15.190:6789/0 conn(0x7f6e70072760 legacy=0x7f6e701615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:00.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 -- 172.21.15.181:0/4057566376 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e70071f60 con 0x7f6e700ce6a0 2023-10-16T09:44:00.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75921700 1 --2- 172.21.15.181:0/4057566376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e7006a8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e600098f0 tx=0x7f6e60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd96089b33f02bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:00.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75120700 1 -- 172.21.15.181:0/4057566376 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000b8a0 con 0x7f6e700ce6a0 2023-10-16T09:44:00.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75120700 1 -- 172.21.15.181:0/4057566376 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6000ba00 con 0x7f6e700ce6a0 2023-10-16T09:44:00.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.924+0000 7f6e75120700 1 -- 172.21.15.181:0/4057566376 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000bd90 con 0x7f6e700ce6a0 2023-10-16T09:44:00.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 -- 172.21.15.181:0/4057566376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 msgr2=0x7f6e7006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:00.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 --2- 172.21.15.181:0/4057566376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e7006a8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e600098f0 tx=0x7f6e60009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:00.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 -- 172.21.15.181:0/4057566376 shutdown_connections 2023-10-16T09:44:00.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 --2- 172.21.15.181:0/4057566376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 --2- 172.21.15.181:0/4057566376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e7006a8a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 -- 172.21.15.181:0/4057566376 >> 172.21.15.181:0/4057566376 conn(0x7f6e700cd010 msgr2=0x7f6e700cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:00.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 -- 172.21.15.181:0/4057566376 shutdown_connections 2023-10-16T09:44:00.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.925+0000 7f6e78386700 1 -- 172.21.15.181:0/4057566376 wait complete. 2023-10-16T09:44:00.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.927+0000 7f6e78386700 1 Processor -- start 2023-10-16T09:44:00.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.927+0000 7f6e78386700 1 -- start start 2023-10-16T09:44:00.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.927+0000 7f6e78386700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e70072760 0x7f6e700b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:00.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e78386700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e700c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:00.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e78386700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:00.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e78386700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700b31b0 con 0x7f6e700ce6a0 2023-10-16T09:44:00.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e78386700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700b3a50 con 0x7f6e70072760 2023-10-16T09:44:00.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e78386700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700b6050 con 0x7f6e7015da70 2023-10-16T09:44:00.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e700c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:00.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e700c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35948/0 (socket says 172.21.15.181:35948) 2023-10-16T09:44:00.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 -- 172.21.15.181:0/1713583453 learned_addr learned my addr 172.21.15.181:0/1713583453 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e76122700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e70072760 0x7f6e700b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e76923700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 -- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 msgr2=0x7f6e700ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 -- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e70072760 msgr2=0x7f6e700b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e70072760 0x7f6e700b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 -- 172.21.15.181:0/1713583453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e60009550 con 0x7f6e700ce6a0 2023-10-16T09:44:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e76923700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e76122700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e70072760 0x7f6e700b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e75921700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e700c5c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e60000c00 tx=0x7f6e60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e66ffd700 1 -- 172.21.15.181:0/1713583453 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60004480 con 0x7f6e700ce6a0 2023-10-16T09:44:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e66ffd700 1 -- 172.21.15.181:0/1713583453 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6001f070 con 0x7f6e700ce6a0 2023-10-16T09:44:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e700c6510 con 0x7f6e700ce6a0 2023-10-16T09:44:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e66ffd700 1 -- 172.21.15.181:0/1713583453 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e600038c0 con 0x7f6e700ce6a0 2023-10-16T09:44:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.928+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e700b36e0 con 0x7f6e700ce6a0 2023-10-16T09:44:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.929+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e700b3bc0 con 0x7f6e700ce6a0 2023-10-16T09:44:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.930+0000 7f6e66ffd700 1 -- 172.21.15.181:0/1713583453 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e60021030 con 0x7f6e700ce6a0 2023-10-16T09:44:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.930+0000 7f6e66ffd700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e5406af00 0x7f6e5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.930+0000 7f6e66ffd700 1 -- 172.21.15.181:0/1713583453 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6e6008ab60 con 0x7f6e700ce6a0 2023-10-16T09:44:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.930+0000 7f6e76122700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e5406af00 0x7f6e5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.931+0000 7f6e76122700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e5406af00 0x7f6e5406d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009a00 tx=0x7f6e6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:00.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:00.935+0000 7f6e66ffd700 1 -- 172.21.15.181:0/1713583453 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6e60055d80 con 0x7f6e700ce6a0 2023-10-16T09:44:01.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.169+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6e7006f6c0 con 0x7f6e700ce6a0 2023-10-16T09:44:01.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.803+0000 7f6e66ffd700 1 -- 172.21.15.181:0/1713583453 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6e600593a0 con 0x7f6e700ce6a0 2023-10-16T09:44:01.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e5406af00 msgr2=0x7f6e5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:01.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e5406af00 0x7f6e5406d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009a00 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2023-10-16T09:44:01.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 msgr2=0x7f6e700c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:01.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e700c5c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e60000c00 tx=0x7f6e60004750 comp rx=0 tx=0).stop 2023-10-16T09:44:01.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 shutdown_connections 2023-10-16T09:44:01.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e5406af00 0x7f6e5406d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e7015da70 0x7f6e700ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e70072760 0x7f6e700b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 --2- 172.21.15.181:0/1713583453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e700ce6a0 0x7f6e700c5c10 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e60000c00 tx=0x7f6e60004750 comp rx=0 tx=0).stop 2023-10-16T09:44:01.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.807+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 >> 172.21.15.181:0/1713583453 conn(0x7f6e700cd010 msgr2=0x7f6e700b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:01.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.808+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 shutdown_connections 2023-10-16T09:44:01.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.808+0000 7f6e78386700 1 -- 172.21.15.181:0/1713583453 wait complete. 2023-10-16T09:44:01.811 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 2 and data pool 3 2023-10-16T09:44:01.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:01.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.947+0000 7f4dbfd18700 1 Processor -- start 2023-10-16T09:44:01.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.947+0000 7f4dbfd18700 1 -- start start 2023-10-16T09:44:01.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbfd18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db8159730 0x7f4db815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:01.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbfd18700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4db80cc400 0x7f4db80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:01.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbfd18700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4db80cd3f0 0x7f4db806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:01.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbfd18700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db80cd880 con 0x7f4db80cd3f0 2023-10-16T09:44:01.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbfd18700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db80cee70 con 0x7f4db80cc400 2023-10-16T09:44:01.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbfd18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db8159b10 con 0x7f4db8159730 2023-10-16T09:44:01.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4db80cd3f0 0x7f4db806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:01.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4db80cd3f0 0x7f4db806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35954/0 (socket says 172.21.15.181:35954) 2023-10-16T09:44:01.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 -- 172.21.15.181:0/753892796 learned_addr learned my addr 172.21.15.181:0/753892796 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:01.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbd2b3700 1 --2- 172.21.15.181:0/753892796 >> v2:172.21.15.190:3300/0 conn(0x7f4db80cc400 0x7f4db80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:01.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbdab4700 1 --2- 172.21.15.181:0/753892796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db8159730 0x7f4db815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:01.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 -- 172.21.15.181:0/753892796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db8159730 msgr2=0x7f4db815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:01.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 --2- 172.21.15.181:0/753892796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db8159730 0x7f4db815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 -- 172.21.15.181:0/753892796 >> v2:172.21.15.190:3300/0 conn(0x7f4db80cc400 msgr2=0x7f4db80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:01.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 --2- 172.21.15.181:0/753892796 >> v2:172.21.15.190:3300/0 conn(0x7f4db80cc400 0x7f4db80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.948+0000 7f4dbe2b5700 1 -- 172.21.15.181:0/753892796 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80ce540 con 0x7f4db80cd3f0 2023-10-16T09:44:01.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbe2b5700 1 --2- 172.21.15.181:0/753892796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db806a8a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4db80705e0 tx=0x7f4db000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632046852b6c1f5e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:01.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbcab2700 1 -- 172.21.15.181:0/753892796 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db000e070 con 0x7f4db80cd3f0 2023-10-16T09:44:01.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbcab2700 1 -- 172.21.15.181:0/753892796 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db000b9c0 con 0x7f4db80cd3f0 2023-10-16T09:44:01.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbcab2700 1 -- 172.21.15.181:0/753892796 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db000bd70 con 0x7f4db80cd3f0 2023-10-16T09:44:01.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 -- 172.21.15.181:0/753892796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 msgr2=0x7f4db806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:01.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/753892796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db806a8a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4db80705e0 tx=0x7f4db000b330 comp rx=0 tx=0).stop 2023-10-16T09:44:01.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 -- 172.21.15.181:0/753892796 shutdown_connections 2023-10-16T09:44:01.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/753892796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db806a8a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/753892796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db8159730 0x7f4db815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/753892796 >> v2:172.21.15.190:3300/0 conn(0x7f4db80cc400 0x7f4db80cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 -- 172.21.15.181:0/753892796 >> 172.21.15.181:0/753892796 conn(0x7f4db80d04a0 msgr2=0x7f4db80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:01.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 -- 172.21.15.181:0/753892796 shutdown_connections 2023-10-16T09:44:01.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.949+0000 7f4dbfd18700 1 -- 172.21.15.181:0/753892796 wait complete. 2023-10-16T09:44:01.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 Processor -- start 2023-10-16T09:44:01.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 -- start start 2023-10-16T09:44:01.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db80cc400 0x7f4db80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:01.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:01.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db8159730 0x7f4db80b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db814ff70 con 0x7f4db80cd3f0 2023-10-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80b6090 con 0x7f4db8159730 2023-10-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbfd18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80b39f0 con 0x7f4db80cc400 2023-10-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbd2b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbd2b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db814fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35958/0 (socket says 172.21.15.181:35958) 2023-10-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbd2b3700 1 -- 172.21.15.181:0/3244598291 learned_addr learned my addr 172.21.15.181:0/3244598291 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:01.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbdab4700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db80cc400 0x7f4db80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:01.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.951+0000 7f4dbe2b5700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db8159730 0x7f4db80b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:01.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbd2b3700 1 -- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db80cc400 msgr2=0x7f4db80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:01.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbd2b3700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db80cc400 0x7f4db80b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbd2b3700 1 -- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db8159730 msgr2=0x7f4db80b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:01.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbd2b3700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db8159730 0x7f4db80b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:01.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbd2b3700 1 -- 172.21.15.181:0/3244598291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db0009b30 con 0x7f4db80cd3f0 2023-10-16T09:44:01.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbd2b3700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db814fa30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e810 tx=0x7f4db400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:01.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dae7fc700 1 -- 172.21.15.181:0/3244598291 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db400c070 con 0x7f4db80cd3f0 2023-10-16T09:44:01.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dae7fc700 1 -- 172.21.15.181:0/3244598291 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db4013610 con 0x7f4db80cd3f0 2023-10-16T09:44:01.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db80b3cd0 con 0x7f4db80cd3f0 2023-10-16T09:44:01.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dae7fc700 1 -- 172.21.15.181:0/3244598291 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db40139a0 con 0x7f4db80cd3f0 2023-10-16T09:44:01.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.952+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db80b4290 con 0x7f4db80cd3f0 2023-10-16T09:44:01.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.953+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db8067d20 con 0x7f4db80cd3f0 2023-10-16T09:44:01.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.953+0000 7f4dae7fc700 1 -- 172.21.15.181:0/3244598291 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4db4013b00 con 0x7f4db80cd3f0 2023-10-16T09:44:01.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.954+0000 7f4dae7fc700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:01.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.954+0000 7f4dbdab4700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:01.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.954+0000 7f4dae7fc700 1 -- 172.21.15.181:0/3244598291 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4db4089310 con 0x7f4db80cd3f0 2023-10-16T09:44:01.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.954+0000 7f4dbdab4700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006fd0 tx=0x7f4da8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:01.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:01.958+0000 7f4dae7fc700 1 -- 172.21.15.181:0/3244598291 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4db4054330 con 0x7f4db80cd3f0 2023-10-16T09:44:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.150+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4db80c3920 con 0x7f4db80cd3f0 2023-10-16T09:44:02.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.151+0000 7f4dae7fc700 1 -- 172.21.15.181:0/3244598291 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f4db4009440 con 0x7f4db80cd3f0 2023-10-16T09:44:02.153 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:02.153 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":21,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:01.170505+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:02.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.153+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d9c06af00 msgr2=0x7f4d9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006fd0 tx=0x7f4da8008040 comp rx=0 tx=0).stop 2023-10-16T09:44:02.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 msgr2=0x7f4db814fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db814fa30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4db400e810 tx=0x7f4db400eb20 comp rx=0 tx=0).stop 2023-10-16T09:44:02.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 shutdown_connections 2023-10-16T09:44:02.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db80cc400 0x7f4db80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db8159730 0x7f4db80b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 --2- 172.21.15.181:0/3244598291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db80cd3f0 0x7f4db814fa30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 >> 172.21.15.181:0/3244598291 conn(0x7f4db80d04a0 msgr2=0x7f4db81506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:02.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 shutdown_connections 2023-10-16T09:44:02.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.154+0000 7f4dbfd18700 1 -- 172.21.15.181:0/3244598291 wait complete. 2023-10-16T09:44:02.167 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:02.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.287+0000 7faca0c07700 1 Processor -- start 2023-10-16T09:44:02.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.287+0000 7faca0c07700 1 -- start start 2023-10-16T09:44:02.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7faca0c07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac9c066a70 0x7fac9c0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7faca0c07700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c15ef20 con 0x7fac9c06e560 2023-10-16T09:44:02.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7faca0c07700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c1621d0 con 0x7fac9c0665a0 2023-10-16T09:44:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7faca0c07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c161fe0 con 0x7fac9c066a70 2023-10-16T09:44:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fac9c06e560 0x7fac9c06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47042/0 (socket says 172.21.15.181:47042) 2023-10-16T09:44:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9a59c700 1 -- 172.21.15.181:0/2378862776 learned_addr learned my addr 172.21.15.181:0/2378862776 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541501151 0 0) 0x7fac9c15ef20 con 0x7fac9c06e560 2023-10-16T09:44:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac99d9b700 1 --2- 172.21.15.181:0/2378862776 >> v2:172.21.15.190:3300/0 conn(0x7fac9c066a70 0x7fac9c0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac840035b0 con 0x7fac9c06e560 2023-10-16T09:44:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4182699636 0 0) 0x7fac840035b0 con 0x7fac9c06e560 2023-10-16T09:44:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 >> v2:172.21.15.190:3300/0 conn(0x7fac9c066a70 msgr2=0x7fac9c0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 --2- 172.21.15.181:0/2378862776 >> v2:172.21.15.190:3300/0 conn(0x7fac9c066a70 0x7fac9c0cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 >> v1:172.21.15.190:6789/0 conn(0x7fac9c0665a0 legacy=0x7fac9c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c0cf270 con 0x7fac9c06e560 2023-10-16T09:44:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fac8c001670 con 0x7fac9c06e560 2023-10-16T09:44:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 >> v1:172.21.15.181:6789/0 conn(0x7fac9c06e560 legacy=0x7fac9c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac84003a00 0x7fac84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac84004850 0x7fac84004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84005210 0x7fac8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8400c080 con 0x7fac84004850 2023-10-16T09:44:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8400c1f0 con 0x7fac84005210 2023-10-16T09:44:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.288+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8400c3b0 con 0x7fac84003a00 2023-10-16T09:44:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac99d9b700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac84004850 0x7fac84004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac9a59c700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac84003a00 0x7fac84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac9ad9d700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84005210 0x7fac8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac99d9b700 1 -- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac84003a00 msgr2=0x7fac84003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac99d9b700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac84003a00 0x7fac84003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac99d9b700 1 -- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84005210 msgr2=0x7fac8400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac99d9b700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84005210 0x7fac8400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac9ad9d700 1 -- 172.21.15.181:0/2378862776 reap_dead start 2023-10-16T09:44:02.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac99d9b700 1 -- 172.21.15.181:0/2378862776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840035b0 con 0x7fac84004850 2023-10-16T09:44:02.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac99d9b700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac84004850 0x7fac84004cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac90009b60 tx=0x7fac84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dcd06ec680da64 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:02.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000e6c0 con 0x7fac84004850 2023-10-16T09:44:02.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9000e820 con 0x7fac84004850 2023-10-16T09:44:02.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.289+0000 7fac9959a700 1 -- 172.21.15.181:0/2378862776 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000eb90 con 0x7fac84004850 2023-10-16T09:44:02.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.290+0000 7faca0c07700 1 -- 172.21.15.181:0/2378862776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac84004850 msgr2=0x7fac84004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.290+0000 7faca0c07700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac84004850 0x7fac84004cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac90009b60 tx=0x7fac84005880 comp rx=0 tx=0).stop 2023-10-16T09:44:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.290+0000 7faca0c07700 1 -- 172.21.15.181:0/2378862776 shutdown_connections 2023-10-16T09:44:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.290+0000 7faca0c07700 1 --2- 172.21.15.181:0/2378862776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac84004850 0x7fac84004cd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.290+0000 7faca0c07700 1 -- 172.21.15.181:0/2378862776 >> 172.21.15.181:0/2378862776 conn(0x7fac9c0b16e0 msgr2=0x7fac9c0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.290+0000 7faca0c07700 1 -- 172.21.15.181:0/2378862776 shutdown_connections 2023-10-16T09:44:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.290+0000 7faca0c07700 1 -- 172.21.15.181:0/2378862776 wait complete. 2023-10-16T09:44:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.291+0000 7faca0c07700 1 Processor -- start 2023-10-16T09:44:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.291+0000 7faca0c07700 1 -- start start 2023-10-16T09:44:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84004850 0x7fac9c0a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c1f4210 0x7fac9c09de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9c0c55d0 0x7fac9c1f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c09e350 con 0x7fac9c1f4210 2023-10-16T09:44:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0a09a0 con 0x7fac84004850 2023-10-16T09:44:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c15b5a0 con 0x7fac9c0c55d0 2023-10-16T09:44:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c1f4210 0x7fac9c09de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c1f4210 0x7fac9c09de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35978/0 (socket says 172.21.15.181:35978) 2023-10-16T09:44:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 -- 172.21.15.181:0/108308747 learned_addr learned my addr 172.21.15.181:0/108308747 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 -- 172.21.15.181:0/108308747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9c0c55d0 msgr2=0x7fac9c1f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9c0c55d0 0x7fac9c1f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 -- 172.21.15.181:0/108308747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84004850 msgr2=0x7fac9c0a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84004850 0x7fac9c0a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 -- 172.21.15.181:0/108308747 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c15b880 con 0x7fac9c1f4210 2023-10-16T09:44:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac99d9b700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c1f4210 0x7fac9c09de10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c1c0 tx=0x7fac900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac8affd700 1 -- 172.21.15.181:0/108308747 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000e6c0 con 0x7fac9c1f4210 2023-10-16T09:44:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac8affd700 1 -- 172.21.15.181:0/108308747 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac90003ad0 con 0x7fac9c1f4210 2023-10-16T09:44:02.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c15bb60 con 0x7fac9c1f4210 2023-10-16T09:44:02.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7fac8affd700 1 -- 172.21.15.181:0/108308747 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac900045d0 con 0x7fac9c1f4210 2023-10-16T09:44:02.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.292+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c0c5a80 con 0x7fac9c1f4210 2023-10-16T09:44:02.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.293+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c067d20 con 0x7fac9c1f4210 2023-10-16T09:44:02.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.294+0000 7fac8affd700 1 -- 172.21.15.181:0/108308747 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac90013470 con 0x7fac9c1f4210 2023-10-16T09:44:02.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.294+0000 7fac8affd700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac7c06af00 0x7fac7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.294+0000 7fac8affd700 1 -- 172.21.15.181:0/108308747 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac9008ad90 con 0x7fac9c1f4210 2023-10-16T09:44:02.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.294+0000 7fac9a59c700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac7c06af00 0x7fac7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.294+0000 7fac9a59c700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac7c06af00 0x7fac7c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001050 tx=0x7fac8c001b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:02.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.298+0000 7fac8affd700 1 -- 172.21.15.181:0/108308747 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac90055ed0 con 0x7fac9c1f4210 2023-10-16T09:44:02.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.490+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac9c15bcf0 con 0x7fac9c1f4210 2023-10-16T09:44:02.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.492+0000 7fac8affd700 1 -- 172.21.15.181:0/108308747 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16554 (secure 0 0 0) 0x7fac90017020 con 0x7fac9c1f4210 2023-10-16T09:44:02.547 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:02.547 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":21,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:01.170505+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:02.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac7c06af00 msgr2=0x7fac7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac7c06af00 0x7fac7c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001050 tx=0x7fac8c001b20 comp rx=0 tx=0).stop 2023-10-16T09:44:02.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c1f4210 msgr2=0x7fac9c09de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c1f4210 0x7fac9c09de10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c1c0 tx=0x7fac900047c0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 shutdown_connections 2023-10-16T09:44:02.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac7c06af00 0x7fac7c06d3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9c0c55d0 0x7fac9c1f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84004850 0x7fac9c0a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 --2- 172.21.15.181:0/108308747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac9c1f4210 0x7fac9c09de10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 >> 172.21.15.181:0/108308747 conn(0x7fac9c0b16e0 msgr2=0x7fac9c0ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:02.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 shutdown_connections 2023-10-16T09:44:02.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.494+0000 7faca0c07700 1 -- 172.21.15.181:0/108308747 wait complete. 2023-10-16T09:44:02.552 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T09:44:02.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.673+0000 7f416a6c1700 1 Processor -- start 2023-10-16T09:44:02.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.673+0000 7f416a6c1700 1 -- start start 2023-10-16T09:44:02.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f416a6c1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb2b0 0x7f416406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f416a6c1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41640cb690 0x7f41640b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f416a6c1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4164070590 con 0x7f41640b2ba0 2023-10-16T09:44:02.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f416a6c1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4164156330 con 0x7f41640cb690 2023-10-16T09:44:02.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f416a6c1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416406fcf0 con 0x7f41640cb2b0 2023-10-16T09:44:02.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f41637fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41640cb690 0x7f41640b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f41637fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41640cb690 0x7f41640b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35986/0 (socket says 172.21.15.181:35986) 2023-10-16T09:44:02.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f4168c5e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f41640b2ba0 0x7f416415f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47066/0 (socket says 172.21.15.181:47066) 2023-10-16T09:44:02.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.674+0000 7f41637fe700 1 -- 172.21.15.181:0/235742662 learned_addr learned my addr 172.21.15.181:0/235742662 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:02.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f4163fff700 1 --2- 172.21.15.181:0/235742662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb2b0 0x7f416406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f41637fe700 1 -- 172.21.15.181:0/235742662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb2b0 msgr2=0x7f416406f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:02.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f41637fe700 1 --2- 172.21.15.181:0/235742662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb2b0 0x7f416406f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f41637fe700 1 -- 172.21.15.181:0/235742662 >> v1:172.21.15.181:6789/0 conn(0x7f41640b2ba0 legacy=0x7f416415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f41637fe700 1 -- 172.21.15.181:0/235742662 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416406ffd0 con 0x7f41640cb690 2023-10-16T09:44:02.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f4162ffd700 1 -- 172.21.15.181:0/235742662 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624033247 0 0) 0x7f4164070590 con 0x7f41640b2ba0 2023-10-16T09:44:02.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f41637fe700 1 --2- 172.21.15.181:0/235742662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb690 0x7f41640b2660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f41540098f0 tx=0x7f4154009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f484cb80a28f893 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:02.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f4162ffd700 1 -- 172.21.15.181:0/235742662 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400b8a0 con 0x7f41640cb690 2023-10-16T09:44:02.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f4162ffd700 1 -- 172.21.15.181:0/235742662 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415400ba00 con 0x7f41640cb690 2023-10-16T09:44:02.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f4162ffd700 1 -- 172.21.15.181:0/235742662 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400bd90 con 0x7f41640cb690 2023-10-16T09:44:02.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f416a6c1700 1 -- 172.21.15.181:0/235742662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb690 msgr2=0x7f41640b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f416a6c1700 1 --2- 172.21.15.181:0/235742662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb690 0x7f41640b2660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f41540098f0 tx=0x7f4154009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:02.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f416a6c1700 1 -- 172.21.15.181:0/235742662 shutdown_connections 2023-10-16T09:44:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f416a6c1700 1 --2- 172.21.15.181:0/235742662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb2b0 0x7f416406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f416a6c1700 1 --2- 172.21.15.181:0/235742662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb690 0x7f41640b2660 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.675+0000 7f416a6c1700 1 -- 172.21.15.181:0/235742662 >> 172.21.15.181:0/235742662 conn(0x7f41640cd010 msgr2=0x7f41640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.676+0000 7f416a6c1700 1 -- 172.21.15.181:0/235742662 shutdown_connections 2023-10-16T09:44:02.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.676+0000 7f416a6c1700 1 -- 172.21.15.181:0/235742662 wait complete. 2023-10-16T09:44:02.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.677+0000 7f416a6c1700 1 Processor -- start 2023-10-16T09:44:02.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.677+0000 7f416a6c1700 1 -- start start 2023-10-16T09:44:02.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41640b2ba0 0x7f4164152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb2b0 0x7f41640c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb690 0x7f41640ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640c85b0 con 0x7f41640cb2b0 2023-10-16T09:44:02.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640c8a40 con 0x7f41640b2ba0 2023-10-16T09:44:02.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41640c79a0 con 0x7f41640cb690 2023-10-16T09:44:02.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb2b0 0x7f41640c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb2b0 0x7f41640c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:35992/0 (socket says 172.21.15.181:35992) 2023-10-16T09:44:02.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 -- 172.21.15.181:0/3263255156 learned_addr learned my addr 172.21.15.181:0/3263255156 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:02.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f4168c5e700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb690 0x7f41640ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f4163fff700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41640b2ba0 0x7f4164152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 -- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb690 msgr2=0x7f41640ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb690 0x7f41640ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 -- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41640b2ba0 msgr2=0x7f4164152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:02.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41640b2ba0 0x7f4164152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:02.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 -- 172.21.15.181:0/3263255156 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154009550 con 0x7f41640cb2b0 2023-10-16T09:44:02.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f4163fff700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41640b2ba0 0x7f4164152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:02.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f41637fe700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb2b0 0x7f41640c9ee0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f4154004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:02.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f4160ff9700 1 -- 172.21.15.181:0/3263255156 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540036a0 con 0x7f41640cb2b0 2023-10-16T09:44:02.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f4160ff9700 1 -- 172.21.15.181:0/3263255156 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415401f070 con 0x7f41640cb2b0 2023-10-16T09:44:02.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41640c7c80 con 0x7f41640cb2b0 2023-10-16T09:44:02.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f4160ff9700 1 -- 172.21.15.181:0/3263255156 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154003a40 con 0x7f41640cb2b0 2023-10-16T09:44:02.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.678+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41640c8240 con 0x7f41640cb2b0 2023-10-16T09:44:02.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.679+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41640c83d0 con 0x7f41640cb2b0 2023-10-16T09:44:02.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.680+0000 7f4160ff9700 1 -- 172.21.15.181:0/3263255156 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4154003ba0 con 0x7f41640cb2b0 2023-10-16T09:44:02.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.680+0000 7f4160ff9700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f415006af00 0x7f415006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:02.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.680+0000 7f4160ff9700 1 -- 172.21.15.181:0/3263255156 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4154021070 con 0x7f41640cb2b0 2023-10-16T09:44:02.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.680+0000 7f4163fff700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f415006af00 0x7f415006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:02.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.680+0000 7f4163fff700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f415006af00 0x7f415006d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4158006fd0 tx=0x7f4158008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:02.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.684+0000 7f4160ff9700 1 -- 172.21.15.181:0/3263255156 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4154056000 con 0x7f41640cb2b0 2023-10-16T09:44:02.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:02.927+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f41640c86f0 con 0x7f41640cb2b0 2023-10-16T09:44:03.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.818+0000 7f4160ff9700 1 -- 172.21.15.181:0/3263255156 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4154010380 con 0x7f41640cb2b0 2023-10-16T09:44:03.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.821+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f415006af00 msgr2=0x7f415006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:03.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.821+0000 7f416a6c1700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f415006af00 0x7f415006d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4158006fd0 tx=0x7f4158008040 comp rx=0 tx=0).stop 2023-10-16T09:44:03.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.821+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb2b0 msgr2=0x7f41640c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:03.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.821+0000 7f416a6c1700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb2b0 0x7f41640c9ee0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c00 tx=0x7f4154004750 comp rx=0 tx=0).stop 2023-10-16T09:44:03.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 shutdown_connections 2023-10-16T09:44:03.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f415006af00 0x7f415006d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41640cb690 0x7f41640ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41640b2ba0 0x7f4164152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 --2- 172.21.15.181:0/3263255156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41640cb2b0 0x7f41640c9ee0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 >> 172.21.15.181:0/3263255156 conn(0x7f41640cd010 msgr2=0x7f41640d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:03.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 shutdown_connections 2023-10-16T09:44:03.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.822+0000 7f416a6c1700 1 -- 172.21.15.181:0/3263255156 wait complete. 2023-10-16T09:44:03.837 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T09:44:03.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T09:44:03.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.966+0000 7fac8afc5700 1 Processor -- start 2023-10-16T09:44:03.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.966+0000 7fac8afc5700 1 -- start start 2023-10-16T09:44:03.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac8afc5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 0x7fac84071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac8afc5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac840ccb60 con 0x7fac840b2010 2023-10-16T09:44:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac8afc5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac840ce1f0 con 0x7fac840ce680 2023-10-16T09:44:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac8afc5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac84072120 con 0x7fac840cc6e0 2023-10-16T09:44:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac88d61700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fac840b2010 0x7fac840b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47076/0 (socket says 172.21.15.181:47076) 2023-10-16T09:44:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac88d61700 1 -- 172.21.15.181:0/2608639333 learned_addr learned my addr 172.21.15.181:0/2608639333 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:03.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac83fff700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 0x7fac84071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166201813 0 0) 0x7fac78001110 con 0x7fac840b2010 2023-10-16T09:44:03.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac740035b0 con 0x7fac840b2010 2023-10-16T09:44:03.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368918141 0 0) 0x7fac840ce1f0 con 0x7fac840ce680 2023-10-16T09:44:03.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac78001110 con 0x7fac840ce680 2023-10-16T09:44:03.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387073716 0 0) 0x7fac740035b0 con 0x7fac840b2010 2023-10-16T09:44:03.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 msgr2=0x7fac84071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:03.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 0x7fac84071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 >> v1:172.21.15.190:6789/0 conn(0x7fac840ce680 legacy=0x7fac840ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:03.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840ccda0 con 0x7fac840b2010 2023-10-16T09:44:03.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.967+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fac78001800 con 0x7fac840b2010 2023-10-16T09:44:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 >> v1:172.21.15.181:6789/0 conn(0x7fac840b2010 legacy=0x7fac840b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003bc0 0x7fac74003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac74004850 0x7fac74004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 0x7fac74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac84072120 con 0x7fac74004850 2023-10-16T09:44:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7400bac0 con 0x7fac74003bc0 2023-10-16T09:44:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7400bc50 con 0x7fac840cc6e0 2023-10-16T09:44:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac83fff700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac74004850 0x7fac74004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac88d61700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003bc0 0x7fac74003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac89562700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 0x7fac74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac83fff700 1 -- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 msgr2=0x7fac74005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac83fff700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac840cc6e0 0x7fac74005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac83fff700 1 -- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003bc0 msgr2=0x7fac74003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac83fff700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003bc0 0x7fac74003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac83fff700 1 -- 172.21.15.181:0/2608639333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740035b0 con 0x7fac74004850 2023-10-16T09:44:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac83fff700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac74004850 0x7fac74004cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac70009b60 tx=0x7fac74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e86fb3f45ad008 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7000e6c0 con 0x7fac74004850 2023-10-16T09:44:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7000e820 con 0x7fac74004850 2023-10-16T09:44:03.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.968+0000 7fac837fe700 1 -- 172.21.15.181:0/2608639333 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7000eb90 con 0x7fac74004850 2023-10-16T09:44:03.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.969+0000 7fac8afc5700 1 -- 172.21.15.181:0/2608639333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac74004850 msgr2=0x7fac74004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:03.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.969+0000 7fac8afc5700 1 --2- 172.21.15.181:0/2608639333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac74004850 0x7fac74004cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac70009b60 tx=0x7fac74005880 comp rx=0 tx=0).stop 2023-10-16T09:44:03.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.969+0000 7fac89562700 1 -- 172.21.15.181:0/2608639333 reap_dead start 2023-10-16T09:44:03.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.969+0000 7fac8afc5700 1 -- 172.21.15.181:0/2608639333 shutdown_connections 2023-10-16T09:44:03.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.969+0000 7fac8afc5700 1 -- 172.21.15.181:0/2608639333 >> 172.21.15.181:0/2608639333 conn(0x7fac840cd010 msgr2=0x7fac840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:03.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.969+0000 7fac8afc5700 1 -- 172.21.15.181:0/2608639333 shutdown_connections 2023-10-16T09:44:03.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.969+0000 7fac8afc5700 1 -- 172.21.15.181:0/2608639333 wait complete. 2023-10-16T09:44:03.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.970+0000 7fac8afc5700 1 Processor -- start 2023-10-16T09:44:03.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.970+0000 7fac8afc5700 1 -- start start 2023-10-16T09:44:03.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac8afc5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004850 0x7fac84143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac8afc5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac840c60a0 0x7fac840b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac8afc5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac840b1080 0x7fac8406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac8afc5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac840b1500 con 0x7fac840c60a0 2023-10-16T09:44:03.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac8afc5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac84144480 con 0x7fac840b1080 2023-10-16T09:44:03.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac8afc5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac841f49b0 con 0x7fac74004850 2023-10-16T09:44:03.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac840c60a0 0x7fac840b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac840c60a0 0x7fac840b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36012/0 (socket says 172.21.15.181:36012) 2023-10-16T09:44:03.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 -- 172.21.15.181:0/3544486911 learned_addr learned my addr 172.21.15.181:0/3544486911 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:03.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac89562700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac840b1080 0x7fac8406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac88d61700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004850 0x7fac84143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 -- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004850 msgr2=0x7fac84143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:03.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004850 0x7fac84143f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 -- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac840b1080 msgr2=0x7fac8406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:03.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac840b1080 0x7fac8406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:03.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 -- 172.21.15.181:0/3544486911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac841f4c90 con 0x7fac840c60a0 2023-10-16T09:44:03.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac89562700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac840b1080 0x7fac8406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:03.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac83fff700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac840c60a0 0x7fac840b0b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e4b0 tx=0x7fac700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:03.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.971+0000 7fac817fa700 1 -- 172.21.15.181:0/3544486911 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70021070 con 0x7fac840c60a0 2023-10-16T09:44:03.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.972+0000 7fac817fa700 1 -- 172.21.15.181:0/3544486911 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac70004590 con 0x7fac840c60a0 2023-10-16T09:44:03.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.972+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac841f4f70 con 0x7fac840c60a0 2023-10-16T09:44:03.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.972+0000 7fac817fa700 1 -- 172.21.15.181:0/3544486911 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7000e6c0 con 0x7fac840c60a0 2023-10-16T09:44:03.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.972+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac841f5530 con 0x7fac840c60a0 2023-10-16T09:44:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.973+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac840b1460 con 0x7fac840c60a0 2023-10-16T09:44:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.973+0000 7fac817fa700 1 -- 172.21.15.181:0/3544486911 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac70025070 con 0x7fac840c60a0 2023-10-16T09:44:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.973+0000 7fac817fa700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac6806af00 0x7fac6806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:03.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.973+0000 7fac817fa700 1 -- 172.21.15.181:0/3544486911 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac7008a700 con 0x7fac840c60a0 2023-10-16T09:44:03.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.973+0000 7fac88d61700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac6806af00 0x7fac6806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:03.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.974+0000 7fac88d61700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac6806af00 0x7fac6806d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac78001050 tx=0x7fac78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:03.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:03.978+0000 7fac817fa700 1 -- 172.21.15.181:0/3544486911 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac70055960 con 0x7fac840c60a0 2023-10-16T09:44:04.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.215+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fac84154f10 con 0x7fac840c60a0 2023-10-16T09:44:04.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.819+0000 7fac817fa700 1 -- 172.21.15.181:0/3544486911 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fac70058f80 con 0x7fac840c60a0 2023-10-16T09:44:04.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac6806af00 msgr2=0x7fac6806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:04.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac6806af00 0x7fac6806d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac78001050 tx=0x7fac78009040 comp rx=0 tx=0).stop 2023-10-16T09:44:04.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac840c60a0 msgr2=0x7fac840b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:04.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac840c60a0 0x7fac840b0b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e4b0 tx=0x7fac700047c0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 shutdown_connections 2023-10-16T09:44:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fac6806af00 0x7fac6806d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004850 0x7fac84143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac840b1080 0x7fac8406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 --2- 172.21.15.181:0/3544486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac840c60a0 0x7fac840b0b40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 >> 172.21.15.181:0/3544486911 conn(0x7fac840cd010 msgr2=0x7fac840cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:04.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 shutdown_connections 2023-10-16T09:44:04.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.822+0000 7fac8afc5700 1 -- 172.21.15.181:0/3544486911 wait complete. 2023-10-16T09:44:04.838 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T09:44:04.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:04.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.962+0000 7f40b0075700 1 Processor -- start 2023-10-16T09:44:04.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.963+0000 7f40b0075700 1 -- start start 2023-10-16T09:44:04.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.963+0000 7f40b0075700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40a80ce7f0 0x7f40a806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.963+0000 7f40b0075700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 0x7f40a80cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.963+0000 7f40b0075700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a80cc470 con 0x7f40a80d0180 2023-10-16T09:44:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.963+0000 7f40b0075700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a80d0610 con 0x7f40a80ce7f0 2023-10-16T09:44:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.963+0000 7f40b0075700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a806ca30 con 0x7f40a80d2390 2023-10-16T09:44:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40a80ce7f0 0x7f40a806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40a80ce7f0 0x7f40a806c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36018/0 (socket says 172.21.15.181:36018) 2023-10-16T09:44:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ae612700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f40a80d0180 0x7f40a8161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47100/0 (socket says 172.21.15.181:47100) 2023-10-16T09:44:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 -- 172.21.15.181:0/934160209 learned_addr learned my addr 172.21.15.181:0/934160209 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ad610700 1 --2- 172.21.15.181:0/934160209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 0x7f40a80cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 -- 172.21.15.181:0/934160209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 msgr2=0x7f40a80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 --2- 172.21.15.181:0/934160209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 0x7f40a80cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 -- 172.21.15.181:0/934160209 >> v1:172.21.15.181:6789/0 conn(0x7f40a80d0180 legacy=0x7f40a8161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 -- 172.21.15.181:0/934160209 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a806cd10 con 0x7f40a80ce7f0 2023-10-16T09:44:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ace0f700 1 -- 172.21.15.181:0/934160209 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003197069 0 0) 0x7f40a80cc470 con 0x7f40a80d0180 2023-10-16T09:44:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ade11700 1 --2- 172.21.15.181:0/934160209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a806c100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4098006fd0 tx=0x7f4098009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4807ce1850b92b6b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ace0f700 1 -- 172.21.15.181:0/934160209 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098005e30 con 0x7f40a80ce7f0 2023-10-16T09:44:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ace0f700 1 -- 172.21.15.181:0/934160209 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098004030 con 0x7f40a80ce7f0 2023-10-16T09:44:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.964+0000 7f40ace0f700 1 -- 172.21.15.181:0/934160209 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40980043e0 con 0x7f40a80ce7f0 2023-10-16T09:44:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 -- 172.21.15.181:0/934160209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 msgr2=0x7f40a806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 --2- 172.21.15.181:0/934160209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a806c100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4098006fd0 tx=0x7f4098009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 -- 172.21.15.181:0/934160209 shutdown_connections 2023-10-16T09:44:04.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 --2- 172.21.15.181:0/934160209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a806c100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 --2- 172.21.15.181:0/934160209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 0x7f40a80cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 -- 172.21.15.181:0/934160209 >> 172.21.15.181:0/934160209 conn(0x7f40a809a9a0 msgr2=0x7f40a80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:04.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 -- 172.21.15.181:0/934160209 shutdown_connections 2023-10-16T09:44:04.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.965+0000 7f40b0075700 1 -- 172.21.15.181:0/934160209 wait complete. 2023-10-16T09:44:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.966+0000 7f40b0075700 1 Processor -- start 2023-10-16T09:44:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.966+0000 7f40b0075700 1 -- start start 2023-10-16T09:44:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a8155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a80d0180 0x7f40a809eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 0x7f40a80c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a809e810 con 0x7f40a80ce7f0 2023-10-16T09:44:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a809dee0 con 0x7f40a80d0180 2023-10-16T09:44:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a809e1c0 con 0x7f40a80d2390 2023-10-16T09:44:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a8155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a8155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36024/0 (socket says 172.21.15.181:36024) 2023-10-16T09:44:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 -- 172.21.15.181:0/267898732 learned_addr learned my addr 172.21.15.181:0/267898732 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 -- 172.21.15.181:0/267898732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 msgr2=0x7f40a80c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 0x7f40a80c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 -- 172.21.15.181:0/267898732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a80d0180 msgr2=0x7f40a809eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a80d0180 0x7f40a809eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 -- 172.21.15.181:0/267898732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098009500 con 0x7f40a80ce7f0 2023-10-16T09:44:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40ade11700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a8155ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4098009c10 tx=0x7f4098004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f409e7fc700 1 -- 172.21.15.181:0/267898732 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098003dd0 con 0x7f40a80ce7f0 2023-10-16T09:44:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f409e7fc700 1 -- 172.21.15.181:0/267898732 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409801c420 con 0x7f40a80ce7f0 2023-10-16T09:44:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a809e4a0 con 0x7f40a80ce7f0 2023-10-16T09:44:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f409e7fc700 1 -- 172.21.15.181:0/267898732 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409801c7d0 con 0x7f40a80ce7f0 2023-10-16T09:44:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.967+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a809db20 con 0x7f40a80ce7f0 2023-10-16T09:44:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.968+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a8067d20 con 0x7f40a80ce7f0 2023-10-16T09:44:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.969+0000 7f409e7fc700 1 -- 172.21.15.181:0/267898732 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f409801c930 con 0x7f40a80ce7f0 2023-10-16T09:44:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.969+0000 7f409e7fc700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f408c06af00 0x7f408c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.969+0000 7f409e7fc700 1 -- 172.21.15.181:0/267898732 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f409808a670 con 0x7f40a80ce7f0 2023-10-16T09:44:04.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.969+0000 7f40ad610700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f408c06af00 0x7f408c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:04.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.970+0000 7f40ad610700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f408c06af00 0x7f408c06d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a4006b20 tx=0x7f40a4006ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:04.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:04.974+0000 7f409e7fc700 1 -- 172.21.15.181:0/267898732 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f40980557b0 con 0x7f40a80ce7f0 2023-10-16T09:44:05.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.207+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40a80cef40 con 0x7f40a80ce7f0 2023-10-16T09:44:05.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.207+0000 7f409e7fc700 1 -- 172.21.15.181:0/267898732 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f4098058dd0 con 0x7f40a80ce7f0 2023-10-16T09:44:05.209 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:05.209 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:05.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f408c06af00 msgr2=0x7f408c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f408c06af00 0x7f408c06d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a4006b20 tx=0x7f40a4006ab0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 msgr2=0x7f40a8155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a8155ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4098009c10 tx=0x7f4098004750 comp rx=0 tx=0).stop 2023-10-16T09:44:05.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 shutdown_connections 2023-10-16T09:44:05.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f408c06af00 0x7f408c06d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a80d2390 0x7f40a80c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a80ce7f0 0x7f40a8155ab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 --2- 172.21.15.181:0/267898732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a80d0180 0x7f40a809eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 >> 172.21.15.181:0/267898732 conn(0x7f40a809a9a0 msgr2=0x7f40a80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:05.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.209+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 shutdown_connections 2023-10-16T09:44:05.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.210+0000 7f40b0075700 1 -- 172.21.15.181:0/267898732 wait complete. 2023-10-16T09:44:05.216 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:05.223 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:05.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.342+0000 7f2cfe847700 1 Processor -- start 2023-10-16T09:44:05.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.342+0000 7f2cfe847700 1 -- start start 2023-10-16T09:44:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cfe847700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2cf815eab0 0x7f2cf80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cfe847700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 0x7f2cf815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cfe847700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf806ed00 con 0x7f2cf815e6d0 2023-10-16T09:44:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cfe847700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf80d02d0 con 0x7f2cf815eab0 2023-10-16T09:44:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cfe847700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf815ad40 con 0x7f2cf80cb0c0 2023-10-16T09:44:05.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2cf815e6d0 0x7f2cf815a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47110/0 (socket says 172.21.15.181:47110) 2023-10-16T09:44:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf7fff700 1 -- 172.21.15.181:0/1377730175 learned_addr learned my addr 172.21.15.181:0/1377730175 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf77fe700 1 --2- 172.21.15.181:0/1377730175 >> v2:172.21.15.190:3300/0 conn(0x7f2cf815eab0 0x7f2cf80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cfcde4700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 0x7f2cf815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039882111 0 0) 0x7f2cf806ed00 con 0x7f2cf815e6d0 2023-10-16T09:44:05.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce00035b0 con 0x7f2cf815e6d0 2023-10-16T09:44:05.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441454295 0 0) 0x7f2ce00035b0 con 0x7f2cf815e6d0 2023-10-16T09:44:05.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 msgr2=0x7f2cf815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 0x7f2cf815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 >> v2:172.21.15.190:3300/0 conn(0x7f2cf815eab0 msgr2=0x7f2cf80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 --2- 172.21.15.181:0/1377730175 >> v2:172.21.15.190:3300/0 conn(0x7f2cf815eab0 0x7f2cf80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf806e0f0 con 0x7f2cf815e6d0 2023-10-16T09:44:05.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f2ce80017f0 con 0x7f2cf815e6d0 2023-10-16T09:44:05.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 >> v1:172.21.15.181:6789/0 conn(0x7f2cf815e6d0 legacy=0x7f2cf815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 0x7f2ce0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce0003f80 0x7f2ce0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce0004d90 0x7f2ce0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80d02d0 con 0x7f2ce0004d90 2023-10-16T09:44:05.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf815ad40 con 0x7f2ce0003f80 2023-10-16T09:44:05.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.344+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce000ba10 con 0x7f2cf80cb0c0 2023-10-16T09:44:05.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cfcde4700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce0004d90 0x7f2ce0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cf7fff700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 0x7f2ce0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cf77fe700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce0003f80 0x7f2ce0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cfcde4700 1 -- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 msgr2=0x7f2ce0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cfcde4700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf80cb0c0 0x7f2ce0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cfcde4700 1 -- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce0003f80 msgr2=0x7f2ce0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cfcde4700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ce0003f80 0x7f2ce0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cfcde4700 1 -- 172.21.15.181:0/1377730175 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce00035b0 con 0x7f2ce0004d90 2023-10-16T09:44:05.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cfcde4700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce0004d90 0x7f2ce0005210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009fc0 tx=0x7f2ce0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd8361c6431f2a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:05.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf000ebd0 con 0x7f2ce0004d90 2023-10-16T09:44:05.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf000f040 con 0x7f2ce0004d90 2023-10-16T09:44:05.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.345+0000 7f2cf6ffd700 1 -- 172.21.15.181:0/1377730175 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0014480 con 0x7f2ce0004d90 2023-10-16T09:44:05.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.346+0000 7f2cfe847700 1 -- 172.21.15.181:0/1377730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce0004d90 msgr2=0x7f2ce0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.346+0000 7f2cfe847700 1 --2- 172.21.15.181:0/1377730175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce0004d90 0x7f2ce0005210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009fc0 tx=0x7f2ce0005880 comp rx=0 tx=0).stop 2023-10-16T09:44:05.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.346+0000 7f2cfcde4700 1 -- 172.21.15.181:0/1377730175 reap_dead start 2023-10-16T09:44:05.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.346+0000 7f2cfe847700 1 -- 172.21.15.181:0/1377730175 shutdown_connections 2023-10-16T09:44:05.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.346+0000 7f2cfe847700 1 -- 172.21.15.181:0/1377730175 >> 172.21.15.181:0/1377730175 conn(0x7f2cf80d04a0 msgr2=0x7f2cf80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:05.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.346+0000 7f2cfe847700 1 -- 172.21.15.181:0/1377730175 shutdown_connections 2023-10-16T09:44:05.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.346+0000 7f2cfe847700 1 -- 172.21.15.181:0/1377730175 wait complete. 2023-10-16T09:44:05.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.347+0000 7f2cfe847700 1 Processor -- start 2023-10-16T09:44:05.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.347+0000 7f2cfe847700 1 -- start start 2023-10-16T09:44:05.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cf81504f0 0x7f2cf80af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf80b5300 0x7f2cf80b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf815baa0 0x7f2cf815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80c9360 con 0x7f2cf81504f0 2023-10-16T09:44:05.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80b5cc0 con 0x7f2cf80b5300 2023-10-16T09:44:05.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf815c6f0 con 0x7f2cf815baa0 2023-10-16T09:44:05.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cf81504f0 0x7f2cf80af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cf81504f0 0x7f2cf80af980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36044/0 (socket says 172.21.15.181:36044) 2023-10-16T09:44:05.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 -- 172.21.15.181:0/954802759 learned_addr learned my addr 172.21.15.181:0/954802759 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:05.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf77fe700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf80b5300 0x7f2cf80b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfcde4700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf815baa0 0x7f2cf815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 -- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf815baa0 msgr2=0x7f2cf815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf815baa0 0x7f2cf815bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 -- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf80b5300 msgr2=0x7f2cf80b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf80b5300 0x7f2cf80b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 -- 172.21.15.181:0/954802759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80afec0 con 0x7f2cf81504f0 2023-10-16T09:44:05.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf7fff700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cf81504f0 0x7f2cf80af980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8001130 tx=0x7f2ce800fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:05.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf4ff9700 1 -- 172.21.15.181:0/954802759 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80073a0 con 0x7f2cf81504f0 2023-10-16T09:44:05.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf4ff9700 1 -- 172.21.15.181:0/954802759 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce8007520 con 0x7f2cf81504f0 2023-10-16T09:44:05.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf815c460 con 0x7f2cf81504f0 2023-10-16T09:44:05.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cf4ff9700 1 -- 172.21.15.181:0/954802759 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8005a70 con 0x7f2cf81504f0 2023-10-16T09:44:05.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.348+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf80c8ca0 con 0x7f2cf81504f0 2023-10-16T09:44:05.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.349+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf80c94a0 con 0x7f2cf81504f0 2023-10-16T09:44:05.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.350+0000 7f2cf4ff9700 1 -- 172.21.15.181:0/954802759 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2ce8005bd0 con 0x7f2cf81504f0 2023-10-16T09:44:05.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.350+0000 7f2cf4ff9700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2ce406af30 0x7f2ce406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.350+0000 7f2cf4ff9700 1 -- 172.21.15.181:0/954802759 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ce80897e0 con 0x7f2cf81504f0 2023-10-16T09:44:05.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.350+0000 7f2cf77fe700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2ce406af30 0x7f2ce406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.351+0000 7f2cf77fe700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2ce406af30 0x7f2ce406d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2cf815ab40 tx=0x7f2cec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:05.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.355+0000 7f2cf4ff9700 1 -- 172.21.15.181:0/954802759 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2cf80c94a0 con 0x7f2cf81504f0 2023-10-16T09:44:05.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.549+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cf80c94a0 con 0x7f2cf81504f0 2023-10-16T09:44:05.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.550+0000 7f2cf4ff9700 1 -- 172.21.15.181:0/954802759 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f2cf80c94a0 con 0x7f2cf81504f0 2023-10-16T09:44:05.551 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:05.551 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:05.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2ce406af30 msgr2=0x7f2ce406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2ce406af30 0x7f2ce406d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2cf815ab40 tx=0x7f2cec008040 comp rx=0 tx=0).stop 2023-10-16T09:44:05.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cf81504f0 msgr2=0x7f2cf80af980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cf81504f0 0x7f2cf80af980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8001130 tx=0x7f2ce800fe80 comp rx=0 tx=0).stop 2023-10-16T09:44:05.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 shutdown_connections 2023-10-16T09:44:05.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2ce406af30 0x7f2ce406d3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cf815baa0 0x7f2cf815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cf81504f0 0x7f2cf80af980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 --2- 172.21.15.181:0/954802759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cf80b5300 0x7f2cf80b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 >> 172.21.15.181:0/954802759 conn(0x7f2cf80d04a0 msgr2=0x7f2cf80ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:05.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 shutdown_connections 2023-10-16T09:44:05.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.552+0000 7f2cfe847700 1 -- 172.21.15.181:0/954802759 wait complete. 2023-10-16T09:44:05.566 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T09:44:05.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:44:05.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.685+0000 7f85bc31d700 1 Processor -- start 2023-10-16T09:44:05.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.685+0000 7f85bc31d700 1 -- start start 2023-10-16T09:44:05.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.686+0000 7f85bc31d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 0x7f85b40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.686+0000 7f85bc31d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f85b40cb0c0 0x7f85b415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.686+0000 7f85bc31d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b406ed00 con 0x7f85b415e6d0 2023-10-16T09:44:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.686+0000 7f85bc31d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b40d02d0 con 0x7f85b40cb0c0 2023-10-16T09:44:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.686+0000 7f85bc31d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b415ad40 con 0x7f85b415eab0 2023-10-16T09:44:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.686+0000 7f85ba0b9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f85b415e6d0 0x7f85b415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47130/0 (socket says 172.21.15.181:47130) 2023-10-16T09:44:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba0b9700 1 -- 172.21.15.181:0/2941668819 learned_addr learned my addr 172.21.15.181:0/2941668819 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b98b8700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 0x7f85b40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba8ba700 1 --2- 172.21.15.181:0/2941668819 >> v2:172.21.15.190:3300/0 conn(0x7f85b40cb0c0 0x7f85b415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389597320 0 0) 0x7f85b406ed00 con 0x7f85b415e6d0 2023-10-16T09:44:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a00035b0 con 0x7f85b415e6d0 2023-10-16T09:44:05.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 577646343 0 0) 0x7f85a00035b0 con 0x7f85b415e6d0 2023-10-16T09:44:05.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 msgr2=0x7f85b40d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 0x7f85b40d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 >> v2:172.21.15.190:3300/0 conn(0x7f85b40cb0c0 msgr2=0x7f85b415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 --2- 172.21.15.181:0/2941668819 >> v2:172.21.15.190:3300/0 conn(0x7f85b40cb0c0 0x7f85b415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b406e0f0 con 0x7f85b415e6d0 2023-10-16T09:44:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f85a40017f0 con 0x7f85b415e6d0 2023-10-16T09:44:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 >> v1:172.21.15.181:6789/0 conn(0x7f85b415e6d0 legacy=0x7f85b415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85a0003a00 0x7f85a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 0x7f85a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b40d02d0 con 0x7f85a0004d90 2023-10-16T09:44:05.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b415ad40 con 0x7f85a0003a00 2023-10-16T09:44:05.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a000ba10 con 0x7f85b415eab0 2023-10-16T09:44:05.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba8ba700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba0b9700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85a0003a00 0x7f85a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85b98b8700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 0x7f85a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba8ba700 1 -- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 msgr2=0x7f85a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba8ba700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415eab0 0x7f85a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba8ba700 1 -- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85a0003a00 msgr2=0x7f85a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:05.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.687+0000 7f85ba8ba700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85a0003a00 0x7f85a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85ba8ba700 1 -- 172.21.15.181:0/2941668819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00035b0 con 0x7f85a0004d90 2023-10-16T09:44:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85ba8ba700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85a0005210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009fc0 tx=0x7f85a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d554d11611a045d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00ec30 con 0x7f85a0004d90 2023-10-16T09:44:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac00f040 con 0x7f85a0004d90 2023-10-16T09:44:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85b90b7700 1 -- 172.21.15.181:0/2941668819 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac014480 con 0x7f85a0004d90 2023-10-16T09:44:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85bc31d700 1 -- 172.21.15.181:0/2941668819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 msgr2=0x7f85a0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85bc31d700 1 --2- 172.21.15.181:0/2941668819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85a0005210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009fc0 tx=0x7f85a0005880 comp rx=0 tx=0).stop 2023-10-16T09:44:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85ba8ba700 1 -- 172.21.15.181:0/2941668819 reap_dead start 2023-10-16T09:44:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85bc31d700 1 -- 172.21.15.181:0/2941668819 shutdown_connections 2023-10-16T09:44:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85bc31d700 1 -- 172.21.15.181:0/2941668819 >> 172.21.15.181:0/2941668819 conn(0x7f85b40d04a0 msgr2=0x7f85b40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85bc31d700 1 -- 172.21.15.181:0/2941668819 shutdown_connections 2023-10-16T09:44:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.688+0000 7f85bc31d700 1 -- 172.21.15.181:0/2941668819 wait complete. 2023-10-16T09:44:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 Processor -- start 2023-10-16T09:44:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 -- start start 2023-10-16T09:44:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85b414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b40c59e0 0x7f85b415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415be50 0x7f85b415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b415c820 con 0x7f85a0004d90 2023-10-16T09:44:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b40b3640 con 0x7f85b40c59e0 2023-10-16T09:44:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85bc31d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b40b6230 con 0x7f85b415be50 2023-10-16T09:44:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85ba0b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85b414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85ba0b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85b414e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36064/0 (socket says 172.21.15.181:36064) 2023-10-16T09:44:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.690+0000 7f85ba0b9700 1 -- 172.21.15.181:0/462835691 learned_addr learned my addr 172.21.15.181:0/462835691 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85ba8ba700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415be50 0x7f85b415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85b98b8700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b40c59e0 0x7f85b415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85ba0b9700 1 -- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415be50 msgr2=0x7f85b415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85ba0b9700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415be50 0x7f85b415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85ba0b9700 1 -- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b40c59e0 msgr2=0x7f85b415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85ba0b9700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b40c59e0 0x7f85b415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85ba0b9700 1 -- 172.21.15.181:0/462835691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b40c4c20 con 0x7f85a0004d90 2023-10-16T09:44:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85ba0b9700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85b414e920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85a4007c70 tx=0x7f85a4001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85aaffd700 1 -- 172.21.15.181:0/462835691 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a40072e0 con 0x7f85a0004d90 2023-10-16T09:44:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85aaffd700 1 -- 172.21.15.181:0/462835691 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a4007440 con 0x7f85a0004d90 2023-10-16T09:44:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85aaffd700 1 -- 172.21.15.181:0/462835691 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4005830 con 0x7f85a0004d90 2023-10-16T09:44:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b40c4f00 con 0x7f85a0004d90 2023-10-16T09:44:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.691+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b40c54c0 con 0x7f85a0004d90 2023-10-16T09:44:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.692+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b4067d20 con 0x7f85a0004d90 2023-10-16T09:44:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.692+0000 7f85aaffd700 1 -- 172.21.15.181:0/462835691 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f85a4005990 con 0x7f85a0004d90 2023-10-16T09:44:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.693+0000 7f85aaffd700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f859806af00 0x7f859806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:05.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.693+0000 7f85b98b8700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f859806af00 0x7f859806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:05.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.693+0000 7f85aaffd700 1 -- 172.21.15.181:0/462835691 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85a4089630 con 0x7f85a0004d90 2023-10-16T09:44:05.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.693+0000 7f85b98b8700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f859806af00 0x7f859806d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85b40ca8a0 tx=0x7f85b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:05.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.697+0000 7f85aaffd700 1 -- 172.21.15.181:0/462835691 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f85a4054770 con 0x7f85a0004d90 2023-10-16T09:44:05.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.885+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85b406e6b0 con 0x7f859806af00 2023-10-16T09:44:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.893+0000 7f85aaffd700 1 -- 172.21.15.181:0/462835691 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264702 (secure 0 0 0) 0x7f85b00137d0 con 0x7f859806af00 2023-10-16T09:44:05.897 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f859806af00 msgr2=0x7f859806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f859806af00 0x7f859806d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85b40ca8a0 tx=0x7f85b0008040 comp rx=0 tx=0).stop 2023-10-16T09:44:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 msgr2=0x7f85b414e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85b414e920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85a4007c70 tx=0x7f85a4001890 comp rx=0 tx=0).stop 2023-10-16T09:44:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 shutdown_connections 2023-10-16T09:44:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f859806af00 0x7f859806d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b415be50 0x7f85b415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85a0004d90 0x7f85b414e920 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.896+0000 7f85bc31d700 1 --2- 172.21.15.181:0/462835691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b40c59e0 0x7f85b415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.897+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 >> 172.21.15.181:0/462835691 conn(0x7f85b40d04a0 msgr2=0x7f85b41589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.897+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 shutdown_connections 2023-10-16T09:44:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:05.897+0000 7f85bc31d700 1 -- 172.21.15.181:0/462835691 wait complete. 2023-10-16T09:44:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:44:05.909 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-16T09:44:05.678219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219176,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755544,"statfs":{"total":773094113280,"available":772869677056,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.904416"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793938+0000","last_change":"2023-10-16T09:44:01.820375+0000","last_active":"2023-10-16T09:44:02.793938+0000","last_peered":"2023-10-16T09:44:02.793938+0000","last_clean":"2023-10-16T09:44:02.793938+0000","last_became_active":"2023-10-16T09:44:01.819619+0000","last_became_peered":"2023-10-16T09:44:01.819619+0000","last_unstale":"2023-10-16T09:44:02.793938+0000","last_undegraded":"2023-10-16T09:44:02.793938+0000","last_fullsized":"2023-10-16T09:44:02.793938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792204+0000","last_change":"2023-10-16T09:44:01.800363+0000","last_active":"2023-10-16T09:44:02.792204+0000","last_peered":"2023-10-16T09:44:02.792204+0000","last_clean":"2023-10-16T09:44:02.792204+0000","last_became_active":"2023-10-16T09:44:01.800037+0000","last_became_peered":"2023-10-16T09:44:01.800037+0000","last_unstale":"2023-10-16T09:44:02.792204+0000","last_undegraded":"2023-10-16T09:44:02.792204+0000","last_fullsized":"2023-10-16T09:44:02.792204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896175+0000","last_change":"2023-10-16T09:44:01.806686+0000","last_active":"2023-10-16T09:44:02.896175+0000","last_peered":"2023-10-16T09:44:02.896175+0000","last_clean":"2023-10-16T09:44:02.896175+0000","last_became_active":"2023-10-16T09:44:01.806361+0000","last_became_peered":"2023-10-16T09:44:01.806361+0000","last_unstale":"2023-10-16T09:44:02.896175+0000","last_undegraded":"2023-10-16T09:44:02.896175+0000","last_fullsized":"2023-10-16T09:44:02.896175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793996+0000","last_change":"2023-10-16T09:44:01.802398+0000","last_active":"2023-10-16T09:44:02.793996+0000","last_peered":"2023-10-16T09:44:02.793996+0000","last_clean":"2023-10-16T09:44:02.793996+0000","last_became_active":"2023-10-16T09:44:01.802077+0000","last_became_peered":"2023-10-16T09:44:01.802077+0000","last_unstale":"2023-10-16T09:44:02.793996+0000","last_undegraded":"2023-10-16T09:44:02.793996+0000","last_fullsized":"2023-10-16T09:44:02.793996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793271+0000","last_change":"2023-10-16T09:44:01.817219+0000","last_active":"2023-10-16T09:44:02.793271+0000","last_peered":"2023-10-16T09:44:02.793271+0000","last_clean":"2023-10-16T09:44:02.793271+0000","last_became_active":"2023-10-16T09:44:01.816945+0000","last_became_peered":"2023-10-16T09:44:01.816945+0000","last_unstale":"2023-10-16T09:44:02.793271+0000","last_undegraded":"2023-10-16T09:44:02.793271+0000","last_fullsized":"2023-10-16T09:44:02.793271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794026+0000","last_change":"2023-10-16T09:44:01.807539+0000","last_active":"2023-10-16T09:44:02.794026+0000","last_peered":"2023-10-16T09:44:02.794026+0000","last_clean":"2023-10-16T09:44:02.794026+0000","last_became_active":"2023-10-16T09:44:01.807282+0000","last_became_peered":"2023-10-16T09:44:01.807282+0000","last_unstale":"2023-10-16T09:44:02.794026+0000","last_undegraded":"2023-10-16T09:44:02.794026+0000","last_fullsized":"2023-10-16T09:44:02.794026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792808+0000","last_change":"2023-10-16T09:44:01.816493+0000","last_active":"2023-10-16T09:44:02.792808+0000","last_peered":"2023-10-16T09:44:02.792808+0000","last_clean":"2023-10-16T09:44:02.792808+0000","last_became_active":"2023-10-16T09:44:01.816006+0000","last_became_peered":"2023-10-16T09:44:01.816006+0000","last_unstale":"2023-10-16T09:44:02.792808+0000","last_undegraded":"2023-10-16T09:44:02.792808+0000","last_fullsized":"2023-10-16T09:44:02.792808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045389+0000","last_change":"2023-10-16T09:44:01.802910+0000","last_active":"2023-10-16T09:44:03.045389+0000","last_peered":"2023-10-16T09:44:03.045389+0000","last_clean":"2023-10-16T09:44:03.045389+0000","last_became_active":"2023-10-16T09:44:01.802640+0000","last_became_peered":"2023-10-16T09:44:01.802640+0000","last_unstale":"2023-10-16T09:44:03.045389+0000","last_undegraded":"2023-10-16T09:44:03.045389+0000","last_fullsized":"2023-10-16T09:44:03.045389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793935+0000","last_change":"2023-10-16T09:44:01.818818+0000","last_active":"2023-10-16T09:44:02.793935+0000","last_peered":"2023-10-16T09:44:02.793935+0000","last_clean":"2023-10-16T09:44:02.793935+0000","last_became_active":"2023-10-16T09:44:01.818532+0000","last_became_peered":"2023-10-16T09:44:01.818532+0000","last_unstale":"2023-10-16T09:44:02.793935+0000","last_undegraded":"2023-10-16T09:44:02.793935+0000","last_fullsized":"2023-10-16T09:44:02.793935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046230+0000","last_change":"2023-10-16T09:44:01.808945+0000","last_active":"2023-10-16T09:44:03.046230+0000","last_peered":"2023-10-16T09:44:03.046230+0000","last_clean":"2023-10-16T09:44:03.046230+0000","last_became_active":"2023-10-16T09:44:01.808655+0000","last_became_peered":"2023-10-16T09:44:01.808655+0000","last_unstale":"2023-10-16T09:44:03.046230+0000","last_undegraded":"2023-10-16T09:44:03.046230+0000","last_fullsized":"2023-10-16T09:44:03.046230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045896+0000","last_change":"2023-10-16T09:44:01.808213+0000","last_active":"2023-10-16T09:44:03.045896+0000","last_peered":"2023-10-16T09:44:03.045896+0000","last_clean":"2023-10-16T09:44:03.045896+0000","last_became_active":"2023-10-16T09:44:01.807715+0000","last_became_peered":"2023-10-16T09:44:01.807715+0000","last_unstale":"2023-10-16T09:44:03.045896+0000","last_undegraded":"2023-10-16T09:44:03.045896+0000","last_fullsized":"2023-10-16T09:44:03.045896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046320+0000","last_change":"2023-10-16T09:44:01.801296+0000","last_active":"2023-10-16T09:44:03.046320+0000","last_peered":"2023-10-16T09:44:03.046320+0000","last_clean":"2023-10-16T09:44:03.046320+0000","last_became_active":"2023-10-16T09:44:01.800456+0000","last_became_peered":"2023-10-16T09:44:01.800456+0000","last_unstale":"2023-10-16T09:44:03.046320+0000","last_undegraded":"2023-10-16T09:44:03.046320+0000","last_fullsized":"2023-10-16T09:44:03.046320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:56.560951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045786+0000","last_change":"2023-10-16T09:44:01.800574+0000","last_active":"2023-10-16T09:44:03.045786+0000","last_peered":"2023-10-16T09:44:03.045786+0000","last_clean":"2023-10-16T09:44:03.045786+0000","last_became_active":"2023-10-16T09:44:01.800299+0000","last_became_peered":"2023-10-16T09:44:01.800299+0000","last_unstale":"2023-10-16T09:44:03.045786+0000","last_undegraded":"2023-10-16T09:44:03.045786+0000","last_fullsized":"2023-10-16T09:44:03.045786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895941+0000","last_change":"2023-10-16T09:44:01.797679+0000","last_active":"2023-10-16T09:44:02.895941+0000","last_peered":"2023-10-16T09:44:02.895941+0000","last_clean":"2023-10-16T09:44:02.895941+0000","last_became_active":"2023-10-16T09:44:01.797420+0000","last_became_peered":"2023-10-16T09:44:01.797420+0000","last_unstale":"2023-10-16T09:44:02.895941+0000","last_undegraded":"2023-10-16T09:44:02.895941+0000","last_fullsized":"2023-10-16T09:44:02.895941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791819+0000","last_change":"2023-10-16T09:44:01.801979+0000","last_active":"2023-10-16T09:44:02.791819+0000","last_peered":"2023-10-16T09:44:02.791819+0000","last_clean":"2023-10-16T09:44:02.791819+0000","last_became_active":"2023-10-16T09:44:01.801663+0000","last_became_peered":"2023-10-16T09:44:01.801663+0000","last_unstale":"2023-10-16T09:44:02.791819+0000","last_undegraded":"2023-10-16T09:44:02.791819+0000","last_fullsized":"2023-10-16T09:44:02.791819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793275+0000","last_change":"2023-10-16T09:44:01.808377+0000","last_active":"2023-10-16T09:44:02.793275+0000","last_peered":"2023-10-16T09:44:02.793275+0000","last_clean":"2023-10-16T09:44:02.793275+0000","last_became_active":"2023-10-16T09:44:01.808091+0000","last_became_peered":"2023-10-16T09:44:01.808091+0000","last_unstale":"2023-10-16T09:44:02.793275+0000","last_undegraded":"2023-10-16T09:44:02.793275+0000","last_fullsized":"2023-10-16T09:44:02.793275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794670+0000","last_change":"2023-10-16T09:44:01.815509+0000","last_active":"2023-10-16T09:44:02.794670+0000","last_peered":"2023-10-16T09:44:02.794670+0000","last_clean":"2023-10-16T09:44:02.794670+0000","last_became_active":"2023-10-16T09:44:01.815290+0000","last_became_peered":"2023-10-16T09:44:01.815290+0000","last_unstale":"2023-10-16T09:44:02.794670+0000","last_undegraded":"2023-10-16T09:44:02.794670+0000","last_fullsized":"2023-10-16T09:44:02.794670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045622+0000","last_change":"2023-10-16T09:44:01.804696+0000","last_active":"2023-10-16T09:44:03.045622+0000","last_peered":"2023-10-16T09:44:03.045622+0000","last_clean":"2023-10-16T09:44:03.045622+0000","last_became_active":"2023-10-16T09:44:01.804427+0000","last_became_peered":"2023-10-16T09:44:01.804427+0000","last_unstale":"2023-10-16T09:44:03.045622+0000","last_undegraded":"2023-10-16T09:44:03.045622+0000","last_fullsized":"2023-10-16T09:44:03.045622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:06.338018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794299+0000","last_change":"2023-10-16T09:44:01.815090+0000","last_active":"2023-10-16T09:44:02.794299+0000","last_peered":"2023-10-16T09:44:02.794299+0000","last_clean":"2023-10-16T09:44:02.794299+0000","last_became_active":"2023-10-16T09:44:01.814787+0000","last_became_peered":"2023-10-16T09:44:01.814787+0000","last_unstale":"2023-10-16T09:44:02.794299+0000","last_undegraded":"2023-10-16T09:44:02.794299+0000","last_fullsized":"2023-10-16T09:44:02.794299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.820250+0000","last_change":"2023-10-16T09:44:00.797577+0000","last_active":"2023-10-16T09:44:01.820250+0000","last_peered":"2023-10-16T09:44:01.820250+0000","last_clean":"2023-10-16T09:44:01.820250+0000","last_became_active":"2023-10-16T09:44:00.797323+0000","last_became_peered":"2023-10-16T09:44:00.797323+0000","last_unstale":"2023-10-16T09:44:01.820250+0000","last_undegraded":"2023-10-16T09:44:01.820250+0000","last_fullsized":"2023-10-16T09:44:01.820250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046254+0000","last_change":"2023-10-16T09:44:01.801503+0000","last_active":"2023-10-16T09:44:03.046254+0000","last_peered":"2023-10-16T09:44:03.046254+0000","last_clean":"2023-10-16T09:44:03.046254+0000","last_became_active":"2023-10-16T09:44:01.801078+0000","last_became_peered":"2023-10-16T09:44:01.801078+0000","last_unstale":"2023-10-16T09:44:03.046254+0000","last_undegraded":"2023-10-16T09:44:03.046254+0000","last_fullsized":"2023-10-16T09:44:03.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819479+0000","last_change":"2023-10-16T09:44:00.790199+0000","last_active":"2023-10-16T09:44:01.819479+0000","last_peered":"2023-10-16T09:44:01.819479+0000","last_clean":"2023-10-16T09:44:01.819479+0000","last_became_active":"2023-10-16T09:44:00.789914+0000","last_became_peered":"2023-10-16T09:44:00.789914+0000","last_unstale":"2023-10-16T09:44:01.819479+0000","last_undegraded":"2023-10-16T09:44:01.819479+0000","last_fullsized":"2023-10-16T09:44:01.819479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896280+0000","last_change":"2023-10-16T09:44:01.808873+0000","last_active":"2023-10-16T09:44:02.896280+0000","last_peered":"2023-10-16T09:44:02.896280+0000","last_clean":"2023-10-16T09:44:02.896280+0000","last_became_active":"2023-10-16T09:44:01.808557+0000","last_became_peered":"2023-10-16T09:44:01.808557+0000","last_unstale":"2023-10-16T09:44:02.896280+0000","last_undegraded":"2023-10-16T09:44:02.896280+0000","last_fullsized":"2023-10-16T09:44:02.896280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794605+0000","last_change":"2023-10-16T09:44:00.801288+0000","last_active":"2023-10-16T09:44:02.794605+0000","last_peered":"2023-10-16T09:44:02.794605+0000","last_clean":"2023-10-16T09:44:02.794605+0000","last_became_active":"2023-10-16T09:44:00.801008+0000","last_became_peered":"2023-10-16T09:44:00.801008+0000","last_unstale":"2023-10-16T09:44:02.794605+0000","last_undegraded":"2023-10-16T09:44:02.794605+0000","last_fullsized":"2023-10-16T09:44:02.794605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793760+0000","last_change":"2023-10-16T09:44:01.803802+0000","last_active":"2023-10-16T09:44:02.793760+0000","last_peered":"2023-10-16T09:44:02.793760+0000","last_clean":"2023-10-16T09:44:02.793760+0000","last_became_active":"2023-10-16T09:44:01.803480+0000","last_became_peered":"2023-10-16T09:44:01.803480+0000","last_unstale":"2023-10-16T09:44:02.793760+0000","last_undegraded":"2023-10-16T09:44:02.793760+0000","last_fullsized":"2023-10-16T09:44:02.793760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793452+0000","last_change":"2023-10-16T09:44:00.798160+0000","last_active":"2023-10-16T09:44:02.793452+0000","last_peered":"2023-10-16T09:44:02.793452+0000","last_clean":"2023-10-16T09:44:02.793452+0000","last_became_active":"2023-10-16T09:44:00.797833+0000","last_became_peered":"2023-10-16T09:44:00.797833+0000","last_unstale":"2023-10-16T09:44:02.793452+0000","last_undegraded":"2023-10-16T09:44:02.793452+0000","last_fullsized":"2023-10-16T09:44:02.793452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794244+0000","last_change":"2023-10-16T09:44:01.802107+0000","last_active":"2023-10-16T09:44:02.794244+0000","last_peered":"2023-10-16T09:44:02.794244+0000","last_clean":"2023-10-16T09:44:02.794244+0000","last_became_active":"2023-10-16T09:44:01.801774+0000","last_became_peered":"2023-10-16T09:44:01.801774+0000","last_unstale":"2023-10-16T09:44:02.794244+0000","last_undegraded":"2023-10-16T09:44:02.794244+0000","last_fullsized":"2023-10-16T09:44:02.794244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822531+0000","last_change":"2023-10-16T09:44:00.785046+0000","last_active":"2023-10-16T09:44:01.822531+0000","last_peered":"2023-10-16T09:44:01.822531+0000","last_clean":"2023-10-16T09:44:01.822531+0000","last_became_active":"2023-10-16T09:44:00.784767+0000","last_became_peered":"2023-10-16T09:44:00.784767+0000","last_unstale":"2023-10-16T09:44:01.822531+0000","last_undegraded":"2023-10-16T09:44:01.822531+0000","last_fullsized":"2023-10-16T09:44:01.822531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045674+0000","last_change":"2023-10-16T09:44:01.799372+0000","last_active":"2023-10-16T09:44:03.045674+0000","last_peered":"2023-10-16T09:44:03.045674+0000","last_clean":"2023-10-16T09:44:03.045674+0000","last_became_active":"2023-10-16T09:44:01.799056+0000","last_became_peered":"2023-10-16T09:44:01.799056+0000","last_unstale":"2023-10-16T09:44:03.045674+0000","last_undegraded":"2023-10-16T09:44:03.045674+0000","last_fullsized":"2023-10-16T09:44:03.045674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:44:46.117646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045848+0000","last_change":"2023-10-16T09:44:00.800927+0000","last_active":"2023-10-16T09:44:03.045848+0000","last_peered":"2023-10-16T09:44:03.045848+0000","last_clean":"2023-10-16T09:44:03.045848+0000","last_became_active":"2023-10-16T09:44:00.800533+0000","last_became_peered":"2023-10-16T09:44:00.800533+0000","last_unstale":"2023-10-16T09:44:03.045848+0000","last_undegraded":"2023-10-16T09:44:03.045848+0000","last_fullsized":"2023-10-16T09:44:03.045848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896238+0000","last_change":"2023-10-16T09:44:01.795883+0000","last_active":"2023-10-16T09:44:02.896238+0000","last_peered":"2023-10-16T09:44:02.896238+0000","last_clean":"2023-10-16T09:44:02.896238+0000","last_became_active":"2023-10-16T09:44:01.795613+0000","last_became_peered":"2023-10-16T09:44:01.795613+0000","last_unstale":"2023-10-16T09:44:02.896238+0000","last_undegraded":"2023-10-16T09:44:02.896238+0000","last_fullsized":"2023-10-16T09:44:02.896238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792936+0000","last_change":"2023-10-16T09:44:00.800124+0000","last_active":"2023-10-16T09:44:01.792936+0000","last_peered":"2023-10-16T09:44:01.792936+0000","last_clean":"2023-10-16T09:44:01.792936+0000","last_became_active":"2023-10-16T09:44:00.799861+0000","last_became_peered":"2023-10-16T09:44:00.799861+0000","last_unstale":"2023-10-16T09:44:01.792936+0000","last_undegraded":"2023-10-16T09:44:01.792936+0000","last_fullsized":"2023-10-16T09:44:01.792936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045513+0000","last_change":"2023-10-16T09:44:01.799159+0000","last_active":"2023-10-16T09:44:03.045513+0000","last_peered":"2023-10-16T09:44:03.045513+0000","last_clean":"2023-10-16T09:44:03.045513+0000","last_became_active":"2023-10-16T09:44:01.798882+0000","last_became_peered":"2023-10-16T09:44:01.798882+0000","last_unstale":"2023-10-16T09:44:03.045513+0000","last_undegraded":"2023-10-16T09:44:03.045513+0000","last_fullsized":"2023-10-16T09:44:03.045513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798407+0000","last_change":"2023-10-16T09:44:00.797328+0000","last_active":"2023-10-16T09:44:01.798407+0000","last_peered":"2023-10-16T09:44:01.798407+0000","last_clean":"2023-10-16T09:44:01.798407+0000","last_became_active":"2023-10-16T09:44:00.797038+0000","last_became_peered":"2023-10-16T09:44:00.797038+0000","last_unstale":"2023-10-16T09:44:01.798407+0000","last_undegraded":"2023-10-16T09:44:01.798407+0000","last_fullsized":"2023-10-16T09:44:01.798407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:18.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791948+0000","last_change":"2023-10-16T09:44:01.797648+0000","last_active":"2023-10-16T09:44:02.791948+0000","last_peered":"2023-10-16T09:44:02.791948+0000","last_clean":"2023-10-16T09:44:02.791948+0000","last_became_active":"2023-10-16T09:44:01.797374+0000","last_became_peered":"2023-10-16T09:44:01.797374+0000","last_unstale":"2023-10-16T09:44:02.791948+0000","last_undegraded":"2023-10-16T09:44:02.791948+0000","last_fullsized":"2023-10-16T09:44:02.791948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046347+0000","last_change":"2023-10-16T09:44:00.796911+0000","last_active":"2023-10-16T09:44:03.046347+0000","last_peered":"2023-10-16T09:44:03.046347+0000","last_clean":"2023-10-16T09:44:03.046347+0000","last_became_active":"2023-10-16T09:44:00.796666+0000","last_became_peered":"2023-10-16T09:44:00.796666+0000","last_unstale":"2023-10-16T09:44:03.046347+0000","last_undegraded":"2023-10-16T09:44:03.046347+0000","last_fullsized":"2023-10-16T09:44:03.046347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794228+0000","last_change":"2023-10-16T09:44:01.815668+0000","last_active":"2023-10-16T09:44:02.794228+0000","last_peered":"2023-10-16T09:44:02.794228+0000","last_clean":"2023-10-16T09:44:02.794228+0000","last_became_active":"2023-10-16T09:44:01.815327+0000","last_became_peered":"2023-10-16T09:44:01.815327+0000","last_unstale":"2023-10-16T09:44:02.794228+0000","last_undegraded":"2023-10-16T09:44:02.794228+0000","last_fullsized":"2023-10-16T09:44:02.794228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798169+0000","last_change":"2023-10-16T09:44:00.800951+0000","last_active":"2023-10-16T09:44:01.798169+0000","last_peered":"2023-10-16T09:44:01.798169+0000","last_clean":"2023-10-16T09:44:01.798169+0000","last_became_active":"2023-10-16T09:44:00.800685+0000","last_became_peered":"2023-10-16T09:44:00.800685+0000","last_unstale":"2023-10-16T09:44:01.798169+0000","last_undegraded":"2023-10-16T09:44:01.798169+0000","last_fullsized":"2023-10-16T09:44:01.798169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896480+0000","last_change":"2023-10-16T09:44:01.808665+0000","last_active":"2023-10-16T09:44:02.896480+0000","last_peered":"2023-10-16T09:44:02.896480+0000","last_clean":"2023-10-16T09:44:02.896480+0000","last_became_active":"2023-10-16T09:44:01.808229+0000","last_became_peered":"2023-10-16T09:44:01.808229+0000","last_unstale":"2023-10-16T09:44:02.896480+0000","last_undegraded":"2023-10-16T09:44:02.896480+0000","last_fullsized":"2023-10-16T09:44:02.896480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.824237+0000","last_change":"2023-10-16T09:44:00.797832+0000","last_active":"2023-10-16T09:44:01.824237+0000","last_peered":"2023-10-16T09:44:01.824237+0000","last_clean":"2023-10-16T09:44:01.824237+0000","last_became_active":"2023-10-16T09:44:00.797569+0000","last_became_peered":"2023-10-16T09:44:00.797569+0000","last_unstale":"2023-10-16T09:44:01.824237+0000","last_undegraded":"2023-10-16T09:44:01.824237+0000","last_fullsized":"2023-10-16T09:44:01.824237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046342+0000","last_change":"2023-10-16T09:44:01.820469+0000","last_active":"2023-10-16T09:44:03.046342+0000","last_peered":"2023-10-16T09:44:03.046342+0000","last_clean":"2023-10-16T09:44:03.046342+0000","last_became_active":"2023-10-16T09:44:01.820207+0000","last_became_peered":"2023-10-16T09:44:01.820207+0000","last_unstale":"2023-10-16T09:44:03.046342+0000","last_undegraded":"2023-10-16T09:44:03.046342+0000","last_fullsized":"2023-10-16T09:44:03.046342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:16.100248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045792+0000","last_change":"2023-10-16T09:44:00.794450+0000","last_active":"2023-10-16T09:44:03.045792+0000","last_peered":"2023-10-16T09:44:03.045792+0000","last_clean":"2023-10-16T09:44:03.045792+0000","last_became_active":"2023-10-16T09:44:00.794197+0000","last_became_peered":"2023-10-16T09:44:00.794197+0000","last_unstale":"2023-10-16T09:44:03.045792+0000","last_undegraded":"2023-10-16T09:44:03.045792+0000","last_fullsized":"2023-10-16T09:44:03.045792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794469+0000","last_change":"2023-10-16T09:44:01.817900+0000","last_active":"2023-10-16T09:44:02.794469+0000","last_peered":"2023-10-16T09:44:02.794469+0000","last_clean":"2023-10-16T09:44:02.794469+0000","last_became_active":"2023-10-16T09:44:01.817666+0000","last_became_peered":"2023-10-16T09:44:01.817666+0000","last_unstale":"2023-10-16T09:44:02.794469+0000","last_undegraded":"2023-10-16T09:44:02.794469+0000","last_fullsized":"2023-10-16T09:44:02.794469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822707+0000","last_change":"2023-10-16T09:44:00.794006+0000","last_active":"2023-10-16T09:44:01.822707+0000","last_peered":"2023-10-16T09:44:01.822707+0000","last_clean":"2023-10-16T09:44:01.822707+0000","last_became_active":"2023-10-16T09:44:00.793703+0000","last_became_peered":"2023-10-16T09:44:00.793703+0000","last_unstale":"2023-10-16T09:44:01.822707+0000","last_undegraded":"2023-10-16T09:44:01.822707+0000","last_fullsized":"2023-10-16T09:44:01.822707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895868+0000","last_change":"2023-10-16T09:44:01.815023+0000","last_active":"2023-10-16T09:44:02.895868+0000","last_peered":"2023-10-16T09:44:02.895868+0000","last_clean":"2023-10-16T09:44:02.895868+0000","last_became_active":"2023-10-16T09:44:01.814737+0000","last_became_peered":"2023-10-16T09:44:01.814737+0000","last_unstale":"2023-10-16T09:44:02.895868+0000","last_undegraded":"2023-10-16T09:44:02.895868+0000","last_fullsized":"2023-10-16T09:44:02.895868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:59.336084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.799063+0000","last_change":"2023-10-16T09:44:00.792205+0000","last_active":"2023-10-16T09:44:01.799063+0000","last_peered":"2023-10-16T09:44:01.799063+0000","last_clean":"2023-10-16T09:44:01.799063+0000","last_became_active":"2023-10-16T09:44:00.791860+0000","last_became_peered":"2023-10-16T09:44:00.791860+0000","last_unstale":"2023-10-16T09:44:01.799063+0000","last_undegraded":"2023-10-16T09:44:01.799063+0000","last_fullsized":"2023-10-16T09:44:01.799063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:45.078590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793427+0000","last_change":"2023-10-16T09:44:01.820397+0000","last_active":"2023-10-16T09:44:02.793427+0000","last_peered":"2023-10-16T09:44:02.793427+0000","last_clean":"2023-10-16T09:44:02.793427+0000","last_became_active":"2023-10-16T09:44:01.820151+0000","last_became_peered":"2023-10-16T09:44:01.820151+0000","last_unstale":"2023-10-16T09:44:02.793427+0000","last_undegraded":"2023-10-16T09:44:02.793427+0000","last_fullsized":"2023-10-16T09:44:02.793427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045738+0000","last_change":"2023-10-16T09:44:00.799178+0000","last_active":"2023-10-16T09:44:03.045738+0000","last_peered":"2023-10-16T09:44:03.045738+0000","last_clean":"2023-10-16T09:44:03.045738+0000","last_became_active":"2023-10-16T09:44:00.798866+0000","last_became_peered":"2023-10-16T09:44:00.798866+0000","last_unstale":"2023-10-16T09:44:03.045738+0000","last_undegraded":"2023-10-16T09:44:03.045738+0000","last_fullsized":"2023-10-16T09:44:03.045738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045651+0000","last_change":"2023-10-16T09:44:01.805085+0000","last_active":"2023-10-16T09:44:03.045651+0000","last_peered":"2023-10-16T09:44:03.045651+0000","last_clean":"2023-10-16T09:44:03.045651+0000","last_became_active":"2023-10-16T09:44:01.804493+0000","last_became_peered":"2023-10-16T09:44:01.804493+0000","last_unstale":"2023-10-16T09:44:03.045651+0000","last_undegraded":"2023-10-16T09:44:03.045651+0000","last_fullsized":"2023-10-16T09:44:03.045651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819839+0000","last_change":"2023-10-16T09:44:00.788023+0000","last_active":"2023-10-16T09:44:01.819839+0000","last_peered":"2023-10-16T09:44:01.819839+0000","last_clean":"2023-10-16T09:44:01.819839+0000","last_became_active":"2023-10-16T09:44:00.787751+0000","last_became_peered":"2023-10-16T09:44:00.787751+0000","last_unstale":"2023-10-16T09:44:01.819839+0000","last_undegraded":"2023-10-16T09:44:01.819839+0000","last_fullsized":"2023-10-16T09:44:01.819839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:25:10.084733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793056+0000","last_change":"2023-10-16T09:44:01.818990+0000","last_active":"2023-10-16T09:44:02.793056+0000","last_peered":"2023-10-16T09:44:02.793056+0000","last_clean":"2023-10-16T09:44:02.793056+0000","last_became_active":"2023-10-16T09:44:01.818711+0000","last_became_peered":"2023-10-16T09:44:01.818711+0000","last_unstale":"2023-10-16T09:44:02.793056+0000","last_undegraded":"2023-10-16T09:44:02.793056+0000","last_fullsized":"2023-10-16T09:44:02.793056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.817348+0000","last_change":"2023-10-16T09:44:00.788065+0000","last_active":"2023-10-16T09:44:01.817348+0000","last_peered":"2023-10-16T09:44:01.817348+0000","last_clean":"2023-10-16T09:44:01.817348+0000","last_became_active":"2023-10-16T09:44:00.787742+0000","last_became_peered":"2023-10-16T09:44:00.787742+0000","last_unstale":"2023-10-16T09:44:01.817348+0000","last_undegraded":"2023-10-16T09:44:01.817348+0000","last_fullsized":"2023-10-16T09:44:01.817348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:18.670482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794288+0000","last_change":"2023-10-16T09:44:01.821150+0000","last_active":"2023-10-16T09:44:02.794288+0000","last_peered":"2023-10-16T09:44:02.794288+0000","last_clean":"2023-10-16T09:44:02.794288+0000","last_became_active":"2023-10-16T09:44:01.820830+0000","last_became_peered":"2023-10-16T09:44:01.820830+0000","last_unstale":"2023-10-16T09:44:02.794288+0000","last_undegraded":"2023-10-16T09:44:02.794288+0000","last_fullsized":"2023-10-16T09:44:02.794288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792997+0000","last_change":"2023-10-16T09:44:00.790958+0000","last_active":"2023-10-16T09:44:01.792997+0000","last_peered":"2023-10-16T09:44:01.792997+0000","last_clean":"2023-10-16T09:44:01.792997+0000","last_became_active":"2023-10-16T09:44:00.790679+0000","last_became_peered":"2023-10-16T09:44:00.790679+0000","last_unstale":"2023-10-16T09:44:01.792997+0000","last_undegraded":"2023-10-16T09:44:01.792997+0000","last_fullsized":"2023-10-16T09:44:01.792997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:23.552983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794431+0000","last_change":"2023-10-16T09:44:01.801190+0000","last_active":"2023-10-16T09:44:02.794431+0000","last_peered":"2023-10-16T09:44:02.794431+0000","last_clean":"2023-10-16T09:44:02.794431+0000","last_became_active":"2023-10-16T09:44:01.800889+0000","last_became_peered":"2023-10-16T09:44:01.800889+0000","last_unstale":"2023-10-16T09:44:02.794431+0000","last_undegraded":"2023-10-16T09:44:02.794431+0000","last_fullsized":"2023-10-16T09:44:02.794431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798899+0000","last_change":"2023-10-16T09:44:00.792936+0000","last_active":"2023-10-16T09:44:01.798899+0000","last_peered":"2023-10-16T09:44:01.798899+0000","last_clean":"2023-10-16T09:44:01.798899+0000","last_became_active":"2023-10-16T09:44:00.792595+0000","last_became_peered":"2023-10-16T09:44:00.792595+0000","last_unstale":"2023-10-16T09:44:01.798899+0000","last_undegraded":"2023-10-16T09:44:01.798899+0000","last_fullsized":"2023-10-16T09:44:01.798899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:22.500292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045929+0000","last_change":"2023-10-16T09:44:01.800944+0000","last_active":"2023-10-16T09:44:03.045929+0000","last_peered":"2023-10-16T09:44:03.045929+0000","last_clean":"2023-10-16T09:44:03.045929+0000","last_became_active":"2023-10-16T09:44:01.800569+0000","last_became_peered":"2023-10-16T09:44:01.800569+0000","last_unstale":"2023-10-16T09:44:03.045929+0000","last_undegraded":"2023-10-16T09:44:03.045929+0000","last_fullsized":"2023-10-16T09:44:03.045929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:38:40.688657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791775+0000","last_change":"2023-10-16T09:44:01.805039+0000","last_active":"2023-10-16T09:44:02.791775+0000","last_peered":"2023-10-16T09:44:02.791775+0000","last_clean":"2023-10-16T09:44:02.791775+0000","last_became_active":"2023-10-16T09:44:01.804777+0000","last_became_peered":"2023-10-16T09:44:01.804777+0000","last_unstale":"2023-10-16T09:44:02.791775+0000","last_undegraded":"2023-10-16T09:44:02.791775+0000","last_fullsized":"2023-10-16T09:44:02.791775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800545+0000","last_change":"2023-10-16T09:44:00.790368+0000","last_active":"2023-10-16T09:44:01.800545+0000","last_peered":"2023-10-16T09:44:01.800545+0000","last_clean":"2023-10-16T09:44:01.800545+0000","last_became_active":"2023-10-16T09:44:00.790030+0000","last_became_peered":"2023-10-16T09:44:00.790030+0000","last_unstale":"2023-10-16T09:44:01.800545+0000","last_undegraded":"2023-10-16T09:44:01.800545+0000","last_fullsized":"2023-10-16T09:44:01.800545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:19.134455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045870+0000","last_change":"2023-10-16T09:44:01.802895+0000","last_active":"2023-10-16T09:44:03.045870+0000","last_peered":"2023-10-16T09:44:03.045870+0000","last_clean":"2023-10-16T09:44:03.045870+0000","last_became_active":"2023-10-16T09:44:01.802637+0000","last_became_peered":"2023-10-16T09:44:01.802637+0000","last_unstale":"2023-10-16T09:44:03.045870+0000","last_undegraded":"2023-10-16T09:44:03.045870+0000","last_fullsized":"2023-10-16T09:44:03.045870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798097+0000","last_change":"2023-10-16T09:44:00.793146+0000","last_active":"2023-10-16T09:44:01.798097+0000","last_peered":"2023-10-16T09:44:01.798097+0000","last_clean":"2023-10-16T09:44:01.798097+0000","last_became_active":"2023-10-16T09:44:00.792827+0000","last_became_peered":"2023-10-16T09:44:00.792827+0000","last_unstale":"2023-10-16T09:44:01.798097+0000","last_undegraded":"2023-10-16T09:44:01.798097+0000","last_fullsized":"2023-10-16T09:44:01.798097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.803000+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:01.803000+0000","last_peered":"2023-10-16T09:44:01.803000+0000","last_clean":"2023-10-16T09:44:01.803000+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:01.803000+0000","last_undegraded":"2023-10-16T09:44:01.803000+0000","last_fullsized":"2023-10-16T09:44:01.803000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895844+0000","last_change":"2023-10-16T09:44:01.795766+0000","last_active":"2023-10-16T09:44:02.895844+0000","last_peered":"2023-10-16T09:44:02.895844+0000","last_clean":"2023-10-16T09:44:02.895844+0000","last_became_active":"2023-10-16T09:44:01.795429+0000","last_became_peered":"2023-10-16T09:44:01.795429+0000","last_unstale":"2023-10-16T09:44:02.895844+0000","last_undegraded":"2023-10-16T09:44:02.895844+0000","last_fullsized":"2023-10-16T09:44:02.895844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:24.556381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792852+0000","last_change":"2023-10-16T09:44:00.800131+0000","last_active":"2023-10-16T09:44:02.792852+0000","last_peered":"2023-10-16T09:44:02.792852+0000","last_clean":"2023-10-16T09:44:02.792852+0000","last_became_active":"2023-10-16T09:44:00.799868+0000","last_became_peered":"2023-10-16T09:44:00.799868+0000","last_unstale":"2023-10-16T09:44:02.792852+0000","last_undegraded":"2023-10-16T09:44:02.792852+0000","last_fullsized":"2023-10-16T09:44:02.792852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794561+0000","last_change":"2023-10-16T09:44:00.799180+0000","last_active":"2023-10-16T09:44:02.794561+0000","last_peered":"2023-10-16T09:44:02.794561+0000","last_clean":"2023-10-16T09:44:02.794561+0000","last_became_active":"2023-10-16T09:44:00.798901+0000","last_became_peered":"2023-10-16T09:44:00.798901+0000","last_unstale":"2023-10-16T09:44:02.794561+0000","last_undegraded":"2023-10-16T09:44:02.794561+0000","last_fullsized":"2023-10-16T09:44:02.794561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792018+0000","last_change":"2023-10-16T09:44:01.804991+0000","last_active":"2023-10-16T09:44:02.792018+0000","last_peered":"2023-10-16T09:44:02.792018+0000","last_clean":"2023-10-16T09:44:02.792018+0000","last_became_active":"2023-10-16T09:44:01.804694+0000","last_became_peered":"2023-10-16T09:44:01.804694+0000","last_unstale":"2023-10-16T09:44:02.792018+0000","last_undegraded":"2023-10-16T09:44:02.792018+0000","last_fullsized":"2023-10-16T09:44:02.792018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790556+0000","last_change":"2023-10-16T09:44:00.800156+0000","last_active":"2023-10-16T09:44:01.790556+0000","last_peered":"2023-10-16T09:44:01.790556+0000","last_clean":"2023-10-16T09:44:01.790556+0000","last_became_active":"2023-10-16T09:44:00.799895+0000","last_became_peered":"2023-10-16T09:44:00.799895+0000","last_unstale":"2023-10-16T09:44:01.790556+0000","last_undegraded":"2023-10-16T09:44:01.790556+0000","last_fullsized":"2023-10-16T09:44:01.790556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046312+0000","last_change":"2023-10-16T09:44:01.819462+0000","last_active":"2023-10-16T09:44:03.046312+0000","last_peered":"2023-10-16T09:44:03.046312+0000","last_clean":"2023-10-16T09:44:03.046312+0000","last_became_active":"2023-10-16T09:44:01.819196+0000","last_became_peered":"2023-10-16T09:44:01.819196+0000","last_unstale":"2023-10-16T09:44:03.046312+0000","last_undegraded":"2023-10-16T09:44:03.046312+0000","last_fullsized":"2023-10-16T09:44:03.046312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046068+0000","last_change":"2023-10-16T09:44:00.801223+0000","last_active":"2023-10-16T09:44:03.046068+0000","last_peered":"2023-10-16T09:44:03.046068+0000","last_clean":"2023-10-16T09:44:03.046068+0000","last_became_active":"2023-10-16T09:44:00.800978+0000","last_became_peered":"2023-10-16T09:44:00.800978+0000","last_unstale":"2023-10-16T09:44:03.046068+0000","last_undegraded":"2023-10-16T09:44:03.046068+0000","last_fullsized":"2023-10-16T09:44:03.046068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896394+0000","last_change":"2023-10-16T09:44:01.798851+0000","last_active":"2023-10-16T09:44:02.896394+0000","last_peered":"2023-10-16T09:44:02.896394+0000","last_clean":"2023-10-16T09:44:02.896394+0000","last_became_active":"2023-10-16T09:44:01.798475+0000","last_became_peered":"2023-10-16T09:44:01.798475+0000","last_unstale":"2023-10-16T09:44:02.896394+0000","last_undegraded":"2023-10-16T09:44:02.896394+0000","last_fullsized":"2023-10-16T09:44:02.896394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792734+0000","last_change":"2023-10-16T09:44:00.799681+0000","last_active":"2023-10-16T09:44:02.792734+0000","last_peered":"2023-10-16T09:44:02.792734+0000","last_clean":"2023-10-16T09:44:02.792734+0000","last_became_active":"2023-10-16T09:44:00.799423+0000","last_became_peered":"2023-10-16T09:44:00.799423+0000","last_unstale":"2023-10-16T09:44:02.792734+0000","last_undegraded":"2023-10-16T09:44:02.792734+0000","last_fullsized":"2023-10-16T09:44:02.792734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046475+0000","last_change":"2023-10-16T09:44:01.806654+0000","last_active":"2023-10-16T09:44:03.046475+0000","last_peered":"2023-10-16T09:44:03.046475+0000","last_clean":"2023-10-16T09:44:03.046475+0000","last_became_active":"2023-10-16T09:44:01.805672+0000","last_became_peered":"2023-10-16T09:44:01.805672+0000","last_unstale":"2023-10-16T09:44:03.046475+0000","last_undegraded":"2023-10-16T09:44:03.046475+0000","last_fullsized":"2023-10-16T09:44:03.046475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:41.878512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790462+0000","last_change":"2023-10-16T09:44:00.797519+0000","last_active":"2023-10-16T09:44:01.790462+0000","last_peered":"2023-10-16T09:44:01.790462+0000","last_clean":"2023-10-16T09:44:01.790462+0000","last_became_active":"2023-10-16T09:44:00.797217+0000","last_became_peered":"2023-10-16T09:44:00.797217+0000","last_unstale":"2023-10-16T09:44:01.790462+0000","last_undegraded":"2023-10-16T09:44:01.790462+0000","last_fullsized":"2023-10-16T09:44:01.790462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045449+0000","last_change":"2023-10-16T09:44:01.798478+0000","last_active":"2023-10-16T09:44:03.045449+0000","last_peered":"2023-10-16T09:44:03.045449+0000","last_clean":"2023-10-16T09:44:03.045449+0000","last_became_active":"2023-10-16T09:44:01.798146+0000","last_became_peered":"2023-10-16T09:44:01.798146+0000","last_unstale":"2023-10-16T09:44:03.045449+0000","last_undegraded":"2023-10-16T09:44:03.045449+0000","last_fullsized":"2023-10-16T09:44:03.045449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790384+0000","last_change":"2023-10-16T09:44:00.791453+0000","last_active":"2023-10-16T09:44:01.790384+0000","last_peered":"2023-10-16T09:44:01.790384+0000","last_clean":"2023-10-16T09:44:01.790384+0000","last_became_active":"2023-10-16T09:44:00.791178+0000","last_became_peered":"2023-10-16T09:44:00.791178+0000","last_unstale":"2023-10-16T09:44:01.790384+0000","last_undegraded":"2023-10-16T09:44:01.790384+0000","last_fullsized":"2023-10-16T09:44:01.790384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046137+0000","last_change":"2023-10-16T09:44:01.801224+0000","last_active":"2023-10-16T09:44:03.046137+0000","last_peered":"2023-10-16T09:44:03.046137+0000","last_clean":"2023-10-16T09:44:03.046137+0000","last_became_active":"2023-10-16T09:44:01.800946+0000","last_became_peered":"2023-10-16T09:44:01.800946+0000","last_unstale":"2023-10-16T09:44:03.046137+0000","last_undegraded":"2023-10-16T09:44:03.046137+0000","last_fullsized":"2023-10-16T09:44:03.046137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793527+0000","last_change":"2023-10-16T09:44:00.799160+0000","last_active":"2023-10-16T09:44:02.793527+0000","last_peered":"2023-10-16T09:44:02.793527+0000","last_clean":"2023-10-16T09:44:02.793527+0000","last_became_active":"2023-10-16T09:44:00.798874+0000","last_became_peered":"2023-10-16T09:44:00.798874+0000","last_unstale":"2023-10-16T09:44:02.793527+0000","last_undegraded":"2023-10-16T09:44:02.793527+0000","last_fullsized":"2023-10-16T09:44:02.793527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045858+0000","last_change":"2023-10-16T09:44:01.802604+0000","last_active":"2023-10-16T09:44:03.045858+0000","last_peered":"2023-10-16T09:44:03.045858+0000","last_clean":"2023-10-16T09:44:03.045858+0000","last_became_active":"2023-10-16T09:44:01.801907+0000","last_became_peered":"2023-10-16T09:44:01.801907+0000","last_unstale":"2023-10-16T09:44:03.045858+0000","last_undegraded":"2023-10-16T09:44:03.045858+0000","last_fullsized":"2023-10-16T09:44:03.045858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800616+0000","last_change":"2023-10-16T09:44:00.796930+0000","last_active":"2023-10-16T09:44:01.800616+0000","last_peered":"2023-10-16T09:44:01.800616+0000","last_clean":"2023-10-16T09:44:01.800616+0000","last_became_active":"2023-10-16T09:44:00.796659+0000","last_became_peered":"2023-10-16T09:44:00.796659+0000","last_unstale":"2023-10-16T09:44:01.800616+0000","last_undegraded":"2023-10-16T09:44:01.800616+0000","last_fullsized":"2023-10-16T09:44:01.800616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794538+0000","last_change":"2023-10-16T09:44:01.820645+0000","last_active":"2023-10-16T09:44:02.794538+0000","last_peered":"2023-10-16T09:44:02.794538+0000","last_clean":"2023-10-16T09:44:02.794538+0000","last_became_active":"2023-10-16T09:44:01.820369+0000","last_became_peered":"2023-10-16T09:44:01.820369+0000","last_unstale":"2023-10-16T09:44:02.794538+0000","last_undegraded":"2023-10-16T09:44:02.794538+0000","last_fullsized":"2023-10-16T09:44:02.794538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822381+0000","last_change":"2023-10-16T09:44:00.785075+0000","last_active":"2023-10-16T09:44:01.822381+0000","last_peered":"2023-10-16T09:44:01.822381+0000","last_clean":"2023-10-16T09:44:01.822381+0000","last_became_active":"2023-10-16T09:44:00.784807+0000","last_became_peered":"2023-10-16T09:44:00.784807+0000","last_unstale":"2023-10-16T09:44:01.822381+0000","last_undegraded":"2023-10-16T09:44:01.822381+0000","last_fullsized":"2023-10-16T09:44:01.822381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793947+0000","last_change":"2023-10-16T09:44:01.804040+0000","last_active":"2023-10-16T09:44:02.793947+0000","last_peered":"2023-10-16T09:44:02.793947+0000","last_clean":"2023-10-16T09:44:02.793947+0000","last_became_active":"2023-10-16T09:44:01.803782+0000","last_became_peered":"2023-10-16T09:44:01.803782+0000","last_unstale":"2023-10-16T09:44:02.793947+0000","last_undegraded":"2023-10-16T09:44:02.793947+0000","last_fullsized":"2023-10-16T09:44:02.793947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819101+0000","last_change":"2023-10-16T09:44:00.790542+0000","last_active":"2023-10-16T09:44:01.819101+0000","last_peered":"2023-10-16T09:44:01.819101+0000","last_clean":"2023-10-16T09:44:01.819101+0000","last_became_active":"2023-10-16T09:44:00.790254+0000","last_became_peered":"2023-10-16T09:44:00.790254+0000","last_unstale":"2023-10-16T09:44:01.819101+0000","last_undegraded":"2023-10-16T09:44:01.819101+0000","last_fullsized":"2023-10-16T09:44:01.819101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794115+0000","last_change":"2023-10-16T09:44:01.820628+0000","last_active":"2023-10-16T09:44:02.794115+0000","last_peered":"2023-10-16T09:44:02.794115+0000","last_clean":"2023-10-16T09:44:02.794115+0000","last_became_active":"2023-10-16T09:44:01.820114+0000","last_became_peered":"2023-10-16T09:44:01.820114+0000","last_unstale":"2023-10-16T09:44:02.794115+0000","last_undegraded":"2023-10-16T09:44:02.794115+0000","last_fullsized":"2023-10-16T09:44:02.794115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046278+0000","last_change":"2023-10-16T09:44:01.805259+0000","last_active":"2023-10-16T09:44:03.046278+0000","last_peered":"2023-10-16T09:44:03.046278+0000","last_clean":"2023-10-16T09:44:03.046278+0000","last_became_active":"2023-10-16T09:44:01.804819+0000","last_became_peered":"2023-10-16T09:44:01.804819+0000","last_unstale":"2023-10-16T09:44:03.046278+0000","last_undegraded":"2023-10-16T09:44:03.046278+0000","last_fullsized":"2023-10-16T09:44:03.046278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045666+0000","last_change":"2023-10-16T09:44:01.808510+0000","last_active":"2023-10-16T09:44:03.045666+0000","last_peered":"2023-10-16T09:44:03.045666+0000","last_clean":"2023-10-16T09:44:03.045666+0000","last_became_active":"2023-10-16T09:44:01.808152+0000","last_became_peered":"2023-10-16T09:44:01.808152+0000","last_unstale":"2023-10-16T09:44:03.045666+0000","last_undegraded":"2023-10-16T09:44:03.045666+0000","last_fullsized":"2023-10-16T09:44:03.045666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:13.616924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793846+0000","last_change":"2023-10-16T09:44:01.808070+0000","last_active":"2023-10-16T09:44:02.793846+0000","last_peered":"2023-10-16T09:44:02.793846+0000","last_clean":"2023-10-16T09:44:02.793846+0000","last_became_active":"2023-10-16T09:44:01.807794+0000","last_became_peered":"2023-10-16T09:44:01.807794+0000","last_unstale":"2023-10-16T09:44:02.793846+0000","last_undegraded":"2023-10-16T09:44:02.793846+0000","last_fullsized":"2023-10-16T09:44:02.793846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896204+0000","last_change":"2023-10-16T09:44:01.796651+0000","last_active":"2023-10-16T09:44:02.896204+0000","last_peered":"2023-10-16T09:44:02.896204+0000","last_clean":"2023-10-16T09:44:02.896204+0000","last_became_active":"2023-10-16T09:44:01.796377+0000","last_became_peered":"2023-10-16T09:44:01.796377+0000","last_unstale":"2023-10-16T09:44:02.896204+0000","last_undegraded":"2023-10-16T09:44:02.896204+0000","last_fullsized":"2023-10-16T09:44:02.896204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046174+0000","last_change":"2023-10-16T09:44:01.803048+0000","last_active":"2023-10-16T09:44:03.046174+0000","last_peered":"2023-10-16T09:44:03.046174+0000","last_clean":"2023-10-16T09:44:03.046174+0000","last_became_active":"2023-10-16T09:44:01.802761+0000","last_became_peered":"2023-10-16T09:44:01.802761+0000","last_unstale":"2023-10-16T09:44:03.046174+0000","last_undegraded":"2023-10-16T09:44:03.046174+0000","last_fullsized":"2023-10-16T09:44:03.046174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791913+0000","last_change":"2023-10-16T09:44:01.797818+0000","last_active":"2023-10-16T09:44:02.791913+0000","last_peered":"2023-10-16T09:44:02.791913+0000","last_clean":"2023-10-16T09:44:02.791913+0000","last_became_active":"2023-10-16T09:44:01.797537+0000","last_became_peered":"2023-10-16T09:44:01.797537+0000","last_unstale":"2023-10-16T09:44:02.791913+0000","last_undegraded":"2023-10-16T09:44:02.791913+0000","last_fullsized":"2023-10-16T09:44:02.791913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046247+0000","last_change":"2023-10-16T09:44:01.821446+0000","last_active":"2023-10-16T09:44:03.046247+0000","last_peered":"2023-10-16T09:44:03.046247+0000","last_clean":"2023-10-16T09:44:03.046247+0000","last_became_active":"2023-10-16T09:44:01.821191+0000","last_became_peered":"2023-10-16T09:44:01.821191+0000","last_unstale":"2023-10-16T09:44:03.046247+0000","last_undegraded":"2023-10-16T09:44:03.046247+0000","last_fullsized":"2023-10-16T09:44:03.046247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046237+0000","last_change":"2023-10-16T09:44:01.801738+0000","last_active":"2023-10-16T09:44:03.046237+0000","last_peered":"2023-10-16T09:44:03.046237+0000","last_clean":"2023-10-16T09:44:03.046237+0000","last_became_active":"2023-10-16T09:44:01.801400+0000","last_became_peered":"2023-10-16T09:44:01.801400+0000","last_unstale":"2023-10-16T09:44:03.046237+0000","last_undegraded":"2023-10-16T09:44:03.046237+0000","last_fullsized":"2023-10-16T09:44:03.046237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793441+0000","last_change":"2023-10-16T09:44:01.797904+0000","last_active":"2023-10-16T09:44:02.793441+0000","last_peered":"2023-10-16T09:44:02.793441+0000","last_clean":"2023-10-16T09:44:02.793441+0000","last_became_active":"2023-10-16T09:44:01.797424+0000","last_became_peered":"2023-10-16T09:44:01.797424+0000","last_unstale":"2023-10-16T09:44:02.793441+0000","last_undegraded":"2023-10-16T09:44:02.793441+0000","last_fullsized":"2023-10-16T09:44:02.793441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045818+0000","last_change":"2023-10-16T09:44:01.802067+0000","last_active":"2023-10-16T09:44:03.045818+0000","last_peered":"2023-10-16T09:44:03.045818+0000","last_clean":"2023-10-16T09:44:03.045818+0000","last_became_active":"2023-10-16T09:44:01.801792+0000","last_became_peered":"2023-10-16T09:44:01.801792+0000","last_unstale":"2023-10-16T09:44:03.045818+0000","last_undegraded":"2023-10-16T09:44:03.045818+0000","last_fullsized":"2023-10-16T09:44:03.045818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793644+0000","last_change":"2023-10-16T09:44:01.819606+0000","last_active":"2023-10-16T09:44:02.793644+0000","last_peered":"2023-10-16T09:44:02.793644+0000","last_clean":"2023-10-16T09:44:02.793644+0000","last_became_active":"2023-10-16T09:44:01.819328+0000","last_became_peered":"2023-10-16T09:44:01.819328+0000","last_unstale":"2023-10-16T09:44:02.793644+0000","last_undegraded":"2023-10-16T09:44:02.793644+0000","last_fullsized":"2023-10-16T09:44:02.793644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896102+0000","last_change":"2023-10-16T09:44:01.818209+0000","last_active":"2023-10-16T09:44:02.896102+0000","last_peered":"2023-10-16T09:44:02.896102+0000","last_clean":"2023-10-16T09:44:02.896102+0000","last_became_active":"2023-10-16T09:44:01.817921+0000","last_became_peered":"2023-10-16T09:44:01.817921+0000","last_unstale":"2023-10-16T09:44:02.896102+0000","last_undegraded":"2023-10-16T09:44:02.896102+0000","last_fullsized":"2023-10-16T09:44:02.896102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045707+0000","last_change":"2023-10-16T09:44:01.802316+0000","last_active":"2023-10-16T09:44:03.045707+0000","last_peered":"2023-10-16T09:44:03.045707+0000","last_clean":"2023-10-16T09:44:03.045707+0000","last_became_active":"2023-10-16T09:44:01.802036+0000","last_became_peered":"2023-10-16T09:44:01.802036+0000","last_unstale":"2023-10-16T09:44:03.045707+0000","last_undegraded":"2023-10-16T09:44:03.045707+0000","last_fullsized":"2023-10-16T09:44:03.045707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:44:05.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:44:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.069+0000 7f7748bf5700 1 Processor -- start 2023-10-16T09:44:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.069+0000 7f7748bf5700 1 -- start start 2023-10-16T09:44:06.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f7748bf5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f774415dfd0 0x7f77440af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f7748bf5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77440af450 0x7f774406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f7748bf5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774406f870 con 0x7f774406db80 2023-10-16T09:44:06.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f7748bf5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77440d1160 con 0x7f774415dfd0 2023-10-16T09:44:06.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f7748bf5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77440cc8e0 con 0x7f77440af450 2023-10-16T09:44:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f774415dfd0 0x7f77440af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f774415dfd0 0x7f77440af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36072/0 (socket says 172.21.15.181:36072) 2023-10-16T09:44:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 -- 172.21.15.181:0/2258960541 learned_addr learned my addr 172.21.15.181:0/2258960541 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 -- 172.21.15.181:0/2258960541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77440af450 msgr2=0x7f774406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f7741d9b700 1 --2- 172.21.15.181:0/2258960541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77440af450 0x7f774406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 --2- 172.21.15.181:0/2258960541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77440af450 0x7f774406d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 -- 172.21.15.181:0/2258960541 >> v1:172.21.15.190:6789/0 conn(0x7f774406db80 legacy=0x7f77440cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 -- 172.21.15.181:0/2258960541 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440ccbc0 con 0x7f774415dfd0 2023-10-16T09:44:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774259c700 1 --2- 172.21.15.181:0/2258960541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f774415dfd0 0x7f77440af9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7734006fd0 tx=0x7f7734009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ff0ad90c47d977 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774159a700 1 -- 172.21.15.181:0/2258960541 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734005e30 con 0x7f774415dfd0 2023-10-16T09:44:06.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774159a700 1 -- 172.21.15.181:0/2258960541 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7734006030 con 0x7f774415dfd0 2023-10-16T09:44:06.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.070+0000 7f774159a700 1 -- 172.21.15.181:0/2258960541 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340063c0 con 0x7f774415dfd0 2023-10-16T09:44:06.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 -- 172.21.15.181:0/2258960541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f774415dfd0 msgr2=0x7f77440af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 --2- 172.21.15.181:0/2258960541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f774415dfd0 0x7f77440af9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7734006fd0 tx=0x7f7734009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 -- 172.21.15.181:0/2258960541 shutdown_connections 2023-10-16T09:44:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 --2- 172.21.15.181:0/2258960541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f774415dfd0 0x7f77440af9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 --2- 172.21.15.181:0/2258960541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77440af450 0x7f774406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 -- 172.21.15.181:0/2258960541 >> 172.21.15.181:0/2258960541 conn(0x7f77440d04a0 msgr2=0x7f77440d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 -- 172.21.15.181:0/2258960541 shutdown_connections 2023-10-16T09:44:06.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.071+0000 7f7748bf5700 1 -- 172.21.15.181:0/2258960541 wait complete. 2023-10-16T09:44:06.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.072+0000 7f7748bf5700 1 Processor -- start 2023-10-16T09:44:06.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.072+0000 7f7748bf5700 1 -- start start 2023-10-16T09:44:06.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7748bf5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f774406db80 0x7f7744152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7748bf5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77440af450 0x7f77440ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7748bf5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774415dfd0 0x7f77440ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7748bf5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77440b1220 con 0x7f77440af450 2023-10-16T09:44:06.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7748bf5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77441529c0 con 0x7f774415dfd0 2023-10-16T09:44:06.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7748bf5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77440b0610 con 0x7f774406db80 2023-10-16T09:44:06.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7741d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77440af450 0x7f77440ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7741d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77440af450 0x7f77440ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36078/0 (socket says 172.21.15.181:36078) 2023-10-16T09:44:06.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7741d9b700 1 -- 172.21.15.181:0/1781760861 learned_addr learned my addr 172.21.15.181:0/1781760861 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:06.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f774259c700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f774406db80 0x7f7744152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.073+0000 7f7742d9d700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774415dfd0 0x7f77440ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7741d9b700 1 -- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f774406db80 msgr2=0x7f7744152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7741d9b700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f774406db80 0x7f7744152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7741d9b700 1 -- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774415dfd0 msgr2=0x7f77440ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7741d9b700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774415dfd0 0x7f77440ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7741d9b700 1 -- 172.21.15.181:0/1781760861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734009500 con 0x7f77440af450 2023-10-16T09:44:06.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7741d9b700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77440af450 0x7f77440ce3a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f773800c7f0 tx=0x7f773800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7732ffd700 1 -- 172.21.15.181:0/1781760861 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738004220 con 0x7f77440af450 2023-10-16T09:44:06.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7732ffd700 1 -- 172.21.15.181:0/1781760861 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7738004380 con 0x7f77440af450 2023-10-16T09:44:06.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77440b08f0 con 0x7f77440af450 2023-10-16T09:44:06.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7732ffd700 1 -- 172.21.15.181:0/1781760861 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77380036a0 con 0x7f77440af450 2023-10-16T09:44:06.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.074+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77440b0eb0 con 0x7f77440af450 2023-10-16T09:44:06.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.075+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7744067d20 con 0x7f77440af450 2023-10-16T09:44:06.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.075+0000 7f7732ffd700 1 -- 172.21.15.181:0/1781760861 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7738003800 con 0x7f77440af450 2023-10-16T09:44:06.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.076+0000 7f7732ffd700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f772406af00 0x7f772406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.076+0000 7f774259c700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f772406af00 0x7f772406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.076+0000 7f7732ffd700 1 -- 172.21.15.181:0/1781760861 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7738089100 con 0x7f77440af450 2023-10-16T09:44:06.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.076+0000 7f774259c700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f772406af00 0x7f772406d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7734013040 tx=0x7f7734008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.087 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-10-16T09:44:06.081+0000 7f3b9ce57700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:44:06.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.080+0000 7f7732ffd700 1 -- 172.21.15.181:0/1781760861 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f77380541b0 con 0x7f77440af450 2023-10-16T09:44:06.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.265+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77440cc590 con 0x7f772406af00 2023-10-16T09:44:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.272+0000 7f7732ffd700 1 -- 172.21.15.181:0/1781760861 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264702 (secure 0 0 0) 0x7f7734005d30 con 0x7f772406af00 2023-10-16T09:44:06.276 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f772406af00 msgr2=0x7f772406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f772406af00 0x7f772406d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7734013040 tx=0x7f7734008070 comp rx=0 tx=0).stop 2023-10-16T09:44:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77440af450 msgr2=0x7f77440ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77440af450 0x7f77440ce3a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f773800c7f0 tx=0x7f773800cb00 comp rx=0 tx=0).stop 2023-10-16T09:44:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 shutdown_connections 2023-10-16T09:44:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f772406af00 0x7f772406d3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f774406db80 0x7f7744152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774415dfd0 0x7f77440ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 --2- 172.21.15.181:0/1781760861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77440af450 0x7f77440ce3a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 >> 172.21.15.181:0/1781760861 conn(0x7f77440d04a0 msgr2=0x7f77440cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:06.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 shutdown_connections 2023-10-16T09:44:06.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.275+0000 7f7748bf5700 1 -- 172.21.15.181:0/1781760861 wait complete. 2023-10-16T09:44:06.281 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:44:06.288 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-16T09:44:05.678219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219176,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755544,"statfs":{"total":773094113280,"available":772869677056,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.904416"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793938+0000","last_change":"2023-10-16T09:44:01.820375+0000","last_active":"2023-10-16T09:44:02.793938+0000","last_peered":"2023-10-16T09:44:02.793938+0000","last_clean":"2023-10-16T09:44:02.793938+0000","last_became_active":"2023-10-16T09:44:01.819619+0000","last_became_peered":"2023-10-16T09:44:01.819619+0000","last_unstale":"2023-10-16T09:44:02.793938+0000","last_undegraded":"2023-10-16T09:44:02.793938+0000","last_fullsized":"2023-10-16T09:44:02.793938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792204+0000","last_change":"2023-10-16T09:44:01.800363+0000","last_active":"2023-10-16T09:44:02.792204+0000","last_peered":"2023-10-16T09:44:02.792204+0000","last_clean":"2023-10-16T09:44:02.792204+0000","last_became_active":"2023-10-16T09:44:01.800037+0000","last_became_peered":"2023-10-16T09:44:01.800037+0000","last_unstale":"2023-10-16T09:44:02.792204+0000","last_undegraded":"2023-10-16T09:44:02.792204+0000","last_fullsized":"2023-10-16T09:44:02.792204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896175+0000","last_change":"2023-10-16T09:44:01.806686+0000","last_active":"2023-10-16T09:44:02.896175+0000","last_peered":"2023-10-16T09:44:02.896175+0000","last_clean":"2023-10-16T09:44:02.896175+0000","last_became_active":"2023-10-16T09:44:01.806361+0000","last_became_peered":"2023-10-16T09:44:01.806361+0000","last_unstale":"2023-10-16T09:44:02.896175+0000","last_undegraded":"2023-10-16T09:44:02.896175+0000","last_fullsized":"2023-10-16T09:44:02.896175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793996+0000","last_change":"2023-10-16T09:44:01.802398+0000","last_active":"2023-10-16T09:44:02.793996+0000","last_peered":"2023-10-16T09:44:02.793996+0000","last_clean":"2023-10-16T09:44:02.793996+0000","last_became_active":"2023-10-16T09:44:01.802077+0000","last_became_peered":"2023-10-16T09:44:01.802077+0000","last_unstale":"2023-10-16T09:44:02.793996+0000","last_undegraded":"2023-10-16T09:44:02.793996+0000","last_fullsized":"2023-10-16T09:44:02.793996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793271+0000","last_change":"2023-10-16T09:44:01.817219+0000","last_active":"2023-10-16T09:44:02.793271+0000","last_peered":"2023-10-16T09:44:02.793271+0000","last_clean":"2023-10-16T09:44:02.793271+0000","last_became_active":"2023-10-16T09:44:01.816945+0000","last_became_peered":"2023-10-16T09:44:01.816945+0000","last_unstale":"2023-10-16T09:44:02.793271+0000","last_undegraded":"2023-10-16T09:44:02.793271+0000","last_fullsized":"2023-10-16T09:44:02.793271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794026+0000","last_change":"2023-10-16T09:44:01.807539+0000","last_active":"2023-10-16T09:44:02.794026+0000","last_peered":"2023-10-16T09:44:02.794026+0000","last_clean":"2023-10-16T09:44:02.794026+0000","last_became_active":"2023-10-16T09:44:01.807282+0000","last_became_peered":"2023-10-16T09:44:01.807282+0000","last_unstale":"2023-10-16T09:44:02.794026+0000","last_undegraded":"2023-10-16T09:44:02.794026+0000","last_fullsized":"2023-10-16T09:44:02.794026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792808+0000","last_change":"2023-10-16T09:44:01.816493+0000","last_active":"2023-10-16T09:44:02.792808+0000","last_peered":"2023-10-16T09:44:02.792808+0000","last_clean":"2023-10-16T09:44:02.792808+0000","last_became_active":"2023-10-16T09:44:01.816006+0000","last_became_peered":"2023-10-16T09:44:01.816006+0000","last_unstale":"2023-10-16T09:44:02.792808+0000","last_undegraded":"2023-10-16T09:44:02.792808+0000","last_fullsized":"2023-10-16T09:44:02.792808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045389+0000","last_change":"2023-10-16T09:44:01.802910+0000","last_active":"2023-10-16T09:44:03.045389+0000","last_peered":"2023-10-16T09:44:03.045389+0000","last_clean":"2023-10-16T09:44:03.045389+0000","last_became_active":"2023-10-16T09:44:01.802640+0000","last_became_peered":"2023-10-16T09:44:01.802640+0000","last_unstale":"2023-10-16T09:44:03.045389+0000","last_undegraded":"2023-10-16T09:44:03.045389+0000","last_fullsized":"2023-10-16T09:44:03.045389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793935+0000","last_change":"2023-10-16T09:44:01.818818+0000","last_active":"2023-10-16T09:44:02.793935+0000","last_peered":"2023-10-16T09:44:02.793935+0000","last_clean":"2023-10-16T09:44:02.793935+0000","last_became_active":"2023-10-16T09:44:01.818532+0000","last_became_peered":"2023-10-16T09:44:01.818532+0000","last_unstale":"2023-10-16T09:44:02.793935+0000","last_undegraded":"2023-10-16T09:44:02.793935+0000","last_fullsized":"2023-10-16T09:44:02.793935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046230+0000","last_change":"2023-10-16T09:44:01.808945+0000","last_active":"2023-10-16T09:44:03.046230+0000","last_peered":"2023-10-16T09:44:03.046230+0000","last_clean":"2023-10-16T09:44:03.046230+0000","last_became_active":"2023-10-16T09:44:01.808655+0000","last_became_peered":"2023-10-16T09:44:01.808655+0000","last_unstale":"2023-10-16T09:44:03.046230+0000","last_undegraded":"2023-10-16T09:44:03.046230+0000","last_fullsized":"2023-10-16T09:44:03.046230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045896+0000","last_change":"2023-10-16T09:44:01.808213+0000","last_active":"2023-10-16T09:44:03.045896+0000","last_peered":"2023-10-16T09:44:03.045896+0000","last_clean":"2023-10-16T09:44:03.045896+0000","last_became_active":"2023-10-16T09:44:01.807715+0000","last_became_peered":"2023-10-16T09:44:01.807715+0000","last_unstale":"2023-10-16T09:44:03.045896+0000","last_undegraded":"2023-10-16T09:44:03.045896+0000","last_fullsized":"2023-10-16T09:44:03.045896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046320+0000","last_change":"2023-10-16T09:44:01.801296+0000","last_active":"2023-10-16T09:44:03.046320+0000","last_peered":"2023-10-16T09:44:03.046320+0000","last_clean":"2023-10-16T09:44:03.046320+0000","last_became_active":"2023-10-16T09:44:01.800456+0000","last_became_peered":"2023-10-16T09:44:01.800456+0000","last_unstale":"2023-10-16T09:44:03.046320+0000","last_undegraded":"2023-10-16T09:44:03.046320+0000","last_fullsized":"2023-10-16T09:44:03.046320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:56.560951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045786+0000","last_change":"2023-10-16T09:44:01.800574+0000","last_active":"2023-10-16T09:44:03.045786+0000","last_peered":"2023-10-16T09:44:03.045786+0000","last_clean":"2023-10-16T09:44:03.045786+0000","last_became_active":"2023-10-16T09:44:01.800299+0000","last_became_peered":"2023-10-16T09:44:01.800299+0000","last_unstale":"2023-10-16T09:44:03.045786+0000","last_undegraded":"2023-10-16T09:44:03.045786+0000","last_fullsized":"2023-10-16T09:44:03.045786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895941+0000","last_change":"2023-10-16T09:44:01.797679+0000","last_active":"2023-10-16T09:44:02.895941+0000","last_peered":"2023-10-16T09:44:02.895941+0000","last_clean":"2023-10-16T09:44:02.895941+0000","last_became_active":"2023-10-16T09:44:01.797420+0000","last_became_peered":"2023-10-16T09:44:01.797420+0000","last_unstale":"2023-10-16T09:44:02.895941+0000","last_undegraded":"2023-10-16T09:44:02.895941+0000","last_fullsized":"2023-10-16T09:44:02.895941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791819+0000","last_change":"2023-10-16T09:44:01.801979+0000","last_active":"2023-10-16T09:44:02.791819+0000","last_peered":"2023-10-16T09:44:02.791819+0000","last_clean":"2023-10-16T09:44:02.791819+0000","last_became_active":"2023-10-16T09:44:01.801663+0000","last_became_peered":"2023-10-16T09:44:01.801663+0000","last_unstale":"2023-10-16T09:44:02.791819+0000","last_undegraded":"2023-10-16T09:44:02.791819+0000","last_fullsized":"2023-10-16T09:44:02.791819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793275+0000","last_change":"2023-10-16T09:44:01.808377+0000","last_active":"2023-10-16T09:44:02.793275+0000","last_peered":"2023-10-16T09:44:02.793275+0000","last_clean":"2023-10-16T09:44:02.793275+0000","last_became_active":"2023-10-16T09:44:01.808091+0000","last_became_peered":"2023-10-16T09:44:01.808091+0000","last_unstale":"2023-10-16T09:44:02.793275+0000","last_undegraded":"2023-10-16T09:44:02.793275+0000","last_fullsized":"2023-10-16T09:44:02.793275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794670+0000","last_change":"2023-10-16T09:44:01.815509+0000","last_active":"2023-10-16T09:44:02.794670+0000","last_peered":"2023-10-16T09:44:02.794670+0000","last_clean":"2023-10-16T09:44:02.794670+0000","last_became_active":"2023-10-16T09:44:01.815290+0000","last_became_peered":"2023-10-16T09:44:01.815290+0000","last_unstale":"2023-10-16T09:44:02.794670+0000","last_undegraded":"2023-10-16T09:44:02.794670+0000","last_fullsized":"2023-10-16T09:44:02.794670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045622+0000","last_change":"2023-10-16T09:44:01.804696+0000","last_active":"2023-10-16T09:44:03.045622+0000","last_peered":"2023-10-16T09:44:03.045622+0000","last_clean":"2023-10-16T09:44:03.045622+0000","last_became_active":"2023-10-16T09:44:01.804427+0000","last_became_peered":"2023-10-16T09:44:01.804427+0000","last_unstale":"2023-10-16T09:44:03.045622+0000","last_undegraded":"2023-10-16T09:44:03.045622+0000","last_fullsized":"2023-10-16T09:44:03.045622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:06.338018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794299+0000","last_change":"2023-10-16T09:44:01.815090+0000","last_active":"2023-10-16T09:44:02.794299+0000","last_peered":"2023-10-16T09:44:02.794299+0000","last_clean":"2023-10-16T09:44:02.794299+0000","last_became_active":"2023-10-16T09:44:01.814787+0000","last_became_peered":"2023-10-16T09:44:01.814787+0000","last_unstale":"2023-10-16T09:44:02.794299+0000","last_undegraded":"2023-10-16T09:44:02.794299+0000","last_fullsized":"2023-10-16T09:44:02.794299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.820250+0000","last_change":"2023-10-16T09:44:00.797577+0000","last_active":"2023-10-16T09:44:01.820250+0000","last_peered":"2023-10-16T09:44:01.820250+0000","last_clean":"2023-10-16T09:44:01.820250+0000","last_became_active":"2023-10-16T09:44:00.797323+0000","last_became_peered":"2023-10-16T09:44:00.797323+0000","last_unstale":"2023-10-16T09:44:01.820250+0000","last_undegraded":"2023-10-16T09:44:01.820250+0000","last_fullsized":"2023-10-16T09:44:01.820250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046254+0000","last_change":"2023-10-16T09:44:01.801503+0000","last_active":"2023-10-16T09:44:03.046254+0000","last_peered":"2023-10-16T09:44:03.046254+0000","last_clean":"2023-10-16T09:44:03.046254+0000","last_became_active":"2023-10-16T09:44:01.801078+0000","last_became_peered":"2023-10-16T09:44:01.801078+0000","last_unstale":"2023-10-16T09:44:03.046254+0000","last_undegraded":"2023-10-16T09:44:03.046254+0000","last_fullsized":"2023-10-16T09:44:03.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819479+0000","last_change":"2023-10-16T09:44:00.790199+0000","last_active":"2023-10-16T09:44:01.819479+0000","last_peered":"2023-10-16T09:44:01.819479+0000","last_clean":"2023-10-16T09:44:01.819479+0000","last_became_active":"2023-10-16T09:44:00.789914+0000","last_became_peered":"2023-10-16T09:44:00.789914+0000","last_unstale":"2023-10-16T09:44:01.819479+0000","last_undegraded":"2023-10-16T09:44:01.819479+0000","last_fullsized":"2023-10-16T09:44:01.819479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896280+0000","last_change":"2023-10-16T09:44:01.808873+0000","last_active":"2023-10-16T09:44:02.896280+0000","last_peered":"2023-10-16T09:44:02.896280+0000","last_clean":"2023-10-16T09:44:02.896280+0000","last_became_active":"2023-10-16T09:44:01.808557+0000","last_became_peered":"2023-10-16T09:44:01.808557+0000","last_unstale":"2023-10-16T09:44:02.896280+0000","last_undegraded":"2023-10-16T09:44:02.896280+0000","last_fullsized":"2023-10-16T09:44:02.896280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794605+0000","last_change":"2023-10-16T09:44:00.801288+0000","last_active":"2023-10-16T09:44:02.794605+0000","last_peered":"2023-10-16T09:44:02.794605+0000","last_clean":"2023-10-16T09:44:02.794605+0000","last_became_active":"2023-10-16T09:44:00.801008+0000","last_became_peered":"2023-10-16T09:44:00.801008+0000","last_unstale":"2023-10-16T09:44:02.794605+0000","last_undegraded":"2023-10-16T09:44:02.794605+0000","last_fullsized":"2023-10-16T09:44:02.794605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793760+0000","last_change":"2023-10-16T09:44:01.803802+0000","last_active":"2023-10-16T09:44:02.793760+0000","last_peered":"2023-10-16T09:44:02.793760+0000","last_clean":"2023-10-16T09:44:02.793760+0000","last_became_active":"2023-10-16T09:44:01.803480+0000","last_became_peered":"2023-10-16T09:44:01.803480+0000","last_unstale":"2023-10-16T09:44:02.793760+0000","last_undegraded":"2023-10-16T09:44:02.793760+0000","last_fullsized":"2023-10-16T09:44:02.793760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793452+0000","last_change":"2023-10-16T09:44:00.798160+0000","last_active":"2023-10-16T09:44:02.793452+0000","last_peered":"2023-10-16T09:44:02.793452+0000","last_clean":"2023-10-16T09:44:02.793452+0000","last_became_active":"2023-10-16T09:44:00.797833+0000","last_became_peered":"2023-10-16T09:44:00.797833+0000","last_unstale":"2023-10-16T09:44:02.793452+0000","last_undegraded":"2023-10-16T09:44:02.793452+0000","last_fullsized":"2023-10-16T09:44:02.793452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794244+0000","last_change":"2023-10-16T09:44:01.802107+0000","last_active":"2023-10-16T09:44:02.794244+0000","last_peered":"2023-10-16T09:44:02.794244+0000","last_clean":"2023-10-16T09:44:02.794244+0000","last_became_active":"2023-10-16T09:44:01.801774+0000","last_became_peered":"2023-10-16T09:44:01.801774+0000","last_unstale":"2023-10-16T09:44:02.794244+0000","last_undegraded":"2023-10-16T09:44:02.794244+0000","last_fullsized":"2023-10-16T09:44:02.794244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822531+0000","last_change":"2023-10-16T09:44:00.785046+0000","last_active":"2023-10-16T09:44:01.822531+0000","last_peered":"2023-10-16T09:44:01.822531+0000","last_clean":"2023-10-16T09:44:01.822531+0000","last_became_active":"2023-10-16T09:44:00.784767+0000","last_became_peered":"2023-10-16T09:44:00.784767+0000","last_unstale":"2023-10-16T09:44:01.822531+0000","last_undegraded":"2023-10-16T09:44:01.822531+0000","last_fullsized":"2023-10-16T09:44:01.822531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045674+0000","last_change":"2023-10-16T09:44:01.799372+0000","last_active":"2023-10-16T09:44:03.045674+0000","last_peered":"2023-10-16T09:44:03.045674+0000","last_clean":"2023-10-16T09:44:03.045674+0000","last_became_active":"2023-10-16T09:44:01.799056+0000","last_became_peered":"2023-10-16T09:44:01.799056+0000","last_unstale":"2023-10-16T09:44:03.045674+0000","last_undegraded":"2023-10-16T09:44:03.045674+0000","last_fullsized":"2023-10-16T09:44:03.045674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:44:46.117646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045848+0000","last_change":"2023-10-16T09:44:00.800927+0000","last_active":"2023-10-16T09:44:03.045848+0000","last_peered":"2023-10-16T09:44:03.045848+0000","last_clean":"2023-10-16T09:44:03.045848+0000","last_became_active":"2023-10-16T09:44:00.800533+0000","last_became_peered":"2023-10-16T09:44:00.800533+0000","last_unstale":"2023-10-16T09:44:03.045848+0000","last_undegraded":"2023-10-16T09:44:03.045848+0000","last_fullsized":"2023-10-16T09:44:03.045848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896238+0000","last_change":"2023-10-16T09:44:01.795883+0000","last_active":"2023-10-16T09:44:02.896238+0000","last_peered":"2023-10-16T09:44:02.896238+0000","last_clean":"2023-10-16T09:44:02.896238+0000","last_became_active":"2023-10-16T09:44:01.795613+0000","last_became_peered":"2023-10-16T09:44:01.795613+0000","last_unstale":"2023-10-16T09:44:02.896238+0000","last_undegraded":"2023-10-16T09:44:02.896238+0000","last_fullsized":"2023-10-16T09:44:02.896238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792936+0000","last_change":"2023-10-16T09:44:00.800124+0000","last_active":"2023-10-16T09:44:01.792936+0000","last_peered":"2023-10-16T09:44:01.792936+0000","last_clean":"2023-10-16T09:44:01.792936+0000","last_became_active":"2023-10-16T09:44:00.799861+0000","last_became_peered":"2023-10-16T09:44:00.799861+0000","last_unstale":"2023-10-16T09:44:01.792936+0000","last_undegraded":"2023-10-16T09:44:01.792936+0000","last_fullsized":"2023-10-16T09:44:01.792936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045513+0000","last_change":"2023-10-16T09:44:01.799159+0000","last_active":"2023-10-16T09:44:03.045513+0000","last_peered":"2023-10-16T09:44:03.045513+0000","last_clean":"2023-10-16T09:44:03.045513+0000","last_became_active":"2023-10-16T09:44:01.798882+0000","last_became_peered":"2023-10-16T09:44:01.798882+0000","last_unstale":"2023-10-16T09:44:03.045513+0000","last_undegraded":"2023-10-16T09:44:03.045513+0000","last_fullsized":"2023-10-16T09:44:03.045513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798407+0000","last_change":"2023-10-16T09:44:00.797328+0000","last_active":"2023-10-16T09:44:01.798407+0000","last_peered":"2023-10-16T09:44:01.798407+0000","last_clean":"2023-10-16T09:44:01.798407+0000","last_became_active":"2023-10-16T09:44:00.797038+0000","last_became_peered":"2023-10-16T09:44:00.797038+0000","last_unstale":"2023-10-16T09:44:01.798407+0000","last_undegraded":"2023-10-16T09:44:01.798407+0000","last_fullsized":"2023-10-16T09:44:01.798407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:18.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791948+0000","last_change":"2023-10-16T09:44:01.797648+0000","last_active":"2023-10-16T09:44:02.791948+0000","last_peered":"2023-10-16T09:44:02.791948+0000","last_clean":"2023-10-16T09:44:02.791948+0000","last_became_active":"2023-10-16T09:44:01.797374+0000","last_became_peered":"2023-10-16T09:44:01.797374+0000","last_unstale":"2023-10-16T09:44:02.791948+0000","last_undegraded":"2023-10-16T09:44:02.791948+0000","last_fullsized":"2023-10-16T09:44:02.791948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046347+0000","last_change":"2023-10-16T09:44:00.796911+0000","last_active":"2023-10-16T09:44:03.046347+0000","last_peered":"2023-10-16T09:44:03.046347+0000","last_clean":"2023-10-16T09:44:03.046347+0000","last_became_active":"2023-10-16T09:44:00.796666+0000","last_became_peered":"2023-10-16T09:44:00.796666+0000","last_unstale":"2023-10-16T09:44:03.046347+0000","last_undegraded":"2023-10-16T09:44:03.046347+0000","last_fullsized":"2023-10-16T09:44:03.046347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794228+0000","last_change":"2023-10-16T09:44:01.815668+0000","last_active":"2023-10-16T09:44:02.794228+0000","last_peered":"2023-10-16T09:44:02.794228+0000","last_clean":"2023-10-16T09:44:02.794228+0000","last_became_active":"2023-10-16T09:44:01.815327+0000","last_became_peered":"2023-10-16T09:44:01.815327+0000","last_unstale":"2023-10-16T09:44:02.794228+0000","last_undegraded":"2023-10-16T09:44:02.794228+0000","last_fullsized":"2023-10-16T09:44:02.794228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798169+0000","last_change":"2023-10-16T09:44:00.800951+0000","last_active":"2023-10-16T09:44:01.798169+0000","last_peered":"2023-10-16T09:44:01.798169+0000","last_clean":"2023-10-16T09:44:01.798169+0000","last_became_active":"2023-10-16T09:44:00.800685+0000","last_became_peered":"2023-10-16T09:44:00.800685+0000","last_unstale":"2023-10-16T09:44:01.798169+0000","last_undegraded":"2023-10-16T09:44:01.798169+0000","last_fullsized":"2023-10-16T09:44:01.798169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896480+0000","last_change":"2023-10-16T09:44:01.808665+0000","last_active":"2023-10-16T09:44:02.896480+0000","last_peered":"2023-10-16T09:44:02.896480+0000","last_clean":"2023-10-16T09:44:02.896480+0000","last_became_active":"2023-10-16T09:44:01.808229+0000","last_became_peered":"2023-10-16T09:44:01.808229+0000","last_unstale":"2023-10-16T09:44:02.896480+0000","last_undegraded":"2023-10-16T09:44:02.896480+0000","last_fullsized":"2023-10-16T09:44:02.896480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.824237+0000","last_change":"2023-10-16T09:44:00.797832+0000","last_active":"2023-10-16T09:44:01.824237+0000","last_peered":"2023-10-16T09:44:01.824237+0000","last_clean":"2023-10-16T09:44:01.824237+0000","last_became_active":"2023-10-16T09:44:00.797569+0000","last_became_peered":"2023-10-16T09:44:00.797569+0000","last_unstale":"2023-10-16T09:44:01.824237+0000","last_undegraded":"2023-10-16T09:44:01.824237+0000","last_fullsized":"2023-10-16T09:44:01.824237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046342+0000","last_change":"2023-10-16T09:44:01.820469+0000","last_active":"2023-10-16T09:44:03.046342+0000","last_peered":"2023-10-16T09:44:03.046342+0000","last_clean":"2023-10-16T09:44:03.046342+0000","last_became_active":"2023-10-16T09:44:01.820207+0000","last_became_peered":"2023-10-16T09:44:01.820207+0000","last_unstale":"2023-10-16T09:44:03.046342+0000","last_undegraded":"2023-10-16T09:44:03.046342+0000","last_fullsized":"2023-10-16T09:44:03.046342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:16.100248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045792+0000","last_change":"2023-10-16T09:44:00.794450+0000","last_active":"2023-10-16T09:44:03.045792+0000","last_peered":"2023-10-16T09:44:03.045792+0000","last_clean":"2023-10-16T09:44:03.045792+0000","last_became_active":"2023-10-16T09:44:00.794197+0000","last_became_peered":"2023-10-16T09:44:00.794197+0000","last_unstale":"2023-10-16T09:44:03.045792+0000","last_undegraded":"2023-10-16T09:44:03.045792+0000","last_fullsized":"2023-10-16T09:44:03.045792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794469+0000","last_change":"2023-10-16T09:44:01.817900+0000","last_active":"2023-10-16T09:44:02.794469+0000","last_peered":"2023-10-16T09:44:02.794469+0000","last_clean":"2023-10-16T09:44:02.794469+0000","last_became_active":"2023-10-16T09:44:01.817666+0000","last_became_peered":"2023-10-16T09:44:01.817666+0000","last_unstale":"2023-10-16T09:44:02.794469+0000","last_undegraded":"2023-10-16T09:44:02.794469+0000","last_fullsized":"2023-10-16T09:44:02.794469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822707+0000","last_change":"2023-10-16T09:44:00.794006+0000","last_active":"2023-10-16T09:44:01.822707+0000","last_peered":"2023-10-16T09:44:01.822707+0000","last_clean":"2023-10-16T09:44:01.822707+0000","last_became_active":"2023-10-16T09:44:00.793703+0000","last_became_peered":"2023-10-16T09:44:00.793703+0000","last_unstale":"2023-10-16T09:44:01.822707+0000","last_undegraded":"2023-10-16T09:44:01.822707+0000","last_fullsized":"2023-10-16T09:44:01.822707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895868+0000","last_change":"2023-10-16T09:44:01.815023+0000","last_active":"2023-10-16T09:44:02.895868+0000","last_peered":"2023-10-16T09:44:02.895868+0000","last_clean":"2023-10-16T09:44:02.895868+0000","last_became_active":"2023-10-16T09:44:01.814737+0000","last_became_peered":"2023-10-16T09:44:01.814737+0000","last_unstale":"2023-10-16T09:44:02.895868+0000","last_undegraded":"2023-10-16T09:44:02.895868+0000","last_fullsized":"2023-10-16T09:44:02.895868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:59.336084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.799063+0000","last_change":"2023-10-16T09:44:00.792205+0000","last_active":"2023-10-16T09:44:01.799063+0000","last_peered":"2023-10-16T09:44:01.799063+0000","last_clean":"2023-10-16T09:44:01.799063+0000","last_became_active":"2023-10-16T09:44:00.791860+0000","last_became_peered":"2023-10-16T09:44:00.791860+0000","last_unstale":"2023-10-16T09:44:01.799063+0000","last_undegraded":"2023-10-16T09:44:01.799063+0000","last_fullsized":"2023-10-16T09:44:01.799063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:45.078590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793427+0000","last_change":"2023-10-16T09:44:01.820397+0000","last_active":"2023-10-16T09:44:02.793427+0000","last_peered":"2023-10-16T09:44:02.793427+0000","last_clean":"2023-10-16T09:44:02.793427+0000","last_became_active":"2023-10-16T09:44:01.820151+0000","last_became_peered":"2023-10-16T09:44:01.820151+0000","last_unstale":"2023-10-16T09:44:02.793427+0000","last_undegraded":"2023-10-16T09:44:02.793427+0000","last_fullsized":"2023-10-16T09:44:02.793427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045738+0000","last_change":"2023-10-16T09:44:00.799178+0000","last_active":"2023-10-16T09:44:03.045738+0000","last_peered":"2023-10-16T09:44:03.045738+0000","last_clean":"2023-10-16T09:44:03.045738+0000","last_became_active":"2023-10-16T09:44:00.798866+0000","last_became_peered":"2023-10-16T09:44:00.798866+0000","last_unstale":"2023-10-16T09:44:03.045738+0000","last_undegraded":"2023-10-16T09:44:03.045738+0000","last_fullsized":"2023-10-16T09:44:03.045738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045651+0000","last_change":"2023-10-16T09:44:01.805085+0000","last_active":"2023-10-16T09:44:03.045651+0000","last_peered":"2023-10-16T09:44:03.045651+0000","last_clean":"2023-10-16T09:44:03.045651+0000","last_became_active":"2023-10-16T09:44:01.804493+0000","last_became_peered":"2023-10-16T09:44:01.804493+0000","last_unstale":"2023-10-16T09:44:03.045651+0000","last_undegraded":"2023-10-16T09:44:03.045651+0000","last_fullsized":"2023-10-16T09:44:03.045651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819839+0000","last_change":"2023-10-16T09:44:00.788023+0000","last_active":"2023-10-16T09:44:01.819839+0000","last_peered":"2023-10-16T09:44:01.819839+0000","last_clean":"2023-10-16T09:44:01.819839+0000","last_became_active":"2023-10-16T09:44:00.787751+0000","last_became_peered":"2023-10-16T09:44:00.787751+0000","last_unstale":"2023-10-16T09:44:01.819839+0000","last_undegraded":"2023-10-16T09:44:01.819839+0000","last_fullsized":"2023-10-16T09:44:01.819839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:25:10.084733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793056+0000","last_change":"2023-10-16T09:44:01.818990+0000","last_active":"2023-10-16T09:44:02.793056+0000","last_peered":"2023-10-16T09:44:02.793056+0000","last_clean":"2023-10-16T09:44:02.793056+0000","last_became_active":"2023-10-16T09:44:01.818711+0000","last_became_peered":"2023-10-16T09:44:01.818711+0000","last_unstale":"2023-10-16T09:44:02.793056+0000","last_undegraded":"2023-10-16T09:44:02.793056+0000","last_fullsized":"2023-10-16T09:44:02.793056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.817348+0000","last_change":"2023-10-16T09:44:00.788065+0000","last_active":"2023-10-16T09:44:01.817348+0000","last_peered":"2023-10-16T09:44:01.817348+0000","last_clean":"2023-10-16T09:44:01.817348+0000","last_became_active":"2023-10-16T09:44:00.787742+0000","last_became_peered":"2023-10-16T09:44:00.787742+0000","last_unstale":"2023-10-16T09:44:01.817348+0000","last_undegraded":"2023-10-16T09:44:01.817348+0000","last_fullsized":"2023-10-16T09:44:01.817348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:18.670482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794288+0000","last_change":"2023-10-16T09:44:01.821150+0000","last_active":"2023-10-16T09:44:02.794288+0000","last_peered":"2023-10-16T09:44:02.794288+0000","last_clean":"2023-10-16T09:44:02.794288+0000","last_became_active":"2023-10-16T09:44:01.820830+0000","last_became_peered":"2023-10-16T09:44:01.820830+0000","last_unstale":"2023-10-16T09:44:02.794288+0000","last_undegraded":"2023-10-16T09:44:02.794288+0000","last_fullsized":"2023-10-16T09:44:02.794288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792997+0000","last_change":"2023-10-16T09:44:00.790958+0000","last_active":"2023-10-16T09:44:01.792997+0000","last_peered":"2023-10-16T09:44:01.792997+0000","last_clean":"2023-10-16T09:44:01.792997+0000","last_became_active":"2023-10-16T09:44:00.790679+0000","last_became_peered":"2023-10-16T09:44:00.790679+0000","last_unstale":"2023-10-16T09:44:01.792997+0000","last_undegraded":"2023-10-16T09:44:01.792997+0000","last_fullsized":"2023-10-16T09:44:01.792997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:23.552983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794431+0000","last_change":"2023-10-16T09:44:01.801190+0000","last_active":"2023-10-16T09:44:02.794431+0000","last_peered":"2023-10-16T09:44:02.794431+0000","last_clean":"2023-10-16T09:44:02.794431+0000","last_became_active":"2023-10-16T09:44:01.800889+0000","last_became_peered":"2023-10-16T09:44:01.800889+0000","last_unstale":"2023-10-16T09:44:02.794431+0000","last_undegraded":"2023-10-16T09:44:02.794431+0000","last_fullsized":"2023-10-16T09:44:02.794431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798899+0000","last_change":"2023-10-16T09:44:00.792936+0000","last_active":"2023-10-16T09:44:01.798899+0000","last_peered":"2023-10-16T09:44:01.798899+0000","last_clean":"2023-10-16T09:44:01.798899+0000","last_became_active":"2023-10-16T09:44:00.792595+0000","last_became_peered":"2023-10-16T09:44:00.792595+0000","last_unstale":"2023-10-16T09:44:01.798899+0000","last_undegraded":"2023-10-16T09:44:01.798899+0000","last_fullsized":"2023-10-16T09:44:01.798899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:22.500292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045929+0000","last_change":"2023-10-16T09:44:01.800944+0000","last_active":"2023-10-16T09:44:03.045929+0000","last_peered":"2023-10-16T09:44:03.045929+0000","last_clean":"2023-10-16T09:44:03.045929+0000","last_became_active":"2023-10-16T09:44:01.800569+0000","last_became_peered":"2023-10-16T09:44:01.800569+0000","last_unstale":"2023-10-16T09:44:03.045929+0000","last_undegraded":"2023-10-16T09:44:03.045929+0000","last_fullsized":"2023-10-16T09:44:03.045929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:38:40.688657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791775+0000","last_change":"2023-10-16T09:44:01.805039+0000","last_active":"2023-10-16T09:44:02.791775+0000","last_peered":"2023-10-16T09:44:02.791775+0000","last_clean":"2023-10-16T09:44:02.791775+0000","last_became_active":"2023-10-16T09:44:01.804777+0000","last_became_peered":"2023-10-16T09:44:01.804777+0000","last_unstale":"2023-10-16T09:44:02.791775+0000","last_undegraded":"2023-10-16T09:44:02.791775+0000","last_fullsized":"2023-10-16T09:44:02.791775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800545+0000","last_change":"2023-10-16T09:44:00.790368+0000","last_active":"2023-10-16T09:44:01.800545+0000","last_peered":"2023-10-16T09:44:01.800545+0000","last_clean":"2023-10-16T09:44:01.800545+0000","last_became_active":"2023-10-16T09:44:00.790030+0000","last_became_peered":"2023-10-16T09:44:00.790030+0000","last_unstale":"2023-10-16T09:44:01.800545+0000","last_undegraded":"2023-10-16T09:44:01.800545+0000","last_fullsized":"2023-10-16T09:44:01.800545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:19.134455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045870+0000","last_change":"2023-10-16T09:44:01.802895+0000","last_active":"2023-10-16T09:44:03.045870+0000","last_peered":"2023-10-16T09:44:03.045870+0000","last_clean":"2023-10-16T09:44:03.045870+0000","last_became_active":"2023-10-16T09:44:01.802637+0000","last_became_peered":"2023-10-16T09:44:01.802637+0000","last_unstale":"2023-10-16T09:44:03.045870+0000","last_undegraded":"2023-10-16T09:44:03.045870+0000","last_fullsized":"2023-10-16T09:44:03.045870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798097+0000","last_change":"2023-10-16T09:44:00.793146+0000","last_active":"2023-10-16T09:44:01.798097+0000","last_peered":"2023-10-16T09:44:01.798097+0000","last_clean":"2023-10-16T09:44:01.798097+0000","last_became_active":"2023-10-16T09:44:00.792827+0000","last_became_peered":"2023-10-16T09:44:00.792827+0000","last_unstale":"2023-10-16T09:44:01.798097+0000","last_undegraded":"2023-10-16T09:44:01.798097+0000","last_fullsized":"2023-10-16T09:44:01.798097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.803000+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:01.803000+0000","last_peered":"2023-10-16T09:44:01.803000+0000","last_clean":"2023-10-16T09:44:01.803000+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:01.803000+0000","last_undegraded":"2023-10-16T09:44:01.803000+0000","last_fullsized":"2023-10-16T09:44:01.803000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895844+0000","last_change":"2023-10-16T09:44:01.795766+0000","last_active":"2023-10-16T09:44:02.895844+0000","last_peered":"2023-10-16T09:44:02.895844+0000","last_clean":"2023-10-16T09:44:02.895844+0000","last_became_active":"2023-10-16T09:44:01.795429+0000","last_became_peered":"2023-10-16T09:44:01.795429+0000","last_unstale":"2023-10-16T09:44:02.895844+0000","last_undegraded":"2023-10-16T09:44:02.895844+0000","last_fullsized":"2023-10-16T09:44:02.895844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:24.556381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792852+0000","last_change":"2023-10-16T09:44:00.800131+0000","last_active":"2023-10-16T09:44:02.792852+0000","last_peered":"2023-10-16T09:44:02.792852+0000","last_clean":"2023-10-16T09:44:02.792852+0000","last_became_active":"2023-10-16T09:44:00.799868+0000","last_became_peered":"2023-10-16T09:44:00.799868+0000","last_unstale":"2023-10-16T09:44:02.792852+0000","last_undegraded":"2023-10-16T09:44:02.792852+0000","last_fullsized":"2023-10-16T09:44:02.792852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794561+0000","last_change":"2023-10-16T09:44:00.799180+0000","last_active":"2023-10-16T09:44:02.794561+0000","last_peered":"2023-10-16T09:44:02.794561+0000","last_clean":"2023-10-16T09:44:02.794561+0000","last_became_active":"2023-10-16T09:44:00.798901+0000","last_became_peered":"2023-10-16T09:44:00.798901+0000","last_unstale":"2023-10-16T09:44:02.794561+0000","last_undegraded":"2023-10-16T09:44:02.794561+0000","last_fullsized":"2023-10-16T09:44:02.794561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792018+0000","last_change":"2023-10-16T09:44:01.804991+0000","last_active":"2023-10-16T09:44:02.792018+0000","last_peered":"2023-10-16T09:44:02.792018+0000","last_clean":"2023-10-16T09:44:02.792018+0000","last_became_active":"2023-10-16T09:44:01.804694+0000","last_became_peered":"2023-10-16T09:44:01.804694+0000","last_unstale":"2023-10-16T09:44:02.792018+0000","last_undegraded":"2023-10-16T09:44:02.792018+0000","last_fullsized":"2023-10-16T09:44:02.792018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790556+0000","last_change":"2023-10-16T09:44:00.800156+0000","last_active":"2023-10-16T09:44:01.790556+0000","last_peered":"2023-10-16T09:44:01.790556+0000","last_clean":"2023-10-16T09:44:01.790556+0000","last_became_active":"2023-10-16T09:44:00.799895+0000","last_became_peered":"2023-10-16T09:44:00.799895+0000","last_unstale":"2023-10-16T09:44:01.790556+0000","last_undegraded":"2023-10-16T09:44:01.790556+0000","last_fullsized":"2023-10-16T09:44:01.790556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046312+0000","last_change":"2023-10-16T09:44:01.819462+0000","last_active":"2023-10-16T09:44:03.046312+0000","last_peered":"2023-10-16T09:44:03.046312+0000","last_clean":"2023-10-16T09:44:03.046312+0000","last_became_active":"2023-10-16T09:44:01.819196+0000","last_became_peered":"2023-10-16T09:44:01.819196+0000","last_unstale":"2023-10-16T09:44:03.046312+0000","last_undegraded":"2023-10-16T09:44:03.046312+0000","last_fullsized":"2023-10-16T09:44:03.046312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046068+0000","last_change":"2023-10-16T09:44:00.801223+0000","last_active":"2023-10-16T09:44:03.046068+0000","last_peered":"2023-10-16T09:44:03.046068+0000","last_clean":"2023-10-16T09:44:03.046068+0000","last_became_active":"2023-10-16T09:44:00.800978+0000","last_became_peered":"2023-10-16T09:44:00.800978+0000","last_unstale":"2023-10-16T09:44:03.046068+0000","last_undegraded":"2023-10-16T09:44:03.046068+0000","last_fullsized":"2023-10-16T09:44:03.046068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896394+0000","last_change":"2023-10-16T09:44:01.798851+0000","last_active":"2023-10-16T09:44:02.896394+0000","last_peered":"2023-10-16T09:44:02.896394+0000","last_clean":"2023-10-16T09:44:02.896394+0000","last_became_active":"2023-10-16T09:44:01.798475+0000","last_became_peered":"2023-10-16T09:44:01.798475+0000","last_unstale":"2023-10-16T09:44:02.896394+0000","last_undegraded":"2023-10-16T09:44:02.896394+0000","last_fullsized":"2023-10-16T09:44:02.896394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792734+0000","last_change":"2023-10-16T09:44:00.799681+0000","last_active":"2023-10-16T09:44:02.792734+0000","last_peered":"2023-10-16T09:44:02.792734+0000","last_clean":"2023-10-16T09:44:02.792734+0000","last_became_active":"2023-10-16T09:44:00.799423+0000","last_became_peered":"2023-10-16T09:44:00.799423+0000","last_unstale":"2023-10-16T09:44:02.792734+0000","last_undegraded":"2023-10-16T09:44:02.792734+0000","last_fullsized":"2023-10-16T09:44:02.792734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046475+0000","last_change":"2023-10-16T09:44:01.806654+0000","last_active":"2023-10-16T09:44:03.046475+0000","last_peered":"2023-10-16T09:44:03.046475+0000","last_clean":"2023-10-16T09:44:03.046475+0000","last_became_active":"2023-10-16T09:44:01.805672+0000","last_became_peered":"2023-10-16T09:44:01.805672+0000","last_unstale":"2023-10-16T09:44:03.046475+0000","last_undegraded":"2023-10-16T09:44:03.046475+0000","last_fullsized":"2023-10-16T09:44:03.046475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:41.878512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790462+0000","last_change":"2023-10-16T09:44:00.797519+0000","last_active":"2023-10-16T09:44:01.790462+0000","last_peered":"2023-10-16T09:44:01.790462+0000","last_clean":"2023-10-16T09:44:01.790462+0000","last_became_active":"2023-10-16T09:44:00.797217+0000","last_became_peered":"2023-10-16T09:44:00.797217+0000","last_unstale":"2023-10-16T09:44:01.790462+0000","last_undegraded":"2023-10-16T09:44:01.790462+0000","last_fullsized":"2023-10-16T09:44:01.790462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045449+0000","last_change":"2023-10-16T09:44:01.798478+0000","last_active":"2023-10-16T09:44:03.045449+0000","last_peered":"2023-10-16T09:44:03.045449+0000","last_clean":"2023-10-16T09:44:03.045449+0000","last_became_active":"2023-10-16T09:44:01.798146+0000","last_became_peered":"2023-10-16T09:44:01.798146+0000","last_unstale":"2023-10-16T09:44:03.045449+0000","last_undegraded":"2023-10-16T09:44:03.045449+0000","last_fullsized":"2023-10-16T09:44:03.045449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790384+0000","last_change":"2023-10-16T09:44:00.791453+0000","last_active":"2023-10-16T09:44:01.790384+0000","last_peered":"2023-10-16T09:44:01.790384+0000","last_clean":"2023-10-16T09:44:01.790384+0000","last_became_active":"2023-10-16T09:44:00.791178+0000","last_became_peered":"2023-10-16T09:44:00.791178+0000","last_unstale":"2023-10-16T09:44:01.790384+0000","last_undegraded":"2023-10-16T09:44:01.790384+0000","last_fullsized":"2023-10-16T09:44:01.790384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046137+0000","last_change":"2023-10-16T09:44:01.801224+0000","last_active":"2023-10-16T09:44:03.046137+0000","last_peered":"2023-10-16T09:44:03.046137+0000","last_clean":"2023-10-16T09:44:03.046137+0000","last_became_active":"2023-10-16T09:44:01.800946+0000","last_became_peered":"2023-10-16T09:44:01.800946+0000","last_unstale":"2023-10-16T09:44:03.046137+0000","last_undegraded":"2023-10-16T09:44:03.046137+0000","last_fullsized":"2023-10-16T09:44:03.046137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793527+0000","last_change":"2023-10-16T09:44:00.799160+0000","last_active":"2023-10-16T09:44:02.793527+0000","last_peered":"2023-10-16T09:44:02.793527+0000","last_clean":"2023-10-16T09:44:02.793527+0000","last_became_active":"2023-10-16T09:44:00.798874+0000","last_became_peered":"2023-10-16T09:44:00.798874+0000","last_unstale":"2023-10-16T09:44:02.793527+0000","last_undegraded":"2023-10-16T09:44:02.793527+0000","last_fullsized":"2023-10-16T09:44:02.793527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045858+0000","last_change":"2023-10-16T09:44:01.802604+0000","last_active":"2023-10-16T09:44:03.045858+0000","last_peered":"2023-10-16T09:44:03.045858+0000","last_clean":"2023-10-16T09:44:03.045858+0000","last_became_active":"2023-10-16T09:44:01.801907+0000","last_became_peered":"2023-10-16T09:44:01.801907+0000","last_unstale":"2023-10-16T09:44:03.045858+0000","last_undegraded":"2023-10-16T09:44:03.045858+0000","last_fullsized":"2023-10-16T09:44:03.045858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800616+0000","last_change":"2023-10-16T09:44:00.796930+0000","last_active":"2023-10-16T09:44:01.800616+0000","last_peered":"2023-10-16T09:44:01.800616+0000","last_clean":"2023-10-16T09:44:01.800616+0000","last_became_active":"2023-10-16T09:44:00.796659+0000","last_became_peered":"2023-10-16T09:44:00.796659+0000","last_unstale":"2023-10-16T09:44:01.800616+0000","last_undegraded":"2023-10-16T09:44:01.800616+0000","last_fullsized":"2023-10-16T09:44:01.800616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794538+0000","last_change":"2023-10-16T09:44:01.820645+0000","last_active":"2023-10-16T09:44:02.794538+0000","last_peered":"2023-10-16T09:44:02.794538+0000","last_clean":"2023-10-16T09:44:02.794538+0000","last_became_active":"2023-10-16T09:44:01.820369+0000","last_became_peered":"2023-10-16T09:44:01.820369+0000","last_unstale":"2023-10-16T09:44:02.794538+0000","last_undegraded":"2023-10-16T09:44:02.794538+0000","last_fullsized":"2023-10-16T09:44:02.794538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822381+0000","last_change":"2023-10-16T09:44:00.785075+0000","last_active":"2023-10-16T09:44:01.822381+0000","last_peered":"2023-10-16T09:44:01.822381+0000","last_clean":"2023-10-16T09:44:01.822381+0000","last_became_active":"2023-10-16T09:44:00.784807+0000","last_became_peered":"2023-10-16T09:44:00.784807+0000","last_unstale":"2023-10-16T09:44:01.822381+0000","last_undegraded":"2023-10-16T09:44:01.822381+0000","last_fullsized":"2023-10-16T09:44:01.822381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793947+0000","last_change":"2023-10-16T09:44:01.804040+0000","last_active":"2023-10-16T09:44:02.793947+0000","last_peered":"2023-10-16T09:44:02.793947+0000","last_clean":"2023-10-16T09:44:02.793947+0000","last_became_active":"2023-10-16T09:44:01.803782+0000","last_became_peered":"2023-10-16T09:44:01.803782+0000","last_unstale":"2023-10-16T09:44:02.793947+0000","last_undegraded":"2023-10-16T09:44:02.793947+0000","last_fullsized":"2023-10-16T09:44:02.793947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819101+0000","last_change":"2023-10-16T09:44:00.790542+0000","last_active":"2023-10-16T09:44:01.819101+0000","last_peered":"2023-10-16T09:44:01.819101+0000","last_clean":"2023-10-16T09:44:01.819101+0000","last_became_active":"2023-10-16T09:44:00.790254+0000","last_became_peered":"2023-10-16T09:44:00.790254+0000","last_unstale":"2023-10-16T09:44:01.819101+0000","last_undegraded":"2023-10-16T09:44:01.819101+0000","last_fullsized":"2023-10-16T09:44:01.819101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794115+0000","last_change":"2023-10-16T09:44:01.820628+0000","last_active":"2023-10-16T09:44:02.794115+0000","last_peered":"2023-10-16T09:44:02.794115+0000","last_clean":"2023-10-16T09:44:02.794115+0000","last_became_active":"2023-10-16T09:44:01.820114+0000","last_became_peered":"2023-10-16T09:44:01.820114+0000","last_unstale":"2023-10-16T09:44:02.794115+0000","last_undegraded":"2023-10-16T09:44:02.794115+0000","last_fullsized":"2023-10-16T09:44:02.794115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046278+0000","last_change":"2023-10-16T09:44:01.805259+0000","last_active":"2023-10-16T09:44:03.046278+0000","last_peered":"2023-10-16T09:44:03.046278+0000","last_clean":"2023-10-16T09:44:03.046278+0000","last_became_active":"2023-10-16T09:44:01.804819+0000","last_became_peered":"2023-10-16T09:44:01.804819+0000","last_unstale":"2023-10-16T09:44:03.046278+0000","last_undegraded":"2023-10-16T09:44:03.046278+0000","last_fullsized":"2023-10-16T09:44:03.046278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045666+0000","last_change":"2023-10-16T09:44:01.808510+0000","last_active":"2023-10-16T09:44:03.045666+0000","last_peered":"2023-10-16T09:44:03.045666+0000","last_clean":"2023-10-16T09:44:03.045666+0000","last_became_active":"2023-10-16T09:44:01.808152+0000","last_became_peered":"2023-10-16T09:44:01.808152+0000","last_unstale":"2023-10-16T09:44:03.045666+0000","last_undegraded":"2023-10-16T09:44:03.045666+0000","last_fullsized":"2023-10-16T09:44:03.045666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:13.616924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793846+0000","last_change":"2023-10-16T09:44:01.808070+0000","last_active":"2023-10-16T09:44:02.793846+0000","last_peered":"2023-10-16T09:44:02.793846+0000","last_clean":"2023-10-16T09:44:02.793846+0000","last_became_active":"2023-10-16T09:44:01.807794+0000","last_became_peered":"2023-10-16T09:44:01.807794+0000","last_unstale":"2023-10-16T09:44:02.793846+0000","last_undegraded":"2023-10-16T09:44:02.793846+0000","last_fullsized":"2023-10-16T09:44:02.793846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896204+0000","last_change":"2023-10-16T09:44:01.796651+0000","last_active":"2023-10-16T09:44:02.896204+0000","last_peered":"2023-10-16T09:44:02.896204+0000","last_clean":"2023-10-16T09:44:02.896204+0000","last_became_active":"2023-10-16T09:44:01.796377+0000","last_became_peered":"2023-10-16T09:44:01.796377+0000","last_unstale":"2023-10-16T09:44:02.896204+0000","last_undegraded":"2023-10-16T09:44:02.896204+0000","last_fullsized":"2023-10-16T09:44:02.896204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046174+0000","last_change":"2023-10-16T09:44:01.803048+0000","last_active":"2023-10-16T09:44:03.046174+0000","last_peered":"2023-10-16T09:44:03.046174+0000","last_clean":"2023-10-16T09:44:03.046174+0000","last_became_active":"2023-10-16T09:44:01.802761+0000","last_became_peered":"2023-10-16T09:44:01.802761+0000","last_unstale":"2023-10-16T09:44:03.046174+0000","last_undegraded":"2023-10-16T09:44:03.046174+0000","last_fullsized":"2023-10-16T09:44:03.046174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791913+0000","last_change":"2023-10-16T09:44:01.797818+0000","last_active":"2023-10-16T09:44:02.791913+0000","last_peered":"2023-10-16T09:44:02.791913+0000","last_clean":"2023-10-16T09:44:02.791913+0000","last_became_active":"2023-10-16T09:44:01.797537+0000","last_became_peered":"2023-10-16T09:44:01.797537+0000","last_unstale":"2023-10-16T09:44:02.791913+0000","last_undegraded":"2023-10-16T09:44:02.791913+0000","last_fullsized":"2023-10-16T09:44:02.791913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046247+0000","last_change":"2023-10-16T09:44:01.821446+0000","last_active":"2023-10-16T09:44:03.046247+0000","last_peered":"2023-10-16T09:44:03.046247+0000","last_clean":"2023-10-16T09:44:03.046247+0000","last_became_active":"2023-10-16T09:44:01.821191+0000","last_became_peered":"2023-10-16T09:44:01.821191+0000","last_unstale":"2023-10-16T09:44:03.046247+0000","last_undegraded":"2023-10-16T09:44:03.046247+0000","last_fullsized":"2023-10-16T09:44:03.046247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046237+0000","last_change":"2023-10-16T09:44:01.801738+0000","last_active":"2023-10-16T09:44:03.046237+0000","last_peered":"2023-10-16T09:44:03.046237+0000","last_clean":"2023-10-16T09:44:03.046237+0000","last_became_active":"2023-10-16T09:44:01.801400+0000","last_became_peered":"2023-10-16T09:44:01.801400+0000","last_unstale":"2023-10-16T09:44:03.046237+0000","last_undegraded":"2023-10-16T09:44:03.046237+0000","last_fullsized":"2023-10-16T09:44:03.046237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793441+0000","last_change":"2023-10-16T09:44:01.797904+0000","last_active":"2023-10-16T09:44:02.793441+0000","last_peered":"2023-10-16T09:44:02.793441+0000","last_clean":"2023-10-16T09:44:02.793441+0000","last_became_active":"2023-10-16T09:44:01.797424+0000","last_became_peered":"2023-10-16T09:44:01.797424+0000","last_unstale":"2023-10-16T09:44:02.793441+0000","last_undegraded":"2023-10-16T09:44:02.793441+0000","last_fullsized":"2023-10-16T09:44:02.793441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045818+0000","last_change":"2023-10-16T09:44:01.802067+0000","last_active":"2023-10-16T09:44:03.045818+0000","last_peered":"2023-10-16T09:44:03.045818+0000","last_clean":"2023-10-16T09:44:03.045818+0000","last_became_active":"2023-10-16T09:44:01.801792+0000","last_became_peered":"2023-10-16T09:44:01.801792+0000","last_unstale":"2023-10-16T09:44:03.045818+0000","last_undegraded":"2023-10-16T09:44:03.045818+0000","last_fullsized":"2023-10-16T09:44:03.045818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793644+0000","last_change":"2023-10-16T09:44:01.819606+0000","last_active":"2023-10-16T09:44:02.793644+0000","last_peered":"2023-10-16T09:44:02.793644+0000","last_clean":"2023-10-16T09:44:02.793644+0000","last_became_active":"2023-10-16T09:44:01.819328+0000","last_became_peered":"2023-10-16T09:44:01.819328+0000","last_unstale":"2023-10-16T09:44:02.793644+0000","last_undegraded":"2023-10-16T09:44:02.793644+0000","last_fullsized":"2023-10-16T09:44:02.793644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896102+0000","last_change":"2023-10-16T09:44:01.818209+0000","last_active":"2023-10-16T09:44:02.896102+0000","last_peered":"2023-10-16T09:44:02.896102+0000","last_clean":"2023-10-16T09:44:02.896102+0000","last_became_active":"2023-10-16T09:44:01.817921+0000","last_became_peered":"2023-10-16T09:44:01.817921+0000","last_unstale":"2023-10-16T09:44:02.896102+0000","last_undegraded":"2023-10-16T09:44:02.896102+0000","last_fullsized":"2023-10-16T09:44:02.896102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045707+0000","last_change":"2023-10-16T09:44:01.802316+0000","last_active":"2023-10-16T09:44:03.045707+0000","last_peered":"2023-10-16T09:44:03.045707+0000","last_clean":"2023-10-16T09:44:03.045707+0000","last_became_active":"2023-10-16T09:44:01.802036+0000","last_became_peered":"2023-10-16T09:44:01.802036+0000","last_unstale":"2023-10-16T09:44:03.045707+0000","last_undegraded":"2023-10-16T09:44:03.045707+0000","last_fullsized":"2023-10-16T09:44:03.045707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:44:06.294 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T09:44:06.294 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-16T09:44:06.295 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-16T09:44:06.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-16T09:44:06.296 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-16T09:44:06.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.447+0000 7ff2c2a32700 1 Processor -- start 2023-10-16T09:44:06.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.447+0000 7ff2c2a32700 1 -- start start 2023-10-16T09:44:06.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2c2a32700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15e6d0 0x7ff2bc0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2c2a32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2bc15eab0 0x7ff2bc15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2c2a32700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2bc06ed00 con 0x7ff2bc0cb0c0 2023-10-16T09:44:06.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2c2a32700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2bc0d02d0 con 0x7ff2bc15eab0 2023-10-16T09:44:06.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2c2a32700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc15ad40 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2bb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2bc15eab0 0x7ff2bc15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2c0fcf700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff2bc0cb0c0 0x7ff2bc15a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47164/0 (socket says 172.21.15.181:47164) 2023-10-16T09:44:06.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.448+0000 7ff2c0fcf700 1 -- 172.21.15.181:0/3480923289 learned_addr learned my addr 172.21.15.181:0/3480923289 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:06.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2bbfff700 1 --2- 172.21.15.181:0/3480923289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15e6d0 0x7ff2bc0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2bb7fe700 1 -- 172.21.15.181:0/3480923289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15e6d0 msgr2=0x7ff2bc0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:06.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2bb7fe700 1 --2- 172.21.15.181:0/3480923289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15e6d0 0x7ff2bc0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2bb7fe700 1 -- 172.21.15.181:0/3480923289 >> v1:172.21.15.181:6789/0 conn(0x7ff2bc0cb0c0 legacy=0x7ff2bc15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2baffd700 1 -- 172.21.15.181:0/3480923289 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595563570 0 0) 0x7ff2bc06ed00 con 0x7ff2bc0cb0c0 2023-10-16T09:44:06.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2bb7fe700 1 -- 172.21.15.181:0/3480923289 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc06e0f0 con 0x7ff2bc15eab0 2023-10-16T09:44:06.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2bbfff700 1 --2- 172.21.15.181:0/3480923289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15e6d0 0x7ff2bc0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:06.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2bb7fe700 1 --2- 172.21.15.181:0/3480923289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15eab0 0x7ff2bc15a0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009840 tx=0x7ff2b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e596a576f43b7709 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2baffd700 1 -- 172.21.15.181:0/3480923289 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0011830 con 0x7ff2bc15eab0 2023-10-16T09:44:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2baffd700 1 -- 172.21.15.181:0/3480923289 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b0011990 con 0x7ff2bc15eab0 2023-10-16T09:44:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.449+0000 7ff2baffd700 1 -- 172.21.15.181:0/3480923289 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0011d20 con 0x7ff2bc15eab0 2023-10-16T09:44:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3480923289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15eab0 msgr2=0x7ff2bc15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3480923289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15eab0 0x7ff2bc15a0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009840 tx=0x7ff2b0009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3480923289 shutdown_connections 2023-10-16T09:44:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3480923289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15e6d0 0x7ff2bc0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3480923289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15eab0 0x7ff2bc15a0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3480923289 >> 172.21.15.181:0/3480923289 conn(0x7ff2bc0d04a0 msgr2=0x7ff2bc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3480923289 shutdown_connections 2023-10-16T09:44:06.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.450+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3480923289 wait complete. 2023-10-16T09:44:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.451+0000 7ff2c2a32700 1 Processor -- start 2023-10-16T09:44:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.451+0000 7ff2c2a32700 1 -- start start 2023-10-16T09:44:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c2a32700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2bc0cb0c0 0x7ff2bc150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c2a32700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15e6d0 0x7ff2bc1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c2a32700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15eab0 0x7ff2bc157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c2a32700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc150e60 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c2a32700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0b4220 con 0x7ff2bc0cb0c0 2023-10-16T09:44:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c2a32700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0c5ec0 con 0x7ff2bc15eab0 2023-10-16T09:44:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15e6d0 0x7ff2bc1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15e6d0 0x7ff2bc1571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36090/0 (socket says 172.21.15.181:36090) 2023-10-16T09:44:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 -- 172.21.15.181:0/3790279744 learned_addr learned my addr 172.21.15.181:0/3790279744 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c0fcf700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15eab0 0x7ff2bc157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bbfff700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2bc0cb0c0 0x7ff2bc150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 -- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15eab0 msgr2=0x7ff2bc157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15eab0 0x7ff2bc157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 -- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2bc0cb0c0 msgr2=0x7ff2bc150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2bc0cb0c0 0x7ff2bc150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 -- 172.21.15.181:0/3790279744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00094a0 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bbfff700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2bc0cb0c0 0x7ff2bc150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2bb7fe700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15e6d0 0x7ff2bc1571f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00090b0 tx=0x7ff2b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2b8ff9700 1 -- 172.21.15.181:0/3790279744 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00038a0 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2b8ff9700 1 -- 172.21.15.181:0/3790279744 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b001f070 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.452+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc0c61a0 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.453+0000 7ff2b8ff9700 1 -- 172.21.15.181:0/3790279744 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0003c60 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.453+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc0c6760 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.454+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc067d20 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.454+0000 7ff2b8ff9700 1 -- 172.21.15.181:0/3790279744 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2b001a430 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.454+0000 7ff2b8ff9700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff2a806af00 0x7ff2a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.454+0000 7ff2b8ff9700 1 -- 172.21.15.181:0/3790279744 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2b0021070 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.454+0000 7ff2bbfff700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff2a806af00 0x7ff2a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.455+0000 7ff2bbfff700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff2a806af00 0x7ff2a806d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac006fd0 tx=0x7ff2ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.459+0000 7ff2b8ff9700 1 -- 172.21.15.181:0/3790279744 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff2b00561a0 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.708+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff2bc0cb610 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.711+0000 7ff2b8ff9700 1 -- 172.21.15.181:0/3790279744 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171037 (secure 0 0 0) 0x7ff2b00597c0 con 0x7ff2bc15e6d0 2023-10-16T09:44:06.714 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:06.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff2a806af00 msgr2=0x7ff2a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff2a806af00 0x7ff2a806d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac006fd0 tx=0x7ff2ac008040 comp rx=0 tx=0).stop 2023-10-16T09:44:06.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15e6d0 msgr2=0x7ff2bc1571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15e6d0 0x7ff2bc1571f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00090b0 tx=0x7ff2b0004750 comp rx=0 tx=0).stop 2023-10-16T09:44:06.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 shutdown_connections 2023-10-16T09:44:06.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff2a806af00 0x7ff2a806d3c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2bc15eab0 0x7ff2bc157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2bc0cb0c0 0x7ff2bc150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 --2- 172.21.15.181:0/3790279744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2bc15e6d0 0x7ff2bc1571f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 >> 172.21.15.181:0/3790279744 conn(0x7ff2bc0d04a0 msgr2=0x7ff2bc0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.713+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 shutdown_connections 2023-10-16T09:44:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.714+0000 7ff2c2a32700 1 -- 172.21.15.181:0/3790279744 wait complete. 2023-10-16T09:44:06.727 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2039940952},{"type":"v1","addr":"172.21.15.181:6833","nonce":2039940952}]},"active_addr":"172.21.15.181:6833/2039940952","active_change":"2023-10-16T09:43:49.670235+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":4251066643}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2848692303}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2080914758}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3764257785}]}]} 2023-10-16T09:44:06.731 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-16T09:44:06.731 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-16T09:44:06.732 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.851+0000 7f7135f21700 1 Processor -- start 2023-10-16T09:44:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.851+0000 7f7135f21700 1 -- start start 2023-10-16T09:44:06.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f7135f21700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f713015e6d0 0x7f71300d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f7135f21700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f713015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f7135f21700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713006ed00 con 0x7f713015eab0 2023-10-16T09:44:06.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f7135f21700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71300d02d0 con 0x7f713015e6d0 2023-10-16T09:44:06.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f7135f21700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713015ad40 con 0x7f71300cb0c0 2023-10-16T09:44:06.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f712ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f713015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f712f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f713015e6d0 0x7f71300d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f712effd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f713015eab0 0x7f713015a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59350/0 (socket says 172.21.15.181:59350) 2023-10-16T09:44:06.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.852+0000 7f712effd700 1 -- 172.21.15.181:0/334204842 learned_addr learned my addr 172.21.15.181:0/334204842 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:06.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712e7fc700 1 -- 172.21.15.181:0/334204842 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186018751 0 0) 0x7f713006ed00 con 0x7f713015eab0 2023-10-16T09:44:06.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712e7fc700 1 -- 172.21.15.181:0/334204842 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c0035b0 con 0x7f713015eab0 2023-10-16T09:44:06.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712ffff700 1 -- 172.21.15.181:0/334204842 >> v2:172.21.15.190:3300/0 conn(0x7f713015e6d0 msgr2=0x7f71300d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:06.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712ffff700 1 --2- 172.21.15.181:0/334204842 >> v2:172.21.15.190:3300/0 conn(0x7f713015e6d0 0x7f71300d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712ffff700 1 -- 172.21.15.181:0/334204842 >> v1:172.21.15.190:6789/0 conn(0x7f713015eab0 legacy=0x7f713015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712ffff700 1 -- 172.21.15.181:0/334204842 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713006e0f0 con 0x7f71300cb0c0 2023-10-16T09:44:06.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712f7fe700 1 --2- 172.21.15.181:0/334204842 >> v2:172.21.15.190:3300/0 conn(0x7f713015e6d0 0x7f71300d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:06.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712ffff700 1 --2- 172.21.15.181:0/334204842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f713015a800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7120009840 tx=0x7f7120009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8834095c000dbf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712e7fc700 1 -- 172.21.15.181:0/334204842 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120011830 con 0x7f71300cb0c0 2023-10-16T09:44:06.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712e7fc700 1 -- 172.21.15.181:0/334204842 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120011990 con 0x7f71300cb0c0 2023-10-16T09:44:06.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.853+0000 7f712e7fc700 1 -- 172.21.15.181:0/334204842 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120011d20 con 0x7f71300cb0c0 2023-10-16T09:44:06.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 -- 172.21.15.181:0/334204842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 msgr2=0x7f713015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 --2- 172.21.15.181:0/334204842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f713015a800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7120009840 tx=0x7f7120009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 -- 172.21.15.181:0/334204842 shutdown_connections 2023-10-16T09:44:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 --2- 172.21.15.181:0/334204842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f713015a800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 --2- 172.21.15.181:0/334204842 >> v2:172.21.15.190:3300/0 conn(0x7f713015e6d0 0x7f71300d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 -- 172.21.15.181:0/334204842 >> 172.21.15.181:0/334204842 conn(0x7f71300d04a0 msgr2=0x7f71300d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:06.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 -- 172.21.15.181:0/334204842 shutdown_connections 2023-10-16T09:44:06.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.854+0000 7f7135f21700 1 -- 172.21.15.181:0/334204842 wait complete. 2023-10-16T09:44:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.855+0000 7f7135f21700 1 Processor -- start 2023-10-16T09:44:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.856+0000 7f7135f21700 1 -- start start 2023-10-16T09:44:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.856+0000 7f7135f21700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f71301f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.856+0000 7f7135f21700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f713015e6d0 0x7f71300b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7135f21700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713015eab0 0x7f71301f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7135f21700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71301f44b0 con 0x7f713015eab0 2023-10-16T09:44:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7135f21700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71301f16c0 con 0x7f713015e6d0 2023-10-16T09:44:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7135f21700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71301f19a0 con 0x7f71300cb0c0 2023-10-16T09:44:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713015eab0 0x7f71301f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713015eab0 0x7f71301f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36106/0 (socket says 172.21.15.181:36106) 2023-10-16T09:44:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 -- 172.21.15.181:0/1064466809 learned_addr learned my addr 172.21.15.181:0/1064466809 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712f7fe700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f71301f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712effd700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f713015e6d0 0x7f71300b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 -- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 msgr2=0x7f71301f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f71301f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 -- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f713015e6d0 msgr2=0x7f71300b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f713015e6d0 0x7f71300b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 -- 172.21.15.181:0/1064466809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118009430 con 0x7f713015eab0 2023-10-16T09:44:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f712ffff700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713015eab0 0x7f71301f4c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71200090b0 tx=0x7f7120004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7127fff700 1 -- 172.21.15.181:0/1064466809 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200036a0 con 0x7f713015eab0 2023-10-16T09:44:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7127fff700 1 -- 172.21.15.181:0/1064466809 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712001e070 con 0x7f713015eab0 2023-10-16T09:44:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71200094a0 con 0x7f713015eab0 2023-10-16T09:44:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7127fff700 1 -- 172.21.15.181:0/1064466809 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120003a60 con 0x7f713015eab0 2023-10-16T09:44:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.857+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71301f4700 con 0x7f713015eab0 2023-10-16T09:44:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.858+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7130067d20 con 0x7f713015eab0 2023-10-16T09:44:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.859+0000 7f7127fff700 1 -- 172.21.15.181:0/1064466809 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7120003bc0 con 0x7f713015eab0 2023-10-16T09:44:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.859+0000 7f7127fff700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f711006af00 0x7f711006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.859+0000 7f7127fff700 1 -- 172.21.15.181:0/1064466809 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f712008a490 con 0x7f713015eab0 2023-10-16T09:44:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.859+0000 7f712f7fe700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f711006af00 0x7f711006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.859+0000 7f712f7fe700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f711006af00 0x7f711006d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713006f2d0 tx=0x7f7118008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:06.863+0000 7f7127fff700 1 -- 172.21.15.181:0/1064466809 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f71200555d0 con 0x7f713015eab0 2023-10-16T09:44:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.057+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71300cb610 con 0x7f713015eab0 2023-10-16T09:44:07.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.058+0000 7f7127fff700 1 -- 172.21.15.181:0/1064466809 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f7120058bf0 con 0x7f713015eab0 2023-10-16T09:44:07.062 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:07.062 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:07.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f711006af00 msgr2=0x7f711006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f711006af00 0x7f711006d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713006f2d0 tx=0x7f7118008040 comp rx=0 tx=0).stop 2023-10-16T09:44:07.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713015eab0 msgr2=0x7f71301f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713015eab0 0x7f71301f4c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71200090b0 tx=0x7f7120004750 comp rx=0 tx=0).stop 2023-10-16T09:44:07.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 shutdown_connections 2023-10-16T09:44:07.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f711006af00 0x7f711006d3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71300cb0c0 0x7f71301f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713015eab0 0x7f71301f4c20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 --2- 172.21.15.181:0/1064466809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f713015e6d0 0x7f71300b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 >> 172.21.15.181:0/1064466809 conn(0x7f71300d04a0 msgr2=0x7f71300c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 shutdown_connections 2023-10-16T09:44:07.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.061+0000 7f7135f21700 1 -- 172.21.15.181:0/1064466809 wait complete. 2023-10-16T09:44:07.074 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-16T09:44:07.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:07.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.193+0000 7f7dc6ce6700 1 Processor -- start 2023-10-16T09:44:07.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.193+0000 7f7dc6ce6700 1 -- start start 2023-10-16T09:44:07.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.194+0000 7f7dc6ce6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc015e6d0 0x7f7dc00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.194+0000 7f7dc6ce6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015eab0 0x7f7dc015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.194+0000 7f7dc6ce6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc006ed00 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.194+0000 7f7dc6ce6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc00d02d0 con 0x7f7dc015e6d0 2023-10-16T09:44:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.194+0000 7f7dc6ce6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc015ad40 con 0x7f7dc015eab0 2023-10-16T09:44:07.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.194+0000 7f7dc4a82700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc015e6d0 0x7f7dc00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.194+0000 7f7dc4a82700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc015e6d0 0x7f7dc00d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36110/0 (socket says 172.21.15.181:36110) 2023-10-16T09:44:07.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dc4a82700 1 -- 172.21.15.181:0/2598774125 learned_addr learned my addr 172.21.15.181:0/2598774125 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dbffff700 1 --2- 172.21.15.181:0/2598774125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015eab0 0x7f7dc015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dc4a82700 1 -- 172.21.15.181:0/2598774125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015eab0 msgr2=0x7f7dc015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dc4a82700 1 --2- 172.21.15.181:0/2598774125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015eab0 0x7f7dc015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dc4a82700 1 -- 172.21.15.181:0/2598774125 >> v1:172.21.15.190:6789/0 conn(0x7f7dc00cb0c0 legacy=0x7f7dc015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dc4a82700 1 -- 172.21.15.181:0/2598774125 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc006e0f0 con 0x7f7dc015e6d0 2023-10-16T09:44:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dbffff700 1 --2- 172.21.15.181:0/2598774125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015eab0 0x7f7dc015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dc4a82700 1 --2- 172.21.15.181:0/2598774125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc015e6d0 0x7f7dc00d0fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7db4006fd0 tx=0x7f7db400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808940bc63cf39bb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dbf7fe700 1 -- 172.21.15.181:0/2598774125 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4005e30 con 0x7f7dc015e6d0 2023-10-16T09:44:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dbf7fe700 1 -- 172.21.15.181:0/2598774125 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4004030 con 0x7f7dc015e6d0 2023-10-16T09:44:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.195+0000 7f7dbf7fe700 1 -- 172.21.15.181:0/2598774125 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db40043e0 con 0x7f7dc015e6d0 2023-10-16T09:44:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/2598774125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc015e6d0 msgr2=0x7f7dc00d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/2598774125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc015e6d0 0x7f7dc00d0fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7db4006fd0 tx=0x7f7db400be70 comp rx=0 tx=0).stop 2023-10-16T09:44:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/2598774125 shutdown_connections 2023-10-16T09:44:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/2598774125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc015e6d0 0x7f7dc00d0fc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/2598774125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015eab0 0x7f7dc015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/2598774125 >> 172.21.15.181:0/2598774125 conn(0x7f7dc00d04a0 msgr2=0x7f7dc00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/2598774125 shutdown_connections 2023-10-16T09:44:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.196+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/2598774125 wait complete. 2023-10-16T09:44:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.197+0000 7f7dc6ce6700 1 Processor -- start 2023-10-16T09:44:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.197+0000 7f7dc6ce6700 1 -- start start 2023-10-16T09:44:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc6ce6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc00cb0c0 0x7f7dc014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc6ce6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015e6d0 0x7f7dc00b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc6ce6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc015eab0 0x7f7dc00b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc6ce6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00c5720 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc6ce6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00b4a40 con 0x7f7dc015eab0 2023-10-16T09:44:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc6ce6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00b4d20 con 0x7f7dc015e6d0 2023-10-16T09:44:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc00cb0c0 0x7f7dc014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc00cb0c0 0x7f7dc014da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36114/0 (socket says 172.21.15.181:36114) 2023-10-16T09:44:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 -- 172.21.15.181:0/3267460610 learned_addr learned my addr 172.21.15.181:0/3267460610 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 -- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015e6d0 msgr2=0x7f7dc00b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015e6d0 0x7f7dc00b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 -- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc015eab0 msgr2=0x7f7dc00b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc5283700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc015eab0 0x7f7dc00b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc015eab0 0x7f7dc00b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 -- 172.21.15.181:0/3267460610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db400b7a0 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc5283700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc015eab0 0x7f7dc00b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc4a82700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc00cb0c0 0x7f7dc014da30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7db400beb0 tx=0x7f7db4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dbd7fa700 1 -- 172.21.15.181:0/3267460610 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4020070 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dbd7fa700 1 -- 172.21.15.181:0/3267460610 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db40039b0 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc00c53b0 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.198+0000 7f7dbd7fa700 1 -- 172.21.15.181:0/3267460610 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4003d60 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.199+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc00c4ff0 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.199+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc00c5210 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.200+0000 7f7dbd7fa700 1 -- 172.21.15.181:0/3267460610 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7db4024410 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.200+0000 7f7dbd7fa700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7da406af00 0x7f7da406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.200+0000 7f7dbd7fa700 1 -- 172.21.15.181:0/3267460610 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7db408c800 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.200+0000 7f7dbffff700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7da406af00 0x7f7da406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.201+0000 7f7dbffff700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7da406af00 0x7f7da406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7dc01527c0 tx=0x7f7dac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.205+0000 7f7dbd7fa700 1 -- 172.21.15.181:0/3267460610 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7dc00c5210 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.217 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:44:07.219 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:44:07.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.403+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dc00c5210 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.405+0000 7f7dbd7fa700 1 -- 172.21.15.181:0/3267460610 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f7dc00c5210 con 0x7f7dc00cb0c0 2023-10-16T09:44:07.406 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:07.406 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7da406af00 msgr2=0x7f7da406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7da406af00 0x7f7da406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7dc01527c0 tx=0x7f7dac008040 comp rx=0 tx=0).stop 2023-10-16T09:44:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc00cb0c0 msgr2=0x7f7dc014da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc00cb0c0 0x7f7dc014da30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7db400beb0 tx=0x7f7db4004750 comp rx=0 tx=0).stop 2023-10-16T09:44:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 shutdown_connections 2023-10-16T09:44:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7da406af00 0x7f7da406d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc015eab0 0x7f7dc00b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc00cb0c0 0x7f7dc014da30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 --2- 172.21.15.181:0/3267460610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc015e6d0 0x7f7dc00b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 >> 172.21.15.181:0/3267460610 conn(0x7f7dc00d04a0 msgr2=0x7f7dc01583e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 shutdown_connections 2023-10-16T09:44:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.407+0000 7f7dc6ce6700 1 -- 172.21.15.181:0/3267460610 wait complete. 2023-10-16T09:44:07.420 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-16T09:44:07.421 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-16T09:44:07.421 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-16T09:44:07.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-16T09:44:07.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-16T09:44:07.423 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-16T09:44:07.423 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-16T09:44:07.424 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-16T09:44:07.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.628+0000 7efd6735f700 1 Processor -- start 2023-10-16T09:44:07.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.629+0000 7efd6735f700 1 -- start start 2023-10-16T09:44:07.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.630+0000 7efd6735f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd6006d580 0x7efd600ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.630+0000 7efd6735f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd600cdb10 con 0x7efd6006da80 2023-10-16T09:44:07.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.630+0000 7efd6735f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd600cddf0 con 0x7efd600678d0 2023-10-16T09:44:07.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.630+0000 7efd6735f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd600ce0d0 con 0x7efd6006d580 2023-10-16T09:44:07.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.630+0000 7efd650fb700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efd6006da80 0x7efd601599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47204/0 (socket says 172.21.15.181:47204) 2023-10-16T09:44:07.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.630+0000 7efd650fb700 1 -- 172.21.15.181:0/1094920956 learned_addr learned my addr 172.21.15.181:0/1094920956 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:07.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd658fc700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd6006d580 0x7efd600ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035760615 0 0) 0x7efd600cdb10 con 0x7efd6006da80 2023-10-16T09:44:07.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd4c0035b0 con 0x7efd6006da80 2023-10-16T09:44:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1687158212 0 0) 0x7efd4c0035b0 con 0x7efd6006da80 2023-10-16T09:44:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd6006d580 msgr2=0x7efd600ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd6006d580 0x7efd600ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 >> v1:172.21.15.190:6789/0 conn(0x7efd600678d0 legacy=0x7efd6006ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600ce3b0 con 0x7efd6006da80 2023-10-16T09:44:07.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7efd50001670 con 0x7efd6006da80 2023-10-16T09:44:07.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 >> v1:172.21.15.181:6789/0 conn(0x7efd6006da80 legacy=0x7efd601599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd4c004420 0x7efd4c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd4c0050f0 0x7efd4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd600ce0d0 con 0x7efd4c004420 2023-10-16T09:44:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c00bfb0 con 0x7efd4c003a30 2023-10-16T09:44:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.631+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c00c170 con 0x7efd4c0050f0 2023-10-16T09:44:07.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd650fb700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd650fb700 1 -- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd4c0050f0 msgr2=0x7efd4c00ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:44:07.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd650fb700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd4c0050f0 0x7efd4c00ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd650fb700 1 -- 172.21.15.181:0/1094920956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd4c004420 msgr2=0x7efd4c0048f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:44:07.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd650fb700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd4c004420 0x7efd4c0048f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd650fb700 1 -- 172.21.15.181:0/1094920956 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600ce3b0 con 0x7efd4c003a30 2023-10-16T09:44:07.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd650fb700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd5000c340 tx=0x7efd4c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf29ca4104beff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50005960 con 0x7efd4c003a30 2023-10-16T09:44:07.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd50005ac0 con 0x7efd4c003a30 2023-10-16T09:44:07.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.632+0000 7efd57fff700 1 -- 172.21.15.181:0/1094920956 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50005e70 con 0x7efd4c003a30 2023-10-16T09:44:07.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.633+0000 7efd6735f700 1 -- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd4c003a30 msgr2=0x7efd4c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.633+0000 7efd6735f700 1 --2- 172.21.15.181:0/1094920956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd5000c340 tx=0x7efd4c005880 comp rx=0 tx=0).stop 2023-10-16T09:44:07.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.634+0000 7fc3ad1dd700 1 Processor -- start 2023-10-16T09:44:07.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.634+0000 7fc3ad1dd700 1 -- start start 2023-10-16T09:44:07.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.634+0000 7fe8434d8700 1 Processor -- start 2023-10-16T09:44:07.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fe8434d8700 1 -- start start 2023-10-16T09:44:07.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3ad1dd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc3a815b960 0x7fc3a80b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3ad1dd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 0x7fc3a8071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3ad1dd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a80cd840 con 0x7fc3a80b0ef0 2023-10-16T09:44:07.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3ad1dd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a8071990 con 0x7fc3a815b960 2023-10-16T09:44:07.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3ad1dd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80ce6a0 con 0x7fc3a80ceba0 2023-10-16T09:44:07.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3a759e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc3a80b0ef0 0x7fc3a815f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47210/0 (socket says 172.21.15.181:47210) 2023-10-16T09:44:07.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3a759e700 1 -- 172.21.15.181:0/1872330253 learned_addr learned my addr 172.21.15.181:0/1872330253 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:07.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.635+0000 7fc3a6d9d700 1 --2- 172.21.15.181:0/1872330253 >> v2:172.21.15.190:3300/0 conn(0x7fc3a815b960 0x7fc3a80b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566623885 0 0) 0x7fc3a80cd840 con 0x7fc3a80b0ef0 2023-10-16T09:44:07.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3900035b0 con 0x7fc3a80b0ef0 2023-10-16T09:44:07.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a659c700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 0x7fc3a8071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397017210 0 0) 0x7fc3900035b0 con 0x7fc3a80b0ef0 2023-10-16T09:44:07.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 msgr2=0x7fc3a8071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 0x7fc3a8071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 >> v2:172.21.15.190:3300/0 conn(0x7fc3a815b960 msgr2=0x7fc3a80b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 --2- 172.21.15.181:0/1872330253 >> v2:172.21.15.190:3300/0 conn(0x7fc3a815b960 0x7fc3a80b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe8434d8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe83c0b1600 0x7fe83c0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe8434d8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe83c0ceba0 0x7fe83c071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe8434d8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c0cd840 con 0x7fe83c15b960 2023-10-16T09:44:07.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe8434d8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c071990 con 0x7fe83c0b1600 2023-10-16T09:44:07.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe8434d8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c0ce6a0 con 0x7fe83c0ceba0 2023-10-16T09:44:07.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe840a73700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe83c0b1600 0x7fe83c0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe840a73700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe83c0b1600 0x7fe83c0b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36136/0 (socket says 172.21.15.181:36136) 2023-10-16T09:44:07.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe840a73700 1 -- 172.21.15.181:0/3545462468 learned_addr learned my addr 172.21.15.181:0/3545462468 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe841a75700 1 --2- 172.21.15.181:0/3545462468 >> v2:172.21.15.190:3300/0 conn(0x7fe83c0ceba0 0x7fe83c071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a80cda90 con 0x7fc3a80b0ef0 2023-10-16T09:44:07.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc3a00019f0 con 0x7fc3a80b0ef0 2023-10-16T09:44:07.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 >> v1:172.21.15.181:6789/0 conn(0x7fc3a80b0ef0 legacy=0x7fc3a815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 0x7fc390003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc390003f80 0x7fc390004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc390004d90 0x7fc390005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8071990 con 0x7fc390003f80 2023-10-16T09:44:07.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80ce6a0 con 0x7fc390004d90 2023-10-16T09:44:07.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc390014270 con 0x7fc3a80ceba0 2023-10-16T09:44:07.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a6d9d700 1 --2- 172.21.15.181:0/1872330253 >> v2:172.21.15.190:3300/0 conn(0x7fc3a815b960 0x7fc3a80b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:07.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a759e700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc390004d90 0x7fc390005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe841a75700 1 -- 172.21.15.181:0/3545462468 >> v2:172.21.15.181:3300/0 conn(0x7fe83c0b1600 msgr2=0x7fe83c0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe841a75700 1 --2- 172.21.15.181:0/3545462468 >> v2:172.21.15.181:3300/0 conn(0x7fe83c0b1600 0x7fe83c0b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe841a75700 1 -- 172.21.15.181:0/3545462468 >> v1:172.21.15.190:6789/0 conn(0x7fe83c15b960 legacy=0x7fe83c0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fe841a75700 1 -- 172.21.15.181:0/3545462468 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0cda90 con 0x7fe83c0ceba0 2023-10-16T09:44:07.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a6d9d700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 0x7fc390003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.636+0000 7fc3a659c700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc390003f80 0x7fc390004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7f6326480700 1 Processor -- start 2023-10-16T09:44:07.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7f6326480700 1 -- start start 2023-10-16T09:44:07.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a659c700 1 -- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 msgr2=0x7fc390003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a659c700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3a80ceba0 0x7fc390003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a659c700 1 -- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc390004d90 msgr2=0x7fc390005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a659c700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc390004d90 0x7fc390005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a659c700 1 -- 172.21.15.181:0/1872330253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a80cda90 con 0x7fc390003f80 2023-10-16T09:44:07.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a759e700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc390004d90 0x7fc390005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:07.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fe841a75700 1 --2- 172.21.15.181:0/3545462468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c0ceba0 0x7fe83c071450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0c64d0 tx=0x7fe83400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8c780b4408b0087 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a659c700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc390003f80 0x7fc390004850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009860 tx=0x7fc390005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90fad6064444098 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fe833fff700 1 -- 172.21.15.181:0/3545462468 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400e070 con 0x7fe83c0ceba0 2023-10-16T09:44:07.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fe833fff700 1 -- 172.21.15.181:0/3545462468 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83400bb90 con 0x7fe83c0ceba0 2023-10-16T09:44:07.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fe833fff700 1 -- 172.21.15.181:0/3545462468 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834003a20 con 0x7fe83c0ceba0 2023-10-16T09:44:07.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00c070 con 0x7fc390003f80 2023-10-16T09:44:07.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c004020 con 0x7fc390003f80 2023-10-16T09:44:07.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.637+0000 7fc3a5d9b700 1 -- 172.21.15.181:0/1872330253 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0043b0 con 0x7fc390003f80 2023-10-16T09:44:07.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7f6326480700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f632015b960 0x7f63200b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7fe8434d8700 1 -- 172.21.15.181:0/3545462468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c0ceba0 msgr2=0x7fe83c071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7fe8434d8700 1 --2- 172.21.15.181:0/3545462468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c0ceba0 0x7fe83c071450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0c64d0 tx=0x7fe83400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:07.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7fe8434d8700 1 -- 172.21.15.181:0/3545462468 shutdown_connections 2023-10-16T09:44:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7fe8434d8700 1 --2- 172.21.15.181:0/3545462468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c0ceba0 0x7fe83c071450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7fe8434d8700 1 --2- 172.21.15.181:0/3545462468 >> v2:172.21.15.181:3300/0 conn(0x7fe83c0b1600 0x7fe83c0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7fe8434d8700 1 -- 172.21.15.181:0/3545462468 >> 172.21.15.181:0/3545462468 conn(0x7fe83c06ac90 msgr2=0x7fe83c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7ff9faad5700 1 Processor -- start 2023-10-16T09:44:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7ff9faad5700 1 -- start start 2023-10-16T09:44:07.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/1872330253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc390003f80 msgr2=0x7fc390004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7fc3ad1dd700 1 --2- 172.21.15.181:0/1872330253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc390003f80 0x7fc390004850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009860 tx=0x7fc390005880 comp rx=0 tx=0).stop 2023-10-16T09:44:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9faad5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41622d0 0x7ff9f40cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9faad5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f409b7d0 con 0x7ff9f4161ef0 2023-10-16T09:44:07.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7f6326480700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63200ceba0 0x7f6320071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7f6326480700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7f6326480700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6320071990 con 0x7f63200ceba0 2023-10-16T09:44:07.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7f6326480700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63200b1880 con 0x7f632015b960 2023-10-16T09:44:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.638+0000 7f6326480700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63200cd840 con 0x7f63200b0ef0 2023-10-16T09:44:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f632015b960 0x7f63200b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f632015b960 0x7f63200b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41198/0 (socket says 172.21.15.181:41198) 2023-10-16T09:44:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631ffff700 1 -- 172.21.15.181:0/2832297376 learned_addr learned my addr 172.21.15.181:0/2832297376 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7efd658fc700 1 -- 172.21.15.181:0/1094920956 reap_dead start 2023-10-16T09:44:07.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7fe8434d8700 1 -- 172.21.15.181:0/3545462468 shutdown_connections 2023-10-16T09:44:07.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7efd6735f700 1 -- 172.21.15.181:0/1094920956 shutdown_connections 2023-10-16T09:44:07.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7efd6735f700 1 -- 172.21.15.181:0/1094920956 >> 172.21.15.181:0/1094920956 conn(0x7efd6006a8a0 msgr2=0x7efd6006ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7fc3a759e700 1 -- 172.21.15.181:0/1872330253 reap_dead start 2023-10-16T09:44:07.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9f8871700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff9f4161ef0 0x7ff9f409c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47228/0 (socket says 172.21.15.181:47228) 2023-10-16T09:44:07.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9f8871700 1 -- 172.21.15.181:0/1658463712 learned_addr learned my addr 172.21.15.181:0/1658463712 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:07.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/1872330253 shutdown_connections 2023-10-16T09:44:07.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/1872330253 >> 172.21.15.181:0/1872330253 conn(0x7fc3a806ac90 msgr2=0x7fc3a806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7efd6735f700 1 -- 172.21.15.181:0/1094920956 shutdown_connections 2023-10-16T09:44:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9faad5700 1 -- 172.21.15.181:0/1658463712 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f409ba70 con 0x7ff9f40ce430 2023-10-16T09:44:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9faad5700 1 -- 172.21.15.181:0/1658463712 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40d2080 con 0x7ff9f41622d0 2023-10-16T09:44:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9f3fff700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41622d0 0x7ff9f40cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266161001 0 0) 0x7ff9f409b7d0 con 0x7ff9f4161ef0 2023-10-16T09:44:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e40035b0 con 0x7ff9f4161ef0 2023-10-16T09:44:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992550881 0 0) 0x7ff9f409ba70 con 0x7ff9f40ce430 2023-10-16T09:44:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9f409b7d0 con 0x7ff9f40ce430 2023-10-16T09:44:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362988464 0 0) 0x7ff9e40035b0 con 0x7ff9f4161ef0 2023-10-16T09:44:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41622d0 msgr2=0x7ff9f40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41622d0 0x7ff9f40cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 >> v1:172.21.15.190:6789/0 conn(0x7ff9f40ce430 legacy=0x7ff9f40b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f409bd50 con 0x7ff9f4161ef0 2023-10-16T09:44:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff9e80017f0 con 0x7ff9f4161ef0 2023-10-16T09:44:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 >> v1:172.21.15.181:6789/0 conn(0x7ff9f4161ef0 legacy=0x7ff9f409c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e4004c20 0x7ff9e4005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40037f0 0x7ff9e4003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e40041b0 0x7ff9e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40d2080 con 0x7ff9e40041b0 2023-10-16T09:44:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e400bfb0 con 0x7ff9e40037f0 2023-10-16T09:44:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e400c170 con 0x7ff9e4004c20 2023-10-16T09:44:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f9072700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e40041b0 0x7ff9e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f3fff700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40037f0 0x7ff9e4003c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f9072700 1 -- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e4004c20 msgr2=0x7ff9e4005020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:44:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f9072700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e4004c20 0x7ff9e4005020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f9072700 1 -- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40037f0 msgr2=0x7ff9e4003c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f9072700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40037f0 0x7ff9e4003c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f9072700 1 -- 172.21.15.181:0/1658463712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f409bd50 con 0x7ff9e40041b0 2023-10-16T09:44:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7ff9f3fff700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40037f0 0x7ff9e4003c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7ff9f9072700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e40041b0 0x7ff9e400ba10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00c320 tx=0x7ff9e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ad6bf10460ded8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00ce80 con 0x7ff9e40041b0 2023-10-16T09:44:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec003a00 con 0x7ff9e40041b0 2023-10-16T09:44:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7ff9f37fe700 1 -- 172.21.15.181:0/1658463712 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec003db0 con 0x7ff9e40041b0 2023-10-16T09:44:07.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7efd6735f700 1 -- 172.21.15.181:0/1094920956 wait complete. 2023-10-16T09:44:07.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7efd6735f700 1 Processor -- start 2023-10-16T09:44:07.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7efd6735f700 1 -- start start 2023-10-16T09:44:07.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631f7fe700 1 --2- 172.21.15.181:0/2832297376 >> v2:172.21.15.181:3300/0 conn(0x7f63200ceba0 0x7f6320071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f6324a1d700 1 --2- 172.21.15.181:0/2832297376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631f7fe700 1 -- 172.21.15.181:0/2832297376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 msgr2=0x7f63200b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631f7fe700 1 --2- 172.21.15.181:0/2832297376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631f7fe700 1 -- 172.21.15.181:0/2832297376 >> v2:172.21.15.190:3300/0 conn(0x7f632015b960 msgr2=0x7f63200b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631f7fe700 1 --2- 172.21.15.181:0/2832297376 >> v2:172.21.15.190:3300/0 conn(0x7f632015b960 0x7f63200b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.639+0000 7f631f7fe700 1 -- 172.21.15.181:0/2832297376 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63200ce6a0 con 0x7f63200ceba0 2023-10-16T09:44:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7f631f7fe700 1 --2- 172.21.15.181:0/2832297376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63200ceba0 0x7f6320071450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f632006dc60 tx=0x7f6314009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b7487b0713a485 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7f631effd700 1 -- 172.21.15.181:0/2832297376 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314011830 con 0x7f63200ceba0 2023-10-16T09:44:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7f631effd700 1 -- 172.21.15.181:0/2832297376 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6314011990 con 0x7f63200ceba0 2023-10-16T09:44:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.640+0000 7f631effd700 1 -- 172.21.15.181:0/2832297376 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314011d20 con 0x7f63200ceba0 2023-10-16T09:44:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7f6326480700 1 -- 172.21.15.181:0/2832297376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63200ceba0 msgr2=0x7f6320071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7f6326480700 1 --2- 172.21.15.181:0/2832297376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63200ceba0 0x7f6320071450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f632006dc60 tx=0x7f6314009b20 comp rx=0 tx=0).stop 2023-10-16T09:44:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7f6326480700 1 -- 172.21.15.181:0/2832297376 shutdown_connections 2023-10-16T09:44:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7f6326480700 1 --2- 172.21.15.181:0/2832297376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7f6326480700 1 --2- 172.21.15.181:0/2832297376 >> v2:172.21.15.190:3300/0 conn(0x7f632015b960 0x7f63200b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7f6326480700 1 --2- 172.21.15.181:0/2832297376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63200ceba0 0x7f6320071450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.641+0000 7f6326480700 1 -- 172.21.15.181:0/2832297376 >> 172.21.15.181:0/2832297376 conn(0x7f632006ac90 msgr2=0x7f632006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd6735f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd60158a10 0x7efd601f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd6735f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd601f2b60 0x7efd60169900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd6735f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd601591c0 0x7efd60169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd6735f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd601f46c0 con 0x7efd601f2b60 2023-10-16T09:44:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd6735f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd601f30e0 con 0x7efd60158a10 2023-10-16T09:44:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd6735f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd601f33c0 con 0x7efd601591c0 2023-10-16T09:44:07.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd650fb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd60158a10 0x7efd601f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd650fb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd60158a10 0x7efd601f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41216/0 (socket says 172.21.15.181:41216) 2023-10-16T09:44:07.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7efd650fb700 1 -- 172.21.15.181:0/387450247 learned_addr learned my addr 172.21.15.181:0/387450247 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7ff9faad5700 1 -- 172.21.15.181:0/1658463712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e40041b0 msgr2=0x7ff9e400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7ff9faad5700 1 --2- 172.21.15.181:0/1658463712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9e40041b0 0x7ff9e400ba10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00c320 tx=0x7ff9e4005880 comp rx=0 tx=0).stop 2023-10-16T09:44:07.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7ff9f9072700 1 -- 172.21.15.181:0/1658463712 reap_dead start 2023-10-16T09:44:07.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7ff9faad5700 1 -- 172.21.15.181:0/1658463712 shutdown_connections 2023-10-16T09:44:07.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7ff9faad5700 1 -- 172.21.15.181:0/1658463712 >> 172.21.15.181:0/1658463712 conn(0x7ff9f40d38a0 msgr2=0x7ff9f40d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7ff9faad5700 1 -- 172.21.15.181:0/1658463712 shutdown_connections 2023-10-16T09:44:07.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7ff9faad5700 1 -- 172.21.15.181:0/1658463712 wait complete. 2023-10-16T09:44:07.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd650fb700 1 -- 172.21.15.181:0/387450247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd601591c0 msgr2=0x7efd60169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd650fb700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd601591c0 0x7efd60169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd650fb700 1 -- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd601f2b60 msgr2=0x7efd60169900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:44:07.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd650fb700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd601f2b60 0x7efd60169900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd650fb700 1 -- 172.21.15.181:0/387450247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd601f36a0 con 0x7efd60158a10 2023-10-16T09:44:07.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd650fb700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd60158a10 0x7efd601f4180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efd5000c930 tx=0x7efd50001a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50001ca0 con 0x7efd60158a10 2023-10-16T09:44:07.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd50004930 con 0x7efd60158a10 2023-10-16T09:44:07.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50004cc0 con 0x7efd60158a10 2023-10-16T09:44:07.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd601f3980 con 0x7efd60158a10 2023-10-16T09:44:07.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.642+0000 7fe8434d8700 1 -- 172.21.15.181:0/3545462468 wait complete. 2023-10-16T09:44:07.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.643+0000 7fe8434d8700 1 Processor -- start 2023-10-16T09:44:07.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.644+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6016cbc0 con 0x7efd60158a10 2023-10-16T09:44:07.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.644+0000 7ff9faad5700 1 Processor -- start 2023-10-16T09:44:07.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.644+0000 7ff9faad5700 1 -- start start 2023-10-16T09:44:07.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7f05e1026700 1 Processor -- start 2023-10-16T09:44:07.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.644+0000 7fe8434d8700 1 -- start start 2023-10-16T09:44:07.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe8434d8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe83c0b1600 0x7fe83c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe8434d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83c0ceba0 0x7fe83c0c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe8434d8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c1f5400 0x7fe83c203580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe8434d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c15d2d0 con 0x7fe83c0ceba0 2023-10-16T09:44:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe8434d8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c1f36f0 con 0x7fe83c1f5400 2023-10-16T09:44:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe8434d8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c3550 con 0x7fe83c0b1600 2023-10-16T09:44:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe840a73700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83c0ceba0 0x7fe83c0c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe840a73700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83c0ceba0 0x7fe83c0c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36170/0 (socket says 172.21.15.181:36170) 2023-10-16T09:44:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe840a73700 1 -- 172.21.15.181:0/278008626 learned_addr learned my addr 172.21.15.181:0/278008626 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe841a75700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c1f5400 0x7fe83c203580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe841274700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe83c0b1600 0x7fe83c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7fe840a73700 1 -- 172.21.15.181:0/278008626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe83c0b1600 msgr2=0x7fe83c15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe840a73700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe83c0b1600 0x7fe83c15cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe840a73700 1 -- 172.21.15.181:0/278008626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c1f5400 msgr2=0x7fe83c203580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe840a73700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe83c1f5400 0x7fe83c203580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe840a73700 1 -- 172.21.15.181:0/278008626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009a70 con 0x7fe83c0ceba0 2023-10-16T09:44:07.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9faad5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40041b0 0x7ff9f40cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9faad5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f415f360 0x7ff9f40c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9faad5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41f74e0 0x7ff9f41f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9faad5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40c97a0 con 0x7ff9f415f360 2023-10-16T09:44:07.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9faad5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f40c8dd0 con 0x7ff9e40041b0 2023-10-16T09:44:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9faad5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f415f7e0 con 0x7ff9f41f74e0 2023-10-16T09:44:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9f9072700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41f74e0 0x7ff9f41f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f415f360 0x7ff9f40c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f415f360 0x7ff9f40c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36180/0 (socket says 172.21.15.181:36180) 2023-10-16T09:44:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.645+0000 7ff9f3fff700 1 -- 172.21.15.181:0/2502448833 learned_addr learned my addr 172.21.15.181:0/2502448833 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7ff9f3fff700 1 -- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41f74e0 msgr2=0x7ff9f41f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7ff9f3fff700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f41f74e0 0x7ff9f41f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7ff9f3fff700 1 -- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40041b0 msgr2=0x7ff9f40cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7ff9f3fff700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e40041b0 0x7ff9f40cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7ff9f3fff700 1 -- 172.21.15.181:0/2502448833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f40c81c0 con 0x7ff9f415f360 2023-10-16T09:44:07.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe840a73700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83c0ceba0 0x7fe83c0c3010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe82c012d40 tx=0x7fe82c011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7ff9f3fff700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f415f360 0x7ff9f40c9260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00118b0 tx=0x7ff9e0011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7f6326480700 1 -- 172.21.15.181:0/2832297376 shutdown_connections 2023-10-16T09:44:07.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efd50005030 con 0x7efd60158a10 2023-10-16T09:44:07.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7efd55ffb700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efd4406af00 0x7efd4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd600ce690 con 0x7efd60158a10 2023-10-16T09:44:07.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd5008a5a0 con 0x7efd60158a10 2023-10-16T09:44:07.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7efd55ffb700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd440730a0 0x7efd440754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 --> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd44075b70 con 0x7efd440730a0 2023-10-16T09:44:07.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7f05e1026700 1 -- start start 2023-10-16T09:44:07.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7f6326480700 1 -- 172.21.15.181:0/2832297376 wait complete. 2023-10-16T09:44:07.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7f6326480700 1 Processor -- start 2023-10-16T09:44:07.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00c070 con 0x7fe83c0ceba0 2023-10-16T09:44:07.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c012e20 con 0x7fe83c0ceba0 2023-10-16T09:44:07.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c005a20 con 0x7fe83c0ceba0 2023-10-16T09:44:07.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.646+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c15e0d0 con 0x7fe83c0ceba0 2023-10-16T09:44:07.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c15da80 con 0x7fe83c0ceba0 2023-10-16T09:44:07.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe83c0ce050 con 0x7fe83c0ceba0 2023-10-16T09:44:07.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd50055640 con 0x7efd60158a10 2023-10-16T09:44:07.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7efd658fc700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd440730a0 0x7efd440754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7efd648fa700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efd4406af00 0x7efd4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7efd658fc700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd440730a0 0x7efd440754c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05e1026700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05dc161ef0 0x7f05dc09c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05e1026700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc09cd70 0x7f05dc0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05e1026700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05dc09b7d0 con 0x7f05dc1622d0 2023-10-16T09:44:07.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05e1026700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc0d2080 con 0x7f05dc161ef0 2023-10-16T09:44:07.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05e1026700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc09ba70 con 0x7f05dc09cd70 2023-10-16T09:44:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05dad9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05dc161ef0 0x7f05dc09c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05db59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc09cd70 0x7f05dc0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05db59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc09cd70 0x7f05dc0b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33216/0 (socket says 172.21.15.181:33216) 2023-10-16T09:44:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7f05db59e700 1 -- 172.21.15.181:0/2317468387 learned_addr learned my addr 172.21.15.181:0/2317468387 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7f05db59e700 1 -- 172.21.15.181:0/2317468387 >> v2:172.21.15.190:3300/0 conn(0x7f05dc161ef0 msgr2=0x7f05dc09c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7f05db59e700 1 --2- 172.21.15.181:0/2317468387 >> v2:172.21.15.190:3300/0 conn(0x7f05dc161ef0 0x7f05dc09c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7f05db59e700 1 -- 172.21.15.181:0/2317468387 >> v1:172.21.15.181:6789/0 conn(0x7f05dc1622d0 legacy=0x7f05dc0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7f05db59e700 1 -- 172.21.15.181:0/2317468387 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc09bd50 con 0x7f05dc09cd70 2023-10-16T09:44:07.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7f05db59e700 1 --2- 172.21.15.181:0/2317468387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc09cd70 0x7f05dc0b6d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009e50 tx=0x7f05d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d0c1d8e18cdb6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0004220 con 0x7ff9f415f360 2023-10-16T09:44:07.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0004380 con 0x7ff9f415f360 2023-10-16T09:44:07.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f40c84a0 con 0x7ff9f415f360 2023-10-16T09:44:07.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.647+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00036a0 con 0x7ff9f415f360 2023-10-16T09:44:07.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f40c8a60 con 0x7ff9f415f360 2023-10-16T09:44:07.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7f6326480700 1 -- start start 2023-10-16T09:44:07.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6326480700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6326480700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63200ceba0 0x7f63200c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6326480700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63201f60a0 0x7f63200c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6326480700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63201f6530 con 0x7f63200ceba0 2023-10-16T09:44:07.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6326480700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63201f40a0 con 0x7f63201f60a0 2023-10-16T09:44:07.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6326480700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63201f4380 con 0x7f63200b0ef0 2023-10-16T09:44:07.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6324a1d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63201f60a0 0x7f63200c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6324a1d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63201f60a0 0x7f63200c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41244/0 (socket says 172.21.15.181:41244) 2023-10-16T09:44:07.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f6324a1d700 1 -- 172.21.15.181:0/2694967469 learned_addr learned my addr 172.21.15.181:0/2694967469 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f631ffff700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6324a1d700 1 -- 172.21.15.181:0/2694967469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 msgr2=0x7f63200c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6324a1d700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6324a1d700 1 -- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63200ceba0 msgr2=0x7f63200c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6324a1d700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63200ceba0 0x7f63200c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6324a1d700 1 -- 172.21.15.181:0/2694967469 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63140094a0 con 0x7f63201f60a0 2023-10-16T09:44:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f631ffff700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63200b0ef0 0x7f63200c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6324a1d700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63201f60a0 0x7f63200c2fa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6318007e40 tx=0x7f631800e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f631cff9700 1 -- 172.21.15.181:0/2694967469 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631800ede0 con 0x7f63201f60a0 2023-10-16T09:44:07.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f631cff9700 1 -- 172.21.15.181:0/2694967469 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6318014420 con 0x7f63201f60a0 2023-10-16T09:44:07.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f631cff9700 1 -- 172.21.15.181:0/2694967469 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63180147d0 con 0x7f63201f60a0 2023-10-16T09:44:07.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63201f4660 con 0x7f63201f60a0 2023-10-16T09:44:07.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6320159f90 con 0x7f63201f60a0 2023-10-16T09:44:07.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.648+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/1872330253 shutdown_connections 2023-10-16T09:44:07.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/1872330253 wait complete. 2023-10-16T09:44:07.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7fc3ad1dd700 1 Processor -- start 2023-10-16T09:44:07.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.652+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe82c005b80 con 0x7fe83c0ceba0 2023-10-16T09:44:07.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7efd648fa700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efd4406af00 0x7efd4406d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd580012e0 tx=0x7efd58001710 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== osd.3 v2:172.21.15.181:6817/2943330843 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7efd44075b70 con 0x7efd440730a0 2023-10-16T09:44:07.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7f05d9d9b700 1 -- 172.21.15.181:0/2317468387 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400e070 con 0x7f05dc09cd70 2023-10-16T09:44:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7f05d9d9b700 1 -- 172.21.15.181:0/2317468387 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4003a20 con 0x7f05dc09cd70 2023-10-16T09:44:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.649+0000 7f05d9d9b700 1 -- 172.21.15.181:0/2317468387 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4003dd0 con 0x7f05dc09cd70 2023-10-16T09:44:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f05e1026700 1 -- 172.21.15.181:0/2317468387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc09cd70 msgr2=0x7f05dc0b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f05e1026700 1 --2- 172.21.15.181:0/2317468387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc09cd70 0x7f05dc0b6d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009e50 tx=0x7f05d400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:07.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f05e1026700 1 -- 172.21.15.181:0/2317468387 shutdown_connections 2023-10-16T09:44:07.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f05e1026700 1 --2- 172.21.15.181:0/2317468387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc09cd70 0x7f05dc0b6d20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f05e1026700 1 --2- 172.21.15.181:0/2317468387 >> v2:172.21.15.190:3300/0 conn(0x7f05dc161ef0 0x7f05dc09c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.650+0000 7f05e1026700 1 -- 172.21.15.181:0/2317468387 >> 172.21.15.181:0/2317468387 conn(0x7f05dc0d38a0 msgr2=0x7f05dc0d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f05e1026700 1 -- 172.21.15.181:0/2317468387 shutdown_connections 2023-10-16T09:44:07.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7f05e1026700 1 -- 172.21.15.181:0/2317468387 wait complete. 2023-10-16T09:44:07.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.651+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9f409c310 con 0x7ff9f415f360 2023-10-16T09:44:07.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7f05e1026700 1 Processor -- start 2023-10-16T09:44:07.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.652+0000 7fc3ad1dd700 1 -- start start 2023-10-16T09:44:07.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fc3ad1dd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc390003f80 0x7fc3a814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fc3ad1dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a81eeb10 0x7fc3a80b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fc3ad1dd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a80b4130 0x7fc3a81528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fc3ad1dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80b45e0 con 0x7fc3a81eeb10 2023-10-16T09:44:07.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fc3ad1dd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a814dcd0 con 0x7fc3a80b4130 2023-10-16T09:44:07.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fc3ad1dd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a814dfb0 con 0x7fc390003f80 2023-10-16T09:44:07.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7fc3a6d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc390003f80 0x7fc3a814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.652+0000 7fe831ffb700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe82806b0b0 0x7fe82806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe82c089840 con 0x7fe83c0ceba0 2023-10-16T09:44:07.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fe831ffb700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fe8280732d0 0x7fe8280756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 --> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe828075da0 con 0x7fe8280732d0 2023-10-16T09:44:07.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe82c089ce0 con 0x7fe83c0ceba0 2023-10-16T09:44:07.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7fe841a75700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fe8280732d0 0x7fe8280756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7fe841274700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe82806b0b0 0x7fe82806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fe841274700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe82806b0b0 0x7fe82806d570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0703b0 tx=0x7fe838009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7f05e1026700 1 -- start start 2023-10-16T09:44:07.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05e1026700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05dc09cd70 0x7f05dc163790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05e1026700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05dc161ef0 0x7f05dc15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05e1026700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc1622d0 0x7f05dc0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05e1026700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc163d70 con 0x7f05dc161ef0 2023-10-16T09:44:07.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05e1026700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc164050 con 0x7f05dc09cd70 2023-10-16T09:44:07.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05e1026700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc164330 con 0x7f05dc1622d0 2023-10-16T09:44:07.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05dad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05dc09cd70 0x7f05dc163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05dad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05dc09cd70 0x7f05dc163790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41264/0 (socket says 172.21.15.181:41264) 2023-10-16T09:44:07.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05dad9d700 1 -- 172.21.15.181:0/2033026427 learned_addr learned my addr 172.21.15.181:0/2033026427 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f05db59e700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc1622d0 0x7f05dc0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.653+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff9e0003800 con 0x7ff9f415f360 2023-10-16T09:44:07.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7ff9f17fa700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9d806afd0 0x7ff9d806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9e0089a40 con 0x7ff9f415f360 2023-10-16T09:44:07.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7ff9f17fa700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7ff9d8073170 0x7ff9d8075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 --> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9d8075c40 con 0x7ff9d8073170 2023-10-16T09:44:07.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fe841a75700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fe8280732d0 0x7fe8280756f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7f05dad9d700 1 -- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc1622d0 msgr2=0x7f05dc0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7f05dad9d700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05dc1622d0 0x7f05dc0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7f05dad9d700 1 -- 172.21.15.181:0/2033026427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05dc161ef0 msgr2=0x7f05dc15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7f05dad9d700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05dc161ef0 0x7f05dc15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7f05dad9d700 1 -- 172.21.15.181:0/2033026427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d4009b30 con 0x7f05dc09cd70 2023-10-16T09:44:07.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f631cff9700 1 -- 172.21.15.181:0/2694967469 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6318014930 con 0x7f63201f60a0 2023-10-16T09:44:07.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f631cff9700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f630c06af50 0x7f630c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.654+0000 7f631cff9700 1 -- 172.21.15.181:0/2694967469 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6318089bc0 con 0x7f63201f60a0 2023-10-16T09:44:07.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7f631ffff700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f630c06af50 0x7f630c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7f05dad9d700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05dc09cd70 0x7f05dc163790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e990 tx=0x7f05d000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a80b4130 0x7fc3a81528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a80b4130 0x7fc3a81528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41256/0 (socket says 172.21.15.181:41256) 2023-10-16T09:44:07.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 -- 172.21.15.181:0/3617795278 learned_addr learned my addr 172.21.15.181:0/3617795278 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a659c700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a81eeb10 0x7fc3a80b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 -- 172.21.15.181:0/3617795278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc390003f80 msgr2=0x7fc3a814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc390003f80 0x7fc3a814ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 -- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a81eeb10 msgr2=0x7fc3a80b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a81eeb10 0x7fc3a80b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7fc3a759e700 1 -- 172.21.15.181:0/3617795278 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a814e290 con 0x7fc3a80b4130 2023-10-16T09:44:07.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7ff9f8871700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9d806afd0 0x7ff9d806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.655+0000 7ff9f9072700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7ff9d8073170 0x7ff9d8075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f631ffff700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f630c06af50 0x7f630c06d410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63200703b0 tx=0x7f63100092c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7fc3a759e700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a80b4130 0x7fc3a81528d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80aece0 tx=0x7fc3a0007570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== osd.2 v2:172.21.15.181:6800/4285977707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe828075da0 con 0x7fe8280732d0 2023-10-16T09:44:07.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9e0054ad0 con 0x7ff9f415f360 2023-10-16T09:44:07.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7ff9f9072700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7ff9d8073170 0x7ff9d8075590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7ff9f8871700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9d806afd0 0x7ff9d806d490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40b04e0 tx=0x7ff9e8001130 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f6326480700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f63000015e0 0x7f6300003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 --> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6300006bd0 con 0x7f63000015e0 2023-10-16T09:44:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7fc38f7fe700 1 -- 172.21.15.181:0/3617795278 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0007ba0 con 0x7fc3a80b4130 2023-10-16T09:44:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7fc38f7fe700 1 -- 172.21.15.181:0/3617795278 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0012560 con 0x7fc3a80b4130 2023-10-16T09:44:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7fc38f7fe700 1 -- 172.21.15.181:0/3617795278 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0012910 con 0x7fc3a80b4130 2023-10-16T09:44:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a814e570 con 0x7fc3a80b4130 2023-10-16T09:44:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a81eef90 con 0x7fc3a80b4130 2023-10-16T09:44:07.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f05c37fe700 1 -- 172.21.15.181:0/2033026427 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d00096f0 con 0x7f05dc09cd70 2023-10-16T09:44:07.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f05c37fe700 1 -- 172.21.15.181:0/2033026427 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d000c850 con 0x7f05dc09cd70 2023-10-16T09:44:07.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f05c37fe700 1 -- 172.21.15.181:0/2033026427 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d000eee0 con 0x7f05dc09cd70 2023-10-16T09:44:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc164610 con 0x7f05dc09cd70 2023-10-16T09:44:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.656+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc165470 con 0x7f05dc09cd70 2023-10-16T09:44:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.657+0000 7f631f7fe700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f63000015e0 0x7f6300003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.658+0000 7f631f7fe700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f63000015e0 0x7f6300003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.659+0000 7f631cff9700 1 -- 172.21.15.181:0/2694967469 <== osd.0 v2:172.21.15.181:6816/2908478956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6300006bd0 con 0x7f63000015e0 2023-10-16T09:44:07.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.659+0000 7f05c37fe700 1 -- 172.21.15.181:0/2033026427 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f05d000cbc0 con 0x7f05dc09cd70 2023-10-16T09:44:07.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.659+0000 7f05c37fe700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f05c806afd0 0x7f05c806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.659+0000 7f05c37fe700 1 -- 172.21.15.181:0/2033026427 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d0089940 con 0x7f05dc09cd70 2023-10-16T09:44:07.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.658+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== osd.6 v2:172.21.15.190:6800/487839335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff9d8075c40 con 0x7ff9d8073170 2023-10-16T09:44:07.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.660+0000 7f05e1026700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7f05bc0015e0 0x7f05bc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.660+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 --> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05bc006bd0 con 0x7f05bc0015e0 2023-10-16T09:44:07.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.660+0000 7f05db59e700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7f05bc0015e0 0x7f05bc003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.660+0000 7f05db59e700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7f05bc0015e0 0x7f05bc003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.661+0000 7f05c37fe700 1 -- 172.21.15.181:0/2033026427 <== osd.5 v2:172.21.15.190:6808/646354310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f05bc006bd0 con 0x7f05bc0015e0 2023-10-16T09:44:07.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.658+0000 7fc38f7fe700 1 -- 172.21.15.181:0/3617795278 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc3a0012a70 con 0x7fc3a80b4130 2023-10-16T09:44:07.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.659+0000 7fc38f7fe700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc39406af50 0x7fc39406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.659+0000 7fc38f7fe700 1 -- 172.21.15.181:0/3617795278 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a0089f10 con 0x7fc3a80b4130 2023-10-16T09:44:07.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.661+0000 7fc3ad1dd700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fc3880015e0 0x7fc388003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.661+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 --> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc388006bd0 con 0x7fc3880015e0 2023-10-16T09:44:07.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.662+0000 7fcf4a9f3700 1 Processor -- start 2023-10-16T09:44:07.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.662+0000 7fcf4a9f3700 1 -- start start 2023-10-16T09:44:07.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.663+0000 7fcf4a9f3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf440c9b30 0x7fcf4415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.663+0000 7fcf4a9f3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf44159fb0 con 0x7fcf440b0ef0 2023-10-16T09:44:07.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.663+0000 7fcf4a9f3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf44071280 con 0x7fcf44159bd0 2023-10-16T09:44:07.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.663+0000 7fcf4a9f3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440715f0 con 0x7fcf440c9b30 2023-10-16T09:44:07.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.664+0000 7fcf3bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf440c9b30 0x7fcf4415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.664+0000 7fcf3bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf440c9b30 0x7fcf4415d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36234/0 (socket says 172.21.15.181:36234) 2023-10-16T09:44:07.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.664+0000 7fcf3bfff700 1 -- 172.21.15.181:0/1718463509 learned_addr learned my addr 172.21.15.181:0/1718463509 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.664+0000 7fcf3bfff700 1 -- 172.21.15.181:0/1718463509 >> v1:172.21.15.190:6789/0 conn(0x7fcf44159bd0 legacy=0x7fcf440cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.664+0000 7fcf3bfff700 1 -- 172.21.15.181:0/1718463509 >> v1:172.21.15.181:6789/0 conn(0x7fcf440b0ef0 legacy=0x7fcf44071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:07.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.664+0000 7fcf3bfff700 1 -- 172.21.15.181:0/1718463509 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4415df90 con 0x7fcf440c9b30 2023-10-16T09:44:07.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.664+0000 7fcf3bfff700 1 --2- 172.21.15.181:0/1718463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440c9b30 0x7fcf4415d6b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0088f0 tx=0x7fcf2c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61db06a15dea8c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.665+0000 7fcf437fe700 1 -- 172.21.15.181:0/1718463509 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c010070 con 0x7fcf440c9b30 2023-10-16T09:44:07.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.665+0000 7fcf437fe700 1 -- 172.21.15.181:0/1718463509 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00da00 con 0x7fcf440c9b30 2023-10-16T09:44:07.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.665+0000 7fcf437fe700 1 -- 172.21.15.181:0/1718463509 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00dd90 con 0x7fcf440c9b30 2023-10-16T09:44:07.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.666+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/1718463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440c9b30 msgr2=0x7fcf4415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.666+0000 7fcf4a9f3700 1 --2- 172.21.15.181:0/1718463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440c9b30 0x7fcf4415d6b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0088f0 tx=0x7fcf2c00d330 comp rx=0 tx=0).stop 2023-10-16T09:44:07.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.667+0000 7fc3a6d9d700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc39406af50 0x7fc39406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.667+0000 7f05da59c700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f05c806afd0 0x7f05c806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.667+0000 7fc3a6d9d700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc39406af50 0x7fc39406d410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc398009e60 tx=0x7fc398009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.668+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/1718463509 shutdown_connections 2023-10-16T09:44:07.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.668+0000 7fcf4a9f3700 1 --2- 172.21.15.181:0/1718463509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440c9b30 0x7fcf4415d6b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.668+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/1718463509 >> 172.21.15.181:0/1718463509 conn(0x7fcf4406ac90 msgr2=0x7fcf4406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.668+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/1718463509 shutdown_connections 2023-10-16T09:44:07.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.669+0000 7f05da59c700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f05c806afd0 0x7f05c806d490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05dc162710 tx=0x7f05cc007a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.668+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/1718463509 wait complete. 2023-10-16T09:44:07.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.670+0000 7fcf4a9f3700 1 Processor -- start 2023-10-16T09:44:07.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.670+0000 7fcf4a9f3700 1 -- start start 2023-10-16T09:44:07.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.671+0000 7fcf4a9f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440b0ef0 0x7fcf441ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.671+0000 7fcf4a9f3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c9b30 0x7fcf4415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.671+0000 7fcf4a9f3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44159bd0 0x7fcf4415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.671+0000 7fcf4a9f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440cb100 con 0x7fcf440b0ef0 2023-10-16T09:44:07.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.671+0000 7fcf4a9f3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4415a9a0 con 0x7fcf44159bd0 2023-10-16T09:44:07.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.671+0000 7fcf4a9f3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4415ac80 con 0x7fcf440c9b30 2023-10-16T09:44:07.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.671+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440b0ef0 0x7fcf441ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440b0ef0 0x7fcf441ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36236/0 (socket says 172.21.15.181:36236) 2023-10-16T09:44:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 -- 172.21.15.181:0/234322959 learned_addr learned my addr 172.21.15.181:0/234322959 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fc3a659c700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fc3880015e0 0x7fc388003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fc3a659c700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fc3880015e0 0x7fc388003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf48f90700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44159bd0 0x7fcf4415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf3bfff700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c9b30 0x7fcf4415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 -- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c9b30 msgr2=0x7fcf4415a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf440c9b30 0x7fcf4415a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 -- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44159bd0 msgr2=0x7fcf4415ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44159bd0 0x7fcf4415ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 -- 172.21.15.181:0/234322959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c0085d0 con 0x7fcf440b0ef0 2023-10-16T09:44:07.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf48f90700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf44159bd0 0x7fcf4415ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.672+0000 7fcf43fff700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440b0ef0 0x7fcf441ff9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c4a0 tx=0x7fcf3400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.673+0000 7fc38f7fe700 1 -- 172.21.15.181:0/3617795278 <== osd.1 v2:172.21.15.181:6808/3991385762 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc388006bd0 con 0x7fc3880015e0 2023-10-16T09:44:07.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.673+0000 7f1d46065700 1 Processor -- start 2023-10-16T09:44:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.673+0000 7f1d46065700 1 -- start start 2023-10-16T09:44:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.674+0000 7f1d46065700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d400c9b30 0x7f1d4015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.674+0000 7f1d46065700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d40159fb0 con 0x7f1d40159bd0 2023-10-16T09:44:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.674+0000 7f1d46065700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d40071280 con 0x7f1d400b0ef0 2023-10-16T09:44:07.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.674+0000 7f1d46065700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400715f0 con 0x7f1d400c9b30 2023-10-16T09:44:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.674+0000 7f1d3f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1d40159bd0 0x7f1d400cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47320/0 (socket says 172.21.15.181:47320) 2023-10-16T09:44:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.674+0000 7f1d3f7fe700 1 -- 172.21.15.181:0/2878460492 learned_addr learned my addr 172.21.15.181:0/2878460492 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.674+0000 7f1d3effd700 1 --2- 172.21.15.181:0/2878460492 >> v2:172.21.15.190:3300/0 conn(0x7f1d400c9b30 0x7f1d4015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3e7fc700 1 -- 172.21.15.181:0/2878460492 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430535596 0 0) 0x7f1d30001110 con 0x7f1d40159bd0 2023-10-16T09:44:07.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3e7fc700 1 -- 172.21.15.181:0/2878460492 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d280035b0 con 0x7f1d40159bd0 2023-10-16T09:44:07.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3effd700 1 -- 172.21.15.181:0/2878460492 >> v1:172.21.15.190:6789/0 conn(0x7f1d400b0ef0 legacy=0x7f1d40071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3effd700 1 -- 172.21.15.181:0/2878460492 >> v1:172.21.15.181:6789/0 conn(0x7f1d40159bd0 legacy=0x7f1d400cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3effd700 1 -- 172.21.15.181:0/2878460492 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4015df90 con 0x7f1d400c9b30 2023-10-16T09:44:07.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3effd700 1 --2- 172.21.15.181:0/2878460492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d400c9b30 0x7f1d4015d6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d34008840 tx=0x7f1d3400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1975e090adfd78af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3e7fc700 1 -- 172.21.15.181:0/2878460492 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34010070 con 0x7f1d400c9b30 2023-10-16T09:44:07.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3e7fc700 1 -- 172.21.15.181:0/2878460492 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3400d910 con 0x7f1d400c9b30 2023-10-16T09:44:07.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.675+0000 7f1d3e7fc700 1 -- 172.21.15.181:0/2878460492 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400dc80 con 0x7f1d400c9b30 2023-10-16T09:44:07.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.676+0000 7f1d46065700 1 -- 172.21.15.181:0/2878460492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d400c9b30 msgr2=0x7f1d4015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.676+0000 7f1d46065700 1 --2- 172.21.15.181:0/2878460492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d400c9b30 0x7f1d4015d6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d34008840 tx=0x7f1d3400d330 comp rx=0 tx=0).stop 2023-10-16T09:44:07.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.676+0000 7f1d46065700 1 -- 172.21.15.181:0/2878460492 shutdown_connections 2023-10-16T09:44:07.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.676+0000 7f1d46065700 1 --2- 172.21.15.181:0/2878460492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d400c9b30 0x7f1d4015d6b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.676+0000 7f1d46065700 1 -- 172.21.15.181:0/2878460492 >> 172.21.15.181:0/2878460492 conn(0x7f1d4006ac90 msgr2=0x7f1d4006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.678+0000 7fcf417fa700 1 -- 172.21.15.181:0/234322959 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf340054e0 con 0x7fcf440b0ef0 2023-10-16T09:44:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.678+0000 7fcf417fa700 1 -- 172.21.15.181:0/234322959 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34003680 con 0x7fcf440b0ef0 2023-10-16T09:44:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.678+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4415af60 con 0x7fcf440b0ef0 2023-10-16T09:44:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.678+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4415b520 con 0x7fcf440b0ef0 2023-10-16T09:44:07.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.678+0000 7fcf417fa700 1 -- 172.21.15.181:0/234322959 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34003a30 con 0x7fcf440b0ef0 2023-10-16T09:44:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 --> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe83c0cdd70 con 0x7fe8280732d0 2023-10-16T09:44:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 --> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd60169780 con 0x7efd440730a0 2023-10-16T09:44:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7f1d46065700 1 -- 172.21.15.181:0/2878460492 shutdown_connections 2023-10-16T09:44:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 --> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6300005c50 con 0x7f63000015e0 2023-10-16T09:44:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7fe831ffb700 1 -- 172.21.15.181:0/278008626 <== osd.2 v2:172.21.15.181:6800/4285977707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe834003b80 con 0x7fe8280732d0 2023-10-16T09:44:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7efd55ffb700 1 -- 172.21.15.181:0/387450247 <== osd.3 v2:172.21.15.181:6817/2943330843 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efd5c009660 con 0x7efd440730a0 2023-10-16T09:44:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7f631cff9700 1 -- 172.21.15.181:0/2694967469 <== osd.0 v2:172.21.15.181:6816/2908478956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6300005c50 con 0x7f63000015e0 2023-10-16T09:44:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fe8280732d0 msgr2=0x7fe8280756f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fe8280732d0 0x7fe8280756f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe82806b0b0 msgr2=0x7fe82806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe82806b0b0 0x7fe82806d570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0703b0 tx=0x7fe838009040 comp rx=0 tx=0).stop 2023-10-16T09:44:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83c0ceba0 msgr2=0x7fe83c0c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.680+0000 7fcf417fa700 1 -- 172.21.15.181:0/234322959 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf34003b90 con 0x7fcf440b0ef0 2023-10-16T09:44:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7fcf417fa700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcf2806af20 0x7fcf2806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7fcf417fa700 1 -- 172.21.15.181:0/234322959 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf34088a10 con 0x7fcf440b0ef0 2023-10-16T09:44:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7fcf3bfff700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcf2806af20 0x7fcf2806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 --2- 172.21.15.181:0/278008626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe83c0ceba0 0x7fe83c0c3010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe82c012d40 tx=0x7fe82c011620 comp rx=0 tx=0).stop 2023-10-16T09:44:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe841a75700 1 -- 172.21.15.181:0/278008626 reap_dead start 2023-10-16T09:44:07.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 shutdown_connections 2023-10-16T09:44:07.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 >> 172.21.15.181:0/278008626 conn(0x7fe83c06ac90 msgr2=0x7fe83c070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fcf4a9f3700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fcf1c0015e0 0x7fcf1c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 --> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf1c006bd0 con 0x7fcf1c0015e0 2023-10-16T09:44:07.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.681+0000 7f1d46065700 1 -- 172.21.15.181:0/2878460492 wait complete. 2023-10-16T09:44:07.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.682+0000 7fcf3bfff700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcf2806af20 0x7fcf2806d3e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf440c9f70 tx=0x7fcf2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd440730a0 msgr2=0x7efd440754c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd440730a0 0x7efd440754c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 shutdown_connections 2023-10-16T09:44:07.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f63000015e0 msgr2=0x7f6300003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f63000015e0 0x7f6300003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7fe8434d8700 1 -- 172.21.15.181:0/278008626 wait complete. 2023-10-16T09:44:07.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efd4406af00 msgr2=0x7efd4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efd4406af00 0x7efd4406d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd580012e0 tx=0x7efd58001710 comp rx=0 tx=0).stop 2023-10-16T09:44:07.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd60158a10 msgr2=0x7efd601f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 --2- 172.21.15.181:0/387450247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd60158a10 0x7efd601f4180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efd5000c930 tx=0x7efd50001a00 comp rx=0 tx=0).stop 2023-10-16T09:44:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd658fc700 1 -- 172.21.15.181:0/387450247 reap_dead start 2023-10-16T09:44:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7fcf48f90700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fcf1c0015e0 0x7fcf1c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f1d46065700 1 Processor -- start 2023-10-16T09:44:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 shutdown_connections 2023-10-16T09:44:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 >> 172.21.15.181:0/387450247 conn(0x7efd6006a8a0 msgr2=0x7efd600cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 shutdown_connections 2023-10-16T09:44:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7efd6735f700 1 -- 172.21.15.181:0/387450247 wait complete. 2023-10-16T09:44:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f630c06af50 msgr2=0x7f630c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f630c06af50 0x7f630c06d410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63200703b0 tx=0x7f63100092c0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63201f60a0 msgr2=0x7f63200c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 --2- 172.21.15.181:0/2694967469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63201f60a0 0x7f63200c2fa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6318007e40 tx=0x7f631800e3c0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6324a1d700 1 -- 172.21.15.181:0/2694967469 reap_dead start 2023-10-16T09:44:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 shutdown_connections 2023-10-16T09:44:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 >> 172.21.15.181:0/2694967469 conn(0x7f632006ac90 msgr2=0x7f632006b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 shutdown_connections 2023-10-16T09:44:07.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f6326480700 1 -- 172.21.15.181:0/2694967469 wait complete. 2023-10-16T09:44:07.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.686+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 --> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9f409c030 con 0x7ff9d8073170 2023-10-16T09:44:07.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.687+0000 7ff9f17fa700 1 -- 172.21.15.181:0/2502448833 <== osd.6 v2:172.21.15.190:6800/487839335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9ec00cc80 con 0x7ff9d8073170 2023-10-16T09:44:07.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.687+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7ff9d8073170 msgr2=0x7ff9d8075590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.687+0000 7ff9faad5700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7ff9d8073170 0x7ff9d8075590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9d806afd0 msgr2=0x7ff9d806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9d806afd0 0x7ff9d806d490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40b04e0 tx=0x7ff9e8001130 comp rx=0 tx=0).stop 2023-10-16T09:44:07.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f415f360 msgr2=0x7ff9f40c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 --2- 172.21.15.181:0/2502448833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9f415f360 0x7ff9f40c9260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00118b0 tx=0x7ff9e0011bc0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9f9072700 1 -- 172.21.15.181:0/2502448833 reap_dead start 2023-10-16T09:44:07.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 shutdown_connections 2023-10-16T09:44:07.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 >> 172.21.15.181:0/2502448833 conn(0x7ff9f40d38a0 msgr2=0x7ff9f41614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 shutdown_connections 2023-10-16T09:44:07.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.688+0000 7ff9faad5700 1 -- 172.21.15.181:0/2502448833 wait complete. 2023-10-16T09:44:07.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.683+0000 7f1d46065700 1 -- start start 2023-10-16T09:44:07.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f1d46065700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d400b0ef0 0x7f1d40153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f1d46065700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d400c9b30 0x7f1d400c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f1d46065700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d40159bd0 0x7f1d400c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f1d46065700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400cb100 con 0x7f1d400b0ef0 2023-10-16T09:44:07.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f1d46065700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40153f70 con 0x7f1d40159bd0 2023-10-16T09:44:07.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.684+0000 7f1d46065700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4015a150 con 0x7f1d400c9b30 2023-10-16T09:44:07.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d400b0ef0 0x7f1d40153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d400b0ef0 0x7f1d40153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36252/0 (socket says 172.21.15.181:36252) 2023-10-16T09:44:07.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 -- 172.21.15.181:0/711686531 learned_addr learned my addr 172.21.15.181:0/711686531 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:07.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3ffff700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d40159bd0 0x7f1d400c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 -- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d400c9b30 msgr2=0x7f1d400c8da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:44:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d400c9b30 0x7f1d400c8da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 -- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d40159bd0 msgr2=0x7f1d400c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d40159bd0 0x7f1d400c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 -- 172.21.15.181:0/711686531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34008520 con 0x7f1d400b0ef0 2023-10-16T09:44:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3ffff700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d40159bd0 0x7f1d400c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.685+0000 7f1d3f7fe700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d400b0ef0 0x7f1d40153a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1d30001d50 tx=0x7f1d3000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.694+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30005b70 con 0x7f1d400b0ef0 2023-10-16T09:44:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.694+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30005cd0 con 0x7f1d400b0ef0 2023-10-16T09:44:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.694+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300072f0 con 0x7f1d400b0ef0 2023-10-16T09:44:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.694+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d401558e0 con 0x7f1d400b0ef0 2023-10-16T09:44:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.694+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4015a3a0 con 0x7f1d400b0ef0 2023-10-16T09:44:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.695+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d4015e550 con 0x7f1d400b0ef0 2023-10-16T09:44:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.695+0000 7fcf48f90700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fcf1c0015e0 0x7fcf1c003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.696+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d30007450 con 0x7f1d400b0ef0 2023-10-16T09:44:07.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.696+0000 7fcf417fa700 1 -- 172.21.15.181:0/234322959 <== osd.7 v2:172.21.15.190:6818/4234176418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcf1c006bd0 con 0x7fcf1c0015e0 2023-10-16T09:44:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.696+0000 7f1d27fff700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1d2c06af50 0x7f1d2c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.697+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d300aaea0 con 0x7f1d400b0ef0 2023-10-16T09:44:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.697+0000 7f1d27fff700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7f1d2c073170 0x7f1d2c075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.697+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 --> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d2c075c40 con 0x7f1d2c073170 2023-10-16T09:44:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.697+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d300ab340 con 0x7f1d400b0ef0 2023-10-16T09:44:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.697+0000 7f1d3effd700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1d2c06af50 0x7f1d2c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.697+0000 7f1d3ffff700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7f1d2c073170 0x7f1d2c075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.697+0000 7f1d3effd700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1d2c06af50 0x7f1d2c06d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d400c9f70 tx=0x7f1d34012000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.698+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 --> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05bc005c50 con 0x7f05bc0015e0 2023-10-16T09:44:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.698+0000 7f1d3ffff700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7f1d2c073170 0x7f1d2c075590 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.699+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== osd.4 v2:172.21.15.190:6816/4008105976 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1d2c075c40 con 0x7f1d2c073170 2023-10-16T09:44:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.702+0000 7f05c37fe700 1 -- 172.21.15.181:0/2033026427 <== osd.5 v2:172.21.15.190:6808/646354310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f05bc005c50 con 0x7f05bc0015e0 2023-10-16T09:44:07.796 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:07.796 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T09:44:07.797 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:07.798 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T09:44:07.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.708+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7f05bc0015e0 msgr2=0x7f05bc003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.708+0000 7f05e1026700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7f05bc0015e0 0x7f05bc003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.708+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f05c806afd0 msgr2=0x7f05c806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.708+0000 7f05e1026700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f05c806afd0 0x7f05c806d490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f05dc162710 tx=0x7f05cc007a20 comp rx=0 tx=0).stop 2023-10-16T09:44:07.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.708+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05dc09cd70 msgr2=0x7f05dc163790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.708+0000 7f05e1026700 1 --2- 172.21.15.181:0/2033026427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05dc09cd70 0x7f05dc163790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e990 tx=0x7f05d000eca0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.708+0000 7f05db59e700 1 -- 172.21.15.181:0/2033026427 reap_dead start 2023-10-16T09:44:07.800 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:07.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T09:44:07.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.709+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 shutdown_connections 2023-10-16T09:44:07.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.709+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 >> 172.21.15.181:0/2033026427 conn(0x7f05dc0d38a0 msgr2=0x7f05dc0d3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.709+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 shutdown_connections 2023-10-16T09:44:07.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.709+0000 7f05e1026700 1 -- 172.21.15.181:0/2033026427 wait complete. 2023-10-16T09:44:07.803 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:07.804 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T09:44:07.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.714+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 --> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc388005c50 con 0x7fc3880015e0 2023-10-16T09:44:07.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.714+0000 7fc38f7fe700 1 -- 172.21.15.181:0/3617795278 <== osd.1 v2:172.21.15.181:6808/3991385762 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc388005c50 con 0x7fc3880015e0 2023-10-16T09:44:07.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.715+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fc3880015e0 msgr2=0x7fc388003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.715+0000 7fc3ad1dd700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fc3880015e0 0x7fc388003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.715+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc39406af50 msgr2=0x7fc39406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.715+0000 7fc3ad1dd700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc39406af50 0x7fc39406d410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc398009e60 tx=0x7fc398009210 comp rx=0 tx=0).stop 2023-10-16T09:44:07.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.715+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a80b4130 msgr2=0x7fc3a81528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.715+0000 7fc3ad1dd700 1 --2- 172.21.15.181:0/3617795278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3a80b4130 0x7fc3a81528d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80aece0 tx=0x7fc3a0007570 comp rx=0 tx=0).stop 2023-10-16T09:44:07.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.715+0000 7fc3a759e700 1 -- 172.21.15.181:0/3617795278 reap_dead start 2023-10-16T09:44:07.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.716+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 shutdown_connections 2023-10-16T09:44:07.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.716+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 >> 172.21.15.181:0/3617795278 conn(0x7fc3a806ac90 msgr2=0x7fc3a815ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.716+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 shutdown_connections 2023-10-16T09:44:07.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.716+0000 7fc3ad1dd700 1 -- 172.21.15.181:0/3617795278 wait complete. 2023-10-16T09:44:07.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.718+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 --> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf1c005cb0 con 0x7fcf1c0015e0 2023-10-16T09:44:07.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf417fa700 1 -- 172.21.15.181:0/234322959 <== osd.7 v2:172.21.15.190:6818/4234176418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf1c005cb0 con 0x7fcf1c0015e0 2023-10-16T09:44:07.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fcf1c0015e0 msgr2=0x7fcf1c003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fcf1c0015e0 0x7fcf1c003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcf2806af20 msgr2=0x7fcf2806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcf2806af20 0x7fcf2806d3e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf440c9f70 tx=0x7fcf2c008040 comp rx=0 tx=0).stop 2023-10-16T09:44:07.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440b0ef0 msgr2=0x7fcf441ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 --2- 172.21.15.181:0/234322959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf440b0ef0 0x7fcf441ff9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c4a0 tx=0x7fcf3400c7b0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf48f90700 1 -- 172.21.15.181:0/234322959 reap_dead start 2023-10-16T09:44:07.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 shutdown_connections 2023-10-16T09:44:07.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 >> 172.21.15.181:0/234322959 conn(0x7fcf4406ac90 msgr2=0x7fcf440b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 shutdown_connections 2023-10-16T09:44:07.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.722+0000 7fcf4a9f3700 1 -- 172.21.15.181:0/234322959 wait complete. 2023-10-16T09:44:07.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.724+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 --> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d4015e270 con 0x7f1d2c073170 2023-10-16T09:44:07.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d27fff700 1 -- 172.21.15.181:0/711686531 <== osd.4 v2:172.21.15.190:6816/4008105976 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1d38002050 con 0x7f1d2c073170 2023-10-16T09:44:07.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7f1d2c073170 msgr2=0x7f1d2c075590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7f1d2c073170 0x7f1d2c075590 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1d2c06af50 msgr2=0x7f1d2c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1d2c06af50 0x7f1d2c06d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d400c9f70 tx=0x7f1d34012000 comp rx=0 tx=0).stop 2023-10-16T09:44:07.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d400b0ef0 msgr2=0x7f1d40153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:07.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 --2- 172.21.15.181:0/711686531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d400b0ef0 0x7f1d40153a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1d30001d50 tx=0x7f1d3000c9e0 comp rx=0 tx=0).stop 2023-10-16T09:44:07.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d3ffff700 1 -- 172.21.15.181:0/711686531 reap_dead start 2023-10-16T09:44:07.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 shutdown_connections 2023-10-16T09:44:07.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 >> 172.21.15.181:0/711686531 conn(0x7f1d4006ac90 msgr2=0x7f1d40158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:07.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 shutdown_connections 2023-10-16T09:44:07.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:07.725+0000 7f1d46065700 1 -- 172.21.15.181:0/711686531 wait complete. 2023-10-16T09:44:07.816 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:07.816 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T09:44:07.817 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:07.817 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T09:44:07.818 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:07.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T09:44:07.820 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:07.820 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T09:44:08.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.047+0000 7f5b7dde9700 1 Processor -- start 2023-10-16T09:44:08.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.047+0000 7f5b7dde9700 1 -- start start 2023-10-16T09:44:08.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.048+0000 7f5b7dde9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b780cb110 0x7f5b780d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.048+0000 7f5b7dde9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7806ed00 con 0x7f5b7815e460 2023-10-16T09:44:08.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.048+0000 7f5b7dde9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7806e0f0 con 0x7f5b7815a0f0 2023-10-16T09:44:08.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.048+0000 7f5b7dde9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7806e3d0 con 0x7f5b780cb110 2023-10-16T09:44:08.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.048+0000 7f5b76ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b780cb110 0x7f5b780d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b77fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f5b7815a0f0 0x7f5b7815a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59514/0 (socket says 172.21.15.181:59514) 2023-10-16T09:44:08.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b77fff700 1 -- 172.21.15.181:0/428684059 learned_addr learned my addr 172.21.15.181:0/428684059 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:08.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f017d086700 1 Processor -- start 2023-10-16T09:44:08.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f017d086700 1 -- start start 2023-10-16T09:44:08.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058556263 0 0) 0x7f5b7806e0f0 con 0x7f5b7815a0f0 2023-10-16T09:44:08.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b600035b0 con 0x7f5b7815a0f0 2023-10-16T09:44:08.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 249232841 0 0) 0x7f5b600035b0 con 0x7f5b7815a0f0 2023-10-16T09:44:08.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 >> v2:172.21.15.190:3300/0 conn(0x7f5b780cb110 msgr2=0x7f5b780d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 --2- 172.21.15.181:0/428684059 >> v2:172.21.15.190:3300/0 conn(0x7f5b780cb110 0x7f5b780d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 >> v1:172.21.15.181:6789/0 conn(0x7f5b7815e460 legacy=0x7f5b7815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7806e6b0 con 0x7f5b7815a0f0 2023-10-16T09:44:08.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.049+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5b700019f0 con 0x7f5b7815a0f0 2023-10-16T09:44:08.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 >> v1:172.21.15.190:6789/0 conn(0x7f5b7815a0f0 legacy=0x7f5b7815a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b767fc700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b60003a00 0x7f5b60003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b767fc700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b60004850 0x7f5b60004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b767fc700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60005210 0x7f5b6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7806e3d0 con 0x7f5b60004850 2023-10-16T09:44:08.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60014890 con 0x7f5b60005210 2023-10-16T09:44:08.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60014a50 con 0x7f5b60003a00 2023-10-16T09:44:08.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b76ffd700 1 --2- 172.21.15.181:0/428684059 >> v2:172.21.15.190:3300/0 conn(0x7f5b780cb110 0x7f5b780d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b77fff700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60005210 0x7f5b6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b777fe700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b60003a00 0x7f5b60003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017d086700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f017815e460 0x7f01780d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017d086700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01780cb110 0x7f017815e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017d086700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017806ed00 con 0x7f017815a0f0 2023-10-16T09:44:08.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017d086700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017815ed80 con 0x7f017815e460 2023-10-16T09:44:08.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017d086700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017806e0f0 con 0x7f01780cb110 2023-10-16T09:44:08.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01780cb110 0x7f017815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01780cb110 0x7f017815e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33298/0 (socket says 172.21.15.181:33298) 2023-10-16T09:44:08.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017659c700 1 -- 172.21.15.181:0/1299857185 learned_addr learned my addr 172.21.15.181:0/1299857185 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b777fe700 1 -- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60005210 msgr2=0x7f5b6000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b777fe700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60005210 0x7f5b6000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b777fe700 1 -- 172.21.15.181:0/428684059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b60004850 msgr2=0x7f5b60004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b777fe700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b60004850 0x7f5b60004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b777fe700 1 -- 172.21.15.181:0/428684059 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7806e6b0 con 0x7f5b60003a00 2023-10-16T09:44:08.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b77fff700 1 -- 172.21.15.181:0/428684059 reap_dead start 2023-10-16T09:44:08.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f5b777fe700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b60003a00 0x7f5b60003e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0017e0 tx=0x7f5b60005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0a53afefd30314 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017659c700 1 -- 172.21.15.181:0/1299857185 >> v2:172.21.15.190:3300/0 conn(0x7f017815e460 msgr2=0x7f01780d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.050+0000 7f017659c700 1 --2- 172.21.15.181:0/1299857185 >> v2:172.21.15.190:3300/0 conn(0x7f017815e460 0x7f01780d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7f017659c700 1 -- 172.21.15.181:0/1299857185 >> v1:172.21.15.190:6789/0 conn(0x7f017815a0f0 legacy=0x7f017815a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7f017659c700 1 -- 172.21.15.181:0/1299857185 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017806e3d0 con 0x7f01780cb110 2023-10-16T09:44:08.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0012a0 con 0x7f5b60003a00 2023-10-16T09:44:08.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7f5c21f09700 1 Processor -- start 2023-10-16T09:44:08.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0052d0 con 0x7f5b60003a00 2023-10-16T09:44:08.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7f5b767fc700 1 -- 172.21.15.181:0/428684059 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c003680 con 0x7f5b60003a00 2023-10-16T09:44:08.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7fa931549700 1 Processor -- start 2023-10-16T09:44:08.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7fa931549700 1 -- start start 2023-10-16T09:44:08.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa931549700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c06bdc0 0x7fa92c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa931549700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa92c162f20 0x7fa92c0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa931549700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa92c0cf340 0x7fa92c06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa931549700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92c1633a0 con 0x7fa92c0cf340 2023-10-16T09:44:08.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa931549700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92c0665a0 con 0x7fa92c162f20 2023-10-16T09:44:08.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa931549700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c162460 con 0x7fa92c06bdc0 2023-10-16T09:44:08.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa92b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa92c0cf340 0x7fa92c06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa92b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa92c0cf340 0x7fa92c06cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36284/0 (socket says 172.21.15.181:36284) 2023-10-16T09:44:08.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa92b7fe700 1 -- 172.21.15.181:0/2857639896 learned_addr learned my addr 172.21.15.181:0/2857639896 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7fa9227fc700 1 --2- 172.21.15.181:0/2857639896 >> v2:172.21.15.190:3300/0 conn(0x7fa92c162f20 0x7fa92c0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.051+0000 7f5c21f09700 1 -- start start 2023-10-16T09:44:08.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c21f09700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c1c0c3470 0x7f5c1c0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c21f09700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c0c3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c21f09700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c1c0ad110 0x7f5c1c06eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c21f09700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c06dab0 con 0x7f5c1c0c3470 2023-10-16T09:44:08.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c21f09700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c06dd90 con 0x7f5c1c0ad110 2023-10-16T09:44:08.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c21f09700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c06e070 con 0x7f5c1c0af840 2023-10-16T09:44:08.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c1bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c1c0ad110 0x7f5c1c06eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c1bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c1c0ad110 0x7f5c1c06eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41330/0 (socket says 172.21.15.181:41330) 2023-10-16T09:44:08.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c1bfff700 1 -- 172.21.15.181:0/1505690571 learned_addr learned my addr 172.21.15.181:0/1505690571 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f5c1b7fe700 1 --2- 172.21.15.181:0/1505690571 >> v2:172.21.15.181:3300/0 conn(0x7f5c1c0c3470 0x7f5c1c0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f017659c700 1 --2- 172.21.15.181:0/1299857185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01780cb110 0x7f017815e840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f01680098f0 tx=0x7f0168009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ff963d65c6b4bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5b7dde9700 1 -- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b60003a00 msgr2=0x7f5b60003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b60003a00 0x7f5b60003e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0017e0 tx=0x7f5b60005880 comp rx=0 tx=0).stop 2023-10-16T09:44:08.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7fa92b7fe700 1 -- 172.21.15.181:0/2857639896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c06bdc0 msgr2=0x7fa92c0aed90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:44:08.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7fa92b7fe700 1 --2- 172.21.15.181:0/2857639896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c06bdc0 0x7fa92c0aed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7fa92b7fe700 1 -- 172.21.15.181:0/2857639896 >> v2:172.21.15.190:3300/0 conn(0x7fa92c162f20 msgr2=0x7fa92c0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7fa92b7fe700 1 --2- 172.21.15.181:0/2857639896 >> v2:172.21.15.190:3300/0 conn(0x7fa92c162f20 0x7fa92c0cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7fa92b7fe700 1 -- 172.21.15.181:0/2857639896 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c06c1a0 con 0x7fa92c0cf340 2023-10-16T09:44:08.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7fa9227fc700 1 --2- 172.21.15.181:0/2857639896 >> v2:172.21.15.190:3300/0 conn(0x7fa92c162f20 0x7fa92c0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5b7dde9700 1 -- 172.21.15.181:0/428684059 shutdown_connections 2023-10-16T09:44:08.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/428684059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b60003a00 0x7f5b60003e20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5b7dde9700 1 -- 172.21.15.181:0/428684059 >> 172.21.15.181:0/428684059 conn(0x7f5b780d04a0 msgr2=0x7f5b780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5c1b7fe700 1 -- 172.21.15.181:0/1505690571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 msgr2=0x7f5c1c0c3850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:44:08.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5c1b7fe700 1 --2- 172.21.15.181:0/1505690571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c0c3850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5c1b7fe700 1 -- 172.21.15.181:0/1505690571 >> v2:172.21.15.190:3300/0 conn(0x7f5c1c0ad110 msgr2=0x7f5c1c06eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5c1b7fe700 1 --2- 172.21.15.181:0/1505690571 >> v2:172.21.15.190:3300/0 conn(0x7f5c1c0ad110 0x7f5c1c06eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5c1b7fe700 1 -- 172.21.15.181:0/1505690571 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c06e350 con 0x7f5c1c0c3470 2023-10-16T09:44:08.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f5c1b7fe700 1 --2- 172.21.15.181:0/1505690571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0cf7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c071460 tx=0x7f5c10009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3605c02291a64f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7fa92b7fe700 1 --2- 172.21.15.181:0/2857639896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c0cf340 0x7fa92c06cdf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa92c06c730 tx=0x7fa92400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a0f6dc3cdcfbd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f01b39ce700 1 Processor -- start 2023-10-16T09:44:08.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.052+0000 7f01b39ce700 1 -- start start 2023-10-16T09:44:08.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f01b39ce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac08a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f01b39ce700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f01ac0c9cd0 0x7f01ac08a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f01b39ce700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac08adc0 con 0x7f01ac0cc930 2023-10-16T09:44:08.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f01b39ce700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac0ca4d0 con 0x7f01ac0c9cd0 2023-10-16T09:44:08.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f01b39ce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac16ef80 con 0x7f01ac0cc550 2023-10-16T09:44:08.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.053+0000 7f01b176a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac08a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b1f6b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f01ac0cc930 0x7f01ac0b0ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59540/0 (socket says 172.21.15.181:59540) 2023-10-16T09:44:08.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b1f6b700 1 -- 172.21.15.181:0/2271053602 learned_addr learned my addr 172.21.15.181:0/2271053602 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:08.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b0f69700 1 --2- 172.21.15.181:0/2271053602 >> v2:172.21.15.181:3300/0 conn(0x7f01ac0c9cd0 0x7f01ac08a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f5c1a7fc700 1 -- 172.21.15.181:0/1505690571 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10005e30 con 0x7f5c1c0c3470 2023-10-16T09:44:08.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f5c1a7fc700 1 -- 172.21.15.181:0/1505690571 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c10006030 con 0x7f5c1c0c3470 2023-10-16T09:44:08.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f5c1a7fc700 1 -- 172.21.15.181:0/1505690571 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100063c0 con 0x7f5c1c0c3470 2023-10-16T09:44:08.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f5b7dde9700 1 -- 172.21.15.181:0/428684059 shutdown_connections 2023-10-16T09:44:08.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f5b7dde9700 1 -- 172.21.15.181:0/428684059 wait complete. 2023-10-16T09:44:08.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b0f69700 1 -- 172.21.15.181:0/2271053602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 msgr2=0x7f01ac08a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b0f69700 1 --2- 172.21.15.181:0/2271053602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac08a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b0f69700 1 -- 172.21.15.181:0/2271053602 >> v1:172.21.15.190:6789/0 conn(0x7f01ac0cc930 legacy=0x7f01ac0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b0f69700 1 -- 172.21.15.181:0/2271053602 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac16f260 con 0x7f01ac0c9cd0 2023-10-16T09:44:08.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01b0f69700 1 --2- 172.21.15.181:0/2271053602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 0x7f01ac08a880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f019c009840 tx=0x7f019c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a470c3d53cd0cd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01a3fff700 1 -- 172.21.15.181:0/2271053602 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c011830 con 0x7f01ac0c9cd0 2023-10-16T09:44:08.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01a3fff700 1 -- 172.21.15.181:0/2271053602 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c011990 con 0x7f01ac0c9cd0 2023-10-16T09:44:08.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f01a3fff700 1 -- 172.21.15.181:0/2271053602 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c011d20 con 0x7f01ac0c9cd0 2023-10-16T09:44:08.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7fa92a7fc700 1 -- 172.21.15.181:0/2857639896 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400e070 con 0x7fa92c0cf340 2023-10-16T09:44:08.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f5c21f09700 1 -- 172.21.15.181:0/1505690571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 msgr2=0x7f5c1c0cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.054+0000 7f5c21f09700 1 --2- 172.21.15.181:0/1505690571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0cf7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c071460 tx=0x7f5c10009b00 comp rx=0 tx=0).stop 2023-10-16T09:44:08.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7fa92a7fc700 1 -- 172.21.15.181:0/2857639896 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92400b910 con 0x7fa92c0cf340 2023-10-16T09:44:08.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7fa92a7fc700 1 -- 172.21.15.181:0/2857639896 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400bcc0 con 0x7fa92c0cf340 2023-10-16T09:44:08.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7f0175d9b700 1 -- 172.21.15.181:0/1299857185 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168004230 con 0x7f01780cb110 2023-10-16T09:44:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7f0175d9b700 1 -- 172.21.15.181:0/1299857185 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0168004390 con 0x7f01780cb110 2023-10-16T09:44:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7f0175d9b700 1 -- 172.21.15.181:0/1299857185 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680036a0 con 0x7f01780cb110 2023-10-16T09:44:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7f017d086700 1 -- 172.21.15.181:0/1299857185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01780cb110 msgr2=0x7f017815e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7f017d086700 1 --2- 172.21.15.181:0/1299857185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01780cb110 0x7f017815e840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f01680098f0 tx=0x7f0168009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 -- 172.21.15.181:0/2857639896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c0cf340 msgr2=0x7fa92c06cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 --2- 172.21.15.181:0/2857639896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c0cf340 0x7fa92c06cdf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa92c06c730 tx=0x7fa92400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 -- 172.21.15.181:0/2857639896 shutdown_connections 2023-10-16T09:44:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 --2- 172.21.15.181:0/2857639896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c0cf340 0x7fa92c06cdf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 --2- 172.21.15.181:0/2857639896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c06bdc0 0x7fa92c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 --2- 172.21.15.181:0/2857639896 >> v2:172.21.15.190:3300/0 conn(0x7fa92c162f20 0x7fa92c0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 -- 172.21.15.181:0/2857639896 >> 172.21.15.181:0/2857639896 conn(0x7fa92c0b1780 msgr2=0x7fa92c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7f01b39ce700 1 -- 172.21.15.181:0/2271053602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 msgr2=0x7f01ac08a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.055+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2271053602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 0x7f01ac08a880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f019c009840 tx=0x7f019c009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:08.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f01b39ce700 1 -- 172.21.15.181:0/2271053602 shutdown_connections 2023-10-16T09:44:08.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f5c21f09700 1 -- 172.21.15.181:0/1505690571 shutdown_connections 2023-10-16T09:44:08.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f5c21f09700 1 --2- 172.21.15.181:0/1505690571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0cf7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f5c21f09700 1 --2- 172.21.15.181:0/1505690571 >> v2:172.21.15.190:3300/0 conn(0x7f5c1c0ad110 0x7f5c1c06eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2271053602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac08a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2271053602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 0x7f01ac08a880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f01b39ce700 1 -- 172.21.15.181:0/2271053602 >> 172.21.15.181:0/2271053602 conn(0x7f01ac06ac90 msgr2=0x7f01ac06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f5c21f09700 1 --2- 172.21.15.181:0/1505690571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c0c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f5c21f09700 1 -- 172.21.15.181:0/1505690571 >> 172.21.15.181:0/1505690571 conn(0x7f5c1c0d04a0 msgr2=0x7f5c1c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 -- 172.21.15.181:0/2857639896 shutdown_connections 2023-10-16T09:44:08.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7fa931549700 1 -- 172.21.15.181:0/2857639896 wait complete. 2023-10-16T09:44:08.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f017d086700 1 -- 172.21.15.181:0/1299857185 shutdown_connections 2023-10-16T09:44:08.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f017d086700 1 --2- 172.21.15.181:0/1299857185 >> v2:172.21.15.190:3300/0 conn(0x7f017815e460 0x7f01780d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f017d086700 1 --2- 172.21.15.181:0/1299857185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01780cb110 0x7f017815e840 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f017d086700 1 -- 172.21.15.181:0/1299857185 >> 172.21.15.181:0/1299857185 conn(0x7f01780d04a0 msgr2=0x7f01780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f5b7dde9700 1 Processor -- start 2023-10-16T09:44:08.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f01b39ce700 1 -- 172.21.15.181:0/2271053602 shutdown_connections 2023-10-16T09:44:08.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f01b39ce700 1 -- 172.21.15.181:0/2271053602 wait complete. 2023-10-16T09:44:08.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.056+0000 7f5b7dde9700 1 -- start start 2023-10-16T09:44:08.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b7dde9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60003a00 0x7f5b7814e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b7dde9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b780b5a50 0x7f5b780b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b7dde9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b780b3520 0x7f5b780c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b7dde9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780b6380 con 0x7f5b780b5a50 2023-10-16T09:44:08.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b7dde9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780b1470 con 0x7f5b60003a00 2023-10-16T09:44:08.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b7dde9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780b60a0 con 0x7f5b780b3520 2023-10-16T09:44:08.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b76ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b780b5a50 0x7f5b780b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.057+0000 7f5b76ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b780b5a50 0x7f5b780b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36294/0 (socket says 172.21.15.181:36294) 2023-10-16T09:44:08.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5b76ffd700 1 -- 172.21.15.181:0/1594599000 learned_addr learned my addr 172.21.15.181:0/1594599000 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f017d086700 1 -- 172.21.15.181:0/1299857185 shutdown_connections 2023-10-16T09:44:08.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7fa931549700 1 Processor -- start 2023-10-16T09:44:08.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7fa931549700 1 -- start start 2023-10-16T09:44:08.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa931549700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c06bdc0 0x7fa92c1618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa931549700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c0cf340 0x7fa92c203a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa931549700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c162f20 0x7fa92c203f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa931549700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c0b5aa0 con 0x7fa92c06bdc0 2023-10-16T09:44:08.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa931549700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c151960 con 0x7fa92c0cf340 2023-10-16T09:44:08.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa931549700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c0b57c0 con 0x7fa92c162f20 2023-10-16T09:44:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f017d086700 1 -- 172.21.15.181:0/1299857185 wait complete. 2023-10-16T09:44:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f017d086700 1 Processor -- start 2023-10-16T09:44:08.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5b77fff700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b780b3520 0x7f5b780c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7fda610d6700 1 Processor -- start 2023-10-16T09:44:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7fda610d6700 1 -- start start 2023-10-16T09:44:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fda610d6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c1622d0 0x7fda5c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fda610d6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda5c0ce430 0x7fda5c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fda610d6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c09b7d0 con 0x7fda5c161ef0 2023-10-16T09:44:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f01b39ce700 1 Processor -- start 2023-10-16T09:44:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f01b39ce700 1 -- start start 2023-10-16T09:44:08.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5c21f09700 1 -- 172.21.15.181:0/1505690571 shutdown_connections 2023-10-16T09:44:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fda610d6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0d2080 con 0x7fda5c0ce430 2023-10-16T09:44:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fda610d6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c09ba70 con 0x7fda5c1622d0 2023-10-16T09:44:08.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5b76ffd700 1 -- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b780b3520 msgr2=0x7f5b780c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5b76ffd700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b780b3520 0x7f5b780c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5b76ffd700 1 -- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60003a00 msgr2=0x7f5b7814e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c162f20 0x7fa92c203f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c162f20 0x7fa92c203f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33328/0 (socket says 172.21.15.181:33328) 2023-10-16T09:44:08.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa92b7fe700 1 -- 172.21.15.181:0/145267138 learned_addr learned my addr 172.21.15.181:0/145267138 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7fa9227fc700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c0cf340 0x7fa92c203a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f01b39ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 0x7f01ac0c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f01b39ce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac16c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f01b39ce700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01ac0cc930 0x7f01ac2058b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f01b39ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac16b300 con 0x7f01ac0c9cd0 2023-10-16T09:44:08.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5c21f09700 1 -- 172.21.15.181:0/1505690571 wait complete. 2023-10-16T09:44:08.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f01b39ce700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0c7140 con 0x7f01ac0cc930 2023-10-16T09:44:08.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f01b39ce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0c7420 con 0x7f01ac0cc550 2023-10-16T09:44:08.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.059+0000 7f01b0f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac16c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b0f69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac16c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33330/0 (socket says 172.21.15.181:33330) 2023-10-16T09:44:08.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b0f69700 1 -- 172.21.15.181:0/2325566407 learned_addr learned my addr 172.21.15.181:0/2325566407 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f5c21f09700 1 Processor -- start 2023-10-16T09:44:08.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fa92affd700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c06bdc0 0x7fa92c1618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c1622d0 0x7fda5c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b1f6b700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01ac0cc930 0x7f01ac2058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c1622d0 0x7fda5c0cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33336/0 (socket says 172.21.15.181:33336) 2023-10-16T09:44:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 -- 172.21.15.181:0/2815379203 learned_addr learned my addr 172.21.15.181:0/2815379203 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5b59e700 1 --2- 172.21.15.181:0/2815379203 >> v2:172.21.15.181:3300/0 conn(0x7fda5c0ce430 0x7fda5c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.058+0000 7f5b777fe700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60003a00 0x7f5b7814e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f5b76ffd700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60003a00 0x7f5b7814e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f5b76ffd700 1 -- 172.21.15.181:0/1594599000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b780c4320 con 0x7f5b780b5a50 2023-10-16T09:44:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f017d086700 1 -- start start 2023-10-16T09:44:08.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f5b777fe700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60003a00 0x7f5b7814e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:08.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f5b76ffd700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b780b5a50 0x7f5b780b0f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e900 tx=0x7f5b6800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 -- 172.21.15.181:0/2815379203 >> v2:172.21.15.181:3300/0 conn(0x7fda5c0ce430 msgr2=0x7fda5c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 --2- 172.21.15.181:0/2815379203 >> v2:172.21.15.181:3300/0 conn(0x7fda5c0ce430 0x7fda5c0b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 -- 172.21.15.181:0/2815379203 >> v1:172.21.15.190:6789/0 conn(0x7fda5c161ef0 legacy=0x7fda5c09c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 -- 172.21.15.181:0/2815379203 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c09bd50 con 0x7fda5c1622d0 2023-10-16T09:44:08.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fda5a59c700 1 --2- 172.21.15.181:0/2815379203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c1622d0 0x7fda5c0cdef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fda500098f0 tx=0x7fda50009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c888a09761af507b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fa92b7fe700 1 -- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c0cf340 msgr2=0x7fa92c203a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fa92b7fe700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c0cf340 0x7fa92c203a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fa92b7fe700 1 -- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c06bdc0 msgr2=0x7fa92c1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fa92b7fe700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c06bdc0 0x7fa92c1618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7fa92b7fe700 1 -- 172.21.15.181:0/145267138 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0094a0 con 0x7fa92c162f20 2023-10-16T09:44:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f5b5ffff700 1 -- 172.21.15.181:0/1594599000 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800ee50 con 0x7f5b780b5a50 2023-10-16T09:44:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f5b5ffff700 1 -- 172.21.15.181:0/1594599000 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6800c850 con 0x7f5b780b5a50 2023-10-16T09:44:08.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f5b5ffff700 1 -- 172.21.15.181:0/1594599000 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800cc00 con 0x7f5b780b5a50 2023-10-16T09:44:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b780c4600 con 0x7f5b780b5a50 2023-10-16T09:44:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b780b39a0 con 0x7f5b780b5a50 2023-10-16T09:44:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b176a700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 0x7f01ac0c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b1f6b700 1 -- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 msgr2=0x7f01ac16c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b1f6b700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac16c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b1f6b700 1 -- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 msgr2=0x7f01ac0c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b1f6b700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 0x7f01ac0c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b1f6b700 1 -- 172.21.15.181:0/2325566407 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0094a0 con 0x7f01ac0cc930 2023-10-16T09:44:08.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b0f69700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac16c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.060+0000 7f01b1f6b700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01ac0cc930 0x7f01ac2058b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f01a400caa0 tx=0x7f01a400cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f01a1ffb700 1 -- 172.21.15.181:0/2325566407 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400a9e0 con 0x7f01ac0cc930 2023-10-16T09:44:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f01a1ffb700 1 -- 172.21.15.181:0/2325566407 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400ab40 con 0x7f01ac0cc930 2023-10-16T09:44:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f01a1ffb700 1 -- 172.21.15.181:0/2325566407 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4012400 con 0x7f01ac0cc930 2023-10-16T09:44:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac0c7700 con 0x7f01ac0cc930 2023-10-16T09:44:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac0c7f00 con 0x7f01ac0cc930 2023-10-16T09:44:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7fda59d9b700 1 -- 172.21.15.181:0/2815379203 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000b8a0 con 0x7fda5c1622d0 2023-10-16T09:44:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7fda59d9b700 1 -- 172.21.15.181:0/2815379203 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5000ba00 con 0x7fda5c1622d0 2023-10-16T09:44:08.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7fda59d9b700 1 -- 172.21.15.181:0/2815379203 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000bd70 con 0x7fda5c1622d0 2023-10-16T09:44:08.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7fa92b7fe700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c162f20 0x7fa92c203f70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa92c074830 tx=0x7fa924004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.062+0000 7fa923fff700 1 -- 172.21.15.181:0/145267138 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240092e0 con 0x7fa92c162f20 2023-10-16T09:44:08.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.062+0000 7fa923fff700 1 -- 172.21.15.181:0/145267138 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa924004250 con 0x7fa92c162f20 2023-10-16T09:44:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.062+0000 7fa923fff700 1 -- 172.21.15.181:0/145267138 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924004600 con 0x7fa92c162f20 2023-10-16T09:44:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.062+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa924009b30 con 0x7fa92c162f20 2023-10-16T09:44:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.062+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c151680 con 0x7fa92c162f20 2023-10-16T09:44:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017d086700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780cb110 0x7f01780ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017d086700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f017815a0f0 0x7f01780c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017d086700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017815e460 0x7f0178153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017d086700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01780c8940 con 0x7f01780cb110 2023-10-16T09:44:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017d086700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01780c7ab0 con 0x7f017815e460 2023-10-16T09:44:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017d086700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01780c7cc0 con 0x7f017815a0f0 2023-10-16T09:44:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017815e460 0x7f0178153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f017815a0f0 0x7f01780c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017815e460 0x7f0178153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41368/0 (socket says 172.21.15.181:41368) 2023-10-16T09:44:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f017759e700 1 -- 172.21.15.181:0/551011511 learned_addr learned my addr 172.21.15.181:0/551011511 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.061+0000 7f0176d9d700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780cb110 0x7f01780ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.062+0000 7fda610d6700 1 -- 172.21.15.181:0/2815379203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c1622d0 msgr2=0x7fda5c0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.062+0000 7fda610d6700 1 --2- 172.21.15.181:0/2815379203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c1622d0 0x7fda5c0cdef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fda500098f0 tx=0x7fda50009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f017759e700 1 -- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f017815a0f0 msgr2=0x7f01780c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f017759e700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f017815a0f0 0x7f01780c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f017759e700 1 -- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780cb110 msgr2=0x7f01780ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f017759e700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780cb110 0x7f01780ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f017759e700 1 -- 172.21.15.181:0/551011511 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168009550 con 0x7f017815e460 2023-10-16T09:44:08.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f017659c700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f017815a0f0 0x7f01780c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f0176d9d700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780cb110 0x7f01780ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f01a1ffb700 1 -- 172.21.15.181:0/2325566407 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f01a401b3b0 con 0x7f01ac0cc930 2023-10-16T09:44:08.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9100052f0 con 0x7fa92c162f20 2023-10-16T09:44:08.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.063+0000 7f017759e700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017815e460 0x7f0178153640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f017000c7a0 tx=0x7f017000cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b580052f0 con 0x7f5b780b5a50 2023-10-16T09:44:08.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7fda610d6700 1 -- 172.21.15.181:0/2815379203 shutdown_connections 2023-10-16T09:44:08.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7fda610d6700 1 --2- 172.21.15.181:0/2815379203 >> v2:172.21.15.181:3300/0 conn(0x7fda5c0ce430 0x7fda5c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7fda610d6700 1 --2- 172.21.15.181:0/2815379203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c1622d0 0x7fda5c0cdef0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7fda610d6700 1 -- 172.21.15.181:0/2815379203 >> 172.21.15.181:0/2815379203 conn(0x7fda5c0d38a0 msgr2=0x7fda5c0d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f015f7fe700 1 -- 172.21.15.181:0/551011511 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170003e10 con 0x7f017815e460 2023-10-16T09:44:08.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f01a1ffb700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f019006af00 0x7f019006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f015f7fe700 1 -- 172.21.15.181:0/551011511 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0170005070 con 0x7f017815e460 2023-10-16T09:44:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01780c7fa0 con 0x7f017815e460 2023-10-16T09:44:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01780c8560 con 0x7f017815e460 2023-10-16T09:44:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018c0052f0 con 0x7f01ac0cc930 2023-10-16T09:44:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f01a1ffb700 1 -- 172.21.15.181:0/2325566407 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01a4089080 con 0x7f01ac0cc930 2023-10-16T09:44:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f01b176a700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f019006af00 0x7f019006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f015f7fe700 1 -- 172.21.15.181:0/551011511 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170005420 con 0x7f017815e460 2023-10-16T09:44:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f01b176a700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f019006af00 0x7f019006d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01a8006b20 tx=0x7f01a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f5b5ffff700 1 -- 172.21.15.181:0/1594599000 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5b68013470 con 0x7f5b780b5a50 2023-10-16T09:44:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5b5ffff700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5b6406afd0 0x7f5b6406d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5b5ffff700 1 -- 172.21.15.181:0/1594599000 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b680894c0 con 0x7f5b780b5a50 2023-10-16T09:44:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5b777fe700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5b6406afd0 0x7f5b6406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.064+0000 7f5c21f09700 1 -- start start 2023-10-16T09:44:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c21f09700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c1c0ad110 0x7f5c1c14cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c21f09700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c21f09700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c21f09700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0b41c0 con 0x7f5c1c0c3470 2023-10-16T09:44:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c21f09700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1605d0 con 0x7f5c1c0ad110 2023-10-16T09:44:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c21f09700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c160b90 con 0x7f5c1c0af840 2023-10-16T09:44:08.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36330/0 (socket says 172.21.15.181:36330) 2023-10-16T09:44:08.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c1bfff700 1 -- 172.21.15.181:0/2955913323 learned_addr learned my addr 172.21.15.181:0/2955913323 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c1affd700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f5b777fe700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5b6406afd0 0x7f5b6406d490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c015040 tx=0x7f5b6c007f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7fa923fff700 1 -- 172.21.15.181:0/145267138 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa924004760 con 0x7fa92c162f20 2023-10-16T09:44:08.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7fa923fff700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa90c06af50 0x7fa90c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7fa923fff700 1 -- 172.21.15.181:0/145267138 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa92408ad60 con 0x7fa92c162f20 2023-10-16T09:44:08.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7fa92affd700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa90c06af50 0x7fa90c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.065+0000 7f5c1b7fe700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c1c0ad110 0x7f5c1c14cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f5c1affd700 1 -- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c1c0ad110 msgr2=0x7f5c1c14cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f5c1affd700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c1c0ad110 0x7f5c1c14cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f5c1affd700 1 -- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 msgr2=0x7f5c1c0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f5c1affd700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f5c1affd700 1 -- 172.21.15.181:0/2955913323 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10009430 con 0x7f5c1c0af840 2023-10-16T09:44:08.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7fda610d6700 1 -- 172.21.15.181:0/2815379203 shutdown_connections 2023-10-16T09:44:08.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01580052f0 con 0x7f017815e460 2023-10-16T09:44:08.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7fa92affd700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa90c06af50 0x7fa90c06d410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa914006fd0 tx=0x7fa914009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.066+0000 7f5c1affd700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c14e5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b450 tx=0x7f5c0c00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.067+0000 7f2b0fb89700 1 Processor -- start 2023-10-16T09:44:08.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.067+0000 7fda610d6700 1 -- 172.21.15.181:0/2815379203 wait complete. 2023-10-16T09:44:08.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.067+0000 7f2b0fb89700 1 -- start start 2023-10-16T09:44:08.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0fb89700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b080cbdc0 0x7f2b080ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0fb89700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2b0815aac0 0x7f2b080ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0fb89700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b080cd530 con 0x7f2b0815a6e0 2023-10-16T09:44:08.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0fb89700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b080cd810 con 0x7f2b0815aac0 2023-10-16T09:44:08.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0fb89700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080cdaf0 con 0x7f2b080cbdc0 2023-10-16T09:44:08.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0e126700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2b0815aac0 0x7f2b080ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b080cbdc0 0x7f2b080ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b080cbdc0 0x7f2b080ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33362/0 (socket says 172.21.15.181:33362) 2023-10-16T09:44:08.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0e126700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2b0815aac0 0x7f2b080ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41388/0 (socket says 172.21.15.181:41388) 2023-10-16T09:44:08.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 -- 172.21.15.181:0/2351630958 learned_addr learned my addr 172.21.15.181:0/2351630958 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d925700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2b0815a6e0 0x7f2b080b1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59586/0 (socket says 172.21.15.181:59586) 2023-10-16T09:44:08.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7fdcb6195700 1 Processor -- start 2023-10-16T09:44:08.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 -- 172.21.15.181:0/2351630958 >> v2:172.21.15.190:3300/0 conn(0x7f2b0815aac0 msgr2=0x7f2b080ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 --2- 172.21.15.181:0/2351630958 >> v2:172.21.15.190:3300/0 conn(0x7f2b0815aac0 0x7f2b080ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 -- 172.21.15.181:0/2351630958 >> v1:172.21.15.190:6789/0 conn(0x7f2b0815a6e0 legacy=0x7f2b080b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 -- 172.21.15.181:0/2351630958 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080cddd0 con 0x7f2b080cbdc0 2023-10-16T09:44:08.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0e126700 1 --2- 172.21.15.181:0/2351630958 >> v2:172.21.15.190:3300/0 conn(0x7f2b0815aac0 0x7f2b080ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7fda610d6700 1 Processor -- start 2023-10-16T09:44:08.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7f2b0d124700 1 --2- 172.21.15.181:0/2351630958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b080cbdc0 0x7f2b080ce860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2b040098f0 tx=0x7f2b04009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=410311775cd43aae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7fdcb6195700 1 -- start start 2023-10-16T09:44:08.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.068+0000 7fda610d6700 1 -- start start 2023-10-16T09:44:08.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcb6195700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb006bdc0 0x7fdcb00aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcb6195700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdcb0162f20 0x7fdcb00cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcb6195700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdcb00cf340 0x7fdcb006cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcb6195700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb01633a0 con 0x7fdcb00cf340 2023-10-16T09:44:08.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcb6195700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb00665a0 con 0x7fdcb0162f20 2023-10-16T09:44:08.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcb6195700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb0162460 con 0x7fdcb006bdc0 2023-10-16T09:44:08.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda610d6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0ce430 0x7fda5c0cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda610d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c161ef0 0x7fda5c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda610d6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c1622d0 0x7fda5c0b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda610d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0cc910 con 0x7fda5c161ef0 2023-10-16T09:44:08.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda610d6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c158110 con 0x7fda5c1622d0 2023-10-16T09:44:08.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda610d6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c1583f0 con 0x7fda5c0ce430 2023-10-16T09:44:08.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0ce430 0x7fda5c0cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda5a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c161ef0 0x7fda5c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda5a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c161ef0 0x7fda5c156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36350/0 (socket says 172.21.15.181:36350) 2023-10-16T09:44:08.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda5a59c700 1 -- 172.21.15.181:0/1014368882 learned_addr learned my addr 172.21.15.181:0/1014368882 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0ce430 0x7fda5c0cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33372/0 (socket says 172.21.15.181:33372) 2023-10-16T09:44:08.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdca6ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdcb0162f20 0x7fdcb00cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcaffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdcb00cf340 0x7fdcb006cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb006bdc0 0x7fdcb00aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdca6ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdcb0162f20 0x7fdcb00cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41392/0 (socket says 172.21.15.181:41392) 2023-10-16T09:44:08.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdcaffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdcb00cf340 0x7fdcb006cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36344/0 (socket says 172.21.15.181:36344) 2023-10-16T09:44:08.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.069+0000 7fdca6ffd700 1 -- 172.21.15.181:0/1955809272 learned_addr learned my addr 172.21.15.181:0/1955809272 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fda5b59e700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c1622d0 0x7fda5c0b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fdcaffff700 1 -- 172.21.15.181:0/1955809272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb006bdc0 msgr2=0x7fdcb00aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fdcaffff700 1 --2- 172.21.15.181:0/1955809272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb006bdc0 0x7fdcb00aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fdcaffff700 1 -- 172.21.15.181:0/1955809272 >> v2:172.21.15.190:3300/0 conn(0x7fdcb0162f20 msgr2=0x7fdcb00cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:08.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fdcaffff700 1 --2- 172.21.15.181:0/1955809272 >> v2:172.21.15.190:3300/0 conn(0x7fdcb0162f20 0x7fdcb00cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fdcaffff700 1 -- 172.21.15.181:0/1955809272 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb006c1a0 con 0x7fdcb00cf340 2023-10-16T09:44:08.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fdcaf7fe700 1 --2- 172.21.15.181:0/1955809272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb006bdc0 0x7fdcb00aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fda5ad9d700 1 -- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c1622d0 msgr2=0x7fda5c0b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fda5ad9d700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c1622d0 0x7fda5c0b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fda5ad9d700 1 -- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c161ef0 msgr2=0x7fda5c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fda5ad9d700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c161ef0 0x7fda5c156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fda5ad9d700 1 -- 172.21.15.181:0/1014368882 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda50009550 con 0x7fda5c0ce430 2023-10-16T09:44:08.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fdcaffff700 1 --2- 172.21.15.181:0/1955809272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 0x7fdcb006cdf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdcb006c730 tx=0x7fdca8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565c3555a5f38a65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.070+0000 7fda5ad9d700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0ce430 0x7fda5c0cc3d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda5400cb50 tx=0x7fda5400ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.071+0000 7fdcaeffd700 1 -- 172.21.15.181:0/1955809272 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8011830 con 0x7fdcb00cf340 2023-10-16T09:44:08.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.071+0000 7fdcaeffd700 1 -- 172.21.15.181:0/1955809272 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca8011990 con 0x7fdcb00cf340 2023-10-16T09:44:08.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.071+0000 7fdcaeffd700 1 -- 172.21.15.181:0/1955809272 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8011d20 con 0x7fdcb00cf340 2023-10-16T09:44:08.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7f5b5ffff700 1 -- 172.21.15.181:0/1594599000 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5b68054620 con 0x7f5b780b5a50 2023-10-16T09:44:08.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 -- 172.21.15.181:0/1955809272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 msgr2=0x7fdcb006cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1955809272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 0x7fdcb006cdf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdcb006c730 tx=0x7fdca8009b20 comp rx=0 tx=0).stop 2023-10-16T09:44:08.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 -- 172.21.15.181:0/1955809272 shutdown_connections 2023-10-16T09:44:08.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1955809272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 0x7fdcb006cdf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1955809272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb006bdc0 0x7fdcb00aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1955809272 >> v2:172.21.15.190:3300/0 conn(0x7fdcb0162f20 0x7fdcb00cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 -- 172.21.15.181:0/1955809272 >> 172.21.15.181:0/1955809272 conn(0x7fdcb00b1780 msgr2=0x7fdcb00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 -- 172.21.15.181:0/1955809272 shutdown_connections 2023-10-16T09:44:08.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.072+0000 7fdcb6195700 1 -- 172.21.15.181:0/1955809272 wait complete. 2023-10-16T09:44:08.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.073+0000 7f5c03fff700 1 -- 172.21.15.181:0/2955913323 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c011830 con 0x7f5c1c0af840 2023-10-16T09:44:08.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.073+0000 7f2b0c923700 1 -- 172.21.15.181:0/2351630958 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04011830 con 0x7f2b080cbdc0 2023-10-16T09:44:08.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7fa923fff700 1 -- 172.21.15.181:0/145267138 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa924055f80 con 0x7fa92c162f20 2023-10-16T09:44:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.073+0000 7fda437fe700 1 -- 172.21.15.181:0/1014368882 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54003c00 con 0x7fda5c0ce430 2023-10-16T09:44:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.073+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c1586d0 con 0x7fda5c0ce430 2023-10-16T09:44:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c158c90 con 0x7fda5c0ce430 2023-10-16T09:44:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7fda437fe700 1 -- 172.21.15.181:0/1014368882 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda54003d80 con 0x7fda5c0ce430 2023-10-16T09:44:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7fda437fe700 1 -- 172.21.15.181:0/1014368882 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400aa20 con 0x7fda5c0ce430 2023-10-16T09:44:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.073+0000 7fdcb6195700 1 Processor -- start 2023-10-16T09:44:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7fdcb6195700 1 -- start start 2023-10-16T09:44:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7f01a1ffb700 1 -- 172.21.15.181:0/2325566407 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f01a4054140 con 0x7f01ac0cc930 2023-10-16T09:44:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.073+0000 7f5c03fff700 1 -- 172.21.15.181:0/2955913323 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0119b0 con 0x7f5c1c0af840 2023-10-16T09:44:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.073+0000 7f5c03fff700 1 -- 172.21.15.181:0/2955913323 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c011d60 con 0x7f5c1c0af840 2023-10-16T09:44:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c0b3660 con 0x7f5c1c0af840 2023-10-16T09:44:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c160820 con 0x7f5c1c0af840 2023-10-16T09:44:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f5c03fff700 1 -- 172.21.15.181:0/2955913323 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5c0c01a400 con 0x7f5c1c0af840 2023-10-16T09:44:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7f015f7fe700 1 -- 172.21.15.181:0/551011511 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0170005580 con 0x7f017815e460 2023-10-16T09:44:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7f5c03fff700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5c0806af00 0x7f5c0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7f5c03fff700 1 -- 172.21.15.181:0/2955913323 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c0c0894c0 con 0x7f5c1c0af840 2023-10-16T09:44:08.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fda437fe700 1 -- 172.21.15.181:0/1014368882 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fda5400ab80 con 0x7fda5c0ce430 2023-10-16T09:44:08.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fda437fe700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fda4806af00 0x7fda4806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.077+0000 7fda437fe700 1 -- 172.21.15.181:0/1014368882 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda54089670 con 0x7fda5c0ce430 2023-10-16T09:44:08.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7f015f7fe700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f016406af50 0x7f016406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7f015f7fe700 1 -- 172.21.15.181:0/551011511 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f017008b690 con 0x7f017815e460 2023-10-16T09:44:08.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.077+0000 7fda5a59c700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fda4806af00 0x7fda4806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.077+0000 7f0176d9d700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f016406af50 0x7f016406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.077+0000 7fda5a59c700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fda4806af00 0x7fda4806d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda5c159950 tx=0x7fda50008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.078+0000 7f0176d9d700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f016406af50 0x7f016406d410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f01781f3760 tx=0x7f016c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.078+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c0052f0 con 0x7fda5c0ce430 2023-10-16T09:44:08.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.077+0000 7f5c1b7fe700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5c0806af00 0x7f5c0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.078+0000 7f5c1b7fe700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5c0806af00 0x7f5c0806d3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c00 tx=0x7f5c10008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.078+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bfc0052f0 con 0x7f5c1c0af840 2023-10-16T09:44:08.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7f2b0c923700 1 -- 172.21.15.181:0/2351630958 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b04011990 con 0x7f2b080cbdc0 2023-10-16T09:44:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.074+0000 7f2b0c923700 1 -- 172.21.15.181:0/2351630958 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04011d20 con 0x7f2b080cbdc0 2023-10-16T09:44:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2351630958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b080cbdc0 msgr2=0x7f2b080ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2351630958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b080cbdc0 0x7f2b080ce860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2b040098f0 tx=0x7f2b04009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2351630958 shutdown_connections 2023-10-16T09:44:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2351630958 >> v2:172.21.15.190:3300/0 conn(0x7f2b0815aac0 0x7f2b080ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2351630958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b080cbdc0 0x7f2b080ce860 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2351630958 >> 172.21.15.181:0/2351630958 conn(0x7f2b0806a8a0 msgr2=0x7f2b0806ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2351630958 shutdown_connections 2023-10-16T09:44:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2351630958 wait complete. 2023-10-16T09:44:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.078+0000 7f2b0fb89700 1 Processor -- start 2023-10-16T09:44:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.078+0000 7f2b0fb89700 1 -- start start 2023-10-16T09:44:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcb6195700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcb006bdc0 0x7fdcb00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcb6195700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 0x7fdcb00cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcb6195700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb0162f20 0x7fdcb00cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcb6195700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00c5c70 con 0x7fdcb00cf340 2023-10-16T09:44:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcb6195700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00cb9e0 con 0x7fdcb006bdc0 2023-10-16T09:44:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcb6195700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00c3840 con 0x7fdcb0162f20 2023-10-16T09:44:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcaffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb0162f20 0x7fdcb00cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcaffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb0162f20 0x7fdcb00cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33380/0 (socket says 172.21.15.181:33380) 2023-10-16T09:44:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.075+0000 7fdcaffff700 1 -- 172.21.15.181:0/1691172471 learned_addr learned my addr 172.21.15.181:0/1691172471 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcaffff700 1 -- 172.21.15.181:0/1691172471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcb006bdc0 msgr2=0x7fdcb00c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcaffff700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcb006bdc0 0x7fdcb00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcaffff700 1 -- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 msgr2=0x7fdcb00cae60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:44:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcaffff700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 0x7fdcb00cae60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcaffff700 1 -- 172.21.15.181:0/1691172471 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009a70 con 0x7fdcb0162f20 2023-10-16T09:44:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcaffff700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb0162f20 0x7fdcb00cb3a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009b60 tx=0x7fdca8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcacff9700 1 -- 172.21.15.181:0/1691172471 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8004510 con 0x7fdcb0162f20 2023-10-16T09:44:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca80094a0 con 0x7fdcb0162f20 2023-10-16T09:44:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.076+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb00c3e00 con 0x7fdcb0162f20 2023-10-16T09:44:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.077+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb0067d20 con 0x7fdcb0162f20 2023-10-16T09:44:08.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0fb89700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b080cbdc0 0x7f2b08140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0fb89700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b0815a6e0 0x7f2b0815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0fb89700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b0815aac0 0x7f2b0815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0fb89700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0814db80 con 0x7f2b0815aac0 2023-10-16T09:44:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0fb89700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0815b450 con 0x7f2b080cbdc0 2023-10-16T09:44:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0fb89700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0815b730 con 0x7f2b0815a6e0 2023-10-16T09:44:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0e126700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b0815aac0 0x7f2b0815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0d925700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b080cbdc0 0x7f2b08140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0d124700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b0815a6e0 0x7f2b0815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0e126700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b0815aac0 0x7f2b0815c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36366/0 (socket says 172.21.15.181:36366) 2023-10-16T09:44:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.079+0000 7f2b0e126700 1 -- 172.21.15.181:0/2246597191 learned_addr learned my addr 172.21.15.181:0/2246597191 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0e126700 1 -- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b0815a6e0 msgr2=0x7f2b0815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0e126700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b0815a6e0 0x7f2b0815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0e126700 1 -- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b080cbdc0 msgr2=0x7f2b08140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0e126700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b080cbdc0 0x7f2b08140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0e126700 1 -- 172.21.15.181:0/2246597191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80094a0 con 0x7f2b0815aac0 2023-10-16T09:44:08.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0d124700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b0815a6e0 0x7f2b0815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0d925700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b080cbdc0 0x7f2b08140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:08.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0e126700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b0815aac0 0x7f2b0815c640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2af800e930 tx=0x7f2af800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2afe7fc700 1 -- 172.21.15.181:0/2246597191 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8009700 con 0x7f2b0815aac0 2023-10-16T09:44:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2afe7fc700 1 -- 172.21.15.181:0/2246597191 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8004020 con 0x7f2b0815aac0 2023-10-16T09:44:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b04009550 con 0x7f2b0815aac0 2023-10-16T09:44:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2afe7fc700 1 -- 172.21.15.181:0/2246597191 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af80043b0 con 0x7f2b0815aac0 2023-10-16T09:44:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f015f7fe700 1 -- 172.21.15.181:0/551011511 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f01700568b0 con 0x7f017815e460 2023-10-16T09:44:08.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.080+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0815bcf0 con 0x7f2b0815aac0 2023-10-16T09:44:08.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.082+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b08074f00 con 0x7f2b0815aac0 2023-10-16T09:44:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.082+0000 7f2afe7fc700 1 -- 172.21.15.181:0/2246597191 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2af80036a0 con 0x7f2b0815aac0 2023-10-16T09:44:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.082+0000 7f2afe7fc700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2aec06af00 0x7f2aec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.083+0000 7f2afe7fc700 1 -- 172.21.15.181:0/2246597191 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2af8089220 con 0x7f2b0815aac0 2023-10-16T09:44:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.083+0000 7f2b0d925700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2aec06af00 0x7f2aec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.083+0000 7f2b0d925700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2aec06af00 0x7f2aec06d3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b000012e0 tx=0x7f2b00001b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.084+0000 7fda437fe700 1 -- 172.21.15.181:0/1014368882 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fda540546d0 con 0x7fda5c0ce430 2023-10-16T09:44:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.088+0000 7f5c03fff700 1 -- 172.21.15.181:0/2955913323 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5c0c054570 con 0x7f5c1c0af840 2023-10-16T09:44:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.084+0000 7fdcacff9700 1 -- 172.21.15.181:0/1691172471 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca8030070 con 0x7fdcb0162f20 2023-10-16T09:44:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.084+0000 7fdcacff9700 1 -- 172.21.15.181:0/1691172471 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca80038c0 con 0x7fdcb0162f20 2023-10-16T09:44:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.090+0000 7fdcacff9700 1 -- 172.21.15.181:0/1691172471 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdca8003a20 con 0x7fdcb0162f20 2023-10-16T09:44:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.090+0000 7fdcacff9700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdc9006af00 0x7fdc9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.090+0000 7fdcacff9700 1 -- 172.21.15.181:0/1691172471 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdca8032070 con 0x7fdcb0162f20 2023-10-16T09:44:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.090+0000 7fdcaf7fe700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdc9006af00 0x7fdc9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.091+0000 7fdcaf7fe700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdc9006af00 0x7fdc9006d3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdca000e2b0 tx=0x7fdca000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.095+0000 7fdcacff9700 1 -- 172.21.15.181:0/1691172471 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdca8067010 con 0x7fdcb0162f20 2023-10-16T09:44:08.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.092+0000 7f2afe7fc700 1 -- 172.21.15.181:0/2246597191 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2af80542e0 con 0x7f2b0815aac0 2023-10-16T09:44:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.478+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b580055a0 con 0x7f5b780b5a50 2023-10-16T09:44:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.478+0000 7f5b5ffff700 1 -- 172.21.15.181:0/1594599000 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b68057c40 con 0x7f5b780b5a50 2023-10-16T09:44:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:08.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.478+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f018c0055a0 con 0x7f01ac0cc930 2023-10-16T09:44:08.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.479+0000 7f01a1ffb700 1 -- 172.21.15.181:0/2325566407 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01a4017030 con 0x7f01ac0cc930 2023-10-16T09:44:08.482 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:08.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.481+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa910005870 con 0x7fa92c162f20 2023-10-16T09:44:08.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.481+0000 7fa923fff700 1 -- 172.21.15.181:0/145267138 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa924020020 con 0x7fa92c162f20 2023-10-16T09:44:08.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.484+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0158005870 con 0x7f017815e460 2023-10-16T09:44:08.486 INFO:teuthology.orchestra.run.smithi181.stdout:60129542148 2023-10-16T09:44:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.485+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f019006af00 msgr2=0x7f019006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.485+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f019006af00 0x7f019006d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01a8006b20 tx=0x7f01a8008040 comp rx=0 tx=0).stop 2023-10-16T09:44:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.485+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01ac0cc930 msgr2=0x7f01ac2058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.485+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01ac0cc930 0x7f01ac2058b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f01a400caa0 tx=0x7f01a400cdb0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f015f7fe700 1 -- 172.21.15.181:0/551011511 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0170059ed0 con 0x7f017815e460 2023-10-16T09:44:08.488 INFO:teuthology.orchestra.run.smithi181.stdout:60129542148 2023-10-16T09:44:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 shutdown_connections 2023-10-16T09:44:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f019006af00 0x7f019006d3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01ac0cc930 0x7f01ac2058b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ac0c9cd0 0x7f01ac0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 --2- 172.21.15.181:0/2325566407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ac0cc550 0x7f01ac16c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 >> 172.21.15.181:0/2325566407 conn(0x7f01ac06ac90 msgr2=0x7f01ac0b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 shutdown_connections 2023-10-16T09:44:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.486+0000 7f01b39ce700 1 -- 172.21.15.181:0/2325566407 wait complete. 2023-10-16T09:44:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.487+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fda3c0055a0 con 0x7fda5c0ce430 2023-10-16T09:44:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.487+0000 7fda437fe700 1 -- 172.21.15.181:0/1014368882 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda54057cf0 con 0x7fda5c0ce430 2023-10-16T09:44:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa90c06af50 msgr2=0x7fa90c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa90c06af50 0x7fa90c06d410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa914006fd0 tx=0x7fa914009190 comp rx=0 tx=0).stop 2023-10-16T09:44:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c162f20 msgr2=0x7fa92c203f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c162f20 0x7fa92c203f70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa92c074830 tx=0x7fa924004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 shutdown_connections 2023-10-16T09:44:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa90c06af50 0x7fa90c06d410 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa914006fd0 tx=0x7fa914009190 comp rx=0 tx=0).stop 2023-10-16T09:44:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92c162f20 0x7fa92c203f70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92c06bdc0 0x7fa92c1618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 --2- 172.21.15.181:0/145267138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c0cf340 0x7fa92c203a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 >> 172.21.15.181:0/145267138 conn(0x7fa92c0b1780 msgr2=0x7fa92c155170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 shutdown_connections 2023-10-16T09:44:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f016406af50 msgr2=0x7f016406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f016406af50 0x7f016406d410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f01781f3760 tx=0x7f016c009190 comp rx=0 tx=0).stop 2023-10-16T09:44:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017815e460 msgr2=0x7f0178153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017815e460 0x7f0178153640 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f017000c7a0 tx=0x7f017000cab0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 shutdown_connections 2023-10-16T09:44:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f016406af50 0x7f016406d410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f017815e460 0x7f0178153640 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780cb110 0x7f01780ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 --2- 172.21.15.181:0/551011511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f017815a0f0 0x7f01780c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.488+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 >> 172.21.15.181:0/551011511 conn(0x7f01780d04a0 msgr2=0x7f01780ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.489+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 shutdown_connections 2023-10-16T09:44:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.489+0000 7f017d086700 1 -- 172.21.15.181:0/551011511 wait complete. 2023-10-16T09:44:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5b6406afd0 msgr2=0x7f5b6406d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5b6406afd0 0x7f5b6406d490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c015040 tx=0x7f5b6c007f90 comp rx=0 tx=0).stop 2023-10-16T09:44:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b780b5a50 msgr2=0x7f5b780b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.490+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b780b5a50 0x7f5b780b0f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800e900 tx=0x7f5b6800ec10 comp rx=0 tx=0).stop 2023-10-16T09:44:08.498 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fda4806af00 msgr2=0x7fda4806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fda4806af00 0x7fda4806d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda5c159950 tx=0x7fda50008030 comp rx=0 tx=0).stop 2023-10-16T09:44:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0ce430 msgr2=0x7fda5c0cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0ce430 0x7fda5c0cc3d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda5400cb50 tx=0x7fda5400ce60 comp rx=0 tx=0).stop 2023-10-16T09:44:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 shutdown_connections 2023-10-16T09:44:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5b6406afd0 0x7f5b6406d490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b780b3520 0x7f5b780c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b60003a00 0x7f5b7814e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 --2- 172.21.15.181:0/1594599000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b780b5a50 0x7f5b780b0f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 >> 172.21.15.181:0/1594599000 conn(0x7f5b780d04a0 msgr2=0x7f5b78158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 shutdown_connections 2023-10-16T09:44:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fda4806af00 0x7fda4806d3c0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda5c159950 tx=0x7fda50008030 comp rx=0 tx=0).stop 2023-10-16T09:44:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0ce430 0x7fda5c0cc3d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c1622d0 0x7fda5c0b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 --2- 172.21.15.181:0/1014368882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c161ef0 0x7fda5c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 >> 172.21.15.181:0/1014368882 conn(0x7fda5c0d38a0 msgr2=0x7fda5c0d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 shutdown_connections 2023-10-16T09:44:08.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.491+0000 7f5b7dde9700 1 -- 172.21.15.181:0/1594599000 wait complete. 2023-10-16T09:44:08.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.494+0000 7fa931549700 1 -- 172.21.15.181:0/145267138 wait complete. 2023-10-16T09:44:08.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.494+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b08067d20 con 0x7f2b0815aac0 2023-10-16T09:44:08.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.494+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 shutdown_connections 2023-10-16T09:44:08.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.494+0000 7fda610d6700 1 -- 172.21.15.181:0/1014368882 wait complete. 2023-10-16T09:44:08.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.494+0000 7f2afe7fc700 1 -- 172.21.15.181:0/2246597191 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2af8057900 con 0x7f2b0815aac0 2023-10-16T09:44:08.506 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:08.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.500+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdcb00c3980 con 0x7fdcb0162f20 2023-10-16T09:44:08.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.500+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5bfc005870 con 0x7f5c1c0af840 2023-10-16T09:44:08.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.499+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2aec06af00 msgr2=0x7f2aec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.499+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2aec06af00 0x7f2aec06d3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b000012e0 tx=0x7f2b00001b20 comp rx=0 tx=0).stop 2023-10-16T09:44:08.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.499+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b0815aac0 msgr2=0x7f2b0815c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.499+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b0815aac0 0x7f2b0815c640 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2af800e930 tx=0x7f2af800ec40 comp rx=0 tx=0).stop 2023-10-16T09:44:08.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.501+0000 7fdcacff9700 1 -- 172.21.15.181:0/1691172471 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdcb00c3980 con 0x7fdcb0162f20 2023-10-16T09:44:08.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.501+0000 7f5c03fff700 1 -- 172.21.15.181:0/2955913323 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c0c015030 con 0x7f5c1c0af840 2023-10-16T09:44:08.510 INFO:teuthology.orchestra.run.smithi181.stdout:60129542148 2023-10-16T09:44:08.511 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.504+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5c0806af00 msgr2=0x7f5c0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.504+0000 7f5c21f09700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5c0806af00 0x7f5c0806d3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c00 tx=0x7f5c10008070 comp rx=0 tx=0).stop 2023-10-16T09:44:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.504+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 msgr2=0x7f5c1c14e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.504+0000 7f5c21f09700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c14e5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b450 tx=0x7f5c0c00b760 comp rx=0 tx=0).stop 2023-10-16T09:44:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 shutdown_connections 2023-10-16T09:44:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5c0806af00 0x7f5c0806d3c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c1c0c3470 0x7f5c1c0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c1c0ad110 0x7f5c1c14cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 --2- 172.21.15.181:0/2955913323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c1c0af840 0x7f5c1c14e5f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 >> 172.21.15.181:0/2955913323 conn(0x7f5c1c0d04a0 msgr2=0x7f5c1c0ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 shutdown_connections 2023-10-16T09:44:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7f5c21f09700 1 -- 172.21.15.181:0/2955913323 wait complete. 2023-10-16T09:44:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdc9006af00 msgr2=0x7fdc9006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdc9006af00 0x7fdc9006d3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdca000e2b0 tx=0x7fdca000d040 comp rx=0 tx=0).stop 2023-10-16T09:44:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb0162f20 msgr2=0x7fdcb00cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.505+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb0162f20 0x7fdcb00cb3a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009b60 tx=0x7fdca8004750 comp rx=0 tx=0).stop 2023-10-16T09:44:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 shutdown_connections 2023-10-16T09:44:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdc9006af00 0x7fdc9006d3c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcb0162f20 0x7fdcb00cb3a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcb006bdc0 0x7fdcb00c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7fdcb6195700 1 --2- 172.21.15.181:0/1691172471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcb00cf340 0x7fdcb00cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 >> 172.21.15.181:0/1691172471 conn(0x7fdcb00b1780 msgr2=0x7fdcb006da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 shutdown_connections 2023-10-16T09:44:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2aec06af00 0x7f2aec06d3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b0815aac0 0x7f2b0815c640 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b080cbdc0 0x7f2b08140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 --2- 172.21.15.181:0/2246597191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b0815a6e0 0x7f2b0815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 >> 172.21.15.181:0/2246597191 conn(0x7f2b0806a8a0 msgr2=0x7f2b081598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 shutdown_connections 2023-10-16T09:44:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.506+0000 7f2b0fb89700 1 -- 172.21.15.181:0/2246597191 wait complete. 2023-10-16T09:44:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.507+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 shutdown_connections 2023-10-16T09:44:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:08.507+0000 7fdcb6195700 1 -- 172.21.15.181:0/1691172471 wait complete. 2023-10-16T09:44:08.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2023-10-16T09:44:08.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2023-10-16T09:44:08.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2023-10-16T09:44:08.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2023-10-16T09:44:08.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2023-10-16T09:44:08.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2023-10-16T09:44:08.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2023-10-16T09:44:08.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2023-10-16T09:44:09.522 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T09:44:09.523 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T09:44:09.525 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T09:44:09.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T09:44:09.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T09:44:09.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T09:44:09.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T09:44:09.535 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T09:44:09.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.704+0000 7f101c60b700 1 Processor -- start 2023-10-16T09:44:09.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.704+0000 7f101c60b700 1 -- start start 2023-10-16T09:44:09.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.705+0000 7f101c60b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10140c9b30 0x7f101415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.705+0000 7f101c60b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10140b0ef0 0x7f10140b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.705+0000 7f101c60b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1014159fb0 con 0x7f1014159bd0 2023-10-16T09:44:09.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.705+0000 7f101c60b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10140715f0 con 0x7f10140c9b30 2023-10-16T09:44:09.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.705+0000 7f101c60b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10140cb100 con 0x7f10140b0ef0 2023-10-16T09:44:09.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.706+0000 7f101aba8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10140b0ef0 0x7f10140b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.706+0000 7f101aba8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10140b0ef0 0x7f10140b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41424/0 (socket says 172.21.15.181:41424) 2023-10-16T09:44:09.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.706+0000 7f101aba8700 1 -- 172.21.15.181:0/3643106045 learned_addr learned my addr 172.21.15.181:0/3643106045 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.706+0000 7f101aba8700 1 -- 172.21.15.181:0/3643106045 >> v2:172.21.15.181:3300/0 conn(0x7f10140c9b30 msgr2=0x7f101415d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.706+0000 7f101aba8700 1 --2- 172.21.15.181:0/3643106045 >> v2:172.21.15.181:3300/0 conn(0x7f10140c9b30 0x7f101415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.706+0000 7f101aba8700 1 -- 172.21.15.181:0/3643106045 >> v1:172.21.15.181:6789/0 conn(0x7f1014159bd0 legacy=0x7f10140cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.706+0000 7f101aba8700 1 -- 172.21.15.181:0/3643106045 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140b1880 con 0x7f10140b0ef0 2023-10-16T09:44:09.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.707+0000 7f101aba8700 1 --2- 172.21.15.181:0/3643106045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 0x7f10140b1340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10140b2040 tx=0x7f100c00d650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ac88318f940afc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.707+0000 7f10193a5700 1 -- 172.21.15.181:0/3643106045 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c010070 con 0x7f10140b0ef0 2023-10-16T09:44:09.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.707+0000 7f10193a5700 1 -- 172.21.15.181:0/3643106045 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c00dca0 con 0x7f10140b0ef0 2023-10-16T09:44:09.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.707+0000 7f10193a5700 1 -- 172.21.15.181:0/3643106045 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c007d50 con 0x7f10140b0ef0 2023-10-16T09:44:09.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.708+0000 7f101c60b700 1 -- 172.21.15.181:0/3643106045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 msgr2=0x7f10140b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.708+0000 7f101c60b700 1 --2- 172.21.15.181:0/3643106045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 0x7f10140b1340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10140b2040 tx=0x7f100c00d650 comp rx=0 tx=0).stop 2023-10-16T09:44:09.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.712+0000 7f0496329700 1 Processor -- start 2023-10-16T09:44:09.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.712+0000 7f0496329700 1 -- start start 2023-10-16T09:44:09.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f0496329700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04900cc550 0x7f049008a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f0496329700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0490071fa0 0x7f049008a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f0496329700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049008adc0 con 0x7f04900c9cd0 2023-10-16T09:44:09.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f0496329700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04900ca4d0 con 0x7f04900cc550 2023-10-16T09:44:09.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f0496329700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049016ef80 con 0x7f0490071fa0 2023-10-16T09:44:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04900cc550 0x7f049008a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04900cc550 0x7f049008a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36378/0 (socket says 172.21.15.181:36378) 2023-10-16T09:44:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 -- 172.21.15.181:0/1922121908 learned_addr learned my addr 172.21.15.181:0/1922121908 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f04948c6700 1 --2- 172.21.15.181:0/1922121908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0490071fa0 0x7f049008a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 -- 172.21.15.181:0/1922121908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0490071fa0 msgr2=0x7f049008a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 --2- 172.21.15.181:0/1922121908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0490071fa0 0x7f049008a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 -- 172.21.15.181:0/1922121908 >> v1:172.21.15.190:6789/0 conn(0x7f04900c9cd0 legacy=0x7f04900b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 -- 172.21.15.181:0/1922121908 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049016f260 con 0x7f04900cc550 2023-10-16T09:44:09.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.713+0000 7f048ffff700 1 --2- 172.21.15.181:0/1922121908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900cc550 0x7f049008a340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0488009e50 tx=0x7f048800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c22723042643208 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.716+0000 7f101c60b700 1 -- 172.21.15.181:0/3643106045 shutdown_connections 2023-10-16T09:44:09.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.716+0000 7f101c60b700 1 --2- 172.21.15.181:0/3643106045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 0x7f10140b1340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.716+0000 7f101c60b700 1 --2- 172.21.15.181:0/3643106045 >> v2:172.21.15.181:3300/0 conn(0x7f10140c9b30 0x7f101415d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.716+0000 7f101c60b700 1 -- 172.21.15.181:0/3643106045 >> 172.21.15.181:0/3643106045 conn(0x7f101406ac90 msgr2=0x7f101406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.716+0000 7f101c60b700 1 -- 172.21.15.181:0/3643106045 shutdown_connections 2023-10-16T09:44:09.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.716+0000 7f101c60b700 1 -- 172.21.15.181:0/3643106045 wait complete. 2023-10-16T09:44:09.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.717+0000 7f101c60b700 1 Processor -- start 2023-10-16T09:44:09.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.717+0000 7f101c60b700 1 -- start start 2023-10-16T09:44:09.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f101c60b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 0x7f101414fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f101c60b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10140c9b30 0x7f101415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f101c60b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1014159bd0 0x7f10141565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f101c60b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141512f0 con 0x7f10140c9b30 2023-10-16T09:44:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f101c60b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101415aba0 con 0x7f10140b0ef0 2023-10-16T09:44:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f101c60b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1014151590 con 0x7f1014159bd0 2023-10-16T09:44:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f1019ba6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10140c9b30 0x7f101415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f1019ba6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10140c9b30 0x7f101415a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36386/0 (socket says 172.21.15.181:36386) 2023-10-16T09:44:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f1019ba6700 1 -- 172.21.15.181:0/3556409562 learned_addr learned my addr 172.21.15.181:0/3556409562 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.718+0000 7f101aba8700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1014159bd0 0x7f10141565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.719+0000 7f1019ba6700 1 -- 172.21.15.181:0/3556409562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1014159bd0 msgr2=0x7f10141565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.719+0000 7f1019ba6700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1014159bd0 0x7f10141565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.719+0000 7f1019ba6700 1 -- 172.21.15.181:0/3556409562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 msgr2=0x7f101414fe60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T09:44:09.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.719+0000 7f1019ba6700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 0x7f101414fe60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.719+0000 7f1019ba6700 1 -- 172.21.15.181:0/3556409562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c00d050 con 0x7f10140c9b30 2023-10-16T09:44:09.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.719+0000 7f1019ba6700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10140c9b30 0x7f101415a150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f101000b500 tx=0x7f101000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.722+0000 7f048effd700 1 -- 172.21.15.181:0/1922121908 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488003c20 con 0x7f04900cc550 2023-10-16T09:44:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.722+0000 7f048effd700 1 -- 172.21.15.181:0/1922121908 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488003d80 con 0x7f04900cc550 2023-10-16T09:44:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.722+0000 7f048effd700 1 -- 172.21.15.181:0/1922121908 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800bbc0 con 0x7f04900cc550 2023-10-16T09:44:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f0496329700 1 -- 172.21.15.181:0/1922121908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900cc550 msgr2=0x7f049008a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f0496329700 1 --2- 172.21.15.181:0/1922121908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900cc550 0x7f049008a340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0488009e50 tx=0x7f048800b330 comp rx=0 tx=0).stop 2023-10-16T09:44:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f0496329700 1 -- 172.21.15.181:0/1922121908 shutdown_connections 2023-10-16T09:44:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f0496329700 1 --2- 172.21.15.181:0/1922121908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0490071fa0 0x7f049008a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f0496329700 1 --2- 172.21.15.181:0/1922121908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04900cc550 0x7f049008a340 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f0496329700 1 -- 172.21.15.181:0/1922121908 >> 172.21.15.181:0/1922121908 conn(0x7f049006ac90 msgr2=0x7f049006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f100affd700 1 -- 172.21.15.181:0/3556409562 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010011830 con 0x7f10140c9b30 2023-10-16T09:44:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f100affd700 1 -- 172.21.15.181:0/3556409562 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1010011990 con 0x7f10140c9b30 2023-10-16T09:44:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f100affd700 1 -- 172.21.15.181:0/3556409562 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010011d40 con 0x7f10140c9b30 2023-10-16T09:44:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1014151870 con 0x7f10140c9b30 2023-10-16T09:44:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.723+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101415ae80 con 0x7f10140c9b30 2023-10-16T09:44:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.727+0000 7f100affd700 1 -- 172.21.15.181:0/3556409562 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f101001a400 con 0x7f10140c9b30 2023-10-16T09:44:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.727+0000 7f100affd700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f0ff806af20 0x7f0ff806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.728+0000 7f100affd700 1 -- 172.21.15.181:0/3556409562 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10100894b0 con 0x7f10140c9b30 2023-10-16T09:44:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.728+0000 7f101a3a7700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f0ff806af20 0x7f0ff806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.732+0000 7f0496329700 1 -- 172.21.15.181:0/1922121908 shutdown_connections 2023-10-16T09:44:09.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.732+0000 7f0496329700 1 -- 172.21.15.181:0/1922121908 wait complete. 2023-10-16T09:44:09.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.734+0000 7f0496329700 1 Processor -- start 2023-10-16T09:44:09.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.734+0000 7f0496329700 1 -- start start 2023-10-16T09:44:09.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.735+0000 7f0496329700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490071fa0 0x7f0490208b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.735+0000 7f0496329700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04900c9cd0 0x7f049020cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.735+0000 7f0496329700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04900cc550 0x7f0490202ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.735+0000 7f0496329700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0490209080 con 0x7f0490071fa0 2023-10-16T09:44:09.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.735+0000 7f0496329700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0490209b40 con 0x7f04900c9cd0 2023-10-16T09:44:09.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.735+0000 7f0496329700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049020d100 con 0x7f04900cc550 2023-10-16T09:44:09.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490071fa0 0x7f0490208b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490071fa0 0x7f0490208b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36392/0 (socket says 172.21.15.181:36392) 2023-10-16T09:44:09.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 -- 172.21.15.181:0/651191689 learned_addr learned my addr 172.21.15.181:0/651191689 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f04948c6700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04900cc550 0x7f0490202ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048f7fe700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04900c9cd0 0x7f049020cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 -- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04900cc550 msgr2=0x7f0490202ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04900cc550 0x7f0490202ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 -- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04900c9cd0 msgr2=0x7f049020cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04900c9cd0 0x7f049020cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 -- 172.21.15.181:0/651191689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488009b30 con 0x7f0490071fa0 2023-10-16T09:44:09.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.736+0000 7f048ffff700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490071fa0 0x7f0490208b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f048800b730 tx=0x7f0488004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.739+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014151a90 con 0x7f10140c9b30 2023-10-16T09:44:09.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.739+0000 7f101a3a7700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f0ff806af20 0x7f0ff806d3e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1004001590 tx=0x7f1004007a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.749+0000 7f100affd700 1 -- 172.21.15.181:0/3556409562 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1010054640 con 0x7f10140c9b30 2023-10-16T09:44:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.750+0000 7f048cff9700 1 -- 172.21.15.181:0/651191689 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488010070 con 0x7f0490071fa0 2023-10-16T09:44:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.750+0000 7f048cff9700 1 -- 172.21.15.181:0/651191689 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04880092e0 con 0x7f0490071fa0 2023-10-16T09:44:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.750+0000 7f048cff9700 1 -- 172.21.15.181:0/651191689 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04880047c0 con 0x7f0490071fa0 2023-10-16T09:44:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.750+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04902092a0 con 0x7f0490071fa0 2023-10-16T09:44:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.750+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0490209860 con 0x7f0490071fa0 2023-10-16T09:44:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.751+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049004ebb0 con 0x7f0490071fa0 2023-10-16T09:44:09.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.759+0000 7f048cff9700 1 -- 172.21.15.181:0/651191689 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f048802a3f0 con 0x7f0490071fa0 2023-10-16T09:44:09.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.759+0000 7fc983e7f700 1 Processor -- start 2023-10-16T09:44:09.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.760+0000 7f048cff9700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f047c06afd0 0x7f047c06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.760+0000 7f048cff9700 1 -- 172.21.15.181:0/651191689 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f048808a620 con 0x7f0490071fa0 2023-10-16T09:44:09.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.761+0000 7f048f7fe700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f047c06afd0 0x7f047c06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.760+0000 7fc983e7f700 1 -- start start 2023-10-16T09:44:09.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.762+0000 7fc983e7f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc97c15d9f0 0x7fc97c0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.762+0000 7fc983e7f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c0b1430 0x7fc97c06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.762+0000 7fc983e7f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c06da00 con 0x7fc97c06d1d0 2023-10-16T09:44:09.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.762+0000 7fc983e7f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0b18b0 con 0x7fc97c15d9f0 2023-10-16T09:44:09.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.762+0000 7fc983e7f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0caad0 con 0x7fc97c0b1430 2023-10-16T09:44:09.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.763+0000 7fc981c1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc97c15d9f0 0x7fc97c0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.763+0000 7f048f7fe700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f047c06afd0 0x7f047c06d490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0490209d10 tx=0x7f0484009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.763+0000 7fc98141a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c0b1430 0x7fc97c06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.763+0000 7fc981c1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc97c15d9f0 0x7fc97c0b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41452/0 (socket says 172.21.15.181:41452) 2023-10-16T09:44:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.763+0000 7fc981c1b700 1 -- 172.21.15.181:0/2946238389 learned_addr learned my addr 172.21.15.181:0/2946238389 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.763+0000 7f9610237700 1 Processor -- start 2023-10-16T09:44:09.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7fc981c1b700 1 -- 172.21.15.181:0/2946238389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c0b1430 msgr2=0x7fc97c06cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7fc981c1b700 1 --2- 172.21.15.181:0/2946238389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c0b1430 0x7fc97c06cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7fc981c1b700 1 -- 172.21.15.181:0/2946238389 >> v1:172.21.15.181:6789/0 conn(0x7fc97c06d1d0 legacy=0x7fc97c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7fc981c1b700 1 -- 172.21.15.181:0/2946238389 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c1617a0 con 0x7fc97c15d9f0 2023-10-16T09:44:09.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.764+0000 7f9610237700 1 -- start start 2023-10-16T09:44:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7f9610237700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7f9610237700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96080b0ef0 0x7f96080b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7f9610237700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9608159fb0 con 0x7f96080c9b30 2023-10-16T09:44:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7f9610237700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080715f0 con 0x7f96080b0ef0 2023-10-16T09:44:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.765+0000 7f9610237700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96080b18e0 con 0x7f9608159bd0 2023-10-16T09:44:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96080b0ef0 0x7f96080b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96080b0ef0 0x7f96080b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36408/0 (socket says 172.21.15.181:36408) 2023-10-16T09:44:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 -- 172.21.15.181:0/26015530 learned_addr learned my addr 172.21.15.181:0/26015530 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960dfd3700 1 --2- 172.21.15.181:0/26015530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 -- 172.21.15.181:0/26015530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 msgr2=0x7f96080cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 --2- 172.21.15.181:0/26015530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 -- 172.21.15.181:0/26015530 >> v1:172.21.15.181:6789/0 conn(0x7f96080c9b30 legacy=0x7f960815d6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:44:09.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 -- 172.21.15.181:0/26015530 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608071280 con 0x7f96080b0ef0 2023-10-16T09:44:09.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f048cff9700 1 -- 172.21.15.181:0/651191689 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0488055740 con 0x7f0490071fa0 2023-10-16T09:44:09.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.766+0000 7f960e7d4700 1 --2- 172.21.15.181:0/26015530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080b0ef0 0x7f96080b13a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f96040088f0 tx=0x7f960400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df0ce0b810c243f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.767+0000 7f960cfd1700 1 -- 172.21.15.181:0/26015530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604010070 con 0x7f96080b0ef0 2023-10-16T09:44:09.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.767+0000 7f960cfd1700 1 -- 172.21.15.181:0/26015530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960400da40 con 0x7f96080b0ef0 2023-10-16T09:44:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.767+0000 7f960cfd1700 1 -- 172.21.15.181:0/26015530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400ddb0 con 0x7f96080b0ef0 2023-10-16T09:44:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 -- 172.21.15.181:0/26015530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080b0ef0 msgr2=0x7f96080b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 --2- 172.21.15.181:0/26015530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080b0ef0 0x7f96080b13a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f96040088f0 tx=0x7f960400d330 comp rx=0 tx=0).stop 2023-10-16T09:44:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 -- 172.21.15.181:0/26015530 shutdown_connections 2023-10-16T09:44:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 --2- 172.21.15.181:0/26015530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080b0ef0 0x7f96080b13a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 --2- 172.21.15.181:0/26015530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 -- 172.21.15.181:0/26015530 >> 172.21.15.181:0/26015530 conn(0x7f960806ac90 msgr2=0x7f960806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 -- 172.21.15.181:0/26015530 shutdown_connections 2023-10-16T09:44:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.768+0000 7f9610237700 1 -- 172.21.15.181:0/26015530 wait complete. 2023-10-16T09:44:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.770+0000 7f9610237700 1 Processor -- start 2023-10-16T09:44:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.771+0000 7f9610237700 1 -- start start 2023-10-16T09:44:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f9610237700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96080b0ef0 0x7f960814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f9610237700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080c9b30 0x7f96080c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f9610237700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f9610237700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080cb100 con 0x7f96080c9b30 2023-10-16T09:44:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f9610237700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960814df30 con 0x7f96080b0ef0 2023-10-16T09:44:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f9610237700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96081fb280 con 0x7f9608159bd0 2023-10-16T09:44:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f960d7d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080c9b30 0x7f96080c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f960d7d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080c9b30 0x7f96080c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36412/0 (socket says 172.21.15.181:36412) 2023-10-16T09:44:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f960d7d2700 1 -- 172.21.15.181:0/369623239 learned_addr learned my addr 172.21.15.181:0/369623239 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.772+0000 7f960dfd3700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96080b0ef0 0x7f960814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.773+0000 7f960e7d4700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.773+0000 7f960dfd3700 1 -- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 msgr2=0x7f96080cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.773+0000 7f960dfd3700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.773+0000 7f960dfd3700 1 -- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080c9b30 msgr2=0x7f96080c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.773+0000 7f960dfd3700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080c9b30 0x7f96080c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.773+0000 7f960dfd3700 1 -- 172.21.15.181:0/369623239 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040085d0 con 0x7f96080b0ef0 2023-10-16T09:44:09.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.773+0000 7f960dfd3700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96080b0ef0 0x7f960814d9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f960000cdf0 tx=0x7f960000a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.776+0000 7fc981c1b700 1 --2- 172.21.15.181:0/2946238389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0b0ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098f0 tx=0x7fc978009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a9d029f09f6bf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.776+0000 7fc980c19700 1 -- 172.21.15.181:0/2946238389 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978004230 con 0x7fc97c15d9f0 2023-10-16T09:44:09.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.776+0000 7fc980c19700 1 -- 172.21.15.181:0/2946238389 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978004390 con 0x7fc97c15d9f0 2023-10-16T09:44:09.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.776+0000 7fc980c19700 1 -- 172.21.15.181:0/2946238389 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780036a0 con 0x7fc97c15d9f0 2023-10-16T09:44:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.777+0000 7fc983e7f700 1 -- 172.21.15.181:0/2946238389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 msgr2=0x7fc97c0b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.777+0000 7fc983e7f700 1 --2- 172.21.15.181:0/2946238389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0b0ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098f0 tx=0x7fc978009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.778+0000 7fc983e7f700 1 -- 172.21.15.181:0/2946238389 shutdown_connections 2023-10-16T09:44:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.778+0000 7fc983e7f700 1 --2- 172.21.15.181:0/2946238389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0b0ef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.778+0000 7fc983e7f700 1 --2- 172.21.15.181:0/2946238389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c0b1430 0x7fc97c06cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.778+0000 7fc983e7f700 1 -- 172.21.15.181:0/2946238389 >> 172.21.15.181:0/2946238389 conn(0x7fc97c06ac90 msgr2=0x7fc97c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.784+0000 7fc983e7f700 1 -- 172.21.15.181:0/2946238389 shutdown_connections 2023-10-16T09:44:09.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.784+0000 7fc983e7f700 1 -- 172.21.15.181:0/2946238389 wait complete. 2023-10-16T09:44:09.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.786+0000 7fc983e7f700 1 Processor -- start 2023-10-16T09:44:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.786+0000 7fc983e7f700 1 -- start start 2023-10-16T09:44:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.787+0000 7fc983e7f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c06d1d0 0x7fc97c0c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.787+0000 7fc983e7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97c0b1430 0x7fc97c0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.787+0000 7fc983e7f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc983e7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c1f4080 con 0x7fc97c0b1430 2023-10-16T09:44:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc983e7f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0b5070 con 0x7fc97c15d9f0 2023-10-16T09:44:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc983e7f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c1f36e0 con 0x7fc97c06d1d0 2023-10-16T09:44:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41468/0 (socket says 172.21.15.181:41468) 2023-10-16T09:44:09.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 -- 172.21.15.181:0/1299756169 learned_addr learned my addr 172.21.15.181:0/1299756169 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc981c1b700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c06d1d0 0x7fc97c0c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 -- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c06d1d0 msgr2=0x7fc97c0c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c06d1d0 0x7fc97c0c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 -- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97c0b1430 msgr2=0x7fc97c0b45b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:44:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97c0b1430 0x7fc97c0b45b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.788+0000 7fc98241c700 1 -- 172.21.15.181:0/1299756169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978009550 con 0x7fc97c15d9f0 2023-10-16T09:44:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.789+0000 7fc98241c700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0add70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc97400b990 tx=0x7fc97400bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.789+0000 7fc9727fc700 1 -- 172.21.15.181:0/1299756169 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400c930 con 0x7fc97c15d9f0 2023-10-16T09:44:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.789+0000 7fc9727fc700 1 -- 172.21.15.181:0/1299756169 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97400ca90 con 0x7fc97c15d9f0 2023-10-16T09:44:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.789+0000 7fc9727fc700 1 -- 172.21.15.181:0/1299756169 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ce40 con 0x7fc97c15d9f0 2023-10-16T09:44:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.789+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c1f39c0 con 0x7fc97c15d9f0 2023-10-16T09:44:09.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.789+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c15ae80 con 0x7fc97c15d9f0 2023-10-16T09:44:09.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.792+0000 7fc9727fc700 1 -- 172.21.15.181:0/1299756169 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc974006d50 con 0x7fc97c15d9f0 2023-10-16T09:44:09.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.793+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c1f69c0 con 0x7fc97c15d9f0 2023-10-16T09:44:09.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.793+0000 7fc9727fc700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc96006af00 0x7fc96006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.793+0000 7fc9727fc700 1 -- 172.21.15.181:0/1299756169 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9740890f0 con 0x7fc97c15d9f0 2023-10-16T09:44:09.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.794+0000 7fc981c1b700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc96006af00 0x7fc96006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.792+0000 7f95fe7fc700 1 -- 172.21.15.181:0/369623239 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600016420 con 0x7f96080b0ef0 2023-10-16T09:44:09.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.792+0000 7f95fe7fc700 1 -- 172.21.15.181:0/369623239 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600016580 con 0x7f96080b0ef0 2023-10-16T09:44:09.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.792+0000 7f95fe7fc700 1 -- 172.21.15.181:0/369623239 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96000168f0 con 0x7f96080b0ef0 2023-10-16T09:44:09.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.792+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960815b9e0 con 0x7f96080b0ef0 2023-10-16T09:44:09.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.792+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96081fb7b0 con 0x7f96080b0ef0 2023-10-16T09:44:09.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.794+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f00052f0 con 0x7f96080b0ef0 2023-10-16T09:44:09.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.794+0000 7fc981c1b700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc96006af00 0x7fc96006d3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc978009130 tx=0x7fc978008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.800+0000 7f95fe7fc700 1 -- 172.21.15.181:0/369623239 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9600016a50 con 0x7f96080b0ef0 2023-10-16T09:44:09.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.800+0000 7f95fe7fc700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f95ec06afd0 0x7f95ec06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.801+0000 7f95fe7fc700 1 -- 172.21.15.181:0/369623239 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9600012070 con 0x7f96080b0ef0 2023-10-16T09:44:09.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.801+0000 7f960d7d2700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f95ec06afd0 0x7f95ec06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.798+0000 7fc9727fc700 1 -- 172.21.15.181:0/1299756169 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc974054110 con 0x7fc97c15d9f0 2023-10-16T09:44:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.810+0000 7f718d806700 1 Processor -- start 2023-10-16T09:44:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.810+0000 7f718d806700 1 -- start start 2023-10-16T09:44:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f718d806700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f718815daf0 0x7f71880c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f718d806700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815ded0 0x7f718806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f718d806700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718815b410 con 0x7f71880c2cb0 2023-10-16T09:44:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f718d806700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71880c1f50 con 0x7f718815daf0 2023-10-16T09:44:09.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f718d806700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71880cfa70 con 0x7f718815ded0 2023-10-16T09:44:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f71867fc700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f71880c2cb0 0x7f71880c2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59676/0 (socket says 172.21.15.181:59676) 2023-10-16T09:44:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f71867fc700 1 -- 172.21.15.181:0/4042660310 learned_addr learned my addr 172.21.15.181:0/4042660310 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f7186ffd700 1 --2- 172.21.15.181:0/4042660310 >> v2:172.21.15.190:3300/0 conn(0x7f718815daf0 0x7f71880c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.811+0000 7f71877fe700 1 --2- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815ded0 0x7f718806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7186ffd700 1 -- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815ded0 msgr2=0x7f718806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7186ffd700 1 --2- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815ded0 0x7f718806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7186ffd700 1 -- 172.21.15.181:0/4042660310 >> v1:172.21.15.190:6789/0 conn(0x7f71880c2cb0 legacy=0x7f71880c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7186ffd700 1 -- 172.21.15.181:0/4042660310 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71880c2a80 con 0x7f718815daf0 2023-10-16T09:44:09.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7186ffd700 1 --2- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718815daf0 0x7f71880c31e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7170009870 tx=0x7f7170009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7970a9cbc1ebf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7185ffb700 1 -- 172.21.15.181:0/4042660310 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000c880 con 0x7f718815daf0 2023-10-16T09:44:09.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.801+0000 7f960d7d2700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f95ec06afd0 0x7f95ec06d490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9608159470 tx=0x7f95f8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f95fe7fc700 1 -- 172.21.15.181:0/369623239 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f96000555e0 con 0x7f96080b0ef0 2023-10-16T09:44:09.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7185ffb700 1 -- 172.21.15.181:0/4042660310 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717000c9e0 con 0x7f718815daf0 2023-10-16T09:44:09.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.812+0000 7f7185ffb700 1 -- 172.21.15.181:0/4042660310 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000cd70 con 0x7f718815daf0 2023-10-16T09:44:09.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.813+0000 7f718d806700 1 -- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718815daf0 msgr2=0x7f71880c31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.813+0000 7f718d806700 1 --2- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718815daf0 0x7f71880c31e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7170009870 tx=0x7f7170009b80 comp rx=0 tx=0).stop 2023-10-16T09:44:09.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.814+0000 7f718d806700 1 -- 172.21.15.181:0/4042660310 shutdown_connections 2023-10-16T09:44:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.814+0000 7f718d806700 1 --2- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815ded0 0x7f718806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.814+0000 7f718d806700 1 --2- 172.21.15.181:0/4042660310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718815daf0 0x7f71880c31e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.814+0000 7f718d806700 1 -- 172.21.15.181:0/4042660310 >> 172.21.15.181:0/4042660310 conn(0x7f71880b1780 msgr2=0x7f71880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.829+0000 7f718d806700 1 -- 172.21.15.181:0/4042660310 shutdown_connections 2023-10-16T09:44:09.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.829+0000 7f718d806700 1 -- 172.21.15.181:0/4042660310 wait complete. 2023-10-16T09:44:09.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.831+0000 7f718d806700 1 Processor -- start 2023-10-16T09:44:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.831+0000 7f718d806700 1 -- start start 2023-10-16T09:44:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.832+0000 7f718d806700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815daf0 0x7f71880aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.832+0000 7f718d806700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f718815ded0 0x7f71881f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.832+0000 7f718d806700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718806e200 0x7f71881f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.832+0000 7f718d806700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880c4760 con 0x7f718815ded0 2023-10-16T09:44:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.832+0000 7f718d806700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880aef40 con 0x7f718806e200 2023-10-16T09:44:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.832+0000 7f718d806700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71880ab970 con 0x7f718815daf0 2023-10-16T09:44:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f718815ded0 0x7f71881f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f7186ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815daf0 0x7f71880aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f718815ded0 0x7f71881f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36432/0 (socket says 172.21.15.181:36432) 2023-10-16T09:44:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 -- 172.21.15.181:0/4285000921 learned_addr learned my addr 172.21.15.181:0/4285000921 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 -- 172.21.15.181:0/4285000921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815daf0 msgr2=0x7f71880aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815daf0 0x7f71880aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 -- 172.21.15.181:0/4285000921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718806e200 msgr2=0x7f71881f76f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T09:44:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718806e200 0x7f71881f76f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.833+0000 7f71867fc700 1 -- 172.21.15.181:0/4285000921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700094a0 con 0x7f718815ded0 2023-10-16T09:44:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.834+0000 7f71867fc700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f718815ded0 0x7f71881f71b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f717800c500 tx=0x7f717800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.840+0000 7f717ffff700 1 -- 172.21.15.181:0/4285000921 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178005d70 con 0x7f718815ded0 2023-10-16T09:44:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.840+0000 7f717ffff700 1 -- 172.21.15.181:0/4285000921 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71780070d0 con 0x7f718815ded0 2023-10-16T09:44:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.840+0000 7f717ffff700 1 -- 172.21.15.181:0/4285000921 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178007440 con 0x7f718815ded0 2023-10-16T09:44:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.841+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71880abc50 con 0x7f718815ded0 2023-10-16T09:44:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.841+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71880c43f0 con 0x7f718815ded0 2023-10-16T09:44:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.842+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7188067d20 con 0x7f718815ded0 2023-10-16T09:44:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.838+0000 7f2e9c8df700 1 Processor -- start 2023-10-16T09:44:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.838+0000 7f2e9c8df700 1 -- start start 2023-10-16T09:44:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.839+0000 7f2e9c8df700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e980c9cd0 0x7f2e98070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.839+0000 7f2e9c8df700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2e980cc930 0x7f2e980714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.839+0000 7f2e9c8df700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e980ca4d0 con 0x7f2e980cc550 2023-10-16T09:44:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.839+0000 7f2e9c8df700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9816ef80 con 0x7f2e980c9cd0 2023-10-16T09:44:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.839+0000 7f2e9c8df700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9816f260 con 0x7f2e980cc930 2023-10-16T09:44:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.840+0000 7f2e9659c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2e980cc550 0x7f2e9808a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59690/0 (socket says 172.21.15.181:59690) 2023-10-16T09:44:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.840+0000 7f2e9659c700 1 -- 172.21.15.181:0/2884735475 learned_addr learned my addr 172.21.15.181:0/2884735475 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.842+0000 7f2e96d9d700 1 --2- 172.21.15.181:0/2884735475 >> v2:172.21.15.190:3300/0 conn(0x7f2e980cc930 0x7f2e980714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.843+0000 7f717ffff700 1 -- 172.21.15.181:0/4285000921 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f71780075a0 con 0x7f718815ded0 2023-10-16T09:44:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f717ffff700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f716806af50 0x7f716806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f717ffff700 1 -- 172.21.15.181:0/4285000921 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71780888f0 con 0x7f718815ded0 2023-10-16T09:44:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f7186ffd700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f716806af50 0x7f716806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.845+0000 7f7186ffd700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f716806af50 0x7f716806d410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71700090b0 tx=0x7f7170008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f2e95d9b700 1 --2- 172.21.15.181:0/2884735475 >> v2:172.21.15.181:3300/0 conn(0x7f2e980c9cd0 0x7f2e98070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f2e95d9b700 1 -- 172.21.15.181:0/2884735475 >> v2:172.21.15.190:3300/0 conn(0x7f2e980cc930 msgr2=0x7f2e980714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f2e95d9b700 1 --2- 172.21.15.181:0/2884735475 >> v2:172.21.15.190:3300/0 conn(0x7f2e980cc930 0x7f2e980714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f2e95d9b700 1 -- 172.21.15.181:0/2884735475 >> v1:172.21.15.190:6789/0 conn(0x7f2e980cc550 legacy=0x7f2e9808a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.844+0000 7f2e95d9b700 1 -- 172.21.15.181:0/2884735475 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980ca720 con 0x7f2e980c9cd0 2023-10-16T09:44:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.845+0000 7f2e95d9b700 1 --2- 172.21.15.181:0/2884735475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980c9cd0 0x7f2e98070f90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2e980b1f70 tx=0x7f2e8800a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d369765c178ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.853+0000 7f2e9559a700 1 -- 172.21.15.181:0/2884735475 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88015df0 con 0x7f2e980c9cd0 2023-10-16T09:44:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.853+0000 7f717ffff700 1 -- 172.21.15.181:0/4285000921 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7178053a30 con 0x7f718815ded0 2023-10-16T09:44:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.853+0000 7f2e9559a700 1 -- 172.21.15.181:0/2884735475 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88004030 con 0x7f2e980c9cd0 2023-10-16T09:44:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.853+0000 7f2e9559a700 1 -- 172.21.15.181:0/2884735475 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880043e0 con 0x7f2e980c9cd0 2023-10-16T09:44:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.853+0000 7f5e864a2700 1 Processor -- start 2023-10-16T09:44:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.854+0000 7f2e9c8df700 1 -- 172.21.15.181:0/2884735475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980c9cd0 msgr2=0x7f2e98070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.854+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/2884735475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980c9cd0 0x7f2e98070f90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2e980b1f70 tx=0x7f2e8800a370 comp rx=0 tx=0).stop 2023-10-16T09:44:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.853+0000 7f5e864a2700 1 -- start start 2023-10-16T09:44:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.845+0000 7fed2aad0700 1 Processor -- start 2023-10-16T09:44:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.845+0000 7fed2aad0700 1 -- start start 2023-10-16T09:44:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.847+0000 7fed2aad0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c038cd0 0x7fed1c005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.847+0000 7fed2aad0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1c053650 con 0x7fed1c03a8f0 2023-10-16T09:44:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.847+0000 7fed2aad0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1c053920 con 0x7fed1c100f90 2023-10-16T09:44:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.847+0000 7fed2aad0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c0549c0 con 0x7fed1c038cd0 2023-10-16T09:44:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.847+0000 7fed2a2cf700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fed1c03a8f0 0x7fed1c0064a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47524/0 (socket says 172.21.15.181:47524) 2023-10-16T09:44:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.847+0000 7fed2a2cf700 1 -- 172.21.15.181:0/4270205562 learned_addr learned my addr 172.21.15.181:0/4270205562 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.849+0000 7fed292cd700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c038cd0 0x7fed1c005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.853+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238632711 0 0) 0x7fed1c053650 con 0x7fed1c03a8f0 2023-10-16T09:44:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.854+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed100035b0 con 0x7fed1c03a8f0 2023-10-16T09:44:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.854+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383617772 0 0) 0x7fed1c053920 con 0x7fed1c100f90 2023-10-16T09:44:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.854+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed1c053650 con 0x7fed1c100f90 2023-10-16T09:44:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.854+0000 7f5e864a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c2cb0 0x7f5e800c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.854+0000 7f5e864a2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e800665a0 0x7f5e800c2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e864a2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e800c28b0 0x7f5e8006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e864a2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e800c1f50 con 0x7f5e800665a0 2023-10-16T09:44:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e864a2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e800cfa70 con 0x7f5e800c28b0 2023-10-16T09:44:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e864a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8015e900 con 0x7f5e800c2cb0 2023-10-16T09:44:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e800665a0 0x7f5e800c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e800665a0 0x7f5e800c2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36450/0 (socket says 172.21.15.181:36450) 2023-10-16T09:44:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 -- 172.21.15.181:0/440863985 learned_addr learned my addr 172.21.15.181:0/440863985 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 -- 172.21.15.181:0/440863985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c2cb0 msgr2=0x7f5e800c31e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:44:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 --2- 172.21.15.181:0/440863985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c2cb0 0x7f5e800c31e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 -- 172.21.15.181:0/440863985 >> v2:172.21.15.190:3300/0 conn(0x7f5e800c28b0 msgr2=0x7f5e8006a8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:44:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 --2- 172.21.15.181:0/440863985 >> v2:172.21.15.190:3300/0 conn(0x7f5e800c28b0 0x7f5e8006a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 -- 172.21.15.181:0/440863985 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800c15b0 con 0x7f5e800665a0 2023-10-16T09:44:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327160959 0 0) 0x7fed100035b0 con 0x7fed1c03a8f0 2023-10-16T09:44:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c038cd0 msgr2=0x7fed1c005f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c038cd0 0x7fed1c005f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 >> v1:172.21.15.190:6789/0 conn(0x7fed1c100f90 legacy=0x7fed1c052f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c053c00 con 0x7fed1c03a8f0 2023-10-16T09:44:09.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fed2404f400 con 0x7fed1c03a8f0 2023-10-16T09:44:09.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 >> v1:172.21.15.181:6789/0 conn(0x7fed1c03a8f0 legacy=0x7fed1c0064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed10004c20 0x7fed10005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100037f0 0x7fed10003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed100041b0 0x7fed1000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c0549c0 con 0x7fed100041b0 2023-10-16T09:44:09.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1000bf50 con 0x7fed100037f0 2023-10-16T09:44:09.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1000c090 con 0x7fed10004c20 2023-10-16T09:44:09.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7fed2a2cf700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed100041b0 0x7fed1000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.855+0000 7f5e7f7fe700 1 --2- 172.21.15.181:0/440863985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c2370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e800b1710 tx=0x7f5e70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedeb1a00147ff68 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed2a2cf700 1 -- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed10004c20 msgr2=0x7fed10005020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T09:44:09.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed2a2cf700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed10004c20 0x7fed10005020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed2a2cf700 1 -- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100037f0 msgr2=0x7fed10003c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:44:09.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed2a2cf700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100037f0 0x7fed10003c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed2a2cf700 1 -- 172.21.15.181:0/4270205562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c053c00 con 0x7fed100041b0 2023-10-16T09:44:09.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7f5e7effd700 1 -- 172.21.15.181:0/440863985 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000b860 con 0x7f5e800665a0 2023-10-16T09:44:09.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7f5e7effd700 1 -- 172.21.15.181:0/440863985 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000b9c0 con 0x7f5e800665a0 2023-10-16T09:44:09.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7f5e7effd700 1 -- 172.21.15.181:0/440863985 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000bd50 con 0x7f5e800665a0 2023-10-16T09:44:09.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed2a2cf700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed100041b0 0x7fed1000ba10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fed24068950 tx=0x7fed10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babd22b44371326a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2406cba0 con 0x7fed100041b0 2023-10-16T09:44:09.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2407d070 con 0x7fed100041b0 2023-10-16T09:44:09.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.856+0000 7fed28acc700 1 -- 172.21.15.181:0/4270205562 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2406a8a0 con 0x7fed100041b0 2023-10-16T09:44:09.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 -- 172.21.15.181:0/440863985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 msgr2=0x7f5e800c2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 --2- 172.21.15.181:0/440863985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c2370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e800b1710 tx=0x7f5e70009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:09.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f2e9c8df700 1 -- 172.21.15.181:0/2884735475 shutdown_connections 2023-10-16T09:44:09.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/2884735475 >> v2:172.21.15.190:3300/0 conn(0x7f2e980cc930 0x7f2e980714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/2884735475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980c9cd0 0x7f2e98070f90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f2e9c8df700 1 -- 172.21.15.181:0/2884735475 >> 172.21.15.181:0/2884735475 conn(0x7f2e9806ac90 msgr2=0x7f2e9806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 -- 172.21.15.181:0/440863985 shutdown_connections 2023-10-16T09:44:09.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 --2- 172.21.15.181:0/440863985 >> v2:172.21.15.190:3300/0 conn(0x7f5e800c28b0 0x7f5e8006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 --2- 172.21.15.181:0/440863985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c2cb0 0x7f5e800c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 --2- 172.21.15.181:0/440863985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c2370 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 -- 172.21.15.181:0/440863985 >> 172.21.15.181:0/440863985 conn(0x7f5e800b1780 msgr2=0x7f5e800b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f2e9c8df700 1 -- 172.21.15.181:0/2884735475 shutdown_connections 2023-10-16T09:44:09.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f2e9c8df700 1 -- 172.21.15.181:0/2884735475 wait complete. 2023-10-16T09:44:09.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7fed2aad0700 1 -- 172.21.15.181:0/4270205562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed100041b0 msgr2=0x7fed1000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7fed2aad0700 1 --2- 172.21.15.181:0/4270205562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed100041b0 0x7fed1000ba10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fed24068950 tx=0x7fed10005880 comp rx=0 tx=0).stop 2023-10-16T09:44:09.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7fed2a2cf700 1 -- 172.21.15.181:0/4270205562 reap_dead start 2023-10-16T09:44:09.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7fed2aad0700 1 -- 172.21.15.181:0/4270205562 shutdown_connections 2023-10-16T09:44:09.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7fed2aad0700 1 -- 172.21.15.181:0/4270205562 >> 172.21.15.181:0/4270205562 conn(0x7fed1c054b90 msgr2=0x7fed1c054fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:09.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.857+0000 7f5e864a2700 1 -- 172.21.15.181:0/440863985 shutdown_connections 2023-10-16T09:44:09.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.859+0000 7f2e9c8df700 1 Processor -- start 2023-10-16T09:44:09.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.859+0000 7f2e9c8df700 1 -- start start 2023-10-16T09:44:09.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.860+0000 7f2e9c8df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e980c9cd0 0x7f2e9808a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.860+0000 7f2e9c8df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980cc550 0x7f2e980c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.860+0000 7f2e9c8df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e980cc930 0x7f2e980c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.860+0000 7f2e9c8df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e981622b0 con 0x7f2e980cc550 2023-10-16T09:44:09.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.860+0000 7f2e9c8df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e98162590 con 0x7f2e980cc930 2023-10-16T09:44:09.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.860+0000 7f2e9c8df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e98162870 con 0x7f2e980c9cd0 2023-10-16T09:44:09.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e980c9cd0 0x7f2e9808a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e95d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980cc550 0x7f2e980c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e980c9cd0 0x7f2e9808a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33486/0 (socket says 172.21.15.181:33486) 2023-10-16T09:44:09.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e95d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980cc550 0x7f2e980c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36462/0 (socket says 172.21.15.181:36462) 2023-10-16T09:44:09.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 -- 172.21.15.181:0/3748604434 learned_addr learned my addr 172.21.15.181:0/3748604434 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.858+0000 7f5e864a2700 1 -- 172.21.15.181:0/440863985 wait complete. 2023-10-16T09:44:09.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.859+0000 7f5e864a2700 1 Processor -- start 2023-10-16T09:44:09.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.860+0000 7f5e864a2700 1 -- start start 2023-10-16T09:44:09.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e864a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e864a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c28b0 0x7f5e8014bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e864a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e800bf600 0x7f5e800bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e864a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e800c5a10 con 0x7f5e800665a0 2023-10-16T09:44:09.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e864a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e800c6840 con 0x7f5e800bf600 2023-10-16T09:44:09.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e864a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e800c5c30 con 0x7f5e800c28b0 2023-10-16T09:44:09.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e96d9d700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e980cc930 0x7f2e980c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36470/0 (socket says 172.21.15.181:36470) 2023-10-16T09:44:09.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f5e7ffff700 1 -- 172.21.15.181:0/3547824851 learned_addr learned my addr 172.21.15.181:0/3547824851 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 -- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e980cc930 msgr2=0x7f2e980c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e980cc930 0x7f2e980c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 -- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980cc550 msgr2=0x7f2e980c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980cc550 0x7f2e980c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.861+0000 7f2e9659c700 1 -- 172.21.15.181:0/3748604434 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8800a630 con 0x7f2e980c9cd0 2023-10-16T09:44:09.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f2e9659c700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e980c9cd0 0x7f2e9808a080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c780 tx=0x7f2e8c00ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f5e7ffff700 1 -- 172.21.15.181:0/3547824851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c28b0 msgr2=0x7f5e8014bf50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T09:44:09.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f5e7ffff700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c28b0 0x7f5e8014bf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f5e7ffff700 1 -- 172.21.15.181:0/3547824851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e800bf600 msgr2=0x7f5e800bfab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T09:44:09.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f5e7ffff700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e800bf600 0x7f5e800bfab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f5e7ffff700 1 -- 172.21.15.181:0/3547824851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70009550 con 0x7f5e800665a0 2023-10-16T09:44:09.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f5e7ffff700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c3810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400b500 tx=0x7f5e7400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f2e86ffd700 1 -- 172.21.15.181:0/3748604434 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c004c80 con 0x7f2e980c9cd0 2023-10-16T09:44:09.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f2e86ffd700 1 -- 172.21.15.181:0/3748604434 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c004de0 con 0x7f2e980c9cd0 2023-10-16T09:44:09.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f2e86ffd700 1 -- 172.21.15.181:0/3748604434 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c007310 con 0x7f2e980c9cd0 2023-10-16T09:44:09.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e98162b50 con 0x7f2e980c9cd0 2023-10-16T09:44:09.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.862+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e98161510 con 0x7f2e980c9cd0 2023-10-16T09:44:09.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.864+0000 7fed2aad0700 1 -- 172.21.15.181:0/4270205562 shutdown_connections 2023-10-16T09:44:09.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.864+0000 7fed2aad0700 1 -- 172.21.15.181:0/4270205562 wait complete. 2023-10-16T09:44:09.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.865+0000 7fed2aad0700 1 Processor -- start 2023-10-16T09:44:09.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.865+0000 7fed2aad0700 1 -- start start 2023-10-16T09:44:09.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed2aad0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100041b0 0x7fed1c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed2aad0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed1c1a2df0 0x7fed1c034760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed2aad0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c0ff590 0x7fed1c0ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed2aad0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c0fff50 con 0x7fed1c1a2df0 2023-10-16T09:44:09.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed2aad0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c19eaf0 con 0x7fed100041b0 2023-10-16T09:44:09.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed2aad0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c19ed30 con 0x7fed1c0ff590 2023-10-16T09:44:09.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed292cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed1c1a2df0 0x7fed1c034760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed292cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed1c1a2df0 0x7fed1c034760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36476/0 (socket says 172.21.15.181:36476) 2023-10-16T09:44:09.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7fed292cd700 1 -- 172.21.15.181:0/386446358 learned_addr learned my addr 172.21.15.181:0/386446358 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.865+0000 7f2e86ffd700 1 -- 172.21.15.181:0/3748604434 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2e8c007470 con 0x7f2e980c9cd0 2023-10-16T09:44:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7f2e86ffd700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e8006b0b0 0x7f2e8006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.866+0000 7f2e86ffd700 1 -- 172.21.15.181:0/3748604434 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e8c089210 con 0x7f2e980c9cd0 2023-10-16T09:44:09.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed29ace700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100041b0 0x7fed1c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed292cd700 1 -- 172.21.15.181:0/386446358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c0ff590 msgr2=0x7fed1c0ffa10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T09:44:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed292cd700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c0ff590 0x7fed1c0ffa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed292cd700 1 -- 172.21.15.181:0/386446358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100041b0 msgr2=0x7fed1c19c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:09.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed292cd700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100041b0 0x7fed1c19c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:09.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed292cd700 1 -- 172.21.15.181:0/386446358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c19f010 con 0x7fed1c1a2df0 2023-10-16T09:44:09.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed292cd700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed1c1a2df0 0x7fed1c034760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed200079a0 tx=0x7fed20007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed167fc700 1 -- 172.21.15.181:0/386446358 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20004440 con 0x7fed1c1a2df0 2023-10-16T09:44:09.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed167fc700 1 -- 172.21.15.181:0/386446358 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200045a0 con 0x7fed1c1a2df0 2023-10-16T09:44:09.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed167fc700 1 -- 172.21.15.181:0/386446358 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200038c0 con 0x7fed1c1a2df0 2023-10-16T09:44:09.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1c19f2f0 con 0x7fed1c1a2df0 2023-10-16T09:44:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.867+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1c19f8b0 con 0x7fed1c1a2df0 2023-10-16T09:44:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.869+0000 7fed167fc700 1 -- 172.21.15.181:0/386446358 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fed20003a20 con 0x7fed1c1a2df0 2023-10-16T09:44:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.870+0000 7fed167fc700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fed0806af00 0x7fed0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.870+0000 7fed167fc700 1 -- 172.21.15.181:0/386446358 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed20089190 con 0x7fed1c1a2df0 2023-10-16T09:44:09.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.870+0000 7fed29ace700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fed0806af00 0x7fed0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.870+0000 7fed29ace700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fed0806af00 0x7fed0806d3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed18000f60 tx=0x7fed18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.874+0000 7f2e95d9b700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e8006b0b0 0x7f2e8006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.875+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1c008aa0 con 0x7fed1c1a2df0 2023-10-16T09:44:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.875+0000 7f2e95d9b700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e8006b0b0 0x7f2e8006d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e88000c00 tx=0x7f2e88019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.875+0000 7f5e7cff9700 1 -- 172.21.15.181:0/3547824851 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74011830 con 0x7f5e800665a0 2023-10-16T09:44:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.875+0000 7f5e7cff9700 1 -- 172.21.15.181:0/3547824851 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e74011990 con 0x7f5e800665a0 2023-10-16T09:44:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.875+0000 7f5e7cff9700 1 -- 172.21.15.181:0/3547824851 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74011d20 con 0x7f5e800665a0 2023-10-16T09:44:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.875+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e800c5f10 con 0x7f5e800665a0 2023-10-16T09:44:09.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.875+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e800c64d0 con 0x7f5e800665a0 2023-10-16T09:44:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.880+0000 7fed167fc700 1 -- 172.21.15.181:0/386446358 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fed20054240 con 0x7fed1c1a2df0 2023-10-16T09:44:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.881+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e600052f0 con 0x7f5e800665a0 2023-10-16T09:44:09.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.881+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7c0052f0 con 0x7f2e980c9cd0 2023-10-16T09:44:09.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.882+0000 7f5e7cff9700 1 -- 172.21.15.181:0/3547824851 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e7401a400 con 0x7f5e800665a0 2023-10-16T09:44:09.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.883+0000 7f5e7cff9700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6c06afd0 0x7f5e6c06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:09.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.883+0000 7f5e7cff9700 1 -- 172.21.15.181:0/3547824851 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e74089570 con 0x7f5e800665a0 2023-10-16T09:44:09.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.883+0000 7f5e7f7fe700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6c06afd0 0x7f5e6c06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:09.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.884+0000 7f5e7f7fe700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6c06afd0 0x7f5e6c06d490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e8015a2b0 tx=0x7f5e70008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:09.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.888+0000 7f2e86ffd700 1 -- 172.21.15.181:0/3748604434 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2e8c0543a0 con 0x7f2e980c9cd0 2023-10-16T09:44:09.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:09.888+0000 7f5e7cff9700 1 -- 172.21.15.181:0/3547824851 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5e74054690 con 0x7f5e800665a0 2023-10-16T09:44:10.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.119+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f101404ebb0 con 0x7f10140c9b30 2023-10-16T09:44:10.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.119+0000 7f100affd700 1 -- 172.21.15.181:0/3556409562 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1010015030 con 0x7f10140c9b30 2023-10-16T09:44:10.123 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:10.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f0ff806af20 msgr2=0x7f0ff806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f0ff806af20 0x7f0ff806d3e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1004001590 tx=0x7f1004007a20 comp rx=0 tx=0).stop 2023-10-16T09:44:10.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10140c9b30 msgr2=0x7f101415a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10140c9b30 0x7f101415a150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f101000b500 tx=0x7f101000b810 comp rx=0 tx=0).stop 2023-10-16T09:44:10.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 shutdown_connections 2023-10-16T09:44:10.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f0ff806af20 0x7f0ff806d3e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1014159bd0 0x7f10141565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10140b0ef0 0x7f101414fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 --2- 172.21.15.181:0/3556409562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10140c9b30 0x7f101415a150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.123+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 >> 172.21.15.181:0/3556409562 conn(0x7f101406ac90 msgr2=0x7f101406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.124+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 shutdown_connections 2023-10-16T09:44:10.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.124+0000 7f101c60b700 1 -- 172.21.15.181:0/3556409562 wait complete. 2023-10-16T09:44:10.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2023-10-16T09:44:10.143 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.180+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f049004ebb0 con 0x7f0490071fa0 2023-10-16T09:44:10.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.180+0000 7f048cff9700 1 -- 172.21.15.181:0/651191689 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0488058d60 con 0x7f0490071fa0 2023-10-16T09:44:10.182 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:10.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f047c06afd0 msgr2=0x7f047c06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f047c06afd0 0x7f047c06d490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0490209d10 tx=0x7f0484009040 comp rx=0 tx=0).stop 2023-10-16T09:44:10.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490071fa0 msgr2=0x7f0490208b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490071fa0 0x7f0490208b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f048800b730 tx=0x7f0488004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 shutdown_connections 2023-10-16T09:44:10.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f047c06afd0 0x7f047c06d490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04900cc550 0x7f0490202ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490071fa0 0x7f0490208b40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 --2- 172.21.15.181:0/651191689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04900c9cd0 0x7f049020cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.184+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 >> 172.21.15.181:0/651191689 conn(0x7f049006ac90 msgr2=0x7f04900723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.185+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 shutdown_connections 2023-10-16T09:44:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.185+0000 7f0496329700 1 -- 172.21.15.181:0/651191689 wait complete. 2023-10-16T09:44:10.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.192+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f95f0005870 con 0x7f96080b0ef0 2023-10-16T09:44:10.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.193+0000 7f95fe7fc700 1 -- 172.21.15.181:0/369623239 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9600024090 con 0x7f96080b0ef0 2023-10-16T09:44:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.196+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f95ec06afd0 msgr2=0x7f95ec06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.196+0000 7f9610237700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f95ec06afd0 0x7f95ec06d490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9608159470 tx=0x7f95f8009190 comp rx=0 tx=0).stop 2023-10-16T09:44:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.196+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96080b0ef0 msgr2=0x7f960814d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.196+0000 7f9610237700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96080b0ef0 0x7f960814d9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f960000cdf0 tx=0x7f960000a570 comp rx=0 tx=0).stop 2023-10-16T09:44:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.197+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 shutdown_connections 2023-10-16T09:44:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.197+0000 7f9610237700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f95ec06afd0 0x7f95ec06d490 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.197+0000 7f9610237700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9608159bd0 0x7f96080cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.197+0000 7f9610237700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96080b0ef0 0x7f960814d9f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.197+0000 7f9610237700 1 --2- 172.21.15.181:0/369623239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96080c9b30 0x7f96080c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.197+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 >> 172.21.15.181:0/369623239 conn(0x7f960806ac90 msgr2=0x7f96080b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.201+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 shutdown_connections 2023-10-16T09:44:10.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.201+0000 7f9610237700 1 -- 172.21.15.181:0/369623239 wait complete. 2023-10-16T09:44:10.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2023-10-16T09:44:10.204 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.207+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc97c04ebb0 con 0x7fc97c15d9f0 2023-10-16T09:44:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.208+0000 7fc9727fc700 1 -- 172.21.15.181:0/1299756169 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc974019020 con 0x7fc97c15d9f0 2023-10-16T09:44:10.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.210+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f71880c48a0 con 0x7f718815ded0 2023-10-16T09:44:10.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.210+0000 7f717ffff700 1 -- 172.21.15.181:0/4285000921 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7178017020 con 0x7f718815ded0 2023-10-16T09:44:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.212+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f716806af50 msgr2=0x7f716806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.212+0000 7f718d806700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f716806af50 0x7f716806d410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71700090b0 tx=0x7f7170008030 comp rx=0 tx=0).stop 2023-10-16T09:44:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.212+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f718815ded0 msgr2=0x7f71881f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.213+0000 7f718d806700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f718815ded0 0x7f71881f71b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f717800c500 tx=0x7f717800c810 comp rx=0 tx=0).stop 2023-10-16T09:44:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 shutdown_connections 2023-10-16T09:44:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f716806af50 0x7f716806d410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f718815daf0 0x7f71880aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f718806e200 0x7f71881f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 --2- 172.21.15.181:0/4285000921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f718815ded0 0x7f71881f71b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 >> 172.21.15.181:0/4285000921 conn(0x7f71880b1780 msgr2=0x7f71880b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 shutdown_connections 2023-10-16T09:44:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.215+0000 7f718d806700 1 -- 172.21.15.181:0/4285000921 wait complete. 2023-10-16T09:44:10.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.216+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc96006af00 msgr2=0x7fc96006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.216+0000 7fc983e7f700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc96006af00 0x7fc96006d3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc978009130 tx=0x7fc978008030 comp rx=0 tx=0).stop 2023-10-16T09:44:10.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.216+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 msgr2=0x7fc97c0add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.216+0000 7fc983e7f700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0add70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc97400b990 tx=0x7fc97400bca0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.217+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 shutdown_connections 2023-10-16T09:44:10.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.217+0000 7fc983e7f700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc96006af00 0x7fc96006d3c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.217+0000 7fc983e7f700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc97c06d1d0 0x7fc97c0c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.217+0000 7fc983e7f700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc97c15d9f0 0x7fc97c0add70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.217+0000 7fc983e7f700 1 --2- 172.21.15.181:0/1299756169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97c0b1430 0x7fc97c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.217+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 >> 172.21.15.181:0/1299756169 conn(0x7fc97c06ac90 msgr2=0x7fc97c15b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2023-10-16T09:44:10.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.221+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 shutdown_connections 2023-10-16T09:44:10.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.221+0000 7fc983e7f700 1 -- 172.21.15.181:0/1299756169 wait complete. 2023-10-16T09:44:10.222 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2023-10-16T09:44:10.230 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2023-10-16T09:44:10.235 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.252+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fed1c0053d0 con 0x7fed1c1a2df0 2023-10-16T09:44:10.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.253+0000 7fed167fc700 1 -- 172.21.15.181:0/386446358 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed2001b310 con 0x7fed1c1a2df0 2023-10-16T09:44:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fed0806af00 msgr2=0x7fed0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fed0806af00 0x7fed0806d3c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed18000f60 tx=0x7fed18009040 comp rx=0 tx=0).stop 2023-10-16T09:44:10.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed1c1a2df0 msgr2=0x7fed1c034760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed1c1a2df0 0x7fed1c034760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed200079a0 tx=0x7fed20007d60 comp rx=0 tx=0).stop 2023-10-16T09:44:10.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 shutdown_connections 2023-10-16T09:44:10.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fed0806af00 0x7fed0806d3c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed1c0ff590 0x7fed1c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed100041b0 0x7fed1c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 --2- 172.21.15.181:0/386446358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed1c1a2df0 0x7fed1c034760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 >> 172.21.15.181:0/386446358 conn(0x7fed1c054b90 msgr2=0x7fed1c052740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 shutdown_connections 2023-10-16T09:44:10.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.255+0000 7fed2aad0700 1 -- 172.21.15.181:0/386446358 wait complete. 2023-10-16T09:44:10.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2023-10-16T09:44:10.268 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.292+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2e7c005870 con 0x7f2e980c9cd0 2023-10-16T09:44:10.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.293+0000 7f2e86ffd700 1 -- 172.21.15.181:0/3748604434 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e8c0579c0 con 0x7f2e980c9cd0 2023-10-16T09:44:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:60129542149 2023-10-16T09:44:10.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e8006b0b0 msgr2=0x7f2e8006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e8006b0b0 0x7f2e8006d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e88000c00 tx=0x7f2e88019000 comp rx=0 tx=0).stop 2023-10-16T09:44:10.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e980c9cd0 msgr2=0x7f2e9808a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e980c9cd0 0x7f2e9808a080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c780 tx=0x7f2e8c00ca90 comp rx=0 tx=0).stop 2023-10-16T09:44:10.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 shutdown_connections 2023-10-16T09:44:10.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e8006b0b0 0x7f2e8006d570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e980c9cd0 0x7f2e9808a080 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e980cc930 0x7f2e980c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 --2- 172.21.15.181:0/3748604434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e980cc550 0x7f2e980c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 >> 172.21.15.181:0/3748604434 conn(0x7f2e9806ac90 msgr2=0x7f2e980b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 shutdown_connections 2023-10-16T09:44:10.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.296+0000 7f2e9c8df700 1 -- 172.21.15.181:0/3748604434 wait complete. 2023-10-16T09:44:10.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.299+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5e600055a0 con 0x7f5e800665a0 2023-10-16T09:44:10.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.299+0000 7f5e7cff9700 1 -- 172.21.15.181:0/3547824851 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5e74015020 con 0x7f5e800665a0 2023-10-16T09:44:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:60129542150 2023-10-16T09:44:10.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6c06afd0 msgr2=0x7f5e6c06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6c06afd0 0x7f5e6c06d490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e8015a2b0 tx=0x7f5e70008030 comp rx=0 tx=0).stop 2023-10-16T09:44:10.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 msgr2=0x7f5e800c3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c3810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400b500 tx=0x7f5e7400b8c0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 shutdown_connections 2023-10-16T09:44:10.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6c06afd0 0x7f5e6c06d490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e800bf600 0x7f5e800bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e800665a0 0x7f5e800c3810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 --2- 172.21.15.181:0/3547824851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e800c28b0 0x7f5e8014bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 >> 172.21.15.181:0/3547824851 conn(0x7f5e800b1780 msgr2=0x7f5e800b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 shutdown_connections 2023-10-16T09:44:10.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.303+0000 7f5e864a2700 1 -- 172.21.15.181:0/3547824851 wait complete. 2023-10-16T09:44:10.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2023-10-16T09:44:10.313 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2023-10-16T09:44:10.317 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:10.317 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T09:44:10.317 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:44:10.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.475+0000 7f5f738bc700 1 Processor -- start 2023-10-16T09:44:10.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.475+0000 7f5f738bc700 1 -- start start 2023-10-16T09:44:10.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f738bc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f6c15dfd0 0x7f5f6c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f738bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f6c0cd470 0x7f5f6c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f738bc700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6c06f870 con 0x7f5f6c0af450 2023-10-16T09:44:10.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f738bc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c0cde40 con 0x7f5f6c0cd470 2023-10-16T09:44:10.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f738bc700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c0d1160 con 0x7f5f6c15dfd0 2023-10-16T09:44:10.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f71e59700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f6c0cd470 0x7f5f6c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f71e59700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f6c0cd470 0x7f5f6c0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36488/0 (socket says 172.21.15.181:36488) 2023-10-16T09:44:10.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.476+0000 7f5f71e59700 1 -- 172.21.15.181:0/173819063 learned_addr learned my addr 172.21.15.181:0/173819063 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:10.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f71658700 1 --2- 172.21.15.181:0/173819063 >> v2:172.21.15.190:3300/0 conn(0x7f5f6c15dfd0 0x7f5f6c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f71e59700 1 -- 172.21.15.181:0/173819063 >> v2:172.21.15.190:3300/0 conn(0x7f5f6c15dfd0 msgr2=0x7f5f6c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:10.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f71e59700 1 --2- 172.21.15.181:0/173819063 >> v2:172.21.15.190:3300/0 conn(0x7f5f6c15dfd0 0x7f5f6c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f71e59700 1 -- 172.21.15.181:0/173819063 >> v1:172.21.15.190:6789/0 conn(0x7f5f6c0af450 legacy=0x7f5f6c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f71e59700 1 -- 172.21.15.181:0/173819063 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c0cc8e0 con 0x7f5f6c0cd470 2023-10-16T09:44:10.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f71e59700 1 --2- 172.21.15.181:0/173819063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0cd470 0x7f5f6c0cd900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c070340 tx=0x7f5f6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52aed7a35ee45b2c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:10.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f63fff700 1 -- 172.21.15.181:0/173819063 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400e070 con 0x7f5f6c0cd470 2023-10-16T09:44:10.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f63fff700 1 -- 172.21.15.181:0/173819063 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6400b9c0 con 0x7f5f6c0cd470 2023-10-16T09:44:10.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f63fff700 1 -- 172.21.15.181:0/173819063 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400bd70 con 0x7f5f6c0cd470 2023-10-16T09:44:10.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f738bc700 1 -- 172.21.15.181:0/173819063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0cd470 msgr2=0x7f5f6c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.477+0000 7f5f738bc700 1 --2- 172.21.15.181:0/173819063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0cd470 0x7f5f6c0cd900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c070340 tx=0x7f5f6400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:10.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.478+0000 7f5f738bc700 1 -- 172.21.15.181:0/173819063 shutdown_connections 2023-10-16T09:44:10.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.478+0000 7f5f738bc700 1 --2- 172.21.15.181:0/173819063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0cd470 0x7f5f6c0cd900 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.478+0000 7f5f738bc700 1 --2- 172.21.15.181:0/173819063 >> v2:172.21.15.190:3300/0 conn(0x7f5f6c15dfd0 0x7f5f6c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.478+0000 7f5f738bc700 1 -- 172.21.15.181:0/173819063 >> 172.21.15.181:0/173819063 conn(0x7f5f6c0d04a0 msgr2=0x7f5f6c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.478+0000 7f5f738bc700 1 -- 172.21.15.181:0/173819063 shutdown_connections 2023-10-16T09:44:10.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.478+0000 7f5f738bc700 1 -- 172.21.15.181:0/173819063 wait complete. 2023-10-16T09:44:10.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.479+0000 7f5f738bc700 1 Processor -- start 2023-10-16T09:44:10.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.479+0000 7f5f738bc700 1 -- start start 2023-10-16T09:44:10.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f738bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0af450 0x7f5f6c1f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f738bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f6c0cd470 0x7f5f6c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f738bc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f6c15dfd0 0x7f5f6c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f738bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1f2d40 con 0x7f5f6c0af450 2023-10-16T09:44:10.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f738bc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0aff00 con 0x7f5f6c15dfd0 2023-10-16T09:44:10.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f738bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0b07a0 con 0x7f5f6c0cd470 2023-10-16T09:44:10.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71658700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0af450 0x7f5f6c1f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71658700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0af450 0x7f5f6c1f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36490/0 (socket says 172.21.15.181:36490) 2023-10-16T09:44:10.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71658700 1 -- 172.21.15.181:0/3271217333 learned_addr learned my addr 172.21.15.181:0/3271217333 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:10.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71e59700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f6c15dfd0 0x7f5f6c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f70e57700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f6c0cd470 0x7f5f6c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71658700 1 -- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f6c0cd470 msgr2=0x7f5f6c0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71658700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f6c0cd470 0x7f5f6c0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71658700 1 -- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f6c15dfd0 msgr2=0x7f5f6c0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.480+0000 7f5f71658700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f6c15dfd0 0x7f5f6c0c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f71658700 1 -- 172.21.15.181:0/3271217333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009b30 con 0x7f5f6c0af450 2023-10-16T09:44:10.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f70e57700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f6c0cd470 0x7f5f6c0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:10.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f71658700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0af450 0x7f5f6c1f2800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e760 tx=0x7f5f5c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:10.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f61ffb700 1 -- 172.21.15.181:0/3271217333 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c005c20 con 0x7f5f6c0af450 2023-10-16T09:44:10.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f61ffb700 1 -- 172.21.15.181:0/3271217333 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c005da0 con 0x7f5f6c0af450 2023-10-16T09:44:10.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c15d3b0 con 0x7f5f6c0af450 2023-10-16T09:44:10.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f61ffb700 1 -- 172.21.15.181:0/3271217333 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c006260 con 0x7f5f6c0af450 2023-10-16T09:44:10.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.481+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c0b0430 con 0x7f5f6c0af450 2023-10-16T09:44:10.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.482+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6c067d20 con 0x7f5f6c0af450 2023-10-16T09:44:10.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.482+0000 7f5f61ffb700 1 -- 172.21.15.181:0/3271217333 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f5c0063c0 con 0x7f5f6c0af450 2023-10-16T09:44:10.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.482+0000 7f5f61ffb700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5f5006aed0 0x7f5f5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.483+0000 7f5f61ffb700 1 -- 172.21.15.181:0/3271217333 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f5c088df0 con 0x7f5f6c0af450 2023-10-16T09:44:10.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.483+0000 7f5f70e57700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5f5006aed0 0x7f5f5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.483+0000 7f5f70e57700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5f5006aed0 0x7f5f5006d390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c1f6300 tx=0x7f5f68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:10.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.487+0000 7f5f61ffb700 1 -- 172.21.15.181:0/3271217333 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5f5c053f30 con 0x7f5f6c0af450 2023-10-16T09:44:10.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.676+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f6c0ccf50 con 0x7f5f5006aed0 2023-10-16T09:44:10.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.683+0000 7f5f61ffb700 1 -- 172.21.15.181:0/3271217333 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264585 (secure 0 0 0) 0x7f5f68001450 con 0x7f5f5006aed0 2023-10-16T09:44:10.687 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:10.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5f5006aed0 msgr2=0x7f5f5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5f5006aed0 0x7f5f5006d390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c1f6300 tx=0x7f5f68009040 comp rx=0 tx=0).stop 2023-10-16T09:44:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0af450 msgr2=0x7f5f6c1f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0af450 0x7f5f6c1f2800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e760 tx=0x7f5f5c00ea70 comp rx=0 tx=0).stop 2023-10-16T09:44:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 shutdown_connections 2023-10-16T09:44:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5f5006aed0 0x7f5f5006d390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f6c15dfd0 0x7f5f6c0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0af450 0x7f5f6c1f2800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 --2- 172.21.15.181:0/3271217333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f6c0cd470 0x7f5f6c0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 >> 172.21.15.181:0/3271217333 conn(0x7f5f6c0d04a0 msgr2=0x7f5f6c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 shutdown_connections 2023-10-16T09:44:10.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.686+0000 7f5f738bc700 1 -- 172.21.15.181:0/3271217333 wait complete. 2023-10-16T09:44:10.692 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:44:10.699 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T09:44:09.679360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219176,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755544,"statfs":{"total":773094113280,"available":772869677056,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002065"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793938+0000","last_change":"2023-10-16T09:44:01.820375+0000","last_active":"2023-10-16T09:44:02.793938+0000","last_peered":"2023-10-16T09:44:02.793938+0000","last_clean":"2023-10-16T09:44:02.793938+0000","last_became_active":"2023-10-16T09:44:01.819619+0000","last_became_peered":"2023-10-16T09:44:01.819619+0000","last_unstale":"2023-10-16T09:44:02.793938+0000","last_undegraded":"2023-10-16T09:44:02.793938+0000","last_fullsized":"2023-10-16T09:44:02.793938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792204+0000","last_change":"2023-10-16T09:44:01.800363+0000","last_active":"2023-10-16T09:44:02.792204+0000","last_peered":"2023-10-16T09:44:02.792204+0000","last_clean":"2023-10-16T09:44:02.792204+0000","last_became_active":"2023-10-16T09:44:01.800037+0000","last_became_peered":"2023-10-16T09:44:01.800037+0000","last_unstale":"2023-10-16T09:44:02.792204+0000","last_undegraded":"2023-10-16T09:44:02.792204+0000","last_fullsized":"2023-10-16T09:44:02.792204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896175+0000","last_change":"2023-10-16T09:44:01.806686+0000","last_active":"2023-10-16T09:44:02.896175+0000","last_peered":"2023-10-16T09:44:02.896175+0000","last_clean":"2023-10-16T09:44:02.896175+0000","last_became_active":"2023-10-16T09:44:01.806361+0000","last_became_peered":"2023-10-16T09:44:01.806361+0000","last_unstale":"2023-10-16T09:44:02.896175+0000","last_undegraded":"2023-10-16T09:44:02.896175+0000","last_fullsized":"2023-10-16T09:44:02.896175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793996+0000","last_change":"2023-10-16T09:44:01.802398+0000","last_active":"2023-10-16T09:44:02.793996+0000","last_peered":"2023-10-16T09:44:02.793996+0000","last_clean":"2023-10-16T09:44:02.793996+0000","last_became_active":"2023-10-16T09:44:01.802077+0000","last_became_peered":"2023-10-16T09:44:01.802077+0000","last_unstale":"2023-10-16T09:44:02.793996+0000","last_undegraded":"2023-10-16T09:44:02.793996+0000","last_fullsized":"2023-10-16T09:44:02.793996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793271+0000","last_change":"2023-10-16T09:44:01.817219+0000","last_active":"2023-10-16T09:44:02.793271+0000","last_peered":"2023-10-16T09:44:02.793271+0000","last_clean":"2023-10-16T09:44:02.793271+0000","last_became_active":"2023-10-16T09:44:01.816945+0000","last_became_peered":"2023-10-16T09:44:01.816945+0000","last_unstale":"2023-10-16T09:44:02.793271+0000","last_undegraded":"2023-10-16T09:44:02.793271+0000","last_fullsized":"2023-10-16T09:44:02.793271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794026+0000","last_change":"2023-10-16T09:44:01.807539+0000","last_active":"2023-10-16T09:44:02.794026+0000","last_peered":"2023-10-16T09:44:02.794026+0000","last_clean":"2023-10-16T09:44:02.794026+0000","last_became_active":"2023-10-16T09:44:01.807282+0000","last_became_peered":"2023-10-16T09:44:01.807282+0000","last_unstale":"2023-10-16T09:44:02.794026+0000","last_undegraded":"2023-10-16T09:44:02.794026+0000","last_fullsized":"2023-10-16T09:44:02.794026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792808+0000","last_change":"2023-10-16T09:44:01.816493+0000","last_active":"2023-10-16T09:44:02.792808+0000","last_peered":"2023-10-16T09:44:02.792808+0000","last_clean":"2023-10-16T09:44:02.792808+0000","last_became_active":"2023-10-16T09:44:01.816006+0000","last_became_peered":"2023-10-16T09:44:01.816006+0000","last_unstale":"2023-10-16T09:44:02.792808+0000","last_undegraded":"2023-10-16T09:44:02.792808+0000","last_fullsized":"2023-10-16T09:44:02.792808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045389+0000","last_change":"2023-10-16T09:44:01.802910+0000","last_active":"2023-10-16T09:44:03.045389+0000","last_peered":"2023-10-16T09:44:03.045389+0000","last_clean":"2023-10-16T09:44:03.045389+0000","last_became_active":"2023-10-16T09:44:01.802640+0000","last_became_peered":"2023-10-16T09:44:01.802640+0000","last_unstale":"2023-10-16T09:44:03.045389+0000","last_undegraded":"2023-10-16T09:44:03.045389+0000","last_fullsized":"2023-10-16T09:44:03.045389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793935+0000","last_change":"2023-10-16T09:44:01.818818+0000","last_active":"2023-10-16T09:44:02.793935+0000","last_peered":"2023-10-16T09:44:02.793935+0000","last_clean":"2023-10-16T09:44:02.793935+0000","last_became_active":"2023-10-16T09:44:01.818532+0000","last_became_peered":"2023-10-16T09:44:01.818532+0000","last_unstale":"2023-10-16T09:44:02.793935+0000","last_undegraded":"2023-10-16T09:44:02.793935+0000","last_fullsized":"2023-10-16T09:44:02.793935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046230+0000","last_change":"2023-10-16T09:44:01.808945+0000","last_active":"2023-10-16T09:44:03.046230+0000","last_peered":"2023-10-16T09:44:03.046230+0000","last_clean":"2023-10-16T09:44:03.046230+0000","last_became_active":"2023-10-16T09:44:01.808655+0000","last_became_peered":"2023-10-16T09:44:01.808655+0000","last_unstale":"2023-10-16T09:44:03.046230+0000","last_undegraded":"2023-10-16T09:44:03.046230+0000","last_fullsized":"2023-10-16T09:44:03.046230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045896+0000","last_change":"2023-10-16T09:44:01.808213+0000","last_active":"2023-10-16T09:44:03.045896+0000","last_peered":"2023-10-16T09:44:03.045896+0000","last_clean":"2023-10-16T09:44:03.045896+0000","last_became_active":"2023-10-16T09:44:01.807715+0000","last_became_peered":"2023-10-16T09:44:01.807715+0000","last_unstale":"2023-10-16T09:44:03.045896+0000","last_undegraded":"2023-10-16T09:44:03.045896+0000","last_fullsized":"2023-10-16T09:44:03.045896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046320+0000","last_change":"2023-10-16T09:44:01.801296+0000","last_active":"2023-10-16T09:44:03.046320+0000","last_peered":"2023-10-16T09:44:03.046320+0000","last_clean":"2023-10-16T09:44:03.046320+0000","last_became_active":"2023-10-16T09:44:01.800456+0000","last_became_peered":"2023-10-16T09:44:01.800456+0000","last_unstale":"2023-10-16T09:44:03.046320+0000","last_undegraded":"2023-10-16T09:44:03.046320+0000","last_fullsized":"2023-10-16T09:44:03.046320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:56.560951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045786+0000","last_change":"2023-10-16T09:44:01.800574+0000","last_active":"2023-10-16T09:44:03.045786+0000","last_peered":"2023-10-16T09:44:03.045786+0000","last_clean":"2023-10-16T09:44:03.045786+0000","last_became_active":"2023-10-16T09:44:01.800299+0000","last_became_peered":"2023-10-16T09:44:01.800299+0000","last_unstale":"2023-10-16T09:44:03.045786+0000","last_undegraded":"2023-10-16T09:44:03.045786+0000","last_fullsized":"2023-10-16T09:44:03.045786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895941+0000","last_change":"2023-10-16T09:44:01.797679+0000","last_active":"2023-10-16T09:44:02.895941+0000","last_peered":"2023-10-16T09:44:02.895941+0000","last_clean":"2023-10-16T09:44:02.895941+0000","last_became_active":"2023-10-16T09:44:01.797420+0000","last_became_peered":"2023-10-16T09:44:01.797420+0000","last_unstale":"2023-10-16T09:44:02.895941+0000","last_undegraded":"2023-10-16T09:44:02.895941+0000","last_fullsized":"2023-10-16T09:44:02.895941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791819+0000","last_change":"2023-10-16T09:44:01.801979+0000","last_active":"2023-10-16T09:44:02.791819+0000","last_peered":"2023-10-16T09:44:02.791819+0000","last_clean":"2023-10-16T09:44:02.791819+0000","last_became_active":"2023-10-16T09:44:01.801663+0000","last_became_peered":"2023-10-16T09:44:01.801663+0000","last_unstale":"2023-10-16T09:44:02.791819+0000","last_undegraded":"2023-10-16T09:44:02.791819+0000","last_fullsized":"2023-10-16T09:44:02.791819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793275+0000","last_change":"2023-10-16T09:44:01.808377+0000","last_active":"2023-10-16T09:44:02.793275+0000","last_peered":"2023-10-16T09:44:02.793275+0000","last_clean":"2023-10-16T09:44:02.793275+0000","last_became_active":"2023-10-16T09:44:01.808091+0000","last_became_peered":"2023-10-16T09:44:01.808091+0000","last_unstale":"2023-10-16T09:44:02.793275+0000","last_undegraded":"2023-10-16T09:44:02.793275+0000","last_fullsized":"2023-10-16T09:44:02.793275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794670+0000","last_change":"2023-10-16T09:44:01.815509+0000","last_active":"2023-10-16T09:44:02.794670+0000","last_peered":"2023-10-16T09:44:02.794670+0000","last_clean":"2023-10-16T09:44:02.794670+0000","last_became_active":"2023-10-16T09:44:01.815290+0000","last_became_peered":"2023-10-16T09:44:01.815290+0000","last_unstale":"2023-10-16T09:44:02.794670+0000","last_undegraded":"2023-10-16T09:44:02.794670+0000","last_fullsized":"2023-10-16T09:44:02.794670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045622+0000","last_change":"2023-10-16T09:44:01.804696+0000","last_active":"2023-10-16T09:44:03.045622+0000","last_peered":"2023-10-16T09:44:03.045622+0000","last_clean":"2023-10-16T09:44:03.045622+0000","last_became_active":"2023-10-16T09:44:01.804427+0000","last_became_peered":"2023-10-16T09:44:01.804427+0000","last_unstale":"2023-10-16T09:44:03.045622+0000","last_undegraded":"2023-10-16T09:44:03.045622+0000","last_fullsized":"2023-10-16T09:44:03.045622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:06.338018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794299+0000","last_change":"2023-10-16T09:44:01.815090+0000","last_active":"2023-10-16T09:44:02.794299+0000","last_peered":"2023-10-16T09:44:02.794299+0000","last_clean":"2023-10-16T09:44:02.794299+0000","last_became_active":"2023-10-16T09:44:01.814787+0000","last_became_peered":"2023-10-16T09:44:01.814787+0000","last_unstale":"2023-10-16T09:44:02.794299+0000","last_undegraded":"2023-10-16T09:44:02.794299+0000","last_fullsized":"2023-10-16T09:44:02.794299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.820250+0000","last_change":"2023-10-16T09:44:00.797577+0000","last_active":"2023-10-16T09:44:01.820250+0000","last_peered":"2023-10-16T09:44:01.820250+0000","last_clean":"2023-10-16T09:44:01.820250+0000","last_became_active":"2023-10-16T09:44:00.797323+0000","last_became_peered":"2023-10-16T09:44:00.797323+0000","last_unstale":"2023-10-16T09:44:01.820250+0000","last_undegraded":"2023-10-16T09:44:01.820250+0000","last_fullsized":"2023-10-16T09:44:01.820250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046254+0000","last_change":"2023-10-16T09:44:01.801503+0000","last_active":"2023-10-16T09:44:03.046254+0000","last_peered":"2023-10-16T09:44:03.046254+0000","last_clean":"2023-10-16T09:44:03.046254+0000","last_became_active":"2023-10-16T09:44:01.801078+0000","last_became_peered":"2023-10-16T09:44:01.801078+0000","last_unstale":"2023-10-16T09:44:03.046254+0000","last_undegraded":"2023-10-16T09:44:03.046254+0000","last_fullsized":"2023-10-16T09:44:03.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819479+0000","last_change":"2023-10-16T09:44:00.790199+0000","last_active":"2023-10-16T09:44:01.819479+0000","last_peered":"2023-10-16T09:44:01.819479+0000","last_clean":"2023-10-16T09:44:01.819479+0000","last_became_active":"2023-10-16T09:44:00.789914+0000","last_became_peered":"2023-10-16T09:44:00.789914+0000","last_unstale":"2023-10-16T09:44:01.819479+0000","last_undegraded":"2023-10-16T09:44:01.819479+0000","last_fullsized":"2023-10-16T09:44:01.819479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896280+0000","last_change":"2023-10-16T09:44:01.808873+0000","last_active":"2023-10-16T09:44:02.896280+0000","last_peered":"2023-10-16T09:44:02.896280+0000","last_clean":"2023-10-16T09:44:02.896280+0000","last_became_active":"2023-10-16T09:44:01.808557+0000","last_became_peered":"2023-10-16T09:44:01.808557+0000","last_unstale":"2023-10-16T09:44:02.896280+0000","last_undegraded":"2023-10-16T09:44:02.896280+0000","last_fullsized":"2023-10-16T09:44:02.896280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794605+0000","last_change":"2023-10-16T09:44:00.801288+0000","last_active":"2023-10-16T09:44:02.794605+0000","last_peered":"2023-10-16T09:44:02.794605+0000","last_clean":"2023-10-16T09:44:02.794605+0000","last_became_active":"2023-10-16T09:44:00.801008+0000","last_became_peered":"2023-10-16T09:44:00.801008+0000","last_unstale":"2023-10-16T09:44:02.794605+0000","last_undegraded":"2023-10-16T09:44:02.794605+0000","last_fullsized":"2023-10-16T09:44:02.794605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793760+0000","last_change":"2023-10-16T09:44:01.803802+0000","last_active":"2023-10-16T09:44:02.793760+0000","last_peered":"2023-10-16T09:44:02.793760+0000","last_clean":"2023-10-16T09:44:02.793760+0000","last_became_active":"2023-10-16T09:44:01.803480+0000","last_became_peered":"2023-10-16T09:44:01.803480+0000","last_unstale":"2023-10-16T09:44:02.793760+0000","last_undegraded":"2023-10-16T09:44:02.793760+0000","last_fullsized":"2023-10-16T09:44:02.793760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793452+0000","last_change":"2023-10-16T09:44:00.798160+0000","last_active":"2023-10-16T09:44:02.793452+0000","last_peered":"2023-10-16T09:44:02.793452+0000","last_clean":"2023-10-16T09:44:02.793452+0000","last_became_active":"2023-10-16T09:44:00.797833+0000","last_became_peered":"2023-10-16T09:44:00.797833+0000","last_unstale":"2023-10-16T09:44:02.793452+0000","last_undegraded":"2023-10-16T09:44:02.793452+0000","last_fullsized":"2023-10-16T09:44:02.793452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794244+0000","last_change":"2023-10-16T09:44:01.802107+0000","last_active":"2023-10-16T09:44:02.794244+0000","last_peered":"2023-10-16T09:44:02.794244+0000","last_clean":"2023-10-16T09:44:02.794244+0000","last_became_active":"2023-10-16T09:44:01.801774+0000","last_became_peered":"2023-10-16T09:44:01.801774+0000","last_unstale":"2023-10-16T09:44:02.794244+0000","last_undegraded":"2023-10-16T09:44:02.794244+0000","last_fullsized":"2023-10-16T09:44:02.794244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822531+0000","last_change":"2023-10-16T09:44:00.785046+0000","last_active":"2023-10-16T09:44:01.822531+0000","last_peered":"2023-10-16T09:44:01.822531+0000","last_clean":"2023-10-16T09:44:01.822531+0000","last_became_active":"2023-10-16T09:44:00.784767+0000","last_became_peered":"2023-10-16T09:44:00.784767+0000","last_unstale":"2023-10-16T09:44:01.822531+0000","last_undegraded":"2023-10-16T09:44:01.822531+0000","last_fullsized":"2023-10-16T09:44:01.822531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045674+0000","last_change":"2023-10-16T09:44:01.799372+0000","last_active":"2023-10-16T09:44:03.045674+0000","last_peered":"2023-10-16T09:44:03.045674+0000","last_clean":"2023-10-16T09:44:03.045674+0000","last_became_active":"2023-10-16T09:44:01.799056+0000","last_became_peered":"2023-10-16T09:44:01.799056+0000","last_unstale":"2023-10-16T09:44:03.045674+0000","last_undegraded":"2023-10-16T09:44:03.045674+0000","last_fullsized":"2023-10-16T09:44:03.045674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:44:46.117646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045848+0000","last_change":"2023-10-16T09:44:00.800927+0000","last_active":"2023-10-16T09:44:03.045848+0000","last_peered":"2023-10-16T09:44:03.045848+0000","last_clean":"2023-10-16T09:44:03.045848+0000","last_became_active":"2023-10-16T09:44:00.800533+0000","last_became_peered":"2023-10-16T09:44:00.800533+0000","last_unstale":"2023-10-16T09:44:03.045848+0000","last_undegraded":"2023-10-16T09:44:03.045848+0000","last_fullsized":"2023-10-16T09:44:03.045848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896238+0000","last_change":"2023-10-16T09:44:01.795883+0000","last_active":"2023-10-16T09:44:02.896238+0000","last_peered":"2023-10-16T09:44:02.896238+0000","last_clean":"2023-10-16T09:44:02.896238+0000","last_became_active":"2023-10-16T09:44:01.795613+0000","last_became_peered":"2023-10-16T09:44:01.795613+0000","last_unstale":"2023-10-16T09:44:02.896238+0000","last_undegraded":"2023-10-16T09:44:02.896238+0000","last_fullsized":"2023-10-16T09:44:02.896238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792936+0000","last_change":"2023-10-16T09:44:00.800124+0000","last_active":"2023-10-16T09:44:01.792936+0000","last_peered":"2023-10-16T09:44:01.792936+0000","last_clean":"2023-10-16T09:44:01.792936+0000","last_became_active":"2023-10-16T09:44:00.799861+0000","last_became_peered":"2023-10-16T09:44:00.799861+0000","last_unstale":"2023-10-16T09:44:01.792936+0000","last_undegraded":"2023-10-16T09:44:01.792936+0000","last_fullsized":"2023-10-16T09:44:01.792936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045513+0000","last_change":"2023-10-16T09:44:01.799159+0000","last_active":"2023-10-16T09:44:03.045513+0000","last_peered":"2023-10-16T09:44:03.045513+0000","last_clean":"2023-10-16T09:44:03.045513+0000","last_became_active":"2023-10-16T09:44:01.798882+0000","last_became_peered":"2023-10-16T09:44:01.798882+0000","last_unstale":"2023-10-16T09:44:03.045513+0000","last_undegraded":"2023-10-16T09:44:03.045513+0000","last_fullsized":"2023-10-16T09:44:03.045513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798407+0000","last_change":"2023-10-16T09:44:00.797328+0000","last_active":"2023-10-16T09:44:01.798407+0000","last_peered":"2023-10-16T09:44:01.798407+0000","last_clean":"2023-10-16T09:44:01.798407+0000","last_became_active":"2023-10-16T09:44:00.797038+0000","last_became_peered":"2023-10-16T09:44:00.797038+0000","last_unstale":"2023-10-16T09:44:01.798407+0000","last_undegraded":"2023-10-16T09:44:01.798407+0000","last_fullsized":"2023-10-16T09:44:01.798407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:18.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791948+0000","last_change":"2023-10-16T09:44:01.797648+0000","last_active":"2023-10-16T09:44:02.791948+0000","last_peered":"2023-10-16T09:44:02.791948+0000","last_clean":"2023-10-16T09:44:02.791948+0000","last_became_active":"2023-10-16T09:44:01.797374+0000","last_became_peered":"2023-10-16T09:44:01.797374+0000","last_unstale":"2023-10-16T09:44:02.791948+0000","last_undegraded":"2023-10-16T09:44:02.791948+0000","last_fullsized":"2023-10-16T09:44:02.791948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046347+0000","last_change":"2023-10-16T09:44:00.796911+0000","last_active":"2023-10-16T09:44:03.046347+0000","last_peered":"2023-10-16T09:44:03.046347+0000","last_clean":"2023-10-16T09:44:03.046347+0000","last_became_active":"2023-10-16T09:44:00.796666+0000","last_became_peered":"2023-10-16T09:44:00.796666+0000","last_unstale":"2023-10-16T09:44:03.046347+0000","last_undegraded":"2023-10-16T09:44:03.046347+0000","last_fullsized":"2023-10-16T09:44:03.046347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794228+0000","last_change":"2023-10-16T09:44:01.815668+0000","last_active":"2023-10-16T09:44:02.794228+0000","last_peered":"2023-10-16T09:44:02.794228+0000","last_clean":"2023-10-16T09:44:02.794228+0000","last_became_active":"2023-10-16T09:44:01.815327+0000","last_became_peered":"2023-10-16T09:44:01.815327+0000","last_unstale":"2023-10-16T09:44:02.794228+0000","last_undegraded":"2023-10-16T09:44:02.794228+0000","last_fullsized":"2023-10-16T09:44:02.794228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798169+0000","last_change":"2023-10-16T09:44:00.800951+0000","last_active":"2023-10-16T09:44:01.798169+0000","last_peered":"2023-10-16T09:44:01.798169+0000","last_clean":"2023-10-16T09:44:01.798169+0000","last_became_active":"2023-10-16T09:44:00.800685+0000","last_became_peered":"2023-10-16T09:44:00.800685+0000","last_unstale":"2023-10-16T09:44:01.798169+0000","last_undegraded":"2023-10-16T09:44:01.798169+0000","last_fullsized":"2023-10-16T09:44:01.798169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896480+0000","last_change":"2023-10-16T09:44:01.808665+0000","last_active":"2023-10-16T09:44:02.896480+0000","last_peered":"2023-10-16T09:44:02.896480+0000","last_clean":"2023-10-16T09:44:02.896480+0000","last_became_active":"2023-10-16T09:44:01.808229+0000","last_became_peered":"2023-10-16T09:44:01.808229+0000","last_unstale":"2023-10-16T09:44:02.896480+0000","last_undegraded":"2023-10-16T09:44:02.896480+0000","last_fullsized":"2023-10-16T09:44:02.896480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.824237+0000","last_change":"2023-10-16T09:44:00.797832+0000","last_active":"2023-10-16T09:44:01.824237+0000","last_peered":"2023-10-16T09:44:01.824237+0000","last_clean":"2023-10-16T09:44:01.824237+0000","last_became_active":"2023-10-16T09:44:00.797569+0000","last_became_peered":"2023-10-16T09:44:00.797569+0000","last_unstale":"2023-10-16T09:44:01.824237+0000","last_undegraded":"2023-10-16T09:44:01.824237+0000","last_fullsized":"2023-10-16T09:44:01.824237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046342+0000","last_change":"2023-10-16T09:44:01.820469+0000","last_active":"2023-10-16T09:44:03.046342+0000","last_peered":"2023-10-16T09:44:03.046342+0000","last_clean":"2023-10-16T09:44:03.046342+0000","last_became_active":"2023-10-16T09:44:01.820207+0000","last_became_peered":"2023-10-16T09:44:01.820207+0000","last_unstale":"2023-10-16T09:44:03.046342+0000","last_undegraded":"2023-10-16T09:44:03.046342+0000","last_fullsized":"2023-10-16T09:44:03.046342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:16.100248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045792+0000","last_change":"2023-10-16T09:44:00.794450+0000","last_active":"2023-10-16T09:44:03.045792+0000","last_peered":"2023-10-16T09:44:03.045792+0000","last_clean":"2023-10-16T09:44:03.045792+0000","last_became_active":"2023-10-16T09:44:00.794197+0000","last_became_peered":"2023-10-16T09:44:00.794197+0000","last_unstale":"2023-10-16T09:44:03.045792+0000","last_undegraded":"2023-10-16T09:44:03.045792+0000","last_fullsized":"2023-10-16T09:44:03.045792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794469+0000","last_change":"2023-10-16T09:44:01.817900+0000","last_active":"2023-10-16T09:44:02.794469+0000","last_peered":"2023-10-16T09:44:02.794469+0000","last_clean":"2023-10-16T09:44:02.794469+0000","last_became_active":"2023-10-16T09:44:01.817666+0000","last_became_peered":"2023-10-16T09:44:01.817666+0000","last_unstale":"2023-10-16T09:44:02.794469+0000","last_undegraded":"2023-10-16T09:44:02.794469+0000","last_fullsized":"2023-10-16T09:44:02.794469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822707+0000","last_change":"2023-10-16T09:44:00.794006+0000","last_active":"2023-10-16T09:44:01.822707+0000","last_peered":"2023-10-16T09:44:01.822707+0000","last_clean":"2023-10-16T09:44:01.822707+0000","last_became_active":"2023-10-16T09:44:00.793703+0000","last_became_peered":"2023-10-16T09:44:00.793703+0000","last_unstale":"2023-10-16T09:44:01.822707+0000","last_undegraded":"2023-10-16T09:44:01.822707+0000","last_fullsized":"2023-10-16T09:44:01.822707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895868+0000","last_change":"2023-10-16T09:44:01.815023+0000","last_active":"2023-10-16T09:44:02.895868+0000","last_peered":"2023-10-16T09:44:02.895868+0000","last_clean":"2023-10-16T09:44:02.895868+0000","last_became_active":"2023-10-16T09:44:01.814737+0000","last_became_peered":"2023-10-16T09:44:01.814737+0000","last_unstale":"2023-10-16T09:44:02.895868+0000","last_undegraded":"2023-10-16T09:44:02.895868+0000","last_fullsized":"2023-10-16T09:44:02.895868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:59.336084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.799063+0000","last_change":"2023-10-16T09:44:00.792205+0000","last_active":"2023-10-16T09:44:01.799063+0000","last_peered":"2023-10-16T09:44:01.799063+0000","last_clean":"2023-10-16T09:44:01.799063+0000","last_became_active":"2023-10-16T09:44:00.791860+0000","last_became_peered":"2023-10-16T09:44:00.791860+0000","last_unstale":"2023-10-16T09:44:01.799063+0000","last_undegraded":"2023-10-16T09:44:01.799063+0000","last_fullsized":"2023-10-16T09:44:01.799063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:45.078590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793427+0000","last_change":"2023-10-16T09:44:01.820397+0000","last_active":"2023-10-16T09:44:02.793427+0000","last_peered":"2023-10-16T09:44:02.793427+0000","last_clean":"2023-10-16T09:44:02.793427+0000","last_became_active":"2023-10-16T09:44:01.820151+0000","last_became_peered":"2023-10-16T09:44:01.820151+0000","last_unstale":"2023-10-16T09:44:02.793427+0000","last_undegraded":"2023-10-16T09:44:02.793427+0000","last_fullsized":"2023-10-16T09:44:02.793427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045738+0000","last_change":"2023-10-16T09:44:00.799178+0000","last_active":"2023-10-16T09:44:03.045738+0000","last_peered":"2023-10-16T09:44:03.045738+0000","last_clean":"2023-10-16T09:44:03.045738+0000","last_became_active":"2023-10-16T09:44:00.798866+0000","last_became_peered":"2023-10-16T09:44:00.798866+0000","last_unstale":"2023-10-16T09:44:03.045738+0000","last_undegraded":"2023-10-16T09:44:03.045738+0000","last_fullsized":"2023-10-16T09:44:03.045738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045651+0000","last_change":"2023-10-16T09:44:01.805085+0000","last_active":"2023-10-16T09:44:03.045651+0000","last_peered":"2023-10-16T09:44:03.045651+0000","last_clean":"2023-10-16T09:44:03.045651+0000","last_became_active":"2023-10-16T09:44:01.804493+0000","last_became_peered":"2023-10-16T09:44:01.804493+0000","last_unstale":"2023-10-16T09:44:03.045651+0000","last_undegraded":"2023-10-16T09:44:03.045651+0000","last_fullsized":"2023-10-16T09:44:03.045651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819839+0000","last_change":"2023-10-16T09:44:00.788023+0000","last_active":"2023-10-16T09:44:01.819839+0000","last_peered":"2023-10-16T09:44:01.819839+0000","last_clean":"2023-10-16T09:44:01.819839+0000","last_became_active":"2023-10-16T09:44:00.787751+0000","last_became_peered":"2023-10-16T09:44:00.787751+0000","last_unstale":"2023-10-16T09:44:01.819839+0000","last_undegraded":"2023-10-16T09:44:01.819839+0000","last_fullsized":"2023-10-16T09:44:01.819839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:25:10.084733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793056+0000","last_change":"2023-10-16T09:44:01.818990+0000","last_active":"2023-10-16T09:44:02.793056+0000","last_peered":"2023-10-16T09:44:02.793056+0000","last_clean":"2023-10-16T09:44:02.793056+0000","last_became_active":"2023-10-16T09:44:01.818711+0000","last_became_peered":"2023-10-16T09:44:01.818711+0000","last_unstale":"2023-10-16T09:44:02.793056+0000","last_undegraded":"2023-10-16T09:44:02.793056+0000","last_fullsized":"2023-10-16T09:44:02.793056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.817348+0000","last_change":"2023-10-16T09:44:00.788065+0000","last_active":"2023-10-16T09:44:01.817348+0000","last_peered":"2023-10-16T09:44:01.817348+0000","last_clean":"2023-10-16T09:44:01.817348+0000","last_became_active":"2023-10-16T09:44:00.787742+0000","last_became_peered":"2023-10-16T09:44:00.787742+0000","last_unstale":"2023-10-16T09:44:01.817348+0000","last_undegraded":"2023-10-16T09:44:01.817348+0000","last_fullsized":"2023-10-16T09:44:01.817348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:18.670482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794288+0000","last_change":"2023-10-16T09:44:01.821150+0000","last_active":"2023-10-16T09:44:02.794288+0000","last_peered":"2023-10-16T09:44:02.794288+0000","last_clean":"2023-10-16T09:44:02.794288+0000","last_became_active":"2023-10-16T09:44:01.820830+0000","last_became_peered":"2023-10-16T09:44:01.820830+0000","last_unstale":"2023-10-16T09:44:02.794288+0000","last_undegraded":"2023-10-16T09:44:02.794288+0000","last_fullsized":"2023-10-16T09:44:02.794288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792997+0000","last_change":"2023-10-16T09:44:00.790958+0000","last_active":"2023-10-16T09:44:01.792997+0000","last_peered":"2023-10-16T09:44:01.792997+0000","last_clean":"2023-10-16T09:44:01.792997+0000","last_became_active":"2023-10-16T09:44:00.790679+0000","last_became_peered":"2023-10-16T09:44:00.790679+0000","last_unstale":"2023-10-16T09:44:01.792997+0000","last_undegraded":"2023-10-16T09:44:01.792997+0000","last_fullsized":"2023-10-16T09:44:01.792997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:23.552983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794431+0000","last_change":"2023-10-16T09:44:01.801190+0000","last_active":"2023-10-16T09:44:02.794431+0000","last_peered":"2023-10-16T09:44:02.794431+0000","last_clean":"2023-10-16T09:44:02.794431+0000","last_became_active":"2023-10-16T09:44:01.800889+0000","last_became_peered":"2023-10-16T09:44:01.800889+0000","last_unstale":"2023-10-16T09:44:02.794431+0000","last_undegraded":"2023-10-16T09:44:02.794431+0000","last_fullsized":"2023-10-16T09:44:02.794431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798899+0000","last_change":"2023-10-16T09:44:00.792936+0000","last_active":"2023-10-16T09:44:01.798899+0000","last_peered":"2023-10-16T09:44:01.798899+0000","last_clean":"2023-10-16T09:44:01.798899+0000","last_became_active":"2023-10-16T09:44:00.792595+0000","last_became_peered":"2023-10-16T09:44:00.792595+0000","last_unstale":"2023-10-16T09:44:01.798899+0000","last_undegraded":"2023-10-16T09:44:01.798899+0000","last_fullsized":"2023-10-16T09:44:01.798899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:22.500292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045929+0000","last_change":"2023-10-16T09:44:01.800944+0000","last_active":"2023-10-16T09:44:03.045929+0000","last_peered":"2023-10-16T09:44:03.045929+0000","last_clean":"2023-10-16T09:44:03.045929+0000","last_became_active":"2023-10-16T09:44:01.800569+0000","last_became_peered":"2023-10-16T09:44:01.800569+0000","last_unstale":"2023-10-16T09:44:03.045929+0000","last_undegraded":"2023-10-16T09:44:03.045929+0000","last_fullsized":"2023-10-16T09:44:03.045929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:38:40.688657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791775+0000","last_change":"2023-10-16T09:44:01.805039+0000","last_active":"2023-10-16T09:44:02.791775+0000","last_peered":"2023-10-16T09:44:02.791775+0000","last_clean":"2023-10-16T09:44:02.791775+0000","last_became_active":"2023-10-16T09:44:01.804777+0000","last_became_peered":"2023-10-16T09:44:01.804777+0000","last_unstale":"2023-10-16T09:44:02.791775+0000","last_undegraded":"2023-10-16T09:44:02.791775+0000","last_fullsized":"2023-10-16T09:44:02.791775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800545+0000","last_change":"2023-10-16T09:44:00.790368+0000","last_active":"2023-10-16T09:44:01.800545+0000","last_peered":"2023-10-16T09:44:01.800545+0000","last_clean":"2023-10-16T09:44:01.800545+0000","last_became_active":"2023-10-16T09:44:00.790030+0000","last_became_peered":"2023-10-16T09:44:00.790030+0000","last_unstale":"2023-10-16T09:44:01.800545+0000","last_undegraded":"2023-10-16T09:44:01.800545+0000","last_fullsized":"2023-10-16T09:44:01.800545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:19.134455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045870+0000","last_change":"2023-10-16T09:44:01.802895+0000","last_active":"2023-10-16T09:44:03.045870+0000","last_peered":"2023-10-16T09:44:03.045870+0000","last_clean":"2023-10-16T09:44:03.045870+0000","last_became_active":"2023-10-16T09:44:01.802637+0000","last_became_peered":"2023-10-16T09:44:01.802637+0000","last_unstale":"2023-10-16T09:44:03.045870+0000","last_undegraded":"2023-10-16T09:44:03.045870+0000","last_fullsized":"2023-10-16T09:44:03.045870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798097+0000","last_change":"2023-10-16T09:44:00.793146+0000","last_active":"2023-10-16T09:44:01.798097+0000","last_peered":"2023-10-16T09:44:01.798097+0000","last_clean":"2023-10-16T09:44:01.798097+0000","last_became_active":"2023-10-16T09:44:00.792827+0000","last_became_peered":"2023-10-16T09:44:00.792827+0000","last_unstale":"2023-10-16T09:44:01.798097+0000","last_undegraded":"2023-10-16T09:44:01.798097+0000","last_fullsized":"2023-10-16T09:44:01.798097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.803000+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:01.803000+0000","last_peered":"2023-10-16T09:44:01.803000+0000","last_clean":"2023-10-16T09:44:01.803000+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:01.803000+0000","last_undegraded":"2023-10-16T09:44:01.803000+0000","last_fullsized":"2023-10-16T09:44:01.803000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895844+0000","last_change":"2023-10-16T09:44:01.795766+0000","last_active":"2023-10-16T09:44:02.895844+0000","last_peered":"2023-10-16T09:44:02.895844+0000","last_clean":"2023-10-16T09:44:02.895844+0000","last_became_active":"2023-10-16T09:44:01.795429+0000","last_became_peered":"2023-10-16T09:44:01.795429+0000","last_unstale":"2023-10-16T09:44:02.895844+0000","last_undegraded":"2023-10-16T09:44:02.895844+0000","last_fullsized":"2023-10-16T09:44:02.895844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:24.556381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792852+0000","last_change":"2023-10-16T09:44:00.800131+0000","last_active":"2023-10-16T09:44:02.792852+0000","last_peered":"2023-10-16T09:44:02.792852+0000","last_clean":"2023-10-16T09:44:02.792852+0000","last_became_active":"2023-10-16T09:44:00.799868+0000","last_became_peered":"2023-10-16T09:44:00.799868+0000","last_unstale":"2023-10-16T09:44:02.792852+0000","last_undegraded":"2023-10-16T09:44:02.792852+0000","last_fullsized":"2023-10-16T09:44:02.792852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794561+0000","last_change":"2023-10-16T09:44:00.799180+0000","last_active":"2023-10-16T09:44:02.794561+0000","last_peered":"2023-10-16T09:44:02.794561+0000","last_clean":"2023-10-16T09:44:02.794561+0000","last_became_active":"2023-10-16T09:44:00.798901+0000","last_became_peered":"2023-10-16T09:44:00.798901+0000","last_unstale":"2023-10-16T09:44:02.794561+0000","last_undegraded":"2023-10-16T09:44:02.794561+0000","last_fullsized":"2023-10-16T09:44:02.794561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792018+0000","last_change":"2023-10-16T09:44:01.804991+0000","last_active":"2023-10-16T09:44:02.792018+0000","last_peered":"2023-10-16T09:44:02.792018+0000","last_clean":"2023-10-16T09:44:02.792018+0000","last_became_active":"2023-10-16T09:44:01.804694+0000","last_became_peered":"2023-10-16T09:44:01.804694+0000","last_unstale":"2023-10-16T09:44:02.792018+0000","last_undegraded":"2023-10-16T09:44:02.792018+0000","last_fullsized":"2023-10-16T09:44:02.792018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792039+0000","last_change":"2023-10-16T09:44:00.800156+0000","last_active":"2023-10-16T09:44:02.792039+0000","last_peered":"2023-10-16T09:44:02.792039+0000","last_clean":"2023-10-16T09:44:02.792039+0000","last_became_active":"2023-10-16T09:44:00.799895+0000","last_became_peered":"2023-10-16T09:44:00.799895+0000","last_unstale":"2023-10-16T09:44:02.792039+0000","last_undegraded":"2023-10-16T09:44:02.792039+0000","last_fullsized":"2023-10-16T09:44:02.792039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046312+0000","last_change":"2023-10-16T09:44:01.819462+0000","last_active":"2023-10-16T09:44:03.046312+0000","last_peered":"2023-10-16T09:44:03.046312+0000","last_clean":"2023-10-16T09:44:03.046312+0000","last_became_active":"2023-10-16T09:44:01.819196+0000","last_became_peered":"2023-10-16T09:44:01.819196+0000","last_unstale":"2023-10-16T09:44:03.046312+0000","last_undegraded":"2023-10-16T09:44:03.046312+0000","last_fullsized":"2023-10-16T09:44:03.046312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046068+0000","last_change":"2023-10-16T09:44:00.801223+0000","last_active":"2023-10-16T09:44:03.046068+0000","last_peered":"2023-10-16T09:44:03.046068+0000","last_clean":"2023-10-16T09:44:03.046068+0000","last_became_active":"2023-10-16T09:44:00.800978+0000","last_became_peered":"2023-10-16T09:44:00.800978+0000","last_unstale":"2023-10-16T09:44:03.046068+0000","last_undegraded":"2023-10-16T09:44:03.046068+0000","last_fullsized":"2023-10-16T09:44:03.046068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896394+0000","last_change":"2023-10-16T09:44:01.798851+0000","last_active":"2023-10-16T09:44:02.896394+0000","last_peered":"2023-10-16T09:44:02.896394+0000","last_clean":"2023-10-16T09:44:02.896394+0000","last_became_active":"2023-10-16T09:44:01.798475+0000","last_became_peered":"2023-10-16T09:44:01.798475+0000","last_unstale":"2023-10-16T09:44:02.896394+0000","last_undegraded":"2023-10-16T09:44:02.896394+0000","last_fullsized":"2023-10-16T09:44:02.896394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792734+0000","last_change":"2023-10-16T09:44:00.799681+0000","last_active":"2023-10-16T09:44:02.792734+0000","last_peered":"2023-10-16T09:44:02.792734+0000","last_clean":"2023-10-16T09:44:02.792734+0000","last_became_active":"2023-10-16T09:44:00.799423+0000","last_became_peered":"2023-10-16T09:44:00.799423+0000","last_unstale":"2023-10-16T09:44:02.792734+0000","last_undegraded":"2023-10-16T09:44:02.792734+0000","last_fullsized":"2023-10-16T09:44:02.792734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046475+0000","last_change":"2023-10-16T09:44:01.806654+0000","last_active":"2023-10-16T09:44:03.046475+0000","last_peered":"2023-10-16T09:44:03.046475+0000","last_clean":"2023-10-16T09:44:03.046475+0000","last_became_active":"2023-10-16T09:44:01.805672+0000","last_became_peered":"2023-10-16T09:44:01.805672+0000","last_unstale":"2023-10-16T09:44:03.046475+0000","last_undegraded":"2023-10-16T09:44:03.046475+0000","last_fullsized":"2023-10-16T09:44:03.046475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:41.878512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790462+0000","last_change":"2023-10-16T09:44:00.797519+0000","last_active":"2023-10-16T09:44:01.790462+0000","last_peered":"2023-10-16T09:44:01.790462+0000","last_clean":"2023-10-16T09:44:01.790462+0000","last_became_active":"2023-10-16T09:44:00.797217+0000","last_became_peered":"2023-10-16T09:44:00.797217+0000","last_unstale":"2023-10-16T09:44:01.790462+0000","last_undegraded":"2023-10-16T09:44:01.790462+0000","last_fullsized":"2023-10-16T09:44:01.790462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045449+0000","last_change":"2023-10-16T09:44:01.798478+0000","last_active":"2023-10-16T09:44:03.045449+0000","last_peered":"2023-10-16T09:44:03.045449+0000","last_clean":"2023-10-16T09:44:03.045449+0000","last_became_active":"2023-10-16T09:44:01.798146+0000","last_became_peered":"2023-10-16T09:44:01.798146+0000","last_unstale":"2023-10-16T09:44:03.045449+0000","last_undegraded":"2023-10-16T09:44:03.045449+0000","last_fullsized":"2023-10-16T09:44:03.045449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790384+0000","last_change":"2023-10-16T09:44:00.791453+0000","last_active":"2023-10-16T09:44:01.790384+0000","last_peered":"2023-10-16T09:44:01.790384+0000","last_clean":"2023-10-16T09:44:01.790384+0000","last_became_active":"2023-10-16T09:44:00.791178+0000","last_became_peered":"2023-10-16T09:44:00.791178+0000","last_unstale":"2023-10-16T09:44:01.790384+0000","last_undegraded":"2023-10-16T09:44:01.790384+0000","last_fullsized":"2023-10-16T09:44:01.790384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046137+0000","last_change":"2023-10-16T09:44:01.801224+0000","last_active":"2023-10-16T09:44:03.046137+0000","last_peered":"2023-10-16T09:44:03.046137+0000","last_clean":"2023-10-16T09:44:03.046137+0000","last_became_active":"2023-10-16T09:44:01.800946+0000","last_became_peered":"2023-10-16T09:44:01.800946+0000","last_unstale":"2023-10-16T09:44:03.046137+0000","last_undegraded":"2023-10-16T09:44:03.046137+0000","last_fullsized":"2023-10-16T09:44:03.046137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793527+0000","last_change":"2023-10-16T09:44:00.799160+0000","last_active":"2023-10-16T09:44:02.793527+0000","last_peered":"2023-10-16T09:44:02.793527+0000","last_clean":"2023-10-16T09:44:02.793527+0000","last_became_active":"2023-10-16T09:44:00.798874+0000","last_became_peered":"2023-10-16T09:44:00.798874+0000","last_unstale":"2023-10-16T09:44:02.793527+0000","last_undegraded":"2023-10-16T09:44:02.793527+0000","last_fullsized":"2023-10-16T09:44:02.793527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045858+0000","last_change":"2023-10-16T09:44:01.802604+0000","last_active":"2023-10-16T09:44:03.045858+0000","last_peered":"2023-10-16T09:44:03.045858+0000","last_clean":"2023-10-16T09:44:03.045858+0000","last_became_active":"2023-10-16T09:44:01.801907+0000","last_became_peered":"2023-10-16T09:44:01.801907+0000","last_unstale":"2023-10-16T09:44:03.045858+0000","last_undegraded":"2023-10-16T09:44:03.045858+0000","last_fullsized":"2023-10-16T09:44:03.045858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800616+0000","last_change":"2023-10-16T09:44:00.796930+0000","last_active":"2023-10-16T09:44:01.800616+0000","last_peered":"2023-10-16T09:44:01.800616+0000","last_clean":"2023-10-16T09:44:01.800616+0000","last_became_active":"2023-10-16T09:44:00.796659+0000","last_became_peered":"2023-10-16T09:44:00.796659+0000","last_unstale":"2023-10-16T09:44:01.800616+0000","last_undegraded":"2023-10-16T09:44:01.800616+0000","last_fullsized":"2023-10-16T09:44:01.800616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794538+0000","last_change":"2023-10-16T09:44:01.820645+0000","last_active":"2023-10-16T09:44:02.794538+0000","last_peered":"2023-10-16T09:44:02.794538+0000","last_clean":"2023-10-16T09:44:02.794538+0000","last_became_active":"2023-10-16T09:44:01.820369+0000","last_became_peered":"2023-10-16T09:44:01.820369+0000","last_unstale":"2023-10-16T09:44:02.794538+0000","last_undegraded":"2023-10-16T09:44:02.794538+0000","last_fullsized":"2023-10-16T09:44:02.794538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822381+0000","last_change":"2023-10-16T09:44:00.785075+0000","last_active":"2023-10-16T09:44:01.822381+0000","last_peered":"2023-10-16T09:44:01.822381+0000","last_clean":"2023-10-16T09:44:01.822381+0000","last_became_active":"2023-10-16T09:44:00.784807+0000","last_became_peered":"2023-10-16T09:44:00.784807+0000","last_unstale":"2023-10-16T09:44:01.822381+0000","last_undegraded":"2023-10-16T09:44:01.822381+0000","last_fullsized":"2023-10-16T09:44:01.822381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793947+0000","last_change":"2023-10-16T09:44:01.804040+0000","last_active":"2023-10-16T09:44:02.793947+0000","last_peered":"2023-10-16T09:44:02.793947+0000","last_clean":"2023-10-16T09:44:02.793947+0000","last_became_active":"2023-10-16T09:44:01.803782+0000","last_became_peered":"2023-10-16T09:44:01.803782+0000","last_unstale":"2023-10-16T09:44:02.793947+0000","last_undegraded":"2023-10-16T09:44:02.793947+0000","last_fullsized":"2023-10-16T09:44:02.793947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819101+0000","last_change":"2023-10-16T09:44:00.790542+0000","last_active":"2023-10-16T09:44:01.819101+0000","last_peered":"2023-10-16T09:44:01.819101+0000","last_clean":"2023-10-16T09:44:01.819101+0000","last_became_active":"2023-10-16T09:44:00.790254+0000","last_became_peered":"2023-10-16T09:44:00.790254+0000","last_unstale":"2023-10-16T09:44:01.819101+0000","last_undegraded":"2023-10-16T09:44:01.819101+0000","last_fullsized":"2023-10-16T09:44:01.819101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794115+0000","last_change":"2023-10-16T09:44:01.820628+0000","last_active":"2023-10-16T09:44:02.794115+0000","last_peered":"2023-10-16T09:44:02.794115+0000","last_clean":"2023-10-16T09:44:02.794115+0000","last_became_active":"2023-10-16T09:44:01.820114+0000","last_became_peered":"2023-10-16T09:44:01.820114+0000","last_unstale":"2023-10-16T09:44:02.794115+0000","last_undegraded":"2023-10-16T09:44:02.794115+0000","last_fullsized":"2023-10-16T09:44:02.794115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046278+0000","last_change":"2023-10-16T09:44:01.805259+0000","last_active":"2023-10-16T09:44:03.046278+0000","last_peered":"2023-10-16T09:44:03.046278+0000","last_clean":"2023-10-16T09:44:03.046278+0000","last_became_active":"2023-10-16T09:44:01.804819+0000","last_became_peered":"2023-10-16T09:44:01.804819+0000","last_unstale":"2023-10-16T09:44:03.046278+0000","last_undegraded":"2023-10-16T09:44:03.046278+0000","last_fullsized":"2023-10-16T09:44:03.046278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045666+0000","last_change":"2023-10-16T09:44:01.808510+0000","last_active":"2023-10-16T09:44:03.045666+0000","last_peered":"2023-10-16T09:44:03.045666+0000","last_clean":"2023-10-16T09:44:03.045666+0000","last_became_active":"2023-10-16T09:44:01.808152+0000","last_became_peered":"2023-10-16T09:44:01.808152+0000","last_unstale":"2023-10-16T09:44:03.045666+0000","last_undegraded":"2023-10-16T09:44:03.045666+0000","last_fullsized":"2023-10-16T09:44:03.045666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:13.616924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793846+0000","last_change":"2023-10-16T09:44:01.808070+0000","last_active":"2023-10-16T09:44:02.793846+0000","last_peered":"2023-10-16T09:44:02.793846+0000","last_clean":"2023-10-16T09:44:02.793846+0000","last_became_active":"2023-10-16T09:44:01.807794+0000","last_became_peered":"2023-10-16T09:44:01.807794+0000","last_unstale":"2023-10-16T09:44:02.793846+0000","last_undegraded":"2023-10-16T09:44:02.793846+0000","last_fullsized":"2023-10-16T09:44:02.793846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896204+0000","last_change":"2023-10-16T09:44:01.796651+0000","last_active":"2023-10-16T09:44:02.896204+0000","last_peered":"2023-10-16T09:44:02.896204+0000","last_clean":"2023-10-16T09:44:02.896204+0000","last_became_active":"2023-10-16T09:44:01.796377+0000","last_became_peered":"2023-10-16T09:44:01.796377+0000","last_unstale":"2023-10-16T09:44:02.896204+0000","last_undegraded":"2023-10-16T09:44:02.896204+0000","last_fullsized":"2023-10-16T09:44:02.896204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046174+0000","last_change":"2023-10-16T09:44:01.803048+0000","last_active":"2023-10-16T09:44:03.046174+0000","last_peered":"2023-10-16T09:44:03.046174+0000","last_clean":"2023-10-16T09:44:03.046174+0000","last_became_active":"2023-10-16T09:44:01.802761+0000","last_became_peered":"2023-10-16T09:44:01.802761+0000","last_unstale":"2023-10-16T09:44:03.046174+0000","last_undegraded":"2023-10-16T09:44:03.046174+0000","last_fullsized":"2023-10-16T09:44:03.046174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791913+0000","last_change":"2023-10-16T09:44:01.797818+0000","last_active":"2023-10-16T09:44:02.791913+0000","last_peered":"2023-10-16T09:44:02.791913+0000","last_clean":"2023-10-16T09:44:02.791913+0000","last_became_active":"2023-10-16T09:44:01.797537+0000","last_became_peered":"2023-10-16T09:44:01.797537+0000","last_unstale":"2023-10-16T09:44:02.791913+0000","last_undegraded":"2023-10-16T09:44:02.791913+0000","last_fullsized":"2023-10-16T09:44:02.791913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046247+0000","last_change":"2023-10-16T09:44:01.821446+0000","last_active":"2023-10-16T09:44:03.046247+0000","last_peered":"2023-10-16T09:44:03.046247+0000","last_clean":"2023-10-16T09:44:03.046247+0000","last_became_active":"2023-10-16T09:44:01.821191+0000","last_became_peered":"2023-10-16T09:44:01.821191+0000","last_unstale":"2023-10-16T09:44:03.046247+0000","last_undegraded":"2023-10-16T09:44:03.046247+0000","last_fullsized":"2023-10-16T09:44:03.046247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046237+0000","last_change":"2023-10-16T09:44:01.801738+0000","last_active":"2023-10-16T09:44:03.046237+0000","last_peered":"2023-10-16T09:44:03.046237+0000","last_clean":"2023-10-16T09:44:03.046237+0000","last_became_active":"2023-10-16T09:44:01.801400+0000","last_became_peered":"2023-10-16T09:44:01.801400+0000","last_unstale":"2023-10-16T09:44:03.046237+0000","last_undegraded":"2023-10-16T09:44:03.046237+0000","last_fullsized":"2023-10-16T09:44:03.046237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793441+0000","last_change":"2023-10-16T09:44:01.797904+0000","last_active":"2023-10-16T09:44:02.793441+0000","last_peered":"2023-10-16T09:44:02.793441+0000","last_clean":"2023-10-16T09:44:02.793441+0000","last_became_active":"2023-10-16T09:44:01.797424+0000","last_became_peered":"2023-10-16T09:44:01.797424+0000","last_unstale":"2023-10-16T09:44:02.793441+0000","last_undegraded":"2023-10-16T09:44:02.793441+0000","last_fullsized":"2023-10-16T09:44:02.793441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045818+0000","last_change":"2023-10-16T09:44:01.802067+0000","last_active":"2023-10-16T09:44:03.045818+0000","last_peered":"2023-10-16T09:44:03.045818+0000","last_clean":"2023-10-16T09:44:03.045818+0000","last_became_active":"2023-10-16T09:44:01.801792+0000","last_became_peered":"2023-10-16T09:44:01.801792+0000","last_unstale":"2023-10-16T09:44:03.045818+0000","last_undegraded":"2023-10-16T09:44:03.045818+0000","last_fullsized":"2023-10-16T09:44:03.045818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793644+0000","last_change":"2023-10-16T09:44:01.819606+0000","last_active":"2023-10-16T09:44:02.793644+0000","last_peered":"2023-10-16T09:44:02.793644+0000","last_clean":"2023-10-16T09:44:02.793644+0000","last_became_active":"2023-10-16T09:44:01.819328+0000","last_became_peered":"2023-10-16T09:44:01.819328+0000","last_unstale":"2023-10-16T09:44:02.793644+0000","last_undegraded":"2023-10-16T09:44:02.793644+0000","last_fullsized":"2023-10-16T09:44:02.793644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896102+0000","last_change":"2023-10-16T09:44:01.818209+0000","last_active":"2023-10-16T09:44:02.896102+0000","last_peered":"2023-10-16T09:44:02.896102+0000","last_clean":"2023-10-16T09:44:02.896102+0000","last_became_active":"2023-10-16T09:44:01.817921+0000","last_became_peered":"2023-10-16T09:44:01.817921+0000","last_unstale":"2023-10-16T09:44:02.896102+0000","last_undegraded":"2023-10-16T09:44:02.896102+0000","last_fullsized":"2023-10-16T09:44:02.896102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045707+0000","last_change":"2023-10-16T09:44:01.802316+0000","last_active":"2023-10-16T09:44:03.045707+0000","last_peered":"2023-10-16T09:44:03.045707+0000","last_clean":"2023-10-16T09:44:03.045707+0000","last_became_active":"2023-10-16T09:44:01.802036+0000","last_became_peered":"2023-10-16T09:44:01.802036+0000","last_unstale":"2023-10-16T09:44:03.045707+0000","last_undegraded":"2023-10-16T09:44:03.045707+0000","last_fullsized":"2023-10-16T09:44:03.045707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5090000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3260000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4349999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4160000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2189999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4649999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1140000000000008}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:44:10.704 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:44:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.860+0000 7f51c0584700 1 Processor -- start 2023-10-16T09:44:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.860+0000 7f51c0584700 1 -- start start 2023-10-16T09:44:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51c0584700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51b80cd2d0 0x7f51b80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51c0584700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b80cc510 con 0x7f51b80d0370 2023-10-16T09:44:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51c0584700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b80cc7f0 con 0x7f51b815b990 2023-10-16T09:44:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51c0584700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b80ccad0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51beb21700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51b80cd2d0 0x7f51b80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51bdb1f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f51b80d0370 0x7f51b80ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47576/0 (socket says 172.21.15.181:47576) 2023-10-16T09:44:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51bdb1f700 1 -- 172.21.15.181:0/1169404778 learned_addr learned my addr 172.21.15.181:0/1169404778 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51beb21700 1 -- 172.21.15.181:0/1169404778 >> v1:172.21.15.190:6789/0 conn(0x7f51b815b990 legacy=0x7f51b80cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51beb21700 1 -- 172.21.15.181:0/1169404778 >> v1:172.21.15.181:6789/0 conn(0x7f51b80d0370 legacy=0x7f51b80ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51beb21700 1 -- 172.21.15.181:0/1169404778 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b80ccdb0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51bd31e700 1 -- 172.21.15.181:0/1169404778 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245962670 0 0) 0x7f51b80cc510 con 0x7f51b80d0370 2023-10-16T09:44:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51beb21700 1 --2- 172.21.15.181:0/1169404778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80ce8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009d90 tx=0x7f51b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b096574dfbfab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51bd31e700 1 -- 172.21.15.181:0/1169404778 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b000e070 con 0x7f51b80cd2d0 2023-10-16T09:44:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51bd31e700 1 -- 172.21.15.181:0/1169404778 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b000bb90 con 0x7f51b80cd2d0 2023-10-16T09:44:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.861+0000 7f51bd31e700 1 -- 172.21.15.181:0/1169404778 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0003a20 con 0x7f51b80cd2d0 2023-10-16T09:44:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.862+0000 7f51c0584700 1 -- 172.21.15.181:0/1169404778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 msgr2=0x7f51b80ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.862+0000 7f51c0584700 1 --2- 172.21.15.181:0/1169404778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80ce8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009d90 tx=0x7f51b000b330 comp rx=0 tx=0).stop 2023-10-16T09:44:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.862+0000 7f51c0584700 1 -- 172.21.15.181:0/1169404778 shutdown_connections 2023-10-16T09:44:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.862+0000 7f51c0584700 1 --2- 172.21.15.181:0/1169404778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80ce8f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.862+0000 7f51c0584700 1 -- 172.21.15.181:0/1169404778 >> 172.21.15.181:0/1169404778 conn(0x7f51b809a9a0 msgr2=0x7f51b80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.862+0000 7f51c0584700 1 -- 172.21.15.181:0/1169404778 shutdown_connections 2023-10-16T09:44:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.862+0000 7f51c0584700 1 -- 172.21.15.181:0/1169404778 wait complete. 2023-10-16T09:44:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.863+0000 7f51c0584700 1 Processor -- start 2023-10-16T09:44:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.863+0000 7f51c0584700 1 -- start start 2023-10-16T09:44:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51c0584700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51c0584700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51b80d0370 0x7f51b81f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51c0584700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51b815b990 0x7f51b81f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51c0584700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80b71c0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51c0584700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81f56c0 con 0x7f51b80d0370 2023-10-16T09:44:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51c0584700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80b5860 con 0x7f51b815b990 2023-10-16T09:44:10.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36504/0 (socket says 172.21.15.181:36504) 2023-10-16T09:44:10.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 -- 172.21.15.181:0/2650493287 learned_addr learned my addr 172.21.15.181:0/2650493287 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:10.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51bdb1f700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51b80d0370 0x7f51b81f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51beb21700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51b815b990 0x7f51b81f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 -- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51b815b990 msgr2=0x7f51b81f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51b815b990 0x7f51b81f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 -- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51b80d0370 msgr2=0x7f51b81f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:10.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51b80d0370 0x7f51b81f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:10.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 -- 172.21.15.181:0/2650493287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0009a70 con 0x7f51b80cd2d0 2023-10-16T09:44:10.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.864+0000 7f51be320700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80c8750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c520 tx=0x7f51b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:10.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.865+0000 7f51aeffd700 1 -- 172.21.15.181:0/2650493287 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40054e0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.865+0000 7f51aeffd700 1 -- 172.21.15.181:0/2650493287 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40036d0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.865+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b80b6ee0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.865+0000 7f51aeffd700 1 -- 172.21.15.181:0/2650493287 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4003a80 con 0x7f51b80cd2d0 2023-10-16T09:44:10.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.865+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b80b54f0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.865+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b80b5070 con 0x7f51b80cd2d0 2023-10-16T09:44:10.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.866+0000 7f51aeffd700 1 -- 172.21.15.181:0/2650493287 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f51b4003be0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.866+0000 7f51aeffd700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f519c06af00 0x7f519c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:10.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.866+0000 7f51aeffd700 1 -- 172.21.15.181:0/2650493287 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51b4088ab0 con 0x7f51b80cd2d0 2023-10-16T09:44:10.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.866+0000 7f51bdb1f700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f519c06af00 0x7f519c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:10.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.867+0000 7f51bdb1f700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f519c06af00 0x7f519c06d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51a8000f60 tx=0x7f51a8007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:10.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:10.871+0000 7f51aeffd700 1 -- 172.21.15.181:0/2650493287 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f51b80b5070 con 0x7f51b80cd2d0 2023-10-16T09:44:11.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.059+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51b80cc200 con 0x7f519c06af00 2023-10-16T09:44:11.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.066+0000 7f51aeffd700 1 -- 172.21.15.181:0/2650493287 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264585 (secure 0 0 0) 0x7f51a8012380 con 0x7f519c06af00 2023-10-16T09:44:11.070 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:11.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f519c06af00 msgr2=0x7f519c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f519c06af00 0x7f519c06d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51a8000f60 tx=0x7f51a8007c90 comp rx=0 tx=0).stop 2023-10-16T09:44:11.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 msgr2=0x7f51b80c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80c8750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c520 tx=0x7f51b400c8e0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 shutdown_connections 2023-10-16T09:44:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f519c06af00 0x7f519c06d3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51b815b990 0x7f51b81f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cd2d0 0x7f51b80c8750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 --2- 172.21.15.181:0/2650493287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51b80d0370 0x7f51b81f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.069+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 >> 172.21.15.181:0/2650493287 conn(0x7f51b809a9a0 msgr2=0x7f51b80cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:11.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.070+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 shutdown_connections 2023-10-16T09:44:11.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.070+0000 7f51c0584700 1 -- 172.21.15.181:0/2650493287 wait complete. 2023-10-16T09:44:11.075 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:44:11.082 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T09:44:09.679360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219176,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755544,"statfs":{"total":773094113280,"available":772869677056,"internally_reserved":0,"allocated":3055616,"data_stored":1816468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002065"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793938+0000","last_change":"2023-10-16T09:44:01.820375+0000","last_active":"2023-10-16T09:44:02.793938+0000","last_peered":"2023-10-16T09:44:02.793938+0000","last_clean":"2023-10-16T09:44:02.793938+0000","last_became_active":"2023-10-16T09:44:01.819619+0000","last_became_peered":"2023-10-16T09:44:01.819619+0000","last_unstale":"2023-10-16T09:44:02.793938+0000","last_undegraded":"2023-10-16T09:44:02.793938+0000","last_fullsized":"2023-10-16T09:44:02.793938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792204+0000","last_change":"2023-10-16T09:44:01.800363+0000","last_active":"2023-10-16T09:44:02.792204+0000","last_peered":"2023-10-16T09:44:02.792204+0000","last_clean":"2023-10-16T09:44:02.792204+0000","last_became_active":"2023-10-16T09:44:01.800037+0000","last_became_peered":"2023-10-16T09:44:01.800037+0000","last_unstale":"2023-10-16T09:44:02.792204+0000","last_undegraded":"2023-10-16T09:44:02.792204+0000","last_fullsized":"2023-10-16T09:44:02.792204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896175+0000","last_change":"2023-10-16T09:44:01.806686+0000","last_active":"2023-10-16T09:44:02.896175+0000","last_peered":"2023-10-16T09:44:02.896175+0000","last_clean":"2023-10-16T09:44:02.896175+0000","last_became_active":"2023-10-16T09:44:01.806361+0000","last_became_peered":"2023-10-16T09:44:01.806361+0000","last_unstale":"2023-10-16T09:44:02.896175+0000","last_undegraded":"2023-10-16T09:44:02.896175+0000","last_fullsized":"2023-10-16T09:44:02.896175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793996+0000","last_change":"2023-10-16T09:44:01.802398+0000","last_active":"2023-10-16T09:44:02.793996+0000","last_peered":"2023-10-16T09:44:02.793996+0000","last_clean":"2023-10-16T09:44:02.793996+0000","last_became_active":"2023-10-16T09:44:01.802077+0000","last_became_peered":"2023-10-16T09:44:01.802077+0000","last_unstale":"2023-10-16T09:44:02.793996+0000","last_undegraded":"2023-10-16T09:44:02.793996+0000","last_fullsized":"2023-10-16T09:44:02.793996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793271+0000","last_change":"2023-10-16T09:44:01.817219+0000","last_active":"2023-10-16T09:44:02.793271+0000","last_peered":"2023-10-16T09:44:02.793271+0000","last_clean":"2023-10-16T09:44:02.793271+0000","last_became_active":"2023-10-16T09:44:01.816945+0000","last_became_peered":"2023-10-16T09:44:01.816945+0000","last_unstale":"2023-10-16T09:44:02.793271+0000","last_undegraded":"2023-10-16T09:44:02.793271+0000","last_fullsized":"2023-10-16T09:44:02.793271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794026+0000","last_change":"2023-10-16T09:44:01.807539+0000","last_active":"2023-10-16T09:44:02.794026+0000","last_peered":"2023-10-16T09:44:02.794026+0000","last_clean":"2023-10-16T09:44:02.794026+0000","last_became_active":"2023-10-16T09:44:01.807282+0000","last_became_peered":"2023-10-16T09:44:01.807282+0000","last_unstale":"2023-10-16T09:44:02.794026+0000","last_undegraded":"2023-10-16T09:44:02.794026+0000","last_fullsized":"2023-10-16T09:44:02.794026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792808+0000","last_change":"2023-10-16T09:44:01.816493+0000","last_active":"2023-10-16T09:44:02.792808+0000","last_peered":"2023-10-16T09:44:02.792808+0000","last_clean":"2023-10-16T09:44:02.792808+0000","last_became_active":"2023-10-16T09:44:01.816006+0000","last_became_peered":"2023-10-16T09:44:01.816006+0000","last_unstale":"2023-10-16T09:44:02.792808+0000","last_undegraded":"2023-10-16T09:44:02.792808+0000","last_fullsized":"2023-10-16T09:44:02.792808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045389+0000","last_change":"2023-10-16T09:44:01.802910+0000","last_active":"2023-10-16T09:44:03.045389+0000","last_peered":"2023-10-16T09:44:03.045389+0000","last_clean":"2023-10-16T09:44:03.045389+0000","last_became_active":"2023-10-16T09:44:01.802640+0000","last_became_peered":"2023-10-16T09:44:01.802640+0000","last_unstale":"2023-10-16T09:44:03.045389+0000","last_undegraded":"2023-10-16T09:44:03.045389+0000","last_fullsized":"2023-10-16T09:44:03.045389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793935+0000","last_change":"2023-10-16T09:44:01.818818+0000","last_active":"2023-10-16T09:44:02.793935+0000","last_peered":"2023-10-16T09:44:02.793935+0000","last_clean":"2023-10-16T09:44:02.793935+0000","last_became_active":"2023-10-16T09:44:01.818532+0000","last_became_peered":"2023-10-16T09:44:01.818532+0000","last_unstale":"2023-10-16T09:44:02.793935+0000","last_undegraded":"2023-10-16T09:44:02.793935+0000","last_fullsized":"2023-10-16T09:44:02.793935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046230+0000","last_change":"2023-10-16T09:44:01.808945+0000","last_active":"2023-10-16T09:44:03.046230+0000","last_peered":"2023-10-16T09:44:03.046230+0000","last_clean":"2023-10-16T09:44:03.046230+0000","last_became_active":"2023-10-16T09:44:01.808655+0000","last_became_peered":"2023-10-16T09:44:01.808655+0000","last_unstale":"2023-10-16T09:44:03.046230+0000","last_undegraded":"2023-10-16T09:44:03.046230+0000","last_fullsized":"2023-10-16T09:44:03.046230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045896+0000","last_change":"2023-10-16T09:44:01.808213+0000","last_active":"2023-10-16T09:44:03.045896+0000","last_peered":"2023-10-16T09:44:03.045896+0000","last_clean":"2023-10-16T09:44:03.045896+0000","last_became_active":"2023-10-16T09:44:01.807715+0000","last_became_peered":"2023-10-16T09:44:01.807715+0000","last_unstale":"2023-10-16T09:44:03.045896+0000","last_undegraded":"2023-10-16T09:44:03.045896+0000","last_fullsized":"2023-10-16T09:44:03.045896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046320+0000","last_change":"2023-10-16T09:44:01.801296+0000","last_active":"2023-10-16T09:44:03.046320+0000","last_peered":"2023-10-16T09:44:03.046320+0000","last_clean":"2023-10-16T09:44:03.046320+0000","last_became_active":"2023-10-16T09:44:01.800456+0000","last_became_peered":"2023-10-16T09:44:01.800456+0000","last_unstale":"2023-10-16T09:44:03.046320+0000","last_undegraded":"2023-10-16T09:44:03.046320+0000","last_fullsized":"2023-10-16T09:44:03.046320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:56.560951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045786+0000","last_change":"2023-10-16T09:44:01.800574+0000","last_active":"2023-10-16T09:44:03.045786+0000","last_peered":"2023-10-16T09:44:03.045786+0000","last_clean":"2023-10-16T09:44:03.045786+0000","last_became_active":"2023-10-16T09:44:01.800299+0000","last_became_peered":"2023-10-16T09:44:01.800299+0000","last_unstale":"2023-10-16T09:44:03.045786+0000","last_undegraded":"2023-10-16T09:44:03.045786+0000","last_fullsized":"2023-10-16T09:44:03.045786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895941+0000","last_change":"2023-10-16T09:44:01.797679+0000","last_active":"2023-10-16T09:44:02.895941+0000","last_peered":"2023-10-16T09:44:02.895941+0000","last_clean":"2023-10-16T09:44:02.895941+0000","last_became_active":"2023-10-16T09:44:01.797420+0000","last_became_peered":"2023-10-16T09:44:01.797420+0000","last_unstale":"2023-10-16T09:44:02.895941+0000","last_undegraded":"2023-10-16T09:44:02.895941+0000","last_fullsized":"2023-10-16T09:44:02.895941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791819+0000","last_change":"2023-10-16T09:44:01.801979+0000","last_active":"2023-10-16T09:44:02.791819+0000","last_peered":"2023-10-16T09:44:02.791819+0000","last_clean":"2023-10-16T09:44:02.791819+0000","last_became_active":"2023-10-16T09:44:01.801663+0000","last_became_peered":"2023-10-16T09:44:01.801663+0000","last_unstale":"2023-10-16T09:44:02.791819+0000","last_undegraded":"2023-10-16T09:44:02.791819+0000","last_fullsized":"2023-10-16T09:44:02.791819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793275+0000","last_change":"2023-10-16T09:44:01.808377+0000","last_active":"2023-10-16T09:44:02.793275+0000","last_peered":"2023-10-16T09:44:02.793275+0000","last_clean":"2023-10-16T09:44:02.793275+0000","last_became_active":"2023-10-16T09:44:01.808091+0000","last_became_peered":"2023-10-16T09:44:01.808091+0000","last_unstale":"2023-10-16T09:44:02.793275+0000","last_undegraded":"2023-10-16T09:44:02.793275+0000","last_fullsized":"2023-10-16T09:44:02.793275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794670+0000","last_change":"2023-10-16T09:44:01.815509+0000","last_active":"2023-10-16T09:44:02.794670+0000","last_peered":"2023-10-16T09:44:02.794670+0000","last_clean":"2023-10-16T09:44:02.794670+0000","last_became_active":"2023-10-16T09:44:01.815290+0000","last_became_peered":"2023-10-16T09:44:01.815290+0000","last_unstale":"2023-10-16T09:44:02.794670+0000","last_undegraded":"2023-10-16T09:44:02.794670+0000","last_fullsized":"2023-10-16T09:44:02.794670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045622+0000","last_change":"2023-10-16T09:44:01.804696+0000","last_active":"2023-10-16T09:44:03.045622+0000","last_peered":"2023-10-16T09:44:03.045622+0000","last_clean":"2023-10-16T09:44:03.045622+0000","last_became_active":"2023-10-16T09:44:01.804427+0000","last_became_peered":"2023-10-16T09:44:01.804427+0000","last_unstale":"2023-10-16T09:44:03.045622+0000","last_undegraded":"2023-10-16T09:44:03.045622+0000","last_fullsized":"2023-10-16T09:44:03.045622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:06.338018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794299+0000","last_change":"2023-10-16T09:44:01.815090+0000","last_active":"2023-10-16T09:44:02.794299+0000","last_peered":"2023-10-16T09:44:02.794299+0000","last_clean":"2023-10-16T09:44:02.794299+0000","last_became_active":"2023-10-16T09:44:01.814787+0000","last_became_peered":"2023-10-16T09:44:01.814787+0000","last_unstale":"2023-10-16T09:44:02.794299+0000","last_undegraded":"2023-10-16T09:44:02.794299+0000","last_fullsized":"2023-10-16T09:44:02.794299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.820250+0000","last_change":"2023-10-16T09:44:00.797577+0000","last_active":"2023-10-16T09:44:01.820250+0000","last_peered":"2023-10-16T09:44:01.820250+0000","last_clean":"2023-10-16T09:44:01.820250+0000","last_became_active":"2023-10-16T09:44:00.797323+0000","last_became_peered":"2023-10-16T09:44:00.797323+0000","last_unstale":"2023-10-16T09:44:01.820250+0000","last_undegraded":"2023-10-16T09:44:01.820250+0000","last_fullsized":"2023-10-16T09:44:01.820250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046254+0000","last_change":"2023-10-16T09:44:01.801503+0000","last_active":"2023-10-16T09:44:03.046254+0000","last_peered":"2023-10-16T09:44:03.046254+0000","last_clean":"2023-10-16T09:44:03.046254+0000","last_became_active":"2023-10-16T09:44:01.801078+0000","last_became_peered":"2023-10-16T09:44:01.801078+0000","last_unstale":"2023-10-16T09:44:03.046254+0000","last_undegraded":"2023-10-16T09:44:03.046254+0000","last_fullsized":"2023-10-16T09:44:03.046254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:44.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819479+0000","last_change":"2023-10-16T09:44:00.790199+0000","last_active":"2023-10-16T09:44:01.819479+0000","last_peered":"2023-10-16T09:44:01.819479+0000","last_clean":"2023-10-16T09:44:01.819479+0000","last_became_active":"2023-10-16T09:44:00.789914+0000","last_became_peered":"2023-10-16T09:44:00.789914+0000","last_unstale":"2023-10-16T09:44:01.819479+0000","last_undegraded":"2023-10-16T09:44:01.819479+0000","last_fullsized":"2023-10-16T09:44:01.819479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896280+0000","last_change":"2023-10-16T09:44:01.808873+0000","last_active":"2023-10-16T09:44:02.896280+0000","last_peered":"2023-10-16T09:44:02.896280+0000","last_clean":"2023-10-16T09:44:02.896280+0000","last_became_active":"2023-10-16T09:44:01.808557+0000","last_became_peered":"2023-10-16T09:44:01.808557+0000","last_unstale":"2023-10-16T09:44:02.896280+0000","last_undegraded":"2023-10-16T09:44:02.896280+0000","last_fullsized":"2023-10-16T09:44:02.896280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794605+0000","last_change":"2023-10-16T09:44:00.801288+0000","last_active":"2023-10-16T09:44:02.794605+0000","last_peered":"2023-10-16T09:44:02.794605+0000","last_clean":"2023-10-16T09:44:02.794605+0000","last_became_active":"2023-10-16T09:44:00.801008+0000","last_became_peered":"2023-10-16T09:44:00.801008+0000","last_unstale":"2023-10-16T09:44:02.794605+0000","last_undegraded":"2023-10-16T09:44:02.794605+0000","last_fullsized":"2023-10-16T09:44:02.794605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793760+0000","last_change":"2023-10-16T09:44:01.803802+0000","last_active":"2023-10-16T09:44:02.793760+0000","last_peered":"2023-10-16T09:44:02.793760+0000","last_clean":"2023-10-16T09:44:02.793760+0000","last_became_active":"2023-10-16T09:44:01.803480+0000","last_became_peered":"2023-10-16T09:44:01.803480+0000","last_unstale":"2023-10-16T09:44:02.793760+0000","last_undegraded":"2023-10-16T09:44:02.793760+0000","last_fullsized":"2023-10-16T09:44:02.793760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793452+0000","last_change":"2023-10-16T09:44:00.798160+0000","last_active":"2023-10-16T09:44:02.793452+0000","last_peered":"2023-10-16T09:44:02.793452+0000","last_clean":"2023-10-16T09:44:02.793452+0000","last_became_active":"2023-10-16T09:44:00.797833+0000","last_became_peered":"2023-10-16T09:44:00.797833+0000","last_unstale":"2023-10-16T09:44:02.793452+0000","last_undegraded":"2023-10-16T09:44:02.793452+0000","last_fullsized":"2023-10-16T09:44:02.793452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794244+0000","last_change":"2023-10-16T09:44:01.802107+0000","last_active":"2023-10-16T09:44:02.794244+0000","last_peered":"2023-10-16T09:44:02.794244+0000","last_clean":"2023-10-16T09:44:02.794244+0000","last_became_active":"2023-10-16T09:44:01.801774+0000","last_became_peered":"2023-10-16T09:44:01.801774+0000","last_unstale":"2023-10-16T09:44:02.794244+0000","last_undegraded":"2023-10-16T09:44:02.794244+0000","last_fullsized":"2023-10-16T09:44:02.794244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822531+0000","last_change":"2023-10-16T09:44:00.785046+0000","last_active":"2023-10-16T09:44:01.822531+0000","last_peered":"2023-10-16T09:44:01.822531+0000","last_clean":"2023-10-16T09:44:01.822531+0000","last_became_active":"2023-10-16T09:44:00.784767+0000","last_became_peered":"2023-10-16T09:44:00.784767+0000","last_unstale":"2023-10-16T09:44:01.822531+0000","last_undegraded":"2023-10-16T09:44:01.822531+0000","last_fullsized":"2023-10-16T09:44:01.822531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045674+0000","last_change":"2023-10-16T09:44:01.799372+0000","last_active":"2023-10-16T09:44:03.045674+0000","last_peered":"2023-10-16T09:44:03.045674+0000","last_clean":"2023-10-16T09:44:03.045674+0000","last_became_active":"2023-10-16T09:44:01.799056+0000","last_became_peered":"2023-10-16T09:44:01.799056+0000","last_unstale":"2023-10-16T09:44:03.045674+0000","last_undegraded":"2023-10-16T09:44:03.045674+0000","last_fullsized":"2023-10-16T09:44:03.045674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:44:46.117646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045848+0000","last_change":"2023-10-16T09:44:00.800927+0000","last_active":"2023-10-16T09:44:03.045848+0000","last_peered":"2023-10-16T09:44:03.045848+0000","last_clean":"2023-10-16T09:44:03.045848+0000","last_became_active":"2023-10-16T09:44:00.800533+0000","last_became_peered":"2023-10-16T09:44:00.800533+0000","last_unstale":"2023-10-16T09:44:03.045848+0000","last_undegraded":"2023-10-16T09:44:03.045848+0000","last_fullsized":"2023-10-16T09:44:03.045848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896238+0000","last_change":"2023-10-16T09:44:01.795883+0000","last_active":"2023-10-16T09:44:02.896238+0000","last_peered":"2023-10-16T09:44:02.896238+0000","last_clean":"2023-10-16T09:44:02.896238+0000","last_became_active":"2023-10-16T09:44:01.795613+0000","last_became_peered":"2023-10-16T09:44:01.795613+0000","last_unstale":"2023-10-16T09:44:02.896238+0000","last_undegraded":"2023-10-16T09:44:02.896238+0000","last_fullsized":"2023-10-16T09:44:02.896238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792936+0000","last_change":"2023-10-16T09:44:00.800124+0000","last_active":"2023-10-16T09:44:01.792936+0000","last_peered":"2023-10-16T09:44:01.792936+0000","last_clean":"2023-10-16T09:44:01.792936+0000","last_became_active":"2023-10-16T09:44:00.799861+0000","last_became_peered":"2023-10-16T09:44:00.799861+0000","last_unstale":"2023-10-16T09:44:01.792936+0000","last_undegraded":"2023-10-16T09:44:01.792936+0000","last_fullsized":"2023-10-16T09:44:01.792936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045513+0000","last_change":"2023-10-16T09:44:01.799159+0000","last_active":"2023-10-16T09:44:03.045513+0000","last_peered":"2023-10-16T09:44:03.045513+0000","last_clean":"2023-10-16T09:44:03.045513+0000","last_became_active":"2023-10-16T09:44:01.798882+0000","last_became_peered":"2023-10-16T09:44:01.798882+0000","last_unstale":"2023-10-16T09:44:03.045513+0000","last_undegraded":"2023-10-16T09:44:03.045513+0000","last_fullsized":"2023-10-16T09:44:03.045513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798407+0000","last_change":"2023-10-16T09:44:00.797328+0000","last_active":"2023-10-16T09:44:01.798407+0000","last_peered":"2023-10-16T09:44:01.798407+0000","last_clean":"2023-10-16T09:44:01.798407+0000","last_became_active":"2023-10-16T09:44:00.797038+0000","last_became_peered":"2023-10-16T09:44:00.797038+0000","last_unstale":"2023-10-16T09:44:01.798407+0000","last_undegraded":"2023-10-16T09:44:01.798407+0000","last_fullsized":"2023-10-16T09:44:01.798407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:18.584437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791948+0000","last_change":"2023-10-16T09:44:01.797648+0000","last_active":"2023-10-16T09:44:02.791948+0000","last_peered":"2023-10-16T09:44:02.791948+0000","last_clean":"2023-10-16T09:44:02.791948+0000","last_became_active":"2023-10-16T09:44:01.797374+0000","last_became_peered":"2023-10-16T09:44:01.797374+0000","last_unstale":"2023-10-16T09:44:02.791948+0000","last_undegraded":"2023-10-16T09:44:02.791948+0000","last_fullsized":"2023-10-16T09:44:02.791948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046347+0000","last_change":"2023-10-16T09:44:00.796911+0000","last_active":"2023-10-16T09:44:03.046347+0000","last_peered":"2023-10-16T09:44:03.046347+0000","last_clean":"2023-10-16T09:44:03.046347+0000","last_became_active":"2023-10-16T09:44:00.796666+0000","last_became_peered":"2023-10-16T09:44:00.796666+0000","last_unstale":"2023-10-16T09:44:03.046347+0000","last_undegraded":"2023-10-16T09:44:03.046347+0000","last_fullsized":"2023-10-16T09:44:03.046347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794228+0000","last_change":"2023-10-16T09:44:01.815668+0000","last_active":"2023-10-16T09:44:02.794228+0000","last_peered":"2023-10-16T09:44:02.794228+0000","last_clean":"2023-10-16T09:44:02.794228+0000","last_became_active":"2023-10-16T09:44:01.815327+0000","last_became_peered":"2023-10-16T09:44:01.815327+0000","last_unstale":"2023-10-16T09:44:02.794228+0000","last_undegraded":"2023-10-16T09:44:02.794228+0000","last_fullsized":"2023-10-16T09:44:02.794228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:29:37.972925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798169+0000","last_change":"2023-10-16T09:44:00.800951+0000","last_active":"2023-10-16T09:44:01.798169+0000","last_peered":"2023-10-16T09:44:01.798169+0000","last_clean":"2023-10-16T09:44:01.798169+0000","last_became_active":"2023-10-16T09:44:00.800685+0000","last_became_peered":"2023-10-16T09:44:00.800685+0000","last_unstale":"2023-10-16T09:44:01.798169+0000","last_undegraded":"2023-10-16T09:44:01.798169+0000","last_fullsized":"2023-10-16T09:44:01.798169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:38.141086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896480+0000","last_change":"2023-10-16T09:44:01.808665+0000","last_active":"2023-10-16T09:44:02.896480+0000","last_peered":"2023-10-16T09:44:02.896480+0000","last_clean":"2023-10-16T09:44:02.896480+0000","last_became_active":"2023-10-16T09:44:01.808229+0000","last_became_peered":"2023-10-16T09:44:01.808229+0000","last_unstale":"2023-10-16T09:44:02.896480+0000","last_undegraded":"2023-10-16T09:44:02.896480+0000","last_fullsized":"2023-10-16T09:44:02.896480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.824237+0000","last_change":"2023-10-16T09:44:00.797832+0000","last_active":"2023-10-16T09:44:01.824237+0000","last_peered":"2023-10-16T09:44:01.824237+0000","last_clean":"2023-10-16T09:44:01.824237+0000","last_became_active":"2023-10-16T09:44:00.797569+0000","last_became_peered":"2023-10-16T09:44:00.797569+0000","last_unstale":"2023-10-16T09:44:01.824237+0000","last_undegraded":"2023-10-16T09:44:01.824237+0000","last_fullsized":"2023-10-16T09:44:01.824237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046342+0000","last_change":"2023-10-16T09:44:01.820469+0000","last_active":"2023-10-16T09:44:03.046342+0000","last_peered":"2023-10-16T09:44:03.046342+0000","last_clean":"2023-10-16T09:44:03.046342+0000","last_became_active":"2023-10-16T09:44:01.820207+0000","last_became_peered":"2023-10-16T09:44:01.820207+0000","last_unstale":"2023-10-16T09:44:03.046342+0000","last_undegraded":"2023-10-16T09:44:03.046342+0000","last_fullsized":"2023-10-16T09:44:03.046342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:16.100248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045792+0000","last_change":"2023-10-16T09:44:00.794450+0000","last_active":"2023-10-16T09:44:03.045792+0000","last_peered":"2023-10-16T09:44:03.045792+0000","last_clean":"2023-10-16T09:44:03.045792+0000","last_became_active":"2023-10-16T09:44:00.794197+0000","last_became_peered":"2023-10-16T09:44:00.794197+0000","last_unstale":"2023-10-16T09:44:03.045792+0000","last_undegraded":"2023-10-16T09:44:03.045792+0000","last_fullsized":"2023-10-16T09:44:03.045792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794469+0000","last_change":"2023-10-16T09:44:01.817900+0000","last_active":"2023-10-16T09:44:02.794469+0000","last_peered":"2023-10-16T09:44:02.794469+0000","last_clean":"2023-10-16T09:44:02.794469+0000","last_became_active":"2023-10-16T09:44:01.817666+0000","last_became_peered":"2023-10-16T09:44:01.817666+0000","last_unstale":"2023-10-16T09:44:02.794469+0000","last_undegraded":"2023-10-16T09:44:02.794469+0000","last_fullsized":"2023-10-16T09:44:02.794469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822707+0000","last_change":"2023-10-16T09:44:00.794006+0000","last_active":"2023-10-16T09:44:01.822707+0000","last_peered":"2023-10-16T09:44:01.822707+0000","last_clean":"2023-10-16T09:44:01.822707+0000","last_became_active":"2023-10-16T09:44:00.793703+0000","last_became_peered":"2023-10-16T09:44:00.793703+0000","last_unstale":"2023-10-16T09:44:01.822707+0000","last_undegraded":"2023-10-16T09:44:01.822707+0000","last_fullsized":"2023-10-16T09:44:01.822707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:19.442167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895868+0000","last_change":"2023-10-16T09:44:01.815023+0000","last_active":"2023-10-16T09:44:02.895868+0000","last_peered":"2023-10-16T09:44:02.895868+0000","last_clean":"2023-10-16T09:44:02.895868+0000","last_became_active":"2023-10-16T09:44:01.814737+0000","last_became_peered":"2023-10-16T09:44:01.814737+0000","last_unstale":"2023-10-16T09:44:02.895868+0000","last_undegraded":"2023-10-16T09:44:02.895868+0000","last_fullsized":"2023-10-16T09:44:02.895868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:59.336084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.799063+0000","last_change":"2023-10-16T09:44:00.792205+0000","last_active":"2023-10-16T09:44:01.799063+0000","last_peered":"2023-10-16T09:44:01.799063+0000","last_clean":"2023-10-16T09:44:01.799063+0000","last_became_active":"2023-10-16T09:44:00.791860+0000","last_became_peered":"2023-10-16T09:44:00.791860+0000","last_unstale":"2023-10-16T09:44:01.799063+0000","last_undegraded":"2023-10-16T09:44:01.799063+0000","last_fullsized":"2023-10-16T09:44:01.799063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:45.078590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793427+0000","last_change":"2023-10-16T09:44:01.820397+0000","last_active":"2023-10-16T09:44:02.793427+0000","last_peered":"2023-10-16T09:44:02.793427+0000","last_clean":"2023-10-16T09:44:02.793427+0000","last_became_active":"2023-10-16T09:44:01.820151+0000","last_became_peered":"2023-10-16T09:44:01.820151+0000","last_unstale":"2023-10-16T09:44:02.793427+0000","last_undegraded":"2023-10-16T09:44:02.793427+0000","last_fullsized":"2023-10-16T09:44:02.793427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045738+0000","last_change":"2023-10-16T09:44:00.799178+0000","last_active":"2023-10-16T09:44:03.045738+0000","last_peered":"2023-10-16T09:44:03.045738+0000","last_clean":"2023-10-16T09:44:03.045738+0000","last_became_active":"2023-10-16T09:44:00.798866+0000","last_became_peered":"2023-10-16T09:44:00.798866+0000","last_unstale":"2023-10-16T09:44:03.045738+0000","last_undegraded":"2023-10-16T09:44:03.045738+0000","last_fullsized":"2023-10-16T09:44:03.045738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045651+0000","last_change":"2023-10-16T09:44:01.805085+0000","last_active":"2023-10-16T09:44:03.045651+0000","last_peered":"2023-10-16T09:44:03.045651+0000","last_clean":"2023-10-16T09:44:03.045651+0000","last_became_active":"2023-10-16T09:44:01.804493+0000","last_became_peered":"2023-10-16T09:44:01.804493+0000","last_unstale":"2023-10-16T09:44:03.045651+0000","last_undegraded":"2023-10-16T09:44:03.045651+0000","last_fullsized":"2023-10-16T09:44:03.045651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:11.787967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819839+0000","last_change":"2023-10-16T09:44:00.788023+0000","last_active":"2023-10-16T09:44:01.819839+0000","last_peered":"2023-10-16T09:44:01.819839+0000","last_clean":"2023-10-16T09:44:01.819839+0000","last_became_active":"2023-10-16T09:44:00.787751+0000","last_became_peered":"2023-10-16T09:44:00.787751+0000","last_unstale":"2023-10-16T09:44:01.819839+0000","last_undegraded":"2023-10-16T09:44:01.819839+0000","last_fullsized":"2023-10-16T09:44:01.819839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:25:10.084733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793056+0000","last_change":"2023-10-16T09:44:01.818990+0000","last_active":"2023-10-16T09:44:02.793056+0000","last_peered":"2023-10-16T09:44:02.793056+0000","last_clean":"2023-10-16T09:44:02.793056+0000","last_became_active":"2023-10-16T09:44:01.818711+0000","last_became_peered":"2023-10-16T09:44:01.818711+0000","last_unstale":"2023-10-16T09:44:02.793056+0000","last_undegraded":"2023-10-16T09:44:02.793056+0000","last_fullsized":"2023-10-16T09:44:02.793056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.817348+0000","last_change":"2023-10-16T09:44:00.788065+0000","last_active":"2023-10-16T09:44:01.817348+0000","last_peered":"2023-10-16T09:44:01.817348+0000","last_clean":"2023-10-16T09:44:01.817348+0000","last_became_active":"2023-10-16T09:44:00.787742+0000","last_became_peered":"2023-10-16T09:44:00.787742+0000","last_unstale":"2023-10-16T09:44:01.817348+0000","last_undegraded":"2023-10-16T09:44:01.817348+0000","last_fullsized":"2023-10-16T09:44:01.817348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:48:18.670482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794288+0000","last_change":"2023-10-16T09:44:01.821150+0000","last_active":"2023-10-16T09:44:02.794288+0000","last_peered":"2023-10-16T09:44:02.794288+0000","last_clean":"2023-10-16T09:44:02.794288+0000","last_became_active":"2023-10-16T09:44:01.820830+0000","last_became_peered":"2023-10-16T09:44:01.820830+0000","last_unstale":"2023-10-16T09:44:02.794288+0000","last_undegraded":"2023-10-16T09:44:02.794288+0000","last_fullsized":"2023-10-16T09:44:02.794288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:38.892535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.792997+0000","last_change":"2023-10-16T09:44:00.790958+0000","last_active":"2023-10-16T09:44:01.792997+0000","last_peered":"2023-10-16T09:44:01.792997+0000","last_clean":"2023-10-16T09:44:01.792997+0000","last_became_active":"2023-10-16T09:44:00.790679+0000","last_became_peered":"2023-10-16T09:44:00.790679+0000","last_unstale":"2023-10-16T09:44:01.792997+0000","last_undegraded":"2023-10-16T09:44:01.792997+0000","last_fullsized":"2023-10-16T09:44:01.792997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:23.552983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794431+0000","last_change":"2023-10-16T09:44:01.801190+0000","last_active":"2023-10-16T09:44:02.794431+0000","last_peered":"2023-10-16T09:44:02.794431+0000","last_clean":"2023-10-16T09:44:02.794431+0000","last_became_active":"2023-10-16T09:44:01.800889+0000","last_became_peered":"2023-10-16T09:44:01.800889+0000","last_unstale":"2023-10-16T09:44:02.794431+0000","last_undegraded":"2023-10-16T09:44:02.794431+0000","last_fullsized":"2023-10-16T09:44:02.794431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:42.115516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798899+0000","last_change":"2023-10-16T09:44:00.792936+0000","last_active":"2023-10-16T09:44:01.798899+0000","last_peered":"2023-10-16T09:44:01.798899+0000","last_clean":"2023-10-16T09:44:01.798899+0000","last_became_active":"2023-10-16T09:44:00.792595+0000","last_became_peered":"2023-10-16T09:44:00.792595+0000","last_unstale":"2023-10-16T09:44:01.798899+0000","last_undegraded":"2023-10-16T09:44:01.798899+0000","last_fullsized":"2023-10-16T09:44:01.798899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:22.500292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045929+0000","last_change":"2023-10-16T09:44:01.800944+0000","last_active":"2023-10-16T09:44:03.045929+0000","last_peered":"2023-10-16T09:44:03.045929+0000","last_clean":"2023-10-16T09:44:03.045929+0000","last_became_active":"2023-10-16T09:44:01.800569+0000","last_became_peered":"2023-10-16T09:44:01.800569+0000","last_unstale":"2023-10-16T09:44:03.045929+0000","last_undegraded":"2023-10-16T09:44:03.045929+0000","last_fullsized":"2023-10-16T09:44:03.045929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:38:40.688657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791775+0000","last_change":"2023-10-16T09:44:01.805039+0000","last_active":"2023-10-16T09:44:02.791775+0000","last_peered":"2023-10-16T09:44:02.791775+0000","last_clean":"2023-10-16T09:44:02.791775+0000","last_became_active":"2023-10-16T09:44:01.804777+0000","last_became_peered":"2023-10-16T09:44:01.804777+0000","last_unstale":"2023-10-16T09:44:02.791775+0000","last_undegraded":"2023-10-16T09:44:02.791775+0000","last_fullsized":"2023-10-16T09:44:02.791775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800545+0000","last_change":"2023-10-16T09:44:00.790368+0000","last_active":"2023-10-16T09:44:01.800545+0000","last_peered":"2023-10-16T09:44:01.800545+0000","last_clean":"2023-10-16T09:44:01.800545+0000","last_became_active":"2023-10-16T09:44:00.790030+0000","last_became_peered":"2023-10-16T09:44:00.790030+0000","last_unstale":"2023-10-16T09:44:01.800545+0000","last_undegraded":"2023-10-16T09:44:01.800545+0000","last_fullsized":"2023-10-16T09:44:01.800545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:19.134455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045870+0000","last_change":"2023-10-16T09:44:01.802895+0000","last_active":"2023-10-16T09:44:03.045870+0000","last_peered":"2023-10-16T09:44:03.045870+0000","last_clean":"2023-10-16T09:44:03.045870+0000","last_became_active":"2023-10-16T09:44:01.802637+0000","last_became_peered":"2023-10-16T09:44:01.802637+0000","last_unstale":"2023-10-16T09:44:03.045870+0000","last_undegraded":"2023-10-16T09:44:03.045870+0000","last_fullsized":"2023-10-16T09:44:03.045870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.798097+0000","last_change":"2023-10-16T09:44:00.793146+0000","last_active":"2023-10-16T09:44:01.798097+0000","last_peered":"2023-10-16T09:44:01.798097+0000","last_clean":"2023-10-16T09:44:01.798097+0000","last_became_active":"2023-10-16T09:44:00.792827+0000","last_became_peered":"2023-10-16T09:44:00.792827+0000","last_unstale":"2023-10-16T09:44:01.798097+0000","last_undegraded":"2023-10-16T09:44:01.798097+0000","last_fullsized":"2023-10-16T09:44:01.798097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.803000+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:01.803000+0000","last_peered":"2023-10-16T09:44:01.803000+0000","last_clean":"2023-10-16T09:44:01.803000+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:01.803000+0000","last_undegraded":"2023-10-16T09:44:01.803000+0000","last_fullsized":"2023-10-16T09:44:01.803000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.895844+0000","last_change":"2023-10-16T09:44:01.795766+0000","last_active":"2023-10-16T09:44:02.895844+0000","last_peered":"2023-10-16T09:44:02.895844+0000","last_clean":"2023-10-16T09:44:02.895844+0000","last_became_active":"2023-10-16T09:44:01.795429+0000","last_became_peered":"2023-10-16T09:44:01.795429+0000","last_unstale":"2023-10-16T09:44:02.895844+0000","last_undegraded":"2023-10-16T09:44:02.895844+0000","last_fullsized":"2023-10-16T09:44:02.895844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:24.556381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792852+0000","last_change":"2023-10-16T09:44:00.800131+0000","last_active":"2023-10-16T09:44:02.792852+0000","last_peered":"2023-10-16T09:44:02.792852+0000","last_clean":"2023-10-16T09:44:02.792852+0000","last_became_active":"2023-10-16T09:44:00.799868+0000","last_became_peered":"2023-10-16T09:44:00.799868+0000","last_unstale":"2023-10-16T09:44:02.792852+0000","last_undegraded":"2023-10-16T09:44:02.792852+0000","last_fullsized":"2023-10-16T09:44:02.792852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:52.613458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794561+0000","last_change":"2023-10-16T09:44:00.799180+0000","last_active":"2023-10-16T09:44:02.794561+0000","last_peered":"2023-10-16T09:44:02.794561+0000","last_clean":"2023-10-16T09:44:02.794561+0000","last_became_active":"2023-10-16T09:44:00.798901+0000","last_became_peered":"2023-10-16T09:44:00.798901+0000","last_unstale":"2023-10-16T09:44:02.794561+0000","last_undegraded":"2023-10-16T09:44:02.794561+0000","last_fullsized":"2023-10-16T09:44:02.794561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:10.784568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792018+0000","last_change":"2023-10-16T09:44:01.804991+0000","last_active":"2023-10-16T09:44:02.792018+0000","last_peered":"2023-10-16T09:44:02.792018+0000","last_clean":"2023-10-16T09:44:02.792018+0000","last_became_active":"2023-10-16T09:44:01.804694+0000","last_became_peered":"2023-10-16T09:44:01.804694+0000","last_unstale":"2023-10-16T09:44:02.792018+0000","last_undegraded":"2023-10-16T09:44:02.792018+0000","last_fullsized":"2023-10-16T09:44:02.792018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792039+0000","last_change":"2023-10-16T09:44:00.800156+0000","last_active":"2023-10-16T09:44:02.792039+0000","last_peered":"2023-10-16T09:44:02.792039+0000","last_clean":"2023-10-16T09:44:02.792039+0000","last_became_active":"2023-10-16T09:44:00.799895+0000","last_became_peered":"2023-10-16T09:44:00.799895+0000","last_unstale":"2023-10-16T09:44:02.792039+0000","last_undegraded":"2023-10-16T09:44:02.792039+0000","last_fullsized":"2023-10-16T09:44:02.792039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046312+0000","last_change":"2023-10-16T09:44:01.819462+0000","last_active":"2023-10-16T09:44:03.046312+0000","last_peered":"2023-10-16T09:44:03.046312+0000","last_clean":"2023-10-16T09:44:03.046312+0000","last_became_active":"2023-10-16T09:44:01.819196+0000","last_became_peered":"2023-10-16T09:44:01.819196+0000","last_unstale":"2023-10-16T09:44:03.046312+0000","last_undegraded":"2023-10-16T09:44:03.046312+0000","last_fullsized":"2023-10-16T09:44:03.046312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046068+0000","last_change":"2023-10-16T09:44:00.801223+0000","last_active":"2023-10-16T09:44:03.046068+0000","last_peered":"2023-10-16T09:44:03.046068+0000","last_clean":"2023-10-16T09:44:03.046068+0000","last_became_active":"2023-10-16T09:44:00.800978+0000","last_became_peered":"2023-10-16T09:44:00.800978+0000","last_unstale":"2023-10-16T09:44:03.046068+0000","last_undegraded":"2023-10-16T09:44:03.046068+0000","last_fullsized":"2023-10-16T09:44:03.046068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896394+0000","last_change":"2023-10-16T09:44:01.798851+0000","last_active":"2023-10-16T09:44:02.896394+0000","last_peered":"2023-10-16T09:44:02.896394+0000","last_clean":"2023-10-16T09:44:02.896394+0000","last_became_active":"2023-10-16T09:44:01.798475+0000","last_became_peered":"2023-10-16T09:44:01.798475+0000","last_unstale":"2023-10-16T09:44:02.896394+0000","last_undegraded":"2023-10-16T09:44:02.896394+0000","last_fullsized":"2023-10-16T09:44:02.896394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.792734+0000","last_change":"2023-10-16T09:44:00.799681+0000","last_active":"2023-10-16T09:44:02.792734+0000","last_peered":"2023-10-16T09:44:02.792734+0000","last_clean":"2023-10-16T09:44:02.792734+0000","last_became_active":"2023-10-16T09:44:00.799423+0000","last_became_peered":"2023-10-16T09:44:00.799423+0000","last_unstale":"2023-10-16T09:44:02.792734+0000","last_undegraded":"2023-10-16T09:44:02.792734+0000","last_fullsized":"2023-10-16T09:44:02.792734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046475+0000","last_change":"2023-10-16T09:44:01.806654+0000","last_active":"2023-10-16T09:44:03.046475+0000","last_peered":"2023-10-16T09:44:03.046475+0000","last_clean":"2023-10-16T09:44:03.046475+0000","last_became_active":"2023-10-16T09:44:01.805672+0000","last_became_peered":"2023-10-16T09:44:01.805672+0000","last_unstale":"2023-10-16T09:44:03.046475+0000","last_undegraded":"2023-10-16T09:44:03.046475+0000","last_fullsized":"2023-10-16T09:44:03.046475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:41.878512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790462+0000","last_change":"2023-10-16T09:44:00.797519+0000","last_active":"2023-10-16T09:44:01.790462+0000","last_peered":"2023-10-16T09:44:01.790462+0000","last_clean":"2023-10-16T09:44:01.790462+0000","last_became_active":"2023-10-16T09:44:00.797217+0000","last_became_peered":"2023-10-16T09:44:00.797217+0000","last_unstale":"2023-10-16T09:44:01.790462+0000","last_undegraded":"2023-10-16T09:44:01.790462+0000","last_fullsized":"2023-10-16T09:44:01.790462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:12.613526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045449+0000","last_change":"2023-10-16T09:44:01.798478+0000","last_active":"2023-10-16T09:44:03.045449+0000","last_peered":"2023-10-16T09:44:03.045449+0000","last_clean":"2023-10-16T09:44:03.045449+0000","last_became_active":"2023-10-16T09:44:01.798146+0000","last_became_peered":"2023-10-16T09:44:01.798146+0000","last_unstale":"2023-10-16T09:44:03.045449+0000","last_undegraded":"2023-10-16T09:44:03.045449+0000","last_fullsized":"2023-10-16T09:44:03.045449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:23.503690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.790384+0000","last_change":"2023-10-16T09:44:00.791453+0000","last_active":"2023-10-16T09:44:01.790384+0000","last_peered":"2023-10-16T09:44:01.790384+0000","last_clean":"2023-10-16T09:44:01.790384+0000","last_became_active":"2023-10-16T09:44:00.791178+0000","last_became_peered":"2023-10-16T09:44:00.791178+0000","last_unstale":"2023-10-16T09:44:01.790384+0000","last_undegraded":"2023-10-16T09:44:01.790384+0000","last_fullsized":"2023-10-16T09:44:01.790384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:05.334620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046137+0000","last_change":"2023-10-16T09:44:01.801224+0000","last_active":"2023-10-16T09:44:03.046137+0000","last_peered":"2023-10-16T09:44:03.046137+0000","last_clean":"2023-10-16T09:44:03.046137+0000","last_became_active":"2023-10-16T09:44:01.800946+0000","last_became_peered":"2023-10-16T09:44:01.800946+0000","last_unstale":"2023-10-16T09:44:03.046137+0000","last_undegraded":"2023-10-16T09:44:03.046137+0000","last_fullsized":"2023-10-16T09:44:03.046137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:39.233557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793527+0000","last_change":"2023-10-16T09:44:00.799160+0000","last_active":"2023-10-16T09:44:02.793527+0000","last_peered":"2023-10-16T09:44:02.793527+0000","last_clean":"2023-10-16T09:44:02.793527+0000","last_became_active":"2023-10-16T09:44:00.798874+0000","last_became_peered":"2023-10-16T09:44:00.798874+0000","last_unstale":"2023-10-16T09:44:02.793527+0000","last_undegraded":"2023-10-16T09:44:02.793527+0000","last_fullsized":"2023-10-16T09:44:02.793527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:31:05.724901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045858+0000","last_change":"2023-10-16T09:44:01.802604+0000","last_active":"2023-10-16T09:44:03.045858+0000","last_peered":"2023-10-16T09:44:03.045858+0000","last_clean":"2023-10-16T09:44:03.045858+0000","last_became_active":"2023-10-16T09:44:01.801907+0000","last_became_peered":"2023-10-16T09:44:01.801907+0000","last_unstale":"2023-10-16T09:44:03.045858+0000","last_undegraded":"2023-10-16T09:44:03.045858+0000","last_fullsized":"2023-10-16T09:44:03.045858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:19.587835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.800616+0000","last_change":"2023-10-16T09:44:00.796930+0000","last_active":"2023-10-16T09:44:01.800616+0000","last_peered":"2023-10-16T09:44:01.800616+0000","last_clean":"2023-10-16T09:44:01.800616+0000","last_became_active":"2023-10-16T09:44:00.796659+0000","last_became_peered":"2023-10-16T09:44:00.796659+0000","last_unstale":"2023-10-16T09:44:01.800616+0000","last_undegraded":"2023-10-16T09:44:01.800616+0000","last_fullsized":"2023-10-16T09:44:01.800616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:12:48.682590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794538+0000","last_change":"2023-10-16T09:44:01.820645+0000","last_active":"2023-10-16T09:44:02.794538+0000","last_peered":"2023-10-16T09:44:02.794538+0000","last_clean":"2023-10-16T09:44:02.794538+0000","last_became_active":"2023-10-16T09:44:01.820369+0000","last_became_peered":"2023-10-16T09:44:01.820369+0000","last_unstale":"2023-10-16T09:44:02.794538+0000","last_undegraded":"2023-10-16T09:44:02.794538+0000","last_fullsized":"2023-10-16T09:44:02.794538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:45.764214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.822381+0000","last_change":"2023-10-16T09:44:00.785075+0000","last_active":"2023-10-16T09:44:01.822381+0000","last_peered":"2023-10-16T09:44:01.822381+0000","last_clean":"2023-10-16T09:44:01.822381+0000","last_became_active":"2023-10-16T09:44:00.784807+0000","last_became_peered":"2023-10-16T09:44:00.784807+0000","last_unstale":"2023-10-16T09:44:01.822381+0000","last_undegraded":"2023-10-16T09:44:01.822381+0000","last_fullsized":"2023-10-16T09:44:01.822381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:58.332686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793947+0000","last_change":"2023-10-16T09:44:01.804040+0000","last_active":"2023-10-16T09:44:02.793947+0000","last_peered":"2023-10-16T09:44:02.793947+0000","last_clean":"2023-10-16T09:44:02.793947+0000","last_became_active":"2023-10-16T09:44:01.803782+0000","last_became_peered":"2023-10-16T09:44:01.803782+0000","last_unstale":"2023-10-16T09:44:02.793947+0000","last_undegraded":"2023-10-16T09:44:02.793947+0000","last_fullsized":"2023-10-16T09:44:02.793947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T09:44:01.819101+0000","last_change":"2023-10-16T09:44:00.790542+0000","last_active":"2023-10-16T09:44:01.819101+0000","last_peered":"2023-10-16T09:44:01.819101+0000","last_clean":"2023-10-16T09:44:01.819101+0000","last_became_active":"2023-10-16T09:44:00.790254+0000","last_became_peered":"2023-10-16T09:44:00.790254+0000","last_unstale":"2023-10-16T09:44:01.819101+0000","last_undegraded":"2023-10-16T09:44:01.819101+0000","last_fullsized":"2023-10-16T09:44:01.819101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:59.767702+0000","last_clean_scrub_stamp":"2023-10-16T09:43:59.767702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:33.165576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.794115+0000","last_change":"2023-10-16T09:44:01.820628+0000","last_active":"2023-10-16T09:44:02.794115+0000","last_peered":"2023-10-16T09:44:02.794115+0000","last_clean":"2023-10-16T09:44:02.794115+0000","last_became_active":"2023-10-16T09:44:01.820114+0000","last_became_peered":"2023-10-16T09:44:01.820114+0000","last_unstale":"2023-10-16T09:44:02.794115+0000","last_undegraded":"2023-10-16T09:44:02.794115+0000","last_fullsized":"2023-10-16T09:44:02.794115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:21.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046278+0000","last_change":"2023-10-16T09:44:01.805259+0000","last_active":"2023-10-16T09:44:03.046278+0000","last_peered":"2023-10-16T09:44:03.046278+0000","last_clean":"2023-10-16T09:44:03.046278+0000","last_became_active":"2023-10-16T09:44:01.804819+0000","last_became_peered":"2023-10-16T09:44:01.804819+0000","last_unstale":"2023-10-16T09:44:03.046278+0000","last_undegraded":"2023-10-16T09:44:03.046278+0000","last_fullsized":"2023-10-16T09:44:03.046278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:29.783957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045666+0000","last_change":"2023-10-16T09:44:01.808510+0000","last_active":"2023-10-16T09:44:03.045666+0000","last_peered":"2023-10-16T09:44:03.045666+0000","last_clean":"2023-10-16T09:44:03.045666+0000","last_became_active":"2023-10-16T09:44:01.808152+0000","last_became_peered":"2023-10-16T09:44:01.808152+0000","last_unstale":"2023-10-16T09:44:03.045666+0000","last_undegraded":"2023-10-16T09:44:03.045666+0000","last_fullsized":"2023-10-16T09:44:03.045666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:13.616924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793846+0000","last_change":"2023-10-16T09:44:01.808070+0000","last_active":"2023-10-16T09:44:02.793846+0000","last_peered":"2023-10-16T09:44:02.793846+0000","last_clean":"2023-10-16T09:44:02.793846+0000","last_became_active":"2023-10-16T09:44:01.807794+0000","last_became_peered":"2023-10-16T09:44:01.807794+0000","last_unstale":"2023-10-16T09:44:02.793846+0000","last_undegraded":"2023-10-16T09:44:02.793846+0000","last_fullsized":"2023-10-16T09:44:02.793846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896204+0000","last_change":"2023-10-16T09:44:01.796651+0000","last_active":"2023-10-16T09:44:02.896204+0000","last_peered":"2023-10-16T09:44:02.896204+0000","last_clean":"2023-10-16T09:44:02.896204+0000","last_became_active":"2023-10-16T09:44:01.796377+0000","last_became_peered":"2023-10-16T09:44:01.796377+0000","last_unstale":"2023-10-16T09:44:02.896204+0000","last_undegraded":"2023-10-16T09:44:02.896204+0000","last_fullsized":"2023-10-16T09:44:02.896204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046174+0000","last_change":"2023-10-16T09:44:01.803048+0000","last_active":"2023-10-16T09:44:03.046174+0000","last_peered":"2023-10-16T09:44:03.046174+0000","last_clean":"2023-10-16T09:44:03.046174+0000","last_became_active":"2023-10-16T09:44:01.802761+0000","last_became_peered":"2023-10-16T09:44:01.802761+0000","last_unstale":"2023-10-16T09:44:03.046174+0000","last_undegraded":"2023-10-16T09:44:03.046174+0000","last_fullsized":"2023-10-16T09:44:03.046174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:30.046112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.791913+0000","last_change":"2023-10-16T09:44:01.797818+0000","last_active":"2023-10-16T09:44:02.791913+0000","last_peered":"2023-10-16T09:44:02.791913+0000","last_clean":"2023-10-16T09:44:02.791913+0000","last_became_active":"2023-10-16T09:44:01.797537+0000","last_became_peered":"2023-10-16T09:44:01.797537+0000","last_unstale":"2023-10-16T09:44:02.791913+0000","last_undegraded":"2023-10-16T09:44:02.791913+0000","last_fullsized":"2023-10-16T09:44:02.791913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:39.488077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046247+0000","last_change":"2023-10-16T09:44:01.821446+0000","last_active":"2023-10-16T09:44:03.046247+0000","last_peered":"2023-10-16T09:44:03.046247+0000","last_clean":"2023-10-16T09:44:03.046247+0000","last_became_active":"2023-10-16T09:44:01.821191+0000","last_became_peered":"2023-10-16T09:44:01.821191+0000","last_unstale":"2023-10-16T09:44:03.046247+0000","last_undegraded":"2023-10-16T09:44:03.046247+0000","last_fullsized":"2023-10-16T09:44:03.046247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:46.081988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.046237+0000","last_change":"2023-10-16T09:44:01.801738+0000","last_active":"2023-10-16T09:44:03.046237+0000","last_peered":"2023-10-16T09:44:03.046237+0000","last_clean":"2023-10-16T09:44:03.046237+0000","last_became_active":"2023-10-16T09:44:01.801400+0000","last_became_peered":"2023-10-16T09:44:01.801400+0000","last_unstale":"2023-10-16T09:44:03.046237+0000","last_undegraded":"2023-10-16T09:44:03.046237+0000","last_fullsized":"2023-10-16T09:44:03.046237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:42:53.134955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793441+0000","last_change":"2023-10-16T09:44:01.797904+0000","last_active":"2023-10-16T09:44:02.793441+0000","last_peered":"2023-10-16T09:44:02.793441+0000","last_clean":"2023-10-16T09:44:02.793441+0000","last_became_active":"2023-10-16T09:44:01.797424+0000","last_became_peered":"2023-10-16T09:44:01.797424+0000","last_unstale":"2023-10-16T09:44:02.793441+0000","last_undegraded":"2023-10-16T09:44:02.793441+0000","last_fullsized":"2023-10-16T09:44:02.793441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045818+0000","last_change":"2023-10-16T09:44:01.802067+0000","last_active":"2023-10-16T09:44:03.045818+0000","last_peered":"2023-10-16T09:44:03.045818+0000","last_clean":"2023-10-16T09:44:03.045818+0000","last_became_active":"2023-10-16T09:44:01.801792+0000","last_became_peered":"2023-10-16T09:44:01.801792+0000","last_unstale":"2023-10-16T09:44:03.045818+0000","last_undegraded":"2023-10-16T09:44:03.045818+0000","last_fullsized":"2023-10-16T09:44:03.045818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:51.244162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.793644+0000","last_change":"2023-10-16T09:44:01.819606+0000","last_active":"2023-10-16T09:44:02.793644+0000","last_peered":"2023-10-16T09:44:02.793644+0000","last_clean":"2023-10-16T09:44:02.793644+0000","last_became_active":"2023-10-16T09:44:01.819328+0000","last_became_peered":"2023-10-16T09:44:01.819328+0000","last_unstale":"2023-10-16T09:44:02.793644+0000","last_undegraded":"2023-10-16T09:44:02.793644+0000","last_fullsized":"2023-10-16T09:44:02.793644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:14:17.322240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:02.896102+0000","last_change":"2023-10-16T09:44:01.818209+0000","last_active":"2023-10-16T09:44:02.896102+0000","last_peered":"2023-10-16T09:44:02.896102+0000","last_clean":"2023-10-16T09:44:02.896102+0000","last_became_active":"2023-10-16T09:44:01.817921+0000","last_became_peered":"2023-10-16T09:44:01.817921+0000","last_unstale":"2023-10-16T09:44:02.896102+0000","last_undegraded":"2023-10-16T09:44:02.896102+0000","last_fullsized":"2023-10-16T09:44:02.896102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T09:44:03.045707+0000","last_change":"2023-10-16T09:44:01.802316+0000","last_active":"2023-10-16T09:44:03.045707+0000","last_peered":"2023-10-16T09:44:03.045707+0000","last_clean":"2023-10-16T09:44:03.045707+0000","last_became_active":"2023-10-16T09:44:01.802036+0000","last_became_peered":"2023-10-16T09:44:01.802036+0000","last_unstale":"2023-10-16T09:44:03.045707+0000","last_undegraded":"2023-10-16T09:44:03.045707+0000","last_fullsized":"2023-10-16T09:44:03.045707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:00.771100+0000","last_clean_scrub_stamp":"2023-10-16T09:44:00.771100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:46:39.144484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":669814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5090000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3260000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4349999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4160000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2189999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4649999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":78990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1140000000000008}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:44:11.088 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T09:44:11.088 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-16T09:44:11.089 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-16T09:44:11.089 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-16T09:44:11.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.240+0000 7ff371cfc700 1 Processor -- start 2023-10-16T09:44:11.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.240+0000 7ff371cfc700 1 -- start start 2023-10-16T09:44:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.241+0000 7ff371cfc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 0x7ff36c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.241+0000 7ff371cfc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff36c0cb0c0 0x7ff36c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.241+0000 7ff371cfc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36c06ed00 con 0x7ff36c15eab0 2023-10-16T09:44:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.241+0000 7ff371cfc700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff36c0d02d0 con 0x7ff36c0cb0c0 2023-10-16T09:44:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.241+0000 7ff371cfc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c15ad40 con 0x7ff36c15e6d0 2023-10-16T09:44:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.241+0000 7ff36affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff36c15eab0 0x7ff36c15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47592/0 (socket says 172.21.15.181:47592) 2023-10-16T09:44:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.241+0000 7ff36affd700 1 -- 172.21.15.181:0/3534671230 learned_addr learned my addr 172.21.15.181:0/3534671230 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 --2- 172.21.15.181:0/3534671230 >> v2:172.21.15.190:3300/0 conn(0x7ff36c0cb0c0 0x7ff36c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36b7fe700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 0x7ff36c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395312190 0 0) 0x7ff36c06ed00 con 0x7ff36c15eab0 2023-10-16T09:44:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3540035b0 con 0x7ff36c15eab0 2023-10-16T09:44:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1874124123 0 0) 0x7ff3540035b0 con 0x7ff36c15eab0 2023-10-16T09:44:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 msgr2=0x7ff36c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 0x7ff36c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 >> v2:172.21.15.190:3300/0 conn(0x7ff36c0cb0c0 msgr2=0x7ff36c15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 --2- 172.21.15.181:0/3534671230 >> v2:172.21.15.190:3300/0 conn(0x7ff36c0cb0c0 0x7ff36c15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c06e0f0 con 0x7ff36c15eab0 2023-10-16T09:44:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff360001670 con 0x7ff36c15eab0 2023-10-16T09:44:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 >> v1:172.21.15.181:6789/0 conn(0x7ff36c15eab0 legacy=0x7ff36c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 0x7ff354003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff354003f80 0x7ff354004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff354004d90 0x7ff354005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0d02d0 con 0x7ff354004d90 2023-10-16T09:44:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c15ad40 con 0x7ff354003f80 2023-10-16T09:44:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35400dc20 con 0x7ff36c15e6d0 2023-10-16T09:44:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff354004d90 0x7ff354005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36b7fe700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 0x7ff354003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36affd700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff354003f80 0x7ff354004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 -- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 msgr2=0x7ff354003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15e6d0 0x7ff354003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 -- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff354003f80 msgr2=0x7ff354004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff354003f80 0x7ff354004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 -- 172.21.15.181:0/3534671230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540035b0 con 0x7ff354004d90 2023-10-16T09:44:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.242+0000 7ff36bfff700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff354004d90 0x7ff354005210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff364009fc0 tx=0x7ff354005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2651262519faa04c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36400ebd0 con 0x7ff354004d90 2023-10-16T09:44:11.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36400f040 con 0x7ff354004d90 2023-10-16T09:44:11.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff36a7fc700 1 -- 172.21.15.181:0/3534671230 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364014480 con 0x7ff354004d90 2023-10-16T09:44:11.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff371cfc700 1 -- 172.21.15.181:0/3534671230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff354004d90 msgr2=0x7ff354005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff371cfc700 1 --2- 172.21.15.181:0/3534671230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff354004d90 0x7ff354005210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff364009fc0 tx=0x7ff354005880 comp rx=0 tx=0).stop 2023-10-16T09:44:11.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff36bfff700 1 -- 172.21.15.181:0/3534671230 reap_dead start 2023-10-16T09:44:11.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff371cfc700 1 -- 172.21.15.181:0/3534671230 shutdown_connections 2023-10-16T09:44:11.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff371cfc700 1 -- 172.21.15.181:0/3534671230 >> 172.21.15.181:0/3534671230 conn(0x7ff36c0d04a0 msgr2=0x7ff36c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:11.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff371cfc700 1 -- 172.21.15.181:0/3534671230 shutdown_connections 2023-10-16T09:44:11.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.243+0000 7ff371cfc700 1 -- 172.21.15.181:0/3534671230 wait complete. 2023-10-16T09:44:11.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.244+0000 7ff371cfc700 1 Processor -- start 2023-10-16T09:44:11.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff371cfc700 1 -- start start 2023-10-16T09:44:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff371cfc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff36c0b57b0 0x7ff36c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff371cfc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15b690 0x7ff36c0c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff371cfc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff36c0c6310 0x7ff36c0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff371cfc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c15bb10 con 0x7ff36c0c6310 2023-10-16T09:44:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff371cfc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0c5940 con 0x7ff36c0b57b0 2023-10-16T09:44:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff371cfc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0c4d30 con 0x7ff36c15b690 2023-10-16T09:44:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff36bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff36c0c6310 0x7ff36c0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff36bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff36c0c6310 0x7ff36c0c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36528/0 (socket says 172.21.15.181:36528) 2023-10-16T09:44:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff36bfff700 1 -- 172.21.15.181:0/1088633744 learned_addr learned my addr 172.21.15.181:0/1088633744 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff36b7fe700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff36c0b57b0 0x7ff36c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.245+0000 7ff36affd700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15b690 0x7ff36c0c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff36bfff700 1 -- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15b690 msgr2=0x7ff36c0c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff36bfff700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15b690 0x7ff36c0c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff36bfff700 1 -- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff36c0b57b0 msgr2=0x7ff36c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff36bfff700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff36c0b57b0 0x7ff36c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff36bfff700 1 -- 172.21.15.181:0/1088633744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c0c5010 con 0x7ff36c0c6310 2023-10-16T09:44:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff36bfff700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff36c0c6310 0x7ff36c0c6790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff364016000 tx=0x7ff364009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff353fff700 1 -- 172.21.15.181:0/1088633744 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364007000 con 0x7ff36c0c6310 2023-10-16T09:44:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff353fff700 1 -- 172.21.15.181:0/1088633744 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364015a80 con 0x7ff36c0c6310 2023-10-16T09:44:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c0c52f0 con 0x7ff36c0c6310 2023-10-16T09:44:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff353fff700 1 -- 172.21.15.181:0/1088633744 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364015e90 con 0x7ff36c0c6310 2023-10-16T09:44:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.246+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c15bf40 con 0x7ff36c0c6310 2023-10-16T09:44:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.247+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c067d20 con 0x7ff36c0c6310 2023-10-16T09:44:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.247+0000 7ff353fff700 1 -- 172.21.15.181:0/1088633744 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff36401d420 con 0x7ff36c0c6310 2023-10-16T09:44:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.248+0000 7ff353fff700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff35806af00 0x7ff35806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.248+0000 7ff353fff700 1 -- 172.21.15.181:0/1088633744 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff36408ab70 con 0x7ff36c0c6310 2023-10-16T09:44:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.248+0000 7ff36b7fe700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff35806af00 0x7ff35806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.248+0000 7ff36b7fe700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff35806af00 0x7ff35806d3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009c40 tx=0x7ff35c0091c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.252+0000 7ff353fff700 1 -- 172.21.15.181:0/1088633744 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff364055e90 con 0x7ff36c0c6310 2023-10-16T09:44:11.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.488+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff36c06de30 con 0x7ff36c0c6310 2023-10-16T09:44:11.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.488+0000 7ff353fff700 1 -- 172.21.15.181:0/1088633744 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff3640594b0 con 0x7ff36c0c6310 2023-10-16T09:44:11.490 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:11.490 INFO:teuthology.orchestra.run.smithi181.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-16T09:44:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.490+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff35806af00 msgr2=0x7ff35806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.490+0000 7ff371cfc700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff35806af00 0x7ff35806d3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009c40 tx=0x7ff35c0091c0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.490+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff36c0c6310 msgr2=0x7ff36c0c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.490+0000 7ff371cfc700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff36c0c6310 0x7ff36c0c6790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff364016000 tx=0x7ff364009750 comp rx=0 tx=0).stop 2023-10-16T09:44:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.490+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 shutdown_connections 2023-10-16T09:44:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.491+0000 7ff371cfc700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff35806af00 0x7ff35806d3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.491+0000 7ff371cfc700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff36c0c6310 0x7ff36c0c6790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.491+0000 7ff371cfc700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff36c0b57b0 0x7ff36c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.491+0000 7ff371cfc700 1 --2- 172.21.15.181:0/1088633744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff36c15b690 0x7ff36c0c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.491+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 >> 172.21.15.181:0/1088633744 conn(0x7ff36c0d04a0 msgr2=0x7ff36c0ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:11.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.491+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 shutdown_connections 2023-10-16T09:44:11.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.491+0000 7ff371cfc700 1 -- 172.21.15.181:0/1088633744 wait complete. 2023-10-16T09:44:11.503 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-16T09:44:11.504 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:11.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.622+0000 7f1d1ff6f700 1 Processor -- start 2023-10-16T09:44:11.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.623+0000 7f1d1ff6f700 1 -- start start 2023-10-16T09:44:11.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.623+0000 7f1d1ff6f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d1815b920 0x7f1d180ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.623+0000 7f1d1ff6f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.623+0000 7f1d1ff6f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d180b43d0 con 0x7f1d180ce610 2023-10-16T09:44:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.623+0000 7f1d1ff6f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d180d1970 con 0x7f1d1815b920 2023-10-16T09:44:11.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.623+0000 7f1d1ff6f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180d1c50 con 0x7f1d180cc2e0 2023-10-16T09:44:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1dd0b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1d1815b920 0x7f1d180ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1e50c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1d50a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1d180ce610 0x7f1d180cbbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59784/0 (socket says 172.21.15.181:59784) 2023-10-16T09:44:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1d50a700 1 -- 172.21.15.181:0/1151094494 learned_addr learned my addr 172.21.15.181:0/1151094494 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:11.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1e50c700 1 -- 172.21.15.181:0/1151094494 >> v2:172.21.15.190:3300/0 conn(0x7f1d1815b920 msgr2=0x7f1d180ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:11.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1e50c700 1 --2- 172.21.15.181:0/1151094494 >> v2:172.21.15.190:3300/0 conn(0x7f1d1815b920 0x7f1d180ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1e50c700 1 -- 172.21.15.181:0/1151094494 >> v1:172.21.15.190:6789/0 conn(0x7f1d180ce610 legacy=0x7f1d180cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1e50c700 1 -- 172.21.15.181:0/1151094494 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180d1f30 con 0x7f1d180cc2e0 2023-10-16T09:44:11.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1dd0b700 1 --2- 172.21.15.181:0/1151094494 >> v2:172.21.15.190:3300/0 conn(0x7f1d1815b920 0x7f1d180ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:11.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1e50c700 1 --2- 172.21.15.181:0/1151094494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180b4860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009d90 tx=0x7f1d1000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa6799b845d7c47 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1cd09700 1 -- 172.21.15.181:0/1151094494 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000e070 con 0x7f1d180cc2e0 2023-10-16T09:44:11.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1cd09700 1 -- 172.21.15.181:0/1151094494 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1000bb20 con 0x7f1d180cc2e0 2023-10-16T09:44:11.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.624+0000 7f1d1cd09700 1 -- 172.21.15.181:0/1151094494 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10012430 con 0x7f1d180cc2e0 2023-10-16T09:44:11.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/1151094494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 msgr2=0x7f1d180b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/1151094494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180b4860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009d90 tx=0x7f1d1000b330 comp rx=0 tx=0).stop 2023-10-16T09:44:11.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/1151094494 shutdown_connections 2023-10-16T09:44:11.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/1151094494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180b4860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/1151094494 >> v2:172.21.15.190:3300/0 conn(0x7f1d1815b920 0x7f1d180ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/1151094494 >> 172.21.15.181:0/1151094494 conn(0x7f1d180665a0 msgr2=0x7f1d180669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:11.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/1151094494 shutdown_connections 2023-10-16T09:44:11.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.625+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/1151094494 wait complete. 2023-10-16T09:44:11.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.626+0000 7f1d1ff6f700 1 Processor -- start 2023-10-16T09:44:11.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.626+0000 7f1d1ff6f700 1 -- start start 2023-10-16T09:44:11.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1ff6f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1ff6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d180ce610 0x7f1d180c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1ff6f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1815b920 0x7f1d180c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1ff6f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1806d5e0 con 0x7f1d180ce610 2023-10-16T09:44:11.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1ff6f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1806d8c0 con 0x7f1d1815b920 2023-10-16T09:44:11.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1ff6f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180c61a0 con 0x7f1d180cc2e0 2023-10-16T09:44:11.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d180ce610 0x7f1d180c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d180ce610 0x7f1d180c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36538/0 (socket says 172.21.15.181:36538) 2023-10-16T09:44:11.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 -- 172.21.15.181:0/236217779 learned_addr learned my addr 172.21.15.181:0/236217779 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:11.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1e50c700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1815b920 0x7f1d180c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1dd0b700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 -- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 msgr2=0x7f1d180cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 -- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1815b920 msgr2=0x7f1d180c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1815b920 0x7f1d180c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 -- 172.21.15.181:0/236217779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080094a0 con 0x7f1d180ce610 2023-10-16T09:44:11.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.627+0000 7f1d1d50a700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d180ce610 0x7f1d180c8a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c520 tx=0x7f1d1400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.628+0000 7f1d0e7fc700 1 -- 172.21.15.181:0/236217779 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14005d70 con 0x7f1d180ce610 2023-10-16T09:44:11.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.628+0000 7f1d0e7fc700 1 -- 172.21.15.181:0/236217779 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d140070d0 con 0x7f1d180ce610 2023-10-16T09:44:11.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.628+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d10009a70 con 0x7f1d180ce610 2023-10-16T09:44:11.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.628+0000 7f1d0e7fc700 1 -- 172.21.15.181:0/236217779 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14007440 con 0x7f1d180ce610 2023-10-16T09:44:11.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.628+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d180c6760 con 0x7f1d180ce610 2023-10-16T09:44:11.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.629+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d180c68f0 con 0x7f1d180ce610 2023-10-16T09:44:11.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.629+0000 7f1d0e7fc700 1 -- 172.21.15.181:0/236217779 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d140075a0 con 0x7f1d180ce610 2023-10-16T09:44:11.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.629+0000 7f1d0e7fc700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1cfc06af00 0x7f1cfc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.629+0000 7f1d0e7fc700 1 -- 172.21.15.181:0/236217779 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d14088830 con 0x7f1d180ce610 2023-10-16T09:44:11.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.629+0000 7f1d1dd0b700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1cfc06af00 0x7f1cfc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.630+0000 7f1d1dd0b700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1cfc06af00 0x7f1cfc06d3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009cd0 tx=0x7f1d08009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.634+0000 7f1d0e7fc700 1 -- 172.21.15.181:0/236217779 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1d14053990 con 0x7f1d180ce610 2023-10-16T09:44:11.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.828+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d180b30d0 con 0x7f1d180ce610 2023-10-16T09:44:11.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.829+0000 7f1d0e7fc700 1 -- 172.21.15.181:0/236217779 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f1d14017090 con 0x7f1d180ce610 2023-10-16T09:44:11.831 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:11.832 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:11.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1cfc06af00 msgr2=0x7f1cfc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1cfc06af00 0x7f1cfc06d3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009cd0 tx=0x7f1d08009210 comp rx=0 tx=0).stop 2023-10-16T09:44:11.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d180ce610 msgr2=0x7f1d180c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d180ce610 0x7f1d180c8a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c520 tx=0x7f1d1400c830 comp rx=0 tx=0).stop 2023-10-16T09:44:11.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 shutdown_connections 2023-10-16T09:44:11.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1cfc06af00 0x7f1cfc06d3c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d180cc2e0 0x7f1d180cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1815b920 0x7f1d180c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 --2- 172.21.15.181:0/236217779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d180ce610 0x7f1d180c8a10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 >> 172.21.15.181:0/236217779 conn(0x7f1d180665a0 msgr2=0x7f1d180b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:11.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 shutdown_connections 2023-10-16T09:44:11.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.832+0000 7f1d1ff6f700 1 -- 172.21.15.181:0/236217779 wait complete. 2023-10-16T09:44:11.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:11.847 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.968+0000 7f3e51655700 1 Processor -- start 2023-10-16T09:44:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.968+0000 7f3e51655700 1 -- start start 2023-10-16T09:44:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e51655700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e4c0b2020 0x7f3e4c06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e51655700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e51655700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c15b550 con 0x7f3e4c0cb6a0 2023-10-16T09:44:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e51655700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c06d2f0 con 0x7f3e4c0b2020 2023-10-16T09:44:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e51655700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0b3ed0 con 0x7f3e4c0cc1b0 2023-10-16T09:44:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e4c0b2020 0x7f3e4c06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e4c0b2020 0x7f3e4c06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36546/0 (socket says 172.21.15.181:36546) 2023-10-16T09:44:11.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 -- 172.21.15.181:0/4034026433 learned_addr learned my addr 172.21.15.181:0/4034026433 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:11.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4b7fe700 1 --2- 172.21.15.181:0/4034026433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 -- 172.21.15.181:0/4034026433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 msgr2=0x7f3e4c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:11.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 --2- 172.21.15.181:0/4034026433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 -- 172.21.15.181:0/4034026433 >> v1:172.21.15.190:6789/0 conn(0x7f3e4c0cb6a0 legacy=0x7f3e4c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 -- 172.21.15.181:0/4034026433 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c15b830 con 0x7f3e4c0b2020 2023-10-16T09:44:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e4a7fc700 1 --2- 172.21.15.181:0/4034026433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c06d710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000bbc0 tx=0x7f3e4000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb10e58238685c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e49ffb700 1 -- 172.21.15.181:0/4034026433 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000dd50 con 0x7f3e4c0b2020 2023-10-16T09:44:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e49ffb700 1 -- 172.21.15.181:0/4034026433 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40010470 con 0x7f3e4c0b2020 2023-10-16T09:44:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.969+0000 7f3e49ffb700 1 -- 172.21.15.181:0/4034026433 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40010800 con 0x7f3e4c0b2020 2023-10-16T09:44:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 -- 172.21.15.181:0/4034026433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 msgr2=0x7f3e4c06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 --2- 172.21.15.181:0/4034026433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c06d710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000bbc0 tx=0x7f3e4000bed0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 -- 172.21.15.181:0/4034026433 shutdown_connections 2023-10-16T09:44:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 --2- 172.21.15.181:0/4034026433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 --2- 172.21.15.181:0/4034026433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c06d710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 -- 172.21.15.181:0/4034026433 >> 172.21.15.181:0/4034026433 conn(0x7f3e4c06a8a0 msgr2=0x7f3e4c06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 -- 172.21.15.181:0/4034026433 shutdown_connections 2023-10-16T09:44:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.970+0000 7f3e51655700 1 -- 172.21.15.181:0/4034026433 wait complete. 2023-10-16T09:44:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.971+0000 7f3e51655700 1 Processor -- start 2023-10-16T09:44:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.971+0000 7f3e51655700 1 -- start start 2023-10-16T09:44:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e51655700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e51655700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c0cb6a0 0x7f3e4c0c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e51655700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e51655700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c155e00 con 0x7f3e4c0b2020 2023-10-16T09:44:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e51655700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1560e0 con 0x7f3e4c0cb6a0 2023-10-16T09:44:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e51655700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1563c0 con 0x7f3e4c0cc1b0 2023-10-16T09:44:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c1555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36550/0 (socket says 172.21.15.181:36550) 2023-10-16T09:44:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 -- 172.21.15.181:0/2446058997 learned_addr learned my addr 172.21.15.181:0/2446058997 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4a7fc700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c0cb6a0 0x7f3e4c0c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4b7fe700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 -- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 msgr2=0x7f3e4c0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 -- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c0cb6a0 msgr2=0x7f3e4c0c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c0cb6a0 0x7f3e4c0c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:11.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.972+0000 7f3e4affd700 1 -- 172.21.15.181:0/2446058997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4000b7f0 con 0x7f3e4c0b2020 2023-10-16T09:44:11.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.973+0000 7f3e4affd700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c1555b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00c4e0 tx=0x7f3e3c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.973+0000 7f3e337fe700 1 -- 172.21.15.181:0/2446058997 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0052d0 con 0x7f3e4c0b2020 2023-10-16T09:44:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.973+0000 7f3e337fe700 1 -- 172.21.15.181:0/2446058997 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c005430 con 0x7f3e4c0b2020 2023-10-16T09:44:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.973+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c1566a0 con 0x7f3e4c0b2020 2023-10-16T09:44:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.973+0000 7f3e337fe700 1 -- 172.21.15.181:0/2446058997 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0038c0 con 0x7f3e4c0b2020 2023-10-16T09:44:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.973+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c156ea0 con 0x7f3e4c0b2020 2023-10-16T09:44:11.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.974+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c156bc0 con 0x7f3e4c0b2020 2023-10-16T09:44:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.974+0000 7f3e337fe700 1 -- 172.21.15.181:0/2446058997 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e3c003a20 con 0x7f3e4c0b2020 2023-10-16T09:44:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.974+0000 7f3e337fe700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f3e3806af00 0x7f3e3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.975+0000 7f3e4a7fc700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f3e3806af00 0x7f3e3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.975+0000 7f3e337fe700 1 -- 172.21.15.181:0/2446058997 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e3c088960 con 0x7f3e4c0b2020 2023-10-16T09:44:11.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.975+0000 7f3e4a7fc700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f3e3806af00 0x7f3e3806d3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000bf10 tx=0x7f3e40013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:11.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:11.979+0000 7f3e337fe700 1 -- 172.21.15.181:0/2446058997 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3e4c156bc0 con 0x7f3e4c0b2020 2023-10-16T09:44:12.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.170+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e4c156bc0 con 0x7f3e4c0b2020 2023-10-16T09:44:12.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.170+0000 7f3e337fe700 1 -- 172.21.15.181:0/2446058997 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e4c156bc0 con 0x7f3e4c0b2020 2023-10-16T09:44:12.173 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:12.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f3e3806af00 msgr2=0x7f3e3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f3e3806af00 0x7f3e3806d3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000bf10 tx=0x7f3e40013000 comp rx=0 tx=0).stop 2023-10-16T09:44:12.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 msgr2=0x7f3e4c1555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c1555b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00c4e0 tx=0x7f3e3c00c7f0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 shutdown_connections 2023-10-16T09:44:12.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f3e3806af00 0x7f3e3806d3c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e4c0cc1b0 0x7f3e4c0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e4c0b2020 0x7f3e4c1555b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 --2- 172.21.15.181:0/2446058997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e4c0cb6a0 0x7f3e4c0c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 >> 172.21.15.181:0/2446058997 conn(0x7f3e4c06a8a0 msgr2=0x7f3e4c15e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:12.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 shutdown_connections 2023-10-16T09:44:12.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.173+0000 7f3e51655700 1 -- 172.21.15.181:0/2446058997 wait complete. 2023-10-16T09:44:12.187 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:12.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.308+0000 7ff682d72700 1 Processor -- start 2023-10-16T09:44:12.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.308+0000 7ff682d72700 1 -- start start 2023-10-16T09:44:12.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff682d72700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0cdd00 0x7ff67c06f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff682d72700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff67c0666a0 0x7ff67c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff682d72700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0ca300 con 0x7ff67c0ce0e0 2023-10-16T09:44:12.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff682d72700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c06e5e0 con 0x7ff67c0666a0 2023-10-16T09:44:12.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff682d72700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0ca580 con 0x7ff67c0cdd00 2023-10-16T09:44:12.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff68130f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff67c0ce0e0 0x7ff67c06ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47640/0 (socket says 172.21.15.181:47640) 2023-10-16T09:44:12.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff68130f700 1 -- 172.21.15.181:0/2412540752 learned_addr learned my addr 172.21.15.181:0/2412540752 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:12.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff680b0e700 1 --2- 172.21.15.181:0/2412540752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0cdd00 0x7ff67c06f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff67bfff700 1 --2- 172.21.15.181:0/2412540752 >> v2:172.21.15.190:3300/0 conn(0x7ff67c0666a0 0x7ff67c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff67b7fe700 1 -- 172.21.15.181:0/2412540752 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334422892 0 0) 0x7ff67c0ca300 con 0x7ff67c0ce0e0 2023-10-16T09:44:12.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff67b7fe700 1 -- 172.21.15.181:0/2412540752 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c0035b0 con 0x7ff67c0ce0e0 2023-10-16T09:44:12.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff680b0e700 1 -- 172.21.15.181:0/2412540752 >> v2:172.21.15.190:3300/0 conn(0x7ff67c0666a0 msgr2=0x7ff67c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:12.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff680b0e700 1 --2- 172.21.15.181:0/2412540752 >> v2:172.21.15.190:3300/0 conn(0x7ff67c0666a0 0x7ff67c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff680b0e700 1 -- 172.21.15.181:0/2412540752 >> v1:172.21.15.181:6789/0 conn(0x7ff67c0ce0e0 legacy=0x7ff67c06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff680b0e700 1 -- 172.21.15.181:0/2412540752 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c0ca860 con 0x7ff67c0cdd00 2023-10-16T09:44:12.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.309+0000 7ff680b0e700 1 --2- 172.21.15.181:0/2412540752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0cdd00 0x7ff67c06f5f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff670009840 tx=0x7ff670009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af8c58218fe506 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:12.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff67b7fe700 1 -- 172.21.15.181:0/2412540752 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670011830 con 0x7ff67c0cdd00 2023-10-16T09:44:12.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff67b7fe700 1 -- 172.21.15.181:0/2412540752 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670011990 con 0x7ff67c0cdd00 2023-10-16T09:44:12.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff67b7fe700 1 -- 172.21.15.181:0/2412540752 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670011d00 con 0x7ff67c0cdd00 2023-10-16T09:44:12.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 -- 172.21.15.181:0/2412540752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0cdd00 msgr2=0x7ff67c06f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 --2- 172.21.15.181:0/2412540752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0cdd00 0x7ff67c06f5f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff670009840 tx=0x7ff670009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:12.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 -- 172.21.15.181:0/2412540752 shutdown_connections 2023-10-16T09:44:12.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 --2- 172.21.15.181:0/2412540752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0cdd00 0x7ff67c06f5f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 --2- 172.21.15.181:0/2412540752 >> v2:172.21.15.190:3300/0 conn(0x7ff67c0666a0 0x7ff67c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 -- 172.21.15.181:0/2412540752 >> 172.21.15.181:0/2412540752 conn(0x7ff67c0cc730 msgr2=0x7ff67c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:12.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 -- 172.21.15.181:0/2412540752 shutdown_connections 2023-10-16T09:44:12.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.310+0000 7ff682d72700 1 -- 172.21.15.181:0/2412540752 wait complete. 2023-10-16T09:44:12.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.311+0000 7ff682d72700 1 Processor -- start 2023-10-16T09:44:12.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.311+0000 7ff682d72700 1 -- start start 2023-10-16T09:44:12.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff682d72700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0666a0 0x7ff67c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff682d72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff67c0cdd00 0x7ff67c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff682d72700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0ce0e0 0x7ff67c0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff682d72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c15d6c0 con 0x7ff67c0cdd00 2023-10-16T09:44:12.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff682d72700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c4290 con 0x7ff67c0666a0 2023-10-16T09:44:12.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff682d72700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c1e7e60 con 0x7ff67c0ce0e0 2023-10-16T09:44:12.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff67bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff67c0cdd00 0x7ff67c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff67bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff67c0cdd00 0x7ff67c0c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36566/0 (socket says 172.21.15.181:36566) 2023-10-16T09:44:12.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff680b0e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0666a0 0x7ff67c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff67bfff700 1 -- 172.21.15.181:0/1481711795 learned_addr learned my addr 172.21.15.181:0/1481711795 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:12.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.312+0000 7ff68130f700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0ce0e0 0x7ff67c0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff67bfff700 1 -- 172.21.15.181:0/1481711795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0ce0e0 msgr2=0x7ff67c0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff67bfff700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0ce0e0 0x7ff67c0c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff67bfff700 1 -- 172.21.15.181:0/1481711795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0666a0 msgr2=0x7ff67c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff67bfff700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0666a0 0x7ff67c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff67bfff700 1 -- 172.21.15.181:0/1481711795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700094a0 con 0x7ff67c0cdd00 2023-10-16T09:44:12.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff67bfff700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff67c0cdd00 0x7ff67c0c8710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff66800e8a0 tx=0x7ff66800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:12.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff6797fa700 1 -- 172.21.15.181:0/1481711795 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668009590 con 0x7ff67c0cdd00 2023-10-16T09:44:12.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff6797fa700 1 -- 172.21.15.181:0/1481711795 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff668013680 con 0x7ff67c0cdd00 2023-10-16T09:44:12.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c1e8140 con 0x7ff67c0cdd00 2023-10-16T09:44:12.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff6797fa700 1 -- 172.21.15.181:0/1481711795 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6680139f0 con 0x7ff67c0cdd00 2023-10-16T09:44:12.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.313+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c0c3eb0 con 0x7ff67c0cdd00 2023-10-16T09:44:12.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.314+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c0cd870 con 0x7ff67c0cdd00 2023-10-16T09:44:12.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.314+0000 7ff6797fa700 1 -- 172.21.15.181:0/1481711795 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff668013b50 con 0x7ff67c0cdd00 2023-10-16T09:44:12.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.315+0000 7ff6797fa700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff66006af00 0x7ff66006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.315+0000 7ff680b0e700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff66006af00 0x7ff66006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.315+0000 7ff6797fa700 1 -- 172.21.15.181:0/1481711795 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6680894e0 con 0x7ff67c0cdd00 2023-10-16T09:44:12.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.315+0000 7ff680b0e700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff66006af00 0x7ff66006d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff670000c00 tx=0x7ff67001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:12.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.319+0000 7ff6797fa700 1 -- 172.21.15.181:0/1481711795 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6680544f0 con 0x7ff67c0cdd00 2023-10-16T09:44:12.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.511+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff67c0af900 con 0x7ff67c0cdd00 2023-10-16T09:44:12.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.511+0000 7ff6797fa700 1 -- 172.21.15.181:0/1481711795 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff668057b10 con 0x7ff67c0cdd00 2023-10-16T09:44:12.514 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:12.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff66006af00 msgr2=0x7ff66006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff66006af00 0x7ff66006d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff670000c00 tx=0x7ff67001c040 comp rx=0 tx=0).stop 2023-10-16T09:44:12.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff67c0cdd00 msgr2=0x7ff67c0c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff67c0cdd00 0x7ff67c0c8710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff66800e8a0 tx=0x7ff66800ebb0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 shutdown_connections 2023-10-16T09:44:12.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff66006af00 0x7ff66006d3c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67c0ce0e0 0x7ff67c0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff67c0666a0 0x7ff67c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 --2- 172.21.15.181:0/1481711795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff67c0cdd00 0x7ff67c0c8710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 >> 172.21.15.181:0/1481711795 conn(0x7ff67c0cc730 msgr2=0x7ff67c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:12.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 shutdown_connections 2023-10-16T09:44:12.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.513+0000 7ff682d72700 1 -- 172.21.15.181:0/1481711795 wait complete. 2023-10-16T09:44:12.528 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:12.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.649+0000 7f4d3374e700 1 Processor -- start 2023-10-16T09:44:12.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.649+0000 7f4d3374e700 1 -- start start 2023-10-16T09:44:12.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d3374e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d2c0cc6e0 0x7f4d2c071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d3374e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d2c0ce680 0x7f4d2c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d3374e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2c0ce1f0 con 0x7f4d2c0b2010 2023-10-16T09:44:12.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d3374e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c0ccb90 con 0x7f4d2c0cc6e0 2023-10-16T09:44:12.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d3374e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c072120 con 0x7f4d2c0ce680 2023-10-16T09:44:12.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d2c0cc6e0 0x7f4d2c071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d2c0cc6e0 0x7f4d2c071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36574/0 (socket says 172.21.15.181:36574) 2023-10-16T09:44:12.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d314ea700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4d2c0b2010 0x7f4d2c0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47650/0 (socket says 172.21.15.181:47650) 2023-10-16T09:44:12.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 -- 172.21.15.181:0/3663718995 learned_addr learned my addr 172.21.15.181:0/3663718995 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:12.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d31ceb700 1 --2- 172.21.15.181:0/3663718995 >> v2:172.21.15.190:3300/0 conn(0x7f4d2c0ce680 0x7f4d2c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 -- 172.21.15.181:0/3663718995 >> v2:172.21.15.190:3300/0 conn(0x7f4d2c0ce680 msgr2=0x7f4d2c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:12.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 --2- 172.21.15.181:0/3663718995 >> v2:172.21.15.190:3300/0 conn(0x7f4d2c0ce680 0x7f4d2c0ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 -- 172.21.15.181:0/3663718995 >> v1:172.21.15.181:6789/0 conn(0x7f4d2c0b2010 legacy=0x7f4d2c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 -- 172.21.15.181:0/3663718995 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0ccda0 con 0x7f4d2c0cc6e0 2023-10-16T09:44:12.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d23fff700 1 -- 172.21.15.181:0/3663718995 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639617870 0 0) 0x7f4d2c0ce1f0 con 0x7f4d2c0b2010 2023-10-16T09:44:12.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d30ce9700 1 --2- 172.21.15.181:0/3663718995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0cc6e0 0x7f4d2c071be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c071a30 tx=0x7f4d1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb06052cc481923 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:12.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d23fff700 1 -- 172.21.15.181:0/3663718995 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c011830 con 0x7f4d2c0cc6e0 2023-10-16T09:44:12.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d23fff700 1 -- 172.21.15.181:0/3663718995 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c011990 con 0x7f4d2c0cc6e0 2023-10-16T09:44:12.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.650+0000 7f4d23fff700 1 -- 172.21.15.181:0/3663718995 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c011d20 con 0x7f4d2c0cc6e0 2023-10-16T09:44:12.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 -- 172.21.15.181:0/3663718995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0cc6e0 msgr2=0x7f4d2c071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 --2- 172.21.15.181:0/3663718995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0cc6e0 0x7f4d2c071be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c071a30 tx=0x7f4d1c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 -- 172.21.15.181:0/3663718995 shutdown_connections 2023-10-16T09:44:12.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 --2- 172.21.15.181:0/3663718995 >> v2:172.21.15.190:3300/0 conn(0x7f4d2c0ce680 0x7f4d2c0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 --2- 172.21.15.181:0/3663718995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0cc6e0 0x7f4d2c071be0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 -- 172.21.15.181:0/3663718995 >> 172.21.15.181:0/3663718995 conn(0x7f4d2c0cd010 msgr2=0x7f4d2c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:12.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 -- 172.21.15.181:0/3663718995 shutdown_connections 2023-10-16T09:44:12.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.651+0000 7f4d3374e700 1 -- 172.21.15.181:0/3663718995 wait complete. 2023-10-16T09:44:12.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.652+0000 7f4d3374e700 1 Processor -- start 2023-10-16T09:44:12.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.652+0000 7f4d3374e700 1 -- start start 2023-10-16T09:44:12.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d3374e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0b2010 0x7f4d2c1503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d3374e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d2c0cc6e0 0x7f4d2c1e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d3374e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2c0ce680 0x7f4d2c1e7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d3374e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c150e30 con 0x7f4d2c0b2010 2023-10-16T09:44:12.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d3374e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c1508f0 con 0x7f4d2c0cc6e0 2023-10-16T09:44:12.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d3374e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0c7400 con 0x7f4d2c0ce680 2023-10-16T09:44:12.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0b2010 0x7f4d2c1503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0b2010 0x7f4d2c1503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36580/0 (socket says 172.21.15.181:36580) 2023-10-16T09:44:12.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 -- 172.21.15.181:0/206938438 learned_addr learned my addr 172.21.15.181:0/206938438 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d30ce9700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d2c0cc6e0 0x7f4d2c1e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d31ceb700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2c0ce680 0x7f4d2c1e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 -- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2c0ce680 msgr2=0x7f4d2c1e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2c0ce680 0x7f4d2c1e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 -- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d2c0cc6e0 msgr2=0x7f4d2c1e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d2c0cc6e0 0x7f4d2c1e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 -- 172.21.15.181:0/206938438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c009550 con 0x7f4d2c0b2010 2023-10-16T09:44:12.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.653+0000 7f4d314ea700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0b2010 0x7f4d2c1503b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c550 tx=0x7f4d2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:12.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.654+0000 7f4d21ffb700 1 -- 172.21.15.181:0/206938438 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28004c80 con 0x7f4d2c0b2010 2023-10-16T09:44:12.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.654+0000 7f4d21ffb700 1 -- 172.21.15.181:0/206938438 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d28004e00 con 0x7f4d2c0b2010 2023-10-16T09:44:12.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.654+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c0ca430 con 0x7f4d2c0b2010 2023-10-16T09:44:12.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.654+0000 7f4d21ffb700 1 -- 172.21.15.181:0/206938438 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28007310 con 0x7f4d2c0b2010 2023-10-16T09:44:12.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.654+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c0c9670 con 0x7f4d2c0b2010 2023-10-16T09:44:12.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.655+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c0b6510 con 0x7f4d2c0b2010 2023-10-16T09:44:12.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.655+0000 7f4d21ffb700 1 -- 172.21.15.181:0/206938438 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4d28007470 con 0x7f4d2c0b2010 2023-10-16T09:44:12.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.655+0000 7f4d21ffb700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d1006af00 0x7f4d1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.655+0000 7f4d21ffb700 1 -- 172.21.15.181:0/206938438 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d28089240 con 0x7f4d2c0b2010 2023-10-16T09:44:12.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.655+0000 7f4d30ce9700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d1006af00 0x7f4d1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.656+0000 7f4d30ce9700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d1006af00 0x7f4d1006d3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c014040 tx=0x7f4d1c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:12.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.660+0000 7f4d21ffb700 1 -- 172.21.15.181:0/206938438 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4d28054380 con 0x7f4d2c0b2010 2023-10-16T09:44:12.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.855+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d2c0ceca0 con 0x7f4d2c0b2010 2023-10-16T09:44:12.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.855+0000 7f4d21ffb700 1 -- 172.21.15.181:0/206938438 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d280579a0 con 0x7f4d2c0b2010 2023-10-16T09:44:12.858 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:12.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d1006af00 msgr2=0x7f4d1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d1006af00 0x7f4d1006d3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c014040 tx=0x7f4d1c008030 comp rx=0 tx=0).stop 2023-10-16T09:44:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0b2010 msgr2=0x7f4d2c1503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0b2010 0x7f4d2c1503b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c550 tx=0x7f4d2800c910 comp rx=0 tx=0).stop 2023-10-16T09:44:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 shutdown_connections 2023-10-16T09:44:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4d1006af00 0x7f4d1006d3c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d2c0ce680 0x7f4d2c1e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d2c0b2010 0x7f4d2c1503b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 --2- 172.21.15.181:0/206938438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d2c0cc6e0 0x7f4d2c1e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 >> 172.21.15.181:0/206938438 conn(0x7f4d2c0cd010 msgr2=0x7f4d2c159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:12.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 shutdown_connections 2023-10-16T09:44:12.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.857+0000 7f4d3374e700 1 -- 172.21.15.181:0/206938438 wait complete. 2023-10-16T09:44:12.872 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:12.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.990+0000 7ff4f0d08700 1 Processor -- start 2023-10-16T09:44:12.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.990+0000 7ff4f0d08700 1 -- start start 2023-10-16T09:44:12.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4f0d08700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4ec15dfd0 0x7ff4ec0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4f0d08700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4ec0cd470 0x7ff4ec0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:12.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4f0d08700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec06f870 con 0x7ff4ec0af450 2023-10-16T09:44:12.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4f0d08700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0cde40 con 0x7ff4ec15dfd0 2023-10-16T09:44:12.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4f0d08700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0d1160 con 0x7ff4ec0cd470 2023-10-16T09:44:12.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4ec15dfd0 0x7ff4ec0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4ec15dfd0 0x7ff4ec0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36586/0 (socket says 172.21.15.181:36586) 2023-10-16T09:44:12.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 -- 172.21.15.181:0/663426581 learned_addr learned my addr 172.21.15.181:0/663426581 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:12.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ead9d700 1 --2- 172.21.15.181:0/663426581 >> v2:172.21.15.190:3300/0 conn(0x7ff4ec0cd470 0x7ff4ec0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:12.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 -- 172.21.15.181:0/663426581 >> v2:172.21.15.190:3300/0 conn(0x7ff4ec0cd470 msgr2=0x7ff4ec0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:12.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 --2- 172.21.15.181:0/663426581 >> v2:172.21.15.190:3300/0 conn(0x7ff4ec0cd470 0x7ff4ec0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:12.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 -- 172.21.15.181:0/663426581 >> v1:172.21.15.190:6789/0 conn(0x7ff4ec0af450 legacy=0x7ff4ec06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:12.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 -- 172.21.15.181:0/663426581 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0cc8e0 con 0x7ff4ec15dfd0 2023-10-16T09:44:12.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4ea59c700 1 --2- 172.21.15.181:0/663426581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec15dfd0 0x7ff4ec0af9a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0af1c0 tx=0x7ff4dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9343351bf5d9d76 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:12.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4e959a700 1 -- 172.21.15.181:0/663426581 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc005e30 con 0x7ff4ec15dfd0 2023-10-16T09:44:12.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4e959a700 1 -- 172.21.15.181:0/663426581 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc006030 con 0x7ff4ec15dfd0 2023-10-16T09:44:12.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.991+0000 7ff4e959a700 1 -- 172.21.15.181:0/663426581 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0063e0 con 0x7ff4ec15dfd0 2023-10-16T09:44:12.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 -- 172.21.15.181:0/663426581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec15dfd0 msgr2=0x7ff4ec0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/663426581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec15dfd0 0x7ff4ec0af9a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0af1c0 tx=0x7ff4dc009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 -- 172.21.15.181:0/663426581 shutdown_connections 2023-10-16T09:44:13.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/663426581 >> v2:172.21.15.190:3300/0 conn(0x7ff4ec0cd470 0x7ff4ec0cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/663426581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec15dfd0 0x7ff4ec0af9a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 -- 172.21.15.181:0/663426581 >> 172.21.15.181:0/663426581 conn(0x7ff4ec0d04a0 msgr2=0x7ff4ec0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:13.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 -- 172.21.15.181:0/663426581 shutdown_connections 2023-10-16T09:44:13.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.992+0000 7ff4f0d08700 1 -- 172.21.15.181:0/663426581 wait complete. 2023-10-16T09:44:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.993+0000 7ff4f0d08700 1 Processor -- start 2023-10-16T09:44:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.993+0000 7ff4f0d08700 1 -- start start 2023-10-16T09:44:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4f0d08700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4ec0af450 0x7ff4ec0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4f0d08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec0cd470 0x7ff4ec0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4f0d08700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4ec15dfd0 0x7ff4ec0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4f0d08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec1552d0 con 0x7ff4ec0cd470 2023-10-16T09:44:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4f0d08700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec1f2f90 con 0x7ff4ec0af450 2023-10-16T09:44:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4f0d08700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0c6e70 con 0x7ff4ec15dfd0 2023-10-16T09:44:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec0cd470 0x7ff4ec0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec0cd470 0x7ff4ec0b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36594/0 (socket says 172.21.15.181:36594) 2023-10-16T09:44:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 -- 172.21.15.181:0/2052183435 learned_addr learned my addr 172.21.15.181:0/2052183435 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4ea59c700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4ec0af450 0x7ff4ec0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4ead9d700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4ec15dfd0 0x7ff4ec0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 -- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4ec15dfd0 msgr2=0x7ff4ec0c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4ec15dfd0 0x7ff4ec0c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 -- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4ec0af450 msgr2=0x7ff4ec0b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4ec0af450 0x7ff4ec0b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.994+0000 7ff4e9d9b700 1 -- 172.21.15.181:0/2052183435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc009500 con 0x7ff4ec0cd470 2023-10-16T09:44:13.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.995+0000 7ff4e9d9b700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec0cd470 0x7ff4ec0b0830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b4a0 tx=0x7ff4e000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.995+0000 7ff4daffd700 1 -- 172.21.15.181:0/2052183435 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0011830 con 0x7ff4ec0cd470 2023-10-16T09:44:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.995+0000 7ff4daffd700 1 -- 172.21.15.181:0/2052183435 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e0011990 con 0x7ff4ec0cd470 2023-10-16T09:44:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.995+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec0b0ee0 con 0x7ff4ec0cd470 2023-10-16T09:44:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.995+0000 7ff4daffd700 1 -- 172.21.15.181:0/2052183435 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0011d40 con 0x7ff4ec0cd470 2023-10-16T09:44:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.995+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec0b14a0 con 0x7ff4ec0cd470 2023-10-16T09:44:13.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.996+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec0cb610 con 0x7ff4ec0cd470 2023-10-16T09:44:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.996+0000 7ff4daffd700 1 -- 172.21.15.181:0/2052183435 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff4e001a400 con 0x7ff4ec0cd470 2023-10-16T09:44:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.996+0000 7ff4daffd700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff4cc06af00 0x7ff4cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.997+0000 7ff4daffd700 1 -- 172.21.15.181:0/2052183435 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4e00891b0 con 0x7ff4ec0cd470 2023-10-16T09:44:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.997+0000 7ff4ea59c700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff4cc06af00 0x7ff4cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:12.997+0000 7ff4ea59c700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff4cc06af00 0x7ff4cc06d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0af1c0 tx=0x7ff4dc008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.001+0000 7ff4daffd700 1 -- 172.21.15.181:0/2052183435 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff4e0054260 con 0x7ff4ec0cd470 2023-10-16T09:44:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.235+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4ec067d20 con 0x7ff4ec0cd470 2023-10-16T09:44:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.236+0000 7ff4daffd700 1 -- 172.21.15.181:0/2052183435 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7ff4e0057880 con 0x7ff4ec0cd470 2023-10-16T09:44:13.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:13.239 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:13.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff4cc06af00 msgr2=0x7ff4cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff4cc06af00 0x7ff4cc06d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0af1c0 tx=0x7ff4dc008070 comp rx=0 tx=0).stop 2023-10-16T09:44:13.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec0cd470 msgr2=0x7ff4ec0b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec0cd470 0x7ff4ec0b0830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b4a0 tx=0x7ff4e000b7b0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 shutdown_connections 2023-10-16T09:44:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff4cc06af00 0x7ff4cc06d3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4ec15dfd0 0x7ff4ec0c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4ec0af450 0x7ff4ec0b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 --2- 172.21.15.181:0/2052183435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ec0cd470 0x7ff4ec0b0830 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 >> 172.21.15.181:0/2052183435 conn(0x7ff4ec0d04a0 msgr2=0x7ff4ec0ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 shutdown_connections 2023-10-16T09:44:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.238+0000 7ff4f0d08700 1 -- 172.21.15.181:0/2052183435 wait complete. 2023-10-16T09:44:13.243 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:13.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:13.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.370+0000 7fc2b5ef4700 1 Processor -- start 2023-10-16T09:44:13.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.370+0000 7fc2b5ef4700 1 -- start start 2023-10-16T09:44:13.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2b5ef4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc2b015b990 0x7fc2b00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2b5ef4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2b5ef4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b00cc510 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2b5ef4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b00d24f0 con 0x7fc2b015b990 2023-10-16T09:44:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2b5ef4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00cc7f0 con 0x7fc2b00d0370 2023-10-16T09:44:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2aeffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc2b00cd2d0 0x7fc2b00cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47680/0 (socket says 172.21.15.181:47680) 2023-10-16T09:44:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2aeffd700 1 -- 172.21.15.181:0/2779076492 learned_addr learned my addr 172.21.15.181:0/2779076492 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2af7fe700 1 --2- 172.21.15.181:0/2779076492 >> v2:172.21.15.190:3300/0 conn(0x7fc2b015b990 0x7fc2b00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2affff700 1 --2- 172.21.15.181:0/2779076492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2ae7fc700 1 -- 172.21.15.181:0/2779076492 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427015412 0 0) 0x7fc2b00cc510 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.371+0000 7fc2ae7fc700 1 -- 172.21.15.181:0/2779076492 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2980035b0 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2affff700 1 -- 172.21.15.181:0/2779076492 >> v2:172.21.15.190:3300/0 conn(0x7fc2b015b990 msgr2=0x7fc2b00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2affff700 1 --2- 172.21.15.181:0/2779076492 >> v2:172.21.15.190:3300/0 conn(0x7fc2b015b990 0x7fc2b00ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2affff700 1 -- 172.21.15.181:0/2779076492 >> v1:172.21.15.181:6789/0 conn(0x7fc2b00cd2d0 legacy=0x7fc2b00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2ae7fc700 1 -- 172.21.15.181:0/2779076492 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 227516913 0 0) 0x7fc2980035b0 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2affff700 1 -- 172.21.15.181:0/2779076492 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b00ccad0 con 0x7fc2b00d0370 2023-10-16T09:44:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2affff700 1 --2- 172.21.15.181:0/2779076492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00d1fb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009d90 tx=0x7fc2a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c1587b462b2c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2ae7fc700 1 -- 172.21.15.181:0/2779076492 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a800e070 con 0x7fc2b00d0370 2023-10-16T09:44:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2ae7fc700 1 -- 172.21.15.181:0/2779076492 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a8003a20 con 0x7fc2b00d0370 2023-10-16T09:44:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2ae7fc700 1 -- 172.21.15.181:0/2779076492 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0001670 con 0x7fc2b00d0370 2023-10-16T09:44:13.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/2779076492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 msgr2=0x7fc2b00d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.372+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/2779076492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00d1fb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009d90 tx=0x7fc2a800b330 comp rx=0 tx=0).stop 2023-10-16T09:44:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.373+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/2779076492 shutdown_connections 2023-10-16T09:44:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.373+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/2779076492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00d1fb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.373+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/2779076492 >> v2:172.21.15.190:3300/0 conn(0x7fc2b015b990 0x7fc2b00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.373+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/2779076492 >> 172.21.15.181:0/2779076492 conn(0x7fc2b009a9a0 msgr2=0x7fc2b00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:13.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.373+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/2779076492 shutdown_connections 2023-10-16T09:44:13.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.373+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/2779076492 wait complete. 2023-10-16T09:44:13.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.374+0000 7fc2b5ef4700 1 Processor -- start 2023-10-16T09:44:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.374+0000 7fc2b5ef4700 1 -- start start 2023-10-16T09:44:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2b5ef4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b00cd2d0 0x7fc2b0154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2b5ef4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2b5ef4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b015b990 0x7fc2b00ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2b5ef4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00ca040 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2b5ef4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b01554b0 con 0x7fc2b015b990 2023-10-16T09:44:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2b5ef4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00c9a30 con 0x7fc2b00d0370 2023-10-16T09:44:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b00cd2d0 0x7fc2b0154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b00cd2d0 0x7fc2b0154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36606/0 (socket says 172.21.15.181:36606) 2023-10-16T09:44:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.375+0000 7fc2af7fe700 1 -- 172.21.15.181:0/3406444776 learned_addr learned my addr 172.21.15.181:0/3406444776 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2aeffd700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2affff700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b015b990 0x7fc2b00ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2af7fe700 1 -- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 msgr2=0x7fc2b00cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2af7fe700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2af7fe700 1 -- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b015b990 msgr2=0x7fc2b00ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2af7fe700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b015b990 0x7fc2b00ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2af7fe700 1 -- 172.21.15.181:0/3406444776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8009a70 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2af7fe700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b00cd2d0 0x7fc2b0154f70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400e900 tx=0x7fc2a400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc297fff700 1 -- 172.21.15.181:0/3406444776 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4009650 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc297fff700 1 -- 172.21.15.181:0/3406444776 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a400c850 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc297fff700 1 -- 172.21.15.181:0/3406444776 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400cbc0 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b0157f30 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.376+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b00c9cd0 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.377+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b00c9890 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.377+0000 7fc297fff700 1 -- 172.21.15.181:0/3406444776 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc2a400cd20 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.378+0000 7fc297fff700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc29c06af00 0x7fc29c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.378+0000 7fc2aeffd700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc29c06af00 0x7fc29c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.378+0000 7fc297fff700 1 -- 172.21.15.181:0/3406444776 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2a4089ad0 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.378+0000 7fc2aeffd700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc29c06af00 0x7fc29c06d3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0001eb0 tx=0x7fc2a0001c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.382+0000 7fc297fff700 1 -- 172.21.15.181:0/3406444776 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc2b00c9890 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.578+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2b00c9890 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.579+0000 7fc297fff700 1 -- 172.21.15.181:0/3406444776 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fc2b00c9890 con 0x7fc2b00cd2d0 2023-10-16T09:44:13.582 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:13.582 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:13.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc29c06af00 msgr2=0x7fc29c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc29c06af00 0x7fc29c06d3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0001eb0 tx=0x7fc2a0001c60 comp rx=0 tx=0).stop 2023-10-16T09:44:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b00cd2d0 msgr2=0x7fc2b0154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b00cd2d0 0x7fc2b0154f70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400e900 tx=0x7fc2a400ecc0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 shutdown_connections 2023-10-16T09:44:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc29c06af00 0x7fc29c06d3c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b015b990 0x7fc2b00ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b00cd2d0 0x7fc2b0154f70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 --2- 172.21.15.181:0/3406444776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2b00d0370 0x7fc2b00cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 >> 172.21.15.181:0/3406444776 conn(0x7fc2b009a9a0 msgr2=0x7fc2b00cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 shutdown_connections 2023-10-16T09:44:13.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.581+0000 7fc2b5ef4700 1 -- 172.21.15.181:0/3406444776 wait complete. 2023-10-16T09:44:13.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:13.596 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.718+0000 7feb2b32b700 1 Processor -- start 2023-10-16T09:44:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.718+0000 7feb2b32b700 1 -- start start 2023-10-16T09:44:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb2b32b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb240cc6e0 0x7feb24071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb2b32b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240ce680 0x7feb240ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb2b32b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb240ce1f0 con 0x7feb240b2010 2023-10-16T09:44:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb2b32b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb240ccb90 con 0x7feb240cc6e0 2023-10-16T09:44:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb2b32b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb24072120 con 0x7feb240ce680 2023-10-16T09:44:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb240cc6e0 0x7feb24071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb240cc6e0 0x7feb24071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36616/0 (socket says 172.21.15.181:36616) 2023-10-16T09:44:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 -- 172.21.15.181:0/839358898 learned_addr learned my addr 172.21.15.181:0/839358898 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb298c8700 1 --2- 172.21.15.181:0/839358898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240ce680 0x7feb240ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 -- 172.21.15.181:0/839358898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240ce680 msgr2=0x7feb240ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 --2- 172.21.15.181:0/839358898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240ce680 0x7feb240ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 -- 172.21.15.181:0/839358898 >> v1:172.21.15.190:6789/0 conn(0x7feb240b2010 legacy=0x7feb240b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 -- 172.21.15.181:0/839358898 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240ccda0 con 0x7feb240cc6e0 2023-10-16T09:44:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb288c6700 1 --2- 172.21.15.181:0/839358898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240cc6e0 0x7feb24071be0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb1400bb90 tx=0x7feb1400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bb1ecba6db0f0e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.719+0000 7feb1bfff700 1 -- 172.21.15.181:0/839358898 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1400dd30 con 0x7feb240cc6e0 2023-10-16T09:44:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb1bfff700 1 -- 172.21.15.181:0/839358898 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1400de90 con 0x7feb240cc6e0 2023-10-16T09:44:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb1bfff700 1 -- 172.21.15.181:0/839358898 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14010690 con 0x7feb240cc6e0 2023-10-16T09:44:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 -- 172.21.15.181:0/839358898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240cc6e0 msgr2=0x7feb24071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 --2- 172.21.15.181:0/839358898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240cc6e0 0x7feb24071be0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb1400bb90 tx=0x7feb1400bea0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 -- 172.21.15.181:0/839358898 shutdown_connections 2023-10-16T09:44:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 --2- 172.21.15.181:0/839358898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240ce680 0x7feb240ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 --2- 172.21.15.181:0/839358898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240cc6e0 0x7feb24071be0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 -- 172.21.15.181:0/839358898 >> 172.21.15.181:0/839358898 conn(0x7feb240cd010 msgr2=0x7feb240cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 -- 172.21.15.181:0/839358898 shutdown_connections 2023-10-16T09:44:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.720+0000 7feb2b32b700 1 -- 172.21.15.181:0/839358898 wait complete. 2023-10-16T09:44:13.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.721+0000 7feb2b32b700 1 Processor -- start 2023-10-16T09:44:13.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.721+0000 7feb2b32b700 1 -- start start 2023-10-16T09:44:13.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb2b32b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb240b2010 0x7feb240b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb2b32b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240cc6e0 0x7feb240c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb2b32b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240ce680 0x7feb24151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb2b32b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240c85e0 con 0x7feb240ce680 2023-10-16T09:44:13.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb2b32b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240c88c0 con 0x7feb240b2010 2023-10-16T09:44:13.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb2b32b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb240c8ba0 con 0x7feb240cc6e0 2023-10-16T09:44:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240ce680 0x7feb24151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240ce680 0x7feb24151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36622/0 (socket says 172.21.15.181:36622) 2023-10-16T09:44:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 -- 172.21.15.181:0/470832055 learned_addr learned my addr 172.21.15.181:0/470832055 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb290c7700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb240b2010 0x7feb240b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb288c6700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240cc6e0 0x7feb240c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 -- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240cc6e0 msgr2=0x7feb240c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240cc6e0 0x7feb240c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 -- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb240b2010 msgr2=0x7feb240b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb240b2010 0x7feb240b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.722+0000 7feb298c8700 1 -- 172.21.15.181:0/470832055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1400b7f0 con 0x7feb240ce680 2023-10-16T09:44:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.723+0000 7feb298c8700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240ce680 0x7feb24151630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00efc0 tx=0x7feb1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.723+0000 7feb19ffb700 1 -- 172.21.15.181:0/470832055 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c009ea0 con 0x7feb240ce680 2023-10-16T09:44:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.723+0000 7feb19ffb700 1 -- 172.21.15.181:0/470832055 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c00f040 con 0x7feb240ce680 2023-10-16T09:44:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.723+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb240c8e80 con 0x7feb240ce680 2023-10-16T09:44:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.723+0000 7feb19ffb700 1 -- 172.21.15.181:0/470832055 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c00ced0 con 0x7feb240ce680 2023-10-16T09:44:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.723+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb240c9ed0 con 0x7feb240ce680 2023-10-16T09:44:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.724+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb240c91f0 con 0x7feb240ce680 2023-10-16T09:44:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.724+0000 7feb19ffb700 1 -- 172.21.15.181:0/470832055 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feb1c014480 con 0x7feb240ce680 2023-10-16T09:44:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.725+0000 7feb19ffb700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7feb0806af00 0x7feb0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.725+0000 7feb290c7700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7feb0806af00 0x7feb0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.725+0000 7feb19ffb700 1 -- 172.21.15.181:0/470832055 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb1c089da0 con 0x7feb240ce680 2023-10-16T09:44:13.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.725+0000 7feb290c7700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7feb0806af00 0x7feb0806d3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feb20007fd0 tx=0x7feb20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:13.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.729+0000 7feb19ffb700 1 -- 172.21.15.181:0/470832055 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feb240c91f0 con 0x7feb240ce680 2023-10-16T09:44:13.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.926+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb240c91f0 con 0x7feb240ce680 2023-10-16T09:44:13.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.927+0000 7feb19ffb700 1 -- 172.21.15.181:0/470832055 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb240c91f0 con 0x7feb240ce680 2023-10-16T09:44:13.929 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:13.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7feb0806af00 msgr2=0x7feb0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7feb0806af00 0x7feb0806d3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feb20007fd0 tx=0x7feb20009040 comp rx=0 tx=0).stop 2023-10-16T09:44:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240ce680 msgr2=0x7feb24151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240ce680 0x7feb24151630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00efc0 tx=0x7feb1c00c3a0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 shutdown_connections 2023-10-16T09:44:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7feb0806af00 0x7feb0806d3c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb240ce680 0x7feb24151630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb240b2010 0x7feb240b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 --2- 172.21.15.181:0/470832055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb240cc6e0 0x7feb240c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 >> 172.21.15.181:0/470832055 conn(0x7feb240cd010 msgr2=0x7feb240b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 shutdown_connections 2023-10-16T09:44:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:13.929+0000 7feb2b32b700 1 -- 172.21.15.181:0/470832055 wait complete. 2023-10-16T09:44:13.943 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:14.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.066+0000 7f430b80c700 1 Processor -- start 2023-10-16T09:44:14.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.066+0000 7f430b80c700 1 -- start start 2023-10-16T09:44:14.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f430b80c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f430415da70 0x7f43040ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f430b80c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43040ce6a0 0x7f430406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f430b80c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43040cef00 con 0x7f4304072760 2023-10-16T09:44:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f430b80c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4304072b40 con 0x7f43040ce6a0 2023-10-16T09:44:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f430b80c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43040cec20 con 0x7f430415da70 2023-10-16T09:44:14.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43040ce6a0 0x7f430406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43040ce6a0 0x7f430406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36630/0 (socket says 172.21.15.181:36630) 2023-10-16T09:44:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 -- 172.21.15.181:0/3533808094 learned_addr learned my addr 172.21.15.181:0/3533808094 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f43095a8700 1 --2- 172.21.15.181:0/3533808094 >> v2:172.21.15.190:3300/0 conn(0x7f430415da70 0x7f43040ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 -- 172.21.15.181:0/3533808094 >> v2:172.21.15.190:3300/0 conn(0x7f430415da70 msgr2=0x7f43040ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:14.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 --2- 172.21.15.181:0/3533808094 >> v2:172.21.15.190:3300/0 conn(0x7f430415da70 0x7f43040ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 -- 172.21.15.181:0/3533808094 >> v1:172.21.15.190:6789/0 conn(0x7f4304072760 legacy=0x7f43041615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:14.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 -- 172.21.15.181:0/3533808094 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304071c80 con 0x7f43040ce6a0 2023-10-16T09:44:14.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f4308da7700 1 --2- 172.21.15.181:0/3533808094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f430406a8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f43040675e0 tx=0x7f42f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13cf8370ea73b9a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.067+0000 7f42fbfff700 1 -- 172.21.15.181:0/3533808094 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f400b870 con 0x7f43040ce6a0 2023-10-16T09:44:14.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f42fbfff700 1 -- 172.21.15.181:0/3533808094 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f400b9d0 con 0x7f43040ce6a0 2023-10-16T09:44:14.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f42fbfff700 1 -- 172.21.15.181:0/3533808094 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f400bd60 con 0x7f43040ce6a0 2023-10-16T09:44:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 -- 172.21.15.181:0/3533808094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 msgr2=0x7f430406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 --2- 172.21.15.181:0/3533808094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f430406a8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f43040675e0 tx=0x7f42f4009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 -- 172.21.15.181:0/3533808094 shutdown_connections 2023-10-16T09:44:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 --2- 172.21.15.181:0/3533808094 >> v2:172.21.15.190:3300/0 conn(0x7f430415da70 0x7f43040ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 --2- 172.21.15.181:0/3533808094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f430406a8a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 -- 172.21.15.181:0/3533808094 >> 172.21.15.181:0/3533808094 conn(0x7f43040cd010 msgr2=0x7f43040cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 -- 172.21.15.181:0/3533808094 shutdown_connections 2023-10-16T09:44:14.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.068+0000 7f430b80c700 1 -- 172.21.15.181:0/3533808094 wait complete. 2023-10-16T09:44:14.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.069+0000 7f430b80c700 1 Processor -- start 2023-10-16T09:44:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.069+0000 7f430b80c700 1 -- start start 2023-10-16T09:44:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f430b80c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4304072760 0x7f4304144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f430b80c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f43040b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f430b80c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430415da70 0x7f43040c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f430b80c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43040c6500 con 0x7f43040ce6a0 2023-10-16T09:44:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f430b80c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43040c5600 con 0x7f4304072760 2023-10-16T09:44:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f430b80c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43041450d0 con 0x7f430415da70 2023-10-16T09:44:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f43040b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f43040b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36636/0 (socket says 172.21.15.181:36636) 2023-10-16T09:44:14.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 -- 172.21.15.181:0/280520900 learned_addr learned my addr 172.21.15.181:0/280520900 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:14.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4309da9700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430415da70 0x7f43040c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f43095a8700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4304072760 0x7f4304144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 -- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430415da70 msgr2=0x7f43040c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430415da70 0x7f43040c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 -- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4304072760 msgr2=0x7f4304144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4304072760 0x7f4304144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4308da7700 1 -- 172.21.15.181:0/280520900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4009550 con 0x7f43040ce6a0 2023-10-16T09:44:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f43095a8700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4304072760 0x7f4304144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.070+0000 7f4309da9700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430415da70 0x7f43040c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:14.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.071+0000 7f4308da7700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f43040b5a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f42f4000c00 tx=0x7f42f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.071+0000 7f42f9ffb700 1 -- 172.21.15.181:0/280520900 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4004540 con 0x7f43040ce6a0 2023-10-16T09:44:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.071+0000 7f42f9ffb700 1 -- 172.21.15.181:0/280520900 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f40036a0 con 0x7f43040ce6a0 2023-10-16T09:44:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.071+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43040c5880 con 0x7f43040ce6a0 2023-10-16T09:44:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.071+0000 7f42f9ffb700 1 -- 172.21.15.181:0/280520900 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4003a30 con 0x7f43040ce6a0 2023-10-16T09:44:14.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.071+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43040c5e40 con 0x7f43040ce6a0 2023-10-16T09:44:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.072+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43040c6640 con 0x7f43040ce6a0 2023-10-16T09:44:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.073+0000 7f42f9ffb700 1 -- 172.21.15.181:0/280520900 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f42f4021070 con 0x7f43040ce6a0 2023-10-16T09:44:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.073+0000 7f42f9ffb700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f42e806af00 0x7f42e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.073+0000 7f42f9ffb700 1 -- 172.21.15.181:0/280520900 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42f408afe0 con 0x7f43040ce6a0 2023-10-16T09:44:14.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.073+0000 7f43095a8700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f42e806af00 0x7f42e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.074+0000 7f43095a8700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f42e806af00 0x7f42e806d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43000099f0 tx=0x7f4300008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.077+0000 7f42f9ffb700 1 -- 172.21.15.181:0/280520900 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f42f4056320 con 0x7f43040ce6a0 2023-10-16T09:44:14.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.267+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43040c6640 con 0x7f43040ce6a0 2023-10-16T09:44:14.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.267+0000 7f42f9ffb700 1 -- 172.21.15.181:0/280520900 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43040c6640 con 0x7f43040ce6a0 2023-10-16T09:44:14.270 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:14.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f42e806af00 msgr2=0x7f42e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f42e806af00 0x7f42e806d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43000099f0 tx=0x7f4300008040 comp rx=0 tx=0).stop 2023-10-16T09:44:14.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 msgr2=0x7f43040b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f43040b5a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f42f4000c00 tx=0x7f42f4004750 comp rx=0 tx=0).stop 2023-10-16T09:44:14.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 shutdown_connections 2023-10-16T09:44:14.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f42e806af00 0x7f42e806d3c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430415da70 0x7f43040c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4304072760 0x7f4304144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 --2- 172.21.15.181:0/280520900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43040ce6a0 0x7f43040b5a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.269+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 >> 172.21.15.181:0/280520900 conn(0x7f43040cd010 msgr2=0x7f43040afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:14.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.270+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 shutdown_connections 2023-10-16T09:44:14.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.270+0000 7f430b80c700 1 -- 172.21.15.181:0/280520900 wait complete. 2023-10-16T09:44:14.284 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:14.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.405+0000 7f2d4597c700 1 Processor -- start 2023-10-16T09:44:14.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.405+0000 7f2d4597c700 1 -- start start 2023-10-16T09:44:14.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.406+0000 7f2d4597c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 0x7f2d40070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.406+0000 7f2d4597c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400b1d90 con 0x7f2d40071760 2023-10-16T09:44:14.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.406+0000 7f2d4597c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d40159790 con 0x7f2d4015e3d0 2023-10-16T09:44:14.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.406+0000 7f2d4597c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4015cf70 con 0x7f2d401593b0 2023-10-16T09:44:14.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.406+0000 7f2d3e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2d40071760 0x7f2d400b0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47720/0 (socket says 172.21.15.181:47720) 2023-10-16T09:44:14.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.406+0000 7f2d3e7fc700 1 -- 172.21.15.181:0/2802589309 learned_addr learned my addr 172.21.15.181:0/2802589309 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:14.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3effd700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 0x7f2d40070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557948913 0 0) 0x7f2d400b1d90 con 0x7f2d40071760 2023-10-16T09:44:14.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d280035b0 con 0x7f2d40071760 2023-10-16T09:44:14.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243107099 0 0) 0x7f2d280035b0 con 0x7f2d40071760 2023-10-16T09:44:14.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 msgr2=0x7f2d40070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:14.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 0x7f2d40070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 >> v1:172.21.15.190:6789/0 conn(0x7f2d4015e3d0 legacy=0x7f2d400b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4015c320 con 0x7f2d40071760 2023-10-16T09:44:14.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f2d30001670 con 0x7f2d40071760 2023-10-16T09:44:14.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 >> v1:172.21.15.181:6789/0 conn(0x7f2d40071760 legacy=0x7f2d400b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d28003a30 0x7f2d28003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 0x7f2d28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28004d90 0x7f2d28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4015cf70 con 0x7f2d28004d90 2023-10-16T09:44:14.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2800ba10 con 0x7f2d28003a30 2023-10-16T09:44:14.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2800bbd0 con 0x7f2d401593b0 2023-10-16T09:44:14.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3f7fe700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28004d90 0x7f2d28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3effd700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d28003a30 0x7f2d28003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3e7fc700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 0x7f2d28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3f7fe700 1 -- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 msgr2=0x7f2d28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:14.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3f7fe700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d401593b0 0x7f2d28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3f7fe700 1 -- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d28003a30 msgr2=0x7f2d28003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:14.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3f7fe700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d28003a30 0x7f2d28003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.407+0000 7f2d3f7fe700 1 -- 172.21.15.181:0/2802589309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d280035b0 con 0x7f2d28004d90 2023-10-16T09:44:14.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d3f7fe700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28004d90 0x7f2d28005210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c340 tx=0x7f2d28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91762dff85dffcc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38001f20 con 0x7f2d28004d90 2023-10-16T09:44:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3800ce80 con 0x7f2d28004d90 2023-10-16T09:44:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d3dffb700 1 -- 172.21.15.181:0/2802589309 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38012660 con 0x7f2d28004d90 2023-10-16T09:44:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d4597c700 1 -- 172.21.15.181:0/2802589309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28004d90 msgr2=0x7f2d28005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d4597c700 1 --2- 172.21.15.181:0/2802589309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d28004d90 0x7f2d28005210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c340 tx=0x7f2d28005880 comp rx=0 tx=0).stop 2023-10-16T09:44:14.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d3f7fe700 1 -- 172.21.15.181:0/2802589309 reap_dead start 2023-10-16T09:44:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d4597c700 1 -- 172.21.15.181:0/2802589309 shutdown_connections 2023-10-16T09:44:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d4597c700 1 -- 172.21.15.181:0/2802589309 >> 172.21.15.181:0/2802589309 conn(0x7f2d4006aed0 msgr2=0x7f2d4006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d4597c700 1 -- 172.21.15.181:0/2802589309 shutdown_connections 2023-10-16T09:44:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.408+0000 7f2d4597c700 1 -- 172.21.15.181:0/2802589309 wait complete. 2023-10-16T09:44:14.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.409+0000 7f2d4597c700 1 Processor -- start 2023-10-16T09:44:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d4597c700 1 -- start start 2023-10-16T09:44:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d4597c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d40154360 0x7f2d401f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d4597c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4015b0d0 0x7f2d400b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d4597c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d400b0c50 0x7f2d400b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d4597c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400ce0c0 con 0x7f2d4015b0d0 2023-10-16T09:44:14.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d4597c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400acb80 con 0x7f2d40154360 2023-10-16T09:44:14.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d4597c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401f6ca0 con 0x7f2d400b0c50 2023-10-16T09:44:14.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d3e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4015b0d0 0x7f2d400b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d3e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4015b0d0 0x7f2d400b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36652/0 (socket says 172.21.15.181:36652) 2023-10-16T09:44:14.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d3e7fc700 1 -- 172.21.15.181:0/2687033155 learned_addr learned my addr 172.21.15.181:0/2687033155 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:14.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d3f7fe700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d400b0c50 0x7f2d400b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.410+0000 7f2d3effd700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d40154360 0x7f2d401f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d3e7fc700 1 -- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d400b0c50 msgr2=0x7f2d400b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d3e7fc700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d400b0c50 0x7f2d400b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d3e7fc700 1 -- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d40154360 msgr2=0x7f2d401f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d3e7fc700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d40154360 0x7f2d401f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d3e7fc700 1 -- 172.21.15.181:0/2687033155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400affa0 con 0x7f2d4015b0d0 2023-10-16T09:44:14.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d3e7fc700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4015b0d0 0x7f2d400b0710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d300072c0 tx=0x7f2d30007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d27fff700 1 -- 172.21.15.181:0/2687033155 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300077a0 con 0x7f2d4015b0d0 2023-10-16T09:44:14.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d27fff700 1 -- 172.21.15.181:0/2687033155 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3000f870 con 0x7f2d4015b0d0 2023-10-16T09:44:14.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d400b0280 con 0x7f2d4015b0d0 2023-10-16T09:44:14.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d27fff700 1 -- 172.21.15.181:0/2687033155 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000fc20 con 0x7f2d4015b0d0 2023-10-16T09:44:14.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.411+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d401f71b0 con 0x7f2d4015b0d0 2023-10-16T09:44:14.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.412+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4004ebb0 con 0x7f2d4015b0d0 2023-10-16T09:44:14.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.412+0000 7f2d27fff700 1 -- 172.21.15.181:0/2687033155 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2d3000fd80 con 0x7f2d4015b0d0 2023-10-16T09:44:14.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.413+0000 7f2d27fff700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2d2c06af00 0x7f2d2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.413+0000 7f2d3effd700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2d2c06af00 0x7f2d2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.413+0000 7f2d27fff700 1 -- 172.21.15.181:0/2687033155 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d30089610 con 0x7f2d4015b0d0 2023-10-16T09:44:14.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.413+0000 7f2d3effd700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2d2c06af00 0x7f2d2c06d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009ba0 tx=0x7f2d34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.417+0000 7f2d27fff700 1 -- 172.21.15.181:0/2687033155 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d300582a0 con 0x7f2d4015b0d0 2023-10-16T09:44:14.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.608+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d400aecf0 con 0x7f2d4015b0d0 2023-10-16T09:44:14.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.609+0000 7f2d27fff700 1 -- 172.21.15.181:0/2687033155 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d30057e30 con 0x7f2d4015b0d0 2023-10-16T09:44:14.611 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:14.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2d2c06af00 msgr2=0x7f2d2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2d2c06af00 0x7f2d2c06d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009ba0 tx=0x7f2d34008040 comp rx=0 tx=0).stop 2023-10-16T09:44:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4015b0d0 msgr2=0x7f2d400b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4015b0d0 0x7f2d400b0710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d300072c0 tx=0x7f2d30007680 comp rx=0 tx=0).stop 2023-10-16T09:44:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 shutdown_connections 2023-10-16T09:44:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2d2c06af00 0x7f2d2c06d3c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d400b0c50 0x7f2d400b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d40154360 0x7f2d401f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 --2- 172.21.15.181:0/2687033155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4015b0d0 0x7f2d400b0710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 >> 172.21.15.181:0/2687033155 conn(0x7f2d4006aed0 msgr2=0x7f2d4015e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:14.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 shutdown_connections 2023-10-16T09:44:14.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.611+0000 7f2d4597c700 1 -- 172.21.15.181:0/2687033155 wait complete. 2023-10-16T09:44:14.625 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:14.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.743+0000 7f53bb89b700 1 Processor -- start 2023-10-16T09:44:14.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.744+0000 7f53bb89b700 1 -- start start 2023-10-16T09:44:14.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.744+0000 7f53bb89b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53b415dfd0 0x7f53b40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.744+0000 7f53bb89b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53b40cd470 0x7f53b40cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.744+0000 7f53bb89b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b406f870 con 0x7f53b40af450 2023-10-16T09:44:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.744+0000 7f53bb89b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40cde40 con 0x7f53b40cd470 2023-10-16T09:44:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.744+0000 7f53bb89b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40d1160 con 0x7f53b415dfd0 2023-10-16T09:44:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b9e38700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53b40cd470 0x7f53b40cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b8e36700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f53b40af450 0x7f53b406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47736/0 (socket says 172.21.15.181:47736) 2023-10-16T09:44:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b8e36700 1 -- 172.21.15.181:0/1655647193 learned_addr learned my addr 172.21.15.181:0/1655647193 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b9637700 1 --2- 172.21.15.181:0/1655647193 >> v2:172.21.15.190:3300/0 conn(0x7f53b415dfd0 0x7f53b40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b9e38700 1 -- 172.21.15.181:0/1655647193 >> v2:172.21.15.190:3300/0 conn(0x7f53b415dfd0 msgr2=0x7f53b40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:14.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b9e38700 1 --2- 172.21.15.181:0/1655647193 >> v2:172.21.15.190:3300/0 conn(0x7f53b415dfd0 0x7f53b40af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b9e38700 1 -- 172.21.15.181:0/1655647193 >> v1:172.21.15.181:6789/0 conn(0x7f53b40af450 legacy=0x7f53b406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b9e38700 1 -- 172.21.15.181:0/1655647193 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b40cc8e0 con 0x7f53b40cd470 2023-10-16T09:44:14.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53abfff700 1 -- 172.21.15.181:0/1655647193 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506221355 0 0) 0x7f53b406f870 con 0x7f53b40af450 2023-10-16T09:44:14.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53b9e38700 1 --2- 172.21.15.181:0/1655647193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b40cd900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f53b4070340 tx=0x7f53ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b433fe1cb4b895 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53abfff700 1 -- 172.21.15.181:0/1655647193 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00e070 con 0x7f53b40cd470 2023-10-16T09:44:14.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53abfff700 1 -- 172.21.15.181:0/1655647193 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac00bbd0 con 0x7f53b40cd470 2023-10-16T09:44:14.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.745+0000 7f53abfff700 1 -- 172.21.15.181:0/1655647193 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac012430 con 0x7f53b40cd470 2023-10-16T09:44:14.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 -- 172.21.15.181:0/1655647193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 msgr2=0x7f53b40cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 --2- 172.21.15.181:0/1655647193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b40cd900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f53b4070340 tx=0x7f53ac00b330 comp rx=0 tx=0).stop 2023-10-16T09:44:14.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 -- 172.21.15.181:0/1655647193 shutdown_connections 2023-10-16T09:44:14.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 --2- 172.21.15.181:0/1655647193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b40cd900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 --2- 172.21.15.181:0/1655647193 >> v2:172.21.15.190:3300/0 conn(0x7f53b415dfd0 0x7f53b40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 -- 172.21.15.181:0/1655647193 >> 172.21.15.181:0/1655647193 conn(0x7f53b40d04a0 msgr2=0x7f53b40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:14.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 -- 172.21.15.181:0/1655647193 shutdown_connections 2023-10-16T09:44:14.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.746+0000 7f53bb89b700 1 -- 172.21.15.181:0/1655647193 wait complete. 2023-10-16T09:44:14.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.747+0000 7f53bb89b700 1 Processor -- start 2023-10-16T09:44:14.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.747+0000 7f53bb89b700 1 -- start start 2023-10-16T09:44:14.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53bb89b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b40af450 0x7f53b40ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53bb89b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b414c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53bb89b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b415dfd0 0x7f53b40b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53bb89b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41e5560 con 0x7f53b40cd470 2023-10-16T09:44:14.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53bb89b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40c57b0 con 0x7f53b40af450 2023-10-16T09:44:14.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53bb89b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40b42a0 con 0x7f53b415dfd0 2023-10-16T09:44:14.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b414c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b414c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36664/0 (socket says 172.21.15.181:36664) 2023-10-16T09:44:14.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 -- 172.21.15.181:0/3900152819 learned_addr learned my addr 172.21.15.181:0/3900152819 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:14.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b9637700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b40af450 0x7f53b40ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b9e38700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b415dfd0 0x7f53b40b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 -- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b415dfd0 msgr2=0x7f53b40b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b415dfd0 0x7f53b40b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 -- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b40af450 msgr2=0x7f53b40ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:14.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b40af450 0x7f53b40ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:14.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 -- 172.21.15.181:0/3900152819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac009a70 con 0x7f53b40cd470 2023-10-16T09:44:14.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b9e38700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b415dfd0 0x7f53b40b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:14.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53b8e36700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b414c7d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53b000c580 tx=0x7f53b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53a9ffb700 1 -- 172.21.15.181:0/3900152819 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0004c80 con 0x7f53b40cd470 2023-10-16T09:44:14.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53a9ffb700 1 -- 172.21.15.181:0/3900152819 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b0004e00 con 0x7f53b40cd470 2023-10-16T09:44:14.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b40b3fc0 con 0x7f53b40cd470 2023-10-16T09:44:14.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.748+0000 7f53a9ffb700 1 -- 172.21.15.181:0/3900152819 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0007310 con 0x7f53b40cd470 2023-10-16T09:44:14.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.749+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b40c5440 con 0x7f53b40cd470 2023-10-16T09:44:14.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.750+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b4067d20 con 0x7f53b40cd470 2023-10-16T09:44:14.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.750+0000 7f53a9ffb700 1 -- 172.21.15.181:0/3900152819 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53b0007470 con 0x7f53b40cd470 2023-10-16T09:44:14.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.750+0000 7f53a9ffb700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f539806af00 0x7f539806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:14.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.750+0000 7f53a9ffb700 1 -- 172.21.15.181:0/3900152819 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53b00892b0 con 0x7f53b40cd470 2023-10-16T09:44:14.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.750+0000 7f53b9637700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f539806af00 0x7f539806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:14.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.751+0000 7f53b9637700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f539806af00 0x7f539806d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006fd0 tx=0x7f53a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:14.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.755+0000 7f53a9ffb700 1 -- 172.21.15.181:0/3900152819 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f53b0054360 con 0x7f53b40cd470 2023-10-16T09:44:14.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.995+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53b406cc30 con 0x7f53b40cd470 2023-10-16T09:44:14.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.995+0000 7f53a9ffb700 1 -- 172.21.15.181:0/3900152819 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f53b0057980 con 0x7f53b40cd470 2023-10-16T09:44:14.998 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:14.998 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:15.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f539806af00 msgr2=0x7f539806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f539806af00 0x7f539806d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006fd0 tx=0x7f53a4008040 comp rx=0 tx=0).stop 2023-10-16T09:44:15.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 msgr2=0x7f53b414c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b414c7d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53b000c580 tx=0x7f53b000c940 comp rx=0 tx=0).stop 2023-10-16T09:44:15.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 shutdown_connections 2023-10-16T09:44:15.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f539806af00 0x7f539806d3c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53b415dfd0 0x7f53b40b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b40af450 0x7f53b40ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 --2- 172.21.15.181:0/3900152819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b40cd470 0x7f53b414c7d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 >> 172.21.15.181:0/3900152819 conn(0x7f53b40d04a0 msgr2=0x7f53b40ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:15.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 shutdown_connections 2023-10-16T09:44:15.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:14.998+0000 7f53bb89b700 1 -- 172.21.15.181:0/3900152819 wait complete. 2023-10-16T09:44:15.003 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:15.012 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.130+0000 7ff2a7844700 1 Processor -- start 2023-10-16T09:44:15.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.130+0000 7ff2a7844700 1 -- start start 2023-10-16T09:44:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a7844700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2a015dfd0 0x7ff2a00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a7844700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2a00af450 0x7ff2a006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a7844700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a006db80 0x7ff2a00cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a7844700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a006f870 con 0x7ff2a015dfd0 2023-10-16T09:44:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a7844700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a00d1160 con 0x7ff2a00af450 2023-10-16T09:44:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a7844700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00cc8e0 con 0x7ff2a006db80 2023-10-16T09:44:15.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2a015dfd0 0x7ff2a00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2a015dfd0 0x7ff2a00af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36670/0 (socket says 172.21.15.181:36670) 2023-10-16T09:44:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 -- 172.21.15.181:0/2931516698 learned_addr learned my addr 172.21.15.181:0/2931516698 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a4ddf700 1 --2- 172.21.15.181:0/2931516698 >> v2:172.21.15.190:3300/0 conn(0x7ff2a00af450 0x7ff2a006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a5de1700 1 --2- 172.21.15.181:0/2931516698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a006db80 0x7ff2a00cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 -- 172.21.15.181:0/2931516698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a006db80 msgr2=0x7ff2a00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 --2- 172.21.15.181:0/2931516698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a006db80 0x7ff2a00cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 -- 172.21.15.181:0/2931516698 >> v2:172.21.15.190:3300/0 conn(0x7ff2a00af450 msgr2=0x7ff2a006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 --2- 172.21.15.181:0/2931516698 >> v2:172.21.15.190:3300/0 conn(0x7ff2a00af450 0x7ff2a006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 -- 172.21.15.181:0/2931516698 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00ccbc0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a5de1700 1 --2- 172.21.15.181:0/2931516698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a006db80 0x7ff2a00cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a4ddf700 1 --2- 172.21.15.181:0/2931516698 >> v2:172.21.15.190:3300/0 conn(0x7ff2a00af450 0x7ff2a006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.131+0000 7ff2a55e0700 1 --2- 172.21.15.181:0/2931516698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00af9a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b3070 tx=0x7ff29c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584a2603f6e46a51 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff297fff700 1 -- 172.21.15.181:0/2931516698 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c005e30 con 0x7ff2a015dfd0 2023-10-16T09:44:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff297fff700 1 -- 172.21.15.181:0/2931516698 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c004030 con 0x7ff2a015dfd0 2023-10-16T09:44:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff297fff700 1 -- 172.21.15.181:0/2931516698 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0043e0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 -- 172.21.15.181:0/2931516698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 msgr2=0x7ff2a00af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 --2- 172.21.15.181:0/2931516698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00af9a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b3070 tx=0x7ff29c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 -- 172.21.15.181:0/2931516698 shutdown_connections 2023-10-16T09:44:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 --2- 172.21.15.181:0/2931516698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00af9a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 --2- 172.21.15.181:0/2931516698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a006db80 0x7ff2a00cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 --2- 172.21.15.181:0/2931516698 >> v2:172.21.15.190:3300/0 conn(0x7ff2a00af450 0x7ff2a006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 -- 172.21.15.181:0/2931516698 >> 172.21.15.181:0/2931516698 conn(0x7ff2a00d04a0 msgr2=0x7ff2a00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 -- 172.21.15.181:0/2931516698 shutdown_connections 2023-10-16T09:44:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.132+0000 7ff2a7844700 1 -- 172.21.15.181:0/2931516698 wait complete. 2023-10-16T09:44:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.133+0000 7ff2a7844700 1 Processor -- start 2023-10-16T09:44:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.133+0000 7ff2a7844700 1 -- start start 2023-10-16T09:44:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a7844700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a006db80 0x7ff2a00c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a7844700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a00af450 0x7ff2a0155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a7844700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a7844700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00b00a0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a7844700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a015cbb0 con 0x7ff2a006db80 2023-10-16T09:44:15.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a7844700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00c7ab0 con 0x7ff2a00af450 2023-10-16T09:44:15.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a5de1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a5de1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36678/0 (socket says 172.21.15.181:36678) 2023-10-16T09:44:15.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a5de1700 1 -- 172.21.15.181:0/3915233483 learned_addr learned my addr 172.21.15.181:0/3915233483 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:15.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a55e0700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a006db80 0x7ff2a00c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a4ddf700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a00af450 0x7ff2a0155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a5de1700 1 -- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a00af450 msgr2=0x7ff2a0155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.134+0000 7ff2a5de1700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a00af450 0x7ff2a0155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a5de1700 1 -- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a006db80 msgr2=0x7ff2a00c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a5de1700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a006db80 0x7ff2a00c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a5de1700 1 -- 172.21.15.181:0/3915233483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009500 con 0x7ff2a015dfd0 2023-10-16T09:44:15.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a55e0700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a006db80 0x7ff2a00c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:15.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a4ddf700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a00af450 0x7ff2a0155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:15.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a5de1700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00c7e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff29800cef0 tx=0x7ff29800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:15.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff295ffb700 1 -- 172.21.15.181:0/3915233483 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800add0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff295ffb700 1 -- 172.21.15.181:0/3915233483 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298015440 con 0x7ff2a015dfd0 2023-10-16T09:44:15.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a015c5f0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff295ffb700 1 -- 172.21.15.181:0/3915233483 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2980157f0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.135+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a00af9e0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.136+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a00b01e0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.136+0000 7ff295ffb700 1 -- 172.21.15.181:0/3915233483 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff29800d430 con 0x7ff2a015dfd0 2023-10-16T09:44:15.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.137+0000 7ff295ffb700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff28406aed0 0x7ff28406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.137+0000 7ff2a55e0700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff28406aed0 0x7ff28406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.137+0000 7ff295ffb700 1 -- 172.21.15.181:0/3915233483 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff298089b80 con 0x7ff2a015dfd0 2023-10-16T09:44:15.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.137+0000 7ff2a55e0700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff28406aed0 0x7ff28406d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b3070 tx=0x7ff29c008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:15.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.141+0000 7ff295ffb700 1 -- 172.21.15.181:0/3915233483 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff2a00b01e0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.337+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2a00b01e0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.339+0000 7ff295ffb700 1 -- 172.21.15.181:0/3915233483 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7ff2a00b01e0 con 0x7ff2a015dfd0 2023-10-16T09:44:15.341 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:15.341 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:15.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff28406aed0 msgr2=0x7ff28406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff28406aed0 0x7ff28406d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b3070 tx=0x7ff29c008070 comp rx=0 tx=0).stop 2023-10-16T09:44:15.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 msgr2=0x7ff2a00c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00c7e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff29800cef0 tx=0x7ff29800a340 comp rx=0 tx=0).stop 2023-10-16T09:44:15.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 shutdown_connections 2023-10-16T09:44:15.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff28406aed0 0x7ff28406d390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a015dfd0 0x7ff2a00c7e10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a006db80 0x7ff2a00c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 --2- 172.21.15.181:0/3915233483 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a00af450 0x7ff2a0155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 >> 172.21.15.181:0/3915233483 conn(0x7ff2a00d04a0 msgr2=0x7ff2a00cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:15.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 shutdown_connections 2023-10-16T09:44:15.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.341+0000 7ff2a7844700 1 -- 172.21.15.181:0/3915233483 wait complete. 2023-10-16T09:44:15.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:15.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.474+0000 7fa3e85c1700 1 Processor -- start 2023-10-16T09:44:15.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.475+0000 7fa3e85c1700 1 -- start start 2023-10-16T09:44:15.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.475+0000 7fa3e85c1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa3e015dfd0 0x7fa3e00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e85c1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e006f870 con 0x7fa3e00cd470 2023-10-16T09:44:15.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e85c1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e00d1160 con 0x7fa3e00af450 2023-10-16T09:44:15.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e85c1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e00cc8e0 con 0x7fa3e015dfd0 2023-10-16T09:44:15.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e6b5e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa3e00cd470 0x7fa3e006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47762/0 (socket says 172.21.15.181:47762) 2023-10-16T09:44:15.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e6b5e700 1 -- 172.21.15.181:0/3306719973 learned_addr learned my addr 172.21.15.181:0/3306719973 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:15.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e635d700 1 --2- 172.21.15.181:0/3306719973 >> v2:172.21.15.190:3300/0 conn(0x7fa3e015dfd0 0x7fa3e00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e535b700 1 -- 172.21.15.181:0/3306719973 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501414128 0 0) 0x7fa3e006f870 con 0x7fa3e00cd470 2023-10-16T09:44:15.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e535b700 1 -- 172.21.15.181:0/3306719973 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3cc0035b0 con 0x7fa3e00cd470 2023-10-16T09:44:15.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e635d700 1 -- 172.21.15.181:0/3306719973 >> v1:172.21.15.190:6789/0 conn(0x7fa3e00af450 legacy=0x7fa3e006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e635d700 1 -- 172.21.15.181:0/3306719973 >> v1:172.21.15.181:6789/0 conn(0x7fa3e00cd470 legacy=0x7fa3e006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e635d700 1 -- 172.21.15.181:0/3306719973 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00ccbc0 con 0x7fa3e015dfd0 2023-10-16T09:44:15.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e635d700 1 --2- 172.21.15.181:0/3306719973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e015dfd0 0x7fa3e00af9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009820 tx=0x7fa3dc009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5428c21f2f4aef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:15.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.476+0000 7fa3e535b700 1 -- 172.21.15.181:0/3306719973 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc005e30 con 0x7fa3e015dfd0 2023-10-16T09:44:15.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e535b700 1 -- 172.21.15.181:0/3306719973 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc006030 con 0x7fa3e015dfd0 2023-10-16T09:44:15.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e535b700 1 -- 172.21.15.181:0/3306719973 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0063c0 con 0x7fa3e015dfd0 2023-10-16T09:44:15.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e85c1700 1 -- 172.21.15.181:0/3306719973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e015dfd0 msgr2=0x7fa3e00af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/3306719973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e015dfd0 0x7fa3e00af9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009820 tx=0x7fa3dc009b30 comp rx=0 tx=0).stop 2023-10-16T09:44:15.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e85c1700 1 -- 172.21.15.181:0/3306719973 shutdown_connections 2023-10-16T09:44:15.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/3306719973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e015dfd0 0x7fa3e00af9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e85c1700 1 -- 172.21.15.181:0/3306719973 >> 172.21.15.181:0/3306719973 conn(0x7fa3e00d04a0 msgr2=0x7fa3e00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:15.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e85c1700 1 -- 172.21.15.181:0/3306719973 shutdown_connections 2023-10-16T09:44:15.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.477+0000 7fa3e85c1700 1 -- 172.21.15.181:0/3306719973 wait complete. 2023-10-16T09:44:15.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.478+0000 7fa3e85c1700 1 Processor -- start 2023-10-16T09:44:15.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.478+0000 7fa3e85c1700 1 -- start start 2023-10-16T09:44:15.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e85c1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e00af450 0x7fa3e0154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e85c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3e00cd470 0x7fa3e00c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e85c1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3e015dfd0 0x7fa3e0159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e85c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00b0bb0 con 0x7fa3e00cd470 2023-10-16T09:44:15.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e85c1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e01550e0 con 0x7fa3e00af450 2023-10-16T09:44:15.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e85c1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00affa0 con 0x7fa3e015dfd0 2023-10-16T09:44:15.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3e00cd470 0x7fa3e00c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3e00cd470 0x7fa3e00c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36688/0 (socket says 172.21.15.181:36688) 2023-10-16T09:44:15.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 -- 172.21.15.181:0/1970159658 learned_addr learned my addr 172.21.15.181:0/1970159658 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:15.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e6b5e700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3e015dfd0 0x7fa3e0159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e635d700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e00af450 0x7fa3e0154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 -- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3e015dfd0 msgr2=0x7fa3e0159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3e015dfd0 0x7fa3e0159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 -- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e00af450 msgr2=0x7fa3e0154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e00af450 0x7fa3e0154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.479+0000 7fa3e5b5c700 1 -- 172.21.15.181:0/1970159658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc009430 con 0x7fa3e00cd470 2023-10-16T09:44:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.480+0000 7fa3e5b5c700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3e00cd470 0x7fa3e00c3fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c4e0 tx=0x7fa3d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.480+0000 7fa3d6ffd700 1 -- 172.21.15.181:0/1970159658 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00052d0 con 0x7fa3e00cd470 2023-10-16T09:44:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.480+0000 7fa3d6ffd700 1 -- 172.21.15.181:0/1970159658 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0005430 con 0x7fa3e00cd470 2023-10-16T09:44:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.480+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e00b0280 con 0x7fa3e00cd470 2023-10-16T09:44:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.480+0000 7fa3d6ffd700 1 -- 172.21.15.181:0/1970159658 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00038c0 con 0x7fa3e00cd470 2023-10-16T09:44:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.480+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e00b0840 con 0x7fa3e00cd470 2023-10-16T09:44:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.481+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e00b0d20 con 0x7fa3e00cd470 2023-10-16T09:44:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.481+0000 7fa3d6ffd700 1 -- 172.21.15.181:0/1970159658 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa3d0003a20 con 0x7fa3e00cd470 2023-10-16T09:44:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.481+0000 7fa3d6ffd700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa3c406af00 0x7fa3c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.482+0000 7fa3d6ffd700 1 -- 172.21.15.181:0/1970159658 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3d0088ac0 con 0x7fa3e00cd470 2023-10-16T09:44:15.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.482+0000 7fa3e635d700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa3c406af00 0x7fa3c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:15.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.482+0000 7fa3e635d700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa3c406af00 0x7fa3c406d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc013040 tx=0x7fa3dc008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:15.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.486+0000 7fa3d6ffd700 1 -- 172.21.15.181:0/1970159658 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa3d0053b70 con 0x7fa3e00cd470 2023-10-16T09:44:15.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.721+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3e0067d20 con 0x7fa3e00cd470 2023-10-16T09:44:15.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.721+0000 7fa3d6ffd700 1 -- 172.21.15.181:0/1970159658 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7fa3d0017090 con 0x7fa3e00cd470 2023-10-16T09:44:15.724 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:15.724 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:15.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa3c406af00 msgr2=0x7fa3c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa3c406af00 0x7fa3c406d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc013040 tx=0x7fa3dc008070 comp rx=0 tx=0).stop 2023-10-16T09:44:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3e00cd470 msgr2=0x7fa3e00c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3e00cd470 0x7fa3e00c3fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c4e0 tx=0x7fa3d000c8a0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 shutdown_connections 2023-10-16T09:44:15.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa3c406af00 0x7fa3c406d3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3e015dfd0 0x7fa3e0159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3e00af450 0x7fa3e0154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 --2- 172.21.15.181:0/1970159658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3e00cd470 0x7fa3e00c3fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:15.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 >> 172.21.15.181:0/1970159658 conn(0x7fa3e00d04a0 msgr2=0x7fa3e00adbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:15.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 shutdown_connections 2023-10-16T09:44:15.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:15.723+0000 7fa3e85c1700 1 -- 172.21.15.181:0/1970159658 wait complete. 2023-10-16T09:44:15.728 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:16.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T09:44:01.170821+0000', 'modified': '2023-10-16T09:44:04.215750+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/1843354940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 1843354940}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1843354940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T09:44:16.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T09:44:16.141 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-16T09:44:16.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.262+0000 7f7e227f7700 1 Processor -- start 2023-10-16T09:44:16.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.262+0000 7f7e227f7700 1 -- start start 2023-10-16T09:44:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.263+0000 7f7e227f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c15b9f0 0x7f7e1c0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.263+0000 7f7e227f7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e1c0cfb50 0x7f7e1c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.263+0000 7f7e227f7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1c06b370 con 0x7f7e1c0cc400 2023-10-16T09:44:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.263+0000 7f7e227f7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1c06b800 con 0x7f7e1c0cfb50 2023-10-16T09:44:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.263+0000 7f7e227f7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c06aa40 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.263+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c15b9f0 0x7f7e1c0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.263+0000 7f7e20d94700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e1c0cfb50 0x7f7e1c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1b7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f7e1c0cc400 0x7f7e1c0cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59950/0 (socket says 172.21.15.181:59950) 2023-10-16T09:44:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e20d94700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e1c0cfb50 0x7f7e1c0ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41744/0 (socket says 172.21.15.181:41744) 2023-10-16T09:44:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1b7fe700 1 -- 172.21.15.181:0/2716210842 learned_addr learned my addr 172.21.15.181:0/2716210842 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c15b9f0 0x7f7e1c0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:33720/0 (socket says 172.21.15.181:33720) 2023-10-16T09:44:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1affd700 1 -- 172.21.15.181:0/2716210842 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263179857 0 0) 0x7f7e1c06b370 con 0x7f7e1c0cc400 2023-10-16T09:44:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1affd700 1 -- 172.21.15.181:0/2716210842 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e040035b0 con 0x7f7e1c0cc400 2023-10-16T09:44:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e20d94700 1 -- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c15b9f0 msgr2=0x7f7e1c0cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e20d94700 1 --2- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c15b9f0 0x7f7e1c0cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e20d94700 1 -- 172.21.15.181:0/2716210842 >> v1:172.21.15.190:6789/0 conn(0x7f7e1c0cc400 legacy=0x7f7e1c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e20d94700 1 -- 172.21.15.181:0/2716210842 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c06ad20 con 0x7f7e1c0cfb50 2023-10-16T09:44:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c15b9f0 0x7f7e1c0cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e20d94700 1 --2- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 0x7f7e1c0ad110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009d90 tx=0x7f7e1400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a353d6e69d36c6da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1affd700 1 -- 172.21.15.181:0/2716210842 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1400e070 con 0x7f7e1c0cfb50 2023-10-16T09:44:16.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1affd700 1 -- 172.21.15.181:0/2716210842 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1400bb20 con 0x7f7e1c0cfb50 2023-10-16T09:44:16.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.264+0000 7f7e1affd700 1 -- 172.21.15.181:0/2716210842 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1400be90 con 0x7f7e1c0cfb50 2023-10-16T09:44:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 -- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 msgr2=0x7f7e1c0ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 --2- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 0x7f7e1c0ad110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009d90 tx=0x7f7e1400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:16.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 -- 172.21.15.181:0/2716210842 shutdown_connections 2023-10-16T09:44:16.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 --2- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 0x7f7e1c0ad110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 --2- 172.21.15.181:0/2716210842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c15b9f0 0x7f7e1c0cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 -- 172.21.15.181:0/2716210842 >> 172.21.15.181:0/2716210842 conn(0x7f7e1c0d04a0 msgr2=0x7f7e1c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:16.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 -- 172.21.15.181:0/2716210842 shutdown_connections 2023-10-16T09:44:16.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.265+0000 7f7e227f7700 1 -- 172.21.15.181:0/2716210842 wait complete. 2023-10-16T09:44:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.266+0000 7f7e227f7700 1 Processor -- start 2023-10-16T09:44:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.266+0000 7f7e227f7700 1 -- start start 2023-10-16T09:44:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e227f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c0cc400 0x7f7e1c151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e227f7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 0x7f7e1c09dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e227f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e1c15b9f0 0x7f7e1c0c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e227f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1521b0 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e227f7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0c4ab0 con 0x7f7e1c0cfb50 2023-10-16T09:44:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e227f7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0c4d90 con 0x7f7e1c0cc400 2023-10-16T09:44:16.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e1c15b9f0 0x7f7e1c0c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e1c15b9f0 0x7f7e1c0c3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36704/0 (socket says 172.21.15.181:36704) 2023-10-16T09:44:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 -- 172.21.15.181:0/230626746 learned_addr learned my addr 172.21.15.181:0/230626746 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 0x7f7e1c09dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c0cc400 0x7f7e1c151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 -- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c0cc400 msgr2=0x7f7e1c151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c0cc400 0x7f7e1c151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 -- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 msgr2=0x7f7e1c09dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 0x7f7e1c09dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 -- 172.21.15.181:0/230626746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10009430 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c0cc400 0x7f7e1c151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.267+0000 7f7e20d94700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e1c15b9f0 0x7f7e1c0c3f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e14014040 tx=0x7f7e14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.268+0000 7f7e18ff9700 1 -- 172.21.15.181:0/230626746 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e140048b0 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.268+0000 7f7e18ff9700 1 -- 172.21.15.181:0/230626746 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e14020070 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.268+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e14009a70 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.268+0000 7f7e18ff9700 1 -- 172.21.15.181:0/230626746 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e140276c0 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.268+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c09d5c0 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.269+0000 7f7e18ff9700 1 -- 172.21.15.181:0/230626746 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e14012430 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.269+0000 7f7e18ff9700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e0806af00 0x7f7e0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.270+0000 7f7e18ff9700 1 -- 172.21.15.181:0/230626746 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e14022070 con 0x7f7e1c15b9f0 2023-10-16T09:44:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.270+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e0806af00 0x7f7e0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.270+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e0806af00 0x7f7e0806d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0c6f40 tx=0x7f7e10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 Processor -- start 2023-10-16T09:44:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 -- start start 2023-10-16T09:44:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa58a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa58fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478a9a2c20 con 0x56478aa51220 2023-10-16T09:44:16.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478a987c10 con 0x56478aa56630 2023-10-16T09:44:16.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56478a985530 con 0x56478aa2c010 2023-10-16T09:44:16.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa58a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa58a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36708/0 (socket says 172.21.15.181:36708) 2023-10-16T09:44:16.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.272+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4079676732 learned_addr learned my addr 172.21.15.181:0/4079676732 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:16.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e20d94700 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa58fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 msgr2=0x56478aa2c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 msgr2=0x56478aa58fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa58fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4079676732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56478a93e3a0 con 0x56478aa51220 2023-10-16T09:44:16.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e20d94700 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:16.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa58fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:16.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa58a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e10011e00 tx=0x7f7e10014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36122b4aed9be93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e01ffb700 1 -- 172.21.15.181:0/4079676732 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10011e30 con 0x56478aa51220 2023-10-16T09:44:16.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e01ffb700 1 -- 172.21.15.181:0/4079676732 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e10015c60 con 0x56478aa51220 2023-10-16T09:44:16.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e01ffb700 1 -- 172.21.15.181:0/4079676732 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10005c20 con 0x56478aa51220 2023-10-16T09:44:16.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4079676732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 msgr2=0x56478aa58a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.273+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa58a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e10011e00 tx=0x7f7e10014000 comp rx=0 tx=0).stop 2023-10-16T09:44:16.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4079676732 shutdown_connections 2023-10-16T09:44:16.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa58a70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4079676732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa58fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4079676732 >> 172.21.15.181:0/4079676732 conn(0x56478a95bd80 msgr2=0x56478aa76ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:16.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4079676732 shutdown_connections 2023-10-16T09:44:16.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4079676732 wait complete. 2023-10-16T09:44:16.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 -- start start 2023-10-16T09:44:16.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.274+0000 7f7e2e9e1b80 1 Processor -- start 2023-10-16T09:44:16.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa54250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa2c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa2c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478a980e20 con 0x56478aa51220 2023-10-16T09:44:16.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478a91d6c0 con 0x56478aa56630 2023-10-16T09:44:16.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56478a91d360 con 0x56478aa2c010 2023-10-16T09:44:16.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e2e9e1b80 10 client.0 _ll_register_callbacks cb 0x56478a77a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:44:16.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa2c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa2c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36716/0 (socket says 172.21.15.181:36716) 2023-10-16T09:44:16.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 -- 172.21.15.181:0/390642424 learned_addr learned my addr 172.21.15.181:0/390642424 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:16.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa54250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e20d94700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa2c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 -- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 msgr2=0x56478aa54250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa54250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 -- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 msgr2=0x56478aa2c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa2c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1b7fe700 1 -- 172.21.15.181:0/390642424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e10011ae0 con 0x56478aa51220 2023-10-16T09:44:16.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e20d94700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa2c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:16.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.275+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa54250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:16.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa2c450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c000f60 tx=0x7f7e0c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddc5bf624c3821a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7dfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:44:16.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7dfaffd700 1 -- 172.21.15.181:0/390642424 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c001af0 con 0x56478aa51220 2023-10-16T09:44:16.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7dfaffd700 1 -- 172.21.15.181:0/390642424 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e0c001c50 con 0x56478aa51220 2023-10-16T09:44:16.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56478aa215b0 con 0x56478aa51220 2023-10-16T09:44:16.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7dfaffd700 1 -- 172.21.15.181:0/390642424 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c014400 con 0x56478aa51220 2023-10-16T09:44:16.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 10 client.4617 fetch_fsmap learned FSMap version 8 2023-10-16T09:44:16.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56478a7abd90 con 0x56478aa51220 2023-10-16T09:44:16.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7dfaffd700 1 -- 172.21.15.181:0/390642424 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56478aa215b0 con 0x56478aa51220 2023-10-16T09:44:16.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7dfaffd700 1 -- 172.21.15.181:0/390642424 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e0c014560 con 0x56478aa51220 2023-10-16T09:44:16.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 10 client.4617 fetch_fsmap finished waiting for FSMap version 8 2023-10-16T09:44:16.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 10 client.4617 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T09:44:16.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 20 client.4617 populate_metadata read hostname 'smithi181' 2023-10-16T09:44:16.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aa84e60 0x56478aa82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 4 client.4617 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T09:44:16.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 --> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56478a979e00 con 0x56478aa84e60 2023-10-16T09:44:16.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.276+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aa84e60 0x56478aa82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.277+0000 7f7dfaffd700 10 client.4617 ms_handle_connect on v2:172.21.15.190:6833/1843354940 2023-10-16T09:44:16.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.277+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aa84e60 0x56478aa82d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e6c7f8370d6442 server_cookie=df433e8f7276f97b in_seq=0 out_seq=0 2023-10-16T09:44:16.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.277+0000 7f7dfaffd700 1 -- 172.21.15.181:0/390642424 <== mds.0 v2:172.21.15.190:6833/1843354940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56478a979e00 con 0x56478aa84e60 2023-10-16T09:44:16.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.277+0000 7f7dfaffd700 10 client.4617 handle_command_reply: tid=0 2023-10-16T09:44:16.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.277+0000 7f7e2e9e1b80 1 client.4617 shutdown 2023-10-16T09:44:16.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 msgr2=0x56478aa2c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa2c450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c000f60 tx=0x7f7e0c005f90 comp rx=0 tx=0).stop 2023-10-16T09:44:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 shutdown_connections 2023-10-16T09:44:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aa84e60 0x56478aa82d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa54250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aa2c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7dfaffd700 0 client.4617 ms_handle_reset on v2:172.21.15.190:6833/1843354940 2023-10-16T09:44:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/390642424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa51220 0x56478aa2c450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 >> 172.21.15.181:0/390642424 conn(0x56478a95bd80 msgr2=0x56478aa76f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 shutdown_connections 2023-10-16T09:44:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/390642424 wait complete. 2023-10-16T09:44:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.278+0000 7f7e2e9e1b80 20 client.4617 trim_cache size 0 max 16384 2023-10-16T09:44:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.295+0000 7f7e2e9e1b80 1 Processor -- start 2023-10-16T09:44:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.295+0000 7f7e2e9e1b80 1 -- start start 2023-10-16T09:44:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa51220 0x56478aa831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa56630 0x56478aa836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478a8018d0 con 0x56478aa56630 2023-10-16T09:44:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478aa53450 con 0x56478aa51220 2023-10-16T09:44:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56478aa043c0 con 0x56478aa2c010 2023-10-16T09:44:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa56630 0x56478aa836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa56630 0x56478aa836e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36726/0 (socket says 172.21.15.181:36726) 2023-10-16T09:44:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 -- 172.21.15.181:0/2396456137 learned_addr learned my addr 172.21.15.181:0/2396456137 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa51220 0x56478aa831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 -- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 msgr2=0x56478aa2e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 -- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa51220 msgr2=0x56478aa831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa51220 0x56478aa831a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 -- 172.21.15.181:0/2396456137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56478a794c10 con 0x56478aa56630 2023-10-16T09:44:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.296+0000 7f7e20d94700 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa56630 0x56478aa836e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7e14089ab0 tx=0x7f7e140567c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adeb5cc5b68e1e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e027fc700 1 -- 172.21.15.181:0/2396456137 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14089ea0 con 0x56478aa56630 2023-10-16T09:44:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e027fc700 1 -- 172.21.15.181:0/2396456137 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e14070030 con 0x56478aa56630 2023-10-16T09:44:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e027fc700 1 -- 172.21.15.181:0/2396456137 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1405a350 con 0x56478aa56630 2023-10-16T09:44:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/2396456137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa56630 msgr2=0x56478aa836e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa56630 0x56478aa836e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7e14089ab0 tx=0x7f7e140567c0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/2396456137 shutdown_connections 2023-10-16T09:44:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa2c010 0x56478aa2e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa56630 0x56478aa836e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/2396456137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa51220 0x56478aa831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/2396456137 >> 172.21.15.181:0/2396456137 conn(0x56478a95bd80 msgr2=0x56478aa73560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/2396456137 shutdown_connections 2023-10-16T09:44:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.297+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/2396456137 wait complete. 2023-10-16T09:44:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.298+0000 7f7e2e9e1b80 1 -- start start 2023-10-16T09:44:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.298+0000 7f7e2e9e1b80 1 Processor -- start 2023-10-16T09:44:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa2c010 0x56478aa70f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa51220 0x56478aa4f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aaddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478aa71840 con 0x56478aa2c010 2023-10-16T09:44:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56478aa5bef0 con 0x56478aa56630 2023-10-16T09:44:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56478aa5c140 con 0x56478aa51220 2023-10-16T09:44:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 10 client.0 _ll_register_callbacks cb 0x7f7e0c0070e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:44:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa2c010 0x56478aa70f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa2c010 0x56478aa70f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36728/0 (socket says 172.21.15.181:36728) 2023-10-16T09:44:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4274604343 learned_addr learned my addr 172.21.15.181:0/4274604343 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e20d94700 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aaddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa51220 0x56478aa4f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa51220 msgr2=0x56478aa4f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa51220 0x56478aa4f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 msgr2=0x56478aaddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aaddf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 -- 172.21.15.181:0/4274604343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c09d5c0 con 0x56478aa2c010 2023-10-16T09:44:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e1bfff700 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa2c010 0x56478aa70f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56478aa2ffa0 tx=0x7f7e100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4e0a0fddb8ed90 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7dfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:44:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7dfaffd700 1 -- 172.21.15.181:0/4274604343 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10031050 con 0x56478aa2c010 2023-10-16T09:44:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7dfaffd700 1 -- 172.21.15.181:0/4274604343 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e10005aa0 con 0x56478aa2c010 2023-10-16T09:44:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.299+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e10015c60 con 0x56478aa2c010 2023-10-16T09:44:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7dfaffd700 1 -- 172.21.15.181:0/4274604343 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e10015e90 con 0x56478aa2c010 2023-10-16T09:44:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 10 client.4623 fetch_fsmap learned FSMap version 8 2023-10-16T09:44:16.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56478a7d2200 con 0x56478aa2c010 2023-10-16T09:44:16.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7dfaffd700 1 -- 172.21.15.181:0/4274604343 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e10015c60 con 0x56478aa2c010 2023-10-16T09:44:16.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7dfaffd700 1 -- 172.21.15.181:0/4274604343 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e10005c60 con 0x56478aa2c010 2023-10-16T09:44:16.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 10 client.4623 fetch_fsmap finished waiting for FSMap version 8 2023-10-16T09:44:16.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 10 client.4623 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T09:44:16.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 20 client.4623 populate_metadata read hostname 'smithi181' 2023-10-16T09:44:16.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aade710 0x56478aa51810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 4 client.4623 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T09:44:16.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 --> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56478a979b20 con 0x56478aade710 2023-10-16T09:44:16.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aade710 0x56478aa51810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7dfaffd700 10 client.4623 ms_handle_connect on v2:172.21.15.190:6833/1843354940 2023-10-16T09:44:16.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.300+0000 7f7e1b7fe700 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aade710 0x56478aa51810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ded61341f9ae94 server_cookie=82e6206d191f7852 in_seq=0 out_seq=0 2023-10-16T09:44:16.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7dfaffd700 1 -- 172.21.15.181:0/4274604343 <== mds.0 v2:172.21.15.190:6833/1843354940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56478a979b20 con 0x56478aade710 2023-10-16T09:44:16.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7dfaffd700 10 client.4623 handle_command_reply: tid=0 2023-10-16T09:44:16.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 client.4623 shutdown 2023-10-16T09:44:16.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa2c010 msgr2=0x56478aa70f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa2c010 0x56478aa70f60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56478aa2ffa0 tx=0x7f7e100047c0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 shutdown_connections 2023-10-16T09:44:16.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:6833/1843354940,v1:172.21.15.190:6835/1843354940] conn(0x56478aade710 0x56478aa51810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56478aa56630 0x56478aaddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7dfaffd700 0 client.4623 ms_handle_reset on v2:172.21.15.190:6833/1843354940 2023-10-16T09:44:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56478aa2c010 0x56478aa70f60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 --2- 172.21.15.181:0/4274604343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56478aa51220 0x56478aa4f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 >> 172.21.15.181:0/4274604343 conn(0x56478a95bd80 msgr2=0x56478aa559f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:16.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 shutdown_connections 2023-10-16T09:44:16.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 1 -- 172.21.15.181:0/4274604343 wait complete. 2023-10-16T09:44:16.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e2e9e1b80 20 client.4623 trim_cache size 0 max 16384 2023-10-16T09:44:16.332 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:44:16.333 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T09:44:16.333 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T09:44:16.333 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 4272, 2023-10-16T09:44:16.333 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T09:44:16.334 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T09:44:16.334 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T09:44:16.334 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 14.498710427000001, 2023-10-16T09:44:16.334 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 8, 2023-10-16T09:44:16.334 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 22, 2023-10-16T09:44:16.335 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 22, 2023-10-16T09:44:16.335 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 21.229455994999999 2023-10-16T09:44:16.335 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:44:16.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e0806af00 msgr2=0x7f7e0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e0806af00 0x7f7e0806d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0c6f40 tx=0x7f7e10008040 comp rx=0 tx=0).stop 2023-10-16T09:44:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e1c15b9f0 msgr2=0x7f7e1c0c3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e1c15b9f0 0x7f7e1c0c3f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7e14014040 tx=0x7f7e14004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 shutdown_connections 2023-10-16T09:44:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e0806af00 0x7f7e0806d3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e1c0cc400 0x7f7e1c151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e1c15b9f0 0x7f7e1c0c3f40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.301+0000 7f7e227f7700 1 --2- 172.21.15.181:0/230626746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e1c0cfb50 0x7f7e1c09dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.302+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 >> 172.21.15.181:0/230626746 conn(0x7f7e1c0d04a0 msgr2=0x7f7e1c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.302+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 shutdown_connections 2023-10-16T09:44:16.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.302+0000 7f7e227f7700 1 -- 172.21.15.181:0/230626746 wait complete. 2023-10-16T09:44:16.339 INFO:teuthology.run_tasks:Running task kclient... 2023-10-16T09:44:16.361 INFO:tasks.kclient:Mounting kernel clients... 2023-10-16T09:44:16.361 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-16T09:44:16.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.362 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T09:44:16.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.380 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-16T09:44:16.449 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-16T09:44:16.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:44:16.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.451 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T09:44:16.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.508 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-16T09:44:16.574 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-16T09:44:16.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:44:16.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.576 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-10-16T09:44:16.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-10-16T09:44:16.670 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-10-16T09:44:16.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:44:16.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.672 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-10-16T09:44:16.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:16.732 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-10-16T09:44:16.803 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-10-16T09:44:16.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T09:44:16.805 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-16T09:44:16.805 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:44:16.806 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:44:16.806 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:44:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.926+0000 7f6216f30700 1 Processor -- start 2023-10-16T09:44:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.926+0000 7f6216f30700 1 -- start start 2023-10-16T09:44:16.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.926+0000 7f6216f30700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f621015dfd0 0x7f62100af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.926+0000 7f6216f30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100af450 0x7f621006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.926+0000 7f6216f30700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621006f870 con 0x7f621006db80 2023-10-16T09:44:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.926+0000 7f6216f30700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100d1160 con 0x7f621015dfd0 2023-10-16T09:44:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.926+0000 7f6216f30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100cc8e0 con 0x7f62100af450 2023-10-16T09:44:16.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f6214ccc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f621015dfd0 0x7f62100af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f620ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100af450 0x7f621006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f62154cd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f621006db80 0x7f62100cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:59992/0 (socket says 172.21.15.181:59992) 2023-10-16T09:44:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f6214ccc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f621015dfd0 0x7f62100af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:41786/0 (socket says 172.21.15.181:41786) 2023-10-16T09:44:16.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f62154cd700 1 -- 172.21.15.181:0/3036452152 learned_addr learned my addr 172.21.15.181:0/3036452152 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:16.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f6214ccc700 1 -- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100af450 msgr2=0x7f621006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:16.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f6214ccc700 1 --2- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100af450 0x7f621006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f6214ccc700 1 -- 172.21.15.181:0/3036452152 >> v1:172.21.15.190:6789/0 conn(0x7f621006db80 legacy=0x7f62100cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f6214ccc700 1 -- 172.21.15.181:0/3036452152 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100ccbc0 con 0x7f621015dfd0 2023-10-16T09:44:16.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f620ffff700 1 --2- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100af450 0x7f621006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:16.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f6214ccc700 1 --2- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015dfd0 0x7f62100af9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009820 tx=0x7f61f8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c590897a1235734 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f620f7fe700 1 -- 172.21.15.181:0/3036452152 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8005e30 con 0x7f621015dfd0 2023-10-16T09:44:16.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f620f7fe700 1 -- 172.21.15.181:0/3036452152 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f8006030 con 0x7f621015dfd0 2023-10-16T09:44:16.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.927+0000 7f620f7fe700 1 -- 172.21.15.181:0/3036452152 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f80063c0 con 0x7f621015dfd0 2023-10-16T09:44:16.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 -- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015dfd0 msgr2=0x7f62100af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 --2- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015dfd0 0x7f62100af9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009820 tx=0x7f61f8009b30 comp rx=0 tx=0).stop 2023-10-16T09:44:16.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 -- 172.21.15.181:0/3036452152 shutdown_connections 2023-10-16T09:44:16.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 --2- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015dfd0 0x7f62100af9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 --2- 172.21.15.181:0/3036452152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100af450 0x7f621006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 -- 172.21.15.181:0/3036452152 >> 172.21.15.181:0/3036452152 conn(0x7f62100d04a0 msgr2=0x7f62100d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:16.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 -- 172.21.15.181:0/3036452152 shutdown_connections 2023-10-16T09:44:16.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.928+0000 7f6216f30700 1 -- 172.21.15.181:0/3036452152 wait complete. 2023-10-16T09:44:16.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.929+0000 7f6216f30700 1 Processor -- start 2023-10-16T09:44:16.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.929+0000 7f6216f30700 1 -- start start 2023-10-16T09:44:16.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f6216f30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f621006db80 0x7f62101546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f6216f30700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62100af450 0x7f62100b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f6216f30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621015dfd0 0x7f62100b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f6216f30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100b0df0 con 0x7f621015dfd0 2023-10-16T09:44:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f6216f30700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100aff00 con 0x7f62100af450 2023-10-16T09:44:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f6216f30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100b01e0 con 0x7f621006db80 2023-10-16T09:44:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f62154cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621015dfd0 0x7f62100b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f62154cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621015dfd0 0x7f62100b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36746/0 (socket says 172.21.15.181:36746) 2023-10-16T09:44:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f62154cd700 1 -- 172.21.15.181:0/2379691218 learned_addr learned my addr 172.21.15.181:0/2379691218 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f6214ccc700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f621006db80 0x7f62101546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.930+0000 7f620ffff700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62100af450 0x7f62100b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f62154cd700 1 -- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f621006db80 msgr2=0x7f62101546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f62154cd700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f621006db80 0x7f62101546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f62154cd700 1 -- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62100af450 msgr2=0x7f62100b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f62154cd700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62100af450 0x7f62100b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f62154cd700 1 -- 172.21.15.181:0/2379691218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000094a0 con 0x7f621015dfd0 2023-10-16T09:44:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f620ffff700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62100af450 0x7f62100b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f62154cd700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621015dfd0 0x7f62100b1280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f620800cbc0 tx=0x7f620800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f620d7fa700 1 -- 172.21.15.181:0/2379691218 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800aaa0 con 0x7f621015dfd0 2023-10-16T09:44:16.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f620d7fa700 1 -- 172.21.15.181:0/2379691218 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620800ac00 con 0x7f621015dfd0 2023-10-16T09:44:16.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8009430 con 0x7f621015dfd0 2023-10-16T09:44:16.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f620d7fa700 1 -- 172.21.15.181:0/2379691218 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208012400 con 0x7f621015dfd0 2023-10-16T09:44:16.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.931+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62100b07a0 con 0x7f621015dfd0 2023-10-16T09:44:16.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.932+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62100b0f60 con 0x7f621015dfd0 2023-10-16T09:44:16.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.932+0000 7f620d7fa700 1 -- 172.21.15.181:0/2379691218 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f620801b3b0 con 0x7f621015dfd0 2023-10-16T09:44:16.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.933+0000 7f620d7fa700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f61f006af00 0x7f61f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:16.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.933+0000 7f620d7fa700 1 -- 172.21.15.181:0/2379691218 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62080890b0 con 0x7f621015dfd0 2023-10-16T09:44:16.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.933+0000 7f6214ccc700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f61f006af00 0x7f61f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:16.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.933+0000 7f6214ccc700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f61f006af00 0x7f61f006d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009b70 tx=0x7f61f8008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:16.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:16.937+0000 7f620d7fa700 1 -- 172.21.15.181:0/2379691218 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f62080542a0 con 0x7f621015dfd0 2023-10-16T09:44:17.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.175+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6210067d20 con 0x7f621015dfd0 2023-10-16T09:44:17.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.175+0000 7f620d7fa700 1 -- 172.21.15.181:0/2379691218 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62080578c0 con 0x7f621015dfd0 2023-10-16T09:44:17.177 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:44:17.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f61f006af00 msgr2=0x7f61f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:17.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f61f006af00 0x7f61f006d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009b70 tx=0x7f61f8008070 comp rx=0 tx=0).stop 2023-10-16T09:44:17.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621015dfd0 msgr2=0x7f62100b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:17.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621015dfd0 0x7f62100b1280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f620800cbc0 tx=0x7f620800ced0 comp rx=0 tx=0).stop 2023-10-16T09:44:17.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 shutdown_connections 2023-10-16T09:44:17.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f61f006af00 0x7f61f006d3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:17.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f621006db80 0x7f62101546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:17.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621015dfd0 0x7f62100b1280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:17.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 --2- 172.21.15.181:0/2379691218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62100af450 0x7f62100b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:17.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 >> 172.21.15.181:0/2379691218 conn(0x7f62100d04a0 msgr2=0x7f62100addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:17.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 shutdown_connections 2023-10-16T09:44:17.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:17.177+0000 7f6216f30700 1 -- 172.21.15.181:0/2379691218 wait complete. 2023-10-16T09:44:17.191 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:44:17.192 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:44:17.192 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T09:44:17.192 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T09:44:17.192 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:44:17.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:44:17.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:44:17.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:44:17.194 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:44:17.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.194 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T09:44:17.211 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:17.212 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:17.212 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:17.212 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:17.212 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:17.213 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:17.213 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:17.213 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:17.214 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:17.214 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:17.214 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:17.214 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:17.214 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:44:17.215 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:17.215 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:44:17.215 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42413sec preferred_lft 42413sec 2023-10-16T09:44:17.215 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T09:44:17.215 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:17.216 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T09:44:17.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.217 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:17.217 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:17.217 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2023-10-16T09:44:17.217 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2023-10-16T09:44:17.218 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2023-10-16T09:44:17.218 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T09:44:17.218 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:17.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.380 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:44:17.408 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T09:44:17.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.409 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T09:44:17.492 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T09:44:17.492 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:44:17.492 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:44:17.492 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:44:17.493 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:44:17.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.494 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:17.494 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:17.494 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:44:17.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:44:17.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:44:17.495 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:17.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.731 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T09:44:17.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.749 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-10-16T09:44:17.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.806 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:17.807 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:17.807 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:44:17.807 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-16T09:44:17.808 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:17.899 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-16T09:44:17.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:17.900 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:17.900 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:17.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-16T09:44:17.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T09:44:17.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-16T09:44:17.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-16T09:44:17.902 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-16T09:44:17.902 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:18.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:18.108 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:18.108 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:18.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T09:44:18.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T09:44:18.109 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:18.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:18.211 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T09:44:18.231 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:44:18.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:18.231 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:44:18.287 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T09:44:18.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:18.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:44:18.379 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:44:18.379 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:44:18.379 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:44:18.498 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:44:18.499 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:44:18.499 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:44:18.499 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:18.499 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:18.573 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:44:18.574 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:44:18.574 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:44:18.574 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:18.574 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:18.574 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T09:44:18.575 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T09:44:18.575 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T09:44:18.576 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:44:18.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:18.577 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:44:18.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:18.604 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:18.696 INFO:teuthology.orchestra.run.smithi181.stdout:client4493 2023-10-16T09:44:18.697 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:18.697 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:18.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:18.715 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:18.805 INFO:teuthology.orchestra.run.smithi181.stdout:client4493 2023-10-16T09:44:18.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:18.806 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:18.851 INFO:teuthology.orchestra.run.smithi181.stdout:client4493 2023-10-16T09:44:18.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:18.852 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0cf369f-f4d9-40c8-9248-54aac3970136.client4493/status') 2023-10-16T09:44:18.904 INFO:teuthology.orchestra.run.smithi181.stdout:instance: client.4493 (3)172.21.15.181:0/516980899 2023-10-16T09:44:18.905 INFO:teuthology.orchestra.run.smithi181.stdout:blocklisted: false 2023-10-16T09:44:18.906 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2023-10-16T09:44:18.906 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2023-10-16T09:44:18.906 INFO:teuthology.orchestra.run.smithi181.stderr:70 bytes copied, 3.0087e-05 s, 2.3 MB/s 2023-10-16T09:44:18.907 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-16T09:44:18.907 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:44:18.907 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T09:44:18.908 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:44:19.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.027+0000 7f2e1d18f700 1 Processor -- start 2023-10-16T09:44:19.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.027+0000 7f2e1d18f700 1 -- start start 2023-10-16T09:44:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e1d18f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e1815de50 0x7f2e180b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e1d18f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2e1806bdb0 0x7f2e180cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e1d18f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e18161fc0 con 0x7f2e18161ac0 2023-10-16T09:44:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e1d18f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e180b4010 con 0x7f2e1815de50 2023-10-16T09:44:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e1d18f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e181622a0 con 0x7f2e1806bdb0 2023-10-16T09:44:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e1815de50 0x7f2e180b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e1815de50 0x7f2e180b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36766/0 (socket says 172.21.15.181:36766) 2023-10-16T09:44:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 -- 172.21.15.181:0/4059814563 learned_addr learned my addr 172.21.15.181:0/4059814563 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e1659c700 1 --2- 172.21.15.181:0/4059814563 >> v2:172.21.15.190:3300/0 conn(0x7f2e1806bdb0 0x7f2e180cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 -- 172.21.15.181:0/4059814563 >> v2:172.21.15.190:3300/0 conn(0x7f2e1806bdb0 msgr2=0x7f2e180cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 --2- 172.21.15.181:0/4059814563 >> v2:172.21.15.190:3300/0 conn(0x7f2e1806bdb0 0x7f2e180cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 -- 172.21.15.181:0/4059814563 >> v1:172.21.15.190:6789/0 conn(0x7f2e18161ac0 legacy=0x7f2e180cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 -- 172.21.15.181:0/4059814563 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18162580 con 0x7f2e1815de50 2023-10-16T09:44:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.028+0000 7f2e16d9d700 1 --2- 172.21.15.181:0/4059814563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1815de50 0x7f2e180b3ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e180c50a0 tx=0x7f2e0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f285923c747505e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e15d9b700 1 -- 172.21.15.181:0/4059814563 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c011830 con 0x7f2e1815de50 2023-10-16T09:44:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e15d9b700 1 -- 172.21.15.181:0/4059814563 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c011990 con 0x7f2e1815de50 2023-10-16T09:44:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e15d9b700 1 -- 172.21.15.181:0/4059814563 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c011d20 con 0x7f2e1815de50 2023-10-16T09:44:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 -- 172.21.15.181:0/4059814563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1815de50 msgr2=0x7f2e180b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/4059814563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1815de50 0x7f2e180b3ad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e180c50a0 tx=0x7f2e0c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 -- 172.21.15.181:0/4059814563 shutdown_connections 2023-10-16T09:44:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/4059814563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1815de50 0x7f2e180b3ad0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/4059814563 >> v2:172.21.15.190:3300/0 conn(0x7f2e1806bdb0 0x7f2e180cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 -- 172.21.15.181:0/4059814563 >> 172.21.15.181:0/4059814563 conn(0x7f2e180665a0 msgr2=0x7f2e18066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 -- 172.21.15.181:0/4059814563 shutdown_connections 2023-10-16T09:44:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.029+0000 7f2e1d18f700 1 -- 172.21.15.181:0/4059814563 wait complete. 2023-10-16T09:44:19.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.031+0000 7f2e1d18f700 1 Processor -- start 2023-10-16T09:44:19.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.031+0000 7f2e1d18f700 1 -- start start 2023-10-16T09:44:19.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1d18f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1806bdb0 0x7f2e1815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:19.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1d18f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1815de50 0x7f2e18151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:19.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1d18f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18161ac0 0x7f2e180ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:19.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1d18f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180c5310 con 0x7f2e1806bdb0 2023-10-16T09:44:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1d18f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180c55f0 con 0x7f2e1815de50 2023-10-16T09:44:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1d18f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180c58d0 con 0x7f2e18161ac0 2023-10-16T09:44:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e16d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1806bdb0 0x7f2e1815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e16d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1806bdb0 0x7f2e1815eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36770/0 (socket says 172.21.15.181:36770) 2023-10-16T09:44:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e16d9d700 1 -- 172.21.15.181:0/3947113319 learned_addr learned my addr 172.21.15.181:0/3947113319 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1759e700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18161ac0 0x7f2e180ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.032+0000 7f2e1659c700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1815de50 0x7f2e18151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e16d9d700 1 -- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18161ac0 msgr2=0x7f2e180ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e16d9d700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18161ac0 0x7f2e180ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e16d9d700 1 -- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1815de50 msgr2=0x7f2e18151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e16d9d700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1815de50 0x7f2e18151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e16d9d700 1 -- 172.21.15.181:0/3947113319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009550 con 0x7f2e1806bdb0 2023-10-16T09:44:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e16d9d700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1806bdb0 0x7f2e1815eb10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2dff7fe700 1 -- 172.21.15.181:0/3947113319 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c004380 con 0x7f2e1806bdb0 2023-10-16T09:44:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2dff7fe700 1 -- 172.21.15.181:0/3947113319 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c01f070 con 0x7f2e1806bdb0 2023-10-16T09:44:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e180c5bb0 con 0x7f2e1806bdb0 2023-10-16T09:44:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2dff7fe700 1 -- 172.21.15.181:0/3947113319 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0036a0 con 0x7f2e1806bdb0 2023-10-16T09:44:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.033+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e18151960 con 0x7f2e1806bdb0 2023-10-16T09:44:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.034+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1815d670 con 0x7f2e1806bdb0 2023-10-16T09:44:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.034+0000 7f2dff7fe700 1 -- 172.21.15.181:0/3947113319 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2e0c021030 con 0x7f2e1806bdb0 2023-10-16T09:44:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.035+0000 7f2dff7fe700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e0406af30 0x7f2e0406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.035+0000 7f2dff7fe700 1 -- 172.21.15.181:0/3947113319 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e0c08a980 con 0x7f2e1806bdb0 2023-10-16T09:44:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.035+0000 7f2e1659c700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e0406af30 0x7f2e0406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.035+0000 7f2e1659c700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e0406af30 0x7f2e0406d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e180c9840 tx=0x7f2e08009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:19.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.039+0000 7f2dff7fe700 1 -- 172.21.15.181:0/3947113319 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2e0c055a40 con 0x7f2e1806bdb0 2023-10-16T09:44:19.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.279+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e180c5450 con 0x7f2e1806bdb0 2023-10-16T09:44:19.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.279+0000 7f2dff7fe700 1 -- 172.21.15.181:0/3947113319 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2e180c5450 con 0x7f2e1806bdb0 2023-10-16T09:44:19.283 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:44:19.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e0406af30 msgr2=0x7f2e0406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:19.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e0406af30 0x7f2e0406d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e180c9840 tx=0x7f2e08009210 comp rx=0 tx=0).stop 2023-10-16T09:44:19.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1806bdb0 msgr2=0x7f2e1815eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:19.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1806bdb0 0x7f2e1815eb10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000c00 tx=0x7f2e0c004750 comp rx=0 tx=0).stop 2023-10-16T09:44:19.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 shutdown_connections 2023-10-16T09:44:19.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f2e0406af30 0x7f2e0406d3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18161ac0 0x7f2e180ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1806bdb0 0x7f2e1815eb10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 --2- 172.21.15.181:0/3947113319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e1815de50 0x7f2e18151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:19.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 >> 172.21.15.181:0/3947113319 conn(0x7f2e180665a0 msgr2=0x7f2e180b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:19.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 shutdown_connections 2023-10-16T09:44:19.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:19.282+0000 7f2e1d18f700 1 -- 172.21.15.181:0/3947113319 wait complete. 2023-10-16T09:44:19.296 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:44:19.296 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:44:19.296 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T09:44:19.297 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-16T09:44:19.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T09:44:19.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:44:19.298 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:44:19.298 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:44:19.298 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-16T09:44:19.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.299 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T09:44:19.317 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:19.317 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:19.317 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:19.318 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.318 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:19.318 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.318 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:19.319 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:19.319 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:19.319 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:19.319 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:19.319 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:19.320 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:44:19.320 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:19.320 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:44:19.320 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42411sec preferred_lft 42411sec 2023-10-16T09:44:19.320 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T09:44:19.321 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.321 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:44:19.321 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:19.321 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:44:19.321 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.322 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T09:44:19.322 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.322 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:44:19.322 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:44:19.323 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link tentative 2023-10-16T09:44:19.323 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.324 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:44:19.382 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T09:44:19.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.384 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T09:44:19.442 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T09:44:19.443 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:44:19.443 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:44:19.443 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:44:19.443 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:44:19.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.444 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:19.445 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:19.445 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:44:19.445 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:44:19.446 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:44:19.446 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:19.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.548 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T09:44:19.564 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T09:44:19.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.565 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-10-16T09:44:19.620 INFO:teuthology.orchestra.run.smithi181.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-16T09:44:19.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.621 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:19.622 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:19.622 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T09:44:19.622 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-16T09:44:19.623 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:19.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.714 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T09:44:19.756 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:19.756 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:19.756 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:19.756 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.757 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:19.757 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.757 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:44:19.757 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 92:55:bd:6a:03:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:44:19.757 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:44:19.758 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.758 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::9055:bdff:fe6a:3e7/64 scope link tentative 2023-10-16T09:44:19.758 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.772 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T09:44:19.849 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:19.849 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:19.849 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:19.849 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.850 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:19.850 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.850 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:44:19.850 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 92:55:bd:6a:03:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:44:19.850 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:44:19.851 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.851 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::9055:bdff:fe6a:3e7/64 scope link tentative 2023-10-16T09:44:19.851 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:19.859 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-16T09:44:19.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:19.860 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:19.860 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:19.860 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-16T09:44:19.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T09:44:19.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-16T09:44:19.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-16T09:44:19.862 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-16T09:44:19.862 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:20.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:20.098 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:44:20.098 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:44:20.099 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.1 up 2023-10-16T09:44:20.099 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T09:44:20.099 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:44:20.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:20.220 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-16T09:44:20.240 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-16T09:44:20.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:20.241 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-16T09:44:20.297 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T09:44:20.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:20.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-16T09:44:20.382 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-16T09:44:20.383 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-16T09:44:20.383 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:44:20.391 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-16T09:44:20.391 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-16T09:44:20.392 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:44:20.392 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:20.392 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:20.466 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-16T09:44:20.466 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-16T09:44:20.466 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:44:20.466 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:20.467 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:20.467 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T09:44:20.467 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T09:44:20.467 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T09:44:20.468 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:44:20.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:20.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-16T09:44:20.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:20.495 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:20.567 INFO:teuthology.orchestra.run.smithi181.stdout:client4499 2023-10-16T09:44:20.567 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:20.567 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:20.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:20.625 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:20.697 INFO:teuthology.orchestra.run.smithi181.stdout:client4499 2023-10-16T09:44:20.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:20.697 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:20.768 INFO:teuthology.orchestra.run.smithi181.stdout:client4499 2023-10-16T09:44:20.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:20.769 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0cf369f-f4d9-40c8-9248-54aac3970136.client4499/status') 2023-10-16T09:44:20.848 INFO:teuthology.orchestra.run.smithi181.stdout:instance: client.4499 (3)172.21.15.181:0/3703383652 2023-10-16T09:44:20.849 INFO:teuthology.orchestra.run.smithi181.stdout:blocklisted: false 2023-10-16T09:44:20.850 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2023-10-16T09:44:20.850 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2023-10-16T09:44:20.850 INFO:teuthology.orchestra.run.smithi181.stderr:71 bytes copied, 2.566e-05 s, 2.8 MB/s 2023-10-16T09:44:20.850 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-16T09:44:20.851 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:44:20.851 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T09:44:20.851 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:44:21.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.008+0000 7f409ff14700 1 Processor -- start 2023-10-16T09:44:21.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.008+0000 7f409ff14700 1 -- start start 2023-10-16T09:44:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409ff14700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40980ce7f0 0x7f409806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409ff14700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40980cc470 con 0x7f40980d0350 2023-10-16T09:44:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409ff14700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409806ca30 con 0x7f40980d2390 2023-10-16T09:44:21.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409ff14700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409806cd10 con 0x7f40980ce7f0 2023-10-16T09:44:21.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409dcb0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40980ce7f0 0x7f409806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:21.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409e4b1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f40980d0350 0x7f40980cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47862/0 (socket says 172.21.15.181:47862) 2023-10-16T09:44:21.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409e4b1700 1 -- 172.21.15.181:0/291228846 learned_addr learned my addr 172.21.15.181:0/291228846 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:21.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.009+0000 7f409dcb0700 1 -- 172.21.15.181:0/291228846 >> v1:172.21.15.190:6789/0 conn(0x7f40980d2390 legacy=0x7f40980cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ccae700 1 -- 172.21.15.181:0/291228846 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019442065 0 0) 0x7f40980cc470 con 0x7f40980d0350 2023-10-16T09:44:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409dcb0700 1 -- 172.21.15.181:0/291228846 >> v1:172.21.15.181:6789/0 conn(0x7f40980d0350 legacy=0x7f40980cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409dcb0700 1 -- 172.21.15.181:0/291228846 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409806cff0 con 0x7f40980ce7f0 2023-10-16T09:44:21.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409dcb0700 1 --2- 172.21.15.181:0/291228846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980ce7f0 0x7f409806c100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4094009820 tx=0x7f4094009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dfd6480b5b6dd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ccae700 1 -- 172.21.15.181:0/291228846 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094005e30 con 0x7f40980ce7f0 2023-10-16T09:44:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ccae700 1 -- 172.21.15.181:0/291228846 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094006030 con 0x7f40980ce7f0 2023-10-16T09:44:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ccae700 1 -- 172.21.15.181:0/291228846 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940063e0 con 0x7f40980ce7f0 2023-10-16T09:44:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ff14700 1 -- 172.21.15.181:0/291228846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980ce7f0 msgr2=0x7f409806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ff14700 1 --2- 172.21.15.181:0/291228846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980ce7f0 0x7f409806c100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4094009820 tx=0x7f4094009b30 comp rx=0 tx=0).stop 2023-10-16T09:44:21.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ff14700 1 -- 172.21.15.181:0/291228846 shutdown_connections 2023-10-16T09:44:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ff14700 1 --2- 172.21.15.181:0/291228846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980ce7f0 0x7f409806c100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.010+0000 7f409ff14700 1 -- 172.21.15.181:0/291228846 >> 172.21.15.181:0/291228846 conn(0x7f409809a9a0 msgr2=0x7f40980d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.011+0000 7f409ff14700 1 -- 172.21.15.181:0/291228846 shutdown_connections 2023-10-16T09:44:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.011+0000 7f409ff14700 1 -- 172.21.15.181:0/291228846 wait complete. 2023-10-16T09:44:21.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.012+0000 7f409ff14700 1 Processor -- start 2023-10-16T09:44:21.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.012+0000 7f409ff14700 1 -- start start 2023-10-16T09:44:21.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.012+0000 7f409ff14700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980ce7f0 0x7f409814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.012+0000 7f409ff14700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40980d0350 0x7f40980c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409ff14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980d2390 0x7f40980ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409ff14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40980c5f60 con 0x7f40980d2390 2023-10-16T09:44:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409ff14700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409814fe80 con 0x7f40980d0350 2023-10-16T09:44:21.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409ff14700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40980c5350 con 0x7f40980ce7f0 2023-10-16T09:44:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980d2390 0x7f40980ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980d2390 0x7f40980ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36792/0 (socket says 172.21.15.181:36792) 2023-10-16T09:44:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 -- 172.21.15.181:0/3227897853 learned_addr learned my addr 172.21.15.181:0/3227897853 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:21.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409dcb0700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980ce7f0 0x7f409814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409d4af700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40980d0350 0x7f40980c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 -- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980ce7f0 msgr2=0x7f409814f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980ce7f0 0x7f409814f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 -- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40980d0350 msgr2=0x7f40980c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40980d0350 0x7f40980c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:21.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 -- 172.21.15.181:0/3227897853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009430 con 0x7f40980d2390 2023-10-16T09:44:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409d4af700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40980d0350 0x7f40980c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409e4b1700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980d2390 0x7f40980ca700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f409000cc10 tx=0x7f409000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f408e7fc700 1 -- 172.21.15.181:0/3227897853 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409000ab50 con 0x7f40980d2390 2023-10-16T09:44:21.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f408e7fc700 1 -- 172.21.15.181:0/3227897853 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409000acd0 con 0x7f40980d2390 2023-10-16T09:44:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40980c5630 con 0x7f40980d2390 2023-10-16T09:44:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f408e7fc700 1 -- 172.21.15.181:0/3227897853 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090007310 con 0x7f40980d2390 2023-10-16T09:44:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.013+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40980c5bf0 con 0x7f40980d2390 2023-10-16T09:44:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.014+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40980c6110 con 0x7f40980d2390 2023-10-16T09:44:21.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.015+0000 7f408e7fc700 1 -- 172.21.15.181:0/3227897853 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4090007470 con 0x7f40980d2390 2023-10-16T09:44:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.015+0000 7f408e7fc700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f407c06af00 0x7f407c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.015+0000 7f408e7fc700 1 -- 172.21.15.181:0/3227897853 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4090089360 con 0x7f40980d2390 2023-10-16T09:44:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.015+0000 7f409dcb0700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f407c06af00 0x7f407c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:21.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.016+0000 7f409dcb0700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f407c06af00 0x7f407c06d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4094000c00 tx=0x7f4094008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:21.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.020+0000 7f408e7fc700 1 -- 172.21.15.181:0/3227897853 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4090054410 con 0x7f40980d2390 2023-10-16T09:44:21.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.258+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4098067d20 con 0x7f40980d2390 2023-10-16T09:44:21.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.259+0000 7f408e7fc700 1 -- 172.21.15.181:0/3227897853 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4090057a30 con 0x7f40980d2390 2023-10-16T09:44:21.261 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:44:21.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f407c06af00 msgr2=0x7f407c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:21.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f407c06af00 0x7f407c06d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4094000c00 tx=0x7f4094008070 comp rx=0 tx=0).stop 2023-10-16T09:44:21.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980d2390 msgr2=0x7f40980ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:21.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980d2390 0x7f40980ca700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f409000cc10 tx=0x7f409000cf20 comp rx=0 tx=0).stop 2023-10-16T09:44:21.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 shutdown_connections 2023-10-16T09:44:21.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f407c06af00 0x7f407c06d3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:21.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40980ce7f0 0x7f409814f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:21.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40980d2390 0x7f40980ca700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:21.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 --2- 172.21.15.181:0/3227897853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40980d0350 0x7f40980c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:21.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 >> 172.21.15.181:0/3227897853 conn(0x7f409809a9a0 msgr2=0x7f4098070160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:21.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 shutdown_connections 2023-10-16T09:44:21.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:21.261+0000 7f409ff14700 1 -- 172.21.15.181:0/3227897853 wait complete. 2023-10-16T09:44:21.275 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:44:21.275 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:44:21.275 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-10-16T09:44:21.276 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-16T09:44:21.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T09:44:21.276 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:44:21.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:44:21.277 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:44:21.277 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-16T09:44:21.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.278 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-10-16T09:44:21.299 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:21.300 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:21.300 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:21.300 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:21.300 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:21.300 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:21.301 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:21.301 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:21.301 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:21.301 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:21.301 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:21.302 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:21.302 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:44:21.302 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:21.302 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:44:21.302 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42412sec preferred_lft 42412sec 2023-10-16T09:44:21.303 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ccc1:8389:e10a:d1c6/64 scope link noprefixroute 2023-10-16T09:44:21.303 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:21.303 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T09:44:21.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.304 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:21.304 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:21.304 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2023-10-16T09:44:21.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2023-10-16T09:44:21.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2023-10-16T09:44:21.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T09:44:21.306 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:21.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.480 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:44:21.508 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-10-16T09:44:21.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.510 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-10-16T09:44:21.582 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-10-16T09:44:21.582 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:44:21.582 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:44:21.582 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:44:21.583 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:44:21.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.583 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:21.584 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:21.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:44:21.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:44:21.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:44:21.585 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:21.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.817 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-10-16T09:44:21.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.834 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-10-16T09:44:21.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.891 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:21.891 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:21.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T09:44:21.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-16T09:44:21.892 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:21.986 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-16T09:44:21.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:21.987 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:21.987 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:21.987 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-16T09:44:21.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T09:44:21.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-16T09:44:21.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-16T09:44:21.989 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-16T09:44:21.989 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:22.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:22.177 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:22.178 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:22.178 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2023-10-16T09:44:22.178 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T09:44:22.179 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:22.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:22.278 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-16T09:44:22.297 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-16T09:44:22.298 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:22.298 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-16T09:44:22.353 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T09:44:22.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:22.354 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-16T09:44:22.455 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-16T09:44:22.455 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-16T09:44:22.455 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2023-10-16T09:44:22.560 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-16T09:44:22.560 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-16T09:44:22.560 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2023-10-16T09:44:22.561 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:22.561 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:22.603 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-16T09:44:22.604 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-16T09:44:22.604 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2023-10-16T09:44:22.604 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:22.604 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:22.605 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T09:44:22.605 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T09:44:22.605 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T09:44:22.619 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:44:22.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:22.619 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-16T09:44:22.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:22.647 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:22.718 INFO:teuthology.orchestra.run.smithi190.stdout:client4505 2023-10-16T09:44:22.718 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:22.718 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:22.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:22.776 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:22.847 INFO:teuthology.orchestra.run.smithi190.stdout:client4505 2023-10-16T09:44:22.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:22.848 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:22.919 INFO:teuthology.orchestra.run.smithi190.stdout:client4505 2023-10-16T09:44:22.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:22.921 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0cf369f-f4d9-40c8-9248-54aac3970136.client4505/status') 2023-10-16T09:44:22.999 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4505 (3)192.168.0.1:0/2438537688 2023-10-16T09:44:23.000 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2023-10-16T09:44:23.000 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2023-10-16T09:44:23.000 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2023-10-16T09:44:23.000 INFO:teuthology.orchestra.run.smithi190.stderr:69 bytes copied, 3.7427e-05 s, 1.8 MB/s 2023-10-16T09:44:23.001 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-16T09:44:23.002 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T09:44:23.002 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T09:44:23.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:44:23.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.135+0000 7efee5d3c700 1 Processor -- start 2023-10-16T09:44:23.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.135+0000 7efee5d3c700 1 -- start start 2023-10-16T09:44:23.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efee5d3c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee015b990 0x7efee00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:23.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efee5d3c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efee00cd2d0 0x7efee00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:23.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efee5d3c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee00cc510 con 0x7efee00cfc60 2023-10-16T09:44:23.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efee5d3c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00d24f0 con 0x7efee00cd2d0 2023-10-16T09:44:23.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efee5d3c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00cc7f0 con 0x7efee015b990 2023-10-16T09:44:23.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efedffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efee00cfc60 0x7efee00ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47878/0 (socket says 172.21.15.181:47878) 2023-10-16T09:44:23.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efedffff700 1 -- 172.21.15.181:0/4103241892 learned_addr learned my addr 172.21.15.181:0/4103241892 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:23.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efedf7fe700 1 --2- 172.21.15.181:0/4103241892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee015b990 0x7efee00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:23.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.136+0000 7efedeffd700 1 --2- 172.21.15.181:0/4103241892 >> v2:172.21.15.190:3300/0 conn(0x7efee00cd2d0 0x7efee00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:23.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efede7fc700 1 -- 172.21.15.181:0/4103241892 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409342998 0 0) 0x7efee00cc510 con 0x7efee00cfc60 2023-10-16T09:44:23.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efede7fc700 1 -- 172.21.15.181:0/4103241892 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec80035b0 con 0x7efee00cfc60 2023-10-16T09:44:23.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efedf7fe700 1 -- 172.21.15.181:0/4103241892 >> v2:172.21.15.190:3300/0 conn(0x7efee00cd2d0 msgr2=0x7efee00d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:23.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efedf7fe700 1 --2- 172.21.15.181:0/4103241892 >> v2:172.21.15.190:3300/0 conn(0x7efee00cd2d0 0x7efee00d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efedf7fe700 1 -- 172.21.15.181:0/4103241892 >> v1:172.21.15.181:6789/0 conn(0x7efee00cfc60 legacy=0x7efee00ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:23.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efedf7fe700 1 -- 172.21.15.181:0/4103241892 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00ccad0 con 0x7efee015b990 2023-10-16T09:44:23.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efedeffd700 1 --2- 172.21.15.181:0/4103241892 >> v2:172.21.15.190:3300/0 conn(0x7efee00cd2d0 0x7efee00d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:23.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efedf7fe700 1 --2- 172.21.15.181:0/4103241892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee015b990 0x7efee00ce8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efed0009820 tx=0x7efed0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9cad4531f66ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:23.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efede7fc700 1 -- 172.21.15.181:0/4103241892 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0005e30 con 0x7efee015b990 2023-10-16T09:44:23.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efede7fc700 1 -- 172.21.15.181:0/4103241892 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0006030 con 0x7efee015b990 2023-10-16T09:44:23.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.137+0000 7efede7fc700 1 -- 172.21.15.181:0/4103241892 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00063c0 con 0x7efee015b990 2023-10-16T09:44:23.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 -- 172.21.15.181:0/4103241892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee015b990 msgr2=0x7efee00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:23.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 --2- 172.21.15.181:0/4103241892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee015b990 0x7efee00ce8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efed0009820 tx=0x7efed0009b30 comp rx=0 tx=0).stop 2023-10-16T09:44:23.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 -- 172.21.15.181:0/4103241892 shutdown_connections 2023-10-16T09:44:23.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 --2- 172.21.15.181:0/4103241892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee015b990 0x7efee00ce8f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 --2- 172.21.15.181:0/4103241892 >> v2:172.21.15.190:3300/0 conn(0x7efee00cd2d0 0x7efee00d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 -- 172.21.15.181:0/4103241892 >> 172.21.15.181:0/4103241892 conn(0x7efee009a9a0 msgr2=0x7efee00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:23.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 -- 172.21.15.181:0/4103241892 shutdown_connections 2023-10-16T09:44:23.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.138+0000 7efee5d3c700 1 -- 172.21.15.181:0/4103241892 wait complete. 2023-10-16T09:44:23.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.139+0000 7efee5d3c700 1 Processor -- start 2023-10-16T09:44:23.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.139+0000 7efee5d3c700 1 -- start start 2023-10-16T09:44:23.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efee5d3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efee00cd2d0 0x7efee01464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:23.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efee5d3c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee00cfc60 0x7efee0145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:23.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efee5d3c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee015b990 0x7efee006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:23.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efee5d3c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0146250 con 0x7efee00cd2d0 2023-10-16T09:44:23.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efee5d3c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00c4f50 con 0x7efee015b990 2023-10-16T09:44:23.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efee5d3c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00c5180 con 0x7efee00cfc60 2023-10-16T09:44:23.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efee00cd2d0 0x7efee01464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:23.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efee00cd2d0 0x7efee01464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36802/0 (socket says 172.21.15.181:36802) 2023-10-16T09:44:23.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 -- 172.21.15.181:0/2669781439 learned_addr learned my addr 172.21.15.181:0/2669781439 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:23.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedeffd700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee00cfc60 0x7efee0145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:23.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedffff700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee015b990 0x7efee006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:23.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 -- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee00cfc60 msgr2=0x7efee0145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:23.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee00cfc60 0x7efee0145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 -- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee015b990 msgr2=0x7efee006f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:23.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee015b990 0x7efee006f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 -- 172.21.15.181:0/2669781439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40094a0 con 0x7efee00cd2d0 2023-10-16T09:44:23.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.140+0000 7efedf7fe700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efee00cd2d0 0x7efee01464b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efed0000c00 tx=0x7efed0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:23.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.141+0000 7efec7fff700 1 -- 172.21.15.181:0/2669781439 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00061a0 con 0x7efee00cd2d0 2023-10-16T09:44:23.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.141+0000 7efec7fff700 1 -- 172.21.15.181:0/2669781439 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0020070 con 0x7efee00cd2d0 2023-10-16T09:44:23.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.141+0000 7efec7fff700 1 -- 172.21.15.181:0/2669781439 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0006560 con 0x7efee00cd2d0 2023-10-16T09:44:23.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.141+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed0009430 con 0x7efee00cd2d0 2023-10-16T09:44:23.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.141+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee00c5740 con 0x7efee00cd2d0 2023-10-16T09:44:23.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.142+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee00b2fc0 con 0x7efee00cd2d0 2023-10-16T09:44:23.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.142+0000 7efec7fff700 1 -- 172.21.15.181:0/2669781439 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efed0022030 con 0x7efee00cd2d0 2023-10-16T09:44:23.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.142+0000 7efec7fff700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efecc06af00 0x7efecc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:23.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.142+0000 7efec7fff700 1 -- 172.21.15.181:0/2669781439 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efed008ab00 con 0x7efee00cd2d0 2023-10-16T09:44:23.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.143+0000 7efedeffd700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efecc06af00 0x7efecc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:23.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.143+0000 7efedeffd700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efecc06af00 0x7efecc06d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efee00d2060 tx=0x7efed4009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:23.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.147+0000 7efec7fff700 1 -- 172.21.15.181:0/2669781439 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efed0055bb0 con 0x7efee00cd2d0 2023-10-16T09:44:23.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.382+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efee0067d20 con 0x7efee00cd2d0 2023-10-16T09:44:23.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.382+0000 7efec7fff700 1 -- 172.21.15.181:0/2669781439 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7efed00591d0 con 0x7efee00cd2d0 2023-10-16T09:44:23.384 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:44:23.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efecc06af00 msgr2=0x7efecc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efecc06af00 0x7efecc06d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efee00d2060 tx=0x7efed4009210 comp rx=0 tx=0).stop 2023-10-16T09:44:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efee00cd2d0 msgr2=0x7efee01464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efee00cd2d0 0x7efee01464b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efed0000c00 tx=0x7efed0004750 comp rx=0 tx=0).stop 2023-10-16T09:44:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 shutdown_connections 2023-10-16T09:44:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efecc06af00 0x7efecc06d3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efee015b990 0x7efee006f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efee00cd2d0 0x7efee01464b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 --2- 172.21.15.181:0/2669781439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efee00cfc60 0x7efee0145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 >> 172.21.15.181:0/2669781439 conn(0x7efee009a9a0 msgr2=0x7efee0070170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 shutdown_connections 2023-10-16T09:44:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:23.384+0000 7efee5d3c700 1 -- 172.21.15.181:0/2669781439 wait complete. 2023-10-16T09:44:23.398 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T09:44:23.398 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:44:23.399 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-10-16T09:44:23.399 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-16T09:44:23.399 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T09:44:23.399 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T09:44:23.400 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T09:44:23.400 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:44:23.400 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-16T09:44:23.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.401 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-10-16T09:44:23.418 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:23.418 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:23.419 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:23.419 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.419 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:23.419 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.419 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:23.420 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:23.420 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:23.420 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:23.420 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:44:23.421 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:23.421 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:44:23.421 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:23.422 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:44:23.422 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42410sec preferred_lft 42410sec 2023-10-16T09:44:23.422 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ccc1:8389:e10a:d1c6/64 scope link noprefixroute 2023-10-16T09:44:23.423 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.423 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:44:23.423 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether ba:2e:dd:01:fa:18 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:44:23.424 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:44:23.424 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.424 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::b079:6ff:fea1:651b/64 scope link 2023-10-16T09:44:23.425 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.425 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:44:23.426 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether ba:2e:dd:01:fa:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T09:44:23.426 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::b82e:ddff:fe01:fa18/64 scope link tentative 2023-10-16T09:44:23.426 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.428 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:44:23.484 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-10-16T09:44:23.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.486 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-10-16T09:44:23.544 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-10-16T09:44:23.544 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:44:23.545 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:44:23.545 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:44:23.545 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:44:23.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.546 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:23.546 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:23.546 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:44:23.547 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:44:23.547 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:44:23.547 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:23.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.649 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-10-16T09:44:23.666 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T09:44:23.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.667 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-10-16T09:44:23.722 INFO:teuthology.orchestra.run.smithi190.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-16T09:44:23.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.723 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:23.723 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:23.724 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T09:44:23.724 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-16T09:44:23.724 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:23.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.816 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T09:44:23.849 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:23.850 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:23.850 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:23.850 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.850 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:23.851 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.851 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:44:23.851 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 82:25:08:cc:43:55 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:44:23.851 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:44:23.851 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.851 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::8025:8ff:fecc:4355/64 scope link 2023-10-16T09:44:23.852 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.867 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T09:44:23.952 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:44:23.952 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:44:23.953 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:44:23.953 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.953 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-10-16T09:44:23.953 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.953 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:44:23.954 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 82:25:08:cc:43:55 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T09:44:23.954 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T09:44:23.954 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.954 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::8025:8ff:fecc:4355/64 scope link 2023-10-16T09:44:23.955 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:44:23.971 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-16T09:44:23.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:23.972 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:23.973 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:23.973 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-16T09:44:23.974 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T09:44:23.974 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-16T09:44:23.975 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-16T09:44:23.975 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-16T09:44:23.976 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:24.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:24.215 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-10-16T09:44:24.215 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-10-16T09:44:24.216 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.1 up 2023-10-16T09:44:24.216 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T09:44:24.216 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-10-16T09:44:24.321 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:24.322 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-16T09:44:24.341 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-16T09:44:24.342 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:44:24.342 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-16T09:44:24.398 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T09:44:24.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:24.398 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-16T09:44:24.483 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-16T09:44:24.484 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-16T09:44:24.484 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2023-10-16T09:44:24.492 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-16T09:44:24.492 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-16T09:44:24.492 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2023-10-16T09:44:24.493 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:24.493 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:24.524 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-16T09:44:24.524 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-16T09:44:24.524 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2023-10-16T09:44:24.525 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:44:24.525 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:44:24.525 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T09:44:24.525 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T09:44:24.525 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T09:44:24.528 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:44:24.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:24.528 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-16T09:44:24.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:24.561 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:24.633 INFO:teuthology.orchestra.run.smithi190.stdout:client4514 2023-10-16T09:44:24.633 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:24.633 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:24.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:24.692 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:24.763 INFO:teuthology.orchestra.run.smithi190.stdout:client4514 2023-10-16T09:44:24.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:24.764 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:44:24.834 INFO:teuthology.orchestra.run.smithi190.stdout:client4514 2023-10-16T09:44:24.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:24.835 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0cf369f-f4d9-40c8-9248-54aac3970136.client4514/status') 2023-10-16T09:44:24.914 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4514 (3)192.168.0.2:0/531382334 2023-10-16T09:44:24.915 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2023-10-16T09:44:24.916 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2023-10-16T09:44:24.916 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2023-10-16T09:44:24.916 INFO:teuthology.orchestra.run.smithi190.stderr:68 bytes copied, 2.1868e-05 s, 3.1 MB/s 2023-10-16T09:44:24.916 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-16T09:44:24.935 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:25.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.065+0000 7fe99413d700 1 Processor -- start 2023-10-16T09:44:25.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.066+0000 7fe99413d700 1 -- start start 2023-10-16T09:44:25.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.066+0000 7fe99413d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15eab0 0x7fe98c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.066+0000 7fe99413d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe98c0cb0c0 0x7fe98c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.066+0000 7fe99413d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c06ed00 con 0x7fe98c15e6d0 2023-10-16T09:44:25.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.066+0000 7fe99413d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c0d02d0 con 0x7fe98c0cb0c0 2023-10-16T09:44:25.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.066+0000 7fe99413d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c15ad40 con 0x7fe98c15eab0 2023-10-16T09:44:25.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe9926da700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe98c0cb0c0 0x7fe98c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe991ed9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe98c15e6d0 0x7fe98c15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47888/0 (socket says 172.21.15.181:47888) 2023-10-16T09:44:25.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe991ed9700 1 -- 172.21.15.181:0/1995616401 learned_addr learned my addr 172.21.15.181:0/1995616401 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:25.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe9916d8700 1 --2- 172.21.15.181:0/1995616401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15eab0 0x7fe98c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe9926da700 1 -- 172.21.15.181:0/1995616401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15eab0 msgr2=0x7fe98c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe9926da700 1 --2- 172.21.15.181:0/1995616401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15eab0 0x7fe98c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe9926da700 1 -- 172.21.15.181:0/1995616401 >> v1:172.21.15.181:6789/0 conn(0x7fe98c15e6d0 legacy=0x7fe98c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe9926da700 1 -- 172.21.15.181:0/1995616401 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c06e0f0 con 0x7fe98c0cb0c0 2023-10-16T09:44:25.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe990ed7700 1 -- 172.21.15.181:0/1995616401 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345088413 0 0) 0x7fe98c06ed00 con 0x7fe98c15e6d0 2023-10-16T09:44:25.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe9926da700 1 --2- 172.21.15.181:0/1995616401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c0cb0c0 0x7fe98c15a800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe984009d90 tx=0x7fe98400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac4cbdd1138c7d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe990ed7700 1 -- 172.21.15.181:0/1995616401 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400e070 con 0x7fe98c0cb0c0 2023-10-16T09:44:25.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe990ed7700 1 -- 172.21.15.181:0/1995616401 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98400bb20 con 0x7fe98c0cb0c0 2023-10-16T09:44:25.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.067+0000 7fe990ed7700 1 -- 172.21.15.181:0/1995616401 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984012430 con 0x7fe98c0cb0c0 2023-10-16T09:44:25.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 -- 172.21.15.181:0/1995616401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c0cb0c0 msgr2=0x7fe98c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 --2- 172.21.15.181:0/1995616401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c0cb0c0 0x7fe98c15a800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe984009d90 tx=0x7fe98400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:25.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 -- 172.21.15.181:0/1995616401 shutdown_connections 2023-10-16T09:44:25.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 --2- 172.21.15.181:0/1995616401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c0cb0c0 0x7fe98c15a800 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 --2- 172.21.15.181:0/1995616401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15eab0 0x7fe98c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 -- 172.21.15.181:0/1995616401 >> 172.21.15.181:0/1995616401 conn(0x7fe98c0d04a0 msgr2=0x7fe98c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:25.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 -- 172.21.15.181:0/1995616401 shutdown_connections 2023-10-16T09:44:25.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.068+0000 7fe99413d700 1 -- 172.21.15.181:0/1995616401 wait complete. 2023-10-16T09:44:25.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.069+0000 7fe99413d700 1 Processor -- start 2023-10-16T09:44:25.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.069+0000 7fe99413d700 1 -- start start 2023-10-16T09:44:25.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe99413d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0cb0c0 0x7fe98c0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe99413d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15e6d0 0x7fe98c0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe99413d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c15eab0 0x7fe98c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe99413d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0b01d0 con 0x7fe98c15eab0 2023-10-16T09:44:25.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe99413d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0b29c0 con 0x7fe98c0cb0c0 2023-10-16T09:44:25.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe99413d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0b2e50 con 0x7fe98c15e6d0 2023-10-16T09:44:25.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c15eab0 0x7fe98c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c15eab0 0x7fe98c0b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36820/0 (socket says 172.21.15.181:36820) 2023-10-16T09:44:25.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 -- 172.21.15.181:0/2837293618 learned_addr learned my addr 172.21.15.181:0/2837293618 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:25.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9916d8700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15e6d0 0x7fe98c0b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe991ed9700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0cb0c0 0x7fe98c0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 -- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15e6d0 msgr2=0x7fe98c0b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15e6d0 0x7fe98c0b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 -- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0cb0c0 msgr2=0x7fe98c0c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0cb0c0 0x7fe98c0c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 -- 172.21.15.181:0/2837293618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984009a70 con 0x7fe98c15eab0 2023-10-16T09:44:25.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.070+0000 7fe9926da700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c15eab0 0x7fe98c0b1750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe98c071580 tx=0x7fe984004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.071+0000 7fe9827fc700 1 -- 172.21.15.181:0/2837293618 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984012430 con 0x7fe98c15eab0 2023-10-16T09:44:25.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.071+0000 7fe9827fc700 1 -- 172.21.15.181:0/2837293618 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe984020070 con 0x7fe98c15eab0 2023-10-16T09:44:25.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.071+0000 7fe9827fc700 1 -- 172.21.15.181:0/2837293618 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984004280 con 0x7fe98c15eab0 2023-10-16T09:44:25.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.071+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c0b1db0 con 0x7fe98c15eab0 2023-10-16T09:44:25.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.071+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c0b2370 con 0x7fe98c15eab0 2023-10-16T09:44:25.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.071+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c0b2b70 con 0x7fe98c15eab0 2023-10-16T09:44:25.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.072+0000 7fe9827fc700 1 -- 172.21.15.181:0/2837293618 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe9840043e0 con 0x7fe98c15eab0 2023-10-16T09:44:25.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.072+0000 7fe9827fc700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe97006af00 0x7fe97006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.072+0000 7fe9827fc700 1 -- 172.21.15.181:0/2837293618 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe984022070 con 0x7fe98c15eab0 2023-10-16T09:44:25.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.072+0000 7fe991ed9700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe97006af00 0x7fe97006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.073+0000 7fe991ed9700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe97006af00 0x7fe97006d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe97c007fd0 tx=0x7fe97c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.077+0000 7fe9827fc700 1 -- 172.21.15.181:0/2837293618 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe98c0b2b70 con 0x7fe98c15eab0 2023-10-16T09:44:25.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.269+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe98c0b2b30 con 0x7fe98c15eab0 2023-10-16T09:44:25.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.270+0000 7fe9827fc700 1 -- 172.21.15.181:0/2837293618 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fe9840562d0 con 0x7fe98c15eab0 2023-10-16T09:44:25.272 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:25.272 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:25.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe97006af00 msgr2=0x7fe97006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe97006af00 0x7fe97006d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe97c007fd0 tx=0x7fe97c009040 comp rx=0 tx=0).stop 2023-10-16T09:44:25.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c15eab0 msgr2=0x7fe98c0b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c15eab0 0x7fe98c0b1750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe98c071580 tx=0x7fe984004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 shutdown_connections 2023-10-16T09:44:25.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe97006af00 0x7fe97006d3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe98c15eab0 0x7fe98c0b1750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe98c0cb0c0 0x7fe98c0c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 --2- 172.21.15.181:0/2837293618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe98c15e6d0 0x7fe98c0b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 >> 172.21.15.181:0/2837293618 conn(0x7fe98c0d04a0 msgr2=0x7fe98c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:25.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 shutdown_connections 2023-10-16T09:44:25.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.272+0000 7fe99413d700 1 -- 172.21.15.181:0/2837293618 wait complete. 2023-10-16T09:44:25.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:25.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:25.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.408+0000 7f689909f700 1 Processor -- start 2023-10-16T09:44:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.408+0000 7f689909f700 1 -- start start 2023-10-16T09:44:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f689909f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 0x7f68940ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f689909f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68940cef00 con 0x7f689406afb0 2023-10-16T09:44:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f689909f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68940cec20 con 0x7f68940ce6a0 2023-10-16T09:44:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f689909f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6894071c80 con 0x7f689415da70 2023-10-16T09:44:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f689359e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f689406afb0 0x7f68941615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47906/0 (socket says 172.21.15.181:47906) 2023-10-16T09:44:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f689359e700 1 -- 172.21.15.181:0/2925349992 learned_addr learned my addr 172.21.15.181:0/2925349992 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f6892d9d700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 0x7f68940ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612060646 0 0) 0x7f68940cef00 con 0x7f689406afb0 2023-10-16T09:44:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c0035b0 con 0x7f689406afb0 2023-10-16T09:44:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957612836 0 0) 0x7f68940cec20 con 0x7f68940ce6a0 2023-10-16T09:44:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68940cef00 con 0x7f68940ce6a0 2023-10-16T09:44:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.409+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 715160553 0 0) 0x7f687c0035b0 con 0x7f689406afb0 2023-10-16T09:44:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 msgr2=0x7f68940ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 0x7f68940ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 >> v1:172.21.15.190:6789/0 conn(0x7f68940ce6a0 legacy=0x7f689406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894071f60 con 0x7f689406afb0 2023-10-16T09:44:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f688c0019f0 con 0x7f689406afb0 2023-10-16T09:44:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 >> v1:172.21.15.181:6789/0 conn(0x7f689406afb0 legacy=0x7f68941615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f687c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c004850 0x7f687c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 0x7f687c0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894071c80 con 0x7f687c003bc0 2023-10-16T09:44:25.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c00bbd0 con 0x7f687c004850 2023-10-16T09:44:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c00bd90 con 0x7f689415da70 2023-10-16T09:44:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6892d9d700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f687c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f689259c700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c004850 0x7f687c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f689359e700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 0x7f687c0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6892d9d700 1 -- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 msgr2=0x7f687c0052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6892d9d700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 0x7f687c0052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6892d9d700 1 -- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c004850 msgr2=0x7f687c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6892d9d700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c004850 0x7f687c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6892d9d700 1 -- 172.21.15.181:0/2925349992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0035b0 con 0x7f687c003bc0 2023-10-16T09:44:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f689359e700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415da70 0x7f687c0052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6892d9d700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f687c003fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6888009b60 tx=0x7f687c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbf0bb6303d6a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800e6c0 con 0x7f687c003bc0 2023-10-16T09:44:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688800e820 con 0x7f687c003bc0 2023-10-16T09:44:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.410+0000 7f6891d9b700 1 -- 172.21.15.181:0/2925349992 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800eb90 con 0x7f687c003bc0 2023-10-16T09:44:25.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.411+0000 7f689909f700 1 -- 172.21.15.181:0/2925349992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 msgr2=0x7f687c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.411+0000 7f689909f700 1 --2- 172.21.15.181:0/2925349992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f687c003fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6888009b60 tx=0x7f687c005880 comp rx=0 tx=0).stop 2023-10-16T09:44:25.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.411+0000 7f689359e700 1 -- 172.21.15.181:0/2925349992 reap_dead start 2023-10-16T09:44:25.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.411+0000 7f689909f700 1 -- 172.21.15.181:0/2925349992 shutdown_connections 2023-10-16T09:44:25.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.411+0000 7f689909f700 1 -- 172.21.15.181:0/2925349992 >> 172.21.15.181:0/2925349992 conn(0x7f68940cd010 msgr2=0x7f68940cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.411+0000 7f689909f700 1 -- 172.21.15.181:0/2925349992 shutdown_connections 2023-10-16T09:44:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.411+0000 7f689909f700 1 -- 172.21.15.181:0/2925349992 wait complete. 2023-10-16T09:44:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.412+0000 7f689909f700 1 Processor -- start 2023-10-16T09:44:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.412+0000 7f689909f700 1 -- start start 2023-10-16T09:44:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689909f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f689415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689909f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68940ca660 0x7f689415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689909f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415a900 0x7f689415fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689909f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689415fad0 con 0x7f687c003bc0 2023-10-16T09:44:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689909f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689415f230 con 0x7f68940ca660 2023-10-16T09:44:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689909f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689415f510 con 0x7f689415a900 2023-10-16T09:44:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f689415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f689415e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36836/0 (socket says 172.21.15.181:36836) 2023-10-16T09:44:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 -- 172.21.15.181:0/3231851027 learned_addr learned my addr 172.21.15.181:0/3231851027 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689259c700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68940ca660 0x7f689415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f689359e700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415a900 0x7f689415fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 -- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415a900 msgr2=0x7f689415fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415a900 0x7f689415fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 -- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68940ca660 msgr2=0x7f689415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68940ca660 0x7f689415a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.413+0000 7f6892d9d700 1 -- 172.21.15.181:0/3231851027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689415f7f0 con 0x7f687c003bc0 2023-10-16T09:44:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.414+0000 7f6892d9d700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f689415e9e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f687c00bba0 tx=0x7f68880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.414+0000 7f687b7fe700 1 -- 172.21.15.181:0/3231851027 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800e6c0 con 0x7f687c003bc0 2023-10-16T09:44:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.414+0000 7f687b7fe700 1 -- 172.21.15.181:0/3231851027 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68880038c0 con 0x7f687c003bc0 2023-10-16T09:44:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.414+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68940aca20 con 0x7f687c003bc0 2023-10-16T09:44:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.414+0000 7f687b7fe700 1 -- 172.21.15.181:0/3231851027 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888003c70 con 0x7f687c003bc0 2023-10-16T09:44:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.414+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68941601e0 con 0x7f687c003bc0 2023-10-16T09:44:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.415+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68940acbe0 con 0x7f687c003bc0 2023-10-16T09:44:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.415+0000 7f687b7fe700 1 -- 172.21.15.181:0/3231851027 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6888013470 con 0x7f687c003bc0 2023-10-16T09:44:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.416+0000 7f687b7fe700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f688006af00 0x7f688006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.416+0000 7f687b7fe700 1 -- 172.21.15.181:0/3231851027 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f688808aea0 con 0x7f687c003bc0 2023-10-16T09:44:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.416+0000 7f689259c700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f688006af00 0x7f688006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.416+0000 7f689259c700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f688006af00 0x7f688006d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f68940b6120 tx=0x7f6884009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.420+0000 7f687b7fe700 1 -- 172.21.15.181:0/3231851027 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6888056060 con 0x7f687c003bc0 2023-10-16T09:44:25.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.612+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68940acd90 con 0x7f687c003bc0 2023-10-16T09:44:25.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.612+0000 7f687b7fe700 1 -- 172.21.15.181:0/3231851027 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6888059680 con 0x7f687c003bc0 2023-10-16T09:44:25.615 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:25.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f688006af00 msgr2=0x7f688006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f688006af00 0x7f688006d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f68940b6120 tx=0x7f6884009040 comp rx=0 tx=0).stop 2023-10-16T09:44:25.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 msgr2=0x7f689415e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f689415e9e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f687c00bba0 tx=0x7f68880047c0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 shutdown_connections 2023-10-16T09:44:25.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f688006af00 0x7f688006d3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f689415a900 0x7f689415fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f687c003bc0 0x7f689415e9e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 --2- 172.21.15.181:0/3231851027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68940ca660 0x7f689415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 >> 172.21.15.181:0/3231851027 conn(0x7f68940cd010 msgr2=0x7f689415bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 shutdown_connections 2023-10-16T09:44:25.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.615+0000 7f689909f700 1 -- 172.21.15.181:0/3231851027 wait complete. 2023-10-16T09:44:25.629 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:25.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.750+0000 7f7e6ee9a700 1 Processor -- start 2023-10-16T09:44:25.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.750+0000 7f7e6ee9a700 1 -- start start 2023-10-16T09:44:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e6ee9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e6815da70 0x7f7e680ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e6ee9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 0x7f7e681615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e6ee9a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680cef00 con 0x7f7e680ce6a0 2023-10-16T09:44:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e6ee9a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e680cec20 con 0x7f7e6815da70 2023-10-16T09:44:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e6ee9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68071c80 con 0x7f7e6806afb0 2023-10-16T09:44:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e67fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e680ce6a0 0x7f7e6806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47924/0 (socket says 172.21.15.181:47924) 2023-10-16T09:44:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e67fff700 1 -- 172.21.15.181:0/1044282708 learned_addr learned my addr 172.21.15.181:0/1044282708 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e6d437700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 0x7f7e681615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.751+0000 7f7e6cc36700 1 --2- 172.21.15.181:0/1044282708 >> v2:172.21.15.190:3300/0 conn(0x7f7e6815da70 0x7f7e680ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283761198 0 0) 0x7f7e680cef00 con 0x7f7e680ce6a0 2023-10-16T09:44:25.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e580035b0 con 0x7f7e680ce6a0 2023-10-16T09:44:25.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3175396185 0 0) 0x7f7e580035b0 con 0x7f7e680ce6a0 2023-10-16T09:44:25.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 msgr2=0x7f7e681615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 0x7f7e681615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 >> v2:172.21.15.190:3300/0 conn(0x7f7e6815da70 msgr2=0x7f7e680ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 --2- 172.21.15.181:0/1044282708 >> v2:172.21.15.190:3300/0 conn(0x7f7e6815da70 0x7f7e680ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68071f60 con 0x7f7e680ce6a0 2023-10-16T09:44:25.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e6d437700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 0x7f7e681615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:25.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7e54001670 con 0x7f7e680ce6a0 2023-10-16T09:44:25.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 >> v1:172.21.15.181:6789/0 conn(0x7f7e680ce6a0 legacy=0x7f7e6806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 0x7f7e58003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58003f40 0x7f7e58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58004d90 0x7f7e58005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680cec20 con 0x7f7e58003f40 2023-10-16T09:44:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68071c80 con 0x7f7e58004d90 2023-10-16T09:44:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5800bac0 con 0x7f7e6806afb0 2023-10-16T09:44:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e67fff700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58003f40 0x7f7e58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e6cc36700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 0x7f7e58003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e6d437700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58004d90 0x7f7e58005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e67fff700 1 -- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 msgr2=0x7f7e58003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e67fff700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6806afb0 0x7f7e58003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e67fff700 1 -- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58004d90 msgr2=0x7f7e58005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e67fff700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58004d90 0x7f7e58005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e67fff700 1 -- 172.21.15.181:0/1044282708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60009a70 con 0x7f7e58003f40 2023-10-16T09:44:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.752+0000 7f7e67fff700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58003f40 0x7f7e58004850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400c320 tx=0x7f7e58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0419c1105d691e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400ce60 con 0x7f7e58003f40 2023-10-16T09:44:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e54005960 con 0x7f7e58003f40 2023-10-16T09:44:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e677fe700 1 -- 172.21.15.181:0/1044282708 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54005cf0 con 0x7f7e58003f40 2023-10-16T09:44:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1044282708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58003f40 msgr2=0x7f7e58004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e6ee9a700 1 --2- 172.21.15.181:0/1044282708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58003f40 0x7f7e58004850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400c320 tx=0x7f7e58005880 comp rx=0 tx=0).stop 2023-10-16T09:44:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e6d437700 1 -- 172.21.15.181:0/1044282708 reap_dead start 2023-10-16T09:44:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1044282708 shutdown_connections 2023-10-16T09:44:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1044282708 >> 172.21.15.181:0/1044282708 conn(0x7f7e680cd010 msgr2=0x7f7e680cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1044282708 shutdown_connections 2023-10-16T09:44:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.753+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1044282708 wait complete. 2023-10-16T09:44:25.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.754+0000 7f7e6ee9a700 1 Processor -- start 2023-10-16T09:44:25.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.754+0000 7f7e6ee9a700 1 -- start start 2023-10-16T09:44:25.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6ee9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58003f40 0x7f7e681f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6ee9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e680ad260 0x7f7e6815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6ee9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6815be40 0x7f7e680b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6ee9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6815c2d0 con 0x7f7e680ad260 2023-10-16T09:44:25.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6ee9a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681f7370 con 0x7f7e58003f40 2023-10-16T09:44:25.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6ee9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681f67d0 con 0x7f7e6815be40 2023-10-16T09:44:25.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e67fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e680ad260 0x7f7e6815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e67fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e680ad260 0x7f7e6815b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36858/0 (socket says 172.21.15.181:36858) 2023-10-16T09:44:25.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e67fff700 1 -- 172.21.15.181:0/1089654408 learned_addr learned my addr 172.21.15.181:0/1089654408 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:25.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6d437700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6815be40 0x7f7e680b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.755+0000 7f7e6cc36700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58003f40 0x7f7e681f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e67fff700 1 -- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6815be40 msgr2=0x7f7e680b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e67fff700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6815be40 0x7f7e680b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e67fff700 1 -- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58003f40 msgr2=0x7f7e681f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e67fff700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58003f40 0x7f7e681f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e67fff700 1 -- 172.21.15.181:0/1089654408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e681f6a40 con 0x7f7e680ad260 2023-10-16T09:44:25.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e67fff700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e680ad260 0x7f7e6815b900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e54000c00 tx=0x7f7e54001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e657fa700 1 -- 172.21.15.181:0/1089654408 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54000fe0 con 0x7f7e680ad260 2023-10-16T09:44:25.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e657fa700 1 -- 172.21.15.181:0/1089654408 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e54005e80 con 0x7f7e680ad260 2023-10-16T09:44:25.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e657fa700 1 -- 172.21.15.181:0/1089654408 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54004ca0 con 0x7f7e680ad260 2023-10-16T09:44:25.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e681f6d20 con 0x7f7e680ad260 2023-10-16T09:44:25.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.756+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e680b3a10 con 0x7f7e680ad260 2023-10-16T09:44:25.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.757+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6815c250 con 0x7f7e680ad260 2023-10-16T09:44:25.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.757+0000 7f7e657fa700 1 -- 172.21.15.181:0/1089654408 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e540070d0 con 0x7f7e680ad260 2023-10-16T09:44:25.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.758+0000 7f7e657fa700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e4c06af00 0x7f7e4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:25.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.758+0000 7f7e657fa700 1 -- 172.21.15.181:0/1089654408 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e5408a9c0 con 0x7f7e680ad260 2023-10-16T09:44:25.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.758+0000 7f7e6cc36700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e4c06af00 0x7f7e4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:25.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.758+0000 7f7e6cc36700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e4c06af00 0x7f7e4c06d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009c60 tx=0x7f7e5c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:25.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.762+0000 7f7e657fa700 1 -- 172.21.15.181:0/1089654408 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e54055bc0 con 0x7f7e680ad260 2023-10-16T09:44:25.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.961+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e6815d290 con 0x7f7e680ad260 2023-10-16T09:44:25.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.962+0000 7f7e657fa700 1 -- 172.21.15.181:0/1089654408 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e54007da0 con 0x7f7e680ad260 2023-10-16T09:44:25.964 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:25.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e4c06af00 msgr2=0x7f7e4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e4c06af00 0x7f7e4c06d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009c60 tx=0x7f7e5c009210 comp rx=0 tx=0).stop 2023-10-16T09:44:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e680ad260 msgr2=0x7f7e6815b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e680ad260 0x7f7e6815b900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e54000c00 tx=0x7f7e54001a00 comp rx=0 tx=0).stop 2023-10-16T09:44:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 shutdown_connections 2023-10-16T09:44:25.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e4c06af00 0x7f7e4c06d3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6815be40 0x7f7e680b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e58003f40 0x7f7e681f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 --2- 172.21.15.181:0/1089654408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e680ad260 0x7f7e6815b900 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:25.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 >> 172.21.15.181:0/1089654408 conn(0x7f7e680cd010 msgr2=0x7f7e6806f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:25.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 shutdown_connections 2023-10-16T09:44:25.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:25.964+0000 7f7e6ee9a700 1 -- 172.21.15.181:0/1089654408 wait complete. 2023-10-16T09:44:25.978 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.100+0000 7f77a4c97700 1 Processor -- start 2023-10-16T09:44:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.100+0000 7f77a4c97700 1 -- start start 2023-10-16T09:44:26.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f77a4c97700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77a015f150 0x7f77a00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f77a4c97700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a00b2ae0 con 0x7f77a00ce4f0 2023-10-16T09:44:26.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f77a4c97700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a00b2dc0 con 0x7f77a00cd170 2023-10-16T09:44:26.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f77a4c97700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a00ce9a0 con 0x7f77a015f150 2023-10-16T09:44:26.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f77a00ce4f0 0x7f77a00b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47944/0 (socket says 172.21.15.181:47944) 2023-10-16T09:44:26.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779dd9b700 1 -- 172.21.15.181:0/598088623 learned_addr learned my addr 172.21.15.181:0/598088623 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:26.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779e59c700 1 --2- 172.21.15.181:0/598088623 >> v2:172.21.15.190:3300/0 conn(0x7f77a015f150 0x7f77a00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810624492 0 0) 0x7f77a00b2ae0 con 0x7f77a00ce4f0 2023-10-16T09:44:26.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77880035b0 con 0x7f77a00ce4f0 2023-10-16T09:44:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707997001 0 0) 0x7f77a00b2dc0 con 0x7f77a00cd170 2023-10-16T09:44:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a00b2ae0 con 0x7f77a00cd170 2023-10-16T09:44:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779e59c700 1 -- 172.21.15.181:0/598088623 >> v1:172.21.15.190:6789/0 conn(0x7f77a00cd170 legacy=0x7f77a00b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779e59c700 1 -- 172.21.15.181:0/598088623 >> v1:172.21.15.181:6789/0 conn(0x7f77a00ce4f0 legacy=0x7f77a00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1746875040 0 0) 0x7f77880035b0 con 0x7f77a00ce4f0 2023-10-16T09:44:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.101+0000 7f779e59c700 1 -- 172.21.15.181:0/598088623 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a00cf9d0 con 0x7f77a015f150 2023-10-16T09:44:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f779e59c700 1 --2- 172.21.15.181:0/598088623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a015f150 0x7f77a00cdfb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7790009870 tx=0x7f7790009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e234f181fae70fbf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794001670 con 0x7f77a015f150 2023-10-16T09:44:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7790004030 con 0x7f77a015f150 2023-10-16T09:44:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f779d59a700 1 -- 172.21.15.181:0/598088623 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77900043c0 con 0x7f77a015f150 2023-10-16T09:44:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f77a4c97700 1 -- 172.21.15.181:0/598088623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a015f150 msgr2=0x7f77a00cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f77a4c97700 1 --2- 172.21.15.181:0/598088623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a015f150 0x7f77a00cdfb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7790009870 tx=0x7f7790009b80 comp rx=0 tx=0).stop 2023-10-16T09:44:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f77a4c97700 1 -- 172.21.15.181:0/598088623 shutdown_connections 2023-10-16T09:44:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f77a4c97700 1 --2- 172.21.15.181:0/598088623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a015f150 0x7f77a00cdfb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f77a4c97700 1 -- 172.21.15.181:0/598088623 >> 172.21.15.181:0/598088623 conn(0x7f77a009a930 msgr2=0x7f77a00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f77a4c97700 1 -- 172.21.15.181:0/598088623 shutdown_connections 2023-10-16T09:44:26.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.102+0000 7f77a4c97700 1 -- 172.21.15.181:0/598088623 wait complete. 2023-10-16T09:44:26.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.104+0000 7f77a4c97700 1 Processor -- start 2023-10-16T09:44:26.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.104+0000 7f77a4c97700 1 -- start start 2023-10-16T09:44:26.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f77a4c97700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a00cd170 0x7f77a01576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f77a4c97700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77a00ce4f0 0x7f77a015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f77a4c97700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a015f150 0x7f77a00c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f77a4c97700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a01f45e0 con 0x7f77a015f150 2023-10-16T09:44:26.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f77a4c97700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a0157bf0 con 0x7f77a00cd170 2023-10-16T09:44:26.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f77a4c97700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a00cac80 con 0x7f77a00ce4f0 2023-10-16T09:44:26.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a015f150 0x7f77a00c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a015f150 0x7f77a00c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36874/0 (socket says 172.21.15.181:36874) 2023-10-16T09:44:26.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 -- 172.21.15.181:0/369126925 learned_addr learned my addr 172.21.15.181:0/369126925 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779e59c700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a00cd170 0x7f77a01576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779dd9b700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77a00ce4f0 0x7f77a015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 -- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77a00ce4f0 msgr2=0x7f77a015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77a00ce4f0 0x7f77a015c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 -- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a00cd170 msgr2=0x7f77a01576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a00cd170 0x7f77a01576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779ed9d700 1 -- 172.21.15.181:0/369126925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77900094a0 con 0x7f77a015f150 2023-10-16T09:44:26.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.105+0000 7f779e59c700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a00cd170 0x7f77a01576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:26.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.106+0000 7f779ed9d700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a015f150 0x7f77a00c8ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f779800cbc0 tx=0x7f779800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.106+0000 7f778effd700 1 -- 172.21.15.181:0/369126925 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798003c00 con 0x7f77a015f150 2023-10-16T09:44:26.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.106+0000 7f778effd700 1 -- 172.21.15.181:0/369126925 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798003d60 con 0x7f77a015f150 2023-10-16T09:44:26.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.106+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a00caf60 con 0x7f77a015f150 2023-10-16T09:44:26.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.106+0000 7f778effd700 1 -- 172.21.15.181:0/369126925 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800abb0 con 0x7f77a015f150 2023-10-16T09:44:26.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.106+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a00cb520 con 0x7f77a015f150 2023-10-16T09:44:26.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.107+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a01f4750 con 0x7f77a015f150 2023-10-16T09:44:26.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.107+0000 7f778effd700 1 -- 172.21.15.181:0/369126925 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f779800ad10 con 0x7f77a015f150 2023-10-16T09:44:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.107+0000 7f778effd700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f778006af00 0x7f778006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.107+0000 7f778effd700 1 -- 172.21.15.181:0/369126925 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7798089700 con 0x7f77a015f150 2023-10-16T09:44:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.108+0000 7f779e59c700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f778006af00 0x7f778006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.108+0000 7f779e59c700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f778006af00 0x7f778006d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7790009790 tx=0x7f779001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.112+0000 7f778effd700 1 -- 172.21.15.181:0/369126925 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f77980547b0 con 0x7f77a015f150 2023-10-16T09:44:26.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.304+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77a0067d20 con 0x7f77a015f150 2023-10-16T09:44:26.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.304+0000 7f778effd700 1 -- 172.21.15.181:0/369126925 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7798057dd0 con 0x7f77a015f150 2023-10-16T09:44:26.307 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:26.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f778006af00 msgr2=0x7f778006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f778006af00 0x7f778006d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7790009790 tx=0x7f779001c040 comp rx=0 tx=0).stop 2023-10-16T09:44:26.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a015f150 msgr2=0x7f77a00c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a015f150 0x7f77a00c8ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f779800cbc0 tx=0x7f779800cf80 comp rx=0 tx=0).stop 2023-10-16T09:44:26.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 shutdown_connections 2023-10-16T09:44:26.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f778006af00 0x7f778006d3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a015f150 0x7f77a00c8ad0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77a00cd170 0x7f77a01576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 --2- 172.21.15.181:0/369126925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77a00ce4f0 0x7f77a015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 >> 172.21.15.181:0/369126925 conn(0x7f77a009a930 msgr2=0x7f77a01584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:26.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 shutdown_connections 2023-10-16T09:44:26.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.306+0000 7f77a4c97700 1 -- 172.21.15.181:0/369126925 wait complete. 2023-10-16T09:44:26.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:26.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.438+0000 7fae26e30700 1 Processor -- start 2023-10-16T09:44:26.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.438+0000 7fae26e30700 1 -- start start 2023-10-16T09:44:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.439+0000 7fae26e30700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fae200cdb80 0x7fae200af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.439+0000 7fae26e30700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2006f870 con 0x7fae200af450 2023-10-16T09:44:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.439+0000 7fae26e30700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae200d1160 con 0x7fae2015dfd0 2023-10-16T09:44:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.439+0000 7fae26e30700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae200cc8e0 con 0x7fae200cdb80 2023-10-16T09:44:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.439+0000 7fae1ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fae200af450 0x7fae200cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47958/0 (socket says 172.21.15.181:47958) 2023-10-16T09:44:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.439+0000 7fae1ffff700 1 -- 172.21.15.181:0/1102901455 learned_addr learned my addr 172.21.15.181:0/1102901455 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 --2- 172.21.15.181:0/1102901455 >> v2:172.21.15.190:3300/0 conn(0x7fae200cdb80 0x7fae200af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280575338 0 0) 0x7fae2006f870 con 0x7fae200af450 2023-10-16T09:44:26.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae100035b0 con 0x7fae200af450 2023-10-16T09:44:26.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172603214 0 0) 0x7fae100035b0 con 0x7fae200af450 2023-10-16T09:44:26.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 >> v2:172.21.15.190:3300/0 conn(0x7fae200cdb80 msgr2=0x7fae200af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:26.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 --2- 172.21.15.181:0/1102901455 >> v2:172.21.15.190:3300/0 conn(0x7fae200cdb80 0x7fae200af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 >> v1:172.21.15.190:6789/0 conn(0x7fae2015dfd0 legacy=0x7fae2006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200ccbc0 con 0x7fae200af450 2023-10-16T09:44:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fae0c001670 con 0x7fae200af450 2023-10-16T09:44:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 >> v1:172.21.15.181:6789/0 conn(0x7fae200af450 legacy=0x7fae200cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae10003a00 0x7fae10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae10004850 0x7fae10004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae1000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200cc8e0 con 0x7fae10005210 2023-10-16T09:44:26.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1000c1b0 con 0x7fae10004850 2023-10-16T09:44:26.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1000c370 con 0x7fae10003a00 2023-10-16T09:44:26.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae1000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae1ffff700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae10004850 0x7fae10004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae24bcc700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae10003a00 0x7fae10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 -- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae10003a00 msgr2=0x7fae10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:26.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae10003a00 0x7fae10003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 -- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae10004850 msgr2=0x7fae10004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae10004850 0x7fae10004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 -- 172.21.15.181:0/1102901455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100035b0 con 0x7fae10005210 2023-10-16T09:44:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae24bcc700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae10003a00 0x7fae10003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 -- 172.21.15.181:0/1102901455 reap_dead start 2023-10-16T09:44:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.440+0000 7fae253cd700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae1000ba10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fae18009f90 tx=0x7fae10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939b9064c5ddcf26 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18003a20 con 0x7fae10005210 2023-10-16T09:44:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1800f040 con 0x7fae10005210 2023-10-16T09:44:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae1f7fe700 1 -- 172.21.15.181:0/1102901455 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18003de0 con 0x7fae10005210 2023-10-16T09:44:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae26e30700 1 -- 172.21.15.181:0/1102901455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 msgr2=0x7fae1000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae26e30700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae1000ba10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fae18009f90 tx=0x7fae10005880 comp rx=0 tx=0).stop 2023-10-16T09:44:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae26e30700 1 -- 172.21.15.181:0/1102901455 shutdown_connections 2023-10-16T09:44:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae26e30700 1 --2- 172.21.15.181:0/1102901455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae1000ba10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae26e30700 1 -- 172.21.15.181:0/1102901455 >> 172.21.15.181:0/1102901455 conn(0x7fae200d04a0 msgr2=0x7fae200d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae26e30700 1 -- 172.21.15.181:0/1102901455 shutdown_connections 2023-10-16T09:44:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.441+0000 7fae26e30700 1 -- 172.21.15.181:0/1102901455 wait complete. 2023-10-16T09:44:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.442+0000 7fae26e30700 1 Processor -- start 2023-10-16T09:44:26.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.442+0000 7fae26e30700 1 -- start start 2023-10-16T09:44:26.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae26e30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae200c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae26e30700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae2014b680 0x7fae2014bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae26e30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae200c60f0 0x7fae200c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae26e30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200b4390 con 0x7fae10005210 2023-10-16T09:44:26.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae26e30700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2014c040 con 0x7fae2014b680 2023-10-16T09:44:26.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae26e30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae200c6e40 con 0x7fae200c60f0 2023-10-16T09:44:26.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae24bcc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae200c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae24bcc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae200c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36890/0 (socket says 172.21.15.181:36890) 2023-10-16T09:44:26.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae24bcc700 1 -- 172.21.15.181:0/927233522 learned_addr learned my addr 172.21.15.181:0/927233522 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:26.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae253cd700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae200c60f0 0x7fae200c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.443+0000 7fae1ffff700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae2014b680 0x7fae2014bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae24bcc700 1 -- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae200c60f0 msgr2=0x7fae200c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae24bcc700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae200c60f0 0x7fae200c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae24bcc700 1 -- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae2014b680 msgr2=0x7fae2014bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae24bcc700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae2014b680 0x7fae2014bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae24bcc700 1 -- 172.21.15.181:0/927233522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200b39f0 con 0x7fae10005210 2023-10-16T09:44:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae24bcc700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae200c4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae14000c00 tx=0x7fae1400e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae1d7fa700 1 -- 172.21.15.181:0/927233522 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400ee40 con 0x7fae10005210 2023-10-16T09:44:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae1d7fa700 1 -- 172.21.15.181:0/927233522 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae140052e0 con 0x7fae10005210 2023-10-16T09:44:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae200b3cd0 con 0x7fae10005210 2023-10-16T09:44:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae1d7fa700 1 -- 172.21.15.181:0/927233522 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140036d0 con 0x7fae10005210 2023-10-16T09:44:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.444+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae200c6ae0 con 0x7fae10005210 2023-10-16T09:44:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.445+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae20067d20 con 0x7fae10005210 2023-10-16T09:44:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.445+0000 7fae1d7fa700 1 -- 172.21.15.181:0/927233522 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fae14003830 con 0x7fae10005210 2023-10-16T09:44:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.446+0000 7fae1d7fa700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fae0406af00 0x7fae0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.446+0000 7fae1d7fa700 1 -- 172.21.15.181:0/927233522 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae14088a30 con 0x7fae10005210 2023-10-16T09:44:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.446+0000 7fae1ffff700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fae0406af00 0x7fae0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.446+0000 7fae1ffff700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fae0406af00 0x7fae0406d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fae200ca670 tx=0x7fae0c000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.450+0000 7fae1d7fa700 1 -- 172.21.15.181:0/927233522 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fae14053c20 con 0x7fae10005210 2023-10-16T09:44:26.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.646+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae20067a70 con 0x7fae10005210 2023-10-16T09:44:26.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.647+0000 7fae1d7fa700 1 -- 172.21.15.181:0/927233522 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fae14019020 con 0x7fae10005210 2023-10-16T09:44:26.650 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:26.650 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:26.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.649+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fae0406af00 msgr2=0x7fae0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.649+0000 7fae26e30700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fae0406af00 0x7fae0406d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fae200ca670 tx=0x7fae0c000ed0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.649+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 msgr2=0x7fae200c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.649+0000 7fae26e30700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae200c4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae14000c00 tx=0x7fae1400e7c0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 shutdown_connections 2023-10-16T09:44:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fae0406af00 0x7fae0406d3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae200c60f0 0x7fae200c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae10005210 0x7fae200c4240 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 --2- 172.21.15.181:0/927233522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae2014b680 0x7fae2014bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 >> 172.21.15.181:0/927233522 conn(0x7fae200d04a0 msgr2=0x7fae200cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:26.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 shutdown_connections 2023-10-16T09:44:26.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.650+0000 7fae26e30700 1 -- 172.21.15.181:0/927233522 wait complete. 2023-10-16T09:44:26.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:26.665 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:26.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.786+0000 7f73251ef700 1 Processor -- start 2023-10-16T09:44:26.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.786+0000 7f73251ef700 1 -- start start 2023-10-16T09:44:26.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f73251ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f732015da70 0x7f73200ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f73251ef700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200ce6a0 0x7f732006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f73251ef700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73200cef00 con 0x7f7320072760 2023-10-16T09:44:26.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f73251ef700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200cec20 con 0x7f732015da70 2023-10-16T09:44:26.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f73251ef700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7320071c80 con 0x7f73200ce6a0 2023-10-16T09:44:26.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f731ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f732015da70 0x7f73200ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f731f59e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7320072760 0x7f73201615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47978/0 (socket says 172.21.15.181:47978) 2023-10-16T09:44:26.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f731ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f732015da70 0x7f73200ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36898/0 (socket says 172.21.15.181:36898) 2023-10-16T09:44:26.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f731f59e700 1 -- 172.21.15.181:0/2651108372 learned_addr learned my addr 172.21.15.181:0/2651108372 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:26.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.787+0000 7f731e59c700 1 --2- 172.21.15.181:0/2651108372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200ce6a0 0x7f732006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731ed9d700 1 -- 172.21.15.181:0/2651108372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200ce6a0 msgr2=0x7f732006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:26.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731ed9d700 1 --2- 172.21.15.181:0/2651108372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200ce6a0 0x7f732006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731ed9d700 1 -- 172.21.15.181:0/2651108372 >> v1:172.21.15.181:6789/0 conn(0x7f7320072760 legacy=0x7f73201615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731ed9d700 1 -- 172.21.15.181:0/2651108372 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320071f60 con 0x7f732015da70 2023-10-16T09:44:26.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731dd9b700 1 -- 172.21.15.181:0/2651108372 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374310192 0 0) 0x7f73200cef00 con 0x7f7320072760 2023-10-16T09:44:26.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731e59c700 1 --2- 172.21.15.181:0/2651108372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200ce6a0 0x7f732006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:26.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731ed9d700 1 --2- 172.21.15.181:0/2651108372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200ce160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7314009840 tx=0x7f7314009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507506aca43f8c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731dd9b700 1 -- 172.21.15.181:0/2651108372 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314011830 con 0x7f732015da70 2023-10-16T09:44:26.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731dd9b700 1 -- 172.21.15.181:0/2651108372 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314011990 con 0x7f732015da70 2023-10-16T09:44:26.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f731dd9b700 1 -- 172.21.15.181:0/2651108372 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314011d20 con 0x7f732015da70 2023-10-16T09:44:26.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 -- 172.21.15.181:0/2651108372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 msgr2=0x7f73200ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 --2- 172.21.15.181:0/2651108372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200ce160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7314009840 tx=0x7f7314009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:26.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 -- 172.21.15.181:0/2651108372 shutdown_connections 2023-10-16T09:44:26.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 --2- 172.21.15.181:0/2651108372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200ce160 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 --2- 172.21.15.181:0/2651108372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200ce6a0 0x7f732006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 -- 172.21.15.181:0/2651108372 >> 172.21.15.181:0/2651108372 conn(0x7f73200cd010 msgr2=0x7f73200cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:26.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 -- 172.21.15.181:0/2651108372 shutdown_connections 2023-10-16T09:44:26.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.788+0000 7f73251ef700 1 -- 172.21.15.181:0/2651108372 wait complete. 2023-10-16T09:44:26.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.790+0000 7f73251ef700 1 Processor -- start 2023-10-16T09:44:26.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.790+0000 7f73251ef700 1 -- start start 2023-10-16T09:44:26.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73251ef700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7320072760 0x7f73201f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73251ef700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200ce6a0 0x7f73200aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73251ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73251ef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b41a0 con 0x7f732015da70 2023-10-16T09:44:26.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73251ef700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b3520 con 0x7f73200ce6a0 2023-10-16T09:44:26.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73251ef700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b3800 con 0x7f7320072760 2023-10-16T09:44:26.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36908/0 (socket says 172.21.15.181:36908) 2023-10-16T09:44:26.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 -- 172.21.15.181:0/3653605800 learned_addr learned my addr 172.21.15.181:0/3653605800 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:26.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731ed9d700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7320072760 0x7f73201f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731e59c700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200ce6a0 0x7f73200aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 -- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7320072760 msgr2=0x7f73201f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7320072760 0x7f73201f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 -- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200ce6a0 msgr2=0x7f73200aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200ce6a0 0x7f73200aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 -- 172.21.15.181:0/3653605800 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140094a0 con 0x7f732015da70 2023-10-16T09:44:26.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f731f59e700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200c7660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f731800cba0 tx=0x7f731800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73077fe700 1 -- 172.21.15.181:0/3653605800 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731800aaa0 con 0x7f732015da70 2023-10-16T09:44:26.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73077fe700 1 -- 172.21.15.181:0/3653605800 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731800ac20 con 0x7f732015da70 2023-10-16T09:44:26.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73200b3ae0 con 0x7f732015da70 2023-10-16T09:44:26.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.791+0000 7f73077fe700 1 -- 172.21.15.181:0/3653605800 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180070d0 con 0x7f732015da70 2023-10-16T09:44:26.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.792+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73200b31b0 con 0x7f732015da70 2023-10-16T09:44:26.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.792+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73200b42e0 con 0x7f732015da70 2023-10-16T09:44:26.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.793+0000 7f73077fe700 1 -- 172.21.15.181:0/3653605800 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7318007230 con 0x7f732015da70 2023-10-16T09:44:26.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.793+0000 7f73077fe700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f730c06af00 0x7f730c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:26.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.793+0000 7f73077fe700 1 -- 172.21.15.181:0/3653605800 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7318089240 con 0x7f732015da70 2023-10-16T09:44:26.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.793+0000 7f731ed9d700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f730c06af00 0x7f730c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:26.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.794+0000 7f731ed9d700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f730c06af00 0x7f730c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73140090b0 tx=0x7f731401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:26.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.798+0000 7f73077fe700 1 -- 172.21.15.181:0/3653605800 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f73200b42e0 con 0x7f732015da70 2023-10-16T09:44:26.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.992+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73200b42e0 con 0x7f732015da70 2023-10-16T09:44:26.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.993+0000 7f73077fe700 1 -- 172.21.15.181:0/3653605800 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73200b42e0 con 0x7f732015da70 2023-10-16T09:44:26.995 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.994+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f730c06af00 msgr2=0x7f730c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.994+0000 7f73251ef700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f730c06af00 0x7f730c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73140090b0 tx=0x7f731401c040 comp rx=0 tx=0).stop 2023-10-16T09:44:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 msgr2=0x7f73200c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200c7660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f731800cba0 tx=0x7f731800ceb0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 shutdown_connections 2023-10-16T09:44:26.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f730c06af00 0x7f730c06d3c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7320072760 0x7f73201f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f732015da70 0x7f73200c7660 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 --2- 172.21.15.181:0/3653605800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200ce6a0 0x7f73200aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:26.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 >> 172.21.15.181:0/3653605800 conn(0x7f73200cd010 msgr2=0x7f73200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:26.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 shutdown_connections 2023-10-16T09:44:26.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:26.995+0000 7f73251ef700 1 -- 172.21.15.181:0/3653605800 wait complete. 2023-10-16T09:44:27.009 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.131+0000 7f34b2600700 1 Processor -- start 2023-10-16T09:44:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.131+0000 7f34b2600700 1 -- start start 2023-10-16T09:44:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.131+0000 7f34b2600700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34ac0ce6c0 0x7f34ac0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.131+0000 7f34b2600700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac0cc6a0 0x7f34ac0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.131+0000 7f34b2600700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac0b1f70 con 0x7f34ac15f150 2023-10-16T09:44:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.131+0000 7f34b2600700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac0b2ae0 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.131+0000 7f34b2600700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0b2dc0 con 0x7f34ac0cc6a0 2023-10-16T09:44:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34abfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f34ac15f150 0x7f34ac0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:47990/0 (socket says 172.21.15.181:47990) 2023-10-16T09:44:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34abfff700 1 -- 172.21.15.181:0/2391930297 learned_addr learned my addr 172.21.15.181:0/2391930297 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34b0b9d700 1 --2- 172.21.15.181:0/2391930297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac0cc6a0 0x7f34ac0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34ab7fe700 1 --2- 172.21.15.181:0/2391930297 >> v2:172.21.15.190:3300/0 conn(0x7f34ac0ce6c0 0x7f34ac0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34aaffd700 1 -- 172.21.15.181:0/2391930297 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151700681 0 0) 0x7f34ac0b1f70 con 0x7f34ac15f150 2023-10-16T09:44:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34aaffd700 1 -- 172.21.15.181:0/2391930297 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34940035b0 con 0x7f34ac15f150 2023-10-16T09:44:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34aaffd700 1 -- 172.21.15.181:0/2391930297 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2035384135 0 0) 0x7f34940035b0 con 0x7f34ac15f150 2023-10-16T09:44:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34b0b9d700 1 -- 172.21.15.181:0/2391930297 >> v2:172.21.15.190:3300/0 conn(0x7f34ac0ce6c0 msgr2=0x7f34ac0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34b0b9d700 1 --2- 172.21.15.181:0/2391930297 >> v2:172.21.15.190:3300/0 conn(0x7f34ac0ce6c0 0x7f34ac0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34b0b9d700 1 -- 172.21.15.181:0/2391930297 >> v1:172.21.15.181:6789/0 conn(0x7f34ac15f150 legacy=0x7f34ac0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34b0b9d700 1 -- 172.21.15.181:0/2391930297 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac0cf9d0 con 0x7f34ac0cc6a0 2023-10-16T09:44:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34ab7fe700 1 --2- 172.21.15.181:0/2391930297 >> v2:172.21.15.190:3300/0 conn(0x7f34ac0ce6c0 0x7f34ac0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.132+0000 7f34b0b9d700 1 --2- 172.21.15.181:0/2391930297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac0cc6a0 0x7f34ac0ccb10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009d90 tx=0x7f34a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8463b2eb499f964 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34aaffd700 1 -- 172.21.15.181:0/2391930297 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400e070 con 0x7f34ac0cc6a0 2023-10-16T09:44:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34aaffd700 1 -- 172.21.15.181:0/2391930297 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a4003a20 con 0x7f34ac0cc6a0 2023-10-16T09:44:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34aaffd700 1 -- 172.21.15.181:0/2391930297 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c001670 con 0x7f34ac0cc6a0 2023-10-16T09:44:27.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 -- 172.21.15.181:0/2391930297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac0cc6a0 msgr2=0x7f34ac0ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 --2- 172.21.15.181:0/2391930297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac0cc6a0 0x7f34ac0ccb10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009d90 tx=0x7f34a400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:27.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 -- 172.21.15.181:0/2391930297 shutdown_connections 2023-10-16T09:44:27.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 --2- 172.21.15.181:0/2391930297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac0cc6a0 0x7f34ac0ccb10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 --2- 172.21.15.181:0/2391930297 >> v2:172.21.15.190:3300/0 conn(0x7f34ac0ce6c0 0x7f34ac0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 -- 172.21.15.181:0/2391930297 >> 172.21.15.181:0/2391930297 conn(0x7f34ac09a930 msgr2=0x7f34ac0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 -- 172.21.15.181:0/2391930297 shutdown_connections 2023-10-16T09:44:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.133+0000 7f34b2600700 1 -- 172.21.15.181:0/2391930297 wait complete. 2023-10-16T09:44:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.134+0000 7f34b2600700 1 Processor -- start 2023-10-16T09:44:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.135+0000 7f34b2600700 1 -- start start 2023-10-16T09:44:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.135+0000 7f34b2600700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34ac0cc6a0 0x7f34ac0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.135+0000 7f34b2600700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34ac0ce6c0 0x7f34ac06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.135+0000 7f34b2600700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac15f150 0x7f34ac1f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.135+0000 7f34b2600700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0c5e50 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.135+0000 7f34b2600700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac06cde0 con 0x7f34ac0cc6a0 2023-10-16T09:44:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.135+0000 7f34b2600700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac06d9f0 con 0x7f34ac15f150 2023-10-16T09:44:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34ac0ce6c0 0x7f34ac06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34ac0ce6c0 0x7f34ac06c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36920/0 (socket says 172.21.15.181:36920) 2023-10-16T09:44:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 -- 172.21.15.181:0/331296618 learned_addr learned my addr 172.21.15.181:0/331296618 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34b0b9d700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac15f150 0x7f34ac1f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34abfff700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34ac0cc6a0 0x7f34ac0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 -- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac15f150 msgr2=0x7f34ac1f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac15f150 0x7f34ac1f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 -- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34ac0cc6a0 msgr2=0x7f34ac0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34ac0cc6a0 0x7f34ac0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 -- 172.21.15.181:0/331296618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00094a0 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34ab7fe700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34ac0ce6c0 0x7f34ac06c8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ec40 tx=0x7f34a000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34a8ff9700 1 -- 172.21.15.181:0/331296618 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a000c850 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34a8ff9700 1 -- 172.21.15.181:0/331296618 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a000c9b0 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a4009a70 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34a8ff9700 1 -- 172.21.15.181:0/331296618 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a000cd20 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.136+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac0c54b0 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.137+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac0c5f90 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.138+0000 7f34a8ff9700 1 -- 172.21.15.181:0/331296618 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f34a0013470 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.138+0000 7f34a8ff9700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f349806af00 0x7f349806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.138+0000 7f34a8ff9700 1 -- 172.21.15.181:0/331296618 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34a0089650 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.138+0000 7f34abfff700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f349806af00 0x7f349806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.138+0000 7f34abfff700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f349806af00 0x7f349806d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f349c000f60 tx=0x7f349c001e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.142+0000 7f34a8ff9700 1 -- 172.21.15.181:0/331296618 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f34ac0c5f90 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.334+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34ac0c5f90 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.334+0000 7f34a8ff9700 1 -- 172.21.15.181:0/331296618 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34ac0c5f90 con 0x7f34ac0ce6c0 2023-10-16T09:44:27.336 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:27.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f349806af00 msgr2=0x7f349806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f349806af00 0x7f349806d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f349c000f60 tx=0x7f349c001e00 comp rx=0 tx=0).stop 2023-10-16T09:44:27.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34ac0ce6c0 msgr2=0x7f34ac06c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34ac0ce6c0 0x7f34ac06c8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ec40 tx=0x7f34a000ef50 comp rx=0 tx=0).stop 2023-10-16T09:44:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 shutdown_connections 2023-10-16T09:44:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f349806af00 0x7f349806d3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34ac15f150 0x7f34ac1f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34ac0cc6a0 0x7f34ac0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 --2- 172.21.15.181:0/331296618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34ac0ce6c0 0x7f34ac06c8a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 >> 172.21.15.181:0/331296618 conn(0x7f34ac09a930 msgr2=0x7f34ac15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 shutdown_connections 2023-10-16T09:44:27.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.336+0000 7f34b2600700 1 -- 172.21.15.181:0/331296618 wait complete. 2023-10-16T09:44:27.350 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.473+0000 7f42553d5700 1 Processor -- start 2023-10-16T09:44:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f42553d5700 1 -- start start 2023-10-16T09:44:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f42553d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f425015f150 0x7f42500cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f42553d5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500b2ae0 con 0x7f42500cd170 2023-10-16T09:44:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f42553d5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500b2dc0 con 0x7f42500ce4f0 2023-10-16T09:44:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f42553d5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42500ce9a0 con 0x7f425015f150 2023-10-16T09:44:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f424effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f425015f150 0x7f42500cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f424f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f42500cd170 0x7f42500b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48006/0 (socket says 172.21.15.181:48006) 2023-10-16T09:44:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.474+0000 7f424f7fe700 1 -- 172.21.15.181:0/12233195 learned_addr learned my addr 172.21.15.181:0/12233195 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424effd700 1 -- 172.21.15.181:0/12233195 >> v1:172.21.15.190:6789/0 conn(0x7f42500ce4f0 legacy=0x7f42500b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424effd700 1 -- 172.21.15.181:0/12233195 >> v1:172.21.15.181:6789/0 conn(0x7f42500cd170 legacy=0x7f42500b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424effd700 1 -- 172.21.15.181:0/12233195 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500cf9d0 con 0x7f425015f150 2023-10-16T09:44:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424dffb700 1 -- 172.21.15.181:0/12233195 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016373097 0 0) 0x7f42500b2ae0 con 0x7f42500cd170 2023-10-16T09:44:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424effd700 1 --2- 172.21.15.181:0/12233195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425015f150 0x7f42500cdfb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4244009870 tx=0x7f4244009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add4b64598c50016 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424dffb700 1 -- 172.21.15.181:0/12233195 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244004230 con 0x7f425015f150 2023-10-16T09:44:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424dffb700 1 -- 172.21.15.181:0/12233195 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4244004390 con 0x7f425015f150 2023-10-16T09:44:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.475+0000 7f424dffb700 1 -- 172.21.15.181:0/12233195 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440036a0 con 0x7f425015f150 2023-10-16T09:44:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.476+0000 7f42553d5700 1 -- 172.21.15.181:0/12233195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425015f150 msgr2=0x7f42500cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.476+0000 7f42553d5700 1 --2- 172.21.15.181:0/12233195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425015f150 0x7f42500cdfb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4244009870 tx=0x7f4244009b80 comp rx=0 tx=0).stop 2023-10-16T09:44:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.476+0000 7f42553d5700 1 -- 172.21.15.181:0/12233195 shutdown_connections 2023-10-16T09:44:27.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.476+0000 7f42553d5700 1 --2- 172.21.15.181:0/12233195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425015f150 0x7f42500cdfb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.476+0000 7f42553d5700 1 -- 172.21.15.181:0/12233195 >> 172.21.15.181:0/12233195 conn(0x7f425009a930 msgr2=0x7f42500d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:27.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.476+0000 7f42553d5700 1 -- 172.21.15.181:0/12233195 shutdown_connections 2023-10-16T09:44:27.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.476+0000 7f42553d5700 1 -- 172.21.15.181:0/12233195 wait complete. 2023-10-16T09:44:27.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.477+0000 7f42553d5700 1 Processor -- start 2023-10-16T09:44:27.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.477+0000 7f42553d5700 1 -- start start 2023-10-16T09:44:27.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f42553d5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42500cd170 0x7f425006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f42553d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42500ce4f0 0x7f42501e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f42553d5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f425015f150 0x7f4250152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f42553d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425006c4d0 con 0x7f42500ce4f0 2023-10-16T09:44:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f42553d5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425006c7b0 con 0x7f425015f150 2023-10-16T09:44:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f42553d5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425006ca90 con 0x7f42500cd170 2023-10-16T09:44:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f424e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42500ce4f0 0x7f42501e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f424e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42500ce4f0 0x7f42501e4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36932/0 (socket says 172.21.15.181:36932) 2023-10-16T09:44:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f424e7fc700 1 -- 172.21.15.181:0/3159856967 learned_addr learned my addr 172.21.15.181:0/3159856967 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f424effd700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42500cd170 0x7f425006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.478+0000 7f424f7fe700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f425015f150 0x7f4250152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f424e7fc700 1 -- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42500cd170 msgr2=0x7f425006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f424e7fc700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42500cd170 0x7f425006d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f424e7fc700 1 -- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f425015f150 msgr2=0x7f4250152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f424e7fc700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f425015f150 0x7f4250152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f424e7fc700 1 -- 172.21.15.181:0/3159856967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440094a0 con 0x7f42500ce4f0 2023-10-16T09:44:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f424e7fc700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42500ce4f0 0x7f42501e4ce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f424000c4d0 tx=0x7f424000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f42377fe700 1 -- 172.21.15.181:0/3159856967 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011790 con 0x7f42500ce4f0 2023-10-16T09:44:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f42377fe700 1 -- 172.21.15.181:0/3159856967 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42400118f0 con 0x7f42500ce4f0 2023-10-16T09:44:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425006cd70 con 0x7f42500ce4f0 2023-10-16T09:44:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f42377fe700 1 -- 172.21.15.181:0/3159856967 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011ca0 con 0x7f42500ce4f0 2023-10-16T09:44:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.479+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425006bcc0 con 0x7f42500ce4f0 2023-10-16T09:44:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.480+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425006d290 con 0x7f42500ce4f0 2023-10-16T09:44:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.480+0000 7f42377fe700 1 -- 172.21.15.181:0/3159856967 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f424001c460 con 0x7f42500ce4f0 2023-10-16T09:44:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.481+0000 7f42377fe700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f423c06af00 0x7f423c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.481+0000 7f424effd700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f423c06af00 0x7f423c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.481+0000 7f42377fe700 1 -- 172.21.15.181:0/3159856967 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4240089300 con 0x7f42500ce4f0 2023-10-16T09:44:27.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.481+0000 7f424effd700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f423c06af00 0x7f423c06d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4244009840 tx=0x7f424401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.485+0000 7f42377fe700 1 -- 172.21.15.181:0/3159856967 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f42400543b0 con 0x7f42500ce4f0 2023-10-16T09:44:27.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.676+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42500cfc80 con 0x7f42500ce4f0 2023-10-16T09:44:27.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.676+0000 7f42377fe700 1 -- 172.21.15.181:0/3159856967 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42400579d0 con 0x7f42500ce4f0 2023-10-16T09:44:27.679 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:27.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f423c06af00 msgr2=0x7f423c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f423c06af00 0x7f423c06d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4244009840 tx=0x7f424401c040 comp rx=0 tx=0).stop 2023-10-16T09:44:27.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42500ce4f0 msgr2=0x7f42501e4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42500ce4f0 0x7f42501e4ce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f424000c4d0 tx=0x7f424000c7e0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 shutdown_connections 2023-10-16T09:44:27.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f423c06af00 0x7f423c06d3c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42500cd170 0x7f425006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f425015f150 0x7f4250152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 --2- 172.21.15.181:0/3159856967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42500ce4f0 0x7f42501e4ce0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 >> 172.21.15.181:0/3159856967 conn(0x7f425009a930 msgr2=0x7f42500d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:27.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 shutdown_connections 2023-10-16T09:44:27.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.678+0000 7f42553d5700 1 -- 172.21.15.181:0/3159856967 wait complete. 2023-10-16T09:44:27.693 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.811+0000 7f4a2452a700 1 Processor -- start 2023-10-16T09:44:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.811+0000 7f4a2452a700 1 -- start start 2023-10-16T09:44:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.811+0000 7f4a2452a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a1c0cfaa0 0x7f4a1c15fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a2452a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c15bb80 con 0x7f4a1c070bb0 2023-10-16T09:44:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a2452a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0718a0 con 0x7f4a1c15b7a0 2023-10-16T09:44:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a2452a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c160300 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a21ac5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a1c0cfaa0 0x7f4a1c15fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a22ac7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4a1c070bb0 0x7f4a1c0ca3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48020/0 (socket says 172.21.15.181:48020) 2023-10-16T09:44:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a21ac5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a1c0cfaa0 0x7f4a1c15fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36940/0 (socket says 172.21.15.181:36940) 2023-10-16T09:44:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a22ac7700 1 -- 172.21.15.181:0/4089423154 learned_addr learned my addr 172.21.15.181:0/4089423154 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a21ac5700 1 -- 172.21.15.181:0/4089423154 >> v1:172.21.15.190:6789/0 conn(0x7f4a1c15b7a0 legacy=0x7f4a1c0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a21ac5700 1 -- 172.21.15.181:0/4089423154 >> v1:172.21.15.181:6789/0 conn(0x7f4a1c070bb0 legacy=0x7f4a1c0ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a21ac5700 1 -- 172.21.15.181:0/4089423154 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0712e0 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a212c4700 1 -- 172.21.15.181:0/4089423154 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35956597 0 0) 0x7f4a1c15bb80 con 0x7f4a1c070bb0 2023-10-16T09:44:27.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a21ac5700 1 --2- 172.21.15.181:0/4089423154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c15fdc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009840 tx=0x7f4a18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269bbbb1da10c0b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a212c4700 1 -- 172.21.15.181:0/4089423154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18011830 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a212c4700 1 -- 172.21.15.181:0/4089423154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a18011990 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.812+0000 7f4a212c4700 1 -- 172.21.15.181:0/4089423154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18011d20 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.813+0000 7f4a2452a700 1 -- 172.21.15.181:0/4089423154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 msgr2=0x7f4a1c15fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.813+0000 7f4a2452a700 1 --2- 172.21.15.181:0/4089423154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c15fdc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009840 tx=0x7f4a18009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:27.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.813+0000 7f4a2452a700 1 -- 172.21.15.181:0/4089423154 shutdown_connections 2023-10-16T09:44:27.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.813+0000 7f4a2452a700 1 --2- 172.21.15.181:0/4089423154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c15fdc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.813+0000 7f4a2452a700 1 -- 172.21.15.181:0/4089423154 >> 172.21.15.181:0/4089423154 conn(0x7f4a1c06aed0 msgr2=0x7f4a1c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:27.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.813+0000 7f4a2452a700 1 -- 172.21.15.181:0/4089423154 shutdown_connections 2023-10-16T09:44:27.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.813+0000 7f4a2452a700 1 -- 172.21.15.181:0/4089423154 wait complete. 2023-10-16T09:44:27.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.814+0000 7f4a2452a700 1 Processor -- start 2023-10-16T09:44:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.814+0000 7f4a2452a700 1 -- start start 2023-10-16T09:44:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a2452a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c070bb0 0x7f4a1c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a2452a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a2452a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a1c15b7a0 0x7f4a1c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a2452a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0c8810 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a2452a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0c7a60 con 0x7f4a1c15b7a0 2023-10-16T09:44:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a2452a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c158490 con 0x7f4a1c070bb0 2023-10-16T09:44:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36944/0 (socket says 172.21.15.181:36944) 2023-10-16T09:44:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 -- 172.21.15.181:0/1772009893 learned_addr learned my addr 172.21.15.181:0/1772009893 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 -- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c070bb0 msgr2=0x7f4a1c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a222c6700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c070bb0 0x7f4a1c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a22ac7700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a1c15b7a0 0x7f4a1c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c070bb0 0x7f4a1c0c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 -- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a1c15b7a0 msgr2=0x7f4a1c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a1c15b7a0 0x7f4a1c0cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 -- 172.21.15.181:0/1772009893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180094a0 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.815+0000 7f4a21ac5700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c157f50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a180090b0 tx=0x7f4a18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.816+0000 7f4a12ffd700 1 -- 172.21.15.181:0/1772009893 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1801a900 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.816+0000 7f4a12ffd700 1 -- 172.21.15.181:0/1772009893 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1801aa60 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.816+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0c7c90 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.816+0000 7f4a12ffd700 1 -- 172.21.15.181:0/1772009893 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1801add0 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.816+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0c8250 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.817+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c04ebb0 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.817+0000 7f4a12ffd700 1 -- 172.21.15.181:0/1772009893 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a18022070 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.818+0000 7f4a12ffd700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4a0006af00 0x7f4a0006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.818+0000 7f4a222c6700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4a0006af00 0x7f4a0006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.818+0000 7f4a12ffd700 1 -- 172.21.15.181:0/1772009893 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a1808a820 con 0x7f4a1c0cfaa0 2023-10-16T09:44:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.818+0000 7f4a222c6700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4a0006af00 0x7f4a0006d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c000f60 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:27.822+0000 7f4a12ffd700 1 -- 172.21.15.181:0/1772009893 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a18055a10 con 0x7f4a1c0cfaa0 2023-10-16T09:44:28.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.016+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a1c0b1d40 con 0x7f4a1c0cfaa0 2023-10-16T09:44:28.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.017+0000 7f4a12ffd700 1 -- 172.21.15.181:0/1772009893 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f4a18029090 con 0x7f4a1c0cfaa0 2023-10-16T09:44:28.020 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:28.020 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:28.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4a0006af00 msgr2=0x7f4a0006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4a0006af00 0x7f4a0006d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c000f60 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2023-10-16T09:44:28.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 msgr2=0x7f4a1c157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c157f50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a180090b0 tx=0x7f4a18004750 comp rx=0 tx=0).stop 2023-10-16T09:44:28.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 shutdown_connections 2023-10-16T09:44:28.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4a0006af00 0x7f4a0006d3c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a1c070bb0 0x7f4a1c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a1c15b7a0 0x7f4a1c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 --2- 172.21.15.181:0/1772009893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a1c0cfaa0 0x7f4a1c157f50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 >> 172.21.15.181:0/1772009893 conn(0x7f4a1c06aed0 msgr2=0x7f4a1c0b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:28.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 shutdown_connections 2023-10-16T09:44:28.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.019+0000 7f4a2452a700 1 -- 172.21.15.181:0/1772009893 wait complete. 2023-10-16T09:44:28.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:28.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.156+0000 7facf8275700 1 Processor -- start 2023-10-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.156+0000 7facf8275700 1 -- start start 2023-10-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf8275700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf0072760 0x7facf006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf8275700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facf006ade0 0x7facf01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf8275700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf006b230 con 0x7facf015da70 2023-10-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf8275700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf00cef00 con 0x7facf006ade0 2023-10-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf8275700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf00cec20 con 0x7facf0072760 2023-10-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf6812700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facf006ade0 0x7facf01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf6812700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facf006ade0 0x7facf01615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36956/0 (socket says 172.21.15.181:36956) 2023-10-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.157+0000 7facf6812700 1 -- 172.21.15.181:0/2589663081 learned_addr learned my addr 172.21.15.181:0/2589663081 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf5810700 1 --2- 172.21.15.181:0/2589663081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf0072760 0x7facf006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf6812700 1 -- 172.21.15.181:0/2589663081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf0072760 msgr2=0x7facf006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf6812700 1 --2- 172.21.15.181:0/2589663081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf0072760 0x7facf006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf6812700 1 -- 172.21.15.181:0/2589663081 >> v1:172.21.15.190:6789/0 conn(0x7facf015da70 legacy=0x7facf00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf6812700 1 -- 172.21.15.181:0/2589663081 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0071c80 con 0x7facf006ade0 2023-10-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf5810700 1 --2- 172.21.15.181:0/2589663081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf0072760 0x7facf006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf6812700 1 --2- 172.21.15.181:0/2589663081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf01615f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7face8009e50 tx=0x7face800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9af3fdd9a7eea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf500f700 1 -- 172.21.15.181:0/2589663081 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face800e070 con 0x7facf006ade0 2023-10-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf500f700 1 -- 172.21.15.181:0/2589663081 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8003a20 con 0x7facf006ade0 2023-10-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf500f700 1 -- 172.21.15.181:0/2589663081 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8003dd0 con 0x7facf006ade0 2023-10-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf8275700 1 -- 172.21.15.181:0/2589663081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 msgr2=0x7facf01615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf8275700 1 --2- 172.21.15.181:0/2589663081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf01615f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7face8009e50 tx=0x7face800b330 comp rx=0 tx=0).stop 2023-10-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf8275700 1 -- 172.21.15.181:0/2589663081 shutdown_connections 2023-10-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.158+0000 7facf8275700 1 --2- 172.21.15.181:0/2589663081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf01615f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.159+0000 7facf8275700 1 --2- 172.21.15.181:0/2589663081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf0072760 0x7facf006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.159+0000 7facf8275700 1 -- 172.21.15.181:0/2589663081 >> 172.21.15.181:0/2589663081 conn(0x7facf00cd010 msgr2=0x7facf00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.159+0000 7facf8275700 1 -- 172.21.15.181:0/2589663081 shutdown_connections 2023-10-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.159+0000 7facf8275700 1 -- 172.21.15.181:0/2589663081 wait complete. 2023-10-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 Processor -- start 2023-10-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 -- start start 2023-10-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf014a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf0072760 0x7facf00af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf015da70 0x7facf00c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00c5860 con 0x7facf006ade0 2023-10-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00c5b40 con 0x7facf0072760 2023-10-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.160+0000 7facf8275700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf00c5e20 con 0x7facf015da70 2023-10-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf014a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf014a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36958/0 (socket says 172.21.15.181:36958) 2023-10-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 -- 172.21.15.181:0/2543429790 learned_addr learned my addr 172.21.15.181:0/2543429790 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf5810700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf0072760 0x7facf00af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6812700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf015da70 0x7facf00c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 -- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf015da70 msgr2=0x7facf00c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf015da70 0x7facf00c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 -- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf0072760 msgr2=0x7facf00af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf0072760 0x7facf00af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 -- 172.21.15.181:0/2543429790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face8009b30 con 0x7facf006ade0 2023-10-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf6011700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf014a750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7face000c4d0 tx=0x7face000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7face6ffd700 1 -- 172.21.15.181:0/2543429790 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011790 con 0x7facf006ade0 2023-10-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7face6ffd700 1 -- 172.21.15.181:0/2543429790 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face00118f0 con 0x7facf006ade0 2023-10-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf00c6100 con 0x7facf006ade0 2023-10-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7face6ffd700 1 -- 172.21.15.181:0/2543429790 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011ca0 con 0x7facf006ade0 2023-10-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.161+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf00c54c0 con 0x7facf006ade0 2023-10-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.162+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf00cd8b0 con 0x7facf006ade0 2023-10-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.162+0000 7face6ffd700 1 -- 172.21.15.181:0/2543429790 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7face001c460 con 0x7facf006ade0 2023-10-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.163+0000 7face6ffd700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7facd406af00 0x7facd406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.163+0000 7face6ffd700 1 -- 172.21.15.181:0/2543429790 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7face00894b0 con 0x7facf006ade0 2023-10-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.163+0000 7facf5810700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7facd406af00 0x7facd406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.163+0000 7facf5810700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7facd406af00 0x7facd406d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7facec006fd0 tx=0x7facec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.167+0000 7face6ffd700 1 -- 172.21.15.181:0/2543429790 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7face0054560 con 0x7facf006ade0 2023-10-16T09:44:28.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.357+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facf0067ca0 con 0x7facf006ade0 2023-10-16T09:44:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.357+0000 7face6ffd700 1 -- 172.21.15.181:0/2543429790 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7face0057b80 con 0x7facf006ade0 2023-10-16T09:44:28.360 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7facd406af00 msgr2=0x7facd406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7facd406af00 0x7facd406d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7facec006fd0 tx=0x7facec008040 comp rx=0 tx=0).stop 2023-10-16T09:44:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 msgr2=0x7facf014a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf014a750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7face000c4d0 tx=0x7face000c890 comp rx=0 tx=0).stop 2023-10-16T09:44:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 shutdown_connections 2023-10-16T09:44:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7facd406af00 0x7facd406d3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facf015da70 0x7facf00c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf006ade0 0x7facf014a750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 --2- 172.21.15.181:0/2543429790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf0072760 0x7facf00af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.359+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 >> 172.21.15.181:0/2543429790 conn(0x7facf00cd010 msgr2=0x7facf00ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:28.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.360+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 shutdown_connections 2023-10-16T09:44:28.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.360+0000 7facf8275700 1 -- 172.21.15.181:0/2543429790 wait complete. 2023-10-16T09:44:28.374 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.495+0000 7f1c34011700 1 Processor -- start 2023-10-16T09:44:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.495+0000 7f1c34011700 1 -- start start 2023-10-16T09:44:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c34011700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1c2c0cc6e0 0x7f1c2c071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c34011700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c0ce1f0 con 0x7f1c2c0b2010 2023-10-16T09:44:28.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c34011700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c0ccda0 con 0x7f1c2c0ce680 2023-10-16T09:44:28.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c34011700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c0ccb90 con 0x7f1c2c0cc6e0 2023-10-16T09:44:28.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c31dad700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1c2c0b2010 0x7f1c2c0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48044/0 (socket says 172.21.15.181:48044) 2023-10-16T09:44:28.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c31dad700 1 -- 172.21.15.181:0/507821274 learned_addr learned my addr 172.21.15.181:0/507821274 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:28.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c315ac700 1 --2- 172.21.15.181:0/507821274 >> v2:172.21.15.190:3300/0 conn(0x7f1c2c0cc6e0 0x7f1c2c071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561075819 0 0) 0x7f1c2c0ce1f0 con 0x7f1c2c0b2010 2023-10-16T09:44:28.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c180035b0 con 0x7f1c2c0b2010 2023-10-16T09:44:28.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2422699725 0 0) 0x7f1c180035b0 con 0x7f1c2c0b2010 2023-10-16T09:44:28.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 >> v2:172.21.15.190:3300/0 conn(0x7f1c2c0cc6e0 msgr2=0x7f1c2c071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 --2- 172.21.15.181:0/507821274 >> v2:172.21.15.190:3300/0 conn(0x7f1c2c0cc6e0 0x7f1c2c071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 >> v1:172.21.15.190:6789/0 conn(0x7f1c2c0ce680 legacy=0x7f1c2c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0727e0 con 0x7f1c2c0b2010 2023-10-16T09:44:28.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f1c1c001670 con 0x7f1c2c0b2010 2023-10-16T09:44:28.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 >> v1:172.21.15.181:6789/0 conn(0x7f1c2c0b2010 legacy=0x7f1c2c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18003a00 0x7f1c18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18004400 0x7f1c180048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180050d0 0x7f1c1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0ccb90 con 0x7f1c18004400 2023-10-16T09:44:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1800c030 con 0x7f1c18003a00 2023-10-16T09:44:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1800c1f0 con 0x7f1c180050d0 2023-10-16T09:44:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.496+0000 7f1c315ac700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18004400 0x7f1c180048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c31dad700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18003a00 0x7f1c18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c325ae700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180050d0 0x7f1c1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c315ac700 1 -- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180050d0 msgr2=0x7f1c1800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c315ac700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180050d0 0x7f1c1800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c315ac700 1 -- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18003a00 msgr2=0x7f1c18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c315ac700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18003a00 0x7f1c18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c325ae700 1 -- 172.21.15.181:0/507821274 reap_dead start 2023-10-16T09:44:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c315ac700 1 -- 172.21.15.181:0/507821274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180035b0 con 0x7f1c18004400 2023-10-16T09:44:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c315ac700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18004400 0x7f1c180048d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009b60 tx=0x7f1c18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797bfc9190173b6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800e6c0 con 0x7f1c18004400 2023-10-16T09:44:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2800e820 con 0x7f1c18004400 2023-10-16T09:44:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c30dab700 1 -- 172.21.15.181:0/507821274 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800eb90 con 0x7f1c18004400 2023-10-16T09:44:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c34011700 1 -- 172.21.15.181:0/507821274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18004400 msgr2=0x7f1c180048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.497+0000 7f1c34011700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18004400 0x7f1c180048d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009b60 tx=0x7f1c18005880 comp rx=0 tx=0).stop 2023-10-16T09:44:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.498+0000 7f1c34011700 1 -- 172.21.15.181:0/507821274 shutdown_connections 2023-10-16T09:44:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.498+0000 7f1c34011700 1 --2- 172.21.15.181:0/507821274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18004400 0x7f1c180048d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.498+0000 7f1c34011700 1 -- 172.21.15.181:0/507821274 >> 172.21.15.181:0/507821274 conn(0x7f1c2c0cd010 msgr2=0x7f1c2c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:28.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.498+0000 7f1c34011700 1 -- 172.21.15.181:0/507821274 shutdown_connections 2023-10-16T09:44:28.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.498+0000 7f1c34011700 1 -- 172.21.15.181:0/507821274 wait complete. 2023-10-16T09:44:28.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.499+0000 7f1c34011700 1 Processor -- start 2023-10-16T09:44:28.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.499+0000 7f1c34011700 1 -- start start 2023-10-16T09:44:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.499+0000 7f1c34011700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18004400 0x7f1c2c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.499+0000 7f1c34011700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c2c1564b0 0x7f1c2c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c34011700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c2c154cb0 0x7f1c2c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c34011700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0b3150 con 0x7f1c2c1564b0 2023-10-16T09:44:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c34011700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0b3430 con 0x7f1c18004400 2023-10-16T09:44:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c34011700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0b3710 con 0x7f1c2c154cb0 2023-10-16T09:44:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c2c1564b0 0x7f1c2c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c2c1564b0 0x7f1c2c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:36980/0 (socket says 172.21.15.181:36980) 2023-10-16T09:44:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 -- 172.21.15.181:0/1674581432 learned_addr learned my addr 172.21.15.181:0/1674581432 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c325ae700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c2c154cb0 0x7f1c2c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c31dad700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18004400 0x7f1c2c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 -- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c2c154cb0 msgr2=0x7f1c2c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c2c154cb0 0x7f1c2c155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 -- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18004400 msgr2=0x7f1c2c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18004400 0x7f1c2c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 -- 172.21.15.181:0/1674581432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0b39f0 con 0x7f1c2c1564b0 2023-10-16T09:44:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c31dad700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18004400 0x7f1c2c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c315ac700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c2c1564b0 0x7f1c2c156910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c000 tx=0x7f1c280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c227fc700 1 -- 172.21.15.181:0/1674581432 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800e6c0 con 0x7f1c2c1564b0 2023-10-16T09:44:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c227fc700 1 -- 172.21.15.181:0/1674581432 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28003ad0 con 0x7f1c2c1564b0 2023-10-16T09:44:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c0b3cd0 con 0x7f1c2c1564b0 2023-10-16T09:44:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c227fc700 1 -- 172.21.15.181:0/1674581432 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28003e80 con 0x7f1c2c1564b0 2023-10-16T09:44:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.500+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c0adb70 con 0x7f1c2c1564b0 2023-10-16T09:44:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.501+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c14ca70 con 0x7f1c2c1564b0 2023-10-16T09:44:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.502+0000 7f1c227fc700 1 -- 172.21.15.181:0/1674581432 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1c28013470 con 0x7f1c2c1564b0 2023-10-16T09:44:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.502+0000 7f1c227fc700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c1006af00 0x7f1c1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.502+0000 7f1c227fc700 1 -- 172.21.15.181:0/1674581432 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c2808b180 con 0x7f1c2c1564b0 2023-10-16T09:44:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.502+0000 7f1c31dad700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c1006af00 0x7f1c1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.502+0000 7f1c31dad700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c1006af00 0x7f1c1006d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0697f0 tx=0x7f1c1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.506+0000 7f1c227fc700 1 -- 172.21.15.181:0/1674581432 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c2c14ca70 con 0x7f1c2c1564b0 2023-10-16T09:44:28.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.695+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c2c14ca70 con 0x7f1c2c1564b0 2023-10-16T09:44:28.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.696+0000 7f1c227fc700 1 -- 172.21.15.181:0/1674581432 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c2c14ca70 con 0x7f1c2c1564b0 2023-10-16T09:44:28.698 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:28.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c1006af00 msgr2=0x7f1c1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c1006af00 0x7f1c1006d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0697f0 tx=0x7f1c1c009040 comp rx=0 tx=0).stop 2023-10-16T09:44:28.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c2c1564b0 msgr2=0x7f1c2c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c2c1564b0 0x7f1c2c156910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c000 tx=0x7f1c280047c0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 shutdown_connections 2023-10-16T09:44:28.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c1006af00 0x7f1c1006d3c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c2c154cb0 0x7f1c2c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c18004400 0x7f1c2c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 --2- 172.21.15.181:0/1674581432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c2c1564b0 0x7f1c2c156910 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 >> 172.21.15.181:0/1674581432 conn(0x7f1c2c0cd010 msgr2=0x7f1c2c158990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:28.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 shutdown_connections 2023-10-16T09:44:28.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.698+0000 7f1c34011700 1 -- 172.21.15.181:0/1674581432 wait complete. 2023-10-16T09:44:28.712 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:28.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.834+0000 7f6ad1f80700 1 Processor -- start 2023-10-16T09:44:28.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.834+0000 7f6ad1f80700 1 -- start start 2023-10-16T09:44:28.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.834+0000 7f6ad1f80700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0cc6e0 0x7f6acc071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.834+0000 7f6ad1f80700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0ccb60 con 0x7f6acc0b2010 2023-10-16T09:44:28.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.834+0000 7f6ad1f80700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0ce1f0 con 0x7f6acc0ce680 2023-10-16T09:44:28.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.834+0000 7f6ad1f80700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc072120 con 0x7f6acc0cc6e0 2023-10-16T09:44:28.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6acb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6acc0b2010 0x7f6acc0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48064/0 (socket says 172.21.15.181:48064) 2023-10-16T09:44:28.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6acb7fe700 1 -- 172.21.15.181:0/3668903605 learned_addr learned my addr 172.21.15.181:0/3668903605 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:28.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6acaffd700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0cc6e0 0x7f6acc071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994871764 0 0) 0x7f6abc001110 con 0x7f6acc0b2010 2023-10-16T09:44:28.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab40035b0 con 0x7f6acc0b2010 2023-10-16T09:44:28.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639182192 0 0) 0x7f6acc0ce1f0 con 0x7f6acc0ce680 2023-10-16T09:44:28.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6abc001110 con 0x7f6acc0ce680 2023-10-16T09:44:28.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381921078 0 0) 0x7f6ab40035b0 con 0x7f6acc0b2010 2023-10-16T09:44:28.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0cc6e0 msgr2=0x7f6acc071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0cc6e0 0x7f6acc071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 >> v1:172.21.15.190:6789/0 conn(0x7f6acc0ce680 legacy=0x7f6acc0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0ccda0 con 0x7f6acc0b2010 2023-10-16T09:44:28.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f6abc001800 con 0x7f6acc0b2010 2023-10-16T09:44:28.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 >> v1:172.21.15.181:6789/0 conn(0x7f6acc0b2010 legacy=0x7f6acc0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab4003bc0 0x7f6ab4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab4004850 0x7f6ab4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab4005210 0x7f6ab400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc072120 con 0x7f6ab4005210 2023-10-16T09:44:28.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab400c030 con 0x7f6ab4003bc0 2023-10-16T09:44:28.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab400c1f0 con 0x7f6ab4004850 2023-10-16T09:44:28.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6acaffd700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0cc6e0 0x7f6acc071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:28.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6acbfff700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab4005210 0x7f6ab400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6acb7fe700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab4003bc0 0x7f6ab4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.835+0000 7f6acaffd700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab4004850 0x7f6ab4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6acbfff700 1 -- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab4004850 msgr2=0x7f6ab4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6acbfff700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab4004850 0x7f6ab4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6acbfff700 1 -- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab4003bc0 msgr2=0x7f6ab4003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:28.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6acbfff700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab4003bc0 0x7f6ab4003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6acbfff700 1 -- 172.21.15.181:0/3668903605 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab40035b0 con 0x7f6ab4005210 2023-10-16T09:44:28.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6acbfff700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab4005210 0x7f6ab400ba10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c320 tx=0x7f6ab4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67d998865e1b147 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4001f20 con 0x7f6ab4005210 2023-10-16T09:44:28.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac4003a00 con 0x7f6ab4005210 2023-10-16T09:44:28.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6aca7fc700 1 -- 172.21.15.181:0/3668903605 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400cea0 con 0x7f6ab4005210 2023-10-16T09:44:28.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6ad1f80700 1 -- 172.21.15.181:0/3668903605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab4005210 msgr2=0x7f6ab400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6ad1f80700 1 --2- 172.21.15.181:0/3668903605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab4005210 0x7f6ab400ba10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c320 tx=0x7f6ab4005880 comp rx=0 tx=0).stop 2023-10-16T09:44:28.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6acbfff700 1 -- 172.21.15.181:0/3668903605 reap_dead start 2023-10-16T09:44:28.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6ad1f80700 1 -- 172.21.15.181:0/3668903605 shutdown_connections 2023-10-16T09:44:28.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6ad1f80700 1 -- 172.21.15.181:0/3668903605 >> 172.21.15.181:0/3668903605 conn(0x7f6acc0cd010 msgr2=0x7f6acc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:28.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6ad1f80700 1 -- 172.21.15.181:0/3668903605 shutdown_connections 2023-10-16T09:44:28.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.836+0000 7f6ad1f80700 1 -- 172.21.15.181:0/3668903605 wait complete. 2023-10-16T09:44:28.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.838+0000 7f6ad1f80700 1 Processor -- start 2023-10-16T09:44:28.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.838+0000 7f6ad1f80700 1 -- start start 2023-10-16T09:44:28.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.838+0000 7f6ad1f80700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0ae3d0 0x7f6acc1f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.838+0000 7f6ad1f80700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6acc1f47b0 0x7f6acc0c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ad1f80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc0d0a70 0x7f6acc0d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ad1f80700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0c9cf0 con 0x7f6acc0d0a70 2023-10-16T09:44:28.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ad1f80700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0d1430 con 0x7f6acc1f47b0 2023-10-16T09:44:28.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ad1f80700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1f4530 con 0x7f6acc0ae3d0 2023-10-16T09:44:28.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc0d0a70 0x7f6acc0d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc0d0a70 0x7f6acc0d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37002/0 (socket says 172.21.15.181:37002) 2023-10-16T09:44:28.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 -- 172.21.15.181:0/1055015897 learned_addr learned my addr 172.21.15.181:0/1055015897 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:28.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acb7fe700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0ae3d0 0x7f6acc1f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acaffd700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6acc1f47b0 0x7f6acc0c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 -- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0ae3d0 msgr2=0x7f6acc1f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0ae3d0 0x7f6acc1f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 -- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6acc1f47b0 msgr2=0x7f6acc0c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:28.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6acc1f47b0 0x7f6acc0c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:28.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 -- 172.21.15.181:0/1055015897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc1f39b0 con 0x7f6acc0d0a70 2023-10-16T09:44:28.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6acbfff700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc0d0a70 0x7f6acc0d0ef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4015040 tx=0x7f6ac4001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ab3fff700 1 -- 172.21.15.181:0/1055015897 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40050d0 con 0x7f6acc0d0a70 2023-10-16T09:44:28.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ab3fff700 1 -- 172.21.15.181:0/1055015897 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac4005250 con 0x7f6acc0d0a70 2023-10-16T09:44:28.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc1f3c90 con 0x7f6acc0d0a70 2023-10-16T09:44:28.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ab3fff700 1 -- 172.21.15.181:0/1055015897 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4005600 con 0x7f6acc0d0a70 2023-10-16T09:44:28.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.839+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc1f4250 con 0x7f6acc0d0a70 2023-10-16T09:44:28.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.840+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc0b6510 con 0x7f6acc0d0a70 2023-10-16T09:44:28.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.841+0000 7f6ab3fff700 1 -- 172.21.15.181:0/1055015897 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ac4019040 con 0x7f6acc0d0a70 2023-10-16T09:44:28.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.841+0000 7f6ab3fff700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ab806af00 0x7f6ab806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:28.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.841+0000 7f6ab3fff700 1 -- 172.21.15.181:0/1055015897 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ac408a580 con 0x7f6acc0d0a70 2023-10-16T09:44:28.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.841+0000 7f6acb7fe700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ab806af00 0x7f6ab806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:28.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.841+0000 7f6acb7fe700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ab806af00 0x7f6ab806d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6abc001cf0 tx=0x7f6abc001c80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:28.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:28.846+0000 7f6ab3fff700 1 -- 172.21.15.181:0/1055015897 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6ac4055780 con 0x7f6acc0d0a70 2023-10-16T09:44:29.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.039+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6acc0b1120 con 0x7f6acc0d0a70 2023-10-16T09:44:29.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.039+0000 7f6ab3fff700 1 -- 172.21.15.181:0/1055015897 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ac4002080 con 0x7f6acc0d0a70 2023-10-16T09:44:29.042 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:29.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.041+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ab806af00 msgr2=0x7f6ab806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.041+0000 7f6ad1f80700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ab806af00 0x7f6ab806d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6abc001cf0 tx=0x7f6abc001c80 comp rx=0 tx=0).stop 2023-10-16T09:44:29.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.041+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc0d0a70 msgr2=0x7f6acc0d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.041+0000 7f6ad1f80700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc0d0a70 0x7f6acc0d0ef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4015040 tx=0x7f6ac4001650 comp rx=0 tx=0).stop 2023-10-16T09:44:29.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 shutdown_connections 2023-10-16T09:44:29.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ab806af00 0x7f6ab806d3c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6acc0ae3d0 0x7f6acc1f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc0d0a70 0x7f6acc0d0ef0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 --2- 172.21.15.181:0/1055015897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6acc1f47b0 0x7f6acc0c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 >> 172.21.15.181:0/1055015897 conn(0x7f6acc0cd010 msgr2=0x7f6acc0b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:29.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 shutdown_connections 2023-10-16T09:44:29.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.042+0000 7f6ad1f80700 1 -- 172.21.15.181:0/1055015897 wait complete. 2023-10-16T09:44:29.056 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:29.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.174+0000 7f5957a4b700 1 Processor -- start 2023-10-16T09:44:29.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.175+0000 7f5957a4b700 1 -- start start 2023-10-16T09:44:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.175+0000 7f5957a4b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f595015dc40 0x7f595006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.175+0000 7f5957a4b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595006ade0 0x7f59500c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.175+0000 7f5957a4b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59500c2bf0 0x7f595015e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.175+0000 7f5957a4b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500b2710 con 0x7f595015dc40 2023-10-16T09:44:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f5957a4b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500c3080 con 0x7f59500c2bf0 2023-10-16T09:44:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f5957a4b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500c18f0 con 0x7f595006ade0 2023-10-16T09:44:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f595015dc40 0x7f595006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f595015dc40 0x7f595006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37006/0 (socket says 172.21.15.181:37006) 2023-10-16T09:44:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 -- 172.21.15.181:0/1981530759 learned_addr learned my addr 172.21.15.181:0/1981530759 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f5954fe6700 1 --2- 172.21.15.181:0/1981530759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595006ade0 0x7f59500c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f5955fe8700 1 --2- 172.21.15.181:0/1981530759 >> v2:172.21.15.190:3300/0 conn(0x7f59500c2bf0 0x7f595015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 -- 172.21.15.181:0/1981530759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595006ade0 msgr2=0x7f59500c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 --2- 172.21.15.181:0/1981530759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595006ade0 0x7f59500c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 -- 172.21.15.181:0/1981530759 >> v2:172.21.15.190:3300/0 conn(0x7f59500c2bf0 msgr2=0x7f595015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 --2- 172.21.15.181:0/1981530759 >> v2:172.21.15.190:3300/0 conn(0x7f59500c2bf0 0x7f595015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 -- 172.21.15.181:0/1981530759 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500c1bd0 con 0x7f595015dc40 2023-10-16T09:44:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f59557e7700 1 --2- 172.21.15.181:0/1981530759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015dc40 0x7f595006a8a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5950072a90 tx=0x7f594000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0ff2589ba0068d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f5947fff700 1 -- 172.21.15.181:0/1981530759 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000dc20 con 0x7f595015dc40 2023-10-16T09:44:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f5947fff700 1 -- 172.21.15.181:0/1981530759 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594000dd80 con 0x7f595015dc40 2023-10-16T09:44:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.176+0000 7f5947fff700 1 -- 172.21.15.181:0/1981530759 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940017630 con 0x7f595015dc40 2023-10-16T09:44:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 -- 172.21.15.181:0/1981530759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015dc40 msgr2=0x7f595006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 --2- 172.21.15.181:0/1981530759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015dc40 0x7f595006a8a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5950072a90 tx=0x7f594000be70 comp rx=0 tx=0).stop 2023-10-16T09:44:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 -- 172.21.15.181:0/1981530759 shutdown_connections 2023-10-16T09:44:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 --2- 172.21.15.181:0/1981530759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f595015dc40 0x7f595006a8a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 --2- 172.21.15.181:0/1981530759 >> v2:172.21.15.190:3300/0 conn(0x7f59500c2bf0 0x7f595015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 --2- 172.21.15.181:0/1981530759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595006ade0 0x7f59500c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 -- 172.21.15.181:0/1981530759 >> 172.21.15.181:0/1981530759 conn(0x7f59500b1780 msgr2=0x7f59500b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 -- 172.21.15.181:0/1981530759 shutdown_connections 2023-10-16T09:44:29.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.177+0000 7f5957a4b700 1 -- 172.21.15.181:0/1981530759 wait complete. 2023-10-16T09:44:29.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.178+0000 7f5957a4b700 1 Processor -- start 2023-10-16T09:44:29.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.178+0000 7f5957a4b700 1 -- start start 2023-10-16T09:44:29.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5957a4b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595006ade0 0x7f59501559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5957a4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59500c2bf0 0x7f595006de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5957a4b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595015dc40 0x7f595006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5957a4b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950158710 con 0x7f59500c2bf0 2023-10-16T09:44:29.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5957a4b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59501e2d30 con 0x7f595006ade0 2023-10-16T09:44:29.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5957a4b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5950157f80 con 0x7f595015dc40 2023-10-16T09:44:29.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59500c2bf0 0x7f595006de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59500c2bf0 0x7f595006de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37014/0 (socket says 172.21.15.181:37014) 2023-10-16T09:44:29.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 -- 172.21.15.181:0/557237454 learned_addr learned my addr 172.21.15.181:0/557237454 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:29.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5955fe8700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595015dc40 0x7f595006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f59557e7700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595006ade0 0x7f59501559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 -- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595015dc40 msgr2=0x7f595006e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595015dc40 0x7f595006e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 -- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595006ade0 msgr2=0x7f59501559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595006ade0 0x7f59501559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 -- 172.21.15.181:0/557237454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594000b7f0 con 0x7f59500c2bf0 2023-10-16T09:44:29.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5955fe8700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595015dc40 0x7f595006e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:29.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f59557e7700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595006ade0 0x7f59501559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:29.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.179+0000 7f5954fe6700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59500c2bf0 0x7f595006de40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e810 tx=0x7f594c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.180+0000 7f5945ffb700 1 -- 172.21.15.181:0/557237454 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c013610 con 0x7f59500c2bf0 2023-10-16T09:44:29.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.180+0000 7f5945ffb700 1 -- 172.21.15.181:0/557237454 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c013770 con 0x7f59500c2bf0 2023-10-16T09:44:29.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.180+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950158960 con 0x7f59500c2bf0 2023-10-16T09:44:29.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.180+0000 7f5945ffb700 1 -- 172.21.15.181:0/557237454 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c013b00 con 0x7f59500c2bf0 2023-10-16T09:44:29.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.180+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5950158f20 con 0x7f59500c2bf0 2023-10-16T09:44:29.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.181+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950067d20 con 0x7f59500c2bf0 2023-10-16T09:44:29.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.181+0000 7f5945ffb700 1 -- 172.21.15.181:0/557237454 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f594c013c60 con 0x7f59500c2bf0 2023-10-16T09:44:29.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.181+0000 7f5945ffb700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f593406af00 0x7f593406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.182+0000 7f5945ffb700 1 -- 172.21.15.181:0/557237454 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f594c0891b0 con 0x7f59500c2bf0 2023-10-16T09:44:29.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.182+0000 7f59557e7700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f593406af00 0x7f593406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.182+0000 7f59557e7700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f593406af00 0x7f593406d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f594000bae0 tx=0x7f594000bf90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.186+0000 7f5945ffb700 1 -- 172.21.15.181:0/557237454 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f594c054250 con 0x7f59500c2bf0 2023-10-16T09:44:29.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.379+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59500c1e80 con 0x7f59500c2bf0 2023-10-16T09:44:29.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.380+0000 7f5945ffb700 1 -- 172.21.15.181:0/557237454 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f594c057870 con 0x7f59500c2bf0 2023-10-16T09:44:29.382 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:29.382 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:29.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f593406af00 msgr2=0x7f593406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f593406af00 0x7f593406d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f594000bae0 tx=0x7f594000bf90 comp rx=0 tx=0).stop 2023-10-16T09:44:29.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59500c2bf0 msgr2=0x7f595006de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59500c2bf0 0x7f595006de40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e810 tx=0x7f594c00ebd0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 shutdown_connections 2023-10-16T09:44:29.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f593406af00 0x7f593406d3c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f595015dc40 0x7f595006e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595006ade0 0x7f59501559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 --2- 172.21.15.181:0/557237454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59500c2bf0 0x7f595006de40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 >> 172.21.15.181:0/557237454 conn(0x7f59500b1780 msgr2=0x7f59500cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:29.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 shutdown_connections 2023-10-16T09:44:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.382+0000 7f5957a4b700 1 -- 172.21.15.181:0/557237454 wait complete. 2023-10-16T09:44:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:29.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:29.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.518+0000 7f4c410a2700 1 Processor -- start 2023-10-16T09:44:29.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.518+0000 7f4c410a2700 1 -- start start 2023-10-16T09:44:29.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c410a2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c3c0ce6c0 0x7f4c3c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c410a2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c3c0cc6a0 0x7f4c3c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c410a2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0b1f70 con 0x7f4c3c15f150 2023-10-16T09:44:29.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c410a2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c0b2ae0 con 0x7f4c3c0cc6a0 2023-10-16T09:44:29.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c410a2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c0b2dc0 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c3c0cc6a0 0x7f4c3c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c3c0cc6a0 0x7f4c3c0ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37024/0 (socket says 172.21.15.181:37024) 2023-10-16T09:44:29.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 -- 172.21.15.181:0/2318128850 learned_addr learned my addr 172.21.15.181:0/2318128850 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:29.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3a59c700 1 --2- 172.21.15.181:0/2318128850 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c0ce6c0 0x7f4c3c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 -- 172.21.15.181:0/2318128850 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c0ce6c0 msgr2=0x7f4c3c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:29.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 --2- 172.21.15.181:0/2318128850 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c0ce6c0 0x7f4c3c0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 -- 172.21.15.181:0/2318128850 >> v1:172.21.15.190:6789/0 conn(0x7f4c3c15f150 legacy=0x7f4c3c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 -- 172.21.15.181:0/2318128850 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0cf9d0 con 0x7f4c3c0cc6a0 2023-10-16T09:44:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c3b59e700 1 --2- 172.21.15.181:0/2318128850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cc6a0 0x7f4c3c0ccad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c06b210 tx=0x7f4c3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cf44779ff3980c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c39d9b700 1 -- 172.21.15.181:0/2318128850 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400e070 con 0x7f4c3c0cc6a0 2023-10-16T09:44:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c39d9b700 1 -- 172.21.15.181:0/2318128850 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400b910 con 0x7f4c3c0cc6a0 2023-10-16T09:44:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.519+0000 7f4c39d9b700 1 -- 172.21.15.181:0/2318128850 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400bcc0 con 0x7f4c3c0cc6a0 2023-10-16T09:44:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 -- 172.21.15.181:0/2318128850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cc6a0 msgr2=0x7f4c3c0ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 --2- 172.21.15.181:0/2318128850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cc6a0 0x7f4c3c0ccad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c06b210 tx=0x7f4c3400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 -- 172.21.15.181:0/2318128850 shutdown_connections 2023-10-16T09:44:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 --2- 172.21.15.181:0/2318128850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cc6a0 0x7f4c3c0ccad0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 --2- 172.21.15.181:0/2318128850 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c0ce6c0 0x7f4c3c0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 -- 172.21.15.181:0/2318128850 >> 172.21.15.181:0/2318128850 conn(0x7f4c3c09a930 msgr2=0x7f4c3c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:29.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 -- 172.21.15.181:0/2318128850 shutdown_connections 2023-10-16T09:44:29.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.520+0000 7f4c410a2700 1 -- 172.21.15.181:0/2318128850 wait complete. 2023-10-16T09:44:29.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.521+0000 7f4c410a2700 1 Processor -- start 2023-10-16T09:44:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c410a2700 1 -- start start 2023-10-16T09:44:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c410a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c0cc6a0 0x7f4c3c0cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c410a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0ce6c0 0x7f4c3c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c410a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c15f150 0x7f4c3c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c410a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c15a850 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c410a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0c8870 con 0x7f4c3c15f150 2023-10-16T09:44:29.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c410a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0c8b50 con 0x7f4c3c0cc6a0 2023-10-16T09:44:29.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0ce6c0 0x7f4c3c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c3a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0ce6c0 0x7f4c3c0c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37028/0 (socket says 172.21.15.181:37028) 2023-10-16T09:44:29.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c3a59c700 1 -- 172.21.15.181:0/654154875 learned_addr learned my addr 172.21.15.181:0/654154875 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:29.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.522+0000 7f4c3ad9d700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c0cc6a0 0x7f4c3c0cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c3b59e700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c15f150 0x7f4c3c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c3a59c700 1 -- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c0cc6a0 msgr2=0x7f4c3c0cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c3a59c700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c0cc6a0 0x7f4c3c0cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c3a59c700 1 -- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c15f150 msgr2=0x7f4c3c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c3a59c700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c15f150 0x7f4c3c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c3a59c700 1 -- 172.21.15.181:0/654154875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009b30 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c3a59c700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0ce6c0 0x7f4c3c0c9630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e9b0 tx=0x7f4c3000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c237fe700 1 -- 172.21.15.181:0/654154875 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30009710 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c237fe700 1 -- 172.21.15.181:0/654154875 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3000ee50 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c237fe700 1 -- 172.21.15.181:0/654154875 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000ca50 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c0c8e30 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.523+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c151ba0 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.524+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c067d20 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.524+0000 7f4c237fe700 1 -- 172.21.15.181:0/654154875 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4c3000cbb0 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.525+0000 7f4c237fe700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4c2806af00 0x7f4c2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.525+0000 7f4c237fe700 1 -- 172.21.15.181:0/654154875 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c30089640 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.525+0000 7f4c3ad9d700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4c2806af00 0x7f4c2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.525+0000 7f4c3ad9d700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4c2806af00 0x7f4c2806d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c007fd0 tx=0x7f4c2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.529+0000 7f4c237fe700 1 -- 172.21.15.181:0/654154875 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c300546f0 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.722+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c3c1596e0 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.723+0000 7f4c237fe700 1 -- 172.21.15.181:0/654154875 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c3000ce60 con 0x7f4c3c0ce6c0 2023-10-16T09:44:29.725 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:29.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4c2806af00 msgr2=0x7f4c2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4c2806af00 0x7f4c2806d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c007fd0 tx=0x7f4c2c009040 comp rx=0 tx=0).stop 2023-10-16T09:44:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0ce6c0 msgr2=0x7f4c3c0c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0ce6c0 0x7f4c3c0c9630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e9b0 tx=0x7f4c3000ecc0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 shutdown_connections 2023-10-16T09:44:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4c2806af00 0x7f4c2806d3c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c0cc6a0 0x7f4c3c0cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c15f150 0x7f4c3c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 --2- 172.21.15.181:0/654154875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0ce6c0 0x7f4c3c0c9630 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 >> 172.21.15.181:0/654154875 conn(0x7f4c3c09a930 msgr2=0x7f4c3c15d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 shutdown_connections 2023-10-16T09:44:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.725+0000 7f4c410a2700 1 -- 172.21.15.181:0/654154875 wait complete. 2023-10-16T09:44:29.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.860+0000 7ff808670700 1 Processor -- start 2023-10-16T09:44:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.860+0000 7ff808670700 1 -- start start 2023-10-16T09:44:29.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.861+0000 7ff808670700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff80015a1f0 0x7ff80015db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.861+0000 7ff808670700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80006e4b0 con 0x7ff800159e10 2023-10-16T09:44:29.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.861+0000 7ff808670700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80006ed50 con 0x7ff80006bef0 2023-10-16T09:44:29.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.861+0000 7ff808670700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8000cecb0 con 0x7ff80015a1f0 2023-10-16T09:44:29.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff806c0d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff80015a1f0 0x7ff80015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff80640c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff800159e10 0x7ff8000ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48112/0 (socket says 172.21.15.181:48112) 2023-10-16T09:44:29.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff80640c700 1 -- 172.21.15.181:0/232500532 learned_addr learned my addr 172.21.15.181:0/232500532 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:29.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff806c0d700 1 -- 172.21.15.181:0/232500532 >> v1:172.21.15.190:6789/0 conn(0x7ff80006bef0 legacy=0x7ff8000caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff80540a700 1 -- 172.21.15.181:0/232500532 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636714868 0 0) 0x7ff80006e4b0 con 0x7ff800159e10 2023-10-16T09:44:29.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff806c0d700 1 -- 172.21.15.181:0/232500532 >> v1:172.21.15.181:6789/0 conn(0x7ff800159e10 legacy=0x7ff8000ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff806c0d700 1 -- 172.21.15.181:0/232500532 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000ceea0 con 0x7ff80015a1f0 2023-10-16T09:44:29.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff806c0d700 1 --2- 172.21.15.181:0/232500532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80015a1f0 0x7ff80015db90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800aea0 tx=0x7ff7f800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545cba71d6379816 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff80540a700 1 -- 172.21.15.181:0/232500532 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800f070 con 0x7ff80015a1f0 2023-10-16T09:44:29.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff80540a700 1 -- 172.21.15.181:0/232500532 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f800c9c0 con 0x7ff80015a1f0 2023-10-16T09:44:29.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.862+0000 7ff80540a700 1 -- 172.21.15.181:0/232500532 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800cd70 con 0x7ff80015a1f0 2023-10-16T09:44:29.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.863+0000 7ff808670700 1 -- 172.21.15.181:0/232500532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80015a1f0 msgr2=0x7ff80015db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.863+0000 7ff808670700 1 --2- 172.21.15.181:0/232500532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80015a1f0 0x7ff80015db90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800aea0 tx=0x7ff7f800c330 comp rx=0 tx=0).stop 2023-10-16T09:44:29.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.863+0000 7ff808670700 1 -- 172.21.15.181:0/232500532 shutdown_connections 2023-10-16T09:44:29.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.863+0000 7ff808670700 1 --2- 172.21.15.181:0/232500532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80015a1f0 0x7ff80015db90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.863+0000 7ff808670700 1 -- 172.21.15.181:0/232500532 >> 172.21.15.181:0/232500532 conn(0x7ff8000cc730 msgr2=0x7ff8000ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:29.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.863+0000 7ff808670700 1 -- 172.21.15.181:0/232500532 shutdown_connections 2023-10-16T09:44:29.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.863+0000 7ff808670700 1 -- 172.21.15.181:0/232500532 wait complete. 2023-10-16T09:44:29.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.864+0000 7ff808670700 1 Processor -- start 2023-10-16T09:44:29.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.864+0000 7ff808670700 1 -- start start 2023-10-16T09:44:29.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff808670700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80006bef0 0x7ff8000c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff808670700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff800159e10 0x7ff800152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff808670700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff80015a1f0 0x7ff80015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff808670700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff800155dd0 con 0x7ff80006bef0 2023-10-16T09:44:29.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff808670700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff800156020 con 0x7ff800159e10 2023-10-16T09:44:29.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff808670700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff800156300 con 0x7ff80015a1f0 2023-10-16T09:44:29.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80006bef0 0x7ff8000c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80006bef0 0x7ff8000c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37040/0 (socket says 172.21.15.181:37040) 2023-10-16T09:44:29.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 -- 172.21.15.181:0/3418640928 learned_addr learned my addr 172.21.15.181:0/3418640928 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:29.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff806c0d700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff80015a1f0 0x7ff80015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 -- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff80015a1f0 msgr2=0x7ff80015b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff80015a1f0 0x7ff80015b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 -- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff800159e10 msgr2=0x7ff800152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:29.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff805c0b700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff800159e10 0x7ff800152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff800159e10 0x7ff800152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:29.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 -- 172.21.15.181:0/3418640928 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f800ab80 con 0x7ff80006bef0 2023-10-16T09:44:29.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff806c0d700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff80015a1f0 0x7ff80015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:29.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff805c0b700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff800159e10 0x7ff800152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:29.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff80640c700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80006bef0 0x7ff8000c6540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000c520 tx=0x7ff7f000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff7f6ffd700 1 -- 172.21.15.181:0/3418640928 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0004c80 con 0x7ff80006bef0 2023-10-16T09:44:29.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff7f6ffd700 1 -- 172.21.15.181:0/3418640928 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f0004e00 con 0x7ff80006bef0 2023-10-16T09:44:29.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff7f6ffd700 1 -- 172.21.15.181:0/3418640928 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0007310 con 0x7ff80006bef0 2023-10-16T09:44:29.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.865+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8001565e0 con 0x7ff80006bef0 2023-10-16T09:44:29.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.866+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8000c6a80 con 0x7ff80006bef0 2023-10-16T09:44:29.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.866+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8000cd5d0 con 0x7ff80006bef0 2023-10-16T09:44:29.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.867+0000 7ff7f6ffd700 1 -- 172.21.15.181:0/3418640928 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff7f0007470 con 0x7ff80006bef0 2023-10-16T09:44:29.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.868+0000 7ff7f6ffd700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff7e406af00 0x7ff7e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:29.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.868+0000 7ff7f6ffd700 1 -- 172.21.15.181:0/3418640928 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7f0089a60 con 0x7ff80006bef0 2023-10-16T09:44:29.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.868+0000 7ff805c0b700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff7e406af00 0x7ff7e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:29.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.868+0000 7ff805c0b700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff7e406af00 0x7ff7e406d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff8000cae80 tx=0x7ff7fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:29.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:29.871+0000 7ff7f6ffd700 1 -- 172.21.15.181:0/3418640928 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff7f0054af0 con 0x7ff80006bef0 2023-10-16T09:44:30.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.063+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff800154180 con 0x7ff80006bef0 2023-10-16T09:44:30.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.064+0000 7ff7f6ffd700 1 -- 172.21.15.181:0/3418640928 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7f0058110 con 0x7ff80006bef0 2023-10-16T09:44:30.066 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:30.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.065+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff7e406af00 msgr2=0x7ff7e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff7e406af00 0x7ff7e406d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff8000cae80 tx=0x7ff7fc009040 comp rx=0 tx=0).stop 2023-10-16T09:44:30.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80006bef0 msgr2=0x7ff8000c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80006bef0 0x7ff8000c6540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000c520 tx=0x7ff7f000c8e0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 shutdown_connections 2023-10-16T09:44:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff7e406af00 0x7ff7e406d3c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff80015a1f0 0x7ff80015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff80006bef0 0x7ff8000c6540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 --2- 172.21.15.181:0/3418640928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff800159e10 0x7ff800152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 >> 172.21.15.181:0/3418640928 conn(0x7ff8000cc730 msgr2=0x7ff80015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:30.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 shutdown_connections 2023-10-16T09:44:30.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.066+0000 7ff808670700 1 -- 172.21.15.181:0/3418640928 wait complete. 2023-10-16T09:44:30.080 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.200+0000 7f836e76e700 1 Processor -- start 2023-10-16T09:44:30.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.200+0000 7f836e76e700 1 -- start start 2023-10-16T09:44:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f836e76e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f83680cf270 0x7f836806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f836e76e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83680cb340 con 0x7f836815ac20 2023-10-16T09:44:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f836e76e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836815b0d0 con 0x7f8368158360 2023-10-16T09:44:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f836e76e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836815b3b0 con 0x7f83680cf270 2023-10-16T09:44:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f836cd0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f83680cf270 0x7f836806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f836cd0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f83680cf270 0x7f836806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37052/0 (socket says 172.21.15.181:37052) 2023-10-16T09:44:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f836cd0b700 1 -- 172.21.15.181:0/4147230931 learned_addr learned my addr 172.21.15.181:0/4147230931 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.201+0000 7f83677fe700 1 --1- 172.21.15.181:0/4147230931 >> v1:172.21.15.181:6789/0 conn(0x7f836815ac20 0x7f83680ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48126/0 (socket says 172.21.15.181:48126) 2023-10-16T09:44:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f836cd0b700 1 -- 172.21.15.181:0/4147230931 >> v1:172.21.15.190:6789/0 conn(0x7f8368158360 legacy=0x7f836815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f836cd0b700 1 -- 172.21.15.181:0/4147230931 >> v1:172.21.15.181:6789/0 conn(0x7f836815ac20 legacy=0x7f83680ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f836cd0b700 1 -- 172.21.15.181:0/4147230931 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836815b690 con 0x7f83680cf270 2023-10-16T09:44:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f8366ffd700 1 -- 172.21.15.181:0/4147230931 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289196200 0 0) 0x7f83680cb340 con 0x7f836815ac20 2023-10-16T09:44:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f836cd0b700 1 --2- 172.21.15.181:0/4147230931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f836806a8a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f836000d150 tx=0x7f836000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f81480a4350952 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f8366ffd700 1 -- 172.21.15.181:0/4147230931 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360010070 con 0x7f83680cf270 2023-10-16T09:44:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f8366ffd700 1 -- 172.21.15.181:0/4147230931 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836000ddc0 con 0x7f83680cf270 2023-10-16T09:44:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.202+0000 7f8366ffd700 1 -- 172.21.15.181:0/4147230931 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000b310 con 0x7f83680cf270 2023-10-16T09:44:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.203+0000 7f836e76e700 1 -- 172.21.15.181:0/4147230931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 msgr2=0x7f836806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.203+0000 7f836e76e700 1 --2- 172.21.15.181:0/4147230931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f836806a8a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f836000d150 tx=0x7f836000d460 comp rx=0 tx=0).stop 2023-10-16T09:44:30.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.203+0000 7f836e76e700 1 -- 172.21.15.181:0/4147230931 shutdown_connections 2023-10-16T09:44:30.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.203+0000 7f836e76e700 1 --2- 172.21.15.181:0/4147230931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f836806a8a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.203+0000 7f836e76e700 1 -- 172.21.15.181:0/4147230931 >> 172.21.15.181:0/4147230931 conn(0x7f83680cc730 msgr2=0x7f83680ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:30.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.203+0000 7f836e76e700 1 -- 172.21.15.181:0/4147230931 shutdown_connections 2023-10-16T09:44:30.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.203+0000 7f836e76e700 1 -- 172.21.15.181:0/4147230931 wait complete. 2023-10-16T09:44:30.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.204+0000 7f836e76e700 1 Processor -- start 2023-10-16T09:44:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.204+0000 7f836e76e700 1 -- start start 2023-10-16T09:44:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836e76e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f83681490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836e76e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8368158360 0x7f83680c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836e76e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836815ac20 0x7f836815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836e76e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8368149620 con 0x7f83680cf270 2023-10-16T09:44:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836e76e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83680c4110 con 0x7f836815ac20 2023-10-16T09:44:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836e76e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83680c43f0 con 0x7f8368158360 2023-10-16T09:44:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f83681490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f83681490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37054/0 (socket says 172.21.15.181:37054) 2023-10-16T09:44:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 -- 172.21.15.181:0/3368643027 learned_addr learned my addr 172.21.15.181:0/3368643027 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f83677fe700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8368158360 0x7f83680c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836cd0b700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836815ac20 0x7f836815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 -- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8368158360 msgr2=0x7f83680c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8368158360 0x7f83680c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 -- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836815ac20 msgr2=0x7f836815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836815ac20 0x7f836815cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 -- 172.21.15.181:0/3368643027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836000bd40 con 0x7f83680cf270 2023-10-16T09:44:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836cd0b700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836815ac20 0x7f836815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8367fff700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f83681490e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c4e0 tx=0x7f835c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8364ff9700 1 -- 172.21.15.181:0/3368643027 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0052d0 con 0x7f83680cf270 2023-10-16T09:44:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8364ff9700 1 -- 172.21.15.181:0/3368643027 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c005430 con 0x7f83680cf270 2023-10-16T09:44:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83680c46d0 con 0x7f83680cf270 2023-10-16T09:44:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.205+0000 7f8364ff9700 1 -- 172.21.15.181:0/3368643027 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0038c0 con 0x7f83680cf270 2023-10-16T09:44:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.206+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83680c4c90 con 0x7f83680cf270 2023-10-16T09:44:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.206+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83680c4eb0 con 0x7f83680cf270 2023-10-16T09:44:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.207+0000 7f8364ff9700 1 -- 172.21.15.181:0/3368643027 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f835c003a20 con 0x7f83680cf270 2023-10-16T09:44:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.207+0000 7f8364ff9700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f835406af00 0x7f835406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.207+0000 7f8364ff9700 1 -- 172.21.15.181:0/3368643027 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f835c088a20 con 0x7f83680cf270 2023-10-16T09:44:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.207+0000 7f83677fe700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f835406af00 0x7f835406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.208+0000 7f83677fe700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f835406af00 0x7f835406d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f83680af050 tx=0x7f8358007d90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.212+0000 7f8364ff9700 1 -- 172.21.15.181:0/3368643027 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f835c053b60 con 0x7f83680cf270 2023-10-16T09:44:30.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.402+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f836806ad00 con 0x7f83680cf270 2023-10-16T09:44:30.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.403+0000 7f8364ff9700 1 -- 172.21.15.181:0/3368643027 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f835c0170a0 con 0x7f83680cf270 2023-10-16T09:44:30.405 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:30.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f835406af00 msgr2=0x7f835406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f835406af00 0x7f835406d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f83680af050 tx=0x7f8358007d90 comp rx=0 tx=0).stop 2023-10-16T09:44:30.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 msgr2=0x7f83681490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f83681490e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c4e0 tx=0x7f835c00c8a0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 shutdown_connections 2023-10-16T09:44:30.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f835406af00 0x7f835406d3c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f836815ac20 0x7f836815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83680cf270 0x7f83681490e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 --2- 172.21.15.181:0/3368643027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8368158360 0x7f83680c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 >> 172.21.15.181:0/3368643027 conn(0x7f83680cc730 msgr2=0x7f83680b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:30.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 shutdown_connections 2023-10-16T09:44:30.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.405+0000 7f836e76e700 1 -- 172.21.15.181:0/3368643027 wait complete. 2023-10-16T09:44:30.431 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-16T09:44:30.432 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T09:44:30.432 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:30.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.551+0000 7fb270116700 1 Processor -- start 2023-10-16T09:44:30.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.551+0000 7fb270116700 1 -- start start 2023-10-16T09:44:30.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb270116700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb26815e6d0 0x7fb2680d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb270116700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb26815eab0 0x7fb26815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb270116700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26806ed00 con 0x7fb2680cb0c0 2023-10-16T09:44:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb270116700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2680d02d0 con 0x7fb26815e6d0 2023-10-16T09:44:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb270116700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26815ad40 con 0x7fb26815eab0 2023-10-16T09:44:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb26815e6d0 0x7fb2680d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb26815e6d0 0x7fb2680d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37062/0 (socket says 172.21.15.181:37062) 2023-10-16T09:44:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 -- 172.21.15.181:0/3645359307 learned_addr learned my addr 172.21.15.181:0/3645359307 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26d6b1700 1 --2- 172.21.15.181:0/3645359307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb26815eab0 0x7fb26815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 -- 172.21.15.181:0/3645359307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb26815eab0 msgr2=0x7fb26815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 --2- 172.21.15.181:0/3645359307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb26815eab0 0x7fb26815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 -- 172.21.15.181:0/3645359307 >> v1:172.21.15.190:6789/0 conn(0x7fb2680cb0c0 legacy=0x7fb26815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 -- 172.21.15.181:0/3645359307 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26806e0f0 con 0x7fb26815e6d0 2023-10-16T09:44:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26deb2700 1 --2- 172.21.15.181:0/3645359307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680d0fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb264006fd0 tx=0x7fb264009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a889427974b41db0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26ceb0700 1 -- 172.21.15.181:0/3645359307 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264005e30 con 0x7fb26815e6d0 2023-10-16T09:44:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26ceb0700 1 -- 172.21.15.181:0/3645359307 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb264004030 con 0x7fb26815e6d0 2023-10-16T09:44:30.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.552+0000 7fb26ceb0700 1 -- 172.21.15.181:0/3645359307 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2640043e0 con 0x7fb26815e6d0 2023-10-16T09:44:30.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 -- 172.21.15.181:0/3645359307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 msgr2=0x7fb2680d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 --2- 172.21.15.181:0/3645359307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680d0fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb264006fd0 tx=0x7fb264009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 -- 172.21.15.181:0/3645359307 shutdown_connections 2023-10-16T09:44:30.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 --2- 172.21.15.181:0/3645359307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680d0fc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 --2- 172.21.15.181:0/3645359307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb26815eab0 0x7fb26815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 -- 172.21.15.181:0/3645359307 >> 172.21.15.181:0/3645359307 conn(0x7fb2680d04a0 msgr2=0x7fb2680d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:30.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 -- 172.21.15.181:0/3645359307 shutdown_connections 2023-10-16T09:44:30.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.553+0000 7fb270116700 1 -- 172.21.15.181:0/3645359307 wait complete. 2023-10-16T09:44:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.554+0000 7fb270116700 1 Processor -- start 2023-10-16T09:44:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.554+0000 7fb270116700 1 -- start start 2023-10-16T09:44:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb270116700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2680cb0c0 0x7fb268158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb270116700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb270116700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb26815eab0 0x7fb2680c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb270116700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680ca3d0 con 0x7fb26815e6d0 2023-10-16T09:44:30.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb270116700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680b4600 con 0x7fb26815eab0 2023-10-16T09:44:30.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb270116700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680b39f0 con 0x7fb2680cb0c0 2023-10-16T09:44:30.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37070/0 (socket says 172.21.15.181:37070) 2023-10-16T09:44:30.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 -- 172.21.15.181:0/3420324890 learned_addr learned my addr 172.21.15.181:0/3420324890 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:30.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26e6b3700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb26815eab0 0x7fb2680c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26deb2700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2680cb0c0 0x7fb268158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 -- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2680cb0c0 msgr2=0x7fb268158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2680cb0c0 0x7fb268158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 -- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb26815eab0 msgr2=0x7fb2680c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb26815eab0 0x7fb2680c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 -- 172.21.15.181:0/3420324890 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb264009500 con 0x7fb26815e6d0 2023-10-16T09:44:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.555+0000 7fb26d6b1700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680b32f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb25800e9e0 tx=0x7fb25800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.556+0000 7fb25e7fc700 1 -- 172.21.15.181:0/3420324890 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258009710 con 0x7fb26815e6d0 2023-10-16T09:44:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.556+0000 7fb25e7fc700 1 -- 172.21.15.181:0/3420324890 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25800ee80 con 0x7fb26815e6d0 2023-10-16T09:44:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.556+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2680b3cd0 con 0x7fb26815e6d0 2023-10-16T09:44:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.556+0000 7fb25e7fc700 1 -- 172.21.15.181:0/3420324890 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800ca50 con 0x7fb26815e6d0 2023-10-16T09:44:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.556+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2680b4290 con 0x7fb26815e6d0 2023-10-16T09:44:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.556+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2680b4790 con 0x7fb26815e6d0 2023-10-16T09:44:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.557+0000 7fb25e7fc700 1 -- 172.21.15.181:0/3420324890 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb25800cbb0 con 0x7fb26815e6d0 2023-10-16T09:44:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.557+0000 7fb25e7fc700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb24c06af00 0x7fb24c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.557+0000 7fb25e7fc700 1 -- 172.21.15.181:0/3420324890 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2580895e0 con 0x7fb26815e6d0 2023-10-16T09:44:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.558+0000 7fb26deb2700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb24c06af00 0x7fb24c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.558+0000 7fb26deb2700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb24c06af00 0x7fb24c06d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb264006fd0 tx=0x7fb264008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.562+0000 7fb25e7fc700 1 -- 172.21.15.181:0/3420324890 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb258054600 con 0x7fb26815e6d0 2023-10-16T09:44:30.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.753+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb268067d20 con 0x7fb26815e6d0 2023-10-16T09:44:30.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.755+0000 7fb25e7fc700 1 -- 172.21.15.181:0/3420324890 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fb258057c20 con 0x7fb26815e6d0 2023-10-16T09:44:30.757 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:30.757 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb24c06af00 msgr2=0x7fb24c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb24c06af00 0x7fb24c06d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb264006fd0 tx=0x7fb264008070 comp rx=0 tx=0).stop 2023-10-16T09:44:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 msgr2=0x7fb2680b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680b32f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb25800e9e0 tx=0x7fb25800ecf0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 shutdown_connections 2023-10-16T09:44:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb24c06af00 0x7fb24c06d3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2680cb0c0 0x7fb268158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb26815eab0 0x7fb2680c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 --2- 172.21.15.181:0/3420324890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb26815e6d0 0x7fb2680b32f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 >> 172.21.15.181:0/3420324890 conn(0x7fb2680d04a0 msgr2=0x7fb268159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:30.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 shutdown_connections 2023-10-16T09:44:30.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.757+0000 7fb270116700 1 -- 172.21.15.181:0/3420324890 wait complete. 2023-10-16T09:44:30.771 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:30.772 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:30.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.893+0000 7f37d2479700 1 Processor -- start 2023-10-16T09:44:30.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.893+0000 7f37d2479700 1 -- start start 2023-10-16T09:44:30.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37d2479700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f37cc15f150 0x7f37cc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37d2479700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37cc0ce4f0 0x7f37cc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37d2479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37d2479700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37cc0ce9a0 con 0x7f37cc0ce4f0 2023-10-16T09:44:30.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37d2479700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37cc0b2ae0 con 0x7f37cc15f150 2023-10-16T09:44:30.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37d2479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0b2dc0 con 0x7f37cc0b14a0 2023-10-16T09:44:30.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37cb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37cc0ce4f0 0x7f37cc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37cb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37cc0ce4f0 0x7f37cc0b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37076/0 (socket says 172.21.15.181:37076) 2023-10-16T09:44:30.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37cb7fe700 1 -- 172.21.15.181:0/3686815318 learned_addr learned my addr 172.21.15.181:0/3686815318 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:30.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37d0a16700 1 --2- 172.21.15.181:0/3686815318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.894+0000 7f37cbfff700 1 --2- 172.21.15.181:0/3686815318 >> v2:172.21.15.190:3300/0 conn(0x7f37cc15f150 0x7f37cc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37cb7fe700 1 -- 172.21.15.181:0/3686815318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 msgr2=0x7f37cc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:30.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37cb7fe700 1 --2- 172.21.15.181:0/3686815318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37cb7fe700 1 -- 172.21.15.181:0/3686815318 >> v2:172.21.15.190:3300/0 conn(0x7f37cc15f150 msgr2=0x7f37cc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:30.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37cb7fe700 1 --2- 172.21.15.181:0/3686815318 >> v2:172.21.15.190:3300/0 conn(0x7f37cc15f150 0x7f37cc0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37cb7fe700 1 -- 172.21.15.181:0/3686815318 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc0cf9d0 con 0x7f37cc0ce4f0 2023-10-16T09:44:30.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37cbfff700 1 --2- 172.21.15.181:0/3686815318 >> v2:172.21.15.190:3300/0 conn(0x7f37cc15f150 0x7f37cc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:30.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37cb7fe700 1 --2- 172.21.15.181:0/3686815318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc0b0f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0098f0 tx=0x7f37bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52a855f00b0864 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37caffd700 1 -- 172.21.15.181:0/3686815318 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc011830 con 0x7f37cc0ce4f0 2023-10-16T09:44:30.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37caffd700 1 -- 172.21.15.181:0/3686815318 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc011990 con 0x7f37cc0ce4f0 2023-10-16T09:44:30.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37caffd700 1 -- 172.21.15.181:0/3686815318 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc011d20 con 0x7f37cc0ce4f0 2023-10-16T09:44:30.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37d2479700 1 -- 172.21.15.181:0/3686815318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc0ce4f0 msgr2=0x7f37cc0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.895+0000 7f37d2479700 1 --2- 172.21.15.181:0/3686815318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc0b0f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0098f0 tx=0x7f37bc009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:30.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.896+0000 7f37d2479700 1 -- 172.21.15.181:0/3686815318 shutdown_connections 2023-10-16T09:44:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.896+0000 7f37d2479700 1 --2- 172.21.15.181:0/3686815318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.896+0000 7f37d2479700 1 --2- 172.21.15.181:0/3686815318 >> v2:172.21.15.190:3300/0 conn(0x7f37cc15f150 0x7f37cc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.896+0000 7f37d2479700 1 --2- 172.21.15.181:0/3686815318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc0b0f60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.896+0000 7f37d2479700 1 -- 172.21.15.181:0/3686815318 >> 172.21.15.181:0/3686815318 conn(0x7f37cc09a930 msgr2=0x7f37cc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:30.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.896+0000 7f37d2479700 1 -- 172.21.15.181:0/3686815318 shutdown_connections 2023-10-16T09:44:30.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.896+0000 7f37d2479700 1 -- 172.21.15.181:0/3686815318 wait complete. 2023-10-16T09:44:30.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.897+0000 7f37d2479700 1 Processor -- start 2023-10-16T09:44:30.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.897+0000 7f37d2479700 1 -- start start 2023-10-16T09:44:30.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc1f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc15f150 0x7f37cc0c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc1f53d0 con 0x7f37cc15f150 2023-10-16T09:44:30.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0c9a20 con 0x7f37cc0ce4f0 2023-10-16T09:44:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0ca9e0 con 0x7f37cc0b14a0 2023-10-16T09:44:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc15f150 0x7f37cc0c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc15f150 0x7f37cc0c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37086/0 (socket says 172.21.15.181:37086) 2023-10-16T09:44:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 -- 172.21.15.181:0/303415247 learned_addr learned my addr 172.21.15.181:0/303415247 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:30.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37cbfff700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37cb7fe700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc1f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 -- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 msgr2=0x7f37cc0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 -- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc0ce4f0 msgr2=0x7f37cc1f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:30.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc1f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 -- 172.21.15.181:0/303415247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc009550 con 0x7f37cc15f150 2023-10-16T09:44:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37cb7fe700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37cbfff700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d0a16700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc15f150 0x7f37cc0c94e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f37c400ef00 tx=0x7f37c400b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37c8ff9700 1 -- 172.21.15.181:0/303415247 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4010040 con 0x7f37cc15f150 2023-10-16T09:44:30.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37c8ff9700 1 -- 172.21.15.181:0/303415247 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c400bb70 con 0x7f37cc15f150 2023-10-16T09:44:30.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc0cacc0 con 0x7f37cc15f150 2023-10-16T09:44:30.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37c8ff9700 1 -- 172.21.15.181:0/303415247 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4014400 con 0x7f37cc15f150 2023-10-16T09:44:30.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.898+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc0c62b0 con 0x7f37cc15f150 2023-10-16T09:44:30.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.899+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc0c64a0 con 0x7f37cc15f150 2023-10-16T09:44:30.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.900+0000 7f37c8ff9700 1 -- 172.21.15.181:0/303415247 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f37c4014560 con 0x7f37cc15f150 2023-10-16T09:44:30.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.900+0000 7f37c8ff9700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f37b806af00 0x7f37b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:30.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.900+0000 7f37c8ff9700 1 -- 172.21.15.181:0/303415247 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37c40899b0 con 0x7f37cc15f150 2023-10-16T09:44:30.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.900+0000 7f37cbfff700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f37b806af00 0x7f37b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:30.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.901+0000 7f37cbfff700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f37b806af00 0x7f37b806d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37c00099f0 tx=0x7f37c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:30.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:30.905+0000 7f37c8ff9700 1 -- 172.21.15.181:0/303415247 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f37c4054a70 con 0x7f37cc15f150 2023-10-16T09:44:31.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.095+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37cc067d20 con 0x7f37cc15f150 2023-10-16T09:44:31.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.096+0000 7f37c8ff9700 1 -- 172.21.15.181:0/303415247 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37c4058090 con 0x7f37cc15f150 2023-10-16T09:44:31.098 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:31.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f37b806af00 msgr2=0x7f37b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f37b806af00 0x7f37b806d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37c00099f0 tx=0x7f37c0008040 comp rx=0 tx=0).stop 2023-10-16T09:44:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc15f150 msgr2=0x7f37cc0c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc15f150 0x7f37cc0c94e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f37c400ef00 tx=0x7f37c400b0b0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 shutdown_connections 2023-10-16T09:44:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f37b806af00 0x7f37b806d3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37cc0b14a0 0x7f37cc0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37cc15f150 0x7f37cc0c94e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 --2- 172.21.15.181:0/303415247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc0ce4f0 0x7f37cc1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 >> 172.21.15.181:0/303415247 conn(0x7f37cc09a930 msgr2=0x7f37cc0cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 shutdown_connections 2023-10-16T09:44:31.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.098+0000 7f37d2479700 1 -- 172.21.15.181:0/303415247 wait complete. 2023-10-16T09:44:31.112 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:31.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.234+0000 7fc466cf0700 1 Processor -- start 2023-10-16T09:44:31.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.234+0000 7fc466cf0700 1 -- start start 2023-10-16T09:44:31.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc466cf0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 0x7fc4600cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc466cf0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc460071be0 0x7fc4600b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc466cf0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4600ce1f0 con 0x7fc46006f3b0 2023-10-16T09:44:31.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc466cf0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4600b2930 con 0x7fc460071be0 2023-10-16T09:44:31.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc466cf0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600ccda0 con 0x7fc4600b2010 2023-10-16T09:44:31.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc46528d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc46006f3b0 0x7fc46006f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48168/0 (socket says 172.21.15.181:48168) 2023-10-16T09:44:31.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc46528d700 1 -- 172.21.15.181:0/2925482553 learned_addr learned my addr 172.21.15.181:0/2925482553 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:31.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc464a8c700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 0x7fc4600cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595952518 0 0) 0x7fc4600ce1f0 con 0x7fc46006f3b0 2023-10-16T09:44:31.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4500035b0 con 0x7fc46006f3b0 2023-10-16T09:44:31.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45ffff700 1 --2- 172.21.15.181:0/2925482553 >> v2:172.21.15.190:3300/0 conn(0x7fc460071be0 0x7fc4600b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3239156472 0 0) 0x7fc4500035b0 con 0x7fc46006f3b0 2023-10-16T09:44:31.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 msgr2=0x7fc4600cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45f7fe700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 0x7fc4600cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 >> v2:172.21.15.190:3300/0 conn(0x7fc460071be0 msgr2=0x7fc4600b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.235+0000 7fc45f7fe700 1 --2- 172.21.15.181:0/2925482553 >> v2:172.21.15.190:3300/0 conn(0x7fc460071be0 0x7fc4600b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4600727e0 con 0x7fc46006f3b0 2023-10-16T09:44:31.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc4580019f0 con 0x7fc46006f3b0 2023-10-16T09:44:31.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 >> v1:172.21.15.181:6789/0 conn(0x7fc46006f3b0 legacy=0x7fc46006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 0x7fc450003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450003f80 0x7fc450004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc450004d90 0x7fc450005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600b2930 con 0x7fc450004d90 2023-10-16T09:44:31.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600ccda0 con 0x7fc450003f80 2023-10-16T09:44:31.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45000ba60 con 0x7fc4600b2010 2023-10-16T09:44:31.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc46528d700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc450004d90 0x7fc450005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc464a8c700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 0x7fc450003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45ffff700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450003f80 0x7fc450004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc46528d700 1 -- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 msgr2=0x7fc450003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc46528d700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4600b2010 0x7fc450003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc46528d700 1 -- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450003f80 msgr2=0x7fc450004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc46528d700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450003f80 0x7fc450004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc46528d700 1 -- 172.21.15.181:0/2925482553 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4500035b0 con 0x7fc450004d90 2023-10-16T09:44:31.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc46528d700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc450004d90 0x7fc450005210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc45800a8d0 tx=0x7fc450005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5cc00d7518c5df server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800a340 con 0x7fc450004d90 2023-10-16T09:44:31.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc458012600 con 0x7fc450004d90 2023-10-16T09:44:31.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.236+0000 7fc45f7fe700 1 -- 172.21.15.181:0/2925482553 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580129b0 con 0x7fc450004d90 2023-10-16T09:44:31.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.237+0000 7fc466cf0700 1 -- 172.21.15.181:0/2925482553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc450004d90 msgr2=0x7fc450005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.237+0000 7fc466cf0700 1 --2- 172.21.15.181:0/2925482553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc450004d90 0x7fc450005210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc45800a8d0 tx=0x7fc450005880 comp rx=0 tx=0).stop 2023-10-16T09:44:31.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.237+0000 7fc46528d700 1 -- 172.21.15.181:0/2925482553 reap_dead start 2023-10-16T09:44:31.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.237+0000 7fc466cf0700 1 -- 172.21.15.181:0/2925482553 shutdown_connections 2023-10-16T09:44:31.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.237+0000 7fc466cf0700 1 -- 172.21.15.181:0/2925482553 >> 172.21.15.181:0/2925482553 conn(0x7fc4600cd010 msgr2=0x7fc4600cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:31.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.237+0000 7fc466cf0700 1 -- 172.21.15.181:0/2925482553 shutdown_connections 2023-10-16T09:44:31.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.237+0000 7fc466cf0700 1 -- 172.21.15.181:0/2925482553 wait complete. 2023-10-16T09:44:31.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.238+0000 7fc466cf0700 1 Processor -- start 2023-10-16T09:44:31.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.238+0000 7fc466cf0700 1 -- start start 2023-10-16T09:44:31.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc466cf0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450004d90 0x7fc46014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc466cf0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc46015cac0 0x7fc4600c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc466cf0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc46015cf40 0x7fc4600c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc466cf0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4601500f0 con 0x7fc46015cac0 2023-10-16T09:44:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc466cf0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4601f36a0 con 0x7fc450004d90 2023-10-16T09:44:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc466cf0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4601f3940 con 0x7fc46015cf40 2023-10-16T09:44:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc46015cac0 0x7fc4600c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc46015cac0 0x7fc4600c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37104/0 (socket says 172.21.15.181:37104) 2023-10-16T09:44:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 -- 172.21.15.181:0/1077782192 learned_addr learned my addr 172.21.15.181:0/1077782192 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc46528d700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc46015cf40 0x7fc4600c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc464a8c700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450004d90 0x7fc46014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 -- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc46015cf40 msgr2=0x7fc4600c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc46015cf40 0x7fc4600c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 -- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450004d90 msgr2=0x7fc46014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450004d90 0x7fc46014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.239+0000 7fc45ffff700 1 -- 172.21.15.181:0/1077782192 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4601f3c20 con 0x7fc46015cac0 2023-10-16T09:44:31.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.240+0000 7fc45ffff700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc46015cac0 0x7fc4600c9910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0116d0 tx=0x7fc44c0119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.240+0000 7fc45d7fa700 1 -- 172.21.15.181:0/1077782192 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00c070 con 0x7fc46015cac0 2023-10-16T09:44:31.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.240+0000 7fc45d7fa700 1 -- 172.21.15.181:0/1077782192 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c015870 con 0x7fc46015cac0 2023-10-16T09:44:31.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.240+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4601f3f00 con 0x7fc46015cac0 2023-10-16T09:44:31.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.240+0000 7fc45d7fa700 1 -- 172.21.15.181:0/1077782192 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c015be0 con 0x7fc46015cac0 2023-10-16T09:44:31.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.240+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46015c540 con 0x7fc46015cac0 2023-10-16T09:44:31.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.241+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46015c730 con 0x7fc46015cac0 2023-10-16T09:44:31.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.241+0000 7fc45d7fa700 1 -- 172.21.15.181:0/1077782192 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc44c015d40 con 0x7fc46015cac0 2023-10-16T09:44:31.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.242+0000 7fc45d7fa700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc44406af00 0x7fc44406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.242+0000 7fc45d7fa700 1 -- 172.21.15.181:0/1077782192 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc44c0896f0 con 0x7fc46015cac0 2023-10-16T09:44:31.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.242+0000 7fc464a8c700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc44406af00 0x7fc44406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.242+0000 7fc464a8c700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc44406af00 0x7fc44406d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc454009c60 tx=0x7fc454009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.246+0000 7fc45d7fa700 1 -- 172.21.15.181:0/1077782192 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc46015c730 con 0x7fc46015cac0 2023-10-16T09:44:31.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.437+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc46015c730 con 0x7fc46015cac0 2023-10-16T09:44:31.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.437+0000 7fc45d7fa700 1 -- 172.21.15.181:0/1077782192 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc44c019020 con 0x7fc46015cac0 2023-10-16T09:44:31.440 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:31.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc44406af00 msgr2=0x7fc44406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc44406af00 0x7fc44406d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc454009c60 tx=0x7fc454009210 comp rx=0 tx=0).stop 2023-10-16T09:44:31.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc46015cac0 msgr2=0x7fc4600c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc46015cac0 0x7fc4600c9910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0116d0 tx=0x7fc44c0119e0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 shutdown_connections 2023-10-16T09:44:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc44406af00 0x7fc44406d3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc46015cf40 0x7fc4600c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc450004d90 0x7fc46014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 --2- 172.21.15.181:0/1077782192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc46015cac0 0x7fc4600c9910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.439+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 >> 172.21.15.181:0/1077782192 conn(0x7fc4600cd010 msgr2=0x7fc4600cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.440+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 shutdown_connections 2023-10-16T09:44:31.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.440+0000 7fc466cf0700 1 -- 172.21.15.181:0/1077782192 wait complete. 2023-10-16T09:44:31.454 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.575+0000 7f359b28a700 1 Processor -- start 2023-10-16T09:44:31.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.575+0000 7f359b28a700 1 -- start start 2023-10-16T09:44:31.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.576+0000 7f359b28a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f359415da70 0x7f35940ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.576+0000 7f359b28a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 0x7f359406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.576+0000 7f359b28a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35940cef00 con 0x7f3594072760 2023-10-16T09:44:31.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.576+0000 7f359b28a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35940cec20 con 0x7f359415da70 2023-10-16T09:44:31.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.576+0000 7f359b28a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3594071c80 con 0x7f35940ce6a0 2023-10-16T09:44:31.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.576+0000 7f3599827700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3594072760 0x7f35941615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48190/0 (socket says 172.21.15.181:48190) 2023-10-16T09:44:31.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.576+0000 7f3599827700 1 -- 172.21.15.181:0/1605776439 learned_addr learned my addr 172.21.15.181:0/1605776439 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:31.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 0x7f359406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3599026700 1 --2- 172.21.15.181:0/1605776439 >> v2:172.21.15.190:3300/0 conn(0x7f359415da70 0x7f35940ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187820303 0 0) 0x7f35940cef00 con 0x7f3594072760 2023-10-16T09:44:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35800035b0 con 0x7f3594072760 2023-10-16T09:44:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2925914345 0 0) 0x7f35800035b0 con 0x7f3594072760 2023-10-16T09:44:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 msgr2=0x7f359406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 0x7f359406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 >> v2:172.21.15.190:3300/0 conn(0x7f359415da70 msgr2=0x7f35940ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 --2- 172.21.15.181:0/1605776439 >> v2:172.21.15.190:3300/0 conn(0x7f359415da70 0x7f35940ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594071f60 con 0x7f3594072760 2023-10-16T09:44:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f358c0019f0 con 0x7f3594072760 2023-10-16T09:44:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 0x7f359406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3599026700 1 --2- 172.21.15.181:0/1605776439 >> v2:172.21.15.190:3300/0 conn(0x7f359415da70 0x7f35940ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 >> v1:172.21.15.181:6789/0 conn(0x7f3594072760 legacy=0x7f35941615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 0x7f3580003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f3580004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3580004d90 0x7f3580005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35940cec20 con 0x7f3580003f40 2023-10-16T09:44:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594071c80 con 0x7f3580004d90 2023-10-16T09:44:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358000bac0 con 0x7f35940ce6a0 2023-10-16T09:44:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f3580004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3599026700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 0x7f3580003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3599827700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3580004d90 0x7f3580005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 -- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 msgr2=0x7f3580003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940ce6a0 0x7f3580003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 -- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3580004d90 msgr2=0x7f3580005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3580004d90 0x7f3580005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.577+0000 7f3598825700 1 -- 172.21.15.181:0/1605776439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35900094a0 con 0x7f3580003f40 2023-10-16T09:44:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f3598825700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f3580004850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3590009d30 tx=0x7f3580005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fb42c91bcece3b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590009e80 con 0x7f3580003f40 2023-10-16T09:44:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359000e9a0 con 0x7f3580003f40 2023-10-16T09:44:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f358bfff700 1 -- 172.21.15.181:0/1605776439 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359000ed10 con 0x7f3580003f40 2023-10-16T09:44:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f359b28a700 1 -- 172.21.15.181:0/1605776439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 msgr2=0x7f3580004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f359b28a700 1 --2- 172.21.15.181:0/1605776439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f3580004850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3590009d30 tx=0x7f3580005880 comp rx=0 tx=0).stop 2023-10-16T09:44:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f3599827700 1 -- 172.21.15.181:0/1605776439 reap_dead start 2023-10-16T09:44:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f359b28a700 1 -- 172.21.15.181:0/1605776439 shutdown_connections 2023-10-16T09:44:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f359b28a700 1 -- 172.21.15.181:0/1605776439 >> 172.21.15.181:0/1605776439 conn(0x7f35940cd010 msgr2=0x7f35940cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f359b28a700 1 -- 172.21.15.181:0/1605776439 shutdown_connections 2023-10-16T09:44:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.578+0000 7f359b28a700 1 -- 172.21.15.181:0/1605776439 wait complete. 2023-10-16T09:44:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 Processor -- start 2023-10-16T09:44:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 -- start start 2023-10-16T09:44:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f359415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940c79f0 0x7f35940c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359415ebe0 0x7f359415f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35941604b0 con 0x7f3580003f40 2023-10-16T09:44:31.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359415f5d0 con 0x7f359415ebe0 2023-10-16T09:44:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f359b28a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359415aa80 con 0x7f35940c79f0 2023-10-16T09:44:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f3599026700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f359415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f3599026700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f359415a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37120/0 (socket says 172.21.15.181:37120) 2023-10-16T09:44:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f3599026700 1 -- 172.21.15.181:0/2115944863 learned_addr learned my addr 172.21.15.181:0/2115944863 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.580+0000 7f3598825700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940c79f0 0x7f35940c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3599827700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359415ebe0 0x7f359415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3599026700 1 -- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940c79f0 msgr2=0x7f35940c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3599026700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940c79f0 0x7f35940c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3599026700 1 -- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359415ebe0 msgr2=0x7f359415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3599026700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359415ebe0 0x7f359415f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3599026700 1 -- 172.21.15.181:0/2115944863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35940af9e0 con 0x7f3580003f40 2023-10-16T09:44:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3599026700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f359415a010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3584011c20 tx=0x7f3584011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3589ffb700 1 -- 172.21.15.181:0/2115944863 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358400fa00 con 0x7f3580003f40 2023-10-16T09:44:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3589ffb700 1 -- 172.21.15.181:0/2115944863 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358400fb60 con 0x7f3580003f40 2023-10-16T09:44:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359415a550 con 0x7f3580003f40 2023-10-16T09:44:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f3589ffb700 1 -- 172.21.15.181:0/2115944863 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584004020 con 0x7f3580003f40 2023-10-16T09:44:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.581+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359415e100 con 0x7f3580003f40 2023-10-16T09:44:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.582+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594070040 con 0x7f3580003f40 2023-10-16T09:44:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.583+0000 7f3589ffb700 1 -- 172.21.15.181:0/2115944863 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3584004180 con 0x7f3580003f40 2023-10-16T09:44:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.583+0000 7f3589ffb700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f357806af00 0x7f357806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.583+0000 7f3589ffb700 1 -- 172.21.15.181:0/2115944863 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3584089b10 con 0x7f3580003f40 2023-10-16T09:44:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.583+0000 7f3598825700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f357806af00 0x7f357806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.583+0000 7f3598825700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f357806af00 0x7f357806d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35900090b0 tx=0x7f3590008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.587+0000 7f3589ffb700 1 -- 172.21.15.181:0/2115944863 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3584054c50 con 0x7f3580003f40 2023-10-16T09:44:31.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.780+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f359406fd60 con 0x7f3580003f40 2023-10-16T09:44:31.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.780+0000 7f3589ffb700 1 -- 172.21.15.181:0/2115944863 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3584022070 con 0x7f3580003f40 2023-10-16T09:44:31.783 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:31.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f357806af00 msgr2=0x7f357806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f357806af00 0x7f357806d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35900090b0 tx=0x7f3590008030 comp rx=0 tx=0).stop 2023-10-16T09:44:31.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 msgr2=0x7f359415a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f359415a010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3584011c20 tx=0x7f3584011f30 comp rx=0 tx=0).stop 2023-10-16T09:44:31.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 shutdown_connections 2023-10-16T09:44:31.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f357806af00 0x7f357806d3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359415ebe0 0x7f359415f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3580003f40 0x7f359415a010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 --2- 172.21.15.181:0/2115944863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35940c79f0 0x7f35940c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 >> 172.21.15.181:0/2115944863 conn(0x7f35940cd010 msgr2=0x7f359406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:31.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 shutdown_connections 2023-10-16T09:44:31.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.783+0000 7f359b28a700 1 -- 172.21.15.181:0/2115944863 wait complete. 2023-10-16T09:44:31.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-16T09:44:31.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.917+0000 7ff109e18700 1 Processor -- start 2023-10-16T09:44:31.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.917+0000 7ff109e18700 1 -- start start 2023-10-16T09:44:31.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff109e18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 0x7ff1040af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff109e18700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10406f870 con 0x7ff1040af450 2023-10-16T09:44:31.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff109e18700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1040d1160 con 0x7ff10415dfd0 2023-10-16T09:44:31.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff109e18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040cc8e0 con 0x7ff1040cdb80 2023-10-16T09:44:31.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff102ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff1040af450 0x7ff1040cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48208/0 (socket says 172.21.15.181:48208) 2023-10-16T09:44:31.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff102ffd700 1 -- 172.21.15.181:0/1334397660 learned_addr learned my addr 172.21.15.181:0/1334397660 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:31.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff103fff700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 0x7ff1040af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903638104 0 0) 0x7ff10406f870 con 0x7ff1040af450 2023-10-16T09:44:31.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.918+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec0035b0 con 0x7ff1040af450 2023-10-16T09:44:31.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1825717698 0 0) 0x7ff0ec0035b0 con 0x7ff1040af450 2023-10-16T09:44:31.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 msgr2=0x7ff1040af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 0x7ff1040af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 >> v1:172.21.15.190:6789/0 conn(0x7ff10415dfd0 legacy=0x7ff10406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040ccbc0 con 0x7ff1040af450 2023-10-16T09:44:31.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff0f4001670 con 0x7ff1040af450 2023-10-16T09:44:31.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 >> v1:172.21.15.181:6789/0 conn(0x7ff1040af450 legacy=0x7ff1040cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec003a30 0x7ff0ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 0x7ff0ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ec004d90 0x7ff0ec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040cc8e0 con 0x7ff0ec003a30 2023-10-16T09:44:31.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec00baf0 con 0x7ff0ec004d90 2023-10-16T09:44:31.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec00bcb0 con 0x7ff1040cdb80 2023-10-16T09:44:31.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1037fe700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec003a30 0x7ff0ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff102ffd700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 0x7ff0ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff103fff700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ec004d90 0x7ff0ec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1037fe700 1 -- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 msgr2=0x7ff0ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1037fe700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1040cdb80 0x7ff0ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1037fe700 1 -- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ec004d90 msgr2=0x7ff0ec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:31.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1037fe700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ec004d90 0x7ff0ec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1037fe700 1 -- 172.21.15.181:0/1334397660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0035b0 con 0x7ff0ec003a30 2023-10-16T09:44:31.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff103fff700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0ec004d90 0x7ff0ec005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:31.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1037fe700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec003a30 0x7ff0ec003e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80018e0 tx=0x7ff0ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06ea0bf9c403e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800c660 con 0x7ff0ec003a30 2023-10-16T09:44:31.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f800c7e0 con 0x7ff0ec003a30 2023-10-16T09:44:31.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.919+0000 7ff1027fc700 1 -- 172.21.15.181:0/1334397660 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800cb90 con 0x7ff0ec003a30 2023-10-16T09:44:31.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.920+0000 7ff109e18700 1 -- 172.21.15.181:0/1334397660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec003a30 msgr2=0x7ff0ec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.920+0000 7ff109e18700 1 --2- 172.21.15.181:0/1334397660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0ec003a30 0x7ff0ec003e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80018e0 tx=0x7ff0ec005880 comp rx=0 tx=0).stop 2023-10-16T09:44:31.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.920+0000 7ff103fff700 1 -- 172.21.15.181:0/1334397660 reap_dead start 2023-10-16T09:44:31.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.920+0000 7ff109e18700 1 -- 172.21.15.181:0/1334397660 shutdown_connections 2023-10-16T09:44:31.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.920+0000 7ff109e18700 1 -- 172.21.15.181:0/1334397660 >> 172.21.15.181:0/1334397660 conn(0x7ff1040d04a0 msgr2=0x7ff1040d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:31.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.920+0000 7ff109e18700 1 -- 172.21.15.181:0/1334397660 shutdown_connections 2023-10-16T09:44:31.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.920+0000 7ff109e18700 1 -- 172.21.15.181:0/1334397660 wait complete. 2023-10-16T09:44:31.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.921+0000 7ff109e18700 1 Processor -- start 2023-10-16T09:44:31.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.921+0000 7ff109e18700 1 -- start start 2023-10-16T09:44:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff109e18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0ec003a30 0x7ff1040b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff109e18700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1040c9f70 0x7ff1040aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff109e18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040b5e20 0x7ff10406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff109e18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040b1460 con 0x7ff1040b5e20 2023-10-16T09:44:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff109e18700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040b07e0 con 0x7ff1040c9f70 2023-10-16T09:44:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff109e18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040b0ac0 con 0x7ff0ec003a30 2023-10-16T09:44:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff103fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040b5e20 0x7ff10406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff103fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040b5e20 0x7ff10406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37144/0 (socket says 172.21.15.181:37144) 2023-10-16T09:44:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff103fff700 1 -- 172.21.15.181:0/2811888812 learned_addr learned my addr 172.21.15.181:0/2811888812 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff102ffd700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1040c9f70 0x7ff1040aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.922+0000 7ff1037fe700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0ec003a30 0x7ff1040b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff103fff700 1 -- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0ec003a30 msgr2=0x7ff1040b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff103fff700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0ec003a30 0x7ff1040b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff103fff700 1 -- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1040c9f70 msgr2=0x7ff1040aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff103fff700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1040c9f70 0x7ff1040aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff103fff700 1 -- 172.21.15.181:0/2811888812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040b0da0 con 0x7ff1040b5e20 2023-10-16T09:44:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff1037fe700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0ec003a30 0x7ff1040b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff103fff700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040b5e20 0x7ff10406a8a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff104070310 tx=0x7ff0fc0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff0ebfff700 1 -- 172.21.15.181:0/2811888812 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc01b070 con 0x7ff1040b5e20 2023-10-16T09:44:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff0ebfff700 1 -- 172.21.15.181:0/2811888812 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00f040 con 0x7ff1040b5e20 2023-10-16T09:44:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1040b1080 con 0x7ff1040b5e20 2023-10-16T09:44:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff0ebfff700 1 -- 172.21.15.181:0/2811888812 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00e050 con 0x7ff1040b5e20 2023-10-16T09:44:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.923+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1040b0440 con 0x7ff1040b5e20 2023-10-16T09:44:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.924+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff104067d20 con 0x7ff1040b5e20 2023-10-16T09:44:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.924+0000 7ff0ebfff700 1 -- 172.21.15.181:0/2811888812 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff0fc003a20 con 0x7ff1040b5e20 2023-10-16T09:44:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.925+0000 7ff0ebfff700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff0f006af00 0x7ff0f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:31.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.925+0000 7ff0ebfff700 1 -- 172.21.15.181:0/2811888812 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0fc089840 con 0x7ff1040b5e20 2023-10-16T09:44:31.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.925+0000 7ff1037fe700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff0f006af00 0x7ff0f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:31.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.925+0000 7ff1037fe700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff0f006af00 0x7ff0f006d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80018e0 tx=0x7ff0f8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:31.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:31.929+0000 7ff0ebfff700 1 -- 172.21.15.181:0/2811888812 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0fc054980 con 0x7ff1040b5e20 2023-10-16T09:44:32.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.203+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7ff1040cd6f0 con 0x7ff1040b5e20 2023-10-16T09:44:32.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.911+0000 7ff0ebfff700 1 -- 172.21.15.181:0/2811888812 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff0fc057fa0 con 0x7ff1040b5e20 2023-10-16T09:44:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff0f006af00 msgr2=0x7ff0f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff0f006af00 0x7ff0f006d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80018e0 tx=0x7ff0f8009030 comp rx=0 tx=0).stop 2023-10-16T09:44:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040b5e20 msgr2=0x7ff10406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040b5e20 0x7ff10406a8a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff104070310 tx=0x7ff0fc0104a0 comp rx=0 tx=0).stop 2023-10-16T09:44:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 shutdown_connections 2023-10-16T09:44:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff0f006af00 0x7ff0f006d3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0ec003a30 0x7ff1040b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040b5e20 0x7ff10406a8a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 --2- 172.21.15.181:0/2811888812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1040c9f70 0x7ff1040aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:32.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 >> 172.21.15.181:0/2811888812 conn(0x7ff1040d04a0 msgr2=0x7ff10406d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:32.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.914+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 shutdown_connections 2023-10-16T09:44:32.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:32.915+0000 7ff109e18700 1 -- 172.21.15.181:0/2811888812 wait complete. 2023-10-16T09:44:32.931 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T09:44:33.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.058+0000 7fcd27a40700 1 Processor -- start 2023-10-16T09:44:33.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.059+0000 7fcd27a40700 1 -- start start 2023-10-16T09:44:33.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.059+0000 7fcd27a40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd2015dbd0 0x7fcd200d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.059+0000 7fcd27a40700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcd200d0610 0x7fcd200cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.059+0000 7fcd27a40700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2015dfb0 con 0x7fcd200ce2c0 2023-10-16T09:44:33.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.059+0000 7fcd27a40700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd200ce740 con 0x7fcd200d0610 2023-10-16T09:44:33.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.059+0000 7fcd27a40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200cd3a0 con 0x7fcd2015dbd0 2023-10-16T09:44:33.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcd200d0610 0x7fcd200cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcd200d0610 0x7fcd200cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37148/0 (socket says 172.21.15.181:37148) 2023-10-16T09:44:33.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 -- 172.21.15.181:0/3286619522 learned_addr learned my addr 172.21.15.181:0/3286619522 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:33.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd257dc700 1 --2- 172.21.15.181:0/3286619522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd2015dbd0 0x7fcd200d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 -- 172.21.15.181:0/3286619522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd2015dbd0 msgr2=0x7fcd200d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:33.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 --2- 172.21.15.181:0/3286619522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd2015dbd0 0x7fcd200d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 -- 172.21.15.181:0/3286619522 >> v1:172.21.15.190:6789/0 conn(0x7fcd200ce2c0 legacy=0x7fcd20161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 -- 172.21.15.181:0/3286619522 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd200cd830 con 0x7fcd200d0610 2023-10-16T09:44:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd257dc700 1 --2- 172.21.15.181:0/3286619522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd2015dbd0 0x7fcd200d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd24fdb700 1 --2- 172.21.15.181:0/3286619522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd200d0610 0x7fcd200cdd80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0098f0 tx=0x7fcd1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338da6d9e3548132 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:33.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd17fff700 1 -- 172.21.15.181:0/3286619522 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c004230 con 0x7fcd200d0610 2023-10-16T09:44:33.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd17fff700 1 -- 172.21.15.181:0/3286619522 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c004390 con 0x7fcd200d0610 2023-10-16T09:44:33.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.060+0000 7fcd17fff700 1 -- 172.21.15.181:0/3286619522 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0036a0 con 0x7fcd200d0610 2023-10-16T09:44:33.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 -- 172.21.15.181:0/3286619522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd200d0610 msgr2=0x7fcd200cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 --2- 172.21.15.181:0/3286619522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd200d0610 0x7fcd200cdd80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0098f0 tx=0x7fcd1c009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:33.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 -- 172.21.15.181:0/3286619522 shutdown_connections 2023-10-16T09:44:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 --2- 172.21.15.181:0/3286619522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd2015dbd0 0x7fcd200d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 --2- 172.21.15.181:0/3286619522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd200d0610 0x7fcd200cdd80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 -- 172.21.15.181:0/3286619522 >> 172.21.15.181:0/3286619522 conn(0x7fcd2009a930 msgr2=0x7fcd200d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 -- 172.21.15.181:0/3286619522 shutdown_connections 2023-10-16T09:44:33.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.061+0000 7fcd27a40700 1 -- 172.21.15.181:0/3286619522 wait complete. 2023-10-16T09:44:33.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.062+0000 7fcd27a40700 1 Processor -- start 2023-10-16T09:44:33.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.062+0000 7fcd27a40700 1 -- start start 2023-10-16T09:44:33.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd27a40700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd200ce2c0 0x7fcd20155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd27a40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200d0610 0x7fcd200c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd27a40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2015dbd0 0x7fcd201593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd27a40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2015e200 con 0x7fcd2015dbd0 2023-10-16T09:44:33.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd27a40700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2015eaa0 con 0x7fcd200ce2c0 2023-10-16T09:44:33.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd27a40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200cb380 con 0x7fcd200d0610 2023-10-16T09:44:33.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2015dbd0 0x7fcd201593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2015dbd0 0x7fcd201593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37158/0 (socket says 172.21.15.181:37158) 2023-10-16T09:44:33.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 -- 172.21.15.181:0/1163157787 learned_addr learned my addr 172.21.15.181:0/1163157787 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:33.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd24fdb700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200d0610 0x7fcd200c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd257dc700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd200ce2c0 0x7fcd20155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 -- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200d0610 msgr2=0x7fcd200c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200d0610 0x7fcd200c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 -- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd200ce2c0 msgr2=0x7fcd20155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd200ce2c0 0x7fcd20155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 -- 172.21.15.181:0/1163157787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009550 con 0x7fcd2015dbd0 2023-10-16T09:44:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd24fdb700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200d0610 0x7fcd200c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd257dc700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd200ce2c0 0x7fcd20155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd25fdd700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2015dbd0 0x7fcd201593e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b970 tx=0x7fcd1800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:33.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1163157787 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800c9b0 con 0x7fcd2015dbd0 2023-10-16T09:44:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.063+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1163157787 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1800cb10 con 0x7fcd2015dbd0 2023-10-16T09:44:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.064+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2015e450 con 0x7fcd2015dbd0 2023-10-16T09:44:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.064+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1163157787 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18014440 con 0x7fcd2015dbd0 2023-10-16T09:44:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.064+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd200c8c10 con 0x7fcd2015dbd0 2023-10-16T09:44:33.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.065+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd20067d20 con 0x7fcd2015dbd0 2023-10-16T09:44:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.065+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1163157787 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcd18003a00 con 0x7fcd2015dbd0 2023-10-16T09:44:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.066+0000 7fcd15ffb700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcd0406af00 0x7fcd0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.066+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1163157787 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd18089320 con 0x7fcd2015dbd0 2023-10-16T09:44:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.066+0000 7fcd257dc700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcd0406af00 0x7fcd0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.066+0000 7fcd257dc700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcd0406af00 0x7fcd0406d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd10006fd0 tx=0x7fcd10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:33.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.070+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1163157787 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcd18054540 con 0x7fcd2015dbd0 2023-10-16T09:44:33.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.262+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcd2015ebe0 con 0x7fcd2015dbd0 2023-10-16T09:44:33.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.265+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1163157787 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcd18003cb0 con 0x7fcd2015dbd0 2023-10-16T09:44:33.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcd0406af00 msgr2=0x7fcd0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcd0406af00 0x7fcd0406d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd10006fd0 tx=0x7fcd10008040 comp rx=0 tx=0).stop 2023-10-16T09:44:33.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2015dbd0 msgr2=0x7fcd201593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2015dbd0 0x7fcd201593e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b970 tx=0x7fcd1800bd30 comp rx=0 tx=0).stop 2023-10-16T09:44:33.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 shutdown_connections 2023-10-16T09:44:33.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcd0406af00 0x7fcd0406d3c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2015dbd0 0x7fcd201593e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd200ce2c0 0x7fcd20155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 --2- 172.21.15.181:0/1163157787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd200d0610 0x7fcd200c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 >> 172.21.15.181:0/1163157787 conn(0x7fcd2009a930 msgr2=0x7fcd200d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:33.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 shutdown_connections 2023-10-16T09:44:33.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.267+0000 7fcd27a40700 1 -- 172.21.15.181:0/1163157787 wait complete. 2023-10-16T09:44:33.282 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:44:33.282 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:44:33.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:44:33.374 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:44:33.375 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:33.375 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:33.393 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:33.393 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:33.449 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:33.450 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:33.507 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:33.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.664+0000 7fd132548700 1 Processor -- start 2023-10-16T09:44:33.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.664+0000 7fd132548700 1 -- start start 2023-10-16T09:44:33.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd132548700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd12c0d0370 0x7fd12c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd132548700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 0x7fd12c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd132548700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c0cc510 con 0x7fd12c15b990 2023-10-16T09:44:33.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd132548700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0d24f0 con 0x7fd12c0d0370 2023-10-16T09:44:33.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd132548700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0cc7f0 con 0x7fd12c0cd2d0 2023-10-16T09:44:33.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd12c15b990 0x7fd12c0cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48240/0 (socket says 172.21.15.181:48240) 2023-10-16T09:44:33.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12bfff700 1 -- 172.21.15.181:0/1789760718 learned_addr learned my addr 172.21.15.181:0/1789760718 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:33.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034789996 0 0) 0x7fd12c0cc510 con 0x7fd12c15b990 2023-10-16T09:44:33.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/1789760718 >> v2:172.21.15.190:3300/0 conn(0x7fd12c0d0370 0x7fd12c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1140035b0 con 0x7fd12c15b990 2023-10-16T09:44:33.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd130ae5700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 0x7fd12c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3021520123 0 0) 0x7fd1140035b0 con 0x7fd12c15b990 2023-10-16T09:44:33.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 msgr2=0x7fd12c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:33.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 0x7fd12c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 >> v2:172.21.15.190:3300/0 conn(0x7fd12c0d0370 msgr2=0x7fd12c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:33.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 --2- 172.21.15.181:0/1789760718 >> v2:172.21.15.190:3300/0 conn(0x7fd12c0d0370 0x7fd12c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c0ccad0 con 0x7fd12c15b990 2023-10-16T09:44:33.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.665+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd1200017f0 con 0x7fd12c15b990 2023-10-16T09:44:33.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 >> v1:172.21.15.181:6789/0 conn(0x7fd12c15b990 legacy=0x7fd12c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 0x7fd114003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd114004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114004d90 0x7fd114005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0d24f0 con 0x7fd114003f80 2023-10-16T09:44:33.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0cc7f0 con 0x7fd114004d90 2023-10-16T09:44:33.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400ba10 con 0x7fd12c0cd2d0 2023-10-16T09:44:33.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd114004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd130ae5700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114004d90 0x7fd114005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12bfff700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 0x7fd114003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12b7fe700 1 -- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 msgr2=0x7fd114003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:33.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0cd2d0 0x7fd114003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12b7fe700 1 -- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114004d90 msgr2=0x7fd114005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:33.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd114004d90 0x7fd114005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12b7fe700 1 -- 172.21.15.181:0/1789760718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140035b0 con 0x7fd114003f80 2023-10-16T09:44:33.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd114004850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009b60 tx=0x7fd114005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656d47e2bd615f41 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:33.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00e660 con 0x7fd114003f80 2023-10-16T09:44:33.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c00e7c0 con 0x7fd114003f80 2023-10-16T09:44:33.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.666+0000 7fd12affd700 1 -- 172.21.15.181:0/1789760718 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00eb30 con 0x7fd114003f80 2023-10-16T09:44:33.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.667+0000 7fd132548700 1 -- 172.21.15.181:0/1789760718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 msgr2=0x7fd114004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.667+0000 7fd132548700 1 --2- 172.21.15.181:0/1789760718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd114004850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009b60 tx=0x7fd114005880 comp rx=0 tx=0).stop 2023-10-16T09:44:33.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.667+0000 7fd130ae5700 1 -- 172.21.15.181:0/1789760718 reap_dead start 2023-10-16T09:44:33.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.667+0000 7fd132548700 1 -- 172.21.15.181:0/1789760718 shutdown_connections 2023-10-16T09:44:33.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.667+0000 7fd132548700 1 -- 172.21.15.181:0/1789760718 >> 172.21.15.181:0/1789760718 conn(0x7fd12c09a9a0 msgr2=0x7fd12c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:33.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.667+0000 7fd132548700 1 -- 172.21.15.181:0/1789760718 shutdown_connections 2023-10-16T09:44:33.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.667+0000 7fd132548700 1 -- 172.21.15.181:0/1789760718 wait complete. 2023-10-16T09:44:33.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.668+0000 7fd132548700 1 Processor -- start 2023-10-16T09:44:33.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.668+0000 7fd132548700 1 -- start start 2023-10-16T09:44:33.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd132548700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd12c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd132548700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12c06d1c0 0x7fd12c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd132548700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0c9df0 0x7fd12c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd132548700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0ca7b0 con 0x7fd114003f80 2023-10-16T09:44:33.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd132548700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c06bcc0 con 0x7fd12c06d1c0 2023-10-16T09:44:33.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd132548700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c06bfa0 con 0x7fd12c0c9df0 2023-10-16T09:44:33.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd12c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd12c1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37174/0 (socket says 172.21.15.181:37174) 2023-10-16T09:44:33.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 -- 172.21.15.181:0/3671312362 learned_addr learned my addr 172.21.15.181:0/3671312362 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:33.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd130ae5700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0c9df0 0x7fd12c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12c06d1c0 0x7fd12c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 -- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0c9df0 msgr2=0x7fd12c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0c9df0 0x7fd12c0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 -- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12c06d1c0 msgr2=0x7fd12c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12c06d1c0 0x7fd12c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 -- 172.21.15.181:0/3671312362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c06c280 con 0x7fd114003f80 2023-10-16T09:44:33.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd12bfff700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd12c1590e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd12000f850 tx=0x7fd12000fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:33.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.669+0000 7fd128ff9700 1 -- 172.21.15.181:0/3671312362 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1200072e0 con 0x7fd114003f80 2023-10-16T09:44:33.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.670+0000 7fd128ff9700 1 -- 172.21.15.181:0/3671312362 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd120007440 con 0x7fd114003f80 2023-10-16T09:44:33.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.670+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c06c560 con 0x7fd114003f80 2023-10-16T09:44:33.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.670+0000 7fd128ff9700 1 -- 172.21.15.181:0/3671312362 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120005830 con 0x7fd114003f80 2023-10-16T09:44:33.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.670+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c06b9b0 con 0x7fd114003f80 2023-10-16T09:44:33.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.671+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c067d20 con 0x7fd114003f80 2023-10-16T09:44:33.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.671+0000 7fd128ff9700 1 -- 172.21.15.181:0/3671312362 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd120005990 con 0x7fd114003f80 2023-10-16T09:44:33.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.671+0000 7fd128ff9700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd11806af00 0x7fd11806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:33.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.672+0000 7fd128ff9700 1 -- 172.21.15.181:0/3671312362 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd120089490 con 0x7fd114003f80 2023-10-16T09:44:33.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.672+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd11806af00 0x7fd11806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:33.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.672+0000 7fd12b7fe700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd11806af00 0x7fd11806d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0090b0 tx=0x7fd11c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:33.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.676+0000 7fd128ff9700 1 -- 172.21.15.181:0/3671312362 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd120054540 con 0x7fd114003f80 2023-10-16T09:44:33.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.873+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd12c0b73f0 con 0x7fd114003f80 2023-10-16T09:44:33.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.874+0000 7fd128ff9700 1 -- 172.21.15.181:0/3671312362 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fd120057b60 con 0x7fd114003f80 2023-10-16T09:44:33.876 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:33.877 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:33.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd11806af00 msgr2=0x7fd11806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd11806af00 0x7fd11806d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0090b0 tx=0x7fd11c008030 comp rx=0 tx=0).stop 2023-10-16T09:44:33.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 msgr2=0x7fd12c1590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:33.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd12c1590e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd12000f850 tx=0x7fd12000fb60 comp rx=0 tx=0).stop 2023-10-16T09:44:33.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 shutdown_connections 2023-10-16T09:44:33.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd11806af00 0x7fd11806d3c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd12c0c9df0 0x7fd12c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd114003f80 0x7fd12c1590e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 --2- 172.21.15.181:0/3671312362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12c06d1c0 0x7fd12c06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:33.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 >> 172.21.15.181:0/3671312362 conn(0x7fd12c09a9a0 msgr2=0x7fd12c070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:33.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 shutdown_connections 2023-10-16T09:44:33.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:33.877+0000 7fd132548700 1 -- 172.21.15.181:0/3671312362 wait complete. 2023-10-16T09:44:33.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:33.892 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:34.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.014+0000 7f7e412fb700 1 Processor -- start 2023-10-16T09:44:34.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.014+0000 7f7e412fb700 1 -- start start 2023-10-16T09:44:34.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e412fb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c0b2010 0x7f7e3c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e412fb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e3c071be0 0x7f7e3c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e412fb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e3c06f3b0 0x7f7e3c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e412fb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c0ce1f0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e412fb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c0b2930 con 0x7f7e3c071be0 2023-10-16T09:44:34.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e412fb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0ccda0 con 0x7f7e3c0b2010 2023-10-16T09:44:34.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e3b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e3c06f3b0 0x7f7e3c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e3b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e3c06f3b0 0x7f7e3c06f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37186/0 (socket says 172.21.15.181:37186) 2023-10-16T09:44:34.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e3b7fe700 1 -- 172.21.15.181:0/2856383773 learned_addr learned my addr 172.21.15.181:0/2856383773 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:34.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e3a7fc700 1 --2- 172.21.15.181:0/2856383773 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c071be0 0x7f7e3c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.015+0000 7f7e3affd700 1 --2- 172.21.15.181:0/2856383773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c0b2010 0x7f7e3c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e3b7fe700 1 -- 172.21.15.181:0/2856383773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c0b2010 msgr2=0x7f7e3c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:34.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e3b7fe700 1 --2- 172.21.15.181:0/2856383773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c0b2010 0x7f7e3c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e3b7fe700 1 -- 172.21.15.181:0/2856383773 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c071be0 msgr2=0x7f7e3c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:34.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e3b7fe700 1 --2- 172.21.15.181:0/2856383773 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c071be0 0x7f7e3c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e3b7fe700 1 -- 172.21.15.181:0/2856383773 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0727e0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e3a7fc700 1 --2- 172.21.15.181:0/2856383773 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c071be0 0x7f7e3c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:34.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e3b7fe700 1 --2- 172.21.15.181:0/2856383773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c06f840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c067600 tx=0x7f7e3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277cb9ae3d126219 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e39ffb700 1 -- 172.21.15.181:0/2856383773 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400e070 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e39ffb700 1 -- 172.21.15.181:0/2856383773 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3400b9c0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e39ffb700 1 -- 172.21.15.181:0/2856383773 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400bd70 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e412fb700 1 -- 172.21.15.181:0/2856383773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 msgr2=0x7f7e3c06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2856383773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c06f840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c067600 tx=0x7f7e3400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e412fb700 1 -- 172.21.15.181:0/2856383773 shutdown_connections 2023-10-16T09:44:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2856383773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c06f840 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2856383773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c0b2010 0x7f7e3c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2856383773 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c071be0 0x7f7e3c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.016+0000 7f7e412fb700 1 -- 172.21.15.181:0/2856383773 >> 172.21.15.181:0/2856383773 conn(0x7f7e3c0cd010 msgr2=0x7f7e3c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:34.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.017+0000 7f7e412fb700 1 -- 172.21.15.181:0/2856383773 shutdown_connections 2023-10-16T09:44:34.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.017+0000 7f7e412fb700 1 -- 172.21.15.181:0/2856383773 wait complete. 2023-10-16T09:44:34.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.018+0000 7f7e412fb700 1 Processor -- start 2023-10-16T09:44:34.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.018+0000 7f7e412fb700 1 -- start start 2023-10-16T09:44:34.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c071be0 0x7f7e3c14ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c0b2010 0x7f7e3c0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c15baf0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c8e30 con 0x7f7e3c0b2010 2023-10-16T09:44:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c1571b0 con 0x7f7e3c071be0 2023-10-16T09:44:34.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37188/0 (socket says 172.21.15.181:37188) 2023-10-16T09:44:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 -- 172.21.15.181:0/2149175902 learned_addr learned my addr 172.21.15.181:0/2149175902 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3a7fc700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c071be0 0x7f7e3c14ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3b7fe700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c0b2010 0x7f7e3c0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 -- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c071be0 msgr2=0x7f7e3c14ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c071be0 0x7f7e3c14ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 -- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c0b2010 msgr2=0x7f7e3c0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c0b2010 0x7f7e3c0b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 -- 172.21.15.181:0/2149175902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34009b30 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3b7fe700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c0b2010 0x7f7e3c0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e3affd700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c153e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000e810 tx=0x7f7e3000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e237fe700 1 -- 172.21.15.181:0/2149175902 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30013610 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e237fe700 1 -- 172.21.15.181:0/2149175902 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e30013770 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c156ed0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e237fe700 1 -- 172.21.15.181:0/2149175902 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30013b20 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.019+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c0c8ac0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.020+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c074e40 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.021+0000 7f7e237fe700 1 -- 172.21.15.181:0/2149175902 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e30013c80 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.021+0000 7f7e237fe700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e2806af00 0x7f7e2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.021+0000 7f7e237fe700 1 -- 172.21.15.181:0/2149175902 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e300892c0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.021+0000 7f7e3a7fc700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e2806af00 0x7f7e2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.021+0000 7f7e3a7fc700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e2806af00 0x7f7e2806d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c15b3b0 tx=0x7f7e2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.026+0000 7f7e237fe700 1 -- 172.21.15.181:0/2149175902 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e30054380 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.215+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e3c0b6510 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.215+0000 7f7e237fe700 1 -- 172.21.15.181:0/2149175902 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e300579a0 con 0x7f7e3c06f3b0 2023-10-16T09:44:34.217 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:34.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e2806af00 msgr2=0x7f7e2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e2806af00 0x7f7e2806d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c15b3b0 tx=0x7f7e2c008040 comp rx=0 tx=0).stop 2023-10-16T09:44:34.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 msgr2=0x7f7e3c153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c153e10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000e810 tx=0x7f7e3000ebd0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 shutdown_connections 2023-10-16T09:44:34.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e2806af00 0x7f7e2806d3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c0b2010 0x7f7e3c0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e3c06f3b0 0x7f7e3c153e10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 --2- 172.21.15.181:0/2149175902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c071be0 0x7f7e3c14ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 >> 172.21.15.181:0/2149175902 conn(0x7f7e3c0cd010 msgr2=0x7f7e3c0ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:34.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 shutdown_connections 2023-10-16T09:44:34.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.217+0000 7f7e412fb700 1 -- 172.21.15.181:0/2149175902 wait complete. 2023-10-16T09:44:34.231 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:34.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.352+0000 7f56f085b700 1 Processor -- start 2023-10-16T09:44:34.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.352+0000 7f56f085b700 1 -- start start 2023-10-16T09:44:34.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56f085b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56ec0b2010 0x7f56ec0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56f085b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56ec0ce680 0x7f56ec0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56f085b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0ce1f0 con 0x7f56ec071be0 2023-10-16T09:44:34.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56f085b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec072090 con 0x7f56ec0b2010 2023-10-16T09:44:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56f085b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec0ccda0 con 0x7f56ec0ce680 2023-10-16T09:44:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56ea59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56ec0b2010 0x7f56ec0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56e9d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f56ec071be0 0x7f56ec0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48274/0 (socket says 172.21.15.181:48274) 2023-10-16T09:44:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.353+0000 7f56e9d9b700 1 -- 172.21.15.181:0/2584166147 learned_addr learned my addr 172.21.15.181:0/2584166147 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:34.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56e959a700 1 -- 172.21.15.181:0/2584166147 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326944228 0 0) 0x7f56ec0ce1f0 con 0x7f56ec071be0 2023-10-16T09:44:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56e959a700 1 -- 172.21.15.181:0/2584166147 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d40035b0 con 0x7f56ec071be0 2023-10-16T09:44:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56ead9d700 1 --2- 172.21.15.181:0/2584166147 >> v2:172.21.15.190:3300/0 conn(0x7f56ec0ce680 0x7f56ec0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56ea59c700 1 -- 172.21.15.181:0/2584166147 >> v2:172.21.15.190:3300/0 conn(0x7f56ec0ce680 msgr2=0x7f56ec0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56ea59c700 1 --2- 172.21.15.181:0/2584166147 >> v2:172.21.15.190:3300/0 conn(0x7f56ec0ce680 0x7f56ec0ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56ea59c700 1 -- 172.21.15.181:0/2584166147 >> v1:172.21.15.181:6789/0 conn(0x7f56ec071be0 legacy=0x7f56ec0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56ea59c700 1 -- 172.21.15.181:0/2584166147 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0727e0 con 0x7f56ec0b2010 2023-10-16T09:44:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56ea59c700 1 --2- 172.21.15.181:0/2584166147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0b2010 0x7f56ec0cc6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f56ec071a30 tx=0x7f56e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5fc032ff0b07d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56e959a700 1 -- 172.21.15.181:0/2584166147 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0011830 con 0x7f56ec0b2010 2023-10-16T09:44:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56e959a700 1 -- 172.21.15.181:0/2584166147 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0011990 con 0x7f56ec0b2010 2023-10-16T09:44:34.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56e959a700 1 -- 172.21.15.181:0/2584166147 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0011d20 con 0x7f56ec0b2010 2023-10-16T09:44:34.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56f085b700 1 -- 172.21.15.181:0/2584166147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0b2010 msgr2=0x7f56ec0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56f085b700 1 --2- 172.21.15.181:0/2584166147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0b2010 0x7f56ec0cc6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f56ec071a30 tx=0x7f56e0009b20 comp rx=0 tx=0).stop 2023-10-16T09:44:34.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56f085b700 1 -- 172.21.15.181:0/2584166147 shutdown_connections 2023-10-16T09:44:34.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56f085b700 1 --2- 172.21.15.181:0/2584166147 >> v2:172.21.15.190:3300/0 conn(0x7f56ec0ce680 0x7f56ec0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56f085b700 1 --2- 172.21.15.181:0/2584166147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0b2010 0x7f56ec0cc6e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.354+0000 7f56f085b700 1 -- 172.21.15.181:0/2584166147 >> 172.21.15.181:0/2584166147 conn(0x7f56ec0cd010 msgr2=0x7f56ec0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:34.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.355+0000 7f56f085b700 1 -- 172.21.15.181:0/2584166147 shutdown_connections 2023-10-16T09:44:34.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.355+0000 7f56f085b700 1 -- 172.21.15.181:0/2584166147 wait complete. 2023-10-16T09:44:34.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.356+0000 7f56f085b700 1 Processor -- start 2023-10-16T09:44:34.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.356+0000 7f56f085b700 1 -- start start 2023-10-16T09:44:34.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec071be0 0x7f56ec1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ec0b2010 0x7f56ec0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0ce680 0x7f56ec14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0c9960 con 0x7f56ec0ce680 2023-10-16T09:44:34.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1517a0 con 0x7f56ec071be0 2023-10-16T09:44:34.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec151a80 con 0x7f56ec0b2010 2023-10-16T09:44:34.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0ce680 0x7f56ec14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0ce680 0x7f56ec14fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37206/0 (socket says 172.21.15.181:37206) 2023-10-16T09:44:34.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 -- 172.21.15.181:0/138706025 learned_addr learned my addr 172.21.15.181:0/138706025 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:34.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56e9d9b700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ec0b2010 0x7f56ec0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ea59c700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec071be0 0x7f56ec1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 -- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ec0b2010 msgr2=0x7f56ec0c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ec0b2010 0x7f56ec0c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 -- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec071be0 msgr2=0x7f56ec1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec071be0 0x7f56ec1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 -- 172.21.15.181:0/138706025 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094a0 con 0x7f56ec0ce680 2023-10-16T09:44:34.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ea59c700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec071be0 0x7f56ec1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:34.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56ead9d700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0ce680 0x7f56ec14fdf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56e400ce80 tx=0x7f56e400e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56daffd700 1 -- 172.21.15.181:0/138706025 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4011070 con 0x7f56ec0ce680 2023-10-16T09:44:34.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56daffd700 1 -- 172.21.15.181:0/138706025 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e400c2e0 con 0x7f56ec0ce680 2023-10-16T09:44:34.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec151d60 con 0x7f56ec0ce680 2023-10-16T09:44:34.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56daffd700 1 -- 172.21.15.181:0/138706025 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4003c40 con 0x7f56ec0ce680 2023-10-16T09:44:34.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.357+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec150330 con 0x7f56ec0ce680 2023-10-16T09:44:34.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.358+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec0b6510 con 0x7f56ec0ce680 2023-10-16T09:44:34.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.359+0000 7f56daffd700 1 -- 172.21.15.181:0/138706025 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f56e400e980 con 0x7f56ec0ce680 2023-10-16T09:44:34.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.359+0000 7f56daffd700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f56cc06af00 0x7f56cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.359+0000 7f56daffd700 1 -- 172.21.15.181:0/138706025 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56e408b350 con 0x7f56ec0ce680 2023-10-16T09:44:34.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.359+0000 7f56ea59c700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f56cc06af00 0x7f56cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.360+0000 7f56ea59c700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f56cc06af00 0x7f56cc06d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56e00090b0 tx=0x7f56e001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.364+0000 7f56daffd700 1 -- 172.21.15.181:0/138706025 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f56e4056490 con 0x7f56ec0ce680 2023-10-16T09:44:34.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.554+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56ec0b4590 con 0x7f56ec0ce680 2023-10-16T09:44:34.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.555+0000 7f56daffd700 1 -- 172.21.15.181:0/138706025 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56e4059ab0 con 0x7f56ec0ce680 2023-10-16T09:44:34.557 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:34.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f56cc06af00 msgr2=0x7f56cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f56cc06af00 0x7f56cc06d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56e00090b0 tx=0x7f56e001c040 comp rx=0 tx=0).stop 2023-10-16T09:44:34.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0ce680 msgr2=0x7f56ec14fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0ce680 0x7f56ec14fdf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f56e400ce80 tx=0x7f56e400e330 comp rx=0 tx=0).stop 2023-10-16T09:44:34.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 shutdown_connections 2023-10-16T09:44:34.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f56cc06af00 0x7f56cc06d3c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ec0ce680 0x7f56ec14fdf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec071be0 0x7f56ec1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 --2- 172.21.15.181:0/138706025 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ec0b2010 0x7f56ec0c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 >> 172.21.15.181:0/138706025 conn(0x7f56ec0cd010 msgr2=0x7f56ec0b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:34.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 shutdown_connections 2023-10-16T09:44:34.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.557+0000 7f56f085b700 1 -- 172.21.15.181:0/138706025 wait complete. 2023-10-16T09:44:34.571 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:34.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.692+0000 7f660e303700 1 Processor -- start 2023-10-16T09:44:34.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.692+0000 7f660e303700 1 -- start start 2023-10-16T09:44:34.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660e303700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f660815da70 0x7f66080ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660e303700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66080ce6a0 0x7f660806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660e303700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6608072760 0x7f660806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660e303700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66080cef00 con 0x7f660815da70 2023-10-16T09:44:34.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660e303700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66080cec20 con 0x7f6608072760 2023-10-16T09:44:34.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660e303700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6608071c80 con 0x7f66080ce6a0 2023-10-16T09:44:34.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f660815da70 0x7f66080ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f660815da70 0x7f66080ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37212/0 (socket says 172.21.15.181:37212) 2023-10-16T09:44:34.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 -- 172.21.15.181:0/4102841306 learned_addr learned my addr 172.21.15.181:0/4102841306 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:34.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f66077fe700 1 --2- 172.21.15.181:0/4102841306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66080ce6a0 0x7f660806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660c8a0700 1 --2- 172.21.15.181:0/4102841306 >> v2:172.21.15.190:3300/0 conn(0x7f6608072760 0x7f660806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 -- 172.21.15.181:0/4102841306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66080ce6a0 msgr2=0x7f660806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:34.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 --2- 172.21.15.181:0/4102841306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66080ce6a0 0x7f660806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 -- 172.21.15.181:0/4102841306 >> v2:172.21.15.190:3300/0 conn(0x7f6608072760 msgr2=0x7f660806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:34.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 --2- 172.21.15.181:0/4102841306 >> v2:172.21.15.190:3300/0 conn(0x7f6608072760 0x7f660806ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f6607fff700 1 -- 172.21.15.181:0/4102841306 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608071f60 con 0x7f660815da70 2023-10-16T09:44:34.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f66077fe700 1 --2- 172.21.15.181:0/4102841306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66080ce6a0 0x7f660806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:34.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.693+0000 7f660c8a0700 1 --2- 172.21.15.181:0/4102841306 >> v2:172.21.15.190:3300/0 conn(0x7f6608072760 0x7f660806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:34.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f6607fff700 1 --2- 172.21.15.181:0/4102841306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080ce160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6608161530 tx=0x7f65fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70eadf21d414a1e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f6606ffd700 1 -- 172.21.15.181:0/4102841306 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00b870 con 0x7f660815da70 2023-10-16T09:44:34.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f6606ffd700 1 -- 172.21.15.181:0/4102841306 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00b9d0 con 0x7f660815da70 2023-10-16T09:44:34.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f6606ffd700 1 -- 172.21.15.181:0/4102841306 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00bd60 con 0x7f660815da70 2023-10-16T09:44:34.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 -- 172.21.15.181:0/4102841306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 msgr2=0x7f66080ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 --2- 172.21.15.181:0/4102841306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080ce160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6608161530 tx=0x7f65fc009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 -- 172.21.15.181:0/4102841306 shutdown_connections 2023-10-16T09:44:34.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 --2- 172.21.15.181:0/4102841306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080ce160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 --2- 172.21.15.181:0/4102841306 >> v2:172.21.15.190:3300/0 conn(0x7f6608072760 0x7f660806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 --2- 172.21.15.181:0/4102841306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66080ce6a0 0x7f660806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 -- 172.21.15.181:0/4102841306 >> 172.21.15.181:0/4102841306 conn(0x7f66080cd010 msgr2=0x7f66080cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:34.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 -- 172.21.15.181:0/4102841306 shutdown_connections 2023-10-16T09:44:34.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.694+0000 7f660e303700 1 -- 172.21.15.181:0/4102841306 wait complete. 2023-10-16T09:44:34.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.695+0000 7f660e303700 1 Processor -- start 2023-10-16T09:44:34.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660e303700 1 -- start start 2023-10-16T09:44:34.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660e303700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6608072760 0x7f66080b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660e303700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66080ce6a0 0x7f66081f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660e303700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660e303700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66081f4460 con 0x7f660815da70 2023-10-16T09:44:34.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660e303700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66081f45c0 con 0x7f66080ce6a0 2023-10-16T09:44:34.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660e303700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66081f3ea0 con 0x7f6608072760 2023-10-16T09:44:34.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660c8a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660c8a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37218/0 (socket says 172.21.15.181:37218) 2023-10-16T09:44:34.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.696+0000 7f660c8a0700 1 -- 172.21.15.181:0/3486573648 learned_addr learned my addr 172.21.15.181:0/3486573648 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:34.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f6607fff700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6608072760 0x7f66080b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f66077fe700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66080ce6a0 0x7f66081f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660c8a0700 1 -- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6608072760 msgr2=0x7f66080b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660c8a0700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6608072760 0x7f66080b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660c8a0700 1 -- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66080ce6a0 msgr2=0x7f66081f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660c8a0700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66080ce6a0 0x7f66081f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660c8a0700 1 -- 172.21.15.181:0/3486573648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc009550 con 0x7f660815da70 2023-10-16T09:44:34.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660c8a0700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080cac70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f660000cd10 tx=0x7f660000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f6604ff9700 1 -- 172.21.15.181:0/3486573648 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000ce60 con 0x7f660815da70 2023-10-16T09:44:34.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f6604ff9700 1 -- 172.21.15.181:0/3486573648 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660000f480 con 0x7f660815da70 2023-10-16T09:44:34.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f6604ff9700 1 -- 172.21.15.181:0/3486573648 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000f810 con 0x7f660815da70 2023-10-16T09:44:34.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66081f4180 con 0x7f660815da70 2023-10-16T09:44:34.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.697+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66081f4a90 con 0x7f660815da70 2023-10-16T09:44:34.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.698+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66081f4c80 con 0x7f660815da70 2023-10-16T09:44:34.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.698+0000 7f6604ff9700 1 -- 172.21.15.181:0/3486573648 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f660000b040 con 0x7f660815da70 2023-10-16T09:44:34.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.699+0000 7f6604ff9700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f65f406af00 0x7f65f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:34.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.699+0000 7f6604ff9700 1 -- 172.21.15.181:0/3486573648 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6600089bb0 con 0x7f660815da70 2023-10-16T09:44:34.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.699+0000 7f6607fff700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f65f406af00 0x7f65f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:34.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.699+0000 7f6607fff700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f65f406af00 0x7f65f406d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65fc000c00 tx=0x7f65fc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:34.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.703+0000 7f6604ff9700 1 -- 172.21.15.181:0/3486573648 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6600054cf0 con 0x7f660815da70 2023-10-16T09:44:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.895+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f660806add0 con 0x7f660815da70 2023-10-16T09:44:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.895+0000 7f6604ff9700 1 -- 172.21.15.181:0/3486573648 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f660001d020 con 0x7f660815da70 2023-10-16T09:44:34.898 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f65f406af00 msgr2=0x7f65f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f65f406af00 0x7f65f406d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65fc000c00 tx=0x7f65fc008030 comp rx=0 tx=0).stop 2023-10-16T09:44:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 msgr2=0x7f66080cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080cac70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f660000cd10 tx=0x7f660000a560 comp rx=0 tx=0).stop 2023-10-16T09:44:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 shutdown_connections 2023-10-16T09:44:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f65f406af00 0x7f65f406d3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6608072760 0x7f66080b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f660815da70 0x7f66080cac70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 --2- 172.21.15.181:0/3486573648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66080ce6a0 0x7f66081f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.897+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 >> 172.21.15.181:0/3486573648 conn(0x7f66080cd010 msgr2=0x7f660815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.898+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 shutdown_connections 2023-10-16T09:44:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:34.898+0000 7f660e303700 1 -- 172.21.15.181:0/3486573648 wait complete. 2023-10-16T09:44:34.912 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:44:35.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.032+0000 7f6b04192700 1 Processor -- start 2023-10-16T09:44:35.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.032+0000 7f6b04192700 1 -- start start 2023-10-16T09:44:35.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b04192700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6afc15df60 0x7f6afc0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b04192700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6afc06a8a0 0x7f6afc06ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b04192700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6afc0cddf0 con 0x7f6afc0d0fc0 2023-10-16T09:44:35.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b04192700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6afc06b290 con 0x7f6afc06a8a0 2023-10-16T09:44:35.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b04192700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6afc0af9c0 con 0x7f6afc15df60 2023-10-16T09:44:35.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b0272f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6afc06a8a0 0x7f6afc06ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b0172d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6afc0d0fc0 0x7f6afc0cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48300/0 (socket says 172.21.15.181:48300) 2023-10-16T09:44:35.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b0172d700 1 -- 172.21.15.181:0/1732447687 learned_addr learned my addr 172.21.15.181:0/1732447687 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:35.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b0272f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6afc06a8a0 0x7f6afc06ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37224/0 (socket says 172.21.15.181:37224) 2023-10-16T09:44:35.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.033+0000 7f6b01f2e700 1 --2- 172.21.15.181:0/1732447687 >> v2:172.21.15.190:3300/0 conn(0x7f6afc15df60 0x7f6afc0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b0272f700 1 -- 172.21.15.181:0/1732447687 >> v2:172.21.15.190:3300/0 conn(0x7f6afc15df60 msgr2=0x7f6afc0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:35.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b0272f700 1 --2- 172.21.15.181:0/1732447687 >> v2:172.21.15.190:3300/0 conn(0x7f6afc15df60 0x7f6afc0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b0272f700 1 -- 172.21.15.181:0/1732447687 >> v1:172.21.15.181:6789/0 conn(0x7f6afc0d0fc0 legacy=0x7f6afc0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b0272f700 1 -- 172.21.15.181:0/1732447687 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc0ccf50 con 0x7f6afc06a8a0 2023-10-16T09:44:35.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b00f2c700 1 -- 172.21.15.181:0/1732447687 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359027249 0 0) 0x7f6afc0cddf0 con 0x7f6afc0d0fc0 2023-10-16T09:44:35.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b01f2e700 1 --2- 172.21.15.181:0/1732447687 >> v2:172.21.15.190:3300/0 conn(0x7f6afc15df60 0x7f6afc0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:35.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b0272f700 1 --2- 172.21.15.181:0/1732447687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc06ad50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6afc072040 tx=0x7f6af400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152ecf5d13a4df43 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b00f2c700 1 -- 172.21.15.181:0/1732447687 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400e070 con 0x7f6afc06a8a0 2023-10-16T09:44:35.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b00f2c700 1 -- 172.21.15.181:0/1732447687 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af400bbd0 con 0x7f6afc06a8a0 2023-10-16T09:44:35.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b00f2c700 1 -- 172.21.15.181:0/1732447687 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4012430 con 0x7f6afc06a8a0 2023-10-16T09:44:35.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b04192700 1 -- 172.21.15.181:0/1732447687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 msgr2=0x7f6afc06ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b04192700 1 --2- 172.21.15.181:0/1732447687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc06ad50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6afc072040 tx=0x7f6af400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:35.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.034+0000 7f6b04192700 1 -- 172.21.15.181:0/1732447687 shutdown_connections 2023-10-16T09:44:35.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.035+0000 7f6b04192700 1 --2- 172.21.15.181:0/1732447687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc06ad50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.035+0000 7f6b04192700 1 --2- 172.21.15.181:0/1732447687 >> v2:172.21.15.190:3300/0 conn(0x7f6afc15df60 0x7f6afc0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.035+0000 7f6b04192700 1 -- 172.21.15.181:0/1732447687 >> 172.21.15.181:0/1732447687 conn(0x7f6afc0d04a0 msgr2=0x7f6afc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:35.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.035+0000 7f6b04192700 1 -- 172.21.15.181:0/1732447687 shutdown_connections 2023-10-16T09:44:35.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.035+0000 7f6b04192700 1 -- 172.21.15.181:0/1732447687 wait complete. 2023-10-16T09:44:35.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 Processor -- start 2023-10-16T09:44:35.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 -- start start 2023-10-16T09:44:35.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc0b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6afc0d0fc0 0x7f6afc0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6afc15df60 0x7f6afc0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0b3920 con 0x7f6afc06a8a0 2023-10-16T09:44:35.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc0b32f0 con 0x7f6afc0d0fc0 2023-10-16T09:44:35.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.036+0000 7f6b04192700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc154d20 con 0x7f6afc15df60 2023-10-16T09:44:35.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc0b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc0b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37228/0 (socket says 172.21.15.181:37228) 2023-10-16T09:44:35.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 -- 172.21.15.181:0/2979838110 learned_addr learned my addr 172.21.15.181:0/2979838110 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:35.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b0272f700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6afc15df60 0x7f6afc0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b0172d700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6afc0d0fc0 0x7f6afc0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 -- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6afc15df60 msgr2=0x7f6afc0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6afc15df60 0x7f6afc0cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 -- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6afc0d0fc0 msgr2=0x7f6afc0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6afc0d0fc0 0x7f6afc0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 -- 172.21.15.181:0/2979838110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4009a70 con 0x7f6afc06a8a0 2023-10-16T09:44:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b01f2e700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc0b5480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e840 tx=0x7f6aec00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6af27fc700 1 -- 172.21.15.181:0/2979838110 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec005c20 con 0x7f6afc06a8a0 2023-10-16T09:44:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6af27fc700 1 -- 172.21.15.181:0/2979838110 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec005d80 con 0x7f6afc06a8a0 2023-10-16T09:44:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc0c9f60 con 0x7f6afc06a8a0 2023-10-16T09:44:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6af27fc700 1 -- 172.21.15.181:0/2979838110 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec006250 con 0x7f6afc06a8a0 2023-10-16T09:44:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.037+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc0b3540 con 0x7f6afc06a8a0 2023-10-16T09:44:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.038+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc0ca480 con 0x7f6afc06a8a0 2023-10-16T09:44:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.039+0000 7f6af27fc700 1 -- 172.21.15.181:0/2979838110 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6aec0063b0 con 0x7f6afc06a8a0 2023-10-16T09:44:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.039+0000 7f6af27fc700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ae006af00 0x7f6ae006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.039+0000 7f6af27fc700 1 -- 172.21.15.181:0/2979838110 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6aec089020 con 0x7f6afc06a8a0 2023-10-16T09:44:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.039+0000 7f6b0172d700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ae006af00 0x7f6ae006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.039+0000 7f6b0172d700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ae006af00 0x7f6ae006d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000f60 tx=0x7f6af8007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.043+0000 7f6af27fc700 1 -- 172.21.15.181:0/2979838110 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6aec0540e0 con 0x7f6afc06a8a0 2023-10-16T09:44:35.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.249+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6afc067d20 con 0x7f6afc06a8a0 2023-10-16T09:44:35.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.250+0000 7f6af27fc700 1 -- 172.21.15.181:0/2979838110 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6aec00c450 con 0x7f6afc06a8a0 2023-10-16T09:44:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ae006af00 msgr2=0x7f6ae006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ae006af00 0x7f6ae006d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000f60 tx=0x7f6af8007be0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 msgr2=0x7f6afc0b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc0b5480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e840 tx=0x7f6aec00ec00 comp rx=0 tx=0).stop 2023-10-16T09:44:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 shutdown_connections 2023-10-16T09:44:35.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6ae006af00 0x7f6ae006d3c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6afc15df60 0x7f6afc0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6afc06a8a0 0x7f6afc0b5480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 --2- 172.21.15.181:0/2979838110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6afc0d0fc0 0x7f6afc0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 >> 172.21.15.181:0/2979838110 conn(0x7f6afc0d04a0 msgr2=0x7f6afc0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:35.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 shutdown_connections 2023-10-16T09:44:35.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.252+0000 7f6b04192700 1 -- 172.21.15.181:0/2979838110 wait complete. 2023-10-16T09:44:35.257 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2023-10-16T09:44:35.266 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T09:44:35.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:35.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-16T09:44:35.301 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:44:35.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:35.302 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T09:44:35.368 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:35.369 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:35.390 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:35.390 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:35.450 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:35.451 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:35.511 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:35.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.630+0000 7fe4dca58700 1 Processor -- start 2023-10-16T09:44:35.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.630+0000 7fe4dca58700 1 -- start start 2023-10-16T09:44:35.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.630+0000 7fe4dca58700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80d2390 0x7fe4d80cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4dca58700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d80cc470 con 0x7fe4d80ce7f0 2023-10-16T09:44:35.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4dca58700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d806ca30 con 0x7fe4d80d0180 2023-10-16T09:44:35.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4dca58700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80d0610 con 0x7fe4d80d2390 2023-10-16T09:44:35.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe4d80ce7f0 0x7fe4d806c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48312/0 (socket says 172.21.15.181:48312) 2023-10-16T09:44:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d659c700 1 -- 172.21.15.181:0/1130830530 learned_addr learned my addr 172.21.15.181:0/1130830530 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d5d9b700 1 --2- 172.21.15.181:0/1130830530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80d2390 0x7fe4d80cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236989593 0 0) 0x7fe4d80cc470 con 0x7fe4d80ce7f0 2023-10-16T09:44:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4c00035b0 con 0x7fe4d80ce7f0 2023-10-16T09:44:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360256061 0 0) 0x7fe4d806ca30 con 0x7fe4d80d0180 2023-10-16T09:44:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d80cc470 con 0x7fe4d80d0180 2023-10-16T09:44:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3949735804 0 0) 0x7fe4c00035b0 con 0x7fe4d80ce7f0 2023-10-16T09:44:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d5d9b700 1 -- 172.21.15.181:0/1130830530 >> v1:172.21.15.190:6789/0 conn(0x7fe4d80d0180 legacy=0x7fe4d80cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d5d9b700 1 -- 172.21.15.181:0/1130830530 >> v1:172.21.15.181:6789/0 conn(0x7fe4d80ce7f0 legacy=0x7fe4d806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d5d9b700 1 -- 172.21.15.181:0/1130830530 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d806cd10 con 0x7fe4d80d2390 2023-10-16T09:44:35.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.631+0000 7fe4d5d9b700 1 --2- 172.21.15.181:0/1130830530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80d2390 0x7fe4d80cfc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009870 tx=0x7fe4c8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a21059b536c4908 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0017f0 con 0x7fe4d80d2390 2023-10-16T09:44:35.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8004030 con 0x7fe4d80d2390 2023-10-16T09:44:35.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4d559a700 1 -- 172.21.15.181:0/1130830530 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80043a0 con 0x7fe4d80d2390 2023-10-16T09:44:35.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4dca58700 1 -- 172.21.15.181:0/1130830530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80d2390 msgr2=0x7fe4d80cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1130830530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80d2390 0x7fe4d80cfc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009870 tx=0x7fe4c8009b80 comp rx=0 tx=0).stop 2023-10-16T09:44:35.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4dca58700 1 -- 172.21.15.181:0/1130830530 shutdown_connections 2023-10-16T09:44:35.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1130830530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80d2390 0x7fe4d80cfc40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4dca58700 1 -- 172.21.15.181:0/1130830530 >> 172.21.15.181:0/1130830530 conn(0x7fe4d809a9a0 msgr2=0x7fe4d80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4dca58700 1 -- 172.21.15.181:0/1130830530 shutdown_connections 2023-10-16T09:44:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.632+0000 7fe4dca58700 1 -- 172.21.15.181:0/1130830530 wait complete. 2023-10-16T09:44:35.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 Processor -- start 2023-10-16T09:44:35.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 -- start start 2023-10-16T09:44:35.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80ce7f0 0x7fe4d80ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4d80d0180 0x7fe4d815b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d80d2390 0x7fe4d815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81f4450 con 0x7fe4d80d0180 2023-10-16T09:44:35.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81f3fc0 con 0x7fe4d80d2390 2023-10-16T09:44:35.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4dca58700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8158e00 con 0x7fe4d80ce7f0 2023-10-16T09:44:35.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4d5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4d80d0180 0x7fe4d815b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4d5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4d80d0180 0x7fe4d815b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37242/0 (socket says 172.21.15.181:37242) 2023-10-16T09:44:35.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.634+0000 7fe4d5d9b700 1 -- 172.21.15.181:0/1410835863 learned_addr learned my addr 172.21.15.181:0/1410835863 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:35.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d659c700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80ce7f0 0x7fe4d80ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d6d9d700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d80d2390 0x7fe4d815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d5d9b700 1 -- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80ce7f0 msgr2=0x7fe4d80ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d5d9b700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80ce7f0 0x7fe4d80ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d5d9b700 1 -- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d80d2390 msgr2=0x7fe4d815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d5d9b700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d80d2390 0x7fe4d815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d5d9b700 1 -- 172.21.15.181:0/1410835863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80094a0 con 0x7fe4d80d0180 2023-10-16T09:44:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4d5d9b700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4d80d0180 0x7fe4d815b980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80097c0 tx=0x7fe4c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4c6ffd700 1 -- 172.21.15.181:0/1410835863 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8003e60 con 0x7fe4d80d0180 2023-10-16T09:44:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4c6ffd700 1 -- 172.21.15.181:0/1410835863 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c801f070 con 0x7fe4d80d0180 2023-10-16T09:44:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4c6ffd700 1 -- 172.21.15.181:0/1410835863 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8011870 con 0x7fe4d80d0180 2023-10-16T09:44:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d81f3690 con 0x7fe4d80d0180 2023-10-16T09:44:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.635+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d81f3c50 con 0x7fe4d80d0180 2023-10-16T09:44:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.636+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d80cebd0 con 0x7fe4d80d0180 2023-10-16T09:44:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.636+0000 7fe4c6ffd700 1 -- 172.21.15.181:0/1410835863 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe4c80119d0 con 0x7fe4d80d0180 2023-10-16T09:44:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.637+0000 7fe4c6ffd700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe4b806af00 0x7fe4b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.637+0000 7fe4c6ffd700 1 -- 172.21.15.181:0/1410835863 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4c8021070 con 0x7fe4d80d0180 2023-10-16T09:44:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.637+0000 7fe4d659c700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe4b806af00 0x7fe4b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.637+0000 7fe4d659c700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe4b806af00 0x7fe4b806d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001900 tx=0x7fe4cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.641+0000 7fe4c6ffd700 1 -- 172.21.15.181:0/1410835863 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe4c8055ea0 con 0x7fe4d80d0180 2023-10-16T09:44:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.833+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4d8067d20 con 0x7fe4d80d0180 2023-10-16T09:44:35.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.834+0000 7fe4c6ffd700 1 -- 172.21.15.181:0/1410835863 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fe4c802b020 con 0x7fe4d80d0180 2023-10-16T09:44:35.837 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:35.837 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:35.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe4b806af00 msgr2=0x7fe4b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe4b806af00 0x7fe4b806d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001900 tx=0x7fe4cc009040 comp rx=0 tx=0).stop 2023-10-16T09:44:35.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4d80d0180 msgr2=0x7fe4d815b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4d80d0180 0x7fe4d815b980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80097c0 tx=0x7fe4c8004750 comp rx=0 tx=0).stop 2023-10-16T09:44:35.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 shutdown_connections 2023-10-16T09:44:35.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe4b806af00 0x7fe4b806d3c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4d80ce7f0 0x7fe4d80ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d80d2390 0x7fe4d815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 --2- 172.21.15.181:0/1410835863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4d80d0180 0x7fe4d815b980 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 >> 172.21.15.181:0/1410835863 conn(0x7fe4d809a9a0 msgr2=0x7fe4d8070360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:35.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 shutdown_connections 2023-10-16T09:44:35.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.837+0000 7fe4dca58700 1 -- 172.21.15.181:0/1410835863 wait complete. 2023-10-16T09:44:35.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:35.852 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:35.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.974+0000 7f50cd79f700 1 Processor -- start 2023-10-16T09:44:35.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.974+0000 7f50cd79f700 1 -- start start 2023-10-16T09:44:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50cd79f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50c815da70 0x7f50c80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50cd79f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50c80ce6a0 0x7f50c806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50cd79f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c8072760 0x7f50c806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50cd79f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80cef00 con 0x7f50c815da70 2023-10-16T09:44:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50cd79f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80cec20 con 0x7f50c80ce6a0 2023-10-16T09:44:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50cd79f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8071c80 con 0x7f50c8072760 2023-10-16T09:44:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50c815da70 0x7f50c80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50c815da70 0x7f50c80ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37248/0 (socket says 172.21.15.181:37248) 2023-10-16T09:44:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1620439699 learned_addr learned my addr 172.21.15.181:0/1620439699 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:35.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c77fe700 1 --2- 172.21.15.181:0/1620439699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c8072760 0x7f50c806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c67fc700 1 --2- 172.21.15.181:0/1620439699 >> v2:172.21.15.190:3300/0 conn(0x7f50c80ce6a0 0x7f50c806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1620439699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c8072760 msgr2=0x7f50c806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:35.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 --2- 172.21.15.181:0/1620439699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c8072760 0x7f50c806ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1620439699 >> v2:172.21.15.190:3300/0 conn(0x7f50c80ce6a0 msgr2=0x7f50c806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 --2- 172.21.15.181:0/1620439699 >> v2:172.21.15.190:3300/0 conn(0x7f50c80ce6a0 0x7f50c806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1620439699 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8071f60 con 0x7f50c815da70 2023-10-16T09:44:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.975+0000 7f50c67fc700 1 --2- 172.21.15.181:0/1620439699 >> v2:172.21.15.190:3300/0 conn(0x7f50c80ce6a0 0x7f50c806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50c6ffd700 1 --2- 172.21.15.181:0/1620439699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c815da70 0x7f50c80ce160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f50c806ef60 tx=0x7f50bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b12434d2bd5b64 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50c5ffb700 1 -- 172.21.15.181:0/1620439699 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00b870 con 0x7f50c815da70 2023-10-16T09:44:35.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50c5ffb700 1 -- 172.21.15.181:0/1620439699 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc00b9d0 con 0x7f50c815da70 2023-10-16T09:44:35.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50c5ffb700 1 -- 172.21.15.181:0/1620439699 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00bd60 con 0x7f50c815da70 2023-10-16T09:44:35.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 -- 172.21.15.181:0/1620439699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c815da70 msgr2=0x7f50c80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1620439699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c815da70 0x7f50c80ce160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f50c806ef60 tx=0x7f50bc009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 -- 172.21.15.181:0/1620439699 shutdown_connections 2023-10-16T09:44:35.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1620439699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c815da70 0x7f50c80ce160 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1620439699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c8072760 0x7f50c806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1620439699 >> v2:172.21.15.190:3300/0 conn(0x7f50c80ce6a0 0x7f50c806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 -- 172.21.15.181:0/1620439699 >> 172.21.15.181:0/1620439699 conn(0x7f50c80cd010 msgr2=0x7f50c80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:35.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 -- 172.21.15.181:0/1620439699 shutdown_connections 2023-10-16T09:44:35.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.976+0000 7f50cd79f700 1 -- 172.21.15.181:0/1620439699 wait complete. 2023-10-16T09:44:35.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 Processor -- start 2023-10-16T09:44:35.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 -- start start 2023-10-16T09:44:35.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c8072760 0x7f50c8159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c80ce6a0 0x7f50c80c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c815da70 0x7f50c80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c815a290 con 0x7f50c8072760 2023-10-16T09:44:35.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80ad590 con 0x7f50c815da70 2023-10-16T09:44:35.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50cd79f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80ad870 con 0x7f50c80ce6a0 2023-10-16T09:44:35.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50c6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c8072760 0x7f50c8159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50c6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c8072760 0x7f50c8159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37254/0 (socket says 172.21.15.181:37254) 2023-10-16T09:44:35.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1274964692 learned_addr learned my addr 172.21.15.181:0/1274964692 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:35.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50c67fc700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c80ce6a0 0x7f50c80c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.978+0000 7f50c77fe700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c815da70 0x7f50c80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c80ce6a0 msgr2=0x7f50c80c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50c6ffd700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c80ce6a0 0x7f50c80c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c815da70 msgr2=0x7f50c80c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:35.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50c6ffd700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c815da70 0x7f50c80c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50c6ffd700 1 -- 172.21.15.181:0/1274964692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc009550 con 0x7f50c8072760 2023-10-16T09:44:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50c6ffd700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c8072760 0x7f50c8159d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50bc000c00 tx=0x7f50bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50af7fe700 1 -- 172.21.15.181:0/1274964692 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004480 con 0x7f50c8072760 2023-10-16T09:44:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50af7fe700 1 -- 172.21.15.181:0/1274964692 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc0045e0 con 0x7f50c8072760 2023-10-16T09:44:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c80adb50 con 0x7f50c8072760 2023-10-16T09:44:35.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50af7fe700 1 -- 172.21.15.181:0/1274964692 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0038e0 con 0x7f50c8072760 2023-10-16T09:44:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.979+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c80ae110 con 0x7f50c8072760 2023-10-16T09:44:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.980+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c806f6c0 con 0x7f50c8072760 2023-10-16T09:44:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.980+0000 7f50af7fe700 1 -- 172.21.15.181:0/1274964692 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f50bc021070 con 0x7f50c8072760 2023-10-16T09:44:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.981+0000 7f50af7fe700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50b406af00 0x7f50b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.981+0000 7f50af7fe700 1 -- 172.21.15.181:0/1274964692 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50bc08aaa0 con 0x7f50c8072760 2023-10-16T09:44:35.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.981+0000 7f50c67fc700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50b406af00 0x7f50b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:35.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.981+0000 7f50c67fc700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50b406af00 0x7f50b406d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50c80c69e0 tx=0x7f50b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:35.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:35.985+0000 7f50af7fe700 1 -- 172.21.15.181:0/1274964692 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50bc055b40 con 0x7f50c8072760 2023-10-16T09:44:36.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.177+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50c8067ca0 con 0x7f50c8072760 2023-10-16T09:44:36.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.177+0000 7f50af7fe700 1 -- 172.21.15.181:0/1274964692 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50bc015020 con 0x7f50c8072760 2023-10-16T09:44:36.179 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:36.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50b406af00 msgr2=0x7f50b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50b406af00 0x7f50b406d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50c80c69e0 tx=0x7f50b8008040 comp rx=0 tx=0).stop 2023-10-16T09:44:36.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c8072760 msgr2=0x7f50c8159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c8072760 0x7f50c8159d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f50bc000c00 tx=0x7f50bc004750 comp rx=0 tx=0).stop 2023-10-16T09:44:36.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 shutdown_connections 2023-10-16T09:44:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50b406af00 0x7f50b406d3c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c815da70 0x7f50c80c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50c8072760 0x7f50c8159d50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 --2- 172.21.15.181:0/1274964692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50c80ce6a0 0x7f50c80c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 >> 172.21.15.181:0/1274964692 conn(0x7f50c80cd010 msgr2=0x7f50c815ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:36.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 shutdown_connections 2023-10-16T09:44:36.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.179+0000 7f50cd79f700 1 -- 172.21.15.181:0/1274964692 wait complete. 2023-10-16T09:44:36.193 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:36.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.314+0000 7f19919c3700 1 Processor -- start 2023-10-16T09:44:36.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.314+0000 7f19919c3700 1 -- start start 2023-10-16T09:44:36.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f19919c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0d01d0 0x7f198c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f19919c3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198c15ea40 0x7f198c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f19919c3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0cf580 con 0x7f198c161530 2023-10-16T09:44:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f19919c3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c0d4290 con 0x7f198c15ea40 2023-10-16T09:44:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f19919c3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0d0af0 con 0x7f198c0d01d0 2023-10-16T09:44:36.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198c15ea40 0x7f198c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f198c161530 0x7f198c1619c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48342/0 (socket says 172.21.15.181:48342) 2023-10-16T09:44:36.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198b7fe700 1 -- 172.21.15.181:0/3352238349 learned_addr learned my addr 172.21.15.181:0/3352238349 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:36.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198a7fc700 1 -- 172.21.15.181:0/3352238349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0d01d0 msgr2=0x7f198c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:36.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198affd700 1 --2- 172.21.15.181:0/3352238349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0d01d0 0x7f198c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f1989ffb700 1 -- 172.21.15.181:0/3352238349 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721016674 0 0) 0x7f198c0cf580 con 0x7f198c161530 2023-10-16T09:44:36.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198a7fc700 1 --2- 172.21.15.181:0/3352238349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0d01d0 0x7f198c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198a7fc700 1 -- 172.21.15.181:0/3352238349 >> v1:172.21.15.181:6789/0 conn(0x7f198c161530 legacy=0x7f198c1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198a7fc700 1 -- 172.21.15.181:0/3352238349 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0ce970 con 0x7f198c15ea40 2023-10-16T09:44:36.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198affd700 1 --2- 172.21.15.181:0/3352238349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0d01d0 0x7f198c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:36.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f198a7fc700 1 --2- 172.21.15.181:0/3352238349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c15ea40 0x7f198c0d05b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1980009840 tx=0x7f1980009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615670fc5ecf48ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f1989ffb700 1 -- 172.21.15.181:0/3352238349 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980011830 con 0x7f198c15ea40 2023-10-16T09:44:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f1989ffb700 1 -- 172.21.15.181:0/3352238349 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980011990 con 0x7f198c15ea40 2023-10-16T09:44:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.315+0000 7f1989ffb700 1 -- 172.21.15.181:0/3352238349 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980011d20 con 0x7f198c15ea40 2023-10-16T09:44:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 -- 172.21.15.181:0/3352238349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c15ea40 msgr2=0x7f198c0d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 --2- 172.21.15.181:0/3352238349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c15ea40 0x7f198c0d05b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1980009840 tx=0x7f1980009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 -- 172.21.15.181:0/3352238349 shutdown_connections 2023-10-16T09:44:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 --2- 172.21.15.181:0/3352238349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0d01d0 0x7f198c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 --2- 172.21.15.181:0/3352238349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c15ea40 0x7f198c0d05b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 -- 172.21.15.181:0/3352238349 >> 172.21.15.181:0/3352238349 conn(0x7f198c09cb60 msgr2=0x7f198c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:36.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 -- 172.21.15.181:0/3352238349 shutdown_connections 2023-10-16T09:44:36.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.316+0000 7f19919c3700 1 -- 172.21.15.181:0/3352238349 wait complete. 2023-10-16T09:44:36.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.317+0000 7f19919c3700 1 Processor -- start 2023-10-16T09:44:36.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.317+0000 7f19919c3700 1 -- start start 2023-10-16T09:44:36.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f19919c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c0d01d0 0x7f198c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f19919c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c15ea40 0x7f198c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f19919c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c161530 0x7f198c15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f19919c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0c9320 con 0x7f198c0d01d0 2023-10-16T09:44:36.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f19919c3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0cc500 con 0x7f198c161530 2023-10-16T09:44:36.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f19919c3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0cc7e0 con 0x7f198c15ea40 2023-10-16T09:44:36.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c0d01d0 0x7f198c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c0d01d0 0x7f198c0cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37268/0 (socket says 172.21.15.181:37268) 2023-10-16T09:44:36.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 -- 172.21.15.181:0/175499424 learned_addr learned my addr 172.21.15.181:0/175499424 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:36.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198a7fc700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c15ea40 0x7f198c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198b7fe700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c161530 0x7f198c15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 -- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c15ea40 msgr2=0x7f198c0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c15ea40 0x7f198c0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 -- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c161530 msgr2=0x7f198c15aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c161530 0x7f198c15aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198affd700 1 -- 172.21.15.181:0/175499424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19800094a0 con 0x7f198c0d01d0 2023-10-16T09:44:36.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198a7fc700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c15ea40 0x7f198c0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:36.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.318+0000 7f198b7fe700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c161530 0x7f198c15aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:36.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.319+0000 7f198affd700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c0d01d0 0x7f198c0cbe70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c7f0 tx=0x7f197c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:36.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.319+0000 7f1973fff700 1 -- 172.21.15.181:0/175499424 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c011800 con 0x7f198c0d01d0 2023-10-16T09:44:36.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.319+0000 7f1973fff700 1 -- 172.21.15.181:0/175499424 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c011960 con 0x7f198c0d01d0 2023-10-16T09:44:36.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.319+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c0ccac0 con 0x7f198c0d01d0 2023-10-16T09:44:36.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.319+0000 7f1973fff700 1 -- 172.21.15.181:0/175499424 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c011d10 con 0x7f198c0d01d0 2023-10-16T09:44:36.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.319+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c15af50 con 0x7f198c0d01d0 2023-10-16T09:44:36.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.320+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c15b0e0 con 0x7f198c0d01d0 2023-10-16T09:44:36.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.320+0000 7f1973fff700 1 -- 172.21.15.181:0/175499424 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f197c018360 con 0x7f198c0d01d0 2023-10-16T09:44:36.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.320+0000 7f1973fff700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f197806af00 0x7f197806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.321+0000 7f1973fff700 1 -- 172.21.15.181:0/175499424 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f197c089590 con 0x7f198c0d01d0 2023-10-16T09:44:36.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.321+0000 7f198a7fc700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f197806af00 0x7f197806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.321+0000 7f198a7fc700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f197806af00 0x7f197806d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f19800090b0 tx=0x7f198001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:36.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.325+0000 7f1973fff700 1 -- 172.21.15.181:0/175499424 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f197c054640 con 0x7f198c0d01d0 2023-10-16T09:44:36.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.517+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f198c0cd0b0 con 0x7f198c0d01d0 2023-10-16T09:44:36.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.517+0000 7f1973fff700 1 -- 172.21.15.181:0/175499424 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f197c057c60 con 0x7f198c0d01d0 2023-10-16T09:44:36.520 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:36.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f197806af00 msgr2=0x7f197806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f197806af00 0x7f197806d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f19800090b0 tx=0x7f198001c040 comp rx=0 tx=0).stop 2023-10-16T09:44:36.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c0d01d0 msgr2=0x7f198c0cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c0d01d0 0x7f198c0cbe70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c7f0 tx=0x7f197c00cbb0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 shutdown_connections 2023-10-16T09:44:36.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f197806af00 0x7f197806d3c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c161530 0x7f198c15aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c0d01d0 0x7f198c0cbe70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 --2- 172.21.15.181:0/175499424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c15ea40 0x7f198c0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 >> 172.21.15.181:0/175499424 conn(0x7f198c09cb60 msgr2=0x7f198c0cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:36.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 shutdown_connections 2023-10-16T09:44:36.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.520+0000 7f19919c3700 1 -- 172.21.15.181:0/175499424 wait complete. 2023-10-16T09:44:36.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:36.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.656+0000 7fa947267700 1 Processor -- start 2023-10-16T09:44:36.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.656+0000 7fa947267700 1 -- start start 2023-10-16T09:44:36.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa947267700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa94015dbd0 0x7fa9400d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa947267700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400d0610 0x7fa9400cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa947267700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94015dfb0 con 0x7fa9400ce2c0 2023-10-16T09:44:36.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa947267700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400ce740 con 0x7fa94015dbd0 2023-10-16T09:44:36.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa947267700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400cd3a0 con 0x7fa9400d0610 2023-10-16T09:44:36.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945003700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa94015dbd0 0x7fa9400d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945003700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa94015dbd0 0x7fa9400d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37276/0 (socket says 172.21.15.181:37276) 2023-10-16T09:44:36.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945804700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa9400ce2c0 0x7fa940161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:60530/0 (socket says 172.21.15.181:60530) 2023-10-16T09:44:36.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945003700 1 -- 172.21.15.181:0/2586827337 learned_addr learned my addr 172.21.15.181:0/2586827337 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:36.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa944802700 1 --2- 172.21.15.181:0/2586827337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400d0610 0x7fa9400cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945003700 1 -- 172.21.15.181:0/2586827337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400d0610 msgr2=0x7fa9400cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:36.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945003700 1 --2- 172.21.15.181:0/2586827337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400d0610 0x7fa9400cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa937fff700 1 -- 172.21.15.181:0/2586827337 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829398174 0 0) 0x7fa938001490 con 0x7fa9400ce2c0 2023-10-16T09:44:36.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945003700 1 -- 172.21.15.181:0/2586827337 >> v1:172.21.15.190:6789/0 conn(0x7fa9400ce2c0 legacy=0x7fa940161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.657+0000 7fa945003700 1 -- 172.21.15.181:0/2586827337 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400cd830 con 0x7fa94015dbd0 2023-10-16T09:44:36.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa945003700 1 --2- 172.21.15.181:0/2586827337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa9400d00d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0098f0 tx=0x7fa93c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df052fa39ee426a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:36.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa937fff700 1 -- 172.21.15.181:0/2586827337 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c011830 con 0x7fa94015dbd0 2023-10-16T09:44:36.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa937fff700 1 -- 172.21.15.181:0/2586827337 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c011990 con 0x7fa94015dbd0 2023-10-16T09:44:36.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa937fff700 1 -- 172.21.15.181:0/2586827337 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c011d20 con 0x7fa94015dbd0 2023-10-16T09:44:36.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa947267700 1 -- 172.21.15.181:0/2586827337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 msgr2=0x7fa9400d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa947267700 1 --2- 172.21.15.181:0/2586827337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa9400d00d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0098f0 tx=0x7fa93c009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:36.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa947267700 1 -- 172.21.15.181:0/2586827337 shutdown_connections 2023-10-16T09:44:36.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa947267700 1 --2- 172.21.15.181:0/2586827337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa9400d00d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa947267700 1 --2- 172.21.15.181:0/2586827337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400d0610 0x7fa9400cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa947267700 1 -- 172.21.15.181:0/2586827337 >> 172.21.15.181:0/2586827337 conn(0x7fa94009a930 msgr2=0x7fa9400d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:36.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.658+0000 7fa947267700 1 -- 172.21.15.181:0/2586827337 shutdown_connections 2023-10-16T09:44:36.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.659+0000 7fa947267700 1 -- 172.21.15.181:0/2586827337 wait complete. 2023-10-16T09:44:36.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 Processor -- start 2023-10-16T09:44:36.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 -- start start 2023-10-16T09:44:36.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400ce2c0 0x7fa9401f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9400d0610 0x7fa940160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa940151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9401613c0 con 0x7fa94015dbd0 2023-10-16T09:44:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94015fde0 con 0x7fa9400d0610 2023-10-16T09:44:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.660+0000 7fa947267700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9401600c0 con 0x7fa9400ce2c0 2023-10-16T09:44:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa940151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa940151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37284/0 (socket says 172.21.15.181:37284) 2023-10-16T09:44:36.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 -- 172.21.15.181:0/3037365470 learned_addr learned my addr 172.21.15.181:0/3037365470 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:36.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945003700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400ce2c0 0x7fa9401f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa944802700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9400d0610 0x7fa940160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 -- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400ce2c0 msgr2=0x7fa9401f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400ce2c0 0x7fa9401f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 -- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9400d0610 msgr2=0x7fa940160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9400d0610 0x7fa940160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 -- 172.21.15.181:0/3037365470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c009550 con 0x7fa94015dbd0 2023-10-16T09:44:36.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa945804700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa940151f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa93800cc10 tx=0x7fa93800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:36.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa935ffb700 1 -- 172.21.15.181:0/3037365470 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93800aaa0 con 0x7fa94015dbd0 2023-10-16T09:44:36.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa935ffb700 1 -- 172.21.15.181:0/3037365470 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93800ac20 con 0x7fa94015dbd0 2023-10-16T09:44:36.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa935ffb700 1 -- 172.21.15.181:0/3037365470 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9380070d0 con 0x7fa94015dbd0 2023-10-16T09:44:36.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9401603a0 con 0x7fa94015dbd0 2023-10-16T09:44:36.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.661+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa940160960 con 0x7fa94015dbd0 2023-10-16T09:44:36.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.662+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa940067d20 con 0x7fa94015dbd0 2023-10-16T09:44:36.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.663+0000 7fa935ffb700 1 -- 172.21.15.181:0/3037365470 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa938007230 con 0x7fa94015dbd0 2023-10-16T09:44:36.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.663+0000 7fa935ffb700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa92406af00 0x7fa92406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:36.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.663+0000 7fa935ffb700 1 -- 172.21.15.181:0/3037365470 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa938088dd0 con 0x7fa94015dbd0 2023-10-16T09:44:36.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.663+0000 7fa945003700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa92406af00 0x7fa92406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:36.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.663+0000 7fa945003700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa92406af00 0x7fa92406d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009c40 tx=0x7fa93c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:36.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.667+0000 7fa935ffb700 1 -- 172.21.15.181:0/3037365470 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa938053f10 con 0x7fa94015dbd0 2023-10-16T09:44:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.859+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9400d03f0 con 0x7fa94015dbd0 2023-10-16T09:44:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.860+0000 7fa935ffb700 1 -- 172.21.15.181:0/3037365470 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa93808a0a0 con 0x7fa94015dbd0 2023-10-16T09:44:36.862 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa92406af00 msgr2=0x7fa92406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa92406af00 0x7fa92406d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009c40 tx=0x7fa93c008030 comp rx=0 tx=0).stop 2023-10-16T09:44:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 msgr2=0x7fa940151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa940151f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa93800cc10 tx=0x7fa93800cf20 comp rx=0 tx=0).stop 2023-10-16T09:44:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 shutdown_connections 2023-10-16T09:44:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa92406af00 0x7fa92406d3c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9400ce2c0 0x7fa9401f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa94015dbd0 0x7fa940151f80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 --2- 172.21.15.181:0/3037365470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9400d0610 0x7fa940160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.862+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 >> 172.21.15.181:0/3037365470 conn(0x7fa94009a930 msgr2=0x7fa94015b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.863+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 shutdown_connections 2023-10-16T09:44:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.863+0000 7fa947267700 1 -- 172.21.15.181:0/3037365470 wait complete. 2023-10-16T09:44:36.877 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:44:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.996+0000 7fe130099700 1 Processor -- start 2023-10-16T09:44:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.996+0000 7fe130099700 1 -- start start 2023-10-16T09:44:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe130099700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe12806d210 0x7fe1280cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe130099700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1280cc4c0 0x7fe1280cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe130099700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1280cf460 con 0x7fe1280ce300 2023-10-16T09:44:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe130099700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1280d0380 con 0x7fe1280cc4c0 2023-10-16T09:44:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe130099700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280cc940 con 0x7fe12806d210 2023-10-16T09:44:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12d634700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1280cc4c0 0x7fe1280cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12d634700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1280cc4c0 0x7fe1280cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37288/0 (socket says 172.21.15.181:37288) 2023-10-16T09:44:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12d634700 1 -- 172.21.15.181:0/1117931959 learned_addr learned my addr 172.21.15.181:0/1117931959 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12de35700 1 --2- 172.21.15.181:0/1117931959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe12806d210 0x7fe1280cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12d634700 1 -- 172.21.15.181:0/1117931959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe12806d210 msgr2=0x7fe1280cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12d634700 1 --2- 172.21.15.181:0/1117931959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe12806d210 0x7fe1280cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12d634700 1 -- 172.21.15.181:0/1117931959 >> v1:172.21.15.190:6789/0 conn(0x7fe1280ce300 legacy=0x7fe12815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.997+0000 7fe12d634700 1 -- 172.21.15.181:0/1117931959 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12806c700 con 0x7fe1280cc4c0 2023-10-16T09:44:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe12d634700 1 --2- 172.21.15.181:0/1117931959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280cc4c0 0x7fe1280cddc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1240098f0 tx=0x7fe124009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636d0412069a116f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe12ce33700 1 -- 172.21.15.181:0/1117931959 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124011830 con 0x7fe1280cc4c0 2023-10-16T09:44:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe12ce33700 1 -- 172.21.15.181:0/1117931959 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe124011990 con 0x7fe1280cc4c0 2023-10-16T09:44:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe12ce33700 1 -- 172.21.15.181:0/1117931959 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124011d20 con 0x7fe1280cc4c0 2023-10-16T09:44:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 -- 172.21.15.181:0/1117931959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280cc4c0 msgr2=0x7fe1280cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 --2- 172.21.15.181:0/1117931959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280cc4c0 0x7fe1280cddc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1240098f0 tx=0x7fe124009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 -- 172.21.15.181:0/1117931959 shutdown_connections 2023-10-16T09:44:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 --2- 172.21.15.181:0/1117931959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe12806d210 0x7fe1280cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 --2- 172.21.15.181:0/1117931959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280cc4c0 0x7fe1280cddc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 -- 172.21.15.181:0/1117931959 >> 172.21.15.181:0/1117931959 conn(0x7fe12809a9a0 msgr2=0x7fe1280d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 -- 172.21.15.181:0/1117931959 shutdown_connections 2023-10-16T09:44:37.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:36.998+0000 7fe130099700 1 -- 172.21.15.181:0/1117931959 wait complete. 2023-10-16T09:44:37.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 Processor -- start 2023-10-16T09:44:37.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 -- start start 2023-10-16T09:44:37.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe12806d210 0x7fe12815dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1280cc4c0 0x7fe1280c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280ce300 0x7fe1280c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12809c1d0 con 0x7fe1280ce300 2023-10-16T09:44:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12809f270 con 0x7fe12806d210 2023-10-16T09:44:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe130099700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12809b410 con 0x7fe1280cc4c0 2023-10-16T09:44:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe12e636700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280ce300 0x7fe1280c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe12e636700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280ce300 0x7fe1280c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37298/0 (socket says 172.21.15.181:37298) 2023-10-16T09:44:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.000+0000 7fe12e636700 1 -- 172.21.15.181:0/1060524425 learned_addr learned my addr 172.21.15.181:0/1060524425 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12d634700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1280cc4c0 0x7fe1280c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12de35700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe12806d210 0x7fe12815dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12e636700 1 -- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1280cc4c0 msgr2=0x7fe1280c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12e636700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1280cc4c0 0x7fe1280c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12e636700 1 -- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe12806d210 msgr2=0x7fe12815dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12e636700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe12806d210 0x7fe12815dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12e636700 1 -- 172.21.15.181:0/1060524425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124009550 con 0x7fe1280ce300 2023-10-16T09:44:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe12e636700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280ce300 0x7fe1280c7700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe12000cc50 tx=0x7fe12000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe11e7fc700 1 -- 172.21.15.181:0/1060524425 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120003c00 con 0x7fe1280ce300 2023-10-16T09:44:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe11e7fc700 1 -- 172.21.15.181:0/1060524425 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe120003d60 con 0x7fe1280ce300 2023-10-16T09:44:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12809b6f0 con 0x7fe1280ce300 2023-10-16T09:44:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe11e7fc700 1 -- 172.21.15.181:0/1060524425 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000aab0 con 0x7fe1280ce300 2023-10-16T09:44:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.001+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12809bcb0 con 0x7fe1280ce300 2023-10-16T09:44:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.002+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe128067d20 con 0x7fe1280ce300 2023-10-16T09:44:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.003+0000 7fe11e7fc700 1 -- 172.21.15.181:0/1060524425 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe12000ac10 con 0x7fe1280ce300 2023-10-16T09:44:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.003+0000 7fe11e7fc700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe10c06af00 0x7fe10c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.003+0000 7fe11e7fc700 1 -- 172.21.15.181:0/1060524425 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe120089ee0 con 0x7fe1280ce300 2023-10-16T09:44:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.003+0000 7fe12de35700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe10c06af00 0x7fe10c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.003+0000 7fe12de35700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe10c06af00 0x7fe10c06d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe1280b75e0 tx=0x7fe118009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.007+0000 7fe11e7fc700 1 -- 172.21.15.181:0/1060524425 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe120055100 con 0x7fe1280ce300 2023-10-16T09:44:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.214+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe1280cfcb0 con 0x7fe1280ce300 2023-10-16T09:44:37.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.215+0000 7fe11e7fc700 1 -- 172.21.15.181:0/1060524425 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe120058720 con 0x7fe1280ce300 2023-10-16T09:44:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe10c06af00 msgr2=0x7fe10c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe10c06af00 0x7fe10c06d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe1280b75e0 tx=0x7fe118009210 comp rx=0 tx=0).stop 2023-10-16T09:44:37.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280ce300 msgr2=0x7fe1280c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280ce300 0x7fe1280c7700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe12000cc50 tx=0x7fe12000cf60 comp rx=0 tx=0).stop 2023-10-16T09:44:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 shutdown_connections 2023-10-16T09:44:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe10c06af00 0x7fe10c06d3c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280ce300 0x7fe1280c7700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe12806d210 0x7fe12815dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 --2- 172.21.15.181:0/1060524425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1280cc4c0 0x7fe1280c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 >> 172.21.15.181:0/1060524425 conn(0x7fe12809a9a0 msgr2=0x7fe12806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:37.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 shutdown_connections 2023-10-16T09:44:37.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.217+0000 7fe130099700 1 -- 172.21.15.181:0/1060524425 wait complete. 2023-10-16T09:44:37.222 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2023-10-16T09:44:37.231 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-16T09:44:37.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:37.232 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-16T09:44:37.277 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:44:37.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:37.278 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T09:44:37.283 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:44:37.285 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:44:37.311 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:37.311 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:37.343 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:37.343 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:37.400 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:44:37.401 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:37.457 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:37.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.614+0000 7ffa1b9cf700 1 Processor -- start 2023-10-16T09:44:37.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.615+0000 7ffa1b9cf700 1 -- start start 2023-10-16T09:44:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.615+0000 7ffa1b9cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa140cfaa0 0x7ffa1415fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.615+0000 7ffa1b9cf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1415bb80 con 0x7ffa1415b7a0 2023-10-16T09:44:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.615+0000 7ffa1b9cf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa140718a0 con 0x7ffa14070bb0 2023-10-16T09:44:37.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.615+0000 7ffa1b9cf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa14160300 con 0x7ffa140cfaa0 2023-10-16T09:44:37.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa18f6a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa140cfaa0 0x7ffa1415fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa1976b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ffa1415b7a0 0x7ffa140cf390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48380/0 (socket says 172.21.15.181:48380) 2023-10-16T09:44:37.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa18f6a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa140cfaa0 0x7ffa1415fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37302/0 (socket says 172.21.15.181:37302) 2023-10-16T09:44:37.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa1976b700 1 -- 172.21.15.181:0/4265681499 learned_addr learned my addr 172.21.15.181:0/4265681499 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:37.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa18f6a700 1 -- 172.21.15.181:0/4265681499 >> v1:172.21.15.190:6789/0 conn(0x7ffa14070bb0 legacy=0x7ffa140ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa0bfff700 1 -- 172.21.15.181:0/4265681499 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364836246 0 0) 0x7ffa1415bb80 con 0x7ffa1415b7a0 2023-10-16T09:44:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa18f6a700 1 -- 172.21.15.181:0/4265681499 >> v1:172.21.15.181:6789/0 conn(0x7ffa1415b7a0 legacy=0x7ffa140cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa18f6a700 1 -- 172.21.15.181:0/4265681499 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa140712e0 con 0x7ffa140cfaa0 2023-10-16T09:44:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa18f6a700 1 --2- 172.21.15.181:0/4265681499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa140cfaa0 0x7ffa1415fdc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009840 tx=0x7ffa10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222ba915b8258710 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa0bfff700 1 -- 172.21.15.181:0/4265681499 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10004230 con 0x7ffa140cfaa0 2023-10-16T09:44:37.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa0bfff700 1 -- 172.21.15.181:0/4265681499 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa10004390 con 0x7ffa140cfaa0 2023-10-16T09:44:37.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.616+0000 7ffa0bfff700 1 -- 172.21.15.181:0/4265681499 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa100036a0 con 0x7ffa140cfaa0 2023-10-16T09:44:37.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.617+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/4265681499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa140cfaa0 msgr2=0x7ffa1415fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.617+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/4265681499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa140cfaa0 0x7ffa1415fdc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009840 tx=0x7ffa10009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:37.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.617+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/4265681499 shutdown_connections 2023-10-16T09:44:37.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.617+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/4265681499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa140cfaa0 0x7ffa1415fdc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.617+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/4265681499 >> 172.21.15.181:0/4265681499 conn(0x7ffa1406aed0 msgr2=0x7ffa140b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:37.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.617+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/4265681499 shutdown_connections 2023-10-16T09:44:37.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.617+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/4265681499 wait complete. 2023-10-16T09:44:37.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.618+0000 7ffa1b9cf700 1 Processor -- start 2023-10-16T09:44:37.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.618+0000 7ffa1b9cf700 1 -- start start 2023-10-16T09:44:37.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1b9cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14070bb0 0x7ffa141f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1b9cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140cfaa0 0x7ffa140c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1b9cf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415b7a0 0x7ffa141528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1b9cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141f7380 con 0x7ffa14070bb0 2023-10-16T09:44:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1b9cf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140c4940 con 0x7ffa1415b7a0 2023-10-16T09:44:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1b9cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140b3da0 con 0x7ffa140cfaa0 2023-10-16T09:44:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14070bb0 0x7ffa141f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14070bb0 0x7ffa141f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37308/0 (socket says 172.21.15.181:37308) 2023-10-16T09:44:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 -- 172.21.15.181:0/2981662354 learned_addr learned my addr 172.21.15.181:0/2981662354 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa19f6c700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415b7a0 0x7ffa141528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa18f6a700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140cfaa0 0x7ffa140c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 -- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140cfaa0 msgr2=0x7ffa140c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140cfaa0 0x7ffa140c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 -- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415b7a0 msgr2=0x7ffa141528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415b7a0 0x7ffa141528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa1976b700 1 -- 172.21.15.181:0/2981662354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa100094a0 con 0x7ffa14070bb0 2023-10-16T09:44:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.619+0000 7ffa19f6c700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415b7a0 0x7ffa141528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.620+0000 7ffa1976b700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14070bb0 0x7ffa141f6450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa040011e0 tx=0x7ffa0400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.620+0000 7ffa09ffb700 1 -- 172.21.15.181:0/2981662354 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04004c80 con 0x7ffa14070bb0 2023-10-16T09:44:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.620+0000 7ffa09ffb700 1 -- 172.21.15.181:0/2981662354 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa04004e00 con 0x7ffa14070bb0 2023-10-16T09:44:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.620+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa14152e10 con 0x7ffa14070bb0 2023-10-16T09:44:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.620+0000 7ffa09ffb700 1 -- 172.21.15.181:0/2981662354 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04007310 con 0x7ffa14070bb0 2023-10-16T09:44:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.620+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa141f6dc0 con 0x7ffa14070bb0 2023-10-16T09:44:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.621+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1404ebb0 con 0x7ffa14070bb0 2023-10-16T09:44:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.621+0000 7ffa09ffb700 1 -- 172.21.15.181:0/2981662354 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffa04007470 con 0x7ffa14070bb0 2023-10-16T09:44:37.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.621+0000 7ffa09ffb700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9f806af00 0x7ff9f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.622+0000 7ffa09ffb700 1 -- 172.21.15.181:0/2981662354 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa04089380 con 0x7ffa14070bb0 2023-10-16T09:44:37.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.622+0000 7ffa18f6a700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9f806af00 0x7ff9f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.622+0000 7ffa18f6a700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9f806af00 0x7ff9f806d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa10000c00 tx=0x7ffa1001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.626+0000 7ffa09ffb700 1 -- 172.21.15.181:0/2981662354 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffa04054430 con 0x7ffa14070bb0 2023-10-16T09:44:37.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.820+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa140b1d40 con 0x7ffa14070bb0 2023-10-16T09:44:37.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.821+0000 7ffa09ffb700 1 -- 172.21.15.181:0/2981662354 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7ffa04057a50 con 0x7ffa14070bb0 2023-10-16T09:44:37.823 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:37.824 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:37.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9f806af00 msgr2=0x7ff9f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9f806af00 0x7ff9f806d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa10000c00 tx=0x7ffa1001c040 comp rx=0 tx=0).stop 2023-10-16T09:44:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14070bb0 msgr2=0x7ffa141f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14070bb0 0x7ffa141f6450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa040011e0 tx=0x7ffa0400cae0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 shutdown_connections 2023-10-16T09:44:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff9f806af00 0x7ff9f806d3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa1415b7a0 0x7ffa141528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14070bb0 0x7ffa141f6450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 --2- 172.21.15.181:0/2981662354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa140cfaa0 0x7ffa140c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 >> 172.21.15.181:0/2981662354 conn(0x7ffa1406aed0 msgr2=0x7ffa140b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:37.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 shutdown_connections 2023-10-16T09:44:37.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.823+0000 7ffa1b9cf700 1 -- 172.21.15.181:0/2981662354 wait complete. 2023-10-16T09:44:37.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:37.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:37.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.961+0000 7f1c9abec700 1 Processor -- start 2023-10-16T09:44:37.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.961+0000 7f1c9abec700 1 -- start start 2023-10-16T09:44:37.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.961+0000 7f1c9abec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9415da70 0x7f1c940ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c9abec700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c9406afb0 0x7f1c941615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c9abec700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c940cef00 con 0x7f1c940ce6a0 2023-10-16T09:44:37.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c9abec700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c940cec20 con 0x7f1c9406afb0 2023-10-16T09:44:37.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c9abec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94071c80 con 0x7f1c9415da70 2023-10-16T09:44:37.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c9406afb0 0x7f1c941615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c9406afb0 0x7f1c941615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37318/0 (socket says 172.21.15.181:37318) 2023-10-16T09:44:37.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 -- 172.21.15.181:0/2935237857 learned_addr learned my addr 172.21.15.181:0/2935237857 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:37.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 -- 172.21.15.181:0/2935237857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9415da70 msgr2=0x7f1c940ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:37.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 --2- 172.21.15.181:0/2935237857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9415da70 0x7f1c940ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 -- 172.21.15.181:0/2935237857 >> v1:172.21.15.190:6789/0 conn(0x7f1c940ce6a0 legacy=0x7f1c9406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 -- 172.21.15.181:0/2935237857 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94071f60 con 0x7f1c9406afb0 2023-10-16T09:44:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c99189700 1 --2- 172.21.15.181:0/2935237857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c9406afb0 0x7f1c941615f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009e50 tx=0x7f1c8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9542326f84f40760 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c937fe700 1 -- 172.21.15.181:0/2935237857 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00e070 con 0x7f1c9406afb0 2023-10-16T09:44:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c937fe700 1 -- 172.21.15.181:0/2935237857 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c003a20 con 0x7f1c9406afb0 2023-10-16T09:44:37.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.962+0000 7f1c937fe700 1 -- 172.21.15.181:0/2935237857 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c003dd0 con 0x7f1c9406afb0 2023-10-16T09:44:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 -- 172.21.15.181:0/2935237857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c9406afb0 msgr2=0x7f1c941615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2935237857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c9406afb0 0x7f1c941615f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009e50 tx=0x7f1c8c00b330 comp rx=0 tx=0).stop 2023-10-16T09:44:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 -- 172.21.15.181:0/2935237857 shutdown_connections 2023-10-16T09:44:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2935237857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c9406afb0 0x7f1c941615f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2935237857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9415da70 0x7f1c940ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 -- 172.21.15.181:0/2935237857 >> 172.21.15.181:0/2935237857 conn(0x7f1c940cd010 msgr2=0x7f1c940cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:37.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 -- 172.21.15.181:0/2935237857 shutdown_connections 2023-10-16T09:44:37.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.963+0000 7f1c9abec700 1 -- 172.21.15.181:0/2935237857 wait complete. 2023-10-16T09:44:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.964+0000 7f1c9abec700 1 Processor -- start 2023-10-16T09:44:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.964+0000 7f1c9abec700 1 -- start start 2023-10-16T09:44:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c9abec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9406afb0 0x7f1c94158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c9abec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c940ce6a0 0x7f1c940ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c9abec700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c9415da70 0x7f1c940c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c9abec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c6740 con 0x7f1c940ce6a0 2023-10-16T09:44:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c9abec700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c7ad0 con 0x7f1c9415da70 2023-10-16T09:44:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c9abec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c7db0 con 0x7f1c9406afb0 2023-10-16T09:44:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c940ce6a0 0x7f1c940ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c940ce6a0 0x7f1c940ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37322/0 (socket says 172.21.15.181:37322) 2023-10-16T09:44:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 -- 172.21.15.181:0/2890266721 learned_addr learned my addr 172.21.15.181:0/2890266721 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c99189700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c9415da70 0x7f1c940c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c98988700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9406afb0 0x7f1c94158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 -- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9406afb0 msgr2=0x7f1c94158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9406afb0 0x7f1c94158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 -- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c9415da70 msgr2=0x7f1c940c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c9415da70 0x7f1c940c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c93fff700 1 -- 172.21.15.181:0/2890266721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c009b30 con 0x7f1c940ce6a0 2023-10-16T09:44:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c98988700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9406afb0 0x7f1c94158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.965+0000 7f1c99189700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c9415da70 0x7f1c940c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.966+0000 7f1c93fff700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c940ce6a0 0x7f1c940ca320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c4a0 tx=0x7f1c8000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.966+0000 7f1c917fa700 1 -- 172.21.15.181:0/2890266721 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c800054e0 con 0x7f1c940ce6a0 2023-10-16T09:44:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.966+0000 7f1c917fa700 1 -- 172.21.15.181:0/2890266721 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c80003680 con 0x7f1c940ce6a0 2023-10-16T09:44:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.966+0000 7f1c917fa700 1 -- 172.21.15.181:0/2890266721 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c80003a30 con 0x7f1c940ce6a0 2023-10-16T09:44:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.966+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c940c60f0 con 0x7f1c940ce6a0 2023-10-16T09:44:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.966+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c940ca860 con 0x7f1c940ce6a0 2023-10-16T09:44:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.967+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c940cca70 con 0x7f1c940ce6a0 2023-10-16T09:44:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.967+0000 7f1c917fa700 1 -- 172.21.15.181:0/2890266721 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1c80003b90 con 0x7f1c940ce6a0 2023-10-16T09:44:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.967+0000 7f1c917fa700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c7806af00 0x7f1c7806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.968+0000 7f1c917fa700 1 -- 172.21.15.181:0/2890266721 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c80088670 con 0x7f1c940ce6a0 2023-10-16T09:44:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.968+0000 7f1c98988700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c7806af00 0x7f1c7806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.968+0000 7f1c98988700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c7806af00 0x7f1c7806d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c88007900 tx=0x7f1c88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:37.972+0000 7f1c917fa700 1 -- 172.21.15.181:0/2890266721 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c940cca70 con 0x7f1c940ce6a0 2023-10-16T09:44:38.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.164+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c940cca70 con 0x7f1c940ce6a0 2023-10-16T09:44:38.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.164+0000 7f1c917fa700 1 -- 172.21.15.181:0/2890266721 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c940cca70 con 0x7f1c940ce6a0 2023-10-16T09:44:38.167 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c7806af00 msgr2=0x7f1c7806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c7806af00 0x7f1c7806d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c88007900 tx=0x7f1c88008040 comp rx=0 tx=0).stop 2023-10-16T09:44:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c940ce6a0 msgr2=0x7f1c940ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c940ce6a0 0x7f1c940ca320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c4a0 tx=0x7f1c8000c860 comp rx=0 tx=0).stop 2023-10-16T09:44:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 shutdown_connections 2023-10-16T09:44:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1c7806af00 0x7f1c7806d3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c9406afb0 0x7f1c94158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c9415da70 0x7f1c940c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 --2- 172.21.15.181:0/2890266721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c940ce6a0 0x7f1c940ca320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.166+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 >> 172.21.15.181:0/2890266721 conn(0x7f1c940cd010 msgr2=0x7f1c940cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:38.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.167+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 shutdown_connections 2023-10-16T09:44:38.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.167+0000 7f1c9abec700 1 -- 172.21.15.181:0/2890266721 wait complete. 2023-10-16T09:44:38.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:38.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.302+0000 7ff241a68700 1 Processor -- start 2023-10-16T09:44:38.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.302+0000 7ff241a68700 1 -- start start 2023-10-16T09:44:38.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff241a68700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff23c15b850 0x7ff23c0665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff241a68700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff23c15fee0 0x7ff23c0c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff241a68700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c06e4b0 con 0x7ff23c0c3d40 2023-10-16T09:44:38.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff241a68700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c06ed50 con 0x7ff23c15b850 2023-10-16T09:44:38.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff241a68700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c0cb0f0 con 0x7ff23c15fee0 2023-10-16T09:44:38.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff23c15b850 0x7ff23c0665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff23c15b850 0x7ff23c0665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37328/0 (socket says 172.21.15.181:37328) 2023-10-16T09:44:38.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 -- 172.21.15.181:0/2982819808 learned_addr learned my addr 172.21.15.181:0/2982819808 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:38.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23affd700 1 --2- 172.21.15.181:0/2982819808 >> v2:172.21.15.190:3300/0 conn(0x7ff23c15fee0 0x7ff23c0c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 -- 172.21.15.181:0/2982819808 >> v2:172.21.15.190:3300/0 conn(0x7ff23c15fee0 msgr2=0x7ff23c0c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:38.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 --2- 172.21.15.181:0/2982819808 >> v2:172.21.15.190:3300/0 conn(0x7ff23c15fee0 0x7ff23c0c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 -- 172.21.15.181:0/2982819808 >> v1:172.21.15.190:6789/0 conn(0x7ff23c0c3d40 legacy=0x7ff23c06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:38.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 -- 172.21.15.181:0/2982819808 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c066b80 con 0x7ff23c15b850 2023-10-16T09:44:38.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23affd700 1 --2- 172.21.15.181:0/2982819808 >> v2:172.21.15.190:3300/0 conn(0x7ff23c15fee0 0x7ff23c0c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:38.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23b7fe700 1 --2- 172.21.15.181:0/2982819808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c15b850 0x7ff23c0665a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff23c067360 tx=0x7ff230009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa247816f81bb19 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:38.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.303+0000 7ff23a7fc700 1 -- 172.21.15.181:0/2982819808 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230004230 con 0x7ff23c15b850 2023-10-16T09:44:38.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff23a7fc700 1 -- 172.21.15.181:0/2982819808 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230004390 con 0x7ff23c15b850 2023-10-16T09:44:38.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff23a7fc700 1 -- 172.21.15.181:0/2982819808 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300036a0 con 0x7ff23c15b850 2023-10-16T09:44:38.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 -- 172.21.15.181:0/2982819808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c15b850 msgr2=0x7ff23c0665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 --2- 172.21.15.181:0/2982819808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c15b850 0x7ff23c0665a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff23c067360 tx=0x7ff230009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 -- 172.21.15.181:0/2982819808 shutdown_connections 2023-10-16T09:44:38.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 --2- 172.21.15.181:0/2982819808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c15b850 0x7ff23c0665a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 --2- 172.21.15.181:0/2982819808 >> v2:172.21.15.190:3300/0 conn(0x7ff23c15fee0 0x7ff23c0c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 -- 172.21.15.181:0/2982819808 >> 172.21.15.181:0/2982819808 conn(0x7ff23c0cc730 msgr2=0x7ff23c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:38.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 -- 172.21.15.181:0/2982819808 shutdown_connections 2023-10-16T09:44:38.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.304+0000 7ff241a68700 1 -- 172.21.15.181:0/2982819808 wait complete. 2023-10-16T09:44:38.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.305+0000 7ff241a68700 1 Processor -- start 2023-10-16T09:44:38.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.305+0000 7ff241a68700 1 -- start start 2023-10-16T09:44:38.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff241a68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c0c3d40 0x7ff23c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff241a68700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff23c15b850 0x7ff23c1f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff241a68700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff23c15fee0 0x7ff23c1f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff241a68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c1f3030 con 0x7ff23c0c3d40 2023-10-16T09:44:38.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff241a68700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0b5890 con 0x7ff23c15fee0 2023-10-16T09:44:38.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff241a68700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0b4900 con 0x7ff23c15b850 2023-10-16T09:44:38.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff23b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c0c3d40 0x7ff23c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff23b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c0c3d40 0x7ff23c0b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37334/0 (socket says 172.21.15.181:37334) 2023-10-16T09:44:38.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff23b7fe700 1 -- 172.21.15.181:0/1591119037 learned_addr learned my addr 172.21.15.181:0/1591119037 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:38.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff23affd700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff23c15b850 0x7ff23c1f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.306+0000 7ff23bfff700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff23c15fee0 0x7ff23c1f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff23b7fe700 1 -- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff23c15b850 msgr2=0x7ff23c1f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff23b7fe700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff23c15b850 0x7ff23c1f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff23b7fe700 1 -- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff23c15fee0 msgr2=0x7ff23c1f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff23b7fe700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff23c15fee0 0x7ff23c1f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff23b7fe700 1 -- 172.21.15.181:0/1591119037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230009550 con 0x7ff23c0c3d40 2023-10-16T09:44:38.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff23b7fe700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c0c3d40 0x7ff23c0b5350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff23c070de0 tx=0x7ff230004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:38.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff223fff700 1 -- 172.21.15.181:0/1591119037 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230003c90 con 0x7ff23c0c3d40 2023-10-16T09:44:38.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff223fff700 1 -- 172.21.15.181:0/1591119037 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23001f070 con 0x7ff23c0c3d40 2023-10-16T09:44:38.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c0b4ec0 con 0x7ff23c0c3d40 2023-10-16T09:44:38.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff223fff700 1 -- 172.21.15.181:0/1591119037 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230011870 con 0x7ff23c0c3d40 2023-10-16T09:44:38.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.307+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c0ade10 con 0x7ff23c0c3d40 2023-10-16T09:44:38.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.308+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c0cd5d0 con 0x7ff23c0c3d40 2023-10-16T09:44:38.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.308+0000 7ff223fff700 1 -- 172.21.15.181:0/1591119037 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff230021030 con 0x7ff23c0c3d40 2023-10-16T09:44:38.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.309+0000 7ff223fff700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff22806af00 0x7ff22806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.309+0000 7ff23affd700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff22806af00 0x7ff22806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.309+0000 7ff223fff700 1 -- 172.21.15.181:0/1591119037 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff23008aa60 con 0x7ff23c0c3d40 2023-10-16T09:44:38.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.309+0000 7ff23affd700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff22806af00 0x7ff22806d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0af270 tx=0x7ff22c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:38.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.314+0000 7ff223fff700 1 -- 172.21.15.181:0/1591119037 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff230055ba0 con 0x7ff23c0c3d40 2023-10-16T09:44:38.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.502+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff23c0adfa0 con 0x7ff23c0c3d40 2023-10-16T09:44:38.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.503+0000 7ff223fff700 1 -- 172.21.15.181:0/1591119037 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2300591c0 con 0x7ff23c0c3d40 2023-10-16T09:44:38.505 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:38.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff22806af00 msgr2=0x7ff22806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff22806af00 0x7ff22806d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0af270 tx=0x7ff22c008040 comp rx=0 tx=0).stop 2023-10-16T09:44:38.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c0c3d40 msgr2=0x7ff23c0b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c0c3d40 0x7ff23c0b5350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff23c070de0 tx=0x7ff230004750 comp rx=0 tx=0).stop 2023-10-16T09:44:38.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 shutdown_connections 2023-10-16T09:44:38.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff22806af00 0x7ff22806d3c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff23c15fee0 0x7ff23c1f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff23c0c3d40 0x7ff23c0b5350 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 --2- 172.21.15.181:0/1591119037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff23c15b850 0x7ff23c1f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 >> 172.21.15.181:0/1591119037 conn(0x7ff23c0cc730 msgr2=0x7ff23c0c5460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:38.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 shutdown_connections 2023-10-16T09:44:38.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.505+0000 7ff241a68700 1 -- 172.21.15.181:0/1591119037 wait complete. 2023-10-16T09:44:38.519 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:38.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.640+0000 7fdc1f539700 1 Processor -- start 2023-10-16T09:44:38.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.640+0000 7fdc1f539700 1 -- start start 2023-10-16T09:44:38.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1f539700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc18072760 0x7fdc1806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1f539700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc1806ade0 0x7fdc181615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1f539700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1806b230 con 0x7fdc1815da70 2023-10-16T09:44:38.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1f539700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc180cef00 con 0x7fdc1806ade0 2023-10-16T09:44:38.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1f539700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180cec20 con 0x7fdc18072760 2023-10-16T09:44:38.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1dad6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc1806ade0 0x7fdc181615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1dad6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc1806ade0 0x7fdc181615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37346/0 (socket says 172.21.15.181:37346) 2023-10-16T09:44:38.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1dad6700 1 -- 172.21.15.181:0/673773420 learned_addr learned my addr 172.21.15.181:0/673773420 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:38.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.641+0000 7fdc1cad4700 1 --2- 172.21.15.181:0/673773420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc18072760 0x7fdc1806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1dad6700 1 -- 172.21.15.181:0/673773420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc18072760 msgr2=0x7fdc1806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:38.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1dad6700 1 --2- 172.21.15.181:0/673773420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc18072760 0x7fdc1806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1dad6700 1 -- 172.21.15.181:0/673773420 >> v1:172.21.15.190:6789/0 conn(0x7fdc1815da70 legacy=0x7fdc180ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1dad6700 1 -- 172.21.15.181:0/673773420 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc18071c80 con 0x7fdc1806ade0 2023-10-16T09:44:38.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1cad4700 1 --2- 172.21.15.181:0/673773420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc18072760 0x7fdc1806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:38.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1dad6700 1 --2- 172.21.15.181:0/673773420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181615f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009e50 tx=0x7fdc1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de44d0d976de75e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:38.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc0ffff700 1 -- 172.21.15.181:0/673773420 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000e070 con 0x7fdc1806ade0 2023-10-16T09:44:38.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc0ffff700 1 -- 172.21.15.181:0/673773420 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1000b9c0 con 0x7fdc1806ade0 2023-10-16T09:44:38.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc0ffff700 1 -- 172.21.15.181:0/673773420 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000bd70 con 0x7fdc1806ade0 2023-10-16T09:44:38.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1f539700 1 -- 172.21.15.181:0/673773420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 msgr2=0x7fdc181615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1f539700 1 --2- 172.21.15.181:0/673773420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181615f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009e50 tx=0x7fdc1000b330 comp rx=0 tx=0).stop 2023-10-16T09:44:38.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1f539700 1 -- 172.21.15.181:0/673773420 shutdown_connections 2023-10-16T09:44:38.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1f539700 1 --2- 172.21.15.181:0/673773420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181615f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1f539700 1 --2- 172.21.15.181:0/673773420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc18072760 0x7fdc1806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.642+0000 7fdc1f539700 1 -- 172.21.15.181:0/673773420 >> 172.21.15.181:0/673773420 conn(0x7fdc180cd010 msgr2=0x7fdc180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:38.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.643+0000 7fdc1f539700 1 -- 172.21.15.181:0/673773420 shutdown_connections 2023-10-16T09:44:38.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.643+0000 7fdc1f539700 1 -- 172.21.15.181:0/673773420 wait complete. 2023-10-16T09:44:38.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 Processor -- start 2023-10-16T09:44:38.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 -- start start 2023-10-16T09:44:38.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc18072760 0x7fdc1814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc1815da70 0x7fdc180cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc181f3330 con 0x7fdc1806ade0 2023-10-16T09:44:38.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18147470 con 0x7fdc18072760 2023-10-16T09:44:38.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.644+0000 7fdc1f539700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1814a700 con 0x7fdc1815da70 2023-10-16T09:44:38.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37348/0 (socket says 172.21.15.181:37348) 2023-10-16T09:44:38.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 -- 172.21.15.181:0/703349939 learned_addr learned my addr 172.21.15.181:0/703349939 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:38.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 -- 172.21.15.181:0/703349939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc1815da70 msgr2=0x7fdc180cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc1815da70 0x7fdc180cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 -- 172.21.15.181:0/703349939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc18072760 msgr2=0x7fdc1814a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc18072760 0x7fdc1814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 -- 172.21.15.181:0/703349939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc10009b30 con 0x7fdc1806ade0 2023-10-16T09:44:38.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1d2d5700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181f2df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c450 tx=0x7fdc0800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:38.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc0dffb700 1 -- 172.21.15.181:0/703349939 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc080052d0 con 0x7fdc1806ade0 2023-10-16T09:44:38.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc0dffb700 1 -- 172.21.15.181:0/703349939 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08005430 con 0x7fdc1806ade0 2023-10-16T09:44:38.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc181f6cb0 con 0x7fdc1806ade0 2023-10-16T09:44:38.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc0dffb700 1 -- 172.21.15.181:0/703349939 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc080038c0 con 0x7fdc1806ade0 2023-10-16T09:44:38.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.645+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc181f7550 con 0x7fdc1806ade0 2023-10-16T09:44:38.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.646+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc181f70d0 con 0x7fdc1806ade0 2023-10-16T09:44:38.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.647+0000 7fdc0dffb700 1 -- 172.21.15.181:0/703349939 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc08003a20 con 0x7fdc1806ade0 2023-10-16T09:44:38.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.647+0000 7fdc0dffb700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdbfc06af30 0x7fdbfc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.647+0000 7fdc0dffb700 1 -- 172.21.15.181:0/703349939 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc08088820 con 0x7fdc1806ade0 2023-10-16T09:44:38.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.647+0000 7fdc1cad4700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdbfc06af30 0x7fdbfc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.647+0000 7fdc1cad4700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdbfc06af30 0x7fdbfc06d3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc14006fd0 tx=0x7fdc14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:38.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.652+0000 7fdc0dffb700 1 -- 172.21.15.181:0/703349939 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc181f70d0 con 0x7fdc1806ade0 2023-10-16T09:44:38.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.845+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc1806b230 con 0x7fdc1806ade0 2023-10-16T09:44:38.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.845+0000 7fdc0dffb700 1 -- 172.21.15.181:0/703349939 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc1806b230 con 0x7fdc1806ade0 2023-10-16T09:44:38.847 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:38.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.847+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdbfc06af30 msgr2=0x7fdbfc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.847+0000 7fdc1f539700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdbfc06af30 0x7fdbfc06d3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc14006fd0 tx=0x7fdc14008040 comp rx=0 tx=0).stop 2023-10-16T09:44:38.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.847+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 msgr2=0x7fdc181f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.847+0000 7fdc1f539700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181f2df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800c450 tx=0x7fdc0800c810 comp rx=0 tx=0).stop 2023-10-16T09:44:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 shutdown_connections 2023-10-16T09:44:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdbfc06af30 0x7fdbfc06d3f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc1815da70 0x7fdc180cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc1806ade0 0x7fdc181f2df0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 --2- 172.21.15.181:0/703349939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc18072760 0x7fdc1814a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 >> 172.21.15.181:0/703349939 conn(0x7fdc180cd010 msgr2=0x7fdc180b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 shutdown_connections 2023-10-16T09:44:38.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.848+0000 7fdc1f539700 1 -- 172.21.15.181:0/703349939 wait complete. 2023-10-16T09:44:38.862 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:44:38.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.983+0000 7f50e7ed7700 1 Processor -- start 2023-10-16T09:44:38.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.983+0000 7f50e7ed7700 1 -- start start 2023-10-16T09:44:38.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e7ed7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50e015daf0 0x7f50e00c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e7ed7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e015b410 con 0x7f50e015ded0 2023-10-16T09:44:38.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e7ed7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e00c2a80 con 0x7f50e00c2cb0 2023-10-16T09:44:38.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e7ed7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e00c1f50 con 0x7f50e015daf0 2023-10-16T09:44:38.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e6474700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f50e015ded0 0x7f50e006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48434/0 (socket says 172.21.15.181:48434) 2023-10-16T09:44:38.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e6474700 1 -- 172.21.15.181:0/3139962382 learned_addr learned my addr 172.21.15.181:0/3139962382 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:38.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e5c73700 1 --2- 172.21.15.181:0/3139962382 >> v2:172.21.15.190:3300/0 conn(0x7f50e015daf0 0x7f50e00c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782542655 0 0) 0x7f50e015b410 con 0x7f50e015ded0 2023-10-16T09:44:38.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50cc0035b0 con 0x7f50e015ded0 2023-10-16T09:44:38.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603835962 0 0) 0x7f50cc0035b0 con 0x7f50e015ded0 2023-10-16T09:44:38.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 >> v2:172.21.15.190:3300/0 conn(0x7f50e015daf0 msgr2=0x7f50e00c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:38.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 --2- 172.21.15.181:0/3139962382 >> v2:172.21.15.190:3300/0 conn(0x7f50e015daf0 0x7f50e00c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 >> v1:172.21.15.190:6789/0 conn(0x7f50e00c2cb0 legacy=0x7f50e00c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e015e900 con 0x7f50e015ded0 2023-10-16T09:44:38.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f50d80019f0 con 0x7f50e015ded0 2023-10-16T09:44:38.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 >> v1:172.21.15.181:6789/0 conn(0x7f50e015ded0 legacy=0x7f50e006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50cc003a00 0x7f50cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc004850 0x7f50cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc005210 0x7f50cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:38.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00c1f50 con 0x7f50cc004850 2023-10-16T09:44:38.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc00c0b0 con 0x7f50cc005210 2023-10-16T09:44:38.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc00c270 con 0x7f50cc003a00 2023-10-16T09:44:38.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e5472700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc004850 0x7f50cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.984+0000 7f50e5c73700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50cc003a00 0x7f50cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e6474700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc005210 0x7f50cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:38.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e5472700 1 -- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50cc003a00 msgr2=0x7f50cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:38.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e5472700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50cc003a00 0x7f50cc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e5472700 1 -- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc005210 msgr2=0x7f50cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:38.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e5472700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc005210 0x7f50cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e6474700 1 -- 172.21.15.181:0/3139962382 reap_dead start 2023-10-16T09:44:38.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e5472700 1 -- 172.21.15.181:0/3139962382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc0035b0 con 0x7f50cc004850 2023-10-16T09:44:38.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e5472700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc004850 0x7f50cc004cd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001800 tx=0x7f50cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c604e2f5bd909712 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:38.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d00012f0 con 0x7f50cc004850 2023-10-16T09:44:38.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d0005310 con 0x7f50cc004850 2023-10-16T09:44:38.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e4c71700 1 -- 172.21.15.181:0/3139962382 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0006860 con 0x7f50cc004850 2023-10-16T09:44:38.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e7ed7700 1 -- 172.21.15.181:0/3139962382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc004850 msgr2=0x7f50cc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:38.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc004850 0x7f50cc004cd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001800 tx=0x7f50cc005880 comp rx=0 tx=0).stop 2023-10-16T09:44:38.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.985+0000 7f50e7ed7700 1 -- 172.21.15.181:0/3139962382 shutdown_connections 2023-10-16T09:44:38.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.986+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/3139962382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc004850 0x7f50cc004cd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:38.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.986+0000 7f50e7ed7700 1 -- 172.21.15.181:0/3139962382 >> 172.21.15.181:0/3139962382 conn(0x7f50e00b1780 msgr2=0x7f50e00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:38.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.986+0000 7f50e7ed7700 1 -- 172.21.15.181:0/3139962382 shutdown_connections 2023-10-16T09:44:38.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.986+0000 7f50e7ed7700 1 -- 172.21.15.181:0/3139962382 wait complete. 2023-10-16T09:44:38.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.987+0000 7f50e7ed7700 1 Processor -- start 2023-10-16T09:44:38.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.987+0000 7f50e7ed7700 1 -- start start 2023-10-16T09:44:39.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.987+0000 7f50e7ed7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc004850 0x7f50e01f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.987+0000 7f50e7ed7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e00cb170 0x7f50e00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.987+0000 7f50e7ed7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e00b27f0 0x7f50e00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.987+0000 7f50e7ed7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00cbb30 con 0x7f50e00cb170 2023-10-16T09:44:39.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.987+0000 7f50e7ed7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00c4dd0 con 0x7f50cc004850 2023-10-16T09:44:39.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e7ed7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00b34e0 con 0x7f50e00b27f0 2023-10-16T09:44:39.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e00cb170 0x7f50e00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e00cb170 0x7f50e00cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37368/0 (socket says 172.21.15.181:37368) 2023-10-16T09:44:39.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 -- 172.21.15.181:0/2666549448 learned_addr learned my addr 172.21.15.181:0/2666549448 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:39.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5c73700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc004850 0x7f50e01f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e6474700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e00b27f0 0x7f50e00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 -- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e00b27f0 msgr2=0x7f50e00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e00b27f0 0x7f50e00b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 -- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc004850 msgr2=0x7f50e01f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc004850 0x7f50e01f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 -- 172.21.15.181:0/2666549448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e00b37c0 con 0x7f50e00cb170 2023-10-16T09:44:39.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e5472700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e00cb170 0x7f50e00cb5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f50d0014040 tx=0x7f50d0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50d67fc700 1 -- 172.21.15.181:0/2666549448 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0001290 con 0x7f50e00cb170 2023-10-16T09:44:39.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50d67fc700 1 -- 172.21.15.181:0/2666549448 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d00050d0 con 0x7f50e00cb170 2023-10-16T09:44:39.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e00b3aa0 con 0x7f50e00cb170 2023-10-16T09:44:39.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50d67fc700 1 -- 172.21.15.181:0/2666549448 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0007280 con 0x7f50e00cb170 2023-10-16T09:44:39.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.988+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e00b31e0 con 0x7f50e00cb170 2023-10-16T09:44:39.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.989+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e0067d20 con 0x7f50e00cb170 2023-10-16T09:44:39.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.989+0000 7f50d67fc700 1 -- 172.21.15.181:0/2666549448 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f50d00073e0 con 0x7f50e00cb170 2023-10-16T09:44:39.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.990+0000 7f50d67fc700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50c406af00 0x7f50c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.990+0000 7f50d67fc700 1 -- 172.21.15.181:0/2666549448 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50d008a4b0 con 0x7f50e00cb170 2023-10-16T09:44:39.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.990+0000 7f50e5c73700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50c406af00 0x7f50c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.990+0000 7f50e5c73700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50c406af00 0x7f50c406d3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009ba0 tx=0x7f50dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:38.994+0000 7f50d67fc700 1 -- 172.21.15.181:0/2666549448 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50d0055690 con 0x7f50e00cb170 2023-10-16T09:44:39.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.202+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50e00c1060 con 0x7f50e00cb170 2023-10-16T09:44:39.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.202+0000 7f50d67fc700 1 -- 172.21.15.181:0/2666549448 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f50d0058cb0 con 0x7f50e00cb170 2023-10-16T09:44:39.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.204+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50c406af00 msgr2=0x7f50c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.204+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50c406af00 0x7f50c406d3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009ba0 tx=0x7f50dc008040 comp rx=0 tx=0).stop 2023-10-16T09:44:39.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.204+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e00cb170 msgr2=0x7f50e00cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.204+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e00cb170 0x7f50e00cb5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f50d0014040 tx=0x7f50d0001120 comp rx=0 tx=0).stop 2023-10-16T09:44:39.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.204+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 shutdown_connections 2023-10-16T09:44:39.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.204+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f50c406af00 0x7f50c406d3c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.205+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e00b27f0 0x7f50e00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.205+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50cc004850 0x7f50e01f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.205+0000 7f50e7ed7700 1 --2- 172.21.15.181:0/2666549448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e00cb170 0x7f50e00cb5f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.205+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 >> 172.21.15.181:0/2666549448 conn(0x7f50e00b1780 msgr2=0x7f50e00b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:39.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.205+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 shutdown_connections 2023-10-16T09:44:39.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.205+0000 7f50e7ed7700 1 -- 172.21.15.181:0/2666549448 wait complete. 2023-10-16T09:44:39.210 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2023-10-16T09:44:39.219 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-16T09:44:39.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:39.220 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-16T09:44:39.258 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:44:39.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:39.259 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T09:44:39.315 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:39.316 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:39.337 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:39.338 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:39.398 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:44:39.398 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:44:39.459 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:39.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.578+0000 7fce6a7df700 1 Processor -- start 2023-10-16T09:44:39.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.578+0000 7fce6a7df700 1 -- start start 2023-10-16T09:44:39.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce6a7df700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce640cb0c0 0x7fce640d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce6a7df700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6406ed00 con 0x7fce6415eab0 2023-10-16T09:44:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce6a7df700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce640d02d0 con 0x7fce6415e6d0 2023-10-16T09:44:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce6a7df700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6406e0f0 con 0x7fce640cb0c0 2023-10-16T09:44:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce637fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fce6415eab0 0x7fce6415a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48456/0 (socket says 172.21.15.181:48456) 2023-10-16T09:44:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce637fe700 1 -- 172.21.15.181:0/685454327 learned_addr learned my addr 172.21.15.181:0/685454327 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186626277 0 0) 0x7fce6406ed00 con 0x7fce6415eab0 2023-10-16T09:44:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce68d7c700 1 --2- 172.21.15.181:0/685454327 >> v2:172.21.15.190:3300/0 conn(0x7fce640cb0c0 0x7fce640d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce4c0057c0 con 0x7fce6415eab0 2023-10-16T09:44:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2968870115 0 0) 0x7fce4c0057c0 con 0x7fce6415eab0 2023-10-16T09:44:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 >> v2:172.21.15.190:3300/0 conn(0x7fce640cb0c0 msgr2=0x7fce640d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:39.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 --2- 172.21.15.181:0/685454327 >> v2:172.21.15.190:3300/0 conn(0x7fce640cb0c0 0x7fce640d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 >> v1:172.21.15.190:6789/0 conn(0x7fce6415e6d0 legacy=0x7fce6415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6406e3d0 con 0x7fce6415eab0 2023-10-16T09:44:39.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fce54001670 con 0x7fce6415eab0 2023-10-16T09:44:39.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 >> v1:172.21.15.181:6789/0 conn(0x7fce6415eab0 legacy=0x7fce6415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c005c10 0x7fce4c006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.579+0000 7fce62ffd700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c006a60 0x7fce4c006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce62ffd700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce4c00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6406e0f0 con 0x7fce4c007420 2023-10-16T09:44:39.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c00e2c0 con 0x7fce4c005c10 2023-10-16T09:44:39.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c00e480 con 0x7fce4c006a60 2023-10-16T09:44:39.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce4c00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce63fff700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c005c10 0x7fce4c006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce637fe700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c006a60 0x7fce4c006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 -- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c006a60 msgr2=0x7fce4c006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:39.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c006a60 0x7fce4c006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 -- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c005c10 msgr2=0x7fce4c006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:39.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce4c005c10 0x7fce4c006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 -- 172.21.15.181:0/685454327 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0057c0 con 0x7fce4c007420 2023-10-16T09:44:39.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce637fe700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce4c006a60 0x7fce4c006ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:39.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 -- 172.21.15.181:0/685454327 reap_dead start 2023-10-16T09:44:39.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce68d7c700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce4c00dc20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009fc0 tx=0x7fce4c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f527626984c23c99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c003a20 con 0x7fce4c007420 2023-10-16T09:44:39.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c00f040 con 0x7fce4c007420 2023-10-16T09:44:39.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.580+0000 7fce62ffd700 1 -- 172.21.15.181:0/685454327 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c003de0 con 0x7fce4c007420 2023-10-16T09:44:39.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.581+0000 7fce6a7df700 1 -- 172.21.15.181:0/685454327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 msgr2=0x7fce4c00dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.581+0000 7fce6a7df700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce4c00dc20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009fc0 tx=0x7fce4c007a90 comp rx=0 tx=0).stop 2023-10-16T09:44:39.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.581+0000 7fce6a7df700 1 -- 172.21.15.181:0/685454327 shutdown_connections 2023-10-16T09:44:39.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.581+0000 7fce6a7df700 1 --2- 172.21.15.181:0/685454327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce4c00dc20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.581+0000 7fce6a7df700 1 -- 172.21.15.181:0/685454327 >> 172.21.15.181:0/685454327 conn(0x7fce640d04a0 msgr2=0x7fce640d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:39.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.581+0000 7fce6a7df700 1 -- 172.21.15.181:0/685454327 shutdown_connections 2023-10-16T09:44:39.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.581+0000 7fce6a7df700 1 -- 172.21.15.181:0/685454327 wait complete. 2023-10-16T09:44:39.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.582+0000 7fce6a7df700 1 Processor -- start 2023-10-16T09:44:39.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.582+0000 7fce6a7df700 1 -- start start 2023-10-16T09:44:39.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce6a7df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce640c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce6a7df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce64149230 0x7fce640c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce6a7df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce640c9870 0x7fce6415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce6a7df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce640c9cf0 con 0x7fce4c007420 2023-10-16T09:44:39.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce6a7df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce640c79c0 con 0x7fce64149230 2023-10-16T09:44:39.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce6a7df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce640c7ca0 con 0x7fce640c9870 2023-10-16T09:44:39.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce640c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce640c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37388/0 (socket says 172.21.15.181:37388) 2023-10-16T09:44:39.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 -- 172.21.15.181:0/3670621544 learned_addr learned my addr 172.21.15.181:0/3670621544 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:39.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce637fe700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce64149230 0x7fce640c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce68d7c700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce640c9870 0x7fce6415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 -- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce640c9870 msgr2=0x7fce6415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce640c9870 0x7fce6415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 -- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce64149230 msgr2=0x7fce640c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce64149230 0x7fce640c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 -- 172.21.15.181:0/3670621544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6415d060 con 0x7fce4c007420 2023-10-16T09:44:39.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce68d7c700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce640c9870 0x7fce6415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:39.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce63fff700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce640c9ef0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce5800f7f0 tx=0x7fce5800fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.583+0000 7fce60ff9700 1 -- 172.21.15.181:0/3670621544 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580052e0 con 0x7fce4c007420 2023-10-16T09:44:39.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.584+0000 7fce60ff9700 1 -- 172.21.15.181:0/3670621544 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58005440 con 0x7fce4c007420 2023-10-16T09:44:39.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.584+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6415d340 con 0x7fce4c007420 2023-10-16T09:44:39.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.584+0000 7fce60ff9700 1 -- 172.21.15.181:0/3670621544 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58003910 con 0x7fce4c007420 2023-10-16T09:44:39.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.584+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce640ca430 con 0x7fce4c007420 2023-10-16T09:44:39.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.584+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce640ca5c0 con 0x7fce4c007420 2023-10-16T09:44:39.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.585+0000 7fce60ff9700 1 -- 172.21.15.181:0/3670621544 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fce58003a70 con 0x7fce4c007420 2023-10-16T09:44:39.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.586+0000 7fce60ff9700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fce5006af00 0x7fce5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.586+0000 7fce60ff9700 1 -- 172.21.15.181:0/3670621544 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce58089c60 con 0x7fce4c007420 2023-10-16T09:44:39.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.586+0000 7fce637fe700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fce5006af00 0x7fce5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.586+0000 7fce637fe700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fce5006af00 0x7fce5006d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce640c41d0 tx=0x7fce54007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.590+0000 7fce60ff9700 1 -- 172.21.15.181:0/3670621544 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fce58054e80 con 0x7fce4c007420 2023-10-16T09:44:39.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.783+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce64067d20 con 0x7fce4c007420 2023-10-16T09:44:39.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.784+0000 7fce60ff9700 1 -- 172.21.15.181:0/3670621544 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fce580584a0 con 0x7fce4c007420 2023-10-16T09:44:39.786 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:39.787 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:39.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fce5006af00 msgr2=0x7fce5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fce5006af00 0x7fce5006d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce640c41d0 tx=0x7fce54007040 comp rx=0 tx=0).stop 2023-10-16T09:44:39.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 msgr2=0x7fce640c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce640c9ef0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce5800f7f0 tx=0x7fce5800fb00 comp rx=0 tx=0).stop 2023-10-16T09:44:39.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 shutdown_connections 2023-10-16T09:44:39.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fce5006af00 0x7fce5006d3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce640c9870 0x7fce6415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4c007420 0x7fce640c9ef0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 --2- 172.21.15.181:0/3670621544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce64149230 0x7fce640c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.786+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 >> 172.21.15.181:0/3670621544 conn(0x7fce640d04a0 msgr2=0x7fce640cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:39.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.787+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 shutdown_connections 2023-10-16T09:44:39.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.787+0000 7fce6a7df700 1 -- 172.21.15.181:0/3670621544 wait complete. 2023-10-16T09:44:39.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:39.802 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:39.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.923+0000 7f328c27a700 1 Processor -- start 2023-10-16T09:44:39.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.923+0000 7f328c27a700 1 -- start start 2023-10-16T09:44:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328c27a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840b2010 0x7f32840cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328c27a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32840ce680 0x7f32840ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328c27a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32840ce1f0 con 0x7f3284071be0 2023-10-16T09:44:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328c27a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32840ccda0 con 0x7f32840ce680 2023-10-16T09:44:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328c27a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840727e0 con 0x7f32840b2010 2023-10-16T09:44:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f3289815700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3284071be0 0x7f32840b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48478/0 (socket says 172.21.15.181:48478) 2023-10-16T09:44:39.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f3289815700 1 -- 172.21.15.181:0/446217042 learned_addr learned my addr 172.21.15.181:0/446217042 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:39.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328a016700 1 --2- 172.21.15.181:0/446217042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840b2010 0x7f32840cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328a817700 1 --2- 172.21.15.181:0/446217042 >> v2:172.21.15.190:3300/0 conn(0x7f32840ce680 0x7f32840ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f3289014700 1 -- 172.21.15.181:0/446217042 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616646650 0 0) 0x7f32840ce1f0 con 0x7f3284071be0 2023-10-16T09:44:39.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f3289014700 1 -- 172.21.15.181:0/446217042 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32700035b0 con 0x7f3284071be0 2023-10-16T09:44:39.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328a016700 1 -- 172.21.15.181:0/446217042 >> v2:172.21.15.190:3300/0 conn(0x7f32840ce680 msgr2=0x7f32840ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:39.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328a016700 1 --2- 172.21.15.181:0/446217042 >> v2:172.21.15.190:3300/0 conn(0x7f32840ce680 0x7f32840ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f3289014700 1 -- 172.21.15.181:0/446217042 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2700599573 0 0) 0x7f32700035b0 con 0x7f3284071be0 2023-10-16T09:44:39.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328a016700 1 -- 172.21.15.181:0/446217042 >> v1:172.21.15.181:6789/0 conn(0x7f3284071be0 legacy=0x7f32840b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.925+0000 7f328a016700 1 -- 172.21.15.181:0/446217042 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3284072ac0 con 0x7f32840b2010 2023-10-16T09:44:39.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.926+0000 7f328a016700 1 --2- 172.21.15.181:0/446217042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840b2010 0x7f32840cc6e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3280009840 tx=0x7f3280009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7840f456ffd0e9be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.928+0000 7f3289014700 1 -- 172.21.15.181:0/446217042 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274001670 con 0x7f32840b2010 2023-10-16T09:44:39.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.928+0000 7f3289014700 1 -- 172.21.15.181:0/446217042 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3280004030 con 0x7f32840b2010 2023-10-16T09:44:39.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.928+0000 7f3289014700 1 -- 172.21.15.181:0/446217042 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32800043c0 con 0x7f32840b2010 2023-10-16T09:44:39.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 -- 172.21.15.181:0/446217042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840b2010 msgr2=0x7f32840cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 --2- 172.21.15.181:0/446217042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840b2010 0x7f32840cc6e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3280009840 tx=0x7f3280009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:39.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 -- 172.21.15.181:0/446217042 shutdown_connections 2023-10-16T09:44:39.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 --2- 172.21.15.181:0/446217042 >> v2:172.21.15.190:3300/0 conn(0x7f32840ce680 0x7f32840ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 --2- 172.21.15.181:0/446217042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840b2010 0x7f32840cc6e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 -- 172.21.15.181:0/446217042 >> 172.21.15.181:0/446217042 conn(0x7f32840cd010 msgr2=0x7f32840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:39.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 -- 172.21.15.181:0/446217042 shutdown_connections 2023-10-16T09:44:39.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.929+0000 7f328c27a700 1 -- 172.21.15.181:0/446217042 wait complete. 2023-10-16T09:44:39.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 Processor -- start 2023-10-16T09:44:39.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 -- start start 2023-10-16T09:44:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3284071be0 0x7f32840c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32840b2010 0x7f32840c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840ce680 0x7f32840c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840d00a0 con 0x7f32840b2010 2023-10-16T09:44:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840c8050 con 0x7f3284071be0 2023-10-16T09:44:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.931+0000 7f328c27a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840c8860 con 0x7f32840ce680 2023-10-16T09:44:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32840b2010 0x7f32840c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32840b2010 0x7f32840c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37404/0 (socket says 172.21.15.181:37404) 2023-10-16T09:44:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 -- 172.21.15.181:0/2696579628 learned_addr learned my addr 172.21.15.181:0/2696579628 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f328a016700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3284071be0 0x7f32840c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f328a817700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840ce680 0x7f32840c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 -- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840ce680 msgr2=0x7f32840c1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840ce680 0x7f32840c1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 -- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3284071be0 msgr2=0x7f32840c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3284071be0 0x7f32840c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 -- 172.21.15.181:0/2696579628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800094a0 con 0x7f32840b2010 2023-10-16T09:44:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f328a817700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840ce680 0x7f32840c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f3289815700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32840b2010 0x7f32840c5e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f32740011e0 tx=0x7f327400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f327affd700 1 -- 172.21.15.181:0/2696579628 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274004c80 con 0x7f32840b2010 2023-10-16T09:44:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f327affd700 1 -- 172.21.15.181:0/2696579628 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3274004e00 con 0x7f32840b2010 2023-10-16T09:44:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32840c82a0 con 0x7f32840b2010 2023-10-16T09:44:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f327affd700 1 -- 172.21.15.181:0/2696579628 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274007310 con 0x7f32840b2010 2023-10-16T09:44:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.932+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32840ca220 con 0x7f32840b2010 2023-10-16T09:44:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.933+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32840cc0d0 con 0x7f32840b2010 2023-10-16T09:44:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.933+0000 7f327affd700 1 -- 172.21.15.181:0/2696579628 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3274007470 con 0x7f32840b2010 2023-10-16T09:44:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.934+0000 7f327affd700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f326806af00 0x7f326806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.934+0000 7f327affd700 1 -- 172.21.15.181:0/2696579628 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3274089380 con 0x7f32840b2010 2023-10-16T09:44:39.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.934+0000 7f328a016700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f326806af00 0x7f326806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:39.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.934+0000 7f328a016700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f326806af00 0x7f326806d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f32800090b0 tx=0x7f3280008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:39.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:39.938+0000 7f327affd700 1 -- 172.21.15.181:0/2696579628 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3274054430 con 0x7f32840b2010 2023-10-16T09:44:40.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.134+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32840b6510 con 0x7f32840b2010 2023-10-16T09:44:40.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.134+0000 7f327affd700 1 -- 172.21.15.181:0/2696579628 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3274057a50 con 0x7f32840b2010 2023-10-16T09:44:40.136 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:40.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f326806af00 msgr2=0x7f326806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f326806af00 0x7f326806d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f32800090b0 tx=0x7f3280008030 comp rx=0 tx=0).stop 2023-10-16T09:44:40.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32840b2010 msgr2=0x7f32840c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32840b2010 0x7f32840c5e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f32740011e0 tx=0x7f327400c9e0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 shutdown_connections 2023-10-16T09:44:40.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f326806af00 0x7f326806d3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32840ce680 0x7f32840c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3284071be0 0x7f32840c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 --2- 172.21.15.181:0/2696579628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32840b2010 0x7f32840c5e50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 >> 172.21.15.181:0/2696579628 conn(0x7f32840cd010 msgr2=0x7f32840cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:40.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 shutdown_connections 2023-10-16T09:44:40.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.136+0000 7f328c27a700 1 -- 172.21.15.181:0/2696579628 wait complete. 2023-10-16T09:44:40.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:40.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.272+0000 7fa026fe2700 1 Processor -- start 2023-10-16T09:44:40.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.272+0000 7fa026fe2700 1 -- start start 2023-10-16T09:44:40.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa026fe2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa02015b850 0x7fa0200665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa026fe2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa02015fee0 0x7fa0200c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa026fe2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02006e4b0 con 0x7fa0200c3d40 2023-10-16T09:44:40.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa026fe2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02006ed50 con 0x7fa02015fee0 2023-10-16T09:44:40.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa026fe2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0200cb0f0 con 0x7fa02015b850 2023-10-16T09:44:40.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa01ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa02015fee0 0x7fa0200c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa01ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa02015fee0 0x7fa0200c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37412/0 (socket says 172.21.15.181:37412) 2023-10-16T09:44:40.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa02557f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa0200c3d40 0x7fa02006f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48492/0 (socket says 172.21.15.181:48492) 2023-10-16T09:44:40.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.273+0000 7fa01ffff700 1 -- 172.21.15.181:0/3674530312 learned_addr learned my addr 172.21.15.181:0/3674530312 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:40.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01ffff700 1 -- 172.21.15.181:0/3674530312 >> v2:172.21.15.190:3300/0 conn(0x7fa02015b850 msgr2=0x7fa0200665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:40.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01ffff700 1 --2- 172.21.15.181:0/3674530312 >> v2:172.21.15.190:3300/0 conn(0x7fa02015b850 0x7fa0200665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01ffff700 1 -- 172.21.15.181:0/3674530312 >> v1:172.21.15.181:6789/0 conn(0x7fa0200c3d40 legacy=0x7fa02006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01f7fe700 1 -- 172.21.15.181:0/3674530312 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237794735 0 0) 0x7fa02006e4b0 con 0x7fa0200c3d40 2023-10-16T09:44:40.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01ffff700 1 -- 172.21.15.181:0/3674530312 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020066b80 con 0x7fa02015fee0 2023-10-16T09:44:40.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01ffff700 1 --2- 172.21.15.181:0/3674530312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015fee0 0x7fa0200c3800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0200726a0 tx=0x7fa00c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9553228a34b74040 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01f7fe700 1 -- 172.21.15.181:0/3674530312 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00b870 con 0x7fa02015fee0 2023-10-16T09:44:40.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01f7fe700 1 -- 172.21.15.181:0/3674530312 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00b9d0 con 0x7fa02015fee0 2023-10-16T09:44:40.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa01f7fe700 1 -- 172.21.15.181:0/3674530312 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00bd60 con 0x7fa02015fee0 2023-10-16T09:44:40.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa026fe2700 1 -- 172.21.15.181:0/3674530312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015fee0 msgr2=0x7fa0200c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa026fe2700 1 --2- 172.21.15.181:0/3674530312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015fee0 0x7fa0200c3800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0200726a0 tx=0x7fa00c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa026fe2700 1 -- 172.21.15.181:0/3674530312 shutdown_connections 2023-10-16T09:44:40.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa026fe2700 1 --2- 172.21.15.181:0/3674530312 >> v2:172.21.15.190:3300/0 conn(0x7fa02015b850 0x7fa0200665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa026fe2700 1 --2- 172.21.15.181:0/3674530312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015fee0 0x7fa0200c3800 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.274+0000 7fa026fe2700 1 -- 172.21.15.181:0/3674530312 >> 172.21.15.181:0/3674530312 conn(0x7fa0200cc730 msgr2=0x7fa0200ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:40.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.275+0000 7fa026fe2700 1 -- 172.21.15.181:0/3674530312 shutdown_connections 2023-10-16T09:44:40.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.275+0000 7fa026fe2700 1 -- 172.21.15.181:0/3674530312 wait complete. 2023-10-16T09:44:40.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.276+0000 7fa026fe2700 1 Processor -- start 2023-10-16T09:44:40.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.276+0000 7fa026fe2700 1 -- start start 2023-10-16T09:44:40.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0200c3d40 0x7fa0201f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015b850 0x7fa0201f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa02015fee0 0x7fa0200b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02015cc80 con 0x7fa02015b850 2023-10-16T09:44:40.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02015e3e0 con 0x7fa0200c3d40 2023-10-16T09:44:40.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201f1330 con 0x7fa02015fee0 2023-10-16T09:44:40.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015b850 0x7fa0201f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015b850 0x7fa0201f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37418/0 (socket says 172.21.15.181:37418) 2023-10-16T09:44:40.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 -- 172.21.15.181:0/1542557679 learned_addr learned my addr 172.21.15.181:0/1542557679 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:40.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa02557f700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa02015fee0 0x7fa0200b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 -- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa02015fee0 msgr2=0x7fa0200b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa024d7e700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0200c3d40 0x7fa0201f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa02015fee0 0x7fa0200b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 -- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0200c3d40 msgr2=0x7fa0201f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0200c3d40 0x7fa0201f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 -- 172.21.15.181:0/1542557679 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c009550 con 0x7fa02015b850 2023-10-16T09:44:40.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa02557f700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa02015fee0 0x7fa0200b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:40.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa024d7e700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0200c3d40 0x7fa0201f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:40.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01ffff700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015b850 0x7fa0201f3ce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000c00 tx=0x7fa00c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01d7fa700 1 -- 172.21.15.181:0/1542557679 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c004500 con 0x7fa02015b850 2023-10-16T09:44:40.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01d7fa700 1 -- 172.21.15.181:0/1542557679 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c0036a0 con 0x7fa02015b850 2023-10-16T09:44:40.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0201f1610 con 0x7fa02015b850 2023-10-16T09:44:40.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa01d7fa700 1 -- 172.21.15.181:0/1542557679 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c003a10 con 0x7fa02015b850 2023-10-16T09:44:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.277+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201f1bd0 con 0x7fa02015b850 2023-10-16T09:44:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.278+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0201f1d60 con 0x7fa02015b850 2023-10-16T09:44:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.279+0000 7fa01d7fa700 1 -- 172.21.15.181:0/1542557679 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa00c003b70 con 0x7fa02015b850 2023-10-16T09:44:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.279+0000 7fa01d7fa700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa00406af00 0x7fa00406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.279+0000 7fa01d7fa700 1 -- 172.21.15.181:0/1542557679 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa00c08ac10 con 0x7fa02015b850 2023-10-16T09:44:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.279+0000 7fa024d7e700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa00406af00 0x7fa00406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.280+0000 7fa024d7e700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa00406af00 0x7fa00406d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0201ee9b0 tx=0x7fa014008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.284+0000 7fa01d7fa700 1 -- 172.21.15.181:0/1542557679 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa00c055d50 con 0x7fa02015b850 2023-10-16T09:44:40.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.475+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0200cd5d0 con 0x7fa02015b850 2023-10-16T09:44:40.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.476+0000 7fa01d7fa700 1 -- 172.21.15.181:0/1542557679 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa00c0200a0 con 0x7fa02015b850 2023-10-16T09:44:40.478 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa00406af00 msgr2=0x7fa00406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa00406af00 0x7fa00406d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0201ee9b0 tx=0x7fa014008040 comp rx=0 tx=0).stop 2023-10-16T09:44:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015b850 msgr2=0x7fa0201f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015b850 0x7fa0201f3ce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000c00 tx=0x7fa00c004750 comp rx=0 tx=0).stop 2023-10-16T09:44:40.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 shutdown_connections 2023-10-16T09:44:40.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa00406af00 0x7fa00406d3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa02015fee0 0x7fa0200b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0200c3d40 0x7fa0201f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 --2- 172.21.15.181:0/1542557679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa02015b850 0x7fa0201f3ce0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 >> 172.21.15.181:0/1542557679 conn(0x7fa0200cc730 msgr2=0x7fa0200c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:40.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 shutdown_connections 2023-10-16T09:44:40.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.478+0000 7fa026fe2700 1 -- 172.21.15.181:0/1542557679 wait complete. 2023-10-16T09:44:40.492 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.614+0000 7f0586e6e700 1 Processor -- start 2023-10-16T09:44:40.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.614+0000 7f0586e6e700 1 -- start start 2023-10-16T09:44:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f0586e6e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 0x7f05800ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f0586e6e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05800ce6a0 0x7f058006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f0586e6e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800cef00 con 0x7f0580072760 2023-10-16T09:44:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f0586e6e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05800cec20 con 0x7f05800ce6a0 2023-10-16T09:44:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f0586e6e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0580071c80 con 0x7f058015da70 2023-10-16T09:44:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f058540b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0580072760 0x7f05801615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48506/0 (socket says 172.21.15.181:48506) 2023-10-16T09:44:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f058540b700 1 -- 172.21.15.181:0/1377835529 learned_addr learned my addr 172.21.15.181:0/1377835529 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f0584c0a700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 0x7f05800ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f057ffff700 1 --2- 172.21.15.181:0/1377835529 >> v2:172.21.15.190:3300/0 conn(0x7f05800ce6a0 0x7f058006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115501463 0 0) 0x7f05800cef00 con 0x7f0580072760 2023-10-16T09:44:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05700035b0 con 0x7f0580072760 2023-10-16T09:44:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572388709 0 0) 0x7f05700035b0 con 0x7f0580072760 2023-10-16T09:44:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 msgr2=0x7f05800ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.615+0000 7f057f7fe700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 0x7f05800ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 >> v2:172.21.15.190:3300/0 conn(0x7f05800ce6a0 msgr2=0x7f058006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 --2- 172.21.15.181:0/1377835529 >> v2:172.21.15.190:3300/0 conn(0x7f05800ce6a0 0x7f058006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0580071f60 con 0x7f0580072760 2023-10-16T09:44:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f05780019f0 con 0x7f0580072760 2023-10-16T09:44:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057ffff700 1 --2- 172.21.15.181:0/1377835529 >> v2:172.21.15.190:3300/0 conn(0x7f05800ce6a0 0x7f058006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 >> v1:172.21.15.181:6789/0 conn(0x7f0580072760 legacy=0x7f05801615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0570003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 0x7f0570004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0570004d90 0x7f0570005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800cec20 con 0x7f0570003a00 2023-10-16T09:44:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0580071c80 con 0x7f0570004d90 2023-10-16T09:44:40.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057000ba90 con 0x7f058015da70 2023-10-16T09:44:40.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f0584c0a700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0570003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057ffff700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 0x7f0570004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f058540b700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0570004d90 0x7f0570005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f0584c0a700 1 -- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 msgr2=0x7f0570004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:40.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f0584c0a700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015da70 0x7f0570004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f0584c0a700 1 -- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0570004d90 msgr2=0x7f0570005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:40.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f0584c0a700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0570004d90 0x7f0570005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f0584c0a700 1 -- 172.21.15.181:0/1377835529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700035b0 con 0x7f0570003a00 2023-10-16T09:44:40.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f058540b700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0570004d90 0x7f0570005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:40.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f0584c0a700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0570003e20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0574009b40 tx=0x7f0570005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5856978fd8dda2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400e6c0 con 0x7f0570003a00 2023-10-16T09:44:40.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057400e820 con 0x7f0570003a00 2023-10-16T09:44:40.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.616+0000 7f057f7fe700 1 -- 172.21.15.181:0/1377835529 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400ebd0 con 0x7f0570003a00 2023-10-16T09:44:40.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.617+0000 7f0586e6e700 1 -- 172.21.15.181:0/1377835529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 msgr2=0x7f0570003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.617+0000 7f0586e6e700 1 --2- 172.21.15.181:0/1377835529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0570003e20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0574009b40 tx=0x7f0570005880 comp rx=0 tx=0).stop 2023-10-16T09:44:40.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.617+0000 7f058540b700 1 -- 172.21.15.181:0/1377835529 reap_dead start 2023-10-16T09:44:40.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.617+0000 7f0586e6e700 1 -- 172.21.15.181:0/1377835529 shutdown_connections 2023-10-16T09:44:40.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.617+0000 7f0586e6e700 1 -- 172.21.15.181:0/1377835529 >> 172.21.15.181:0/1377835529 conn(0x7f05800cd010 msgr2=0x7f05800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:40.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.617+0000 7f0586e6e700 1 -- 172.21.15.181:0/1377835529 shutdown_connections 2023-10-16T09:44:40.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.617+0000 7f0586e6e700 1 -- 172.21.15.181:0/1377835529 wait complete. 2023-10-16T09:44:40.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.618+0000 7f0586e6e700 1 Processor -- start 2023-10-16T09:44:40.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.618+0000 7f0586e6e700 1 -- start start 2023-10-16T09:44:40.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0586e6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0580157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0586e6e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05800c6390 0x7f05800b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0586e6e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800b26e0 0x7f05800c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0586e6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800c9660 con 0x7f0570003a00 2023-10-16T09:44:40.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0586e6e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800b2b60 con 0x7f05800c6390 2023-10-16T09:44:40.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0586e6e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0580157cb0 con 0x7f05800b26e0 2023-10-16T09:44:40.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0580157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0580157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37436/0 (socket says 172.21.15.181:37436) 2023-10-16T09:44:40.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 -- 172.21.15.181:0/3243023519 learned_addr learned my addr 172.21.15.181:0/3243023519 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:40.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f058540b700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800b26e0 0x7f05800c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f057ffff700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05800c6390 0x7f05800b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 -- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800b26e0 msgr2=0x7f05800c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800b26e0 0x7f05800c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 -- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05800c6390 msgr2=0x7f05800b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05800c6390 0x7f05800b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.619+0000 7f0584c0a700 1 -- 172.21.15.181:0/3243023519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800c9380 con 0x7f0570003a00 2023-10-16T09:44:40.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.620+0000 7f0584c0a700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0580157770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05700047d0 tx=0x7f05740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.620+0000 7f057d7fa700 1 -- 172.21.15.181:0/3243023519 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574021070 con 0x7f0570003a00 2023-10-16T09:44:40.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.620+0000 7f057d7fa700 1 -- 172.21.15.181:0/3243023519 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574013980 con 0x7f0570003a00 2023-10-16T09:44:40.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.620+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800c58b0 con 0x7f0570003a00 2023-10-16T09:44:40.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.620+0000 7f057d7fa700 1 -- 172.21.15.181:0/3243023519 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400e6c0 con 0x7f0570003a00 2023-10-16T09:44:40.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.620+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05800c5e70 con 0x7f0570003a00 2023-10-16T09:44:40.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.621+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05800c5710 con 0x7f0570003a00 2023-10-16T09:44:40.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.621+0000 7f057d7fa700 1 -- 172.21.15.181:0/3243023519 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0574025070 con 0x7f0570003a00 2023-10-16T09:44:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.622+0000 7f057d7fa700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f056406af00 0x7f056406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.622+0000 7f057d7fa700 1 -- 172.21.15.181:0/3243023519 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f057408a950 con 0x7f0570003a00 2023-10-16T09:44:40.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.622+0000 7f057ffff700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f056406af00 0x7f056406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.622+0000 7f057ffff700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f056406af00 0x7f056406d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0580158420 tx=0x7f056c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.626+0000 7f057d7fa700 1 -- 172.21.15.181:0/3243023519 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f05800c5710 con 0x7f0570003a00 2023-10-16T09:44:40.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.820+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f05800c5710 con 0x7f0570003a00 2023-10-16T09:44:40.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.820+0000 7f057d7fa700 1 -- 172.21.15.181:0/3243023519 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f05800c5710 con 0x7f0570003a00 2023-10-16T09:44:40.823 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:40.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f056406af00 msgr2=0x7f056406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f056406af00 0x7f056406d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0580158420 tx=0x7f056c008040 comp rx=0 tx=0).stop 2023-10-16T09:44:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 msgr2=0x7f0580157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0580157770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05700047d0 tx=0x7f05740047c0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 shutdown_connections 2023-10-16T09:44:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f056406af00 0x7f056406d3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800b26e0 0x7f05800c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570003a00 0x7f0580157770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.822+0000 7f0586e6e700 1 --2- 172.21.15.181:0/3243023519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05800c6390 0x7f05800b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.823+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 >> 172.21.15.181:0/3243023519 conn(0x7f05800cd010 msgr2=0x7f05800b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:40.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.823+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 shutdown_connections 2023-10-16T09:44:40.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.823+0000 7f0586e6e700 1 -- 172.21.15.181:0/3243023519 wait complete. 2023-10-16T09:44:40.837 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:44:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.958+0000 7f28372b1700 1 Processor -- start 2023-10-16T09:44:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.958+0000 7f28372b1700 1 -- start start 2023-10-16T09:44:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f28372b1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f283015df60 0x7f28300cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f28372b1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f283006a8a0 0x7f283006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f28372b1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28300cddf0 con 0x7f28300d0fc0 2023-10-16T09:44:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f28372b1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283006b290 con 0x7f283015df60 2023-10-16T09:44:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f28372b1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28300af9c0 con 0x7f283006a8a0 2023-10-16T09:44:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283504d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f283015df60 0x7f28300cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283484c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f28300d0fc0 0x7f28300cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48522/0 (socket says 172.21.15.181:48522) 2023-10-16T09:44:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283484c700 1 -- 172.21.15.181:0/3449203541 learned_addr learned my addr 172.21.15.181:0/3449203541 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283584e700 1 --2- 172.21.15.181:0/3449203541 >> v2:172.21.15.190:3300/0 conn(0x7f283006a8a0 0x7f283006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283504d700 1 -- 172.21.15.181:0/3449203541 >> v2:172.21.15.190:3300/0 conn(0x7f283006a8a0 msgr2=0x7f283006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283504d700 1 --2- 172.21.15.181:0/3449203541 >> v2:172.21.15.190:3300/0 conn(0x7f283006a8a0 0x7f283006ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283504d700 1 -- 172.21.15.181:0/3449203541 >> v1:172.21.15.181:6789/0 conn(0x7f28300d0fc0 legacy=0x7f28300cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f2827fff700 1 -- 172.21.15.181:0/3449203541 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966787281 0 0) 0x7f28300cddf0 con 0x7f28300d0fc0 2023-10-16T09:44:40.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.959+0000 7f283504d700 1 -- 172.21.15.181:0/3449203541 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300ccf50 con 0x7f283015df60 2023-10-16T09:44:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f283504d700 1 --2- 172.21.15.181:0/3449203541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f283015df60 0x7f28300cae20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f283006f9e0 tx=0x7f2820009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c699834ff69df96 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f2827fff700 1 -- 172.21.15.181:0/3449203541 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820005e30 con 0x7f283015df60 2023-10-16T09:44:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f2827fff700 1 -- 172.21.15.181:0/3449203541 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2820006030 con 0x7f283015df60 2023-10-16T09:44:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f2827fff700 1 -- 172.21.15.181:0/3449203541 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28200063c0 con 0x7f283015df60 2023-10-16T09:44:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 -- 172.21.15.181:0/3449203541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f283015df60 msgr2=0x7f28300cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 --2- 172.21.15.181:0/3449203541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f283015df60 0x7f28300cae20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f283006f9e0 tx=0x7f2820009b00 comp rx=0 tx=0).stop 2023-10-16T09:44:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 -- 172.21.15.181:0/3449203541 shutdown_connections 2023-10-16T09:44:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 --2- 172.21.15.181:0/3449203541 >> v2:172.21.15.190:3300/0 conn(0x7f283006a8a0 0x7f283006ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 --2- 172.21.15.181:0/3449203541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f283015df60 0x7f28300cae20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 -- 172.21.15.181:0/3449203541 >> 172.21.15.181:0/3449203541 conn(0x7f28300d04a0 msgr2=0x7f28300d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 -- 172.21.15.181:0/3449203541 shutdown_connections 2023-10-16T09:44:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.960+0000 7f28372b1700 1 -- 172.21.15.181:0/3449203541 wait complete. 2023-10-16T09:44:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 Processor -- start 2023-10-16T09:44:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 -- start start 2023-10-16T09:44:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f283006a8a0 0x7f28301598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28300d0fc0 0x7f28301f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f283015df60 0x7f28301f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300b57f0 con 0x7f28300d0fc0 2023-10-16T09:44:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300b5fc0 con 0x7f283015df60 2023-10-16T09:44:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f28372b1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2830159de0 con 0x7f283006a8a0 2023-10-16T09:44:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f283484c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28300d0fc0 0x7f28301f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f283484c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28300d0fc0 0x7f28301f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37450/0 (socket says 172.21.15.181:37450) 2023-10-16T09:44:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.962+0000 7f283484c700 1 -- 172.21.15.181:0/3536344082 learned_addr learned my addr 172.21.15.181:0/3536344082 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283504d700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f283006a8a0 0x7f28301598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283584e700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f283015df60 0x7f28301f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283484c700 1 -- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f283006a8a0 msgr2=0x7f28301598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283484c700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f283006a8a0 0x7f28301598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283484c700 1 -- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f283015df60 msgr2=0x7f28301f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:40.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283484c700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f283015df60 0x7f28301f3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:40.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283484c700 1 -- 172.21.15.181:0/3536344082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820009430 con 0x7f28300d0fc0 2023-10-16T09:44:40.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283504d700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f283006a8a0 0x7f28301598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:40.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f283484c700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28300d0fc0 0x7f28301f2c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f282c0011e0 tx=0x7f282c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f2825ffb700 1 -- 172.21.15.181:0/3536344082 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c004c80 con 0x7f28300d0fc0 2023-10-16T09:44:40.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f2825ffb700 1 -- 172.21.15.181:0/3536344082 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c004e00 con 0x7f28300d0fc0 2023-10-16T09:44:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28300cfa30 con 0x7f28300d0fc0 2023-10-16T09:44:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f2825ffb700 1 -- 172.21.15.181:0/3536344082 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c007310 con 0x7f28300d0fc0 2023-10-16T09:44:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.963+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28300b5ce0 con 0x7f28300d0fc0 2023-10-16T09:44:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.964+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2830067d20 con 0x7f28300d0fc0 2023-10-16T09:44:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.964+0000 7f2825ffb700 1 -- 172.21.15.181:0/3536344082 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f282c007470 con 0x7f28300d0fc0 2023-10-16T09:44:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.965+0000 7f2825ffb700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f281406af00 0x7f281406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.965+0000 7f2825ffb700 1 -- 172.21.15.181:0/3536344082 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f282c0893f0 con 0x7f28300d0fc0 2023-10-16T09:44:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.965+0000 7f283504d700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f281406af00 0x7f281406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.965+0000 7f283504d700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f281406af00 0x7f281406d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2820013040 tx=0x7f2820008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:40.969+0000 7f2825ffb700 1 -- 172.21.15.181:0/3536344082 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f282c0544a0 con 0x7f28300d0fc0 2023-10-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.179+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28300ca7d0 con 0x7f28300d0fc0 2023-10-16T09:44:41.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.179+0000 7f2825ffb700 1 -- 172.21.15.181:0/3536344082 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f282c057ac0 con 0x7f28300d0fc0 2023-10-16T09:44:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f281406af00 msgr2=0x7f281406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f281406af00 0x7f281406d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2820013040 tx=0x7f2820008070 comp rx=0 tx=0).stop 2023-10-16T09:44:41.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28300d0fc0 msgr2=0x7f28301f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28300d0fc0 0x7f28301f2c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f282c0011e0 tx=0x7f282c00cae0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 shutdown_connections 2023-10-16T09:44:41.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f281406af00 0x7f281406d3c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f283006a8a0 0x7f28301598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f283015df60 0x7f28301f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 --2- 172.21.15.181:0/3536344082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28300d0fc0 0x7f28301f2c10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 >> 172.21.15.181:0/3536344082 conn(0x7f28300d04a0 msgr2=0x7f283015c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:41.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 shutdown_connections 2023-10-16T09:44:41.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.182+0000 7f28372b1700 1 -- 172.21.15.181:0/3536344082 wait complete. 2023-10-16T09:44:41.187 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2023-10-16T09:44:41.196 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-16T09:44:41.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:41.197 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-16T09:44:41.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T09:44:41.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:44:41.238 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T09:44:41.297 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T09:44:41.297 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:41.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.416+0000 7fafa05c4700 1 Processor -- start 2023-10-16T09:44:41.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.416+0000 7fafa05c4700 1 -- start start 2023-10-16T09:44:41.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7fafa05c4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf980d0370 0x7faf980ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7fafa05c4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf980cd2d0 0x7faf980d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7fafa05c4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf980cc510 con 0x7faf9815b990 2023-10-16T09:44:41.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7fafa05c4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf980d24f0 con 0x7faf980cd2d0 2023-10-16T09:44:41.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7fafa05c4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf980cc7f0 con 0x7faf980d0370 2023-10-16T09:44:41.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7faf9eb61700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf980cd2d0 0x7faf980d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7faf9e360700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faf9815b990 0x7faf980cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48534/0 (socket says 172.21.15.181:48534) 2023-10-16T09:44:41.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.417+0000 7faf9e360700 1 -- 172.21.15.181:0/2928689458 learned_addr learned my addr 172.21.15.181:0/2928689458 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:41.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9db5f700 1 --2- 172.21.15.181:0/2928689458 >> v2:172.21.15.190:3300/0 conn(0x7faf980d0370 0x7faf980ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9eb61700 1 -- 172.21.15.181:0/2928689458 >> v2:172.21.15.190:3300/0 conn(0x7faf980d0370 msgr2=0x7faf980ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:41.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9eb61700 1 --2- 172.21.15.181:0/2928689458 >> v2:172.21.15.190:3300/0 conn(0x7faf980d0370 0x7faf980ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9eb61700 1 -- 172.21.15.181:0/2928689458 >> v1:172.21.15.181:6789/0 conn(0x7faf9815b990 legacy=0x7faf980cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9d35e700 1 -- 172.21.15.181:0/2928689458 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545117702 0 0) 0x7faf980cc510 con 0x7faf9815b990 2023-10-16T09:44:41.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9eb61700 1 -- 172.21.15.181:0/2928689458 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980ccad0 con 0x7faf980cd2d0 2023-10-16T09:44:41.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9db5f700 1 --2- 172.21.15.181:0/2928689458 >> v2:172.21.15.190:3300/0 conn(0x7faf980d0370 0x7faf980ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:41.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9eb61700 1 --2- 172.21.15.181:0/2928689458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980cd2d0 0x7faf980d1fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf980b4230 tx=0x7faf9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3341ce417d91db4f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:41.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9d35e700 1 -- 172.21.15.181:0/2928689458 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9000e070 con 0x7faf980cd2d0 2023-10-16T09:44:41.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9d35e700 1 -- 172.21.15.181:0/2928689458 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9000bbd0 con 0x7faf980cd2d0 2023-10-16T09:44:41.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.418+0000 7faf9d35e700 1 -- 172.21.15.181:0/2928689458 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90012430 con 0x7faf980cd2d0 2023-10-16T09:44:41.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 -- 172.21.15.181:0/2928689458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980cd2d0 msgr2=0x7faf980d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 --2- 172.21.15.181:0/2928689458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980cd2d0 0x7faf980d1fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf980b4230 tx=0x7faf9000b330 comp rx=0 tx=0).stop 2023-10-16T09:44:41.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 -- 172.21.15.181:0/2928689458 shutdown_connections 2023-10-16T09:44:41.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 --2- 172.21.15.181:0/2928689458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980cd2d0 0x7faf980d1fb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 --2- 172.21.15.181:0/2928689458 >> v2:172.21.15.190:3300/0 conn(0x7faf980d0370 0x7faf980ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 -- 172.21.15.181:0/2928689458 >> 172.21.15.181:0/2928689458 conn(0x7faf9809a9a0 msgr2=0x7faf980d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:41.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 -- 172.21.15.181:0/2928689458 shutdown_connections 2023-10-16T09:44:41.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.419+0000 7fafa05c4700 1 -- 172.21.15.181:0/2928689458 wait complete. 2023-10-16T09:44:41.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.420+0000 7fafa05c4700 1 Processor -- start 2023-10-16T09:44:41.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.420+0000 7fafa05c4700 1 -- start start 2023-10-16T09:44:41.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf980cd2d0 0x7faf98151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980d0370 0x7faf9815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9815b990 0x7faf980c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b53d0 con 0x7faf980d0370 2023-10-16T09:44:41.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980c5920 con 0x7faf980cd2d0 2023-10-16T09:44:41.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980c4ff0 con 0x7faf9815b990 2023-10-16T09:44:41.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980d0370 0x7faf9815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980d0370 0x7faf9815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37464/0 (socket says 172.21.15.181:37464) 2023-10-16T09:44:41.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 -- 172.21.15.181:0/975007760 learned_addr learned my addr 172.21.15.181:0/975007760 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:41.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9e360700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf980cd2d0 0x7faf98151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9eb61700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9815b990 0x7faf980c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 -- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9815b990 msgr2=0x7faf980c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9815b990 0x7faf980c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 -- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf980cd2d0 msgr2=0x7faf98151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf980cd2d0 0x7faf98151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 -- 172.21.15.181:0/975007760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90009a70 con 0x7faf980d0370 2023-10-16T09:44:41.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf9db5f700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980d0370 0x7faf9815cc60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf9400c7f0 tx=0x7faf9400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:41.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf8effd700 1 -- 172.21.15.181:0/975007760 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94011800 con 0x7faf980d0370 2023-10-16T09:44:41.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf8effd700 1 -- 172.21.15.181:0/975007760 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94011960 con 0x7faf980d0370 2023-10-16T09:44:41.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf980c52d0 con 0x7faf980d0370 2023-10-16T09:44:41.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7faf8effd700 1 -- 172.21.15.181:0/975007760 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94011cf0 con 0x7faf980d0370 2023-10-16T09:44:41.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.421+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf980b6920 con 0x7faf980d0370 2023-10-16T09:44:41.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.422+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf98067d20 con 0x7faf980d0370 2023-10-16T09:44:41.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.423+0000 7faf8effd700 1 -- 172.21.15.181:0/975007760 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faf94018360 con 0x7faf980d0370 2023-10-16T09:44:41.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.423+0000 7faf8effd700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7faf7c06af00 0x7faf7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.423+0000 7faf8effd700 1 -- 172.21.15.181:0/975007760 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf94089450 con 0x7faf980d0370 2023-10-16T09:44:41.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.423+0000 7faf9e360700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7faf7c06af00 0x7faf7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.424+0000 7faf9e360700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7faf7c06af00 0x7faf7c06d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faf981f4340 tx=0x7faf88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:41.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.428+0000 7faf8effd700 1 -- 172.21.15.181:0/975007760 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faf94054500 con 0x7faf980d0370 2023-10-16T09:44:41.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.660+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf980cfa40 con 0x7faf980d0370 2023-10-16T09:44:41.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.660+0000 7faf8effd700 1 -- 172.21.15.181:0/975007760 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7faf94057b20 con 0x7faf980d0370 2023-10-16T09:44:41.663 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:41.663 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:41.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7faf7c06af00 msgr2=0x7faf7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7faf7c06af00 0x7faf7c06d3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faf981f4340 tx=0x7faf88009040 comp rx=0 tx=0).stop 2023-10-16T09:44:41.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980d0370 msgr2=0x7faf9815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980d0370 0x7faf9815cc60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf9400c7f0 tx=0x7faf9400cb00 comp rx=0 tx=0).stop 2023-10-16T09:44:41.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 shutdown_connections 2023-10-16T09:44:41.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7faf7c06af00 0x7faf7c06d3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf9815b990 0x7faf980c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf980cd2d0 0x7faf98151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 --2- 172.21.15.181:0/975007760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf980d0370 0x7faf9815cc60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 >> 172.21.15.181:0/975007760 conn(0x7faf9809a9a0 msgr2=0x7faf980d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:41.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 shutdown_connections 2023-10-16T09:44:41.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.662+0000 7fafa05c4700 1 -- 172.21.15.181:0/975007760 wait complete. 2023-10-16T09:44:41.668 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:41.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:41.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.795+0000 7f4ef4fb2700 1 Processor -- start 2023-10-16T09:44:41.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.795+0000 7f4ef4fb2700 1 -- start start 2023-10-16T09:44:41.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4ef4fb2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 0x7f4ef00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4ef4fb2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef006f870 con 0x7f4ef00cd470 2023-10-16T09:44:41.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4ef4fb2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef00d1160 con 0x7f4ef00af450 2023-10-16T09:44:41.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4ef4fb2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00cc8e0 con 0x7f4ef015dfd0 2023-10-16T09:44:41.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eeed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4ef00cd470 0x7f4ef006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48548/0 (socket says 172.21.15.181:48548) 2023-10-16T09:44:41.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eeed9d700 1 -- 172.21.15.181:0/1832648893 learned_addr learned my addr 172.21.15.181:0/1832648893 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:41.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534204638 0 0) 0x7f4ef006f870 con 0x7f4ef00cd470 2023-10-16T09:44:41.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eee59c700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 0x7f4ef00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed80035b0 con 0x7f4ef00cd470 2023-10-16T09:44:41.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850474778 0 0) 0x7f4ed80035b0 con 0x7f4ef00cd470 2023-10-16T09:44:41.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 msgr2=0x7f4ef00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:41.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 0x7f4ef00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 >> v1:172.21.15.190:6789/0 conn(0x7f4ef00af450 legacy=0x7f4ef006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00ccbc0 con 0x7f4ef00cd470 2023-10-16T09:44:41.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4ee80019f0 con 0x7f4ef00cd470 2023-10-16T09:44:41.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 >> v1:172.21.15.181:6789/0 conn(0x7f4ef00cd470 legacy=0x7f4ef006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 0x7f4ed8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.796+0000 7f4eed59a700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed8003fb0 0x7f4ed8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eed59a700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ed8004d90 0x7f4ed8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00cc8e0 con 0x7f4ed8003fb0 2023-10-16T09:44:41.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800bba0 con 0x7f4ed8004d90 2023-10-16T09:44:41.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800bd60 con 0x7f4ef015dfd0 2023-10-16T09:44:41.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eee59c700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 0x7f4ed8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eedd9b700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed8003fb0 0x7f4ed8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eeed9d700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ed8004d90 0x7f4ed8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eedd9b700 1 -- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 msgr2=0x7f4ed8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:41.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eedd9b700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ef015dfd0 0x7f4ed8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eedd9b700 1 -- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ed8004d90 msgr2=0x7f4ed8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:41.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eedd9b700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ed8004d90 0x7f4ed8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eedd9b700 1 -- 172.21.15.181:0/1832648893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80035b0 con 0x7f4ed8003fb0 2023-10-16T09:44:41.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eedd9b700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed8003fb0 0x7f4ed8004850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40017c0 tx=0x7f4ed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2fdf54bb21263d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:41.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40052d0 con 0x7f4ed8003fb0 2023-10-16T09:44:41.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4005430 con 0x7f4ed8003fb0 2023-10-16T09:44:41.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.797+0000 7f4eed59a700 1 -- 172.21.15.181:0/1832648893 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40038c0 con 0x7f4ed8003fb0 2023-10-16T09:44:41.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.798+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/1832648893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed8003fb0 msgr2=0x7f4ed8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.798+0000 7f4ef4fb2700 1 --2- 172.21.15.181:0/1832648893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ed8003fb0 0x7f4ed8004850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40017c0 tx=0x7f4ed8005880 comp rx=0 tx=0).stop 2023-10-16T09:44:41.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.798+0000 7f4eeed9d700 1 -- 172.21.15.181:0/1832648893 reap_dead start 2023-10-16T09:44:41.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.798+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/1832648893 shutdown_connections 2023-10-16T09:44:41.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.798+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/1832648893 >> 172.21.15.181:0/1832648893 conn(0x7f4ef00d04a0 msgr2=0x7f4ef00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:41.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.798+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/1832648893 shutdown_connections 2023-10-16T09:44:41.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.798+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/1832648893 wait complete. 2023-10-16T09:44:41.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.799+0000 7f4ef4fb2700 1 Processor -- start 2023-10-16T09:44:41.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4ef4fb2700 1 -- start start 2023-10-16T09:44:41.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4ef4fb2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ed8003fb0 0x7f4ef014d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4ef4fb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ef014de70 0x7f4ef00c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4ef4fb2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ef00ce3a0 0x7f4ef00ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4ef4fb2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00ced60 con 0x7f4ef014de70 2023-10-16T09:44:41.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4ef4fb2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00c6e00 con 0x7f4ef00ce3a0 2023-10-16T09:44:41.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4ef4fb2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00c5d60 con 0x7f4ed8003fb0 2023-10-16T09:44:41.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4eedd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ef014de70 0x7f4ef00c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4eedd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ef014de70 0x7f4ef00c54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37482/0 (socket says 172.21.15.181:37482) 2023-10-16T09:44:41.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4eedd9b700 1 -- 172.21.15.181:0/3875232251 learned_addr learned my addr 172.21.15.181:0/3875232251 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:41.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4eee59c700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ed8003fb0 0x7f4ef014d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.800+0000 7f4eeed9d700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ef00ce3a0 0x7f4ef00ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4eedd9b700 1 -- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ed8003fb0 msgr2=0x7f4ef014d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4eedd9b700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ed8003fb0 0x7f4ef014d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4eedd9b700 1 -- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ef00ce3a0 msgr2=0x7f4ef00ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:41.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4eedd9b700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ef00ce3a0 0x7f4ef00ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:41.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4eedd9b700 1 -- 172.21.15.181:0/3875232251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00c6040 con 0x7f4ef014de70 2023-10-16T09:44:41.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4eedd9b700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ef014de70 0x7f4ef00c54f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4006a90 tx=0x7f4ee4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:41.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4edeffd700 1 -- 172.21.15.181:0/3875232251 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4007240 con 0x7f4ef014de70 2023-10-16T09:44:41.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4edeffd700 1 -- 172.21.15.181:0/3875232251 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee40073a0 con 0x7f4ef014de70 2023-10-16T09:44:41.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef00c6320 con 0x7f4ef014de70 2023-10-16T09:44:41.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4edeffd700 1 -- 172.21.15.181:0/3875232251 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4001290 con 0x7f4ef014de70 2023-10-16T09:44:41.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.801+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef00c68e0 con 0x7f4ef014de70 2023-10-16T09:44:41.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.802+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef00c6aa0 con 0x7f4ef014de70 2023-10-16T09:44:41.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.803+0000 7f4edeffd700 1 -- 172.21.15.181:0/3875232251 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4ee4003680 con 0x7f4ef014de70 2023-10-16T09:44:41.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.803+0000 7f4edeffd700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4ed006af00 0x7f4ed006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:41.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.804+0000 7f4edeffd700 1 -- 172.21.15.181:0/3875232251 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ee408a760 con 0x7f4ef014de70 2023-10-16T09:44:41.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.804+0000 7f4eee59c700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4ed006af00 0x7f4ed006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:41.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.804+0000 7f4eee59c700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4ed006af00 0x7f4ed006d3c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009c40 tx=0x7f4ee00091c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:41.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:41.808+0000 7f4edeffd700 1 -- 172.21.15.181:0/3875232251 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4ee4055a40 con 0x7f4ef014de70 2023-10-16T09:44:42.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.002+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ef00ced60 con 0x7f4ef014de70 2023-10-16T09:44:42.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.003+0000 7f4edeffd700 1 -- 172.21.15.181:0/3875232251 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f4ee4059060 con 0x7f4ef014de70 2023-10-16T09:44:42.006 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:42.006 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:42.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.005+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4ed006af00 msgr2=0x7f4ed006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.005+0000 7f4ef4fb2700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4ed006af00 0x7f4ed006d3c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009c40 tx=0x7f4ee00091c0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ef014de70 msgr2=0x7f4ef00c54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ef014de70 0x7f4ef00c54f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4006a90 tx=0x7f4ee4001120 comp rx=0 tx=0).stop 2023-10-16T09:44:42.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 shutdown_connections 2023-10-16T09:44:42.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f4ed006af00 0x7f4ed006d3c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ed8003fb0 0x7f4ef014d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ef00ce3a0 0x7f4ef00ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 --2- 172.21.15.181:0/3875232251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ef014de70 0x7f4ef00c54f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 >> 172.21.15.181:0/3875232251 conn(0x7f4ef00d04a0 msgr2=0x7f4ef006d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:42.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 shutdown_connections 2023-10-16T09:44:42.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.006+0000 7f4ef4fb2700 1 -- 172.21.15.181:0/3875232251 wait complete. 2023-10-16T09:44:42.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:43:59.531648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:00.138849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:42.021 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:42.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.142+0000 7fcb37999700 1 Processor -- start 2023-10-16T09:44:42.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.143+0000 7fcb37999700 1 -- start start 2023-10-16T09:44:42.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.143+0000 7fcb37999700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb300cc6e0 0x7fcb30071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.143+0000 7fcb37999700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb300ce1f0 con 0x7fcb300ce680 2023-10-16T09:44:42.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.143+0000 7fcb37999700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb300ccda0 con 0x7fcb300b2010 2023-10-16T09:44:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.143+0000 7fcb37999700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb300ccb90 con 0x7fcb300cc6e0 2023-10-16T09:44:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb34f34700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb300cc6e0 0x7fcb30071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb34f34700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb300cc6e0 0x7fcb30071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37490/0 (socket says 172.21.15.181:37490) 2023-10-16T09:44:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb35f36700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcb300ce680 0x7fcb300ceaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48570/0 (socket says 172.21.15.181:48570) 2023-10-16T09:44:42.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb34f34700 1 -- 172.21.15.181:0/2075804744 learned_addr learned my addr 172.21.15.181:0/2075804744 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb34f34700 1 -- 172.21.15.181:0/2075804744 >> v1:172.21.15.190:6789/0 conn(0x7fcb300b2010 legacy=0x7fcb300b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb34f34700 1 -- 172.21.15.181:0/2075804744 >> v1:172.21.15.181:6789/0 conn(0x7fcb300ce680 legacy=0x7fcb300ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb34f34700 1 -- 172.21.15.181:0/2075804744 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb300727e0 con 0x7fcb300cc6e0 2023-10-16T09:44:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb27fff700 1 -- 172.21.15.181:0/2075804744 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887714131 0 0) 0x7fcb300ce1f0 con 0x7fcb300ce680 2023-10-16T09:44:42.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb34f34700 1 --2- 172.21.15.181:0/2075804744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300cc6e0 0x7fcb30071be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098f0 tx=0x7fcb2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475a466fd224ba84 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb27fff700 1 -- 172.21.15.181:0/2075804744 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00b8a0 con 0x7fcb300cc6e0 2023-10-16T09:44:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb27fff700 1 -- 172.21.15.181:0/2075804744 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c00ba00 con 0x7fcb300cc6e0 2023-10-16T09:44:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.144+0000 7fcb27fff700 1 -- 172.21.15.181:0/2075804744 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00bd90 con 0x7fcb300cc6e0 2023-10-16T09:44:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.145+0000 7fcb37999700 1 -- 172.21.15.181:0/2075804744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300cc6e0 msgr2=0x7fcb30071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.145+0000 7fcb37999700 1 --2- 172.21.15.181:0/2075804744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300cc6e0 0x7fcb30071be0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098f0 tx=0x7fcb2c009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.145+0000 7fcb37999700 1 -- 172.21.15.181:0/2075804744 shutdown_connections 2023-10-16T09:44:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.145+0000 7fcb37999700 1 --2- 172.21.15.181:0/2075804744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300cc6e0 0x7fcb30071be0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.145+0000 7fcb37999700 1 -- 172.21.15.181:0/2075804744 >> 172.21.15.181:0/2075804744 conn(0x7fcb300cd010 msgr2=0x7fcb300cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.145+0000 7fcb37999700 1 -- 172.21.15.181:0/2075804744 shutdown_connections 2023-10-16T09:44:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.145+0000 7fcb37999700 1 -- 172.21.15.181:0/2075804744 wait complete. 2023-10-16T09:44:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.146+0000 7fcb37999700 1 Processor -- start 2023-10-16T09:44:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.146+0000 7fcb37999700 1 -- start start 2023-10-16T09:44:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb37999700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb300b2010 0x7fcb3014eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb37999700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb300cc6e0 0x7fcb30153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb37999700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300ce680 0x7fcb300cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb37999700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30155ad0 con 0x7fcb300ce680 2023-10-16T09:44:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb37999700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3014f2e0 con 0x7fcb300cc6e0 2023-10-16T09:44:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb37999700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30154a30 con 0x7fcb300b2010 2023-10-16T09:44:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb35f36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300ce680 0x7fcb300cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb35f36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300ce680 0x7fcb300cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37498/0 (socket says 172.21.15.181:37498) 2023-10-16T09:44:42.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.147+0000 7fcb35f36700 1 -- 172.21.15.181:0/2345996870 learned_addr learned my addr 172.21.15.181:0/2345996870 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:42.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb34f34700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb300cc6e0 0x7fcb30153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35735700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb300b2010 0x7fcb3014eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35f36700 1 -- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb300b2010 msgr2=0x7fcb3014eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35f36700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb300b2010 0x7fcb3014eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35f36700 1 -- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb300cc6e0 msgr2=0x7fcb30153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35f36700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb300cc6e0 0x7fcb30153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35f36700 1 -- 172.21.15.181:0/2345996870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009550 con 0x7fcb300ce680 2023-10-16T09:44:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35735700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb300b2010 0x7fcb3014eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb35f36700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300ce680 0x7fcb300cf150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb28002230 tx=0x7fcb2800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb25ffb700 1 -- 172.21.15.181:0/2345996870 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800ab70 con 0x7fcb300ce680 2023-10-16T09:44:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb25ffb700 1 -- 172.21.15.181:0/2345996870 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2800acf0 con 0x7fcb300ce680 2023-10-16T09:44:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb30154d10 con 0x7fcb300ce680 2023-10-16T09:44:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb25ffb700 1 -- 172.21.15.181:0/2345996870 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28007310 con 0x7fcb300ce680 2023-10-16T09:44:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.148+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb301552d0 con 0x7fcb300ce680 2023-10-16T09:44:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.149+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb300b6510 con 0x7fcb300ce680 2023-10-16T09:44:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.150+0000 7fcb25ffb700 1 -- 172.21.15.181:0/2345996870 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb28007470 con 0x7fcb300ce680 2023-10-16T09:44:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.150+0000 7fcb25ffb700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcb1406af00 0x7fcb1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.150+0000 7fcb25ffb700 1 -- 172.21.15.181:0/2345996870 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb280893d0 con 0x7fcb300ce680 2023-10-16T09:44:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.150+0000 7fcb35735700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcb1406af00 0x7fcb1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.150+0000 7fcb35735700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcb1406af00 0x7fcb1406d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000f60 tx=0x7fcb20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.154+0000 7fcb25ffb700 1 -- 172.21.15.181:0/2345996870 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcb28054510 con 0x7fcb300ce680 2023-10-16T09:44:42.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.344+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb300ced00 con 0x7fcb300ce680 2023-10-16T09:44:42.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.344+0000 7fcb25ffb700 1 -- 172.21.15.181:0/2345996870 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb28018020 con 0x7fcb300ce680 2023-10-16T09:44:42.346 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:42.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcb1406af00 msgr2=0x7fcb1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcb1406af00 0x7fcb1406d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000f60 tx=0x7fcb20009040 comp rx=0 tx=0).stop 2023-10-16T09:44:42.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300ce680 msgr2=0x7fcb300cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300ce680 0x7fcb300cf150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb28002230 tx=0x7fcb2800cf50 comp rx=0 tx=0).stop 2023-10-16T09:44:42.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 shutdown_connections 2023-10-16T09:44:42.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fcb1406af00 0x7fcb1406d3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb300b2010 0x7fcb3014eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb300ce680 0x7fcb300cf150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 --2- 172.21.15.181:0/2345996870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb300cc6e0 0x7fcb30153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 >> 172.21.15.181:0/2345996870 conn(0x7fcb300cd010 msgr2=0x7fcb30159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:42.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 shutdown_connections 2023-10-16T09:44:42.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.346+0000 7fcb37999700 1 -- 172.21.15.181:0/2345996870 wait complete. 2023-10-16T09:44:42.360 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:44:42.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.482+0000 7f1ec2d56700 1 Processor -- start 2023-10-16T09:44:42.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.482+0000 7f1ec2d56700 1 -- start start 2023-10-16T09:44:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec2d56700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1ebc15da70 0x7f1ebc0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec2d56700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ebc06afb0 0x7f1ebc1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec2d56700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0cef00 con 0x7f1ebc0ce6a0 2023-10-16T09:44:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec2d56700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0cec20 con 0x7f1ebc06afb0 2023-10-16T09:44:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec2d56700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc071c80 con 0x7f1ebc15da70 2023-10-16T09:44:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec12f3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ebc06afb0 0x7f1ebc1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec12f3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ebc06afb0 0x7f1ebc1615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37502/0 (socket says 172.21.15.181:37502) 2023-10-16T09:44:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec12f3700 1 -- 172.21.15.181:0/2296659131 learned_addr learned my addr 172.21.15.181:0/2296659131 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ebbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1ebc0ce6a0 0x7f1ebc06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48580/0 (socket says 172.21.15.181:48580) 2023-10-16T09:44:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec0af2700 1 --2- 172.21.15.181:0/2296659131 >> v2:172.21.15.190:3300/0 conn(0x7f1ebc15da70 0x7f1ebc0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec12f3700 1 -- 172.21.15.181:0/2296659131 >> v2:172.21.15.190:3300/0 conn(0x7f1ebc15da70 msgr2=0x7f1ebc0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec12f3700 1 --2- 172.21.15.181:0/2296659131 >> v2:172.21.15.190:3300/0 conn(0x7f1ebc15da70 0x7f1ebc0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec12f3700 1 -- 172.21.15.181:0/2296659131 >> v1:172.21.15.181:6789/0 conn(0x7f1ebc0ce6a0 legacy=0x7f1ebc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ec12f3700 1 -- 172.21.15.181:0/2296659131 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc071f60 con 0x7f1ebc06afb0 2023-10-16T09:44:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.483+0000 7f1ebb7fe700 1 -- 172.21.15.181:0/2296659131 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948184157 0 0) 0x7f1ebc0cef00 con 0x7f1ebc0ce6a0 2023-10-16T09:44:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec12f3700 1 --2- 172.21.15.181:0/2296659131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc1615f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0675e0 tx=0x7f1eb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d0e8196128dc52 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ebb7fe700 1 -- 172.21.15.181:0/2296659131 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400e070 con 0x7f1ebc06afb0 2023-10-16T09:44:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ebb7fe700 1 -- 172.21.15.181:0/2296659131 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb400b9c0 con 0x7f1ebc06afb0 2023-10-16T09:44:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ebb7fe700 1 -- 172.21.15.181:0/2296659131 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400bd70 con 0x7f1ebc06afb0 2023-10-16T09:44:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 -- 172.21.15.181:0/2296659131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc06afb0 msgr2=0x7f1ebc1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/2296659131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc1615f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0675e0 tx=0x7f1eb400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 -- 172.21.15.181:0/2296659131 shutdown_connections 2023-10-16T09:44:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/2296659131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc1615f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/2296659131 >> v2:172.21.15.190:3300/0 conn(0x7f1ebc15da70 0x7f1ebc0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 -- 172.21.15.181:0/2296659131 >> 172.21.15.181:0/2296659131 conn(0x7f1ebc0cd010 msgr2=0x7f1ebc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 -- 172.21.15.181:0/2296659131 shutdown_connections 2023-10-16T09:44:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.484+0000 7f1ec2d56700 1 -- 172.21.15.181:0/2296659131 wait complete. 2023-10-16T09:44:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.485+0000 7f1ec2d56700 1 Processor -- start 2023-10-16T09:44:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.485+0000 7f1ec2d56700 1 -- start start 2023-10-16T09:44:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec2d56700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec2d56700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ebc0ce6a0 0x7f1ebc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec2d56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc15da70 0x7f1ebc0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec2d56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc1f6f10 con 0x7f1ebc15da70 2023-10-16T09:44:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec2d56700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc1f6620 con 0x7f1ebc06afb0 2023-10-16T09:44:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec2d56700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c6dd0 con 0x7f1ebc0ce6a0 2023-10-16T09:44:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec12f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc15da70 0x7f1ebc0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec12f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc15da70 0x7f1ebc0b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37506/0 (socket says 172.21.15.181:37506) 2023-10-16T09:44:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.486+0000 7f1ec12f3700 1 -- 172.21.15.181:0/204798654 learned_addr learned my addr 172.21.15.181:0/204798654 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec0af2700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ebbfff700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ebc0ce6a0 0x7f1ebc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec12f3700 1 -- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ebc0ce6a0 msgr2=0x7f1ebc0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec12f3700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ebc0ce6a0 0x7f1ebc0b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec12f3700 1 -- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ebc06afb0 msgr2=0x7f1ebc0b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec12f3700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc0b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec12f3700 1 -- 172.21.15.181:0/204798654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4009b30 con 0x7f1ebc15da70 2023-10-16T09:44:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec0af2700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc0b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec12f3700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc15da70 0x7f1ebc0b5370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc06f140 tx=0x7f1eb4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1eb97fa700 1 -- 172.21.15.181:0/204798654 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40099b0 con 0x7f1ebc15da70 2023-10-16T09:44:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1eb97fa700 1 -- 172.21.15.181:0/204798654 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40092e0 con 0x7f1ebc15da70 2023-10-16T09:44:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc1f68c0 con 0x7f1ebc15da70 2023-10-16T09:44:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1eb97fa700 1 -- 172.21.15.181:0/204798654 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb401b650 con 0x7f1ebc15da70 2023-10-16T09:44:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.487+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc1f73a0 con 0x7f1ebc15da70 2023-10-16T09:44:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.488+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc1f7080 con 0x7f1ebc15da70 2023-10-16T09:44:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.488+0000 7f1eb97fa700 1 -- 172.21.15.181:0/204798654 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1eb401b7b0 con 0x7f1ebc15da70 2023-10-16T09:44:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.489+0000 7f1eb97fa700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1ea006af00 0x7f1ea006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.489+0000 7f1eb97fa700 1 -- 172.21.15.181:0/204798654 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eb408af30 con 0x7f1ebc15da70 2023-10-16T09:44:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.489+0000 7f1ec0af2700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1ea006af00 0x7f1ea006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.489+0000 7f1ec0af2700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1ea006af00 0x7f1ea006d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009a00 tx=0x7f1eb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.493+0000 7f1eb97fa700 1 -- 172.21.15.181:0/204798654 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1eb4055ff0 con 0x7f1ebc15da70 2023-10-16T09:44:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.684+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ebc06f6c0 con 0x7f1ebc15da70 2023-10-16T09:44:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.685+0000 7f1eb97fa700 1 -- 172.21.15.181:0/204798654 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1eb4059610 con 0x7f1ebc15da70 2023-10-16T09:44:42.687 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:44:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1ea006af00 msgr2=0x7f1ea006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1ea006af00 0x7f1ea006d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009a00 tx=0x7f1eb0008040 comp rx=0 tx=0).stop 2023-10-16T09:44:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc15da70 msgr2=0x7f1ebc0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc15da70 0x7f1ebc0b5370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc06f140 tx=0x7f1eb4004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 shutdown_connections 2023-10-16T09:44:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1ea006af00 0x7f1ea006d3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ebc15da70 0x7f1ebc0b5370 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ebc06afb0 0x7f1ebc0b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 --2- 172.21.15.181:0/204798654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ebc0ce6a0 0x7f1ebc0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 >> 172.21.15.181:0/204798654 conn(0x7f1ebc0cd010 msgr2=0x7f1ebc0d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 shutdown_connections 2023-10-16T09:44:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.687+0000 7f1ec2d56700 1 -- 172.21.15.181:0/204798654 wait complete. 2023-10-16T09:44:42.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:44:42.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597d025700 1 Processor -- start 2023-10-16T09:44:42.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597d025700 1 -- start start 2023-10-16T09:44:42.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597d025700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59780cc6e0 0x7f5978071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597d025700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59780ce680 0x7f59780ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597d025700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59780ce1f0 con 0x7f59780b2010 2023-10-16T09:44:42.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597d025700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59780ccb90 con 0x7f59780cc6e0 2023-10-16T09:44:42.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597d025700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5978072120 con 0x7f59780ce680 2023-10-16T09:44:42.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.823+0000 7f597659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59780cc6e0 0x7f5978071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59780cc6e0 0x7f5978071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37514/0 (socket says 172.21.15.181:37514) 2023-10-16T09:44:42.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597659c700 1 -- 172.21.15.181:0/483493024 learned_addr learned my addr 172.21.15.181:0/483493024 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:42.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597759e700 1 --2- 172.21.15.181:0/483493024 >> v2:172.21.15.190:3300/0 conn(0x7f59780ce680 0x7f59780ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597659c700 1 -- 172.21.15.181:0/483493024 >> v2:172.21.15.190:3300/0 conn(0x7f59780ce680 msgr2=0x7f59780ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:42.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597659c700 1 --2- 172.21.15.181:0/483493024 >> v2:172.21.15.190:3300/0 conn(0x7f59780ce680 0x7f59780ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597659c700 1 -- 172.21.15.181:0/483493024 >> v1:172.21.15.190:6789/0 conn(0x7f59780b2010 legacy=0x7f59780b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597659c700 1 -- 172.21.15.181:0/483493024 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780ccda0 con 0x7f59780cc6e0 2023-10-16T09:44:42.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f597659c700 1 --2- 172.21.15.181:0/483493024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780cc6e0 0x7f5978071be0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5978071a30 tx=0x7f596800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0c230ba773c077 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f5975d9b700 1 -- 172.21.15.181:0/483493024 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800dc70 con 0x7f59780cc6e0 2023-10-16T09:44:42.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f5975d9b700 1 -- 172.21.15.181:0/483493024 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800ddd0 con 0x7f59780cc6e0 2023-10-16T09:44:42.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.824+0000 7f5975d9b700 1 -- 172.21.15.181:0/483493024 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968010690 con 0x7f59780cc6e0 2023-10-16T09:44:42.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 -- 172.21.15.181:0/483493024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780cc6e0 msgr2=0x7f5978071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 --2- 172.21.15.181:0/483493024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780cc6e0 0x7f5978071be0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5978071a30 tx=0x7f596800be70 comp rx=0 tx=0).stop 2023-10-16T09:44:42.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 -- 172.21.15.181:0/483493024 shutdown_connections 2023-10-16T09:44:42.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 --2- 172.21.15.181:0/483493024 >> v2:172.21.15.190:3300/0 conn(0x7f59780ce680 0x7f59780ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 --2- 172.21.15.181:0/483493024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780cc6e0 0x7f5978071be0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 -- 172.21.15.181:0/483493024 >> 172.21.15.181:0/483493024 conn(0x7f59780cd010 msgr2=0x7f59780cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:42.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 -- 172.21.15.181:0/483493024 shutdown_connections 2023-10-16T09:44:42.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.825+0000 7f597d025700 1 -- 172.21.15.181:0/483493024 wait complete. 2023-10-16T09:44:42.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.826+0000 7f597d025700 1 Processor -- start 2023-10-16T09:44:42.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.826+0000 7f597d025700 1 -- start start 2023-10-16T09:44:42.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597d025700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780b2010 0x7f597814b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597d025700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59780cc6e0 0x7f5978157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597d025700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59780ce680 0x7f59780c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597d025700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780ca480 con 0x7f59780b2010 2023-10-16T09:44:42.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597d025700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780ac9b0 con 0x7f59780ce680 2023-10-16T09:44:42.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597d025700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59780acc90 con 0x7f59780cc6e0 2023-10-16T09:44:42.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f5976d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780b2010 0x7f597814b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f5976d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780b2010 0x7f597814b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37518/0 (socket says 172.21.15.181:37518) 2023-10-16T09:44:42.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f5976d9d700 1 -- 172.21.15.181:0/3532479613 learned_addr learned my addr 172.21.15.181:0/3532479613 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:42.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597759e700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59780ce680 0x7f59780c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.827+0000 7f597659c700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59780cc6e0 0x7f5978157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f5976d9d700 1 -- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59780cc6e0 msgr2=0x7f5978157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f5976d9d700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59780cc6e0 0x7f5978157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f5976d9d700 1 -- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59780ce680 msgr2=0x7f59780c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:42.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f5976d9d700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59780ce680 0x7f59780c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:42.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f5976d9d700 1 -- 172.21.15.181:0/3532479613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596800b7f0 con 0x7f59780b2010 2023-10-16T09:44:42.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f5976d9d700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780b2010 0x7f597814b6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c450 tx=0x7f596c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f595f7fe700 1 -- 172.21.15.181:0/3532479613 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0052d0 con 0x7f59780b2010 2023-10-16T09:44:42.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f595f7fe700 1 -- 172.21.15.181:0/3532479613 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c005430 con 0x7f59780b2010 2023-10-16T09:44:42.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f595f7fe700 1 -- 172.21.15.181:0/3532479613 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0038c0 con 0x7f59780b2010 2023-10-16T09:44:42.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59780acf70 con 0x7f59780b2010 2023-10-16T09:44:42.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.828+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59780ca100 con 0x7f59780b2010 2023-10-16T09:44:42.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.829+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59780ad490 con 0x7f59780b2010 2023-10-16T09:44:42.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.829+0000 7f595f7fe700 1 -- 172.21.15.181:0/3532479613 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f596c003a20 con 0x7f59780b2010 2023-10-16T09:44:42.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.830+0000 7f595f7fe700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f596406af30 0x7f596406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:42.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.830+0000 7f597659c700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f596406af30 0x7f596406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:42.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.830+0000 7f595f7fe700 1 -- 172.21.15.181:0/3532479613 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f596c088860 con 0x7f59780b2010 2023-10-16T09:44:42.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.830+0000 7f597659c700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f596406af30 0x7f596406d3f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f596800beb0 tx=0x7f596801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:42.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:42.834+0000 7f595f7fe700 1 -- 172.21.15.181:0/3532479613 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f596c0539a0 con 0x7f59780b2010 2023-10-16T09:44:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.030+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59780b6510 con 0x7f59780b2010 2023-10-16T09:44:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.030+0000 7f595f7fe700 1 -- 172.21.15.181:0/3532479613 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f596c0170a0 con 0x7f59780b2010 2023-10-16T09:44:43.033 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:44:43.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f596406af30 msgr2=0x7f596406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f596406af30 0x7f596406d3f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f596800beb0 tx=0x7f596801c040 comp rx=0 tx=0).stop 2023-10-16T09:44:43.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780b2010 msgr2=0x7f597814b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780b2010 0x7f597814b6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c450 tx=0x7f596c00c760 comp rx=0 tx=0).stop 2023-10-16T09:44:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 shutdown_connections 2023-10-16T09:44:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f596406af30 0x7f596406d3f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59780ce680 0x7f59780c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59780b2010 0x7f597814b6d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 --2- 172.21.15.181:0/3532479613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59780cc6e0 0x7f5978157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 >> 172.21.15.181:0/3532479613 conn(0x7f59780cd010 msgr2=0x7f59780b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:43.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.032+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 shutdown_connections 2023-10-16T09:44:43.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.033+0000 7f597d025700 1 -- 172.21.15.181:0/3532479613 wait complete. 2023-10-16T09:44:43.047 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:43.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.166+0000 7f9cf3c6c700 1 Processor -- start 2023-10-16T09:44:43.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.166+0000 7f9cf3c6c700 1 -- start start 2023-10-16T09:44:43.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf3c6c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0af450 0x7f9cec0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf3c6c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9cec0cd470 0x7f9cec0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf3c6c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec06f870 con 0x7f9cec15dfd0 2023-10-16T09:44:43.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf3c6c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec0d1160 con 0x7f9cec0cd470 2023-10-16T09:44:43.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf3c6c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0cc8e0 con 0x7f9cec0af450 2023-10-16T09:44:43.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf2209700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9cec0cd470 0x7f9cec0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf2209700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9cec0cd470 0x7f9cec0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37530/0 (socket says 172.21.15.181:37530) 2023-10-16T09:44:43.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf2209700 1 -- 172.21.15.181:0/3822352415 learned_addr learned my addr 172.21.15.181:0/3822352415 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:43.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf1207700 1 --2- 172.21.15.181:0/3822352415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0af450 0x7f9cec0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf2209700 1 -- 172.21.15.181:0/3822352415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0af450 msgr2=0x7f9cec0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:43.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf2209700 1 --2- 172.21.15.181:0/3822352415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0af450 0x7f9cec0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf2209700 1 -- 172.21.15.181:0/3822352415 >> v1:172.21.15.190:6789/0 conn(0x7f9cec15dfd0 legacy=0x7f9cec06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:43.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.167+0000 7f9cf2209700 1 -- 172.21.15.181:0/3822352415 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0ccbc0 con 0x7f9cec0cd470 2023-10-16T09:44:43.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf2209700 1 --2- 172.21.15.181:0/3822352415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0cd470 0x7f9cec0cd920 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009e50 tx=0x7f9ce400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0f190d2c309ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf0a06700 1 -- 172.21.15.181:0/3822352415 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400e070 con 0x7f9cec0cd470 2023-10-16T09:44:43.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf0a06700 1 -- 172.21.15.181:0/3822352415 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce4003a20 con 0x7f9cec0cd470 2023-10-16T09:44:43.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf0a06700 1 -- 172.21.15.181:0/3822352415 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4003dd0 con 0x7f9cec0cd470 2023-10-16T09:44:43.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3822352415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0cd470 msgr2=0x7f9cec0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3822352415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0cd470 0x7f9cec0cd920 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009e50 tx=0x7f9ce400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:43.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3822352415 shutdown_connections 2023-10-16T09:44:43.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3822352415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0cd470 0x7f9cec0cd920 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3822352415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0af450 0x7f9cec0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3822352415 >> 172.21.15.181:0/3822352415 conn(0x7f9cec0d04a0 msgr2=0x7f9cec0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:43.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3822352415 shutdown_connections 2023-10-16T09:44:43.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.168+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3822352415 wait complete. 2023-10-16T09:44:43.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 Processor -- start 2023-10-16T09:44:43.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 -- start start 2023-10-16T09:44:43.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0af450 0x7f9cec0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0cd470 0x7f9cec151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec15dfd0 0x7f9cec159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec152e40 con 0x7f9cec0af450 2023-10-16T09:44:43.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0b0ea0 con 0x7f9cec15dfd0 2023-10-16T09:44:43.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf3c6c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec152370 con 0x7f9cec0cd470 2023-10-16T09:44:43.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf1a08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0af450 0x7f9cec0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf1a08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0af450 0x7f9cec0aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37532/0 (socket says 172.21.15.181:37532) 2023-10-16T09:44:43.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.170+0000 7f9cf1a08700 1 -- 172.21.15.181:0/3795419253 learned_addr learned my addr 172.21.15.181:0/3795419253 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:43.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf2209700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec15dfd0 0x7f9cec159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1207700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0cd470 0x7f9cec151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1a08700 1 -- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0cd470 msgr2=0x7f9cec151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1a08700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0cd470 0x7f9cec151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1a08700 1 -- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec15dfd0 msgr2=0x7f9cec159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1a08700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec15dfd0 0x7f9cec159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1a08700 1 -- 172.21.15.181:0/3795419253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4009b30 con 0x7f9cec0af450 2023-10-16T09:44:43.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1207700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0cd470 0x7f9cec151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:43.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf1a08700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0af450 0x7f9cec0aff00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b450 tx=0x7f9cdc00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9ce27fc700 1 -- 172.21.15.181:0/3795419253 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004020 con 0x7f9cec0af450 2023-10-16T09:44:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9ce27fc700 1 -- 172.21.15.181:0/3795419253 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc004180 con 0x7f9cec0af450 2023-10-16T09:44:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec0b0600 con 0x7f9cec0af450 2023-10-16T09:44:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9ce27fc700 1 -- 172.21.15.181:0/3795419253 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004530 con 0x7f9cec0af450 2023-10-16T09:44:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.171+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec0b0bc0 con 0x7f9cec0af450 2023-10-16T09:44:43.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.172+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec067d20 con 0x7f9cec0af450 2023-10-16T09:44:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.172+0000 7f9ce27fc700 1 -- 172.21.15.181:0/3795419253 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9cdc0036d0 con 0x7f9cec0af450 2023-10-16T09:44:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.173+0000 7f9ce27fc700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9cd006af30 0x7f9cd006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.173+0000 7f9cf1207700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9cd006af30 0x7f9cd006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.173+0000 7f9ce27fc700 1 -- 172.21.15.181:0/3795419253 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cdc089070 con 0x7f9cec0af450 2023-10-16T09:44:43.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.173+0000 7f9cf1207700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9cd006af30 0x7f9cd006d3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9cec152ac0 tx=0x7f9ce8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.177+0000 7f9ce27fc700 1 -- 172.21.15.181:0/3795419253 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9cdc0541b0 con 0x7f9cec0af450 2023-10-16T09:44:43.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.414+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cec06f3d0 con 0x7f9cec0af450 2023-10-16T09:44:43.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.414+0000 7f9ce27fc700 1 -- 172.21.15.181:0/3795419253 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f9cdc0577d0 con 0x7f9cec0af450 2023-10-16T09:44:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:43.417 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:43.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9cd006af30 msgr2=0x7f9cd006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9cd006af30 0x7f9cd006d3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9cec152ac0 tx=0x7f9ce8008040 comp rx=0 tx=0).stop 2023-10-16T09:44:43.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0af450 msgr2=0x7f9cec0aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0af450 0x7f9cec0aff00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b450 tx=0x7f9cdc00b760 comp rx=0 tx=0).stop 2023-10-16T09:44:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 shutdown_connections 2023-10-16T09:44:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9cd006af30 0x7f9cd006d3f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cec15dfd0 0x7f9cec159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec0af450 0x7f9cec0aff00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 --2- 172.21.15.181:0/3795419253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cec0cd470 0x7f9cec151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 >> 172.21.15.181:0/3795419253 conn(0x7f9cec0d04a0 msgr2=0x7f9cec15cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 shutdown_connections 2023-10-16T09:44:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.416+0000 7f9cf3c6c700 1 -- 172.21.15.181:0/3795419253 wait complete. 2023-10-16T09:44:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:43.431 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:43.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.549+0000 7efc16b4a700 1 Processor -- start 2023-10-16T09:44:43.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.549+0000 7efc16b4a700 1 -- start start 2023-10-16T09:44:43.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc16b4a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc1015a550 0x7efc100cb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc16b4a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015ed90 0x7efc1006e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc16b4a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc100cb3f0 con 0x7efc1015e1d0 2023-10-16T09:44:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc16b4a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc100caac0 con 0x7efc1015a550 2023-10-16T09:44:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc16b4a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc100cada0 con 0x7efc1015ed90 2023-10-16T09:44:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc148e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc1015a550 0x7efc100cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc150e7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efc1015e1d0 0x7efc100cdfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48620/0 (socket says 172.21.15.181:48620) 2023-10-16T09:44:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc148e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc1015a550 0x7efc100cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37540/0 (socket says 172.21.15.181:37540) 2023-10-16T09:44:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc150e7700 1 -- 172.21.15.181:0/971653887 learned_addr learned my addr 172.21.15.181:0/971653887 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc0ffff700 1 --2- 172.21.15.181:0/971653887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015ed90 0x7efc1006e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc148e6700 1 -- 172.21.15.181:0/971653887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015ed90 msgr2=0x7efc1006e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:43.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc148e6700 1 --2- 172.21.15.181:0/971653887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015ed90 0x7efc1006e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc148e6700 1 -- 172.21.15.181:0/971653887 >> v1:172.21.15.181:6789/0 conn(0x7efc1015e1d0 legacy=0x7efc100cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc148e6700 1 -- 172.21.15.181:0/971653887 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc100cb080 con 0x7efc1015a550 2023-10-16T09:44:43.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.550+0000 7efc0f7fe700 1 -- 172.21.15.181:0/971653887 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592666723 0 0) 0x7efc100cb3f0 con 0x7efc1015e1d0 2023-10-16T09:44:43.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc148e6700 1 --2- 172.21.15.181:0/971653887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015a550 0x7efc100cb880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efc04009820 tx=0x7efc04009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f7034f6818ffc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc0f7fe700 1 -- 172.21.15.181:0/971653887 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04005e30 con 0x7efc1015a550 2023-10-16T09:44:43.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc0f7fe700 1 -- 172.21.15.181:0/971653887 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc04006030 con 0x7efc1015a550 2023-10-16T09:44:43.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc0f7fe700 1 -- 172.21.15.181:0/971653887 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040063e0 con 0x7efc1015a550 2023-10-16T09:44:43.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 -- 172.21.15.181:0/971653887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015a550 msgr2=0x7efc100cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 --2- 172.21.15.181:0/971653887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015a550 0x7efc100cb880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efc04009820 tx=0x7efc04009b30 comp rx=0 tx=0).stop 2023-10-16T09:44:43.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 -- 172.21.15.181:0/971653887 shutdown_connections 2023-10-16T09:44:43.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 --2- 172.21.15.181:0/971653887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015a550 0x7efc100cb880 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 --2- 172.21.15.181:0/971653887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015ed90 0x7efc1006e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 -- 172.21.15.181:0/971653887 >> 172.21.15.181:0/971653887 conn(0x7efc100d0460 msgr2=0x7efc100d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:43.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 -- 172.21.15.181:0/971653887 shutdown_connections 2023-10-16T09:44:43.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.551+0000 7efc16b4a700 1 -- 172.21.15.181:0/971653887 wait complete. 2023-10-16T09:44:43.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.552+0000 7efc16b4a700 1 Processor -- start 2023-10-16T09:44:43.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.552+0000 7efc16b4a700 1 -- start start 2023-10-16T09:44:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc16b4a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc1015a550 0x7efc10151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc16b4a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015e1d0 0x7efc100c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc16b4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015ed90 0x7efc100cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc16b4a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100b5e80 con 0x7efc1015ed90 2023-10-16T09:44:43.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc16b4a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100b4de0 con 0x7efc1015a550 2023-10-16T09:44:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc16b4a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc100b50c0 con 0x7efc1015e1d0 2023-10-16T09:44:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc150e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015ed90 0x7efc100cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc150e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015ed90 0x7efc100cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37548/0 (socket says 172.21.15.181:37548) 2023-10-16T09:44:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.553+0000 7efc150e7700 1 -- 172.21.15.181:0/1750326906 learned_addr learned my addr 172.21.15.181:0/1750326906 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc148e6700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc1015a550 0x7efc10151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc0ffff700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015e1d0 0x7efc100c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc150e7700 1 -- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015e1d0 msgr2=0x7efc100c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc150e7700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015e1d0 0x7efc100c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc150e7700 1 -- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc1015a550 msgr2=0x7efc10151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc150e7700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc1015a550 0x7efc10151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc150e7700 1 -- 172.21.15.181:0/1750326906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc04009430 con 0x7efc1015ed90 2023-10-16T09:44:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc0ffff700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015e1d0 0x7efc100c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc150e7700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015ed90 0x7efc100cf590 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc08002230 tx=0x7efc0800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc0d7fa700 1 -- 172.21.15.181:0/1750326906 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800ab70 con 0x7efc1015ed90 2023-10-16T09:44:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc0d7fa700 1 -- 172.21.15.181:0/1750326906 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0800acf0 con 0x7efc1015ed90 2023-10-16T09:44:43.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc100b53a0 con 0x7efc1015ed90 2023-10-16T09:44:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc0d7fa700 1 -- 172.21.15.181:0/1750326906 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08007310 con 0x7efc1015ed90 2023-10-16T09:44:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.554+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc100b5960 con 0x7efc1015ed90 2023-10-16T09:44:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.555+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc100b5b80 con 0x7efc1015ed90 2023-10-16T09:44:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.556+0000 7efc0d7fa700 1 -- 172.21.15.181:0/1750326906 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efc08007470 con 0x7efc1015ed90 2023-10-16T09:44:43.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.556+0000 7efc0d7fa700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.556+0000 7efc0d7fa700 1 -- 172.21.15.181:0/1750326906 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc080893f0 con 0x7efc1015ed90 2023-10-16T09:44:43.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.556+0000 7efc148e6700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.556+0000 7efc148e6700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efbf406af00 0x7efbf406d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc04013040 tx=0x7efc04008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.560+0000 7efc0d7fa700 1 -- 172.21.15.181:0/1750326906 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efc08054530 con 0x7efc1015ed90 2023-10-16T09:44:43.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.751+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc10067d20 con 0x7efc1015ed90 2023-10-16T09:44:43.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.752+0000 7efc0d7fa700 1 -- 172.21.15.181:0/1750326906 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7efc0800ae60 con 0x7efc1015ed90 2023-10-16T09:44:43.755 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:43.755 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:43.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efbf406af00 msgr2=0x7efbf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efbf406af00 0x7efbf406d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc04013040 tx=0x7efc04008070 comp rx=0 tx=0).stop 2023-10-16T09:44:43.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015ed90 msgr2=0x7efc100cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015ed90 0x7efc100cf590 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc08002230 tx=0x7efc0800cf50 comp rx=0 tx=0).stop 2023-10-16T09:44:43.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 shutdown_connections 2023-10-16T09:44:43.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1015ed90 0x7efc100cf590 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc1015a550 0x7efc10151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 --2- 172.21.15.181:0/1750326906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc1015e1d0 0x7efc100c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 >> 172.21.15.181:0/1750326906 conn(0x7efc100d0460 msgr2=0x7efc100d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:43.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 shutdown_connections 2023-10-16T09:44:43.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.755+0000 7efc16b4a700 1 -- 172.21.15.181:0/1750326906 wait complete. 2023-10-16T09:44:43.769 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T09:44:43.770 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.888+0000 7f6768123700 1 Processor -- start 2023-10-16T09:44:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.888+0000 7f6768123700 1 -- start start 2023-10-16T09:44:43.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6768123700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67600cdfb0 0x7f676015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6768123700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f676006cdf0 0x7f67600cb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6768123700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67600cabd0 con 0x7f67600cc0c0 2023-10-16T09:44:43.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6768123700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67600ca5c0 con 0x7f67600cdfb0 2023-10-16T09:44:43.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6768123700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6760066740 con 0x7f676006cdf0 2023-10-16T09:44:43.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67600cdfb0 0x7f676015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67600cdfb0 0x7f676015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37552/0 (socket says 172.21.15.181:37552) 2023-10-16T09:44:43.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 -- 172.21.15.181:0/2331109930 learned_addr learned my addr 172.21.15.181:0/2331109930 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:43.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f67666c0700 1 --2- 172.21.15.181:0/2331109930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f676006cdf0 0x7f67600cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 -- 172.21.15.181:0/2331109930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f676006cdf0 msgr2=0x7f67600cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:43.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 --2- 172.21.15.181:0/2331109930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f676006cdf0 0x7f67600cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 -- 172.21.15.181:0/2331109930 >> v1:172.21.15.190:6789/0 conn(0x7f67600cc0c0 legacy=0x7f676015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:43.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 -- 172.21.15.181:0/2331109930 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760066a20 con 0x7f67600cdfb0 2023-10-16T09:44:43.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.889+0000 7f6765ebf700 1 --2- 172.21.15.181:0/2331109930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67600cdfb0 0x7f676015d8b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f675c00bb90 tx=0x7f675c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0576b651541099 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6764ebd700 1 -- 172.21.15.181:0/2331109930 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c00dd10 con 0x7f67600cdfb0 2023-10-16T09:44:43.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6764ebd700 1 -- 172.21.15.181:0/2331109930 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c00de70 con 0x7f67600cdfb0 2023-10-16T09:44:43.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6764ebd700 1 -- 172.21.15.181:0/2331109930 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c004230 con 0x7f67600cdfb0 2023-10-16T09:44:43.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 -- 172.21.15.181:0/2331109930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67600cdfb0 msgr2=0x7f676015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 --2- 172.21.15.181:0/2331109930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67600cdfb0 0x7f676015d8b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f675c00bb90 tx=0x7f675c00bea0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 -- 172.21.15.181:0/2331109930 shutdown_connections 2023-10-16T09:44:43.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 --2- 172.21.15.181:0/2331109930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f676006cdf0 0x7f67600cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 --2- 172.21.15.181:0/2331109930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67600cdfb0 0x7f676015d8b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 -- 172.21.15.181:0/2331109930 >> 172.21.15.181:0/2331109930 conn(0x7f67600b1780 msgr2=0x7f67600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:43.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 -- 172.21.15.181:0/2331109930 shutdown_connections 2023-10-16T09:44:43.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.890+0000 7f6768123700 1 -- 172.21.15.181:0/2331109930 wait complete. 2023-10-16T09:44:43.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.891+0000 7f6768123700 1 Processor -- start 2023-10-16T09:44:43.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.891+0000 7f6768123700 1 -- start start 2023-10-16T09:44:43.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6768123700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676006cdf0 0x7f67600a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6768123700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67600cc0c0 0x7f676015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6768123700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67600cdfb0 0x7f67601501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6768123700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676015a640 con 0x7f676006cdf0 2023-10-16T09:44:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6768123700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676015a8e0 con 0x7f67600cc0c0 2023-10-16T09:44:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6768123700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676015abc0 con 0x7f67600cdfb0 2023-10-16T09:44:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6765ebf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676006cdf0 0x7f67600a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6765ebf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676006cdf0 0x7f67600a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37558/0 (socket says 172.21.15.181:37558) 2023-10-16T09:44:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6765ebf700 1 -- 172.21.15.181:0/849569283 learned_addr learned my addr 172.21.15.181:0/849569283 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f67656be700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67600cc0c0 0x7f676015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f67666c0700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67600cdfb0 0x7f67601501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6765ebf700 1 -- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67600cdfb0 msgr2=0x7f67601501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6765ebf700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67600cdfb0 0x7f67601501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6765ebf700 1 -- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67600cc0c0 msgr2=0x7f676015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.892+0000 7f6765ebf700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67600cc0c0 0x7f676015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f6765ebf700 1 -- 172.21.15.181:0/849569283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c00b7f0 con 0x7f676006cdf0 2023-10-16T09:44:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f67666c0700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67600cdfb0 0x7f67601501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f67656be700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67600cc0c0 0x7f676015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:43.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f6765ebf700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676006cdf0 0x7f67600a0510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6760074830 tx=0x7f675c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f67567fc700 1 -- 172.21.15.181:0/849569283 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c00dcc0 con 0x7f676006cdf0 2023-10-16T09:44:43.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f67567fc700 1 -- 172.21.15.181:0/849569283 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c010470 con 0x7f676006cdf0 2023-10-16T09:44:43.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676015aea0 con 0x7f676006cdf0 2023-10-16T09:44:43.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f67567fc700 1 -- 172.21.15.181:0/849569283 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c0107e0 con 0x7f676006cdf0 2023-10-16T09:44:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.893+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676015b460 con 0x7f676006cdf0 2023-10-16T09:44:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.894+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6760067d20 con 0x7f676006cdf0 2023-10-16T09:44:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.894+0000 7f67567fc700 1 -- 172.21.15.181:0/849569283 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f675c0193f0 con 0x7f676006cdf0 2023-10-16T09:44:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.895+0000 7f67567fc700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f674406af00 0x7f674406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.895+0000 7f67567fc700 1 -- 172.21.15.181:0/849569283 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f675c08b590 con 0x7f676006cdf0 2023-10-16T09:44:43.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.895+0000 7f67656be700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f674406af00 0x7f674406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:43.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.895+0000 7f67656be700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f674406af00 0x7f674406d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f67600c40a0 tx=0x7f6750009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:43.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:43.899+0000 7f67567fc700 1 -- 172.21.15.181:0/849569283 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f675c056630 con 0x7f676006cdf0 2023-10-16T09:44:44.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.138+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67600cf510 con 0x7f676006cdf0 2023-10-16T09:44:44.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.139+0000 7f67567fc700 1 -- 172.21.15.181:0/849569283 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f675c006d80 con 0x7f676006cdf0 2023-10-16T09:44:44.141 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:44.142 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:44.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f674406af00 msgr2=0x7f674406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f674406af00 0x7f674406d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f67600c40a0 tx=0x7f6750009040 comp rx=0 tx=0).stop 2023-10-16T09:44:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676006cdf0 msgr2=0x7f67600a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676006cdf0 0x7f67600a0510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6760074830 tx=0x7f675c004750 comp rx=0 tx=0).stop 2023-10-16T09:44:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 shutdown_connections 2023-10-16T09:44:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f674406af00 0x7f674406d3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67600cdfb0 0x7f67601501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f676006cdf0 0x7f67600a0510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 --2- 172.21.15.181:0/849569283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67600cc0c0 0x7f676015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 >> 172.21.15.181:0/849569283 conn(0x7f67600b1780 msgr2=0x7f67600c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:44.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 shutdown_connections 2023-10-16T09:44:44.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.141+0000 7f6768123700 1 -- 172.21.15.181:0/849569283 wait complete. 2023-10-16T09:44:44.146 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:44.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:44.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.273+0000 7f869de95700 1 Processor -- start 2023-10-16T09:44:44.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.273+0000 7f869de95700 1 -- start start 2023-10-16T09:44:44.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f869de95700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f869815eab0 0x7f86980d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f869de95700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86980cb0c0 0x7f869815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f869de95700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869806ed00 con 0x7f869815e6d0 2023-10-16T09:44:44.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f869de95700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86980d02d0 con 0x7f869815eab0 2023-10-16T09:44:44.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f869de95700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869815ad40 con 0x7f86980cb0c0 2023-10-16T09:44:44.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8696ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f869815eab0 0x7f86980d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8696ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f869815eab0 0x7f86980d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37568/0 (socket says 172.21.15.181:37568) 2023-10-16T09:44:44.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8696ffd700 1 -- 172.21.15.181:0/965530129 learned_addr learned my addr 172.21.15.181:0/965530129 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:44.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8697fff700 1 --2- 172.21.15.181:0/965530129 >> v2:172.21.15.190:3300/0 conn(0x7f86980cb0c0 0x7f869815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8696ffd700 1 -- 172.21.15.181:0/965530129 >> v2:172.21.15.190:3300/0 conn(0x7f86980cb0c0 msgr2=0x7f869815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:44.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8696ffd700 1 --2- 172.21.15.181:0/965530129 >> v2:172.21.15.190:3300/0 conn(0x7f86980cb0c0 0x7f869815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8696ffd700 1 -- 172.21.15.181:0/965530129 >> v1:172.21.15.190:6789/0 conn(0x7f869815e6d0 legacy=0x7f869815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:44.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.274+0000 7f8696ffd700 1 -- 172.21.15.181:0/965530129 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869806e0f0 con 0x7f869815eab0 2023-10-16T09:44:44.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f8696ffd700 1 --2- 172.21.15.181:0/965530129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815eab0 0x7f86980d0fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8698071650 tx=0x7f868800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b75871234325ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:44.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f86967fc700 1 -- 172.21.15.181:0/965530129 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868800dcf0 con 0x7f869815eab0 2023-10-16T09:44:44.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f86967fc700 1 -- 172.21.15.181:0/965530129 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868800de50 con 0x7f869815eab0 2023-10-16T09:44:44.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f86967fc700 1 -- 172.21.15.181:0/965530129 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688004250 con 0x7f869815eab0 2023-10-16T09:44:44.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 -- 172.21.15.181:0/965530129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815eab0 msgr2=0x7f86980d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 --2- 172.21.15.181:0/965530129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815eab0 0x7f86980d0fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8698071650 tx=0x7f868800be70 comp rx=0 tx=0).stop 2023-10-16T09:44:44.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 -- 172.21.15.181:0/965530129 shutdown_connections 2023-10-16T09:44:44.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 --2- 172.21.15.181:0/965530129 >> v2:172.21.15.190:3300/0 conn(0x7f86980cb0c0 0x7f869815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 --2- 172.21.15.181:0/965530129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815eab0 0x7f86980d0fc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 -- 172.21.15.181:0/965530129 >> 172.21.15.181:0/965530129 conn(0x7f86980d04a0 msgr2=0x7f86980d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:44.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 -- 172.21.15.181:0/965530129 shutdown_connections 2023-10-16T09:44:44.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.275+0000 7f869de95700 1 -- 172.21.15.181:0/965530129 wait complete. 2023-10-16T09:44:44.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.276+0000 7f869de95700 1 Processor -- start 2023-10-16T09:44:44.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f869de95700 1 -- start start 2023-10-16T09:44:44.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f869de95700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86980cb0c0 0x7f86980c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f869de95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815e6d0 0x7f869814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f869de95700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f869815eab0 0x7f86980adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f869de95700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869814f1b0 con 0x7f869815e6d0 2023-10-16T09:44:44.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f869de95700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980b2eb0 con 0x7f869815eab0 2023-10-16T09:44:44.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f869de95700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c9c80 con 0x7f86980cb0c0 2023-10-16T09:44:44.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.277+0000 7f8696ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815e6d0 0x7f869814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815e6d0 0x7f869814e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37574/0 (socket says 172.21.15.181:37574) 2023-10-16T09:44:44.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 -- 172.21.15.181:0/4081074604 learned_addr learned my addr 172.21.15.181:0/4081074604 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:44.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f86977fe700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86980cb0c0 0x7f86980c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 -- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86980cb0c0 msgr2=0x7f86980c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86980cb0c0 0x7f86980c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8697fff700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f869815eab0 0x7f86980adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 -- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f869815eab0 msgr2=0x7f86980adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f869815eab0 0x7f86980adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 -- 172.21.15.181:0/4081074604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868800b7f0 con 0x7f869815e6d0 2023-10-16T09:44:44.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f8696ffd700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815e6d0 0x7f869814e6f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8698071580 tx=0x7f8688004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:44.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f868ffff700 1 -- 172.21.15.181:0/4081074604 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688020070 con 0x7f869815e6d0 2023-10-16T09:44:44.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f868ffff700 1 -- 172.21.15.181:0/4081074604 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8688004030 con 0x7f869815e6d0 2023-10-16T09:44:44.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869814ec30 con 0x7f869815e6d0 2023-10-16T09:44:44.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f868ffff700 1 -- 172.21.15.181:0/4081074604 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688010470 con 0x7f869815e6d0 2023-10-16T09:44:44.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.278+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86980b2860 con 0x7f869815e6d0 2023-10-16T09:44:44.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.279+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86980670b0 con 0x7f869815e6d0 2023-10-16T09:44:44.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.279+0000 7f868ffff700 1 -- 172.21.15.181:0/4081074604 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f86880193f0 con 0x7f869815e6d0 2023-10-16T09:44:44.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.280+0000 7f868ffff700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f867806af00 0x7f867806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.280+0000 7f86977fe700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f867806af00 0x7f867806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.280+0000 7f868ffff700 1 -- 172.21.15.181:0/4081074604 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f868808c920 con 0x7f869815e6d0 2023-10-16T09:44:44.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.280+0000 7f86977fe700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f867806af00 0x7f867806d3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f868000a7d0 tx=0x7f8680008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:44.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.285+0000 7f868ffff700 1 -- 172.21.15.181:0/4081074604 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8688057a60 con 0x7f869815e6d0 2023-10-16T09:44:44.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.519+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f869814ec30 con 0x7f869815e6d0 2023-10-16T09:44:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.519+0000 7f868ffff700 1 -- 172.21.15.181:0/4081074604 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f869814ec30 con 0x7f869815e6d0 2023-10-16T09:44:44.522 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:44.522 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2467574686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2467574686},{"type":"v1","addr":"172.21.15.181:6837","nonce":2467574686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3657988941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3657988941},{"type":"v1","addr":"172.21.15.181:6836","nonce":3657988941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1507612323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1507612323},{"type":"v1","addr":"172.21.15.190:6834","nonce":1507612323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:01.170821+0000","modified":"2023-10-16T09:44:04.215750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1843354940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1843354940},{"type":"v1","addr":"172.21.15.190:6835","nonce":1843354940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T09:44:44.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f867806af00 msgr2=0x7f867806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f867806af00 0x7f867806d3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f868000a7d0 tx=0x7f8680008040 comp rx=0 tx=0).stop 2023-10-16T09:44:44.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815e6d0 msgr2=0x7f869814e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815e6d0 0x7f869814e6f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8698071580 tx=0x7f8688004750 comp rx=0 tx=0).stop 2023-10-16T09:44:44.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 shutdown_connections 2023-10-16T09:44:44.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f867806af00 0x7f867806d3c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86980cb0c0 0x7f86980c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f869815eab0 0x7f86980adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 --2- 172.21.15.181:0/4081074604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f869815e6d0 0x7f869814e6f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 >> 172.21.15.181:0/4081074604 conn(0x7f86980d04a0 msgr2=0x7f86980aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:44.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 shutdown_connections 2023-10-16T09:44:44.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.522+0000 7f869de95700 1 -- 172.21.15.181:0/4081074604 wait complete. 2023-10-16T09:44:44.527 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 8 2023-10-16T09:44:44.536 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.656+0000 7f20960f5700 1 Processor -- start 2023-10-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.656+0000 7f20960f5700 1 -- start start 2023-10-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.656+0000 7f20960f5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900af450 0x7f20900af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.656+0000 7f20960f5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20900cd470 0x7f20900cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.656+0000 7f20960f5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209006f870 con 0x7f209015dfd0 2023-10-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.656+0000 7f20960f5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20900d1160 con 0x7f20900cd470 2023-10-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.656+0000 7f20960f5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20900cc8e0 con 0x7f20900af450 2023-10-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20900cd470 0x7f20900cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f209015dfd0 0x7f209006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48658/0 (socket says 172.21.15.181:48658) 2023-10-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208f7fe700 1 -- 172.21.15.181:0/2234470345 learned_addr learned my addr 172.21.15.181:0/2234470345 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208effd700 1 --2- 172.21.15.181:0/2234470345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900af450 0x7f20900af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208ffff700 1 -- 172.21.15.181:0/2234470345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900af450 msgr2=0x7f20900af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208ffff700 1 --2- 172.21.15.181:0/2234470345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900af450 0x7f20900af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208e7fc700 1 -- 172.21.15.181:0/2234470345 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585019673 0 0) 0x7f209006f870 con 0x7f209015dfd0 2023-10-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208ffff700 1 -- 172.21.15.181:0/2234470345 >> v1:172.21.15.181:6789/0 conn(0x7f209015dfd0 legacy=0x7f209006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208ffff700 1 -- 172.21.15.181:0/2234470345 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900ccbc0 con 0x7f20900cd470 2023-10-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208ffff700 1 --2- 172.21.15.181:0/2234470345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20900cd470 0x7f20900cd920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2088009d90 tx=0x7f208800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34563787bf575e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208e7fc700 1 -- 172.21.15.181:0/2234470345 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800e070 con 0x7f20900cd470 2023-10-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208e7fc700 1 -- 172.21.15.181:0/2234470345 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800bbd0 con 0x7f20900cd470 2023-10-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.657+0000 7f208e7fc700 1 -- 172.21.15.181:0/2234470345 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088012430 con 0x7f20900cd470 2023-10-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 -- 172.21.15.181:0/2234470345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20900cd470 msgr2=0x7f20900cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 --2- 172.21.15.181:0/2234470345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20900cd470 0x7f20900cd920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2088009d90 tx=0x7f208800b330 comp rx=0 tx=0).stop 2023-10-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 -- 172.21.15.181:0/2234470345 shutdown_connections 2023-10-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 --2- 172.21.15.181:0/2234470345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20900cd470 0x7f20900cd920 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 --2- 172.21.15.181:0/2234470345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900af450 0x7f20900af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 -- 172.21.15.181:0/2234470345 >> 172.21.15.181:0/2234470345 conn(0x7f20900d04a0 msgr2=0x7f20900d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 -- 172.21.15.181:0/2234470345 shutdown_connections 2023-10-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.658+0000 7f20960f5700 1 -- 172.21.15.181:0/2234470345 wait complete. 2023-10-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.659+0000 7f20960f5700 1 Processor -- start 2023-10-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.659+0000 7f20960f5700 1 -- start start 2023-10-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f20960f5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20900af450 0x7f2090159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f20960f5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900cd470 0x7f20901e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f20960f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209015dfd0 0x7f209015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f20960f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20901e59d0 con 0x7f209015dfd0 2023-10-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f20960f5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20901e5cb0 con 0x7f20900af450 2023-10-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f20960f5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20901e5f90 con 0x7f20900cd470 2023-10-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209015dfd0 0x7f209015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209015dfd0 0x7f209015cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37590/0 (socket says 172.21.15.181:37590) 2023-10-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 -- 172.21.15.181:0/886736441 learned_addr learned my addr 172.21.15.181:0/886736441 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208effd700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900cd470 0x7f20901e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208f7fe700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20900af450 0x7f2090159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 -- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900cd470 msgr2=0x7f20901e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900cd470 0x7f20901e7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 -- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20900af450 msgr2=0x7f2090159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20900af450 0x7f2090159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 -- 172.21.15.181:0/886736441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2088009a70 con 0x7f209015dfd0 2023-10-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.660+0000 7f208ffff700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209015dfd0 0x7f209015cd20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20900716a0 tx=0x7f2088004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.661+0000 7f2077fff700 1 -- 172.21.15.181:0/886736441 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880098f0 con 0x7f209015dfd0 2023-10-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.661+0000 7f2077fff700 1 -- 172.21.15.181:0/886736441 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2088020070 con 0x7f209015dfd0 2023-10-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.661+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20901e6270 con 0x7f209015dfd0 2023-10-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.661+0000 7f2077fff700 1 -- 172.21.15.181:0/886736441 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880276a0 con 0x7f209015dfd0 2023-10-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.661+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209015a1c0 con 0x7f209015dfd0 2023-10-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.662+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20901e6790 con 0x7f209015dfd0 2023-10-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.662+0000 7f2077fff700 1 -- 172.21.15.181:0/886736441 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2088027800 con 0x7f209015dfd0 2023-10-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.662+0000 7f2077fff700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f207c06af00 0x7f207c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.662+0000 7f2077fff700 1 -- 172.21.15.181:0/886736441 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2088022070 con 0x7f209015dfd0 2023-10-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.662+0000 7f208f7fe700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f207c06af00 0x7f207c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.663+0000 7f208f7fe700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f207c06af00 0x7f207c06d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2080001750 tx=0x7f2080009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.667+0000 7f2077fff700 1 -- 172.21.15.181:0/886736441 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f20901e6790 con 0x7f209015dfd0 2023-10-16T09:44:44.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.861+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20901e6790 con 0x7f209015dfd0 2023-10-16T09:44:44.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.862+0000 7f2077fff700 1 -- 172.21.15.181:0/886736441 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f20901e6790 con 0x7f209015dfd0 2023-10-16T09:44:44.864 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:44.864 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":22,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:02.785877+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:43:59.531648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:00.138849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f207c06af00 msgr2=0x7f207c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f207c06af00 0x7f207c06d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2080001750 tx=0x7f2080009040 comp rx=0 tx=0).stop 2023-10-16T09:44:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209015dfd0 msgr2=0x7f209015cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209015dfd0 0x7f209015cd20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20900716a0 tx=0x7f2088004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 shutdown_connections 2023-10-16T09:44:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f207c06af00 0x7f207c06d3c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209015dfd0 0x7f209015cd20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20900af450 0x7f2090159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 --2- 172.21.15.181:0/886736441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20900cd470 0x7f20901e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 >> 172.21.15.181:0/886736441 conn(0x7f20900d04a0 msgr2=0x7f209015bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 shutdown_connections 2023-10-16T09:44:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.864+0000 7f20960f5700 1 -- 172.21.15.181:0/886736441 wait complete. 2023-10-16T09:44:44.878 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T09:44:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.999+0000 7f6440eaa700 1 Processor -- start 2023-10-16T09:44:45.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:44.999+0000 7f6440eaa700 1 -- start start 2023-10-16T09:44:45.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f6440eaa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f6440eaa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f643c0d0fc0 0x7f643c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f6440eaa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c0ccf50 con 0x7f643c06a8a0 2023-10-16T09:44:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f6440eaa700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c0af9c0 con 0x7f643c0d0fc0 2023-10-16T09:44:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f6440eaa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0cc600 con 0x7f643c15df60 2023-10-16T09:44:45.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f643a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f6439d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f643c0d0fc0 0x7f643c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f643ad9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f643c06a8a0 0x7f643c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:60848/0 (socket says 172.21.15.181:60848) 2023-10-16T09:44:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f643ad9d700 1 -- 172.21.15.181:0/3869950721 learned_addr learned my addr 172.21.15.181:0/3869950721 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f643a59c700 1 -- 172.21.15.181:0/3869950721 >> v2:172.21.15.190:3300/0 conn(0x7f643c0d0fc0 msgr2=0x7f643c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f643a59c700 1 --2- 172.21.15.181:0/3869950721 >> v2:172.21.15.190:3300/0 conn(0x7f643c0d0fc0 0x7f643c0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f643a59c700 1 -- 172.21.15.181:0/3869950721 >> v1:172.21.15.190:6789/0 conn(0x7f643c06a8a0 legacy=0x7f643c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f643a59c700 1 -- 172.21.15.181:0/3869950721 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c0cc8a0 con 0x7f643c15df60 2023-10-16T09:44:45.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.000+0000 7f6439d9b700 1 --2- 172.21.15.181:0/3869950721 >> v2:172.21.15.190:3300/0 conn(0x7f643c0d0fc0 0x7f643c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:45.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f643a59c700 1 --2- 172.21.15.181:0/3869950721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0cae20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f642c009820 tx=0x7f642c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1831570e3dcff029 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:45.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f643959a700 1 -- 172.21.15.181:0/3869950721 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c005e30 con 0x7f643c15df60 2023-10-16T09:44:45.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f643959a700 1 -- 172.21.15.181:0/3869950721 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c006030 con 0x7f643c15df60 2023-10-16T09:44:45.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f643959a700 1 -- 172.21.15.181:0/3869950721 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0063c0 con 0x7f643c15df60 2023-10-16T09:44:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 -- 172.21.15.181:0/3869950721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 msgr2=0x7f643c0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 --2- 172.21.15.181:0/3869950721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0cae20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f642c009820 tx=0x7f642c009b30 comp rx=0 tx=0).stop 2023-10-16T09:44:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 -- 172.21.15.181:0/3869950721 shutdown_connections 2023-10-16T09:44:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 --2- 172.21.15.181:0/3869950721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0cae20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 --2- 172.21.15.181:0/3869950721 >> v2:172.21.15.190:3300/0 conn(0x7f643c0d0fc0 0x7f643c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 -- 172.21.15.181:0/3869950721 >> 172.21.15.181:0/3869950721 conn(0x7f643c0d04a0 msgr2=0x7f643c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 -- 172.21.15.181:0/3869950721 shutdown_connections 2023-10-16T09:44:45.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.001+0000 7f6440eaa700 1 -- 172.21.15.181:0/3869950721 wait complete. 2023-10-16T09:44:45.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 Processor -- start 2023-10-16T09:44:45.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 -- start start 2023-10-16T09:44:45.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f643c06a8a0 0x7f643c1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c0d0fc0 0x7f643c0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0c7820 con 0x7f643c06a8a0 2023-10-16T09:44:45.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0c50a0 con 0x7f643c0d0fc0 2023-10-16T09:44:45.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.003+0000 7f6440eaa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0c4420 con 0x7f643c15df60 2023-10-16T09:44:45.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f643c06a8a0 0x7f643c1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f643c06a8a0 0x7f643c1507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37600/0 (socket says 172.21.15.181:37600) 2023-10-16T09:44:45.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 -- 172.21.15.181:0/333280015 learned_addr learned my addr 172.21.15.181:0/333280015 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:45.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f6439d9b700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c0d0fc0 0x7f643c0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643ad9d700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 -- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 msgr2=0x7f643c0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 -- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c0d0fc0 msgr2=0x7f643c0c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c0d0fc0 0x7f643c0c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 -- 172.21.15.181:0/333280015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300094a0 con 0x7f643c06a8a0 2023-10-16T09:44:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f643a59c700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f643c06a8a0 0x7f643c1507a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f642c000c00 tx=0x7f642c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f642affd700 1 -- 172.21.15.181:0/333280015 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0063a0 con 0x7f643c06a8a0 2023-10-16T09:44:45.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f642affd700 1 -- 172.21.15.181:0/333280015 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c020070 con 0x7f643c06a8a0 2023-10-16T09:44:45.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c009430 con 0x7f643c06a8a0 2023-10-16T09:44:45.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f642affd700 1 -- 172.21.15.181:0/333280015 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c006760 con 0x7f643c06a8a0 2023-10-16T09:44:45.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.004+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c0c4cc0 con 0x7f643c06a8a0 2023-10-16T09:44:45.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.005+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c0c4810 con 0x7f643c06a8a0 2023-10-16T09:44:45.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.006+0000 7f642affd700 1 -- 172.21.15.181:0/333280015 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f642c022030 con 0x7f643c06a8a0 2023-10-16T09:44:45.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.006+0000 7f642affd700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f641c06af00 0x7f641c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.006+0000 7f642affd700 1 -- 172.21.15.181:0/333280015 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f642c08aa10 con 0x7f643c06a8a0 2023-10-16T09:44:45.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.006+0000 7f6439d9b700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f641c06af00 0x7f641c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.006+0000 7f6439d9b700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f641c06af00 0x7f641c06d3c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f643c0b4320 tx=0x7f6430009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.010+0000 7f642affd700 1 -- 172.21.15.181:0/333280015 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f642c055ad0 con 0x7f643c06a8a0 2023-10-16T09:44:45.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.246+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f643c067d20 con 0x7f643c06a8a0 2023-10-16T09:44:45.249 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T09:44:45.247+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T09:44:45.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.248+0000 7f642affd700 1 -- 172.21.15.181:0/333280015 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f642c0590f0 con 0x7f643c06a8a0 2023-10-16T09:44:45.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f641c06af00 msgr2=0x7f641c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f641c06af00 0x7f641c06d3c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f643c0b4320 tx=0x7f6430009210 comp rx=0 tx=0).stop 2023-10-16T09:44:45.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f643c06a8a0 msgr2=0x7f643c1507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f643c06a8a0 0x7f643c1507a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f642c000c00 tx=0x7f642c004750 comp rx=0 tx=0).stop 2023-10-16T09:44:45.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 shutdown_connections 2023-10-16T09:44:45.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f641c06af00 0x7f641c06d3c0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f643c0b4320 tx=0x7f6430009210 comp rx=0 tx=0).stop 2023-10-16T09:44:45.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15df60 0x7f643c0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f643c06a8a0 0x7f643c1507a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 --2- 172.21.15.181:0/333280015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f643c0d0fc0 0x7f643c0c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.251+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 >> 172.21.15.181:0/333280015 conn(0x7f643c0d04a0 msgr2=0x7f643c06e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:45.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.252+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 shutdown_connections 2023-10-16T09:44:45.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.252+0000 7f6440eaa700 1 -- 172.21.15.181:0/333280015 wait complete. 2023-10-16T09:44:45.258 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T09:44:45.266 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T09:44:45.292 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T09:44:45.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.386+0000 7f21f3779700 1 Processor -- start 2023-10-16T09:44:45.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.386+0000 7f21f3779700 1 -- start start 2023-10-16T09:44:45.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.387+0000 7f21f3779700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21ec06afb0 0x7f21ec1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.387+0000 7f21f3779700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ec0cef00 con 0x7f21ec072760 2023-10-16T09:44:45.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.387+0000 7f21f3779700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ec0cec20 con 0x7f21ec15da70 2023-10-16T09:44:45.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.387+0000 7f21f3779700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ec071c80 con 0x7f21ec06afb0 2023-10-16T09:44:45.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.387+0000 7f21f1d16700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21ec06afb0 0x7f21ec1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.387+0000 7f21f0d14700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f21ec072760 0x7f21ec06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48688/0 (socket says 172.21.15.181:48688) 2023-10-16T09:44:45.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.387+0000 7f21f0d14700 1 -- 172.21.15.181:0/2127115128 learned_addr learned my addr 172.21.15.181:0/2127115128 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:45.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f1d16700 1 -- 172.21.15.181:0/2127115128 >> v1:172.21.15.190:6789/0 conn(0x7f21ec15da70 legacy=0x7f21ec0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f1d16700 1 -- 172.21.15.181:0/2127115128 >> v1:172.21.15.181:6789/0 conn(0x7f21ec072760 legacy=0x7f21ec06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f1d16700 1 -- 172.21.15.181:0/2127115128 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec071f60 con 0x7f21ec06afb0 2023-10-16T09:44:45.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f1d16700 1 --2- 172.21.15.181:0/2127115128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1615f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009d90 tx=0x7f21e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717a3d895ad4e2e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:45.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21e3fff700 1 -- 172.21.15.181:0/2127115128 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e400e070 con 0x7f21ec06afb0 2023-10-16T09:44:45.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21e3fff700 1 -- 172.21.15.181:0/2127115128 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e400bbd0 con 0x7f21ec06afb0 2023-10-16T09:44:45.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21e3fff700 1 -- 172.21.15.181:0/2127115128 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4012430 con 0x7f21ec06afb0 2023-10-16T09:44:45.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f3779700 1 -- 172.21.15.181:0/2127115128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 msgr2=0x7f21ec1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f3779700 1 --2- 172.21.15.181:0/2127115128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1615f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009d90 tx=0x7f21e400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:45.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f3779700 1 -- 172.21.15.181:0/2127115128 shutdown_connections 2023-10-16T09:44:45.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f3779700 1 --2- 172.21.15.181:0/2127115128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1615f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.388+0000 7f21f3779700 1 -- 172.21.15.181:0/2127115128 >> 172.21.15.181:0/2127115128 conn(0x7f21ec0cd010 msgr2=0x7f21ec0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:45.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.389+0000 7f21f3779700 1 -- 172.21.15.181:0/2127115128 shutdown_connections 2023-10-16T09:44:45.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.389+0000 7f21f3779700 1 -- 172.21.15.181:0/2127115128 wait complete. 2023-10-16T09:44:45.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 Processor -- start 2023-10-16T09:44:45.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 -- start start 2023-10-16T09:44:45.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21ec072760 0x7f21ec1f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21ec15da70 0x7f21ec154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec0ad290 con 0x7f21ec06afb0 2023-10-16T09:44:45.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec0ad570 con 0x7f21ec15da70 2023-10-16T09:44:45.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.390+0000 7f21f3779700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec0ad850 con 0x7f21ec072760 2023-10-16T09:44:45.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37614/0 (socket says 172.21.15.181:37614) 2023-10-16T09:44:45.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 -- 172.21.15.181:0/128623189 learned_addr learned my addr 172.21.15.181:0/128623189 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:45.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 -- 172.21.15.181:0/128623189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21ec072760 msgr2=0x7f21ec1f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21ec072760 0x7f21ec1f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 -- 172.21.15.181:0/128623189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21ec15da70 msgr2=0x7f21ec154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:45.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1d16700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21ec15da70 0x7f21ec154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21ec15da70 0x7f21ec154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:45.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 -- 172.21.15.181:0/128623189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4009a70 con 0x7f21ec06afb0 2023-10-16T09:44:45.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f1515700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1f78e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00c4e0 tx=0x7f21dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:45.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21e1ffb700 1 -- 172.21.15.181:0/128623189 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0052d0 con 0x7f21ec06afb0 2023-10-16T09:44:45.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21e1ffb700 1 -- 172.21.15.181:0/128623189 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc005430 con 0x7f21ec06afb0 2023-10-16T09:44:45.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ec0adb30 con 0x7f21ec06afb0 2023-10-16T09:44:45.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21e1ffb700 1 -- 172.21.15.181:0/128623189 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0038c0 con 0x7f21ec06afb0 2023-10-16T09:44:45.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.391+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ec1f74f0 con 0x7f21ec06afb0 2023-10-16T09:44:45.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.392+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ec1f76b0 con 0x7f21ec06afb0 2023-10-16T09:44:45.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.392+0000 7f21e1ffb700 1 -- 172.21.15.181:0/128623189 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f21dc003a20 con 0x7f21ec06afb0 2023-10-16T09:44:45.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.393+0000 7f21e1ffb700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f21d006af00 0x7f21d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:45.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.393+0000 7f21e1ffb700 1 -- 172.21.15.181:0/128623189 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f21dc088ac0 con 0x7f21ec06afb0 2023-10-16T09:44:45.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.393+0000 7f21f0d14700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f21d006af00 0x7f21d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:45.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.393+0000 7f21f0d14700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f21d006af00 0x7f21d006d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f21ec1f6ca0 tx=0x7f21e8007ad0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:45.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.397+0000 7f21e1ffb700 1 -- 172.21.15.181:0/128623189 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21dc053ba0 con 0x7f21ec06afb0 2023-10-16T09:44:45.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:45.632+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f21ec072210 con 0x7f21ec06afb0 2023-10-16T09:44:46.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.253+0000 7f21e1ffb700 1 -- 172.21.15.181:0/128623189 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f21dc0571c0 con 0x7f21ec06afb0 2023-10-16T09:44:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f21d006af00 msgr2=0x7f21d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f21d006af00 0x7f21d006d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f21ec1f6ca0 tx=0x7f21e8007ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 msgr2=0x7f21ec1f78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:46.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1f78e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00c4e0 tx=0x7f21dc00c8a0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 shutdown_connections 2023-10-16T09:44:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f21d006af00 0x7f21d006d3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21ec15da70 0x7f21ec154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21ec06afb0 0x7f21ec1f78e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 --2- 172.21.15.181:0/128623189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21ec072760 0x7f21ec1f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.255+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 >> 172.21.15.181:0/128623189 conn(0x7f21ec0cd010 msgr2=0x7f21ec161560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.256+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 shutdown_connections 2023-10-16T09:44:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.256+0000 7f21f3779700 1 -- 172.21.15.181:0/128623189 wait complete. 2023-10-16T09:44:46.271 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T09:44:46.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 Processor -- start 2023-10-16T09:44:46.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 -- start start 2023-10-16T09:44:46.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb58815eea0 0x7fb5880b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:46.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5880b0f70 0x7fb5880ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:46.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5880ce640 0x7fb5880cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:46.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5880d23c0 con 0x7fb5880b0f70 2023-10-16T09:44:46.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5880cd990 con 0x7fb58815eea0 2023-10-16T09:44:46.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58f884700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880cdc70 con 0x7fb5880ce640 2023-10-16T09:44:46.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5880b0f70 0x7fb5880ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:46.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5880b0f70 0x7fb5880ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37624/0 (socket says 172.21.15.181:37624) 2023-10-16T09:44:46.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 -- 172.21.15.181:0/40916769 learned_addr learned my addr 172.21.15.181:0/40916769 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:46.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58de21700 1 --2- 172.21.15.181:0/40916769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5880ce640 0x7fb5880cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:46.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58d620700 1 --2- 172.21.15.181:0/40916769 >> v2:172.21.15.190:3300/0 conn(0x7fb58815eea0 0x7fb5880b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:46.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 -- 172.21.15.181:0/40916769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5880ce640 msgr2=0x7fb5880cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:46.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 --2- 172.21.15.181:0/40916769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5880ce640 0x7fb5880cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 -- 172.21.15.181:0/40916769 >> v2:172.21.15.190:3300/0 conn(0x7fb58815eea0 msgr2=0x7fb5880b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:46.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 --2- 172.21.15.181:0/40916769 >> v2:172.21.15.190:3300/0 conn(0x7fb58815eea0 0x7fb5880b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 -- 172.21.15.181:0/40916769 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5880cce60 con 0x7fb5880b0f70 2023-10-16T09:44:46.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58d620700 1 --2- 172.21.15.181:0/40916769 >> v2:172.21.15.190:3300/0 conn(0x7fb58815eea0 0x7fb5880b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:46.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.397+0000 7fb58ce1f700 1 --2- 172.21.15.181:0/40916769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880ce100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5880b4180 tx=0x7fb578009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698e2d70c45b6889 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:46.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb57ffff700 1 -- 172.21.15.181:0/40916769 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800b870 con 0x7fb5880b0f70 2023-10-16T09:44:46.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb57ffff700 1 -- 172.21.15.181:0/40916769 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57800b9d0 con 0x7fb5880b0f70 2023-10-16T09:44:46.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb57ffff700 1 -- 172.21.15.181:0/40916769 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800bd60 con 0x7fb5880b0f70 2023-10-16T09:44:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 -- 172.21.15.181:0/40916769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 msgr2=0x7fb5880ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 --2- 172.21.15.181:0/40916769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880ce100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb5880b4180 tx=0x7fb578009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 -- 172.21.15.181:0/40916769 shutdown_connections 2023-10-16T09:44:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 --2- 172.21.15.181:0/40916769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5880ce640 0x7fb5880cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 --2- 172.21.15.181:0/40916769 >> v2:172.21.15.190:3300/0 conn(0x7fb58815eea0 0x7fb5880b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 --2- 172.21.15.181:0/40916769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880ce100 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 -- 172.21.15.181:0/40916769 >> 172.21.15.181:0/40916769 conn(0x7fb5880d1610 msgr2=0x7fb5880d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 -- 172.21.15.181:0/40916769 shutdown_connections 2023-10-16T09:44:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.398+0000 7fb58f884700 1 -- 172.21.15.181:0/40916769 wait complete. 2023-10-16T09:44:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.399+0000 7fb58f884700 1 Processor -- start 2023-10-16T09:44:46.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.399+0000 7fb58f884700 1 -- start start 2023-10-16T09:44:46.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58f884700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:46.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58f884700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5880ce640 0x7fb588159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:46.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58f884700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58815eea0 0x7fb5880c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:46.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58f884700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880c5b90 con 0x7fb5880b0f70 2023-10-16T09:44:46.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58f884700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880c5e70 con 0x7fb5880ce640 2023-10-16T09:44:46.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58f884700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880b4a40 con 0x7fb58815eea0 2023-10-16T09:44:46.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58d620700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:46.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58d620700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37628/0 (socket says 172.21.15.181:37628) 2023-10-16T09:44:46.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58d620700 1 -- 172.21.15.181:0/1337832602 learned_addr learned my addr 172.21.15.181:0/1337832602 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:46.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58de21700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58815eea0 0x7fb5880c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:46.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58ce1f700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5880ce640 0x7fb588159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:46.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58d620700 1 -- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58815eea0 msgr2=0x7fb5880c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:46.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58d620700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58815eea0 0x7fb5880c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58d620700 1 -- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5880ce640 msgr2=0x7fb588159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:46.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.400+0000 7fb58d620700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5880ce640 0x7fb588159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:46.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.401+0000 7fb58d620700 1 -- 172.21.15.181:0/1337832602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578009550 con 0x7fb5880b0f70 2023-10-16T09:44:46.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.401+0000 7fb58d620700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880b5240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb58400e810 tx=0x7fb58400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:46.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.401+0000 7fb57dffb700 1 -- 172.21.15.181:0/1337832602 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58400c070 con 0x7fb5880b0f70 2023-10-16T09:44:46.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.401+0000 7fb57dffb700 1 -- 172.21.15.181:0/1337832602 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb584004230 con 0x7fb5880b0f70 2023-10-16T09:44:46.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.401+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5880b4d20 con 0x7fb5880b0f70 2023-10-16T09:44:46.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.401+0000 7fb57dffb700 1 -- 172.21.15.181:0/1337832602 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840036a0 con 0x7fb5880b0f70 2023-10-16T09:44:46.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.401+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5880b70a0 con 0x7fb5880b0f70 2023-10-16T09:44:46.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.402+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5880b5090 con 0x7fb5880b0f70 2023-10-16T09:44:46.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.402+0000 7fb57dffb700 1 -- 172.21.15.181:0/1337832602 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb584003800 con 0x7fb5880b0f70 2023-10-16T09:44:46.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.403+0000 7fb57dffb700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb56c06af00 0x7fb56c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:46.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.403+0000 7fb57dffb700 1 -- 172.21.15.181:0/1337832602 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb5840892b0 con 0x7fb5880b0f70 2023-10-16T09:44:46.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.403+0000 7fb58ce1f700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb56c06af00 0x7fb56c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:46.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.403+0000 7fb58ce1f700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb56c06af00 0x7fb56c06d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb578014040 tx=0x7fb578008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:46.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.408+0000 7fb57dffb700 1 -- 172.21.15.181:0/1337832602 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb584054300 con 0x7fb5880b0f70 2023-10-16T09:44:46.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:46.597+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb5880b48a0 con 0x7fb5880b0f70 2023-10-16T09:44:47.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.254+0000 7fb57dffb700 1 -- 172.21.15.181:0/1337832602 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb5880b48a0 con 0x7fb5880b0f70 2023-10-16T09:44:47.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb56c06af00 msgr2=0x7fb56c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb56c06af00 0x7fb56c06d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb578014040 tx=0x7fb578008030 comp rx=0 tx=0).stop 2023-10-16T09:44:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 msgr2=0x7fb5880b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880b5240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb58400e810 tx=0x7fb58400eb20 comp rx=0 tx=0).stop 2023-10-16T09:44:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 shutdown_connections 2023-10-16T09:44:47.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb56c06af00 0x7fb56c06d3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58815eea0 0x7fb5880c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5880b0f70 0x7fb5880b5240 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 --2- 172.21.15.181:0/1337832602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5880ce640 0x7fb588159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.258+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 >> 172.21.15.181:0/1337832602 conn(0x7fb5880d1610 msgr2=0x7fb5880d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:47.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.259+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 shutdown_connections 2023-10-16T09:44:47.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.259+0000 7fb58f884700 1 -- 172.21.15.181:0/1337832602 wait complete. 2023-10-16T09:44:47.263 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T09:44:47.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T09:44:47.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.404+0000 7fdbbaa23700 1 Processor -- start 2023-10-16T09:44:47.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.404+0000 7fdbbaa23700 1 -- start start 2023-10-16T09:44:47.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.405+0000 7fdbbaa23700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdbb415e610 0x7fdbb415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:47.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.405+0000 7fdbbaa23700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:47.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.405+0000 7fdbbaa23700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb406ed90 con 0x7fdbb40665a0 2023-10-16T09:44:47.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.405+0000 7fdbbaa23700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb415f260 con 0x7fdbb415e610 2023-10-16T09:44:47.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.405+0000 7fdbbaa23700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40ceb40 con 0x7fdbb40cf040 2023-10-16T09:44:47.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdbb415e610 0x7fdbb415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:47.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb37fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fdbb40665a0 0x7fdbb406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:60888/0 (socket says 172.21.15.181:60888) 2023-10-16T09:44:47.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb37fe700 1 -- 172.21.15.181:0/2022824421 learned_addr learned my addr 172.21.15.181:0/2022824421 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:47.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb8fc0700 1 --2- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:47.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb3fff700 1 -- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 msgr2=0x7fdbb40cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:47.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb3fff700 1 --2- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb3fff700 1 -- 172.21.15.181:0/2022824421 >> v1:172.21.15.190:6789/0 conn(0x7fdbb40665a0 legacy=0x7fdbb406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:47.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb3fff700 1 -- 172.21.15.181:0/2022824421 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb406e7d0 con 0x7fdbb415e610 2023-10-16T09:44:47.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb2ffd700 1 -- 172.21.15.181:0/2022824421 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752725742 0 0) 0x7fdbb406ed90 con 0x7fdbb40665a0 2023-10-16T09:44:47.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb3fff700 1 --2- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 0x7fdbb415e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009840 tx=0x7fdba4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e84f8b0fdb3292e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:47.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb2ffd700 1 -- 172.21.15.181:0/2022824421 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4011830 con 0x7fdbb415e610 2023-10-16T09:44:47.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb2ffd700 1 -- 172.21.15.181:0/2022824421 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba4011990 con 0x7fdbb415e610 2023-10-16T09:44:47.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.406+0000 7fdbb2ffd700 1 -- 172.21.15.181:0/2022824421 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4011d20 con 0x7fdbb415e610 2023-10-16T09:44:47.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 -- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 msgr2=0x7fdbb415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:47.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 0x7fdbb415e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009840 tx=0x7fdba4009b50 comp rx=0 tx=0).stop 2023-10-16T09:44:47.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 -- 172.21.15.181:0/2022824421 shutdown_connections 2023-10-16T09:44:47.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/2022824421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 0x7fdbb415e9f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 -- 172.21.15.181:0/2022824421 >> 172.21.15.181:0/2022824421 conn(0x7fdbb40b1780 msgr2=0x7fdbb40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:47.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 -- 172.21.15.181:0/2022824421 shutdown_connections 2023-10-16T09:44:47.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.407+0000 7fdbbaa23700 1 -- 172.21.15.181:0/2022824421 wait complete. 2023-10-16T09:44:47.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.408+0000 7fdbbaa23700 1 Processor -- start 2023-10-16T09:44:47.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.408+0000 7fdbbaa23700 1 -- start start 2023-10-16T09:44:47.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbbaa23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40665a0 0x7fdbb40b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:47.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbbaa23700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:47.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbbaa23700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 0x7fdbb409df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbbaa23700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41598b0 con 0x7fdbb40665a0 2023-10-16T09:44:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbbaa23700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4159e70 con 0x7fdbb415e610 2023-10-16T09:44:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbbaa23700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb409e830 con 0x7fdbb40cf040 2023-10-16T09:44:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40665a0 0x7fdbb40b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:47.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40665a0 0x7fdbb40b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37642/0 (socket says 172.21.15.181:37642) 2023-10-16T09:44:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb3fff700 1 -- 172.21.15.181:0/1190848795 learned_addr learned my addr 172.21.15.181:0/1190848795 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb8fc0700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 0x7fdbb409df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb37fe700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb3fff700 1 -- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 msgr2=0x7fdbb40a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:47.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb3fff700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb3fff700 1 -- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 msgr2=0x7fdbb409df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:47.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.409+0000 7fdbb3fff700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 0x7fdbb409df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:47.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.410+0000 7fdbb3fff700 1 -- 172.21.15.181:0/1190848795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40094a0 con 0x7fdbb40665a0 2023-10-16T09:44:47.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.410+0000 7fdbb3fff700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40665a0 0x7fdbb40b0780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdba40090b0 tx=0x7fdba4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:47.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.410+0000 7fdbb0ff9700 1 -- 172.21.15.181:0/1190848795 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4004550 con 0x7fdbb40665a0 2023-10-16T09:44:47.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.410+0000 7fdbb0ff9700 1 -- 172.21.15.181:0/1190848795 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba401e070 con 0x7fdbb40665a0 2023-10-16T09:44:47.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.410+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb415b350 con 0x7fdbb40665a0 2023-10-16T09:44:47.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.410+0000 7fdbb0ff9700 1 -- 172.21.15.181:0/1190848795 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40038e0 con 0x7fdbb40665a0 2023-10-16T09:44:47.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.410+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb4159b00 con 0x7fdbb40665a0 2023-10-16T09:44:47.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.411+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb4159c90 con 0x7fdbb40665a0 2023-10-16T09:44:47.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.411+0000 7fdbb0ff9700 1 -- 172.21.15.181:0/1190848795 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdba4003a40 con 0x7fdbb40665a0 2023-10-16T09:44:47.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.411+0000 7fdbb0ff9700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdba006af00 0x7fdba006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:47.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.412+0000 7fdbb0ff9700 1 -- 172.21.15.181:0/1190848795 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fdba408a310 con 0x7fdbb40665a0 2023-10-16T09:44:47.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.412+0000 7fdbb37fe700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdba006af00 0x7fdba006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:47.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.412+0000 7fdbb37fe700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdba006af00 0x7fdba006d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40c33d0 tx=0x7fdba8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:47.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.417+0000 7fdbb0ff9700 1 -- 172.21.15.181:0/1190848795 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdba40555b0 con 0x7fdbb40665a0 2023-10-16T09:44:47.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:47.610+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdbb406f7c0 con 0x7fdbb40665a0 2023-10-16T09:44:48.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.260+0000 7fdbb0ff9700 1 -- 172.21.15.181:0/1190848795 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdba4058bd0 con 0x7fdbb40665a0 2023-10-16T09:44:48.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdba006af00 msgr2=0x7fdba006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:48.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdba006af00 0x7fdba006d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40c33d0 tx=0x7fdba8007de0 comp rx=0 tx=0).stop 2023-10-16T09:44:48.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40665a0 msgr2=0x7fdbb40b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:48.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40665a0 0x7fdbb40b0780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdba40090b0 tx=0x7fdba4004750 comp rx=0 tx=0).stop 2023-10-16T09:44:48.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 shutdown_connections 2023-10-16T09:44:48.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdba006af00 0x7fdba006d3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:48.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb415e610 0x7fdbb409df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:48.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40665a0 0x7fdbb40b0780 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdba40090b0 tx=0x7fdba4004750 comp rx=0 tx=0).stop 2023-10-16T09:44:48.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 --2- 172.21.15.181:0/1190848795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbb40cf040 0x7fdbb40a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:48.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.265+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 >> 172.21.15.181:0/1190848795 conn(0x7fdbb40b1780 msgr2=0x7fdbb40ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:48.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.266+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 shutdown_connections 2023-10-16T09:44:48.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.268+0000 7fdbbaa23700 1 -- 172.21.15.181:0/1190848795 wait complete. 2023-10-16T09:44:48.271 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T09:44:48.287 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T09:44:48.288 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T09:44:48.288 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T09:44:48.288 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:44:48.289 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T09:44:48.289 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T09:44:48.289 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T09:44:48.289 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:44:48.290 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T09:44:48.290 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T09:44:48.290 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T09:44:48.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:44:48.291 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T09:44:48.291 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T09:44:48.292 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T09:44:48.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:44:48.298 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T09:44:48.299 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T09:44:48.300 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T09:44:48.300 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T09:44:48.303 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T09:44:48.303 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T09:44:48.305 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T09:44:48.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T09:44:48.341 INFO:tasks.ceph.mds.a:Started 2023-10-16T09:44:48.342 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:48.343 INFO:tasks.ceph.mds.c:Started 2023-10-16T09:44:48.344 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:48.347 INFO:tasks.ceph.mds.b:Started 2023-10-16T09:44:48.347 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:48.348 INFO:tasks.ceph.mds.d:Started 2023-10-16T09:44:48.349 DEBUG:teuthology.parallel:result is None 2023-10-16T09:44:48.349 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T09:44:48.424 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T09:44:48.424 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T09:44:48.425 INFO:tasks.ceph.mds.a.smithi181.stderr:2023-10-16T09:44:48.423+0000 7f8efe1e7700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-16T09:44:48.432 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T09:44:48.433 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T09:44:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.478+0000 7f74ca8c3700 1 Processor -- start 2023-10-16T09:44:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.478+0000 7f74ca8c3700 1 -- start start 2023-10-16T09:44:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.478+0000 7f74ca8c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40cfc60 0x7f74c40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74ca8c3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c40d24f0 con 0x7f74c415bd50 2023-10-16T09:44:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74ca8c3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c415c320 con 0x7f74c40b05d0 2023-10-16T09:44:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74ca8c3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c415ccc0 con 0x7f74c40cfc60 2023-10-16T09:44:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74bbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f74c415bd50 0x7f74c40d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48754/0 (socket says 172.21.15.181:48754) 2023-10-16T09:44:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74bbfff700 1 -- 172.21.15.181:0/1835553416 learned_addr learned my addr 172.21.15.181:0/1835553416 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c3fff700 1 --2- 172.21.15.181:0/1835553416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40cfc60 0x7f74c40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293264102 0 0) 0x7f74c40d24f0 con 0x7f74c415bd50 2023-10-16T09:44:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b00035b0 con 0x7f74c415bd50 2023-10-16T09:44:48.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295963643 0 0) 0x7f74c415c320 con 0x7f74c40b05d0 2023-10-16T09:44:48.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c40d24f0 con 0x7f74c40b05d0 2023-10-16T09:44:48.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c3fff700 1 -- 172.21.15.181:0/1835553416 >> v1:172.21.15.190:6789/0 conn(0x7f74c40b05d0 legacy=0x7f74c40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:48.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3466685646 0 0) 0x7f74b00035b0 con 0x7f74c415bd50 2023-10-16T09:44:48.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c3fff700 1 -- 172.21.15.181:0/1835553416 >> v1:172.21.15.181:6789/0 conn(0x7f74c415bd50 legacy=0x7f74c40d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:48.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c3fff700 1 -- 172.21.15.181:0/1835553416 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c415c600 con 0x7f74c40cfc60 2023-10-16T09:44:48.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c3fff700 1 --2- 172.21.15.181:0/1835553416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40cfc60 0x7f74c40d1fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009870 tx=0x7f74b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6fe561bcb98a51d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:48.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.479+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac001670 con 0x7f74c40cfc60 2023-10-16T09:44:48.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4004030 con 0x7f74c40cfc60 2023-10-16T09:44:48.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74c37fe700 1 -- 172.21.15.181:0/1835553416 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b40043c0 con 0x7f74c40cfc60 2023-10-16T09:44:48.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74ca8c3700 1 -- 172.21.15.181:0/1835553416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40cfc60 msgr2=0x7f74c40d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:48.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/1835553416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40cfc60 0x7f74c40d1fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009870 tx=0x7f74b4009b80 comp rx=0 tx=0).stop 2023-10-16T09:44:48.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74ca8c3700 1 -- 172.21.15.181:0/1835553416 shutdown_connections 2023-10-16T09:44:48.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/1835553416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40cfc60 0x7f74c40d1fb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:48.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74ca8c3700 1 -- 172.21.15.181:0/1835553416 >> 172.21.15.181:0/1835553416 conn(0x7f74c409a9a0 msgr2=0x7f74c40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:48.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74ca8c3700 1 -- 172.21.15.181:0/1835553416 shutdown_connections 2023-10-16T09:44:48.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.480+0000 7f74ca8c3700 1 -- 172.21.15.181:0/1835553416 wait complete. 2023-10-16T09:44:48.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.482+0000 7f74ca8c3700 1 Processor -- start 2023-10-16T09:44:48.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.482+0000 7f74ca8c3700 1 -- start start 2023-10-16T09:44:48.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40b05d0 0x7f74c41597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:48.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74c40cfc60 0x7f74c4157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:48.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c415bd50 0x7f74c406f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:48.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41580c0 con 0x7f74c415bd50 2023-10-16T09:44:48.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4158ab0 con 0x7f74c40cfc60 2023-10-16T09:44:48.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4159350 con 0x7f74c40b05d0 2023-10-16T09:44:48.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c415bd50 0x7f74c406f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:48.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c415bd50 0x7f74c406f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37684/0 (socket says 172.21.15.181:37684) 2023-10-16T09:44:48.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 -- 172.21.15.181:0/3703119404 learned_addr learned my addr 172.21.15.181:0/3703119404 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:48.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74bbfff700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74c40cfc60 0x7f74c4157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:48.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c3fff700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40b05d0 0x7f74c41597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:48.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 -- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40b05d0 msgr2=0x7f74c41597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:48.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40b05d0 0x7f74c41597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:48.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 -- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74c40cfc60 msgr2=0x7f74c4157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:48.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74c40cfc60 0x7f74c4157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:48.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 -- 172.21.15.181:0/3703119404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40094a0 con 0x7f74c415bd50 2023-10-16T09:44:48.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74bbfff700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74c40cfc60 0x7f74c4157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:48.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c3fff700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40b05d0 0x7f74c41597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:48.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c8e60700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c415bd50 0x7f74c406f880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00cc50 tx=0x7f74bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:48.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c17fa700 1 -- 172.21.15.181:0/3703119404 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00ada0 con 0x7f74c415bd50 2023-10-16T09:44:48.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c17fa700 1 -- 172.21.15.181:0/3703119404 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc00ce50 con 0x7f74c415bd50 2023-10-16T09:44:48.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c41f3ec0 con 0x7f74c415bd50 2023-10-16T09:44:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74c17fa700 1 -- 172.21.15.181:0/3703119404 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc003c40 con 0x7f74c415bd50 2023-10-16T09:44:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.483+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c4158fe0 con 0x7f74c415bd50 2023-10-16T09:44:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.484+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c4067d20 con 0x7f74c415bd50 2023-10-16T09:44:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.485+0000 7f74c17fa700 1 -- 172.21.15.181:0/3703119404 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f74bc0050d0 con 0x7f74c415bd50 2023-10-16T09:44:48.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.485+0000 7f74c17fa700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f74a406af00 0x7f74a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:48.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.485+0000 7f74c17fa700 1 -- 172.21.15.181:0/3703119404 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f74bc089420 con 0x7f74c415bd50 2023-10-16T09:44:48.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.485+0000 7f74c3fff700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f74a406af00 0x7f74a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:48.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.486+0000 7f74c3fff700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f74a406af00 0x7f74a406d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74c415a9e0 tx=0x7f74b4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:48.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.490+0000 7f74c17fa700 1 -- 172.21.15.181:0/3703119404 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f74bc0547d0 con 0x7f74c415bd50 2023-10-16T09:44:48.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:48.695+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f74c4159170 con 0x7f74c415bd50 2023-10-16T09:44:49.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.806+0000 7f74c17fa700 1 -- 172.21.15.181:0/3703119404 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f74bc057df0 con 0x7f74c415bd50 2023-10-16T09:44:49.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f74a406af00 msgr2=0x7f74a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:49.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f74a406af00 0x7f74a406d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74c415a9e0 tx=0x7f74b4008030 comp rx=0 tx=0).stop 2023-10-16T09:44:49.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c415bd50 msgr2=0x7f74c406f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:49.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c415bd50 0x7f74c406f880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00cc50 tx=0x7f74bc00a340 comp rx=0 tx=0).stop 2023-10-16T09:44:49.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 shutdown_connections 2023-10-16T09:44:49.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f74a406af00 0x7f74a406d3c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74c40b05d0 0x7f74c41597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c415bd50 0x7f74c406f880 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 --2- 172.21.15.181:0/3703119404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74c40cfc60 0x7f74c4157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 >> 172.21.15.181:0/3703119404 conn(0x7f74c409a9a0 msgr2=0x7f74c40cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:49.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 shutdown_connections 2023-10-16T09:44:49.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.809+0000 7f74ca8c3700 1 -- 172.21.15.181:0/3703119404 wait complete. 2023-10-16T09:44:49.813 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T09:44:49.825 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T09:44:49.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.951+0000 7f7c4eb0d700 1 Processor -- start 2023-10-16T09:44:49.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.951+0000 7f7c4eb0d700 1 -- start start 2023-10-16T09:44:49.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c4eb0d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c480cd2d0 0x7f7c480ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c4eb0d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c480cc510 con 0x7f7c480d0370 2023-10-16T09:44:49.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c4eb0d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c480cc7f0 con 0x7f7c4815b990 2023-10-16T09:44:49.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c4eb0d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c480ccad0 con 0x7f7c480cd2d0 2023-10-16T09:44:49.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c47fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7c480d0370 0x7f7c480ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48772/0 (socket says 172.21.15.181:48772) 2023-10-16T09:44:49.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c47fff700 1 -- 172.21.15.181:0/1312332003 learned_addr learned my addr 172.21.15.181:0/1312332003 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:49.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c4d0aa700 1 --2- 172.21.15.181:0/1312332003 >> v2:172.21.15.190:3300/0 conn(0x7f7c480cd2d0 0x7f7c480ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:49.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846192038 0 0) 0x7f7c480cc510 con 0x7f7c480d0370 2023-10-16T09:44:49.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c380035b0 con 0x7f7c480d0370 2023-10-16T09:44:49.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397935776 0 0) 0x7f7c380035b0 con 0x7f7c480d0370 2023-10-16T09:44:49.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 >> v2:172.21.15.190:3300/0 conn(0x7f7c480cd2d0 msgr2=0x7f7c480ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:49.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 --2- 172.21.15.181:0/1312332003 >> v2:172.21.15.190:3300/0 conn(0x7f7c480cd2d0 0x7f7c480ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 >> v1:172.21.15.190:6789/0 conn(0x7f7c4815b990 legacy=0x7f7c480cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:49.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c480ccdb0 con 0x7f7c480d0370 2023-10-16T09:44:49.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7c34001670 con 0x7f7c480d0370 2023-10-16T09:44:49.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 >> v1:172.21.15.181:6789/0 conn(0x7f7c480d0370 legacy=0x7f7c480ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:49.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c38003a00 0x7f7c38003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.952+0000 7f7c477fe700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c38004850 0x7f7c38004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c477fe700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38005210 0x7f7c3800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c480ccad0 con 0x7f7c38003a00 2023-10-16T09:44:49.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3800c0c0 con 0x7f7c38004850 2023-10-16T09:44:49.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3800c280 con 0x7f7c38005210 2023-10-16T09:44:49.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4c8a9700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c38003a00 0x7f7c38003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:49.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c47fff700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c38004850 0x7f7c38004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:49.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4d0aa700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38005210 0x7f7c3800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:49.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4c8a9700 1 -- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38005210 msgr2=0x7f7c3800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:49.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4c8a9700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38005210 0x7f7c3800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4c8a9700 1 -- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c38004850 msgr2=0x7f7c38004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:49.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4c8a9700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c38004850 0x7f7c38004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4c8a9700 1 -- 172.21.15.181:0/1312332003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380035b0 con 0x7f7c38003a00 2023-10-16T09:44:49.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4d0aa700 1 -- 172.21.15.181:0/1312332003 reap_dead start 2023-10-16T09:44:49.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c47fff700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c38004850 0x7f7c38004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:49.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c4c8a9700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c38003a00 0x7f7c38003e20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c001820 tx=0x7f7c38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a2d6e3fd0b959 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:49.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0012e0 con 0x7f7c38003a00 2023-10-16T09:44:49.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c0052d0 con 0x7f7c38003a00 2023-10-16T09:44:49.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.953+0000 7f7c477fe700 1 -- 172.21.15.181:0/1312332003 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0036d0 con 0x7f7c38003a00 2023-10-16T09:44:49.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.954+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/1312332003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c38003a00 msgr2=0x7f7c38003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:49.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.954+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c38003a00 0x7f7c38003e20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c001820 tx=0x7f7c38005880 comp rx=0 tx=0).stop 2023-10-16T09:44:49.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.954+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/1312332003 shutdown_connections 2023-10-16T09:44:49.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.954+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/1312332003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c38003a00 0x7f7c38003e20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.954+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/1312332003 >> 172.21.15.181:0/1312332003 conn(0x7f7c4809a9a0 msgr2=0x7f7c480d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:49.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.954+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/1312332003 shutdown_connections 2023-10-16T09:44:49.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.954+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/1312332003 wait complete. 2023-10-16T09:44:49.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.955+0000 7f7c4eb0d700 1 Processor -- start 2023-10-16T09:44:49.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.955+0000 7f7c4eb0d700 1 -- start start 2023-10-16T09:44:49.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c4eb0d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38003a00 0x7f7c480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c4eb0d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c481591e0 0x7f7c480b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c4eb0d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c4815e2a0 0x7f7c4815e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c4eb0d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4815ec40 con 0x7f7c481591e0 2023-10-16T09:44:49.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c4eb0d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c480b6920 con 0x7f7c4815e2a0 2023-10-16T09:44:49.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c4eb0d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c480b5a90 con 0x7f7c38003a00 2023-10-16T09:44:49.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c481591e0 0x7f7c480b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:49.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c481591e0 0x7f7c480b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37704/0 (socket says 172.21.15.181:37704) 2023-10-16T09:44:49.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 -- 172.21.15.181:0/853871908 learned_addr learned my addr 172.21.15.181:0/853871908 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:49.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 -- 172.21.15.181:0/853871908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38003a00 msgr2=0x7f7c480b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:49.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38003a00 0x7f7c480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 -- 172.21.15.181:0/853871908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c4815e2a0 msgr2=0x7f7c4815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:49.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c4815e2a0 0x7f7c4815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:49.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 -- 172.21.15.181:0/853871908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c480b6db0 con 0x7f7c481591e0 2023-10-16T09:44:49.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c47fff700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c481591e0 0x7f7c480b5550 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d7a0 tx=0x7f7c3400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:49.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.956+0000 7f7c457fa700 1 -- 172.21.15.181:0/853871908 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34004c80 con 0x7f7c481591e0 2023-10-16T09:44:49.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.957+0000 7f7c457fa700 1 -- 172.21.15.181:0/853871908 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c34004de0 con 0x7f7c481591e0 2023-10-16T09:44:49.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.957+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c480b5d10 con 0x7f7c481591e0 2023-10-16T09:44:49.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.957+0000 7f7c457fa700 1 -- 172.21.15.181:0/853871908 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c340072f0 con 0x7f7c481591e0 2023-10-16T09:44:49.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.957+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c480b62d0 con 0x7f7c481591e0 2023-10-16T09:44:49.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.957+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c48067d20 con 0x7f7c481591e0 2023-10-16T09:44:49.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.958+0000 7f7c457fa700 1 -- 172.21.15.181:0/853871908 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7c34007450 con 0x7f7c481591e0 2023-10-16T09:44:49.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.958+0000 7f7c457fa700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c2c06af00 0x7f7c2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:49.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.958+0000 7f7c457fa700 1 -- 172.21.15.181:0/853871908 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7c34088d00 con 0x7f7c481591e0 2023-10-16T09:44:49.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.958+0000 7f7c4c8a9700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c2c06af00 0x7f7c2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:49.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.959+0000 7f7c4c8a9700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c2c06af00 0x7f7c2c06d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c000ef0 tx=0x7f7c3c011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:49.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:49.963+0000 7f7c457fa700 1 -- 172.21.15.181:0/853871908 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7c34054100 con 0x7f7c481591e0 2023-10-16T09:44:50.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.236+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7c480b6a60 con 0x7f7c481591e0 2023-10-16T09:44:50.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.236+0000 7f7c457fa700 1 -- 172.21.15.181:0/853871908 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f7c480b6a60 con 0x7f7c481591e0 2023-10-16T09:44:50.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:50.238 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:44:50.238 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T09:44:50.238 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.239 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T09:44:50.239 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T09:44:50.239 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.239 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.240 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:44:50.240 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:44:50.240 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.240 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.241 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T09:44:50.241 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T09:44:50.241 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.241 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.242 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:44:50.242 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:44:50.242 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.242 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.243 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T09:44:50.243 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T09:44:50.243 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.243 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.244 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:44:50.244 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:44:50.244 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.244 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.245 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.245 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.245 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T09:44:50.245 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T09:44:50.245 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.245 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.246 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:44:50.246 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:44:50.246 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.246 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T09:44:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T09:44:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.247 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T09:44:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T09:44:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.249 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.250 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T09:44:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T09:44:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.251 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T09:44:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T09:44:50.252 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.254 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T09:44:50.254 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T09:44:50.254 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.254 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.254 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.255 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.255 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.255 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T09:44:50.256 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T09:44:50.256 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.256 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.256 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.256 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.256 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.257 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T09:44:50.257 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T09:44:50.257 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.257 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.258 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.258 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.258 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.259 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T09:44:50.259 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T09:44:50.259 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.259 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:44:50.259 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:44:50.259 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.260 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.260 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T09:44:50.260 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T09:44:50.260 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.261 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.261 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:44:50.261 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:44:50.261 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:44:50.262 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.262 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T09:44:50.262 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T09:44:50.262 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.263 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.263 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:44:50.263 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:44:50.263 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:44:50.264 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.264 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T09:44:50.265 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T09:44:50.265 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.265 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.265 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:44:50.265 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:44:50.266 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:44:50.266 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.266 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.266 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T09:44:50.266 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T09:44:50.267 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.267 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:44:50.267 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:44:50.267 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:44:50.267 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:44:50.268 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.268 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T09:44:50.268 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T09:44:50.268 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.269 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:44:50.269 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:44:50.269 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:44:50.269 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.269 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.270 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-mds", 2023-10-16T09:44:50.270 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDQBS1lMwF3ARAAD+7WNNzCmerR/WsBO/4A7A==", 2023-10-16T09:44:50.270 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.270 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-mds" 2023-10-16T09:44:50.271 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.271 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-mgr", 2023-10-16T09:44:50.271 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDQBS1lxwp3ARAAjEYvDEbHdJI+RDyybEt/Jw==", 2023-10-16T09:44:50.272 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.272 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-16T09:44:50.272 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.272 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.273 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-osd", 2023-10-16T09:44:50.273 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDQBS1l1RN3ARAAoP7asiVxyjvTqn11yoPOAg==", 2023-10-16T09:44:50.273 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.273 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-osd" 2023-10-16T09:44:50.273 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.274 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rbd", 2023-10-16T09:44:50.274 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDQBS1lYBt3ARAAwMDyPkte5P4criHvpuJMPQ==", 2023-10-16T09:44:50.274 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.275 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-16T09:44:50.275 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.275 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.275 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-16T09:44:50.275 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDQBS1l4SJ3ARAAG/mI16l5UmUwoBzZIdL2Wg==", 2023-10-16T09:44:50.276 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.276 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-16T09:44:50.276 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.277 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rgw", 2023-10-16T09:44:50.277 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDQBS1lPSp3ARAARqZO2eF4vn3ZWSKZof8bXw==", 2023-10-16T09:44:50.277 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.277 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-16T09:44:50.277 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.278 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.278 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T09:44:50.278 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T09:44:50.278 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.278 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:44:50.279 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:44:50.279 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.279 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:44:50.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:50.279 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T09:44:50.280 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T09:44:50.280 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:44:50.280 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:44:50.280 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:44:50.280 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:44:50.280 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.281 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:50.281 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T09:44:50.281 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:44:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c2c06af00 msgr2=0x7f7c2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c2c06af00 0x7f7c2c06d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c000ef0 tx=0x7f7c3c011000 comp rx=0 tx=0).stop 2023-10-16T09:44:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c481591e0 msgr2=0x7f7c480b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c481591e0 0x7f7c480b5550 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d7a0 tx=0x7f7c3400db60 comp rx=0 tx=0).stop 2023-10-16T09:44:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 shutdown_connections 2023-10-16T09:44:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c2c06af00 0x7f7c2c06d3c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c38003a00 0x7f7c480b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c4815e2a0 0x7f7c4815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 --2- 172.21.15.181:0/853871908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c481591e0 0x7f7c480b5550 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 >> 172.21.15.181:0/853871908 conn(0x7f7c4809a9a0 msgr2=0x7f7c480ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 shutdown_connections 2023-10-16T09:44:50.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.239+0000 7f7c4eb0d700 1 -- 172.21.15.181:0/853871908 wait complete. 2023-10-16T09:44:50.286 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-16T09:44:50.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.407+0000 7f8b41eac700 1 Processor -- start 2023-10-16T09:44:50.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.407+0000 7f8b41eac700 1 -- start start 2023-10-16T09:44:50.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b41eac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8b3c0cd950 0x7f8b3c0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b41eac700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b3c0cf980 0x7f8b3c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b41eac700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c06b370 con 0x7f8b3c15b9f0 2023-10-16T09:44:50.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b41eac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c06b800 con 0x7f8b3c0cd950 2023-10-16T09:44:50.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b41eac700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cde00 con 0x7f8b3c0cf980 2023-10-16T09:44:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8b3c0cd950 0x7f8b3c0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8b3c0cd950 0x7f8b3c0cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37712/0 (socket says 172.21.15.181:37712) 2023-10-16T09:44:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 -- 172.21.15.181:0/277460033 learned_addr learned my addr 172.21.15.181:0/277460033 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8b3c15b9f0 0x7f8b3c0cc400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48790/0 (socket says 172.21.15.181:48790) 2023-10-16T09:44:50.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 -- 172.21.15.181:0/277460033 >> v2:172.21.15.190:3300/0 conn(0x7f8b3c0cf980 msgr2=0x7f8b3c0ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:50.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 --2- 172.21.15.181:0/277460033 >> v2:172.21.15.190:3300/0 conn(0x7f8b3c0cf980 0x7f8b3c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 -- 172.21.15.181:0/277460033 >> v1:172.21.15.181:6789/0 conn(0x7f8b3c15b9f0 legacy=0x7f8b3c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 -- 172.21.15.181:0/277460033 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0cfdc0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3a7fc700 1 -- 172.21.15.181:0/277460033 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122761556 0 0) 0x7f8b3c06b370 con 0x7f8b3c15b9f0 2023-10-16T09:44:50.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3affd700 1 --2- 172.21.15.181:0/277460033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c0cf440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c070580 tx=0x7f8b24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c1c93a9c008a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:50.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3a7fc700 1 -- 172.21.15.181:0/277460033 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2400b860 con 0x7f8b3c0cd950 2023-10-16T09:44:50.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.408+0000 7f8b3a7fc700 1 -- 172.21.15.181:0/277460033 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2400b9c0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b3a7fc700 1 -- 172.21.15.181:0/277460033 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2400bd50 con 0x7f8b3c0cd950 2023-10-16T09:44:50.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 -- 172.21.15.181:0/277460033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 msgr2=0x7f8b3c0cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 --2- 172.21.15.181:0/277460033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c0cf440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c070580 tx=0x7f8b24009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:50.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 -- 172.21.15.181:0/277460033 shutdown_connections 2023-10-16T09:44:50.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 --2- 172.21.15.181:0/277460033 >> v2:172.21.15.190:3300/0 conn(0x7f8b3c0cf980 0x7f8b3c0ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 --2- 172.21.15.181:0/277460033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c0cf440 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 -- 172.21.15.181:0/277460033 >> 172.21.15.181:0/277460033 conn(0x7f8b3c0d04a0 msgr2=0x7f8b3c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:50.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 -- 172.21.15.181:0/277460033 shutdown_connections 2023-10-16T09:44:50.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.409+0000 7f8b41eac700 1 -- 172.21.15.181:0/277460033 wait complete. 2023-10-16T09:44:50.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.411+0000 7f8b41eac700 1 Processor -- start 2023-10-16T09:44:50.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.411+0000 7f8b41eac700 1 -- start start 2023-10-16T09:44:50.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b41eac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c1e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b41eac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b3c0cf980 0x7f8b3c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b41eac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b3c15b9f0 0x7f8b3c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b41eac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0b42e0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b41eac700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c155380 con 0x7f8b3c15b9f0 2023-10-16T09:44:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b41eac700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c06f660 con 0x7f8b3c0cf980 2023-10-16T09:44:50.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c1e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c1e5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37714/0 (socket says 172.21.15.181:37714) 2023-10-16T09:44:50.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 -- 172.21.15.181:0/257695704 learned_addr learned my addr 172.21.15.181:0/257695704 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:50.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3affd700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b3c0cf980 0x7f8b3c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 -- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b3c0cf980 msgr2=0x7f8b3c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b3c0cf980 0x7f8b3c154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3bfff700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b3c15b9f0 0x7f8b3c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 -- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b3c15b9f0 msgr2=0x7f8b3c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b3c15b9f0 0x7f8b3c0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 -- 172.21.15.181:0/257695704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24009550 con 0x7f8b3c0cd950 2023-10-16T09:44:50.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3bfff700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b3c15b9f0 0x7f8b3c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:50.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b3b7fe700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c1e5100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c4a0 tx=0x7f8b2c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:50.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b33fff700 1 -- 172.21.15.181:0/257695704 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0074e0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b33fff700 1 -- 172.21.15.181:0/257695704 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c007640 con 0x7f8b3c0cd950 2023-10-16T09:44:50.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c1e57e0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.412+0000 7f8b33fff700 1 -- 172.21.15.181:0/257695704 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c005a50 con 0x7f8b3c0cd950 2023-10-16T09:44:50.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.413+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c1e5da0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.413+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c0b4420 con 0x7f8b3c0cd950 2023-10-16T09:44:50.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.414+0000 7f8b33fff700 1 -- 172.21.15.181:0/257695704 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8b2c005bb0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.414+0000 7f8b33fff700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8b1c06af00 0x7f8b1c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.414+0000 7f8b33fff700 1 -- 172.21.15.181:0/257695704 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8b2c088790 con 0x7f8b3c0cd950 2023-10-16T09:44:50.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.414+0000 7f8b3affd700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8b1c06af00 0x7f8b1c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.415+0000 7f8b3affd700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8b1c06af00 0x7f8b1c06d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0b3f90 tx=0x7f8b24008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:50.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.418+0000 7f8b33fff700 1 -- 172.21.15.181:0/257695704 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8b2c053b90 con 0x7f8b3c0cd950 2023-10-16T09:44:50.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.689+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f8b3c067d20 con 0x7f8b3c0cd950 2023-10-16T09:44:50.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.691+0000 7f8b33fff700 1 -- 172.21.15.181:0/257695704 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8b2c0170a0 con 0x7f8b3c0cd950 2023-10-16T09:44:50.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8b1c06af00 msgr2=0x7f8b1c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8b1c06af00 0x7f8b1c06d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0b3f90 tx=0x7f8b24008030 comp rx=0 tx=0).stop 2023-10-16T09:44:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 msgr2=0x7f8b3c1e5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c1e5100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c4a0 tx=0x7f8b2c00c860 comp rx=0 tx=0).stop 2023-10-16T09:44:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 shutdown_connections 2023-10-16T09:44:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8b1c06af00 0x7f8b1c06d3c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b3c15b9f0 0x7f8b3c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b3c0cd950 0x7f8b3c1e5100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 --2- 172.21.15.181:0/257695704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b3c0cf980 0x7f8b3c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 >> 172.21.15.181:0/257695704 conn(0x7f8b3c0d04a0 msgr2=0x7f8b3c0b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 shutdown_connections 2023-10-16T09:44:50.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.693+0000 7f8b41eac700 1 -- 172.21.15.181:0/257695704 wait complete. 2023-10-16T09:44:50.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-16T09:44:50.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.826+0000 7f7e77ceb700 1 Processor -- start 2023-10-16T09:44:50.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.826+0000 7f7e77ceb700 1 -- start start 2023-10-16T09:44:50.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.827+0000 7f7e77ceb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e700cfc60 0x7f7e700d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.827+0000 7f7e77ceb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e700b05d0 0x7f7e700cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.827+0000 7f7e77ceb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700d24f0 con 0x7f7e7015bd50 2023-10-16T09:44:50.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.827+0000 7f7e77ceb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7015ccc0 con 0x7f7e700b05d0 2023-10-16T09:44:50.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.827+0000 7f7e77ceb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7015c320 con 0x7f7e700cfc60 2023-10-16T09:44:50.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e75286700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e7015bd50 0x7f7e700d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48800/0 (socket says 172.21.15.181:48800) 2023-10-16T09:44:50.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e75286700 1 -- 172.21.15.181:0/3635824827 learned_addr learned my addr 172.21.15.181:0/3635824827 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:50.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e76288700 1 --2- 172.21.15.181:0/3635824827 >> v2:172.21.15.181:3300/0 conn(0x7f7e700b05d0 0x7f7e700cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e75a87700 1 --2- 172.21.15.181:0/3635824827 >> v2:172.21.15.190:3300/0 conn(0x7f7e700cfc60 0x7f7e700d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e74a85700 1 -- 172.21.15.181:0/3635824827 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065154940 0 0) 0x7f7e700d24f0 con 0x7f7e7015bd50 2023-10-16T09:44:50.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e74a85700 1 -- 172.21.15.181:0/3635824827 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e5c0035b0 con 0x7f7e7015bd50 2023-10-16T09:44:50.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e76288700 1 -- 172.21.15.181:0/3635824827 >> v2:172.21.15.190:3300/0 conn(0x7f7e700cfc60 msgr2=0x7f7e700d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:50.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e76288700 1 --2- 172.21.15.181:0/3635824827 >> v2:172.21.15.190:3300/0 conn(0x7f7e700cfc60 0x7f7e700d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e76288700 1 -- 172.21.15.181:0/3635824827 >> v1:172.21.15.181:6789/0 conn(0x7f7e7015bd50 legacy=0x7f7e700d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e74a85700 1 -- 172.21.15.181:0/3635824827 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408123056 0 0) 0x7f7e5c0035b0 con 0x7f7e7015bd50 2023-10-16T09:44:50.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e76288700 1 -- 172.21.15.181:0/3635824827 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7015c600 con 0x7f7e700b05d0 2023-10-16T09:44:50.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e76288700 1 --2- 172.21.15.181:0/3635824827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e700b05d0 0x7f7e700cc200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e7008bc90 tx=0x7f7e6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ea4c1fe6b2b0f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:50.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e74a85700 1 -- 172.21.15.181:0/3635824827 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800e070 con 0x7f7e700b05d0 2023-10-16T09:44:50.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e74a85700 1 -- 172.21.15.181:0/3635824827 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68003a20 con 0x7f7e700b05d0 2023-10-16T09:44:50.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.828+0000 7f7e74a85700 1 -- 172.21.15.181:0/3635824827 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60001670 con 0x7f7e700b05d0 2023-10-16T09:44:50.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3635824827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e700b05d0 msgr2=0x7f7e700cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3635824827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e700b05d0 0x7f7e700cc200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e7008bc90 tx=0x7f7e6800b330 comp rx=0 tx=0).stop 2023-10-16T09:44:50.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3635824827 shutdown_connections 2023-10-16T09:44:50.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3635824827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e700b05d0 0x7f7e700cc200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3635824827 >> v2:172.21.15.190:3300/0 conn(0x7f7e700cfc60 0x7f7e700d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3635824827 >> 172.21.15.181:0/3635824827 conn(0x7f7e7009a9a0 msgr2=0x7f7e700d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:50.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3635824827 shutdown_connections 2023-10-16T09:44:50.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.829+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3635824827 wait complete. 2023-10-16T09:44:50.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.830+0000 7f7e77ceb700 1 Processor -- start 2023-10-16T09:44:50.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.830+0000 7f7e77ceb700 1 -- start start 2023-10-16T09:44:50.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e77ceb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e700b05d0 0x7f7e7014aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e77ceb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e700cfc60 0x7f7e700c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e77ceb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7015bd50 0x7f7e700b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e77ceb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7006bf30 con 0x7f7e7015bd50 2023-10-16T09:44:50.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e77ceb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700ca610 con 0x7f7e700b05d0 2023-10-16T09:44:50.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e77ceb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700ca050 con 0x7f7e700cfc60 2023-10-16T09:44:50.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7015bd50 0x7f7e700b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7015bd50 0x7f7e700b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37730/0 (socket says 172.21.15.181:37730) 2023-10-16T09:44:50.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 -- 172.21.15.181:0/3255490972 learned_addr learned my addr 172.21.15.181:0/3255490972 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:50.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e75a87700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e700b05d0 0x7f7e7014aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e75286700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e700cfc60 0x7f7e700c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 -- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e700cfc60 msgr2=0x7f7e700c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e700cfc60 0x7f7e700c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 -- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e700b05d0 msgr2=0x7f7e7014aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e700b05d0 0x7f7e7014aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 -- 172.21.15.181:0/3255490972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68009a70 con 0x7f7e7015bd50 2023-10-16T09:44:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.831+0000 7f7e76288700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7015bd50 0x7f7e700b1a00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b020 tx=0x7f7e68004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:50.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.832+0000 7f7e667fc700 1 -- 172.21.15.181:0/3255490972 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68004670 con 0x7f7e7015bd50 2023-10-16T09:44:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.832+0000 7f7e667fc700 1 -- 172.21.15.181:0/3255490972 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e680047d0 con 0x7f7e7015bd50 2023-10-16T09:44:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.832+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e700ca330 con 0x7f7e7015bd50 2023-10-16T09:44:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.832+0000 7f7e667fc700 1 -- 172.21.15.181:0/3255490972 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68012670 con 0x7f7e7015bd50 2023-10-16T09:44:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.832+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7014af60 con 0x7f7e7015bd50 2023-10-16T09:44:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.832+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7006b9b0 con 0x7f7e7015bd50 2023-10-16T09:44:50.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.833+0000 7f7e667fc700 1 -- 172.21.15.181:0/3255490972 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e680127d0 con 0x7f7e7015bd50 2023-10-16T09:44:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.833+0000 7f7e667fc700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5406af00 0x7f7e5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.833+0000 7f7e667fc700 1 -- 172.21.15.181:0/3255490972 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7e6808b350 con 0x7f7e7015bd50 2023-10-16T09:44:50.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.833+0000 7f7e75a87700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5406af00 0x7f7e5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:50.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.834+0000 7f7e75a87700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5406af00 0x7f7e5406d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009b70 tx=0x7f7e6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:50.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:50.838+0000 7f7e667fc700 1 -- 172.21.15.181:0/3255490972 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e7006b9b0 con 0x7f7e7015bd50 2023-10-16T09:44:51.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.108+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f7e7006b9b0 con 0x7f7e7015bd50 2023-10-16T09:44:51.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.111+0000 7f7e667fc700 1 -- 172.21.15.181:0/3255490972 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7e7006b9b0 con 0x7f7e7015bd50 2023-10-16T09:44:51.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5406af00 msgr2=0x7f7e5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5406af00 0x7f7e5406d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009b70 tx=0x7f7e6c008040 comp rx=0 tx=0).stop 2023-10-16T09:44:51.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7015bd50 msgr2=0x7f7e700b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7015bd50 0x7f7e700b1a00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b020 tx=0x7f7e68004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 shutdown_connections 2023-10-16T09:44:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5406af00 0x7f7e5406d3c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7015bd50 0x7f7e700b1a00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e700b05d0 0x7f7e7014aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 --2- 172.21.15.181:0/3255490972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e700cfc60 0x7f7e700c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 >> 172.21.15.181:0/3255490972 conn(0x7f7e7009a9a0 msgr2=0x7f7e7015fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 shutdown_connections 2023-10-16T09:44:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.113+0000 7f7e77ceb700 1 -- 172.21.15.181:0/3255490972 wait complete. 2023-10-16T09:44:51.126 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-16T09:44:51.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.246+0000 7efe032f4700 1 Processor -- start 2023-10-16T09:44:51.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.246+0000 7efe032f4700 1 -- start start 2023-10-16T09:44:51.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe032f4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdfc15df60 0x7efdfc0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe032f4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc06ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe032f4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc0cddf0 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe032f4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc06b290 con 0x7efdfc15df60 2023-10-16T09:44:51.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe032f4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0af9c0 con 0x7efdfc06a8a0 2023-10-16T09:44:51.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01090700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdfc15df60 0x7efdfc0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01891700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc06ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01090700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdfc15df60 0x7efdfc0cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:42784/0 (socket says 172.21.15.181:42784) 2023-10-16T09:44:51.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01891700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc06ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:34762/0 (socket says 172.21.15.181:34762) 2023-10-16T09:44:51.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01090700 1 -- 172.21.15.181:0/951847059 learned_addr learned my addr 172.21.15.181:0/951847059 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:51.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01891700 1 -- 172.21.15.181:0/951847059 >> v2:172.21.15.190:3300/0 conn(0x7efdfc15df60 msgr2=0x7efdfc0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:51.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01891700 1 --2- 172.21.15.181:0/951847059 >> v2:172.21.15.190:3300/0 conn(0x7efdfc15df60 0x7efdfc0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01891700 1 -- 172.21.15.181:0/951847059 >> v1:172.21.15.190:6789/0 conn(0x7efdfc0d0fc0 legacy=0x7efdfc0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.247+0000 7efe01891700 1 -- 172.21.15.181:0/951847059 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0ccf50 con 0x7efdfc06a8a0 2023-10-16T09:44:51.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe01090700 1 --2- 172.21.15.181:0/951847059 >> v2:172.21.15.190:3300/0 conn(0x7efdfc15df60 0x7efdfc0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:51.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe01891700 1 --2- 172.21.15.181:0/951847059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc06ad50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009d90 tx=0x7efdf400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6fe03dd216aa1e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:51.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efdf3fff700 1 -- 172.21.15.181:0/951847059 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf400e070 con 0x7efdfc06a8a0 2023-10-16T09:44:51.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efdf3fff700 1 -- 172.21.15.181:0/951847059 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf400bb20 con 0x7efdfc06a8a0 2023-10-16T09:44:51.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efdf3fff700 1 -- 172.21.15.181:0/951847059 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4012430 con 0x7efdfc06a8a0 2023-10-16T09:44:51.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe032f4700 1 -- 172.21.15.181:0/951847059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 msgr2=0x7efdfc06ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe032f4700 1 --2- 172.21.15.181:0/951847059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc06ad50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009d90 tx=0x7efdf400b330 comp rx=0 tx=0).stop 2023-10-16T09:44:51.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe032f4700 1 -- 172.21.15.181:0/951847059 shutdown_connections 2023-10-16T09:44:51.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe032f4700 1 --2- 172.21.15.181:0/951847059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc06ad50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe032f4700 1 --2- 172.21.15.181:0/951847059 >> v2:172.21.15.190:3300/0 conn(0x7efdfc15df60 0x7efdfc0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.248+0000 7efe032f4700 1 -- 172.21.15.181:0/951847059 >> 172.21.15.181:0/951847059 conn(0x7efdfc0d04a0 msgr2=0x7efdfc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:51.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.249+0000 7efe032f4700 1 -- 172.21.15.181:0/951847059 shutdown_connections 2023-10-16T09:44:51.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.249+0000 7efe032f4700 1 -- 172.21.15.181:0/951847059 wait complete. 2023-10-16T09:44:51.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 Processor -- start 2023-10-16T09:44:51.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 -- start start 2023-10-16T09:44:51.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc1e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0d0fc0 0x7efdfc147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdfc15df60 0x7efdfc147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1e4120 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1e3510 con 0x7efdfc15df60 2023-10-16T09:44:51.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.250+0000 7efe032f4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1e37f0 con 0x7efdfc06a8a0 2023-10-16T09:44:51.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0d0fc0 0x7efdfc147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0d0fc0 0x7efdfc147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37740/0 (socket says 172.21.15.181:37740) 2023-10-16T09:44:51.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 -- 172.21.15.181:0/1296192621 learned_addr learned my addr 172.21.15.181:0/1296192621 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:51.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe01891700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdfc15df60 0x7efdfc147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe01090700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc1e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 -- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 msgr2=0x7efdfc1e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc1e45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 -- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdfc15df60 msgr2=0x7efdfc147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdfc15df60 0x7efdfc147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 -- 172.21.15.181:0/1296192621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8009500 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe0088f700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0d0fc0 0x7efdfc147100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c4e0 tx=0x7efdec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:51.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efdf1ffb700 1 -- 172.21.15.181:0/1296192621 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0052d0 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efdf1ffb700 1 -- 172.21.15.181:0/1296192621 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec005430 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efdf1ffb700 1 -- 172.21.15.181:0/1296192621 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0038c0 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4009a70 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.251+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc1e3db0 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.252+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc1e4290 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.253+0000 7efdf1ffb700 1 -- 172.21.15.181:0/1296192621 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efdec003a20 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.253+0000 7efdf1ffb700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efde006af30 0x7efde006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.253+0000 7efdf1ffb700 1 -- 172.21.15.181:0/1296192621 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efdec088580 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.253+0000 7efe01090700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efde006af30 0x7efde006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.253+0000 7efe01090700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efde006af30 0x7efde006d3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdf8006fd0 tx=0x7efdf8009290 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:51.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.257+0000 7efdf1ffb700 1 -- 172.21.15.181:0/1296192621 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efdec053a10 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.529+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7efdfc067d20 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.531+0000 7efdf1ffb700 1 -- 172.21.15.181:0/1296192621 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7efdec057030 con 0x7efdfc0d0fc0 2023-10-16T09:44:51.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efde006af30 msgr2=0x7efde006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efde006af30 0x7efde006d3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdf8006fd0 tx=0x7efdf8009290 comp rx=0 tx=0).stop 2023-10-16T09:44:51.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0d0fc0 msgr2=0x7efdfc147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0d0fc0 0x7efdfc147100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c4e0 tx=0x7efdec00c7f0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 shutdown_connections 2023-10-16T09:44:51.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7efde006af30 0x7efde006d3f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdfc06a8a0 0x7efdfc1e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdfc15df60 0x7efdfc147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 --2- 172.21.15.181:0/1296192621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0d0fc0 0x7efdfc147100 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 >> 172.21.15.181:0/1296192621 conn(0x7efdfc0d04a0 msgr2=0x7efdfc15bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:51.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 shutdown_connections 2023-10-16T09:44:51.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.534+0000 7efe032f4700 1 -- 172.21.15.181:0/1296192621 wait complete. 2023-10-16T09:44:51.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-16T09:44:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.667+0000 7f7cb122e700 1 Processor -- start 2023-10-16T09:44:51.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.667+0000 7f7cb122e700 1 -- start start 2023-10-16T09:44:51.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7cb122e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7cac15e460 0x7f7cac0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7cb122e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac0cb110 0x7f7cac15e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7cb122e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac06ed00 con 0x7f7cac15a0f0 2023-10-16T09:44:51.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7cb122e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac15ed80 con 0x7f7cac15e460 2023-10-16T09:44:51.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7cb122e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06e0f0 con 0x7f7cac0cb110 2023-10-16T09:44:51.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7caad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7cac15e460 0x7f7cac0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7caad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7cac15e460 0x7f7cac0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37746/0 (socket says 172.21.15.181:37746) 2023-10-16T09:44:51.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7caad9d700 1 -- 172.21.15.181:0/915134919 learned_addr learned my addr 172.21.15.181:0/915134919 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:51.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.668+0000 7f7caa59c700 1 --2- 172.21.15.181:0/915134919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac0cb110 0x7f7cac15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7caad9d700 1 -- 172.21.15.181:0/915134919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac0cb110 msgr2=0x7f7cac15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:51.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7caad9d700 1 --2- 172.21.15.181:0/915134919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac0cb110 0x7f7cac15e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7caad9d700 1 -- 172.21.15.181:0/915134919 >> v1:172.21.15.190:6789/0 conn(0x7f7cac15a0f0 legacy=0x7f7cac15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7caad9d700 1 -- 172.21.15.181:0/915134919 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac06e3d0 con 0x7f7cac15e460 2023-10-16T09:44:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7caad9d700 1 --2- 172.21.15.181:0/915134919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0d0fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c53f6ae678ce55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7ca9d9b700 1 -- 172.21.15.181:0/915134919 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0005e30 con 0x7f7cac15e460 2023-10-16T09:44:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7ca9d9b700 1 -- 172.21.15.181:0/915134919 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0004030 con 0x7f7cac15e460 2023-10-16T09:44:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7ca9d9b700 1 -- 172.21.15.181:0/915134919 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00043e0 con 0x7f7cac15e460 2023-10-16T09:44:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 -- 172.21.15.181:0/915134919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 msgr2=0x7f7cac0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 --2- 172.21.15.181:0/915134919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0d0fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0006fd0 tx=0x7f7ca0009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 -- 172.21.15.181:0/915134919 shutdown_connections 2023-10-16T09:44:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 --2- 172.21.15.181:0/915134919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0d0fc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 --2- 172.21.15.181:0/915134919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac0cb110 0x7f7cac15e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 -- 172.21.15.181:0/915134919 >> 172.21.15.181:0/915134919 conn(0x7f7cac0d04a0 msgr2=0x7f7cac0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:51.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 -- 172.21.15.181:0/915134919 shutdown_connections 2023-10-16T09:44:51.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.669+0000 7f7cb122e700 1 -- 172.21.15.181:0/915134919 wait complete. 2023-10-16T09:44:51.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.671+0000 7f7cb122e700 1 Processor -- start 2023-10-16T09:44:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.671+0000 7f7cb122e700 1 -- start start 2023-10-16T09:44:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cac0cb110 0x7f7cac0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac15a0f0 0x7f7cac0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0b16b0 con 0x7f7cac15e460 2023-10-16T09:44:51.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0b3ab0 con 0x7f7cac0cb110 2023-10-16T09:44:51.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c40a0 con 0x7f7cac15a0f0 2023-10-16T09:44:51.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37754/0 (socket says 172.21.15.181:37754) 2023-10-16T09:44:51.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 -- 172.21.15.181:0/2267967518 learned_addr learned my addr 172.21.15.181:0/2267967518 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:51.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7caa59c700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac15a0f0 0x7f7cac0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7caad9d700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cac0cb110 0x7f7cac0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 -- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac15a0f0 msgr2=0x7f7cac0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac15a0f0 0x7f7cac0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 -- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cac0cb110 msgr2=0x7f7cac0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cac0cb110 0x7f7cac0b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 -- 172.21.15.181:0/2267967518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0009500 con 0x7f7cac15e460 2023-10-16T09:44:51.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7caad9d700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cac0cb110 0x7f7cac0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:51.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cab59e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0b1b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400cc50 tx=0x7f7ca400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:51.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7c937fe700 1 -- 172.21.15.181:0/2267967518 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4003c00 con 0x7f7cac15e460 2023-10-16T09:44:51.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7c937fe700 1 -- 172.21.15.181:0/2267967518 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4003d60 con 0x7f7cac15e460 2023-10-16T09:44:51.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7c937fe700 1 -- 172.21.15.181:0/2267967518 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400ab60 con 0x7f7cac15e460 2023-10-16T09:44:51.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac0b1060 con 0x7f7cac15e460 2023-10-16T09:44:51.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.672+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac0c3d90 con 0x7f7cac15e460 2023-10-16T09:44:51.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.673+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac06f170 con 0x7f7cac15e460 2023-10-16T09:44:51.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.674+0000 7f7c937fe700 1 -- 172.21.15.181:0/2267967518 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ca400acc0 con 0x7f7cac15e460 2023-10-16T09:44:51.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.674+0000 7f7c937fe700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c9806af00 0x7f7c9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:51.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.674+0000 7f7c937fe700 1 -- 172.21.15.181:0/2267967518 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ca40893b0 con 0x7f7cac15e460 2023-10-16T09:44:51.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.674+0000 7f7caad9d700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c9806af00 0x7f7c9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:51.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.675+0000 7f7caad9d700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c9806af00 0x7f7c9806d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009130 tx=0x7f7ca0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:51.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.679+0000 7f7c937fe700 1 -- 172.21.15.181:0/2267967518 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7cac06f170 con 0x7f7cac15e460 2023-10-16T09:44:51.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.952+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f7cac06f170 con 0x7f7cac15e460 2023-10-16T09:44:51.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.954+0000 7f7c937fe700 1 -- 172.21.15.181:0/2267967518 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7cac06f170 con 0x7f7cac15e460 2023-10-16T09:44:51.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c9806af00 msgr2=0x7f7c9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c9806af00 0x7f7c9806d3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009130 tx=0x7f7ca0008070 comp rx=0 tx=0).stop 2023-10-16T09:44:51.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 msgr2=0x7f7cac0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:51.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0b1b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400cc50 tx=0x7f7ca400cf60 comp rx=0 tx=0).stop 2023-10-16T09:44:51.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 shutdown_connections 2023-10-16T09:44:51.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7c9806af00 0x7f7c9806d3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cac15e460 0x7f7cac0b1b40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cac0cb110 0x7f7cac0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 --2- 172.21.15.181:0/2267967518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cac15a0f0 0x7f7cac0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:51.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 >> 172.21.15.181:0/2267967518 conn(0x7f7cac0d04a0 msgr2=0x7f7cac0ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:51.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 shutdown_connections 2023-10-16T09:44:51.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:51.956+0000 7f7cb122e700 1 -- 172.21.15.181:0/2267967518 wait complete. 2023-10-16T09:44:51.969 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-16T09:44:52.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.090+0000 7fa8e0d8f700 1 Processor -- start 2023-10-16T09:44:52.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.090+0000 7fa8e0d8f700 1 -- start start 2023-10-16T09:44:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.091+0000 7fa8e0d8f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa8dc15df60 0x7fa8dc0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.091+0000 7fa8e0d8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 0x7fa8dc0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.091+0000 7fa8e0d8f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc0ccf50 con 0x7fa8dc06a8a0 2023-10-16T09:44:52.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.091+0000 7fa8e0d8f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0af9c0 con 0x7fa8dc15df60 2023-10-16T09:44:52.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.091+0000 7fa8e0d8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0cc600 con 0x7fa8dc0d0fc0 2023-10-16T09:44:52.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa8dc06a8a0 0x7fa8dc06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48840/0 (socket says 172.21.15.181:48840) 2023-10-16T09:44:52.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 -- 172.21.15.181:0/1323607143 learned_addr learned my addr 172.21.15.181:0/1323607143 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8da59c700 1 --2- 172.21.15.181:0/1323607143 >> v2:172.21.15.190:3300/0 conn(0x7fa8dc15df60 0x7fa8dc0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d9d9b700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 0x7fa8dc0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496513934 0 0) 0x7fa8dc0ccf50 con 0x7fa8dc06a8a0 2023-10-16T09:44:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8c40035b0 con 0x7fa8dc06a8a0 2023-10-16T09:44:52.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071682614 0 0) 0x7fa8c40035b0 con 0x7fa8dc06a8a0 2023-10-16T09:44:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 msgr2=0x7fa8dc0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 0x7fa8dc0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 >> v2:172.21.15.190:3300/0 conn(0x7fa8dc15df60 msgr2=0x7fa8dc0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 --2- 172.21.15.181:0/1323607143 >> v2:172.21.15.190:3300/0 conn(0x7fa8dc15df60 0x7fa8dc0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0cc8a0 con 0x7fa8dc06a8a0 2023-10-16T09:44:52.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa8d40019f0 con 0x7fa8dc06a8a0 2023-10-16T09:44:52.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 >> v1:172.21.15.181:6789/0 conn(0x7fa8dc06a8a0 legacy=0x7fa8dc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8c4003a00 0x7fa8c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 0x7fa8c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0cc600 con 0x7fa8c4004d90 2023-10-16T09:44:52.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c400ba10 con 0x7fa8c4003a00 2023-10-16T09:44:52.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c400bb50 con 0x7fa8dc0d0fc0 2023-10-16T09:44:52.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8da59c700 1 --2- 172.21.15.181:0/1323607143 >> v2:172.21.15.190:3300/0 conn(0x7fa8dc15df60 0x7fa8dc0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:52.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d9d9b700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 0x7fa8c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8da59c700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8c4003a00 0x7fa8c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 -- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 msgr2=0x7fa8c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:52.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 0x7fa8c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 -- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8c4003a00 msgr2=0x7fa8c4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:52.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8c4003a00 0x7fa8c4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8dad9d700 1 -- 172.21.15.181:0/1323607143 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40035b0 con 0x7fa8c4004d90 2023-10-16T09:44:52.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.092+0000 7fa8d9d9b700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0d0fc0 0x7fa8c4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:52.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8dad9d700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005210 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400a860 tx=0x7fa8c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977dbd82a25537ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4012600 con 0x7fa8c4004d90 2023-10-16T09:44:52.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4012780 con 0x7fa8c4004d90 2023-10-16T09:44:52.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8d959a700 1 -- 172.21.15.181:0/1323607143 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4012b30 con 0x7fa8c4004d90 2023-10-16T09:44:52.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/1323607143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 msgr2=0x7fa8c4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8e0d8f700 1 --2- 172.21.15.181:0/1323607143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8c4005210 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400a860 tx=0x7fa8c4005880 comp rx=0 tx=0).stop 2023-10-16T09:44:52.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8dad9d700 1 -- 172.21.15.181:0/1323607143 reap_dead start 2023-10-16T09:44:52.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/1323607143 shutdown_connections 2023-10-16T09:44:52.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/1323607143 >> 172.21.15.181:0/1323607143 conn(0x7fa8dc0d04a0 msgr2=0x7fa8dc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:52.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/1323607143 shutdown_connections 2023-10-16T09:44:52.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.093+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/1323607143 wait complete. 2023-10-16T09:44:52.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.095+0000 7fa8e0d8f700 1 Processor -- start 2023-10-16T09:44:52.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.095+0000 7fa8e0d8f700 1 -- start start 2023-10-16T09:44:52.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.095+0000 7fa8e0d8f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8dc1489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8e0d8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8dc0aff00 0x7fa8dc0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8e0d8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0b1f40 0x7fa8dc0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8e0d8f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0b08c0 con 0x7fa8c4004d90 2023-10-16T09:44:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8e0d8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc15f750 con 0x7fa8dc0aff00 2023-10-16T09:44:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8e0d8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0b2c70 con 0x7fa8dc0b1f40 2023-10-16T09:44:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8dc1489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8dc1489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37770/0 (socket says 172.21.15.181:37770) 2023-10-16T09:44:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 -- 172.21.15.181:0/3862293570 learned_addr learned my addr 172.21.15.181:0/3862293570 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8d9d9b700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8dc0aff00 0x7fa8dc0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8dad9d700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0b1f40 0x7fa8dc0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 -- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0b1f40 msgr2=0x7fa8dc0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0b1f40 0x7fa8dc0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 -- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8dc0aff00 msgr2=0x7fa8dc0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8dc0aff00 0x7fa8dc0b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 -- 172.21.15.181:0/3862293570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc15ee20 con 0x7fa8c4004d90 2023-10-16T09:44:52.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8d9d9b700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8dc0aff00 0x7fa8dc0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:52.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8da59c700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8dc1489e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0011930 tx=0x7fa8d0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8caffd700 1 -- 172.21.15.181:0/3862293570 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000f8d0 con 0x7fa8c4004d90 2023-10-16T09:44:52.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8caffd700 1 -- 172.21.15.181:0/3862293570 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d000fa30 con 0x7fa8c4004d90 2023-10-16T09:44:52.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8caffd700 1 -- 172.21.15.181:0/3862293570 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000fda0 con 0x7fa8c4004d90 2023-10-16T09:44:52.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc15f100 con 0x7fa8c4004d90 2023-10-16T09:44:52.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.096+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc0b2900 con 0x7fa8c4004d90 2023-10-16T09:44:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.097+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc067d20 con 0x7fa8c4004d90 2023-10-16T09:44:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.098+0000 7fa8caffd700 1 -- 172.21.15.181:0/3862293570 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa8d0015460 con 0x7fa8c4004d90 2023-10-16T09:44:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.098+0000 7fa8caffd700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa8bc06af00 0x7fa8bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.098+0000 7fa8caffd700 1 -- 172.21.15.181:0/3862293570 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8d0089c90 con 0x7fa8c4004d90 2023-10-16T09:44:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.098+0000 7fa8d9d9b700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa8bc06af00 0x7fa8bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.098+0000 7fa8d9d9b700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa8bc06af00 0x7fa8bc06d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0d0380 tx=0x7fa8cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.102+0000 7fa8caffd700 1 -- 172.21.15.181:0/3862293570 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa8d0055090 con 0x7fa8c4004d90 2023-10-16T09:44:52.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.371+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa8dc0af9c0 con 0x7fa8c4004d90 2023-10-16T09:44:52.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.373+0000 7fa8caffd700 1 -- 172.21.15.181:0/3862293570 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa8d00586b0 con 0x7fa8c4004d90 2023-10-16T09:44:52.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa8bc06af00 msgr2=0x7fa8bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa8bc06af00 0x7fa8bc06d3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0d0380 tx=0x7fa8cc008040 comp rx=0 tx=0).stop 2023-10-16T09:44:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 msgr2=0x7fa8dc1489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8dc1489e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0011930 tx=0x7fa8d0011cf0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 shutdown_connections 2023-10-16T09:44:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fa8bc06af00 0x7fa8bc06d3c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8dc0b1f40 0x7fa8dc0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8c4004d90 0x7fa8dc1489e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 --2- 172.21.15.181:0/3862293570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8dc0aff00 0x7fa8dc0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 >> 172.21.15.181:0/3862293570 conn(0x7fa8dc0d04a0 msgr2=0x7fa8dc06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:52.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 shutdown_connections 2023-10-16T09:44:52.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.375+0000 7fa8e0d8f700 1 -- 172.21.15.181:0/3862293570 wait complete. 2023-10-16T09:44:52.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-16T09:44:52.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.508+0000 7f260e151700 1 Processor -- start 2023-10-16T09:44:52.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.509+0000 7f260e151700 1 -- start start 2023-10-16T09:44:52.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.509+0000 7f260e151700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f260815b9f0 0x7f26080cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.509+0000 7f260e151700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080cc400 0x7f26080cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.509+0000 7f260e151700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260806b370 con 0x7f26080cf980 2023-10-16T09:44:52.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.509+0000 7f260e151700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260806b800 con 0x7f260815b9f0 2023-10-16T09:44:52.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.509+0000 7f260e151700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26080cfdc0 con 0x7f26080cc400 2023-10-16T09:44:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.509+0000 7f2607fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f26080cf980 0x7f26080ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48860/0 (socket says 172.21.15.181:48860) 2023-10-16T09:44:52.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f2607fff700 1 -- 172.21.15.181:0/1086799601 learned_addr learned my addr 172.21.15.181:0/1086799601 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:52.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26077fe700 1 --2- 172.21.15.181:0/1086799601 >> v2:172.21.15.190:3300/0 conn(0x7f260815b9f0 0x7f26080cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f2606ffd700 1 --2- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080cc400 0x7f26080cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26067fc700 1 -- 172.21.15.181:0/1086799601 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810634533 0 0) 0x7f260806b370 con 0x7f26080cf980 2023-10-16T09:44:52.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26067fc700 1 -- 172.21.15.181:0/1086799601 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25f00035b0 con 0x7f26080cf980 2023-10-16T09:44:52.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26077fe700 1 -- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080cc400 msgr2=0x7f26080cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:52.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26077fe700 1 --2- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080cc400 0x7f26080cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26077fe700 1 -- 172.21.15.181:0/1086799601 >> v1:172.21.15.181:6789/0 conn(0x7f26080cf980 legacy=0x7f26080ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26067fc700 1 -- 172.21.15.181:0/1086799601 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 727020728 0 0) 0x7f25f00035b0 con 0x7f26080cf980 2023-10-16T09:44:52.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26077fe700 1 -- 172.21.15.181:0/1086799601 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260806aa40 con 0x7f260815b9f0 2023-10-16T09:44:52.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26077fe700 1 --2- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f260815b9f0 0x7f26080cd950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2600009de0 tx=0x7f260000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e18d62d28c7f4a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26067fc700 1 -- 172.21.15.181:0/1086799601 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260000e070 con 0x7f260815b9f0 2023-10-16T09:44:52.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26067fc700 1 -- 172.21.15.181:0/1086799601 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600003a20 con 0x7f260815b9f0 2023-10-16T09:44:52.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.510+0000 7f26067fc700 1 -- 172.21.15.181:0/1086799601 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8001670 con 0x7f260815b9f0 2023-10-16T09:44:52.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 -- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f260815b9f0 msgr2=0x7f26080cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 --2- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f260815b9f0 0x7f26080cd950 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2600009de0 tx=0x7f260000b330 comp rx=0 tx=0).stop 2023-10-16T09:44:52.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 -- 172.21.15.181:0/1086799601 shutdown_connections 2023-10-16T09:44:52.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 --2- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f260815b9f0 0x7f26080cd950 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 --2- 172.21.15.181:0/1086799601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26080cc400 0x7f26080cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 -- 172.21.15.181:0/1086799601 >> 172.21.15.181:0/1086799601 conn(0x7f26080d04a0 msgr2=0x7f26080d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:52.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 -- 172.21.15.181:0/1086799601 shutdown_connections 2023-10-16T09:44:52.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.511+0000 7f260e151700 1 -- 172.21.15.181:0/1086799601 wait complete. 2023-10-16T09:44:52.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.512+0000 7f260e151700 1 Processor -- start 2023-10-16T09:44:52.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.512+0000 7f260e151700 1 -- start start 2023-10-16T09:44:52.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f260e151700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080cc400 0x7f260814c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f260e151700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26080cf980 0x7f26081e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f260e151700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260815b9f0 0x7f26081e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f260e151700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608158fa0 con 0x7f26080cf980 2023-10-16T09:44:52.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f260e151700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26080b3f00 con 0x7f26080cc400 2023-10-16T09:44:52.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f260e151700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26080b41e0 con 0x7f260815b9f0 2023-10-16T09:44:52.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2606ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26080cf980 0x7f26081e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2606ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26080cf980 0x7f26081e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37786/0 (socket says 172.21.15.181:37786) 2023-10-16T09:44:52.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2606ffd700 1 -- 172.21.15.181:0/2927836287 learned_addr learned my addr 172.21.15.181:0/2927836287 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:52.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f26077fe700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080cc400 0x7f260814c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2607fff700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260815b9f0 0x7f26081e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2606ffd700 1 -- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260815b9f0 msgr2=0x7f26081e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2606ffd700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260815b9f0 0x7f26081e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2606ffd700 1 -- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080cc400 msgr2=0x7f260814c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.513+0000 7f2606ffd700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080cc400 0x7f260814c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f2606ffd700 1 -- 172.21.15.181:0/2927836287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2600009a70 con 0x7f26080cf980 2023-10-16T09:44:52.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f2607fff700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260815b9f0 0x7f26081e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:52.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f26077fe700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080cc400 0x7f260814c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:52.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f2606ffd700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26080cf980 0x7f26081e5660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9b0 tx=0x7f25fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f25effff700 1 -- 172.21.15.181:0/2927836287 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc009710 con 0x7f26080cf980 2023-10-16T09:44:52.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f25effff700 1 -- 172.21.15.181:0/2927836287 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc00c850 con 0x7f26080cf980 2023-10-16T09:44:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26080b44c0 con 0x7f26080cf980 2023-10-16T09:44:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f25effff700 1 -- 172.21.15.181:0/2927836287 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00cbc0 con 0x7f26080cf980 2023-10-16T09:44:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.514+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260814ce40 con 0x7f26080cf980 2023-10-16T09:44:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.515+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2608067d20 con 0x7f26080cf980 2023-10-16T09:44:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.515+0000 7f25effff700 1 -- 172.21.15.181:0/2927836287 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25fc00cd20 con 0x7f26080cf980 2023-10-16T09:44:52.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.516+0000 7f25effff700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f25f406af00 0x7f25f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.516+0000 7f25effff700 1 -- 172.21.15.181:0/2927836287 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25fc089380 con 0x7f26080cf980 2023-10-16T09:44:52.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.516+0000 7f26077fe700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f25f406af00 0x7f25f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.516+0000 7f26077fe700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f25f406af00 0x7f25f406d3c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2600000f80 tx=0x7f2600011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.520+0000 7f25effff700 1 -- 172.21.15.181:0/2927836287 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f25fc054780 con 0x7f26080cf980 2023-10-16T09:44:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.791+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f26080cf5b0 con 0x7f26080cf980 2023-10-16T09:44:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.792+0000 7f25effff700 1 -- 172.21.15.181:0/2927836287 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f25fc057da0 con 0x7f26080cf980 2023-10-16T09:44:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f25f406af00 msgr2=0x7f25f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f25f406af00 0x7f25f406d3c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2600000f80 tx=0x7f2600011000 comp rx=0 tx=0).stop 2023-10-16T09:44:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26080cf980 msgr2=0x7f26081e5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26080cf980 0x7f26081e5660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9b0 tx=0x7f25fc00ed70 comp rx=0 tx=0).stop 2023-10-16T09:44:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 shutdown_connections 2023-10-16T09:44:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f25f406af00 0x7f25f406d3c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f260815b9f0 0x7f26081e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26080cc400 0x7f260814c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 --2- 172.21.15.181:0/2927836287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26080cf980 0x7f26081e5660 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 >> 172.21.15.181:0/2927836287 conn(0x7f26080d04a0 msgr2=0x7f260815aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 shutdown_connections 2023-10-16T09:44:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.795+0000 7f260e151700 1 -- 172.21.15.181:0/2927836287 wait complete. 2023-10-16T09:44:52.808 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:52.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.928+0000 7fe97df81700 1 Processor -- start 2023-10-16T09:44:52.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.928+0000 7fe97df81700 1 -- start start 2023-10-16T09:44:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe97df81700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe97815dfd0 0x7fe9780af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe97df81700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe9780cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe97df81700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97806f870 con 0x7fe9780af450 2023-10-16T09:44:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe97df81700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780cde40 con 0x7fe97815dfd0 2023-10-16T09:44:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe97df81700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780d1160 con 0x7fe9780cd470 2023-10-16T09:44:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe9777fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe97815dfd0 0x7fe9780af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe976ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe9780af450 0x7fe97806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48872/0 (socket says 172.21.15.181:48872) 2023-10-16T09:44:52.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe976ffd700 1 -- 172.21.15.181:0/3974953727 learned_addr learned my addr 172.21.15.181:0/3974953727 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:52.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe977fff700 1 --2- 172.21.15.181:0/3974953727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe9780cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe9777fe700 1 -- 172.21.15.181:0/3974953727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 msgr2=0x7fe9780cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:52.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe9777fe700 1 --2- 172.21.15.181:0/3974953727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe9780cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe9777fe700 1 -- 172.21.15.181:0/3974953727 >> v1:172.21.15.181:6789/0 conn(0x7fe9780af450 legacy=0x7fe97806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe9777fe700 1 -- 172.21.15.181:0/3974953727 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780cc8e0 con 0x7fe97815dfd0 2023-10-16T09:44:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe9767fc700 1 -- 172.21.15.181:0/3974953727 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268943205 0 0) 0x7fe97806f870 con 0x7fe9780af450 2023-10-16T09:44:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.929+0000 7fe977fff700 1 --2- 172.21.15.181:0/3974953727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe9780cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe9777fe700 1 --2- 172.21.15.181:0/3974953727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97815dfd0 0x7fe9780af9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe960009820 tx=0x7fe960009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67282c4b60818d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe9767fc700 1 -- 172.21.15.181:0/3974953727 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960005e30 con 0x7fe97815dfd0 2023-10-16T09:44:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe9767fc700 1 -- 172.21.15.181:0/3974953727 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960006030 con 0x7fe97815dfd0 2023-10-16T09:44:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe9767fc700 1 -- 172.21.15.181:0/3974953727 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600063e0 con 0x7fe97815dfd0 2023-10-16T09:44:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 -- 172.21.15.181:0/3974953727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97815dfd0 msgr2=0x7fe9780af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 --2- 172.21.15.181:0/3974953727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97815dfd0 0x7fe9780af9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe960009820 tx=0x7fe960009b30 comp rx=0 tx=0).stop 2023-10-16T09:44:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 -- 172.21.15.181:0/3974953727 shutdown_connections 2023-10-16T09:44:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 --2- 172.21.15.181:0/3974953727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe9780cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 --2- 172.21.15.181:0/3974953727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97815dfd0 0x7fe9780af9a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 -- 172.21.15.181:0/3974953727 >> 172.21.15.181:0/3974953727 conn(0x7fe9780d04a0 msgr2=0x7fe9780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 -- 172.21.15.181:0/3974953727 shutdown_connections 2023-10-16T09:44:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.930+0000 7fe97df81700 1 -- 172.21.15.181:0/3974953727 wait complete. 2023-10-16T09:44:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.932+0000 7fe97df81700 1 Processor -- start 2023-10-16T09:44:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.932+0000 7fe97df81700 1 -- start start 2023-10-16T09:44:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe97df81700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780af450 0x7fe9780b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe97df81700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe978157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe97df81700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe97815dfd0 0x7fe9780b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe97df81700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780b4df0 con 0x7fe9780af450 2023-10-16T09:44:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe97df81700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780c88b0 con 0x7fe97815dfd0 2023-10-16T09:44:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe97df81700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9781582f0 con 0x7fe9780cd470 2023-10-16T09:44:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780af450 0x7fe9780b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780af450 0x7fe9780b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37798/0 (socket says 172.21.15.181:37798) 2023-10-16T09:44:52.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 -- 172.21.15.181:0/571452530 learned_addr learned my addr 172.21.15.181:0/571452530 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:52.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe976ffd700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe978157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe977fff700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe97815dfd0 0x7fe9780b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 -- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 msgr2=0x7fe978157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe978157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 -- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe97815dfd0 msgr2=0x7fe9780b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:52.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe97815dfd0 0x7fe9780b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:52.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.933+0000 7fe9777fe700 1 -- 172.21.15.181:0/571452530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960009430 con 0x7fe9780af450 2023-10-16T09:44:52.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.934+0000 7fe9777fe700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780af450 0x7fe9780b5d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.934+0000 7fe96ffff700 1 -- 172.21.15.181:0/571452530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960004040 con 0x7fe9780af450 2023-10-16T09:44:52.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.934+0000 7fe96ffff700 1 -- 172.21.15.181:0/571452530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960020070 con 0x7fe9780af450 2023-10-16T09:44:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.934+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9780b4550 con 0x7fe9780af450 2023-10-16T09:44:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.934+0000 7fe96ffff700 1 -- 172.21.15.181:0/571452530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960004400 con 0x7fe9780af450 2023-10-16T09:44:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.934+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9780b4b10 con 0x7fe9780af450 2023-10-16T09:44:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.935+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9780c89f0 con 0x7fe9780af450 2023-10-16T09:44:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.935+0000 7fe96ffff700 1 -- 172.21.15.181:0/571452530 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe960022030 con 0x7fe9780af450 2023-10-16T09:44:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.936+0000 7fe96ffff700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe95806af00 0x7fe95806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.936+0000 7fe976ffd700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe95806af00 0x7fe95806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.936+0000 7fe96ffff700 1 -- 172.21.15.181:0/571452530 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe96008a910 con 0x7fe9780af450 2023-10-16T09:44:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.936+0000 7fe976ffd700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe95806af00 0x7fe95806d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe968000f60 tx=0x7fe968007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:52.940+0000 7fe96ffff700 1 -- 172.21.15.181:0/571452530 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe9780c89f0 con 0x7fe9780af450 2023-10-16T09:44:53.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.133+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9780c89f0 con 0x7fe9780af450 2023-10-16T09:44:53.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.134+0000 7fe96ffff700 1 -- 172.21.15.181:0/571452530 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12893 (secure 0 0 0) 0x7fe9780c89f0 con 0x7fe9780af450 2023-10-16T09:44:53.135 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:53.136 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":30,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:49.805054+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:53.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe95806af00 msgr2=0x7fe95806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe95806af00 0x7fe95806d3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe968000f60 tx=0x7fe968007e70 comp rx=0 tx=0).stop 2023-10-16T09:44:53.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780af450 msgr2=0x7fe9780b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780af450 0x7fe9780b5d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960004750 comp rx=0 tx=0).stop 2023-10-16T09:44:53.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 shutdown_connections 2023-10-16T09:44:53.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe95806af00 0x7fe95806d3c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe97815dfd0 0x7fe9780b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780af450 0x7fe9780b5d90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.136+0000 7fe97df81700 1 --2- 172.21.15.181:0/571452530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9780cd470 0x7fe978157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.137+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 >> 172.21.15.181:0/571452530 conn(0x7fe9780d04a0 msgr2=0x7fe9780aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:53.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.137+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 shutdown_connections 2023-10-16T09:44:53.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.137+0000 7fe97df81700 1 -- 172.21.15.181:0/571452530 wait complete. 2023-10-16T09:44:53.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:44:53.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:44:53.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.272+0000 7f7441f9a700 1 Processor -- start 2023-10-16T09:44:53.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.272+0000 7f7441f9a700 1 -- start start 2023-10-16T09:44:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f7441f9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 0x7f743c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f7441f9a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c15dfb0 con 0x7f743c15dbd0 2023-10-16T09:44:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f7441f9a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c0cd3a0 con 0x7f743c0cdd80 2023-10-16T09:44:53.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f7441f9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0cd830 con 0x7f743c0ce200 2023-10-16T09:44:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f743b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f743c15dbd0 0x7f743c0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48884/0 (socket says 172.21.15.181:48884) 2023-10-16T09:44:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f743b7fe700 1 -- 172.21.15.181:0/3552534262 learned_addr learned my addr 172.21.15.181:0/3552534262 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650223785 0 0) 0x7f7424001110 con 0x7f743c15dbd0 2023-10-16T09:44:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f743bfff700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 0x7f743c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.273+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74280035b0 con 0x7f743c15dbd0 2023-10-16T09:44:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401808679 0 0) 0x7f74280035b0 con 0x7f743c15dbd0 2023-10-16T09:44:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 msgr2=0x7f743c06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 0x7f743c06be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 >> v1:172.21.15.190:6789/0 conn(0x7f743c0cdd80 legacy=0x7f743c06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0d1fc0 con 0x7f743c15dbd0 2023-10-16T09:44:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7424001800 con 0x7f743c15dbd0 2023-10-16T09:44:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 >> v1:172.21.15.181:6789/0 conn(0x7f743c15dbd0 legacy=0x7f743c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 0x7f7428003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003fb0 0x7f7428004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7428004d90 0x7f7428005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0cd830 con 0x7f7428004d90 2023-10-16T09:44:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742800baf0 con 0x7f7428003fb0 2023-10-16T09:44:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742800bcb0 con 0x7f743c0ce200 2023-10-16T09:44:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743bfff700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7428004d90 0x7f7428005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743b7fe700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 0x7f7428003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743affd700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003fb0 0x7f7428004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743bfff700 1 -- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 msgr2=0x7f7428003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:53.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743bfff700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f743c0ce200 0x7f7428003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743bfff700 1 -- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003fb0 msgr2=0x7f7428004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:53.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743bfff700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7428003fb0 0x7f7428004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743bfff700 1 -- 172.21.15.181:0/3552534262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280035b0 con 0x7f7428004d90 2023-10-16T09:44:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743bfff700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7428004d90 0x7f7428005210 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7434009fc0 tx=0x7f7428005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeec51bb7e51b08 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743400ec30 con 0x7f7428004d90 2023-10-16T09:44:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743400f040 con 0x7f7428004d90 2023-10-16T09:44:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.274+0000 7f743a7fc700 1 -- 172.21.15.181:0/3552534262 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434014480 con 0x7f7428004d90 2023-10-16T09:44:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.275+0000 7f7441f9a700 1 -- 172.21.15.181:0/3552534262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7428004d90 msgr2=0x7f7428005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.275+0000 7f7441f9a700 1 --2- 172.21.15.181:0/3552534262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7428004d90 0x7f7428005210 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7434009fc0 tx=0x7f7428005880 comp rx=0 tx=0).stop 2023-10-16T09:44:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.275+0000 7f743bfff700 1 -- 172.21.15.181:0/3552534262 reap_dead start 2023-10-16T09:44:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.275+0000 7f7441f9a700 1 -- 172.21.15.181:0/3552534262 shutdown_connections 2023-10-16T09:44:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.275+0000 7f7441f9a700 1 -- 172.21.15.181:0/3552534262 >> 172.21.15.181:0/3552534262 conn(0x7f743c09a930 msgr2=0x7f743c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.275+0000 7f7441f9a700 1 -- 172.21.15.181:0/3552534262 shutdown_connections 2023-10-16T09:44:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.275+0000 7f7441f9a700 1 -- 172.21.15.181:0/3552534262 wait complete. 2023-10-16T09:44:53.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.276+0000 7f7441f9a700 1 Processor -- start 2023-10-16T09:44:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.276+0000 7f7441f9a700 1 -- start start 2023-10-16T09:44:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f7441f9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7428004d90 0x7f743c0c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f7441f9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f743c0ca0e0 0x7f743c0b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f7441f9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f743c14d9f0 0x7f743c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f7441f9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0b1630 con 0x7f743c0ca0e0 2023-10-16T09:44:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f7441f9a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c9040 con 0x7f743c14d9f0 2023-10-16T09:44:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f7441f9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c9320 con 0x7f7428004d90 2023-10-16T09:44:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f743affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f743c0ca0e0 0x7f743c0b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f743affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f743c0ca0e0 0x7f743c0b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37820/0 (socket says 172.21.15.181:37820) 2023-10-16T09:44:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f743affd700 1 -- 172.21.15.181:0/244324916 learned_addr learned my addr 172.21.15.181:0/244324916 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f743b7fe700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7428004d90 0x7f743c0c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.277+0000 7f743bfff700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f743c14d9f0 0x7f743c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.278+0000 7f743affd700 1 -- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7428004d90 msgr2=0x7f743c0c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.278+0000 7f743affd700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7428004d90 0x7f743c0c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.278+0000 7f743affd700 1 -- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f743c14d9f0 msgr2=0x7f743c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.278+0000 7f743affd700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f743c14d9f0 0x7f743c0ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.278+0000 7f743affd700 1 -- 172.21.15.181:0/244324916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0c9600 con 0x7f743c0ca0e0 2023-10-16T09:44:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.278+0000 7f743bfff700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f743c14d9f0 0x7f743c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.278+0000 7f743affd700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f743c0ca0e0 0x7f743c0b10f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f742c00f820 tx=0x7f742c00fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.279+0000 7f7433fff700 1 -- 172.21.15.181:0/244324916 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c0052e0 con 0x7f743c0ca0e0 2023-10-16T09:44:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.279+0000 7f7433fff700 1 -- 172.21.15.181:0/244324916 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c005440 con 0x7f743c0ca0e0 2023-10-16T09:44:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.279+0000 7f7433fff700 1 -- 172.21.15.181:0/244324916 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c003880 con 0x7f743c0ca0e0 2023-10-16T09:44:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.279+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c0c98e0 con 0x7f743c0ca0e0 2023-10-16T09:44:53.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.279+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c14dea0 con 0x7f743c0ca0e0 2023-10-16T09:44:53.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.280+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c067d20 con 0x7f743c0ca0e0 2023-10-16T09:44:53.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.280+0000 7f7433fff700 1 -- 172.21.15.181:0/244324916 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f742c0039e0 con 0x7f743c0ca0e0 2023-10-16T09:44:53.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.281+0000 7f7433fff700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f741c06af00 0x7f741c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.281+0000 7f7433fff700 1 -- 172.21.15.181:0/244324916 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f742c089040 con 0x7f743c0ca0e0 2023-10-16T09:44:53.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.281+0000 7f743b7fe700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f741c06af00 0x7f741c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.281+0000 7f743b7fe700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f741c06af00 0x7f741c06d3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7424001270 tx=0x7f7424007b00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.285+0000 7f7433fff700 1 -- 172.21.15.181:0/244324916 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f742c054440 con 0x7f743c0ca0e0 2023-10-16T09:44:53.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.472+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f743c0ccd40 con 0x7f743c0ca0e0 2023-10-16T09:44:53.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.473+0000 7f7433fff700 1 -- 172.21.15.181:0/244324916 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f742c057a60 con 0x7f743c0ca0e0 2023-10-16T09:44:53.474 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:44:53.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f741c06af00 msgr2=0x7f741c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f741c06af00 0x7f741c06d3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7424001270 tx=0x7f7424007b00 comp rx=0 tx=0).stop 2023-10-16T09:44:53.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f743c0ca0e0 msgr2=0x7f743c0b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f743c0ca0e0 0x7f743c0b10f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f742c00f820 tx=0x7f742c00fb30 comp rx=0 tx=0).stop 2023-10-16T09:44:53.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 shutdown_connections 2023-10-16T09:44:53.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f741c06af00 0x7f741c06d3c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7428004d90 0x7f743c0c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f743c14d9f0 0x7f743c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 --2- 172.21.15.181:0/244324916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f743c0ca0e0 0x7f743c0b10f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 >> 172.21.15.181:0/244324916 conn(0x7f743c09a930 msgr2=0x7f743c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:53.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 shutdown_connections 2023-10-16T09:44:53.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.475+0000 7f7441f9a700 1 -- 172.21.15.181:0/244324916 wait complete. 2023-10-16T09:44:53.488 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T09:44:53.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.608+0000 7fc8c2022700 1 Processor -- start 2023-10-16T09:44:53.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.608+0000 7fc8c2022700 1 -- start start 2023-10-16T09:44:53.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8c2022700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc159730 0x7fc8bc15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8c2022700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8bc0cc400 0x7fc8bc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8c2022700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc0d0310 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8c2022700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8bc0cd880 con 0x7fc8bc0cc400 2023-10-16T09:44:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8c2022700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0cee70 con 0x7fc8bc159730 2023-10-16T09:44:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8baffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8bc0cc400 0x7fc8bc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8bb7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc159730 0x7fc8bc15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8baffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8bc0cc400 0x7fc8bc0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:42878/0 (socket says 172.21.15.181:42878) 2023-10-16T09:44:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8baffd700 1 -- 172.21.15.181:0/647970061 learned_addr learned my addr 172.21.15.181:0/647970061 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8baffd700 1 -- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc159730 msgr2=0x7fc8bc15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:53.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8baffd700 1 --2- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc159730 0x7fc8bc15cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8baffd700 1 -- 172.21.15.181:0/647970061 >> v1:172.21.15.190:6789/0 conn(0x7fc8bc0cd3f0 legacy=0x7fc8bc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8baffd700 1 -- 172.21.15.181:0/647970061 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc159b10 con 0x7fc8bc0cc400 2023-10-16T09:44:53.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.609+0000 7fc8bb7fe700 1 --2- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc159730 0x7fc8bc15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:53.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8baffd700 1 --2- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc0cc400 0x7fc8bc0cceb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0098f0 tx=0x7fc8ac009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb52b9f5fb157072 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8ba7fc700 1 -- 172.21.15.181:0/647970061 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac011830 con 0x7fc8bc0cc400 2023-10-16T09:44:53.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8ba7fc700 1 -- 172.21.15.181:0/647970061 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac011990 con 0x7fc8bc0cc400 2023-10-16T09:44:53.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8ba7fc700 1 -- 172.21.15.181:0/647970061 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac011d20 con 0x7fc8bc0cc400 2023-10-16T09:44:53.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 -- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc0cc400 msgr2=0x7fc8bc0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 --2- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc0cc400 0x7fc8bc0cceb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0098f0 tx=0x7fc8ac009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:53.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 -- 172.21.15.181:0/647970061 shutdown_connections 2023-10-16T09:44:53.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 --2- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc159730 0x7fc8bc15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 --2- 172.21.15.181:0/647970061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc0cc400 0x7fc8bc0cceb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 -- 172.21.15.181:0/647970061 >> 172.21.15.181:0/647970061 conn(0x7fc8bc0d04a0 msgr2=0x7fc8bc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:53.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 -- 172.21.15.181:0/647970061 shutdown_connections 2023-10-16T09:44:53.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.610+0000 7fc8c2022700 1 -- 172.21.15.181:0/647970061 wait complete. 2023-10-16T09:44:53.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.611+0000 7fc8c2022700 1 Processor -- start 2023-10-16T09:44:53.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8c2022700 1 -- start start 2023-10-16T09:44:53.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8c2022700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc0cc400 0x7fc8bc14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8c2022700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cd3f0 0x7fc8bc06f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8c2022700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc159730 0x7fc8bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8c2022700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0b3d00 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8c2022700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0b3f50 con 0x7fc8bc159730 2023-10-16T09:44:53.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8c2022700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0b4230 con 0x7fc8bc0cc400 2023-10-16T09:44:53.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cd3f0 0x7fc8bc06f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cd3f0 0x7fc8bc06f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37834/0 (socket says 172.21.15.181:37834) 2023-10-16T09:44:53.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.612+0000 7fc8baffd700 1 -- 172.21.15.181:0/1915063966 learned_addr learned my addr 172.21.15.181:0/1915063966 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:53.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8bbfff700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc159730 0x7fc8bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8bb7fe700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc0cc400 0x7fc8bc14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8baffd700 1 -- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc0cc400 msgr2=0x7fc8bc14fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8baffd700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc0cc400 0x7fc8bc14fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8baffd700 1 -- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc159730 msgr2=0x7fc8bc06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8baffd700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc159730 0x7fc8bc06e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8baffd700 1 -- 172.21.15.181:0/1915063966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a4009430 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8bbfff700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc159730 0x7fc8bc06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:53.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8baffd700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cd3f0 0x7fc8bc06f000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac000c00 tx=0x7fc8ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8b3fff700 1 -- 172.21.15.181:0/1915063966 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac004570 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8b3fff700 1 -- 172.21.15.181:0/1915063966 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac01f070 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac009550 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8b3fff700 1 -- 172.21.15.181:0/1915063966 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0038a0 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.613+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc0b47f0 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.614+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc067d20 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.614+0000 7fc8b3fff700 1 -- 172.21.15.181:0/1915063966 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc8ac021030 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.615+0000 7fc8b3fff700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc89c06af00 0x7fc89c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.615+0000 7fc8b3fff700 1 -- 172.21.15.181:0/1915063966 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc8ac08a780 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.615+0000 7fc8bb7fe700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc89c06af00 0x7fc89c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.615+0000 7fc8bb7fe700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc89c06af00 0x7fc89c06d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009c40 tx=0x7fc8a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.619+0000 7fc8b3fff700 1 -- 172.21.15.181:0/1915063966 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc8ac055ca0 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.813+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc8bc066a40 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.813+0000 7fc8b3fff700 1 -- 172.21.15.181:0/1915063966 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc8ac015090 con 0x7fc8bc0cd3f0 2023-10-16T09:44:53.815 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:53.815 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:44:53.815 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:44:53.816 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T09:44:53.816 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T09:44:53.816 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:44:53.816 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:44:53.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc89c06af00 msgr2=0x7fc89c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc89c06af00 0x7fc89c06d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009c40 tx=0x7fc8a4008040 comp rx=0 tx=0).stop 2023-10-16T09:44:53.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cd3f0 msgr2=0x7fc8bc06f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cd3f0 0x7fc8bc06f000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac000c00 tx=0x7fc8ac004750 comp rx=0 tx=0).stop 2023-10-16T09:44:53.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 shutdown_connections 2023-10-16T09:44:53.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fc89c06af00 0x7fc89c06d3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8bc0cc400 0x7fc8bc14fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8bc159730 0x7fc8bc06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 --2- 172.21.15.181:0/1915063966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cd3f0 0x7fc8bc06f000 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.815+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 >> 172.21.15.181:0/1915063966 conn(0x7fc8bc0d04a0 msgr2=0x7fc8bc158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:53.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.816+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 shutdown_connections 2023-10-16T09:44:53.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.816+0000 7fc8c2022700 1 -- 172.21.15.181:0/1915063966 wait complete. 2023-10-16T09:44:53.828 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T09:44:53.829 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T09:44:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.952+0000 7f5e7da9f700 1 Processor -- start 2023-10-16T09:44:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.952+0000 7f5e7da9f700 1 -- start start 2023-10-16T09:44:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e7da9f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e7806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e7da9f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e780d0fc0 0x7f5e780cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e7da9f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e780d0060 0x7f5e7806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e7da9f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e78162190 con 0x7f5e780d0fc0 2023-10-16T09:44:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e7da9f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e78162650 con 0x7f5e780d0060 2023-10-16T09:44:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e7da9f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e78162370 con 0x7f5e780ce610 2023-10-16T09:44:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e780d0fc0 0x7f5e780cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e780d0fc0 0x7f5e780cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37842/0 (socket says 172.21.15.181:37842) 2023-10-16T09:44:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 -- 172.21.15.181:0/2875378111 learned_addr learned my addr 172.21.15.181:0/2875378111 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e777fe700 1 --2- 172.21.15.181:0/2875378111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e7806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e77fff700 1 --2- 172.21.15.181:0/2875378111 >> v2:172.21.15.190:3300/0 conn(0x7f5e780d0060 0x7f5e7806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 -- 172.21.15.181:0/2875378111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 msgr2=0x7f5e7806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 --2- 172.21.15.181:0/2875378111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e7806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 -- 172.21.15.181:0/2875378111 >> v2:172.21.15.190:3300/0 conn(0x7f5e780d0060 msgr2=0x7f5e7806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 --2- 172.21.15.181:0/2875378111 >> v2:172.21.15.190:3300/0 conn(0x7f5e780d0060 0x7f5e7806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 -- 172.21.15.181:0/2875378111 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7806f0f0 con 0x7f5e780d0fc0 2023-10-16T09:44:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.953+0000 7f5e76ffd700 1 --2- 172.21.15.181:0/2875378111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e780cfb20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5e78071880 tx=0x7f5e68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee449db0fc8aeeb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e767fc700 1 -- 172.21.15.181:0/2875378111 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800b870 con 0x7f5e780d0fc0 2023-10-16T09:44:53.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e767fc700 1 -- 172.21.15.181:0/2875378111 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6800b9d0 con 0x7f5e780d0fc0 2023-10-16T09:44:53.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e767fc700 1 -- 172.21.15.181:0/2875378111 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800bd60 con 0x7f5e780d0fc0 2023-10-16T09:44:53.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 -- 172.21.15.181:0/2875378111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 msgr2=0x7f5e780cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/2875378111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e780cfb20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5e78071880 tx=0x7f5e68009bd0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 -- 172.21.15.181:0/2875378111 shutdown_connections 2023-10-16T09:44:53.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/2875378111 >> v2:172.21.15.190:3300/0 conn(0x7f5e780d0060 0x7f5e7806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/2875378111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e7806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/2875378111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e780cfb20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 -- 172.21.15.181:0/2875378111 >> 172.21.15.181:0/2875378111 conn(0x7f5e780d04a0 msgr2=0x7f5e780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:53.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 -- 172.21.15.181:0/2875378111 shutdown_connections 2023-10-16T09:44:53.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.954+0000 7f5e7da9f700 1 -- 172.21.15.181:0/2875378111 wait complete. 2023-10-16T09:44:53.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.956+0000 7f5e7da9f700 1 Processor -- start 2023-10-16T09:44:53.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.956+0000 7f5e7da9f700 1 -- start start 2023-10-16T09:44:53.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e7da9f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e780c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e7da9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e78151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e7da9f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e781515a0 0x7f5e780c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e7da9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780c8180 con 0x7f5e780d0fc0 2023-10-16T09:44:53.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e7da9f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780c8460 con 0x7f5e781515a0 2023-10-16T09:44:53.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e7da9f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780c8740 con 0x7f5e780ce610 2023-10-16T09:44:53.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e78151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e78151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37846/0 (socket says 172.21.15.181:37846) 2023-10-16T09:44:53.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 -- 172.21.15.181:0/1563121252 learned_addr learned my addr 172.21.15.181:0/1563121252 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:53.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e777fe700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e780c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e77fff700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e781515a0 0x7f5e780c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 -- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 msgr2=0x7f5e780c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e780c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 -- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e781515a0 msgr2=0x7f5e780c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:53.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e781515a0 0x7f5e780c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:53.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 -- 172.21.15.181:0/1563121252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68009550 con 0x7f5e780d0fc0 2023-10-16T09:44:53.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e777fe700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e780c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:53.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e77fff700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e781515a0 0x7f5e780c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:53.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.957+0000 7f5e76ffd700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e78151060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5e68000c00 tx=0x7f5e68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.958+0000 7f5e5ffff700 1 -- 172.21.15.181:0/1563121252 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68004540 con 0x7f5e780d0fc0 2023-10-16T09:44:53.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.958+0000 7f5e5ffff700 1 -- 172.21.15.181:0/1563121252 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e680036a0 con 0x7f5e780d0fc0 2023-10-16T09:44:53.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.958+0000 7f5e5ffff700 1 -- 172.21.15.181:0/1563121252 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68003a50 con 0x7f5e780d0fc0 2023-10-16T09:44:53.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.958+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e780c8a20 con 0x7f5e780d0fc0 2023-10-16T09:44:53.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.958+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e78151bf0 con 0x7f5e780d0fc0 2023-10-16T09:44:53.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.959+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e78067d20 con 0x7f5e780d0fc0 2023-10-16T09:44:53.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.959+0000 7f5e5ffff700 1 -- 172.21.15.181:0/1563121252 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e68021070 con 0x7f5e780d0fc0 2023-10-16T09:44:53.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.959+0000 7f5e5ffff700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6406af00 0x7f5e6406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:53.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.959+0000 7f5e777fe700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6406af00 0x7f5e6406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:53.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.959+0000 7f5e5ffff700 1 -- 172.21.15.181:0/1563121252 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e6808a9e0 con 0x7f5e780d0fc0 2023-10-16T09:44:53.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.960+0000 7f5e777fe700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6406af00 0x7f5e6406d3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006fd0 tx=0x7f5e6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:53.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:53.964+0000 7f5e5ffff700 1 -- 172.21.15.181:0/1563121252 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5e68055f90 con 0x7f5e780d0fc0 2023-10-16T09:44:54.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.156+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5e78066fb0 con 0x7f5e780d0fc0 2023-10-16T09:44:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.778+0000 7f5e5ffff700 1 -- 172.21.15.181:0/1563121252 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5e68015020 con 0x7f5e780d0fc0 2023-10-16T09:44:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6406af00 msgr2=0x7f5e6406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6406af00 0x7f5e6406d3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006fd0 tx=0x7f5e6c008040 comp rx=0 tx=0).stop 2023-10-16T09:44:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 msgr2=0x7f5e78151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e78151060 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5e68000c00 tx=0x7f5e68004750 comp rx=0 tx=0).stop 2023-10-16T09:44:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 shutdown_connections 2023-10-16T09:44:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5e6406af00 0x7f5e6406d3c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce610 0x7f5e780c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e781515a0 0x7f5e780c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 --2- 172.21.15.181:0/1563121252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780d0fc0 0x7f5e78151060 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 >> 172.21.15.181:0/1563121252 conn(0x7f5e780d04a0 msgr2=0x7f5e780b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.782+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 shutdown_connections 2023-10-16T09:44:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.783+0000 7f5e7da9f700 1 -- 172.21.15.181:0/1563121252 wait complete. 2023-10-16T09:44:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T09:44:54.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T09:44:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.928+0000 7fd1cbaef700 1 Processor -- start 2023-10-16T09:44:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.929+0000 7fd1cbaef700 1 -- start start 2023-10-16T09:44:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.929+0000 7fd1cbaef700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1c4072760 0x7fd1c406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.929+0000 7fd1cbaef700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.929+0000 7fd1cbaef700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c406b210 con 0x7fd1c415da70 2023-10-16T09:44:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.929+0000 7fd1cbaef700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c40cef00 con 0x7fd1c4072760 2023-10-16T09:44:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.929+0000 7fd1cbaef700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c40cec20 con 0x7fd1c406ade0 2023-10-16T09:44:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1c908a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1c4072760 0x7fd1c406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1ca08c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1c908a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1c4072760 0x7fd1c406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:42904/0 (socket says 172.21.15.181:42904) 2023-10-16T09:44:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1ca08c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c41615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:34884/0 (socket says 172.21.15.181:34884) 2023-10-16T09:44:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1c908a700 1 -- 172.21.15.181:0/3317303548 learned_addr learned my addr 172.21.15.181:0/3317303548 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1ca08c700 1 -- 172.21.15.181:0/3317303548 >> v2:172.21.15.190:3300/0 conn(0x7fd1c4072760 msgr2=0x7fd1c406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1ca08c700 1 --2- 172.21.15.181:0/3317303548 >> v2:172.21.15.190:3300/0 conn(0x7fd1c4072760 0x7fd1c406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1ca08c700 1 -- 172.21.15.181:0/3317303548 >> v1:172.21.15.190:6789/0 conn(0x7fd1c415da70 legacy=0x7fd1c40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1ca08c700 1 -- 172.21.15.181:0/3317303548 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4071c80 con 0x7fd1c406ade0 2023-10-16T09:44:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1c908a700 1 --2- 172.21.15.181:0/3317303548 >> v2:172.21.15.190:3300/0 conn(0x7fd1c4072760 0x7fd1c406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:44:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.930+0000 7fd1ca08c700 1 --2- 172.21.15.181:0/3317303548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c41615f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009d90 tx=0x7fd1bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0cbcdcb253bc457 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1c8889700 1 -- 172.21.15.181:0/3317303548 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00e070 con 0x7fd1c406ade0 2023-10-16T09:44:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1c8889700 1 -- 172.21.15.181:0/3317303548 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc00bb20 con 0x7fd1c406ade0 2023-10-16T09:44:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1c8889700 1 -- 172.21.15.181:0/3317303548 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc012430 con 0x7fd1c406ade0 2023-10-16T09:44:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 -- 172.21.15.181:0/3317303548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 msgr2=0x7fd1c41615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/3317303548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c41615f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009d90 tx=0x7fd1bc00b330 comp rx=0 tx=0).stop 2023-10-16T09:44:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 -- 172.21.15.181:0/3317303548 shutdown_connections 2023-10-16T09:44:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/3317303548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c41615f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/3317303548 >> v2:172.21.15.190:3300/0 conn(0x7fd1c4072760 0x7fd1c406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 -- 172.21.15.181:0/3317303548 >> 172.21.15.181:0/3317303548 conn(0x7fd1c40cd010 msgr2=0x7fd1c40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 -- 172.21.15.181:0/3317303548 shutdown_connections 2023-10-16T09:44:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.931+0000 7fd1cbaef700 1 -- 172.21.15.181:0/3317303548 wait complete. 2023-10-16T09:44:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.932+0000 7fd1cbaef700 1 Processor -- start 2023-10-16T09:44:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.932+0000 7fd1cbaef700 1 -- start start 2023-10-16T09:44:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1cbaef700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c40c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1cbaef700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c4072760 0x7fd1c40ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1cbaef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c415da70 0x7fd1c40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1cbaef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c40ad7d0 con 0x7fd1c415da70 2023-10-16T09:44:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1cbaef700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c40ada00 con 0x7fd1c4072760 2023-10-16T09:44:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1cbaef700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c40adce0 con 0x7fd1c406ade0 2023-10-16T09:44:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1ca08c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c415da70 0x7fd1c40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1ca08c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c415da70 0x7fd1c40cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37864/0 (socket says 172.21.15.181:37864) 2023-10-16T09:44:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1ca08c700 1 -- 172.21.15.181:0/4066492923 learned_addr learned my addr 172.21.15.181:0/4066492923 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1c988b700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c40c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.933+0000 7fd1c908a700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c4072760 0x7fd1c40ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ca08c700 1 -- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 msgr2=0x7fd1c40c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ca08c700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c40c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ca08c700 1 -- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c4072760 msgr2=0x7fd1c40ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ca08c700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c4072760 0x7fd1c40ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ca08c700 1 -- 172.21.15.181:0/4066492923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0009550 con 0x7fd1c415da70 2023-10-16T09:44:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ca08c700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c415da70 0x7fd1c40cbc20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc014040 tx=0x7fd1bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ba7fc700 1 -- 172.21.15.181:0/4066492923 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc012430 con 0x7fd1c415da70 2023-10-16T09:44:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ba7fc700 1 -- 172.21.15.181:0/4066492923 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc020070 con 0x7fd1c415da70 2023-10-16T09:44:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc009a70 con 0x7fd1c415da70 2023-10-16T09:44:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1ba7fc700 1 -- 172.21.15.181:0/4066492923 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc003ea0 con 0x7fd1c415da70 2023-10-16T09:44:54.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.934+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c40ae2a0 con 0x7fd1c415da70 2023-10-16T09:44:54.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.935+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c406f6c0 con 0x7fd1c415da70 2023-10-16T09:44:54.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.935+0000 7fd1ba7fc700 1 -- 172.21.15.181:0/4066492923 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd1bc004280 con 0x7fd1c415da70 2023-10-16T09:44:54.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.936+0000 7fd1ba7fc700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd1a806af00 0x7fd1a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:54.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.936+0000 7fd1c988b700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd1a806af00 0x7fd1a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:54.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.936+0000 7fd1ba7fc700 1 -- 172.21.15.181:0/4066492923 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd1bc022070 con 0x7fd1c415da70 2023-10-16T09:44:54.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.936+0000 7fd1c988b700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd1a806af00 0x7fd1a806d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4007fd0 tx=0x7fd1b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:54.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:54.940+0000 7fd1ba7fc700 1 -- 172.21.15.181:0/4066492923 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1bc056710 con 0x7fd1c415da70 2023-10-16T09:44:55.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.135+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd1c415d130 con 0x7fd1c415da70 2023-10-16T09:44:55.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.783+0000 7fd1ba7fc700 1 -- 172.21.15.181:0/4066492923 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd1bc012e60 con 0x7fd1c415da70 2023-10-16T09:44:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd1a806af00 msgr2=0x7fd1a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd1a806af00 0x7fd1a806d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4007fd0 tx=0x7fd1b4009040 comp rx=0 tx=0).stop 2023-10-16T09:44:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c415da70 msgr2=0x7fd1c40cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c415da70 0x7fd1c40cbc20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc014040 tx=0x7fd1bc004ad0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 shutdown_connections 2023-10-16T09:44:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd1a806af00 0x7fd1a806d3c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1c406ade0 0x7fd1c40c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c415da70 0x7fd1c40cbc20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 --2- 172.21.15.181:0/4066492923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1c4072760 0x7fd1c40ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 >> 172.21.15.181:0/4066492923 conn(0x7fd1c40cd010 msgr2=0x7fd1c406fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:55.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 shutdown_connections 2023-10-16T09:44:55.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.787+0000 7fd1cbaef700 1 -- 172.21.15.181:0/4066492923 wait complete. 2023-10-16T09:44:55.791 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T09:44:55.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T09:44:55.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.938+0000 7f710db8d700 1 Processor -- start 2023-10-16T09:44:55.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.938+0000 7f710db8d700 1 -- start start 2023-10-16T09:44:55.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.938+0000 7f710db8d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f710815f150 0x7f71080cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:55.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.938+0000 7f710db8d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080b2ae0 con 0x7f71080cd170 2023-10-16T09:44:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.938+0000 7f710db8d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080b2dc0 con 0x7f71080ce4f0 2023-10-16T09:44:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.938+0000 7f710db8d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71080ce9a0 con 0x7f710815f150 2023-10-16T09:44:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.938+0000 7f71077fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f710815f150 0x7f71080cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f7107fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f71080cd170 0x7f71080b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48950/0 (socket says 172.21.15.181:48950) 2023-10-16T09:44:55.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f7107fff700 1 -- 172.21.15.181:0/3425980576 learned_addr learned my addr 172.21.15.181:0/3425980576 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:55.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71077fe700 1 -- 172.21.15.181:0/3425980576 >> v1:172.21.15.190:6789/0 conn(0x7f71080ce4f0 legacy=0x7f71080b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:55.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71077fe700 1 -- 172.21.15.181:0/3425980576 >> v1:172.21.15.181:6789/0 conn(0x7f71080cd170 legacy=0x7f71080b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:55.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71077fe700 1 -- 172.21.15.181:0/3425980576 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080cf9d0 con 0x7f710815f150 2023-10-16T09:44:55.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71067fc700 1 -- 172.21.15.181:0/3425980576 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478159890 0 0) 0x7f71080b2ae0 con 0x7f71080cd170 2023-10-16T09:44:55.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71077fe700 1 --2- 172.21.15.181:0/3425980576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f71080cdfb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009870 tx=0x7f70f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665e4e1459387f07 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:55.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71067fc700 1 -- 172.21.15.181:0/3425980576 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0011830 con 0x7f710815f150 2023-10-16T09:44:55.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71067fc700 1 -- 172.21.15.181:0/3425980576 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0011990 con 0x7f710815f150 2023-10-16T09:44:55.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.939+0000 7f71067fc700 1 -- 172.21.15.181:0/3425980576 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0011d20 con 0x7f710815f150 2023-10-16T09:44:55.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.940+0000 7f710db8d700 1 -- 172.21.15.181:0/3425980576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 msgr2=0x7f71080cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:55.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.940+0000 7f710db8d700 1 --2- 172.21.15.181:0/3425980576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f71080cdfb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009870 tx=0x7f70f0009b80 comp rx=0 tx=0).stop 2023-10-16T09:44:55.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.940+0000 7f710db8d700 1 -- 172.21.15.181:0/3425980576 shutdown_connections 2023-10-16T09:44:55.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.940+0000 7f710db8d700 1 --2- 172.21.15.181:0/3425980576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f71080cdfb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.940+0000 7f710db8d700 1 -- 172.21.15.181:0/3425980576 >> 172.21.15.181:0/3425980576 conn(0x7f710809a930 msgr2=0x7f71080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:55.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.940+0000 7f710db8d700 1 -- 172.21.15.181:0/3425980576 shutdown_connections 2023-10-16T09:44:55.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.940+0000 7f710db8d700 1 -- 172.21.15.181:0/3425980576 wait complete. 2023-10-16T09:44:55.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.941+0000 7f710db8d700 1 Processor -- start 2023-10-16T09:44:55.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.941+0000 7f710db8d700 1 -- start start 2023-10-16T09:44:55.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f710db8d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71080cd170 0x7f710814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:55.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f710db8d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71080ce4f0 0x7f7108150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:55.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f710db8d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f710815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:55.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f710db8d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108151e90 con 0x7f710815f150 2023-10-16T09:44:55.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f710db8d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108158f70 con 0x7f71080ce4f0 2023-10-16T09:44:55.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f710db8d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7108159250 con 0x7f71080cd170 2023-10-16T09:44:55.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f710815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:55.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f710815a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37876/0 (socket says 172.21.15.181:37876) 2023-10-16T09:44:55.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 -- 172.21.15.181:0/3496802796 learned_addr learned my addr 172.21.15.181:0/3496802796 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:55.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f71077fe700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71080cd170 0x7f710814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:55.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7106ffd700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71080ce4f0 0x7f7108150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:55.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 -- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71080cd170 msgr2=0x7f710814fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:55.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71080cd170 0x7f710814fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 -- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71080ce4f0 msgr2=0x7f7108150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:55.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71080ce4f0 0x7f7108150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:55.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f7107fff700 1 -- 172.21.15.181:0/3496802796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00094a0 con 0x7f710815f150 2023-10-16T09:44:55.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.942+0000 7f71077fe700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71080cd170 0x7f710814fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:55.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.943+0000 7f7107fff700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f710815a010 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f710000cba0 tx=0x7f710000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:55.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.943+0000 7f70fffff700 1 -- 172.21.15.181:0/3496802796 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000aad0 con 0x7f710815f150 2023-10-16T09:44:55.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.943+0000 7f70fffff700 1 -- 172.21.15.181:0/3496802796 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710000ac50 con 0x7f710815f150 2023-10-16T09:44:55.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.943+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7108159530 con 0x7f710815f150 2023-10-16T09:44:55.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.943+0000 7f70fffff700 1 -- 172.21.15.181:0/3496802796 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000070d0 con 0x7f710815f150 2023-10-16T09:44:55.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.943+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7108159af0 con 0x7f710815f150 2023-10-16T09:44:55.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.944+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7108159d10 con 0x7f710815f150 2023-10-16T09:44:55.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.944+0000 7f70fffff700 1 -- 172.21.15.181:0/3496802796 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7100007230 con 0x7f710815f150 2023-10-16T09:44:55.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.945+0000 7f70fffff700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f70e806af00 0x7f70e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:55.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.945+0000 7f70fffff700 1 -- 172.21.15.181:0/3496802796 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f71000892d0 con 0x7f710815f150 2023-10-16T09:44:55.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.945+0000 7f71077fe700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f70e806af00 0x7f70e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:55.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.945+0000 7f71077fe700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f70e806af00 0x7f70e806d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70f00090b0 tx=0x7f70f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:55.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:55.949+0000 7f70fffff700 1 -- 172.21.15.181:0/3496802796 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f71000543e0 con 0x7f710815f150 2023-10-16T09:44:56.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.184+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7108067d20 con 0x7f710815f150 2023-10-16T09:44:56.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.785+0000 7f70fffff700 1 -- 172.21.15.181:0/3496802796 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7100057a00 con 0x7f710815f150 2023-10-16T09:44:56.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.789+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f70e806af00 msgr2=0x7f70e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:56.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.789+0000 7f710db8d700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f70e806af00 0x7f70e806d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70f00090b0 tx=0x7f70f0008030 comp rx=0 tx=0).stop 2023-10-16T09:44:56.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.789+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 msgr2=0x7f710815a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:56.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.789+0000 7f710db8d700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f710815a010 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f710000cba0 tx=0x7f710000ceb0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.790+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 shutdown_connections 2023-10-16T09:44:56.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.790+0000 7f710db8d700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f70e806af00 0x7f70e806d3c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.790+0000 7f710db8d700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71080cd170 0x7f710814fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.790+0000 7f710db8d700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f710815f150 0x7f710815a010 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.790+0000 7f710db8d700 1 --2- 172.21.15.181:0/3496802796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71080ce4f0 0x7f7108150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.790+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 >> 172.21.15.181:0/3496802796 conn(0x7f710809a930 msgr2=0x7f71080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:56.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.791+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 shutdown_connections 2023-10-16T09:44:56.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.791+0000 7f710db8d700 1 -- 172.21.15.181:0/3496802796 wait complete. 2023-10-16T09:44:56.798 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 4 and data pool 5 2023-10-16T09:44:56.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:56.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.948+0000 7fe885cea700 1 Processor -- start 2023-10-16T09:44:56.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.948+0000 7fe885cea700 1 -- start start 2023-10-16T09:44:56.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe885cea700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88015dfd0 0x7fe8800af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:56.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe885cea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8800af450 0x7fe88006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:56.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe885cea700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88006f870 con 0x7fe88006db80 2023-10-16T09:44:56.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe885cea700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800d1160 con 0x7fe8800af450 2023-10-16T09:44:56.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe885cea700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800cc8e0 con 0x7fe88015dfd0 2023-10-16T09:44:56.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8800af450 0x7fe88006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:56.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8800af450 0x7fe88006d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37896/0 (socket says 172.21.15.181:37896) 2023-10-16T09:44:56.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 -- 172.21.15.181:0/371071652 learned_addr learned my addr 172.21.15.181:0/371071652 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:56.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87f7fe700 1 --2- 172.21.15.181:0/371071652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88015dfd0 0x7fe8800af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:56.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 -- 172.21.15.181:0/371071652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88015dfd0 msgr2=0x7fe8800af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:56.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 --2- 172.21.15.181:0/371071652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88015dfd0 0x7fe8800af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 -- 172.21.15.181:0/371071652 >> v1:172.21.15.190:6789/0 conn(0x7fe88006db80 legacy=0x7fe8800cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:56.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 -- 172.21.15.181:0/371071652 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8800ccbc0 con 0x7fe8800af450 2023-10-16T09:44:56.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87f7fe700 1 --2- 172.21.15.181:0/371071652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88015dfd0 0x7fe8800af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:56.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.949+0000 7fe87effd700 1 --2- 172.21.15.181:0/371071652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8800af450 0x7fe88006d640 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098f0 tx=0x7fe870009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0699c71a99f8fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:56.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe87e7fc700 1 -- 172.21.15.181:0/371071652 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000b8a0 con 0x7fe8800af450 2023-10-16T09:44:56.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe87e7fc700 1 -- 172.21.15.181:0/371071652 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000ba00 con 0x7fe8800af450 2023-10-16T09:44:56.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe87e7fc700 1 -- 172.21.15.181:0/371071652 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000bd90 con 0x7fe8800af450 2023-10-16T09:44:56.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 -- 172.21.15.181:0/371071652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8800af450 msgr2=0x7fe88006d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:56.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 --2- 172.21.15.181:0/371071652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8800af450 0x7fe88006d640 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098f0 tx=0x7fe870009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:56.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 -- 172.21.15.181:0/371071652 shutdown_connections 2023-10-16T09:44:56.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 --2- 172.21.15.181:0/371071652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88015dfd0 0x7fe8800af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 --2- 172.21.15.181:0/371071652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8800af450 0x7fe88006d640 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 -- 172.21.15.181:0/371071652 >> 172.21.15.181:0/371071652 conn(0x7fe8800d04a0 msgr2=0x7fe8800d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:56.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 -- 172.21.15.181:0/371071652 shutdown_connections 2023-10-16T09:44:56.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.950+0000 7fe885cea700 1 -- 172.21.15.181:0/371071652 wait complete. 2023-10-16T09:44:56.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.951+0000 7fe885cea700 1 Processor -- start 2023-10-16T09:44:56.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe885cea700 1 -- start start 2023-10-16T09:44:56.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe885cea700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88006db80 0x7fe8800b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:56.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe885cea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8800af450 0x7fe880153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:56.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe885cea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe88015dfd0 0x7fe8800b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:56.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe885cea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880159ee0 con 0x7fe88015dfd0 2023-10-16T09:44:56.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe885cea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800b18a0 con 0x7fe8800af450 2023-10-16T09:44:56.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe885cea700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe880153eb0 con 0x7fe88006db80 2023-10-16T09:44:56.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe87ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe88015dfd0 0x7fe8800b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:56.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe87ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe88015dfd0 0x7fe8800b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37904/0 (socket says 172.21.15.181:37904) 2023-10-16T09:44:56.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.952+0000 7fe87ffff700 1 -- 172.21.15.181:0/3030739219 learned_addr learned my addr 172.21.15.181:0/3030739219 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:56.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87f7fe700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88006db80 0x7fe8800b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:56.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87effd700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8800af450 0x7fe880153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:56.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87ffff700 1 -- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88006db80 msgr2=0x7fe8800b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:56.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87ffff700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88006db80 0x7fe8800b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87ffff700 1 -- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8800af450 msgr2=0x7fe880153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:56.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87ffff700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8800af450 0x7fe880153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:56.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87ffff700 1 -- 172.21.15.181:0/3030739219 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe870009550 con 0x7fe88015dfd0 2023-10-16T09:44:56.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87effd700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8800af450 0x7fe880153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:56.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe87ffff700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe88015dfd0 0x7fe8800b2430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe87800cc50 tx=0x7fe87800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:56.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe867fff700 1 -- 172.21.15.181:0/3030739219 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87800ad80 con 0x7fe88015dfd0 2023-10-16T09:44:56.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe867fff700 1 -- 172.21.15.181:0/3030739219 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87800ce50 con 0x7fe88015dfd0 2023-10-16T09:44:56.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8800b02c0 con 0x7fe88015dfd0 2023-10-16T09:44:56.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe867fff700 1 -- 172.21.15.181:0/3030739219 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe878003c40 con 0x7fe88015dfd0 2023-10-16T09:44:56.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.953+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8800b0880 con 0x7fe88015dfd0 2023-10-16T09:44:56.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.954+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe880072780 con 0x7fe88015dfd0 2023-10-16T09:44:56.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.954+0000 7fe867fff700 1 -- 172.21.15.181:0/3030739219 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8780050d0 con 0x7fe88015dfd0 2023-10-16T09:44:56.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.955+0000 7fe867fff700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe86c06af00 0x7fe86c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:56.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.955+0000 7fe867fff700 1 -- 172.21.15.181:0/3030739219 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8780896f0 con 0x7fe88015dfd0 2023-10-16T09:44:56.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.955+0000 7fe87f7fe700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe86c06af00 0x7fe86c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:56.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.955+0000 7fe87f7fe700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe86c06af00 0x7fe86c06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c920 tx=0x7fe874008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:56.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:56.959+0000 7fe867fff700 1 -- 172.21.15.181:0/3030739219 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe8780547a0 con 0x7fe88015dfd0 2023-10-16T09:44:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.155+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe880067d20 con 0x7fe88015dfd0 2023-10-16T09:44:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.156+0000 7fe867fff700 1 -- 172.21.15.181:0/3030739219 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16618 (secure 0 0 0) 0x7fe878057dc0 con 0x7fe88015dfd0 2023-10-16T09:44:57.157 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:57.158 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":33,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:56.185411+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:44:54.175032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:55.152824+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe86c06af00 msgr2=0x7fe86c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe86c06af00 0x7fe86c06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c920 tx=0x7fe874008040 comp rx=0 tx=0).stop 2023-10-16T09:44:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe88015dfd0 msgr2=0x7fe8800b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe88015dfd0 0x7fe8800b2430 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe87800cc50 tx=0x7fe87800a340 comp rx=0 tx=0).stop 2023-10-16T09:44:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 shutdown_connections 2023-10-16T09:44:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fe86c06af00 0x7fe86c06d3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe88006db80 0x7fe8800b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe88015dfd0 0x7fe8800b2430 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 --2- 172.21.15.181:0/3030739219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8800af450 0x7fe880153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 >> 172.21.15.181:0/3030739219 conn(0x7fe8800d04a0 msgr2=0x7fe88015ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 shutdown_connections 2023-10-16T09:44:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.158+0000 7fe885cea700 1 -- 172.21.15.181:0/3030739219 wait complete. 2023-10-16T09:44:57.172 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:44:57.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.290+0000 7fb69579e700 1 Processor -- start 2023-10-16T09:44:57.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.290+0000 7fb69579e700 1 -- start start 2023-10-16T09:44:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.291+0000 7fb69579e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 0x7fb6900cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.291+0000 7fb69579e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb69006e740 0x7fb6900aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.291+0000 7fb69579e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900cf1f0 con 0x7fb6900d0b70 2023-10-16T09:44:57.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.291+0000 7fb69579e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69006e5b0 con 0x7fb69006e740 2023-10-16T09:44:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.291+0000 7fb69579e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900afd10 con 0x7fb69006d910 2023-10-16T09:44:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.291+0000 7fb68f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb6900d0b70 0x7fb69006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48988/0 (socket says 172.21.15.181:48988) 2023-10-16T09:44:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.291+0000 7fb68f7fe700 1 -- 172.21.15.181:0/1605790738 learned_addr learned my addr 172.21.15.181:0/1605790738 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338088065 0 0) 0x7fb6900cf1f0 con 0x7fb6900d0b70 2023-10-16T09:44:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb67c0035b0 con 0x7fb6900d0b70 2023-10-16T09:44:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb694f9d700 1 --2- 172.21.15.181:0/1605790738 >> v2:172.21.15.190:3300/0 conn(0x7fb69006e740 0x7fb6900aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68ffff700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 0x7fb6900cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 359109654 0 0) 0x7fb67c0035b0 con 0x7fb6900d0b70 2023-10-16T09:44:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 msgr2=0x7fb6900cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 0x7fb6900cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 >> v2:172.21.15.190:3300/0 conn(0x7fb69006e740 msgr2=0x7fb6900aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 --2- 172.21.15.181:0/1605790738 >> v2:172.21.15.190:3300/0 conn(0x7fb69006e740 0x7fb6900aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900afa30 con 0x7fb6900d0b70 2023-10-16T09:44:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fb678001670 con 0x7fb6900d0b70 2023-10-16T09:44:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 >> v1:172.21.15.181:6789/0 conn(0x7fb6900d0b70 legacy=0x7fb69006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb67c003a00 0x7fb67c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 0x7fb67c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c004d90 0x7fb67c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69006e5b0 con 0x7fb67c003a00 2023-10-16T09:44:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900afd10 con 0x7fb67c004d90 2023-10-16T09:44:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c00ba10 con 0x7fb69006d910 2023-10-16T09:44:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68ffff700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb67c003a00 0x7fb67c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68f7fe700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 0x7fb67c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb694f9d700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c004d90 0x7fb67c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68ffff700 1 -- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 msgr2=0x7fb67c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68ffff700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb69006d910 0x7fb67c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68ffff700 1 -- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c004d90 msgr2=0x7fb67c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68ffff700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c004d90 0x7fb67c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb68ffff700 1 -- 172.21.15.181:0/1605790738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0035b0 con 0x7fb67c003a00 2023-10-16T09:44:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.292+0000 7fb694f9d700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c004d90 0x7fb67c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb68ffff700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb67c003a00 0x7fb67c003e20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb680009b40 tx=0x7fb67c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a523a5a0fdaeb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680005a20 con 0x7fb67c003a00 2023-10-16T09:44:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680005b80 con 0x7fb67c003a00 2023-10-16T09:44:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb68effd700 1 -- 172.21.15.181:0/1605790738 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000e6c0 con 0x7fb67c003a00 2023-10-16T09:44:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb69579e700 1 -- 172.21.15.181:0/1605790738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb67c003a00 msgr2=0x7fb67c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb69579e700 1 --2- 172.21.15.181:0/1605790738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb67c003a00 0x7fb67c003e20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb680009b40 tx=0x7fb67c005880 comp rx=0 tx=0).stop 2023-10-16T09:44:57.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb694f9d700 1 -- 172.21.15.181:0/1605790738 reap_dead start 2023-10-16T09:44:57.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb69579e700 1 -- 172.21.15.181:0/1605790738 shutdown_connections 2023-10-16T09:44:57.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb69579e700 1 -- 172.21.15.181:0/1605790738 >> 172.21.15.181:0/1605790738 conn(0x7fb69006c430 msgr2=0x7fb6900d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:57.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb69579e700 1 -- 172.21.15.181:0/1605790738 shutdown_connections 2023-10-16T09:44:57.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.293+0000 7fb69579e700 1 -- 172.21.15.181:0/1605790738 wait complete. 2023-10-16T09:44:57.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.295+0000 7fb69579e700 1 Processor -- start 2023-10-16T09:44:57.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.295+0000 7fb69579e700 1 -- start start 2023-10-16T09:44:57.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c003a00 0x7fb69014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6900c6270 0x7fb6900c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb69014b330 0x7fb69014b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69014a360 con 0x7fb69014b330 2023-10-16T09:44:57.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69014aea0 con 0x7fb67c003a00 2023-10-16T09:44:57.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69014d1e0 con 0x7fb6900c6270 2023-10-16T09:44:57.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb69014b330 0x7fb69014b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb69014b330 0x7fb69014b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37926/0 (socket says 172.21.15.181:37926) 2023-10-16T09:44:57.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 -- 172.21.15.181:0/3472347404 learned_addr learned my addr 172.21.15.181:0/3472347404 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:57.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb68ffff700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c003a00 0x7fb69014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb68f7fe700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6900c6270 0x7fb6900c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 -- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6900c6270 msgr2=0x7fb6900c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6900c6270 0x7fb6900c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 -- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c003a00 msgr2=0x7fb69014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c003a00 0x7fb69014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 -- 172.21.15.181:0/3472347404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69014a570 con 0x7fb69014b330 2023-10-16T09:44:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb68f7fe700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6900c6270 0x7fb6900c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:44:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb694f9d700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb69014b330 0x7fb69014b7b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb6900c3d30 tx=0x7fb6880104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb68cff9700 1 -- 172.21.15.181:0/3472347404 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68801b070 con 0x7fb69014b330 2023-10-16T09:44:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb68cff9700 1 -- 172.21.15.181:0/3472347404 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800f040 con 0x7fb69014b330 2023-10-16T09:44:57.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb68cff9700 1 -- 172.21.15.181:0/3472347404 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e050 con 0x7fb69014b330 2023-10-16T09:44:57.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69014a850 con 0x7fb69014b330 2023-10-16T09:44:57.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.296+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6900c6c30 con 0x7fb69014b330 2023-10-16T09:44:57.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.297+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb690067d20 con 0x7fb69014b330 2023-10-16T09:44:57.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.298+0000 7fb68cff9700 1 -- 172.21.15.181:0/3472347404 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb688015440 con 0x7fb69014b330 2023-10-16T09:44:57.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.298+0000 7fb68cff9700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb67006af00 0x7fb67006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.298+0000 7fb68ffff700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb67006af00 0x7fb67006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.298+0000 7fb68cff9700 1 -- 172.21.15.181:0/3472347404 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6880895b0 con 0x7fb69014b330 2023-10-16T09:44:57.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.299+0000 7fb68ffff700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb67006af00 0x7fb67006d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb680000c00 tx=0x7fb680008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:57.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.303+0000 7fb68cff9700 1 -- 172.21.15.181:0/3472347404 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb6880545d0 con 0x7fb69014b330 2023-10-16T09:44:57.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.495+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6900b5730 con 0x7fb69014b330 2023-10-16T09:44:57.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.496+0000 7fb68cff9700 1 -- 172.21.15.181:0/3472347404 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16618 (secure 0 0 0) 0x7fb688007ea0 con 0x7fb69014b330 2023-10-16T09:44:57.498 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:44:57.498 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":33,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:56.185411+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:44:54.175032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:55.152824+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:44:57.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb67006af00 msgr2=0x7fb67006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb67006af00 0x7fb67006d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb680000c00 tx=0x7fb680008030 comp rx=0 tx=0).stop 2023-10-16T09:44:57.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb69014b330 msgr2=0x7fb69014b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb69014b330 0x7fb69014b7b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb6900c3d30 tx=0x7fb6880104a0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 shutdown_connections 2023-10-16T09:44:57.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb67006af00 0x7fb67006d3c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb69014b330 0x7fb69014b7b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb67c003a00 0x7fb69014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 --2- 172.21.15.181:0/3472347404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6900c6270 0x7fb6900c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.498+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 >> 172.21.15.181:0/3472347404 conn(0x7fb69006c430 msgr2=0x7fb6900b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:57.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.499+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 shutdown_connections 2023-10-16T09:44:57.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.499+0000 7fb69579e700 1 -- 172.21.15.181:0/3472347404 wait complete. 2023-10-16T09:44:57.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T09:44:57.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.634+0000 7f53cdb25700 1 Processor -- start 2023-10-16T09:44:57.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.634+0000 7f53cdb25700 1 -- start start 2023-10-16T09:44:57.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.635+0000 7f53cdb25700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 0x7f53c8071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.635+0000 7f53cdb25700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c80ccb60 con 0x7f53c80b2010 2023-10-16T09:44:57.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.635+0000 7f53cdb25700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c80ce1f0 con 0x7f53c80ce680 2023-10-16T09:44:57.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.635+0000 7f53cdb25700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8072120 con 0x7f53c80cc6e0 2023-10-16T09:44:57.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.635+0000 7f53c77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f53c80b2010 0x7f53c80b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49008/0 (socket says 172.21.15.181:49008) 2023-10-16T09:44:57.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.635+0000 7f53c77fe700 1 -- 172.21.15.181:0/2486743881 learned_addr learned my addr 172.21.15.181:0/2486743881 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:44:57.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c6ffd700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 0x7f53c8071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26159700 0 0) 0x7f53b0001110 con 0x7f53c80b2010 2023-10-16T09:44:57.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b40035b0 con 0x7f53c80b2010 2023-10-16T09:44:57.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2320426471 0 0) 0x7f53b40035b0 con 0x7f53c80b2010 2023-10-16T09:44:57.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 msgr2=0x7f53c8071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:57.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 0x7f53c8071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 >> v1:172.21.15.190:6789/0 conn(0x7f53c80ce680 legacy=0x7f53c80ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80ccda0 con 0x7f53c80b2010 2023-10-16T09:44:57.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f53b0001800 con 0x7f53c80b2010 2023-10-16T09:44:57.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 >> v1:172.21.15.181:6789/0 conn(0x7f53c80b2010 legacy=0x7f53c80b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 0x7f53b4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4003fb0 0x7f53b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b4004d90 0x7f53b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8072120 con 0x7f53b4004d90 2023-10-16T09:44:57.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b400baf0 con 0x7f53b4003fb0 2023-10-16T09:44:57.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b400bcb0 con 0x7f53c80cc6e0 2023-10-16T09:44:57.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c7fff700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b4004d90 0x7f53b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c77fe700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 0x7f53b4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c6ffd700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4003fb0 0x7f53b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c7fff700 1 -- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 msgr2=0x7f53b4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:57.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c7fff700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cc6e0 0x7f53b4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c7fff700 1 -- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4003fb0 msgr2=0x7f53b4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:57.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c7fff700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53b4003fb0 0x7f53b4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c7fff700 1 -- 172.21.15.181:0/2486743881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b40035b0 con 0x7f53b4004d90 2023-10-16T09:44:57.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.636+0000 7f53c7fff700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b4004d90 0x7f53b4005210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f53c000c190 tx=0x7f53b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832c5bac1dbdd5d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:57.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0001db0 con 0x7f53b4004d90 2023-10-16T09:44:57.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c000ccd0 con 0x7f53b4004d90 2023-10-16T09:44:57.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53c67fc700 1 -- 172.21.15.181:0/2486743881 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0003c40 con 0x7f53b4004d90 2023-10-16T09:44:57.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53cdb25700 1 -- 172.21.15.181:0/2486743881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b4004d90 msgr2=0x7f53b4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53cdb25700 1 --2- 172.21.15.181:0/2486743881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53b4004d90 0x7f53b4005210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f53c000c190 tx=0x7f53b4005880 comp rx=0 tx=0).stop 2023-10-16T09:44:57.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53c7fff700 1 -- 172.21.15.181:0/2486743881 reap_dead start 2023-10-16T09:44:57.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53cdb25700 1 -- 172.21.15.181:0/2486743881 shutdown_connections 2023-10-16T09:44:57.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53cdb25700 1 -- 172.21.15.181:0/2486743881 >> 172.21.15.181:0/2486743881 conn(0x7f53c80cd010 msgr2=0x7f53c80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:57.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53cdb25700 1 -- 172.21.15.181:0/2486743881 shutdown_connections 2023-10-16T09:44:57.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.637+0000 7f53cdb25700 1 -- 172.21.15.181:0/2486743881 wait complete. 2023-10-16T09:44:57.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.638+0000 7f53cdb25700 1 Processor -- start 2023-10-16T09:44:57.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53cdb25700 1 -- start start 2023-10-16T09:44:57.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53cdb25700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80aad80 0x7f53c814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53cdb25700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c814ca90 0x7f53c80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53cdb25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c80b50f0 0x7f53c81f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53cdb25700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c814cf10 con 0x7f53c80b50f0 2023-10-16T09:44:57.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53cdb25700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80b5570 con 0x7f53c814ca90 2023-10-16T09:44:57.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53cdb25700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c814a270 con 0x7f53c80aad80 2023-10-16T09:44:57.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c80b50f0 0x7f53c81f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c80b50f0 0x7f53c81f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37946/0 (socket says 172.21.15.181:37946) 2023-10-16T09:44:57.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53c7fff700 1 -- 172.21.15.181:0/279806634 learned_addr learned my addr 172.21.15.181:0/279806634 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:57.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.639+0000 7f53c6ffd700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c814ca90 0x7f53c80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53c77fe700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80aad80 0x7f53c814c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53c7fff700 1 -- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80aad80 msgr2=0x7f53c814c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53c7fff700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80aad80 0x7f53c814c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53c7fff700 1 -- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c814ca90 msgr2=0x7f53c80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:57.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53c7fff700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c814ca90 0x7f53c80b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:57.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53c7fff700 1 -- 172.21.15.181:0/279806634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c814d110 con 0x7f53c80b50f0 2023-10-16T09:44:57.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53c7fff700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c80b50f0 0x7f53c81f4620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005040 tx=0x7f53c00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:57.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53bffff700 1 -- 172.21.15.181:0/279806634 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00050d0 con 0x7f53c80b50f0 2023-10-16T09:44:57.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53bffff700 1 -- 172.21.15.181:0/279806634 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c0005250 con 0x7f53c80b50f0 2023-10-16T09:44:57.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c814d3f0 con 0x7f53c80b50f0 2023-10-16T09:44:57.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53bffff700 1 -- 172.21.15.181:0/279806634 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0005400 con 0x7f53c80b50f0 2023-10-16T09:44:57.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.640+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c814a810 con 0x7f53c80b50f0 2023-10-16T09:44:57.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.641+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c80b6510 con 0x7f53c80b50f0 2023-10-16T09:44:57.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.642+0000 7f53bffff700 1 -- 172.21.15.181:0/279806634 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53c0019040 con 0x7f53c80b50f0 2023-10-16T09:44:57.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.642+0000 7f53bffff700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f53a806af00 0x7f53a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:57.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.642+0000 7f53bffff700 1 -- 172.21.15.181:0/279806634 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53c008a310 con 0x7f53c80b50f0 2023-10-16T09:44:57.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.642+0000 7f53c77fe700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f53a806af00 0x7f53a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:57.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.642+0000 7f53c77fe700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f53a806af00 0x7f53a806d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53b0001270 tx=0x7f53b00012e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:57.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.646+0000 7f53bffff700 1 -- 172.21.15.181:0/279806634 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f53c00556e0 con 0x7f53c80b50f0 2023-10-16T09:44:57.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:57.890+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f53c80cc370 con 0x7f53c80b50f0 2023-10-16T09:44:58.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.822+0000 7f53bffff700 1 -- 172.21.15.181:0/279806634 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f53c0058d00 con 0x7f53c80b50f0 2023-10-16T09:44:58.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f53a806af00 msgr2=0x7f53a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:58.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f53a806af00 0x7f53a806d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f53b0001270 tx=0x7f53b00012e0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c80b50f0 msgr2=0x7f53c81f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c80b50f0 0x7f53c81f4620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005040 tx=0x7f53c00014a0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 shutdown_connections 2023-10-16T09:44:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f53a806af00 0x7f53a806d3c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80aad80 0x7f53c814c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c80b50f0 0x7f53c81f4620 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 --2- 172.21.15.181:0/279806634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c814ca90 0x7f53c80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 >> 172.21.15.181:0/279806634 conn(0x7f53c80cd010 msgr2=0x7f53c80b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 shutdown_connections 2023-10-16T09:44:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.825+0000 7f53cdb25700 1 -- 172.21.15.181:0/279806634 wait complete. 2023-10-16T09:44:58.841 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T09:44:58.841 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T09:44:58.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.971+0000 7f5c1016b700 1 Processor -- start 2023-10-16T09:44:58.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.971+0000 7f5c1016b700 1 -- start start 2023-10-16T09:44:58.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c1016b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c0806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:58.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c1016b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c080b5fd0 0x7f5c0815f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c1016b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c08070590 con 0x7f5c080cb2b0 2023-10-16T09:44:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c1016b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c08156330 con 0x7f5c080b5fd0 2023-10-16T09:44:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c1016b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0806fcf0 con 0x7f5c080cb690 2023-10-16T09:44:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c080b5fd0 0x7f5c0815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c080b5fd0 0x7f5c0815f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37954/0 (socket says 172.21.15.181:37954) 2023-10-16T09:44:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 -- 172.21.15.181:0/1610536665 learned_addr learned my addr 172.21.15.181:0/1610536665 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0d706700 1 --2- 172.21.15.181:0/1610536665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c0806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 -- 172.21.15.181:0/1610536665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 msgr2=0x7f5c0806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:58.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 --2- 172.21.15.181:0/1610536665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c0806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 -- 172.21.15.181:0/1610536665 >> v1:172.21.15.190:6789/0 conn(0x7f5c080cb2b0 legacy=0x7f5c080b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 -- 172.21.15.181:0/1610536665 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0806ffd0 con 0x7f5c080b5fd0 2023-10-16T09:44:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.972+0000 7f5c0e708700 1 --2- 172.21.15.181:0/1610536665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080b5fd0 0x7f5c0815f420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009e50 tx=0x7f5c0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b5e492448c79cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c0cf05700 1 -- 172.21.15.181:0/1610536665 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000e070 con 0x7f5c080b5fd0 2023-10-16T09:44:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c0cf05700 1 -- 172.21.15.181:0/1610536665 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0000b9c0 con 0x7f5c080b5fd0 2023-10-16T09:44:58.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c0cf05700 1 -- 172.21.15.181:0/1610536665 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0000bd70 con 0x7f5c080b5fd0 2023-10-16T09:44:58.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 -- 172.21.15.181:0/1610536665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080b5fd0 msgr2=0x7f5c0815f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:58.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 --2- 172.21.15.181:0/1610536665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080b5fd0 0x7f5c0815f420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009e50 tx=0x7f5c0000b330 comp rx=0 tx=0).stop 2023-10-16T09:44:58.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 -- 172.21.15.181:0/1610536665 shutdown_connections 2023-10-16T09:44:58.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 --2- 172.21.15.181:0/1610536665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080b5fd0 0x7f5c0815f420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 --2- 172.21.15.181:0/1610536665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c0806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 -- 172.21.15.181:0/1610536665 >> 172.21.15.181:0/1610536665 conn(0x7f5c080cd010 msgr2=0x7f5c080cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:58.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 -- 172.21.15.181:0/1610536665 shutdown_connections 2023-10-16T09:44:58.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.973+0000 7f5c1016b700 1 -- 172.21.15.181:0/1610536665 wait complete. 2023-10-16T09:44:58.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.974+0000 7f5c1016b700 1 Processor -- start 2023-10-16T09:44:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.974+0000 7f5c1016b700 1 -- start start 2023-10-16T09:44:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c1016b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c080b5fd0 0x7f5c080c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c1016b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cb2b0 0x7f5c08145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c1016b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c080b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:58.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c1016b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08146240 con 0x7f5c080cb2b0 2023-10-16T09:44:58.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c1016b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c081464b0 con 0x7f5c080b5fd0 2023-10-16T09:44:58.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c1016b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08146790 con 0x7f5c080cb690 2023-10-16T09:44:58.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c0d706700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cb2b0 0x7f5c08145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:58.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c0d706700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cb2b0 0x7f5c08145830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37958/0 (socket says 172.21.15.181:37958) 2023-10-16T09:44:58.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c0d706700 1 -- 172.21.15.181:0/805310757 learned_addr learned my addr 172.21.15.181:0/805310757 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:58.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c0e708700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c080b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.975+0000 7f5c0df07700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c080b5fd0 0x7f5c080c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c0d706700 1 -- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 msgr2=0x7f5c080b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c0d706700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c080b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c0d706700 1 -- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c080b5fd0 msgr2=0x7f5c080c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c0d706700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c080b5fd0 0x7f5c080c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c0d706700 1 -- 172.21.15.181:0/805310757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c00009b30 con 0x7f5c080cb2b0 2023-10-16T09:44:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c0d706700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cb2b0 0x7f5c08145830 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d800 tx=0x7f5bf800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5bfe7fc700 1 -- 172.21.15.181:0/805310757 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80125f0 con 0x7f5c080cb2b0 2023-10-16T09:44:58.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5bfe7fc700 1 -- 172.21.15.181:0/805310757 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8012750 con 0x7f5c080cb2b0 2023-10-16T09:44:58.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5bfe7fc700 1 -- 172.21.15.181:0/805310757 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8012b00 con 0x7f5c080cb2b0 2023-10-16T09:44:58.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c08146a70 con 0x7f5c080cb2b0 2023-10-16T09:44:58.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.976+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c080c4ef0 con 0x7f5c080cb2b0 2023-10-16T09:44:58.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.977+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c081470c0 con 0x7f5c080cb2b0 2023-10-16T09:44:58.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.977+0000 7f5bfe7fc700 1 -- 172.21.15.181:0/805310757 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5bf8012c60 con 0x7f5c080cb2b0 2023-10-16T09:44:58.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.978+0000 7f5bfe7fc700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5bec06af00 0x7f5bec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:58.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.978+0000 7f5bfe7fc700 1 -- 172.21.15.181:0/805310757 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bf8089610 con 0x7f5c080cb2b0 2023-10-16T09:44:58.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.978+0000 7f5c0df07700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5bec06af00 0x7f5bec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:58.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.978+0000 7f5c0df07700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5bec06af00 0x7f5bec06d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c04007fd0 tx=0x7f5c04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:58.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:58.982+0000 7f5bfe7fc700 1 -- 172.21.15.181:0/805310757 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5bf8054770 con 0x7f5c080cb2b0 2023-10-16T09:44:59.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.222+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5c081540f0 con 0x7f5c080cb2b0 2023-10-16T09:44:59.268 INFO:tasks.ceph.mds.c.smithi181.stderr:2023-10-16T09:44:59.267+0000 7f77983c3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:44:59.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.826+0000 7f5bfe7fc700 1 -- 172.21.15.181:0/805310757 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5bf8057d90 con 0x7f5c080cb2b0 2023-10-16T09:44:59.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5bec06af00 msgr2=0x7f5bec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:59.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5bec06af00 0x7f5bec06d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c04007fd0 tx=0x7f5c04009040 comp rx=0 tx=0).stop 2023-10-16T09:44:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cb2b0 msgr2=0x7f5c08145830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cb2b0 0x7f5c08145830 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d800 tx=0x7f5bf800db10 comp rx=0 tx=0).stop 2023-10-16T09:44:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 shutdown_connections 2023-10-16T09:44:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f5bec06af00 0x7f5bec06d3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080cb690 0x7f5c080b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c080b5fd0 0x7f5c080c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 --2- 172.21.15.181:0/805310757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cb2b0 0x7f5c08145830 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 >> 172.21.15.181:0/805310757 conn(0x7f5c080cd010 msgr2=0x7f5c08155a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 shutdown_connections 2023-10-16T09:44:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.829+0000 7f5c1016b700 1 -- 172.21.15.181:0/805310757 wait complete. 2023-10-16T09:44:59.843 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T09:44:59.843 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:44:59.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.963+0000 7fb9b6e92700 1 Processor -- start 2023-10-16T09:44:59.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.963+0000 7fb9b6e92700 1 -- start start 2023-10-16T09:44:59.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9b6e92700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb9b00af450 0x7fb9b00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:59.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9b6e92700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b00cd470 0x7fb9b00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:59.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9b6e92700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b006f870 con 0x7fb9b015dfd0 2023-10-16T09:44:59.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9b6e92700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b00d1160 con 0x7fb9b00af450 2023-10-16T09:44:59.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9b6e92700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00cc8e0 con 0x7fb9b00cd470 2023-10-16T09:44:59.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9affff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb9b00af450 0x7fb9b00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:59.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9affff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb9b00af450 0x7fb9b00af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37966/0 (socket says 172.21.15.181:37966) 2023-10-16T09:44:59.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.964+0000 7fb9affff700 1 -- 172.21.15.181:0/296203001 learned_addr learned my addr 172.21.15.181:0/296203001 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:59.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b542f700 1 --2- 172.21.15.181:0/296203001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b00cd470 0x7fb9b00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:59.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9affff700 1 -- 172.21.15.181:0/296203001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b00cd470 msgr2=0x7fb9b00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:59.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9affff700 1 --2- 172.21.15.181:0/296203001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b00cd470 0x7fb9b00cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9affff700 1 -- 172.21.15.181:0/296203001 >> v1:172.21.15.190:6789/0 conn(0x7fb9b015dfd0 legacy=0x7fb9b006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:44:59.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9affff700 1 -- 172.21.15.181:0/296203001 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00ccbc0 con 0x7fb9b00af450 2023-10-16T09:44:59.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b542f700 1 --2- 172.21.15.181:0/296203001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b00cd470 0x7fb9b00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:59.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9affff700 1 --2- 172.21.15.181:0/296203001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b00af9a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0098f0 tx=0x7fb99c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742bd57982eaa107 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:59.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9af7fe700 1 -- 172.21.15.181:0/296203001 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c004230 con 0x7fb9b00af450 2023-10-16T09:44:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9af7fe700 1 -- 172.21.15.181:0/296203001 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c004390 con 0x7fb9b00af450 2023-10-16T09:44:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9af7fe700 1 -- 172.21.15.181:0/296203001 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0036a0 con 0x7fb9b00af450 2023-10-16T09:44:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 -- 172.21.15.181:0/296203001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 msgr2=0x7fb9b00af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/296203001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b00af9a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0098f0 tx=0x7fb99c009c00 comp rx=0 tx=0).stop 2023-10-16T09:44:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 -- 172.21.15.181:0/296203001 shutdown_connections 2023-10-16T09:44:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/296203001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b00cd470 0x7fb9b00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/296203001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b00af9a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 -- 172.21.15.181:0/296203001 >> 172.21.15.181:0/296203001 conn(0x7fb9b00d04a0 msgr2=0x7fb9b00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:44:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 -- 172.21.15.181:0/296203001 shutdown_connections 2023-10-16T09:44:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.965+0000 7fb9b6e92700 1 -- 172.21.15.181:0/296203001 wait complete. 2023-10-16T09:44:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 Processor -- start 2023-10-16T09:44:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 -- start start 2023-10-16T09:44:59.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b0155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:59.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b00cd470 0x7fb9b015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:59.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b015dfd0 0x7fb9b00b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:59.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b01604b0 con 0x7fb9b00af450 2023-10-16T09:44:59.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0156420 con 0x7fb9b00cd470 2023-10-16T09:44:59.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.967+0000 7fb9b6e92700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b015f830 con 0x7fb9b015dfd0 2023-10-16T09:44:59.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b0155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:59.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b0155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37972/0 (socket says 172.21.15.181:37972) 2023-10-16T09:44:59.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 -- 172.21.15.181:0/49814673 learned_addr learned my addr 172.21.15.181:0/49814673 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:44:59.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 -- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b015dfd0 msgr2=0x7fb9b00b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:59.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b015dfd0 0x7fb9b00b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 -- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b00cd470 msgr2=0x7fb9b015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:44:59.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9affff700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b00cd470 0x7fb9b015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:59.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b00cd470 0x7fb9b015ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:44:59.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 -- 172.21.15.181:0/49814673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c009550 con 0x7fb9b00af450 2023-10-16T09:44:59.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9affff700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b00cd470 0x7fb9b015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:44:59.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b4c2e700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b0155ee0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b450 tx=0x7fb9a400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:59.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9ad7fa700 1 -- 172.21.15.181:0/49814673 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004020 con 0x7fb9b00af450 2023-10-16T09:44:59.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9ad7fa700 1 -- 172.21.15.181:0/49814673 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a4004180 con 0x7fb9b00af450 2023-10-16T09:44:59.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b015fb10 con 0x7fb9b00af450 2023-10-16T09:44:59.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9ad7fa700 1 -- 172.21.15.181:0/49814673 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004530 con 0x7fb9b00af450 2023-10-16T09:44:59.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.968+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b01600d0 con 0x7fb9b00af450 2023-10-16T09:44:59.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.969+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b0067ab0 con 0x7fb9b00af450 2023-10-16T09:44:59.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.970+0000 7fb9ad7fa700 1 -- 172.21.15.181:0/49814673 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb9a40036d0 con 0x7fb9b00af450 2023-10-16T09:44:59.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.970+0000 7fb9ad7fa700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb99406af00 0x7fb99406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:44:59.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.970+0000 7fb9ad7fa700 1 -- 172.21.15.181:0/49814673 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9a4088e30 con 0x7fb9b00af450 2023-10-16T09:44:59.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.970+0000 7fb9affff700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb99406af00 0x7fb99406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:44:59.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.970+0000 7fb9affff700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb99406af00 0x7fb99406d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00cbfe0 tx=0x7fb99c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:44:59.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:44:59.974+0000 7fb9ad7fa700 1 -- 172.21.15.181:0/49814673 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb9a4053ee0 con 0x7fb9b00af450 2023-10-16T09:45:00.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.211+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b0067d20 con 0x7fb9b00af450 2023-10-16T09:45:00.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.211+0000 7fb9ad7fa700 1 -- 172.21.15.181:0/49814673 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fb9a4015070 con 0x7fb9b00af450 2023-10-16T09:45:00.213 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:00.213 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3974533571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3974533571},{"type":"v1","addr":"172.21.15.190:6834","nonce":3974533571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3064364132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3064364132},{"type":"v1","addr":"172.21.15.181:6837","nonce":3064364132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4021280674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4021280674},{"type":"v1","addr":"172.21.15.190:6835","nonce":4021280674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:56.185727+0000","modified":"2023-10-16T09:44:59.222716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6836/760205137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":760205137},{"type":"v1","addr":"172.21.15.181:6836","nonce":760205137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:45:00.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.213+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb99406af00 msgr2=0x7fb99406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.213+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb99406af00 0x7fb99406d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00cbfe0 tx=0x7fb99c008030 comp rx=0 tx=0).stop 2023-10-16T09:45:00.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.213+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 msgr2=0x7fb9b0155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.213+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b0155ee0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b450 tx=0x7fb9a400b810 comp rx=0 tx=0).stop 2023-10-16T09:45:00.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 shutdown_connections 2023-10-16T09:45:00.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fb99406af00 0x7fb99406d3c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b015dfd0 0x7fb9b00b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00af450 0x7fb9b0155ee0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 --2- 172.21.15.181:0/49814673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b00cd470 0x7fb9b015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 >> 172.21.15.181:0/49814673 conn(0x7fb9b00d04a0 msgr2=0x7fb9b00cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:00.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 shutdown_connections 2023-10-16T09:45:00.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.214+0000 7fb9b6e92700 1 -- 172.21.15.181:0/49814673 wait complete. 2023-10-16T09:45:00.218 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2023-10-16T09:45:00.227 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:45:00.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.345+0000 7f4522189700 1 Processor -- start 2023-10-16T09:45:00.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.345+0000 7f4522189700 1 -- start start 2023-10-16T09:45:00.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.346+0000 7f4522189700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0ce7f0 0x7f451c06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.346+0000 7f4522189700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f451c0d2390 0x7f451c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.346+0000 7f4522189700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451c0cc470 con 0x7f451c0d0180 2023-10-16T09:45:00.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.346+0000 7f4522189700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451c0d0630 con 0x7f451c0d2390 2023-10-16T09:45:00.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.346+0000 7f4522189700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c06ca30 con 0x7f451c0ce7f0 2023-10-16T09:45:00.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f4512ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f451c0d2390 0x7f451c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0ce7f0 0x7f451c06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f4512ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f451c0d2390 0x7f451c0cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:43028/0 (socket says 172.21.15.181:43028) 2023-10-16T09:45:00.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f4512ffd700 1 -- 172.21.15.181:0/2581928389 learned_addr learned my addr 172.21.15.181:0/2581928389 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:00.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451b7fe700 1 -- 172.21.15.181:0/2581928389 >> v2:172.21.15.190:3300/0 conn(0x7f451c0d2390 msgr2=0x7f451c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:00.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451b7fe700 1 --2- 172.21.15.181:0/2581928389 >> v2:172.21.15.190:3300/0 conn(0x7f451c0d2390 0x7f451c0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451b7fe700 1 -- 172.21.15.181:0/2581928389 >> v1:172.21.15.190:6789/0 conn(0x7f451c0d0180 legacy=0x7f451c161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451affd700 1 -- 172.21.15.181:0/2581928389 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426666009 0 0) 0x7f451c0cc470 con 0x7f451c0d0180 2023-10-16T09:45:00.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451b7fe700 1 -- 172.21.15.181:0/2581928389 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c06cd10 con 0x7f451c0ce7f0 2023-10-16T09:45:00.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f4512ffd700 1 --2- 172.21.15.181:0/2581928389 >> v2:172.21.15.190:3300/0 conn(0x7f451c0d2390 0x7f451c0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:00.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451b7fe700 1 --2- 172.21.15.181:0/2581928389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0ce7f0 0x7f451c06c100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f450c009820 tx=0x7f450c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79d0deff18ba780 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:00.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451affd700 1 -- 172.21.15.181:0/2581928389 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c005e30 con 0x7f451c0ce7f0 2023-10-16T09:45:00.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451affd700 1 -- 172.21.15.181:0/2581928389 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c006030 con 0x7f451c0ce7f0 2023-10-16T09:45:00.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.347+0000 7f451affd700 1 -- 172.21.15.181:0/2581928389 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c0063c0 con 0x7f451c0ce7f0 2023-10-16T09:45:00.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 -- 172.21.15.181:0/2581928389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0ce7f0 msgr2=0x7f451c06c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 --2- 172.21.15.181:0/2581928389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0ce7f0 0x7f451c06c100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f450c009820 tx=0x7f450c009b30 comp rx=0 tx=0).stop 2023-10-16T09:45:00.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 -- 172.21.15.181:0/2581928389 shutdown_connections 2023-10-16T09:45:00.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 --2- 172.21.15.181:0/2581928389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0ce7f0 0x7f451c06c100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 --2- 172.21.15.181:0/2581928389 >> v2:172.21.15.190:3300/0 conn(0x7f451c0d2390 0x7f451c0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 -- 172.21.15.181:0/2581928389 >> 172.21.15.181:0/2581928389 conn(0x7f451c09a9a0 msgr2=0x7f451c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:00.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 -- 172.21.15.181:0/2581928389 shutdown_connections 2023-10-16T09:45:00.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.348+0000 7f4522189700 1 -- 172.21.15.181:0/2581928389 wait complete. 2023-10-16T09:45:00.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.349+0000 7f4522189700 1 Processor -- start 2023-10-16T09:45:00.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.349+0000 7f4522189700 1 -- start start 2023-10-16T09:45:00.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f4522189700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451c0ce7f0 0x7f451c160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f4522189700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0d0180 0x7f451c0b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f4522189700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c0d2390 0x7f451c0b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f4522189700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c09d440 con 0x7f451c0d2390 2023-10-16T09:45:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f4522189700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c09b4b0 con 0x7f451c0ce7f0 2023-10-16T09:45:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f4522189700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c09b790 con 0x7f451c0d0180 2023-10-16T09:45:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f451bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c0d2390 0x7f451c0b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f451bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c0d2390 0x7f451c0b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:37986/0 (socket says 172.21.15.181:37986) 2023-10-16T09:45:00.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f451bfff700 1 -- 172.21.15.181:0/2073633478 learned_addr learned my addr 172.21.15.181:0/2073633478 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:00.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f451b7fe700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451c0ce7f0 0x7f451c160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f4512ffd700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0d0180 0x7f451c0b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f451bfff700 1 -- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0d0180 msgr2=0x7f451c0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.350+0000 7f451bfff700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0d0180 0x7f451c0b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f451bfff700 1 -- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451c0ce7f0 msgr2=0x7f451c160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f451bfff700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451c0ce7f0 0x7f451c160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f451bfff700 1 -- 172.21.15.181:0/2073633478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4504009550 con 0x7f451c0d2390 2023-10-16T09:45:00.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f4512ffd700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0d0180 0x7f451c0b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:00.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f451bfff700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c0d2390 0x7f451c0b6ec0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f451400c760 tx=0x7f451400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:00.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f4518ff9700 1 -- 172.21.15.181:0/2073633478 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514003e30 con 0x7f451c0d2390 2023-10-16T09:45:00.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f4518ff9700 1 -- 172.21.15.181:0/2073633478 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4514005070 con 0x7f451c0d2390 2023-10-16T09:45:00.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c009430 con 0x7f451c0d2390 2023-10-16T09:45:00.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f4518ff9700 1 -- 172.21.15.181:0/2073633478 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514005400 con 0x7f451c0d2390 2023-10-16T09:45:00.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.351+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451c09bd50 con 0x7f451c0d2390 2023-10-16T09:45:00.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.352+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451c067d20 con 0x7f451c0d2390 2023-10-16T09:45:00.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.352+0000 7f4518ff9700 1 -- 172.21.15.181:0/2073633478 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4514005560 con 0x7f451c0d2390 2023-10-16T09:45:00.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.353+0000 7f4518ff9700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f44fc06af00 0x7f44fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.353+0000 7f451b7fe700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f44fc06af00 0x7f44fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.353+0000 7f4518ff9700 1 -- 172.21.15.181:0/2073633478 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45140891d0 con 0x7f451c0d2390 2023-10-16T09:45:00.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.353+0000 7f451b7fe700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f44fc06af00 0x7f44fc06d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f450c000c00 tx=0x7f450c008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:00.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.357+0000 7f4518ff9700 1 -- 172.21.15.181:0/2073633478 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4514054280 con 0x7f451c0d2390 2023-10-16T09:45:00.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.548+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f451c0cef40 con 0x7f451c0d2390 2023-10-16T09:45:00.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.549+0000 7f4518ff9700 1 -- 172.21.15.181:0/2073633478 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f45140578a0 con 0x7f451c0d2390 2023-10-16T09:45:00.551 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:00.551 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":34,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:44:57.785363+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:44:54.175032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:55.152824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:45:00.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f44fc06af00 msgr2=0x7f44fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f44fc06af00 0x7f44fc06d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f450c000c00 tx=0x7f450c008070 comp rx=0 tx=0).stop 2023-10-16T09:45:00.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c0d2390 msgr2=0x7f451c0b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c0d2390 0x7f451c0b6ec0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f451400c760 tx=0x7f451400ca70 comp rx=0 tx=0).stop 2023-10-16T09:45:00.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 shutdown_connections 2023-10-16T09:45:00.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f44fc06af00 0x7f44fc06d3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f451c0d2390 0x7f451c0b6ec0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451c0ce7f0 0x7f451c160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 --2- 172.21.15.181:0/2073633478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f451c0d0180 0x7f451c0b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 >> 172.21.15.181:0/2073633478 conn(0x7f451c09a9a0 msgr2=0x7f451c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 shutdown_connections 2023-10-16T09:45:00.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.552+0000 7f4522189700 1 -- 172.21.15.181:0/2073633478 wait complete. 2023-10-16T09:45:00.565 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T09:45:00.566 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:45:00.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.684+0000 7f97baed3700 1 Processor -- start 2023-10-16T09:45:00.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.684+0000 7f97baed3700 1 -- start start 2023-10-16T09:45:00.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.685+0000 7f97baed3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97b415cd20 0x7f97b40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.685+0000 7f97baed3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b40cd840 con 0x7f97b40cd3f0 2023-10-16T09:45:00.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.685+0000 7f97baed3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b40cee70 con 0x7f97b4159730 2023-10-16T09:45:00.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.685+0000 7f97baed3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b4159b10 con 0x7f97b415cd20 2023-10-16T09:45:00.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.685+0000 7f97b9470700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f97b40cd3f0 0x7f97b40cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49072/0 (socket says 172.21.15.181:49072) 2023-10-16T09:45:00.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.685+0000 7f97b9470700 1 -- 172.21.15.181:0/3101363504 learned_addr learned my addr 172.21.15.181:0/3101363504 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:00.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.685+0000 7f97b3fff700 1 --2- 172.21.15.181:0/3101363504 >> v2:172.21.15.190:3300/0 conn(0x7f97b415cd20 0x7f97b40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335997037 0 0) 0x7f97ac001490 con 0x7f97b40cd3f0 2023-10-16T09:45:00.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a40035b0 con 0x7f97b40cd3f0 2023-10-16T09:45:00.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 536086745 0 0) 0x7f97a40035b0 con 0x7f97b40cd3f0 2023-10-16T09:45:00.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 >> v2:172.21.15.190:3300/0 conn(0x7f97b415cd20 msgr2=0x7f97b40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:00.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 --2- 172.21.15.181:0/3101363504 >> v2:172.21.15.190:3300/0 conn(0x7f97b415cd20 0x7f97b40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 >> v1:172.21.15.190:6789/0 conn(0x7f97b4159730 legacy=0x7f97b40cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40ce540 con 0x7f97b40cd3f0 2023-10-16T09:45:00.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f97ac001b80 con 0x7f97b40cd3f0 2023-10-16T09:45:00.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 >> v1:172.21.15.181:6789/0 conn(0x7f97b40cd3f0 legacy=0x7f97b40cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4003a00 0x7f97a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a4004850 0x7f97a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97a4005210 0x7f97a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4159b10 con 0x7f97a4004850 2023-10-16T09:45:00.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a400c0b0 con 0x7f97a4003a00 2023-10-16T09:45:00.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a400c270 con 0x7f97a4005210 2023-10-16T09:45:00.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b3fff700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a4004850 0x7f97a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b8c6f700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4003a00 0x7f97a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b9470700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97a4005210 0x7f97a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b3fff700 1 -- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97a4005210 msgr2=0x7f97a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:00.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b3fff700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97a4005210 0x7f97a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b3fff700 1 -- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4003a00 msgr2=0x7f97a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:00.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b3fff700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4003a00 0x7f97a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b3fff700 1 -- 172.21.15.181:0/3101363504 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40035b0 con 0x7f97a4004850 2023-10-16T09:45:00.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b9470700 1 -- 172.21.15.181:0/3101363504 reap_dead start 2023-10-16T09:45:00.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b3fff700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a4004850 0x7f97a4004cd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009b60 tx=0x7f97a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c71fbbc8d02a7c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:00.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.686+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000e6c0 con 0x7f97a4004850 2023-10-16T09:45:00.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000e820 con 0x7f97a4004850 2023-10-16T09:45:00.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97b37fe700 1 -- 172.21.15.181:0/3101363504 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000ebb0 con 0x7f97a4004850 2023-10-16T09:45:00.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97baed3700 1 -- 172.21.15.181:0/3101363504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a4004850 msgr2=0x7f97a4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97baed3700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a4004850 0x7f97a4004cd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009b60 tx=0x7f97a4005880 comp rx=0 tx=0).stop 2023-10-16T09:45:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97baed3700 1 -- 172.21.15.181:0/3101363504 shutdown_connections 2023-10-16T09:45:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97baed3700 1 --2- 172.21.15.181:0/3101363504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a4004850 0x7f97a4004cd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97baed3700 1 -- 172.21.15.181:0/3101363504 >> 172.21.15.181:0/3101363504 conn(0x7f97b40d04a0 msgr2=0x7f97b40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97baed3700 1 -- 172.21.15.181:0/3101363504 shutdown_connections 2023-10-16T09:45:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.687+0000 7f97baed3700 1 -- 172.21.15.181:0/3101363504 wait complete. 2023-10-16T09:45:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.688+0000 7f97baed3700 1 Processor -- start 2023-10-16T09:45:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.688+0000 7f97baed3700 1 -- start start 2023-10-16T09:45:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97baed3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4004850 0x7f97b4150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97baed3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b40b3500 0x7f97b41f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97baed3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b40b4a60 0x7f97b40c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97baed3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4151010 con 0x7f97b40b3500 2023-10-16T09:45:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97baed3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40c9ff0 con 0x7f97a4004850 2023-10-16T09:45:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97baed3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41f7c20 con 0x7f97b40b4a60 2023-10-16T09:45:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97b3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b40b3500 0x7f97b41f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97b8c6f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4004850 0x7f97b4150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97b8c6f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4004850 0x7f97b4150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:43052/0 (socket says 172.21.15.181:43052) 2023-10-16T09:45:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97b3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b40b3500 0x7f97b41f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38006/0 (socket says 172.21.15.181:38006) 2023-10-16T09:45:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97b8c6f700 1 -- 172.21.15.181:0/895546333 learned_addr learned my addr 172.21.15.181:0/895546333 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.689+0000 7f97b9470700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b40b4a60 0x7f97b40c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b3fff700 1 -- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b40b4a60 msgr2=0x7f97b40c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b3fff700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b40b4a60 0x7f97b40c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b3fff700 1 -- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4004850 msgr2=0x7f97b4150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b3fff700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4004850 0x7f97b4150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b3fff700 1 -- 172.21.15.181:0/895546333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40b39c0 con 0x7f97b40b3500 2023-10-16T09:45:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b8c6f700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4004850 0x7f97b4150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b3fff700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b40b3500 0x7f97b41f76e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f97a0019000 tx=0x7f97a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b17fa700 1 -- 172.21.15.181:0/895546333 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000e6c0 con 0x7f97b40b3500 2023-10-16T09:45:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b17fa700 1 -- 172.21.15.181:0/895546333 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a00038e0 con 0x7f97b40b3500 2023-10-16T09:45:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b4151010 con 0x7f97b40b3500 2023-10-16T09:45:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97b17fa700 1 -- 172.21.15.181:0/895546333 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0003c90 con 0x7f97b40b3500 2023-10-16T09:45:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.690+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b40b3f80 con 0x7f97b40b3500 2023-10-16T09:45:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.691+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4067d20 con 0x7f97b40b3500 2023-10-16T09:45:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.691+0000 7f97b17fa700 1 -- 172.21.15.181:0/895546333 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97a0013470 con 0x7f97b40b3500 2023-10-16T09:45:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.692+0000 7f97b17fa700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f979806af00 0x7f979806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.692+0000 7f97b8c6f700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f979806af00 0x7f979806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.692+0000 7f97b17fa700 1 -- 172.21.15.181:0/895546333 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97a008ae90 con 0x7f97b40b3500 2023-10-16T09:45:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.692+0000 7f97b8c6f700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f979806af00 0x7f979806d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f97b41f4850 tx=0x7f97a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.696+0000 7f97b17fa700 1 -- 172.21.15.181:0/895546333 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f97a00560f0 con 0x7f97b40b3500 2023-10-16T09:45:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.885+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97b40ceb00 con 0x7f979806af00 2023-10-16T09:45:00.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.892+0000 7f97b17fa700 1 -- 172.21.15.181:0/895546333 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251584 (secure 0 0 0) 0x7f97a8012b40 con 0x7f979806af00 2023-10-16T09:45:00.896 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:00.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f979806af00 msgr2=0x7f979806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f979806af00 0x7f979806d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f97b41f4850 tx=0x7f97a8009040 comp rx=0 tx=0).stop 2023-10-16T09:45:00.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b40b3500 msgr2=0x7f97b41f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:00.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b40b3500 0x7f97b41f76e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f97a0019000 tx=0x7f97a00047c0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 shutdown_connections 2023-10-16T09:45:00.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f979806af00 0x7f979806d3c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97b40b4a60 0x7f97b40c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a4004850 0x7f97b4150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 --2- 172.21.15.181:0/895546333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b40b3500 0x7f97b41f76e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:00.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 >> 172.21.15.181:0/895546333 conn(0x7f97b40d04a0 msgr2=0x7f97b4159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:00.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 shutdown_connections 2023-10-16T09:45:00.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:00.895+0000 7f97baed3700 1 -- 172.21.15.181:0/895546333 wait complete. 2023-10-16T09:45:00.900 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:45:00.908 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-16T09:44:59.689705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":219960,"kb_used_data":3752,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754760,"statfs":{"total":773094113280,"available":772868874240,"internally_reserved":0,"allocated":3842048,"data_stored":2158432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.909786"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783396+0000","last_change":"2023-10-16T09:44:54.783396+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783396+0000","last_undegraded":"2023-10-16T09:44:54.783396+0000","last_fullsized":"2023-10-16T09:44:54.783396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.789240+0000","last_change":"2023-10-16T09:44:54.789240+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.789240+0000","last_undegraded":"2023-10-16T09:44:54.789240+0000","last_fullsized":"2023-10-16T09:44:54.789240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788635+0000","last_change":"2023-10-16T09:44:54.788635+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788635+0000","last_undegraded":"2023-10-16T09:44:54.788635+0000","last_fullsized":"2023-10-16T09:44:54.788635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783564+0000","last_change":"2023-10-16T09:44:54.783564+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783564+0000","last_undegraded":"2023-10-16T09:44:54.783564+0000","last_fullsized":"2023-10-16T09:44:54.783564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788637+0000","last_change":"2023-10-16T09:44:54.788637+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788637+0000","last_undegraded":"2023-10-16T09:44:54.788637+0000","last_fullsized":"2023-10-16T09:44:54.788637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782624+0000","last_change":"2023-10-16T09:44:54.782624+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782624+0000","last_undegraded":"2023-10-16T09:44:54.782624+0000","last_fullsized":"2023-10-16T09:44:54.782624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782157+0000","last_change":"2023-10-16T09:44:54.782157+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782157+0000","last_undegraded":"2023-10-16T09:44:54.782157+0000","last_fullsized":"2023-10-16T09:44:54.782157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782906+0000","last_change":"2023-10-16T09:44:54.782906+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782906+0000","last_undegraded":"2023-10-16T09:44:54.782906+0000","last_fullsized":"2023-10-16T09:44:54.782906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788664+0000","last_change":"2023-10-16T09:44:54.788664+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788664+0000","last_undegraded":"2023-10-16T09:44:54.788664+0000","last_fullsized":"2023-10-16T09:44:54.788664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.785238+0000","last_change":"2023-10-16T09:44:54.785238+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.785238+0000","last_undegraded":"2023-10-16T09:44:54.785238+0000","last_fullsized":"2023-10-16T09:44:54.785238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782961+0000","last_change":"2023-10-16T09:44:54.782961+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782961+0000","last_undegraded":"2023-10-16T09:44:54.782961+0000","last_fullsized":"2023-10-16T09:44:54.782961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783462+0000","last_change":"2023-10-16T09:44:54.783462+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783462+0000","last_undegraded":"2023-10-16T09:44:54.783462+0000","last_fullsized":"2023-10-16T09:44:54.783462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.785279+0000","last_change":"2023-10-16T09:44:54.785279+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.785279+0000","last_undegraded":"2023-10-16T09:44:54.785279+0000","last_fullsized":"2023-10-16T09:44:54.785279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788799+0000","last_change":"2023-10-16T09:44:54.788799+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788799+0000","last_undegraded":"2023-10-16T09:44:54.788799+0000","last_fullsized":"2023-10-16T09:44:54.788799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788771+0000","last_change":"2023-10-16T09:44:54.788771+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788771+0000","last_undegraded":"2023-10-16T09:44:54.788771+0000","last_fullsized":"2023-10-16T09:44:54.788771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788636+0000","last_change":"2023-10-16T09:44:54.788636+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788636+0000","last_undegraded":"2023-10-16T09:44:54.788636+0000","last_fullsized":"2023-10-16T09:44:54.788636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.789408+0000","last_change":"2023-10-16T09:44:54.789408+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.789408+0000","last_undegraded":"2023-10-16T09:44:54.789408+0000","last_fullsized":"2023-10-16T09:44:54.789408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782157+0000","last_change":"2023-10-16T09:44:54.782157+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782157+0000","last_undegraded":"2023-10-16T09:44:54.782157+0000","last_fullsized":"2023-10-16T09:44:54.782157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784847+0000","last_change":"2023-10-16T09:44:54.784847+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784847+0000","last_undegraded":"2023-10-16T09:44:54.784847+0000","last_fullsized":"2023-10-16T09:44:54.784847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784904+0000","last_change":"2023-10-16T09:44:54.784904+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784904+0000","last_undegraded":"2023-10-16T09:44:54.784904+0000","last_fullsized":"2023-10-16T09:44:54.784904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784819+0000","last_change":"2023-10-16T09:44:54.784819+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784819+0000","last_undegraded":"2023-10-16T09:44:54.784819+0000","last_fullsized":"2023-10-16T09:44:54.784819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788613+0000","last_change":"2023-10-16T09:44:54.788613+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788613+0000","last_undegraded":"2023-10-16T09:44:54.788613+0000","last_fullsized":"2023-10-16T09:44:54.788613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782970+0000","last_change":"2023-10-16T09:44:54.782970+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782970+0000","last_undegraded":"2023-10-16T09:44:54.782970+0000","last_fullsized":"2023-10-16T09:44:54.782970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":173,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-16T09:44:49.920853+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:49.920853+0000","last_peered":"2023-10-16T09:44:49.920853+0000","last_clean":"2023-10-16T09:44:49.920853+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:49.920853+0000","last_undegraded":"2023-10-16T09:44:49.920853+0000","last_fullsized":"2023-10-16T09:44:49.920853+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":925696,"data_stored":712580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":925696,"data_stored":712580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:45:00.915 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:45:01.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.068+0000 7f45576f7700 1 Processor -- start 2023-10-16T09:45:01.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.069+0000 7f45576f7700 1 -- start start 2023-10-16T09:45:01.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.069+0000 7f45576f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015e6d0 0x7f45500d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.069+0000 7f45576f7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f455015eab0 0x7f455015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.069+0000 7f45576f7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455006ed00 con 0x7f45500cb0c0 2023-10-16T09:45:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.069+0000 7f45576f7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45500d02d0 con 0x7f455015eab0 2023-10-16T09:45:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.069+0000 7f45576f7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455015ad40 con 0x7f455015e6d0 2023-10-16T09:45:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4554c92700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f455015eab0 0x7f455015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4555c94700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f45500cb0c0 0x7f455015a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49092/0 (socket says 172.21.15.181:49092) 2023-10-16T09:45:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4555c94700 1 -- 172.21.15.181:0/2975240320 learned_addr learned my addr 172.21.15.181:0/2975240320 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4555493700 1 --2- 172.21.15.181:0/2975240320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015e6d0 0x7f45500d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4554c92700 1 -- 172.21.15.181:0/2975240320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015e6d0 msgr2=0x7f45500d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4554c92700 1 --2- 172.21.15.181:0/2975240320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015e6d0 0x7f45500d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4554c92700 1 -- 172.21.15.181:0/2975240320 >> v1:172.21.15.181:6789/0 conn(0x7f45500cb0c0 legacy=0x7f455015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4554c92700 1 -- 172.21.15.181:0/2975240320 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455006e0f0 con 0x7f455015eab0 2023-10-16T09:45:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4547fff700 1 -- 172.21.15.181:0/2975240320 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608629059 0 0) 0x7f455006ed00 con 0x7f45500cb0c0 2023-10-16T09:45:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4554c92700 1 --2- 172.21.15.181:0/2975240320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455015eab0 0x7f455015a0f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f454c009840 tx=0x7f454c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931dd664cb580aa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4547fff700 1 -- 172.21.15.181:0/2975240320 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c011830 con 0x7f455015eab0 2023-10-16T09:45:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4547fff700 1 -- 172.21.15.181:0/2975240320 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c011990 con 0x7f455015eab0 2023-10-16T09:45:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.070+0000 7f4547fff700 1 -- 172.21.15.181:0/2975240320 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c011d20 con 0x7f455015eab0 2023-10-16T09:45:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 -- 172.21.15.181:0/2975240320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455015eab0 msgr2=0x7f455015a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 --2- 172.21.15.181:0/2975240320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455015eab0 0x7f455015a0f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f454c009840 tx=0x7f454c009b50 comp rx=0 tx=0).stop 2023-10-16T09:45:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 -- 172.21.15.181:0/2975240320 shutdown_connections 2023-10-16T09:45:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 --2- 172.21.15.181:0/2975240320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015e6d0 0x7f45500d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 --2- 172.21.15.181:0/2975240320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455015eab0 0x7f455015a0f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 -- 172.21.15.181:0/2975240320 >> 172.21.15.181:0/2975240320 conn(0x7f45500d04a0 msgr2=0x7f45500d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 -- 172.21.15.181:0/2975240320 shutdown_connections 2023-10-16T09:45:01.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.071+0000 7f45576f7700 1 -- 172.21.15.181:0/2975240320 wait complete. 2023-10-16T09:45:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.072+0000 7f45576f7700 1 Processor -- start 2023-10-16T09:45:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.072+0000 7f45576f7700 1 -- start start 2023-10-16T09:45:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f45576f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45500cb0c0 0x7f45501575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f45576f7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f455015e6d0 0x7f45500b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f45576f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015eab0 0x7f45500b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f45576f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455015d150 con 0x7f45500cb0c0 2023-10-16T09:45:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f45576f7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500c6480 con 0x7f455015e6d0 2023-10-16T09:45:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f45576f7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500c69b0 con 0x7f455015eab0 2023-10-16T09:45:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45500cb0c0 0x7f45501575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45500cb0c0 0x7f45501575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38018/0 (socket says 172.21.15.181:38018) 2023-10-16T09:45:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 -- 172.21.15.181:0/2865683358 learned_addr learned my addr 172.21.15.181:0/2865683358 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4554c92700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f455015e6d0 0x7f45500b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555c94700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015eab0 0x7f45500b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 -- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015eab0 msgr2=0x7f45500b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015eab0 0x7f45500b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 -- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f455015e6d0 msgr2=0x7f45500b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f455015e6d0 0x7f45500b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 -- 172.21.15.181:0/2865683358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0094a0 con 0x7f45500cb0c0 2023-10-16T09:45:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4554c92700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f455015e6d0 0x7f45500b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.073+0000 7f4555493700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45500cb0c0 0x7f45501575d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f454000e9b0 tx=0x7f454000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.074+0000 7f4545ffb700 1 -- 172.21.15.181:0/2865683358 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540009670 con 0x7f45500cb0c0 2023-10-16T09:45:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.074+0000 7f4545ffb700 1 -- 172.21.15.181:0/2865683358 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000c850 con 0x7f45500cb0c0 2023-10-16T09:45:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.074+0000 7f4545ffb700 1 -- 172.21.15.181:0/2865683358 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000cbe0 con 0x7f45500cb0c0 2023-10-16T09:45:01.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.074+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45500c66d0 con 0x7f45500cb0c0 2023-10-16T09:45:01.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.074+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455015cd70 con 0x7f45500cb0c0 2023-10-16T09:45:01.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.075+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4550067d20 con 0x7f45500cb0c0 2023-10-16T09:45:01.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.075+0000 7f4545ffb700 1 -- 172.21.15.181:0/2865683358 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f454000cd40 con 0x7f45500cb0c0 2023-10-16T09:45:01.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.075+0000 7f4545ffb700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f453406af00 0x7f453406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.076+0000 7f4545ffb700 1 -- 172.21.15.181:0/2865683358 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4540089990 con 0x7f45500cb0c0 2023-10-16T09:45:01.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.076+0000 7f4554c92700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f453406af00 0x7f453406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.076+0000 7f4554c92700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f453406af00 0x7f453406d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.080+0000 7f4545ffb700 1 -- 172.21.15.181:0/2865683358 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4540054ad0 con 0x7f45500cb0c0 2023-10-16T09:45:01.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.266+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f455006e6b0 con 0x7f453406af00 2023-10-16T09:45:01.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.273+0000 7f4545ffb700 1 -- 172.21.15.181:0/2865683358 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251584 (secure 0 0 0) 0x7f454c011cb0 con 0x7f453406af00 2023-10-16T09:45:01.278 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:01.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f453406af00 msgr2=0x7f453406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f453406af00 0x7f453406d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c008030 comp rx=0 tx=0).stop 2023-10-16T09:45:01.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45500cb0c0 msgr2=0x7f45501575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45500cb0c0 0x7f45501575d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f454000e9b0 tx=0x7f454000ed70 comp rx=0 tx=0).stop 2023-10-16T09:45:01.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 shutdown_connections 2023-10-16T09:45:01.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f453406af00 0x7f453406d3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f455015eab0 0x7f45500b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45500cb0c0 0x7f45501575d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 --2- 172.21.15.181:0/2865683358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f455015e6d0 0x7f45500b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 >> 172.21.15.181:0/2865683358 conn(0x7f45500d04a0 msgr2=0x7f45500cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 shutdown_connections 2023-10-16T09:45:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.276+0000 7f45576f7700 1 -- 172.21.15.181:0/2865683358 wait complete. 2023-10-16T09:45:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:45:01.289 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-16T09:44:59.689705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":219960,"kb_used_data":3752,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754760,"statfs":{"total":773094113280,"available":772868874240,"internally_reserved":0,"allocated":3842048,"data_stored":2158432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.909786"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783396+0000","last_change":"2023-10-16T09:44:54.783396+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783396+0000","last_undegraded":"2023-10-16T09:44:54.783396+0000","last_fullsized":"2023-10-16T09:44:54.783396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.789240+0000","last_change":"2023-10-16T09:44:54.789240+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.789240+0000","last_undegraded":"2023-10-16T09:44:54.789240+0000","last_fullsized":"2023-10-16T09:44:54.789240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788635+0000","last_change":"2023-10-16T09:44:54.788635+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788635+0000","last_undegraded":"2023-10-16T09:44:54.788635+0000","last_fullsized":"2023-10-16T09:44:54.788635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783564+0000","last_change":"2023-10-16T09:44:54.783564+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783564+0000","last_undegraded":"2023-10-16T09:44:54.783564+0000","last_fullsized":"2023-10-16T09:44:54.783564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788637+0000","last_change":"2023-10-16T09:44:54.788637+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788637+0000","last_undegraded":"2023-10-16T09:44:54.788637+0000","last_fullsized":"2023-10-16T09:44:54.788637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782624+0000","last_change":"2023-10-16T09:44:54.782624+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782624+0000","last_undegraded":"2023-10-16T09:44:54.782624+0000","last_fullsized":"2023-10-16T09:44:54.782624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782157+0000","last_change":"2023-10-16T09:44:54.782157+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782157+0000","last_undegraded":"2023-10-16T09:44:54.782157+0000","last_fullsized":"2023-10-16T09:44:54.782157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782906+0000","last_change":"2023-10-16T09:44:54.782906+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782906+0000","last_undegraded":"2023-10-16T09:44:54.782906+0000","last_fullsized":"2023-10-16T09:44:54.782906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788664+0000","last_change":"2023-10-16T09:44:54.788664+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788664+0000","last_undegraded":"2023-10-16T09:44:54.788664+0000","last_fullsized":"2023-10-16T09:44:54.788664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.785238+0000","last_change":"2023-10-16T09:44:54.785238+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.785238+0000","last_undegraded":"2023-10-16T09:44:54.785238+0000","last_fullsized":"2023-10-16T09:44:54.785238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782961+0000","last_change":"2023-10-16T09:44:54.782961+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782961+0000","last_undegraded":"2023-10-16T09:44:54.782961+0000","last_fullsized":"2023-10-16T09:44:54.782961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783462+0000","last_change":"2023-10-16T09:44:54.783462+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783462+0000","last_undegraded":"2023-10-16T09:44:54.783462+0000","last_fullsized":"2023-10-16T09:44:54.783462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.785279+0000","last_change":"2023-10-16T09:44:54.785279+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.785279+0000","last_undegraded":"2023-10-16T09:44:54.785279+0000","last_fullsized":"2023-10-16T09:44:54.785279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788799+0000","last_change":"2023-10-16T09:44:54.788799+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788799+0000","last_undegraded":"2023-10-16T09:44:54.788799+0000","last_fullsized":"2023-10-16T09:44:54.788799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788771+0000","last_change":"2023-10-16T09:44:54.788771+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788771+0000","last_undegraded":"2023-10-16T09:44:54.788771+0000","last_fullsized":"2023-10-16T09:44:54.788771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788636+0000","last_change":"2023-10-16T09:44:54.788636+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788636+0000","last_undegraded":"2023-10-16T09:44:54.788636+0000","last_fullsized":"2023-10-16T09:44:54.788636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.789408+0000","last_change":"2023-10-16T09:44:54.789408+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.789408+0000","last_undegraded":"2023-10-16T09:44:54.789408+0000","last_fullsized":"2023-10-16T09:44:54.789408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782157+0000","last_change":"2023-10-16T09:44:54.782157+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782157+0000","last_undegraded":"2023-10-16T09:44:54.782157+0000","last_fullsized":"2023-10-16T09:44:54.782157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784847+0000","last_change":"2023-10-16T09:44:54.784847+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784847+0000","last_undegraded":"2023-10-16T09:44:54.784847+0000","last_fullsized":"2023-10-16T09:44:54.784847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784904+0000","last_change":"2023-10-16T09:44:54.784904+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784904+0000","last_undegraded":"2023-10-16T09:44:54.784904+0000","last_fullsized":"2023-10-16T09:44:54.784904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784819+0000","last_change":"2023-10-16T09:44:54.784819+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784819+0000","last_undegraded":"2023-10-16T09:44:54.784819+0000","last_fullsized":"2023-10-16T09:44:54.784819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788613+0000","last_change":"2023-10-16T09:44:54.788613+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788613+0000","last_undegraded":"2023-10-16T09:44:54.788613+0000","last_fullsized":"2023-10-16T09:44:54.788613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782970+0000","last_change":"2023-10-16T09:44:54.782970+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782970+0000","last_undegraded":"2023-10-16T09:44:54.782970+0000","last_fullsized":"2023-10-16T09:44:54.782970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":173,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-16T09:44:49.920853+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:49.920853+0000","last_peered":"2023-10-16T09:44:49.920853+0000","last_clean":"2023-10-16T09:44:49.920853+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:49.920853+0000","last_undegraded":"2023-10-16T09:44:49.920853+0000","last_fullsized":"2023-10-16T09:44:49.920853+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":925696,"data_stored":712580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":925696,"data_stored":712580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:45:01.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:45:01.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.449+0000 7f1e6246c700 1 Processor -- start 2023-10-16T09:45:01.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.449+0000 7f1e6246c700 1 -- start start 2023-10-16T09:45:01.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e6246c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e5c0ce7f0 0x7f1e5c06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e6246c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0d0350 0x7f1e5c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e6246c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0cc470 con 0x7f1e5c0d2390 2023-10-16T09:45:01.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e6246c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c06ca30 con 0x7f1e5c0ce7f0 2023-10-16T09:45:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e6246c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c06cd10 con 0x7f1e5c0d0350 2023-10-16T09:45:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e5c0ce7f0 0x7f1e5c06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1e5c0d2390 0x7f1e5c0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49102/0 (socket says 172.21.15.181:49102) 2023-10-16T09:45:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5b7fe700 1 -- 172.21.15.181:0/3618066496 learned_addr learned my addr 172.21.15.181:0/3618066496 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:01.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e60a09700 1 --2- 172.21.15.181:0/3618066496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0d0350 0x7f1e5c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5bfff700 1 -- 172.21.15.181:0/3618066496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0d0350 msgr2=0x7f1e5c0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5bfff700 1 --2- 172.21.15.181:0/3618066496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0d0350 0x7f1e5c0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5bfff700 1 -- 172.21.15.181:0/3618066496 >> v1:172.21.15.181:6789/0 conn(0x7f1e5c0d2390 legacy=0x7f1e5c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5bfff700 1 -- 172.21.15.181:0/3618066496 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c06cff0 con 0x7f1e5c0ce7f0 2023-10-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.450+0000 7f1e5affd700 1 -- 172.21.15.181:0/3618066496 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968606572 0 0) 0x7f1e5c0cc470 con 0x7f1e5c0d2390 2023-10-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e5bfff700 1 --2- 172.21.15.181:0/3618066496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0ce7f0 0x7f1e5c06c100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009820 tx=0x7f1e50009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177d037c42ec113c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e5affd700 1 -- 172.21.15.181:0/3618066496 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50005e30 con 0x7f1e5c0ce7f0 2023-10-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e5affd700 1 -- 172.21.15.181:0/3618066496 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50006030 con 0x7f1e5c0ce7f0 2023-10-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e5affd700 1 -- 172.21.15.181:0/3618066496 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500063e0 con 0x7f1e5c0ce7f0 2023-10-16T09:45:01.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 -- 172.21.15.181:0/3618066496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0ce7f0 msgr2=0x7f1e5c06c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 --2- 172.21.15.181:0/3618066496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0ce7f0 0x7f1e5c06c100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009820 tx=0x7f1e50009b30 comp rx=0 tx=0).stop 2023-10-16T09:45:01.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 -- 172.21.15.181:0/3618066496 shutdown_connections 2023-10-16T09:45:01.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 --2- 172.21.15.181:0/3618066496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0d0350 0x7f1e5c0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 --2- 172.21.15.181:0/3618066496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0ce7f0 0x7f1e5c06c100 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 -- 172.21.15.181:0/3618066496 >> 172.21.15.181:0/3618066496 conn(0x7f1e5c09a9a0 msgr2=0x7f1e5c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:01.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 -- 172.21.15.181:0/3618066496 shutdown_connections 2023-10-16T09:45:01.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.451+0000 7f1e6246c700 1 -- 172.21.15.181:0/3618066496 wait complete. 2023-10-16T09:45:01.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.452+0000 7f1e6246c700 1 Processor -- start 2023-10-16T09:45:01.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.453+0000 7f1e6246c700 1 -- start start 2023-10-16T09:45:01.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.453+0000 7f1e6246c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0ce7f0 0x7f1e5c152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.453+0000 7f1e6246c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0d0350 0x7f1e5c0b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.453+0000 7f1e6246c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0d2390 0x7f1e5c09bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.453+0000 7f1e6246c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0b58e0 con 0x7f1e5c0d0350 2023-10-16T09:45:01.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.453+0000 7f1e6246c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1565d0 con 0x7f1e5c0d2390 2023-10-16T09:45:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.453+0000 7f1e6246c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c1568b0 con 0x7f1e5c0ce7f0 2023-10-16T09:45:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0d0350 0x7f1e5c0b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0d0350 0x7f1e5c0b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38032/0 (socket says 172.21.15.181:38032) 2023-10-16T09:45:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 -- 172.21.15.181:0/2408352134 learned_addr learned my addr 172.21.15.181:0/2408352134 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:01.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5bfff700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0ce7f0 0x7f1e5c152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e60a09700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0d2390 0x7f1e5c09bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 -- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0ce7f0 msgr2=0x7f1e5c152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0ce7f0 0x7f1e5c152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 -- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0d2390 msgr2=0x7f1e5c09bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0d2390 0x7f1e5c09bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 -- 172.21.15.181:0/2408352134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e50009430 con 0x7f1e5c0d0350 2023-10-16T09:45:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5bfff700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0ce7f0 0x7f1e5c152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e5b7fe700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0d0350 0x7f1e5c0b5b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0011e0 tx=0x7f1e4c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e58ff9700 1 -- 172.21.15.181:0/2408352134 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c004c80 con 0x7f1e5c0d0350 2023-10-16T09:45:01.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e58ff9700 1 -- 172.21.15.181:0/2408352134 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c004e00 con 0x7f1e5c0d0350 2023-10-16T09:45:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c09dd30 con 0x7f1e5c0d0350 2023-10-16T09:45:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e58ff9700 1 -- 172.21.15.181:0/2408352134 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c007310 con 0x7f1e5c0d0350 2023-10-16T09:45:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.454+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c09e2f0 con 0x7f1e5c0d0350 2023-10-16T09:45:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.455+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c09e510 con 0x7f1e5c0d0350 2023-10-16T09:45:01.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.455+0000 7f1e58ff9700 1 -- 172.21.15.181:0/2408352134 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1e4c007470 con 0x7f1e5c0d0350 2023-10-16T09:45:01.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.456+0000 7f1e58ff9700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1e4806af30 0x7f1e4806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.456+0000 7f1e5bfff700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1e4806af30 0x7f1e4806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.456+0000 7f1e58ff9700 1 -- 172.21.15.181:0/2408352134 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e4c0893c0 con 0x7f1e5c0d0350 2023-10-16T09:45:01.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.456+0000 7f1e5bfff700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1e4806af30 0x7f1e4806d3f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e50013040 tx=0x7f1e50008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.460+0000 7f1e58ff9700 1 -- 172.21.15.181:0/2408352134 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e4c054470 con 0x7f1e5c0d0350 2023-10-16T09:45:01.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.648+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e5c06d2d0 con 0x7f1e4806af30 2023-10-16T09:45:01.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.655+0000 7f1e58ff9700 1 -- 172.21.15.181:0/2408352134 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251584 (secure 0 0 0) 0x7f1e50005db0 con 0x7f1e4806af30 2023-10-16T09:45:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:01.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1e4806af30 msgr2=0x7f1e4806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1e4806af30 0x7f1e4806d3f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e50013040 tx=0x7f1e50008070 comp rx=0 tx=0).stop 2023-10-16T09:45:01.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0d0350 msgr2=0x7f1e5c0b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0d0350 0x7f1e5c0b5b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0011e0 tx=0x7f1e4c00cae0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 shutdown_connections 2023-10-16T09:45:01.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f1e4806af30 0x7f1e4806d3f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c0ce7f0 0x7f1e5c152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0d2390 0x7f1e5c09bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 --2- 172.21.15.181:0/2408352134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e5c0d0350 0x7f1e5c0b5b80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 >> 172.21.15.181:0/2408352134 conn(0x7f1e5c09a9a0 msgr2=0x7f1e5c0c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 shutdown_connections 2023-10-16T09:45:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.658+0000 7f1e6246c700 1 -- 172.21.15.181:0/2408352134 wait complete. 2023-10-16T09:45:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:45:01.671 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-16T09:44:59.689705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":219960,"kb_used_data":3752,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754760,"statfs":{"total":773094113280,"available":772868874240,"internally_reserved":0,"allocated":3842048,"data_stored":2158432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.909786"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783396+0000","last_change":"2023-10-16T09:44:54.783396+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783396+0000","last_undegraded":"2023-10-16T09:44:54.783396+0000","last_fullsized":"2023-10-16T09:44:54.783396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.789240+0000","last_change":"2023-10-16T09:44:54.789240+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.789240+0000","last_undegraded":"2023-10-16T09:44:54.789240+0000","last_fullsized":"2023-10-16T09:44:54.789240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788635+0000","last_change":"2023-10-16T09:44:54.788635+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788635+0000","last_undegraded":"2023-10-16T09:44:54.788635+0000","last_fullsized":"2023-10-16T09:44:54.788635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783564+0000","last_change":"2023-10-16T09:44:54.783564+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783564+0000","last_undegraded":"2023-10-16T09:44:54.783564+0000","last_fullsized":"2023-10-16T09:44:54.783564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788637+0000","last_change":"2023-10-16T09:44:54.788637+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788637+0000","last_undegraded":"2023-10-16T09:44:54.788637+0000","last_fullsized":"2023-10-16T09:44:54.788637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782624+0000","last_change":"2023-10-16T09:44:54.782624+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782624+0000","last_undegraded":"2023-10-16T09:44:54.782624+0000","last_fullsized":"2023-10-16T09:44:54.782624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782157+0000","last_change":"2023-10-16T09:44:54.782157+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782157+0000","last_undegraded":"2023-10-16T09:44:54.782157+0000","last_fullsized":"2023-10-16T09:44:54.782157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782906+0000","last_change":"2023-10-16T09:44:54.782906+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782906+0000","last_undegraded":"2023-10-16T09:44:54.782906+0000","last_fullsized":"2023-10-16T09:44:54.782906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788664+0000","last_change":"2023-10-16T09:44:54.788664+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788664+0000","last_undegraded":"2023-10-16T09:44:54.788664+0000","last_fullsized":"2023-10-16T09:44:54.788664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.785238+0000","last_change":"2023-10-16T09:44:54.785238+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.785238+0000","last_undegraded":"2023-10-16T09:44:54.785238+0000","last_fullsized":"2023-10-16T09:44:54.785238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782961+0000","last_change":"2023-10-16T09:44:54.782961+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782961+0000","last_undegraded":"2023-10-16T09:44:54.782961+0000","last_fullsized":"2023-10-16T09:44:54.782961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.783462+0000","last_change":"2023-10-16T09:44:54.783462+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.783462+0000","last_undegraded":"2023-10-16T09:44:54.783462+0000","last_fullsized":"2023-10-16T09:44:54.783462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.785279+0000","last_change":"2023-10-16T09:44:54.785279+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.785279+0000","last_undegraded":"2023-10-16T09:44:54.785279+0000","last_fullsized":"2023-10-16T09:44:54.785279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788799+0000","last_change":"2023-10-16T09:44:54.788799+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788799+0000","last_undegraded":"2023-10-16T09:44:54.788799+0000","last_fullsized":"2023-10-16T09:44:54.788799+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788771+0000","last_change":"2023-10-16T09:44:54.788771+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788771+0000","last_undegraded":"2023-10-16T09:44:54.788771+0000","last_fullsized":"2023-10-16T09:44:54.788771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788636+0000","last_change":"2023-10-16T09:44:54.788636+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788636+0000","last_undegraded":"2023-10-16T09:44:54.788636+0000","last_fullsized":"2023-10-16T09:44:54.788636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.789408+0000","last_change":"2023-10-16T09:44:54.789408+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.789408+0000","last_undegraded":"2023-10-16T09:44:54.789408+0000","last_fullsized":"2023-10-16T09:44:54.789408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782157+0000","last_change":"2023-10-16T09:44:54.782157+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782157+0000","last_undegraded":"2023-10-16T09:44:54.782157+0000","last_fullsized":"2023-10-16T09:44:54.782157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784847+0000","last_change":"2023-10-16T09:44:54.784847+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784847+0000","last_undegraded":"2023-10-16T09:44:54.784847+0000","last_fullsized":"2023-10-16T09:44:54.784847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784904+0000","last_change":"2023-10-16T09:44:54.784904+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784904+0000","last_undegraded":"2023-10-16T09:44:54.784904+0000","last_fullsized":"2023-10-16T09:44:54.784904+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.784819+0000","last_change":"2023-10-16T09:44:54.784819+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.784819+0000","last_undegraded":"2023-10-16T09:44:54.784819+0000","last_fullsized":"2023-10-16T09:44:54.784819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.788613+0000","last_change":"2023-10-16T09:44:54.788613+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.788613+0000","last_undegraded":"2023-10-16T09:44:54.788613+0000","last_fullsized":"2023-10-16T09:44:54.788613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-10-16T09:44:54.782970+0000","last_change":"2023-10-16T09:44:54.782970+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.782970+0000","last_undegraded":"2023-10-16T09:44:54.782970+0000","last_fullsized":"2023-10-16T09:44:54.782970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:55.779326+0000","last_change":"2023-10-16T09:44:55.779326+0000","last_active":"2023-10-16T09:44:55.779326+0000","last_peered":"2023-10-16T09:44:55.779326+0000","last_clean":"2023-10-16T09:44:55.779326+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:55.779326+0000","last_undegraded":"2023-10-16T09:44:55.779326+0000","last_fullsized":"2023-10-16T09:44:55.779326+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T09:44:54.777401+0000","last_change":"2023-10-16T09:44:54.777401+0000","last_active":"2023-10-16T09:44:54.777401+0000","last_peered":"2023-10-16T09:44:54.777401+0000","last_clean":"2023-10-16T09:44:54.777401+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T09:44:54.777401+0000","last_undegraded":"2023-10-16T09:44:54.777401+0000","last_fullsized":"2023-10-16T09:44:54.777401+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":173,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-16T09:44:49.920853+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:49.920853+0000","last_peered":"2023-10-16T09:44:49.920853+0000","last_clean":"2023-10-16T09:44:49.920853+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:49.920853+0000","last_undegraded":"2023-10-16T09:44:49.920853+0000","last_fullsized":"2023-10-16T09:44:49.920853+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":46,"acting":46,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":925696,"data_stored":712580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":925696,"data_stored":712580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":331776,"data_stored":122212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:45:01.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-10-16T09:45:01.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-10-16T09:45:01.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-10-16T09:45:01.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-10-16T09:45:01.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-10-16T09:45:01.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-10-16T09:45:01.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-10-16T09:45:01.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-10-16T09:45:01.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-10-16T09:45:01.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-10-16T09:45:01.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-10-16T09:45:01.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-10-16T09:45:01.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-10-16T09:45:01.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-10-16T09:45:01.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-10-16T09:45:01.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-10-16T09:45:01.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-10-16T09:45:01.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-10-16T09:45:01.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-10-16T09:45:01.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-10-16T09:45:01.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-10-16T09:45:01.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-10-16T09:45:01.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-10-16T09:45:01.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-10-16T09:45:01.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-10-16T09:45:01.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-10-16T09:45:01.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-10-16T09:45:01.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-10-16T09:45:01.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-10-16T09:45:01.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-10-16T09:45:01.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-10-16T09:45:01.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-10-16T09:45:01.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-10-16T09:45:01.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-10-16T09:45:01.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-10-16T09:45:01.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-10-16T09:45:01.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-10-16T09:45:01.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-10-16T09:45:01.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.783396+0000', 'last_change': '2023-10-16T09:44:54.783396+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.783396+0000', 'last_undegraded': '2023-10-16T09:44:54.783396+0000', 'last_fullsized': '2023-10-16T09:44:54.783396+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T09:45:01.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-10-16T09:45:01.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-10-16T09:45:01.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-10-16T09:45:01.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-10-16T09:45:01.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.789240+0000', 'last_change': '2023-10-16T09:44:54.789240+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.789240+0000', 'last_undegraded': '2023-10-16T09:44:54.789240+0000', 'last_fullsized': '2023-10-16T09:44:54.789240+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-10-16T09:45:01.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-10-16T09:45:01.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.788635+0000', 'last_change': '2023-10-16T09:44:54.788635+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.788635+0000', 'last_undegraded': '2023-10-16T09:44:54.788635+0000', 'last_fullsized': '2023-10-16T09:44:54.788635+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-10-16T09:45:01.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-10-16T09:45:01.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.783564+0000', 'last_change': '2023-10-16T09:44:54.783564+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.783564+0000', 'last_undegraded': '2023-10-16T09:44:54.783564+0000', 'last_fullsized': '2023-10-16T09:44:54.783564+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T09:45:01.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-10-16T09:45:01.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-10-16T09:45:01.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.788637+0000', 'last_change': '2023-10-16T09:44:54.788637+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.788637+0000', 'last_undegraded': '2023-10-16T09:44:54.788637+0000', 'last_fullsized': '2023-10-16T09:44:54.788637+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-10-16T09:45:01.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-10-16T09:45:01.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.782624+0000', 'last_change': '2023-10-16T09:44:54.782624+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.782624+0000', 'last_undegraded': '2023-10-16T09:44:54.782624+0000', 'last_fullsized': '2023-10-16T09:44:54.782624+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T09:45:01.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-10-16T09:45:01.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-10-16T09:45:01.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-10-16T09:45:01.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-10-16T09:45:01.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.782157+0000', 'last_change': '2023-10-16T09:44:54.782157+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.782157+0000', 'last_undegraded': '2023-10-16T09:44:54.782157+0000', 'last_fullsized': '2023-10-16T09:44:54.782157+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-16T09:45:01.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-10-16T09:45:01.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-10-16T09:45:01.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.782906+0000', 'last_change': '2023-10-16T09:44:54.782906+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.782906+0000', 'last_undegraded': '2023-10-16T09:44:54.782906+0000', 'last_fullsized': '2023-10-16T09:44:54.782906+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T09:45:01.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-10-16T09:45:01.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-10-16T09:45:01.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.788664+0000', 'last_change': '2023-10-16T09:44:54.788664+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.788664+0000', 'last_undegraded': '2023-10-16T09:44:54.788664+0000', 'last_fullsized': '2023-10-16T09:44:54.788664+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-10-16T09:45:01.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-10-16T09:45:01.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.785238+0000', 'last_change': '2023-10-16T09:44:54.785238+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.785238+0000', 'last_undegraded': '2023-10-16T09:44:54.785238+0000', 'last_fullsized': '2023-10-16T09:44:54.785238+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T09:45:01.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-10-16T09:45:01.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-10-16T09:45:01.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.782961+0000', 'last_change': '2023-10-16T09:44:54.782961+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.782961+0000', 'last_undegraded': '2023-10-16T09:44:54.782961+0000', 'last_fullsized': '2023-10-16T09:44:54.782961+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T09:45:01.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-10-16T09:45:01.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-10-16T09:45:01.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.783462+0000', 'last_change': '2023-10-16T09:44:54.783462+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.783462+0000', 'last_undegraded': '2023-10-16T09:44:54.783462+0000', 'last_fullsized': '2023-10-16T09:44:54.783462+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T09:45:01.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-10-16T09:45:01.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-10-16T09:45:01.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.785279+0000', 'last_change': '2023-10-16T09:44:54.785279+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.785279+0000', 'last_undegraded': '2023-10-16T09:44:54.785279+0000', 'last_fullsized': '2023-10-16T09:44:54.785279+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T09:45:01.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-10-16T09:45:01.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-10-16T09:45:01.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.788799+0000', 'last_change': '2023-10-16T09:44:54.788799+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.788799+0000', 'last_undegraded': '2023-10-16T09:44:54.788799+0000', 'last_fullsized': '2023-10-16T09:44:54.788799+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-10-16T09:45:01.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-10-16T09:45:01.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.788771+0000', 'last_change': '2023-10-16T09:44:54.788771+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.788771+0000', 'last_undegraded': '2023-10-16T09:44:54.788771+0000', 'last_fullsized': '2023-10-16T09:44:54.788771+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-10-16T09:45:01.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-10-16T09:45:01.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.788636+0000', 'last_change': '2023-10-16T09:44:54.788636+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.788636+0000', 'last_undegraded': '2023-10-16T09:44:54.788636+0000', 'last_fullsized': '2023-10-16T09:44:54.788636+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-10-16T09:45:01.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-10-16T09:45:01.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-10-16T09:45:01.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-10-16T09:45:01.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-10-16T09:45:01.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-10-16T09:45:01.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.789408+0000', 'last_change': '2023-10-16T09:44:54.789408+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.789408+0000', 'last_undegraded': '2023-10-16T09:44:54.789408+0000', 'last_fullsized': '2023-10-16T09:44:54.789408+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-10-16T09:45:01.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-10-16T09:45:01.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.782157+0000', 'last_change': '2023-10-16T09:44:54.782157+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.782157+0000', 'last_undegraded': '2023-10-16T09:44:54.782157+0000', 'last_fullsized': '2023-10-16T09:44:54.782157+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-16T09:45:01.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-10-16T09:45:01.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-10-16T09:45:01.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.784847+0000', 'last_change': '2023-10-16T09:44:54.784847+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.784847+0000', 'last_undegraded': '2023-10-16T09:44:54.784847+0000', 'last_fullsized': '2023-10-16T09:44:54.784847+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T09:45:01.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-10-16T09:45:01.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-10-16T09:45:01.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-10-16T09:45:01.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-10-16T09:45:01.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.784904+0000', 'last_change': '2023-10-16T09:44:54.784904+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.784904+0000', 'last_undegraded': '2023-10-16T09:44:54.784904+0000', 'last_fullsized': '2023-10-16T09:44:54.784904+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T09:45:01.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-10-16T09:45:01.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-10-16T09:45:01.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-10-16T09:45:01.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-10-16T09:45:01.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.784819+0000', 'last_change': '2023-10-16T09:44:54.784819+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.784819+0000', 'last_undegraded': '2023-10-16T09:44:54.784819+0000', 'last_fullsized': '2023-10-16T09:44:54.784819+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-16T09:45:01.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-10-16T09:45:01.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-10-16T09:45:01.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.788613+0000', 'last_change': '2023-10-16T09:44:54.788613+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.788613+0000', 'last_undegraded': '2023-10-16T09:44:54.788613+0000', 'last_fullsized': '2023-10-16T09:44:54.788613+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-16T09:45:01.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-10-16T09:45:01.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-10-16T09:45:01.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-10-16T09:44:54.782970+0000', 'last_change': '2023-10-16T09:44:54.782970+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.782970+0000', 'last_undegraded': '2023-10-16T09:44:54.782970+0000', 'last_fullsized': '2023-10-16T09:44:54.782970+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-16T09:45:01.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-10-16T09:45:01.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:55.779326+0000', 'last_change': '2023-10-16T09:44:55.779326+0000', 'last_active': '2023-10-16T09:44:55.779326+0000', 'last_peered': '2023-10-16T09:44:55.779326+0000', 'last_clean': '2023-10-16T09:44:55.779326+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:55.779326+0000', 'last_undegraded': '2023-10-16T09:44:55.779326+0000', 'last_fullsized': '2023-10-16T09:44:55.779326+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:55.779326+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-10-16T09:45:01.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-16T09:44:54.777401+0000', 'last_change': '2023-10-16T09:44:54.777401+0000', 'last_active': '2023-10-16T09:44:54.777401+0000', 'last_peered': '2023-10-16T09:44:54.777401+0000', 'last_clean': '2023-10-16T09:44:54.777401+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T09:44:54.777401+0000', 'last_undegraded': '2023-10-16T09:44:54.777401+0000', 'last_fullsized': '2023-10-16T09:44:54.777401+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'last_clean_scrub_stamp': '2023-10-16T09:44:54.777401+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-16T09:45:01.752 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-16T09:45:01.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.871+0000 7f9b97ac3700 1 Processor -- start 2023-10-16T09:45:01.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.871+0000 7f9b97ac3700 1 -- start start 2023-10-16T09:45:01.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b97ac3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b9015b990 0x7f9b900ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b97ac3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b900d0370 0x7f9b900d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b97ac3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b900cc510 con 0x7f9b900cd2d0 2023-10-16T09:45:01.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b97ac3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b900d24f0 con 0x7f9b9015b990 2023-10-16T09:45:01.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b97ac3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b900cc7f0 con 0x7f9b900d0370 2023-10-16T09:45:01.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b9585f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b9015b990 0x7f9b900ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b9585f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b9015b990 0x7f9b900ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38038/0 (socket says 172.21.15.181:38038) 2023-10-16T09:45:01.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b9585f700 1 -- 172.21.15.181:0/3767417616 learned_addr learned my addr 172.21.15.181:0/3767417616 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:01.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b96060700 1 --2- 172.21.15.181:0/3767417616 >> v2:172.21.15.190:3300/0 conn(0x7f9b900d0370 0x7f9b900d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b9585f700 1 -- 172.21.15.181:0/3767417616 >> v2:172.21.15.190:3300/0 conn(0x7f9b900d0370 msgr2=0x7f9b900d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:01.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b9585f700 1 --2- 172.21.15.181:0/3767417616 >> v2:172.21.15.190:3300/0 conn(0x7f9b900d0370 0x7f9b900d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b9585f700 1 -- 172.21.15.181:0/3767417616 >> v1:172.21.15.190:6789/0 conn(0x7f9b900cd2d0 legacy=0x7f9b900cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:01.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b9585f700 1 -- 172.21.15.181:0/3767417616 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900ccad0 con 0x7f9b9015b990 2023-10-16T09:45:01.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.872+0000 7f9b96060700 1 --2- 172.21.15.181:0/3767417616 >> v2:172.21.15.190:3300/0 conn(0x7f9b900d0370 0x7f9b900d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:45:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b9585f700 1 --2- 172.21.15.181:0/3767417616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900ce8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9b9009ae40 tx=0x7f9b8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2ba4212f9af3b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b9485d700 1 -- 172.21.15.181:0/3767417616 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c005e30 con 0x7f9b9015b990 2023-10-16T09:45:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b9485d700 1 -- 172.21.15.181:0/3767417616 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c006030 con 0x7f9b9015b990 2023-10-16T09:45:01.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b9485d700 1 -- 172.21.15.181:0/3767417616 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0063e0 con 0x7f9b9015b990 2023-10-16T09:45:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 -- 172.21.15.181:0/3767417616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 msgr2=0x7f9b900ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/3767417616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900ce8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9b9009ae40 tx=0x7f9b8c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 -- 172.21.15.181:0/3767417616 shutdown_connections 2023-10-16T09:45:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/3767417616 >> v2:172.21.15.190:3300/0 conn(0x7f9b900d0370 0x7f9b900d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/3767417616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900ce8f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 -- 172.21.15.181:0/3767417616 >> 172.21.15.181:0/3767417616 conn(0x7f9b9009a9a0 msgr2=0x7f9b900d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 -- 172.21.15.181:0/3767417616 shutdown_connections 2023-10-16T09:45:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.873+0000 7f9b97ac3700 1 -- 172.21.15.181:0/3767417616 wait complete. 2023-10-16T09:45:01.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.874+0000 7f9b97ac3700 1 Processor -- start 2023-10-16T09:45:01.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b97ac3700 1 -- start start 2023-10-16T09:45:01.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b97ac3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b900cd2d0 0x7f9b9014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b97ac3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b900d0370 0x7f9b9006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b97ac3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b97ac3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900b5380 con 0x7f9b9015b990 2023-10-16T09:45:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b97ac3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900b4a50 con 0x7f9b900cd2d0 2023-10-16T09:45:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b97ac3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900b4d30 con 0x7f9b900d0370 2023-10-16T09:45:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b96060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b96060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38046/0 (socket says 172.21.15.181:38046) 2023-10-16T09:45:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b96060700 1 -- 172.21.15.181:0/674541801 learned_addr learned my addr 172.21.15.181:0/674541801 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b9585f700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b900cd2d0 0x7f9b9014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.875+0000 7f9b9505e700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b900d0370 0x7f9b9006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b96060700 1 -- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b900d0370 msgr2=0x7f9b9006fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b96060700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b900d0370 0x7f9b9006fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b96060700 1 -- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b900cd2d0 msgr2=0x7f9b9014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:01.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b96060700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b900cd2d0 0x7f9b9014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:01.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b96060700 1 -- 172.21.15.181:0/674541801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c009500 con 0x7f9b9015b990 2023-10-16T09:45:01.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b96060700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900c8750 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800ce80 tx=0x7f9b8800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b867fc700 1 -- 172.21.15.181:0/674541801 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800ad40 con 0x7f9b9015b990 2023-10-16T09:45:01.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b867fc700 1 -- 172.21.15.181:0/674541801 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88015440 con 0x7f9b9015b990 2023-10-16T09:45:01.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b900b5010 con 0x7f9b9015b990 2023-10-16T09:45:01.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b867fc700 1 -- 172.21.15.181:0/674541801 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800aea0 con 0x7f9b9015b990 2023-10-16T09:45:01.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.876+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9006eef0 con 0x7f9b9015b990 2023-10-16T09:45:01.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.877+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b900b01c0 con 0x7f9b9015b990 2023-10-16T09:45:01.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.877+0000 7f9b867fc700 1 -- 172.21.15.181:0/674541801 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b8800d430 con 0x7f9b9015b990 2023-10-16T09:45:01.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.878+0000 7f9b867fc700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9b7406af00 0x7f9b7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:01.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.878+0000 7f9b9585f700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9b7406af00 0x7f9b7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:01.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.878+0000 7f9b867fc700 1 -- 172.21.15.181:0/674541801 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b88089a30 con 0x7f9b9015b990 2023-10-16T09:45:01.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.878+0000 7f9b9585f700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9b7406af00 0x7f9b7406d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b9009ae40 tx=0x7f9b8c008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:01.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:01.882+0000 7f9b867fc700 1 -- 172.21.15.181:0/674541801 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9b88054ae0 con 0x7f9b9015b990 2023-10-16T09:45:02.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.153+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9b90067d20 con 0x7f9b9015b990 2023-10-16T09:45:02.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.153+0000 7f9b867fc700 1 -- 172.21.15.181:0/674541801 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1559 (secure 0 0 0) 0x7f9b8801a080 con 0x7f9b9015b990 2023-10-16T09:45:02.155 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:02.155 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":78,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1697449434,"num_in_osds":8,"osd_in_since":1697449426,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":225673216,"bytes_avail":772868440064,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5049}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-16T09:44:55.689045+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-16T09:45:02.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.155+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9b7406af00 msgr2=0x7f9b7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.155+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9b7406af00 0x7f9b7406d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b9009ae40 tx=0x7f9b8c008070 comp rx=0 tx=0).stop 2023-10-16T09:45:02.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.155+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 msgr2=0x7f9b900c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.155+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900c8750 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800ce80 tx=0x7f9b8800ceb0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 shutdown_connections 2023-10-16T09:45:02.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9b7406af00 0x7f9b7406d3c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b9015b990 0x7f9b900c8750 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b900cd2d0 0x7f9b9014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 --2- 172.21.15.181:0/674541801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b900d0370 0x7f9b9006fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 >> 172.21.15.181:0/674541801 conn(0x7f9b9009a9a0 msgr2=0x7f9b9015abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:02.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 shutdown_connections 2023-10-16T09:45:02.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.156+0000 7f9b97ac3700 1 -- 172.21.15.181:0/674541801 wait complete. 2023-10-16T09:45:02.169 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-16T09:45:02.169 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:45:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.289+0000 7f6ea3139700 1 Processor -- start 2023-10-16T09:45:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.289+0000 7f6ea3139700 1 -- start start 2023-10-16T09:45:02.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea3139700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e9c15dfd0 0x7f6e9c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea3139700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c06f870 con 0x7f6e9c0cd470 2023-10-16T09:45:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea3139700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c0d1160 con 0x7f6e9c0af450 2023-10-16T09:45:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea3139700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c0cc8e0 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea0ed5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e9c15dfd0 0x7f6e9c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea16d6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6e9c0cd470 0x7f6e9c06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49132/0 (socket says 172.21.15.181:49132) 2023-10-16T09:45:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea16d6700 1 -- 172.21.15.181:0/3009973035 learned_addr learned my addr 172.21.15.181:0/3009973035 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea0ed5700 1 -- 172.21.15.181:0/3009973035 >> v1:172.21.15.190:6789/0 conn(0x7f6e9c0af450 legacy=0x7f6e9c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea0ed5700 1 -- 172.21.15.181:0/3009973035 >> v1:172.21.15.181:6789/0 conn(0x7f6e9c0cd470 legacy=0x7f6e9c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea0ed5700 1 -- 172.21.15.181:0/3009973035 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0ccbc0 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6e9b7fe700 1 -- 172.21.15.181:0/3009973035 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213892778 0 0) 0x7f6e9c06f870 con 0x7f6e9c0cd470 2023-10-16T09:45:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6ea0ed5700 1 --2- 172.21.15.181:0/3009973035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0af9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009820 tx=0x7f6e90009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de593fa9f1697e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.290+0000 7f6e9b7fe700 1 -- 172.21.15.181:0/3009973035 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90005e30 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6e9b7fe700 1 -- 172.21.15.181:0/3009973035 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e90004030 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6e9b7fe700 1 -- 172.21.15.181:0/3009973035 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900043c0 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6ea3139700 1 -- 172.21.15.181:0/3009973035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 msgr2=0x7f6e9c0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6ea3139700 1 --2- 172.21.15.181:0/3009973035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0af9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009820 tx=0x7f6e90009b30 comp rx=0 tx=0).stop 2023-10-16T09:45:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6ea3139700 1 -- 172.21.15.181:0/3009973035 shutdown_connections 2023-10-16T09:45:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6ea3139700 1 --2- 172.21.15.181:0/3009973035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0af9a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6ea3139700 1 -- 172.21.15.181:0/3009973035 >> 172.21.15.181:0/3009973035 conn(0x7f6e9c0d04a0 msgr2=0x7f6e9c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6ea3139700 1 -- 172.21.15.181:0/3009973035 shutdown_connections 2023-10-16T09:45:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.291+0000 7f6ea3139700 1 -- 172.21.15.181:0/3009973035 wait complete. 2023-10-16T09:45:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.292+0000 7f6ea3139700 1 Processor -- start 2023-10-16T09:45:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.292+0000 7f6ea3139700 1 -- start start 2023-10-16T09:45:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea3139700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9c0af450 0x7f6e9c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea3139700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e9c0cd470 0x7f6e9c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea3139700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea3139700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15d350 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea3139700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0b46e0 con 0x7f6e9c0af450 2023-10-16T09:45:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea3139700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0b49c0 con 0x7f6e9c0cd470 2023-10-16T09:45:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea16d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea16d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38060/0 (socket says 172.21.15.181:38060) 2023-10-16T09:45:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea16d6700 1 -- 172.21.15.181:0/1235784919 learned_addr learned my addr 172.21.15.181:0/1235784919 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6ea0ed5700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9c0af450 0x7f6e9c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.293+0000 7f6e9bfff700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e9c0cd470 0x7f6e9c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea16d6700 1 -- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e9c0cd470 msgr2=0x7f6e9c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea16d6700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e9c0cd470 0x7f6e9c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea16d6700 1 -- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9c0af450 msgr2=0x7f6e9c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea16d6700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9c0af450 0x7f6e9c157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea16d6700 1 -- 172.21.15.181:0/1235784919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e90009430 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea16d6700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0ce3a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400cca0 tx=0x7f6e9400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6e997fa700 1 -- 172.21.15.181:0/1235784919 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400ab70 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6e997fa700 1 -- 172.21.15.181:0/1235784919 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9400acd0 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c0b4ca0 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6e997fa700 1 -- 172.21.15.181:0/1235784919 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94007310 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.294+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c0b5260 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.295+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9c0b5480 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.296+0000 7f6e997fa700 1 -- 172.21.15.181:0/1235784919 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e94007470 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.296+0000 7f6e997fa700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e8006af00 0x7f6e8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.296+0000 7f6e997fa700 1 -- 172.21.15.181:0/1235784919 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e94089680 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.296+0000 7f6ea0ed5700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e8006af00 0x7f6e8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:02.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.296+0000 7f6ea0ed5700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e8006af00 0x7f6e8006d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e90000c00 tx=0x7f6e90008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:02.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.300+0000 7f6e997fa700 1 -- 172.21.15.181:0/1235784919 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6e94054810 con 0x7f6e9c15dfd0 2023-10-16T09:45:02.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.488+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e9c0cc590 con 0x7f6e8006af00 2023-10-16T09:45:02.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.496+0000 7f6e997fa700 1 -- 172.21.15.181:0/1235784919 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289452 (secure 0 0 0) 0x7f6e90021070 con 0x7f6e8006af00 2023-10-16T09:45:02.500 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:02.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e8006af00 msgr2=0x7f6e8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e8006af00 0x7f6e8006d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e90000c00 tx=0x7f6e90008070 comp rx=0 tx=0).stop 2023-10-16T09:45:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 msgr2=0x7f6e9c0ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0ce3a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400cca0 tx=0x7f6e9400a060 comp rx=0 tx=0).stop 2023-10-16T09:45:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 shutdown_connections 2023-10-16T09:45:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6e8006af00 0x7f6e8006d3c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e9c15dfd0 0x7f6e9c0ce3a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e9c0af450 0x7f6e9c157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 --2- 172.21.15.181:0/1235784919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e9c0cd470 0x7f6e9c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 >> 172.21.15.181:0/1235784919 conn(0x7f6e9c0d04a0 msgr2=0x7f6e9c15a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 shutdown_connections 2023-10-16T09:45:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:02.500+0000 7f6ea3139700 1 -- 172.21.15.181:0/1235784919 wait complete. 2023-10-16T09:45:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:45:02.513 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-10-16T09:45:01.690092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220384,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754336,"statfs":{"total":773094113280,"available":772868440064,"internally_reserved":0,"allocated":4300800,"data_stored":2364524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001178"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812734+0000","last_change":"2023-10-16T09:44:56.802788+0000","last_active":"2023-10-16T09:44:57.812734+0000","last_peered":"2023-10-16T09:44:57.812734+0000","last_clean":"2023-10-16T09:44:57.812734+0000","last_became_active":"2023-10-16T09:44:56.801293+0000","last_became_peered":"2023-10-16T09:44:56.801293+0000","last_unstale":"2023-10-16T09:44:57.812734+0000","last_undegraded":"2023-10-16T09:44:57.812734+0000","last_fullsized":"2023-10-16T09:44:57.812734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:01.325830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789003+0000","last_change":"2023-10-16T09:44:56.816768+0000","last_active":"2023-10-16T09:44:57.789003+0000","last_peered":"2023-10-16T09:44:57.789003+0000","last_clean":"2023-10-16T09:44:57.789003+0000","last_became_active":"2023-10-16T09:44:56.814617+0000","last_became_peered":"2023-10-16T09:44:56.814617+0000","last_unstale":"2023-10-16T09:44:57.789003+0000","last_undegraded":"2023-10-16T09:44:57.789003+0000","last_fullsized":"2023-10-16T09:44:57.789003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:51:59.527316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790065+0000","last_change":"2023-10-16T09:44:56.798977+0000","last_active":"2023-10-16T09:44:57.790065+0000","last_peered":"2023-10-16T09:44:57.790065+0000","last_clean":"2023-10-16T09:44:57.790065+0000","last_became_active":"2023-10-16T09:44:56.798301+0000","last_became_peered":"2023-10-16T09:44:56.798301+0000","last_unstale":"2023-10-16T09:44:57.790065+0000","last_undegraded":"2023-10-16T09:44:57.790065+0000","last_fullsized":"2023-10-16T09:44:57.790065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:34.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381306+0000","last_change":"2023-10-16T09:44:56.807900+0000","last_active":"2023-10-16T09:44:58.381306+0000","last_peered":"2023-10-16T09:44:58.381306+0000","last_clean":"2023-10-16T09:44:58.381306+0000","last_became_active":"2023-10-16T09:44:56.807244+0000","last_became_peered":"2023-10-16T09:44:56.807244+0000","last_unstale":"2023-10-16T09:44:58.381306+0000","last_undegraded":"2023-10-16T09:44:58.381306+0000","last_fullsized":"2023-10-16T09:44:58.381306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:43:47.651626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789871+0000","last_change":"2023-10-16T09:44:56.810090+0000","last_active":"2023-10-16T09:44:57.789871+0000","last_peered":"2023-10-16T09:44:57.789871+0000","last_clean":"2023-10-16T09:44:57.789871+0000","last_became_active":"2023-10-16T09:44:56.809820+0000","last_became_peered":"2023-10-16T09:44:56.809820+0000","last_unstale":"2023-10-16T09:44:57.789871+0000","last_undegraded":"2023-10-16T09:44:57.789871+0000","last_fullsized":"2023-10-16T09:44:57.789871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:27.237857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788580+0000","last_change":"2023-10-16T09:44:56.814396+0000","last_active":"2023-10-16T09:44:57.788580+0000","last_peered":"2023-10-16T09:44:57.788580+0000","last_clean":"2023-10-16T09:44:57.788580+0000","last_became_active":"2023-10-16T09:44:56.814106+0000","last_became_peered":"2023-10-16T09:44:56.814106+0000","last_unstale":"2023-10-16T09:44:57.788580+0000","last_undegraded":"2023-10-16T09:44:57.788580+0000","last_fullsized":"2023-10-16T09:44:57.788580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:38:27.019733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722536+0000","last_change":"2023-10-16T09:44:56.810408+0000","last_active":"2023-10-16T09:44:58.722536+0000","last_peered":"2023-10-16T09:44:58.722536+0000","last_clean":"2023-10-16T09:44:58.722536+0000","last_became_active":"2023-10-16T09:44:56.810068+0000","last_became_peered":"2023-10-16T09:44:56.810068+0000","last_unstale":"2023-10-16T09:44:58.722536+0000","last_undegraded":"2023-10-16T09:44:58.722536+0000","last_fullsized":"2023-10-16T09:44:58.722536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:09.120120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722082+0000","last_change":"2023-10-16T09:44:56.807818+0000","last_active":"2023-10-16T09:44:58.722082+0000","last_peered":"2023-10-16T09:44:58.722082+0000","last_clean":"2023-10-16T09:44:58.722082+0000","last_became_active":"2023-10-16T09:44:56.807435+0000","last_became_peered":"2023-10-16T09:44:56.807435+0000","last_unstale":"2023-10-16T09:44:58.722082+0000","last_undegraded":"2023-10-16T09:44:58.722082+0000","last_fullsized":"2023-10-16T09:44:58.722082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:17.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189514+0000","last_change":"2023-10-16T09:44:56.803007+0000","last_active":"2023-10-16T09:44:58.189514+0000","last_peered":"2023-10-16T09:44:58.189514+0000","last_clean":"2023-10-16T09:44:58.189514+0000","last_became_active":"2023-10-16T09:44:56.802499+0000","last_became_peered":"2023-10-16T09:44:56.802499+0000","last_unstale":"2023-10-16T09:44:58.189514+0000","last_undegraded":"2023-10-16T09:44:58.189514+0000","last_fullsized":"2023-10-16T09:44:58.189514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:19.270890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381558+0000","last_change":"2023-10-16T09:44:56.818068+0000","last_active":"2023-10-16T09:44:58.381558+0000","last_peered":"2023-10-16T09:44:58.381558+0000","last_clean":"2023-10-16T09:44:58.381558+0000","last_became_active":"2023-10-16T09:44:56.817799+0000","last_became_peered":"2023-10-16T09:44:56.817799+0000","last_unstale":"2023-10-16T09:44:58.381558+0000","last_undegraded":"2023-10-16T09:44:58.381558+0000","last_fullsized":"2023-10-16T09:44:58.381558+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:17.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722465+0000","last_change":"2023-10-16T09:44:56.803456+0000","last_active":"2023-10-16T09:44:58.722465+0000","last_peered":"2023-10-16T09:44:58.722465+0000","last_clean":"2023-10-16T09:44:58.722465+0000","last_became_active":"2023-10-16T09:44:56.803140+0000","last_became_peered":"2023-10-16T09:44:56.803140+0000","last_unstale":"2023-10-16T09:44:58.722465+0000","last_undegraded":"2023-10-16T09:44:58.722465+0000","last_fullsized":"2023-10-16T09:44:58.722465+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:14.058492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789841+0000","last_change":"2023-10-16T09:44:56.818281+0000","last_active":"2023-10-16T09:44:57.789841+0000","last_peered":"2023-10-16T09:44:57.789841+0000","last_clean":"2023-10-16T09:44:57.789841+0000","last_became_active":"2023-10-16T09:44:56.817903+0000","last_became_peered":"2023-10-16T09:44:56.817903+0000","last_unstale":"2023-10-16T09:44:57.789841+0000","last_undegraded":"2023-10-16T09:44:57.789841+0000","last_fullsized":"2023-10-16T09:44:57.789841+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:57.785495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789780+0000","last_change":"2023-10-16T09:44:56.806961+0000","last_active":"2023-10-16T09:44:57.789780+0000","last_peered":"2023-10-16T09:44:57.789780+0000","last_clean":"2023-10-16T09:44:57.789780+0000","last_became_active":"2023-10-16T09:44:56.806699+0000","last_became_peered":"2023-10-16T09:44:56.806699+0000","last_unstale":"2023-10-16T09:44:57.789780+0000","last_undegraded":"2023-10-16T09:44:57.789780+0000","last_fullsized":"2023-10-16T09:44:57.789780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:12:28.672873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788683+0000","last_change":"2023-10-16T09:44:56.814342+0000","last_active":"2023-10-16T09:44:57.788683+0000","last_peered":"2023-10-16T09:44:57.788683+0000","last_clean":"2023-10-16T09:44:57.788683+0000","last_became_active":"2023-10-16T09:44:56.813974+0000","last_became_peered":"2023-10-16T09:44:56.813974+0000","last_unstale":"2023-10-16T09:44:57.788683+0000","last_undegraded":"2023-10-16T09:44:57.788683+0000","last_fullsized":"2023-10-16T09:44:57.788683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:32.192715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789661+0000","last_change":"2023-10-16T09:44:56.804490+0000","last_active":"2023-10-16T09:44:57.789661+0000","last_peered":"2023-10-16T09:44:57.789661+0000","last_clean":"2023-10-16T09:44:57.789661+0000","last_became_active":"2023-10-16T09:44:56.804185+0000","last_became_peered":"2023-10-16T09:44:56.804185+0000","last_unstale":"2023-10-16T09:44:57.789661+0000","last_undegraded":"2023-10-16T09:44:57.789661+0000","last_fullsized":"2023-10-16T09:44:57.789661+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:14.058492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722557+0000","last_change":"2023-10-16T09:44:56.817373+0000","last_active":"2023-10-16T09:44:58.722557+0000","last_peered":"2023-10-16T09:44:58.722557+0000","last_clean":"2023-10-16T09:44:58.722557+0000","last_became_active":"2023-10-16T09:44:56.817085+0000","last_became_peered":"2023-10-16T09:44:56.817085+0000","last_unstale":"2023-10-16T09:44:58.722557+0000","last_undegraded":"2023-10-16T09:44:58.722557+0000","last_fullsized":"2023-10-16T09:44:58.722557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:21.710655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722658+0000","last_change":"2023-10-16T09:44:56.805810+0000","last_active":"2023-10-16T09:44:58.722658+0000","last_peered":"2023-10-16T09:44:58.722658+0000","last_clean":"2023-10-16T09:44:58.722658+0000","last_became_active":"2023-10-16T09:44:56.805492+0000","last_became_peered":"2023-10-16T09:44:56.805492+0000","last_unstale":"2023-10-16T09:44:58.722658+0000","last_undegraded":"2023-10-16T09:44:58.722658+0000","last_fullsized":"2023-10-16T09:44:58.722658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:42.010506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789140+0000","last_change":"2023-10-16T09:44:56.816595+0000","last_active":"2023-10-16T09:44:57.789140+0000","last_peered":"2023-10-16T09:44:57.789140+0000","last_clean":"2023-10-16T09:44:57.789140+0000","last_became_active":"2023-10-16T09:44:56.814296+0000","last_became_peered":"2023-10-16T09:44:56.814296+0000","last_unstale":"2023-10-16T09:44:57.789140+0000","last_undegraded":"2023-10-16T09:44:57.789140+0000","last_fullsized":"2023-10-16T09:44:57.789140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:38.501059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788719+0000","last_change":"2023-10-16T09:44:56.811146+0000","last_active":"2023-10-16T09:44:57.788719+0000","last_peered":"2023-10-16T09:44:57.788719+0000","last_clean":"2023-10-16T09:44:57.788719+0000","last_became_active":"2023-10-16T09:44:56.810894+0000","last_became_peered":"2023-10-16T09:44:56.810894+0000","last_unstale":"2023-10-16T09:44:57.788719+0000","last_undegraded":"2023-10-16T09:44:57.788719+0000","last_fullsized":"2023-10-16T09:44:57.788719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:07.823284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789867+0000","last_change":"2023-10-16T09:44:56.811436+0000","last_active":"2023-10-16T09:44:57.789867+0000","last_peered":"2023-10-16T09:44:57.789867+0000","last_clean":"2023-10-16T09:44:57.789867+0000","last_became_active":"2023-10-16T09:44:56.811154+0000","last_became_peered":"2023-10-16T09:44:56.811154+0000","last_unstale":"2023-10-16T09:44:57.789867+0000","last_undegraded":"2023-10-16T09:44:57.789867+0000","last_fullsized":"2023-10-16T09:44:57.789867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:09.746836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189112+0000","last_change":"2023-10-16T09:44:56.807191+0000","last_active":"2023-10-16T09:44:58.189112+0000","last_peered":"2023-10-16T09:44:58.189112+0000","last_clean":"2023-10-16T09:44:58.189112+0000","last_became_active":"2023-10-16T09:44:56.806875+0000","last_became_peered":"2023-10-16T09:44:56.806875+0000","last_unstale":"2023-10-16T09:44:58.189112+0000","last_undegraded":"2023-10-16T09:44:58.189112+0000","last_fullsized":"2023-10-16T09:44:58.189112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:20.173958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189082+0000","last_change":"2023-10-16T09:44:56.818182+0000","last_active":"2023-10-16T09:44:58.189082+0000","last_peered":"2023-10-16T09:44:58.189082+0000","last_clean":"2023-10-16T09:44:58.189082+0000","last_became_active":"2023-10-16T09:44:56.817890+0000","last_became_peered":"2023-10-16T09:44:56.817890+0000","last_unstale":"2023-10-16T09:44:58.189082+0000","last_undegraded":"2023-10-16T09:44:58.189082+0000","last_fullsized":"2023-10-16T09:44:58.189082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:29.923256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428765+0000","last_change":"2023-10-16T09:44:56.812353+0000","last_active":"2023-10-16T09:44:58.428765+0000","last_peered":"2023-10-16T09:44:58.428765+0000","last_clean":"2023-10-16T09:44:58.428765+0000","last_became_active":"2023-10-16T09:44:56.811994+0000","last_became_peered":"2023-10-16T09:44:56.811994+0000","last_unstale":"2023-10-16T09:44:58.428765+0000","last_undegraded":"2023-10-16T09:44:58.428765+0000","last_fullsized":"2023-10-16T09:44:58.428765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:25.271373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789470+0000","last_change":"2023-10-16T09:44:56.807324+0000","last_active":"2023-10-16T09:44:57.789470+0000","last_peered":"2023-10-16T09:44:57.789470+0000","last_clean":"2023-10-16T09:44:57.789470+0000","last_became_active":"2023-10-16T09:44:56.807041+0000","last_became_peered":"2023-10-16T09:44:56.807041+0000","last_unstale":"2023-10-16T09:44:57.789470+0000","last_undegraded":"2023-10-16T09:44:57.789470+0000","last_fullsized":"2023-10-16T09:44:57.789470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:02.475372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381620+0000","last_change":"2023-10-16T09:44:56.805136+0000","last_active":"2023-10-16T09:44:58.381620+0000","last_peered":"2023-10-16T09:44:58.381620+0000","last_clean":"2023-10-16T09:44:58.381620+0000","last_became_active":"2023-10-16T09:44:56.804879+0000","last_became_peered":"2023-10-16T09:44:56.804879+0000","last_unstale":"2023-10-16T09:44:58.381620+0000","last_undegraded":"2023-10-16T09:44:58.381620+0000","last_fullsized":"2023-10-16T09:44:58.381620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:32.410416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789146+0000","last_change":"2023-10-16T09:44:56.816589+0000","last_active":"2023-10-16T09:44:57.789146+0000","last_peered":"2023-10-16T09:44:57.789146+0000","last_clean":"2023-10-16T09:44:57.789146+0000","last_became_active":"2023-10-16T09:44:56.814006+0000","last_became_peered":"2023-10-16T09:44:56.814006+0000","last_unstale":"2023-10-16T09:44:57.789146+0000","last_undegraded":"2023-10-16T09:44:57.789146+0000","last_fullsized":"2023-10-16T09:44:57.789146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:29.557059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381639+0000","last_change":"2023-10-16T09:44:56.808137+0000","last_active":"2023-10-16T09:44:58.381639+0000","last_peered":"2023-10-16T09:44:58.381639+0000","last_clean":"2023-10-16T09:44:58.381639+0000","last_became_active":"2023-10-16T09:44:56.807685+0000","last_became_peered":"2023-10-16T09:44:56.807685+0000","last_unstale":"2023-10-16T09:44:58.381639+0000","last_undegraded":"2023-10-16T09:44:58.381639+0000","last_fullsized":"2023-10-16T09:44:58.381639+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:02.475372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789885+0000","last_change":"2023-10-16T09:44:56.804238+0000","last_active":"2023-10-16T09:44:57.789885+0000","last_peered":"2023-10-16T09:44:57.789885+0000","last_clean":"2023-10-16T09:44:57.789885+0000","last_became_active":"2023-10-16T09:44:56.803899+0000","last_became_peered":"2023-10-16T09:44:56.803899+0000","last_unstale":"2023-10-16T09:44:57.789885+0000","last_undegraded":"2023-10-16T09:44:57.789885+0000","last_fullsized":"2023-10-16T09:44:57.789885+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:21.710655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189141+0000","last_change":"2023-10-16T09:44:56.818972+0000","last_active":"2023-10-16T09:44:58.189141+0000","last_peered":"2023-10-16T09:44:58.189141+0000","last_clean":"2023-10-16T09:44:58.189141+0000","last_became_active":"2023-10-16T09:44:56.818641+0000","last_became_peered":"2023-10-16T09:44:56.818641+0000","last_unstale":"2023-10-16T09:44:58.189141+0000","last_undegraded":"2023-10-16T09:44:58.189141+0000","last_fullsized":"2023-10-16T09:44:58.189141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:52.793141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722281+0000","last_change":"2023-10-16T09:44:56.799942+0000","last_active":"2023-10-16T09:44:58.722281+0000","last_peered":"2023-10-16T09:44:58.722281+0000","last_clean":"2023-10-16T09:44:58.722281+0000","last_became_active":"2023-10-16T09:44:56.799652+0000","last_became_peered":"2023-10-16T09:44:56.799652+0000","last_unstale":"2023-10-16T09:44:58.722281+0000","last_undegraded":"2023-10-16T09:44:58.722281+0000","last_fullsized":"2023-10-16T09:44:58.722281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:25.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812676+0000","last_change":"2023-10-16T09:44:56.809197+0000","last_active":"2023-10-16T09:44:57.812676+0000","last_peered":"2023-10-16T09:44:57.812676+0000","last_clean":"2023-10-16T09:44:57.812676+0000","last_became_active":"2023-10-16T09:44:56.808905+0000","last_became_peered":"2023-10-16T09:44:56.808905+0000","last_unstale":"2023-10-16T09:44:57.812676+0000","last_undegraded":"2023-10-16T09:44:57.812676+0000","last_fullsized":"2023-10-16T09:44:57.812676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:31.365459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722655+0000","last_change":"2023-10-16T09:44:56.806839+0000","last_active":"2023-10-16T09:44:58.722655+0000","last_peered":"2023-10-16T09:44:58.722655+0000","last_clean":"2023-10-16T09:44:58.722655+0000","last_became_active":"2023-10-16T09:44:56.806531+0000","last_became_peered":"2023-10-16T09:44:56.806531+0000","last_unstale":"2023-10-16T09:44:58.722655+0000","last_undegraded":"2023-10-16T09:44:58.722655+0000","last_fullsized":"2023-10-16T09:44:58.722655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:34.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428936+0000","last_change":"2023-10-16T09:44:56.812485+0000","last_active":"2023-10-16T09:44:58.428936+0000","last_peered":"2023-10-16T09:44:58.428936+0000","last_clean":"2023-10-16T09:44:58.428936+0000","last_became_active":"2023-10-16T09:44:56.812061+0000","last_became_peered":"2023-10-16T09:44:56.812061+0000","last_unstale":"2023-10-16T09:44:58.428936+0000","last_undegraded":"2023-10-16T09:44:58.428936+0000","last_fullsized":"2023-10-16T09:44:58.428936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:47.916598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812812+0000","last_change":"2023-10-16T09:44:55.800280+0000","last_active":"2023-10-16T09:44:57.812812+0000","last_peered":"2023-10-16T09:44:57.812812+0000","last_clean":"2023-10-16T09:44:57.812812+0000","last_became_active":"2023-10-16T09:44:55.799942+0000","last_became_peered":"2023-10-16T09:44:55.799942+0000","last_unstale":"2023-10-16T09:44:57.812812+0000","last_undegraded":"2023-10-16T09:44:57.812812+0000","last_fullsized":"2023-10-16T09:44:57.812812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:18:47.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812733+0000","last_change":"2023-10-16T09:44:56.804130+0000","last_active":"2023-10-16T09:44:57.812733+0000","last_peered":"2023-10-16T09:44:57.812733+0000","last_clean":"2023-10-16T09:44:57.812733+0000","last_became_active":"2023-10-16T09:44:56.803840+0000","last_became_peered":"2023-10-16T09:44:56.803840+0000","last_unstale":"2023-10-16T09:44:57.812733+0000","last_undegraded":"2023-10-16T09:44:57.812733+0000","last_fullsized":"2023-10-16T09:44:57.812733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:25.271373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428878+0000","last_change":"2023-10-16T09:44:55.804801+0000","last_active":"2023-10-16T09:44:58.428878+0000","last_peered":"2023-10-16T09:44:58.428878+0000","last_clean":"2023-10-16T09:44:58.428878+0000","last_became_active":"2023-10-16T09:44:55.804524+0000","last_became_peered":"2023-10-16T09:44:55.804524+0000","last_unstale":"2023-10-16T09:44:58.428878+0000","last_undegraded":"2023-10-16T09:44:58.428878+0000","last_fullsized":"2023-10-16T09:44:58.428878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:55.256612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189174+0000","last_change":"2023-10-16T09:44:56.818522+0000","last_active":"2023-10-16T09:44:58.189174+0000","last_peered":"2023-10-16T09:44:58.189174+0000","last_clean":"2023-10-16T09:44:58.189174+0000","last_became_active":"2023-10-16T09:44:56.818278+0000","last_became_peered":"2023-10-16T09:44:56.818278+0000","last_unstale":"2023-10-16T09:44:58.189174+0000","last_undegraded":"2023-10-16T09:44:58.189174+0000","last_fullsized":"2023-10-16T09:44:58.189174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:21.572345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.820330+0000","last_change":"2023-10-16T09:44:55.801884+0000","last_active":"2023-10-16T09:44:56.820330+0000","last_peered":"2023-10-16T09:44:56.820330+0000","last_clean":"2023-10-16T09:44:56.820330+0000","last_became_active":"2023-10-16T09:44:55.801654+0000","last_became_peered":"2023-10-16T09:44:55.801654+0000","last_unstale":"2023-10-16T09:44:56.820330+0000","last_undegraded":"2023-10-16T09:44:56.820330+0000","last_fullsized":"2023-10-16T09:44:56.820330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:41.008581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381593+0000","last_change":"2023-10-16T09:44:56.807961+0000","last_active":"2023-10-16T09:44:58.381593+0000","last_peered":"2023-10-16T09:44:58.381593+0000","last_clean":"2023-10-16T09:44:58.381593+0000","last_became_active":"2023-10-16T09:44:56.807320+0000","last_became_peered":"2023-10-16T09:44:56.807320+0000","last_unstale":"2023-10-16T09:44:58.381593+0000","last_undegraded":"2023-10-16T09:44:58.381593+0000","last_fullsized":"2023-10-16T09:44:58.381593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:25.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789903+0000","last_change":"2023-10-16T09:44:55.803670+0000","last_active":"2023-10-16T09:44:57.789903+0000","last_peered":"2023-10-16T09:44:57.789903+0000","last_clean":"2023-10-16T09:44:57.789903+0000","last_became_active":"2023-10-16T09:44:55.803243+0000","last_became_peered":"2023-10-16T09:44:55.803243+0000","last_unstale":"2023-10-16T09:44:57.789903+0000","last_undegraded":"2023-10-16T09:44:57.789903+0000","last_fullsized":"2023-10-16T09:44:57.789903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:38:59.831210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722334+0000","last_change":"2023-10-16T09:44:56.801591+0000","last_active":"2023-10-16T09:44:58.722334+0000","last_peered":"2023-10-16T09:44:58.722334+0000","last_clean":"2023-10-16T09:44:58.722334+0000","last_became_active":"2023-10-16T09:44:56.801303+0000","last_became_peered":"2023-10-16T09:44:56.801303+0000","last_unstale":"2023-10-16T09:44:58.722334+0000","last_undegraded":"2023-10-16T09:44:58.722334+0000","last_fullsized":"2023-10-16T09:44:58.722334+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:02.475372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.818370+0000","last_change":"2023-10-16T09:44:55.803450+0000","last_active":"2023-10-16T09:44:56.818370+0000","last_peered":"2023-10-16T09:44:56.818370+0000","last_clean":"2023-10-16T09:44:56.818370+0000","last_became_active":"2023-10-16T09:44:55.803110+0000","last_became_peered":"2023-10-16T09:44:55.803110+0000","last_unstale":"2023-10-16T09:44:56.818370+0000","last_undegraded":"2023-10-16T09:44:56.818370+0000","last_fullsized":"2023-10-16T09:44:56.818370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:01.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789788+0000","last_change":"2023-10-16T09:44:56.805027+0000","last_active":"2023-10-16T09:44:57.789788+0000","last_peered":"2023-10-16T09:44:57.789788+0000","last_clean":"2023-10-16T09:44:57.789788+0000","last_became_active":"2023-10-16T09:44:56.804715+0000","last_became_peered":"2023-10-16T09:44:56.804715+0000","last_unstale":"2023-10-16T09:44:57.789788+0000","last_undegraded":"2023-10-16T09:44:57.789788+0000","last_fullsized":"2023-10-16T09:44:57.789788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:04.066331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.817621+0000","last_change":"2023-10-16T09:44:55.803751+0000","last_active":"2023-10-16T09:44:56.817621+0000","last_peered":"2023-10-16T09:44:56.817621+0000","last_clean":"2023-10-16T09:44:56.817621+0000","last_became_active":"2023-10-16T09:44:55.803491+0000","last_became_peered":"2023-10-16T09:44:55.803491+0000","last_unstale":"2023-10-16T09:44:56.817621+0000","last_undegraded":"2023-10-16T09:44:56.817621+0000","last_fullsized":"2023-10-16T09:44:56.817621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:33.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189019+0000","last_change":"2023-10-16T09:44:56.809642+0000","last_active":"2023-10-16T09:44:58.189019+0000","last_peered":"2023-10-16T09:44:58.189019+0000","last_clean":"2023-10-16T09:44:58.189019+0000","last_became_active":"2023-10-16T09:44:56.809396+0000","last_became_peered":"2023-10-16T09:44:56.809396+0000","last_unstale":"2023-10-16T09:44:58.189019+0000","last_undegraded":"2023-10-16T09:44:58.189019+0000","last_fullsized":"2023-10-16T09:44:58.189019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:54.581883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.803972+0000","last_change":"2023-10-16T09:44:55.805779+0000","last_active":"2023-10-16T09:44:56.803972+0000","last_peered":"2023-10-16T09:44:56.803972+0000","last_clean":"2023-10-16T09:44:56.803972+0000","last_became_active":"2023-10-16T09:44:55.805466+0000","last_became_peered":"2023-10-16T09:44:55.805466+0000","last_unstale":"2023-10-16T09:44:56.803972+0000","last_undegraded":"2023-10-16T09:44:56.803972+0000","last_fullsized":"2023-10-16T09:44:56.803972+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:03.064406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381249+0000","last_change":"2023-10-16T09:44:56.797572+0000","last_active":"2023-10-16T09:44:58.381249+0000","last_peered":"2023-10-16T09:44:58.381249+0000","last_clean":"2023-10-16T09:44:58.381249+0000","last_became_active":"2023-10-16T09:44:56.797084+0000","last_became_peered":"2023-10-16T09:44:56.797084+0000","last_unstale":"2023-10-16T09:44:58.381249+0000","last_undegraded":"2023-10-16T09:44:58.381249+0000","last_fullsized":"2023-10-16T09:44:58.381249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:08:30.600822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.799894+0000","last_change":"2023-10-16T09:44:55.803251+0000","last_active":"2023-10-16T09:44:56.799894+0000","last_peered":"2023-10-16T09:44:56.799894+0000","last_clean":"2023-10-16T09:44:56.799894+0000","last_became_active":"2023-10-16T09:44:55.802925+0000","last_became_peered":"2023-10-16T09:44:55.802925+0000","last_unstale":"2023-10-16T09:44:56.799894+0000","last_undegraded":"2023-10-16T09:44:56.799894+0000","last_fullsized":"2023-10-16T09:44:56.799894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:26.340087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790027+0000","last_change":"2023-10-16T09:44:56.818058+0000","last_active":"2023-10-16T09:44:57.790027+0000","last_peered":"2023-10-16T09:44:57.790027+0000","last_clean":"2023-10-16T09:44:57.790027+0000","last_became_active":"2023-10-16T09:44:56.817736+0000","last_became_peered":"2023-10-16T09:44:56.817736+0000","last_unstale":"2023-10-16T09:44:57.790027+0000","last_undegraded":"2023-10-16T09:44:57.790027+0000","last_fullsized":"2023-10-16T09:44:57.790027+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:42.010506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.804241+0000","last_change":"2023-10-16T09:44:55.806160+0000","last_active":"2023-10-16T09:44:56.804241+0000","last_peered":"2023-10-16T09:44:56.804241+0000","last_clean":"2023-10-16T09:44:56.804241+0000","last_became_active":"2023-10-16T09:44:55.805818+0000","last_became_peered":"2023-10-16T09:44:55.805818+0000","last_unstale":"2023-10-16T09:44:56.804241+0000","last_undegraded":"2023-10-16T09:44:56.804241+0000","last_fullsized":"2023-10-16T09:44:56.804241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:08.118195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788923+0000","last_change":"2023-10-16T09:44:56.810480+0000","last_active":"2023-10-16T09:44:57.788923+0000","last_peered":"2023-10-16T09:44:57.788923+0000","last_clean":"2023-10-16T09:44:57.788923+0000","last_became_active":"2023-10-16T09:44:56.810198+0000","last_became_peered":"2023-10-16T09:44:56.810198+0000","last_unstale":"2023-10-16T09:44:57.788923+0000","last_undegraded":"2023-10-16T09:44:57.788923+0000","last_fullsized":"2023-10-16T09:44:57.788923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:46.160206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428931+0000","last_change":"2023-10-16T09:44:55.803506+0000","last_active":"2023-10-16T09:44:58.428931+0000","last_peered":"2023-10-16T09:44:58.428931+0000","last_clean":"2023-10-16T09:44:58.428931+0000","last_became_active":"2023-10-16T09:44:55.803232+0000","last_became_peered":"2023-10-16T09:44:55.803232+0000","last_unstale":"2023-10-16T09:44:58.428931+0000","last_undegraded":"2023-10-16T09:44:58.428931+0000","last_fullsized":"2023-10-16T09:44:58.428931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:09.850844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788718+0000","last_change":"2023-10-16T09:44:56.816232+0000","last_active":"2023-10-16T09:44:57.788718+0000","last_peered":"2023-10-16T09:44:57.788718+0000","last_clean":"2023-10-16T09:44:57.788718+0000","last_became_active":"2023-10-16T09:44:56.815980+0000","last_became_peered":"2023-10-16T09:44:56.815980+0000","last_unstale":"2023-10-16T09:44:57.788718+0000","last_undegraded":"2023-10-16T09:44:57.788718+0000","last_fullsized":"2023-10-16T09:44:57.788718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:37.765122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.811802+0000","last_change":"2023-10-16T09:44:55.797411+0000","last_active":"2023-10-16T09:44:56.811802+0000","last_peered":"2023-10-16T09:44:56.811802+0000","last_clean":"2023-10-16T09:44:56.811802+0000","last_became_active":"2023-10-16T09:44:55.797121+0000","last_became_peered":"2023-10-16T09:44:55.797121+0000","last_unstale":"2023-10-16T09:44:56.811802+0000","last_undegraded":"2023-10-16T09:44:56.811802+0000","last_fullsized":"2023-10-16T09:44:56.811802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:24.771136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789997+0000","last_change":"2023-10-16T09:44:56.799562+0000","last_active":"2023-10-16T09:44:57.789997+0000","last_peered":"2023-10-16T09:44:57.789997+0000","last_clean":"2023-10-16T09:44:57.789997+0000","last_became_active":"2023-10-16T09:44:56.798274+0000","last_became_peered":"2023-10-16T09:44:56.798274+0000","last_unstale":"2023-10-16T09:44:57.789997+0000","last_undegraded":"2023-10-16T09:44:57.789997+0000","last_fullsized":"2023-10-16T09:44:57.789997+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:09.120120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.824354+0000","last_change":"2023-10-16T09:44:55.804174+0000","last_active":"2023-10-16T09:44:56.824354+0000","last_peered":"2023-10-16T09:44:56.824354+0000","last_clean":"2023-10-16T09:44:56.824354+0000","last_became_active":"2023-10-16T09:44:55.803860+0000","last_became_peered":"2023-10-16T09:44:55.803860+0000","last_unstale":"2023-10-16T09:44:56.824354+0000","last_undegraded":"2023-10-16T09:44:56.824354+0000","last_fullsized":"2023-10-16T09:44:56.824354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:56.783570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381258+0000","last_change":"2023-10-16T09:44:56.797512+0000","last_active":"2023-10-16T09:44:58.381258+0000","last_peered":"2023-10-16T09:44:58.381258+0000","last_clean":"2023-10-16T09:44:58.381258+0000","last_became_active":"2023-10-16T09:44:56.797185+0000","last_became_peered":"2023-10-16T09:44:56.797185+0000","last_unstale":"2023-10-16T09:44:58.381258+0000","last_undegraded":"2023-10-16T09:44:58.381258+0000","last_fullsized":"2023-10-16T09:44:58.381258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:40.280433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.811314+0000","last_change":"2023-10-16T09:44:55.802245+0000","last_active":"2023-10-16T09:44:56.811314+0000","last_peered":"2023-10-16T09:44:56.811314+0000","last_clean":"2023-10-16T09:44:56.811314+0000","last_became_active":"2023-10-16T09:44:55.801952+0000","last_became_peered":"2023-10-16T09:44:55.801952+0000","last_unstale":"2023-10-16T09:44:56.811314+0000","last_undegraded":"2023-10-16T09:44:56.811314+0000","last_fullsized":"2023-10-16T09:44:56.811314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:19:57.048737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788957+0000","last_change":"2023-10-16T09:44:56.809135+0000","last_active":"2023-10-16T09:44:57.788957+0000","last_peered":"2023-10-16T09:44:57.788957+0000","last_clean":"2023-10-16T09:44:57.788957+0000","last_became_active":"2023-10-16T09:44:56.808859+0000","last_became_peered":"2023-10-16T09:44:56.808859+0000","last_unstale":"2023-10-16T09:44:57.788957+0000","last_undegraded":"2023-10-16T09:44:57.788957+0000","last_fullsized":"2023-10-16T09:44:57.788957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:39.281794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.790319+0000","last_change":"2023-10-16T09:44:55.795060+0000","last_active":"2023-10-16T09:44:56.790319+0000","last_peered":"2023-10-16T09:44:56.790319+0000","last_clean":"2023-10-16T09:44:56.790319+0000","last_became_active":"2023-10-16T09:44:55.794776+0000","last_became_peered":"2023-10-16T09:44:55.794776+0000","last_unstale":"2023-10-16T09:44:56.790319+0000","last_undegraded":"2023-10-16T09:44:56.790319+0000","last_fullsized":"2023-10-16T09:44:56.790319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:08.744911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722494+0000","last_change":"2023-10-16T09:44:56.807538+0000","last_active":"2023-10-16T09:44:58.722494+0000","last_peered":"2023-10-16T09:44:58.722494+0000","last_clean":"2023-10-16T09:44:58.722494+0000","last_became_active":"2023-10-16T09:44:56.806706+0000","last_became_peered":"2023-10-16T09:44:56.806706+0000","last_unstale":"2023-10-16T09:44:58.722494+0000","last_undegraded":"2023-10-16T09:44:58.722494+0000","last_fullsized":"2023-10-16T09:44:58.722494+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:04.066331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.804178+0000","last_change":"2023-10-16T09:44:55.803176+0000","last_active":"2023-10-16T09:44:56.804178+0000","last_peered":"2023-10-16T09:44:56.804178+0000","last_clean":"2023-10-16T09:44:56.804178+0000","last_became_active":"2023-10-16T09:44:55.802803+0000","last_became_peered":"2023-10-16T09:44:55.802803+0000","last_unstale":"2023-10-16T09:44:56.804178+0000","last_undegraded":"2023-10-16T09:44:56.804178+0000","last_fullsized":"2023-10-16T09:44:56.804178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.708730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812676+0000","last_change":"2023-10-16T09:44:56.802287+0000","last_active":"2023-10-16T09:44:57.812676+0000","last_peered":"2023-10-16T09:44:57.812676+0000","last_clean":"2023-10-16T09:44:57.812676+0000","last_became_active":"2023-10-16T09:44:56.801929+0000","last_became_peered":"2023-10-16T09:44:56.801929+0000","last_unstale":"2023-10-16T09:44:57.812676+0000","last_undegraded":"2023-10-16T09:44:57.812676+0000","last_fullsized":"2023-10-16T09:44:57.812676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:10.852769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.822603+0000","last_change":"2023-10-16T09:44:55.807648+0000","last_active":"2023-10-16T09:44:56.822603+0000","last_peered":"2023-10-16T09:44:56.822603+0000","last_clean":"2023-10-16T09:44:56.822603+0000","last_became_active":"2023-10-16T09:44:55.807375+0000","last_became_peered":"2023-10-16T09:44:55.807375+0000","last_unstale":"2023-10-16T09:44:56.822603+0000","last_undegraded":"2023-10-16T09:44:56.822603+0000","last_fullsized":"2023-10-16T09:44:56.822603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:01.473446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189391+0000","last_change":"2023-10-16T09:44:56.809884+0000","last_active":"2023-10-16T09:44:58.189391+0000","last_peered":"2023-10-16T09:44:58.189391+0000","last_clean":"2023-10-16T09:44:58.189391+0000","last_became_active":"2023-10-16T09:44:56.809518+0000","last_became_peered":"2023-10-16T09:44:56.809518+0000","last_unstale":"2023-10-16T09:44:58.189391+0000","last_undegraded":"2023-10-16T09:44:58.189391+0000","last_fullsized":"2023-10-16T09:44:58.189391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:39.561624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.809991+0000","last_change":"2023-10-16T09:44:55.803515+0000","last_active":"2023-10-16T09:44:56.809991+0000","last_peered":"2023-10-16T09:44:56.809991+0000","last_clean":"2023-10-16T09:44:56.809991+0000","last_became_active":"2023-10-16T09:44:55.803225+0000","last_became_peered":"2023-10-16T09:44:55.803225+0000","last_unstale":"2023-10-16T09:44:56.809991+0000","last_undegraded":"2023-10-16T09:44:56.809991+0000","last_fullsized":"2023-10-16T09:44:56.809991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:24.269448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790139+0000","last_change":"2023-10-16T09:44:56.806417+0000","last_active":"2023-10-16T09:44:57.790139+0000","last_peered":"2023-10-16T09:44:57.790139+0000","last_clean":"2023-10-16T09:44:57.790139+0000","last_became_active":"2023-10-16T09:44:56.805632+0000","last_became_peered":"2023-10-16T09:44:56.805632+0000","last_unstale":"2023-10-16T09:44:57.790139+0000","last_undegraded":"2023-10-16T09:44:57.790139+0000","last_fullsized":"2023-10-16T09:44:57.790139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:32.192715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.809364+0000","last_change":"2023-10-16T09:44:55.803364+0000","last_active":"2023-10-16T09:44:56.809364+0000","last_peered":"2023-10-16T09:44:56.809364+0000","last_clean":"2023-10-16T09:44:56.809364+0000","last_became_active":"2023-10-16T09:44:55.803063+0000","last_became_peered":"2023-10-16T09:44:55.803063+0000","last_unstale":"2023-10-16T09:44:56.809364+0000","last_undegraded":"2023-10-16T09:44:56.809364+0000","last_fullsized":"2023-10-16T09:44:56.809364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:35:08.435699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381463+0000","last_change":"2023-10-16T09:44:56.807966+0000","last_active":"2023-10-16T09:44:58.381463+0000","last_peered":"2023-10-16T09:44:58.381463+0000","last_clean":"2023-10-16T09:44:58.381463+0000","last_became_active":"2023-10-16T09:44:56.807616+0000","last_became_peered":"2023-10-16T09:44:56.807616+0000","last_unstale":"2023-10-16T09:44:58.381463+0000","last_undegraded":"2023-10-16T09:44:58.381463+0000","last_fullsized":"2023-10-16T09:44:58.381463+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:51.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.808673+0000","last_change":"2023-10-16T09:44:55.805347+0000","last_active":"2023-10-16T09:44:56.808673+0000","last_peered":"2023-10-16T09:44:56.808673+0000","last_clean":"2023-10-16T09:44:56.808673+0000","last_became_active":"2023-10-16T09:44:55.805054+0000","last_became_peered":"2023-10-16T09:44:55.805054+0000","last_unstale":"2023-10-16T09:44:56.808673+0000","last_undegraded":"2023-10-16T09:44:56.808673+0000","last_fullsized":"2023-10-16T09:44:56.808673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:27.087072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789833+0000","last_change":"2023-10-16T09:44:56.811326+0000","last_active":"2023-10-16T09:44:57.789833+0000","last_peered":"2023-10-16T09:44:57.789833+0000","last_clean":"2023-10-16T09:44:57.789833+0000","last_became_active":"2023-10-16T09:44:56.810995+0000","last_became_peered":"2023-10-16T09:44:56.810995+0000","last_unstale":"2023-10-16T09:44:57.789833+0000","last_undegraded":"2023-10-16T09:44:57.789833+0000","last_fullsized":"2023-10-16T09:44:57.789833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:06.524695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.802551+0000","last_change":"2023-10-16T09:44:55.803440+0000","last_active":"2023-10-16T09:44:56.802551+0000","last_peered":"2023-10-16T09:44:56.802551+0000","last_clean":"2023-10-16T09:44:56.802551+0000","last_became_active":"2023-10-16T09:44:55.803141+0000","last_became_peered":"2023-10-16T09:44:55.803141+0000","last_unstale":"2023-10-16T09:44:56.802551+0000","last_undegraded":"2023-10-16T09:44:56.802551+0000","last_fullsized":"2023-10-16T09:44:56.802551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:13.236917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789741+0000","last_change":"2023-10-16T09:44:56.806094+0000","last_active":"2023-10-16T09:44:57.789741+0000","last_peered":"2023-10-16T09:44:57.789741+0000","last_clean":"2023-10-16T09:44:57.789741+0000","last_became_active":"2023-10-16T09:44:56.805871+0000","last_became_peered":"2023-10-16T09:44:56.805871+0000","last_unstale":"2023-10-16T09:44:57.789741+0000","last_undegraded":"2023-10-16T09:44:57.789741+0000","last_fullsized":"2023-10-16T09:44:57.789741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:45.961536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428856+0000","last_change":"2023-10-16T09:44:55.810953+0000","last_active":"2023-10-16T09:44:58.428856+0000","last_peered":"2023-10-16T09:44:58.428856+0000","last_clean":"2023-10-16T09:44:58.428856+0000","last_became_active":"2023-10-16T09:44:55.810677+0000","last_became_peered":"2023-10-16T09:44:55.810677+0000","last_unstale":"2023-10-16T09:44:58.428856+0000","last_undegraded":"2023-10-16T09:44:58.428856+0000","last_fullsized":"2023-10-16T09:44:58.428856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:12:45.052666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789695+0000","last_change":"2023-10-16T09:44:56.805014+0000","last_active":"2023-10-16T09:44:57.789695+0000","last_peered":"2023-10-16T09:44:57.789695+0000","last_clean":"2023-10-16T09:44:57.789695+0000","last_became_active":"2023-10-16T09:44:56.804733+0000","last_became_peered":"2023-10-16T09:44:56.804733+0000","last_unstale":"2023-10-16T09:44:57.789695+0000","last_undegraded":"2023-10-16T09:44:57.789695+0000","last_fullsized":"2023-10-16T09:44:57.789695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:13.678959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812507+0000","last_change":"2023-10-16T09:44:55.801848+0000","last_active":"2023-10-16T09:44:57.812507+0000","last_peered":"2023-10-16T09:44:57.812507+0000","last_clean":"2023-10-16T09:44:57.812507+0000","last_became_active":"2023-10-16T09:44:55.801533+0000","last_became_peered":"2023-10-16T09:44:55.801533+0000","last_unstale":"2023-10-16T09:44:57.812507+0000","last_undegraded":"2023-10-16T09:44:57.812507+0000","last_fullsized":"2023-10-16T09:44:57.812507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:06:16.539616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789567+0000","last_change":"2023-10-16T09:44:56.806835+0000","last_active":"2023-10-16T09:44:57.789567+0000","last_peered":"2023-10-16T09:44:57.789567+0000","last_clean":"2023-10-16T09:44:57.789567+0000","last_became_active":"2023-10-16T09:44:56.806572+0000","last_became_peered":"2023-10-16T09:44:56.806572+0000","last_unstale":"2023-10-16T09:44:57.789567+0000","last_undegraded":"2023-10-16T09:44:57.789567+0000","last_fullsized":"2023-10-16T09:44:57.789567+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:58:40.004337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.799624+0000","last_change":"2023-10-16T09:44:55.801917+0000","last_active":"2023-10-16T09:44:56.799624+0000","last_peered":"2023-10-16T09:44:56.799624+0000","last_clean":"2023-10-16T09:44:56.799624+0000","last_became_active":"2023-10-16T09:44:55.801641+0000","last_became_peered":"2023-10-16T09:44:55.801641+0000","last_unstale":"2023-10-16T09:44:56.799624+0000","last_undegraded":"2023-10-16T09:44:56.799624+0000","last_fullsized":"2023-10-16T09:44:56.799624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:24.355232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722321+0000","last_change":"2023-10-16T09:44:56.803314+0000","last_active":"2023-10-16T09:44:58.722321+0000","last_peered":"2023-10-16T09:44:58.722321+0000","last_clean":"2023-10-16T09:44:58.722321+0000","last_became_active":"2023-10-16T09:44:56.803027+0000","last_became_peered":"2023-10-16T09:44:56.803027+0000","last_unstale":"2023-10-16T09:44:58.722321+0000","last_undegraded":"2023-10-16T09:44:58.722321+0000","last_fullsized":"2023-10-16T09:44:58.722321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:32.410416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.790722+0000","last_change":"2023-10-16T09:44:55.800277+0000","last_active":"2023-10-16T09:44:56.790722+0000","last_peered":"2023-10-16T09:44:56.790722+0000","last_clean":"2023-10-16T09:44:56.790722+0000","last_became_active":"2023-10-16T09:44:55.799950+0000","last_became_peered":"2023-10-16T09:44:55.799950+0000","last_unstale":"2023-10-16T09:44:56.790722+0000","last_undegraded":"2023-10-16T09:44:56.790722+0000","last_fullsized":"2023-10-16T09:44:56.790722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:50.064075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789665+0000","last_change":"2023-10-16T09:44:56.811816+0000","last_active":"2023-10-16T09:44:57.789665+0000","last_peered":"2023-10-16T09:44:57.789665+0000","last_clean":"2023-10-16T09:44:57.789665+0000","last_became_active":"2023-10-16T09:44:56.811579+0000","last_became_peered":"2023-10-16T09:44:56.811579+0000","last_unstale":"2023-10-16T09:44:57.789665+0000","last_undegraded":"2023-10-16T09:44:57.789665+0000","last_fullsized":"2023-10-16T09:44:57.789665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:08.998408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.812098+0000","last_change":"2023-10-16T09:44:55.795185+0000","last_active":"2023-10-16T09:44:56.812098+0000","last_peered":"2023-10-16T09:44:56.812098+0000","last_clean":"2023-10-16T09:44:56.812098+0000","last_became_active":"2023-10-16T09:44:55.794880+0000","last_became_peered":"2023-10-16T09:44:55.794880+0000","last_unstale":"2023-10-16T09:44:56.812098+0000","last_undegraded":"2023-10-16T09:44:56.812098+0000","last_fullsized":"2023-10-16T09:44:56.812098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:09.850844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790122+0000","last_change":"2023-10-16T09:44:56.805133+0000","last_active":"2023-10-16T09:44:57.790122+0000","last_peered":"2023-10-16T09:44:57.790122+0000","last_clean":"2023-10-16T09:44:57.790122+0000","last_became_active":"2023-10-16T09:44:56.798012+0000","last_became_peered":"2023-10-16T09:44:56.798012+0000","last_unstale":"2023-10-16T09:44:57.790122+0000","last_undegraded":"2023-10-16T09:44:57.790122+0000","last_fullsized":"2023-10-16T09:44:57.790122+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:38:27.019733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.430216+0000","last_change":"2023-10-16T09:44:55.802637+0000","last_active":"2023-10-16T09:44:58.430216+0000","last_peered":"2023-10-16T09:44:58.430216+0000","last_clean":"2023-10-16T09:44:58.430216+0000","last_became_active":"2023-10-16T09:44:55.802375+0000","last_became_peered":"2023-10-16T09:44:55.802375+0000","last_unstale":"2023-10-16T09:44:58.430216+0000","last_undegraded":"2023-10-16T09:44:58.430216+0000","last_fullsized":"2023-10-16T09:44:58.430216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:30.363534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428875+0000","last_change":"2023-10-16T09:44:56.813550+0000","last_active":"2023-10-16T09:44:58.428875+0000","last_peered":"2023-10-16T09:44:58.428875+0000","last_clean":"2023-10-16T09:44:58.428875+0000","last_became_active":"2023-10-16T09:44:56.813288+0000","last_became_peered":"2023-10-16T09:44:56.813288+0000","last_unstale":"2023-10-16T09:44:58.428875+0000","last_undegraded":"2023-10-16T09:44:58.428875+0000","last_fullsized":"2023-10-16T09:44:58.428875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:01.325830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.791773+0000","last_change":"2023-10-16T09:44:55.805218+0000","last_active":"2023-10-16T09:44:56.791773+0000","last_peered":"2023-10-16T09:44:56.791773+0000","last_clean":"2023-10-16T09:44:56.791773+0000","last_became_active":"2023-10-16T09:44:55.804922+0000","last_became_peered":"2023-10-16T09:44:55.804922+0000","last_unstale":"2023-10-16T09:44:56.791773+0000","last_undegraded":"2023-10-16T09:44:56.791773+0000","last_fullsized":"2023-10-16T09:44:56.791773+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:46.914673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812467+0000","last_change":"2023-10-16T09:44:56.805897+0000","last_active":"2023-10-16T09:44:57.812467+0000","last_peered":"2023-10-16T09:44:57.812467+0000","last_clean":"2023-10-16T09:44:57.812467+0000","last_became_active":"2023-10-16T09:44:56.805621+0000","last_became_peered":"2023-10-16T09:44:56.805621+0000","last_unstale":"2023-10-16T09:44:57.812467+0000","last_undegraded":"2023-10-16T09:44:57.812467+0000","last_fullsized":"2023-10-16T09:44:57.812467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:56.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428802+0000","last_change":"2023-10-16T09:44:55.801484+0000","last_active":"2023-10-16T09:44:58.428802+0000","last_peered":"2023-10-16T09:44:58.428802+0000","last_clean":"2023-10-16T09:44:58.428802+0000","last_became_active":"2023-10-16T09:44:55.801193+0000","last_became_peered":"2023-10-16T09:44:55.801193+0000","last_unstale":"2023-10-16T09:44:58.428802+0000","last_undegraded":"2023-10-16T09:44:58.428802+0000","last_fullsized":"2023-10-16T09:44:58.428802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:18:47.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428967+0000","last_change":"2023-10-16T09:44:56.806894+0000","last_active":"2023-10-16T09:44:58.428967+0000","last_peered":"2023-10-16T09:44:58.428967+0000","last_clean":"2023-10-16T09:44:58.428967+0000","last_became_active":"2023-10-16T09:44:56.806596+0000","last_became_peered":"2023-10-16T09:44:56.806596+0000","last_unstale":"2023-10-16T09:44:58.428967+0000","last_undegraded":"2023-10-16T09:44:58.428967+0000","last_fullsized":"2023-10-16T09:44:58.428967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:03.733810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.790458+0000","last_change":"2023-10-16T09:44:55.792347+0000","last_active":"2023-10-16T09:44:56.790458+0000","last_peered":"2023-10-16T09:44:56.790458+0000","last_clean":"2023-10-16T09:44:56.790458+0000","last_became_active":"2023-10-16T09:44:55.792073+0000","last_became_peered":"2023-10-16T09:44:55.792073+0000","last_unstale":"2023-10-16T09:44:56.790458+0000","last_undegraded":"2023-10-16T09:44:56.790458+0000","last_fullsized":"2023-10-16T09:44:56.790458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:26.235932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788414+0000","last_change":"2023-10-16T09:44:56.817067+0000","last_active":"2023-10-16T09:44:57.788414+0000","last_peered":"2023-10-16T09:44:57.788414+0000","last_clean":"2023-10-16T09:44:57.788414+0000","last_became_active":"2023-10-16T09:44:56.816905+0000","last_became_peered":"2023-10-16T09:44:56.816905+0000","last_unstale":"2023-10-16T09:44:57.788414+0000","last_undegraded":"2023-10-16T09:44:57.788414+0000","last_fullsized":"2023-10-16T09:44:57.788414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:34.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.799817+0000","last_change":"2023-10-16T09:44:55.801193+0000","last_active":"2023-10-16T09:44:56.799817+0000","last_peered":"2023-10-16T09:44:56.799817+0000","last_clean":"2023-10-16T09:44:56.799817+0000","last_became_active":"2023-10-16T09:44:55.800923+0000","last_became_peered":"2023-10-16T09:44:55.800923+0000","last_unstale":"2023-10-16T09:44:56.799817+0000","last_undegraded":"2023-10-16T09:44:56.799817+0000","last_fullsized":"2023-10-16T09:44:56.799817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:35:15.291908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.188912+0000","last_change":"2023-10-16T09:44:56.814220+0000","last_active":"2023-10-16T09:44:58.188912+0000","last_peered":"2023-10-16T09:44:58.188912+0000","last_clean":"2023-10-16T09:44:58.188912+0000","last_became_active":"2023-10-16T09:44:56.813944+0000","last_became_peered":"2023-10-16T09:44:56.813944+0000","last_unstale":"2023-10-16T09:44:58.188912+0000","last_undegraded":"2023-10-16T09:44:58.188912+0000","last_fullsized":"2023-10-16T09:44:58.188912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:10.246138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.803595+0000","last_change":"2023-10-16T09:44:55.806449+0000","last_active":"2023-10-16T09:44:56.803595+0000","last_peered":"2023-10-16T09:44:56.803595+0000","last_clean":"2023-10-16T09:44:56.803595+0000","last_became_active":"2023-10-16T09:44:55.806201+0000","last_became_peered":"2023-10-16T09:44:55.806201+0000","last_unstale":"2023-10-16T09:44:56.803595+0000","last_undegraded":"2023-10-16T09:44:56.803595+0000","last_fullsized":"2023-10-16T09:44:56.803595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:13.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722525+0000","last_change":"2023-10-16T09:44:56.807830+0000","last_active":"2023-10-16T09:44:58.722525+0000","last_peered":"2023-10-16T09:44:58.722525+0000","last_clean":"2023-10-16T09:44:58.722525+0000","last_became_active":"2023-10-16T09:44:56.807547+0000","last_became_peered":"2023-10-16T09:44:56.807547+0000","last_unstale":"2023-10-16T09:44:58.722525+0000","last_undegraded":"2023-10-16T09:44:58.722525+0000","last_fullsized":"2023-10-16T09:44:58.722525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:57.785495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812507+0000","last_change":"2023-10-16T09:44:55.801763+0000","last_active":"2023-10-16T09:44:57.812507+0000","last_peered":"2023-10-16T09:44:57.812507+0000","last_clean":"2023-10-16T09:44:57.812507+0000","last_became_active":"2023-10-16T09:44:55.801471+0000","last_became_peered":"2023-10-16T09:44:55.801471+0000","last_unstale":"2023-10-16T09:44:57.812507+0000","last_undegraded":"2023-10-16T09:44:57.812507+0000","last_fullsized":"2023-10-16T09:44:57.812507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:26.235932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.789023+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:56.789023+0000","last_peered":"2023-10-16T09:44:56.789023+0000","last_clean":"2023-10-16T09:44:56.789023+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:56.789023+0000","last_undegraded":"2023-10-16T09:44:56.789023+0000","last_fullsized":"2023-10-16T09:44:56.789023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":978944,"data_stored":737829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.63800000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46100000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.39500000000000002}]},{"osd":5,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.57799999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":6.194,"5min":6.194,"15min":6.194},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":6.1609999999999996,"5min":6.1609999999999996,"15min":6.1609999999999996},"last":0.39400000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":8.5600000000000005,"5min":8.5600000000000005,"15min":8.5600000000000005},"last":0.624},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":6.2249999999999996,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.67000000000000004}]},{"osd":4,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56000000000000005}]},{"osd":5,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65900000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":149005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36199999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.624},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.60599999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":737739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":5.5090000000000003,"5min":5.5090000000000003,"15min":5.5090000000000003},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":6.4950000000000001,"5min":6.4950000000000001,"15min":6.4950000000000001},"last":0.44800000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.374}]},{"osd":3,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":8.3260000000000005,"5min":8.3260000000000005,"15min":8.3260000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":5.4809999999999999,"5min":5.4809999999999999,"15min":5.4809999999999999},"last":0.35399999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.4349999999999996,"5min":6.4349999999999996,"15min":6.4349999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.62}]},{"osd":5,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":5.4160000000000004,"5min":5.4160000000000004,"15min":5.4160000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.3869999999999996,"5min":6.3869999999999996,"15min":6.3869999999999996},"last":0.63300000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":8.2189999999999994,"5min":8.2189999999999994,"15min":8.2189999999999994},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.4560000000000004,"5min":5.4560000000000004,"15min":5.4560000000000004},"last":0.51900000000000002}]},{"osd":7,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":6.4649999999999999,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":8.266,"5min":8.266,"15min":8.266},"last":0.56000000000000005}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":147441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.52100000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67400000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70099999999999996}]},{"osd":6,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.66000000000000003}]},{"osd":7,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.73099999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":149005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.317}]},{"osd":1,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":6.2240000000000002},"last":0.36499999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.29299999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.4420000000000002,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":0.43099999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.505}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":147475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.71599999999999997}]},{"osd":3,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.63600000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76200000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66300000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0370000000000008,"5min":8.0370000000000008,"15min":8.0370000000000008},"last":0.51800000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":8.2110000000000003,"5min":8.2110000000000003,"15min":8.2110000000000003},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.1140000000000008,"5min":8.1140000000000008,"15min":8.1140000000000008},"last":0.434},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":8.3000000000000007,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.54800000000000004}]},{"osd":3,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":9.8810000000000002,"5min":9.8810000000000002,"15min":9.8810000000000002},"last":0.57399999999999995}]},{"osd":4,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.63100000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.66200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:45:05.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:45:05.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.652+0000 7f7f02cd6700 1 Processor -- start 2023-10-16T09:45:05.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.652+0000 7f7f02cd6700 1 -- start start 2023-10-16T09:45:05.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f02cd6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7efc0cab40 0x7f7efc0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f02cd6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7efc15e7f0 0x7f7efc06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f02cd6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0cc310 con 0x7f7efc0b1ee0 2023-10-16T09:45:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f02cd6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc15e0f0 con 0x7f7efc15e7f0 2023-10-16T09:45:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f02cd6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc0cf820 con 0x7f7efc0cab40 2023-10-16T09:45:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7efc15e7f0 0x7f7efc06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7efc15e7f0 0x7f7efc06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38068/0 (socket says 172.21.15.181:38068) 2023-10-16T09:45:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 -- 172.21.15.181:0/3177064994 learned_addr learned my addr 172.21.15.181:0/3177064994 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7efbfff700 1 --2- 172.21.15.181:0/3177064994 >> v2:172.21.15.190:3300/0 conn(0x7f7efc0cab40 0x7f7efc0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 -- 172.21.15.181:0/3177064994 >> v2:172.21.15.190:3300/0 conn(0x7f7efc0cab40 msgr2=0x7f7efc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 --2- 172.21.15.181:0/3177064994 >> v2:172.21.15.190:3300/0 conn(0x7f7efc0cab40 0x7f7efc0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 -- 172.21.15.181:0/3177064994 >> v1:172.21.15.190:6789/0 conn(0x7f7efc0b1ee0 legacy=0x7f7efc0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 -- 172.21.15.181:0/3177064994 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0cc590 con 0x7f7efc15e7f0 2023-10-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7f01273700 1 --2- 172.21.15.181:0/3177064994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc06a8a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009e50 tx=0x7f7ef400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5747e97207403114 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7efb7fe700 1 -- 172.21.15.181:0/3177064994 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400e070 con 0x7f7efc15e7f0 2023-10-16T09:45:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7efb7fe700 1 -- 172.21.15.181:0/3177064994 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4003a20 con 0x7f7efc15e7f0 2023-10-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.653+0000 7f7efb7fe700 1 -- 172.21.15.181:0/3177064994 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4003dd0 con 0x7f7efc15e7f0 2023-10-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 -- 172.21.15.181:0/3177064994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 msgr2=0x7f7efc06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/3177064994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc06a8a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009e50 tx=0x7f7ef400b330 comp rx=0 tx=0).stop 2023-10-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 -- 172.21.15.181:0/3177064994 shutdown_connections 2023-10-16T09:45:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/3177064994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc06a8a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/3177064994 >> v2:172.21.15.190:3300/0 conn(0x7f7efc0cab40 0x7f7efc0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 -- 172.21.15.181:0/3177064994 >> 172.21.15.181:0/3177064994 conn(0x7f7efc0d0460 msgr2=0x7f7efc0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 -- 172.21.15.181:0/3177064994 shutdown_connections 2023-10-16T09:45:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.654+0000 7f7f02cd6700 1 -- 172.21.15.181:0/3177064994 wait complete. 2023-10-16T09:45:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.655+0000 7f7f02cd6700 1 Processor -- start 2023-10-16T09:45:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.655+0000 7f7f02cd6700 1 -- start start 2023-10-16T09:45:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f02cd6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7efc0b1ee0 0x7f7efc15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f02cd6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7efc0cab40 0x7f7efc0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f02cd6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f02cd6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0b5ce0 con 0x7f7efc15e7f0 2023-10-16T09:45:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f02cd6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1f3030 con 0x7f7efc0b1ee0 2023-10-16T09:45:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f02cd6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0b3e20 con 0x7f7efc0cab40 2023-10-16T09:45:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc0b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38074/0 (socket says 172.21.15.181:38074) 2023-10-16T09:45:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 -- 172.21.15.181:0/2392882542 learned_addr learned my addr 172.21.15.181:0/2392882542 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f00a72700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7efc0b1ee0 0x7f7efc15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7efbfff700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7efc0cab40 0x7f7efc0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 -- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7efc0cab40 msgr2=0x7f7efc0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7efc0cab40 0x7f7efc0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 -- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7efc0b1ee0 msgr2=0x7f7efc15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7efc0b1ee0 0x7f7efc15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.656+0000 7f7f01273700 1 -- 172.21.15.181:0/2392882542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4009b30 con 0x7f7efc15e7f0 2023-10-16T09:45:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.657+0000 7f7f01273700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc0b04a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7efc070600 tx=0x7f7ef4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.657+0000 7f7ef97fa700 1 -- 172.21.15.181:0/2392882542 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4004440 con 0x7f7efc15e7f0 2023-10-16T09:45:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.657+0000 7f7ef97fa700 1 -- 172.21.15.181:0/2392882542 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef40045c0 con 0x7f7efc15e7f0 2023-10-16T09:45:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.657+0000 7f7ef97fa700 1 -- 172.21.15.181:0/2392882542 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4004970 con 0x7f7efc15e7f0 2023-10-16T09:45:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.657+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0b4100 con 0x7f7efc15e7f0 2023-10-16T09:45:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.657+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc0b46c0 con 0x7f7efc15e7f0 2023-10-16T09:45:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.658+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc067d20 con 0x7f7efc15e7f0 2023-10-16T09:45:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.658+0000 7f7ef97fa700 1 -- 172.21.15.181:0/2392882542 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ef4025450 con 0x7f7efc15e7f0 2023-10-16T09:45:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.658+0000 7f7ef97fa700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7ee006af00 0x7f7ee006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.659+0000 7f7ef97fa700 1 -- 172.21.15.181:0/2392882542 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ef408abb0 con 0x7f7efc15e7f0 2023-10-16T09:45:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.659+0000 7f7f00a72700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7ee006af00 0x7f7ee006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.659+0000 7f7f00a72700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7ee006af00 0x7f7ee006d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0006fd0 tx=0x7f7ef00090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.663+0000 7f7ef97fa700 1 -- 172.21.15.181:0/2392882542 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7ef4059780 con 0x7f7efc15e7f0 2023-10-16T09:45:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.853+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7efc0ccb50 con 0x7f7ee006af00 2023-10-16T09:45:05.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.860+0000 7f7ef97fa700 1 -- 172.21.15.181:0/2392882542 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289213 (secure 0 0 0) 0x7f7ef0009e20 con 0x7f7ee006af00 2023-10-16T09:45:05.866 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7ee006af00 msgr2=0x7f7ee006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7ee006af00 0x7f7ee006d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0006fd0 tx=0x7f7ef00090e0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 msgr2=0x7f7efc0b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc0b04a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7efc070600 tx=0x7f7ef4004ad0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 shutdown_connections 2023-10-16T09:45:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7ee006af00 0x7f7ee006d3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7efc15e7f0 0x7f7efc0b04a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7efc0b1ee0 0x7f7efc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 --2- 172.21.15.181:0/2392882542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7efc0cab40 0x7f7efc0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 >> 172.21.15.181:0/2392882542 conn(0x7f7efc0d0460 msgr2=0x7f7efc0d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 shutdown_connections 2023-10-16T09:45:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:05.864+0000 7f7f02cd6700 1 -- 172.21.15.181:0/2392882542 wait complete. 2023-10-16T09:45:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:45:05.878 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-16T09:45:05.691113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220384,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754336,"statfs":{"total":773094113280,"available":772868440064,"internally_reserved":0,"allocated":4300800,"data_stored":2364524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.902889"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812734+0000","last_change":"2023-10-16T09:44:56.802788+0000","last_active":"2023-10-16T09:44:57.812734+0000","last_peered":"2023-10-16T09:44:57.812734+0000","last_clean":"2023-10-16T09:44:57.812734+0000","last_became_active":"2023-10-16T09:44:56.801293+0000","last_became_peered":"2023-10-16T09:44:56.801293+0000","last_unstale":"2023-10-16T09:44:57.812734+0000","last_undegraded":"2023-10-16T09:44:57.812734+0000","last_fullsized":"2023-10-16T09:44:57.812734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:01.325830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789003+0000","last_change":"2023-10-16T09:44:56.816768+0000","last_active":"2023-10-16T09:44:57.789003+0000","last_peered":"2023-10-16T09:44:57.789003+0000","last_clean":"2023-10-16T09:44:57.789003+0000","last_became_active":"2023-10-16T09:44:56.814617+0000","last_became_peered":"2023-10-16T09:44:56.814617+0000","last_unstale":"2023-10-16T09:44:57.789003+0000","last_undegraded":"2023-10-16T09:44:57.789003+0000","last_fullsized":"2023-10-16T09:44:57.789003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:51:59.527316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790065+0000","last_change":"2023-10-16T09:44:56.798977+0000","last_active":"2023-10-16T09:44:57.790065+0000","last_peered":"2023-10-16T09:44:57.790065+0000","last_clean":"2023-10-16T09:44:57.790065+0000","last_became_active":"2023-10-16T09:44:56.798301+0000","last_became_peered":"2023-10-16T09:44:56.798301+0000","last_unstale":"2023-10-16T09:44:57.790065+0000","last_undegraded":"2023-10-16T09:44:57.790065+0000","last_fullsized":"2023-10-16T09:44:57.790065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:34.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381306+0000","last_change":"2023-10-16T09:44:56.807900+0000","last_active":"2023-10-16T09:44:58.381306+0000","last_peered":"2023-10-16T09:44:58.381306+0000","last_clean":"2023-10-16T09:44:58.381306+0000","last_became_active":"2023-10-16T09:44:56.807244+0000","last_became_peered":"2023-10-16T09:44:56.807244+0000","last_unstale":"2023-10-16T09:44:58.381306+0000","last_undegraded":"2023-10-16T09:44:58.381306+0000","last_fullsized":"2023-10-16T09:44:58.381306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:43:47.651626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789871+0000","last_change":"2023-10-16T09:44:56.810090+0000","last_active":"2023-10-16T09:44:57.789871+0000","last_peered":"2023-10-16T09:44:57.789871+0000","last_clean":"2023-10-16T09:44:57.789871+0000","last_became_active":"2023-10-16T09:44:56.809820+0000","last_became_peered":"2023-10-16T09:44:56.809820+0000","last_unstale":"2023-10-16T09:44:57.789871+0000","last_undegraded":"2023-10-16T09:44:57.789871+0000","last_fullsized":"2023-10-16T09:44:57.789871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:27.237857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788580+0000","last_change":"2023-10-16T09:44:56.814396+0000","last_active":"2023-10-16T09:44:57.788580+0000","last_peered":"2023-10-16T09:44:57.788580+0000","last_clean":"2023-10-16T09:44:57.788580+0000","last_became_active":"2023-10-16T09:44:56.814106+0000","last_became_peered":"2023-10-16T09:44:56.814106+0000","last_unstale":"2023-10-16T09:44:57.788580+0000","last_undegraded":"2023-10-16T09:44:57.788580+0000","last_fullsized":"2023-10-16T09:44:57.788580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:38:27.019733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722536+0000","last_change":"2023-10-16T09:44:56.810408+0000","last_active":"2023-10-16T09:44:58.722536+0000","last_peered":"2023-10-16T09:44:58.722536+0000","last_clean":"2023-10-16T09:44:58.722536+0000","last_became_active":"2023-10-16T09:44:56.810068+0000","last_became_peered":"2023-10-16T09:44:56.810068+0000","last_unstale":"2023-10-16T09:44:58.722536+0000","last_undegraded":"2023-10-16T09:44:58.722536+0000","last_fullsized":"2023-10-16T09:44:58.722536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:09.120120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722082+0000","last_change":"2023-10-16T09:44:56.807818+0000","last_active":"2023-10-16T09:44:58.722082+0000","last_peered":"2023-10-16T09:44:58.722082+0000","last_clean":"2023-10-16T09:44:58.722082+0000","last_became_active":"2023-10-16T09:44:56.807435+0000","last_became_peered":"2023-10-16T09:44:56.807435+0000","last_unstale":"2023-10-16T09:44:58.722082+0000","last_undegraded":"2023-10-16T09:44:58.722082+0000","last_fullsized":"2023-10-16T09:44:58.722082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:17.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189514+0000","last_change":"2023-10-16T09:44:56.803007+0000","last_active":"2023-10-16T09:44:58.189514+0000","last_peered":"2023-10-16T09:44:58.189514+0000","last_clean":"2023-10-16T09:44:58.189514+0000","last_became_active":"2023-10-16T09:44:56.802499+0000","last_became_peered":"2023-10-16T09:44:56.802499+0000","last_unstale":"2023-10-16T09:44:58.189514+0000","last_undegraded":"2023-10-16T09:44:58.189514+0000","last_fullsized":"2023-10-16T09:44:58.189514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:19.270890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381558+0000","last_change":"2023-10-16T09:44:56.818068+0000","last_active":"2023-10-16T09:44:58.381558+0000","last_peered":"2023-10-16T09:44:58.381558+0000","last_clean":"2023-10-16T09:44:58.381558+0000","last_became_active":"2023-10-16T09:44:56.817799+0000","last_became_peered":"2023-10-16T09:44:56.817799+0000","last_unstale":"2023-10-16T09:44:58.381558+0000","last_undegraded":"2023-10-16T09:44:58.381558+0000","last_fullsized":"2023-10-16T09:44:58.381558+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:17.482238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722465+0000","last_change":"2023-10-16T09:44:56.803456+0000","last_active":"2023-10-16T09:44:58.722465+0000","last_peered":"2023-10-16T09:44:58.722465+0000","last_clean":"2023-10-16T09:44:58.722465+0000","last_became_active":"2023-10-16T09:44:56.803140+0000","last_became_peered":"2023-10-16T09:44:56.803140+0000","last_unstale":"2023-10-16T09:44:58.722465+0000","last_undegraded":"2023-10-16T09:44:58.722465+0000","last_fullsized":"2023-10-16T09:44:58.722465+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:14.058492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789841+0000","last_change":"2023-10-16T09:44:56.818281+0000","last_active":"2023-10-16T09:44:57.789841+0000","last_peered":"2023-10-16T09:44:57.789841+0000","last_clean":"2023-10-16T09:44:57.789841+0000","last_became_active":"2023-10-16T09:44:56.817903+0000","last_became_peered":"2023-10-16T09:44:56.817903+0000","last_unstale":"2023-10-16T09:44:57.789841+0000","last_undegraded":"2023-10-16T09:44:57.789841+0000","last_fullsized":"2023-10-16T09:44:57.789841+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:57.785495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789780+0000","last_change":"2023-10-16T09:44:56.806961+0000","last_active":"2023-10-16T09:44:57.789780+0000","last_peered":"2023-10-16T09:44:57.789780+0000","last_clean":"2023-10-16T09:44:57.789780+0000","last_became_active":"2023-10-16T09:44:56.806699+0000","last_became_peered":"2023-10-16T09:44:56.806699+0000","last_unstale":"2023-10-16T09:44:57.789780+0000","last_undegraded":"2023-10-16T09:44:57.789780+0000","last_fullsized":"2023-10-16T09:44:57.789780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:12:28.672873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788683+0000","last_change":"2023-10-16T09:44:56.814342+0000","last_active":"2023-10-16T09:44:57.788683+0000","last_peered":"2023-10-16T09:44:57.788683+0000","last_clean":"2023-10-16T09:44:57.788683+0000","last_became_active":"2023-10-16T09:44:56.813974+0000","last_became_peered":"2023-10-16T09:44:56.813974+0000","last_unstale":"2023-10-16T09:44:57.788683+0000","last_undegraded":"2023-10-16T09:44:57.788683+0000","last_fullsized":"2023-10-16T09:44:57.788683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:32.192715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789661+0000","last_change":"2023-10-16T09:44:56.804490+0000","last_active":"2023-10-16T09:44:57.789661+0000","last_peered":"2023-10-16T09:44:57.789661+0000","last_clean":"2023-10-16T09:44:57.789661+0000","last_became_active":"2023-10-16T09:44:56.804185+0000","last_became_peered":"2023-10-16T09:44:56.804185+0000","last_unstale":"2023-10-16T09:44:57.789661+0000","last_undegraded":"2023-10-16T09:44:57.789661+0000","last_fullsized":"2023-10-16T09:44:57.789661+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:14.058492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722557+0000","last_change":"2023-10-16T09:44:56.817373+0000","last_active":"2023-10-16T09:44:58.722557+0000","last_peered":"2023-10-16T09:44:58.722557+0000","last_clean":"2023-10-16T09:44:58.722557+0000","last_became_active":"2023-10-16T09:44:56.817085+0000","last_became_peered":"2023-10-16T09:44:56.817085+0000","last_unstale":"2023-10-16T09:44:58.722557+0000","last_undegraded":"2023-10-16T09:44:58.722557+0000","last_fullsized":"2023-10-16T09:44:58.722557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:21.710655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722658+0000","last_change":"2023-10-16T09:44:56.805810+0000","last_active":"2023-10-16T09:44:58.722658+0000","last_peered":"2023-10-16T09:44:58.722658+0000","last_clean":"2023-10-16T09:44:58.722658+0000","last_became_active":"2023-10-16T09:44:56.805492+0000","last_became_peered":"2023-10-16T09:44:56.805492+0000","last_unstale":"2023-10-16T09:44:58.722658+0000","last_undegraded":"2023-10-16T09:44:58.722658+0000","last_fullsized":"2023-10-16T09:44:58.722658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:42.010506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789140+0000","last_change":"2023-10-16T09:44:56.816595+0000","last_active":"2023-10-16T09:44:57.789140+0000","last_peered":"2023-10-16T09:44:57.789140+0000","last_clean":"2023-10-16T09:44:57.789140+0000","last_became_active":"2023-10-16T09:44:56.814296+0000","last_became_peered":"2023-10-16T09:44:56.814296+0000","last_unstale":"2023-10-16T09:44:57.789140+0000","last_undegraded":"2023-10-16T09:44:57.789140+0000","last_fullsized":"2023-10-16T09:44:57.789140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:38.501059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788719+0000","last_change":"2023-10-16T09:44:56.811146+0000","last_active":"2023-10-16T09:44:57.788719+0000","last_peered":"2023-10-16T09:44:57.788719+0000","last_clean":"2023-10-16T09:44:57.788719+0000","last_became_active":"2023-10-16T09:44:56.810894+0000","last_became_peered":"2023-10-16T09:44:56.810894+0000","last_unstale":"2023-10-16T09:44:57.788719+0000","last_undegraded":"2023-10-16T09:44:57.788719+0000","last_fullsized":"2023-10-16T09:44:57.788719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:07.823284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789867+0000","last_change":"2023-10-16T09:44:56.811436+0000","last_active":"2023-10-16T09:44:57.789867+0000","last_peered":"2023-10-16T09:44:57.789867+0000","last_clean":"2023-10-16T09:44:57.789867+0000","last_became_active":"2023-10-16T09:44:56.811154+0000","last_became_peered":"2023-10-16T09:44:56.811154+0000","last_unstale":"2023-10-16T09:44:57.789867+0000","last_undegraded":"2023-10-16T09:44:57.789867+0000","last_fullsized":"2023-10-16T09:44:57.789867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:09.746836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189112+0000","last_change":"2023-10-16T09:44:56.807191+0000","last_active":"2023-10-16T09:44:58.189112+0000","last_peered":"2023-10-16T09:44:58.189112+0000","last_clean":"2023-10-16T09:44:58.189112+0000","last_became_active":"2023-10-16T09:44:56.806875+0000","last_became_peered":"2023-10-16T09:44:56.806875+0000","last_unstale":"2023-10-16T09:44:58.189112+0000","last_undegraded":"2023-10-16T09:44:58.189112+0000","last_fullsized":"2023-10-16T09:44:58.189112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:20.173958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189082+0000","last_change":"2023-10-16T09:44:56.818182+0000","last_active":"2023-10-16T09:44:58.189082+0000","last_peered":"2023-10-16T09:44:58.189082+0000","last_clean":"2023-10-16T09:44:58.189082+0000","last_became_active":"2023-10-16T09:44:56.817890+0000","last_became_peered":"2023-10-16T09:44:56.817890+0000","last_unstale":"2023-10-16T09:44:58.189082+0000","last_undegraded":"2023-10-16T09:44:58.189082+0000","last_fullsized":"2023-10-16T09:44:58.189082+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:29.923256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428765+0000","last_change":"2023-10-16T09:44:56.812353+0000","last_active":"2023-10-16T09:44:58.428765+0000","last_peered":"2023-10-16T09:44:58.428765+0000","last_clean":"2023-10-16T09:44:58.428765+0000","last_became_active":"2023-10-16T09:44:56.811994+0000","last_became_peered":"2023-10-16T09:44:56.811994+0000","last_unstale":"2023-10-16T09:44:58.428765+0000","last_undegraded":"2023-10-16T09:44:58.428765+0000","last_fullsized":"2023-10-16T09:44:58.428765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:25.271373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789470+0000","last_change":"2023-10-16T09:44:56.807324+0000","last_active":"2023-10-16T09:44:57.789470+0000","last_peered":"2023-10-16T09:44:57.789470+0000","last_clean":"2023-10-16T09:44:57.789470+0000","last_became_active":"2023-10-16T09:44:56.807041+0000","last_became_peered":"2023-10-16T09:44:56.807041+0000","last_unstale":"2023-10-16T09:44:57.789470+0000","last_undegraded":"2023-10-16T09:44:57.789470+0000","last_fullsized":"2023-10-16T09:44:57.789470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:02.475372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381620+0000","last_change":"2023-10-16T09:44:56.805136+0000","last_active":"2023-10-16T09:44:58.381620+0000","last_peered":"2023-10-16T09:44:58.381620+0000","last_clean":"2023-10-16T09:44:58.381620+0000","last_became_active":"2023-10-16T09:44:56.804879+0000","last_became_peered":"2023-10-16T09:44:56.804879+0000","last_unstale":"2023-10-16T09:44:58.381620+0000","last_undegraded":"2023-10-16T09:44:58.381620+0000","last_fullsized":"2023-10-16T09:44:58.381620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:32.410416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789146+0000","last_change":"2023-10-16T09:44:56.816589+0000","last_active":"2023-10-16T09:44:57.789146+0000","last_peered":"2023-10-16T09:44:57.789146+0000","last_clean":"2023-10-16T09:44:57.789146+0000","last_became_active":"2023-10-16T09:44:56.814006+0000","last_became_peered":"2023-10-16T09:44:56.814006+0000","last_unstale":"2023-10-16T09:44:57.789146+0000","last_undegraded":"2023-10-16T09:44:57.789146+0000","last_fullsized":"2023-10-16T09:44:57.789146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:29.557059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381639+0000","last_change":"2023-10-16T09:44:56.808137+0000","last_active":"2023-10-16T09:44:58.381639+0000","last_peered":"2023-10-16T09:44:58.381639+0000","last_clean":"2023-10-16T09:44:58.381639+0000","last_became_active":"2023-10-16T09:44:56.807685+0000","last_became_peered":"2023-10-16T09:44:56.807685+0000","last_unstale":"2023-10-16T09:44:58.381639+0000","last_undegraded":"2023-10-16T09:44:58.381639+0000","last_fullsized":"2023-10-16T09:44:58.381639+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:02.475372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789885+0000","last_change":"2023-10-16T09:44:56.804238+0000","last_active":"2023-10-16T09:44:57.789885+0000","last_peered":"2023-10-16T09:44:57.789885+0000","last_clean":"2023-10-16T09:44:57.789885+0000","last_became_active":"2023-10-16T09:44:56.803899+0000","last_became_peered":"2023-10-16T09:44:56.803899+0000","last_unstale":"2023-10-16T09:44:57.789885+0000","last_undegraded":"2023-10-16T09:44:57.789885+0000","last_fullsized":"2023-10-16T09:44:57.789885+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:21.710655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189141+0000","last_change":"2023-10-16T09:44:56.818972+0000","last_active":"2023-10-16T09:44:58.189141+0000","last_peered":"2023-10-16T09:44:58.189141+0000","last_clean":"2023-10-16T09:44:58.189141+0000","last_became_active":"2023-10-16T09:44:56.818641+0000","last_became_peered":"2023-10-16T09:44:56.818641+0000","last_unstale":"2023-10-16T09:44:58.189141+0000","last_undegraded":"2023-10-16T09:44:58.189141+0000","last_fullsized":"2023-10-16T09:44:58.189141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:52.793141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722281+0000","last_change":"2023-10-16T09:44:56.799942+0000","last_active":"2023-10-16T09:44:58.722281+0000","last_peered":"2023-10-16T09:44:58.722281+0000","last_clean":"2023-10-16T09:44:58.722281+0000","last_became_active":"2023-10-16T09:44:56.799652+0000","last_became_peered":"2023-10-16T09:44:56.799652+0000","last_unstale":"2023-10-16T09:44:58.722281+0000","last_undegraded":"2023-10-16T09:44:58.722281+0000","last_fullsized":"2023-10-16T09:44:58.722281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:25.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812676+0000","last_change":"2023-10-16T09:44:56.809197+0000","last_active":"2023-10-16T09:44:57.812676+0000","last_peered":"2023-10-16T09:44:57.812676+0000","last_clean":"2023-10-16T09:44:57.812676+0000","last_became_active":"2023-10-16T09:44:56.808905+0000","last_became_peered":"2023-10-16T09:44:56.808905+0000","last_unstale":"2023-10-16T09:44:57.812676+0000","last_undegraded":"2023-10-16T09:44:57.812676+0000","last_fullsized":"2023-10-16T09:44:57.812676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:31.365459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722655+0000","last_change":"2023-10-16T09:44:56.806839+0000","last_active":"2023-10-16T09:44:58.722655+0000","last_peered":"2023-10-16T09:44:58.722655+0000","last_clean":"2023-10-16T09:44:58.722655+0000","last_became_active":"2023-10-16T09:44:56.806531+0000","last_became_peered":"2023-10-16T09:44:56.806531+0000","last_unstale":"2023-10-16T09:44:58.722655+0000","last_undegraded":"2023-10-16T09:44:58.722655+0000","last_fullsized":"2023-10-16T09:44:58.722655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:34.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428936+0000","last_change":"2023-10-16T09:44:56.812485+0000","last_active":"2023-10-16T09:44:58.428936+0000","last_peered":"2023-10-16T09:44:58.428936+0000","last_clean":"2023-10-16T09:44:58.428936+0000","last_became_active":"2023-10-16T09:44:56.812061+0000","last_became_peered":"2023-10-16T09:44:56.812061+0000","last_unstale":"2023-10-16T09:44:58.428936+0000","last_undegraded":"2023-10-16T09:44:58.428936+0000","last_fullsized":"2023-10-16T09:44:58.428936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:47.916598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812812+0000","last_change":"2023-10-16T09:44:55.800280+0000","last_active":"2023-10-16T09:44:57.812812+0000","last_peered":"2023-10-16T09:44:57.812812+0000","last_clean":"2023-10-16T09:44:57.812812+0000","last_became_active":"2023-10-16T09:44:55.799942+0000","last_became_peered":"2023-10-16T09:44:55.799942+0000","last_unstale":"2023-10-16T09:44:57.812812+0000","last_undegraded":"2023-10-16T09:44:57.812812+0000","last_fullsized":"2023-10-16T09:44:57.812812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:18:47.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812733+0000","last_change":"2023-10-16T09:44:56.804130+0000","last_active":"2023-10-16T09:44:57.812733+0000","last_peered":"2023-10-16T09:44:57.812733+0000","last_clean":"2023-10-16T09:44:57.812733+0000","last_became_active":"2023-10-16T09:44:56.803840+0000","last_became_peered":"2023-10-16T09:44:56.803840+0000","last_unstale":"2023-10-16T09:44:57.812733+0000","last_undegraded":"2023-10-16T09:44:57.812733+0000","last_fullsized":"2023-10-16T09:44:57.812733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:25.271373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428878+0000","last_change":"2023-10-16T09:44:55.804801+0000","last_active":"2023-10-16T09:44:58.428878+0000","last_peered":"2023-10-16T09:44:58.428878+0000","last_clean":"2023-10-16T09:44:58.428878+0000","last_became_active":"2023-10-16T09:44:55.804524+0000","last_became_peered":"2023-10-16T09:44:55.804524+0000","last_unstale":"2023-10-16T09:44:58.428878+0000","last_undegraded":"2023-10-16T09:44:58.428878+0000","last_fullsized":"2023-10-16T09:44:58.428878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:55.256612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189174+0000","last_change":"2023-10-16T09:44:56.818522+0000","last_active":"2023-10-16T09:44:58.189174+0000","last_peered":"2023-10-16T09:44:58.189174+0000","last_clean":"2023-10-16T09:44:58.189174+0000","last_became_active":"2023-10-16T09:44:56.818278+0000","last_became_peered":"2023-10-16T09:44:56.818278+0000","last_unstale":"2023-10-16T09:44:58.189174+0000","last_undegraded":"2023-10-16T09:44:58.189174+0000","last_fullsized":"2023-10-16T09:44:58.189174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:21.572345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.820330+0000","last_change":"2023-10-16T09:44:55.801884+0000","last_active":"2023-10-16T09:44:56.820330+0000","last_peered":"2023-10-16T09:44:56.820330+0000","last_clean":"2023-10-16T09:44:56.820330+0000","last_became_active":"2023-10-16T09:44:55.801654+0000","last_became_peered":"2023-10-16T09:44:55.801654+0000","last_unstale":"2023-10-16T09:44:56.820330+0000","last_undegraded":"2023-10-16T09:44:56.820330+0000","last_fullsized":"2023-10-16T09:44:56.820330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:41.008581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381593+0000","last_change":"2023-10-16T09:44:56.807961+0000","last_active":"2023-10-16T09:44:58.381593+0000","last_peered":"2023-10-16T09:44:58.381593+0000","last_clean":"2023-10-16T09:44:58.381593+0000","last_became_active":"2023-10-16T09:44:56.807320+0000","last_became_peered":"2023-10-16T09:44:56.807320+0000","last_unstale":"2023-10-16T09:44:58.381593+0000","last_undegraded":"2023-10-16T09:44:58.381593+0000","last_fullsized":"2023-10-16T09:44:58.381593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:25.773061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789903+0000","last_change":"2023-10-16T09:44:55.803670+0000","last_active":"2023-10-16T09:44:57.789903+0000","last_peered":"2023-10-16T09:44:57.789903+0000","last_clean":"2023-10-16T09:44:57.789903+0000","last_became_active":"2023-10-16T09:44:55.803243+0000","last_became_peered":"2023-10-16T09:44:55.803243+0000","last_unstale":"2023-10-16T09:44:57.789903+0000","last_undegraded":"2023-10-16T09:44:57.789903+0000","last_fullsized":"2023-10-16T09:44:57.789903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:38:59.831210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722334+0000","last_change":"2023-10-16T09:44:56.801591+0000","last_active":"2023-10-16T09:44:58.722334+0000","last_peered":"2023-10-16T09:44:58.722334+0000","last_clean":"2023-10-16T09:44:58.722334+0000","last_became_active":"2023-10-16T09:44:56.801303+0000","last_became_peered":"2023-10-16T09:44:56.801303+0000","last_unstale":"2023-10-16T09:44:58.722334+0000","last_undegraded":"2023-10-16T09:44:58.722334+0000","last_fullsized":"2023-10-16T09:44:58.722334+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:02.475372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.818370+0000","last_change":"2023-10-16T09:44:55.803450+0000","last_active":"2023-10-16T09:44:56.818370+0000","last_peered":"2023-10-16T09:44:56.818370+0000","last_clean":"2023-10-16T09:44:56.818370+0000","last_became_active":"2023-10-16T09:44:55.803110+0000","last_became_peered":"2023-10-16T09:44:55.803110+0000","last_unstale":"2023-10-16T09:44:56.818370+0000","last_undegraded":"2023-10-16T09:44:56.818370+0000","last_fullsized":"2023-10-16T09:44:56.818370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:01.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789788+0000","last_change":"2023-10-16T09:44:56.805027+0000","last_active":"2023-10-16T09:44:57.789788+0000","last_peered":"2023-10-16T09:44:57.789788+0000","last_clean":"2023-10-16T09:44:57.789788+0000","last_became_active":"2023-10-16T09:44:56.804715+0000","last_became_peered":"2023-10-16T09:44:56.804715+0000","last_unstale":"2023-10-16T09:44:57.789788+0000","last_undegraded":"2023-10-16T09:44:57.789788+0000","last_fullsized":"2023-10-16T09:44:57.789788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:04.066331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.817621+0000","last_change":"2023-10-16T09:44:55.803751+0000","last_active":"2023-10-16T09:44:56.817621+0000","last_peered":"2023-10-16T09:44:56.817621+0000","last_clean":"2023-10-16T09:44:56.817621+0000","last_became_active":"2023-10-16T09:44:55.803491+0000","last_became_peered":"2023-10-16T09:44:55.803491+0000","last_unstale":"2023-10-16T09:44:56.817621+0000","last_undegraded":"2023-10-16T09:44:56.817621+0000","last_fullsized":"2023-10-16T09:44:56.817621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:13:33.357992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189019+0000","last_change":"2023-10-16T09:44:56.809642+0000","last_active":"2023-10-16T09:44:58.189019+0000","last_peered":"2023-10-16T09:44:58.189019+0000","last_clean":"2023-10-16T09:44:58.189019+0000","last_became_active":"2023-10-16T09:44:56.809396+0000","last_became_peered":"2023-10-16T09:44:56.809396+0000","last_unstale":"2023-10-16T09:44:58.189019+0000","last_undegraded":"2023-10-16T09:44:58.189019+0000","last_fullsized":"2023-10-16T09:44:58.189019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:54.581883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.803972+0000","last_change":"2023-10-16T09:44:55.805779+0000","last_active":"2023-10-16T09:44:56.803972+0000","last_peered":"2023-10-16T09:44:56.803972+0000","last_clean":"2023-10-16T09:44:56.803972+0000","last_became_active":"2023-10-16T09:44:55.805466+0000","last_became_peered":"2023-10-16T09:44:55.805466+0000","last_unstale":"2023-10-16T09:44:56.803972+0000","last_undegraded":"2023-10-16T09:44:56.803972+0000","last_fullsized":"2023-10-16T09:44:56.803972+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:03.064406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381249+0000","last_change":"2023-10-16T09:44:56.797572+0000","last_active":"2023-10-16T09:44:58.381249+0000","last_peered":"2023-10-16T09:44:58.381249+0000","last_clean":"2023-10-16T09:44:58.381249+0000","last_became_active":"2023-10-16T09:44:56.797084+0000","last_became_peered":"2023-10-16T09:44:56.797084+0000","last_unstale":"2023-10-16T09:44:58.381249+0000","last_undegraded":"2023-10-16T09:44:58.381249+0000","last_fullsized":"2023-10-16T09:44:58.381249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:08:30.600822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.799894+0000","last_change":"2023-10-16T09:44:55.803251+0000","last_active":"2023-10-16T09:44:56.799894+0000","last_peered":"2023-10-16T09:44:56.799894+0000","last_clean":"2023-10-16T09:44:56.799894+0000","last_became_active":"2023-10-16T09:44:55.802925+0000","last_became_peered":"2023-10-16T09:44:55.802925+0000","last_unstale":"2023-10-16T09:44:56.799894+0000","last_undegraded":"2023-10-16T09:44:56.799894+0000","last_fullsized":"2023-10-16T09:44:56.799894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:26.340087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790027+0000","last_change":"2023-10-16T09:44:56.818058+0000","last_active":"2023-10-16T09:44:57.790027+0000","last_peered":"2023-10-16T09:44:57.790027+0000","last_clean":"2023-10-16T09:44:57.790027+0000","last_became_active":"2023-10-16T09:44:56.817736+0000","last_became_peered":"2023-10-16T09:44:56.817736+0000","last_unstale":"2023-10-16T09:44:57.790027+0000","last_undegraded":"2023-10-16T09:44:57.790027+0000","last_fullsized":"2023-10-16T09:44:57.790027+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:42.010506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.804241+0000","last_change":"2023-10-16T09:44:55.806160+0000","last_active":"2023-10-16T09:44:56.804241+0000","last_peered":"2023-10-16T09:44:56.804241+0000","last_clean":"2023-10-16T09:44:56.804241+0000","last_became_active":"2023-10-16T09:44:55.805818+0000","last_became_peered":"2023-10-16T09:44:55.805818+0000","last_unstale":"2023-10-16T09:44:56.804241+0000","last_undegraded":"2023-10-16T09:44:56.804241+0000","last_fullsized":"2023-10-16T09:44:56.804241+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:08.118195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788923+0000","last_change":"2023-10-16T09:44:56.810480+0000","last_active":"2023-10-16T09:44:57.788923+0000","last_peered":"2023-10-16T09:44:57.788923+0000","last_clean":"2023-10-16T09:44:57.788923+0000","last_became_active":"2023-10-16T09:44:56.810198+0000","last_became_peered":"2023-10-16T09:44:56.810198+0000","last_unstale":"2023-10-16T09:44:57.788923+0000","last_undegraded":"2023-10-16T09:44:57.788923+0000","last_fullsized":"2023-10-16T09:44:57.788923+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:46.160206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428931+0000","last_change":"2023-10-16T09:44:55.803506+0000","last_active":"2023-10-16T09:44:58.428931+0000","last_peered":"2023-10-16T09:44:58.428931+0000","last_clean":"2023-10-16T09:44:58.428931+0000","last_became_active":"2023-10-16T09:44:55.803232+0000","last_became_peered":"2023-10-16T09:44:55.803232+0000","last_unstale":"2023-10-16T09:44:58.428931+0000","last_undegraded":"2023-10-16T09:44:58.428931+0000","last_fullsized":"2023-10-16T09:44:58.428931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:09.850844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788718+0000","last_change":"2023-10-16T09:44:56.816232+0000","last_active":"2023-10-16T09:44:57.788718+0000","last_peered":"2023-10-16T09:44:57.788718+0000","last_clean":"2023-10-16T09:44:57.788718+0000","last_became_active":"2023-10-16T09:44:56.815980+0000","last_became_peered":"2023-10-16T09:44:56.815980+0000","last_unstale":"2023-10-16T09:44:57.788718+0000","last_undegraded":"2023-10-16T09:44:57.788718+0000","last_fullsized":"2023-10-16T09:44:57.788718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:37.765122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.811802+0000","last_change":"2023-10-16T09:44:55.797411+0000","last_active":"2023-10-16T09:44:56.811802+0000","last_peered":"2023-10-16T09:44:56.811802+0000","last_clean":"2023-10-16T09:44:56.811802+0000","last_became_active":"2023-10-16T09:44:55.797121+0000","last_became_peered":"2023-10-16T09:44:55.797121+0000","last_unstale":"2023-10-16T09:44:56.811802+0000","last_undegraded":"2023-10-16T09:44:56.811802+0000","last_fullsized":"2023-10-16T09:44:56.811802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:24.771136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789997+0000","last_change":"2023-10-16T09:44:56.799562+0000","last_active":"2023-10-16T09:44:57.789997+0000","last_peered":"2023-10-16T09:44:57.789997+0000","last_clean":"2023-10-16T09:44:57.789997+0000","last_became_active":"2023-10-16T09:44:56.798274+0000","last_became_peered":"2023-10-16T09:44:56.798274+0000","last_unstale":"2023-10-16T09:44:57.789997+0000","last_undegraded":"2023-10-16T09:44:57.789997+0000","last_fullsized":"2023-10-16T09:44:57.789997+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:09.120120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.824354+0000","last_change":"2023-10-16T09:44:55.804174+0000","last_active":"2023-10-16T09:44:56.824354+0000","last_peered":"2023-10-16T09:44:56.824354+0000","last_clean":"2023-10-16T09:44:56.824354+0000","last_became_active":"2023-10-16T09:44:55.803860+0000","last_became_peered":"2023-10-16T09:44:55.803860+0000","last_unstale":"2023-10-16T09:44:56.824354+0000","last_undegraded":"2023-10-16T09:44:56.824354+0000","last_fullsized":"2023-10-16T09:44:56.824354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:56.783570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381258+0000","last_change":"2023-10-16T09:44:56.797512+0000","last_active":"2023-10-16T09:44:58.381258+0000","last_peered":"2023-10-16T09:44:58.381258+0000","last_clean":"2023-10-16T09:44:58.381258+0000","last_became_active":"2023-10-16T09:44:56.797185+0000","last_became_peered":"2023-10-16T09:44:56.797185+0000","last_unstale":"2023-10-16T09:44:58.381258+0000","last_undegraded":"2023-10-16T09:44:58.381258+0000","last_fullsized":"2023-10-16T09:44:58.381258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:40.280433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.811314+0000","last_change":"2023-10-16T09:44:55.802245+0000","last_active":"2023-10-16T09:44:56.811314+0000","last_peered":"2023-10-16T09:44:56.811314+0000","last_clean":"2023-10-16T09:44:56.811314+0000","last_became_active":"2023-10-16T09:44:55.801952+0000","last_became_peered":"2023-10-16T09:44:55.801952+0000","last_unstale":"2023-10-16T09:44:56.811314+0000","last_undegraded":"2023-10-16T09:44:56.811314+0000","last_fullsized":"2023-10-16T09:44:56.811314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:19:57.048737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788957+0000","last_change":"2023-10-16T09:44:56.809135+0000","last_active":"2023-10-16T09:44:57.788957+0000","last_peered":"2023-10-16T09:44:57.788957+0000","last_clean":"2023-10-16T09:44:57.788957+0000","last_became_active":"2023-10-16T09:44:56.808859+0000","last_became_peered":"2023-10-16T09:44:56.808859+0000","last_unstale":"2023-10-16T09:44:57.788957+0000","last_undegraded":"2023-10-16T09:44:57.788957+0000","last_fullsized":"2023-10-16T09:44:57.788957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:39.281794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.790319+0000","last_change":"2023-10-16T09:44:55.795060+0000","last_active":"2023-10-16T09:44:56.790319+0000","last_peered":"2023-10-16T09:44:56.790319+0000","last_clean":"2023-10-16T09:44:56.790319+0000","last_became_active":"2023-10-16T09:44:55.794776+0000","last_became_peered":"2023-10-16T09:44:55.794776+0000","last_unstale":"2023-10-16T09:44:56.790319+0000","last_undegraded":"2023-10-16T09:44:56.790319+0000","last_fullsized":"2023-10-16T09:44:56.790319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:08.744911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722494+0000","last_change":"2023-10-16T09:44:56.807538+0000","last_active":"2023-10-16T09:44:58.722494+0000","last_peered":"2023-10-16T09:44:58.722494+0000","last_clean":"2023-10-16T09:44:58.722494+0000","last_became_active":"2023-10-16T09:44:56.806706+0000","last_became_peered":"2023-10-16T09:44:56.806706+0000","last_unstale":"2023-10-16T09:44:58.722494+0000","last_undegraded":"2023-10-16T09:44:58.722494+0000","last_fullsized":"2023-10-16T09:44:58.722494+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:04.066331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.804178+0000","last_change":"2023-10-16T09:44:55.803176+0000","last_active":"2023-10-16T09:44:56.804178+0000","last_peered":"2023-10-16T09:44:56.804178+0000","last_clean":"2023-10-16T09:44:56.804178+0000","last_became_active":"2023-10-16T09:44:55.802803+0000","last_became_peered":"2023-10-16T09:44:55.802803+0000","last_unstale":"2023-10-16T09:44:56.804178+0000","last_undegraded":"2023-10-16T09:44:56.804178+0000","last_fullsized":"2023-10-16T09:44:56.804178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.708730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812676+0000","last_change":"2023-10-16T09:44:56.802287+0000","last_active":"2023-10-16T09:44:57.812676+0000","last_peered":"2023-10-16T09:44:57.812676+0000","last_clean":"2023-10-16T09:44:57.812676+0000","last_became_active":"2023-10-16T09:44:56.801929+0000","last_became_peered":"2023-10-16T09:44:56.801929+0000","last_unstale":"2023-10-16T09:44:57.812676+0000","last_undegraded":"2023-10-16T09:44:57.812676+0000","last_fullsized":"2023-10-16T09:44:57.812676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:10.852769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.822603+0000","last_change":"2023-10-16T09:44:55.807648+0000","last_active":"2023-10-16T09:44:56.822603+0000","last_peered":"2023-10-16T09:44:56.822603+0000","last_clean":"2023-10-16T09:44:56.822603+0000","last_became_active":"2023-10-16T09:44:55.807375+0000","last_became_peered":"2023-10-16T09:44:55.807375+0000","last_unstale":"2023-10-16T09:44:56.822603+0000","last_undegraded":"2023-10-16T09:44:56.822603+0000","last_fullsized":"2023-10-16T09:44:56.822603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:05:01.473446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.189391+0000","last_change":"2023-10-16T09:44:56.809884+0000","last_active":"2023-10-16T09:44:58.189391+0000","last_peered":"2023-10-16T09:44:58.189391+0000","last_clean":"2023-10-16T09:44:58.189391+0000","last_became_active":"2023-10-16T09:44:56.809518+0000","last_became_peered":"2023-10-16T09:44:56.809518+0000","last_unstale":"2023-10-16T09:44:58.189391+0000","last_undegraded":"2023-10-16T09:44:58.189391+0000","last_fullsized":"2023-10-16T09:44:58.189391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:39.561624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.809991+0000","last_change":"2023-10-16T09:44:55.803515+0000","last_active":"2023-10-16T09:44:56.809991+0000","last_peered":"2023-10-16T09:44:56.809991+0000","last_clean":"2023-10-16T09:44:56.809991+0000","last_became_active":"2023-10-16T09:44:55.803225+0000","last_became_peered":"2023-10-16T09:44:55.803225+0000","last_unstale":"2023-10-16T09:44:56.809991+0000","last_undegraded":"2023-10-16T09:44:56.809991+0000","last_fullsized":"2023-10-16T09:44:56.809991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:24.269448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790139+0000","last_change":"2023-10-16T09:44:56.806417+0000","last_active":"2023-10-16T09:44:57.790139+0000","last_peered":"2023-10-16T09:44:57.790139+0000","last_clean":"2023-10-16T09:44:57.790139+0000","last_became_active":"2023-10-16T09:44:56.805632+0000","last_became_peered":"2023-10-16T09:44:56.805632+0000","last_unstale":"2023-10-16T09:44:57.790139+0000","last_undegraded":"2023-10-16T09:44:57.790139+0000","last_fullsized":"2023-10-16T09:44:57.790139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:32.192715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.809364+0000","last_change":"2023-10-16T09:44:55.803364+0000","last_active":"2023-10-16T09:44:56.809364+0000","last_peered":"2023-10-16T09:44:56.809364+0000","last_clean":"2023-10-16T09:44:56.809364+0000","last_became_active":"2023-10-16T09:44:55.803063+0000","last_became_peered":"2023-10-16T09:44:55.803063+0000","last_unstale":"2023-10-16T09:44:56.809364+0000","last_undegraded":"2023-10-16T09:44:56.809364+0000","last_fullsized":"2023-10-16T09:44:56.809364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:35:08.435699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.381463+0000","last_change":"2023-10-16T09:44:56.807966+0000","last_active":"2023-10-16T09:44:58.381463+0000","last_peered":"2023-10-16T09:44:58.381463+0000","last_clean":"2023-10-16T09:44:58.381463+0000","last_became_active":"2023-10-16T09:44:56.807616+0000","last_became_peered":"2023-10-16T09:44:56.807616+0000","last_unstale":"2023-10-16T09:44:58.381463+0000","last_undegraded":"2023-10-16T09:44:58.381463+0000","last_fullsized":"2023-10-16T09:44:58.381463+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:51.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.808673+0000","last_change":"2023-10-16T09:44:55.805347+0000","last_active":"2023-10-16T09:44:56.808673+0000","last_peered":"2023-10-16T09:44:56.808673+0000","last_clean":"2023-10-16T09:44:56.808673+0000","last_became_active":"2023-10-16T09:44:55.805054+0000","last_became_peered":"2023-10-16T09:44:55.805054+0000","last_unstale":"2023-10-16T09:44:56.808673+0000","last_undegraded":"2023-10-16T09:44:56.808673+0000","last_fullsized":"2023-10-16T09:44:56.808673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:27.087072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789833+0000","last_change":"2023-10-16T09:44:56.811326+0000","last_active":"2023-10-16T09:44:57.789833+0000","last_peered":"2023-10-16T09:44:57.789833+0000","last_clean":"2023-10-16T09:44:57.789833+0000","last_became_active":"2023-10-16T09:44:56.810995+0000","last_became_peered":"2023-10-16T09:44:56.810995+0000","last_unstale":"2023-10-16T09:44:57.789833+0000","last_undegraded":"2023-10-16T09:44:57.789833+0000","last_fullsized":"2023-10-16T09:44:57.789833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:06.524695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.802551+0000","last_change":"2023-10-16T09:44:55.803440+0000","last_active":"2023-10-16T09:44:56.802551+0000","last_peered":"2023-10-16T09:44:56.802551+0000","last_clean":"2023-10-16T09:44:56.802551+0000","last_became_active":"2023-10-16T09:44:55.803141+0000","last_became_peered":"2023-10-16T09:44:55.803141+0000","last_unstale":"2023-10-16T09:44:56.802551+0000","last_undegraded":"2023-10-16T09:44:56.802551+0000","last_fullsized":"2023-10-16T09:44:56.802551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:13.236917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789741+0000","last_change":"2023-10-16T09:44:56.806094+0000","last_active":"2023-10-16T09:44:57.789741+0000","last_peered":"2023-10-16T09:44:57.789741+0000","last_clean":"2023-10-16T09:44:57.789741+0000","last_became_active":"2023-10-16T09:44:56.805871+0000","last_became_peered":"2023-10-16T09:44:56.805871+0000","last_unstale":"2023-10-16T09:44:57.789741+0000","last_undegraded":"2023-10-16T09:44:57.789741+0000","last_fullsized":"2023-10-16T09:44:57.789741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:45.961536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428856+0000","last_change":"2023-10-16T09:44:55.810953+0000","last_active":"2023-10-16T09:44:58.428856+0000","last_peered":"2023-10-16T09:44:58.428856+0000","last_clean":"2023-10-16T09:44:58.428856+0000","last_became_active":"2023-10-16T09:44:55.810677+0000","last_became_peered":"2023-10-16T09:44:55.810677+0000","last_unstale":"2023-10-16T09:44:58.428856+0000","last_undegraded":"2023-10-16T09:44:58.428856+0000","last_fullsized":"2023-10-16T09:44:58.428856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:12:45.052666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789695+0000","last_change":"2023-10-16T09:44:56.805014+0000","last_active":"2023-10-16T09:44:57.789695+0000","last_peered":"2023-10-16T09:44:57.789695+0000","last_clean":"2023-10-16T09:44:57.789695+0000","last_became_active":"2023-10-16T09:44:56.804733+0000","last_became_peered":"2023-10-16T09:44:56.804733+0000","last_unstale":"2023-10-16T09:44:57.789695+0000","last_undegraded":"2023-10-16T09:44:57.789695+0000","last_fullsized":"2023-10-16T09:44:57.789695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:13.678959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812507+0000","last_change":"2023-10-16T09:44:55.801848+0000","last_active":"2023-10-16T09:44:57.812507+0000","last_peered":"2023-10-16T09:44:57.812507+0000","last_clean":"2023-10-16T09:44:57.812507+0000","last_became_active":"2023-10-16T09:44:55.801533+0000","last_became_peered":"2023-10-16T09:44:55.801533+0000","last_unstale":"2023-10-16T09:44:57.812507+0000","last_undegraded":"2023-10-16T09:44:57.812507+0000","last_fullsized":"2023-10-16T09:44:57.812507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:06:16.539616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789567+0000","last_change":"2023-10-16T09:44:56.806835+0000","last_active":"2023-10-16T09:44:57.789567+0000","last_peered":"2023-10-16T09:44:57.789567+0000","last_clean":"2023-10-16T09:44:57.789567+0000","last_became_active":"2023-10-16T09:44:56.806572+0000","last_became_peered":"2023-10-16T09:44:56.806572+0000","last_unstale":"2023-10-16T09:44:57.789567+0000","last_undegraded":"2023-10-16T09:44:57.789567+0000","last_fullsized":"2023-10-16T09:44:57.789567+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:58:40.004337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.799624+0000","last_change":"2023-10-16T09:44:55.801917+0000","last_active":"2023-10-16T09:44:56.799624+0000","last_peered":"2023-10-16T09:44:56.799624+0000","last_clean":"2023-10-16T09:44:56.799624+0000","last_became_active":"2023-10-16T09:44:55.801641+0000","last_became_peered":"2023-10-16T09:44:55.801641+0000","last_unstale":"2023-10-16T09:44:56.799624+0000","last_undegraded":"2023-10-16T09:44:56.799624+0000","last_fullsized":"2023-10-16T09:44:56.799624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:24.355232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722321+0000","last_change":"2023-10-16T09:44:56.803314+0000","last_active":"2023-10-16T09:44:58.722321+0000","last_peered":"2023-10-16T09:44:58.722321+0000","last_clean":"2023-10-16T09:44:58.722321+0000","last_became_active":"2023-10-16T09:44:56.803027+0000","last_became_peered":"2023-10-16T09:44:56.803027+0000","last_unstale":"2023-10-16T09:44:58.722321+0000","last_undegraded":"2023-10-16T09:44:58.722321+0000","last_fullsized":"2023-10-16T09:44:58.722321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:32.410416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.790722+0000","last_change":"2023-10-16T09:44:55.800277+0000","last_active":"2023-10-16T09:44:56.790722+0000","last_peered":"2023-10-16T09:44:56.790722+0000","last_clean":"2023-10-16T09:44:56.790722+0000","last_became_active":"2023-10-16T09:44:55.799950+0000","last_became_peered":"2023-10-16T09:44:55.799950+0000","last_unstale":"2023-10-16T09:44:56.790722+0000","last_undegraded":"2023-10-16T09:44:56.790722+0000","last_fullsized":"2023-10-16T09:44:56.790722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:50.064075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.789665+0000","last_change":"2023-10-16T09:44:56.811816+0000","last_active":"2023-10-16T09:44:57.789665+0000","last_peered":"2023-10-16T09:44:57.789665+0000","last_clean":"2023-10-16T09:44:57.789665+0000","last_became_active":"2023-10-16T09:44:56.811579+0000","last_became_peered":"2023-10-16T09:44:56.811579+0000","last_unstale":"2023-10-16T09:44:57.789665+0000","last_undegraded":"2023-10-16T09:44:57.789665+0000","last_fullsized":"2023-10-16T09:44:57.789665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:08.998408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.812098+0000","last_change":"2023-10-16T09:44:55.795185+0000","last_active":"2023-10-16T09:44:56.812098+0000","last_peered":"2023-10-16T09:44:56.812098+0000","last_clean":"2023-10-16T09:44:56.812098+0000","last_became_active":"2023-10-16T09:44:55.794880+0000","last_became_peered":"2023-10-16T09:44:55.794880+0000","last_unstale":"2023-10-16T09:44:56.812098+0000","last_undegraded":"2023-10-16T09:44:56.812098+0000","last_fullsized":"2023-10-16T09:44:56.812098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:09.850844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.790122+0000","last_change":"2023-10-16T09:44:56.805133+0000","last_active":"2023-10-16T09:44:57.790122+0000","last_peered":"2023-10-16T09:44:57.790122+0000","last_clean":"2023-10-16T09:44:57.790122+0000","last_became_active":"2023-10-16T09:44:56.798012+0000","last_became_peered":"2023-10-16T09:44:56.798012+0000","last_unstale":"2023-10-16T09:44:57.790122+0000","last_undegraded":"2023-10-16T09:44:57.790122+0000","last_fullsized":"2023-10-16T09:44:57.790122+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:38:27.019733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.430216+0000","last_change":"2023-10-16T09:44:55.802637+0000","last_active":"2023-10-16T09:44:58.430216+0000","last_peered":"2023-10-16T09:44:58.430216+0000","last_clean":"2023-10-16T09:44:58.430216+0000","last_became_active":"2023-10-16T09:44:55.802375+0000","last_became_peered":"2023-10-16T09:44:55.802375+0000","last_unstale":"2023-10-16T09:44:58.430216+0000","last_undegraded":"2023-10-16T09:44:58.430216+0000","last_fullsized":"2023-10-16T09:44:58.430216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:46:30.363534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428875+0000","last_change":"2023-10-16T09:44:56.813550+0000","last_active":"2023-10-16T09:44:58.428875+0000","last_peered":"2023-10-16T09:44:58.428875+0000","last_clean":"2023-10-16T09:44:58.428875+0000","last_became_active":"2023-10-16T09:44:56.813288+0000","last_became_peered":"2023-10-16T09:44:56.813288+0000","last_unstale":"2023-10-16T09:44:58.428875+0000","last_undegraded":"2023-10-16T09:44:58.428875+0000","last_fullsized":"2023-10-16T09:44:58.428875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:01.325830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.791773+0000","last_change":"2023-10-16T09:44:55.805218+0000","last_active":"2023-10-16T09:44:56.791773+0000","last_peered":"2023-10-16T09:44:56.791773+0000","last_clean":"2023-10-16T09:44:56.791773+0000","last_became_active":"2023-10-16T09:44:55.804922+0000","last_became_peered":"2023-10-16T09:44:55.804922+0000","last_unstale":"2023-10-16T09:44:56.791773+0000","last_undegraded":"2023-10-16T09:44:56.791773+0000","last_fullsized":"2023-10-16T09:44:56.791773+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:46.914673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812467+0000","last_change":"2023-10-16T09:44:56.805897+0000","last_active":"2023-10-16T09:44:57.812467+0000","last_peered":"2023-10-16T09:44:57.812467+0000","last_clean":"2023-10-16T09:44:57.812467+0000","last_became_active":"2023-10-16T09:44:56.805621+0000","last_became_peered":"2023-10-16T09:44:56.805621+0000","last_unstale":"2023-10-16T09:44:57.812467+0000","last_undegraded":"2023-10-16T09:44:57.812467+0000","last_fullsized":"2023-10-16T09:44:57.812467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:56.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428802+0000","last_change":"2023-10-16T09:44:55.801484+0000","last_active":"2023-10-16T09:44:58.428802+0000","last_peered":"2023-10-16T09:44:58.428802+0000","last_clean":"2023-10-16T09:44:58.428802+0000","last_became_active":"2023-10-16T09:44:55.801193+0000","last_became_peered":"2023-10-16T09:44:55.801193+0000","last_unstale":"2023-10-16T09:44:58.428802+0000","last_undegraded":"2023-10-16T09:44:58.428802+0000","last_fullsized":"2023-10-16T09:44:58.428802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:18:47.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.428967+0000","last_change":"2023-10-16T09:44:56.806894+0000","last_active":"2023-10-16T09:44:58.428967+0000","last_peered":"2023-10-16T09:44:58.428967+0000","last_clean":"2023-10-16T09:44:58.428967+0000","last_became_active":"2023-10-16T09:44:56.806596+0000","last_became_peered":"2023-10-16T09:44:56.806596+0000","last_unstale":"2023-10-16T09:44:58.428967+0000","last_undegraded":"2023-10-16T09:44:58.428967+0000","last_fullsized":"2023-10-16T09:44:58.428967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:03.733810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.790458+0000","last_change":"2023-10-16T09:44:55.792347+0000","last_active":"2023-10-16T09:44:56.790458+0000","last_peered":"2023-10-16T09:44:56.790458+0000","last_clean":"2023-10-16T09:44:56.790458+0000","last_became_active":"2023-10-16T09:44:55.792073+0000","last_became_peered":"2023-10-16T09:44:55.792073+0000","last_unstale":"2023-10-16T09:44:56.790458+0000","last_undegraded":"2023-10-16T09:44:56.790458+0000","last_fullsized":"2023-10-16T09:44:56.790458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:26.235932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.788414+0000","last_change":"2023-10-16T09:44:56.817067+0000","last_active":"2023-10-16T09:44:57.788414+0000","last_peered":"2023-10-16T09:44:57.788414+0000","last_clean":"2023-10-16T09:44:57.788414+0000","last_became_active":"2023-10-16T09:44:56.816905+0000","last_became_peered":"2023-10-16T09:44:56.816905+0000","last_unstale":"2023-10-16T09:44:57.788414+0000","last_undegraded":"2023-10-16T09:44:57.788414+0000","last_fullsized":"2023-10-16T09:44:57.788414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:34.929737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.799817+0000","last_change":"2023-10-16T09:44:55.801193+0000","last_active":"2023-10-16T09:44:56.799817+0000","last_peered":"2023-10-16T09:44:56.799817+0000","last_clean":"2023-10-16T09:44:56.799817+0000","last_became_active":"2023-10-16T09:44:55.800923+0000","last_became_peered":"2023-10-16T09:44:55.800923+0000","last_unstale":"2023-10-16T09:44:56.799817+0000","last_undegraded":"2023-10-16T09:44:56.799817+0000","last_fullsized":"2023-10-16T09:44:56.799817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:35:15.291908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.188912+0000","last_change":"2023-10-16T09:44:56.814220+0000","last_active":"2023-10-16T09:44:58.188912+0000","last_peered":"2023-10-16T09:44:58.188912+0000","last_clean":"2023-10-16T09:44:58.188912+0000","last_became_active":"2023-10-16T09:44:56.813944+0000","last_became_peered":"2023-10-16T09:44:56.813944+0000","last_unstale":"2023-10-16T09:44:58.188912+0000","last_undegraded":"2023-10-16T09:44:58.188912+0000","last_fullsized":"2023-10-16T09:44:58.188912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:10.246138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.803595+0000","last_change":"2023-10-16T09:44:55.806449+0000","last_active":"2023-10-16T09:44:56.803595+0000","last_peered":"2023-10-16T09:44:56.803595+0000","last_clean":"2023-10-16T09:44:56.803595+0000","last_became_active":"2023-10-16T09:44:55.806201+0000","last_became_peered":"2023-10-16T09:44:55.806201+0000","last_unstale":"2023-10-16T09:44:56.803595+0000","last_undegraded":"2023-10-16T09:44:56.803595+0000","last_fullsized":"2023-10-16T09:44:56.803595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:13.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:58.722525+0000","last_change":"2023-10-16T09:44:56.807830+0000","last_active":"2023-10-16T09:44:58.722525+0000","last_peered":"2023-10-16T09:44:58.722525+0000","last_clean":"2023-10-16T09:44:58.722525+0000","last_became_active":"2023-10-16T09:44:56.807547+0000","last_became_peered":"2023-10-16T09:44:56.807547+0000","last_unstale":"2023-10-16T09:44:58.722525+0000","last_undegraded":"2023-10-16T09:44:58.722525+0000","last_fullsized":"2023-10-16T09:44:58.722525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:55.779326+0000","last_clean_scrub_stamp":"2023-10-16T09:44:55.779326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:57.785495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-16T09:44:57.812507+0000","last_change":"2023-10-16T09:44:55.801763+0000","last_active":"2023-10-16T09:44:57.812507+0000","last_peered":"2023-10-16T09:44:57.812507+0000","last_clean":"2023-10-16T09:44:57.812507+0000","last_became_active":"2023-10-16T09:44:55.801471+0000","last_became_peered":"2023-10-16T09:44:55.801471+0000","last_unstale":"2023-10-16T09:44:57.812507+0000","last_undegraded":"2023-10-16T09:44:57.812507+0000","last_fullsized":"2023-10-16T09:44:57.812507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:44:54.777401+0000","last_clean_scrub_stamp":"2023-10-16T09:44:54.777401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:26.235932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-16T09:44:56.789023+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:44:56.789023+0000","last_peered":"2023-10-16T09:44:56.789023+0000","last_clean":"2023-10-16T09:44:56.789023+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:44:56.789023+0000","last_undegraded":"2023-10-16T09:44:56.789023+0000","last_fullsized":"2023-10-16T09:44:56.789023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":978944,"data_stored":737829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.377}]},{"osd":2,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.45200000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.58799999999999997}]},{"osd":4,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.629},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60699999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":6.194,"5min":6.194,"15min":6.194},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":6.1609999999999996,"5min":6.1609999999999996,"15min":6.1609999999999996},"last":0.56899999999999995}]},{"osd":1,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.438},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":8.5600000000000005,"5min":8.5600000000000005,"15min":8.5600000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.52800000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.503},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":6.2249999999999996,"5min":6.2249999999999996,"15min":6.2249999999999996},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63400000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":149005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33300000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.315}]},{"osd":4,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59499999999999997}]},{"osd":5,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.628}]},{"osd":6,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 16 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57199999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":737739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":5.5090000000000003,"5min":5.5090000000000003,"15min":5.5090000000000003},"last":0.372},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":6.4950000000000001,"5min":6.4950000000000001,"15min":6.4950000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.437},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.496}]},{"osd":3,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":8.3260000000000005,"5min":8.3260000000000005,"15min":8.3260000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":5.4809999999999999,"5min":5.4809999999999999,"15min":5.4809999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.4349999999999996,"5min":6.4349999999999996,"15min":6.4349999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":5.4160000000000004,"5min":5.4160000000000004,"15min":5.4160000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.3869999999999996,"5min":6.3869999999999996,"15min":6.3869999999999996},"last":0.76800000000000002}]},{"osd":6,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":8.2189999999999994,"5min":8.2189999999999994,"15min":8.2189999999999994},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":5.4560000000000004,"5min":5.4560000000000004,"15min":5.4560000000000004},"last":0.80000000000000004}]},{"osd":7,"last update":"Mon Oct 16 09:44:55 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":6.4649999999999999,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":8.266,"5min":8.266,"15min":8.266},"last":0.85599999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":147441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.47299999999999998}]},{"osd":3,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.499}]},{"osd":4,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.627}]},{"osd":5,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58799999999999997}]},{"osd":7,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.70699999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":397312,"data_stored":149005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.29999999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":6.2240000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.38},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32100000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.4420000000000002,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":0.56699999999999995}]},{"osd":5,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Mon Oct 16 09:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.70799999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":147475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.44700000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.47099999999999997}]},{"osd":2,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.498},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57699999999999996}]},{"osd":3,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.55700000000000005}]},{"osd":5,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.626},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.63700000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Oct 16 09:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65200000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0370000000000008,"5min":8.0370000000000008,"15min":8.0370000000000008},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":8.2110000000000003,"5min":8.2110000000000003,"15min":8.2110000000000003},"last":0.40799999999999997}]},{"osd":2,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.1140000000000008,"5min":8.1140000000000008,"15min":8.1140000000000008},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":8.3000000000000007,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.44800000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.502},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":9.8810000000000002,"5min":9.8810000000000002,"15min":9.8810000000000002},"last":0.64600000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:45:05.885 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T09:45:05.885 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T09:45:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.044+0000 7fdfca6da700 1 Processor -- start 2023-10-16T09:45:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.044+0000 7fdfca6da700 1 -- start start 2023-10-16T09:45:06.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.044+0000 7fdfca6da700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 0x7fdfc41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.044+0000 7fdfca6da700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc40cef00 con 0x7fdfc4072760 2023-10-16T09:45:06.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.044+0000 7fdfca6da700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc40cec20 con 0x7fdfc415da70 2023-10-16T09:45:06.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.044+0000 7fdfca6da700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4071c80 con 0x7fdfc406afb0 2023-10-16T09:45:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdfc4072760 0x7fdfc406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49158/0 (socket says 172.21.15.181:49158) 2023-10-16T09:45:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc37fe700 1 -- 172.21.15.181:0/3960501138 learned_addr learned my addr 172.21.15.181:0/3960501138 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc8c77700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 0x7fdfc41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399250967 0 0) 0x7fdfc40cef00 con 0x7fdfc4072760 2023-10-16T09:45:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfac0035b0 con 0x7fdfc4072760 2023-10-16T09:45:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276121941 0 0) 0x7fdfc40cec20 con 0x7fdfc415da70 2023-10-16T09:45:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc40cef00 con 0x7fdfc415da70 2023-10-16T09:45:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3503443861 0 0) 0x7fdfac0035b0 con 0x7fdfc4072760 2023-10-16T09:45:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 msgr2=0x7fdfc41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 0x7fdfc41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 >> v1:172.21.15.190:6789/0 conn(0x7fdfc415da70 legacy=0x7fdfc40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4071f60 con 0x7fdfc4072760 2023-10-16T09:45:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc8c77700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 0x7fdfc41615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fdfb8001670 con 0x7fdfc4072760 2023-10-16T09:45:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 >> v1:172.21.15.181:6789/0 conn(0x7fdfc4072760 legacy=0x7fdfc406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 0x7fdfac003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfac004100 0x7fdfac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfac004d90 0x7fdfac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4071c80 con 0x7fdfac004d90 2023-10-16T09:45:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac00bc00 con 0x7fdfac004100 2023-10-16T09:45:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac00bdc0 con 0x7fdfc406afb0 2023-10-16T09:45:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc8c77700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfac004d90 0x7fdfac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc37fe700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfac004100 0x7fdfac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.045+0000 7fdfc3fff700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 0x7fdfac003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc8c77700 1 -- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 msgr2=0x7fdfac003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc8c77700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc406afb0 0x7fdfac003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc8c77700 1 -- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfac004100 msgr2=0x7fdfac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc8c77700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfac004100 0x7fdfac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc8c77700 1 -- 172.21.15.181:0/3960501138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc009a70 con 0x7fdfac004d90 2023-10-16T09:45:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc8c77700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfac004d90 0x7fdfac005210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e350 tx=0x7fdfac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d580b66c14916ab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00ee10 con 0x7fdfac004d90 2023-10-16T09:45:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc00f040 con 0x7fdfac004d90 2023-10-16T09:45:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc2ffd700 1 -- 172.21.15.181:0/3960501138 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003c60 con 0x7fdfac004d90 2023-10-16T09:45:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfca6da700 1 -- 172.21.15.181:0/3960501138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfac004d90 msgr2=0x7fdfac005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfca6da700 1 --2- 172.21.15.181:0/3960501138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfac004d90 0x7fdfac005210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e350 tx=0x7fdfac005880 comp rx=0 tx=0).stop 2023-10-16T09:45:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfc8c77700 1 -- 172.21.15.181:0/3960501138 reap_dead start 2023-10-16T09:45:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfca6da700 1 -- 172.21.15.181:0/3960501138 shutdown_connections 2023-10-16T09:45:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfca6da700 1 -- 172.21.15.181:0/3960501138 >> 172.21.15.181:0/3960501138 conn(0x7fdfc40cd010 msgr2=0x7fdfc40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfca6da700 1 -- 172.21.15.181:0/3960501138 shutdown_connections 2023-10-16T09:45:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.046+0000 7fdfca6da700 1 -- 172.21.15.181:0/3960501138 wait complete. 2023-10-16T09:45:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.048+0000 7fdfca6da700 1 Processor -- start 2023-10-16T09:45:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.048+0000 7fdfca6da700 1 -- start start 2023-10-16T09:45:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.048+0000 7fdfca6da700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfc415c2c0 0x7fdfc4160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.048+0000 7fdfca6da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfc40b0110 0x7fdfc40c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.048+0000 7fdfca6da700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc40c63c0 0x7fdfc415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfca6da700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40c6870 con 0x7fdfc40b0110 2023-10-16T09:45:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfca6da700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b2170 con 0x7fdfc415c2c0 2023-10-16T09:45:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfca6da700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b23c0 con 0x7fdfc40c63c0 2023-10-16T09:45:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfc40b0110 0x7fdfc40c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfc40b0110 0x7fdfc40c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38092/0 (socket says 172.21.15.181:38092) 2023-10-16T09:45:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 -- 172.21.15.181:0/2931362197 learned_addr learned my addr 172.21.15.181:0/2931362197 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc8c77700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc40c63c0 0x7fdfc415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc3fff700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfc415c2c0 0x7fdfc4160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 -- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc40c63c0 msgr2=0x7fdfc415eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc40c63c0 0x7fdfc415eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 -- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfc415c2c0 msgr2=0x7fdfc4160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfc415c2c0 0x7fdfc4160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 -- 172.21.15.181:0/2931362197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40b26a0 con 0x7fdfc40b0110 2023-10-16T09:45:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc3fff700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfc415c2c0 0x7fdfc4160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:45:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc37fe700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfc40b0110 0x7fdfc40c5e80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8007370 tx=0x7fdfb8007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc0ff9700 1 -- 172.21.15.181:0/2931362197 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8001060 con 0x7fdfc40b0110 2023-10-16T09:45:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc0ff9700 1 -- 172.21.15.181:0/2931362197 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb800f870 con 0x7fdfc40b0110 2023-10-16T09:45:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfc0ff9700 1 -- 172.21.15.181:0/2931362197 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800fc20 con 0x7fdfc40b0110 2023-10-16T09:45:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc40b2980 con 0x7fdfc40b0110 2023-10-16T09:45:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.049+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc415c6a0 con 0x7fdfc40b0110 2023-10-16T09:45:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.050+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc40b2ea0 con 0x7fdfc40b0110 2023-10-16T09:45:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.051+0000 7fdfc0ff9700 1 -- 172.21.15.181:0/2931362197 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdfb800fd80 con 0x7fdfc40b0110 2023-10-16T09:45:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.051+0000 7fdfc0ff9700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdfb006af00 0x7fdfb006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.051+0000 7fdfc0ff9700 1 -- 172.21.15.181:0/2931362197 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfb80895d0 con 0x7fdfc40b0110 2023-10-16T09:45:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.051+0000 7fdfc3fff700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdfb006af00 0x7fdfb006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.051+0000 7fdfc3fff700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdfb006af00 0x7fdfb006d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40010e0 tx=0x7fdfb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.056+0000 7fdfc0ff9700 1 -- 172.21.15.181:0/2931362197 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdfc40b2ea0 con 0x7fdfc40b0110 2023-10-16T09:45:06.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.326+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdfc40b2ea0 con 0x7fdfc40b0110 2023-10-16T09:45:06.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.327+0000 7fdfc0ff9700 1 -- 172.21.15.181:0/2931362197 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdfc40b2ea0 con 0x7fdfc40b0110 2023-10-16T09:45:06.330 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T09:45:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T09:45:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T09:45:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T09:45:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T09:45:06.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdfb006af00 msgr2=0x7fdfb006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdfb006af00 0x7fdfb006d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40010e0 tx=0x7fdfb4009040 comp rx=0 tx=0).stop 2023-10-16T09:45:06.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfc40b0110 msgr2=0x7fdfc40c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfc40b0110 0x7fdfc40c5e80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8007370 tx=0x7fdfb8007730 comp rx=0 tx=0).stop 2023-10-16T09:45:06.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 shutdown_connections 2023-10-16T09:45:06.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fdfb006af00 0x7fdfb006d3c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfc40c63c0 0x7fdfc415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfc415c2c0 0x7fdfc4160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 --2- 172.21.15.181:0/2931362197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdfc40b0110 0x7fdfc40c5e80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 >> 172.21.15.181:0/2931362197 conn(0x7fdfc40cd010 msgr2=0x7fdfc40b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:06.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 shutdown_connections 2023-10-16T09:45:06.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.330+0000 7fdfca6da700 1 -- 172.21.15.181:0/2931362197 wait complete. 2023-10-16T09:45:06.335 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T09:45:06.345 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:45:06.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.463+0000 7f9c3befe700 1 Processor -- start 2023-10-16T09:45:06.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.464+0000 7f9c3befe700 1 -- start start 2023-10-16T09:45:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.464+0000 7f9c3befe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c3415dfd0 0x7f9c340af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.464+0000 7f9c3befe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3406f870 con 0x7f9c340cd470 2023-10-16T09:45:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.464+0000 7f9c3befe700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340d1160 con 0x7f9c340af450 2023-10-16T09:45:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.464+0000 7f9c3befe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c340cc8e0 con 0x7f9c3415dfd0 2023-10-16T09:45:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c39c9a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c3415dfd0 0x7f9c340af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c3a49b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9c340cd470 0x7f9c3406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49180/0 (socket says 172.21.15.181:49180) 2023-10-16T09:45:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c3a49b700 1 -- 172.21.15.181:0/3404771497 learned_addr learned my addr 172.21.15.181:0/3404771497 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c39c9a700 1 -- 172.21.15.181:0/3404771497 >> v1:172.21.15.190:6789/0 conn(0x7f9c340af450 legacy=0x7f9c3406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:06.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c39c9a700 1 -- 172.21.15.181:0/3404771497 >> v1:172.21.15.181:6789/0 conn(0x7f9c340cd470 legacy=0x7f9c3406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c39c9a700 1 -- 172.21.15.181:0/3404771497 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340ccbc0 con 0x7f9c3415dfd0 2023-10-16T09:45:06.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c38c98700 1 -- 172.21.15.181:0/3404771497 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479876126 0 0) 0x7f9c3406f870 con 0x7f9c340cd470 2023-10-16T09:45:06.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c39c9a700 1 --2- 172.21.15.181:0/3404771497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3415dfd0 0x7f9c340af9a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009820 tx=0x7f9c24009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b22b20cf925726d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:06.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c38c98700 1 -- 172.21.15.181:0/3404771497 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24005e30 con 0x7f9c3415dfd0 2023-10-16T09:45:06.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c38c98700 1 -- 172.21.15.181:0/3404771497 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24004030 con 0x7f9c3415dfd0 2023-10-16T09:45:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.465+0000 7f9c38c98700 1 -- 172.21.15.181:0/3404771497 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240043c0 con 0x7f9c3415dfd0 2023-10-16T09:45:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.466+0000 7f9c3befe700 1 -- 172.21.15.181:0/3404771497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3415dfd0 msgr2=0x7f9c340af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.466+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3404771497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3415dfd0 0x7f9c340af9a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009820 tx=0x7f9c24009b30 comp rx=0 tx=0).stop 2023-10-16T09:45:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.466+0000 7f9c3befe700 1 -- 172.21.15.181:0/3404771497 shutdown_connections 2023-10-16T09:45:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.466+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3404771497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3415dfd0 0x7f9c340af9a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.466+0000 7f9c3befe700 1 -- 172.21.15.181:0/3404771497 >> 172.21.15.181:0/3404771497 conn(0x7f9c340d04a0 msgr2=0x7f9c340d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.466+0000 7f9c3befe700 1 -- 172.21.15.181:0/3404771497 shutdown_connections 2023-10-16T09:45:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.466+0000 7f9c3befe700 1 -- 172.21.15.181:0/3404771497 wait complete. 2023-10-16T09:45:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.467+0000 7f9c3befe700 1 Processor -- start 2023-10-16T09:45:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.467+0000 7f9c3befe700 1 -- start start 2023-10-16T09:45:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c340af450 0x7f9c340b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c340cd470 0x7f9c341548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c3415dfd0 0x7f9c340adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c341f39a0 con 0x7f9c340af450 2023-10-16T09:45:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c341f3070 con 0x7f9c3415dfd0 2023-10-16T09:45:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c341f3350 con 0x7f9c340cd470 2023-10-16T09:45:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c340af450 0x7f9c340b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c340af450 0x7f9c340b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38104/0 (socket says 172.21.15.181:38104) 2023-10-16T09:45:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 -- 172.21.15.181:0/3558532401 learned_addr learned my addr 172.21.15.181:0/3558532401 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 -- 172.21.15.181:0/3558532401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c340cd470 msgr2=0x7f9c341548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c340cd470 0x7f9c341548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 -- 172.21.15.181:0/3558532401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c3415dfd0 msgr2=0x7f9c340adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c3415dfd0 0x7f9c340adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 -- 172.21.15.181:0/3558532401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c24009430 con 0x7f9c340af450 2023-10-16T09:45:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c39c9a700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c340af450 0x7f9c340b5be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c24000c00 tx=0x7f9c24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c2a7fc700 1 -- 172.21.15.181:0/3558532401 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2401d430 con 0x7f9c340af450 2023-10-16T09:45:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c2a7fc700 1 -- 172.21.15.181:0/3558532401 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2401d590 con 0x7f9c340af450 2023-10-16T09:45:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c2a7fc700 1 -- 172.21.15.181:0/3558532401 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2401d940 con 0x7f9c340af450 2023-10-16T09:45:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c341f3630 con 0x7f9c340af450 2023-10-16T09:45:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.468+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c341f2d80 con 0x7f9c340af450 2023-10-16T09:45:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.469+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c34067d20 con 0x7f9c340af450 2023-10-16T09:45:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.470+0000 7f9c2a7fc700 1 -- 172.21.15.181:0/3558532401 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9c2401daa0 con 0x7f9c340af450 2023-10-16T09:45:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.470+0000 7f9c2a7fc700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9c1806af00 0x7f9c1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.470+0000 7f9c2a7fc700 1 -- 172.21.15.181:0/3558532401 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c2408ab50 con 0x7f9c340af450 2023-10-16T09:45:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.470+0000 7f9c39499700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9c1806af00 0x7f9c1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:06.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.471+0000 7f9c39499700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9c1806af00 0x7f9c1806d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c3409e460 tx=0x7f9c30007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:06.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.475+0000 7f9c2a7fc700 1 -- 172.21.15.181:0/3558532401 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9c24055db0 con 0x7f9c340af450 2023-10-16T09:45:06.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.712+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c340cb4d0 con 0x7f9c340af450 2023-10-16T09:45:06.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.713+0000 7f9c2a7fc700 1 -- 172.21.15.181:0/3558532401 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f9c2402b090 con 0x7f9c340af450 2023-10-16T09:45:06.715 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:45:06.716 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3974533571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3974533571},{"type":"v1","addr":"172.21.15.190:6834","nonce":3974533571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3064364132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3064364132},{"type":"v1","addr":"172.21.15.181:6837","nonce":3064364132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4021280674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4021280674},{"type":"v1","addr":"172.21.15.190:6835","nonce":4021280674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:56.185727+0000","modified":"2023-10-16T09:44:59.222716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6836/760205137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":760205137},{"type":"v1","addr":"172.21.15.181:6836","nonce":760205137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:45:06.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9c1806af00 msgr2=0x7f9c1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9c1806af00 0x7f9c1806d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c3409e460 tx=0x7f9c30007a40 comp rx=0 tx=0).stop 2023-10-16T09:45:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c340af450 msgr2=0x7f9c340b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c340af450 0x7f9c340b5be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9c24000c00 tx=0x7f9c24004750 comp rx=0 tx=0).stop 2023-10-16T09:45:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 shutdown_connections 2023-10-16T09:45:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f9c1806af00 0x7f9c1806d3c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c3415dfd0 0x7f9c340adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c340af450 0x7f9c340b5be0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 --2- 172.21.15.181:0/3558532401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c340cd470 0x7f9c341548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 >> 172.21.15.181:0/3558532401 conn(0x7f9c340d04a0 msgr2=0x7f9c3415d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 shutdown_connections 2023-10-16T09:45:06.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:06.715+0000 7f9c3befe700 1 -- 172.21.15.181:0/3558532401 wait complete. 2023-10-16T09:45:06.720 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2023-10-16T09:45:07.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T09:44:56.185727+0000', 'modified': '2023-10-16T09:44:59.222716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5049': {'gid': 5049, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6836/760205137', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 760205137}, {'type': 'v1', 'addr': '172.21.15.181:6836', 'nonce': 760205137}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T09:45:07.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T09:45:07.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T09:45:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.256+0000 7f6d3a85b700 1 Processor -- start 2023-10-16T09:45:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.256+0000 7f6d3a85b700 1 -- start start 2023-10-16T09:45:07.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d3a85b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d3415dd50 0x7f6d340d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d3a85b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d340d2300 0x7f6d340ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d3a85b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d340b0410 con 0x7f6d340d1e80 2023-10-16T09:45:07.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d3a85b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d340b3c00 con 0x7f6d340d2300 2023-10-16T09:45:07.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d3a85b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3415e130 con 0x7f6d3415dd50 2023-10-16T09:45:07.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d340d2300 0x7f6d340ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d340d2300 0x7f6d340ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38116/0 (socket says 172.21.15.181:38116) 2023-10-16T09:45:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 -- 172.21.15.181:0/1752690072 learned_addr learned my addr 172.21.15.181:0/1752690072 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1752690072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d3415dd50 0x7f6d340d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 -- 172.21.15.181:0/1752690072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d3415dd50 msgr2=0x7f6d340d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 --2- 172.21.15.181:0/1752690072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d3415dd50 0x7f6d340d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 -- 172.21.15.181:0/1752690072 >> v1:172.21.15.190:6789/0 conn(0x7f6d340d1e80 legacy=0x7f6d340ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 -- 172.21.15.181:0/1752690072 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d340b2ca0 con 0x7f6d340d2300 2023-10-16T09:45:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1752690072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d3415dd50 0x7f6d340d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d38df8700 1 --2- 172.21.15.181:0/1752690072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d340d2300 0x7f6d340ceb50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009e50 tx=0x7f6d2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61947abf2fe7349 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d32ffd700 1 -- 172.21.15.181:0/1752690072 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00e070 con 0x7f6d340d2300 2023-10-16T09:45:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d32ffd700 1 -- 172.21.15.181:0/1752690072 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c00b9c0 con 0x7f6d340d2300 2023-10-16T09:45:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.257+0000 7f6d32ffd700 1 -- 172.21.15.181:0/1752690072 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00bd70 con 0x7f6d340d2300 2023-10-16T09:45:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 -- 172.21.15.181:0/1752690072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d340d2300 msgr2=0x7f6d340ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/1752690072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d340d2300 0x7f6d340ceb50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009e50 tx=0x7f6d2c00b330 comp rx=0 tx=0).stop 2023-10-16T09:45:07.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 -- 172.21.15.181:0/1752690072 shutdown_connections 2023-10-16T09:45:07.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/1752690072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d340d2300 0x7f6d340ceb50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/1752690072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d3415dd50 0x7f6d340d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 -- 172.21.15.181:0/1752690072 >> 172.21.15.181:0/1752690072 conn(0x7f6d3408af90 msgr2=0x7f6d340d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 -- 172.21.15.181:0/1752690072 shutdown_connections 2023-10-16T09:45:07.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.258+0000 7f6d3a85b700 1 -- 172.21.15.181:0/1752690072 wait complete. 2023-10-16T09:45:07.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.260+0000 7f6d3a85b700 1 Processor -- start 2023-10-16T09:45:07.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.260+0000 7f6d3a85b700 1 -- start start 2023-10-16T09:45:07.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d3a85b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d340d1e80 0x7f6d340cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d3a85b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d340d2300 0x7f6d341f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d3a85b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3415dd50 0x7f6d3415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d3a85b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3409db10 con 0x7f6d3415dd50 2023-10-16T09:45:07.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d3a85b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3409d590 con 0x7f6d340d2300 2023-10-16T09:45:07.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d3a85b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3414f0a0 con 0x7f6d340d1e80 2023-10-16T09:45:07.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3415dd50 0x7f6d3415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3415dd50 0x7f6d3415f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38122/0 (socket says 172.21.15.181:38122) 2023-10-16T09:45:07.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 -- 172.21.15.181:0/60052349 learned_addr learned my addr 172.21.15.181:0/60052349 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 -- 172.21.15.181:0/60052349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d340d1e80 msgr2=0x7f6d340cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d340d1e80 0x7f6d340cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 -- 172.21.15.181:0/60052349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d340d2300 msgr2=0x7f6d341f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d337fe700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d340d2300 0x7f6d341f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d340d2300 0x7f6d341f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 -- 172.21.15.181:0/60052349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c009b30 con 0x7f6d3415dd50 2023-10-16T09:45:07.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d38df8700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3415dd50 0x7f6d3415f2d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d3408d0d0 tx=0x7f6d2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d30ff9700 1 -- 172.21.15.181:0/60052349 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c004960 con 0x7f6d3415dd50 2023-10-16T09:45:07.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d30ff9700 1 -- 172.21.15.181:0/60052349 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c0099b0 con 0x7f6d3415dd50 2023-10-16T09:45:07.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d30ff9700 1 -- 172.21.15.181:0/60052349 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c01c680 con 0x7f6d3415dd50 2023-10-16T09:45:07.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.261+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3409d830 con 0x7f6d3415dd50 2023-10-16T09:45:07.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.262+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d340cb370 con 0x7f6d3415dd50 2023-10-16T09:45:07.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.263+0000 7f6d30ff9700 1 -- 172.21.15.181:0/60052349 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d2c01c7e0 con 0x7f6d3415dd50 2023-10-16T09:45:07.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.264+0000 7f6d30ff9700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6d2006af00 0x7f6d2006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.264+0000 7f6d30ff9700 1 -- 172.21.15.181:0/60052349 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d2c08b2c0 con 0x7f6d3415dd50 2023-10-16T09:45:07.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.264+0000 7f6d33fff700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6d2006af00 0x7f6d2006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.264+0000 7f6d33fff700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6d2006af00 0x7f6d2006d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d24006f30 tx=0x7f6d24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.265+0000 7f6d46a45b80 1 Processor -- start 2023-10-16T09:45:07.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d46a45b80 1 -- start start 2023-10-16T09:45:07.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e717a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e717fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e6eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e661c20 con 0x55f29e710220 2023-10-16T09:45:07.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e646c10 con 0x55f29e6eb010 2023-10-16T09:45:07.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f29e644530 con 0x55f29e715630 2023-10-16T09:45:07.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e717a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e717a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38126/0 (socket says 172.21.15.181:38126) 2023-10-16T09:45:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d33fff700 1 -- 172.21.15.181:0/1504714023 learned_addr learned my addr 172.21.15.181:0/1504714023 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d38df8700 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e6eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.266+0000 7f6d337fe700 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e717fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d33fff700 1 -- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 msgr2=0x55f29e717fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e717fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d33fff700 1 -- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 msgr2=0x55f29e6eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e6eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d33fff700 1 -- 172.21.15.181:0/1504714023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f29e5fd3a0 con 0x55f29e710220 2023-10-16T09:45:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d337fe700 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e717fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e717a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d24015910 tx=0x7f6d24011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e4532d02927333 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d19ffb700 1 -- 172.21.15.181:0/1504714023 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d240136f0 con 0x55f29e710220 2023-10-16T09:45:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d19ffb700 1 -- 172.21.15.181:0/1504714023 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d24013850 con 0x55f29e710220 2023-10-16T09:45:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.267+0000 7f6d19ffb700 1 -- 172.21.15.181:0/1504714023 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24013bc0 con 0x55f29e710220 2023-10-16T09:45:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1504714023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 msgr2=0x55f29e717a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e717a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d24015910 tx=0x7f6d24011000 comp rx=0 tx=0).stop 2023-10-16T09:45:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1504714023 shutdown_connections 2023-10-16T09:45:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e6eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e717a70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1504714023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e717fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1504714023 >> 172.21.15.181:0/1504714023 conn(0x55f29e61ad80 msgr2=0x55f29e735ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1504714023 shutdown_connections 2023-10-16T09:45:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1504714023 wait complete. 2023-10-16T09:45:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 -- start start 2023-10-16T09:45:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.268+0000 7f6d46a45b80 1 Processor -- start 2023-10-16T09:45:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e70f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e710220 0x55f29e713ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e715630 0x55f29e6eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e63fe20 con 0x55f29e6eb010 2023-10-16T09:45:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e5dc6c0 con 0x55f29e715630 2023-10-16T09:45:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f29e5dc360 con 0x55f29e710220 2023-10-16T09:45:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d46a45b80 10 client.0 _ll_register_callbacks cb 0x55f29e4397b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:45:07.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e70f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e70f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38132/0 (socket says 172.21.15.181:38132) 2023-10-16T09:45:07.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d33fff700 1 -- 172.21.15.181:0/4066225235 learned_addr learned my addr 172.21.15.181:0/4066225235 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d337fe700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e710220 0x55f29e713ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.269+0000 7f6d38df8700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e715630 0x55f29e6eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d33fff700 1 -- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e710220 msgr2=0x55f29e713ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d33fff700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e710220 0x55f29e713ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d33fff700 1 -- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e715630 msgr2=0x55f29e6eb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d33fff700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e715630 0x55f29e6eb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d33fff700 1 -- 172.21.15.181:0/4066225235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d240155f0 con 0x55f29e6eb010 2023-10-16T09:45:07.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d38df8700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e715630 0x55f29e6eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:07.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d33fff700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e70f2b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d24015910 tx=0x7f6d24006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9451f208cfdb1a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:45:07.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d12ffd700 1 -- 172.21.15.181:0/4066225235 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24032650 con 0x55f29e6eb010 2023-10-16T09:45:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d12ffd700 1 -- 172.21.15.181:0/4066225235 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d240327b0 con 0x55f29e6eb010 2023-10-16T09:45:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f29e6e05b0 con 0x55f29e6eb010 2023-10-16T09:45:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d12ffd700 1 -- 172.21.15.181:0/4066225235 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d24032970 con 0x55f29e6eb010 2023-10-16T09:45:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 10 client.5256 fetch_fsmap learned FSMap version 16 2023-10-16T09:45:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f29e46ad90 con 0x55f29e6eb010 2023-10-16T09:45:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d12ffd700 1 -- 172.21.15.181:0/4066225235 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f29e6e05b0 con 0x55f29e6eb010 2023-10-16T09:45:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d12ffd700 1 -- 172.21.15.181:0/4066225235 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d24030070 con 0x55f29e6eb010 2023-10-16T09:45:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 10 client.5256 fetch_fsmap finished waiting for FSMap version 16 2023-10-16T09:45:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 10 client.5256 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T09:45:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 20 client.5256 populate_metadata read hostname 'smithi181' 2023-10-16T09:45:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e710b00 0x55f29e710ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 4 client.5256 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T09:45:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 --> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f29e638e00 con 0x55f29e710b00 2023-10-16T09:45:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.270+0000 7f6d337fe700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e710b00 0x55f29e710ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d12ffd700 10 client.5256 ms_handle_connect on v2:172.21.15.181:6834/760205137 2023-10-16T09:45:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d337fe700 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e710b00 0x55f29e710ee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6ad2072a5aec7f server_cookie=9aad3591d87d9c68 in_seq=0 out_seq=0 2023-10-16T09:45:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d12ffd700 1 -- 172.21.15.181:0/4066225235 <== mds.0 v2:172.21.15.181:6834/760205137 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f29e638e00 con 0x55f29e710b00 2023-10-16T09:45:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d12ffd700 10 client.5256 handle_command_reply: tid=0 2023-10-16T09:45:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 client.5256 shutdown 2023-10-16T09:45:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 msgr2=0x55f29e70f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e70f2b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d24015910 tx=0x7f6d24006e20 comp rx=0 tx=0).stop 2023-10-16T09:45:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 shutdown_connections 2023-10-16T09:45:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e715630 0x55f29e6eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e70f2b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e710b00 0x55f29e710ee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/4066225235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e710220 0x55f29e713ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d12ffd700 0 client.5256 ms_handle_reset on v2:172.21.15.181:6834/760205137 2023-10-16T09:45:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.271+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 >> 172.21.15.181:0/4066225235 conn(0x55f29e61ad80 msgr2=0x55f29e733450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.272+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 shutdown_connections 2023-10-16T09:45:07.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.272+0000 7f6d46a45b80 1 -- 172.21.15.181:0/4066225235 wait complete. 2023-10-16T09:45:07.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.272+0000 7f6d46a45b80 20 client.5256 trim_cache size 0 max 16384 2023-10-16T09:45:07.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.289+0000 7f6d46a45b80 1 Processor -- start 2023-10-16T09:45:07.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.289+0000 7f6d46a45b80 1 -- start start 2023-10-16T09:45:07.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e736250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e710220 0x55f29e6e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e72c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e714bb0 con 0x55f29e6eb010 2023-10-16T09:45:07.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e4c08d0 con 0x55f29e710220 2023-10-16T09:45:07.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f29e6eefa0 con 0x55f29e715630 2023-10-16T09:45:07.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e736250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e736250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38140/0 (socket says 172.21.15.181:38140) 2023-10-16T09:45:07.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 -- 172.21.15.181:0/1021493271 learned_addr learned my addr 172.21.15.181:0/1021493271 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d38df8700 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e72c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 -- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 msgr2=0x55f29e72c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e72c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 -- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e710220 msgr2=0x55f29e6e1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d337fe700 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e710220 0x55f29e6e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e710220 0x55f29e6e1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 -- 172.21.15.181:0/1021493271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f29e453c10 con 0x55f29e6eb010 2023-10-16T09:45:07.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e736250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400c300 tx=0x7f6d2402a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26b34086ea02829 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d1a7fc700 1 -- 172.21.15.181:0/1021493271 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24013ba0 con 0x55f29e6eb010 2023-10-16T09:45:07.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d1a7fc700 1 -- 172.21.15.181:0/1021493271 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2402c070 con 0x55f29e6eb010 2023-10-16T09:45:07.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.290+0000 7f6d1a7fc700 1 -- 172.21.15.181:0/1021493271 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24032690 con 0x55f29e6eb010 2023-10-16T09:45:07.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1021493271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 msgr2=0x55f29e736250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e736250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400c300 tx=0x7f6d2402a070 comp rx=0 tx=0).stop 2023-10-16T09:45:07.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1021493271 shutdown_connections 2023-10-16T09:45:07.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e72c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e6eb010 0x55f29e736250 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1021493271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e710220 0x55f29e6e1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1021493271 >> 172.21.15.181:0/1021493271 conn(0x55f29e61ad80 msgr2=0x55f29e70f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1021493271 shutdown_connections 2023-10-16T09:45:07.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.291+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1021493271 wait complete. 2023-10-16T09:45:07.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 -- start start 2023-10-16T09:45:07.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 Processor -- start 2023-10-16T09:45:07.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e71b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e6ec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e6e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e79a610 con 0x55f29e710220 2023-10-16T09:45:07.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29e732c70 con 0x55f29e6eb010 2023-10-16T09:45:07.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.292+0000 7f6d46a45b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f29e6e8ec0 con 0x55f29e715630 2023-10-16T09:45:07.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d46a45b80 10 client.0 _ll_register_callbacks cb 0x7f6d2400c550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:45:07.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e6ec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e6ec890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38148/0 (socket says 172.21.15.181:38148) 2023-10-16T09:45:07.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 -- 172.21.15.181:0/1284109692 learned_addr learned my addr 172.21.15.181:0/1284109692 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d38df8700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e6e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e71b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 -- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 msgr2=0x55f29e6e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e6e7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 -- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 msgr2=0x55f29e71b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e71b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 -- 172.21.15.181:0/1284109692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f29e46ad90 con 0x55f29e710220 2023-10-16T09:45:07.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e71b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:07.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d337fe700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e6ec890 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55f29e6e96e0 tx=0x7f6d28001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967f14708833dfef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:45:07.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d12ffd700 1 -- 172.21.15.181:0/1284109692 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801a070 con 0x55f29e710220 2023-10-16T09:45:07.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d12ffd700 1 -- 172.21.15.181:0/1284109692 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d280053e0 con 0x55f29e710220 2023-10-16T09:45:07.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d24013850 con 0x55f29e710220 2023-10-16T09:45:07.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d12ffd700 1 -- 172.21.15.181:0/1284109692 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d280158b0 con 0x55f29e710220 2023-10-16T09:45:07.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d46a45b80 10 client.5262 fetch_fsmap learned FSMap version 16 2023-10-16T09:45:07.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f29e491200 con 0x55f29e710220 2023-10-16T09:45:07.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d12ffd700 1 -- 172.21.15.181:0/1284109692 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d24013850 con 0x55f29e710220 2023-10-16T09:45:07.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.293+0000 7f6d12ffd700 1 -- 172.21.15.181:0/1284109692 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d28012050 con 0x55f29e710220 2023-10-16T09:45:07.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 10 client.5262 fetch_fsmap finished waiting for FSMap version 16 2023-10-16T09:45:07.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 10 client.5262 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T09:45:07.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 20 client.5262 populate_metadata read hostname 'smithi181' 2023-10-16T09:45:07.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e744480 0x55f29e799550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 4 client.5262 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T09:45:07.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 --> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f29e638b20 con 0x55f29e744480 2023-10-16T09:45:07.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e744480 0x55f29e799550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d12ffd700 10 client.5262 ms_handle_connect on v2:172.21.15.181:6834/760205137 2023-10-16T09:45:07.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d33fff700 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e744480 0x55f29e799550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9771d0c1075912 server_cookie=4f913a6950e03736 in_seq=0 out_seq=0 2023-10-16T09:45:07.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d12ffd700 1 -- 172.21.15.181:0/1284109692 <== mds.0 v2:172.21.15.181:6834/760205137 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f29e638b20 con 0x55f29e744480 2023-10-16T09:45:07.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d12ffd700 10 client.5262 handle_command_reply: tid=0 2023-10-16T09:45:07.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 client.5262 shutdown 2023-10-16T09:45:07.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 msgr2=0x55f29e6ec890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e6ec890 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55f29e6e96e0 tx=0x7f6d28001310 comp rx=0 tx=0).stop 2023-10-16T09:45:07.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 shutdown_connections 2023-10-16T09:45:07.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:6834/760205137,v1:172.21.15.181:6836/760205137] conn(0x55f29e744480 0x55f29e799550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f29e715630 0x55f29e6e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d12ffd700 0 client.5262 ms_handle_reset on v2:172.21.15.181:6834/760205137 2023-10-16T09:45:07.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f29e6eb010 0x55f29e71b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 --2- 172.21.15.181:0/1284109692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f29e710220 0x55f29e6ec890 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 >> 172.21.15.181:0/1284109692 conn(0x55f29e61ad80 msgr2=0x55f29e741250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 shutdown_connections 2023-10-16T09:45:07.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.294+0000 7f6d46a45b80 1 -- 172.21.15.181:0/1284109692 wait complete. 2023-10-16T09:45:07.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d46a45b80 20 client.5262 trim_cache size 0 max 16384 2023-10-16T09:45:07.328 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:45:07.328 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T09:45:07.329 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T09:45:07.329 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 5049, 2023-10-16T09:45:07.329 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T09:45:07.329 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T09:45:07.329 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T09:45:07.330 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 10.509893869000001, 2023-10-16T09:45:07.330 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 16, 2023-10-16T09:45:07.330 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 34, 2023-10-16T09:45:07.330 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 34, 2023-10-16T09:45:07.331 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 18.870841749 2023-10-16T09:45:07.331 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:45:07.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6d2006af00 msgr2=0x7f6d2006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6d2006af00 0x7f6d2006d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d24006f30 tx=0x7f6d24008040 comp rx=0 tx=0).stop 2023-10-16T09:45:07.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3415dd50 msgr2=0x7f6d3415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3415dd50 0x7f6d3415f2d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d3408d0d0 tx=0x7f6d2c004ad0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 shutdown_connections 2023-10-16T09:45:07.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f6d2006af00 0x7f6d2006d3c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d340d1e80 0x7f6d340cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3415dd50 0x7f6d3415f2d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 --2- 172.21.15.181:0/60052349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d340d2300 0x7f6d341f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 >> 172.21.15.181:0/60052349 conn(0x7f6d3408af90 msgr2=0x7f6d340b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 shutdown_connections 2023-10-16T09:45:07.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.295+0000 7f6d3a85b700 1 -- 172.21.15.181:0/60052349 wait complete. 2023-10-16T09:45:07.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:45:07.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:45:07.412 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:45:07.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.457+0000 7f170847a700 1 Processor -- start 2023-10-16T09:45:07.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.457+0000 7f170847a700 1 -- start start 2023-10-16T09:45:07.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.457+0000 7f170847a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170015b990 0x7f17000ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.457+0000 7f170847a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17000cd2d0 0x7f17000d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f170847a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17000cc510 con 0x7f17000cfc60 2023-10-16T09:45:07.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f170847a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17000d24f0 con 0x7f17000cd2d0 2023-10-16T09:45:07.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f170847a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17000cc7f0 con 0x7f170015b990 2023-10-16T09:45:07.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1706216700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170015b990 0x7f17000ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1705a15700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17000cd2d0 0x7f17000d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1705a15700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17000cd2d0 0x7f17000d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:43206/0 (socket says 172.21.15.181:43206) 2023-10-16T09:45:07.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1706216700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170015b990 0x7f17000ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:35180/0 (socket says 172.21.15.181:35180) 2023-10-16T09:45:07.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1705a15700 1 -- 172.21.15.181:0/491141654 learned_addr learned my addr 172.21.15.181:0/491141654 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1706216700 1 -- 172.21.15.181:0/491141654 >> v2:172.21.15.190:3300/0 conn(0x7f17000cd2d0 msgr2=0x7f17000d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:07.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1706216700 1 --2- 172.21.15.181:0/491141654 >> v2:172.21.15.190:3300/0 conn(0x7f17000cd2d0 0x7f17000d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1706216700 1 -- 172.21.15.181:0/491141654 >> v1:172.21.15.190:6789/0 conn(0x7f17000cfc60 legacy=0x7f17000ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1706216700 1 -- 172.21.15.181:0/491141654 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17000ccad0 con 0x7f170015b990 2023-10-16T09:45:07.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1705a15700 1 --2- 172.21.15.181:0/491141654 >> v2:172.21.15.190:3300/0 conn(0x7f17000cd2d0 0x7f17000d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:07.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.458+0000 7f1706216700 1 --2- 172.21.15.181:0/491141654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170015b990 0x7f17000ce8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009820 tx=0x7f16f0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941f83d6f19584a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f1705214700 1 -- 172.21.15.181:0/491141654 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0005e30 con 0x7f170015b990 2023-10-16T09:45:07.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f1705214700 1 -- 172.21.15.181:0/491141654 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f0006030 con 0x7f170015b990 2023-10-16T09:45:07.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f1705214700 1 -- 172.21.15.181:0/491141654 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f00063c0 con 0x7f170015b990 2023-10-16T09:45:07.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 -- 172.21.15.181:0/491141654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170015b990 msgr2=0x7f17000ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 --2- 172.21.15.181:0/491141654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170015b990 0x7f17000ce8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009820 tx=0x7f16f0009b30 comp rx=0 tx=0).stop 2023-10-16T09:45:07.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 -- 172.21.15.181:0/491141654 shutdown_connections 2023-10-16T09:45:07.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 --2- 172.21.15.181:0/491141654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f170015b990 0x7f17000ce8f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 --2- 172.21.15.181:0/491141654 >> v2:172.21.15.190:3300/0 conn(0x7f17000cd2d0 0x7f17000d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 -- 172.21.15.181:0/491141654 >> 172.21.15.181:0/491141654 conn(0x7f170009a9a0 msgr2=0x7f17000d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 -- 172.21.15.181:0/491141654 shutdown_connections 2023-10-16T09:45:07.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.459+0000 7f170847a700 1 -- 172.21.15.181:0/491141654 wait complete. 2023-10-16T09:45:07.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.460+0000 7f170847a700 1 Processor -- start 2023-10-16T09:45:07.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.460+0000 7f170847a700 1 -- start start 2023-10-16T09:45:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f170847a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17000cd2d0 0x7f1700151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f170847a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17000cfc60 0x7f170015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f170847a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170015b990 0x7f17001f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f170847a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170015c9b0 con 0x7f17000cfc60 2023-10-16T09:45:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f170847a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170015cc90 con 0x7f170015b990 2023-10-16T09:45:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f170847a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170015cf70 con 0x7f17000cd2d0 2023-10-16T09:45:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f1705a15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17000cfc60 0x7f170015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f1705a15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17000cfc60 0x7f170015c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38162/0 (socket says 172.21.15.181:38162) 2023-10-16T09:45:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f1705a15700 1 -- 172.21.15.181:0/2534974745 learned_addr learned my addr 172.21.15.181:0/2534974745 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:07.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f1706216700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17000cd2d0 0x7f1700151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.461+0000 7f1706a17700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170015b990 0x7f17001f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f1705a15700 1 -- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17000cd2d0 msgr2=0x7f1700151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f1705a15700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17000cd2d0 0x7f1700151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f1705a15700 1 -- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170015b990 msgr2=0x7f17001f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f1705a15700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170015b990 0x7f17001f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f1705a15700 1 -- 172.21.15.181:0/2534974745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc009550 con 0x7f17000cfc60 2023-10-16T09:45:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f1705a15700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17000cfc60 0x7f170015c0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e9b0 tx=0x7f16fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f16f6ffd700 1 -- 172.21.15.181:0/2534974745 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00ee50 con 0x7f17000cfc60 2023-10-16T09:45:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f16f6ffd700 1 -- 172.21.15.181:0/2534974745 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00c850 con 0x7f17000cfc60 2023-10-16T09:45:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f16f6ffd700 1 -- 172.21.15.181:0/2534974745 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00cbc0 con 0x7f17000cfc60 2023-10-16T09:45:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f0009430 con 0x7f17000cfc60 2023-10-16T09:45:07.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.462+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170015d530 con 0x7f17000cfc60 2023-10-16T09:45:07.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.463+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1700067d20 con 0x7f17000cfc60 2023-10-16T09:45:07.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.463+0000 7f16f6ffd700 1 -- 172.21.15.181:0/2534974745 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f16fc00cd20 con 0x7f17000cfc60 2023-10-16T09:45:07.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.464+0000 7f16f6ffd700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f16e406af00 0x7f16e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:07.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.464+0000 7f16f6ffd700 1 -- 172.21.15.181:0/2534974745 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16fc089920 con 0x7f17000cfc60 2023-10-16T09:45:07.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.464+0000 7f1706216700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f16e406af00 0x7f16e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:07.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.464+0000 7f1706216700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f16e406af00 0x7f16e406d3c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000c00 tx=0x7f16f0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:07.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.468+0000 7f16f6ffd700 1 -- 172.21.15.181:0/2534974745 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f16fc0549e0 con 0x7f17000cfc60 2023-10-16T09:45:07.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.702+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f17000cfa40 con 0x7f17000cfc60 2023-10-16T09:45:07.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.703+0000 7f16f6ffd700 1 -- 172.21.15.181:0/2534974745 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f16fc058000 con 0x7f17000cfc60 2023-10-16T09:45:07.705 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:45:07.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f16e406af00 msgr2=0x7f16e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f16e406af00 0x7f16e406d3c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000c00 tx=0x7f16f0008070 comp rx=0 tx=0).stop 2023-10-16T09:45:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17000cfc60 msgr2=0x7f170015c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17000cfc60 0x7f170015c0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e9b0 tx=0x7f16fc00ecc0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 shutdown_connections 2023-10-16T09:45:07.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f16e406af00 0x7f16e406d3c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17000cd2d0 0x7f1700151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f170015b990 0x7f17001f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 --2- 172.21.15.181:0/2534974745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17000cfc60 0x7f170015c0b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:07.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 >> 172.21.15.181:0/2534974745 conn(0x7f170009a9a0 msgr2=0x7f17000ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:07.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 shutdown_connections 2023-10-16T09:45:07.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:07.705+0000 7f170847a700 1 -- 172.21.15.181:0/2534974745 wait complete. 2023-10-16T09:45:07.719 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T09:45:07.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:45:07.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T09:45:07.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T09:45:07.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:45:07.721 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T09:45:07.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T09:45:07.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:45:07.722 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:45:07.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:07.723 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T09:45:07.777 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:45:07.777 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:45:07.777 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:45:07.777 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:45:07.778 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T09:45:07.778 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:45:07.778 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:45:07.778 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:45:07.778 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:45:07.779 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:45:07.779 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:45:07.779 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T09:45:07.779 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:45:07.779 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T09:45:07.780 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:45:07.780 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42362sec preferred_lft 42362sec 2023-10-16T09:45:07.780 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T09:45:07.780 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:45:07.780 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:45:07.781 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:45:07.781 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:45:07.781 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:45:07.781 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T09:45:07.781 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:45:07.782 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:45:07.782 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:45:07.782 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T09:45:07.782 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:45:07.782 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:45:07.783 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T09:45:07.783 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T09:45:07.783 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:45:07.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:07.784 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:45:07.841 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T09:45:07.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:07.843 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T09:45:07.902 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T09:45:07.902 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:45:07.902 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:45:07.902 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:45:07.903 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:45:07.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:07.903 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:45:07.904 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:45:07.904 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:45:07.904 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:45:07.905 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:45:07.905 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:45:08.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:08.006 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T09:45:08.023 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T09:45:08.023 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T09:45:08.024 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:45:08.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:08.024 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T09:45:08.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:45:08.091 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T09:45:08.147 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:45:08.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:45:08.148 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:45:08.205 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T09:45:08.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:08.206 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:45:08.290 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:45:08.290 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:45:08.291 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:45:08.299 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:45:08.299 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:45:08.299 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:45:08.300 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:45:08.300 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:45:08.373 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:45:08.373 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:45:08.373 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:45:08.373 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:45:08.374 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:45:08.374 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T09:45:08.374 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T09:45:08.374 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T09:45:08.375 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:45:08.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:08.376 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:45:08.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:08.403 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:45:08.476 INFO:teuthology.orchestra.run.smithi181.stdout:client5059 2023-10-16T09:45:08.476 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:45:08.477 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:45:08.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T09:45:08.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.692+0000 7fd162a1b700 1 Processor -- start 2023-10-16T09:45:08.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.692+0000 7fd162a1b700 1 -- start start 2023-10-16T09:45:08.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.692+0000 7fd162a1b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd15c15e460 0x7fd15c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:08.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.692+0000 7fd162a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c15e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:08.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.692+0000 7fd162a1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c06ed00 con 0x7fd15c15a0f0 2023-10-16T09:45:08.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.692+0000 7fd162a1b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15c15ed80 con 0x7fd15c15e460 2023-10-16T09:45:08.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.692+0000 7fd162a1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c06e0f0 con 0x7fd15c0cb110 2023-10-16T09:45:08.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd15c15e460 0x7fd15c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:08.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd15c15e460 0x7fd15c0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38172/0 (socket says 172.21.15.181:38172) 2023-10-16T09:45:08.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 -- 172.21.15.181:0/839133183 learned_addr learned my addr 172.21.15.181:0/839133183 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:08.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15b7fe700 1 --2- 172.21.15.181:0/839133183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:08.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 -- 172.21.15.181:0/839133183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 msgr2=0x7fd15c15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:08.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 --2- 172.21.15.181:0/839133183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c15e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 -- 172.21.15.181:0/839133183 >> v1:172.21.15.190:6789/0 conn(0x7fd15c15a0f0 legacy=0x7fd15c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:08.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 -- 172.21.15.181:0/839133183 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c06e3d0 con 0x7fd15c15e460 2023-10-16T09:45:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15bfff700 1 --2- 172.21.15.181:0/839133183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15e460 0x7fd15c0d0fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe213b18f6dcb709 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15affd700 1 -- 172.21.15.181:0/839133183 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c005e30 con 0x7fd15c15e460 2023-10-16T09:45:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15affd700 1 -- 172.21.15.181:0/839133183 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c004030 con 0x7fd15c15e460 2023-10-16T09:45:08.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.693+0000 7fd15affd700 1 -- 172.21.15.181:0/839133183 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0043e0 con 0x7fd15c15e460 2023-10-16T09:45:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 -- 172.21.15.181:0/839133183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15e460 msgr2=0x7fd15c0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 --2- 172.21.15.181:0/839133183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15e460 0x7fd15c0d0fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 -- 172.21.15.181:0/839133183 shutdown_connections 2023-10-16T09:45:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 --2- 172.21.15.181:0/839133183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15e460 0x7fd15c0d0fc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 --2- 172.21.15.181:0/839133183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c15e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 -- 172.21.15.181:0/839133183 >> 172.21.15.181:0/839133183 conn(0x7fd15c0d04a0 msgr2=0x7fd15c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 -- 172.21.15.181:0/839133183 shutdown_connections 2023-10-16T09:45:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.694+0000 7fd162a1b700 1 -- 172.21.15.181:0/839133183 wait complete. 2023-10-16T09:45:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.695+0000 7fd162a1b700 1 Processor -- start 2023-10-16T09:45:08.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.695+0000 7fd162a1b700 1 -- start start 2023-10-16T09:45:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd162a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd162a1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15a0f0 0x7fd15c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd162a1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd15c15e460 0x7fd15c1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd162a1b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c0b0990 con 0x7fd15c15a0f0 2023-10-16T09:45:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd162a1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c0b0c70 con 0x7fd15c15e460 2023-10-16T09:45:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd162a1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c0b0f50 con 0x7fd15c0cb110 2023-10-16T09:45:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15a0f0 0x7fd15c0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15a0f0 0x7fd15c0c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38180/0 (socket says 172.21.15.181:38180) 2023-10-16T09:45:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 -- 172.21.15.181:0/702725738 learned_addr learned my addr 172.21.15.181:0/702725738 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15bfff700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 -- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 msgr2=0x7fd15c158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd160fb8700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd15c15e460 0x7fd15c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 -- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd15c15e460 msgr2=0x7fd15c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd15c15e460 0x7fd15c1532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 -- 172.21.15.181:0/702725738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c009500 con 0x7fd15c15a0f0 2023-10-16T09:45:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd15b7fe700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15a0f0 0x7fd15c0c8d30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd15000d800 tx=0x7fd15000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.696+0000 7fd158ff9700 1 -- 172.21.15.181:0/702725738 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500125f0 con 0x7fd15c15a0f0 2023-10-16T09:45:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.697+0000 7fd158ff9700 1 -- 172.21.15.181:0/702725738 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150012750 con 0x7fd15c15a0f0 2023-10-16T09:45:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.697+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c0b1230 con 0x7fd15c15a0f0 2023-10-16T09:45:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.697+0000 7fd158ff9700 1 -- 172.21.15.181:0/702725738 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150012ae0 con 0x7fd15c15a0f0 2023-10-16T09:45:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.697+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c0c9f80 con 0x7fd15c15a0f0 2023-10-16T09:45:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.697+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15c06f170 con 0x7fd15c15a0f0 2023-10-16T09:45:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.698+0000 7fd158ff9700 1 -- 172.21.15.181:0/702725738 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd150012c40 con 0x7fd15c15a0f0 2023-10-16T09:45:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.698+0000 7fd158ff9700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd14806af00 0x7fd14806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.698+0000 7fd158ff9700 1 -- 172.21.15.181:0/702725738 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd150089a90 con 0x7fd15c15a0f0 2023-10-16T09:45:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.698+0000 7fd15bfff700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd14806af00 0x7fd14806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.699+0000 7fd15bfff700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd14806af00 0x7fd14806d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd14c014040 tx=0x7fd14c008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.703+0000 7fd158ff9700 1 -- 172.21.15.181:0/702725738 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd15c06f170 con 0x7fd15c15a0f0 2023-10-16T09:45:08.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.926+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd15c06e990 con 0x7fd14806af00 2023-10-16T09:45:08.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.928+0000 7fd158ff9700 1 -- 172.21.15.181:0/702725738 <== mgr.4107 v2:172.21.15.181:6832/2039940952 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd14c006c40 con 0x7fd14806af00 2023-10-16T09:45:08.930 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:45:08.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:45:08.930 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T09:45:08.930 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:45:08.930 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:45:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.929+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd14806af00 msgr2=0x7fd14806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.929+0000 7fd162a1b700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd14806af00 0x7fd14806d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd14c014040 tx=0x7fd14c008070 comp rx=0 tx=0).stop 2023-10-16T09:45:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15a0f0 msgr2=0x7fd15c0c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:08.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15a0f0 0x7fd15c0c8d30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd15000d800 tx=0x7fd15000dbc0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 shutdown_connections 2023-10-16T09:45:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7fd14806af00 0x7fd14806d3c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd15c0cb110 0x7fd15c158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd15c15e460 0x7fd15c1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 --2- 172.21.15.181:0/702725738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd15c15a0f0 0x7fd15c0c8d30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:08.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 >> 172.21.15.181:0/702725738 conn(0x7fd15c0d04a0 msgr2=0x7fd15c156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:08.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 shutdown_connections 2023-10-16T09:45:08.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:08.930+0000 7fd162a1b700 1 -- 172.21.15.181:0/702725738 wait complete. 2023-10-16T09:45:08.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T09:45:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.064+0000 7f8a7ae59700 1 Processor -- start 2023-10-16T09:45:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.064+0000 7f8a7ae59700 1 -- start start 2023-10-16T09:45:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.064+0000 7f8a7ae59700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a740d07d0 0x7f8a740b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.064+0000 7f8a7ae59700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740d2960 con 0x7f8a7415d9e0 2023-10-16T09:45:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.064+0000 7f8a7ae59700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740d4180 con 0x7f8a740d24e0 2023-10-16T09:45:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a7ae59700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a740d20c0 con 0x7f8a740d07d0 2023-10-16T09:45:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a793f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a740d07d0 0x7f8a740b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a78bf5700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8a7415d9e0 0x7f8a740cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49264/0 (socket says 172.21.15.181:49264) 2023-10-16T09:45:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a78bf5700 1 -- 172.21.15.181:0/358283620 learned_addr learned my addr 172.21.15.181:0/358283620 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a793f6700 1 -- 172.21.15.181:0/358283620 >> v1:172.21.15.190:6789/0 conn(0x7f8a740d24e0 legacy=0x7f8a740d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a793f6700 1 -- 172.21.15.181:0/358283620 >> v1:172.21.15.181:6789/0 conn(0x7f8a7415d9e0 legacy=0x7f8a740cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a737fe700 1 -- 172.21.15.181:0/358283620 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966356782 0 0) 0x7f8a740d2960 con 0x7f8a7415d9e0 2023-10-16T09:45:09.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a793f6700 1 -- 172.21.15.181:0/358283620 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740d1440 con 0x7f8a740d07d0 2023-10-16T09:45:09.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a793f6700 1 --2- 172.21.15.181:0/358283620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d07d0 0x7f8a740b6620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009d90 tx=0x7f8a6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5de79700ab0ba6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:09.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a737fe700 1 -- 172.21.15.181:0/358283620 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00e070 con 0x7f8a740d07d0 2023-10-16T09:45:09.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a737fe700 1 -- 172.21.15.181:0/358283620 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c00bbd0 con 0x7f8a740d07d0 2023-10-16T09:45:09.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.065+0000 7f8a737fe700 1 -- 172.21.15.181:0/358283620 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c003a20 con 0x7f8a740d07d0 2023-10-16T09:45:09.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.066+0000 7f8a7ae59700 1 -- 172.21.15.181:0/358283620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d07d0 msgr2=0x7f8a740b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:09.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.066+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/358283620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d07d0 0x7f8a740b6620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009d90 tx=0x7f8a6c00b330 comp rx=0 tx=0).stop 2023-10-16T09:45:09.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.066+0000 7f8a7ae59700 1 -- 172.21.15.181:0/358283620 shutdown_connections 2023-10-16T09:45:09.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.066+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/358283620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d07d0 0x7f8a740b6620 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.066+0000 7f8a7ae59700 1 -- 172.21.15.181:0/358283620 >> 172.21.15.181:0/358283620 conn(0x7f8a7409cb40 msgr2=0x7f8a740d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:09.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.066+0000 7f8a7ae59700 1 -- 172.21.15.181:0/358283620 shutdown_connections 2023-10-16T09:45:09.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.066+0000 7f8a7ae59700 1 -- 172.21.15.181:0/358283620 wait complete. 2023-10-16T09:45:09.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.067+0000 7f8a7ae59700 1 Processor -- start 2023-10-16T09:45:09.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.067+0000 7f8a7ae59700 1 -- start start 2023-10-16T09:45:09.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a7ae59700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a740d07d0 0x7f8a7415ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:09.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a7ae59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d24e0 0x7f8a741f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:09.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a7ae59700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7415d9e0 0x7f8a740c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:09.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a7ae59700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a741f6c40 con 0x7f8a740d24e0 2023-10-16T09:45:09.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a7ae59700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740b6ae0 con 0x7f8a740d07d0 2023-10-16T09:45:09.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a7ae59700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740b6dc0 con 0x7f8a7415d9e0 2023-10-16T09:45:09.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d24e0 0x7f8a741f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:09.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d24e0 0x7f8a741f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38194/0 (socket says 172.21.15.181:38194) 2023-10-16T09:45:09.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 -- 172.21.15.181:0/1379814697 learned_addr learned my addr 172.21.15.181:0/1379814697 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:09.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a78bf5700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a740d07d0 0x7f8a7415ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:09.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a793f6700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7415d9e0 0x7f8a740c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:09.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 -- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7415d9e0 msgr2=0x7f8a740c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:09.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7415d9e0 0x7f8a740c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 -- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a740d07d0 msgr2=0x7f8a7415ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:09.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a740d07d0 0x7f8a7415ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a73fff700 1 -- 172.21.15.181:0/1379814697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009a70 con 0x7f8a740d24e0 2023-10-16T09:45:09.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.068+0000 7f8a793f6700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7415d9e0 0x7f8a740c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:09.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.069+0000 7f8a73fff700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d24e0 0x7f8a741f7060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c4a0 tx=0x7f8a6000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:09.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.069+0000 7f8a717fa700 1 -- 172.21.15.181:0/1379814697 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600054e0 con 0x7f8a740d24e0 2023-10-16T09:45:09.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.069+0000 7f8a717fa700 1 -- 172.21.15.181:0/1379814697 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60003680 con 0x7f8a740d24e0 2023-10-16T09:45:09.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.069+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a741f6860 con 0x7f8a740d24e0 2023-10-16T09:45:09.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.069+0000 7f8a717fa700 1 -- 172.21.15.181:0/1379814697 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60003a30 con 0x7f8a740d24e0 2023-10-16T09:45:09.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.069+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a741f75a0 con 0x7f8a740d24e0 2023-10-16T09:45:09.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.070+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a741f6d80 con 0x7f8a740d24e0 2023-10-16T09:45:09.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.070+0000 7f8a717fa700 1 -- 172.21.15.181:0/1379814697 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a60003b90 con 0x7f8a740d24e0 2023-10-16T09:45:09.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.070+0000 7f8a717fa700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8a5806af00 0x7f8a5806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:09.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.070+0000 7f8a717fa700 1 -- 172.21.15.181:0/1379814697 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a60088a90 con 0x7f8a740d24e0 2023-10-16T09:45:09.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.071+0000 7f8a78bf5700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8a5806af00 0x7f8a5806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:09.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.071+0000 7f8a78bf5700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8a5806af00 0x7f8a5806d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000f60 tx=0x7f8a68007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:09.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.075+0000 7f8a717fa700 1 -- 172.21.15.181:0/1379814697 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a60053b40 con 0x7f8a740d24e0 2023-10-16T09:45:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.326+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f8a74067d20 con 0x7f8a740d24e0 2023-10-16T09:45:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.852+0000 7f8a717fa700 1 -- 172.21.15.181:0/1379814697 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8a60017020 con 0x7f8a740d24e0 2023-10-16T09:45:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.856+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8a5806af00 msgr2=0x7f8a5806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.856+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8a5806af00 0x7f8a5806d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000f60 tx=0x7f8a68007e70 comp rx=0 tx=0).stop 2023-10-16T09:45:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.856+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d24e0 msgr2=0x7f8a741f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.856+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d24e0 0x7f8a741f7060 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c4a0 tx=0x7f8a6000c7b0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 shutdown_connections 2023-10-16T09:45:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f8a5806af00 0x7f8a5806d3c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7415d9e0 0x7f8a740c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a740d07d0 0x7f8a7415ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 --2- 172.21.15.181:0/1379814697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740d24e0 0x7f8a741f7060 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 >> 172.21.15.181:0/1379814697 conn(0x7f8a7409cb40 msgr2=0x7f8a7415f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 shutdown_connections 2023-10-16T09:45:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:09.857+0000 7f8a7ae59700 1 -- 172.21.15.181:0/1379814697 wait complete. 2023-10-16T09:45:09.873 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T09:45:09.998 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:09.996+0000 7feba4c77200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:45:10.005 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.003+0000 7f2572f86200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:45:10.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.006+0000 7f55fe364700 1 Processor -- start 2023-10-16T09:45:10.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.006+0000 7f55fe364700 1 -- start start 2023-10-16T09:45:10.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.006+0000 7f55fe364700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55fe364700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f806b230 con 0x7f55f815da70 2023-10-16T09:45:10.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55fe364700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f80cef00 con 0x7f55f806ade0 2023-10-16T09:45:10.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55fe364700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80cec20 con 0x7f55f8072760 2023-10-16T09:45:10.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f55f815da70 0x7f55f80ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49304/0 (socket says 172.21.15.181:49304) 2023-10-16T09:45:10.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f7fff700 1 -- 172.21.15.181:0/2720315534 learned_addr learned my addr 172.21.15.181:0/2720315534 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:10.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f77fe700 1 --2- 172.21.15.181:0/2720315534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f6ffd700 1 -- 172.21.15.181:0/2720315534 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085682389 0 0) 0x7f55e8001110 con 0x7f55f815da70 2023-10-16T09:45:10.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f6ffd700 1 -- 172.21.15.181:0/2720315534 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e00035b0 con 0x7f55f815da70 2023-10-16T09:45:10.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f6ffd700 1 -- 172.21.15.181:0/2720315534 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807827501 0 0) 0x7f55f80cef00 con 0x7f55f806ade0 2023-10-16T09:45:10.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f6ffd700 1 -- 172.21.15.181:0/2720315534 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e8001110 con 0x7f55f806ade0 2023-10-16T09:45:10.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f77fe700 1 -- 172.21.15.181:0/2720315534 >> v1:172.21.15.190:6789/0 conn(0x7f55f806ade0 legacy=0x7f55f81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f77fe700 1 -- 172.21.15.181:0/2720315534 >> v1:172.21.15.181:6789/0 conn(0x7f55f815da70 legacy=0x7f55f80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f77fe700 1 -- 172.21.15.181:0/2720315534 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8071c80 con 0x7f55f8072760 2023-10-16T09:45:10.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.007+0000 7f55f77fe700 1 --2- 172.21.15.181:0/2720315534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f806a8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009870 tx=0x7f55ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6da8f7fb29fca59 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:10.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55f6ffd700 1 -- 172.21.15.181:0/2720315534 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011830 con 0x7f55f8072760 2023-10-16T09:45:10.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55f6ffd700 1 -- 172.21.15.181:0/2720315534 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec011990 con 0x7f55f8072760 2023-10-16T09:45:10.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55f6ffd700 1 -- 172.21.15.181:0/2720315534 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011d20 con 0x7f55f8072760 2023-10-16T09:45:10.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55fe364700 1 -- 172.21.15.181:0/2720315534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 msgr2=0x7f55f806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55fe364700 1 --2- 172.21.15.181:0/2720315534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f806a8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009870 tx=0x7f55ec009b80 comp rx=0 tx=0).stop 2023-10-16T09:45:10.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55fe364700 1 -- 172.21.15.181:0/2720315534 shutdown_connections 2023-10-16T09:45:10.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55fe364700 1 --2- 172.21.15.181:0/2720315534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f806a8a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55fe364700 1 -- 172.21.15.181:0/2720315534 >> 172.21.15.181:0/2720315534 conn(0x7f55f80cd010 msgr2=0x7f55f80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:10.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55fe364700 1 -- 172.21.15.181:0/2720315534 shutdown_connections 2023-10-16T09:45:10.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.008+0000 7f55fe364700 1 -- 172.21.15.181:0/2720315534 wait complete. 2023-10-16T09:45:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.009+0000 7f55fe364700 1 Processor -- start 2023-10-16T09:45:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fe364700 1 -- start start 2023-10-16T09:45:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fe364700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55f806ade0 0x7f55f81f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fe364700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f80c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fe364700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f815da70 0x7f55f80ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fe364700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c5a20 con 0x7f55f815da70 2023-10-16T09:45:10.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fe364700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f81f30a0 con 0x7f55f806ade0 2023-10-16T09:45:10.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fe364700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c4600 con 0x7f55f8072760 2023-10-16T09:45:10.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fc901700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f815da70 0x7f55f80ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55f7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55f806ade0 0x7f55f81f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55f7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55f806ade0 0x7f55f81f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:43282/0 (socket says 172.21.15.181:43282) 2023-10-16T09:45:10.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55fc901700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f815da70 0x7f55f80ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38236/0 (socket says 172.21.15.181:38236) 2023-10-16T09:45:10.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.010+0000 7f55f7fff700 1 -- 172.21.15.181:0/3254391000 learned_addr learned my addr 172.21.15.181:0/3254391000 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55f77fe700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f80c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fc901700 1 -- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 msgr2=0x7f55f80c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fc901700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f80c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fc901700 1 -- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55f806ade0 msgr2=0x7f55f81f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fc901700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55f806ade0 0x7f55f81f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fc901700 1 -- 172.21.15.181:0/3254391000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0094a0 con 0x7f55f815da70 2023-10-16T09:45:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55f7fff700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55f806ade0 0x7f55f81f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fc901700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f815da70 0x7f55f80ac880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f55f000cb10 tx=0x7f55f000ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55f4ff9700 1 -- 172.21.15.181:0/3254391000 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0003c00 con 0x7f55f815da70 2023-10-16T09:45:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55f4ff9700 1 -- 172.21.15.181:0/3254391000 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0003d60 con 0x7f55f815da70 2023-10-16T09:45:10.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55f4ff9700 1 -- 172.21.15.181:0/3254391000 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000aa20 con 0x7f55f815da70 2023-10-16T09:45:10.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f80af830 con 0x7f55f815da70 2023-10-16T09:45:10.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.011+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f80acdc0 con 0x7f55f815da70 2023-10-16T09:45:10.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.012+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f806f6c0 con 0x7f55f815da70 2023-10-16T09:45:10.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.013+0000 7f55f4ff9700 1 -- 172.21.15.181:0/3254391000 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f55f000ab80 con 0x7f55f815da70 2023-10-16T09:45:10.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.013+0000 7f55f4ff9700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 0x7f55e406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.013+0000 7f55f7fff700 1 -- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 msgr2=0x7f55e406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:10.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.013+0000 7f55f4ff9700 1 -- 172.21.15.181:0/3254391000 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55f0089500 con 0x7f55f815da70 2023-10-16T09:45:10.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.013+0000 7f55f7fff700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 0x7f55e406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:45:10.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.017+0000 7f55f4ff9700 1 -- 172.21.15.181:0/3254391000 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f55f00545b0 con 0x7f55f815da70 2023-10-16T09:45:10.065 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.063+0000 7feba4c77200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:45:10.085 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.083+0000 7f2572f86200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:45:10.137 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.136+0000 7feba4c77200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:45:10.159 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.157+0000 7f2572f86200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:45:10.214 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.212+0000 7feba4c77200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:45:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.214+0000 7f55f7fff700 1 -- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 msgr2=0x7f55e406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.214+0000 7f55f7fff700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 0x7f55e406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:45:10.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.232+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f55f80af550 con 0x7f55f815da70 2023-10-16T09:45:10.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.235+0000 7f55f4ff9700 1 -- 172.21.15.181:0/3254391000 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f55f0057bd0 con 0x7f55f815da70 2023-10-16T09:45:10.238 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.236+0000 7f2572f86200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:45:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 msgr2=0x7f55e406d410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:45:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 0x7f55e406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f815da70 msgr2=0x7f55f80ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f815da70 0x7f55f80ac880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f55f000cb10 tx=0x7f55f000ce20 comp rx=0 tx=0).stop 2023-10-16T09:45:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 shutdown_connections 2023-10-16T09:45:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f55e406af50 0x7f55e406d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f815da70 0x7f55f80ac880 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55f806ade0 0x7f55f81f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.237+0000 7f55fe364700 1 --2- 172.21.15.181:0/3254391000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55f8072760 0x7f55f80c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.238+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 >> 172.21.15.181:0/3254391000 conn(0x7f55f80cd010 msgr2=0x7f55f80ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.238+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 shutdown_connections 2023-10-16T09:45:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.238+0000 7f55fe364700 1 -- 172.21.15.181:0/3254391000 wait complete. 2023-10-16T09:45:10.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T09:45:10.355 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.353+0000 7feba4c77200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:45:10.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.375+0000 7f7e731aa700 1 Processor -- start 2023-10-16T09:45:10.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.375+0000 7f7e731aa700 1 -- start start 2023-10-16T09:45:10.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e731aa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e6c15ea40 0x7f7e6c0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e731aa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c0cf580 con 0x7f7e6c0d01d0 2023-10-16T09:45:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e731aa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c0ce970 con 0x7f7e6c0ce2d0 2023-10-16T09:45:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e731aa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c0d4290 con 0x7f7e6c15ea40 2023-10-16T09:45:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e71747700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e6c15ea40 0x7f7e6c0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e70f46700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e6c0d01d0 0x7f7e6c0d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49364/0 (socket says 172.21.15.181:49364) 2023-10-16T09:45:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e70f46700 1 -- 172.21.15.181:0/118912565 learned_addr learned my addr 172.21.15.181:0/118912565 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e71747700 1 -- 172.21.15.181:0/118912565 >> v1:172.21.15.190:6789/0 conn(0x7f7e6c0ce2d0 legacy=0x7f7e6c0d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e71747700 1 -- 172.21.15.181:0/118912565 >> v1:172.21.15.181:6789/0 conn(0x7f7e6c0d01d0 legacy=0x7f7e6c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e6b7fe700 1 -- 172.21.15.181:0/118912565 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076980802 0 0) 0x7f7e6c0ce970 con 0x7f7e6c0ce2d0 2023-10-16T09:45:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e71747700 1 -- 172.21.15.181:0/118912565 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0cec50 con 0x7f7e6c15ea40 2023-10-16T09:45:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e71747700 1 --2- 172.21.15.181:0/118912565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c15ea40 0x7f7e6c0d1c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009d90 tx=0x7f7e6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cae7c7eb45c4a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:10.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e6b7fe700 1 -- 172.21.15.181:0/118912565 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400e070 con 0x7f7e6c15ea40 2023-10-16T09:45:10.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e6b7fe700 1 -- 172.21.15.181:0/118912565 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6400bad0 con 0x7f7e6c15ea40 2023-10-16T09:45:10.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.377+0000 7f7e6b7fe700 1 -- 172.21.15.181:0/118912565 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400be80 con 0x7f7e6c15ea40 2023-10-16T09:45:10.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.378+0000 7f7e731aa700 1 -- 172.21.15.181:0/118912565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c15ea40 msgr2=0x7f7e6c0d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.378+0000 7f7e731aa700 1 --2- 172.21.15.181:0/118912565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c15ea40 0x7f7e6c0d1c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009d90 tx=0x7f7e6400b330 comp rx=0 tx=0).stop 2023-10-16T09:45:10.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.378+0000 7f7e731aa700 1 -- 172.21.15.181:0/118912565 shutdown_connections 2023-10-16T09:45:10.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.378+0000 7f7e731aa700 1 --2- 172.21.15.181:0/118912565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c15ea40 0x7f7e6c0d1c10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.378+0000 7f7e731aa700 1 -- 172.21.15.181:0/118912565 >> 172.21.15.181:0/118912565 conn(0x7f7e6c09cb60 msgr2=0x7f7e6c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:10.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.378+0000 7f7e731aa700 1 -- 172.21.15.181:0/118912565 shutdown_connections 2023-10-16T09:45:10.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.378+0000 7f7e731aa700 1 -- 172.21.15.181:0/118912565 wait complete. 2023-10-16T09:45:10.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.379+0000 7f7e731aa700 1 Processor -- start 2023-10-16T09:45:10.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.379+0000 7f7e731aa700 1 -- start start 2023-10-16T09:45:10.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e731aa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e6c0ce2d0 0x7f7e6c0c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e731aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c0d01d0 0x7f7e6c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e731aa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6c15ea40 0x7f7e6c0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e731aa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0c8f40 con 0x7f7e6c0d01d0 2023-10-16T09:45:10.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e731aa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0c8390 con 0x7f7e6c0ce2d0 2023-10-16T09:45:10.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e731aa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b3330 con 0x7f7e6c15ea40 2023-10-16T09:45:10.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e6bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c0d01d0 0x7f7e6c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e6bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c0d01d0 0x7f7e6c0c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38294/0 (socket says 172.21.15.181:38294) 2023-10-16T09:45:10.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e6bfff700 1 -- 172.21.15.181:0/503064180 learned_addr learned my addr 172.21.15.181:0/503064180 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:10.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e71747700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6c15ea40 0x7f7e6c0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.380+0000 7f7e70f46700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e6c0ce2d0 0x7f7e6c0c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e71747700 1 -- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e6c0ce2d0 msgr2=0x7f7e6c0c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e71747700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e6c0ce2d0 0x7f7e6c0c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e71747700 1 -- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c0d01d0 msgr2=0x7f7e6c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e71747700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c0d01d0 0x7f7e6c0c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e71747700 1 -- 172.21.15.181:0/503064180 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e64009a70 con 0x7f7e6c15ea40 2023-10-16T09:45:10.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e70f46700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e6c0ce2d0 0x7f7e6c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:10.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e6bfff700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c0d01d0 0x7f7e6c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:10.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e71747700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6c15ea40 0x7f7e6c0ca890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e64015000 tx=0x7f7e64004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:10.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e697fa700 1 -- 172.21.15.181:0/503064180 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64012630 con 0x7f7e6c15ea40 2023-10-16T09:45:10.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e697fa700 1 -- 172.21.15.181:0/503064180 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e64012790 con 0x7f7e6c15ea40 2023-10-16T09:45:10.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c0b3610 con 0x7f7e6c15ea40 2023-10-16T09:45:10.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e697fa700 1 -- 172.21.15.181:0/503064180 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64012b40 con 0x7f7e6c15ea40 2023-10-16T09:45:10.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.381+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c0b3bd0 con 0x7f7e6c15ea40 2023-10-16T09:45:10.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.382+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c04ebb0 con 0x7f7e6c15ea40 2023-10-16T09:45:10.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.382+0000 7f7e697fa700 1 -- 172.21.15.181:0/503064180 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f7e64012ca0 con 0x7f7e6c15ea40 2023-10-16T09:45:10.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.383+0000 7f7e697fa700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 0x7f7e5006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.383+0000 7f7e70f46700 1 -- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 msgr2=0x7f7e5006d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:10.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.383+0000 7f7e697fa700 1 -- 172.21.15.181:0/503064180 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e6408b850 con 0x7f7e6c15ea40 2023-10-16T09:45:10.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.383+0000 7f7e70f46700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 0x7f7e5006d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:45:10.393 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.385+0000 7f2572f86200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:45:10.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.388+0000 7f7e697fa700 1 -- 172.21.15.181:0/503064180 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e64056990 con 0x7f7e6c15ea40 2023-10-16T09:45:10.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.584+0000 7f7e70f46700 1 -- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 msgr2=0x7f7e5006d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:10.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.584+0000 7f7e70f46700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 0x7f7e5006d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:45:10.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.589+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f7e6c0d0e00 con 0x7f7e6c15ea40 2023-10-16T09:45:10.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.597+0000 7f7e697fa700 1 -- 172.21.15.181:0/503064180 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7e64059fb0 con 0x7f7e6c15ea40 2023-10-16T09:45:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 msgr2=0x7f7e5006d410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:45:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 0x7f7e5006d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6c15ea40 msgr2=0x7f7e6c0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6c15ea40 0x7f7e6c0ca890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e64015000 tx=0x7f7e64004ad0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 shutdown_connections 2023-10-16T09:45:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7f7e5006af50 0x7f7e5006d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e6c15ea40 0x7f7e6c0ca890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e6c0ce2d0 0x7f7e6c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 --2- 172.21.15.181:0/503064180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e6c0d01d0 0x7f7e6c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.599+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 >> 172.21.15.181:0/503064180 conn(0x7f7e6c09cb60 msgr2=0x7f7e6c15eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:10.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.600+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 shutdown_connections 2023-10-16T09:45:10.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.600+0000 7f7e731aa700 1 -- 172.21.15.181:0/503064180 wait complete. 2023-10-16T09:45:10.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:10.614 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T09:45:10.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:10.634 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-16T09:45:10.678 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.677+0000 7feba4c77200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:45:10.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1m 2023-10-16T09:45:10.727 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.725+0000 7f2572f86200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:45:10.793 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.791+0000 7feba4c77200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:45:10.848 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.847+0000 7f2572f86200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:45:10.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.851+0000 7ff11defe700 1 Processor -- start 2023-10-16T09:45:10.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff11defe700 1 -- start start 2023-10-16T09:45:10.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff11defe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1180b2010 0x7ff1180cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff11defe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff118071be0 0x7ff1180b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff11defe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1180ce1f0 con 0x7ff11806f3b0 2023-10-16T09:45:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff11defe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff118072090 con 0x7ff1180b2010 2023-10-16T09:45:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff11defe700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1180b2930 con 0x7ff118071be0 2023-10-16T09:45:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff1177fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1180b2010 0x7ff1180cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.852+0000 7ff117fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff11806f3b0 0x7ff11806f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49426/0 (socket says 172.21.15.181:49426) 2023-10-16T09:45:10.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff117fff700 1 -- 172.21.15.181:0/4236163730 learned_addr learned my addr 172.21.15.181:0/4236163730 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff116ffd700 1 --2- 172.21.15.181:0/4236163730 >> v2:172.21.15.190:3300/0 conn(0x7ff118071be0 0x7ff1180b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1177fe700 1 -- 172.21.15.181:0/4236163730 >> v2:172.21.15.190:3300/0 conn(0x7ff118071be0 msgr2=0x7ff1180b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1177fe700 1 --2- 172.21.15.181:0/4236163730 >> v2:172.21.15.190:3300/0 conn(0x7ff118071be0 0x7ff1180b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1177fe700 1 -- 172.21.15.181:0/4236163730 >> v1:172.21.15.181:6789/0 conn(0x7ff11806f3b0 legacy=0x7ff11806f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1177fe700 1 -- 172.21.15.181:0/4236163730 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1180ccda0 con 0x7ff1180b2010 2023-10-16T09:45:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1167fc700 1 -- 172.21.15.181:0/4236163730 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585929920 0 0) 0x7ff1180ce1f0 con 0x7ff11806f3b0 2023-10-16T09:45:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1177fe700 1 --2- 172.21.15.181:0/4236163730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1180b2010 0x7ff1180cc6e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff11815e1c0 tx=0x7ff10800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a294d6f2c83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1167fc700 1 -- 172.21.15.181:0/4236163730 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10800e070 con 0x7ff1180b2010 2023-10-16T09:45:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1167fc700 1 -- 172.21.15.181:0/4236163730 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10800bb20 con 0x7ff1180b2010 2023-10-16T09:45:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.853+0000 7ff1167fc700 1 -- 172.21.15.181:0/4236163730 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108012430 con 0x7ff1180b2010 2023-10-16T09:45:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 -- 172.21.15.181:0/4236163730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1180b2010 msgr2=0x7ff1180cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 --2- 172.21.15.181:0/4236163730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1180b2010 0x7ff1180cc6e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff11815e1c0 tx=0x7ff10800b330 comp rx=0 tx=0).stop 2023-10-16T09:45:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 -- 172.21.15.181:0/4236163730 shutdown_connections 2023-10-16T09:45:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 --2- 172.21.15.181:0/4236163730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1180b2010 0x7ff1180cc6e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 --2- 172.21.15.181:0/4236163730 >> v2:172.21.15.190:3300/0 conn(0x7ff118071be0 0x7ff1180b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 -- 172.21.15.181:0/4236163730 >> 172.21.15.181:0/4236163730 conn(0x7ff1180cd010 msgr2=0x7ff1180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 -- 172.21.15.181:0/4236163730 shutdown_connections 2023-10-16T09:45:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.854+0000 7ff11defe700 1 -- 172.21.15.181:0/4236163730 wait complete. 2023-10-16T09:45:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.855+0000 7ff11defe700 1 Processor -- start 2023-10-16T09:45:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.855+0000 7ff11defe700 1 -- start start 2023-10-16T09:45:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff11defe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11806f3b0 0x7ff118150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff11defe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff118071be0 0x7ff1180b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff11defe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1180b2010 0x7ff1180b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff11defe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11815ad00 con 0x7ff11806f3b0 2023-10-16T09:45:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff11defe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11815d280 con 0x7ff1180b2010 2023-10-16T09:45:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff11defe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11815d710 con 0x7ff118071be0 2023-10-16T09:45:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff1177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11806f3b0 0x7ff118150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff1177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11806f3b0 0x7ff118150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38352/0 (socket says 172.21.15.181:38352) 2023-10-16T09:45:10.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff1177fe700 1 -- 172.21.15.181:0/1644768901 learned_addr learned my addr 172.21.15.181:0/1644768901 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff117fff700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1180b2010 0x7ff1180b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.856+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff118071be0 0x7ff1180b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff1177fe700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff118071be0 msgr2=0x7ff1180b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff1177fe700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff118071be0 0x7ff1180b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff1177fe700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1180b2010 msgr2=0x7ff1180b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff1177fe700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1180b2010 0x7ff1180b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff1177fe700 1 -- 172.21.15.181:0/1644768901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108009a70 con 0x7ff11806f3b0 2023-10-16T09:45:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff118071be0 0x7ff1180b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff117fff700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1180b2010 0x7ff1180b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff1177fe700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11806f3b0 0x7ff118150b60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff108014040 tx=0x7ff108004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108012430 con 0x7ff11806f3b0 2023-10-16T09:45:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108020070 con 0x7ff11806f3b0 2023-10-16T09:45:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1180ca0c0 con 0x7ff11806f3b0 2023-10-16T09:45:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.857+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108004280 con 0x7ff11806f3b0 2023-10-16T09:45:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.858+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11815cc30 con 0x7ff11806f3b0 2023-10-16T09:45:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.859+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11815d560 con 0x7ff11806f3b0 2023-10-16T09:45:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.859+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7ff1080043e0 con 0x7ff11806f3b0 2023-10-16T09:45:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.860+0000 7ff0fffff700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 0x7ff10406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.860+0000 7ff116ffd700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 msgr2=0x7ff10406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.860+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff108022070 con 0x7ff11806f3b0 2023-10-16T09:45:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.860+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 0x7ff10406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:45:10.870 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.865+0000 7feba4c77200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:45:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:10.864+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff108056330 con 0x7ff11806f3b0 2023-10-16T09:45:10.927 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:10.925+0000 7f2572f86200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:45:10.947 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:10.945+0000 7feba4c77200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:45:11.008 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:11.006+0000 7f2572f86200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:45:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:11.061+0000 7ff116ffd700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 msgr2=0x7ff10406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:11.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:11.061+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 0x7ff10406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:45:11.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:11.091+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 --> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff118072ac0 con 0x7ff10406af50 2023-10-16T09:45:11.314 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:11.313+0000 7feba4c77200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:45:11.380 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:11.378+0000 7f2572f86200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:45:11.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:11.462+0000 7ff116ffd700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 msgr2=0x7ff10406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:11.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:11.462+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 0x7ff10406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T09:45:11.878 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:11.876+0000 7feba4c77200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:45:11.946 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:11.944+0000 7f2572f86200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:45:11.947 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:11.945+0000 7feba4c77200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:45:12.014 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:12.013+0000 7f2572f86200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:45:12.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:12.263+0000 7ff116ffd700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 msgr2=0x7ff10406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:12.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:12.263+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 0x7ff10406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T09:45:12.572 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:12.570+0000 7feba4c77200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:45:12.642 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:12.641+0000 7f2572f86200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:45:12.648 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:12.646+0000 7feba4c77200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:45:12.719 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:12.717+0000 7f2572f86200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:45:12.790 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:12.788+0000 7feba4c77200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:45:12.865 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:12.863+0000 7f2572f86200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:45:13.018 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:13.016+0000 7feba4c77200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:45:13.090 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:13.088+0000 7feba4c77200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:45:13.092 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:13.090+0000 7f2572f86200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:45:13.164 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:13.163+0000 7f2572f86200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:45:13.353 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:13.351+0000 7feba4c77200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:45:13.428 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:13.426+0000 7f2572f86200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:45:13.568 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:13.566+0000 7feba4c77200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:45:13.647 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:13.645+0000 7f2572f86200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:45:13.714 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:13.712+0000 7feba4c77200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:45:13.795 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:13.793+0000 7f2572f86200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:45:13.826 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:13.823+0000 7feba4c77200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:45:13.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:13.864+0000 7ff116ffd700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 msgr2=0x7ff10406d410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2039940952 2023-10-16T09:45:13.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:13.864+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 0x7ff10406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T09:45:13.907 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:13.905+0000 7f2572f86200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:45:13.977 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:13.974+0000 7feba4c77200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:45:14.061 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:14.059+0000 7f2572f86200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:45:14.238 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:14.234+0000 7feba4c77200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:45:14.322 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:14.321+0000 7f2572f86200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:45:14.393 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:14.390+0000 7feba4c77200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:45:14.480 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:14.478+0000 7f2572f86200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:45:14.906 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:45:14.904+0000 7feba4c77200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:45:14.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:14.974+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 7) v1 ==== 89246+0+0 (secure 0 0 0) 0x7ff108056700 con 0x7ff11806f3b0 2023-10-16T09:45:15.002 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.000+0000 7f2572f86200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:45:15.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:15.025+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 8) v1 ==== 88957+0+0 (secure 0 0 0) 0x7ff1080576e0 con 0x7ff11806f3b0 2023-10-16T09:45:15.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:15.026+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 msgr2=0x7ff10406d410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:45:15.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:15.026+0000 7ff0fffff700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/2039940952,v1:172.21.15.181:6833/2039940952] conn(0x7ff10406af50 0x7ff10406d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:15.210 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e5f61700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.210 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e5f61700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.210 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e5f61700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.211 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e5f61700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.211 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e5f61700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.211 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e7f65700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.212 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e7f65700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.212 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e7f65700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.212 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e7f65700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:15.213 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:45:15.208+0000 7f24e7f65700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:45:16.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.027+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 9) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff108056fd0 con 0x7ff11806f3b0 2023-10-16T09:45:16.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.027+0000 7ff0fffff700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff10401ca90 0x7ff10401ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.027+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff10401d3b0 con 0x7ff10401ca90 2023-10-16T09:45:16.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.028+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff10401ca90 0x7ff10401ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.030+0000 7ff116ffd700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff10401ca90 0x7ff10401ce70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00a010 tx=0x7ff10c009210 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:16.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.119+0000 7ff0fffff700 1 -- 172.21.15.181:0/1644768901 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff10401d3b0 con 0x7ff10401ca90 2023-10-16T09:45:16.122 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-16T09:45:16.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff10401ca90 msgr2=0x7ff10401ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff10401ca90 0x7ff10401ce70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00a010 tx=0x7ff10c009210 comp rx=0 tx=0).stop 2023-10-16T09:45:16.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11806f3b0 msgr2=0x7ff118150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 --2- 172.21.15.181:0/1644768901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11806f3b0 0x7ff118150b60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff108014040 tx=0x7ff108004ad0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff117fff700 1 -- 172.21.15.181:0/1644768901 reap_dead start 2023-10-16T09:45:16.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 shutdown_connections 2023-10-16T09:45:16.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 >> 172.21.15.181:0/1644768901 conn(0x7ff1180cd010 msgr2=0x7ff1180afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:16.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 shutdown_connections 2023-10-16T09:45:16.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.122+0000 7ff11defe700 1 -- 172.21.15.181:0/1644768901 wait complete. 2023-10-16T09:45:16.137 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-16T09:45:16.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:16.137 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-16T09:45:16.156 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1m 2023-10-16T09:45:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.316+0000 7f3217245700 1 Processor -- start 2023-10-16T09:45:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.316+0000 7f3217245700 1 -- start start 2023-10-16T09:45:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f3217245700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32100cdd00 0x7f321006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f3217245700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32100ce0e0 0x7f321006afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f3217245700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100ca300 con 0x7f32100666a0 2023-10-16T09:45:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f3217245700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321006e5e0 con 0x7f32100ce0e0 2023-10-16T09:45:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f3217245700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100ca580 con 0x7f32100cdd00 2023-10-16T09:45:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f32157e2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32100ce0e0 0x7f321006afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f320ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f32100666a0 0x7f321006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49628/0 (socket says 172.21.15.181:49628) 2023-10-16T09:45:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f320ffff700 1 -- 172.21.15.181:0/1365015805 learned_addr learned my addr 172.21.15.181:0/1365015805 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f3214fe1700 1 --2- 172.21.15.181:0/1365015805 >> v2:172.21.15.190:3300/0 conn(0x7f32100cdd00 0x7f321006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f32157e2700 1 -- 172.21.15.181:0/1365015805 >> v2:172.21.15.190:3300/0 conn(0x7f32100cdd00 msgr2=0x7f321006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f32157e2700 1 --2- 172.21.15.181:0/1365015805 >> v2:172.21.15.190:3300/0 conn(0x7f32100cdd00 0x7f321006f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f32157e2700 1 -- 172.21.15.181:0/1365015805 >> v1:172.21.15.181:6789/0 conn(0x7f32100666a0 legacy=0x7f321006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f32157e2700 1 -- 172.21.15.181:0/1365015805 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100ca860 con 0x7f32100ce0e0 2023-10-16T09:45:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.317+0000 7f320f7fe700 1 -- 172.21.15.181:0/1365015805 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284792964 0 0) 0x7f32100ca300 con 0x7f32100666a0 2023-10-16T09:45:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f32157e2700 1 --2- 172.21.15.181:0/1365015805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100ce0e0 0x7f321006afb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3210070b50 tx=0x7f320800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c913e2394c9c7f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f320f7fe700 1 -- 172.21.15.181:0/1365015805 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320800e070 con 0x7f32100ce0e0 2023-10-16T09:45:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f320f7fe700 1 -- 172.21.15.181:0/1365015805 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320800bbd0 con 0x7f32100ce0e0 2023-10-16T09:45:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f320f7fe700 1 -- 172.21.15.181:0/1365015805 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208012430 con 0x7f32100ce0e0 2023-10-16T09:45:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 -- 172.21.15.181:0/1365015805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100ce0e0 msgr2=0x7f321006afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 --2- 172.21.15.181:0/1365015805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100ce0e0 0x7f321006afb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3210070b50 tx=0x7f320800b330 comp rx=0 tx=0).stop 2023-10-16T09:45:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 -- 172.21.15.181:0/1365015805 shutdown_connections 2023-10-16T09:45:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 --2- 172.21.15.181:0/1365015805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100ce0e0 0x7f321006afb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 --2- 172.21.15.181:0/1365015805 >> v2:172.21.15.190:3300/0 conn(0x7f32100cdd00 0x7f321006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 -- 172.21.15.181:0/1365015805 >> 172.21.15.181:0/1365015805 conn(0x7f32100cc730 msgr2=0x7f32100ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 -- 172.21.15.181:0/1365015805 shutdown_connections 2023-10-16T09:45:16.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.318+0000 7f3217245700 1 -- 172.21.15.181:0/1365015805 wait complete. 2023-10-16T09:45:16.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.320+0000 7f3217245700 1 Processor -- start 2023-10-16T09:45:16.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.320+0000 7f3217245700 1 -- start start 2023-10-16T09:45:16.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.320+0000 7f3217245700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100666a0 0x7f32100c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.320+0000 7f3217245700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32100cdd00 0x7f32101e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3217245700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32100ce0e0 0x7f321014e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3217245700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32101e9720 con 0x7f32100666a0 2023-10-16T09:45:16.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3217245700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100c7460 con 0x7f32100ce0e0 2023-10-16T09:45:16.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3217245700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321014e540 con 0x7f32100cdd00 2023-10-16T09:45:16.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100666a0 0x7f32100c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100666a0 0x7f32100c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38556/0 (socket says 172.21.15.181:38556) 2023-10-16T09:45:16.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 -- 172.21.15.181:0/2279237834 learned_addr learned my addr 172.21.15.181:0/2279237834 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:16.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f32157e2700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32100ce0e0 0x7f321014e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f320ffff700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32100cdd00 0x7f32101e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 -- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32100cdd00 msgr2=0x7f32101e91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32100cdd00 0x7f32101e91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 -- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32100ce0e0 msgr2=0x7f321014e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32100ce0e0 0x7f321014e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 -- 172.21.15.181:0/2279237834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208009a70 con 0x7f32100666a0 2023-10-16T09:45:16.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3214fe1700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100666a0 0x7f32100c6d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f320400e810 tx=0x7f320400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:16.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f320d7fa700 1 -- 172.21.15.181:0/2279237834 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204013610 con 0x7f32100666a0 2023-10-16T09:45:16.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f320d7fa700 1 -- 172.21.15.181:0/2279237834 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3204013770 con 0x7f32100666a0 2023-10-16T09:45:16.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f320d7fa700 1 -- 172.21.15.181:0/2279237834 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204013b20 con 0x7f32100666a0 2023-10-16T09:45:16.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32100c9690 con 0x7f32100666a0 2023-10-16T09:45:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.321+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100c9c50 con 0x7f32100666a0 2023-10-16T09:45:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.322+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32101e6c60 con 0x7f32100666a0 2023-10-16T09:45:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.323+0000 7f320d7fa700 1 -- 172.21.15.181:0/2279237834 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3204013c80 con 0x7f32100666a0 2023-10-16T09:45:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.323+0000 7f320d7fa700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f31f406ada0 0x7f31f406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.323+0000 7f320d7fa700 1 -- 172.21.15.181:0/2279237834 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f32040890d0 con 0x7f32100666a0 2023-10-16T09:45:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.323+0000 7f320ffff700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f31f406ada0 0x7f31f406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.323+0000 7f320ffff700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f31f406ada0 0x7f31f406d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f32100c6730 tx=0x7f31fc007be0 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:16.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.328+0000 7f320d7fa700 1 -- 172.21.15.181:0/2279237834 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3204054340 con 0x7f32100666a0 2023-10-16T09:45:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.543+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f32100cd5b0 con 0x7f31f406ada0 2023-10-16T09:45:16.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.634+0000 7f320d7fa700 1 -- 172.21.15.181:0/2279237834 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f31fc005aa0 con 0x7f31f406ada0 2023-10-16T09:45:16.637 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-16T09:45:16.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f31f406ada0 msgr2=0x7f31f406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f31f406ada0 0x7f31f406d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f32100c6730 tx=0x7f31fc007be0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100666a0 msgr2=0x7f32100c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100666a0 0x7f32100c6d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f320400e810 tx=0x7f320400ebd0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 shutdown_connections 2023-10-16T09:45:16.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f31f406ada0 0x7f31f406d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32100ce0e0 0x7f321014e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32100666a0 0x7f32100c6d80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 --2- 172.21.15.181:0/2279237834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32100cdd00 0x7f32101e91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 >> 172.21.15.181:0/2279237834 conn(0x7f32100cc730 msgr2=0x7f321006f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:16.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 shutdown_connections 2023-10-16T09:45:16.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.636+0000 7f3217245700 1 -- 172.21.15.181:0/2279237834 wait complete. 2023-10-16T09:45:16.650 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-16T09:45:16.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:16.651 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-16T09:45:16.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1m 2023-10-16T09:45:16.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.831+0000 7fa0782ce700 1 Processor -- start 2023-10-16T09:45:16.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.831+0000 7fa0782ce700 1 -- start start 2023-10-16T09:45:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa0782ce700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa070072760 0x7fa07006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa0782ce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa07006ade0 0x7fa0701615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa0782ce700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07006b210 con 0x7fa07015da70 2023-10-16T09:45:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa0782ce700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0700cef00 con 0x7fa070072760 2023-10-16T09:45:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa0782ce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700cec20 con 0x7fa07006ade0 2023-10-16T09:45:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa07606a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa07015da70 0x7fa0700ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49642/0 (socket says 172.21.15.181:49642) 2023-10-16T09:45:16.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa07606a700 1 -- 172.21.15.181:0/1584120464 learned_addr learned my addr 172.21.15.181:0/1584120464 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:16.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa07686b700 1 --2- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa07006ade0 0x7fa0701615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.832+0000 7fa075869700 1 --2- 172.21.15.181:0/1584120464 >> v2:172.21.15.190:3300/0 conn(0x7fa070072760 0x7fa07006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075068700 1 -- 172.21.15.181:0/1584120464 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766758128 0 0) 0x7fa07006b210 con 0x7fa07015da70 2023-10-16T09:45:16.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075068700 1 -- 172.21.15.181:0/1584120464 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c0035b0 con 0x7fa07015da70 2023-10-16T09:45:16.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075869700 1 -- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa07006ade0 msgr2=0x7fa0701615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:16.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075869700 1 --2- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa07006ade0 0x7fa0701615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075869700 1 -- 172.21.15.181:0/1584120464 >> v1:172.21.15.181:6789/0 conn(0x7fa07015da70 legacy=0x7fa0700ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075869700 1 -- 172.21.15.181:0/1584120464 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa070071c80 con 0x7fa070072760 2023-10-16T09:45:16.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa07686b700 1 --2- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa07006ade0 0x7fa0701615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:16.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075869700 1 --2- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070072760 0x7fa07006a8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009840 tx=0x7fa06c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af71a84ddffd51b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:16.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075068700 1 -- 172.21.15.181:0/1584120464 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c011830 con 0x7fa070072760 2023-10-16T09:45:16.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075068700 1 -- 172.21.15.181:0/1584120464 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c011990 con 0x7fa070072760 2023-10-16T09:45:16.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.833+0000 7fa075068700 1 -- 172.21.15.181:0/1584120464 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c011d00 con 0x7fa070072760 2023-10-16T09:45:16.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 -- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070072760 msgr2=0x7fa07006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 --2- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070072760 0x7fa07006a8a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009840 tx=0x7fa06c009b50 comp rx=0 tx=0).stop 2023-10-16T09:45:16.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 -- 172.21.15.181:0/1584120464 shutdown_connections 2023-10-16T09:45:16.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 --2- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa07006ade0 0x7fa0701615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 --2- 172.21.15.181:0/1584120464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa070072760 0x7fa07006a8a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 -- 172.21.15.181:0/1584120464 >> 172.21.15.181:0/1584120464 conn(0x7fa0700cd010 msgr2=0x7fa0700cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:16.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 -- 172.21.15.181:0/1584120464 shutdown_connections 2023-10-16T09:45:16.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.834+0000 7fa0782ce700 1 -- 172.21.15.181:0/1584120464 wait complete. 2023-10-16T09:45:16.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.835+0000 7fa0782ce700 1 Processor -- start 2023-10-16T09:45:16.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.835+0000 7fa0782ce700 1 -- start start 2023-10-16T09:45:16.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa0782ce700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa07006ade0 0x7fa0700c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa0782ce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa070072760 0x7fa070159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa0782ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07015da70 0x7fa070160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa0782ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701f65c0 con 0x7fa07015da70 2023-10-16T09:45:16.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa0782ce700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07015a000 con 0x7fa07006ade0 2023-10-16T09:45:16.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa0782ce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0701611c0 con 0x7fa070072760 2023-10-16T09:45:16.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07015da70 0x7fa070160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07015da70 0x7fa070160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38574/0 (socket says 172.21.15.181:38574) 2023-10-16T09:45:16.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 -- 172.21.15.181:0/3856918512 learned_addr learned my addr 172.21.15.181:0/3856918512 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:16.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa075869700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa070072760 0x7fa070159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07606a700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa07006ade0 0x7fa0700c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 -- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa070072760 msgr2=0x7fa070159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa070072760 0x7fa070159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 -- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa07006ade0 msgr2=0x7fa0700c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:16.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa07006ade0 0x7fa0700c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:16.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 -- 172.21.15.181:0/3856918512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068009a70 con 0x7fa07015da70 2023-10-16T09:45:16.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07606a700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa07006ade0 0x7fa0700c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:16.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa07686b700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07015da70 0x7fa070160900 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:16.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.836+0000 7fa066ffd700 1 -- 172.21.15.181:0/3856918512 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800cb40 con 0x7fa07015da70 2023-10-16T09:45:16.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.837+0000 7fa066ffd700 1 -- 172.21.15.181:0/3856918512 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06800f040 con 0x7fa07015da70 2023-10-16T09:45:16.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.837+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c0094a0 con 0x7fa07015da70 2023-10-16T09:45:16.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.837+0000 7fa066ffd700 1 -- 172.21.15.181:0/3856918512 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068003a20 con 0x7fa07015da70 2023-10-16T09:45:16.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.837+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0701f6af0 con 0x7fa07015da70 2023-10-16T09:45:16.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.838+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07006f6c0 con 0x7fa07015da70 2023-10-16T09:45:16.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.838+0000 7fa066ffd700 1 -- 172.21.15.181:0/3856918512 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fa068014480 con 0x7fa07015da70 2023-10-16T09:45:16.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.838+0000 7fa066ffd700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa05406ada0 0x7fa05406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:16.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.838+0000 7fa066ffd700 1 -- 172.21.15.181:0/3856918512 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa068089df0 con 0x7fa07015da70 2023-10-16T09:45:16.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.838+0000 7fa07606a700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa05406ada0 0x7fa05406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:16.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.839+0000 7fa07606a700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa05406ada0 0x7fa05406d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa060005910 tx=0x7fa060009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:16.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:16.843+0000 7fa066ffd700 1 -- 172.21.15.181:0/3856918512 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa068055060 con 0x7fa07015da70 2023-10-16T09:45:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.065+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa070072240 con 0x7fa05406ada0 2023-10-16T09:45:17.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.136+0000 7fa066ffd700 1 -- 172.21.15.181:0/3856918512 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa060001d10 con 0x7fa05406ada0 2023-10-16T09:45:17.139 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-16T09:45:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa05406ada0 msgr2=0x7fa05406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa05406ada0 0x7fa05406d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa060005910 tx=0x7fa060009040 comp rx=0 tx=0).stop 2023-10-16T09:45:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07015da70 msgr2=0x7fa070160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07015da70 0x7fa070160900 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa06800efc0 tx=0x7fa06800c3a0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 shutdown_connections 2023-10-16T09:45:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa05406ada0 0x7fa05406d260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07015da70 0x7fa070160900 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa07006ade0 0x7fa0700c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 --2- 172.21.15.181:0/3856918512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa070072760 0x7fa070159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 >> 172.21.15.181:0/3856918512 conn(0x7fa0700cd010 msgr2=0x7fa07015ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 shutdown_connections 2023-10-16T09:45:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.139+0000 7fa0782ce700 1 -- 172.21.15.181:0/3856918512 wait complete. 2023-10-16T09:45:17.153 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-16T09:45:17.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:17.153 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-16T09:45:17.171 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1m 2023-10-16T09:45:17.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.333+0000 7fb6e2280700 1 Processor -- start 2023-10-16T09:45:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.333+0000 7fb6e2280700 1 -- start start 2023-10-16T09:45:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6e2280700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6dc15da70 0x7fb6dc0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6e2280700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6e2280700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc0cef00 con 0x7fb6dc072760 2023-10-16T09:45:17.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6e2280700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6dc0cec20 con 0x7fb6dc15da70 2023-10-16T09:45:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6e2280700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc071c80 con 0x7fb6dc0ce6a0 2023-10-16T09:45:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6dc15da70 0x7fb6dc0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6dc15da70 0x7fb6dc0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38596/0 (socket says 172.21.15.181:38596) 2023-10-16T09:45:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2318625842 learned_addr learned my addr 172.21.15.181:0/2318625842 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:17.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6db7fe700 1 --2- 172.21.15.181:0/2318625842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2318625842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 msgr2=0x7fb6dc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:17.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 --2- 172.21.15.181:0/2318625842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2318625842 >> v1:172.21.15.190:6789/0 conn(0x7fb6dc072760 legacy=0x7fb6dc1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2318625842 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc071f60 con 0x7fb6dc15da70 2023-10-16T09:45:17.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6dbfff700 1 --2- 172.21.15.181:0/2318625842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc15da70 0x7fb6dc0ce160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0098f0 tx=0x7fb6cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6d0baf6e83f4e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:17.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6daffd700 1 -- 172.21.15.181:0/2318625842 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00b8a0 con 0x7fb6dc15da70 2023-10-16T09:45:17.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6daffd700 1 -- 172.21.15.181:0/2318625842 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc00ba00 con 0x7fb6dc15da70 2023-10-16T09:45:17.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.334+0000 7fb6daffd700 1 -- 172.21.15.181:0/2318625842 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00bd90 con 0x7fb6dc15da70 2023-10-16T09:45:17.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 -- 172.21.15.181:0/2318625842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc15da70 msgr2=0x7fb6dc0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2318625842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc15da70 0x7fb6dc0ce160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0098f0 tx=0x7fb6cc009c00 comp rx=0 tx=0).stop 2023-10-16T09:45:17.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 -- 172.21.15.181:0/2318625842 shutdown_connections 2023-10-16T09:45:17.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2318625842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc15da70 0x7fb6dc0ce160 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2318625842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 -- 172.21.15.181:0/2318625842 >> 172.21.15.181:0/2318625842 conn(0x7fb6dc0cd010 msgr2=0x7fb6dc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:17.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 -- 172.21.15.181:0/2318625842 shutdown_connections 2023-10-16T09:45:17.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.335+0000 7fb6e2280700 1 -- 172.21.15.181:0/2318625842 wait complete. 2023-10-16T09:45:17.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.336+0000 7fb6e2280700 1 Processor -- start 2023-10-16T09:45:17.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.336+0000 7fb6e2280700 1 -- start start 2023-10-16T09:45:17.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6e2280700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc072760 0x7fb6dc0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6e2280700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc1f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6e2280700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6dc15da70 0x7fb6dc0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6e2280700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0c5f40 con 0x7fb6dc072760 2023-10-16T09:45:17.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6e2280700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0c5330 con 0x7fb6dc15da70 2023-10-16T09:45:17.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6e2280700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0c5610 con 0x7fb6dc0ce6a0 2023-10-16T09:45:17.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6dbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc072760 0x7fb6dc0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6dbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc072760 0x7fb6dc0c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38602/0 (socket says 172.21.15.181:38602) 2023-10-16T09:45:17.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2276462319 learned_addr learned my addr 172.21.15.181:0/2276462319 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:17.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6db7fe700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc1f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.337+0000 7fb6e081d700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6dc15da70 0x7fb6dc0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 msgr2=0x7fb6dc1f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6dbfff700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc1f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6dc15da70 msgr2=0x7fb6dc0b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6dbfff700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6dc15da70 0x7fb6dc0b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6dbfff700 1 -- 172.21.15.181:0/2276462319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc009550 con 0x7fb6dc072760 2023-10-16T09:45:17.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6dbfff700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc072760 0x7fb6dc0c90c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc000c00 tx=0x7fb6cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:17.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6d8ff9700 1 -- 172.21.15.181:0/2276462319 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc004300 con 0x7fb6dc072760 2023-10-16T09:45:17.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6d8ff9700 1 -- 172.21.15.181:0/2276462319 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc01f070 con 0x7fb6dc072760 2023-10-16T09:45:17.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc0c58f0 con 0x7fb6dc072760 2023-10-16T09:45:17.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6d8ff9700 1 -- 172.21.15.181:0/2276462319 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0036a0 con 0x7fb6dc072760 2023-10-16T09:45:17.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.338+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc0c9600 con 0x7fb6dc072760 2023-10-16T09:45:17.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.339+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc06f6c0 con 0x7fb6dc072760 2023-10-16T09:45:17.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.339+0000 7fb6d8ff9700 1 -- 172.21.15.181:0/2276462319 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fb6cc021030 con 0x7fb6dc072760 2023-10-16T09:45:17.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.340+0000 7fb6d8ff9700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb6c806ada0 0x7fb6c806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.340+0000 7fb6d8ff9700 1 -- 172.21.15.181:0/2276462319 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb6cc08a670 con 0x7fb6dc072760 2023-10-16T09:45:17.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.340+0000 7fb6db7fe700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb6c806ada0 0x7fb6c806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.340+0000 7fb6db7fe700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb6c806ada0 0x7fb6c806d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc1f66b0 tx=0x7fb6d0008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:17.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.344+0000 7fb6d8ff9700 1 -- 172.21.15.181:0/2276462319 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb6cc055a00 con 0x7fb6dc072760 2023-10-16T09:45:17.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.561+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6dc15de50 con 0x7fb6c806ada0 2023-10-16T09:45:17.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.625+0000 7fb6d8ff9700 1 -- 172.21.15.181:0/2276462319 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb6d0009e10 con 0x7fb6c806ada0 2023-10-16T09:45:17.628 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-16T09:45:17.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.627+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb6c806ada0 msgr2=0x7fb6c806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.627+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb6c806ada0 0x7fb6c806d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc1f66b0 tx=0x7fb6d0008040 comp rx=0 tx=0).stop 2023-10-16T09:45:17.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.627+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc072760 msgr2=0x7fb6dc0c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.627+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc072760 0x7fb6dc0c90c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc000c00 tx=0x7fb6cc004750 comp rx=0 tx=0).stop 2023-10-16T09:45:17.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 shutdown_connections 2023-10-16T09:45:17.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb6c806ada0 0x7fb6c806d260 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6dc15da70 0x7fb6dc0b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6dc072760 0x7fb6dc0c90c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 --2- 172.21.15.181:0/2276462319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6dc0ce6a0 0x7fb6dc1f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 >> 172.21.15.181:0/2276462319 conn(0x7fb6dc0cd010 msgr2=0x7fb6dc0b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:17.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 shutdown_connections 2023-10-16T09:45:17.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.628+0000 7fb6e2280700 1 -- 172.21.15.181:0/2276462319 wait complete. 2023-10-16T09:45:17.641 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-16T09:45:17.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:17.642 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-16T09:45:17.661 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1m 2023-10-16T09:45:17.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.822+0000 7f6e3b6f6700 1 Processor -- start 2023-10-16T09:45:17.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.822+0000 7f6e3b6f6700 1 -- start start 2023-10-16T09:45:17.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e3b6f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e34159e50 0x7f6e34071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e3b6f6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415ea60 con 0x7f6e3415db30 2023-10-16T09:45:17.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e3b6f6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415e1c0 con 0x7f6e340cc5e0 2023-10-16T09:45:17.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e3b6f6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3415dfe0 con 0x7f6e34159e50 2023-10-16T09:45:17.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e39492700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e34159e50 0x7f6e34071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e39492700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e34159e50 0x7f6e34071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38610/0 (socket says 172.21.15.181:38610) 2023-10-16T09:45:17.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e39492700 1 -- 172.21.15.181:0/3236696618 learned_addr learned my addr 172.21.15.181:0/3236696618 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:17.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e39492700 1 -- 172.21.15.181:0/3236696618 >> v1:172.21.15.190:6789/0 conn(0x7f6e340cc5e0 legacy=0x7f6e340ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:17.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e39492700 1 -- 172.21.15.181:0/3236696618 >> v1:172.21.15.181:6789/0 conn(0x7f6e3415db30 legacy=0x7f6e340cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e39492700 1 -- 172.21.15.181:0/3236696618 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3415e4a0 con 0x7f6e34159e50 2023-10-16T09:45:17.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e39492700 1 --2- 172.21.15.181:0/3236696618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e34071460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000bb90 tx=0x7f6e3000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430f0d90164be702 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:17.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.823+0000 7f6e2bfff700 1 -- 172.21.15.181:0/3236696618 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3000dc00 con 0x7f6e34159e50 2023-10-16T09:45:17.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e2bfff700 1 -- 172.21.15.181:0/3236696618 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3000dd60 con 0x7f6e34159e50 2023-10-16T09:45:17.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e2bfff700 1 -- 172.21.15.181:0/3236696618 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30004270 con 0x7f6e34159e50 2023-10-16T09:45:17.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/3236696618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 msgr2=0x7f6e34071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/3236696618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e34071460 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000bb90 tx=0x7f6e3000bea0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/3236696618 shutdown_connections 2023-10-16T09:45:17.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/3236696618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e34071460 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/3236696618 >> 172.21.15.181:0/3236696618 conn(0x7f6e3406aed0 msgr2=0x7f6e3406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:17.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/3236696618 shutdown_connections 2023-10-16T09:45:17.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.824+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/3236696618 wait complete. 2023-10-16T09:45:17.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.825+0000 7f6e3b6f6700 1 Processor -- start 2023-10-16T09:45:17.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.825+0000 7f6e3b6f6700 1 -- start start 2023-10-16T09:45:17.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e3b6f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e340cc5e0 0x7f6e340b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e3b6f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e340aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e3b6f6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3415db30 0x7f6e34156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e3b6f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340c4ad0 con 0x7f6e34159e50 2023-10-16T09:45:17.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e3b6f6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34155660 con 0x7f6e340cc5e0 2023-10-16T09:45:17.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e3b6f6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340c41a0 con 0x7f6e3415db30 2023-10-16T09:45:17.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e340aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e340aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38618/0 (socket says 172.21.15.181:38618) 2023-10-16T09:45:17.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 -- 172.21.15.181:0/1196974207 learned_addr learned my addr 172.21.15.181:0/1196974207 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:17.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e39492700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e340cc5e0 0x7f6e340b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e39c93700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3415db30 0x7f6e34156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 -- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3415db30 msgr2=0x7f6e34156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3415db30 0x7f6e34156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 -- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e340cc5e0 msgr2=0x7f6e340b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:17.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e340cc5e0 0x7f6e340b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e38c91700 1 -- 172.21.15.181:0/1196974207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3000b7f0 con 0x7f6e34159e50 2023-10-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e39492700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e340cc5e0 0x7f6e340b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.826+0000 7f6e39c93700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3415db30 0x7f6e34156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.827+0000 7f6e38c91700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e340aee50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c430 tx=0x7f6e2400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.827+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240052d0 con 0x7f6e34159e50 2023-10-16T09:45:17.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.827+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24005450 con 0x7f6e34159e50 2023-10-16T09:45:17.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.827+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e340c4480 con 0x7f6e34159e50 2023-10-16T09:45:17.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.827+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24006aa0 con 0x7f6e34159e50 2023-10-16T09:45:17.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.827+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e341552f0 con 0x7f6e34159e50 2023-10-16T09:45:17.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.828+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340c4c80 con 0x7f6e34159e50 2023-10-16T09:45:17.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.828+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f6e24003680 con 0x7f6e34159e50 2023-10-16T09:45:17.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.828+0000 7f6e29ffb700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f6e1806ada0 0x7f6e1806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:17.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.828+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6e240885a0 con 0x7f6e34159e50 2023-10-16T09:45:17.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.829+0000 7f6e39492700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f6e1806ada0 0x7f6e1806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:17.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.829+0000 7f6e39492700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f6e1806ada0 0x7f6e1806d260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e30019040 tx=0x7f6e30008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:17.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:17.833+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6e340c4c80 con 0x7f6e34159e50 2023-10-16T09:45:18.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.031+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f6e24053c60 con 0x7f6e34159e50 2023-10-16T09:45:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.056+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e340cde70 con 0x7f6e1806ada0 2023-10-16T09:45:18.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.134+0000 7f6e29ffb700 1 -- 172.21.15.181:0/1196974207 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6e30004030 con 0x7f6e1806ada0 2023-10-16T09:45:18.137 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-16T09:45:18.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f6e1806ada0 msgr2=0x7f6e1806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f6e1806ada0 0x7f6e1806d260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e30019040 tx=0x7f6e30008040 comp rx=0 tx=0).stop 2023-10-16T09:45:18.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 msgr2=0x7f6e340aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e340aee50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c430 tx=0x7f6e2400c7f0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 shutdown_connections 2023-10-16T09:45:18.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3415db30 0x7f6e34156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f6e1806ada0 0x7f6e1806d260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e340cc5e0 0x7f6e340b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 --2- 172.21.15.181:0/1196974207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e34159e50 0x7f6e340aee50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 >> 172.21.15.181:0/1196974207 conn(0x7f6e3406aed0 msgr2=0x7f6e3415c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:18.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 shutdown_connections 2023-10-16T09:45:18.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.137+0000 7f6e3b6f6700 1 -- 172.21.15.181:0/1196974207 wait complete. 2023-10-16T09:45:18.151 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-16T09:45:18.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:18.152 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-16T09:45:18.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1m 2023-10-16T09:45:18.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.330+0000 7ff30769d700 1 Processor -- start 2023-10-16T09:45:18.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.331+0000 7ff30769d700 1 -- start start 2023-10-16T09:45:18.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.331+0000 7ff30769d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3000cc6e0 0x7ff300071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.331+0000 7ff30769d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3000ccb60 con 0x7ff3000ce680 2023-10-16T09:45:18.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.331+0000 7ff30769d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3000ce1f0 con 0x7ff3000b2010 2023-10-16T09:45:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.331+0000 7ff30769d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff300072120 con 0x7ff3000cc6e0 2023-10-16T09:45:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff305c3a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff3000ce680 0x7ff3000ceb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49706/0 (socket says 172.21.15.181:49706) 2023-10-16T09:45:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff305c3a700 1 -- 172.21.15.181:0/3009519815 learned_addr learned my addr 172.21.15.181:0/3009519815 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff304c38700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3000cc6e0 0x7ff300071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610409240 0 0) 0x7ff2f8001490 con 0x7ff3000ce680 2023-10-16T09:45:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ec0035b0 con 0x7ff3000ce680 2023-10-16T09:45:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034122829 0 0) 0x7ff3000ce1f0 con 0x7ff3000b2010 2023-10-16T09:45:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f8001490 con 0x7ff3000b2010 2023-10-16T09:45:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1874372599 0 0) 0x7ff2ec0035b0 con 0x7ff3000ce680 2023-10-16T09:45:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3000cc6e0 msgr2=0x7ff300071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3000cc6e0 0x7ff300071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 >> v1:172.21.15.190:6789/0 conn(0x7ff3000b2010 legacy=0x7ff3000b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000ccda0 con 0x7ff3000ce680 2023-10-16T09:45:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff2f8001b80 con 0x7ff3000ce680 2023-10-16T09:45:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 >> v1:172.21.15.181:6789/0 conn(0x7ff3000ce680 legacy=0x7ff3000ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff2ec003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2ec004850 0x7ff2ec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec005210 0x7ff2ec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff300072120 con 0x7ff2ec003bc0 2023-10-16T09:45:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec00c000 con 0x7ff2ec004850 2023-10-16T09:45:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec00c1c0 con 0x7ff2ec005210 2023-10-16T09:45:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff304c38700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3000cc6e0 0x7ff300071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff305439700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff2ec003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff305c3a700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec005210 0x7ff2ec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.332+0000 7ff304c38700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2ec004850 0x7ff2ec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff305439700 1 -- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec005210 msgr2=0x7ff2ec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff305439700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec005210 0x7ff2ec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff305439700 1 -- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2ec004850 msgr2=0x7ff2ec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff305439700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2ec004850 0x7ff2ec004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff305439700 1 -- 172.21.15.181:0/3009519815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec0035b0 con 0x7ff2ec003bc0 2023-10-16T09:45:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff305439700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff2ec003fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0001930 tx=0x7ff2ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3c84463f301aa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00052d0 con 0x7ff2ec003bc0 2023-10-16T09:45:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f0005430 con 0x7ff2ec003bc0 2023-10-16T09:45:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff2f7fff700 1 -- 172.21.15.181:0/3009519815 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00038c0 con 0x7ff2ec003bc0 2023-10-16T09:45:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff30769d700 1 -- 172.21.15.181:0/3009519815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 msgr2=0x7ff2ec003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff30769d700 1 --2- 172.21.15.181:0/3009519815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff2ec003fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0001930 tx=0x7ff2ec005880 comp rx=0 tx=0).stop 2023-10-16T09:45:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff305c3a700 1 -- 172.21.15.181:0/3009519815 reap_dead start 2023-10-16T09:45:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff30769d700 1 -- 172.21.15.181:0/3009519815 shutdown_connections 2023-10-16T09:45:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.333+0000 7ff30769d700 1 -- 172.21.15.181:0/3009519815 >> 172.21.15.181:0/3009519815 conn(0x7ff3000cd010 msgr2=0x7ff3000cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.334+0000 7ff30769d700 1 -- 172.21.15.181:0/3009519815 shutdown_connections 2023-10-16T09:45:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.334+0000 7ff30769d700 1 -- 172.21.15.181:0/3009519815 wait complete. 2023-10-16T09:45:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 Processor -- start 2023-10-16T09:45:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 -- start start 2023-10-16T09:45:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff300156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3000d0590 0x7ff3000d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3001f6550 0x7ff3001f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000d0f50 con 0x7ff2ec003bc0 2023-10-16T09:45:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3001f3850 con 0x7ff3000d0590 2023-10-16T09:45:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.335+0000 7ff30769d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3001f2f20 con 0x7ff3001f6550 2023-10-16T09:45:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff300156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff300156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38638/0 (socket says 172.21.15.181:38638) 2023-10-16T09:45:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 -- 172.21.15.181:0/282033244 learned_addr learned my addr 172.21.15.181:0/282033244 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff304c38700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3000d0590 0x7ff3000d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305c3a700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3001f6550 0x7ff3001f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 -- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3001f6550 msgr2=0x7ff3001f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3001f6550 0x7ff3001f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 -- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3000d0590 msgr2=0x7ff3000d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3000d0590 0x7ff3000d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 -- 172.21.15.181:0/282033244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3001f3200 con 0x7ff2ec003bc0 2023-10-16T09:45:18.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305c3a700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3001f6550 0x7ff3001f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:18.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff305439700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff300156d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c3c0 tx=0x7ff2f00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:18.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff2f5ffb700 1 -- 172.21.15.181:0/282033244 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0001d60 con 0x7ff2ec003bc0 2023-10-16T09:45:18.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff2f5ffb700 1 -- 172.21.15.181:0/282033244 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f00013e0 con 0x7ff2ec003bc0 2023-10-16T09:45:18.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3001f34e0 con 0x7ff2ec003bc0 2023-10-16T09:45:18.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff2f5ffb700 1 -- 172.21.15.181:0/282033244 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000c870 con 0x7ff2ec003bc0 2023-10-16T09:45:18.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.336+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3001f3ce0 con 0x7ff2ec003bc0 2023-10-16T09:45:18.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.337+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3001f3b30 con 0x7ff2ec003bc0 2023-10-16T09:45:18.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.338+0000 7ff2f5ffb700 1 -- 172.21.15.181:0/282033244 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7ff2f0019070 con 0x7ff2ec003bc0 2023-10-16T09:45:18.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.338+0000 7ff2f5ffb700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2e406b070 0x7ff2e406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.338+0000 7ff2f5ffb700 1 -- 172.21.15.181:0/282033244 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff2f008a7f0 con 0x7ff2ec003bc0 2023-10-16T09:45:18.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.338+0000 7ff304c38700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2e406b070 0x7ff2e406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.339+0000 7ff304c38700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2e406b070 0x7ff2e406d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff3000cdd70 tx=0x7ff2fc008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:18.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.343+0000 7ff2f5ffb700 1 -- 172.21.15.181:0/282033244 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff2f0055680 con 0x7ff2ec003bc0 2023-10-16T09:45:18.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.563+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3000727e0 con 0x7ff2e406b070 2023-10-16T09:45:18.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.636+0000 7ff2f5ffb700 1 -- 172.21.15.181:0/282033244 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff2fc009e80 con 0x7ff2e406b070 2023-10-16T09:45:18.638 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-16T09:45:18.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2e406b070 msgr2=0x7ff2e406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2e406b070 0x7ff2e406d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff3000cdd70 tx=0x7ff2fc008040 comp rx=0 tx=0).stop 2023-10-16T09:45:18.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 msgr2=0x7ff300156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff300156d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c3c0 tx=0x7ff2f00012d0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 shutdown_connections 2023-10-16T09:45:18.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3001f6550 0x7ff3001f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2e406b070 0x7ff2e406d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec003bc0 0x7ff300156d90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 --2- 172.21.15.181:0/282033244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3000d0590 0x7ff3000d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 >> 172.21.15.181:0/282033244 conn(0x7ff3000cd010 msgr2=0x7ff3000b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 shutdown_connections 2023-10-16T09:45:18.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.638+0000 7ff30769d700 1 -- 172.21.15.181:0/282033244 wait complete. 2023-10-16T09:45:18.652 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-16T09:45:18.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:18.654 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-16T09:45:18.672 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1m 2023-10-16T09:45:18.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.835+0000 7f653af41700 1 Processor -- start 2023-10-16T09:45:18.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.835+0000 7f653af41700 1 -- start start 2023-10-16T09:45:18.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f653af41700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f65340cb2b0 0x7f653406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f653af41700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65340cb690 0x7f65340b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f653af41700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f653af41700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6534070590 con 0x7f65340cb690 2023-10-16T09:45:18.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f653af41700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6534156330 con 0x7f65340cb2b0 2023-10-16T09:45:18.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f653af41700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653406fcf0 con 0x7f65340b2ba0 2023-10-16T09:45:18.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6533fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65340cb690 0x7f65340b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6533fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65340cb690 0x7f65340b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38646/0 (socket says 172.21.15.181:38646) 2023-10-16T09:45:18.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6533fff700 1 -- 172.21.15.181:0/2354727342 learned_addr learned my addr 172.21.15.181:0/2354727342 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:18.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6538cdd700 1 --2- 172.21.15.181:0/2354727342 >> v2:172.21.15.190:3300/0 conn(0x7f65340cb2b0 0x7f653406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f65394de700 1 --2- 172.21.15.181:0/2354727342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6533fff700 1 -- 172.21.15.181:0/2354727342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 msgr2=0x7f653415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:18.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6533fff700 1 --2- 172.21.15.181:0/2354727342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6533fff700 1 -- 172.21.15.181:0/2354727342 >> v2:172.21.15.190:3300/0 conn(0x7f65340cb2b0 msgr2=0x7f653406f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:18.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.836+0000 7f6533fff700 1 --2- 172.21.15.181:0/2354727342 >> v2:172.21.15.190:3300/0 conn(0x7f65340cb2b0 0x7f653406f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f6533fff700 1 -- 172.21.15.181:0/2354727342 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653406ffd0 con 0x7f65340cb690 2023-10-16T09:45:18.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f6533fff700 1 --2- 172.21.15.181:0/2354727342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb690 0x7f65340b2660 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65340d1600 tx=0x7f651c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4258554a641576 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:18.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f65337fe700 1 -- 172.21.15.181:0/2354727342 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00b870 con 0x7f65340cb690 2023-10-16T09:45:18.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f65337fe700 1 -- 172.21.15.181:0/2354727342 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c00b9d0 con 0x7f65340cb690 2023-10-16T09:45:18.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f65337fe700 1 -- 172.21.15.181:0/2354727342 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00bd60 con 0x7f65340cb690 2023-10-16T09:45:18.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 -- 172.21.15.181:0/2354727342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb690 msgr2=0x7f65340b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 --2- 172.21.15.181:0/2354727342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb690 0x7f65340b2660 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65340d1600 tx=0x7f651c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 -- 172.21.15.181:0/2354727342 shutdown_connections 2023-10-16T09:45:18.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 --2- 172.21.15.181:0/2354727342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 --2- 172.21.15.181:0/2354727342 >> v2:172.21.15.190:3300/0 conn(0x7f65340cb2b0 0x7f653406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 --2- 172.21.15.181:0/2354727342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb690 0x7f65340b2660 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 -- 172.21.15.181:0/2354727342 >> 172.21.15.181:0/2354727342 conn(0x7f65340cd010 msgr2=0x7f65340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:18.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 -- 172.21.15.181:0/2354727342 shutdown_connections 2023-10-16T09:45:18.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.837+0000 7f653af41700 1 -- 172.21.15.181:0/2354727342 wait complete. 2023-10-16T09:45:18.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.839+0000 7f653af41700 1 Processor -- start 2023-10-16T09:45:18.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.839+0000 7f653af41700 1 -- start start 2023-10-16T09:45:18.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.839+0000 7f653af41700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.839+0000 7f653af41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb2b0 0x7f65340c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f653af41700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65340cb690 0x7f6534154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f653af41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65341550b0 con 0x7f65340cb2b0 2023-10-16T09:45:18.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f653af41700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653415c890 con 0x7f65340cb690 2023-10-16T09:45:18.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f653af41700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653415aa20 con 0x7f65340b2ba0 2023-10-16T09:45:18.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb2b0 0x7f65340c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb2b0 0x7f65340c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38652/0 (socket says 172.21.15.181:38652) 2023-10-16T09:45:18.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 -- 172.21.15.181:0/2957752802 learned_addr learned my addr 172.21.15.181:0/2957752802 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:18.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6538cdd700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f65394de700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65340cb690 0x7f6534154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 -- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 msgr2=0x7f653415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 -- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65340cb690 msgr2=0x7f6534154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:18.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65340cb690 0x7f6534154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:18.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 -- 172.21.15.181:0/2957752802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c009550 con 0x7f65340cb2b0 2023-10-16T09:45:18.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f6533fff700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb2b0 0x7f65340c5e20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f651c000c00 tx=0x7f651c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:18.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f65317fa700 1 -- 172.21.15.181:0/2957752802 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c004520 con 0x7f65340cb2b0 2023-10-16T09:45:18.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f65317fa700 1 -- 172.21.15.181:0/2957752802 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c01e070 con 0x7f65340cb2b0 2023-10-16T09:45:18.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653415ad00 con 0x7f65340cb2b0 2023-10-16T09:45:18.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f65317fa700 1 -- 172.21.15.181:0/2957752802 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c0038c0 con 0x7f65340cb2b0 2023-10-16T09:45:18.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.840+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653415b2c0 con 0x7f65340cb2b0 2023-10-16T09:45:18.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.841+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653406f4f0 con 0x7f65340cb2b0 2023-10-16T09:45:18.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.842+0000 7f65317fa700 1 -- 172.21.15.181:0/2957752802 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f651c003a20 con 0x7f65340cb2b0 2023-10-16T09:45:18.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.842+0000 7f65317fa700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f651406b070 0x7f651406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:18.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.842+0000 7f65317fa700 1 -- 172.21.15.181:0/2957752802 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f651c020070 con 0x7f65340cb2b0 2023-10-16T09:45:18.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.842+0000 7f6538cdd700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f651406b070 0x7f651406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:18.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.843+0000 7f6538cdd700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f651406b070 0x7f651406d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6524009930 tx=0x7f6524008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:18.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:18.847+0000 7f65317fa700 1 -- 172.21.15.181:0/2957752802 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f651c090020 con 0x7f65340cb2b0 2023-10-16T09:45:19.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.066+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f65340702b0 con 0x7f651406b070 2023-10-16T09:45:19.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.139+0000 7f65317fa700 1 -- 172.21.15.181:0/2957752802 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f65240095c0 con 0x7f651406b070 2023-10-16T09:45:19.141 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-16T09:45:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f651406b070 msgr2=0x7f651406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f651406b070 0x7f651406d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6524009930 tx=0x7f6524008040 comp rx=0 tx=0).stop 2023-10-16T09:45:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb2b0 msgr2=0x7f65340c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb2b0 0x7f65340c5e20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f651c000c00 tx=0x7f651c004750 comp rx=0 tx=0).stop 2023-10-16T09:45:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 shutdown_connections 2023-10-16T09:45:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65340b2ba0 0x7f653415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f651406b070 0x7f651406d530 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65340cb690 0x7f6534154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 --2- 172.21.15.181:0/2957752802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65340cb2b0 0x7f65340c5e20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 >> 172.21.15.181:0/2957752802 conn(0x7f65340cd010 msgr2=0x7f6534155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 shutdown_connections 2023-10-16T09:45:19.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.141+0000 7f653af41700 1 -- 172.21.15.181:0/2957752802 wait complete. 2023-10-16T09:45:19.155 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-16T09:45:19.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:19.156 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-16T09:45:19.175 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1m 2023-10-16T09:45:19.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.336+0000 7f1ab2566700 1 Processor -- start 2023-10-16T09:45:19.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.336+0000 7f1ab2566700 1 -- start start 2023-10-16T09:45:19.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab2566700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1aac15da70 0x7f1aac0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab2566700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab2566700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1aac072760 0x7f1aac06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab2566700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac0cef00 con 0x7f1aac072760 2023-10-16T09:45:19.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab2566700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac0cec20 con 0x7f1aac15da70 2023-10-16T09:45:19.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab2566700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac071c80 con 0x7f1aac0ce6a0 2023-10-16T09:45:19.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1aac072760 0x7f1aac06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1aac072760 0x7f1aac06ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38662/0 (socket says 172.21.15.181:38662) 2023-10-16T09:45:19.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 -- 172.21.15.181:0/522184419 learned_addr learned my addr 172.21.15.181:0/522184419 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:19.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 -- 172.21.15.181:0/522184419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 msgr2=0x7f1aac06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:19.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1aab7fe700 1 --2- 172.21.15.181:0/522184419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 --2- 172.21.15.181:0/522184419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 -- 172.21.15.181:0/522184419 >> v2:172.21.15.190:3300/0 conn(0x7f1aac15da70 msgr2=0x7f1aac0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:19.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1aabfff700 1 --2- 172.21.15.181:0/522184419 >> v2:172.21.15.190:3300/0 conn(0x7f1aac15da70 0x7f1aac0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 --2- 172.21.15.181:0/522184419 >> v2:172.21.15.190:3300/0 conn(0x7f1aac15da70 0x7f1aac0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 -- 172.21.15.181:0/522184419 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac071f60 con 0x7f1aac072760 2023-10-16T09:45:19.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1ab0b03700 1 --2- 172.21.15.181:0/522184419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac06ade0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0675e0 tx=0x7f1aa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0338c4eefb60fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:19.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1aaaffd700 1 -- 172.21.15.181:0/522184419 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400e070 con 0x7f1aac072760 2023-10-16T09:45:19.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1aaaffd700 1 -- 172.21.15.181:0/522184419 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa400b9c0 con 0x7f1aac072760 2023-10-16T09:45:19.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.337+0000 7f1aaaffd700 1 -- 172.21.15.181:0/522184419 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400bd70 con 0x7f1aac072760 2023-10-16T09:45:19.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 -- 172.21.15.181:0/522184419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 msgr2=0x7f1aac06ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 --2- 172.21.15.181:0/522184419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac06ade0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0675e0 tx=0x7f1aa400b330 comp rx=0 tx=0).stop 2023-10-16T09:45:19.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 -- 172.21.15.181:0/522184419 shutdown_connections 2023-10-16T09:45:19.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 --2- 172.21.15.181:0/522184419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac06ade0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 --2- 172.21.15.181:0/522184419 >> v2:172.21.15.190:3300/0 conn(0x7f1aac15da70 0x7f1aac0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 --2- 172.21.15.181:0/522184419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 -- 172.21.15.181:0/522184419 >> 172.21.15.181:0/522184419 conn(0x7f1aac0cd010 msgr2=0x7f1aac0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:19.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 -- 172.21.15.181:0/522184419 shutdown_connections 2023-10-16T09:45:19.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.338+0000 7f1ab2566700 1 -- 172.21.15.181:0/522184419 wait complete. 2023-10-16T09:45:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.340+0000 7f1ab2566700 1 Processor -- start 2023-10-16T09:45:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.340+0000 7f1ab2566700 1 -- start start 2023-10-16T09:45:19.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1aac15da70 0x7f1aac1f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0acdb0 con 0x7f1aac072760 2023-10-16T09:45:19.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0ad090 con 0x7f1aac15da70 2023-10-16T09:45:19.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0ad370 con 0x7f1aac0ce6a0 2023-10-16T09:45:19.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38664/0 (socket says 172.21.15.181:38664) 2023-10-16T09:45:19.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 -- 172.21.15.181:0/1353106465 learned_addr learned my addr 172.21.15.181:0/1353106465 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:19.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab0b03700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1aac15da70 0x7f1aac1f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aab7fe700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 -- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 msgr2=0x7f1aac0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac0ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 -- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1aac15da70 msgr2=0x7f1aac1f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1aac15da70 0x7f1aac1f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 -- 172.21.15.181:0/1353106465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4009b30 con 0x7f1aac072760 2023-10-16T09:45:19.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aabfff700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac15ff20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c7f0 tx=0x7f1a9c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:19.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aa8ff9700 1 -- 172.21.15.181:0/1353106465 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c011800 con 0x7f1aac072760 2023-10-16T09:45:19.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aa8ff9700 1 -- 172.21.15.181:0/1353106465 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c011960 con 0x7f1aac072760 2023-10-16T09:45:19.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac1f30c0 con 0x7f1aac072760 2023-10-16T09:45:19.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1aa8ff9700 1 -- 172.21.15.181:0/1353106465 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c011cf0 con 0x7f1aac072760 2023-10-16T09:45:19.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.341+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac0aedc0 con 0x7f1aac072760 2023-10-16T09:45:19.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.342+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac158230 con 0x7f1aac072760 2023-10-16T09:45:19.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.343+0000 7f1aa8ff9700 1 -- 172.21.15.181:0/1353106465 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 10) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f1a9c018360 con 0x7f1aac072760 2023-10-16T09:45:19.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.343+0000 7f1aa8ff9700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1a9806b070 0x7f1a9806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.343+0000 7f1aa8ff9700 1 -- 172.21.15.181:0/1353106465 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f1a9c0899a0 con 0x7f1aac072760 2023-10-16T09:45:19.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.343+0000 7f1aab7fe700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1a9806b070 0x7f1a9806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.343+0000 7f1aab7fe700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1a9806b070 0x7f1a9806d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0b19b0 tx=0x7f1aa0008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:19.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.347+0000 7f1aa8ff9700 1 -- 172.21.15.181:0/1353106465 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a9c054740 con 0x7f1aac072760 2023-10-16T09:45:19.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.569+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1aac072240 con 0x7f1a9806b070 2023-10-16T09:45:19.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.642+0000 7f1aa8ff9700 1 -- 172.21.15.181:0/1353106465 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1aa000c050 con 0x7f1a9806b070 2023-10-16T09:45:19.644 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-16T09:45:19.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1a9806b070 msgr2=0x7f1a9806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1a9806b070 0x7f1a9806d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0b19b0 tx=0x7f1aa0008040 comp rx=0 tx=0).stop 2023-10-16T09:45:19.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 msgr2=0x7f1aac15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac15ff20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c7f0 tx=0x7f1a9c00cbb0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 shutdown_connections 2023-10-16T09:45:19.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1a9806b070 0x7f1a9806d530 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1aac15da70 0x7f1aac1f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aac072760 0x7f1aac15ff20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 --2- 172.21.15.181:0/1353106465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1aac0ce6a0 0x7f1aac0ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 >> 172.21.15.181:0/1353106465 conn(0x7f1aac0cd010 msgr2=0x7f1aac161cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:19.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 shutdown_connections 2023-10-16T09:45:19.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.644+0000 7f1ab2566700 1 -- 172.21.15.181:0/1353106465 wait complete. 2023-10-16T09:45:19.658 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-16T09:45:19.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:19.659 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-16T09:45:19.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1m 2023-10-16T09:45:19.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.843+0000 7fe08c040700 1 Processor -- start 2023-10-16T09:45:19.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.844+0000 7fe08c040700 1 -- start start 2023-10-16T09:45:19.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.844+0000 7fe08c040700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08415da70 0x7fe0840ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.844+0000 7fe08c040700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe0840ce6a0 0x7fe08406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.844+0000 7fe08c040700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840cef00 con 0x7fe084072760 2023-10-16T09:45:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.844+0000 7fe08c040700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840cec20 con 0x7fe0840ce6a0 2023-10-16T09:45:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.844+0000 7fe08c040700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe084071c80 con 0x7fe08415da70 2023-10-16T09:45:19.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe08a5dd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe084072760 0x7fe0841615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49754/0 (socket says 172.21.15.181:49754) 2023-10-16T09:45:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe08a5dd700 1 -- 172.21.15.181:0/1332018945 learned_addr learned my addr 172.21.15.181:0/1332018945 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe089ddc700 1 --2- 172.21.15.181:0/1332018945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08415da70 0x7fe0840ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe0895db700 1 --2- 172.21.15.181:0/1332018945 >> v2:172.21.15.190:3300/0 conn(0x7fe0840ce6a0 0x7fe08406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe088dda700 1 -- 172.21.15.181:0/1332018945 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271548032 0 0) 0x7fe0840cef00 con 0x7fe084072760 2023-10-16T09:45:19.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe088dda700 1 -- 172.21.15.181:0/1332018945 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0700035b0 con 0x7fe084072760 2023-10-16T09:45:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe089ddc700 1 -- 172.21.15.181:0/1332018945 >> v2:172.21.15.190:3300/0 conn(0x7fe0840ce6a0 msgr2=0x7fe08406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe089ddc700 1 --2- 172.21.15.181:0/1332018945 >> v2:172.21.15.190:3300/0 conn(0x7fe0840ce6a0 0x7fe08406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe089ddc700 1 -- 172.21.15.181:0/1332018945 >> v1:172.21.15.181:6789/0 conn(0x7fe084072760 legacy=0x7fe0841615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe089ddc700 1 -- 172.21.15.181:0/1332018945 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084071f60 con 0x7fe08415da70 2023-10-16T09:45:19.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe0895db700 1 --2- 172.21.15.181:0/1332018945 >> v2:172.21.15.190:3300/0 conn(0x7fe0840ce6a0 0x7fe08406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe089ddc700 1 --2- 172.21.15.181:0/1332018945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08415da70 0x7fe0840ce160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009d90 tx=0x7fe07c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8eeb8e61ddca8e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe088dda700 1 -- 172.21.15.181:0/1332018945 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00e070 con 0x7fe08415da70 2023-10-16T09:45:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe088dda700 1 -- 172.21.15.181:0/1332018945 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00bb20 con 0x7fe08415da70 2023-10-16T09:45:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.845+0000 7fe088dda700 1 -- 172.21.15.181:0/1332018945 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00be90 con 0x7fe08415da70 2023-10-16T09:45:19.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 -- 172.21.15.181:0/1332018945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08415da70 msgr2=0x7fe0840ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 --2- 172.21.15.181:0/1332018945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08415da70 0x7fe0840ce160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009d90 tx=0x7fe07c00b330 comp rx=0 tx=0).stop 2023-10-16T09:45:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 -- 172.21.15.181:0/1332018945 shutdown_connections 2023-10-16T09:45:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 --2- 172.21.15.181:0/1332018945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe08415da70 0x7fe0840ce160 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 --2- 172.21.15.181:0/1332018945 >> v2:172.21.15.190:3300/0 conn(0x7fe0840ce6a0 0x7fe08406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 -- 172.21.15.181:0/1332018945 >> 172.21.15.181:0/1332018945 conn(0x7fe0840cd010 msgr2=0x7fe0840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 -- 172.21.15.181:0/1332018945 shutdown_connections 2023-10-16T09:45:19.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.846+0000 7fe08c040700 1 -- 172.21.15.181:0/1332018945 wait complete. 2023-10-16T09:45:19.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.847+0000 7fe08c040700 1 Processor -- start 2023-10-16T09:45:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.847+0000 7fe08c040700 1 -- start start 2023-10-16T09:45:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08c040700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe084072760 0x7fe0840ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08c040700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0840ce6a0 0x7fe0841519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08c040700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08415da70 0x7fe0840aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08c040700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840ae0c0 con 0x7fe084072760 2023-10-16T09:45:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08c040700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840ae360 con 0x7fe08415da70 2023-10-16T09:45:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08c040700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840ae640 con 0x7fe0840ce6a0 2023-10-16T09:45:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe084072760 0x7fe0840ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe084072760 0x7fe0840ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38678/0 (socket says 172.21.15.181:38678) 2023-10-16T09:45:19.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 -- 172.21.15.181:0/1230467996 learned_addr learned my addr 172.21.15.181:0/1230467996 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe0895db700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0840ce6a0 0x7fe0841519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08a5dd700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08415da70 0x7fe0840aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 -- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0840ce6a0 msgr2=0x7fe0841519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0840ce6a0 0x7fe0841519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 -- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08415da70 msgr2=0x7fe0840aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08415da70 0x7fe0840aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe089ddc700 1 -- 172.21.15.181:0/1230467996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0740094a0 con 0x7fe084072760 2023-10-16T09:45:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.848+0000 7fe08a5dd700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08415da70 0x7fe0840aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.849+0000 7fe089ddc700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe084072760 0x7fe0840ad290 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe07c014040 tx=0x7fe07c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.849+0000 7fe07a7fc700 1 -- 172.21.15.181:0/1230467996 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c004930 con 0x7fe084072760 2023-10-16T09:45:19.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.849+0000 7fe07a7fc700 1 -- 172.21.15.181:0/1230467996 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c020070 con 0x7fe084072760 2023-10-16T09:45:19.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.849+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c009a70 con 0x7fe084072760 2023-10-16T09:45:19.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.849+0000 7fe07a7fc700 1 -- 172.21.15.181:0/1230467996 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0276c0 con 0x7fe084072760 2023-10-16T09:45:19.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.849+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0840adb90 con 0x7fe084072760 2023-10-16T09:45:19.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.850+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0840aec90 con 0x7fe084072760 2023-10-16T09:45:19.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.850+0000 7fe07a7fc700 1 -- 172.21.15.181:0/1230467996 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fe07c012430 con 0x7fe084072760 2023-10-16T09:45:19.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.851+0000 7fe07a7fc700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fe06806b130 0x7fe06806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:19.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.851+0000 7fe0895db700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fe06806b130 0x7fe06806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:19.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.851+0000 7fe07a7fc700 1 -- 172.21.15.181:0/1230467996 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe07c022070 con 0x7fe084072760 2023-10-16T09:45:19.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.851+0000 7fe0895db700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fe06806b130 0x7fe06806d5f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe0840cc750 tx=0x7fe074009210 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:19.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:19.855+0000 7fe07a7fc700 1 -- 172.21.15.181:0/1230467996 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe07c056660 con 0x7fe084072760 2023-10-16T09:45:20.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.075+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe08415de50 con 0x7fe06806b130 2023-10-16T09:45:20.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.143+0000 7fe07a7fc700 1 -- 172.21.15.181:0/1230467996 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe074009e80 con 0x7fe06806b130 2023-10-16T09:45:20.145 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-16T09:45:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fe06806b130 msgr2=0x7fe06806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fe06806b130 0x7fe06806d5f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe0840cc750 tx=0x7fe074009210 comp rx=0 tx=0).stop 2023-10-16T09:45:20.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe084072760 msgr2=0x7fe0840ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe084072760 0x7fe0840ad290 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe07c014040 tx=0x7fe07c004ad0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 shutdown_connections 2023-10-16T09:45:20.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fe06806b130 0x7fe06806d5f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08415da70 0x7fe0840aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe084072760 0x7fe0840ad290 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 --2- 172.21.15.181:0/1230467996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0840ce6a0 0x7fe0841519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 >> 172.21.15.181:0/1230467996 conn(0x7fe0840cd010 msgr2=0x7fe08415c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:20.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 shutdown_connections 2023-10-16T09:45:20.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.145+0000 7fe08c040700 1 -- 172.21.15.181:0/1230467996 wait complete. 2023-10-16T09:45:20.159 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-16T09:45:20.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:45:20.160 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-16T09:45:20.178 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1m 2023-10-16T09:45:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.339+0000 7ff2f1819700 1 Processor -- start 2023-10-16T09:45:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.339+0000 7ff2f1819700 1 -- start start 2023-10-16T09:45:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2f1819700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2ec15ea40 0x7ff2ec0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2f1819700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ec0cf580 con 0x7ff2ec0ce2d0 2023-10-16T09:45:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2f1819700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ec0ce970 con 0x7ff2ec0d01d0 2023-10-16T09:45:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2f1819700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ec0d4290 con 0x7ff2ec15ea40 2023-10-16T09:45:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2ea7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff2ec0ce2d0 0x7ff2ec0d1500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49766/0 (socket says 172.21.15.181:49766) 2023-10-16T09:45:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2ea7fc700 1 -- 172.21.15.181:0/3565734114 learned_addr learned my addr 172.21.15.181:0/3565734114 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2eb7fe700 1 --2- 172.21.15.181:0/3565734114 >> v2:172.21.15.190:3300/0 conn(0x7ff2ec15ea40 0x7ff2ec0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275135428 0 0) 0x7ff2ec0cf580 con 0x7ff2ec0ce2d0 2023-10-16T09:45:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2d80035b0 con 0x7ff2ec0ce2d0 2023-10-16T09:45:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525092105 0 0) 0x7ff2d80035b0 con 0x7ff2ec0ce2d0 2023-10-16T09:45:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 >> v2:172.21.15.190:3300/0 conn(0x7ff2ec15ea40 msgr2=0x7ff2ec0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 --2- 172.21.15.181:0/3565734114 >> v2:172.21.15.190:3300/0 conn(0x7ff2ec15ea40 0x7ff2ec0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 >> v1:172.21.15.190:6789/0 conn(0x7ff2ec0d01d0 legacy=0x7ff2ec0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec0cec50 con 0x7ff2ec0ce2d0 2023-10-16T09:45:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff2d4001670 con 0x7ff2ec0ce2d0 2023-10-16T09:45:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 >> v1:172.21.15.181:6789/0 conn(0x7ff2ec0ce2d0 legacy=0x7ff2ec0d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.340+0000 7ff2e9ffb700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d8003a00 0x7ff2d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d8004850 0x7ff2d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8005210 0x7ff2d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d800c080 con 0x7ff2d8003a00 2023-10-16T09:45:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d800c1f0 con 0x7ff2d8005210 2023-10-16T09:45:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d800c3b0 con 0x7ff2d8004850 2023-10-16T09:45:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eaffd700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d8003a00 0x7ff2d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eb7fe700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8005210 0x7ff2d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2ea7fc700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d8004850 0x7ff2d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eaffd700 1 -- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d8004850 msgr2=0x7ff2d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eaffd700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2d8004850 0x7ff2d8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eaffd700 1 -- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8005210 msgr2=0x7ff2d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eaffd700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8005210 0x7ff2d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eb7fe700 1 -- 172.21.15.181:0/3565734114 reap_dead start 2023-10-16T09:45:20.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eaffd700 1 -- 172.21.15.181:0/3565734114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80035b0 con 0x7ff2d8003a00 2023-10-16T09:45:20.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2eaffd700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d8003a00 0x7ff2d8003e20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0017c0 tx=0x7ff2d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320bb74d76305b89 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:20.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0052d0 con 0x7ff2d8003a00 2023-10-16T09:45:20.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2dc005450 con 0x7ff2d8003a00 2023-10-16T09:45:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.341+0000 7ff2e9ffb700 1 -- 172.21.15.181:0/3565734114 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0038c0 con 0x7ff2d8003a00 2023-10-16T09:45:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.342+0000 7ff2f1819700 1 -- 172.21.15.181:0/3565734114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d8003a00 msgr2=0x7ff2d8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.342+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d8003a00 0x7ff2d8003e20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0017c0 tx=0x7ff2d8005880 comp rx=0 tx=0).stop 2023-10-16T09:45:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.342+0000 7ff2f1819700 1 -- 172.21.15.181:0/3565734114 shutdown_connections 2023-10-16T09:45:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.342+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3565734114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d8003a00 0x7ff2d8003e20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.342+0000 7ff2f1819700 1 -- 172.21.15.181:0/3565734114 >> 172.21.15.181:0/3565734114 conn(0x7ff2ec09cb60 msgr2=0x7ff2ec0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.342+0000 7ff2f1819700 1 -- 172.21.15.181:0/3565734114 shutdown_connections 2023-10-16T09:45:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.342+0000 7ff2f1819700 1 -- 172.21.15.181:0/3565734114 wait complete. 2023-10-16T09:45:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.343+0000 7ff2f1819700 1 Processor -- start 2023-10-16T09:45:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.343+0000 7ff2f1819700 1 -- start start 2023-10-16T09:45:20.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2f1819700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8003a00 0x7ff2ec0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2f1819700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec0b8220 0x7ff2ec1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2f1819700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec15df70 0x7ff2ec1f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2f1819700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec0b9490 con 0x7ff2ec0b8220 2023-10-16T09:45:20.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2f1819700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec0b6380 con 0x7ff2d8003a00 2023-10-16T09:45:20.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2f1819700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec1f76e0 con 0x7ff2ec15df70 2023-10-16T09:45:20.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec0b8220 0x7ff2ec1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec0b8220 0x7ff2ec1f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38700/0 (socket says 172.21.15.181:38700) 2023-10-16T09:45:20.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 -- 172.21.15.181:0/3874007623 learned_addr learned my addr 172.21.15.181:0/3874007623 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:20.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2eb7fe700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec15df70 0x7ff2ec1f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2eaffd700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8003a00 0x7ff2ec0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 -- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec15df70 msgr2=0x7ff2ec1f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec15df70 0x7ff2ec1f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 -- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8003a00 msgr2=0x7ff2ec0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8003a00 0x7ff2ec0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 -- 172.21.15.181:0/3874007623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec0b86a0 con 0x7ff2ec0b8220 2023-10-16T09:45:20.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2ea7fc700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec0b8220 0x7ff2ec1f71a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400eaf0 tx=0x7ff2d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:20.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.344+0000 7ff2e3fff700 1 -- 172.21.15.181:0/3874007623 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4005e00 con 0x7ff2ec0b8220 2023-10-16T09:45:20.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.345+0000 7ff2e3fff700 1 -- 172.21.15.181:0/3874007623 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d400c7b0 con 0x7ff2ec0b8220 2023-10-16T09:45:20.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.345+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ec0b8970 con 0x7ff2ec0b8220 2023-10-16T09:45:20.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.345+0000 7ff2e3fff700 1 -- 172.21.15.181:0/3874007623 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400cb40 con 0x7ff2ec0b8220 2023-10-16T09:45:20.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.345+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ec0b91b0 con 0x7ff2ec0b8220 2023-10-16T09:45:20.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.346+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ec0b64f0 con 0x7ff2ec0b8220 2023-10-16T09:45:20.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.346+0000 7ff2e3fff700 1 -- 172.21.15.181:0/3874007623 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff2d400cca0 con 0x7ff2ec0b8220 2023-10-16T09:45:20.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.346+0000 7ff2e3fff700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2cc06b130 0x7ff2cc06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.347+0000 7ff2e3fff700 1 -- 172.21.15.181:0/3874007623 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff2d4089890 con 0x7ff2ec0b8220 2023-10-16T09:45:20.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.347+0000 7ff2eaffd700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2cc06b130 0x7ff2cc06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.347+0000 7ff2eaffd700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2cc06b130 0x7ff2cc06d5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005040 tx=0x7ff2dc009030 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:20.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.351+0000 7ff2e3fff700 1 -- 172.21.15.181:0/3874007623 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff2d4054520 con 0x7ff2ec0b8220 2023-10-16T09:45:20.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.570+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2ec0ce750 con 0x7ff2cc06b130 2023-10-16T09:45:20.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.641+0000 7ff2e3fff700 1 -- 172.21.15.181:0/3874007623 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff2dc0051d0 con 0x7ff2cc06b130 2023-10-16T09:45:20.644 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-16T09:45:20.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2cc06b130 msgr2=0x7ff2cc06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2cc06b130 0x7ff2cc06d5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005040 tx=0x7ff2dc009030 comp rx=0 tx=0).stop 2023-10-16T09:45:20.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec0b8220 msgr2=0x7ff2ec1f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec0b8220 0x7ff2ec1f71a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400eaf0 tx=0x7ff2d400eeb0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 shutdown_connections 2023-10-16T09:45:20.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2ec15df70 0x7ff2ec1f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2cc06b130 0x7ff2cc06d5f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d8003a00 0x7ff2ec0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 --2- 172.21.15.181:0/3874007623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec0b8220 0x7ff2ec1f71a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 >> 172.21.15.181:0/3874007623 conn(0x7ff2ec09cb60 msgr2=0x7ff2ec15cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:20.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 shutdown_connections 2023-10-16T09:45:20.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.643+0000 7ff2f1819700 1 -- 172.21.15.181:0/3874007623 wait complete. 2023-10-16T09:45:20.657 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-16T09:45:20.657 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1m 2023-10-16T09:45:20.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.778+0000 7f612793e700 1 Processor -- start 2023-10-16T09:45:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.778+0000 7f612793e700 1 -- start start 2023-10-16T09:45:20.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f612793e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f612015dbd0 0x7f61200d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f612793e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61200d0610 0x7f61200cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f612793e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612015dfb0 con 0x7f61200ce2c0 2023-10-16T09:45:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f612793e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61200ce740 con 0x7f61200d0610 2023-10-16T09:45:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f612793e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61200cd3a0 con 0x7f612015dbd0 2023-10-16T09:45:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f6124ed9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61200d0610 0x7f61200cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f6124ed9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61200d0610 0x7f61200cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38708/0 (socket says 172.21.15.181:38708) 2023-10-16T09:45:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f6124ed9700 1 -- 172.21.15.181:0/3266260505 learned_addr learned my addr 172.21.15.181:0/3266260505 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.779+0000 7f6125edb700 1 --1- 172.21.15.181:0/3266260505 >> v1:172.21.15.181:6789/0 conn(0x7f61200ce2c0 0x7f6120161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49788/0 (socket says 172.21.15.181:49788) 2023-10-16T09:45:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f61256da700 1 --2- 172.21.15.181:0/3266260505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f612015dbd0 0x7f61200d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6124ed9700 1 -- 172.21.15.181:0/3266260505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f612015dbd0 msgr2=0x7f61200d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6124ed9700 1 --2- 172.21.15.181:0/3266260505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f612015dbd0 0x7f61200d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6124ed9700 1 -- 172.21.15.181:0/3266260505 >> v1:172.21.15.181:6789/0 conn(0x7f61200ce2c0 legacy=0x7f6120161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6124ed9700 1 -- 172.21.15.181:0/3266260505 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61200cd830 con 0x7f61200d0610 2023-10-16T09:45:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6117fff700 1 -- 172.21.15.181:0/3266260505 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317110446 0 0) 0x7f6118001490 con 0x7f61200ce2c0 2023-10-16T09:45:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6124ed9700 1 --2- 172.21.15.181:0/3266260505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61200d0610 0x7f61200cdd80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f611c0098f0 tx=0x7f611c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aad19d38a6a374 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6117fff700 1 -- 172.21.15.181:0/3266260505 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c00b8a0 con 0x7f61200d0610 2023-10-16T09:45:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6117fff700 1 -- 172.21.15.181:0/3266260505 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c00ba00 con 0x7f61200d0610 2023-10-16T09:45:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.780+0000 7f6117fff700 1 -- 172.21.15.181:0/3266260505 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c00bd90 con 0x7f61200d0610 2023-10-16T09:45:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 -- 172.21.15.181:0/3266260505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61200d0610 msgr2=0x7f61200cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 --2- 172.21.15.181:0/3266260505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61200d0610 0x7f61200cdd80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f611c0098f0 tx=0x7f611c009c00 comp rx=0 tx=0).stop 2023-10-16T09:45:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 -- 172.21.15.181:0/3266260505 shutdown_connections 2023-10-16T09:45:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 --2- 172.21.15.181:0/3266260505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f612015dbd0 0x7f61200d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 --2- 172.21.15.181:0/3266260505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61200d0610 0x7f61200cdd80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 -- 172.21.15.181:0/3266260505 >> 172.21.15.181:0/3266260505 conn(0x7f612009a930 msgr2=0x7f61200d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 -- 172.21.15.181:0/3266260505 shutdown_connections 2023-10-16T09:45:20.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.781+0000 7f612793e700 1 -- 172.21.15.181:0/3266260505 wait complete. 2023-10-16T09:45:20.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.782+0000 7f612793e700 1 Processor -- start 2023-10-16T09:45:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.782+0000 7f612793e700 1 -- start start 2023-10-16T09:45:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f612793e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61200ce2c0 0x7f6120151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f612793e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61200d0610 0x7f61200b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f612793e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f612015dbd0 0x7f61200c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f612793e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200b4220 con 0x7f612015dbd0 2023-10-16T09:45:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f612793e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6120160650 con 0x7f61200ce2c0 2023-10-16T09:45:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f612793e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6120160930 con 0x7f61200d0610 2023-10-16T09:45:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f6125edb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f612015dbd0 0x7f61200c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f6125edb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f612015dbd0 0x7f61200c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38716/0 (socket says 172.21.15.181:38716) 2023-10-16T09:45:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f6125edb700 1 -- 172.21.15.181:0/3596954436 learned_addr learned my addr 172.21.15.181:0/3596954436 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f6124ed9700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61200d0610 0x7f61200b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.783+0000 7f61256da700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61200ce2c0 0x7f6120151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6125edb700 1 -- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61200d0610 msgr2=0x7f61200b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6125edb700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61200d0610 0x7f61200b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6125edb700 1 -- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61200ce2c0 msgr2=0x7f6120151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6125edb700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61200ce2c0 0x7f6120151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6125edb700 1 -- 172.21.15.181:0/3596954436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c009550 con 0x7f612015dbd0 2023-10-16T09:45:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6125edb700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f612015dbd0 0x7f61200c9850 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f611800cca0 tx=0x7f611800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6115ffb700 1 -- 172.21.15.181:0/3596954436 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800ab10 con 0x7f612015dbd0 2023-10-16T09:45:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6115ffb700 1 -- 172.21.15.181:0/3596954436 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611800ac90 con 0x7f612015dbd0 2023-10-16T09:45:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6120160c10 con 0x7f612015dbd0 2023-10-16T09:45:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f6115ffb700 1 -- 172.21.15.181:0/3596954436 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180070d0 con 0x7f612015dbd0 2023-10-16T09:45:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.784+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6120152390 con 0x7f612015dbd0 2023-10-16T09:45:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.785+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61200b4390 con 0x7f612015dbd0 2023-10-16T09:45:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.785+0000 7f6115ffb700 1 -- 172.21.15.181:0/3596954436 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f6118007250 con 0x7f612015dbd0 2023-10-16T09:45:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.786+0000 7f6115ffb700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f610406b100 0x7f610406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.786+0000 7f6115ffb700 1 -- 172.21.15.181:0/3596954436 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6118089690 con 0x7f612015dbd0 2023-10-16T09:45:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.786+0000 7f61256da700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f610406b100 0x7f610406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.786+0000 7f61256da700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f610406b100 0x7f610406d5c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6110006b20 tx=0x7f6110008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:20.790+0000 7f6115ffb700 1 -- 172.21.15.181:0/3596954436 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61200b4390 con 0x7f612015dbd0 2023-10-16T09:45:21.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.015+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f61200cc790 con 0x7f610406b100 2023-10-16T09:45:21.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.070+0000 7f6115ffb700 1 -- 172.21.15.181:0/3596954436 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f61100076b0 con 0x7f610406b100 2023-10-16T09:45:21.073 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-16T09:45:21.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.072+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f610406b100 msgr2=0x7f610406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.072+0000 7f612793e700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f610406b100 0x7f610406d5c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6110006b20 tx=0x7f6110008040 comp rx=0 tx=0).stop 2023-10-16T09:45:21.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f612015dbd0 msgr2=0x7f61200c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f612015dbd0 0x7f61200c9850 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f611800cca0 tx=0x7f611800a060 comp rx=0 tx=0).stop 2023-10-16T09:45:21.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 shutdown_connections 2023-10-16T09:45:21.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f610406b100 0x7f610406d5c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f612015dbd0 0x7f61200c9850 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61200ce2c0 0x7f6120151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 --2- 172.21.15.181:0/3596954436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61200d0610 0x7f61200b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 >> 172.21.15.181:0/3596954436 conn(0x7f612009a930 msgr2=0x7f61200701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:21.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 shutdown_connections 2023-10-16T09:45:21.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.073+0000 7f612793e700 1 -- 172.21.15.181:0/3596954436 wait complete. 2023-10-16T09:45:21.086 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-16T09:45:21.087 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1m 2023-10-16T09:45:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.208+0000 7feb301e0700 1 Processor -- start 2023-10-16T09:45:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.208+0000 7feb301e0700 1 -- start start 2023-10-16T09:45:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb301e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb2806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb301e0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb2806ade0 0x7feb281615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb301e0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2806b230 con 0x7feb2815da70 2023-10-16T09:45:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb301e0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb280cef00 con 0x7feb2806ade0 2023-10-16T09:45:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb301e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280cec20 con 0x7feb28072760 2023-10-16T09:45:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb2806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb2806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:35750/0 (socket says 172.21.15.181:35750) 2023-10-16T09:45:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2e77d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb2806ade0 0x7feb281615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 -- 172.21.15.181:0/4012438328 learned_addr learned my addr 172.21.15.181:0/4012438328 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 -- 172.21.15.181:0/4012438328 >> v2:172.21.15.190:3300/0 conn(0x7feb2806ade0 msgr2=0x7feb281615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 --2- 172.21.15.181:0/4012438328 >> v2:172.21.15.190:3300/0 conn(0x7feb2806ade0 0x7feb281615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 -- 172.21.15.181:0/4012438328 >> v1:172.21.15.190:6789/0 conn(0x7feb2815da70 legacy=0x7feb280ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 -- 172.21.15.181:0/4012438328 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28071c80 con 0x7feb28072760 2023-10-16T09:45:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2e77d700 1 --2- 172.21.15.181:0/4012438328 >> v2:172.21.15.190:3300/0 conn(0x7feb2806ade0 0x7feb281615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:45:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2d77b700 1 --2- 172.21.15.181:0/4012438328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb2806a8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb240098f0 tx=0x7feb24009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3bdfc2c8cf85f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.209+0000 7feb2cf7a700 1 -- 172.21.15.181:0/4012438328 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24011830 con 0x7feb28072760 2023-10-16T09:45:21.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb2cf7a700 1 -- 172.21.15.181:0/4012438328 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24011990 con 0x7feb28072760 2023-10-16T09:45:21.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb2cf7a700 1 -- 172.21.15.181:0/4012438328 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24011d20 con 0x7feb28072760 2023-10-16T09:45:21.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 -- 172.21.15.181:0/4012438328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 msgr2=0x7feb2806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 --2- 172.21.15.181:0/4012438328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb2806a8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb240098f0 tx=0x7feb24009c00 comp rx=0 tx=0).stop 2023-10-16T09:45:21.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 -- 172.21.15.181:0/4012438328 shutdown_connections 2023-10-16T09:45:21.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 --2- 172.21.15.181:0/4012438328 >> v2:172.21.15.190:3300/0 conn(0x7feb2806ade0 0x7feb281615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 --2- 172.21.15.181:0/4012438328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb2806a8a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 -- 172.21.15.181:0/4012438328 >> 172.21.15.181:0/4012438328 conn(0x7feb280cd010 msgr2=0x7feb280cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:21.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 -- 172.21.15.181:0/4012438328 shutdown_connections 2023-10-16T09:45:21.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.210+0000 7feb301e0700 1 -- 172.21.15.181:0/4012438328 wait complete. 2023-10-16T09:45:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.211+0000 7feb301e0700 1 Processor -- start 2023-10-16T09:45:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.211+0000 7feb301e0700 1 -- start start 2023-10-16T09:45:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb301e0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb2806ade0 0x7feb280af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb301e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb281f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb301e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2815da70 0x7feb280b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb301e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2815c5c0 con 0x7feb2815da70 2023-10-16T09:45:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb301e0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb281f7770 con 0x7feb2806ade0 2023-10-16T09:45:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb301e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280b4d80 con 0x7feb28072760 2023-10-16T09:45:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2e77d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2815da70 0x7feb280b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2e77d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2815da70 0x7feb280b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38730/0 (socket says 172.21.15.181:38730) 2023-10-16T09:45:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2e77d700 1 -- 172.21.15.181:0/3326187063 learned_addr learned my addr 172.21.15.181:0/3326187063 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2d77b700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb281f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2df7c700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb2806ade0 0x7feb280af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2e77d700 1 -- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 msgr2=0x7feb281f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2e77d700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb281f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2e77d700 1 -- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb2806ade0 msgr2=0x7feb280af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.212+0000 7feb2e77d700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb2806ade0 0x7feb280af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.213+0000 7feb2e77d700 1 -- 172.21.15.181:0/3326187063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20009a70 con 0x7feb2815da70 2023-10-16T09:45:21.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.213+0000 7feb2e77d700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2815da70 0x7feb280b4840 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb2000efc0 tx=0x7feb2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:21.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.213+0000 7feb1e7fc700 1 -- 172.21.15.181:0/3326187063 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2000cb40 con 0x7feb2815da70 2023-10-16T09:45:21.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.213+0000 7feb1e7fc700 1 -- 172.21.15.181:0/3326187063 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2000f040 con 0x7feb2815da70 2023-10-16T09:45:21.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.213+0000 7feb1e7fc700 1 -- 172.21.15.181:0/3326187063 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20003a20 con 0x7feb2815da70 2023-10-16T09:45:21.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.213+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb24009550 con 0x7feb2815da70 2023-10-16T09:45:21.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.213+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb280ae700 con 0x7feb2815da70 2023-10-16T09:45:21.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.214+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb280cd710 con 0x7feb2815da70 2023-10-16T09:45:21.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.214+0000 7feb1e7fc700 1 -- 172.21.15.181:0/3326187063 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7feb20014480 con 0x7feb2815da70 2023-10-16T09:45:21.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.215+0000 7feb1e7fc700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7feb0c06b100 0x7feb0c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.215+0000 7feb2df7c700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7feb0c06b100 0x7feb0c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.215+0000 7feb1e7fc700 1 -- 172.21.15.181:0/3326187063 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7feb2008a520 con 0x7feb2815da70 2023-10-16T09:45:21.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.215+0000 7feb2df7c700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7feb0c06b100 0x7feb0c06d5c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb18001620 tx=0x7feb18009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:21.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.219+0000 7feb1e7fc700 1 -- 172.21.15.181:0/3326187063 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feb20055140 con 0x7feb2815da70 2023-10-16T09:45:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.438+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7feb28072240 con 0x7feb0c06b100 2023-10-16T09:45:21.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.493+0000 7feb1e7fc700 1 -- 172.21.15.181:0/3326187063 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7feb18001280 con 0x7feb0c06b100 2023-10-16T09:45:21.495 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-16T09:45:21.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7feb0c06b100 msgr2=0x7feb0c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7feb0c06b100 0x7feb0c06d5c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb18001620 tx=0x7feb18009040 comp rx=0 tx=0).stop 2023-10-16T09:45:21.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2815da70 msgr2=0x7feb280b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2815da70 0x7feb280b4840 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb2000efc0 tx=0x7feb2000c3a0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 shutdown_connections 2023-10-16T09:45:21.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7feb0c06b100 0x7feb0c06d5c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb2815da70 0x7feb280b4840 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb2806ade0 0x7feb280af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 --2- 172.21.15.181:0/3326187063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb28072760 0x7feb281f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 >> 172.21.15.181:0/3326187063 conn(0x7feb280cd010 msgr2=0x7feb280b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:21.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 shutdown_connections 2023-10-16T09:45:21.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.495+0000 7feb301e0700 1 -- 172.21.15.181:0/3326187063 wait complete. 2023-10-16T09:45:21.509 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-16T09:45:21.509 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1m 2023-10-16T09:45:21.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.631+0000 7f9fb96b2700 1 Processor -- start 2023-10-16T09:45:21.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.631+0000 7f9fb96b2700 1 -- start start 2023-10-16T09:45:21.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.631+0000 7f9fb96b2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9fb4071220 0x7f9fb415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.631+0000 7f9fb96b2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb4159790 con 0x7f9fb4071760 2023-10-16T09:45:21.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.631+0000 7f9fb96b2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb415c320 con 0x7f9fb41593b0 2023-10-16T09:45:21.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.631+0000 7f9fb96b2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb415cf70 con 0x7f9fb4071220 2023-10-16T09:45:21.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9fb4071760 0x7f9fb415d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49816/0 (socket says 172.21.15.181:49816) 2023-10-16T09:45:21.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb37fe700 1 -- 172.21.15.181:0/1748990684 learned_addr learned my addr 172.21.15.181:0/1748990684 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:21.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb27fc700 1 --2- 172.21.15.181:0/1748990684 >> v2:172.21.15.190:3300/0 conn(0x7f9fb4071220 0x7f9fb415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb1ffb700 1 -- 172.21.15.181:0/1748990684 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265713533 0 0) 0x7f9fb4159790 con 0x7f9fb4071760 2023-10-16T09:45:21.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb1ffb700 1 -- 172.21.15.181:0/1748990684 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f9c0035b0 con 0x7f9fb4071760 2023-10-16T09:45:21.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb1ffb700 1 -- 172.21.15.181:0/1748990684 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750213588 0 0) 0x7f9fb415c320 con 0x7f9fb41593b0 2023-10-16T09:45:21.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb1ffb700 1 -- 172.21.15.181:0/1748990684 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb4159790 con 0x7f9fb41593b0 2023-10-16T09:45:21.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb27fc700 1 -- 172.21.15.181:0/1748990684 >> v1:172.21.15.190:6789/0 conn(0x7f9fb41593b0 legacy=0x7f9fb4070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb27fc700 1 -- 172.21.15.181:0/1748990684 >> v1:172.21.15.181:6789/0 conn(0x7f9fb4071760 legacy=0x7f9fb415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb27fc700 1 -- 172.21.15.181:0/1748990684 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb415c600 con 0x7f9fb4071220 2023-10-16T09:45:21.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb27fc700 1 --2- 172.21.15.181:0/1748990684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb4071220 0x7f9fb415d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009840 tx=0x7f9fa8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c1654b76bf4e74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:21.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb1ffb700 1 -- 172.21.15.181:0/1748990684 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8011830 con 0x7f9fb4071220 2023-10-16T09:45:21.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb1ffb700 1 -- 172.21.15.181:0/1748990684 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8011990 con 0x7f9fb4071220 2023-10-16T09:45:21.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.632+0000 7f9fb1ffb700 1 -- 172.21.15.181:0/1748990684 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8011d20 con 0x7f9fb4071220 2023-10-16T09:45:21.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.633+0000 7f9fb96b2700 1 -- 172.21.15.181:0/1748990684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb4071220 msgr2=0x7f9fb415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.633+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/1748990684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb4071220 0x7f9fb415d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009840 tx=0x7f9fa8009b50 comp rx=0 tx=0).stop 2023-10-16T09:45:21.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.633+0000 7f9fb96b2700 1 -- 172.21.15.181:0/1748990684 shutdown_connections 2023-10-16T09:45:21.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.633+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/1748990684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb4071220 0x7f9fb415d3c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.633+0000 7f9fb96b2700 1 -- 172.21.15.181:0/1748990684 >> 172.21.15.181:0/1748990684 conn(0x7f9fb406aed0 msgr2=0x7f9fb406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:21.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.633+0000 7f9fb96b2700 1 -- 172.21.15.181:0/1748990684 shutdown_connections 2023-10-16T09:45:21.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.633+0000 7f9fb96b2700 1 -- 172.21.15.181:0/1748990684 wait complete. 2023-10-16T09:45:21.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.634+0000 7f9fb96b2700 1 Processor -- start 2023-10-16T09:45:21.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.634+0000 7f9fb96b2700 1 -- start start 2023-10-16T09:45:21.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb96b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb4071220 0x7f9fb40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb96b2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fb4071760 0x7f9fb40b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb96b2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb41593b0 0x7f9fb40b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb96b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41536a0 con 0x7f9fb4071220 2023-10-16T09:45:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb96b2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41530e0 con 0x7f9fb41593b0 2023-10-16T09:45:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb96b2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41533c0 con 0x7f9fb4071760 2023-10-16T09:45:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb4071220 0x7f9fb40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb4071220 0x7f9fb40c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38740/0 (socket says 172.21.15.181:38740) 2023-10-16T09:45:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb2ffd700 1 -- 172.21.15.181:0/3744950189 learned_addr learned my addr 172.21.15.181:0/3744950189 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb37fe700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb41593b0 0x7f9fb40b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.635+0000 7f9fb27fc700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fb4071760 0x7f9fb40b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb2ffd700 1 -- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fb4071760 msgr2=0x7f9fb40b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb2ffd700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fb4071760 0x7f9fb40b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb2ffd700 1 -- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb41593b0 msgr2=0x7f9fb40b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb2ffd700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb41593b0 0x7f9fb40b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb2ffd700 1 -- 172.21.15.181:0/3744950189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80094a0 con 0x7f9fb4071220 2023-10-16T09:45:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb2ffd700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb4071220 0x7f9fb40c5710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400c500 tx=0x7f9fa400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3744950189 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40052d0 con 0x7f9fb4071220 2023-10-16T09:45:21.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3744950189 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa4005450 con 0x7f9fb4071220 2023-10-16T09:45:21.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb40aeff0 con 0x7f9fb4071220 2023-10-16T09:45:21.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3744950189 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40038c0 con 0x7f9fb4071220 2023-10-16T09:45:21.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.636+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb40c5c50 con 0x7f9fb4071220 2023-10-16T09:45:21.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.637+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb404ebb0 con 0x7f9fb4071220 2023-10-16T09:45:21.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.637+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3744950189 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f9fa4003a20 con 0x7f9fb4071220 2023-10-16T09:45:21.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.638+0000 7f9f9b7fe700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9fa006b100 0x7f9fa006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:21.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.638+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3744950189 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9fa4088d50 con 0x7f9fb4071220 2023-10-16T09:45:21.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.638+0000 7f9fb27fc700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9fa006b100 0x7f9fa006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:21.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.638+0000 7f9fb27fc700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9fa006b100 0x7f9fa006d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009fd0 tx=0x7f9fa801c040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:21.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.642+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3744950189 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9fa408c030 con 0x7f9fb4071220 2023-10-16T09:45:21.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.863+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fb4153250 con 0x7f9fa006b100 2023-10-16T09:45:21.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.925+0000 7f9f9b7fe700 1 -- 172.21.15.181:0/3744950189 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9fb4153250 con 0x7f9fa006b100 2023-10-16T09:45:21.927 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-16T09:45:21.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9fa006b100 msgr2=0x7f9fa006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9fa006b100 0x7f9fa006d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009fd0 tx=0x7f9fa801c040 comp rx=0 tx=0).stop 2023-10-16T09:45:21.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb4071220 msgr2=0x7f9fb40c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:21.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb4071220 0x7f9fb40c5710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400c500 tx=0x7f9fa400c810 comp rx=0 tx=0).stop 2023-10-16T09:45:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 shutdown_connections 2023-10-16T09:45:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9fa006b100 0x7f9fa006d5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fb41593b0 0x7f9fb40b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fb4071220 0x7f9fb40c5710 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 --2- 172.21.15.181:0/3744950189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fb4071760 0x7f9fb40b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 >> 172.21.15.181:0/3744950189 conn(0x7f9fb406aed0 msgr2=0x7f9fb4157550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 shutdown_connections 2023-10-16T09:45:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:21.927+0000 7f9fb96b2700 1 -- 172.21.15.181:0/3744950189 wait complete. 2023-10-16T09:45:21.941 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-16T09:45:21.942 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1m 2023-10-16T09:45:22.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.063+0000 7fd9306a7700 1 Processor -- start 2023-10-16T09:45:22.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.063+0000 7fd9306a7700 1 -- start start 2023-10-16T09:45:22.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.064+0000 7fd9306a7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd928071be0 0x7fd9280ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.064+0000 7fd9306a7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9280ce1f0 con 0x7fd9280b2010 2023-10-16T09:45:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.064+0000 7fd9306a7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9280ccda0 con 0x7fd9280cc6e0 2023-10-16T09:45:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.064+0000 7fd9306a7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280ccb90 con 0x7fd928071be0 2023-10-16T09:45:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.064+0000 7fd92e443700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd9280b2010 0x7fd9280b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49826/0 (socket says 172.21.15.181:49826) 2023-10-16T09:45:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.064+0000 7fd92e443700 1 -- 172.21.15.181:0/2493511054 learned_addr learned my addr 172.21.15.181:0/2493511054 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:22.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131005533 0 0) 0x7fd9280ce1f0 con 0x7fd9280b2010 2023-10-16T09:45:22.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92ec44700 1 --2- 172.21.15.181:0/2493511054 >> v2:172.21.15.190:3300/0 conn(0x7fd928071be0 0x7fd9280ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9140035b0 con 0x7fd9280b2010 2023-10-16T09:45:22.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 81619474 0 0) 0x7fd9140035b0 con 0x7fd9280b2010 2023-10-16T09:45:22.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 >> v2:172.21.15.190:3300/0 conn(0x7fd928071be0 msgr2=0x7fd9280ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:22.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 --2- 172.21.15.181:0/2493511054 >> v2:172.21.15.190:3300/0 conn(0x7fd928071be0 0x7fd9280ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 >> v1:172.21.15.190:6789/0 conn(0x7fd9280cc6e0 legacy=0x7fd9280ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280727e0 con 0x7fd9280b2010 2023-10-16T09:45:22.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd918001670 con 0x7fd9280b2010 2023-10-16T09:45:22.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 >> v1:172.21.15.181:6789/0 conn(0x7fd9280b2010 legacy=0x7fd9280b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd914003a00 0x7fd914003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914004850 0x7fd914004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd914005210 0x7fd91400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280ccb90 con 0x7fd914003a00 2023-10-16T09:45:22.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91400c0b0 con 0x7fd914004850 2023-10-16T09:45:22.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91400e480 con 0x7fd914005210 2023-10-16T09:45:22.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92e443700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd914003a00 0x7fd914003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92dc42700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914004850 0x7fd914004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92ec44700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd914005210 0x7fd91400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92e443700 1 -- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd914005210 msgr2=0x7fd91400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:22.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92e443700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd914005210 0x7fd91400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92e443700 1 -- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914004850 msgr2=0x7fd914004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:22.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92e443700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914004850 0x7fd914004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92ec44700 1 -- 172.21.15.181:0/2493511054 reap_dead start 2023-10-16T09:45:22.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.065+0000 7fd92e443700 1 -- 172.21.15.181:0/2493511054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140035b0 con 0x7fd914003a00 2023-10-16T09:45:22.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd92e443700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd914003a00 0x7fd914003e20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c240 tx=0x7fd914005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b81363852540fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918007c10 con 0x7fd914003a00 2023-10-16T09:45:22.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91800cd80 con 0x7fd914003a00 2023-10-16T09:45:22.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd92d441700 1 -- 172.21.15.181:0/2493511054 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918005ba0 con 0x7fd914003a00 2023-10-16T09:45:22.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd9306a7700 1 -- 172.21.15.181:0/2493511054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd914003a00 msgr2=0x7fd914003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd9306a7700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd914003a00 0x7fd914003e20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c240 tx=0x7fd914005880 comp rx=0 tx=0).stop 2023-10-16T09:45:22.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd9306a7700 1 -- 172.21.15.181:0/2493511054 shutdown_connections 2023-10-16T09:45:22.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd9306a7700 1 --2- 172.21.15.181:0/2493511054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd914003a00 0x7fd914003e20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd9306a7700 1 -- 172.21.15.181:0/2493511054 >> 172.21.15.181:0/2493511054 conn(0x7fd9280cd010 msgr2=0x7fd9280cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd9306a7700 1 -- 172.21.15.181:0/2493511054 shutdown_connections 2023-10-16T09:45:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.066+0000 7fd9306a7700 1 -- 172.21.15.181:0/2493511054 wait complete. 2023-10-16T09:45:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 Processor -- start 2023-10-16T09:45:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 -- start start 2023-10-16T09:45:22.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914003a00 0x7fd9280af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9280afe70 0x7fd92815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92815b0a0 0x7fd928140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92815b550 con 0x7fd92815b0a0 2023-10-16T09:45:22.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280b02f0 con 0x7fd914003a00 2023-10-16T09:45:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd9306a7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd928141000 con 0x7fd9280afe70 2023-10-16T09:45:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd92ec44700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92815b0a0 0x7fd928140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd92ec44700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92815b0a0 0x7fd928140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38764/0 (socket says 172.21.15.181:38764) 2023-10-16T09:45:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.068+0000 7fd92ec44700 1 -- 172.21.15.181:0/92102043 learned_addr learned my addr 172.21.15.181:0/92102043 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92e443700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914003a00 0x7fd9280af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92dc42700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9280afe70 0x7fd92815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92ec44700 1 -- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9280afe70 msgr2=0x7fd92815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92ec44700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9280afe70 0x7fd92815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92ec44700 1 -- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914003a00 msgr2=0x7fd9280af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92ec44700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914003a00 0x7fd9280af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92ec44700 1 -- 172.21.15.181:0/92102043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9281f21c0 con 0x7fd92815b0a0 2023-10-16T09:45:22.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd92ec44700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92815b0a0 0x7fd928140290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd9280675d0 tx=0x7fd9200114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd91effd700 1 -- 172.21.15.181:0/92102043 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000e070 con 0x7fd92815b0a0 2023-10-16T09:45:22.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd91effd700 1 -- 172.21.15.181:0/92102043 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd920003c30 con 0x7fd92815b0a0 2023-10-16T09:45:22.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9281f24a0 con 0x7fd92815b0a0 2023-10-16T09:45:22.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd91effd700 1 -- 172.21.15.181:0/92102043 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920003de0 con 0x7fd92815b0a0 2023-10-16T09:45:22.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.069+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9281f2a60 con 0x7fd92815b0a0 2023-10-16T09:45:22.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.070+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9280b6510 con 0x7fd92815b0a0 2023-10-16T09:45:22.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.070+0000 7fd91effd700 1 -- 172.21.15.181:0/92102043 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fd920011a10 con 0x7fd92815b0a0 2023-10-16T09:45:22.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.071+0000 7fd91effd700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd90c06b100 0x7fd90c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.071+0000 7fd91effd700 1 -- 172.21.15.181:0/92102043 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd920089ee0 con 0x7fd92815b0a0 2023-10-16T09:45:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.071+0000 7fd92e443700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd90c06b100 0x7fd90c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.071+0000 7fd92e443700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd90c06b100 0x7fd90c06d5c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd918000c00 tx=0x7fd918007000 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.075+0000 7fd91effd700 1 -- 172.21.15.181:0/92102043 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd920054b00 con 0x7fd92815b0a0 2023-10-16T09:45:22.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.296+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd928072da0 con 0x7fd90c06b100 2023-10-16T09:45:22.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.341+0000 7fd91effd700 1 -- 172.21.15.181:0/92102043 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd918007b30 con 0x7fd90c06b100 2023-10-16T09:45:22.343 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-16T09:45:22.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd90c06b100 msgr2=0x7fd90c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd90c06b100 0x7fd90c06d5c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd918000c00 tx=0x7fd918007000 comp rx=0 tx=0).stop 2023-10-16T09:45:22.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92815b0a0 msgr2=0x7fd928140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92815b0a0 0x7fd928140290 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd9280675d0 tx=0x7fd9200114b0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 shutdown_connections 2023-10-16T09:45:22.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd90c06b100 0x7fd90c06d5c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92815b0a0 0x7fd928140290 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd914003a00 0x7fd9280af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 --2- 172.21.15.181:0/92102043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9280afe70 0x7fd92815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 >> 172.21.15.181:0/92102043 conn(0x7fd9280cd010 msgr2=0x7fd9280b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:22.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 shutdown_connections 2023-10-16T09:45:22.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.343+0000 7fd9306a7700 1 -- 172.21.15.181:0/92102043 wait complete. 2023-10-16T09:45:22.357 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-16T09:45:22.358 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1m 2023-10-16T09:45:22.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4049e40700 1 Processor -- start 2023-10-16T09:45:22.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4049e40700 1 -- start start 2023-10-16T09:45:22.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4049e40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f404406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4049e40700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f404415ac20 0x7f404415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4049e40700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404415f150 con 0x7f40440ceb60 2023-10-16T09:45:22.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4049e40700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440cb340 con 0x7f404415ac20 2023-10-16T09:45:22.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4049e40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404415b0d0 con 0x7f4044158360 2023-10-16T09:45:22.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.480+0000 7f4042ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f404415ac20 0x7f404415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f4042ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f404415ac20 0x7f404415ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38768/0 (socket says 172.21.15.181:38768) 2023-10-16T09:45:22.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f4042ffd700 1 -- 172.21.15.181:0/844483002 learned_addr learned my addr 172.21.15.181:0/844483002 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:22.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f40437fe700 1 --2- 172.21.15.181:0/844483002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f404406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f4042ffd700 1 -- 172.21.15.181:0/844483002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 msgr2=0x7f404406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:22.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f4042ffd700 1 --2- 172.21.15.181:0/844483002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f404406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f4042ffd700 1 -- 172.21.15.181:0/844483002 >> v1:172.21.15.181:6789/0 conn(0x7f40440ceb60 legacy=0x7f40440cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f4042ffd700 1 -- 172.21.15.181:0/844483002 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404415b3b0 con 0x7f404415ac20 2023-10-16T09:45:22.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f4042ffd700 1 --2- 172.21.15.181:0/844483002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404415ec10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f40340098f0 tx=0x7f4034009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac42768fcdf72a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f40427fc700 1 -- 172.21.15.181:0/844483002 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403400b8a0 con 0x7f404415ac20 2023-10-16T09:45:22.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f40427fc700 1 -- 172.21.15.181:0/844483002 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403400ba00 con 0x7f404415ac20 2023-10-16T09:45:22.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.481+0000 7f40427fc700 1 -- 172.21.15.181:0/844483002 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403400bd90 con 0x7f404415ac20 2023-10-16T09:45:22.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 -- 172.21.15.181:0/844483002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 msgr2=0x7f404415ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 --2- 172.21.15.181:0/844483002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404415ec10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f40340098f0 tx=0x7f4034009c00 comp rx=0 tx=0).stop 2023-10-16T09:45:22.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 -- 172.21.15.181:0/844483002 shutdown_connections 2023-10-16T09:45:22.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 --2- 172.21.15.181:0/844483002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f404406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 --2- 172.21.15.181:0/844483002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404415ec10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 -- 172.21.15.181:0/844483002 >> 172.21.15.181:0/844483002 conn(0x7f40440cc730 msgr2=0x7f40440ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:22.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 -- 172.21.15.181:0/844483002 shutdown_connections 2023-10-16T09:45:22.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.482+0000 7f4049e40700 1 -- 172.21.15.181:0/844483002 wait complete. 2023-10-16T09:45:22.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.483+0000 7f4049e40700 1 Processor -- start 2023-10-16T09:45:22.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.483+0000 7f4049e40700 1 -- start start 2023-10-16T09:45:22.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4049e40700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40440ceb60 0x7f4044151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4049e40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f4044152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4049e40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404414fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4049e40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404415a2a0 con 0x7f404415ac20 2023-10-16T09:45:22.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4049e40700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4044152060 con 0x7f40440ceb60 2023-10-16T09:45:22.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4049e40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404414f860 con 0x7f4044158360 2023-10-16T09:45:22.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404414fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404414fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38776/0 (socket says 172.21.15.181:38776) 2023-10-16T09:45:22.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 -- 172.21.15.181:0/603028356 learned_addr learned my addr 172.21.15.181:0/603028356 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:22.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4042ffd700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f4044152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 -- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 msgr2=0x7f4044152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f4044152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 -- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40440ceb60 msgr2=0x7f4044151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f40437fe700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40440ceb60 0x7f4044151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40440ceb60 0x7f4044151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4043fff700 1 -- 172.21.15.181:0/603028356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034009550 con 0x7f404415ac20 2023-10-16T09:45:22.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.484+0000 7f4042ffd700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f4044152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:45:22.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.485+0000 7f4043fff700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404414fa90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f403c00cc50 tx=0x7f403c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.485+0000 7f403bfff700 1 -- 172.21.15.181:0/603028356 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c00ad80 con 0x7f404415ac20 2023-10-16T09:45:22.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.485+0000 7f403bfff700 1 -- 172.21.15.181:0/603028356 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c00ce50 con 0x7f404415ac20 2023-10-16T09:45:22.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.485+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4044158870 con 0x7f404415ac20 2023-10-16T09:45:22.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.485+0000 7f403bfff700 1 -- 172.21.15.181:0/603028356 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c003c40 con 0x7f404415ac20 2023-10-16T09:45:22.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.485+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4044158b10 con 0x7f404415ac20 2023-10-16T09:45:22.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.486+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4044066850 con 0x7f404415ac20 2023-10-16T09:45:22.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.486+0000 7f403bfff700 1 -- 172.21.15.181:0/603028356 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f403c0050d0 con 0x7f404415ac20 2023-10-16T09:45:22.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.487+0000 7f403bfff700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f402806b100 0x7f402806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.487+0000 7f403bfff700 1 -- 172.21.15.181:0/603028356 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f403c089b70 con 0x7f404415ac20 2023-10-16T09:45:22.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.487+0000 7f40437fe700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f402806b100 0x7f402806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.487+0000 7f40437fe700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f402806b100 0x7f402806d5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f402c009950 tx=0x7f402c008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.491+0000 7f403bfff700 1 -- 172.21.15.181:0/603028356 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f403c054930 con 0x7f404415ac20 2023-10-16T09:45:22.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.714+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f404415b970 con 0x7f402806b100 2023-10-16T09:45:22.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.766+0000 7f403bfff700 1 -- 172.21.15.181:0/603028356 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f402c0095e0 con 0x7f402806b100 2023-10-16T09:45:22.768 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-16T09:45:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f402806b100 msgr2=0x7f402806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f402806b100 0x7f402806d5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f402c009950 tx=0x7f402c008040 comp rx=0 tx=0).stop 2023-10-16T09:45:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 msgr2=0x7f404414fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404414fa90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f403c00cc50 tx=0x7f403c00a340 comp rx=0 tx=0).stop 2023-10-16T09:45:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 shutdown_connections 2023-10-16T09:45:22.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f402806b100 0x7f402806d5c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f404415ac20 0x7f404414fa90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40440ceb60 0x7f4044151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 --2- 172.21.15.181:0/603028356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4044158360 0x7f4044152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 >> 172.21.15.181:0/603028356 conn(0x7f40440cc730 msgr2=0x7f4044071200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:22.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 shutdown_connections 2023-10-16T09:45:22.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.768+0000 7f4049e40700 1 -- 172.21.15.181:0/603028356 wait complete. 2023-10-16T09:45:22.782 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-16T09:45:22.783 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1m 2023-10-16T09:45:22.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.905+0000 7ff8343f5700 1 Processor -- start 2023-10-16T09:45:22.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.905+0000 7ff8343f5700 1 -- start start 2023-10-16T09:45:22.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff8343f5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff82c15dbd0 0x7ff82c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff8343f5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82c15dfb0 con 0x7ff82c0d0610 2023-10-16T09:45:22.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff8343f5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82c0cd3a0 con 0x7ff82c0ce490 2023-10-16T09:45:22.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff8343f5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0cd830 con 0x7ff82c15dbd0 2023-10-16T09:45:22.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff831990700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff82c0d0610 0x7ff82c0cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49858/0 (socket says 172.21.15.181:49858) 2023-10-16T09:45:22.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff831990700 1 -- 172.21.15.181:0/3210835190 learned_addr learned my addr 172.21.15.181:0/3210835190 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:22.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131085996 0 0) 0x7ff81c001110 con 0x7ff82c0d0610 2023-10-16T09:45:22.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8180035b0 con 0x7ff82c0d0610 2023-10-16T09:45:22.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff832191700 1 --2- 172.21.15.181:0/3210835190 >> v2:172.21.15.190:3300/0 conn(0x7ff82c15dbd0 0x7ff82c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337234180 0 0) 0x7ff8180035b0 con 0x7ff82c0d0610 2023-10-16T09:45:22.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 >> v2:172.21.15.190:3300/0 conn(0x7ff82c15dbd0 msgr2=0x7ff82c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:22.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 --2- 172.21.15.181:0/3210835190 >> v2:172.21.15.190:3300/0 conn(0x7ff82c15dbd0 0x7ff82c0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 >> v1:172.21.15.190:6789/0 conn(0x7ff82c0ce490 legacy=0x7ff82c06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0d1fc0 con 0x7ff82c0d0610 2023-10-16T09:45:22.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff81c001800 con 0x7ff82c0d0610 2023-10-16T09:45:22.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 >> v1:172.21.15.181:6789/0 conn(0x7ff82c0d0610 legacy=0x7ff82c0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff818003a00 0x7ff818003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff818004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.906+0000 7ff83118f700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818005210 0x7ff81800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0cd830 con 0x7ff818004850 2023-10-16T09:45:22.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81800c0b0 con 0x7ff818005210 2023-10-16T09:45:22.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81800c270 con 0x7ff818003a00 2023-10-16T09:45:22.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff831990700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff818004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff832191700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff818003a00 0x7ff818003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff832992700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818005210 0x7ff81800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff831990700 1 -- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff818003a00 msgr2=0x7ff818003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:22.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff831990700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff818003a00 0x7ff818003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff831990700 1 -- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818005210 msgr2=0x7ff81800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:22.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff831990700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818005210 0x7ff81800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff831990700 1 -- 172.21.15.181:0/3210835190 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8180035b0 con 0x7ff818004850 2023-10-16T09:45:22.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff832992700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818005210 0x7ff81800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:22.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff832191700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff818003a00 0x7ff818003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:22.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff832992700 1 -- 172.21.15.181:0/3210835190 reap_dead start 2023-10-16T09:45:22.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff831990700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff818004cd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c210 tx=0x7ff818005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990e6f18a10850c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c007db0 con 0x7ff818004850 2023-10-16T09:45:22.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c00cdb0 con 0x7ff818004850 2023-10-16T09:45:22.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.907+0000 7ff83118f700 1 -- 172.21.15.181:0/3210835190 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c005b80 con 0x7ff818004850 2023-10-16T09:45:22.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.908+0000 7ff8343f5700 1 -- 172.21.15.181:0/3210835190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 msgr2=0x7ff818004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.908+0000 7ff8343f5700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff818004cd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c210 tx=0x7ff818005880 comp rx=0 tx=0).stop 2023-10-16T09:45:22.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.908+0000 7ff8343f5700 1 -- 172.21.15.181:0/3210835190 shutdown_connections 2023-10-16T09:45:22.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.908+0000 7ff8343f5700 1 --2- 172.21.15.181:0/3210835190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff818004cd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.908+0000 7ff8343f5700 1 -- 172.21.15.181:0/3210835190 >> 172.21.15.181:0/3210835190 conn(0x7ff82c09a930 msgr2=0x7ff82c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:22.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.908+0000 7ff8343f5700 1 -- 172.21.15.181:0/3210835190 shutdown_connections 2023-10-16T09:45:22.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.908+0000 7ff8343f5700 1 -- 172.21.15.181:0/3210835190 wait complete. 2023-10-16T09:45:22.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.909+0000 7ff8343f5700 1 Processor -- start 2023-10-16T09:45:22.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.909+0000 7ff8343f5700 1 -- start start 2023-10-16T09:45:22.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff8343f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff82c0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff8343f5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0b5e00 0x7ff82c15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff8343f5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c158270 0x7ff82c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff8343f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c161400 con 0x7ff818004850 2023-10-16T09:45:22.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff8343f5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0b7220 con 0x7ff82c158270 2023-10-16T09:45:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff8343f5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c160e80 con 0x7ff82c0b5e00 2023-10-16T09:45:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff832191700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff82c0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff832191700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff82c0b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38790/0 (socket says 172.21.15.181:38790) 2023-10-16T09:45:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff832191700 1 -- 172.21.15.181:0/2550860241 learned_addr learned my addr 172.21.15.181:0/2550860241 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.910+0000 7ff831990700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0b5e00 0x7ff82c15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff832992700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c158270 0x7ff82c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff832191700 1 -- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0b5e00 msgr2=0x7ff82c15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff832191700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0b5e00 0x7ff82c15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff832191700 1 -- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c158270 msgr2=0x7ff82c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:22.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff832191700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c158270 0x7ff82c0b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:22.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff832191700 1 -- 172.21.15.181:0/2550860241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c161120 con 0x7ff818004850 2023-10-16T09:45:22.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff832191700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff82c0b47f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff828011590 tx=0x7ff828011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff822ffd700 1 -- 172.21.15.181:0/2550860241 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828004220 con 0x7ff818004850 2023-10-16T09:45:22.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff822ffd700 1 -- 172.21.15.181:0/2550860241 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828004380 con 0x7ff818004850 2023-10-16T09:45:22.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c0b6eb0 con 0x7ff818004850 2023-10-16T09:45:22.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff822ffd700 1 -- 172.21.15.181:0/2550860241 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8280036a0 con 0x7ff818004850 2023-10-16T09:45:22.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.911+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c15e690 con 0x7ff818004850 2023-10-16T09:45:22.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.912+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c0b6d10 con 0x7ff818004850 2023-10-16T09:45:22.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.912+0000 7ff822ffd700 1 -- 172.21.15.181:0/2550860241 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff828003800 con 0x7ff818004850 2023-10-16T09:45:22.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.913+0000 7ff822ffd700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff81006b100 0x7ff81006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.913+0000 7ff822ffd700 1 -- 172.21.15.181:0/2550860241 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff828089d70 con 0x7ff818004850 2023-10-16T09:45:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.913+0000 7ff831990700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff81006b100 0x7ff81006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.913+0000 7ff831990700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff81006b100 0x7ff81006d5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff81c015040 tx=0x7ff81c007000 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:22.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:22.917+0000 7ff822ffd700 1 -- 172.21.15.181:0/2550860241 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff82c0b6d10 con 0x7ff818004850 2023-10-16T09:45:23.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.139+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff82c0cca70 con 0x7ff81006b100 2023-10-16T09:45:23.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.200+0000 7ff822ffd700 1 -- 172.21.15.181:0/2550860241 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff81c001110 con 0x7ff81006b100 2023-10-16T09:45:23.203 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-16T09:45:23.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff81006b100 msgr2=0x7ff81006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff81006b100 0x7ff81006d5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff81c015040 tx=0x7ff81c007000 comp rx=0 tx=0).stop 2023-10-16T09:45:23.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 msgr2=0x7ff82c0b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff82c0b47f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff828011590 tx=0x7ff828011950 comp rx=0 tx=0).stop 2023-10-16T09:45:23.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 shutdown_connections 2023-10-16T09:45:23.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff81006b100 0x7ff81006d5c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff82c158270 0x7ff82c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff818004850 0x7ff82c0b47f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 --2- 172.21.15.181:0/2550860241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82c0b5e00 0x7ff82c15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 >> 172.21.15.181:0/2550860241 conn(0x7ff82c09a930 msgr2=0x7ff82c06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:23.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 shutdown_connections 2023-10-16T09:45:23.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.203+0000 7ff8343f5700 1 -- 172.21.15.181:0/2550860241 wait complete. 2023-10-16T09:45:23.217 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-16T09:45:23.217 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1m 2023-10-16T09:45:23.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.338+0000 7f9b1c187700 1 Processor -- start 2023-10-16T09:45:23.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.339+0000 7f9b1c187700 1 -- start start 2023-10-16T09:45:23.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.339+0000 7f9b1c187700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b1406afb0 0x7f9b141615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.339+0000 7f9b1c187700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b140cef00 con 0x7f9b14072760 2023-10-16T09:45:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.339+0000 7f9b1c187700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b140cec20 con 0x7f9b1415da70 2023-10-16T09:45:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.339+0000 7f9b1c187700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b14071c80 con 0x7f9b1406afb0 2023-10-16T09:45:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b19722700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9b14072760 0x7f9b1406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49876/0 (socket says 172.21.15.181:49876) 2023-10-16T09:45:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b19722700 1 -- 172.21.15.181:0/564303472 learned_addr learned my addr 172.21.15.181:0/564303472 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:23.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b1a724700 1 --2- 172.21.15.181:0/564303472 >> v2:172.21.15.190:3300/0 conn(0x7f9b1406afb0 0x7f9b141615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352066281 0 0) 0x7f9b140cef00 con 0x7f9b14072760 2023-10-16T09:45:23.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b000035b0 con 0x7f9b14072760 2023-10-16T09:45:23.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210166477 0 0) 0x7f9b000035b0 con 0x7f9b14072760 2023-10-16T09:45:23.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 >> v2:172.21.15.190:3300/0 conn(0x7f9b1406afb0 msgr2=0x7f9b141615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 --2- 172.21.15.181:0/564303472 >> v2:172.21.15.190:3300/0 conn(0x7f9b1406afb0 0x7f9b141615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 >> v1:172.21.15.190:6789/0 conn(0x7f9b1415da70 legacy=0x7f9b140ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b14071f60 con 0x7f9b14072760 2023-10-16T09:45:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f9b10001670 con 0x7f9b14072760 2023-10-16T09:45:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 >> v1:172.21.15.181:6789/0 conn(0x7f9b14072760 legacy=0x7f9b1406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b00003a00 0x7f9b00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b00004850 0x7f9b00004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b14071c80 con 0x7f9b00005330 2023-10-16T09:45:23.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0000c200 con 0x7f9b00004850 2023-10-16T09:45:23.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0000c3c0 con 0x7f9b00003a00 2023-10-16T09:45:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b1a724700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b19722700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b00004850 0x7f9b00004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.340+0000 7f9b19f23700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b00003a00 0x7f9b00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1a724700 1 -- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b00003a00 msgr2=0x7f9b00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1a724700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b00003a00 0x7f9b00003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1a724700 1 -- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b00004850 msgr2=0x7f9b00004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1a724700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b00004850 0x7f9b00004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1a724700 1 -- 172.21.15.181:0/564303472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b000035b0 con 0x7f9b00005330 2023-10-16T09:45:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1a724700 1 -- 172.21.15.181:0/564303472 reap_dead start 2023-10-16T09:45:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1a724700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b0000ba10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009f90 tx=0x7f9b00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484d657c814f5c40 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:23.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c003a20 con 0x7f9b00005330 2023-10-16T09:45:23.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c00f040 con 0x7f9b00005330 2023-10-16T09:45:23.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b18f21700 1 -- 172.21.15.181:0/564303472 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c003dc0 con 0x7f9b00005330 2023-10-16T09:45:23.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1c187700 1 -- 172.21.15.181:0/564303472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 msgr2=0x7f9b0000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1c187700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b0000ba10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009f90 tx=0x7f9b00005880 comp rx=0 tx=0).stop 2023-10-16T09:45:23.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1c187700 1 -- 172.21.15.181:0/564303472 shutdown_connections 2023-10-16T09:45:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1c187700 1 --2- 172.21.15.181:0/564303472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b0000ba10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1c187700 1 -- 172.21.15.181:0/564303472 >> 172.21.15.181:0/564303472 conn(0x7f9b140cd010 msgr2=0x7f9b140cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.341+0000 7f9b1c187700 1 -- 172.21.15.181:0/564303472 shutdown_connections 2023-10-16T09:45:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.342+0000 7f9b1c187700 1 -- 172.21.15.181:0/564303472 wait complete. 2023-10-16T09:45:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 Processor -- start 2023-10-16T09:45:23.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 -- start start 2023-10-16T09:45:23.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b141f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b1415f020 0x7f9b140c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b1415bbe0 0x7f9b1415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b14160210 con 0x7f9b00005330 2023-10-16T09:45:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1415c5a0 con 0x7f9b1415bbe0 2023-10-16T09:45:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.343+0000 7f9b1c187700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140c61f0 con 0x7f9b1415f020 2023-10-16T09:45:23.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b141f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b141f2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38810/0 (socket says 172.21.15.181:38810) 2023-10-16T09:45:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 -- 172.21.15.181:0/2620537127 learned_addr learned my addr 172.21.15.181:0/2620537127 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19722700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b1415f020 0x7f9b140c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b1a724700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b1415bbe0 0x7f9b1415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 -- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b1415f020 msgr2=0x7f9b140c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b1415f020 0x7f9b140c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 -- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b1415bbe0 msgr2=0x7f9b1415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b1415bbe0 0x7f9b1415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 -- 172.21.15.181:0/2620537127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1415f590 con 0x7f9b00005330 2023-10-16T09:45:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19722700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b1415f020 0x7f9b140c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:23.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b19f23700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b141f2e20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400f850 tx=0x7f9b0400fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:23.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b0a7fc700 1 -- 172.21.15.181:0/2620537127 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b040050d0 con 0x7f9b00005330 2023-10-16T09:45:23.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b0a7fc700 1 -- 172.21.15.181:0/2620537127 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04005230 con 0x7f9b00005330 2023-10-16T09:45:23.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b0a7fc700 1 -- 172.21.15.181:0/2620537127 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04003680 con 0x7f9b00005330 2023-10-16T09:45:23.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1415f870 con 0x7f9b00005330 2023-10-16T09:45:23.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.344+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1415fe30 con 0x7f9b00005330 2023-10-16T09:45:23.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.345+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b14160480 con 0x7f9b00005330 2023-10-16T09:45:23.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.346+0000 7f9b0a7fc700 1 -- 172.21.15.181:0/2620537127 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f9b040037e0 con 0x7f9b00005330 2023-10-16T09:45:23.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.346+0000 7f9b0a7fc700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9af806b100 0x7f9af806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.346+0000 7f9b0a7fc700 1 -- 172.21.15.181:0/2620537127 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9b040897a0 con 0x7f9b00005330 2023-10-16T09:45:23.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.346+0000 7f9b19722700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9af806b100 0x7f9af806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.346+0000 7f9b19722700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9af806b100 0x7f9af806d5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b1415c7f0 tx=0x7f9b10009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:23.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.350+0000 7f9b0a7fc700 1 -- 172.21.15.181:0/2620537127 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9b040544d0 con 0x7f9b00005330 2023-10-16T09:45:23.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.571+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b1415de50 con 0x7f9af806b100 2023-10-16T09:45:23.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.627+0000 7f9b0a7fc700 1 -- 172.21.15.181:0/2620537127 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9b10004e70 con 0x7f9af806b100 2023-10-16T09:45:23.630 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-16T09:45:23.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9af806b100 msgr2=0x7f9af806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9af806b100 0x7f9af806d5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b1415c7f0 tx=0x7f9b10009040 comp rx=0 tx=0).stop 2023-10-16T09:45:23.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 msgr2=0x7f9b141f2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b141f2e20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400f850 tx=0x7f9b0400fc10 comp rx=0 tx=0).stop 2023-10-16T09:45:23.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 shutdown_connections 2023-10-16T09:45:23.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f9af806b100 0x7f9af806d5c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b1415bbe0 0x7f9b1415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b00005330 0x7f9b141f2e20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 --2- 172.21.15.181:0/2620537127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b1415f020 0x7f9b140c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 >> 172.21.15.181:0/2620537127 conn(0x7f9b140cd010 msgr2=0x7f9b140cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:23.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 shutdown_connections 2023-10-16T09:45:23.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.630+0000 7f9b1c187700 1 -- 172.21.15.181:0/2620537127 wait complete. 2023-10-16T09:45:23.644 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-16T09:45:23.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1m 2023-10-16T09:45:23.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.765+0000 7fbfdc3e0700 1 Processor -- start 2023-10-16T09:45:23.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.765+0000 7fbfdc3e0700 1 -- start start 2023-10-16T09:45:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.766+0000 7fbfdc3e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 0x7fbfd40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.766+0000 7fbfdc3e0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd40b2ae0 con 0x7fbfd40ce4f0 2023-10-16T09:45:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.766+0000 7fbfdc3e0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd40b2dc0 con 0x7fbfd40cd170 2023-10-16T09:45:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.766+0000 7fbfdc3e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40ce970 con 0x7fbfd415f150 2023-10-16T09:45:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.766+0000 7fbfd997b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbfd40ce4f0 0x7fbfd40b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49896/0 (socket says 172.21.15.181:49896) 2023-10-16T09:45:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.766+0000 7fbfd997b700 1 -- 172.21.15.181:0/3840272634 learned_addr learned my addr 172.21.15.181:0/3840272634 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda17c700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 0x7fbfd40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725264996 0 0) 0x7fbfd40b2ae0 con 0x7fbfd40ce4f0 2023-10-16T09:45:23.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc00035b0 con 0x7fbfd40ce4f0 2023-10-16T09:45:23.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3890857189 0 0) 0x7fbfc00035b0 con 0x7fbfd40ce4f0 2023-10-16T09:45:23.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 msgr2=0x7fbfd40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:23.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 0x7fbfd40cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 >> v1:172.21.15.190:6789/0 conn(0x7fbfd40cd170 legacy=0x7fbfd40b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40cf9d0 con 0x7fbfd40ce4f0 2023-10-16T09:45:23.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fbfd0001670 con 0x7fbfd40ce4f0 2023-10-16T09:45:23.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 >> v1:172.21.15.181:6789/0 conn(0x7fbfd40ce4f0 legacy=0x7fbfd40b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0003a30 0x7fbfc0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 0x7fbfc0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc0004d90 0x7fbfc0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40ce970 con 0x7fbfc0004d90 2023-10-16T09:45:23.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc000bac0 con 0x7fbfc0003a30 2023-10-16T09:45:23.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc000bc80 con 0x7fbfd415f150 2023-10-16T09:45:23.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda97d700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc0004d90 0x7fbfc0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfd997b700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 0x7fbfc0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda17c700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0003a30 0x7fbfc0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda97d700 1 -- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 msgr2=0x7fbfc0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda97d700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd415f150 0x7fbfc0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda97d700 1 -- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0003a30 msgr2=0x7fbfc0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda97d700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0003a30 0x7fbfc0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda97d700 1 -- 172.21.15.181:0/3840272634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00035b0 con 0x7fbfc0004d90 2023-10-16T09:45:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.767+0000 7fbfda97d700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc0004d90 0x7fbfc0005210 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00c160 tx=0x7fbfc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a1aab1db16806c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc001db0 con 0x7fbfc0004d90 2023-10-16T09:45:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc00cca0 con 0x7fbfc0004d90 2023-10-16T09:45:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfd917a700 1 -- 172.21.15.181:0/3840272634 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc003c40 con 0x7fbfc0004d90 2023-10-16T09:45:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/3840272634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc0004d90 msgr2=0x7fbfc0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfdc3e0700 1 --2- 172.21.15.181:0/3840272634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc0004d90 0x7fbfc0005210 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00c160 tx=0x7fbfc0005880 comp rx=0 tx=0).stop 2023-10-16T09:45:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfda97d700 1 -- 172.21.15.181:0/3840272634 reap_dead start 2023-10-16T09:45:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/3840272634 shutdown_connections 2023-10-16T09:45:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/3840272634 >> 172.21.15.181:0/3840272634 conn(0x7fbfd409a930 msgr2=0x7fbfd40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/3840272634 shutdown_connections 2023-10-16T09:45:23.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.768+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/3840272634 wait complete. 2023-10-16T09:45:23.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.770+0000 7fbfdc3e0700 1 Processor -- start 2023-10-16T09:45:23.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.770+0000 7fbfdc3e0700 1 -- start start 2023-10-16T09:45:23.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0004d90 0x7fbfd415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd40c6860 0x7fbfd40c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd406f970 0x7fbfd406fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4070360 con 0x7fbfd40c6860 2023-10-16T09:45:23.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40c6440 con 0x7fbfc0004d90 2023-10-16T09:45:23.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40c6dd0 con 0x7fbfd406f970 2023-10-16T09:45:23.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd40c6860 0x7fbfd40c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd40c6860 0x7fbfd40c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38832/0 (socket says 172.21.15.181:38832) 2023-10-16T09:45:23.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 -- 172.21.15.181:0/1376755335 learned_addr learned my addr 172.21.15.181:0/1376755335 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:23.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfda17c700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0004d90 0x7fbfd415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfda97d700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd406f970 0x7fbfd406fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 -- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd406f970 msgr2=0x7fbfd406fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd406f970 0x7fbfd406fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 -- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0004d90 msgr2=0x7fbfd415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:23.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0004d90 0x7fbfd415ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:23.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 -- 172.21.15.181:0/1376755335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40c70b0 con 0x7fbfd40c6860 2023-10-16T09:45:23.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfd997b700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd40c6860 0x7fbfd40c5f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00072c0 tx=0x7fbfd0007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:23.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfcaffd700 1 -- 172.21.15.181:0/1376755335 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00014a0 con 0x7fbfd40c6860 2023-10-16T09:45:23.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfcaffd700 1 -- 172.21.15.181:0/1376755335 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd000f8d0 con 0x7fbfd40c6860 2023-10-16T09:45:23.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40c7390 con 0x7fbfd40c6860 2023-10-16T09:45:23.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfcaffd700 1 -- 172.21.15.181:0/1376755335 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000fc80 con 0x7fbfd40c6860 2023-10-16T09:45:23.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.771+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd40c7950 con 0x7fbfd40c6860 2023-10-16T09:45:23.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.772+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd4067d20 con 0x7fbfd40c6860 2023-10-16T09:45:23.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.773+0000 7fbfcaffd700 1 -- 172.21.15.181:0/1376755335 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fbfd001d460 con 0x7fbfd40c6860 2023-10-16T09:45:23.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.773+0000 7fbfcaffd700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfb806b100 0x7fbfb806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:23.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.773+0000 7fbfcaffd700 1 -- 172.21.15.181:0/1376755335 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fbfd0089ad0 con 0x7fbfd40c6860 2023-10-16T09:45:23.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.773+0000 7fbfda17c700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfb806b100 0x7fbfb806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:23.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.774+0000 7fbfda17c700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfb806b100 0x7fbfb806d5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009ba0 tx=0x7fbfc4008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:23.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:23.778+0000 7fbfcaffd700 1 -- 172.21.15.181:0/1376755335 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfd0054890 con 0x7fbfd40c6860 2023-10-16T09:45:24.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.001+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfd40cff90 con 0x7fbfb806b100 2023-10-16T09:45:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.054+0000 7fbfcaffd700 1 -- 172.21.15.181:0/1376755335 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbfc40097f0 con 0x7fbfb806b100 2023-10-16T09:45:24.056 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-16T09:45:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.056+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfb806b100 msgr2=0x7fbfb806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.056+0000 7fbfdc3e0700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfb806b100 0x7fbfb806d5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009ba0 tx=0x7fbfc4008040 comp rx=0 tx=0).stop 2023-10-16T09:45:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.056+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd40c6860 msgr2=0x7fbfd40c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.056+0000 7fbfdc3e0700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd40c6860 0x7fbfd40c5f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00072c0 tx=0x7fbfd0007680 comp rx=0 tx=0).stop 2023-10-16T09:45:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 shutdown_connections 2023-10-16T09:45:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd406f970 0x7fbfd406fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfb806b100 0x7fbfb806d5c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc0004d90 0x7fbfd415ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 --2- 172.21.15.181:0/1376755335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd40c6860 0x7fbfd40c5f00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 >> 172.21.15.181:0/1376755335 conn(0x7fbfd409a930 msgr2=0x7fbfd40af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:24.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 shutdown_connections 2023-10-16T09:45:24.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.057+0000 7fbfdc3e0700 1 -- 172.21.15.181:0/1376755335 wait complete. 2023-10-16T09:45:24.071 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-16T09:45:24.071 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1m 2023-10-16T09:45:24.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.193+0000 7f5b85bd0700 1 Processor -- start 2023-10-16T09:45:24.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.193+0000 7f5b85bd0700 1 -- start start 2023-10-16T09:45:24.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b85bd0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b8006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b85bd0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8006b230 con 0x7f5b8015da70 2023-10-16T09:45:24.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b85bd0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800cef00 con 0x7f5b8006ade0 2023-10-16T09:45:24.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b85bd0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cec20 con 0x7f5b80072760 2023-10-16T09:45:24.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5b8015da70 0x7f5b800ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49916/0 (socket says 172.21.15.181:49916) 2023-10-16T09:45:24.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/396582317 learned_addr learned my addr 172.21.15.181:0/396582317 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:45:24.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653162270 0 0) 0x7f5b74001110 con 0x7f5b8015da70 2023-10-16T09:45:24.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b680035b0 con 0x7f5b8015da70 2023-10-16T09:45:24.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7effd700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b8006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060009861 0 0) 0x7f5b680035b0 con 0x7f5b8015da70 2023-10-16T09:45:24.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 msgr2=0x7f5b8006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:24.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b8006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 >> v1:172.21.15.190:6789/0 conn(0x7f5b8006ade0 legacy=0x7f5b801615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80071c80 con 0x7f5b8015da70 2023-10-16T09:45:24.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5b74001800 con 0x7f5b8015da70 2023-10-16T09:45:24.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7effd700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b8006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:24.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.194+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 >> v1:172.21.15.181:6789/0 conn(0x7f5b8015da70 legacy=0x7f5b800ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b68003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b68004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b68004d90 0x7f5b68005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cec20 con 0x7f5b68003a30 2023-10-16T09:45:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6800bc70 con 0x7f5b68004d90 2023-10-16T09:45:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6800be30 con 0x7f5b80072760 2023-10-16T09:45:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7f7fe700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b68003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7effd700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b68004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7ffff700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b68004d90 0x7f5b68005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 msgr2=0x7f5b68004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:24.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7f7fe700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b68004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b68004d90 msgr2=0x7f5b68005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:24.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7f7fe700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b68004d90 0x7f5b68005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/396582317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680035b0 con 0x7f5b68003a30 2023-10-16T09:45:24.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7effd700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b80072760 0x7f5b68004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:45:24.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7f7fe700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b68003e50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c240 tx=0x7f5b68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65afb7174f7d6232 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74007db0 con 0x7f5b68003a30 2023-10-16T09:45:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400ce00 con 0x7f5b68003a30 2023-10-16T09:45:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.195+0000 7f5b7e7fc700 1 -- 172.21.15.181:0/396582317 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74005ba0 con 0x7f5b68003a30 2023-10-16T09:45:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.196+0000 7f5b85bd0700 1 -- 172.21.15.181:0/396582317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 msgr2=0x7f5b68003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.196+0000 7f5b85bd0700 1 --2- 172.21.15.181:0/396582317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b68003e50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400c240 tx=0x7f5b68005880 comp rx=0 tx=0).stop 2023-10-16T09:45:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.196+0000 7f5b7ffff700 1 -- 172.21.15.181:0/396582317 reap_dead start 2023-10-16T09:45:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.196+0000 7f5b85bd0700 1 -- 172.21.15.181:0/396582317 shutdown_connections 2023-10-16T09:45:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.196+0000 7f5b85bd0700 1 -- 172.21.15.181:0/396582317 >> 172.21.15.181:0/396582317 conn(0x7f5b800cd010 msgr2=0x7f5b800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.196+0000 7f5b85bd0700 1 -- 172.21.15.181:0/396582317 shutdown_connections 2023-10-16T09:45:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.196+0000 7f5b85bd0700 1 -- 172.21.15.181:0/396582317 wait complete. 2023-10-16T09:45:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.197+0000 7f5b85bd0700 1 Processor -- start 2023-10-16T09:45:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.197+0000 7f5b85bd0700 1 -- start start 2023-10-16T09:45:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b85bd0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b800c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b85bd0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b800c7740 0x7f5b800c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b85bd0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b800b31b0 0x7f5b800b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b85bd0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800c84b0 con 0x7f5b68003a30 2023-10-16T09:45:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b85bd0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800c8790 con 0x7f5b800b31b0 2023-10-16T09:45:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b85bd0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800c8a70 con 0x7f5b800c7740 2023-10-16T09:45:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b800c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b800c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38850/0 (socket says 172.21.15.181:38850) 2023-10-16T09:45:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/3464298949 learned_addr learned my addr 172.21.15.181:0/3464298949 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7effd700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b800c7740 0x7f5b800c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7ffff700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b800b31b0 0x7f5b800b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b800c7740 msgr2=0x7f5b800c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b800c7740 0x7f5b800c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b800b31b0 msgr2=0x7f5b800b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b800b31b0 0x7f5b800b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 -- 172.21.15.181:0/3464298949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800c8d50 con 0x7f5b68003a30 2023-10-16T09:45:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b7f7fe700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b800c9c70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800bc40 tx=0x7f5b74001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.198+0000 7f5b67fff700 1 -- 172.21.15.181:0/3464298949 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740017c0 con 0x7f5b68003a30 2023-10-16T09:45:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.199+0000 7f5b67fff700 1 -- 172.21.15.181:0/3464298949 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b74007d50 con 0x7f5b68003a30 2023-10-16T09:45:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.199+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b800c9030 con 0x7f5b68003a30 2023-10-16T09:45:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.199+0000 7f5b67fff700 1 -- 172.21.15.181:0/3464298949 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74007330 con 0x7f5b68003a30 2023-10-16T09:45:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.199+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b800ca1b0 con 0x7f5b68003a30 2023-10-16T09:45:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.199+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b800c4e00 con 0x7f5b68003a30 2023-10-16T09:45:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.200+0000 7f5b67fff700 1 -- 172.21.15.181:0/3464298949 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f5b74019030 con 0x7f5b68003a30 2023-10-16T09:45:24.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.200+0000 7f5b67fff700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b6c06b100 0x7f5b6c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.200+0000 7f5b67fff700 1 -- 172.21.15.181:0/3464298949 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5b7408aea0 con 0x7f5b68003a30 2023-10-16T09:45:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.200+0000 7f5b7effd700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b6c06b100 0x7f5b6c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.201+0000 7f5b7effd700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b6c06b100 0x7f5b6c06d5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b800b1d60 tx=0x7f5b70008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.205+0000 7f5b67fff700 1 -- 172.21.15.181:0/3464298949 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5b74055c60 con 0x7f5b68003a30 2023-10-16T09:45:24.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.425+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b80072240 con 0x7f5b6c06b100 2023-10-16T09:45:24.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.475+0000 7f5b67fff700 1 -- 172.21.15.181:0/3464298949 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5b7000c040 con 0x7f5b6c06b100 2023-10-16T09:45:24.478 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-16T09:45:24.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.477+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b6c06b100 msgr2=0x7f5b6c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.477+0000 7f5b85bd0700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b6c06b100 0x7f5b6c06d5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b800b1d60 tx=0x7f5b70008040 comp rx=0 tx=0).stop 2023-10-16T09:45:24.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 msgr2=0x7f5b800c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b800c9c70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800bc40 tx=0x7f5b74001b90 comp rx=0 tx=0).stop 2023-10-16T09:45:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 shutdown_connections 2023-10-16T09:45:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b6c06b100 0x7f5b6c06d5c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b800b31b0 0x7f5b800b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68003a30 0x7f5b800c9c70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 --2- 172.21.15.181:0/3464298949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b800c7740 0x7f5b800c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 >> 172.21.15.181:0/3464298949 conn(0x7f5b800cd010 msgr2=0x7f5b8015cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:24.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 shutdown_connections 2023-10-16T09:45:24.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.478+0000 7f5b85bd0700 1 -- 172.21.15.181:0/3464298949 wait complete. 2023-10-16T09:45:24.492 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-16T09:45:24.492 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1m 2023-10-16T09:45:24.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.613+0000 7f71737c6700 1 Processor -- start 2023-10-16T09:45:24.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f71737c6700 1 -- start start 2023-10-16T09:45:24.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f71737c6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f716c15da70 0x7f716c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f71737c6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c0ce6a0 0x7f716c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f71737c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f716c072760 0x7f716c06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f71737c6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c0cef00 con 0x7f716c072760 2023-10-16T09:45:24.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f71737c6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c0cec20 con 0x7f716c15da70 2023-10-16T09:45:24.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f71737c6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c071c80 con 0x7f716c0ce6a0 2023-10-16T09:45:24.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.614+0000 7f7171d63700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f716c072760 0x7f716c06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f716c072760 0x7f716c06ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38862/0 (socket says 172.21.15.181:38862) 2023-10-16T09:45:24.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 -- 172.21.15.181:0/2218320613 learned_addr learned my addr 172.21.15.181:0/2218320613 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:24.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7170d61700 1 --2- 172.21.15.181:0/2218320613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c0ce6a0 0x7f716c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171562700 1 --2- 172.21.15.181:0/2218320613 >> v2:172.21.15.190:3300/0 conn(0x7f716c15da70 0x7f716c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 -- 172.21.15.181:0/2218320613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c0ce6a0 msgr2=0x7f716c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:24.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 --2- 172.21.15.181:0/2218320613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c0ce6a0 0x7f716c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 -- 172.21.15.181:0/2218320613 >> v2:172.21.15.190:3300/0 conn(0x7f716c15da70 msgr2=0x7f716c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:45:24.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 --2- 172.21.15.181:0/2218320613 >> v2:172.21.15.190:3300/0 conn(0x7f716c15da70 0x7f716c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 -- 172.21.15.181:0/2218320613 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c071f60 con 0x7f716c072760 2023-10-16T09:45:24.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7171d63700 1 --2- 172.21.15.181:0/2218320613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c06ade0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f716c0675e0 tx=0x7f716400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c3aa272ca1cb3b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:24.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7163fff700 1 -- 172.21.15.181:0/2218320613 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400e070 con 0x7f716c072760 2023-10-16T09:45:24.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7163fff700 1 -- 172.21.15.181:0/2218320613 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716400b910 con 0x7f716c072760 2023-10-16T09:45:24.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.615+0000 7f7163fff700 1 -- 172.21.15.181:0/2218320613 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400bcc0 con 0x7f716c072760 2023-10-16T09:45:24.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 -- 172.21.15.181:0/2218320613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 msgr2=0x7f716c06ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 --2- 172.21.15.181:0/2218320613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c06ade0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f716c0675e0 tx=0x7f716400b330 comp rx=0 tx=0).stop 2023-10-16T09:45:24.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 -- 172.21.15.181:0/2218320613 shutdown_connections 2023-10-16T09:45:24.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 --2- 172.21.15.181:0/2218320613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c06ade0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 --2- 172.21.15.181:0/2218320613 >> v2:172.21.15.190:3300/0 conn(0x7f716c15da70 0x7f716c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 --2- 172.21.15.181:0/2218320613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c0ce6a0 0x7f716c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 -- 172.21.15.181:0/2218320613 >> 172.21.15.181:0/2218320613 conn(0x7f716c0cd010 msgr2=0x7f716c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:24.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 -- 172.21.15.181:0/2218320613 shutdown_connections 2023-10-16T09:45:24.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.616+0000 7f71737c6700 1 -- 172.21.15.181:0/2218320613 wait complete. 2023-10-16T09:45:24.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.617+0000 7f71737c6700 1 Processor -- start 2023-10-16T09:45:24.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.617+0000 7f71737c6700 1 -- start start 2023-10-16T09:45:24.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f71737c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f71737c6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c0ce6a0 0x7f716c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f71737c6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c15da70 0x7f716c1f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f71737c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0acdb0 con 0x7f716c072760 2023-10-16T09:45:24.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f71737c6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0ad090 con 0x7f716c0ce6a0 2023-10-16T09:45:24.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f71737c6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0ad370 con 0x7f716c15da70 2023-10-16T09:45:24.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38864/0 (socket says 172.21.15.181:38864) 2023-10-16T09:45:24.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 -- 172.21.15.181:0/490724415 learned_addr learned my addr 172.21.15.181:0/490724415 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:45:24.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171d63700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c15da70 0x7f716c1f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7170d61700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c0ce6a0 0x7f716c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 -- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c15da70 msgr2=0x7f716c1f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c15da70 0x7f716c1f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 -- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c0ce6a0 msgr2=0x7f716c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c0ce6a0 0x7f716c0ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 -- 172.21.15.181:0/490724415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164009b30 con 0x7f716c072760 2023-10-16T09:45:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7171562700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c15ff20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f715c00e9b0 tx=0x7f715c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7161ffb700 1 -- 172.21.15.181:0/490724415 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c009710 con 0x7f716c072760 2023-10-16T09:45:24.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7161ffb700 1 -- 172.21.15.181:0/490724415 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c00c850 con 0x7f716c072760 2023-10-16T09:45:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c1f30c0 con 0x7f716c072760 2023-10-16T09:45:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.618+0000 7f7161ffb700 1 -- 172.21.15.181:0/490724415 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00cbc0 con 0x7f716c072760 2023-10-16T09:45:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.619+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c0aedc0 con 0x7f716c072760 2023-10-16T09:45:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.619+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c158230 con 0x7f716c072760 2023-10-16T09:45:24.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.620+0000 7f7161ffb700 1 -- 172.21.15.181:0/490724415 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f715c00cd20 con 0x7f716c072760 2023-10-16T09:45:24.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.620+0000 7f7161ffb700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f715006b100 0x7f715006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:45:24.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.620+0000 7f7161ffb700 1 -- 172.21.15.181:0/490724415 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f715c089b50 con 0x7f716c072760 2023-10-16T09:45:24.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.620+0000 7f7170d61700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f715006b100 0x7f715006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:45:24.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.621+0000 7f7170d61700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f715006b100 0x7f715006d5c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f716c0b19b0 tx=0x7f7168008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:45:24.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.625+0000 7f7161ffb700 1 -- 172.21.15.181:0/490724415 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f715c054880 con 0x7f716c072760 2023-10-16T09:45:24.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.850+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f716c072240 con 0x7f715006b100 2023-10-16T09:45:24.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.902+0000 7f7161ffb700 1 -- 172.21.15.181:0/490724415 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7168009820 con 0x7f715006b100 2023-10-16T09:45:24.905 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-16T09:45:24.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.904+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f715006b100 msgr2=0x7f715006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.904+0000 7f71737c6700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f715006b100 0x7f715006d5c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f716c0b19b0 tx=0x7f7168008040 comp rx=0 tx=0).stop 2023-10-16T09:45:24.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.904+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 msgr2=0x7f716c15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:45:24.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.904+0000 7f71737c6700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c15ff20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f715c00e9b0 tx=0x7f715c00ed70 comp rx=0 tx=0).stop 2023-10-16T09:45:24.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 shutdown_connections 2023-10-16T09:45:24.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f716c15da70 0x7f716c1f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f715006b100 0x7f715006d5c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c072760 0x7f716c15ff20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 --2- 172.21.15.181:0/490724415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c0ce6a0 0x7f716c0ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:45:24.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 >> 172.21.15.181:0/490724415 conn(0x7f716c0cd010 msgr2=0x7f716c161cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:45:24.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 shutdown_connections 2023-10-16T09:45:24.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:45:24.905+0000 7f71737c6700 1 -- 172.21.15.181:0/490724415 wait complete. 2023-10-16T09:45:24.918 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-16T09:45:37.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:45:37.486 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:46:00.067 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:46:00.064+0000 7f24d16b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:46:00.067 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:46:00.064+0000 7f24d16b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:46:00.068 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:46:00.064+0000 7f24d16b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:46:00.068 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:46:00.064+0000 7f24d16b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:46:00.068 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:46:00.064+0000 7f24d16b8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:46:07.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:46:07.549 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:46:37.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:46:37.622 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:07.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:07.712 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:37.780 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:47:37.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:07.851 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:07.854 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:37.966 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:48:37.970 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:08.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:08.039 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:38.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:49:38.103 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:50:08.171 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:50:08.175 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:50:38.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:50:38.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:00.361 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:51:00.360+0000 7f24d8746700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.361 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:51:00.360+0000 7f24d8746700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.362 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:51:00.360+0000 7f24d8746700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.362 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:51:00.360+0000 7f24d8746700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:00.362 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:51:00.360+0000 7f24d8746700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:51:08.331 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:08.334 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:38.394 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:51:38.397 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:08.466 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:08.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:38.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:52:38.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:53:08.626 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:53:08.630 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:53:38.714 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:53:38.717 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:08.786 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:08.789 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:38.858 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:54:38.862 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:08.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:08.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:39.043 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:55:39.046 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:09.107 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:09.110 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:19.920 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1m 2023-10-16T09:56:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.056+0000 7f4d1f67e700 1 Processor -- start 2023-10-16T09:56:20.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.056+0000 7f4d1f67e700 1 -- start start 2023-10-16T09:56:20.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1f67e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d1815f150 0x7f4d180cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1f67e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180ce4f0 0x7f4d180b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1f67e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d180b2ae0 con 0x7f4d180b14a0 2023-10-16T09:56:20.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1f67e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d180ce970 con 0x7f4d1815f150 2023-10-16T09:56:20.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1f67e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b2dc0 con 0x7f4d180ce4f0 2023-10-16T09:56:20.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1d41a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d1815f150 0x7f4d180cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1d41a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d1815f150 0x7f4d180cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38898/0 (socket says 172.21.15.181:38898) 2023-10-16T09:56:20.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1d41a700 1 -- 172.21.15.181:0/3653820063 learned_addr learned my addr 172.21.15.181:0/3653820063 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:20.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1d41a700 1 -- 172.21.15.181:0/3653820063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180ce4f0 msgr2=0x7f4d180b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:20.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1d41a700 1 --2- 172.21.15.181:0/3653820063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180ce4f0 0x7f4d180b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1d41a700 1 -- 172.21.15.181:0/3653820063 >> v1:172.21.15.190:6789/0 conn(0x7f4d180b14a0 legacy=0x7f4d180b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:20.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.057+0000 7f4d1d41a700 1 -- 172.21.15.181:0/3653820063 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180cf9d0 con 0x7f4d1815f150 2023-10-16T09:56:20.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1d41a700 1 --2- 172.21.15.181:0/3653820063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d1815f150 0x7f4d180cdfb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4d080098f0 tx=0x7f4d08009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633aa0c3af3937b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d0ffff700 1 -- 172.21.15.181:0/3653820063 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08004230 con 0x7f4d1815f150 2023-10-16T09:56:20.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d0ffff700 1 -- 172.21.15.181:0/3653820063 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08004390 con 0x7f4d1815f150 2023-10-16T09:56:20.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d0ffff700 1 -- 172.21.15.181:0/3653820063 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d080036a0 con 0x7f4d1815f150 2023-10-16T09:56:20.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3653820063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d1815f150 msgr2=0x7f4d180cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3653820063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d1815f150 0x7f4d180cdfb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4d080098f0 tx=0x7f4d08009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:20.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3653820063 shutdown_connections 2023-10-16T09:56:20.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3653820063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d1815f150 0x7f4d180cdfb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3653820063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180ce4f0 0x7f4d180b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3653820063 >> 172.21.15.181:0/3653820063 conn(0x7f4d1809a930 msgr2=0x7f4d180d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:20.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3653820063 shutdown_connections 2023-10-16T09:56:20.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.058+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3653820063 wait complete. 2023-10-16T09:56:20.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.060+0000 7f4d1f67e700 1 Processor -- start 2023-10-16T09:56:20.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.060+0000 7f4d1f67e700 1 -- start start 2023-10-16T09:56:20.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1f67e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180b14a0 0x7f4d180b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1f67e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d180ce4f0 0x7f4d1814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1f67e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1815f150 0x7f4d180b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1f67e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b5fc0 con 0x7f4d180ce4f0 2023-10-16T09:56:20.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1f67e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b53b0 con 0x7f4d180b14a0 2023-10-16T09:56:20.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1f67e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b5690 con 0x7f4d1815f150 2023-10-16T09:56:20.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d180ce4f0 0x7f4d1814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d180ce4f0 0x7f4d1814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38906/0 (socket says 172.21.15.181:38906) 2023-10-16T09:56:20.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 -- 172.21.15.181:0/3799037468 learned_addr learned my addr 172.21.15.181:0/3799037468 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:20.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1d41a700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180b14a0 0x7f4d180b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1dc1b700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1815f150 0x7f4d180b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 -- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1815f150 msgr2=0x7f4d180b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1815f150 0x7f4d180b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 -- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180b14a0 msgr2=0x7f4d180b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180b14a0 0x7f4d180b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.061+0000 7f4d1cc19700 1 -- 172.21.15.181:0/3799037468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08009550 con 0x7f4d180ce4f0 2023-10-16T09:56:20.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.062+0000 7f4d1cc19700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d180ce4f0 0x7f4d1814dcd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400b530 tx=0x7f4d1400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.062+0000 7f4d0dffb700 1 -- 172.21.15.181:0/3799037468 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14011830 con 0x7f4d180ce4f0 2023-10-16T09:56:20.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.062+0000 7f4d0dffb700 1 -- 172.21.15.181:0/3799037468 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d140119b0 con 0x7f4d180ce4f0 2023-10-16T09:56:20.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.062+0000 7f4d0dffb700 1 -- 172.21.15.181:0/3799037468 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14011d60 con 0x7f4d180ce4f0 2023-10-16T09:56:20.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.062+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d180b5970 con 0x7f4d180ce4f0 2023-10-16T09:56:20.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.062+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1814e210 con 0x7f4d180ce4f0 2023-10-16T09:56:20.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.063+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d180b6130 con 0x7f4d180ce4f0 2023-10-16T09:56:20.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.063+0000 7f4d0dffb700 1 -- 172.21.15.181:0/3799037468 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4d1401a400 con 0x7f4d180ce4f0 2023-10-16T09:56:20.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.064+0000 7f4d0dffb700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4cfc06b100 0x7f4cfc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.064+0000 7f4d0dffb700 1 -- 172.21.15.181:0/3799037468 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4d14089b50 con 0x7f4d180ce4f0 2023-10-16T09:56:20.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.064+0000 7f4d1d41a700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4cfc06b100 0x7f4cfc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.064+0000 7f4d1d41a700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4cfc06b100 0x7f4cfc06d5c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009130 tx=0x7f4d08008030 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.068+0000 7f4d0dffb700 1 -- 172.21.15.181:0/3799037468 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4d14054880 con 0x7f4d180ce4f0 2023-10-16T09:56:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.289+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d180cfcb0 con 0x7f4cfc06b100 2023-10-16T09:56:20.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.357+0000 7f4d0dffb700 1 -- 172.21.15.181:0/3799037468 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4d08020070 con 0x7f4cfc06b100 2023-10-16T09:56:20.359 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-16T09:56:20.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4cfc06b100 msgr2=0x7f4cfc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4cfc06b100 0x7f4cfc06d5c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009130 tx=0x7f4d08008030 comp rx=0 tx=0).stop 2023-10-16T09:56:20.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d180ce4f0 msgr2=0x7f4d1814dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d180ce4f0 0x7f4d1814dcd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400b530 tx=0x7f4d1400b8f0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 shutdown_connections 2023-10-16T09:56:20.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1815f150 0x7f4d180b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4cfc06b100 0x7f4cfc06d5c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180b14a0 0x7f4d180b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 --2- 172.21.15.181:0/3799037468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d180ce4f0 0x7f4d1814dcd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 >> 172.21.15.181:0/3799037468 conn(0x7f4d1809a930 msgr2=0x7f4d1815e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:20.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 shutdown_connections 2023-10-16T09:56:20.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.359+0000 7f4d1f67e700 1 -- 172.21.15.181:0/3799037468 wait complete. 2023-10-16T09:56:20.372 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-16T09:56:20.373 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1m 2023-10-16T09:56:20.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.495+0000 7f5b27d55700 1 Processor -- start 2023-10-16T09:56:20.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b27d55700 1 -- start start 2023-10-16T09:56:20.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b27d55700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b2015da70 0x7f5b200ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b27d55700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b200ce6a0 0x7f5b2006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b27d55700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200cef00 con 0x7f5b20072760 2023-10-16T09:56:20.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b27d55700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b200cec20 con 0x7f5b200ce6a0 2023-10-16T09:56:20.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b27d55700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20071c80 con 0x7f5b2015da70 2023-10-16T09:56:20.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b252f0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b200ce6a0 0x7f5b2006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b262f2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5b20072760 0x7f5b201615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:49994/0 (socket says 172.21.15.181:49994) 2023-10-16T09:56:20.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.496+0000 7f5b262f2700 1 -- 172.21.15.181:0/2268732597 learned_addr learned my addr 172.21.15.181:0/2268732597 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:20.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b25af1700 1 --2- 172.21.15.181:0/2268732597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b2015da70 0x7f5b200ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b24aef700 1 -- 172.21.15.181:0/2268732597 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159047439 0 0) 0x7f5b200cef00 con 0x7f5b20072760 2023-10-16T09:56:20.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b24aef700 1 -- 172.21.15.181:0/2268732597 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b0c0035b0 con 0x7f5b20072760 2023-10-16T09:56:20.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b252f0700 1 -- 172.21.15.181:0/2268732597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b2015da70 msgr2=0x7f5b200ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:20.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b252f0700 1 --2- 172.21.15.181:0/2268732597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b2015da70 0x7f5b200ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b252f0700 1 -- 172.21.15.181:0/2268732597 >> v1:172.21.15.181:6789/0 conn(0x7f5b20072760 legacy=0x7f5b201615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b252f0700 1 -- 172.21.15.181:0/2268732597 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b20071f60 con 0x7f5b200ce6a0 2023-10-16T09:56:20.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b252f0700 1 --2- 172.21.15.181:0/2268732597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b200ce6a0 0x7f5b2006a8a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009840 tx=0x7f5b1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf54666bfed1b0aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b24aef700 1 -- 172.21.15.181:0/2268732597 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c011830 con 0x7f5b200ce6a0 2023-10-16T09:56:20.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b24aef700 1 -- 172.21.15.181:0/2268732597 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c011990 con 0x7f5b200ce6a0 2023-10-16T09:56:20.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.497+0000 7f5b24aef700 1 -- 172.21.15.181:0/2268732597 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c011d20 con 0x7f5b200ce6a0 2023-10-16T09:56:20.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 -- 172.21.15.181:0/2268732597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b200ce6a0 msgr2=0x7f5b2006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 --2- 172.21.15.181:0/2268732597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b200ce6a0 0x7f5b2006a8a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009840 tx=0x7f5b1c009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:20.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 -- 172.21.15.181:0/2268732597 shutdown_connections 2023-10-16T09:56:20.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 --2- 172.21.15.181:0/2268732597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b2015da70 0x7f5b200ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 --2- 172.21.15.181:0/2268732597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b200ce6a0 0x7f5b2006a8a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 -- 172.21.15.181:0/2268732597 >> 172.21.15.181:0/2268732597 conn(0x7f5b200cd010 msgr2=0x7f5b200cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:20.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 -- 172.21.15.181:0/2268732597 shutdown_connections 2023-10-16T09:56:20.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.498+0000 7f5b27d55700 1 -- 172.21.15.181:0/2268732597 wait complete. 2023-10-16T09:56:20.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.499+0000 7f5b27d55700 1 Processor -- start 2023-10-16T09:56:20.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.499+0000 7f5b27d55700 1 -- start start 2023-10-16T09:56:20.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b20072760 0x7f5b201f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b200ce6a0 0x7f5b20160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b2015da70 0x7f5b201f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200c7050 con 0x7f5b20072760 2023-10-16T09:56:20.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b201f4150 con 0x7f5b2015da70 2023-10-16T09:56:20.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b201f6c70 con 0x7f5b200ce6a0 2023-10-16T09:56:20.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b20072760 0x7f5b201f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b20072760 0x7f5b201f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38918/0 (socket says 172.21.15.181:38918) 2023-10-16T09:56:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 -- 172.21.15.181:0/1329079641 learned_addr learned my addr 172.21.15.181:0/1329079641 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b252f0700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b200ce6a0 0x7f5b20160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b262f2700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b2015da70 0x7f5b201f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 -- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b200ce6a0 msgr2=0x7f5b20160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b200ce6a0 0x7f5b20160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 -- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b2015da70 msgr2=0x7f5b201f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b2015da70 0x7f5b201f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 -- 172.21.15.181:0/1329079641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0094a0 con 0x7f5b20072760 2023-10-16T09:56:20.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b25af1700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b20072760 0x7f5b201f3c10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000e9b0 tx=0x7f5b1000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b167fc700 1 -- 172.21.15.181:0/1329079641 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10009710 con 0x7f5b20072760 2023-10-16T09:56:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b167fc700 1 -- 172.21.15.181:0/1329079641 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000c850 con 0x7f5b20072760 2023-10-16T09:56:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b167fc700 1 -- 172.21.15.181:0/1329079641 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000cbc0 con 0x7f5b20072760 2023-10-16T09:56:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b201f6f50 con 0x7f5b20072760 2023-10-16T09:56:20.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.500+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201f7510 con 0x7f5b20072760 2023-10-16T09:56:20.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.501+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2006f6c0 con 0x7f5b20072760 2023-10-16T09:56:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.502+0000 7f5b167fc700 1 -- 172.21.15.181:0/1329079641 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f5b1000cd20 con 0x7f5b20072760 2023-10-16T09:56:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.502+0000 7f5b167fc700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b0406b100 0x7f5b0406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.502+0000 7f5b167fc700 1 -- 172.21.15.181:0/1329079641 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5b10089ad0 con 0x7f5b20072760 2023-10-16T09:56:20.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.502+0000 7f5b252f0700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b0406b100 0x7f5b0406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.503+0000 7f5b252f0700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b0406b100 0x7f5b0406d5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c000c00 tx=0x7f5b1c008030 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.507+0000 7f5b167fc700 1 -- 172.21.15.181:0/1329079641 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5b10054890 con 0x7f5b20072760 2023-10-16T09:56:20.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.729+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b2015de50 con 0x7f5b0406b100 2023-10-16T09:56:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.796+0000 7f5b167fc700 1 -- 172.21.15.181:0/1329079641 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5b1c011630 con 0x7f5b0406b100 2023-10-16T09:56:20.798 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-16T09:56:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b0406b100 msgr2=0x7f5b0406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b0406b100 0x7f5b0406d5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c000c00 tx=0x7f5b1c008030 comp rx=0 tx=0).stop 2023-10-16T09:56:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b20072760 msgr2=0x7f5b201f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b20072760 0x7f5b201f3c10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000e9b0 tx=0x7f5b1000ed70 comp rx=0 tx=0).stop 2023-10-16T09:56:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 shutdown_connections 2023-10-16T09:56:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5b0406b100 0x7f5b0406d5c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b2015da70 0x7f5b201f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b20072760 0x7f5b201f3c10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 --2- 172.21.15.181:0/1329079641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b200ce6a0 0x7f5b20160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 >> 172.21.15.181:0/1329079641 conn(0x7f5b200cd010 msgr2=0x7f5b200b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:20.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 shutdown_connections 2023-10-16T09:56:20.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.798+0000 7f5b27d55700 1 -- 172.21.15.181:0/1329079641 wait complete. 2023-10-16T09:56:20.811 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-16T09:56:20.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1m 2023-10-16T09:56:20.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.934+0000 7fbc59acb700 1 Processor -- start 2023-10-16T09:56:20.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.934+0000 7fbc59acb700 1 -- start start 2023-10-16T09:56:20.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc59acb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc540b2010 0x7fbc540cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc59acb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc540ce680 0x7fbc540ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc59acb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc540ce1f0 con 0x7fbc54071be0 2023-10-16T09:56:20.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc59acb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc54072090 con 0x7fbc540b2010 2023-10-16T09:56:20.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc59acb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540ccda0 con 0x7fbc540ce680 2023-10-16T09:56:20.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc537fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc540b2010 0x7fbc540cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc537fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc540b2010 0x7fbc540cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38926/0 (socket says 172.21.15.181:38926) 2023-10-16T09:56:20.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.935+0000 7fbc537fe700 1 -- 172.21.15.181:0/1533880687 learned_addr learned my addr 172.21.15.181:0/1533880687 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:20.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc53fff700 1 --2- 172.21.15.181:0/1533880687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc540ce680 0x7fbc540ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc537fe700 1 -- 172.21.15.181:0/1533880687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc540ce680 msgr2=0x7fbc540ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:20.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc537fe700 1 --2- 172.21.15.181:0/1533880687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc540ce680 0x7fbc540ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc537fe700 1 -- 172.21.15.181:0/1533880687 >> v1:172.21.15.190:6789/0 conn(0x7fbc54071be0 legacy=0x7fbc540b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc537fe700 1 -- 172.21.15.181:0/1533880687 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540727e0 con 0x7fbc540b2010 2023-10-16T09:56:20.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc537fe700 1 --2- 172.21.15.181:0/1533880687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540b2010 0x7fbc540cc6e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00bb90 tx=0x7fbc3c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d889b83713bacd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc527fc700 1 -- 172.21.15.181:0/1533880687 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00dd50 con 0x7fbc540b2010 2023-10-16T09:56:20.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc527fc700 1 -- 172.21.15.181:0/1533880687 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c010470 con 0x7fbc540b2010 2023-10-16T09:56:20.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc527fc700 1 -- 172.21.15.181:0/1533880687 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c010800 con 0x7fbc540b2010 2023-10-16T09:56:20.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc59acb700 1 -- 172.21.15.181:0/1533880687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540b2010 msgr2=0x7fbc540cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.936+0000 7fbc59acb700 1 --2- 172.21.15.181:0/1533880687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540b2010 0x7fbc540cc6e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00bb90 tx=0x7fbc3c00bea0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.937+0000 7fbc59acb700 1 -- 172.21.15.181:0/1533880687 shutdown_connections 2023-10-16T09:56:20.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.937+0000 7fbc59acb700 1 --2- 172.21.15.181:0/1533880687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc540ce680 0x7fbc540ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.937+0000 7fbc59acb700 1 --2- 172.21.15.181:0/1533880687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540b2010 0x7fbc540cc6e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.937+0000 7fbc59acb700 1 -- 172.21.15.181:0/1533880687 >> 172.21.15.181:0/1533880687 conn(0x7fbc540cd010 msgr2=0x7fbc540cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:20.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.937+0000 7fbc59acb700 1 -- 172.21.15.181:0/1533880687 shutdown_connections 2023-10-16T09:56:20.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.937+0000 7fbc59acb700 1 -- 172.21.15.181:0/1533880687 wait complete. 2023-10-16T09:56:20.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.938+0000 7fbc59acb700 1 Processor -- start 2023-10-16T09:56:20.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.938+0000 7fbc59acb700 1 -- start start 2023-10-16T09:56:20.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc54071be0 0x7fbc540b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc540b2010 0x7fbc541529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540ce680 0x7fbc540b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540b5880 con 0x7fbc540ce680 2023-10-16T09:56:20.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540c8710 con 0x7fbc540b2010 2023-10-16T09:56:20.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5415b1a0 con 0x7fbc54071be0 2023-10-16T09:56:20.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540ce680 0x7fbc540b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540ce680 0x7fbc540b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38936/0 (socket says 172.21.15.181:38936) 2023-10-16T09:56:20.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 -- 172.21.15.181:0/585040292 learned_addr learned my addr 172.21.15.181:0/585040292 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:20.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc537fe700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc54071be0 0x7fbc540b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc52ffd700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc540b2010 0x7fbc541529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 -- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc54071be0 msgr2=0x7fbc540b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc54071be0 0x7fbc540b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 -- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc540b2010 msgr2=0x7fbc541529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc540b2010 0x7fbc541529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 -- 172.21.15.181:0/585040292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c00b7f0 con 0x7fbc540ce680 2023-10-16T09:56:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc537fe700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc54071be0 0x7fbc540b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc52ffd700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc540b2010 0x7fbc541529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc53fff700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540ce680 0x7fbc540b4680 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007e60 tx=0x7fbc4c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc4bfff700 1 -- 172.21.15.181:0/585040292 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00ede0 con 0x7fbc540ce680 2023-10-16T09:56:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc4bfff700 1 -- 172.21.15.181:0/585040292 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c014420 con 0x7fbc540ce680 2023-10-16T09:56:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5415b480 con 0x7fbc540ce680 2023-10-16T09:56:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc4bfff700 1 -- 172.21.15.181:0/585040292 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0147d0 con 0x7fbc540ce680 2023-10-16T09:56:20.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.939+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc54158d30 con 0x7fbc540ce680 2023-10-16T09:56:20.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.940+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc540b1df0 con 0x7fbc540ce680 2023-10-16T09:56:20.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.941+0000 7fbc4bfff700 1 -- 172.21.15.181:0/585040292 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fbc4c014930 con 0x7fbc540ce680 2023-10-16T09:56:20.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.941+0000 7fbc4bfff700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbc3406b100 0x7fbc3406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:20.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.941+0000 7fbc4bfff700 1 -- 172.21.15.181:0/585040292 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fbc4c08a1c0 con 0x7fbc540ce680 2023-10-16T09:56:20.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.941+0000 7fbc537fe700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbc3406b100 0x7fbc3406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:20.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.941+0000 7fbc537fe700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbc3406b100 0x7fbc3406d5c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00bee0 tx=0x7fbc3c013000 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:20.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:20.945+0000 7fbc4bfff700 1 -- 172.21.15.181:0/585040292 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc4c054ef0 con 0x7fbc540ce680 2023-10-16T09:56:21.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.169+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc54072da0 con 0x7fbc3406b100 2023-10-16T09:56:21.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.245+0000 7fbc4bfff700 1 -- 172.21.15.181:0/585040292 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbc3c00dc70 con 0x7fbc3406b100 2023-10-16T09:56:21.247 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-16T09:56:21.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbc3406b100 msgr2=0x7fbc3406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbc3406b100 0x7fbc3406d5c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00bee0 tx=0x7fbc3c013000 comp rx=0 tx=0).stop 2023-10-16T09:56:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540ce680 msgr2=0x7fbc540b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540ce680 0x7fbc540b4680 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007e60 tx=0x7fbc4c00e3c0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 shutdown_connections 2023-10-16T09:56:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc54071be0 0x7fbc540b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbc3406b100 0x7fbc3406d5c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc540ce680 0x7fbc540b4680 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 --2- 172.21.15.181:0/585040292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc540b2010 0x7fbc541529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 >> 172.21.15.181:0/585040292 conn(0x7fbc540cd010 msgr2=0x7fbc540d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:21.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 shutdown_connections 2023-10-16T09:56:21.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.247+0000 7fbc59acb700 1 -- 172.21.15.181:0/585040292 wait complete. 2023-10-16T09:56:21.261 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-16T09:56:21.261 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1m 2023-10-16T09:56:21.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.382+0000 7f98ee8ba700 1 Processor -- start 2023-10-16T09:56:21.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.382+0000 7f98ee8ba700 1 -- start start 2023-10-16T09:56:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.383+0000 7f98ee8ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 0x7f98e80ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.383+0000 7f98ee8ba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e806e4b0 con 0x7f98e806bef0 2023-10-16T09:56:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.383+0000 7f98ee8ba700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e806ed50 con 0x7f98e815a1f0 2023-10-16T09:56:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.383+0000 7f98ee8ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80ceea0 con 0x7f98e8159e10 2023-10-16T09:56:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.383+0000 7f98e77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f98e806bef0 0x7f98e80caba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50020/0 (socket says 172.21.15.181:50020) 2023-10-16T09:56:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.383+0000 7f98e77fe700 1 -- 172.21.15.181:0/4113307934 learned_addr learned my addr 172.21.15.181:0/4113307934 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e7fff700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 0x7f98e80ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115836012 0 0) 0x7f98e806e4b0 con 0x7f98e806bef0 2023-10-16T09:56:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d00035b0 con 0x7f98e806bef0 2023-10-16T09:56:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1821687634 0 0) 0x7f98d00035b0 con 0x7f98e806bef0 2023-10-16T09:56:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 msgr2=0x7f98e80ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 0x7f98e80ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 >> v1:172.21.15.190:6789/0 conn(0x7f98e815a1f0 legacy=0x7f98e815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e806e700 con 0x7f98e806bef0 2023-10-16T09:56:21.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f98d8001670 con 0x7f98e806bef0 2023-10-16T09:56:21.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 >> v1:172.21.15.181:6789/0 conn(0x7f98e806bef0 legacy=0x7f98e80caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 0x7f98d0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0003fb0 0x7f98d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d0004d90 0x7f98d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80ceea0 con 0x7f98d0004d90 2023-10-16T09:56:21.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d000bb80 con 0x7f98d0003fb0 2023-10-16T09:56:21.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d000bd40 con 0x7f98e8159e10 2023-10-16T09:56:21.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98ece57700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d0004d90 0x7f98d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e7fff700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 0x7f98d0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98e77fe700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0003fb0 0x7f98d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98ece57700 1 -- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 msgr2=0x7f98d0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98ece57700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8159e10 0x7f98d0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98ece57700 1 -- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0003fb0 msgr2=0x7f98d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:21.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98ece57700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98d0003fb0 0x7f98d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98ece57700 1 -- 172.21.15.181:0/4113307934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00035b0 con 0x7f98d0004d90 2023-10-16T09:56:21.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.384+0000 7f98ece57700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d0004d90 0x7f98d0005210 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c160 tx=0x7f98d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50619fb030f091ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:21.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0001db0 con 0x7f98d0004d90 2023-10-16T09:56:21.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e000cca0 con 0x7f98d0004d90 2023-10-16T09:56:21.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98e6ffd700 1 -- 172.21.15.181:0/4113307934 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0003c40 con 0x7f98d0004d90 2023-10-16T09:56:21.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98ee8ba700 1 -- 172.21.15.181:0/4113307934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d0004d90 msgr2=0x7f98d0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98ee8ba700 1 --2- 172.21.15.181:0/4113307934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d0004d90 0x7f98d0005210 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c160 tx=0x7f98d0005880 comp rx=0 tx=0).stop 2023-10-16T09:56:21.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98ece57700 1 -- 172.21.15.181:0/4113307934 reap_dead start 2023-10-16T09:56:21.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98ee8ba700 1 -- 172.21.15.181:0/4113307934 shutdown_connections 2023-10-16T09:56:21.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98ee8ba700 1 -- 172.21.15.181:0/4113307934 >> 172.21.15.181:0/4113307934 conn(0x7f98e80cc730 msgr2=0x7f98e80ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:21.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98ee8ba700 1 -- 172.21.15.181:0/4113307934 shutdown_connections 2023-10-16T09:56:21.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.385+0000 7f98ee8ba700 1 -- 172.21.15.181:0/4113307934 wait complete. 2023-10-16T09:56:21.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.386+0000 7f98ee8ba700 1 Processor -- start 2023-10-16T09:56:21.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ee8ba700 1 -- start start 2023-10-16T09:56:21.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ee8ba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e814ccb0 0x7f98e8149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ee8ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814d090 0x7f98e814a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ee8ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80c5110 0x7f98e80c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ee8ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c5e20 con 0x7f98e814d090 2023-10-16T09:56:21.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ee8ba700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c6100 con 0x7f98e814ccb0 2023-10-16T09:56:21.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ee8ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c63e0 con 0x7f98e80c5110 2023-10-16T09:56:21.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814d090 0x7f98e814a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814d090 0x7f98e814a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38954/0 (socket says 172.21.15.181:38954) 2023-10-16T09:56:21.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98e77fe700 1 -- 172.21.15.181:0/3201126110 learned_addr learned my addr 172.21.15.181:0/3201126110 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:21.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98e7fff700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e814ccb0 0x7f98e8149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.387+0000 7f98ece57700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80c5110 0x7f98e80c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e77fe700 1 -- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80c5110 msgr2=0x7f98e80c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e77fe700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80c5110 0x7f98e80c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e77fe700 1 -- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e814ccb0 msgr2=0x7f98e8149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e77fe700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e814ccb0 0x7f98e8149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e77fe700 1 -- 172.21.15.181:0/3201126110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80c66c0 con 0x7f98e814d090 2023-10-16T09:56:21.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e77fe700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814d090 0x7f98e814a190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f98d80072c0 tx=0x7f98d80075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:21.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e4ff9700 1 -- 172.21.15.181:0/3201126110 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800f6c0 con 0x7f98e814d090 2023-10-16T09:56:21.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e4ff9700 1 -- 172.21.15.181:0/3201126110 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d800f820 con 0x7f98e814d090 2023-10-16T09:56:21.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e80c69a0 con 0x7f98e814d090 2023-10-16T09:56:21.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98e4ff9700 1 -- 172.21.15.181:0/3201126110 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800fbd0 con 0x7f98e814d090 2023-10-16T09:56:21.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.388+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e80c6d80 con 0x7f98e814d090 2023-10-16T09:56:21.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.389+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e80cd5d0 con 0x7f98e814d090 2023-10-16T09:56:21.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.389+0000 7f98e4ff9700 1 -- 172.21.15.181:0/3201126110 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f98d800fd30 con 0x7f98e814d090 2023-10-16T09:56:21.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.390+0000 7f98e4ff9700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f98d406b100 0x7f98d406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.390+0000 7f98e4ff9700 1 -- 172.21.15.181:0/3201126110 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f98d8089ac0 con 0x7f98e814d090 2023-10-16T09:56:21.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.390+0000 7f98e7fff700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f98d406b100 0x7f98d406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.390+0000 7f98e7fff700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f98d406b100 0x7f98d406d5c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009c60 tx=0x7f98dc009210 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:21.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.394+0000 7f98e4ff9700 1 -- 172.21.15.181:0/3201126110 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f98d80548a0 con 0x7f98e814d090 2023-10-16T09:56:21.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.614+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f98e80cb2b0 con 0x7f98d406b100 2023-10-16T09:56:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.688+0000 7f98e4ff9700 1 -- 172.21.15.181:0/3201126110 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f98dc019070 con 0x7f98d406b100 2023-10-16T09:56:21.690 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-16T09:56:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f98d406b100 msgr2=0x7f98d406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f98d406b100 0x7f98d406d5c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009c60 tx=0x7f98dc009210 comp rx=0 tx=0).stop 2023-10-16T09:56:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814d090 msgr2=0x7f98e814a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814d090 0x7f98e814a190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f98d80072c0 tx=0x7f98d80075d0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 shutdown_connections 2023-10-16T09:56:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80c5110 0x7f98e80c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f98d406b100 0x7f98d406d5c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e814ccb0 0x7f98e8149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 --2- 172.21.15.181:0/3201126110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814d090 0x7f98e814a190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 >> 172.21.15.181:0/3201126110 conn(0x7f98e80cc730 msgr2=0x7f98e815a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:21.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 shutdown_connections 2023-10-16T09:56:21.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.690+0000 7f98ee8ba700 1 -- 172.21.15.181:0/3201126110 wait complete. 2023-10-16T09:56:21.704 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-16T09:56:21.704 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1m 2023-10-16T09:56:21.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.825+0000 7fd26bd3e700 1 Processor -- start 2023-10-16T09:56:21.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.826+0000 7fd26bd3e700 1 -- start start 2023-10-16T09:56:21.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.826+0000 7fd26bd3e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd26415da70 0x7fd2640ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.826+0000 7fd26bd3e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2640ce6a0 0x7fd26406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.826+0000 7fd26bd3e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2640cef00 con 0x7fd264072760 2023-10-16T09:56:21.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.826+0000 7fd26bd3e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2640cec20 con 0x7fd26415da70 2023-10-16T09:56:21.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.826+0000 7fd26bd3e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd264071c80 con 0x7fd2640ce6a0 2023-10-16T09:56:21.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd26415da70 0x7fd2640ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd26415da70 0x7fd2640ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38960/0 (socket says 172.21.15.181:38960) 2023-10-16T09:56:21.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 -- 172.21.15.181:0/3934094520 learned_addr learned my addr 172.21.15.181:0/3934094520 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:21.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd2692d9700 1 --2- 172.21.15.181:0/3934094520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2640ce6a0 0x7fd26406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 -- 172.21.15.181:0/3934094520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2640ce6a0 msgr2=0x7fd26406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:21.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 --2- 172.21.15.181:0/3934094520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2640ce6a0 0x7fd26406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 -- 172.21.15.181:0/3934094520 >> v1:172.21.15.190:6789/0 conn(0x7fd264072760 legacy=0x7fd2641615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:21.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 -- 172.21.15.181:0/3934094520 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd264071f60 con 0x7fd26415da70 2023-10-16T09:56:21.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd269ada700 1 --2- 172.21.15.181:0/3934094520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26415da70 0x7fd2640ce160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd2540098f0 tx=0x7fd254009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588a07cdb9e60cf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:21.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd268ad8700 1 -- 172.21.15.181:0/3934094520 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254004230 con 0x7fd26415da70 2023-10-16T09:56:21.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd268ad8700 1 -- 172.21.15.181:0/3934094520 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd254004390 con 0x7fd26415da70 2023-10-16T09:56:21.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.827+0000 7fd268ad8700 1 -- 172.21.15.181:0/3934094520 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2540036a0 con 0x7fd26415da70 2023-10-16T09:56:21.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 -- 172.21.15.181:0/3934094520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26415da70 msgr2=0x7fd2640ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/3934094520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26415da70 0x7fd2640ce160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd2540098f0 tx=0x7fd254009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 -- 172.21.15.181:0/3934094520 shutdown_connections 2023-10-16T09:56:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/3934094520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd26415da70 0x7fd2640ce160 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/3934094520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2640ce6a0 0x7fd26406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 -- 172.21.15.181:0/3934094520 >> 172.21.15.181:0/3934094520 conn(0x7fd2640cd010 msgr2=0x7fd2640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 -- 172.21.15.181:0/3934094520 shutdown_connections 2023-10-16T09:56:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.828+0000 7fd26bd3e700 1 -- 172.21.15.181:0/3934094520 wait complete. 2023-10-16T09:56:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.829+0000 7fd26bd3e700 1 Processor -- start 2023-10-16T09:56:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.829+0000 7fd26bd3e700 1 -- start start 2023-10-16T09:56:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd264072760 0x7fd264151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2640ce6a0 0x7fd2640c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd26415da70 0x7fd2640ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640b3470 con 0x7fd264072760 2023-10-16T09:56:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264151ff0 con 0x7fd2640ce6a0 2023-10-16T09:56:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2640b49d0 con 0x7fd26415da70 2023-10-16T09:56:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd264072760 0x7fd264151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd264072760 0x7fd264151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38964/0 (socket says 172.21.15.181:38964) 2023-10-16T09:56:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 -- 172.21.15.181:0/2749590012 learned_addr learned my addr 172.21.15.181:0/2749590012 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:21.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 -- 172.21.15.181:0/2749590012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd26415da70 msgr2=0x7fd2640ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd26415da70 0x7fd2640ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 -- 172.21.15.181:0/2749590012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2640ce6a0 msgr2=0x7fd2640c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:21.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2640ce6a0 0x7fd2640c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:21.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 -- 172.21.15.181:0/2749590012 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254009550 con 0x7fd264072760 2023-10-16T09:56:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd269ada700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd264072760 0x7fd264151ab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd254009130 tx=0x7fd254004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd25a7fc700 1 -- 172.21.15.181:0/2749590012 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254003e10 con 0x7fd264072760 2023-10-16T09:56:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd25a7fc700 1 -- 172.21.15.181:0/2749590012 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25400b660 con 0x7fd264072760 2023-10-16T09:56:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2640b4c70 con 0x7fd264072760 2023-10-16T09:56:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd25a7fc700 1 -- 172.21.15.181:0/2749590012 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400ba10 con 0x7fd264072760 2023-10-16T09:56:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.830+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2640b5230 con 0x7fd264072760 2023-10-16T09:56:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.831+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26406f6c0 con 0x7fd264072760 2023-10-16T09:56:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.832+0000 7fd25a7fc700 1 -- 172.21.15.181:0/2749590012 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd25400bb70 con 0x7fd264072760 2023-10-16T09:56:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.832+0000 7fd25a7fc700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd24806b130 0x7fd24806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.832+0000 7fd25a7fc700 1 -- 172.21.15.181:0/2749590012 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd25408b310 con 0x7fd264072760 2023-10-16T09:56:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.832+0000 7fd2692d9700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd24806b130 0x7fd24806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.833+0000 7fd2692d9700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd24806b130 0x7fd24806d5f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd2640b6430 tx=0x7fd260008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:21.837+0000 7fd25a7fc700 1 -- 172.21.15.181:0/2749590012 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd2540560d0 con 0x7fd264072760 2023-10-16T09:56:22.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.055+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd26415de50 con 0x7fd24806b130 2023-10-16T09:56:22.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.127+0000 7fd25a7fc700 1 -- 172.21.15.181:0/2749590012 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd260009100 con 0x7fd24806b130 2023-10-16T09:56:22.129 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-16T09:56:22.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd24806b130 msgr2=0x7fd24806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd24806b130 0x7fd24806d5f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd2640b6430 tx=0x7fd260008040 comp rx=0 tx=0).stop 2023-10-16T09:56:22.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd264072760 msgr2=0x7fd264151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd264072760 0x7fd264151ab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd254009130 tx=0x7fd254004750 comp rx=0 tx=0).stop 2023-10-16T09:56:22.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 shutdown_connections 2023-10-16T09:56:22.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd26415da70 0x7fd2640ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd24806b130 0x7fd24806d5f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd264072760 0x7fd264151ab0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 --2- 172.21.15.181:0/2749590012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2640ce6a0 0x7fd2640c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 >> 172.21.15.181:0/2749590012 conn(0x7fd2640cd010 msgr2=0x7fd2640cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:22.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 shutdown_connections 2023-10-16T09:56:22.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.129+0000 7fd26bd3e700 1 -- 172.21.15.181:0/2749590012 wait complete. 2023-10-16T09:56:22.143 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-16T09:56:22.144 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1m 2023-10-16T09:56:22.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.265+0000 7f8c4ef6b700 1 Processor -- start 2023-10-16T09:56:22.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.265+0000 7f8c4ef6b700 1 -- start start 2023-10-16T09:56:22.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4ef6b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 0x7f8c480cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4ef6b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c480ce680 0x7f8c480ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4ef6b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c480ce1f0 con 0x7f8c48071be0 2023-10-16T09:56:22.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4ef6b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c480ccda0 con 0x7f8c480ce680 2023-10-16T09:56:22.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4ef6b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480727e0 con 0x7f8c480b2010 2023-10-16T09:56:22.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c47fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8c48071be0 0x7f8c480b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50052/0 (socket says 172.21.15.181:50052) 2023-10-16T09:56:22.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c47fff700 1 -- 172.21.15.181:0/1316391001 learned_addr learned my addr 172.21.15.181:0/1316391001 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:22.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561490163 0 0) 0x7f8c480ce1f0 con 0x7f8c48071be0 2023-10-16T09:56:22.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c380035b0 con 0x7f8c48071be0 2023-10-16T09:56:22.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4cd07700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 0x7f8c480cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4d508700 1 --2- 172.21.15.181:0/1316391001 >> v2:172.21.15.190:3300/0 conn(0x7f8c480ce680 0x7f8c480ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414350652 0 0) 0x7f8c380035b0 con 0x7f8c48071be0 2023-10-16T09:56:22.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 msgr2=0x7f8c480cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:22.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 0x7f8c480cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 >> v2:172.21.15.190:3300/0 conn(0x7f8c480ce680 msgr2=0x7f8c480ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:22.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 --2- 172.21.15.181:0/1316391001 >> v2:172.21.15.190:3300/0 conn(0x7f8c480ce680 0x7f8c480ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48072ac0 con 0x7f8c48071be0 2023-10-16T09:56:22.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f8c34001670 con 0x7f8c48071be0 2023-10-16T09:56:22.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c4cd07700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 0x7f8c480cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:22.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.266+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 >> v1:172.21.15.181:6789/0 conn(0x7f8c48071be0 legacy=0x7f8c480b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 0x7f8c38003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c38003ff0 0x7f8c38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c38004d90 0x7f8c38005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480ccda0 con 0x7f8c38003ff0 2023-10-16T09:56:22.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480727e0 con 0x7f8c38004d90 2023-10-16T09:56:22.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3800baf0 con 0x7f8c480b2010 2023-10-16T09:56:22.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c4d508700 1 --2- 172.21.15.181:0/1316391001 >> v2:172.21.15.190:3300/0 conn(0x7f8c480ce680 0x7f8c480ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:22.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c47fff700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c38003ff0 0x7f8c38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c4cd07700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 0x7f8c38003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c4d508700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c38004d90 0x7f8c38005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c47fff700 1 -- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 msgr2=0x7f8c38003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:22.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c47fff700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c480b2010 0x7f8c38003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c47fff700 1 -- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c38004d90 msgr2=0x7f8c38005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:22.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c47fff700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c38004d90 0x7f8c38005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c47fff700 1 -- 172.21.15.181:0/1316391001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380035b0 con 0x7f8c38003ff0 2023-10-16T09:56:22.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c47fff700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c38003ff0 0x7f8c38004850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c320 tx=0x7f8c38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8781abe28672ed83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:22.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400ce60 con 0x7f8c38003ff0 2023-10-16T09:56:22.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c34005960 con 0x7f8c38003ff0 2023-10-16T09:56:22.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.267+0000 7f8c477fe700 1 -- 172.21.15.181:0/1316391001 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34005cf0 con 0x7f8c38003ff0 2023-10-16T09:56:22.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.268+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1316391001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c38003ff0 msgr2=0x7f8c38004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.268+0000 7f8c4ef6b700 1 --2- 172.21.15.181:0/1316391001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c38003ff0 0x7f8c38004850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c320 tx=0x7f8c38005880 comp rx=0 tx=0).stop 2023-10-16T09:56:22.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.268+0000 7f8c4d508700 1 -- 172.21.15.181:0/1316391001 reap_dead start 2023-10-16T09:56:22.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.268+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1316391001 shutdown_connections 2023-10-16T09:56:22.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.268+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1316391001 >> 172.21.15.181:0/1316391001 conn(0x7f8c480cd010 msgr2=0x7f8c480cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:22.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.268+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1316391001 shutdown_connections 2023-10-16T09:56:22.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.268+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1316391001 wait complete. 2023-10-16T09:56:22.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.269+0000 7f8c4ef6b700 1 Processor -- start 2023-10-16T09:56:22.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.269+0000 7f8c4ef6b700 1 -- start start 2023-10-16T09:56:22.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4ef6b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c38003ff0 0x7f8c481ef350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4ef6b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c48157280 0x7f8c480c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4ef6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c48157700 0x7f8c4814ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4ef6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481561e0 con 0x7f8c48157700 2023-10-16T09:56:22.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4ef6b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481564c0 con 0x7f8c48157280 2023-10-16T09:56:22.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4ef6b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481567a0 con 0x7f8c38003ff0 2023-10-16T09:56:22.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c48157700 0x7f8c4814ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c48157700 0x7f8c4814ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38988/0 (socket says 172.21.15.181:38988) 2023-10-16T09:56:22.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 -- 172.21.15.181:0/1909529759 learned_addr learned my addr 172.21.15.181:0/1909529759 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:22.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4cd07700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c38003ff0 0x7f8c481ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c47fff700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c48157280 0x7f8c480c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 -- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c38003ff0 msgr2=0x7f8c481ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c38003ff0 0x7f8c481ef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 -- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c48157280 msgr2=0x7f8c480c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c48157280 0x7f8c480c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 -- 172.21.15.181:0/1909529759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48156a80 con 0x7f8c48157700 2023-10-16T09:56:22.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4cd07700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c38003ff0 0x7f8c481ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:22.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.270+0000 7f8c4d508700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c48157700 0x7f8c4814ba90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8c480cea60 tx=0x7f8c40004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:22.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.271+0000 7f8c457fa700 1 -- 172.21.15.181:0/1909529759 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40007b80 con 0x7f8c48157700 2023-10-16T09:56:22.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.271+0000 7f8c457fa700 1 -- 172.21.15.181:0/1909529759 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4000f050 con 0x7f8c48157700 2023-10-16T09:56:22.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.271+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c48156d60 con 0x7f8c48157700 2023-10-16T09:56:22.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.271+0000 7f8c457fa700 1 -- 172.21.15.181:0/1909529759 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40014410 con 0x7f8c48157700 2023-10-16T09:56:22.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.271+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c481ef890 con 0x7f8c48157700 2023-10-16T09:56:22.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.272+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48156f50 con 0x7f8c48157700 2023-10-16T09:56:22.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.272+0000 7f8c457fa700 1 -- 172.21.15.181:0/1909529759 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f8c40014570 con 0x7f8c48157700 2023-10-16T09:56:22.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.272+0000 7f8c457fa700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f8c2c06b130 0x7f8c2c06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.273+0000 7f8c457fa700 1 -- 172.21.15.181:0/1909529759 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8c40089fe0 con 0x7f8c48157700 2023-10-16T09:56:22.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.273+0000 7f8c4cd07700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f8c2c06b130 0x7f8c2c06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.273+0000 7f8c4cd07700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f8c2c06b130 0x7f8c2c06d5f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0045d0 tx=0x7f8c3c013040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:22.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.277+0000 7f8c457fa700 1 -- 172.21.15.181:0/1909529759 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8c40054c00 con 0x7f8c48157700 2023-10-16T09:56:22.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.497+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c48072060 con 0x7f8c2c06b130 2023-10-16T09:56:22.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.569+0000 7f8c457fa700 1 -- 172.21.15.181:0/1909529759 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8c3c004220 con 0x7f8c2c06b130 2023-10-16T09:56:22.571 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-16T09:56:22.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f8c2c06b130 msgr2=0x7f8c2c06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f8c2c06b130 0x7f8c2c06d5f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0045d0 tx=0x7f8c3c013040 comp rx=0 tx=0).stop 2023-10-16T09:56:22.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c48157700 msgr2=0x7f8c4814ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c48157700 0x7f8c4814ba90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8c480cea60 tx=0x7f8c40004970 comp rx=0 tx=0).stop 2023-10-16T09:56:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 shutdown_connections 2023-10-16T09:56:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c38003ff0 0x7f8c481ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f8c2c06b130 0x7f8c2c06d5f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c48157700 0x7f8c4814ba90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 --2- 172.21.15.181:0/1909529759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c48157280 0x7f8c480c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 >> 172.21.15.181:0/1909529759 conn(0x7f8c480cd010 msgr2=0x7f8c480b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 shutdown_connections 2023-10-16T09:56:22.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.571+0000 7f8c4ef6b700 1 -- 172.21.15.181:0/1909529759 wait complete. 2023-10-16T09:56:22.584 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-16T09:56:22.585 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1m 2023-10-16T09:56:22.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.706+0000 7fb9bf01d700 1 Processor -- start 2023-10-16T09:56:22.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.706+0000 7fb9bf01d700 1 -- start start 2023-10-16T09:56:22.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9bf01d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9bf01d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb9b806afb0 0x7fb9b81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9bf01d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b80cef00 con 0x7fb9b80ce6a0 2023-10-16T09:56:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9bf01d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b80cec20 con 0x7fb9b806afb0 2023-10-16T09:56:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9bf01d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b8071c80 con 0x7fb9b815da70 2023-10-16T09:56:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9b7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb9b80ce6a0 0x7fb9b806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50070/0 (socket says 172.21.15.181:50070) 2023-10-16T09:56:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9b7fff700 1 -- 172.21.15.181:0/1970325605 learned_addr learned my addr 172.21.15.181:0/1970325605 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9bcdb9700 1 --2- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9bd5ba700 1 --2- 172.21.15.181:0/1970325605 >> v2:172.21.15.190:3300/0 conn(0x7fb9b806afb0 0x7fb9b81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9b77fe700 1 -- 172.21.15.181:0/1970325605 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180381152 0 0) 0x7fb9b80cef00 con 0x7fb9b80ce6a0 2023-10-16T09:56:22.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.707+0000 7fb9b77fe700 1 -- 172.21.15.181:0/1970325605 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a80035b0 con 0x7fb9b80ce6a0 2023-10-16T09:56:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9bd5ba700 1 -- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 msgr2=0x7fb9b80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9bd5ba700 1 --2- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9bd5ba700 1 -- 172.21.15.181:0/1970325605 >> v1:172.21.15.181:6789/0 conn(0x7fb9b80ce6a0 legacy=0x7fb9b806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9b77fe700 1 -- 172.21.15.181:0/1970325605 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889780991 0 0) 0x7fb9a80035b0 con 0x7fb9b80ce6a0 2023-10-16T09:56:22.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9bd5ba700 1 -- 172.21.15.181:0/1970325605 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b8071f60 con 0x7fb9b806afb0 2023-10-16T09:56:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9bcdb9700 1 --2- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9bd5ba700 1 --2- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b806afb0 0x7fb9b81615f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009d90 tx=0x7fb9b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca3e46b15430621a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9b77fe700 1 -- 172.21.15.181:0/1970325605 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b000e070 con 0x7fb9b806afb0 2023-10-16T09:56:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9b77fe700 1 -- 172.21.15.181:0/1970325605 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0003a20 con 0x7fb9b806afb0 2023-10-16T09:56:22.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.708+0000 7fb9b77fe700 1 -- 172.21.15.181:0/1970325605 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4001670 con 0x7fb9b806afb0 2023-10-16T09:56:22.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 -- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b806afb0 msgr2=0x7fb9b81615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b806afb0 0x7fb9b81615f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009d90 tx=0x7fb9b000b330 comp rx=0 tx=0).stop 2023-10-16T09:56:22.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 -- 172.21.15.181:0/1970325605 shutdown_connections 2023-10-16T09:56:22.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b806afb0 0x7fb9b81615f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/1970325605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 -- 172.21.15.181:0/1970325605 >> 172.21.15.181:0/1970325605 conn(0x7fb9b80cd010 msgr2=0x7fb9b80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:22.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 -- 172.21.15.181:0/1970325605 shutdown_connections 2023-10-16T09:56:22.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.709+0000 7fb9bf01d700 1 -- 172.21.15.181:0/1970325605 wait complete. 2023-10-16T09:56:22.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.710+0000 7fb9bf01d700 1 Processor -- start 2023-10-16T09:56:22.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.710+0000 7fb9bf01d700 1 -- start start 2023-10-16T09:56:22.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bf01d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b806afb0 0x7fb9b81f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bf01d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b80ce6a0 0x7fb9b8149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bf01d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b8149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bf01d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80ae5a0 con 0x7fb9b806afb0 2023-10-16T09:56:22.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bf01d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80ae880 con 0x7fb9b80ce6a0 2023-10-16T09:56:22.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bf01d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80aeb60 con 0x7fb9b815da70 2023-10-16T09:56:22.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b806afb0 0x7fb9b81f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b806afb0 0x7fb9b81f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:38998/0 (socket says 172.21.15.181:38998) 2023-10-16T09:56:22.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 -- 172.21.15.181:0/2183291356 learned_addr learned my addr 172.21.15.181:0/2183291356 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:22.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bd5ba700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b8149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9b7fff700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b80ce6a0 0x7fb9b8149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 -- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 msgr2=0x7fb9b8149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b8149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 -- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b80ce6a0 msgr2=0x7fb9b8149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:22.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b80ce6a0 0x7fb9b8149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:22.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 -- 172.21.15.181:0/2183291356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0094a0 con 0x7fb9b806afb0 2023-10-16T09:56:22.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bcdb9700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b806afb0 0x7fb9b81f2940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00ebe0 tx=0x7fb9ac00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:22.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9b57fa700 1 -- 172.21.15.181:0/2183291356 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00c850 con 0x7fb9b806afb0 2023-10-16T09:56:22.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9b57fa700 1 -- 172.21.15.181:0/2183291356 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac00c9b0 con 0x7fb9b806afb0 2023-10-16T09:56:22.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b0009a70 con 0x7fb9b806afb0 2023-10-16T09:56:22.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.711+0000 7fb9b57fa700 1 -- 172.21.15.181:0/2183291356 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00cd40 con 0x7fb9b806afb0 2023-10-16T09:56:22.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.712+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b80af120 con 0x7fb9b806afb0 2023-10-16T09:56:22.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.712+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b80ae6e0 con 0x7fb9b806afb0 2023-10-16T09:56:22.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.713+0000 7fb9b57fa700 1 -- 172.21.15.181:0/2183291356 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fb9ac013470 con 0x7fb9b806afb0 2023-10-16T09:56:22.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.713+0000 7fb9b57fa700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb99c06b100 0x7fb99c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.713+0000 7fb9b57fa700 1 -- 172.21.15.181:0/2183291356 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb9ac089ae0 con 0x7fb9b806afb0 2023-10-16T09:56:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.713+0000 7fb9b7fff700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb99c06b100 0x7fb99c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.714+0000 7fb9b7fff700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb99c06b100 0x7fb99c06d5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9b81f6590 tx=0x7fb9a4001e40 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:22.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.718+0000 7fb9b57fa700 1 -- 172.21.15.181:0/2183291356 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb9b80ae6e0 con 0x7fb9b806afb0 2023-10-16T09:56:22.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:22.939+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9b815de50 con 0x7fb99c06b100 2023-10-16T09:56:23.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.016+0000 7fb9b57fa700 1 -- 172.21.15.181:0/2183291356 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb9a400f690 con 0x7fb99c06b100 2023-10-16T09:56:23.018 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-16T09:56:23.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb99c06b100 msgr2=0x7fb99c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb99c06b100 0x7fb99c06d5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9b81f6590 tx=0x7fb9a4001e40 comp rx=0 tx=0).stop 2023-10-16T09:56:23.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b806afb0 msgr2=0x7fb9b81f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b806afb0 0x7fb9b81f2940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00ebe0 tx=0x7fb9ac00eef0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 shutdown_connections 2023-10-16T09:56:23.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9b815da70 0x7fb9b8149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fb99c06b100 0x7fb99c06d5c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b806afb0 0x7fb9b81f2940 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 --2- 172.21.15.181:0/2183291356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9b80ce6a0 0x7fb9b8149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 >> 172.21.15.181:0/2183291356 conn(0x7fb9b80cd010 msgr2=0x7fb9b80b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:23.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 shutdown_connections 2023-10-16T09:56:23.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.018+0000 7fb9bf01d700 1 -- 172.21.15.181:0/2183291356 wait complete. 2023-10-16T09:56:23.031 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-16T09:56:23.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1m 2023-10-16T09:56:23.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.153+0000 7fa471c70700 1 Processor -- start 2023-10-16T09:56:23.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa471c70700 1 -- start start 2023-10-16T09:56:23.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa471c70700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa46c0cc6e0 0x7fa46c071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa471c70700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0ce680 0x7fa46c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa471c70700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46c0ce1f0 con 0x7fa46c0b2010 2023-10-16T09:56:23.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa471c70700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46c0ccb90 con 0x7fa46c0cc6e0 2023-10-16T09:56:23.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa471c70700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c072120 con 0x7fa46c0ce680 2023-10-16T09:56:23.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa46b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa46c0b2010 0x7fa46c0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50084/0 (socket says 172.21.15.181:50084) 2023-10-16T09:56:23.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.154+0000 7fa46b7fe700 1 -- 172.21.15.181:0/4133047825 learned_addr learned my addr 172.21.15.181:0/4133047825 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:23.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46bfff700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0ce680 0x7fa46c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46affd700 1 --2- 172.21.15.181:0/4133047825 >> v2:172.21.15.190:3300/0 conn(0x7fa46c0cc6e0 0x7fa46c071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124814606 0 0) 0x7fa46c0ce1f0 con 0x7fa46c0b2010 2023-10-16T09:56:23.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4580035b0 con 0x7fa46c0b2010 2023-10-16T09:56:23.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688305345 0 0) 0x7fa4580035b0 con 0x7fa46c0b2010 2023-10-16T09:56:23.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0ce680 msgr2=0x7fa46c0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:23.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0ce680 0x7fa46c0ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 >> v2:172.21.15.190:3300/0 conn(0x7fa46c0cc6e0 msgr2=0x7fa46c071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:23.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 --2- 172.21.15.181:0/4133047825 >> v2:172.21.15.190:3300/0 conn(0x7fa46c0cc6e0 0x7fa46c071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c0ccda0 con 0x7fa46c0b2010 2023-10-16T09:56:23.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa454001670 con 0x7fa46c0b2010 2023-10-16T09:56:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46affd700 1 --2- 172.21.15.181:0/4133047825 >> v2:172.21.15.190:3300/0 conn(0x7fa46c0cc6e0 0x7fa46c071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 >> v1:172.21.15.181:6789/0 conn(0x7fa46c0b2010 legacy=0x7fa46c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa458003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa458004850 0x7fa458004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa458005210 0x7fa45800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c0ccb90 con 0x7fa458003a00 2023-10-16T09:56:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c072120 con 0x7fa458004850 2023-10-16T09:56:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45800e160 con 0x7fa458005210 2023-10-16T09:56:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46bfff700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0ce680 0x7fa46c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46b7fe700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa458003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46affd700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa458004850 0x7fa458004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46bfff700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa458005210 0x7fa45800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46b7fe700 1 -- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa458005210 msgr2=0x7fa45800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:23.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46b7fe700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa458005210 0x7fa45800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46b7fe700 1 -- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa458004850 msgr2=0x7fa458004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:23.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46b7fe700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa458004850 0x7fa458004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46b7fe700 1 -- 172.21.15.181:0/4133047825 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4580035b0 con 0x7fa458003a00 2023-10-16T09:56:23.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.155+0000 7fa46affd700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa458004850 0x7fa458004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:23.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa46b7fe700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa458003e20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa45400c350 tx=0x7fa458005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bbef7f47e235d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:23.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45400ce50 con 0x7fa458003a00 2023-10-16T09:56:23.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa454005960 con 0x7fa458003a00 2023-10-16T09:56:23.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa46a7fc700 1 -- 172.21.15.181:0/4133047825 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454005d10 con 0x7fa458003a00 2023-10-16T09:56:23.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa471c70700 1 -- 172.21.15.181:0/4133047825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 msgr2=0x7fa458003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa471c70700 1 --2- 172.21.15.181:0/4133047825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa458003e20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa45400c350 tx=0x7fa458005880 comp rx=0 tx=0).stop 2023-10-16T09:56:23.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa46bfff700 1 -- 172.21.15.181:0/4133047825 reap_dead start 2023-10-16T09:56:23.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa471c70700 1 -- 172.21.15.181:0/4133047825 shutdown_connections 2023-10-16T09:56:23.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa471c70700 1 -- 172.21.15.181:0/4133047825 >> 172.21.15.181:0/4133047825 conn(0x7fa46c0cd010 msgr2=0x7fa46c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:23.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa471c70700 1 -- 172.21.15.181:0/4133047825 shutdown_connections 2023-10-16T09:56:23.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.156+0000 7fa471c70700 1 -- 172.21.15.181:0/4133047825 wait complete. 2023-10-16T09:56:23.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.157+0000 7fa471c70700 1 Processor -- start 2023-10-16T09:56:23.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa471c70700 1 -- start start 2023-10-16T09:56:23.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa471c70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa46c0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa471c70700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0c9740 0x7fa46c1ee440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa471c70700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa46c0cf600 0x7fa46c0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa471c70700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c0cffc0 con 0x7fa458003a00 2023-10-16T09:56:23.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa471c70700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c0c8a50 con 0x7fa46c0cf600 2023-10-16T09:56:23.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa471c70700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c0c7ba0 con 0x7fa46c0c9740 2023-10-16T09:56:23.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.158+0000 7fa46b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa46c0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa46c0c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39018/0 (socket says 172.21.15.181:39018) 2023-10-16T09:56:23.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 -- 172.21.15.181:0/932405229 learned_addr learned my addr 172.21.15.181:0/932405229 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:23.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46affd700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0c9740 0x7fa46c1ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46bfff700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa46c0cf600 0x7fa46c0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 -- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0c9740 msgr2=0x7fa46c1ee440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0c9740 0x7fa46c1ee440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 -- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa46c0cf600 msgr2=0x7fa46c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa46c0cf600 0x7fa46c0cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 -- 172.21.15.181:0/932405229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c15d600 con 0x7fa458003a00 2023-10-16T09:56:23.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46bfff700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa46c0cf600 0x7fa46c0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:23.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa46b7fe700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa46c0c7660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa454000c00 tx=0x7fa454001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:23.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa463fff700 1 -- 172.21.15.181:0/932405229 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454004a80 con 0x7fa458003a00 2023-10-16T09:56:23.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa463fff700 1 -- 172.21.15.181:0/932405229 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa454004c00 con 0x7fa458003a00 2023-10-16T09:56:23.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa463fff700 1 -- 172.21.15.181:0/932405229 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454004db0 con 0x7fa458003a00 2023-10-16T09:56:23.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c0c7e40 con 0x7fa458003a00 2023-10-16T09:56:23.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.159+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c0c8400 con 0x7fa458003a00 2023-10-16T09:56:23.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.160+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46c0c8bc0 con 0x7fa458003a00 2023-10-16T09:56:23.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.160+0000 7fa463fff700 1 -- 172.21.15.181:0/932405229 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fa4540070d0 con 0x7fa458003a00 2023-10-16T09:56:23.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.161+0000 7fa463fff700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa44c06b100 0x7fa44c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.161+0000 7fa463fff700 1 -- 172.21.15.181:0/932405229 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa45408aeb0 con 0x7fa458003a00 2023-10-16T09:56:23.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.161+0000 7fa46affd700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa44c06b100 0x7fa44c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.161+0000 7fa46affd700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa44c06b100 0x7fa44c06d5c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0cdf90 tx=0x7fa45c008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:23.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.165+0000 7fa463fff700 1 -- 172.21.15.181:0/932405229 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa454055e50 con 0x7fa458003a00 2023-10-16T09:56:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.385+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa46c0cffc0 con 0x7fa44c06b100 2023-10-16T09:56:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.456+0000 7fa463fff700 1 -- 172.21.15.181:0/932405229 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa46c0cffc0 con 0x7fa44c06b100 2023-10-16T09:56:23.458 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-16T09:56:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa44c06b100 msgr2=0x7fa44c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa44c06b100 0x7fa44c06d5c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0cdf90 tx=0x7fa45c008040 comp rx=0 tx=0).stop 2023-10-16T09:56:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 msgr2=0x7fa46c0c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa46c0c7660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa454000c00 tx=0x7fa454001a00 comp rx=0 tx=0).stop 2023-10-16T09:56:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 shutdown_connections 2023-10-16T09:56:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa44c06b100 0x7fa44c06d5c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa46c0cf600 0x7fa46c0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa458003a00 0x7fa46c0c7660 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 --2- 172.21.15.181:0/932405229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa46c0c9740 0x7fa46c1ee440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.458+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 >> 172.21.15.181:0/932405229 conn(0x7fa46c0cd010 msgr2=0x7fa46c158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:23.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.459+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 shutdown_connections 2023-10-16T09:56:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.459+0000 7fa471c70700 1 -- 172.21.15.181:0/932405229 wait complete. 2023-10-16T09:56:23.472 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-16T09:56:23.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1m 2023-10-16T09:56:23.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.594+0000 7f5f2f032700 1 Processor -- start 2023-10-16T09:56:23.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.594+0000 7f5f2f032700 1 -- start start 2023-10-16T09:56:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.594+0000 7f5f2f032700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f280b2010 0x7f5f280cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2f032700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f28071be0 0x7f5f280b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2f032700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f2806f3b0 0x7f5f2806f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2f032700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280ce1f0 con 0x7f5f280b2010 2023-10-16T09:56:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2f032700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f28072090 con 0x7f5f28071be0 2023-10-16T09:56:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2f032700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280b2930 con 0x7f5f2806f3b0 2023-10-16T09:56:23.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f280b2010 0x7f5f280cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f280b2010 0x7f5f280cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39026/0 (socket says 172.21.15.181:39026) 2023-10-16T09:56:23.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 -- 172.21.15.181:0/2217507680 learned_addr learned my addr 172.21.15.181:0/2217507680 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:23.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f27fff700 1 --2- 172.21.15.181:0/2217507680 >> v2:172.21.15.190:3300/0 conn(0x7f5f28071be0 0x7f5f280b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2d5cf700 1 --2- 172.21.15.181:0/2217507680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f2806f3b0 0x7f5f2806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 -- 172.21.15.181:0/2217507680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f2806f3b0 msgr2=0x7f5f2806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 --2- 172.21.15.181:0/2217507680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f2806f3b0 0x7f5f2806f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 -- 172.21.15.181:0/2217507680 >> v2:172.21.15.190:3300/0 conn(0x7f5f28071be0 msgr2=0x7f5f280b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 --2- 172.21.15.181:0/2217507680 >> v2:172.21.15.190:3300/0 conn(0x7f5f28071be0 0x7f5f280b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 -- 172.21.15.181:0/2217507680 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280ccda0 con 0x7f5f280b2010 2023-10-16T09:56:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f27fff700 1 --2- 172.21.15.181:0/2217507680 >> v2:172.21.15.190:3300/0 conn(0x7f5f28071be0 0x7f5f280b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f2cdce700 1 --2- 172.21.15.181:0/2217507680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f280b2010 0x7f5f280cc6e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5f2815e1c0 tx=0x7f5f1c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a1295164be7219 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f277fe700 1 -- 172.21.15.181:0/2217507680 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00dd30 con 0x7f5f280b2010 2023-10-16T09:56:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f277fe700 1 -- 172.21.15.181:0/2217507680 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00de90 con 0x7f5f280b2010 2023-10-16T09:56:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.595+0000 7f5f277fe700 1 -- 172.21.15.181:0/2217507680 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c010690 con 0x7f5f280b2010 2023-10-16T09:56:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 -- 172.21.15.181:0/2217507680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f280b2010 msgr2=0x7f5f280cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 --2- 172.21.15.181:0/2217507680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f280b2010 0x7f5f280cc6e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5f2815e1c0 tx=0x7f5f1c00be70 comp rx=0 tx=0).stop 2023-10-16T09:56:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 -- 172.21.15.181:0/2217507680 shutdown_connections 2023-10-16T09:56:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 --2- 172.21.15.181:0/2217507680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f280b2010 0x7f5f280cc6e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 --2- 172.21.15.181:0/2217507680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f2806f3b0 0x7f5f2806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 --2- 172.21.15.181:0/2217507680 >> v2:172.21.15.190:3300/0 conn(0x7f5f28071be0 0x7f5f280b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 -- 172.21.15.181:0/2217507680 >> 172.21.15.181:0/2217507680 conn(0x7f5f280cd010 msgr2=0x7f5f280cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 -- 172.21.15.181:0/2217507680 shutdown_connections 2023-10-16T09:56:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.596+0000 7f5f2f032700 1 -- 172.21.15.181:0/2217507680 wait complete. 2023-10-16T09:56:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.597+0000 7f5f2f032700 1 Processor -- start 2023-10-16T09:56:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.597+0000 7f5f2f032700 1 -- start start 2023-10-16T09:56:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2f032700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2806f3b0 0x7f5f281523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2f032700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f28071be0 0x7f5f280b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2f032700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f280b2010 0x7f5f280cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2f032700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28155310 con 0x7f5f2806f3b0 2023-10-16T09:56:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2f032700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280b31c0 con 0x7f5f280b2010 2023-10-16T09:56:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2f032700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280b34a0 con 0x7f5f28071be0 2023-10-16T09:56:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2806f3b0 0x7f5f281523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2806f3b0 0x7f5f281523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39032/0 (socket says 172.21.15.181:39032) 2023-10-16T09:56:23.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 -- 172.21.15.181:0/3708969239 learned_addr learned my addr 172.21.15.181:0/3708969239 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:23.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f27fff700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f28071be0 0x7f5f280b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2d5cf700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f280b2010 0x7f5f280cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 -- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f28071be0 msgr2=0x7f5f280b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f28071be0 0x7f5f280b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 -- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f280b2010 msgr2=0x7f5f280cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f280b2010 0x7f5f280cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 -- 172.21.15.181:0/3708969239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c00b7f0 con 0x7f5f2806f3b0 2023-10-16T09:56:23.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2d5cf700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f280b2010 0x7f5f280cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:23.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.598+0000 7f5f2cdce700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2806f3b0 0x7f5f281523d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00beb0 tx=0x7f5f1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:23.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.599+0000 7f5f257fa700 1 -- 172.21.15.181:0/3708969239 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c01f070 con 0x7f5f2806f3b0 2023-10-16T09:56:23.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.599+0000 7f5f257fa700 1 -- 172.21.15.181:0/3708969239 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c003b20 con 0x7f5f2806f3b0 2023-10-16T09:56:23.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.599+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f280b3780 con 0x7f5f2806f3b0 2023-10-16T09:56:23.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.599+0000 7f5f257fa700 1 -- 172.21.15.181:0/3708969239 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c010470 con 0x7f5f2806f3b0 2023-10-16T09:56:23.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.599+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f280b3d40 con 0x7f5f2806f3b0 2023-10-16T09:56:23.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.600+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f280b6510 con 0x7f5f2806f3b0 2023-10-16T09:56:23.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.600+0000 7f5f257fa700 1 -- 172.21.15.181:0/3708969239 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f5f1c019400 con 0x7f5f2806f3b0 2023-10-16T09:56:23.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.600+0000 7f5f257fa700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5f0c06b100 0x7f5f0c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:23.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.600+0000 7f5f257fa700 1 -- 172.21.15.181:0/3708969239 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5f1c08cf70 con 0x7f5f2806f3b0 2023-10-16T09:56:23.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.600+0000 7f5f27fff700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5f0c06b100 0x7f5f0c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:23.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.601+0000 7f5f27fff700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5f0c06b100 0x7f5f0c06d5c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f280b2670 tx=0x7f5f14008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:23.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.605+0000 7f5f257fa700 1 -- 172.21.15.181:0/3708969239 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5f1c0bffe0 con 0x7f5f2806f3b0 2023-10-16T09:56:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.824+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f28072ac0 con 0x7f5f0c06b100 2023-10-16T09:56:23.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.899+0000 7f5f257fa700 1 -- 172.21.15.181:0/3708969239 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5f1400c040 con 0x7f5f0c06b100 2023-10-16T09:56:23.902 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-16T09:56:23.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.901+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5f0c06b100 msgr2=0x7f5f0c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.901+0000 7f5f2f032700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5f0c06b100 0x7f5f0c06d5c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f280b2670 tx=0x7f5f14008040 comp rx=0 tx=0).stop 2023-10-16T09:56:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.901+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2806f3b0 msgr2=0x7f5f281523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.901+0000 7f5f2f032700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2806f3b0 0x7f5f281523d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00beb0 tx=0x7f5f1c004750 comp rx=0 tx=0).stop 2023-10-16T09:56:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 shutdown_connections 2023-10-16T09:56:23.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f5f0c06b100 0x7f5f0c06d5c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f280b2010 0x7f5f280cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2806f3b0 0x7f5f281523d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 --2- 172.21.15.181:0/3708969239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f28071be0 0x7f5f280b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:23.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 >> 172.21.15.181:0/3708969239 conn(0x7f5f280cd010 msgr2=0x7f5f280b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:23.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 shutdown_connections 2023-10-16T09:56:23.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:23.902+0000 7f5f2f032700 1 -- 172.21.15.181:0/3708969239 wait complete. 2023-10-16T09:56:23.916 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-16T09:56:23.916 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1m 2023-10-16T09:56:24.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.038+0000 7ff20ec45700 1 Processor -- start 2023-10-16T09:56:24.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.038+0000 7ff20ec45700 1 -- start start 2023-10-16T09:56:24.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.039+0000 7ff20ec45700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2080d01d0 0x7ff2080ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.039+0000 7ff20ec45700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d1500 0x7ff2080d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.039+0000 7ff20ec45700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2080cf580 con 0x7ff20815ea40 2023-10-16T09:56:24.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.039+0000 7ff20ec45700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2080d4290 con 0x7ff2080d01d0 2023-10-16T09:56:24.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.039+0000 7ff20ec45700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080ce970 con 0x7ff2080d1500 2023-10-16T09:56:24.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20c9e1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2080d01d0 0x7ff2080ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20d1e2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d1500 0x7ff2080d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20d1e2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d1500 0x7ff2080d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:36070/0 (socket says 172.21.15.181:36070) 2023-10-16T09:56:24.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20c9e1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2080d01d0 0x7ff2080ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:44090/0 (socket says 172.21.15.181:44090) 2023-10-16T09:56:24.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20d1e2700 1 -- 172.21.15.181:0/1692475564 learned_addr learned my addr 172.21.15.181:0/1692475564 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:24.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff207fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff20815ea40 0x7ff2080d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:34060/0 (socket says 172.21.15.181:34060) 2023-10-16T09:56:24.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff2077fe700 1 -- 172.21.15.181:0/1692475564 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254140712 0 0) 0x7ff2080cf580 con 0x7ff20815ea40 2023-10-16T09:56:24.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff2077fe700 1 -- 172.21.15.181:0/1692475564 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1f80035b0 con 0x7ff20815ea40 2023-10-16T09:56:24.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20c9e1700 1 -- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d1500 msgr2=0x7ff2080d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:24.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20c9e1700 1 --2- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d1500 0x7ff2080d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20c9e1700 1 -- 172.21.15.181:0/1692475564 >> v1:172.21.15.190:6789/0 conn(0x7ff20815ea40 legacy=0x7ff2080d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20c9e1700 1 -- 172.21.15.181:0/1692475564 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080cec50 con 0x7ff2080d01d0 2023-10-16T09:56:24.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20d1e2700 1 --2- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d1500 0x7ff2080d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:24.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff20c9e1700 1 --2- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d01d0 0x7ff2080ce2d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009840 tx=0x7ff1fc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f747ae3ca8e5646b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff2077fe700 1 -- 172.21.15.181:0/1692475564 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc011830 con 0x7ff2080d01d0 2023-10-16T09:56:24.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.040+0000 7ff2077fe700 1 -- 172.21.15.181:0/1692475564 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc011990 con 0x7ff2080d01d0 2023-10-16T09:56:24.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff2077fe700 1 -- 172.21.15.181:0/1692475564 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc011d20 con 0x7ff2080d01d0 2023-10-16T09:56:24.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 -- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d01d0 msgr2=0x7ff2080ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d01d0 0x7ff2080ce2d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009840 tx=0x7ff1fc009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:24.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 -- 172.21.15.181:0/1692475564 shutdown_connections 2023-10-16T09:56:24.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d1500 0x7ff2080d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1692475564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d01d0 0x7ff2080ce2d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 -- 172.21.15.181:0/1692475564 >> 172.21.15.181:0/1692475564 conn(0x7ff20809cb60 msgr2=0x7ff2080d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:24.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 -- 172.21.15.181:0/1692475564 shutdown_connections 2023-10-16T09:56:24.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.041+0000 7ff20ec45700 1 -- 172.21.15.181:0/1692475564 wait complete. 2023-10-16T09:56:24.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.042+0000 7ff20ec45700 1 Processor -- start 2023-10-16T09:56:24.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.042+0000 7ff20ec45700 1 -- start start 2023-10-16T09:56:24.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20ec45700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d01d0 0x7ff208152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20ec45700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d1500 0x7ff2080b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20ec45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff20815ea40 0x7ff2080b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20ec45700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080c7420 con 0x7ff20815ea40 2023-10-16T09:56:24.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20ec45700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080c7700 con 0x7ff2080d1500 2023-10-16T09:56:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20ec45700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080c79e0 con 0x7ff2080d01d0 2023-10-16T09:56:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20d1e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff20815ea40 0x7ff2080b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20d1e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff20815ea40 0x7ff2080b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39050/0 (socket says 172.21.15.181:39050) 2023-10-16T09:56:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20d1e2700 1 -- 172.21.15.181:0/1349423849 learned_addr learned my addr 172.21.15.181:0/1349423849 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff207fff700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d1500 0x7ff2080b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.043+0000 7ff20c9e1700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d01d0 0x7ff208152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20d1e2700 1 -- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d01d0 msgr2=0x7ff208152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20d1e2700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d01d0 0x7ff208152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20d1e2700 1 -- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d1500 msgr2=0x7ff2080b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20d1e2700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d1500 0x7ff2080b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20d1e2700 1 -- 172.21.15.181:0/1349423849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200009b30 con 0x7ff20815ea40 2023-10-16T09:56:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20d1e2700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff20815ea40 0x7ff2080b3660 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff20000efc0 tx=0x7ff20000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff2057fa700 1 -- 172.21.15.181:0/1349423849 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200018070 con 0x7ff20815ea40 2023-10-16T09:56:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff2057fa700 1 -- 172.21.15.181:0/1349423849 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20000f040 con 0x7ff20815ea40 2023-10-16T09:56:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc0094a0 con 0x7ff20815ea40 2023-10-16T09:56:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff2057fa700 1 -- 172.21.15.181:0/1349423849 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200003c20 con 0x7ff20815ea40 2023-10-16T09:56:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.044+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2080c7fa0 con 0x7ff20815ea40 2023-10-16T09:56:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.045+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20804ebb0 con 0x7ff20815ea40 2023-10-16T09:56:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.045+0000 7ff2057fa700 1 -- 172.21.15.181:0/1349423849 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff20000c880 con 0x7ff20815ea40 2023-10-16T09:56:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.046+0000 7ff2057fa700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff1ec06b100 0x7ff1ec06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.046+0000 7ff2057fa700 1 -- 172.21.15.181:0/1349423849 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff200089d30 con 0x7ff20815ea40 2023-10-16T09:56:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.046+0000 7ff20c9e1700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff1ec06b100 0x7ff1ec06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.046+0000 7ff20c9e1700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff1ec06b100 0x7ff1ec06d5c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc000c00 tx=0x7ff1fc01c040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.050+0000 7ff2057fa700 1 -- 172.21.15.181:0/1349423849 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff200054960 con 0x7ff20815ea40 2023-10-16T09:56:24.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.269+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2080c7870 con 0x7ff1ec06b100 2023-10-16T09:56:24.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.342+0000 7ff2057fa700 1 -- 172.21.15.181:0/1349423849 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff2080c7870 con 0x7ff1ec06b100 2023-10-16T09:56:24.344 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-16T09:56:24.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff1ec06b100 msgr2=0x7ff1ec06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff1ec06b100 0x7ff1ec06d5c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc000c00 tx=0x7ff1fc01c040 comp rx=0 tx=0).stop 2023-10-16T09:56:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff20815ea40 msgr2=0x7ff2080b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff20815ea40 0x7ff2080b3660 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff20000efc0 tx=0x7ff20000c3a0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 shutdown_connections 2023-10-16T09:56:24.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2080d01d0 0x7ff208152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff1ec06b100 0x7ff1ec06d5c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff20815ea40 0x7ff2080b3660 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 --2- 172.21.15.181:0/1349423849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2080d1500 0x7ff2080b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 >> 172.21.15.181:0/1349423849 conn(0x7ff20809cb60 msgr2=0x7ff208160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:24.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 shutdown_connections 2023-10-16T09:56:24.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.344+0000 7ff20ec45700 1 -- 172.21.15.181:0/1349423849 wait complete. 2023-10-16T09:56:24.358 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-16T09:56:24.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:24.359 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-16T09:56:24.377 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:24.378 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:24.378 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:24.378 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:24.378 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:24.378 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:24.379 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:24.379 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:24.379 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:24.379 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:24.380 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:24.380 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:24.380 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-10-16T09:56:24.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.538+0000 7fc9d9e1a700 1 Processor -- start 2023-10-16T09:56:24.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.538+0000 7fc9d9e1a700 1 -- start start 2023-10-16T09:56:24.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d9e1a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9d40cc6e0 0x7fc9d4071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d9e1a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9d40ce680 0x7fc9d40ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d9e1a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d40ce1f0 con 0x7fc9d40b2010 2023-10-16T09:56:24.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d9e1a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d40ccb90 con 0x7fc9d40cc6e0 2023-10-16T09:56:24.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d9e1a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d4072120 con 0x7fc9d40ce680 2023-10-16T09:56:24.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9d40cc6e0 0x7fc9d4071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc9d40b2010 0x7fc9d40b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50132/0 (socket says 172.21.15.181:50132) 2023-10-16T09:56:24.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d37fe700 1 -- 172.21.15.181:0/647753511 learned_addr learned my addr 172.21.15.181:0/647753511 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:24.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d2ffd700 1 -- 172.21.15.181:0/647753511 >> v2:172.21.15.190:3300/0 conn(0x7fc9d40ce680 msgr2=0x7fc9d40ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:24.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d3fff700 1 --2- 172.21.15.181:0/647753511 >> v2:172.21.15.190:3300/0 conn(0x7fc9d40ce680 0x7fc9d40ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d2ffd700 1 --2- 172.21.15.181:0/647753511 >> v2:172.21.15.190:3300/0 conn(0x7fc9d40ce680 0x7fc9d40ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d27fc700 1 -- 172.21.15.181:0/647753511 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174401418 0 0) 0x7fc9d40ce1f0 con 0x7fc9d40b2010 2023-10-16T09:56:24.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d2ffd700 1 -- 172.21.15.181:0/647753511 >> v1:172.21.15.181:6789/0 conn(0x7fc9d40b2010 legacy=0x7fc9d40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d2ffd700 1 -- 172.21.15.181:0/647753511 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d40ccda0 con 0x7fc9d40cc6e0 2023-10-16T09:56:24.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.539+0000 7fc9d3fff700 1 --2- 172.21.15.181:0/647753511 >> v2:172.21.15.190:3300/0 conn(0x7fc9d40ce680 0x7fc9d40ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:24.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d2ffd700 1 --2- 172.21.15.181:0/647753511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40cc6e0 0x7fc9d4071be0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4071a30 tx=0x7fc9c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e580c1b73a64549 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d27fc700 1 -- 172.21.15.181:0/647753511 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4011830 con 0x7fc9d40cc6e0 2023-10-16T09:56:24.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d27fc700 1 -- 172.21.15.181:0/647753511 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4011990 con 0x7fc9d40cc6e0 2023-10-16T09:56:24.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d27fc700 1 -- 172.21.15.181:0/647753511 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4011d20 con 0x7fc9d40cc6e0 2023-10-16T09:56:24.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/647753511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40cc6e0 msgr2=0x7fc9d4071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/647753511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40cc6e0 0x7fc9d4071be0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4071a30 tx=0x7fc9c4009b20 comp rx=0 tx=0).stop 2023-10-16T09:56:24.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/647753511 shutdown_connections 2023-10-16T09:56:24.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/647753511 >> v2:172.21.15.190:3300/0 conn(0x7fc9d40ce680 0x7fc9d40ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/647753511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40cc6e0 0x7fc9d4071be0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/647753511 >> 172.21.15.181:0/647753511 conn(0x7fc9d40cd010 msgr2=0x7fc9d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:24.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/647753511 shutdown_connections 2023-10-16T09:56:24.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.540+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/647753511 wait complete. 2023-10-16T09:56:24.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 Processor -- start 2023-10-16T09:56:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 -- start start 2023-10-16T09:56:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d40b2010 0x7fc9d414fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9d40cc6e0 0x7fc9d40c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40ce680 0x7fc9d4151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c8c50 con 0x7fc9d40ce680 2023-10-16T09:56:24.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c8040 con 0x7fc9d40b2010 2023-10-16T09:56:24.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.542+0000 7fc9d9e1a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c8320 con 0x7fc9d40cc6e0 2023-10-16T09:56:24.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40ce680 0x7fc9d4151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40ce680 0x7fc9d4151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39064/0 (socket says 172.21.15.181:39064) 2023-10-16T09:56:24.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 -- 172.21.15.181:0/1870261465 learned_addr learned my addr 172.21.15.181:0/1870261465 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:24.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d37fe700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d40b2010 0x7fc9d414fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 -- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9d40cc6e0 msgr2=0x7fc9d40c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d2ffd700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9d40cc6e0 0x7fc9d40c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9d40cc6e0 0x7fc9d40c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 -- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d40b2010 msgr2=0x7fc9d414fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d40b2010 0x7fc9d414fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 -- 172.21.15.181:0/1870261465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c40094a0 con 0x7fc9d40ce680 2023-10-16T09:56:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d37fe700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d40b2010 0x7fc9d414fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d2ffd700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9d40cc6e0 0x7fc9d40c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d3fff700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40ce680 0x7fc9d4151fd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00cef0 tx=0x7fc9cc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9bbfff700 1 -- 172.21.15.181:0/1870261465 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00ae50 con 0x7fc9d40ce680 2023-10-16T09:56:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9bbfff700 1 -- 172.21.15.181:0/1870261465 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc015440 con 0x7fc9d40ce680 2023-10-16T09:56:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d40c8600 con 0x7fc9d40ce680 2023-10-16T09:56:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9bbfff700 1 -- 172.21.15.181:0/1870261465 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0157f0 con 0x7fc9d40ce680 2023-10-16T09:56:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.543+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4151b90 con 0x7fc9d40ce680 2023-10-16T09:56:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.544+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d40c8dc0 con 0x7fc9d40ce680 2023-10-16T09:56:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.545+0000 7fc9bbfff700 1 -- 172.21.15.181:0/1870261465 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc9cc00d430 con 0x7fc9d40ce680 2023-10-16T09:56:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.545+0000 7fc9bbfff700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fc9c006b100 0x7fc9c006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.545+0000 7fc9bbfff700 1 -- 172.21.15.181:0/1870261465 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc9cc08a0f0 con 0x7fc9d40ce680 2023-10-16T09:56:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.545+0000 7fc9d37fe700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fc9c006b100 0x7fc9c006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.545+0000 7fc9d37fe700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fc9c006b100 0x7fc9c006d5c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8007fd0 tx=0x7fc9c8009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.549+0000 7fc9bbfff700 1 -- 172.21.15.181:0/1870261465 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc9cc054da0 con 0x7fc9d40ce680 2023-10-16T09:56:24.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.764+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9d4072ac0 con 0x7fc9c006b100 2023-10-16T09:56:24.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.772+0000 7fc9bbfff700 1 -- 172.21.15.181:0/1870261465 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fc9c8001370 con 0x7fc9c006b100 2023-10-16T09:56:24.774 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:24.774 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:16", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:24.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fc9c006b100 msgr2=0x7fc9c006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fc9c006b100 0x7fc9c006d5c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8007fd0 tx=0x7fc9c8009040 comp rx=0 tx=0).stop 2023-10-16T09:56:24.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40ce680 msgr2=0x7fc9d4151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40ce680 0x7fc9d4151fd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00cef0 tx=0x7fc9cc00a340 comp rx=0 tx=0).stop 2023-10-16T09:56:24.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 shutdown_connections 2023-10-16T09:56:24.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fc9c006b100 0x7fc9c006d5c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d40ce680 0x7fc9d4151fd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9d40b2010 0x7fc9d414fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 --2- 172.21.15.181:0/1870261465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9d40cc6e0 0x7fc9d40c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 >> 172.21.15.181:0/1870261465 conn(0x7fc9d40cd010 msgr2=0x7fc9d40cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:24.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 shutdown_connections 2023-10-16T09:56:24.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.774+0000 7fc9d9e1a700 1 -- 172.21.15.181:0/1870261465 wait complete. 2023-10-16T09:56:24.787 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:16", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:24.788 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:16', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:24.788 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:24.789 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:24.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:24.790 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-16T09:56:24.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:24.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:24.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:24.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:24.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:24.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:24.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:24.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:24.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:24.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:24.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:24.810 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:24.810 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-10-16T09:56:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.967+0000 7f957da67700 1 Processor -- start 2023-10-16T09:56:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.967+0000 7f957da67700 1 -- start start 2023-10-16T09:56:24.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f957da67700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95780d01d0 0x7f95780ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f957da67700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95780d1500 0x7f95780d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f957da67700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95780cf580 con 0x7f957815ea40 2023-10-16T09:56:24.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f957da67700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780d4290 con 0x7f95780d01d0 2023-10-16T09:56:24.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f957da67700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780ce970 con 0x7f95780d1500 2023-10-16T09:56:24.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f95777fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95780d01d0 0x7f95780ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f9576ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f957815ea40 0x7f95780d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50146/0 (socket says 172.21.15.181:50146) 2023-10-16T09:56:24.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f9576ffd700 1 -- 172.21.15.181:0/2053483248 learned_addr learned my addr 172.21.15.181:0/2053483248 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:24.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f9577fff700 1 --2- 172.21.15.181:0/2053483248 >> v2:172.21.15.190:3300/0 conn(0x7f95780d1500 0x7f95780d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f95777fe700 1 -- 172.21.15.181:0/2053483248 >> v2:172.21.15.190:3300/0 conn(0x7f95780d1500 msgr2=0x7f95780d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:24.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f95777fe700 1 --2- 172.21.15.181:0/2053483248 >> v2:172.21.15.190:3300/0 conn(0x7f95780d1500 0x7f95780d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f95777fe700 1 -- 172.21.15.181:0/2053483248 >> v1:172.21.15.181:6789/0 conn(0x7f957815ea40 legacy=0x7f95780d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.968+0000 7f95777fe700 1 -- 172.21.15.181:0/2053483248 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780cec50 con 0x7f95780d01d0 2023-10-16T09:56:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f95777fe700 1 --2- 172.21.15.181:0/2053483248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95780d01d0 0x7f95780ce2d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f95781625d0 tx=0x7f9560009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80017bf2c2dd3a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f95767fc700 1 -- 172.21.15.181:0/2053483248 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560011830 con 0x7f95780d01d0 2023-10-16T09:56:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f95767fc700 1 -- 172.21.15.181:0/2053483248 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560011990 con 0x7f95780d01d0 2023-10-16T09:56:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f95767fc700 1 -- 172.21.15.181:0/2053483248 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560011d20 con 0x7f95780d01d0 2023-10-16T09:56:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 -- 172.21.15.181:0/2053483248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95780d01d0 msgr2=0x7f95780ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 --2- 172.21.15.181:0/2053483248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95780d01d0 0x7f95780ce2d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f95781625d0 tx=0x7f9560009b20 comp rx=0 tx=0).stop 2023-10-16T09:56:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 -- 172.21.15.181:0/2053483248 shutdown_connections 2023-10-16T09:56:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 --2- 172.21.15.181:0/2053483248 >> v2:172.21.15.190:3300/0 conn(0x7f95780d1500 0x7f95780d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 --2- 172.21.15.181:0/2053483248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95780d01d0 0x7f95780ce2d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 -- 172.21.15.181:0/2053483248 >> 172.21.15.181:0/2053483248 conn(0x7f957809cb60 msgr2=0x7f95780d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:24.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 -- 172.21.15.181:0/2053483248 shutdown_connections 2023-10-16T09:56:24.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.969+0000 7f957da67700 1 -- 172.21.15.181:0/2053483248 wait complete. 2023-10-16T09:56:24.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.971+0000 7f957da67700 1 Processor -- start 2023-10-16T09:56:24.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.971+0000 7f957da67700 1 -- start start 2023-10-16T09:56:24.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f957da67700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95780d01d0 0x7f95781f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f957da67700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95780d1500 0x7f9578158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f957da67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f957815ea40 0x7f957815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f957da67700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780b9500 con 0x7f957815ea40 2023-10-16T09:56:24.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f957da67700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780b7bd0 con 0x7f95780d01d0 2023-10-16T09:56:24.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f957da67700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95781f4520 con 0x7f95780d1500 2023-10-16T09:56:24.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f9577fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f957815ea40 0x7f957815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f95777fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95780d01d0 0x7f95781f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f9577fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f957815ea40 0x7f957815abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39078/0 (socket says 172.21.15.181:39078) 2023-10-16T09:56:24.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f9577fff700 1 -- 172.21.15.181:0/3544179199 learned_addr learned my addr 172.21.15.181:0/3544179199 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:24.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.972+0000 7f9576ffd700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95780d1500 0x7f9578158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f9577fff700 1 -- 172.21.15.181:0/3544179199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95780d1500 msgr2=0x7f9578158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f9577fff700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95780d1500 0x7f9578158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f9577fff700 1 -- 172.21.15.181:0/3544179199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95780d01d0 msgr2=0x7f95781f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:24.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f9577fff700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95780d01d0 0x7f95781f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:24.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f9577fff700 1 -- 172.21.15.181:0/3544179199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600094a0 con 0x7f957815ea40 2023-10-16T09:56:24.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f9577fff700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f957815ea40 0x7f957815abe0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9570007e70 tx=0x7f957000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f956ffff700 1 -- 172.21.15.181:0/3544179199 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000ed30 con 0x7f957815ea40 2023-10-16T09:56:24.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f956ffff700 1 -- 172.21.15.181:0/3544179199 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957000eeb0 con 0x7f957815ea40 2023-10-16T09:56:24.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95780b7610 con 0x7f957815ea40 2023-10-16T09:56:24.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f956ffff700 1 -- 172.21.15.181:0/3544179199 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570014640 con 0x7f957815ea40 2023-10-16T09:56:24.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.973+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95780c8af0 con 0x7f957815ea40 2023-10-16T09:56:24.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.974+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957804ebb0 con 0x7f957815ea40 2023-10-16T09:56:24.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.974+0000 7f956ffff700 1 -- 172.21.15.181:0/3544179199 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f95700147a0 con 0x7f957815ea40 2023-10-16T09:56:24.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.975+0000 7f956ffff700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f955806b100 0x7f955806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:24.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.975+0000 7f956ffff700 1 -- 172.21.15.181:0/3544179199 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f957008a090 con 0x7f957815ea40 2023-10-16T09:56:24.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.975+0000 7f95777fe700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f955806b100 0x7f955806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:24.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.975+0000 7f95777fe700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f955806b100 0x7f955806d5c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f956001c040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:24.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:24.979+0000 7f956ffff700 1 -- 172.21.15.181:0/3544179199 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9570054e50 con 0x7f957815ea40 2023-10-16T09:56:25.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.197+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95780cf210 con 0x7f955806b100 2023-10-16T09:56:25.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.205+0000 7f956ffff700 1 -- 172.21.15.181:0/3544179199 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f9560011630 con 0x7f955806b100 2023-10-16T09:56:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:25.209 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:16", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:25.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f955806b100 msgr2=0x7f955806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f955806b100 0x7f955806d5c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9560000c00 tx=0x7f956001c040 comp rx=0 tx=0).stop 2023-10-16T09:56:25.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f957815ea40 msgr2=0x7f957815abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f957815ea40 0x7f957815abe0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9570007e70 tx=0x7f957000e3c0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 shutdown_connections 2023-10-16T09:56:25.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f955806b100 0x7f955806d5c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f957815ea40 0x7f957815abe0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95780d01d0 0x7f95781f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 --2- 172.21.15.181:0/3544179199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95780d1500 0x7f9578158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 >> 172.21.15.181:0/3544179199 conn(0x7f957809cb60 msgr2=0x7f957815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:25.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 shutdown_connections 2023-10-16T09:56:25.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.207+0000 7f957da67700 1 -- 172.21.15.181:0/3544179199 wait complete. 2023-10-16T09:56:25.220 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:16", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:25.221 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:16', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:25.221 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:25.221 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:25.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:25.222 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-16T09:56:25.240 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:25.240 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:25.240 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:25.241 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:25.241 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:25.241 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:25.241 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:25.241 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:25.242 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:25.242 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:25.242 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:25.243 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:25.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-10-16T09:56:25.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.400+0000 7f2617558700 1 Processor -- start 2023-10-16T09:56:25.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.400+0000 7f2617558700 1 -- start start 2023-10-16T09:56:25.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.401+0000 7f2617558700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f261006afb0 0x7f26101615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.401+0000 7f2617558700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26100cef00 con 0x7f2610072760 2023-10-16T09:56:25.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.401+0000 7f2617558700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26100cec20 con 0x7f261015da70 2023-10-16T09:56:25.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.401+0000 7f2617558700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2610071c80 con 0x7f261006afb0 2023-10-16T09:56:25.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.401+0000 7f2615af5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f261006afb0 0x7f26101615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.401+0000 7f2614af3700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2610072760 0x7f261006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50162/0 (socket says 172.21.15.181:50162) 2023-10-16T09:56:25.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.401+0000 7f2614af3700 1 -- 172.21.15.181:0/3100807529 learned_addr learned my addr 172.21.15.181:0/3100807529 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:25.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2615af5700 1 -- 172.21.15.181:0/3100807529 >> v1:172.21.15.190:6789/0 conn(0x7f261015da70 legacy=0x7f26100ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:25.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2615af5700 1 -- 172.21.15.181:0/3100807529 >> v1:172.21.15.181:6789/0 conn(0x7f2610072760 legacy=0x7f261006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2615af5700 1 -- 172.21.15.181:0/3100807529 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610071f60 con 0x7f261006afb0 2023-10-16T09:56:25.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2607fff700 1 -- 172.21.15.181:0/3100807529 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908695634 0 0) 0x7f26100cef00 con 0x7f2610072760 2023-10-16T09:56:25.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2615af5700 1 --2- 172.21.15.181:0/3100807529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006afb0 0x7f26101615f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2608009d90 tx=0x7f260800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b83d03007a36dcf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:25.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2607fff700 1 -- 172.21.15.181:0/3100807529 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800e070 con 0x7f261006afb0 2023-10-16T09:56:25.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2607fff700 1 -- 172.21.15.181:0/3100807529 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260800bb90 con 0x7f261006afb0 2023-10-16T09:56:25.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2607fff700 1 -- 172.21.15.181:0/3100807529 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608003a20 con 0x7f261006afb0 2023-10-16T09:56:25.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2617558700 1 -- 172.21.15.181:0/3100807529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006afb0 msgr2=0x7f26101615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2617558700 1 --2- 172.21.15.181:0/3100807529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006afb0 0x7f26101615f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2608009d90 tx=0x7f260800b330 comp rx=0 tx=0).stop 2023-10-16T09:56:25.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2617558700 1 -- 172.21.15.181:0/3100807529 shutdown_connections 2023-10-16T09:56:25.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.402+0000 7f2617558700 1 --2- 172.21.15.181:0/3100807529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006afb0 0x7f26101615f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.403+0000 7f2617558700 1 -- 172.21.15.181:0/3100807529 >> 172.21.15.181:0/3100807529 conn(0x7f26100cd010 msgr2=0x7f26100cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.403+0000 7f2617558700 1 -- 172.21.15.181:0/3100807529 shutdown_connections 2023-10-16T09:56:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.403+0000 7f2617558700 1 -- 172.21.15.181:0/3100807529 wait complete. 2023-10-16T09:56:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.404+0000 7f2617558700 1 Processor -- start 2023-10-16T09:56:25.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.404+0000 7f2617558700 1 -- start start 2023-10-16T09:56:25.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2617558700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f261006afb0 0x7f26100b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2617558700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610072760 0x7f26100ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2617558700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261015da70 0x7f26100aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2617558700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26100c4c40 con 0x7f261015da70 2023-10-16T09:56:25.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2617558700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26100b4270 con 0x7f2610072760 2023-10-16T09:56:25.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2617558700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26100b4550 con 0x7f261006afb0 2023-10-16T09:56:25.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261015da70 0x7f26100aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261015da70 0x7f26100aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39092/0 (socket says 172.21.15.181:39092) 2023-10-16T09:56:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 -- 172.21.15.181:0/1606347562 learned_addr learned my addr 172.21.15.181:0/1606347562 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2614af3700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610072760 0x7f26100ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 -- 172.21.15.181:0/1606347562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f261006afb0 msgr2=0x7f26100b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f261006afb0 0x7f26100b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 -- 172.21.15.181:0/1606347562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610072760 msgr2=0x7f26100ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610072760 0x7f26100ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.405+0000 7f2615af5700 1 -- 172.21.15.181:0/1606347562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608009a70 con 0x7f261015da70 2023-10-16T09:56:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.406+0000 7f2615af5700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261015da70 0x7f26100aebe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f261006f140 tx=0x7f2608004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.406+0000 7f2605ffb700 1 -- 172.21.15.181:0/1606347562 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608012630 con 0x7f261015da70 2023-10-16T09:56:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.406+0000 7f2605ffb700 1 -- 172.21.15.181:0/1606347562 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26080127b0 con 0x7f261015da70 2023-10-16T09:56:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.406+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26100b4830 con 0x7f261015da70 2023-10-16T09:56:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.406+0000 7f2605ffb700 1 -- 172.21.15.181:0/1606347562 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608012b60 con 0x7f261015da70 2023-10-16T09:56:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.406+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26100b5b50 con 0x7f261015da70 2023-10-16T09:56:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.407+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26100b4d50 con 0x7f261015da70 2023-10-16T09:56:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.407+0000 7f2605ffb700 1 -- 172.21.15.181:0/1606347562 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f2608012cc0 con 0x7f261015da70 2023-10-16T09:56:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.408+0000 7f2605ffb700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25f406b100 0x7f25f406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.408+0000 7f2605ffb700 1 -- 172.21.15.181:0/1606347562 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f260808bf50 con 0x7f261015da70 2023-10-16T09:56:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.408+0000 7f26152f4700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25f406b100 0x7f25f406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.408+0000 7f26152f4700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25f406b100 0x7f25f406d5c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2600003df0 tx=0x7f2600007a40 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:25.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.412+0000 7f2605ffb700 1 -- 172.21.15.181:0/1606347562 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f26100b4d50 con 0x7f261015da70 2023-10-16T09:56:25.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.625+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f261015de50 con 0x7f25f406b100 2023-10-16T09:56:25.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.633+0000 7f2605ffb700 1 -- 172.21.15.181:0/1606347562 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f2600003a80 con 0x7f25f406b100 2023-10-16T09:56:25.635 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:25.635 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:17", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25f406b100 msgr2=0x7f25f406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25f406b100 0x7f25f406d5c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2600003df0 tx=0x7f2600007a40 comp rx=0 tx=0).stop 2023-10-16T09:56:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261015da70 msgr2=0x7f26100aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261015da70 0x7f26100aebe0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f261006f140 tx=0x7f2608004ad0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 shutdown_connections 2023-10-16T09:56:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f261006afb0 0x7f26100b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25f406b100 0x7f25f406d5c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261015da70 0x7f26100aebe0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 --2- 172.21.15.181:0/1606347562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610072760 0x7f26100ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.635+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 >> 172.21.15.181:0/1606347562 conn(0x7f26100cd010 msgr2=0x7f2610161560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.636+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 shutdown_connections 2023-10-16T09:56:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.636+0000 7f2617558700 1 -- 172.21.15.181:0/1606347562 wait complete. 2023-10-16T09:56:25.649 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:17", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:25.649 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:17', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:25.650 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:25.650 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:25.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:25.651 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-16T09:56:25.668 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:25.669 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:25.669 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:25.669 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:25.669 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:25.669 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:25.670 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:25.670 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:25.670 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:25.670 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:25.670 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:25.671 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:25.671 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-10-16T09:56:25.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.828+0000 7fbfcff6b700 1 Processor -- start 2023-10-16T09:56:25.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.828+0000 7fbfcff6b700 1 -- start start 2023-10-16T09:56:25.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcff6b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cb2b0 0x7fbfc806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcff6b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfc80cb690 0x7fbfc80b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcff6b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc8070590 con 0x7fbfc80b2ba0 2023-10-16T09:56:25.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcff6b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc8156330 con 0x7fbfc80cb690 2023-10-16T09:56:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcff6b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc806fcf0 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcd506700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfc80cb690 0x7fbfc80b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcd506700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfc80cb690 0x7fbfc80b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39096/0 (socket says 172.21.15.181:39096) 2023-10-16T09:56:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcd506700 1 -- 172.21.15.181:0/389309608 learned_addr learned my addr 172.21.15.181:0/389309608 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcd506700 1 -- 172.21.15.181:0/389309608 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cb2b0 msgr2=0x7fbfc806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.829+0000 7fbfcdd07700 1 --2- 172.21.15.181:0/389309608 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cb2b0 0x7fbfc806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcd506700 1 --2- 172.21.15.181:0/389309608 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cb2b0 0x7fbfc806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcd506700 1 -- 172.21.15.181:0/389309608 >> v1:172.21.15.190:6789/0 conn(0x7fbfc80b2ba0 legacy=0x7fbfc815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcd506700 1 -- 172.21.15.181:0/389309608 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc806ffd0 con 0x7fbfc80cb690 2023-10-16T09:56:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcdd07700 1 --2- 172.21.15.181:0/389309608 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cb2b0 0x7fbfc806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcd506700 1 --2- 172.21.15.181:0/389309608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb690 0x7fbfc80b2660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80d1600 tx=0x7fbfb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db252f6360239ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfccd05700 1 -- 172.21.15.181:0/389309608 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8004230 con 0x7fbfc80cb690 2023-10-16T09:56:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfccd05700 1 -- 172.21.15.181:0/389309608 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb8004390 con 0x7fbfc80cb690 2023-10-16T09:56:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfccd05700 1 -- 172.21.15.181:0/389309608 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80036a0 con 0x7fbfc80cb690 2023-10-16T09:56:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcff6b700 1 -- 172.21.15.181:0/389309608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb690 msgr2=0x7fbfc80b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/389309608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb690 0x7fbfc80b2660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80d1600 tx=0x7fbfb8009bd0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcff6b700 1 -- 172.21.15.181:0/389309608 shutdown_connections 2023-10-16T09:56:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/389309608 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cb2b0 0x7fbfc806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/389309608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb690 0x7fbfc80b2660 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.830+0000 7fbfcff6b700 1 -- 172.21.15.181:0/389309608 >> 172.21.15.181:0/389309608 conn(0x7fbfc80cd010 msgr2=0x7fbfc80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:25.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.831+0000 7fbfcff6b700 1 -- 172.21.15.181:0/389309608 shutdown_connections 2023-10-16T09:56:25.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.831+0000 7fbfcff6b700 1 -- 172.21.15.181:0/389309608 wait complete. 2023-10-16T09:56:25.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 Processor -- start 2023-10-16T09:56:25.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 -- start start 2023-10-16T09:56:25.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc80b2ba0 0x7fbfc80c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb2b0 0x7fbfc8148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80cb690 0x7fbfc80c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc815b710 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc815ab00 con 0x7fbfc80b2ba0 2023-10-16T09:56:25.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.832+0000 7fbfcff6b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc815ade0 con 0x7fbfc80cb690 2023-10-16T09:56:25.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb2b0 0x7fbfc8148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb2b0 0x7fbfc8148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39102/0 (socket says 172.21.15.181:39102) 2023-10-16T09:56:25.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 -- 172.21.15.181:0/2594133576 learned_addr learned my addr 172.21.15.181:0/2594133576 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:25.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcdd07700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc80b2ba0 0x7fbfc80c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfce508700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80cb690 0x7fbfc80c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 -- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80cb690 msgr2=0x7fbfc80c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80cb690 0x7fbfc80c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 -- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc80b2ba0 msgr2=0x7fbfc80c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:25.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc80b2ba0 0x7fbfc80c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:25.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 -- 172.21.15.181:0/2594133576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8009550 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcdd07700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc80b2ba0 0x7fbfc80c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:25.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfce508700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80cb690 0x7fbfc80c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:25.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcd506700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb2b0 0x7fbfc8148510 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8004000 tx=0x7fbfb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:25.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfbe7fc700 1 -- 172.21.15.181:0/2594133576 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8011630 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfbe7fc700 1 -- 172.21.15.181:0/2594133576 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb8011790 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfbe7fc700 1 -- 172.21.15.181:0/2594133576 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8011b40 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc815b0c0 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.833+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80c4e90 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.834+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc806f4f0 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.835+0000 7fbfbe7fc700 1 -- 172.21.15.181:0/2594133576 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fbfb8011ca0 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.835+0000 7fbfbe7fc700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfac06b100 0x7fbfac06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:25.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.835+0000 7fbfbe7fc700 1 -- 172.21.15.181:0/2594133576 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fbfb808afc0 con 0x7fbfc80cb2b0 2023-10-16T09:56:25.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.835+0000 7fbfcdd07700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfac06b100 0x7fbfac06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:25.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.835+0000 7fbfcdd07700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfac06b100 0x7fbfac06d5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40089b0 tx=0x7fbfc4008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:25.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:25.839+0000 7fbfbe7fc700 1 -- 172.21.15.181:0/2594133576 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfb80bdfe0 con 0x7fbfc80cb2b0 2023-10-16T09:56:26.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.058+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfc80b2140 con 0x7fbfac06b100 2023-10-16T09:56:26.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.065+0000 7fbfbe7fc700 1 -- 172.21.15.181:0/2594133576 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fbfc400c040 con 0x7fbfac06b100 2023-10-16T09:56:26.067 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:26.067 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:17", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:26.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.067+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfac06b100 msgr2=0x7fbfac06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfac06b100 0x7fbfac06d5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40089b0 tx=0x7fbfc4008040 comp rx=0 tx=0).stop 2023-10-16T09:56:26.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb2b0 msgr2=0x7fbfc8148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb2b0 0x7fbfc8148510 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8004000 tx=0x7fbfb8004750 comp rx=0 tx=0).stop 2023-10-16T09:56:26.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 shutdown_connections 2023-10-16T09:56:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80cb690 0x7fbfc80c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fbfac06b100 0x7fbfac06d5c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc80b2ba0 0x7fbfc80c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 --2- 172.21.15.181:0/2594133576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cb2b0 0x7fbfc8148510 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 >> 172.21.15.181:0/2594133576 conn(0x7fbfc80cd010 msgr2=0x7fbfc80b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:26.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 shutdown_connections 2023-10-16T09:56:26.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.068+0000 7fbfcff6b700 1 -- 172.21.15.181:0/2594133576 wait complete. 2023-10-16T09:56:26.081 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:17", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:26.081 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:17', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:26.082 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:26.082 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:26.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:26.083 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-16T09:56:26.101 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:26.101 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:26.101 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:26.101 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:26.102 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:26.102 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:26.102 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:26.103 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:26.103 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:26.103 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:26.103 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:26.104 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:26.104 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-10-16T09:56:26.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.262+0000 7fa53c325700 1 Processor -- start 2023-10-16T09:56:26.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.262+0000 7fa53c325700 1 -- start start 2023-10-16T09:56:26.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53c325700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa534072760 0x7fa53406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53c325700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53406b210 con 0x7fa53415da70 2023-10-16T09:56:26.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53c325700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5340cef00 con 0x7fa53406ade0 2023-10-16T09:56:26.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53c325700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5340cec20 con 0x7fa534072760 2023-10-16T09:56:26.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53a0c1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa53415da70 0x7fa5340ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50186/0 (socket says 172.21.15.181:50186) 2023-10-16T09:56:26.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53a0c1700 1 -- 172.21.15.181:0/2891631294 learned_addr learned my addr 172.21.15.181:0/2891631294 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:26.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5398c0700 1 --2- 172.21.15.181:0/2891631294 >> v2:172.21.15.190:3300/0 conn(0x7fa534072760 0x7fa53406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485737153 0 0) 0x7fa53406b210 con 0x7fa53415da70 2023-10-16T09:56:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5200035b0 con 0x7fa53415da70 2023-10-16T09:56:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2392494650 0 0) 0x7fa5200035b0 con 0x7fa53415da70 2023-10-16T09:56:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 >> v2:172.21.15.190:3300/0 conn(0x7fa534072760 msgr2=0x7fa53406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:26.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 --2- 172.21.15.181:0/2891631294 >> v2:172.21.15.190:3300/0 conn(0x7fa534072760 0x7fa53406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 >> v1:172.21.15.190:6789/0 conn(0x7fa53406ade0 legacy=0x7fa5341615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534071c80 con 0x7fa53415da70 2023-10-16T09:56:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa524001670 con 0x7fa53415da70 2023-10-16T09:56:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 >> v1:172.21.15.181:6789/0 conn(0x7fa53415da70 legacy=0x7fa5340ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa520003a00 0x7fa520003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa520004850 0x7fa520004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa52000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340cec20 con 0x7fa520005210 2023-10-16T09:56:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52000c160 con 0x7fa520003a00 2023-10-16T09:56:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52000c320 con 0x7fa520004850 2023-10-16T09:56:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53a8c2700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa52000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa53a0c1700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa520003a00 0x7fa520003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.263+0000 7fa5398c0700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa520004850 0x7fa520004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53a8c2700 1 -- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa520004850 msgr2=0x7fa520004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:26.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53a8c2700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa520004850 0x7fa520004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53a8c2700 1 -- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa520003a00 msgr2=0x7fa520003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53a8c2700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa520003a00 0x7fa520003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53a8c2700 1 -- 172.21.15.181:0/2891631294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200035b0 con 0x7fa520005210 2023-10-16T09:56:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53a8c2700 1 -- 172.21.15.181:0/2891631294 reap_dead start 2023-10-16T09:56:26.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53a8c2700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa52000ba10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c290 tx=0x7fa520005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c30ac8b4e5e03dd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:26.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c001e70 con 0x7fa520005210 2023-10-16T09:56:26.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c00ce70 con 0x7fa520005210 2023-10-16T09:56:26.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa5390bf700 1 -- 172.21.15.181:0/2891631294 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c003c20 con 0x7fa520005210 2023-10-16T09:56:26.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53c325700 1 -- 172.21.15.181:0/2891631294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 msgr2=0x7fa52000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.264+0000 7fa53c325700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa52000ba10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c290 tx=0x7fa520005880 comp rx=0 tx=0).stop 2023-10-16T09:56:26.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.265+0000 7fa53c325700 1 -- 172.21.15.181:0/2891631294 shutdown_connections 2023-10-16T09:56:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.265+0000 7fa53c325700 1 --2- 172.21.15.181:0/2891631294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa52000ba10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.265+0000 7fa53c325700 1 -- 172.21.15.181:0/2891631294 >> 172.21.15.181:0/2891631294 conn(0x7fa5340cd010 msgr2=0x7fa5340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.265+0000 7fa53c325700 1 -- 172.21.15.181:0/2891631294 shutdown_connections 2023-10-16T09:56:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.265+0000 7fa53c325700 1 -- 172.21.15.181:0/2891631294 wait complete. 2023-10-16T09:56:26.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 Processor -- start 2023-10-16T09:56:26.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 -- start start 2023-10-16T09:56:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa5340c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5340b1bf0 0x7fa5340ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5340b2070 0x7fa5340af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5341f3280 con 0x7fa520005210 2023-10-16T09:56:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340c4600 con 0x7fa5340b1bf0 2023-10-16T09:56:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.266+0000 7fa53c325700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5341f7830 con 0x7fa5340b2070 2023-10-16T09:56:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa5340c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa5340c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39120/0 (socket says 172.21.15.181:39120) 2023-10-16T09:56:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 -- 172.21.15.181:0/2706800719 learned_addr learned my addr 172.21.15.181:0/2706800719 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa5398c0700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5340b1bf0 0x7fa5340ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a8c2700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5340b2070 0x7fa5340af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 -- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5340b2070 msgr2=0x7fa5340af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5340b2070 0x7fa5340af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 -- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5340b1bf0 msgr2=0x7fa5340ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5340b1bf0 0x7fa5340ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 -- 172.21.15.181:0/2706800719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5341f6cb0 con 0x7fa520005210 2023-10-16T09:56:26.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53a0c1700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa5340c40c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa524007290 tx=0x7fa524007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:26.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa52affd700 1 -- 172.21.15.181:0/2706800719 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52400f640 con 0x7fa520005210 2023-10-16T09:56:26.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa52affd700 1 -- 172.21.15.181:0/2706800719 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52400f7a0 con 0x7fa520005210 2023-10-16T09:56:26.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa52affd700 1 -- 172.21.15.181:0/2706800719 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52400fb50 con 0x7fa520005210 2023-10-16T09:56:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5341f6f90 con 0x7fa520005210 2023-10-16T09:56:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.267+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5341f7550 con 0x7fa520005210 2023-10-16T09:56:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.268+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5341f33c0 con 0x7fa520005210 2023-10-16T09:56:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.268+0000 7fa52affd700 1 -- 172.21.15.181:0/2706800719 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fa52400fcb0 con 0x7fa520005210 2023-10-16T09:56:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.269+0000 7fa52affd700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa51806b100 0x7fa51806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.269+0000 7fa52affd700 1 -- 172.21.15.181:0/2706800719 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa524089a10 con 0x7fa520005210 2023-10-16T09:56:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.269+0000 7fa5398c0700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa51806b100 0x7fa51806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.269+0000 7fa5398c0700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa51806b100 0x7fa51806d5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa530009b20 tx=0x7fa530008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:26.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.273+0000 7fa52affd700 1 -- 172.21.15.181:0/2706800719 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa5341f33c0 con 0x7fa520005210 2023-10-16T09:56:26.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.492+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa534072240 con 0x7fa51806b100 2023-10-16T09:56:26.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.500+0000 7fa52affd700 1 -- 172.21.15.181:0/2706800719 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fa530009790 con 0x7fa51806b100 2023-10-16T09:56:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:26.502 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:18", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:26.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa51806b100 msgr2=0x7fa51806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa51806b100 0x7fa51806d5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa530009b20 tx=0x7fa530008040 comp rx=0 tx=0).stop 2023-10-16T09:56:26.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 msgr2=0x7fa5340c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa5340c40c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa524007290 tx=0x7fa524007650 comp rx=0 tx=0).stop 2023-10-16T09:56:26.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 shutdown_connections 2023-10-16T09:56:26.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5340b2070 0x7fa5340af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa51806b100 0x7fa51806d5c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa520005210 0x7fa5340c40c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 --2- 172.21.15.181:0/2706800719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5340b1bf0 0x7fa5340ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 >> 172.21.15.181:0/2706800719 conn(0x7fa5340cd010 msgr2=0x7fa534160ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:26.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 shutdown_connections 2023-10-16T09:56:26.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.503+0000 7fa53c325700 1 -- 172.21.15.181:0/2706800719 wait complete. 2023-10-16T09:56:26.516 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:18", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:26.517 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:18', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:26.517 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:26.517 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:26.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:26.518 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-16T09:56:26.536 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:26.536 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:26.536 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:26.536 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:26.537 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:26.537 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:26.537 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:26.537 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:26.537 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:26.538 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:26.538 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:26.538 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:26.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-10-16T09:56:26.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.697+0000 7ff88773c700 1 Processor -- start 2023-10-16T09:56:26.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.697+0000 7ff88773c700 1 -- start start 2023-10-16T09:56:26.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff88773c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff8800b2010 0x7ff8800cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff88773c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8800ce680 0x7ff8800ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff88773c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8800ce1f0 con 0x7ff880071be0 2023-10-16T09:56:26.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff88773c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff880072090 con 0x7ff8800b2010 2023-10-16T09:56:26.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff88773c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800ccda0 con 0x7ff8800ce680 2023-10-16T09:56:26.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff885cd9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8800ce680 0x7ff8800ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff8854d8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff8800b2010 0x7ff8800cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff885cd9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8800ce680 0x7ff8800ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:36158/0 (socket says 172.21.15.181:36158) 2023-10-16T09:56:26.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff885cd9700 1 -- 172.21.15.181:0/3842368245 learned_addr learned my addr 172.21.15.181:0/3842368245 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:26.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff8854d8700 1 -- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8800ce680 msgr2=0x7ff8800ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:26.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff8854d8700 1 --2- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8800ce680 0x7ff8800ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff8854d8700 1 -- 172.21.15.181:0/3842368245 >> v1:172.21.15.190:6789/0 conn(0x7ff880071be0 legacy=0x7ff8800b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff8854d8700 1 -- 172.21.15.181:0/3842368245 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800727e0 con 0x7ff8800b2010 2023-10-16T09:56:26.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff885cd9700 1 --2- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8800ce680 0x7ff8800ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:26.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.698+0000 7ff8854d8700 1 --2- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 0x7ff8800cc6e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009840 tx=0x7ff87c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84b555a92776a03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:26.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff877fff700 1 -- 172.21.15.181:0/3842368245 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c011830 con 0x7ff8800b2010 2023-10-16T09:56:26.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff877fff700 1 -- 172.21.15.181:0/3842368245 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c011990 con 0x7ff8800b2010 2023-10-16T09:56:26.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff877fff700 1 -- 172.21.15.181:0/3842368245 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c011d20 con 0x7ff8800b2010 2023-10-16T09:56:26.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 -- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 msgr2=0x7ff8800cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 --2- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 0x7ff8800cc6e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009840 tx=0x7ff87c009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:26.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 -- 172.21.15.181:0/3842368245 shutdown_connections 2023-10-16T09:56:26.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 --2- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8800ce680 0x7ff8800ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 --2- 172.21.15.181:0/3842368245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 0x7ff8800cc6e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 -- 172.21.15.181:0/3842368245 >> 172.21.15.181:0/3842368245 conn(0x7ff8800cd010 msgr2=0x7ff8800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:26.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 -- 172.21.15.181:0/3842368245 shutdown_connections 2023-10-16T09:56:26.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.699+0000 7ff88773c700 1 -- 172.21.15.181:0/3842368245 wait complete. 2023-10-16T09:56:26.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.700+0000 7ff88773c700 1 Processor -- start 2023-10-16T09:56:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.700+0000 7ff88773c700 1 -- start start 2023-10-16T09:56:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff88773c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff880071be0 0x7ff88015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff88773c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 0x7ff880158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff88773c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8800ce680 0x7ff880158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff88773c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800c8a10 con 0x7ff8800ce680 2023-10-16T09:56:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff88773c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff880158ac0 con 0x7ff8800b2010 2023-10-16T09:56:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff88773c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800c8390 con 0x7ff880071be0 2023-10-16T09:56:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff885cd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8800ce680 0x7ff880158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff885cd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8800ce680 0x7ff880158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39138/0 (socket says 172.21.15.181:39138) 2023-10-16T09:56:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff885cd9700 1 -- 172.21.15.181:0/1322180225 learned_addr learned my addr 172.21.15.181:0/1322180225 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff8854d8700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff880071be0 0x7ff88015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.701+0000 7ff884cd7700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 0x7ff880158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff885cd9700 1 -- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff880071be0 msgr2=0x7ff88015b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff885cd9700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff880071be0 0x7ff88015b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff885cd9700 1 -- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 msgr2=0x7ff880158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff885cd9700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 0x7ff880158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff885cd9700 1 -- 172.21.15.181:0/1322180225 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878009b30 con 0x7ff8800ce680 2023-10-16T09:56:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff885cd9700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8800ce680 0x7ff880158580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff878009f10 tx=0x7ff87800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff875ffb700 1 -- 172.21.15.181:0/1322180225 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878018070 con 0x7ff8800ce680 2023-10-16T09:56:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff875ffb700 1 -- 172.21.15.181:0/1322180225 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87800f040 con 0x7ff8800ce680 2023-10-16T09:56:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff875ffb700 1 -- 172.21.15.181:0/1322180225 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878003c40 con 0x7ff8800ce680 2023-10-16T09:56:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c0094a0 con 0x7ff8800ce680 2023-10-16T09:56:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.702+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8800c8630 con 0x7ff8800ce680 2023-10-16T09:56:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.703+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8800cc2f0 con 0x7ff8800ce680 2023-10-16T09:56:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.703+0000 7ff875ffb700 1 -- 172.21.15.181:0/1322180225 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff87800c940 con 0x7ff8800ce680 2023-10-16T09:56:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.704+0000 7ff875ffb700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff86406b100 0x7ff86406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.704+0000 7ff8854d8700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff86406b100 0x7ff86406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.704+0000 7ff875ffb700 1 -- 172.21.15.181:0/1322180225 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff878089c80 con 0x7ff8800ce680 2023-10-16T09:56:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.704+0000 7ff8854d8700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff86406b100 0x7ff86406d5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff87c000c00 tx=0x7ff87c01c040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.708+0000 7ff875ffb700 1 -- 172.21.15.181:0/1322180225 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff878054930 con 0x7ff8800ce680 2023-10-16T09:56:26.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.923+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff880072da0 con 0x7ff86406b100 2023-10-16T09:56:26.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.931+0000 7ff875ffb700 1 -- 172.21.15.181:0/1322180225 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff87c011b80 con 0x7ff86406b100 2023-10-16T09:56:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:26.933 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:18", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff86406b100 msgr2=0x7ff86406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff86406b100 0x7ff86406d5c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff87c000c00 tx=0x7ff87c01c040 comp rx=0 tx=0).stop 2023-10-16T09:56:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8800ce680 msgr2=0x7ff880158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8800ce680 0x7ff880158580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff878009f10 tx=0x7ff87800c3a0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 shutdown_connections 2023-10-16T09:56:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff880071be0 0x7ff88015b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff86406b100 0x7ff86406d5c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8800ce680 0x7ff880158580 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 --2- 172.21.15.181:0/1322180225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8800b2010 0x7ff880158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 >> 172.21.15.181:0/1322180225 conn(0x7ff8800cd010 msgr2=0x7ff8800cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 shutdown_connections 2023-10-16T09:56:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:26.933+0000 7ff88773c700 1 -- 172.21.15.181:0/1322180225 wait complete. 2023-10-16T09:56:26.946 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:18", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:26.947 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:18', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:26.947 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:26.948 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:26.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:26.948 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-16T09:56:26.966 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:26.967 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:26.967 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:26.967 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:26.967 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:26.968 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:26.968 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:26.968 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:26.968 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:26.968 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:26.969 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:26.969 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:26.970 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-10-16T09:56:27.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.127+0000 7ff47b868700 1 Processor -- start 2023-10-16T09:56:27.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.127+0000 7ff47b868700 1 -- start start 2023-10-16T09:56:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff47b868700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4740ce2d0 0x7ff47415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff47b868700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 0x7ff4740d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff47b868700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47415ef80 con 0x7ff4740d01d0 2023-10-16T09:56:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff47b868700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4740d4290 con 0x7ff4740ce2d0 2023-10-16T09:56:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff47b868700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740ce750 con 0x7ff4740d1500 2023-10-16T09:56:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff479604700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff4740d01d0 0x7ff4740d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50220/0 (socket says 172.21.15.181:50220) 2023-10-16T09:56:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff479604700 1 -- 172.21.15.181:0/583860242 learned_addr learned my addr 172.21.15.181:0/583860242 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff478e03700 1 --2- 172.21.15.181:0/583860242 >> v2:172.21.15.190:3300/0 conn(0x7ff4740ce2d0 0x7ff47415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff479e05700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 0x7ff4740d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248232788 0 0) 0x7ff470001110 con 0x7ff4740d01d0 2023-10-16T09:56:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.128+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4600035b0 con 0x7ff4740d01d0 2023-10-16T09:56:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 819915256 0 0) 0x7ff4600035b0 con 0x7ff4740d01d0 2023-10-16T09:56:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 msgr2=0x7ff4740d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 0x7ff4740d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 >> v2:172.21.15.190:3300/0 conn(0x7ff4740ce2d0 msgr2=0x7ff47415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 --2- 172.21.15.181:0/583860242 >> v2:172.21.15.190:3300/0 conn(0x7ff4740ce2d0 0x7ff47415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740cf580 con 0x7ff4740d01d0 2023-10-16T09:56:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff470001800 con 0x7ff4740d01d0 2023-10-16T09:56:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 >> v1:172.21.15.181:6789/0 conn(0x7ff4740d01d0 legacy=0x7ff4740d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460003a30 0x7ff460003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 0x7ff460004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff460004d90 0x7ff460005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740ce750 con 0x7ff460004d90 2023-10-16T09:56:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46000ba10 con 0x7ff460003a30 2023-10-16T09:56:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46000bb50 con 0x7ff4740d1500 2023-10-16T09:56:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479e05700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff460004d90 0x7ff460005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479604700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460003a30 0x7ff460003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff478e03700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 0x7ff460004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479e05700 1 -- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 msgr2=0x7ff460004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479e05700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 0x7ff460004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479e05700 1 -- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460003a30 msgr2=0x7ff460003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479e05700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460003a30 0x7ff460003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479e05700 1 -- 172.21.15.181:0/583860242 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600035b0 con 0x7ff460004d90 2023-10-16T09:56:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff478e03700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4740d1500 0x7ff460004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff479e05700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff460004d90 0x7ff460005210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009fc0 tx=0x7ff460005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c074a14f521d3d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c00ebd0 con 0x7ff460004d90 2023-10-16T09:56:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00f040 con 0x7ff460004d90 2023-10-16T09:56:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.129+0000 7ff46bfff700 1 -- 172.21.15.181:0/583860242 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c014480 con 0x7ff460004d90 2023-10-16T09:56:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.130+0000 7ff47b868700 1 -- 172.21.15.181:0/583860242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff460004d90 msgr2=0x7ff460005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.130+0000 7ff47b868700 1 --2- 172.21.15.181:0/583860242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff460004d90 0x7ff460005210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009fc0 tx=0x7ff460005880 comp rx=0 tx=0).stop 2023-10-16T09:56:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.130+0000 7ff479e05700 1 -- 172.21.15.181:0/583860242 reap_dead start 2023-10-16T09:56:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.130+0000 7ff47b868700 1 -- 172.21.15.181:0/583860242 shutdown_connections 2023-10-16T09:56:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.130+0000 7ff47b868700 1 -- 172.21.15.181:0/583860242 >> 172.21.15.181:0/583860242 conn(0x7ff47409cb60 msgr2=0x7ff4740d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.130+0000 7ff47b868700 1 -- 172.21.15.181:0/583860242 shutdown_connections 2023-10-16T09:56:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.130+0000 7ff47b868700 1 -- 172.21.15.181:0/583860242 wait complete. 2023-10-16T09:56:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.131+0000 7ff47b868700 1 Processor -- start 2023-10-16T09:56:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.131+0000 7ff47b868700 1 -- start start 2023-10-16T09:56:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff47b868700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460004d90 0x7ff4740b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff47b868700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4741f6240 0x7ff4740cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff47b868700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4740cd680 0x7ff4740b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff47b868700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c7330 con 0x7ff4740cd680 2023-10-16T09:56:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff47b868700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c7610 con 0x7ff460004d90 2023-10-16T09:56:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff47b868700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740c78f0 con 0x7ff4741f6240 2023-10-16T09:56:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4740cd680 0x7ff4740b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4740cd680 0x7ff4740b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39158/0 (socket says 172.21.15.181:39158) 2023-10-16T09:56:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 -- 172.21.15.181:0/3211032361 learned_addr learned my addr 172.21.15.181:0/3211032361 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff478e03700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4741f6240 0x7ff4740cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479604700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460004d90 0x7ff4740b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 -- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4741f6240 msgr2=0x7ff4740cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4741f6240 0x7ff4740cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 -- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460004d90 msgr2=0x7ff4740b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460004d90 0x7ff4740b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479e05700 1 -- 172.21.15.181:0/3211032361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740c7bd0 con 0x7ff4740cd680 2023-10-16T09:56:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.132+0000 7ff479604700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460004d90 0x7ff4740b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.133+0000 7ff479e05700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4740cd680 0x7ff4740b5380 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff46c016000 tx=0x7ff46c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.133+0000 7ff469ffb700 1 -- 172.21.15.181:0/3211032361 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c007000 con 0x7ff4740cd680 2023-10-16T09:56:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.133+0000 7ff469ffb700 1 -- 172.21.15.181:0/3211032361 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c022070 con 0x7ff4740cd680 2023-10-16T09:56:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.133+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4740c7eb0 con 0x7ff4740cd680 2023-10-16T09:56:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.133+0000 7ff469ffb700 1 -- 172.21.15.181:0/3211032361 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c029480 con 0x7ff4740cd680 2023-10-16T09:56:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.133+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4740c6f90 con 0x7ff4740cd680 2023-10-16T09:56:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.134+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47404ebb0 con 0x7ff4740cd680 2023-10-16T09:56:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.134+0000 7ff469ffb700 1 -- 172.21.15.181:0/3211032361 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff46c0295e0 con 0x7ff4740cd680 2023-10-16T09:56:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.135+0000 7ff469ffb700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff45806b130 0x7ff45806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.135+0000 7ff469ffb700 1 -- 172.21.15.181:0/3211032361 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff46c024070 con 0x7ff4740cd680 2023-10-16T09:56:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.135+0000 7ff479604700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff45806b130 0x7ff45806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.135+0000 7ff479604700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff45806b130 0x7ff45806d5f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4700012b0 tx=0x7ff470009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.139+0000 7ff469ffb700 1 -- 172.21.15.181:0/3211032361 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff46c056800 con 0x7ff4740cd680 2023-10-16T09:56:27.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.359+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4740cec50 con 0x7ff45806b130 2023-10-16T09:56:27.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.366+0000 7ff469ffb700 1 -- 172.21.15.181:0/3211032361 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff470000ed0 con 0x7ff45806b130 2023-10-16T09:56:27.368 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:27.368 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:19", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:27.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff45806b130 msgr2=0x7ff45806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff45806b130 0x7ff45806d5f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4700012b0 tx=0x7ff470009040 comp rx=0 tx=0).stop 2023-10-16T09:56:27.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4740cd680 msgr2=0x7ff4740b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4740cd680 0x7ff4740b5380 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff46c016000 tx=0x7ff46c009750 comp rx=0 tx=0).stop 2023-10-16T09:56:27.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 shutdown_connections 2023-10-16T09:56:27.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff45806b130 0x7ff45806d5f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4740cd680 0x7ff4740b5380 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff460004d90 0x7ff4740b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 --2- 172.21.15.181:0/3211032361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4741f6240 0x7ff4740cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 >> 172.21.15.181:0/3211032361 conn(0x7ff47409cb60 msgr2=0x7ff47406d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:27.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 shutdown_connections 2023-10-16T09:56:27.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.369+0000 7ff47b868700 1 -- 172.21.15.181:0/3211032361 wait complete. 2023-10-16T09:56:27.382 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:19", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:27.383 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:19', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:27.383 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:27.383 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:27.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:27.384 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-16T09:56:27.402 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:27.402 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:27.402 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:27.402 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:27.403 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:27.403 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:27.403 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:27.403 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:27.404 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:27.404 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:27.404 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:27.404 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:27.405 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-10-16T09:56:27.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.562+0000 7f3c5af76700 1 Processor -- start 2023-10-16T09:56:27.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.562+0000 7f3c5af76700 1 -- start start 2023-10-16T09:56:27.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c5af76700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5406afb0 0x7f3c541615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c5af76700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540cef00 con 0x7f3c5415da70 2023-10-16T09:56:27.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c5af76700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540cec20 con 0x7f3c54072760 2023-10-16T09:56:27.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c5af76700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54071c80 con 0x7f3c5406afb0 2023-10-16T09:56:27.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c58d12700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3c5415da70 0x7f3c540ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50240/0 (socket says 172.21.15.181:50240) 2023-10-16T09:56:27.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c58d12700 1 -- 172.21.15.181:0/296717127 learned_addr learned my addr 172.21.15.181:0/296717127 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:27.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c59513700 1 --2- 172.21.15.181:0/296717127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5406afb0 0x7f3c541615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.563+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131746670 0 0) 0x7f3c540cef00 con 0x7f3c5415da70 2023-10-16T09:56:27.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c440035b0 con 0x7f3c5415da70 2023-10-16T09:56:27.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977664165 0 0) 0x7f3c540cec20 con 0x7f3c54072760 2023-10-16T09:56:27.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c540cef00 con 0x7f3c54072760 2023-10-16T09:56:27.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3013356903 0 0) 0x7f3c440035b0 con 0x7f3c5415da70 2023-10-16T09:56:27.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c59513700 1 -- 172.21.15.181:0/296717127 >> v1:172.21.15.190:6789/0 conn(0x7f3c54072760 legacy=0x7f3c5406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c59513700 1 -- 172.21.15.181:0/296717127 >> v1:172.21.15.181:6789/0 conn(0x7f3c5415da70 legacy=0x7f3c540ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c59513700 1 -- 172.21.15.181:0/296717127 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54071f60 con 0x7f3c5406afb0 2023-10-16T09:56:27.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c59513700 1 --2- 172.21.15.181:0/296717127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5406afb0 0x7f3c541615f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009dc0 tx=0x7f3c4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3dc3604ed59d83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:27.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00e070 con 0x7f3c5406afb0 2023-10-16T09:56:27.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c003a20 con 0x7f3c5406afb0 2023-10-16T09:56:27.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.564+0000 7f3c537fe700 1 -- 172.21.15.181:0/296717127 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48001670 con 0x7f3c5406afb0 2023-10-16T09:56:27.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.565+0000 7f3c5af76700 1 -- 172.21.15.181:0/296717127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5406afb0 msgr2=0x7f3c541615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.565+0000 7f3c5af76700 1 --2- 172.21.15.181:0/296717127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5406afb0 0x7f3c541615f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009dc0 tx=0x7f3c4c00b330 comp rx=0 tx=0).stop 2023-10-16T09:56:27.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.565+0000 7f3c5af76700 1 -- 172.21.15.181:0/296717127 shutdown_connections 2023-10-16T09:56:27.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.565+0000 7f3c5af76700 1 --2- 172.21.15.181:0/296717127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5406afb0 0x7f3c541615f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.565+0000 7f3c5af76700 1 -- 172.21.15.181:0/296717127 >> 172.21.15.181:0/296717127 conn(0x7f3c540cd010 msgr2=0x7f3c540cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:27.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.565+0000 7f3c5af76700 1 -- 172.21.15.181:0/296717127 shutdown_connections 2023-10-16T09:56:27.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.565+0000 7f3c5af76700 1 -- 172.21.15.181:0/296717127 wait complete. 2023-10-16T09:56:27.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.566+0000 7f3c5af76700 1 Processor -- start 2023-10-16T09:56:27.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.566+0000 7f3c5af76700 1 -- start start 2023-10-16T09:56:27.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c5af76700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c5406afb0 0x7f3c540c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c5af76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c54072760 0x7f3c54152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c5af76700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5415da70 0x7f3c540d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c5af76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540c7ba0 con 0x7f3c54072760 2023-10-16T09:56:27.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c5af76700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540c7e20 con 0x7f3c5406afb0 2023-10-16T09:56:27.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c5af76700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540c8100 con 0x7f3c5415da70 2023-10-16T09:56:27.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c54072760 0x7f3c54152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c54072760 0x7f3c54152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39168/0 (socket says 172.21.15.181:39168) 2023-10-16T09:56:27.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 -- 172.21.15.181:0/1223191623 learned_addr learned my addr 172.21.15.181:0/1223191623 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:27.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c58d12700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c5406afb0 0x7f3c540c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c59513700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5415da70 0x7f3c540d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 -- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5415da70 msgr2=0x7f3c540d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5415da70 0x7f3c540d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 -- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c5406afb0 msgr2=0x7f3c540c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c5406afb0 0x7f3c540c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 -- 172.21.15.181:0/1223191623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009a70 con 0x7f3c54072760 2023-10-16T09:56:27.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c59513700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5415da70 0x7f3c540d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:27.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c53fff700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c54072760 0x7f3c54152120 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c520 tx=0x7f3c4000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:27.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.567+0000 7f3c517fa700 1 -- 172.21.15.181:0/1223191623 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400052d0 con 0x7f3c54072760 2023-10-16T09:56:27.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.568+0000 7f3c517fa700 1 -- 172.21.15.181:0/1223191623 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c40005430 con 0x7f3c54072760 2023-10-16T09:56:27.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.568+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c540c83e0 con 0x7f3c54072760 2023-10-16T09:56:27.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.568+0000 7f3c517fa700 1 -- 172.21.15.181:0/1223191623 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400038c0 con 0x7f3c54072760 2023-10-16T09:56:27.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.568+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c540b3ea0 con 0x7f3c54072760 2023-10-16T09:56:27.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.568+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5406f6c0 con 0x7f3c54072760 2023-10-16T09:56:27.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.569+0000 7f3c517fa700 1 -- 172.21.15.181:0/1223191623 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f3c40003a20 con 0x7f3c54072760 2023-10-16T09:56:27.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.569+0000 7f3c517fa700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3c3806b130 0x7f3c3806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.569+0000 7f3c517fa700 1 -- 172.21.15.181:0/1223191623 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f3c40088f90 con 0x7f3c54072760 2023-10-16T09:56:27.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.569+0000 7f3c58d12700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3c3806b130 0x7f3c3806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.570+0000 7f3c58d12700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3c3806b130 0x7f3c3806d5f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3c48000f60 tx=0x7f3c48001c00 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:27.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.574+0000 7f3c517fa700 1 -- 172.21.15.181:0/1223191623 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3c400bbfe0 con 0x7f3c54072760 2023-10-16T09:56:27.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.789+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c5415de50 con 0x7f3c3806b130 2023-10-16T09:56:27.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.797+0000 7f3c517fa700 1 -- 172.21.15.181:0/1223191623 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f3c48001510 con 0x7f3c3806b130 2023-10-16T09:56:27.799 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:27.799 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:19", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:27.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3c3806b130 msgr2=0x7f3c3806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3c3806b130 0x7f3c3806d5f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3c48000f60 tx=0x7f3c48001c00 comp rx=0 tx=0).stop 2023-10-16T09:56:27.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c54072760 msgr2=0x7f3c54152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:27.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c54072760 0x7f3c54152120 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c520 tx=0x7f3c4000c8e0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 shutdown_connections 2023-10-16T09:56:27.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c5415da70 0x7f3c540d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3c3806b130 0x7f3c3806d5f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c5406afb0 0x7f3c540c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 --2- 172.21.15.181:0/1223191623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c54072760 0x7f3c54152120 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:27.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 >> 172.21.15.181:0/1223191623 conn(0x7f3c540cd010 msgr2=0x7f3c5415c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:27.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 shutdown_connections 2023-10-16T09:56:27.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.800+0000 7f3c5af76700 1 -- 172.21.15.181:0/1223191623 wait complete. 2023-10-16T09:56:27.813 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:19", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:27.814 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:19', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:27.814 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:27.815 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:27.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:27.815 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-16T09:56:27.833 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:27.833 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:27.833 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:27.833 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:27.834 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:27.834 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:27.834 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:27.834 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:27.834 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:27.835 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:27.835 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:27.835 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:27.836 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-10-16T09:56:27.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.994+0000 7ff30f107700 1 Processor -- start 2023-10-16T09:56:27.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.994+0000 7ff30f107700 1 -- start start 2023-10-16T09:56:27.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff30f107700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3080ce6c0 0x7ff3080cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff30f107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080cc6a0 0x7ff3080ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:27.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff30f107700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3080b1f70 con 0x7ff30815f150 2023-10-16T09:56:27.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff30f107700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3080b2ae0 con 0x7ff3080ce6c0 2023-10-16T09:56:27.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff30f107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080b2dc0 con 0x7ff3080cc6a0 2023-10-16T09:56:27.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3080ce6c0 0x7ff3080cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:27.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3080ce6c0 0x7ff3080cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39178/0 (socket says 172.21.15.181:39178) 2023-10-16T09:56:28.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 -- 172.21.15.181:0/2205473326 learned_addr learned my addr 172.21.15.181:0/2205473326 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:28.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff30d6a4700 1 --2- 172.21.15.181:0/2205473326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080cc6a0 0x7ff3080ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 -- 172.21.15.181:0/2205473326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080cc6a0 msgr2=0x7ff3080ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:28.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 --2- 172.21.15.181:0/2205473326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080cc6a0 0x7ff3080ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 -- 172.21.15.181:0/2205473326 >> v1:172.21.15.190:6789/0 conn(0x7ff30815f150 legacy=0x7ff3080cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 -- 172.21.15.181:0/2205473326 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080cf9d0 con 0x7ff3080ce6c0 2023-10-16T09:56:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff307fff700 1 --2- 172.21.15.181:0/2205473326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3080ce6c0 0x7ff3080cc160 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40098f0 tx=0x7ff2f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5712330f7b6510e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff3077fe700 1 -- 172.21.15.181:0/2205473326 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4004230 con 0x7ff3080ce6c0 2023-10-16T09:56:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff3077fe700 1 -- 172.21.15.181:0/2205473326 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4004390 con 0x7ff3080ce6c0 2023-10-16T09:56:28.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.995+0000 7ff3077fe700 1 -- 172.21.15.181:0/2205473326 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40036a0 con 0x7ff3080ce6c0 2023-10-16T09:56:28.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 -- 172.21.15.181:0/2205473326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3080ce6c0 msgr2=0x7ff3080cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 --2- 172.21.15.181:0/2205473326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3080ce6c0 0x7ff3080cc160 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40098f0 tx=0x7ff2f4009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:28.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 -- 172.21.15.181:0/2205473326 shutdown_connections 2023-10-16T09:56:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 --2- 172.21.15.181:0/2205473326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080cc6a0 0x7ff3080ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 --2- 172.21.15.181:0/2205473326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3080ce6c0 0x7ff3080cc160 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 -- 172.21.15.181:0/2205473326 >> 172.21.15.181:0/2205473326 conn(0x7ff30809a930 msgr2=0x7ff3080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 -- 172.21.15.181:0/2205473326 shutdown_connections 2023-10-16T09:56:28.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.996+0000 7ff30f107700 1 -- 172.21.15.181:0/2205473326 wait complete. 2023-10-16T09:56:28.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.997+0000 7ff30f107700 1 Processor -- start 2023-10-16T09:56:28.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.997+0000 7ff30f107700 1 -- start start 2023-10-16T09:56:28.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.998+0000 7ff30f107700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3080cc6a0 0x7ff308151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.998+0000 7ff30f107700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080ce6c0 0x7ff30815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.998+0000 7ff30f107700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30815f150 0x7ff30815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30f107700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080c6780 con 0x7ff30815f150 2023-10-16T09:56:28.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30f107700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080c9f80 con 0x7ff3080cc6a0 2023-10-16T09:56:28.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30f107700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081520e0 con 0x7ff3080ce6c0 2023-10-16T09:56:28.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30815f150 0x7ff30815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30815f150 0x7ff30815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39186/0 (socket says 172.21.15.181:39186) 2023-10-16T09:56:28.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 -- 172.21.15.181:0/1612119668 learned_addr learned my addr 172.21.15.181:0/1612119668 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:28.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30cea3700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3080cc6a0 0x7ff308151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff307fff700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080ce6c0 0x7ff30815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 -- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080ce6c0 msgr2=0x7ff30815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080ce6c0 0x7ff30815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 -- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3080cc6a0 msgr2=0x7ff308151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3080cc6a0 0x7ff308151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 -- 172.21.15.181:0/1612119668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009550 con 0x7ff30815f150 2023-10-16T09:56:28.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff307fff700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080ce6c0 0x7ff30815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:28.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30d6a4700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30815f150 0x7ff30815a730 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff30000cda0 tx=0x7ff30000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff3057fa700 1 -- 172.21.15.181:0/1612119668 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300003a20 con 0x7ff30815f150 2023-10-16T09:56:28.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff3057fa700 1 -- 172.21.15.181:0/1612119668 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff300003b80 con 0x7ff30815f150 2023-10-16T09:56:28.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3080c64a0 con 0x7ff30815f150 2023-10-16T09:56:28.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff3057fa700 1 -- 172.21.15.181:0/1612119668 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300004020 con 0x7ff30815f150 2023-10-16T09:56:28.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:27.999+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3080c9c10 con 0x7ff30815f150 2023-10-16T09:56:28.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.000+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff308067d20 con 0x7ff30815f150 2023-10-16T09:56:28.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.001+0000 7ff3057fa700 1 -- 172.21.15.181:0/1612119668 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ff300004180 con 0x7ff30815f150 2023-10-16T09:56:28.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.001+0000 7ff3057fa700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2ec06b100 0x7ff2ec06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.001+0000 7ff3057fa700 1 -- 172.21.15.181:0/1612119668 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff300012070 con 0x7ff30815f150 2023-10-16T09:56:28.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.001+0000 7ff30cea3700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2ec06b100 0x7ff2ec06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.002+0000 7ff30cea3700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2ec06b100 0x7ff2ec06d5c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009870 tx=0x7ff2fc008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.006+0000 7ff3057fa700 1 -- 172.21.15.181:0/1612119668 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff300055020 con 0x7ff30815f150 2023-10-16T09:56:28.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.221+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3080cff90 con 0x7ff2ec06b100 2023-10-16T09:56:28.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.230+0000 7ff3057fa700 1 -- 172.21.15.181:0/1612119668 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff2fc0090c0 con 0x7ff2ec06b100 2023-10-16T09:56:28.231 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:28.232 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:20", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:28.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2ec06b100 msgr2=0x7ff2ec06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2ec06b100 0x7ff2ec06d5c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009870 tx=0x7ff2fc008040 comp rx=0 tx=0).stop 2023-10-16T09:56:28.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30815f150 msgr2=0x7ff30815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30815f150 0x7ff30815a730 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff30000cda0 tx=0x7ff30000a570 comp rx=0 tx=0).stop 2023-10-16T09:56:28.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 shutdown_connections 2023-10-16T09:56:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7ff2ec06b100 0x7ff2ec06d5c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30815f150 0x7ff30815a730 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3080cc6a0 0x7ff308151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 --2- 172.21.15.181:0/1612119668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3080ce6c0 0x7ff30815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 >> 172.21.15.181:0/1612119668 conn(0x7ff30809a930 msgr2=0x7ff30806ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 shutdown_connections 2023-10-16T09:56:28.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.232+0000 7ff30f107700 1 -- 172.21.15.181:0/1612119668 wait complete. 2023-10-16T09:56:28.245 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:20", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:28.246 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:20', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:28.246 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:28.246 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:28.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:28.247 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-16T09:56:28.265 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:28.265 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:28.265 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:28.265 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:28.266 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:28.266 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:28.266 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:28.266 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:28.266 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:28.267 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:28.267 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:28.267 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-09_46_00_UTC', 'scheduled-2023-10-16-09_47_00_UTC', 'scheduled-2023-10-16-09_48_00_UTC', 'scheduled-2023-10-16-09_49_00_UTC', 'scheduled-2023-10-16-09_50_00_UTC', 'scheduled-2023-10-16-09_51_00_UTC', 'scheduled-2023-10-16-09_52_00_UTC', 'scheduled-2023-10-16-09_53_00_UTC', 'scheduled-2023-10-16-09_54_00_UTC', 'scheduled-2023-10-16-09_55_00_UTC', 'scheduled-2023-10-16-09_56_00_UTC'] 2023-10-16T09:56:28.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-10-16T09:56:28.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.426+0000 7f72db243700 1 Processor -- start 2023-10-16T09:56:28.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.426+0000 7f72db243700 1 -- start start 2023-10-16T09:56:28.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72db243700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72d40cc6e0 0x7f72d4071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72db243700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40ce680 0x7f72d40ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72db243700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40ce1f0 con 0x7f72d40b2010 2023-10-16T09:56:28.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72db243700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d40ccb90 con 0x7f72d40cc6e0 2023-10-16T09:56:28.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72db243700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4072120 con 0x7f72d40ce680 2023-10-16T09:56:28.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72d3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72d40cc6e0 0x7f72d4071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72d3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72d40cc6e0 0x7f72d4071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39192/0 (socket says 172.21.15.181:39192) 2023-10-16T09:56:28.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72d3fff700 1 -- 172.21.15.181:0/641195611 learned_addr learned my addr 172.21.15.181:0/641195611 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:28.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.427+0000 7f72d97e0700 1 --2- 172.21.15.181:0/641195611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40ce680 0x7f72d40ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d3fff700 1 -- 172.21.15.181:0/641195611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40ce680 msgr2=0x7f72d40ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:28.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d3fff700 1 --2- 172.21.15.181:0/641195611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40ce680 0x7f72d40ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d3fff700 1 -- 172.21.15.181:0/641195611 >> v1:172.21.15.190:6789/0 conn(0x7f72d40b2010 legacy=0x7f72d40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d3fff700 1 -- 172.21.15.181:0/641195611 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40ccda0 con 0x7f72d40cc6e0 2023-10-16T09:56:28.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d3fff700 1 --2- 172.21.15.181:0/641195611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40cc6e0 0x7f72d4071be0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098f0 tx=0x7f72c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9375097c03b194a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d37fe700 1 -- 172.21.15.181:0/641195611 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0012b80 con 0x7f72d40cc6e0 2023-10-16T09:56:28.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d37fe700 1 -- 172.21.15.181:0/641195611 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c0012ce0 con 0x7f72d40cc6e0 2023-10-16T09:56:28.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72d37fe700 1 -- 172.21.15.181:0/641195611 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c001b620 con 0x7f72d40cc6e0 2023-10-16T09:56:28.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72db243700 1 -- 172.21.15.181:0/641195611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40cc6e0 msgr2=0x7f72d4071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.428+0000 7f72db243700 1 --2- 172.21.15.181:0/641195611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40cc6e0 0x7f72d4071be0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098f0 tx=0x7f72c0009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:28.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.429+0000 7f72db243700 1 -- 172.21.15.181:0/641195611 shutdown_connections 2023-10-16T09:56:28.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.429+0000 7f72db243700 1 --2- 172.21.15.181:0/641195611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40ce680 0x7f72d40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.429+0000 7f72db243700 1 --2- 172.21.15.181:0/641195611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40cc6e0 0x7f72d4071be0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.429+0000 7f72db243700 1 -- 172.21.15.181:0/641195611 >> 172.21.15.181:0/641195611 conn(0x7f72d40cd010 msgr2=0x7f72d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:28.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.429+0000 7f72db243700 1 -- 172.21.15.181:0/641195611 shutdown_connections 2023-10-16T09:56:28.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.429+0000 7f72db243700 1 -- 172.21.15.181:0/641195611 wait complete. 2023-10-16T09:56:28.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.430+0000 7f72db243700 1 Processor -- start 2023-10-16T09:56:28.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.430+0000 7f72db243700 1 -- start start 2023-10-16T09:56:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d40b2010 0x7f72d40b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40cc6e0 0x7f72d40c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40ce680 0x7f72d4151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40c85e0 con 0x7f72d40ce680 2023-10-16T09:56:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40c88c0 con 0x7f72d40b2010 2023-10-16T09:56:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40c8ba0 con 0x7f72d40cc6e0 2023-10-16T09:56:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40ce680 0x7f72d4151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40ce680 0x7f72d4151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39200/0 (socket says 172.21.15.181:39200) 2023-10-16T09:56:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 -- 172.21.15.181:0/991427884 learned_addr learned my addr 172.21.15.181:0/991427884 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d3fff700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40cc6e0 0x7f72d40c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d8fdf700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d40b2010 0x7f72d40b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 -- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40cc6e0 msgr2=0x7f72d40c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40cc6e0 0x7f72d40c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 -- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d40b2010 msgr2=0x7f72d40b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d40b2010 0x7f72d40b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 -- 172.21.15.181:0/991427884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0009550 con 0x7f72d40ce680 2023-10-16T09:56:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d8fdf700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d40b2010 0x7f72d40b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d97e0700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40ce680 0x7f72d4151630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00efc0 tx=0x7f72cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d17fa700 1 -- 172.21.15.181:0/991427884 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc009ea0 con 0x7f72d40ce680 2023-10-16T09:56:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d17fa700 1 -- 172.21.15.181:0/991427884 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc00f040 con 0x7f72d40ce680 2023-10-16T09:56:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d40c8e80 con 0x7f72d40ce680 2023-10-16T09:56:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72d17fa700 1 -- 172.21.15.181:0/991427884 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc014480 con 0x7f72d40ce680 2023-10-16T09:56:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.431+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d40c9ed0 con 0x7f72d40ce680 2023-10-16T09:56:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.432+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d40c91f0 con 0x7f72d40ce680 2023-10-16T09:56:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.433+0000 7f72d17fa700 1 -- 172.21.15.181:0/991427884 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f72cc0145e0 con 0x7f72d40ce680 2023-10-16T09:56:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.433+0000 7f72d17fa700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f72b806b100 0x7f72b806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.433+0000 7f72d17fa700 1 -- 172.21.15.181:0/991427884 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f72cc08a3e0 con 0x7f72d40ce680 2023-10-16T09:56:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.433+0000 7f72d8fdf700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f72b806b100 0x7f72b806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.433+0000 7f72d8fdf700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f72b806b100 0x7f72b806d5c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f72c8000f60 tx=0x7f72c8009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.437+0000 7f72d17fa700 1 -- 172.21.15.181:0/991427884 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f72d40c91f0 con 0x7f72d40ce680 2023-10-16T09:56:28.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.654+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72d40c8a30 con 0x7f72b806b100 2023-10-16T09:56:28.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.662+0000 7f72d17fa700 1 -- 172.21.15.181:0/991427884 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f72d40c8a30 con 0x7f72b806b100 2023-10-16T09:56:28.664 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:28.664 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:20", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:28.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f72b806b100 msgr2=0x7f72b806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f72b806b100 0x7f72b806d5c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f72c8000f60 tx=0x7f72c8009040 comp rx=0 tx=0).stop 2023-10-16T09:56:28.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40ce680 msgr2=0x7f72d4151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40ce680 0x7f72d4151630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00efc0 tx=0x7f72cc00c3a0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 shutdown_connections 2023-10-16T09:56:28.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f72b806b100 0x7f72b806d5c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d40ce680 0x7f72d4151630 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d40b2010 0x7f72d40b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 --2- 172.21.15.181:0/991427884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72d40cc6e0 0x7f72d40c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 >> 172.21.15.181:0/991427884 conn(0x7f72d40cd010 msgr2=0x7f72d40b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 shutdown_connections 2023-10-16T09:56:28.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.665+0000 7f72db243700 1 -- 172.21.15.181:0/991427884 wait complete. 2023-10-16T09:56:28.678 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T09:45:20", "first": "2023-10-16T09:46:00", "last": "2023-10-16T09:56:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-16T09:56:28.679 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T09:45:20', 'first': '2023-10-16T09:46:00', 'last': '2023-10-16T09:56:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-16T09:56:28.679 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T09:56:28.679 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T09:56:28.680 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1m 2023-10-16T09:56:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.801+0000 7f556a13d700 1 Processor -- start 2023-10-16T09:56:28.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.802+0000 7f556a13d700 1 -- start start 2023-10-16T09:56:28.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.802+0000 7f556a13d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f556415da70 0x7f55640ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.802+0000 7f556a13d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55640cef00 con 0x7f55640ce6a0 2023-10-16T09:56:28.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.802+0000 7f556a13d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55640cec20 con 0x7f556406afb0 2023-10-16T09:56:28.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.802+0000 7f556a13d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5564071c80 con 0x7f556415da70 2023-10-16T09:56:28.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f5562ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f55640ce6a0 0x7f556406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50284/0 (socket says 172.21.15.181:50284) 2023-10-16T09:56:28.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f5562ffd700 1 -- 172.21.15.181:0/2001869721 learned_addr learned my addr 172.21.15.181:0/2001869721 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:28.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55637fe700 1 --2- 172.21.15.181:0/2001869721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f556415da70 0x7f55640ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55627fc700 1 -- 172.21.15.181:0/2001869721 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977127064 0 0) 0x7f55640cef00 con 0x7f55640ce6a0 2023-10-16T09:56:28.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55627fc700 1 -- 172.21.15.181:0/2001869721 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55500035b0 con 0x7f55640ce6a0 2023-10-16T09:56:28.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55637fe700 1 -- 172.21.15.181:0/2001869721 >> v1:172.21.15.190:6789/0 conn(0x7f556406afb0 legacy=0x7f55641615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55627fc700 1 -- 172.21.15.181:0/2001869721 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353270627 0 0) 0x7f55500035b0 con 0x7f55640ce6a0 2023-10-16T09:56:28.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55637fe700 1 -- 172.21.15.181:0/2001869721 >> v1:172.21.15.181:6789/0 conn(0x7f55640ce6a0 legacy=0x7f556406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55637fe700 1 -- 172.21.15.181:0/2001869721 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5564071f60 con 0x7f556415da70 2023-10-16T09:56:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55637fe700 1 --2- 172.21.15.181:0/2001869721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f556415da70 0x7f55640ce160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f554c009870 tx=0x7f554c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2a39f329dccade server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55627fc700 1 -- 172.21.15.181:0/2001869721 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554001670 con 0x7f556415da70 2023-10-16T09:56:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55627fc700 1 -- 172.21.15.181:0/2001869721 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c004030 con 0x7f556415da70 2023-10-16T09:56:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.803+0000 7f55627fc700 1 -- 172.21.15.181:0/2001869721 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0043a0 con 0x7f556415da70 2023-10-16T09:56:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.804+0000 7f556a13d700 1 -- 172.21.15.181:0/2001869721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f556415da70 msgr2=0x7f55640ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.804+0000 7f556a13d700 1 --2- 172.21.15.181:0/2001869721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f556415da70 0x7f55640ce160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f554c009870 tx=0x7f554c009b80 comp rx=0 tx=0).stop 2023-10-16T09:56:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.804+0000 7f556a13d700 1 -- 172.21.15.181:0/2001869721 shutdown_connections 2023-10-16T09:56:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.804+0000 7f556a13d700 1 --2- 172.21.15.181:0/2001869721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f556415da70 0x7f55640ce160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.804+0000 7f556a13d700 1 -- 172.21.15.181:0/2001869721 >> 172.21.15.181:0/2001869721 conn(0x7f55640cd010 msgr2=0x7f55640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.804+0000 7f556a13d700 1 -- 172.21.15.181:0/2001869721 shutdown_connections 2023-10-16T09:56:28.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.804+0000 7f556a13d700 1 -- 172.21.15.181:0/2001869721 wait complete. 2023-10-16T09:56:28.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.805+0000 7f556a13d700 1 Processor -- start 2023-10-16T09:56:28.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.805+0000 7f556a13d700 1 -- start start 2023-10-16T09:56:28.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f556a13d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406afb0 0x7f55641f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f556a13d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55640ce6a0 0x7f55640b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f556a13d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f556415da70 0x7f55640b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f556a13d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55640b3580 con 0x7f556406afb0 2023-10-16T09:56:28.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f556a13d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55640b5780 con 0x7f556415da70 2023-10-16T09:56:28.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f556a13d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55640b5a60 con 0x7f55640ce6a0 2023-10-16T09:56:28.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406afb0 0x7f55641f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406afb0 0x7f55641f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39210/0 (socket says 172.21.15.181:39210) 2023-10-16T09:56:28.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 -- 172.21.15.181:0/3692891352 learned_addr learned my addr 172.21.15.181:0/3692891352 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:28.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f5562ffd700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55640ce6a0 0x7f55640b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f5563fff700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f556415da70 0x7f55640b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 -- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55640ce6a0 msgr2=0x7f55640b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55640ce6a0 0x7f55640b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 -- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f556415da70 msgr2=0x7f55640b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f556415da70 0x7f55640b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f55637fe700 1 -- 172.21.15.181:0/3692891352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0094a0 con 0x7f556406afb0 2023-10-16T09:56:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f5562ffd700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55640ce6a0 0x7f55640b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.806+0000 7f5563fff700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f556415da70 0x7f55640b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.807+0000 7f55637fe700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406afb0 0x7f55641f2b00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f554c000c00 tx=0x7f554c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.807+0000 7f555bfff700 1 -- 172.21.15.181:0/3692891352 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c011830 con 0x7f556406afb0 2023-10-16T09:56:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.807+0000 7f555bfff700 1 -- 172.21.15.181:0/3692891352 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c01f070 con 0x7f556406afb0 2023-10-16T09:56:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.807+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55640b5d40 con 0x7f556406afb0 2023-10-16T09:56:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.807+0000 7f555bfff700 1 -- 172.21.15.181:0/3692891352 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c011bd0 con 0x7f556406afb0 2023-10-16T09:56:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.807+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55641f3040 con 0x7f556406afb0 2023-10-16T09:56:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.808+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55640b6260 con 0x7f556406afb0 2023-10-16T09:56:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.808+0000 7f555bfff700 1 -- 172.21.15.181:0/3692891352 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f554c011d30 con 0x7f556406afb0 2023-10-16T09:56:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.808+0000 7f555bfff700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f554406b100 0x7f554406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.809+0000 7f555bfff700 1 -- 172.21.15.181:0/3692891352 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f554c021070 con 0x7f556406afb0 2023-10-16T09:56:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.809+0000 7f5562ffd700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f554406b100 0x7f554406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:28.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.809+0000 7f5562ffd700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f554406b100 0x7f554406d5c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f55540011e0 tx=0x7f5554009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:28.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:28.813+0000 7f555bfff700 1 -- 172.21.15.181:0/3692891352 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f55640b6260 con 0x7f556406afb0 2023-10-16T09:56:29.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.032+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f556415de50 con 0x7f554406b100 2023-10-16T09:56:29.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.117+0000 7f555bfff700 1 -- 172.21.15.181:0/3692891352 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f5554007df0 con 0x7f554406b100 2023-10-16T09:56:29.119 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-16T09:56:29.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f554406b100 msgr2=0x7f554406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:29.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f554406b100 0x7f554406d5c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f55540011e0 tx=0x7f5554009040 comp rx=0 tx=0).stop 2023-10-16T09:56:29.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406afb0 msgr2=0x7f55641f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:29.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406afb0 0x7f55641f2b00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f554c000c00 tx=0x7f554c004750 comp rx=0 tx=0).stop 2023-10-16T09:56:29.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 shutdown_connections 2023-10-16T09:56:29.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f554406b100 0x7f554406d5c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:29.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f556415da70 0x7f55640b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:29.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f556406afb0 0x7f55641f2b00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:29.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 --2- 172.21.15.181:0/3692891352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55640ce6a0 0x7f55640b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:29.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 >> 172.21.15.181:0/3692891352 conn(0x7f55640cd010 msgr2=0x7f556406f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:29.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 shutdown_connections 2023-10-16T09:56:29.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:29.119+0000 7f556a13d700 1 -- 172.21.15.181:0/3692891352 wait complete. 2023-10-16T09:56:29.133 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-16T09:56:29.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.133 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-16T09:56:29.151 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:29.151 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:29.151 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:29.151 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:29.152 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:29.152 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:29.152 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:29.152 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:29.152 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:29.153 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:29.153 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:29.153 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:29.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.154 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:29.214 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:29.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.214 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:29.276 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:29.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.277 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:29.341 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:29.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.342 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:29.406 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:29.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.407 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:29.475 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:29.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.475 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:29.545 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:29.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.546 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:29.611 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:29.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.612 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:29.675 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:29.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.675 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:29.741 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:29.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.742 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:29.804 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:29.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.805 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:29.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:29.867 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-16T09:56:29.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1m 2023-10-16T09:56:30.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.087+0000 7fa034a94700 1 Processor -- start 2023-10-16T09:56:30.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.087+0000 7fa034a94700 1 -- start start 2023-10-16T09:56:30.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.088+0000 7fa034a94700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa03015a5f0 0x7fa0300cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.088+0000 7fa034a94700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03015b510 con 0x7fa0300cd570 2023-10-16T09:56:30.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.088+0000 7fa034a94700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03015b7f0 con 0x7fa0300cd0c0 2023-10-16T09:56:30.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.088+0000 7fa034a94700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0300720e0 con 0x7fa03015a5f0 2023-10-16T09:56:30.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa0300cd570 0x7fa030072500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50300/0 (socket says 172.21.15.181:50300) 2023-10-16T09:56:30.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02ed9d700 1 -- 172.21.15.181:0/4008775837 learned_addr learned my addr 172.21.15.181:0/4008775837 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:30.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02e59c700 1 --2- 172.21.15.181:0/4008775837 >> v2:172.21.15.190:3300/0 conn(0x7fa03015a5f0 0x7fa0300cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213073415 0 0) 0x7fa03015b510 con 0x7fa0300cd570 2023-10-16T09:56:30.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0180035b0 con 0x7fa0300cd570 2023-10-16T09:56:30.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399631105 0 0) 0x7fa0180035b0 con 0x7fa0300cd570 2023-10-16T09:56:30.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 >> v2:172.21.15.190:3300/0 conn(0x7fa03015a5f0 msgr2=0x7fa0300cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:30.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 --2- 172.21.15.181:0/4008775837 >> v2:172.21.15.190:3300/0 conn(0x7fa03015a5f0 0x7fa0300cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 >> v1:172.21.15.190:6789/0 conn(0x7fa0300cd0c0 legacy=0x7fa03015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:30.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03015bad0 con 0x7fa0300cd570 2023-10-16T09:56:30.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa0280019f0 con 0x7fa0300cd570 2023-10-16T09:56:30.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 >> v1:172.21.15.181:6789/0 conn(0x7fa0300cd570 legacy=0x7fa030072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:30.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018003a00 0x7fa018003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018004850 0x7fa018004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa018005330 0x7fa01800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300720e0 con 0x7fa018005330 2023-10-16T09:56:30.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01800c200 con 0x7fa018003a00 2023-10-16T09:56:30.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01800c3c0 con 0x7fa018004850 2023-10-16T09:56:30.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02ed9d700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa018005330 0x7fa01800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02e59c700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018003a00 0x7fa018003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.089+0000 7fa02dd9b700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018004850 0x7fa018004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02ed9d700 1 -- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018004850 msgr2=0x7fa018004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:30.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02ed9d700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018004850 0x7fa018004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02ed9d700 1 -- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018003a00 msgr2=0x7fa018003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:30.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02ed9d700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018003a00 0x7fa018003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02ed9d700 1 -- 172.21.15.181:0/4008775837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180035b0 con 0x7fa018005330 2023-10-16T09:56:30.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02ed9d700 1 -- 172.21.15.181:0/4008775837 reap_dead start 2023-10-16T09:56:30.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02ed9d700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa018005330 0x7fa01800ba10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa02800a800 tx=0x7fa018005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e79ad0b32ba5fbe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:30.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800a2f0 con 0x7fa018005330 2023-10-16T09:56:30.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028012600 con 0x7fa018005330 2023-10-16T09:56:30.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa02d59a700 1 -- 172.21.15.181:0/4008775837 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0280129b0 con 0x7fa018005330 2023-10-16T09:56:30.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa034a94700 1 -- 172.21.15.181:0/4008775837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa018005330 msgr2=0x7fa01800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:30.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa034a94700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa018005330 0x7fa01800ba10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa02800a800 tx=0x7fa018005880 comp rx=0 tx=0).stop 2023-10-16T09:56:30.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa034a94700 1 -- 172.21.15.181:0/4008775837 shutdown_connections 2023-10-16T09:56:30.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa034a94700 1 --2- 172.21.15.181:0/4008775837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa018005330 0x7fa01800ba10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.090+0000 7fa034a94700 1 -- 172.21.15.181:0/4008775837 >> 172.21.15.181:0/4008775837 conn(0x7fa03006aed0 msgr2=0x7fa03006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:30.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.091+0000 7fa034a94700 1 -- 172.21.15.181:0/4008775837 shutdown_connections 2023-10-16T09:56:30.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.091+0000 7fa034a94700 1 -- 172.21.15.181:0/4008775837 wait complete. 2023-10-16T09:56:30.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.092+0000 7fa034a94700 1 Processor -- start 2023-10-16T09:56:30.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.092+0000 7fa034a94700 1 -- start start 2023-10-16T09:56:30.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018005330 0x7fa030153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0300ac880 0x7fa030157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0300ae1a0 0x7fa030158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300b2a40 con 0x7fa0300ae1a0 2023-10-16T09:56:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300b31e0 con 0x7fa0300ac880 2023-10-16T09:56:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300b2080 con 0x7fa018005330 2023-10-16T09:56:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0300ae1a0 0x7fa030158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0300ae1a0 0x7fa030158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39234/0 (socket says 172.21.15.181:39234) 2023-10-16T09:56:30.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 -- 172.21.15.181:0/1791782728 learned_addr learned my addr 172.21.15.181:0/1791782728 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:30.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02e59c700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018005330 0x7fa030153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02dd9b700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0300ac880 0x7fa030157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 -- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018005330 msgr2=0x7fa030153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:30.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018005330 0x7fa030153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 -- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0300ac880 msgr2=0x7fa030157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:30.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0300ac880 0x7fa030157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 -- 172.21.15.181:0/1791782728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300b53b0 con 0x7fa0300ae1a0 2023-10-16T09:56:30.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02dd9b700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0300ac880 0x7fa030157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa02ed9d700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0300ae1a0 0x7fa030158200 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0300ce3b0 tx=0x7fa028001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa01effd700 1 -- 172.21.15.181:0/1791782728 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0280017b0 con 0x7fa0300ae1a0 2023-10-16T09:56:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa01effd700 1 -- 172.21.15.181:0/1791782728 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028001910 con 0x7fa0300ae1a0 2023-10-16T09:56:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa01effd700 1 -- 172.21.15.181:0/1791782728 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02801d660 con 0x7fa0300ae1a0 2023-10-16T09:56:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0300b2c20 con 0x7fa0300ae1a0 2023-10-16T09:56:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.093+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300b2290 con 0x7fa0300ae1a0 2023-10-16T09:56:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.094+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03004ebb0 con 0x7fa0300ae1a0 2023-10-16T09:56:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.095+0000 7fa01effd700 1 -- 172.21.15.181:0/1791782728 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fa02801d7c0 con 0x7fa0300ae1a0 2023-10-16T09:56:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.095+0000 7fa01effd700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa01006b100 0x7fa01006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.095+0000 7fa01effd700 1 -- 172.21.15.181:0/1791782728 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa02808b850 con 0x7fa0300ae1a0 2023-10-16T09:56:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.095+0000 7fa02e59c700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa01006b100 0x7fa01006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.096+0000 7fa02e59c700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa01006b100 0x7fa01006d5c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa024009b20 tx=0x7fa024008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.100+0000 7fa01effd700 1 -- 172.21.15.181:0/1791782728 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa028056610 con 0x7fa0300ae1a0 2023-10-16T09:56:30.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.320+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa030071c90 con 0x7fa01006b100 2023-10-16T09:56:30.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.406+0000 7fa01effd700 1 -- 172.21.15.181:0/1791782728 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa024009790 con 0x7fa01006b100 2023-10-16T09:56:30.408 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-16T09:56:30.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa01006b100 msgr2=0x7fa01006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:30.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa01006b100 0x7fa01006d5c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa024009b20 tx=0x7fa024008040 comp rx=0 tx=0).stop 2023-10-16T09:56:30.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0300ae1a0 msgr2=0x7fa030158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:30.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0300ae1a0 0x7fa030158200 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0300ce3b0 tx=0x7fa028001d80 comp rx=0 tx=0).stop 2023-10-16T09:56:30.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 shutdown_connections 2023-10-16T09:56:30.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa018005330 0x7fa030153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fa01006b100 0x7fa01006d5c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0300ae1a0 0x7fa030158200 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 --2- 172.21.15.181:0/1791782728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0300ac880 0x7fa030157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:30.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 >> 172.21.15.181:0/1791782728 conn(0x7fa03006aed0 msgr2=0x7fa03006d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:30.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 shutdown_connections 2023-10-16T09:56:30.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:30.408+0000 7fa034a94700 1 -- 172.21.15.181:0/1791782728 wait complete. 2023-10-16T09:56:30.422 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-16T09:56:30.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.422 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-16T09:56:30.440 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:30.440 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:30.440 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:30.440 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:30.441 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:30.441 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:30.441 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:30.441 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:30.441 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:30.442 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:30.442 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:30.442 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:30.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.443 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:30.504 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:30.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.505 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:30.567 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:30.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.568 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:30.630 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:30.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.631 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:30.695 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:30.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.696 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:30.758 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:30.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.759 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:30.822 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:30.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.823 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:30.885 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:30.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.885 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:30.948 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:30.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:30.949 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:31.011 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:31.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.012 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:31.077 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:31.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.078 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:31.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.142 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-16T09:56:31.200 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1m 2023-10-16T09:56:31.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.361+0000 7f4db890a700 1 Processor -- start 2023-10-16T09:56:31.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.362+0000 7f4db890a700 1 -- start start 2023-10-16T09:56:31.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.362+0000 7f4db890a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4db40cc6e0 0x7f4db4071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:31.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.362+0000 7f4db890a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40ce680 0x7f4db40ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:31.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.362+0000 7f4db890a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db40ce1f0 con 0x7f4db40b2010 2023-10-16T09:56:31.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.362+0000 7f4db890a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db40ccb90 con 0x7f4db40cc6e0 2023-10-16T09:56:31.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.362+0000 7f4db890a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4072120 con 0x7f4db40ce680 2023-10-16T09:56:31.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4db40cc6e0 0x7f4db4071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:31.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4db40cc6e0 0x7f4db4071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39240/0 (socket says 172.21.15.181:39240) 2023-10-16T09:56:31.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 -- 172.21.15.181:0/3022168768 learned_addr learned my addr 172.21.15.181:0/3022168768 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:31.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db2d9d700 1 --2- 172.21.15.181:0/3022168768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40ce680 0x7f4db40ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:31.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 -- 172.21.15.181:0/3022168768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40ce680 msgr2=0x7f4db40ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:31.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 --2- 172.21.15.181:0/3022168768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40ce680 0x7f4db40ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 -- 172.21.15.181:0/3022168768 >> v1:172.21.15.190:6789/0 conn(0x7f4db40b2010 legacy=0x7f4db40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:31.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 -- 172.21.15.181:0/3022168768 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db40ccda0 con 0x7f4db40cc6e0 2023-10-16T09:56:31.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db2d9d700 1 --2- 172.21.15.181:0/3022168768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40ce680 0x7f4db40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:31.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db1d9b700 1 --2- 172.21.15.181:0/3022168768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40cc6e0 0x7f4db4071be0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4da400bb90 tx=0x7f4da400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3ebe17ef6a90c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:31.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db159a700 1 -- 172.21.15.181:0/3022168768 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da400dd10 con 0x7f4db40cc6e0 2023-10-16T09:56:31.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db159a700 1 -- 172.21.15.181:0/3022168768 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da400de70 con 0x7f4db40cc6e0 2023-10-16T09:56:31.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.363+0000 7f4db159a700 1 -- 172.21.15.181:0/3022168768 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da4004230 con 0x7f4db40cc6e0 2023-10-16T09:56:31.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 -- 172.21.15.181:0/3022168768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40cc6e0 msgr2=0x7f4db4071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:31.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 --2- 172.21.15.181:0/3022168768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40cc6e0 0x7f4db4071be0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4da400bb90 tx=0x7f4da400bea0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 -- 172.21.15.181:0/3022168768 shutdown_connections 2023-10-16T09:56:31.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 --2- 172.21.15.181:0/3022168768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40ce680 0x7f4db40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 --2- 172.21.15.181:0/3022168768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40cc6e0 0x7f4db4071be0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 -- 172.21.15.181:0/3022168768 >> 172.21.15.181:0/3022168768 conn(0x7f4db40cd010 msgr2=0x7f4db40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:31.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 -- 172.21.15.181:0/3022168768 shutdown_connections 2023-10-16T09:56:31.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.364+0000 7f4db890a700 1 -- 172.21.15.181:0/3022168768 wait complete. 2023-10-16T09:56:31.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.365+0000 7f4db890a700 1 Processor -- start 2023-10-16T09:56:31.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.365+0000 7f4db890a700 1 -- start start 2023-10-16T09:56:31.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db890a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40b2010 0x7f4db414cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:31.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db890a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db40cc6e0 0x7f4db415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:31.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db890a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40ce680 0x7f4db415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:31.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db890a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db415afa0 con 0x7f4db40ce680 2023-10-16T09:56:31.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db890a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4151920 con 0x7f4db40cc6e0 2023-10-16T09:56:31.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db890a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4151c00 con 0x7f4db40b2010 2023-10-16T09:56:31.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40ce680 0x7f4db415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:31.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40ce680 0x7f4db415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39248/0 (socket says 172.21.15.181:39248) 2023-10-16T09:56:31.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 -- 172.21.15.181:0/3543316529 learned_addr learned my addr 172.21.15.181:0/3543316529 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:31.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db259c700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40b2010 0x7f4db414cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:31.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db1d9b700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db40cc6e0 0x7f4db415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:31.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 -- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40b2010 msgr2=0x7f4db414cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:31.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40b2010 0x7f4db414cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 -- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db40cc6e0 msgr2=0x7f4db415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:31.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db40cc6e0 0x7f4db415aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.366+0000 7f4db2d9d700 1 -- 172.21.15.181:0/3543316529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da400b7f0 con 0x7f4db40ce680 2023-10-16T09:56:31.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.367+0000 7f4db2d9d700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40ce680 0x7f4db415c5a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00cef0 tx=0x7f4dac00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:31.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.367+0000 7f4da2ffd700 1 -- 172.21.15.181:0/3543316529 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00ae50 con 0x7f4db40ce680 2023-10-16T09:56:31.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.367+0000 7f4da2ffd700 1 -- 172.21.15.181:0/3543316529 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac015440 con 0x7f4db40ce680 2023-10-16T09:56:31.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.367+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db4151ee0 con 0x7f4db40ce680 2023-10-16T09:56:31.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.367+0000 7f4da2ffd700 1 -- 172.21.15.181:0/3543316529 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0157f0 con 0x7f4db40ce680 2023-10-16T09:56:31.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.367+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db414f7e0 con 0x7f4db40ce680 2023-10-16T09:56:31.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.368+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db4155c50 con 0x7f4db40ce680 2023-10-16T09:56:31.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.368+0000 7f4da2ffd700 1 -- 172.21.15.181:0/3543316529 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4dac00d430 con 0x7f4db40ce680 2023-10-16T09:56:31.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.369+0000 7f4da2ffd700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4d9406b130 0x7f4d9406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:31.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.369+0000 7f4db259c700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4d9406b130 0x7f4d9406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:31.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.369+0000 7f4da2ffd700 1 -- 172.21.15.181:0/3543316529 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4dac08a040 con 0x7f4db40ce680 2023-10-16T09:56:31.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.369+0000 7f4db259c700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4d9406b130 0x7f4d9406d5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4da800a7d0 tx=0x7f4da8008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:31.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.373+0000 7f4da2ffd700 1 -- 172.21.15.181:0/3543316529 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4db4155c50 con 0x7f4db40ce680 2023-10-16T09:56:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.588+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4db4151a90 con 0x7f4d9406b130 2023-10-16T09:56:31.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.671+0000 7f4da2ffd700 1 -- 172.21.15.181:0/3543316529 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f4db4151a90 con 0x7f4d9406b130 2023-10-16T09:56:31.673 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-16T09:56:31.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4d9406b130 msgr2=0x7f4d9406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:31.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4d9406b130 0x7f4d9406d5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4da800a7d0 tx=0x7f4da8008040 comp rx=0 tx=0).stop 2023-10-16T09:56:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40ce680 msgr2=0x7f4db415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40ce680 0x7f4db415c5a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00cef0 tx=0x7f4dac00a340 comp rx=0 tx=0).stop 2023-10-16T09:56:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 shutdown_connections 2023-10-16T09:56:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4db40b2010 0x7f4db414cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4d9406b130 0x7f4d9406d5f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4db40ce680 0x7f4db415c5a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 --2- 172.21.15.181:0/3543316529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4db40cc6e0 0x7f4db415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:31.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 >> 172.21.15.181:0/3543316529 conn(0x7f4db40cd010 msgr2=0x7f4db406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:31.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.673+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 shutdown_connections 2023-10-16T09:56:31.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:31.674+0000 7f4db890a700 1 -- 172.21.15.181:0/3543316529 wait complete. 2023-10-16T09:56:31.687 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-16T09:56:31.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.687 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-16T09:56:31.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:31.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:31.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:31.706 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:31.706 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:31.706 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:31.706 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:31.706 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:31.707 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:31.707 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:31.707 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:31.707 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:31.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.708 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:31.773 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:31.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.774 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:31.837 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:31.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.838 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:31.900 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:31.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.901 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:31.963 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:31.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:31.964 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:32.026 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:32.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.027 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:32.089 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:32.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.090 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:32.152 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:32.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.153 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:32.215 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:32.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.216 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:32.284 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:32.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.285 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:32.347 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:32.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.348 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:32.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.411 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-16T09:56:32.470 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1m 2023-10-16T09:56:32.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 Processor -- start 2023-10-16T09:56:32.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 -- start start 2023-10-16T09:56:32.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56ac0d01d0 0x7f56ac0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:32.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac15ea40 0x7f56ac0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56ac161530 0x7f56ac1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0d4290 con 0x7f56ac0d01d0 2023-10-16T09:56:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0cf580 con 0x7f56ac161530 2023-10-16T09:56:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b2e98700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0d0af0 con 0x7f56ac15ea40 2023-10-16T09:56:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.631+0000 7f56b0c34700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56ac0d01d0 0x7f56ac0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:32.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56ac0d01d0 0x7f56ac0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39252/0 (socket says 172.21.15.181:39252) 2023-10-16T09:56:32.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 -- 172.21.15.181:0/4105832167 learned_addr learned my addr 172.21.15.181:0/4105832167 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:32.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 -- 172.21.15.181:0/4105832167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac15ea40 msgr2=0x7f56ac0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:32.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56abfff700 1 --2- 172.21.15.181:0/4105832167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac15ea40 0x7f56ac0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:32.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b1435700 1 --2- 172.21.15.181:0/4105832167 >> v2:172.21.15.190:3300/0 conn(0x7f56ac161530 0x7f56ac1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:32.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 --2- 172.21.15.181:0/4105832167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac15ea40 0x7f56ac0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 -- 172.21.15.181:0/4105832167 >> v2:172.21.15.190:3300/0 conn(0x7f56ac161530 msgr2=0x7f56ac1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:32.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 --2- 172.21.15.181:0/4105832167 >> v2:172.21.15.190:3300/0 conn(0x7f56ac161530 0x7f56ac1619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 -- 172.21.15.181:0/4105832167 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0ce970 con 0x7f56ac0d01d0 2023-10-16T09:56:32.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56abfff700 1 --2- 172.21.15.181:0/4105832167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac15ea40 0x7f56ac0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:32.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b1435700 1 --2- 172.21.15.181:0/4105832167 >> v2:172.21.15.190:3300/0 conn(0x7f56ac161530 0x7f56ac1619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:32.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56b0c34700 1 --2- 172.21.15.181:0/4105832167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0ce2d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f56ac08daa0 tx=0x7f56a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a4864e2849d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:32.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56ab7fe700 1 -- 172.21.15.181:0/4105832167 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a000b870 con 0x7f56ac0d01d0 2023-10-16T09:56:32.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56ab7fe700 1 -- 172.21.15.181:0/4105832167 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a000b9d0 con 0x7f56ac0d01d0 2023-10-16T09:56:32.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.632+0000 7f56ab7fe700 1 -- 172.21.15.181:0/4105832167 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a000bd60 con 0x7f56ac0d01d0 2023-10-16T09:56:32.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 -- 172.21.15.181:0/4105832167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 msgr2=0x7f56ac0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:32.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 --2- 172.21.15.181:0/4105832167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0ce2d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f56ac08daa0 tx=0x7f56a0009bd0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 -- 172.21.15.181:0/4105832167 shutdown_connections 2023-10-16T09:56:32.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 --2- 172.21.15.181:0/4105832167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0ce2d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 --2- 172.21.15.181:0/4105832167 >> v2:172.21.15.190:3300/0 conn(0x7f56ac161530 0x7f56ac1619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 --2- 172.21.15.181:0/4105832167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac15ea40 0x7f56ac0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 -- 172.21.15.181:0/4105832167 >> 172.21.15.181:0/4105832167 conn(0x7f56ac09cb60 msgr2=0x7f56ac0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:32.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 -- 172.21.15.181:0/4105832167 shutdown_connections 2023-10-16T09:56:32.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.633+0000 7f56b2e98700 1 -- 172.21.15.181:0/4105832167 wait complete. 2023-10-16T09:56:32.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.634+0000 7f56b2e98700 1 Processor -- start 2023-10-16T09:56:32.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.634+0000 7f56b2e98700 1 -- start start 2023-10-16T09:56:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.635+0000 7f56b2e98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.635+0000 7f56b2e98700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ac15ea40 0x7f56ac158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.635+0000 7f56b2e98700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac161530 0x7f56ac0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.635+0000 7f56b2e98700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac159360 con 0x7f56ac0d01d0 2023-10-16T09:56:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.635+0000 7f56b2e98700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0b39b0 con 0x7f56ac15ea40 2023-10-16T09:56:32.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.635+0000 7f56b2e98700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac0b3c90 con 0x7f56ac161530 2023-10-16T09:56:32.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.635+0000 7f56b0c34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:32.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39258/0 (socket says 172.21.15.181:39258) 2023-10-16T09:56:32.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 -- 172.21.15.181:0/2486679781 learned_addr learned my addr 172.21.15.181:0/2486679781 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b1435700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac161530 0x7f56ac0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56abfff700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ac15ea40 0x7f56ac158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 -- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac161530 msgr2=0x7f56ac0b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac161530 0x7f56ac0b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 -- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ac15ea40 msgr2=0x7f56ac158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ac15ea40 0x7f56ac158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 -- 172.21.15.181:0/2486679781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0009550 con 0x7f56ac0d01d0 2023-10-16T09:56:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56abfff700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ac15ea40 0x7f56ac158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b0c34700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0c8050 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56a0014040 tx=0x7f56a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56a97fa700 1 -- 172.21.15.181:0/2486679781 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004310 con 0x7f56ac0d01d0 2023-10-16T09:56:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56a97fa700 1 -- 172.21.15.181:0/2486679781 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a001e070 con 0x7f56ac0d01d0 2023-10-16T09:56:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56a97fa700 1 -- 172.21.15.181:0/2486679781 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00036a0 con 0x7f56ac0d01d0 2023-10-16T09:56:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac0b3f70 con 0x7f56ac0d01d0 2023-10-16T09:56:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.636+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac0b4530 con 0x7f56ac0d01d0 2023-10-16T09:56:32.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.637+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac0d2eb0 con 0x7f56ac0d01d0 2023-10-16T09:56:32.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.637+0000 7f56a97fa700 1 -- 172.21.15.181:0/2486679781 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f56a0020030 con 0x7f56ac0d01d0 2023-10-16T09:56:32.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.638+0000 7f56a97fa700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f569006b130 0x7f569006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:32.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.638+0000 7f56a97fa700 1 -- 172.21.15.181:0/2486679781 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f56a008aa20 con 0x7f56ac0d01d0 2023-10-16T09:56:32.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.638+0000 7f56abfff700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f569006b130 0x7f569006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:32.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.638+0000 7f56abfff700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f569006b130 0x7f569006d5f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f569800aa50 tx=0x7f5698008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:32.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.642+0000 7f56a97fa700 1 -- 172.21.15.181:0/2486679781 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f56a00557e0 con 0x7f56ac0d01d0 2023-10-16T09:56:32.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.865+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f56ac0cef30 con 0x7f569006b130 2023-10-16T09:56:32.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.948+0000 7f56a97fa700 1 -- 172.21.15.181:0/2486679781 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f569800a270 con 0x7f569006b130 2023-10-16T09:56:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-16T09:56:32.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.950+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f569006b130 msgr2=0x7f569006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:32.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.950+0000 7f56b2e98700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f569006b130 0x7f569006d5f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f569800aa50 tx=0x7f5698008040 comp rx=0 tx=0).stop 2023-10-16T09:56:32.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 msgr2=0x7f56ac0c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:32.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0c8050 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56a0014040 tx=0x7f56a0004750 comp rx=0 tx=0).stop 2023-10-16T09:56:32.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 shutdown_connections 2023-10-16T09:56:32.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56ac161530 0x7f56ac0b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f569006b130 0x7f569006d5f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56ac0d01d0 0x7f56ac0c8050 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 --2- 172.21.15.181:0/2486679781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ac15ea40 0x7f56ac158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:32.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 >> 172.21.15.181:0/2486679781 conn(0x7f56ac09cb60 msgr2=0x7f56ac1619c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:32.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 shutdown_connections 2023-10-16T09:56:32.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:32.951+0000 7f56b2e98700 1 -- 172.21.15.181:0/2486679781 wait complete. 2023-10-16T09:56:32.964 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-16T09:56:32.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.965 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-16T09:56:32.983 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:32.983 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:32.984 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:32.984 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:32.984 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:32.984 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:32.984 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:32.985 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:32.985 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:32.985 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:32.985 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:32.985 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:32.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:32.986 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:33.048 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:33.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.049 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:33.111 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:33.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.112 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:33.174 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:33.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.175 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:33.238 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:33.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.239 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:33.301 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:33.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.302 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:33.364 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:33.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.365 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:33.428 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:33.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.429 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:33.491 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:33.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.492 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:33.554 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:33.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.555 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:33.617 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:33.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.618 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:33.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:33.680 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-16T09:56:33.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1m 2023-10-16T09:56:33.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.900+0000 7f7939f7e700 1 Processor -- start 2023-10-16T09:56:33.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.900+0000 7f7939f7e700 1 -- start start 2023-10-16T09:56:33.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7939f7e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7934072760 0x7f793406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:33.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7939f7e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793406ade0 0x7f79341615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:33.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7939f7e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793406b210 con 0x7f793415da70 2023-10-16T09:56:33.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7939f7e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79340cef00 con 0x7f7934072760 2023-10-16T09:56:33.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7939f7e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79340cec20 con 0x7f793406ade0 2023-10-16T09:56:33.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7932ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7934072760 0x7f793406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:33.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7932ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7934072760 0x7f793406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39268/0 (socket says 172.21.15.181:39268) 2023-10-16T09:56:33.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.901+0000 7f7932ffd700 1 -- 172.21.15.181:0/3124932154 learned_addr learned my addr 172.21.15.181:0/3124932154 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:33.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7933fff700 1 --2- 172.21.15.181:0/3124932154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793406ade0 0x7f79341615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:33.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7932ffd700 1 -- 172.21.15.181:0/3124932154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793406ade0 msgr2=0x7f79341615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:33.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7932ffd700 1 --2- 172.21.15.181:0/3124932154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793406ade0 0x7f79341615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7932ffd700 1 -- 172.21.15.181:0/3124932154 >> v1:172.21.15.190:6789/0 conn(0x7f793415da70 legacy=0x7f79340ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7932ffd700 1 -- 172.21.15.181:0/3124932154 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7934071c80 con 0x7f7934072760 2023-10-16T09:56:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7932ffd700 1 --2- 172.21.15.181:0/3124932154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7934072760 0x7f793406a8a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f79240098f0 tx=0x7f7924009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56af9a30eae4f94 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f79327fc700 1 -- 172.21.15.181:0/3124932154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924004230 con 0x7f7934072760 2023-10-16T09:56:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f79327fc700 1 -- 172.21.15.181:0/3124932154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7924004390 con 0x7f7934072760 2023-10-16T09:56:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f79327fc700 1 -- 172.21.15.181:0/3124932154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240036a0 con 0x7f7934072760 2023-10-16T09:56:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7939f7e700 1 -- 172.21.15.181:0/3124932154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7934072760 msgr2=0x7f793406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3124932154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7934072760 0x7f793406a8a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f79240098f0 tx=0x7f7924009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.902+0000 7f7939f7e700 1 -- 172.21.15.181:0/3124932154 shutdown_connections 2023-10-16T09:56:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.903+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3124932154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793406ade0 0x7f79341615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.903+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3124932154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7934072760 0x7f793406a8a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.903+0000 7f7939f7e700 1 -- 172.21.15.181:0/3124932154 >> 172.21.15.181:0/3124932154 conn(0x7f79340cd010 msgr2=0x7f79340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.903+0000 7f7939f7e700 1 -- 172.21.15.181:0/3124932154 shutdown_connections 2023-10-16T09:56:33.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.903+0000 7f7939f7e700 1 -- 172.21.15.181:0/3124932154 wait complete. 2023-10-16T09:56:33.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.904+0000 7f7939f7e700 1 Processor -- start 2023-10-16T09:56:33.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.904+0000 7f7939f7e700 1 -- start start 2023-10-16T09:56:33.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f793406ade0 0x7f7934160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:33.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7934072760 0x7f793415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:33.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793415da70 0x7f793415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:33.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79341f7370 con 0x7f793406ade0 2023-10-16T09:56:33.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7934160fa0 con 0x7f7934072760 2023-10-16T09:56:33.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79341f4290 con 0x7f793415da70 2023-10-16T09:56:33.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f793406ade0 0x7f7934160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:33.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f793406ade0 0x7f7934160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39272/0 (socket says 172.21.15.181:39272) 2023-10-16T09:56:33.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 -- 172.21.15.181:0/3852528173 learned_addr learned my addr 172.21.15.181:0/3852528173 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:33.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7933fff700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793415da70 0x7f793415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:33.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7932ffd700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7934072760 0x7f793415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:33.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 -- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793415da70 msgr2=0x7f793415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:33.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793415da70 0x7f793415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:33.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 -- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7934072760 msgr2=0x7f793415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:33.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7934072760 0x7f793415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:33.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 -- 172.21.15.181:0/3852528173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924009550 con 0x7f793406ade0 2023-10-16T09:56:33.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7932ffd700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7934072760 0x7f793415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:33.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f79337fe700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f793406ade0 0x7f7934160a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f792800b470 tx=0x7f792800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:33.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f791bfff700 1 -- 172.21.15.181:0/3852528173 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928011830 con 0x7f793406ade0 2023-10-16T09:56:33.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f791bfff700 1 -- 172.21.15.181:0/3852528173 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7928011990 con 0x7f793406ade0 2023-10-16T09:56:33.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79341f69d0 con 0x7f793406ade0 2023-10-16T09:56:33.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f791bfff700 1 -- 172.21.15.181:0/3852528173 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928011d40 con 0x7f793406ade0 2023-10-16T09:56:33.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.905+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79341f6f90 con 0x7f793406ade0 2023-10-16T09:56:33.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.907+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793406f6c0 con 0x7f793406ade0 2023-10-16T09:56:33.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.907+0000 7f791bfff700 1 -- 172.21.15.181:0/3852528173 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f792801a400 con 0x7f793406ade0 2023-10-16T09:56:33.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.907+0000 7f791bfff700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f792006b100 0x7f792006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:33.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.907+0000 7f791bfff700 1 -- 172.21.15.181:0/3852528173 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7928089b10 con 0x7f793406ade0 2023-10-16T09:56:33.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.907+0000 7f7932ffd700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f792006b100 0x7f792006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:33.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.908+0000 7f7932ffd700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f792006b100 0x7f792006d5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79340c4350 tx=0x7f7924008030 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:33.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:33.912+0000 7f791bfff700 1 -- 172.21.15.181:0/3852528173 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7928054840 con 0x7f793406ade0 2023-10-16T09:56:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.181+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7934072240 con 0x7f792006b100 2023-10-16T09:56:34.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.430+0000 7f791bfff700 1 -- 172.21.15.181:0/3852528173 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f7924020070 con 0x7f792006b100 2023-10-16T09:56:34.432 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-16T09:56:34.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f792006b100 msgr2=0x7f792006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:34.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f792006b100 0x7f792006d5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79340c4350 tx=0x7f7924008030 comp rx=0 tx=0).stop 2023-10-16T09:56:34.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f793406ade0 msgr2=0x7f7934160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:34.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f793406ade0 0x7f7934160a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f792800b470 tx=0x7f792800b830 comp rx=0 tx=0).stop 2023-10-16T09:56:34.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 shutdown_connections 2023-10-16T09:56:34.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f793415da70 0x7f793415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:34.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f792006b100 0x7f792006d5c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:34.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f793406ade0 0x7f7934160a60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:34.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 --2- 172.21.15.181:0/3852528173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7934072760 0x7f793415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:34.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.432+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 >> 172.21.15.181:0/3852528173 conn(0x7f79340cd010 msgr2=0x7f79340cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:34.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.433+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 shutdown_connections 2023-10-16T09:56:34.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:34.433+0000 7f7939f7e700 1 -- 172.21.15.181:0/3852528173 wait complete. 2023-10-16T09:56:34.446 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-16T09:56:34.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.447 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-16T09:56:34.465 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:34.465 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:34.465 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:34.466 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:34.466 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:34.466 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:34.466 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:34.467 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:34.467 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:34.467 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:34.467 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:34.467 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:34.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.468 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:34.530 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:34.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.532 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:34.594 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:34.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.595 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:34.658 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:34.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.659 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:34.721 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:34.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.722 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:34.784 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:34.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.785 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:34.847 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:34.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.848 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:34.910 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:34.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.911 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:34.974 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:34.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:34.975 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:35.055 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:35.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.056 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:35.118 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:35.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.119 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:35.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.181 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-16T09:56:35.240 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1m 2023-10-16T09:56:35.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a94159700 1 Processor -- start 2023-10-16T09:56:35.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a94159700 1 -- start start 2023-10-16T09:56:35.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a94159700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 0x7f4a8c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a94159700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c0cb340 con 0x7f4a8c15ac20 2023-10-16T09:56:35.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a94159700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c15b0d0 con 0x7f4a8c0ceb60 2023-10-16T09:56:35.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a94159700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c15b3b0 con 0x7f4a8c158360 2023-10-16T09:56:35.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a916f4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4a8c15ac20 0x7f4a8c15ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50362/0 (socket says 172.21.15.181:50362) 2023-10-16T09:56:35.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.402+0000 7f4a916f4700 1 -- 172.21.15.181:0/869561585 learned_addr learned my addr 172.21.15.181:0/869561585 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:35.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a91ef5700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 0x7f4a8c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492590548 0 0) 0x7f4a8c0cb340 con 0x7f4a8c15ac20 2023-10-16T09:56:35.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a780035b0 con 0x7f4a8c15ac20 2023-10-16T09:56:35.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2736310484 0 0) 0x7f4a780035b0 con 0x7f4a8c15ac20 2023-10-16T09:56:35.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 msgr2=0x7f4a8c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:35.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 0x7f4a8c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 >> v1:172.21.15.190:6789/0 conn(0x7f4a8c0ceb60 legacy=0x7f4a8c0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:35.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c15b690 con 0x7f4a8c15ac20 2023-10-16T09:56:35.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4a88001670 con 0x7f4a8c15ac20 2023-10-16T09:56:35.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 >> v1:172.21.15.181:6789/0 conn(0x7f4a8c15ac20 legacy=0x7f4a8c15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:35.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 0x7f4a78003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a78003f70 0x7f4a78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a78004d90 0x7f4a78005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c15b3b0 con 0x7f4a78004d90 2023-10-16T09:56:35.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7800ba70 con 0x7f4a78003f70 2023-10-16T09:56:35.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7800bc30 con 0x7f4a8c158360 2023-10-16T09:56:35.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a926f6700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a78004d90 0x7f4a78005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a91ef5700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 0x7f4a78003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a916f4700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a78003f70 0x7f4a78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a926f6700 1 -- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 msgr2=0x7f4a78003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:35.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a926f6700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c158360 0x7f4a78003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a926f6700 1 -- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a78003f70 msgr2=0x7f4a78004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:35.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a926f6700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a78003f70 0x7f4a78004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.403+0000 7f4a926f6700 1 -- 172.21.15.181:0/869561585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780035b0 con 0x7f4a78004d90 2023-10-16T09:56:35.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a926f6700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a78004d90 0x7f4a78005210 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c290 tx=0x7f4a78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a90b9fabd307e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:35.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84001e70 con 0x7f4a78004d90 2023-10-16T09:56:35.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8400ce50 con 0x7f4a78004d90 2023-10-16T09:56:35.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a90ef3700 1 -- 172.21.15.181:0/869561585 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84012660 con 0x7f4a78004d90 2023-10-16T09:56:35.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a94159700 1 -- 172.21.15.181:0/869561585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a78004d90 msgr2=0x7f4a78005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:35.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a94159700 1 --2- 172.21.15.181:0/869561585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a78004d90 0x7f4a78005210 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c290 tx=0x7f4a78005880 comp rx=0 tx=0).stop 2023-10-16T09:56:35.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a926f6700 1 -- 172.21.15.181:0/869561585 reap_dead start 2023-10-16T09:56:35.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a94159700 1 -- 172.21.15.181:0/869561585 shutdown_connections 2023-10-16T09:56:35.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a94159700 1 -- 172.21.15.181:0/869561585 >> 172.21.15.181:0/869561585 conn(0x7f4a8c0cc730 msgr2=0x7f4a8c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:35.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a94159700 1 -- 172.21.15.181:0/869561585 shutdown_connections 2023-10-16T09:56:35.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.404+0000 7f4a94159700 1 -- 172.21.15.181:0/869561585 wait complete. 2023-10-16T09:56:35.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 Processor -- start 2023-10-16T09:56:35.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 -- start start 2023-10-16T09:56:35.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a8c0c6280 0x7f4a8c15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c0ad290 0x7f4a8c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a8c0c6660 0x7f4a8c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0ad790 con 0x7f4a8c0c6660 2023-10-16T09:56:35.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0ada70 con 0x7f4a8c0c6280 2023-10-16T09:56:35.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.406+0000 7f4a94159700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0add50 con 0x7f4a8c0ad290 2023-10-16T09:56:35.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a8c0c6660 0x7f4a8c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a8c0c6660 0x7f4a8c06e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39296/0 (socket says 172.21.15.181:39296) 2023-10-16T09:56:35.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 -- 172.21.15.181:0/1730790596 learned_addr learned my addr 172.21.15.181:0/1730790596 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:35.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a916f4700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c0ad290 0x7f4a8c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a91ef5700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a8c0c6280 0x7f4a8c15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 -- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c0ad290 msgr2=0x7f4a8c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:35.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c0ad290 0x7f4a8c06e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 -- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a8c0c6280 msgr2=0x7f4a8c15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:35.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a8c0c6280 0x7f4a8c15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 -- 172.21.15.181:0/1730790596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0ae030 con 0x7f4a8c0c6660 2023-10-16T09:56:35.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a926f6700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a8c0c6660 0x7f4a8c06e9f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a84013000 tx=0x7f4a840015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:35.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a827fc700 1 -- 172.21.15.181:0/1730790596 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84003c80 con 0x7f4a8c0c6660 2023-10-16T09:56:35.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a827fc700 1 -- 172.21.15.181:0/1730790596 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a84020070 con 0x7f4a8c0c6660 2023-10-16T09:56:35.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c0ae310 con 0x7f4a8c0c6660 2023-10-16T09:56:35.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a827fc700 1 -- 172.21.15.181:0/1730790596 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84005300 con 0x7f4a8c0c6660 2023-10-16T09:56:35.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.407+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c0ae830 con 0x7f4a8c0c6660 2023-10-16T09:56:35.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.408+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8c0ae500 con 0x7f4a8c0c6660 2023-10-16T09:56:35.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.409+0000 7f4a827fc700 1 -- 172.21.15.181:0/1730790596 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4a84022030 con 0x7f4a8c0c6660 2023-10-16T09:56:35.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.409+0000 7f4a827fc700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4a7006b100 0x7f4a7006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:35.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.409+0000 7f4a91ef5700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4a7006b100 0x7f4a7006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:35.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.409+0000 7f4a827fc700 1 -- 172.21.15.181:0/1730790596 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f4a8408ac80 con 0x7f4a8c0c6660 2023-10-16T09:56:35.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.409+0000 7f4a91ef5700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4a7006b100 0x7f4a7006d5c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009b80 tx=0x7f4a7c008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:35.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.414+0000 7f4a827fc700 1 -- 172.21.15.181:0/1730790596 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a84055850 con 0x7f4a8c0c6660 2023-10-16T09:56:35.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.634+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a8c06f880 con 0x7f4a7006b100 2023-10-16T09:56:35.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.716+0000 7f4a827fc700 1 -- 172.21.15.181:0/1730790596 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f4a7c0097d0 con 0x7f4a7006b100 2023-10-16T09:56:35.718 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-16T09:56:35.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4a7006b100 msgr2=0x7f4a7006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:35.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4a7006b100 0x7f4a7006d5c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009b80 tx=0x7f4a7c008040 comp rx=0 tx=0).stop 2023-10-16T09:56:35.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a8c0c6660 msgr2=0x7f4a8c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:35.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a8c0c6660 0x7f4a8c06e9f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a84013000 tx=0x7f4a840015b0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 shutdown_connections 2023-10-16T09:56:35.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f4a7006b100 0x7f4a7006d5c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a8c0c6660 0x7f4a8c06e9f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a8c0c6280 0x7f4a8c15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 --2- 172.21.15.181:0/1730790596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a8c0ad290 0x7f4a8c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:35.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 >> 172.21.15.181:0/1730790596 conn(0x7f4a8c0cc730 msgr2=0x7f4a8c0af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:35.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 shutdown_connections 2023-10-16T09:56:35.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:35.718+0000 7f4a94159700 1 -- 172.21.15.181:0/1730790596 wait complete. 2023-10-16T09:56:35.732 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-16T09:56:35.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.733 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-16T09:56:35.751 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:35.751 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:35.751 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:35.751 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:35.752 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:35.752 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:35.752 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:35.752 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:35.752 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:35.753 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:35.753 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:35.753 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:35.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.754 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:35.814 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:35.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.815 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:35.876 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:35.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.877 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:35.939 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:35.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:35.940 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:36.002 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:36.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.003 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:36.068 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:36.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.069 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:36.131 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:36.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.131 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:36.195 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:36.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.195 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:36.261 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:36.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.262 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:36.324 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:36.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.325 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:36.387 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:36.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.388 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:36.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:36.450 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-16T09:56:36.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1m 2023-10-16T09:56:36.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.672+0000 7f201f34d700 1 Processor -- start 2023-10-16T09:56:36.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.672+0000 7f201f34d700 1 -- start start 2023-10-16T09:56:36.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201f34d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20180b2010 0x7f20180cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:36.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201f34d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:36.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201f34d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20180ce1f0 con 0x7f2018071be0 2023-10-16T09:56:36.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201f34d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2018072090 con 0x7f20180b2010 2023-10-16T09:56:36.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201f34d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20180ccda0 con 0x7f20180ce680 2023-10-16T09:56:36.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20180b2010 0x7f20180cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:36.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20180b2010 0x7f20180cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39300/0 (socket says 172.21.15.181:39300) 2023-10-16T09:56:36.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 -- 172.21.15.181:0/2364417654 learned_addr learned my addr 172.21.15.181:0/2364417654 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:36.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d8ea700 1 --2- 172.21.15.181:0/2364417654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:36.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 -- 172.21.15.181:0/2364417654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 msgr2=0x7f20180ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:36.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 --2- 172.21.15.181:0/2364417654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 -- 172.21.15.181:0/2364417654 >> v1:172.21.15.190:6789/0 conn(0x7f2018071be0 legacy=0x7f20180b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:36.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 -- 172.21.15.181:0/2364417654 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180727e0 con 0x7f20180b2010 2023-10-16T09:56:36.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.673+0000 7f201d0e9700 1 --2- 172.21.15.181:0/2364417654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20180b2010 0x7f20180cc6e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f20080098f0 tx=0x7f2008009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee39bdbe68d02ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:36.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f200ffff700 1 -- 172.21.15.181:0/2364417654 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008011830 con 0x7f20180b2010 2023-10-16T09:56:36.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f200ffff700 1 -- 172.21.15.181:0/2364417654 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008011990 con 0x7f20180b2010 2023-10-16T09:56:36.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f200ffff700 1 -- 172.21.15.181:0/2364417654 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008011d20 con 0x7f20180b2010 2023-10-16T09:56:36.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 -- 172.21.15.181:0/2364417654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20180b2010 msgr2=0x7f20180cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:36.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 --2- 172.21.15.181:0/2364417654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20180b2010 0x7f20180cc6e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f20080098f0 tx=0x7f2008009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:36.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 -- 172.21.15.181:0/2364417654 shutdown_connections 2023-10-16T09:56:36.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 --2- 172.21.15.181:0/2364417654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 --2- 172.21.15.181:0/2364417654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20180b2010 0x7f20180cc6e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 -- 172.21.15.181:0/2364417654 >> 172.21.15.181:0/2364417654 conn(0x7f20180cd010 msgr2=0x7f20180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:36.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 -- 172.21.15.181:0/2364417654 shutdown_connections 2023-10-16T09:56:36.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.674+0000 7f201f34d700 1 -- 172.21.15.181:0/2364417654 wait complete. 2023-10-16T09:56:36.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.675+0000 7f201f34d700 1 Processor -- start 2023-10-16T09:56:36.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.675+0000 7f201f34d700 1 -- start start 2023-10-16T09:56:36.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201f34d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2018071be0 0x7f20180b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:36.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201f34d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20180b2010 0x7f2018147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:36.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201f34d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:36.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201f34d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20181f3450 con 0x7f2018071be0 2023-10-16T09:56:36.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201f34d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b1c10 con 0x7f20180b2010 2023-10-16T09:56:36.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201f34d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20180b1090 con 0x7f20180ce680 2023-10-16T09:56:36.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201d0e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2018071be0 0x7f20180b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:36.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201d0e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2018071be0 0x7f20180b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39308/0 (socket says 172.21.15.181:39308) 2023-10-16T09:56:36.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201d0e9700 1 -- 172.21.15.181:0/3598796237 learned_addr learned my addr 172.21.15.181:0/3598796237 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:36.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201c8e8700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20180b2010 0x7f2018147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:36.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.676+0000 7f201d8ea700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:36.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201d0e9700 1 -- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 msgr2=0x7f20180b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:36.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201d0e9700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201d0e9700 1 -- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20180b2010 msgr2=0x7f2018147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:36.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201d0e9700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20180b2010 0x7f2018147300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201d0e9700 1 -- 172.21.15.181:0/3598796237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2008009550 con 0x7f2018071be0 2023-10-16T09:56:36.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201d0e9700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2018071be0 0x7f20180b0720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2008000c00 tx=0x7f2008004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:36.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f200dffb700 1 -- 172.21.15.181:0/3598796237 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008004480 con 0x7f2018071be0 2023-10-16T09:56:36.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f200dffb700 1 -- 172.21.15.181:0/3598796237 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200801f070 con 0x7f2018071be0 2023-10-16T09:56:36.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20180b1370 con 0x7f2018071be0 2023-10-16T09:56:36.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f200dffb700 1 -- 172.21.15.181:0/3598796237 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080038c0 con 0x7f2018071be0 2023-10-16T09:56:36.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.677+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20180b1930 con 0x7f2018071be0 2023-10-16T09:56:36.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.678+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20180b6510 con 0x7f2018071be0 2023-10-16T09:56:36.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.678+0000 7f200dffb700 1 -- 172.21.15.181:0/3598796237 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f2008021030 con 0x7f2018071be0 2023-10-16T09:56:36.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.679+0000 7f200dffb700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1ffc06b100 0x7f1ffc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:36.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.679+0000 7f200dffb700 1 -- 172.21.15.181:0/3598796237 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f200808b150 con 0x7f2018071be0 2023-10-16T09:56:36.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.679+0000 7f201c8e8700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1ffc06b100 0x7f1ffc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:36.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.679+0000 7f201c8e8700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1ffc06b100 0x7f1ffc06d5c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20180b06b0 tx=0x7f2014009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:36.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.683+0000 7f200dffb700 1 -- 172.21.15.181:0/3598796237 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2008055c90 con 0x7f2018071be0 2023-10-16T09:56:36.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.905+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2018072da0 con 0x7f1ffc06b100 2023-10-16T09:56:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.988+0000 7f200dffb700 1 -- 172.21.15.181:0/3598796237 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f20140014c0 con 0x7f1ffc06b100 2023-10-16T09:56:36.989 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-16T09:56:36.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1ffc06b100 msgr2=0x7f1ffc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:36.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1ffc06b100 0x7f1ffc06d5c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20180b06b0 tx=0x7f2014009040 comp rx=0 tx=0).stop 2023-10-16T09:56:36.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2018071be0 msgr2=0x7f20180b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:36.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2018071be0 0x7f20180b0720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2008000c00 tx=0x7f2008004750 comp rx=0 tx=0).stop 2023-10-16T09:56:36.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 shutdown_connections 2023-10-16T09:56:36.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20180ce680 0x7f20180b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f1ffc06b100 0x7f1ffc06d5c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2018071be0 0x7f20180b0720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 --2- 172.21.15.181:0/3598796237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20180b2010 0x7f2018147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 >> 172.21.15.181:0/3598796237 conn(0x7f20180cd010 msgr2=0x7f20180b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 shutdown_connections 2023-10-16T09:56:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:36.990+0000 7f201f34d700 1 -- 172.21.15.181:0/3598796237 wait complete. 2023-10-16T09:56:37.003 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-16T09:56:37.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.004 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-16T09:56:37.023 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:37.023 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:37.023 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:37.024 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:37.024 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:37.024 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:37.024 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:37.024 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:37.025 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:37.025 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:37.025 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:37.026 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:37.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.027 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:37.104 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:37.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.105 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:37.167 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:37.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.168 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:37.229 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:37.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.230 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:37.304 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:37.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.305 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:37.367 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:37.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.367 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:37.429 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:37.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.430 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:37.492 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:37.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.493 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:37.554 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:37.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.555 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:37.617 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:37.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.617 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:37.679 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:37.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.680 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:37.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:37.742 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-16T09:56:37.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1m 2023-10-16T09:56:37.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.963+0000 7fee15878700 1 Processor -- start 2023-10-16T09:56:37.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.964+0000 7fee15878700 1 -- start start 2023-10-16T09:56:37.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.964+0000 7fee15878700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee10071460 0x7fee1015db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.964+0000 7fee15878700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1015ea60 con 0x7fee10159e50 2023-10-16T09:56:37.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.964+0000 7fee15878700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1015e1c0 con 0x7fee100cc5e0 2023-10-16T09:56:37.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.964+0000 7fee15878700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1015e4a0 con 0x7fee10071460 2023-10-16T09:56:37.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fee10159e50 0x7fee100cbed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50392/0 (socket says 172.21.15.181:50392) 2023-10-16T09:56:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0effd700 1 -- 172.21.15.181:0/183862378 learned_addr learned my addr 172.21.15.181:0/183862378 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985659444 0 0) 0x7fee1015ea60 con 0x7fee10159e50 2023-10-16T09:56:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0f7fe700 1 --2- 172.21.15.181:0/183862378 >> v2:172.21.15.190:3300/0 conn(0x7fee10071460 0x7fee1015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf80046c0 con 0x7fee10159e50 2023-10-16T09:56:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2053699074 0 0) 0x7fedf80046c0 con 0x7fee10159e50 2023-10-16T09:56:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 >> v2:172.21.15.190:3300/0 conn(0x7fee10071460 msgr2=0x7fee1015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 --2- 172.21.15.181:0/183862378 >> v2:172.21.15.190:3300/0 conn(0x7fee10071460 0x7fee1015db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 >> v1:172.21.15.190:6789/0 conn(0x7fee100cc5e0 legacy=0x7fee100ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1015e780 con 0x7fee10159e50 2023-10-16T09:56:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fee04001670 con 0x7fee10159e50 2023-10-16T09:56:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 >> v1:172.21.15.181:6789/0 conn(0x7fee10159e50 legacy=0x7fee100cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf8004b10 0x7fedf8004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf8005960 0x7fedf8005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fedf800cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1015e4a0 con 0x7fedf8006320 2023-10-16T09:56:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf800d2c0 con 0x7fedf8005960 2023-10-16T09:56:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf800d480 con 0x7fedf8004b10 2023-10-16T09:56:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0f7fe700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fedf800cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0effd700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf8004b10 0x7fedf8004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.965+0000 7fee0e7fc700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf8005960 0x7fedf8005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0f7fe700 1 -- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf8004b10 msgr2=0x7fedf8004f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0f7fe700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf8004b10 0x7fedf8004f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0f7fe700 1 -- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf8005960 msgr2=0x7fedf8005de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0f7fe700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf8005960 0x7fedf8005de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0f7fe700 1 -- 172.21.15.181:0/183862378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80046c0 con 0x7fedf8006320 2023-10-16T09:56:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0f7fe700 1 -- 172.21.15.181:0/183862378 reap_dead start 2023-10-16T09:56:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0f7fe700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fedf800cb20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fee08009fc0 tx=0x7fedf8006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1cb0c4cec2539f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08003c20 con 0x7fedf8006320 2023-10-16T09:56:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0800f040 con 0x7fedf8006320 2023-10-16T09:56:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee0dffb700 1 -- 172.21.15.181:0/183862378 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0800ea30 con 0x7fedf8006320 2023-10-16T09:56:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee15878700 1 -- 172.21.15.181:0/183862378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 msgr2=0x7fedf800cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee15878700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fedf800cb20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fee08009fc0 tx=0x7fedf8006990 comp rx=0 tx=0).stop 2023-10-16T09:56:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee15878700 1 -- 172.21.15.181:0/183862378 shutdown_connections 2023-10-16T09:56:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee15878700 1 --2- 172.21.15.181:0/183862378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fedf800cb20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee15878700 1 -- 172.21.15.181:0/183862378 >> 172.21.15.181:0/183862378 conn(0x7fee1006aed0 msgr2=0x7fee1006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee15878700 1 -- 172.21.15.181:0/183862378 shutdown_connections 2023-10-16T09:56:37.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.966+0000 7fee15878700 1 -- 172.21.15.181:0/183862378 wait complete. 2023-10-16T09:56:37.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.968+0000 7fee15878700 1 Processor -- start 2023-10-16T09:56:37.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.968+0000 7fee15878700 1 -- start start 2023-10-16T09:56:37.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.968+0000 7fee15878700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fee10152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.968+0000 7fee15878700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee100b2b20 0x7fee10150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.968+0000 7fee15878700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee10157a90 0x7fee10157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee15878700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10158480 con 0x7fedf8006320 2023-10-16T09:56:37.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee15878700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100b3290 con 0x7fee100b2b20 2023-10-16T09:56:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee15878700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee100b3570 con 0x7fee10157a90 2023-10-16T09:56:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fee10152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fee10152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39324/0 (socket says 172.21.15.181:39324) 2023-10-16T09:56:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 -- 172.21.15.181:0/1572711112 learned_addr learned my addr 172.21.15.181:0/1572711112 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0f7fe700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee10157a90 0x7fee10157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0e7fc700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee100b2b20 0x7fee10150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 -- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee10157a90 msgr2=0x7fee10157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee10157a90 0x7fee10157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 -- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee100b2b20 msgr2=0x7fee10150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee100b2b20 0x7fee10150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 -- 172.21.15.181:0/1572711112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100b3850 con 0x7fedf8006320 2023-10-16T09:56:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0f7fe700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee10157a90 0x7fee10157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0e7fc700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee100b2b20 0x7fee10150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee0effd700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fee10152720 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fee040072c0 tx=0x7fee04007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fedf7fff700 1 -- 172.21.15.181:0/1572711112 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee040077a0 con 0x7fedf8006320 2023-10-16T09:56:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fedf7fff700 1 -- 172.21.15.181:0/1572711112 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0400f8d0 con 0x7fedf8006320 2023-10-16T09:56:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fedf7fff700 1 -- 172.21.15.181:0/1572711112 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0400fc80 con 0x7fedf8006320 2023-10-16T09:56:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee100b3b30 con 0x7fedf8006320 2023-10-16T09:56:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.969+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee10150f50 con 0x7fedf8006320 2023-10-16T09:56:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.970+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee100cde50 con 0x7fedf8006320 2023-10-16T09:56:37.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.971+0000 7fedf7fff700 1 -- 172.21.15.181:0/1572711112 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fee0401d460 con 0x7fedf8006320 2023-10-16T09:56:37.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.971+0000 7fedf7fff700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fedfc06b100 0x7fedfc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:37.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.971+0000 7fedf7fff700 1 -- 172.21.15.181:0/1572711112 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6903+0+0 (secure 0 0 0) 0x7fee04089e90 con 0x7fedf8006320 2023-10-16T09:56:37.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.971+0000 7fee0e7fc700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fedfc06b100 0x7fedfc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:37.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.971+0000 7fee0e7fc700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fedfc06b100 0x7fedfc06d5c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fee10154a70 tx=0x7fee00009040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:37.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:37.975+0000 7fedf7fff700 1 -- 172.21.15.181:0/1572711112 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fee100cde50 con 0x7fedf8006320 2023-10-16T09:56:38.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.211+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fee100cde50 con 0x7fedfc06b100 2023-10-16T09:56:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.291+0000 7fedf7fff700 1 -- 172.21.15.181:0/1572711112 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fee100cde50 con 0x7fedfc06b100 2023-10-16T09:56:38.293 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-16T09:56:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fedfc06b100 msgr2=0x7fedfc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fedfc06b100 0x7fedfc06d5c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fee10154a70 tx=0x7fee00009040 comp rx=0 tx=0).stop 2023-10-16T09:56:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 msgr2=0x7fee10152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fee10152720 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fee040072c0 tx=0x7fee04007680 comp rx=0 tx=0).stop 2023-10-16T09:56:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 shutdown_connections 2023-10-16T09:56:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee10157a90 0x7fee10157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fedfc06b100 0x7fedfc06d5c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf8006320 0x7fee10152720 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 --2- 172.21.15.181:0/1572711112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee100b2b20 0x7fee10150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 >> 172.21.15.181:0/1572711112 conn(0x7fee1006aed0 msgr2=0x7fee101538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 shutdown_connections 2023-10-16T09:56:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:38.294+0000 7fee15878700 1 -- 172.21.15.181:0/1572711112 wait complete. 2023-10-16T09:56:38.306 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-16T09:56:38.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.307 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-16T09:56:38.325 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:38.325 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:38.325 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:38.326 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:38.326 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:38.326 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:38.326 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:38.326 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:38.327 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:38.327 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:38.327 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:38.327 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:38.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.328 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:38.406 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:38.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.407 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:38.468 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:38.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.469 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:38.534 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:38.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.535 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:38.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:38.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.597 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:38.660 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:38.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.660 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:38.722 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:38.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.723 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:38.786 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:38.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.787 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:38.848 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:38.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.849 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:38.911 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:38.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:38.912 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:39.016 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:39.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.017 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:39.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.041 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-16T09:56:39.101 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1m 2023-10-16T09:56:39.178 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:39.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:56:39.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.267+0000 7fdb73dc8700 1 Processor -- start 2023-10-16T09:56:39.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.268+0000 7fdb73dc8700 1 -- start start 2023-10-16T09:56:39.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.268+0000 7fdb73dc8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c0cdd80 0x7fdb6c0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:39.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.268+0000 7fdb73dc8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdb6c06b6f0 0x7fdb6c06bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:39.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.268+0000 7fdb73dc8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6c15dfb0 con 0x7fdb6c15dbd0 2023-10-16T09:56:39.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.268+0000 7fdb73dc8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6c0ce740 con 0x7fdb6c06b6f0 2023-10-16T09:56:39.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.268+0000 7fdb73dc8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0cd3a0 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c0cdd80 0x7fdb6c0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:39.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb72365700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdb6c06b6f0 0x7fdb6c06bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:39.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c0cdd80 0x7fdb6c0ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:36360/0 (socket says 172.21.15.181:36360) 2023-10-16T09:56:39.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71b64700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fdb6c15dbd0 0x7fdb6c0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:34352/0 (socket says 172.21.15.181:34352) 2023-10-16T09:56:39.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 -- 172.21.15.181:0/2368366958 learned_addr learned my addr 172.21.15.181:0/2368366958 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:39.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 -- 172.21.15.181:0/2368366958 >> v2:172.21.15.190:3300/0 conn(0x7fdb6c06b6f0 msgr2=0x7fdb6c06bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:39.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 --2- 172.21.15.181:0/2368366958 >> v2:172.21.15.190:3300/0 conn(0x7fdb6c06b6f0 0x7fdb6c06bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 -- 172.21.15.181:0/2368366958 >> v1:172.21.15.190:6789/0 conn(0x7fdb6c15dbd0 legacy=0x7fdb6c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:39.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 -- 172.21.15.181:0/2368366958 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c0cd830 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb70b62700 1 -- 172.21.15.181:0/2368366958 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184332636 0 0) 0x7fdb5c001110 con 0x7fdb6c15dbd0 2023-10-16T09:56:39.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb72365700 1 --2- 172.21.15.181:0/2368366958 >> v2:172.21.15.190:3300/0 conn(0x7fdb6c06b6f0 0x7fdb6c06bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:39.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb71363700 1 --2- 172.21.15.181:0/2368366958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c0cdd80 0x7fdb6c0ce200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb680098f0 tx=0x7fdb68009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87a333fbb8f08dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:39.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.269+0000 7fdb70b62700 1 -- 172.21.15.181:0/2368366958 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68011830 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb70b62700 1 -- 172.21.15.181:0/2368366958 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb68011990 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb70b62700 1 -- 172.21.15.181:0/2368366958 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68011d20 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 -- 172.21.15.181:0/2368366958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c0cdd80 msgr2=0x7fdb6c0ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:39.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/2368366958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c0cdd80 0x7fdb6c0ce200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb680098f0 tx=0x7fdb68009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:39.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 -- 172.21.15.181:0/2368366958 shutdown_connections 2023-10-16T09:56:39.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/2368366958 >> v2:172.21.15.190:3300/0 conn(0x7fdb6c06b6f0 0x7fdb6c06bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/2368366958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c0cdd80 0x7fdb6c0ce200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 -- 172.21.15.181:0/2368366958 >> 172.21.15.181:0/2368366958 conn(0x7fdb6c09a930 msgr2=0x7fdb6c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:39.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 -- 172.21.15.181:0/2368366958 shutdown_connections 2023-10-16T09:56:39.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.270+0000 7fdb73dc8700 1 -- 172.21.15.181:0/2368366958 wait complete. 2023-10-16T09:56:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.271+0000 7fdb73dc8700 1 Processor -- start 2023-10-16T09:56:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.271+0000 7fdb73dc8700 1 -- start start 2023-10-16T09:56:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb73dc8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c06b6f0 0x7fdb6c157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb73dc8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6c0cdd80 0x7fdb6c0b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:39.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb73dc8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c15dbd0 0x7fdb6c15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:39.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb73dc8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c158520 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb73dc8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0ca510 con 0x7fdb6c15dbd0 2023-10-16T09:56:39.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb73dc8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c0ca7f0 con 0x7fdb6c06b6f0 2023-10-16T09:56:39.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71363700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6c0cdd80 0x7fdb6c0b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:39.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71363700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6c0cdd80 0x7fdb6c0b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39340/0 (socket says 172.21.15.181:39340) 2023-10-16T09:56:39.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71363700 1 -- 172.21.15.181:0/1856392417 learned_addr learned my addr 172.21.15.181:0/1856392417 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:39.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71b64700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c06b6f0 0x7fdb6c157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:39.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb72365700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c15dbd0 0x7fdb6c15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:39.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71363700 1 -- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c06b6f0 msgr2=0x7fdb6c157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:39.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71363700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c06b6f0 0x7fdb6c157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71363700 1 -- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c15dbd0 msgr2=0x7fdb6c15fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:39.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.272+0000 7fdb71363700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c15dbd0 0x7fdb6c15fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.273+0000 7fdb71363700 1 -- 172.21.15.181:0/1856392417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb64009a70 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.273+0000 7fdb71363700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6c0cdd80 0x7fdb6c0b6800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009c40 tx=0x7fdb68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:39.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.273+0000 7fdb627fc700 1 -- 172.21.15.181:0/1856392417 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68004450 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.273+0000 7fdb627fc700 1 -- 172.21.15.181:0/1856392417 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6801e070 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.273+0000 7fdb627fc700 1 -- 172.21.15.181:0/1856392417 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb680038e0 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.273+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb68009550 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.273+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c0cadb0 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.274+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6c067d20 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.274+0000 7fdb627fc700 1 -- 172.21.15.181:0/1856392417 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fdb68003a40 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.274+0000 7fdb627fc700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdb5006b100 0x7fdb5006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:39.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.275+0000 7fdb627fc700 1 -- 172.21.15.181:0/1856392417 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7235+0+0 (secure 0 0 0) 0x7fdb6808ae80 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.275+0000 7fdb71b64700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdb5006b100 0x7fdb5006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:39.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.275+0000 7fdb71b64700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdb5006b100 0x7fdb5006d5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c000f60 tx=0x7fdb5c007c20 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:39.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.279+0000 7fdb627fc700 1 -- 172.21.15.181:0/1856392417 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdb68055620 con 0x7fdb6c0cdd80 2023-10-16T09:56:39.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.499+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb6c0cc790 con 0x7fdb5006b100 2023-10-16T09:56:39.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.623+0000 7fdb627fc700 1 -- 172.21.15.181:0/1856392417 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fdb5c0049b0 con 0x7fdb5006b100 2023-10-16T09:56:39.624 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-16T09:56:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.625+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdb5006b100 msgr2=0x7fdb5006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.625+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdb5006b100 0x7fdb5006d5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c000f60 tx=0x7fdb5c007c20 comp rx=0 tx=0).stop 2023-10-16T09:56:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.625+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6c0cdd80 msgr2=0x7fdb6c0b6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.625+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6c0cdd80 0x7fdb6c0b6800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009c40 tx=0x7fdb68004750 comp rx=0 tx=0).stop 2023-10-16T09:56:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 shutdown_connections 2023-10-16T09:56:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb6c06b6f0 0x7fdb6c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdb5006b100 0x7fdb5006d5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c15dbd0 0x7fdb6c15fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 --2- 172.21.15.181:0/1856392417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6c0cdd80 0x7fdb6c0b6800 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:39.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 >> 172.21.15.181:0/1856392417 conn(0x7fdb6c09a930 msgr2=0x7fdb6c1615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:39.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 shutdown_connections 2023-10-16T09:56:39.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:39.626+0000 7fdb73dc8700 1 -- 172.21.15.181:0/1856392417 wait complete. 2023-10-16T09:56:39.638 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-16T09:56:39.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.639 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-16T09:56:39.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:39.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:39.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:39.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:39.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:39.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:39.698 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:39.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.699 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:39.759 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:39.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.760 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:39.823 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:39.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.824 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:39.889 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:39.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.890 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:39.970 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:39.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:39.971 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:40.033 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:40.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:40.034 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:40.096 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:40.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:40.097 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:40.160 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:40.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:40.161 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:40.223 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:40.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:40.242 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:40.286 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:40.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:40.287 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:40.349 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:40.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:40.350 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:40.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:40.414 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-16T09:56:40.474 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1m 2023-10-16T09:56:40.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.641+0000 7f62dec93700 1 Processor -- start 2023-10-16T09:56:40.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.641+0000 7f62dec93700 1 -- start start 2023-10-16T09:56:40.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.641+0000 7f62dec93700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d01d0 0x7f62d80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:40.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.641+0000 7f62dec93700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62d80d1500 0x7f62d80d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:40.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.641+0000 7f62dec93700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80cf580 con 0x7f62d815ea40 2023-10-16T09:56:40.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.641+0000 7f62dec93700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80d4290 con 0x7f62d80d1500 2023-10-16T09:56:40.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.641+0000 7f62dec93700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80ce970 con 0x7f62d80d01d0 2023-10-16T09:56:40.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dd230700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62d80d1500 0x7f62d80d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:40.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dca2f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d01d0 0x7f62d80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62d7fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f62d815ea40 0x7f62d80d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:34366/0 (socket says 172.21.15.181:34366) 2023-10-16T09:56:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62d7fff700 1 -- 172.21.15.181:0/556215861 learned_addr learned my addr 172.21.15.181:0/556215861 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dca2f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d01d0 0x7f62d80ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:36372/0 (socket says 172.21.15.181:36372) 2023-10-16T09:56:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62d77fe700 1 -- 172.21.15.181:0/556215861 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721602093 0 0) 0x7f62d80cf580 con 0x7f62d815ea40 2023-10-16T09:56:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dca2f700 1 -- 172.21.15.181:0/556215861 >> v2:172.21.15.190:3300/0 conn(0x7f62d80d1500 msgr2=0x7f62d80d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dca2f700 1 --2- 172.21.15.181:0/556215861 >> v2:172.21.15.190:3300/0 conn(0x7f62d80d1500 0x7f62d80d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dca2f700 1 -- 172.21.15.181:0/556215861 >> v1:172.21.15.190:6789/0 conn(0x7f62d815ea40 legacy=0x7f62d80d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dca2f700 1 -- 172.21.15.181:0/556215861 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80cec50 con 0x7f62d80d01d0 2023-10-16T09:56:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.642+0000 7f62dca2f700 1 --2- 172.21.15.181:0/556215861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d01d0 0x7f62d80ce2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009870 tx=0x7f62cc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82e1daf6e857c08c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62d77fe700 1 -- 172.21.15.181:0/556215861 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc011830 con 0x7f62d80d01d0 2023-10-16T09:56:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62d77fe700 1 -- 172.21.15.181:0/556215861 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc011990 con 0x7f62d80d01d0 2023-10-16T09:56:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62d77fe700 1 -- 172.21.15.181:0/556215861 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc011d20 con 0x7f62d80d01d0 2023-10-16T09:56:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 -- 172.21.15.181:0/556215861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d01d0 msgr2=0x7f62d80ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 --2- 172.21.15.181:0/556215861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d01d0 0x7f62d80ce2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009870 tx=0x7f62cc009b80 comp rx=0 tx=0).stop 2023-10-16T09:56:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 -- 172.21.15.181:0/556215861 shutdown_connections 2023-10-16T09:56:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 --2- 172.21.15.181:0/556215861 >> v2:172.21.15.190:3300/0 conn(0x7f62d80d1500 0x7f62d80d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 --2- 172.21.15.181:0/556215861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d01d0 0x7f62d80ce2d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 -- 172.21.15.181:0/556215861 >> 172.21.15.181:0/556215861 conn(0x7f62d809cb60 msgr2=0x7f62d80d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 -- 172.21.15.181:0/556215861 shutdown_connections 2023-10-16T09:56:40.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.643+0000 7f62dec93700 1 -- 172.21.15.181:0/556215861 wait complete. 2023-10-16T09:56:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.644+0000 7f62dec93700 1 Processor -- start 2023-10-16T09:56:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.644+0000 7f62dec93700 1 -- start start 2023-10-16T09:56:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dec93700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d80d01d0 0x7f62d80c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dec93700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d1500 0x7f62d80cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dec93700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62d815ea40 0x7f62d80c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dec93700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80cb600 con 0x7f62d815ea40 2023-10-16T09:56:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dec93700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80c7fb0 con 0x7f62d80d01d0 2023-10-16T09:56:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dec93700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80c8250 con 0x7f62d80d1500 2023-10-16T09:56:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dd230700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62d815ea40 0x7f62d80c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:40.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62d7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d1500 0x7f62d80cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:40.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dd230700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62d815ea40 0x7f62d80c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39352/0 (socket says 172.21.15.181:39352) 2023-10-16T09:56:40.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dd230700 1 -- 172.21.15.181:0/2373586834 learned_addr learned my addr 172.21.15.181:0/2373586834 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:40.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.645+0000 7f62dca2f700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d80d01d0 0x7f62d80c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:40.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dd230700 1 -- 172.21.15.181:0/2373586834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d1500 msgr2=0x7f62d80cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:40.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dd230700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d1500 0x7f62d80cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dd230700 1 -- 172.21.15.181:0/2373586834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d80d01d0 msgr2=0x7f62d80c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:40.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dd230700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d80d01d0 0x7f62d80c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dd230700 1 -- 172.21.15.181:0/2373586834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0094a0 con 0x7f62d815ea40 2023-10-16T09:56:40.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dd230700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62d815ea40 0x7f62d80c7a70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f62d000efc0 tx=0x7f62d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:40.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62d57fa700 1 -- 172.21.15.181:0/2373586834 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0009ea0 con 0x7f62d815ea40 2023-10-16T09:56:40.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62d57fa700 1 -- 172.21.15.181:0/2373586834 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000f040 con 0x7f62d815ea40 2023-10-16T09:56:40.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d80c8530 con 0x7f62d815ea40 2023-10-16T09:56:40.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62d57fa700 1 -- 172.21.15.181:0/2373586834 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000ced0 con 0x7f62d815ea40 2023-10-16T09:56:40.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.646+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d80c8af0 con 0x7f62d815ea40 2023-10-16T09:56:40.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.647+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d804ebb0 con 0x7f62d815ea40 2023-10-16T09:56:40.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.647+0000 7f62d57fa700 1 -- 172.21.15.181:0/2373586834 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f62d0014480 con 0x7f62d815ea40 2023-10-16T09:56:40.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.648+0000 7f62d57fa700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f62bc06b100 0x7f62bc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:40.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.648+0000 7f62d57fa700 1 -- 172.21.15.181:0/2373586834 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6631+0+0 (secure 0 0 0) 0x7f62d008a990 con 0x7f62d815ea40 2023-10-16T09:56:40.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.648+0000 7f62dca2f700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f62bc06b100 0x7f62bc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:40.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.648+0000 7f62dca2f700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f62bc06b100 0x7f62bc06d5c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009840 tx=0x7f62cc01c040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:40.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.652+0000 7f62d57fa700 1 -- 172.21.15.181:0/2373586834 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f62d0055290 con 0x7f62d815ea40 2023-10-16T09:56:40.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.882+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 --> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f62d80cf210 con 0x7f62bc06b100 2023-10-16T09:56:40.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.985+0000 7f62d57fa700 1 -- 172.21.15.181:0/2373586834 <== mgr.5289 v2:172.21.15.181:6832/3948589065 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f62cc011b80 con 0x7f62bc06b100 2023-10-16T09:56:40.986 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-16T09:56:40.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.987+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f62bc06b100 msgr2=0x7f62bc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:40.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.987+0000 7f62dec93700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f62bc06b100 0x7f62bc06d5c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009840 tx=0x7f62cc01c040 comp rx=0 tx=0).stop 2023-10-16T09:56:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.987+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62d815ea40 msgr2=0x7f62d80c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.987+0000 7f62dec93700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62d815ea40 0x7f62d80c7a70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f62d000efc0 tx=0x7f62d000c3a0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 shutdown_connections 2023-10-16T09:56:40.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f62bc06b100 0x7f62bc06d5c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62d815ea40 0x7f62d80c7a70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62d80d01d0 0x7f62d80c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 --2- 172.21.15.181:0/2373586834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62d80d1500 0x7f62d80cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 >> 172.21.15.181:0/2373586834 conn(0x7f62d809cb60 msgr2=0x7f62d80d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 shutdown_connections 2023-10-16T09:56:40.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:40.988+0000 7f62dec93700 1 -- 172.21.15.181:0/2373586834 wait complete. 2023-10-16T09:56:41.000 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-16T09:56:41.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.001 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-16T09:56:41.018 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:41.018 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:41.019 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:41.019 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:41.019 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:41.019 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:41.019 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:41.020 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:41.020 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:41.020 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:41.020 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:41.020 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_46_00_UTC 2023-10-16T09:56:41.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.021 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_46_00_UTC) 2023-10-16T09:56:41.087 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_47_00_UTC 2023-10-16T09:56:41.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.088 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_47_00_UTC) 2023-10-16T09:56:41.157 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_48_00_UTC 2023-10-16T09:56:41.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.158 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_48_00_UTC) 2023-10-16T09:56:41.221 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_49_00_UTC 2023-10-16T09:56:41.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.222 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_49_00_UTC) 2023-10-16T09:56:41.284 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_50_00_UTC 2023-10-16T09:56:41.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.285 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_50_00_UTC) 2023-10-16T09:56:41.347 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_51_00_UTC 2023-10-16T09:56:41.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.348 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_51_00_UTC) 2023-10-16T09:56:41.410 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_52_00_UTC 2023-10-16T09:56:41.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.411 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_52_00_UTC) 2023-10-16T09:56:41.473 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_53_00_UTC 2023-10-16T09:56:41.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.474 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_53_00_UTC) 2023-10-16T09:56:41.537 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_54_00_UTC 2023-10-16T09:56:41.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.538 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_54_00_UTC) 2023-10-16T09:56:41.601 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_55_00_UTC 2023-10-16T09:56:41.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.601 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_55_00_UTC) 2023-10-16T09:56:41.664 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_56_00_UTC 2023-10-16T09:56:41.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.665 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-16-09_56_00_UTC) 2023-10-16T09:56:41.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:41.728 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-16T09:56:41.788 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T09:56:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.961+0000 7f01f4499700 1 Processor -- start 2023-10-16T09:56:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.961+0000 7f01f4499700 1 -- start start 2023-10-16T09:56:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.962+0000 7f01f4499700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f01ec159b50 0x7f01ec0b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.962+0000 7f01f4499700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0cdb50 con 0x7f01ec15dd30 2023-10-16T09:56:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.962+0000 7f01f4499700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0c9e70 con 0x7f01ec0cca60 2023-10-16T09:56:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.962+0000 7f01f4499700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec159f30 con 0x7f01ec159b50 2023-10-16T09:56:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.963+0000 7f01f1a34700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f01ec0cca60 0x7f01ec15d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:34380/0 (socket says 172.21.15.181:34380) 2023-10-16T09:56:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.963+0000 7f01f1a34700 1 -- 172.21.15.181:0/1676367425 learned_addr learned my addr 172.21.15.181:0/1676367425 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.963+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836841392 0 0) 0x7f01ec0c9e70 con 0x7f01ec0cca60 2023-10-16T09:56:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.963+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d80035b0 con 0x7f01ec0cca60 2023-10-16T09:56:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742051266 0 0) 0x7f01ec0cdb50 con 0x7f01ec15dd30 2023-10-16T09:56:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec0c9e70 con 0x7f01ec15dd30 2023-10-16T09:56:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254464603 0 0) 0x7f01ec0c9e70 con 0x7f01ec15dd30 2023-10-16T09:56:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 >> v2:172.21.15.181:3300/0 conn(0x7f01ec159b50 msgr2=0x7f01ec0b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 --2- 172.21.15.181:0/1676367425 >> v2:172.21.15.181:3300/0 conn(0x7f01ec159b50 0x7f01ec0b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 >> v1:172.21.15.190:6789/0 conn(0x7f01ec0cca60 legacy=0x7f01ec15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec15e200 con 0x7f01ec15dd30 2023-10-16T09:56:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f01e40019f0 con 0x7f01ec15dd30 2023-10-16T09:56:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 >> v1:172.21.15.181:6789/0 conn(0x7f01ec15dd30 legacy=0x7f01ec164990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8003be0 0x7f01d8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d8004850 0x7f01d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01d8005330 0x7f01d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec159f30 con 0x7f01d8004850 2023-10-16T09:56:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d800c280 con 0x7f01d8003be0 2023-10-16T09:56:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.964+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8014c50 con 0x7f01d8005330 2023-10-16T09:56:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.965+0000 7f01f2a36700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01d8005330 0x7f01d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.965+0000 7f01f1a34700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d8004850 0x7f01d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f2a36700 1 -- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8003be0 msgr2=0x7f01d8003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T09:56:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f2a36700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8003be0 0x7f01d8003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f2a36700 1 -- 172.21.15.181:0/1676367425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d8004850 msgr2=0x7f01d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f2a36700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01d8004850 0x7f01d8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f2a36700 1 -- 172.21.15.181:0/1676367425 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0c9e70 con 0x7f01d8005330 2023-10-16T09:56:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f2a36700 1 -- 172.21.15.181:0/1676367425 reap_dead start 2023-10-16T09:56:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f2a36700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01d8005330 0x7f01d800ba10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0cd170 tx=0x7f01d8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206f2a79c2df1830 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.966+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4016070 con 0x7f01d8005330 2023-10-16T09:56:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.967+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4012440 con 0x7f01d8005330 2023-10-16T09:56:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.967+0000 7f01f1233700 1 -- 172.21.15.181:0/1676367425 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40127f0 con 0x7f01d8005330 2023-10-16T09:56:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.967+0000 7f01f4499700 1 -- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01d8005330 msgr2=0x7f01d800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.967+0000 7f01f4499700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01d8005330 0x7f01d800ba10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0cd170 tx=0x7f01d8005880 comp rx=0 tx=0).stop 2023-10-16T09:56:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.968+0000 7f01f4499700 1 -- 172.21.15.181:0/1676367425 shutdown_connections 2023-10-16T09:56:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.968+0000 7f01f4499700 1 --2- 172.21.15.181:0/1676367425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01d8005330 0x7f01d800ba10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.968+0000 7f01f4499700 1 -- 172.21.15.181:0/1676367425 >> 172.21.15.181:0/1676367425 conn(0x7f01ec0cbf70 msgr2=0x7f01ec0cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.968+0000 7f01f4499700 1 -- 172.21.15.181:0/1676367425 shutdown_connections 2023-10-16T09:56:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.969+0000 7f01f4499700 1 -- 172.21.15.181:0/1676367425 wait complete. 2023-10-16T09:56:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.970+0000 7f01f4499700 1 Processor -- start 2023-10-16T09:56:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.971+0000 7f01f4499700 1 -- start start 2023-10-16T09:56:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f4499700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8005330 0x7f01ec15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f4499700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ec156e60 0x7f01ec148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f4499700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ec149250 0x7f01ec1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f4499700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b3530 con 0x7f01ec149250 2023-10-16T09:56:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f4499700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b37d0 con 0x7f01d8005330 2023-10-16T09:56:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f4499700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b3ab0 con 0x7f01ec156e60 2023-10-16T09:56:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f2a36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ec149250 0x7f01ec1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f2a36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ec149250 0x7f01ec1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39370/0 (socket says 172.21.15.181:39370) 2023-10-16T09:56:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.972+0000 7f01f2a36700 1 -- 172.21.15.181:0/2040760948 learned_addr learned my addr 172.21.15.181:0/2040760948 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2235700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8005330 0x7f01ec15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2a36700 1 -- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ec156e60 msgr2=0x7f01ec148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2a36700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ec156e60 0x7f01ec148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2a36700 1 -- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8005330 msgr2=0x7f01ec15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2a36700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8005330 0x7f01ec15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2a36700 1 -- 172.21.15.181:0/2040760948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0b3d90 con 0x7f01ec149250 2023-10-16T09:56:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2235700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8005330 0x7f01ec15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.973+0000 7f01f2a36700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ec149250 0x7f01ec1588a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01ec06dfb0 tx=0x7f01e4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.974+0000 7f01e2ffd700 1 -- 172.21.15.181:0/2040760948 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4016030 con 0x7f01ec149250 2023-10-16T09:56:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.974+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec0b4070 con 0x7f01ec149250 2023-10-16T09:56:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.974+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec148dd0 con 0x7f01ec149250 2023-10-16T09:56:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.974+0000 7f01e2ffd700 1 -- 172.21.15.181:0/2040760948 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4017040 con 0x7f01ec149250 2023-10-16T09:56:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.974+0000 7f01e2ffd700 1 -- 172.21.15.181:0/2040760948 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4005710 con 0x7f01ec149250 2023-10-16T09:56:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.975+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d40052f0 con 0x7f01ec149250 2023-10-16T09:56:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.976+0000 7f01e2ffd700 1 -- 172.21.15.181:0/2040760948 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 14) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f01e40125b0 con 0x7f01ec149250 2023-10-16T09:56:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.976+0000 7f01e2ffd700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f01d006b100 0x7f01d006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.976+0000 7f01f2235700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f01d006b100 0x7f01d006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.977+0000 7f01f2235700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f01d006b100 0x7f01d006d5c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0697f0 tx=0x7f01e8008040 comp rx=0 tx=0).ready entity=mgr.5289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.978+0000 7f01e2ffd700 1 -- 172.21.15.181:0/2040760948 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6807+0+0 (secure 0 0 0) 0x7f01e4059410 con 0x7f01ec149250 2023-10-16T09:56:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:41.981+0000 7f01e2ffd700 1 -- 172.21.15.181:0/2040760948 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f01e408f060 con 0x7f01ec149250 2023-10-16T09:56:42.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.238+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f01d4005b20 con 0x7f01ec149250 2023-10-16T09:56:42.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.946+0000 7f01e2ffd700 1 -- 172.21.15.181:0/2040760948 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v15) v1 ==== 93+0+0 (secure 0 0 0) 0x7f01e4055090 con 0x7f01ec149250 2023-10-16T09:56:42.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.954+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f01d006b100 msgr2=0x7f01d006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:42.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.954+0000 7f01f4499700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f01d006b100 0x7f01d006d5c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0697f0 tx=0x7f01e8008040 comp rx=0 tx=0).stop 2023-10-16T09:56:42.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.954+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ec149250 msgr2=0x7f01ec1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:42.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.954+0000 7f01f4499700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ec149250 0x7f01ec1588a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01ec06dfb0 tx=0x7f01e4001cb0 comp rx=0 tx=0).stop 2023-10-16T09:56:42.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 shutdown_connections 2023-10-16T09:56:42.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f01d006b100 0x7f01d006d5c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:42.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01ec149250 0x7f01ec1588a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:42.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01d8005330 0x7f01ec15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:42.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 --2- 172.21.15.181:0/2040760948 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01ec156e60 0x7f01ec148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:42.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 >> 172.21.15.181:0/2040760948 conn(0x7f01ec0cbf70 msgr2=0x7f01ec157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:42.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 shutdown_connections 2023-10-16T09:56:42.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:42.955+0000 7f01f4499700 1 -- 172.21.15.181:0/2040760948 wait complete. 2023-10-16T09:56:42.977 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:56:42.978 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:56:43.030 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:56:43.071 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:56:43.072 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:56:43.072 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:43.090 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:56:43.090 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:43.113 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:43.112+0000 7f643894d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:56:43.117 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:43.115+0000 7fd4d85a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:56:43.130 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:56:43.182 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:43.180+0000 7fd4d85a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:56:43.184 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:43.183+0000 7f643894d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:56:43.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:43.253+0000 7fd4d85a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:56:43.259 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:43.258+0000 7f643894d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:56:43.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.294+0000 7f822951f700 1 Processor -- start 2023-10-16T09:56:43.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.294+0000 7f822951f700 1 -- start start 2023-10-16T09:56:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f822951f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82240d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f822951f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82240cb0c0 0x7f822415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f822951f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822406ed00 con 0x7f822415eab0 2023-10-16T09:56:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f822951f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82240d02d0 con 0x7f82240cb0c0 2023-10-16T09:56:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f822951f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822415ad40 con 0x7f822415e6d0 2023-10-16T09:56:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f82237fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82240cb0c0 0x7f822415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f82237fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82240cb0c0 0x7f822415a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39426/0 (socket says 172.21.15.181:39426) 2023-10-16T09:56:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f82237fe700 1 -- 172.21.15.181:0/3020271132 learned_addr learned my addr 172.21.15.181:0/3020271132 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f82227fc700 1 --1- 172.21.15.181:0/3020271132 >> v1:172.21.15.181:6789/0 conn(0x7f822415eab0 0x7f822415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50502/0 (socket says 172.21.15.181:50502) 2023-10-16T09:56:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.295+0000 7f8222ffd700 1 --2- 172.21.15.181:0/3020271132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82240d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f82237fe700 1 -- 172.21.15.181:0/3020271132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 msgr2=0x7f82240d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f82237fe700 1 --2- 172.21.15.181:0/3020271132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82240d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f82237fe700 1 -- 172.21.15.181:0/3020271132 >> v1:172.21.15.181:6789/0 conn(0x7f822415eab0 legacy=0x7f822415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f82237fe700 1 -- 172.21.15.181:0/3020271132 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822406e0f0 con 0x7f82240cb0c0 2023-10-16T09:56:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f82237fe700 1 --2- 172.21.15.181:0/3020271132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f822415a800 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f821c00d150 tx=0x7f821c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9408e820595e1e49 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f8221ffb700 1 -- 172.21.15.181:0/3020271132 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c010070 con 0x7f82240cb0c0 2023-10-16T09:56:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f8221ffb700 1 -- 172.21.15.181:0/3020271132 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c00dd10 con 0x7f82240cb0c0 2023-10-16T09:56:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.296+0000 7f8221ffb700 1 -- 172.21.15.181:0/3020271132 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00b310 con 0x7f82240cb0c0 2023-10-16T09:56:43.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 -- 172.21.15.181:0/3020271132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 msgr2=0x7f822415a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 --2- 172.21.15.181:0/3020271132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f822415a800 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f821c00d150 tx=0x7f821c00d460 comp rx=0 tx=0).stop 2023-10-16T09:56:43.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 -- 172.21.15.181:0/3020271132 shutdown_connections 2023-10-16T09:56:43.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 --2- 172.21.15.181:0/3020271132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f822415a800 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 --2- 172.21.15.181:0/3020271132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82240d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 -- 172.21.15.181:0/3020271132 >> 172.21.15.181:0/3020271132 conn(0x7f82240d04a0 msgr2=0x7f82240d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:43.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 -- 172.21.15.181:0/3020271132 shutdown_connections 2023-10-16T09:56:43.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.297+0000 7f822951f700 1 -- 172.21.15.181:0/3020271132 wait complete. 2023-10-16T09:56:43.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.298+0000 7f822951f700 1 Processor -- start 2023-10-16T09:56:43.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.298+0000 7f822951f700 1 -- start start 2023-10-16T09:56:43.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f822951f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f82240c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f822951f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82241f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f822951f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f822415eab0 0x7f82241f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f822951f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822415df80 con 0x7f82240cb0c0 2023-10-16T09:56:43.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f822951f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822415d370 con 0x7f822415eab0 2023-10-16T09:56:43.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f822951f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822415d650 con 0x7f822415e6d0 2023-10-16T09:56:43.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f82240c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f82240c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39428/0 (socket says 172.21.15.181:39428) 2023-10-16T09:56:43.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 -- 172.21.15.181:0/1714090113 learned_addr learned my addr 172.21.15.181:0/1714090113 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:43.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f82227fc700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82241f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f82237fe700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f822415eab0 0x7f82241f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 -- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 msgr2=0x7f82241f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82241f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 -- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f822415eab0 msgr2=0x7f82241f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f822415eab0 0x7f82241f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.299+0000 7f8222ffd700 1 -- 172.21.15.181:0/1714090113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c00bd40 con 0x7f82240cb0c0 2023-10-16T09:56:43.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.300+0000 7f8222ffd700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f82240c2d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f821400e900 tx=0x7f821400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:43.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.300+0000 7f820b7fe700 1 -- 172.21.15.181:0/1714090113 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214013680 con 0x7f82240cb0c0 2023-10-16T09:56:43.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.300+0000 7f820b7fe700 1 -- 172.21.15.181:0/1714090113 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82140137e0 con 0x7f82240cb0c0 2023-10-16T09:56:43.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.300+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822415d930 con 0x7f82240cb0c0 2023-10-16T09:56:43.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.300+0000 7f820b7fe700 1 -- 172.21.15.181:0/1714090113 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214013b50 con 0x7f82240cb0c0 2023-10-16T09:56:43.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.300+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822415afd0 con 0x7f82240cb0c0 2023-10-16T09:56:43.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.301+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82240c44d0 con 0x7f82240cb0c0 2023-10-16T09:56:43.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.301+0000 7f820b7fe700 1 -- 172.21.15.181:0/1714090113 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f8214013cb0 con 0x7f82240cb0c0 2023-10-16T09:56:43.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.302+0000 7f820b7fe700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 0x7f821006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.302+0000 7f82227fc700 1 -- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 msgr2=0x7f821006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:43.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.302+0000 7f820b7fe700 1 -- 172.21.15.181:0/1714090113 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f8214089f90 con 0x7f82240cb0c0 2023-10-16T09:56:43.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.302+0000 7f82227fc700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 0x7f821006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:43.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.306+0000 7f820b7fe700 1 -- 172.21.15.181:0/1714090113 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f82140bcfe0 con 0x7f82240cb0c0 2023-10-16T09:56:43.333 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:43.331+0000 7fd4d85a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:56:43.354 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:43.353+0000 7f643894d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:56:43.501 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:43.499+0000 7fd4d85a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:56:43.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.503+0000 7f82227fc700 1 -- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 msgr2=0x7f821006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:43.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.503+0000 7f82227fc700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 0x7f821006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:43.525 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:43.525+0000 7f643894d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:56:43.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.549+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f822415b190 con 0x7f82240cb0c0 2023-10-16T09:56:43.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.550+0000 7f820b7fe700 1 -- 172.21.15.181:0/1714090113 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+18354 (secure 0 0 0) 0x7f8214021020 con 0x7f82240cb0c0 2023-10-16T09:56:43.551 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:43.551 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":45,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:56:42.936392+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:44:54.175032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:55.152824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3764257785":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6833/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6832/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/4251066643":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2848692303":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2080914758":"2023-10-17T09:45:15.015302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":6,"length":1},{"begin":8,"length":3},{"begin":13,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":4},{"begin":24,"length":1},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":3},{"begin":34,"length":1},{"begin":36,"length":1},{"begin":38,"length":1},{"begin":40,"length":4},{"begin":45,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":3},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":61,"length":4},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":70,"length":1},{"begin":73,"length":3},{"begin":77,"length":1},{"begin":79,"length":1},{"begin":81,"length":2},{"begin":84,"length":3},{"begin":88,"length":1},{"begin":90,"length":1},{"begin":92,"length":1},{"begin":94,"length":3},{"begin":98,"length":1},{"begin":100,"length":1},{"begin":102,"length":2},{"begin":105,"length":2},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":155,"length":66}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":10,"length":1},{"begin":21,"length":2},{"begin":32,"length":1},{"begin":43,"length":1},{"begin":53,"length":1},{"begin":64,"length":1},{"begin":75,"length":1},{"begin":86,"length":1},{"begin":96,"length":1},{"begin":103,"length":1},{"begin":210,"length":11}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:56:43.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 msgr2=0x7f821006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 0x7f821006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 msgr2=0x7f82240c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f82240c2d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f821400e900 tx=0x7f821400ec10 comp rx=0 tx=0).stop 2023-10-16T09:56:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 shutdown_connections 2023-10-16T09:56:43.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f821006b0b0 0x7f821006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f822415eab0 0x7f82241f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82240cb0c0 0x7f82240c2d70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 --2- 172.21.15.181:0/1714090113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f822415e6d0 0x7f82241f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.552+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 >> 172.21.15.181:0/1714090113 conn(0x7f82240d04a0 msgr2=0x7f82240aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:43.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.553+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 shutdown_connections 2023-10-16T09:56:43.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.553+0000 7f822951f700 1 -- 172.21.15.181:0/1714090113 wait complete. 2023-10-16T09:56:43.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:44:54.175032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:55.152824+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 45, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:56:43.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:56:43.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.729+0000 7f2226eaf700 1 Processor -- start 2023-10-16T09:56:43.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.729+0000 7f2226eaf700 1 -- start start 2023-10-16T09:56:43.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f2226eaf700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f222015f150 0x7f22200cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f2226eaf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22200ce4f0 0x7f22200b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f2226eaf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22200b2ae0 con 0x7f22200b14a0 2023-10-16T09:56:43.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f2226eaf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22200ce9a0 con 0x7f22200ce4f0 2023-10-16T09:56:43.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f2226eaf700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22200b2dc0 con 0x7f222015f150 2023-10-16T09:56:43.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22200ce4f0 0x7f22200b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f222544c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f22200b14a0 0x7f22200b18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50538/0 (socket says 172.21.15.181:50538) 2023-10-16T09:56:43.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f222544c700 1 -- 172.21.15.181:0/2355003913 learned_addr learned my addr 172.21.15.181:0/2355003913 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:43.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221ffff700 1 --2- 172.21.15.181:0/2355003913 >> v2:172.21.15.181:3300/0 conn(0x7f22200ce4f0 0x7f22200b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39458/0 (socket says 172.21.15.181:39458) 2023-10-16T09:56:43.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f2224c4b700 1 --2- 172.21.15.181:0/2355003913 >> v2:172.21.15.190:3300/0 conn(0x7f222015f150 0x7f22200cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221ffff700 1 -- 172.21.15.181:0/2355003913 >> v2:172.21.15.190:3300/0 conn(0x7f222015f150 msgr2=0x7f22200cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:43.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221ffff700 1 --2- 172.21.15.181:0/2355003913 >> v2:172.21.15.190:3300/0 conn(0x7f222015f150 0x7f22200cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221ffff700 1 -- 172.21.15.181:0/2355003913 >> v1:172.21.15.181:6789/0 conn(0x7f22200b14a0 legacy=0x7f22200b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221ffff700 1 -- 172.21.15.181:0/2355003913 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22200cf9d0 con 0x7f22200ce4f0 2023-10-16T09:56:43.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221f7fe700 1 -- 172.21.15.181:0/2355003913 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583546787 0 0) 0x7f22200b2ae0 con 0x7f22200b14a0 2023-10-16T09:56:43.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.730+0000 7f221ffff700 1 --2- 172.21.15.181:0/2355003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200ce4f0 0x7f22200b0f60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f220c009840 tx=0x7f220c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ef48d2316f8313 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:43.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f221f7fe700 1 -- 172.21.15.181:0/2355003913 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c011830 con 0x7f22200ce4f0 2023-10-16T09:56:43.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f221f7fe700 1 -- 172.21.15.181:0/2355003913 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c011990 con 0x7f22200ce4f0 2023-10-16T09:56:43.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f221f7fe700 1 -- 172.21.15.181:0/2355003913 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c011d20 con 0x7f22200ce4f0 2023-10-16T09:56:43.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 -- 172.21.15.181:0/2355003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200ce4f0 msgr2=0x7f22200b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2355003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200ce4f0 0x7f22200b0f60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f220c009840 tx=0x7f220c009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:43.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 -- 172.21.15.181:0/2355003913 shutdown_connections 2023-10-16T09:56:43.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2355003913 >> v2:172.21.15.190:3300/0 conn(0x7f222015f150 0x7f22200cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2355003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200ce4f0 0x7f22200b0f60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 -- 172.21.15.181:0/2355003913 >> 172.21.15.181:0/2355003913 conn(0x7f222009a930 msgr2=0x7f22200d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:43.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 -- 172.21.15.181:0/2355003913 shutdown_connections 2023-10-16T09:56:43.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.731+0000 7f2226eaf700 1 -- 172.21.15.181:0/2355003913 wait complete. 2023-10-16T09:56:43.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.732+0000 7f2226eaf700 1 Processor -- start 2023-10-16T09:56:43.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2226eaf700 1 -- start start 2023-10-16T09:56:43.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2226eaf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200b14a0 0x7f22200c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2226eaf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22200ce4f0 0x7f222006b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2226eaf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f222015f150 0x7f222015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2226eaf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f68a0 con 0x7f22200b14a0 2023-10-16T09:56:43.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2226eaf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f3820 con 0x7f222015f150 2023-10-16T09:56:43.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2226eaf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22201f3b00 con 0x7f22200ce4f0 2023-10-16T09:56:43.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.733+0000 7f2224c4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200b14a0 0x7f22200c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200b14a0 0x7f22200c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39464/0 (socket says 172.21.15.181:39464) 2023-10-16T09:56:43.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 -- 172.21.15.181:0/2931566837 learned_addr learned my addr 172.21.15.181:0/2931566837 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:43.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f222544c700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f222015f150 0x7f222015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f221ffff700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22200ce4f0 0x7f222006b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:43.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 -- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22200ce4f0 msgr2=0x7f222006b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22200ce4f0 0x7f222006b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 -- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f222015f150 msgr2=0x7f222015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f222015f150 0x7f222015a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 -- 172.21.15.181:0/2931566837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0094a0 con 0x7f22200b14a0 2023-10-16T09:56:43.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2224c4b700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200b14a0 0x7f22200c5130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9b0 tx=0x7f221400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:43.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f221d7fa700 1 -- 172.21.15.181:0/2931566837 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214009710 con 0x7f22200b14a0 2023-10-16T09:56:43.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f221d7fa700 1 -- 172.21.15.181:0/2931566837 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221400ee50 con 0x7f22200b14a0 2023-10-16T09:56:43.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22201f3de0 con 0x7f22200b14a0 2023-10-16T09:56:43.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f221d7fa700 1 -- 172.21.15.181:0/2931566837 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221400ca50 con 0x7f22200b14a0 2023-10-16T09:56:43.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.734+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22200c5670 con 0x7f22200b14a0 2023-10-16T09:56:43.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.735+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22201f6bf0 con 0x7f22200b14a0 2023-10-16T09:56:43.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.736+0000 7f221d7fa700 1 -- 172.21.15.181:0/2931566837 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f221400cbb0 con 0x7f22200b14a0 2023-10-16T09:56:43.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.736+0000 7f221d7fa700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 0x7f220406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:43.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.736+0000 7f221ffff700 1 -- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 msgr2=0x7f220406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:43.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.736+0000 7f221d7fa700 1 -- 172.21.15.181:0/2931566837 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f2214089d40 con 0x7f22200b14a0 2023-10-16T09:56:43.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.736+0000 7f221ffff700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 0x7f220406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:43.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.741+0000 7f221d7fa700 1 -- 172.21.15.181:0/2931566837 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f22140546f0 con 0x7f22200b14a0 2023-10-16T09:56:43.823 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:43.821+0000 7fd4d85a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:56:43.863 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:43.863+0000 7f643894d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:56:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.937+0000 7f221ffff700 1 -- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 msgr2=0x7f220406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.937+0000 7f221ffff700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 0x7f220406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:43.941 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:43.939+0000 7fd4d85a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:56:43.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.941+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f222009ad10 con 0x7f22200b14a0 2023-10-16T09:56:43.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.942+0000 7f221d7fa700 1 -- 172.21.15.181:0/2931566837 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22140210a0 con 0x7f22200b14a0 2023-10-16T09:56:43.942 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:56:43.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 msgr2=0x7f220406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:43.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 0x7f220406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200b14a0 msgr2=0x7f22200c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:43.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200b14a0 0x7f22200c5130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9b0 tx=0x7f221400ecc0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 shutdown_connections 2023-10-16T09:56:43.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f220406b0b0 0x7f220406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f222015f150 0x7f222015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22200b14a0 0x7f22200c5130 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 --2- 172.21.15.181:0/2931566837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22200ce4f0 0x7f222006b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:43.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 >> 172.21.15.181:0/2931566837 conn(0x7f222009a930 msgr2=0x7f22201e6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:43.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 shutdown_connections 2023-10-16T09:56:43.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:43.945+0000 7f2226eaf700 1 -- 172.21.15.181:0/2931566837 wait complete. 2023-10-16T09:56:43.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:56:43.991 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:43.991+0000 7f643894d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:56:44.015 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:44.013+0000 7fd4d85a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:56:44.068 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:44.068+0000 7f643894d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:56:44.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.088+0000 7f9556932700 1 Processor -- start 2023-10-16T09:56:44.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.088+0000 7f9556932700 1 -- start start 2023-10-16T09:56:44.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9556932700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95500d07d0 0x7f95500b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9556932700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500d2960 con 0x7f955015d9e0 2023-10-16T09:56:44.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9556932700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500d4180 con 0x7f95500d24e0 2023-10-16T09:56:44.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9556932700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500d20c0 con 0x7f95500d07d0 2023-10-16T09:56:44.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f954ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f955015d9e0 0x7f95500cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50550/0 (socket says 172.21.15.181:50550) 2023-10-16T09:56:44.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f954ffff700 1 -- 172.21.15.181:0/3794891819 learned_addr learned my addr 172.21.15.181:0/3794891819 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:44.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9554ecf700 1 --2- 172.21.15.181:0/3794891819 >> v2:172.21.15.181:3300/0 conn(0x7f95500d07d0 0x7f95500b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f954effd700 1 -- 172.21.15.181:0/3794891819 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078242666 0 0) 0x7f95500d2960 con 0x7f955015d9e0 2023-10-16T09:56:44.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f954effd700 1 -- 172.21.15.181:0/3794891819 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95380035b0 con 0x7f955015d9e0 2023-10-16T09:56:44.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9554ecf700 1 -- 172.21.15.181:0/3794891819 >> v1:172.21.15.190:6789/0 conn(0x7f95500d24e0 legacy=0x7f95500d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9554ecf700 1 -- 172.21.15.181:0/3794891819 >> v1:172.21.15.181:6789/0 conn(0x7f955015d9e0 legacy=0x7f95500cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.089+0000 7f9554ecf700 1 -- 172.21.15.181:0/3794891819 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500d1440 con 0x7f95500d07d0 2023-10-16T09:56:44.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f9554ecf700 1 --2- 172.21.15.181:0/3794891819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d07d0 0x7f95500b6620 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9548009d90 tx=0x7f954800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e28f6956e3deee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:44.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f954effd700 1 -- 172.21.15.181:0/3794891819 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954800e070 con 0x7f95500d07d0 2023-10-16T09:56:44.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f954effd700 1 -- 172.21.15.181:0/3794891819 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954800bb20 con 0x7f95500d07d0 2023-10-16T09:56:44.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f954effd700 1 -- 172.21.15.181:0/3794891819 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548012430 con 0x7f95500d07d0 2023-10-16T09:56:44.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f9556932700 1 -- 172.21.15.181:0/3794891819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d07d0 msgr2=0x7f95500b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f9556932700 1 --2- 172.21.15.181:0/3794891819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d07d0 0x7f95500b6620 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9548009d90 tx=0x7f954800b330 comp rx=0 tx=0).stop 2023-10-16T09:56:44.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f9556932700 1 -- 172.21.15.181:0/3794891819 shutdown_connections 2023-10-16T09:56:44.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f9556932700 1 --2- 172.21.15.181:0/3794891819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d07d0 0x7f95500b6620 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f9556932700 1 -- 172.21.15.181:0/3794891819 >> 172.21.15.181:0/3794891819 conn(0x7f955009cb40 msgr2=0x7f95500d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:44.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.090+0000 7f9556932700 1 -- 172.21.15.181:0/3794891819 shutdown_connections 2023-10-16T09:56:44.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.091+0000 7f9556932700 1 -- 172.21.15.181:0/3794891819 wait complete. 2023-10-16T09:56:44.096 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:44.094+0000 7fd4d85a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:56:44.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.092+0000 7f9556932700 1 Processor -- start 2023-10-16T09:56:44.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.092+0000 7f9556932700 1 -- start start 2023-10-16T09:56:44.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.092+0000 7f9556932700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95500d07d0 0x7f95500cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.092+0000 7f9556932700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d24e0 0x7f955015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.092+0000 7f9556932700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015d9e0 0x7f955015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f9556932700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500cc320 con 0x7f95500d24e0 2023-10-16T09:56:44.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f9556932700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c91e0 con 0x7f955015d9e0 2023-10-16T09:56:44.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f9556932700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c85d0 con 0x7f95500d07d0 2023-10-16T09:56:44.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d24e0 0x7f955015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d24e0 0x7f955015b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39480/0 (socket says 172.21.15.181:39480) 2023-10-16T09:56:44.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 -- 172.21.15.181:0/3223599652 learned_addr learned my addr 172.21.15.181:0/3223599652 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:44.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954ffff700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95500d07d0 0x7f95500cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f9554ecf700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015d9e0 0x7f955015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 -- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95500d07d0 msgr2=0x7f95500cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95500d07d0 0x7f95500cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 -- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015d9e0 msgr2=0x7f955015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015d9e0 0x7f955015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 -- 172.21.15.181:0/3223599652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548009a70 con 0x7f95500d24e0 2023-10-16T09:56:44.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954f7fe700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d24e0 0x7f955015b5e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f954400c700 tx=0x7f954400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:44.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954cff9700 1 -- 172.21.15.181:0/3223599652 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440052d0 con 0x7f95500d24e0 2023-10-16T09:56:44.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954cff9700 1 -- 172.21.15.181:0/3223599652 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9544005430 con 0x7f95500d24e0 2023-10-16T09:56:44.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f954cff9700 1 -- 172.21.15.181:0/3223599652 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544003880 con 0x7f95500d24e0 2023-10-16T09:56:44.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95500c88b0 con 0x7f95500d24e0 2023-10-16T09:56:44.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.093+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500c8e70 con 0x7f95500d24e0 2023-10-16T09:56:44.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.094+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9550067d20 con 0x7f95500d24e0 2023-10-16T09:56:44.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.095+0000 7f954cff9700 1 -- 172.21.15.181:0/3223599652 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f95440039e0 con 0x7f95500d24e0 2023-10-16T09:56:44.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.095+0000 7f954cff9700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 0x7f953c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.095+0000 7f954ffff700 1 -- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 msgr2=0x7f953c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:44.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.095+0000 7f954cff9700 1 -- 172.21.15.181:0/3223599652 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f95440890d0 con 0x7f95500d24e0 2023-10-16T09:56:44.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.096+0000 7f954ffff700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 0x7f953c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:44.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.100+0000 7f954cff9700 1 -- 172.21.15.181:0/3223599652 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9544053a90 con 0x7f95500d24e0 2023-10-16T09:56:44.153 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:44.153+0000 7f643894d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:56:44.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.297+0000 7f954ffff700 1 -- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 msgr2=0x7f953c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:44.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.297+0000 7f954ffff700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 0x7f953c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:44.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.298+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95500d0e50 con 0x7f95500d24e0 2023-10-16T09:56:44.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.298+0000 7f954cff9700 1 -- 172.21.15.181:0/3223599652 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95440570b0 con 0x7f95500d24e0 2023-10-16T09:56:44.299 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:56:44.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 msgr2=0x7f953c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:44.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 0x7f953c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d24e0 msgr2=0x7f955015b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d24e0 0x7f955015b5e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f954400c700 tx=0x7f954400ca10 comp rx=0 tx=0).stop 2023-10-16T09:56:44.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 shutdown_connections 2023-10-16T09:56:44.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95500d07d0 0x7f95500cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f953c06b0b0 0x7f953c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015d9e0 0x7f955015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 --2- 172.21.15.181:0/3223599652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95500d24e0 0x7f955015b5e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 >> 172.21.15.181:0/3223599652 conn(0x7f955009cb40 msgr2=0x7f955015f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:44.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.300+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 shutdown_connections 2023-10-16T09:56:44.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.301+0000 7f9556932700 1 -- 172.21.15.181:0/3223599652 wait complete. 2023-10-16T09:56:44.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:56:44.467 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:44.466+0000 7fd4d85a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:56:44.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.479+0000 7fd02a6b6700 1 Processor -- start 2023-10-16T09:56:44.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.480+0000 7fd02a6b6700 1 -- start start 2023-10-16T09:56:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.480+0000 7fd02a6b6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0240b2010 0x7fd0240cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.480+0000 7fd02a6b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0240b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.480+0000 7fd02a6b6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240ce1f0 con 0x7fd02406f3b0 2023-10-16T09:56:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.480+0000 7fd02a6b6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0240b2930 con 0x7fd0240b2010 2023-10-16T09:56:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.480+0000 7fd02a6b6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240ccda0 con 0x7fd024071be0 2023-10-16T09:56:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd01bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0240b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd023fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0240b2010 0x7fd0240cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd028c53700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd02406f3b0 0x7fd02406f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:34534/0 (socket says 172.21.15.181:34534) 2023-10-16T09:56:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd023fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0240b2010 0x7fd0240cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:44560/0 (socket says 172.21.15.181:44560) 2023-10-16T09:56:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd028c53700 1 -- 172.21.15.181:0/2835337232 learned_addr learned my addr 172.21.15.181:0/2835337232 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd01bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0240b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:36538/0 (socket says 172.21.15.181:36538) 2023-10-16T09:56:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd023fff700 1 -- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 msgr2=0x7fd0240b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd023fff700 1 --2- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0240b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd023fff700 1 -- 172.21.15.181:0/2835337232 >> v1:172.21.15.190:6789/0 conn(0x7fd02406f3b0 legacy=0x7fd02406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd023fff700 1 -- 172.21.15.181:0/2835337232 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240727e0 con 0x7fd0240b2010 2023-10-16T09:56:44.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd01bfff700 1 --2- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0240b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:44.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd023fff700 1 --2- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0240b2010 0x7fd0240cc6e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd01400bae0 tx=0x7fd01400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=388d5212a9da2fcd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:44.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd0237fe700 1 -- 172.21.15.181:0/2835337232 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400dc70 con 0x7fd0240b2010 2023-10-16T09:56:44.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd0237fe700 1 -- 172.21.15.181:0/2835337232 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01400ddd0 con 0x7fd0240b2010 2023-10-16T09:56:44.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.481+0000 7fd0237fe700 1 -- 172.21.15.181:0/2835337232 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014010690 con 0x7fd0240b2010 2023-10-16T09:56:44.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 -- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0240b2010 msgr2=0x7fd0240cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0240b2010 0x7fd0240cc6e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd01400bae0 tx=0x7fd01400bdf0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 -- 172.21.15.181:0/2835337232 shutdown_connections 2023-10-16T09:56:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0240b2010 0x7fd0240cc6e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/2835337232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0240b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 -- 172.21.15.181:0/2835337232 >> 172.21.15.181:0/2835337232 conn(0x7fd0240cd010 msgr2=0x7fd0240cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:44.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 -- 172.21.15.181:0/2835337232 shutdown_connections 2023-10-16T09:56:44.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.482+0000 7fd02a6b6700 1 -- 172.21.15.181:0/2835337232 wait complete. 2023-10-16T09:56:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.483+0000 7fd02a6b6700 1 Processor -- start 2023-10-16T09:56:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.483+0000 7fd02a6b6700 1 -- start start 2023-10-16T09:56:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd02a6b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd02406f3b0 0x7fd02413e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd02a6b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0241f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd02a6b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0240b2010 0x7fd02415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd02a6b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240cf050 con 0x7fd0240b2010 2023-10-16T09:56:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd02a6b6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02415db70 con 0x7fd02406f3b0 2023-10-16T09:56:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd02a6b6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240ce720 con 0x7fd024071be0 2023-10-16T09:56:44.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd028c53700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0240b2010 0x7fd02415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd028c53700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0240b2010 0x7fd02415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39520/0 (socket says 172.21.15.181:39520) 2023-10-16T09:56:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd028c53700 1 -- 172.21.15.181:0/781828173 learned_addr learned my addr 172.21.15.181:0/781828173 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd01bfff700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0241f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.484+0000 7fd023fff700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd02406f3b0 0x7fd02413e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd028c53700 1 -- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 msgr2=0x7fd0241f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd028c53700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0241f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd028c53700 1 -- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd02406f3b0 msgr2=0x7fd02413e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd028c53700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd02406f3b0 0x7fd02413e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd028c53700 1 -- 172.21.15.181:0/781828173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0094a0 con 0x7fd0240b2010 2023-10-16T09:56:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd023fff700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd02406f3b0 0x7fd02413e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd01bfff700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0241f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd028c53700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0240b2010 0x7fd02415ced0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00cbc0 tx=0x7fd01c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd0217fa700 1 -- 172.21.15.181:0/781828173 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c00a9e0 con 0x7fd0240b2010 2023-10-16T09:56:44.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd0217fa700 1 -- 172.21.15.181:0/781828173 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c00ab40 con 0x7fd0240b2010 2023-10-16T09:56:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01400b740 con 0x7fd0240b2010 2023-10-16T09:56:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd0217fa700 1 -- 172.21.15.181:0/781828173 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c012400 con 0x7fd0240b2010 2023-10-16T09:56:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.485+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0240cece0 con 0x7fd0240b2010 2023-10-16T09:56:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.486+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0240cf1c0 con 0x7fd0240b2010 2023-10-16T09:56:44.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.486+0000 7fd0217fa700 1 -- 172.21.15.181:0/781828173 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fd01c01b3b0 con 0x7fd0240b2010 2023-10-16T09:56:44.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.487+0000 7fd0217fa700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 0x7fd00406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.487+0000 7fd023fff700 1 -- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 msgr2=0x7fd00406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:44.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.487+0000 7fd0217fa700 1 -- 172.21.15.181:0/781828173 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fd01c089840 con 0x7fd0240b2010 2023-10-16T09:56:44.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.487+0000 7fd023fff700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 0x7fd00406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:44.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.491+0000 7fd0217fa700 1 -- 172.21.15.181:0/781828173 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd01c0542a0 con 0x7fd0240b2010 2023-10-16T09:56:44.542 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:44.542+0000 7f643894d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:56:44.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.688+0000 7fd023fff700 1 -- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 msgr2=0x7fd00406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:44.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.688+0000 7fd023fff700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 0x7fd00406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:44.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.696+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0240b6510 con 0x7fd0240b2010 2023-10-16T09:56:44.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.696+0000 7fd0217fa700 1 -- 172.21.15.181:0/781828173 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd01c0578c0 con 0x7fd0240b2010 2023-10-16T09:56:44.697 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:56:44.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 msgr2=0x7fd00406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:44.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 0x7fd00406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0240b2010 msgr2=0x7fd02415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0240b2010 0x7fd02415ced0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00cbc0 tx=0x7fd01c00ced0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 shutdown_connections 2023-10-16T09:56:44.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fd00406b0b0 0x7fd00406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0240b2010 0x7fd02415ced0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd02406f3b0 0x7fd02413e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 --2- 172.21.15.181:0/781828173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd024071be0 0x7fd0241f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.698+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 >> 172.21.15.181:0/781828173 conn(0x7fd0240cd010 msgr2=0x7fd024158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:44.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.699+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 shutdown_connections 2023-10-16T09:56:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.699+0000 7fd02a6b6700 1 -- 172.21.15.181:0/781828173 wait complete. 2023-10-16T09:56:44.712 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:56:44.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.835+0000 7f9797865700 1 Processor -- start 2023-10-16T09:56:44.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.835+0000 7f9797865700 1 -- start start 2023-10-16T09:56:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9797865700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97900d0fc0 0x7f97900cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9797865700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f979006a8a0 0x7f979006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9797865700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97900cddf0 con 0x7f979015df60 2023-10-16T09:56:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9797865700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979006b290 con 0x7f979006a8a0 2023-10-16T09:56:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9797865700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97900af9c0 con 0x7f97900d0fc0 2023-10-16T09:56:44.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9794e00700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97900d0fc0 0x7f97900cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9794e00700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97900d0fc0 0x7f97900cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:44578/0 (socket says 172.21.15.181:44578) 2023-10-16T09:56:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9794e00700 1 -- 172.21.15.181:0/3525917845 learned_addr learned my addr 172.21.15.181:0/3525917845 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9795e02700 1 --2- 172.21.15.181:0/3525917845 >> v2:172.21.15.181:3300/0 conn(0x7f979006a8a0 0x7f979006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9795e02700 1 -- 172.21.15.181:0/3525917845 >> v2:172.21.15.190:3300/0 conn(0x7f97900d0fc0 msgr2=0x7f97900cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.836+0000 7f9795e02700 1 --2- 172.21.15.181:0/3525917845 >> v2:172.21.15.190:3300/0 conn(0x7f97900d0fc0 0x7f97900cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.837+0000 7f9795e02700 1 -- 172.21.15.181:0/3525917845 >> v1:172.21.15.190:6789/0 conn(0x7f979015df60 legacy=0x7f97900cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.837+0000 7f9787fff700 1 -- 172.21.15.181:0/3525917845 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453549570 0 0) 0x7f97900cddf0 con 0x7f979015df60 2023-10-16T09:56:44.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.837+0000 7f9795e02700 1 -- 172.21.15.181:0/3525917845 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900ccf50 con 0x7f979006a8a0 2023-10-16T09:56:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.837+0000 7f9795e02700 1 --2- 172.21.15.181:0/3525917845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f979006ad50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f979006c990 tx=0x7f978800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0ceed4cff727b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.837+0000 7f9787fff700 1 -- 172.21.15.181:0/3525917845 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978800e070 con 0x7f979006a8a0 2023-10-16T09:56:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.837+0000 7f9787fff700 1 -- 172.21.15.181:0/3525917845 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978800bb20 con 0x7f979006a8a0 2023-10-16T09:56:44.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.837+0000 7f9787fff700 1 -- 172.21.15.181:0/3525917845 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788012430 con 0x7f979006a8a0 2023-10-16T09:56:44.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 -- 172.21.15.181:0/3525917845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 msgr2=0x7f979006ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 --2- 172.21.15.181:0/3525917845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f979006ad50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f979006c990 tx=0x7f978800b330 comp rx=0 tx=0).stop 2023-10-16T09:56:44.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 -- 172.21.15.181:0/3525917845 shutdown_connections 2023-10-16T09:56:44.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 --2- 172.21.15.181:0/3525917845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f979006ad50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 --2- 172.21.15.181:0/3525917845 >> v2:172.21.15.190:3300/0 conn(0x7f97900d0fc0 0x7f97900cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 -- 172.21.15.181:0/3525917845 >> 172.21.15.181:0/3525917845 conn(0x7f97900d04a0 msgr2=0x7f97900d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:44.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 -- 172.21.15.181:0/3525917845 shutdown_connections 2023-10-16T09:56:44.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.838+0000 7f9797865700 1 -- 172.21.15.181:0/3525917845 wait complete. 2023-10-16T09:56:44.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.839+0000 7f9797865700 1 Processor -- start 2023-10-16T09:56:44.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.839+0000 7f9797865700 1 -- start start 2023-10-16T09:56:44.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9797865700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f97900c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9797865700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97900d0fc0 0x7f9790150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9797865700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979015df60 0x7f97900c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9797865700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790151460 con 0x7f979006a8a0 2023-10-16T09:56:44.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9797865700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97900c5210 con 0x7f97900d0fc0 2023-10-16T09:56:44.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9797865700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97900c4600 con 0x7f979015df60 2023-10-16T09:56:44.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f97900c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f97900c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39532/0 (socket says 172.21.15.181:39532) 2023-10-16T09:56:44.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 -- 172.21.15.181:0/2801053548 learned_addr learned my addr 172.21.15.181:0/2801053548 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:44.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9794e00700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97900d0fc0 0x7f9790150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795e02700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979015df60 0x7f97900c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:44.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 -- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979015df60 msgr2=0x7f97900c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979015df60 0x7f97900c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 -- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97900d0fc0 msgr2=0x7f9790150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:44.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97900d0fc0 0x7f9790150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:44.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 -- 172.21.15.181:0/2801053548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9788009a70 con 0x7f979006a8a0 2023-10-16T09:56:44.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9795601700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f97900c3f00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f978c00c4a0 tx=0x7f978c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:44.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9785ffb700 1 -- 172.21.15.181:0/2801053548 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0054e0 con 0x7f979006a8a0 2023-10-16T09:56:44.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9785ffb700 1 -- 172.21.15.181:0/2801053548 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c0036d0 con 0x7f979006a8a0 2023-10-16T09:56:44.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97900c48e0 con 0x7f979006a8a0 2023-10-16T09:56:44.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.840+0000 7f9785ffb700 1 -- 172.21.15.181:0/2801053548 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c003a80 con 0x7f979006a8a0 2023-10-16T09:56:44.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.841+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97900c4ea0 con 0x7f979006a8a0 2023-10-16T09:56:44.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.842+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97900c54f0 con 0x7f979006a8a0 2023-10-16T09:56:44.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.842+0000 7f9785ffb700 1 -- 172.21.15.181:0/2801053548 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f978c003be0 con 0x7f979006a8a0 2023-10-16T09:56:44.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.842+0000 7f9785ffb700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 0x7f977406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:44.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.842+0000 7f9794e00700 1 -- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 msgr2=0x7f977406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:44.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.843+0000 7f9785ffb700 1 -- 172.21.15.181:0/2801053548 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f978c089130 con 0x7f979006a8a0 2023-10-16T09:56:44.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.843+0000 7f9794e00700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 0x7f977406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:44.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:44.847+0000 7f9785ffb700 1 -- 172.21.15.181:0/2801053548 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f978c053ae0 con 0x7f979006a8a0 2023-10-16T09:56:45.041 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:45.040+0000 7fd4d85a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:56:45.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.044+0000 7f9794e00700 1 -- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 msgr2=0x7f977406d540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:45.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.044+0000 7f9794e00700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 0x7f977406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:45.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.095+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f97900c5060 con 0x7f979006a8a0 2023-10-16T09:56:45.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.095+0000 7f9785ffb700 1 -- 172.21.15.181:0/2801053548 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v47) v1 ==== 80+0+360 (secure 0 0 0) 0x7f978c017180 con 0x7f979006a8a0 2023-10-16T09:56:45.097 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3764257785 2023-10-17T09:45:15.015302+0000 2023-10-16T09:56:45.097 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/2039940952 2023-10-17T09:45:15.015302+0000 2023-10-16T09:56:45.097 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/2039940952 2023-10-17T09:45:15.015302+0000 2023-10-16T09:56:45.097 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/4251066643 2023-10-17T09:45:15.015302+0000 2023-10-16T09:56:45.098 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2848692303 2023-10-17T09:45:15.015302+0000 2023-10-16T09:56:45.098 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2080914758 2023-10-17T09:45:15.015302+0000 2023-10-16T09:56:45.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 msgr2=0x7f977406d540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:45.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 0x7f977406d540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 msgr2=0x7f97900c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f97900c3f00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f978c00c4a0 tx=0x7f978c00c7b0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 shutdown_connections 2023-10-16T09:56:45.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979015df60 0x7f97900c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f977406b080 0x7f977406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f979006a8a0 0x7f97900c3f00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 --2- 172.21.15.181:0/2801053548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97900d0fc0 0x7f9790150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 >> 172.21.15.181:0/2801053548 conn(0x7f97900d04a0 msgr2=0x7f979015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:45.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 shutdown_connections 2023-10-16T09:56:45.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.098+0000 7f9797865700 1 -- 172.21.15.181:0/2801053548 wait complete. 2023-10-16T09:56:45.102 INFO:teuthology.orchestra.run.smithi181.stderr:listed 6 entries 2023-10-16T09:56:45.109 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:45.108+0000 7fd4d85a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:56:45.112 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T09:56:45.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:45.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T09:56:45.141 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:45.141+0000 7f643894d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:56:45.153 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:56:45.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:56:45.154 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T09:56:45.211 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:56:45.211 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:45.219 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:45.218+0000 7f643894d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:56:45.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:56:45.229 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:56:45.229 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:45.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:56:45.246 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:56:45.247 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:45.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:56:45.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:56:45.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.411+0000 7f3d0a7f6700 1 Processor -- start 2023-10-16T09:56:45.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.411+0000 7f3d0a7f6700 1 -- start start 2023-10-16T09:56:45.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.411+0000 7f3d0a7f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d040d0370 0x7f3d040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.411+0000 7f3d0a7f6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d040cc510 con 0x7f3d0415b990 2023-10-16T09:56:45.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.411+0000 7f3d0a7f6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d040cc7f0 con 0x7f3d040cd2d0 2023-10-16T09:56:45.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.411+0000 7f3d0a7f6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d040ccad0 con 0x7f3d040d0370 2023-10-16T09:56:45.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d040d0370 0x7f3d040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d03fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3d0415b990 0x7f3d040cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50620/0 (socket says 172.21.15.181:50620) 2023-10-16T09:56:45.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d03fff700 1 -- 172.21.15.181:0/2776002423 learned_addr learned my addr 172.21.15.181:0/2776002423 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:45.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d040d0370 0x7f3d040ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39544/0 (socket says 172.21.15.181:39544) 2023-10-16T09:56:45.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d037fe700 1 -- 172.21.15.181:0/2776002423 >> v1:172.21.15.190:6789/0 conn(0x7f3d040cd2d0 legacy=0x7f3d040ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d037fe700 1 -- 172.21.15.181:0/2776002423 >> v1:172.21.15.181:6789/0 conn(0x7f3d0415b990 legacy=0x7f3d040cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d02ffd700 1 -- 172.21.15.181:0/2776002423 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191680587 0 0) 0x7f3d040cc510 con 0x7f3d0415b990 2023-10-16T09:56:45.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d037fe700 1 -- 172.21.15.181:0/2776002423 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040ccdb0 con 0x7f3d040d0370 2023-10-16T09:56:45.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d037fe700 1 --2- 172.21.15.181:0/2776002423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040d0370 0x7f3d040ce8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009840 tx=0x7f3cf8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38772f894ff1a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:45.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d02ffd700 1 -- 172.21.15.181:0/2776002423 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8011830 con 0x7f3d040d0370 2023-10-16T09:56:45.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d02ffd700 1 -- 172.21.15.181:0/2776002423 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf8011990 con 0x7f3d040d0370 2023-10-16T09:56:45.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.412+0000 7f3d02ffd700 1 -- 172.21.15.181:0/2776002423 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8011d20 con 0x7f3d040d0370 2023-10-16T09:56:45.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.413+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/2776002423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040d0370 msgr2=0x7f3d040ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.413+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/2776002423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040d0370 0x7f3d040ce8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009840 tx=0x7f3cf8009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:45.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.413+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/2776002423 shutdown_connections 2023-10-16T09:56:45.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.413+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/2776002423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040d0370 0x7f3d040ce8f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.413+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/2776002423 >> 172.21.15.181:0/2776002423 conn(0x7f3d0409a9a0 msgr2=0x7f3d040d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:45.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.413+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/2776002423 shutdown_connections 2023-10-16T09:56:45.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.413+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/2776002423 wait complete. 2023-10-16T09:56:45.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.414+0000 7f3d0a7f6700 1 Processor -- start 2023-10-16T09:56:45.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.414+0000 7f3d0a7f6700 1 -- start start 2023-10-16T09:56:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d0a7f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040cd2d0 0x7f3d040b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d0a7f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d040d0370 0x7f3d040b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d0a7f6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0415b990 0x7f3d04154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d0a7f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040c9f30 con 0x7f3d040cd2d0 2023-10-16T09:56:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d0a7f6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040b6990 con 0x7f3d040d0370 2023-10-16T09:56:45.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d0a7f6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040b6c60 con 0x7f3d0415b990 2023-10-16T09:56:45.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040cd2d0 0x7f3d040b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040cd2d0 0x7f3d040b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39548/0 (socket says 172.21.15.181:39548) 2023-10-16T09:56:45.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 -- 172.21.15.181:0/1339011109 learned_addr learned my addr 172.21.15.181:0/1339011109 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:45.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d037fe700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d040d0370 0x7f3d040b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d08d93700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0415b990 0x7f3d04154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 -- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0415b990 msgr2=0x7f3d04154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0415b990 0x7f3d04154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 -- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d040d0370 msgr2=0x7f3d040b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d040d0370 0x7f3d040b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.415+0000 7f3d03fff700 1 -- 172.21.15.181:0/1339011109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80094a0 con 0x7f3d040cd2d0 2023-10-16T09:56:45.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.416+0000 7f3d03fff700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040cd2d0 0x7f3d040b42a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c6d0 tx=0x7f3cf400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:45.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.416+0000 7f3d00ff9700 1 -- 172.21.15.181:0/1339011109 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40074e0 con 0x7f3d040cd2d0 2023-10-16T09:56:45.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.416+0000 7f3d00ff9700 1 -- 172.21.15.181:0/1339011109 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf4007660 con 0x7f3d040cd2d0 2023-10-16T09:56:45.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.416+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d040b6f40 con 0x7f3d040cd2d0 2023-10-16T09:56:45.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.416+0000 7f3d00ff9700 1 -- 172.21.15.181:0/1339011109 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4005a70 con 0x7f3d040cd2d0 2023-10-16T09:56:45.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.416+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d040ca4b0 con 0x7f3d040cd2d0 2023-10-16T09:56:45.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.417+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d040b7180 con 0x7f3d040cd2d0 2023-10-16T09:56:45.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.417+0000 7f3d00ff9700 1 -- 172.21.15.181:0/1339011109 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f3cf4005bd0 con 0x7f3d040cd2d0 2023-10-16T09:56:45.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.418+0000 7f3d00ff9700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 0x7f3cf006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.418+0000 7f3d037fe700 1 -- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 msgr2=0x7f3cf006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:45.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.418+0000 7f3d00ff9700 1 -- 172.21.15.181:0/1339011109 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7f3cf4089040 con 0x7f3d040cd2d0 2023-10-16T09:56:45.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.418+0000 7f3d037fe700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 0x7f3cf006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:45.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.422+0000 7f3d00ff9700 1 -- 172.21.15.181:0/1339011109 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3cf4053bb0 con 0x7f3d040cd2d0 2023-10-16T09:56:45.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.619+0000 7f3d037fe700 1 -- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 msgr2=0x7f3cf006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:45.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.619+0000 7f3d037fe700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 0x7f3cf006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:45.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.664+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d040ca6d0 con 0x7f3d040cd2d0 2023-10-16T09:56:45.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.664+0000 7f3d00ff9700 1 -- 172.21.15.181:0/1339011109 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f3cf4017090 con 0x7f3d040cd2d0 2023-10-16T09:56:45.665 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:45.666 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3974533571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3974533571},{"type":"v1","addr":"172.21.15.190:6834","nonce":3974533571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3064364132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3064364132},{"type":"v1","addr":"172.21.15.181:6837","nonce":3064364132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4021280674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4021280674},{"type":"v1","addr":"172.21.15.190:6835","nonce":4021280674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:56.185727+0000","modified":"2023-10-16T09:44:59.222716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6836/760205137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":760205137},{"type":"v1","addr":"172.21.15.181:6836","nonce":760205137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:56:45.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 msgr2=0x7f3cf006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:45.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 0x7f3cf006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040cd2d0 msgr2=0x7f3d040b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040cd2d0 0x7f3d040b42a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c6d0 tx=0x7f3cf400c9e0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 shutdown_connections 2023-10-16T09:56:45.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0415b990 0x7f3d04154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f3cf006b0b0 0x7f3cf006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d040cd2d0 0x7f3d040b42a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 --2- 172.21.15.181:0/1339011109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d040d0370 0x7f3d040b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 >> 172.21.15.181:0/1339011109 conn(0x7f3d0409a9a0 msgr2=0x7f3d0415abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:45.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 shutdown_connections 2023-10-16T09:56:45.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.668+0000 7f3d0a7f6700 1 -- 172.21.15.181:0/1339011109 wait complete. 2023-10-16T09:56:45.671 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2023-10-16T09:56:45.686 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:56:45.742 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:45.741+0000 7fd4d85a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:56:45.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.813+0000 7f8400857700 1 Processor -- start 2023-10-16T09:56:45.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.813+0000 7f8400857700 1 -- start start 2023-10-16T09:56:45.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.814+0000 7f8400857700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f83fc0d2390 0x7f83fc0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.814+0000 7f8400857700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc0cc470 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.814+0000 7f8400857700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc06ca30 con 0x7f83fc0d0180 2023-10-16T09:56:45.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.814+0000 7f8400857700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc06c810 con 0x7f83fc0d2390 2023-10-16T09:56:45.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.814+0000 7f83fa59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f83fc0ce7f0 0x7f83fc06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50636/0 (socket says 172.21.15.181:50636) 2023-10-16T09:56:45.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.814+0000 7f83fa59c700 1 -- 172.21.15.181:0/2701802067 learned_addr learned my addr 172.21.15.181:0/2701802067 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:45.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.814+0000 7f83f9d9b700 1 --2- 172.21.15.181:0/2701802067 >> v2:172.21.15.190:3300/0 conn(0x7f83fc0d2390 0x7f83fc0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f959a700 1 -- 172.21.15.181:0/2701802067 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588825102 0 0) 0x7f83fc0cc470 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f959a700 1 -- 172.21.15.181:0/2701802067 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e40035b0 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f9d9b700 1 -- 172.21.15.181:0/2701802067 >> v1:172.21.15.190:6789/0 conn(0x7f83fc0d0180 legacy=0x7f83fc0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f9d9b700 1 -- 172.21.15.181:0/2701802067 >> v1:172.21.15.181:6789/0 conn(0x7f83fc0ce7f0 legacy=0x7f83fc06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f9d9b700 1 -- 172.21.15.181:0/2701802067 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc06cd10 con 0x7f83fc0d2390 2023-10-16T09:56:45.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f9d9b700 1 --2- 172.21.15.181:0/2701802067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0cfc40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009840 tx=0x7f83f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57cd9025926d02b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:45.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f959a700 1 -- 172.21.15.181:0/2701802067 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0011830 con 0x7f83fc0d2390 2023-10-16T09:56:45.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f959a700 1 -- 172.21.15.181:0/2701802067 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0011990 con 0x7f83fc0d2390 2023-10-16T09:56:45.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.815+0000 7f83f959a700 1 -- 172.21.15.181:0/2701802067 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0011d00 con 0x7f83fc0d2390 2023-10-16T09:56:45.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.816+0000 7f8400857700 1 -- 172.21.15.181:0/2701802067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 msgr2=0x7f83fc0cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.816+0000 7f8400857700 1 --2- 172.21.15.181:0/2701802067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0cfc40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009840 tx=0x7f83f0009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:45.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.816+0000 7f8400857700 1 -- 172.21.15.181:0/2701802067 shutdown_connections 2023-10-16T09:56:45.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.816+0000 7f8400857700 1 --2- 172.21.15.181:0/2701802067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0cfc40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.816+0000 7f8400857700 1 -- 172.21.15.181:0/2701802067 >> 172.21.15.181:0/2701802067 conn(0x7f83fc09a9a0 msgr2=0x7f83fc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:45.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.816+0000 7f8400857700 1 -- 172.21.15.181:0/2701802067 shutdown_connections 2023-10-16T09:56:45.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.816+0000 7f8400857700 1 -- 172.21.15.181:0/2701802067 wait complete. 2023-10-16T09:56:45.822 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:45.819+0000 7fd4d85a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:56:45.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.817+0000 7f8400857700 1 Processor -- start 2023-10-16T09:56:45.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.817+0000 7f8400857700 1 -- start start 2023-10-16T09:56:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.818+0000 7f8400857700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83fc0ce7f0 0x7f83fc0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.818+0000 7f8400857700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83fc0d0180 0x7f83fc0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.818+0000 7f8400857700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.818+0000 7f8400857700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0c8560 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.818+0000 7f8400857700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc1587b0 con 0x7f83fc0d2390 2023-10-16T09:56:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.818+0000 7f8400857700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc157ba0 con 0x7f83fc0d0180 2023-10-16T09:56:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83fc0ce7f0 0x7f83fc0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83fc0ce7f0 0x7f83fc0c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39566/0 (socket says 172.21.15.181:39566) 2023-10-16T09:56:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 -- 172.21.15.181:0/114900029 learned_addr learned my addr 172.21.15.181:0/114900029 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:45.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fad9d700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83f9d9b700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83fc0d0180 0x7f83fc0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:45.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 -- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83fc0d0180 msgr2=0x7f83fc0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83fc0d0180 0x7f83fc0b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 -- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 msgr2=0x7f83fc0b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:45.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:45.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 -- 172.21.15.181:0/114900029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00094a0 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fad9d700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:45.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83fa59c700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83fc0ce7f0 0x7f83fc0c8020 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f83ec001260 tx=0x7f83ec00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:45.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83eaffd700 1 -- 172.21.15.181:0/114900029 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0074e0 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83eaffd700 1 -- 172.21.15.181:0/114900029 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec007640 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f83eaffd700 1 -- 172.21.15.181:0/114900029 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec005a50 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc157e80 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.819+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc158440 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.820+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc067d20 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.820+0000 7f83eaffd700 1 -- 172.21.15.181:0/114900029 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f83ec005bb0 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.821+0000 7f83eaffd700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 0x7f83dc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:45.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.821+0000 7f83f9d9b700 1 -- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 msgr2=0x7f83dc06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:45.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.821+0000 7f83eaffd700 1 -- 172.21.15.181:0/114900029 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7f83ec089040 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.821+0000 7f83f9d9b700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 0x7f83dc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:45.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:45.825+0000 7f83eaffd700 1 -- 172.21.15.181:0/114900029 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f83ec053b30 con 0x7f83fc0ce7f0 2023-10-16T09:56:45.885 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:45.884+0000 7f643894d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:56:45.963 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:45.962+0000 7fd4d85a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:56:45.963 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:45.963+0000 7f643894d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:56:46.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.022+0000 7f83f9d9b700 1 -- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 msgr2=0x7f83dc06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:46.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.022+0000 7f83f9d9b700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 0x7f83dc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:46.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.025+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83fc0cef40 con 0x7f83fc0ce7f0 2023-10-16T09:56:46.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.027+0000 7f83eaffd700 1 -- 172.21.15.181:0/114900029 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17682 (secure 0 0 0) 0x7f83ec007e60 con 0x7f83fc0ce7f0 2023-10-16T09:56:46.028 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:46.028 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":47,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:56:44.924958+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:44:54.175032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:55.152824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3764257785":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6833/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6832/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/4251066643":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2848692303":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2080914758":"2023-10-17T09:45:15.015302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":107,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:56:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 msgr2=0x7f83dc06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 0x7f83dc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83fc0ce7f0 msgr2=0x7f83fc0c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83fc0ce7f0 0x7f83fc0c8020 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f83ec001260 tx=0x7f83ec00c7b0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 shutdown_connections 2023-10-16T09:56:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f83dc06b0b0 0x7f83dc06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83fc0d2390 0x7f83fc0b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83fc0ce7f0 0x7f83fc0c8020 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 --2- 172.21.15.181:0/114900029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83fc0d0180 0x7f83fc0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 >> 172.21.15.181:0/114900029 conn(0x7f83fc09a9a0 msgr2=0x7f83fc070510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 shutdown_connections 2023-10-16T09:56:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.029+0000 7f8400857700 1 -- 172.21.15.181:0/114900029 wait complete. 2023-10-16T09:56:46.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:44:54.175032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:44:55.152824+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 221, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:56:46.043 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:56:46.122 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:46.121+0000 7f643894d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:56:46.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.177+0000 7fdffc939700 1 Processor -- start 2023-10-16T09:56:46.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.177+0000 7fdffc939700 1 -- start start 2023-10-16T09:56:46.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdffc939700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdff815da70 0x7fdff80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdffc939700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdff80ce6a0 0x7fdff806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdffc939700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80cef00 con 0x7fdff8072760 2023-10-16T09:56:46.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdffc939700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff8072b40 con 0x7fdff815da70 2023-10-16T09:56:46.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdffc939700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff80cec20 con 0x7fdff80ce6a0 2023-10-16T09:56:46.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdff659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdff815da70 0x7fdff80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdff6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdff8072760 0x7fdff81615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50672/0 (socket says 172.21.15.181:50672) 2023-10-16T09:56:46.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdff6d9d700 1 -- 172.21.15.181:0/2354793350 learned_addr learned my addr 172.21.15.181:0/2354793350 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:46.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdff659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdff815da70 0x7fdff80ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39590/0 (socket says 172.21.15.181:39590) 2023-10-16T09:56:46.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.178+0000 7fdff5d9b700 1 --2- 172.21.15.181:0/2354793350 >> v2:172.21.15.190:3300/0 conn(0x7fdff80ce6a0 0x7fdff806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff659c700 1 -- 172.21.15.181:0/2354793350 >> v2:172.21.15.190:3300/0 conn(0x7fdff80ce6a0 msgr2=0x7fdff806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:46.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff659c700 1 --2- 172.21.15.181:0/2354793350 >> v2:172.21.15.190:3300/0 conn(0x7fdff80ce6a0 0x7fdff806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff659c700 1 -- 172.21.15.181:0/2354793350 >> v1:172.21.15.181:6789/0 conn(0x7fdff8072760 legacy=0x7fdff81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff559a700 1 -- 172.21.15.181:0/2354793350 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782533073 0 0) 0x7fdff80cef00 con 0x7fdff8072760 2023-10-16T09:56:46.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff659c700 1 -- 172.21.15.181:0/2354793350 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff8071c80 con 0x7fdff815da70 2023-10-16T09:56:46.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff659c700 1 --2- 172.21.15.181:0/2354793350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80ce160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdff806ef60 tx=0x7fdfe8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd78eaaae538e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:46.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff559a700 1 -- 172.21.15.181:0/2354793350 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8011830 con 0x7fdff815da70 2023-10-16T09:56:46.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff559a700 1 -- 172.21.15.181:0/2354793350 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe8011990 con 0x7fdff815da70 2023-10-16T09:56:46.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.179+0000 7fdff559a700 1 -- 172.21.15.181:0/2354793350 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8011d20 con 0x7fdff815da70 2023-10-16T09:56:46.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 -- 172.21.15.181:0/2354793350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 msgr2=0x7fdff80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 --2- 172.21.15.181:0/2354793350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80ce160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdff806ef60 tx=0x7fdfe8009b20 comp rx=0 tx=0).stop 2023-10-16T09:56:46.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 -- 172.21.15.181:0/2354793350 shutdown_connections 2023-10-16T09:56:46.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 --2- 172.21.15.181:0/2354793350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80ce160 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 --2- 172.21.15.181:0/2354793350 >> v2:172.21.15.190:3300/0 conn(0x7fdff80ce6a0 0x7fdff806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 -- 172.21.15.181:0/2354793350 >> 172.21.15.181:0/2354793350 conn(0x7fdff80cd010 msgr2=0x7fdff80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:46.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 -- 172.21.15.181:0/2354793350 shutdown_connections 2023-10-16T09:56:46.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.180+0000 7fdffc939700 1 -- 172.21.15.181:0/2354793350 wait complete. 2023-10-16T09:56:46.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.182+0000 7fdffc939700 1 Processor -- start 2023-10-16T09:56:46.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.182+0000 7fdffc939700 1 -- start start 2023-10-16T09:56:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.182+0000 7fdffc939700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff8072760 0x7fdff8151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdffc939700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdff80ce6a0 0x7fdff80b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdffc939700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdffc939700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff8152010 con 0x7fdff815da70 2023-10-16T09:56:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdffc939700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80c6a30 con 0x7fdff8072760 2023-10-16T09:56:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdffc939700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80c6190 con 0x7fdff80ce6a0 2023-10-16T09:56:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff8072760 0x7fdff8151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39600/0 (socket says 172.21.15.181:39600) 2023-10-16T09:56:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 -- 172.21.15.181:0/1923302846 learned_addr learned my addr 172.21.15.181:0/1923302846 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff5d9b700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdff80ce6a0 0x7fdff80b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 -- 172.21.15.181:0/1923302846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdff80ce6a0 msgr2=0x7fdff80b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdff80ce6a0 0x7fdff80b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 -- 172.21.15.181:0/1923302846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff8072760 msgr2=0x7fdff8151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff8072760 0x7fdff8151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 -- 172.21.15.181:0/1923302846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe80094a0 con 0x7fdff815da70 2023-10-16T09:56:46.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdff6d9d700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80b25f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdff000cba0 tx=0x7fdff000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:46.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdfe6ffd700 1 -- 172.21.15.181:0/1923302846 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000aaa0 con 0x7fdff815da70 2023-10-16T09:56:46.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdfe6ffd700 1 -- 172.21.15.181:0/1923302846 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff000ac20 con 0x7fdff815da70 2023-10-16T09:56:46.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff80c6470 con 0x7fdff815da70 2023-10-16T09:56:46.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdfe6ffd700 1 -- 172.21.15.181:0/1923302846 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00070d0 con 0x7fdff815da70 2023-10-16T09:56:46.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.183+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff815ebe0 con 0x7fdff815da70 2023-10-16T09:56:46.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.185+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff815ea40 con 0x7fdff815da70 2023-10-16T09:56:46.192 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:46.187+0000 7fd4d85a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:56:46.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.186+0000 7fdfe6ffd700 1 -- 172.21.15.181:0/1923302846 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fdff0007230 con 0x7fdff815da70 2023-10-16T09:56:46.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.186+0000 7fdfe6ffd700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 0x7fdfd806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.186+0000 7fdff659c700 1 -- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 msgr2=0x7fdfd806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:46.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.186+0000 7fdfe6ffd700 1 -- 172.21.15.181:0/1923302846 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7fdff00898c0 con 0x7fdff815da70 2023-10-16T09:56:46.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.186+0000 7fdff659c700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 0x7fdfd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:46.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.191+0000 7fdfe6ffd700 1 -- 172.21.15.181:0/1923302846 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdff815ea40 con 0x7fdff815da70 2023-10-16T09:56:46.261 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:46.260+0000 7fd4d85a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:56:46.357 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:46.356+0000 7f643894d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:56:46.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.387+0000 7fdff659c700 1 -- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 msgr2=0x7fdfd806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:46.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.387+0000 7fdff659c700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 0x7fdfd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:46.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.389+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdff815ea40 con 0x7fdff815da70 2023-10-16T09:56:46.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.389+0000 7fdfe6ffd700 1 -- 172.21.15.181:0/1923302846 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdff0018020 con 0x7fdff815da70 2023-10-16T09:56:46.390 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:56:46.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.391+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 msgr2=0x7fdfd806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:46.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.391+0000 7fdffc939700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 0x7fdfd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.391+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 msgr2=0x7fdff80b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.391+0000 7fdffc939700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80b25f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdff000cba0 tx=0x7fdff000ceb0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.391+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 shutdown_connections 2023-10-16T09:56:46.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.391+0000 7fdffc939700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7fdfd806b0b0 0x7fdfd806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.391+0000 7fdffc939700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff815da70 0x7fdff80b25f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.392+0000 7fdffc939700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdff8072760 0x7fdff8151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.392+0000 7fdffc939700 1 --2- 172.21.15.181:0/1923302846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdff80ce6a0 0x7fdff80b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.392+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 >> 172.21.15.181:0/1923302846 conn(0x7fdff80cd010 msgr2=0x7fdff80b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:46.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.392+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 shutdown_connections 2023-10-16T09:56:46.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.392+0000 7fdffc939700 1 -- 172.21.15.181:0/1923302846 wait complete. 2023-10-16T09:56:46.405 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:56:46.435 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:46.434+0000 7f643894d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:56:46.524 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:46.522+0000 7fd4d85a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:56:46.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.534+0000 7f50de685700 1 Processor -- start 2023-10-16T09:56:46.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.535+0000 7f50de685700 1 -- start start 2023-10-16T09:56:46.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.535+0000 7f50de685700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50d815dbd0 0x7f50d80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.535+0000 7f50de685700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50d80d0610 0x7f50d80cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.535+0000 7f50de685700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d815dfb0 con 0x7f50d80ce2c0 2023-10-16T09:56:46.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.535+0000 7f50de685700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d80ce770 con 0x7f50d815dbd0 2023-10-16T09:56:46.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.535+0000 7f50de685700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d80cd3a0 con 0x7f50d80d0610 2023-10-16T09:56:46.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50d815dbd0 0x7f50d80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50d815dbd0 0x7f50d80d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39606/0 (socket says 172.21.15.181:39606) 2023-10-16T09:56:46.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 -- 172.21.15.181:0/1279187709 learned_addr learned my addr 172.21.15.181:0/1279187709 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:46.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d77fe700 1 --2- 172.21.15.181:0/1279187709 >> v2:172.21.15.190:3300/0 conn(0x7f50d80d0610 0x7f50d80cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 -- 172.21.15.181:0/1279187709 >> v2:172.21.15.190:3300/0 conn(0x7f50d80d0610 msgr2=0x7f50d80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:46.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 --2- 172.21.15.181:0/1279187709 >> v2:172.21.15.190:3300/0 conn(0x7f50d80d0610 0x7f50d80cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 -- 172.21.15.181:0/1279187709 >> v1:172.21.15.190:6789/0 conn(0x7f50d80ce2c0 legacy=0x7f50d8161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 -- 172.21.15.181:0/1279187709 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80cd830 con 0x7f50d815dbd0 2023-10-16T09:56:46.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d7fff700 1 --2- 172.21.15.181:0/1279187709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d80d00d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f50d80cb990 tx=0x7f50c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd0129997ae25e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:46.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d6ffd700 1 -- 172.21.15.181:0/1279187709 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8011830 con 0x7f50d815dbd0 2023-10-16T09:56:46.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d6ffd700 1 -- 172.21.15.181:0/1279187709 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8011990 con 0x7f50d815dbd0 2023-10-16T09:56:46.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.536+0000 7f50d6ffd700 1 -- 172.21.15.181:0/1279187709 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8011d20 con 0x7f50d815dbd0 2023-10-16T09:56:46.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 -- 172.21.15.181:0/1279187709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 msgr2=0x7f50d80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 --2- 172.21.15.181:0/1279187709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d80d00d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f50d80cb990 tx=0x7f50c8009bd0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 -- 172.21.15.181:0/1279187709 shutdown_connections 2023-10-16T09:56:46.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 --2- 172.21.15.181:0/1279187709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d80d00d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 --2- 172.21.15.181:0/1279187709 >> v2:172.21.15.190:3300/0 conn(0x7f50d80d0610 0x7f50d80cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 -- 172.21.15.181:0/1279187709 >> 172.21.15.181:0/1279187709 conn(0x7f50d809a930 msgr2=0x7f50d80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:46.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 -- 172.21.15.181:0/1279187709 shutdown_connections 2023-10-16T09:56:46.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.537+0000 7f50de685700 1 -- 172.21.15.181:0/1279187709 wait complete. 2023-10-16T09:56:46.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.538+0000 7f50de685700 1 Processor -- start 2023-10-16T09:56:46.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.538+0000 7f50de685700 1 -- start start 2023-10-16T09:56:46.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50de685700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d80ce2c0 0x7f50d81f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50de685700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d80d0610 0x7f50d815fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50de685700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d81602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50de685700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80c9cb0 con 0x7f50d815dbd0 2023-10-16T09:56:46.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50de685700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80c90e0 con 0x7f50d80d0610 2023-10-16T09:56:46.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50de685700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80c9380 con 0x7f50d80ce2c0 2023-10-16T09:56:46.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50dcc22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d81602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50d77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d80d0610 0x7f50d815fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50dcc22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d81602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39616/0 (socket says 172.21.15.181:39616) 2023-10-16T09:56:46.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50dcc22700 1 -- 172.21.15.181:0/919975326 learned_addr learned my addr 172.21.15.181:0/919975326 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:46.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.539+0000 7f50d7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d80ce2c0 0x7f50d81f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50dcc22700 1 -- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d80ce2c0 msgr2=0x7f50d81f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50dcc22700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d80ce2c0 0x7f50d81f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50dcc22700 1 -- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d80d0610 msgr2=0x7f50d815fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50dcc22700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d80d0610 0x7f50d815fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50dcc22700 1 -- 172.21.15.181:0/919975326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8009550 con 0x7f50d815dbd0 2023-10-16T09:56:46.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50d77fe700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d80d0610 0x7f50d815fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:46.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50d7fff700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d80ce2c0 0x7f50d81f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:46.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50dcc22700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d81602a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f50d000cb50 tx=0x7f50d000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:46.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50d4ff9700 1 -- 172.21.15.181:0/919975326 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0003c00 con 0x7f50d815dbd0 2023-10-16T09:56:46.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50d4ff9700 1 -- 172.21.15.181:0/919975326 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d0003d60 con 0x7f50d815dbd0 2023-10-16T09:56:46.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d80c9660 con 0x7f50d815dbd0 2023-10-16T09:56:46.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50d4ff9700 1 -- 172.21.15.181:0/919975326 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d000aa20 con 0x7f50d815dbd0 2023-10-16T09:56:46.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.540+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d81f3220 con 0x7f50d815dbd0 2023-10-16T09:56:46.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.541+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d80c9e20 con 0x7f50d815dbd0 2023-10-16T09:56:46.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.541+0000 7f50d4ff9700 1 -- 172.21.15.181:0/919975326 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f50d000ab80 con 0x7f50d815dbd0 2023-10-16T09:56:46.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.542+0000 7f50d4ff9700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 0x7f50c406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.542+0000 7f50d7fff700 1 -- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 msgr2=0x7f50c406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:46.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.542+0000 7f50d4ff9700 1 -- 172.21.15.181:0/919975326 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7f50d0089ab0 con 0x7f50d815dbd0 2023-10-16T09:56:46.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.542+0000 7f50d7fff700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 0x7f50c406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:46.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.546+0000 7f50d4ff9700 1 -- 172.21.15.181:0/919975326 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50d0054580 con 0x7f50d815dbd0 2023-10-16T09:56:46.712 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:46.711+0000 7f643894d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:56:46.741 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:46.740+0000 7fd4d85a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:56:46.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.743+0000 7f50d7fff700 1 -- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 msgr2=0x7f50c406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:46.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.743+0000 7f50d7fff700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 0x7f50c406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:46.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.745+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50d8067d20 con 0x7f50d815dbd0 2023-10-16T09:56:46.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.746+0000 7f50d4ff9700 1 -- 172.21.15.181:0/919975326 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f50d0057ba0 con 0x7f50d815dbd0 2023-10-16T09:56:46.747 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:56:46.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 msgr2=0x7f50c406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:46.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 0x7f50c406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 msgr2=0x7f50d81602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d81602a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f50d000cb50 tx=0x7f50d000ce60 comp rx=0 tx=0).stop 2023-10-16T09:56:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 shutdown_connections 2023-10-16T09:56:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d80ce2c0 0x7f50d81f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f50c406b0b0 0x7f50c406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d815dbd0 0x7f50d81602a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 --2- 172.21.15.181:0/919975326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d80d0610 0x7f50d815fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 >> 172.21.15.181:0/919975326 conn(0x7f50d809a930 msgr2=0x7f50d8070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 shutdown_connections 2023-10-16T09:56:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.748+0000 7f50de685700 1 -- 172.21.15.181:0/919975326 wait complete. 2023-10-16T09:56:46.761 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:56:46.886 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:46.885+0000 7fd4d85a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:56:46.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.930+0000 7f466cba5700 1 Processor -- start 2023-10-16T09:56:46.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.930+0000 7f466cba5700 1 -- start start 2023-10-16T09:56:46.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466cba5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46680ce6c0 0x7f46680cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466cba5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46680b1f70 con 0x7f46680cc6a0 2023-10-16T09:56:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466cba5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46680b2ae0 con 0x7f466815f150 2023-10-16T09:56:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466cba5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46680b2dc0 con 0x7f46680ce6c0 2023-10-16T09:56:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f4666d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f46680cc6a0 0x7f46680ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50704/0 (socket says 172.21.15.181:50704) 2023-10-16T09:56:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f4666d9d700 1 -- 172.21.15.181:0/2687648852 learned_addr learned my addr 172.21.15.181:0/2687648852 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f4665d9b700 1 --2- 172.21.15.181:0/2687648852 >> v2:172.21.15.190:3300/0 conn(0x7f46680ce6c0 0x7f46680cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466559a700 1 -- 172.21.15.181:0/2687648852 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221151744 0 0) 0x7f46680b1f70 con 0x7f46680cc6a0 2023-10-16T09:56:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466559a700 1 -- 172.21.15.181:0/2687648852 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46500035b0 con 0x7f46680cc6a0 2023-10-16T09:56:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466559a700 1 -- 172.21.15.181:0/2687648852 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611568439 0 0) 0x7f46680b2ae0 con 0x7f466815f150 2023-10-16T09:56:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f466559a700 1 -- 172.21.15.181:0/2687648852 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46680b1f70 con 0x7f466815f150 2023-10-16T09:56:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f4665d9b700 1 -- 172.21.15.181:0/2687648852 >> v1:172.21.15.190:6789/0 conn(0x7f466815f150 legacy=0x7f46680cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f4665d9b700 1 -- 172.21.15.181:0/2687648852 >> v1:172.21.15.181:6789/0 conn(0x7f46680cc6a0 legacy=0x7f46680ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.931+0000 7f4665d9b700 1 -- 172.21.15.181:0/2687648852 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46680cf9d0 con 0x7f46680ce6c0 2023-10-16T09:56:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f4665d9b700 1 --2- 172.21.15.181:0/2687648852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 0x7f46680cc160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f465c009840 tx=0x7f465c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89542d9945acd1a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466559a700 1 -- 172.21.15.181:0/2687648852 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011830 con 0x7f46680ce6c0 2023-10-16T09:56:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466559a700 1 -- 172.21.15.181:0/2687648852 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c011990 con 0x7f46680ce6c0 2023-10-16T09:56:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466559a700 1 -- 172.21.15.181:0/2687648852 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011d20 con 0x7f46680ce6c0 2023-10-16T09:56:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466cba5700 1 -- 172.21.15.181:0/2687648852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 msgr2=0x7f46680cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466cba5700 1 --2- 172.21.15.181:0/2687648852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 0x7f46680cc160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f465c009840 tx=0x7f465c009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466cba5700 1 -- 172.21.15.181:0/2687648852 shutdown_connections 2023-10-16T09:56:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466cba5700 1 --2- 172.21.15.181:0/2687648852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 0x7f46680cc160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.932+0000 7f466cba5700 1 -- 172.21.15.181:0/2687648852 >> 172.21.15.181:0/2687648852 conn(0x7f466809a930 msgr2=0x7f46680d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.933+0000 7f466cba5700 1 -- 172.21.15.181:0/2687648852 shutdown_connections 2023-10-16T09:56:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.933+0000 7f466cba5700 1 -- 172.21.15.181:0/2687648852 wait complete. 2023-10-16T09:56:46.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.934+0000 7f466cba5700 1 Processor -- start 2023-10-16T09:56:46.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.934+0000 7f466cba5700 1 -- start start 2023-10-16T09:56:46.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466cba5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46680cc6a0 0x7f46681f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466cba5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 0x7f46681f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466cba5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466815f150 0x7f46680c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466cba5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46681593c0 con 0x7f466815f150 2023-10-16T09:56:46.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466cba5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46680b3190 con 0x7f46680ce6c0 2023-10-16T09:56:46.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466cba5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46680c7f60 con 0x7f46680cc6a0 2023-10-16T09:56:46.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466815f150 0x7f46680c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466815f150 0x7f46680c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39632/0 (socket says 172.21.15.181:39632) 2023-10-16T09:56:46.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 -- 172.21.15.181:0/1541310437 learned_addr learned my addr 172.21.15.181:0/1541310437 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:46.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4665d9b700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 0x7f46681f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466659c700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46680cc6a0 0x7f46681f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:46.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 -- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46680cc6a0 msgr2=0x7f46681f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46680cc6a0 0x7f46681f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 -- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 msgr2=0x7f46681f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:46.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 0x7f46681f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:46.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 -- 172.21.15.181:0/1541310437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0094a0 con 0x7f466815f150 2023-10-16T09:56:46.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4666d9d700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466815f150 0x7f46680c7a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f466000cba0 tx=0x7f466000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:46.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4656ffd700 1 -- 172.21.15.181:0/1541310437 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000aa40 con 0x7f466815f150 2023-10-16T09:56:46.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4656ffd700 1 -- 172.21.15.181:0/1541310437 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466000aba0 con 0x7f466815f150 2023-10-16T09:56:46.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46680c8c40 con 0x7f466815f150 2023-10-16T09:56:46.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.935+0000 7f4656ffd700 1 -- 172.21.15.181:0/1541310437 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660012400 con 0x7f466815f150 2023-10-16T09:56:46.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.936+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46680c9200 con 0x7f466815f150 2023-10-16T09:56:46.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.937+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4668067d20 con 0x7f466815f150 2023-10-16T09:56:46.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.937+0000 7f4656ffd700 1 -- 172.21.15.181:0/1541310437 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f466001b3b0 con 0x7f466815f150 2023-10-16T09:56:46.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.937+0000 7f4656ffd700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 0x7f464806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:46.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.937+0000 7f466659c700 1 -- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 msgr2=0x7f464806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:46.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.937+0000 7f4656ffd700 1 -- 172.21.15.181:0/1541310437 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7f46600898a0 con 0x7f466815f150 2023-10-16T09:56:46.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.938+0000 7f466659c700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 0x7f464806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:46.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:46.942+0000 7f4656ffd700 1 -- 172.21.15.181:0/1541310437 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4660054410 con 0x7f466815f150 2023-10-16T09:56:46.947 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:46.945+0000 7f643894d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:56:46.999 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:46.998+0000 7fd4d85a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:56:47.101 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:47.100+0000 7f643894d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:56:47.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.139+0000 7f466659c700 1 -- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 msgr2=0x7f464806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:47.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.139+0000 7f466659c700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 0x7f464806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:47.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.142+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46680af290 con 0x7f466815f150 2023-10-16T09:56:47.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.142+0000 7f4656ffd700 1 -- 172.21.15.181:0/1541310437 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4660057a30 con 0x7f466815f150 2023-10-16T09:56:47.143 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:56:47.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 msgr2=0x7f464806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:47.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 0x7f464806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466815f150 msgr2=0x7f46680c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466815f150 0x7f46680c7a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f466000cba0 tx=0x7f466000ceb0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 shutdown_connections 2023-10-16T09:56:47.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46680cc6a0 0x7f46681f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f464806b0b0 0x7f464806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f466815f150 0x7f46680c7a20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 --2- 172.21.15.181:0/1541310437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46680ce6c0 0x7f46681f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 >> 172.21.15.181:0/1541310437 conn(0x7f466809a930 msgr2=0x7f4668070660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:47.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 shutdown_connections 2023-10-16T09:56:47.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.145+0000 7f466cba5700 1 -- 172.21.15.181:0/1541310437 wait complete. 2023-10-16T09:56:47.151 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:47.150+0000 7fd4d85a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:56:47.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:56:47.221 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:47.220+0000 7f643894d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:56:47.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.280+0000 7f9380c0a700 1 Processor -- start 2023-10-16T09:56:47.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.281+0000 7f9380c0a700 1 -- start start 2023-10-16T09:56:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.281+0000 7f9380c0a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c15dc40 0x7f937c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.281+0000 7f9380c0a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f937c0c2dc0 0x7f937c15e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.281+0000 7f9380c0a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937c0b2710 con 0x7f937c06ade0 2023-10-16T09:56:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.281+0000 7f9380c0a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937c0c18f0 con 0x7f937c0c2dc0 2023-10-16T09:56:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.281+0000 7f9380c0a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0c1bd0 con 0x7f937c15dc40 2023-10-16T09:56:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f937c0c2dc0 0x7f937c15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f937c0c2dc0 0x7f937c15e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39642/0 (socket says 172.21.15.181:39642) 2023-10-16T09:56:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 -- 172.21.15.181:0/3420265969 learned_addr learned my addr 172.21.15.181:0/3420265969 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937a59c700 1 --2- 172.21.15.181:0/3420265969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c15dc40 0x7f937c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 -- 172.21.15.181:0/3420265969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c15dc40 msgr2=0x7f937c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 --2- 172.21.15.181:0/3420265969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c15dc40 0x7f937c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 -- 172.21.15.181:0/3420265969 >> v1:172.21.15.190:6789/0 conn(0x7f937c06ade0 legacy=0x7f937c0c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 -- 172.21.15.181:0/3420265969 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0c1eb0 con 0x7f937c0c2dc0 2023-10-16T09:56:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937959a700 1 -- 172.21.15.181:0/3420265969 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150487044 0 0) 0x7f9370001110 con 0x7f937c06ade0 2023-10-16T09:56:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937ad9d700 1 --2- 172.21.15.181:0/3420265969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c0c2dc0 0x7f937c15e020 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9374009e50 tx=0x7f937400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e933c33ce5eb61b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937959a700 1 -- 172.21.15.181:0/3420265969 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400e070 con 0x7f937c0c2dc0 2023-10-16T09:56:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937959a700 1 -- 172.21.15.181:0/3420265969 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937400b910 con 0x7f937c0c2dc0 2023-10-16T09:56:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.282+0000 7f937959a700 1 -- 172.21.15.181:0/3420265969 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937400bcc0 con 0x7f937c0c2dc0 2023-10-16T09:56:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 -- 172.21.15.181:0/3420265969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c0c2dc0 msgr2=0x7f937c15e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 --2- 172.21.15.181:0/3420265969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c0c2dc0 0x7f937c15e020 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9374009e50 tx=0x7f937400b330 comp rx=0 tx=0).stop 2023-10-16T09:56:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 -- 172.21.15.181:0/3420265969 shutdown_connections 2023-10-16T09:56:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 --2- 172.21.15.181:0/3420265969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c0c2dc0 0x7f937c15e020 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 --2- 172.21.15.181:0/3420265969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c15dc40 0x7f937c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 -- 172.21.15.181:0/3420265969 >> 172.21.15.181:0/3420265969 conn(0x7f937c0b1780 msgr2=0x7f937c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 -- 172.21.15.181:0/3420265969 shutdown_connections 2023-10-16T09:56:47.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.283+0000 7f9380c0a700 1 -- 172.21.15.181:0/3420265969 wait complete. 2023-10-16T09:56:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.284+0000 7f9380c0a700 1 Processor -- start 2023-10-16T09:56:47.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.284+0000 7f9380c0a700 1 -- start start 2023-10-16T09:56:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f9380c0a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c06ade0 0x7f937c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f9380c0a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0c2dc0 0x7f937c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f9380c0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c15dc40 0x7f937c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f9380c0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c06edc0 con 0x7f937c15dc40 2023-10-16T09:56:47.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f9380c0a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c159490 con 0x7f937c0c2dc0 2023-10-16T09:56:47.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f9380c0a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0b2bb0 con 0x7f937c06ade0 2023-10-16T09:56:47.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f937ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c15dc40 0x7f937c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f937ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c15dc40 0x7f937c06df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39648/0 (socket says 172.21.15.181:39648) 2023-10-16T09:56:47.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f937ad9d700 1 -- 172.21.15.181:0/498509478 learned_addr learned my addr 172.21.15.181:0/498509478 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:47.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f937a59c700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c06ade0 0x7f937c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.285+0000 7f9379d9b700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0c2dc0 0x7f937c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f937ad9d700 1 -- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c06ade0 msgr2=0x7f937c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f937ad9d700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c06ade0 0x7f937c159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f937ad9d700 1 -- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0c2dc0 msgr2=0x7f937c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f937ad9d700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0c2dc0 0x7f937c153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f937ad9d700 1 -- 172.21.15.181:0/498509478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9374009b30 con 0x7f937c15dc40 2023-10-16T09:56:47.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f937ad9d700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c15dc40 0x7f937c06df70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f937400b680 tx=0x7f9374004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:47.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f936affd700 1 -- 172.21.15.181:0/498509478 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93740099b0 con 0x7f937c15dc40 2023-10-16T09:56:47.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f936affd700 1 -- 172.21.15.181:0/498509478 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93740092e0 con 0x7f937c15dc40 2023-10-16T09:56:47.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c06eae0 con 0x7f937c15dc40 2023-10-16T09:56:47.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f936affd700 1 -- 172.21.15.181:0/498509478 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374004470 con 0x7f937c15dc40 2023-10-16T09:56:47.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.286+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c159120 con 0x7f937c15dc40 2023-10-16T09:56:47.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.287+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937c158ca0 con 0x7f937c15dc40 2023-10-16T09:56:47.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.288+0000 7f936affd700 1 -- 172.21.15.181:0/498509478 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f93740045d0 con 0x7f937c15dc40 2023-10-16T09:56:47.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.288+0000 7f936affd700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 0x7f935c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.288+0000 7f937a59c700 1 -- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 msgr2=0x7f935c06d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:47.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.288+0000 7f936affd700 1 -- 172.21.15.181:0/498509478 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7f937408b060 con 0x7f937c15dc40 2023-10-16T09:56:47.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.288+0000 7f937a59c700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 0x7f935c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:47.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.292+0000 7f936affd700 1 -- 172.21.15.181:0/498509478 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f937c158ca0 con 0x7f937c15dc40 2023-10-16T09:56:47.383 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:47.383+0000 7f643894d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:56:47.411 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:47.410+0000 7fd4d85a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:56:47.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.489+0000 7f937a59c700 1 -- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 msgr2=0x7f935c06d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:47.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.489+0000 7f937a59c700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 0x7f935c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:47.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.538+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f937c158ca0 con 0x7f937c15dc40 2023-10-16T09:56:47.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.538+0000 7f936affd700 1 -- 172.21.15.181:0/498509478 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f937c158ca0 con 0x7f937c15dc40 2023-10-16T09:56:47.539 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:47.539 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3974533571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3974533571},{"type":"v1","addr":"172.21.15.190:6834","nonce":3974533571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3064364132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3064364132},{"type":"v1","addr":"172.21.15.181:6837","nonce":3064364132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4021280674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4021280674},{"type":"v1","addr":"172.21.15.190:6835","nonce":4021280674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:56.185727+0000","modified":"2023-10-16T09:44:59.222716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6836/760205137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":760205137},{"type":"v1","addr":"172.21.15.181:6836","nonce":760205137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:56:47.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.540+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 msgr2=0x7f935c06d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.540+0000 7f9380c0a700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 0x7f935c06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c15dc40 msgr2=0x7f937c06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c15dc40 0x7f937c06df70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f937400b680 tx=0x7f9374004b20 comp rx=0 tx=0).stop 2023-10-16T09:56:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 shutdown_connections 2023-10-16T09:56:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f937c06ade0 0x7f937c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f935c06b0e0 0x7f935c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f937c15dc40 0x7f937c06df70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 --2- 172.21.15.181:0/498509478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0c2dc0 0x7f937c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 >> 172.21.15.181:0/498509478 conn(0x7f937c0b1780 msgr2=0x7f937c165670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 shutdown_connections 2023-10-16T09:56:47.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.541+0000 7f9380c0a700 1 -- 172.21.15.181:0/498509478 wait complete. 2023-10-16T09:56:47.544 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2023-10-16T09:56:47.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:56:47.566 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:47.565+0000 7fd4d85a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:56:47.657 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:47.657+0000 7f643894d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:56:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.676+0000 7f25c22c2700 1 Processor -- start 2023-10-16T09:56:47.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.676+0000 7f25c22c2700 1 -- start start 2023-10-16T09:56:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25c22c2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25bc15e6d0 0x7f25bc0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25c22c2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f25bc0cb0c0 0x7f25bc15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25c22c2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25bc06ed00 con 0x7f25bc15eab0 2023-10-16T09:56:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25c22c2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc0d02d0 con 0x7f25bc15e6d0 2023-10-16T09:56:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25c22c2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc15ad40 con 0x7f25bc0cb0c0 2023-10-16T09:56:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25bbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25bc15e6d0 0x7f25bc0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25bbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25bc15e6d0 0x7f25bc0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39656/0 (socket says 172.21.15.181:39656) 2023-10-16T09:56:47.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.677+0000 7f25bbfff700 1 -- 172.21.15.181:0/3678135410 learned_addr learned my addr 172.21.15.181:0/3678135410 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:47.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25c085f700 1 --2- 172.21.15.181:0/3678135410 >> v2:172.21.15.190:3300/0 conn(0x7f25bc0cb0c0 0x7f25bc15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25bbfff700 1 -- 172.21.15.181:0/3678135410 >> v2:172.21.15.190:3300/0 conn(0x7f25bc0cb0c0 msgr2=0x7f25bc15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:47.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25bbfff700 1 --2- 172.21.15.181:0/3678135410 >> v2:172.21.15.190:3300/0 conn(0x7f25bc0cb0c0 0x7f25bc15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25bbfff700 1 -- 172.21.15.181:0/3678135410 >> v1:172.21.15.190:6789/0 conn(0x7f25bc15eab0 legacy=0x7f25bc15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25bbfff700 1 -- 172.21.15.181:0/3678135410 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc06e0f0 con 0x7f25bc15e6d0 2023-10-16T09:56:47.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25bbfff700 1 --2- 172.21.15.181:0/3678135410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15e6d0 0x7f25bc0d0fc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f25bc071650 tx=0x7f25ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5b0d9c004b2dfe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:47.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25baffd700 1 -- 172.21.15.181:0/3678135410 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac005e30 con 0x7f25bc15e6d0 2023-10-16T09:56:47.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25baffd700 1 -- 172.21.15.181:0/3678135410 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac004030 con 0x7f25bc15e6d0 2023-10-16T09:56:47.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.678+0000 7f25baffd700 1 -- 172.21.15.181:0/3678135410 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac0043e0 con 0x7f25bc15e6d0 2023-10-16T09:56:47.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 -- 172.21.15.181:0/3678135410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15e6d0 msgr2=0x7f25bc0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 --2- 172.21.15.181:0/3678135410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15e6d0 0x7f25bc0d0fc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f25bc071650 tx=0x7f25ac009bd0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 -- 172.21.15.181:0/3678135410 shutdown_connections 2023-10-16T09:56:47.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 --2- 172.21.15.181:0/3678135410 >> v2:172.21.15.190:3300/0 conn(0x7f25bc0cb0c0 0x7f25bc15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 --2- 172.21.15.181:0/3678135410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15e6d0 0x7f25bc0d0fc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 -- 172.21.15.181:0/3678135410 >> 172.21.15.181:0/3678135410 conn(0x7f25bc0d04a0 msgr2=0x7f25bc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:47.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 -- 172.21.15.181:0/3678135410 shutdown_connections 2023-10-16T09:56:47.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.679+0000 7f25c22c2700 1 -- 172.21.15.181:0/3678135410 wait complete. 2023-10-16T09:56:47.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.680+0000 7f25c22c2700 1 Processor -- start 2023-10-16T09:56:47.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.680+0000 7f25c22c2700 1 -- start start 2023-10-16T09:56:47.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c22c2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc0cb0c0 0x7f25bc14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c22c2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25bc15e6d0 0x7f25bc0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c22c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15eab0 0x7f25bc15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c22c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0c9550 con 0x7f25bc15eab0 2023-10-16T09:56:47.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c22c2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0c88d0 con 0x7f25bc0cb0c0 2023-10-16T09:56:47.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c22c2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0c8bb0 con 0x7f25bc15e6d0 2023-10-16T09:56:47.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15eab0 0x7f25bc15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15eab0 0x7f25bc15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39664/0 (socket says 172.21.15.181:39664) 2023-10-16T09:56:47.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 -- 172.21.15.181:0/2659340879 learned_addr learned my addr 172.21.15.181:0/2659340879 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:47.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25bbfff700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc0cb0c0 0x7f25bc14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25bb7fe700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25bc15e6d0 0x7f25bc0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:47.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 -- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25bc15e6d0 msgr2=0x7f25bc0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25bc15e6d0 0x7f25bc0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 -- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc0cb0c0 msgr2=0x7f25bc14c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc0cb0c0 0x7f25bc14c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 -- 172.21.15.181:0/2659340879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac009500 con 0x7f25bc15eab0 2023-10-16T09:56:47.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25bbfff700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc0cb0c0 0x7f25bc14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:47.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.681+0000 7f25c085f700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15eab0 0x7f25bc15ae80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f25b400efc0 tx=0x7f25b400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:47.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.682+0000 7f25b8ff9700 1 -- 172.21.15.181:0/2659340879 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b400cad0 con 0x7f25bc15eab0 2023-10-16T09:56:47.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.682+0000 7f25b8ff9700 1 -- 172.21.15.181:0/2659340879 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b400f040 con 0x7f25bc15eab0 2023-10-16T09:56:47.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.682+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc0c8e90 con 0x7f25bc15eab0 2023-10-16T09:56:47.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.682+0000 7f25b8ff9700 1 -- 172.21.15.181:0/2659340879 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b400ce70 con 0x7f25bc15eab0 2023-10-16T09:56:47.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.682+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc14cca0 con 0x7f25bc15eab0 2023-10-16T09:56:47.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.683+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc067d20 con 0x7f25bc15eab0 2023-10-16T09:56:47.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.683+0000 7f25b8ff9700 1 -- 172.21.15.181:0/2659340879 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f25b4014480 con 0x7f25bc15eab0 2023-10-16T09:56:47.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.683+0000 7f25b8ff9700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 0x7f25a806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:47.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.683+0000 7f25bbfff700 1 -- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 msgr2=0x7f25a806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:47.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.683+0000 7f25b8ff9700 1 -- 172.21.15.181:0/2659340879 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7f25b408a2b0 con 0x7f25bc15eab0 2023-10-16T09:56:47.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.683+0000 7f25bbfff700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 0x7f25a806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:47.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.688+0000 7f25b8ff9700 1 -- 172.21.15.181:0/2659340879 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f25b4054e80 con 0x7f25bc15eab0 2023-10-16T09:56:47.826 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:47.825+0000 7f643894d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:56:47.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.884+0000 7f25bbfff700 1 -- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 msgr2=0x7f25a806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:47.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.884+0000 7f25bbfff700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 0x7f25a806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:47.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.888+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25bc0cb610 con 0x7f25bc15eab0 2023-10-16T09:56:47.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.890+0000 7f25b8ff9700 1 -- 172.21.15.181:0/2659340879 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17682 (secure 0 0 0) 0x7f25b40584a0 con 0x7f25bc15eab0 2023-10-16T09:56:47.890 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:47.891 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":47,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:56:44.924958+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:44:54.175032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:55.152824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3764257785":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6833/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6832/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/4251066643":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2848692303":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2080914758":"2023-10-17T09:45:15.015302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":107,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:56:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 msgr2=0x7f25a806d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 0x7f25a806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15eab0 msgr2=0x7f25bc15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15eab0 0x7f25bc15ae80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f25b400efc0 tx=0x7f25b400c0c0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 shutdown_connections 2023-10-16T09:56:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f25a806b100 0x7f25a806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25bc15eab0 0x7f25bc15ae80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc0cb0c0 0x7f25bc14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 --2- 172.21.15.181:0/2659340879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25bc15e6d0 0x7f25bc0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 >> 172.21.15.181:0/2659340879 conn(0x7f25bc0d04a0 msgr2=0x7f25bc0ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 shutdown_connections 2023-10-16T09:56:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:47.892+0000 7f25c22c2700 1 -- 172.21.15.181:0/2659340879 wait complete. 2023-10-16T09:56:47.905 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T09:56:47.906 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:56:48.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.029+0000 7f45ba505700 1 Processor -- start 2023-10-16T09:56:48.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.029+0000 7f45ba505700 1 -- start start 2023-10-16T09:56:48.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.029+0000 7f45ba505700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b415e6d0 0x7f45b40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45ba505700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45b415eab0 0x7f45b415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45ba505700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b406ed00 con 0x7f45b40cb0c0 2023-10-16T09:56:48.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45ba505700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b40d02d0 con 0x7f45b415eab0 2023-10-16T09:56:48.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45ba505700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b415ad40 con 0x7f45b415e6d0 2023-10-16T09:56:48.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45b415eab0 0x7f45b415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b8aa2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f45b40cb0c0 0x7f45b415a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50752/0 (socket says 172.21.15.181:50752) 2023-10-16T09:56:48.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b8aa2700 1 -- 172.21.15.181:0/4046594698 learned_addr learned my addr 172.21.15.181:0/4046594698 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:48.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b3fff700 1 --2- 172.21.15.181:0/4046594698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b415e6d0 0x7f45b40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b37fe700 1 -- 172.21.15.181:0/4046594698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b415e6d0 msgr2=0x7f45b40d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:48.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b37fe700 1 --2- 172.21.15.181:0/4046594698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b415e6d0 0x7f45b40d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b37fe700 1 -- 172.21.15.181:0/4046594698 >> v1:172.21.15.181:6789/0 conn(0x7f45b40cb0c0 legacy=0x7f45b415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b2ffd700 1 -- 172.21.15.181:0/4046594698 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613979004 0 0) 0x7f45b406ed00 con 0x7f45b40cb0c0 2023-10-16T09:56:48.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b37fe700 1 -- 172.21.15.181:0/4046594698 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b406e0f0 con 0x7f45b415eab0 2023-10-16T09:56:48.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b37fe700 1 --2- 172.21.15.181:0/4046594698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415eab0 0x7f45b415a0f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009840 tx=0x7f45a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740c265d4326aecf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:48.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b2ffd700 1 -- 172.21.15.181:0/4046594698 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011830 con 0x7f45b415eab0 2023-10-16T09:56:48.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b2ffd700 1 -- 172.21.15.181:0/4046594698 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a8011990 con 0x7f45b415eab0 2023-10-16T09:56:48.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.030+0000 7f45b2ffd700 1 -- 172.21.15.181:0/4046594698 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011d20 con 0x7f45b415eab0 2023-10-16T09:56:48.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 -- 172.21.15.181:0/4046594698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415eab0 msgr2=0x7f45b415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 --2- 172.21.15.181:0/4046594698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415eab0 0x7f45b415a0f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009840 tx=0x7f45a8009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:48.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 -- 172.21.15.181:0/4046594698 shutdown_connections 2023-10-16T09:56:48.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 --2- 172.21.15.181:0/4046594698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b415e6d0 0x7f45b40d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 --2- 172.21.15.181:0/4046594698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415eab0 0x7f45b415a0f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 -- 172.21.15.181:0/4046594698 >> 172.21.15.181:0/4046594698 conn(0x7f45b40d04a0 msgr2=0x7f45b40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:48.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 -- 172.21.15.181:0/4046594698 shutdown_connections 2023-10-16T09:56:48.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.031+0000 7f45ba505700 1 -- 172.21.15.181:0/4046594698 wait complete. 2023-10-16T09:56:48.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.032+0000 7f45ba505700 1 Processor -- start 2023-10-16T09:56:48.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.032+0000 7f45ba505700 1 -- start start 2023-10-16T09:56:48.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45ba505700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b40cb0c0 0x7f45b40c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45ba505700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415e6d0 0x7f45b40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45ba505700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b415eab0 0x7f45b40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45ba505700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c5a10 con 0x7f45b415e6d0 2023-10-16T09:56:48.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45ba505700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c5cf0 con 0x7f45b415eab0 2023-10-16T09:56:48.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45ba505700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c5fd0 con 0x7f45b40cb0c0 2023-10-16T09:56:48.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415e6d0 0x7f45b40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45b3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b40cb0c0 0x7f45b40c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415e6d0 0x7f45b40c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39678/0 (socket says 172.21.15.181:39678) 2023-10-16T09:56:48.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45b37fe700 1 -- 172.21.15.181:0/3237845525 learned_addr learned my addr 172.21.15.181:0/3237845525 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:48.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.033+0000 7f45b8aa2700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b415eab0 0x7f45b40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b37fe700 1 -- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b40cb0c0 msgr2=0x7f45b40c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b37fe700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b40cb0c0 0x7f45b40c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b37fe700 1 -- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b415eab0 msgr2=0x7f45b40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b37fe700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b415eab0 0x7f45b40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b37fe700 1 -- 172.21.15.181:0/3237845525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a80094a0 con 0x7f45b415e6d0 2023-10-16T09:56:48.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b3fff700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b40cb0c0 0x7f45b40c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:48.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b37fe700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415e6d0 0x7f45b40c9670 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009fd0 tx=0x7f45a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:48.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b0ff9700 1 -- 172.21.15.181:0/3237845525 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a80036a0 con 0x7f45b415e6d0 2023-10-16T09:56:48.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b0ff9700 1 -- 172.21.15.181:0/3237845525 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a801f070 con 0x7f45b415e6d0 2023-10-16T09:56:48.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b40c62b0 con 0x7f45b415e6d0 2023-10-16T09:56:48.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45b0ff9700 1 -- 172.21.15.181:0/3237845525 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8003a60 con 0x7f45b415e6d0 2023-10-16T09:56:48.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.034+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b40c6ab0 con 0x7f45b415e6d0 2023-10-16T09:56:48.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.035+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b40c67d0 con 0x7f45b415e6d0 2023-10-16T09:56:48.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.035+0000 7f45b0ff9700 1 -- 172.21.15.181:0/3237845525 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f45a8003bc0 con 0x7f45b415e6d0 2023-10-16T09:56:48.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.036+0000 7f45b0ff9700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 0x7f45a006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.036+0000 7f45b3fff700 1 -- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 msgr2=0x7f45a006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:48.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.036+0000 7f45b0ff9700 1 -- 172.21.15.181:0/3237845525 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6407+0+0 (secure 0 0 0) 0x7f45a8021070 con 0x7f45b415e6d0 2023-10-16T09:56:48.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.036+0000 7f45b3fff700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 0x7f45a006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:56:48.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.040+0000 7f45b0ff9700 1 -- 172.21.15.181:0/3237845525 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f45b40c67d0 con 0x7f45b415e6d0 2023-10-16T09:56:48.084 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:56:48.083+0000 7fd4d85a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:56:48.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.237+0000 7f45b3fff700 1 -- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 msgr2=0x7f45a006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3948589065 2023-10-16T09:56:48.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.237+0000 7f45b3fff700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 0x7f45a006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:56:48.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.286+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45b40c67d0 con 0x7f45b415e6d0 2023-10-16T09:56:48.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.287+0000 7f45b0ff9700 1 -- 172.21.15.181:0/3237845525 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f45b40c67d0 con 0x7f45b415e6d0 2023-10-16T09:56:48.287 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:48.288 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3974533571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3974533571},{"type":"v1","addr":"172.21.15.190:6834","nonce":3974533571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3064364132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3064364132},{"type":"v1","addr":"172.21.15.181:6837","nonce":3064364132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4021280674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4021280674},{"type":"v1","addr":"172.21.15.190:6835","nonce":4021280674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:56.185727+0000","modified":"2023-10-16T09:44:59.222716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6836/760205137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":760205137},{"type":"v1","addr":"172.21.15.181:6836","nonce":760205137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:56:48.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 msgr2=0x7f45a006d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:56:48.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 0x7f45a006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415e6d0 msgr2=0x7f45b40c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415e6d0 0x7f45b40c9670 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009fd0 tx=0x7f45a8004750 comp rx=0 tx=0).stop 2023-10-16T09:56:48.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 shutdown_connections 2023-10-16T09:56:48.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45b40cb0c0 0x7f45b40c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:6832/3948589065,v1:172.21.15.181:6833/3948589065] conn(0x7f45a006b0e0 0x7f45a006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45b415eab0 0x7f45b40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 --2- 172.21.15.181:0/3237845525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b415e6d0 0x7f45b40c9670 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 >> 172.21.15.181:0/3237845525 conn(0x7f45b40d04a0 msgr2=0x7f45b4159cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:48.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 shutdown_connections 2023-10-16T09:56:48.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.289+0000 7f45ba505700 1 -- 172.21.15.181:0/3237845525 wait complete. 2023-10-16T09:56:48.292 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2023-10-16T09:56:48.302 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:56:48.387 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.386+0000 7f643894d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:56:48.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.431+0000 7feb38753700 1 Processor -- start 2023-10-16T09:56:48.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.431+0000 7feb38753700 1 -- start start 2023-10-16T09:56:48.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb38753700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb300d1e60 0x7feb300d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb38753700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb300b3fa0 0x7feb300b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb38753700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb300b2f00 con 0x7feb3015dc20 2023-10-16T09:56:48.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb38753700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300b3b80 con 0x7feb300d1e60 2023-10-16T09:56:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb38753700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb300d2820 con 0x7feb300b3fa0 2023-10-16T09:56:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb300b3fa0 0x7feb300b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb300b3fa0 0x7feb300b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:36736/0 (socket says 172.21.15.181:36736) 2023-10-16T09:56:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 -- 172.21.15.181:0/720420477 learned_addr learned my addr 172.21.15.181:0/720420477 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb35cee700 1 --2- 172.21.15.181:0/720420477 >> v2:172.21.15.190:3300/0 conn(0x7feb300d1e60 0x7feb300d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 -- 172.21.15.181:0/720420477 >> v2:172.21.15.190:3300/0 conn(0x7feb300d1e60 msgr2=0x7feb300d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 --2- 172.21.15.181:0/720420477 >> v2:172.21.15.190:3300/0 conn(0x7feb300d1e60 0x7feb300d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 -- 172.21.15.181:0/720420477 >> v1:172.21.15.190:6789/0 conn(0x7feb3015dc20 legacy=0x7feb300d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 -- 172.21.15.181:0/720420477 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300b31e0 con 0x7feb300b3fa0 2023-10-16T09:56:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.432+0000 7feb36cf0700 1 --2- 172.21.15.181:0/720420477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb300b3fa0 0x7feb300b4410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb28009e50 tx=0x7feb2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92258a2b74909487 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:48.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb354ed700 1 -- 172.21.15.181:0/720420477 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800e070 con 0x7feb300b3fa0 2023-10-16T09:56:48.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb354ed700 1 -- 172.21.15.181:0/720420477 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2800b910 con 0x7feb300b3fa0 2023-10-16T09:56:48.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb354ed700 1 -- 172.21.15.181:0/720420477 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800bcc0 con 0x7feb300b3fa0 2023-10-16T09:56:48.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 -- 172.21.15.181:0/720420477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb300b3fa0 msgr2=0x7feb300b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 --2- 172.21.15.181:0/720420477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb300b3fa0 0x7feb300b4410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb28009e50 tx=0x7feb2800b330 comp rx=0 tx=0).stop 2023-10-16T09:56:48.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 -- 172.21.15.181:0/720420477 shutdown_connections 2023-10-16T09:56:48.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 --2- 172.21.15.181:0/720420477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb300b3fa0 0x7feb300b4410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 --2- 172.21.15.181:0/720420477 >> v2:172.21.15.190:3300/0 conn(0x7feb300d1e60 0x7feb300d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 -- 172.21.15.181:0/720420477 >> 172.21.15.181:0/720420477 conn(0x7feb3008af90 msgr2=0x7feb300d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:48.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 -- 172.21.15.181:0/720420477 shutdown_connections 2023-10-16T09:56:48.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.433+0000 7feb38753700 1 -- 172.21.15.181:0/720420477 wait complete. 2023-10-16T09:56:48.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.434+0000 7feb38753700 1 Processor -- start 2023-10-16T09:56:48.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.434+0000 7feb38753700 1 -- start start 2023-10-16T09:56:48.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.435+0000 7feb38753700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb300b3fa0 0x7feb3009f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.435+0000 7feb38753700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb300d1e60 0x7feb3015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.435+0000 7feb38753700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb3015dc20 0x7feb300ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.435+0000 7feb38753700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3009fbe0 con 0x7feb300b3fa0 2023-10-16T09:56:48.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.435+0000 7feb38753700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3009e640 con 0x7feb300d1e60 2023-10-16T09:56:48.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.435+0000 7feb38753700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3009e8e0 con 0x7feb3015dc20 2023-10-16T09:56:48.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.435+0000 7feb364ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb300b3fa0 0x7feb3009f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb300b3fa0 0x7feb3009f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39712/0 (socket says 172.21.15.181:39712) 2023-10-16T09:56:48.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 -- 172.21.15.181:0/2947783297 learned_addr learned my addr 172.21.15.181:0/2947783297 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:48.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb36cf0700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb3015dc20 0x7feb300ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb35cee700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb300d1e60 0x7feb3015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 -- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb3015dc20 msgr2=0x7feb300ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb3015dc20 0x7feb300ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 -- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb300d1e60 msgr2=0x7feb3015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb300d1e60 0x7feb3015b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 -- 172.21.15.181:0/2947783297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28009b30 con 0x7feb300b3fa0 2023-10-16T09:56:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb36cf0700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb3015dc20 0x7feb300ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb35cee700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb300d1e60 0x7feb3015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb364ef700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb300b3fa0 0x7feb3009f6a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00c4d0 tx=0x7feb2c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb26ffd700 1 -- 172.21.15.181:0/2947783297 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0052d0 con 0x7feb300b3fa0 2023-10-16T09:56:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb26ffd700 1 -- 172.21.15.181:0/2947783297 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c005430 con 0x7feb300b3fa0 2023-10-16T09:56:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3009ebc0 con 0x7feb300b3fa0 2023-10-16T09:56:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb26ffd700 1 -- 172.21.15.181:0/2947783297 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0038f0 con 0x7feb300b3fa0 2023-10-16T09:56:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.436+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3009f180 con 0x7feb300b3fa0 2023-10-16T09:56:48.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.437+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3009f370 con 0x7feb300b3fa0 2023-10-16T09:56:48.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.438+0000 7feb26ffd700 1 -- 172.21.15.181:0/2947783297 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 88940+0+0 (secure 0 0 0) 0x7feb2c003a50 con 0x7feb300b3fa0 2023-10-16T09:56:48.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.438+0000 7feb26ffd700 1 -- 172.21.15.181:0/2947783297 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6750+0+0 (secure 0 0 0) 0x7feb2c088890 con 0x7feb300b3fa0 2023-10-16T09:56:48.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.443+0000 7feb26ffd700 1 -- 172.21.15.181:0/2947783297 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feb2c053980 con 0x7feb300b3fa0 2023-10-16T09:56:48.473 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b4179700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.473 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b4179700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.473 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b4179700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.473 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b4179700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.473 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b4179700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.473 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b1133700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.474 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b1133700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.474 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b1133700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.474 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b1133700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.474 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:56:48.472+0000 7f63b1133700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:56:48.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.689+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb3015cef0 con 0x7feb300b3fa0 2023-10-16T09:56:48.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.690+0000 7feb26ffd700 1 -- 172.21.15.181:0/2947783297 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4922 (secure 0 0 0) 0x7feb2c056fa0 con 0x7feb300b3fa0 2023-10-16T09:56:48.691 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:48.691 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3974533571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3974533571},{"type":"v1","addr":"172.21.15.190:6834","nonce":3974533571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3064364132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3064364132},{"type":"v1","addr":"172.21.15.181:6837","nonce":3064364132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4021280674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4021280674},{"type":"v1","addr":"172.21.15.190:6835","nonce":4021280674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:44:56.185727+0000","modified":"2023-10-16T09:44:59.222716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6836/760205137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":760205137},{"type":"v1","addr":"172.21.15.181:6836","nonce":760205137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-16T09:56:48.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb300b3fa0 msgr2=0x7feb3009f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb300b3fa0 0x7feb3009f6a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00c4d0 tx=0x7feb2c00c7e0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 shutdown_connections 2023-10-16T09:56:48.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb3015dc20 0x7feb300ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb300b3fa0 0x7feb3009f6a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 --2- 172.21.15.181:0/2947783297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb300d1e60 0x7feb3015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 >> 172.21.15.181:0/2947783297 conn(0x7feb3008af90 msgr2=0x7feb300b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:48.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 shutdown_connections 2023-10-16T09:56:48.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.692+0000 7feb38753700 1 -- 172.21.15.181:0/2947783297 wait complete. 2023-10-16T09:56:48.695 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2023-10-16T09:56:48.705 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:56:48.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.825+0000 7f067a4fa700 1 Processor -- start 2023-10-16T09:56:48.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.825+0000 7f067a4fa700 1 -- start start 2023-10-16T09:56:48.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f067a4fa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f067415b990 0x7f06740ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f067a4fa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06740cd2d0 0x7f06740d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f067a4fa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f06740cfc60 0x7f06740d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f067a4fa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06740d0630 con 0x7f06740cd2d0 2023-10-16T09:56:48.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f067a4fa700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06740d24f0 con 0x7f06740cfc60 2023-10-16T09:56:48.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f067a4fa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740cc510 con 0x7f067415b990 2023-10-16T09:56:48.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f06737fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06740cd2d0 0x7f06740d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f06737fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06740cd2d0 0x7f06740d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39772/0 (socket says 172.21.15.181:39772) 2023-10-16T09:56:48.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.826+0000 7f06737fe700 1 -- 172.21.15.181:0/3186844922 learned_addr learned my addr 172.21.15.181:0/3186844922 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:48.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f0673fff700 1 --2- 172.21.15.181:0/3186844922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f067415b990 0x7f06740ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f0678a97700 1 --2- 172.21.15.181:0/3186844922 >> v2:172.21.15.190:3300/0 conn(0x7f06740cfc60 0x7f06740d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f06737fe700 1 -- 172.21.15.181:0/3186844922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f067415b990 msgr2=0x7f06740ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:48.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f06737fe700 1 --2- 172.21.15.181:0/3186844922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f067415b990 0x7f06740ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f06737fe700 1 -- 172.21.15.181:0/3186844922 >> v2:172.21.15.190:3300/0 conn(0x7f06740cfc60 msgr2=0x7f06740d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:48.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f06737fe700 1 --2- 172.21.15.181:0/3186844922 >> v2:172.21.15.190:3300/0 conn(0x7f06740cfc60 0x7f06740d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f06737fe700 1 -- 172.21.15.181:0/3186844922 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06740cc7f0 con 0x7f06740cd2d0 2023-10-16T09:56:48.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f06737fe700 1 --2- 172.21.15.181:0/3186844922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06740cd2d0 0x7f06740d1fb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f06740cbc60 tx=0x7f0668009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e08718a20c558a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:48.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f0672ffd700 1 -- 172.21.15.181:0/3186844922 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066800b870 con 0x7f06740cd2d0 2023-10-16T09:56:48.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f0672ffd700 1 -- 172.21.15.181:0/3186844922 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066800b9d0 con 0x7f06740cd2d0 2023-10-16T09:56:48.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.827+0000 7f0672ffd700 1 -- 172.21.15.181:0/3186844922 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066800bd60 con 0x7f06740cd2d0 2023-10-16T09:56:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 -- 172.21.15.181:0/3186844922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06740cd2d0 msgr2=0x7f06740d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 --2- 172.21.15.181:0/3186844922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06740cd2d0 0x7f06740d1fb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f06740cbc60 tx=0x7f0668009bd0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 -- 172.21.15.181:0/3186844922 shutdown_connections 2023-10-16T09:56:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 --2- 172.21.15.181:0/3186844922 >> v2:172.21.15.190:3300/0 conn(0x7f06740cfc60 0x7f06740d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 --2- 172.21.15.181:0/3186844922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f067415b990 0x7f06740ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 --2- 172.21.15.181:0/3186844922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06740cd2d0 0x7f06740d1fb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 -- 172.21.15.181:0/3186844922 >> 172.21.15.181:0/3186844922 conn(0x7f067409a9a0 msgr2=0x7f06740d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:48.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 -- 172.21.15.181:0/3186844922 shutdown_connections 2023-10-16T09:56:48.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.828+0000 7f067a4fa700 1 -- 172.21.15.181:0/3186844922 wait complete. 2023-10-16T09:56:48.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.829+0000 7f067a4fa700 1 Processor -- start 2023-10-16T09:56:48.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.829+0000 7f067a4fa700 1 -- start start 2023-10-16T09:56:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f067a4fa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740cd2d0 0x7f0674150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f067a4fa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cfc60 0x7f067415e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f067a4fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f067415b990 0x7f06740c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f067a4fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067415e090 con 0x7f067415b990 2023-10-16T09:56:48.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f067a4fa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067415d480 con 0x7f06740cd2d0 2023-10-16T09:56:48.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f067a4fa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067415d760 con 0x7f06740cfc60 2023-10-16T09:56:48.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f067415b990 0x7f06740c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f067415b990 0x7f06740c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39782/0 (socket says 172.21.15.181:39782) 2023-10-16T09:56:48.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 -- 172.21.15.181:0/1732159394 learned_addr learned my addr 172.21.15.181:0/1732159394 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:48.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0673fff700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740cd2d0 0x7f0674150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f06737fe700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cfc60 0x7f067415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:48.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 -- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cfc60 msgr2=0x7f067415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cfc60 0x7f067415e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 -- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740cd2d0 msgr2=0x7f0674150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:48.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740cd2d0 0x7f0674150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:48.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 -- 172.21.15.181:0/1732159394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0668009550 con 0x7f067415b990 2023-10-16T09:56:48.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0678a97700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f067415b990 0x7f06740c9270 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f066c007e60 tx=0x7f066c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:48.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0670ff9700 1 -- 172.21.15.181:0/1732159394 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00ede0 con 0x7f067415b990 2023-10-16T09:56:48.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0670ff9700 1 -- 172.21.15.181:0/1732159394 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c014420 con 0x7f067415b990 2023-10-16T09:56:48.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067415da40 con 0x7f067415b990 2023-10-16T09:56:48.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.830+0000 7f0670ff9700 1 -- 172.21.15.181:0/1732159394 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0147d0 con 0x7f067415b990 2023-10-16T09:56:48.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.831+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067415d170 con 0x7f067415b990 2023-10-16T09:56:48.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.831+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0674067d20 con 0x7f067415b990 2023-10-16T09:56:48.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.832+0000 7f0670ff9700 1 -- 172.21.15.181:0/1732159394 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f066c014930 con 0x7f067415b990 2023-10-16T09:56:48.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.832+0000 7f0670ff9700 1 -- 172.21.15.181:0/1732159394 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6750+0+0 (secure 0 0 0) 0x7f066c089c90 con 0x7f067415b990 2023-10-16T09:56:48.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:48.837+0000 7f0670ff9700 1 -- 172.21.15.181:0/1732159394 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f066c054c90 con 0x7f067415b990 2023-10-16T09:56:49.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.032+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f067415e1d0 con 0x7f067415b990 2023-10-16T09:56:49.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.033+0000 7f0670ff9700 1 -- 172.21.15.181:0/1732159394 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f066c0582b0 con 0x7f067415b990 2023-10-16T09:56:49.034 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:49.034 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":48,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:56:48.399518+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:44:54.175032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-16T09:44:55.152824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/360434361":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/673841945":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/2553110707":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/3542266377":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/2885824130":"2023-10-17T09:56:48.399477+0000","172.21.15.181:6833/3948589065":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/2080914758":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2848692303":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/4251066643":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/3468053237":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/4053208405":"2023-10-17T09:56:48.399477+0000","172.21.15.181:6832/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6832/3948589065":"2023-10-17T09:56:48.399477+0000","172.21.15.181:6833/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/3764257785":"2023-10-17T09:45:15.015302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:56:49.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.035+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f067415b990 msgr2=0x7f06740c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.035+0000 7f067a4fa700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f067415b990 0x7f06740c9270 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f066c007e60 tx=0x7f066c00e3c0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.035+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 shutdown_connections 2023-10-16T09:56:49.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.035+0000 7f067a4fa700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f067415b990 0x7f06740c9270 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.035+0000 7f067a4fa700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740cd2d0 0x7f0674150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.035+0000 7f067a4fa700 1 --2- 172.21.15.181:0/1732159394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cfc60 0x7f067415e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.035+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 >> 172.21.15.181:0/1732159394 conn(0x7f067409a9a0 msgr2=0x7f067406fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:49.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.036+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 shutdown_connections 2023-10-16T09:56:49.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.036+0000 7f067a4fa700 1 -- 172.21.15.181:0/1732159394 wait complete. 2023-10-16T09:56:49.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T09:56:49.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.169+0000 7f4064edd700 1 Processor -- start 2023-10-16T09:56:49.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.169+0000 7f4064edd700 1 -- start start 2023-10-16T09:56:49.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.170+0000 7f4064edd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 0x7f40600cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.170+0000 7f4064edd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f406006a8a0 0x7f406006ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.170+0000 7f4064edd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40600cddf0 con 0x7f40600d0fc0 2023-10-16T09:56:49.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.170+0000 7f4064edd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40600af9c0 con 0x7f406006a8a0 2023-10-16T09:56:49.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.170+0000 7f4064edd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406006b270 con 0x7f406015df60 2023-10-16T09:56:49.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f40600d0fc0 0x7f40600cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50862/0 (socket says 172.21.15.181:50862) 2023-10-16T09:56:49.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405dd9b700 1 -- 172.21.15.181:0/3032964684 learned_addr learned my addr 172.21.15.181:0/3032964684 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:49.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405e59c700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 0x7f40600cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405ed9d700 1 --2- 172.21.15.181:0/3032964684 >> v2:172.21.15.190:3300/0 conn(0x7f406006a8a0 0x7f406006ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568821280 0 0) 0x7f40600cddf0 con 0x7f40600d0fc0 2023-10-16T09:56:49.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40480035b0 con 0x7f40600d0fc0 2023-10-16T09:56:49.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1344250472 0 0) 0x7f40480035b0 con 0x7f40600d0fc0 2023-10-16T09:56:49.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 msgr2=0x7f40600cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:49.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 0x7f40600cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 >> v2:172.21.15.190:3300/0 conn(0x7f406006a8a0 msgr2=0x7f406006ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:49.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 --2- 172.21.15.181:0/3032964684 >> v2:172.21.15.190:3300/0 conn(0x7f406006a8a0 0x7f406006ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40600ccf50 con 0x7f40600d0fc0 2023-10-16T09:56:49.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4050001670 con 0x7f40600d0fc0 2023-10-16T09:56:49.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 >> v1:172.21.15.181:6789/0 conn(0x7f40600d0fc0 legacy=0x7f40600cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 0x7f4048003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4048003f80 0x7f4048004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4048004d90 0x7f4048005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406006b270 con 0x7f4048003f80 2023-10-16T09:56:49.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404800ba10 con 0x7f4048004d90 2023-10-16T09:56:49.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404800bb50 con 0x7f406015df60 2023-10-16T09:56:49.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405dd9b700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4048003f80 0x7f4048004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405e59c700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 0x7f4048003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.171+0000 7f405ed9d700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4048004d90 0x7f4048005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405dd9b700 1 -- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 msgr2=0x7f4048003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:49.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405dd9b700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f406015df60 0x7f4048003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405dd9b700 1 -- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4048004d90 msgr2=0x7f4048005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:49.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405dd9b700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4048004d90 0x7f4048005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405dd9b700 1 -- 172.21.15.181:0/3032964684 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480035b0 con 0x7f4048003f80 2023-10-16T09:56:49.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405dd9b700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4048003f80 0x7f4048004850 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f405000c2f0 tx=0x7f4048005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d057960a0b1be316 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:49.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405000cdd0 con 0x7f4048003f80 2023-10-16T09:56:49.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4050005960 con 0x7f4048003f80 2023-10-16T09:56:49.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405d59a700 1 -- 172.21.15.181:0/3032964684 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050005d10 con 0x7f4048003f80 2023-10-16T09:56:49.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f4064edd700 1 -- 172.21.15.181:0/3032964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4048003f80 msgr2=0x7f4048004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f4064edd700 1 --2- 172.21.15.181:0/3032964684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4048003f80 0x7f4048004850 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f405000c2f0 tx=0x7f4048005880 comp rx=0 tx=0).stop 2023-10-16T09:56:49.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f405ed9d700 1 -- 172.21.15.181:0/3032964684 reap_dead start 2023-10-16T09:56:49.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f4064edd700 1 -- 172.21.15.181:0/3032964684 shutdown_connections 2023-10-16T09:56:49.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f4064edd700 1 -- 172.21.15.181:0/3032964684 >> 172.21.15.181:0/3032964684 conn(0x7f40600d04a0 msgr2=0x7f40600d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:49.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f4064edd700 1 -- 172.21.15.181:0/3032964684 shutdown_connections 2023-10-16T09:56:49.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.172+0000 7f4064edd700 1 -- 172.21.15.181:0/3032964684 wait complete. 2023-10-16T09:56:49.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.174+0000 7f4064edd700 1 Processor -- start 2023-10-16T09:56:49.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.174+0000 7f4064edd700 1 -- start start 2023-10-16T09:56:49.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4048003f80 0x7f40600b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40600b0090 0x7f40600b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40600b32f0 0x7f40600c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40600c9360 con 0x7f40600b32f0 2023-10-16T09:56:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40600b0510 con 0x7f40600b0090 2023-10-16T09:56:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40600b5680 con 0x7f4048003f80 2023-10-16T09:56:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40600b32f0 0x7f40600c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40600b32f0 0x7f40600c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39800/0 (socket says 172.21.15.181:39800) 2023-10-16T09:56:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 -- 172.21.15.181:0/2084648039 learned_addr learned my addr 172.21.15.181:0/2084648039 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 -- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4048003f80 msgr2=0x7f40600b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405e59c700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4048003f80 0x7f40600b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4048003f80 0x7f40600b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 -- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40600b0090 msgr2=0x7f40600b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405dd9b700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40600b0090 0x7f40600b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40600b0090 0x7f40600b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 -- 172.21.15.181:0/2084648039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40600b6120 con 0x7f40600b32f0 2023-10-16T09:56:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f405ed9d700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40600b32f0 0x7f40600c5fa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4058011bf0 tx=0x7f405800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405800e040 con 0x7f40600b32f0 2023-10-16T09:56:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405800f040 con 0x7f40600b32f0 2023-10-16T09:56:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40600c8a30 con 0x7f40600b32f0 2023-10-16T09:56:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405800fe50 con 0x7f40600b32f0 2023-10-16T09:56:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.175+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40600c8ff0 con 0x7f40600b32f0 2023-10-16T09:56:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.176+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40600c9180 con 0x7f40600b32f0 2023-10-16T09:56:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.177+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4058022460 con 0x7f40600b32f0 2023-10-16T09:56:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.177+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6750+0+0 (secure 0 0 0) 0x7f4058089a30 con 0x7f40600b32f0 2023-10-16T09:56:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.181+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4058054b20 con 0x7f40600b32f0 2023-10-16T09:56:49.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.413+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 17) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f40580545d0 con 0x7f40600b32f0 2023-10-16T09:56:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.414+0000 7f404effd700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4040070b10 0x7f4040072f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.415+0000 7f405e59c700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4040070b10 0x7f4040072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.415+0000 7f405e59c700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4040070b10 0x7f4040072f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f40540099a0 tx=0x7f4054008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:49.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.423+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4060067d20 con 0x7f40600b32f0 2023-10-16T09:56:49.424 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T09:56:49.424+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T09:56:49.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.425+0000 7f404effd700 1 -- 172.21.15.181:0/2084648039 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4058017250 con 0x7f40600b32f0 2023-10-16T09:56:49.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4040070b10 msgr2=0x7f4040072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4040070b10 0x7f4040072f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f40540099a0 tx=0x7f4054008040 comp rx=0 tx=0).stop 2023-10-16T09:56:49.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40600b32f0 msgr2=0x7f40600c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40600b32f0 0x7f40600c5fa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4058011bf0 tx=0x7f405800f1b0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 shutdown_connections 2023-10-16T09:56:49.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4040070b10 0x7f4040072f00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4048003f80 0x7f40600b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40600b32f0 0x7f40600c5fa0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 --2- 172.21.15.181:0/2084648039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40600b0090 0x7f40600b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.427+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 >> 172.21.15.181:0/2084648039 conn(0x7f40600d04a0 msgr2=0x7f40600cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:49.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.428+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 shutdown_connections 2023-10-16T09:56:49.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.428+0000 7f4064edd700 1 -- 172.21.15.181:0/2084648039 wait complete. 2023-10-16T09:56:49.431 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T09:56:49.443 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T09:56:49.465 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T09:56:49.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.564+0000 7fd5e4b05700 1 Processor -- start 2023-10-16T09:56:49.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.565+0000 7fd5e4b05700 1 -- start start 2023-10-16T09:56:49.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.565+0000 7fd5e4b05700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e015da70 0x7fd5e00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.565+0000 7fd5e4b05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5e00ce6a0 0x7fd5e006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.565+0000 7fd5e4b05700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5e0072760 0x7fd5e006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.565+0000 7fd5e4b05700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00cef00 con 0x7fd5e00ce6a0 2023-10-16T09:56:49.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.565+0000 7fd5e4b05700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00cec20 con 0x7fd5e0072760 2023-10-16T09:56:49.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.565+0000 7fd5e4b05700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0071c80 con 0x7fd5e015da70 2023-10-16T09:56:49.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5e00ce6a0 0x7fd5e006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5e00ce6a0 0x7fd5e006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39842/0 (socket says 172.21.15.181:39842) 2023-10-16T09:56:49.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 -- 172.21.15.181:0/3022800289 learned_addr learned my addr 172.21.15.181:0/3022800289 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:49.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5de59c700 1 --2- 172.21.15.181:0/3022800289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e015da70 0x7fd5e00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ded9d700 1 --2- 172.21.15.181:0/3022800289 >> v2:172.21.15.190:3300/0 conn(0x7fd5e0072760 0x7fd5e006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 -- 172.21.15.181:0/3022800289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e015da70 msgr2=0x7fd5e00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:49.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 --2- 172.21.15.181:0/3022800289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e015da70 0x7fd5e00ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 -- 172.21.15.181:0/3022800289 >> v2:172.21.15.190:3300/0 conn(0x7fd5e0072760 msgr2=0x7fd5e006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:49.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 --2- 172.21.15.181:0/3022800289 >> v2:172.21.15.190:3300/0 conn(0x7fd5e0072760 0x7fd5e006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 -- 172.21.15.181:0/3022800289 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0071f60 con 0x7fd5e00ce6a0 2023-10-16T09:56:49.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ded9d700 1 --2- 172.21.15.181:0/3022800289 >> v2:172.21.15.190:3300/0 conn(0x7fd5e0072760 0x7fd5e006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:49.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5ddd9b700 1 --2- 172.21.15.181:0/3022800289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e00ce6a0 0x7fd5e006a8a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0161530 tx=0x7fd5d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d7a6d80aba1ddd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:49.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5dd59a700 1 -- 172.21.15.181:0/3022800289 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4011830 con 0x7fd5e00ce6a0 2023-10-16T09:56:49.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5dd59a700 1 -- 172.21.15.181:0/3022800289 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4011990 con 0x7fd5e00ce6a0 2023-10-16T09:56:49.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.566+0000 7fd5dd59a700 1 -- 172.21.15.181:0/3022800289 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4011d20 con 0x7fd5e00ce6a0 2023-10-16T09:56:49.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 -- 172.21.15.181:0/3022800289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e00ce6a0 msgr2=0x7fd5e006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/3022800289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e00ce6a0 0x7fd5e006a8a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0161530 tx=0x7fd5d4009bd0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 -- 172.21.15.181:0/3022800289 shutdown_connections 2023-10-16T09:56:49.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/3022800289 >> v2:172.21.15.190:3300/0 conn(0x7fd5e0072760 0x7fd5e006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/3022800289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e015da70 0x7fd5e00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/3022800289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e00ce6a0 0x7fd5e006a8a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 -- 172.21.15.181:0/3022800289 >> 172.21.15.181:0/3022800289 conn(0x7fd5e00cd010 msgr2=0x7fd5e00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:49.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 -- 172.21.15.181:0/3022800289 shutdown_connections 2023-10-16T09:56:49.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.567+0000 7fd5e4b05700 1 -- 172.21.15.181:0/3022800289 wait complete. 2023-10-16T09:56:49.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.568+0000 7fd5e4b05700 1 Processor -- start 2023-10-16T09:56:49.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.568+0000 7fd5e4b05700 1 -- start start 2023-10-16T09:56:49.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5e4b05700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e0072760 0x7fd5e01519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5e4b05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e00ce6a0 0x7fd5e00aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5e4b05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e015da70 0x7fd5e00af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5e4b05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00ad620 con 0x7fd5e015da70 2023-10-16T09:56:49.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5e4b05700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00ad900 con 0x7fd5e00ce6a0 2023-10-16T09:56:49.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5e4b05700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00adbe0 con 0x7fd5e0072760 2023-10-16T09:56:49.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e015da70 0x7fd5e00af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e015da70 0x7fd5e00af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39850/0 (socket says 172.21.15.181:39850) 2023-10-16T09:56:49.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 -- 172.21.15.181:0/2124776186 learned_addr learned my addr 172.21.15.181:0/2124776186 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:49.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5de59c700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e0072760 0x7fd5e01519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ddd9b700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e00ce6a0 0x7fd5e00aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 -- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e0072760 msgr2=0x7fd5e01519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e0072760 0x7fd5e01519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 -- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e00ce6a0 msgr2=0x7fd5e00aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:49.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e00ce6a0 0x7fd5e00aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:49.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 -- 172.21.15.181:0/2124776186 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4009550 con 0x7fd5e015da70 2023-10-16T09:56:49.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.569+0000 7fd5ded9d700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e015da70 0x7fd5e00af260 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007db0 tx=0x7fd5d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:49.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.570+0000 7fd5ceffd700 1 -- 172.21.15.181:0/2124776186 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800ddd0 con 0x7fd5e015da70 2023-10-16T09:56:49.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.570+0000 7fd5ceffd700 1 -- 172.21.15.181:0/2124776186 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800f040 con 0x7fd5e015da70 2023-10-16T09:56:49.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.570+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e00adec0 con 0x7fd5e015da70 2023-10-16T09:56:49.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.570+0000 7fd5ceffd700 1 -- 172.21.15.181:0/2124776186 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8003c60 con 0x7fd5e015da70 2023-10-16T09:56:49.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.570+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e0151ef0 con 0x7fd5e015da70 2023-10-16T09:56:49.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.571+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e006f6c0 con 0x7fd5e015da70 2023-10-16T09:56:49.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.571+0000 7fd5ceffd700 1 -- 172.21.15.181:0/2124776186 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd5d8004030 con 0x7fd5e015da70 2023-10-16T09:56:49.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.571+0000 7fd5ceffd700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd5c006ad50 0x7fd5c006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.572+0000 7fd5de59c700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd5c006ad50 0x7fd5c006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.572+0000 7fd5ceffd700 1 -- 172.21.15.181:0/2124776186 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6836+0+0 (secure 0 0 0) 0x7fd5d8089720 con 0x7fd5e015da70 2023-10-16T09:56:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.572+0000 7fd5de59c700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd5c006ad50 0x7fd5c006d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ab0 tx=0x7fd5d0008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.576+0000 7fd5ceffd700 1 -- 172.21.15.181:0/2124776186 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5d80546a0 con 0x7fd5e015da70 2023-10-16T09:56:49.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:49.816+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5e00b1ae0 con 0x7fd5e015da70 2023-10-16T09:56:50.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.426+0000 7fd5ceffd700 1 -- 172.21.15.181:0/2124776186 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd5d8057cc0 con 0x7fd5e015da70 2023-10-16T09:56:50.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd5c006ad50 msgr2=0x7fd5c006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:50.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd5c006ad50 0x7fd5c006d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ab0 tx=0x7fd5d0008040 comp rx=0 tx=0).stop 2023-10-16T09:56:50.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e015da70 msgr2=0x7fd5e00af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e015da70 0x7fd5e00af260 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007db0 tx=0x7fd5d800d3b0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 shutdown_connections 2023-10-16T09:56:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd5c006ad50 0x7fd5c006d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5e0072760 0x7fd5e01519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e015da70 0x7fd5e00af260 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 --2- 172.21.15.181:0/2124776186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5e00ce6a0 0x7fd5e00aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 >> 172.21.15.181:0/2124776186 conn(0x7fd5e00cd010 msgr2=0x7fd5e006f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:50.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 shutdown_connections 2023-10-16T09:56:50.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.429+0000 7fd5e4b05700 1 -- 172.21.15.181:0/2124776186 wait complete. 2023-10-16T09:56:50.444 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T09:56:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.608+0000 7efe13b94700 1 Processor -- start 2023-10-16T09:56:50.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.609+0000 7efe13b94700 1 -- start start 2023-10-16T09:56:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.609+0000 7efe13b94700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe0c0d0fc0 0x7efe0c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.609+0000 7efe13b94700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0c162650 con 0x7efe0c0ce610 2023-10-16T09:56:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe13b94700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0c0d0230 con 0x7efe0c0ce9f0 2023-10-16T09:56:50.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe13b94700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0c162190 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1112f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe0c0d0fc0 0x7efe0c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe11930700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efe0c0ce610 0x7efe0c0cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50952/0 (socket says 172.21.15.181:50952) 2023-10-16T09:56:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1112f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe0c0d0fc0 0x7efe0c06d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39874/0 (socket says 172.21.15.181:39874) 2023-10-16T09:56:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe11930700 1 -- 172.21.15.181:0/1591363578 learned_addr learned my addr 172.21.15.181:0/1591363578 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:50.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1112f700 1 -- 172.21.15.181:0/1591363578 >> v1:172.21.15.190:6789/0 conn(0x7efe0c0ce9f0 legacy=0x7efe0c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1112f700 1 -- 172.21.15.181:0/1591363578 >> v1:172.21.15.181:6789/0 conn(0x7efe0c0ce610 legacy=0x7efe0c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1112f700 1 -- 172.21.15.181:0/1591363578 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c162370 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1092e700 1 -- 172.21.15.181:0/1591363578 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775753846 0 0) 0x7efe0c162650 con 0x7efe0c0ce610 2023-10-16T09:56:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1112f700 1 --2- 172.21.15.181:0/1591363578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c06d660 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efe08009840 tx=0x7efe08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499de2b992a9c83b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:50.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1092e700 1 -- 172.21.15.181:0/1591363578 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08004230 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1092e700 1 -- 172.21.15.181:0/1591363578 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08004390 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.610+0000 7efe1092e700 1 -- 172.21.15.181:0/1591363578 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080036a0 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.611+0000 7efe13b94700 1 -- 172.21.15.181:0/1591363578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 msgr2=0x7efe0c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.611+0000 7efe13b94700 1 --2- 172.21.15.181:0/1591363578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c06d660 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efe08009840 tx=0x7efe08009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:50.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.611+0000 7efe13b94700 1 -- 172.21.15.181:0/1591363578 shutdown_connections 2023-10-16T09:56:50.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.611+0000 7efe13b94700 1 --2- 172.21.15.181:0/1591363578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c06d660 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.611+0000 7efe13b94700 1 -- 172.21.15.181:0/1591363578 >> 172.21.15.181:0/1591363578 conn(0x7efe0c0d04a0 msgr2=0x7efe0c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:50.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.611+0000 7efe13b94700 1 -- 172.21.15.181:0/1591363578 shutdown_connections 2023-10-16T09:56:50.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.611+0000 7efe13b94700 1 -- 172.21.15.181:0/1591363578 wait complete. 2023-10-16T09:56:50.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.612+0000 7efe13b94700 1 Processor -- start 2023-10-16T09:56:50.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.612+0000 7efe13b94700 1 -- start start 2023-10-16T09:56:50.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe13b94700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe0c0ce610 0x7efe0c14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:50.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe13b94700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe0c0ce9f0 0x7efe0c14e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:50.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe13b94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:50.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe13b94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c09ad00 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe13b94700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c14de30 con 0x7efe0c0ce9f0 2023-10-16T09:56:50.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe13b94700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c09a080 con 0x7efe0c0ce610 2023-10-16T09:56:50.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:50.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39884/0 (socket says 172.21.15.181:39884) 2023-10-16T09:56:50.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 -- 172.21.15.181:0/3233372469 learned_addr learned my addr 172.21.15.181:0/3233372469 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:50.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe1112f700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe0c0ce9f0 0x7efe0c14e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:50.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe11930700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe0c0ce610 0x7efe0c14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:50.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 -- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe0c0ce610 msgr2=0x7efe0c14d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:50.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe0c0ce610 0x7efe0c14d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 -- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe0c0ce9f0 msgr2=0x7efe0c14e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:50.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe0c0ce9f0 0x7efe0c14e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:50.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.613+0000 7efe12131700 1 -- 172.21.15.181:0/3233372469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe080094a0 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.614+0000 7efe12131700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c160850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efe0400cba0 tx=0x7efe0400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:50.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.614+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400ab80 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.614+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0400ace0 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.614+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c09a360 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.614+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04012660 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.614+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c09a920 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.615+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 89067+0+0 (secure 0 0 0) 0x7efe040127c0 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.615+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c09ae40 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.616+0000 7efe027fc700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7efdf006ad50 0x7efdf006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:50.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.616+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6836+0+0 (secure 0 0 0) 0x7efe04089e10 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.616+0000 7efe11930700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7efdf006ad50 0x7efdf006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:50.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.616+0000 7efe11930700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7efdf006ad50 0x7efdf006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efdfc007fd0 tx=0x7efdfc009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:50.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.620+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efe0c09ae40 con 0x7efe0c0d0fc0 2023-10-16T09:56:50.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:50.815+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efe0c09ae40 con 0x7efe0c0d0fc0 2023-10-16T09:56:51.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.422+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 18) v1 ==== 89275+0+0 (secure 0 0 0) 0x7efe04055160 con 0x7efe0c0d0fc0 2023-10-16T09:56:51.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.428+0000 7efe027fc700 1 -- 172.21.15.181:0/3233372469 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7efe040716c0 con 0x7efe0c0d0fc0 2023-10-16T09:56:51.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7efdf006ad50 msgr2=0x7efdf006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:51.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7efdf006ad50 0x7efdf006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efdfc007fd0 tx=0x7efdfc009040 comp rx=0 tx=0).stop 2023-10-16T09:56:51.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 msgr2=0x7efe0c160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:51.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c160850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efe0400cba0 tx=0x7efe0400cf60 comp rx=0 tx=0).stop 2023-10-16T09:56:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 shutdown_connections 2023-10-16T09:56:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7efdf006ad50 0x7efdf006d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe0c0ce610 0x7efe0c14d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe0c0d0fc0 0x7efe0c160850 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 --2- 172.21.15.181:0/3233372469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe0c0ce9f0 0x7efe0c14e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.432+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 >> 172.21.15.181:0/3233372469 conn(0x7efe0c0d04a0 msgr2=0x7efe0c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.433+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 shutdown_connections 2023-10-16T09:56:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.433+0000 7efe13b94700 1 -- 172.21.15.181:0/3233372469 wait complete. 2023-10-16T09:56:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T09:56:51.453 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T09:56:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.586+0000 7f8442cf2700 1 Processor -- start 2023-10-16T09:56:51.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.586+0000 7f8442cf2700 1 -- start start 2023-10-16T09:56:51.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f8442cf2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f843c0ce3a0 0x7f843c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:51.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f8442cf2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0cd3f0 0x7f843c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f8442cf2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0ce7e0 con 0x7f843c0afa10 2023-10-16T09:56:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f8442cf2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0cd880 con 0x7f843c0ce3a0 2023-10-16T09:56:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f8442cf2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c15b540 con 0x7f843c0cd3f0 2023-10-16T09:56:51.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f843bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f843c0ce3a0 0x7f843c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:51.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f843bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f843c0ce3a0 0x7f843c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39888/0 (socket says 172.21.15.181:39888) 2023-10-16T09:56:51.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f843bfff700 1 -- 172.21.15.181:0/2339663434 learned_addr learned my addr 172.21.15.181:0/2339663434 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:51.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.587+0000 7f844128f700 1 --2- 172.21.15.181:0/2339663434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0cd3f0 0x7f843c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:51.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843bfff700 1 -- 172.21.15.181:0/2339663434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0cd3f0 msgr2=0x7f843c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:51.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843bfff700 1 --2- 172.21.15.181:0/2339663434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0cd3f0 0x7f843c0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843bfff700 1 -- 172.21.15.181:0/2339663434 >> v1:172.21.15.190:6789/0 conn(0x7f843c0afa10 legacy=0x7f843c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:51.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843bfff700 1 -- 172.21.15.181:0/2339663434 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c15b820 con 0x7f843c0ce3a0 2023-10-16T09:56:51.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843bfff700 1 --2- 172.21.15.181:0/2339663434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0ce3a0 0x7f843c0cceb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f84280098f0 tx=0x7f8428009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf459112035b8a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:51.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843b7fe700 1 -- 172.21.15.181:0/2339663434 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842800b8a0 con 0x7f843c0ce3a0 2023-10-16T09:56:51.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843b7fe700 1 -- 172.21.15.181:0/2339663434 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842800ba00 con 0x7f843c0ce3a0 2023-10-16T09:56:51.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f843b7fe700 1 -- 172.21.15.181:0/2339663434 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842800bd90 con 0x7f843c0ce3a0 2023-10-16T09:56:51.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 -- 172.21.15.181:0/2339663434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0ce3a0 msgr2=0x7f843c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:51.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 --2- 172.21.15.181:0/2339663434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0ce3a0 0x7f843c0cceb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f84280098f0 tx=0x7f8428009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:51.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 -- 172.21.15.181:0/2339663434 shutdown_connections 2023-10-16T09:56:51.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 --2- 172.21.15.181:0/2339663434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0cd3f0 0x7f843c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 --2- 172.21.15.181:0/2339663434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0ce3a0 0x7f843c0cceb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 -- 172.21.15.181:0/2339663434 >> 172.21.15.181:0/2339663434 conn(0x7f843c0d04a0 msgr2=0x7f843c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:51.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 -- 172.21.15.181:0/2339663434 shutdown_connections 2023-10-16T09:56:51.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.588+0000 7f8442cf2700 1 -- 172.21.15.181:0/2339663434 wait complete. 2023-10-16T09:56:51.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.590+0000 7f8442cf2700 1 Processor -- start 2023-10-16T09:56:51.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.590+0000 7f8442cf2700 1 -- start start 2023-10-16T09:56:51.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.590+0000 7f8442cf2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0afa10 0x7f843c1f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:51.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8442cf2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843c0cd3f0 0x7f843c15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8442cf2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0ce3a0 0x7f843c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8442cf2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0b5a80 con 0x7f843c0afa10 2023-10-16T09:56:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8442cf2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0c48c0 con 0x7f843c0cd3f0 2023-10-16T09:56:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8442cf2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0b51e0 con 0x7f843c0ce3a0 2023-10-16T09:56:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0afa10 0x7f843c1f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0afa10 0x7f843c1f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39894/0 (socket says 172.21.15.181:39894) 2023-10-16T09:56:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 -- 172.21.15.181:0/3406490195 learned_addr learned my addr 172.21.15.181:0/3406490195 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f843bfff700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843c0cd3f0 0x7f843c15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:51.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f844128f700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0ce3a0 0x7f843c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 -- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0ce3a0 msgr2=0x7f843c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0ce3a0 0x7f843c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 -- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843c0cd3f0 msgr2=0x7f843c15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843c0cd3f0 0x7f843c15c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:51.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 -- 172.21.15.181:0/3406490195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428009550 con 0x7f843c0afa10 2023-10-16T09:56:51.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f843bfff700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843c0cd3f0 0x7f843c15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:51.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8440a8e700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0afa10 0x7f843c1f4890 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f843000c4d0 tx=0x7f843000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:51.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f84397fa700 1 -- 172.21.15.181:0/3406490195 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300052d0 con 0x7f843c0afa10 2023-10-16T09:56:51.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f84397fa700 1 -- 172.21.15.181:0/3406490195 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430005430 con 0x7f843c0afa10 2023-10-16T09:56:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c0b54c0 con 0x7f843c0afa10 2023-10-16T09:56:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f84397fa700 1 -- 172.21.15.181:0/3406490195 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300038f0 con 0x7f843c0afa10 2023-10-16T09:56:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.591+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c1f4dd0 con 0x7f843c0afa10 2023-10-16T09:56:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.592+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c0ccd20 con 0x7f843c0afa10 2023-10-16T09:56:51.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.593+0000 7f84397fa700 1 -- 172.21.15.181:0/3406490195 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8430003a50 con 0x7f843c0afa10 2023-10-16T09:56:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.593+0000 7f84397fa700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f842006af90 0x7f842006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.593+0000 7f84397fa700 1 -- 172.21.15.181:0/3406490195 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f8430088d30 con 0x7f843c0afa10 2023-10-16T09:56:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.593+0000 7f843bfff700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f842006af90 0x7f842006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:51.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.594+0000 7f843bfff700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f842006af90 0x7f842006d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8428009840 tx=0x7f8428008030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:51.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.598+0000 7f84397fa700 1 -- 172.21.15.181:0/3406490195 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8430053bd0 con 0x7f843c0afa10 2023-10-16T09:56:51.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:51.791+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f843c0c4720 con 0x7f843c0afa10 2023-10-16T09:56:52.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.429+0000 7f84397fa700 1 -- 172.21.15.181:0/3406490195 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) v1 ==== 169+0+0 (secure 0 0 0) 0x7f843c0c4720 con 0x7f843c0afa10 2023-10-16T09:56:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.433+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f842006af90 msgr2=0x7f842006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.433+0000 7f8442cf2700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f842006af90 0x7f842006d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8428009840 tx=0x7f8428008030 comp rx=0 tx=0).stop 2023-10-16T09:56:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.433+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0afa10 msgr2=0x7f843c1f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.433+0000 7f8442cf2700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0afa10 0x7f843c1f4890 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f843000c4d0 tx=0x7f843000c890 comp rx=0 tx=0).stop 2023-10-16T09:56:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 shutdown_connections 2023-10-16T09:56:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f842006af90 0x7f842006d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f843c0ce3a0 0x7f843c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f843c0afa10 0x7f843c1f4890 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 --2- 172.21.15.181:0/3406490195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843c0cd3f0 0x7f843c15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 >> 172.21.15.181:0/3406490195 conn(0x7f843c0d04a0 msgr2=0x7f843c0b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 shutdown_connections 2023-10-16T09:56:52.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.434+0000 7f8442cf2700 1 -- 172.21.15.181:0/3406490195 wait complete. 2023-10-16T09:56:52.438 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T09:56:52.452 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T09:56:52.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.579+0000 7f9fc2ed9700 1 Processor -- start 2023-10-16T09:56:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.579+0000 7f9fc2ed9700 1 -- start start 2023-10-16T09:56:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fc2ed9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9fbc15da70 0x7f9fbc0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fc2ed9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc0cef00 con 0x7f9fbc0ce6a0 2023-10-16T09:56:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fc2ed9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc0cec20 con 0x7f9fbc06afb0 2023-10-16T09:56:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fc2ed9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc071c80 con 0x7f9fbc15da70 2023-10-16T09:56:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fc0c75700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9fbc15da70 0x7f9fbc0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fbbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9fbc0ce6a0 0x7f9fbc06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50980/0 (socket says 172.21.15.181:50980) 2023-10-16T09:56:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fbbfff700 1 -- 172.21.15.181:0/2055985433 learned_addr learned my addr 172.21.15.181:0/2055985433 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.580+0000 7f9fc0c75700 1 --2- 172.21.15.181:0/2055985433 >> v2:172.21.15.181:3300/0 conn(0x7f9fbc15da70 0x7f9fbc0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39902/0 (socket says 172.21.15.181:39902) 2023-10-16T09:56:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc0c75700 1 -- 172.21.15.181:0/2055985433 >> v1:172.21.15.190:6789/0 conn(0x7f9fbc06afb0 legacy=0x7f9fbc1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc0c75700 1 -- 172.21.15.181:0/2055985433 >> v1:172.21.15.181:6789/0 conn(0x7f9fbc0ce6a0 legacy=0x7f9fbc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc0c75700 1 -- 172.21.15.181:0/2055985433 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc071f60 con 0x7f9fbc15da70 2023-10-16T09:56:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fbb7fe700 1 -- 172.21.15.181:0/2055985433 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859504724 0 0) 0x7f9fbc0cef00 con 0x7f9fbc0ce6a0 2023-10-16T09:56:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc0c75700 1 --2- 172.21.15.181:0/2055985433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc0ce160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009840 tx=0x7f9fb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf913fe953c068e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fbb7fe700 1 -- 172.21.15.181:0/2055985433 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0004230 con 0x7f9fbc15da70 2023-10-16T09:56:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fbb7fe700 1 -- 172.21.15.181:0/2055985433 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb0004390 con 0x7f9fbc15da70 2023-10-16T09:56:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fbb7fe700 1 -- 172.21.15.181:0/2055985433 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00036a0 con 0x7f9fbc15da70 2023-10-16T09:56:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/2055985433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 msgr2=0x7f9fbc0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/2055985433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc0ce160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009840 tx=0x7f9fb0009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/2055985433 shutdown_connections 2023-10-16T09:56:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/2055985433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc0ce160 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.581+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/2055985433 >> 172.21.15.181:0/2055985433 conn(0x7f9fbc0cd010 msgr2=0x7f9fbc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.582+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/2055985433 shutdown_connections 2023-10-16T09:56:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.582+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/2055985433 wait complete. 2023-10-16T09:56:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.583+0000 7f9fc2ed9700 1 Processor -- start 2023-10-16T09:56:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.583+0000 7f9fc2ed9700 1 -- start start 2023-10-16T09:56:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc06afb0 0x7f9fbc15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ce6a0 0x7f9fbc15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc1605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc15a970 con 0x7f9fbc15da70 2023-10-16T09:56:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc160150 con 0x7f9fbc06afb0 2023-10-16T09:56:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc15f540 con 0x7f9fbc0ce6a0 2023-10-16T09:56:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc1605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc1605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39912/0 (socket says 172.21.15.181:39912) 2023-10-16T09:56:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 -- 172.21.15.181:0/4239576565 learned_addr learned my addr 172.21.15.181:0/4239576565 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fbbfff700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ce6a0 0x7f9fbc15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc0c75700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc06afb0 0x7f9fbc15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 -- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ce6a0 msgr2=0x7f9fbc15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ce6a0 0x7f9fbc15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 -- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc06afb0 msgr2=0x7f9fbc15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc06afb0 0x7f9fbc15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 -- 172.21.15.181:0/4239576565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00094a0 con 0x7f9fbc15da70 2023-10-16T09:56:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fbbfff700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ce6a0 0x7f9fbc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc0c75700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc06afb0 0x7f9fbc15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc1476700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc1605e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400b9a0 tx=0x7f9fb400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fb97fa700 1 -- 172.21.15.181:0/4239576565 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400c9b0 con 0x7f9fbc15da70 2023-10-16T09:56:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fb97fa700 1 -- 172.21.15.181:0/4239576565 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb400cb10 con 0x7f9fbc15da70 2023-10-16T09:56:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc15f820 con 0x7f9fbc15da70 2023-10-16T09:56:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fb97fa700 1 -- 172.21.15.181:0/4239576565 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4014440 con 0x7f9fbc15da70 2023-10-16T09:56:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.584+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc15fde0 con 0x7f9fbc15da70 2023-10-16T09:56:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.585+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc06f6c0 con 0x7f9fbc15da70 2023-10-16T09:56:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.586+0000 7f9fb97fa700 1 -- 172.21.15.181:0/4239576565 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9fb401d3b0 con 0x7f9fbc15da70 2023-10-16T09:56:52.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.586+0000 7f9fb97fa700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f9fa006af90 0x7f9fa006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.586+0000 7f9fc0c75700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f9fa006af90 0x7f9fa006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.586+0000 7f9fb97fa700 1 -- 172.21.15.181:0/4239576565 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7f9fb4089360 con 0x7f9fbc15da70 2023-10-16T09:56:52.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.587+0000 7f9fc0c75700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f9fa006af90 0x7f9fa006d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009b90 tx=0x7f9fb0008030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:52.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.591+0000 7f9fb97fa700 1 -- 172.21.15.181:0/4239576565 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9fb4054340 con 0x7f9fbc15da70 2023-10-16T09:56:52.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.783+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f9fbc0d06c0 con 0x7f9fbc15da70 2023-10-16T09:56:52.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.787+0000 7f9fb97fa700 1 -- 172.21.15.181:0/4239576565 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v13) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9fb4057960 con 0x7f9fbc15da70 2023-10-16T09:56:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f9fa006af90 msgr2=0x7f9fa006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f9fa006af90 0x7f9fa006d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009b90 tx=0x7f9fb0008030 comp rx=0 tx=0).stop 2023-10-16T09:56:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 msgr2=0x7f9fbc1605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc1605e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400b9a0 tx=0x7f9fb400bd60 comp rx=0 tx=0).stop 2023-10-16T09:56:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 shutdown_connections 2023-10-16T09:56:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f9fa006af90 0x7f9fa006d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc15da70 0x7f9fbc1605e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc06afb0 0x7f9fbc15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 --2- 172.21.15.181:0/4239576565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ce6a0 0x7f9fbc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 >> 172.21.15.181:0/4239576565 conn(0x7f9fbc0cd010 msgr2=0x7f9fbc0b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 shutdown_connections 2023-10-16T09:56:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.789+0000 7f9fc2ed9700 1 -- 172.21.15.181:0/4239576565 wait complete. 2023-10-16T09:56:52.804 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T09:56:52.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.925+0000 7fc7a3703700 1 Processor -- start 2023-10-16T09:56:52.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.925+0000 7fc7a3703700 1 -- start start 2023-10-16T09:56:52.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a3703700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc79c15da70 0x7fc79c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a3703700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0cef00 con 0x7fc79c06afb0 2023-10-16T09:56:52.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a3703700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0cec20 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a3703700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79c071c80 con 0x7fc79c15da70 2023-10-16T09:56:52.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a149f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc79c15da70 0x7fc79c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a1ca0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc79c06afb0 0x7fc79c1615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:50996/0 (socket says 172.21.15.181:50996) 2023-10-16T09:56:52.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a1ca0700 1 -- 172.21.15.181:0/3820527559 learned_addr learned my addr 172.21.15.181:0/3820527559 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:52.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a149f700 1 -- 172.21.15.181:0/3820527559 >> v1:172.21.15.190:6789/0 conn(0x7fc79c0ce6a0 legacy=0x7fc79c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:52.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a149f700 1 -- 172.21.15.181:0/3820527559 >> v1:172.21.15.181:6789/0 conn(0x7fc79c06afb0 legacy=0x7fc79c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a149f700 1 -- 172.21.15.181:0/3820527559 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c071f60 con 0x7fc79c15da70 2023-10-16T09:56:52.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc793fff700 1 -- 172.21.15.181:0/3820527559 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456927648 0 0) 0x7fc79c0cef00 con 0x7fc79c06afb0 2023-10-16T09:56:52.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc7a149f700 1 --2- 172.21.15.181:0/3820527559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c15da70 0x7fc79c0ce160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc798009840 tx=0x7fc798009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6197ac82d34df70 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc793fff700 1 -- 172.21.15.181:0/3820527559 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798004230 con 0x7fc79c15da70 2023-10-16T09:56:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc793fff700 1 -- 172.21.15.181:0/3820527559 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc798004390 con 0x7fc79c15da70 2023-10-16T09:56:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.926+0000 7fc793fff700 1 -- 172.21.15.181:0/3820527559 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7980036a0 con 0x7fc79c15da70 2023-10-16T09:56:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.927+0000 7fc7a3703700 1 -- 172.21.15.181:0/3820527559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c15da70 msgr2=0x7fc79c0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.927+0000 7fc7a3703700 1 --2- 172.21.15.181:0/3820527559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c15da70 0x7fc79c0ce160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc798009840 tx=0x7fc798009b50 comp rx=0 tx=0).stop 2023-10-16T09:56:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.927+0000 7fc7a3703700 1 -- 172.21.15.181:0/3820527559 shutdown_connections 2023-10-16T09:56:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.927+0000 7fc7a3703700 1 --2- 172.21.15.181:0/3820527559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c15da70 0x7fc79c0ce160 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.927+0000 7fc7a3703700 1 -- 172.21.15.181:0/3820527559 >> 172.21.15.181:0/3820527559 conn(0x7fc79c0cd010 msgr2=0x7fc79c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.927+0000 7fc7a3703700 1 -- 172.21.15.181:0/3820527559 shutdown_connections 2023-10-16T09:56:52.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.927+0000 7fc7a3703700 1 -- 172.21.15.181:0/3820527559 wait complete. 2023-10-16T09:56:52.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.928+0000 7fc7a3703700 1 Processor -- start 2023-10-16T09:56:52.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.928+0000 7fc7a3703700 1 -- start start 2023-10-16T09:56:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a3703700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79c06afb0 0x7fc79c14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a3703700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c0ce6a0 0x7fc79c1f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a3703700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc79c15da70 0x7fc79c1f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a3703700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0b4a70 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a3703700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c1f4380 con 0x7fc79c15da70 2023-10-16T09:56:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a3703700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0b4400 con 0x7fc79c06afb0 2023-10-16T09:56:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c0ce6a0 0x7fc79c1f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c0ce6a0 0x7fc79c1f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39922/0 (socket says 172.21.15.181:39922) 2023-10-16T09:56:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 -- 172.21.15.181:0/1003275666 learned_addr learned my addr 172.21.15.181:0/1003275666 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a1ca0700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc79c15da70 0x7fc79c1f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a149f700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79c06afb0 0x7fc79c14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 -- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79c06afb0 msgr2=0x7fc79c14fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79c06afb0 0x7fc79c14fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 -- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc79c15da70 msgr2=0x7fc79c1f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc79c15da70 0x7fc79c1f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 -- 172.21.15.181:0/1003275666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980094a0 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.929+0000 7fc7a0c9e700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c0ce6a0 0x7fc79c1f3160 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c4d0 tx=0x7fc78c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.930+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0054e0 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.930+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c003680 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.930+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c1f3cf0 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.930+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c003a30 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.930+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c0b4690 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.931+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79c0b4bb0 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.931+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc78c003b90 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.932+0000 7fc791ffb700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc78006af90 0x7fc78006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.932+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7fc78c088710 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.932+0000 7fc7a149f700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc78006af90 0x7fc78006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.932+0000 7fc7a149f700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc78006af90 0x7fc78006d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc79c160780 tx=0x7fc798008030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.936+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc79c0b4bb0 con 0x7fc79c0ce6a0 2023-10-16T09:56:52.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:52.971+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc78c053b40 con 0x7fc79c0ce6a0 2023-10-16T09:56:53.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.141+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fc79c0b4bb0 con 0x7fc79c0ce6a0 2023-10-16T09:56:53.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.144+0000 7fc791ffb700 1 -- 172.21.15.181:0/1003275666 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v14) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc79c0b4bb0 con 0x7fc79c0ce6a0 2023-10-16T09:56:53.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc78006af90 msgr2=0x7fc78006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc78006af90 0x7fc78006d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc79c160780 tx=0x7fc798008030 comp rx=0 tx=0).stop 2023-10-16T09:56:53.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c0ce6a0 msgr2=0x7fc79c1f3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c0ce6a0 0x7fc79c1f3160 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00c4d0 tx=0x7fc78c00c890 comp rx=0 tx=0).stop 2023-10-16T09:56:53.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 shutdown_connections 2023-10-16T09:56:53.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc78006af90 0x7fc78006d450 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79c06afb0 0x7fc79c14fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc79c15da70 0x7fc79c1f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 --2- 172.21.15.181:0/1003275666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc79c0ce6a0 0x7fc79c1f3160 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 >> 172.21.15.181:0/1003275666 conn(0x7fc79c0cd010 msgr2=0x7fc79c06fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:53.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 shutdown_connections 2023-10-16T09:56:53.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.147+0000 7fc7a3703700 1 -- 172.21.15.181:0/1003275666 wait complete. 2023-10-16T09:56:53.161 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T09:56:53.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.281+0000 7f830cdd5700 1 Processor -- start 2023-10-16T09:56:53.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.281+0000 7f830cdd5700 1 -- start start 2023-10-16T09:56:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.282+0000 7f830cdd5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f830815dcd0 0x7f83080ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.282+0000 7f830cdd5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f830806a8a0 0x7f830806acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.282+0000 7f830cdd5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830806b210 con 0x7f8308162450 2023-10-16T09:56:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.282+0000 7f830cdd5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83080cec70 con 0x7f830806a8a0 2023-10-16T09:56:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.282+0000 7f830cdd5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83080cef50 con 0x7f830815dcd0 2023-10-16T09:56:53.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f830806a8a0 0x7f830806acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f830806a8a0 0x7f830806acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39932/0 (socket says 172.21.15.181:39932) 2023-10-16T09:56:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 -- 172.21.15.181:0/2673690146 learned_addr learned my addr 172.21.15.181:0/2673690146 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f830659c700 1 --2- 172.21.15.181:0/2673690146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f830815dcd0 0x7f83080ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 -- 172.21.15.181:0/2673690146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f830815dcd0 msgr2=0x7f83080ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 --2- 172.21.15.181:0/2673690146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f830815dcd0 0x7f83080ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 -- 172.21.15.181:0/2673690146 >> v1:172.21.15.190:6789/0 conn(0x7f8308162450 legacy=0x7f83080caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 -- 172.21.15.181:0/2673690146 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83080cf230 con 0x7f830806a8a0 2023-10-16T09:56:53.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f830659c700 1 --2- 172.21.15.181:0/2673690146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f830815dcd0 0x7f83080ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f8306d9d700 1 --2- 172.21.15.181:0/2673690146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830806a8a0 0x7f830806acd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8300009e50 tx=0x7f830000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12a9881e25d0c70 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f830559a700 1 -- 172.21.15.181:0/2673690146 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000e070 con 0x7f830806a8a0 2023-10-16T09:56:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f830559a700 1 -- 172.21.15.181:0/2673690146 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830000b9c0 con 0x7f830806a8a0 2023-10-16T09:56:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.283+0000 7f830559a700 1 -- 172.21.15.181:0/2673690146 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000bd70 con 0x7f830806a8a0 2023-10-16T09:56:53.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 -- 172.21.15.181:0/2673690146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830806a8a0 msgr2=0x7f830806acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 --2- 172.21.15.181:0/2673690146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830806a8a0 0x7f830806acd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8300009e50 tx=0x7f830000b330 comp rx=0 tx=0).stop 2023-10-16T09:56:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 -- 172.21.15.181:0/2673690146 shutdown_connections 2023-10-16T09:56:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 --2- 172.21.15.181:0/2673690146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830806a8a0 0x7f830806acd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 --2- 172.21.15.181:0/2673690146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f830815dcd0 0x7f83080ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 -- 172.21.15.181:0/2673690146 >> 172.21.15.181:0/2673690146 conn(0x7f83080d05d0 msgr2=0x7f83080d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 -- 172.21.15.181:0/2673690146 shutdown_connections 2023-10-16T09:56:53.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.284+0000 7f830cdd5700 1 -- 172.21.15.181:0/2673690146 wait complete. 2023-10-16T09:56:53.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.285+0000 7f830cdd5700 1 Processor -- start 2023-10-16T09:56:53.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.285+0000 7f830cdd5700 1 -- start start 2023-10-16T09:56:53.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f830cdd5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830806a8a0 0x7f8308072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f830cdd5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830815dcd0 0x7f83080b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f830cdd5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8308162450 0x7f83080b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f830cdd5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83080b6120 con 0x7f830815dcd0 2023-10-16T09:56:53.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f830cdd5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308148600 con 0x7f830806a8a0 2023-10-16T09:56:53.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f830cdd5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83081f6cb0 con 0x7f8308162450 2023-10-16T09:56:53.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830815dcd0 0x7f83080b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830815dcd0 0x7f83080b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39936/0 (socket says 172.21.15.181:39936) 2023-10-16T09:56:53.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 -- 172.21.15.181:0/275587172 learned_addr learned my addr 172.21.15.181:0/275587172 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:53.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8306d9d700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8308162450 0x7f83080b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f830659c700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830806a8a0 0x7f8308072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 -- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8308162450 msgr2=0x7f83080b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8308162450 0x7f83080b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 -- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830806a8a0 msgr2=0x7f8308072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830806a8a0 0x7f8308072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 -- 172.21.15.181:0/275587172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300009b30 con 0x7f830815dcd0 2023-10-16T09:56:53.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.286+0000 7f8305d9b700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830815dcd0 0x7f83080b56a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c450 tx=0x7f82f000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:53.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.287+0000 7f82feffd700 1 -- 172.21.15.181:0/275587172 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00052d0 con 0x7f830815dcd0 2023-10-16T09:56:53.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.287+0000 7f82feffd700 1 -- 172.21.15.181:0/275587172 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0005430 con 0x7f830815dcd0 2023-10-16T09:56:53.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.287+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081f6f90 con 0x7f830815dcd0 2023-10-16T09:56:53.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.287+0000 7f82feffd700 1 -- 172.21.15.181:0/275587172 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00038c0 con 0x7f830815dcd0 2023-10-16T09:56:53.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.287+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081f7550 con 0x7f830815dcd0 2023-10-16T09:56:53.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.287+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83080c5ee0 con 0x7f830815dcd0 2023-10-16T09:56:53.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.288+0000 7f82feffd700 1 -- 172.21.15.181:0/275587172 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f82f0003a20 con 0x7f830815dcd0 2023-10-16T09:56:53.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.288+0000 7f82feffd700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f82e806b020 0x7f82e806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.288+0000 7f82feffd700 1 -- 172.21.15.181:0/275587172 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7f82f0088b70 con 0x7f830815dcd0 2023-10-16T09:56:53.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.288+0000 7f830659c700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f82e806b020 0x7f82e806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.289+0000 7f830659c700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f82e806b020 0x7f82e806d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009930 tx=0x7f82f8008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:53.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.293+0000 7f82feffd700 1 -- 172.21.15.181:0/275587172 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f82f0053a00 con 0x7f830815dcd0 2023-10-16T09:56:53.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.487+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f830804ebb0 con 0x7f830815dcd0 2023-10-16T09:56:53.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.490+0000 7f82feffd700 1 -- 172.21.15.181:0/275587172 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f82f0057020 con 0x7f830815dcd0 2023-10-16T09:56:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f82e806b020 msgr2=0x7f82e806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f82e806b020 0x7f82e806d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009930 tx=0x7f82f8008040 comp rx=0 tx=0).stop 2023-10-16T09:56:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830815dcd0 msgr2=0x7f83080b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830815dcd0 0x7f83080b56a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c450 tx=0x7f82f000c810 comp rx=0 tx=0).stop 2023-10-16T09:56:53.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 shutdown_connections 2023-10-16T09:56:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f82e806b020 0x7f82e806d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8308162450 0x7f83080b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830806a8a0 0x7f8308072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 --2- 172.21.15.181:0/275587172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f830815dcd0 0x7f83080b56a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 >> 172.21.15.181:0/275587172 conn(0x7f83080d05d0 msgr2=0x7f83080cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:53.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 shutdown_connections 2023-10-16T09:56:53.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.492+0000 7f830cdd5700 1 -- 172.21.15.181:0/275587172 wait complete. 2023-10-16T09:56:53.506 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-16T09:56:53.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.624+0000 7f3c151b9700 1 Processor -- start 2023-10-16T09:56:53.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.625+0000 7f3c151b9700 1 -- start start 2023-10-16T09:56:53.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.625+0000 7f3c151b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c1015dc40 0x7f3c1006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.625+0000 7f3c151b9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c100b2710 con 0x7f3c1006ade0 2023-10-16T09:56:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.625+0000 7f3c151b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c100c18f0 con 0x7f3c100c2dc0 2023-10-16T09:56:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.625+0000 7f3c151b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c100c1bd0 con 0x7f3c1015dc40 2023-10-16T09:56:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.625+0000 7f3c0e59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3c1006ade0 0x7f3c100c26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51020/0 (socket says 172.21.15.181:51020) 2023-10-16T09:56:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.625+0000 7f3c0e59c700 1 -- 172.21.15.181:0/2672670537 learned_addr learned my addr 172.21.15.181:0/2672670537 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0ed9d700 1 --2- 172.21.15.181:0/2672670537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c1015dc40 0x7f3c1006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0dd9b700 1 -- 172.21.15.181:0/2672670537 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112888399 0 0) 0x7f3c00001110 con 0x7f3c1006ade0 2023-10-16T09:56:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0dd9b700 1 -- 172.21.15.181:0/2672670537 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf80035b0 con 0x7f3c1006ade0 2023-10-16T09:56:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0dd9b700 1 -- 172.21.15.181:0/2672670537 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850873904 0 0) 0x7f3c100c18f0 con 0x7f3c100c2dc0 2023-10-16T09:56:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0dd9b700 1 -- 172.21.15.181:0/2672670537 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c00001110 con 0x7f3c100c2dc0 2023-10-16T09:56:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0ed9d700 1 -- 172.21.15.181:0/2672670537 >> v1:172.21.15.190:6789/0 conn(0x7f3c100c2dc0 legacy=0x7f3c1015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0ed9d700 1 -- 172.21.15.181:0/2672670537 >> v1:172.21.15.181:6789/0 conn(0x7f3c1006ade0 legacy=0x7f3c100c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0ed9d700 1 -- 172.21.15.181:0/2672670537 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100c1eb0 con 0x7f3c1015dc40 2023-10-16T09:56:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0ed9d700 1 --2- 172.21.15.181:0/2672670537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c1015dc40 0x7f3c1006a8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009870 tx=0x7f3c04009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37bd18dc6062c1b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0dd9b700 1 -- 172.21.15.181:0/2672670537 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04011830 con 0x7f3c1015dc40 2023-10-16T09:56:53.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0dd9b700 1 -- 172.21.15.181:0/2672670537 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c04011990 con 0x7f3c1015dc40 2023-10-16T09:56:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.626+0000 7f3c0dd9b700 1 -- 172.21.15.181:0/2672670537 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04011d00 con 0x7f3c1015dc40 2023-10-16T09:56:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.627+0000 7f3c151b9700 1 -- 172.21.15.181:0/2672670537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c1015dc40 msgr2=0x7f3c1006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.627+0000 7f3c151b9700 1 --2- 172.21.15.181:0/2672670537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c1015dc40 0x7f3c1006a8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009870 tx=0x7f3c04009b80 comp rx=0 tx=0).stop 2023-10-16T09:56:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.627+0000 7f3c151b9700 1 -- 172.21.15.181:0/2672670537 shutdown_connections 2023-10-16T09:56:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.627+0000 7f3c151b9700 1 --2- 172.21.15.181:0/2672670537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c1015dc40 0x7f3c1006a8a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.627+0000 7f3c151b9700 1 -- 172.21.15.181:0/2672670537 >> 172.21.15.181:0/2672670537 conn(0x7f3c100b1780 msgr2=0x7f3c100b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.627+0000 7f3c151b9700 1 -- 172.21.15.181:0/2672670537 shutdown_connections 2023-10-16T09:56:53.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.627+0000 7f3c151b9700 1 -- 172.21.15.181:0/2672670537 wait complete. 2023-10-16T09:56:53.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.628+0000 7f3c151b9700 1 Processor -- start 2023-10-16T09:56:53.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.628+0000 7f3c151b9700 1 -- start start 2023-10-16T09:56:53.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c151b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1006ade0 0x7f3c100ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c151b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c100c2dc0 0x7f3c100c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c151b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c1015dc40 0x7f3c10167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c151b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c100c0e00 con 0x7f3c1015dc40 2023-10-16T09:56:53.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c151b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c100af250 con 0x7f3c1006ade0 2023-10-16T09:56:53.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c151b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1006d630 con 0x7f3c100c2dc0 2023-10-16T09:56:53.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c1015dc40 0x7f3c10167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c1015dc40 0x7f3c10167fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39952/0 (socket says 172.21.15.181:39952) 2023-10-16T09:56:53.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 -- 172.21.15.181:0/1401411244 learned_addr learned my addr 172.21.15.181:0/1401411244 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:53.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0ed9d700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1006ade0 0x7f3c100ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0e59c700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c100c2dc0 0x7f3c100c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 -- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c100c2dc0 msgr2=0x7f3c100c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c100c2dc0 0x7f3c100c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 -- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1006ade0 msgr2=0x7f3c100ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:53.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1006ade0 0x7f3c100ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:53.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.629+0000 7f3c0f59e700 1 -- 172.21.15.181:0/1401411244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c040094a0 con 0x7f3c1015dc40 2023-10-16T09:56:53.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.630+0000 7f3c0f59e700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c1015dc40 0x7f3c10167fa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800cbf0 tx=0x7f3c0800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:53.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.630+0000 7f3bf77fe700 1 -- 172.21.15.181:0/1401411244 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08003c50 con 0x7f3c1015dc40 2023-10-16T09:56:53.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.630+0000 7f3bf77fe700 1 -- 172.21.15.181:0/1401411244 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08003dd0 con 0x7f3c1015dc40 2023-10-16T09:56:53.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.630+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c10168c00 con 0x7f3c1015dc40 2023-10-16T09:56:53.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.630+0000 7f3bf77fe700 1 -- 172.21.15.181:0/1401411244 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800aa60 con 0x7f3c1015dc40 2023-10-16T09:56:53.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.630+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1006d2c0 con 0x7f3c1015dc40 2023-10-16T09:56:53.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.631+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c10067d20 con 0x7f3c1015dc40 2023-10-16T09:56:53.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.631+0000 7f3bf77fe700 1 -- 172.21.15.181:0/1401411244 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3c0800abc0 con 0x7f3c1015dc40 2023-10-16T09:56:53.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.632+0000 7f3bf77fe700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3bfc06b020 0x7f3bfc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:53.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.632+0000 7f3bf77fe700 1 -- 172.21.15.181:0/1401411244 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7f3c08089800 con 0x7f3c1015dc40 2023-10-16T09:56:53.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.632+0000 7f3c0ed9d700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3bfc06b020 0x7f3bfc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:53.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.632+0000 7f3c0ed9d700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3bfc06b020 0x7f3bfc06d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3c040090b0 tx=0x7f3c04008030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:53.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.636+0000 7f3bf77fe700 1 -- 172.21.15.181:0/1401411244 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3c08054720 con 0x7f3c1015dc40 2023-10-16T09:56:53.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:53.909+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f3c100c6580 con 0x7f3c1015dc40 2023-10-16T09:56:54.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.431+0000 7f3bf77fe700 1 -- 172.21.15.181:0/1401411244 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v455) v1 ==== 158+0+0 (secure 0 0 0) 0x7f3c08057d40 con 0x7f3c1015dc40 2023-10-16T09:56:54.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3bfc06b020 msgr2=0x7f3bfc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3bfc06b020 0x7f3bfc06d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3c040090b0 tx=0x7f3c04008030 comp rx=0 tx=0).stop 2023-10-16T09:56:54.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c1015dc40 msgr2=0x7f3c10167fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c1015dc40 0x7f3c10167fa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800cbf0 tx=0x7f3c0800cf00 comp rx=0 tx=0).stop 2023-10-16T09:56:54.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 shutdown_connections 2023-10-16T09:56:54.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3bfc06b020 0x7f3bfc06d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c1015dc40 0x7f3c10167fa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1006ade0 0x7f3c100ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 --2- 172.21.15.181:0/1401411244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c100c2dc0 0x7f3c100c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 >> 172.21.15.181:0/1401411244 conn(0x7f3c100b1780 msgr2=0x7f3c1015a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:54.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 shutdown_connections 2023-10-16T09:56:54.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.434+0000 7f3c151b9700 1 -- 172.21.15.181:0/1401411244 wait complete. 2023-10-16T09:56:54.449 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-16T09:56:54.449 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-16T09:56:54.450 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T09:56:54.450 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:56:54.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.570+0000 7f4af2985700 1 Processor -- start 2023-10-16T09:56:54.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4af2985700 1 -- start start 2023-10-16T09:56:54.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4af2985700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4aec0ce7f0 0x7f4aec06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4af2985700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 0x7f4aec0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4af2985700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec0cc470 con 0x7f4aec0d2390 2023-10-16T09:56:54.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4af2985700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec06ca30 con 0x7f4aec0ce7f0 2023-10-16T09:56:54.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4af2985700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec06cd10 con 0x7f4aec0d0350 2023-10-16T09:56:54.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4aeb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4aec0d2390 0x7f4aec0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51034/0 (socket says 172.21.15.181:51034) 2023-10-16T09:56:54.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.571+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/3994387348 learned_addr learned my addr 172.21.15.181:0/3994387348 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:54.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058228779 0 0) 0x7f4aec0cc470 con 0x7f4aec0d2390 2023-10-16T09:56:54.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ad80035b0 con 0x7f4aec0d2390 2023-10-16T09:56:54.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aebfff700 1 --2- 172.21.15.181:0/3994387348 >> v2:172.21.15.190:3300/0 conn(0x7f4aec0ce7f0 0x7f4aec06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4af0f22700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 0x7f4aec0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1466024273 0 0) 0x7f4ad80035b0 con 0x7f4aec0d2390 2023-10-16T09:56:54.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 msgr2=0x7f4aec0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:54.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 0x7f4aec0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 >> v2:172.21.15.190:3300/0 conn(0x7f4aec0ce7f0 msgr2=0x7f4aec06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:54.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 --2- 172.21.15.181:0/3994387348 >> v2:172.21.15.190:3300/0 conn(0x7f4aec0ce7f0 0x7f4aec06c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec06cff0 con 0x7f4aec0d2390 2023-10-16T09:56:54.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4ad4001670 con 0x7f4aec0d2390 2023-10-16T09:56:54.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 >> v1:172.21.15.181:6789/0 conn(0x7f4aec0d2390 legacy=0x7f4aec0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8003a00 0x7f4ad8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ad8004850 0x7f4ad8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 0x7f4ad8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec06ca30 con 0x7f4ad8004850 2023-10-16T09:56:54.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec06cd10 con 0x7f4ad8003a00 2023-10-16T09:56:54.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad800ba10 con 0x7f4aec0d0350 2023-10-16T09:56:54.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeb7fe700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ad8004850 0x7f4ad8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aebfff700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8003a00 0x7f4ad8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4af0f22700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 0x7f4ad8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 msgr2=0x7f4ad8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:54.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeb7fe700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0d0350 0x7f4ad8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8003a00 msgr2=0x7f4ad8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:54.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeb7fe700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8003a00 0x7f4ad8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/3994387348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80035b0 con 0x7f4ad8004850 2023-10-16T09:56:54.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.572+0000 7f4aebfff700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8003a00 0x7f4ad8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:54.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4aeb7fe700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ad8004850 0x7f4ad8004cd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c370 tx=0x7f4ad8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e880ed522eb95054 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:54.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007c60 con 0x7f4ad8004850 2023-10-16T09:56:54.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4005960 con 0x7f4ad8004850 2023-10-16T09:56:54.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4aeaffd700 1 -- 172.21.15.181:0/3994387348 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4005d10 con 0x7f4ad8004850 2023-10-16T09:56:54.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4af2985700 1 -- 172.21.15.181:0/3994387348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ad8004850 msgr2=0x7f4ad8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4af2985700 1 --2- 172.21.15.181:0/3994387348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ad8004850 0x7f4ad8004cd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c370 tx=0x7f4ad8005880 comp rx=0 tx=0).stop 2023-10-16T09:56:54.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4af0f22700 1 -- 172.21.15.181:0/3994387348 reap_dead start 2023-10-16T09:56:54.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4af2985700 1 -- 172.21.15.181:0/3994387348 shutdown_connections 2023-10-16T09:56:54.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4af2985700 1 -- 172.21.15.181:0/3994387348 >> 172.21.15.181:0/3994387348 conn(0x7f4aec09a9a0 msgr2=0x7f4aec0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4af2985700 1 -- 172.21.15.181:0/3994387348 shutdown_connections 2023-10-16T09:56:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.573+0000 7f4af2985700 1 -- 172.21.15.181:0/3994387348 wait complete. 2023-10-16T09:56:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 Processor -- start 2023-10-16T09:56:54.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 -- start start 2023-10-16T09:56:54.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8004850 0x7f4aec158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aec157a00 0x7f4aec0c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0b3510 0x7f4aec0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec157e80 con 0x7f4aec157a00 2023-10-16T09:56:54.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec1588f0 con 0x7f4ad8004850 2023-10-16T09:56:54.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.575+0000 7f4af2985700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec1f34b0 con 0x7f4aec0b3510 2023-10-16T09:56:54.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aec157a00 0x7f4aec0c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aec157a00 0x7f4aec0c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39968/0 (socket says 172.21.15.181:39968) 2023-10-16T09:56:54.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/2126095675 learned_addr learned my addr 172.21.15.181:0/2126095675 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:54.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4af0f22700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0b3510 0x7f4aec0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aebfff700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8004850 0x7f4aec158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0b3510 msgr2=0x7f4aec0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0b3510 0x7f4aec0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8004850 msgr2=0x7f4aec158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8004850 0x7f4aec158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 -- 172.21.15.181:0/2126095675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec1f3790 con 0x7f4aec157a00 2023-10-16T09:56:54.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4af0f22700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0b3510 0x7f4aec0b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:54.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4aeb7fe700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aec157a00 0x7f4aec0c69e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c960 tx=0x7f4ad4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:54.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4ae8ff9700 1 -- 172.21.15.181:0/2126095675 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4004870 con 0x7f4aec157a00 2023-10-16T09:56:54.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4ae8ff9700 1 -- 172.21.15.181:0/2126095675 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad40049d0 con 0x7f4aec157a00 2023-10-16T09:56:54.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec1f3a70 con 0x7f4aec157a00 2023-10-16T09:56:54.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4ae8ff9700 1 -- 172.21.15.181:0/2126095675 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4004d80 con 0x7f4aec157a00 2023-10-16T09:56:54.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.576+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec0b3ed0 con 0x7f4aec157a00 2023-10-16T09:56:54.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.577+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec067d20 con 0x7f4aec157a00 2023-10-16T09:56:54.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.578+0000 7f4ae8ff9700 1 -- 172.21.15.181:0/2126095675 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4ad40070d0 con 0x7f4aec157a00 2023-10-16T09:56:54.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.578+0000 7f4ae8ff9700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4acc06b020 0x7f4acc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.578+0000 7f4ae8ff9700 1 -- 172.21.15.181:0/2126095675 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7f4ad408b010 con 0x7f4aec157a00 2023-10-16T09:56:54.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.578+0000 7f4aebfff700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4acc06b020 0x7f4acc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.579+0000 7f4aebfff700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4acc06b020 0x7f4acc06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009a10 tx=0x7f4adc008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:54.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.583+0000 7f4ae8ff9700 1 -- 172.21.15.181:0/2126095675 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4ad4056050 con 0x7f4aec157a00 2023-10-16T09:56:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.776+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4aec0cef40 con 0x7f4aec157a00 2023-10-16T09:56:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.777+0000 7f4ae8ff9700 1 -- 172.21.15.181:0/2126095675 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+14577 (secure 0 0 0) 0x7f4ad4007dc0 con 0x7f4aec157a00 2023-10-16T09:56:54.779 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:54.779 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":51,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:56:52.428836+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/760205137":"2023-10-17T09:56:49.424078+0000","172.21.15.181:0/3764257785":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6833/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6832/3948589065":"2023-10-17T09:56:48.399477+0000","172.21.15.181:6832/2039940952":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/4053208405":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/3468053237":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/4251066643":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/2080914758":"2023-10-17T09:45:15.015302+0000","172.21.15.181:6833/3948589065":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/2885824130":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/2848692303":"2023-10-17T09:45:15.015302+0000","172.21.15.181:0/3542266377":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/2553110707":"2023-10-17T09:56:48.399477+0000","172.21.15.181:0/673841945":"2023-10-17T09:56:48.399477+0000","172.21.15.181:6834/760205137":"2023-10-17T09:56:49.424078+0000","172.21.15.181:0/360434361":"2023-10-17T09:56:48.399477+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:56:54.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4acc06b020 msgr2=0x7f4acc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4acc06b020 0x7f4acc06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009a10 tx=0x7f4adc008040 comp rx=0 tx=0).stop 2023-10-16T09:56:54.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aec157a00 msgr2=0x7f4aec0c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aec157a00 0x7f4aec0c69e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c960 tx=0x7f4ad4000fe0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 shutdown_connections 2023-10-16T09:56:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f4acc06b020 0x7f4acc06d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4aec0b3510 0x7f4aec0b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad8004850 0x7f4aec158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 --2- 172.21.15.181:0/2126095675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4aec157a00 0x7f4aec0c69e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 >> 172.21.15.181:0/2126095675 conn(0x7f4aec09a9a0 msgr2=0x7f4aec06bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 shutdown_connections 2023-10-16T09:56:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.780+0000 7f4af2985700 1 -- 172.21.15.181:0/2126095675 wait complete. 2023-10-16T09:56:54.794 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:56:54.794 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:56:54.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.916+0000 7fa511518700 1 Processor -- start 2023-10-16T09:56:54.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.916+0000 7fa511518700 1 -- start start 2023-10-16T09:56:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa511518700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa511518700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa50c0d1500 0x7fa50c0d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa511518700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c0cf580 con 0x7fa50c15ea40 2023-10-16T09:56:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa511518700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c0d4290 con 0x7fa50c0d1500 2023-10-16T09:56:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa511518700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0ce970 con 0x7fa50c0d01d0 2023-10-16T09:56:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa50affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa50b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa50c0d1500 0x7fa50c0d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa50affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:37000/0 (socket says 172.21.15.181:37000) 2023-10-16T09:56:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa50a7fc700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa50c15ea40 0x7fa50c0d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:34994/0 (socket says 172.21.15.181:34994) 2023-10-16T09:56:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.917+0000 7fa50affd700 1 -- 172.21.15.181:0/904538928 learned_addr learned my addr 172.21.15.181:0/904538928 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa50affd700 1 -- 172.21.15.181:0/904538928 >> v2:172.21.15.190:3300/0 conn(0x7fa50c0d1500 msgr2=0x7fa50c0d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa50affd700 1 --2- 172.21.15.181:0/904538928 >> v2:172.21.15.190:3300/0 conn(0x7fa50c0d1500 0x7fa50c0d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa50affd700 1 -- 172.21.15.181:0/904538928 >> v1:172.21.15.190:6789/0 conn(0x7fa50c15ea40 legacy=0x7fa50c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa50affd700 1 -- 172.21.15.181:0/904538928 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c0cec50 con 0x7fa50c0d01d0 2023-10-16T09:56:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa50affd700 1 --2- 172.21.15.181:0/904538928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0ce2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa500009920 tx=0x7fa500009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627ed9ae6210a392 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa509ffb700 1 -- 172.21.15.181:0/904538928 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500011830 con 0x7fa50c0d01d0 2023-10-16T09:56:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa509ffb700 1 -- 172.21.15.181:0/904538928 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa500011990 con 0x7fa50c0d01d0 2023-10-16T09:56:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa509ffb700 1 -- 172.21.15.181:0/904538928 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500011d20 con 0x7fa50c0d01d0 2023-10-16T09:56:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa511518700 1 -- 172.21.15.181:0/904538928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 msgr2=0x7fa50c0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa511518700 1 --2- 172.21.15.181:0/904538928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0ce2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa500009920 tx=0x7fa500009c30 comp rx=0 tx=0).stop 2023-10-16T09:56:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa511518700 1 -- 172.21.15.181:0/904538928 shutdown_connections 2023-10-16T09:56:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa511518700 1 --2- 172.21.15.181:0/904538928 >> v2:172.21.15.190:3300/0 conn(0x7fa50c0d1500 0x7fa50c0d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa511518700 1 --2- 172.21.15.181:0/904538928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0ce2d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.918+0000 7fa511518700 1 -- 172.21.15.181:0/904538928 >> 172.21.15.181:0/904538928 conn(0x7fa50c09cb60 msgr2=0x7fa50c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:54.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.919+0000 7fa511518700 1 -- 172.21.15.181:0/904538928 shutdown_connections 2023-10-16T09:56:54.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.919+0000 7fa511518700 1 -- 172.21.15.181:0/904538928 wait complete. 2023-10-16T09:56:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.920+0000 7fa511518700 1 Processor -- start 2023-10-16T09:56:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.920+0000 7fa511518700 1 -- start start 2023-10-16T09:56:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa511518700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa511518700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50c0d1500 0x7fa50c151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa511518700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa50c15ea40 0x7fa50c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa511518700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0b5270 con 0x7fa50c15ea40 2023-10-16T09:56:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa511518700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0b43e0 con 0x7fa50c0d1500 2023-10-16T09:56:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa511518700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0b45f0 con 0x7fa50c0d01d0 2023-10-16T09:56:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa50c15ea40 0x7fa50c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa50c15ea40 0x7fa50c0caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39984/0 (socket says 172.21.15.181:39984) 2023-10-16T09:56:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 -- 172.21.15.181:0/3378714925 learned_addr learned my addr 172.21.15.181:0/3378714925 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50affd700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50a7fc700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50c0d1500 0x7fa50c151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 -- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 msgr2=0x7fa50c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 -- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50c0d1500 msgr2=0x7fa50c151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50c0d1500 0x7fa50c151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 -- 172.21.15.181:0/3378714925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500009550 con 0x7fa50c15ea40 2023-10-16T09:56:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa50b7fe700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa50c15ea40 0x7fa50c0caac0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa50400efc0 tx=0x7fa50400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa4f37fe700 1 -- 172.21.15.181:0/3378714925 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504009ea0 con 0x7fa50c15ea40 2023-10-16T09:56:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.921+0000 7fa4f37fe700 1 -- 172.21.15.181:0/3378714925 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50400f040 con 0x7fa50c15ea40 2023-10-16T09:56:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.922+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c0b48d0 con 0x7fa50c15ea40 2023-10-16T09:56:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.922+0000 7fa4f37fe700 1 -- 172.21.15.181:0/3378714925 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400ced0 con 0x7fa50c15ea40 2023-10-16T09:56:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.922+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c0b4e90 con 0x7fa50c15ea40 2023-10-16T09:56:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.922+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c04ebb0 con 0x7fa50c15ea40 2023-10-16T09:56:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.923+0000 7fa4f37fe700 1 -- 172.21.15.181:0/3378714925 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa504014480 con 0x7fa50c15ea40 2023-10-16T09:56:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.923+0000 7fa4f37fe700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fa4f806b020 0x7fa4f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.923+0000 7fa4f37fe700 1 -- 172.21.15.181:0/3378714925 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7fa50408a0b0 con 0x7fa50c15ea40 2023-10-16T09:56:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.923+0000 7fa50affd700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fa4f806b020 0x7fa4f806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.924+0000 7fa50affd700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fa4f806b020 0x7fa4f806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa500000c00 tx=0x7fa500008030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:54.928+0000 7fa4f37fe700 1 -- 172.21.15.181:0/3378714925 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa504054ec0 con 0x7fa50c15ea40 2023-10-16T09:56:55.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.118+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa50c0d0e00 con 0x7fa50c15ea40 2023-10-16T09:56:55.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.118+0000 7fa4f37fe700 1 -- 172.21.15.181:0/3378714925 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa504018020 con 0x7fa50c15ea40 2023-10-16T09:56:55.121 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:56:55.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fa4f806b020 msgr2=0x7fa4f806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fa4f806b020 0x7fa4f806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa500000c00 tx=0x7fa500008030 comp rx=0 tx=0).stop 2023-10-16T09:56:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa50c15ea40 msgr2=0x7fa50c0caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa50c15ea40 0x7fa50c0caac0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa50400efc0 tx=0x7fa50400c3a0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 shutdown_connections 2023-10-16T09:56:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fa4f806b020 0x7fa4f806d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50c0d01d0 0x7fa50c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa50c15ea40 0x7fa50c0caac0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 --2- 172.21.15.181:0/3378714925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50c0d1500 0x7fa50c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 >> 172.21.15.181:0/3378714925 conn(0x7fa50c09cb60 msgr2=0x7fa50c06bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:55.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 shutdown_connections 2023-10-16T09:56:55.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.121+0000 7fa511518700 1 -- 172.21.15.181:0/3378714925 wait complete. 2023-10-16T09:56:55.135 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-16T09:56:55.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.253+0000 7f3d2924f700 1 Processor -- start 2023-10-16T09:56:55.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.253+0000 7f3d2924f700 1 -- start start 2023-10-16T09:56:55.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.253+0000 7f3d2924f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3d240c26b0 0x7f3d240c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.253+0000 7f3d2924f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d240b2710 con 0x7f3d2415e020 2023-10-16T09:56:55.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.253+0000 7f3d2924f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d240c18f0 con 0x7f3d2415dc40 2023-10-16T09:56:55.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.253+0000 7f3d2924f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2415ebc0 con 0x7f3d240c26b0 2023-10-16T09:56:55.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d2359e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3d2415e020 0x7f3d2415e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51070/0 (socket says 172.21.15.181:51070) 2023-10-16T09:56:55.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d2359e700 1 -- 172.21.15.181:0/1904090809 learned_addr learned my addr 172.21.15.181:0/1904090809 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:55.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d2259c700 1 --2- 172.21.15.181:0/1904090809 >> v2:172.21.15.190:3300/0 conn(0x7f3d240c26b0 0x7f3d240c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179833342 0 0) 0x7f3d1c0014e0 con 0x7f3d2415e020 2023-10-16T09:56:55.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0c0035b0 con 0x7f3d2415e020 2023-10-16T09:56:55.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 563132866 0 0) 0x7f3d0c0035b0 con 0x7f3d2415e020 2023-10-16T09:56:55.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 >> v2:172.21.15.190:3300/0 conn(0x7f3d240c26b0 msgr2=0x7f3d240c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:55.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 --2- 172.21.15.181:0/1904090809 >> v2:172.21.15.190:3300/0 conn(0x7f3d240c26b0 0x7f3d240c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 >> v1:172.21.15.190:6789/0 conn(0x7f3d2415dc40 legacy=0x7f3d2406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:55.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d240c1bd0 con 0x7f3d2415e020 2023-10-16T09:56:55.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f3d1c001bd0 con 0x7f3d2415e020 2023-10-16T09:56:55.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 >> v1:172.21.15.181:6789/0 conn(0x7f3d2415e020 legacy=0x7f3d2415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c003a00 0x7f3d0c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d0c004850 0x7f3d0c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c005210 0x7f3d0c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2415ebc0 con 0x7f3d0c003a00 2023-10-16T09:56:55.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c00d1c0 con 0x7f3d0c004850 2023-10-16T09:56:55.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c00d380 con 0x7f3d0c005210 2023-10-16T09:56:55.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c003a00 0x7f3d0c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d2259c700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d0c004850 0x7f3d0c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.254+0000 7f3d2359e700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c005210 0x7f3d0c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d22d9d700 1 -- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c005210 msgr2=0x7f3d0c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:55.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c005210 0x7f3d0c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d22d9d700 1 -- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d0c004850 msgr2=0x7f3d0c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:55.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d0c004850 0x7f3d0c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d22d9d700 1 -- 172.21.15.181:0/1904090809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c0035b0 con 0x7f3d0c003a00 2023-10-16T09:56:55.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2259c700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d0c004850 0x7f3d0c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:55.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2359e700 1 -- 172.21.15.181:0/1904090809 reap_dead start 2023-10-16T09:56:55.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c003a00 0x7f3d0c003e20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3d180017c0 tx=0x7f3d0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70ca3a201668f7e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:55.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180052d0 con 0x7f3d0c003a00 2023-10-16T09:56:55.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d18005430 con 0x7f3d0c003a00 2023-10-16T09:56:55.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d21d9b700 1 -- 172.21.15.181:0/1904090809 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180038c0 con 0x7f3d0c003a00 2023-10-16T09:56:55.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2924f700 1 -- 172.21.15.181:0/1904090809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c003a00 msgr2=0x7f3d0c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:55.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2924f700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c003a00 0x7f3d0c003e20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3d180017c0 tx=0x7f3d0c005880 comp rx=0 tx=0).stop 2023-10-16T09:56:55.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2924f700 1 -- 172.21.15.181:0/1904090809 shutdown_connections 2023-10-16T09:56:55.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2924f700 1 --2- 172.21.15.181:0/1904090809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c003a00 0x7f3d0c003e20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2924f700 1 -- 172.21.15.181:0/1904090809 >> 172.21.15.181:0/1904090809 conn(0x7f3d240b1780 msgr2=0x7f3d240b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:55.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2924f700 1 -- 172.21.15.181:0/1904090809 shutdown_connections 2023-10-16T09:56:55.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.255+0000 7f3d2924f700 1 -- 172.21.15.181:0/1904090809 wait complete. 2023-10-16T09:56:55.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.257+0000 7f3d2924f700 1 Processor -- start 2023-10-16T09:56:55.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.257+0000 7f3d2924f700 1 -- start start 2023-10-16T09:56:55.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c003a00 0x7f3d24153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d240720b0 0x7f3d241f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240b0b20 0x7f3d241f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241e4bc0 con 0x7f3d240720b0 2023-10-16T09:56:55.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240c87c0 con 0x7f3d240b0b20 2023-10-16T09:56:55.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240c9620 con 0x7f3d0c003a00 2023-10-16T09:56:55.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d240720b0 0x7f3d241f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d240720b0 0x7f3d241f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40002/0 (socket says 172.21.15.181:40002) 2023-10-16T09:56:55.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 -- 172.21.15.181:0/2505809070 learned_addr learned my addr 172.21.15.181:0/2505809070 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:55.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c003a00 0x7f3d24153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2359e700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240b0b20 0x7f3d241f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 -- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c003a00 msgr2=0x7f3d24153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:55.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c003a00 0x7f3d24153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 -- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240b0b20 msgr2=0x7f3d241f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:55.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240b0b20 0x7f3d241f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:55.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 -- 172.21.15.181:0/2505809070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24153e70 con 0x7f3d240720b0 2023-10-16T09:56:55.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c003a00 0x7f3d24153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:55.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2359e700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240b0b20 0x7f3d241f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:55.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2259c700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d240720b0 0x7f3d241f14f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3d14011930 tx=0x7f3d14011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:55.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d0b7fe700 1 -- 172.21.15.181:0/2505809070 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14004220 con 0x7f3d240720b0 2023-10-16T09:56:55.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d0b7fe700 1 -- 172.21.15.181:0/2505809070 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d14004380 con 0x7f3d240720b0 2023-10-16T09:56:55.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d240cbb40 con 0x7f3d240720b0 2023-10-16T09:56:55.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d0b7fe700 1 -- 172.21.15.181:0/2505809070 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d140036a0 con 0x7f3d240720b0 2023-10-16T09:56:55.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.258+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d240c8cf0 con 0x7f3d240720b0 2023-10-16T09:56:55.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.259+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d24067d20 con 0x7f3d240720b0 2023-10-16T09:56:55.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.260+0000 7f3d0b7fe700 1 -- 172.21.15.181:0/2505809070 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3d14003800 con 0x7f3d240720b0 2023-10-16T09:56:55.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.260+0000 7f3d0b7fe700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3d1006b050 0x7f3d1006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:55.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.260+0000 7f3d0b7fe700 1 -- 172.21.15.181:0/2505809070 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7f3d14089bb0 con 0x7f3d240720b0 2023-10-16T09:56:55.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.260+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3d1006b050 0x7f3d1006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:55.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.261+0000 7f3d22d9d700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3d1006b050 0x7f3d1006d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3d18000c00 tx=0x7f3d18009030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:55.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.265+0000 7f3d0b7fe700 1 -- 172.21.15.181:0/2505809070 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3d14054a40 con 0x7f3d240720b0 2023-10-16T09:56:55.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:55.537+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f3d24072510 con 0x7f3d240720b0 2023-10-16T09:56:56.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.440+0000 7f3d0b7fe700 1 -- 172.21.15.181:0/2505809070 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v457) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3d14022020 con 0x7f3d240720b0 2023-10-16T09:56:56.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3d1006b050 msgr2=0x7f3d1006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3d1006b050 0x7f3d1006d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3d18000c00 tx=0x7f3d18009030 comp rx=0 tx=0).stop 2023-10-16T09:56:56.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d240720b0 msgr2=0x7f3d241f14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d240720b0 0x7f3d241f14f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3d14011930 tx=0x7f3d14011cf0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 shutdown_connections 2023-10-16T09:56:56.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3d1006b050 0x7f3d1006d510 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d0c003a00 0x7f3d24153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d240b0b20 0x7f3d241f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 --2- 172.21.15.181:0/2505809070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d240720b0 0x7f3d241f14f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 >> 172.21.15.181:0/2505809070 conn(0x7f3d240b1780 msgr2=0x7f3d240cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:56.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 shutdown_connections 2023-10-16T09:56:56.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.443+0000 7f3d2924f700 1 -- 172.21.15.181:0/2505809070 wait complete. 2023-10-16T09:56:56.460 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T09:56:56.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.587+0000 7fb82065e700 1 Processor -- start 2023-10-16T09:56:56.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.587+0000 7fb82065e700 1 -- start start 2023-10-16T09:56:56.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb82065e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb81815d9e0 0x7fb8180d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:56.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb82065e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8180d07d0 0x7fb8180cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:56.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb82065e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8180d4180 con 0x7fb8180cefa0 2023-10-16T09:56:56.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb82065e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8180d20c0 con 0x7fb8180d07d0 2023-10-16T09:56:56.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb82065e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180d1440 con 0x7fb81815d9e0 2023-10-16T09:56:56.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb81ebfb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8180d07d0 0x7fb8180cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:56.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb81dbf9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb8180cefa0 0x7fb8180d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51086/0 (socket says 172.21.15.181:51086) 2023-10-16T09:56:56.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb81ebfb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8180d07d0 0x7fb8180cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40012/0 (socket says 172.21.15.181:40012) 2023-10-16T09:56:56.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb81dbf9700 1 -- 172.21.15.181:0/2983433181 learned_addr learned my addr 172.21.15.181:0/2983433181 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:56.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.588+0000 7fb81e3fa700 1 --2- 172.21.15.181:0/2983433181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb81815d9e0 0x7fb8180d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:56.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81ebfb700 1 -- 172.21.15.181:0/2983433181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb81815d9e0 msgr2=0x7fb8180d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:56.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81ebfb700 1 --2- 172.21.15.181:0/2983433181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb81815d9e0 0x7fb8180d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81ebfb700 1 -- 172.21.15.181:0/2983433181 >> v1:172.21.15.181:6789/0 conn(0x7fb8180cefa0 legacy=0x7fb8180d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81d3f8700 1 -- 172.21.15.181:0/2983433181 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043947415 0 0) 0x7fb8180d4180 con 0x7fb8180cefa0 2023-10-16T09:56:56.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81ebfb700 1 -- 172.21.15.181:0/2983433181 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8180d1720 con 0x7fb8180d07d0 2023-10-16T09:56:56.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81ebfb700 1 --2- 172.21.15.181:0/2983433181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8180d07d0 0x7fb8180cf420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb810009d90 tx=0x7fb81000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34463005070a4b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:56.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81d3f8700 1 -- 172.21.15.181:0/2983433181 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000e070 con 0x7fb8180d07d0 2023-10-16T09:56:56.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81d3f8700 1 -- 172.21.15.181:0/2983433181 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81000bbd0 con 0x7fb8180d07d0 2023-10-16T09:56:56.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb81d3f8700 1 -- 172.21.15.181:0/2983433181 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810012430 con 0x7fb8180d07d0 2023-10-16T09:56:56.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb82065e700 1 -- 172.21.15.181:0/2983433181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8180d07d0 msgr2=0x7fb8180cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.589+0000 7fb82065e700 1 --2- 172.21.15.181:0/2983433181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8180d07d0 0x7fb8180cf420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb810009d90 tx=0x7fb81000b330 comp rx=0 tx=0).stop 2023-10-16T09:56:56.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.590+0000 7fb82065e700 1 -- 172.21.15.181:0/2983433181 shutdown_connections 2023-10-16T09:56:56.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.590+0000 7fb82065e700 1 --2- 172.21.15.181:0/2983433181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8180d07d0 0x7fb8180cf420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.590+0000 7fb82065e700 1 --2- 172.21.15.181:0/2983433181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb81815d9e0 0x7fb8180d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.590+0000 7fb82065e700 1 -- 172.21.15.181:0/2983433181 >> 172.21.15.181:0/2983433181 conn(0x7fb81809cb40 msgr2=0x7fb8180d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:56.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.590+0000 7fb82065e700 1 -- 172.21.15.181:0/2983433181 shutdown_connections 2023-10-16T09:56:56.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.590+0000 7fb82065e700 1 -- 172.21.15.181:0/2983433181 wait complete. 2023-10-16T09:56:56.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 Processor -- start 2023-10-16T09:56:56.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 -- start start 2023-10-16T09:56:56.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8180cefa0 0x7fb8180b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:56.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8180d07d0 0x7fb818159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:56.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81815d9e0 0x7fb8180cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:56.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180b2aa0 con 0x7fb81815d9e0 2023-10-16T09:56:56.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180b2d80 con 0x7fb8180d07d0 2023-10-16T09:56:56.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.591+0000 7fb82065e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180b3060 con 0x7fb8180cefa0 2023-10-16T09:56:56.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81815d9e0 0x7fb8180cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:56.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81815d9e0 0x7fb8180cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40018/0 (socket says 172.21.15.181:40018) 2023-10-16T09:56:56.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 -- 172.21.15.181:0/951820486 learned_addr learned my addr 172.21.15.181:0/951820486 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:56.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81e3fa700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8180cefa0 0x7fb8180b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:56.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81dbf9700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8180d07d0 0x7fb818159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:56.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 -- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8180cefa0 msgr2=0x7fb8180b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8180cefa0 0x7fb8180b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 -- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8180d07d0 msgr2=0x7fb818159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8180d07d0 0x7fb818159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 -- 172.21.15.181:0/951820486 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810009a70 con 0x7fb81815d9e0 2023-10-16T09:56:56.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb81ebfb700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81815d9e0 0x7fb8180cbe90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb810013000 tx=0x7fb810004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:56.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb80effd700 1 -- 172.21.15.181:0/951820486 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810012430 con 0x7fb81815d9e0 2023-10-16T09:56:56.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb80effd700 1 -- 172.21.15.181:0/951820486 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb810020070 con 0x7fb81815d9e0 2023-10-16T09:56:56.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8180b3340 con 0x7fb81815d9e0 2023-10-16T09:56:56.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb80effd700 1 -- 172.21.15.181:0/951820486 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810004280 con 0x7fb81815d9e0 2023-10-16T09:56:56.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.592+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81815a430 con 0x7fb81815d9e0 2023-10-16T09:56:56.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.593+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8180b3860 con 0x7fb81815d9e0 2023-10-16T09:56:56.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.593+0000 7fb80effd700 1 -- 172.21.15.181:0/951820486 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb8100043e0 con 0x7fb81815d9e0 2023-10-16T09:56:56.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.594+0000 7fb80effd700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb7fc06b020 0x7fb7fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:56.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.594+0000 7fb81e3fa700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb7fc06b020 0x7fb7fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:56.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.594+0000 7fb80effd700 1 -- 172.21.15.181:0/951820486 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7fb810022070 con 0x7fb81815d9e0 2023-10-16T09:56:56.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.594+0000 7fb81e3fa700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb7fc06b020 0x7fb7fc06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb808009950 tx=0x7fb808008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:56.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.599+0000 7fb80effd700 1 -- 172.21.15.181:0/951820486 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8180b3860 con 0x7fb81815d9e0 2023-10-16T09:56:56.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.794+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb8180b3860 con 0x7fb81815d9e0 2023-10-16T09:56:56.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.797+0000 7fb80effd700 1 -- 172.21.15.181:0/951820486 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb8180b3860 con 0x7fb81815d9e0 2023-10-16T09:56:56.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb7fc06b020 msgr2=0x7fb7fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb7fc06b020 0x7fb7fc06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb808009950 tx=0x7fb808008040 comp rx=0 tx=0).stop 2023-10-16T09:56:56.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81815d9e0 msgr2=0x7fb8180cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:56.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81815d9e0 0x7fb8180cbe90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb810013000 tx=0x7fb810004ad0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 shutdown_connections 2023-10-16T09:56:56.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb7fc06b020 0x7fb7fc06d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8180cefa0 0x7fb8180b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81815d9e0 0x7fb8180cbe90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 --2- 172.21.15.181:0/951820486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8180d07d0 0x7fb818159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:56.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 >> 172.21.15.181:0/951820486 conn(0x7fb81809cb40 msgr2=0x7fb818161280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:56.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 shutdown_connections 2023-10-16T09:56:56.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:56.799+0000 7fb82065e700 1 -- 172.21.15.181:0/951820486 wait complete. 2023-10-16T09:56:56.813 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:56:56.814 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:56:56.858 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:56:56.909 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:56:56.909 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:56:56.910 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:56.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:56:56.927 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:56:56.928 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:56.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:56:56.948 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:56:56.948 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:56.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:56:56.985 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:56:56.985 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:56:57.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:56:57.007 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T09:56:57.007 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:56:57.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.142+0000 7ffad7bc1700 1 Processor -- start 2023-10-16T09:56:57.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.142+0000 7ffad7bc1700 1 -- start start 2023-10-16T09:56:57.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad7bc1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 0x7ffad00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad7bc1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad006ed00 con 0x7ffad00cb0c0 2023-10-16T09:56:57.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad7bc1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad00d02d0 con 0x7ffad015eab0 2023-10-16T09:56:57.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad7bc1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad006e0f0 con 0x7ffad015e6d0 2023-10-16T09:56:57.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad615e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ffad00cb0c0 0x7ffad015a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51102/0 (socket says 172.21.15.181:51102) 2023-10-16T09:56:57.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad615e700 1 -- 172.21.15.181:0/4085459844 learned_addr learned my addr 172.21.15.181:0/4085459844 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:57.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad595d700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 0x7ffad00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446805649 0 0) 0x7ffad006ed00 con 0x7ffad00cb0c0 2023-10-16T09:56:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffabc0035b0 con 0x7ffad00cb0c0 2023-10-16T09:56:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747962757 0 0) 0x7ffad00d02d0 con 0x7ffad015eab0 2023-10-16T09:56:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad006ed00 con 0x7ffad015eab0 2023-10-16T09:56:57.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2835600286 0 0) 0x7ffabc0035b0 con 0x7ffad00cb0c0 2023-10-16T09:56:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 msgr2=0x7ffad00d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 0x7ffad00d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 >> v1:172.21.15.190:6789/0 conn(0x7ffad015eab0 legacy=0x7ffad015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad006e3d0 con 0x7ffad00cb0c0 2023-10-16T09:56:57.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ffac80019f0 con 0x7ffad00cb0c0 2023-10-16T09:56:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.143+0000 7ffad595d700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 0x7ffad00d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:56:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 >> v1:172.21.15.181:6789/0 conn(0x7ffad00cb0c0 legacy=0x7ffad015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 0x7ffabc003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffabc004150 0x7ffabc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffabc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad006e0f0 con 0x7ffabc004d90 2023-10-16T09:56:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc00bb20 con 0x7ffabc004150 2023-10-16T09:56:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc00bce0 con 0x7ffad015e6d0 2023-10-16T09:56:57.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad615e700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffabc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad595d700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 0x7ffabc003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad515c700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffabc004150 0x7ffabc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad615e700 1 -- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 msgr2=0x7ffabc003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad615e700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015e6d0 0x7ffabc003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad615e700 1 -- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffabc004150 msgr2=0x7ffabc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad615e700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffabc004150 0x7ffabc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad615e700 1 -- 172.21.15.181:0/4085459844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc009430 con 0x7ffabc004d90 2023-10-16T09:56:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad615e700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffabc005210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffac800a6e0 tx=0x7ffabc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d96273bc37d2679 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80072d0 con 0x7ffabc004d90 2023-10-16T09:56:57.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac8007450 con 0x7ffabc004d90 2023-10-16T09:56:57.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.144+0000 7ffad495b700 1 -- 172.21.15.181:0/4085459844 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8007800 con 0x7ffabc004d90 2023-10-16T09:56:57.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.145+0000 7ffad7bc1700 1 -- 172.21.15.181:0/4085459844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 msgr2=0x7ffabc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.145+0000 7ffad7bc1700 1 --2- 172.21.15.181:0/4085459844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffabc005210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffac800a6e0 tx=0x7ffabc005880 comp rx=0 tx=0).stop 2023-10-16T09:56:57.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.145+0000 7ffad615e700 1 -- 172.21.15.181:0/4085459844 reap_dead start 2023-10-16T09:56:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.145+0000 7ffad7bc1700 1 -- 172.21.15.181:0/4085459844 shutdown_connections 2023-10-16T09:56:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.145+0000 7ffad7bc1700 1 -- 172.21.15.181:0/4085459844 >> 172.21.15.181:0/4085459844 conn(0x7ffad00d04a0 msgr2=0x7ffad00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.145+0000 7ffad7bc1700 1 -- 172.21.15.181:0/4085459844 shutdown_connections 2023-10-16T09:56:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.145+0000 7ffad7bc1700 1 -- 172.21.15.181:0/4085459844 wait complete. 2023-10-16T09:56:57.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.146+0000 7ffad7bc1700 1 Processor -- start 2023-10-16T09:56:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.146+0000 7ffad7bc1700 1 -- start start 2023-10-16T09:56:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad7bc1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffad015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad7bc1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad014f790 0x7ffad00af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad7bc1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad00b5580 0x7ffad00b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad7bc1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00b5f40 con 0x7ffabc004d90 2023-10-16T09:56:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad7bc1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00b3670 con 0x7ffad00b5580 2023-10-16T09:56:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad7bc1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad015cf70 con 0x7ffad014f790 2023-10-16T09:56:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffad015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffad015ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40034/0 (socket says 172.21.15.181:40034) 2023-10-16T09:56:57.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 -- 172.21.15.181:0/3116541952 learned_addr learned my addr 172.21.15.181:0/3116541952 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 -- 172.21.15.181:0/3116541952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad014f790 msgr2=0x7ffad00af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad014f790 0x7ffad00af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 -- 172.21.15.181:0/3116541952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad00b5580 msgr2=0x7ffad00b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad615e700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad00b5580 0x7ffad00b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad00b5580 0x7ffad00b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 -- 172.21.15.181:0/3116541952 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00b2070 con 0x7ffabc004d90 2023-10-16T09:56:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.147+0000 7ffad595d700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffad015ca30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffacc011d00 tx=0x7ffacc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:57.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.148+0000 7ffac67fc700 1 -- 172.21.15.181:0/3116541952 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00fb30 con 0x7ffabc004d90 2023-10-16T09:56:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.148+0000 7ffac67fc700 1 -- 172.21.15.181:0/3116541952 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc00fc90 con 0x7ffabc004d90 2023-10-16T09:56:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.148+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad00b2350 con 0x7ffabc004d90 2023-10-16T09:56:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.148+0000 7ffac67fc700 1 -- 172.21.15.181:0/3116541952 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc014440 con 0x7ffabc004d90 2023-10-16T09:56:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.148+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad00b2910 con 0x7ffabc004d90 2023-10-16T09:56:57.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.148+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad00ae930 con 0x7ffabc004d90 2023-10-16T09:56:57.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.149+0000 7ffac67fc700 1 -- 172.21.15.181:0/3116541952 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffacc01d400 con 0x7ffabc004d90 2023-10-16T09:56:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.150+0000 7ffac67fc700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ffab406b050 0x7ffab406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.150+0000 7ffac67fc700 1 -- 172.21.15.181:0/3116541952 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7ffacc08a5e0 con 0x7ffabc004d90 2023-10-16T09:56:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.150+0000 7ffad515c700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ffab406b050 0x7ffab406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.150+0000 7ffad515c700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ffab406b050 0x7ffab406d510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffac0007a30 tx=0x7ffac0009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:57.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.154+0000 7ffac67fc700 1 -- 172.21.15.181:0/3116541952 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffacc055500 con 0x7ffabc004d90 2023-10-16T09:56:57.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.389+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffad0067d20 con 0x7ffabc004d90 2023-10-16T09:56:57.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.390+0000 7ffac67fc700 1 -- 172.21.15.181:0/3116541952 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3705 (secure 0 0 0) 0x7ffacc022020 con 0x7ffabc004d90 2023-10-16T09:56:57.392 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:57.392 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3974533571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3974533571},{"type":"v1","addr":"172.21.15.190:6834","nonce":3974533571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3064364132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3064364132},{"type":"v1","addr":"172.21.15.181:6837","nonce":3064364132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4021280674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4021280674},{"type":"v1","addr":"172.21.15.190:6835","nonce":4021280674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/84272236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":84272236},{"type":"v1","addr":"172.21.15.181:6836","nonce":84272236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2023-10-16T09:56:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ffab406b050 msgr2=0x7ffab406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ffab406b050 0x7ffab406d510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffac0007a30 tx=0x7ffac0009040 comp rx=0 tx=0).stop 2023-10-16T09:56:57.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 msgr2=0x7ffad015ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffad015ca30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffacc011d00 tx=0x7ffacc00f520 comp rx=0 tx=0).stop 2023-10-16T09:56:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 shutdown_connections 2023-10-16T09:56:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ffab406b050 0x7ffab406d510 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad00b5580 0x7ffad00b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc004d90 0x7ffad015ca30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 --2- 172.21.15.181:0/3116541952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad014f790 0x7ffad00af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 >> 172.21.15.181:0/3116541952 conn(0x7ffad00d04a0 msgr2=0x7ffad00c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 shutdown_connections 2023-10-16T09:56:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.392+0000 7ffad7bc1700 1 -- 172.21.15.181:0/3116541952 wait complete. 2023-10-16T09:56:57.396 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 18 2023-10-16T09:56:57.406 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T09:56:57.407 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T09:56:57.407 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T09:56:57.407 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:56:57.408 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T09:56:57.408 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T09:56:57.408 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T09:56:57.408 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:56:57.409 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T09:56:57.409 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T09:56:57.409 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T09:56:57.410 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:56:57.410 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T09:56:57.410 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T09:56:57.411 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T09:56:57.411 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:56:57.416 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T09:56:57.416 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T09:56:57.417 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T09:56:57.417 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T09:56:57.421 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T09:56:57.421 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T09:56:57.422 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T09:56:57.422 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T09:56:57.459 INFO:tasks.ceph.mds.a:Started 2023-10-16T09:56:57.459 DEBUG:teuthology.parallel:result is None 2023-10-16T09:56:57.460 INFO:tasks.ceph.mds.c:Started 2023-10-16T09:56:57.461 DEBUG:teuthology.parallel:result is None 2023-10-16T09:56:57.463 INFO:tasks.ceph.mds.b:Started 2023-10-16T09:56:57.464 DEBUG:teuthology.parallel:result is None 2023-10-16T09:56:57.464 INFO:tasks.ceph.mds.d:Started 2023-10-16T09:56:57.465 DEBUG:teuthology.parallel:result is None 2023-10-16T09:56:57.465 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T09:56:57.538 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T09:56:57.539 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T09:56:57.550 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T09:56:57.551 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T09:56:57.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.591+0000 7fb485fb4700 1 Processor -- start 2023-10-16T09:56:57.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.591+0000 7fb485fb4700 1 -- start start 2023-10-16T09:56:57.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb485fb4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4800c3320 0x7fb48006cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb485fb4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cd8f0 0x7fb4800cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb485fb4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4800cbf90 con 0x7fb4800cde40 2023-10-16T09:56:57.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb485fb4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48015e170 con 0x7fb4800c3320 2023-10-16T09:56:57.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb485fb4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb480066640 con 0x7fb4800cd8f0 2023-10-16T09:56:57.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb47effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4800c3320 0x7fb48006cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb47effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4800c3320 0x7fb48006cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40066/0 (socket says 172.21.15.181:40066) 2023-10-16T09:56:57.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb47effd700 1 -- 172.21.15.181:0/2422968597 learned_addr learned my addr 172.21.15.181:0/2422968597 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:57.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb47ffff700 1 --2- 172.21.15.181:0/2422968597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cd8f0 0x7fb4800cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb47effd700 1 -- 172.21.15.181:0/2422968597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cd8f0 msgr2=0x7fb4800cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:57.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.592+0000 7fb47effd700 1 --2- 172.21.15.181:0/2422968597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cd8f0 0x7fb4800cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb47effd700 1 -- 172.21.15.181:0/2422968597 >> v1:172.21.15.190:6789/0 conn(0x7fb4800cde40 legacy=0x7fb4800ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb47effd700 1 -- 172.21.15.181:0/2422968597 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800b22a0 con 0x7fb4800c3320 2023-10-16T09:56:57.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb47effd700 1 --2- 172.21.15.181:0/2422968597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb48006cdf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb4680098f0 tx=0x7fb468009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d059f5f444356c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:57.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb47e7fc700 1 -- 172.21.15.181:0/2422968597 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468011830 con 0x7fb4800c3320 2023-10-16T09:56:57.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb47e7fc700 1 -- 172.21.15.181:0/2422968597 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468011990 con 0x7fb4800c3320 2023-10-16T09:56:57.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb47e7fc700 1 -- 172.21.15.181:0/2422968597 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468011d20 con 0x7fb4800c3320 2023-10-16T09:56:57.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 -- 172.21.15.181:0/2422968597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 msgr2=0x7fb48006cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2422968597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb48006cdf0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb4680098f0 tx=0x7fb468009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:57.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 -- 172.21.15.181:0/2422968597 shutdown_connections 2023-10-16T09:56:57.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2422968597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cd8f0 0x7fb4800cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2422968597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb48006cdf0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 -- 172.21.15.181:0/2422968597 >> 172.21.15.181:0/2422968597 conn(0x7fb4800b1780 msgr2=0x7fb4800b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:57.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 -- 172.21.15.181:0/2422968597 shutdown_connections 2023-10-16T09:56:57.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.593+0000 7fb485fb4700 1 -- 172.21.15.181:0/2422968597 wait complete. 2023-10-16T09:56:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 Processor -- start 2023-10-16T09:56:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 -- start start 2023-10-16T09:56:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb4800b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4800cd8f0 0x7fb4800c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cde40 0x7fb48006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800b4d00 con 0x7fb4800c3320 2023-10-16T09:56:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800c8d00 con 0x7fb4800cd8f0 2023-10-16T09:56:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.595+0000 7fb485fb4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800c9b20 con 0x7fb4800cde40 2023-10-16T09:56:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb4800b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb4800b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40070/0 (socket says 172.21.15.181:40070) 2023-10-16T09:56:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 -- 172.21.15.181:0/2443442660 learned_addr learned my addr 172.21.15.181:0/2443442660 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47effd700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4800cd8f0 0x7fb4800c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47ffff700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cde40 0x7fb48006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 -- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cde40 msgr2=0x7fb48006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cde40 0x7fb48006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 -- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4800cd8f0 msgr2=0x7fb4800c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4800cd8f0 0x7fb4800c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 -- 172.21.15.181:0/2443442660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468009550 con 0x7fb4800c3320 2023-10-16T09:56:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb47f7fe700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb4800b47c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c4e0 tx=0x7fb47000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb477fff700 1 -- 172.21.15.181:0/2443442660 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4700052d0 con 0x7fb4800c3320 2023-10-16T09:56:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb477fff700 1 -- 172.21.15.181:0/2443442660 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb470005430 con 0x7fb4800c3320 2023-10-16T09:56:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4800c8fa0 con 0x7fb4800c3320 2023-10-16T09:56:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb477fff700 1 -- 172.21.15.181:0/2443442660 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4700038c0 con 0x7fb4800c3320 2023-10-16T09:56:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.596+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4800c9560 con 0x7fb4800c3320 2023-10-16T09:56:57.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.597+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb480067d20 con 0x7fb4800c3320 2023-10-16T09:56:57.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.598+0000 7fb477fff700 1 -- 172.21.15.181:0/2443442660 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb470003a20 con 0x7fb4800c3320 2023-10-16T09:56:57.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.598+0000 7fb477fff700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb46006b020 0x7fb46006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:57.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.598+0000 7fb477fff700 1 -- 172.21.15.181:0/2443442660 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5984+0+0 (secure 0 0 0) 0x7fb470088b10 con 0x7fb4800c3320 2023-10-16T09:56:57.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.598+0000 7fb47effd700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb46006b020 0x7fb46006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:57.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.598+0000 7fb47effd700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb46006b020 0x7fb46006d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb468014040 tx=0x7fb468008030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:57.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.602+0000 7fb477fff700 1 -- 172.21.15.181:0/2443442660 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb470053a30 con 0x7fb4800c3320 2023-10-16T09:56:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:57.807+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb4800b5940 con 0x7fb4800c3320 2023-10-16T09:56:58.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.451+0000 7fb477fff700 1 -- 172.21.15.181:0/2443442660 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb470017080 con 0x7fb4800c3320 2023-10-16T09:56:58.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb46006b020 msgr2=0x7fb46006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb46006b020 0x7fb46006d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb468014040 tx=0x7fb468008030 comp rx=0 tx=0).stop 2023-10-16T09:56:58.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 msgr2=0x7fb4800b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb4800b47c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c4e0 tx=0x7fb47000c7f0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 shutdown_connections 2023-10-16T09:56:58.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb46006b020 0x7fb46006d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4800cde40 0x7fb48006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4800c3320 0x7fb4800b47c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 --2- 172.21.15.181:0/2443442660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4800cd8f0 0x7fb4800c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 >> 172.21.15.181:0/2443442660 conn(0x7fb4800b1780 msgr2=0x7fb4800712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:58.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 shutdown_connections 2023-10-16T09:56:58.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.454+0000 7fb485fb4700 1 -- 172.21.15.181:0/2443442660 wait complete. 2023-10-16T09:56:58.459 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T09:56:58.471 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T09:56:58.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.591+0000 7fe4fae6e700 1 Processor -- start 2023-10-16T09:56:58.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.591+0000 7fe4fae6e700 1 -- start start 2023-10-16T09:56:58.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.592+0000 7fe4fae6e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4f415e6d0 0x7fe4f40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:58.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.592+0000 7fe4fae6e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f406ed00 con 0x7fe4f415eab0 2023-10-16T09:56:58.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.592+0000 7fe4fae6e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f40d02d0 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.592+0000 7fe4fae6e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f406e0f0 con 0x7fe4f415e6d0 2023-10-16T09:56:58.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.592+0000 7fe4f8c0a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4f415e6d0 0x7fe4f40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:58.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.592+0000 7fe4f3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe4f415eab0 0x7fe4f415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51158/0 (socket says 172.21.15.181:51158) 2023-10-16T09:56:58.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.592+0000 7fe4f3fff700 1 -- 172.21.15.181:0/3167585816 learned_addr learned my addr 172.21.15.181:0/3167585816 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:58.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f8c0a700 1 -- 172.21.15.181:0/3167585816 >> v1:172.21.15.190:6789/0 conn(0x7fe4f40cb0c0 legacy=0x7fe4f415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:58.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f8c0a700 1 -- 172.21.15.181:0/3167585816 >> v1:172.21.15.181:6789/0 conn(0x7fe4f415eab0 legacy=0x7fe4f415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f37fe700 1 -- 172.21.15.181:0/3167585816 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710594356 0 0) 0x7fe4f406ed00 con 0x7fe4f415eab0 2023-10-16T09:56:58.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f8c0a700 1 -- 172.21.15.181:0/3167585816 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f406e3d0 con 0x7fe4f415e6d0 2023-10-16T09:56:58.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f8c0a700 1 --2- 172.21.15.181:0/3167585816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f415e6d0 0x7fe4f40d0fc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009820 tx=0x7fe4e8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348c13f0deb2cfdd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:58.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f37fe700 1 -- 172.21.15.181:0/3167585816 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8005e30 con 0x7fe4f415e6d0 2023-10-16T09:56:58.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f37fe700 1 -- 172.21.15.181:0/3167585816 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e8004030 con 0x7fe4f415e6d0 2023-10-16T09:56:58.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4f37fe700 1 -- 172.21.15.181:0/3167585816 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80043c0 con 0x7fe4f415e6d0 2023-10-16T09:56:58.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3167585816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f415e6d0 msgr2=0x7fe4f40d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3167585816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f415e6d0 0x7fe4f40d0fc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009820 tx=0x7fe4e8009b30 comp rx=0 tx=0).stop 2023-10-16T09:56:58.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3167585816 shutdown_connections 2023-10-16T09:56:58.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3167585816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f415e6d0 0x7fe4f40d0fc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3167585816 >> 172.21.15.181:0/3167585816 conn(0x7fe4f40d04a0 msgr2=0x7fe4f40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:58.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.593+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3167585816 shutdown_connections 2023-10-16T09:56:58.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.594+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3167585816 wait complete. 2023-10-16T09:56:58.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.595+0000 7fe4fae6e700 1 Processor -- start 2023-10-16T09:56:58.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.595+0000 7fe4fae6e700 1 -- start start 2023-10-16T09:56:58.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.595+0000 7fe4fae6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f40cb0c0 0x7fe4f4157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:58.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.595+0000 7fe4fae6e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f415e6d0 0x7fe4f415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:58.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4fae6e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f415eab0 0x7fe4f415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:58.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4fae6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f415c430 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4fae6e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f4158010 con 0x7fe4f415eab0 2023-10-16T09:56:58.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4fae6e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f415cd90 con 0x7fe4f415e6d0 2023-10-16T09:56:58.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f40cb0c0 0x7fe4f4157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:58.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f40cb0c0 0x7fe4f4157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40084/0 (socket says 172.21.15.181:40084) 2023-10-16T09:56:58.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 -- 172.21.15.181:0/3795545704 learned_addr learned my addr 172.21.15.181:0/3795545704 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:58.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f3fff700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f415e6d0 0x7fe4f415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:58.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f940b700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f415eab0 0x7fe4f415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:58.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 -- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f415e6d0 msgr2=0x7fe4f415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f415e6d0 0x7fe4f415c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 -- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f415eab0 msgr2=0x7fe4f415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f415eab0 0x7fe4f415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 -- 172.21.15.181:0/3795545704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e8009430 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f3fff700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f415e6d0 0x7fe4f415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:58.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f940b700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f415eab0 0x7fe4f415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:56:58.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f8c0a700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f40cb0c0 0x7fe4f4157ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8000c00 tx=0x7fe4e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:58.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f17fa700 1 -- 172.21.15.181:0/3795545704 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e801c620 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f17fa700 1 -- 172.21.15.181:0/3795545704 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e801c7a0 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4f17fa700 1 -- 172.21.15.181:0/3795545704 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e801cb50 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f415b7b0 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.596+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f415bd70 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.597+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f415c6a0 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.598+0000 7fe4f17fa700 1 -- 172.21.15.181:0/3795545704 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe4e801ccb0 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.598+0000 7fe4f17fa700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fe4d806b020 0x7fe4d806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:58.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.598+0000 7fe4f17fa700 1 -- 172.21.15.181:0/3795545704 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe4e808a970 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.598+0000 7fe4f3fff700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fe4d806b020 0x7fe4d806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:58.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.598+0000 7fe4f3fff700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fe4d806b020 0x7fe4d806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0007fd0 tx=0x7fe4e0009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:58.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.602+0000 7fe4f17fa700 1 -- 172.21.15.181:0/3795545704 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe4e8055b60 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.875+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe4f4067d20 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.876+0000 7fe4f17fa700 1 -- 172.21.15.181:0/3795545704 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe4e8016090 con 0x7fe4f40cb0c0 2023-10-16T09:56:58.878 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:58.878 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:56:58.879 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T09:56:58.879 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.879 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T09:56:58.879 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T09:56:58.880 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.880 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.880 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:56:58.880 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:56:58.880 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.881 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.881 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.881 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T09:56:58.881 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T09:56:58.882 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.882 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.882 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:56:58.882 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:56:58.882 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.883 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.883 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.883 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T09:56:58.883 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T09:56:58.883 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.884 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.884 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:56:58.884 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:56:58.884 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.885 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.885 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.885 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T09:56:58.885 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T09:56:58.885 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.886 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.886 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:56:58.886 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:56:58.886 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.886 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.887 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T09:56:58.887 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T09:56:58.887 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.888 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.888 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.888 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.888 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.888 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.889 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T09:56:58.889 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T09:56:58.889 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.889 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.889 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.890 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.890 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.890 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.890 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T09:56:58.890 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T09:56:58.891 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.891 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.891 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.891 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.891 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.892 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T09:56:58.892 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T09:56:58.892 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.892 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.893 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.893 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.893 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.893 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.893 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T09:56:58.894 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T09:56:58.894 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.894 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.894 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.894 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.895 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.895 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T09:56:58.895 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T09:56:58.895 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.896 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.896 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.896 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.896 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.897 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T09:56:58.897 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T09:56:58.897 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.897 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.897 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.898 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.898 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.898 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T09:56:58.899 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T09:56:58.899 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.899 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:56:58.899 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:56:58.899 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.900 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.900 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.900 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T09:56:58.900 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T09:56:58.901 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.901 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.901 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T09:56:58.901 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T09:56:58.901 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.902 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T09:56:58.902 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T09:56:58.902 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.903 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.903 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:56:58.903 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:56:58.903 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:56:58.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.904 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.904 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T09:56:58.904 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T09:56:58.904 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.905 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.905 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:56:58.905 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:56:58.905 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:56:58.905 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.906 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T09:56:58.906 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T09:56:58.906 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.906 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:56:58.907 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:56:58.907 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:56:58.907 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:56:58.907 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.907 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.908 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T09:56:58.908 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T09:56:58.908 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.908 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:56:58.908 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:56:58.908 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:56:58.909 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.909 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.909 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.909 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T09:56:58.910 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T09:56:58.910 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.910 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:56:58.910 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:56:58.911 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:56:58.911 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:58.911 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T09:56:58.912 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T09:56:58.912 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:56:58.912 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:56:58.912 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:56:58.912 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:56:58.913 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.913 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:58.913 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T09:56:58.913 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:56:58.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fe4d806b020 msgr2=0x7fe4d806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fe4d806b020 0x7fe4d806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0007fd0 tx=0x7fe4e0009040 comp rx=0 tx=0).stop 2023-10-16T09:56:58.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f40cb0c0 msgr2=0x7fe4f4157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:58.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f40cb0c0 0x7fe4f4157ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8000c00 tx=0x7fe4e8004750 comp rx=0 tx=0).stop 2023-10-16T09:56:58.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 shutdown_connections 2023-10-16T09:56:58.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fe4d806b020 0x7fe4d806d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4f415eab0 0x7fe4f415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f40cb0c0 0x7fe4f4157ad0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 --2- 172.21.15.181:0/3795545704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4f415e6d0 0x7fe4f415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:58.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 >> 172.21.15.181:0/3795545704 conn(0x7fe4f40d04a0 msgr2=0x7fe4f40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:58.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 shutdown_connections 2023-10-16T09:56:58.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:58.878+0000 7fe4fae6e700 1 -- 172.21.15.181:0/3795545704 wait complete. 2023-10-16T09:56:58.918 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:56:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.036+0000 7f59f0cae700 1 Processor -- start 2023-10-16T09:56:59.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.036+0000 7f59f0cae700 1 -- start start 2023-10-16T09:56:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59f0cae700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59ec0ce7f0 0x7f59ec06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59f0cae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59ec0d2390 0x7f59ec0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59f0cae700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0cc470 con 0x7f59ec0d0180 2023-10-16T09:56:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59f0cae700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0d05f0 con 0x7f59ec0d2390 2023-10-16T09:56:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59f0cae700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec06ca30 con 0x7f59ec0ce7f0 2023-10-16T09:56:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59ec0d2390 0x7f59ec0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59ec0d2390 0x7f59ec0cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40094/0 (socket says 172.21.15.181:40094) 2023-10-16T09:56:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 -- 172.21.15.181:0/3386251619 learned_addr learned my addr 172.21.15.181:0/3386251619 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 -- 172.21.15.181:0/3386251619 >> v2:172.21.15.190:3300/0 conn(0x7f59ec0ce7f0 msgr2=0x7f59ec06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59ea59c700 1 --2- 172.21.15.181:0/3386251619 >> v2:172.21.15.190:3300/0 conn(0x7f59ec0ce7f0 0x7f59ec06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 --2- 172.21.15.181:0/3386251619 >> v2:172.21.15.190:3300/0 conn(0x7f59ec0ce7f0 0x7f59ec06c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 -- 172.21.15.181:0/3386251619 >> v1:172.21.15.190:6789/0 conn(0x7f59ec0d0180 legacy=0x7f59ec161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 -- 172.21.15.181:0/3386251619 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec06cd10 con 0x7f59ec0d2390 2023-10-16T09:56:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.037+0000 7f59e9d9b700 1 --2- 172.21.15.181:0/3386251619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d2390 0x7f59ec0cfc40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f59ec09af40 tx=0x7f59e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862f46b47adb22dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59e959a700 1 -- 172.21.15.181:0/3386251619 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004230 con 0x7f59ec0d2390 2023-10-16T09:56:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59e959a700 1 -- 172.21.15.181:0/3386251619 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e0004390 con 0x7f59ec0d2390 2023-10-16T09:56:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59e959a700 1 -- 172.21.15.181:0/3386251619 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e00036a0 con 0x7f59ec0d2390 2023-10-16T09:56:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 -- 172.21.15.181:0/3386251619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d2390 msgr2=0x7f59ec0cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 --2- 172.21.15.181:0/3386251619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d2390 0x7f59ec0cfc40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f59ec09af40 tx=0x7f59e0009bd0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 -- 172.21.15.181:0/3386251619 shutdown_connections 2023-10-16T09:56:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 --2- 172.21.15.181:0/3386251619 >> v2:172.21.15.190:3300/0 conn(0x7f59ec0ce7f0 0x7f59ec06c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 --2- 172.21.15.181:0/3386251619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d2390 0x7f59ec0cfc40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 -- 172.21.15.181:0/3386251619 >> 172.21.15.181:0/3386251619 conn(0x7f59ec09a9a0 msgr2=0x7f59ec0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 -- 172.21.15.181:0/3386251619 shutdown_connections 2023-10-16T09:56:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.038+0000 7f59f0cae700 1 -- 172.21.15.181:0/3386251619 wait complete. 2023-10-16T09:56:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.039+0000 7f59f0cae700 1 Processor -- start 2023-10-16T09:56:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59f0cae700 1 -- start start 2023-10-16T09:56:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59f0cae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59ec0ce7f0 0x7f59ec15edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59f0cae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d0180 0x7f59ec0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59f0cae700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59ec0d2390 0x7f59ec0b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59f0cae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0ae7d0 con 0x7f59ec0d0180 2023-10-16T09:56:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59f0cae700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec09b880 con 0x7f59ec0ce7f0 2023-10-16T09:56:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59f0cae700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec09bb60 con 0x7f59ec0d2390 2023-10-16T09:56:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59e9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d0180 0x7f59ec0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59e9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d0180 0x7f59ec0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40100/0 (socket says 172.21.15.181:40100) 2023-10-16T09:56:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.040+0000 7f59e9d9b700 1 -- 172.21.15.181:0/2831909546 learned_addr learned my addr 172.21.15.181:0/2831909546 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59ea59c700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59ec0ce7f0 0x7f59ec15edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59ead9d700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59ec0d2390 0x7f59ec0b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59e9d9b700 1 -- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59ec0d2390 msgr2=0x7f59ec0b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59e9d9b700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59ec0d2390 0x7f59ec0b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59e9d9b700 1 -- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59ec0ce7f0 msgr2=0x7f59ec15edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59e9d9b700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59ec0ce7f0 0x7f59ec15edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59e9d9b700 1 -- 172.21.15.181:0/2831909546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0009550 con 0x7f59ec0d0180 2023-10-16T09:56:59.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59e9d9b700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d0180 0x7f59ec0ae290 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0674d0 tx=0x7f59e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59daffd700 1 -- 172.21.15.181:0/2831909546 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0003c90 con 0x7f59ec0d0180 2023-10-16T09:56:59.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59daffd700 1 -- 172.21.15.181:0/2831909546 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e001f070 con 0x7f59ec0d0180 2023-10-16T09:56:59.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59daffd700 1 -- 172.21.15.181:0/2831909546 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0011870 con 0x7f59ec0d0180 2023-10-16T09:56:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec09be40 con 0x7f59ec0d0180 2023-10-16T09:56:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.041+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec09c400 con 0x7f59ec0d0180 2023-10-16T09:56:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.042+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec067d20 con 0x7f59ec0d0180 2023-10-16T09:56:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.042+0000 7f59daffd700 1 -- 172.21.15.181:0/2831909546 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f59e0021030 con 0x7f59ec0d0180 2023-10-16T09:56:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.043+0000 7f59daffd700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f59cc06b020 0x7f59cc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.043+0000 7f59daffd700 1 -- 172.21.15.181:0/2831909546 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f59e008aaf0 con 0x7f59ec0d0180 2023-10-16T09:56:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.043+0000 7f59ea59c700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f59cc06b020 0x7f59cc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.043+0000 7f59ea59c700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f59cc06b020 0x7f59cc06d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00aaa0 tx=0x7f59dc008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.047+0000 7f59daffd700 1 -- 172.21.15.181:0/2831909546 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f59e0055e00 con 0x7f59ec0d0180 2023-10-16T09:56:59.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.239+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59ec0cef40 con 0x7f59ec0d0180 2023-10-16T09:56:59.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.240+0000 7f59daffd700 1 -- 172.21.15.181:0/2831909546 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+13493 (secure 0 0 0) 0x7f59e0029090 con 0x7f59ec0d0180 2023-10-16T09:56:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:59.242 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":55,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:56:58.463418+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:56:59.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f59cc06b020 msgr2=0x7f59cc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f59cc06b020 0x7f59cc06d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00aaa0 tx=0x7f59dc008040 comp rx=0 tx=0).stop 2023-10-16T09:56:59.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d0180 msgr2=0x7f59ec0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d0180 0x7f59ec0ae290 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0674d0 tx=0x7f59e0004750 comp rx=0 tx=0).stop 2023-10-16T09:56:59.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 shutdown_connections 2023-10-16T09:56:59.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f59cc06b020 0x7f59cc06d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59ec0d2390 0x7f59ec0b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59ec0ce7f0 0x7f59ec15edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 --2- 172.21.15.181:0/2831909546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59ec0d0180 0x7f59ec0ae290 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 >> 172.21.15.181:0/2831909546 conn(0x7f59ec09a9a0 msgr2=0x7f59ec06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:59.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 shutdown_connections 2023-10-16T09:56:59.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.242+0000 7f59f0cae700 1 -- 172.21.15.181:0/2831909546 wait complete. 2023-10-16T09:56:59.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:56:59.257 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:56:59.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.378+0000 7fd0e7dbd700 1 Processor -- start 2023-10-16T09:56:59.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.379+0000 7fd0e7dbd700 1 -- start start 2023-10-16T09:56:59.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.379+0000 7fd0e7dbd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00cc6e0 0x7fd0e0071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.379+0000 7fd0e7dbd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0e00ce680 0x7fd0e00ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.379+0000 7fd0e7dbd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e00ccb60 con 0x7fd0e00b2010 2023-10-16T09:56:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.379+0000 7fd0e7dbd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e0072120 con 0x7fd0e00ce680 2023-10-16T09:56:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.379+0000 7fd0e7dbd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00ce1f0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00cc6e0 0x7fd0e0071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e635a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0e00ce680 0x7fd0e00ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00cc6e0 0x7fd0e0071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:37134/0 (socket says 172.21.15.181:37134) 2023-10-16T09:56:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e635a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0e00ce680 0x7fd0e00ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:45160/0 (socket says 172.21.15.181:45160) 2023-10-16T09:56:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 -- 172.21.15.181:0/1943414614 learned_addr learned my addr 172.21.15.181:0/1943414614 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:59.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 -- 172.21.15.181:0/1943414614 >> v2:172.21.15.190:3300/0 conn(0x7fd0e00ce680 msgr2=0x7fd0e00ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:59.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 --2- 172.21.15.181:0/1943414614 >> v2:172.21.15.190:3300/0 conn(0x7fd0e00ce680 0x7fd0e00ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 -- 172.21.15.181:0/1943414614 >> v1:172.21.15.190:6789/0 conn(0x7fd0e00b2010 legacy=0x7fd0e00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 -- 172.21.15.181:0/1943414614 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e00ccda0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e5358700 1 --2- 172.21.15.181:0/1943414614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00cc6e0 0x7fd0e0071be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098f0 tx=0x7fd0dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc198c5032333307 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e4b57700 1 -- 172.21.15.181:0/1943414614 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc011830 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e4b57700 1 -- 172.21.15.181:0/1943414614 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc011990 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.380+0000 7fd0e4b57700 1 -- 172.21.15.181:0/1943414614 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc011d20 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/1943414614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00cc6e0 msgr2=0x7fd0e0071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/1943414614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00cc6e0 0x7fd0e0071be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098f0 tx=0x7fd0dc009c00 comp rx=0 tx=0).stop 2023-10-16T09:56:59.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/1943414614 shutdown_connections 2023-10-16T09:56:59.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/1943414614 >> v2:172.21.15.190:3300/0 conn(0x7fd0e00ce680 0x7fd0e00ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/1943414614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00cc6e0 0x7fd0e0071be0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/1943414614 >> 172.21.15.181:0/1943414614 conn(0x7fd0e00cd010 msgr2=0x7fd0e00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:59.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/1943414614 shutdown_connections 2023-10-16T09:56:59.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.381+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/1943414614 wait complete. 2023-10-16T09:56:59.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.382+0000 7fd0e7dbd700 1 Processor -- start 2023-10-16T09:56:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.382+0000 7fd0e7dbd700 1 -- start start 2023-10-16T09:56:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e7dbd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0e00b2010 0x7fd0e014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e7dbd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e00cc6e0 0x7fd0e00c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e7dbd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00ce680 0x7fd0e00c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e7dbd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e014bfb0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e7dbd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00c8c20 con 0x7fd0e00b2010 2023-10-16T09:56:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e7dbd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e00c8f00 con 0x7fd0e00ce680 2023-10-16T09:56:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e00cc6e0 0x7fd0e00c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e00cc6e0 0x7fd0e00c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40114/0 (socket says 172.21.15.181:40114) 2023-10-16T09:56:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 -- 172.21.15.181:0/2273745998 learned_addr learned my addr 172.21.15.181:0/2273745998 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5b59700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0e00b2010 0x7fd0e014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e635a700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00ce680 0x7fd0e00c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 -- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00ce680 msgr2=0x7fd0e00c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00ce680 0x7fd0e00c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 -- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0e00b2010 msgr2=0x7fd0e014ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0e00b2010 0x7fd0e014ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.383+0000 7fd0e5358700 1 -- 172.21.15.181:0/2273745998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009550 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.384+0000 7fd0e5358700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e00cc6e0 0x7fd0e00c81d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc014040 tx=0x7fd0dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.384+0000 7fd0d67fc700 1 -- 172.21.15.181:0/2273745998 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc004510 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.384+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e00c91e0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.384+0000 7fd0d67fc700 1 -- 172.21.15.181:0/2273745998 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc01e070 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.384+0000 7fd0d67fc700 1 -- 172.21.15.181:0/2273745998 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0038e0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.384+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e00c97a0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.385+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e00b6510 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.385+0000 7fd0d67fc700 1 -- 172.21.15.181:0/2273745998 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd0dc003a40 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.385+0000 7fd0d67fc700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd0c406b020 0x7fd0c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.386+0000 7fd0d67fc700 1 -- 172.21.15.181:0/2273745998 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd0dc08a450 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.386+0000 7fd0e5b59700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd0c406b020 0x7fd0c406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.386+0000 7fd0e5b59700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd0c406b020 0x7fd0c406d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0007fd0 tx=0x7fd0d0009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.390+0000 7fd0d67fc700 1 -- 172.21.15.181:0/2273745998 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd0dc0555c0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.590+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0e00d0ac0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.591+0000 7fd0d67fc700 1 -- 172.21.15.181:0/2273745998 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0dc058be0 con 0x7fd0e00cc6e0 2023-10-16T09:56:59.592 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:56:59.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd0c406b020 msgr2=0x7fd0c406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd0c406b020 0x7fd0c406d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0007fd0 tx=0x7fd0d0009040 comp rx=0 tx=0).stop 2023-10-16T09:56:59.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e00cc6e0 msgr2=0x7fd0e00c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e00cc6e0 0x7fd0e00c81d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc014040 tx=0x7fd0dc004750 comp rx=0 tx=0).stop 2023-10-16T09:56:59.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 shutdown_connections 2023-10-16T09:56:59.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fd0c406b020 0x7fd0c406d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0e00ce680 0x7fd0e00c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0e00b2010 0x7fd0e014ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 --2- 172.21.15.181:0/2273745998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e00cc6e0 0x7fd0e00c81d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 >> 172.21.15.181:0/2273745998 conn(0x7fd0e00cd010 msgr2=0x7fd0e00b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:59.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 shutdown_connections 2023-10-16T09:56:59.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.593+0000 7fd0e7dbd700 1 -- 172.21.15.181:0/2273745998 wait complete. 2023-10-16T09:56:59.607 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T09:56:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.727+0000 7f0bbb86a700 1 Processor -- start 2023-10-16T09:56:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.727+0000 7f0bbb86a700 1 -- start start 2023-10-16T09:56:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0bbb86a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0bb40cb0c0 0x7f0bb40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0bbb86a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb406ed00 con 0x7f0bb415e6d0 2023-10-16T09:56:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0bbb86a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb40d02d0 con 0x7f0bb415eab0 2023-10-16T09:56:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0bbb86a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb406e0f0 con 0x7f0bb40cb0c0 2023-10-16T09:56:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0bb9606700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0bb415e6d0 0x7f0bb415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51200/0 (socket says 172.21.15.181:51200) 2023-10-16T09:56:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0bb9606700 1 -- 172.21.15.181:0/1868183206 learned_addr learned my addr 172.21.15.181:0/1868183206 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:56:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0bb9e07700 1 --2- 172.21.15.181:0/1868183206 >> v2:172.21.15.190:3300/0 conn(0x7f0bb40cb0c0 0x7f0bb40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446639960 0 0) 0x7f0bb406ed00 con 0x7f0bb415e6d0 2023-10-16T09:56:59.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.728+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba00035b0 con 0x7f0bb415e6d0 2023-10-16T09:56:59.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3699094666 0 0) 0x7f0ba00035b0 con 0x7f0bb415e6d0 2023-10-16T09:56:59.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 >> v2:172.21.15.190:3300/0 conn(0x7f0bb40cb0c0 msgr2=0x7f0bb40d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:59.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 --2- 172.21.15.181:0/1868183206 >> v2:172.21.15.190:3300/0 conn(0x7f0bb40cb0c0 0x7f0bb40d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 >> v1:172.21.15.190:6789/0 conn(0x7f0bb415eab0 legacy=0x7f0bb415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb406e3d0 con 0x7f0bb415e6d0 2023-10-16T09:56:59.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f0ba40017f0 con 0x7f0bb415e6d0 2023-10-16T09:56:59.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 >> v1:172.21.15.181:6789/0 conn(0x7f0bb415e6d0 legacy=0x7f0bb415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ba0003a00 0x7f0ba0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0004850 0x7f0ba0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba0005210 0x7f0ba000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb406e0f0 con 0x7f0ba0003a00 2023-10-16T09:56:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba000c0b0 con 0x7f0ba0004850 2023-10-16T09:56:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba000c270 con 0x7f0ba0005210 2023-10-16T09:56:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9606700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ba0003a00 0x7f0ba0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb8e05700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0004850 0x7f0ba0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9e07700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba0005210 0x7f0ba000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9606700 1 -- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba0005210 msgr2=0x7f0ba000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9606700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba0005210 0x7f0ba000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9606700 1 -- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0004850 msgr2=0x7f0ba0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:56:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9606700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0004850 0x7f0ba0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9606700 1 -- 172.21.15.181:0/1868183206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00035b0 con 0x7f0ba0003a00 2023-10-16T09:56:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9e07700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ba0005210 0x7f0ba000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:56:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9e07700 1 -- 172.21.15.181:0/1868183206 reap_dead start 2023-10-16T09:56:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0bb9606700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ba0003a00 0x7f0ba0003e20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400c240 tx=0x7f0ba0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aef943a821c7e58 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.729+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400cde0 con 0x7f0ba0003a00 2023-10-16T09:56:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4007b80 con 0x7f0ba0003a00 2023-10-16T09:56:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0babfff700 1 -- 172.21.15.181:0/1868183206 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40072d0 con 0x7f0ba0003a00 2023-10-16T09:56:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0bbb86a700 1 -- 172.21.15.181:0/1868183206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ba0003a00 msgr2=0x7f0ba0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ba0003a00 0x7f0ba0003e20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400c240 tx=0x7f0ba0005880 comp rx=0 tx=0).stop 2023-10-16T09:56:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0bbb86a700 1 -- 172.21.15.181:0/1868183206 shutdown_connections 2023-10-16T09:56:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/1868183206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ba0003a00 0x7f0ba0003e20 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0bbb86a700 1 -- 172.21.15.181:0/1868183206 >> 172.21.15.181:0/1868183206 conn(0x7f0bb40d04a0 msgr2=0x7f0bb40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0bbb86a700 1 -- 172.21.15.181:0/1868183206 shutdown_connections 2023-10-16T09:56:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.730+0000 7f0bbb86a700 1 -- 172.21.15.181:0/1868183206 wait complete. 2023-10-16T09:56:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.731+0000 7f0bbb86a700 1 Processor -- start 2023-10-16T09:56:59.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.731+0000 7f0bbb86a700 1 -- start start 2023-10-16T09:56:59.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bbb86a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0003a00 0x7f0bb41e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bbb86a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb41f4170 0x7f0bb41f0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bbb86a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bb41f05b0 0x7f0bb41e6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bbb86a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41f0a40 con 0x7f0bb41f05b0 2023-10-16T09:56:59.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bbb86a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41f66d0 con 0x7f0ba0003a00 2023-10-16T09:56:59.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bbb86a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41f69b0 con 0x7f0bb41f4170 2023-10-16T09:56:59.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bb9e07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bb41f05b0 0x7f0bb41e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bb9e07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bb41f05b0 0x7f0bb41e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40138/0 (socket says 172.21.15.181:40138) 2023-10-16T09:56:59.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bb9e07700 1 -- 172.21.15.181:0/3769581559 learned_addr learned my addr 172.21.15.181:0/3769581559 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:56:59.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bb8e05700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb41f4170 0x7f0bb41f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.732+0000 7f0bb9606700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0003a00 0x7f0bb41e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bb9e07700 1 -- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb41f4170 msgr2=0x7f0bb41f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bb9e07700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb41f4170 0x7f0bb41f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bb9e07700 1 -- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0003a00 msgr2=0x7f0bb41e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bb9e07700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0003a00 0x7f0bb41e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bb9e07700 1 -- 172.21.15.181:0/3769581559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb41f6c90 con 0x7f0bb41f05b0 2023-10-16T09:56:59.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bb9e07700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bb41f05b0 0x7f0bb41e6050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4071b20 tx=0x7f0bac0101c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0ba9ffb700 1 -- 172.21.15.181:0/3769581559 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00e050 con 0x7f0bb41f05b0 2023-10-16T09:56:59.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0ba9ffb700 1 -- 172.21.15.181:0/3769581559 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac00f040 con 0x7f0bb41f05b0 2023-10-16T09:56:59.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0ba9ffb700 1 -- 172.21.15.181:0/3769581559 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac003c40 con 0x7f0bb41f05b0 2023-10-16T09:56:59.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb41f6f70 con 0x7f0bb41f05b0 2023-10-16T09:56:59.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.733+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb41e4450 con 0x7f0bb41f05b0 2023-10-16T09:56:59.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.734+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb4067d20 con 0x7f0bb41f05b0 2023-10-16T09:56:59.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.735+0000 7f0ba9ffb700 1 -- 172.21.15.181:0/3769581559 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0bac010a60 con 0x7f0bb41f05b0 2023-10-16T09:56:59.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.735+0000 7f0ba9ffb700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f0b9806b020 0x7f0b9806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:56:59.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.735+0000 7f0ba9ffb700 1 -- 172.21.15.181:0/3769581559 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0bac089830 con 0x7f0bb41f05b0 2023-10-16T09:56:59.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.735+0000 7f0bb9606700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f0b9806b020 0x7f0b9806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:56:59.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.735+0000 7f0bb9606700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f0b9806b020 0x7f0b9806d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4000ef0 tx=0x7f0ba4007000 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:56:59.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.739+0000 7f0ba9ffb700 1 -- 172.21.15.181:0/3769581559 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0bac054910 con 0x7f0bb41f05b0 2023-10-16T09:56:59.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.935+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0bb40cb610 con 0x7f0bb41f05b0 2023-10-16T09:56:59.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.936+0000 7f0ba9ffb700 1 -- 172.21.15.181:0/3769581559 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0bac01a060 con 0x7f0bb41f05b0 2023-10-16T09:56:59.938 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:56:59.938 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:56:59.939 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:56:59.939 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T09:56:59.939 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T09:56:59.939 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:56:59.940 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:56:59.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f0b9806b020 msgr2=0x7f0b9806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f0b9806b020 0x7f0b9806d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4000ef0 tx=0x7f0ba4007000 comp rx=0 tx=0).stop 2023-10-16T09:56:59.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bb41f05b0 msgr2=0x7f0bb41e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:56:59.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bb41f05b0 0x7f0bb41e6050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4071b20 tx=0x7f0bac0101c0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 shutdown_connections 2023-10-16T09:56:59.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f0b9806b020 0x7f0b9806d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bb41f05b0 0x7f0bb41e6050 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ba0003a00 0x7f0bb41e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 --2- 172.21.15.181:0/3769581559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb41f4170 0x7f0bb41f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:56:59.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 >> 172.21.15.181:0/3769581559 conn(0x7f0bb40d04a0 msgr2=0x7f0bb40ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:56:59.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 shutdown_connections 2023-10-16T09:56:59.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:56:59.938+0000 7f0bbb86a700 1 -- 172.21.15.181:0/3769581559 wait complete. 2023-10-16T09:56:59.951 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T09:56:59.952 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T09:57:00.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.074+0000 7f7f5d07e700 1 Processor -- start 2023-10-16T09:57:00.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.074+0000 7f7f5d07e700 1 -- start start 2023-10-16T09:57:00.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5d07e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580ce3a0 0x7f7f580cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5d07e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f580cd3f0 0x7f7f580cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5d07e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f580cd850 con 0x7f7f580afa10 2023-10-16T09:57:00.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5d07e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f580ce7e0 con 0x7f7f580cd3f0 2023-10-16T09:57:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5d07e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5815b540 con 0x7f7f580ce3a0 2023-10-16T09:57:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f580cd3f0 0x7f7f580cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f580cd3f0 0x7f7f580cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40146/0 (socket says 172.21.15.181:40146) 2023-10-16T09:57:00.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.075+0000 7f7f5759e700 1 -- 172.21.15.181:0/2059830392 learned_addr learned my addr 172.21.15.181:0/2059830392 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5659c700 1 --2- 172.21.15.181:0/2059830392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580ce3a0 0x7f7f580cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5759e700 1 -- 172.21.15.181:0/2059830392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580ce3a0 msgr2=0x7f7f580cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5759e700 1 --2- 172.21.15.181:0/2059830392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580ce3a0 0x7f7f580cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5759e700 1 -- 172.21.15.181:0/2059830392 >> v1:172.21.15.181:6789/0 conn(0x7f7f580afa10 legacy=0x7f7f5806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5759e700 1 -- 172.21.15.181:0/2059830392 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5815b820 con 0x7f7f580cd3f0 2023-10-16T09:57:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f55d9b700 1 -- 172.21.15.181:0/2059830392 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848544206 0 0) 0x7f7f580cd850 con 0x7f7f580afa10 2023-10-16T09:57:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5759e700 1 --2- 172.21.15.181:0/2059830392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580cd3f0 0x7f7f580cb8b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009e50 tx=0x7f7f5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27763e6bea1ffaff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f55d9b700 1 -- 172.21.15.181:0/2059830392 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000e070 con 0x7f7f580cd3f0 2023-10-16T09:57:00.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f55d9b700 1 -- 172.21.15.181:0/2059830392 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5000b9c0 con 0x7f7f580cd3f0 2023-10-16T09:57:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f55d9b700 1 -- 172.21.15.181:0/2059830392 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000bd70 con 0x7f7f580cd3f0 2023-10-16T09:57:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5d07e700 1 -- 172.21.15.181:0/2059830392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580cd3f0 msgr2=0x7f7f580cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/2059830392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580cd3f0 0x7f7f580cb8b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009e50 tx=0x7f7f5000b330 comp rx=0 tx=0).stop 2023-10-16T09:57:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5d07e700 1 -- 172.21.15.181:0/2059830392 shutdown_connections 2023-10-16T09:57:00.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/2059830392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580cd3f0 0x7f7f580cb8b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/2059830392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580ce3a0 0x7f7f580cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.076+0000 7f7f5d07e700 1 -- 172.21.15.181:0/2059830392 >> 172.21.15.181:0/2059830392 conn(0x7f7f580d04a0 msgr2=0x7f7f580d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.077+0000 7f7f5d07e700 1 -- 172.21.15.181:0/2059830392 shutdown_connections 2023-10-16T09:57:00.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.077+0000 7f7f5d07e700 1 -- 172.21.15.181:0/2059830392 wait complete. 2023-10-16T09:57:00.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.078+0000 7f7f5d07e700 1 Processor -- start 2023-10-16T09:57:00.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.078+0000 7f7f5d07e700 1 -- start start 2023-10-16T09:57:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5d07e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580afa10 0x7f7f58147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5d07e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580cd3f0 0x7f7f5815d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5d07e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f580ce3a0 0x7f7f5815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5d07e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f58147cc0 con 0x7f7f580afa10 2023-10-16T09:57:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5d07e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f58148630 con 0x7f7f580ce3a0 2023-10-16T09:57:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5d07e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f580c6cc0 con 0x7f7f580cd3f0 2023-10-16T09:57:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580afa10 0x7f7f58147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580afa10 0x7f7f58147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40148/0 (socket says 172.21.15.181:40148) 2023-10-16T09:57:00.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 -- 172.21.15.181:0/4189092819 learned_addr learned my addr 172.21.15.181:0/4189092819 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:00.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 -- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580cd3f0 msgr2=0x7f7f5815d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580cd3f0 0x7f7f5815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 -- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f580ce3a0 msgr2=0x7f7f5815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5759e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f580ce3a0 0x7f7f5815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f580ce3a0 0x7f7f5815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 -- 172.21.15.181:0/4189092819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f50009b30 con 0x7f7f580afa10 2023-10-16T09:57:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5759e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f580ce3a0 0x7f7f5815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f56d9d700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580afa10 0x7f7f58147780 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c5a0 tx=0x7f7f4c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:00.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f3f7fe700 1 -- 172.21.15.181:0/4189092819 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0054e0 con 0x7f7f580afa10 2023-10-16T09:57:00.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f3f7fe700 1 -- 172.21.15.181:0/4189092819 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c003680 con 0x7f7f580afa10 2023-10-16T09:57:00.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f580c69e0 con 0x7f7f580afa10 2023-10-16T09:57:00.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.079+0000 7f7f3f7fe700 1 -- 172.21.15.181:0/4189092819 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c003a30 con 0x7f7f580afa10 2023-10-16T09:57:00.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.080+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f581482c0 con 0x7f7f580afa10 2023-10-16T09:57:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.081+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f580ccd20 con 0x7f7f580afa10 2023-10-16T09:57:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.081+0000 7f7f3f7fe700 1 -- 172.21.15.181:0/4189092819 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7f4c003b90 con 0x7f7f580afa10 2023-10-16T09:57:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.081+0000 7f7f3f7fe700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f7f4406b020 0x7f7f4406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.081+0000 7f7f5659c700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f7f4406b020 0x7f7f4406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.081+0000 7f7f3f7fe700 1 -- 172.21.15.181:0/4189092819 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7f4c088ba0 con 0x7f7f580afa10 2023-10-16T09:57:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.082+0000 7f7f5659c700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f7f4406b020 0x7f7f4406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f58157fc0 tx=0x7f7f48008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:00.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.086+0000 7f7f3f7fe700 1 -- 172.21.15.181:0/4189092819 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7f4c053c60 con 0x7f7f580afa10 2023-10-16T09:57:00.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.278+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7f58148450 con 0x7f7f580afa10 2023-10-16T09:57:00.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.466+0000 7f7f3f7fe700 1 -- 172.21.15.181:0/4189092819 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7f4c017020 con 0x7f7f580afa10 2023-10-16T09:57:00.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f7f4406b020 msgr2=0x7f7f4406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f7f4406b020 0x7f7f4406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f58157fc0 tx=0x7f7f48008040 comp rx=0 tx=0).stop 2023-10-16T09:57:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580afa10 msgr2=0x7f7f58147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580afa10 0x7f7f58147780 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c5a0 tx=0x7f7f4c00c960 comp rx=0 tx=0).stop 2023-10-16T09:57:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 shutdown_connections 2023-10-16T09:57:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f7f4406b020 0x7f7f4406d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f580ce3a0 0x7f7f5815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f580afa10 0x7f7f58147780 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 --2- 172.21.15.181:0/4189092819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f580cd3f0 0x7f7f5815d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 >> 172.21.15.181:0/4189092819 conn(0x7f7f580d04a0 msgr2=0x7f7f580b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 shutdown_connections 2023-10-16T09:57:00.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.468+0000 7f7f5d07e700 1 -- 172.21.15.181:0/4189092819 wait complete. 2023-10-16T09:57:00.473 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T09:57:00.486 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T09:57:00.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.612+0000 7f04ad418700 1 Processor -- start 2023-10-16T09:57:00.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.612+0000 7f04ad418700 1 -- start start 2023-10-16T09:57:00.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.612+0000 7f04ad418700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04a815dcd0 0x7f04a80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04ad418700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a80cec70 con 0x7f04a8162450 2023-10-16T09:57:00.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04ad418700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a80cef50 con 0x7f04a806a8a0 2023-10-16T09:57:00.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04ad418700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a80cf230 con 0x7f04a815dcd0 2023-10-16T09:57:00.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04a815dcd0 0x7f04a80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a67fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f04a8162450 0x7f04a80caf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51236/0 (socket says 172.21.15.181:51236) 2023-10-16T09:57:00.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04a815dcd0 0x7f04a80ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40156/0 (socket says 172.21.15.181:40156) 2023-10-16T09:57:00.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a67fc700 1 -- 172.21.15.181:0/1859796746 learned_addr learned my addr 172.21.15.181:0/1859796746 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:00.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a6ffd700 1 -- 172.21.15.181:0/1859796746 >> v1:172.21.15.190:6789/0 conn(0x7f04a806a8a0 legacy=0x7f04a806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:00.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a6ffd700 1 -- 172.21.15.181:0/1859796746 >> v1:172.21.15.181:6789/0 conn(0x7f04a8162450 legacy=0x7f04a80caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a5ffb700 1 -- 172.21.15.181:0/1859796746 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825637186 0 0) 0x7f04a80cec70 con 0x7f04a8162450 2023-10-16T09:57:00.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a6ffd700 1 -- 172.21.15.181:0/1859796746 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8071ca0 con 0x7f04a815dcd0 2023-10-16T09:57:00.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a6ffd700 1 --2- 172.21.15.181:0/1859796746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815dcd0 0x7f04a80ce3b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f049c009840 tx=0x7f049c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2bb5f18cbe3000 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:00.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a5ffb700 1 -- 172.21.15.181:0/1859796746 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c011830 con 0x7f04a815dcd0 2023-10-16T09:57:00.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a5ffb700 1 -- 172.21.15.181:0/1859796746 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c011990 con 0x7f04a815dcd0 2023-10-16T09:57:00.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.613+0000 7f04a5ffb700 1 -- 172.21.15.181:0/1859796746 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c011d20 con 0x7f04a815dcd0 2023-10-16T09:57:00.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.614+0000 7f04ad418700 1 -- 172.21.15.181:0/1859796746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815dcd0 msgr2=0x7f04a80ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.614+0000 7f04ad418700 1 --2- 172.21.15.181:0/1859796746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815dcd0 0x7f04a80ce3b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f049c009840 tx=0x7f049c009b50 comp rx=0 tx=0).stop 2023-10-16T09:57:00.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.614+0000 7f04ad418700 1 -- 172.21.15.181:0/1859796746 shutdown_connections 2023-10-16T09:57:00.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.614+0000 7f04ad418700 1 --2- 172.21.15.181:0/1859796746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815dcd0 0x7f04a80ce3b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.614+0000 7f04ad418700 1 -- 172.21.15.181:0/1859796746 >> 172.21.15.181:0/1859796746 conn(0x7f04a80d05d0 msgr2=0x7f04a80d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:00.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.614+0000 7f04ad418700 1 -- 172.21.15.181:0/1859796746 shutdown_connections 2023-10-16T09:57:00.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.614+0000 7f04ad418700 1 -- 172.21.15.181:0/1859796746 wait complete. 2023-10-16T09:57:00.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.615+0000 7f04ad418700 1 Processor -- start 2023-10-16T09:57:00.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.615+0000 7f04ad418700 1 -- start start 2023-10-16T09:57:00.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04ad418700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a806a8a0 0x7f04a80c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04ad418700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a815dcd0 0x7f04a80c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04ad418700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8162450 0x7f04a814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04ad418700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c7460 con 0x7f04a806a8a0 2023-10-16T09:57:00.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04ad418700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c6bc0 con 0x7f04a8162450 2023-10-16T09:57:00.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04ad418700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c6ea0 con 0x7f04a815dcd0 2023-10-16T09:57:00.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a806a8a0 0x7f04a80c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a806a8a0 0x7f04a80c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40162/0 (socket says 172.21.15.181:40162) 2023-10-16T09:57:00.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 -- 172.21.15.181:0/3833668742 learned_addr learned my addr 172.21.15.181:0/3833668742 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:00.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a77fe700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8162450 0x7f04a814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 -- 172.21.15.181:0/3833668742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a815dcd0 msgr2=0x7f04a80c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a815dcd0 0x7f04a80c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 -- 172.21.15.181:0/3833668742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8162450 msgr2=0x7f04a814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:00.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8162450 0x7f04a814aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:00.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.616+0000 7f04a6ffd700 1 -- 172.21.15.181:0/3833668742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c0094a0 con 0x7f04a806a8a0 2023-10-16T09:57:00.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.617+0000 7f04a6ffd700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a806a8a0 0x7f04a80c4860 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f049c0090b0 tx=0x7f049c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:00.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.617+0000 7f048f7fe700 1 -- 172.21.15.181:0/3833668742 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c01a980 con 0x7f04a806a8a0 2023-10-16T09:57:00.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.617+0000 7f048f7fe700 1 -- 172.21.15.181:0/3833668742 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c01aae0 con 0x7f04a806a8a0 2023-10-16T09:57:00.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.617+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a80c7180 con 0x7f04a806a8a0 2023-10-16T09:57:00.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.617+0000 7f048f7fe700 1 -- 172.21.15.181:0/3833668742 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c01ae90 con 0x7f04a806a8a0 2023-10-16T09:57:00.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.617+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a80c41a0 con 0x7f04a806a8a0 2023-10-16T09:57:00.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.618+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a80c4330 con 0x7f04a806a8a0 2023-10-16T09:57:00.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.618+0000 7f048f7fe700 1 -- 172.21.15.181:0/3833668742 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f049c022070 con 0x7f04a806a8a0 2023-10-16T09:57:00.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.619+0000 7f048f7fe700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f049406b020 0x7f049406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:00.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.619+0000 7f048f7fe700 1 -- 172.21.15.181:0/3833668742 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5623+0+0 (secure 0 0 0) 0x7f049c08aa90 con 0x7f04a806a8a0 2023-10-16T09:57:00.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.619+0000 7f04a67fc700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f049406b020 0x7f049406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:00.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.619+0000 7f04a67fc700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f049406b020 0x7f049406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0498000c00 tx=0x7f0498007a50 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:00.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.623+0000 7f048f7fe700 1 -- 172.21.15.181:0/3833668742 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f049c055c30 con 0x7f04a806a8a0 2023-10-16T09:57:00.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:00.819+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f04a80adbb0 con 0x7f04a806a8a0 2023-10-16T09:57:01.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.474+0000 7f048f7fe700 1 -- 172.21.15.181:0/3833668742 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7f049c029020 con 0x7f04a806a8a0 2023-10-16T09:57:01.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f049406b020 msgr2=0x7f049406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:01.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f049406b020 0x7f049406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0498000c00 tx=0x7f0498007a50 comp rx=0 tx=0).stop 2023-10-16T09:57:01.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a806a8a0 msgr2=0x7f04a80c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:01.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a806a8a0 0x7f04a80c4860 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f049c0090b0 tx=0x7f049c004750 comp rx=0 tx=0).stop 2023-10-16T09:57:01.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 shutdown_connections 2023-10-16T09:57:01.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f049406b020 0x7f049406d4e0 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0498000c00 tx=0x7f0498007a50 comp rx=0 tx=0).stop 2023-10-16T09:57:01.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8162450 0x7f04a814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a806a8a0 0x7f04a80c4860 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 --2- 172.21.15.181:0/3833668742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a815dcd0 0x7f04a80c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.479+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 >> 172.21.15.181:0/3833668742 conn(0x7f04a80d05d0 msgr2=0x7f04a8160330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:01.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.480+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 shutdown_connections 2023-10-16T09:57:01.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.480+0000 7f04ad418700 1 -- 172.21.15.181:0/3833668742 wait complete. 2023-10-16T09:57:01.484 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T09:57:01.506 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T09:57:01.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.633+0000 7f36270e2700 1 Processor -- start 2023-10-16T09:57:01.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.633+0000 7f36270e2700 1 -- start start 2023-10-16T09:57:01.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f36270e2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3620072760 0x7f362006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f36270e2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f362006ade0 0x7f36201615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f36270e2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362006b210 con 0x7f362015da70 2023-10-16T09:57:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f36270e2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36200cef00 con 0x7f362006ade0 2023-10-16T09:57:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f36270e2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36200cec20 con 0x7f3620072760 2023-10-16T09:57:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f362567f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f362006ade0 0x7f36201615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f362567f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f362006ade0 0x7f36201615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40174/0 (socket says 172.21.15.181:40174) 2023-10-16T09:57:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f362567f700 1 -- 172.21.15.181:0/4070804540 learned_addr learned my addr 172.21.15.181:0/4070804540 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f361ffff700 1 --2- 172.21.15.181:0/4070804540 >> v2:172.21.15.190:3300/0 conn(0x7f3620072760 0x7f362006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f362567f700 1 -- 172.21.15.181:0/4070804540 >> v2:172.21.15.190:3300/0 conn(0x7f3620072760 msgr2=0x7f362006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.634+0000 7f362567f700 1 --2- 172.21.15.181:0/4070804540 >> v2:172.21.15.190:3300/0 conn(0x7f3620072760 0x7f362006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f362567f700 1 -- 172.21.15.181:0/4070804540 >> v1:172.21.15.190:6789/0 conn(0x7f362015da70 legacy=0x7f36200ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f362567f700 1 -- 172.21.15.181:0/4070804540 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620071c80 con 0x7f362006ade0 2023-10-16T09:57:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f361ffff700 1 --2- 172.21.15.181:0/4070804540 >> v2:172.21.15.190:3300/0 conn(0x7f3620072760 0x7f362006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f362567f700 1 --2- 172.21.15.181:0/4070804540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f362006ade0 0x7f36201615f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f362006ef60 tx=0x7f361800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20c0c4281c54cba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:01.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f361f7fe700 1 -- 172.21.15.181:0/4070804540 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800e070 con 0x7f362006ade0 2023-10-16T09:57:01.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f361f7fe700 1 -- 172.21.15.181:0/4070804540 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618003a20 con 0x7f362006ade0 2023-10-16T09:57:01.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f361f7fe700 1 -- 172.21.15.181:0/4070804540 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618003dd0 con 0x7f362006ade0 2023-10-16T09:57:01.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 -- 172.21.15.181:0/4070804540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f362006ade0 msgr2=0x7f36201615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:01.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 --2- 172.21.15.181:0/4070804540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f362006ade0 0x7f36201615f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f362006ef60 tx=0x7f361800b330 comp rx=0 tx=0).stop 2023-10-16T09:57:01.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 -- 172.21.15.181:0/4070804540 shutdown_connections 2023-10-16T09:57:01.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 --2- 172.21.15.181:0/4070804540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f362006ade0 0x7f36201615f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 --2- 172.21.15.181:0/4070804540 >> v2:172.21.15.190:3300/0 conn(0x7f3620072760 0x7f362006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 -- 172.21.15.181:0/4070804540 >> 172.21.15.181:0/4070804540 conn(0x7f36200cd010 msgr2=0x7f36200cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:01.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 -- 172.21.15.181:0/4070804540 shutdown_connections 2023-10-16T09:57:01.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.635+0000 7f36270e2700 1 -- 172.21.15.181:0/4070804540 wait complete. 2023-10-16T09:57:01.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 Processor -- start 2023-10-16T09:57:01.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 -- start start 2023-10-16T09:57:01.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f362006ade0 0x7f3620157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:01.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3620072760 0x7f36200ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:01.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f362015da70 0x7f36200adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:01.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200c6a40 con 0x7f3620072760 2023-10-16T09:57:01.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200c5ec0 con 0x7f362015da70 2023-10-16T09:57:01.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.637+0000 7f36270e2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200c61a0 con 0x7f362006ade0 2023-10-16T09:57:01.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3620072760 0x7f36200ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:01.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3620072760 0x7f36200ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40178/0 (socket says 172.21.15.181:40178) 2023-10-16T09:57:01.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 -- 172.21.15.181:0/1481006441 learned_addr learned my addr 172.21.15.181:0/1481006441 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:01.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f3624e7e700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f362006ade0 0x7f3620157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:01.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f362567f700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f362015da70 0x7f36200adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:01.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 -- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f362006ade0 msgr2=0x7f3620157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:01.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f362006ade0 0x7f3620157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 -- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f362015da70 msgr2=0x7f36200adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:01.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f362015da70 0x7f36200adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:01.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 -- 172.21.15.181:0/1481006441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618009b30 con 0x7f3620072760 2023-10-16T09:57:01.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f3624e7e700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f362006ade0 0x7f3620157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:01.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f362567f700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f362015da70 0x7f36200adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:01.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361ffff700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3620072760 0x7f36200ada90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c750 tx=0x7f360c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:01.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361d7fa700 1 -- 172.21.15.181:0/1481006441 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c011800 con 0x7f3620072760 2023-10-16T09:57:01.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36200c6480 con 0x7f3620072760 2023-10-16T09:57:01.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361d7fa700 1 -- 172.21.15.181:0/1481006441 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c011960 con 0x7f3620072760 2023-10-16T09:57:01.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f361d7fa700 1 -- 172.21.15.181:0/1481006441 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c011cd0 con 0x7f3620072760 2023-10-16T09:57:01.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.638+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36200c5ae0 con 0x7f3620072760 2023-10-16T09:57:01.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.639+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362006f6c0 con 0x7f3620072760 2023-10-16T09:57:01.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.640+0000 7f361d7fa700 1 -- 172.21.15.181:0/1481006441 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f360c023480 con 0x7f3620072760 2023-10-16T09:57:01.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.640+0000 7f361d7fa700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f360406b020 0x7f360406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:01.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.641+0000 7f361d7fa700 1 -- 172.21.15.181:0/1481006441 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6005+0+0 (secure 0 0 0) 0x7f360c089ee0 con 0x7f3620072760 2023-10-16T09:57:01.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.641+0000 7f3624e7e700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f360406b020 0x7f360406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:01.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.641+0000 7f3624e7e700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f360406b020 0x7f360406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3614003fc0 tx=0x7f3614009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:01.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.645+0000 7f361d7fa700 1 -- 172.21.15.181:0/1481006441 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f360c054c30 con 0x7f3620072760 2023-10-16T09:57:01.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:01.883+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3620067ca0 con 0x7f3620072760 2023-10-16T09:57:02.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.476+0000 7f361d7fa700 1 -- 172.21.15.181:0/1481006441 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f360c01e0c0 con 0x7f3620072760 2023-10-16T09:57:02.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f360406b020 msgr2=0x7f360406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f360406b020 0x7f360406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3614003fc0 tx=0x7f3614009040 comp rx=0 tx=0).stop 2023-10-16T09:57:02.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3620072760 msgr2=0x7f36200ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3620072760 0x7f36200ada90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c750 tx=0x7f360c00cb10 comp rx=0 tx=0).stop 2023-10-16T09:57:02.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 shutdown_connections 2023-10-16T09:57:02.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f360406b020 0x7f360406d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f362006ade0 0x7f3620157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f362015da70 0x7f36200adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 --2- 172.21.15.181:0/1481006441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3620072760 0x7f36200ada90 secure :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c750 tx=0x7f360c00cb10 comp rx=0 tx=0).stop 2023-10-16T09:57:02.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 >> 172.21.15.181:0/1481006441 conn(0x7f36200cd010 msgr2=0x7f36200cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:02.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 shutdown_connections 2023-10-16T09:57:02.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.480+0000 7f36270e2700 1 -- 172.21.15.181:0/1481006441 wait complete. 2023-10-16T09:57:02.488 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 6 and data pool 7 2023-10-16T09:57:02.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:57:02.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.633+0000 7f1323e48700 1 Processor -- start 2023-10-16T09:57:02.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.633+0000 7f1323e48700 1 -- start start 2023-10-16T09:57:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1323e48700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f131c15dfd0 0x7f131c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1323e48700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c0af450 0x7f131c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1323e48700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131c06f870 con 0x7f131c06db80 2023-10-16T09:57:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1323e48700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131c0d1160 con 0x7f131c15dfd0 2023-10-16T09:57:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1323e48700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0cc8e0 con 0x7f131c0af450 2023-10-16T09:57:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f131c15dfd0 0x7f131c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f131c15dfd0 0x7f131c0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40184/0 (socket says 172.21.15.181:40184) 2023-10-16T09:57:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f13223e5700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f131c06db80 0x7f131c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51266/0 (socket says 172.21.15.181:51266) 2023-10-16T09:57:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 -- 172.21.15.181:0/1147961044 learned_addr learned my addr 172.21.15.181:0/1147961044 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:02.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f13213e3700 1 --2- 172.21.15.181:0/1147961044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c0af450 0x7f131c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 -- 172.21.15.181:0/1147961044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c0af450 msgr2=0x7f131c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:02.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 --2- 172.21.15.181:0/1147961044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c0af450 0x7f131c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 -- 172.21.15.181:0/1147961044 >> v1:172.21.15.181:6789/0 conn(0x7f131c06db80 legacy=0x7f131c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 -- 172.21.15.181:0/1147961044 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0ccbc0 con 0x7f131c15dfd0 2023-10-16T09:57:02.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1320be2700 1 -- 172.21.15.181:0/1147961044 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668448881 0 0) 0x7f131c06f870 con 0x7f131c06db80 2023-10-16T09:57:02.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1321be4700 1 --2- 172.21.15.181:0/1147961044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c15dfd0 0x7f131c0af9a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1318006fd0 tx=0x7f1318009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d907a667a68424b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:02.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1320be2700 1 -- 172.21.15.181:0/1147961044 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318005e30 con 0x7f131c15dfd0 2023-10-16T09:57:02.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1320be2700 1 -- 172.21.15.181:0/1147961044 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318004030 con 0x7f131c15dfd0 2023-10-16T09:57:02.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.634+0000 7f1320be2700 1 -- 172.21.15.181:0/1147961044 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13180043e0 con 0x7f131c15dfd0 2023-10-16T09:57:02.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 -- 172.21.15.181:0/1147961044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c15dfd0 msgr2=0x7f131c0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 --2- 172.21.15.181:0/1147961044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c15dfd0 0x7f131c0af9a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1318006fd0 tx=0x7f1318009bd0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 -- 172.21.15.181:0/1147961044 shutdown_connections 2023-10-16T09:57:02.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 --2- 172.21.15.181:0/1147961044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c15dfd0 0x7f131c0af9a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 --2- 172.21.15.181:0/1147961044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c0af450 0x7f131c06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 -- 172.21.15.181:0/1147961044 >> 172.21.15.181:0/1147961044 conn(0x7f131c0d04a0 msgr2=0x7f131c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:02.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 -- 172.21.15.181:0/1147961044 shutdown_connections 2023-10-16T09:57:02.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.635+0000 7f1323e48700 1 -- 172.21.15.181:0/1147961044 wait complete. 2023-10-16T09:57:02.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.636+0000 7f1323e48700 1 Processor -- start 2023-10-16T09:57:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.636+0000 7f1323e48700 1 -- start start 2023-10-16T09:57:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1323e48700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c06db80 0x7f131c15e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1323e48700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c0af450 0x7f131c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1323e48700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c15dfd0 0x7f131c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1323e48700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c15f630 con 0x7f131c06db80 2023-10-16T09:57:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1323e48700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c15ea20 con 0x7f131c0af450 2023-10-16T09:57:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1323e48700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c15ed00 con 0x7f131c15dfd0 2023-10-16T09:57:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c06db80 0x7f131c15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c06db80 0x7f131c15e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40192/0 (socket says 172.21.15.181:40192) 2023-10-16T09:57:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 -- 172.21.15.181:0/1872747933 learned_addr learned my addr 172.21.15.181:0/1872747933 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f13223e5700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c15dfd0 0x7f131c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f13213e3700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c0af450 0x7f131c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 -- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c15dfd0 msgr2=0x7f131c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c15dfd0 0x7f131c0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 -- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c0af450 msgr2=0x7f131c0b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c0af450 0x7f131c0b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.637+0000 7f1321be4700 1 -- 172.21.15.181:0/1872747933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318009500 con 0x7f131c06db80 2023-10-16T09:57:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.638+0000 7f1321be4700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c06db80 0x7f131c15e440 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.638+0000 7f13127fc700 1 -- 172.21.15.181:0/1872747933 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318003ce0 con 0x7f131c06db80 2023-10-16T09:57:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.638+0000 7f13127fc700 1 -- 172.21.15.181:0/1872747933 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318003e60 con 0x7f131c06db80 2023-10-16T09:57:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.638+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c15efe0 con 0x7f131c06db80 2023-10-16T09:57:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.638+0000 7f13127fc700 1 -- 172.21.15.181:0/1872747933 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318011660 con 0x7f131c06db80 2023-10-16T09:57:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.638+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c0b60b0 con 0x7f131c06db80 2023-10-16T09:57:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.639+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c067d20 con 0x7f131c06db80 2023-10-16T09:57:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.639+0000 7f13127fc700 1 -- 172.21.15.181:0/1872747933 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f13180117c0 con 0x7f131c06db80 2023-10-16T09:57:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.640+0000 7f13127fc700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f130006b020 0x7f130006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.640+0000 7f13127fc700 1 -- 172.21.15.181:0/1872747933 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f131808adb0 con 0x7f131c06db80 2023-10-16T09:57:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.640+0000 7f13213e3700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f130006b020 0x7f130006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.640+0000 7f13213e3700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f130006b020 0x7f130006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f131c067c90 tx=0x7f130c008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.644+0000 7f13127fc700 1 -- 172.21.15.181:0/1872747933 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1318055c50 con 0x7f131c06db80 2023-10-16T09:57:02.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.838+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f131c0cd410 con 0x7f131c06db80 2023-10-16T09:57:02.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.840+0000 7f13127fc700 1 -- 172.21.15.181:0/1872747933 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+17162 (secure 0 0 0) 0x7f1318059270 con 0x7f131c06db80 2023-10-16T09:57:02.841 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:02.842 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:57:01.885136+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:57:00.297618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T09:57:00.838546+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:57:02.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f130006b020 msgr2=0x7f130006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f130006b020 0x7f130006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f131c067c90 tx=0x7f130c008040 comp rx=0 tx=0).stop 2023-10-16T09:57:02.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c06db80 msgr2=0x7f131c15e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c06db80 0x7f131c15e440 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1318000c00 tx=0x7f1318004750 comp rx=0 tx=0).stop 2023-10-16T09:57:02.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 shutdown_connections 2023-10-16T09:57:02.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f130006b020 0x7f130006d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f131c15dfd0 0x7f131c0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f131c06db80 0x7f131c15e440 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 --2- 172.21.15.181:0/1872747933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f131c0af450 0x7f131c0b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 >> 172.21.15.181:0/1872747933 conn(0x7f131c0d04a0 msgr2=0x7f131c0ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:02.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 shutdown_connections 2023-10-16T09:57:02.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.842+0000 7f1323e48700 1 -- 172.21.15.181:0/1872747933 wait complete. 2023-10-16T09:57:02.856 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:57:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.975+0000 7f771cfb8700 1 Processor -- start 2023-10-16T09:57:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.976+0000 7f771cfb8700 1 -- start start 2023-10-16T09:57:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.976+0000 7f771cfb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f771815b990 0x7f77180ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.976+0000 7f771cfb8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77180cd2d0 0x7f77180d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.976+0000 7f771cfb8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77180cfc60 0x7f77180d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.976+0000 7f771cfb8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180d0630 con 0x7f77180cd2d0 2023-10-16T09:57:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.976+0000 7f771cfb8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180d24f0 con 0x7f77180cfc60 2023-10-16T09:57:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.976+0000 7f771cfb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77180cc510 con 0x7f771815b990 2023-10-16T09:57:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77180cd2d0 0x7f77180d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77180cd2d0 0x7f77180d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40198/0 (socket says 172.21.15.181:40198) 2023-10-16T09:57:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 -- 172.21.15.181:0/365252196 learned_addr learned my addr 172.21.15.181:0/365252196 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7716d9d700 1 --2- 172.21.15.181:0/365252196 >> v2:172.21.15.190:3300/0 conn(0x7f77180cfc60 0x7f77180d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f771659c700 1 --2- 172.21.15.181:0/365252196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f771815b990 0x7f77180ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 -- 172.21.15.181:0/365252196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f771815b990 msgr2=0x7f77180ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 --2- 172.21.15.181:0/365252196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f771815b990 0x7f77180ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 -- 172.21.15.181:0/365252196 >> v2:172.21.15.190:3300/0 conn(0x7f77180cfc60 msgr2=0x7f77180d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 --2- 172.21.15.181:0/365252196 >> v2:172.21.15.190:3300/0 conn(0x7f77180cfc60 0x7f77180d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 -- 172.21.15.181:0/365252196 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180cc7f0 con 0x7f77180cd2d0 2023-10-16T09:57:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7716d9d700 1 --2- 172.21.15.181:0/365252196 >> v2:172.21.15.190:3300/0 conn(0x7f77180cfc60 0x7f77180d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f7715d9b700 1 --2- 172.21.15.181:0/365252196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cd2d0 0x7f77180d1fb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f77180cbc60 tx=0x7f7708009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6548d42b8ba14736 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f771559a700 1 -- 172.21.15.181:0/365252196 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800b870 con 0x7f77180cd2d0 2023-10-16T09:57:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f771559a700 1 -- 172.21.15.181:0/365252196 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800b9d0 con 0x7f77180cd2d0 2023-10-16T09:57:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.977+0000 7f771559a700 1 -- 172.21.15.181:0/365252196 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800bd60 con 0x7f77180cd2d0 2023-10-16T09:57:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 -- 172.21.15.181:0/365252196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cd2d0 msgr2=0x7f77180d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 --2- 172.21.15.181:0/365252196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cd2d0 0x7f77180d1fb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f77180cbc60 tx=0x7f7708009bd0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 -- 172.21.15.181:0/365252196 shutdown_connections 2023-10-16T09:57:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 --2- 172.21.15.181:0/365252196 >> v2:172.21.15.190:3300/0 conn(0x7f77180cfc60 0x7f77180d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 --2- 172.21.15.181:0/365252196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f771815b990 0x7f77180ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 --2- 172.21.15.181:0/365252196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cd2d0 0x7f77180d1fb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 -- 172.21.15.181:0/365252196 >> 172.21.15.181:0/365252196 conn(0x7f771809a9a0 msgr2=0x7f77180d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 -- 172.21.15.181:0/365252196 shutdown_connections 2023-10-16T09:57:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.978+0000 7f771cfb8700 1 -- 172.21.15.181:0/365252196 wait complete. 2023-10-16T09:57:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.979+0000 7f771cfb8700 1 Processor -- start 2023-10-16T09:57:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.979+0000 7f771cfb8700 1 -- start start 2023-10-16T09:57:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771cfb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180cd2d0 0x7f77180c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771cfb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cfc60 0x7f7718150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771cfb8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815b990 0x7f77180ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771cfb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77181f54c0 con 0x7f77180cfc60 2023-10-16T09:57:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771cfb8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180c9880 con 0x7f771815b990 2023-10-16T09:57:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771cfb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77181509d0 con 0x7f77180cd2d0 2023-10-16T09:57:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cfc60 0x7f7718150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cfc60 0x7f7718150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40206/0 (socket says 172.21.15.181:40206) 2023-10-16T09:57:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 -- 172.21.15.181:0/897979398 learned_addr learned my addr 172.21.15.181:0/897979398 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7716d9d700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815b990 0x7f77180ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771659c700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180cd2d0 0x7f77180c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 -- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180cd2d0 msgr2=0x7f77180c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180cd2d0 0x7f77180c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 -- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815b990 msgr2=0x7f77180ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815b990 0x7f77180ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f7715d9b700 1 -- 172.21.15.181:0/897979398 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708009550 con 0x7f77180cfc60 2023-10-16T09:57:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.980+0000 7f771659c700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180cd2d0 0x7f77180c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.981+0000 7f7715d9b700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cfc60 0x7f7718150490 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7708000c00 tx=0x7f7708004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.981+0000 7f7706ffd700 1 -- 172.21.15.181:0/897979398 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708004520 con 0x7f77180cfc60 2023-10-16T09:57:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.981+0000 7f7706ffd700 1 -- 172.21.15.181:0/897979398 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770801e070 con 0x7f77180cfc60 2023-10-16T09:57:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.981+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77181f51e0 con 0x7f77180cfc60 2023-10-16T09:57:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.981+0000 7f7706ffd700 1 -- 172.21.15.181:0/897979398 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080038c0 con 0x7f77180cfc60 2023-10-16T09:57:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.981+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77180c8b40 con 0x7f77180cfc60 2023-10-16T09:57:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.982+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7718067d20 con 0x7f77180cfc60 2023-10-16T09:57:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.982+0000 7f7706ffd700 1 -- 172.21.15.181:0/897979398 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7708003a20 con 0x7f77180cfc60 2023-10-16T09:57:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.982+0000 7f7706ffd700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f76f806b020 0x7f76f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.983+0000 7f7706ffd700 1 -- 172.21.15.181:0/897979398 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f7708020070 con 0x7f77180cfc60 2023-10-16T09:57:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.983+0000 7f771659c700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f76f806b020 0x7f76f806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.983+0000 7f771659c700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f76f806b020 0x7f76f806d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f770c006fd0 tx=0x7f770c008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:02.987+0000 7f7706ffd700 1 -- 172.21.15.181:0/897979398 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7708055c20 con 0x7f77180cfc60 2023-10-16T09:57:03.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.179+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77180cfa40 con 0x7f77180cfc60 2023-10-16T09:57:03.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.180+0000 7f7706ffd700 1 -- 172.21.15.181:0/897979398 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+17162 (secure 0 0 0) 0x7f7708015020 con 0x7f77180cfc60 2023-10-16T09:57:03.182 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:03.183 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:57:01.885136+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:57:00.297618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T09:57:00.838546+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:57:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f76f806b020 msgr2=0x7f76f806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f76f806b020 0x7f76f806d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f770c006fd0 tx=0x7f770c008040 comp rx=0 tx=0).stop 2023-10-16T09:57:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cfc60 msgr2=0x7f7718150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cfc60 0x7f7718150490 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7708000c00 tx=0x7f7708004750 comp rx=0 tx=0).stop 2023-10-16T09:57:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 shutdown_connections 2023-10-16T09:57:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f76f806b020 0x7f76f806d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77180cd2d0 0x7f77180c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815b990 0x7f77180ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 --2- 172.21.15.181:0/897979398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77180cfc60 0x7f7718150490 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 >> 172.21.15.181:0/897979398 conn(0x7f771809a9a0 msgr2=0x7f7718070500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 shutdown_connections 2023-10-16T09:57:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.183+0000 7f771cfb8700 1 -- 172.21.15.181:0/897979398 wait complete. 2023-10-16T09:57:03.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T09:57:03.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.318+0000 7fee88d71700 1 Processor -- start 2023-10-16T09:57:03.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.319+0000 7fee88d71700 1 -- start start 2023-10-16T09:57:03.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.319+0000 7fee88d71700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee84072760 0x7fee8406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.319+0000 7fee88d71700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8406b210 con 0x7fee8406ade0 2023-10-16T09:57:03.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.319+0000 7fee88d71700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840cef00 con 0x7fee8415da70 2023-10-16T09:57:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.319+0000 7fee88d71700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee840cec20 con 0x7fee84072760 2023-10-16T09:57:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee82d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fee8406ade0 0x7fee841615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51292/0 (socket says 172.21.15.181:51292) 2023-10-16T09:57:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee82d9d700 1 -- 172.21.15.181:0/1748650708 learned_addr learned my addr 172.21.15.181:0/1748650708 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee81d9b700 1 --2- 172.21.15.181:0/1748650708 >> v2:172.21.15.190:3300/0 conn(0x7fee84072760 0x7fee8406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18768824 0 0) 0x7fee8406b210 con 0x7fee8406ade0 2023-10-16T09:57:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee6c0035b0 con 0x7fee8406ade0 2023-10-16T09:57:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3792251887 0 0) 0x7fee6c0035b0 con 0x7fee8406ade0 2023-10-16T09:57:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 >> v2:172.21.15.190:3300/0 conn(0x7fee84072760 msgr2=0x7fee8406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 --2- 172.21.15.181:0/1748650708 >> v2:172.21.15.190:3300/0 conn(0x7fee84072760 0x7fee8406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 >> v1:172.21.15.190:6789/0 conn(0x7fee8415da70 legacy=0x7fee840ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84071c80 con 0x7fee8406ade0 2023-10-16T09:57:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fee7c0019f0 con 0x7fee8406ade0 2023-10-16T09:57:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 >> v1:172.21.15.181:6789/0 conn(0x7fee8406ade0 legacy=0x7fee841615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee6c003a00 0x7fee6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c005330 0x7fee6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840cec20 con 0x7fee6c004850 2023-10-16T09:57:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c00c200 con 0x7fee6c003a00 2023-10-16T09:57:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c00c3c0 con 0x7fee6c005330 2023-10-16T09:57:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee81d9b700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee8259c700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee6c003a00 0x7fee6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.320+0000 7fee82d9d700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c005330 0x7fee6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee81d9b700 1 -- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c005330 msgr2=0x7fee6c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:03.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee81d9b700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c005330 0x7fee6c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee81d9b700 1 -- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee6c003a00 msgr2=0x7fee6c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:03.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee81d9b700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee6c003a00 0x7fee6c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee82d9d700 1 -- 172.21.15.181:0/1748650708 reap_dead start 2023-10-16T09:57:03.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee81d9b700 1 -- 172.21.15.181:0/1748650708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c0035b0 con 0x7fee6c004850 2023-10-16T09:57:03.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee81d9b700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fee74009b30 tx=0x7fee6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd18830cd443971 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:03.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400e6c0 con 0x7fee6c004850 2023-10-16T09:57:03.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7400e820 con 0x7fee6c004850 2023-10-16T09:57:03.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee8159a700 1 -- 172.21.15.181:0/1748650708 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400eb90 con 0x7fee6c004850 2023-10-16T09:57:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee88d71700 1 -- 172.21.15.181:0/1748650708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c004850 msgr2=0x7fee6c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee88d71700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fee74009b30 tx=0x7fee6c005880 comp rx=0 tx=0).stop 2023-10-16T09:57:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.321+0000 7fee88d71700 1 -- 172.21.15.181:0/1748650708 shutdown_connections 2023-10-16T09:57:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.322+0000 7fee88d71700 1 --2- 172.21.15.181:0/1748650708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c004850 0x7fee6c004cd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.322+0000 7fee88d71700 1 -- 172.21.15.181:0/1748650708 >> 172.21.15.181:0/1748650708 conn(0x7fee840cd010 msgr2=0x7fee840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:03.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.322+0000 7fee88d71700 1 -- 172.21.15.181:0/1748650708 shutdown_connections 2023-10-16T09:57:03.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.322+0000 7fee88d71700 1 -- 172.21.15.181:0/1748650708 wait complete. 2023-10-16T09:57:03.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.323+0000 7fee88d71700 1 Processor -- start 2023-10-16T09:57:03.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.323+0000 7fee88d71700 1 -- start start 2023-10-16T09:57:03.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee88d71700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c004850 0x7fee841522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee88d71700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee84155f30 0x7fee840c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee88d71700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee840b0950 0x7fee840b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee88d71700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee840b1310 con 0x7fee840b0950 2023-10-16T09:57:03.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee88d71700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841527f0 con 0x7fee84155f30 2023-10-16T09:57:03.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee88d71700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee840c54b0 con 0x7fee6c004850 2023-10-16T09:57:03.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee82d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee840b0950 0x7fee840b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee82d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee840b0950 0x7fee840b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40226/0 (socket says 172.21.15.181:40226) 2023-10-16T09:57:03.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee82d9d700 1 -- 172.21.15.181:0/2868341756 learned_addr learned my addr 172.21.15.181:0/2868341756 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:03.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee8259c700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c004850 0x7fee841522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.324+0000 7fee81d9b700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee84155f30 0x7fee840c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee82d9d700 1 -- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c004850 msgr2=0x7fee841522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:03.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee82d9d700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c004850 0x7fee841522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee82d9d700 1 -- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee84155f30 msgr2=0x7fee840c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:03.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee82d9d700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee84155f30 0x7fee840c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:03.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee82d9d700 1 -- 172.21.15.181:0/2868341756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840c4600 con 0x7fee840b0950 2023-10-16T09:57:03.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee82d9d700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee840b0950 0x7fee840b0dd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fee8406f170 tx=0x7fee7c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:03.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee72ffd700 1 -- 172.21.15.181:0/2868341756 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c017070 con 0x7fee840b0950 2023-10-16T09:57:03.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee72ffd700 1 -- 172.21.15.181:0/2868341756 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c00eaa0 con 0x7fee840b0950 2023-10-16T09:57:03.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee840c48a0 con 0x7fee840b0950 2023-10-16T09:57:03.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee72ffd700 1 -- 172.21.15.181:0/2868341756 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00ec30 con 0x7fee840b0950 2023-10-16T09:57:03.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.325+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee840c4e60 con 0x7fee840b0950 2023-10-16T09:57:03.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.326+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee840c3e10 con 0x7fee840b0950 2023-10-16T09:57:03.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.326+0000 7fee72ffd700 1 -- 172.21.15.181:0/2868341756 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fee7c00ed90 con 0x7fee840b0950 2023-10-16T09:57:03.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.327+0000 7fee72ffd700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fee6406b020 0x7fee6406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:03.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.327+0000 7fee72ffd700 1 -- 172.21.15.181:0/2868341756 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6105+0+0 (secure 0 0 0) 0x7fee7c089380 con 0x7fee840b0950 2023-10-16T09:57:03.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.327+0000 7fee8259c700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fee6406b020 0x7fee6406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:03.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.327+0000 7fee8259c700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fee6406b020 0x7fee6406d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:03.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.331+0000 7fee72ffd700 1 -- 172.21.15.181:0/2868341756 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fee7c057c20 con 0x7fee840b0950 2023-10-16T09:57:03.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:03.574+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fee8406f6c0 con 0x7fee840b0950 2023-10-16T09:57:04.468 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-10-16T09:57:04.467+0000 7fedd4bfa700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:57:04.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.511+0000 7fee72ffd700 1 -- 172.21.15.181:0/2868341756 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fee7c0577b0 con 0x7fee840b0950 2023-10-16T09:57:04.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fee6406b020 msgr2=0x7fee6406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:04.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fee6406b020 0x7fee6406d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78009040 comp rx=0 tx=0).stop 2023-10-16T09:57:04.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee840b0950 msgr2=0x7fee840b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:04.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee840b0950 0x7fee840b0dd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fee8406f170 tx=0x7fee7c00e440 comp rx=0 tx=0).stop 2023-10-16T09:57:04.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 shutdown_connections 2023-10-16T09:57:04.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fee6406b020 0x7fee6406d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee6c004850 0x7fee841522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee840b0950 0x7fee840b0dd0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 --2- 172.21.15.181:0/2868341756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee84155f30 0x7fee840c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 >> 172.21.15.181:0/2868341756 conn(0x7fee840cd010 msgr2=0x7fee840cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:04.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 shutdown_connections 2023-10-16T09:57:04.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.514+0000 7fee88d71700 1 -- 172.21.15.181:0/2868341756 wait complete. 2023-10-16T09:57:04.531 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T09:57:04.531 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T09:57:04.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.659+0000 7fc428a31700 1 Processor -- start 2023-10-16T09:57:04.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.659+0000 7fc428a31700 1 -- start start 2023-10-16T09:57:04.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc428a31700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4240b2010 0x7fc4240cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:04.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc428a31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:04.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc428a31700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc42406f3b0 0x7fc42406f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:04.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc428a31700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240ce1f0 con 0x7fc42406f3b0 2023-10-16T09:57:04.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc428a31700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240b2930 con 0x7fc4240b2010 2023-10-16T09:57:04.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc428a31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240ccda0 con 0x7fc424071be0 2023-10-16T09:57:04.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc42406f3b0 0x7fc42406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:04.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc42406f3b0 0x7fc42406f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40232/0 (socket says 172.21.15.181:40232) 2023-10-16T09:57:04.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 -- 172.21.15.181:0/1078417712 learned_addr learned my addr 172.21.15.181:0/1078417712 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:04.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc421d9b700 1 --2- 172.21.15.181:0/1078417712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:04.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc42259c700 1 --2- 172.21.15.181:0/1078417712 >> v2:172.21.15.190:3300/0 conn(0x7fc4240b2010 0x7fc4240cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:04.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 -- 172.21.15.181:0/1078417712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 msgr2=0x7fc4240b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:04.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 --2- 172.21.15.181:0/1078417712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 -- 172.21.15.181:0/1078417712 >> v2:172.21.15.190:3300/0 conn(0x7fc4240b2010 msgr2=0x7fc4240cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:04.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 --2- 172.21.15.181:0/1078417712 >> v2:172.21.15.190:3300/0 conn(0x7fc4240b2010 0x7fc4240cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 -- 172.21.15.181:0/1078417712 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240727e0 con 0x7fc42406f3b0 2023-10-16T09:57:04.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc422d9d700 1 --2- 172.21.15.181:0/1078417712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc42406f840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc424070430 tx=0x7fc41c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201ab4dff2474b55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:04.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc42159a700 1 -- 172.21.15.181:0/1078417712 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00e070 con 0x7fc42406f3b0 2023-10-16T09:57:04.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc42159a700 1 -- 172.21.15.181:0/1078417712 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c00b9c0 con 0x7fc42406f3b0 2023-10-16T09:57:04.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.660+0000 7fc42159a700 1 -- 172.21.15.181:0/1078417712 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00bd70 con 0x7fc42406f3b0 2023-10-16T09:57:04.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 -- 172.21.15.181:0/1078417712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 msgr2=0x7fc42406f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:04.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 --2- 172.21.15.181:0/1078417712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc42406f840 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc424070430 tx=0x7fc41c00b330 comp rx=0 tx=0).stop 2023-10-16T09:57:04.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 -- 172.21.15.181:0/1078417712 shutdown_connections 2023-10-16T09:57:04.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 --2- 172.21.15.181:0/1078417712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc42406f840 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 --2- 172.21.15.181:0/1078417712 >> v2:172.21.15.190:3300/0 conn(0x7fc4240b2010 0x7fc4240cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 --2- 172.21.15.181:0/1078417712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 -- 172.21.15.181:0/1078417712 >> 172.21.15.181:0/1078417712 conn(0x7fc4240cd010 msgr2=0x7fc4240cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:04.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 -- 172.21.15.181:0/1078417712 shutdown_connections 2023-10-16T09:57:04.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.661+0000 7fc428a31700 1 -- 172.21.15.181:0/1078417712 wait complete. 2023-10-16T09:57:04.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.662+0000 7fc428a31700 1 Processor -- start 2023-10-16T09:57:04.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.662+0000 7fc428a31700 1 -- start start 2023-10-16T09:57:04.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc428a31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc4240ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:04.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc428a31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:04.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc428a31700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240b2010 0x7fc42414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc428a31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241f6b40 con 0x7fc42406f3b0 2023-10-16T09:57:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc428a31700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241f6e20 con 0x7fc4240b2010 2023-10-16T09:57:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc428a31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241f7100 con 0x7fc424071be0 2023-10-16T09:57:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc4240ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:04.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc4240ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40234/0 (socket says 172.21.15.181:40234) 2023-10-16T09:57:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 -- 172.21.15.181:0/2487035261 learned_addr learned my addr 172.21.15.181:0/2487035261 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc421d9b700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc422d9d700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240b2010 0x7fc42414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:04.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 -- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 msgr2=0x7fc4240b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 -- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240b2010 msgr2=0x7fc42414f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240b2010 0x7fc42414f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.663+0000 7fc42259c700 1 -- 172.21.15.181:0/2487035261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c009b30 con 0x7fc42406f3b0 2023-10-16T09:57:04.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.664+0000 7fc42259c700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc4240ad7d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c7f0 tx=0x7fc41400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:04.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.664+0000 7fc412ffd700 1 -- 172.21.15.181:0/2487035261 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414011800 con 0x7fc42406f3b0 2023-10-16T09:57:04.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.664+0000 7fc412ffd700 1 -- 172.21.15.181:0/2487035261 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414011960 con 0x7fc42406f3b0 2023-10-16T09:57:04.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.664+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4241f73e0 con 0x7fc42406f3b0 2023-10-16T09:57:04.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.664+0000 7fc412ffd700 1 -- 172.21.15.181:0/2487035261 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414011cf0 con 0x7fc42406f3b0 2023-10-16T09:57:04.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.664+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240b5480 con 0x7fc42406f3b0 2023-10-16T09:57:04.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.665+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4241f7900 con 0x7fc42406f3b0 2023-10-16T09:57:04.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.665+0000 7fc412ffd700 1 -- 172.21.15.181:0/2487035261 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc414018360 con 0x7fc42406f3b0 2023-10-16T09:57:04.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.665+0000 7fc412ffd700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc40406b020 0x7fc40406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:04.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.666+0000 7fc412ffd700 1 -- 172.21.15.181:0/2487035261 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7fc4140899e0 con 0x7fc42406f3b0 2023-10-16T09:57:04.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.666+0000 7fc421d9b700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc40406b020 0x7fc40406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:04.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.666+0000 7fc421d9b700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc40406b020 0x7fc40406d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc424158810 tx=0x7fc418008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:04.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.670+0000 7fc412ffd700 1 -- 172.21.15.181:0/2487035261 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc4241f7900 con 0x7fc42406f3b0 2023-10-16T09:57:04.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:04.905+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc4241f7900 con 0x7fc42406f3b0 2023-10-16T09:57:05.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.516+0000 7fc412ffd700 1 -- 172.21.15.181:0/2487035261 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc4241f7900 con 0x7fc42406f3b0 2023-10-16T09:57:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc40406b020 msgr2=0x7fc40406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc40406b020 0x7fc40406d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc424158810 tx=0x7fc418008040 comp rx=0 tx=0).stop 2023-10-16T09:57:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 msgr2=0x7fc4240ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc4240ad7d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c7f0 tx=0x7fc41400cbb0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 shutdown_connections 2023-10-16T09:57:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc40406b020 0x7fc40406d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240b2010 0x7fc42414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc42406f3b0 0x7fc4240ad7d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 --2- 172.21.15.181:0/2487035261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424071be0 0x7fc4240b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 >> 172.21.15.181:0/2487035261 conn(0x7fc4240cd010 msgr2=0x7fc424159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.519+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 shutdown_connections 2023-10-16T09:57:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.520+0000 7fc428a31700 1 -- 172.21.15.181:0/2487035261 wait complete. 2023-10-16T09:57:05.534 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T09:57:05.535 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:57:05.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.655+0000 7f45d8eb6700 1 Processor -- start 2023-10-16T09:57:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.655+0000 7f45d8eb6700 1 -- start start 2023-10-16T09:57:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.655+0000 7f45d8eb6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f45d40af450 0x7f45d40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.655+0000 7f45d8eb6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45d40cd470 0x7f45d40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.655+0000 7f45d8eb6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d406f870 con 0x7f45d415dfd0 2023-10-16T09:57:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.655+0000 7f45d8eb6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d40d1160 con 0x7f45d40cd470 2023-10-16T09:57:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.655+0000 7f45d8eb6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d40cc8e0 con 0x7f45d40af450 2023-10-16T09:57:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45d40cd470 0x7f45d40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45d40cd470 0x7f45d40cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40242/0 (socket says 172.21.15.181:40242) 2023-10-16T09:57:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 -- 172.21.15.181:0/2783781987 learned_addr learned my addr 172.21.15.181:0/2783781987 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d1d9b700 1 --2- 172.21.15.181:0/2783781987 >> v2:172.21.15.190:3300/0 conn(0x7f45d40af450 0x7f45d40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 -- 172.21.15.181:0/2783781987 >> v2:172.21.15.190:3300/0 conn(0x7f45d40af450 msgr2=0x7f45d40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 --2- 172.21.15.181:0/2783781987 >> v2:172.21.15.190:3300/0 conn(0x7f45d40af450 0x7f45d40af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 -- 172.21.15.181:0/2783781987 >> v1:172.21.15.190:6789/0 conn(0x7f45d415dfd0 legacy=0x7f45d406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 -- 172.21.15.181:0/2783781987 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d40ccbc0 con 0x7f45d40cd470 2023-10-16T09:57:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d2d9d700 1 --2- 172.21.15.181:0/2783781987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d40cd470 0x7f45d40cd920 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f45d40b3070 tx=0x7f45cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcc15c66025b0ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d159a700 1 -- 172.21.15.181:0/2783781987 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc00e070 con 0x7f45d40cd470 2023-10-16T09:57:05.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d159a700 1 -- 172.21.15.181:0/2783781987 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc003a20 con 0x7f45d40cd470 2023-10-16T09:57:05.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.656+0000 7f45d159a700 1 -- 172.21.15.181:0/2783781987 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc003dd0 con 0x7f45d40cd470 2023-10-16T09:57:05.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 -- 172.21.15.181:0/2783781987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d40cd470 msgr2=0x7f45d40cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:05.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/2783781987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d40cd470 0x7f45d40cd920 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f45d40b3070 tx=0x7f45cc00b330 comp rx=0 tx=0).stop 2023-10-16T09:57:05.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 -- 172.21.15.181:0/2783781987 shutdown_connections 2023-10-16T09:57:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/2783781987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d40cd470 0x7f45d40cd920 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/2783781987 >> v2:172.21.15.190:3300/0 conn(0x7f45d40af450 0x7f45d40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 -- 172.21.15.181:0/2783781987 >> 172.21.15.181:0/2783781987 conn(0x7f45d40d04a0 msgr2=0x7f45d40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 -- 172.21.15.181:0/2783781987 shutdown_connections 2023-10-16T09:57:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.657+0000 7f45d8eb6700 1 -- 172.21.15.181:0/2783781987 wait complete. 2023-10-16T09:57:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.658+0000 7f45d8eb6700 1 Processor -- start 2023-10-16T09:57:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.658+0000 7f45d8eb6700 1 -- start start 2023-10-16T09:57:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d8eb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45d40af450 0x7f45d41542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d8eb6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45d40cd470 0x7f45d40c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d8eb6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d415dfd0 0x7f45d406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d8eb6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40ca300 con 0x7f45d415dfd0 2023-10-16T09:57:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d8eb6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40c96f0 con 0x7f45d40cd470 2023-10-16T09:57:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d8eb6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40c99d0 con 0x7f45d40af450 2023-10-16T09:57:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d415dfd0 0x7f45d406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d415dfd0 0x7f45d406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40248/0 (socket says 172.21.15.181:40248) 2023-10-16T09:57:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 -- 172.21.15.181:0/1516584522 learned_addr learned my addr 172.21.15.181:0/1516584522 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 -- 172.21.15.181:0/1516584522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45d40af450 msgr2=0x7f45d41542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45d40af450 0x7f45d41542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 -- 172.21.15.181:0/1516584522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45d40cd470 msgr2=0x7f45d40c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45d40cd470 0x7f45d40c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 -- 172.21.15.181:0/1516584522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc009b30 con 0x7f45d415dfd0 2023-10-16T09:57:05.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.659+0000 7f45d2d9d700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d415dfd0 0x7f45d406a8a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f45cc015040 tx=0x7f45cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:05.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.660+0000 7f45c2ffd700 1 -- 172.21.15.181:0/1516584522 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004440 con 0x7f45d415dfd0 2023-10-16T09:57:05.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.660+0000 7f45c2ffd700 1 -- 172.21.15.181:0/1516584522 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc0045a0 con 0x7f45d415dfd0 2023-10-16T09:57:05.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.660+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d40c9cb0 con 0x7f45d415dfd0 2023-10-16T09:57:05.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.660+0000 7f45c2ffd700 1 -- 172.21.15.181:0/1516584522 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004950 con 0x7f45d415dfd0 2023-10-16T09:57:05.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.660+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d4154820 con 0x7f45d415dfd0 2023-10-16T09:57:05.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.661+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d406f600 con 0x7f45d415dfd0 2023-10-16T09:57:05.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.661+0000 7f45c2ffd700 1 -- 172.21.15.181:0/1516584522 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f45cc012430 con 0x7f45d415dfd0 2023-10-16T09:57:05.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.661+0000 7f45c2ffd700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f45b406b020 0x7f45b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:05.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.662+0000 7f45d259c700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f45b406b020 0x7f45b406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:05.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.662+0000 7f45c2ffd700 1 -- 172.21.15.181:0/1516584522 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f45cc08ae40 con 0x7f45d415dfd0 2023-10-16T09:57:05.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.662+0000 7f45d259c700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f45b406b020 0x7f45b406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45c80096f0 tx=0x7f45c8006c40 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:05.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.666+0000 7f45c2ffd700 1 -- 172.21.15.181:0/1516584522 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f45cc055bd0 con 0x7f45d415dfd0 2023-10-16T09:57:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.899+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45d4067d20 con 0x7f45d415dfd0 2023-10-16T09:57:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.900+0000 7f45c2ffd700 1 -- 172.21.15.181:0/1516584522 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f45cc017090 con 0x7f45d415dfd0 2023-10-16T09:57:05.902 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:05.902 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5925,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/47573574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":47573574},{"type":"v1","addr":"172.21.15.181:6836","nonce":47573574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3518254514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3518254514},{"type":"v1","addr":"172.21.15.181:6837","nonce":3518254514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4292050143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4292050143},{"type":"v1","addr":"172.21.15.190:6835","nonce":4292050143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:57:01.885491+0000","modified":"2023-10-16T09:57:04.906845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5943":{"gid":5943,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2407452630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2407452630},{"type":"v1","addr":"172.21.15.190:6834","nonce":2407452630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T09:57:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.901+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f45b406b020 msgr2=0x7f45b406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.901+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f45b406b020 0x7f45b406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45c80096f0 tx=0x7f45c8006c40 comp rx=0 tx=0).stop 2023-10-16T09:57:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d415dfd0 msgr2=0x7f45d406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d415dfd0 0x7f45d406a8a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f45cc015040 tx=0x7f45cc004ad0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 shutdown_connections 2023-10-16T09:57:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f45b406b020 0x7f45b406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45d40af450 0x7f45d41542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45d415dfd0 0x7f45d406a8a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 --2- 172.21.15.181:0/1516584522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45d40cd470 0x7f45d40c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 >> 172.21.15.181:0/1516584522 conn(0x7f45d40d04a0 msgr2=0x7f45d40cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 shutdown_connections 2023-10-16T09:57:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:05.902+0000 7f45d8eb6700 1 -- 172.21.15.181:0/1516584522 wait complete. 2023-10-16T09:57:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 25 2023-10-16T09:57:05.916 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:57:06.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.035+0000 7f6598e30700 1 Processor -- start 2023-10-16T09:57:06.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.035+0000 7f6598e30700 1 -- start start 2023-10-16T09:57:06.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f6598e30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 0x7f65940af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f6598e30700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659406f870 con 0x7f65940cd470 2023-10-16T09:57:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f6598e30700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940d1160 con 0x7f65940af450 2023-10-16T09:57:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f6598e30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65940cc8e0 con 0x7f659415dfd0 2023-10-16T09:57:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f6592d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f65940cd470 0x7f659406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51336/0 (socket says 172.21.15.181:51336) 2023-10-16T09:57:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f6592d9d700 1 -- 172.21.15.181:0/386242481 learned_addr learned my addr 172.21.15.181:0/386242481 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659259c700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 0x7f65940af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239921308 0 0) 0x7f659406f870 con 0x7f65940cd470 2023-10-16T09:57:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f657c0035b0 con 0x7f65940cd470 2023-10-16T09:57:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044325295 0 0) 0x7f657c0035b0 con 0x7f65940cd470 2023-10-16T09:57:06.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 msgr2=0x7f65940af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 0x7f65940af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 >> v1:172.21.15.190:6789/0 conn(0x7f65940af450 legacy=0x7f659406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940ccbc0 con 0x7f65940cd470 2023-10-16T09:57:06.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f658c0019f0 con 0x7f65940cd470 2023-10-16T09:57:06.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 >> v1:172.21.15.181:6789/0 conn(0x7f65940cd470 legacy=0x7f659406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c003a30 0x7f657c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 0x7f657c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657c004d90 0x7f657c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940cc8e0 con 0x7f657c004d90 2023-10-16T09:57:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c00bb40 con 0x7f657c003a30 2023-10-16T09:57:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.036+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c00bd00 con 0x7f659415dfd0 2023-10-16T09:57:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6592d9d700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657c004d90 0x7f657c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f659259c700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c003a30 0x7f657c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6591d9b700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 0x7f657c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6592d9d700 1 -- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 msgr2=0x7f657c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6592d9d700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f659415dfd0 0x7f657c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6592d9d700 1 -- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c003a30 msgr2=0x7f657c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6592d9d700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c003a30 0x7f657c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6592d9d700 1 -- 172.21.15.181:0/386242481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0035b0 con 0x7f657c004d90 2023-10-16T09:57:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f6592d9d700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657c004d90 0x7f657c005210 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f658c00a830 tx=0x7f657c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf53c953ce419370 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00a2f0 con 0x7f657c004d90 2023-10-16T09:57:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c012620 con 0x7f657c004d90 2023-10-16T09:57:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.037+0000 7f659159a700 1 -- 172.21.15.181:0/386242481 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0129d0 con 0x7f657c004d90 2023-10-16T09:57:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.038+0000 7f6598e30700 1 -- 172.21.15.181:0/386242481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657c004d90 msgr2=0x7f657c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.038+0000 7f6598e30700 1 --2- 172.21.15.181:0/386242481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657c004d90 0x7f657c005210 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f658c00a830 tx=0x7f657c005880 comp rx=0 tx=0).stop 2023-10-16T09:57:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.038+0000 7f6592d9d700 1 -- 172.21.15.181:0/386242481 reap_dead start 2023-10-16T09:57:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.038+0000 7f6598e30700 1 -- 172.21.15.181:0/386242481 shutdown_connections 2023-10-16T09:57:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.038+0000 7f6598e30700 1 -- 172.21.15.181:0/386242481 >> 172.21.15.181:0/386242481 conn(0x7f65940d04a0 msgr2=0x7f65940d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.038+0000 7f6598e30700 1 -- 172.21.15.181:0/386242481 shutdown_connections 2023-10-16T09:57:06.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.038+0000 7f6598e30700 1 -- 172.21.15.181:0/386242481 wait complete. 2023-10-16T09:57:06.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.039+0000 7f6598e30700 1 Processor -- start 2023-10-16T09:57:06.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.039+0000 7f6598e30700 1 -- start start 2023-10-16T09:57:06.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6598e30700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c004d90 0x7f6594159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6598e30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65940b54c0 0x7f65940b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6598e30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65940b0aa0 0x7f65940b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6598e30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940b5e80 con 0x7f65940b0aa0 2023-10-16T09:57:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6598e30700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941599a0 con 0x7f657c004d90 2023-10-16T09:57:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6598e30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65941f6850 con 0x7f65940b54c0 2023-10-16T09:57:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65940b0aa0 0x7f65940b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65940b0aa0 0x7f65940b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40270/0 (socket says 172.21.15.181:40270) 2023-10-16T09:57:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 -- 172.21.15.181:0/3175150818 learned_addr learned my addr 172.21.15.181:0/3175150818 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6591d9b700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65940b54c0 0x7f65940b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f659259c700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c004d90 0x7f6594159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 -- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65940b54c0 msgr2=0x7f65940b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65940b54c0 0x7f65940b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 -- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c004d90 msgr2=0x7f6594159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c004d90 0x7f6594159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 -- 172.21.15.181:0/3175150818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65941f6b30 con 0x7f65940b0aa0 2023-10-16T09:57:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6592d9d700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65940b0aa0 0x7f65940b0f20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f658c012000 tx=0x7f658c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6582ffd700 1 -- 172.21.15.181:0/3175150818 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c025070 con 0x7f65940b0aa0 2023-10-16T09:57:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.040+0000 7f6582ffd700 1 -- 172.21.15.181:0/3175150818 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c001280 con 0x7f65940b0aa0 2023-10-16T09:57:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.041+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65941f6e10 con 0x7f65940b0aa0 2023-10-16T09:57:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.041+0000 7f6582ffd700 1 -- 172.21.15.181:0/3175150818 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0017b0 con 0x7f65940b0aa0 2023-10-16T09:57:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.041+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65941f73d0 con 0x7f65940b0aa0 2023-10-16T09:57:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.042+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594067d20 con 0x7f65940b0aa0 2023-10-16T09:57:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.042+0000 7f6582ffd700 1 -- 172.21.15.181:0/3175150818 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f658c01e430 con 0x7f65940b0aa0 2023-10-16T09:57:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.042+0000 7f6582ffd700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f657406b020 0x7f657406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.042+0000 7f6582ffd700 1 -- 172.21.15.181:0/3175150818 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f658c08b0e0 con 0x7f65940b0aa0 2023-10-16T09:57:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.042+0000 7f659259c700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f657406b020 0x7f657406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.043+0000 7f659259c700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f657406b020 0x7f657406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6584009b80 tx=0x7f6584008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.047+0000 7f6582ffd700 1 -- 172.21.15.181:0/3175150818 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f658c055f80 con 0x7f65940b0aa0 2023-10-16T09:57:06.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.240+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65940ad1b0 con 0x7f65940b0aa0 2023-10-16T09:57:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.241+0000 7f6582ffd700 1 -- 172.21.15.181:0/3175150818 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17149 (secure 0 0 0) 0x7f658c0595a0 con 0x7f65940b0aa0 2023-10-16T09:57:06.243 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:06.244 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":59,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:57:03.477451+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:57:00.297618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T09:57:00.838546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:57:06.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.243+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f657406b020 msgr2=0x7f657406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.243+0000 7f6598e30700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f657406b020 0x7f657406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6584009b80 tx=0x7f6584008040 comp rx=0 tx=0).stop 2023-10-16T09:57:06.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65940b0aa0 msgr2=0x7f65940b0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65940b0aa0 0x7f65940b0f20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f658c012000 tx=0x7f658c001d80 comp rx=0 tx=0).stop 2023-10-16T09:57:06.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 shutdown_connections 2023-10-16T09:57:06.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f657406b020 0x7f657406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65940b0aa0 0x7f65940b0f20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f657c004d90 0x7f6594159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 --2- 172.21.15.181:0/3175150818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65940b54c0 0x7f65940b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 >> 172.21.15.181:0/3175150818 conn(0x7f65940d04a0 msgr2=0x7f65940d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:06.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 shutdown_connections 2023-10-16T09:57:06.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.244+0000 7f6598e30700 1 -- 172.21.15.181:0/3175150818 wait complete. 2023-10-16T09:57:06.258 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T09:57:06.258 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:57:06.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.378+0000 7f5636e24700 1 Processor -- start 2023-10-16T09:57:06.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.378+0000 7f5636e24700 1 -- start start 2023-10-16T09:57:06.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.378+0000 7f5636e24700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5630159730 0x7f563015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.378+0000 7f5636e24700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.378+0000 7f5636e24700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56300d0310 con 0x7f56300cc860 2023-10-16T09:57:06.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.378+0000 7f5636e24700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56300cee70 con 0x7f5630159730 2023-10-16T09:57:06.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.378+0000 7f5636e24700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5630159b10 con 0x7f56300cc400 2023-10-16T09:57:06.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f5634bc0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5630159730 0x7f563015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f56353c1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f56300cc860 0x7f563006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51354/0 (socket says 172.21.15.181:51354) 2023-10-16T09:57:06.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f56353c1700 1 -- 172.21.15.181:0/3161831228 learned_addr learned my addr 172.21.15.181:0/3161831228 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:06.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f562ffff700 1 --2- 172.21.15.181:0/3161831228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f5634bc0700 1 -- 172.21.15.181:0/3161831228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 msgr2=0x7f56300cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f5634bc0700 1 --2- 172.21.15.181:0/3161831228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f5634bc0700 1 -- 172.21.15.181:0/3161831228 >> v1:172.21.15.181:6789/0 conn(0x7f56300cc860 legacy=0x7f563006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f5634bc0700 1 -- 172.21.15.181:0/3161831228 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56300ce540 con 0x7f5630159730 2023-10-16T09:57:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f562f7fe700 1 -- 172.21.15.181:0/3161831228 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190670418 0 0) 0x7f56300d0310 con 0x7f56300cc860 2023-10-16T09:57:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f562ffff700 1 --2- 172.21.15.181:0/3161831228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f5634bc0700 1 --2- 172.21.15.181:0/3161831228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563015cd20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5624009820 tx=0x7f5624009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d07e6384f18cb72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f562f7fe700 1 -- 172.21.15.181:0/3161831228 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624005e30 con 0x7f5630159730 2023-10-16T09:57:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f562f7fe700 1 -- 172.21.15.181:0/3161831228 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5624006030 con 0x7f5630159730 2023-10-16T09:57:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.379+0000 7f562f7fe700 1 -- 172.21.15.181:0/3161831228 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56240063e0 con 0x7f5630159730 2023-10-16T09:57:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 -- 172.21.15.181:0/3161831228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 msgr2=0x7f563015cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 --2- 172.21.15.181:0/3161831228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563015cd20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5624009820 tx=0x7f5624009b30 comp rx=0 tx=0).stop 2023-10-16T09:57:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 -- 172.21.15.181:0/3161831228 shutdown_connections 2023-10-16T09:57:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 --2- 172.21.15.181:0/3161831228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563015cd20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 --2- 172.21.15.181:0/3161831228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 -- 172.21.15.181:0/3161831228 >> 172.21.15.181:0/3161831228 conn(0x7f56300d04a0 msgr2=0x7f56300d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 -- 172.21.15.181:0/3161831228 shutdown_connections 2023-10-16T09:57:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.380+0000 7f5636e24700 1 -- 172.21.15.181:0/3161831228 wait complete. 2023-10-16T09:57:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.381+0000 7f5636e24700 1 Processor -- start 2023-10-16T09:57:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.381+0000 7f5636e24700 1 -- start start 2023-10-16T09:57:06.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f5636e24700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f5636e24700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56300cc860 0x7f563014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f5636e24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f5636e24700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56300c8410 con 0x7f5630159730 2023-10-16T09:57:06.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f5636e24700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56300ca810 con 0x7f56300cc860 2023-10-16T09:57:06.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f5636e24700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56300c9c00 con 0x7f56300cc400 2023-10-16T09:57:06.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563014d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40282/0 (socket says 172.21.15.181:40282) 2023-10-16T09:57:06.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 -- 172.21.15.181:0/943930892 learned_addr learned my addr 172.21.15.181:0/943930892 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:06.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 -- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 msgr2=0x7f56300c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f562ffff700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56300cc860 0x7f563014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f5634bc0700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 -- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56300cc860 msgr2=0x7f563014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56300cc860 0x7f563014d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.382+0000 7f56353c1700 1 -- 172.21.15.181:0/943930892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624009430 con 0x7f5630159730 2023-10-16T09:57:06.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.383+0000 7f56353c1700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563014d770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f562800cbe0 tx=0x7f562800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.383+0000 7f562d7fa700 1 -- 172.21.15.181:0/943930892 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562800abc0 con 0x7f5630159730 2023-10-16T09:57:06.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.383+0000 7f562d7fa700 1 -- 172.21.15.181:0/943930892 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562800ad40 con 0x7f5630159730 2023-10-16T09:57:06.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.383+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56300c9ee0 con 0x7f5630159730 2023-10-16T09:57:06.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.383+0000 7f562d7fa700 1 -- 172.21.15.181:0/943930892 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628007310 con 0x7f5630159730 2023-10-16T09:57:06.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.383+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56300ca4a0 con 0x7f5630159730 2023-10-16T09:57:06.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.384+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56300ca950 con 0x7f5630159730 2023-10-16T09:57:06.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.384+0000 7f562d7fa700 1 -- 172.21.15.181:0/943930892 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5628007470 con 0x7f5630159730 2023-10-16T09:57:06.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.384+0000 7f562d7fa700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f561406b020 0x7f561406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.385+0000 7f5634bc0700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f561406b020 0x7f561406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.385+0000 7f562d7fa700 1 -- 172.21.15.181:0/943930892 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f5628089740 con 0x7f5630159730 2023-10-16T09:57:06.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.385+0000 7f5634bc0700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f561406b020 0x7f561406d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5624009b70 tx=0x7f5624008070 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.389+0000 7f562d7fa700 1 -- 172.21.15.181:0/943930892 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f56300ca950 con 0x7f5630159730 2023-10-16T09:57:06.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.580+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 --> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56300ceb00 con 0x7f561406b020 2023-10-16T09:57:06.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.589+0000 7f562d7fa700 1 -- 172.21.15.181:0/943930892 <== mgr.5685 v2:172.21.15.181:6832/3219577148 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289773 (secure 0 0 0) 0x7f5624006c00 con 0x7f561406b020 2023-10-16T09:57:06.594 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:06.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f561406b020 msgr2=0x7f561406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f561406b020 0x7f561406d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5624009b70 tx=0x7f5624008070 comp rx=0 tx=0).stop 2023-10-16T09:57:06.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 msgr2=0x7f563014d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563014d770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f562800cbe0 tx=0x7f562800a060 comp rx=0 tx=0).stop 2023-10-16T09:57:06.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 shutdown_connections 2023-10-16T09:57:06.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f561406b020 0x7f561406d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56300cc400 0x7f56300c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5630159730 0x7f563014d770 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 --2- 172.21.15.181:0/943930892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56300cc860 0x7f563014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 >> 172.21.15.181:0/943930892 conn(0x7f56300d04a0 msgr2=0x7f56300b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:06.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 shutdown_connections 2023-10-16T09:57:06.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.592+0000 7f5636e24700 1 -- 172.21.15.181:0/943930892 wait complete. 2023-10-16T09:57:06.599 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:57:06.606 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T09:57:06.419534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":292204,"kb_used_data":6640,"kb_used_omap":12,"kb_used_meta":285427,"kb_avail":754682516,"statfs":{"total":773094113280,"available":772794896384,"internally_reserved":0,"allocated":6799360,"data_stored":3761244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":292277816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.949265"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482733+0000","last_change":"2023-10-16T09:57:02.498783+0000","last_active":"2023-10-16T09:57:03.482733+0000","last_peered":"2023-10-16T09:57:03.482733+0000","last_clean":"2023-10-16T09:57:03.482733+0000","last_became_active":"2023-10-16T09:57:02.498490+0000","last_became_peered":"2023-10-16T09:57:02.498490+0000","last_unstale":"2023-10-16T09:57:03.482733+0000","last_undegraded":"2023-10-16T09:57:03.482733+0000","last_fullsized":"2023-10-16T09:57:03.482733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:34.135197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721330+0000","last_change":"2023-10-16T09:57:02.499533+0000","last_active":"2023-10-16T09:57:03.721330+0000","last_peered":"2023-10-16T09:57:03.721330+0000","last_clean":"2023-10-16T09:57:03.721330+0000","last_became_active":"2023-10-16T09:57:02.499186+0000","last_became_peered":"2023-10-16T09:57:02.499186+0000","last_unstale":"2023-10-16T09:57:03.721330+0000","last_undegraded":"2023-10-16T09:57:03.721330+0000","last_fullsized":"2023-10-16T09:57:03.721330+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:53.007837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496728+0000","last_change":"2023-10-16T09:57:02.491247+0000","last_active":"2023-10-16T09:57:03.496728+0000","last_peered":"2023-10-16T09:57:03.496728+0000","last_clean":"2023-10-16T09:57:03.496728+0000","last_became_active":"2023-10-16T09:57:02.491000+0000","last_became_peered":"2023-10-16T09:57:02.491000+0000","last_unstale":"2023-10-16T09:57:03.496728+0000","last_undegraded":"2023-10-16T09:57:03.496728+0000","last_fullsized":"2023-10-16T09:57:03.496728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:40:39.459389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721810+0000","last_change":"2023-10-16T09:57:02.500876+0000","last_active":"2023-10-16T09:57:03.721810+0000","last_peered":"2023-10-16T09:57:03.721810+0000","last_clean":"2023-10-16T09:57:03.721810+0000","last_became_active":"2023-10-16T09:57:02.500600+0000","last_became_peered":"2023-10-16T09:57:02.500600+0000","last_unstale":"2023-10-16T09:57:03.721810+0000","last_undegraded":"2023-10-16T09:57:03.721810+0000","last_fullsized":"2023-10-16T09:57:03.721810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:15.032313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721333+0000","last_change":"2023-10-16T09:57:02.499369+0000","last_active":"2023-10-16T09:57:03.721333+0000","last_peered":"2023-10-16T09:57:03.721333+0000","last_clean":"2023-10-16T09:57:03.721333+0000","last_became_active":"2023-10-16T09:57:02.499054+0000","last_became_peered":"2023-10-16T09:57:02.499054+0000","last_unstale":"2023-10-16T09:57:03.721333+0000","last_undegraded":"2023-10-16T09:57:03.721333+0000","last_fullsized":"2023-10-16T09:57:03.721333+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:23.723489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483356+0000","last_change":"2023-10-16T09:57:02.498533+0000","last_active":"2023-10-16T09:57:03.483356+0000","last_peered":"2023-10-16T09:57:03.483356+0000","last_clean":"2023-10-16T09:57:03.483356+0000","last_became_active":"2023-10-16T09:57:02.498250+0000","last_became_peered":"2023-10-16T09:57:02.498250+0000","last_unstale":"2023-10-16T09:57:03.483356+0000","last_undegraded":"2023-10-16T09:57:03.483356+0000","last_fullsized":"2023-10-16T09:57:03.483356+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:53.089815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482383+0000","last_change":"2023-10-16T09:57:02.506988+0000","last_active":"2023-10-16T09:57:03.482383+0000","last_peered":"2023-10-16T09:57:03.482383+0000","last_clean":"2023-10-16T09:57:03.482383+0000","last_became_active":"2023-10-16T09:57:02.506691+0000","last_became_peered":"2023-10-16T09:57:02.506691+0000","last_unstale":"2023-10-16T09:57:03.482383+0000","last_undegraded":"2023-10-16T09:57:03.482383+0000","last_fullsized":"2023-10-16T09:57:03.482383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:12.193189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721408+0000","last_change":"2023-10-16T09:57:02.496239+0000","last_active":"2023-10-16T09:57:03.721408+0000","last_peered":"2023-10-16T09:57:03.721408+0000","last_clean":"2023-10-16T09:57:03.721408+0000","last_became_active":"2023-10-16T09:57:02.495719+0000","last_became_peered":"2023-10-16T09:57:02.495719+0000","last_unstale":"2023-10-16T09:57:03.721408+0000","last_undegraded":"2023-10-16T09:57:03.721408+0000","last_fullsized":"2023-10-16T09:57:03.721408+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:15.820414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.722061+0000","last_change":"2023-10-16T09:57:02.504072+0000","last_active":"2023-10-16T09:57:03.722061+0000","last_peered":"2023-10-16T09:57:03.722061+0000","last_clean":"2023-10-16T09:57:03.722061+0000","last_became_active":"2023-10-16T09:57:02.503790+0000","last_became_peered":"2023-10-16T09:57:02.503790+0000","last_unstale":"2023-10-16T09:57:03.722061+0000","last_undegraded":"2023-10-16T09:57:03.722061+0000","last_fullsized":"2023-10-16T09:57:03.722061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:47.535638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721883+0000","last_change":"2023-10-16T09:57:02.507792+0000","last_active":"2023-10-16T09:57:03.721883+0000","last_peered":"2023-10-16T09:57:03.721883+0000","last_clean":"2023-10-16T09:57:03.721883+0000","last_became_active":"2023-10-16T09:57:02.507534+0000","last_became_peered":"2023-10-16T09:57:02.507534+0000","last_unstale":"2023-10-16T09:57:03.721883+0000","last_undegraded":"2023-10-16T09:57:03.721883+0000","last_fullsized":"2023-10-16T09:57:03.721883+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:11.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721690+0000","last_change":"2023-10-16T09:57:02.503379+0000","last_active":"2023-10-16T09:57:03.721690+0000","last_peered":"2023-10-16T09:57:03.721690+0000","last_clean":"2023-10-16T09:57:03.721690+0000","last_became_active":"2023-10-16T09:57:02.503086+0000","last_became_peered":"2023-10-16T09:57:02.503086+0000","last_unstale":"2023-10-16T09:57:03.721690+0000","last_undegraded":"2023-10-16T09:57:03.721690+0000","last_fullsized":"2023-10-16T09:57:03.721690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:31.904995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721430+0000","last_change":"2023-10-16T09:57:02.489457+0000","last_active":"2023-10-16T09:57:03.721430+0000","last_peered":"2023-10-16T09:57:03.721430+0000","last_clean":"2023-10-16T09:57:03.721430+0000","last_became_active":"2023-10-16T09:57:02.489207+0000","last_became_peered":"2023-10-16T09:57:02.489207+0000","last_unstale":"2023-10-16T09:57:03.721430+0000","last_undegraded":"2023-10-16T09:57:03.721430+0000","last_fullsized":"2023-10-16T09:57:03.721430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:06.726322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720948+0000","last_change":"2023-10-16T09:57:02.489644+0000","last_active":"2023-10-16T09:57:03.720948+0000","last_peered":"2023-10-16T09:57:03.720948+0000","last_clean":"2023-10-16T09:57:03.720948+0000","last_became_active":"2023-10-16T09:57:02.489347+0000","last_became_peered":"2023-10-16T09:57:02.489347+0000","last_unstale":"2023-10-16T09:57:03.720948+0000","last_undegraded":"2023-10-16T09:57:03.720948+0000","last_fullsized":"2023-10-16T09:57:03.720948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:34.661286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721621+0000","last_change":"2023-10-16T09:57:02.503795+0000","last_active":"2023-10-16T09:57:03.721621+0000","last_peered":"2023-10-16T09:57:03.721621+0000","last_clean":"2023-10-16T09:57:03.721621+0000","last_became_active":"2023-10-16T09:57:02.503497+0000","last_became_peered":"2023-10-16T09:57:02.503497+0000","last_unstale":"2023-10-16T09:57:03.721621+0000","last_undegraded":"2023-10-16T09:57:03.721621+0000","last_fullsized":"2023-10-16T09:57:03.721621+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:21.797779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483218+0000","last_change":"2023-10-16T09:57:02.493085+0000","last_active":"2023-10-16T09:57:03.483218+0000","last_peered":"2023-10-16T09:57:03.483218+0000","last_clean":"2023-10-16T09:57:03.483218+0000","last_became_active":"2023-10-16T09:57:02.492645+0000","last_became_peered":"2023-10-16T09:57:02.492645+0000","last_unstale":"2023-10-16T09:57:03.483218+0000","last_undegraded":"2023-10-16T09:57:03.483218+0000","last_fullsized":"2023-10-16T09:57:03.483218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:25.512535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.481788+0000","last_change":"2023-10-16T09:57:02.500796+0000","last_active":"2023-10-16T09:57:03.481788+0000","last_peered":"2023-10-16T09:57:03.481788+0000","last_clean":"2023-10-16T09:57:03.481788+0000","last_became_active":"2023-10-16T09:57:02.499965+0000","last_became_peered":"2023-10-16T09:57:02.499965+0000","last_unstale":"2023-10-16T09:57:03.481788+0000","last_undegraded":"2023-10-16T09:57:03.481788+0000","last_fullsized":"2023-10-16T09:57:03.481788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:42:34.299120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.497019+0000","last_change":"2023-10-16T09:57:02.492631+0000","last_active":"2023-10-16T09:57:03.497019+0000","last_peered":"2023-10-16T09:57:03.497019+0000","last_clean":"2023-10-16T09:57:03.497019+0000","last_became_active":"2023-10-16T09:57:02.492346+0000","last_became_peered":"2023-10-16T09:57:02.492346+0000","last_unstale":"2023-10-16T09:57:03.497019+0000","last_undegraded":"2023-10-16T09:57:03.497019+0000","last_fullsized":"2023-10-16T09:57:03.497019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:25.090871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496866+0000","last_change":"2023-10-16T09:57:02.502993+0000","last_active":"2023-10-16T09:57:03.496866+0000","last_peered":"2023-10-16T09:57:03.496866+0000","last_clean":"2023-10-16T09:57:03.496866+0000","last_became_active":"2023-10-16T09:57:02.501704+0000","last_became_peered":"2023-10-16T09:57:02.501704+0000","last_unstale":"2023-10-16T09:57:03.496866+0000","last_undegraded":"2023-10-16T09:57:03.496866+0000","last_fullsized":"2023-10-16T09:57:03.496866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:02.091343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482784+0000","last_change":"2023-10-16T09:57:02.500568+0000","last_active":"2023-10-16T09:57:03.482784+0000","last_peered":"2023-10-16T09:57:03.482784+0000","last_clean":"2023-10-16T09:57:03.482784+0000","last_became_active":"2023-10-16T09:57:02.500306+0000","last_became_peered":"2023-10-16T09:57:02.500306+0000","last_unstale":"2023-10-16T09:57:03.482784+0000","last_undegraded":"2023-10-16T09:57:03.482784+0000","last_fullsized":"2023-10-16T09:57:03.482784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:15.032313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720904+0000","last_change":"2023-10-16T09:57:02.503799+0000","last_active":"2023-10-16T09:57:03.720904+0000","last_peered":"2023-10-16T09:57:03.720904+0000","last_clean":"2023-10-16T09:57:03.720904+0000","last_became_active":"2023-10-16T09:57:02.503016+0000","last_became_peered":"2023-10-16T09:57:02.503016+0000","last_unstale":"2023-10-16T09:57:03.720904+0000","last_undegraded":"2023-10-16T09:57:03.720904+0000","last_fullsized":"2023-10-16T09:57:03.720904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:52.220943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483703+0000","last_change":"2023-10-16T09:57:02.495104+0000","last_active":"2023-10-16T09:57:03.483703+0000","last_peered":"2023-10-16T09:57:03.483703+0000","last_clean":"2023-10-16T09:57:03.483703+0000","last_became_active":"2023-10-16T09:57:02.494853+0000","last_became_peered":"2023-10-16T09:57:02.494853+0000","last_unstale":"2023-10-16T09:57:03.483703+0000","last_undegraded":"2023-10-16T09:57:03.483703+0000","last_fullsized":"2023-10-16T09:57:03.483703+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:51.237459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.722021+0000","last_change":"2023-10-16T09:57:02.502012+0000","last_active":"2023-10-16T09:57:03.722021+0000","last_peered":"2023-10-16T09:57:03.722021+0000","last_clean":"2023-10-16T09:57:03.722021+0000","last_became_active":"2023-10-16T09:57:02.501705+0000","last_became_peered":"2023-10-16T09:57:02.501705+0000","last_unstale":"2023-10-16T09:57:03.722021+0000","last_undegraded":"2023-10-16T09:57:03.722021+0000","last_fullsized":"2023-10-16T09:57:03.722021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:26:46.145532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482051+0000","last_change":"2023-10-16T09:57:02.502215+0000","last_active":"2023-10-16T09:57:03.482051+0000","last_peered":"2023-10-16T09:57:03.482051+0000","last_clean":"2023-10-16T09:57:03.482051+0000","last_became_active":"2023-10-16T09:57:02.501719+0000","last_became_peered":"2023-10-16T09:57:02.501719+0000","last_unstale":"2023-10-16T09:57:03.482051+0000","last_undegraded":"2023-10-16T09:57:03.482051+0000","last_fullsized":"2023-10-16T09:57:03.482051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:50.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482784+0000","last_change":"2023-10-16T09:57:02.497108+0000","last_active":"2023-10-16T09:57:03.482784+0000","last_peered":"2023-10-16T09:57:03.482784+0000","last_clean":"2023-10-16T09:57:03.482784+0000","last_became_active":"2023-10-16T09:57:02.495278+0000","last_became_peered":"2023-10-16T09:57:02.495278+0000","last_unstale":"2023-10-16T09:57:03.482784+0000","last_undegraded":"2023-10-16T09:57:03.482784+0000","last_fullsized":"2023-10-16T09:57:03.482784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:17.554907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482668+0000","last_change":"2023-10-16T09:57:02.502373+0000","last_active":"2023-10-16T09:57:03.482668+0000","last_peered":"2023-10-16T09:57:03.482668+0000","last_clean":"2023-10-16T09:57:03.482668+0000","last_became_active":"2023-10-16T09:57:02.502070+0000","last_became_peered":"2023-10-16T09:57:02.502070+0000","last_unstale":"2023-10-16T09:57:03.482668+0000","last_undegraded":"2023-10-16T09:57:03.482668+0000","last_fullsized":"2023-10-16T09:57:03.482668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:05.838939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720649+0000","last_change":"2023-10-16T09:57:02.490211+0000","last_active":"2023-10-16T09:57:03.720649+0000","last_peered":"2023-10-16T09:57:03.720649+0000","last_clean":"2023-10-16T09:57:03.720649+0000","last_became_active":"2023-10-16T09:57:02.489933+0000","last_became_peered":"2023-10-16T09:57:02.489933+0000","last_unstale":"2023-10-16T09:57:03.720649+0000","last_undegraded":"2023-10-16T09:57:03.720649+0000","last_fullsized":"2023-10-16T09:57:03.720649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:32.183411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496788+0000","last_change":"2023-10-16T09:57:02.491747+0000","last_active":"2023-10-16T09:57:03.496788+0000","last_peered":"2023-10-16T09:57:03.496788+0000","last_clean":"2023-10-16T09:57:03.496788+0000","last_became_active":"2023-10-16T09:57:02.491440+0000","last_became_peered":"2023-10-16T09:57:02.491440+0000","last_unstale":"2023-10-16T09:57:03.496788+0000","last_undegraded":"2023-10-16T09:57:03.496788+0000","last_fullsized":"2023-10-16T09:57:03.496788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:18.356265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721278+0000","last_change":"2023-10-16T09:57:02.504386+0000","last_active":"2023-10-16T09:57:03.721278+0000","last_peered":"2023-10-16T09:57:03.721278+0000","last_clean":"2023-10-16T09:57:03.721278+0000","last_became_active":"2023-10-16T09:57:02.504151+0000","last_became_peered":"2023-10-16T09:57:02.504151+0000","last_unstale":"2023-10-16T09:57:03.721278+0000","last_undegraded":"2023-10-16T09:57:03.721278+0000","last_fullsized":"2023-10-16T09:57:03.721278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:08.491805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482142+0000","last_change":"2023-10-16T09:57:02.502230+0000","last_active":"2023-10-16T09:57:03.482142+0000","last_peered":"2023-10-16T09:57:03.482142+0000","last_clean":"2023-10-16T09:57:03.482142+0000","last_became_active":"2023-10-16T09:57:02.501915+0000","last_became_peered":"2023-10-16T09:57:02.501915+0000","last_unstale":"2023-10-16T09:57:03.482142+0000","last_undegraded":"2023-10-16T09:57:03.482142+0000","last_fullsized":"2023-10-16T09:57:03.482142+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:15.820414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496668+0000","last_change":"2023-10-16T09:57:02.494785+0000","last_active":"2023-10-16T09:57:03.496668+0000","last_peered":"2023-10-16T09:57:03.496668+0000","last_clean":"2023-10-16T09:57:03.496668+0000","last_became_active":"2023-10-16T09:57:02.494513+0000","last_became_peered":"2023-10-16T09:57:02.494513+0000","last_unstale":"2023-10-16T09:57:03.496668+0000","last_undegraded":"2023-10-16T09:57:03.496668+0000","last_fullsized":"2023-10-16T09:57:03.496668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:07:19.933800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483611+0000","last_change":"2023-10-16T09:57:02.485713+0000","last_active":"2023-10-16T09:57:03.483611+0000","last_peered":"2023-10-16T09:57:03.483611+0000","last_clean":"2023-10-16T09:57:03.483611+0000","last_became_active":"2023-10-16T09:57:02.485411+0000","last_became_peered":"2023-10-16T09:57:02.485411+0000","last_unstale":"2023-10-16T09:57:03.483611+0000","last_undegraded":"2023-10-16T09:57:03.483611+0000","last_fullsized":"2023-10-16T09:57:03.483611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:43.271549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720777+0000","last_change":"2023-10-16T09:57:02.504167+0000","last_active":"2023-10-16T09:57:03.720777+0000","last_peered":"2023-10-16T09:57:03.720777+0000","last_clean":"2023-10-16T09:57:03.720777+0000","last_became_active":"2023-10-16T09:57:02.501615+0000","last_became_peered":"2023-10-16T09:57:02.501615+0000","last_unstale":"2023-10-16T09:57:03.720777+0000","last_undegraded":"2023-10-16T09:57:03.720777+0000","last_fullsized":"2023-10-16T09:57:03.720777+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:34.466216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720551+0000","last_change":"2023-10-16T09:57:02.500624+0000","last_active":"2023-10-16T09:57:03.720551+0000","last_peered":"2023-10-16T09:57:03.720551+0000","last_clean":"2023-10-16T09:57:03.720551+0000","last_became_active":"2023-10-16T09:57:02.499796+0000","last_became_peered":"2023-10-16T09:57:02.499796+0000","last_unstale":"2023-10-16T09:57:03.720551+0000","last_undegraded":"2023-10-16T09:57:03.720551+0000","last_fullsized":"2023-10-16T09:57:03.720551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:01.654953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"58'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483485+0000","last_change":"2023-10-16T09:57:01.497614+0000","last_active":"2023-10-16T09:57:03.483485+0000","last_peered":"2023-10-16T09:57:03.483485+0000","last_clean":"2023-10-16T09:57:03.483485+0000","last_became_active":"2023-10-16T09:57:01.497299+0000","last_became_peered":"2023-10-16T09:57:01.497299+0000","last_unstale":"2023-10-16T09:57:03.483485+0000","last_undegraded":"2023-10-16T09:57:03.483485+0000","last_fullsized":"2023-10-16T09:57:03.483485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:51.002135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482466+0000","last_change":"2023-10-16T09:57:02.496202+0000","last_active":"2023-10-16T09:57:03.482466+0000","last_peered":"2023-10-16T09:57:03.482466+0000","last_clean":"2023-10-16T09:57:03.482466+0000","last_became_active":"2023-10-16T09:57:02.495908+0000","last_became_peered":"2023-10-16T09:57:02.495908+0000","last_unstale":"2023-10-16T09:57:03.482466+0000","last_undegraded":"2023-10-16T09:57:03.482466+0000","last_fullsized":"2023-10-16T09:57:03.482466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:21.797779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496940+0000","last_change":"2023-10-16T09:57:01.486358+0000","last_active":"2023-10-16T09:57:03.496940+0000","last_peered":"2023-10-16T09:57:03.496940+0000","last_clean":"2023-10-16T09:57:03.496940+0000","last_became_active":"2023-10-16T09:57:01.485889+0000","last_became_peered":"2023-10-16T09:57:01.485889+0000","last_unstale":"2023-10-16T09:57:03.496940+0000","last_undegraded":"2023-10-16T09:57:03.496940+0000","last_fullsized":"2023-10-16T09:57:03.496940+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:23:23.647317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721267+0000","last_change":"2023-10-16T09:57:02.485785+0000","last_active":"2023-10-16T09:57:03.721267+0000","last_peered":"2023-10-16T09:57:03.721267+0000","last_clean":"2023-10-16T09:57:03.721267+0000","last_became_active":"2023-10-16T09:57:02.485507+0000","last_became_peered":"2023-10-16T09:57:02.485507+0000","last_unstale":"2023-10-16T09:57:03.721267+0000","last_undegraded":"2023-10-16T09:57:03.721267+0000","last_fullsized":"2023-10-16T09:57:03.721267+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:29.641894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720892+0000","last_change":"2023-10-16T09:57:01.495721+0000","last_active":"2023-10-16T09:57:03.720892+0000","last_peered":"2023-10-16T09:57:03.720892+0000","last_clean":"2023-10-16T09:57:03.720892+0000","last_became_active":"2023-10-16T09:57:01.495458+0000","last_became_peered":"2023-10-16T09:57:01.495458+0000","last_unstale":"2023-10-16T09:57:03.720892+0000","last_undegraded":"2023-10-16T09:57:03.720892+0000","last_fullsized":"2023-10-16T09:57:03.720892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:31.333631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720803+0000","last_change":"2023-10-16T09:57:02.507754+0000","last_active":"2023-10-16T09:57:03.720803+0000","last_peered":"2023-10-16T09:57:03.720803+0000","last_clean":"2023-10-16T09:57:03.720803+0000","last_became_active":"2023-10-16T09:57:02.507504+0000","last_became_peered":"2023-10-16T09:57:02.507504+0000","last_unstale":"2023-10-16T09:57:03.720803+0000","last_undegraded":"2023-10-16T09:57:03.720803+0000","last_fullsized":"2023-10-16T09:57:03.720803+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:38:43.919266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.481851+0000","last_change":"2023-10-16T09:57:01.493487+0000","last_active":"2023-10-16T09:57:03.481851+0000","last_peered":"2023-10-16T09:57:03.481851+0000","last_clean":"2023-10-16T09:57:03.481851+0000","last_became_active":"2023-10-16T09:57:01.493244+0000","last_became_peered":"2023-10-16T09:57:01.493244+0000","last_unstale":"2023-10-16T09:57:03.481851+0000","last_undegraded":"2023-10-16T09:57:03.481851+0000","last_fullsized":"2023-10-16T09:57:03.481851+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:22.716457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496658+0000","last_change":"2023-10-16T09:57:02.495317+0000","last_active":"2023-10-16T09:57:03.496658+0000","last_peered":"2023-10-16T09:57:03.496658+0000","last_clean":"2023-10-16T09:57:03.496658+0000","last_became_active":"2023-10-16T09:57:02.495069+0000","last_became_peered":"2023-10-16T09:57:02.495069+0000","last_unstale":"2023-10-16T09:57:03.496658+0000","last_undegraded":"2023-10-16T09:57:03.496658+0000","last_fullsized":"2023-10-16T09:57:03.496658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:14.899883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483828+0000","last_change":"2023-10-16T09:57:01.499386+0000","last_active":"2023-10-16T09:57:03.483828+0000","last_peered":"2023-10-16T09:57:03.483828+0000","last_clean":"2023-10-16T09:57:03.483828+0000","last_became_active":"2023-10-16T09:57:01.499050+0000","last_became_peered":"2023-10-16T09:57:01.499050+0000","last_unstale":"2023-10-16T09:57:03.483828+0000","last_undegraded":"2023-10-16T09:57:03.483828+0000","last_fullsized":"2023-10-16T09:57:03.483828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:39:24.013149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721728+0000","last_change":"2023-10-16T09:57:02.503892+0000","last_active":"2023-10-16T09:57:03.721728+0000","last_peered":"2023-10-16T09:57:03.721728+0000","last_clean":"2023-10-16T09:57:03.721728+0000","last_became_active":"2023-10-16T09:57:02.503612+0000","last_became_peered":"2023-10-16T09:57:02.503612+0000","last_unstale":"2023-10-16T09:57:03.721728+0000","last_undegraded":"2023-10-16T09:57:03.721728+0000","last_fullsized":"2023-10-16T09:57:03.721728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:05.838939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496792+0000","last_change":"2023-10-16T09:57:01.490574+0000","last_active":"2023-10-16T09:57:03.496792+0000","last_peered":"2023-10-16T09:57:03.496792+0000","last_clean":"2023-10-16T09:57:03.496792+0000","last_became_active":"2023-10-16T09:57:01.490282+0000","last_became_peered":"2023-10-16T09:57:01.490282+0000","last_unstale":"2023-10-16T09:57:03.496792+0000","last_undegraded":"2023-10-16T09:57:03.496792+0000","last_fullsized":"2023-10-16T09:57:03.496792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:12:51.430819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721675+0000","last_change":"2023-10-16T09:57:02.495657+0000","last_active":"2023-10-16T09:57:03.721675+0000","last_peered":"2023-10-16T09:57:03.721675+0000","last_clean":"2023-10-16T09:57:03.721675+0000","last_became_active":"2023-10-16T09:57:02.495387+0000","last_became_peered":"2023-10-16T09:57:02.495387+0000","last_unstale":"2023-10-16T09:57:03.721675+0000","last_undegraded":"2023-10-16T09:57:03.721675+0000","last_fullsized":"2023-10-16T09:57:03.721675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:58.467074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721039+0000","last_change":"2023-10-16T09:57:01.495934+0000","last_active":"2023-10-16T09:57:03.721039+0000","last_peered":"2023-10-16T09:57:03.721039+0000","last_clean":"2023-10-16T09:57:03.721039+0000","last_became_active":"2023-10-16T09:57:01.495715+0000","last_became_peered":"2023-10-16T09:57:01.495715+0000","last_unstale":"2023-10-16T09:57:03.721039+0000","last_undegraded":"2023-10-16T09:57:03.721039+0000","last_fullsized":"2023-10-16T09:57:03.721039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:59.481940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482081+0000","last_change":"2023-10-16T09:57:02.502453+0000","last_active":"2023-10-16T09:57:03.482081+0000","last_peered":"2023-10-16T09:57:03.482081+0000","last_clean":"2023-10-16T09:57:03.482081+0000","last_became_active":"2023-10-16T09:57:02.502114+0000","last_became_peered":"2023-10-16T09:57:02.502114+0000","last_unstale":"2023-10-16T09:57:03.482081+0000","last_undegraded":"2023-10-16T09:57:03.482081+0000","last_fullsized":"2023-10-16T09:57:03.482081+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:15.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721520+0000","last_change":"2023-10-16T09:57:01.494248+0000","last_active":"2023-10-16T09:57:03.721520+0000","last_peered":"2023-10-16T09:57:03.721520+0000","last_clean":"2023-10-16T09:57:03.721520+0000","last_became_active":"2023-10-16T09:57:01.493562+0000","last_became_peered":"2023-10-16T09:57:01.493562+0000","last_unstale":"2023-10-16T09:57:03.721520+0000","last_undegraded":"2023-10-16T09:57:03.721520+0000","last_fullsized":"2023-10-16T09:57:03.721520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:49.088283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496951+0000","last_change":"2023-10-16T09:57:02.501979+0000","last_active":"2023-10-16T09:57:03.496951+0000","last_peered":"2023-10-16T09:57:03.496951+0000","last_clean":"2023-10-16T09:57:03.496951+0000","last_became_active":"2023-10-16T09:57:02.501578+0000","last_became_peered":"2023-10-16T09:57:02.501578+0000","last_unstale":"2023-10-16T09:57:03.496951+0000","last_undegraded":"2023-10-16T09:57:03.496951+0000","last_fullsized":"2023-10-16T09:57:03.496951+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:02:51.464571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496995+0000","last_change":"2023-10-16T09:57:01.484026+0000","last_active":"2023-10-16T09:57:03.496995+0000","last_peered":"2023-10-16T09:57:03.496995+0000","last_clean":"2023-10-16T09:57:03.496995+0000","last_became_active":"2023-10-16T09:57:01.482945+0000","last_became_peered":"2023-10-16T09:57:01.482945+0000","last_unstale":"2023-10-16T09:57:03.496995+0000","last_undegraded":"2023-10-16T09:57:03.496995+0000","last_fullsized":"2023-10-16T09:57:03.496995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:05:51.952699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482581+0000","last_change":"2023-10-16T09:57:02.501470+0000","last_active":"2023-10-16T09:57:03.482581+0000","last_peered":"2023-10-16T09:57:03.482581+0000","last_clean":"2023-10-16T09:57:03.482581+0000","last_became_active":"2023-10-16T09:57:02.501197+0000","last_became_peered":"2023-10-16T09:57:02.501197+0000","last_unstale":"2023-10-16T09:57:03.482581+0000","last_undegraded":"2023-10-16T09:57:03.482581+0000","last_fullsized":"2023-10-16T09:57:03.482581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:13:25.472249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721955+0000","last_change":"2023-10-16T09:57:01.498263+0000","last_active":"2023-10-16T09:57:03.721955+0000","last_peered":"2023-10-16T09:57:03.721955+0000","last_clean":"2023-10-16T09:57:03.721955+0000","last_became_active":"2023-10-16T09:57:01.497972+0000","last_became_peered":"2023-10-16T09:57:01.497972+0000","last_unstale":"2023-10-16T09:57:03.721955+0000","last_undegraded":"2023-10-16T09:57:03.721955+0000","last_fullsized":"2023-10-16T09:57:03.721955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:14.028616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483607+0000","last_change":"2023-10-16T09:57:02.504550+0000","last_active":"2023-10-16T09:57:03.483607+0000","last_peered":"2023-10-16T09:57:03.483607+0000","last_clean":"2023-10-16T09:57:03.483607+0000","last_became_active":"2023-10-16T09:57:02.504265+0000","last_became_peered":"2023-10-16T09:57:02.504265+0000","last_unstale":"2023-10-16T09:57:03.483607+0000","last_undegraded":"2023-10-16T09:57:03.483607+0000","last_fullsized":"2023-10-16T09:57:03.483607+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:21.238124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"58'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482512+0000","last_change":"2023-10-16T09:57:01.492222+0000","last_active":"2023-10-16T09:57:03.482512+0000","last_peered":"2023-10-16T09:57:03.482512+0000","last_clean":"2023-10-16T09:57:03.482512+0000","last_became_active":"2023-10-16T09:57:01.492007+0000","last_became_peered":"2023-10-16T09:57:01.492007+0000","last_unstale":"2023-10-16T09:57:03.482512+0000","last_undegraded":"2023-10-16T09:57:03.482512+0000","last_fullsized":"2023-10-16T09:57:03.482512+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:38.639067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482242+0000","last_change":"2023-10-16T09:57:02.508665+0000","last_active":"2023-10-16T09:57:03.482242+0000","last_peered":"2023-10-16T09:57:03.482242+0000","last_clean":"2023-10-16T09:57:03.482242+0000","last_became_active":"2023-10-16T09:57:02.508407+0000","last_became_peered":"2023-10-16T09:57:02.508407+0000","last_unstale":"2023-10-16T09:57:03.482242+0000","last_undegraded":"2023-10-16T09:57:03.482242+0000","last_fullsized":"2023-10-16T09:57:03.482242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:44.151216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"58'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720942+0000","last_change":"2023-10-16T09:57:01.483356+0000","last_active":"2023-10-16T09:57:03.720942+0000","last_peered":"2023-10-16T09:57:03.720942+0000","last_clean":"2023-10-16T09:57:03.720942+0000","last_became_active":"2023-10-16T09:57:01.483076+0000","last_became_peered":"2023-10-16T09:57:01.483076+0000","last_unstale":"2023-10-16T09:57:03.720942+0000","last_undegraded":"2023-10-16T09:57:03.720942+0000","last_fullsized":"2023-10-16T09:57:03.720942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:25:49.254281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482688+0000","last_change":"2023-10-16T09:57:02.497819+0000","last_active":"2023-10-16T09:57:03.482688+0000","last_peered":"2023-10-16T09:57:03.482688+0000","last_clean":"2023-10-16T09:57:03.482688+0000","last_became_active":"2023-10-16T09:57:02.497541+0000","last_became_peered":"2023-10-16T09:57:02.497541+0000","last_unstale":"2023-10-16T09:57:03.482688+0000","last_undegraded":"2023-10-16T09:57:03.482688+0000","last_fullsized":"2023-10-16T09:57:03.482688+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:58.467074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483872+0000","last_change":"2023-10-16T09:57:01.497404+0000","last_active":"2023-10-16T09:57:03.483872+0000","last_peered":"2023-10-16T09:57:03.483872+0000","last_clean":"2023-10-16T09:57:03.483872+0000","last_became_active":"2023-10-16T09:57:01.497106+0000","last_became_peered":"2023-10-16T09:57:01.497106+0000","last_unstale":"2023-10-16T09:57:03.483872+0000","last_undegraded":"2023-10-16T09:57:03.483872+0000","last_fullsized":"2023-10-16T09:57:03.483872+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:22:29.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721870+0000","last_change":"2023-10-16T09:57:02.506387+0000","last_active":"2023-10-16T09:57:03.721870+0000","last_peered":"2023-10-16T09:57:03.721870+0000","last_clean":"2023-10-16T09:57:03.721870+0000","last_became_active":"2023-10-16T09:57:02.505679+0000","last_became_peered":"2023-10-16T09:57:02.505679+0000","last_unstale":"2023-10-16T09:57:03.721870+0000","last_undegraded":"2023-10-16T09:57:03.721870+0000","last_fullsized":"2023-10-16T09:57:03.721870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:32.457071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482278+0000","last_change":"2023-10-16T09:57:01.489882+0000","last_active":"2023-10-16T09:57:03.482278+0000","last_peered":"2023-10-16T09:57:03.482278+0000","last_clean":"2023-10-16T09:57:03.482278+0000","last_became_active":"2023-10-16T09:57:01.489594+0000","last_became_peered":"2023-10-16T09:57:01.489594+0000","last_unstale":"2023-10-16T09:57:03.482278+0000","last_undegraded":"2023-10-16T09:57:03.482278+0000","last_fullsized":"2023-10-16T09:57:03.482278+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:28.634862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483210+0000","last_change":"2023-10-16T09:57:02.498475+0000","last_active":"2023-10-16T09:57:03.483210+0000","last_peered":"2023-10-16T09:57:03.483210+0000","last_clean":"2023-10-16T09:57:03.483210+0000","last_became_active":"2023-10-16T09:57:02.498069+0000","last_became_peered":"2023-10-16T09:57:02.498069+0000","last_unstale":"2023-10-16T09:57:03.483210+0000","last_undegraded":"2023-10-16T09:57:03.483210+0000","last_fullsized":"2023-10-16T09:57:03.483210+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:34.135197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483304+0000","last_change":"2023-10-16T09:57:01.497381+0000","last_active":"2023-10-16T09:57:03.483304+0000","last_peered":"2023-10-16T09:57:03.483304+0000","last_clean":"2023-10-16T09:57:03.483304+0000","last_became_active":"2023-10-16T09:57:01.497088+0000","last_became_peered":"2023-10-16T09:57:01.497088+0000","last_unstale":"2023-10-16T09:57:03.483304+0000","last_undegraded":"2023-10-16T09:57:03.483304+0000","last_fullsized":"2023-10-16T09:57:03.483304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:43.144184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482621+0000","last_change":"2023-10-16T09:57:02.508912+0000","last_active":"2023-10-16T09:57:03.482621+0000","last_peered":"2023-10-16T09:57:03.482621+0000","last_clean":"2023-10-16T09:57:03.482621+0000","last_became_active":"2023-10-16T09:57:02.508637+0000","last_became_peered":"2023-10-16T09:57:02.508637+0000","last_unstale":"2023-10-16T09:57:03.482621+0000","last_undegraded":"2023-10-16T09:57:03.482621+0000","last_fullsized":"2023-10-16T09:57:03.482621+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:39.727242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"58'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721435+0000","last_change":"2023-10-16T09:57:01.494437+0000","last_active":"2023-10-16T09:57:03.721435+0000","last_peered":"2023-10-16T09:57:03.721435+0000","last_clean":"2023-10-16T09:57:03.721435+0000","last_became_active":"2023-10-16T09:57:01.493953+0000","last_became_peered":"2023-10-16T09:57:01.493953+0000","last_unstale":"2023-10-16T09:57:03.721435+0000","last_undegraded":"2023-10-16T09:57:03.721435+0000","last_fullsized":"2023-10-16T09:57:03.721435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:36.973274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720891+0000","last_change":"2023-10-16T09:57:02.497058+0000","last_active":"2023-10-16T09:57:03.720891+0000","last_peered":"2023-10-16T09:57:03.720891+0000","last_clean":"2023-10-16T09:57:03.720891+0000","last_became_active":"2023-10-16T09:57:02.496778+0000","last_became_peered":"2023-10-16T09:57:02.496778+0000","last_unstale":"2023-10-16T09:57:03.720891+0000","last_undegraded":"2023-10-16T09:57:03.720891+0000","last_fullsized":"2023-10-16T09:57:03.720891+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:15.708521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720831+0000","last_change":"2023-10-16T09:57:01.496294+0000","last_active":"2023-10-16T09:57:03.720831+0000","last_peered":"2023-10-16T09:57:03.720831+0000","last_clean":"2023-10-16T09:57:03.720831+0000","last_became_active":"2023-10-16T09:57:01.495974+0000","last_became_peered":"2023-10-16T09:57:01.495974+0000","last_unstale":"2023-10-16T09:57:03.720831+0000","last_undegraded":"2023-10-16T09:57:03.720831+0000","last_fullsized":"2023-10-16T09:57:03.720831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:31.825080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.722154+0000","last_change":"2023-10-16T09:57:02.496707+0000","last_active":"2023-10-16T09:57:03.722154+0000","last_peered":"2023-10-16T09:57:03.722154+0000","last_clean":"2023-10-16T09:57:03.722154+0000","last_became_active":"2023-10-16T09:57:02.496407+0000","last_became_peered":"2023-10-16T09:57:02.496407+0000","last_unstale":"2023-10-16T09:57:03.722154+0000","last_undegraded":"2023-10-16T09:57:03.722154+0000","last_fullsized":"2023-10-16T09:57:03.722154+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:34.650018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496831+0000","last_change":"2023-10-16T09:57:01.486168+0000","last_active":"2023-10-16T09:57:03.496831+0000","last_peered":"2023-10-16T09:57:03.496831+0000","last_clean":"2023-10-16T09:57:03.496831+0000","last_became_active":"2023-10-16T09:57:01.485528+0000","last_became_peered":"2023-10-16T09:57:01.485528+0000","last_unstale":"2023-10-16T09:57:03.496831+0000","last_undegraded":"2023-10-16T09:57:03.496831+0000","last_fullsized":"2023-10-16T09:57:03.496831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:00.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482761+0000","last_change":"2023-10-16T09:57:02.495682+0000","last_active":"2023-10-16T09:57:03.482761+0000","last_peered":"2023-10-16T09:57:03.482761+0000","last_clean":"2023-10-16T09:57:03.482761+0000","last_became_active":"2023-10-16T09:57:02.495092+0000","last_became_peered":"2023-10-16T09:57:02.495092+0000","last_unstale":"2023-10-16T09:57:03.482761+0000","last_undegraded":"2023-10-16T09:57:03.482761+0000","last_fullsized":"2023-10-16T09:57:03.482761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:38.272845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721224+0000","last_change":"2023-10-16T09:57:01.498949+0000","last_active":"2023-10-16T09:57:03.721224+0000","last_peered":"2023-10-16T09:57:03.721224+0000","last_clean":"2023-10-16T09:57:03.721224+0000","last_became_active":"2023-10-16T09:57:01.498663+0000","last_became_peered":"2023-10-16T09:57:01.498663+0000","last_unstale":"2023-10-16T09:57:03.721224+0000","last_undegraded":"2023-10-16T09:57:03.721224+0000","last_fullsized":"2023-10-16T09:57:03.721224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:19.054392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482917+0000","last_change":"2023-10-16T09:57:02.503089+0000","last_active":"2023-10-16T09:57:03.482917+0000","last_peered":"2023-10-16T09:57:03.482917+0000","last_clean":"2023-10-16T09:57:03.482917+0000","last_became_active":"2023-10-16T09:57:02.502819+0000","last_became_peered":"2023-10-16T09:57:02.502819+0000","last_unstale":"2023-10-16T09:57:03.482917+0000","last_undegraded":"2023-10-16T09:57:03.482917+0000","last_fullsized":"2023-10-16T09:57:03.482917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:32.457071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483679+0000","last_change":"2023-10-16T09:57:01.498498+0000","last_active":"2023-10-16T09:57:03.483679+0000","last_peered":"2023-10-16T09:57:03.483679+0000","last_clean":"2023-10-16T09:57:03.483679+0000","last_became_active":"2023-10-16T09:57:01.498188+0000","last_became_peered":"2023-10-16T09:57:01.498188+0000","last_unstale":"2023-10-16T09:57:03.483679+0000","last_undegraded":"2023-10-16T09:57:03.483679+0000","last_fullsized":"2023-10-16T09:57:03.483679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:22.365919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483590+0000","last_change":"2023-10-16T09:57:02.490682+0000","last_active":"2023-10-16T09:57:03.483590+0000","last_peered":"2023-10-16T09:57:03.483590+0000","last_clean":"2023-10-16T09:57:03.483590+0000","last_became_active":"2023-10-16T09:57:02.490328+0000","last_became_peered":"2023-10-16T09:57:02.490328+0000","last_unstale":"2023-10-16T09:57:03.483590+0000","last_undegraded":"2023-10-16T09:57:03.483590+0000","last_fullsized":"2023-10-16T09:57:03.483590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:32.643920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483935+0000","last_change":"2023-10-16T09:57:01.499258+0000","last_active":"2023-10-16T09:57:03.483935+0000","last_peered":"2023-10-16T09:57:03.483935+0000","last_clean":"2023-10-16T09:57:03.483935+0000","last_became_active":"2023-10-16T09:57:01.498677+0000","last_became_peered":"2023-10-16T09:57:01.498677+0000","last_unstale":"2023-10-16T09:57:03.483935+0000","last_undegraded":"2023-10-16T09:57:03.483935+0000","last_fullsized":"2023-10-16T09:57:03.483935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:27.428129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720966+0000","last_change":"2023-10-16T09:57:02.499562+0000","last_active":"2023-10-16T09:57:03.720966+0000","last_peered":"2023-10-16T09:57:03.720966+0000","last_clean":"2023-10-16T09:57:03.720966+0000","last_became_active":"2023-10-16T09:57:02.499275+0000","last_became_peered":"2023-10-16T09:57:02.499275+0000","last_unstale":"2023-10-16T09:57:03.720966+0000","last_undegraded":"2023-10-16T09:57:03.720966+0000","last_fullsized":"2023-10-16T09:57:03.720966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:42.435404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721012+0000","last_change":"2023-10-16T09:57:01.489676+0000","last_active":"2023-10-16T09:57:03.721012+0000","last_peered":"2023-10-16T09:57:03.721012+0000","last_clean":"2023-10-16T09:57:03.721012+0000","last_became_active":"2023-10-16T09:57:01.489323+0000","last_became_peered":"2023-10-16T09:57:01.489323+0000","last_unstale":"2023-10-16T09:57:03.721012+0000","last_undegraded":"2023-10-16T09:57:03.721012+0000","last_fullsized":"2023-10-16T09:57:03.721012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:41.717699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720868+0000","last_change":"2023-10-16T09:57:02.497905+0000","last_active":"2023-10-16T09:57:03.720868+0000","last_peered":"2023-10-16T09:57:03.720868+0000","last_clean":"2023-10-16T09:57:03.720868+0000","last_became_active":"2023-10-16T09:57:02.497217+0000","last_became_peered":"2023-10-16T09:57:02.497217+0000","last_unstale":"2023-10-16T09:57:03.720868+0000","last_undegraded":"2023-10-16T09:57:03.720868+0000","last_fullsized":"2023-10-16T09:57:03.720868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:32.070606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721434+0000","last_change":"2023-10-16T09:57:01.498404+0000","last_active":"2023-10-16T09:57:03.721434+0000","last_peered":"2023-10-16T09:57:03.721434+0000","last_clean":"2023-10-16T09:57:03.721434+0000","last_became_active":"2023-10-16T09:57:01.497919+0000","last_became_peered":"2023-10-16T09:57:01.497919+0000","last_unstale":"2023-10-16T09:57:03.721434+0000","last_undegraded":"2023-10-16T09:57:03.721434+0000","last_fullsized":"2023-10-16T09:57:03.721434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:44.665735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721445+0000","last_change":"2023-10-16T09:57:02.502769+0000","last_active":"2023-10-16T09:57:03.721445+0000","last_peered":"2023-10-16T09:57:03.721445+0000","last_clean":"2023-10-16T09:57:03.721445+0000","last_became_active":"2023-10-16T09:57:02.502474+0000","last_became_peered":"2023-10-16T09:57:02.502474+0000","last_unstale":"2023-10-16T09:57:03.721445+0000","last_undegraded":"2023-10-16T09:57:03.721445+0000","last_fullsized":"2023-10-16T09:57:03.721445+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:38.272845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.481707+0000","last_change":"2023-10-16T09:57:01.494332+0000","last_active":"2023-10-16T09:57:03.481707+0000","last_peered":"2023-10-16T09:57:03.481707+0000","last_clean":"2023-10-16T09:57:03.481707+0000","last_became_active":"2023-10-16T09:57:01.494040+0000","last_became_peered":"2023-10-16T09:57:01.494040+0000","last_unstale":"2023-10-16T09:57:03.481707+0000","last_undegraded":"2023-10-16T09:57:03.481707+0000","last_fullsized":"2023-10-16T09:57:03.481707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:09.030855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482595+0000","last_change":"2023-10-16T09:57:02.497941+0000","last_active":"2023-10-16T09:57:03.482595+0000","last_peered":"2023-10-16T09:57:03.482595+0000","last_clean":"2023-10-16T09:57:03.482595+0000","last_became_active":"2023-10-16T09:57:02.497503+0000","last_became_peered":"2023-10-16T09:57:02.497503+0000","last_unstale":"2023-10-16T09:57:03.482595+0000","last_undegraded":"2023-10-16T09:57:03.482595+0000","last_fullsized":"2023-10-16T09:57:03.482595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:31.904995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483441+0000","last_change":"2023-10-16T09:57:01.496444+0000","last_active":"2023-10-16T09:57:03.483441+0000","last_peered":"2023-10-16T09:57:03.483441+0000","last_clean":"2023-10-16T09:57:03.483441+0000","last_became_active":"2023-10-16T09:57:01.496156+0000","last_became_peered":"2023-10-16T09:57:01.496156+0000","last_unstale":"2023-10-16T09:57:03.483441+0000","last_undegraded":"2023-10-16T09:57:03.483441+0000","last_fullsized":"2023-10-16T09:57:03.483441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:57:08.056788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721752+0000","last_change":"2023-10-16T09:57:02.496832+0000","last_active":"2023-10-16T09:57:03.721752+0000","last_peered":"2023-10-16T09:57:03.721752+0000","last_clean":"2023-10-16T09:57:03.721752+0000","last_became_active":"2023-10-16T09:57:02.496563+0000","last_became_peered":"2023-10-16T09:57:02.496563+0000","last_unstale":"2023-10-16T09:57:03.721752+0000","last_undegraded":"2023-10-16T09:57:03.721752+0000","last_fullsized":"2023-10-16T09:57:03.721752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:17.554907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482602+0000","last_change":"2023-10-16T09:57:01.492346+0000","last_active":"2023-10-16T09:57:03.482602+0000","last_peered":"2023-10-16T09:57:03.482602+0000","last_clean":"2023-10-16T09:57:03.482602+0000","last_became_active":"2023-10-16T09:57:01.492119+0000","last_became_peered":"2023-10-16T09:57:01.492119+0000","last_unstale":"2023-10-16T09:57:03.482602+0000","last_undegraded":"2023-10-16T09:57:03.482602+0000","last_fullsized":"2023-10-16T09:57:03.482602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:46.730975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482413+0000","last_change":"2023-10-16T09:57:02.504229+0000","last_active":"2023-10-16T09:57:03.482413+0000","last_peered":"2023-10-16T09:57:03.482413+0000","last_clean":"2023-10-16T09:57:03.482413+0000","last_became_active":"2023-10-16T09:57:02.503617+0000","last_became_peered":"2023-10-16T09:57:02.503617+0000","last_unstale":"2023-10-16T09:57:03.482413+0000","last_undegraded":"2023-10-16T09:57:03.482413+0000","last_fullsized":"2023-10-16T09:57:03.482413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:57:09.063820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720492+0000","last_change":"2023-10-16T09:57:01.495576+0000","last_active":"2023-10-16T09:57:03.720492+0000","last_peered":"2023-10-16T09:57:03.720492+0000","last_clean":"2023-10-16T09:57:03.720492+0000","last_became_active":"2023-10-16T09:57:01.495343+0000","last_became_peered":"2023-10-16T09:57:01.495343+0000","last_unstale":"2023-10-16T09:57:03.720492+0000","last_undegraded":"2023-10-16T09:57:03.720492+0000","last_fullsized":"2023-10-16T09:57:03.720492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:47.083472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483475+0000","last_change":"2023-10-16T09:57:02.489394+0000","last_active":"2023-10-16T09:57:03.483475+0000","last_peered":"2023-10-16T09:57:03.483475+0000","last_clean":"2023-10-16T09:57:03.483475+0000","last_became_active":"2023-10-16T09:57:02.486244+0000","last_became_peered":"2023-10-16T09:57:02.486244+0000","last_unstale":"2023-10-16T09:57:03.483475+0000","last_undegraded":"2023-10-16T09:57:03.483475+0000","last_fullsized":"2023-10-16T09:57:03.483475+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:15.392064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482130+0000","last_change":"2023-10-16T09:57:01.488437+0000","last_active":"2023-10-16T09:57:03.482130+0000","last_peered":"2023-10-16T09:57:03.482130+0000","last_clean":"2023-10-16T09:57:03.482130+0000","last_became_active":"2023-10-16T09:57:01.488143+0000","last_became_peered":"2023-10-16T09:57:01.488143+0000","last_unstale":"2023-10-16T09:57:03.482130+0000","last_undegraded":"2023-10-16T09:57:03.482130+0000","last_fullsized":"2023-10-16T09:57:03.482130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:07.484773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483785+0000","last_change":"2023-10-16T09:57:02.499160+0000","last_active":"2023-10-16T09:57:03.483785+0000","last_peered":"2023-10-16T09:57:03.483785+0000","last_clean":"2023-10-16T09:57:03.483785+0000","last_became_active":"2023-10-16T09:57:02.498862+0000","last_became_peered":"2023-10-16T09:57:02.498862+0000","last_unstale":"2023-10-16T09:57:03.483785+0000","last_undegraded":"2023-10-16T09:57:03.483785+0000","last_fullsized":"2023-10-16T09:57:03.483785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:20.259232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496790+0000","last_change":"2023-10-16T09:57:01.497430+0000","last_active":"2023-10-16T09:57:03.496790+0000","last_peered":"2023-10-16T09:57:03.496790+0000","last_clean":"2023-10-16T09:57:03.496790+0000","last_became_active":"2023-10-16T09:57:01.497070+0000","last_became_peered":"2023-10-16T09:57:01.497070+0000","last_unstale":"2023-10-16T09:57:03.496790+0000","last_undegraded":"2023-10-16T09:57:03.496790+0000","last_fullsized":"2023-10-16T09:57:03.496790+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:14.885153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721051+0000","last_change":"2023-10-16T09:57:02.501544+0000","last_active":"2023-10-16T09:57:03.721051+0000","last_peered":"2023-10-16T09:57:03.721051+0000","last_clean":"2023-10-16T09:57:03.721051+0000","last_became_active":"2023-10-16T09:57:02.501258+0000","last_became_peered":"2023-10-16T09:57:02.501258+0000","last_unstale":"2023-10-16T09:57:03.721051+0000","last_undegraded":"2023-10-16T09:57:03.721051+0000","last_fullsized":"2023-10-16T09:57:03.721051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:10.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720945+0000","last_change":"2023-10-16T09:57:01.489733+0000","last_active":"2023-10-16T09:57:03.720945+0000","last_peered":"2023-10-16T09:57:03.720945+0000","last_clean":"2023-10-16T09:57:03.720945+0000","last_became_active":"2023-10-16T09:57:01.489315+0000","last_became_peered":"2023-10-16T09:57:01.489315+0000","last_unstale":"2023-10-16T09:57:03.720945+0000","last_undegraded":"2023-10-16T09:57:03.720945+0000","last_fullsized":"2023-10-16T09:57:03.720945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:16.646321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496825+0000","last_change":"2023-10-16T09:57:02.499008+0000","last_active":"2023-10-16T09:57:03.496825+0000","last_peered":"2023-10-16T09:57:03.496825+0000","last_clean":"2023-10-16T09:57:03.496825+0000","last_became_active":"2023-10-16T09:57:02.498717+0000","last_became_peered":"2023-10-16T09:57:02.498717+0000","last_unstale":"2023-10-16T09:57:03.496825+0000","last_undegraded":"2023-10-16T09:57:03.496825+0000","last_fullsized":"2023-10-16T09:57:03.496825+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:56:29.754255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721512+0000","last_change":"2023-10-16T09:57:01.499227+0000","last_active":"2023-10-16T09:57:03.721512+0000","last_peered":"2023-10-16T09:57:03.721512+0000","last_clean":"2023-10-16T09:57:03.721512+0000","last_became_active":"2023-10-16T09:57:01.498968+0000","last_became_peered":"2023-10-16T09:57:01.498968+0000","last_unstale":"2023-10-16T09:57:03.721512+0000","last_undegraded":"2023-10-16T09:57:03.721512+0000","last_fullsized":"2023-10-16T09:57:03.721512+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:35:00.655882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721957+0000","last_change":"2023-10-16T09:57:02.499730+0000","last_active":"2023-10-16T09:57:03.721957+0000","last_peered":"2023-10-16T09:57:03.721957+0000","last_clean":"2023-10-16T09:57:03.721957+0000","last_became_active":"2023-10-16T09:57:02.498886+0000","last_became_peered":"2023-10-16T09:57:02.498886+0000","last_unstale":"2023-10-16T09:57:03.721957+0000","last_undegraded":"2023-10-16T09:57:03.721957+0000","last_fullsized":"2023-10-16T09:57:03.721957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:41.784225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721423+0000","last_change":"2023-10-16T09:57:01.496300+0000","last_active":"2023-10-16T09:57:03.721423+0000","last_peered":"2023-10-16T09:57:03.721423+0000","last_clean":"2023-10-16T09:57:03.721423+0000","last_became_active":"2023-10-16T09:57:01.496004+0000","last_became_peered":"2023-10-16T09:57:01.496004+0000","last_unstale":"2023-10-16T09:57:03.721423+0000","last_undegraded":"2023-10-16T09:57:03.721423+0000","last_fullsized":"2023-10-16T09:57:03.721423+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:28.527406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"49'139","reported_seq":284,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T09:57:02.488626+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:57:02.488626+0000","last_peered":"2023-10-16T09:57:02.488626+0000","last_clean":"2023-10-16T09:57:02.488626+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:57:02.488626+0000","last_undegraded":"2023-10-16T09:57:02.488626+0000","last_fullsized":"2023-10-16T09:57:02.488626+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542305,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":914093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.248},"max":{"1min":0.70399999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.78300000000000003},"last":0.438}]},{"osd":1,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.25700000000000001},"max":{"1min":0.71099999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.437,"5min":0.41499999999999998,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.39500000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.46100000000000002,"5min":0.42099999999999999,"15min":0.308},"max":{"1min":0.68600000000000005,"5min":0.79400000000000004,"15min":0.80400000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.753,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.81200000000000006},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.34399999999999997},"max":{"1min":0.74199999999999999,"5min":0.83399999999999996,"15min":0.83699999999999997},"last":0.67100000000000004}]},{"osd":4,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60199999999999998,"5min":0.78300000000000003,"15min":0.86299999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76200000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77700000000000002,"5min":0.875,"15min":0.875},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":0.84299999999999997,"15min":1.0489999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72099999999999997,"5min":0.84199999999999997,"15min":1.373},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73699999999999999,"5min":0.85999999999999999,"15min":1.4099999999999999},"last":0.63100000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":704512,"data_stored":324307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58399999999999996,"15min":0.66000000000000003},"min":{"1min":0.34999999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68100000000000005,"5min":0.81999999999999995,"15min":6.194},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60499999999999998,"15min":0.65300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71899999999999997,"5min":0.78200000000000003,"15min":6.1609999999999996},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.44500000000000001,"5min":0.34399999999999997,"15min":0.28799999999999998},"max":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":1.054},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.77300000000000002,"15min":0.876},"last":0.61599999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58699999999999997,"15min":0.69599999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.78800000000000003,"15min":8.5600000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69599999999999995,"5min":0.754,"15min":1.1060000000000001},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71199999999999997,"5min":0.81899999999999995,"15min":1.175},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61199999999999999,"15min":0.68300000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77000000000000002,"5min":0.81399999999999995,"15min":6.2249999999999996},"last":0.63}]},{"osd":4,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.497,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.54400000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51400000000000001,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.33400000000000002,"15min":0.30199999999999999},"max":{"1min":0.68999999999999995,"5min":0.77500000000000002,"15min":0.84699999999999998},"last":0.439}]},{"osd":5,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":1.1240000000000001},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.40500000000000003,"5min":0.32500000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.83299999999999996,"15min":0.879},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56699999999999995,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35399999999999998},"max":{"1min":0.75800000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45700000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49904,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321936,"statfs":{"total":96636764160,"available":96585662464,"internally_reserved":0,"allocated":688128,"data_stored":317009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.216},"max":{"1min":0.47699999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42699999999999999,"15min":0.39800000000000002},"min":{"1min":0.33600000000000002,"5min":0.251,"15min":0.19600000000000001},"max":{"1min":0.55800000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.28699999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.38300000000000001},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.622},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.442,"15min":0.42099999999999999},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.30599999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.65800000000000003,"5min":0.66600000000000004,"15min":0.82199999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44700000000000001,"15min":0.42399999999999999},"min":{"1min":0.31,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.65900000000000003},"last":0.35999999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.378},"max":{"1min":0.90100000000000002,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80900000000000005,"5min":0.93500000000000005,"15min":0.97899999999999998},"last":0.47299999999999998}]},{"osd":5,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64000000000000001,"15min":0.63500000000000001},"min":{"1min":0.49399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.95799999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.47299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.879,"5min":0.93400000000000005,"15min":0.95399999999999996},"last":0.438}]},{"osd":6,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.47999999999999998,"5min":0.434,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":0.92200000000000004,"15min":1.0109999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.79900000000000004,"5min":0.90400000000000003,"15min":0.93799999999999994},"last":0.45400000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67500000000000004,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84699999999999998,"5min":0.89000000000000001,"15min":0.91700000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.65100000000000002},"min":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.41499999999999998},"max":{"1min":0.87,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.497}]}]},{"osd":0,"up_from":14,"seq":60129542305,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1282048,"data_stored":907341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.44700000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.191},"max":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":5.5090000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.502},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59399999999999997,"5min":0.62,"15min":6.4950000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41699999999999998,"15min":0.40200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26000000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.69599999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.437,"15min":0.46200000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59499999999999997,"5min":0.63600000000000001,"15min":2.7559999999999998},"last":0.378}]},{"osd":3,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44500000000000001,"15min":0.54500000000000004},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.20699999999999999},"max":{"1min":0.61699999999999999,"5min":0.67500000000000004,"15min":8.3260000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.55200000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.69699999999999995,"15min":5.4809999999999999},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63700000000000001,"15min":0.753},"min":{"1min":0.45600000000000002,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.82999999999999996,"5min":0.877,"15min":6.4349999999999996},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64300000000000002,"15min":0.73499999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.36699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":8.2970000000000006},"last":0.50700000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.71799999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":5.4160000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.51100000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":6.3869999999999996},"last":0.41799999999999998}]},{"osd":6,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64900000000000002,"15min":0.745},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.32900000000000001},"max":{"1min":0.88100000000000001,"5min":0.92400000000000004,"15min":8.2189999999999994},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":5.4560000000000004},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.80500000000000005},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.45200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":6.4649999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66600000000000004,"15min":0.76600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":8.266},"last":0.47999999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542304,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36036,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335804,"statfs":{"total":96636764160,"available":96599863296,"internally_reserved":0,"allocated":712704,"data_stored":325269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.36599999999999999},"min":{"1min":0.27800000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.54200000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.25900000000000001,"15min":0.24399999999999999},"max":{"1min":0.55400000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.28899999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42199999999999999,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.253,"15min":0.24299999999999999},"max":{"1min":0.56699999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.443,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.58199999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35199999999999998}]},{"osd":3,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.40200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.22900000000000001},"max":{"1min":0.53200000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.31},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.49099999999999999,"15min":0.45500000000000002},"min":{"1min":0.34799999999999998,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.60299999999999998,"5min":0.68500000000000005,"15min":0.72899999999999998},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63300000000000001,"15min":0.60999999999999999},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64600000000000002,"15min":0.621},"min":{"1min":0.47499999999999998,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.92700000000000005,"15min":0.95699999999999996},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.624},"min":{"1min":0.504,"5min":0.434,"15min":0.371},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.97699999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.628},"min":{"1min":0.45100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.42799999999999999},"max":{"1min":0.93799999999999994,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.63700000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.64900000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.65400000000000003},"min":{"1min":0.503,"5min":0.44600000000000001,"15min":0.39600000000000002},"max":{"1min":0.83399999999999996,"5min":0.998,"15min":0.998},"last":0.625}]}]},{"osd":3,"up_from":14,"seq":60129542305,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":324273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.375},"min":{"1min":0.223,"5min":0.219,"15min":0.192},"max":{"1min":0.63900000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.435,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.27400000000000002,"15min":0.254},"max":{"1min":0.56299999999999994,"5min":0.63100000000000001,"15min":1.0780000000000001},"last":0.495}]},{"osd":1,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.219},"max":{"1min":0.58699999999999997,"5min":0.72199999999999998,"15min":2.254},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.434,"15min":0.47399999999999998},"min":{"1min":0.318,"5min":0.27600000000000002,"15min":0.246},"max":{"1min":0.754,"5min":0.754,"15min":6.2240000000000002},"last":0.51900000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44800000000000001,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.26000000000000001,"15min":0.25},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":2.2909999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46000000000000002,"15min":0.435},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.28199999999999997},"max":{"1min":0.66600000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40400000000000003},"max":{"1min":0.85099999999999998,"5min":0.95799999999999996,"15min":1.24},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.51400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85299999999999998,"5min":0.93600000000000005,"15min":2.4420000000000002},"last":0.91700000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.879,"5min":0.97599999999999998,"15min":2.4089999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.42899999999999999,"15min":0.39200000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.996},"last":0.93300000000000005}]},{"osd":6,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65500000000000003,"15min":0.63800000000000001},"min":{"1min":0.52700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85699999999999998,"5min":0.96699999999999997,"15min":0.98399999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.379},"max":{"1min":0.86799999999999999,"5min":0.99099999999999999,"15min":1.1619999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.377,"15min":0.377},"max":{"1min":0.90400000000000003,"5min":0.96399999999999997,"15min":1.206},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91300000000000003,"5min":0.97499999999999998,"15min":2.3740000000000001},"last":0.94999999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542304,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54084,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317756,"statfs":{"total":96636764160,"available":96581382144,"internally_reserved":0,"allocated":708608,"data_stored":325247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.186,"15min":0.186},"max":{"1min":0.752,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.248},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.49399999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.85399999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81999999999999995,"5min":0.82799999999999996,"15min":0.89100000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57499999999999996},"min":{"1min":0.50600000000000001,"5min":0.39300000000000002,"15min":0.35099999999999998},"max":{"1min":0.77900000000000003,"5min":0.83299999999999996,"15min":0.86499999999999999},"last":0.624},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45600000000000002,"5min":0.41199999999999998,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.505},"min":{"1min":0.37,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.81000000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.80400000000000005,"15min":0.86899999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.40300000000000002,"5min":0.32800000000000001,"15min":0.32200000000000001},"max":{"1min":0.78900000000000003,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85999999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.373,"15min":0.33300000000000002},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.84599999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.95999999999999996},"last":0.78400000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":323705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60799999999999998,"15min":0.57099999999999995},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.84399999999999997,"15min":0.92900000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.71199999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":8.0370000000000008},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.24399999999999999},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.98599999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.70199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38,"15min":0.307},"max":{"1min":0.71299999999999997,"5min":0.92000000000000004,"15min":8.2110000000000003},"last":0.59199999999999997}]},{"osd":2,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59799999999999998,"15min":0.69299999999999995},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":8.1140000000000008},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61099999999999999,"15min":0.69599999999999995},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.72499999999999998,"5min":0.81799999999999995,"15min":8.3000000000000007},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.629,"15min":0.62},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.89000000000000001,"15min":1.03},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.749},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.874,"15min":9.8810000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.34499999999999997,"5min":0.28999999999999998,"15min":0.26000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":1.0449999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51400000000000001,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.81000000000000005,"15min":1.0780000000000001},"last":0.36099999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.55300000000000005},"min":{"1min":0.38300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":1.599},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.83099999999999996,"15min":1.006},"last":0.442}]},{"osd":7,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.373,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.85699999999999998,"15min":1.1339999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.436,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79500000000000004,"5min":0.84799999999999998,"15min":1.889},"last":0.46999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:57:06.613 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:57:06.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.764+0000 7f4218436700 1 Processor -- start 2023-10-16T09:57:06.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.765+0000 7f4218436700 1 -- start start 2023-10-16T09:57:06.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.765+0000 7f4218436700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 0x7f42100cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.765+0000 7f4218436700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f421015e7f0 0x7f421006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.765+0000 7f4218436700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100cc310 con 0x7f42100cf2e0 2023-10-16T09:57:06.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.765+0000 7f4218436700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421015e0f0 con 0x7f421015e7f0 2023-10-16T09:57:06.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.765+0000 7f4218436700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100cf760 con 0x7f42100b1ee0 2023-10-16T09:57:06.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42159d1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f42100cf2e0 0x7f42100b22c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51366/0 (socket says 172.21.15.181:51366) 2023-10-16T09:57:06.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42159d1700 1 -- 172.21.15.181:0/2154869519 learned_addr learned my addr 172.21.15.181:0/2154869519 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:06.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42169d3700 1 --2- 172.21.15.181:0/2154869519 >> v2:172.21.15.190:3300/0 conn(0x7f421015e7f0 0x7f421006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42161d2700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 0x7f42100cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464894578 0 0) 0x7f42100cc310 con 0x7f42100cf2e0 2023-10-16T09:57:06.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41fc0035b0 con 0x7f42100cf2e0 2023-10-16T09:57:06.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2501570942 0 0) 0x7f41fc0035b0 con 0x7f42100cf2e0 2023-10-16T09:57:06.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 msgr2=0x7f42100cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 0x7f42100cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 >> v2:172.21.15.190:3300/0 conn(0x7f421015e7f0 msgr2=0x7f421006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 --2- 172.21.15.181:0/2154869519 >> v2:172.21.15.190:3300/0 conn(0x7f421015e7f0 0x7f421006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100cc590 con 0x7f42100cf2e0 2023-10-16T09:57:06.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f420c001670 con 0x7f42100cf2e0 2023-10-16T09:57:06.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 >> v1:172.21.15.181:6789/0 conn(0x7f42100cf2e0 legacy=0x7f42100b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41fc003a00 0x7f41fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f41fc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 0x7f41fc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421015e0f0 con 0x7f41fc004850 2023-10-16T09:57:06.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42100cf760 con 0x7f41fc003a00 2023-10-16T09:57:06.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc00ba10 con 0x7f42100b1ee0 2023-10-16T09:57:06.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42159d1700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f41fc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42161d2700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41fc003a00 0x7f41fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.766+0000 7f42169d3700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 0x7f41fc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42159d1700 1 -- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 msgr2=0x7f41fc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42159d1700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100b1ee0 0x7f41fc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42159d1700 1 -- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41fc003a00 msgr2=0x7f41fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:06.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42159d1700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41fc003a00 0x7f41fc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42159d1700 1 -- 172.21.15.181:0/2154869519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0035b0 con 0x7f41fc004850 2023-10-16T09:57:06.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42159d1700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f41fc004cd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c350 tx=0x7f41fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc1ecba1634ad3f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c00ce50 con 0x7f41fc004850 2023-10-16T09:57:06.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c005960 con 0x7f41fc004850 2023-10-16T09:57:06.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.767+0000 7f42151d0700 1 -- 172.21.15.181:0/2154869519 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c005d10 con 0x7f41fc004850 2023-10-16T09:57:06.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.768+0000 7f4218436700 1 -- 172.21.15.181:0/2154869519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 msgr2=0x7f41fc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.768+0000 7f4218436700 1 --2- 172.21.15.181:0/2154869519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f41fc004cd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c350 tx=0x7f41fc005880 comp rx=0 tx=0).stop 2023-10-16T09:57:06.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.768+0000 7f42169d3700 1 -- 172.21.15.181:0/2154869519 reap_dead start 2023-10-16T09:57:06.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.768+0000 7f4218436700 1 -- 172.21.15.181:0/2154869519 shutdown_connections 2023-10-16T09:57:06.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.768+0000 7f4218436700 1 -- 172.21.15.181:0/2154869519 >> 172.21.15.181:0/2154869519 conn(0x7f42100d0460 msgr2=0x7f42100d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:06.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.768+0000 7f4218436700 1 -- 172.21.15.181:0/2154869519 shutdown_connections 2023-10-16T09:57:06.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.768+0000 7f4218436700 1 -- 172.21.15.181:0/2154869519 wait complete. 2023-10-16T09:57:06.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.769+0000 7f4218436700 1 Processor -- start 2023-10-16T09:57:06.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.769+0000 7f4218436700 1 -- start start 2023-10-16T09:57:06.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f421014dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421006a8a0 0x7f421006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100c5500 0x7f42100c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42100c6ef0 con 0x7f41fc004850 2023-10-16T09:57:06.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421006b260 con 0x7f421006a8a0 2023-10-16T09:57:06.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421014e130 con 0x7f42100c5500 2023-10-16T09:57:06.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f421014dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f421014dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40298/0 (socket says 172.21.15.181:40298) 2023-10-16T09:57:06.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 -- 172.21.15.181:0/1318075573 learned_addr learned my addr 172.21.15.181:0/1318075573 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:06.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42159d1700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421006a8a0 0x7f421006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42169d3700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100c5500 0x7f42100c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 -- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100c5500 msgr2=0x7f42100c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100c5500 0x7f42100c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 -- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421006a8a0 msgr2=0x7f421006ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421006a8a0 0x7f421006ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 -- 172.21.15.181:0/1318075573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100c6270 con 0x7f41fc004850 2023-10-16T09:57:06.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42159d1700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421006a8a0 0x7f421006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:06.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42169d3700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100c5500 0x7f42100c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:06.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f42161d2700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f421014dbf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4200009fd0 tx=0x7f4200011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4206ffd700 1 -- 172.21.15.181:0/1318075573 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200009610 con 0x7f41fc004850 2023-10-16T09:57:06.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4206ffd700 1 -- 172.21.15.181:0/1318075573 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200005c30 con 0x7f41fc004850 2023-10-16T09:57:06.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42100c6550 con 0x7f41fc004850 2023-10-16T09:57:06.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4206ffd700 1 -- 172.21.15.181:0/1318075573 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000f730 con 0x7f41fc004850 2023-10-16T09:57:06.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.770+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42100c6b10 con 0x7f41fc004850 2023-10-16T09:57:06.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.771+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4210067d20 con 0x7f41fc004850 2023-10-16T09:57:06.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.772+0000 7f4206ffd700 1 -- 172.21.15.181:0/1318075573 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f420000f890 con 0x7f41fc004850 2023-10-16T09:57:06.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.772+0000 7f4206ffd700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f41f406b020 0x7f41f406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:06.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.772+0000 7f4206ffd700 1 -- 172.21.15.181:0/1318075573 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f4200089f50 con 0x7f41fc004850 2023-10-16T09:57:06.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.772+0000 7f42159d1700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f41f406b020 0x7f41f406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:06.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.772+0000 7f42159d1700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f41f406b020 0x7f41f406d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f421006b530 tx=0x7f420c00a000 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:06.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.777+0000 7f4206ffd700 1 -- 172.21.15.181:0/1318075573 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4200054df0 con 0x7f41fc004850 2023-10-16T09:57:06.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.966+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 --> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42100ccb50 con 0x7f41f406b020 2023-10-16T09:57:06.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.974+0000 7f4206ffd700 1 -- 172.21.15.181:0/1318075573 <== mgr.5685 v2:172.21.15.181:6832/3219577148 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289773 (secure 0 0 0) 0x7f420c00cd50 con 0x7f41f406b020 2023-10-16T09:57:06.979 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:06.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f41f406b020 msgr2=0x7f41f406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f41f406b020 0x7f41f406d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f421006b530 tx=0x7f420c00a000 comp rx=0 tx=0).stop 2023-10-16T09:57:06.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 msgr2=0x7f421014dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:06.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f421014dbf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4200009fd0 tx=0x7f4200011d30 comp rx=0 tx=0).stop 2023-10-16T09:57:06.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 shutdown_connections 2023-10-16T09:57:06.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f41f406b020 0x7f41f406d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42100c5500 0x7f42100c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc004850 0x7f421014dbf0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 --2- 172.21.15.181:0/1318075573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421006a8a0 0x7f421006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:06.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 >> 172.21.15.181:0/1318075573 conn(0x7f42100d0460 msgr2=0x7f421015d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:06.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 shutdown_connections 2023-10-16T09:57:06.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:06.977+0000 7f4218436700 1 -- 172.21.15.181:0/1318075573 wait complete. 2023-10-16T09:57:06.984 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:57:06.991 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T09:57:06.419534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":292204,"kb_used_data":6640,"kb_used_omap":12,"kb_used_meta":285427,"kb_avail":754682516,"statfs":{"total":773094113280,"available":772794896384,"internally_reserved":0,"allocated":6799360,"data_stored":3761244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":292277816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.949265"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482733+0000","last_change":"2023-10-16T09:57:02.498783+0000","last_active":"2023-10-16T09:57:03.482733+0000","last_peered":"2023-10-16T09:57:03.482733+0000","last_clean":"2023-10-16T09:57:03.482733+0000","last_became_active":"2023-10-16T09:57:02.498490+0000","last_became_peered":"2023-10-16T09:57:02.498490+0000","last_unstale":"2023-10-16T09:57:03.482733+0000","last_undegraded":"2023-10-16T09:57:03.482733+0000","last_fullsized":"2023-10-16T09:57:03.482733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:34.135197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721330+0000","last_change":"2023-10-16T09:57:02.499533+0000","last_active":"2023-10-16T09:57:03.721330+0000","last_peered":"2023-10-16T09:57:03.721330+0000","last_clean":"2023-10-16T09:57:03.721330+0000","last_became_active":"2023-10-16T09:57:02.499186+0000","last_became_peered":"2023-10-16T09:57:02.499186+0000","last_unstale":"2023-10-16T09:57:03.721330+0000","last_undegraded":"2023-10-16T09:57:03.721330+0000","last_fullsized":"2023-10-16T09:57:03.721330+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:53.007837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496728+0000","last_change":"2023-10-16T09:57:02.491247+0000","last_active":"2023-10-16T09:57:03.496728+0000","last_peered":"2023-10-16T09:57:03.496728+0000","last_clean":"2023-10-16T09:57:03.496728+0000","last_became_active":"2023-10-16T09:57:02.491000+0000","last_became_peered":"2023-10-16T09:57:02.491000+0000","last_unstale":"2023-10-16T09:57:03.496728+0000","last_undegraded":"2023-10-16T09:57:03.496728+0000","last_fullsized":"2023-10-16T09:57:03.496728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:40:39.459389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721810+0000","last_change":"2023-10-16T09:57:02.500876+0000","last_active":"2023-10-16T09:57:03.721810+0000","last_peered":"2023-10-16T09:57:03.721810+0000","last_clean":"2023-10-16T09:57:03.721810+0000","last_became_active":"2023-10-16T09:57:02.500600+0000","last_became_peered":"2023-10-16T09:57:02.500600+0000","last_unstale":"2023-10-16T09:57:03.721810+0000","last_undegraded":"2023-10-16T09:57:03.721810+0000","last_fullsized":"2023-10-16T09:57:03.721810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:15.032313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721333+0000","last_change":"2023-10-16T09:57:02.499369+0000","last_active":"2023-10-16T09:57:03.721333+0000","last_peered":"2023-10-16T09:57:03.721333+0000","last_clean":"2023-10-16T09:57:03.721333+0000","last_became_active":"2023-10-16T09:57:02.499054+0000","last_became_peered":"2023-10-16T09:57:02.499054+0000","last_unstale":"2023-10-16T09:57:03.721333+0000","last_undegraded":"2023-10-16T09:57:03.721333+0000","last_fullsized":"2023-10-16T09:57:03.721333+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:23.723489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483356+0000","last_change":"2023-10-16T09:57:02.498533+0000","last_active":"2023-10-16T09:57:03.483356+0000","last_peered":"2023-10-16T09:57:03.483356+0000","last_clean":"2023-10-16T09:57:03.483356+0000","last_became_active":"2023-10-16T09:57:02.498250+0000","last_became_peered":"2023-10-16T09:57:02.498250+0000","last_unstale":"2023-10-16T09:57:03.483356+0000","last_undegraded":"2023-10-16T09:57:03.483356+0000","last_fullsized":"2023-10-16T09:57:03.483356+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:53.089815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482383+0000","last_change":"2023-10-16T09:57:02.506988+0000","last_active":"2023-10-16T09:57:03.482383+0000","last_peered":"2023-10-16T09:57:03.482383+0000","last_clean":"2023-10-16T09:57:03.482383+0000","last_became_active":"2023-10-16T09:57:02.506691+0000","last_became_peered":"2023-10-16T09:57:02.506691+0000","last_unstale":"2023-10-16T09:57:03.482383+0000","last_undegraded":"2023-10-16T09:57:03.482383+0000","last_fullsized":"2023-10-16T09:57:03.482383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:12.193189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721408+0000","last_change":"2023-10-16T09:57:02.496239+0000","last_active":"2023-10-16T09:57:03.721408+0000","last_peered":"2023-10-16T09:57:03.721408+0000","last_clean":"2023-10-16T09:57:03.721408+0000","last_became_active":"2023-10-16T09:57:02.495719+0000","last_became_peered":"2023-10-16T09:57:02.495719+0000","last_unstale":"2023-10-16T09:57:03.721408+0000","last_undegraded":"2023-10-16T09:57:03.721408+0000","last_fullsized":"2023-10-16T09:57:03.721408+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:15.820414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.722061+0000","last_change":"2023-10-16T09:57:02.504072+0000","last_active":"2023-10-16T09:57:03.722061+0000","last_peered":"2023-10-16T09:57:03.722061+0000","last_clean":"2023-10-16T09:57:03.722061+0000","last_became_active":"2023-10-16T09:57:02.503790+0000","last_became_peered":"2023-10-16T09:57:02.503790+0000","last_unstale":"2023-10-16T09:57:03.722061+0000","last_undegraded":"2023-10-16T09:57:03.722061+0000","last_fullsized":"2023-10-16T09:57:03.722061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:47.535638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721883+0000","last_change":"2023-10-16T09:57:02.507792+0000","last_active":"2023-10-16T09:57:03.721883+0000","last_peered":"2023-10-16T09:57:03.721883+0000","last_clean":"2023-10-16T09:57:03.721883+0000","last_became_active":"2023-10-16T09:57:02.507534+0000","last_became_peered":"2023-10-16T09:57:02.507534+0000","last_unstale":"2023-10-16T09:57:03.721883+0000","last_undegraded":"2023-10-16T09:57:03.721883+0000","last_fullsized":"2023-10-16T09:57:03.721883+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:11.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721690+0000","last_change":"2023-10-16T09:57:02.503379+0000","last_active":"2023-10-16T09:57:03.721690+0000","last_peered":"2023-10-16T09:57:03.721690+0000","last_clean":"2023-10-16T09:57:03.721690+0000","last_became_active":"2023-10-16T09:57:02.503086+0000","last_became_peered":"2023-10-16T09:57:02.503086+0000","last_unstale":"2023-10-16T09:57:03.721690+0000","last_undegraded":"2023-10-16T09:57:03.721690+0000","last_fullsized":"2023-10-16T09:57:03.721690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:31.904995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721430+0000","last_change":"2023-10-16T09:57:02.489457+0000","last_active":"2023-10-16T09:57:03.721430+0000","last_peered":"2023-10-16T09:57:03.721430+0000","last_clean":"2023-10-16T09:57:03.721430+0000","last_became_active":"2023-10-16T09:57:02.489207+0000","last_became_peered":"2023-10-16T09:57:02.489207+0000","last_unstale":"2023-10-16T09:57:03.721430+0000","last_undegraded":"2023-10-16T09:57:03.721430+0000","last_fullsized":"2023-10-16T09:57:03.721430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:06.726322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720948+0000","last_change":"2023-10-16T09:57:02.489644+0000","last_active":"2023-10-16T09:57:03.720948+0000","last_peered":"2023-10-16T09:57:03.720948+0000","last_clean":"2023-10-16T09:57:03.720948+0000","last_became_active":"2023-10-16T09:57:02.489347+0000","last_became_peered":"2023-10-16T09:57:02.489347+0000","last_unstale":"2023-10-16T09:57:03.720948+0000","last_undegraded":"2023-10-16T09:57:03.720948+0000","last_fullsized":"2023-10-16T09:57:03.720948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:34.661286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721621+0000","last_change":"2023-10-16T09:57:02.503795+0000","last_active":"2023-10-16T09:57:03.721621+0000","last_peered":"2023-10-16T09:57:03.721621+0000","last_clean":"2023-10-16T09:57:03.721621+0000","last_became_active":"2023-10-16T09:57:02.503497+0000","last_became_peered":"2023-10-16T09:57:02.503497+0000","last_unstale":"2023-10-16T09:57:03.721621+0000","last_undegraded":"2023-10-16T09:57:03.721621+0000","last_fullsized":"2023-10-16T09:57:03.721621+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:21.797779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483218+0000","last_change":"2023-10-16T09:57:02.493085+0000","last_active":"2023-10-16T09:57:03.483218+0000","last_peered":"2023-10-16T09:57:03.483218+0000","last_clean":"2023-10-16T09:57:03.483218+0000","last_became_active":"2023-10-16T09:57:02.492645+0000","last_became_peered":"2023-10-16T09:57:02.492645+0000","last_unstale":"2023-10-16T09:57:03.483218+0000","last_undegraded":"2023-10-16T09:57:03.483218+0000","last_fullsized":"2023-10-16T09:57:03.483218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:25.512535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.481788+0000","last_change":"2023-10-16T09:57:02.500796+0000","last_active":"2023-10-16T09:57:03.481788+0000","last_peered":"2023-10-16T09:57:03.481788+0000","last_clean":"2023-10-16T09:57:03.481788+0000","last_became_active":"2023-10-16T09:57:02.499965+0000","last_became_peered":"2023-10-16T09:57:02.499965+0000","last_unstale":"2023-10-16T09:57:03.481788+0000","last_undegraded":"2023-10-16T09:57:03.481788+0000","last_fullsized":"2023-10-16T09:57:03.481788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:42:34.299120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.497019+0000","last_change":"2023-10-16T09:57:02.492631+0000","last_active":"2023-10-16T09:57:03.497019+0000","last_peered":"2023-10-16T09:57:03.497019+0000","last_clean":"2023-10-16T09:57:03.497019+0000","last_became_active":"2023-10-16T09:57:02.492346+0000","last_became_peered":"2023-10-16T09:57:02.492346+0000","last_unstale":"2023-10-16T09:57:03.497019+0000","last_undegraded":"2023-10-16T09:57:03.497019+0000","last_fullsized":"2023-10-16T09:57:03.497019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:14:25.090871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496866+0000","last_change":"2023-10-16T09:57:02.502993+0000","last_active":"2023-10-16T09:57:03.496866+0000","last_peered":"2023-10-16T09:57:03.496866+0000","last_clean":"2023-10-16T09:57:03.496866+0000","last_became_active":"2023-10-16T09:57:02.501704+0000","last_became_peered":"2023-10-16T09:57:02.501704+0000","last_unstale":"2023-10-16T09:57:03.496866+0000","last_undegraded":"2023-10-16T09:57:03.496866+0000","last_fullsized":"2023-10-16T09:57:03.496866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:02.091343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482784+0000","last_change":"2023-10-16T09:57:02.500568+0000","last_active":"2023-10-16T09:57:03.482784+0000","last_peered":"2023-10-16T09:57:03.482784+0000","last_clean":"2023-10-16T09:57:03.482784+0000","last_became_active":"2023-10-16T09:57:02.500306+0000","last_became_peered":"2023-10-16T09:57:02.500306+0000","last_unstale":"2023-10-16T09:57:03.482784+0000","last_undegraded":"2023-10-16T09:57:03.482784+0000","last_fullsized":"2023-10-16T09:57:03.482784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:15.032313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720904+0000","last_change":"2023-10-16T09:57:02.503799+0000","last_active":"2023-10-16T09:57:03.720904+0000","last_peered":"2023-10-16T09:57:03.720904+0000","last_clean":"2023-10-16T09:57:03.720904+0000","last_became_active":"2023-10-16T09:57:02.503016+0000","last_became_peered":"2023-10-16T09:57:02.503016+0000","last_unstale":"2023-10-16T09:57:03.720904+0000","last_undegraded":"2023-10-16T09:57:03.720904+0000","last_fullsized":"2023-10-16T09:57:03.720904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:52.220943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483703+0000","last_change":"2023-10-16T09:57:02.495104+0000","last_active":"2023-10-16T09:57:03.483703+0000","last_peered":"2023-10-16T09:57:03.483703+0000","last_clean":"2023-10-16T09:57:03.483703+0000","last_became_active":"2023-10-16T09:57:02.494853+0000","last_became_peered":"2023-10-16T09:57:02.494853+0000","last_unstale":"2023-10-16T09:57:03.483703+0000","last_undegraded":"2023-10-16T09:57:03.483703+0000","last_fullsized":"2023-10-16T09:57:03.483703+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:51.237459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.722021+0000","last_change":"2023-10-16T09:57:02.502012+0000","last_active":"2023-10-16T09:57:03.722021+0000","last_peered":"2023-10-16T09:57:03.722021+0000","last_clean":"2023-10-16T09:57:03.722021+0000","last_became_active":"2023-10-16T09:57:02.501705+0000","last_became_peered":"2023-10-16T09:57:02.501705+0000","last_unstale":"2023-10-16T09:57:03.722021+0000","last_undegraded":"2023-10-16T09:57:03.722021+0000","last_fullsized":"2023-10-16T09:57:03.722021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:26:46.145532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482051+0000","last_change":"2023-10-16T09:57:02.502215+0000","last_active":"2023-10-16T09:57:03.482051+0000","last_peered":"2023-10-16T09:57:03.482051+0000","last_clean":"2023-10-16T09:57:03.482051+0000","last_became_active":"2023-10-16T09:57:02.501719+0000","last_became_peered":"2023-10-16T09:57:02.501719+0000","last_unstale":"2023-10-16T09:57:03.482051+0000","last_undegraded":"2023-10-16T09:57:03.482051+0000","last_fullsized":"2023-10-16T09:57:03.482051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:50.095315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482784+0000","last_change":"2023-10-16T09:57:02.497108+0000","last_active":"2023-10-16T09:57:03.482784+0000","last_peered":"2023-10-16T09:57:03.482784+0000","last_clean":"2023-10-16T09:57:03.482784+0000","last_became_active":"2023-10-16T09:57:02.495278+0000","last_became_peered":"2023-10-16T09:57:02.495278+0000","last_unstale":"2023-10-16T09:57:03.482784+0000","last_undegraded":"2023-10-16T09:57:03.482784+0000","last_fullsized":"2023-10-16T09:57:03.482784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:17.554907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482668+0000","last_change":"2023-10-16T09:57:02.502373+0000","last_active":"2023-10-16T09:57:03.482668+0000","last_peered":"2023-10-16T09:57:03.482668+0000","last_clean":"2023-10-16T09:57:03.482668+0000","last_became_active":"2023-10-16T09:57:02.502070+0000","last_became_peered":"2023-10-16T09:57:02.502070+0000","last_unstale":"2023-10-16T09:57:03.482668+0000","last_undegraded":"2023-10-16T09:57:03.482668+0000","last_fullsized":"2023-10-16T09:57:03.482668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:05.838939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720649+0000","last_change":"2023-10-16T09:57:02.490211+0000","last_active":"2023-10-16T09:57:03.720649+0000","last_peered":"2023-10-16T09:57:03.720649+0000","last_clean":"2023-10-16T09:57:03.720649+0000","last_became_active":"2023-10-16T09:57:02.489933+0000","last_became_peered":"2023-10-16T09:57:02.489933+0000","last_unstale":"2023-10-16T09:57:03.720649+0000","last_undegraded":"2023-10-16T09:57:03.720649+0000","last_fullsized":"2023-10-16T09:57:03.720649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:32.183411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496788+0000","last_change":"2023-10-16T09:57:02.491747+0000","last_active":"2023-10-16T09:57:03.496788+0000","last_peered":"2023-10-16T09:57:03.496788+0000","last_clean":"2023-10-16T09:57:03.496788+0000","last_became_active":"2023-10-16T09:57:02.491440+0000","last_became_peered":"2023-10-16T09:57:02.491440+0000","last_unstale":"2023-10-16T09:57:03.496788+0000","last_undegraded":"2023-10-16T09:57:03.496788+0000","last_fullsized":"2023-10-16T09:57:03.496788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:18.356265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721278+0000","last_change":"2023-10-16T09:57:02.504386+0000","last_active":"2023-10-16T09:57:03.721278+0000","last_peered":"2023-10-16T09:57:03.721278+0000","last_clean":"2023-10-16T09:57:03.721278+0000","last_became_active":"2023-10-16T09:57:02.504151+0000","last_became_peered":"2023-10-16T09:57:02.504151+0000","last_unstale":"2023-10-16T09:57:03.721278+0000","last_undegraded":"2023-10-16T09:57:03.721278+0000","last_fullsized":"2023-10-16T09:57:03.721278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:08.491805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482142+0000","last_change":"2023-10-16T09:57:02.502230+0000","last_active":"2023-10-16T09:57:03.482142+0000","last_peered":"2023-10-16T09:57:03.482142+0000","last_clean":"2023-10-16T09:57:03.482142+0000","last_became_active":"2023-10-16T09:57:02.501915+0000","last_became_peered":"2023-10-16T09:57:02.501915+0000","last_unstale":"2023-10-16T09:57:03.482142+0000","last_undegraded":"2023-10-16T09:57:03.482142+0000","last_fullsized":"2023-10-16T09:57:03.482142+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:15.820414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496668+0000","last_change":"2023-10-16T09:57:02.494785+0000","last_active":"2023-10-16T09:57:03.496668+0000","last_peered":"2023-10-16T09:57:03.496668+0000","last_clean":"2023-10-16T09:57:03.496668+0000","last_became_active":"2023-10-16T09:57:02.494513+0000","last_became_peered":"2023-10-16T09:57:02.494513+0000","last_unstale":"2023-10-16T09:57:03.496668+0000","last_undegraded":"2023-10-16T09:57:03.496668+0000","last_fullsized":"2023-10-16T09:57:03.496668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:07:19.933800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483611+0000","last_change":"2023-10-16T09:57:02.485713+0000","last_active":"2023-10-16T09:57:03.483611+0000","last_peered":"2023-10-16T09:57:03.483611+0000","last_clean":"2023-10-16T09:57:03.483611+0000","last_became_active":"2023-10-16T09:57:02.485411+0000","last_became_peered":"2023-10-16T09:57:02.485411+0000","last_unstale":"2023-10-16T09:57:03.483611+0000","last_undegraded":"2023-10-16T09:57:03.483611+0000","last_fullsized":"2023-10-16T09:57:03.483611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:38:43.271549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720777+0000","last_change":"2023-10-16T09:57:02.504167+0000","last_active":"2023-10-16T09:57:03.720777+0000","last_peered":"2023-10-16T09:57:03.720777+0000","last_clean":"2023-10-16T09:57:03.720777+0000","last_became_active":"2023-10-16T09:57:02.501615+0000","last_became_peered":"2023-10-16T09:57:02.501615+0000","last_unstale":"2023-10-16T09:57:03.720777+0000","last_undegraded":"2023-10-16T09:57:03.720777+0000","last_fullsized":"2023-10-16T09:57:03.720777+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:34.466216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720551+0000","last_change":"2023-10-16T09:57:02.500624+0000","last_active":"2023-10-16T09:57:03.720551+0000","last_peered":"2023-10-16T09:57:03.720551+0000","last_clean":"2023-10-16T09:57:03.720551+0000","last_became_active":"2023-10-16T09:57:02.499796+0000","last_became_peered":"2023-10-16T09:57:02.499796+0000","last_unstale":"2023-10-16T09:57:03.720551+0000","last_undegraded":"2023-10-16T09:57:03.720551+0000","last_fullsized":"2023-10-16T09:57:03.720551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:01.654953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"58'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483485+0000","last_change":"2023-10-16T09:57:01.497614+0000","last_active":"2023-10-16T09:57:03.483485+0000","last_peered":"2023-10-16T09:57:03.483485+0000","last_clean":"2023-10-16T09:57:03.483485+0000","last_became_active":"2023-10-16T09:57:01.497299+0000","last_became_peered":"2023-10-16T09:57:01.497299+0000","last_unstale":"2023-10-16T09:57:03.483485+0000","last_undegraded":"2023-10-16T09:57:03.483485+0000","last_fullsized":"2023-10-16T09:57:03.483485+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:19:51.002135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482466+0000","last_change":"2023-10-16T09:57:02.496202+0000","last_active":"2023-10-16T09:57:03.482466+0000","last_peered":"2023-10-16T09:57:03.482466+0000","last_clean":"2023-10-16T09:57:03.482466+0000","last_became_active":"2023-10-16T09:57:02.495908+0000","last_became_peered":"2023-10-16T09:57:02.495908+0000","last_unstale":"2023-10-16T09:57:03.482466+0000","last_undegraded":"2023-10-16T09:57:03.482466+0000","last_fullsized":"2023-10-16T09:57:03.482466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:21.797779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496940+0000","last_change":"2023-10-16T09:57:01.486358+0000","last_active":"2023-10-16T09:57:03.496940+0000","last_peered":"2023-10-16T09:57:03.496940+0000","last_clean":"2023-10-16T09:57:03.496940+0000","last_became_active":"2023-10-16T09:57:01.485889+0000","last_became_peered":"2023-10-16T09:57:01.485889+0000","last_unstale":"2023-10-16T09:57:03.496940+0000","last_undegraded":"2023-10-16T09:57:03.496940+0000","last_fullsized":"2023-10-16T09:57:03.496940+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:23:23.647317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721267+0000","last_change":"2023-10-16T09:57:02.485785+0000","last_active":"2023-10-16T09:57:03.721267+0000","last_peered":"2023-10-16T09:57:03.721267+0000","last_clean":"2023-10-16T09:57:03.721267+0000","last_became_active":"2023-10-16T09:57:02.485507+0000","last_became_peered":"2023-10-16T09:57:02.485507+0000","last_unstale":"2023-10-16T09:57:03.721267+0000","last_undegraded":"2023-10-16T09:57:03.721267+0000","last_fullsized":"2023-10-16T09:57:03.721267+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:29.641894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720892+0000","last_change":"2023-10-16T09:57:01.495721+0000","last_active":"2023-10-16T09:57:03.720892+0000","last_peered":"2023-10-16T09:57:03.720892+0000","last_clean":"2023-10-16T09:57:03.720892+0000","last_became_active":"2023-10-16T09:57:01.495458+0000","last_became_peered":"2023-10-16T09:57:01.495458+0000","last_unstale":"2023-10-16T09:57:03.720892+0000","last_undegraded":"2023-10-16T09:57:03.720892+0000","last_fullsized":"2023-10-16T09:57:03.720892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:31.333631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720803+0000","last_change":"2023-10-16T09:57:02.507754+0000","last_active":"2023-10-16T09:57:03.720803+0000","last_peered":"2023-10-16T09:57:03.720803+0000","last_clean":"2023-10-16T09:57:03.720803+0000","last_became_active":"2023-10-16T09:57:02.507504+0000","last_became_peered":"2023-10-16T09:57:02.507504+0000","last_unstale":"2023-10-16T09:57:03.720803+0000","last_undegraded":"2023-10-16T09:57:03.720803+0000","last_fullsized":"2023-10-16T09:57:03.720803+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:38:43.919266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.481851+0000","last_change":"2023-10-16T09:57:01.493487+0000","last_active":"2023-10-16T09:57:03.481851+0000","last_peered":"2023-10-16T09:57:03.481851+0000","last_clean":"2023-10-16T09:57:03.481851+0000","last_became_active":"2023-10-16T09:57:01.493244+0000","last_became_peered":"2023-10-16T09:57:01.493244+0000","last_unstale":"2023-10-16T09:57:03.481851+0000","last_undegraded":"2023-10-16T09:57:03.481851+0000","last_fullsized":"2023-10-16T09:57:03.481851+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:22.716457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496658+0000","last_change":"2023-10-16T09:57:02.495317+0000","last_active":"2023-10-16T09:57:03.496658+0000","last_peered":"2023-10-16T09:57:03.496658+0000","last_clean":"2023-10-16T09:57:03.496658+0000","last_became_active":"2023-10-16T09:57:02.495069+0000","last_became_peered":"2023-10-16T09:57:02.495069+0000","last_unstale":"2023-10-16T09:57:03.496658+0000","last_undegraded":"2023-10-16T09:57:03.496658+0000","last_fullsized":"2023-10-16T09:57:03.496658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:14.899883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483828+0000","last_change":"2023-10-16T09:57:01.499386+0000","last_active":"2023-10-16T09:57:03.483828+0000","last_peered":"2023-10-16T09:57:03.483828+0000","last_clean":"2023-10-16T09:57:03.483828+0000","last_became_active":"2023-10-16T09:57:01.499050+0000","last_became_peered":"2023-10-16T09:57:01.499050+0000","last_unstale":"2023-10-16T09:57:03.483828+0000","last_undegraded":"2023-10-16T09:57:03.483828+0000","last_fullsized":"2023-10-16T09:57:03.483828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:39:24.013149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721728+0000","last_change":"2023-10-16T09:57:02.503892+0000","last_active":"2023-10-16T09:57:03.721728+0000","last_peered":"2023-10-16T09:57:03.721728+0000","last_clean":"2023-10-16T09:57:03.721728+0000","last_became_active":"2023-10-16T09:57:02.503612+0000","last_became_peered":"2023-10-16T09:57:02.503612+0000","last_unstale":"2023-10-16T09:57:03.721728+0000","last_undegraded":"2023-10-16T09:57:03.721728+0000","last_fullsized":"2023-10-16T09:57:03.721728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:18:05.838939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496792+0000","last_change":"2023-10-16T09:57:01.490574+0000","last_active":"2023-10-16T09:57:03.496792+0000","last_peered":"2023-10-16T09:57:03.496792+0000","last_clean":"2023-10-16T09:57:03.496792+0000","last_became_active":"2023-10-16T09:57:01.490282+0000","last_became_peered":"2023-10-16T09:57:01.490282+0000","last_unstale":"2023-10-16T09:57:03.496792+0000","last_undegraded":"2023-10-16T09:57:03.496792+0000","last_fullsized":"2023-10-16T09:57:03.496792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:12:51.430819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721675+0000","last_change":"2023-10-16T09:57:02.495657+0000","last_active":"2023-10-16T09:57:03.721675+0000","last_peered":"2023-10-16T09:57:03.721675+0000","last_clean":"2023-10-16T09:57:03.721675+0000","last_became_active":"2023-10-16T09:57:02.495387+0000","last_became_peered":"2023-10-16T09:57:02.495387+0000","last_unstale":"2023-10-16T09:57:03.721675+0000","last_undegraded":"2023-10-16T09:57:03.721675+0000","last_fullsized":"2023-10-16T09:57:03.721675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:58.467074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721039+0000","last_change":"2023-10-16T09:57:01.495934+0000","last_active":"2023-10-16T09:57:03.721039+0000","last_peered":"2023-10-16T09:57:03.721039+0000","last_clean":"2023-10-16T09:57:03.721039+0000","last_became_active":"2023-10-16T09:57:01.495715+0000","last_became_peered":"2023-10-16T09:57:01.495715+0000","last_unstale":"2023-10-16T09:57:03.721039+0000","last_undegraded":"2023-10-16T09:57:03.721039+0000","last_fullsized":"2023-10-16T09:57:03.721039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:59.481940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482081+0000","last_change":"2023-10-16T09:57:02.502453+0000","last_active":"2023-10-16T09:57:03.482081+0000","last_peered":"2023-10-16T09:57:03.482081+0000","last_clean":"2023-10-16T09:57:03.482081+0000","last_became_active":"2023-10-16T09:57:02.502114+0000","last_became_peered":"2023-10-16T09:57:02.502114+0000","last_unstale":"2023-10-16T09:57:03.482081+0000","last_undegraded":"2023-10-16T09:57:03.482081+0000","last_fullsized":"2023-10-16T09:57:03.482081+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:15.035648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721520+0000","last_change":"2023-10-16T09:57:01.494248+0000","last_active":"2023-10-16T09:57:03.721520+0000","last_peered":"2023-10-16T09:57:03.721520+0000","last_clean":"2023-10-16T09:57:03.721520+0000","last_became_active":"2023-10-16T09:57:01.493562+0000","last_became_peered":"2023-10-16T09:57:01.493562+0000","last_unstale":"2023-10-16T09:57:03.721520+0000","last_undegraded":"2023-10-16T09:57:03.721520+0000","last_fullsized":"2023-10-16T09:57:03.721520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:49.088283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496951+0000","last_change":"2023-10-16T09:57:02.501979+0000","last_active":"2023-10-16T09:57:03.496951+0000","last_peered":"2023-10-16T09:57:03.496951+0000","last_clean":"2023-10-16T09:57:03.496951+0000","last_became_active":"2023-10-16T09:57:02.501578+0000","last_became_peered":"2023-10-16T09:57:02.501578+0000","last_unstale":"2023-10-16T09:57:03.496951+0000","last_undegraded":"2023-10-16T09:57:03.496951+0000","last_fullsized":"2023-10-16T09:57:03.496951+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:02:51.464571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496995+0000","last_change":"2023-10-16T09:57:01.484026+0000","last_active":"2023-10-16T09:57:03.496995+0000","last_peered":"2023-10-16T09:57:03.496995+0000","last_clean":"2023-10-16T09:57:03.496995+0000","last_became_active":"2023-10-16T09:57:01.482945+0000","last_became_peered":"2023-10-16T09:57:01.482945+0000","last_unstale":"2023-10-16T09:57:03.496995+0000","last_undegraded":"2023-10-16T09:57:03.496995+0000","last_fullsized":"2023-10-16T09:57:03.496995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:05:51.952699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482581+0000","last_change":"2023-10-16T09:57:02.501470+0000","last_active":"2023-10-16T09:57:03.482581+0000","last_peered":"2023-10-16T09:57:03.482581+0000","last_clean":"2023-10-16T09:57:03.482581+0000","last_became_active":"2023-10-16T09:57:02.501197+0000","last_became_peered":"2023-10-16T09:57:02.501197+0000","last_unstale":"2023-10-16T09:57:03.482581+0000","last_undegraded":"2023-10-16T09:57:03.482581+0000","last_fullsized":"2023-10-16T09:57:03.482581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:13:25.472249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721955+0000","last_change":"2023-10-16T09:57:01.498263+0000","last_active":"2023-10-16T09:57:03.721955+0000","last_peered":"2023-10-16T09:57:03.721955+0000","last_clean":"2023-10-16T09:57:03.721955+0000","last_became_active":"2023-10-16T09:57:01.497972+0000","last_became_peered":"2023-10-16T09:57:01.497972+0000","last_unstale":"2023-10-16T09:57:03.721955+0000","last_undegraded":"2023-10-16T09:57:03.721955+0000","last_fullsized":"2023-10-16T09:57:03.721955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:14.028616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483607+0000","last_change":"2023-10-16T09:57:02.504550+0000","last_active":"2023-10-16T09:57:03.483607+0000","last_peered":"2023-10-16T09:57:03.483607+0000","last_clean":"2023-10-16T09:57:03.483607+0000","last_became_active":"2023-10-16T09:57:02.504265+0000","last_became_peered":"2023-10-16T09:57:02.504265+0000","last_unstale":"2023-10-16T09:57:03.483607+0000","last_undegraded":"2023-10-16T09:57:03.483607+0000","last_fullsized":"2023-10-16T09:57:03.483607+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:21.238124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"58'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482512+0000","last_change":"2023-10-16T09:57:01.492222+0000","last_active":"2023-10-16T09:57:03.482512+0000","last_peered":"2023-10-16T09:57:03.482512+0000","last_clean":"2023-10-16T09:57:03.482512+0000","last_became_active":"2023-10-16T09:57:01.492007+0000","last_became_peered":"2023-10-16T09:57:01.492007+0000","last_unstale":"2023-10-16T09:57:03.482512+0000","last_undegraded":"2023-10-16T09:57:03.482512+0000","last_fullsized":"2023-10-16T09:57:03.482512+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:38.639067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482242+0000","last_change":"2023-10-16T09:57:02.508665+0000","last_active":"2023-10-16T09:57:03.482242+0000","last_peered":"2023-10-16T09:57:03.482242+0000","last_clean":"2023-10-16T09:57:03.482242+0000","last_became_active":"2023-10-16T09:57:02.508407+0000","last_became_peered":"2023-10-16T09:57:02.508407+0000","last_unstale":"2023-10-16T09:57:03.482242+0000","last_undegraded":"2023-10-16T09:57:03.482242+0000","last_fullsized":"2023-10-16T09:57:03.482242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:44.151216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"58'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720942+0000","last_change":"2023-10-16T09:57:01.483356+0000","last_active":"2023-10-16T09:57:03.720942+0000","last_peered":"2023-10-16T09:57:03.720942+0000","last_clean":"2023-10-16T09:57:03.720942+0000","last_became_active":"2023-10-16T09:57:01.483076+0000","last_became_peered":"2023-10-16T09:57:01.483076+0000","last_unstale":"2023-10-16T09:57:03.720942+0000","last_undegraded":"2023-10-16T09:57:03.720942+0000","last_fullsized":"2023-10-16T09:57:03.720942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:25:49.254281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482688+0000","last_change":"2023-10-16T09:57:02.497819+0000","last_active":"2023-10-16T09:57:03.482688+0000","last_peered":"2023-10-16T09:57:03.482688+0000","last_clean":"2023-10-16T09:57:03.482688+0000","last_became_active":"2023-10-16T09:57:02.497541+0000","last_became_peered":"2023-10-16T09:57:02.497541+0000","last_unstale":"2023-10-16T09:57:03.482688+0000","last_undegraded":"2023-10-16T09:57:03.482688+0000","last_fullsized":"2023-10-16T09:57:03.482688+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:58.467074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483872+0000","last_change":"2023-10-16T09:57:01.497404+0000","last_active":"2023-10-16T09:57:03.483872+0000","last_peered":"2023-10-16T09:57:03.483872+0000","last_clean":"2023-10-16T09:57:03.483872+0000","last_became_active":"2023-10-16T09:57:01.497106+0000","last_became_peered":"2023-10-16T09:57:01.497106+0000","last_unstale":"2023-10-16T09:57:03.483872+0000","last_undegraded":"2023-10-16T09:57:03.483872+0000","last_fullsized":"2023-10-16T09:57:03.483872+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:22:29.462828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721870+0000","last_change":"2023-10-16T09:57:02.506387+0000","last_active":"2023-10-16T09:57:03.721870+0000","last_peered":"2023-10-16T09:57:03.721870+0000","last_clean":"2023-10-16T09:57:03.721870+0000","last_became_active":"2023-10-16T09:57:02.505679+0000","last_became_peered":"2023-10-16T09:57:02.505679+0000","last_unstale":"2023-10-16T09:57:03.721870+0000","last_undegraded":"2023-10-16T09:57:03.721870+0000","last_fullsized":"2023-10-16T09:57:03.721870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:32.457071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482278+0000","last_change":"2023-10-16T09:57:01.489882+0000","last_active":"2023-10-16T09:57:03.482278+0000","last_peered":"2023-10-16T09:57:03.482278+0000","last_clean":"2023-10-16T09:57:03.482278+0000","last_became_active":"2023-10-16T09:57:01.489594+0000","last_became_peered":"2023-10-16T09:57:01.489594+0000","last_unstale":"2023-10-16T09:57:03.482278+0000","last_undegraded":"2023-10-16T09:57:03.482278+0000","last_fullsized":"2023-10-16T09:57:03.482278+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:28.634862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483210+0000","last_change":"2023-10-16T09:57:02.498475+0000","last_active":"2023-10-16T09:57:03.483210+0000","last_peered":"2023-10-16T09:57:03.483210+0000","last_clean":"2023-10-16T09:57:03.483210+0000","last_became_active":"2023-10-16T09:57:02.498069+0000","last_became_peered":"2023-10-16T09:57:02.498069+0000","last_unstale":"2023-10-16T09:57:03.483210+0000","last_undegraded":"2023-10-16T09:57:03.483210+0000","last_fullsized":"2023-10-16T09:57:03.483210+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:34.135197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483304+0000","last_change":"2023-10-16T09:57:01.497381+0000","last_active":"2023-10-16T09:57:03.483304+0000","last_peered":"2023-10-16T09:57:03.483304+0000","last_clean":"2023-10-16T09:57:03.483304+0000","last_became_active":"2023-10-16T09:57:01.497088+0000","last_became_peered":"2023-10-16T09:57:01.497088+0000","last_unstale":"2023-10-16T09:57:03.483304+0000","last_undegraded":"2023-10-16T09:57:03.483304+0000","last_fullsized":"2023-10-16T09:57:03.483304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:43.144184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482621+0000","last_change":"2023-10-16T09:57:02.508912+0000","last_active":"2023-10-16T09:57:03.482621+0000","last_peered":"2023-10-16T09:57:03.482621+0000","last_clean":"2023-10-16T09:57:03.482621+0000","last_became_active":"2023-10-16T09:57:02.508637+0000","last_became_peered":"2023-10-16T09:57:02.508637+0000","last_unstale":"2023-10-16T09:57:03.482621+0000","last_undegraded":"2023-10-16T09:57:03.482621+0000","last_fullsized":"2023-10-16T09:57:03.482621+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:39.727242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"58'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721435+0000","last_change":"2023-10-16T09:57:01.494437+0000","last_active":"2023-10-16T09:57:03.721435+0000","last_peered":"2023-10-16T09:57:03.721435+0000","last_clean":"2023-10-16T09:57:03.721435+0000","last_became_active":"2023-10-16T09:57:01.493953+0000","last_became_peered":"2023-10-16T09:57:01.493953+0000","last_unstale":"2023-10-16T09:57:03.721435+0000","last_undegraded":"2023-10-16T09:57:03.721435+0000","last_fullsized":"2023-10-16T09:57:03.721435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:36.973274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720891+0000","last_change":"2023-10-16T09:57:02.497058+0000","last_active":"2023-10-16T09:57:03.720891+0000","last_peered":"2023-10-16T09:57:03.720891+0000","last_clean":"2023-10-16T09:57:03.720891+0000","last_became_active":"2023-10-16T09:57:02.496778+0000","last_became_peered":"2023-10-16T09:57:02.496778+0000","last_unstale":"2023-10-16T09:57:03.720891+0000","last_undegraded":"2023-10-16T09:57:03.720891+0000","last_fullsized":"2023-10-16T09:57:03.720891+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:15.708521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720831+0000","last_change":"2023-10-16T09:57:01.496294+0000","last_active":"2023-10-16T09:57:03.720831+0000","last_peered":"2023-10-16T09:57:03.720831+0000","last_clean":"2023-10-16T09:57:03.720831+0000","last_became_active":"2023-10-16T09:57:01.495974+0000","last_became_peered":"2023-10-16T09:57:01.495974+0000","last_unstale":"2023-10-16T09:57:03.720831+0000","last_undegraded":"2023-10-16T09:57:03.720831+0000","last_fullsized":"2023-10-16T09:57:03.720831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:31.825080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.722154+0000","last_change":"2023-10-16T09:57:02.496707+0000","last_active":"2023-10-16T09:57:03.722154+0000","last_peered":"2023-10-16T09:57:03.722154+0000","last_clean":"2023-10-16T09:57:03.722154+0000","last_became_active":"2023-10-16T09:57:02.496407+0000","last_became_peered":"2023-10-16T09:57:02.496407+0000","last_unstale":"2023-10-16T09:57:03.722154+0000","last_undegraded":"2023-10-16T09:57:03.722154+0000","last_fullsized":"2023-10-16T09:57:03.722154+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:34.650018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496831+0000","last_change":"2023-10-16T09:57:01.486168+0000","last_active":"2023-10-16T09:57:03.496831+0000","last_peered":"2023-10-16T09:57:03.496831+0000","last_clean":"2023-10-16T09:57:03.496831+0000","last_became_active":"2023-10-16T09:57:01.485528+0000","last_became_peered":"2023-10-16T09:57:01.485528+0000","last_unstale":"2023-10-16T09:57:03.496831+0000","last_undegraded":"2023-10-16T09:57:03.496831+0000","last_fullsized":"2023-10-16T09:57:03.496831+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:00.263476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482761+0000","last_change":"2023-10-16T09:57:02.495682+0000","last_active":"2023-10-16T09:57:03.482761+0000","last_peered":"2023-10-16T09:57:03.482761+0000","last_clean":"2023-10-16T09:57:03.482761+0000","last_became_active":"2023-10-16T09:57:02.495092+0000","last_became_peered":"2023-10-16T09:57:02.495092+0000","last_unstale":"2023-10-16T09:57:03.482761+0000","last_undegraded":"2023-10-16T09:57:03.482761+0000","last_fullsized":"2023-10-16T09:57:03.482761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:38.272845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721224+0000","last_change":"2023-10-16T09:57:01.498949+0000","last_active":"2023-10-16T09:57:03.721224+0000","last_peered":"2023-10-16T09:57:03.721224+0000","last_clean":"2023-10-16T09:57:03.721224+0000","last_became_active":"2023-10-16T09:57:01.498663+0000","last_became_peered":"2023-10-16T09:57:01.498663+0000","last_unstale":"2023-10-16T09:57:03.721224+0000","last_undegraded":"2023-10-16T09:57:03.721224+0000","last_fullsized":"2023-10-16T09:57:03.721224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:19.054392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482917+0000","last_change":"2023-10-16T09:57:02.503089+0000","last_active":"2023-10-16T09:57:03.482917+0000","last_peered":"2023-10-16T09:57:03.482917+0000","last_clean":"2023-10-16T09:57:03.482917+0000","last_became_active":"2023-10-16T09:57:02.502819+0000","last_became_peered":"2023-10-16T09:57:02.502819+0000","last_unstale":"2023-10-16T09:57:03.482917+0000","last_undegraded":"2023-10-16T09:57:03.482917+0000","last_fullsized":"2023-10-16T09:57:03.482917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:32.457071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483679+0000","last_change":"2023-10-16T09:57:01.498498+0000","last_active":"2023-10-16T09:57:03.483679+0000","last_peered":"2023-10-16T09:57:03.483679+0000","last_clean":"2023-10-16T09:57:03.483679+0000","last_became_active":"2023-10-16T09:57:01.498188+0000","last_became_peered":"2023-10-16T09:57:01.498188+0000","last_unstale":"2023-10-16T09:57:03.483679+0000","last_undegraded":"2023-10-16T09:57:03.483679+0000","last_fullsized":"2023-10-16T09:57:03.483679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:22.365919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483590+0000","last_change":"2023-10-16T09:57:02.490682+0000","last_active":"2023-10-16T09:57:03.483590+0000","last_peered":"2023-10-16T09:57:03.483590+0000","last_clean":"2023-10-16T09:57:03.483590+0000","last_became_active":"2023-10-16T09:57:02.490328+0000","last_became_peered":"2023-10-16T09:57:02.490328+0000","last_unstale":"2023-10-16T09:57:03.483590+0000","last_undegraded":"2023-10-16T09:57:03.483590+0000","last_fullsized":"2023-10-16T09:57:03.483590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:32.643920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483935+0000","last_change":"2023-10-16T09:57:01.499258+0000","last_active":"2023-10-16T09:57:03.483935+0000","last_peered":"2023-10-16T09:57:03.483935+0000","last_clean":"2023-10-16T09:57:03.483935+0000","last_became_active":"2023-10-16T09:57:01.498677+0000","last_became_peered":"2023-10-16T09:57:01.498677+0000","last_unstale":"2023-10-16T09:57:03.483935+0000","last_undegraded":"2023-10-16T09:57:03.483935+0000","last_fullsized":"2023-10-16T09:57:03.483935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:27.428129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720966+0000","last_change":"2023-10-16T09:57:02.499562+0000","last_active":"2023-10-16T09:57:03.720966+0000","last_peered":"2023-10-16T09:57:03.720966+0000","last_clean":"2023-10-16T09:57:03.720966+0000","last_became_active":"2023-10-16T09:57:02.499275+0000","last_became_peered":"2023-10-16T09:57:02.499275+0000","last_unstale":"2023-10-16T09:57:03.720966+0000","last_undegraded":"2023-10-16T09:57:03.720966+0000","last_fullsized":"2023-10-16T09:57:03.720966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:42.435404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721012+0000","last_change":"2023-10-16T09:57:01.489676+0000","last_active":"2023-10-16T09:57:03.721012+0000","last_peered":"2023-10-16T09:57:03.721012+0000","last_clean":"2023-10-16T09:57:03.721012+0000","last_became_active":"2023-10-16T09:57:01.489323+0000","last_became_peered":"2023-10-16T09:57:01.489323+0000","last_unstale":"2023-10-16T09:57:03.721012+0000","last_undegraded":"2023-10-16T09:57:03.721012+0000","last_fullsized":"2023-10-16T09:57:03.721012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:41.717699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720868+0000","last_change":"2023-10-16T09:57:02.497905+0000","last_active":"2023-10-16T09:57:03.720868+0000","last_peered":"2023-10-16T09:57:03.720868+0000","last_clean":"2023-10-16T09:57:03.720868+0000","last_became_active":"2023-10-16T09:57:02.497217+0000","last_became_peered":"2023-10-16T09:57:02.497217+0000","last_unstale":"2023-10-16T09:57:03.720868+0000","last_undegraded":"2023-10-16T09:57:03.720868+0000","last_fullsized":"2023-10-16T09:57:03.720868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:32.070606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721434+0000","last_change":"2023-10-16T09:57:01.498404+0000","last_active":"2023-10-16T09:57:03.721434+0000","last_peered":"2023-10-16T09:57:03.721434+0000","last_clean":"2023-10-16T09:57:03.721434+0000","last_became_active":"2023-10-16T09:57:01.497919+0000","last_became_peered":"2023-10-16T09:57:01.497919+0000","last_unstale":"2023-10-16T09:57:03.721434+0000","last_undegraded":"2023-10-16T09:57:03.721434+0000","last_fullsized":"2023-10-16T09:57:03.721434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:44.665735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721445+0000","last_change":"2023-10-16T09:57:02.502769+0000","last_active":"2023-10-16T09:57:03.721445+0000","last_peered":"2023-10-16T09:57:03.721445+0000","last_clean":"2023-10-16T09:57:03.721445+0000","last_became_active":"2023-10-16T09:57:02.502474+0000","last_became_peered":"2023-10-16T09:57:02.502474+0000","last_unstale":"2023-10-16T09:57:03.721445+0000","last_undegraded":"2023-10-16T09:57:03.721445+0000","last_fullsized":"2023-10-16T09:57:03.721445+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:38.272845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.481707+0000","last_change":"2023-10-16T09:57:01.494332+0000","last_active":"2023-10-16T09:57:03.481707+0000","last_peered":"2023-10-16T09:57:03.481707+0000","last_clean":"2023-10-16T09:57:03.481707+0000","last_became_active":"2023-10-16T09:57:01.494040+0000","last_became_peered":"2023-10-16T09:57:01.494040+0000","last_unstale":"2023-10-16T09:57:03.481707+0000","last_undegraded":"2023-10-16T09:57:03.481707+0000","last_fullsized":"2023-10-16T09:57:03.481707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:09.030855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482595+0000","last_change":"2023-10-16T09:57:02.497941+0000","last_active":"2023-10-16T09:57:03.482595+0000","last_peered":"2023-10-16T09:57:03.482595+0000","last_clean":"2023-10-16T09:57:03.482595+0000","last_became_active":"2023-10-16T09:57:02.497503+0000","last_became_peered":"2023-10-16T09:57:02.497503+0000","last_unstale":"2023-10-16T09:57:03.482595+0000","last_undegraded":"2023-10-16T09:57:03.482595+0000","last_fullsized":"2023-10-16T09:57:03.482595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:31.904995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483441+0000","last_change":"2023-10-16T09:57:01.496444+0000","last_active":"2023-10-16T09:57:03.483441+0000","last_peered":"2023-10-16T09:57:03.483441+0000","last_clean":"2023-10-16T09:57:03.483441+0000","last_became_active":"2023-10-16T09:57:01.496156+0000","last_became_peered":"2023-10-16T09:57:01.496156+0000","last_unstale":"2023-10-16T09:57:03.483441+0000","last_undegraded":"2023-10-16T09:57:03.483441+0000","last_fullsized":"2023-10-16T09:57:03.483441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:57:08.056788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721752+0000","last_change":"2023-10-16T09:57:02.496832+0000","last_active":"2023-10-16T09:57:03.721752+0000","last_peered":"2023-10-16T09:57:03.721752+0000","last_clean":"2023-10-16T09:57:03.721752+0000","last_became_active":"2023-10-16T09:57:02.496563+0000","last_became_peered":"2023-10-16T09:57:02.496563+0000","last_unstale":"2023-10-16T09:57:03.721752+0000","last_undegraded":"2023-10-16T09:57:03.721752+0000","last_fullsized":"2023-10-16T09:57:03.721752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:17.554907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482602+0000","last_change":"2023-10-16T09:57:01.492346+0000","last_active":"2023-10-16T09:57:03.482602+0000","last_peered":"2023-10-16T09:57:03.482602+0000","last_clean":"2023-10-16T09:57:03.482602+0000","last_became_active":"2023-10-16T09:57:01.492119+0000","last_became_peered":"2023-10-16T09:57:01.492119+0000","last_unstale":"2023-10-16T09:57:03.482602+0000","last_undegraded":"2023-10-16T09:57:03.482602+0000","last_fullsized":"2023-10-16T09:57:03.482602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:53:46.730975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482413+0000","last_change":"2023-10-16T09:57:02.504229+0000","last_active":"2023-10-16T09:57:03.482413+0000","last_peered":"2023-10-16T09:57:03.482413+0000","last_clean":"2023-10-16T09:57:03.482413+0000","last_became_active":"2023-10-16T09:57:02.503617+0000","last_became_peered":"2023-10-16T09:57:02.503617+0000","last_unstale":"2023-10-16T09:57:03.482413+0000","last_undegraded":"2023-10-16T09:57:03.482413+0000","last_fullsized":"2023-10-16T09:57:03.482413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:57:09.063820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720492+0000","last_change":"2023-10-16T09:57:01.495576+0000","last_active":"2023-10-16T09:57:03.720492+0000","last_peered":"2023-10-16T09:57:03.720492+0000","last_clean":"2023-10-16T09:57:03.720492+0000","last_became_active":"2023-10-16T09:57:01.495343+0000","last_became_peered":"2023-10-16T09:57:01.495343+0000","last_unstale":"2023-10-16T09:57:03.720492+0000","last_undegraded":"2023-10-16T09:57:03.720492+0000","last_fullsized":"2023-10-16T09:57:03.720492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:47.083472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483475+0000","last_change":"2023-10-16T09:57:02.489394+0000","last_active":"2023-10-16T09:57:03.483475+0000","last_peered":"2023-10-16T09:57:03.483475+0000","last_clean":"2023-10-16T09:57:03.483475+0000","last_became_active":"2023-10-16T09:57:02.486244+0000","last_became_peered":"2023-10-16T09:57:02.486244+0000","last_unstale":"2023-10-16T09:57:03.483475+0000","last_undegraded":"2023-10-16T09:57:03.483475+0000","last_fullsized":"2023-10-16T09:57:03.483475+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:15.392064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.482130+0000","last_change":"2023-10-16T09:57:01.488437+0000","last_active":"2023-10-16T09:57:03.482130+0000","last_peered":"2023-10-16T09:57:03.482130+0000","last_clean":"2023-10-16T09:57:03.482130+0000","last_became_active":"2023-10-16T09:57:01.488143+0000","last_became_peered":"2023-10-16T09:57:01.488143+0000","last_unstale":"2023-10-16T09:57:03.482130+0000","last_undegraded":"2023-10-16T09:57:03.482130+0000","last_fullsized":"2023-10-16T09:57:03.482130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:07.484773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.483785+0000","last_change":"2023-10-16T09:57:02.499160+0000","last_active":"2023-10-16T09:57:03.483785+0000","last_peered":"2023-10-16T09:57:03.483785+0000","last_clean":"2023-10-16T09:57:03.483785+0000","last_became_active":"2023-10-16T09:57:02.498862+0000","last_became_peered":"2023-10-16T09:57:02.498862+0000","last_unstale":"2023-10-16T09:57:03.483785+0000","last_undegraded":"2023-10-16T09:57:03.483785+0000","last_fullsized":"2023-10-16T09:57:03.483785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:20.259232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496790+0000","last_change":"2023-10-16T09:57:01.497430+0000","last_active":"2023-10-16T09:57:03.496790+0000","last_peered":"2023-10-16T09:57:03.496790+0000","last_clean":"2023-10-16T09:57:03.496790+0000","last_became_active":"2023-10-16T09:57:01.497070+0000","last_became_peered":"2023-10-16T09:57:01.497070+0000","last_unstale":"2023-10-16T09:57:03.496790+0000","last_undegraded":"2023-10-16T09:57:03.496790+0000","last_fullsized":"2023-10-16T09:57:03.496790+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:14.885153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721051+0000","last_change":"2023-10-16T09:57:02.501544+0000","last_active":"2023-10-16T09:57:03.721051+0000","last_peered":"2023-10-16T09:57:03.721051+0000","last_clean":"2023-10-16T09:57:03.721051+0000","last_became_active":"2023-10-16T09:57:02.501258+0000","last_became_peered":"2023-10-16T09:57:02.501258+0000","last_unstale":"2023-10-16T09:57:03.721051+0000","last_undegraded":"2023-10-16T09:57:03.721051+0000","last_fullsized":"2023-10-16T09:57:03.721051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:10.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.720945+0000","last_change":"2023-10-16T09:57:01.489733+0000","last_active":"2023-10-16T09:57:03.720945+0000","last_peered":"2023-10-16T09:57:03.720945+0000","last_clean":"2023-10-16T09:57:03.720945+0000","last_became_active":"2023-10-16T09:57:01.489315+0000","last_became_peered":"2023-10-16T09:57:01.489315+0000","last_unstale":"2023-10-16T09:57:03.720945+0000","last_undegraded":"2023-10-16T09:57:03.720945+0000","last_fullsized":"2023-10-16T09:57:03.720945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:16.646321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.496825+0000","last_change":"2023-10-16T09:57:02.499008+0000","last_active":"2023-10-16T09:57:03.496825+0000","last_peered":"2023-10-16T09:57:03.496825+0000","last_clean":"2023-10-16T09:57:03.496825+0000","last_became_active":"2023-10-16T09:57:02.498717+0000","last_became_peered":"2023-10-16T09:57:02.498717+0000","last_unstale":"2023-10-16T09:57:03.496825+0000","last_undegraded":"2023-10-16T09:57:03.496825+0000","last_fullsized":"2023-10-16T09:57:03.496825+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:56:29.754255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721512+0000","last_change":"2023-10-16T09:57:01.499227+0000","last_active":"2023-10-16T09:57:03.721512+0000","last_peered":"2023-10-16T09:57:03.721512+0000","last_clean":"2023-10-16T09:57:03.721512+0000","last_became_active":"2023-10-16T09:57:01.498968+0000","last_became_peered":"2023-10-16T09:57:01.498968+0000","last_unstale":"2023-10-16T09:57:03.721512+0000","last_undegraded":"2023-10-16T09:57:03.721512+0000","last_fullsized":"2023-10-16T09:57:03.721512+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:35:00.655882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721957+0000","last_change":"2023-10-16T09:57:02.499730+0000","last_active":"2023-10-16T09:57:03.721957+0000","last_peered":"2023-10-16T09:57:03.721957+0000","last_clean":"2023-10-16T09:57:03.721957+0000","last_became_active":"2023-10-16T09:57:02.498886+0000","last_became_peered":"2023-10-16T09:57:02.498886+0000","last_unstale":"2023-10-16T09:57:03.721957+0000","last_undegraded":"2023-10-16T09:57:03.721957+0000","last_fullsized":"2023-10-16T09:57:03.721957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:01.467864+0000","last_clean_scrub_stamp":"2023-10-16T09:57:01.467864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:41.784225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-16T09:57:03.721423+0000","last_change":"2023-10-16T09:57:01.496300+0000","last_active":"2023-10-16T09:57:03.721423+0000","last_peered":"2023-10-16T09:57:03.721423+0000","last_clean":"2023-10-16T09:57:03.721423+0000","last_became_active":"2023-10-16T09:57:01.496004+0000","last_became_peered":"2023-10-16T09:57:01.496004+0000","last_unstale":"2023-10-16T09:57:03.721423+0000","last_undegraded":"2023-10-16T09:57:03.721423+0000","last_fullsized":"2023-10-16T09:57:03.721423+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:57:00.460832+0000","last_clean_scrub_stamp":"2023-10-16T09:57:00.460832+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:28.527406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"49'139","reported_seq":284,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-16T09:57:02.488626+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:57:02.488626+0000","last_peered":"2023-10-16T09:57:02.488626+0000","last_clean":"2023-10-16T09:57:02.488626+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:57:02.488626+0000","last_undegraded":"2023-10-16T09:57:02.488626+0000","last_fullsized":"2023-10-16T09:57:02.488626+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542305,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":914093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.248},"max":{"1min":0.70399999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.78300000000000003},"last":0.438}]},{"osd":1,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.25700000000000001},"max":{"1min":0.71099999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.437,"5min":0.41499999999999998,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.39500000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.46100000000000002,"5min":0.42099999999999999,"15min":0.308},"max":{"1min":0.68600000000000005,"5min":0.79400000000000004,"15min":0.80400000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.753,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.377,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.81200000000000006},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.34399999999999997},"max":{"1min":0.74199999999999999,"5min":0.83399999999999996,"15min":0.83699999999999997},"last":0.67100000000000004}]},{"osd":4,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60199999999999998,"5min":0.78300000000000003,"15min":0.86299999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.497,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76200000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.53000000000000003},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77700000000000002,"5min":0.875,"15min":0.875},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":0.79000000000000004,"5min":0.84299999999999997,"15min":1.0489999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Mon Oct 16 09:56:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72099999999999997,"5min":0.84199999999999997,"15min":1.373},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.39800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73699999999999999,"5min":0.85999999999999999,"15min":1.4099999999999999},"last":0.63100000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":704512,"data_stored":324307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58399999999999996,"15min":0.66000000000000003},"min":{"1min":0.34999999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68100000000000005,"5min":0.81999999999999995,"15min":6.194},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60499999999999998,"15min":0.65300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71899999999999997,"5min":0.78200000000000003,"15min":6.1609999999999996},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.44500000000000001,"5min":0.34399999999999997,"15min":0.28799999999999998},"max":{"1min":0.76600000000000001,"5min":0.79500000000000004,"15min":1.054},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.39800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73099999999999998,"5min":0.77300000000000002,"15min":0.876},"last":0.61599999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58699999999999997,"15min":0.69599999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.78800000000000003,"15min":8.5600000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69599999999999995,"5min":0.754,"15min":1.1060000000000001},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71199999999999997,"5min":0.81899999999999995,"15min":1.175},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61199999999999999,"15min":0.68300000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77000000000000002,"5min":0.81399999999999995,"15min":6.2249999999999996},"last":0.63}]},{"osd":4,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.497,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.54400000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51400000000000001,"15min":0.53600000000000003},"min":{"1min":0.377,"5min":0.33400000000000002,"15min":0.30199999999999999},"max":{"1min":0.68999999999999995,"5min":0.77500000000000002,"15min":0.84699999999999998},"last":0.439}]},{"osd":5,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":1.1240000000000001},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.40500000000000003,"5min":0.32500000000000001,"15min":0.31900000000000001},"max":{"1min":0.80300000000000005,"5min":0.83299999999999996,"15min":0.879},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56699999999999995,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35399999999999998},"max":{"1min":0.75800000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45700000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49904,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321936,"statfs":{"total":96636764160,"available":96585662464,"internally_reserved":0,"allocated":688128,"data_stored":317009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23699999999999999,"15min":0.216},"max":{"1min":0.47699999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42699999999999999,"15min":0.39800000000000002},"min":{"1min":0.33600000000000002,"5min":0.251,"15min":0.19600000000000001},"max":{"1min":0.55800000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.28699999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.38300000000000001},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.622},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.442,"15min":0.42099999999999999},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.30599999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46800000000000003,"15min":0.45100000000000001},"min":{"1min":0.36099999999999999,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.65800000000000003,"5min":0.66600000000000004,"15min":0.82199999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44700000000000001,"15min":0.42399999999999999},"min":{"1min":0.31,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.65900000000000003},"last":0.35999999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.378},"max":{"1min":0.90100000000000002,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80900000000000005,"5min":0.93500000000000005,"15min":0.97899999999999998},"last":0.47299999999999998}]},{"osd":5,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64000000000000001,"15min":0.63500000000000001},"min":{"1min":0.49399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.95799999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.47299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.879,"5min":0.93400000000000005,"15min":0.95399999999999996},"last":0.438}]},{"osd":6,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.47999999999999998,"5min":0.434,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":0.92200000000000004,"15min":1.0109999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.79900000000000004,"5min":0.90400000000000003,"15min":0.93799999999999994},"last":0.45400000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67500000000000004,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84699999999999998,"5min":0.89000000000000001,"15min":0.91700000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67500000000000004,"15min":0.65100000000000002},"min":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.41499999999999998},"max":{"1min":0.87,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.497}]}]},{"osd":0,"up_from":14,"seq":60129542305,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1282048,"data_stored":907341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.377,"15min":0.44700000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.191},"max":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":5.5090000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.502},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59399999999999997,"5min":0.62,"15min":6.4950000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41699999999999998,"15min":0.40200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.26000000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.69599999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.437,"15min":0.46200000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59499999999999997,"5min":0.63600000000000001,"15min":2.7559999999999998},"last":0.378}]},{"osd":3,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44500000000000001,"15min":0.54500000000000004},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.20699999999999999},"max":{"1min":0.61699999999999999,"5min":0.67500000000000004,"15min":8.3260000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48999999999999999,"15min":0.55200000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.30099999999999999},"max":{"1min":0.65700000000000003,"5min":0.69699999999999995,"15min":5.4809999999999999},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63700000000000001,"15min":0.753},"min":{"1min":0.45600000000000002,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.82999999999999996,"5min":0.877,"15min":6.4349999999999996},"last":0.439},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64300000000000002,"15min":0.73499999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.36699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":8.2970000000000006},"last":0.50700000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.71799999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":5.4160000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.77900000000000003},"min":{"1min":0.51100000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":6.3869999999999996},"last":0.41799999999999998}]},{"osd":6,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64900000000000002,"15min":0.745},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.32900000000000001},"max":{"1min":0.88100000000000001,"5min":0.92400000000000004,"15min":8.2189999999999994},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":5.4560000000000004},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.80500000000000005},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.45200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":6.4649999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66600000000000004,"15min":0.76600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":8.266},"last":0.47999999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542304,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36036,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335804,"statfs":{"total":96636764160,"available":96599863296,"internally_reserved":0,"allocated":712704,"data_stored":325269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39400000000000002,"15min":0.36599999999999999},"min":{"1min":0.27800000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.54200000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41299999999999998,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.25900000000000001,"15min":0.24399999999999999},"max":{"1min":0.55400000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.28899999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42199999999999999,"15min":0.39600000000000002},"min":{"1min":0.26700000000000002,"5min":0.253,"15min":0.24299999999999999},"max":{"1min":0.56699999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.443,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.58199999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35199999999999998}]},{"osd":3,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.40200000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.22900000000000001},"max":{"1min":0.53200000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.31},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.49099999999999999,"15min":0.45500000000000002},"min":{"1min":0.34799999999999998,"5min":0.30599999999999999,"15min":0.27800000000000002},"max":{"1min":0.60299999999999998,"5min":0.68500000000000005,"15min":0.72899999999999998},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63300000000000001,"15min":0.60999999999999999},"min":{"1min":0.47399999999999998,"5min":0.42699999999999999,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64600000000000002,"15min":0.621},"min":{"1min":0.47499999999999998,"5min":0.39700000000000002,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.92700000000000005,"15min":0.95699999999999996},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.624},"min":{"1min":0.504,"5min":0.434,"15min":0.371},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.97699999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66100000000000003,"15min":0.628},"min":{"1min":0.45100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67800000000000005,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.42799999999999999},"max":{"1min":0.93799999999999994,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.63700000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.64900000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68000000000000005,"15min":0.65400000000000003},"min":{"1min":0.503,"5min":0.44600000000000001,"15min":0.39600000000000002},"max":{"1min":0.83399999999999996,"5min":0.998,"15min":0.998},"last":0.625}]}]},{"osd":3,"up_from":14,"seq":60129542305,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":324273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.375},"min":{"1min":0.223,"5min":0.219,"15min":0.192},"max":{"1min":0.63900000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.435,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.27400000000000002,"15min":0.254},"max":{"1min":0.56299999999999994,"5min":0.63100000000000001,"15min":1.0780000000000001},"last":0.495}]},{"osd":1,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.41499999999999998,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.219},"max":{"1min":0.58699999999999997,"5min":0.72199999999999998,"15min":2.254},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.434,"15min":0.47399999999999998},"min":{"1min":0.318,"5min":0.27600000000000002,"15min":0.246},"max":{"1min":0.754,"5min":0.754,"15min":6.2240000000000002},"last":0.51900000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44800000000000001,"15min":0.44600000000000001},"min":{"1min":0.30599999999999999,"5min":0.26000000000000001,"15min":0.25},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":2.2909999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46000000000000002,"15min":0.435},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.28199999999999997},"max":{"1min":0.66600000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40400000000000003},"max":{"1min":0.85099999999999998,"5min":0.95799999999999996,"15min":1.24},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.51400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85299999999999998,"5min":0.93600000000000005,"15min":2.4420000000000002},"last":0.91700000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.879,"5min":0.97599999999999998,"15min":2.4089999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.42899999999999999,"15min":0.39200000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.996},"last":0.93300000000000005}]},{"osd":6,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65500000000000003,"15min":0.63800000000000001},"min":{"1min":0.52700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85699999999999998,"5min":0.96699999999999997,"15min":0.98399999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.379},"max":{"1min":0.86799999999999999,"5min":0.99099999999999999,"15min":1.1619999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.505,"5min":0.377,"15min":0.377},"max":{"1min":0.90400000000000003,"5min":0.96399999999999997,"15min":1.206},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91300000000000003,"5min":0.97499999999999998,"15min":2.3740000000000001},"last":0.94999999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542304,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54084,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317756,"statfs":{"total":96636764160,"available":96581382144,"internally_reserved":0,"allocated":708608,"data_stored":325247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.186,"15min":0.186},"max":{"1min":0.752,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.42299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72199999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.33000000000000002,"5min":0.308,"15min":0.248},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.49399999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.85399999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.47399999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81999999999999995,"5min":0.82799999999999996,"15min":0.89100000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57499999999999996},"min":{"1min":0.50600000000000001,"5min":0.39300000000000002,"15min":0.35099999999999998},"max":{"1min":0.77900000000000003,"5min":0.83299999999999996,"15min":0.86499999999999999},"last":0.624},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45600000000000002,"5min":0.41199999999999998,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.505},"min":{"1min":0.37,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72199999999999998,"5min":0.75,"15min":0.81000000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.52900000000000003},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":0.80400000000000005,"15min":0.86899999999999999},"last":0.69399999999999995}]},{"osd":6,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.40300000000000002,"5min":0.32800000000000001,"15min":0.32200000000000001},"max":{"1min":0.78900000000000003,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.379,"15min":0.35599999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85999999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:56:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.373,"15min":0.33300000000000002},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.84599999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.36099999999999999},"max":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.95999999999999996},"last":0.78400000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":323705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60799999999999998,"15min":0.57099999999999995},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.253},"max":{"1min":0.67200000000000004,"5min":0.84399999999999997,"15min":0.92900000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.71199999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77300000000000002,"5min":0.82299999999999995,"15min":8.0370000000000008},"last":0.57599999999999996}]},{"osd":1,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.57799999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.24399999999999999},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.98599999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.70199999999999996},"min":{"1min":0.38400000000000001,"5min":0.38,"15min":0.307},"max":{"1min":0.71299999999999997,"5min":0.92000000000000004,"15min":8.2110000000000003},"last":0.59199999999999997}]},{"osd":2,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59799999999999998,"15min":0.69299999999999995},"min":{"1min":0.42599999999999999,"5min":0.33300000000000002,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":8.1140000000000008},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61099999999999999,"15min":0.69599999999999995},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.72499999999999998,"5min":0.81799999999999995,"15min":8.3000000000000007},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.629,"15min":0.62},"min":{"1min":0.49099999999999999,"5min":0.42899999999999999,"15min":0.312},"max":{"1min":0.78600000000000003,"5min":0.89000000000000001,"15min":1.03},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.628,"15min":0.749},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.874,"15min":9.8810000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.34499999999999997,"5min":0.28999999999999998,"15min":0.26000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":1.0449999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51400000000000001,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71899999999999997,"5min":0.81000000000000005,"15min":1.0780000000000001},"last":0.36099999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.55300000000000005},"min":{"1min":0.38300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.75700000000000001,"5min":0.77000000000000002,"15min":1.599},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.38400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.83099999999999996,"15min":1.006},"last":0.442}]},{"osd":7,"last update":"Mon Oct 16 09:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.373,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.85699999999999998,"15min":1.1339999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.436,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79500000000000004,"5min":0.84799999999999998,"15min":1.889},"last":0.46999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:57:06.997 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T09:57:06.997 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T09:57:07.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.157+0000 7fdc6d9d2700 1 Processor -- start 2023-10-16T09:57:07.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.157+0000 7fdc6d9d2700 1 -- start start 2023-10-16T09:57:07.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc6d9d2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 0x7fdc68071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc6d9d2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdc680ce680 0x7fdc680ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc6d9d2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc680ccb60 con 0x7fdc680b2010 2023-10-16T09:57:07.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc6d9d2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc68072120 con 0x7fdc680ce680 2023-10-16T09:57:07.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc6d9d2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680ce1f0 con 0x7fdc680cc6e0 2023-10-16T09:57:07.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc66ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdc680b2010 0x7fdc680b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51384/0 (socket says 172.21.15.181:51384) 2023-10-16T09:57:07.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc66ffd700 1 -- 172.21.15.181:0/4210989950 learned_addr learned my addr 172.21.15.181:0/4210989950 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:07.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc667fc700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 0x7fdc68071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc677fe700 1 --2- 172.21.15.181:0/4210989950 >> v2:172.21.15.190:3300/0 conn(0x7fdc680ce680 0x7fdc680ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489295922 0 0) 0x7fdc5c001110 con 0x7fdc680b2010 2023-10-16T09:57:07.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc500057c0 con 0x7fdc680b2010 2023-10-16T09:57:07.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570868270 0 0) 0x7fdc500057c0 con 0x7fdc680b2010 2023-10-16T09:57:07.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 msgr2=0x7fdc68071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:07.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 0x7fdc68071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 >> v2:172.21.15.190:3300/0 conn(0x7fdc680ce680 msgr2=0x7fdc680ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:07.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 --2- 172.21.15.181:0/4210989950 >> v2:172.21.15.190:3300/0 conn(0x7fdc680ce680 0x7fdc680ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680ccda0 con 0x7fdc680b2010 2023-10-16T09:57:07.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fdc5c001800 con 0x7fdc680b2010 2023-10-16T09:57:07.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 >> v1:172.21.15.181:6789/0 conn(0x7fdc680b2010 legacy=0x7fdc680b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 0x7fdc50005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc50006190 0x7fdc50006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc50006fa0 0x7fdc50007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68072120 con 0x7fdc50006fa0 2023-10-16T09:57:07.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680ce1f0 con 0x7fdc50006190 2023-10-16T09:57:07.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5000dc20 con 0x7fdc680cc6e0 2023-10-16T09:57:07.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.159+0000 7fdc677fe700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc50006fa0 0x7fdc50007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc667fc700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc50006190 0x7fdc50006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc66ffd700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 0x7fdc50005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc677fe700 1 -- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 msgr2=0x7fdc50005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:07.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc677fe700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680cc6e0 0x7fdc50005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc677fe700 1 -- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc50006190 msgr2=0x7fdc50006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:07.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc677fe700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc50006190 0x7fdc50006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc677fe700 1 -- 172.21.15.181:0/4210989950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500057c0 con 0x7fdc50006fa0 2023-10-16T09:57:07.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc677fe700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc50006fa0 0x7fdc50007420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009fc0 tx=0x7fdc50007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f0fe2e97bc8e27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:07.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000ebd0 con 0x7fdc50006fa0 2023-10-16T09:57:07.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000f040 con 0x7fdc50006fa0 2023-10-16T09:57:07.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.160+0000 7fdc65ffb700 1 -- 172.21.15.181:0/4210989950 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60014480 con 0x7fdc50006fa0 2023-10-16T09:57:07.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.161+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/4210989950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc50006fa0 msgr2=0x7fdc50007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.161+0000 7fdc6d9d2700 1 --2- 172.21.15.181:0/4210989950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc50006fa0 0x7fdc50007420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009fc0 tx=0x7fdc50007a90 comp rx=0 tx=0).stop 2023-10-16T09:57:07.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.161+0000 7fdc677fe700 1 -- 172.21.15.181:0/4210989950 reap_dead start 2023-10-16T09:57:07.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.161+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/4210989950 shutdown_connections 2023-10-16T09:57:07.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.161+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/4210989950 >> 172.21.15.181:0/4210989950 conn(0x7fdc680cd010 msgr2=0x7fdc680cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:07.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.161+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/4210989950 shutdown_connections 2023-10-16T09:57:07.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.161+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/4210989950 wait complete. 2023-10-16T09:57:07.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.162+0000 7fdc6d9d2700 1 Processor -- start 2023-10-16T09:57:07.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.162+0000 7fdc6d9d2700 1 -- start start 2023-10-16T09:57:07.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc6d9d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc681ee800 0x7fdc680c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc6d9d2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680c7660 0x7fdc6815bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc6d9d2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc680c99b0 0x7fdc680c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc6d9d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680ca3a0 con 0x7fdc681ee800 2023-10-16T09:57:07.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc6d9d2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6814b4b0 con 0x7fdc680c99b0 2023-10-16T09:57:07.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc6d9d2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680c8a90 con 0x7fdc680c7660 2023-10-16T09:57:07.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc66ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc681ee800 0x7fdc680c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc66ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc681ee800 0x7fdc680c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40318/0 (socket says 172.21.15.181:40318) 2023-10-16T09:57:07.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.163+0000 7fdc66ffd700 1 -- 172.21.15.181:0/1375591277 learned_addr learned my addr 172.21.15.181:0/1375591277 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:07.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc667fc700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680c7660 0x7fdc6815bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc677fe700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc680c99b0 0x7fdc680c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc66ffd700 1 -- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680c7660 msgr2=0x7fdc6815bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc66ffd700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680c7660 0x7fdc6815bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc66ffd700 1 -- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc680c99b0 msgr2=0x7fdc680c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc66ffd700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc680c99b0 0x7fdc680c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc66ffd700 1 -- 172.21.15.181:0/1375591277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680d02a0 con 0x7fdc681ee800 2023-10-16T09:57:07.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc66ffd700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc681ee800 0x7fdc680c8550 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000fb0 tx=0x7fdc5c00f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:07.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc4ffff700 1 -- 172.21.15.181:0/1375591277 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c007e00 con 0x7fdc681ee800 2023-10-16T09:57:07.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc4ffff700 1 -- 172.21.15.181:0/1375591277 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c00fa30 con 0x7fdc681ee800 2023-10-16T09:57:07.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc680c8c80 con 0x7fdc681ee800 2023-10-16T09:57:07.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc4ffff700 1 -- 172.21.15.181:0/1375591277 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00fde0 con 0x7fdc681ee800 2023-10-16T09:57:07.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.164+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc680b3370 con 0x7fdc681ee800 2023-10-16T09:57:07.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.165+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc680b6510 con 0x7fdc681ee800 2023-10-16T09:57:07.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.165+0000 7fdc4ffff700 1 -- 172.21.15.181:0/1375591277 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdc5c01d460 con 0x7fdc681ee800 2023-10-16T09:57:07.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.166+0000 7fdc4ffff700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fdc5406b020 0x7fdc5406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.166+0000 7fdc4ffff700 1 -- 172.21.15.181:0/1375591277 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7fdc5c0898d0 con 0x7fdc681ee800 2023-10-16T09:57:07.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.166+0000 7fdc667fc700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fdc5406b020 0x7fdc5406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.166+0000 7fdc667fc700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fdc5406b020 0x7fdc5406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc68071e70 tx=0x7fdc58008040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:07.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.170+0000 7fdc4ffff700 1 -- 172.21.15.181:0/1375591277 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc5c0546e0 con 0x7fdc681ee800 2023-10-16T09:57:07.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.439+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdc680ccb60 con 0x7fdc681ee800 2023-10-16T09:57:07.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.441+0000 7fdc4ffff700 1 -- 172.21.15.181:0/1375591277 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdc680ccb60 con 0x7fdc681ee800 2023-10-16T09:57:07.443 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T09:57:07.443 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T09:57:07.443 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T09:57:07.443 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T09:57:07.444 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T09:57:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fdc5406b020 msgr2=0x7fdc5406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fdc5406b020 0x7fdc5406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc68071e70 tx=0x7fdc58008040 comp rx=0 tx=0).stop 2023-10-16T09:57:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc681ee800 msgr2=0x7fdc680c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc681ee800 0x7fdc680c8550 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000fb0 tx=0x7fdc5c00f4b0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 shutdown_connections 2023-10-16T09:57:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fdc5406b020 0x7fdc5406d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc680c99b0 0x7fdc680c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc681ee800 0x7fdc680c8550 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 --2- 172.21.15.181:0/1375591277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc680c7660 0x7fdc6815bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 >> 172.21.15.181:0/1375591277 conn(0x7fdc680cd010 msgr2=0x7fdc680b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:07.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 shutdown_connections 2023-10-16T09:57:07.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.444+0000 7fdc6d9d2700 1 -- 172.21.15.181:0/1375591277 wait complete. 2023-10-16T09:57:07.448 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T09:57:07.458 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:57:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.579+0000 7ff616235700 1 Processor -- start 2023-10-16T09:57:07.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.579+0000 7ff616235700 1 -- start start 2023-10-16T09:57:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff616235700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff61015dfd0 0x7ff6100af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff616235700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6100cd470 0x7ff6100cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff616235700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61006f870 con 0x7ff6100af450 2023-10-16T09:57:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff616235700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6100cde40 con 0x7ff61015dfd0 2023-10-16T09:57:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff616235700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100d1160 con 0x7ff6100cd470 2023-10-16T09:57:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff60f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff61015dfd0 0x7ff6100af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff60f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff61015dfd0 0x7ff6100af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40326/0 (socket says 172.21.15.181:40326) 2023-10-16T09:57:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff60effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff6100af450 0x7ff61006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51406/0 (socket says 172.21.15.181:51406) 2023-10-16T09:57:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff60f7fe700 1 -- 172.21.15.181:0/3669663970 learned_addr learned my addr 172.21.15.181:0/3669663970 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.580+0000 7ff60ffff700 1 --2- 172.21.15.181:0/3669663970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6100cd470 0x7ff6100cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60f7fe700 1 -- 172.21.15.181:0/3669663970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6100cd470 msgr2=0x7ff6100cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60f7fe700 1 --2- 172.21.15.181:0/3669663970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6100cd470 0x7ff6100cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60f7fe700 1 -- 172.21.15.181:0/3669663970 >> v1:172.21.15.181:6789/0 conn(0x7ff6100af450 legacy=0x7ff61006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60e7fc700 1 -- 172.21.15.181:0/3669663970 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972140749 0 0) 0x7ff61006f870 con 0x7ff6100af450 2023-10-16T09:57:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60f7fe700 1 -- 172.21.15.181:0/3669663970 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100cc8e0 con 0x7ff61015dfd0 2023-10-16T09:57:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60f7fe700 1 --2- 172.21.15.181:0/3669663970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61015dfd0 0x7ff6100af9a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d567ad3bbdc6a37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60e7fc700 1 -- 172.21.15.181:0/3669663970 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604005e30 con 0x7ff61015dfd0 2023-10-16T09:57:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60e7fc700 1 -- 172.21.15.181:0/3669663970 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604004030 con 0x7ff61015dfd0 2023-10-16T09:57:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff60e7fc700 1 -- 172.21.15.181:0/3669663970 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040043e0 con 0x7ff61015dfd0 2023-10-16T09:57:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 -- 172.21.15.181:0/3669663970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61015dfd0 msgr2=0x7ff6100af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 --2- 172.21.15.181:0/3669663970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61015dfd0 0x7ff6100af9a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604009bd0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 -- 172.21.15.181:0/3669663970 shutdown_connections 2023-10-16T09:57:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 --2- 172.21.15.181:0/3669663970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6100cd470 0x7ff6100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 --2- 172.21.15.181:0/3669663970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff61015dfd0 0x7ff6100af9a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 -- 172.21.15.181:0/3669663970 >> 172.21.15.181:0/3669663970 conn(0x7ff6100d04a0 msgr2=0x7ff6100d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 -- 172.21.15.181:0/3669663970 shutdown_connections 2023-10-16T09:57:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.581+0000 7ff616235700 1 -- 172.21.15.181:0/3669663970 wait complete. 2023-10-16T09:57:07.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.583+0000 7ff616235700 1 Processor -- start 2023-10-16T09:57:07.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.583+0000 7ff616235700 1 -- start start 2023-10-16T09:57:07.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff616235700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6100af450 0x7ff6100aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff616235700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6100cd470 0x7ff6101461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff616235700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff61015dfd0 0x7ff6100b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff616235700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610146700 con 0x7ff6100af450 2023-10-16T09:57:07.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff616235700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b2e90 con 0x7ff6100cd470 2023-10-16T09:57:07.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff616235700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b1b90 con 0x7ff61015dfd0 2023-10-16T09:57:07.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6100af450 0x7ff6100aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6100af450 0x7ff6100aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40332/0 (socket says 172.21.15.181:40332) 2023-10-16T09:57:07.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 -- 172.21.15.181:0/2956942087 learned_addr learned my addr 172.21.15.181:0/2956942087 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:07.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60ffff700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff61015dfd0 0x7ff6100b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60effd700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6100cd470 0x7ff6101461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 -- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff61015dfd0 msgr2=0x7ff6100b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff61015dfd0 0x7ff6100b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 -- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6100cd470 msgr2=0x7ff6101461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6100cd470 0x7ff6101461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.584+0000 7ff60f7fe700 1 -- 172.21.15.181:0/2956942087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604009500 con 0x7ff6100af450 2023-10-16T09:57:07.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.585+0000 7ff60f7fe700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6100af450 0x7ff6100aff00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:07.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.585+0000 7ff5f7fff700 1 -- 172.21.15.181:0/2956942087 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604003d90 con 0x7ff6100af450 2023-10-16T09:57:07.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.585+0000 7ff5f7fff700 1 -- 172.21.15.181:0/2956942087 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604011420 con 0x7ff6100af450 2023-10-16T09:57:07.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.585+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6100b1e70 con 0x7ff6100af450 2023-10-16T09:57:07.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.585+0000 7ff5f7fff700 1 -- 172.21.15.181:0/2956942087 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040117d0 con 0x7ff6100af450 2023-10-16T09:57:07.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.585+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6100b2430 con 0x7ff6100af450 2023-10-16T09:57:07.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.586+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff610067d20 con 0x7ff6100af450 2023-10-16T09:57:07.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.586+0000 7ff5f7fff700 1 -- 172.21.15.181:0/2956942087 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff604011930 con 0x7ff6100af450 2023-10-16T09:57:07.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.586+0000 7ff5f7fff700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff5fc06b020 0x7ff5fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:07.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.587+0000 7ff5f7fff700 1 -- 172.21.15.181:0/2956942087 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7ff60408aa90 con 0x7ff6100af450 2023-10-16T09:57:07.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.587+0000 7ff60effd700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff5fc06b020 0x7ff5fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:07.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.587+0000 7ff60effd700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff5fc06b020 0x7ff5fc06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff6101f6b80 tx=0x7ff600007c90 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:07.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.591+0000 7ff5f7fff700 1 -- 172.21.15.181:0/2956942087 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff604055930 con 0x7ff6100af450 2023-10-16T09:57:07.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.828+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6100cfaa0 con 0x7ff6100af450 2023-10-16T09:57:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.828+0000 7ff5f7fff700 1 -- 172.21.15.181:0/2956942087 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7ff604058f50 con 0x7ff6100af450 2023-10-16T09:57:07.830 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:07.831 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5925,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/47573574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":47573574},{"type":"v1","addr":"172.21.15.181:6836","nonce":47573574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3518254514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3518254514},{"type":"v1","addr":"172.21.15.181:6837","nonce":3518254514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4292050143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4292050143},{"type":"v1","addr":"172.21.15.190:6835","nonce":4292050143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:57:01.885491+0000","modified":"2023-10-16T09:57:04.906845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5943":{"gid":5943,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2407452630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2407452630},{"type":"v1","addr":"172.21.15.190:6834","nonce":2407452630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T09:57:07.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff5fc06b020 msgr2=0x7ff5fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff5fc06b020 0x7ff5fc06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff6101f6b80 tx=0x7ff600007c90 comp rx=0 tx=0).stop 2023-10-16T09:57:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6100af450 msgr2=0x7ff6100aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6100af450 0x7ff6100aff00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff604004750 comp rx=0 tx=0).stop 2023-10-16T09:57:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 shutdown_connections 2023-10-16T09:57:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff5fc06b020 0x7ff5fc06d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff61015dfd0 0x7ff6100b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6100af450 0x7ff6100aff00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 --2- 172.21.15.181:0/2956942087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6100cd470 0x7ff6101461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 >> 172.21.15.181:0/2956942087 conn(0x7ff6100d04a0 msgr2=0x7ff61015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 shutdown_connections 2023-10-16T09:57:07.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:07.830+0000 7ff616235700 1 -- 172.21.15.181:0/2956942087 wait complete. 2023-10-16T09:57:07.835 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 25 2023-10-16T09:57:08.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T09:57:01.885491+0000', 'modified': '2023-10-16T09:57:04.906845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5943}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5943': {'gid': 5943, 'name': 'b', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6834/2407452630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 2407452630}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 2407452630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T09:57:08.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T09:57:08.250 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T09:57:08.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.379+0000 7ff06316b700 1 Processor -- start 2023-10-16T09:57:08.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.379+0000 7ff06316b700 1 -- start start 2023-10-16T09:57:08.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff06316b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff05c15df60 0x7ff05c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff06316b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c0d0fc0 0x7ff05c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff06316b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05c0ccf50 con 0x7ff05c06a8a0 2023-10-16T09:57:08.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff06316b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05c0af9c0 con 0x7ff05c15df60 2023-10-16T09:57:08.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff06316b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0cc600 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff060f07700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff05c15df60 0x7ff05c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff061708700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff05c06a8a0 0x7ff05c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51422/0 (socket says 172.21.15.181:51422) 2023-10-16T09:57:08.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff061708700 1 -- 172.21.15.181:0/1715368644 learned_addr learned my addr 172.21.15.181:0/1715368644 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:08.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff060f07700 1 -- 172.21.15.181:0/1715368644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c0d0fc0 msgr2=0x7ff05c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff060f07700 1 --2- 172.21.15.181:0/1715368644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c0d0fc0 0x7ff05c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff060f07700 1 -- 172.21.15.181:0/1715368644 >> v1:172.21.15.181:6789/0 conn(0x7ff05c06a8a0 legacy=0x7ff05c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff060f07700 1 -- 172.21.15.181:0/1715368644 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c0cc8a0 con 0x7ff05c15df60 2023-10-16T09:57:08.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.380+0000 7ff05b7fe700 1 -- 172.21.15.181:0/1715368644 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500405045 0 0) 0x7ff05c0ccf50 con 0x7ff05c06a8a0 2023-10-16T09:57:08.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff060f07700 1 --2- 172.21.15.181:0/1715368644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c15df60 0x7ff05c0cae20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff050009820 tx=0x7ff050009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9167e27e311aa32 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff05b7fe700 1 -- 172.21.15.181:0/1715368644 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050005e30 con 0x7ff05c15df60 2023-10-16T09:57:08.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff05b7fe700 1 -- 172.21.15.181:0/1715368644 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff050006030 con 0x7ff05c15df60 2023-10-16T09:57:08.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff05b7fe700 1 -- 172.21.15.181:0/1715368644 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0500063e0 con 0x7ff05c15df60 2023-10-16T09:57:08.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 -- 172.21.15.181:0/1715368644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c15df60 msgr2=0x7ff05c0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 --2- 172.21.15.181:0/1715368644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c15df60 0x7ff05c0cae20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff050009820 tx=0x7ff050009b30 comp rx=0 tx=0).stop 2023-10-16T09:57:08.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 -- 172.21.15.181:0/1715368644 shutdown_connections 2023-10-16T09:57:08.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 --2- 172.21.15.181:0/1715368644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c15df60 0x7ff05c0cae20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 --2- 172.21.15.181:0/1715368644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c0d0fc0 0x7ff05c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 -- 172.21.15.181:0/1715368644 >> 172.21.15.181:0/1715368644 conn(0x7ff05c0d04a0 msgr2=0x7ff05c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 -- 172.21.15.181:0/1715368644 shutdown_connections 2023-10-16T09:57:08.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.381+0000 7ff06316b700 1 -- 172.21.15.181:0/1715368644 wait complete. 2023-10-16T09:57:08.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.382+0000 7ff06316b700 1 Processor -- start 2023-10-16T09:57:08.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.382+0000 7ff06316b700 1 -- start start 2023-10-16T09:57:08.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff06316b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c06a8a0 0x7ff05c14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff06316b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c0d0fc0 0x7ff05c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff06316b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff05c15df60 0x7ff05c0c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff06316b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0c4210 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff06316b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0c6ec0 con 0x7ff05c15df60 2023-10-16T09:57:08.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff06316b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c14ef40 con 0x7ff05c06a8a0 2023-10-16T09:57:08.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff05bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c0d0fc0 0x7ff05c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff05bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c0d0fc0 0x7ff05c0c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40348/0 (socket says 172.21.15.181:40348) 2023-10-16T09:57:08.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.383+0000 7ff05bfff700 1 -- 172.21.15.181:0/2908557738 learned_addr learned my addr 172.21.15.181:0/2908557738 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:08.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff060f07700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c06a8a0 0x7ff05c14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff061708700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff05c15df60 0x7ff05c0c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff05bfff700 1 -- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c06a8a0 msgr2=0x7ff05c14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c06a8a0 0x7ff05c14e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff05bfff700 1 -- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff05c15df60 msgr2=0x7ff05c0c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff05c15df60 0x7ff05c0c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff05bfff700 1 -- 172.21.15.181:0/2908557738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff050009430 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff061708700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff05c15df60 0x7ff05c0c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff060f07700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c06a8a0 0x7ff05c14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:08.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c0d0fc0 0x7ff05c0c6980 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff04800b530 tx=0x7ff04800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff0597fa700 1 -- 172.21.15.181:0/2908557738 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048011830 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff0597fa700 1 -- 172.21.15.181:0/2908557738 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff048011990 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c0b40d0 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff0597fa700 1 -- 172.21.15.181:0/2908557738 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048011d40 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.384+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c0c5020 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.386+0000 7ff0597fa700 1 -- 172.21.15.181:0/2908557738 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff04801a400 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.386+0000 7ff0597fa700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff04006b020 0x7ff04006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.386+0000 7ff0597fa700 1 -- 172.21.15.181:0/2908557738 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7ff048089860 con 0x7ff05c0d0fc0 2023-10-16T09:57:08.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.386+0000 7ff060f07700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff04006b020 0x7ff04006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.386+0000 7ff060f07700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff04006b020 0x7ff04006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff050013040 tx=0x7ff050008070 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.388+0000 7ff06f355b80 1 Processor -- start 2023-10-16T09:57:08.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.388+0000 7ff06f355b80 1 -- start start 2023-10-16T09:57:08.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c696ff5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c696fc96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff06f355b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c696f3fc20 con 0x55c696ff3630 2023-10-16T09:57:08.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c696f24c10 con 0x55c696fc9010 2023-10-16T09:57:08.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c696f22530 con 0x55c696fee220 2023-10-16T09:57:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40356/0 (socket says 172.21.15.181:40356) 2023-10-16T09:57:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 -- 172.21.15.181:0/4075767013 learned_addr learned my addr 172.21.15.181:0/4075767013 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 -- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 msgr2=0x55c696ff5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff060f07700 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c696ff5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff061708700 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c696fc96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c696ff5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 -- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 msgr2=0x55c696fc96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c696fc96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 -- 172.21.15.181:0/4075767013 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c696edb3a0 con 0x55c696ff3630 2023-10-16T09:57:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff061708700 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c696fc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff05bfff700 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff5fb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff0480548c0 tx=0x7ff048057da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490776166d96f783 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff0467fc700 1 -- 172.21.15.181:0/4075767013 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04805c510 con 0x55c696ff3630 2023-10-16T09:57:08.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff0467fc700 1 -- 172.21.15.181:0/4075767013 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0480606c0 con 0x55c696ff3630 2023-10-16T09:57:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.389+0000 7ff0467fc700 1 -- 172.21.15.181:0/4075767013 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048058620 con 0x55c696ff3630 2023-10-16T09:57:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 -- 172.21.15.181:0/4075767013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 msgr2=0x55c696ff5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff5fb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff0480548c0 tx=0x7ff048057da0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 -- 172.21.15.181:0/4075767013 shutdown_connections 2023-10-16T09:57:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c696ff5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c696fc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 --2- 172.21.15.181:0/4075767013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff5fb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 -- 172.21.15.181:0/4075767013 >> 172.21.15.181:0/4075767013 conn(0x55c696ef8d80 msgr2=0x55c697013ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 -- 172.21.15.181:0/4075767013 shutdown_connections 2023-10-16T09:57:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.390+0000 7ff06f355b80 1 -- 172.21.15.181:0/4075767013 wait complete. 2023-10-16T09:57:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.391+0000 7ff06f355b80 1 -- start start 2023-10-16T09:57:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.391+0000 7ff06f355b80 1 Processor -- start 2023-10-16T09:57:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696ff9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c696f1de20 con 0x55c696fc9010 2023-10-16T09:57:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c696eba6c0 con 0x55c696fee220 2023-10-16T09:57:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c696eba360 con 0x55c696ff3630 2023-10-16T09:57:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 10 client.0 _ll_register_callbacks cb 0x55c696d177b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:57:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696ff9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696ff9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40360/0 (socket says 172.21.15.181:40360) 2023-10-16T09:57:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 -- 172.21.15.181:0/2061730397 learned_addr learned my addr 172.21.15.181:0/2061730397 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff061708700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 -- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 msgr2=0x55c696ff4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 -- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 msgr2=0x55c696ff4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 -- 172.21.15.181:0/2061730397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c0c5020 con 0x55c696fc9010 2023-10-16T09:57:08.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff060f07700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696ff9420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff050006d30 tx=0x7ff0500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1204052b46751c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff03b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:57:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff03b7fe700 1 -- 172.21.15.181:0/2061730397 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05002c070 con 0x55c696fc9010 2023-10-16T09:57:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff03b7fe700 1 -- 172.21.15.181:0/2061730397 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7ff050032c60 con 0x55c696fc9010 2023-10-16T09:57:08.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c696fbe5b0 con 0x55c696fc9010 2023-10-16T09:57:08.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff03b7fe700 1 -- 172.21.15.181:0/2061730397 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff050006a10 con 0x55c696fc9010 2023-10-16T09:57:08.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 10 client.6081 fetch_fsmap learned FSMap version 25 2023-10-16T09:57:08.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.392+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c696d48d90 con 0x55c696fc9010 2023-10-16T09:57:08.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff03b7fe700 1 -- 172.21.15.181:0/2061730397 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c696fbe5b0 con 0x55c696fc9010 2023-10-16T09:57:08.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff03b7fe700 1 -- 172.21.15.181:0/2061730397 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff0500038b0 con 0x55c696fc9010 2023-10-16T09:57:08.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff06f355b80 10 client.6081 fetch_fsmap finished waiting for FSMap version 25 2023-10-16T09:57:08.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff06f355b80 10 client.6081 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T09:57:08.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff06f355b80 20 client.6081 populate_metadata read hostname 'smithi181' 2023-10-16T09:57:08.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff6f00 0x55c696fec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff06f355b80 4 client.6081 mds_command: new command op to 5943 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T09:57:08.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 --> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c696f16e00 con 0x55c696ff6f00 2023-10-16T09:57:08.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff6f00 0x55c696fec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff03b7fe700 10 client.6081 ms_handle_connect on v2:172.21.15.190:6832/2407452630 2023-10-16T09:57:08.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.393+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff6f00 0x55c696fec9e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ece503b1ab2155d server_cookie=5c3a7887149592df in_seq=0 out_seq=0 2023-10-16T09:57:08.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff03b7fe700 1 -- 172.21.15.181:0/2061730397 <== mds.0 v2:172.21.15.190:6832/2407452630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c696f16e00 con 0x55c696ff6f00 2023-10-16T09:57:08.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff03b7fe700 10 client.6081 handle_command_reply: tid=0 2023-10-16T09:57:08.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 client.6081 shutdown 2023-10-16T09:57:08.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 msgr2=0x55c696ff9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696ff9420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff050006d30 tx=0x7ff0500047c0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 shutdown_connections 2023-10-16T09:57:08.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff6f00 0x55c696fec9e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff03b7fe700 0 client.6081 ms_handle_reset on v2:172.21.15.190:6832/2407452630 2023-10-16T09:57:08.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696ff9420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2061730397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 >> 172.21.15.181:0/2061730397 conn(0x55c696ef8d80 msgr2=0x55c697013f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 shutdown_connections 2023-10-16T09:57:08.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.394+0000 7ff06f355b80 1 -- 172.21.15.181:0/2061730397 wait complete. 2023-10-16T09:57:08.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.395+0000 7ff06f355b80 20 client.6081 trim_cache size 0 max 16384 2023-10-16T09:57:08.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.412+0000 7ff06f355b80 1 Processor -- start 2023-10-16T09:57:08.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.412+0000 7ff06f355b80 1 -- start start 2023-10-16T09:57:08.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696fc0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff06f355b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c696d9e8d0 con 0x55c696fc9010 2023-10-16T09:57:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69700a830 con 0x55c696fee220 2023-10-16T09:57:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c696fbe3b0 con 0x55c696ff3630 2023-10-16T09:57:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696fc0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696fc0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40368/0 (socket says 172.21.15.181:40368) 2023-10-16T09:57:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 -- 172.21.15.181:0/2847572463 learned_addr learned my addr 172.21.15.181:0/2847572463 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff05bfff700 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff061708700 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 -- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 msgr2=0x55c696ff9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 -- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 msgr2=0x55c696ff90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 -- 172.21.15.181:0/2847572463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c696fc7f20 con 0x55c696fc9010 2023-10-16T09:57:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff060f07700 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696fc0300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff050006a70 tx=0x7ff05002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372deb26f29baa2c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff046ffd700 1 -- 172.21.15.181:0/2847572463 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050005c30 con 0x55c696fc9010 2023-10-16T09:57:08.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff046ffd700 1 -- 172.21.15.181:0/2847572463 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0500038b0 con 0x55c696fc9010 2023-10-16T09:57:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.413+0000 7ff046ffd700 1 -- 172.21.15.181:0/2847572463 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050005d90 con 0x55c696fc9010 2023-10-16T09:57:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 -- 172.21.15.181:0/2847572463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 msgr2=0x55c696fc0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696fc0300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff050006a70 tx=0x7ff05002c000 comp rx=0 tx=0).stop 2023-10-16T09:57:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 -- 172.21.15.181:0/2847572463 shutdown_connections 2023-10-16T09:57:08.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696ff3630 0x55c696ff9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696fc9010 0x55c696fc0300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 --2- 172.21.15.181:0/2847572463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fee220 0x55c696ff90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 -- 172.21.15.181:0/2847572463 >> 172.21.15.181:0/2847572463 conn(0x55c696ef8d80 msgr2=0x55c697013f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 -- 172.21.15.181:0/2847572463 shutdown_connections 2023-10-16T09:57:08.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 -- 172.21.15.181:0/2847572463 wait complete. 2023-10-16T09:57:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.414+0000 7ff06f355b80 1 -- start start 2023-10-16T09:57:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 1 Processor -- start 2023-10-16T09:57:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c6970109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c697010f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c696fca9b0 con 0x55c696ff3630 2023-10-16T09:57:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c696fcc140 con 0x55c696fc9010 2023-10-16T09:57:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c697077800 con 0x55c696fee220 2023-10-16T09:57:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff06f355b80 10 client.0 _ll_register_callbacks cb 0x7ff050006590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:57:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff061708700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff061708700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40378/0 (socket says 172.21.15.181:40378) 2023-10-16T09:57:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.415+0000 7ff061708700 1 -- 172.21.15.181:0/192376049 learned_addr learned my addr 172.21.15.181:0/192376049 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff060f07700 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c6970109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff05bfff700 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c697010f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff061708700 1 -- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 msgr2=0x55c697010f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff061708700 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c697010f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff061708700 1 -- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 msgr2=0x55c6970109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff061708700 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c6970109f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff061708700 1 -- 172.21.15.181:0/192376049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c696d48d90 con 0x55c696ff3630 2023-10-16T09:57:08.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff061708700 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff1270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff054000f80 tx=0x7ff054007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba5cc29b569a1e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff03b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:57:08.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff03b7fe700 1 -- 172.21.15.181:0/192376049 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054016890 con 0x55c696ff3630 2023-10-16T09:57:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff03b7fe700 1 -- 172.21.15.181:0/192376049 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7ff0540169f0 con 0x55c696ff3630 2023-10-16T09:57:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0480606c0 con 0x55c696ff3630 2023-10-16T09:57:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff03b7fe700 1 -- 172.21.15.181:0/192376049 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff054016bb0 con 0x55c696ff3630 2023-10-16T09:57:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 10 client.6087 fetch_fsmap learned FSMap version 25 2023-10-16T09:57:08.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c696ff8f20 con 0x55c696ff3630 2023-10-16T09:57:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff03b7fe700 1 -- 172.21.15.181:0/192376049 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0480606c0 con 0x55c696ff3630 2023-10-16T09:57:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff03b7fe700 1 -- 172.21.15.181:0/192376049 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff054016d10 con 0x55c696ff3630 2023-10-16T09:57:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 10 client.6087 fetch_fsmap finished waiting for FSMap version 25 2023-10-16T09:57:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 10 client.6087 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T09:57:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 20 client.6087 populate_metadata read hostname 'smithi181' 2023-10-16T09:57:08.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff19f0 0x55c696fcae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 4 client.6087 mds_command: new command op to 5943 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T09:57:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.416+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 --> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c696f16b20 con 0x55c696ff19f0 2023-10-16T09:57:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff060f07700 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff19f0 0x55c696fcae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff03b7fe700 10 client.6087 ms_handle_connect on v2:172.21.15.190:6832/2407452630 2023-10-16T09:57:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff060f07700 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff19f0 0x55c696fcae00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9878d5e8686636a6 server_cookie=eb4fe2f0663c6e87 in_seq=0 out_seq=0 2023-10-16T09:57:08.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff03b7fe700 1 -- 172.21.15.181:0/192376049 <== mds.0 v2:172.21.15.190:6832/2407452630 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c696f16b20 con 0x55c696ff19f0 2023-10-16T09:57:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff03b7fe700 10 client.6087 handle_command_reply: tid=0 2023-10-16T09:57:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 client.6087 shutdown 2023-10-16T09:57:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 msgr2=0x55c696ff1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff1270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff054000f80 tx=0x7ff054007680 comp rx=0 tx=0).stop 2023-10-16T09:57:08.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 shutdown_connections 2023-10-16T09:57:08.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:6832/2407452630,v1:172.21.15.190:6834/2407452630] conn(0x55c696ff19f0 0x55c696fcae00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c696ff3630 0x55c696ff1270 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c696fc9010 0x55c6970109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff03b7fe700 0 client.6087 ms_handle_reset on v2:172.21.15.190:6832/2407452630 2023-10-16T09:57:08.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 --2- 172.21.15.181:0/192376049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c696fee220 0x55c697010f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.417+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 >> 172.21.15.181:0/192376049 conn(0x55c696ef8d80 msgr2=0x55c696ff90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 shutdown_connections 2023-10-16T09:57:08.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06f355b80 1 -- 172.21.15.181:0/192376049 wait complete. 2023-10-16T09:57:08.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06f355b80 20 client.6087 trim_cache size 0 max 16384 2023-10-16T09:57:08.445 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:57:08.445 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T09:57:08.445 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T09:57:08.445 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 5943, 2023-10-16T09:57:08.445 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T09:57:08.446 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T09:57:08.446 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T09:57:08.446 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9407884270000002, 2023-10-16T09:57:08.446 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 25, 2023-10-16T09:57:08.446 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 59, 2023-10-16T09:57:08.447 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 59, 2023-10-16T09:57:08.447 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.866788397000001 2023-10-16T09:57:08.447 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff04006b020 msgr2=0x7ff04006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff04006b020 0x7ff04006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff050013040 tx=0x7ff050008070 comp rx=0 tx=0).stop 2023-10-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c0d0fc0 msgr2=0x7ff05c0c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c0d0fc0 0x7ff05c0c6980 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff04800b530 tx=0x7ff04800b8f0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 shutdown_connections 2023-10-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff04006b020 0x7ff04006d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff05c06a8a0 0x7ff05c14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff05c15df60 0x7ff05c0c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 --2- 172.21.15.181:0/2908557738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff05c0d0fc0 0x7ff05c0c6980 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 >> 172.21.15.181:0/2908557738 conn(0x7ff05c0d04a0 msgr2=0x7ff05c0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 shutdown_connections 2023-10-16T09:57:08.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.418+0000 7ff06316b700 1 -- 172.21.15.181:0/2908557738 wait complete. 2023-10-16T09:57:08.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:57:08.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e99e8700 1 Processor -- start 2023-10-16T09:57:08.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e99e8700 1 -- start start 2023-10-16T09:57:08.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e99e8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0e40cf860 0x7ff0e4168820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e99e8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40d1640 con 0x7ff0e415d9b0 2023-10-16T09:57:08.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e99e8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40d1920 con 0x7ff0e41621c0 2023-10-16T09:57:08.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e99e8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40d1c00 con 0x7ff0e40cf860 2023-10-16T09:57:08.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff0e415d9b0 0x7ff0e40b39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51460/0 (socket says 172.21.15.181:51460) 2023-10-16T09:57:08.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.572+0000 7ff0e2ffd700 1 -- 172.21.15.181:0/589194320 learned_addr learned my addr 172.21.15.181:0/589194320 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:08.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e37fe700 1 --2- 172.21.15.181:0/589194320 >> v2:172.21.15.190:3300/0 conn(0x7ff0e40cf860 0x7ff0e4168820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462702767 0 0) 0x7ff0e40d1640 con 0x7ff0e415d9b0 2023-10-16T09:57:08.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0cc0035b0 con 0x7ff0e415d9b0 2023-10-16T09:57:08.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433527251 0 0) 0x7ff0cc0035b0 con 0x7ff0e415d9b0 2023-10-16T09:57:08.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 >> v2:172.21.15.190:3300/0 conn(0x7ff0e40cf860 msgr2=0x7ff0e4168820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 --2- 172.21.15.181:0/589194320 >> v2:172.21.15.190:3300/0 conn(0x7ff0e40cf860 0x7ff0e4168820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 >> v1:172.21.15.190:6789/0 conn(0x7ff0e41621c0 legacy=0x7ff0e40cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40d1ee0 con 0x7ff0e415d9b0 2023-10-16T09:57:08.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff0d8001670 con 0x7ff0e415d9b0 2023-10-16T09:57:08.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 >> v1:172.21.15.181:6789/0 conn(0x7ff0e415d9b0 legacy=0x7ff0e40b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0cc003a30 0x7ff0cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0cc005210 0x7ff0cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40d1c00 con 0x7ff0cc004850 2023-10-16T09:57:08.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc00c0e0 con 0x7ff0cc005210 2023-10-16T09:57:08.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc00c2a0 con 0x7ff0cc003a30 2023-10-16T09:57:08.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e37fe700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0cc005210 0x7ff0cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e2ffd700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0cc003a30 0x7ff0cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e27fc700 1 -- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0cc003a30 msgr2=0x7ff0cc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0cc003a30 0x7ff0cc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e27fc700 1 -- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0cc005210 msgr2=0x7ff0cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:08.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0cc005210 0x7ff0cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e27fc700 1 -- 172.21.15.181:0/589194320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0035b0 con 0x7ff0cc004850 2023-10-16T09:57:08.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.573+0000 7ff0e37fe700 1 -- 172.21.15.181:0/589194320 reap_dead start 2023-10-16T09:57:08.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0cc004cd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40017c0 tx=0x7ff0cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fbbf8761c85ffb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40052d0 con 0x7ff0cc004850 2023-10-16T09:57:08.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d4005450 con 0x7ff0cc004850 2023-10-16T09:57:08.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e1ffb700 1 -- 172.21.15.181:0/589194320 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40038c0 con 0x7ff0cc004850 2023-10-16T09:57:08.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e99e8700 1 -- 172.21.15.181:0/589194320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 msgr2=0x7ff0cc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0cc004cd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40017c0 tx=0x7ff0cc005880 comp rx=0 tx=0).stop 2023-10-16T09:57:08.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e99e8700 1 -- 172.21.15.181:0/589194320 shutdown_connections 2023-10-16T09:57:08.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/589194320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0cc004cd0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e99e8700 1 -- 172.21.15.181:0/589194320 >> 172.21.15.181:0/589194320 conn(0x7ff0e40665a0 msgr2=0x7ff0e40669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e99e8700 1 -- 172.21.15.181:0/589194320 shutdown_connections 2023-10-16T09:57:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.574+0000 7ff0e99e8700 1 -- 172.21.15.181:0/589194320 wait complete. 2023-10-16T09:57:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.575+0000 7ff0e99e8700 1 Processor -- start 2023-10-16T09:57:08.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.576+0000 7ff0e99e8700 1 -- start start 2023-10-16T09:57:08.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.576+0000 7ff0e99e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0e41601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.576+0000 7ff0e99e8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40c74b0 0x7ff0e40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.576+0000 7ff0e99e8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e4151580 0x7ff0e4151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.576+0000 7ff0e99e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40c7e70 con 0x7ff0cc004850 2023-10-16T09:57:08.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.576+0000 7ff0e99e8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4151f40 con 0x7ff0e40c74b0 2023-10-16T09:57:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.576+0000 7ff0e99e8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40c86f0 con 0x7ff0e4151580 2023-10-16T09:57:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0e41601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0e41601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40394/0 (socket says 172.21.15.181:40394) 2023-10-16T09:57:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 -- 172.21.15.181:0/1047699014 learned_addr learned my addr 172.21.15.181:0/1047699014 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:08.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40c74b0 0x7ff0e40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e37fe700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e4151580 0x7ff0e4151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 -- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e4151580 msgr2=0x7ff0e4151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e4151580 0x7ff0e4151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 -- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40c74b0 msgr2=0x7ff0e40c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40c74b0 0x7ff0e40c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 -- 172.21.15.181:0/1047699014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40c89d0 con 0x7ff0cc004850 2023-10-16T09:57:08.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40c74b0 0x7ff0e40c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:08.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e2ffd700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0e41601a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80072c0 tx=0x7ff0d8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0cbfff700 1 -- 172.21.15.181:0/1047699014 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80077a0 con 0x7ff0cc004850 2023-10-16T09:57:08.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0cbfff700 1 -- 172.21.15.181:0/1047699014 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d800f8d0 con 0x7ff0cc004850 2023-10-16T09:57:08.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e40c8cb0 con 0x7ff0cc004850 2023-10-16T09:57:08.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0cbfff700 1 -- 172.21.15.181:0/1047699014 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800fc80 con 0x7ff0cc004850 2023-10-16T09:57:08.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.577+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e40c9270 con 0x7ff0cc004850 2023-10-16T09:57:08.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.578+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e404ebb0 con 0x7ff0cc004850 2023-10-16T09:57:08.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.579+0000 7ff0cbfff700 1 -- 172.21.15.181:0/1047699014 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff0d801d460 con 0x7ff0cc004850 2023-10-16T09:57:08.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.579+0000 7ff0cbfff700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff0d006b020 0x7ff0d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:08.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.579+0000 7ff0cbfff700 1 -- 172.21.15.181:0/1047699014 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7ff0d8089930 con 0x7ff0cc004850 2023-10-16T09:57:08.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.579+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff0d006b020 0x7ff0d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:08.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.579+0000 7ff0e27fc700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff0d006b020 0x7ff0d006d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4000c00 tx=0x7ff0d4009030 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:08.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.583+0000 7ff0cbfff700 1 -- 172.21.15.181:0/1047699014 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0d8054740 con 0x7ff0cc004850 2023-10-16T09:57:08.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.816+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff0e41591a0 con 0x7ff0cc004850 2023-10-16T09:57:08.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.817+0000 7ff0cbfff700 1 -- 172.21.15.181:0/1047699014 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff0d8057d60 con 0x7ff0cc004850 2023-10-16T09:57:08.819 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:57:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff0d006b020 msgr2=0x7ff0d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff0d006b020 0x7ff0d006d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4000c00 tx=0x7ff0d4009030 comp rx=0 tx=0).stop 2023-10-16T09:57:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 msgr2=0x7ff0e41601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:08.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0e41601a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80072c0 tx=0x7ff0d8007680 comp rx=0 tx=0).stop 2023-10-16T09:57:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 shutdown_connections 2023-10-16T09:57:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7ff0d006b020 0x7ff0d006d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e4151580 0x7ff0e4151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc004850 0x7ff0e41601a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 --2- 172.21.15.181:0/1047699014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40c74b0 0x7ff0e40c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 >> 172.21.15.181:0/1047699014 conn(0x7ff0e40665a0 msgr2=0x7ff0e40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 shutdown_connections 2023-10-16T09:57:08.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:08.819+0000 7ff0e99e8700 1 -- 172.21.15.181:0/1047699014 wait complete. 2023-10-16T09:57:08.833 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T09:57:08.833 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:57:08.834 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T09:57:08.834 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T09:57:08.834 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:57:08.835 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T09:57:08.835 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T09:57:08.835 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:57:08.836 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:57:08.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:08.836 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T09:57:08.853 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:57:08.854 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:57:08.854 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:57:08.854 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:57:08.854 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T09:57:08.854 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:57:08.855 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:57:08.855 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:57:08.855 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:57:08.855 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:57:08.855 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:57:08.856 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T09:57:08.856 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:57:08.856 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T09:57:08.856 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:57:08.856 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41640sec preferred_lft 41640sec 2023-10-16T09:57:08.857 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T09:57:08.857 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:57:08.857 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:57:08.857 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:57:08.857 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:57:08.857 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:57:08.858 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T09:57:08.858 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:57:08.858 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:57:08.858 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:57:08.858 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T09:57:08.859 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:57:08.859 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:57:08.859 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T09:57:08.859 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T09:57:08.859 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:57:08.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:08.860 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:57:08.918 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T09:57:08.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:08.920 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T09:57:08.979 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T09:57:08.979 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:57:08.979 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:57:08.980 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:57:08.980 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:57:08.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:08.980 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:57:08.981 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:57:08.981 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:57:08.981 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:57:08.982 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:57:08.982 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:57:09.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:09.084 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T09:57:09.100 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T09:57:09.101 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T09:57:09.101 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:57:09.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:09.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T09:57:09.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:57:09.167 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T09:57:09.223 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:57:09.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:57:09.223 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:57:09.279 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T09:57:09.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:09.280 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:57:09.365 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:57:09.366 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:57:09.366 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:57:09.374 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:57:09.374 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:57:09.375 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:57:09.375 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:57:09.375 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:57:09.402 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:57:09.402 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:57:09.403 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:57:09.403 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:57:09.403 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:57:09.403 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T09:57:09.403 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T09:57:09.404 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T09:57:09.421 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:57:09.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:09.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:57:09.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:09.449 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:57:09.536 INFO:teuthology.orchestra.run.smithi181.stdout:client6096 2023-10-16T09:57:09.536 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:57:09.537 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:57:09.553 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T09:57:09.617 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:09.638 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:09.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.718+0000 7fc93d8d4700 1 Processor -- start 2023-10-16T09:57:09.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.718+0000 7fc93d8d4700 1 -- start start 2023-10-16T09:57:09.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc93d8d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc938160f00 0x7fc9380d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:09.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc93d8d4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380ce2e0 con 0x7fc9380d2550 2023-10-16T09:57:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc93d8d4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380b09e0 con 0x7fc9380b0500 2023-10-16T09:57:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc93d8d4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380b0cc0 con 0x7fc938160f00 2023-10-16T09:57:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc936ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc938160f00 0x7fc9380d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc936ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc938160f00 0x7fc9380d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40406/0 (socket says 172.21.15.181:40406) 2023-10-16T09:57:09.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc9377fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc9380d2550 0x7fc9380b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51484/0 (socket says 172.21.15.181:51484) 2023-10-16T09:57:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc936ffd700 1 -- 172.21.15.181:0/2519454398 learned_addr learned my addr 172.21.15.181:0/2519454398 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc936ffd700 1 -- 172.21.15.181:0/2519454398 >> v1:172.21.15.190:6789/0 conn(0x7fc9380b0500 legacy=0x7fc9380b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc936ffd700 1 -- 172.21.15.181:0/2519454398 >> v1:172.21.15.181:6789/0 conn(0x7fc9380d2550 legacy=0x7fc9380b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.719+0000 7fc936ffd700 1 -- 172.21.15.181:0/2519454398 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380b0fa0 con 0x7fc938160f00 2023-10-16T09:57:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc936ffd700 1 --2- 172.21.15.181:0/2519454398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc938160f00 0x7fc9380d19b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc92c006fd0 tx=0x7fc92c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9e11c968ce0f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:09.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc935ffb700 1 -- 172.21.15.181:0/2519454398 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c005e30 con 0x7fc938160f00 2023-10-16T09:57:09.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc935ffb700 1 -- 172.21.15.181:0/2519454398 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c004030 con 0x7fc938160f00 2023-10-16T09:57:09.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc935ffb700 1 -- 172.21.15.181:0/2519454398 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0043e0 con 0x7fc938160f00 2023-10-16T09:57:09.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc93d8d4700 1 -- 172.21.15.181:0/2519454398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc938160f00 msgr2=0x7fc9380d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:09.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/2519454398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc938160f00 0x7fc9380d19b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc92c006fd0 tx=0x7fc92c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc93d8d4700 1 -- 172.21.15.181:0/2519454398 shutdown_connections 2023-10-16T09:57:09.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/2519454398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc938160f00 0x7fc9380d19b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc93d8d4700 1 -- 172.21.15.181:0/2519454398 >> 172.21.15.181:0/2519454398 conn(0x7fc93809cc10 msgr2=0x7fc9380d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:09.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc93d8d4700 1 -- 172.21.15.181:0/2519454398 shutdown_connections 2023-10-16T09:57:09.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.720+0000 7fc93d8d4700 1 -- 172.21.15.181:0/2519454398 wait complete. 2023-10-16T09:57:09.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.721+0000 7fc93d8d4700 1 Processor -- start 2023-10-16T09:57:09.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc93d8d4700 1 -- start start 2023-10-16T09:57:09.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc93d8d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9380b0500 0x7fc938155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:09.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc93d8d4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9380d2550 0x7fc9381f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:09.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc93d8d4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938160f00 0x7fc9380ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:09.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc93d8d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc938154e20 con 0x7fc9380b0500 2023-10-16T09:57:09.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc93d8d4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9381560b0 con 0x7fc9380d2550 2023-10-16T09:57:09.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc93d8d4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc938155090 con 0x7fc938160f00 2023-10-16T09:57:09.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc936ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9380b0500 0x7fc938155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc936ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9380b0500 0x7fc938155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40412/0 (socket says 172.21.15.181:40412) 2023-10-16T09:57:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc936ffd700 1 -- 172.21.15.181:0/1241264819 learned_addr learned my addr 172.21.15.181:0/1241264819 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.722+0000 7fc9377fe700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938160f00 0x7fc9380ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc9367fc700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9380d2550 0x7fc9381f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc936ffd700 1 -- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938160f00 msgr2=0x7fc9380ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:09.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc936ffd700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938160f00 0x7fc9380ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc936ffd700 1 -- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9380d2550 msgr2=0x7fc9381f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc936ffd700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9380d2550 0x7fc9381f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc936ffd700 1 -- 172.21.15.181:0/1241264819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c009500 con 0x7fc9380b0500 2023-10-16T09:57:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc936ffd700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9380b0500 0x7fc938155b70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009c10 tx=0x7fc92c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:09.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc91ffff700 1 -- 172.21.15.181:0/1241264819 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c004580 con 0x7fc9380b0500 2023-10-16T09:57:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc91ffff700 1 -- 172.21.15.181:0/1241264819 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c011420 con 0x7fc9380b0500 2023-10-16T09:57:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc938155370 con 0x7fc9380b0500 2023-10-16T09:57:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc91ffff700 1 -- 172.21.15.181:0/1241264819 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0117d0 con 0x7fc9380b0500 2023-10-16T09:57:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.723+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9381f31b0 con 0x7fc9380b0500 2023-10-16T09:57:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.724+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc938155890 con 0x7fc9380b0500 2023-10-16T09:57:09.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.724+0000 7fc91ffff700 1 -- 172.21.15.181:0/1241264819 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc92c011930 con 0x7fc9380b0500 2023-10-16T09:57:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.725+0000 7fc91ffff700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc92406b020 0x7fc92406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.725+0000 7fc91ffff700 1 -- 172.21.15.181:0/1241264819 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7fc92c08aad0 con 0x7fc9380b0500 2023-10-16T09:57:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.725+0000 7fc9367fc700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc92406b020 0x7fc92406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.725+0000 7fc9367fc700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc92406b020 0x7fc92406d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9380a1700 tx=0x7fc928009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:09.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.729+0000 7fc91ffff700 1 -- 172.21.15.181:0/1241264819 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc938155890 con 0x7fc9380b0500 2023-10-16T09:57:09.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.950+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 --> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9380b1280 con 0x7fc92406b020 2023-10-16T09:57:09.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.951+0000 7fc91ffff700 1 -- 172.21.15.181:0/1241264819 <== mgr.5685 v2:172.21.15.181:6832/3219577148 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc928001d60 con 0x7fc92406b020 2023-10-16T09:57:09.953 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:57:09.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:57:09.953 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T09:57:09.954 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:57:09.954 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:57:09.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc92406b020 msgr2=0x7fc92406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:09.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc92406b020 0x7fc92406d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9380a1700 tx=0x7fc928009040 comp rx=0 tx=0).stop 2023-10-16T09:57:09.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9380b0500 msgr2=0x7fc938155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:09.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9380b0500 0x7fc938155b70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009c10 tx=0x7fc92c004750 comp rx=0 tx=0).stop 2023-10-16T09:57:09.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 shutdown_connections 2023-10-16T09:57:09.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fc92406b020 0x7fc92406d4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc938160f00 0x7fc9380ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9380b0500 0x7fc938155b70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 --2- 172.21.15.181:0/1241264819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9380d2550 0x7fc9381f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:09.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 >> 172.21.15.181:0/1241264819 conn(0x7fc93809cc10 msgr2=0x7fc9380b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:09.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 shutdown_connections 2023-10-16T09:57:09.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:09.954+0000 7fc93d8d4700 1 -- 172.21.15.181:0/1241264819 wait complete. 2023-10-16T09:57:09.967 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T09:57:10.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.128+0000 7fb257b2f700 1 Processor -- start 2023-10-16T09:57:10.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.128+0000 7fb257b2f700 1 -- start start 2023-10-16T09:57:10.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.128+0000 7fb257b2f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 0x7fb250166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb257b2f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2500b1f70 con 0x7fb2500ce6c0 2023-10-16T09:57:10.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb257b2f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2500b2ae0 con 0x7fb25015f150 2023-10-16T09:57:10.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb257b2f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500cd880 con 0x7fb2501646b0 2023-10-16T09:57:10.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2550ca700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb2500ce6c0 0x7fb2500cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51496/0 (socket says 172.21.15.181:51496) 2023-10-16T09:57:10.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2550ca700 1 -- 172.21.15.181:0/4128095312 learned_addr learned my addr 172.21.15.181:0/4128095312 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:10.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2560cc700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 0x7fb250166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999854225 0 0) 0x7fb2500b1f70 con 0x7fb2500ce6c0 2023-10-16T09:57:10.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb23c0035b0 con 0x7fb2500ce6c0 2023-10-16T09:57:10.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200616580 0 0) 0x7fb23c0035b0 con 0x7fb2500ce6c0 2023-10-16T09:57:10.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 msgr2=0x7fb250166aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:10.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 0x7fb250166aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 >> v1:172.21.15.190:6789/0 conn(0x7fb25015f150 legacy=0x7fb2500cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500b2dc0 con 0x7fb2500ce6c0 2023-10-16T09:57:10.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fb24c001670 con 0x7fb2500ce6c0 2023-10-16T09:57:10.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 >> v1:172.21.15.181:6789/0 conn(0x7fb2500ce6c0 legacy=0x7fb2500cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c003a30 0x7fb23c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 0x7fb23c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb23c004d90 0x7fb23c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500cd880 con 0x7fb23c004d90 2023-10-16T09:57:10.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c00bac0 con 0x7fb23c003a30 2023-10-16T09:57:10.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c00bc80 con 0x7fb2501646b0 2023-10-16T09:57:10.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2560cc700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb23c004d90 0x7fb23c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2558cb700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c003a30 0x7fb23c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.129+0000 7fb2550ca700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 0x7fb23c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2560cc700 1 -- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 msgr2=0x7fb23c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:10.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2560cc700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 0x7fb23c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2560cc700 1 -- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c003a30 msgr2=0x7fb23c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:10.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2560cc700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c003a30 0x7fb23c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2560cc700 1 -- 172.21.15.181:0/4128095312 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0035b0 con 0x7fb23c004d90 2023-10-16T09:57:10.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2558cb700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb23c003a30 0x7fb23c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:10.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2550ca700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2501646b0 0x7fb23c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:10.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2560cc700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb23c004d90 0x7fb23c005210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb2500ceb90 tx=0x7fb23c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f87774b2f59d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:10.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800ebd0 con 0x7fb23c004d90 2023-10-16T09:57:10.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24800f040 con 0x7fb23c004d90 2023-10-16T09:57:10.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2548c9700 1 -- 172.21.15.181:0/4128095312 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248014480 con 0x7fb23c004d90 2023-10-16T09:57:10.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb257b2f700 1 -- 172.21.15.181:0/4128095312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb23c004d90 msgr2=0x7fb23c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb257b2f700 1 --2- 172.21.15.181:0/4128095312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb23c004d90 0x7fb23c005210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb2500ceb90 tx=0x7fb23c005880 comp rx=0 tx=0).stop 2023-10-16T09:57:10.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb2560cc700 1 -- 172.21.15.181:0/4128095312 reap_dead start 2023-10-16T09:57:10.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb257b2f700 1 -- 172.21.15.181:0/4128095312 shutdown_connections 2023-10-16T09:57:10.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.130+0000 7fb257b2f700 1 -- 172.21.15.181:0/4128095312 >> 172.21.15.181:0/4128095312 conn(0x7fb25009a930 msgr2=0x7fb2500d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:10.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.131+0000 7fb257b2f700 1 -- 172.21.15.181:0/4128095312 shutdown_connections 2023-10-16T09:57:10.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.131+0000 7fb257b2f700 1 -- 172.21.15.181:0/4128095312 wait complete. 2023-10-16T09:57:10.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.132+0000 7fb257b2f700 1 Processor -- start 2023-10-16T09:57:10.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.132+0000 7fb257b2f700 1 -- start start 2023-10-16T09:57:10.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c004d90 0x7fb2500ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2500b6d90 0x7fb2500b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2500b4680 0x7fb2500c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500af0a0 con 0x7fb2500b6d90 2023-10-16T09:57:10.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501543a0 con 0x7fb2500b4680 2023-10-16T09:57:10.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500aeae0 con 0x7fb23c004d90 2023-10-16T09:57:10.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2500b6d90 0x7fb2500b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2500b6d90 0x7fb2500b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40432/0 (socket says 172.21.15.181:40432) 2023-10-16T09:57:10.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 -- 172.21.15.181:0/2431743033 learned_addr learned my addr 172.21.15.181:0/2431743033 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:10.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2558cb700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c004d90 0x7fb2500ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2560cc700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2500b4680 0x7fb2500c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 -- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c004d90 msgr2=0x7fb2500ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c004d90 0x7fb2500ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 -- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2500b4680 msgr2=0x7fb2500c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2500b4680 0x7fb2500c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 -- 172.21.15.181:0/2431743033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500aedc0 con 0x7fb2500b6d90 2023-10-16T09:57:10.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2560cc700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2500b4680 0x7fb2500c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:10.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2550ca700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2500b6d90 0x7fb2500b4140 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00eb20 tx=0x7fb24c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:10.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2467fc700 1 -- 172.21.15.181:0/2431743033 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00c780 con 0x7fb2500b6d90 2023-10-16T09:57:10.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2467fc700 1 -- 172.21.15.181:0/2431743033 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c00c8e0 con 0x7fb2500b6d90 2023-10-16T09:57:10.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb250154030 con 0x7fb2500b6d90 2023-10-16T09:57:10.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb2467fc700 1 -- 172.21.15.181:0/2431743033 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00cc50 con 0x7fb2500b6d90 2023-10-16T09:57:10.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.133+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb250153990 con 0x7fb2500b6d90 2023-10-16T09:57:10.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.134+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb250153e90 con 0x7fb2500b6d90 2023-10-16T09:57:10.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.135+0000 7fb2467fc700 1 -- 172.21.15.181:0/2431743033 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb24c01d460 con 0x7fb2500b6d90 2023-10-16T09:57:10.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.135+0000 7fb2467fc700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb23406b020 0x7fb23406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.135+0000 7fb2467fc700 1 -- 172.21.15.181:0/2431743033 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7fb24c0897e0 con 0x7fb2500b6d90 2023-10-16T09:57:10.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.135+0000 7fb2558cb700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb23406b020 0x7fb23406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.136+0000 7fb2558cb700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb23406b020 0x7fb23406d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb240000c00 tx=0x7fb240009040 comp rx=0 tx=0).ready entity=mgr.5685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:10.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.140+0000 7fb2467fc700 1 -- 172.21.15.181:0/2431743033 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb250153e90 con 0x7fb2500b6d90 2023-10-16T09:57:10.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.389+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb250153e90 con 0x7fb2500b6d90 2023-10-16T09:57:10.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.534+0000 7fb2467fc700 1 -- 172.21.15.181:0/2431743033 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb250153e90 con 0x7fb2500b6d90 2023-10-16T09:57:10.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.538+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb23406b020 msgr2=0x7fb23406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.538+0000 7fb257b2f700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb23406b020 0x7fb23406d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb240000c00 tx=0x7fb240009040 comp rx=0 tx=0).stop 2023-10-16T09:57:10.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.538+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2500b6d90 msgr2=0x7fb2500b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.538+0000 7fb257b2f700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2500b6d90 0x7fb2500b4140 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00eb20 tx=0x7fb24c00eee0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.539+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 shutdown_connections 2023-10-16T09:57:10.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.539+0000 7fb257b2f700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7fb23406b020 0x7fb23406d4e0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb240000c00 tx=0x7fb240009040 comp rx=0 tx=0).stop 2023-10-16T09:57:10.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.539+0000 7fb257b2f700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb23c004d90 0x7fb2500ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.539+0000 7fb257b2f700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2500b4680 0x7fb2500c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.539+0000 7fb257b2f700 1 --2- 172.21.15.181:0/2431743033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2500b6d90 0x7fb2500b4140 secure :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00eb20 tx=0x7fb24c00eee0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.539+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 >> 172.21.15.181:0/2431743033 conn(0x7fb25009a930 msgr2=0x7fb250158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:10.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.539+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 shutdown_connections 2023-10-16T09:57:10.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.540+0000 7fb257b2f700 1 -- 172.21.15.181:0/2431743033 wait complete. 2023-10-16T09:57:10.555 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T09:57:10.680 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:10.679+0000 7f9a64597200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:57:10.683 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:10.681+0000 7f10dbf0c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:57:10.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.682+0000 7f6bf48a2700 1 Processor -- start 2023-10-16T09:57:10.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.682+0000 7f6bf48a2700 1 -- start start 2023-10-16T09:57:10.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.682+0000 7f6bf48a2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6bf00ce2d0 0x7f6bf015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.682+0000 7f6bf48a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf00d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bf48a2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf015ef80 con 0x7f6bf00d01d0 2023-10-16T09:57:10.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bf48a2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf00d4290 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bf48a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf00ce750 con 0x7f6bf00d1500 2023-10-16T09:57:10.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6bf00ce2d0 0x7f6bf015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6bf00ce2d0 0x7f6bf015ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40464/0 (socket says 172.21.15.181:40464) 2023-10-16T09:57:10.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 -- 172.21.15.181:0/2070929280 learned_addr learned my addr 172.21.15.181:0/2070929280 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:10.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6beed9d700 1 --2- 172.21.15.181:0/2070929280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf00d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 -- 172.21.15.181:0/2070929280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 msgr2=0x7f6bf00d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:10.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 --2- 172.21.15.181:0/2070929280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf00d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 -- 172.21.15.181:0/2070929280 >> v1:172.21.15.190:6789/0 conn(0x7f6bf00d01d0 legacy=0x7f6bf00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 -- 172.21.15.181:0/2070929280 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf00cf580 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bedd9b700 1 --2- 172.21.15.181:0/2070929280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf015ea40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009920 tx=0x7f6be4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171c2788165e5eab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:10.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bed59a700 1 -- 172.21.15.181:0/2070929280 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4011830 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bed59a700 1 -- 172.21.15.181:0/2070929280 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be4011990 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.683+0000 7f6bed59a700 1 -- 172.21.15.181:0/2070929280 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4011d20 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 -- 172.21.15.181:0/2070929280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 msgr2=0x7f6bf015ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/2070929280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf015ea40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009920 tx=0x7f6be4009c30 comp rx=0 tx=0).stop 2023-10-16T09:57:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 -- 172.21.15.181:0/2070929280 shutdown_connections 2023-10-16T09:57:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/2070929280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf00d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/2070929280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf015ea40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 -- 172.21.15.181:0/2070929280 >> 172.21.15.181:0/2070929280 conn(0x7f6bf009cb60 msgr2=0x7f6bf00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:10.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 -- 172.21.15.181:0/2070929280 shutdown_connections 2023-10-16T09:57:10.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.684+0000 7f6bf48a2700 1 -- 172.21.15.181:0/2070929280 wait complete. 2023-10-16T09:57:10.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.685+0000 7f6bf48a2700 1 Processor -- start 2023-10-16T09:57:10.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bf48a2700 1 -- start start 2023-10-16T09:57:10.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bf48a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf0151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bf48a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf00d01d0 0x7f6bf00ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bf48a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bf48a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf0152390 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bf48a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf00c8020 con 0x7f6bf00d01d0 2023-10-16T09:57:10.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bf48a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf00c8300 con 0x7f6bf00d1500 2023-10-16T09:57:10.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf0151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf0151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40468/0 (socket says 172.21.15.181:40468) 2023-10-16T09:57:10.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bee59c700 1 -- 172.21.15.181:0/599263194 learned_addr learned my addr 172.21.15.181:0/599263194 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:10.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6bedd9b700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf00d01d0 0x7f6bf00ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.686+0000 7f6beed9d700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:10.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bee59c700 1 -- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 msgr2=0x7f6bf015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bee59c700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bee59c700 1 -- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf00d01d0 msgr2=0x7f6bf00ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bee59c700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf00d01d0 0x7f6bf00ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bee59c700 1 -- 172.21.15.181:0/599263194 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be4009550 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bee59c700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf0151e50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6be000c4e0 tx=0x7f6be000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:10.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bdeffd700 1 -- 172.21.15.181:0/599263194 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be00052d0 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bdeffd700 1 -- 172.21.15.181:0/599263194 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be0005430 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bdeffd700 1 -- 172.21.15.181:0/599263194 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be00038c0 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf00c85e0 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.687+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf00cd560 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.688+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf00c8b00 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.688+0000 7f6bdeffd700 1 -- 172.21.15.181:0/599263194 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 20) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f6be0003a20 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.689+0000 7f6bdeffd700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 0x7f6bd006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:10.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.689+0000 7f6bedd9b700 1 -- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 msgr2=0x7f6bd006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:10.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.689+0000 7f6bdeffd700 1 -- 172.21.15.181:0/599263194 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f6be0088b20 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.689+0000 7f6bedd9b700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 0x7f6bd006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:57:10.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.693+0000 7f6bdeffd700 1 -- 172.21.15.181:0/599263194 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6bf00c8b00 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.747 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:10.746+0000 7f9a64597200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:57:10.752 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:10.751+0000 7f10dbf0c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:57:10.820 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:10.818+0000 7f9a64597200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:57:10.826 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:10.825+0000 7f10dbf0c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:57:10.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.890+0000 7f6bedd9b700 1 -- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 msgr2=0x7f6bd006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:10.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.890+0000 7f6bedd9b700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 0x7f6bd006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:57:10.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.893+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f6bf015ef80 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.896 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:10.895+0000 7f9a64597200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:57:10.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.897+0000 7f6bdeffd700 1 -- 172.21.15.181:0/599263194 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6bf015ef80 con 0x7f6bf00ce2d0 2023-10-16T09:57:10.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 msgr2=0x7f6bd006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:57:10.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 0x7f6bd006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 msgr2=0x7f6bf0151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:10.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf0151e50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6be000c4e0 tx=0x7f6be000c7f0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 shutdown_connections 2023-10-16T09:57:10.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f6bd006b070 0x7f6bd006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bf00d1500 0x7f6bf015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bf00ce2d0 0x7f6bf0151e50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 --2- 172.21.15.181:0/599263194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf00d01d0 0x7f6bf00ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:10.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 >> 172.21.15.181:0/599263194 conn(0x7f6bf009cb60 msgr2=0x7f6bf015be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:10.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 shutdown_connections 2023-10-16T09:57:10.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:10.899+0000 7f6bf48a2700 1 -- 172.21.15.181:0/599263194 wait complete. 2023-10-16T09:57:10.906 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:10.905+0000 7f10dbf0c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:57:10.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T09:57:11.036 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:11.035+0000 7f9a64597200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:57:11.053 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:11.052+0000 7f10dbf0c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:57:11.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.112+0000 7f3f59906700 1 Processor -- start 2023-10-16T09:57:11.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.112+0000 7f3f59906700 1 -- start start 2023-10-16T09:57:11.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f59906700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f540b2010 0x7f3f540cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f59906700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f54071be0 0x7f3f540b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f59906700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540ce1f0 con 0x7f3f5406f3b0 2023-10-16T09:57:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f59906700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540b2930 con 0x7f3f540b2010 2023-10-16T09:57:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f59906700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540ccda0 con 0x7f3f54071be0 2023-10-16T09:57:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f52ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f540b2010 0x7f3f540cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f537fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3f5406f3b0 0x7f3f5406f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51590/0 (socket says 172.21.15.181:51590) 2023-10-16T09:57:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f537fe700 1 -- 172.21.15.181:0/745630984 learned_addr learned my addr 172.21.15.181:0/745630984 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.113+0000 7f3f527fc700 1 --2- 172.21.15.181:0/745630984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f54071be0 0x7f3f540b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f52ffd700 1 -- 172.21.15.181:0/745630984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f54071be0 msgr2=0x7f3f540b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f52ffd700 1 --2- 172.21.15.181:0/745630984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f54071be0 0x7f3f540b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f52ffd700 1 -- 172.21.15.181:0/745630984 >> v1:172.21.15.181:6789/0 conn(0x7f3f5406f3b0 legacy=0x7f3f5406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f51ffb700 1 -- 172.21.15.181:0/745630984 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642048516 0 0) 0x7f3f540ce1f0 con 0x7f3f5406f3b0 2023-10-16T09:57:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f52ffd700 1 -- 172.21.15.181:0/745630984 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540727e0 con 0x7f3f540b2010 2023-10-16T09:57:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f52ffd700 1 --2- 172.21.15.181:0/745630984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540b2010 0x7f3f540cc6e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400bae0 tx=0x7f3f4400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e68877187874045 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f51ffb700 1 -- 172.21.15.181:0/745630984 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4400dc70 con 0x7f3f540b2010 2023-10-16T09:57:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f51ffb700 1 -- 172.21.15.181:0/745630984 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4400ddd0 con 0x7f3f540b2010 2023-10-16T09:57:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.114+0000 7f3f51ffb700 1 -- 172.21.15.181:0/745630984 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44010690 con 0x7f3f540b2010 2023-10-16T09:57:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 -- 172.21.15.181:0/745630984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540b2010 msgr2=0x7f3f540cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 --2- 172.21.15.181:0/745630984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540b2010 0x7f3f540cc6e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400bae0 tx=0x7f3f4400bdf0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 -- 172.21.15.181:0/745630984 shutdown_connections 2023-10-16T09:57:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 --2- 172.21.15.181:0/745630984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540b2010 0x7f3f540cc6e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 --2- 172.21.15.181:0/745630984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f54071be0 0x7f3f540b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 -- 172.21.15.181:0/745630984 >> 172.21.15.181:0/745630984 conn(0x7f3f540cd010 msgr2=0x7f3f540cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 -- 172.21.15.181:0/745630984 shutdown_connections 2023-10-16T09:57:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.115+0000 7f3f59906700 1 -- 172.21.15.181:0/745630984 wait complete. 2023-10-16T09:57:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.116+0000 7f3f59906700 1 Processor -- start 2023-10-16T09:57:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.116+0000 7f3f59906700 1 -- start start 2023-10-16T09:57:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f59906700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5406f3b0 0x7f3f54151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f59906700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f54071be0 0x7f3f541539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f59906700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f540b2010 0x7f3f540ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f59906700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54153ef0 con 0x7f3f54071be0 2023-10-16T09:57:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f59906700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cf040 con 0x7f3f5406f3b0 2023-10-16T09:57:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f59906700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cf320 con 0x7f3f540b2010 2023-10-16T09:57:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f54071be0 0x7f3f541539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f54071be0 0x7f3f541539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40520/0 (socket says 172.21.15.181:40520) 2023-10-16T09:57:11.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 -- 172.21.15.181:0/2846371805 learned_addr learned my addr 172.21.15.181:0/2846371805 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:11.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f537fe700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f540b2010 0x7f3f540ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f52ffd700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5406f3b0 0x7f3f54151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 -- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f540b2010 msgr2=0x7f3f540ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f540b2010 0x7f3f540ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 -- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5406f3b0 msgr2=0x7f3f54151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5406f3b0 0x7f3f54151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.117+0000 7f3f527fc700 1 -- 172.21.15.181:0/2846371805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4400b740 con 0x7f3f54071be0 2023-10-16T09:57:11.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.118+0000 7f3f527fc700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f54071be0 0x7f3f541539b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800e9b0 tx=0x7f3f4800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:11.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.118+0000 7f3f3bfff700 1 -- 172.21.15.181:0/2846371805 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48009710 con 0x7f3f54071be0 2023-10-16T09:57:11.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.118+0000 7f3f3bfff700 1 -- 172.21.15.181:0/2846371805 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4800ee50 con 0x7f3f54071be0 2023-10-16T09:57:11.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.118+0000 7f3f3bfff700 1 -- 172.21.15.181:0/2846371805 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800ca50 con 0x7f3f54071be0 2023-10-16T09:57:11.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.118+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f540cf600 con 0x7f3f54071be0 2023-10-16T09:57:11.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.118+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f540d0490 con 0x7f3f54071be0 2023-10-16T09:57:11.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.119+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f540c7640 con 0x7f3f54071be0 2023-10-16T09:57:11.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.120+0000 7f3f3bfff700 1 -- 172.21.15.181:0/2846371805 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 20) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f3f4800cbb0 con 0x7f3f54071be0 2023-10-16T09:57:11.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.120+0000 7f3f3bfff700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 0x7f3f4006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.120+0000 7f3f52ffd700 1 -- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 msgr2=0x7f3f4006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:11.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.120+0000 7f3f3bfff700 1 -- 172.21.15.181:0/2846371805 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f3f48089e80 con 0x7f3f54071be0 2023-10-16T09:57:11.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.120+0000 7f3f52ffd700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 0x7f3f4006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:57:11.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.124+0000 7f3f3bfff700 1 -- 172.21.15.181:0/2846371805 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3f48054d70 con 0x7f3f54071be0 2023-10-16T09:57:11.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.321+0000 7f3f52ffd700 1 -- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 msgr2=0x7f3f4006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:11.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.321+0000 7f3f52ffd700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 0x7f3f4006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:57:11.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.333+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f3f540c7a20 con 0x7f3f54071be0 2023-10-16T09:57:11.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.336+0000 7f3f3bfff700 1 -- 172.21.15.181:0/2846371805 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3f540c7a20 con 0x7f3f54071be0 2023-10-16T09:57:11.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 msgr2=0x7f3f4006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:57:11.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 0x7f3f4006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f54071be0 msgr2=0x7f3f541539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f54071be0 0x7f3f541539b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800e9b0 tx=0x7f3f4800ecc0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 shutdown_connections 2023-10-16T09:57:11.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f3f4006b070 0x7f3f4006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f540b2010 0x7f3f540ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5406f3b0 0x7f3f54151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 --2- 172.21.15.181:0/2846371805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f54071be0 0x7f3f541539b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.338+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 >> 172.21.15.181:0/2846371805 conn(0x7f3f540cd010 msgr2=0x7f3f541598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:11.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.339+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 shutdown_connections 2023-10-16T09:57:11.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.339+0000 7f3f59906700 1 -- 172.21.15.181:0/2846371805 wait complete. 2023-10-16T09:57:11.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:11.353 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T09:57:11.359 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:11.359+0000 7f9a64597200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:57:11.373 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T09:57:11.400 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:11.398+0000 7f10dbf0c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:57:11.479 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:11.478+0000 7f9a64597200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:57:11.524 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:11.522+0000 7f10dbf0c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:57:11.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.526+0000 7f2f2d4df700 1 Processor -- start 2023-10-16T09:57:11.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.526+0000 7f2f2d4df700 1 -- start start 2023-10-16T09:57:11.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f2d4df700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f2815da70 0x7f2f280ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f2d4df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f2806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f2d4df700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f280cef00 con 0x7f2f28072760 2023-10-16T09:57:11.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f2d4df700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f280cec20 con 0x7f2f2815da70 2023-10-16T09:57:11.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f2d4df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28071c80 con 0x7f2f280ce6a0 2023-10-16T09:57:11.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f26ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f2815da70 0x7f2f280ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f267fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f2806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f277fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2f28072760 0x7f2f281615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:35586/0 (socket says 172.21.15.181:35586) 2023-10-16T09:57:11.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f277fe700 1 -- 172.21.15.181:0/3188260500 learned_addr learned my addr 172.21.15.181:0/3188260500 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:11.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f267fc700 1 -- 172.21.15.181:0/3188260500 >> v2:172.21.15.190:3300/0 conn(0x7f2f2815da70 msgr2=0x7f2f280ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:11.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f267fc700 1 --2- 172.21.15.181:0/3188260500 >> v2:172.21.15.190:3300/0 conn(0x7f2f2815da70 0x7f2f280ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f267fc700 1 -- 172.21.15.181:0/3188260500 >> v1:172.21.15.190:6789/0 conn(0x7f2f28072760 legacy=0x7f2f281615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f267fc700 1 -- 172.21.15.181:0/3188260500 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28071f60 con 0x7f2f280ce6a0 2023-10-16T09:57:11.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f25ffb700 1 -- 172.21.15.181:0/3188260500 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83290640 0 0) 0x7f2f280cef00 con 0x7f2f28072760 2023-10-16T09:57:11.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f26ffd700 1 --2- 172.21.15.181:0/3188260500 >> v2:172.21.15.190:3300/0 conn(0x7f2f2815da70 0x7f2f280ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:57:11.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.527+0000 7f2f267fc700 1 --2- 172.21.15.181:0/3188260500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f2806a8a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009840 tx=0x7f2f1c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3f1fe88b96e8d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:11.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f25ffb700 1 -- 172.21.15.181:0/3188260500 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c011830 con 0x7f2f280ce6a0 2023-10-16T09:57:11.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f25ffb700 1 -- 172.21.15.181:0/3188260500 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c011990 con 0x7f2f280ce6a0 2023-10-16T09:57:11.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f25ffb700 1 -- 172.21.15.181:0/3188260500 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c011d20 con 0x7f2f280ce6a0 2023-10-16T09:57:11.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 -- 172.21.15.181:0/3188260500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 msgr2=0x7f2f2806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 --2- 172.21.15.181:0/3188260500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f2806a8a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009840 tx=0x7f2f1c009b50 comp rx=0 tx=0).stop 2023-10-16T09:57:11.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 -- 172.21.15.181:0/3188260500 shutdown_connections 2023-10-16T09:57:11.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 --2- 172.21.15.181:0/3188260500 >> v2:172.21.15.190:3300/0 conn(0x7f2f2815da70 0x7f2f280ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 --2- 172.21.15.181:0/3188260500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f2806a8a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 -- 172.21.15.181:0/3188260500 >> 172.21.15.181:0/3188260500 conn(0x7f2f280cd010 msgr2=0x7f2f280cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:11.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 -- 172.21.15.181:0/3188260500 shutdown_connections 2023-10-16T09:57:11.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.528+0000 7f2f2d4df700 1 -- 172.21.15.181:0/3188260500 wait complete. 2023-10-16T09:57:11.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.530+0000 7f2f2d4df700 1 Processor -- start 2023-10-16T09:57:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.530+0000 7f2f2d4df700 1 -- start start 2023-10-16T09:57:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f2d4df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f28072760 0x7f2f280b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f2d4df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f281f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f2d4df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f2815da70 0x7f2f281f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f2d4df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280b6280 con 0x7f2f28072760 2023-10-16T09:57:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f2d4df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280ad670 con 0x7f2f2815da70 2023-10-16T09:57:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f2d4df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280ad950 con 0x7f2f280ce6a0 2023-10-16T09:57:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f28072760 0x7f2f280b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f28072760 0x7f2f280b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40568/0 (socket says 172.21.15.181:40568) 2023-10-16T09:57:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 -- 172.21.15.181:0/166176083 learned_addr learned my addr 172.21.15.181:0/166176083 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f281f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f277fe700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f2815da70 0x7f2f281f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 msgr2=0x7f2f281f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f280ce6a0 0x7f2f281f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f2815da70 msgr2=0x7f2f281f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:11.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f2815da70 0x7f2f281f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:11.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.531+0000 7f2f26ffd700 1 -- 172.21.15.181:0/166176083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180094a0 con 0x7f2f28072760 2023-10-16T09:57:11.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.532+0000 7f2f26ffd700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f28072760 0x7f2f280b5c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800ebe0 tx=0x7f2f1800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:11.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.532+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18009760 con 0x7f2f28072760 2023-10-16T09:57:11.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.532+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1800c850 con 0x7f2f28072760 2023-10-16T09:57:11.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.532+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c0094a0 con 0x7f2f28072760 2023-10-16T09:57:11.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.532+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800cbe0 con 0x7f2f28072760 2023-10-16T09:57:11.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.532+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f280adf10 con 0x7f2f28072760 2023-10-16T09:57:11.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.533+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f28161000 con 0x7f2f28072760 2023-10-16T09:57:11.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.534+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 20) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2f1800cd40 con 0x7f2f28072760 2023-10-16T09:57:11.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.534+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 0x7f2f1406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:11.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.534+0000 7f2f267fc700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 msgr2=0x7f2f1406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:11.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.534+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f2f18089980 con 0x7f2f28072760 2023-10-16T09:57:11.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.534+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 0x7f2f1406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:57:11.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.539+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2f18054780 con 0x7f2f28072760 2023-10-16T09:57:11.554 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:11.553+0000 7f9a64597200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:57:11.603 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:11.602+0000 7f10dbf0c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:57:11.634 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:11.632+0000 7f9a64597200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:57:11.685 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:11.684+0000 7f10dbf0c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:57:11.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.735+0000 7f2f267fc700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 msgr2=0x7f2f1406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:11.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.735+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 0x7f2f1406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:57:11.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:11.767+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 --> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f2815de50 con 0x7f2f1406b070 2023-10-16T09:57:11.999 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:11.998+0000 7f9a64597200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:57:12.057 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:12.055+0000 7f10dbf0c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:57:12.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:12.136+0000 7f2f267fc700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 msgr2=0x7f2f1406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:12.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:12.136+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 0x7f2f1406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T09:57:12.566 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:12.565+0000 7f9a64597200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:57:12.626 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:12.625+0000 7f10dbf0c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:57:12.634 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:12.634+0000 7f9a64597200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:57:12.696 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:12.694+0000 7f10dbf0c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:57:12.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:12.937+0000 7f2f267fc700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 msgr2=0x7f2f1406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:12.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:12.937+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 0x7f2f1406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T09:57:13.256 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:13.254+0000 7f9a64597200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:57:13.324 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:13.323+0000 7f10dbf0c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:57:13.332 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:13.331+0000 7f9a64597200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:57:13.401 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:13.400+0000 7f10dbf0c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:57:13.472 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:13.471+0000 7f9a64597200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:57:13.545 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:13.543+0000 7f10dbf0c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:57:13.695 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:13.694+0000 7f9a64597200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:57:13.767 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:13.766+0000 7f9a64597200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:57:13.773 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:13.772+0000 7f10dbf0c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:57:13.847 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:13.845+0000 7f10dbf0c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:57:14.032 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:14.030+0000 7f9a64597200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:57:14.111 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:14.110+0000 7f10dbf0c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:57:14.247 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:14.246+0000 7f9a64597200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:57:14.334 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:14.333+0000 7f10dbf0c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:57:14.392 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:14.391+0000 7f9a64597200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:57:14.484 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:14.483+0000 7f10dbf0c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:57:14.504 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:14.503+0000 7f9a64597200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:14.538+0000 7f2f267fc700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 msgr2=0x7f2f1406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3219577148 2023-10-16T09:57:14.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:14.538+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 0x7f2f1406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T09:57:14.598 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:14.596+0000 7f10dbf0c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:57:14.656 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:14.655+0000 7f9a64597200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:57:14.753 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:14.752+0000 7f10dbf0c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:57:14.919 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:14.918+0000 7f9a64597200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:57:15.015 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.014+0000 7f10dbf0c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:57:15.075 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:15.074+0000 7f9a64597200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:57:15.177 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.175+0000 7f10dbf0c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:57:15.592 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:57:15.590+0000 7f9a64597200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:57:15.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:15.661+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 21) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2f18054b50 con 0x7f2f28072760 2023-10-16T09:57:15.707 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.705+0000 7f10dbf0c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:57:15.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:15.729+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 22) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f2f18087910 con 0x7f2f28072760 2023-10-16T09:57:15.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:15.730+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 msgr2=0x7f2f1406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:57:15.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:15.730+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3219577148,v1:172.21.15.181:6833/3219577148] conn(0x7f2f1406b070 0x7f2f1406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:15.887 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f104eee7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.887 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f104eee7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.887 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f104eee7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.888 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f104eee7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.888 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f104eee7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.888 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f1051eed700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.888 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f1051eed700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.888 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f1051eed700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.889 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f1051eed700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:15.889 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:57:15.885+0000 7f1051eed700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:57:16.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.734+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 23) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2f18087bc0 con 0x7f2f28072760 2023-10-16T09:57:16.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.734+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2f1401d250 0x7f2f140754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:16.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.734+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 --> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f14071070 con 0x7f2f1401d250 2023-10-16T09:57:16.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.734+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2f1401d250 0x7f2f140754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:16.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.735+0000 7f2f267fc700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2f1401d250 0x7f2f140754d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0090b0 tx=0x7f2f1c01c040 comp rx=0 tx=0).ready entity=mgr.6120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:16.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.801+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/166176083 <== mgr.6120 v2:172.21.15.181:6832/3708044156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2f14071070 con 0x7f2f1401d250 2023-10-16T09:57:16.803 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1 2023-10-16T09:57:16.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2f1401d250 msgr2=0x7f2f140754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:16.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2f1401d250 0x7f2f140754d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0090b0 tx=0x7f2f1c01c040 comp rx=0 tx=0).stop 2023-10-16T09:57:16.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f28072760 msgr2=0x7f2f280b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:16.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 --2- 172.21.15.181:0/166176083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f28072760 0x7f2f280b5c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800ebe0 tx=0x7f2f1800eef0 comp rx=0 tx=0).stop 2023-10-16T09:57:16.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f277fe700 1 -- 172.21.15.181:0/166176083 reap_dead start 2023-10-16T09:57:16.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 shutdown_connections 2023-10-16T09:57:16.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 >> 172.21.15.181:0/166176083 conn(0x7f2f280cd010 msgr2=0x7f2f2806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 shutdown_connections 2023-10-16T09:57:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.803+0000 7f2f2d4df700 1 -- 172.21.15.181:0/166176083 wait complete. 2023-10-16T09:57:16.817 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T09:57:16.818 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2m 2023-10-16T09:57:16.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.940+0000 7fb8c2ffc700 1 Processor -- start 2023-10-16T09:57:16.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.940+0000 7fb8c2ffc700 1 -- start start 2023-10-16T09:57:16.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.940+0000 7fb8c2ffc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc15dbd0 0x7fb8bc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:16.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.940+0000 7fb8c2ffc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8bc0ce490 0x7fb8bc06b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:16.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.940+0000 7fb8c2ffc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8bc15dfb0 con 0x7fb8bc0d0610 2023-10-16T09:57:16.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.940+0000 7fb8c2ffc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8bc0cd3a0 con 0x7fb8bc0ce490 2023-10-16T09:57:16.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.940+0000 7fb8c2ffc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0cd830 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c1599700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8bc0ce490 0x7fb8bc06b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:16.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8bbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb8bc0d0610 0x7fb8bc0cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51822/0 (socket says 172.21.15.181:51822) 2023-10-16T09:57:16.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8bbfff700 1 -- 172.21.15.181:0/3066904966 learned_addr learned my addr 172.21.15.181:0/3066904966 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c0d98700 1 --2- 172.21.15.181:0/3066904966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc15dbd0 0x7fb8bc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c1599700 1 -- 172.21.15.181:0/3066904966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc15dbd0 msgr2=0x7fb8bc0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c1599700 1 --2- 172.21.15.181:0/3066904966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc15dbd0 0x7fb8bc0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c1599700 1 -- 172.21.15.181:0/3066904966 >> v1:172.21.15.181:6789/0 conn(0x7fb8bc0d0610 legacy=0x7fb8bc0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8bb7fe700 1 -- 172.21.15.181:0/3066904966 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249430518 0 0) 0x7fb8a8001110 con 0x7fb8bc0d0610 2023-10-16T09:57:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c1599700 1 -- 172.21.15.181:0/3066904966 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc0d1fc0 con 0x7fb8bc0ce490 2023-10-16T09:57:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c0d98700 1 --2- 172.21.15.181:0/3066904966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc15dbd0 0x7fb8bc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:57:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8c1599700 1 --2- 172.21.15.181:0/3066904966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc0ce490 0x7fb8bc06b6f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009d90 tx=0x7fb8b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae8441c62000c0e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8bb7fe700 1 -- 172.21.15.181:0/3066904966 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400e070 con 0x7fb8bc0ce490 2023-10-16T09:57:16.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8bb7fe700 1 -- 172.21.15.181:0/3066904966 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b400bbd0 con 0x7fb8bc0ce490 2023-10-16T09:57:16.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.941+0000 7fb8bb7fe700 1 -- 172.21.15.181:0/3066904966 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4012430 con 0x7fb8bc0ce490 2023-10-16T09:57:16.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/3066904966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc0ce490 msgr2=0x7fb8bc06b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:16.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/3066904966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc0ce490 0x7fb8bc06b6f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009d90 tx=0x7fb8b400b330 comp rx=0 tx=0).stop 2023-10-16T09:57:16.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/3066904966 shutdown_connections 2023-10-16T09:57:16.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/3066904966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc0ce490 0x7fb8bc06b6f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:16.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/3066904966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc15dbd0 0x7fb8bc0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:16.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/3066904966 >> 172.21.15.181:0/3066904966 conn(0x7fb8bc09a930 msgr2=0x7fb8bc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:16.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/3066904966 shutdown_connections 2023-10-16T09:57:16.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.942+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/3066904966 wait complete. 2023-10-16T09:57:16.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.943+0000 7fb8c2ffc700 1 Processor -- start 2023-10-16T09:57:16.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.943+0000 7fb8c2ffc700 1 -- start start 2023-10-16T09:57:16.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c2ffc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc0ce490 0x7fb8bc1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:16.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c2ffc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8bc0d0610 0x7fb8bc15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:16.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c2ffc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc15dbd0 0x7fb8bc15e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:16.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c2ffc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc158f20 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c2ffc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0b61c0 con 0x7fb8bc0d0610 2023-10-16T09:57:16.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c2ffc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0b6460 con 0x7fb8bc0ce490 2023-10-16T09:57:16.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc15dbd0 0x7fb8bc15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:16.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc15dbd0 0x7fb8bc15e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40754/0 (socket says 172.21.15.181:40754) 2023-10-16T09:57:16.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 -- 172.21.15.181:0/4098931166 learned_addr learned my addr 172.21.15.181:0/4098931166 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:16.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c0d98700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc0ce490 0x7fb8bc1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:16.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8bbfff700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8bc0d0610 0x7fb8bc15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:16.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 -- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc0ce490 msgr2=0x7fb8bc1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:16.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc0ce490 0x7fb8bc1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:16.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 -- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8bc0d0610 msgr2=0x7fb8bc15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:16.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8bc0d0610 0x7fb8bc15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:16.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c1599700 1 -- 172.21.15.181:0/4098931166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4009a70 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8c0d98700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc0ce490 0x7fb8bc1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:16.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.944+0000 7fb8bbfff700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8bc0d0610 0x7fb8bc15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:57:16.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.945+0000 7fb8c1599700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc15dbd0 0x7fb8bc15e690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4014040 tx=0x7fb8b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:16.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.945+0000 7fb8b97fa700 1 -- 172.21.15.181:0/4098931166 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4012430 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.945+0000 7fb8b97fa700 1 -- 172.21.15.181:0/4098931166 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b4020070 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.945+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc0b6740 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.945+0000 7fb8b97fa700 1 -- 172.21.15.181:0/4098931166 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4004280 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.945+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc0b6d00 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.946+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc067d20 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.946+0000 7fb8b97fa700 1 -- 172.21.15.181:0/4098931166 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 23) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fb8b40043e0 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.946+0000 7fb8b97fa700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fb8a006ada0 0x7fb8a006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:16.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.946+0000 7fb8b97fa700 1 -- 172.21.15.181:0/4098931166 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fb8b4022070 con 0x7fb8bc15dbd0 2023-10-16T09:57:16.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.946+0000 7fb8c0d98700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fb8a006ada0 0x7fb8a006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:16.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.947+0000 7fb8c0d98700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fb8a006ada0 0x7fb8a006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009a00 tx=0x7fb8b0008040 comp rx=0 tx=0).ready entity=mgr.6120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:16.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:16.951+0000 7fb8b97fa700 1 -- 172.21.15.181:0/4098931166 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8b40564a0 con 0x7fb8bc15dbd0 2023-10-16T09:57:17.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.168+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 --> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8bc0cca70 con 0x7fb8a006ada0 2023-10-16T09:57:17.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.221+0000 7fb8b97fa700 1 -- 172.21.15.181:0/4098931166 <== mgr.6120 v2:172.21.15.181:6832/3708044156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb8b0009610 con 0x7fb8a006ada0 2023-10-16T09:57:17.223 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1 2023-10-16T09:57:17.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fb8a006ada0 msgr2=0x7fb8a006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fb8a006ada0 0x7fb8a006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009a00 tx=0x7fb8b0008040 comp rx=0 tx=0).stop 2023-10-16T09:57:17.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc15dbd0 msgr2=0x7fb8bc15e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc15dbd0 0x7fb8bc15e690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4014040 tx=0x7fb8b4004ad0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 shutdown_connections 2023-10-16T09:57:17.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fb8a006ada0 0x7fb8a006d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8bc0ce490 0x7fb8bc1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8bc15dbd0 0x7fb8bc15e690 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 --2- 172.21.15.181:0/4098931166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8bc0d0610 0x7fb8bc15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 >> 172.21.15.181:0/4098931166 conn(0x7fb8bc09a930 msgr2=0x7fb8bc15cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:17.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 shutdown_connections 2023-10-16T09:57:17.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.224+0000 7fb8c2ffc700 1 -- 172.21.15.181:0/4098931166 wait complete. 2023-10-16T09:57:17.238 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T09:57:17.238 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-09_58_00 in ~43s... 2023-10-16T09:57:17.238 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-09_59_00 in ~103s... 2023-10-16T09:57:17.239 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m', '2m'], expected_retention: [] 2023-10-16T09:57:17.239 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T09:57:17.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.360+0000 7f95d0483700 1 Processor -- start 2023-10-16T09:57:17.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.360+0000 7f95d0483700 1 -- start start 2023-10-16T09:57:17.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.361+0000 7f95d0483700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95c815dbd0 0x7f95c80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:17.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.361+0000 7f95d0483700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c80d0610 0x7f95c80cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:17.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.361+0000 7f95d0483700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c815dfb0 con 0x7f95c80ce2c0 2023-10-16T09:57:17.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.361+0000 7f95d0483700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c80ce740 con 0x7f95c815dbd0 2023-10-16T09:57:17.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.361+0000 7f95d0483700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80cd3a0 con 0x7f95c80d0610 2023-10-16T09:57:17.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95ce21f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95c815dbd0 0x7f95c80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:17.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95cda1e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c80d0610 0x7f95c80cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:17.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95cea20700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f95c80ce2c0 0x7f95c8161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:35778/0 (socket says 172.21.15.181:35778) 2023-10-16T09:57:17.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95cea20700 1 -- 172.21.15.181:0/3814099750 learned_addr learned my addr 172.21.15.181:0/3814099750 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:57:17.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95ce21f700 1 -- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c80d0610 msgr2=0x7f95c80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:57:17.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95ce21f700 1 --2- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c80d0610 0x7f95c80cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95ce21f700 1 -- 172.21.15.181:0/3814099750 >> v1:172.21.15.190:6789/0 conn(0x7f95c80ce2c0 legacy=0x7f95c8161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95ce21f700 1 -- 172.21.15.181:0/3814099750 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80cd830 con 0x7f95c815dbd0 2023-10-16T09:57:17.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95cd21d700 1 -- 172.21.15.181:0/3814099750 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280176025 0 0) 0x7f95c0001490 con 0x7f95c80ce2c0 2023-10-16T09:57:17.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95cda1e700 1 --2- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c80d0610 0x7f95c80cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:57:17.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.362+0000 7f95ce21f700 1 --2- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c815dbd0 0x7f95c80d00d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009840 tx=0x7f95b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc762ac6f211919 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:17.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95cd21d700 1 -- 172.21.15.181:0/3814099750 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8011830 con 0x7f95c815dbd0 2023-10-16T09:57:17.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95cd21d700 1 -- 172.21.15.181:0/3814099750 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b8011990 con 0x7f95c815dbd0 2023-10-16T09:57:17.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95cd21d700 1 -- 172.21.15.181:0/3814099750 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8011d20 con 0x7f95c815dbd0 2023-10-16T09:57:17.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 -- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c815dbd0 msgr2=0x7f95c80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 --2- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c815dbd0 0x7f95c80d00d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009840 tx=0x7f95b8009b50 comp rx=0 tx=0).stop 2023-10-16T09:57:17.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 -- 172.21.15.181:0/3814099750 shutdown_connections 2023-10-16T09:57:17.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 --2- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c815dbd0 0x7f95c80d00d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 --2- 172.21.15.181:0/3814099750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c80d0610 0x7f95c80cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 -- 172.21.15.181:0/3814099750 >> 172.21.15.181:0/3814099750 conn(0x7f95c809a930 msgr2=0x7f95c80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:17.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 -- 172.21.15.181:0/3814099750 shutdown_connections 2023-10-16T09:57:17.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.363+0000 7f95d0483700 1 -- 172.21.15.181:0/3814099750 wait complete. 2023-10-16T09:57:17.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.364+0000 7f95d0483700 1 Processor -- start 2023-10-16T09:57:17.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.364+0000 7f95d0483700 1 -- start start 2023-10-16T09:57:17.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95d0483700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80ce2c0 0x7f95c81f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:17.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95d0483700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c80d0610 0x7f95c8160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:17.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95d0483700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c815dbd0 0x7f95c8151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:17.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95d0483700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81613c0 con 0x7f95c80ce2c0 2023-10-16T09:57:17.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95d0483700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c815fde0 con 0x7f95c80d0610 2023-10-16T09:57:17.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95d0483700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81600c0 con 0x7f95c815dbd0 2023-10-16T09:57:17.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95ce21f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80ce2c0 0x7f95c81f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:17.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95ce21f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80ce2c0 0x7f95c81f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40764/0 (socket says 172.21.15.181:40764) 2023-10-16T09:57:17.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95ce21f700 1 -- 172.21.15.181:0/892240421 learned_addr learned my addr 172.21.15.181:0/892240421 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:57:17.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95cda1e700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c80d0610 0x7f95c8160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:17.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.365+0000 7f95cea20700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c815dbd0 0x7f95c8151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:17.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95cda1e700 1 -- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c815dbd0 msgr2=0x7f95c8151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95cda1e700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c815dbd0 0x7f95c8151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95cda1e700 1 -- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80ce2c0 msgr2=0x7f95c81f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95cda1e700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80ce2c0 0x7f95c81f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95cda1e700 1 -- 172.21.15.181:0/892240421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c40094a0 con 0x7f95c80d0610 2023-10-16T09:57:17.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95ce21f700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80ce2c0 0x7f95c81f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:57:17.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95cda1e700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c80d0610 0x7f95c8160e80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95c400e980 tx=0x7f95c400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:17.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95beffd700 1 -- 172.21.15.181:0/892240421 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c400c690 con 0x7f95c80d0610 2023-10-16T09:57:17.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95beffd700 1 -- 172.21.15.181:0/892240421 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c400c7f0 con 0x7f95c80d0610 2023-10-16T09:57:17.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b80094a0 con 0x7f95c80d0610 2023-10-16T09:57:17.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95beffd700 1 -- 172.21.15.181:0/892240421 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c400cb80 con 0x7f95c80d0610 2023-10-16T09:57:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.366+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c8160680 con 0x7f95c80d0610 2023-10-16T09:57:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.367+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c8067d20 con 0x7f95c80d0610 2023-10-16T09:57:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.367+0000 7f95beffd700 1 -- 172.21.15.181:0/892240421 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 23) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f95c400cce0 con 0x7f95c80d0610 2023-10-16T09:57:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.368+0000 7f95beffd700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f95ac06ada0 0x7f95ac06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:57:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.368+0000 7f95beffd700 1 -- 172.21.15.181:0/892240421 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f95c4089520 con 0x7f95c80d0610 2023-10-16T09:57:17.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.368+0000 7f95ce21f700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f95ac06ada0 0x7f95ac06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:57:17.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.368+0000 7f95ce21f700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f95ac06ada0 0x7f95ac06d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b8008030 comp rx=0 tx=0).ready entity=mgr.6120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:57:17.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.372+0000 7f95beffd700 1 -- 172.21.15.181:0/892240421 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f95c4054620 con 0x7f95c80d0610 2023-10-16T09:57:17.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.592+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 --> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95c80cc790 con 0x7f95ac06ada0 2023-10-16T09:57:17.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.602+0000 7f95beffd700 1 -- 172.21.15.181:0/892240421 <== mgr.6120 v2:172.21.15.181:6832/3708044156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f95b8011c30 con 0x7f95ac06ada0 2023-10-16T09:57:17.604 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:57:17.604 INFO:teuthology.orchestra.run.smithi181.stdout:{"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-16T09:57:17.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f95ac06ada0 msgr2=0x7f95ac06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f95ac06ada0 0x7f95ac06d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b8008030 comp rx=0 tx=0).stop 2023-10-16T09:57:17.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c80d0610 msgr2=0x7f95c8160e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:57:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c80d0610 0x7f95c8160e80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95c400e980 tx=0x7f95c400ec90 comp rx=0 tx=0).stop 2023-10-16T09:57:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 shutdown_connections 2023-10-16T09:57:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f95ac06ada0 0x7f95ac06d260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95c815dbd0 0x7f95c8151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80ce2c0 0x7f95c81f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 --2- 172.21.15.181:0/892240421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95c80d0610 0x7f95c8160e80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:57:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 >> 172.21.15.181:0/892240421 conn(0x7f95c809a930 msgr2=0x7f95c815b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:57:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 shutdown_connections 2023-10-16T09:57:17.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:57:17.604+0000 7f95d0483700 1 -- 172.21.15.181:0/892240421 wait complete. 2023-10-16T09:57:17.618 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-16T09:57:17.619 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m', '2m'], 'retention': [{}, {}]} 2023-10-16T09:57:17.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:17.620 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:17.638 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:17.639 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:17.639 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:18.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:18.641 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:18.663 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:18.663 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:18.663 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:19.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:19.665 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:19.687 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:19.687 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:19.687 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:20.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:20.689 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:20.711 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:20.711 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:20.712 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:21.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:21.713 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:21.735 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:21.735 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:21.736 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:22.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:22.737 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:22.759 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:22.760 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:22.760 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:23.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:23.762 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:23.784 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:23.785 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:23.785 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:24.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:24.786 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:24.808 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:24.808 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:24.809 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:25.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:25.810 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:25.832 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:25.833 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:25.833 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:26.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:26.834 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:26.857 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:26.857 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:26.857 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:27.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:27.859 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:27.881 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:27.881 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:27.882 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:28.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:28.884 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:28.906 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:28.907 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:28.907 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:29.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:29.909 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:29.931 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:29.931 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:29.932 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:30.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:30.933 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:30.955 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:30.955 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:30.956 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:31.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:31.958 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:31.980 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:31.980 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:31.981 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:32.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:32.983 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:33.004 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:33.005 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:33.005 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:34.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:34.007 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:34.028 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:34.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:34.029 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:35.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:35.031 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:35.053 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:35.054 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:35.054 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:36.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:36.056 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:36.078 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:36.078 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:36.079 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:37.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:37.081 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:37.103 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:37.103 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:37.104 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:38.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:38.105 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:38.127 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:38.128 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:38.128 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:39.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:39.130 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:39.152 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:39.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:39.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:39.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:39.699 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:57:40.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:40.155 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:40.178 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:40.179 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:40.179 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:41.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:41.181 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:41.203 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:41.203 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:41.203 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:42.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:42.205 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:42.228 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:42.228 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:42.229 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:43.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:43.231 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:43.253 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:43.253 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:43.254 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:44.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:44.255 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:44.278 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:44.278 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:44.278 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:45.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:45.280 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:45.302 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:45.303 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:45.303 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:46.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:46.305 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:46.328 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:46.328 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:46.328 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:47.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:47.330 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:47.352 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:47.352 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:47.353 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:48.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:48.354 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:48.377 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:48.377 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:48.377 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:49.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:49.379 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:49.401 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:49.401 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:49.402 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:50.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:50.403 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:50.425 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:50.425 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:50.425 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:51.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:51.427 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:51.449 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:51.449 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:51.450 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:52.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:52.451 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:52.473 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:52.473 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:52.474 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:53.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:53.475 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:53.498 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:53.498 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:53.498 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:54.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:54.501 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:54.523 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:54.523 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:54.523 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:55.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:55.525 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:55.550 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:55.550 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:55.551 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:56.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:56.553 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:56.575 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:56.575 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:56.576 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:57.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:57.577 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:57.600 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:57.601 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:57.601 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:58.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:58.602 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:58.624 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:58.625 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:58.625 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:57:59.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:57:59.627 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:57:59.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T09:57:59.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:57:59.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:00.234 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:58:00.232+0000 7f1040ecb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:58:00.234 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:58:00.232+0000 7f1040ecb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:58:00.234 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:58:00.232+0000 7f1040ecb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:58:00.235 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:58:00.232+0000 7f1040ecb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:58:00.235 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:58:00.232+0000 7f1040ecb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:58:00.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:00.651 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:00.673 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:00.673 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:00.673 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:00.674 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:00.674 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-09_58_00_UTC'] 2023-10-16T09:58:00.674 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697450237.2384417, now = 1697450280.6749084, timo = 43 2023-10-16T09:58:01.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:01.676 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:01.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:01.699 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:01.699 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:01.699 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:02.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:02.701 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:02.722 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:02.723 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:02.724 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:02.724 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:03.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:03.726 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:03.747 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:03.748 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:03.748 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:03.749 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:04.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:04.751 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:04.773 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:04.773 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:04.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:04.774 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:05.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:05.776 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:05.799 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:05.799 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:05.800 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:06.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:06.801 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:06.822 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:06.823 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:06.824 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:06.824 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:07.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:07.826 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:07.847 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:07.848 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:07.848 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:07.849 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:08.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:08.850 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:08.871 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:08.874 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:08.874 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:08.874 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:09.780 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:09.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:09.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:09.876 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:09.894 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:09.895 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:09.896 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:09.896 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:10.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:10.898 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:10.920 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:10.920 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:10.920 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:10.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:11.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:11.923 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:11.945 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:11.945 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:11.946 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:11.946 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:12.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:12.948 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:12.970 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:12.971 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:12.971 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:12.972 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:13.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:13.974 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:13.998 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:13.998 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:13.999 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:13.999 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:15.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:15.001 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:15.022 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:15.023 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:15.023 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:15.024 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:16.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:16.025 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:16.049 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:16.050 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:16.050 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:16.051 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:17.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:17.053 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:17.074 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:17.074 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:17.075 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:17.075 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:18.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:18.077 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:18.098 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:18.098 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:18.099 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:18.099 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:19.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:19.101 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:19.123 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:19.123 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:19.124 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:19.124 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:20.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:20.126 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:20.148 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:20.149 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:20.149 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:20.149 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:21.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:21.151 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:21.172 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:21.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:21.174 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:21.174 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:22.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:22.176 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:22.198 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:22.199 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:22.199 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:22.199 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:23.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:23.201 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:23.223 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:23.224 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:23.224 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:23.225 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:24.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:24.227 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:24.248 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:24.250 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:24.250 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:24.251 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:25.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:25.252 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:25.275 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:25.275 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:25.275 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:25.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:26.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:26.278 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:26.299 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:26.300 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:26.301 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:26.301 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:27.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:27.305 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:27.326 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:27.328 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:27.328 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:27.328 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:28.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:28.330 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:28.352 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:28.353 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:28.353 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:28.353 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:29.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:29.358 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:29.379 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:29.380 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:29.380 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:29.381 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:30.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:30.382 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:30.404 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:30.405 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:30.405 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:30.406 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:31.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:31.407 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:31.428 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:31.430 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:31.430 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:31.430 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:32.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:32.432 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:32.453 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:32.455 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:32.455 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:32.455 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:33.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:33.457 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:33.478 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:33.480 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:33.480 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:33.480 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:34.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:34.482 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:34.504 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:34.505 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:34.505 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:34.505 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:35.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:35.507 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:35.528 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:35.529 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:35.530 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:35.530 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:36.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:36.532 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:36.553 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:36.554 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:36.555 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:36.555 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:37.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:37.557 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:37.578 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:37.579 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:37.580 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:37.580 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:38.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:38.582 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:38.603 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:38.604 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:38.605 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:38.605 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:39.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:39.606 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:39.628 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:39.628 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:39.629 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:39.629 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:39.865 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:39.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:58:40.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:40.631 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:40.653 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:40.654 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:40.654 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:40.654 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:41.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:41.656 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:41.677 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:41.679 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:41.679 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:41.679 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:42.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:42.681 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:42.703 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:42.704 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:42.704 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:42.704 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:43.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:43.706 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:43.728 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:43.729 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:43.729 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:43.729 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:44.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:44.731 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:44.753 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:44.754 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:44.755 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:44.755 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:45.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:45.757 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:45.779 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:45.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:45.780 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:45.781 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:46.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:46.782 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:46.803 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:46.804 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:46.805 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:46.805 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:47.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:47.807 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:47.829 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:47.830 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:47.830 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:47.831 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:48.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:48.833 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:48.854 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:48.855 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:48.856 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:48.856 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:49.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:49.858 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:49.879 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:49.880 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:49.881 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:49.881 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:50.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:50.883 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:50.904 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:50.905 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:50.906 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:50.906 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:51.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:51.908 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:51.929 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:51.930 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:51.930 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:51.931 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:52.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:52.932 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:52.953 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:52.954 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:52.954 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:52.955 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:53.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:53.956 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:53.977 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:53.978 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:53.978 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:53.978 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:54.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:54.980 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:55.001 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:55.002 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:55.003 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:55.003 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:56.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:56.005 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:56.026 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:56.027 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:56.027 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:56.028 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:57.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:57.030 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:57.051 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:57.052 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:57.053 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:57.053 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:58.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:58.055 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:58.076 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:58.077 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:58.078 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:58.078 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:58:59.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:58:59.080 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:58:59.101 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:58:59.102 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:58:59.103 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:58:59.103 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:59:00.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:00.104 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:59:00.125 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:59:00.126 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:59:00.126 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T09:59:00.127 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:59:01.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:01.128 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:59:01.149 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:59:01.150 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T09:59:01.151 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-09_59_00_UTC', 'scheduled-2023-10-16-09_58_00_UTC'} 2023-10-16T09:59:01.151 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-09_59_00_UTC'} 2023-10-16T09:59:01.151 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T09:59:01.152 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-09_59_00_UTC'] 2023-10-16T09:59:01.152 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697450237.2384417, now = 1697450341.1525016, timo = 103 2023-10-16T09:59:02.154 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T09:59:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.288+0000 7f89617f9700 1 Processor -- start 2023-10-16T09:59:02.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.288+0000 7f89617f9700 1 -- start start 2023-10-16T09:59:02.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f89617f9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f895c15dce0 0x7f895c0c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f89617f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0aec90 0x7f895c0c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f89617f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c0cd9a0 con 0x7f895c0af090 2023-10-16T09:59:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f89617f9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c0ce540 con 0x7f895c15dce0 2023-10-16T09:59:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f89617f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c06d9e0 con 0x7f895c0aec90 2023-10-16T09:59:02.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f895c15dce0 0x7f895c0c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f895c15dce0 0x7f895c0c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40804/0 (socket says 172.21.15.181:40804) 2023-10-16T09:59:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895affd700 1 -- 172.21.15.181:0/3935070756 learned_addr learned my addr 172.21.15.181:0/3935070756 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895a7fc700 1 --2- 172.21.15.181:0/3935070756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0aec90 0x7f895c0c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895affd700 1 -- 172.21.15.181:0/3935070756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0aec90 msgr2=0x7f895c0c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:02.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895affd700 1 --2- 172.21.15.181:0/3935070756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0aec90 0x7f895c0c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895affd700 1 -- 172.21.15.181:0/3935070756 >> v1:172.21.15.190:6789/0 conn(0x7f895c0af090 legacy=0x7f895c161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.289+0000 7f895affd700 1 -- 172.21.15.181:0/3935070756 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0ce790 con 0x7f895c15dce0 2023-10-16T09:59:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f895affd700 1 --2- 172.21.15.181:0/3935070756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c15dce0 0x7f895c0c4b40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f89500098f0 tx=0x7f8950009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa189d9343c28128 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f8959ffb700 1 -- 172.21.15.181:0/3935070756 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000b8a0 con 0x7f895c15dce0 2023-10-16T09:59:02.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f8959ffb700 1 -- 172.21.15.181:0/3935070756 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895000ba00 con 0x7f895c15dce0 2023-10-16T09:59:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f8959ffb700 1 -- 172.21.15.181:0/3935070756 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000bd90 con 0x7f895c15dce0 2023-10-16T09:59:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 -- 172.21.15.181:0/3935070756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c15dce0 msgr2=0x7f895c0c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 --2- 172.21.15.181:0/3935070756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c15dce0 0x7f895c0c4b40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f89500098f0 tx=0x7f8950009c00 comp rx=0 tx=0).stop 2023-10-16T09:59:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 -- 172.21.15.181:0/3935070756 shutdown_connections 2023-10-16T09:59:02.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 --2- 172.21.15.181:0/3935070756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c15dce0 0x7f895c0c4b40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 --2- 172.21.15.181:0/3935070756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0aec90 0x7f895c0c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 -- 172.21.15.181:0/3935070756 >> 172.21.15.181:0/3935070756 conn(0x7f895c0cc740 msgr2=0x7f895c0ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 -- 172.21.15.181:0/3935070756 shutdown_connections 2023-10-16T09:59:02.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.290+0000 7f89617f9700 1 -- 172.21.15.181:0/3935070756 wait complete. 2023-10-16T09:59:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 Processor -- start 2023-10-16T09:59:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 -- start start 2023-10-16T09:59:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c0aec90 0x7f895c1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0af090 0x7f895c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c15dce0 0x7f895c15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0accf0 con 0x7f895c0aec90 2023-10-16T09:59:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c15f020 con 0x7f895c15dce0 2023-10-16T09:59:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f89617f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c1f3cc0 con 0x7f895c0af090 2023-10-16T09:59:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f895affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c0aec90 0x7f895c1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.292+0000 7f895affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c0aec90 0x7f895c1f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40810/0 (socket says 172.21.15.181:40810) 2023-10-16T09:59:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895affd700 1 -- 172.21.15.181:0/2535785243 learned_addr learned my addr 172.21.15.181:0/2535785243 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895a7fc700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0af090 0x7f895c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895b7fe700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c15dce0 0x7f895c15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895affd700 1 -- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0af090 msgr2=0x7f895c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895affd700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0af090 0x7f895c15e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895affd700 1 -- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c15dce0 msgr2=0x7f895c15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895affd700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c15dce0 0x7f895c15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895affd700 1 -- 172.21.15.181:0/2535785243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009550 con 0x7f895c0aec90 2023-10-16T09:59:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895a7fc700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0af090 0x7f895c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895b7fe700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c15dce0 0x7f895c15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f895affd700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c0aec90 0x7f895c1f33c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8950009c40 tx=0x7f8950004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f89437fe700 1 -- 172.21.15.181:0/2535785243 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89500036a0 con 0x7f895c0aec90 2023-10-16T09:59:02.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f89437fe700 1 -- 172.21.15.181:0/2535785243 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895001e070 con 0x7f895c0aec90 2023-10-16T09:59:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c1f3fa0 con 0x7f895c0aec90 2023-10-16T09:59:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f89437fe700 1 -- 172.21.15.181:0/2535785243 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950003a40 con 0x7f895c0aec90 2023-10-16T09:59:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.293+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0ac910 con 0x7f895c0aec90 2023-10-16T09:59:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.294+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c0acaa0 con 0x7f895c0aec90 2023-10-16T09:59:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.295+0000 7f89437fe700 1 -- 172.21.15.181:0/2535785243 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f8950003ba0 con 0x7f895c0aec90 2023-10-16T09:59:02.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.295+0000 7f89437fe700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f894806b070 0x7f894806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.295+0000 7f89437fe700 1 -- 172.21.15.181:0/2535785243 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f8950020070 con 0x7f895c0aec90 2023-10-16T09:59:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.295+0000 7f895a7fc700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f894806b070 0x7f894806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.295+0000 7f895a7fc700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f894806b070 0x7f894806d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f895c0683e0 tx=0x7f894c008040 comp rx=0 tx=0).ready entity=mgr.6120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:02.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.300+0000 7f89437fe700 1 -- 172.21.15.181:0/2535785243 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f895c0acaa0 con 0x7f895c0aec90 2023-10-16T09:59:02.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.521+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 --> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f895c0ced50 con 0x7f894806b070 2023-10-16T09:59:02.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.587+0000 7f89437fe700 1 -- 172.21.15.181:0/2535785243 <== mgr.6120 v2:172.21.15.181:6832/3708044156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f894c009680 con 0x7f894806b070 2023-10-16T09:59:02.589 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path snap_test_dir1 2023-10-16T09:59:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f894806b070 msgr2=0x7f894806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f894806b070 0x7f894806d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f895c0683e0 tx=0x7f894c008040 comp rx=0 tx=0).stop 2023-10-16T09:59:02.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c0aec90 msgr2=0x7f895c1f33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c0aec90 0x7f895c1f33c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8950009c40 tx=0x7f8950004750 comp rx=0 tx=0).stop 2023-10-16T09:59:02.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 shutdown_connections 2023-10-16T09:59:02.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f894806b070 0x7f894806d530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c15dce0 0x7f895c15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f895c0aec90 0x7f895c1f33c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 --2- 172.21.15.181:0/2535785243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c0af090 0x7f895c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 >> 172.21.15.181:0/2535785243 conn(0x7f895c0cc740 msgr2=0x7f895c06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:02.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 shutdown_connections 2023-10-16T09:59:02.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.589+0000 7f89617f9700 1 -- 172.21.15.181:0/2535785243 wait complete. 2023-10-16T09:59:02.602 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T09:59:02.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:02.603 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T09:59:02.621 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:59:02.622 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-09_59_00_UTC 2023-10-16T09:59:02.622 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-09_58_00_UTC 2023-10-16T09:59:02.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:02.623 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-09_58_00_UTC) 2023-10-16T09:59:02.683 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-09_59_00_UTC 2023-10-16T09:59:02.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:02.684 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-09_59_00_UTC) 2023-10-16T09:59:02.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:02.746 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T09:59:02.805 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T09:59:02.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.965+0000 7f39cf253700 1 Processor -- start 2023-10-16T09:59:02.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.966+0000 7f39cf253700 1 -- start start 2023-10-16T09:59:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.966+0000 7f39cf253700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39c815dbd0 0x7f39c80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.966+0000 7f39cf253700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c815dfb0 con 0x7f39c80ce490 2023-10-16T09:59:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.966+0000 7f39cf253700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c80cd3a0 con 0x7f39c80d0610 2023-10-16T09:59:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.966+0000 7f39cf253700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c80cd830 con 0x7f39c815dbd0 2023-10-16T09:59:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39cd7f0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f39c80ce490 0x7f39c806b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51898/0 (socket says 172.21.15.181:51898) 2023-10-16T09:59:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39cd7f0700 1 -- 172.21.15.181:0/4096256734 learned_addr learned my addr 172.21.15.181:0/4096256734 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39ccfef700 1 --2- 172.21.15.181:0/4096256734 >> v2:172.21.15.190:3300/0 conn(0x7f39c815dbd0 0x7f39c80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799346728 0 0) 0x7f39c0001490 con 0x7f39c80ce490 2023-10-16T09:59:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b80035b0 con 0x7f39c80ce490 2023-10-16T09:59:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659640430 0 0) 0x7f39b80035b0 con 0x7f39c80ce490 2023-10-16T09:59:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 >> v2:172.21.15.190:3300/0 conn(0x7f39c815dbd0 msgr2=0x7f39c80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 --2- 172.21.15.181:0/4096256734 >> v2:172.21.15.190:3300/0 conn(0x7f39c815dbd0 0x7f39c80d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 >> v1:172.21.15.190:6789/0 conn(0x7f39c80d0610 legacy=0x7f39c80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80d1fc0 con 0x7f39c80ce490 2023-10-16T09:59:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f39c0001b80 con 0x7f39c80ce490 2023-10-16T09:59:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 >> v1:172.21.15.181:6789/0 conn(0x7f39c80ce490 legacy=0x7f39c806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39b8003a00 0x7f39b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8004850 0x7f39b8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b8005210 0x7f39b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80cd830 con 0x7f39b8005210 2023-10-16T09:59:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b800c0b0 con 0x7f39b8003a00 2023-10-16T09:59:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b800c270 con 0x7f39b8004850 2023-10-16T09:59:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39cd7f0700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b8005210 0x7f39b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39ccfef700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39b8003a00 0x7f39b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.967+0000 7f39c7fff700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8004850 0x7f39b8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cd7f0700 1 -- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8004850 msgr2=0x7f39b8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cd7f0700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8004850 0x7f39b8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cd7f0700 1 -- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39b8003a00 msgr2=0x7f39b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cd7f0700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39b8003a00 0x7f39b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cd7f0700 1 -- 172.21.15.181:0/4096256734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b80035b0 con 0x7f39b8005210 2023-10-16T09:59:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cd7f0700 1 -- 172.21.15.181:0/4096256734 reap_dead start 2023-10-16T09:59:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cd7f0700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b8005210 0x7f39b800ba10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f39c000a800 tx=0x7f39b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8899861d8c0434b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c000a490 con 0x7f39b8005210 2023-10-16T09:59:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c0012620 con 0x7f39b8005210 2023-10-16T09:59:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39c77fe700 1 -- 172.21.15.181:0/4096256734 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c00129d0 con 0x7f39b8005210 2023-10-16T09:59:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cf253700 1 -- 172.21.15.181:0/4096256734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b8005210 msgr2=0x7f39b800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cf253700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b8005210 0x7f39b800ba10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f39c000a800 tx=0x7f39b8005880 comp rx=0 tx=0).stop 2023-10-16T09:59:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cf253700 1 -- 172.21.15.181:0/4096256734 shutdown_connections 2023-10-16T09:59:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cf253700 1 --2- 172.21.15.181:0/4096256734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b8005210 0x7f39b800ba10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cf253700 1 -- 172.21.15.181:0/4096256734 >> 172.21.15.181:0/4096256734 conn(0x7f39c809a930 msgr2=0x7f39c80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cf253700 1 -- 172.21.15.181:0/4096256734 shutdown_connections 2023-10-16T09:59:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.968+0000 7f39cf253700 1 -- 172.21.15.181:0/4096256734 wait complete. 2023-10-16T09:59:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 Processor -- start 2023-10-16T09:59:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 -- start start 2023-10-16T09:59:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8005210 0x7f39c81f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39c80af220 0x7f39c8155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c80af6a0 0x7f39c81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80b4190 con 0x7f39c80af220 2023-10-16T09:59:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80b43a0 con 0x7f39c80af6a0 2023-10-16T09:59:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.970+0000 7f39cf253700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80b4680 con 0x7f39b8005210 2023-10-16T09:59:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39c80af220 0x7f39c8155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39c80af220 0x7f39c8155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40830/0 (socket says 172.21.15.181:40830) 2023-10-16T09:59:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 -- 172.21.15.181:0/425114500 learned_addr learned my addr 172.21.15.181:0/425114500 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39cd7f0700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c80af6a0 0x7f39c81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39ccfef700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8005210 0x7f39c81f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 -- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8005210 msgr2=0x7f39c81f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8005210 0x7f39c81f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 -- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c80af6a0 msgr2=0x7f39c81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c80af6a0 0x7f39c81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 -- 172.21.15.181:0/425114500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80b4960 con 0x7f39c80af220 2023-10-16T09:59:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39cd7f0700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c80af6a0 0x7f39c81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39ccfef700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8005210 0x7f39c81f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c7fff700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39c80af220 0x7f39c8155060 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f39b400e490 tx=0x7f39b400e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c57fa700 1 -- 172.21.15.181:0/425114500 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40052e0 con 0x7f39c80af220 2023-10-16T09:59:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c57fa700 1 -- 172.21.15.181:0/425114500 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b4005440 con 0x7f39c80af220 2023-10-16T09:59:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39c57fa700 1 -- 172.21.15.181:0/425114500 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40038c0 con 0x7f39c80af220 2023-10-16T09:59:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c80b4c40 con 0x7f39c80af220 2023-10-16T09:59:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.971+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c81f4570 con 0x7f39c80af220 2023-10-16T09:59:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.972+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c8067d20 con 0x7f39c80af220 2023-10-16T09:59:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.973+0000 7f39c57fa700 1 -- 172.21.15.181:0/425114500 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f39b4003a20 con 0x7f39c80af220 2023-10-16T09:59:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.973+0000 7f39c57fa700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f39ac06b070 0x7f39ac06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.973+0000 7f39c57fa700 1 -- 172.21.15.181:0/425114500 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f39b4089110 con 0x7f39c80af220 2023-10-16T09:59:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.973+0000 7f39ccfef700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f39ac06b070 0x7f39ac06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.973+0000 7f39ccfef700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f39ac06b070 0x7f39ac06d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009be0 tx=0x7f39bc008040 comp rx=0 tx=0).ready entity=mgr.6120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:02.978+0000 7f39c57fa700 1 -- 172.21.15.181:0/425114500 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f39b4053e40 con 0x7f39c80af220 2023-10-16T09:59:03.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:03.230+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f39c80cebb0 con 0x7f39c80af220 2023-10-16T09:59:04.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.200+0000 7f39c57fa700 1 -- 172.21.15.181:0/425114500 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v28) v1 ==== 93+0+0 (secure 0 0 0) 0x7f39b4057460 con 0x7f39c80af220 2023-10-16T09:59:04.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f39ac06b070 msgr2=0x7f39ac06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f39ac06b070 0x7f39ac06d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009be0 tx=0x7f39bc008040 comp rx=0 tx=0).stop 2023-10-16T09:59:04.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39c80af220 msgr2=0x7f39c8155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39c80af220 0x7f39c8155060 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f39b400e490 tx=0x7f39b400e850 comp rx=0 tx=0).stop 2023-10-16T09:59:04.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 shutdown_connections 2023-10-16T09:59:04.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f39ac06b070 0x7f39ac06d530 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39b8005210 0x7f39c81f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c80af6a0 0x7f39c81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 --2- 172.21.15.181:0/425114500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39c80af220 0x7f39c8155060 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 >> 172.21.15.181:0/425114500 conn(0x7f39c809a930 msgr2=0x7f39c806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:04.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 shutdown_connections 2023-10-16T09:59:04.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.205+0000 7f39cf253700 1 -- 172.21.15.181:0/425114500 wait complete. 2023-10-16T09:59:04.224 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:59:04.224 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:04.277 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:04.315 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:59:04.315 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:04.316 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:04.333 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:04.333 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:04.346 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:04.344+0000 7f30d3d09200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:59:04.362 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:04.361+0000 7efe1401b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:59:04.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:04.411 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:04.409+0000 7f30d3d09200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:04.430 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:04.428+0000 7efe1401b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:04.483 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:04.481+0000 7f30d3d09200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:59:04.505 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:04.505+0000 7efe1401b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:59:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.545+0000 7fbfdf2fb700 1 Processor -- start 2023-10-16T09:59:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.545+0000 7fbfdf2fb700 1 -- start start 2023-10-16T09:59:04.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdf2fb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80ce7f0 0x7fbfd806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdf2fb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfd80d0350 0x7fbfd80cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdf2fb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd80cc470 con 0x7fbfd80d2390 2023-10-16T09:59:04.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdf2fb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd806ca30 con 0x7fbfd80d0350 2023-10-16T09:59:04.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdf2fb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd806cd10 con 0x7fbfd80ce7f0 2023-10-16T09:59:04.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdd898700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfd80d0350 0x7fbfd80cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdc896700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbfd80d2390 0x7fbfd80cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:51984/0 (socket says 172.21.15.181:51984) 2023-10-16T09:59:04.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdc896700 1 -- 172.21.15.181:0/3251214159 learned_addr learned my addr 172.21.15.181:0/3251214159 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:04.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdd097700 1 --2- 172.21.15.181:0/3251214159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80ce7f0 0x7fbfd806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdd898700 1 -- 172.21.15.181:0/3251214159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80ce7f0 msgr2=0x7fbfd806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:04.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdd898700 1 --2- 172.21.15.181:0/3251214159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80ce7f0 0x7fbfd806c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdd898700 1 -- 172.21.15.181:0/3251214159 >> v1:172.21.15.181:6789/0 conn(0x7fbfd80d2390 legacy=0x7fbfd80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfcffff700 1 -- 172.21.15.181:0/3251214159 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004736176 0 0) 0x7fbfd80cc470 con 0x7fbfd80d2390 2023-10-16T09:59:04.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdd898700 1 -- 172.21.15.181:0/3251214159 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd806cff0 con 0x7fbfd80d0350 2023-10-16T09:59:04.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfdd898700 1 --2- 172.21.15.181:0/3251214159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd80cc930 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009d90 tx=0x7fbfd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16a0454e8df30af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:04.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfcffff700 1 -- 172.21.15.181:0/3251214159 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000e070 con 0x7fbfd80d0350 2023-10-16T09:59:04.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfcffff700 1 -- 172.21.15.181:0/3251214159 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd000bb20 con 0x7fbfd80d0350 2023-10-16T09:59:04.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.546+0000 7fbfcffff700 1 -- 172.21.15.181:0/3251214159 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0012430 con 0x7fbfd80d0350 2023-10-16T09:59:04.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/3251214159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 msgr2=0x7fbfd80cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/3251214159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd80cc930 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009d90 tx=0x7fbfd000b330 comp rx=0 tx=0).stop 2023-10-16T09:59:04.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/3251214159 shutdown_connections 2023-10-16T09:59:04.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/3251214159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd80cc930 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/3251214159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80ce7f0 0x7fbfd806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/3251214159 >> 172.21.15.181:0/3251214159 conn(0x7fbfd809a9a0 msgr2=0x7fbfd80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:04.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/3251214159 shutdown_connections 2023-10-16T09:59:04.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.547+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/3251214159 wait complete. 2023-10-16T09:59:04.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.548+0000 7fbfdf2fb700 1 Processor -- start 2023-10-16T09:59:04.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.548+0000 7fbfdf2fb700 1 -- start start 2023-10-16T09:59:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdf2fb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd80ce7f0 0x7fbfd809d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdf2fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd81f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdf2fb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80d2390 0x7fbfd809ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdf2fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd809d9b0 con 0x7fbfd80d0350 2023-10-16T09:59:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdf2fb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd809dc90 con 0x7fbfd80ce7f0 2023-10-16T09:59:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdf2fb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd809df70 con 0x7fbfd80d2390 2023-10-16T09:59:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdc896700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd81f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdc896700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd81f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40912/0 (socket says 172.21.15.181:40912) 2023-10-16T09:59:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdc896700 1 -- 172.21.15.181:0/2295628081 learned_addr learned my addr 172.21.15.181:0/2295628081 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdd898700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80d2390 0x7fbfd809ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.549+0000 7fbfdd097700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd80ce7f0 0x7fbfd809d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdc896700 1 -- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80d2390 msgr2=0x7fbfd809ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdc896700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80d2390 0x7fbfd809ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdc896700 1 -- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd80ce7f0 msgr2=0x7fbfd809d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdc896700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd80ce7f0 0x7fbfd809d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdc896700 1 -- 172.21.15.181:0/2295628081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0009a70 con 0x7fbfd80d0350 2023-10-16T09:59:04.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdc896700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd81f2bd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80011e0 tx=0x7fbfc800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:04.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfcdffb700 1 -- 172.21.15.181:0/2295628081 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8004c80 con 0x7fbfd80d0350 2023-10-16T09:59:04.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfcdffb700 1 -- 172.21.15.181:0/2295628081 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc8004e00 con 0x7fbfd80d0350 2023-10-16T09:59:04.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfcdffb700 1 -- 172.21.15.181:0/2295628081 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8007310 con 0x7fbfd80d0350 2023-10-16T09:59:04.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd809e250 con 0x7fbfd80d0350 2023-10-16T09:59:04.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.550+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd809ef90 con 0x7fbfd80d0350 2023-10-16T09:59:04.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.551+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd809e770 con 0x7fbfd80d0350 2023-10-16T09:59:04.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.551+0000 7fbfcdffb700 1 -- 172.21.15.181:0/2295628081 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fbfc8007470 con 0x7fbfd80d0350 2023-10-16T09:59:04.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.552+0000 7fbfcdffb700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 0x7fbfbc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.552+0000 7fbfdd097700 1 -- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 msgr2=0x7fbfbc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:04.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.552+0000 7fbfcdffb700 1 -- 172.21.15.181:0/2295628081 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7fbfc8089730 con 0x7fbfd80d0350 2023-10-16T09:59:04.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.552+0000 7fbfdd097700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 0x7fbfbc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:04.567 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:04.559+0000 7f30d3d09200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:59:04.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.557+0000 7fbfcdffb700 1 -- 172.21.15.181:0/2295628081 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfc80543b0 con 0x7fbfd80d0350 2023-10-16T09:59:04.589 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:04.587+0000 7efe1401b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:59:04.701 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:04.700+0000 7f30d3d09200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:59:04.734 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:04.733+0000 7efe1401b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:59:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.753+0000 7fbfdd097700 1 -- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 msgr2=0x7fbfbc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.753+0000 7fbfdd097700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 0x7fbfbc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.758+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfd8067d20 con 0x7fbfd80d0350 2023-10-16T09:59:04.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.759+0000 7fbfcdffb700 1 -- 172.21.15.181:0/2295628081 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17749 (secure 0 0 0) 0x7fbfc80579d0 con 0x7fbfd80d0350 2023-10-16T09:59:04.761 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:04.761 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":62,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:58:17.034994+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:57:00.297618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T09:57:00.838546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2507249913":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2125399270":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2970752475":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/3339446951":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2864335478":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6833/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6832/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/534300178":"2023-10-17T09:57:15.720457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:04.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.761+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 msgr2=0x7fbfbc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.761+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 0x7fbfbc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.761+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 msgr2=0x7fbfd81f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.761+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd81f2bd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80011e0 tx=0x7fbfc800ca30 comp rx=0 tx=0).stop 2023-10-16T09:59:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 shutdown_connections 2023-10-16T09:59:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbfbc06b020 0x7fbfbc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfd80d2390 0x7fbfd809ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd80ce7f0 0x7fbfd809d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 --2- 172.21.15.181:0/2295628081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80d0350 0x7fbfd81f2bd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 >> 172.21.15.181:0/2295628081 conn(0x7fbfd809a9a0 msgr2=0x7fbfd80c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:04.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 shutdown_connections 2023-10-16T09:59:04.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.762+0000 7fbfdf2fb700 1 -- 172.21.15.181:0/2295628081 wait complete. 2023-10-16T09:59:04.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:57:00.297618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:57:00.838546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:04.778 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:04.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.941+0000 7f7508a56700 1 Processor -- start 2023-10-16T09:59:04.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.941+0000 7f7508a56700 1 -- start start 2023-10-16T09:59:04.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.942+0000 7f7508a56700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f75040ce2d0 0x7f750415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.942+0000 7f7508a56700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750415ef80 con 0x7f75040d1500 2023-10-16T09:59:04.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.942+0000 7f7508a56700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75040cf580 con 0x7f75040d01d0 2023-10-16T09:59:04.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.942+0000 7f7508a56700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75040d4290 con 0x7f75040ce2d0 2023-10-16T09:59:04.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.942+0000 7f7501d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f75040ce2d0 0x7f750415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.942+0000 7f7502d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f75040d1500 0x7f75040d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52046/0 (socket says 172.21.15.181:52046) 2023-10-16T09:59:04.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.942+0000 7f7502d9d700 1 -- 172.21.15.181:0/2920063941 learned_addr learned my addr 172.21.15.181:0/2920063941 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:04.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f750159a700 1 -- 172.21.15.181:0/2920063941 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040829533 0 0) 0x7f74fc001490 con 0x7f75040d1500 2023-10-16T09:59:04.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f750159a700 1 -- 172.21.15.181:0/2920063941 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec0035b0 con 0x7f75040d1500 2023-10-16T09:59:04.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f7501d9b700 1 -- 172.21.15.181:0/2920063941 >> v1:172.21.15.190:6789/0 conn(0x7f75040d01d0 legacy=0x7f75040d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f750159a700 1 -- 172.21.15.181:0/2920063941 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780428236 0 0) 0x7f75040cf580 con 0x7f75040d01d0 2023-10-16T09:59:04.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f7501d9b700 1 -- 172.21.15.181:0/2920063941 >> v1:172.21.15.181:6789/0 conn(0x7f75040d1500 legacy=0x7f75040d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f7501d9b700 1 -- 172.21.15.181:0/2920063941 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040ce970 con 0x7f75040ce2d0 2023-10-16T09:59:04.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f7501d9b700 1 --2- 172.21.15.181:0/2920063941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f750415ea40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009840 tx=0x7f74f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a70c549998fab3b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:04.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f750159a700 1 -- 172.21.15.181:0/2920063941 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8011830 con 0x7f75040ce2d0 2023-10-16T09:59:04.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f750159a700 1 -- 172.21.15.181:0/2920063941 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f8011990 con 0x7f75040ce2d0 2023-10-16T09:59:04.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.943+0000 7f750159a700 1 -- 172.21.15.181:0/2920063941 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8011d20 con 0x7f75040ce2d0 2023-10-16T09:59:04.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.944+0000 7f7508a56700 1 -- 172.21.15.181:0/2920063941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 msgr2=0x7f750415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.944+0000 7f7508a56700 1 --2- 172.21.15.181:0/2920063941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f750415ea40 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009840 tx=0x7f74f8009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:04.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.944+0000 7f7508a56700 1 -- 172.21.15.181:0/2920063941 shutdown_connections 2023-10-16T09:59:04.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.944+0000 7f7508a56700 1 --2- 172.21.15.181:0/2920063941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f750415ea40 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.944+0000 7f7508a56700 1 -- 172.21.15.181:0/2920063941 >> 172.21.15.181:0/2920063941 conn(0x7f750409cb60 msgr2=0x7f75040d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:04.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.944+0000 7f7508a56700 1 -- 172.21.15.181:0/2920063941 shutdown_connections 2023-10-16T09:59:04.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.944+0000 7f7508a56700 1 -- 172.21.15.181:0/2920063941 wait complete. 2023-10-16T09:59:04.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.946+0000 7f7508a56700 1 Processor -- start 2023-10-16T09:59:04.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.946+0000 7f7508a56700 1 -- start start 2023-10-16T09:59:04.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7508a56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f75041552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7508a56700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75040d01d0 0x7f75040c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7508a56700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75040d1500 0x7f75040c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7508a56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7504157530 con 0x7f75040ce2d0 2023-10-16T09:59:04.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7508a56700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7504157810 con 0x7f75040d01d0 2023-10-16T09:59:04.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7508a56700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75040b7250 con 0x7f75040d1500 2023-10-16T09:59:04.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f75041552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f75041552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40970/0 (socket says 172.21.15.181:40970) 2023-10-16T09:59:04.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 -- 172.21.15.181:0/2759454245 learned_addr learned my addr 172.21.15.181:0/2759454245 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:04.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7502d9d700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75040d1500 0x7f75040c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7501d9b700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75040d01d0 0x7f75040c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:04.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 -- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75040d1500 msgr2=0x7f75040c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75040d1500 0x7f75040c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 -- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75040d01d0 msgr2=0x7f75040c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:04.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75040d01d0 0x7f75040c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:04.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 -- 172.21.15.181:0/2759454245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f80094a0 con 0x7f75040ce2d0 2023-10-16T09:59:04.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f750259c700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f75041552c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f74f40011e0 tx=0x7f74f400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:04.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f74f2ffd700 1 -- 172.21.15.181:0/2759454245 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f40054e0 con 0x7f75040ce2d0 2023-10-16T09:59:04.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f74f2ffd700 1 -- 172.21.15.181:0/2759454245 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f4003680 con 0x7f75040ce2d0 2023-10-16T09:59:04.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75040b7530 con 0x7f75040ce2d0 2023-10-16T09:59:04.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.947+0000 7f74f2ffd700 1 -- 172.21.15.181:0/2759454245 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4003a30 con 0x7f75040ce2d0 2023-10-16T09:59:04.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.948+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7504157170 con 0x7f75040ce2d0 2023-10-16T09:59:04.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.949+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75040b70a0 con 0x7f75040ce2d0 2023-10-16T09:59:04.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.949+0000 7f74f2ffd700 1 -- 172.21.15.181:0/2759454245 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f74f4003b90 con 0x7f75040ce2d0 2023-10-16T09:59:04.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.949+0000 7f74f2ffd700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 0x7f74e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:04.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.949+0000 7f7501d9b700 1 -- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 msgr2=0x7f74e406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:04.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.949+0000 7f74f2ffd700 1 -- 172.21.15.181:0/2759454245 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f74f4088e90 con 0x7f75040ce2d0 2023-10-16T09:59:04.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.949+0000 7f7501d9b700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 0x7f74e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:04.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:04.954+0000 7f74f2ffd700 1 -- 172.21.15.181:0/2759454245 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f74f4053b20 con 0x7f75040ce2d0 2023-10-16T09:59:05.024 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:05.023+0000 7f30d3d09200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:59:05.072 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:05.070+0000 7efe1401b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:59:05.140 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:05.138+0000 7f30d3d09200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:59:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.149+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f750404ebb0 con 0x7f75040ce2d0 2023-10-16T09:59:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.149+0000 7f74f2ffd700 1 -- 172.21.15.181:0/2759454245 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74f4017020 con 0x7f75040ce2d0 2023-10-16T09:59:05.151 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.150+0000 7f7501d9b700 1 -- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 msgr2=0x7f74e406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:05.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.150+0000 7f7501d9b700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 0x7f74e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:05.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 msgr2=0x7f74e406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:05.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 0x7f74e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 msgr2=0x7f75041552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f75041552c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f74f40011e0 tx=0x7f74f400ca30 comp rx=0 tx=0).stop 2023-10-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 shutdown_connections 2023-10-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74e406b020 0x7f74e406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75040d1500 0x7f75040c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75040ce2d0 0x7f75041552c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 --2- 172.21.15.181:0/2759454245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75040d01d0 0x7f75040c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 >> 172.21.15.181:0/2759454245 conn(0x7f750409cb60 msgr2=0x7f75040b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 shutdown_connections 2023-10-16T09:59:05.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.151+0000 7f7508a56700 1 -- 172.21.15.181:0/2759454245 wait complete. 2023-10-16T09:59:05.167 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:59:05.192 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:05.192+0000 7efe1401b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:59:05.213 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:05.212+0000 7f30d3d09200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:59:05.269 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:05.269+0000 7efe1401b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:59:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.292+0000 7f25067ec700 1 Processor -- start 2023-10-16T09:59:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.292+0000 7f25067ec700 1 -- start start 2023-10-16T09:59:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.293+0000 7f25067ec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.293+0000 7f25067ec700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25000b2ae0 con 0x7f25000ce4f0 2023-10-16T09:59:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.293+0000 7f25067ec700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25000b2dc0 con 0x7f25000cd170 2023-10-16T09:59:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.293+0000 7f25067ec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25000ce970 con 0x7f250015f150 2023-10-16T09:59:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.293+0000 7f24ff7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f25000ce4f0 0x7f25000b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52060/0 (socket says 172.21.15.181:52060) 2023-10-16T09:59:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.293+0000 7f24ff7fe700 1 -- 172.21.15.181:0/3421369816 learned_addr learned my addr 172.21.15.181:0/3421369816 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.293+0000 7f24fffff700 1 --2- 172.21.15.181:0/3421369816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24feffd700 1 -- 172.21.15.181:0/3421369816 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218872098 0 0) 0x7f25000b2ae0 con 0x7f25000ce4f0 2023-10-16T09:59:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24feffd700 1 -- 172.21.15.181:0/3421369816 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e80035b0 con 0x7f25000ce4f0 2023-10-16T09:59:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24feffd700 1 -- 172.21.15.181:0/3421369816 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371542368 0 0) 0x7f25000b2dc0 con 0x7f25000cd170 2023-10-16T09:59:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24feffd700 1 -- 172.21.15.181:0/3421369816 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25000b2ae0 con 0x7f25000cd170 2023-10-16T09:59:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24fffff700 1 -- 172.21.15.181:0/3421369816 >> v1:172.21.15.190:6789/0 conn(0x7f25000cd170 legacy=0x7f25000b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24fffff700 1 -- 172.21.15.181:0/3421369816 >> v1:172.21.15.181:6789/0 conn(0x7f25000ce4f0 legacy=0x7f25000b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24fffff700 1 -- 172.21.15.181:0/3421369816 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000cf9d0 con 0x7f250015f150 2023-10-16T09:59:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24fffff700 1 --2- 172.21.15.181:0/3421369816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000cdfb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009870 tx=0x7f24f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d2fac525509a2a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24feffd700 1 -- 172.21.15.181:0/3421369816 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0011830 con 0x7f250015f150 2023-10-16T09:59:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24feffd700 1 -- 172.21.15.181:0/3421369816 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f0011990 con 0x7f250015f150 2023-10-16T09:59:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.294+0000 7f24feffd700 1 -- 172.21.15.181:0/3421369816 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0011d00 con 0x7f250015f150 2023-10-16T09:59:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.295+0000 7f25067ec700 1 -- 172.21.15.181:0/3421369816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 msgr2=0x7f25000cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.295+0000 7f25067ec700 1 --2- 172.21.15.181:0/3421369816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000cdfb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009870 tx=0x7f24f0009b80 comp rx=0 tx=0).stop 2023-10-16T09:59:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.295+0000 7f25067ec700 1 -- 172.21.15.181:0/3421369816 shutdown_connections 2023-10-16T09:59:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.295+0000 7f25067ec700 1 --2- 172.21.15.181:0/3421369816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000cdfb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.295+0000 7f25067ec700 1 -- 172.21.15.181:0/3421369816 >> 172.21.15.181:0/3421369816 conn(0x7f250009a930 msgr2=0x7f25000d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.295+0000 7f25067ec700 1 -- 172.21.15.181:0/3421369816 shutdown_connections 2023-10-16T09:59:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.295+0000 7f25067ec700 1 -- 172.21.15.181:0/3421369816 wait complete. 2023-10-16T09:59:05.302 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:05.293+0000 7f30d3d09200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:59:05.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.297+0000 7f25067ec700 1 Processor -- start 2023-10-16T09:59:05.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.297+0000 7f25067ec700 1 -- start start 2023-10-16T09:59:05.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f25067ec700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25000cd170 0x7f25001596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f25067ec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25000ce4f0 0x7f25000c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f25067ec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f25067ec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250015e340 con 0x7f25000ce4f0 2023-10-16T09:59:05.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f25067ec700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250006c6b0 con 0x7f25000cd170 2023-10-16T09:59:05.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f25067ec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250006c990 con 0x7f250015f150 2023-10-16T09:59:05.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25000ce4f0 0x7f25000c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25000ce4f0 0x7f25000c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:40988/0 (socket says 172.21.15.181:40988) 2023-10-16T09:59:05.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 -- 172.21.15.181:0/3015187815 learned_addr learned my addr 172.21.15.181:0/3015187815 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:05.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24fffff700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25000cd170 0x7f25001596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f2504d89700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 -- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 msgr2=0x7f25000d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 -- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25000cd170 msgr2=0x7f25001596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25000cd170 0x7f25001596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.298+0000 7f24ff7fe700 1 -- 172.21.15.181:0/3015187815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f00094a0 con 0x7f25000ce4f0 2023-10-16T09:59:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.299+0000 7f24ff7fe700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25000ce4f0 0x7f25000c6610 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c700 tx=0x7f24f400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.299+0000 7f24fcff9700 1 -- 172.21.15.181:0/3015187815 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4005d70 con 0x7f25000ce4f0 2023-10-16T09:59:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.299+0000 7f24fcff9700 1 -- 172.21.15.181:0/3015187815 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f40070d0 con 0x7f25000ce4f0 2023-10-16T09:59:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.299+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250006cc70 con 0x7f25000ce4f0 2023-10-16T09:59:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.299+0000 7f24fcff9700 1 -- 172.21.15.181:0/3015187815 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4007440 con 0x7f25000ce4f0 2023-10-16T09:59:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.299+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250006d230 con 0x7f25000ce4f0 2023-10-16T09:59:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.300+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2500067d20 con 0x7f25000ce4f0 2023-10-16T09:59:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.300+0000 7f24fcff9700 1 -- 172.21.15.181:0/3015187815 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f24f40075a0 con 0x7f25000ce4f0 2023-10-16T09:59:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.301+0000 7f24fcff9700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 0x7f24ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.301+0000 7f24fffff700 1 -- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 msgr2=0x7f24ec06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:05.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.301+0000 7f24fcff9700 1 -- 172.21.15.181:0/3015187815 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f24f4088cf0 con 0x7f25000ce4f0 2023-10-16T09:59:05.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.301+0000 7f24fffff700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 0x7f24ec06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:05.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.305+0000 7f24fcff9700 1 -- 172.21.15.181:0/3015187815 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f24f408d070 con 0x7f25000ce4f0 2023-10-16T09:59:05.357 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:05.356+0000 7efe1401b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:59:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.502+0000 7f24fffff700 1 -- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 msgr2=0x7f24ec06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:05.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.502+0000 7f24fffff700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 0x7f24ec06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:05.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.505+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25000b71c0 con 0x7f25000ce4f0 2023-10-16T09:59:05.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.505+0000 7f24fcff9700 1 -- 172.21.15.181:0/3015187815 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24f40170a0 con 0x7f25000ce4f0 2023-10-16T09:59:05.506 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:59:05.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 msgr2=0x7f24ec06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:05.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 0x7f24ec06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25000ce4f0 msgr2=0x7f25000c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25000ce4f0 0x7f25000c6610 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c700 tx=0x7f24f400ca10 comp rx=0 tx=0).stop 2023-10-16T09:59:05.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 shutdown_connections 2023-10-16T09:59:05.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f24ec06b020 0x7f24ec06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f250015f150 0x7f25000d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25000cd170 0x7f25001596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 --2- 172.21.15.181:0/3015187815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25000ce4f0 0x7f25000c6610 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 >> 172.21.15.181:0/3015187815 conn(0x7f250009a930 msgr2=0x7f25000ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:05.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 shutdown_connections 2023-10-16T09:59:05.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.507+0000 7f25067ec700 1 -- 172.21.15.181:0/3015187815 wait complete. 2023-10-16T09:59:05.521 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:59:05.660 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:05.659+0000 7f30d3d09200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:59:05.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.687+0000 7f83ba367700 1 Processor -- start 2023-10-16T09:59:05.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.687+0000 7f83ba367700 1 -- start start 2023-10-16T09:59:05.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.688+0000 7f83ba367700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40ce6c0 0x7f83b40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.688+0000 7f83ba367700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b40b1f70 con 0x7f83b40cc6a0 2023-10-16T09:59:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.688+0000 7f83ba367700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b40b2ae0 con 0x7f83b415f150 2023-10-16T09:59:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.688+0000 7f83ba367700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40b2dc0 con 0x7f83b40ce6c0 2023-10-16T09:59:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.688+0000 7f83b8904700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f83b40cc6a0 0x7f83b40ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52094/0 (socket says 172.21.15.181:52094) 2023-10-16T09:59:05.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.688+0000 7f83b8904700 1 -- 172.21.15.181:0/2620871540 learned_addr learned my addr 172.21.15.181:0/2620871540 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.688+0000 7f83b37fe700 1 --2- 172.21.15.181:0/2620871540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40ce6c0 0x7f83b40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b2ffd700 1 -- 172.21.15.181:0/2620871540 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794210399 0 0) 0x7f83b40b1f70 con 0x7f83b40cc6a0 2023-10-16T09:59:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b2ffd700 1 -- 172.21.15.181:0/2620871540 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f839c0035b0 con 0x7f83b40cc6a0 2023-10-16T09:59:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b2ffd700 1 -- 172.21.15.181:0/2620871540 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843516455 0 0) 0x7f83b40b2ae0 con 0x7f83b415f150 2023-10-16T09:59:05.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b37fe700 1 -- 172.21.15.181:0/2620871540 >> v1:172.21.15.190:6789/0 conn(0x7f83b415f150 legacy=0x7f83b40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b37fe700 1 -- 172.21.15.181:0/2620871540 >> v1:172.21.15.181:6789/0 conn(0x7f83b40cc6a0 legacy=0x7f83b40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b37fe700 1 -- 172.21.15.181:0/2620871540 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40cf9d0 con 0x7f83b40ce6c0 2023-10-16T09:59:05.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b37fe700 1 --2- 172.21.15.181:0/2620871540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40ce6c0 0x7f83b40cc160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009870 tx=0x7f83a8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e3cab492939ab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:05.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b2ffd700 1 -- 172.21.15.181:0/2620871540 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0019f0 con 0x7f83b40ce6c0 2023-10-16T09:59:05.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b2ffd700 1 -- 172.21.15.181:0/2620871540 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a8004030 con 0x7f83b40ce6c0 2023-10-16T09:59:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.689+0000 7f83b2ffd700 1 -- 172.21.15.181:0/2620871540 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80043c0 con 0x7f83b40ce6c0 2023-10-16T09:59:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.690+0000 7f83ba367700 1 -- 172.21.15.181:0/2620871540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40ce6c0 msgr2=0x7f83b40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.690+0000 7f83ba367700 1 --2- 172.21.15.181:0/2620871540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40ce6c0 0x7f83b40cc160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009870 tx=0x7f83a8009b80 comp rx=0 tx=0).stop 2023-10-16T09:59:05.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.690+0000 7f83ba367700 1 -- 172.21.15.181:0/2620871540 shutdown_connections 2023-10-16T09:59:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.690+0000 7f83ba367700 1 --2- 172.21.15.181:0/2620871540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40ce6c0 0x7f83b40cc160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.690+0000 7f83ba367700 1 -- 172.21.15.181:0/2620871540 >> 172.21.15.181:0/2620871540 conn(0x7f83b409a930 msgr2=0x7f83b40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.690+0000 7f83ba367700 1 -- 172.21.15.181:0/2620871540 shutdown_connections 2023-10-16T09:59:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.690+0000 7f83ba367700 1 -- 172.21.15.181:0/2620871540 wait complete. 2023-10-16T09:59:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.691+0000 7f83ba367700 1 Processor -- start 2023-10-16T09:59:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.691+0000 7f83ba367700 1 -- start start 2023-10-16T09:59:05.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83ba367700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40cc6a0 0x7f83b4159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83ba367700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b40ce6c0 0x7f83b4157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83ba367700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83b415f150 0x7f83b40c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83ba367700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40af2e0 con 0x7f83b415f150 2023-10-16T09:59:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83ba367700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40afdd0 con 0x7f83b40ce6c0 2023-10-16T09:59:05.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83ba367700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b415a300 con 0x7f83b40cc6a0 2023-10-16T09:59:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83b415f150 0x7f83b40c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83b415f150 0x7f83b40c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41022/0 (socket says 172.21.15.181:41022) 2023-10-16T09:59:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 -- 172.21.15.181:0/791670572 learned_addr learned my addr 172.21.15.181:0/791670572 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b37fe700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b40ce6c0 0x7f83b4157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b3fff700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40cc6a0 0x7f83b4159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 -- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40cc6a0 msgr2=0x7f83b4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40cc6a0 0x7f83b4159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 -- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b40ce6c0 msgr2=0x7f83b4157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b40ce6c0 0x7f83b4157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.692+0000 7f83b8904700 1 -- 172.21.15.181:0/791670572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a80094a0 con 0x7f83b415f150 2023-10-16T09:59:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.693+0000 7f83b8904700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83b415f150 0x7f83b40c4260 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f83ac002090 tx=0x7f83ac00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.693+0000 7f83b0ff9700 1 -- 172.21.15.181:0/791670572 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00acd0 con 0x7f83b415f150 2023-10-16T09:59:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.693+0000 7f83b0ff9700 1 -- 172.21.15.181:0/791670572 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac00ae50 con 0x7f83b415f150 2023-10-16T09:59:05.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.693+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b40af530 con 0x7f83b415f150 2023-10-16T09:59:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.693+0000 7f83b0ff9700 1 -- 172.21.15.181:0/791670572 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac007310 con 0x7f83b415f150 2023-10-16T09:59:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.693+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b40afaf0 con 0x7f83b415f150 2023-10-16T09:59:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.694+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b4067d20 con 0x7f83b415f150 2023-10-16T09:59:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.694+0000 7f83b0ff9700 1 -- 172.21.15.181:0/791670572 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f83ac007470 con 0x7f83b415f150 2023-10-16T09:59:05.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.695+0000 7f83b0ff9700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 0x7f83a006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.695+0000 7f83b3fff700 1 -- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 msgr2=0x7f83a006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.695+0000 7f83b0ff9700 1 -- 172.21.15.181:0/791670572 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f83ac0899a0 con 0x7f83b415f150 2023-10-16T09:59:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.695+0000 7f83b3fff700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 0x7f83a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:05.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.699+0000 7f83b0ff9700 1 -- 172.21.15.181:0/791670572 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f83ac0546b0 con 0x7f83b415f150 2023-10-16T09:59:05.742 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:05.741+0000 7efe1401b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:59:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.895+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83b40c8490 con 0x7f83b415f150 2023-10-16T09:59:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.896+0000 7f83b3fff700 1 -- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 msgr2=0x7f83a006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.896+0000 7f83b3fff700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 0x7f83a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.896+0000 7f83b0ff9700 1 -- 172.21.15.181:0/791670572 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f83ac0180a0 con 0x7f83b415f150 2023-10-16T09:59:05.898 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:59:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 msgr2=0x7f83a006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 0x7f83a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83b415f150 msgr2=0x7f83b40c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83b415f150 0x7f83b40c4260 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f83ac002090 tx=0x7f83ac00a340 comp rx=0 tx=0).stop 2023-10-16T09:59:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 shutdown_connections 2023-10-16T09:59:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f83a006b020 0x7f83a006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83b40cc6a0 0x7f83b4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83b415f150 0x7f83b40c4260 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 --2- 172.21.15.181:0/791670572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83b40ce6c0 0x7f83b4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 >> 172.21.15.181:0/791670572 conn(0x7f83b409a930 msgr2=0x7f83b415a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 shutdown_connections 2023-10-16T09:59:05.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:05.898+0000 7f83ba367700 1 -- 172.21.15.181:0/791670572 wait complete. 2023-10-16T09:59:05.911 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:59:06.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.035+0000 7f172b136700 1 Processor -- start 2023-10-16T09:59:06.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.035+0000 7f172b136700 1 -- start start 2023-10-16T09:59:06.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f172b136700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17240cd4b0 0x7f17240cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f172b136700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172406c700 con 0x7f172406d210 2023-10-16T09:59:06.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f172b136700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17240cf460 con 0x7f17240ce4d0 2023-10-16T09:59:06.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f172b136700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17240d0380 con 0x7f17240cd4b0 2023-10-16T09:59:06.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17296d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17240cd4b0 0x7f17240cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f1728ed2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f172406d210 0x7f17240cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52106/0 (socket says 172.21.15.181:52106) 2023-10-16T09:59:06.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f1728ed2700 1 -- 172.21.15.181:0/4185206264 learned_addr learned my addr 172.21.15.181:0/4185206264 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:06.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17296d3700 1 -- 172.21.15.181:0/4185206264 >> v1:172.21.15.190:6789/0 conn(0x7f17240ce4d0 legacy=0x7f172415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17296d3700 1 -- 172.21.15.181:0/4185206264 >> v1:172.21.15.181:6789/0 conn(0x7f172406d210 legacy=0x7f17240cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17296d3700 1 -- 172.21.15.181:0/4185206264 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172406d5f0 con 0x7f17240cd4b0 2023-10-16T09:59:06.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17296d3700 1 --2- 172.21.15.181:0/4185206264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240cd4b0 0x7f17240cc4c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f171c009d90 tx=0x7f171c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a7df5109d0df0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:06.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17237fe700 1 -- 172.21.15.181:0/4185206264 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c00e070 con 0x7f17240cd4b0 2023-10-16T09:59:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17237fe700 1 -- 172.21.15.181:0/4185206264 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c00bbd0 con 0x7f17240cd4b0 2023-10-16T09:59:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.036+0000 7f17237fe700 1 -- 172.21.15.181:0/4185206264 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c012430 con 0x7f17240cd4b0 2023-10-16T09:59:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.037+0000 7f172b136700 1 -- 172.21.15.181:0/4185206264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240cd4b0 msgr2=0x7f17240cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.037+0000 7f172b136700 1 --2- 172.21.15.181:0/4185206264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240cd4b0 0x7f17240cc4c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f171c009d90 tx=0x7f171c00b330 comp rx=0 tx=0).stop 2023-10-16T09:59:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.037+0000 7f172b136700 1 -- 172.21.15.181:0/4185206264 shutdown_connections 2023-10-16T09:59:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.037+0000 7f172b136700 1 --2- 172.21.15.181:0/4185206264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240cd4b0 0x7f17240cc4c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.037+0000 7f172b136700 1 -- 172.21.15.181:0/4185206264 >> 172.21.15.181:0/4185206264 conn(0x7f172409a9a0 msgr2=0x7f17240d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.037+0000 7f172b136700 1 -- 172.21.15.181:0/4185206264 shutdown_connections 2023-10-16T09:59:06.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.037+0000 7f172b136700 1 -- 172.21.15.181:0/4185206264 wait complete. 2023-10-16T09:59:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.038+0000 7f172b136700 1 Processor -- start 2023-10-16T09:59:06.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.038+0000 7f172b136700 1 -- start start 2023-10-16T09:59:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f172b136700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172406d210 0x7f172409b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f172b136700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17240cd4b0 0x7f172415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f172b136700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240ce4d0 0x7f172409cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f172b136700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172409bc60 con 0x7f17240ce4d0 2023-10-16T09:59:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f172b136700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172409bf40 con 0x7f17240cd4b0 2023-10-16T09:59:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f172b136700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172409c220 con 0x7f172406d210 2023-10-16T09:59:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f17296d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240ce4d0 0x7f172409cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f17296d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240ce4d0 0x7f172409cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41038/0 (socket says 172.21.15.181:41038) 2023-10-16T09:59:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f17296d3700 1 -- 172.21.15.181:0/3210761713 learned_addr learned my addr 172.21.15.181:0/3210761713 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f1723fff700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17240cd4b0 0x7f172415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.039+0000 7f1728ed2700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172406d210 0x7f172409b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17296d3700 1 -- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172406d210 msgr2=0x7f172409b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17296d3700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172406d210 0x7f172409b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17296d3700 1 -- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17240cd4b0 msgr2=0x7f172415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17296d3700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17240cd4b0 0x7f172415fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17296d3700 1 -- 172.21.15.181:0/3210761713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c009a70 con 0x7f17240ce4d0 2023-10-16T09:59:06.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17296d3700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240ce4d0 0x7f172409cda0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f171c014040 tx=0x7f171c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:06.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17217fa700 1 -- 172.21.15.181:0/3210761713 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c012d20 con 0x7f17240ce4d0 2023-10-16T09:59:06.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17217fa700 1 -- 172.21.15.181:0/3210761713 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c020070 con 0x7f17240ce4d0 2023-10-16T09:59:06.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172409c500 con 0x7f17240ce4d0 2023-10-16T09:59:06.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f17217fa700 1 -- 172.21.15.181:0/3210761713 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c012ea0 con 0x7f17240ce4d0 2023-10-16T09:59:06.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.040+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1724160240 con 0x7f17240ce4d0 2023-10-16T09:59:06.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.041+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17240c50a0 con 0x7f17240ce4d0 2023-10-16T09:59:06.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.042+0000 7f17217fa700 1 -- 172.21.15.181:0/3210761713 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f171c004280 con 0x7f17240ce4d0 2023-10-16T09:59:06.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.042+0000 7f17217fa700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 0x7f170806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.042+0000 7f1728ed2700 1 -- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 msgr2=0x7f170806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:06.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.042+0000 7f17217fa700 1 -- 172.21.15.181:0/3210761713 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f171c022070 con 0x7f17240ce4d0 2023-10-16T09:59:06.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.042+0000 7f1728ed2700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 0x7f170806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:06.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.046+0000 7f17217fa700 1 -- 172.21.15.181:0/3210761713 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f171c056b60 con 0x7f17240ce4d0 2023-10-16T09:59:06.227 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:06.225+0000 7f30d3d09200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:59:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.243+0000 7f1728ed2700 1 -- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 msgr2=0x7f170806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.243+0000 7f1728ed2700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 0x7f170806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.295+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1724067d20 con 0x7f17240ce4d0 2023-10-16T09:59:06.295 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:06.294+0000 7f30d3d09200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:59:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.295+0000 7f17217fa700 1 -- 172.21.15.181:0/3210761713 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v62) v1 ==== 80+0+477 (secure 0 0 0) 0x7f171c02c020 con 0x7f17240ce4d0 2023-10-16T09:59:06.296 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2507249913 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.297 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2125399270 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.297 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2970752475 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.297 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3339446951 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.297 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2864335478 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.297 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3219577148 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.298 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3219577148 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.298 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/534300178 2023-10-17T09:57:15.720457+0000 2023-10-16T09:59:06.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 msgr2=0x7f170806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 0x7f170806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240ce4d0 msgr2=0x7f172409cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240ce4d0 0x7f172409cda0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f171c014040 tx=0x7f171c004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 shutdown_connections 2023-10-16T09:59:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f170806b020 0x7f170806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f172406d210 0x7f172409b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17240ce4d0 0x7f172409cda0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 --2- 172.21.15.181:0/3210761713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17240cd4b0 0x7f172415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.297+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 >> 172.21.15.181:0/3210761713 conn(0x7f172409a9a0 msgr2=0x7f17240c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:06.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.298+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 shutdown_connections 2023-10-16T09:59:06.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.298+0000 7f172b136700 1 -- 172.21.15.181:0/3210761713 wait complete. 2023-10-16T09:59:06.302 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T09:59:06.312 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T09:59:06.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:06.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T09:59:06.333 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:06.333+0000 7efe1401b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:59:06.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:59:06.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:06.373 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T09:59:06.407 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:06.406+0000 7efe1401b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:59:06.423 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:06.423 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:06.439 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:06.440 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:06.440 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:06.479 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:59:06.480 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:06.480 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:06.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:06.499 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:06.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.641+0000 7f07c6a7e700 1 Processor -- start 2023-10-16T09:59:06.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.641+0000 7f07c6a7e700 1 -- start start 2023-10-16T09:59:06.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c6a7e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cac20 0x7f07c015af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c6a7e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f07c00cb000 0x7f07c015da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c6a7e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c015df40 con 0x7f07c00cc4d0 2023-10-16T09:59:06.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c6a7e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c00cc0b0 con 0x7f07c00cb000 2023-10-16T09:59:06.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c6a7e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c015e210 con 0x7f07c00cac20 2023-10-16T09:59:06.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f07c00cb000 0x7f07c015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c501b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f07c00cc4d0 0x7f07c00cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:36068/0 (socket says 172.21.15.181:36068) 2023-10-16T09:59:06.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c481a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cac20 0x7f07c015af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07c501b700 1 -- 172.21.15.181:0/1958431853 learned_addr learned my addr 172.21.15.181:0/1958431853 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:06.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bffff700 1 -- 172.21.15.181:0/1958431853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cac20 msgr2=0x7f07c015af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:06.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bffff700 1 --2- 172.21.15.181:0/1958431853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cac20 0x7f07c015af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bffff700 1 -- 172.21.15.181:0/1958431853 >> v1:172.21.15.190:6789/0 conn(0x7f07c00cc4d0 legacy=0x7f07c00cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bffff700 1 -- 172.21.15.181:0/1958431853 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c015e4f0 con 0x7f07c00cb000 2023-10-16T09:59:06.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bf7fe700 1 -- 172.21.15.181:0/1958431853 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986659635 0 0) 0x7f07c015df40 con 0x7f07c00cc4d0 2023-10-16T09:59:06.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bffff700 1 --2- 172.21.15.181:0/1958431853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c015da00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009840 tx=0x7f07ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8032950af11e2e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:06.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.643+0000 7f07bf7fe700 1 -- 172.21.15.181:0/1958431853 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac011830 con 0x7f07c00cb000 2023-10-16T09:59:06.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07bf7fe700 1 -- 172.21.15.181:0/1958431853 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac011990 con 0x7f07c00cb000 2023-10-16T09:59:06.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07bf7fe700 1 -- 172.21.15.181:0/1958431853 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac011d20 con 0x7f07c00cb000 2023-10-16T09:59:06.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 -- 172.21.15.181:0/1958431853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 msgr2=0x7f07c015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/1958431853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c015da00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009840 tx=0x7f07ac009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 -- 172.21.15.181:0/1958431853 shutdown_connections 2023-10-16T09:59:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/1958431853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cac20 0x7f07c015af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/1958431853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c015da00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 -- 172.21.15.181:0/1958431853 >> 172.21.15.181:0/1958431853 conn(0x7f07c006aed0 msgr2=0x7f07c00b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:06.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 -- 172.21.15.181:0/1958431853 shutdown_connections 2023-10-16T09:59:06.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.644+0000 7f07c6a7e700 1 -- 172.21.15.181:0/1958431853 wait complete. 2023-10-16T09:59:06.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.646+0000 7f07c6a7e700 1 Processor -- start 2023-10-16T09:59:06.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.646+0000 7f07c6a7e700 1 -- start start 2023-10-16T09:59:06.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.647+0000 7f07c6a7e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07c00cac20 0x7f07c0151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.647+0000 7f07c6a7e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c00c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.647+0000 7f07c6a7e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cc4d0 0x7f07c00c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.647+0000 7f07c6a7e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c00c5db0 con 0x7f07c00cac20 2023-10-16T09:59:06.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.647+0000 7f07c6a7e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c00c31d0 con 0x7f07c00cb000 2023-10-16T09:59:06.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.647+0000 7f07c6a7e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c00c8440 con 0x7f07c00cc4d0 2023-10-16T09:59:06.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07c00cac20 0x7f07c0151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07c00cac20 0x7f07c0151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41050/0 (socket says 172.21.15.181:41050) 2023-10-16T09:59:06.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 -- 172.21.15.181:0/2134263086 learned_addr learned my addr 172.21.15.181:0/2134263086 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:06.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07bffff700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c00c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c501b700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cc4d0 0x7f07c00c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:06.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 -- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cc4d0 msgr2=0x7f07c00c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cc4d0 0x7f07c00c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 -- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 msgr2=0x7f07c00c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c00c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 -- 172.21.15.181:0/2134263086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0094a0 con 0x7f07c00cac20 2023-10-16T09:59:06.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07bffff700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c00c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:06.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c481a700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07c00cac20 0x7f07c0151470 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e9b0 tx=0x7f07b400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:06.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07bd7fa700 1 -- 172.21.15.181:0/2134263086 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4009710 con 0x7f07c00cac20 2023-10-16T09:59:06.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07bd7fa700 1 -- 172.21.15.181:0/2134263086 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b400ee50 con 0x7f07c00cac20 2023-10-16T09:59:06.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c00cbd80 con 0x7f07c00cac20 2023-10-16T09:59:06.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07bd7fa700 1 -- 172.21.15.181:0/2134263086 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b400ca50 con 0x7f07c00cac20 2023-10-16T09:59:06.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.648+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c00c3750 con 0x7f07c00cac20 2023-10-16T09:59:06.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.650+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c004ebb0 con 0x7f07c00cac20 2023-10-16T09:59:06.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.651+0000 7f07bd7fa700 1 -- 172.21.15.181:0/2134263086 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f07b400cbb0 con 0x7f07c00cac20 2023-10-16T09:59:06.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.651+0000 7f07bd7fa700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 0x7f07a406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:06.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.651+0000 7f07bffff700 1 -- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 msgr2=0x7f07a406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:06.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.651+0000 7f07bd7fa700 1 -- 172.21.15.181:0/2134263086 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f07b408a160 con 0x7f07c00cac20 2023-10-16T09:59:06.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.651+0000 7f07bffff700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 0x7f07a406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:06.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.655+0000 7f07bd7fa700 1 -- 172.21.15.181:0/2134263086 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f07b4054ec0 con 0x7f07c00cac20 2023-10-16T09:59:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.852+0000 7f07bffff700 1 -- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 msgr2=0x7f07a406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.852+0000 7f07bffff700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 0x7f07a406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:06.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.895+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07c00cd700 con 0x7f07c00cac20 2023-10-16T09:59:06.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.896+0000 7f07bd7fa700 1 -- 172.21.15.181:0/2134263086 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f07b400ce60 con 0x7f07c00cac20 2023-10-16T09:59:06.897 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:06.897 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5925,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/47573574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":47573574},{"type":"v1","addr":"172.21.15.181:6836","nonce":47573574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3518254514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3518254514},{"type":"v1","addr":"172.21.15.181:6837","nonce":3518254514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4292050143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4292050143},{"type":"v1","addr":"172.21.15.190:6835","nonce":4292050143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:57:01.885491+0000","modified":"2023-10-16T09:57:04.906845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5943":{"gid":5943,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2407452630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2407452630},{"type":"v1","addr":"172.21.15.190:6834","nonce":2407452630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T09:59:06.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 msgr2=0x7f07a406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:06.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 0x7f07a406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07c00cac20 msgr2=0x7f07c0151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:06.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07c00cac20 0x7f07c0151470 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f07b400e9b0 tx=0x7f07b400ecc0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 shutdown_connections 2023-10-16T09:59:06.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f07a406b020 0x7f07a406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07c00cc4d0 0x7f07c00c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07c00cac20 0x7f07c0151470 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 --2- 172.21.15.181:0/2134263086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07c00cb000 0x7f07c00c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:06.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 >> 172.21.15.181:0/2134263086 conn(0x7f07c006aed0 msgr2=0x7f07c00c8780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:06.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 shutdown_connections 2023-10-16T09:59:06.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:06.898+0000 7f07c6a7e700 1 -- 172.21.15.181:0/2134263086 wait complete. 2023-10-16T09:59:06.902 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 25 2023-10-16T09:59:06.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:06.927 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:06.926+0000 7f30d3d09200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:07.004 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:07.003+0000 7f30d3d09200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:59:07.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.038+0000 7fe62f9fb700 1 Processor -- start 2023-10-16T09:59:07.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.038+0000 7fe62f9fb700 1 -- start start 2023-10-16T09:59:07.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62f9fb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe62815b990 0x7fe6280ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62f9fb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6280cd2d0 0x7fe6280d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62f9fb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6280cc510 con 0x7fe6280cfc60 2023-10-16T09:59:07.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62f9fb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6280d24f0 con 0x7fe6280cd2d0 2023-10-16T09:59:07.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62f9fb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6280cc7f0 con 0x7fe62815b990 2023-10-16T09:59:07.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6280cd2d0 0x7fe6280d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6280cd2d0 0x7fe6280d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41062/0 (socket says 172.21.15.181:41062) 2023-10-16T09:59:07.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 -- 172.21.15.181:0/961314093 learned_addr learned my addr 172.21.15.181:0/961314093 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:07.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62d797700 1 --2- 172.21.15.181:0/961314093 >> v2:172.21.15.190:3300/0 conn(0x7fe62815b990 0x7fe6280ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 -- 172.21.15.181:0/961314093 >> v2:172.21.15.190:3300/0 conn(0x7fe62815b990 msgr2=0x7fe6280ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:07.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 --2- 172.21.15.181:0/961314093 >> v2:172.21.15.190:3300/0 conn(0x7fe62815b990 0x7fe6280ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 -- 172.21.15.181:0/961314093 >> v1:172.21.15.190:6789/0 conn(0x7fe6280cfc60 legacy=0x7fe6280ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 -- 172.21.15.181:0/961314093 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280ccad0 con 0x7fe6280cd2d0 2023-10-16T09:59:07.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe62cf96700 1 --2- 172.21.15.181:0/961314093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6280d1fb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe628073000 tx=0x7fe618009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31211600a1d35356 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:07.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.039+0000 7fe61ffff700 1 -- 172.21.15.181:0/961314093 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618011830 con 0x7fe6280cd2d0 2023-10-16T09:59:07.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe61ffff700 1 -- 172.21.15.181:0/961314093 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe618011990 con 0x7fe6280cd2d0 2023-10-16T09:59:07.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe61ffff700 1 -- 172.21.15.181:0/961314093 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618011d20 con 0x7fe6280cd2d0 2023-10-16T09:59:07.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 -- 172.21.15.181:0/961314093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 msgr2=0x7fe6280d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/961314093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6280d1fb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe628073000 tx=0x7fe618009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 -- 172.21.15.181:0/961314093 shutdown_connections 2023-10-16T09:59:07.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/961314093 >> v2:172.21.15.190:3300/0 conn(0x7fe62815b990 0x7fe6280ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/961314093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6280d1fb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 -- 172.21.15.181:0/961314093 >> 172.21.15.181:0/961314093 conn(0x7fe62809a9a0 msgr2=0x7fe6280d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:07.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 -- 172.21.15.181:0/961314093 shutdown_connections 2023-10-16T09:59:07.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.040+0000 7fe62f9fb700 1 -- 172.21.15.181:0/961314093 wait complete. 2023-10-16T09:59:07.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.042+0000 7fe62f9fb700 1 Processor -- start 2023-10-16T09:59:07.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.042+0000 7fe62f9fb700 1 -- start start 2023-10-16T09:59:07.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62f9fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6281f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62f9fb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6280cfc60 0x7fe628159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62f9fb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe62815b990 0x7fe6280b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62f9fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62815a800 con 0x7fe6280cd2d0 2023-10-16T09:59:07.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62f9fb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281f7a70 con 0x7fe62815b990 2023-10-16T09:59:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62f9fb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62815e130 con 0x7fe6280cfc60 2023-10-16T09:59:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62d797700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6281f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62d797700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6281f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41066/0 (socket says 172.21.15.181:41066) 2023-10-16T09:59:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.043+0000 7fe62d797700 1 -- 172.21.15.181:0/1465988060 learned_addr learned my addr 172.21.15.181:0/1465988060 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62df98700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe62815b990 0x7fe6280b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62cf96700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6280cfc60 0x7fe628159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62d797700 1 -- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6280cfc60 msgr2=0x7fe628159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62d797700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6280cfc60 0x7fe628159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62d797700 1 -- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe62815b990 msgr2=0x7fe6280b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62d797700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe62815b990 0x7fe6280b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62d797700 1 -- 172.21.15.181:0/1465988060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe618009550 con 0x7fe6280cd2d0 2023-10-16T09:59:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62df98700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe62815b990 0x7fe6280b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62d797700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6281f7530 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe62400e900 tx=0x7fe62400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe61dffb700 1 -- 172.21.15.181:0/1465988060 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624013680 con 0x7fe6280cd2d0 2023-10-16T09:59:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe61dffb700 1 -- 172.21.15.181:0/1465988060 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6240137e0 con 0x7fe6280cd2d0 2023-10-16T09:59:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62815e410 con 0x7fe6280cd2d0 2023-10-16T09:59:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe61dffb700 1 -- 172.21.15.181:0/1465988060 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624013b50 con 0x7fe6280cd2d0 2023-10-16T09:59:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.044+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62815e9d0 con 0x7fe6280cd2d0 2023-10-16T09:59:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.046+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe628067d20 con 0x7fe6280cd2d0 2023-10-16T09:59:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.046+0000 7fe61dffb700 1 -- 172.21.15.181:0/1465988060 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe624013cb0 con 0x7fe6280cd2d0 2023-10-16T09:59:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.046+0000 7fe61dffb700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 0x7fe60c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.046+0000 7fe62cf96700 1 -- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 msgr2=0x7fe60c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.046+0000 7fe61dffb700 1 -- 172.21.15.181:0/1465988060 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7fe624089d90 con 0x7fe6280cd2d0 2023-10-16T09:59:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.046+0000 7fe62cf96700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 0x7fe60c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.051+0000 7fe61dffb700 1 -- 172.21.15.181:0/1465988060 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe624054a10 con 0x7fe6280cd2d0 2023-10-16T09:59:07.069 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:07.068+0000 7efe1401b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:07.147 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:07.145+0000 7f30d3d09200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:07.150 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:07.149+0000 7efe1401b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:59:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.247+0000 7fe62cf96700 1 -- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 msgr2=0x7fe60c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:07.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.247+0000 7fe62cf96700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 0x7fe60c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:07.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.251+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6280cfa40 con 0x7fe6280cd2d0 2023-10-16T09:59:07.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.253+0000 7fe61dffb700 1 -- 172.21.15.181:0/1465988060 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17749 (secure 0 0 0) 0x7fe624021020 con 0x7fe6280cd2d0 2023-10-16T09:59:07.253 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:07.254 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":62,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:58:17.034994+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:57:00.297618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T09:57:00.838546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2507249913":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2125399270":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2970752475":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/3339446951":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2864335478":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6833/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6832/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/534300178":"2023-10-17T09:57:15.720457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:07.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 msgr2=0x7fe60c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:07.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 0x7fe60c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 msgr2=0x7fe6281f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6281f7530 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe62400e900 tx=0x7fe62400ec10 comp rx=0 tx=0).stop 2023-10-16T09:59:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 shutdown_connections 2023-10-16T09:59:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fe60c06b020 0x7fe60c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe62815b990 0x7fe6280b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6280cd2d0 0x7fe6281f7530 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 --2- 172.21.15.181:0/1465988060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6280cfc60 0x7fe628159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 >> 172.21.15.181:0/1465988060 conn(0x7fe62809a9a0 msgr2=0x7fe6280b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 shutdown_connections 2023-10-16T09:59:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.255+0000 7fe62f9fb700 1 -- 172.21.15.181:0/1465988060 wait complete. 2023-10-16T09:59:07.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:57:00.297618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:57:00.838546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:07.270 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:07.303 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:07.302+0000 7efe1401b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:07.371 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:07.370+0000 7f30d3d09200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:59:07.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.396+0000 7fbaceaee700 1 Processor -- start 2023-10-16T09:59:07.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.396+0000 7fbaceaee700 1 -- start start 2023-10-16T09:59:07.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.396+0000 7fbaceaee700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbac815da70 0x7fbac80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.396+0000 7fbaceaee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac80cef00 con 0x7fbac80ce6a0 2023-10-16T09:59:07.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.396+0000 7fbaceaee700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac80cec20 con 0x7fbac806afb0 2023-10-16T09:59:07.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.396+0000 7fbaceaee700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac8071c80 con 0x7fbac815da70 2023-10-16T09:59:07.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbac80ce6a0 0x7fbac806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52170/0 (socket says 172.21.15.181:52170) 2023-10-16T09:59:07.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac7fff700 1 -- 172.21.15.181:0/3419438654 learned_addr learned my addr 172.21.15.181:0/3419438654 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:07.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbacc88a700 1 --2- 172.21.15.181:0/3419438654 >> v2:172.21.15.190:3300/0 conn(0x7fbac815da70 0x7fbac80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac77fe700 1 -- 172.21.15.181:0/3419438654 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161934569 0 0) 0x7fbac80cef00 con 0x7fbac80ce6a0 2023-10-16T09:59:07.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac77fe700 1 -- 172.21.15.181:0/3419438654 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab80035b0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac77fe700 1 -- 172.21.15.181:0/3419438654 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449382211 0 0) 0x7fbac80cec20 con 0x7fbac806afb0 2023-10-16T09:59:07.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac77fe700 1 -- 172.21.15.181:0/3419438654 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac80cef00 con 0x7fbac806afb0 2023-10-16T09:59:07.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbacc88a700 1 -- 172.21.15.181:0/3419438654 >> v1:172.21.15.190:6789/0 conn(0x7fbac806afb0 legacy=0x7fbac81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbacc88a700 1 -- 172.21.15.181:0/3419438654 >> v1:172.21.15.181:6789/0 conn(0x7fbac80ce6a0 legacy=0x7fbac806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbacc88a700 1 -- 172.21.15.181:0/3419438654 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac8071f60 con 0x7fbac815da70 2023-10-16T09:59:07.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbacc88a700 1 --2- 172.21.15.181:0/3419438654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 0x7fbac80ce160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009840 tx=0x7fbabc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc2d561e183368f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:07.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac77fe700 1 -- 172.21.15.181:0/3419438654 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc011830 con 0x7fbac815da70 2023-10-16T09:59:07.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.397+0000 7fbac77fe700 1 -- 172.21.15.181:0/3419438654 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc011990 con 0x7fbac815da70 2023-10-16T09:59:07.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbac77fe700 1 -- 172.21.15.181:0/3419438654 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc011d20 con 0x7fbac815da70 2023-10-16T09:59:07.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbaceaee700 1 -- 172.21.15.181:0/3419438654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 msgr2=0x7fbac80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3419438654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 0x7fbac80ce160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009840 tx=0x7fbabc009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:07.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbaceaee700 1 -- 172.21.15.181:0/3419438654 shutdown_connections 2023-10-16T09:59:07.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3419438654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 0x7fbac80ce160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbaceaee700 1 -- 172.21.15.181:0/3419438654 >> 172.21.15.181:0/3419438654 conn(0x7fbac80cd010 msgr2=0x7fbac80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:07.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbaceaee700 1 -- 172.21.15.181:0/3419438654 shutdown_connections 2023-10-16T09:59:07.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.398+0000 7fbaceaee700 1 -- 172.21.15.181:0/3419438654 wait complete. 2023-10-16T09:59:07.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.400+0000 7fbaceaee700 1 Processor -- start 2023-10-16T09:59:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.400+0000 7fbaceaee700 1 -- start start 2023-10-16T09:59:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbaceaee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbac806afb0 0x7fbac80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbaceaee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac80ce6a0 0x7fbac8157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbaceaee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 0x7fbac80b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbaceaee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80b36c0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbaceaee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c85b0 con 0x7fbac815da70 2023-10-16T09:59:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbaceaee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c8f00 con 0x7fbac806afb0 2023-10-16T09:59:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac80ce6a0 0x7fbac8157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac80ce6a0 0x7fbac8157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41098/0 (socket says 172.21.15.181:41098) 2023-10-16T09:59:07.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 -- 172.21.15.181:0/3993029927 learned_addr learned my addr 172.21.15.181:0/3993029927 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbacc88a700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbac806afb0 0x7fbac80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbacd08b700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 0x7fbac80b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 -- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbac806afb0 msgr2=0x7fbac80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbac806afb0 0x7fbac80c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 -- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 msgr2=0x7fbac80b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 0x7fbac80b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 -- 172.21.15.181:0/3993029927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0094a0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.401+0000 7fbac7fff700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac80ce6a0 0x7fbac8157800 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c700 tx=0x7fbab400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.402+0000 7fbac57fa700 1 -- 172.21.15.181:0/3993029927 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4005d70 con 0x7fbac80ce6a0 2023-10-16T09:59:07.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.402+0000 7fbac57fa700 1 -- 172.21.15.181:0/3993029927 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab40070d0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.402+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac80c91e0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.402+0000 7fbac57fa700 1 -- 172.21.15.181:0/3993029927 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4007460 con 0x7fbac80ce6a0 2023-10-16T09:59:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.402+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac80b3350 con 0x7fbac80ce6a0 2023-10-16T09:59:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.403+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac80c9700 con 0x7fbac80ce6a0 2023-10-16T09:59:07.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.403+0000 7fbac57fa700 1 -- 172.21.15.181:0/3993029927 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fbab40075c0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.404+0000 7fbac57fa700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 0x7fbaac06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.404+0000 7fbacc88a700 1 -- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 msgr2=0x7fbaac06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.404+0000 7fbac57fa700 1 -- 172.21.15.181:0/3993029927 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7fbab4088fb0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.404+0000 7fbacc88a700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 0x7fbaac06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:07.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.408+0000 7fbac57fa700 1 -- 172.21.15.181:0/3993029927 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbac80c9700 con 0x7fbac80ce6a0 2023-10-16T09:59:07.445 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:07.444+0000 7f30d3d09200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:59:07.557 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:07.557+0000 7efe1401b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:59:07.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.605+0000 7fbacc88a700 1 -- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 msgr2=0x7fbaac06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:07.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.605+0000 7fbacc88a700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 0x7fbaac06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:07.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.626+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbac80b31b0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.626+0000 7fbac57fa700 1 -- 172.21.15.181:0/3993029927 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbac80b31b0 con 0x7fbac80ce6a0 2023-10-16T09:59:07.628 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:07.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.628+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 msgr2=0x7fbaac06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:07.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.628+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 0x7fbaac06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.628+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac80ce6a0 msgr2=0x7fbac8157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.628+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac80ce6a0 0x7fbac8157800 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c700 tx=0x7fbab400ca10 comp rx=0 tx=0).stop 2023-10-16T09:59:07.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.628+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 shutdown_connections 2023-10-16T09:59:07.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.628+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fbaac06b050 0x7fbaac06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.629+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbac806afb0 0x7fbac80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.629+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbac815da70 0x7fbac80b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.629+0000 7fbaceaee700 1 --2- 172.21.15.181:0/3993029927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac80ce6a0 0x7fbac8157800 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.629+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 >> 172.21.15.181:0/3993029927 conn(0x7fbac80cd010 msgr2=0x7fbac80cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:07.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.629+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 shutdown_connections 2023-10-16T09:59:07.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.629+0000 7fbaceaee700 1 -- 172.21.15.181:0/3993029927 wait complete. 2023-10-16T09:59:07.633 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:07.633+0000 7efe1401b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:59:07.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:59:07.710 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:07.709+0000 7f30d3d09200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:59:07.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.813+0000 7f732f590700 1 Processor -- start 2023-10-16T09:59:07.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.813+0000 7f732f590700 1 -- start start 2023-10-16T09:59:07.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732f590700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280ce2d0 0x7f732815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732f590700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732815ef80 con 0x7f73280d1500 2023-10-16T09:59:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732f590700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73280cf580 con 0x7f73280d01d0 2023-10-16T09:59:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732f590700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73280d4290 con 0x7f73280ce2d0 2023-10-16T09:59:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732db2d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f73280d1500 0x7f73280d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52188/0 (socket says 172.21.15.181:52188) 2023-10-16T09:59:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732db2d700 1 -- 172.21.15.181:0/3983844454 learned_addr learned my addr 172.21.15.181:0/3983844454 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732cb2b700 1 --2- 172.21.15.181:0/3983844454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280ce2d0 0x7f732815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f731ffff700 1 -- 172.21.15.181:0/3983844454 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547853333 0 0) 0x7f7320001490 con 0x7f73280d1500 2023-10-16T09:59:07.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f731ffff700 1 -- 172.21.15.181:0/3983844454 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73140035b0 con 0x7f73280d1500 2023-10-16T09:59:07.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f731ffff700 1 -- 172.21.15.181:0/3983844454 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647333097 0 0) 0x7f73280cf580 con 0x7f73280d01d0 2023-10-16T09:59:07.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f731ffff700 1 -- 172.21.15.181:0/3983844454 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7320001490 con 0x7f73280d01d0 2023-10-16T09:59:07.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732cb2b700 1 -- 172.21.15.181:0/3983844454 >> v1:172.21.15.190:6789/0 conn(0x7f73280d01d0 legacy=0x7f73280d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732cb2b700 1 -- 172.21.15.181:0/3983844454 >> v1:172.21.15.181:6789/0 conn(0x7f73280d1500 legacy=0x7f73280d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.814+0000 7f732cb2b700 1 -- 172.21.15.181:0/3983844454 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280ce970 con 0x7f73280ce2d0 2023-10-16T09:59:07.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732cb2b700 1 --2- 172.21.15.181:0/3983844454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280ce2d0 0x7f732815ea40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f73180098a0 tx=0x7f7318009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0214d977458bcbe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:07.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f731ffff700 1 -- 172.21.15.181:0/3983844454 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318011830 con 0x7f73280ce2d0 2023-10-16T09:59:07.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f731ffff700 1 -- 172.21.15.181:0/3983844454 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318011990 con 0x7f73280ce2d0 2023-10-16T09:59:07.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f731ffff700 1 -- 172.21.15.181:0/3983844454 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318011d20 con 0x7f73280ce2d0 2023-10-16T09:59:07.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732f590700 1 -- 172.21.15.181:0/3983844454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280ce2d0 msgr2=0x7f732815ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732f590700 1 --2- 172.21.15.181:0/3983844454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280ce2d0 0x7f732815ea40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f73180098a0 tx=0x7f7318009bb0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732f590700 1 -- 172.21.15.181:0/3983844454 shutdown_connections 2023-10-16T09:59:07.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732f590700 1 --2- 172.21.15.181:0/3983844454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280ce2d0 0x7f732815ea40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732f590700 1 -- 172.21.15.181:0/3983844454 >> 172.21.15.181:0/3983844454 conn(0x7f732809cb60 msgr2=0x7f73280d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:07.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732f590700 1 -- 172.21.15.181:0/3983844454 shutdown_connections 2023-10-16T09:59:07.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.815+0000 7f732f590700 1 -- 172.21.15.181:0/3983844454 wait complete. 2023-10-16T09:59:07.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 Processor -- start 2023-10-16T09:59:07.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 -- start start 2023-10-16T09:59:07.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73280ce2d0 0x7f7328152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280d01d0 0x7f732806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73280d1500 0x7f732806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b85d0 con 0x7f73280ce2d0 2023-10-16T09:59:07.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b7590 con 0x7f73280d1500 2023-10-16T09:59:07.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732f590700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b64f0 con 0x7f73280d01d0 2023-10-16T09:59:07.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.817+0000 7f732d32c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73280ce2d0 0x7f7328152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73280ce2d0 0x7f7328152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41112/0 (socket says 172.21.15.181:41112) 2023-10-16T09:59:07.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 -- 172.21.15.181:0/4092000881 learned_addr learned my addr 172.21.15.181:0/4092000881 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:07.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732db2d700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73280d1500 0x7f732806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732cb2b700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280d01d0 0x7f732806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:07.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 -- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280d01d0 msgr2=0x7f732806ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280d01d0 0x7f732806ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 -- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73280d1500 msgr2=0x7f732806d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:07.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73280d1500 0x7f732806d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:07.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 -- 172.21.15.181:0/4092000881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180094a0 con 0x7f73280ce2d0 2023-10-16T09:59:07.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732db2d700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73280d1500 0x7f732806d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:07.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732d32c700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73280ce2d0 0x7f7328152c50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f732400c520 tx=0x7f732400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:07.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f731dffb700 1 -- 172.21.15.181:0/4092000881 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73240052d0 con 0x7f73280ce2d0 2023-10-16T09:59:07.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73280b67d0 con 0x7f73280ce2d0 2023-10-16T09:59:07.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f731dffb700 1 -- 172.21.15.181:0/4092000881 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7324005430 con 0x7f73280ce2d0 2023-10-16T09:59:07.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f731dffb700 1 -- 172.21.15.181:0/4092000881 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73240038c0 con 0x7f73280ce2d0 2023-10-16T09:59:07.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.818+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73280b6d90 con 0x7f73280ce2d0 2023-10-16T09:59:07.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.820+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732804ebb0 con 0x7f73280ce2d0 2023-10-16T09:59:07.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.820+0000 7f731dffb700 1 -- 172.21.15.181:0/4092000881 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f7324003a20 con 0x7f73280ce2d0 2023-10-16T09:59:07.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.820+0000 7f731dffb700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 0x7f730c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:07.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.820+0000 7f732cb2b700 1 -- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 msgr2=0x7f730c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.820+0000 7f731dffb700 1 -- 172.21.15.181:0/4092000881 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f7324088f40 con 0x7f73280ce2d0 2023-10-16T09:59:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.820+0000 7f732cb2b700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 0x7f730c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:07.829+0000 7f731dffb700 1 -- 172.21.15.181:0/4092000881 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7324053b40 con 0x7f73280ce2d0 2023-10-16T09:59:07.917 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:07.916+0000 7efe1401b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:59:07.928 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:07.927+0000 7f30d3d09200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:59:08.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.021+0000 7f732cb2b700 1 -- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 msgr2=0x7f730c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:08.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.021+0000 7f732cb2b700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 0x7f730c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:08.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.025+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f732815ccf0 con 0x7f73280ce2d0 2023-10-16T09:59:08.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.025+0000 7f731dffb700 1 -- 172.21.15.181:0/4092000881 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7324057160 con 0x7f73280ce2d0 2023-10-16T09:59:08.026 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:59:08.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.027+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 msgr2=0x7f730c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:08.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.027+0000 7f732f590700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 0x7f730c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.027+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73280ce2d0 msgr2=0x7f7328152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.027+0000 7f732f590700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73280ce2d0 0x7f7328152c50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f732400c520 tx=0x7f732400c830 comp rx=0 tx=0).stop 2023-10-16T09:59:08.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 shutdown_connections 2023-10-16T09:59:08.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f730c06b020 0x7f730c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73280d1500 0x7f732806d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73280ce2d0 0x7f7328152c50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 --2- 172.21.15.181:0/4092000881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73280d01d0 0x7f732806ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 >> 172.21.15.181:0/4092000881 conn(0x7f732809cb60 msgr2=0x7f732815f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:08.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 shutdown_connections 2023-10-16T09:59:08.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.028+0000 7f732f590700 1 -- 172.21.15.181:0/4092000881 wait complete. 2023-10-16T09:59:08.041 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:59:08.074 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:08.073+0000 7f30d3d09200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:59:08.150 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:08.149+0000 7efe1401b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:59:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74e71b9700 1 Processor -- start 2023-10-16T09:59:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74e71b9700 1 -- start start 2023-10-16T09:59:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74e71b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e015dbd0 0x7f74e00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74e71b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74e00d0610 0x7f74e00cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74e71b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e015dfb0 con 0x7f74e00ce2c0 2023-10-16T09:59:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74e71b9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e00ce740 con 0x7f74e00d0610 2023-10-16T09:59:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74e71b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00cd3a0 con 0x7f74e015dbd0 2023-10-16T09:59:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.167+0000 7f74dffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74e00d0610 0x7f74e00cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74dffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74e00d0610 0x7f74e00cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41126/0 (socket says 172.21.15.181:41126) 2023-10-16T09:59:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74dffff700 1 -- 172.21.15.181:0/1705404279 learned_addr learned my addr 172.21.15.181:0/1705404279 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74e4f55700 1 --2- 172.21.15.181:0/1705404279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e015dbd0 0x7f74e00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74dffff700 1 -- 172.21.15.181:0/1705404279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e015dbd0 msgr2=0x7f74e00d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:08.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74dffff700 1 --2- 172.21.15.181:0/1705404279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e015dbd0 0x7f74e00d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74dffff700 1 -- 172.21.15.181:0/1705404279 >> v1:172.21.15.190:6789/0 conn(0x7f74e00ce2c0 legacy=0x7f74e0161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74dffff700 1 -- 172.21.15.181:0/1705404279 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00cd830 con 0x7f74e00d0610 2023-10-16T09:59:08.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74dffff700 1 --2- 172.21.15.181:0/1705404279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e00d0610 0x7f74e00cdd80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0098f0 tx=0x7f74cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4378f681e71cd1bd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:08.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74df7fe700 1 -- 172.21.15.181:0/1705404279 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc011830 con 0x7f74e00d0610 2023-10-16T09:59:08.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74df7fe700 1 -- 172.21.15.181:0/1705404279 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc011990 con 0x7f74e00d0610 2023-10-16T09:59:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.168+0000 7f74df7fe700 1 -- 172.21.15.181:0/1705404279 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc011d20 con 0x7f74e00d0610 2023-10-16T09:59:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 -- 172.21.15.181:0/1705404279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e00d0610 msgr2=0x7f74e00cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 --2- 172.21.15.181:0/1705404279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e00d0610 0x7f74e00cdd80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0098f0 tx=0x7f74cc009c00 comp rx=0 tx=0).stop 2023-10-16T09:59:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 -- 172.21.15.181:0/1705404279 shutdown_connections 2023-10-16T09:59:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 --2- 172.21.15.181:0/1705404279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e015dbd0 0x7f74e00d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 --2- 172.21.15.181:0/1705404279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e00d0610 0x7f74e00cdd80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 -- 172.21.15.181:0/1705404279 >> 172.21.15.181:0/1705404279 conn(0x7f74e009a930 msgr2=0x7f74e00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 -- 172.21.15.181:0/1705404279 shutdown_connections 2023-10-16T09:59:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.169+0000 7f74e71b9700 1 -- 172.21.15.181:0/1705404279 wait complete. 2023-10-16T09:59:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.170+0000 7f74e71b9700 1 Processor -- start 2023-10-16T09:59:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.170+0000 7f74e71b9700 1 -- start start 2023-10-16T09:59:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e71b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e00ce2c0 0x7f74e0158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e71b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e00d0610 0x7f74e00b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e71b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e015dbd0 0x7f74e006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e71b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00b59c0 con 0x7f74e015dbd0 2023-10-16T09:59:08.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e71b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00b6320 con 0x7f74e00d0610 2023-10-16T09:59:08.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e71b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0160950 con 0x7f74e00ce2c0 2023-10-16T09:59:08.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e015dbd0 0x7f74e006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e015dbd0 0x7f74e006ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41132/0 (socket says 172.21.15.181:41132) 2023-10-16T09:59:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 -- 172.21.15.181:0/2570793160 learned_addr learned my addr 172.21.15.181:0/2570793160 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74dffff700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e00d0610 0x7f74e00b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e4f55700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e00ce2c0 0x7f74e0158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 -- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e00ce2c0 msgr2=0x7f74e0158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e00ce2c0 0x7f74e0158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 -- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e00d0610 msgr2=0x7f74e00b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e00d0610 0x7f74e00b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 -- 172.21.15.181:0/2570793160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc009550 con 0x7f74e015dbd0 2023-10-16T09:59:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.171+0000 7f74e5756700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e015dbd0 0x7f74e006ed60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f74d800cc50 tx=0x7f74d800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:08.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.172+0000 7f74dd7fa700 1 -- 172.21.15.181:0/2570793160 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8003c00 con 0x7f74e015dbd0 2023-10-16T09:59:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.172+0000 7f74dd7fa700 1 -- 172.21.15.181:0/2570793160 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d8003d60 con 0x7f74e015dbd0 2023-10-16T09:59:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.172+0000 7f74dd7fa700 1 -- 172.21.15.181:0/2570793160 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800aab0 con 0x7f74e015dbd0 2023-10-16T09:59:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.172+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e00b5020 con 0x7f74e015dbd0 2023-10-16T09:59:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.172+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e00b55e0 con 0x7f74e015dbd0 2023-10-16T09:59:08.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.173+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e00b5b00 con 0x7f74e015dbd0 2023-10-16T09:59:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.173+0000 7f74dd7fa700 1 -- 172.21.15.181:0/2570793160 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f74d800ac10 con 0x7f74e015dbd0 2023-10-16T09:59:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.174+0000 7f74dd7fa700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 0x7f74c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.174+0000 7f74e4f55700 1 -- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 msgr2=0x7f74c406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.174+0000 7f74dd7fa700 1 -- 172.21.15.181:0/2570793160 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6507+0+0 (secure 0 0 0) 0x7f74d808a110 con 0x7f74e015dbd0 2023-10-16T09:59:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.174+0000 7f74e4f55700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 0x7f74c406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:08.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.182+0000 7f74dd7fa700 1 -- 172.21.15.181:0/2570793160 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f74d8054e70 con 0x7f74e015dbd0 2023-10-16T09:59:08.187 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:08.186+0000 7f30d3d09200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:59:08.319 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:08.319+0000 7efe1401b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:59:08.346 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:08.345+0000 7f30d3d09200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:59:08.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.375+0000 7f74e4f55700 1 -- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 msgr2=0x7f74c406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:08.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.375+0000 7f74e4f55700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 0x7f74c406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:08.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.391+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74e00b5b00 con 0x7f74e015dbd0 2023-10-16T09:59:08.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.391+0000 7f74dd7fa700 1 -- 172.21.15.181:0/2570793160 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74e00b5b00 con 0x7f74e015dbd0 2023-10-16T09:59:08.392 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:59:08.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 msgr2=0x7f74c406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 0x7f74c406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e015dbd0 msgr2=0x7f74e006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e015dbd0 0x7f74e006ed60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f74d800cc50 tx=0x7f74d800cf60 comp rx=0 tx=0).stop 2023-10-16T09:59:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 shutdown_connections 2023-10-16T09:59:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f74c406b020 0x7f74c406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74e00ce2c0 0x7f74e0158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74e015dbd0 0x7f74e006ed60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 --2- 172.21.15.181:0/2570793160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74e00d0610 0x7f74e00b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 >> 172.21.15.181:0/2570793160 conn(0x7f74e009a930 msgr2=0x7f74e006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:08.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 shutdown_connections 2023-10-16T09:59:08.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.395+0000 7f74e71b9700 1 -- 172.21.15.181:0/2570793160 wait complete. 2023-10-16T09:59:08.408 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:08.440 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:08.439+0000 7efe1401b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:59:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.530+0000 7f2cd91e3700 1 Processor -- start 2023-10-16T09:59:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.530+0000 7f2cd91e3700 1 -- start start 2023-10-16T09:59:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd91e3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2cd415e6d0 0x7f2cd40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd91e3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2cd40cb0c0 0x7f2cd415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd91e3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd406ed00 con 0x7f2cd415eab0 2023-10-16T09:59:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd91e3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40d02d0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd91e3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd415ad40 con 0x7f2cd40cb0c0 2023-10-16T09:59:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2cd415e6d0 0x7f2cd40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2cd415eab0 0x7f2cd415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52216/0 (socket says 172.21.15.181:52216) 2023-10-16T09:59:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2cd415e6d0 0x7f2cd40d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41138/0 (socket says 172.21.15.181:41138) 2023-10-16T09:59:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.531+0000 7f2cd259c700 1 -- 172.21.15.181:0/221602627 learned_addr learned my addr 172.21.15.181:0/221602627 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd359e700 1 --2- 172.21.15.181:0/221602627 >> v2:172.21.15.190:3300/0 conn(0x7f2cd40cb0c0 0x7f2cd415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd2d9d700 1 -- 172.21.15.181:0/221602627 >> v2:172.21.15.190:3300/0 conn(0x7f2cd40cb0c0 msgr2=0x7f2cd415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd2d9d700 1 --2- 172.21.15.181:0/221602627 >> v2:172.21.15.190:3300/0 conn(0x7f2cd40cb0c0 0x7f2cd415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd2d9d700 1 -- 172.21.15.181:0/221602627 >> v1:172.21.15.181:6789/0 conn(0x7f2cd415eab0 legacy=0x7f2cd415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd2d9d700 1 -- 172.21.15.181:0/221602627 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd406e0f0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd2d9d700 1 --2- 172.21.15.181:0/221602627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40d0fc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4071650 tx=0x7f2cc4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94dc9935ea690ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd1d9b700 1 -- 172.21.15.181:0/221602627 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4005e30 con 0x7f2cd415e6d0 2023-10-16T09:59:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd1d9b700 1 -- 172.21.15.181:0/221602627 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc4006030 con 0x7f2cd415e6d0 2023-10-16T09:59:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.532+0000 7f2cd1d9b700 1 -- 172.21.15.181:0/221602627 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40063c0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 -- 172.21.15.181:0/221602627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 msgr2=0x7f2cd40d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/221602627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40d0fc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4071650 tx=0x7f2cc4009b00 comp rx=0 tx=0).stop 2023-10-16T09:59:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 -- 172.21.15.181:0/221602627 shutdown_connections 2023-10-16T09:59:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/221602627 >> v2:172.21.15.190:3300/0 conn(0x7f2cd40cb0c0 0x7f2cd415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/221602627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40d0fc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 -- 172.21.15.181:0/221602627 >> 172.21.15.181:0/221602627 conn(0x7f2cd40d04a0 msgr2=0x7f2cd40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 -- 172.21.15.181:0/221602627 shutdown_connections 2023-10-16T09:59:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.533+0000 7f2cd91e3700 1 -- 172.21.15.181:0/221602627 wait complete. 2023-10-16T09:59:08.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.534+0000 7f2cd91e3700 1 Processor -- start 2023-10-16T09:59:08.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.535+0000 7f2cd91e3700 1 -- start start 2023-10-16T09:59:08.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.535+0000 7f2cd91e3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cd40cb0c0 0x7f2cd4159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.535+0000 7f2cd91e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.535+0000 7f2cd91e3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cd415eab0 0x7f2cd40c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.535+0000 7f2cd91e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd415a390 con 0x7f2cd415e6d0 2023-10-16T09:59:08.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.535+0000 7f2cd91e3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd41589a0 con 0x7f2cd40cb0c0 2023-10-16T09:59:08.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.535+0000 7f2cd91e3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4158c80 con 0x7f2cd415eab0 2023-10-16T09:59:08.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41144/0 (socket says 172.21.15.181:41144) 2023-10-16T09:59:08.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 -- 172.21.15.181:0/1327135127 learned_addr learned my addr 172.21.15.181:0/1327135127 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:08.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd2d9d700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cd40cb0c0 0x7f2cd4159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd359e700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cd415eab0 0x7f2cd40c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 -- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cd415eab0 msgr2=0x7f2cd40c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cd415eab0 0x7f2cd40c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 -- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cd40cb0c0 msgr2=0x7f2cd4159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cd40cb0c0 0x7f2cd4159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 -- 172.21.15.181:0/1327135127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4009430 con 0x7f2cd415e6d0 2023-10-16T09:59:08.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd259c700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40c5130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c490 tx=0x7f2cc800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:08.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cbb7fe700 1 -- 172.21.15.181:0/1327135127 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800cea0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd4158f60 con 0x7f2cd415e6d0 2023-10-16T09:59:08.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cbb7fe700 1 -- 172.21.15.181:0/1327135127 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8005d70 con 0x7f2cd415e6d0 2023-10-16T09:59:08.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cbb7fe700 1 -- 172.21.15.181:0/1327135127 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80072f0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.536+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd4159520 con 0x7f2cd415e6d0 2023-10-16T09:59:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.537+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd40afee0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.538+0000 7f2cbb7fe700 1 -- 172.21.15.181:0/1327135127 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f2cc8007450 con 0x7f2cd415e6d0 2023-10-16T09:59:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.538+0000 7f2cbb7fe700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 0x7f2cc006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.538+0000 7f2cd2d9d700 1 -- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 msgr2=0x7f2cc006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:08.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.538+0000 7f2cbb7fe700 1 -- 172.21.15.181:0/1327135127 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6563+0+0 (secure 0 0 0) 0x7f2cc8088eb0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.538+0000 7f2cd2d9d700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 0x7f2cc006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:08.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.543+0000 7f2cbb7fe700 1 -- 172.21.15.181:0/1327135127 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2cd40afee0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.606 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:08.606+0000 7efe1401b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:59:08.611 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:08.610+0000 7f30d3d09200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:08.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.739+0000 7f2cd2d9d700 1 -- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 msgr2=0x7f2cc006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:08.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.739+0000 7f2cd2d9d700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 0x7f2cc006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:08.771 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:08.770+0000 7f30d3d09200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:59:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.786+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cd40afee0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.787+0000 7f2cbb7fe700 1 -- 172.21.15.181:0/1327135127 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2cd40afee0 con 0x7f2cd415e6d0 2023-10-16T09:59:08.788 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:08.788 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5925,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/47573574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":47573574},{"type":"v1","addr":"172.21.15.181:6836","nonce":47573574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3518254514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3518254514},{"type":"v1","addr":"172.21.15.181:6837","nonce":3518254514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4292050143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4292050143},{"type":"v1","addr":"172.21.15.190:6835","nonce":4292050143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:57:01.885491+0000","modified":"2023-10-16T09:57:04.906845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5943":{"gid":5943,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2407452630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2407452630},{"type":"v1","addr":"172.21.15.190:6834","nonce":2407452630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T09:59:08.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 msgr2=0x7f2cc006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:08.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 0x7f2cc006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 msgr2=0x7f2cd40c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40c5130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c490 tx=0x7f2cc800c7a0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 shutdown_connections 2023-10-16T09:59:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f2cc006b020 0x7f2cc006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cd415eab0 0x7f2cd40c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cd40cb0c0 0x7f2cd4159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 --2- 172.21.15.181:0/1327135127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e6d0 0x7f2cd40c5130 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 >> 172.21.15.181:0/1327135127 conn(0x7f2cd40d04a0 msgr2=0x7f2cd40ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 shutdown_connections 2023-10-16T09:59:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.789+0000 7f2cd91e3700 1 -- 172.21.15.181:0/1327135127 wait complete. 2023-10-16T09:59:08.792 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 25 2023-10-16T09:59:08.802 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:08.884 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:08.883+0000 7efe1401b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:08.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.966+0000 7f5ddad88700 1 Processor -- start 2023-10-16T09:59:08.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.966+0000 7f5ddad88700 1 -- start start 2023-10-16T09:59:08.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5ddad88700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dd415dc20 0x7f5dd40d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5ddad88700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5dd40d1e60 0x7f5dd40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5ddad88700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd40b2f00 con 0x7f5dd40ce4b0 2023-10-16T09:59:08.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5ddad88700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd40b3b80 con 0x7f5dd415dc20 2023-10-16T09:59:08.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5ddad88700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd40d2820 con 0x7f5dd40d1e60 2023-10-16T09:59:08.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dd415dc20 0x7f5dd40d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dd415dc20 0x7f5dd40d41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41152/0 (socket says 172.21.15.181:41152) 2023-10-16T09:59:08.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 -- 172.21.15.181:0/2109522890 learned_addr learned my addr 172.21.15.181:0/2109522890 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:08.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd3fff700 1 --2- 172.21.15.181:0/2109522890 >> v2:172.21.15.190:3300/0 conn(0x7f5dd40d1e60 0x7f5dd40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 -- 172.21.15.181:0/2109522890 >> v2:172.21.15.190:3300/0 conn(0x7f5dd40d1e60 msgr2=0x7f5dd40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:08.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 --2- 172.21.15.181:0/2109522890 >> v2:172.21.15.190:3300/0 conn(0x7f5dd40d1e60 0x7f5dd40d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 -- 172.21.15.181:0/2109522890 >> v1:172.21.15.190:6789/0 conn(0x7f5dd40ce4b0 legacy=0x7f5dd40ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 -- 172.21.15.181:0/2109522890 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40b31e0 con 0x7f5dd415dc20 2023-10-16T09:59:08.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.967+0000 7f5dd8b24700 1 --2- 172.21.15.181:0/2109522890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd40d41c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4161750 tx=0x7f5dc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f88b203309bb1ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:08.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5dd37fe700 1 -- 172.21.15.181:0/2109522890 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8005e30 con 0x7f5dd415dc20 2023-10-16T09:59:08.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5dd37fe700 1 -- 172.21.15.181:0/2109522890 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc8004030 con 0x7f5dd415dc20 2023-10-16T09:59:08.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5dd37fe700 1 -- 172.21.15.181:0/2109522890 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80043e0 con 0x7f5dd415dc20 2023-10-16T09:59:08.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 -- 172.21.15.181:0/2109522890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 msgr2=0x7f5dd40d41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 --2- 172.21.15.181:0/2109522890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd40d41c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4161750 tx=0x7f5dc8009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 -- 172.21.15.181:0/2109522890 shutdown_connections 2023-10-16T09:59:08.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 --2- 172.21.15.181:0/2109522890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd40d41c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 --2- 172.21.15.181:0/2109522890 >> v2:172.21.15.190:3300/0 conn(0x7f5dd40d1e60 0x7f5dd40d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 -- 172.21.15.181:0/2109522890 >> 172.21.15.181:0/2109522890 conn(0x7f5dd408af90 msgr2=0x7f5dd40d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:08.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 -- 172.21.15.181:0/2109522890 shutdown_connections 2023-10-16T09:59:08.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.968+0000 7f5ddad88700 1 -- 172.21.15.181:0/2109522890 wait complete. 2023-10-16T09:59:08.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.969+0000 7f5ddad88700 1 Processor -- start 2023-10-16T09:59:08.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.969+0000 7f5ddad88700 1 -- start start 2023-10-16T09:59:08.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5ddad88700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd40ce4b0 0x7f5dd40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5ddad88700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd40d1e60 0x7f5dd415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5ddad88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5ddad88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40a0d60 con 0x7f5dd415dc20 2023-10-16T09:59:08.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5ddad88700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40cdd30 con 0x7f5dd40d1e60 2023-10-16T09:59:08.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5ddad88700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40a04c0 con 0x7f5dd40ce4b0 2023-10-16T09:59:08.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5dd9325700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5dd9325700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41162/0 (socket says 172.21.15.181:41162) 2023-10-16T09:59:08.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5dd9325700 1 -- 172.21.15.181:0/3001694938 learned_addr learned my addr 172.21.15.181:0/3001694938 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:08.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5dd8b24700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd40ce4b0 0x7f5dd40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.970+0000 7f5dd3fff700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd40d1e60 0x7f5dd415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:08.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd9325700 1 -- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd40ce4b0 msgr2=0x7f5dd40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd9325700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd40ce4b0 0x7f5dd40cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd9325700 1 -- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd40d1e60 msgr2=0x7f5dd415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:08.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd9325700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd40d1e60 0x7f5dd415bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:08.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd9325700 1 -- 172.21.15.181:0/3001694938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8009500 con 0x7f5dd415dc20 2023-10-16T09:59:08.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd9325700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd415c1b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00cc50 tx=0x7f5dcc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:08.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd17fa700 1 -- 172.21.15.181:0/3001694938 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc003c00 con 0x7f5dd415dc20 2023-10-16T09:59:08.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd17fa700 1 -- 172.21.15.181:0/3001694938 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc003d60 con 0x7f5dd415dc20 2023-10-16T09:59:08.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd40a07a0 con 0x7f5dd415dc20 2023-10-16T09:59:08.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5dd17fa700 1 -- 172.21.15.181:0/3001694938 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00aab0 con 0x7f5dd415dc20 2023-10-16T09:59:08.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.971+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd40cb5c0 con 0x7f5dd415dc20 2023-10-16T09:59:08.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.972+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd40cb7e0 con 0x7f5dd415dc20 2023-10-16T09:59:08.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.973+0000 7f5dd17fa700 1 -- 172.21.15.181:0/3001694938 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f5dcc00ac10 con 0x7f5dd415dc20 2023-10-16T09:59:08.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.973+0000 7f5dd17fa700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 0x7f5db806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:08.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.973+0000 7f5dd8b24700 1 -- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 msgr2=0x7f5db806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:08.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.973+0000 7f5dd17fa700 1 -- 172.21.15.181:0/3001694938 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6563+0+0 (secure 0 0 0) 0x7f5dcc089990 con 0x7f5dd415dc20 2023-10-16T09:59:08.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.973+0000 7f5dd8b24700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 0x7f5db806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:08.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:08.977+0000 7f5dd17fa700 1 -- 172.21.15.181:0/3001694938 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5dcc054660 con 0x7f5dd415dc20 2023-10-16T09:59:09.059 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.059+0000 7efe1401b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:59:09.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.174+0000 7f5dd8b24700 1 -- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 msgr2=0x7f5db806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:09.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.174+0000 7f5dd8b24700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 0x7f5db806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:09.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.179+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dd406d4b0 con 0x7f5dd415dc20 2023-10-16T09:59:09.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.181+0000 7f5dd17fa700 1 -- 172.21.15.181:0/3001694938 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17858 (secure 0 0 0) 0x7f5dcc057c80 con 0x7f5dd415dc20 2023-10-16T09:59:09.181 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:09.182 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":63,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:08.205387+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:57:00.297618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T09:57:00.838546+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2507249913":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2125399270":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2970752475":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/3339446951":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2864335478":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6833/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6832/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/534300178":"2023-10-17T09:57:15.720457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:09.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 msgr2=0x7f5db806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 0x7f5db806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 msgr2=0x7f5dd415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd415c1b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00cc50 tx=0x7f5dcc00cf60 comp rx=0 tx=0).stop 2023-10-16T09:59:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 shutdown_connections 2023-10-16T09:59:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7f5db806b020 0x7f5db806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd40ce4b0 0x7f5dd40cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dd415dc20 0x7f5dd415c1b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 --2- 172.21.15.181:0/3001694938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd40d1e60 0x7f5dd415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 >> 172.21.15.181:0/3001694938 conn(0x7f5dd408af90 msgr2=0x7f5dd415cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 shutdown_connections 2023-10-16T09:59:09.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.183+0000 7f5ddad88700 1 -- 172.21.15.181:0/3001694938 wait complete. 2023-10-16T09:59:09.196 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T09:59:09.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:09.298 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:09.297+0000 7f30d3d09200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:59:09.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.361+0000 7fd97ec31700 1 Processor -- start 2023-10-16T09:59:09.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.361+0000 7fd97ec31700 1 -- start start 2023-10-16T09:59:09.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.362+0000 7fd97ec31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 0x7fd9780ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.362+0000 7fd97ec31700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9780cc510 con 0x7fd9780cd2d0 2023-10-16T09:59:09.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.362+0000 7fd97ec31700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9780cc7f0 con 0x7fd9780d0370 2023-10-16T09:59:09.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.362+0000 7fd97ec31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780ccad0 con 0x7fd97815b990 2023-10-16T09:59:09.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.362+0000 7fd977fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd9780cd2d0 0x7fd9780cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52246/0 (socket says 172.21.15.181:52246) 2023-10-16T09:59:09.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.362+0000 7fd977fff700 1 -- 172.21.15.181:0/2341162429 learned_addr learned my addr 172.21.15.181:0/2341162429 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.362+0000 7fd97c9cd700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 0x7fd9780ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829518437 0 0) 0x7fd9780cc510 con 0x7fd9780cd2d0 2023-10-16T09:59:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9680035b0 con 0x7fd9780cd2d0 2023-10-16T09:59:09.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019020580 0 0) 0x7fd9780cc7f0 con 0x7fd9780d0370 2023-10-16T09:59:09.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9780cc510 con 0x7fd9780d0370 2023-10-16T09:59:09.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884169658 0 0) 0x7fd9680035b0 con 0x7fd9780cd2d0 2023-10-16T09:59:09.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 msgr2=0x7fd9780ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:09.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 0x7fd9780ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 >> v1:172.21.15.190:6789/0 conn(0x7fd9780d0370 legacy=0x7fd9780ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780ccdb0 con 0x7fd9780cd2d0 2023-10-16T09:59:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd964001670 con 0x7fd9780cd2d0 2023-10-16T09:59:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 >> v1:172.21.15.181:6789/0 conn(0x7fd9780cd2d0 legacy=0x7fd9780cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 0x7fd968003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd968004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd968004d90 0x7fd968005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780ccad0 con 0x7fd968004170 2023-10-16T09:59:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96800bb50 con 0x7fd968004d90 2023-10-16T09:59:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96800bd10 con 0x7fd97815b990 2023-10-16T09:59:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd977fff700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd968004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd97d1ce700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd968004d90 0x7fd968005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd97c9cd700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 0x7fd968003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd977fff700 1 -- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 msgr2=0x7fd968003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd977fff700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd97815b990 0x7fd968003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd977fff700 1 -- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd968004d90 msgr2=0x7fd968005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd977fff700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd968004d90 0x7fd968005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.363+0000 7fd977fff700 1 -- 172.21.15.181:0/2341162429 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9680035b0 con 0x7fd968004170 2023-10-16T09:59:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd977fff700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd968004850 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c110 tx=0x7fd968005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c0b27213059671 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96400cbf0 con 0x7fd968004170 2023-10-16T09:59:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96400cd50 con 0x7fd968004170 2023-10-16T09:59:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd9777fe700 1 -- 172.21.15.181:0/2341162429 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964005ba0 con 0x7fd968004170 2023-10-16T09:59:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd97ec31700 1 -- 172.21.15.181:0/2341162429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 msgr2=0x7fd968004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd97ec31700 1 --2- 172.21.15.181:0/2341162429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd968004850 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c110 tx=0x7fd968005880 comp rx=0 tx=0).stop 2023-10-16T09:59:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd97d1ce700 1 -- 172.21.15.181:0/2341162429 reap_dead start 2023-10-16T09:59:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd97ec31700 1 -- 172.21.15.181:0/2341162429 shutdown_connections 2023-10-16T09:59:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd97ec31700 1 -- 172.21.15.181:0/2341162429 >> 172.21.15.181:0/2341162429 conn(0x7fd97809a9a0 msgr2=0x7fd9780d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd97ec31700 1 -- 172.21.15.181:0/2341162429 shutdown_connections 2023-10-16T09:59:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.364+0000 7fd97ec31700 1 -- 172.21.15.181:0/2341162429 wait complete. 2023-10-16T09:59:09.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.366+0000 7fd97ec31700 1 Processor -- start 2023-10-16T09:59:09.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.366+0000 7fd97ec31700 1 -- start start 2023-10-16T09:59:09.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd97815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97815c3b0 0x7fd9780c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9780b4350 0x7fd9780b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97815c900 con 0x7fd968004170 2023-10-16T09:59:09.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97815cbe0 con 0x7fd97815c3b0 2023-10-16T09:59:09.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97815cec0 con 0x7fd9780b4350 2023-10-16T09:59:09.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd97815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd97815d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41180/0 (socket says 172.21.15.181:41180) 2023-10-16T09:59:09.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 -- 172.21.15.181:0/303593572 learned_addr learned my addr 172.21.15.181:0/303593572 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:09.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd977fff700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97815c3b0 0x7fd9780c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97d1ce700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9780b4350 0x7fd9780b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 -- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9780b4350 msgr2=0x7fd9780b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9780b4350 0x7fd9780b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 -- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97815c3b0 msgr2=0x7fd9780c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97815c3b0 0x7fd9780c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 -- 172.21.15.181:0/303593572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97815d1a0 con 0x7fd968004170 2023-10-16T09:59:09.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97c9cd700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd97815d9a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0119f0 tx=0x7fd96c011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:09.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd9757fa700 1 -- 172.21.15.181:0/303593572 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c011e90 con 0x7fd968004170 2023-10-16T09:59:09.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd9757fa700 1 -- 172.21.15.181:0/303593572 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c005c30 con 0x7fd968004170 2023-10-16T09:59:09.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97815d480 con 0x7fd968004170 2023-10-16T09:59:09.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd9757fa700 1 -- 172.21.15.181:0/303593572 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00f730 con 0x7fd968004170 2023-10-16T09:59:09.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.367+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9780c8c90 con 0x7fd968004170 2023-10-16T09:59:09.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.368+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97815d7f0 con 0x7fd968004170 2023-10-16T09:59:09.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.369+0000 7fd9757fa700 1 -- 172.21.15.181:0/303593572 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd96c00f890 con 0x7fd968004170 2023-10-16T09:59:09.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.369+0000 7fd9757fa700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 0x7fd95c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.369+0000 7fd977fff700 1 -- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 msgr2=0x7fd95c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:09.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.369+0000 7fd9757fa700 1 -- 172.21.15.181:0/303593572 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6563+0+0 (secure 0 0 0) 0x7fd96c08a4d0 con 0x7fd968004170 2023-10-16T09:59:09.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.369+0000 7fd977fff700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 0x7fd95c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:09.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.377+0000 7fd9757fa700 1 -- 172.21.15.181:0/303593572 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd96c055340 con 0x7fd968004170 2023-10-16T09:59:09.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.570+0000 7fd977fff700 1 -- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 msgr2=0x7fd95c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3708044156 2023-10-16T09:59:09.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.570+0000 7fd977fff700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 0x7fd95c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:09.619 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.619+0000 7efe1401b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:59:09.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.627+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd978067d20 con 0x7fd968004170 2023-10-16T09:59:09.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.628+0000 7fd9757fa700 1 -- 172.21.15.181:0/303593572 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fd96c058960 con 0x7fd968004170 2023-10-16T09:59:09.629 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:09.629 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5925,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/47573574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":47573574},{"type":"v1","addr":"172.21.15.181:6836","nonce":47573574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3518254514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3518254514},{"type":"v1","addr":"172.21.15.181:6837","nonce":3518254514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4292050143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4292050143},{"type":"v1","addr":"172.21.15.190:6835","nonce":4292050143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:57:01.885491+0000","modified":"2023-10-16T09:57:04.906845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5943":{"gid":5943,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2407452630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2407452630},{"type":"v1","addr":"172.21.15.190:6834","nonce":2407452630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T09:59:09.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 msgr2=0x7fd95c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:09.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 0x7fd95c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 msgr2=0x7fd97815d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd97815d9a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0119f0 tx=0x7fd96c011d00 comp rx=0 tx=0).stop 2023-10-16T09:59:09.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 shutdown_connections 2023-10-16T09:59:09.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:6832/3708044156,v1:172.21.15.181:6833/3708044156] conn(0x7fd95c06b020 0x7fd95c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9780b4350 0x7fd9780b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd968004170 0x7fd97815d9a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 --2- 172.21.15.181:0/303593572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd97815c3b0 0x7fd9780c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 >> 172.21.15.181:0/303593572 conn(0x7fd97809a9a0 msgr2=0x7fd97806ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:09.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 shutdown_connections 2023-10-16T09:59:09.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.630+0000 7fd97ec31700 1 -- 172.21.15.181:0/303593572 wait complete. 2023-10-16T09:59:09.634 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 25 2023-10-16T09:59:09.646 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:09.707 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8c801700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.707 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8c801700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.708 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8c801700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.708 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8c801700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.708 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8c801700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.708 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8f807700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.709 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8f807700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.709 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8f807700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.709 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8f807700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.710 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:09.706+0000 7efd8f807700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.776+0000 7fe566886700 1 Processor -- start 2023-10-16T09:59:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.776+0000 7fe566886700 1 -- start start 2023-10-16T09:59:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.776+0000 7fe566886700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe56015b990 0x7fe5600ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.776+0000 7fe566886700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5600d0370 0x7fe5600d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.776+0000 7fe566886700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600cc510 con 0x7fe5600cd2d0 2023-10-16T09:59:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.776+0000 7fe566886700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5600d24f0 con 0x7fe5600d0370 2023-10-16T09:59:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.776+0000 7fe566886700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600cc7f0 con 0x7fe56015b990 2023-10-16T09:59:09.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5600d0370 0x7fe5600d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5600d0370 0x7fe5600d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41266/0 (socket says 172.21.15.181:41266) 2023-10-16T09:59:09.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 -- 172.21.15.181:0/1153239122 learned_addr learned my addr 172.21.15.181:0/1153239122 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:09.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe55ffff700 1 --2- 172.21.15.181:0/1153239122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe56015b990 0x7fe5600ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 -- 172.21.15.181:0/1153239122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe56015b990 msgr2=0x7fe5600ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:09.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 --2- 172.21.15.181:0/1153239122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe56015b990 0x7fe5600ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 -- 172.21.15.181:0/1153239122 >> v1:172.21.15.190:6789/0 conn(0x7fe5600cd2d0 legacy=0x7fe5600cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:09.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 -- 172.21.15.181:0/1153239122 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600ccad0 con 0x7fe5600d0370 2023-10-16T09:59:09.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe564e23700 1 --2- 172.21.15.181:0/1153239122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5600d0370 0x7fe5600d1fb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe558009e50 tx=0x7fe55800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccc201ec7718527 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:09.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe55effd700 1 -- 172.21.15.181:0/1153239122 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800e070 con 0x7fe5600d0370 2023-10-16T09:59:09.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe55effd700 1 -- 172.21.15.181:0/1153239122 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558003a20 con 0x7fe5600d0370 2023-10-16T09:59:09.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.777+0000 7fe55effd700 1 -- 172.21.15.181:0/1153239122 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558003dd0 con 0x7fe5600d0370 2023-10-16T09:59:09.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 -- 172.21.15.181:0/1153239122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5600d0370 msgr2=0x7fe5600d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 --2- 172.21.15.181:0/1153239122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5600d0370 0x7fe5600d1fb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe558009e50 tx=0x7fe55800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 -- 172.21.15.181:0/1153239122 shutdown_connections 2023-10-16T09:59:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 --2- 172.21.15.181:0/1153239122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5600d0370 0x7fe5600d1fb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 --2- 172.21.15.181:0/1153239122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe56015b990 0x7fe5600ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 -- 172.21.15.181:0/1153239122 >> 172.21.15.181:0/1153239122 conn(0x7fe56009a9a0 msgr2=0x7fe5600d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:09.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 -- 172.21.15.181:0/1153239122 shutdown_connections 2023-10-16T09:59:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.778+0000 7fe566886700 1 -- 172.21.15.181:0/1153239122 wait complete. 2023-10-16T09:59:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.779+0000 7fe566886700 1 Processor -- start 2023-10-16T09:59:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.779+0000 7fe566886700 1 -- start start 2023-10-16T09:59:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe566886700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5600cd2d0 0x7fe5601f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe566886700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5600d0370 0x7fe56015dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe566886700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe56015b990 0x7fe5600b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:09.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe566886700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600b4a40 con 0x7fe56015b990 2023-10-16T09:59:09.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe566886700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600b4bd0 con 0x7fe5600d0370 2023-10-16T09:59:09.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe566886700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601f2f20 con 0x7fe5600cd2d0 2023-10-16T09:59:09.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe56015b990 0x7fe5600b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe56015b990 0x7fe5600b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41272/0 (socket says 172.21.15.181:41272) 2023-10-16T09:59:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 -- 172.21.15.181:0/3232551032 learned_addr learned my addr 172.21.15.181:0/3232551032 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe55f7fe700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5600d0370 0x7fe56015dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe55ffff700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5600cd2d0 0x7fe5601f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 -- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5600cd2d0 msgr2=0x7fe5601f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5600cd2d0 0x7fe5601f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 -- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5600d0370 msgr2=0x7fe56015dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5600d0370 0x7fe56015dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 -- 172.21.15.181:0/3232551032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe558009b30 con 0x7fe56015b990 2023-10-16T09:59:09.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe564e23700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe56015b990 0x7fe5600b0c50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe558004000 tx=0x7fe558004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:09.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe55cff9700 1 -- 172.21.15.181:0/3232551032 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558004440 con 0x7fe56015b990 2023-10-16T09:59:09.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe55cff9700 1 -- 172.21.15.181:0/3232551032 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5580045c0 con 0x7fe56015b990 2023-10-16T09:59:09.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe55cff9700 1 -- 172.21.15.181:0/3232551032 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558004970 con 0x7fe56015b990 2023-10-16T09:59:09.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.780+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56006c7a0 con 0x7fe56015b990 2023-10-16T09:59:09.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.781+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56006cd60 con 0x7fe56015b990 2023-10-16T09:59:09.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.781+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5600b4d80 con 0x7fe56015b990 2023-10-16T09:59:09.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.783+0000 7fe55cff9700 1 -- 172.21.15.181:0/3232551032 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 29) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe55802e430 con 0x7fe56015b990 2023-10-16T09:59:09.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.783+0000 7fe55cff9700 1 -- 172.21.15.181:0/3232551032 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6879+0+0 (secure 0 0 0) 0x7fe55808a9c0 con 0x7fe56015b990 2023-10-16T09:59:09.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:09.788+0000 7fe55cff9700 1 -- 172.21.15.181:0/3232551032 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe558055a20 con 0x7fe56015b990 2023-10-16T09:59:09.954 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:09.957 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:10.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.026+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5600b4d40 con 0x7fe56015b990 2023-10-16T09:59:10.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.026+0000 7fe55cff9700 1 -- 172.21.15.181:0/3232551032 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fe558059210 con 0x7fe56015b990 2023-10-16T09:59:10.027 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:10.028 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5925,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/47573574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":47573574},{"type":"v1","addr":"172.21.15.181:6836","nonce":47573574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3518254514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3518254514},{"type":"v1","addr":"172.21.15.181:6837","nonce":3518254514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4292050143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4292050143},{"type":"v1","addr":"172.21.15.190:6835","nonce":4292050143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:57:01.885491+0000","modified":"2023-10-16T09:57:04.906845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5943":{"gid":5943,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2407452630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2407452630},{"type":"v1","addr":"172.21.15.190:6834","nonce":2407452630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-16T09:59:10.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.028+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe56015b990 msgr2=0x7fe5600b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.028+0000 7fe566886700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe56015b990 0x7fe5600b0c50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe558004000 tx=0x7fe558004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.029+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 shutdown_connections 2023-10-16T09:59:10.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.029+0000 7fe566886700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5600cd2d0 0x7fe5601f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.029+0000 7fe566886700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe56015b990 0x7fe5600b0c50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.029+0000 7fe566886700 1 --2- 172.21.15.181:0/3232551032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5600d0370 0x7fe56015dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.029+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 >> 172.21.15.181:0/3232551032 conn(0x7fe56009a9a0 msgr2=0x7fe560073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:10.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.029+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 shutdown_connections 2023-10-16T09:59:10.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.029+0000 7fe566886700 1 -- 172.21.15.181:0/3232551032 wait complete. 2023-10-16T09:59:10.031 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 25 2023-10-16T09:59:10.041 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:10.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.201+0000 7fcf04bd9700 1 Processor -- start 2023-10-16T09:59:10.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.201+0000 7fcf04bd9700 1 -- start start 2023-10-16T09:59:10.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcf04bd9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf0015e6d0 0x7fcf000d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcf04bd9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf000cb0c0 0x7fcf0015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcf04bd9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0006ed00 con 0x7fcf0015eab0 2023-10-16T09:59:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcf04bd9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf000d02d0 con 0x7fcf000cb0c0 2023-10-16T09:59:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcf04bd9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0015ad40 con 0x7fcf0015e6d0 2023-10-16T09:59:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefe59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf0015e6d0 0x7fcf000d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefe59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf0015e6d0 0x7fcf000d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:38300/0 (socket says 172.21.15.181:38300) 2023-10-16T09:59:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefe59c700 1 -- 172.21.15.181:0/221084669 learned_addr learned my addr 172.21.15.181:0/221084669 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefed9d700 1 --2- 172.21.15.181:0/221084669 >> v2:172.21.15.190:3300/0 conn(0x7fcf000cb0c0 0x7fcf0015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefed9d700 1 -- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf0015e6d0 msgr2=0x7fcf000d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefed9d700 1 --2- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf0015e6d0 0x7fcf000d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefed9d700 1 -- 172.21.15.181:0/221084669 >> v1:172.21.15.190:6789/0 conn(0x7fcf0015eab0 legacy=0x7fcf0015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefed9d700 1 -- 172.21.15.181:0/221084669 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0006e0f0 con 0x7fcf000cb0c0 2023-10-16T09:59:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefd59a700 1 -- 172.21.15.181:0/221084669 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995740092 0 0) 0x7fcf0006ed00 con 0x7fcf0015eab0 2023-10-16T09:59:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.202+0000 7fcefe59c700 1 --2- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf0015e6d0 0x7fcf000d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcefed9d700 1 --2- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf000cb0c0 0x7fcf0015a800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27427e55dfdef0c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcefd59a700 1 -- 172.21.15.181:0/221084669 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef800e070 con 0x7fcf000cb0c0 2023-10-16T09:59:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcefd59a700 1 -- 172.21.15.181:0/221084669 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef800bb20 con 0x7fcf000cb0c0 2023-10-16T09:59:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcefd59a700 1 -- 172.21.15.181:0/221084669 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8012430 con 0x7fcf000cb0c0 2023-10-16T09:59:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 -- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf000cb0c0 msgr2=0x7fcf0015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 --2- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf000cb0c0 0x7fcf0015a800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009d90 tx=0x7fcef800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 -- 172.21.15.181:0/221084669 shutdown_connections 2023-10-16T09:59:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 --2- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf000cb0c0 0x7fcf0015a800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 --2- 172.21.15.181:0/221084669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf0015e6d0 0x7fcf000d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 -- 172.21.15.181:0/221084669 >> 172.21.15.181:0/221084669 conn(0x7fcf000d04a0 msgr2=0x7fcf000d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 -- 172.21.15.181:0/221084669 shutdown_connections 2023-10-16T09:59:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.203+0000 7fcf04bd9700 1 -- 172.21.15.181:0/221084669 wait complete. 2023-10-16T09:59:10.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 Processor -- start 2023-10-16T09:59:10.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 -- start start 2023-10-16T09:59:10.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf000cb0c0 0x7fcf00158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf0015e6d0 0x7fcf0015b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf0015eab0 0x7fcf0015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0015dcb0 con 0x7fcf0015e6d0 2023-10-16T09:59:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000ae700 con 0x7fcf0015eab0 2023-10-16T09:59:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.205+0000 7fcf04bd9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000ae9e0 con 0x7fcf000cb0c0 2023-10-16T09:59:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf0015e6d0 0x7fcf0015b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf0015e6d0 0x7fcf0015b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41282/0 (socket says 172.21.15.181:41282) 2023-10-16T09:59:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 -- 172.21.15.181:0/2731125414 learned_addr learned my addr 172.21.15.181:0/2731125414 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefe59c700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf000cb0c0 0x7fcf00158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefed9d700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf0015eab0 0x7fcf0015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 -- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf000cb0c0 msgr2=0x7fcf00158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf000cb0c0 0x7fcf00158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 -- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf0015eab0 msgr2=0x7fcf0015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf0015eab0 0x7fcf0015bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 -- 172.21.15.181:0/2731125414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0009500 con 0x7fcf0015e6d0 2023-10-16T09:59:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefed9d700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf0015eab0 0x7fcf0015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcefdd9b700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf0015e6d0 0x7fcf0015b770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcef40011e0 tx=0x7fcef400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fceeeffd700 1 -- 172.21.15.181:0/2731125414 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40054e0 con 0x7fcf0015e6d0 2023-10-16T09:59:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fceeeffd700 1 -- 172.21.15.181:0/2731125414 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef4003680 con 0x7fcf0015e6d0 2023-10-16T09:59:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef8009a70 con 0x7fcf0015e6d0 2023-10-16T09:59:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fceeeffd700 1 -- 172.21.15.181:0/2731125414 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4003a30 con 0x7fcf0015e6d0 2023-10-16T09:59:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.206+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf000b4750 con 0x7fcf0015e6d0 2023-10-16T09:59:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.207+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf00067d20 con 0x7fcf0015e6d0 2023-10-16T09:59:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.208+0000 7fceeeffd700 1 -- 172.21.15.181:0/2731125414 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 29) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fcef4003b90 con 0x7fcf0015e6d0 2023-10-16T09:59:10.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.208+0000 7fceeeffd700 1 -- 172.21.15.181:0/2731125414 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6879+0+0 (secure 0 0 0) 0x7fcef4088a90 con 0x7fcf0015e6d0 2023-10-16T09:59:10.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.212+0000 7fceeeffd700 1 -- 172.21.15.181:0/2731125414 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcef4053a80 con 0x7fcf0015e6d0 2023-10-16T09:59:10.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.407+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf000cb610 con 0x7fcf0015e6d0 2023-10-16T09:59:10.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.408+0000 7fceeeffd700 1 -- 172.21.15.181:0/2731125414 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18323 (secure 0 0 0) 0x7fcef40570a0 con 0x7fcf0015e6d0 2023-10-16T09:59:10.409 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:10.409 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":64,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:09.633181+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:57:00.297618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-16T09:57:00.838546+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2979005927":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/980850107":"2023-10-17T09:59:09.633143+0000","172.21.15.181:6833/3708044156":"2023-10-17T09:59:09.633143+0000","172.21.15.181:6832/3708044156":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/2177777009":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/1842196234":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/406774785":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/3225128684":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/2507249913":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2125399270":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2970752475":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/3339446951":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2864335478":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6833/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6832/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/534300178":"2023-10-17T09:57:15.720457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:10.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf0015e6d0 msgr2=0x7fcf0015b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf0015e6d0 0x7fcf0015b770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcef40011e0 tx=0x7fcef400cae0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 shutdown_connections 2023-10-16T09:59:10.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf000cb0c0 0x7fcf00158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf0015eab0 0x7fcf0015bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 --2- 172.21.15.181:0/2731125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf0015e6d0 0x7fcf0015b770 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 >> 172.21.15.181:0/2731125414 conn(0x7fcf000d04a0 msgr2=0x7fcf000cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:10.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 shutdown_connections 2023-10-16T09:59:10.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.410+0000 7fcf04bd9700 1 -- 172.21.15.181:0/2731125414 wait complete. 2023-10-16T09:59:10.423 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T09:59:10.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.546+0000 7fc658bcd700 1 Processor -- start 2023-10-16T09:59:10.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.547+0000 7fc658bcd700 1 -- start start 2023-10-16T09:59:10.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.547+0000 7fc658bcd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6540d0fc0 0x7fc6540cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.547+0000 7fc658bcd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65406a8a0 0x7fc65406ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.547+0000 7fc658bcd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6540cddf0 con 0x7fc65415df60 2023-10-16T09:59:10.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.547+0000 7fc658bcd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65406b290 con 0x7fc6540d0fc0 2023-10-16T09:59:10.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.547+0000 7fc658bcd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540af9c0 con 0x7fc65406a8a0 2023-10-16T09:59:10.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc65415df60 0x7fc6540cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52364/0 (socket says 172.21.15.181:52364) 2023-10-16T09:59:10.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65259c700 1 -- 172.21.15.181:0/3568096497 learned_addr learned my addr 172.21.15.181:0/3568096497 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:10.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc652d9d700 1 --2- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65406a8a0 0x7fc65406ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc651d9b700 1 --2- 172.21.15.181:0/3568096497 >> v2:172.21.15.190:3300/0 conn(0x7fc6540d0fc0 0x7fc6540cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65159a700 1 -- 172.21.15.181:0/3568096497 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352139666 0 0) 0x7fc6540cddf0 con 0x7fc65415df60 2023-10-16T09:59:10.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65159a700 1 -- 172.21.15.181:0/3568096497 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6400035b0 con 0x7fc65415df60 2023-10-16T09:59:10.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc651d9b700 1 -- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65406a8a0 msgr2=0x7fc65406ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:10.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc651d9b700 1 --2- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65406a8a0 0x7fc65406ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc651d9b700 1 -- 172.21.15.181:0/3568096497 >> v1:172.21.15.181:6789/0 conn(0x7fc65415df60 legacy=0x7fc6540cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65159a700 1 -- 172.21.15.181:0/3568096497 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 368666148 0 0) 0x7fc6400035b0 con 0x7fc65415df60 2023-10-16T09:59:10.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc651d9b700 1 -- 172.21.15.181:0/3568096497 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540ccf50 con 0x7fc6540d0fc0 2023-10-16T09:59:10.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc652d9d700 1 --2- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65406a8a0 0x7fc65406ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:10.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc651d9b700 1 --2- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 0x7fc6540cae20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc644009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60675bc0c2097f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65159a700 1 -- 172.21.15.181:0/3568096497 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0017f0 con 0x7fc6540d0fc0 2023-10-16T09:59:10.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65159a700 1 -- 172.21.15.181:0/3568096497 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644004030 con 0x7fc6540d0fc0 2023-10-16T09:59:10.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.548+0000 7fc65159a700 1 -- 172.21.15.181:0/3568096497 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440043c0 con 0x7fc6540d0fc0 2023-10-16T09:59:10.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 -- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 msgr2=0x7fc6540cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 --2- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 0x7fc6540cae20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc644009b80 comp rx=0 tx=0).stop 2023-10-16T09:59:10.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 -- 172.21.15.181:0/3568096497 shutdown_connections 2023-10-16T09:59:10.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 --2- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65406a8a0 0x7fc65406ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 --2- 172.21.15.181:0/3568096497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 0x7fc6540cae20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 -- 172.21.15.181:0/3568096497 >> 172.21.15.181:0/3568096497 conn(0x7fc6540d04a0 msgr2=0x7fc6540d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:10.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 -- 172.21.15.181:0/3568096497 shutdown_connections 2023-10-16T09:59:10.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.549+0000 7fc658bcd700 1 -- 172.21.15.181:0/3568096497 wait complete. 2023-10-16T09:59:10.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.550+0000 7fc658bcd700 1 Processor -- start 2023-10-16T09:59:10.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.550+0000 7fc658bcd700 1 -- start start 2023-10-16T09:59:10.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc658bcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65406a8a0 0x7fc654158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc658bcd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 0x7fc6540aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc658bcd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65415df60 0x7fc6540ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc658bcd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c7930 con 0x7fc65406a8a0 2023-10-16T09:59:10.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc658bcd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540b45d0 con 0x7fc6540d0fc0 2023-10-16T09:59:10.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc658bcd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654159090 con 0x7fc65415df60 2023-10-16T09:59:10.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc65259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65406a8a0 0x7fc654158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc65259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65406a8a0 0x7fc654158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41292/0 (socket says 172.21.15.181:41292) 2023-10-16T09:59:10.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc65259c700 1 -- 172.21.15.181:0/4186341466 learned_addr learned my addr 172.21.15.181:0/4186341466 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:10.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc652d9d700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65415df60 0x7fc6540ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.551+0000 7fc651d9b700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 0x7fc6540aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc65259c700 1 -- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65415df60 msgr2=0x7fc6540ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc65259c700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65415df60 0x7fc6540ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc65259c700 1 -- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 msgr2=0x7fc6540aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc65259c700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 0x7fc6540aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc65259c700 1 -- 172.21.15.181:0/4186341466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c009a70 con 0x7fc65406a8a0 2023-10-16T09:59:10.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc65259c700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65406a8a0 0x7fc654158b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c750 tx=0x7fc63c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0074e0 con 0x7fc65406a8a0 2023-10-16T09:59:10.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c007640 con 0x7fc65406a8a0 2023-10-16T09:59:10.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c005a70 con 0x7fc65406a8a0 2023-10-16T09:59:10.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6440094a0 con 0x7fc65406a8a0 2023-10-16T09:59:10.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.552+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540b4b00 con 0x7fc65406a8a0 2023-10-16T09:59:10.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.553+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc654067d20 con 0x7fc65406a8a0 2023-10-16T09:59:10.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.553+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 29) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fc63c005bd0 con 0x7fc65406a8a0 2023-10-16T09:59:10.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.554+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6879+0+0 (secure 0 0 0) 0x7fc63c088a00 con 0x7fc65406a8a0 2023-10-16T09:59:10.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.558+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc63c053a70 con 0x7fc65406a8a0 2023-10-16T09:59:10.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.648+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc63c053520 con 0x7fc65406a8a0 2023-10-16T09:59:10.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.648+0000 7fc64affd700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc634070c80 0x7fc634073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.649+0000 7fc651d9b700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc634070c80 0x7fc634073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.650+0000 7fc651d9b700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc634070c80 0x7fc634073070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc6440097c0 tx=0x7fc64401c040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.800+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc6540b0440 con 0x7fc65406a8a0 2023-10-16T09:59:10.802 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T09:59:10.801+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T09:59:10.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.802+0000 7fc64affd700 1 -- 172.21.15.181:0/4186341466 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc63c073140 con 0x7fc65406a8a0 2023-10-16T09:59:10.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc634070c80 msgr2=0x7fc634073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc634070c80 0x7fc634073070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc6440097c0 tx=0x7fc64401c040 comp rx=0 tx=0).stop 2023-10-16T09:59:10.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65406a8a0 msgr2=0x7fc654158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65406a8a0 0x7fc654158b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c750 tx=0x7fc63c00ca60 comp rx=0 tx=0).stop 2023-10-16T09:59:10.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 shutdown_connections 2023-10-16T09:59:10.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc634070c80 0x7fc634073070 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc65415df60 0x7fc6540ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65406a8a0 0x7fc654158b50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 --2- 172.21.15.181:0/4186341466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc6540d0fc0 0x7fc6540aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 >> 172.21.15.181:0/4186341466 conn(0x7fc6540d04a0 msgr2=0x7fc65415ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:10.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 shutdown_connections 2023-10-16T09:59:10.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.805+0000 7fc658bcd700 1 -- 172.21.15.181:0/4186341466 wait complete. 2023-10-16T09:59:10.811 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T09:59:10.818 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T09:59:10.842 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T09:59:10.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.943+0000 7f3a680df700 1 Processor -- start 2023-10-16T09:59:10.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.943+0000 7f3a680df700 1 -- start start 2023-10-16T09:59:10.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a680df700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a6015f150 0x7f3a600cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a680df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600cd170 0x7f3a600b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a680df700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a600b2ae0 con 0x7f3a600ce4f0 2023-10-16T09:59:10.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a680df700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a600ce9a0 con 0x7f3a6015f150 2023-10-16T09:59:10.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a680df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a600b2dc0 con 0x7f3a600cd170 2023-10-16T09:59:10.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a65e7b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a6015f150 0x7f3a600cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a6567a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3a600ce4f0 0x7f3a600b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52400/0 (socket says 172.21.15.181:52400) 2023-10-16T09:59:10.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a65e7b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a6015f150 0x7f3a600cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41322/0 (socket says 172.21.15.181:41322) 2023-10-16T09:59:10.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a6567a700 1 -- 172.21.15.181:0/518170675 learned_addr learned my addr 172.21.15.181:0/518170675 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:10.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a6667c700 1 --2- 172.21.15.181:0/518170675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600cd170 0x7f3a600b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a65e7b700 1 -- 172.21.15.181:0/518170675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600cd170 msgr2=0x7f3a600b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:10.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a65e7b700 1 --2- 172.21.15.181:0/518170675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600cd170 0x7f3a600b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a65e7b700 1 -- 172.21.15.181:0/518170675 >> v1:172.21.15.181:6789/0 conn(0x7f3a600ce4f0 legacy=0x7f3a600b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a65e7b700 1 -- 172.21.15.181:0/518170675 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600cf9d0 con 0x7f3a6015f150 2023-10-16T09:59:10.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a64e79700 1 -- 172.21.15.181:0/518170675 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247140211 0 0) 0x7f3a600b2ae0 con 0x7f3a600ce4f0 2023-10-16T09:59:10.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.944+0000 7f3a6667c700 1 --2- 172.21.15.181:0/518170675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600cd170 0x7f3a600b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:10.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a65e7b700 1 --2- 172.21.15.181:0/518170675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600cdfb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009870 tx=0x7f3a5c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0be17bc1d1b76f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a64e79700 1 -- 172.21.15.181:0/518170675 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c011830 con 0x7f3a6015f150 2023-10-16T09:59:10.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a64e79700 1 -- 172.21.15.181:0/518170675 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c011990 con 0x7f3a6015f150 2023-10-16T09:59:10.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a64e79700 1 -- 172.21.15.181:0/518170675 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c011d20 con 0x7f3a6015f150 2023-10-16T09:59:10.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 -- 172.21.15.181:0/518170675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 msgr2=0x7f3a600cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 --2- 172.21.15.181:0/518170675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600cdfb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009870 tx=0x7f3a5c009b80 comp rx=0 tx=0).stop 2023-10-16T09:59:10.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 -- 172.21.15.181:0/518170675 shutdown_connections 2023-10-16T09:59:10.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 --2- 172.21.15.181:0/518170675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600cd170 0x7f3a600b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 --2- 172.21.15.181:0/518170675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600cdfb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 -- 172.21.15.181:0/518170675 >> 172.21.15.181:0/518170675 conn(0x7f3a6009a930 msgr2=0x7f3a600d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:10.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 -- 172.21.15.181:0/518170675 shutdown_connections 2023-10-16T09:59:10.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.945+0000 7f3a680df700 1 -- 172.21.15.181:0/518170675 wait complete. 2023-10-16T09:59:10.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.947+0000 7f3a680df700 1 Processor -- start 2023-10-16T09:59:10.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.947+0000 7f3a680df700 1 -- start start 2023-10-16T09:59:10.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a680df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a600cd170 0x7f3a60158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a680df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600ce4f0 0x7f3a6015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a680df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a680df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6015d090 con 0x7f3a6015f150 2023-10-16T09:59:10.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a680df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6015c220 con 0x7f3a600cd170 2023-10-16T09:59:10.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a680df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6015c4c0 con 0x7f3a600ce4f0 2023-10-16T09:59:10.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41332/0 (socket says 172.21.15.181:41332) 2023-10-16T09:59:10.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 -- 172.21.15.181:0/648026634 learned_addr learned my addr 172.21.15.181:0/648026634 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:10.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a65e7b700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a600cd170 0x7f3a60158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6567a700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600ce4f0 0x7f3a6015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 -- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600ce4f0 msgr2=0x7f3a6015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600ce4f0 0x7f3a6015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 -- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a600cd170 msgr2=0x7f3a60158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:10.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a600cd170 0x7f3a60158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:10.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 -- 172.21.15.181:0/648026634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0094a0 con 0x7f3a6015f150 2023-10-16T09:59:10.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a65e7b700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a600cd170 0x7f3a60158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:10.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.948+0000 7f3a6667c700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600c75f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3a58007ef0 tx=0x7f3a5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.949+0000 7f3a567fc700 1 -- 172.21.15.181:0/648026634 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5800de60 con 0x7f3a6015f150 2023-10-16T09:59:10.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.949+0000 7f3a567fc700 1 -- 172.21.15.181:0/648026634 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5800f040 con 0x7f3a6015f150 2023-10-16T09:59:10.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.949+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6015c760 con 0x7f3a6015f150 2023-10-16T09:59:10.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.949+0000 7f3a567fc700 1 -- 172.21.15.181:0/648026634 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58003c60 con 0x7f3a6015f150 2023-10-16T09:59:10.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.949+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6015cd20 con 0x7f3a6015f150 2023-10-16T09:59:10.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.950+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60067d20 con 0x7f3a6015f150 2023-10-16T09:59:10.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.950+0000 7f3a567fc700 1 -- 172.21.15.181:0/648026634 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3a58004030 con 0x7f3a6015f150 2023-10-16T09:59:10.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.950+0000 7f3a567fc700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3a4406ad50 0x7f3a4406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:10.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.950+0000 7f3a567fc700 1 -- 172.21.15.181:0/648026634 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6965+0+0 (secure 0 0 0) 0x7f3a580898e0 con 0x7f3a6015f150 2023-10-16T09:59:10.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.950+0000 7f3a65e7b700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3a4406ad50 0x7f3a4406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:10.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.951+0000 7f3a65e7b700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3a4406ad50 0x7f3a4406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009790 tx=0x7f3a5c01c040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:10.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:10.955+0000 7f3a567fc700 1 -- 172.21.15.181:0/648026634 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3a580547e0 con 0x7f3a6015f150 2023-10-16T09:59:11.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.193+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3a6006f4c0 con 0x7f3a6015f150 2023-10-16T09:59:11.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.803+0000 7f3a567fc700 1 -- 172.21.15.181:0/648026634 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3a58057e00 con 0x7f3a6015f150 2023-10-16T09:59:11.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3a4406ad50 msgr2=0x7f3a4406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:11.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3a4406ad50 0x7f3a4406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009790 tx=0x7f3a5c01c040 comp rx=0 tx=0).stop 2023-10-16T09:59:11.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 msgr2=0x7f3a600c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:11.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600c75f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3a58007ef0 tx=0x7f3a5800d3b0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 shutdown_connections 2023-10-16T09:59:11.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3a4406ad50 0x7f3a4406d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a6015f150 0x7f3a600c75f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a600cd170 0x7f3a60158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 --2- 172.21.15.181:0/648026634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a600ce4f0 0x7f3a6015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 >> 172.21.15.181:0/648026634 conn(0x7f3a6009a930 msgr2=0x7f3a600ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:11.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 shutdown_connections 2023-10-16T09:59:11.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.806+0000 7f3a680df700 1 -- 172.21.15.181:0/648026634 wait complete. 2023-10-16T09:59:11.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T09:59:11.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.943+0000 7f2746154700 1 Processor -- start 2023-10-16T09:59:11.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.943+0000 7f2746154700 1 -- start start 2023-10-16T09:59:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.944+0000 7f2746154700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 0x7f274015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.944+0000 7f2746154700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27400caf40 con 0x7f274006a8a0 2023-10-16T09:59:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.944+0000 7f2746154700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2740160240 con 0x7f27400cf5c0 2023-10-16T09:59:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.944+0000 7f2746154700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274015bd80 con 0x7f27400caaf0 2023-10-16T09:59:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.944+0000 7f273f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f274006a8a0 0x7f274006ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52432/0 (socket says 172.21.15.181:52432) 2023-10-16T09:59:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.944+0000 7f273f7fe700 1 -- 172.21.15.181:0/2843582666 learned_addr learned my addr 172.21.15.181:0/2843582666 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204634989 0 0) 0x7f27400caf40 con 0x7f274006a8a0 2023-10-16T09:59:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 0x7f274015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f272c0035b0 con 0x7f274006a8a0 2023-10-16T09:59:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124778793 0 0) 0x7f272c0035b0 con 0x7f274006a8a0 2023-10-16T09:59:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 msgr2=0x7f274015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 0x7f274015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 >> v1:172.21.15.190:6789/0 conn(0x7f27400cf5c0 legacy=0x7f27400cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274015ff60 con 0x7f274006a8a0 2023-10-16T09:59:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f27280017f0 con 0x7f274006a8a0 2023-10-16T09:59:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 >> v1:172.21.15.181:6789/0 conn(0x7f274006a8a0 legacy=0x7f274006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f272c003a30 0x7f272c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 0x7f272c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f272c004d90 0x7f272c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274015bd80 con 0x7f272c004d90 2023-10-16T09:59:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c00bb50 con 0x7f272c003a30 2023-10-16T09:59:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c00bd10 con 0x7f27400caaf0 2023-10-16T09:59:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f272c004d90 0x7f272c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273effd700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 0x7f272c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273f7fe700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f272c003a30 0x7f272c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 -- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 msgr2=0x7f272c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 0x7f272c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 -- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f272c003a30 msgr2=0x7f272c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f272c003a30 0x7f272c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 -- 172.21.15.181:0/2843582666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0035b0 con 0x7f272c004d90 2023-10-16T09:59:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273effd700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400caaf0 0x7f272c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273f7fe700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f272c003a30 0x7f272c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.945+0000 7f273ffff700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f272c004d90 0x7f272c005210 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2730009b30 tx=0x7f272c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48155545c0ce8720 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000e6c0 con 0x7f272c004d90 2023-10-16T09:59:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273000e820 con 0x7f272c004d90 2023-10-16T09:59:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f273e7fc700 1 -- 172.21.15.181:0/2843582666 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000ebb0 con 0x7f272c004d90 2023-10-16T09:59:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f2746154700 1 -- 172.21.15.181:0/2843582666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f272c004d90 msgr2=0x7f272c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f2746154700 1 --2- 172.21.15.181:0/2843582666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f272c004d90 0x7f272c005210 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2730009b30 tx=0x7f272c005880 comp rx=0 tx=0).stop 2023-10-16T09:59:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f273ffff700 1 -- 172.21.15.181:0/2843582666 reap_dead start 2023-10-16T09:59:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f2746154700 1 -- 172.21.15.181:0/2843582666 shutdown_connections 2023-10-16T09:59:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f2746154700 1 -- 172.21.15.181:0/2843582666 >> 172.21.15.181:0/2843582666 conn(0x7f27400d04a0 msgr2=0x7f27400d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f2746154700 1 -- 172.21.15.181:0/2843582666 shutdown_connections 2023-10-16T09:59:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.946+0000 7f2746154700 1 -- 172.21.15.181:0/2843582666 wait complete. 2023-10-16T09:59:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.947+0000 7f2746154700 1 Processor -- start 2023-10-16T09:59:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f2746154700 1 -- start start 2023-10-16T09:59:11.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f2746154700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27401f08b0 0x7f27400ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f2746154700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400ae370 0x7f274015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f2746154700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f274015e8b0 0x7f27400c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f2746154700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27400ae7f0 con 0x7f274015e8b0 2023-10-16T09:59:11.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f2746154700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274015ed30 con 0x7f27400ae370 2023-10-16T09:59:11.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f2746154700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2740158700 con 0x7f27401f08b0 2023-10-16T09:59:11.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f273ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f274015e8b0 0x7f27400c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f273ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f274015e8b0 0x7f27400c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41368/0 (socket says 172.21.15.181:41368) 2023-10-16T09:59:11.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f273ffff700 1 -- 172.21.15.181:0/1263966864 learned_addr learned my addr 172.21.15.181:0/1263966864 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:11.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f273effd700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400ae370 0x7f274015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.948+0000 7f273f7fe700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27401f08b0 0x7f27400ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f273ffff700 1 -- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27401f08b0 msgr2=0x7f27400ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:11.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f273ffff700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27401f08b0 0x7f27400ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f273ffff700 1 -- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400ae370 msgr2=0x7f274015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f273ffff700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400ae370 0x7f274015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f273ffff700 1 -- 172.21.15.181:0/1263966864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274015ef50 con 0x7f274015e8b0 2023-10-16T09:59:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f273ffff700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f274015e8b0 0x7f27400c4d90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f272c0047b0 tx=0x7f27300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000e6c0 con 0x7f274015e8b0 2023-10-16T09:59:11.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2730013bb0 con 0x7f274015e8b0 2023-10-16T09:59:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27400c3fd0 con 0x7f274015e8b0 2023-10-16T09:59:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273001c450 con 0x7f274015e8b0 2023-10-16T09:59:11.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.949+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27400c4590 con 0x7f274015e8b0 2023-10-16T09:59:11.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.950+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27400ae750 con 0x7f274015e8b0 2023-10-16T09:59:11.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.950+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 30) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f273001c5b0 con 0x7f274015e8b0 2023-10-16T09:59:11.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.951+0000 7f2737fff700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f272006ad50 0x7f272006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:11.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.951+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6965+0+0 (secure 0 0 0) 0x7f2730000e40 con 0x7f274015e8b0 2023-10-16T09:59:11.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.951+0000 7f273f7fe700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f272006ad50 0x7f272006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:11.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.951+0000 7f273f7fe700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f272006ad50 0x7f272006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2728000ef0 tx=0x7f2728009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:11.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:11.955+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f27300558e0 con 0x7f274015e8b0 2023-10-16T09:59:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.147+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2740158f30 con 0x7f274015e8b0 2023-10-16T09:59:12.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.652+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 31) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2730055cb0 con 0x7f274015e8b0 2023-10-16T09:59:12.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.805+0000 7f2737fff700 1 -- 172.21.15.181:0/1263966864 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2730070500 con 0x7f274015e8b0 2023-10-16T09:59:12.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.808+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f272006ad50 msgr2=0x7f272006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:12.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.808+0000 7f2746154700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f272006ad50 0x7f272006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2728000ef0 tx=0x7f2728009040 comp rx=0 tx=0).stop 2023-10-16T09:59:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.808+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f274015e8b0 msgr2=0x7f27400c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.808+0000 7f2746154700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f274015e8b0 0x7f27400c4d90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f272c0047b0 tx=0x7f27300047c0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.808+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 shutdown_connections 2023-10-16T09:59:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.808+0000 7f2746154700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f272006ad50 0x7f272006d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.809+0000 7f2746154700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27401f08b0 0x7f27400ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.809+0000 7f2746154700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f274015e8b0 0x7f27400c4d90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.809+0000 7f2746154700 1 --2- 172.21.15.181:0/1263966864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400ae370 0x7f274015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.809+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 >> 172.21.15.181:0/1263966864 conn(0x7f27400d04a0 msgr2=0x7f27400cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:12.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.809+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 shutdown_connections 2023-10-16T09:59:12.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.809+0000 7f2746154700 1 -- 172.21.15.181:0/1263966864 wait complete. 2023-10-16T09:59:12.812 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T09:59:12.824 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T09:59:12.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.953+0000 7f96df940700 1 Processor -- start 2023-10-16T09:59:12.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.953+0000 7f96df940700 1 -- start start 2023-10-16T09:59:12.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.954+0000 7f96df940700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d80afa10 0x7f96d80ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:12.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.954+0000 7f96df940700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96d806d870 0x7f96d80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:12.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.954+0000 7f96df940700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80cd850 con 0x7f96d80cd3f0 2023-10-16T09:59:12.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.954+0000 7f96df940700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d815b540 con 0x7f96d806d870 2023-10-16T09:59:12.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.954+0000 7f96df940700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815b820 con 0x7f96d80afa10 2023-10-16T09:59:12.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.954+0000 7f96dd6dc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d80afa10 0x7f96d80ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:12.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.954+0000 7f96dcedb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96d806d870 0x7f96d80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:12.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dcedb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96d806d870 0x7f96d80cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:46422/0 (socket says 172.21.15.181:46422) 2023-10-16T09:59:12.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dd6dc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d80afa10 0x7f96d80ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:38398/0 (socket says 172.21.15.181:38398) 2023-10-16T09:59:12.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dcedb700 1 -- 172.21.15.181:0/2936020812 learned_addr learned my addr 172.21.15.181:0/2936020812 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:12.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dcedb700 1 -- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d80afa10 msgr2=0x7f96d80ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:12.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dcedb700 1 --2- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d80afa10 0x7f96d80ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dcedb700 1 -- 172.21.15.181:0/2936020812 >> v1:172.21.15.190:6789/0 conn(0x7f96d80cd3f0 legacy=0x7f96d80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:12.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dcedb700 1 -- 172.21.15.181:0/2936020812 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d815bb00 con 0x7f96d806d870 2023-10-16T09:59:12.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96cbfff700 1 -- 172.21.15.181:0/2936020812 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934777073 0 0) 0x7f96d80cd850 con 0x7f96d80cd3f0 2023-10-16T09:59:12.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dd6dc700 1 --2- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d80afa10 0x7f96d80ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:12.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96dcedb700 1 --2- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d806d870 0x7f96d80cceb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0098f0 tx=0x7f96cc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f75940efac0b653b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:12.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96cbfff700 1 -- 172.21.15.181:0/2936020812 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc011830 con 0x7f96d806d870 2023-10-16T09:59:12.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96cbfff700 1 -- 172.21.15.181:0/2936020812 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc011990 con 0x7f96d806d870 2023-10-16T09:59:12.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.955+0000 7f96cbfff700 1 -- 172.21.15.181:0/2936020812 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc011d20 con 0x7f96d806d870 2023-10-16T09:59:12.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 -- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d806d870 msgr2=0x7f96d80cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:12.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 --2- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d806d870 0x7f96d80cceb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0098f0 tx=0x7f96cc009c00 comp rx=0 tx=0).stop 2023-10-16T09:59:12.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 -- 172.21.15.181:0/2936020812 shutdown_connections 2023-10-16T09:59:12.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 --2- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d80afa10 0x7f96d80ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 --2- 172.21.15.181:0/2936020812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d806d870 0x7f96d80cceb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 -- 172.21.15.181:0/2936020812 >> 172.21.15.181:0/2936020812 conn(0x7f96d80d04a0 msgr2=0x7f96d80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:12.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 -- 172.21.15.181:0/2936020812 shutdown_connections 2023-10-16T09:59:12.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.956+0000 7f96df940700 1 -- 172.21.15.181:0/2936020812 wait complete. 2023-10-16T09:59:12.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.957+0000 7f96df940700 1 Processor -- start 2023-10-16T09:59:12.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.957+0000 7f96df940700 1 -- start start 2023-10-16T09:59:12.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96df940700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d806d870 0x7f96d815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:12.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96df940700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d80afa10 0x7f96d814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:12.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96df940700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d80cd3f0 0x7f96d80b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:12.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96df940700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815ceb0 con 0x7f96d80cd3f0 2023-10-16T09:59:12.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96df940700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d814af10 con 0x7f96d80afa10 2023-10-16T09:59:12.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96df940700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815c2e0 con 0x7f96d806d870 2023-10-16T09:59:12.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96ddedd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d80cd3f0 0x7f96d80b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:12.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96ddedd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d80cd3f0 0x7f96d80b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41382/0 (socket says 172.21.15.181:41382) 2023-10-16T09:59:12.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96ddedd700 1 -- 172.21.15.181:0/715854713 learned_addr learned my addr 172.21.15.181:0/715854713 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:12.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96dcedb700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d80afa10 0x7f96d814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:12.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96dd6dc700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d806d870 0x7f96d815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:12.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96ddedd700 1 -- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d806d870 msgr2=0x7f96d815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:12.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96ddedd700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d806d870 0x7f96d815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96ddedd700 1 -- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d80afa10 msgr2=0x7f96d814a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:12.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.958+0000 7f96ddedd700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d80afa10 0x7f96d814a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:12.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.959+0000 7f96ddedd700 1 -- 172.21.15.181:0/715854713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0009a70 con 0x7f96d80cd3f0 2023-10-16T09:59:12.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.959+0000 7f96ddedd700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d80cd3f0 0x7f96d80b4660 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f96d4001eb0 tx=0x7f96d400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:12.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.959+0000 7f96c9ffb700 1 -- 172.21.15.181:0/715854713 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d40054e0 con 0x7f96d80cd3f0 2023-10-16T09:59:12.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.959+0000 7f96c9ffb700 1 -- 172.21.15.181:0/715854713 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4003680 con 0x7f96d80cd3f0 2023-10-16T09:59:12.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.959+0000 7f96c9ffb700 1 -- 172.21.15.181:0/715854713 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4003a30 con 0x7f96d80cd3f0 2023-10-16T09:59:12.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.959+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc009550 con 0x7f96d80cd3f0 2023-10-16T09:59:12.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.959+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d814bf00 con 0x7f96d80cd3f0 2023-10-16T09:59:12.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.960+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d80ccd20 con 0x7f96d80cd3f0 2023-10-16T09:59:12.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.960+0000 7f96c9ffb700 1 -- 172.21.15.181:0/715854713 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 31) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f96d4003b90 con 0x7f96d80cd3f0 2023-10-16T09:59:12.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.961+0000 7f96c9ffb700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f96bc06af90 0x7f96bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:12.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.961+0000 7f96c9ffb700 1 -- 172.21.15.181:0/715854713 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f96d4088ce0 con 0x7f96d80cd3f0 2023-10-16T09:59:12.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.961+0000 7f96dd6dc700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f96bc06af90 0x7f96bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:12.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.961+0000 7f96dd6dc700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f96bc06af90 0x7f96bc06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009fd0 tx=0x7f96d000b040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:12.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:12.965+0000 7f96c9ffb700 1 -- 172.21.15.181:0/715854713 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f96d4053a70 con 0x7f96d80cd3f0 2023-10-16T09:59:13.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.157+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96d8067d20 con 0x7f96d80cd3f0 2023-10-16T09:59:13.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.807+0000 7f96c9ffb700 1 -- 172.21.15.181:0/715854713 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 169+0+0 (secure 0 0 0) 0x7f96d4017020 con 0x7f96d80cd3f0 2023-10-16T09:59:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f96bc06af90 msgr2=0x7f96bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:13.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f96bc06af90 0x7f96bc06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009fd0 tx=0x7f96d000b040 comp rx=0 tx=0).stop 2023-10-16T09:59:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d80cd3f0 msgr2=0x7f96d80b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:13.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d80cd3f0 0x7f96d80b4660 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f96d4001eb0 tx=0x7f96d400c730 comp rx=0 tx=0).stop 2023-10-16T09:59:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 shutdown_connections 2023-10-16T09:59:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f96bc06af90 0x7f96bc06d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96d806d870 0x7f96d815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d80cd3f0 0x7f96d80b4660 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 --2- 172.21.15.181:0/715854713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d80afa10 0x7f96d814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 >> 172.21.15.181:0/715854713 conn(0x7f96d80d04a0 msgr2=0x7f96d80b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:13.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.810+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 shutdown_connections 2023-10-16T09:59:13.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.811+0000 7f96df940700 1 -- 172.21.15.181:0/715854713 wait complete. 2023-10-16T09:59:13.814 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T09:59:13.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T09:59:13.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.949+0000 7f3338c47700 1 Processor -- start 2023-10-16T09:59:13.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.949+0000 7f3338c47700 1 -- start start 2023-10-16T09:59:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.950+0000 7f3338c47700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3334072760 0x7f333406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.950+0000 7f3338c47700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333406b210 con 0x7f333406ade0 2023-10-16T09:59:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.950+0000 7f3338c47700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340cef00 con 0x7f333415da70 2023-10-16T09:59:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.950+0000 7f3338c47700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33340cec20 con 0x7f3334072760 2023-10-16T09:59:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f3332d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f333406ade0 0x7f33341615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52468/0 (socket says 172.21.15.181:52468) 2023-10-16T09:59:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f3332d9d700 1 -- 172.21.15.181:0/968240503 learned_addr learned my addr 172.21.15.181:0/968240503 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f3331d9b700 1 --2- 172.21.15.181:0/968240503 >> v2:172.21.15.190:3300/0 conn(0x7f3334072760 0x7f333406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723964187 0 0) 0x7f333406b210 con 0x7f333406ade0 2023-10-16T09:59:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f331c0035b0 con 0x7f333406ade0 2023-10-16T09:59:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985457569 0 0) 0x7f331c0035b0 con 0x7f333406ade0 2023-10-16T09:59:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 >> v2:172.21.15.190:3300/0 conn(0x7f3334072760 msgr2=0x7f333406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 --2- 172.21.15.181:0/968240503 >> v2:172.21.15.190:3300/0 conn(0x7f3334072760 0x7f333406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 >> v1:172.21.15.190:6789/0 conn(0x7f333415da70 legacy=0x7f33340ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334071c80 con 0x7f333406ade0 2023-10-16T09:59:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f332c0019f0 con 0x7f333406ade0 2023-10-16T09:59:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 >> v1:172.21.15.181:6789/0 conn(0x7f333406ade0 legacy=0x7f33341615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f331c003a00 0x7f331c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c004850 0x7f331c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331c005330 0x7f331c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340cec20 con 0x7f331c005330 2023-10-16T09:59:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c00c200 con 0x7f331c004850 2023-10-16T09:59:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c00c3c0 con 0x7f331c003a00 2023-10-16T09:59:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f3332d9d700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331c005330 0x7f331c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f3331d9b700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c004850 0x7f331c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.951+0000 7f333259c700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f331c003a00 0x7f331c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3332d9d700 1 -- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f331c003a00 msgr2=0x7f331c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:13.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3332d9d700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f331c003a00 0x7f331c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3332d9d700 1 -- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c004850 msgr2=0x7f331c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3332d9d700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c004850 0x7f331c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3332d9d700 1 -- 172.21.15.181:0/968240503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0035b0 con 0x7f331c005330 2023-10-16T09:59:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3332d9d700 1 -- 172.21.15.181:0/968240503 reap_dead start 2023-10-16T09:59:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3332d9d700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331c005330 0x7f331c00ba10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f332c00a800 tx=0x7f331c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f158d9cc3b266450 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:13.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00a2f0 con 0x7f331c005330 2023-10-16T09:59:13.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c012620 con 0x7f331c005330 2023-10-16T09:59:13.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f333159a700 1 -- 172.21.15.181:0/968240503 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0129d0 con 0x7f331c005330 2023-10-16T09:59:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3338c47700 1 -- 172.21.15.181:0/968240503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331c005330 msgr2=0x7f331c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3338c47700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331c005330 0x7f331c00ba10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f332c00a800 tx=0x7f331c005880 comp rx=0 tx=0).stop 2023-10-16T09:59:13.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3338c47700 1 -- 172.21.15.181:0/968240503 shutdown_connections 2023-10-16T09:59:13.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3338c47700 1 --2- 172.21.15.181:0/968240503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331c005330 0x7f331c00ba10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.952+0000 7f3338c47700 1 -- 172.21.15.181:0/968240503 >> 172.21.15.181:0/968240503 conn(0x7f33340cd010 msgr2=0x7f33340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:13.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.953+0000 7f3338c47700 1 -- 172.21.15.181:0/968240503 shutdown_connections 2023-10-16T09:59:13.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.953+0000 7f3338c47700 1 -- 172.21.15.181:0/968240503 wait complete. 2023-10-16T09:59:13.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.954+0000 7f3338c47700 1 Processor -- start 2023-10-16T09:59:13.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.954+0000 7f3338c47700 1 -- start start 2023-10-16T09:59:13.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3338c47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c005330 0x7f3334158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3338c47700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33340ae410 0x7f33340b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3338c47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33340b20d0 0x7f33340b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3338c47700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b2550 con 0x7f33340ae410 2023-10-16T09:59:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3338c47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b51e0 con 0x7f331c005330 2023-10-16T09:59:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3338c47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33340c5610 con 0x7f33340b20d0 2023-10-16T09:59:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33340ae410 0x7f33340b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33340ae410 0x7f33340b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41400/0 (socket says 172.21.15.181:41400) 2023-10-16T09:59:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 -- 172.21.15.181:0/877734223 learned_addr learned my addr 172.21.15.181:0/877734223 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f333259c700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c005330 0x7f3334158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3332d9d700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33340b20d0 0x7f33340b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 -- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33340b20d0 msgr2=0x7f33340b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:13.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33340b20d0 0x7f33340b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 -- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c005330 msgr2=0x7f3334158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c005330 0x7f3334158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.955+0000 7f3331d9b700 1 -- 172.21.15.181:0/877734223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340b5760 con 0x7f33340ae410 2023-10-16T09:59:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.956+0000 7f3331d9b700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33340ae410 0x7f33340b1b90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3324011a00 tx=0x7f3324011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:13.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.956+0000 7f3322ffd700 1 -- 172.21.15.181:0/877734223 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332400f940 con 0x7f33340ae410 2023-10-16T09:59:13.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.956+0000 7f3322ffd700 1 -- 172.21.15.181:0/877734223 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332400faa0 con 0x7f33340ae410 2023-10-16T09:59:13.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.956+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33340b5a40 con 0x7f33340ae410 2023-10-16T09:59:13.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.956+0000 7f3322ffd700 1 -- 172.21.15.181:0/877734223 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332400fe30 con 0x7f33340ae410 2023-10-16T09:59:13.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.956+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33340b6000 con 0x7f33340ae410 2023-10-16T09:59:13.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.957+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33340c53e0 con 0x7f33340ae410 2023-10-16T09:59:13.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.958+0000 7f3322ffd700 1 -- 172.21.15.181:0/877734223 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 31) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3324014440 con 0x7f33340ae410 2023-10-16T09:59:13.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.958+0000 7f3322ffd700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f331406af90 0x7f331406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:13.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.958+0000 7f3322ffd700 1 -- 172.21.15.181:0/877734223 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f3324089ef0 con 0x7f33340ae410 2023-10-16T09:59:13.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.958+0000 7f333259c700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f331406af90 0x7f331406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:13.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.958+0000 7f333259c700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f331406af90 0x7f331406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f3328009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:13.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:13.963+0000 7f3322ffd700 1 -- 172.21.15.181:0/877734223 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3324054e50 con 0x7f33340ae410 2023-10-16T09:59:14.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.158+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f33340b3460 con 0x7f33340ae410 2023-10-16T09:59:14.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.162+0000 7f3322ffd700 1 -- 172.21.15.181:0/877734223 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v19) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3324058470 con 0x7f33340ae410 2023-10-16T09:59:14.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.164+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f331406af90 msgr2=0x7f331406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.164+0000 7f3338c47700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f331406af90 0x7f331406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f3328009040 comp rx=0 tx=0).stop 2023-10-16T09:59:14.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.164+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33340ae410 msgr2=0x7f33340b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.164+0000 7f3338c47700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33340ae410 0x7f33340b1b90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3324011a00 tx=0x7f3324011dc0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.164+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 shutdown_connections 2023-10-16T09:59:14.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.165+0000 7f3338c47700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f331406af90 0x7f331406d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.165+0000 7f3338c47700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33340b20d0 0x7f33340b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.165+0000 7f3338c47700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f331c005330 0x7f3334158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.165+0000 7f3338c47700 1 --2- 172.21.15.181:0/877734223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33340ae410 0x7f33340b1b90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.165+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 >> 172.21.15.181:0/877734223 conn(0x7f33340cd010 msgr2=0x7f33340d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:14.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.165+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 shutdown_connections 2023-10-16T09:59:14.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.165+0000 7f3338c47700 1 -- 172.21.15.181:0/877734223 wait complete. 2023-10-16T09:59:14.178 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T09:59:14.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.306+0000 7f79f3ac2700 1 Processor -- start 2023-10-16T09:59:14.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.306+0000 7f79f3ac2700 1 -- start start 2023-10-16T09:59:14.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f3ac2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec15da70 0x7f79ec0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f3ac2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f79ec0ce6a0 0x7f79ec06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f3ac2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79ec072760 0x7f79ec06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f3ac2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0cef00 con 0x7f79ec072760 2023-10-16T09:59:14.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f3ac2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0cec20 con 0x7f79ec0ce6a0 2023-10-16T09:59:14.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f3ac2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec071c80 con 0x7f79ec15da70 2023-10-16T09:59:14.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f205f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79ec072760 0x7f79ec06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f205f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79ec072760 0x7f79ec06ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41408/0 (socket says 172.21.15.181:41408) 2023-10-16T09:59:14.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f205f700 1 -- 172.21.15.181:0/3025790619 learned_addr learned my addr 172.21.15.181:0/3025790619 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:14.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f105d700 1 --2- 172.21.15.181:0/3025790619 >> v2:172.21.15.190:3300/0 conn(0x7f79ec0ce6a0 0x7f79ec06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f205f700 1 -- 172.21.15.181:0/3025790619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec15da70 msgr2=0x7f79ec0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:14.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.307+0000 7f79f185e700 1 --2- 172.21.15.181:0/3025790619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec15da70 0x7f79ec0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f205f700 1 --2- 172.21.15.181:0/3025790619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec15da70 0x7f79ec0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f205f700 1 -- 172.21.15.181:0/3025790619 >> v2:172.21.15.190:3300/0 conn(0x7f79ec0ce6a0 msgr2=0x7f79ec06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:14.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f205f700 1 --2- 172.21.15.181:0/3025790619 >> v2:172.21.15.190:3300/0 conn(0x7f79ec0ce6a0 0x7f79ec06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f205f700 1 -- 172.21.15.181:0/3025790619 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec071f60 con 0x7f79ec072760 2023-10-16T09:59:14.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f105d700 1 --2- 172.21.15.181:0/3025790619 >> v2:172.21.15.190:3300/0 conn(0x7f79ec0ce6a0 0x7f79ec06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f205f700 1 --2- 172.21.15.181:0/3025790619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec06ade0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0674f0 tx=0x7f79e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87ff0d22d3ad353 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f085c700 1 -- 172.21.15.181:0/3025790619 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400e070 con 0x7f79ec072760 2023-10-16T09:59:14.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f085c700 1 -- 172.21.15.181:0/3025790619 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400b9c0 con 0x7f79ec072760 2023-10-16T09:59:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f085c700 1 -- 172.21.15.181:0/3025790619 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400bd70 con 0x7f79ec072760 2023-10-16T09:59:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f3ac2700 1 -- 172.21.15.181:0/3025790619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 msgr2=0x7f79ec06ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/3025790619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec06ade0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0674f0 tx=0x7f79e400b330 comp rx=0 tx=0).stop 2023-10-16T09:59:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f3ac2700 1 -- 172.21.15.181:0/3025790619 shutdown_connections 2023-10-16T09:59:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/3025790619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec06ade0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/3025790619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec15da70 0x7f79ec0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/3025790619 >> v2:172.21.15.190:3300/0 conn(0x7f79ec0ce6a0 0x7f79ec06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.308+0000 7f79f3ac2700 1 -- 172.21.15.181:0/3025790619 >> 172.21.15.181:0/3025790619 conn(0x7f79ec0cd010 msgr2=0x7f79ec0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:14.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.309+0000 7f79f3ac2700 1 -- 172.21.15.181:0/3025790619 shutdown_connections 2023-10-16T09:59:14.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.309+0000 7f79f3ac2700 1 -- 172.21.15.181:0/3025790619 wait complete. 2023-10-16T09:59:14.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 Processor -- start 2023-10-16T09:59:14.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 -- start start 2023-10-16T09:59:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec0ce6a0 0x7f79ec160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec15da70 0x7f79ec0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1f2c90 con 0x7f79ec072760 2023-10-16T09:59:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec15e540 con 0x7f79ec15da70 2023-10-16T09:59:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.310+0000 7f79f3ac2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec15e820 con 0x7f79ec0ce6a0 2023-10-16T09:59:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec15f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41410/0 (socket says 172.21.15.181:41410) 2023-10-16T09:59:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 -- 172.21.15.181:0/2261286072 learned_addr learned my addr 172.21.15.181:0/2261286072 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:14.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f105d700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec0ce6a0 0x7f79ec160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f205f700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec15da70 0x7f79ec0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 -- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec0ce6a0 msgr2=0x7f79ec160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec0ce6a0 0x7f79ec160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 -- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec15da70 msgr2=0x7f79ec0b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec15da70 0x7f79ec0b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 -- 172.21.15.181:0/2261286072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e4009b30 con 0x7f79ec072760 2023-10-16T09:59:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f105d700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec0ce6a0 0x7f79ec160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f185e700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec15f5c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c7f0 tx=0x7f79dc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:14.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79e27fc700 1 -- 172.21.15.181:0/2261286072 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc011800 con 0x7f79ec072760 2023-10-16T09:59:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79e27fc700 1 -- 172.21.15.181:0/2261286072 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc011960 con 0x7f79ec072760 2023-10-16T09:59:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec15eb00 con 0x7f79ec072760 2023-10-16T09:59:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79e27fc700 1 -- 172.21.15.181:0/2261286072 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc011cf0 con 0x7f79ec072760 2023-10-16T09:59:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.311+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec15f0c0 con 0x7f79ec072760 2023-10-16T09:59:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.312+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec06f6c0 con 0x7f79ec072760 2023-10-16T09:59:14.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.312+0000 7f79e27fc700 1 -- 172.21.15.181:0/2261286072 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f79dc018360 con 0x7f79ec072760 2023-10-16T09:59:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.313+0000 7f79e27fc700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f79d006b020 0x7f79d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.313+0000 7f79e27fc700 1 -- 172.21.15.181:0/2261286072 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f79dc089950 con 0x7f79ec072760 2023-10-16T09:59:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.313+0000 7f79f105d700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f79d006b020 0x7f79d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.313+0000 7f79f105d700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f79d006b020 0x7f79d006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0b61b0 tx=0x7f79e8008040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.317+0000 7f79e27fc700 1 -- 172.21.15.181:0/2261286072 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f79dc054690 con 0x7f79ec072760 2023-10-16T09:59:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.509+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f79ec15bf80 con 0x7f79ec072760 2023-10-16T09:59:14.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.513+0000 7f79e27fc700 1 -- 172.21.15.181:0/2261286072 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f79dc057cb0 con 0x7f79ec072760 2023-10-16T09:59:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f79d006b020 msgr2=0x7f79d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f79d006b020 0x7f79d006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0b61b0 tx=0x7f79e8008040 comp rx=0 tx=0).stop 2023-10-16T09:59:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 msgr2=0x7f79ec15f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec15f5c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c7f0 tx=0x7f79dc00cbb0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 shutdown_connections 2023-10-16T09:59:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f79d006b020 0x7f79d006d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ec15da70 0x7f79ec0b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec072760 0x7f79ec15f5c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 --2- 172.21.15.181:0/2261286072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ec0ce6a0 0x7f79ec160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 >> 172.21.15.181:0/2261286072 conn(0x7f79ec0cd010 msgr2=0x7f79ec06f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 shutdown_connections 2023-10-16T09:59:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.515+0000 7f79f3ac2700 1 -- 172.21.15.181:0/2261286072 wait complete. 2023-10-16T09:59:14.529 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T09:59:14.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.650+0000 7fd636696700 1 Processor -- start 2023-10-16T09:59:14.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.650+0000 7fd636696700 1 -- start start 2023-10-16T09:59:14.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd636696700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6300ce6c0 0x7fd6300cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd636696700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6300cc6a0 0x7fd6300ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd636696700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6300b1f70 con 0x7fd63015f150 2023-10-16T09:59:14.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd636696700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6300b2ae0 con 0x7fd6300cc6a0 2023-10-16T09:59:14.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd636696700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6300b2dc0 con 0x7fd6300ce6c0 2023-10-16T09:59:14.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd634c33700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6300cc6a0 0x7fd6300ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd62ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd63015f150 0x7fd6300cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52496/0 (socket says 172.21.15.181:52496) 2023-10-16T09:59:14.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd634c33700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6300cc6a0 0x7fd6300ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41422/0 (socket says 172.21.15.181:41422) 2023-10-16T09:59:14.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd62ffff700 1 -- 172.21.15.181:0/1746805987 learned_addr learned my addr 172.21.15.181:0/1746805987 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:14.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd62f7fe700 1 --2- 172.21.15.181:0/1746805987 >> v2:172.21.15.190:3300/0 conn(0x7fd6300ce6c0 0x7fd6300cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd634c33700 1 -- 172.21.15.181:0/1746805987 >> v2:172.21.15.190:3300/0 conn(0x7fd6300ce6c0 msgr2=0x7fd6300cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:14.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd634c33700 1 --2- 172.21.15.181:0/1746805987 >> v2:172.21.15.190:3300/0 conn(0x7fd6300ce6c0 0x7fd6300cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd634c33700 1 -- 172.21.15.181:0/1746805987 >> v1:172.21.15.181:6789/0 conn(0x7fd63015f150 legacy=0x7fd6300cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd634c33700 1 -- 172.21.15.181:0/1746805987 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6300cf9d0 con 0x7fd6300cc6a0 2023-10-16T09:59:14.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd62effd700 1 -- 172.21.15.181:0/1746805987 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921328566 0 0) 0x7fd6300b1f70 con 0x7fd63015f150 2023-10-16T09:59:14.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd634c33700 1 --2- 172.21.15.181:0/1746805987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6300cc6a0 0x7fd6300ccad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd63006b210 tx=0x7fd62800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b943836e99c854aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:14.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd62effd700 1 -- 172.21.15.181:0/1746805987 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62800e070 con 0x7fd6300cc6a0 2023-10-16T09:59:14.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd62effd700 1 -- 172.21.15.181:0/1746805987 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62800bbd0 con 0x7fd6300cc6a0 2023-10-16T09:59:14.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.651+0000 7fd62effd700 1 -- 172.21.15.181:0/1746805987 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628012430 con 0x7fd6300cc6a0 2023-10-16T09:59:14.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 -- 172.21.15.181:0/1746805987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6300cc6a0 msgr2=0x7fd6300ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 --2- 172.21.15.181:0/1746805987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6300cc6a0 0x7fd6300ccad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd63006b210 tx=0x7fd62800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:14.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 -- 172.21.15.181:0/1746805987 shutdown_connections 2023-10-16T09:59:14.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 --2- 172.21.15.181:0/1746805987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6300cc6a0 0x7fd6300ccad0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 --2- 172.21.15.181:0/1746805987 >> v2:172.21.15.190:3300/0 conn(0x7fd6300ce6c0 0x7fd6300cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 -- 172.21.15.181:0/1746805987 >> 172.21.15.181:0/1746805987 conn(0x7fd63009a930 msgr2=0x7fd6300d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:14.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 -- 172.21.15.181:0/1746805987 shutdown_connections 2023-10-16T09:59:14.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.652+0000 7fd636696700 1 -- 172.21.15.181:0/1746805987 wait complete. 2023-10-16T09:59:14.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.653+0000 7fd636696700 1 Processor -- start 2023-10-16T09:59:14.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.653+0000 7fd636696700 1 -- start start 2023-10-16T09:59:14.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd636696700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6300cc6a0 0x7fd63014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd636696700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6300ce6c0 0x7fd6300d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd636696700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015f150 0x7fd6300d1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd636696700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6301e60c0 con 0x7fd63015f150 2023-10-16T09:59:14.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd636696700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6300ae980 con 0x7fd6300cc6a0 2023-10-16T09:59:14.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd636696700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd630150010 con 0x7fd6300ce6c0 2023-10-16T09:59:14.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd634c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015f150 0x7fd6300d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd634c33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015f150 0x7fd6300d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41426/0 (socket says 172.21.15.181:41426) 2023-10-16T09:59:14.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd634c33700 1 -- 172.21.15.181:0/2379706913 learned_addr learned my addr 172.21.15.181:0/2379706913 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:14.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd62ffff700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6300cc6a0 0x7fd63014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd62f7fe700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6300ce6c0 0x7fd6300d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd634c33700 1 -- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6300ce6c0 msgr2=0x7fd6300d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.654+0000 7fd634c33700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6300ce6c0 0x7fd6300d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd634c33700 1 -- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6300cc6a0 msgr2=0x7fd63014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd634c33700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6300cc6a0 0x7fd63014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd634c33700 1 -- 172.21.15.181:0/2379706913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd628009a70 con 0x7fd63015f150 2023-10-16T09:59:14.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd634c33700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015f150 0x7fd6300d1910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd63006b3a0 tx=0x7fd628004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:14.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd62cff9700 1 -- 172.21.15.181:0/2379706913 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628012430 con 0x7fd63015f150 2023-10-16T09:59:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd62cff9700 1 -- 172.21.15.181:0/2379706913 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628020070 con 0x7fd63015f150 2023-10-16T09:59:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd62cff9700 1 -- 172.21.15.181:0/2379706913 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280098f0 con 0x7fd63015f150 2023-10-16T09:59:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6301e4e30 con 0x7fd63015f150 2023-10-16T09:59:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.655+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6300ae330 con 0x7fd63015f150 2023-10-16T09:59:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.656+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd630067d20 con 0x7fd63015f150 2023-10-16T09:59:14.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.656+0000 7fd62cff9700 1 -- 172.21.15.181:0/2379706913 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd628004040 con 0x7fd63015f150 2023-10-16T09:59:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.657+0000 7fd62cff9700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd61c06b020 0x7fd61c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.657+0000 7fd62cff9700 1 -- 172.21.15.181:0/2379706913 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fd628022070 con 0x7fd63015f150 2023-10-16T09:59:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.657+0000 7fd62ffff700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd61c06b020 0x7fd61c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.657+0000 7fd62ffff700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd61c06b020 0x7fd61c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd624007fd0 tx=0x7fd624007de0 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:14.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.661+0000 7fd62cff9700 1 -- 172.21.15.181:0/2379706913 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6280563b0 con 0x7fd63015f150 2023-10-16T09:59:14.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.858+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd6300aeac0 con 0x7fd63015f150 2023-10-16T09:59:14.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.861+0000 7fd62cff9700 1 -- 172.21.15.181:0/2379706913 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd6280599d0 con 0x7fd63015f150 2023-10-16T09:59:14.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd61c06b020 msgr2=0x7fd61c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd61c06b020 0x7fd61c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd624007fd0 tx=0x7fd624007de0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015f150 msgr2=0x7fd6300d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:14.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015f150 0x7fd6300d1910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd63006b3a0 tx=0x7fd628004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 shutdown_connections 2023-10-16T09:59:14.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd61c06b020 0x7fd61c06d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015f150 0x7fd6300d1910 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6300cc6a0 0x7fd63014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 --2- 172.21.15.181:0/2379706913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6300ce6c0 0x7fd6300d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:14.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 >> 172.21.15.181:0/2379706913 conn(0x7fd63009a930 msgr2=0x7fd6300d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:14.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 shutdown_connections 2023-10-16T09:59:14.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.864+0000 7fd636696700 1 -- 172.21.15.181:0/2379706913 wait complete. 2023-10-16T09:59:14.876 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-16T09:59:14.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.996+0000 7f5cf395f700 1 Processor -- start 2023-10-16T09:59:14.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.996+0000 7f5cf395f700 1 -- start start 2023-10-16T09:59:15.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5cf395f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 0x7f5cec0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5cf395f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec06f870 con 0x7f5cec0cd470 2023-10-16T09:59:15.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5cf395f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec0d1160 con 0x7f5cec15dfd0 2023-10-16T09:59:15.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5cf395f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0cc8e0 con 0x7f5cec0af450 2023-10-16T09:59:15.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5cf1efc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5cec0cd470 0x7f5cec06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52514/0 (socket says 172.21.15.181:52514) 2023-10-16T09:59:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5cf1efc700 1 -- 172.21.15.181:0/3072811690 learned_addr learned my addr 172.21.15.181:0/3072811690 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996188791 0 0) 0x7f5cec06f870 con 0x7f5cec0cd470 2023-10-16T09:59:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd80035b0 con 0x7f5cec0cd470 2023-10-16T09:59:15.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 0x7f5cec0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813558750 0 0) 0x7f5cd80035b0 con 0x7f5cec0cd470 2023-10-16T09:59:15.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 msgr2=0x7f5cec0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:15.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 0x7f5cec0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 >> v1:172.21.15.190:6789/0 conn(0x7f5cec15dfd0 legacy=0x7f5cec06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0ccbc0 con 0x7f5cec0cd470 2023-10-16T09:59:15.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.997+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5ce40019f0 con 0x7f5cec0cd470 2023-10-16T09:59:15.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 >> v1:172.21.15.181:6789/0 conn(0x7f5cec0cd470 legacy=0x7f5cec06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8003a30 0x7f5cd8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004850 0x7f5cd8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 0x7f5cd8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0cc8e0 con 0x7f5cd8004850 2023-10-16T09:59:15.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd800bb40 con 0x7f5cd8003a30 2023-10-16T09:59:15.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd800bd00 con 0x7f5cec0af450 2023-10-16T09:59:15.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004850 0x7f5cd8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf16fb700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8003a30 0x7f5cd8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf1efc700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 0x7f5cd8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf0efa700 1 -- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 msgr2=0x7f5cd8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:15.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec0af450 0x7f5cd8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf0efa700 1 -- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8003a30 msgr2=0x7f5cd8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:15.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8003a30 0x7f5cd8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf0efa700 1 -- 172.21.15.181:0/3072811690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80035b0 con 0x7f5cd8004850 2023-10-16T09:59:15.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004850 0x7f5cd8004cd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009b60 tx=0x7f5cd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcccb925d5f21b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:15.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00e6c0 con 0x7f5cd8004850 2023-10-16T09:59:15.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc00e820 con 0x7f5cd8004850 2023-10-16T09:59:15.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.998+0000 7f5ce3fff700 1 -- 172.21.15.181:0/3072811690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00eb90 con 0x7f5cd8004850 2023-10-16T09:59:15.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.999+0000 7f5cf395f700 1 -- 172.21.15.181:0/3072811690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004850 msgr2=0x7f5cd8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.999+0000 7f5cf395f700 1 --2- 172.21.15.181:0/3072811690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004850 0x7f5cd8004cd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009b60 tx=0x7f5cd8005880 comp rx=0 tx=0).stop 2023-10-16T09:59:15.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.999+0000 7f5cf1efc700 1 -- 172.21.15.181:0/3072811690 reap_dead start 2023-10-16T09:59:15.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.999+0000 7f5cf395f700 1 -- 172.21.15.181:0/3072811690 shutdown_connections 2023-10-16T09:59:15.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.999+0000 7f5cf395f700 1 -- 172.21.15.181:0/3072811690 >> 172.21.15.181:0/3072811690 conn(0x7f5cec0d04a0 msgr2=0x7f5cec0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:15.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.999+0000 7f5cf395f700 1 -- 172.21.15.181:0/3072811690 shutdown_connections 2023-10-16T09:59:15.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:14.999+0000 7f5cf395f700 1 -- 172.21.15.181:0/3072811690 wait complete. 2023-10-16T09:59:15.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.000+0000 7f5cf395f700 1 Processor -- start 2023-10-16T09:59:15.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.000+0000 7f5cf395f700 1 -- start start 2023-10-16T09:59:15.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf395f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8004850 0x7f5cec155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf395f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec155b00 0x7f5cec0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf395f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec1563d0 0x7f5cec156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf395f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec156d90 con 0x7f5cec155b00 2023-10-16T09:59:15.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf395f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0c95d0 con 0x7f5cd8004850 2023-10-16T09:59:15.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf395f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0c98b0 con 0x7f5cec1563d0 2023-10-16T09:59:15.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf0efa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec155b00 0x7f5cec0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.001+0000 7f5cf0efa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec155b00 0x7f5cec0c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41446/0 (socket says 172.21.15.181:41446) 2023-10-16T09:59:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf0efa700 1 -- 172.21.15.181:0/2243461732 learned_addr learned my addr 172.21.15.181:0/2243461732 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf16fb700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8004850 0x7f5cec155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf1efc700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec1563d0 0x7f5cec156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf0efa700 1 -- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec1563d0 msgr2=0x7f5cec156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec1563d0 0x7f5cec156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf0efa700 1 -- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8004850 msgr2=0x7f5cec155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8004850 0x7f5cec155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf0efa700 1 -- 172.21.15.181:0/2243461732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0c9b90 con 0x7f5cec155b00 2023-10-16T09:59:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf1efc700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec1563d0 0x7f5cec156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf0efa700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec155b00 0x7f5cec0c8ca0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800bb10 tx=0x7f5cdc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5ce1ffb700 1 -- 172.21.15.181:0/2243461732 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00e6c0 con 0x7f5cec155b00 2023-10-16T09:59:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5ce1ffb700 1 -- 172.21.15.181:0/2243461732 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc003ab0 con 0x7f5cec155b00 2023-10-16T09:59:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec0c9e70 con 0x7f5cec155b00 2023-10-16T09:59:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5ce1ffb700 1 -- 172.21.15.181:0/2243461732 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc003e60 con 0x7f5cec155b00 2023-10-16T09:59:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.002+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1557c0 con 0x7f5cec155b00 2023-10-16T09:59:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.003+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec0ca390 con 0x7f5cec155b00 2023-10-16T09:59:15.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.004+0000 7f5ce1ffb700 1 -- 172.21.15.181:0/2243461732 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5cdc013470 con 0x7f5cec155b00 2023-10-16T09:59:15.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.004+0000 7f5ce1ffb700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f5cd006b050 0x7f5cd006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.004+0000 7f5ce1ffb700 1 -- 172.21.15.181:0/2243461732 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f5cdc08b3f0 con 0x7f5cec155b00 2023-10-16T09:59:15.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.004+0000 7f5cf16fb700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f5cd006b050 0x7f5cd006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.005+0000 7f5cf16fb700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f5cd006b050 0x7f5cd006d510 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80018b0 tx=0x7f5ce8009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:15.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.009+0000 7f5ce1ffb700 1 -- 172.21.15.181:0/2243461732 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5cec0ca390 con 0x7f5cec155b00 2023-10-16T09:59:15.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.282+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f5cec0ca390 con 0x7f5cec155b00 2023-10-16T09:59:15.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.814+0000 7f5ce1ffb700 1 -- 172.21.15.181:0/2243461732 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v546) v1 ==== 154+0+0 (secure 0 0 0) 0x7f5cdc0563e0 con 0x7f5cec155b00 2023-10-16T09:59:15.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.817+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f5cd006b050 msgr2=0x7f5cd006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.817+0000 7f5cf395f700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f5cd006b050 0x7f5cd006d510 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80018b0 tx=0x7f5ce8009040 comp rx=0 tx=0).stop 2023-10-16T09:59:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.817+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec155b00 msgr2=0x7f5cec0c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.817+0000 7f5cf395f700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec155b00 0x7f5cec0c8ca0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800bb10 tx=0x7f5cdc0047c0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.817+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 shutdown_connections 2023-10-16T09:59:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.818+0000 7f5cf395f700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f5cd006b050 0x7f5cd006d510 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.818+0000 7f5cf395f700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cec1563d0 0x7f5cec156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.818+0000 7f5cf395f700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8004850 0x7f5cec155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.818+0000 7f5cf395f700 1 --2- 172.21.15.181:0/2243461732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec155b00 0x7f5cec0c8ca0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.818+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 >> 172.21.15.181:0/2243461732 conn(0x7f5cec0d04a0 msgr2=0x7f5cec06d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.818+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 shutdown_connections 2023-10-16T09:59:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.818+0000 7f5cf395f700 1 -- 172.21.15.181:0/2243461732 wait complete. 2023-10-16T09:59:15.833 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-16T09:59:15.833 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-16T09:59:15.834 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T09:59:15.835 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:15.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.960+0000 7ff7b9a68700 1 Processor -- start 2023-10-16T09:59:15.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.960+0000 7ff7b9a68700 1 -- start start 2023-10-16T09:59:15.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b9a68700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b9a68700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7b40d0350 0x7ff7b40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b9a68700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b40cc470 con 0x7ff7b40d2390 2023-10-16T09:59:15.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b9a68700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b406ca30 con 0x7ff7b40d0350 2023-10-16T09:59:15.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b9a68700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b406cd10 con 0x7ff7b40ce7f0 2023-10-16T09:59:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7b40d0350 0x7ff7b40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7b40d0350 0x7ff7b40cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41452/0 (socket says 172.21.15.181:41452) 2023-10-16T09:59:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.961+0000 7ff7b3fff700 1 -- 172.21.15.181:0/3314043820 learned_addr learned my addr 172.21.15.181:0/3314043820 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b37fe700 1 --2- 172.21.15.181:0/3314043820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b3fff700 1 -- 172.21.15.181:0/3314043820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 msgr2=0x7ff7b406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b3fff700 1 --2- 172.21.15.181:0/3314043820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b406c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b3fff700 1 -- 172.21.15.181:0/3314043820 >> v1:172.21.15.190:6789/0 conn(0x7ff7b40d2390 legacy=0x7ff7b40cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b3fff700 1 -- 172.21.15.181:0/3314043820 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b406cff0 con 0x7ff7b40d0350 2023-10-16T09:59:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b3fff700 1 --2- 172.21.15.181:0/3314043820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d0350 0x7ff7b40cc930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009e50 tx=0x7ff7ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b660d5a6455ebe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b27fc700 1 -- 172.21.15.181:0/3314043820 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00e070 con 0x7ff7b40d0350 2023-10-16T09:59:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b27fc700 1 -- 172.21.15.181:0/3314043820 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac00b9c0 con 0x7ff7b40d0350 2023-10-16T09:59:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b27fc700 1 -- 172.21.15.181:0/3314043820 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00bd70 con 0x7ff7b40d0350 2023-10-16T09:59:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b9a68700 1 -- 172.21.15.181:0/3314043820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d0350 msgr2=0x7ff7b40cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.962+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/3314043820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d0350 0x7ff7b40cc930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009e50 tx=0x7ff7ac00b330 comp rx=0 tx=0).stop 2023-10-16T09:59:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.963+0000 7ff7b9a68700 1 -- 172.21.15.181:0/3314043820 shutdown_connections 2023-10-16T09:59:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.963+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/3314043820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d0350 0x7ff7b40cc930 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.963+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/3314043820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b406c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.963+0000 7ff7b9a68700 1 -- 172.21.15.181:0/3314043820 >> 172.21.15.181:0/3314043820 conn(0x7ff7b409a9a0 msgr2=0x7ff7b40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.963+0000 7ff7b9a68700 1 -- 172.21.15.181:0/3314043820 shutdown_connections 2023-10-16T09:59:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.963+0000 7ff7b9a68700 1 -- 172.21.15.181:0/3314043820 wait complete. 2023-10-16T09:59:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.964+0000 7ff7b9a68700 1 Processor -- start 2023-10-16T09:59:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.964+0000 7ff7b9a68700 1 -- start start 2023-10-16T09:59:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b409b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7b40d0350 0x7ff7b414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d2390 0x7ff7b4150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b415df80 con 0x7ff7b40d2390 2023-10-16T09:59:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4150970 con 0x7ff7b40d0350 2023-10-16T09:59:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b414f390 con 0x7ff7b40ce7f0 2023-10-16T09:59:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d2390 0x7ff7b4150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d2390 0x7ff7b4150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41458/0 (socket says 172.21.15.181:41458) 2023-10-16T09:59:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 -- 172.21.15.181:0/670648053 learned_addr learned my addr 172.21.15.181:0/670648053 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b37fe700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b409b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b2ffd700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7b40d0350 0x7ff7b414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 -- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 msgr2=0x7ff7b409b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b409b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 -- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7b40d0350 msgr2=0x7ff7b414e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7b40d0350 0x7ff7b414e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 -- 172.21.15.181:0/670648053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac009b30 con 0x7ff7b40d2390 2023-10-16T09:59:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b3fff700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d2390 0x7ff7b4150430 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac012000 tx=0x7ff7ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff79bfff700 1 -- 172.21.15.181:0/670648053 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004800 con 0x7ff7b40d2390 2023-10-16T09:59:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff79bfff700 1 -- 172.21.15.181:0/670648053 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac004960 con 0x7ff7b40d2390 2023-10-16T09:59:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b414f670 con 0x7ff7b40d2390 2023-10-16T09:59:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff79bfff700 1 -- 172.21.15.181:0/670648053 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac01b670 con 0x7ff7b40d2390 2023-10-16T09:59:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.965+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b414fc30 con 0x7ff7b40d2390 2023-10-16T09:59:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.966+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b40d10d0 con 0x7ff7b40d2390 2023-10-16T09:59:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.967+0000 7ff79bfff700 1 -- 172.21.15.181:0/670648053 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff7ac01b7d0 con 0x7ff7b40d2390 2023-10-16T09:59:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.967+0000 7ff79bfff700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7ff7a006b020 0x7ff7a006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.967+0000 7ff79bfff700 1 -- 172.21.15.181:0/670648053 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7ff7ac08b200 con 0x7ff7b40d2390 2023-10-16T09:59:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.967+0000 7ff7b37fe700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7ff7a006b020 0x7ff7a006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.968+0000 7ff7b37fe700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7ff7a006b020 0x7ff7a006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8006fd0 tx=0x7ff7a8008040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:15.972+0000 7ff79bfff700 1 -- 172.21.15.181:0/670648053 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff7ac055f20 con 0x7ff7b40d2390 2023-10-16T09:59:16.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.165+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7b4067d20 con 0x7ff7b40d2390 2023-10-16T09:59:16.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.165+0000 7ff79bfff700 1 -- 172.21.15.181:0/670648053 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+14685 (secure 0 0 0) 0x7ff7ac059540 con 0x7ff7b40d2390 2023-10-16T09:59:16.166 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:16.167 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:13.805713+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6832/2407452630":"2023-10-17T09:59:10.801439+0000","172.21.15.190:6834/2407452630":"2023-10-17T09:59:10.801439+0000","172.21.15.181:0/2979005927":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/980850107":"2023-10-17T09:59:09.633143+0000","172.21.15.181:6833/3708044156":"2023-10-17T09:59:09.633143+0000","172.21.15.181:6832/3708044156":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/2177777009":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/1842196234":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/406774785":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/3225128684":"2023-10-17T09:59:09.633143+0000","172.21.15.181:0/2507249913":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2125399270":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2970752475":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/3339446951":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/2864335478":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6833/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:6832/3219577148":"2023-10-17T09:57:15.720457+0000","172.21.15.181:0/534300178":"2023-10-17T09:57:15.720457+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.167+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7ff7a006b020 msgr2=0x7ff7a006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.167+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7ff7a006b020 0x7ff7a006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8006fd0 tx=0x7ff7a8008040 comp rx=0 tx=0).stop 2023-10-16T09:59:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.167+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d2390 msgr2=0x7ff7b4150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.167+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d2390 0x7ff7b4150430 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac012000 tx=0x7ff7ac004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 shutdown_connections 2023-10-16T09:59:16.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7ff7a006b020 0x7ff7a006d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7b40ce7f0 0x7ff7b409b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b40d2390 0x7ff7b4150430 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 --2- 172.21.15.181:0/670648053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7b40d0350 0x7ff7b414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 >> 172.21.15.181:0/670648053 conn(0x7ff7b409a9a0 msgr2=0x7ff7b40705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:16.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 shutdown_connections 2023-10-16T09:59:16.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.168+0000 7ff7b9a68700 1 -- 172.21.15.181:0/670648053 wait complete. 2023-10-16T09:59:16.180 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:16.181 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:16.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.304+0000 7f6951459700 1 Processor -- start 2023-10-16T09:59:16.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.304+0000 7f6951459700 1 -- start start 2023-10-16T09:59:16.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f6951459700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f694c0d01d0 0x7f694c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f6951459700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f694c15ea40 0x7f694c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f6951459700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c0cf580 con 0x7f694c161530 2023-10-16T09:59:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f6951459700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c0d4290 con 0x7f694c15ea40 2023-10-16T09:59:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f6951459700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c0d0af0 con 0x7f694c0d01d0 2023-10-16T09:59:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f694c15ea40 0x7f694c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f694c15ea40 0x7f694c0d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41464/0 (socket says 172.21.15.181:41464) 2023-10-16T09:59:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 -- 172.21.15.181:0/2003795535 learned_addr learned my addr 172.21.15.181:0/2003795535 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694affd700 1 --2- 172.21.15.181:0/2003795535 >> v2:172.21.15.190:3300/0 conn(0x7f694c0d01d0 0x7f694c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 -- 172.21.15.181:0/2003795535 >> v2:172.21.15.190:3300/0 conn(0x7f694c0d01d0 msgr2=0x7f694c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 --2- 172.21.15.181:0/2003795535 >> v2:172.21.15.190:3300/0 conn(0x7f694c0d01d0 0x7f694c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 -- 172.21.15.181:0/2003795535 >> v1:172.21.15.190:6789/0 conn(0x7f694c161530 legacy=0x7f694c1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 -- 172.21.15.181:0/2003795535 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0ce970 con 0x7f694c15ea40 2023-10-16T09:59:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f694a7fc700 1 --2- 172.21.15.181:0/2003795535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c15ea40 0x7f694c0d05b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f694c08e050 tx=0x7f6940009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14f56c01a56bab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.305+0000 7f6949ffb700 1 -- 172.21.15.181:0/2003795535 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940011830 con 0x7f694c15ea40 2023-10-16T09:59:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6949ffb700 1 -- 172.21.15.181:0/2003795535 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6940011990 con 0x7f694c15ea40 2023-10-16T09:59:16.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6949ffb700 1 -- 172.21.15.181:0/2003795535 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940011d20 con 0x7f694c15ea40 2023-10-16T09:59:16.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 -- 172.21.15.181:0/2003795535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c15ea40 msgr2=0x7f694c0d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 --2- 172.21.15.181:0/2003795535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c15ea40 0x7f694c0d05b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f694c08e050 tx=0x7f6940009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 -- 172.21.15.181:0/2003795535 shutdown_connections 2023-10-16T09:59:16.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 --2- 172.21.15.181:0/2003795535 >> v2:172.21.15.190:3300/0 conn(0x7f694c0d01d0 0x7f694c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 --2- 172.21.15.181:0/2003795535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c15ea40 0x7f694c0d05b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 -- 172.21.15.181:0/2003795535 >> 172.21.15.181:0/2003795535 conn(0x7f694c09cb60 msgr2=0x7f694c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 -- 172.21.15.181:0/2003795535 shutdown_connections 2023-10-16T09:59:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.306+0000 7f6951459700 1 -- 172.21.15.181:0/2003795535 wait complete. 2023-10-16T09:59:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.307+0000 7f6951459700 1 Processor -- start 2023-10-16T09:59:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f6951459700 1 -- start start 2023-10-16T09:59:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f6951459700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c0d01d0 0x7f694c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f6951459700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f694c15ea40 0x7f694c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f6951459700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f694c161530 0x7f694c161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f6951459700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c1e8190 con 0x7f694c0d01d0 2023-10-16T09:59:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f6951459700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0c8450 con 0x7f694c15ea40 2023-10-16T09:59:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f6951459700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0c7bb0 con 0x7f694c161530 2023-10-16T09:59:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f694affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c0d01d0 0x7f694c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f694affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c0d01d0 0x7f694c15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41468/0 (socket says 172.21.15.181:41468) 2023-10-16T09:59:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f694affd700 1 -- 172.21.15.181:0/3226731938 learned_addr learned my addr 172.21.15.181:0/3226731938 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f694b7fe700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f694c161530 0x7f694c161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.308+0000 7f694a7fc700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f694c15ea40 0x7f694c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f694affd700 1 -- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f694c161530 msgr2=0x7f694c161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f694affd700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f694c161530 0x7f694c161940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f694affd700 1 -- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f694c15ea40 msgr2=0x7f694c0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f694affd700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f694c15ea40 0x7f694c0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f694affd700 1 -- 172.21.15.181:0/3226731938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940009550 con 0x7f694c0d01d0 2023-10-16T09:59:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f694affd700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c0d01d0 0x7f694c15b1b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f693c00e9b0 tx=0x7f693c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f69337fe700 1 -- 172.21.15.181:0/3226731938 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c009710 con 0x7f694c0d01d0 2023-10-16T09:59:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f69337fe700 1 -- 172.21.15.181:0/3226731938 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c00ee50 con 0x7f694c0d01d0 2023-10-16T09:59:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c0c7e90 con 0x7f694c0d01d0 2023-10-16T09:59:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f69337fe700 1 -- 172.21.15.181:0/3226731938 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00ca50 con 0x7f694c0d01d0 2023-10-16T09:59:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.309+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c1e83c0 con 0x7f694c0d01d0 2023-10-16T09:59:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.310+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c1e8730 con 0x7f694c0d01d0 2023-10-16T09:59:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.310+0000 7f69337fe700 1 -- 172.21.15.181:0/3226731938 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f693c00cbb0 con 0x7f694c0d01d0 2023-10-16T09:59:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.311+0000 7f69337fe700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f693806b020 0x7f693806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.311+0000 7f694a7fc700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f693806b020 0x7f693806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.311+0000 7f69337fe700 1 -- 172.21.15.181:0/3226731938 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f693c089870 con 0x7f694c0d01d0 2023-10-16T09:59:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.311+0000 7f694a7fc700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f693806b020 0x7f693806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6940000c00 tx=0x7f6940008030 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.315+0000 7f69337fe700 1 -- 172.21.15.181:0/3226731938 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f693c0546b0 con 0x7f694c0d01d0 2023-10-16T09:59:16.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.504+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f694c04ebb0 con 0x7f694c0d01d0 2023-10-16T09:59:16.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.505+0000 7f69337fe700 1 -- 172.21.15.181:0/3226731938 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f693c0210a0 con 0x7f694c0d01d0 2023-10-16T09:59:16.506 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:16.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f693806b020 msgr2=0x7f693806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f693806b020 0x7f693806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6940000c00 tx=0x7f6940008030 comp rx=0 tx=0).stop 2023-10-16T09:59:16.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c0d01d0 msgr2=0x7f694c15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c0d01d0 0x7f694c15b1b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f693c00e9b0 tx=0x7f693c00ecc0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 shutdown_connections 2023-10-16T09:59:16.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f693806b020 0x7f693806d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f694c161530 0x7f694c161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f694c0d01d0 0x7f694c15b1b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 --2- 172.21.15.181:0/3226731938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f694c15ea40 0x7f694c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 >> 172.21.15.181:0/3226731938 conn(0x7f694c09cb60 msgr2=0x7f694c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:16.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 shutdown_connections 2023-10-16T09:59:16.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.507+0000 7f6951459700 1 -- 172.21.15.181:0/3226731938 wait complete. 2023-10-16T09:59:16.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-16T09:59:16.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.640+0000 7fa3167a6700 1 Processor -- start 2023-10-16T09:59:16.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.640+0000 7fa3167a6700 1 -- start start 2023-10-16T09:59:16.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa3167a6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31015eab0 0x7fa3100d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa3167a6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31006ed00 con 0x7fa31015e6d0 2023-10-16T09:59:16.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa3167a6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3100d02d0 con 0x7fa3100cb0c0 2023-10-16T09:59:16.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa3167a6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31006e0f0 con 0x7fa31015eab0 2023-10-16T09:59:16.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa31015e6d0 0x7fa31015a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52554/0 (socket says 172.21.15.181:52554) 2023-10-16T09:59:16.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30ffff700 1 -- 172.21.15.181:0/1658275889 learned_addr learned my addr 172.21.15.181:0/1658275889 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:16.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30f7fe700 1 --2- 172.21.15.181:0/1658275889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31015eab0 0x7fa3100d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30effd700 1 -- 172.21.15.181:0/1658275889 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271999383 0 0) 0x7fa31006ed00 con 0x7fa31015e6d0 2023-10-16T09:59:16.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30effd700 1 -- 172.21.15.181:0/1658275889 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f80057c0 con 0x7fa31015e6d0 2023-10-16T09:59:16.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30effd700 1 -- 172.21.15.181:0/1658275889 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52498585 0 0) 0x7fa3100d02d0 con 0x7fa3100cb0c0 2023-10-16T09:59:16.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30effd700 1 -- 172.21.15.181:0/1658275889 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa31006ed00 con 0x7fa3100cb0c0 2023-10-16T09:59:16.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30f7fe700 1 -- 172.21.15.181:0/1658275889 >> v1:172.21.15.190:6789/0 conn(0x7fa3100cb0c0 legacy=0x7fa31015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30f7fe700 1 -- 172.21.15.181:0/1658275889 >> v1:172.21.15.181:6789/0 conn(0x7fa31015e6d0 legacy=0x7fa31015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.641+0000 7fa30f7fe700 1 -- 172.21.15.181:0/1658275889 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31006e3d0 con 0x7fa31015eab0 2023-10-16T09:59:16.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa30f7fe700 1 --2- 172.21.15.181:0/1658275889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31015eab0 0x7fa3100d0fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa300009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2dfc254b53de0a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:16.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa30effd700 1 -- 172.21.15.181:0/1658275889 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300011830 con 0x7fa31015eab0 2023-10-16T09:59:16.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa30effd700 1 -- 172.21.15.181:0/1658275889 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300011990 con 0x7fa31015eab0 2023-10-16T09:59:16.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa30effd700 1 -- 172.21.15.181:0/1658275889 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300011d00 con 0x7fa31015eab0 2023-10-16T09:59:16.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa3167a6700 1 -- 172.21.15.181:0/1658275889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31015eab0 msgr2=0x7fa3100d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1658275889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31015eab0 0x7fa3100d0fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa300009b80 comp rx=0 tx=0).stop 2023-10-16T09:59:16.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa3167a6700 1 -- 172.21.15.181:0/1658275889 shutdown_connections 2023-10-16T09:59:16.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1658275889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa31015eab0 0x7fa3100d0fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa3167a6700 1 -- 172.21.15.181:0/1658275889 >> 172.21.15.181:0/1658275889 conn(0x7fa3100d04a0 msgr2=0x7fa3100d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:16.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa3167a6700 1 -- 172.21.15.181:0/1658275889 shutdown_connections 2023-10-16T09:59:16.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.642+0000 7fa3167a6700 1 -- 172.21.15.181:0/1658275889 wait complete. 2023-10-16T09:59:16.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 Processor -- start 2023-10-16T09:59:16.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 -- start start 2023-10-16T09:59:16.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3100cb0c0 0x7fa3100b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31015e6d0 0x7fa3100c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa31015eab0 0x7fa31014f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3101f6810 con 0x7fa31015eab0 2023-10-16T09:59:16.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100b5670 con 0x7fa31015e6d0 2023-10-16T09:59:16.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.644+0000 7fa3167a6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100c4440 con 0x7fa3100cb0c0 2023-10-16T09:59:16.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa31015eab0 0x7fa31014f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa31015eab0 0x7fa31014f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41486/0 (socket says 172.21.15.181:41486) 2023-10-16T09:59:16.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 -- 172.21.15.181:0/1803987069 learned_addr learned my addr 172.21.15.181:0/1803987069 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:16.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa30ffff700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3100cb0c0 0x7fa3100b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa30f7fe700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31015e6d0 0x7fa3100c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 -- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3100cb0c0 msgr2=0x7fa3100b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3100cb0c0 0x7fa3100b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 -- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31015e6d0 msgr2=0x7fa3100c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:16.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31015e6d0 0x7fa3100c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:16.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 -- 172.21.15.181:0/1803987069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000094a0 con 0x7fa31015eab0 2023-10-16T09:59:16.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa314d43700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa31015eab0 0x7fa31014f2d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa30800cbc0 tx=0x7fa30800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:16.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa30cff9700 1 -- 172.21.15.181:0/1803987069 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308003c00 con 0x7fa31015eab0 2023-10-16T09:59:16.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa30cff9700 1 -- 172.21.15.181:0/1803987069 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa308003d60 con 0x7fa31015eab0 2023-10-16T09:59:16.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3100c4720 con 0x7fa31015eab0 2023-10-16T09:59:16.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa30cff9700 1 -- 172.21.15.181:0/1803987069 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30800ab00 con 0x7fa31015eab0 2023-10-16T09:59:16.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.645+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3100c4ce0 con 0x7fa31015eab0 2023-10-16T09:59:16.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.646+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3101f6980 con 0x7fa31015eab0 2023-10-16T09:59:16.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.647+0000 7fa30cff9700 1 -- 172.21.15.181:0/1803987069 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa30800ac60 con 0x7fa31015eab0 2023-10-16T09:59:16.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.647+0000 7fa30cff9700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fa2fc06b020 0x7fa2fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:16.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.647+0000 7fa30cff9700 1 -- 172.21.15.181:0/1803987069 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fa308089870 con 0x7fa31015eab0 2023-10-16T09:59:16.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.647+0000 7fa30ffff700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fa2fc06b020 0x7fa2fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:16.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.648+0000 7fa30ffff700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fa2fc06b020 0x7fa2fc06d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3040010e0 tx=0x7fa304007c70 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:16.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.652+0000 7fa30cff9700 1 -- 172.21.15.181:0/1803987069 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa308054740 con 0x7fa31015eab0 2023-10-16T09:59:16.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:16.923+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fa310067d20 con 0x7fa31015eab0 2023-10-16T09:59:17.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.820+0000 7fa30cff9700 1 -- 172.21.15.181:0/1803987069 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v548) v1 ==== 159+0+0 (secure 0 0 0) 0x7fa308057d60 con 0x7fa31015eab0 2023-10-16T09:59:17.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fa2fc06b020 msgr2=0x7fa2fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:17.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fa2fc06b020 0x7fa2fc06d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3040010e0 tx=0x7fa304007c70 comp rx=0 tx=0).stop 2023-10-16T09:59:17.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa31015eab0 msgr2=0x7fa31014f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:17.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa31015eab0 0x7fa31014f2d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa30800cbc0 tx=0x7fa30800ced0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 shutdown_connections 2023-10-16T09:59:17.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fa2fc06b020 0x7fa2fc06d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3100cb0c0 0x7fa3100b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa31015eab0 0x7fa31014f2d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 --2- 172.21.15.181:0/1803987069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa31015e6d0 0x7fa3100c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 >> 172.21.15.181:0/1803987069 conn(0x7fa3100d04a0 msgr2=0x7fa3100ada20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:17.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.823+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 shutdown_connections 2023-10-16T09:59:17.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.824+0000 7fa3167a6700 1 -- 172.21.15.181:0/1803987069 wait complete. 2023-10-16T09:59:17.839 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T09:59:17.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.966+0000 7f6d23b6d700 1 Processor -- start 2023-10-16T09:59:17.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.966+0000 7f6d23b6d700 1 -- start start 2023-10-16T09:59:17.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.967+0000 7f6d23b6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d1c0cc6e0 0x7f6d1c071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:17.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.967+0000 7f6d23b6d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6d1c0ce680 0x7f6d1c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:17.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.967+0000 7f6d23b6d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0ce1f0 con 0x7f6d1c0b2010 2023-10-16T09:59:17.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.967+0000 7f6d23b6d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0ccb90 con 0x7f6d1c0cc6e0 2023-10-16T09:59:17.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.967+0000 7f6d23b6d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c072120 con 0x7f6d1c0ce680 2023-10-16T09:59:17.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21108700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d1c0cc6e0 0x7f6d1c071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:17.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21909700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6d1c0b2010 0x7f6d1c0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52568/0 (socket says 172.21.15.181:52568) 2023-10-16T09:59:17.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21108700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d1c0cc6e0 0x7f6d1c071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41490/0 (socket says 172.21.15.181:41490) 2023-10-16T09:59:17.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21909700 1 -- 172.21.15.181:0/1226692126 learned_addr learned my addr 172.21.15.181:0/1226692126 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:17.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21108700 1 -- 172.21.15.181:0/1226692126 >> v2:172.21.15.190:3300/0 conn(0x7f6d1c0ce680 msgr2=0x7f6d1c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:17.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21108700 1 --2- 172.21.15.181:0/1226692126 >> v2:172.21.15.190:3300/0 conn(0x7f6d1c0ce680 0x7f6d1c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21108700 1 -- 172.21.15.181:0/1226692126 >> v1:172.21.15.181:6789/0 conn(0x7f6d1c0b2010 legacy=0x7f6d1c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:17.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21108700 1 -- 172.21.15.181:0/1226692126 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0ccda0 con 0x7f6d1c0cc6e0 2023-10-16T09:59:17.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d20907700 1 -- 172.21.15.181:0/1226692126 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630042576 0 0) 0x7f6d1c0ce1f0 con 0x7f6d1c0b2010 2023-10-16T09:59:17.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d21108700 1 --2- 172.21.15.181:0/1226692126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0cc6e0 0x7f6d1c071be0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c071a30 tx=0x7f6d1800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dff36fec52df4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:17.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d20907700 1 -- 172.21.15.181:0/1226692126 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800dc70 con 0x7f6d1c0cc6e0 2023-10-16T09:59:17.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d20907700 1 -- 172.21.15.181:0/1226692126 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1800ddd0 con 0x7f6d1c0cc6e0 2023-10-16T09:59:17.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.968+0000 7f6d20907700 1 -- 172.21.15.181:0/1226692126 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18010690 con 0x7f6d1c0cc6e0 2023-10-16T09:59:17.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 -- 172.21.15.181:0/1226692126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0cc6e0 msgr2=0x7f6d1c071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:17.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/1226692126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0cc6e0 0x7f6d1c071be0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c071a30 tx=0x7f6d1800bdc0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 -- 172.21.15.181:0/1226692126 shutdown_connections 2023-10-16T09:59:17.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/1226692126 >> v2:172.21.15.190:3300/0 conn(0x7f6d1c0ce680 0x7f6d1c0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/1226692126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0cc6e0 0x7f6d1c071be0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 -- 172.21.15.181:0/1226692126 >> 172.21.15.181:0/1226692126 conn(0x7f6d1c0cd010 msgr2=0x7f6d1c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:17.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 -- 172.21.15.181:0/1226692126 shutdown_connections 2023-10-16T09:59:17.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.969+0000 7f6d23b6d700 1 -- 172.21.15.181:0/1226692126 wait complete. 2023-10-16T09:59:17.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.970+0000 7f6d23b6d700 1 Processor -- start 2023-10-16T09:59:17.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.970+0000 7f6d23b6d700 1 -- start start 2023-10-16T09:59:17.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d23b6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0b2010 0x7f6d1c0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:17.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d23b6d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c0cc6e0 0x7f6d1c14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:17.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d23b6d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0ce680 0x7f6d1c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:17.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d23b6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0ca530 con 0x7f6d1c0b2010 2023-10-16T09:59:17.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d23b6d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c15c240 con 0x7f6d1c0ce680 2023-10-16T09:59:17.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d23b6d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c8a90 con 0x7f6d1c0cc6e0 2023-10-16T09:59:17.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d21909700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0b2010 0x7f6d1c0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:17.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d21909700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0b2010 0x7f6d1c0c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41494/0 (socket says 172.21.15.181:41494) 2023-10-16T09:59:17.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.971+0000 7f6d21909700 1 -- 172.21.15.181:0/2743348328 learned_addr learned my addr 172.21.15.181:0/2743348328 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:17.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d21108700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c0cc6e0 0x7f6d1c14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:17.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d2210a700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0ce680 0x7f6d1c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:17.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d21909700 1 -- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c0cc6e0 msgr2=0x7f6d1c14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:17.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d21909700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c0cc6e0 0x7f6d1c14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d21909700 1 -- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0ce680 msgr2=0x7f6d1c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:17.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d21909700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0ce680 0x7f6d1c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:17.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d21909700 1 -- 172.21.15.181:0/2743348328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1800b740 con 0x7f6d1c0b2010 2023-10-16T09:59:17.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d2210a700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0ce680 0x7f6d1c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:17.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d21909700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0b2010 0x7f6d1c0c8550 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c580 tx=0x7f6d0c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:17.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d127fc700 1 -- 172.21.15.181:0/2743348328 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c004c80 con 0x7f6d1c0b2010 2023-10-16T09:59:17.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d127fc700 1 -- 172.21.15.181:0/2743348328 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c004e00 con 0x7f6d1c0b2010 2023-10-16T09:59:17.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c14ef30 con 0x7f6d1c0b2010 2023-10-16T09:59:17.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d127fc700 1 -- 172.21.15.181:0/2743348328 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c007310 con 0x7f6d1c0b2010 2023-10-16T09:59:17.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.972+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c15c750 con 0x7f6d1c0b2010 2023-10-16T09:59:17.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.973+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c0b6510 con 0x7f6d1c0b2010 2023-10-16T09:59:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.973+0000 7f6d127fc700 1 -- 172.21.15.181:0/2743348328 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6d0c007470 con 0x7f6d1c0b2010 2023-10-16T09:59:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.974+0000 7f6d127fc700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6d0006b020 0x7f6d0006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.974+0000 7f6d127fc700 1 -- 172.21.15.181:0/2743348328 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f6d0c0895f0 con 0x7f6d1c0b2010 2023-10-16T09:59:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.974+0000 7f6d21108700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6d0006b020 0x7f6d0006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:17.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.974+0000 7f6d21108700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6d0006b020 0x7f6d0006d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800b2c0 tx=0x7f6d18013000 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:17.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:17.978+0000 7f6d127fc700 1 -- 172.21.15.181:0/2743348328 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6d0c054430 con 0x7f6d1c0b2010 2023-10-16T09:59:18.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.171+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6d1c0cc340 con 0x7f6d1c0b2010 2023-10-16T09:59:18.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.175+0000 7f6d127fc700 1 -- 172.21.15.181:0/2743348328 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d0c057a50 con 0x7f6d1c0b2010 2023-10-16T09:59:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6d0006b020 msgr2=0x7f6d0006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6d0006b020 0x7f6d0006d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800b2c0 tx=0x7f6d18013000 comp rx=0 tx=0).stop 2023-10-16T09:59:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0b2010 msgr2=0x7f6d1c0c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0b2010 0x7f6d1c0c8550 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c580 tx=0x7f6d0c00c940 comp rx=0 tx=0).stop 2023-10-16T09:59:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 shutdown_connections 2023-10-16T09:59:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6d0006b020 0x7f6d0006d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0ce680 0x7f6d1c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1c0b2010 0x7f6d1c0c8550 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 --2- 172.21.15.181:0/2743348328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d1c0cc6e0 0x7f6d1c14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 >> 172.21.15.181:0/2743348328 conn(0x7f6d1c0cd010 msgr2=0x7f6d1c0b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 shutdown_connections 2023-10-16T09:59:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.177+0000 7f6d23b6d700 1 -- 172.21.15.181:0/2743348328 wait complete. 2023-10-16T09:59:18.190 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:59:18.190 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:18.231 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:18.277 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:59:18.277 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:18.277 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:18.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:59:18.296 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:18.296 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:18.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:18.316 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:18.317 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:18.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:59:18.353 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:18.353 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:18.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:18.375 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T09:59:18.375 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.512+0000 7fe3b0c1d700 1 Processor -- start 2023-10-16T09:59:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.512+0000 7fe3b0c1d700 1 -- start start 2023-10-16T09:59:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3b0c1d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3ac15a550 0x7fe3ac0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3b0c1d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ac0cb3f0 con 0x7fe3ac15e1d0 2023-10-16T09:59:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3b0c1d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ac0caac0 con 0x7fe3ac15ed90 2023-10-16T09:59:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3b0c1d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ac06ecb0 con 0x7fe3ac15a550 2023-10-16T09:59:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3aad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe3ac15e1d0 0x7fe3ac0cdfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52584/0 (socket says 172.21.15.181:52584) 2023-10-16T09:59:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3aad9d700 1 -- 172.21.15.181:0/903149223 learned_addr learned my addr 172.21.15.181:0/903149223 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3aa59c700 1 --2- 172.21.15.181:0/903149223 >> v2:172.21.15.190:3300/0 conn(0x7fe3ac15a550 0x7fe3ac0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209864636 0 0) 0x7fe3ac0cb3f0 con 0x7fe3ac15e1d0 2023-10-16T09:59:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3940035b0 con 0x7fe3ac15e1d0 2023-10-16T09:59:18.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011633032 0 0) 0x7fe3ac0caac0 con 0x7fe3ac15ed90 2023-10-16T09:59:18.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3ac0cb3f0 con 0x7fe3ac15ed90 2023-10-16T09:59:18.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790842195 0 0) 0x7fe3940035b0 con 0x7fe3ac15e1d0 2023-10-16T09:59:18.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 >> v2:172.21.15.190:3300/0 conn(0x7fe3ac15a550 msgr2=0x7fe3ac0cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:18.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 --2- 172.21.15.181:0/903149223 >> v2:172.21.15.190:3300/0 conn(0x7fe3ac15a550 0x7fe3ac0cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 >> v1:172.21.15.190:6789/0 conn(0x7fe3ac15ed90 legacy=0x7fe3ac06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0cada0 con 0x7fe3ac15e1d0 2023-10-16T09:59:18.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.513+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe3a40019f0 con 0x7fe3ac15e1d0 2023-10-16T09:59:18.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 >> v1:172.21.15.181:6789/0 conn(0x7fe3ac15e1d0 legacy=0x7fe3ac0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394003b90 0x7fe394003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe394004850 0x7fe394004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe394005210 0x7fe39400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac06ecb0 con 0x7fe394005210 2023-10-16T09:59:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39400c090 con 0x7fe394003b90 2023-10-16T09:59:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39400c250 con 0x7fe394004850 2023-10-16T09:59:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe394005210 0x7fe39400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aa59c700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394003b90 0x7fe394003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a9d9b700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe394004850 0x7fe394004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 -- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe394004850 msgr2=0x7fe394004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe394004850 0x7fe394004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 -- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394003b90 msgr2=0x7fe394003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394003b90 0x7fe394003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 -- 172.21.15.181:0/903149223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3940035b0 con 0x7fe394005210 2023-10-16T09:59:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 -- 172.21.15.181:0/903149223 reap_dead start 2023-10-16T09:59:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3aad9d700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe394005210 0x7fe39400ba10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400a3b0 tx=0x7fe394005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c0ed56c53a92a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:18.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4010690 con 0x7fe394005210 2023-10-16T09:59:18.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a40107f0 con 0x7fe394005210 2023-10-16T09:59:18.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.514+0000 7fe3a959a700 1 -- 172.21.15.181:0/903149223 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4010ba0 con 0x7fe394005210 2023-10-16T09:59:18.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.515+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/903149223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe394005210 msgr2=0x7fe39400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.515+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe394005210 0x7fe39400ba10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400a3b0 tx=0x7fe394005880 comp rx=0 tx=0).stop 2023-10-16T09:59:18.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.515+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/903149223 shutdown_connections 2023-10-16T09:59:18.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.515+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/903149223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe394005210 0x7fe39400ba10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.515+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/903149223 >> 172.21.15.181:0/903149223 conn(0x7fe3ac0d0460 msgr2=0x7fe3ac0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:18.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.515+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/903149223 shutdown_connections 2023-10-16T09:59:18.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.515+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/903149223 wait complete. 2023-10-16T09:59:18.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.516+0000 7fe3b0c1d700 1 Processor -- start 2023-10-16T09:59:18.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.516+0000 7fe3b0c1d700 1 -- start start 2023-10-16T09:59:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3b0c1d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394005210 0x7fe3ac0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3b0c1d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3ac158da0 0x7fe3ac1f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3b0c1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3ac159220 0x7fe3ac1f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3b0c1d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac1f40b0 con 0x7fe3ac159220 2023-10-16T09:59:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3b0c1d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac1f45b0 con 0x7fe394005210 2023-10-16T09:59:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3b0c1d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac15af30 con 0x7fe3ac158da0 2023-10-16T09:59:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3ac159220 0x7fe3ac1f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3ac159220 0x7fe3ac1f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41518/0 (socket says 172.21.15.181:41518) 2023-10-16T09:59:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 -- 172.21.15.181:0/2365403809 learned_addr learned my addr 172.21.15.181:0/2365403809 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3a9d9b700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3ac158da0 0x7fe3ac1f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aa59c700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394005210 0x7fe3ac0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 -- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3ac158da0 msgr2=0x7fe3ac1f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3ac158da0 0x7fe3ac1f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 -- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394005210 msgr2=0x7fe3ac0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394005210 0x7fe3ac0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.517+0000 7fe3aad9d700 1 -- 172.21.15.181:0/2365403809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac1f42d0 con 0x7fe3ac159220 2023-10-16T09:59:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.518+0000 7fe3aad9d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3ac159220 0x7fe3ac1f3b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400ad90 tx=0x7fe3ac0cb360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.518+0000 7fe39affd700 1 -- 172.21.15.181:0/2365403809 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4020070 con 0x7fe3ac159220 2023-10-16T09:59:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.518+0000 7fe39affd700 1 -- 172.21.15.181:0/2365403809 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a40019c0 con 0x7fe3ac159220 2023-10-16T09:59:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.518+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac0adfc0 con 0x7fe3ac159220 2023-10-16T09:59:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.518+0000 7fe39affd700 1 -- 172.21.15.181:0/2365403809 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4011690 con 0x7fe3ac159220 2023-10-16T09:59:18.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.518+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac0ae580 con 0x7fe3ac159220 2023-10-16T09:59:18.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.519+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac067d20 con 0x7fe3ac159220 2023-10-16T09:59:18.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.519+0000 7fe39affd700 1 -- 172.21.15.181:0/2365403809 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe3a40117f0 con 0x7fe3ac159220 2023-10-16T09:59:18.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.519+0000 7fe39affd700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fe38c06b020 0x7fe38c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.519+0000 7fe39affd700 1 -- 172.21.15.181:0/2365403809 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fe3a408a970 con 0x7fe3ac159220 2023-10-16T09:59:18.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.520+0000 7fe3aa59c700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fe38c06b020 0x7fe38c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.520+0000 7fe3aa59c700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fe38c06b020 0x7fe38c06d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009bc0 tx=0x7fe39c0091c0 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:18.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.524+0000 7fe39affd700 1 -- 172.21.15.181:0/2365403809 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe3a4055a20 con 0x7fe3ac159220 2023-10-16T09:59:18.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.759+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3ac0b6240 con 0x7fe3ac159220 2023-10-16T09:59:18.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.760+0000 7fe39affd700 1 -- 172.21.15.181:0/2365403809 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fe3a402be90 con 0x7fe3ac159220 2023-10-16T09:59:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:18.761 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5925,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/47573574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":47573574},{"type":"v1","addr":"172.21.15.181:6836","nonce":47573574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3518254514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3518254514},{"type":"v1","addr":"172.21.15.181:6837","nonce":3518254514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4292050143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":4292050143},{"type":"v1","addr":"172.21.15.190:6835","nonce":4292050143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5986,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3403197292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3403197292},{"type":"v1","addr":"172.21.15.190:6834","nonce":3403197292}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2023-10-16T09:59:18.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fe38c06b020 msgr2=0x7fe38c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fe38c06b020 0x7fe38c06d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009bc0 tx=0x7fe39c0091c0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3ac159220 msgr2=0x7fe3ac1f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3ac159220 0x7fe3ac1f3b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400ad90 tx=0x7fe3ac0cb360 comp rx=0 tx=0).stop 2023-10-16T09:59:18.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 shutdown_connections 2023-10-16T09:59:18.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fe38c06b020 0x7fe38c06d4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3ac159220 0x7fe3ac1f3b70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe394005210 0x7fe3ac0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 --2- 172.21.15.181:0/2365403809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3ac158da0 0x7fe3ac1f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 >> 172.21.15.181:0/2365403809 conn(0x7fe3ac0d0460 msgr2=0x7fe3ac0ad310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:18.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 shutdown_connections 2023-10-16T09:59:18.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.762+0000 7fe3b0c1d700 1 -- 172.21.15.181:0/2365403809 wait complete. 2023-10-16T09:59:18.765 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 27 2023-10-16T09:59:18.775 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T09:59:18.775 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T09:59:18.775 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T09:59:18.776 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:18.776 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T09:59:18.776 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T09:59:18.777 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T09:59:18.777 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:18.777 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T09:59:18.777 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T09:59:18.778 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T09:59:18.778 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:18.778 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T09:59:18.778 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T09:59:18.779 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T09:59:18.779 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:18.784 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T09:59:18.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T09:59:18.786 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T09:59:18.786 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T09:59:18.788 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T09:59:18.789 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T09:59:18.790 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T09:59:18.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T09:59:18.828 INFO:tasks.ceph.mds.a:Started 2023-10-16T09:59:18.829 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:18.830 INFO:tasks.ceph.mds.c:Started 2023-10-16T09:59:18.830 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:18.832 INFO:tasks.ceph.mds.b:Started 2023-10-16T09:59:18.833 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:18.833 INFO:tasks.ceph.mds.d:Started 2023-10-16T09:59:18.834 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:18.834 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T09:59:18.907 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T09:59:18.908 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T09:59:18.918 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T09:59:18.918 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T09:59:18.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.963+0000 7f6365562700 1 Processor -- start 2023-10-16T09:59:18.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.963+0000 7f6365562700 1 -- start start 2023-10-16T09:59:18.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f6365562700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f63600d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f6365562700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63600d24f0 con 0x7f636015bd50 2023-10-16T09:59:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f6365562700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636015c320 con 0x7f63600cfc60 2023-10-16T09:59:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f6365562700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636015ccc0 con 0x7f63600b05d0 2023-10-16T09:59:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f635e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f636015bd50 0x7f63600cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52626/0 (socket says 172.21.15.181:52626) 2023-10-16T09:59:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f635e7fc700 1 -- 172.21.15.181:0/131765418 learned_addr learned my addr 172.21.15.181:0/131765418 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f635f7fe700 1 --2- 172.21.15.181:0/131765418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f63600d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f635f7fe700 1 -- 172.21.15.181:0/131765418 >> v1:172.21.15.190:6789/0 conn(0x7f63600cfc60 legacy=0x7f63600d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f635f7fe700 1 -- 172.21.15.181:0/131765418 >> v1:172.21.15.181:6789/0 conn(0x7f636015bd50 legacy=0x7f63600cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f635f7fe700 1 -- 172.21.15.181:0/131765418 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636015c600 con 0x7f63600b05d0 2023-10-16T09:59:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.964+0000 7f635f7fe700 1 --2- 172.21.15.181:0/131765418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f63600d1fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6358009dc0 tx=0x7f635800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a66db33ebe718378 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:18.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.966+0000 7f635dffb700 1 -- 172.21.15.181:0/131765418 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635800e070 con 0x7f63600b05d0 2023-10-16T09:59:18.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.967+0000 7f635dffb700 1 -- 172.21.15.181:0/131765418 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800bbd0 con 0x7f63600b05d0 2023-10-16T09:59:18.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.967+0000 7f635dffb700 1 -- 172.21.15.181:0/131765418 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358012430 con 0x7f63600b05d0 2023-10-16T09:59:18.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.967+0000 7f6365562700 1 -- 172.21.15.181:0/131765418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 msgr2=0x7f63600d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.967+0000 7f6365562700 1 --2- 172.21.15.181:0/131765418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f63600d1fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6358009dc0 tx=0x7f635800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:18.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.967+0000 7f6365562700 1 -- 172.21.15.181:0/131765418 shutdown_connections 2023-10-16T09:59:18.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.967+0000 7f6365562700 1 --2- 172.21.15.181:0/131765418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f63600d1fb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.967+0000 7f6365562700 1 -- 172.21.15.181:0/131765418 >> 172.21.15.181:0/131765418 conn(0x7f636009a9a0 msgr2=0x7f63600d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:18.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.968+0000 7f6365562700 1 -- 172.21.15.181:0/131765418 shutdown_connections 2023-10-16T09:59:18.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.968+0000 7f6365562700 1 -- 172.21.15.181:0/131765418 wait complete. 2023-10-16T09:59:18.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.969+0000 7f6365562700 1 Processor -- start 2023-10-16T09:59:18.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.969+0000 7f6365562700 1 -- start start 2023-10-16T09:59:18.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f6365562700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f636014cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f6365562700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63600cfc60 0x7f6360156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f6365562700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636015bd50 0x7f636015f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f6365562700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600b1570 con 0x7f63600cfc60 2023-10-16T09:59:18.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f6365562700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360157130 con 0x7f636015bd50 2023-10-16T09:59:18.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f6365562700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636014d010 con 0x7f63600b05d0 2023-10-16T09:59:18.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636015bd50 0x7f636015f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f635effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f636014cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636015bd50 0x7f636015f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:46606/0 (socket says 172.21.15.181:46606) 2023-10-16T09:59:18.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f635e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63600cfc60 0x7f6360156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f635effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f636014cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:38578/0 (socket says 172.21.15.181:38578) 2023-10-16T09:59:18.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.970+0000 7f635f7fe700 1 -- 172.21.15.181:0/3459258115 learned_addr learned my addr 172.21.15.181:0/3459258115 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:18.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f635e7fc700 1 -- 172.21.15.181:0/3459258115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 msgr2=0x7f636014cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f635e7fc700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f636014cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f635e7fc700 1 -- 172.21.15.181:0/3459258115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636015bd50 msgr2=0x7f636015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:18.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f635e7fc700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636015bd50 0x7f636015f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:18.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f635e7fc700 1 -- 172.21.15.181:0/3459258115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358009a70 con 0x7f63600cfc60 2023-10-16T09:59:18.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f635f7fe700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636015bd50 0x7f636015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:18.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f635e7fc700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63600cfc60 0x7f6360156400 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f635400c3f0 tx=0x7f635400c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:18.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f63477fe700 1 -- 172.21.15.181:0/3459258115 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354005b40 con 0x7f63600cfc60 2023-10-16T09:59:18.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f63477fe700 1 -- 172.21.15.181:0/3459258115 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6354005ca0 con 0x7f63600cfc60 2023-10-16T09:59:18.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f63477fe700 1 -- 172.21.15.181:0/3459258115 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63540072f0 con 0x7f63600cfc60 2023-10-16T09:59:18.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6360156e50 con 0x7f63600cfc60 2023-10-16T09:59:18.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.971+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63600b1200 con 0x7f63600cfc60 2023-10-16T09:59:18.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.972+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63600b1390 con 0x7f63600cfc60 2023-10-16T09:59:18.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.973+0000 7f63477fe700 1 -- 172.21.15.181:0/3459258115 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6354007450 con 0x7f63600cfc60 2023-10-16T09:59:18.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.973+0000 7f63477fe700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f634c06b020 0x7f634c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:18.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.973+0000 7f63477fe700 1 -- 172.21.15.181:0/3459258115 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f6354088bb0 con 0x7f63600cfc60 2023-10-16T09:59:18.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.973+0000 7f635effd700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f634c06b020 0x7f634c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:18.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.973+0000 7f635effd700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f634c06b020 0x7f634c06d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6350000f60 tx=0x7f6350009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:18.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:18.977+0000 7f63477fe700 1 -- 172.21.15.181:0/3459258115 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6354053960 con 0x7f63600cfc60 2023-10-16T09:59:19.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.186+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6360067d20 con 0x7f63600cfc60 2023-10-16T09:59:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.975+0000 7f63477fe700 1 -- 172.21.15.181:0/3459258115 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6354017020 con 0x7f63600cfc60 2023-10-16T09:59:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f634c06b020 msgr2=0x7f634c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:19.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f634c06b020 0x7f634c06d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6350000f60 tx=0x7f6350009040 comp rx=0 tx=0).stop 2023-10-16T09:59:19.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63600cfc60 msgr2=0x7f6360156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:19.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63600cfc60 0x7f6360156400 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f635400c3f0 tx=0x7f635400c700 comp rx=0 tx=0).stop 2023-10-16T09:59:19.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 shutdown_connections 2023-10-16T09:59:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f634c06b020 0x7f634c06d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63600b05d0 0x7f636014cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636015bd50 0x7f636015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:19.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 --2- 172.21.15.181:0/3459258115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63600cfc60 0x7f6360156400 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 >> 172.21.15.181:0/3459258115 conn(0x7f636009a9a0 msgr2=0x7f63600d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 shutdown_connections 2023-10-16T09:59:19.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:19.979+0000 7f6365562700 1 -- 172.21.15.181:0/3459258115 wait complete. 2023-10-16T09:59:19.982 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T09:59:19.995 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T09:59:20.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.120+0000 7fc795b9e700 1 Processor -- start 2023-10-16T09:59:20.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.120+0000 7fc795b9e700 1 -- start start 2023-10-16T09:59:20.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc795b9e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc79015db70 0x7fc790161db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc795b9e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7900af820 0x7fc79006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc795b9e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7900af2d0 0x7fc79006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc795b9e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7900cc610 con 0x7fc7900af820 2023-10-16T09:59:20.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc795b9e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7900cc8f0 con 0x7fc79015db70 2023-10-16T09:59:20.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc795b9e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901615b0 con 0x7fc7900af2d0 2023-10-16T09:59:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc78ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7900af820 0x7fc79006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc78ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7900af820 0x7fc79006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41564/0 (socket says 172.21.15.181:41564) 2023-10-16T09:59:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.121+0000 7fc78ffff700 1 -- 172.21.15.181:0/3885588398 learned_addr learned my addr 172.21.15.181:0/3885588398 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc79539d700 1 --2- 172.21.15.181:0/3885588398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7900af2d0 0x7fc79006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78ffff700 1 -- 172.21.15.181:0/3885588398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7900af2d0 msgr2=0x7fc79006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78ffff700 1 --2- 172.21.15.181:0/3885588398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7900af2d0 0x7fc79006ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc794b9c700 1 --2- 172.21.15.181:0/3885588398 >> v2:172.21.15.190:3300/0 conn(0x7fc79015db70 0x7fc790161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78ffff700 1 -- 172.21.15.181:0/3885588398 >> v2:172.21.15.190:3300/0 conn(0x7fc79015db70 msgr2=0x7fc790161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78ffff700 1 --2- 172.21.15.181:0/3885588398 >> v2:172.21.15.190:3300/0 conn(0x7fc79015db70 0x7fc790161db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78ffff700 1 -- 172.21.15.181:0/3885588398 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790161890 con 0x7fc7900af820 2023-10-16T09:59:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78ffff700 1 --2- 172.21.15.181:0/3885588398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc79006a8a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc79006d400 tx=0x7fc77c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca657b746f85ad46 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78f7fe700 1 -- 172.21.15.181:0/3885588398 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00b870 con 0x7fc7900af820 2023-10-16T09:59:20.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78f7fe700 1 -- 172.21.15.181:0/3885588398 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00b9d0 con 0x7fc7900af820 2023-10-16T09:59:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc78f7fe700 1 -- 172.21.15.181:0/3885588398 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00bd60 con 0x7fc7900af820 2023-10-16T09:59:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc795b9e700 1 -- 172.21.15.181:0/3885588398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 msgr2=0x7fc79006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.122+0000 7fc795b9e700 1 --2- 172.21.15.181:0/3885588398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc79006a8a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc79006d400 tx=0x7fc77c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.123+0000 7fc795b9e700 1 -- 172.21.15.181:0/3885588398 shutdown_connections 2023-10-16T09:59:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.123+0000 7fc795b9e700 1 --2- 172.21.15.181:0/3885588398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7900af2d0 0x7fc79006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.123+0000 7fc795b9e700 1 --2- 172.21.15.181:0/3885588398 >> v2:172.21.15.190:3300/0 conn(0x7fc79015db70 0x7fc790161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.123+0000 7fc795b9e700 1 --2- 172.21.15.181:0/3885588398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc79006a8a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.123+0000 7fc795b9e700 1 -- 172.21.15.181:0/3885588398 >> 172.21.15.181:0/3885588398 conn(0x7fc79006c430 msgr2=0x7fc7900d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:20.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.123+0000 7fc795b9e700 1 -- 172.21.15.181:0/3885588398 shutdown_connections 2023-10-16T09:59:20.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.123+0000 7fc795b9e700 1 -- 172.21.15.181:0/3885588398 wait complete. 2023-10-16T09:59:20.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.124+0000 7fc795b9e700 1 Processor -- start 2023-10-16T09:59:20.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.124+0000 7fc795b9e700 1 -- start start 2023-10-16T09:59:20.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc795b9e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7900af2d0 0x7fc790159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc795b9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc7900b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc795b9e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79015db70 0x7fc7900c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc795b9e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900c5bf0 con 0x7fc7900af820 2023-10-16T09:59:20.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc795b9e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900c5610 con 0x7fc7900af2d0 2023-10-16T09:59:20.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc795b9e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79015a8e0 con 0x7fc79015db70 2023-10-16T09:59:20.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc7900b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc7900b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41570/0 (socket says 172.21.15.181:41570) 2023-10-16T09:59:20.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 -- 172.21.15.181:0/688932383 learned_addr learned my addr 172.21.15.181:0/688932383 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:20.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc79539d700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79015db70 0x7fc7900c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc794b9c700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7900af2d0 0x7fc790159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 -- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79015db70 msgr2=0x7fc7900c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79015db70 0x7fc7900c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 -- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7900af2d0 msgr2=0x7fc790159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7900af2d0 0x7fc790159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 -- 172.21.15.181:0/688932383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c009550 con 0x7fc7900af820 2023-10-16T09:59:20.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc794b9c700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7900af2d0 0x7fc790159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:20.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.125+0000 7fc78ffff700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc7900b2fe0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc77c013000 tx=0x7fc77c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.126+0000 7fc78d7fa700 1 -- 172.21.15.181:0/688932383 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004310 con 0x7fc7900af820 2023-10-16T09:59:20.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.126+0000 7fc78d7fa700 1 -- 172.21.15.181:0/688932383 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c01f070 con 0x7fc7900af820 2023-10-16T09:59:20.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.126+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc790162470 con 0x7fc7900af820 2023-10-16T09:59:20.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.126+0000 7fc78d7fa700 1 -- 172.21.15.181:0/688932383 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0036a0 con 0x7fc7900af820 2023-10-16T09:59:20.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.126+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7900c7ed0 con 0x7fc7900af820 2023-10-16T09:59:20.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.127+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7900c5d30 con 0x7fc7900af820 2023-10-16T09:59:20.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.127+0000 7fc78d7fa700 1 -- 172.21.15.181:0/688932383 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc77c021030 con 0x7fc7900af820 2023-10-16T09:59:20.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.127+0000 7fc78d7fa700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc77406b020 0x7fc77406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.128+0000 7fc794b9c700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc77406b020 0x7fc77406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.128+0000 7fc78d7fa700 1 -- 172.21.15.181:0/688932383 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5281+0+0 (secure 0 0 0) 0x7fc77c08a8e0 con 0x7fc7900af820 2023-10-16T09:59:20.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.128+0000 7fc794b9c700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc77406b020 0x7fc77406d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc784006fd0 tx=0x7fc784008040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.132+0000 7fc78d7fa700 1 -- 172.21.15.181:0/688932383 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7900c5d30 con 0x7fc7900af820 2023-10-16T09:59:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.402+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc7900c5d30 con 0x7fc7900af820 2023-10-16T09:59:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.402+0000 7fc78d7fa700 1 -- 172.21.15.181:0/688932383 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc7900c5d30 con 0x7fc7900af820 2023-10-16T09:59:20.403 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:20.403 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:59:20.404 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T09:59:20.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.404 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T09:59:20.404 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T09:59:20.404 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.405 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.405 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:20.405 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:20.405 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.405 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.406 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T09:59:20.406 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T09:59:20.406 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.406 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.407 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:20.407 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:20.407 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.407 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.408 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T09:59:20.408 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T09:59:20.408 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.408 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.408 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:20.408 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:20.409 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.409 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.409 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T09:59:20.409 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T09:59:20.410 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.410 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.410 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:20.410 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:20.410 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.410 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.411 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T09:59:20.411 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T09:59:20.411 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.411 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.412 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.412 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.412 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.412 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T09:59:20.413 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T09:59:20.413 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.413 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.413 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.413 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.414 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.414 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T09:59:20.414 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T09:59:20.414 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.415 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.415 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.415 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.415 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.416 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T09:59:20.416 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T09:59:20.416 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.416 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.416 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.417 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.417 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.417 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T09:59:20.417 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T09:59:20.418 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.418 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.418 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.418 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.418 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.419 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T09:59:20.419 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T09:59:20.419 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.419 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.419 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.420 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.420 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.420 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T09:59:20.420 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T09:59:20.421 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.421 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.421 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.421 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.421 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.422 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T09:59:20.422 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T09:59:20.422 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.422 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:20.423 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:20.423 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.423 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.424 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T09:59:20.424 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T09:59:20.424 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.424 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.424 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T09:59:20.424 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T09:59:20.425 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.425 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T09:59:20.425 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T09:59:20.426 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.426 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.426 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:59:20.426 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:59:20.426 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:59:20.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.427 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T09:59:20.427 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T09:59:20.427 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:59:20.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:59:20.428 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:59:20.428 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.429 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T09:59:20.429 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T09:59:20.429 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.429 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:20.429 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:59:20.430 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:59:20.430 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:59:20.430 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.430 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T09:59:20.431 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T09:59:20.431 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:59:20.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:20.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:20.432 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.432 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T09:59:20.432 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T09:59:20.433 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:59:20.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:59:20.433 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.433 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:20.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:20.434 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T09:59:20.434 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T09:59:20.434 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:20.434 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:59:20.434 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:59:20.435 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:20.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:20.435 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T09:59:20.435 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:59:20.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc77406b020 msgr2=0x7fc77406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc77406b020 0x7fc77406d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc784006fd0 tx=0x7fc784008040 comp rx=0 tx=0).stop 2023-10-16T09:59:20.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 msgr2=0x7fc7900b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc7900b2fe0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc77c013000 tx=0x7fc77c004750 comp rx=0 tx=0).stop 2023-10-16T09:59:20.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 shutdown_connections 2023-10-16T09:59:20.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc77406b020 0x7fc77406d4e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc79015db70 0x7fc7900c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7900af2d0 0x7fc790159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 --2- 172.21.15.181:0/688932383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7900af820 0x7fc7900b2fe0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 >> 172.21.15.181:0/688932383 conn(0x7fc79006c430 msgr2=0x7fc7900ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:20.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 shutdown_connections 2023-10-16T09:59:20.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.404+0000 7fc795b9e700 1 -- 172.21.15.181:0/688932383 wait complete. 2023-10-16T09:59:20.439 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:20.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.558+0000 7f31fbd9e700 1 Processor -- start 2023-10-16T09:59:20.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.558+0000 7f31fbd9e700 1 -- start start 2023-10-16T09:59:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31fbd9e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31f415ed90 0x7f31f406e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31fbd9e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31f415e1d0 0x7f31f415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31fbd9e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f40cb3f0 con 0x7f31f415a550 2023-10-16T09:59:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31fbd9e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f415eb50 con 0x7f31f415ed90 2023-10-16T09:59:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31fbd9e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f40caac0 con 0x7f31f415e1d0 2023-10-16T09:59:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9339700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31f415ed90 0x7f31f406e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9339700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31f415ed90 0x7f31f406e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41580/0 (socket says 172.21.15.181:41580) 2023-10-16T09:59:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9339700 1 -- 172.21.15.181:0/1532518189 learned_addr learned my addr 172.21.15.181:0/1532518189 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:20.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9b3a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f31f415a550 0x7f31f40cb880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52656/0 (socket says 172.21.15.181:52656) 2023-10-16T09:59:20.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31fa33b700 1 --2- 172.21.15.181:0/1532518189 >> v2:172.21.15.190:3300/0 conn(0x7f31f415e1d0 0x7f31f415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9339700 1 -- 172.21.15.181:0/1532518189 >> v2:172.21.15.190:3300/0 conn(0x7f31f415e1d0 msgr2=0x7f31f415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:20.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9339700 1 --2- 172.21.15.181:0/1532518189 >> v2:172.21.15.190:3300/0 conn(0x7f31f415e1d0 0x7f31f415e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9339700 1 -- 172.21.15.181:0/1532518189 >> v1:172.21.15.181:6789/0 conn(0x7f31f415a550 legacy=0x7f31f40cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f8b38700 1 -- 172.21.15.181:0/1532518189 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152093515 0 0) 0x7f31f40cb3f0 con 0x7f31f415a550 2023-10-16T09:59:20.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.559+0000 7f31f9339700 1 -- 172.21.15.181:0/1532518189 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f40cada0 con 0x7f31f415ed90 2023-10-16T09:59:20.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31f9339700 1 --2- 172.21.15.181:0/1532518189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415ed90 0x7f31f406e5a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f31f40b2fa0 tx=0x7f31f000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3259d4ba85d3a45b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31f8b38700 1 -- 172.21.15.181:0/1532518189 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000dd30 con 0x7f31f415ed90 2023-10-16T09:59:20.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31f8b38700 1 -- 172.21.15.181:0/1532518189 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f000de90 con 0x7f31f415ed90 2023-10-16T09:59:20.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31f8b38700 1 -- 172.21.15.181:0/1532518189 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0010690 con 0x7f31f415ed90 2023-10-16T09:59:20.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 -- 172.21.15.181:0/1532518189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415ed90 msgr2=0x7f31f406e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/1532518189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415ed90 0x7f31f406e5a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f31f40b2fa0 tx=0x7f31f000be70 comp rx=0 tx=0).stop 2023-10-16T09:59:20.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 -- 172.21.15.181:0/1532518189 shutdown_connections 2023-10-16T09:59:20.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/1532518189 >> v2:172.21.15.190:3300/0 conn(0x7f31f415e1d0 0x7f31f415e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/1532518189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415ed90 0x7f31f406e5a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 -- 172.21.15.181:0/1532518189 >> 172.21.15.181:0/1532518189 conn(0x7f31f40d0460 msgr2=0x7f31f40d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:20.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 -- 172.21.15.181:0/1532518189 shutdown_connections 2023-10-16T09:59:20.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.560+0000 7f31fbd9e700 1 -- 172.21.15.181:0/1532518189 wait complete. 2023-10-16T09:59:20.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.562+0000 7f31fbd9e700 1 Processor -- start 2023-10-16T09:59:20.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.562+0000 7f31fbd9e700 1 -- start start 2023-10-16T09:59:20.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31fbd9e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31f415a550 0x7f31f40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31fbd9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415e1d0 0x7f31f40c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31fbd9e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31f415ed90 0x7f31f40c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31fbd9e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41eeaa0 con 0x7f31f415e1d0 2023-10-16T09:59:20.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31fbd9e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40c8240 con 0x7f31f415ed90 2023-10-16T09:59:20.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31fbd9e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41eed20 con 0x7f31f415a550 2023-10-16T09:59:20.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415e1d0 0x7f31f40c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415e1d0 0x7f31f40c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41584/0 (socket says 172.21.15.181:41584) 2023-10-16T09:59:20.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 -- 172.21.15.181:0/983296648 learned_addr learned my addr 172.21.15.181:0/983296648 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:20.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31fa33b700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31f415ed90 0x7f31f40c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 -- 172.21.15.181:0/983296648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31f415a550 msgr2=0x7f31f40b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31f415a550 0x7f31f40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 -- 172.21.15.181:0/983296648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31f415ed90 msgr2=0x7f31f40c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31f415ed90 0x7f31f40c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.563+0000 7f31f9339700 1 -- 172.21.15.181:0/983296648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f000b7f0 con 0x7f31f415e1d0 2023-10-16T09:59:20.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.564+0000 7f31f9339700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415e1d0 0x7f31f40c8f00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f31f000b7c0 tx=0x7f31f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.564+0000 7f31ea7fc700 1 -- 172.21.15.181:0/983296648 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f001f070 con 0x7f31f415e1d0 2023-10-16T09:59:20.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.564+0000 7f31ea7fc700 1 -- 172.21.15.181:0/983296648 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0003ba0 con 0x7f31f415e1d0 2023-10-16T09:59:20.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.564+0000 7f31ea7fc700 1 -- 172.21.15.181:0/983296648 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0024470 con 0x7f31f415e1d0 2023-10-16T09:59:20.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.564+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f41ef000 con 0x7f31f415e1d0 2023-10-16T09:59:20.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.564+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f40b4d10 con 0x7f31f415e1d0 2023-10-16T09:59:20.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.565+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f41ef520 con 0x7f31f415e1d0 2023-10-16T09:59:20.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.565+0000 7f31ea7fc700 1 -- 172.21.15.181:0/983296648 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31f002d430 con 0x7f31f415e1d0 2023-10-16T09:59:20.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.565+0000 7f31ea7fc700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f31d806b020 0x7f31d806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.565+0000 7f31ea7fc700 1 -- 172.21.15.181:0/983296648 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f31f008ca90 con 0x7f31f415e1d0 2023-10-16T09:59:20.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.565+0000 7f31f9b3a700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f31d806b020 0x7f31d806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.566+0000 7f31f9b3a700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f31d806b020 0x7f31d806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f31e40012a0 tx=0x7f31e40078d0 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.570+0000 7f31ea7fc700 1 -- 172.21.15.181:0/983296648 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31f41ef520 con 0x7f31f415e1d0 2023-10-16T09:59:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.763+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31f41ef520 con 0x7f31f415e1d0 2023-10-16T09:59:20.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.763+0000 7f31ea7fc700 1 -- 172.21.15.181:0/983296648 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13537 (secure 0 0 0) 0x7f31f41ef520 con 0x7f31f415e1d0 2023-10-16T09:59:20.765 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:20.766 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":72,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:19.968673+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:20.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.765+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f31d806b020 msgr2=0x7f31d806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.765+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f31d806b020 0x7f31d806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f31e40012a0 tx=0x7f31e40078d0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.765+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415e1d0 msgr2=0x7f31f40c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.765+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415e1d0 0x7f31f40c8f00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f31f000b7c0 tx=0x7f31f0004750 comp rx=0 tx=0).stop 2023-10-16T09:59:20.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 shutdown_connections 2023-10-16T09:59:20.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f31d806b020 0x7f31d806d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31f415a550 0x7f31f40b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31f415ed90 0x7f31f40c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 --2- 172.21.15.181:0/983296648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31f415e1d0 0x7f31f40c8f00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 >> 172.21.15.181:0/983296648 conn(0x7f31f40d0460 msgr2=0x7f31f40cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:20.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 shutdown_connections 2023-10-16T09:59:20.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.766+0000 7f31fbd9e700 1 -- 172.21.15.181:0/983296648 wait complete. 2023-10-16T09:59:20.779 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:20.780 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:20.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.902+0000 7f16f53a4700 1 Processor -- start 2023-10-16T09:59:20.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.902+0000 7f16f53a4700 1 -- start start 2023-10-16T09:59:20.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16f53a4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16f00cb610 0x7f16f00cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16f53a4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f0070d70 con 0x7f16f0159b40 2023-10-16T09:59:20.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16f53a4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f0071010 con 0x7f16f0159f20 2023-10-16T09:59:20.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16f53a4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f00cbfe0 con 0x7f16f00cb610 2023-10-16T09:59:20.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16ef7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16f00cb610 0x7f16f00cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16ef7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16f00cb610 0x7f16f00cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41596/0 (socket says 172.21.15.181:41596) 2023-10-16T09:59:20.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16ef7fe700 1 -- 172.21.15.181:0/1721940528 learned_addr learned my addr 172.21.15.181:0/1721940528 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:20.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.903+0000 7f16eeffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f16f0159b40 0x7f16f006d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52670/0 (socket says 172.21.15.181:52670) 2023-10-16T09:59:20.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16ef7fe700 1 -- 172.21.15.181:0/1721940528 >> v1:172.21.15.190:6789/0 conn(0x7f16f0159f20 legacy=0x7f16f00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16ef7fe700 1 -- 172.21.15.181:0/1721940528 >> v1:172.21.15.181:6789/0 conn(0x7f16f0159b40 legacy=0x7f16f006d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16ef7fe700 1 -- 172.21.15.181:0/1721940528 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f00712f0 con 0x7f16f00cb610 2023-10-16T09:59:20.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16edffb700 1 -- 172.21.15.181:0/1721940528 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535203019 0 0) 0x7f16f0070d70 con 0x7f16f0159b40 2023-10-16T09:59:20.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16ef7fe700 1 --2- 172.21.15.181:0/1721940528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00cb610 0x7f16f00cbaa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f16e800d150 tx=0x7f16e800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe099d3c5a7bb79 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16edffb700 1 -- 172.21.15.181:0/1721940528 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8010070 con 0x7f16f00cb610 2023-10-16T09:59:20.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16edffb700 1 -- 172.21.15.181:0/1721940528 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e800dbb0 con 0x7f16f00cb610 2023-10-16T09:59:20.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.904+0000 7f16edffb700 1 -- 172.21.15.181:0/1721940528 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8016400 con 0x7f16f00cb610 2023-10-16T09:59:20.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.905+0000 7f16f53a4700 1 -- 172.21.15.181:0/1721940528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00cb610 msgr2=0x7f16f00cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.905+0000 7f16f53a4700 1 --2- 172.21.15.181:0/1721940528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00cb610 0x7f16f00cbaa0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f16e800d150 tx=0x7f16e800d460 comp rx=0 tx=0).stop 2023-10-16T09:59:20.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.905+0000 7f16f53a4700 1 -- 172.21.15.181:0/1721940528 shutdown_connections 2023-10-16T09:59:20.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.905+0000 7f16f53a4700 1 --2- 172.21.15.181:0/1721940528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f00cb610 0x7f16f00cbaa0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.905+0000 7f16f53a4700 1 -- 172.21.15.181:0/1721940528 >> 172.21.15.181:0/1721940528 conn(0x7f16f006a8a0 msgr2=0x7f16f006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:20.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.905+0000 7f16f53a4700 1 -- 172.21.15.181:0/1721940528 shutdown_connections 2023-10-16T09:59:20.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.905+0000 7f16f53a4700 1 -- 172.21.15.181:0/1721940528 wait complete. 2023-10-16T09:59:20.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.906+0000 7f16f53a4700 1 Processor -- start 2023-10-16T09:59:20.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.906+0000 7f16f53a4700 1 -- start start 2023-10-16T09:59:20.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16f53a4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16f00cb610 0x7f16f0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16f53a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f0159b40 0x7f16f0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16f53a4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16f0159f20 0x7f16f00b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16f53a4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0159290 con 0x7f16f0159b40 2023-10-16T09:59:20.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16f53a4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f015baa0 con 0x7f16f00cb610 2023-10-16T09:59:20.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16f53a4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f015bd80 con 0x7f16f0159f20 2023-10-16T09:59:20.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16ee7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f0159b40 0x7f16f0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16ee7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f0159b40 0x7f16f0158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41600/0 (socket says 172.21.15.181:41600) 2023-10-16T09:59:20.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.907+0000 7f16ee7fc700 1 -- 172.21.15.181:0/3504415652 learned_addr learned my addr 172.21.15.181:0/3504415652 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:20.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16eeffd700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16f00cb610 0x7f16f0158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16ef7fe700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16f0159f20 0x7f16f00b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16ee7fc700 1 -- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16f0159f20 msgr2=0x7f16f00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16ee7fc700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16f0159f20 0x7f16f00b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16ee7fc700 1 -- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16f00cb610 msgr2=0x7f16f0158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:20.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16ee7fc700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16f00cb610 0x7f16f0158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:20.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16ee7fc700 1 -- 172.21.15.181:0/3504415652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e800bd40 con 0x7f16f0159b40 2023-10-16T09:59:20.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16ee7fc700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f0159b40 0x7f16f0158210 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c450 tx=0x7f16e000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16d77fe700 1 -- 172.21.15.181:0/3504415652 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e00052d0 con 0x7f16f0159b40 2023-10-16T09:59:20.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16d77fe700 1 -- 172.21.15.181:0/3504415652 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e0005430 con 0x7f16f0159b40 2023-10-16T09:59:20.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16d77fe700 1 -- 172.21.15.181:0/3504415652 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e00038c0 con 0x7f16f0159b40 2023-10-16T09:59:20.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f015c060 con 0x7f16f0159b40 2023-10-16T09:59:20.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.908+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f015c620 con 0x7f16f0159b40 2023-10-16T09:59:20.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.909+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f015c840 con 0x7f16f0159b40 2023-10-16T09:59:20.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.909+0000 7f16d77fe700 1 -- 172.21.15.181:0/3504415652 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f16e0003a20 con 0x7f16f0159b40 2023-10-16T09:59:20.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.910+0000 7f16d77fe700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f16dc06b020 0x7f16dc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:20.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.910+0000 7f16d77fe700 1 -- 172.21.15.181:0/3504415652 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f16e00888c0 con 0x7f16f0159b40 2023-10-16T09:59:20.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.910+0000 7f16eeffd700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f16dc06b020 0x7f16dc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:20.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.910+0000 7f16eeffd700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f16dc06b020 0x7f16dc06d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16e4007b10 tx=0x7f16e4007a60 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:20.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:20.914+0000 7f16d77fe700 1 -- 172.21.15.181:0/3504415652 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f16e0053a10 con 0x7f16f0159b40 2023-10-16T09:59:21.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.104+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16f006b830 con 0x7f16f0159b40 2023-10-16T09:59:21.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.105+0000 7f16d77fe700 1 -- 172.21.15.181:0/3504415652 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16e000ce60 con 0x7f16f0159b40 2023-10-16T09:59:21.107 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:21.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f16dc06b020 msgr2=0x7f16dc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f16dc06b020 0x7f16dc06d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16e4007b10 tx=0x7f16e4007a60 comp rx=0 tx=0).stop 2023-10-16T09:59:21.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f0159b40 msgr2=0x7f16f0158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f0159b40 0x7f16f0158210 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c450 tx=0x7f16e000c760 comp rx=0 tx=0).stop 2023-10-16T09:59:21.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 shutdown_connections 2023-10-16T09:59:21.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f16dc06b020 0x7f16dc06d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16f0159f20 0x7f16f00b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16f00cb610 0x7f16f0158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 --2- 172.21.15.181:0/3504415652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16f0159b40 0x7f16f0158210 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 >> 172.21.15.181:0/3504415652 conn(0x7f16f006a8a0 msgr2=0x7f16f00cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:21.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 shutdown_connections 2023-10-16T09:59:21.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.107+0000 7f16f53a4700 1 -- 172.21.15.181:0/3504415652 wait complete. 2023-10-16T09:59:21.121 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T09:59:21.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.240+0000 7f6c13dab700 1 Processor -- start 2023-10-16T09:59:21.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.241+0000 7f6c13dab700 1 -- start start 2023-10-16T09:59:21.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.241+0000 7f6c13dab700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6c0c15eab0 0x7f6c0c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c13dab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6c0c0cb0c0 0x7f6c0c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c13dab700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c06ed00 con 0x7f6c0c15e6d0 2023-10-16T09:59:21.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c13dab700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c0d02d0 con 0x7f6c0c0cb0c0 2023-10-16T09:59:21.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c13dab700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c15ad40 con 0x7f6c0c15eab0 2023-10-16T09:59:21.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c12348700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6c0c0cb0c0 0x7f6c0c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c11b47700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6c0c15e6d0 0x7f6c0c15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52684/0 (socket says 172.21.15.181:52684) 2023-10-16T09:59:21.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c12348700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6c0c0cb0c0 0x7f6c0c15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41610/0 (socket says 172.21.15.181:41610) 2023-10-16T09:59:21.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c11b47700 1 -- 172.21.15.181:0/2327110950 learned_addr learned my addr 172.21.15.181:0/2327110950 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:21.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c11346700 1 --2- 172.21.15.181:0/2327110950 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c15eab0 0x7f6c0c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c12348700 1 -- 172.21.15.181:0/2327110950 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c15eab0 msgr2=0x7f6c0c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:21.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c12348700 1 --2- 172.21.15.181:0/2327110950 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c15eab0 0x7f6c0c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c12348700 1 -- 172.21.15.181:0/2327110950 >> v1:172.21.15.181:6789/0 conn(0x7f6c0c15e6d0 legacy=0x7f6c0c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c12348700 1 -- 172.21.15.181:0/2327110950 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c06e0f0 con 0x7f6c0c0cb0c0 2023-10-16T09:59:21.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c11346700 1 --2- 172.21.15.181:0/2327110950 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c15eab0 0x7f6c0c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:21.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c10b45700 1 -- 172.21.15.181:0/2327110950 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906751004 0 0) 0x7f6c0c06ed00 con 0x7f6c0c15e6d0 2023-10-16T09:59:21.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c12348700 1 --2- 172.21.15.181:0/2327110950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c0cb0c0 0x7f6c0c15a800 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0714b0 tx=0x7f6bfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb321b5ec0bb4ed server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c10b45700 1 -- 172.21.15.181:0/2327110950 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00e070 con 0x7f6c0c0cb0c0 2023-10-16T09:59:21.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c10b45700 1 -- 172.21.15.181:0/2327110950 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc00bbd0 con 0x7f6c0c0cb0c0 2023-10-16T09:59:21.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.242+0000 7f6c10b45700 1 -- 172.21.15.181:0/2327110950 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc012430 con 0x7f6c0c0cb0c0 2023-10-16T09:59:21.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 -- 172.21.15.181:0/2327110950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c0cb0c0 msgr2=0x7f6c0c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 --2- 172.21.15.181:0/2327110950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c0cb0c0 0x7f6c0c15a800 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0714b0 tx=0x7f6bfc00b330 comp rx=0 tx=0).stop 2023-10-16T09:59:21.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 -- 172.21.15.181:0/2327110950 shutdown_connections 2023-10-16T09:59:21.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 --2- 172.21.15.181:0/2327110950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c0cb0c0 0x7f6c0c15a800 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 --2- 172.21.15.181:0/2327110950 >> v2:172.21.15.190:3300/0 conn(0x7f6c0c15eab0 0x7f6c0c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 -- 172.21.15.181:0/2327110950 >> 172.21.15.181:0/2327110950 conn(0x7f6c0c0d04a0 msgr2=0x7f6c0c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 -- 172.21.15.181:0/2327110950 shutdown_connections 2023-10-16T09:59:21.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.243+0000 7f6c13dab700 1 -- 172.21.15.181:0/2327110950 wait complete. 2023-10-16T09:59:21.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.244+0000 7f6c13dab700 1 Processor -- start 2023-10-16T09:59:21.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.244+0000 7f6c13dab700 1 -- start start 2023-10-16T09:59:21.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c13dab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c0cb0c0 0x7f6c0c14e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c13dab700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c15e6d0 0x7f6c0c0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c13dab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c15eab0 0x7f6c0c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c13dab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c7850 con 0x7f6c0c15eab0 2023-10-16T09:59:21.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c13dab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c87b0 con 0x7f6c0c0cb0c0 2023-10-16T09:59:21.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c13dab700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c8a90 con 0x7f6c0c15e6d0 2023-10-16T09:59:21.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c15eab0 0x7f6c0c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c15eab0 0x7f6c0c0c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41616/0 (socket says 172.21.15.181:41616) 2023-10-16T09:59:21.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 -- 172.21.15.181:0/1245251359 learned_addr learned my addr 172.21.15.181:0/1245251359 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:21.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c11b47700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c0cb0c0 0x7f6c0c14e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 -- 172.21.15.181:0/1245251359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c15e6d0 msgr2=0x7f6c0c0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c15e6d0 0x7f6c0c0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 -- 172.21.15.181:0/1245251359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c0cb0c0 msgr2=0x7f6c0c14e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c0cb0c0 0x7f6c0c14e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.245+0000 7f6c12348700 1 -- 172.21.15.181:0/1245251359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc009a70 con 0x7f6c0c15eab0 2023-10-16T09:59:21.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.246+0000 7f6c12348700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c15eab0 0x7f6c0c0c8180 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c071580 tx=0x7f6bfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.246+0000 7f6c0a7fc700 1 -- 172.21.15.181:0/1245251359 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc012430 con 0x7f6c0c15eab0 2023-10-16T09:59:21.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.246+0000 7f6c0a7fc700 1 -- 172.21.15.181:0/1245251359 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc020070 con 0x7f6c0c15eab0 2023-10-16T09:59:21.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.246+0000 7f6c0a7fc700 1 -- 172.21.15.181:0/1245251359 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc004280 con 0x7f6c0c15eab0 2023-10-16T09:59:21.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.246+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c0c8d70 con 0x7f6c0c15eab0 2023-10-16T09:59:21.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.246+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c15c540 con 0x7f6c0c15eab0 2023-10-16T09:59:21.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.247+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c067d20 con 0x7f6c0c15eab0 2023-10-16T09:59:21.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.247+0000 7f6c0a7fc700 1 -- 172.21.15.181:0/1245251359 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6bfc0043e0 con 0x7f6c0c15eab0 2023-10-16T09:59:21.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.247+0000 7f6c0a7fc700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6bf806b020 0x7f6bf806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.248+0000 7f6c0a7fc700 1 -- 172.21.15.181:0/1245251359 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f6bfc022070 con 0x7f6c0c15eab0 2023-10-16T09:59:21.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.248+0000 7f6c11b47700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6bf806b020 0x7f6bf806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.248+0000 7f6c11b47700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6bf806b020 0x7f6bf806d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001670 tx=0x7f6c04009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.252+0000 7f6c0a7fc700 1 -- 172.21.15.181:0/1245251359 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6bfc056910 con 0x7f6c0c15eab0 2023-10-16T09:59:21.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.446+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6c0c0cb610 con 0x7f6c0c15eab0 2023-10-16T09:59:21.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.446+0000 7f6c0a7fc700 1 -- 172.21.15.181:0/1245251359 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6bfc02c060 con 0x7f6c0c15eab0 2023-10-16T09:59:21.448 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:21.449 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:59:21.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:21.449 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T09:59:21.449 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T09:59:21.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:21.450 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:59:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6bf806b020 msgr2=0x7f6bf806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6bf806b020 0x7f6bf806d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001670 tx=0x7f6c04009040 comp rx=0 tx=0).stop 2023-10-16T09:59:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c15eab0 msgr2=0x7f6c0c0c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c15eab0 0x7f6c0c0c8180 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c071580 tx=0x7f6bfc004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 shutdown_connections 2023-10-16T09:59:21.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6bf806b020 0x7f6bf806d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0c15eab0 0x7f6c0c0c8180 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c0c0cb0c0 0x7f6c0c14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 --2- 172.21.15.181:0/1245251359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c0c15e6d0 0x7f6c0c0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 >> 172.21.15.181:0/1245251359 conn(0x7f6c0c0d04a0 msgr2=0x7f6c0c0cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:21.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 shutdown_connections 2023-10-16T09:59:21.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.448+0000 7f6c13dab700 1 -- 172.21.15.181:0/1245251359 wait complete. 2023-10-16T09:59:21.462 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T09:59:21.463 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T09:59:21.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.585+0000 7f757b4fe700 1 Processor -- start 2023-10-16T09:59:21.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.585+0000 7f757b4fe700 1 -- start start 2023-10-16T09:59:21.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f757b4fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f75740cd5c0 0x7f75740cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f757b4fe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75740ce7e0 con 0x7f75740afa10 2023-10-16T09:59:21.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f757b4fe700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757415b540 con 0x7f75740ce3a0 2023-10-16T09:59:21.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f757b4fe700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757415b820 con 0x7f75740cd5c0 2023-10-16T09:59:21.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f757929a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f75740afa10 0x7f757406d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52698/0 (socket says 172.21.15.181:52698) 2023-10-16T09:59:21.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f757929a700 1 -- 172.21.15.181:0/1841863071 learned_addr learned my addr 172.21.15.181:0/1841863071 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:21.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380738799 0 0) 0x7f75740ce7e0 con 0x7f75740afa10 2023-10-16T09:59:21.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f7579a9b700 1 --2- 172.21.15.181:0/1841863071 >> v2:172.21.15.190:3300/0 conn(0x7f75740cd5c0 0x7f75740cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.586+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75600035b0 con 0x7f75740afa10 2023-10-16T09:59:21.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3549683904 0 0) 0x7f75600035b0 con 0x7f75740afa10 2023-10-16T09:59:21.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 >> v2:172.21.15.190:3300/0 conn(0x7f75740cd5c0 msgr2=0x7f75740cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:21.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 --2- 172.21.15.181:0/1841863071 >> v2:172.21.15.190:3300/0 conn(0x7f75740cd5c0 0x7f75740cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 >> v1:172.21.15.190:6789/0 conn(0x7f75740ce3a0 legacy=0x7f75740cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757415bb00 con 0x7f75740afa10 2023-10-16T09:59:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f75640017f0 con 0x7f75740afa10 2023-10-16T09:59:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 >> v1:172.21.15.181:6789/0 conn(0x7f75740afa10 legacy=0x7f757406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7560003a00 0x7f7560003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560004850 0x7f7560004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7560005210 0x7f756000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415b820 con 0x7f7560005210 2023-10-16T09:59:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756000c0c0 con 0x7f7560003a00 2023-10-16T09:59:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756000c280 con 0x7f7560004850 2023-10-16T09:59:21.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7560005210 0x7f756000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f757929a700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7560003a00 0x7f7560003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7578a99700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560004850 0x7f7560004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 -- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560004850 msgr2=0x7f7560004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560004850 0x7f7560004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 -- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7560003a00 msgr2=0x7f7560003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:21.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7560003a00 0x7f7560003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 -- 172.21.15.181:0/1841863071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600035b0 con 0x7f7560005210 2023-10-16T09:59:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f757929a700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7560003a00 0x7f7560003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 -- 172.21.15.181:0/1841863071 reap_dead start 2023-10-16T09:59:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f7579a9b700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7560005210 0x7f756000ba10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f756c009fc0 tx=0x7f7560005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907323822c76294a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c003a20 con 0x7f7560005210 2023-10-16T09:59:21.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c00f040 con 0x7f7560005210 2023-10-16T09:59:21.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.587+0000 7f756bfff700 1 -- 172.21.15.181:0/1841863071 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c003de0 con 0x7f7560005210 2023-10-16T09:59:21.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.588+0000 7f757b4fe700 1 -- 172.21.15.181:0/1841863071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7560005210 msgr2=0x7f756000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.588+0000 7f757b4fe700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7560005210 0x7f756000ba10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f756c009fc0 tx=0x7f7560005880 comp rx=0 tx=0).stop 2023-10-16T09:59:21.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.588+0000 7f757b4fe700 1 -- 172.21.15.181:0/1841863071 shutdown_connections 2023-10-16T09:59:21.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.588+0000 7f757b4fe700 1 --2- 172.21.15.181:0/1841863071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7560005210 0x7f756000ba10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.588+0000 7f757b4fe700 1 -- 172.21.15.181:0/1841863071 >> 172.21.15.181:0/1841863071 conn(0x7f75740d04a0 msgr2=0x7f75740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:21.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.588+0000 7f757b4fe700 1 -- 172.21.15.181:0/1841863071 shutdown_connections 2023-10-16T09:59:21.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.588+0000 7f757b4fe700 1 -- 172.21.15.181:0/1841863071 wait complete. 2023-10-16T09:59:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.590+0000 7f757b4fe700 1 Processor -- start 2023-10-16T09:59:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.590+0000 7f757b4fe700 1 -- start start 2023-10-16T09:59:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757b4fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560005210 0x7f75740adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757b4fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7574158dd0 0x7f7574157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757b4fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75741578b0 0x7f75741efab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757b4fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75741589b0 con 0x7f75741578b0 2023-10-16T09:59:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757b4fe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7574157d30 con 0x7f7574158dd0 2023-10-16T09:59:21.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757b4fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7574158010 con 0x7f7560005210 2023-10-16T09:59:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75741578b0 0x7f75741efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75741578b0 0x7f75741efab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41636/0 (socket says 172.21.15.181:41636) 2023-10-16T09:59:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 -- 172.21.15.181:0/3211340306 learned_addr learned my addr 172.21.15.181:0/3211340306 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757929a700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560005210 0x7f75740adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7578a99700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7574158dd0 0x7f7574157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 -- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560005210 msgr2=0x7f75740adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560005210 0x7f75740adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 -- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7574158dd0 msgr2=0x7f7574157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7574158dd0 0x7f7574157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 -- 172.21.15.181:0/3211340306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75741582f0 con 0x7f75741578b0 2023-10-16T09:59:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7578a99700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7574158dd0 0x7f7574157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7579a9b700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75741578b0 0x7f75741efab0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f757406d380 tx=0x7f756c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7569ffb700 1 -- 172.21.15.181:0/3211340306 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c003e90 con 0x7f75741578b0 2023-10-16T09:59:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7569ffb700 1 -- 172.21.15.181:0/3211340306 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c004230 con 0x7f75741578b0 2023-10-16T09:59:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75741585d0 con 0x7f75741578b0 2023-10-16T09:59:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.591+0000 7f7569ffb700 1 -- 172.21.15.181:0/3211340306 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0045e0 con 0x7f75741578b0 2023-10-16T09:59:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.592+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75740ae1d0 con 0x7f75741578b0 2023-10-16T09:59:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.592+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7574158af0 con 0x7f75741578b0 2023-10-16T09:59:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.593+0000 7f7569ffb700 1 -- 172.21.15.181:0/3211340306 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f756c004740 con 0x7f75741578b0 2023-10-16T09:59:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.593+0000 7f7569ffb700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f755806b020 0x7f755806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.593+0000 7f7569ffb700 1 -- 172.21.15.181:0/3211340306 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f756c08a790 con 0x7f75741578b0 2023-10-16T09:59:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.593+0000 7f757929a700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f755806b020 0x7f755806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.594+0000 7f757929a700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f755806b020 0x7f755806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7564001170 tx=0x7f7564009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.598+0000 7f7569ffb700 1 -- 172.21.15.181:0/3211340306 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7574158af0 con 0x7f75741578b0 2023-10-16T09:59:21.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.792+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7574158af0 con 0x7f75741578b0 2023-10-16T09:59:21.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.831+0000 7f7569ffb700 1 -- 172.21.15.181:0/3211340306 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7574158af0 con 0x7f75741578b0 2023-10-16T09:59:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f755806b020 msgr2=0x7f755806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f755806b020 0x7f755806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7564001170 tx=0x7f7564009040 comp rx=0 tx=0).stop 2023-10-16T09:59:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75741578b0 msgr2=0x7f75741efab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75741578b0 0x7f75741efab0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f757406d380 tx=0x7f756c009750 comp rx=0 tx=0).stop 2023-10-16T09:59:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 shutdown_connections 2023-10-16T09:59:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f755806b020 0x7f755806d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7560005210 0x7f75740adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75741578b0 0x7f75741efab0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 --2- 172.21.15.181:0/3211340306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7574158dd0 0x7f7574157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 >> 172.21.15.181:0/3211340306 conn(0x7f75740d04a0 msgr2=0x7f75740ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 shutdown_connections 2023-10-16T09:59:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.834+0000 7f757b4fe700 1 -- 172.21.15.181:0/3211340306 wait complete. 2023-10-16T09:59:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T09:59:21.851 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T09:59:21.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.977+0000 7fea1e4c5700 1 Processor -- start 2023-10-16T09:59:21.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.977+0000 7fea1e4c5700 1 -- start start 2023-10-16T09:59:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1e4c5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fea180cc6e0 0x7fea18071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1e4c5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fea180ce680 0x7fea180ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1e4c5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea180ce1f0 con 0x7fea180b2010 2023-10-16T09:59:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1e4c5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea180ccb90 con 0x7fea180ce680 2023-10-16T09:59:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1e4c5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea18072120 con 0x7fea180cc6e0 2023-10-16T09:59:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1ca62700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fea180ce680 0x7fea180ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea17fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fea180b2010 0x7fea180b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52718/0 (socket says 172.21.15.181:52718) 2023-10-16T09:59:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea17fff700 1 -- 172.21.15.181:0/768223677 learned_addr learned my addr 172.21.15.181:0/768223677 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea177fe700 1 --2- 172.21.15.181:0/768223677 >> v2:172.21.15.190:3300/0 conn(0x7fea180cc6e0 0x7fea18071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1ca62700 1 -- 172.21.15.181:0/768223677 >> v2:172.21.15.190:3300/0 conn(0x7fea180cc6e0 msgr2=0x7fea18071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1ca62700 1 --2- 172.21.15.181:0/768223677 >> v2:172.21.15.190:3300/0 conn(0x7fea180cc6e0 0x7fea18071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1ca62700 1 -- 172.21.15.181:0/768223677 >> v1:172.21.15.181:6789/0 conn(0x7fea180b2010 legacy=0x7fea180b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1ca62700 1 -- 172.21.15.181:0/768223677 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea180ccda0 con 0x7fea180ce680 2023-10-16T09:59:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea1ca62700 1 --2- 172.21.15.181:0/768223677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180ceaf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fea1815e1c0 tx=0x7fea1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72380d69331258c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea16ffd700 1 -- 172.21.15.181:0/768223677 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1000e070 con 0x7fea180ce680 2023-10-16T09:59:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea16ffd700 1 -- 172.21.15.181:0/768223677 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1000bbd0 con 0x7fea180ce680 2023-10-16T09:59:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.978+0000 7fea16ffd700 1 -- 172.21.15.181:0/768223677 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10012430 con 0x7fea180ce680 2023-10-16T09:59:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 -- 172.21.15.181:0/768223677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 msgr2=0x7fea180ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/768223677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180ceaf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fea1815e1c0 tx=0x7fea1000b330 comp rx=0 tx=0).stop 2023-10-16T09:59:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 -- 172.21.15.181:0/768223677 shutdown_connections 2023-10-16T09:59:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/768223677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180ceaf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/768223677 >> v2:172.21.15.190:3300/0 conn(0x7fea180cc6e0 0x7fea18071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 -- 172.21.15.181:0/768223677 >> 172.21.15.181:0/768223677 conn(0x7fea180cd010 msgr2=0x7fea180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:21.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 -- 172.21.15.181:0/768223677 shutdown_connections 2023-10-16T09:59:21.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.979+0000 7fea1e4c5700 1 -- 172.21.15.181:0/768223677 wait complete. 2023-10-16T09:59:21.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.980+0000 7fea1e4c5700 1 Processor -- start 2023-10-16T09:59:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.981+0000 7fea1e4c5700 1 -- start start 2023-10-16T09:59:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea180b2010 0x7fea180afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea180cc6e0 0x7fea180c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea180c6920 con 0x7fea180ce680 2023-10-16T09:59:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1815c860 con 0x7fea180b2010 2023-10-16T09:59:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1814bab0 con 0x7fea180cc6e0 2023-10-16T09:59:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41650/0 (socket says 172.21.15.181:41650) 2023-10-16T09:59:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 -- 172.21.15.181:0/1477900154 learned_addr learned my addr 172.21.15.181:0/1477900154 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea17fff700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea180b2010 0x7fea180afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea177fe700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea180cc6e0 0x7fea180c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 -- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea180cc6e0 msgr2=0x7fea180c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea180cc6e0 0x7fea180c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 -- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea180b2010 msgr2=0x7fea180afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea180b2010 0x7fea180afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 -- 172.21.15.181:0/1477900154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10009a70 con 0x7fea180ce680 2023-10-16T09:59:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea17fff700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea180b2010 0x7fea180afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1ca62700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180c63e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fea10014040 tx=0x7fea10004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea14ff9700 1 -- 172.21.15.181:0/1477900154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea100098f0 con 0x7fea180ce680 2023-10-16T09:59:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea14ff9700 1 -- 172.21.15.181:0/1477900154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea10020070 con 0x7fea180ce680 2023-10-16T09:59:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1814b4f0 con 0x7fea180ce680 2023-10-16T09:59:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea14ff9700 1 -- 172.21.15.181:0/1477900154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10004280 con 0x7fea180ce680 2023-10-16T09:59:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.982+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1815c4f0 con 0x7fea180ce680 2023-10-16T09:59:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.983+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1815c9d0 con 0x7fea180ce680 2023-10-16T09:59:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.984+0000 7fea14ff9700 1 -- 172.21.15.181:0/1477900154 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fea100043e0 con 0x7fea180ce680 2023-10-16T09:59:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.985+0000 7fea14ff9700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fea0406b020 0x7fea0406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.985+0000 7fea17fff700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fea0406b020 0x7fea0406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.985+0000 7fea14ff9700 1 -- 172.21.15.181:0/1477900154 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5651+0+0 (secure 0 0 0) 0x7fea10022070 con 0x7fea180ce680 2023-10-16T09:59:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.985+0000 7fea17fff700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fea0406b020 0x7fea0406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fea08005910 tx=0x7fea08009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:21.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:21.989+0000 7fea14ff9700 1 -- 172.21.15.181:0/1477900154 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fea10056d30 con 0x7fea180ce680 2023-10-16T09:59:22.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.181+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fea180b6510 con 0x7fea180ce680 2023-10-16T09:59:22.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.834+0000 7fea14ff9700 1 -- 172.21.15.181:0/1477900154 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) v1 ==== 170+0+0 (secure 0 0 0) 0x7fea1002c070 con 0x7fea180ce680 2023-10-16T09:59:22.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.837+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fea0406b020 msgr2=0x7fea0406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:22.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.837+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fea0406b020 0x7fea0406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fea08005910 tx=0x7fea08009040 comp rx=0 tx=0).stop 2023-10-16T09:59:22.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.837+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 msgr2=0x7fea180c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:22.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.837+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180c63e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fea10014040 tx=0x7fea10004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:22.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.838+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 shutdown_connections 2023-10-16T09:59:22.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.838+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fea0406b020 0x7fea0406d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:22.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.838+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea180ce680 0x7fea180c63e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:22.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.838+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea180b2010 0x7fea180afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:22.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.838+0000 7fea1e4c5700 1 --2- 172.21.15.181:0/1477900154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea180cc6e0 0x7fea180c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:22.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.838+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 >> 172.21.15.181:0/1477900154 conn(0x7fea180cd010 msgr2=0x7fea18154470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:22.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.839+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 shutdown_connections 2023-10-16T09:59:22.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.839+0000 7fea1e4c5700 1 -- 172.21.15.181:0/1477900154 wait complete. 2023-10-16T09:59:22.845 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T09:59:22.864 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T09:59:22.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.989+0000 7f3ccee64700 1 Processor -- start 2023-10-16T09:59:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.989+0000 7f3ccee64700 1 -- start start 2023-10-16T09:59:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3ccee64700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3cc80b2010 0x7f3cc80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3ccee64700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8071be0 0x7f3cc80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:22.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3ccee64700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc80ce1f0 con 0x7f3cc806f3b0 2023-10-16T09:59:22.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3ccee64700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80b2930 con 0x7f3cc80b2010 2023-10-16T09:59:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3ccee64700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80ccda0 con 0x7f3cc8071be0 2023-10-16T09:59:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3cc80b2010 0x7f3cc80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3cc80b2010 0x7f3cc80cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41654/0 (socket says 172.21.15.181:41654) 2023-10-16T09:59:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 -- 172.21.15.181:0/31434908 learned_addr learned my addr 172.21.15.181:0/31434908 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cc7fff700 1 --2- 172.21.15.181:0/31434908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8071be0 0x7f3cc80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 -- 172.21.15.181:0/31434908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8071be0 msgr2=0x7f3cc80b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 --2- 172.21.15.181:0/31434908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8071be0 0x7f3cc80b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 -- 172.21.15.181:0/31434908 >> v1:172.21.15.181:6789/0 conn(0x7f3cc806f3b0 legacy=0x7f3cc806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 -- 172.21.15.181:0/31434908 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80727e0 con 0x7f3cc80b2010 2023-10-16T09:59:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cc77fe700 1 -- 172.21.15.181:0/31434908 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878035786 0 0) 0x7f3cc80ce1f0 con 0x7f3cc806f3b0 2023-10-16T09:59:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cccc00700 1 --2- 172.21.15.181:0/31434908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc80b2010 0x7f3cc80cc6e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0098f0 tx=0x7f3cbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf693f6e4fe546af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cc77fe700 1 -- 172.21.15.181:0/31434908 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011830 con 0x7f3cc80b2010 2023-10-16T09:59:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cc77fe700 1 -- 172.21.15.181:0/31434908 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc011990 con 0x7f3cc80b2010 2023-10-16T09:59:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.990+0000 7f3cc77fe700 1 -- 172.21.15.181:0/31434908 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011d20 con 0x7f3cc80b2010 2023-10-16T09:59:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 -- 172.21.15.181:0/31434908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc80b2010 msgr2=0x7f3cc80cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 --2- 172.21.15.181:0/31434908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc80b2010 0x7f3cc80cc6e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0098f0 tx=0x7f3cbc009c00 comp rx=0 tx=0).stop 2023-10-16T09:59:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 -- 172.21.15.181:0/31434908 shutdown_connections 2023-10-16T09:59:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 --2- 172.21.15.181:0/31434908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc80b2010 0x7f3cc80cc6e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 --2- 172.21.15.181:0/31434908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc8071be0 0x7f3cc80b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 -- 172.21.15.181:0/31434908 >> 172.21.15.181:0/31434908 conn(0x7f3cc80cd010 msgr2=0x7f3cc80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 -- 172.21.15.181:0/31434908 shutdown_connections 2023-10-16T09:59:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.991+0000 7f3ccee64700 1 -- 172.21.15.181:0/31434908 wait complete. 2023-10-16T09:59:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.992+0000 7f3ccee64700 1 Processor -- start 2023-10-16T09:59:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.992+0000 7f3ccee64700 1 -- start start 2023-10-16T09:59:23.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3ccee64700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc806f3b0 0x7f3cc81f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:23.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3ccee64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc8071be0 0x7f3cc815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:23.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3ccee64700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80b2010 0x7f3cc80c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:23.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3ccee64700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8150030 con 0x7f3cc8071be0 2023-10-16T09:59:23.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3ccee64700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc81f5820 con 0x7f3cc806f3b0 2023-10-16T09:59:23.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3ccee64700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc814ef90 con 0x7f3cc80b2010 2023-10-16T09:59:23.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3cc7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc8071be0 0x7f3cc815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:23.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3cc7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc8071be0 0x7f3cc815c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41662/0 (socket says 172.21.15.181:41662) 2023-10-16T09:59:23.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3cc7fff700 1 -- 172.21.15.181:0/2216001067 learned_addr learned my addr 172.21.15.181:0/2216001067 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:23.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.993+0000 7f3cccc00700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc806f3b0 0x7f3cc81f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:23.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3ccd401700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80b2010 0x7f3cc80c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:23.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc7fff700 1 -- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80b2010 msgr2=0x7f3cc80c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:23.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc7fff700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80b2010 0x7f3cc80c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc7fff700 1 -- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc806f3b0 msgr2=0x7f3cc81f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:23.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc7fff700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc806f3b0 0x7f3cc81f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc7fff700 1 -- 172.21.15.181:0/2216001067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc009550 con 0x7f3cc8071be0 2023-10-16T09:59:23.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc7fff700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc8071be0 0x7f3cc815c010 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400e870 tx=0x7f3cb400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:23.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc57fa700 1 -- 172.21.15.181:0/2216001067 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4009590 con 0x7f3cc8071be0 2023-10-16T09:59:23.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc57fa700 1 -- 172.21.15.181:0/2216001067 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb4013680 con 0x7f3cc8071be0 2023-10-16T09:59:23.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3cc57fa700 1 -- 172.21.15.181:0/2216001067 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40139f0 con 0x7f3cc8071be0 2023-10-16T09:59:23.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc814f270 con 0x7f3cc8071be0 2023-10-16T09:59:23.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.994+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc814f830 con 0x7f3cc8071be0 2023-10-16T09:59:23.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.995+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc80b6510 con 0x7f3cc8071be0 2023-10-16T09:59:23.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.996+0000 7f3cc57fa700 1 -- 172.21.15.181:0/2216001067 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3cb4013b50 con 0x7f3cc8071be0 2023-10-16T09:59:23.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.996+0000 7f3cc57fa700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3cac06b020 0x7f3cac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:23.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.996+0000 7f3cc57fa700 1 -- 172.21.15.181:0/2216001067 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6033+0+0 (secure 0 0 0) 0x7f3cb4089a70 con 0x7f3cc8071be0 2023-10-16T09:59:23.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.996+0000 7f3cccc00700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3cac06b020 0x7f3cac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:23.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:22.997+0000 7f3cccc00700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3cac06b020 0x7f3cac06d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009130 tx=0x7f3cbc008030 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:23.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.001+0000 7f3cc57fa700 1 -- 172.21.15.181:0/2216001067 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3cb40548d0 con 0x7f3cc8071be0 2023-10-16T09:59:23.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.237+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3cc8158230 con 0x7f3cc8071be0 2023-10-16T09:59:23.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.860+0000 7f3cc57fa700 1 -- 172.21.15.181:0/2216001067 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3cb4057ef0 con 0x7f3cc8071be0 2023-10-16T09:59:23.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3cac06b020 msgr2=0x7f3cac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:23.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3cac06b020 0x7f3cac06d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009130 tx=0x7f3cbc008030 comp rx=0 tx=0).stop 2023-10-16T09:59:23.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc8071be0 msgr2=0x7f3cc815c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:23.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc8071be0 0x7f3cc815c010 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400e870 tx=0x7f3cb400eb80 comp rx=0 tx=0).stop 2023-10-16T09:59:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 shutdown_connections 2023-10-16T09:59:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f3cac06b020 0x7f3cac06d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80b2010 0x7f3cc80c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc806f3b0 0x7f3cc81f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 --2- 172.21.15.181:0/2216001067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc8071be0 0x7f3cc815c010 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:23.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.867+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 >> 172.21.15.181:0/2216001067 conn(0x7f3cc80cd010 msgr2=0x7f3cc8158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:23.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.868+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 shutdown_connections 2023-10-16T09:59:23.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:23.868+0000 7f3ccee64700 1 -- 172.21.15.181:0/2216001067 wait complete. 2023-10-16T09:59:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 8 and data pool 9 2023-10-16T09:59:23.894 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.018+0000 7f102cd6b700 1 Processor -- start 2023-10-16T09:59:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.018+0000 7f102cd6b700 1 -- start start 2023-10-16T09:59:24.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f102cd6b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280af450 0x7f10280af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f102cd6b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102806f870 con 0x7f10280cd470 2023-10-16T09:59:24.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f102cd6b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280d1160 con 0x7f102815dfd0 2023-10-16T09:59:24.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f102cd6b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280cc8e0 con 0x7f10280af450 2023-10-16T09:59:24.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f1026d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f10280cd470 0x7f102806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52750/0 (socket says 172.21.15.181:52750) 2023-10-16T09:59:24.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f1026d9d700 1 -- 172.21.15.181:0/1254076417 learned_addr learned my addr 172.21.15.181:0/1254076417 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:24.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f1025d9b700 1 --2- 172.21.15.181:0/1254076417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280af450 0x7f10280af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.019+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900514956 0 0) 0x7f102806f870 con 0x7f10280cd470 2023-10-16T09:59:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10100035b0 con 0x7f10280cd470 2023-10-16T09:59:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727726882 0 0) 0x7f10280d1160 con 0x7f102815dfd0 2023-10-16T09:59:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f102806f870 con 0x7f102815dfd0 2023-10-16T09:59:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 516971751 0 0) 0x7f10100035b0 con 0x7f10280cd470 2023-10-16T09:59:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f1025d9b700 1 -- 172.21.15.181:0/1254076417 >> v1:172.21.15.190:6789/0 conn(0x7f102815dfd0 legacy=0x7f102806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f1025d9b700 1 -- 172.21.15.181:0/1254076417 >> v1:172.21.15.181:6789/0 conn(0x7f10280cd470 legacy=0x7f102806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f1025d9b700 1 -- 172.21.15.181:0/1254076417 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280ccbc0 con 0x7f10280af450 2023-10-16T09:59:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f1025d9b700 1 --2- 172.21.15.181:0/1254076417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280af450 0x7f10280af9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1018009870 tx=0x7f1018009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f93049d1bddb93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200019f0 con 0x7f10280af450 2023-10-16T09:59:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018004030 con 0x7f10280af450 2023-10-16T09:59:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.020+0000 7f102559a700 1 -- 172.21.15.181:0/1254076417 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180043c0 con 0x7f10280af450 2023-10-16T09:59:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.021+0000 7f102cd6b700 1 -- 172.21.15.181:0/1254076417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280af450 msgr2=0x7f10280af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.021+0000 7f102cd6b700 1 --2- 172.21.15.181:0/1254076417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280af450 0x7f10280af9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1018009870 tx=0x7f1018009b80 comp rx=0 tx=0).stop 2023-10-16T09:59:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.021+0000 7f102cd6b700 1 -- 172.21.15.181:0/1254076417 shutdown_connections 2023-10-16T09:59:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.021+0000 7f102cd6b700 1 --2- 172.21.15.181:0/1254076417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280af450 0x7f10280af9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.021+0000 7f102cd6b700 1 -- 172.21.15.181:0/1254076417 >> 172.21.15.181:0/1254076417 conn(0x7f10280d04a0 msgr2=0x7f10280d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.021+0000 7f102cd6b700 1 -- 172.21.15.181:0/1254076417 shutdown_connections 2023-10-16T09:59:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.021+0000 7f102cd6b700 1 -- 172.21.15.181:0/1254076417 wait complete. 2023-10-16T09:59:24.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.022+0000 7f102cd6b700 1 Processor -- start 2023-10-16T09:59:24.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.022+0000 7f102cd6b700 1 -- start start 2023-10-16T09:59:24.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102cd6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10280af450 0x7f10280b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102cd6b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280cd470 0x7f10281549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102cd6b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102815dfd0 0x7f10280b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102cd6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028154f30 con 0x7f10280af450 2023-10-16T09:59:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102cd6b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280b47a0 con 0x7f102815dfd0 2023-10-16T09:59:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102cd6b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280b4a80 con 0x7f10280cd470 2023-10-16T09:59:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10280af450 0x7f10280b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10280af450 0x7f10280b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41674/0 (socket says 172.21.15.181:41674) 2023-10-16T09:59:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 -- 172.21.15.181:0/792147719 learned_addr learned my addr 172.21.15.181:0/792147719 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 -- 172.21.15.181:0/792147719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280cd470 msgr2=0x7f10281549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280cd470 0x7f10281549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 -- 172.21.15.181:0/792147719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102815dfd0 msgr2=0x7f10280b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102815dfd0 0x7f10280b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 -- 172.21.15.181:0/792147719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10180094a0 con 0x7f10280af450 2023-10-16T09:59:24.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102659c700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10280af450 0x7f10280b5840 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f101c001f00 tx=0x7f101c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f1016ffd700 1 -- 172.21.15.181:0/792147719 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0052d0 con 0x7f10280af450 2023-10-16T09:59:24.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f1016ffd700 1 -- 172.21.15.181:0/792147719 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c005430 con 0x7f10280af450 2023-10-16T09:59:24.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10280b4d60 con 0x7f10280af450 2023-10-16T09:59:24.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.023+0000 7f1016ffd700 1 -- 172.21.15.181:0/792147719 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0038f0 con 0x7f10280af450 2023-10-16T09:59:24.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.024+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10280b5320 con 0x7f10280af450 2023-10-16T09:59:24.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.024+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10280b5540 con 0x7f10280af450 2023-10-16T09:59:24.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.025+0000 7f1016ffd700 1 -- 172.21.15.181:0/792147719 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f101c003a50 con 0x7f10280af450 2023-10-16T09:59:24.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.025+0000 7f1016ffd700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f100806b020 0x7f100806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.025+0000 7f1016ffd700 1 -- 172.21.15.181:0/792147719 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f101c088e50 con 0x7f10280af450 2023-10-16T09:59:24.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.025+0000 7f1025d9b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f100806b020 0x7f100806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.026+0000 7f1025d9b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f100806b020 0x7f100806d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f10281e4fd0 tx=0x7f1018010000 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.030+0000 7f1016ffd700 1 -- 172.21.15.181:0/792147719 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f101c053c40 con 0x7f10280af450 2023-10-16T09:59:24.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.223+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1028067d20 con 0x7f10280af450 2023-10-16T09:59:24.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.224+0000 7f1016ffd700 1 -- 172.21.15.181:0/792147719 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+17258 (secure 0 0 0) 0x7f101c017090 con 0x7f10280af450 2023-10-16T09:59:24.226 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:24.226 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":75,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:23.238969+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:21.812039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:22.200991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f100806b020 msgr2=0x7f100806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f100806b020 0x7f100806d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f10281e4fd0 tx=0x7f1018010000 comp rx=0 tx=0).stop 2023-10-16T09:59:24.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10280af450 msgr2=0x7f10280b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10280af450 0x7f10280b5840 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f101c001f00 tx=0x7f101c00c860 comp rx=0 tx=0).stop 2023-10-16T09:59:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 shutdown_connections 2023-10-16T09:59:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f100806b020 0x7f100806d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f102815dfd0 0x7f10280b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10280af450 0x7f10280b5840 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 --2- 172.21.15.181:0/792147719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10280cd470 0x7f10281549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 >> 172.21.15.181:0/792147719 conn(0x7f10280d04a0 msgr2=0x7f102806d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 shutdown_connections 2023-10-16T09:59:24.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.226+0000 7f102cd6b700 1 -- 172.21.15.181:0/792147719 wait complete. 2023-10-16T09:59:24.240 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:24.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.359+0000 7f2f161ee700 1 Processor -- start 2023-10-16T09:59:24.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.360+0000 7f2f161ee700 1 -- start start 2023-10-16T09:59:24.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.360+0000 7f2f161ee700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f1015dfd0 0x7f2f100af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.360+0000 7f2f161ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.360+0000 7f2f161ee700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1006f870 con 0x7f2f100af450 2023-10-16T09:59:24.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.360+0000 7f2f161ee700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100cde40 con 0x7f2f1015dfd0 2023-10-16T09:59:24.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.360+0000 7f2f161ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100d1160 con 0x7f2f100cd470 2023-10-16T09:59:24.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0f7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f1015dfd0 0x7f2f100af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f07fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2f100af450 0x7f2f1006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:36702/0 (socket says 172.21.15.181:36702) 2023-10-16T09:59:24.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f07fff700 1 -- 172.21.15.181:0/4253237292 learned_addr learned my addr 172.21.15.181:0/4253237292 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:24.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 msgr2=0x7f2f100cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:24.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/4253237292 >> v1:172.21.15.190:6789/0 conn(0x7f2f100af450 legacy=0x7f2f1006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0f7fe700 1 -- 172.21.15.181:0/4253237292 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100cc8e0 con 0x7f2f1015dfd0 2023-10-16T09:59:24.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0effd700 1 -- 172.21.15.181:0/4253237292 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097501406 0 0) 0x7f2f1006f870 con 0x7f2f100af450 2023-10-16T09:59:24.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0ffff700 1 --2- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:24.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1015dfd0 0x7f2f100af9a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009820 tx=0x7f2f00009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e620caad4d37ad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0effd700 1 -- 172.21.15.181:0/4253237292 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00005e30 con 0x7f2f1015dfd0 2023-10-16T09:59:24.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0effd700 1 -- 172.21.15.181:0/4253237292 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00006030 con 0x7f2f1015dfd0 2023-10-16T09:59:24.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.361+0000 7f2f0effd700 1 -- 172.21.15.181:0/4253237292 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f000063c0 con 0x7f2f1015dfd0 2023-10-16T09:59:24.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 -- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1015dfd0 msgr2=0x7f2f100af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 --2- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1015dfd0 0x7f2f100af9a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009820 tx=0x7f2f00009b30 comp rx=0 tx=0).stop 2023-10-16T09:59:24.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 -- 172.21.15.181:0/4253237292 shutdown_connections 2023-10-16T09:59:24.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 --2- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 --2- 172.21.15.181:0/4253237292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1015dfd0 0x7f2f100af9a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 -- 172.21.15.181:0/4253237292 >> 172.21.15.181:0/4253237292 conn(0x7f2f100d04a0 msgr2=0x7f2f100d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:24.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 -- 172.21.15.181:0/4253237292 shutdown_connections 2023-10-16T09:59:24.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.362+0000 7f2f161ee700 1 -- 172.21.15.181:0/4253237292 wait complete. 2023-10-16T09:59:24.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.363+0000 7f2f161ee700 1 Processor -- start 2023-10-16T09:59:24.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.363+0000 7f2f161ee700 1 -- start start 2023-10-16T09:59:24.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f161ee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f100af450 0x7f2f101542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f161ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f161ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1015dfd0 0x7f2f100c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f161ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10154800 con 0x7f2f1015dfd0 2023-10-16T09:59:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f161ee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100b15e0 con 0x7f2f100af450 2023-10-16T09:59:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f161ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100c8b00 con 0x7f2f100cd470 2023-10-16T09:59:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1015dfd0 0x7f2f100c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1015dfd0 0x7f2f100c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41690/0 (socket says 172.21.15.181:41690) 2023-10-16T09:59:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0ffff700 1 -- 172.21.15.181:0/2323431448 learned_addr learned my addr 172.21.15.181:0/2323431448 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f07fff700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f100af450 0x7f2f101542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0ffff700 1 -- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 msgr2=0x7f2f100c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0ffff700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0ffff700 1 -- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f100af450 msgr2=0x7f2f101542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.364+0000 7f2f0ffff700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f100af450 0x7f2f101542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.365+0000 7f2f0ffff700 1 -- 172.21.15.181:0/2323431448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08009a70 con 0x7f2f1015dfd0 2023-10-16T09:59:24.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.365+0000 7f2f0ffff700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1015dfd0 0x7f2f100c9c20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800efc0 tx=0x7f2f0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.365+0000 7f2f0cff9700 1 -- 172.21.15.181:0/2323431448 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800ca90 con 0x7f2f1015dfd0 2023-10-16T09:59:24.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.365+0000 7f2f0cff9700 1 -- 172.21.15.181:0/2323431448 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0800f040 con 0x7f2f1015dfd0 2023-10-16T09:59:24.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.365+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f00009430 con 0x7f2f1015dfd0 2023-10-16T09:59:24.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.365+0000 7f2f0cff9700 1 -- 172.21.15.181:0/2323431448 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800ce10 con 0x7f2f1015dfd0 2023-10-16T09:59:24.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.365+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f100c8260 con 0x7f2f1015dfd0 2023-10-16T09:59:24.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.366+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f100b13b0 con 0x7f2f1015dfd0 2023-10-16T09:59:24.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.366+0000 7f2f0cff9700 1 -- 172.21.15.181:0/2323431448 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2f08014480 con 0x7f2f1015dfd0 2023-10-16T09:59:24.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.367+0000 7f2f0cff9700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f2ef006b050 0x7f2ef006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.367+0000 7f2f0cff9700 1 -- 172.21.15.181:0/2323431448 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f2f08089f80 con 0x7f2f1015dfd0 2023-10-16T09:59:24.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.367+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f2ef006b050 0x7f2ef006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.367+0000 7f2f0f7fe700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f2ef006b050 0x7f2ef006d510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f00000c00 tx=0x7f2f00008070 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.371+0000 7f2f0cff9700 1 -- 172.21.15.181:0/2323431448 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2f080bcfe0 con 0x7f2f1015dfd0 2023-10-16T09:59:24.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.564+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f100b1750 con 0x7f2f1015dfd0 2023-10-16T09:59:24.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.565+0000 7f2f0cff9700 1 -- 172.21.15.181:0/2323431448 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+17258 (secure 0 0 0) 0x7f2f100b13b0 con 0x7f2f1015dfd0 2023-10-16T09:59:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:24.567 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":75,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:23.238969+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:21.812039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:22.200991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:24.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f2ef006b050 msgr2=0x7f2ef006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f2ef006b050 0x7f2ef006d510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f00000c00 tx=0x7f2f00008070 comp rx=0 tx=0).stop 2023-10-16T09:59:24.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1015dfd0 msgr2=0x7f2f100c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1015dfd0 0x7f2f100c9c20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800efc0 tx=0x7f2f0800c3a0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 shutdown_connections 2023-10-16T09:59:24.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f2ef006b050 0x7f2ef006d510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f1015dfd0 0x7f2f100c9c20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f100af450 0x7f2f101542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 --2- 172.21.15.181:0/2323431448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f100cd470 0x7f2f100c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 >> 172.21.15.181:0/2323431448 conn(0x7f2f100d04a0 msgr2=0x7f2f1015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:24.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 shutdown_connections 2023-10-16T09:59:24.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.567+0000 7f2f161ee700 1 -- 172.21.15.181:0/2323431448 wait complete. 2023-10-16T09:59:24.581 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T09:59:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.703+0000 7f46845d4700 1 Processor -- start 2023-10-16T09:59:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.703+0000 7f46845d4700 1 -- start start 2023-10-16T09:59:24.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.704+0000 7f46845d4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f467c06afb0 0x7f467c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.704+0000 7f46845d4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0cef00 con 0x7f467c072760 2023-10-16T09:59:24.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.704+0000 7f46845d4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0cec20 con 0x7f467c15da70 2023-10-16T09:59:24.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.704+0000 7f46845d4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c071c80 con 0x7f467c06afb0 2023-10-16T09:59:24.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.704+0000 7f4681b6f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f467c072760 0x7f467c06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52772/0 (socket says 172.21.15.181:52772) 2023-10-16T09:59:24.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.704+0000 7f4681b6f700 1 -- 172.21.15.181:0/2434288921 learned_addr learned my addr 172.21.15.181:0/2434288921 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:24.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f4682b71700 1 --2- 172.21.15.181:0/2434288921 >> v2:172.21.15.190:3300/0 conn(0x7f467c06afb0 0x7f467c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f468136e700 1 -- 172.21.15.181:0/2434288921 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546273612 0 0) 0x7f467c0cef00 con 0x7f467c072760 2023-10-16T09:59:24.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f468136e700 1 -- 172.21.15.181:0/2434288921 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46680035b0 con 0x7f467c072760 2023-10-16T09:59:24.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f4682b71700 1 -- 172.21.15.181:0/2434288921 >> v1:172.21.15.190:6789/0 conn(0x7f467c15da70 legacy=0x7f467c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f468136e700 1 -- 172.21.15.181:0/2434288921 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4163569352 0 0) 0x7f46680035b0 con 0x7f467c072760 2023-10-16T09:59:24.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f4682b71700 1 -- 172.21.15.181:0/2434288921 >> v1:172.21.15.181:6789/0 conn(0x7f467c072760 legacy=0x7f467c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f4682b71700 1 -- 172.21.15.181:0/2434288921 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c071f60 con 0x7f467c06afb0 2023-10-16T09:59:24.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f4682b71700 1 --2- 172.21.15.181:0/2434288921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1615f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4674009d90 tx=0x7f467400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b475763246f851 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f468136e700 1 -- 172.21.15.181:0/2434288921 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400e070 con 0x7f467c06afb0 2023-10-16T09:59:24.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f468136e700 1 -- 172.21.15.181:0/2434288921 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4674003a20 con 0x7f467c06afb0 2023-10-16T09:59:24.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.705+0000 7f468136e700 1 -- 172.21.15.181:0/2434288921 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678001670 con 0x7f467c06afb0 2023-10-16T09:59:24.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.706+0000 7f46845d4700 1 -- 172.21.15.181:0/2434288921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 msgr2=0x7f467c1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.706+0000 7f46845d4700 1 --2- 172.21.15.181:0/2434288921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1615f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4674009d90 tx=0x7f467400b330 comp rx=0 tx=0).stop 2023-10-16T09:59:24.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.706+0000 7f46845d4700 1 -- 172.21.15.181:0/2434288921 shutdown_connections 2023-10-16T09:59:24.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.706+0000 7f46845d4700 1 --2- 172.21.15.181:0/2434288921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1615f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.706+0000 7f46845d4700 1 -- 172.21.15.181:0/2434288921 >> 172.21.15.181:0/2434288921 conn(0x7f467c0cd010 msgr2=0x7f467c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:24.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.706+0000 7f46845d4700 1 -- 172.21.15.181:0/2434288921 shutdown_connections 2023-10-16T09:59:24.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.706+0000 7f46845d4700 1 -- 172.21.15.181:0/2434288921 wait complete. 2023-10-16T09:59:24.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.707+0000 7f46845d4700 1 Processor -- start 2023-10-16T09:59:24.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.707+0000 7f46845d4700 1 -- start start 2023-10-16T09:59:24.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f46845d4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f46845d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c072760 0x7f467c1f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f46845d4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f467c15da70 0x7f467c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f46845d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1580c0 con 0x7f467c072760 2023-10-16T09:59:24.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f46845d4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0c40c0 con 0x7f467c06afb0 2023-10-16T09:59:24.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f46845d4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0b4e80 con 0x7f467c15da70 2023-10-16T09:59:24.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c072760 0x7f467c1f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c072760 0x7f467c1f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41702/0 (socket says 172.21.15.181:41702) 2023-10-16T09:59:24.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 -- 172.21.15.181:0/432049713 learned_addr learned my addr 172.21.15.181:0/432049713 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:24.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4682370700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 -- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f467c15da70 msgr2=0x7f467c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4682b71700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f467c15da70 0x7f467c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f467c15da70 0x7f467c0b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 -- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 msgr2=0x7f467c1f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:24.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:24.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 -- 172.21.15.181:0/432049713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674009a70 con 0x7f467c072760 2023-10-16T09:59:24.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4682370700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:24.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.708+0000 7f4681b6f700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c072760 0x7f467c1f6720 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f46780011e0 tx=0x7f467800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.709+0000 7f4672ffd700 1 -- 172.21.15.181:0/432049713 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678007560 con 0x7f467c072760 2023-10-16T09:59:24.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.709+0000 7f4672ffd700 1 -- 172.21.15.181:0/432049713 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46780076c0 con 0x7f467c072760 2023-10-16T09:59:24.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.709+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c0b3de0 con 0x7f467c072760 2023-10-16T09:59:24.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.709+0000 7f4672ffd700 1 -- 172.21.15.181:0/432049713 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678004a90 con 0x7f467c072760 2023-10-16T09:59:24.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.709+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c0b4680 con 0x7f467c072760 2023-10-16T09:59:24.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.710+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c072220 con 0x7f467c072760 2023-10-16T09:59:24.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.710+0000 7f4672ffd700 1 -- 172.21.15.181:0/432049713 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4678004bf0 con 0x7f467c072760 2023-10-16T09:59:24.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.711+0000 7f4672ffd700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f466006b020 0x7f466006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:24.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.711+0000 7f4682370700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f466006b020 0x7f466006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:24.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.711+0000 7f4672ffd700 1 -- 172.21.15.181:0/432049713 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f4678088cf0 con 0x7f467c072760 2023-10-16T09:59:24.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.711+0000 7f4682370700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f466006b020 0x7f466006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f466c000f60 tx=0x7f466c009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:24.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.715+0000 7f4672ffd700 1 -- 172.21.15.181:0/432049713 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f467c072220 con 0x7f467c072760 2023-10-16T09:59:24.833 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-10-16T09:59:24.832+0000 7f294453d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:59:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:24.958+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f467c0b41d0 con 0x7f467c072760 2023-10-16T09:59:25.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.870+0000 7f4672ffd700 1 -- 172.21.15.181:0/432049713 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4678018020 con 0x7f467c072760 2023-10-16T09:59:25.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f466006b020 msgr2=0x7f466006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:25.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f466006b020 0x7f466006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f466c000f60 tx=0x7f466c009040 comp rx=0 tx=0).stop 2023-10-16T09:59:25.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c072760 msgr2=0x7f467c1f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:25.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c072760 0x7f467c1f6720 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f46780011e0 tx=0x7f467800cb00 comp rx=0 tx=0).stop 2023-10-16T09:59:25.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 shutdown_connections 2023-10-16T09:59:25.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f466006b020 0x7f466006d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:25.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f467c15da70 0x7f467c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:25.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f467c06afb0 0x7f467c1f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:25.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 --2- 172.21.15.181:0/432049713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c072760 0x7f467c1f6720 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:25.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 >> 172.21.15.181:0/432049713 conn(0x7f467c0cd010 msgr2=0x7f467c161560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:25.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 shutdown_connections 2023-10-16T09:59:25.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:25.873+0000 7f46845d4700 1 -- 172.21.15.181:0/432049713 wait complete. 2023-10-16T09:59:25.889 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T09:59:25.890 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T09:59:26.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.018+0000 7faddec94700 1 Processor -- start 2023-10-16T09:59:26.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.018+0000 7faddec94700 1 -- start start 2023-10-16T09:59:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7faddec94700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fadd815dbd0 0x7fadd80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7faddec94700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd815dfb0 con 0x7fadd80ce490 2023-10-16T09:59:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7faddec94700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd80cd3a0 con 0x7fadd80d0610 2023-10-16T09:59:26.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7faddec94700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd80cd830 con 0x7fadd815dbd0 2023-10-16T09:59:26.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7faddd231700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fadd80ce490 0x7fadd806b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52790/0 (socket says 172.21.15.181:52790) 2023-10-16T09:59:26.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7faddd231700 1 -- 172.21.15.181:0/25859039 learned_addr learned my addr 172.21.15.181:0/25859039 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:26.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7faddca30700 1 --2- 172.21.15.181:0/25859039 >> v2:172.21.15.190:3300/0 conn(0x7fadd815dbd0 0x7fadd80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554967454 0 0) 0x7fadd0001490 con 0x7fadd80ce490 2023-10-16T09:59:26.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadc80035b0 con 0x7fadd80ce490 2023-10-16T09:59:26.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1902297782 0 0) 0x7fadc80035b0 con 0x7fadd80ce490 2023-10-16T09:59:26.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 >> v2:172.21.15.190:3300/0 conn(0x7fadd815dbd0 msgr2=0x7fadd80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:26.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 --2- 172.21.15.181:0/25859039 >> v2:172.21.15.190:3300/0 conn(0x7fadd815dbd0 0x7fadd80d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 >> v1:172.21.15.190:6789/0 conn(0x7fadd80d0610 legacy=0x7fadd80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:26.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80d1fc0 con 0x7fadd80ce490 2023-10-16T09:59:26.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.019+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fadd0001b80 con 0x7fadd80ce490 2023-10-16T09:59:26.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 >> v1:172.21.15.181:6789/0 conn(0x7fadd80ce490 legacy=0x7fadd806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:26.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8003a00 0x7fadc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc8004400 0x7fadc80048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80cd830 con 0x7fadc80050d0 2023-10-16T09:59:26.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc800c030 con 0x7fadc8004400 2023-10-16T09:59:26.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc800c1f0 con 0x7fadc8003a00 2023-10-16T09:59:26.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddca30700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8003a00 0x7fadc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd7fff700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc8004400 0x7fadc80048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 -- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8003a00 msgr2=0x7fadc8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:26.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8003a00 0x7fadc8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 -- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc8004400 msgr2=0x7fadc80048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:26.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc8004400 0x7fadc80048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 -- 172.21.15.181:0/25859039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80035b0 con 0x7fadc80050d0 2023-10-16T09:59:26.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 -- 172.21.15.181:0/25859039 reap_dead start 2023-10-16T09:59:26.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7faddd231700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadc800ba10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fadd000a810 tx=0x7fadc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f19c56d8d7c787b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:26.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0012600 con 0x7fadc80050d0 2023-10-16T09:59:26.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0012780 con 0x7fadc80050d0 2023-10-16T09:59:26.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.020+0000 7fadd77fe700 1 -- 172.21.15.181:0/25859039 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0012b30 con 0x7fadc80050d0 2023-10-16T09:59:26.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.021+0000 7faddec94700 1 -- 172.21.15.181:0/25859039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 msgr2=0x7fadc800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:26.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.021+0000 7faddec94700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadc800ba10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fadd000a810 tx=0x7fadc8005880 comp rx=0 tx=0).stop 2023-10-16T09:59:26.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.021+0000 7faddec94700 1 -- 172.21.15.181:0/25859039 shutdown_connections 2023-10-16T09:59:26.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.021+0000 7faddec94700 1 --2- 172.21.15.181:0/25859039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadc800ba10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.021+0000 7faddec94700 1 -- 172.21.15.181:0/25859039 >> 172.21.15.181:0/25859039 conn(0x7fadd809a930 msgr2=0x7fadd80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:26.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.021+0000 7faddec94700 1 -- 172.21.15.181:0/25859039 shutdown_connections 2023-10-16T09:59:26.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.021+0000 7faddec94700 1 -- 172.21.15.181:0/25859039 wait complete. 2023-10-16T09:59:26.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.022+0000 7faddec94700 1 Processor -- start 2023-10-16T09:59:26.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.022+0000 7faddec94700 1 -- start start 2023-10-16T09:59:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddec94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadd814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddec94700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadd81501a0 0x7fadd814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddec94700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd814cad0 0x7fadd80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddec94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd814cf60 con 0x7fadc80050d0 2023-10-16T09:59:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddec94700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd814abf0 con 0x7fadd814cad0 2023-10-16T09:59:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddec94700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80b5ff0 con 0x7fadd81501a0 2023-10-16T09:59:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadd814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadd814fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41720/0 (socket says 172.21.15.181:41720) 2023-10-16T09:59:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 -- 172.21.15.181:0/2065056789 learned_addr learned my addr 172.21.15.181:0/2065056789 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7fadd7fff700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadd81501a0 0x7fadd814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 -- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadd81501a0 msgr2=0x7fadd814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:26.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadd81501a0 0x7fadd814c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 -- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd814cad0 msgr2=0x7fadd80b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:26.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddd231700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd814cad0 0x7fadd80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd814cad0 0x7fadd80b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.023+0000 7faddca30700 1 -- 172.21.15.181:0/2065056789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80b5d10 con 0x7fadc80050d0 2023-10-16T09:59:26.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.024+0000 7faddca30700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadd814fc60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0119a0 tx=0x7fadcc011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:26.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.024+0000 7fadd57fa700 1 -- 172.21.15.181:0/2065056789 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00f940 con 0x7fadc80050d0 2023-10-16T09:59:26.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.024+0000 7fadd57fa700 1 -- 172.21.15.181:0/2065056789 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc00faa0 con 0x7fadc80050d0 2023-10-16T09:59:26.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.024+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd814a5a0 con 0x7fadc80050d0 2023-10-16T09:59:26.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.024+0000 7fadd57fa700 1 -- 172.21.15.181:0/2065056789 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00fe30 con 0x7fadc80050d0 2023-10-16T09:59:26.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.024+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd80b5560 con 0x7fadc80050d0 2023-10-16T09:59:26.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.025+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd8067d20 con 0x7fadc80050d0 2023-10-16T09:59:26.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.025+0000 7fadd57fa700 1 -- 172.21.15.181:0/2065056789 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fadcc014440 con 0x7fadc80050d0 2023-10-16T09:59:26.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.026+0000 7fadd57fa700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fadbc06b020 0x7fadbc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:26.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.026+0000 7fadd57fa700 1 -- 172.21.15.181:0/2065056789 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fadcc089fc0 con 0x7fadc80050d0 2023-10-16T09:59:26.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.026+0000 7fadd7fff700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fadbc06b020 0x7fadbc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:26.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.026+0000 7fadd7fff700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fadbc06b020 0x7fadbc06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadd8150c10 tx=0x7fadc4009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:26.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.030+0000 7fadd57fa700 1 -- 172.21.15.181:0/2065056789 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fadcc0bcfe0 con 0x7fadc80050d0 2023-10-16T09:59:26.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.269+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fadd80cebb0 con 0x7fadc80050d0 2023-10-16T09:59:26.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.872+0000 7fadd57fa700 1 -- 172.21.15.181:0/2065056789 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fadcc058980 con 0x7fadc80050d0 2023-10-16T09:59:26.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fadbc06b020 msgr2=0x7fadbc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:26.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fadbc06b020 0x7fadbc06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadd8150c10 tx=0x7fadc4009040 comp rx=0 tx=0).stop 2023-10-16T09:59:26.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 msgr2=0x7fadd814fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:26.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadd814fc60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0119a0 tx=0x7fadcc011d60 comp rx=0 tx=0).stop 2023-10-16T09:59:26.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 shutdown_connections 2023-10-16T09:59:26.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fadbc06b020 0x7fadbc06d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadd814cad0 0x7fadd80b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80050d0 0x7fadd814fc60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 --2- 172.21.15.181:0/2065056789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadd81501a0 0x7fadd814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:26.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 >> 172.21.15.181:0/2065056789 conn(0x7fadd809a930 msgr2=0x7fadd80ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:26.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 shutdown_connections 2023-10-16T09:59:26.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:26.875+0000 7faddec94700 1 -- 172.21.15.181:0/2065056789 wait complete. 2023-10-16T09:59:26.890 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T09:59:26.891 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:27.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.012+0000 7fd379ca0700 1 Processor -- start 2023-10-16T09:59:27.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.012+0000 7fd379ca0700 1 -- start start 2023-10-16T09:59:27.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd379ca0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd37415e6d0 0x7fd3740d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd379ca0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37406ed00 con 0x7fd37415eab0 2023-10-16T09:59:27.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd379ca0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3740d02d0 con 0x7fd3740cb0c0 2023-10-16T09:59:27.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd379ca0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37406e0f0 con 0x7fd37415e6d0 2023-10-16T09:59:27.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd3737fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd37415e6d0 0x7fd3740d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd3737fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd37415e6d0 0x7fd3740d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41728/0 (socket says 172.21.15.181:41728) 2023-10-16T09:59:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd3737fe700 1 -- 172.21.15.181:0/126617912 learned_addr learned my addr 172.21.15.181:0/126617912 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.013+0000 7fd3737fe700 1 -- 172.21.15.181:0/126617912 >> v1:172.21.15.190:6789/0 conn(0x7fd3740cb0c0 legacy=0x7fd37415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd3737fe700 1 -- 172.21.15.181:0/126617912 >> v1:172.21.15.181:6789/0 conn(0x7fd37415eab0 legacy=0x7fd37415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd3727fc700 1 -- 172.21.15.181:0/126617912 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190723366 0 0) 0x7fd37406ed00 con 0x7fd37415eab0 2023-10-16T09:59:27.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd3737fe700 1 -- 172.21.15.181:0/126617912 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37406e3d0 con 0x7fd37415e6d0 2023-10-16T09:59:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd3737fe700 1 --2- 172.21.15.181:0/126617912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415e6d0 0x7fd3740d0fc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd368006fd0 tx=0x7fd368009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1304a4ce5fada45a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd3727fc700 1 -- 172.21.15.181:0/126617912 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368005e30 con 0x7fd37415e6d0 2023-10-16T09:59:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd3727fc700 1 -- 172.21.15.181:0/126617912 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368006030 con 0x7fd37415e6d0 2023-10-16T09:59:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd3727fc700 1 -- 172.21.15.181:0/126617912 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680063e0 con 0x7fd37415e6d0 2023-10-16T09:59:27.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd379ca0700 1 -- 172.21.15.181:0/126617912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415e6d0 msgr2=0x7fd3740d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd379ca0700 1 --2- 172.21.15.181:0/126617912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415e6d0 0x7fd3740d0fc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd368006fd0 tx=0x7fd368009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd379ca0700 1 -- 172.21.15.181:0/126617912 shutdown_connections 2023-10-16T09:59:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd379ca0700 1 --2- 172.21.15.181:0/126617912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415e6d0 0x7fd3740d0fc0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd379ca0700 1 -- 172.21.15.181:0/126617912 >> 172.21.15.181:0/126617912 conn(0x7fd3740d04a0 msgr2=0x7fd3740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:27.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.014+0000 7fd379ca0700 1 -- 172.21.15.181:0/126617912 shutdown_connections 2023-10-16T09:59:27.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.015+0000 7fd379ca0700 1 -- 172.21.15.181:0/126617912 wait complete. 2023-10-16T09:59:27.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.016+0000 7fd379ca0700 1 Processor -- start 2023-10-16T09:59:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.016+0000 7fd379ca0700 1 -- start start 2023-10-16T09:59:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3740cb0c0 0x7fd374149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd37415e6d0 0x7fd3740b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415eab0 0x7fd3740b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3741505a0 con 0x7fd37415eab0 2023-10-16T09:59:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740b56e0 con 0x7fd37415e6d0 2023-10-16T09:59:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740b4db0 con 0x7fd3740cb0c0 2023-10-16T09:59:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415eab0 0x7fd3740b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415eab0 0x7fd3740b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41738/0 (socket says 172.21.15.181:41738) 2023-10-16T09:59:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 -- 172.21.15.181:0/1115190336 learned_addr learned my addr 172.21.15.181:0/1115190336 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd3737fe700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3740cb0c0 0x7fd374149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd372ffd700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd37415e6d0 0x7fd3740b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 -- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3740cb0c0 msgr2=0x7fd374149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3740cb0c0 0x7fd374149a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 -- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd37415e6d0 msgr2=0x7fd3740b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd37415e6d0 0x7fd3740b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 -- 172.21.15.181:0/1115190336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368009500 con 0x7fd37415eab0 2023-10-16T09:59:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd373fff700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415eab0 0x7fd3740b3fe0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00b970 tx=0x7fd36c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd35bfff700 1 -- 172.21.15.181:0/1115190336 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00d860 con 0x7fd37415eab0 2023-10-16T09:59:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd35bfff700 1 -- 172.21.15.181:0/1115190336 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c00d9c0 con 0x7fd37415eab0 2023-10-16T09:59:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3740b5090 con 0x7fd37415eab0 2023-10-16T09:59:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd35bfff700 1 -- 172.21.15.181:0/1115190336 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00dd70 con 0x7fd37415eab0 2023-10-16T09:59:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.017+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd374149fa0 con 0x7fd37415eab0 2023-10-16T09:59:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.018+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd374067d20 con 0x7fd37415eab0 2023-10-16T09:59:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.019+0000 7fd35bfff700 1 -- 172.21.15.181:0/1115190336 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd36c00e430 con 0x7fd37415eab0 2023-10-16T09:59:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.019+0000 7fd35bfff700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd36006b050 0x7fd36006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.019+0000 7fd3737fe700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd36006b050 0x7fd36006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.019+0000 7fd35bfff700 1 -- 172.21.15.181:0/1115190336 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fd36c089be0 con 0x7fd37415eab0 2023-10-16T09:59:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.020+0000 7fd3737fe700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd36006b050 0x7fd36006d510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd368013040 tx=0x7fd368008070 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.024+0000 7fd35bfff700 1 -- 172.21.15.181:0/1115190336 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd36c054820 con 0x7fd37415eab0 2023-10-16T09:59:27.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.257+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3740cb610 con 0x7fd37415eab0 2023-10-16T09:59:27.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.258+0000 7fd35bfff700 1 -- 172.21.15.181:0/1115190336 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fd36c00ded0 con 0x7fd37415eab0 2023-10-16T09:59:27.260 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:27.260 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2852689416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2852689416},{"type":"v1","addr":"172.21.15.190:6834","nonce":2852689416}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2993219349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2993219349},{"type":"v1","addr":"172.21.15.181:6836","nonce":2993219349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2504788411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2504788411},{"type":"v1","addr":"172.21.15.181:6837","nonce":2504788411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:23.239300+0000","modified":"2023-10-16T09:59:26.271260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1948728404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1948728404},{"type":"v1","addr":"172.21.15.190:6835","nonce":1948728404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T09:59:27.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd36006b050 msgr2=0x7fd36006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd36006b050 0x7fd36006d510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd368013040 tx=0x7fd368008070 comp rx=0 tx=0).stop 2023-10-16T09:59:27.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415eab0 msgr2=0x7fd3740b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415eab0 0x7fd3740b3fe0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00b970 tx=0x7fd36c00bc80 comp rx=0 tx=0).stop 2023-10-16T09:59:27.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 shutdown_connections 2023-10-16T09:59:27.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fd36006b050 0x7fd36006d510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3740cb0c0 0x7fd374149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37415eab0 0x7fd3740b3fe0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 --2- 172.21.15.181:0/1115190336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd37415e6d0 0x7fd3740b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.260+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 >> 172.21.15.181:0/1115190336 conn(0x7fd3740d04a0 msgr2=0x7fd3740aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:27.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.261+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 shutdown_connections 2023-10-16T09:59:27.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.261+0000 7fd379ca0700 1 -- 172.21.15.181:0/1115190336 wait complete. 2023-10-16T09:59:27.265 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 32 2023-10-16T09:59:27.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:27.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.393+0000 7fbae43b7700 1 Processor -- start 2023-10-16T09:59:27.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.394+0000 7fbae43b7700 1 -- start start 2023-10-16T09:59:27.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.394+0000 7fbae43b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbadc15b990 0x7fbadc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.394+0000 7fbae43b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbadc0d0370 0x7fbadc0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.394+0000 7fbae43b7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbadc0cc510 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.394+0000 7fbae43b7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbadc0d24f0 con 0x7fbadc15b990 2023-10-16T09:59:27.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.394+0000 7fbae43b7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbadc0cc7f0 con 0x7fbadc0d0370 2023-10-16T09:59:27.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae2153700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbadc15b990 0x7fbadc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae1952700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbadc0cd2d0 0x7fbadc0cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52820/0 (socket says 172.21.15.181:52820) 2023-10-16T09:59:27.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae1952700 1 -- 172.21.15.181:0/4023977342 learned_addr learned my addr 172.21.15.181:0/4023977342 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:27.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae2954700 1 --2- 172.21.15.181:0/4023977342 >> v2:172.21.15.190:3300/0 conn(0x7fbadc0d0370 0x7fbadc0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae2153700 1 -- 172.21.15.181:0/4023977342 >> v2:172.21.15.190:3300/0 conn(0x7fbadc0d0370 msgr2=0x7fbadc0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:27.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae2153700 1 --2- 172.21.15.181:0/4023977342 >> v2:172.21.15.190:3300/0 conn(0x7fbadc0d0370 0x7fbadc0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae2153700 1 -- 172.21.15.181:0/4023977342 >> v1:172.21.15.181:6789/0 conn(0x7fbadc0cd2d0 legacy=0x7fbadc0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae1151700 1 -- 172.21.15.181:0/4023977342 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952453340 0 0) 0x7fbadc0cc510 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae2153700 1 -- 172.21.15.181:0/4023977342 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbadc0ccad0 con 0x7fbadc15b990 2023-10-16T09:59:27.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae2153700 1 --2- 172.21.15.181:0/4023977342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc15b990 0x7fbadc0ce8f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbadc09ae40 tx=0x7fbacc009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5653d70f5b4a7821 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae1151700 1 -- 172.21.15.181:0/4023977342 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc005e30 con 0x7fbadc15b990 2023-10-16T09:59:27.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae1151700 1 -- 172.21.15.181:0/4023977342 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc006030 con 0x7fbadc15b990 2023-10-16T09:59:27.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.395+0000 7fbae1151700 1 -- 172.21.15.181:0/4023977342 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0063c0 con 0x7fbadc15b990 2023-10-16T09:59:27.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 -- 172.21.15.181:0/4023977342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc15b990 msgr2=0x7fbadc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4023977342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc15b990 0x7fbadc0ce8f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbadc09ae40 tx=0x7fbacc009b00 comp rx=0 tx=0).stop 2023-10-16T09:59:27.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 -- 172.21.15.181:0/4023977342 shutdown_connections 2023-10-16T09:59:27.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4023977342 >> v2:172.21.15.190:3300/0 conn(0x7fbadc0d0370 0x7fbadc0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4023977342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc15b990 0x7fbadc0ce8f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 -- 172.21.15.181:0/4023977342 >> 172.21.15.181:0/4023977342 conn(0x7fbadc09a9a0 msgr2=0x7fbadc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:27.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 -- 172.21.15.181:0/4023977342 shutdown_connections 2023-10-16T09:59:27.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.396+0000 7fbae43b7700 1 -- 172.21.15.181:0/4023977342 wait complete. 2023-10-16T09:59:27.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.397+0000 7fbae43b7700 1 Processor -- start 2023-10-16T09:59:27.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.397+0000 7fbae43b7700 1 -- start start 2023-10-16T09:59:27.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0cd2d0 0x7fbadc0cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbadc0d0370 0x7fbadc157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbadc15b990 0x7fbadc1f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc1583e0 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc0c6dc0 con 0x7fbadc15b990 2023-10-16T09:59:27.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc1f14e0 con 0x7fbadc0d0370 2023-10-16T09:59:27.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0cd2d0 0x7fbadc0cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0cd2d0 0x7fbadc0cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41748/0 (socket says 172.21.15.181:41748) 2023-10-16T09:59:27.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 -- 172.21.15.181:0/4262759608 learned_addr learned my addr 172.21.15.181:0/4262759608 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:27.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae1952700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbadc0d0370 0x7fbadc157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2954700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbadc15b990 0x7fbadc1f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 -- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbadc0d0370 msgr2=0x7fbadc157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbadc0d0370 0x7fbadc157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 -- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbadc15b990 msgr2=0x7fbadc1f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbadc15b990 0x7fbadc1f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 -- 172.21.15.181:0/4262759608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009430 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2954700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbadc15b990 0x7fbadc1f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:27.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae2153700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0cd2d0 0x7fbadc0cb160 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009fd0 tx=0x7fbacc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbad2ffd700 1 -- 172.21.15.181:0/4262759608 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc004040 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbad2ffd700 1 -- 172.21.15.181:0/4262759608 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc020070 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc1f1790 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbad2ffd700 1 -- 172.21.15.181:0/4262759608 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc004400 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.398+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc1f1d50 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.399+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc067d20 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.400+0000 7fbad2ffd700 1 -- 172.21.15.181:0/4262759608 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbacc022030 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.400+0000 7fbad2ffd700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fbac006b020 0x7fbac006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.400+0000 7fbad2ffd700 1 -- 172.21.15.181:0/4262759608 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fbacc08ad90 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.400+0000 7fbae1952700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fbac006b020 0x7fbac006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.401+0000 7fbae1952700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fbac006b020 0x7fbac006d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000f60 tx=0x7fbad8007be0 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.405+0000 7fbad2ffd700 1 -- 172.21.15.181:0/4262759608 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbacc055b80 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.598+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbadc0cfa40 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.599+0000 7fbad2ffd700 1 -- 172.21.15.181:0/4262759608 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+17245 (secure 0 0 0) 0x7fbacc0591a0 con 0x7fbadc0cd2d0 2023-10-16T09:59:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:27.601 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":76,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:24.858704+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:21.812039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:22.200991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:27.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fbac006b020 msgr2=0x7fbac006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fbac006b020 0x7fbac006d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000f60 tx=0x7fbad8007be0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0cd2d0 msgr2=0x7fbadc0cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0cd2d0 0x7fbadc0cb160 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009fd0 tx=0x7fbacc004750 comp rx=0 tx=0).stop 2023-10-16T09:59:27.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 shutdown_connections 2023-10-16T09:59:27.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fbac006b020 0x7fbac006d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbadc15b990 0x7fbadc1f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0cd2d0 0x7fbadc0cb160 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 --2- 172.21.15.181:0/4262759608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbadc0d0370 0x7fbadc157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 >> 172.21.15.181:0/4262759608 conn(0x7fbadc09a9a0 msgr2=0x7fbadc15e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:27.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 shutdown_connections 2023-10-16T09:59:27.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.601+0000 7fbae43b7700 1 -- 172.21.15.181:0/4262759608 wait complete. 2023-10-16T09:59:27.615 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T09:59:27.615 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:59:27.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.734+0000 7f4221b67700 1 Processor -- start 2023-10-16T09:59:27.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.734+0000 7f4221b67700 1 -- start start 2023-10-16T09:59:27.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f4221b67700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f421c15b8b0 0x7f421c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f4221b67700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421c066c90 con 0x7f421c0b22a0 2023-10-16T09:59:27.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f4221b67700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421c066740 con 0x7f421c06e010 2023-10-16T09:59:27.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f4221b67700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421c0669b0 con 0x7f421c15b8b0 2023-10-16T09:59:27.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f421b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f421c15b8b0 0x7f421c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f421affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f421c0b22a0 0x7f421c15feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52834/0 (socket says 172.21.15.181:52834) 2023-10-16T09:59:27.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f421affd700 1 -- 172.21.15.181:0/3945376362 learned_addr learned my addr 172.21.15.181:0/3945376362 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:27.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f421b7fe700 1 -- 172.21.15.181:0/3945376362 >> v1:172.21.15.190:6789/0 conn(0x7f421c06e010 legacy=0x7f421c06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:27.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f421b7fe700 1 -- 172.21.15.181:0/3945376362 >> v1:172.21.15.181:6789/0 conn(0x7f421c0b22a0 legacy=0x7f421c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.735+0000 7f421a7fc700 1 -- 172.21.15.181:0/3945376362 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306871694 0 0) 0x7f420c001110 con 0x7f421c0b22a0 2023-10-16T09:59:27.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f421b7fe700 1 -- 172.21.15.181:0/3945376362 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c15f6b0 con 0x7f421c15b8b0 2023-10-16T09:59:27.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f421b7fe700 1 --2- 172.21.15.181:0/3945376362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c15b8b0 0x7f421c0cc160 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4204009840 tx=0x7f4204009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddadbf11fd02e514 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f421a7fc700 1 -- 172.21.15.181:0/3945376362 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204011830 con 0x7f421c15b8b0 2023-10-16T09:59:27.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f421a7fc700 1 -- 172.21.15.181:0/3945376362 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4204011990 con 0x7f421c15b8b0 2023-10-16T09:59:27.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f421a7fc700 1 -- 172.21.15.181:0/3945376362 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204011d20 con 0x7f421c15b8b0 2023-10-16T09:59:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f4221b67700 1 -- 172.21.15.181:0/3945376362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c15b8b0 msgr2=0x7f421c0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f4221b67700 1 --2- 172.21.15.181:0/3945376362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c15b8b0 0x7f421c0cc160 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4204009840 tx=0x7f4204009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f4221b67700 1 -- 172.21.15.181:0/3945376362 shutdown_connections 2023-10-16T09:59:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f4221b67700 1 --2- 172.21.15.181:0/3945376362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c15b8b0 0x7f421c0cc160 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f4221b67700 1 -- 172.21.15.181:0/3945376362 >> 172.21.15.181:0/3945376362 conn(0x7f421c0b1780 msgr2=0x7f421c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f4221b67700 1 -- 172.21.15.181:0/3945376362 shutdown_connections 2023-10-16T09:59:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.736+0000 7f4221b67700 1 -- 172.21.15.181:0/3945376362 wait complete. 2023-10-16T09:59:27.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.737+0000 7f4221b67700 1 Processor -- start 2023-10-16T09:59:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f4221b67700 1 -- start start 2023-10-16T09:59:27.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f4221b67700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f421c06e010 0x7f421c09de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f4221b67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c0b22a0 0x7f421c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f4221b67700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421c15b8b0 0x7f421c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f4221b67700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c157480 con 0x7f421c0b22a0 2023-10-16T09:59:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f4221b67700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0ce270 con 0x7f421c15b8b0 2023-10-16T09:59:27.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f4221b67700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c15eea0 con 0x7f421c06e010 2023-10-16T09:59:27.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.738+0000 7f421affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c0b22a0 0x7f421c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c0b22a0 0x7f421c156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41762/0 (socket says 172.21.15.181:41762) 2023-10-16T09:59:27.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 -- 172.21.15.181:0/4129314883 learned_addr learned my addr 172.21.15.181:0/4129314883 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:27.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421bfff700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421c15b8b0 0x7f421c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421b7fe700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f421c06e010 0x7f421c09de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 -- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f421c06e010 msgr2=0x7f421c09de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f421c06e010 0x7f421c09de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 -- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421c15b8b0 msgr2=0x7f421c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421c15b8b0 0x7f421c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 -- 172.21.15.181:0/4129314883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42040094a0 con 0x7f421c0b22a0 2023-10-16T09:59:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421bfff700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421c15b8b0 0x7f421c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f421affd700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c0b22a0 0x7f421c156f40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c520 tx=0x7f420c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f4213fff700 1 -- 172.21.15.181:0/4129314883 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c004c80 con 0x7f421c0b22a0 2023-10-16T09:59:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f4213fff700 1 -- 172.21.15.181:0/4129314883 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c004e20 con 0x7f421c0b22a0 2023-10-16T09:59:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421c0ce4c0 con 0x7f421c0b22a0 2023-10-16T09:59:27.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f4213fff700 1 -- 172.21.15.181:0/4129314883 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c007310 con 0x7f421c0b22a0 2023-10-16T09:59:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.739+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421c0cea80 con 0x7f421c0b22a0 2023-10-16T09:59:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.740+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421c067d20 con 0x7f421c0b22a0 2023-10-16T09:59:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.741+0000 7f4213fff700 1 -- 172.21.15.181:0/4129314883 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f420c007470 con 0x7f421c0b22a0 2023-10-16T09:59:27.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.741+0000 7f4213fff700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f41fc06b020 0x7f41fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.741+0000 7f4213fff700 1 -- 172.21.15.181:0/4129314883 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f420c089550 con 0x7f421c0b22a0 2023-10-16T09:59:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.741+0000 7f421b7fe700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f41fc06b020 0x7f41fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.742+0000 7f421b7fe700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f41fc06b020 0x7f41fc06d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4204009b90 tx=0x7f4204008030 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:27.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.746+0000 7f4213fff700 1 -- 172.21.15.181:0/4129314883 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f420c054340 con 0x7f421c0b22a0 2023-10-16T09:59:27.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.936+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 --> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f421c06ec90 con 0x7f41fc06b020 2023-10-16T09:59:27.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.944+0000 7f4213fff700 1 -- 172.21.15.181:0/4129314883 <== mgr.6222 v2:172.21.15.181:6832/3602430955 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289400 (secure 0 0 0) 0x7f4204011b80 con 0x7f41fc06b020 2023-10-16T09:59:27.949 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:27.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f41fc06b020 msgr2=0x7f41fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f41fc06b020 0x7f41fc06d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4204009b90 tx=0x7f4204008030 comp rx=0 tx=0).stop 2023-10-16T09:59:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c0b22a0 msgr2=0x7f421c156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c0b22a0 0x7f421c156f40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c520 tx=0x7f420c00c8e0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 shutdown_connections 2023-10-16T09:59:27.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f41fc06b020 0x7f41fc06d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f421c06e010 0x7f421c09de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f421c15b8b0 0x7f421c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 --2- 172.21.15.181:0/4129314883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f421c0b22a0 0x7f421c156f40 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:27.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 >> 172.21.15.181:0/4129314883 conn(0x7f421c0b1780 msgr2=0x7f421c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:27.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 shutdown_connections 2023-10-16T09:59:27.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:27.947+0000 7f4221b67700 1 -- 172.21.15.181:0/4129314883 wait complete. 2023-10-16T09:59:27.954 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:59:27.961 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T09:59:27.656795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306376,"kb_used_data":8348,"kb_used_omap":12,"kb_used_meta":297907,"kb_avail":754668344,"statfs":{"total":773094113280,"available":772780384256,"internally_reserved":0,"allocated":8548352,"data_stored":4683888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":305057336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.823297"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886883+0000","last_change":"2023-10-16T09:59:23.871596+0000","last_active":"2023-10-16T09:59:24.886883+0000","last_peered":"2023-10-16T09:59:24.886883+0000","last_clean":"2023-10-16T09:59:24.886883+0000","last_became_active":"2023-10-16T09:59:23.871220+0000","last_became_peered":"2023-10-16T09:59:23.871220+0000","last_unstale":"2023-10-16T09:59:24.886883+0000","last_undegraded":"2023-10-16T09:59:24.886883+0000","last_fullsized":"2023-10-16T09:59:24.886883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:55.500403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861945+0000","last_change":"2023-10-16T09:59:23.867669+0000","last_active":"2023-10-16T09:59:24.861945+0000","last_peered":"2023-10-16T09:59:24.861945+0000","last_clean":"2023-10-16T09:59:24.861945+0000","last_became_active":"2023-10-16T09:59:23.867396+0000","last_became_peered":"2023-10-16T09:59:23.867396+0000","last_unstale":"2023-10-16T09:59:24.861945+0000","last_undegraded":"2023-10-16T09:59:24.861945+0000","last_fullsized":"2023-10-16T09:59:24.861945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:28.613530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863403+0000","last_change":"2023-10-16T09:59:23.861659+0000","last_active":"2023-10-16T09:59:24.863403+0000","last_peered":"2023-10-16T09:59:24.863403+0000","last_clean":"2023-10-16T09:59:24.863403+0000","last_became_active":"2023-10-16T09:59:23.861388+0000","last_became_peered":"2023-10-16T09:59:23.861388+0000","last_unstale":"2023-10-16T09:59:24.863403+0000","last_undegraded":"2023-10-16T09:59:24.863403+0000","last_fullsized":"2023-10-16T09:59:24.863403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:44:32.859542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864403+0000","last_change":"2023-10-16T09:59:23.864723+0000","last_active":"2023-10-16T09:59:24.864403+0000","last_peered":"2023-10-16T09:59:24.864403+0000","last_clean":"2023-10-16T09:59:24.864403+0000","last_became_active":"2023-10-16T09:59:23.864438+0000","last_became_peered":"2023-10-16T09:59:23.864438+0000","last_unstale":"2023-10-16T09:59:24.864403+0000","last_undegraded":"2023-10-16T09:59:24.864403+0000","last_fullsized":"2023-10-16T09:59:24.864403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:00:23.089299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886769+0000","last_change":"2023-10-16T09:59:23.873975+0000","last_active":"2023-10-16T09:59:24.886769+0000","last_peered":"2023-10-16T09:59:24.886769+0000","last_clean":"2023-10-16T09:59:24.886769+0000","last_became_active":"2023-10-16T09:59:23.873663+0000","last_became_peered":"2023-10-16T09:59:23.873663+0000","last_unstale":"2023-10-16T09:59:24.886769+0000","last_undegraded":"2023-10-16T09:59:24.886769+0000","last_fullsized":"2023-10-16T09:59:24.886769+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:30.429025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885670+0000","last_change":"2023-10-16T09:59:23.872889+0000","last_active":"2023-10-16T09:59:24.885670+0000","last_peered":"2023-10-16T09:59:24.885670+0000","last_clean":"2023-10-16T09:59:24.885670+0000","last_became_active":"2023-10-16T09:59:23.868236+0000","last_became_peered":"2023-10-16T09:59:23.868236+0000","last_unstale":"2023-10-16T09:59:24.885670+0000","last_undegraded":"2023-10-16T09:59:24.885670+0000","last_fullsized":"2023-10-16T09:59:24.885670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:30.429025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885705+0000","last_change":"2023-10-16T09:59:23.867279+0000","last_active":"2023-10-16T09:59:24.885705+0000","last_peered":"2023-10-16T09:59:24.885705+0000","last_clean":"2023-10-16T09:59:24.885705+0000","last_became_active":"2023-10-16T09:59:23.866982+0000","last_became_peered":"2023-10-16T09:59:23.866982+0000","last_unstale":"2023-10-16T09:59:24.885705+0000","last_undegraded":"2023-10-16T09:59:24.885705+0000","last_fullsized":"2023-10-16T09:59:24.885705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:05.847190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861680+0000","last_change":"2023-10-16T09:59:23.867731+0000","last_active":"2023-10-16T09:59:24.861680+0000","last_peered":"2023-10-16T09:59:24.861680+0000","last_clean":"2023-10-16T09:59:24.861680+0000","last_became_active":"2023-10-16T09:59:23.867440+0000","last_became_peered":"2023-10-16T09:59:23.867440+0000","last_unstale":"2023-10-16T09:59:24.861680+0000","last_undegraded":"2023-10-16T09:59:24.861680+0000","last_fullsized":"2023-10-16T09:59:24.861680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:03:59.004731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886576+0000","last_change":"2023-10-16T09:59:23.874003+0000","last_active":"2023-10-16T09:59:24.886576+0000","last_peered":"2023-10-16T09:59:24.886576+0000","last_clean":"2023-10-16T09:59:24.886576+0000","last_became_active":"2023-10-16T09:59:23.873735+0000","last_became_peered":"2023-10-16T09:59:23.873735+0000","last_unstale":"2023-10-16T09:59:24.886576+0000","last_undegraded":"2023-10-16T09:59:24.886576+0000","last_fullsized":"2023-10-16T09:59:24.886576+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:04:36.969551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886106+0000","last_change":"2023-10-16T09:59:23.871372+0000","last_active":"2023-10-16T09:59:24.886106+0000","last_peered":"2023-10-16T09:59:24.886106+0000","last_clean":"2023-10-16T09:59:24.886106+0000","last_became_active":"2023-10-16T09:59:23.870964+0000","last_became_peered":"2023-10-16T09:59:23.870964+0000","last_unstale":"2023-10-16T09:59:24.886106+0000","last_undegraded":"2023-10-16T09:59:24.886106+0000","last_fullsized":"2023-10-16T09:59:24.886106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:41.707190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886069+0000","last_change":"2023-10-16T09:59:23.868030+0000","last_active":"2023-10-16T09:59:24.886069+0000","last_peered":"2023-10-16T09:59:24.886069+0000","last_clean":"2023-10-16T09:59:24.886069+0000","last_became_active":"2023-10-16T09:59:23.867764+0000","last_became_peered":"2023-10-16T09:59:23.867764+0000","last_unstale":"2023-10-16T09:59:24.886069+0000","last_undegraded":"2023-10-16T09:59:24.886069+0000","last_fullsized":"2023-10-16T09:59:24.886069+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:33.895462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864554+0000","last_change":"2023-10-16T09:59:23.863343+0000","last_active":"2023-10-16T09:59:24.864554+0000","last_peered":"2023-10-16T09:59:24.864554+0000","last_clean":"2023-10-16T09:59:24.864554+0000","last_became_active":"2023-10-16T09:59:23.863044+0000","last_became_peered":"2023-10-16T09:59:23.863044+0000","last_unstale":"2023-10-16T09:59:24.864554+0000","last_undegraded":"2023-10-16T09:59:24.864554+0000","last_fullsized":"2023-10-16T09:59:24.864554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:35.096865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886289+0000","last_change":"2023-10-16T09:59:23.875396+0000","last_active":"2023-10-16T09:59:24.886289+0000","last_peered":"2023-10-16T09:59:24.886289+0000","last_clean":"2023-10-16T09:59:24.886289+0000","last_became_active":"2023-10-16T09:59:23.875136+0000","last_became_peered":"2023-10-16T09:59:23.875136+0000","last_unstale":"2023-10-16T09:59:24.886289+0000","last_undegraded":"2023-10-16T09:59:24.886289+0000","last_fullsized":"2023-10-16T09:59:24.886289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:02.821717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885865+0000","last_change":"2023-10-16T09:59:23.866898+0000","last_active":"2023-10-16T09:59:24.885865+0000","last_peered":"2023-10-16T09:59:24.885865+0000","last_clean":"2023-10-16T09:59:24.885865+0000","last_became_active":"2023-10-16T09:59:23.866614+0000","last_became_peered":"2023-10-16T09:59:23.866614+0000","last_unstale":"2023-10-16T09:59:24.885865+0000","last_undegraded":"2023-10-16T09:59:24.885865+0000","last_fullsized":"2023-10-16T09:59:24.885865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:10.799717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885728+0000","last_change":"2023-10-16T09:59:23.865712+0000","last_active":"2023-10-16T09:59:24.885728+0000","last_peered":"2023-10-16T09:59:24.885728+0000","last_clean":"2023-10-16T09:59:24.885728+0000","last_became_active":"2023-10-16T09:59:23.865396+0000","last_became_peered":"2023-10-16T09:59:23.865396+0000","last_unstale":"2023-10-16T09:59:24.885728+0000","last_undegraded":"2023-10-16T09:59:24.885728+0000","last_fullsized":"2023-10-16T09:59:24.885728+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:10.083720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885597+0000","last_change":"2023-10-16T09:59:23.873944+0000","last_active":"2023-10-16T09:59:24.885597+0000","last_peered":"2023-10-16T09:59:24.885597+0000","last_clean":"2023-10-16T09:59:24.885597+0000","last_became_active":"2023-10-16T09:59:23.872780+0000","last_became_peered":"2023-10-16T09:59:23.872780+0000","last_unstale":"2023-10-16T09:59:24.885597+0000","last_undegraded":"2023-10-16T09:59:24.885597+0000","last_fullsized":"2023-10-16T09:59:24.885597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:16:05.516422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862324+0000","last_change":"2023-10-16T09:59:23.882778+0000","last_active":"2023-10-16T09:59:24.862324+0000","last_peered":"2023-10-16T09:59:24.862324+0000","last_clean":"2023-10-16T09:59:24.862324+0000","last_became_active":"2023-10-16T09:59:23.881726+0000","last_became_peered":"2023-10-16T09:59:23.881726+0000","last_unstale":"2023-10-16T09:59:24.862324+0000","last_undegraded":"2023-10-16T09:59:24.862324+0000","last_fullsized":"2023-10-16T09:59:24.862324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:19.435418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863376+0000","last_change":"2023-10-16T09:59:23.875314+0000","last_active":"2023-10-16T09:59:24.863376+0000","last_peered":"2023-10-16T09:59:24.863376+0000","last_clean":"2023-10-16T09:59:24.863376+0000","last_became_active":"2023-10-16T09:59:23.875052+0000","last_became_peered":"2023-10-16T09:59:23.875052+0000","last_unstale":"2023-10-16T09:59:24.863376+0000","last_undegraded":"2023-10-16T09:59:24.863376+0000","last_fullsized":"2023-10-16T09:59:24.863376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:38:56.602134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886872+0000","last_change":"2023-10-16T09:59:23.870931+0000","last_active":"2023-10-16T09:59:24.886872+0000","last_peered":"2023-10-16T09:59:24.886872+0000","last_clean":"2023-10-16T09:59:24.886872+0000","last_became_active":"2023-10-16T09:59:23.870639+0000","last_became_peered":"2023-10-16T09:59:23.870639+0000","last_unstale":"2023-10-16T09:59:24.886872+0000","last_undegraded":"2023-10-16T09:59:24.886872+0000","last_fullsized":"2023-10-16T09:59:24.886872+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:15:46.837454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861761+0000","last_change":"2023-10-16T09:59:23.875578+0000","last_active":"2023-10-16T09:59:24.861761+0000","last_peered":"2023-10-16T09:59:24.861761+0000","last_clean":"2023-10-16T09:59:24.861761+0000","last_became_active":"2023-10-16T09:59:23.875282+0000","last_became_peered":"2023-10-16T09:59:23.875282+0000","last_unstale":"2023-10-16T09:59:24.861761+0000","last_undegraded":"2023-10-16T09:59:24.861761+0000","last_fullsized":"2023-10-16T09:59:24.861761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:30.026317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863565+0000","last_change":"2023-10-16T09:59:23.863702+0000","last_active":"2023-10-16T09:59:24.863565+0000","last_peered":"2023-10-16T09:59:24.863565+0000","last_clean":"2023-10-16T09:59:24.863565+0000","last_became_active":"2023-10-16T09:59:23.863411+0000","last_became_peered":"2023-10-16T09:59:23.863411+0000","last_unstale":"2023-10-16T09:59:24.863565+0000","last_undegraded":"2023-10-16T09:59:24.863565+0000","last_fullsized":"2023-10-16T09:59:24.863565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:16.813557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862180+0000","last_change":"2023-10-16T09:59:23.882625+0000","last_active":"2023-10-16T09:59:24.862180+0000","last_peered":"2023-10-16T09:59:24.862180+0000","last_clean":"2023-10-16T09:59:24.862180+0000","last_became_active":"2023-10-16T09:59:23.882349+0000","last_became_peered":"2023-10-16T09:59:23.882349+0000","last_unstale":"2023-10-16T09:59:24.862180+0000","last_undegraded":"2023-10-16T09:59:24.862180+0000","last_fullsized":"2023-10-16T09:59:24.862180+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:36.532587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863607+0000","last_change":"2023-10-16T09:59:23.868274+0000","last_active":"2023-10-16T09:59:24.863607+0000","last_peered":"2023-10-16T09:59:24.863607+0000","last_clean":"2023-10-16T09:59:24.863607+0000","last_became_active":"2023-10-16T09:59:23.867840+0000","last_became_peered":"2023-10-16T09:59:23.867840+0000","last_unstale":"2023-10-16T09:59:24.863607+0000","last_undegraded":"2023-10-16T09:59:24.863607+0000","last_fullsized":"2023-10-16T09:59:24.863607+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:06.040676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886138+0000","last_change":"2023-10-16T09:59:23.876394+0000","last_active":"2023-10-16T09:59:24.886138+0000","last_peered":"2023-10-16T09:59:24.886138+0000","last_clean":"2023-10-16T09:59:24.886138+0000","last_became_active":"2023-10-16T09:59:23.876060+0000","last_became_peered":"2023-10-16T09:59:23.876060+0000","last_unstale":"2023-10-16T09:59:24.886138+0000","last_undegraded":"2023-10-16T09:59:24.886138+0000","last_fullsized":"2023-10-16T09:59:24.886138+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:13:50.899643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886988+0000","last_change":"2023-10-16T09:59:23.864926+0000","last_active":"2023-10-16T09:59:24.886988+0000","last_peered":"2023-10-16T09:59:24.886988+0000","last_clean":"2023-10-16T09:59:24.886988+0000","last_became_active":"2023-10-16T09:59:23.863277+0000","last_became_peered":"2023-10-16T09:59:23.863277+0000","last_unstale":"2023-10-16T09:59:24.886988+0000","last_undegraded":"2023-10-16T09:59:24.886988+0000","last_fullsized":"2023-10-16T09:59:24.886988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:46.877740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864421+0000","last_change":"2023-10-16T09:59:23.862267+0000","last_active":"2023-10-16T09:59:24.864421+0000","last_peered":"2023-10-16T09:59:24.864421+0000","last_clean":"2023-10-16T09:59:24.864421+0000","last_became_active":"2023-10-16T09:59:23.861896+0000","last_became_peered":"2023-10-16T09:59:23.861896+0000","last_unstale":"2023-10-16T09:59:24.864421+0000","last_undegraded":"2023-10-16T09:59:24.864421+0000","last_fullsized":"2023-10-16T09:59:24.864421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:00.864560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885874+0000","last_change":"2023-10-16T09:59:23.873834+0000","last_active":"2023-10-16T09:59:24.885874+0000","last_peered":"2023-10-16T09:59:24.885874+0000","last_clean":"2023-10-16T09:59:24.885874+0000","last_became_active":"2023-10-16T09:59:23.873432+0000","last_became_peered":"2023-10-16T09:59:23.873432+0000","last_unstale":"2023-10-16T09:59:24.885874+0000","last_undegraded":"2023-10-16T09:59:24.885874+0000","last_fullsized":"2023-10-16T09:59:24.885874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:46.877740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886887+0000","last_change":"2023-10-16T09:59:23.874041+0000","last_active":"2023-10-16T09:59:24.886887+0000","last_peered":"2023-10-16T09:59:24.886887+0000","last_clean":"2023-10-16T09:59:24.886887+0000","last_became_active":"2023-10-16T09:59:23.873752+0000","last_became_peered":"2023-10-16T09:59:23.873752+0000","last_unstale":"2023-10-16T09:59:24.886887+0000","last_undegraded":"2023-10-16T09:59:24.886887+0000","last_fullsized":"2023-10-16T09:59:24.886887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:01.092448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861931+0000","last_change":"2023-10-16T09:59:23.869388+0000","last_active":"2023-10-16T09:59:24.861931+0000","last_peered":"2023-10-16T09:59:24.861931+0000","last_clean":"2023-10-16T09:59:24.861931+0000","last_became_active":"2023-10-16T09:59:23.866868+0000","last_became_peered":"2023-10-16T09:59:23.866868+0000","last_unstale":"2023-10-16T09:59:24.861931+0000","last_undegraded":"2023-10-16T09:59:24.861931+0000","last_fullsized":"2023-10-16T09:59:24.861931+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:30.282547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863694+0000","last_change":"2023-10-16T09:59:23.858517+0000","last_active":"2023-10-16T09:59:24.863694+0000","last_peered":"2023-10-16T09:59:24.863694+0000","last_clean":"2023-10-16T09:59:24.863694+0000","last_became_active":"2023-10-16T09:59:23.858219+0000","last_became_peered":"2023-10-16T09:59:23.858219+0000","last_unstale":"2023-10-16T09:59:24.863694+0000","last_undegraded":"2023-10-16T09:59:24.863694+0000","last_fullsized":"2023-10-16T09:59:24.863694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:37.854267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886586+0000","last_change":"2023-10-16T09:59:23.863737+0000","last_active":"2023-10-16T09:59:24.886586+0000","last_peered":"2023-10-16T09:59:24.886586+0000","last_clean":"2023-10-16T09:59:24.886586+0000","last_became_active":"2023-10-16T09:59:23.863447+0000","last_became_peered":"2023-10-16T09:59:23.863447+0000","last_unstale":"2023-10-16T09:59:24.886586+0000","last_undegraded":"2023-10-16T09:59:24.886586+0000","last_fullsized":"2023-10-16T09:59:24.886586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:16:05.516422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885727+0000","last_change":"2023-10-16T09:59:23.874281+0000","last_active":"2023-10-16T09:59:24.885727+0000","last_peered":"2023-10-16T09:59:24.885727+0000","last_clean":"2023-10-16T09:59:24.885727+0000","last_became_active":"2023-10-16T09:59:23.872638+0000","last_became_peered":"2023-10-16T09:59:23.872638+0000","last_unstale":"2023-10-16T09:59:24.885727+0000","last_undegraded":"2023-10-16T09:59:24.885727+0000","last_fullsized":"2023-10-16T09:59:24.885727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:01.092448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885871+0000","last_change":"2023-10-16T09:59:23.869020+0000","last_active":"2023-10-16T09:59:24.885871+0000","last_peered":"2023-10-16T09:59:24.885871+0000","last_clean":"2023-10-16T09:59:24.885871+0000","last_became_active":"2023-10-16T09:59:23.868734+0000","last_became_peered":"2023-10-16T09:59:23.868734+0000","last_unstale":"2023-10-16T09:59:24.885871+0000","last_undegraded":"2023-10-16T09:59:24.885871+0000","last_fullsized":"2023-10-16T09:59:24.885871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:55.500403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"75'4","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886226+0000","last_change":"2023-10-16T09:59:22.853854+0000","last_active":"2023-10-16T09:59:24.886226+0000","last_peered":"2023-10-16T09:59:24.886226+0000","last_clean":"2023-10-16T09:59:24.886226+0000","last_became_active":"2023-10-16T09:59:22.853583+0000","last_became_peered":"2023-10-16T09:59:22.853583+0000","last_unstale":"2023-10-16T09:59:24.886226+0000","last_undegraded":"2023-10-16T09:59:24.886226+0000","last_fullsized":"2023-10-16T09:59:24.886226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:50.002839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886190+0000","last_change":"2023-10-16T09:59:23.869456+0000","last_active":"2023-10-16T09:59:24.886190+0000","last_peered":"2023-10-16T09:59:24.886190+0000","last_clean":"2023-10-16T09:59:24.886190+0000","last_became_active":"2023-10-16T09:59:23.869056+0000","last_became_peered":"2023-10-16T09:59:23.869056+0000","last_unstale":"2023-10-16T09:59:24.886190+0000","last_undegraded":"2023-10-16T09:59:24.886190+0000","last_fullsized":"2023-10-16T09:59:24.886190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:19.631749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862046+0000","last_change":"2023-10-16T09:59:22.848463+0000","last_active":"2023-10-16T09:59:24.862046+0000","last_peered":"2023-10-16T09:59:24.862046+0000","last_clean":"2023-10-16T09:59:24.862046+0000","last_became_active":"2023-10-16T09:59:22.848208+0000","last_became_peered":"2023-10-16T09:59:22.848208+0000","last_unstale":"2023-10-16T09:59:24.862046+0000","last_undegraded":"2023-10-16T09:59:24.862046+0000","last_fullsized":"2023-10-16T09:59:24.862046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:09.064449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885994+0000","last_change":"2023-10-16T09:59:23.866976+0000","last_active":"2023-10-16T09:59:24.885994+0000","last_peered":"2023-10-16T09:59:24.885994+0000","last_clean":"2023-10-16T09:59:24.885994+0000","last_became_active":"2023-10-16T09:59:23.866711+0000","last_became_peered":"2023-10-16T09:59:23.866711+0000","last_unstale":"2023-10-16T09:59:24.885994+0000","last_undegraded":"2023-10-16T09:59:24.885994+0000","last_fullsized":"2023-10-16T09:59:24.885994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:14.455020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886093+0000","last_change":"2023-10-16T09:59:22.848967+0000","last_active":"2023-10-16T09:59:24.886093+0000","last_peered":"2023-10-16T09:59:24.886093+0000","last_clean":"2023-10-16T09:59:24.886093+0000","last_became_active":"2023-10-16T09:59:22.848660+0000","last_became_peered":"2023-10-16T09:59:22.848660+0000","last_unstale":"2023-10-16T09:59:24.886093+0000","last_undegraded":"2023-10-16T09:59:24.886093+0000","last_fullsized":"2023-10-16T09:59:24.886093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886003+0000","last_change":"2023-10-16T09:59:23.873772+0000","last_active":"2023-10-16T09:59:24.886003+0000","last_peered":"2023-10-16T09:59:24.886003+0000","last_clean":"2023-10-16T09:59:24.886003+0000","last_became_active":"2023-10-16T09:59:23.873124+0000","last_became_peered":"2023-10-16T09:59:23.873124+0000","last_unstale":"2023-10-16T09:59:24.886003+0000","last_undegraded":"2023-10-16T09:59:24.886003+0000","last_fullsized":"2023-10-16T09:59:24.886003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:36.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864470+0000","last_change":"2023-10-16T09:59:22.852120+0000","last_active":"2023-10-16T09:59:24.864470+0000","last_peered":"2023-10-16T09:59:24.864470+0000","last_clean":"2023-10-16T09:59:24.864470+0000","last_became_active":"2023-10-16T09:59:22.851826+0000","last_became_peered":"2023-10-16T09:59:22.851826+0000","last_unstale":"2023-10-16T09:59:24.864470+0000","last_undegraded":"2023-10-16T09:59:24.864470+0000","last_fullsized":"2023-10-16T09:59:24.864470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:58.125943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862041+0000","last_change":"2023-10-16T09:59:23.882543+0000","last_active":"2023-10-16T09:59:24.862041+0000","last_peered":"2023-10-16T09:59:24.862041+0000","last_clean":"2023-10-16T09:59:24.862041+0000","last_became_active":"2023-10-16T09:59:23.882242+0000","last_became_peered":"2023-10-16T09:59:23.882242+0000","last_unstale":"2023-10-16T09:59:24.862041+0000","last_undegraded":"2023-10-16T09:59:24.862041+0000","last_fullsized":"2023-10-16T09:59:24.862041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:01.870085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885919+0000","last_change":"2023-10-16T09:59:22.854008+0000","last_active":"2023-10-16T09:59:24.885919+0000","last_peered":"2023-10-16T09:59:24.885919+0000","last_clean":"2023-10-16T09:59:24.885919+0000","last_became_active":"2023-10-16T09:59:22.853695+0000","last_became_peered":"2023-10-16T09:59:22.853695+0000","last_unstale":"2023-10-16T09:59:24.885919+0000","last_undegraded":"2023-10-16T09:59:24.885919+0000","last_fullsized":"2023-10-16T09:59:24.885919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:28:47.452567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861839+0000","last_change":"2023-10-16T09:59:23.874784+0000","last_active":"2023-10-16T09:59:24.861839+0000","last_peered":"2023-10-16T09:59:24.861839+0000","last_clean":"2023-10-16T09:59:24.861839+0000","last_became_active":"2023-10-16T09:59:23.874497+0000","last_became_peered":"2023-10-16T09:59:23.874497+0000","last_unstale":"2023-10-16T09:59:24.861839+0000","last_undegraded":"2023-10-16T09:59:24.861839+0000","last_fullsized":"2023-10-16T09:59:24.861839+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:40:50.582019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885677+0000","last_change":"2023-10-16T09:59:22.849437+0000","last_active":"2023-10-16T09:59:24.885677+0000","last_peered":"2023-10-16T09:59:24.885677+0000","last_clean":"2023-10-16T09:59:24.885677+0000","last_became_active":"2023-10-16T09:59:22.849172+0000","last_became_peered":"2023-10-16T09:59:22.849172+0000","last_unstale":"2023-10-16T09:59:24.885677+0000","last_undegraded":"2023-10-16T09:59:24.885677+0000","last_fullsized":"2023-10-16T09:59:24.885677+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:50.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886214+0000","last_change":"2023-10-16T09:59:23.870219+0000","last_active":"2023-10-16T09:59:24.886214+0000","last_peered":"2023-10-16T09:59:24.886214+0000","last_clean":"2023-10-16T09:59:24.886214+0000","last_became_active":"2023-10-16T09:59:23.869734+0000","last_became_peered":"2023-10-16T09:59:23.869734+0000","last_unstale":"2023-10-16T09:59:24.886214+0000","last_undegraded":"2023-10-16T09:59:24.886214+0000","last_fullsized":"2023-10-16T09:59:24.886214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:23.878030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864140+0000","last_change":"2023-10-16T09:59:22.851804+0000","last_active":"2023-10-16T09:59:24.864140+0000","last_peered":"2023-10-16T09:59:24.864140+0000","last_clean":"2023-10-16T09:59:24.864140+0000","last_became_active":"2023-10-16T09:59:22.851446+0000","last_became_peered":"2023-10-16T09:59:22.851446+0000","last_unstale":"2023-10-16T09:59:24.864140+0000","last_undegraded":"2023-10-16T09:59:24.864140+0000","last_fullsized":"2023-10-16T09:59:24.864140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:49:37.665105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862129+0000","last_change":"2023-10-16T09:59:23.882566+0000","last_active":"2023-10-16T09:59:24.862129+0000","last_peered":"2023-10-16T09:59:24.862129+0000","last_clean":"2023-10-16T09:59:24.862129+0000","last_became_active":"2023-10-16T09:59:23.881771+0000","last_became_peered":"2023-10-16T09:59:23.881771+0000","last_unstale":"2023-10-16T09:59:24.862129+0000","last_undegraded":"2023-10-16T09:59:24.862129+0000","last_fullsized":"2023-10-16T09:59:24.862129+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:32:36.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885668+0000","last_change":"2023-10-16T09:59:22.854268+0000","last_active":"2023-10-16T09:59:24.885668+0000","last_peered":"2023-10-16T09:59:24.885668+0000","last_clean":"2023-10-16T09:59:24.885668+0000","last_became_active":"2023-10-16T09:59:22.853981+0000","last_became_peered":"2023-10-16T09:59:22.853981+0000","last_unstale":"2023-10-16T09:59:24.885668+0000","last_undegraded":"2023-10-16T09:59:24.885668+0000","last_fullsized":"2023-10-16T09:59:24.885668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:05:59.547302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886558+0000","last_change":"2023-10-16T09:59:23.865849+0000","last_active":"2023-10-16T09:59:24.886558+0000","last_peered":"2023-10-16T09:59:24.886558+0000","last_clean":"2023-10-16T09:59:24.886558+0000","last_became_active":"2023-10-16T09:59:23.865567+0000","last_became_peered":"2023-10-16T09:59:23.865567+0000","last_unstale":"2023-10-16T09:59:24.886558+0000","last_undegraded":"2023-10-16T09:59:24.886558+0000","last_fullsized":"2023-10-16T09:59:24.886558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:13.227930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863725+0000","last_change":"2023-10-16T09:59:22.854126+0000","last_active":"2023-10-16T09:59:24.863725+0000","last_peered":"2023-10-16T09:59:24.863725+0000","last_clean":"2023-10-16T09:59:24.863725+0000","last_became_active":"2023-10-16T09:59:22.853844+0000","last_became_peered":"2023-10-16T09:59:22.853844+0000","last_unstale":"2023-10-16T09:59:24.863725+0000","last_undegraded":"2023-10-16T09:59:24.863725+0000","last_fullsized":"2023-10-16T09:59:24.863725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:29.403986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863454+0000","last_change":"2023-10-16T09:59:23.867446+0000","last_active":"2023-10-16T09:59:24.863454+0000","last_peered":"2023-10-16T09:59:24.863454+0000","last_clean":"2023-10-16T09:59:24.863454+0000","last_became_active":"2023-10-16T09:59:23.867106+0000","last_became_peered":"2023-10-16T09:59:23.867106+0000","last_unstale":"2023-10-16T09:59:24.863454+0000","last_undegraded":"2023-10-16T09:59:24.863454+0000","last_fullsized":"2023-10-16T09:59:24.863454+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:27.270775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862128+0000","last_change":"2023-10-16T09:59:22.854038+0000","last_active":"2023-10-16T09:59:24.862128+0000","last_peered":"2023-10-16T09:59:24.862128+0000","last_clean":"2023-10-16T09:59:24.862128+0000","last_became_active":"2023-10-16T09:59:22.853752+0000","last_became_peered":"2023-10-16T09:59:22.853752+0000","last_unstale":"2023-10-16T09:59:24.862128+0000","last_undegraded":"2023-10-16T09:59:24.862128+0000","last_fullsized":"2023-10-16T09:59:24.862128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:42.947785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885670+0000","last_change":"2023-10-16T09:59:23.869198+0000","last_active":"2023-10-16T09:59:24.885670+0000","last_peered":"2023-10-16T09:59:24.885670+0000","last_clean":"2023-10-16T09:59:24.885670+0000","last_became_active":"2023-10-16T09:59:23.868713+0000","last_became_peered":"2023-10-16T09:59:23.868713+0000","last_unstale":"2023-10-16T09:59:24.885670+0000","last_undegraded":"2023-10-16T09:59:24.885670+0000","last_fullsized":"2023-10-16T09:59:24.885670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:03.528976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"75'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863964+0000","last_change":"2023-10-16T09:59:22.849709+0000","last_active":"2023-10-16T09:59:24.863964+0000","last_peered":"2023-10-16T09:59:24.863964+0000","last_clean":"2023-10-16T09:59:24.863964+0000","last_became_active":"2023-10-16T09:59:22.849438+0000","last_became_peered":"2023-10-16T09:59:22.849438+0000","last_unstale":"2023-10-16T09:59:24.863964+0000","last_undegraded":"2023-10-16T09:59:24.863964+0000","last_fullsized":"2023-10-16T09:59:24.863964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:34.092604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863547+0000","last_change":"2023-10-16T09:59:23.859271+0000","last_active":"2023-10-16T09:59:24.863547+0000","last_peered":"2023-10-16T09:59:24.863547+0000","last_clean":"2023-10-16T09:59:24.863547+0000","last_became_active":"2023-10-16T09:59:23.858989+0000","last_became_peered":"2023-10-16T09:59:23.858989+0000","last_unstale":"2023-10-16T09:59:24.863547+0000","last_undegraded":"2023-10-16T09:59:24.863547+0000","last_fullsized":"2023-10-16T09:59:24.863547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:23.935998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"75'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886468+0000","last_change":"2023-10-16T09:59:22.853644+0000","last_active":"2023-10-16T09:59:24.886468+0000","last_peered":"2023-10-16T09:59:24.886468+0000","last_clean":"2023-10-16T09:59:24.886468+0000","last_became_active":"2023-10-16T09:59:22.853312+0000","last_became_peered":"2023-10-16T09:59:22.853312+0000","last_unstale":"2023-10-16T09:59:24.886468+0000","last_undegraded":"2023-10-16T09:59:24.886468+0000","last_fullsized":"2023-10-16T09:59:24.886468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:16.621595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864250+0000","last_change":"2023-10-16T09:59:23.861706+0000","last_active":"2023-10-16T09:59:24.864250+0000","last_peered":"2023-10-16T09:59:24.864250+0000","last_clean":"2023-10-16T09:59:24.864250+0000","last_became_active":"2023-10-16T09:59:23.861391+0000","last_became_peered":"2023-10-16T09:59:23.861391+0000","last_unstale":"2023-10-16T09:59:24.864250+0000","last_undegraded":"2023-10-16T09:59:24.864250+0000","last_fullsized":"2023-10-16T09:59:24.864250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:13.434649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861984+0000","last_change":"2023-10-16T09:59:22.853983+0000","last_active":"2023-10-16T09:59:24.861984+0000","last_peered":"2023-10-16T09:59:24.861984+0000","last_clean":"2023-10-16T09:59:24.861984+0000","last_became_active":"2023-10-16T09:59:22.853696+0000","last_became_peered":"2023-10-16T09:59:22.853696+0000","last_unstale":"2023-10-16T09:59:24.861984+0000","last_undegraded":"2023-10-16T09:59:24.861984+0000","last_fullsized":"2023-10-16T09:59:24.861984+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:46.001474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863317+0000","last_change":"2023-10-16T09:59:23.868422+0000","last_active":"2023-10-16T09:59:24.863317+0000","last_peered":"2023-10-16T09:59:24.863317+0000","last_clean":"2023-10-16T09:59:24.863317+0000","last_became_active":"2023-10-16T09:59:23.867704+0000","last_became_peered":"2023-10-16T09:59:23.867704+0000","last_unstale":"2023-10-16T09:59:24.863317+0000","last_undegraded":"2023-10-16T09:59:24.863317+0000","last_fullsized":"2023-10-16T09:59:24.863317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:58.731906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886114+0000","last_change":"2023-10-16T09:59:22.848206+0000","last_active":"2023-10-16T09:59:24.886114+0000","last_peered":"2023-10-16T09:59:24.886114+0000","last_clean":"2023-10-16T09:59:24.886114+0000","last_became_active":"2023-10-16T09:59:22.847918+0000","last_became_peered":"2023-10-16T09:59:22.847918+0000","last_unstale":"2023-10-16T09:59:24.886114+0000","last_undegraded":"2023-10-16T09:59:24.886114+0000","last_fullsized":"2023-10-16T09:59:24.886114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:35.191382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885659+0000","last_change":"2023-10-16T09:59:23.863919+0000","last_active":"2023-10-16T09:59:24.885659+0000","last_peered":"2023-10-16T09:59:24.885659+0000","last_clean":"2023-10-16T09:59:24.885659+0000","last_became_active":"2023-10-16T09:59:23.863629+0000","last_became_peered":"2023-10-16T09:59:23.863629+0000","last_unstale":"2023-10-16T09:59:24.885659+0000","last_undegraded":"2023-10-16T09:59:24.885659+0000","last_fullsized":"2023-10-16T09:59:24.885659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:27.131791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864185+0000","last_change":"2023-10-16T09:59:22.852360+0000","last_active":"2023-10-16T09:59:24.864185+0000","last_peered":"2023-10-16T09:59:24.864185+0000","last_clean":"2023-10-16T09:59:24.864185+0000","last_became_active":"2023-10-16T09:59:22.852081+0000","last_became_peered":"2023-10-16T09:59:22.852081+0000","last_unstale":"2023-10-16T09:59:24.864185+0000","last_undegraded":"2023-10-16T09:59:24.864185+0000","last_fullsized":"2023-10-16T09:59:24.864185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:42.851600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863381+0000","last_change":"2023-10-16T09:59:23.868100+0000","last_active":"2023-10-16T09:59:24.863381+0000","last_peered":"2023-10-16T09:59:24.863381+0000","last_clean":"2023-10-16T09:59:24.863381+0000","last_became_active":"2023-10-16T09:59:23.867499+0000","last_became_peered":"2023-10-16T09:59:23.867499+0000","last_unstale":"2023-10-16T09:59:24.863381+0000","last_undegraded":"2023-10-16T09:59:24.863381+0000","last_fullsized":"2023-10-16T09:59:24.863381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:08:38.630000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"75'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862941+0000","last_change":"2023-10-16T09:59:22.851868+0000","last_active":"2023-10-16T09:59:24.862941+0000","last_peered":"2023-10-16T09:59:24.862941+0000","last_clean":"2023-10-16T09:59:24.862941+0000","last_became_active":"2023-10-16T09:59:22.851583+0000","last_became_peered":"2023-10-16T09:59:22.851583+0000","last_unstale":"2023-10-16T09:59:24.862941+0000","last_undegraded":"2023-10-16T09:59:24.862941+0000","last_fullsized":"2023-10-16T09:59:24.862941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:59.860300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863563+0000","last_change":"2023-10-16T09:59:23.867652+0000","last_active":"2023-10-16T09:59:24.863563+0000","last_peered":"2023-10-16T09:59:24.863563+0000","last_clean":"2023-10-16T09:59:24.863563+0000","last_became_active":"2023-10-16T09:59:23.867338+0000","last_became_peered":"2023-10-16T09:59:23.867338+0000","last_unstale":"2023-10-16T09:59:24.863563+0000","last_undegraded":"2023-10-16T09:59:24.863563+0000","last_fullsized":"2023-10-16T09:59:24.863563+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:18.049530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862187+0000","last_change":"2023-10-16T09:59:22.854214+0000","last_active":"2023-10-16T09:59:24.862187+0000","last_peered":"2023-10-16T09:59:24.862187+0000","last_clean":"2023-10-16T09:59:24.862187+0000","last_became_active":"2023-10-16T09:59:22.853551+0000","last_became_peered":"2023-10-16T09:59:22.853551+0000","last_unstale":"2023-10-16T09:59:24.862187+0000","last_undegraded":"2023-10-16T09:59:24.862187+0000","last_fullsized":"2023-10-16T09:59:24.862187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:02.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864800+0000","last_change":"2023-10-16T09:59:23.861818+0000","last_active":"2023-10-16T09:59:24.864800+0000","last_peered":"2023-10-16T09:59:24.864800+0000","last_clean":"2023-10-16T09:59:24.864800+0000","last_became_active":"2023-10-16T09:59:23.861173+0000","last_became_peered":"2023-10-16T09:59:23.861173+0000","last_unstale":"2023-10-16T09:59:24.864800+0000","last_undegraded":"2023-10-16T09:59:24.864800+0000","last_fullsized":"2023-10-16T09:59:24.864800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:06.629359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885962+0000","last_change":"2023-10-16T09:59:22.856772+0000","last_active":"2023-10-16T09:59:24.885962+0000","last_peered":"2023-10-16T09:59:24.885962+0000","last_clean":"2023-10-16T09:59:24.885962+0000","last_became_active":"2023-10-16T09:59:22.856449+0000","last_became_peered":"2023-10-16T09:59:22.856449+0000","last_unstale":"2023-10-16T09:59:24.885962+0000","last_undegraded":"2023-10-16T09:59:24.885962+0000","last_fullsized":"2023-10-16T09:59:24.885962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:48.796107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861911+0000","last_change":"2023-10-16T09:59:23.866857+0000","last_active":"2023-10-16T09:59:24.861911+0000","last_peered":"2023-10-16T09:59:24.861911+0000","last_clean":"2023-10-16T09:59:24.861911+0000","last_became_active":"2023-10-16T09:59:23.866555+0000","last_became_peered":"2023-10-16T09:59:23.866555+0000","last_unstale":"2023-10-16T09:59:24.861911+0000","last_undegraded":"2023-10-16T09:59:24.861911+0000","last_fullsized":"2023-10-16T09:59:24.861911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:49.606290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864426+0000","last_change":"2023-10-16T09:59:22.851855+0000","last_active":"2023-10-16T09:59:24.864426+0000","last_peered":"2023-10-16T09:59:24.864426+0000","last_clean":"2023-10-16T09:59:24.864426+0000","last_became_active":"2023-10-16T09:59:22.851516+0000","last_became_peered":"2023-10-16T09:59:22.851516+0000","last_unstale":"2023-10-16T09:59:24.864426+0000","last_undegraded":"2023-10-16T09:59:24.864426+0000","last_fullsized":"2023-10-16T09:59:24.864426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:32.571858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863861+0000","last_change":"2023-10-16T09:59:23.863898+0000","last_active":"2023-10-16T09:59:24.863861+0000","last_peered":"2023-10-16T09:59:24.863861+0000","last_clean":"2023-10-16T09:59:24.863861+0000","last_became_active":"2023-10-16T09:59:23.863644+0000","last_became_peered":"2023-10-16T09:59:23.863644+0000","last_unstale":"2023-10-16T09:59:24.863861+0000","last_undegraded":"2023-10-16T09:59:24.863861+0000","last_fullsized":"2023-10-16T09:59:24.863861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:11.911309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886083+0000","last_change":"2023-10-16T09:59:22.847138+0000","last_active":"2023-10-16T09:59:24.886083+0000","last_peered":"2023-10-16T09:59:24.886083+0000","last_clean":"2023-10-16T09:59:24.886083+0000","last_became_active":"2023-10-16T09:59:22.846821+0000","last_became_peered":"2023-10-16T09:59:22.846821+0000","last_unstale":"2023-10-16T09:59:24.886083+0000","last_undegraded":"2023-10-16T09:59:24.886083+0000","last_fullsized":"2023-10-16T09:59:24.886083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:08.098953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864698+0000","last_change":"2023-10-16T09:59:23.861687+0000","last_active":"2023-10-16T09:59:24.864698+0000","last_peered":"2023-10-16T09:59:24.864698+0000","last_clean":"2023-10-16T09:59:24.864698+0000","last_became_active":"2023-10-16T09:59:23.861394+0000","last_became_peered":"2023-10-16T09:59:23.861394+0000","last_unstale":"2023-10-16T09:59:24.864698+0000","last_undegraded":"2023-10-16T09:59:24.864698+0000","last_fullsized":"2023-10-16T09:59:24.864698+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:48.063641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862103+0000","last_change":"2023-10-16T09:59:22.848603+0000","last_active":"2023-10-16T09:59:24.862103+0000","last_peered":"2023-10-16T09:59:24.862103+0000","last_clean":"2023-10-16T09:59:24.862103+0000","last_became_active":"2023-10-16T09:59:22.848322+0000","last_became_peered":"2023-10-16T09:59:22.848322+0000","last_unstale":"2023-10-16T09:59:24.862103+0000","last_undegraded":"2023-10-16T09:59:24.862103+0000","last_fullsized":"2023-10-16T09:59:24.862103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:03.970575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886473+0000","last_change":"2023-10-16T09:59:23.868919+0000","last_active":"2023-10-16T09:59:24.886473+0000","last_peered":"2023-10-16T09:59:24.886473+0000","last_clean":"2023-10-16T09:59:24.886473+0000","last_became_active":"2023-10-16T09:59:23.868635+0000","last_became_peered":"2023-10-16T09:59:23.868635+0000","last_unstale":"2023-10-16T09:59:24.886473+0000","last_undegraded":"2023-10-16T09:59:24.886473+0000","last_fullsized":"2023-10-16T09:59:24.886473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:22:42.603329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885614+0000","last_change":"2023-10-16T09:59:22.851570+0000","last_active":"2023-10-16T09:59:24.885614+0000","last_peered":"2023-10-16T09:59:24.885614+0000","last_clean":"2023-10-16T09:59:24.885614+0000","last_became_active":"2023-10-16T09:59:22.851300+0000","last_became_peered":"2023-10-16T09:59:22.851300+0000","last_unstale":"2023-10-16T09:59:24.885614+0000","last_undegraded":"2023-10-16T09:59:24.885614+0000","last_fullsized":"2023-10-16T09:59:24.885614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:12.643365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885685+0000","last_change":"2023-10-16T09:59:23.866644+0000","last_active":"2023-10-16T09:59:24.885685+0000","last_peered":"2023-10-16T09:59:24.885685+0000","last_clean":"2023-10-16T09:59:24.885685+0000","last_became_active":"2023-10-16T09:59:23.866265+0000","last_became_peered":"2023-10-16T09:59:23.866265+0000","last_unstale":"2023-10-16T09:59:24.885685+0000","last_undegraded":"2023-10-16T09:59:24.885685+0000","last_fullsized":"2023-10-16T09:59:24.885685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:15:46.837454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862997+0000","last_change":"2023-10-16T09:59:22.854523+0000","last_active":"2023-10-16T09:59:24.862997+0000","last_peered":"2023-10-16T09:59:24.862997+0000","last_clean":"2023-10-16T09:59:24.862997+0000","last_became_active":"2023-10-16T09:59:22.854199+0000","last_became_peered":"2023-10-16T09:59:22.854199+0000","last_unstale":"2023-10-16T09:59:24.862997+0000","last_undegraded":"2023-10-16T09:59:24.862997+0000","last_fullsized":"2023-10-16T09:59:24.862997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:02.867558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864783+0000","last_change":"2023-10-16T09:59:23.864702+0000","last_active":"2023-10-16T09:59:24.864783+0000","last_peered":"2023-10-16T09:59:24.864783+0000","last_clean":"2023-10-16T09:59:24.864783+0000","last_became_active":"2023-10-16T09:59:23.864425+0000","last_became_peered":"2023-10-16T09:59:23.864425+0000","last_unstale":"2023-10-16T09:59:24.864783+0000","last_undegraded":"2023-10-16T09:59:24.864783+0000","last_fullsized":"2023-10-16T09:59:24.864783+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:30.408246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886244+0000","last_change":"2023-10-16T09:59:22.854529+0000","last_active":"2023-10-16T09:59:24.886244+0000","last_peered":"2023-10-16T09:59:24.886244+0000","last_clean":"2023-10-16T09:59:24.886244+0000","last_became_active":"2023-10-16T09:59:22.854175+0000","last_became_peered":"2023-10-16T09:59:22.854175+0000","last_unstale":"2023-10-16T09:59:24.886244+0000","last_undegraded":"2023-10-16T09:59:24.886244+0000","last_fullsized":"2023-10-16T09:59:24.886244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:40.269111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864810+0000","last_change":"2023-10-16T09:59:23.869981+0000","last_active":"2023-10-16T09:59:24.864810+0000","last_peered":"2023-10-16T09:59:24.864810+0000","last_clean":"2023-10-16T09:59:24.864810+0000","last_became_active":"2023-10-16T09:59:23.869668+0000","last_became_peered":"2023-10-16T09:59:23.869668+0000","last_unstale":"2023-10-16T09:59:24.864810+0000","last_undegraded":"2023-10-16T09:59:24.864810+0000","last_fullsized":"2023-10-16T09:59:24.864810+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:03.852538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886050+0000","last_change":"2023-10-16T09:59:22.848213+0000","last_active":"2023-10-16T09:59:24.886050+0000","last_peered":"2023-10-16T09:59:24.886050+0000","last_clean":"2023-10-16T09:59:24.886050+0000","last_became_active":"2023-10-16T09:59:22.847961+0000","last_became_peered":"2023-10-16T09:59:22.847961+0000","last_unstale":"2023-10-16T09:59:24.886050+0000","last_undegraded":"2023-10-16T09:59:24.886050+0000","last_fullsized":"2023-10-16T09:59:24.886050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:41:45.381126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885793+0000","last_change":"2023-10-16T09:59:23.869688+0000","last_active":"2023-10-16T09:59:24.885793+0000","last_peered":"2023-10-16T09:59:24.885793+0000","last_clean":"2023-10-16T09:59:24.885793+0000","last_became_active":"2023-10-16T09:59:23.869412+0000","last_became_peered":"2023-10-16T09:59:23.869412+0000","last_unstale":"2023-10-16T09:59:24.885793+0000","last_undegraded":"2023-10-16T09:59:24.885793+0000","last_fullsized":"2023-10-16T09:59:24.885793+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:33.361606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886385+0000","last_change":"2023-10-16T09:59:22.849304+0000","last_active":"2023-10-16T09:59:24.886385+0000","last_peered":"2023-10-16T09:59:24.886385+0000","last_clean":"2023-10-16T09:59:24.886385+0000","last_became_active":"2023-10-16T09:59:22.849026+0000","last_became_peered":"2023-10-16T09:59:22.849026+0000","last_unstale":"2023-10-16T09:59:24.886385+0000","last_undegraded":"2023-10-16T09:59:24.886385+0000","last_fullsized":"2023-10-16T09:59:24.886385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:43:49.708315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886234+0000","last_change":"2023-10-16T09:59:23.873967+0000","last_active":"2023-10-16T09:59:24.886234+0000","last_peered":"2023-10-16T09:59:24.886234+0000","last_clean":"2023-10-16T09:59:24.886234+0000","last_became_active":"2023-10-16T09:59:23.873466+0000","last_became_peered":"2023-10-16T09:59:23.873466+0000","last_unstale":"2023-10-16T09:59:24.886234+0000","last_undegraded":"2023-10-16T09:59:24.886234+0000","last_fullsized":"2023-10-16T09:59:24.886234+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:13:54.009125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864609+0000","last_change":"2023-10-16T09:59:22.854467+0000","last_active":"2023-10-16T09:59:24.864609+0000","last_peered":"2023-10-16T09:59:24.864609+0000","last_clean":"2023-10-16T09:59:24.864609+0000","last_became_active":"2023-10-16T09:59:22.854170+0000","last_became_peered":"2023-10-16T09:59:22.854170+0000","last_unstale":"2023-10-16T09:59:24.864609+0000","last_undegraded":"2023-10-16T09:59:24.864609+0000","last_fullsized":"2023-10-16T09:59:24.864609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:46.880312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862177+0000","last_change":"2023-10-16T09:59:23.870338+0000","last_active":"2023-10-16T09:59:24.862177+0000","last_peered":"2023-10-16T09:59:24.862177+0000","last_clean":"2023-10-16T09:59:24.862177+0000","last_became_active":"2023-10-16T09:59:23.870027+0000","last_became_peered":"2023-10-16T09:59:23.870027+0000","last_unstale":"2023-10-16T09:59:24.862177+0000","last_undegraded":"2023-10-16T09:59:24.862177+0000","last_fullsized":"2023-10-16T09:59:24.862177+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:03:45.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885497+0000","last_change":"2023-10-16T09:59:22.852999+0000","last_active":"2023-10-16T09:59:24.885497+0000","last_peered":"2023-10-16T09:59:24.885497+0000","last_clean":"2023-10-16T09:59:24.885497+0000","last_became_active":"2023-10-16T09:59:22.852787+0000","last_became_peered":"2023-10-16T09:59:22.852787+0000","last_unstale":"2023-10-16T09:59:24.885497+0000","last_undegraded":"2023-10-16T09:59:24.885497+0000","last_fullsized":"2023-10-16T09:59:24.885497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:13:40.620177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863751+0000","last_change":"2023-10-16T09:59:23.868486+0000","last_active":"2023-10-16T09:59:24.863751+0000","last_peered":"2023-10-16T09:59:24.863751+0000","last_clean":"2023-10-16T09:59:24.863751+0000","last_became_active":"2023-10-16T09:59:23.868122+0000","last_became_peered":"2023-10-16T09:59:23.868122+0000","last_unstale":"2023-10-16T09:59:24.863751+0000","last_undegraded":"2023-10-16T09:59:24.863751+0000","last_fullsized":"2023-10-16T09:59:24.863751+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:54.232434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885848+0000","last_change":"2023-10-16T09:59:22.852181+0000","last_active":"2023-10-16T09:59:24.885848+0000","last_peered":"2023-10-16T09:59:24.885848+0000","last_clean":"2023-10-16T09:59:24.885848+0000","last_became_active":"2023-10-16T09:59:22.851951+0000","last_became_peered":"2023-10-16T09:59:22.851951+0000","last_unstale":"2023-10-16T09:59:24.885848+0000","last_undegraded":"2023-10-16T09:59:24.885848+0000","last_fullsized":"2023-10-16T09:59:24.885848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:53:02.501049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885402+0000","last_change":"2023-10-16T09:59:23.870564+0000","last_active":"2023-10-16T09:59:24.885402+0000","last_peered":"2023-10-16T09:59:24.885402+0000","last_clean":"2023-10-16T09:59:24.885402+0000","last_became_active":"2023-10-16T09:59:23.870291+0000","last_became_peered":"2023-10-16T09:59:23.870291+0000","last_unstale":"2023-10-16T09:59:24.885402+0000","last_undegraded":"2023-10-16T09:59:24.885402+0000","last_fullsized":"2023-10-16T09:59:24.885402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:28.637005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885445+0000","last_change":"2023-10-16T09:59:22.851639+0000","last_active":"2023-10-16T09:59:24.885445+0000","last_peered":"2023-10-16T09:59:24.885445+0000","last_clean":"2023-10-16T09:59:24.885445+0000","last_became_active":"2023-10-16T09:59:22.851306+0000","last_became_peered":"2023-10-16T09:59:22.851306+0000","last_unstale":"2023-10-16T09:59:24.885445+0000","last_undegraded":"2023-10-16T09:59:24.885445+0000","last_fullsized":"2023-10-16T09:59:24.885445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:31.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862224+0000","last_change":"2023-10-16T09:59:23.875558+0000","last_active":"2023-10-16T09:59:24.862224+0000","last_peered":"2023-10-16T09:59:24.862224+0000","last_clean":"2023-10-16T09:59:24.862224+0000","last_became_active":"2023-10-16T09:59:23.874984+0000","last_became_peered":"2023-10-16T09:59:23.874984+0000","last_unstale":"2023-10-16T09:59:24.862224+0000","last_undegraded":"2023-10-16T09:59:24.862224+0000","last_fullsized":"2023-10-16T09:59:24.862224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:12.087522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862014+0000","last_change":"2023-10-16T09:59:22.854092+0000","last_active":"2023-10-16T09:59:24.862014+0000","last_peered":"2023-10-16T09:59:24.862014+0000","last_clean":"2023-10-16T09:59:24.862014+0000","last_became_active":"2023-10-16T09:59:22.853827+0000","last_became_peered":"2023-10-16T09:59:22.853827+0000","last_unstale":"2023-10-16T09:59:24.862014+0000","last_undegraded":"2023-10-16T09:59:24.862014+0000","last_fullsized":"2023-10-16T09:59:24.862014+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:47.257443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862057+0000","last_change":"2023-10-16T09:59:23.876253+0000","last_active":"2023-10-16T09:59:24.862057+0000","last_peered":"2023-10-16T09:59:24.862057+0000","last_clean":"2023-10-16T09:59:24.862057+0000","last_became_active":"2023-10-16T09:59:23.875886+0000","last_became_peered":"2023-10-16T09:59:23.875886+0000","last_unstale":"2023-10-16T09:59:24.862057+0000","last_undegraded":"2023-10-16T09:59:24.862057+0000","last_fullsized":"2023-10-16T09:59:24.862057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:55.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886095+0000","last_change":"2023-10-16T09:59:22.852932+0000","last_active":"2023-10-16T09:59:24.886095+0000","last_peered":"2023-10-16T09:59:24.886095+0000","last_clean":"2023-10-16T09:59:24.886095+0000","last_became_active":"2023-10-16T09:59:22.852659+0000","last_became_peered":"2023-10-16T09:59:22.852659+0000","last_unstale":"2023-10-16T09:59:24.886095+0000","last_undegraded":"2023-10-16T09:59:24.886095+0000","last_fullsized":"2023-10-16T09:59:24.886095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:28.960195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"65'161","reported_seq":378,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-16T09:59:23.857409+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:59:23.857409+0000","last_peered":"2023-10-16T09:59:23.857409+0000","last_clean":"2023-10-16T09:59:23.857409+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:59:23.857409+0000","last_undegraded":"2023-10-16T09:59:23.857409+0000","last_fullsized":"2023-10-16T09:59:23.857409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1515520,"data_stored":1028660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.434},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60099999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.25700000000000001},"max":{"1min":0.72899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.80400000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.623}]},{"osd":3,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.81200000000000006},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.34399999999999997},"max":{"1min":0.755,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53700000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76200000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.318,"15min":0.312},"max":{"1min":0.70299999999999996,"5min":1.006,"15min":1.006},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":1.0169999999999999,"15min":1.0489999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68999999999999995,"5min":0.95199999999999996,"15min":1.373},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.85999999999999999,"15min":1.4099999999999999},"last":0.748}]}]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":438132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56899999999999995,"15min":0.61699999999999999},"min":{"1min":0.497,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":6.194},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58599999999999997,"15min":0.61899999999999999},"min":{"1min":0.49399999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":6.1609999999999996},"last":0.47999999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.47699999999999998,"5min":0.34399999999999997,"15min":0.28799999999999998},"max":{"1min":0.76700000000000002,"5min":0.79500000000000004,"15min":1.054},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78300000000000003,"5min":0.80600000000000005,"15min":0.876},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.34399999999999997,"15min":0.33000000000000002},"max":{"1min":0.79200000000000004,"5min":0.86099999999999999,"15min":8.5600000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76100000000000001,"5min":0.83699999999999997,"15min":1.1060000000000001},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.57799999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.82399999999999995,"15min":1.175},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.64400000000000002},"min":{"1min":0.442,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":6.2249999999999996},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.30299999999999999},"max":{"1min":0.67600000000000005,"5min":0.755,"15min":0.81299999999999994},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.30199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.84699999999999998},"last":0.377}]},{"osd":5,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":1.1240000000000001},"last":0.36699999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.32500000000000001,"15min":0.31900000000000001},"max":{"1min":0.77900000000000003,"5min":0.83499999999999996,"15min":0.879},"last":0.501},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.38,"15min":0.35399999999999998},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.34499999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542331,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":917504,"data_stored":438222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.375,"15min":0.37},"min":{"1min":0.22500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54400000000000004,"5min":1.5,"15min":1.5},"last":0.375},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.245,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.60099999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.23599999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55600000000000005,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.44},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.67300000000000004,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.28699999999999998}]},{"osd":3,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.495,"15min":0.47199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68000000000000005,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.443,"15min":0.436},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.65100000000000002,"5min":1.379,"15min":1.379},"last":0.34100000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85899999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":1.26,"15min":1.26},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78700000000000003,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69699999999999995,"15min":0.67000000000000004},"min":{"1min":0.44500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92100000000000004,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.505},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.50600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92600000000000005,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.43099999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":1030694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.379,"15min":0.41099999999999998},"min":{"1min":0.26700000000000002,"5min":0.19900000000000001,"15min":0.191},"max":{"1min":0.54900000000000004,"5min":0.65400000000000003,"15min":5.5090000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.40699999999999997,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":6.4950000000000001},"last":0.29399999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41399999999999998,"15min":0.40699999999999997},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.69599999999999995},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.42799999999999999,"15min":0.44500000000000001},"min":{"1min":0.29499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59899999999999998,"5min":0.63600000000000001,"15min":2.7559999999999998},"last":0.32600000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.49099999999999999},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.20699999999999999},"max":{"1min":0.58699999999999997,"5min":0.72999999999999998,"15min":8.3260000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48799999999999999,"15min":0.51900000000000002},"min":{"1min":0.40600000000000003,"5min":0.33900000000000002,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":5.4809999999999999},"last":0.309}]},{"osd":4,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.69499999999999995},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.39600000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":6.4349999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.68999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":8.2970000000000006},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":1.004,"5min":1.004,"15min":5.4160000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66000000000000003,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.41799999999999998,"15min":0.40300000000000002},"max":{"1min":0.83999999999999997,"5min":0.96999999999999997,"15min":6.3869999999999996},"last":0.45900000000000002}]},{"osd":6,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.69599999999999995},"min":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.32900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":8.2189999999999994},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.69699999999999995},"min":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.86899999999999999,"5min":1.0089999999999999,"15min":5.4560000000000004},"last":0.46700000000000003}]},{"osd":7,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67600000000000005,"15min":0.73699999999999999},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":6.4649999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66800000000000004,"15min":0.71799999999999997},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.34899999999999998},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":8.266},"last":0.60499999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542331,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":921600,"data_stored":438188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.39700000000000002},"min":{"1min":0.317,"5min":0.223,"15min":0.223},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.27600000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.40300000000000002},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.24299999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.64900000000000002},"last":0.377},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.436,"15min":0.41899999999999998},"min":{"1min":0.36499999999999999,"5min":0.28599999999999998,"15min":0.214},"max":{"1min":0.53300000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.315}]},{"osd":3,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41799999999999998,"15min":0.40600000000000003},"min":{"1min":0.30299999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.54100000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.48499999999999999,"15min":0.46500000000000002},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.496,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80200000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.626},"min":{"1min":0.499,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76900000000000002,"5min":1.091,"15min":1.091},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67000000000000004,"15min":0.63800000000000001},"min":{"1min":0.52400000000000002,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.755,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.54500000000000004,"5min":0.35599999999999998,"15min":0.34999999999999998},"max":{"1min":0.75600000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.53700000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.85099999999999998,"5min":0.95599999999999996,"15min":0.97899999999999998},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.55700000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79400000000000004,"5min":0.98299999999999998,"15min":0.98799999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.54400000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.874,"5min":0.93100000000000005,"15min":0.998},"last":0.58099999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542332,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335596,"statfs":{"total":96636764160,"available":96599650304,"internally_reserved":0,"allocated":921600,"data_stored":438188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.192},"max":{"1min":0.54800000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":1.0780000000000001},"last":0.26900000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":2.254},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26500000000000001,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":0.754,"15min":6.2240000000000002},"last":0.29099999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.26400000000000001,"15min":0.25},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":2.2909999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.64000000000000001,"5min":0.71199999999999997,"15min":0.72299999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78600000000000003,"5min":0.88700000000000001,"15min":1.24},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63700000000000001},"min":{"1min":0.52600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80300000000000005,"5min":1.0820000000000001,"15min":2.4420000000000002},"last":0.314}]},{"osd":5,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.48599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.97599999999999998,"15min":2.4089999999999998},"last":0.374},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85699999999999998,"5min":0.93999999999999995,"15min":0.996},"last":0.38600000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.5,"5min":0.41299999999999998,"15min":0.39200000000000002},"max":{"1min":0.80200000000000005,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.499,"5min":0.39900000000000002,"15min":0.379},"max":{"1min":0.84499999999999997,"5min":0.99099999999999999,"15min":1.1619999999999999},"last":0.41499999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.64500000000000002},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.81599999999999995,"5min":1.3,"15min":1.3},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79100000000000004,"5min":1.4019999999999999,"15min":2.3740000000000001},"last":0.34000000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542331,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54280,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317560,"statfs":{"total":96636764160,"available":96581181440,"internally_reserved":0,"allocated":913408,"data_stored":433020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.76900000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.248,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.75600000000000001,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84599999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.437,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.70499999999999996,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.60699999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.46700000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71399999999999997,"5min":0.81999999999999995,"15min":0.89100000000000001},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.44900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.83199999999999996,"5min":0.83299999999999996,"15min":0.86499999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.48599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.81000000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.32500000000000001},"max":{"1min":0.73399999999999999,"5min":0.80400000000000005,"15min":0.86899999999999999},"last":0.50900000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.32800000000000001,"15min":0.32200000000000001},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.85899999999999999,"15min":0.85999999999999999},"last":0.46300000000000002}]},{"osd":7,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78600000000000003,"5min":0.80700000000000005,"15min":0.84599999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.84599999999999997,"15min":0.95999999999999996},"last":0.433}]}]},{"osd":5,"up_from":14,"seq":60129542332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":438784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.33100000000000002,"15min":0.253},"max":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.92900000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.65700000000000003},"min":{"1min":0.48999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":8.0370000000000008},"last":0.45200000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":0.753,"5min":0.77500000000000002,"15min":0.98599999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59999999999999998,"15min":0.65200000000000002},"min":{"1min":0.46899999999999997,"5min":0.38,"15min":0.307},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":8.2110000000000003},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.64100000000000001},"min":{"1min":0.46700000000000003,"5min":0.315,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":8.1140000000000008},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.64500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.80200000000000005,"5min":0.88700000000000001,"15min":8.3000000000000007},"last":0.438}]},{"osd":3,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.40799999999999997,"5min":0.34599999999999997,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":1.03},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.41299999999999998,"15min":0.33200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":9.8810000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48599999999999999,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.26000000000000001},"max":{"1min":0.70799999999999996,"5min":0.77400000000000002,"15min":1.0449999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.53200000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82599999999999996,"5min":0.86799999999999999,"15min":1.0780000000000001},"last":0.54800000000000004}]},{"osd":6,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":1.599},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.006},"last":0.52400000000000002}]},{"osd":7,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77400000000000002,"5min":0.95199999999999996,"15min":1.1339999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.889},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:59:27.967 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T09:59:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.120+0000 7f6a3c03f700 1 Processor -- start 2023-10-16T09:59:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.120+0000 7f6a3c03f700 1 -- start start 2023-10-16T09:59:28.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a3c03f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 0x7f6a340af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a3c03f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a340cd470 0x7f6a340cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a3c03f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3406f870 con 0x7f6a340af450 2023-10-16T09:59:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a3c03f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340cde40 con 0x7f6a340cd470 2023-10-16T09:59:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a3c03f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340d1160 con 0x7f6a3415dfd0 2023-10-16T09:59:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a395da700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6a340af450 0x7f6a3406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52846/0 (socket says 172.21.15.181:52846) 2023-10-16T09:59:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a395da700 1 -- 172.21.15.181:0/506184640 learned_addr learned my addr 172.21.15.181:0/506184640 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a3a5dc700 1 --2- 172.21.15.181:0/506184640 >> v2:172.21.15.190:3300/0 conn(0x7f6a340cd470 0x7f6a340cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a39ddb700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 0x7f6a340af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293580455 0 0) 0x7f6a3406f870 con 0x7f6a340af450 2023-10-16T09:59:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a200035b0 con 0x7f6a340af450 2023-10-16T09:59:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446368553 0 0) 0x7f6a200035b0 con 0x7f6a340af450 2023-10-16T09:59:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 msgr2=0x7f6a340af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 0x7f6a340af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 >> v2:172.21.15.190:3300/0 conn(0x7f6a340cd470 msgr2=0x7f6a340cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 --2- 172.21.15.181:0/506184640 >> v2:172.21.15.190:3300/0 conn(0x7f6a340cd470 0x7f6a340cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340cc8e0 con 0x7f6a340af450 2023-10-16T09:59:28.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f6a24001670 con 0x7f6a340af450 2023-10-16T09:59:28.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 >> v1:172.21.15.181:6789/0 conn(0x7f6a340af450 legacy=0x7f6a3406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a20003a00 0x7f6a20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 0x7f6a20004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a20004d90 0x7f6a20005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340cde40 con 0x7f6a20003a00 2023-10-16T09:59:28.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340d1160 con 0x7f6a20004d90 2023-10-16T09:59:28.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2000ba10 con 0x7f6a3415dfd0 2023-10-16T09:59:28.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a3a5dc700 1 --2- 172.21.15.181:0/506184640 >> v2:172.21.15.190:3300/0 conn(0x7f6a340cd470 0x7f6a340cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:28.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.121+0000 7f6a39ddb700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a20003a00 0x7f6a20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a395da700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 0x7f6a20004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a3a5dc700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a20004d90 0x7f6a20005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a39ddb700 1 -- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 msgr2=0x7f6a20004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a39ddb700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a3415dfd0 0x7f6a20004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a39ddb700 1 -- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a20004d90 msgr2=0x7f6a20005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a39ddb700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a20004d90 0x7f6a20005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a39ddb700 1 -- 172.21.15.181:0/506184640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200035b0 con 0x7f6a20003a00 2023-10-16T09:59:28.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a39ddb700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a20003a00 0x7f6a20003e20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009b40 tx=0x7f6a20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0e8cffce5e9017 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30005a20 con 0x7f6a20003a00 2023-10-16T09:59:28.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30005b80 con 0x7f6a20003a00 2023-10-16T09:59:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.122+0000 7f6a38dd9700 1 -- 172.21.15.181:0/506184640 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000e6c0 con 0x7f6a20003a00 2023-10-16T09:59:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.123+0000 7f6a3c03f700 1 -- 172.21.15.181:0/506184640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a20003a00 msgr2=0x7f6a20003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.123+0000 7f6a3c03f700 1 --2- 172.21.15.181:0/506184640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a20003a00 0x7f6a20003e20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009b40 tx=0x7f6a20005880 comp rx=0 tx=0).stop 2023-10-16T09:59:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.123+0000 7f6a3a5dc700 1 -- 172.21.15.181:0/506184640 reap_dead start 2023-10-16T09:59:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.123+0000 7f6a3c03f700 1 -- 172.21.15.181:0/506184640 shutdown_connections 2023-10-16T09:59:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.123+0000 7f6a3c03f700 1 -- 172.21.15.181:0/506184640 >> 172.21.15.181:0/506184640 conn(0x7f6a340d04a0 msgr2=0x7f6a340d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:28.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.123+0000 7f6a3c03f700 1 -- 172.21.15.181:0/506184640 shutdown_connections 2023-10-16T09:59:28.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.123+0000 7f6a3c03f700 1 -- 172.21.15.181:0/506184640 wait complete. 2023-10-16T09:59:28.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.124+0000 7f6a3c03f700 1 Processor -- start 2023-10-16T09:59:28.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.124+0000 7f6a3c03f700 1 -- start start 2023-10-16T09:59:28.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a20003a00 0x7f6a34156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a340b32f0 0x7f6a340b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3415c720 0x7f6a3415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340b44a0 con 0x7f6a340b32f0 2023-10-16T09:59:28.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3415d0e0 con 0x7f6a3415c720 2023-10-16T09:59:28.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a34157170 con 0x7f6a20003a00 2023-10-16T09:59:28.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a340b32f0 0x7f6a340b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a340b32f0 0x7f6a340b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41782/0 (socket says 172.21.15.181:41782) 2023-10-16T09:59:28.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 -- 172.21.15.181:0/3737708556 learned_addr learned my addr 172.21.15.181:0/3737708556 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:28.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 -- 172.21.15.181:0/3737708556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a20003a00 msgr2=0x7f6a34156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a20003a00 0x7f6a34156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 -- 172.21.15.181:0/3737708556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3415c720 msgr2=0x7f6a3415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3415c720 0x7f6a3415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 -- 172.21.15.181:0/3737708556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340b3820 con 0x7f6a340b32f0 2023-10-16T09:59:28.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a395da700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a340b32f0 0x7f6a340b50a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e820 tx=0x7f6a2400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a2a7fc700 1 -- 172.21.15.181:0/3737708556 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24012590 con 0x7f6a340b32f0 2023-10-16T09:59:28.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a2a7fc700 1 -- 172.21.15.181:0/3737708556 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a24012710 con 0x7f6a340b32f0 2023-10-16T09:59:28.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a2a7fc700 1 -- 172.21.15.181:0/3737708556 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24012ac0 con 0x7f6a340b32f0 2023-10-16T09:59:28.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a340b3b00 con 0x7f6a340b32f0 2023-10-16T09:59:28.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.125+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340b40c0 con 0x7f6a340b32f0 2023-10-16T09:59:28.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.126+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a340b4280 con 0x7f6a340b32f0 2023-10-16T09:59:28.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.127+0000 7f6a2a7fc700 1 -- 172.21.15.181:0/3737708556 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6a24012c20 con 0x7f6a340b32f0 2023-10-16T09:59:28.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.127+0000 7f6a2a7fc700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6a1806b020 0x7f6a1806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.127+0000 7f6a2a7fc700 1 -- 172.21.15.181:0/3737708556 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f6a24089730 con 0x7f6a340b32f0 2023-10-16T09:59:28.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.127+0000 7f6a39ddb700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6a1806b020 0x7f6a1806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.127+0000 7f6a39ddb700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6a1806b020 0x7f6a1806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000c00 tx=0x7f6a30013000 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.131+0000 7f6a2a7fc700 1 -- 172.21.15.181:0/3737708556 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6a24054520 con 0x7f6a340b32f0 2023-10-16T09:59:28.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.321+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 --> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a340ccf50 con 0x7f6a1806b020 2023-10-16T09:59:28.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.329+0000 7f6a2a7fc700 1 -- 172.21.15.181:0/3737708556 <== mgr.6222 v2:172.21.15.181:6832/3602430955 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289400 (secure 0 0 0) 0x7f6a30005cf0 con 0x7f6a1806b020 2023-10-16T09:59:28.334 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6a1806b020 msgr2=0x7f6a1806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6a1806b020 0x7f6a1806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000c00 tx=0x7f6a30013000 comp rx=0 tx=0).stop 2023-10-16T09:59:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a340b32f0 msgr2=0x7f6a340b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a340b32f0 0x7f6a340b50a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e820 tx=0x7f6a2400ebe0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 shutdown_connections 2023-10-16T09:59:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f6a1806b020 0x7f6a1806d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a20003a00 0x7f6a34156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3415c720 0x7f6a3415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 --2- 172.21.15.181:0/3737708556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a340b32f0 0x7f6a340b50a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 >> 172.21.15.181:0/3737708556 conn(0x7f6a340d04a0 msgr2=0x7f6a3406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 shutdown_connections 2023-10-16T09:59:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.332+0000 7f6a3c03f700 1 -- 172.21.15.181:0/3737708556 wait complete. 2023-10-16T09:59:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T09:59:28.345 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T09:59:27.656795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306376,"kb_used_data":8348,"kb_used_omap":12,"kb_used_meta":297907,"kb_avail":754668344,"statfs":{"total":773094113280,"available":772780384256,"internally_reserved":0,"allocated":8548352,"data_stored":4683888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":305057336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.823297"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886883+0000","last_change":"2023-10-16T09:59:23.871596+0000","last_active":"2023-10-16T09:59:24.886883+0000","last_peered":"2023-10-16T09:59:24.886883+0000","last_clean":"2023-10-16T09:59:24.886883+0000","last_became_active":"2023-10-16T09:59:23.871220+0000","last_became_peered":"2023-10-16T09:59:23.871220+0000","last_unstale":"2023-10-16T09:59:24.886883+0000","last_undegraded":"2023-10-16T09:59:24.886883+0000","last_fullsized":"2023-10-16T09:59:24.886883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:55.500403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861945+0000","last_change":"2023-10-16T09:59:23.867669+0000","last_active":"2023-10-16T09:59:24.861945+0000","last_peered":"2023-10-16T09:59:24.861945+0000","last_clean":"2023-10-16T09:59:24.861945+0000","last_became_active":"2023-10-16T09:59:23.867396+0000","last_became_peered":"2023-10-16T09:59:23.867396+0000","last_unstale":"2023-10-16T09:59:24.861945+0000","last_undegraded":"2023-10-16T09:59:24.861945+0000","last_fullsized":"2023-10-16T09:59:24.861945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:28.613530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863403+0000","last_change":"2023-10-16T09:59:23.861659+0000","last_active":"2023-10-16T09:59:24.863403+0000","last_peered":"2023-10-16T09:59:24.863403+0000","last_clean":"2023-10-16T09:59:24.863403+0000","last_became_active":"2023-10-16T09:59:23.861388+0000","last_became_peered":"2023-10-16T09:59:23.861388+0000","last_unstale":"2023-10-16T09:59:24.863403+0000","last_undegraded":"2023-10-16T09:59:24.863403+0000","last_fullsized":"2023-10-16T09:59:24.863403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:44:32.859542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864403+0000","last_change":"2023-10-16T09:59:23.864723+0000","last_active":"2023-10-16T09:59:24.864403+0000","last_peered":"2023-10-16T09:59:24.864403+0000","last_clean":"2023-10-16T09:59:24.864403+0000","last_became_active":"2023-10-16T09:59:23.864438+0000","last_became_peered":"2023-10-16T09:59:23.864438+0000","last_unstale":"2023-10-16T09:59:24.864403+0000","last_undegraded":"2023-10-16T09:59:24.864403+0000","last_fullsized":"2023-10-16T09:59:24.864403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:00:23.089299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886769+0000","last_change":"2023-10-16T09:59:23.873975+0000","last_active":"2023-10-16T09:59:24.886769+0000","last_peered":"2023-10-16T09:59:24.886769+0000","last_clean":"2023-10-16T09:59:24.886769+0000","last_became_active":"2023-10-16T09:59:23.873663+0000","last_became_peered":"2023-10-16T09:59:23.873663+0000","last_unstale":"2023-10-16T09:59:24.886769+0000","last_undegraded":"2023-10-16T09:59:24.886769+0000","last_fullsized":"2023-10-16T09:59:24.886769+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:30.429025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885670+0000","last_change":"2023-10-16T09:59:23.872889+0000","last_active":"2023-10-16T09:59:24.885670+0000","last_peered":"2023-10-16T09:59:24.885670+0000","last_clean":"2023-10-16T09:59:24.885670+0000","last_became_active":"2023-10-16T09:59:23.868236+0000","last_became_peered":"2023-10-16T09:59:23.868236+0000","last_unstale":"2023-10-16T09:59:24.885670+0000","last_undegraded":"2023-10-16T09:59:24.885670+0000","last_fullsized":"2023-10-16T09:59:24.885670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:59:30.429025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885705+0000","last_change":"2023-10-16T09:59:23.867279+0000","last_active":"2023-10-16T09:59:24.885705+0000","last_peered":"2023-10-16T09:59:24.885705+0000","last_clean":"2023-10-16T09:59:24.885705+0000","last_became_active":"2023-10-16T09:59:23.866982+0000","last_became_peered":"2023-10-16T09:59:23.866982+0000","last_unstale":"2023-10-16T09:59:24.885705+0000","last_undegraded":"2023-10-16T09:59:24.885705+0000","last_fullsized":"2023-10-16T09:59:24.885705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:05.847190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861680+0000","last_change":"2023-10-16T09:59:23.867731+0000","last_active":"2023-10-16T09:59:24.861680+0000","last_peered":"2023-10-16T09:59:24.861680+0000","last_clean":"2023-10-16T09:59:24.861680+0000","last_became_active":"2023-10-16T09:59:23.867440+0000","last_became_peered":"2023-10-16T09:59:23.867440+0000","last_unstale":"2023-10-16T09:59:24.861680+0000","last_undegraded":"2023-10-16T09:59:24.861680+0000","last_fullsized":"2023-10-16T09:59:24.861680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:03:59.004731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886576+0000","last_change":"2023-10-16T09:59:23.874003+0000","last_active":"2023-10-16T09:59:24.886576+0000","last_peered":"2023-10-16T09:59:24.886576+0000","last_clean":"2023-10-16T09:59:24.886576+0000","last_became_active":"2023-10-16T09:59:23.873735+0000","last_became_peered":"2023-10-16T09:59:23.873735+0000","last_unstale":"2023-10-16T09:59:24.886576+0000","last_undegraded":"2023-10-16T09:59:24.886576+0000","last_fullsized":"2023-10-16T09:59:24.886576+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:04:36.969551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886106+0000","last_change":"2023-10-16T09:59:23.871372+0000","last_active":"2023-10-16T09:59:24.886106+0000","last_peered":"2023-10-16T09:59:24.886106+0000","last_clean":"2023-10-16T09:59:24.886106+0000","last_became_active":"2023-10-16T09:59:23.870964+0000","last_became_peered":"2023-10-16T09:59:23.870964+0000","last_unstale":"2023-10-16T09:59:24.886106+0000","last_undegraded":"2023-10-16T09:59:24.886106+0000","last_fullsized":"2023-10-16T09:59:24.886106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:41.707190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886069+0000","last_change":"2023-10-16T09:59:23.868030+0000","last_active":"2023-10-16T09:59:24.886069+0000","last_peered":"2023-10-16T09:59:24.886069+0000","last_clean":"2023-10-16T09:59:24.886069+0000","last_became_active":"2023-10-16T09:59:23.867764+0000","last_became_peered":"2023-10-16T09:59:23.867764+0000","last_unstale":"2023-10-16T09:59:24.886069+0000","last_undegraded":"2023-10-16T09:59:24.886069+0000","last_fullsized":"2023-10-16T09:59:24.886069+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:33.895462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864554+0000","last_change":"2023-10-16T09:59:23.863343+0000","last_active":"2023-10-16T09:59:24.864554+0000","last_peered":"2023-10-16T09:59:24.864554+0000","last_clean":"2023-10-16T09:59:24.864554+0000","last_became_active":"2023-10-16T09:59:23.863044+0000","last_became_peered":"2023-10-16T09:59:23.863044+0000","last_unstale":"2023-10-16T09:59:24.864554+0000","last_undegraded":"2023-10-16T09:59:24.864554+0000","last_fullsized":"2023-10-16T09:59:24.864554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:35.096865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886289+0000","last_change":"2023-10-16T09:59:23.875396+0000","last_active":"2023-10-16T09:59:24.886289+0000","last_peered":"2023-10-16T09:59:24.886289+0000","last_clean":"2023-10-16T09:59:24.886289+0000","last_became_active":"2023-10-16T09:59:23.875136+0000","last_became_peered":"2023-10-16T09:59:23.875136+0000","last_unstale":"2023-10-16T09:59:24.886289+0000","last_undegraded":"2023-10-16T09:59:24.886289+0000","last_fullsized":"2023-10-16T09:59:24.886289+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:02.821717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885865+0000","last_change":"2023-10-16T09:59:23.866898+0000","last_active":"2023-10-16T09:59:24.885865+0000","last_peered":"2023-10-16T09:59:24.885865+0000","last_clean":"2023-10-16T09:59:24.885865+0000","last_became_active":"2023-10-16T09:59:23.866614+0000","last_became_peered":"2023-10-16T09:59:23.866614+0000","last_unstale":"2023-10-16T09:59:24.885865+0000","last_undegraded":"2023-10-16T09:59:24.885865+0000","last_fullsized":"2023-10-16T09:59:24.885865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:10.799717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885728+0000","last_change":"2023-10-16T09:59:23.865712+0000","last_active":"2023-10-16T09:59:24.885728+0000","last_peered":"2023-10-16T09:59:24.885728+0000","last_clean":"2023-10-16T09:59:24.885728+0000","last_became_active":"2023-10-16T09:59:23.865396+0000","last_became_peered":"2023-10-16T09:59:23.865396+0000","last_unstale":"2023-10-16T09:59:24.885728+0000","last_undegraded":"2023-10-16T09:59:24.885728+0000","last_fullsized":"2023-10-16T09:59:24.885728+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:10.083720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885597+0000","last_change":"2023-10-16T09:59:23.873944+0000","last_active":"2023-10-16T09:59:24.885597+0000","last_peered":"2023-10-16T09:59:24.885597+0000","last_clean":"2023-10-16T09:59:24.885597+0000","last_became_active":"2023-10-16T09:59:23.872780+0000","last_became_peered":"2023-10-16T09:59:23.872780+0000","last_unstale":"2023-10-16T09:59:24.885597+0000","last_undegraded":"2023-10-16T09:59:24.885597+0000","last_fullsized":"2023-10-16T09:59:24.885597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:16:05.516422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862324+0000","last_change":"2023-10-16T09:59:23.882778+0000","last_active":"2023-10-16T09:59:24.862324+0000","last_peered":"2023-10-16T09:59:24.862324+0000","last_clean":"2023-10-16T09:59:24.862324+0000","last_became_active":"2023-10-16T09:59:23.881726+0000","last_became_peered":"2023-10-16T09:59:23.881726+0000","last_unstale":"2023-10-16T09:59:24.862324+0000","last_undegraded":"2023-10-16T09:59:24.862324+0000","last_fullsized":"2023-10-16T09:59:24.862324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:19.435418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863376+0000","last_change":"2023-10-16T09:59:23.875314+0000","last_active":"2023-10-16T09:59:24.863376+0000","last_peered":"2023-10-16T09:59:24.863376+0000","last_clean":"2023-10-16T09:59:24.863376+0000","last_became_active":"2023-10-16T09:59:23.875052+0000","last_became_peered":"2023-10-16T09:59:23.875052+0000","last_unstale":"2023-10-16T09:59:24.863376+0000","last_undegraded":"2023-10-16T09:59:24.863376+0000","last_fullsized":"2023-10-16T09:59:24.863376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:38:56.602134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886872+0000","last_change":"2023-10-16T09:59:23.870931+0000","last_active":"2023-10-16T09:59:24.886872+0000","last_peered":"2023-10-16T09:59:24.886872+0000","last_clean":"2023-10-16T09:59:24.886872+0000","last_became_active":"2023-10-16T09:59:23.870639+0000","last_became_peered":"2023-10-16T09:59:23.870639+0000","last_unstale":"2023-10-16T09:59:24.886872+0000","last_undegraded":"2023-10-16T09:59:24.886872+0000","last_fullsized":"2023-10-16T09:59:24.886872+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:15:46.837454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861761+0000","last_change":"2023-10-16T09:59:23.875578+0000","last_active":"2023-10-16T09:59:24.861761+0000","last_peered":"2023-10-16T09:59:24.861761+0000","last_clean":"2023-10-16T09:59:24.861761+0000","last_became_active":"2023-10-16T09:59:23.875282+0000","last_became_peered":"2023-10-16T09:59:23.875282+0000","last_unstale":"2023-10-16T09:59:24.861761+0000","last_undegraded":"2023-10-16T09:59:24.861761+0000","last_fullsized":"2023-10-16T09:59:24.861761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:30.026317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863565+0000","last_change":"2023-10-16T09:59:23.863702+0000","last_active":"2023-10-16T09:59:24.863565+0000","last_peered":"2023-10-16T09:59:24.863565+0000","last_clean":"2023-10-16T09:59:24.863565+0000","last_became_active":"2023-10-16T09:59:23.863411+0000","last_became_peered":"2023-10-16T09:59:23.863411+0000","last_unstale":"2023-10-16T09:59:24.863565+0000","last_undegraded":"2023-10-16T09:59:24.863565+0000","last_fullsized":"2023-10-16T09:59:24.863565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:16.813557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862180+0000","last_change":"2023-10-16T09:59:23.882625+0000","last_active":"2023-10-16T09:59:24.862180+0000","last_peered":"2023-10-16T09:59:24.862180+0000","last_clean":"2023-10-16T09:59:24.862180+0000","last_became_active":"2023-10-16T09:59:23.882349+0000","last_became_peered":"2023-10-16T09:59:23.882349+0000","last_unstale":"2023-10-16T09:59:24.862180+0000","last_undegraded":"2023-10-16T09:59:24.862180+0000","last_fullsized":"2023-10-16T09:59:24.862180+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:36.532587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863607+0000","last_change":"2023-10-16T09:59:23.868274+0000","last_active":"2023-10-16T09:59:24.863607+0000","last_peered":"2023-10-16T09:59:24.863607+0000","last_clean":"2023-10-16T09:59:24.863607+0000","last_became_active":"2023-10-16T09:59:23.867840+0000","last_became_peered":"2023-10-16T09:59:23.867840+0000","last_unstale":"2023-10-16T09:59:24.863607+0000","last_undegraded":"2023-10-16T09:59:24.863607+0000","last_fullsized":"2023-10-16T09:59:24.863607+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:06.040676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886138+0000","last_change":"2023-10-16T09:59:23.876394+0000","last_active":"2023-10-16T09:59:24.886138+0000","last_peered":"2023-10-16T09:59:24.886138+0000","last_clean":"2023-10-16T09:59:24.886138+0000","last_became_active":"2023-10-16T09:59:23.876060+0000","last_became_peered":"2023-10-16T09:59:23.876060+0000","last_unstale":"2023-10-16T09:59:24.886138+0000","last_undegraded":"2023-10-16T09:59:24.886138+0000","last_fullsized":"2023-10-16T09:59:24.886138+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:13:50.899643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886988+0000","last_change":"2023-10-16T09:59:23.864926+0000","last_active":"2023-10-16T09:59:24.886988+0000","last_peered":"2023-10-16T09:59:24.886988+0000","last_clean":"2023-10-16T09:59:24.886988+0000","last_became_active":"2023-10-16T09:59:23.863277+0000","last_became_peered":"2023-10-16T09:59:23.863277+0000","last_unstale":"2023-10-16T09:59:24.886988+0000","last_undegraded":"2023-10-16T09:59:24.886988+0000","last_fullsized":"2023-10-16T09:59:24.886988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:46.877740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864421+0000","last_change":"2023-10-16T09:59:23.862267+0000","last_active":"2023-10-16T09:59:24.864421+0000","last_peered":"2023-10-16T09:59:24.864421+0000","last_clean":"2023-10-16T09:59:24.864421+0000","last_became_active":"2023-10-16T09:59:23.861896+0000","last_became_peered":"2023-10-16T09:59:23.861896+0000","last_unstale":"2023-10-16T09:59:24.864421+0000","last_undegraded":"2023-10-16T09:59:24.864421+0000","last_fullsized":"2023-10-16T09:59:24.864421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:04:00.864560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885874+0000","last_change":"2023-10-16T09:59:23.873834+0000","last_active":"2023-10-16T09:59:24.885874+0000","last_peered":"2023-10-16T09:59:24.885874+0000","last_clean":"2023-10-16T09:59:24.885874+0000","last_became_active":"2023-10-16T09:59:23.873432+0000","last_became_peered":"2023-10-16T09:59:23.873432+0000","last_unstale":"2023-10-16T09:59:24.885874+0000","last_undegraded":"2023-10-16T09:59:24.885874+0000","last_fullsized":"2023-10-16T09:59:24.885874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:46.877740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886887+0000","last_change":"2023-10-16T09:59:23.874041+0000","last_active":"2023-10-16T09:59:24.886887+0000","last_peered":"2023-10-16T09:59:24.886887+0000","last_clean":"2023-10-16T09:59:24.886887+0000","last_became_active":"2023-10-16T09:59:23.873752+0000","last_became_peered":"2023-10-16T09:59:23.873752+0000","last_unstale":"2023-10-16T09:59:24.886887+0000","last_undegraded":"2023-10-16T09:59:24.886887+0000","last_fullsized":"2023-10-16T09:59:24.886887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:01.092448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861931+0000","last_change":"2023-10-16T09:59:23.869388+0000","last_active":"2023-10-16T09:59:24.861931+0000","last_peered":"2023-10-16T09:59:24.861931+0000","last_clean":"2023-10-16T09:59:24.861931+0000","last_became_active":"2023-10-16T09:59:23.866868+0000","last_became_peered":"2023-10-16T09:59:23.866868+0000","last_unstale":"2023-10-16T09:59:24.861931+0000","last_undegraded":"2023-10-16T09:59:24.861931+0000","last_fullsized":"2023-10-16T09:59:24.861931+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:30.282547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863694+0000","last_change":"2023-10-16T09:59:23.858517+0000","last_active":"2023-10-16T09:59:24.863694+0000","last_peered":"2023-10-16T09:59:24.863694+0000","last_clean":"2023-10-16T09:59:24.863694+0000","last_became_active":"2023-10-16T09:59:23.858219+0000","last_became_peered":"2023-10-16T09:59:23.858219+0000","last_unstale":"2023-10-16T09:59:24.863694+0000","last_undegraded":"2023-10-16T09:59:24.863694+0000","last_fullsized":"2023-10-16T09:59:24.863694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:49:37.854267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886586+0000","last_change":"2023-10-16T09:59:23.863737+0000","last_active":"2023-10-16T09:59:24.886586+0000","last_peered":"2023-10-16T09:59:24.886586+0000","last_clean":"2023-10-16T09:59:24.886586+0000","last_became_active":"2023-10-16T09:59:23.863447+0000","last_became_peered":"2023-10-16T09:59:23.863447+0000","last_unstale":"2023-10-16T09:59:24.886586+0000","last_undegraded":"2023-10-16T09:59:24.886586+0000","last_fullsized":"2023-10-16T09:59:24.886586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:16:05.516422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885727+0000","last_change":"2023-10-16T09:59:23.874281+0000","last_active":"2023-10-16T09:59:24.885727+0000","last_peered":"2023-10-16T09:59:24.885727+0000","last_clean":"2023-10-16T09:59:24.885727+0000","last_became_active":"2023-10-16T09:59:23.872638+0000","last_became_peered":"2023-10-16T09:59:23.872638+0000","last_unstale":"2023-10-16T09:59:24.885727+0000","last_undegraded":"2023-10-16T09:59:24.885727+0000","last_fullsized":"2023-10-16T09:59:24.885727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:01.092448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885871+0000","last_change":"2023-10-16T09:59:23.869020+0000","last_active":"2023-10-16T09:59:24.885871+0000","last_peered":"2023-10-16T09:59:24.885871+0000","last_clean":"2023-10-16T09:59:24.885871+0000","last_became_active":"2023-10-16T09:59:23.868734+0000","last_became_peered":"2023-10-16T09:59:23.868734+0000","last_unstale":"2023-10-16T09:59:24.885871+0000","last_undegraded":"2023-10-16T09:59:24.885871+0000","last_fullsized":"2023-10-16T09:59:24.885871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:24:55.500403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"75'4","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886226+0000","last_change":"2023-10-16T09:59:22.853854+0000","last_active":"2023-10-16T09:59:24.886226+0000","last_peered":"2023-10-16T09:59:24.886226+0000","last_clean":"2023-10-16T09:59:24.886226+0000","last_became_active":"2023-10-16T09:59:22.853583+0000","last_became_peered":"2023-10-16T09:59:22.853583+0000","last_unstale":"2023-10-16T09:59:24.886226+0000","last_undegraded":"2023-10-16T09:59:24.886226+0000","last_fullsized":"2023-10-16T09:59:24.886226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:50.002839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886190+0000","last_change":"2023-10-16T09:59:23.869456+0000","last_active":"2023-10-16T09:59:24.886190+0000","last_peered":"2023-10-16T09:59:24.886190+0000","last_clean":"2023-10-16T09:59:24.886190+0000","last_became_active":"2023-10-16T09:59:23.869056+0000","last_became_peered":"2023-10-16T09:59:23.869056+0000","last_unstale":"2023-10-16T09:59:24.886190+0000","last_undegraded":"2023-10-16T09:59:24.886190+0000","last_fullsized":"2023-10-16T09:59:24.886190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:19.631749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862046+0000","last_change":"2023-10-16T09:59:22.848463+0000","last_active":"2023-10-16T09:59:24.862046+0000","last_peered":"2023-10-16T09:59:24.862046+0000","last_clean":"2023-10-16T09:59:24.862046+0000","last_became_active":"2023-10-16T09:59:22.848208+0000","last_became_peered":"2023-10-16T09:59:22.848208+0000","last_unstale":"2023-10-16T09:59:24.862046+0000","last_undegraded":"2023-10-16T09:59:24.862046+0000","last_fullsized":"2023-10-16T09:59:24.862046+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:09.064449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885994+0000","last_change":"2023-10-16T09:59:23.866976+0000","last_active":"2023-10-16T09:59:24.885994+0000","last_peered":"2023-10-16T09:59:24.885994+0000","last_clean":"2023-10-16T09:59:24.885994+0000","last_became_active":"2023-10-16T09:59:23.866711+0000","last_became_peered":"2023-10-16T09:59:23.866711+0000","last_unstale":"2023-10-16T09:59:24.885994+0000","last_undegraded":"2023-10-16T09:59:24.885994+0000","last_fullsized":"2023-10-16T09:59:24.885994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:14.455020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886093+0000","last_change":"2023-10-16T09:59:22.848967+0000","last_active":"2023-10-16T09:59:24.886093+0000","last_peered":"2023-10-16T09:59:24.886093+0000","last_clean":"2023-10-16T09:59:24.886093+0000","last_became_active":"2023-10-16T09:59:22.848660+0000","last_became_peered":"2023-10-16T09:59:22.848660+0000","last_unstale":"2023-10-16T09:59:24.886093+0000","last_undegraded":"2023-10-16T09:59:24.886093+0000","last_fullsized":"2023-10-16T09:59:24.886093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886003+0000","last_change":"2023-10-16T09:59:23.873772+0000","last_active":"2023-10-16T09:59:24.886003+0000","last_peered":"2023-10-16T09:59:24.886003+0000","last_clean":"2023-10-16T09:59:24.886003+0000","last_became_active":"2023-10-16T09:59:23.873124+0000","last_became_peered":"2023-10-16T09:59:23.873124+0000","last_unstale":"2023-10-16T09:59:24.886003+0000","last_undegraded":"2023-10-16T09:59:24.886003+0000","last_fullsized":"2023-10-16T09:59:24.886003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:36.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864470+0000","last_change":"2023-10-16T09:59:22.852120+0000","last_active":"2023-10-16T09:59:24.864470+0000","last_peered":"2023-10-16T09:59:24.864470+0000","last_clean":"2023-10-16T09:59:24.864470+0000","last_became_active":"2023-10-16T09:59:22.851826+0000","last_became_peered":"2023-10-16T09:59:22.851826+0000","last_unstale":"2023-10-16T09:59:24.864470+0000","last_undegraded":"2023-10-16T09:59:24.864470+0000","last_fullsized":"2023-10-16T09:59:24.864470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:58.125943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862041+0000","last_change":"2023-10-16T09:59:23.882543+0000","last_active":"2023-10-16T09:59:24.862041+0000","last_peered":"2023-10-16T09:59:24.862041+0000","last_clean":"2023-10-16T09:59:24.862041+0000","last_became_active":"2023-10-16T09:59:23.882242+0000","last_became_peered":"2023-10-16T09:59:23.882242+0000","last_unstale":"2023-10-16T09:59:24.862041+0000","last_undegraded":"2023-10-16T09:59:24.862041+0000","last_fullsized":"2023-10-16T09:59:24.862041+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:01.870085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885919+0000","last_change":"2023-10-16T09:59:22.854008+0000","last_active":"2023-10-16T09:59:24.885919+0000","last_peered":"2023-10-16T09:59:24.885919+0000","last_clean":"2023-10-16T09:59:24.885919+0000","last_became_active":"2023-10-16T09:59:22.853695+0000","last_became_peered":"2023-10-16T09:59:22.853695+0000","last_unstale":"2023-10-16T09:59:24.885919+0000","last_undegraded":"2023-10-16T09:59:24.885919+0000","last_fullsized":"2023-10-16T09:59:24.885919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:28:47.452567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861839+0000","last_change":"2023-10-16T09:59:23.874784+0000","last_active":"2023-10-16T09:59:24.861839+0000","last_peered":"2023-10-16T09:59:24.861839+0000","last_clean":"2023-10-16T09:59:24.861839+0000","last_became_active":"2023-10-16T09:59:23.874497+0000","last_became_peered":"2023-10-16T09:59:23.874497+0000","last_unstale":"2023-10-16T09:59:24.861839+0000","last_undegraded":"2023-10-16T09:59:24.861839+0000","last_fullsized":"2023-10-16T09:59:24.861839+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:40:50.582019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885677+0000","last_change":"2023-10-16T09:59:22.849437+0000","last_active":"2023-10-16T09:59:24.885677+0000","last_peered":"2023-10-16T09:59:24.885677+0000","last_clean":"2023-10-16T09:59:24.885677+0000","last_became_active":"2023-10-16T09:59:22.849172+0000","last_became_peered":"2023-10-16T09:59:22.849172+0000","last_unstale":"2023-10-16T09:59:24.885677+0000","last_undegraded":"2023-10-16T09:59:24.885677+0000","last_fullsized":"2023-10-16T09:59:24.885677+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:50.830805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886214+0000","last_change":"2023-10-16T09:59:23.870219+0000","last_active":"2023-10-16T09:59:24.886214+0000","last_peered":"2023-10-16T09:59:24.886214+0000","last_clean":"2023-10-16T09:59:24.886214+0000","last_became_active":"2023-10-16T09:59:23.869734+0000","last_became_peered":"2023-10-16T09:59:23.869734+0000","last_unstale":"2023-10-16T09:59:24.886214+0000","last_undegraded":"2023-10-16T09:59:24.886214+0000","last_fullsized":"2023-10-16T09:59:24.886214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:23.878030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864140+0000","last_change":"2023-10-16T09:59:22.851804+0000","last_active":"2023-10-16T09:59:24.864140+0000","last_peered":"2023-10-16T09:59:24.864140+0000","last_clean":"2023-10-16T09:59:24.864140+0000","last_became_active":"2023-10-16T09:59:22.851446+0000","last_became_peered":"2023-10-16T09:59:22.851446+0000","last_unstale":"2023-10-16T09:59:24.864140+0000","last_undegraded":"2023-10-16T09:59:24.864140+0000","last_fullsized":"2023-10-16T09:59:24.864140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:49:37.665105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862129+0000","last_change":"2023-10-16T09:59:23.882566+0000","last_active":"2023-10-16T09:59:24.862129+0000","last_peered":"2023-10-16T09:59:24.862129+0000","last_clean":"2023-10-16T09:59:24.862129+0000","last_became_active":"2023-10-16T09:59:23.881771+0000","last_became_peered":"2023-10-16T09:59:23.881771+0000","last_unstale":"2023-10-16T09:59:24.862129+0000","last_undegraded":"2023-10-16T09:59:24.862129+0000","last_fullsized":"2023-10-16T09:59:24.862129+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:32:36.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885668+0000","last_change":"2023-10-16T09:59:22.854268+0000","last_active":"2023-10-16T09:59:24.885668+0000","last_peered":"2023-10-16T09:59:24.885668+0000","last_clean":"2023-10-16T09:59:24.885668+0000","last_became_active":"2023-10-16T09:59:22.853981+0000","last_became_peered":"2023-10-16T09:59:22.853981+0000","last_unstale":"2023-10-16T09:59:24.885668+0000","last_undegraded":"2023-10-16T09:59:24.885668+0000","last_fullsized":"2023-10-16T09:59:24.885668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:05:59.547302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886558+0000","last_change":"2023-10-16T09:59:23.865849+0000","last_active":"2023-10-16T09:59:24.886558+0000","last_peered":"2023-10-16T09:59:24.886558+0000","last_clean":"2023-10-16T09:59:24.886558+0000","last_became_active":"2023-10-16T09:59:23.865567+0000","last_became_peered":"2023-10-16T09:59:23.865567+0000","last_unstale":"2023-10-16T09:59:24.886558+0000","last_undegraded":"2023-10-16T09:59:24.886558+0000","last_fullsized":"2023-10-16T09:59:24.886558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:13.227930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863725+0000","last_change":"2023-10-16T09:59:22.854126+0000","last_active":"2023-10-16T09:59:24.863725+0000","last_peered":"2023-10-16T09:59:24.863725+0000","last_clean":"2023-10-16T09:59:24.863725+0000","last_became_active":"2023-10-16T09:59:22.853844+0000","last_became_peered":"2023-10-16T09:59:22.853844+0000","last_unstale":"2023-10-16T09:59:24.863725+0000","last_undegraded":"2023-10-16T09:59:24.863725+0000","last_fullsized":"2023-10-16T09:59:24.863725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:29.403986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863454+0000","last_change":"2023-10-16T09:59:23.867446+0000","last_active":"2023-10-16T09:59:24.863454+0000","last_peered":"2023-10-16T09:59:24.863454+0000","last_clean":"2023-10-16T09:59:24.863454+0000","last_became_active":"2023-10-16T09:59:23.867106+0000","last_became_peered":"2023-10-16T09:59:23.867106+0000","last_unstale":"2023-10-16T09:59:24.863454+0000","last_undegraded":"2023-10-16T09:59:24.863454+0000","last_fullsized":"2023-10-16T09:59:24.863454+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:47:27.270775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862128+0000","last_change":"2023-10-16T09:59:22.854038+0000","last_active":"2023-10-16T09:59:24.862128+0000","last_peered":"2023-10-16T09:59:24.862128+0000","last_clean":"2023-10-16T09:59:24.862128+0000","last_became_active":"2023-10-16T09:59:22.853752+0000","last_became_peered":"2023-10-16T09:59:22.853752+0000","last_unstale":"2023-10-16T09:59:24.862128+0000","last_undegraded":"2023-10-16T09:59:24.862128+0000","last_fullsized":"2023-10-16T09:59:24.862128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:29:42.947785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885670+0000","last_change":"2023-10-16T09:59:23.869198+0000","last_active":"2023-10-16T09:59:24.885670+0000","last_peered":"2023-10-16T09:59:24.885670+0000","last_clean":"2023-10-16T09:59:24.885670+0000","last_became_active":"2023-10-16T09:59:23.868713+0000","last_became_peered":"2023-10-16T09:59:23.868713+0000","last_unstale":"2023-10-16T09:59:24.885670+0000","last_undegraded":"2023-10-16T09:59:24.885670+0000","last_fullsized":"2023-10-16T09:59:24.885670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:03.528976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"75'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863964+0000","last_change":"2023-10-16T09:59:22.849709+0000","last_active":"2023-10-16T09:59:24.863964+0000","last_peered":"2023-10-16T09:59:24.863964+0000","last_clean":"2023-10-16T09:59:24.863964+0000","last_became_active":"2023-10-16T09:59:22.849438+0000","last_became_peered":"2023-10-16T09:59:22.849438+0000","last_unstale":"2023-10-16T09:59:24.863964+0000","last_undegraded":"2023-10-16T09:59:24.863964+0000","last_fullsized":"2023-10-16T09:59:24.863964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:34.092604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863547+0000","last_change":"2023-10-16T09:59:23.859271+0000","last_active":"2023-10-16T09:59:24.863547+0000","last_peered":"2023-10-16T09:59:24.863547+0000","last_clean":"2023-10-16T09:59:24.863547+0000","last_became_active":"2023-10-16T09:59:23.858989+0000","last_became_peered":"2023-10-16T09:59:23.858989+0000","last_unstale":"2023-10-16T09:59:24.863547+0000","last_undegraded":"2023-10-16T09:59:24.863547+0000","last_fullsized":"2023-10-16T09:59:24.863547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:23.935998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"75'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886468+0000","last_change":"2023-10-16T09:59:22.853644+0000","last_active":"2023-10-16T09:59:24.886468+0000","last_peered":"2023-10-16T09:59:24.886468+0000","last_clean":"2023-10-16T09:59:24.886468+0000","last_became_active":"2023-10-16T09:59:22.853312+0000","last_became_peered":"2023-10-16T09:59:22.853312+0000","last_unstale":"2023-10-16T09:59:24.886468+0000","last_undegraded":"2023-10-16T09:59:24.886468+0000","last_fullsized":"2023-10-16T09:59:24.886468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:16.621595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864250+0000","last_change":"2023-10-16T09:59:23.861706+0000","last_active":"2023-10-16T09:59:24.864250+0000","last_peered":"2023-10-16T09:59:24.864250+0000","last_clean":"2023-10-16T09:59:24.864250+0000","last_became_active":"2023-10-16T09:59:23.861391+0000","last_became_peered":"2023-10-16T09:59:23.861391+0000","last_unstale":"2023-10-16T09:59:24.864250+0000","last_undegraded":"2023-10-16T09:59:24.864250+0000","last_fullsized":"2023-10-16T09:59:24.864250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:13.434649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861984+0000","last_change":"2023-10-16T09:59:22.853983+0000","last_active":"2023-10-16T09:59:24.861984+0000","last_peered":"2023-10-16T09:59:24.861984+0000","last_clean":"2023-10-16T09:59:24.861984+0000","last_became_active":"2023-10-16T09:59:22.853696+0000","last_became_peered":"2023-10-16T09:59:22.853696+0000","last_unstale":"2023-10-16T09:59:24.861984+0000","last_undegraded":"2023-10-16T09:59:24.861984+0000","last_fullsized":"2023-10-16T09:59:24.861984+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:32:46.001474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863317+0000","last_change":"2023-10-16T09:59:23.868422+0000","last_active":"2023-10-16T09:59:24.863317+0000","last_peered":"2023-10-16T09:59:24.863317+0000","last_clean":"2023-10-16T09:59:24.863317+0000","last_became_active":"2023-10-16T09:59:23.867704+0000","last_became_peered":"2023-10-16T09:59:23.867704+0000","last_unstale":"2023-10-16T09:59:24.863317+0000","last_undegraded":"2023-10-16T09:59:24.863317+0000","last_fullsized":"2023-10-16T09:59:24.863317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:58.731906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886114+0000","last_change":"2023-10-16T09:59:22.848206+0000","last_active":"2023-10-16T09:59:24.886114+0000","last_peered":"2023-10-16T09:59:24.886114+0000","last_clean":"2023-10-16T09:59:24.886114+0000","last_became_active":"2023-10-16T09:59:22.847918+0000","last_became_peered":"2023-10-16T09:59:22.847918+0000","last_unstale":"2023-10-16T09:59:24.886114+0000","last_undegraded":"2023-10-16T09:59:24.886114+0000","last_fullsized":"2023-10-16T09:59:24.886114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:35.191382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885659+0000","last_change":"2023-10-16T09:59:23.863919+0000","last_active":"2023-10-16T09:59:24.885659+0000","last_peered":"2023-10-16T09:59:24.885659+0000","last_clean":"2023-10-16T09:59:24.885659+0000","last_became_active":"2023-10-16T09:59:23.863629+0000","last_became_peered":"2023-10-16T09:59:23.863629+0000","last_unstale":"2023-10-16T09:59:24.885659+0000","last_undegraded":"2023-10-16T09:59:24.885659+0000","last_fullsized":"2023-10-16T09:59:24.885659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:27.131791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864185+0000","last_change":"2023-10-16T09:59:22.852360+0000","last_active":"2023-10-16T09:59:24.864185+0000","last_peered":"2023-10-16T09:59:24.864185+0000","last_clean":"2023-10-16T09:59:24.864185+0000","last_became_active":"2023-10-16T09:59:22.852081+0000","last_became_peered":"2023-10-16T09:59:22.852081+0000","last_unstale":"2023-10-16T09:59:24.864185+0000","last_undegraded":"2023-10-16T09:59:24.864185+0000","last_fullsized":"2023-10-16T09:59:24.864185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:42.851600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863381+0000","last_change":"2023-10-16T09:59:23.868100+0000","last_active":"2023-10-16T09:59:24.863381+0000","last_peered":"2023-10-16T09:59:24.863381+0000","last_clean":"2023-10-16T09:59:24.863381+0000","last_became_active":"2023-10-16T09:59:23.867499+0000","last_became_peered":"2023-10-16T09:59:23.867499+0000","last_unstale":"2023-10-16T09:59:24.863381+0000","last_undegraded":"2023-10-16T09:59:24.863381+0000","last_fullsized":"2023-10-16T09:59:24.863381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:08:38.630000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"75'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862941+0000","last_change":"2023-10-16T09:59:22.851868+0000","last_active":"2023-10-16T09:59:24.862941+0000","last_peered":"2023-10-16T09:59:24.862941+0000","last_clean":"2023-10-16T09:59:24.862941+0000","last_became_active":"2023-10-16T09:59:22.851583+0000","last_became_peered":"2023-10-16T09:59:22.851583+0000","last_unstale":"2023-10-16T09:59:24.862941+0000","last_undegraded":"2023-10-16T09:59:24.862941+0000","last_fullsized":"2023-10-16T09:59:24.862941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:59.860300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863563+0000","last_change":"2023-10-16T09:59:23.867652+0000","last_active":"2023-10-16T09:59:24.863563+0000","last_peered":"2023-10-16T09:59:24.863563+0000","last_clean":"2023-10-16T09:59:24.863563+0000","last_became_active":"2023-10-16T09:59:23.867338+0000","last_became_peered":"2023-10-16T09:59:23.867338+0000","last_unstale":"2023-10-16T09:59:24.863563+0000","last_undegraded":"2023-10-16T09:59:24.863563+0000","last_fullsized":"2023-10-16T09:59:24.863563+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:18.049530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862187+0000","last_change":"2023-10-16T09:59:22.854214+0000","last_active":"2023-10-16T09:59:24.862187+0000","last_peered":"2023-10-16T09:59:24.862187+0000","last_clean":"2023-10-16T09:59:24.862187+0000","last_became_active":"2023-10-16T09:59:22.853551+0000","last_became_peered":"2023-10-16T09:59:22.853551+0000","last_unstale":"2023-10-16T09:59:24.862187+0000","last_undegraded":"2023-10-16T09:59:24.862187+0000","last_fullsized":"2023-10-16T09:59:24.862187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:02.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864800+0000","last_change":"2023-10-16T09:59:23.861818+0000","last_active":"2023-10-16T09:59:24.864800+0000","last_peered":"2023-10-16T09:59:24.864800+0000","last_clean":"2023-10-16T09:59:24.864800+0000","last_became_active":"2023-10-16T09:59:23.861173+0000","last_became_peered":"2023-10-16T09:59:23.861173+0000","last_unstale":"2023-10-16T09:59:24.864800+0000","last_undegraded":"2023-10-16T09:59:24.864800+0000","last_fullsized":"2023-10-16T09:59:24.864800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:06.629359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885962+0000","last_change":"2023-10-16T09:59:22.856772+0000","last_active":"2023-10-16T09:59:24.885962+0000","last_peered":"2023-10-16T09:59:24.885962+0000","last_clean":"2023-10-16T09:59:24.885962+0000","last_became_active":"2023-10-16T09:59:22.856449+0000","last_became_peered":"2023-10-16T09:59:22.856449+0000","last_unstale":"2023-10-16T09:59:24.885962+0000","last_undegraded":"2023-10-16T09:59:24.885962+0000","last_fullsized":"2023-10-16T09:59:24.885962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:48.796107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.861911+0000","last_change":"2023-10-16T09:59:23.866857+0000","last_active":"2023-10-16T09:59:24.861911+0000","last_peered":"2023-10-16T09:59:24.861911+0000","last_clean":"2023-10-16T09:59:24.861911+0000","last_became_active":"2023-10-16T09:59:23.866555+0000","last_became_peered":"2023-10-16T09:59:23.866555+0000","last_unstale":"2023-10-16T09:59:24.861911+0000","last_undegraded":"2023-10-16T09:59:24.861911+0000","last_fullsized":"2023-10-16T09:59:24.861911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:49.606290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"75'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864426+0000","last_change":"2023-10-16T09:59:22.851855+0000","last_active":"2023-10-16T09:59:24.864426+0000","last_peered":"2023-10-16T09:59:24.864426+0000","last_clean":"2023-10-16T09:59:24.864426+0000","last_became_active":"2023-10-16T09:59:22.851516+0000","last_became_peered":"2023-10-16T09:59:22.851516+0000","last_unstale":"2023-10-16T09:59:24.864426+0000","last_undegraded":"2023-10-16T09:59:24.864426+0000","last_fullsized":"2023-10-16T09:59:24.864426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:32.571858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863861+0000","last_change":"2023-10-16T09:59:23.863898+0000","last_active":"2023-10-16T09:59:24.863861+0000","last_peered":"2023-10-16T09:59:24.863861+0000","last_clean":"2023-10-16T09:59:24.863861+0000","last_became_active":"2023-10-16T09:59:23.863644+0000","last_became_peered":"2023-10-16T09:59:23.863644+0000","last_unstale":"2023-10-16T09:59:24.863861+0000","last_undegraded":"2023-10-16T09:59:24.863861+0000","last_fullsized":"2023-10-16T09:59:24.863861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:11.911309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886083+0000","last_change":"2023-10-16T09:59:22.847138+0000","last_active":"2023-10-16T09:59:24.886083+0000","last_peered":"2023-10-16T09:59:24.886083+0000","last_clean":"2023-10-16T09:59:24.886083+0000","last_became_active":"2023-10-16T09:59:22.846821+0000","last_became_peered":"2023-10-16T09:59:22.846821+0000","last_unstale":"2023-10-16T09:59:24.886083+0000","last_undegraded":"2023-10-16T09:59:24.886083+0000","last_fullsized":"2023-10-16T09:59:24.886083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:08.098953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864698+0000","last_change":"2023-10-16T09:59:23.861687+0000","last_active":"2023-10-16T09:59:24.864698+0000","last_peered":"2023-10-16T09:59:24.864698+0000","last_clean":"2023-10-16T09:59:24.864698+0000","last_became_active":"2023-10-16T09:59:23.861394+0000","last_became_peered":"2023-10-16T09:59:23.861394+0000","last_unstale":"2023-10-16T09:59:24.864698+0000","last_undegraded":"2023-10-16T09:59:24.864698+0000","last_fullsized":"2023-10-16T09:59:24.864698+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:48.063641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862103+0000","last_change":"2023-10-16T09:59:22.848603+0000","last_active":"2023-10-16T09:59:24.862103+0000","last_peered":"2023-10-16T09:59:24.862103+0000","last_clean":"2023-10-16T09:59:24.862103+0000","last_became_active":"2023-10-16T09:59:22.848322+0000","last_became_peered":"2023-10-16T09:59:22.848322+0000","last_unstale":"2023-10-16T09:59:24.862103+0000","last_undegraded":"2023-10-16T09:59:24.862103+0000","last_fullsized":"2023-10-16T09:59:24.862103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:03.970575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886473+0000","last_change":"2023-10-16T09:59:23.868919+0000","last_active":"2023-10-16T09:59:24.886473+0000","last_peered":"2023-10-16T09:59:24.886473+0000","last_clean":"2023-10-16T09:59:24.886473+0000","last_became_active":"2023-10-16T09:59:23.868635+0000","last_became_peered":"2023-10-16T09:59:23.868635+0000","last_unstale":"2023-10-16T09:59:24.886473+0000","last_undegraded":"2023-10-16T09:59:24.886473+0000","last_fullsized":"2023-10-16T09:59:24.886473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:22:42.603329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885614+0000","last_change":"2023-10-16T09:59:22.851570+0000","last_active":"2023-10-16T09:59:24.885614+0000","last_peered":"2023-10-16T09:59:24.885614+0000","last_clean":"2023-10-16T09:59:24.885614+0000","last_became_active":"2023-10-16T09:59:22.851300+0000","last_became_peered":"2023-10-16T09:59:22.851300+0000","last_unstale":"2023-10-16T09:59:24.885614+0000","last_undegraded":"2023-10-16T09:59:24.885614+0000","last_fullsized":"2023-10-16T09:59:24.885614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:12.643365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885685+0000","last_change":"2023-10-16T09:59:23.866644+0000","last_active":"2023-10-16T09:59:24.885685+0000","last_peered":"2023-10-16T09:59:24.885685+0000","last_clean":"2023-10-16T09:59:24.885685+0000","last_became_active":"2023-10-16T09:59:23.866265+0000","last_became_peered":"2023-10-16T09:59:23.866265+0000","last_unstale":"2023-10-16T09:59:24.885685+0000","last_undegraded":"2023-10-16T09:59:24.885685+0000","last_fullsized":"2023-10-16T09:59:24.885685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:15:46.837454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862997+0000","last_change":"2023-10-16T09:59:22.854523+0000","last_active":"2023-10-16T09:59:24.862997+0000","last_peered":"2023-10-16T09:59:24.862997+0000","last_clean":"2023-10-16T09:59:24.862997+0000","last_became_active":"2023-10-16T09:59:22.854199+0000","last_became_peered":"2023-10-16T09:59:22.854199+0000","last_unstale":"2023-10-16T09:59:24.862997+0000","last_undegraded":"2023-10-16T09:59:24.862997+0000","last_fullsized":"2023-10-16T09:59:24.862997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:02.867558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864783+0000","last_change":"2023-10-16T09:59:23.864702+0000","last_active":"2023-10-16T09:59:24.864783+0000","last_peered":"2023-10-16T09:59:24.864783+0000","last_clean":"2023-10-16T09:59:24.864783+0000","last_became_active":"2023-10-16T09:59:23.864425+0000","last_became_peered":"2023-10-16T09:59:23.864425+0000","last_unstale":"2023-10-16T09:59:24.864783+0000","last_undegraded":"2023-10-16T09:59:24.864783+0000","last_fullsized":"2023-10-16T09:59:24.864783+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:30.408246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"75'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886244+0000","last_change":"2023-10-16T09:59:22.854529+0000","last_active":"2023-10-16T09:59:24.886244+0000","last_peered":"2023-10-16T09:59:24.886244+0000","last_clean":"2023-10-16T09:59:24.886244+0000","last_became_active":"2023-10-16T09:59:22.854175+0000","last_became_peered":"2023-10-16T09:59:22.854175+0000","last_unstale":"2023-10-16T09:59:24.886244+0000","last_undegraded":"2023-10-16T09:59:24.886244+0000","last_fullsized":"2023-10-16T09:59:24.886244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:40.269111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864810+0000","last_change":"2023-10-16T09:59:23.869981+0000","last_active":"2023-10-16T09:59:24.864810+0000","last_peered":"2023-10-16T09:59:24.864810+0000","last_clean":"2023-10-16T09:59:24.864810+0000","last_became_active":"2023-10-16T09:59:23.869668+0000","last_became_peered":"2023-10-16T09:59:23.869668+0000","last_unstale":"2023-10-16T09:59:24.864810+0000","last_undegraded":"2023-10-16T09:59:24.864810+0000","last_fullsized":"2023-10-16T09:59:24.864810+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:03.852538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886050+0000","last_change":"2023-10-16T09:59:22.848213+0000","last_active":"2023-10-16T09:59:24.886050+0000","last_peered":"2023-10-16T09:59:24.886050+0000","last_clean":"2023-10-16T09:59:24.886050+0000","last_became_active":"2023-10-16T09:59:22.847961+0000","last_became_peered":"2023-10-16T09:59:22.847961+0000","last_unstale":"2023-10-16T09:59:24.886050+0000","last_undegraded":"2023-10-16T09:59:24.886050+0000","last_fullsized":"2023-10-16T09:59:24.886050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:41:45.381126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885793+0000","last_change":"2023-10-16T09:59:23.869688+0000","last_active":"2023-10-16T09:59:24.885793+0000","last_peered":"2023-10-16T09:59:24.885793+0000","last_clean":"2023-10-16T09:59:24.885793+0000","last_became_active":"2023-10-16T09:59:23.869412+0000","last_became_peered":"2023-10-16T09:59:23.869412+0000","last_unstale":"2023-10-16T09:59:24.885793+0000","last_undegraded":"2023-10-16T09:59:24.885793+0000","last_fullsized":"2023-10-16T09:59:24.885793+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:33.361606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886385+0000","last_change":"2023-10-16T09:59:22.849304+0000","last_active":"2023-10-16T09:59:24.886385+0000","last_peered":"2023-10-16T09:59:24.886385+0000","last_clean":"2023-10-16T09:59:24.886385+0000","last_became_active":"2023-10-16T09:59:22.849026+0000","last_became_peered":"2023-10-16T09:59:22.849026+0000","last_unstale":"2023-10-16T09:59:24.886385+0000","last_undegraded":"2023-10-16T09:59:24.886385+0000","last_fullsized":"2023-10-16T09:59:24.886385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:43:49.708315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886234+0000","last_change":"2023-10-16T09:59:23.873967+0000","last_active":"2023-10-16T09:59:24.886234+0000","last_peered":"2023-10-16T09:59:24.886234+0000","last_clean":"2023-10-16T09:59:24.886234+0000","last_became_active":"2023-10-16T09:59:23.873466+0000","last_became_peered":"2023-10-16T09:59:23.873466+0000","last_unstale":"2023-10-16T09:59:24.886234+0000","last_undegraded":"2023-10-16T09:59:24.886234+0000","last_fullsized":"2023-10-16T09:59:24.886234+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:13:54.009125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.864609+0000","last_change":"2023-10-16T09:59:22.854467+0000","last_active":"2023-10-16T09:59:24.864609+0000","last_peered":"2023-10-16T09:59:24.864609+0000","last_clean":"2023-10-16T09:59:24.864609+0000","last_became_active":"2023-10-16T09:59:22.854170+0000","last_became_peered":"2023-10-16T09:59:22.854170+0000","last_unstale":"2023-10-16T09:59:24.864609+0000","last_undegraded":"2023-10-16T09:59:24.864609+0000","last_fullsized":"2023-10-16T09:59:24.864609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:23:46.880312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862177+0000","last_change":"2023-10-16T09:59:23.870338+0000","last_active":"2023-10-16T09:59:24.862177+0000","last_peered":"2023-10-16T09:59:24.862177+0000","last_clean":"2023-10-16T09:59:24.862177+0000","last_became_active":"2023-10-16T09:59:23.870027+0000","last_became_peered":"2023-10-16T09:59:23.870027+0000","last_unstale":"2023-10-16T09:59:24.862177+0000","last_undegraded":"2023-10-16T09:59:24.862177+0000","last_fullsized":"2023-10-16T09:59:24.862177+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:03:45.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885497+0000","last_change":"2023-10-16T09:59:22.852999+0000","last_active":"2023-10-16T09:59:24.885497+0000","last_peered":"2023-10-16T09:59:24.885497+0000","last_clean":"2023-10-16T09:59:24.885497+0000","last_became_active":"2023-10-16T09:59:22.852787+0000","last_became_peered":"2023-10-16T09:59:22.852787+0000","last_unstale":"2023-10-16T09:59:24.885497+0000","last_undegraded":"2023-10-16T09:59:24.885497+0000","last_fullsized":"2023-10-16T09:59:24.885497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:13:40.620177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.863751+0000","last_change":"2023-10-16T09:59:23.868486+0000","last_active":"2023-10-16T09:59:24.863751+0000","last_peered":"2023-10-16T09:59:24.863751+0000","last_clean":"2023-10-16T09:59:24.863751+0000","last_became_active":"2023-10-16T09:59:23.868122+0000","last_became_peered":"2023-10-16T09:59:23.868122+0000","last_unstale":"2023-10-16T09:59:24.863751+0000","last_undegraded":"2023-10-16T09:59:24.863751+0000","last_fullsized":"2023-10-16T09:59:24.863751+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:54.232434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885848+0000","last_change":"2023-10-16T09:59:22.852181+0000","last_active":"2023-10-16T09:59:24.885848+0000","last_peered":"2023-10-16T09:59:24.885848+0000","last_clean":"2023-10-16T09:59:24.885848+0000","last_became_active":"2023-10-16T09:59:22.851951+0000","last_became_peered":"2023-10-16T09:59:22.851951+0000","last_unstale":"2023-10-16T09:59:24.885848+0000","last_undegraded":"2023-10-16T09:59:24.885848+0000","last_fullsized":"2023-10-16T09:59:24.885848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:53:02.501049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885402+0000","last_change":"2023-10-16T09:59:23.870564+0000","last_active":"2023-10-16T09:59:24.885402+0000","last_peered":"2023-10-16T09:59:24.885402+0000","last_clean":"2023-10-16T09:59:24.885402+0000","last_became_active":"2023-10-16T09:59:23.870291+0000","last_became_peered":"2023-10-16T09:59:23.870291+0000","last_unstale":"2023-10-16T09:59:24.885402+0000","last_undegraded":"2023-10-16T09:59:24.885402+0000","last_fullsized":"2023-10-16T09:59:24.885402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:28.637005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.885445+0000","last_change":"2023-10-16T09:59:22.851639+0000","last_active":"2023-10-16T09:59:24.885445+0000","last_peered":"2023-10-16T09:59:24.885445+0000","last_clean":"2023-10-16T09:59:24.885445+0000","last_became_active":"2023-10-16T09:59:22.851306+0000","last_became_peered":"2023-10-16T09:59:22.851306+0000","last_unstale":"2023-10-16T09:59:24.885445+0000","last_undegraded":"2023-10-16T09:59:24.885445+0000","last_fullsized":"2023-10-16T09:59:24.885445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:55:31.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862224+0000","last_change":"2023-10-16T09:59:23.875558+0000","last_active":"2023-10-16T09:59:24.862224+0000","last_peered":"2023-10-16T09:59:24.862224+0000","last_clean":"2023-10-16T09:59:24.862224+0000","last_became_active":"2023-10-16T09:59:23.874984+0000","last_became_peered":"2023-10-16T09:59:23.874984+0000","last_unstale":"2023-10-16T09:59:24.862224+0000","last_undegraded":"2023-10-16T09:59:24.862224+0000","last_fullsized":"2023-10-16T09:59:24.862224+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:12.087522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862014+0000","last_change":"2023-10-16T09:59:22.854092+0000","last_active":"2023-10-16T09:59:24.862014+0000","last_peered":"2023-10-16T09:59:24.862014+0000","last_clean":"2023-10-16T09:59:24.862014+0000","last_became_active":"2023-10-16T09:59:22.853827+0000","last_became_peered":"2023-10-16T09:59:22.853827+0000","last_unstale":"2023-10-16T09:59:24.862014+0000","last_undegraded":"2023-10-16T09:59:24.862014+0000","last_fullsized":"2023-10-16T09:59:24.862014+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:47.257443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.862057+0000","last_change":"2023-10-16T09:59:23.876253+0000","last_active":"2023-10-16T09:59:24.862057+0000","last_peered":"2023-10-16T09:59:24.862057+0000","last_clean":"2023-10-16T09:59:24.862057+0000","last_became_active":"2023-10-16T09:59:23.875886+0000","last_became_peered":"2023-10-16T09:59:23.875886+0000","last_unstale":"2023-10-16T09:59:24.862057+0000","last_undegraded":"2023-10-16T09:59:24.862057+0000","last_fullsized":"2023-10-16T09:59:24.862057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:22.833070+0000","last_clean_scrub_stamp":"2023-10-16T09:59:22.833070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:55.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-16T09:59:24.886095+0000","last_change":"2023-10-16T09:59:22.852932+0000","last_active":"2023-10-16T09:59:24.886095+0000","last_peered":"2023-10-16T09:59:24.886095+0000","last_clean":"2023-10-16T09:59:24.886095+0000","last_became_active":"2023-10-16T09:59:22.852659+0000","last_became_peered":"2023-10-16T09:59:22.852659+0000","last_unstale":"2023-10-16T09:59:24.886095+0000","last_undegraded":"2023-10-16T09:59:24.886095+0000","last_fullsized":"2023-10-16T09:59:24.886095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:21.828810+0000","last_clean_scrub_stamp":"2023-10-16T09:59:21.828810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:28.960195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"65'161","reported_seq":378,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-16T09:59:23.857409+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:59:23.857409+0000","last_peered":"2023-10-16T09:59:23.857409+0000","last_clean":"2023-10-16T09:59:23.857409+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:59:23.857409+0000","last_undegraded":"2023-10-16T09:59:23.857409+0000","last_fullsized":"2023-10-16T09:59:23.857409+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1515520,"data_stored":1028660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.434},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60099999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.25700000000000001},"max":{"1min":0.72899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.80400000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.623}]},{"osd":3,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.81200000000000006},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.34399999999999997},"max":{"1min":0.755,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53700000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76200000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.318,"15min":0.312},"max":{"1min":0.70299999999999996,"5min":1.006,"15min":1.006},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":1.0169999999999999,"15min":1.0489999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68999999999999995,"5min":0.95199999999999996,"15min":1.373},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.85999999999999999,"15min":1.4099999999999999},"last":0.748}]}]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":438132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56899999999999995,"15min":0.61699999999999999},"min":{"1min":0.497,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":6.194},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58599999999999997,"15min":0.61899999999999999},"min":{"1min":0.49399999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":6.1609999999999996},"last":0.47999999999999998}]},{"osd":1,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.47699999999999998,"5min":0.34399999999999997,"15min":0.28799999999999998},"max":{"1min":0.76700000000000002,"5min":0.79500000000000004,"15min":1.054},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78300000000000003,"5min":0.80600000000000005,"15min":0.876},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.64000000000000001},"min":{"1min":0.41599999999999998,"5min":0.34399999999999997,"15min":0.33000000000000002},"max":{"1min":0.79200000000000004,"5min":0.86099999999999999,"15min":8.5600000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76100000000000001,"5min":0.83699999999999997,"15min":1.1060000000000001},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.57799999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.82399999999999995,"15min":1.175},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.64400000000000002},"min":{"1min":0.442,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":6.2249999999999996},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.30299999999999999},"max":{"1min":0.67600000000000005,"5min":0.755,"15min":0.81299999999999994},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.30199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.84699999999999998},"last":0.377}]},{"osd":5,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":0.65700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76000000000000001,"5min":0.78800000000000003,"15min":1.1240000000000001},"last":0.36699999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.32500000000000001,"15min":0.31900000000000001},"max":{"1min":0.77900000000000003,"5min":0.83499999999999996,"15min":0.879},"last":0.501},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.38,"15min":0.35399999999999998},"max":{"1min":0.79500000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.34499999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542331,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":917504,"data_stored":438222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.375,"15min":0.37},"min":{"1min":0.22500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54400000000000004,"5min":1.5,"15min":1.5},"last":0.375},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.245,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.60099999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.23599999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55600000000000005,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.44},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.67300000000000004,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.28699999999999998}]},{"osd":3,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.495,"15min":0.47199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68000000000000005,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.443,"15min":0.436},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.65100000000000002,"5min":1.379,"15min":1.379},"last":0.34100000000000003}]},{"osd":4,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85899999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":1.26,"15min":1.26},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78700000000000003,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69699999999999995,"15min":0.67000000000000004},"min":{"1min":0.44500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92100000000000004,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.505},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.50600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92600000000000005,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.43099999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":1030694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.379,"15min":0.41099999999999998},"min":{"1min":0.26700000000000002,"5min":0.19900000000000001,"15min":0.191},"max":{"1min":0.54900000000000004,"5min":0.65400000000000003,"15min":5.5090000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.40699999999999997,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.23100000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":6.4950000000000001},"last":0.29399999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41399999999999998,"15min":0.40699999999999997},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.69599999999999995},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.42799999999999999,"15min":0.44500000000000001},"min":{"1min":0.29499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59899999999999998,"5min":0.63600000000000001,"15min":2.7559999999999998},"last":0.32600000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.49099999999999999},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.20699999999999999},"max":{"1min":0.58699999999999997,"5min":0.72999999999999998,"15min":8.3260000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48799999999999999,"15min":0.51900000000000002},"min":{"1min":0.40600000000000003,"5min":0.33900000000000002,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":5.4809999999999999},"last":0.309}]},{"osd":4,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.69499999999999995},"min":{"1min":0.47999999999999998,"5min":0.42799999999999999,"15min":0.39600000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":6.4349999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.68999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":8.2970000000000006},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":1.004,"5min":1.004,"15min":5.4160000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66000000000000003,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.41799999999999998,"15min":0.40300000000000002},"max":{"1min":0.83999999999999997,"5min":0.96999999999999997,"15min":6.3869999999999996},"last":0.45900000000000002}]},{"osd":6,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.69599999999999995},"min":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.32900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":8.2189999999999994},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.69699999999999995},"min":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.86899999999999999,"5min":1.0089999999999999,"15min":5.4560000000000004},"last":0.46700000000000003}]},{"osd":7,"last update":"Mon Oct 16 09:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67600000000000005,"15min":0.73699999999999999},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":6.4649999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66800000000000004,"15min":0.71799999999999997},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.34899999999999998},"max":{"1min":0.874,"5min":0.91100000000000003,"15min":8.266},"last":0.60499999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542331,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":921600,"data_stored":438188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.372},"min":{"1min":0.26900000000000002,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.39700000000000002},"min":{"1min":0.317,"5min":0.223,"15min":0.223},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.27600000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.40300000000000002},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.24299999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.64900000000000002},"last":0.377},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.436,"15min":0.41899999999999998},"min":{"1min":0.36499999999999999,"5min":0.28599999999999998,"15min":0.214},"max":{"1min":0.53300000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.315}]},{"osd":3,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41799999999999998,"15min":0.40600000000000003},"min":{"1min":0.30299999999999999,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.54100000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.48499999999999999,"15min":0.46500000000000002},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.496,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80200000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.626},"min":{"1min":0.499,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76900000000000002,"5min":1.091,"15min":1.091},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67000000000000004,"15min":0.63800000000000001},"min":{"1min":0.52400000000000002,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.755,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.54500000000000004,"5min":0.35599999999999998,"15min":0.34999999999999998},"max":{"1min":0.75600000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.53700000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.85099999999999998,"5min":0.95599999999999996,"15min":0.97899999999999998},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.55700000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79400000000000004,"5min":0.98299999999999998,"15min":0.98799999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.54400000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.874,"5min":0.93100000000000005,"15min":0.998},"last":0.58099999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542332,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335596,"statfs":{"total":96636764160,"available":96599650304,"internally_reserved":0,"allocated":921600,"data_stored":438188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.192},"max":{"1min":0.54800000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.41499999999999998},"min":{"1min":0.32800000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":1.0780000000000001},"last":0.26900000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.31,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":2.254},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26500000000000001,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":0.754,"15min":6.2240000000000002},"last":0.29099999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.26400000000000001,"15min":0.25},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":2.2909999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.64000000000000001,"5min":0.71199999999999997,"15min":0.72299999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78600000000000003,"5min":0.88700000000000001,"15min":1.24},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.63700000000000001},"min":{"1min":0.52600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80300000000000005,"5min":1.0820000000000001,"15min":2.4420000000000002},"last":0.314}]},{"osd":5,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.48599999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.97599999999999998,"15min":2.4089999999999998},"last":0.374},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85699999999999998,"5min":0.93999999999999995,"15min":0.996},"last":0.38600000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.5,"5min":0.41299999999999998,"15min":0.39200000000000002},"max":{"1min":0.80200000000000005,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.499,"5min":0.39900000000000002,"15min":0.379},"max":{"1min":0.84499999999999997,"5min":0.99099999999999999,"15min":1.1619999999999999},"last":0.41499999999999998}]},{"osd":7,"last update":"Mon Oct 16 09:58:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.64500000000000002},"min":{"1min":0.51200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.81599999999999995,"5min":1.3,"15min":1.3},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.51500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79100000000000004,"5min":1.4019999999999999,"15min":2.3740000000000001},"last":0.34000000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542331,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54280,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317560,"statfs":{"total":96636764160,"available":96581181440,"internally_reserved":0,"allocated":913408,"data_stored":433020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.38,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.76900000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.248,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.75600000000000001,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.84599999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.437,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.70499999999999996,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.60699999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.46700000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71399999999999997,"5min":0.81999999999999995,"15min":0.89100000000000001},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.44900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.83199999999999996,"5min":0.83299999999999996,"15min":0.86499999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.48599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.81000000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33500000000000002,"15min":0.32500000000000001},"max":{"1min":0.73399999999999999,"5min":0.80400000000000005,"15min":0.86899999999999999},"last":0.50900000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.32800000000000001,"15min":0.32200000000000001},"max":{"1min":0.72399999999999998,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.80100000000000005,"5min":0.85899999999999999,"15min":0.85999999999999999},"last":0.46300000000000002}]},{"osd":7,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78600000000000003,"5min":0.80700000000000005,"15min":0.84599999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42399999999999999,"5min":0.41099999999999998,"15min":0.36099999999999999},"max":{"1min":0.78300000000000003,"5min":0.84599999999999997,"15min":0.95999999999999996},"last":0.433}]}]},{"osd":5,"up_from":14,"seq":60129542332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":438784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.41399999999999998,"5min":0.33100000000000002,"15min":0.253},"max":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.92900000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.65700000000000003},"min":{"1min":0.48999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":8.0370000000000008},"last":0.45200000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":0.753,"5min":0.77500000000000002,"15min":0.98599999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59999999999999998,"15min":0.65200000000000002},"min":{"1min":0.46899999999999997,"5min":0.38,"15min":0.307},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":8.2110000000000003},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.64100000000000001},"min":{"1min":0.46700000000000003,"5min":0.315,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":8.1140000000000008},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59399999999999997,"15min":0.64500000000000002},"min":{"1min":0.45300000000000001,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.80200000000000005,"5min":0.88700000000000001,"15min":8.3000000000000007},"last":0.438}]},{"osd":3,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.40799999999999997,"5min":0.34599999999999997,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":1.03},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.41299999999999998,"15min":0.33200000000000002},"max":{"1min":0.874,"5min":0.874,"15min":9.8810000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48599999999999999,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.26000000000000001},"max":{"1min":0.70799999999999996,"5min":0.77400000000000002,"15min":1.0449999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.53200000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.82599999999999996,"5min":0.86799999999999999,"15min":1.0780000000000001},"last":0.54800000000000004}]},{"osd":6,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":1.599},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":1.006},"last":0.52400000000000002}]},{"osd":7,"last update":"Mon Oct 16 09:58:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77400000000000002,"5min":0.95199999999999996,"15min":1.1339999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.889},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T09:59:28.351 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T09:59:28.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T09:59:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f997d228700 1 Processor -- start 2023-10-16T09:59:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f997d228700 1 -- start start 2023-10-16T09:59:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f997d228700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99780b2010 0x7f99780cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f997d228700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9978071be0 0x7f99780b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f997d228700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99780ce1f0 con 0x7f997806f3b0 2023-10-16T09:59:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f997d228700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9978072090 con 0x7f99780b2010 2023-10-16T09:59:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f997d228700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99780b2930 con 0x7f9978071be0 2023-10-16T09:59:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.511+0000 7f9976d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99780b2010 0x7f99780cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9976d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99780b2010 0x7f99780cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41788/0 (socket says 172.21.15.181:41788) 2023-10-16T09:59:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9976d9d700 1 -- 172.21.15.181:0/2905247344 learned_addr learned my addr 172.21.15.181:0/2905247344 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9976d9d700 1 -- 172.21.15.181:0/2905247344 >> v2:172.21.15.190:3300/0 conn(0x7f9978071be0 msgr2=0x7f99780b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9976d9d700 1 --2- 172.21.15.181:0/2905247344 >> v2:172.21.15.190:3300/0 conn(0x7f9978071be0 0x7f99780b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9976d9d700 1 -- 172.21.15.181:0/2905247344 >> v1:172.21.15.190:6789/0 conn(0x7f997806f3b0 legacy=0x7f997806f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9976d9d700 1 -- 172.21.15.181:0/2905247344 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780ccda0 con 0x7f99780b2010 2023-10-16T09:59:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9976d9d700 1 --2- 172.21.15.181:0/2905247344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780cc6e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f997815e1c0 tx=0x7f996c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc484a196ad9af7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9975d9b700 1 -- 172.21.15.181:0/2905247344 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00dcf0 con 0x7f99780b2010 2023-10-16T09:59:28.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9975d9b700 1 -- 172.21.15.181:0/2905247344 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00de50 con 0x7f99780b2010 2023-10-16T09:59:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.512+0000 7f9975d9b700 1 -- 172.21.15.181:0/2905247344 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c004250 con 0x7f99780b2010 2023-10-16T09:59:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 -- 172.21.15.181:0/2905247344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 msgr2=0x7f99780cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 --2- 172.21.15.181:0/2905247344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780cc6e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f997815e1c0 tx=0x7f996c00be70 comp rx=0 tx=0).stop 2023-10-16T09:59:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 -- 172.21.15.181:0/2905247344 shutdown_connections 2023-10-16T09:59:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 --2- 172.21.15.181:0/2905247344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780cc6e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 --2- 172.21.15.181:0/2905247344 >> v2:172.21.15.190:3300/0 conn(0x7f9978071be0 0x7f99780b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 -- 172.21.15.181:0/2905247344 >> 172.21.15.181:0/2905247344 conn(0x7f99780cd010 msgr2=0x7f99780cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 -- 172.21.15.181:0/2905247344 shutdown_connections 2023-10-16T09:59:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.513+0000 7f997d228700 1 -- 172.21.15.181:0/2905247344 wait complete. 2023-10-16T09:59:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.514+0000 7f997d228700 1 Processor -- start 2023-10-16T09:59:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.514+0000 7f997d228700 1 -- start start 2023-10-16T09:59:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997d228700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f997806f3b0 0x7f9978152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997d228700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9978071be0 0x7f99780ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997d228700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997d228700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c9b40 con 0x7f99780b2010 2023-10-16T09:59:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997d228700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780acd90 con 0x7f9978071be0 2023-10-16T09:59:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997d228700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9978153120 con 0x7f997806f3b0 2023-10-16T09:59:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41798/0 (socket says 172.21.15.181:41798) 2023-10-16T09:59:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 -- 172.21.15.181:0/2426972801 learned_addr learned my addr 172.21.15.181:0/2426972801 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997659c700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9978071be0 0x7f99780ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f9976d9d700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f997806f3b0 0x7f9978152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 -- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f997806f3b0 msgr2=0x7f9978152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f997806f3b0 0x7f9978152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 -- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9978071be0 msgr2=0x7f99780ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9978071be0 0x7f99780ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f997759e700 1 -- 172.21.15.181:0/2426972801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c00b7f0 con 0x7f99780b2010 2023-10-16T09:59:28.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.515+0000 7f9976d9d700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f997806f3b0 0x7f9978152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:28.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.516+0000 7f997759e700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780ad220 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f997000b970 tx=0x7f997000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.516+0000 7f995f7fe700 1 -- 172.21.15.181:0/2426972801 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000d8e0 con 0x7f99780b2010 2023-10-16T09:59:28.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.516+0000 7f995f7fe700 1 -- 172.21.15.181:0/2426972801 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997000da80 con 0x7f99780b2010 2023-10-16T09:59:28.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.516+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99780aec90 con 0x7f99780b2010 2023-10-16T09:59:28.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.516+0000 7f995f7fe700 1 -- 172.21.15.181:0/2426972801 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000de30 con 0x7f99780b2010 2023-10-16T09:59:28.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.516+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99780aca20 con 0x7f99780b2010 2023-10-16T09:59:28.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.517+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99780af1b0 con 0x7f99780b2010 2023-10-16T09:59:28.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.517+0000 7f995f7fe700 1 -- 172.21.15.181:0/2426972801 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9970015420 con 0x7f99780b2010 2023-10-16T09:59:28.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.517+0000 7f995f7fe700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f996406b020 0x7f996406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.518+0000 7f995f7fe700 1 -- 172.21.15.181:0/2426972801 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f9970011070 con 0x7f99780b2010 2023-10-16T09:59:28.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.518+0000 7f9976d9d700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f996406b020 0x7f996406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.518+0000 7f9976d9d700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f996406b020 0x7f996406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f996c000c00 tx=0x7f996c012000 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.522+0000 7f995f7fe700 1 -- 172.21.15.181:0/2426972801 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f997004efe0 con 0x7f99780b2010 2023-10-16T09:59:28.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.792+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f99780ac880 con 0x7f99780b2010 2023-10-16T09:59:28.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.794+0000 7f995f7fe700 1 -- 172.21.15.181:0/2426972801 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f99780ac880 con 0x7f99780b2010 2023-10-16T09:59:28.796 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T09:59:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T09:59:28.797 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T09:59:28.797 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T09:59:28.797 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T09:59:28.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f996406b020 msgr2=0x7f996406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f996406b020 0x7f996406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f996c000c00 tx=0x7f996c012000 comp rx=0 tx=0).stop 2023-10-16T09:59:28.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 msgr2=0x7f99780ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780ad220 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f997000b970 tx=0x7f997000bc80 comp rx=0 tx=0).stop 2023-10-16T09:59:28.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 shutdown_connections 2023-10-16T09:59:28.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f996406b020 0x7f996406d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f997806f3b0 0x7f9978152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99780b2010 0x7f99780ad220 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 --2- 172.21.15.181:0/2426972801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9978071be0 0x7f99780ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 >> 172.21.15.181:0/2426972801 conn(0x7f99780cd010 msgr2=0x7f997806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:28.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 shutdown_connections 2023-10-16T09:59:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.797+0000 7f997d228700 1 -- 172.21.15.181:0/2426972801 wait complete. 2023-10-16T09:59:28.801 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T09:59:28.810 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:28.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.929+0000 7f8b8fea6700 1 Processor -- start 2023-10-16T09:59:28.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.930+0000 7f8b8fea6700 1 -- start start 2023-10-16T09:59:28.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.930+0000 7f8b8fea6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b8815dfd0 0x7f8b880af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.930+0000 7f8b8fea6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 0x7f8b880cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.930+0000 7f8b8fea6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8806f870 con 0x7f8b880af450 2023-10-16T09:59:28.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.930+0000 7f8b8fea6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b880cde40 con 0x7f8b8815dfd0 2023-10-16T09:59:28.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.930+0000 7f8b8fea6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880d1160 con 0x7f8b880cd470 2023-10-16T09:59:28.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8d441700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8b880af450 0x7f8b8806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52882/0 (socket says 172.21.15.181:52882) 2023-10-16T09:59:28.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8d441700 1 -- 172.21.15.181:0/1439939781 learned_addr learned my addr 172.21.15.181:0/1439939781 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:28.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8dc42700 1 --2- 172.21.15.181:0/1439939781 >> v2:172.21.15.190:3300/0 conn(0x7f8b8815dfd0 0x7f8b880af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8e443700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 0x7f8b880cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871531817 0 0) 0x7f8b8806f870 con 0x7f8b880af450 2023-10-16T09:59:28.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b740035b0 con 0x7f8b880af450 2023-10-16T09:59:28.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1555070820 0 0) 0x7f8b740035b0 con 0x7f8b880af450 2023-10-16T09:59:28.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 msgr2=0x7f8b880cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 0x7f8b880cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 >> v2:172.21.15.190:3300/0 conn(0x7f8b8815dfd0 msgr2=0x7f8b880af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 --2- 172.21.15.181:0/1439939781 >> v2:172.21.15.190:3300/0 conn(0x7f8b8815dfd0 0x7f8b880af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b880cc8e0 con 0x7f8b880af450 2023-10-16T09:59:28.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f8b84001670 con 0x7f8b880af450 2023-10-16T09:59:28.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 >> v1:172.21.15.181:6789/0 conn(0x7f8b880af450 legacy=0x7f8b8806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 0x7f8b74003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b74003f80 0x7f8b74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b74004d90 0x7f8b74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880cde40 con 0x7f8b74004d90 2023-10-16T09:59:28.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880d1160 con 0x7f8b74003f80 2023-10-16T09:59:28.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7400ba60 con 0x7f8b880cd470 2023-10-16T09:59:28.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8e443700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b74004d90 0x7f8b74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8dc42700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 0x7f8b74003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.931+0000 7f8b8d441700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b74003f80 0x7f8b74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8e443700 1 -- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 msgr2=0x7f8b74003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8e443700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880cd470 0x7f8b74003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8e443700 1 -- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b74003f80 msgr2=0x7f8b74004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:28.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8e443700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b74003f80 0x7f8b74004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8e443700 1 -- 172.21.15.181:0/1439939781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740035b0 con 0x7f8b74004d90 2023-10-16T09:59:28.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8e443700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b74004d90 0x7f8b74005210 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000e170 tx=0x7f8b74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78a727c469e4b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000ec90 con 0x7f8b74004d90 2023-10-16T09:59:28.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8000f040 con 0x7f8b74004d90 2023-10-16T09:59:28.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8cc40700 1 -- 172.21.15.181:0/1439939781 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80014480 con 0x7f8b74004d90 2023-10-16T09:59:28.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8fea6700 1 -- 172.21.15.181:0/1439939781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b74004d90 msgr2=0x7f8b74005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8fea6700 1 --2- 172.21.15.181:0/1439939781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b74004d90 0x7f8b74005210 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000e170 tx=0x7f8b74005880 comp rx=0 tx=0).stop 2023-10-16T09:59:28.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8e443700 1 -- 172.21.15.181:0/1439939781 reap_dead start 2023-10-16T09:59:28.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8fea6700 1 -- 172.21.15.181:0/1439939781 shutdown_connections 2023-10-16T09:59:28.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8fea6700 1 -- 172.21.15.181:0/1439939781 >> 172.21.15.181:0/1439939781 conn(0x7f8b880d04a0 msgr2=0x7f8b880d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:28.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.932+0000 7f8b8fea6700 1 -- 172.21.15.181:0/1439939781 shutdown_connections 2023-10-16T09:59:28.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.933+0000 7f8b8fea6700 1 -- 172.21.15.181:0/1439939781 wait complete. 2023-10-16T09:59:28.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 Processor -- start 2023-10-16T09:59:28.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 -- start start 2023-10-16T09:59:28.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b88159460 0x7f8b880b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b880b2b40 0x7f8b8806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880b4420 0x7f8b8806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b881f38e0 con 0x7f8b88159460 2023-10-16T09:59:28.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b881f3bc0 con 0x7f8b880b2b40 2023-10-16T09:59:28.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.934+0000 7f8b8fea6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b881f3ea0 con 0x7f8b880b4420 2023-10-16T09:59:28.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b88159460 0x7f8b880b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b88159460 0x7f8b880b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41814/0 (socket says 172.21.15.181:41814) 2023-10-16T09:59:28.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 -- 172.21.15.181:0/3549232008 learned_addr learned my addr 172.21.15.181:0/3549232008 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:28.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8d441700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b880b2b40 0x7f8b8806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8e443700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880b4420 0x7f8b8806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 -- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880b4420 msgr2=0x7f8b8806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880b4420 0x7f8b8806ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 -- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b880b2b40 msgr2=0x7f8b8806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:28.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b880b2b40 0x7f8b8806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:28.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 -- 172.21.15.181:0/3549232008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b881f4180 con 0x7f8b88159460 2023-10-16T09:59:28.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8dc42700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b88159460 0x7f8b880b2600 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8b78011a70 tx=0x7f8b78011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b7e7fc700 1 -- 172.21.15.181:0/3549232008 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78005a20 con 0x7f8b88159460 2023-10-16T09:59:28.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b7e7fc700 1 -- 172.21.15.181:0/3549232008 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78005b80 con 0x7f8b88159460 2023-10-16T09:59:28.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b881f4460 con 0x7f8b88159460 2023-10-16T09:59:28.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b7e7fc700 1 -- 172.21.15.181:0/3549232008 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800fa00 con 0x7f8b88159460 2023-10-16T09:59:28.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.935+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b880c41a0 con 0x7f8b88159460 2023-10-16T09:59:28.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.936+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b88067d20 con 0x7f8b88159460 2023-10-16T09:59:28.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.936+0000 7f8b7e7fc700 1 -- 172.21.15.181:0/3549232008 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8b7800fb60 con 0x7f8b88159460 2023-10-16T09:59:28.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.937+0000 7f8b7e7fc700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f8b6c06b020 0x7f8b6c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:28.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.937+0000 7f8b7e7fc700 1 -- 172.21.15.181:0/3549232008 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f8b78089e20 con 0x7f8b88159460 2023-10-16T09:59:28.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.937+0000 7f8b8d441700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f8b6c06b020 0x7f8b6c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:28.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.937+0000 7f8b8d441700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f8b6c06b020 0x7f8b6c06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b8806f8d0 tx=0x7f8b84009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:28.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:28.941+0000 7f8b7e7fc700 1 -- 172.21.15.181:0/3549232008 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8b78054c10 con 0x7f8b88159460 2023-10-16T09:59:29.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.176+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b880ca8d0 con 0x7f8b88159460 2023-10-16T09:59:29.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.176+0000 7f8b7e7fc700 1 -- 172.21.15.181:0/3549232008 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f8b78022070 con 0x7f8b88159460 2023-10-16T09:59:29.179 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:29.179 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2852689416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2852689416},{"type":"v1","addr":"172.21.15.190:6834","nonce":2852689416}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2993219349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2993219349},{"type":"v1","addr":"172.21.15.181:6836","nonce":2993219349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2504788411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2504788411},{"type":"v1","addr":"172.21.15.181:6837","nonce":2504788411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:23.239300+0000","modified":"2023-10-16T09:59:26.271260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1948728404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1948728404},{"type":"v1","addr":"172.21.15.190:6835","nonce":1948728404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T09:59:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.178+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f8b6c06b020 msgr2=0x7f8b6c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.178+0000 7f8b8fea6700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f8b6c06b020 0x7f8b6c06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b8806f8d0 tx=0x7f8b84009040 comp rx=0 tx=0).stop 2023-10-16T09:59:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.178+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b88159460 msgr2=0x7f8b880b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.178+0000 7f8b8fea6700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b88159460 0x7f8b880b2600 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8b78011a70 tx=0x7f8b78011e30 comp rx=0 tx=0).stop 2023-10-16T09:59:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 shutdown_connections 2023-10-16T09:59:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f8b6c06b020 0x7f8b6c06d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b880b4420 0x7f8b8806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b88159460 0x7f8b880b2600 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 --2- 172.21.15.181:0/3549232008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b880b2b40 0x7f8b8806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 >> 172.21.15.181:0/3549232008 conn(0x7f8b880d04a0 msgr2=0x7f8b8815cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 shutdown_connections 2023-10-16T09:59:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.179+0000 7f8b8fea6700 1 -- 172.21.15.181:0/3549232008 wait complete. 2023-10-16T09:59:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 32 2023-10-16T09:59:29.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T09:59:23.239300+0000', 'modified': '2023-10-16T09:59:26.271260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6474}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6474': {'gid': 6474, 'name': 'd', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/1948728404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 1948728404}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1948728404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T09:59:29.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T09:59:29.595 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-16T09:59:29.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.717+0000 7f9e51ec5700 1 Processor -- start 2023-10-16T09:59:29.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.717+0000 7f9e51ec5700 1 -- start start 2023-10-16T09:59:29.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e51ec5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e4c06d210 0x7f9e4c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e51ec5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e4c0ce4d0 0x7f9e4c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e51ec5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c0cf460 con 0x7f9e4c0cc4c0 2023-10-16T09:59:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e51ec5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c0d0380 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e51ec5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c0cc940 con 0x7f9e4c06d210 2023-10-16T09:59:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e4bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e4c0ce4d0 0x7f9e4c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e4bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e4c0ce4d0 0x7f9e4c15d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41826/0 (socket says 172.21.15.181:41826) 2023-10-16T09:59:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.718+0000 7f9e4bfff700 1 -- 172.21.15.181:0/4084020700 learned_addr learned my addr 172.21.15.181:0/4084020700 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/4084020700 >> v2:172.21.15.190:3300/0 conn(0x7f9e4c06d210 0x7f9e4c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4bfff700 1 -- 172.21.15.181:0/4084020700 >> v2:172.21.15.190:3300/0 conn(0x7f9e4c06d210 msgr2=0x7f9e4c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/4084020700 >> v2:172.21.15.190:3300/0 conn(0x7f9e4c06d210 0x7f9e4c0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4bfff700 1 -- 172.21.15.181:0/4084020700 >> v1:172.21.15.190:6789/0 conn(0x7f9e4c0cc4c0 legacy=0x7f9e4c0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4bfff700 1 -- 172.21.15.181:0/4084020700 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c06c700 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/4084020700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c15d2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c06ed60 tx=0x7f9e4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04d5af90bec7902 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4a7fc700 1 -- 172.21.15.181:0/4084020700 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400e070 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4a7fc700 1 -- 172.21.15.181:0/4084020700 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e44003a20 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e4a7fc700 1 -- 172.21.15.181:0/4084020700 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44003dd0 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e51ec5700 1 -- 172.21.15.181:0/4084020700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 msgr2=0x7f9e4c15d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/4084020700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c15d2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c06ed60 tx=0x7f9e4400b330 comp rx=0 tx=0).stop 2023-10-16T09:59:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e51ec5700 1 -- 172.21.15.181:0/4084020700 shutdown_connections 2023-10-16T09:59:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/4084020700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c15d2f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/4084020700 >> v2:172.21.15.190:3300/0 conn(0x7f9e4c06d210 0x7f9e4c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e51ec5700 1 -- 172.21.15.181:0/4084020700 >> 172.21.15.181:0/4084020700 conn(0x7f9e4c09a9a0 msgr2=0x7f9e4c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.719+0000 7f9e51ec5700 1 -- 172.21.15.181:0/4084020700 shutdown_connections 2023-10-16T09:59:29.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.720+0000 7f9e51ec5700 1 -- 172.21.15.181:0/4084020700 wait complete. 2023-10-16T09:59:29.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.721+0000 7f9e51ec5700 1 Processor -- start 2023-10-16T09:59:29.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.721+0000 7f9e51ec5700 1 -- start start 2023-10-16T09:59:29.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c06d210 0x7f9e4c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c0cc4c0 0x7f9e4c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c1f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0c5e50 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0c6130 con 0x7f9e4c06d210 2023-10-16T09:59:29.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0b4e80 con 0x7f9e4c0cc4c0 2023-10-16T09:59:29.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c1f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c1f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41832/0 (socket says 172.21.15.181:41832) 2023-10-16T09:59:29.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 -- 172.21.15.181:0/3452872761 learned_addr learned my addr 172.21.15.181:0/3452872761 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:29.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c06d210 0x7f9e4c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4affd700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c0cc4c0 0x7f9e4c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 -- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c0cc4c0 msgr2=0x7f9e4c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c0cc4c0 0x7f9e4c0b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 -- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c06d210 msgr2=0x7f9e4c15d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c06d210 0x7f9e4c15d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 -- 172.21.15.181:0/3452872761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44009b30 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c06d210 0x7f9e4c15d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4affd700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c0cc4c0 0x7f9e4c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c1f3300 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9e44015040 tx=0x7f9e44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e33fff700 1 -- 172.21.15.181:0/3452872761 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44004240 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e33fff700 1 -- 172.21.15.181:0/3452872761 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e440043a0 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e33fff700 1 -- 172.21.15.181:0/3452872761 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44004750 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c0b5160 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.722+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c15d6b0 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.724+0000 7f9e33fff700 1 -- 172.21.15.181:0/3452872761 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9e44019070 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.724+0000 7f9e33fff700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9e3806b020 0x7f9e3806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.724+0000 7f9e33fff700 1 -- 172.21.15.181:0/3452872761 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f9e4408a9c0 con 0x7f9e4c0ce4d0 2023-10-16T09:59:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.724+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9e3806b020 0x7f9e3806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.725+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9e3806b020 0x7f9e3806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006fd0 tx=0x7f9e40008040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.726+0000 7f9e5e0afb80 1 Processor -- start 2023-10-16T09:59:29.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.726+0000 7f9e5e0afb80 1 -- start start 2023-10-16T09:59:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933582a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e933582fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e9335566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9334ccc20 con 0x55e933580630 2023-10-16T09:59:29.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9334b1c10 con 0x55e933556010 2023-10-16T09:59:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9334af530 con 0x55e93357b220 2023-10-16T09:59:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e933582fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e933582fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41838/0 (socket says 172.21.15.181:41838) 2023-10-16T09:59:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 -- 172.21.15.181:0/709832266 learned_addr learned my addr 172.21.15.181:0/709832266 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:29.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933582a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e9335566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 -- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 msgr2=0x55e933582a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933582a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 -- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 msgr2=0x55e9335566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e9335566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 -- 172.21.15.181:0/709832266 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9334683a0 con 0x55e933580630 2023-10-16T09:59:29.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.727+0000 7f9e4affd700 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e933582fb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00a9c0 tx=0x7f9e3c00acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323e8ac2071618ea server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e317fa700 1 -- 172.21.15.181:0/709832266 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00ca90 con 0x55e933580630 2023-10-16T09:59:29.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e317fa700 1 -- 172.21.15.181:0/709832266 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c00cbf0 con 0x55e933580630 2023-10-16T09:59:29.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e317fa700 1 -- 172.21.15.181:0/709832266 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0113f0 con 0x55e933580630 2023-10-16T09:59:29.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/709832266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 msgr2=0x55e933582fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e933582fb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00a9c0 tx=0x7f9e3c00acd0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/709832266 shutdown_connections 2023-10-16T09:59:29.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933582a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e9335566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/709832266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e933582fb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/709832266 >> 172.21.15.181:0/709832266 conn(0x55e933485d80 msgr2=0x55e9335a0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/709832266 shutdown_connections 2023-10-16T09:59:29.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.728+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/709832266 wait complete. 2023-10-16T09:59:29.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.729+0000 7f9e5e0afb80 1 -- start start 2023-10-16T09:59:29.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.729+0000 7f9e5e0afb80 1 Processor -- start 2023-10-16T09:59:29.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e933575a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e93357b220 0x55e93357a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e933580630 0x55e9335b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9334aae20 con 0x55e933556010 2023-10-16T09:59:29.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9334476c0 con 0x55e93357b220 2023-10-16T09:59:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e933447360 con 0x55e933580630 2023-10-16T09:59:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 10 client.0 _ll_register_callbacks cb 0x55e9332a47b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:59:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e933575a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e933575a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41842/0 (socket says 172.21.15.181:41842) 2023-10-16T09:59:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/499271363 learned_addr learned my addr 172.21.15.181:0/499271363 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4affd700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e93357b220 0x55e93357a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e933580630 0x55e9335b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e933580630 msgr2=0x55e9335b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e933580630 0x55e9335b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e93357b220 msgr2=0x55e93357a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e93357b220 0x55e93357a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/499271363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c00a5f0 con 0x55e933556010 2023-10-16T09:59:29.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e933580630 0x55e9335b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:29.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e933575a50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9e400136a0 tx=0x7f9e400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccfc8d4c7d88ff8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e2a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:59:29.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/499271363 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000b050 con 0x55e933556010 2023-10-16T09:59:29.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/499271363 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f9e40012e10 con 0x55e933556010 2023-10-16T09:59:29.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.730+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e93354b5b0 con 0x55e933556010 2023-10-16T09:59:29.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/499271363 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e40012560 con 0x55e933556010 2023-10-16T09:59:29.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 10 client.6612 fetch_fsmap learned FSMap version 32 2023-10-16T09:59:29.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e9332d5d90 con 0x55e933556010 2023-10-16T09:59:29.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/499271363 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e93354b5b0 con 0x55e933556010 2023-10-16T09:59:29.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/499271363 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e4001f830 con 0x55e933556010 2023-10-16T09:59:29.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 10 client.6612 fetch_fsmap finished waiting for FSMap version 32 2023-10-16T09:59:29.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 10 client.6612 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T09:59:29.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 20 client.6612 populate_metadata read hostname 'smithi181' 2023-10-16T09:59:29.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e9335839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 4 client.6612 mds_command: new command op to 6474 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T09:59:29.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 --> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9334a3e00 con 0x55e933585b70 2023-10-16T09:59:29.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.731+0000 7f9e4affd700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e9335839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e2a7fc700 10 client.6612 ms_handle_connect on v2:172.21.15.190:6833/1948728404 2023-10-16T09:59:29.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e4affd700 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e9335839c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e298d57b356f611f server_cookie=c254a5aafd41bc41 in_seq=0 out_seq=0 2023-10-16T09:59:29.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/499271363 <== mds.0 v2:172.21.15.190:6833/1948728404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9334a3e00 con 0x55e933585b70 2023-10-16T09:59:29.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e2a7fc700 10 client.6612 handle_command_reply: tid=0 2023-10-16T09:59:29.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 client.6612 shutdown 2023-10-16T09:59:29.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 msgr2=0x55e933575a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e933575a50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9e400136a0 tx=0x7f9e400047c0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 shutdown_connections 2023-10-16T09:59:29.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e9335839c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e933580630 0x55e9335b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e2a7fc700 0 client.6612 ms_handle_reset on v2:172.21.15.190:6833/1948728404 2023-10-16T09:59:29.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e933575a50 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/499271363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e93357b220 0x55e93357a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 >> 172.21.15.181:0/499271363 conn(0x55e933485d80 msgr2=0x55e933556450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 shutdown_connections 2023-10-16T09:59:29.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.732+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/499271363 wait complete. 2023-10-16T09:59:29.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.733+0000 7f9e5e0afb80 20 client.6612 trim_cache size 0 max 16384 2023-10-16T09:59:29.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.750+0000 7f9e5e0afb80 1 Processor -- start 2023-10-16T09:59:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.750+0000 7f9e5e0afb80 1 -- start start 2023-10-16T09:59:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e93357d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e93357d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933580630 0x55e93357c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93332b8d0 con 0x55e933556010 2023-10-16T09:59:29.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e933584cb0 con 0x55e933580630 2023-10-16T09:59:29.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9335a17f0 con 0x55e93357b220 2023-10-16T09:59:29.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e93357d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e93357d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41850/0 (socket says 172.21.15.181:41850) 2023-10-16T09:59:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/3320686554 learned_addr learned my addr 172.21.15.181:0/3320686554 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933580630 0x55e93357c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4affd700 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e93357d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 msgr2=0x55e93357d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e93357d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933580630 msgr2=0x55e93357c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933580630 0x55e93357c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 -- 172.21.15.181:0/3320686554 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93354d210 con 0x55e933556010 2023-10-16T09:59:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933580630 0x55e93357c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.751+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e93357d3d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9e40013670 tx=0x7f9e4001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b454b0bbf6ee1082 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e31ffb700 1 -- 172.21.15.181:0/3320686554 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000b040 con 0x55e933556010 2023-10-16T09:59:29.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e31ffb700 1 -- 172.21.15.181:0/3320686554 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000c040 con 0x55e933556010 2023-10-16T09:59:29.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e31ffb700 1 -- 172.21.15.181:0/3320686554 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40013d40 con 0x55e933556010 2023-10-16T09:59:29.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/3320686554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 msgr2=0x55e93357d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e93357d3d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9e40013670 tx=0x7f9e4001e000 comp rx=0 tx=0).stop 2023-10-16T09:59:29.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/3320686554 shutdown_connections 2023-10-16T09:59:29.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933580630 0x55e93357c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933556010 0x55e93357d3d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/3320686554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e93357d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/3320686554 >> 172.21.15.181:0/3320686554 conn(0x55e933485d80 msgr2=0x55e9335bf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/3320686554 shutdown_connections 2023-10-16T09:59:29.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.752+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/3320686554 wait complete. 2023-10-16T09:59:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.753+0000 7f9e5e0afb80 1 -- start start 2023-10-16T09:59:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.753+0000 7f9e5e0afb80 1 Processor -- start 2023-10-16T09:59:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e93357b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933579020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e5e0afb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e93357d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e933578c00 con 0x55e933580630 2023-10-16T09:59:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9336057b0 con 0x55e933556010 2023-10-16T09:59:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e5e0afb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e93357bdf0 con 0x55e93357b220 2023-10-16T09:59:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e5e0afb80 10 client.0 _ll_register_callbacks cb 0x7f9e40012a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T09:59:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e93357d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e93357d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41860/0 (socket says 172.21.15.181:41860) 2023-10-16T09:59:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 -- 172.21.15.181:0/1492087809 learned_addr learned my addr 172.21.15.181:0/1492087809 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 -- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 msgr2=0x55e933579020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4affd700 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933579020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e93357b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933579020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 -- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 msgr2=0x55e93357b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e93357b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 -- 172.21.15.181:0/1492087809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9332d5d90 con 0x55e933580630 2023-10-16T09:59:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e4bfff700 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e93357d3d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b370 tx=0x7f9e4405d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4efa852d071bd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.754+0000 7f9e2a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T09:59:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/1492087809 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440595d0 con 0x55e933580630 2023-10-16T09:59:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/1492087809 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f9e44059730 con 0x55e933580630 2023-10-16T09:59:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e3c00cbf0 con 0x55e933580630 2023-10-16T09:59:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/1492087809 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e44058860 con 0x55e933580630 2023-10-16T09:59:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 10 client.6618 fetch_fsmap learned FSMap version 32 2023-10-16T09:59:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e9332bec10 con 0x55e933580630 2023-10-16T09:59:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/1492087809 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c00cbf0 con 0x55e933580630 2023-10-16T09:59:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/1492087809 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e440611e0 con 0x55e933580630 2023-10-16T09:59:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 10 client.6618 fetch_fsmap finished waiting for FSMap version 32 2023-10-16T09:59:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 10 client.6618 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T09:59:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 20 client.6618 populate_metadata read hostname 'smithi181' 2023-10-16T09:59:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e933604720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 4 client.6618 mds_command: new command op to 6474 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T09:59:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 --> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e9334a3b20 con 0x55e933585b70 2023-10-16T09:59:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.755+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e933604720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e2a7fc700 10 client.6618 ms_handle_connect on v2:172.21.15.190:6833/1948728404 2023-10-16T09:59:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e4b7fe700 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e933604720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240d1be821127157 server_cookie=90a4c0d21e4d929d in_seq=0 out_seq=0 2023-10-16T09:59:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e2a7fc700 1 -- 172.21.15.181:0/1492087809 <== mds.0 v2:172.21.15.190:6833/1948728404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55e9334a3b20 con 0x55e933585b70 2023-10-16T09:59:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e2a7fc700 10 client.6618 handle_command_reply: tid=0 2023-10-16T09:59:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 client.6618 shutdown 2023-10-16T09:59:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 msgr2=0x55e93357d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e93357d3d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b370 tx=0x7f9e4405d570 comp rx=0 tx=0).stop 2023-10-16T09:59:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 shutdown_connections 2023-10-16T09:59:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e933580630 0x55e93357d3d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e933556010 0x55e93357b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:6833/1948728404,v1:172.21.15.190:6835/1948728404] conn(0x55e933585b70 0x55e933604720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 --2- 172.21.15.181:0/1492087809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e93357b220 0x55e933579020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e2a7fc700 0 client.6618 ms_handle_reset on v2:172.21.15.190:6833/1948728404 2023-10-16T09:59:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 >> 172.21.15.181:0/1492087809 conn(0x55e933485d80 msgr2=0x55e9335b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 shutdown_connections 2023-10-16T09:59:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 1 -- 172.21.15.181:0/1492087809 wait complete. 2023-10-16T09:59:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.756+0000 7f9e5e0afb80 20 client.6618 trim_cache size 0 max 16384 2023-10-16T09:59:29.778 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:59:29.778 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T09:59:29.778 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T09:59:29.778 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 6474, 2023-10-16T09:59:29.779 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T09:59:29.779 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T09:59:29.779 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T09:59:29.779 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.897625648, 2023-10-16T09:59:29.779 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 32, 2023-10-16T09:59:29.780 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 76, 2023-10-16T09:59:29.780 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 76, 2023-10-16T09:59:29.780 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.837843221 2023-10-16T09:59:29.780 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:59:29.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9e3806b020 msgr2=0x7f9e3806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9e3806b020 0x7f9e3806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006fd0 tx=0x7f9e40008040 comp rx=0 tx=0).stop 2023-10-16T09:59:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 msgr2=0x7f9e4c1f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c1f3300 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9e44015040 tx=0x7f9e44004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 shutdown_connections 2023-10-16T09:59:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9e3806b020 0x7f9e3806d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c0ce4d0 0x7f9e4c1f3300 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c06d210 0x7f9e4c15d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 --2- 172.21.15.181:0/3452872761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c0cc4c0 0x7f9e4c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 >> 172.21.15.181:0/3452872761 conn(0x7f9e4c09a9a0 msgr2=0x7f9e4c0c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 shutdown_connections 2023-10-16T09:59:29.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.757+0000 7f9e51ec5700 1 -- 172.21.15.181:0/3452872761 wait complete. 2023-10-16T09:59:29.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T09:59:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.903+0000 7f48913cf700 1 Processor -- start 2023-10-16T09:59:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.903+0000 7f48913cf700 1 -- start start 2023-10-16T09:59:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f48913cf700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f488c0ce7f0 0x7f488c06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f48913cf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c0cc470 con 0x7f488c0d2390 2023-10-16T09:59:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f48913cf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c06ca30 con 0x7f488c0d0350 2023-10-16T09:59:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f48913cf700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c06cd10 con 0x7f488c0ce7f0 2023-10-16T09:59:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f488a7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f488c0d2390 0x7f488c0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52942/0 (socket says 172.21.15.181:52942) 2023-10-16T09:59:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f488a7fc700 1 -- 172.21.15.181:0/1335843048 learned_addr learned my addr 172.21.15.181:0/1335843048 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f488affd700 1 --2- 172.21.15.181:0/1335843048 >> v2:172.21.15.190:3300/0 conn(0x7f488c0ce7f0 0x7f488c06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.904+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878147775 0 0) 0x7f488c0cc470 con 0x7f488c0d2390 2023-10-16T09:59:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48740035b0 con 0x7f488c0d2390 2023-10-16T09:59:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150256469 0 0) 0x7f488c06ca30 con 0x7f488c0d0350 2023-10-16T09:59:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f488c0cc470 con 0x7f488c0d0350 2023-10-16T09:59:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 589751106 0 0) 0x7f48740035b0 con 0x7f488c0d2390 2023-10-16T09:59:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 >> v2:172.21.15.190:3300/0 conn(0x7f488c0ce7f0 msgr2=0x7f488c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 --2- 172.21.15.181:0/1335843048 >> v2:172.21.15.190:3300/0 conn(0x7f488c0ce7f0 0x7f488c06c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 >> v1:172.21.15.190:6789/0 conn(0x7f488c0d0350 legacy=0x7f488c0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c06cff0 con 0x7f488c0d2390 2023-10-16T09:59:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f487c001670 con 0x7f488c0d2390 2023-10-16T09:59:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 >> v1:172.21.15.181:6789/0 conn(0x7f488c0d2390 legacy=0x7f488c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4874003b90 0x7f4874003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874004850 0x7f4874004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4874005210 0x7f487400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c06cd10 con 0x7f4874005210 2023-10-16T09:59:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487400c090 con 0x7f4874003b90 2023-10-16T09:59:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487400c250 con 0x7f4874004850 2023-10-16T09:59:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4874005210 0x7f487400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488affd700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4874003b90 0x7f4874003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488a7fc700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874004850 0x7f4874004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 -- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874004850 msgr2=0x7f4874004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874004850 0x7f4874004cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 -- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4874003b90 msgr2=0x7f4874003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4874003b90 0x7f4874003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 -- 172.21.15.181:0/1335843048 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740035b0 con 0x7f4874005210 2023-10-16T09:59:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 -- 172.21.15.181:0/1335843048 reap_dead start 2023-10-16T09:59:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.905+0000 7f488b7fe700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4874005210 0x7f487400ba10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4884002230 tx=0x7f4874005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6302475cc95b42e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884001f30 con 0x7f4874005210 2023-10-16T09:59:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400ce80 con 0x7f4874005210 2023-10-16T09:59:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f4889ffb700 1 -- 172.21.15.181:0/1335843048 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884003c40 con 0x7f4874005210 2023-10-16T09:59:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f48913cf700 1 -- 172.21.15.181:0/1335843048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4874005210 msgr2=0x7f487400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f48913cf700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4874005210 0x7f487400ba10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4884002230 tx=0x7f4874005880 comp rx=0 tx=0).stop 2023-10-16T09:59:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f48913cf700 1 -- 172.21.15.181:0/1335843048 shutdown_connections 2023-10-16T09:59:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f48913cf700 1 --2- 172.21.15.181:0/1335843048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4874005210 0x7f487400ba10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f48913cf700 1 -- 172.21.15.181:0/1335843048 >> 172.21.15.181:0/1335843048 conn(0x7f488c09a9a0 msgr2=0x7f488c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f48913cf700 1 -- 172.21.15.181:0/1335843048 shutdown_connections 2023-10-16T09:59:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.906+0000 7f48913cf700 1 -- 172.21.15.181:0/1335843048 wait complete. 2023-10-16T09:59:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 Processor -- start 2023-10-16T09:59:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 -- start start 2023-10-16T09:59:29.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874005210 0x7f488c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c09b310 0x7f488c09edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f488c0cd530 0x7f488c15f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0ae7d0 con 0x7f488c09b310 2023-10-16T09:59:29.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c09b880 con 0x7f488c0cd530 2023-10-16T09:59:29.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f48913cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c09bb60 con 0x7f4874005210 2023-10-16T09:59:29.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.908+0000 7f488a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c09b310 0x7f488c09edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c09b310 0x7f488c09edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41878/0 (socket says 172.21.15.181:41878) 2023-10-16T09:59:29.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 -- 172.21.15.181:0/2474767259 learned_addr learned my addr 172.21.15.181:0/2474767259 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:29.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488affd700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874005210 0x7f488c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488b7fe700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f488c0cd530 0x7f488c15f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 -- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874005210 msgr2=0x7f488c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874005210 0x7f488c0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 -- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f488c0cd530 msgr2=0x7f488c15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:29.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f488c0cd530 0x7f488c15f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:29.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 -- 172.21.15.181:0/2474767259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c09be40 con 0x7f488c09b310 2023-10-16T09:59:29.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f488a7fc700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c09b310 0x7f488c09edd0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f487c00db10 tx=0x7f487c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f48737fe700 1 -- 172.21.15.181:0/2474767259 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c004a80 con 0x7f488c09b310 2023-10-16T09:59:29.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f48737fe700 1 -- 172.21.15.181:0/2474767259 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c004be0 con 0x7f488c09b310 2023-10-16T09:59:29.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c09c120 con 0x7f488c09b310 2023-10-16T09:59:29.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f48737fe700 1 -- 172.21.15.181:0/2474767259 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c0070d0 con 0x7f488c09b310 2023-10-16T09:59:29.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.909+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c09f310 con 0x7f488c09b310 2023-10-16T09:59:29.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.910+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c09c640 con 0x7f488c09b310 2023-10-16T09:59:29.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.910+0000 7f48737fe700 1 -- 172.21.15.181:0/2474767259 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f487c007230 con 0x7f488c09b310 2023-10-16T09:59:29.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.911+0000 7f48737fe700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f487806b020 0x7f487806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:29.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.911+0000 7f48737fe700 1 -- 172.21.15.181:0/2474767259 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f487c0895b0 con 0x7f488c09b310 2023-10-16T09:59:29.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.911+0000 7f488affd700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f487806b020 0x7f487806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:29.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.911+0000 7f488affd700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f487806b020 0x7f487806d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4880009a00 tx=0x7f4880008040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:29.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:29.915+0000 7f48737fe700 1 -- 172.21.15.181:0/2474767259 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f488c09c640 con 0x7f488c09b310 2023-10-16T09:59:30.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.155+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f488c09c640 con 0x7f488c09b310 2023-10-16T09:59:30.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.155+0000 7f48737fe700 1 -- 172.21.15.181:0/2474767259 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f488c09c640 con 0x7f488c09b310 2023-10-16T09:59:30.157 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T09:59:30.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f487806b020 msgr2=0x7f487806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:30.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f487806b020 0x7f487806d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4880009a00 tx=0x7f4880008040 comp rx=0 tx=0).stop 2023-10-16T09:59:30.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c09b310 msgr2=0x7f488c09edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:30.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c09b310 0x7f488c09edd0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f487c00db10 tx=0x7f487c00ded0 comp rx=0 tx=0).stop 2023-10-16T09:59:30.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 shutdown_connections 2023-10-16T09:59:30.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f487806b020 0x7f487806d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:30.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4874005210 0x7f488c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:30.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f488c0cd530 0x7f488c15f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:30.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 --2- 172.21.15.181:0/2474767259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c09b310 0x7f488c09edd0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:30.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 >> 172.21.15.181:0/2474767259 conn(0x7f488c09a9a0 msgr2=0x7f488c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:30.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 shutdown_connections 2023-10-16T09:59:30.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:30.157+0000 7f48913cf700 1 -- 172.21.15.181:0/2474767259 wait complete. 2023-10-16T09:59:30.171 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T09:59:30.171 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T09:59:30.171 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T09:59:30.172 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T09:59:30.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T09:59:30.172 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T09:59:30.173 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T09:59:30.173 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T09:59:30.173 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:59:30.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.174 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T09:59:30.191 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T09:59:30.191 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T09:59:30.192 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T09:59:30.192 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:59:30.192 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T09:59:30.192 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:59:30.192 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T09:59:30.194 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T09:59:30.194 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T09:59:30.194 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41499sec preferred_lft 41499sec 2023-10-16T09:59:30.194 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T09:59:30.194 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:59:30.195 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T09:59:30.195 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T09:59:30.195 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T09:59:30.195 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:59:30.195 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T09:59:30.196 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:59:30.196 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:59:30.196 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T09:59:30.196 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T09:59:30.196 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:59:30.197 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T09:59:30.197 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T09:59:30.197 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T09:59:30.197 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T09:59:30.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.198 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T09:59:30.256 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T09:59:30.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.258 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T09:59:30.316 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T09:59:30.316 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T09:59:30.316 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T09:59:30.316 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T09:59:30.317 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T09:59:30.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.317 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T09:59:30.318 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T09:59:30.318 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T09:59:30.319 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T09:59:30.319 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T09:59:30.319 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T09:59:30.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.421 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T09:59:30.437 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T09:59:30.437 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T09:59:30.438 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:59:30.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.438 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T09:59:30.504 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:59:30.504 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T09:59:30.561 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T09:59:30.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T09:59:30.562 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:59:30.617 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T09:59:30.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:59:30.702 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:59:30.703 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:59:30.703 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:59:30.711 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:59:30.712 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:59:30.712 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:59:30.712 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:59:30.712 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:59:30.786 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T09:59:30.786 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T09:59:30.786 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T09:59:30.787 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T09:59:30.787 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T09:59:30.787 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T09:59:30.787 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T09:59:30.787 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T09:59:30.788 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T09:59:30.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.789 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T09:59:30.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:30.817 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T09:59:30.908 INFO:teuthology.orchestra.run.smithi181.stdout:client6166 2023-10-16T09:59:30.909 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:30.909 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:30.926 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T09:59:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.086+0000 7f95febd7700 1 Processor -- start 2023-10-16T09:59:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.086+0000 7f95febd7700 1 -- start start 2023-10-16T09:59:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.087+0000 7f95febd7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 0x7f95f80c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.087+0000 7f95febd7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95f815ded0 0x7f95f806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.087+0000 7f95febd7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f815b410 con 0x7f95f815daf0 2023-10-16T09:59:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.087+0000 7f95febd7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f80c1f50 con 0x7f95f815ded0 2023-10-16T09:59:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.087+0000 7f95febd7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cfa70 con 0x7f95f80c2cb0 2023-10-16T09:59:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.087+0000 7f95fd174700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95f815ded0 0x7f95f806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fd174700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95f815ded0 0x7f95f806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41892/0 (socket says 172.21.15.181:41892) 2023-10-16T09:59:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fc973700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f95f815daf0 0x7f95f80c2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52966/0 (socket says 172.21.15.181:52966) 2023-10-16T09:59:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fd174700 1 -- 172.21.15.181:0/2517520761 learned_addr learned my addr 172.21.15.181:0/2517520761 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95f7fff700 1 --2- 172.21.15.181:0/2517520761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 0x7f95f80c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fd174700 1 -- 172.21.15.181:0/2517520761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 msgr2=0x7f95f80c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fd174700 1 --2- 172.21.15.181:0/2517520761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 0x7f95f80c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fd174700 1 -- 172.21.15.181:0/2517520761 >> v1:172.21.15.181:6789/0 conn(0x7f95f815daf0 legacy=0x7f95f80c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95f77fe700 1 -- 172.21.15.181:0/2517520761 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268279151 0 0) 0x7f95f815b410 con 0x7f95f815daf0 2023-10-16T09:59:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fd174700 1 -- 172.21.15.181:0/2517520761 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80c2a80 con 0x7f95f815ded0 2023-10-16T09:59:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95fd174700 1 --2- 172.21.15.181:0/2517520761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f806a8a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009e80 tx=0x7f95f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a547d11f870e4b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95f77fe700 1 -- 172.21.15.181:0/2517520761 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f000e070 con 0x7f95f815ded0 2023-10-16T09:59:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95f77fe700 1 -- 172.21.15.181:0/2517520761 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f000b980 con 0x7f95f815ded0 2023-10-16T09:59:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.088+0000 7f95f77fe700 1 -- 172.21.15.181:0/2517520761 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f000bd30 con 0x7f95f815ded0 2023-10-16T09:59:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 -- 172.21.15.181:0/2517520761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 msgr2=0x7f95f806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 --2- 172.21.15.181:0/2517520761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f806a8a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009e80 tx=0x7f95f000b330 comp rx=0 tx=0).stop 2023-10-16T09:59:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 -- 172.21.15.181:0/2517520761 shutdown_connections 2023-10-16T09:59:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 --2- 172.21.15.181:0/2517520761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f806a8a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 --2- 172.21.15.181:0/2517520761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 0x7f95f80c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 -- 172.21.15.181:0/2517520761 >> 172.21.15.181:0/2517520761 conn(0x7f95f80b1780 msgr2=0x7f95f80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:31.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 -- 172.21.15.181:0/2517520761 shutdown_connections 2023-10-16T09:59:31.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.089+0000 7f95febd7700 1 -- 172.21.15.181:0/2517520761 wait complete. 2023-10-16T09:59:31.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.090+0000 7f95febd7700 1 Processor -- start 2023-10-16T09:59:31.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.090+0000 7f95febd7700 1 -- start start 2023-10-16T09:59:31.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95febd7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 0x7f95f809e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95febd7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f815daf0 0x7f95f814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95febd7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f80a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95febd7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f809e470 con 0x7f95f815ded0 2023-10-16T09:59:31.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95febd7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815c590 con 0x7f95f815daf0 2023-10-16T09:59:31.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95febd7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f815bfc0 con 0x7f95f80c2cb0 2023-10-16T09:59:31.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f80a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f80a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41898/0 (socket says 172.21.15.181:41898) 2023-10-16T09:59:31.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 -- 172.21.15.181:0/3181012288 learned_addr learned my addr 172.21.15.181:0/3181012288 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95f7fff700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f815daf0 0x7f95f814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 -- 172.21.15.181:0/3181012288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 msgr2=0x7f95f809e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 0x7f95f809e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 -- 172.21.15.181:0/3181012288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f815daf0 msgr2=0x7f95f814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f815daf0 0x7f95f814f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 -- 172.21.15.181:0/3181012288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0009b30 con 0x7f95f815ded0 2023-10-16T09:59:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95fd174700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f80a01d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b730 tx=0x7f95f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95f57fa700 1 -- 172.21.15.181:0/3181012288 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00099b0 con 0x7f95f815ded0 2023-10-16T09:59:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.091+0000 7f95f57fa700 1 -- 172.21.15.181:0/3181012288 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00092e0 con 0x7f95f815ded0 2023-10-16T09:59:31.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.092+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f809deb0 con 0x7f95f815ded0 2023-10-16T09:59:31.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.092+0000 7f95f57fa700 1 -- 172.21.15.181:0/3181012288 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f001b630 con 0x7f95f815ded0 2023-10-16T09:59:31.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.092+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f815c1b0 con 0x7f95f815ded0 2023-10-16T09:59:31.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.093+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f8067d20 con 0x7f95f815ded0 2023-10-16T09:59:31.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.093+0000 7f95f57fa700 1 -- 172.21.15.181:0/3181012288 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f95f001b790 con 0x7f95f815ded0 2023-10-16T09:59:31.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.093+0000 7f95f57fa700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f95dc06b020 0x7f95dc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.093+0000 7f95f57fa700 1 -- 172.21.15.181:0/3181012288 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f95f008af10 con 0x7f95f815ded0 2023-10-16T09:59:31.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.093+0000 7f95fc973700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f95dc06b020 0x7f95dc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.094+0000 7f95fc973700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f95dc06b020 0x7f95dc06d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0011e0 tx=0x7f95ec0079b0 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:31.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.098+0000 7f95f57fa700 1 -- 172.21.15.181:0/3181012288 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f95f0055c80 con 0x7f95f815ded0 2023-10-16T09:59:31.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.319+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 --> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f95f80c15b0 con 0x7f95dc06b020 2023-10-16T09:59:31.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.321+0000 7f95f57fa700 1 -- 172.21.15.181:0/3181012288 <== mgr.6222 v2:172.21.15.181:6832/3602430955 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f95ec004870 con 0x7f95dc06b020 2023-10-16T09:59:31.322 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:59:31.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:31.323 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T09:59:31.323 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:31.323 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:59:31.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f95dc06b020 msgr2=0x7f95dc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f95dc06b020 0x7f95dc06d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0011e0 tx=0x7f95ec0079b0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 msgr2=0x7f95f80a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f80a01d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f95f000b730 tx=0x7f95f0004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 shutdown_connections 2023-10-16T09:59:31.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f95dc06b020 0x7f95dc06d4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95f80c2cb0 0x7f95f809e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95f815ded0 0x7f95f80a01d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 --2- 172.21.15.181:0/3181012288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95f815daf0 0x7f95f814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 >> 172.21.15.181:0/3181012288 conn(0x7f95f80b1780 msgr2=0x7f95f80cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:31.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 shutdown_connections 2023-10-16T09:59:31.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.323+0000 7f95febd7700 1 -- 172.21.15.181:0/3181012288 wait complete. 2023-10-16T09:59:31.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.457+0000 7faccedb6700 1 Processor -- start 2023-10-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.457+0000 7faccedb6700 1 -- start start 2023-10-16T09:59:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7faccedb6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facc80ce6c0 0x7facc80cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7faccedb6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7facc80cc6a0 0x7facc80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7faccedb6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc80b1f70 con 0x7facc815f150 2023-10-16T09:59:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7faccedb6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc80b2ae0 con 0x7facc80ce6c0 2023-10-16T09:59:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7faccedb6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc80b2dc0 con 0x7facc80cc6a0 2023-10-16T09:59:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7facc7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facc80ce6c0 0x7facc80cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7facccb52700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7facc815f150 0x7facc80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52980/0 (socket says 172.21.15.181:52980) 2023-10-16T09:59:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7facccb52700 1 -- 172.21.15.181:0/3888530344 learned_addr learned my addr 172.21.15.181:0/3888530344 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7facc7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facc80ce6c0 0x7facc80cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41902/0 (socket says 172.21.15.181:41902) 2023-10-16T09:59:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.458+0000 7faccd353700 1 --2- 172.21.15.181:0/3888530344 >> v2:172.21.15.190:3300/0 conn(0x7facc80cc6a0 0x7facc80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc7fff700 1 -- 172.21.15.181:0/3888530344 >> v2:172.21.15.190:3300/0 conn(0x7facc80cc6a0 msgr2=0x7facc80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc7fff700 1 --2- 172.21.15.181:0/3888530344 >> v2:172.21.15.190:3300/0 conn(0x7facc80cc6a0 0x7facc80ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc7fff700 1 -- 172.21.15.181:0/3888530344 >> v1:172.21.15.181:6789/0 conn(0x7facc815f150 legacy=0x7facc80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc7fff700 1 -- 172.21.15.181:0/3888530344 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80cf9d0 con 0x7facc80ce6c0 2023-10-16T09:59:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc7fff700 1 --2- 172.21.15.181:0/3888530344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc80ce6c0 0x7facc80cc160 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7facc80ceb70 tx=0x7facb4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf25cacc37d6b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc77fe700 1 -- 172.21.15.181:0/3888530344 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4011830 con 0x7facc80ce6c0 2023-10-16T09:59:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc77fe700 1 -- 172.21.15.181:0/3888530344 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb4011990 con 0x7facc80ce6c0 2023-10-16T09:59:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7facc77fe700 1 -- 172.21.15.181:0/3888530344 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4011d20 con 0x7facc80ce6c0 2023-10-16T09:59:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 -- 172.21.15.181:0/3888530344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc80ce6c0 msgr2=0x7facc80cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 --2- 172.21.15.181:0/3888530344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc80ce6c0 0x7facc80cc160 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7facc80ceb70 tx=0x7facb4009b20 comp rx=0 tx=0).stop 2023-10-16T09:59:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 -- 172.21.15.181:0/3888530344 shutdown_connections 2023-10-16T09:59:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 --2- 172.21.15.181:0/3888530344 >> v2:172.21.15.190:3300/0 conn(0x7facc80cc6a0 0x7facc80ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 --2- 172.21.15.181:0/3888530344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc80ce6c0 0x7facc80cc160 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 -- 172.21.15.181:0/3888530344 >> 172.21.15.181:0/3888530344 conn(0x7facc809a930 msgr2=0x7facc80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 -- 172.21.15.181:0/3888530344 shutdown_connections 2023-10-16T09:59:31.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.459+0000 7faccedb6700 1 -- 172.21.15.181:0/3888530344 wait complete. 2023-10-16T09:59:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.461+0000 7faccedb6700 1 Processor -- start 2023-10-16T09:59:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.461+0000 7faccedb6700 1 -- start start 2023-10-16T09:59:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.461+0000 7faccedb6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc80cc6a0 0x7facc8155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccedb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facc80ce6c0 0x7facc815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccedb6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc815f150 0x7facc80c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccedb6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc815d330 con 0x7facc815f150 2023-10-16T09:59:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccedb6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc80c88e0 con 0x7facc80cc6a0 2023-10-16T09:59:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccedb6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc80b5180 con 0x7facc80ce6c0 2023-10-16T09:59:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc815f150 0x7facc80c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc815f150 0x7facc80c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41910/0 (socket says 172.21.15.181:41910) 2023-10-16T09:59:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 -- 172.21.15.181:0/330811557 learned_addr learned my addr 172.21.15.181:0/330811557 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:31.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7facccb52700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc80cc6a0 0x7facc8155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 -- 172.21.15.181:0/330811557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facc80ce6c0 msgr2=0x7facc815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facc80ce6c0 0x7facc815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 -- 172.21.15.181:0/330811557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc80cc6a0 msgr2=0x7facc8155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc80cc6a0 0x7facc8155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 -- 172.21.15.181:0/330811557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb40094a0 con 0x7facc815f150 2023-10-16T09:59:31.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccd353700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc815f150 0x7facc80c90f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7facc0007e60 tx=0x7facc000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:31.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7facc57fa700 1 -- 172.21.15.181:0/330811557 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000ede0 con 0x7facc815f150 2023-10-16T09:59:31.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7facc57fa700 1 -- 172.21.15.181:0/330811557 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0014420 con 0x7facc815f150 2023-10-16T09:59:31.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc80b4b50 con 0x7facc815f150 2023-10-16T09:59:31.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7facc57fa700 1 -- 172.21.15.181:0/330811557 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00147d0 con 0x7facc815f150 2023-10-16T09:59:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.462+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc80c8e10 con 0x7facc815f150 2023-10-16T09:59:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.463+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc80b52f0 con 0x7facc815f150 2023-10-16T09:59:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.464+0000 7facc57fa700 1 -- 172.21.15.181:0/330811557 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 32) v1 ==== 89331+0+0 (secure 0 0 0) 0x7facc0014930 con 0x7facc815f150 2023-10-16T09:59:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.464+0000 7facc57fa700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7facac06b020 0x7facac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:31.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.464+0000 7facc57fa700 1 -- 172.21.15.181:0/330811557 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7facc0089fd0 con 0x7facc815f150 2023-10-16T09:59:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.464+0000 7facccb52700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7facac06b020 0x7facac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.464+0000 7facccb52700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7facac06b020 0x7facac06d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7facbc005910 tx=0x7facbc009040 comp rx=0 tx=0).ready entity=mgr.6222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:31.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.468+0000 7facc57fa700 1 -- 172.21.15.181:0/330811557 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7facc0054e50 con 0x7facc815f150 2023-10-16T09:59:31.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.717+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7facc8067d20 con 0x7facc815f150 2023-10-16T09:59:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.970+0000 7facc57fa700 1 -- 172.21.15.181:0/330811557 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v33) v1 ==== 92+0+0 (secure 0 0 0) 0x7facc0058470 con 0x7facc815f150 2023-10-16T09:59:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.973+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7facac06b020 msgr2=0x7facac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.973+0000 7faccedb6700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7facac06b020 0x7facac06d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7facbc005910 tx=0x7facbc009040 comp rx=0 tx=0).stop 2023-10-16T09:59:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.973+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc815f150 msgr2=0x7facc80c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.973+0000 7faccedb6700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc815f150 0x7facc80c90f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7facc0007e60 tx=0x7facc000e3c0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 shutdown_connections 2023-10-16T09:59:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7facac06b020 0x7facac06d4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facc815f150 0x7facc80c90f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc80cc6a0 0x7facc8155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 --2- 172.21.15.181:0/330811557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facc80ce6c0 0x7facc815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 >> 172.21.15.181:0/330811557 conn(0x7facc809a930 msgr2=0x7facc80cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 shutdown_connections 2023-10-16T09:59:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:31.975+0000 7faccedb6700 1 -- 172.21.15.181:0/330811557 wait complete. 2023-10-16T09:59:31.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T09:59:32.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.115+0000 7fc4b09c8700 1 Processor -- start 2023-10-16T09:59:32.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.115+0000 7fc4b09c8700 1 -- start start 2023-10-16T09:59:32.118 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.115+0000 7feca52bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:59:32.118 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:32.115+0000 7f7416fd3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:59:32.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4b09c8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4ac0d24e0 0x7fc4ac0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4b09c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4ac0d0600 0x7fc4ac0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4b09c8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ac0d2960 con 0x7fc4ac15d9e0 2023-10-16T09:59:32.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4b09c8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ac0d20c0 con 0x7fc4ac0d24e0 2023-10-16T09:59:32.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4b09c8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac0d0a90 con 0x7fc4ac0d0600 2023-10-16T09:59:32.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4aa59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc4ac15d9e0 0x7fc4ac0cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53018/0 (socket says 172.21.15.181:53018) 2023-10-16T09:59:32.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4aa59c700 1 -- 172.21.15.181:0/1903451758 learned_addr learned my addr 172.21.15.181:0/1903451758 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:32.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4aad9d700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4ac0d0600 0x7fc4ac0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a9d9b700 1 --2- 172.21.15.181:0/1903451758 >> v2:172.21.15.190:3300/0 conn(0x7fc4ac0d24e0 0x7fc4ac0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6212148 0 0) 0x7fc4ac0d2960 con 0x7fc4ac15d9e0 2023-10-16T09:59:32.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4940035b0 con 0x7fc4ac15d9e0 2023-10-16T09:59:32.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3957427788 0 0) 0x7fc4940035b0 con 0x7fc4ac15d9e0 2023-10-16T09:59:32.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4ac0d0600 msgr2=0x7fc4ac0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:32.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4ac0d0600 0x7fc4ac0b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 >> v2:172.21.15.190:3300/0 conn(0x7fc4ac0d24e0 msgr2=0x7fc4ac0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:32.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 --2- 172.21.15.181:0/1903451758 >> v2:172.21.15.190:3300/0 conn(0x7fc4ac0d24e0 0x7fc4ac0d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac0d4180 con 0x7fc4ac15d9e0 2023-10-16T09:59:32.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc49c001670 con 0x7fc4ac15d9e0 2023-10-16T09:59:32.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 >> v1:172.21.15.181:6789/0 conn(0x7fc4ac15d9e0 legacy=0x7fc4ac0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc494003a00 0x7fc494003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494004850 0x7fc494004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc494005210 0x7fc49400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac0d20c0 con 0x7fc494003a00 2023-10-16T09:59:32.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac0d0a90 con 0x7fc494005210 2023-10-16T09:59:32.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.116+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49400bfa0 con 0x7fc494004850 2023-10-16T09:59:32.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4a9d9b700 1 --2- 172.21.15.181:0/1903451758 >> v2:172.21.15.190:3300/0 conn(0x7fc4ac0d24e0 0x7fc4ac0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:32.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aad9d700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4ac0d0600 0x7fc4ac0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:32.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aa59c700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc494003a00 0x7fc494003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4a9d9b700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494004850 0x7fc494004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aad9d700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc494005210 0x7fc49400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aa59c700 1 -- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494004850 msgr2=0x7fc494004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:32.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aa59c700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494004850 0x7fc494004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aa59c700 1 -- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc494005210 msgr2=0x7fc49400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:32.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aa59c700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc494005210 0x7fc49400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aa59c700 1 -- 172.21.15.181:0/1903451758 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940035b0 con 0x7fc494003a00 2023-10-16T09:59:32.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4aa59c700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc494003a00 0x7fc494003e20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00c350 tx=0x7fc494005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7497ee7d732d462 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:32.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00ce50 con 0x7fc494003a00 2023-10-16T09:59:32.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c005960 con 0x7fc494003a00 2023-10-16T09:59:32.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.117+0000 7fc4a959a700 1 -- 172.21.15.181:0/1903451758 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c005d10 con 0x7fc494003a00 2023-10-16T09:59:32.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.118+0000 7fc4b09c8700 1 -- 172.21.15.181:0/1903451758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc494003a00 msgr2=0x7fc494003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.118+0000 7fc4b09c8700 1 --2- 172.21.15.181:0/1903451758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc494003a00 0x7fc494003e20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00c350 tx=0x7fc494005880 comp rx=0 tx=0).stop 2023-10-16T09:59:32.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.118+0000 7fc4aad9d700 1 -- 172.21.15.181:0/1903451758 reap_dead start 2023-10-16T09:59:32.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.118+0000 7fc4b09c8700 1 -- 172.21.15.181:0/1903451758 shutdown_connections 2023-10-16T09:59:32.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.118+0000 7fc4b09c8700 1 -- 172.21.15.181:0/1903451758 >> 172.21.15.181:0/1903451758 conn(0x7fc4ac09cb40 msgr2=0x7fc4ac0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:32.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.118+0000 7fc4b09c8700 1 -- 172.21.15.181:0/1903451758 shutdown_connections 2023-10-16T09:59:32.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.118+0000 7fc4b09c8700 1 -- 172.21.15.181:0/1903451758 wait complete. 2023-10-16T09:59:32.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.119+0000 7fc4b09c8700 1 Processor -- start 2023-10-16T09:59:32.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.119+0000 7fc4b09c8700 1 -- start start 2023-10-16T09:59:32.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4b09c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494003a00 0x7fc4ac14bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4b09c8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4ac1f2d90 0x7fc4ac1f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4b09c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4ac15ef40 0x7fc4ac15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4b09c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac15f900 con 0x7fc4ac15ef40 2023-10-16T09:59:32.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4b09c8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac1f3de0 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4b09c8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac1f3820 con 0x7fc494003a00 2023-10-16T09:59:32.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4aad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4ac15ef40 0x7fc4ac15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4aa59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494003a00 0x7fc4ac14bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4a9d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4ac1f2d90 0x7fc4ac1f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4aad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4ac15ef40 0x7fc4ac15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41956/0 (socket says 172.21.15.181:41956) 2023-10-16T09:59:32.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.120+0000 7fc4aad9d700 1 -- 172.21.15.181:0/2904816272 learned_addr learned my addr 172.21.15.181:0/2904816272 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:32.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4a9d9b700 1 -- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494003a00 msgr2=0x7fc4ac14bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4a9d9b700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494003a00 0x7fc4ac14bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4a9d9b700 1 -- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4ac15ef40 msgr2=0x7fc4ac15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4a9d9b700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4ac15ef40 0x7fc4ac15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4a9d9b700 1 -- 172.21.15.181:0/2904816272 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac1f3b00 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4aa59c700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494003a00 0x7fc4ac14bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:32.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4a9d9b700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4ac1f2d90 0x7fc4ac1f3210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0011a00 tx=0x7fc4a0011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:32.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc49affd700 1 -- 172.21.15.181:0/2904816272 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0004020 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc49affd700 1 -- 172.21.15.181:0/2904816272 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a0004180 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc49affd700 1 -- 172.21.15.181:0/2904816272 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0004530 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac0c72d0 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.121+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac1f3fb0 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.122+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac0c7920 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.122+0000 7fc49affd700 1 -- 172.21.15.181:0/2904816272 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 33) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc4a000f940 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.123+0000 7fc49affd700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 0x7fc48c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.123+0000 7fc4aa59c700 1 -- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 msgr2=0x7fc48c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:32.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.123+0000 7fc49affd700 1 -- 172.21.15.181:0/2904816272 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fc4a0089c20 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.123+0000 7fc4aa59c700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 0x7fc48c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:32.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.128+0000 7fc49affd700 1 -- 172.21.15.181:0/2904816272 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc4a0054aa0 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.183 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.182+0000 7feca52bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:32.186 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:32.185+0000 7f7416fd3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:32.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.254+0000 7feca52bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:59:32.259 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:32.258+0000 7f7416fd3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:59:32.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.324+0000 7fc4aa59c700 1 -- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 msgr2=0x7fc48c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:32.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.324+0000 7fc4aa59c700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 0x7fc48c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:32.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.330+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc4ac067d20 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.333 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.332+0000 7feca52bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:59:32.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.335+0000 7fc49affd700 1 -- 172.21.15.181:0/2904816272 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc4a0022070 con 0x7fc4ac1f2d90 2023-10-16T09:59:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.338+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 msgr2=0x7fc48c06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:32.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.338+0000 7fc4b09c8700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 0x7fc48c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.338+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4ac1f2d90 msgr2=0x7fc4ac1f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.338+0000 7fc4b09c8700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4ac1f2d90 0x7fc4ac1f3210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0011a00 tx=0x7fc4a0011d10 comp rx=0 tx=0).stop 2023-10-16T09:59:32.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 shutdown_connections 2023-10-16T09:59:32.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7fc48c06b070 0x7fc48c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc494003a00 0x7fc4ac14bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4ac15ef40 0x7fc4ac15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 --2- 172.21.15.181:0/2904816272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4ac1f2d90 0x7fc4ac1f3210 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 >> 172.21.15.181:0/2904816272 conn(0x7fc4ac09cb40 msgr2=0x7fc4ac15cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:32.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 shutdown_connections 2023-10-16T09:59:32.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.339+0000 7fc4b09c8700 1 -- 172.21.15.181:0/2904816272 wait complete. 2023-10-16T09:59:32.343 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:32.341+0000 7f7416fd3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:59:32.357 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T09:59:32.474 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.472+0000 7feca52bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:59:32.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.482+0000 7f79365f7700 1 Processor -- start 2023-10-16T09:59:32.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.482+0000 7f79365f7700 1 -- start start 2023-10-16T09:59:32.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f79365f7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7930071be0 0x7f79300ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f79365f7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300ce1f0 con 0x7f79300b2010 2023-10-16T09:59:32.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f79365f7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300ccda0 con 0x7f79300cc6e0 2023-10-16T09:59:32.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f79365f7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79300ccb90 con 0x7f7930071be0 2023-10-16T09:59:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f792ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f79300b2010 0x7f79300b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53084/0 (socket says 172.21.15.181:53084) 2023-10-16T09:59:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f7934b94700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7930071be0 0x7f79300ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f792ffff700 1 -- 172.21.15.181:0/921859694 learned_addr learned my addr 172.21.15.181:0/921859694 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.483+0000 7f792effd700 1 -- 172.21.15.181:0/921859694 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922850696 0 0) 0x7f79300ce1f0 con 0x7f79300b2010 2023-10-16T09:59:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f792effd700 1 -- 172.21.15.181:0/921859694 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79180035b0 con 0x7f79300b2010 2023-10-16T09:59:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f792effd700 1 -- 172.21.15.181:0/921859694 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073711539 0 0) 0x7f79300ccda0 con 0x7f79300cc6e0 2023-10-16T09:59:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f792effd700 1 -- 172.21.15.181:0/921859694 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79300ce1f0 con 0x7f79300cc6e0 2023-10-16T09:59:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f7934b94700 1 -- 172.21.15.181:0/921859694 >> v1:172.21.15.190:6789/0 conn(0x7f79300cc6e0 legacy=0x7f79300ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f7934b94700 1 -- 172.21.15.181:0/921859694 >> v1:172.21.15.181:6789/0 conn(0x7f79300b2010 legacy=0x7f79300b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f7934b94700 1 -- 172.21.15.181:0/921859694 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79300727e0 con 0x7f7930071be0 2023-10-16T09:59:32.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f7934b94700 1 --2- 172.21.15.181:0/921859694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7930071be0 0x7f79300ced90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7928009d90 tx=0x7f792800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cd656040431717 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:32.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f792effd700 1 -- 172.21.15.181:0/921859694 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800e070 con 0x7f7930071be0 2023-10-16T09:59:32.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f792effd700 1 -- 172.21.15.181:0/921859694 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800bae0 con 0x7f7930071be0 2023-10-16T09:59:32.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.484+0000 7f792effd700 1 -- 172.21.15.181:0/921859694 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800be90 con 0x7f7930071be0 2023-10-16T09:59:32.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.485+0000 7f79365f7700 1 -- 172.21.15.181:0/921859694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7930071be0 msgr2=0x7f79300ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.485+0000 7f79365f7700 1 --2- 172.21.15.181:0/921859694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7930071be0 0x7f79300ced90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7928009d90 tx=0x7f792800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:32.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.485+0000 7f79365f7700 1 -- 172.21.15.181:0/921859694 shutdown_connections 2023-10-16T09:59:32.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.485+0000 7f79365f7700 1 --2- 172.21.15.181:0/921859694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7930071be0 0x7f79300ced90 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.485+0000 7f79365f7700 1 -- 172.21.15.181:0/921859694 >> 172.21.15.181:0/921859694 conn(0x7f79300cd010 msgr2=0x7f79300cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:32.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.485+0000 7f79365f7700 1 -- 172.21.15.181:0/921859694 shutdown_connections 2023-10-16T09:59:32.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.485+0000 7f79365f7700 1 -- 172.21.15.181:0/921859694 wait complete. 2023-10-16T09:59:32.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.486+0000 7f79365f7700 1 Processor -- start 2023-10-16T09:59:32.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.486+0000 7f79365f7700 1 -- start start 2023-10-16T09:59:32.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f79365f7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7930071be0 0x7f79300b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f79365f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79300b2010 0x7f79301f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f79365f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79300cc6e0 0x7f79301f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f79365f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79301f7850 con 0x7f79300b2010 2023-10-16T09:59:32.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f79365f7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79301f7370 con 0x7f7930071be0 2023-10-16T09:59:32.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f79365f7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79301f32b0 con 0x7f79300cc6e0 2023-10-16T09:59:32.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79300b2010 0x7f79301f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79300b2010 0x7f79301f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42014/0 (socket says 172.21.15.181:42014) 2023-10-16T09:59:32.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 -- 172.21.15.181:0/1652284456 learned_addr learned my addr 172.21.15.181:0/1652284456 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:32.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792ffff700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7930071be0 0x7f79300b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f7934b94700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79300cc6e0 0x7f79301f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 -- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79300cc6e0 msgr2=0x7f79301f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79300cc6e0 0x7f79301f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 -- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7930071be0 msgr2=0x7f79300b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7930071be0 0x7f79300b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.487+0000 7f792f7fe700 1 -- 172.21.15.181:0/1652284456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928009a70 con 0x7f79300b2010 2023-10-16T09:59:32.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.488+0000 7f792f7fe700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79300b2010 0x7f79301f68f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7924001eb0 tx=0x7f792400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:32.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.488+0000 7f792cff9700 1 -- 172.21.15.181:0/1652284456 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240054e0 con 0x7f79300b2010 2023-10-16T09:59:32.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.488+0000 7f792cff9700 1 -- 172.21.15.181:0/1652284456 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7924003680 con 0x7f79300b2010 2023-10-16T09:59:32.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.488+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79301f7570 con 0x7f79300b2010 2023-10-16T09:59:32.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.488+0000 7f792cff9700 1 -- 172.21.15.181:0/1652284456 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240039f0 con 0x7f79300b2010 2023-10-16T09:59:32.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.488+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79301f37f0 con 0x7f79300b2010 2023-10-16T09:59:32.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.489+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79300b6510 con 0x7f79300b2010 2023-10-16T09:59:32.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.490+0000 7f792cff9700 1 -- 172.21.15.181:0/1652284456 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 33) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7924003b50 con 0x7f79300b2010 2023-10-16T09:59:32.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.490+0000 7f792cff9700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 0x7f791c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.490+0000 7f792ffff700 1 -- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 msgr2=0x7f791c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:32.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.490+0000 7f792cff9700 1 -- 172.21.15.181:0/1652284456 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f79240890b0 con 0x7f79300b2010 2023-10-16T09:59:32.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.490+0000 7f792ffff700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 0x7f791c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:32.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.494+0000 7f792cff9700 1 -- 172.21.15.181:0/1652284456 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7924053fc0 con 0x7f79300b2010 2023-10-16T09:59:32.502 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:32.496+0000 7f7416fd3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:59:32.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.691+0000 7f792ffff700 1 -- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 msgr2=0x7f791c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:32.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.691+0000 7f792ffff700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 0x7f791c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:32.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.696+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f79300668b0 con 0x7f79300b2010 2023-10-16T09:59:32.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.702+0000 7f792cff9700 1 -- 172.21.15.181:0/1652284456 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f79240170c0 con 0x7f79300b2010 2023-10-16T09:59:32.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 msgr2=0x7f791c06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:32.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 0x7f791c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79300b2010 msgr2=0x7f79301f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79300b2010 0x7f79301f68f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7924001eb0 tx=0x7f792400c800 comp rx=0 tx=0).stop 2023-10-16T09:59:32.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 shutdown_connections 2023-10-16T09:59:32.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f791c06b070 0x7f791c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79300cc6e0 0x7f79301f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7930071be0 0x7f79300b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 --2- 172.21.15.181:0/1652284456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79300b2010 0x7f79301f68f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 >> 172.21.15.181:0/1652284456 conn(0x7f79300cd010 msgr2=0x7f7930158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:32.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 shutdown_connections 2023-10-16T09:59:32.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.704+0000 7f79365f7700 1 -- 172.21.15.181:0/1652284456 wait complete. 2023-10-16T09:59:32.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:32.719 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T09:59:32.740 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-16T09:59:32.799 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.797+0000 7feca52bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:59:32.835 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:32.833+0000 7f7416fd3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:59:32.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.903+0000 7f9f97d26700 1 Processor -- start 2023-10-16T09:59:32.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.904+0000 7f9f97d26700 1 -- start start 2023-10-16T09:59:32.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.904+0000 7f9f97d26700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f900ce200 0x7f9f9006be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.904+0000 7f9f97d26700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9015dfb0 con 0x7f9f900cdd80 2023-10-16T09:59:32.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.904+0000 7f9f97d26700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f900cd3a0 con 0x7f9f9015dbd0 2023-10-16T09:59:32.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.904+0000 7f9f97d26700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f900cd830 con 0x7f9f900ce200 2023-10-16T09:59:32.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f952c1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9f900cdd80 0x7f9f9006b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53134/0 (socket says 172.21.15.181:53134) 2023-10-16T09:59:32.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f952c1700 1 -- 172.21.15.181:0/2947379495 learned_addr learned my addr 172.21.15.181:0/2947379495 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:32.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f962c3700 1 --2- 172.21.15.181:0/2947379495 >> v2:172.21.15.190:3300/0 conn(0x7f9f900ce200 0x7f9f9006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907393935 0 0) 0x7f9f80001110 con 0x7f9f900cdd80 2023-10-16T09:59:32.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f7c0035b0 con 0x7f9f900cdd80 2023-10-16T09:59:32.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528912390 0 0) 0x7f9f900cd3a0 con 0x7f9f9015dbd0 2023-10-16T09:59:32.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f80001110 con 0x7f9f9015dbd0 2023-10-16T09:59:32.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318788168 0 0) 0x7f9f7c0035b0 con 0x7f9f900cdd80 2023-10-16T09:59:32.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f962c3700 1 -- 172.21.15.181:0/2947379495 >> v1:172.21.15.190:6789/0 conn(0x7f9f9015dbd0 legacy=0x7f9f900d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f962c3700 1 -- 172.21.15.181:0/2947379495 >> v1:172.21.15.181:6789/0 conn(0x7f9f900cdd80 legacy=0x7f9f9006b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f962c3700 1 -- 172.21.15.181:0/2947379495 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f900ce6b0 con 0x7f9f900ce200 2023-10-16T09:59:32.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f962c3700 1 --2- 172.21.15.181:0/2947379495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900ce200 0x7f9f9006be00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009d90 tx=0x7f9f8800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dd865889c833757 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:32.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800e070 con 0x7f9f900ce200 2023-10-16T09:59:32.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f88003a20 con 0x7f9f900ce200 2023-10-16T09:59:32.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.905+0000 7f9f94ac0700 1 -- 172.21.15.181:0/2947379495 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80001800 con 0x7f9f900ce200 2023-10-16T09:59:32.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.906+0000 7f9f97d26700 1 -- 172.21.15.181:0/2947379495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900ce200 msgr2=0x7f9f9006be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.906+0000 7f9f97d26700 1 --2- 172.21.15.181:0/2947379495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900ce200 0x7f9f9006be00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009d90 tx=0x7f9f8800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:32.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.906+0000 7f9f97d26700 1 -- 172.21.15.181:0/2947379495 shutdown_connections 2023-10-16T09:59:32.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.906+0000 7f9f97d26700 1 --2- 172.21.15.181:0/2947379495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900ce200 0x7f9f9006be00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.906+0000 7f9f97d26700 1 -- 172.21.15.181:0/2947379495 >> 172.21.15.181:0/2947379495 conn(0x7f9f9009a930 msgr2=0x7f9f900d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:32.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.906+0000 7f9f97d26700 1 -- 172.21.15.181:0/2947379495 shutdown_connections 2023-10-16T09:59:32.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.906+0000 7f9f97d26700 1 -- 172.21.15.181:0/2947379495 wait complete. 2023-10-16T09:59:32.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.907+0000 7f9f97d26700 1 Processor -- start 2023-10-16T09:59:32.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f97d26700 1 -- start start 2023-10-16T09:59:32.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f97d26700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900cdd80 0x7f9f90152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f97d26700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f900ce200 0x7f9f901f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f97d26700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9015dbd0 0x7f9f90160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f97d26700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900b5330 con 0x7f9f900ce200 2023-10-16T09:59:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f97d26700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900b5610 con 0x7f9f900cdd80 2023-10-16T09:59:32.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f97d26700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900b58f0 con 0x7f9f9015dbd0 2023-10-16T09:59:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f95ac2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900cdd80 0x7f9f90152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f952c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f900ce200 0x7f9f901f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f95ac2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900cdd80 0x7f9f90152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:47112/0 (socket says 172.21.15.181:47112) 2023-10-16T09:59:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f952c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f900ce200 0x7f9f901f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42064/0 (socket says 172.21.15.181:42064) 2023-10-16T09:59:32.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f95ac2700 1 -- 172.21.15.181:0/895237394 learned_addr learned my addr 172.21.15.181:0/895237394 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:32.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.908+0000 7f9f962c3700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9015dbd0 0x7f9f90160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:32.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f95ac2700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9015dbd0 msgr2=0x7f9f90160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f95ac2700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9015dbd0 0x7f9f90160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f95ac2700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f900ce200 msgr2=0x7f9f901f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:32.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f95ac2700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f900ce200 0x7f9f901f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:32.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f95ac2700 1 -- 172.21.15.181:0/895237394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88009a70 con 0x7f9f900cdd80 2023-10-16T09:59:32.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f95ac2700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f900cdd80 0x7f9f90152660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c520 tx=0x7f9f8c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:32.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0052d0 con 0x7f9f900cdd80 2023-10-16T09:59:32.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c005430 con 0x7f9f900cdd80 2023-10-16T09:59:32.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0038c0 con 0x7f9f900cdd80 2023-10-16T09:59:32.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f97d26700 1 -- 172.21.15.181:0/895237394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f900b5bd0 con 0x7f9f900cdd80 2023-10-16T09:59:32.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.909+0000 7f9f97d26700 1 -- 172.21.15.181:0/895237394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f901f3390 con 0x7f9f900cdd80 2023-10-16T09:59:32.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.910+0000 7f9f97d26700 1 -- 172.21.15.181:0/895237394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f90067d20 con 0x7f9f900cdd80 2023-10-16T09:59:32.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.911+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 33) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9f8c003a20 con 0x7f9f900cdd80 2023-10-16T09:59:32.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.911+0000 7f9f867fc700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 0x7f9f7406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:32.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.911+0000 7f9f952c1700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 msgr2=0x7f9f7406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:32.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.911+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f9f8c088db0 con 0x7f9f900cdd80 2023-10-16T09:59:32.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.911+0000 7f9f952c1700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 0x7f9f7406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:32.924 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.913+0000 7feca52bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:59:32.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:32.916+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9f8c053ba0 con 0x7f9f900cdd80 2023-10-16T09:59:32.955 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:32.954+0000 7f7416fd3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:59:32.989 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:32.987+0000 7feca52bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:59:33.031 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:33.030+0000 7f7416fd3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:59:33.069 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:33.066+0000 7feca52bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:59:33.112 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:33.111+0000 7f7416fd3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:59:33.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:33.112+0000 7f9f952c1700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 msgr2=0x7f9f7406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:33.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:33.112+0000 7f9f952c1700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 0x7f9f7406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:33.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:33.143+0000 7f9f97d26700 1 -- 172.21.15.181:0/895237394 --> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f900cc790 con 0x7f9f7406b070 2023-10-16T09:59:33.436 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:33.435+0000 7feca52bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:59:33.484 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:33.483+0000 7f7416fd3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:59:33.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:33.513+0000 7f9f952c1700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 msgr2=0x7f9f7406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:33.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:33.513+0000 7f9f952c1700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 0x7f9f7406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T09:59:34.002 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:34.001+0000 7feca52bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:59:34.052 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:34.050+0000 7f7416fd3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:59:34.070 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:34.069+0000 7feca52bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:59:34.121 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:34.119+0000 7f7416fd3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:59:34.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:34.314+0000 7f9f952c1700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 msgr2=0x7f9f7406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:34.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:34.314+0000 7f9f952c1700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 0x7f9f7406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T09:59:34.698 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:34.697+0000 7feca52bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:34.748 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:34.746+0000 7f7416fd3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:34.774 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:34.773+0000 7feca52bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:59:34.825 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:34.824+0000 7f7416fd3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:59:34.917 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:34.915+0000 7feca52bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:34.969 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:34.967+0000 7f7416fd3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:35.142 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:35.141+0000 7feca52bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:59:35.195 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:35.194+0000 7f7416fd3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:59:35.215 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:35.214+0000 7feca52bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:59:35.268 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:35.267+0000 7f7416fd3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:59:35.478 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:35.477+0000 7feca52bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:59:35.531 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:35.530+0000 7f7416fd3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:59:35.696 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:35.695+0000 7feca52bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:59:35.751 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:35.750+0000 7f7416fd3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:59:35.842 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:35.841+0000 7feca52bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:59:35.899 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:35.898+0000 7f7416fd3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:59:35.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:35.915+0000 7f9f952c1700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 msgr2=0x7f9f7406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3602430955 2023-10-16T09:59:35.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:35.915+0000 7f9f952c1700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 0x7f9f7406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T09:59:35.954 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:35.953+0000 7feca52bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:59:36.013 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:36.011+0000 7f7416fd3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:59:36.106 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:36.105+0000 7feca52bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:59:36.166 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:36.165+0000 7f7416fd3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:59:36.367 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:36.366+0000 7feca52bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:36.428 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:36.427+0000 7f7416fd3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:36.525 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:36.524+0000 7feca52bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:59:36.589 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:36.587+0000 7f7416fd3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:59:37.044 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:37.043+0000 7feca52bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:59:37.114 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.111+0000 7f7416fd3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:59:37.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:37.112+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 34) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9f8c087120 con 0x7f9f900cdd80 2023-10-16T09:59:37.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:37.135+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mgrmap(e 35) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f9f8c053f70 con 0x7f9f900cdd80 2023-10-16T09:59:37.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:37.136+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 msgr2=0x7f9f7406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:37.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:37.136+0000 7f9f867fc700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/3602430955,v1:172.21.15.181:6833/3602430955] conn(0x7f9f7406b070 0x7f9f7406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:37.280 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f7389fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.281 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f7389fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.281 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f7389fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.281 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f7389fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.281 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f7389fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.282 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f738cfb4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.282 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f738cfb4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.282 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f738cfb4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.282 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f738cfb4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:37.282 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:37.279+0000 7f738cfb4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:38.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.140+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mon.1 v2:172.21.15.190:3300/0 9 ==== mgrmap(e 36) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f9f8c054840 con 0x7f9f900cdd80 2023-10-16T09:59:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.140+0000 7f9f867fc700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f9f740279c0 0x7f9f74027da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.140+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 --> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f74071070 con 0x7f9f740279c0 2023-10-16T09:59:38.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.141+0000 7f9f952c1700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f9f740279c0 0x7f9f74027da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.142+0000 7f9f952c1700 1 --2- 172.21.15.181:0/895237394 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f9f740279c0 0x7f9f74027da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9f80001610 tx=0x7f9f800015a0 comp rx=0 tx=0).ready entity=mgr.6657 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:38.152 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:38.151+0000 7f7394002700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-16T09:59:38.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.151+0000 7f9f867fc700 1 -- 172.21.15.181:0/895237394 <== mgr.6657 v2:172.21.15.181:6832/420719762 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f9f74071070 con 0x7f9f740279c0 2023-10-16T09:59:38.155 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-16T09:59:38.159 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T09:59:38.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:38.160 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T09:59:38.181 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T09:59:38.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.347+0000 7f00687a0700 1 Processor -- start 2023-10-16T09:59:38.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.347+0000 7f00687a0700 1 -- start start 2023-10-16T09:59:38.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f00687a0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00600cc6e0 0x7f0060071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f00687a0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f00600ce680 0x7f00600ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f00687a0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00600ccb60 con 0x7f00600b2010 2023-10-16T09:59:38.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f00687a0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0060072120 con 0x7f00600ce680 2023-10-16T09:59:38.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f00687a0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00600ce1f0 con 0x7f00600cc6e0 2023-10-16T09:59:38.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006653c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f00600b2010 0x7f00600b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53342/0 (socket says 172.21.15.181:53342) 2023-10-16T09:59:38.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006653c700 1 -- 172.21.15.181:0/3196994950 learned_addr learned my addr 172.21.15.181:0/3196994950 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:38.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046998376 0 0) 0x7f005c001110 con 0x7f00600b2010 2023-10-16T09:59:38.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f004c0035b0 con 0x7f00600b2010 2023-10-16T09:59:38.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f0066d3d700 1 --2- 172.21.15.181:0/3196994950 >> v2:172.21.15.190:3300/0 conn(0x7f00600ce680 0x7f00600ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f0065d3b700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00600cc6e0 0x7f0060071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189179655 0 0) 0x7f004c0035b0 con 0x7f00600b2010 2023-10-16T09:59:38.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00600cc6e0 msgr2=0x7f0060071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:38.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00600cc6e0 0x7f0060071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:38.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 >> v2:172.21.15.190:3300/0 conn(0x7f00600ce680 msgr2=0x7f00600ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:38.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 --2- 172.21.15.181:0/3196994950 >> v2:172.21.15.190:3300/0 conn(0x7f00600ce680 0x7f00600ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:38.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600ccda0 con 0x7f00600b2010 2023-10-16T09:59:38.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f005c001800 con 0x7f00600b2010 2023-10-16T09:59:38.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 >> v1:172.21.15.181:6789/0 conn(0x7f00600b2010 legacy=0x7f00600b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:38.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f004c003a00 0x7f004c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c004850 0x7f004c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c005210 0x7f004c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0060072120 con 0x7f004c003a00 2023-10-16T09:59:38.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00600ce1f0 con 0x7f004c005210 2023-10-16T09:59:38.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c00c050 con 0x7f004c004850 2023-10-16T09:59:38.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f0065d3b700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00600cc6e0 0x7f0060071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:38.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.348+0000 7f0066d3d700 1 --2- 172.21.15.181:0/3196994950 >> v2:172.21.15.190:3300/0 conn(0x7f00600ce680 0x7f00600ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:38.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006653c700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f004c003a00 0x7f004c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f0065d3b700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c004850 0x7f004c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f0066d3d700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c005210 0x7f004c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006653c700 1 -- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c004850 msgr2=0x7f004c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006653c700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c004850 0x7f004c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006653c700 1 -- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c005210 msgr2=0x7f004c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006653c700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c005210 0x7f004c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006653c700 1 -- 172.21.15.181:0/3196994950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0035b0 con 0x7f004c003a00 2023-10-16T09:59:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006653c700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f004c003a00 0x7f004c003e20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c320 tx=0x7f004c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e074c452a245f84 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c00ce20 con 0x7f004c003a00 2023-10-16T09:59:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c005960 con 0x7f004c003a00 2023-10-16T09:59:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f006553a700 1 -- 172.21.15.181:0/3196994950 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c005d10 con 0x7f004c003a00 2023-10-16T09:59:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f00687a0700 1 -- 172.21.15.181:0/3196994950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f004c003a00 msgr2=0x7f004c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.349+0000 7f00687a0700 1 --2- 172.21.15.181:0/3196994950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f004c003a00 0x7f004c003e20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c320 tx=0x7f004c005880 comp rx=0 tx=0).stop 2023-10-16T09:59:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.350+0000 7f0066d3d700 1 -- 172.21.15.181:0/3196994950 reap_dead start 2023-10-16T09:59:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.350+0000 7f00687a0700 1 -- 172.21.15.181:0/3196994950 shutdown_connections 2023-10-16T09:59:38.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.350+0000 7f00687a0700 1 -- 172.21.15.181:0/3196994950 >> 172.21.15.181:0/3196994950 conn(0x7f00600cd010 msgr2=0x7f00600cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:38.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.350+0000 7f00687a0700 1 -- 172.21.15.181:0/3196994950 shutdown_connections 2023-10-16T09:59:38.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.350+0000 7f00687a0700 1 -- 172.21.15.181:0/3196994950 wait complete. 2023-10-16T09:59:38.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.351+0000 7f00687a0700 1 Processor -- start 2023-10-16T09:59:38.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.351+0000 7f00687a0700 1 -- start start 2023-10-16T09:59:38.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.351+0000 7f00687a0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c003a00 0x7f006014b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f00687a0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00600cf0e0 0x7f00600cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f00687a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00601f34a0 0x7f00601f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f00687a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00600cfaa0 con 0x7f00601f34a0 2023-10-16T09:59:38.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f00687a0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00600c83c0 con 0x7f00600cf0e0 2023-10-16T09:59:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f00687a0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00601f41d0 con 0x7f004c003a00 2023-10-16T09:59:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00601f34a0 0x7f00601f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00601f34a0 0x7f00601f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42280/0 (socket says 172.21.15.181:42280) 2023-10-16T09:59:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 -- 172.21.15.181:0/3735405007 learned_addr learned my addr 172.21.15.181:0/3735405007 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0065d3b700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00600cf0e0 0x7f00600cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f006653c700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c003a00 0x7f006014b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 -- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c003a00 msgr2=0x7f006014b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c003a00 0x7f006014b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 -- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00600cf0e0 msgr2=0x7f00600cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00600cf0e0 0x7f00600cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:38.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 -- 172.21.15.181:0/3735405007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600c8680 con 0x7f00601f34a0 2023-10-16T09:59:38.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0065d3b700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00600cf0e0 0x7f00600cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:38.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0066d3d700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00601f34a0 0x7f00601f3920 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0060071b70 tx=0x7f0058004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:38.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0056ffd700 1 -- 172.21.15.181:0/3735405007 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058007b80 con 0x7f00601f34a0 2023-10-16T09:59:38.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0056ffd700 1 -- 172.21.15.181:0/3735405007 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005800f050 con 0x7f00601f34a0 2023-10-16T09:59:38.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00600c8960 con 0x7f00601f34a0 2023-10-16T09:59:38.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f0056ffd700 1 -- 172.21.15.181:0/3735405007 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058014410 con 0x7f00601f34a0 2023-10-16T09:59:38.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.352+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00601f3e60 con 0x7f00601f34a0 2023-10-16T09:59:38.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.353+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00600c8b50 con 0x7f00601f34a0 2023-10-16T09:59:38.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.354+0000 7f0056ffd700 1 -- 172.21.15.181:0/3735405007 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 36) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0058014570 con 0x7f00601f34a0 2023-10-16T09:59:38.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.354+0000 7f0056ffd700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f004406ada0 0x7f004406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:38.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.354+0000 7f0056ffd700 1 -- 172.21.15.181:0/3735405007 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f0058089930 con 0x7f00601f34a0 2023-10-16T09:59:38.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.354+0000 7f006653c700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f004406ada0 0x7f004406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:38.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.354+0000 7f006653c700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f004406ada0 0x7f004406d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f005c008fd0 tx=0x7f005c00a000 comp rx=0 tx=0).ready entity=mgr.6657 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:38.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.358+0000 7f0056ffd700 1 -- 172.21.15.181:0/3735405007 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0058054a20 con 0x7f00601f34a0 2023-10-16T09:59:38.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:38.606+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f00600b6510 con 0x7f00601f34a0 2023-10-16T09:59:39.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.142+0000 7f0056ffd700 1 -- 172.21.15.181:0/3735405007 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v37) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0058003cd0 con 0x7f00601f34a0 2023-10-16T09:59:39.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f004406ada0 msgr2=0x7f004406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f004406ada0 0x7f004406d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f005c008fd0 tx=0x7f005c00a000 comp rx=0 tx=0).stop 2023-10-16T09:59:39.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00601f34a0 msgr2=0x7f00601f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00601f34a0 0x7f00601f3920 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0060071b70 tx=0x7f0058004970 comp rx=0 tx=0).stop 2023-10-16T09:59:39.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 shutdown_connections 2023-10-16T09:59:39.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f004406ada0 0x7f004406d260 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f005c008fd0 tx=0x7f005c00a000 comp rx=0 tx=0).stop 2023-10-16T09:59:39.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c003a00 0x7f006014b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00601f34a0 0x7f00601f3920 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 --2- 172.21.15.181:0/3735405007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00600cf0e0 0x7f00600cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.146+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 >> 172.21.15.181:0/3735405007 conn(0x7f00600cd010 msgr2=0x7f00600b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:39.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.147+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 shutdown_connections 2023-10-16T09:59:39.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.147+0000 7f00687a0700 1 -- 172.21.15.181:0/3735405007 wait complete. 2023-10-16T09:59:39.163 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:59:39.164 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:39.202 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:39.245 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:59:39.245 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:39.246 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:39.265 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:39.266 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:39.291 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:39.289+0000 7f60d4a83200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:59:39.296 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:39.295+0000 7f97cc55b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T09:59:39.308 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:39.359 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:39.357+0000 7f60d4a83200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:39.362 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:39.360+0000 7f97cc55b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:39.434 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:39.433+0000 7f97cc55b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:59:39.459 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:39.433+0000 7f60d4a83200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T09:59:39.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.471+0000 7fbb286ca700 1 Processor -- start 2023-10-16T09:59:39.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.471+0000 7fbb286ca700 1 -- start start 2023-10-16T09:59:39.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb286ca700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbb2015e2f0 0x7fbb2015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb286ca700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb200666f0 0x7fbb200cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb286ca700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb200ca540 con 0x7fbb200b23e0 2023-10-16T09:59:39.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb286ca700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2015f230 con 0x7fbb2015e2f0 2023-10-16T09:59:39.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb286ca700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2015ef50 con 0x7fbb200666f0 2023-10-16T09:59:39.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb26466700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbb2015e2f0 0x7fbb2015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb25c65700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbb200b23e0 0x7fbb200cdaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:37358/0 (socket says 172.21.15.181:37358) 2023-10-16T09:59:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb25c65700 1 -- 172.21.15.181:0/1990693485 learned_addr learned my addr 172.21.15.181:0/1990693485 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.472+0000 7fbb26c67700 1 --2- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb200666f0 0x7fbb200cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb26466700 1 -- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb200666f0 msgr2=0x7fbb200cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:39.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb26466700 1 --2- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb200666f0 0x7fbb200cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb26466700 1 -- 172.21.15.181:0/1990693485 >> v1:172.21.15.190:6789/0 conn(0x7fbb200b23e0 legacy=0x7fbb200cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb26466700 1 -- 172.21.15.181:0/1990693485 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb200ca770 con 0x7fbb2015e2f0 2023-10-16T09:59:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb25464700 1 -- 172.21.15.181:0/1990693485 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313924645 0 0) 0x7fbb200ca540 con 0x7fbb200b23e0 2023-10-16T09:59:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb26466700 1 --2- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb2015e2f0 0x7fbb2015e6d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009840 tx=0x7fbb1c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c100c97cb15849f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:39.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb25464700 1 -- 172.21.15.181:0/1990693485 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c011830 con 0x7fbb2015e2f0 2023-10-16T09:59:39.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb25464700 1 -- 172.21.15.181:0/1990693485 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c011990 con 0x7fbb2015e2f0 2023-10-16T09:59:39.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.473+0000 7fbb25464700 1 -- 172.21.15.181:0/1990693485 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c011d20 con 0x7fbb2015e2f0 2023-10-16T09:59:39.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 -- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb2015e2f0 msgr2=0x7fbb2015e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb2015e2f0 0x7fbb2015e6d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009840 tx=0x7fbb1c009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:39.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 -- 172.21.15.181:0/1990693485 shutdown_connections 2023-10-16T09:59:39.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb200666f0 0x7fbb200cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1990693485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb2015e2f0 0x7fbb2015e6d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 -- 172.21.15.181:0/1990693485 >> 172.21.15.181:0/1990693485 conn(0x7fbb200b16e0 msgr2=0x7fbb200b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:39.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 -- 172.21.15.181:0/1990693485 shutdown_connections 2023-10-16T09:59:39.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.474+0000 7fbb286ca700 1 -- 172.21.15.181:0/1990693485 wait complete. 2023-10-16T09:59:39.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.475+0000 7fbb286ca700 1 Processor -- start 2023-10-16T09:59:39.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.475+0000 7fbb286ca700 1 -- start start 2023-10-16T09:59:39.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb286ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb200666f0 0x7fbb2015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb286ca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb200b23e0 0x7fbb200c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb286ca700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb2015e2f0 0x7fbb200c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb286ca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200c9ca0 con 0x7fbb200666f0 2023-10-16T09:59:39.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb286ca700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200c6c30 con 0x7fbb200b23e0 2023-10-16T09:59:39.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb286ca700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200c4610 con 0x7fbb2015e2f0 2023-10-16T09:59:39.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb26466700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb200666f0 0x7fbb2015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb26466700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb200666f0 0x7fbb2015ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42342/0 (socket says 172.21.15.181:42342) 2023-10-16T09:59:39.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb26466700 1 -- 172.21.15.181:0/1269907184 learned_addr learned my addr 172.21.15.181:0/1269907184 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb26c67700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb2015e2f0 0x7fbb200c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.476+0000 7fbb25c65700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb200b23e0 0x7fbb200c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb26c67700 1 -- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb200b23e0 msgr2=0x7fbb200c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb26c67700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb200b23e0 0x7fbb200c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb26c67700 1 -- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb200666f0 msgr2=0x7fbb2015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb26c67700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb200666f0 0x7fbb2015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb26c67700 1 -- 172.21.15.181:0/1269907184 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0094a0 con 0x7fbb2015e2f0 2023-10-16T09:59:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb26466700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb200666f0 0x7fbb2015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb26c67700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb2015e2f0 0x7fbb200c9760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800eba0 tx=0x7fbb1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:39.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb16ffd700 1 -- 172.21.15.181:0/1269907184 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18009a80 con 0x7fbb2015e2f0 2023-10-16T09:59:39.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb16ffd700 1 -- 172.21.15.181:0/1269907184 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1800f040 con 0x7fbb2015e2f0 2023-10-16T09:59:39.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb200c6e80 con 0x7fbb2015e2f0 2023-10-16T09:59:39.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb16ffd700 1 -- 172.21.15.181:0/1269907184 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800cc40 con 0x7fbb2015e2f0 2023-10-16T09:59:39.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.477+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb200c7440 con 0x7fbb2015e2f0 2023-10-16T09:59:39.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.478+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb200c7660 con 0x7fbb2015e2f0 2023-10-16T09:59:39.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.478+0000 7fbb16ffd700 1 -- 172.21.15.181:0/1269907184 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 37) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbb18014480 con 0x7fbb2015e2f0 2023-10-16T09:59:39.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.479+0000 7fbb16ffd700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 0x7fbb0406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.479+0000 7fbb26466700 1 -- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 msgr2=0x7fbb0406d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:39.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.479+0000 7fbb16ffd700 1 -- 172.21.15.181:0/1269907184 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fbb18089ea0 con 0x7fbb2015e2f0 2023-10-16T09:59:39.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.479+0000 7fbb26466700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 0x7fbb0406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:39.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.483+0000 7fbb16ffd700 1 -- 172.21.15.181:0/1269907184 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbb18054f90 con 0x7fbb2015e2f0 2023-10-16T09:59:39.513 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:39.512+0000 7f97cc55b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:59:39.517 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:39.516+0000 7f60d4a83200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T09:59:39.654 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:39.652+0000 7f97cc55b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:59:39.663 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:39.661+0000 7f60d4a83200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T09:59:39.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.680+0000 7fbb26466700 1 -- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 msgr2=0x7fbb0406d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:39.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.680+0000 7fbb26466700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 0x7fbb0406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:39.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.687+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb20067d20 con 0x7fbb2015e2f0 2023-10-16T09:59:39.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.689+0000 7fbb16ffd700 1 -- 172.21.15.181:0/1269907184 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17754 (secure 0 0 0) 0x7fbb18018020 con 0x7fbb2015e2f0 2023-10-16T09:59:39.691 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:39.691 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":77,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:37.126359+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:21.812039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:22.200991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:6832/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/1081249734":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2658775934":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2227004797":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3531530097":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2146555244":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3441419577":"2023-10-17T09:59:37.126305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 msgr2=0x7fbb0406d210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 0x7fbb0406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb2015e2f0 msgr2=0x7fbb200c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb2015e2f0 0x7fbb200c9760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800eba0 tx=0x7fbb1800eeb0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 shutdown_connections 2023-10-16T09:59:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fbb0406ad50 0x7fbb0406d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb2015e2f0 0x7fbb200c9760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb200666f0 0x7fbb2015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 --2- 172.21.15.181:0/1269907184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb200b23e0 0x7fbb200c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 >> 172.21.15.181:0/1269907184 conn(0x7fbb200b16e0 msgr2=0x7fbb20158890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 shutdown_connections 2023-10-16T09:59:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.691+0000 7fbb286ca700 1 -- 172.21.15.181:0/1269907184 wait complete. 2023-10-16T09:59:39.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:59:21.812039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:59:22.200991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:39.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:39.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.873+0000 7f27154c0700 1 Processor -- start 2023-10-16T09:59:39.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.873+0000 7f27154c0700 1 -- start start 2023-10-16T09:59:39.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.873+0000 7f27154c0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2710159e10 0x7f27100ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.873+0000 7f27154c0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271006e4b0 con 0x7f271006bef0 2023-10-16T09:59:39.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.873+0000 7f27154c0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271006ed50 con 0x7f271015a1f0 2023-10-16T09:59:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.873+0000 7f27154c0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100ceea0 con 0x7f2710159e10 2023-10-16T09:59:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2710159e10 0x7f27100ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f271006bef0 0x7f27100caba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53448/0 (socket says 172.21.15.181:53448) 2023-10-16T09:59:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270e7fc700 1 -- 172.21.15.181:0/3449767575 learned_addr learned my addr 172.21.15.181:0/3449767575 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:39.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270effd700 1 -- 172.21.15.181:0/3449767575 >> v1:172.21.15.190:6789/0 conn(0x7f271015a1f0 legacy=0x7f271015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270dffb700 1 -- 172.21.15.181:0/3449767575 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717111794 0 0) 0x7f271006e4b0 con 0x7f271006bef0 2023-10-16T09:59:39.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270effd700 1 -- 172.21.15.181:0/3449767575 >> v1:172.21.15.181:6789/0 conn(0x7f271006bef0 legacy=0x7f27100caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270effd700 1 -- 172.21.15.181:0/3449767575 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271006e700 con 0x7f2710159e10 2023-10-16T09:59:39.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270effd700 1 --2- 172.21.15.181:0/3449767575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2710159e10 0x7f27100ce5a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f27000097c0 tx=0x7f2700009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689e7aded6b1135f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:39.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270dffb700 1 -- 172.21.15.181:0/3449767575 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700004230 con 0x7f2710159e10 2023-10-16T09:59:39.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270dffb700 1 -- 172.21.15.181:0/3449767575 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2700004390 con 0x7f2710159e10 2023-10-16T09:59:39.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.874+0000 7f270dffb700 1 -- 172.21.15.181:0/3449767575 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000036a0 con 0x7f2710159e10 2023-10-16T09:59:39.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.875+0000 7f27154c0700 1 -- 172.21.15.181:0/3449767575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2710159e10 msgr2=0x7f27100ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.875+0000 7f27154c0700 1 --2- 172.21.15.181:0/3449767575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2710159e10 0x7f27100ce5a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f27000097c0 tx=0x7f2700009ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.875+0000 7f27154c0700 1 -- 172.21.15.181:0/3449767575 shutdown_connections 2023-10-16T09:59:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.875+0000 7f27154c0700 1 --2- 172.21.15.181:0/3449767575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2710159e10 0x7f27100ce5a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.875+0000 7f27154c0700 1 -- 172.21.15.181:0/3449767575 >> 172.21.15.181:0/3449767575 conn(0x7f27100cc730 msgr2=0x7f27100ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.875+0000 7f27154c0700 1 -- 172.21.15.181:0/3449767575 shutdown_connections 2023-10-16T09:59:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.875+0000 7f27154c0700 1 -- 172.21.15.181:0/3449767575 wait complete. 2023-10-16T09:59:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.877+0000 7f27154c0700 1 Processor -- start 2023-10-16T09:59:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.877+0000 7f27154c0700 1 -- start start 2023-10-16T09:59:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f27154c0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271006bef0 0x7f27100cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f27154c0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710159e10 0x7f2710148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f27154c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271015a1f0 0x7f2710148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f27154c0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27101f0bd0 con 0x7f271015a1f0 2023-10-16T09:59:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f27154c0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710149180 con 0x7f2710159e10 2023-10-16T09:59:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f27154c0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27101effc0 con 0x7f271006bef0 2023-10-16T09:59:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271015a1f0 0x7f2710148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271015a1f0 0x7f2710148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42380/0 (socket says 172.21.15.181:42380) 2023-10-16T09:59:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271006bef0 0x7f27100cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 -- 172.21.15.181:0/1490772216 learned_addr learned my addr 172.21.15.181:0/1490772216 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270e7fc700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710159e10 0x7f2710148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 -- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271006bef0 msgr2=0x7f27100cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271006bef0 0x7f27100cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 -- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710159e10 msgr2=0x7f2710148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710159e10 0x7f2710148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270f7fe700 1 -- 172.21.15.181:0/1490772216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000094a0 con 0x7f271015a1f0 2023-10-16T09:59:39.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270e7fc700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710159e10 0x7f2710148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.878+0000 7f270effd700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271006bef0 0x7f27100cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.879+0000 7f270f7fe700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271015a1f0 0x7f2710148c40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f270800c910 tx=0x7f270800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.879+0000 7f26f77fe700 1 -- 172.21.15.181:0/1490772216 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708003e30 con 0x7f271015a1f0 2023-10-16T09:59:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.879+0000 7f26f77fe700 1 -- 172.21.15.181:0/1490772216 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2708005070 con 0x7f271015a1f0 2023-10-16T09:59:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.879+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27101f02a0 con 0x7f271015a1f0 2023-10-16T09:59:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.879+0000 7f26f77fe700 1 -- 172.21.15.181:0/1490772216 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708005420 con 0x7f271015a1f0 2023-10-16T09:59:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.879+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27101f0860 con 0x7f271015a1f0 2023-10-16T09:59:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.880+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2710149780 con 0x7f271015a1f0 2023-10-16T09:59:39.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.881+0000 7f26f77fe700 1 -- 172.21.15.181:0/1490772216 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 37) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f2708005580 con 0x7f271015a1f0 2023-10-16T09:59:39.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.881+0000 7f26f77fe700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 0x7f26fc06d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:39.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.881+0000 7f270effd700 1 -- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 msgr2=0x7f26fc06d240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:39.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.881+0000 7f26f77fe700 1 -- 172.21.15.181:0/1490772216 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f2708088be0 con 0x7f271015a1f0 2023-10-16T09:59:39.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.881+0000 7f270effd700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 0x7f26fc06d240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:39.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:39.886+0000 7f26f77fe700 1 -- 172.21.15.181:0/1490772216 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2708053cd0 con 0x7f271015a1f0 2023-10-16T09:59:39.979 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:39.978+0000 7f97cc55b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:59:40.008 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:40.006+0000 7f60d4a83200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T09:59:40.046 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:40.049 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T09:59:40.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.082+0000 7f270effd700 1 -- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 msgr2=0x7f26fc06d240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:40.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.082+0000 7f270effd700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 0x7f26fc06d240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:40.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.088+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f271014d820 con 0x7f271015a1f0 2023-10-16T09:59:40.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.088+0000 7f26f77fe700 1 -- 172.21.15.181:0/1490772216 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2708017070 con 0x7f271015a1f0 2023-10-16T09:59:40.090 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:40.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 msgr2=0x7f26fc06d240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:40.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 0x7f26fc06d240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271015a1f0 msgr2=0x7f2710148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271015a1f0 0x7f2710148c40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f270800c910 tx=0x7f270800cc20 comp rx=0 tx=0).stop 2023-10-16T09:59:40.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 shutdown_connections 2023-10-16T09:59:40.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f26fc06ad80 0x7f26fc06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271006bef0 0x7f27100cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f271015a1f0 0x7f2710148c40 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 --2- 172.21.15.181:0/1490772216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2710159e10 0x7f2710148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 >> 172.21.15.181:0/1490772216 conn(0x7f27100cc730 msgr2=0x7f271014ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:40.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 shutdown_connections 2023-10-16T09:59:40.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.091+0000 7f27154c0700 1 -- 172.21.15.181:0/1490772216 wait complete. 2023-10-16T09:59:40.096 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:40.095+0000 7f97cc55b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:59:40.106 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:59:40.129 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:40.128+0000 7f60d4a83200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T09:59:40.175 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:40.174+0000 7f97cc55b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:59:40.209 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:40.208+0000 7f60d4a83200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T09:59:40.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:40.253+0000 7f97cc55b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:59:40.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.255+0000 7f94def9a700 1 Processor -- start 2023-10-16T09:59:40.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.255+0000 7f94def9a700 1 -- start start 2023-10-16T09:59:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94def9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d815f150 0x7f94d80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94def9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94d80cd170 0x7f94d80b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94def9a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d80b2ae0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94def9a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d80ce970 con 0x7f94d80cd170 2023-10-16T09:59:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94def9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b2dc0 con 0x7f94d815f150 2023-10-16T09:59:40.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94d7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f94d80ce4f0 0x7f94d80b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53466/0 (socket says 172.21.15.181:53466) 2023-10-16T09:59:40.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94d7fff700 1 -- 172.21.15.181:0/1489631623 learned_addr learned my addr 172.21.15.181:0/1489631623 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:40.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94dcd36700 1 --2- 172.21.15.181:0/1489631623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d815f150 0x7f94d80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94dd537700 1 --2- 172.21.15.181:0/1489631623 >> v2:172.21.15.190:3300/0 conn(0x7f94d80cd170 0x7f94d80b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94d77fe700 1 -- 172.21.15.181:0/1489631623 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613422500 0 0) 0x7f94d80b2ae0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94d77fe700 1 -- 172.21.15.181:0/1489631623 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c80035b0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94dcd36700 1 -- 172.21.15.181:0/1489631623 >> v2:172.21.15.190:3300/0 conn(0x7f94d80cd170 msgr2=0x7f94d80b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94dcd36700 1 --2- 172.21.15.181:0/1489631623 >> v2:172.21.15.190:3300/0 conn(0x7f94d80cd170 0x7f94d80b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94dcd36700 1 -- 172.21.15.181:0/1489631623 >> v1:172.21.15.181:6789/0 conn(0x7f94d80ce4f0 legacy=0x7f94d80b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94dcd36700 1 -- 172.21.15.181:0/1489631623 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80cf9d0 con 0x7f94d815f150 2023-10-16T09:59:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.256+0000 7f94dd537700 1 --2- 172.21.15.181:0/1489631623 >> v2:172.21.15.190:3300/0 conn(0x7f94d80cd170 0x7f94d80b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94dcd36700 1 --2- 172.21.15.181:0/1489631623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d815f150 0x7f94d80cdfb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009840 tx=0x7f94cc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80144c9469102db3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94d77fe700 1 -- 172.21.15.181:0/1489631623 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc011830 con 0x7f94d815f150 2023-10-16T09:59:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94d77fe700 1 -- 172.21.15.181:0/1489631623 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc011990 con 0x7f94d815f150 2023-10-16T09:59:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94d77fe700 1 -- 172.21.15.181:0/1489631623 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc011d20 con 0x7f94d815f150 2023-10-16T09:59:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 -- 172.21.15.181:0/1489631623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d815f150 msgr2=0x7f94d80cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 --2- 172.21.15.181:0/1489631623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d815f150 0x7f94d80cdfb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009840 tx=0x7f94cc009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 -- 172.21.15.181:0/1489631623 shutdown_connections 2023-10-16T09:59:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 --2- 172.21.15.181:0/1489631623 >> v2:172.21.15.190:3300/0 conn(0x7f94d80cd170 0x7f94d80b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 --2- 172.21.15.181:0/1489631623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d815f150 0x7f94d80cdfb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 -- 172.21.15.181:0/1489631623 >> 172.21.15.181:0/1489631623 conn(0x7f94d809a930 msgr2=0x7f94d80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 -- 172.21.15.181:0/1489631623 shutdown_connections 2023-10-16T09:59:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.257+0000 7f94def9a700 1 -- 172.21.15.181:0/1489631623 wait complete. 2023-10-16T09:59:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 Processor -- start 2023-10-16T09:59:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 -- start start 2023-10-16T09:59:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d80cd170 0x7f94d8151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d80ce4f0 0x7f94d806c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94d815f150 0x7f94d806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d806da40 con 0x7f94d815f150 2023-10-16T09:59:40.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d806bb50 con 0x7f94d80cd170 2023-10-16T09:59:40.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.259+0000 7f94def9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d806d480 con 0x7f94d80ce4f0 2023-10-16T09:59:40.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94dd537700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94d815f150 0x7f94d806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94dcd36700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d80cd170 0x7f94d8151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d7fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d80ce4f0 0x7f94d806c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94dd537700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94d815f150 0x7f94d806cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42396/0 (socket says 172.21.15.181:42396) 2023-10-16T09:59:40.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94dcd36700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d80cd170 0x7f94d8151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:47442/0 (socket says 172.21.15.181:47442) 2023-10-16T09:59:40.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94dd537700 1 -- 172.21.15.181:0/1965362729 learned_addr learned my addr 172.21.15.181:0/1965362729 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:40.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d7fff700 1 -- 172.21.15.181:0/1965362729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d80cd170 msgr2=0x7f94d8151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d7fff700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d80cd170 0x7f94d8151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d7fff700 1 -- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94d815f150 msgr2=0x7f94d806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d7fff700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94d815f150 0x7f94d806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d7fff700 1 -- 172.21.15.181:0/1965362729 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0009a70 con 0x7f94d80ce4f0 2023-10-16T09:59:40.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d7fff700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d80ce4f0 0x7f94d806c310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c450 tx=0x7f94c400c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:40.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d57fa700 1 -- 172.21.15.181:0/1965362729 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4005d70 con 0x7f94d80ce4f0 2023-10-16T09:59:40.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d57fa700 1 -- 172.21.15.181:0/1965362729 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c400ce20 con 0x7f94d80ce4f0 2023-10-16T09:59:40.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94cc0094a0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94d57fa700 1 -- 172.21.15.181:0/1965362729 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c40072f0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.260+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d806bdf0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.262+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d806b9b0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.262+0000 7f94d57fa700 1 -- 172.21.15.181:0/1965362729 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f94c4007450 con 0x7f94d80ce4f0 2023-10-16T09:59:40.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.262+0000 7f94d57fa700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 0x7f94bc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.262+0000 7f94dcd36700 1 -- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 msgr2=0x7f94bc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:40.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.262+0000 7f94d57fa700 1 -- 172.21.15.181:0/1965362729 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f94c4088d70 con 0x7f94d80ce4f0 2023-10-16T09:59:40.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.262+0000 7f94dcd36700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 0x7f94bc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:40.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.267+0000 7f94d57fa700 1 -- 172.21.15.181:0/1965362729 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f94d806b9b0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.295 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:40.294+0000 7f60d4a83200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T09:59:40.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.463+0000 7f94dcd36700 1 -- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 msgr2=0x7f94bc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:40.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.463+0000 7f94dcd36700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 0x7f94bc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:40.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.466+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94d806b9b0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.467+0000 7f94d57fa700 1 -- 172.21.15.181:0/1965362729 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94d806b9b0 con 0x7f94d80ce4f0 2023-10-16T09:59:40.469 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:59:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 msgr2=0x7f94bc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 0x7f94bc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d80ce4f0 msgr2=0x7f94d806c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d80ce4f0 0x7f94d806c310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c450 tx=0x7f94c400c760 comp rx=0 tx=0).stop 2023-10-16T09:59:40.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 shutdown_connections 2023-10-16T09:59:40.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f94bc06b020 0x7f94bc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94d815f150 0x7f94d806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94d80cd170 0x7f94d8151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 --2- 172.21.15.181:0/1965362729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94d80ce4f0 0x7f94d806c310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 >> 172.21.15.181:0/1965362729 conn(0x7f94d809a930 msgr2=0x7f94d80d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:40.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 shutdown_connections 2023-10-16T09:59:40.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.469+0000 7f94def9a700 1 -- 172.21.15.181:0/1965362729 wait complete. 2023-10-16T09:59:40.483 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:59:40.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.607+0000 7ff7dfe86700 1 Processor -- start 2023-10-16T09:59:40.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.607+0000 7ff7dfe86700 1 -- start start 2023-10-16T09:59:40.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7dfe86700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7d815f150 0x7ff7d80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7dfe86700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80ce4f0 0x7ff7d80b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7dfe86700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d80b2ae0 con 0x7ff7d80b14a0 2023-10-16T09:59:40.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7dfe86700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d80ce970 con 0x7ff7d815f150 2023-10-16T09:59:40.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7dfe86700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d80b2dc0 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7ddc22700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7d815f150 0x7ff7d80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7ddc22700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7d815f150 0x7ff7d80cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42422/0 (socket says 172.21.15.181:42422) 2023-10-16T09:59:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.608+0000 7ff7ddc22700 1 -- 172.21.15.181:0/2557141822 learned_addr learned my addr 172.21.15.181:0/2557141822 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7dd421700 1 --2- 172.21.15.181:0/2557141822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80ce4f0 0x7ff7d80b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7ddc22700 1 -- 172.21.15.181:0/2557141822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80ce4f0 msgr2=0x7ff7d80b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7ddc22700 1 --2- 172.21.15.181:0/2557141822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80ce4f0 0x7ff7d80b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7ddc22700 1 -- 172.21.15.181:0/2557141822 >> v1:172.21.15.190:6789/0 conn(0x7ff7d80b14a0 legacy=0x7ff7d80b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7dcc20700 1 -- 172.21.15.181:0/2557141822 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070689218 0 0) 0x7ff7d80b2ae0 con 0x7ff7d80b14a0 2023-10-16T09:59:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7ddc22700 1 -- 172.21.15.181:0/2557141822 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d80cf9d0 con 0x7ff7d815f150 2023-10-16T09:59:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7ddc22700 1 --2- 172.21.15.181:0/2557141822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d815f150 0x7ff7d80cdfb0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098f0 tx=0x7ff7d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9902147a4b4e678 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7dcc20700 1 -- 172.21.15.181:0/2557141822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4011830 con 0x7ff7d815f150 2023-10-16T09:59:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7dcc20700 1 -- 172.21.15.181:0/2557141822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d4011990 con 0x7ff7d815f150 2023-10-16T09:59:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.609+0000 7ff7dcc20700 1 -- 172.21.15.181:0/2557141822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4011d20 con 0x7ff7d815f150 2023-10-16T09:59:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 -- 172.21.15.181:0/2557141822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d815f150 msgr2=0x7ff7d80cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/2557141822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d815f150 0x7ff7d80cdfb0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098f0 tx=0x7ff7d4009c00 comp rx=0 tx=0).stop 2023-10-16T09:59:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 -- 172.21.15.181:0/2557141822 shutdown_connections 2023-10-16T09:59:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/2557141822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d815f150 0x7ff7d80cdfb0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/2557141822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d80ce4f0 0x7ff7d80b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 -- 172.21.15.181:0/2557141822 >> 172.21.15.181:0/2557141822 conn(0x7ff7d809a930 msgr2=0x7ff7d80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 -- 172.21.15.181:0/2557141822 shutdown_connections 2023-10-16T09:59:40.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.610+0000 7ff7dfe86700 1 -- 172.21.15.181:0/2557141822 wait complete. 2023-10-16T09:59:40.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.611+0000 7ff7dfe86700 1 Processor -- start 2023-10-16T09:59:40.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.611+0000 7ff7dfe86700 1 -- start start 2023-10-16T09:59:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dfe86700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d80b14a0 0x7ff7d815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dfe86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d80ce4f0 0x7ff7d80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dfe86700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d815f150 0x7ff7d814c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dfe86700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8159000 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dfe86700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d815d600 con 0x7ff7d80b14a0 2023-10-16T09:59:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dfe86700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d815c9f0 con 0x7ff7d815f150 2023-10-16T09:59:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d80ce4f0 0x7ff7d80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d80ce4f0 0x7ff7d80c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42430/0 (socket says 172.21.15.181:42430) 2023-10-16T09:59:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 -- 172.21.15.181:0/4115292833 learned_addr learned my addr 172.21.15.181:0/4115292833 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7de423700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d815f150 0x7ff7d814c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7ddc22700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d80b14a0 0x7ff7d815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 -- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d815f150 msgr2=0x7ff7d814c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d815f150 0x7ff7d814c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 -- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d80b14a0 msgr2=0x7ff7d815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d80b14a0 0x7ff7d815da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 -- 172.21.15.181:0/4115292833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009550 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.612+0000 7ff7dd421700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d80ce4f0 0x7ff7d80c7210 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800e9b0 tx=0x7ff7c800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.613+0000 7ff7ce7fc700 1 -- 172.21.15.181:0/4115292833 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8009710 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.613+0000 7ff7ce7fc700 1 -- 172.21.15.181:0/4115292833 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c800ee50 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.613+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d815ccd0 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.613+0000 7ff7ce7fc700 1 -- 172.21.15.181:0/4115292833 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c800ca50 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.613+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d80c7750 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.614+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d815d0f0 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.614+0000 7ff7ce7fc700 1 -- 172.21.15.181:0/4115292833 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7ff7c800cbb0 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.614+0000 7ff7ce7fc700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 0x7ff7bc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.615+0000 7ff7ddc22700 1 -- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 msgr2=0x7ff7bc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.615+0000 7ff7ce7fc700 1 -- 172.21.15.181:0/4115292833 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7ff7c8089a10 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.615+0000 7ff7ddc22700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 0x7ff7bc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:40.625 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:40.624+0000 7f97cc55b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:59:40.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.619+0000 7ff7ce7fc700 1 -- 172.21.15.181:0/4115292833 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff7d815d0f0 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.684 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:40.682+0000 7f60d4a83200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T09:59:40.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.816+0000 7ff7ddc22700 1 -- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 msgr2=0x7ff7bc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:40.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.816+0000 7ff7ddc22700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 0x7ff7bc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:40.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.817+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7d815d0f0 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.818+0000 7ff7ce7fc700 1 -- 172.21.15.181:0/4115292833 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7d815d0f0 con 0x7ff7d80ce4f0 2023-10-16T09:59:40.819 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:59:40.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 msgr2=0x7ff7bc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:40.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 0x7ff7bc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d80ce4f0 msgr2=0x7ff7d80c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d80ce4f0 0x7ff7d80c7210 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800e9b0 tx=0x7ff7c800ecc0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 shutdown_connections 2023-10-16T09:59:40.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff7bc06b020 0x7ff7bc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7d815f150 0x7ff7d814c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7d80b14a0 0x7ff7d815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 --2- 172.21.15.181:0/4115292833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d80ce4f0 0x7ff7d80c7210 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 >> 172.21.15.181:0/4115292833 conn(0x7ff7d809a930 msgr2=0x7ff7d815abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:40.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 shutdown_connections 2023-10-16T09:59:40.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.820+0000 7ff7dfe86700 1 -- 172.21.15.181:0/4115292833 wait complete. 2023-10-16T09:59:40.834 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T09:59:40.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.956+0000 7ff291158700 1 Processor -- start 2023-10-16T09:59:40.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.956+0000 7ff291158700 1 -- start start 2023-10-16T09:59:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.956+0000 7ff291158700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c15dda0 0x7ff28c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.956+0000 7ff291158700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff28c0d2880 0x7ff28c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff291158700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28c0d0940 con 0x7ff28c0d0130 2023-10-16T09:59:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff291158700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28c0d0c20 con 0x7ff28c0d2880 2023-10-16T09:59:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff291158700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c0b2c20 con 0x7ff28c15dda0 2023-10-16T09:59:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff28c0d2880 0x7ff28c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff28c0d0130 0x7ff28c0d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53518/0 (socket says 172.21.15.181:53518) 2023-10-16T09:59:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28a59c700 1 -- 172.21.15.181:0/569744513 learned_addr learned my addr 172.21.15.181:0/569744513 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28b59e700 1 --2- 172.21.15.181:0/569744513 >> v2:172.21.15.181:3300/0 conn(0x7ff28c0d2880 0x7ff28c0b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42442/0 (socket says 172.21.15.181:42442) 2023-10-16T09:59:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28ad9d700 1 --2- 172.21.15.181:0/569744513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c15dda0 0x7ff28c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28b59e700 1 -- 172.21.15.181:0/569744513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c15dda0 msgr2=0x7ff28c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28b59e700 1 --2- 172.21.15.181:0/569744513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c15dda0 0x7ff28c0cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28b59e700 1 -- 172.21.15.181:0/569744513 >> v1:172.21.15.181:6789/0 conn(0x7ff28c0d0130 legacy=0x7ff28c0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28b59e700 1 -- 172.21.15.181:0/569744513 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c0b2f00 con 0x7ff28c0d2880 2023-10-16T09:59:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff289d9b700 1 -- 172.21.15.181:0/569744513 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619292294 0 0) 0x7ff28c0d0940 con 0x7ff28c0d0130 2023-10-16T09:59:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff28b59e700 1 --2- 172.21.15.181:0/569744513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b3700 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff284009d90 tx=0x7ff28400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d628317a7c60c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff289d9b700 1 -- 172.21.15.181:0/569744513 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28400e070 con 0x7ff28c0d2880 2023-10-16T09:59:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff289d9b700 1 -- 172.21.15.181:0/569744513 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28400bb20 con 0x7ff28c0d2880 2023-10-16T09:59:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.957+0000 7ff289d9b700 1 -- 172.21.15.181:0/569744513 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284012430 con 0x7ff28c0d2880 2023-10-16T09:59:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 -- 172.21.15.181:0/569744513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 msgr2=0x7ff28c0b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 --2- 172.21.15.181:0/569744513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b3700 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff284009d90 tx=0x7ff28400b330 comp rx=0 tx=0).stop 2023-10-16T09:59:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 -- 172.21.15.181:0/569744513 shutdown_connections 2023-10-16T09:59:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 --2- 172.21.15.181:0/569744513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b3700 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 --2- 172.21.15.181:0/569744513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c15dda0 0x7ff28c0cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 -- 172.21.15.181:0/569744513 >> 172.21.15.181:0/569744513 conn(0x7ff28c09cb10 msgr2=0x7ff28c0d3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 -- 172.21.15.181:0/569744513 shutdown_connections 2023-10-16T09:59:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.958+0000 7ff291158700 1 -- 172.21.15.181:0/569744513 wait complete. 2023-10-16T09:59:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.959+0000 7ff291158700 1 Processor -- start 2023-10-16T09:59:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.959+0000 7ff291158700 1 -- start start 2023-10-16T09:59:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff291158700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c0d0130 0x7ff28c0c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff291158700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff291158700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff28c15dda0 0x7ff28c0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff291158700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c1f2e90 con 0x7ff28c0d2880 2023-10-16T09:59:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff291158700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c1f7390 con 0x7ff28c15dda0 2023-10-16T09:59:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff291158700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c09d7a0 con 0x7ff28c0d0130 2023-10-16T09:59:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff28a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff28ad9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c0d0130 0x7ff28c0c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff28a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42446/0 (socket says 172.21.15.181:42446) 2023-10-16T09:59:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff28a59c700 1 -- 172.21.15.181:0/3775628607 learned_addr learned my addr 172.21.15.181:0/3775628607 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.960+0000 7ff28b59e700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff28c15dda0 0x7ff28c0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28ad9d700 1 -- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff28c15dda0 msgr2=0x7ff28c0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28ad9d700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff28c15dda0 0x7ff28c0b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28ad9d700 1 -- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 msgr2=0x7ff28c0b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28ad9d700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28ad9d700 1 -- 172.21.15.181:0/3775628607 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284009a70 con 0x7ff28c0d0130 2023-10-16T09:59:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28b59e700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff28c15dda0 0x7ff28c0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28a59c700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff28ad9d700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c0d0130 0x7ff28c0c9dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e870 tx=0x7ff27c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff2737fe700 1 -- 172.21.15.181:0/3775628607 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c013680 con 0x7ff28c0d0130 2023-10-16T09:59:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff2737fe700 1 -- 172.21.15.181:0/3775628607 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c0137e0 con 0x7ff28c0d0130 2023-10-16T09:59:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff2737fe700 1 -- 172.21.15.181:0/3775628607 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c013b50 con 0x7ff28c0d0130 2023-10-16T09:59:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c1f70b0 con 0x7ff28c0d0130 2023-10-16T09:59:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.961+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c1f3360 con 0x7ff28c0d0130 2023-10-16T09:59:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.962+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28c04ebb0 con 0x7ff28c0d0130 2023-10-16T09:59:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.962+0000 7ff2737fe700 1 -- 172.21.15.181:0/3775628607 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7ff27c013cb0 con 0x7ff28c0d0130 2023-10-16T09:59:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.963+0000 7ff2737fe700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 0x7ff27806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.963+0000 7ff28a59c700 1 -- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 msgr2=0x7ff27806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.963+0000 7ff2737fe700 1 -- 172.21.15.181:0/3775628607 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7ff27c089920 con 0x7ff28c0d0130 2023-10-16T09:59:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.963+0000 7ff28a59c700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 0x7ff27806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:40.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:40.967+0000 7ff2737fe700 1 -- 172.21.15.181:0/3775628607 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff27c0545c0 con 0x7ff28c0d0130 2023-10-16T09:59:41.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.164+0000 7ff28a59c700 1 -- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 msgr2=0x7ff27806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:41.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.164+0000 7ff28a59c700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 0x7ff27806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:41.192 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:41.191+0000 7f97cc55b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:59:41.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.216+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff28c0d1f50 con 0x7ff28c0d0130 2023-10-16T09:59:41.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.216+0000 7ff2737fe700 1 -- 172.21.15.181:0/3775628607 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v77) v1 ==== 80+0+478 (secure 0 0 0) 0x7ff27c0211f0 con 0x7ff28c0d0130 2023-10-16T09:59:41.218 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3602430955 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.218 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3602430955 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.219 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1081249734 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.219 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2658775934 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.219 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2227004797 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.219 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3531530097 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.219 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2146555244 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.219 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3441419577 2023-10-17T09:59:37.126305+0000 2023-10-16T09:59:41.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 msgr2=0x7ff27806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:41.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 0x7ff27806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c0d0130 msgr2=0x7ff28c0c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c0d0130 0x7ff28c0c9dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00e870 tx=0x7ff27c00eb80 comp rx=0 tx=0).stop 2023-10-16T09:59:41.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 shutdown_connections 2023-10-16T09:59:41.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7ff27806b020 0x7ff27806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff28c0d0130 0x7ff28c0c9dd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff28c15dda0 0x7ff28c0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 --2- 172.21.15.181:0/3775628607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28c0d2880 0x7ff28c0b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 >> 172.21.15.181:0/3775628607 conn(0x7ff28c09cb10 msgr2=0x7ff28c160870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:41.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 shutdown_connections 2023-10-16T09:59:41.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.219+0000 7ff291158700 1 -- 172.21.15.181:0/3775628607 wait complete. 2023-10-16T09:59:41.223 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T09:59:41.234 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T09:59:41.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:41.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T09:59:41.260 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:41.259+0000 7f97cc55b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:59:41.278 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T09:59:41.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T09:59:41.279 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T09:59:41.279 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:41.277+0000 7f60d4a83200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T09:59:41.335 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:41.335 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:41.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:41.353 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:41.353 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:41.354 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:41.351+0000 7f60d4a83200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T09:59:41.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:59:41.371 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:41.371 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:41.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:41.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:41.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.531+0000 7fb521238700 1 Processor -- start 2023-10-16T09:59:41.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.531+0000 7fb521238700 1 -- start start 2023-10-16T09:59:41.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb521238700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb51c15dc20 0x7fb51c0d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb521238700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0b2f00 con 0x7fb51c0d1e60 2023-10-16T09:59:41.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb521238700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0b31e0 con 0x7fb51c0b4fb0 2023-10-16T09:59:41.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb521238700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0b3b80 con 0x7fb51c15dc20 2023-10-16T09:59:41.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb51a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb51c0d1e60 0x7fb51c0ce4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53532/0 (socket says 172.21.15.181:53532) 2023-10-16T09:59:41.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb51a59c700 1 -- 172.21.15.181:0/1771009627 learned_addr learned my addr 172.21.15.181:0/1771009627 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:41.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb51ad9d700 1 --2- 172.21.15.181:0/1771009627 >> v2:172.21.15.190:3300/0 conn(0x7fb51c15dc20 0x7fb51c0d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb519d9b700 1 -- 172.21.15.181:0/1771009627 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610770822 0 0) 0x7fb51c0b2f00 con 0x7fb51c0d1e60 2023-10-16T09:59:41.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb519d9b700 1 -- 172.21.15.181:0/1771009627 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5040035b0 con 0x7fb51c0d1e60 2023-10-16T09:59:41.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb51ad9d700 1 -- 172.21.15.181:0/1771009627 >> v1:172.21.15.190:6789/0 conn(0x7fb51c0b4fb0 legacy=0x7fb51c0cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb51ad9d700 1 -- 172.21.15.181:0/1771009627 >> v1:172.21.15.181:6789/0 conn(0x7fb51c0d1e60 legacy=0x7fb51c0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb51ad9d700 1 -- 172.21.15.181:0/1771009627 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0b34c0 con 0x7fb51c15dc20 2023-10-16T09:59:41.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.532+0000 7fb51ad9d700 1 --2- 172.21.15.181:0/1771009627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c15dc20 0x7fb51c0d41c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009820 tx=0x7fb50c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2522ce910bc4924e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:41.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb519d9b700 1 -- 172.21.15.181:0/1771009627 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c005e30 con 0x7fb51c15dc20 2023-10-16T09:59:41.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb519d9b700 1 -- 172.21.15.181:0/1771009627 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c006030 con 0x7fb51c15dc20 2023-10-16T09:59:41.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb519d9b700 1 -- 172.21.15.181:0/1771009627 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0063c0 con 0x7fb51c15dc20 2023-10-16T09:59:41.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb521238700 1 -- 172.21.15.181:0/1771009627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c15dc20 msgr2=0x7fb51c0d41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb521238700 1 --2- 172.21.15.181:0/1771009627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c15dc20 0x7fb51c0d41c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009820 tx=0x7fb50c009b30 comp rx=0 tx=0).stop 2023-10-16T09:59:41.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb521238700 1 -- 172.21.15.181:0/1771009627 shutdown_connections 2023-10-16T09:59:41.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb521238700 1 --2- 172.21.15.181:0/1771009627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c15dc20 0x7fb51c0d41c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb521238700 1 -- 172.21.15.181:0/1771009627 >> 172.21.15.181:0/1771009627 conn(0x7fb51c08af90 msgr2=0x7fb51c0d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:41.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb521238700 1 -- 172.21.15.181:0/1771009627 shutdown_connections 2023-10-16T09:59:41.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.533+0000 7fb521238700 1 -- 172.21.15.181:0/1771009627 wait complete. 2023-10-16T09:59:41.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.535+0000 7fb521238700 1 Processor -- start 2023-10-16T09:59:41.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.535+0000 7fb521238700 1 -- start start 2023-10-16T09:59:41.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb51c0b4fb0 0x7fb51c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c0d1e60 0x7fb51c1f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c15dc20 0x7fb51c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0cc590 con 0x7fb51c15dc20 2023-10-16T09:59:41.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c161130 con 0x7fb51c0d1e60 2023-10-16T09:59:41.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c1f35f0 con 0x7fb51c0b4fb0 2023-10-16T09:59:41.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c15dc20 0x7fb51c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c15dc20 0x7fb51c151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42464/0 (socket says 172.21.15.181:42464) 2023-10-16T09:59:41.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 -- 172.21.15.181:0/2140979846 learned_addr learned my addr 172.21.15.181:0/2140979846 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:41.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51ad9d700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb51c0b4fb0 0x7fb51c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51a59c700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c0d1e60 0x7fb51c1f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 -- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb51c0b4fb0 msgr2=0x7fb51c160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb51c0b4fb0 0x7fb51c160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 -- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c0d1e60 msgr2=0x7fb51c1f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c0d1e60 0x7fb51c1f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 -- 172.21.15.181:0/2140979846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c009430 con 0x7fb51c15dc20 2023-10-16T09:59:41.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb51b59e700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c15dc20 0x7fb51c151960 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb51400cc50 tx=0x7fb51400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:41.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb5037fe700 1 -- 172.21.15.181:0/2140979846 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514003c00 con 0x7fb51c15dc20 2023-10-16T09:59:41.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb5037fe700 1 -- 172.21.15.181:0/2140979846 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514003d60 con 0x7fb51c15dc20 2023-10-16T09:59:41.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb5037fe700 1 -- 172.21.15.181:0/2140979846 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400aab0 con 0x7fb51c15dc20 2023-10-16T09:59:41.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c1f4080 con 0x7fb51c15dc20 2023-10-16T09:59:41.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.536+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c1f3ac0 con 0x7fb51c15dc20 2023-10-16T09:59:41.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.537+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c0cc6d0 con 0x7fb51c15dc20 2023-10-16T09:59:41.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.538+0000 7fb5037fe700 1 -- 172.21.15.181:0/2140979846 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb51400ac10 con 0x7fb51c15dc20 2023-10-16T09:59:41.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.538+0000 7fb5037fe700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 0x7fb50806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.538+0000 7fb51ad9d700 1 -- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 msgr2=0x7fb50806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:41.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.538+0000 7fb5037fe700 1 -- 172.21.15.181:0/2140979846 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fb514089a40 con 0x7fb51c15dc20 2023-10-16T09:59:41.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.538+0000 7fb51ad9d700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 0x7fb50806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:41.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.543+0000 7fb5037fe700 1 -- 172.21.15.181:0/2140979846 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb51c0cc6d0 con 0x7fb51c15dc20 2023-10-16T09:59:41.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.739+0000 7fb51ad9d700 1 -- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 msgr2=0x7fb50806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:41.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.739+0000 7fb51ad9d700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 0x7fb50806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:41.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.787+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb51c0cc6d0 con 0x7fb51c15dc20 2023-10-16T09:59:41.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.788+0000 7fb5037fe700 1 -- 172.21.15.181:0/2140979846 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fb51c0cc6d0 con 0x7fb51c15dc20 2023-10-16T09:59:41.790 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:41.790 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2852689416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2852689416},{"type":"v1","addr":"172.21.15.190:6834","nonce":2852689416}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2993219349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2993219349},{"type":"v1","addr":"172.21.15.181:6836","nonce":2993219349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2504788411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2504788411},{"type":"v1","addr":"172.21.15.181:6837","nonce":2504788411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:23.239300+0000","modified":"2023-10-16T09:59:26.271260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1948728404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1948728404},{"type":"v1","addr":"172.21.15.190:6835","nonce":1948728404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T09:59:41.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 msgr2=0x7fb50806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:41.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 0x7fb50806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c15dc20 msgr2=0x7fb51c151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c15dc20 0x7fb51c151960 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb51400cc50 tx=0x7fb51400cf60 comp rx=0 tx=0).stop 2023-10-16T09:59:41.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 shutdown_connections 2023-10-16T09:59:41.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fb50806b020 0x7fb50806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb51c0b4fb0 0x7fb51c160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c15dc20 0x7fb51c151960 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 --2- 172.21.15.181:0/2140979846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb51c0d1e60 0x7fb51c1f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 >> 172.21.15.181:0/2140979846 conn(0x7fb51c08af90 msgr2=0x7fb51c0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:41.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 shutdown_connections 2023-10-16T09:59:41.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.790+0000 7fb521238700 1 -- 172.21.15.181:0/2140979846 wait complete. 2023-10-16T09:59:41.795 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 32 2023-10-16T09:59:41.805 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:41.899 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:41.898+0000 7f97cc55b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:41.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.931+0000 7f2d3c81a700 1 Processor -- start 2023-10-16T09:59:41.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.932+0000 7f2d3c81a700 1 -- start start 2023-10-16T09:59:41.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.932+0000 7f2d3c81a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d3806d5f0 0x7f2d380d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.932+0000 7f2d3c81a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2d38160d20 0x7f2d381611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.932+0000 7f2d3c81a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d381616f0 con 0x7f2d38161b60 2023-10-16T09:59:41.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.932+0000 7f2d3c81a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d38162800 con 0x7f2d3806d5f0 2023-10-16T09:59:41.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d3c81a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d380d43e0 con 0x7f2d38160d20 2023-10-16T09:59:41.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d3806d5f0 0x7f2d380d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d3806d5f0 0x7f2d380d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42472/0 (socket says 172.21.15.181:42472) 2023-10-16T09:59:41.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 -- 172.21.15.181:0/1927859956 learned_addr learned my addr 172.21.15.181:0/1927859956 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:41.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d36d9d700 1 --2- 172.21.15.181:0/1927859956 >> v2:172.21.15.190:3300/0 conn(0x7f2d38160d20 0x7f2d381611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 -- 172.21.15.181:0/1927859956 >> v2:172.21.15.190:3300/0 conn(0x7f2d38160d20 msgr2=0x7f2d381611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:41.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 --2- 172.21.15.181:0/1927859956 >> v2:172.21.15.190:3300/0 conn(0x7f2d38160d20 0x7f2d381611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 -- 172.21.15.181:0/1927859956 >> v1:172.21.15.190:6789/0 conn(0x7f2d38161b60 legacy=0x7f2d38161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 -- 172.21.15.181:0/1927859956 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d380d2560 con 0x7f2d3806d5f0 2023-10-16T09:59:41.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d35d9b700 1 --2- 172.21.15.181:0/1927859956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d380d2020 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2d380d48d0 tx=0x7f2d2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c88ff069a1c1503 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:41.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d3559a700 1 -- 172.21.15.181:0/1927859956 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c011830 con 0x7f2d3806d5f0 2023-10-16T09:59:41.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d3559a700 1 -- 172.21.15.181:0/1927859956 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c011990 con 0x7f2d3806d5f0 2023-10-16T09:59:41.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.933+0000 7f2d3559a700 1 -- 172.21.15.181:0/1927859956 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c011d20 con 0x7f2d3806d5f0 2023-10-16T09:59:41.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1927859956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 msgr2=0x7f2d380d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1927859956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d380d2020 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2d380d48d0 tx=0x7f2d2c009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1927859956 shutdown_connections 2023-10-16T09:59:41.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1927859956 >> v2:172.21.15.190:3300/0 conn(0x7f2d38160d20 0x7f2d381611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1927859956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d380d2020 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1927859956 >> 172.21.15.181:0/1927859956 conn(0x7f2d3809cba0 msgr2=0x7f2d380d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:41.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1927859956 shutdown_connections 2023-10-16T09:59:41.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.934+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1927859956 wait complete. 2023-10-16T09:59:41.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.935+0000 7f2d3c81a700 1 Processor -- start 2023-10-16T09:59:41.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.935+0000 7f2d3c81a700 1 -- start start 2023-10-16T09:59:41.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3c81a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d3814a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3c81a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d38160d20 0x7f2d381f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3c81a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38161b60 0x7f2d3815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3c81a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3815f8b0 con 0x7f2d3806d5f0 2023-10-16T09:59:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3c81a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380b7610 con 0x7f2d38161b60 2023-10-16T09:59:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3c81a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3815f5d0 con 0x7f2d38160d20 2023-10-16T09:59:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d3814a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d3814a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42476/0 (socket says 172.21.15.181:42476) 2023-10-16T09:59:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d3659c700 1 -- 172.21.15.181:0/1860156246 learned_addr learned my addr 172.21.15.181:0/1860156246 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d36d9d700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38161b60 0x7f2d3815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.936+0000 7f2d35d9b700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d38160d20 0x7f2d381f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3659c700 1 -- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d38160d20 msgr2=0x7f2d381f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3659c700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d38160d20 0x7f2d381f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3659c700 1 -- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38161b60 msgr2=0x7f2d3815c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3659c700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38161b60 0x7f2d3815c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3659c700 1 -- 172.21.15.181:0/1860156246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c009550 con 0x7f2d3806d5f0 2023-10-16T09:59:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d35d9b700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d38160d20 0x7f2d381f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:41.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3659c700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d3814a890 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800c4d0 tx=0x7f2d2800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:41.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d26ffd700 1 -- 172.21.15.181:0/1860156246 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d280052d0 con 0x7f2d3806d5f0 2023-10-16T09:59:41.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d26ffd700 1 -- 172.21.15.181:0/1860156246 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d28005430 con 0x7f2d3806d5f0 2023-10-16T09:59:41.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d26ffd700 1 -- 172.21.15.181:0/1860156246 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d280038f0 con 0x7f2d3806d5f0 2023-10-16T09:59:41.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d380b78b0 con 0x7f2d3806d5f0 2023-10-16T09:59:41.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.937+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d381f3280 con 0x7f2d3806d5f0 2023-10-16T09:59:41.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.938+0000 7f2d26ffd700 1 -- 172.21.15.181:0/1860156246 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2d28003a50 con 0x7f2d3806d5f0 2023-10-16T09:59:41.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.938+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d380b7470 con 0x7f2d3806d5f0 2023-10-16T09:59:41.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.939+0000 7f2d26ffd700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 0x7f2d1806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:41.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.939+0000 7f2d35d9b700 1 -- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 msgr2=0x7f2d1806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:41.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.939+0000 7f2d26ffd700 1 -- 172.21.15.181:0/1860156246 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f2d28088ce0 con 0x7f2d3806d5f0 2023-10-16T09:59:41.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.939+0000 7f2d35d9b700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 0x7f2d1806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:41.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:41.943+0000 7f2d26ffd700 1 -- 172.21.15.181:0/1860156246 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d380b7470 con 0x7f2d3806d5f0 2023-10-16T09:59:41.977 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:41.976+0000 7f97cc55b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:59:42.020 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:42.019+0000 7f60d4a83200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:42.100 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:42.098+0000 7f60d4a83200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T09:59:42.120 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:42.119+0000 7f97cc55b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:42.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.140+0000 7f2d35d9b700 1 -- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 msgr2=0x7f2d1806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:42.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.140+0000 7f2d35d9b700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 0x7f2d1806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:42.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.144+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d380b7470 con 0x7f2d3806d5f0 2023-10-16T09:59:42.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.146+0000 7f2d26ffd700 1 -- 172.21.15.181:0/1860156246 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17754 (secure 0 0 0) 0x7f2d380b7470 con 0x7f2d3806d5f0 2023-10-16T09:59:42.148 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:42.148 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":77,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:37.126359+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:21.812039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:22.200991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:6832/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/1081249734":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2658775934":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2227004797":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3531530097":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2146555244":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3441419577":"2023-10-17T09:59:37.126305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 msgr2=0x7f2d1806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:42.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 0x7f2d1806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 msgr2=0x7f2d3814a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d3814a890 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800c4d0 tx=0x7f2d2800c7e0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 shutdown_connections 2023-10-16T09:59:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2d1806b020 0x7f2d1806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38161b60 0x7f2d3815c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d3806d5f0 0x7f2d3814a890 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 --2- 172.21.15.181:0/1860156246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d38160d20 0x7f2d381f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 >> 172.21.15.181:0/1860156246 conn(0x7f2d3809cba0 msgr2=0x7f2d380b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 shutdown_connections 2023-10-16T09:59:42.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.148+0000 7f2d3c81a700 1 -- 172.21.15.181:0/1860156246 wait complete. 2023-10-16T09:59:42.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:59:21.812039+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:59:22.200991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:42.164 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:42.253 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:42.252+0000 7f60d4a83200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T09:59:42.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.289+0000 7f2f5eadc700 1 Processor -- start 2023-10-16T09:59:42.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5eadc700 1 -- start start 2023-10-16T09:59:42.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5eadc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2f58071be0 0x7f2f580ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5eadc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f580ce1f0 con 0x7f2f580b2010 2023-10-16T09:59:42.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5eadc700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f580ccda0 con 0x7f2f580cc6e0 2023-10-16T09:59:42.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5eadc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f580ccb90 con 0x7f2f58071be0 2023-10-16T09:59:42.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5d079700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2f58071be0 0x7f2f580ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5c878700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2f580b2010 0x7f2f580b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53580/0 (socket says 172.21.15.181:53580) 2023-10-16T09:59:42.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.290+0000 7f2f5c878700 1 -- 172.21.15.181:0/3875237095 learned_addr learned my addr 172.21.15.181:0/3875237095 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:42.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.291+0000 7f2f5d079700 1 -- 172.21.15.181:0/3875237095 >> v1:172.21.15.190:6789/0 conn(0x7f2f580cc6e0 legacy=0x7f2f580ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.291+0000 7f2f5d079700 1 -- 172.21.15.181:0/3875237095 >> v1:172.21.15.181:6789/0 conn(0x7f2f580b2010 legacy=0x7f2f580b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.291+0000 7f2f5d079700 1 -- 172.21.15.181:0/3875237095 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f580727e0 con 0x7f2f58071be0 2023-10-16T09:59:42.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.291+0000 7f2f5d079700 1 --2- 172.21.15.181:0/3875237095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f58071be0 0x7f2f580ced90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009d90 tx=0x7f2f5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1a4870a6ac2aca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:42.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.291+0000 7f2f577fe700 1 -- 172.21.15.181:0/3875237095 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5000e070 con 0x7f2f58071be0 2023-10-16T09:59:42.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.291+0000 7f2f577fe700 1 -- 172.21.15.181:0/3875237095 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5000bbd0 con 0x7f2f58071be0 2023-10-16T09:59:42.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.291+0000 7f2f577fe700 1 -- 172.21.15.181:0/3875237095 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50012430 con 0x7f2f58071be0 2023-10-16T09:59:42.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.292+0000 7f2f5eadc700 1 -- 172.21.15.181:0/3875237095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f58071be0 msgr2=0x7f2f580ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.292+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/3875237095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f58071be0 0x7f2f580ced90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009d90 tx=0x7f2f5000b330 comp rx=0 tx=0).stop 2023-10-16T09:59:42.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.292+0000 7f2f5eadc700 1 -- 172.21.15.181:0/3875237095 shutdown_connections 2023-10-16T09:59:42.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.292+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/3875237095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f58071be0 0x7f2f580ced90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.292+0000 7f2f5eadc700 1 -- 172.21.15.181:0/3875237095 >> 172.21.15.181:0/3875237095 conn(0x7f2f580cd010 msgr2=0x7f2f580cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:42.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.292+0000 7f2f5eadc700 1 -- 172.21.15.181:0/3875237095 shutdown_connections 2023-10-16T09:59:42.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.292+0000 7f2f5eadc700 1 -- 172.21.15.181:0/3875237095 wait complete. 2023-10-16T09:59:42.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.293+0000 7f2f5eadc700 1 Processor -- start 2023-10-16T09:59:42.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.293+0000 7f2f5eadc700 1 -- start start 2023-10-16T09:59:42.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5eadc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f58071be0 0x7f2f58150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5eadc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f580b2010 0x7f2f58158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5eadc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f580cc6e0 0x7f2f58158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5eadc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f58151db0 con 0x7f2f580cc6e0 2023-10-16T09:59:42.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5eadc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5815bd80 con 0x7f2f580b2010 2023-10-16T09:59:42.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5eadc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5815c900 con 0x7f2f58071be0 2023-10-16T09:59:42.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f580cc6e0 0x7f2f58158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f580cc6e0 0x7f2f58158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42510/0 (socket says 172.21.15.181:42510) 2023-10-16T09:59:42.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 -- 172.21.15.181:0/1513497108 learned_addr learned my addr 172.21.15.181:0/1513497108 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:42.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5c878700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f58071be0 0x7f2f58150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f57fff700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f580b2010 0x7f2f58158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 -- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f58071be0 msgr2=0x7f2f58150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f58071be0 0x7f2f58150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 -- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f580b2010 msgr2=0x7f2f58158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f580b2010 0x7f2f58158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.294+0000 7f2f5d079700 1 -- 172.21.15.181:0/1513497108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f50009a70 con 0x7f2f580cc6e0 2023-10-16T09:59:42.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.295+0000 7f2f5d079700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f580cc6e0 0x7f2f58158ac0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2f50014040 tx=0x7f2f50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.295+0000 7f2f557fa700 1 -- 172.21.15.181:0/1513497108 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50012d20 con 0x7f2f580cc6e0 2023-10-16T09:59:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.295+0000 7f2f557fa700 1 -- 172.21.15.181:0/1513497108 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f50020070 con 0x7f2f580cc6e0 2023-10-16T09:59:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.295+0000 7f2f557fa700 1 -- 172.21.15.181:0/1513497108 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50012ea0 con 0x7f2f580cc6e0 2023-10-16T09:59:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.295+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5815bfd0 con 0x7f2f580cc6e0 2023-10-16T09:59:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.295+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5815c590 con 0x7f2f580cc6e0 2023-10-16T09:59:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.296+0000 7f2f557fa700 1 -- 172.21.15.181:0/1513497108 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2f50004280 con 0x7f2f580cc6e0 2023-10-16T09:59:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.296+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5815cab0 con 0x7f2f580cc6e0 2023-10-16T09:59:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.296+0000 7f2f557fa700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 0x7f2f3c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.297+0000 7f2f5c878700 1 -- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 msgr2=0x7f2f3c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.297+0000 7f2f557fa700 1 -- 172.21.15.181:0/1513497108 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f2f50022070 con 0x7f2f580cc6e0 2023-10-16T09:59:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.297+0000 7f2f5c878700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 0x7f2f3c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.301+0000 7f2f557fa700 1 -- 172.21.15.181:0/1513497108 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2f50056450 con 0x7f2f580cc6e0 2023-10-16T09:59:42.348 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:42.347+0000 7f97cc55b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:59:42.421 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:42.420+0000 7f97cc55b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:59:42.491 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:42.490+0000 7f60d4a83200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T09:59:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.498+0000 7f2f5c878700 1 -- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 msgr2=0x7f2f3c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.498+0000 7f2f5c878700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 0x7f2f3c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.499+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f580b6510 con 0x7f2f580cc6e0 2023-10-16T09:59:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.500+0000 7f2f557fa700 1 -- 172.21.15.181:0/1513497108 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f50059a70 con 0x7f2f580cc6e0 2023-10-16T09:59:42.501 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 msgr2=0x7f2f3c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 0x7f2f3c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f580cc6e0 msgr2=0x7f2f58158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f580cc6e0 0x7f2f58158ac0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2f50014040 tx=0x7f2f50004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 shutdown_connections 2023-10-16T09:59:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f2f3c06b020 0x7f2f3c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f58071be0 0x7f2f58150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f580cc6e0 0x7f2f58158ac0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 --2- 172.21.15.181:0/1513497108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f580b2010 0x7f2f58158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 >> 172.21.15.181:0/1513497108 conn(0x7f2f580cd010 msgr2=0x7f2f580b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:42.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 shutdown_connections 2023-10-16T09:59:42.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.502+0000 7f2f5eadc700 1 -- 172.21.15.181:0/1513497108 wait complete. 2023-10-16T09:59:42.516 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T09:59:42.574 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:42.572+0000 7f60d4a83200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T09:59:42.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.643+0000 7fc8939b7700 1 Processor -- start 2023-10-16T09:59:42.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.643+0000 7fc8939b7700 1 -- start start 2023-10-16T09:59:42.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.644+0000 7fc8939b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc88c15da70 0x7fc88c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.644+0000 7fc8939b7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c0cef00 con 0x7fc88c0ce6a0 2023-10-16T09:59:42.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.644+0000 7fc8939b7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c0cec20 con 0x7fc88c06afb0 2023-10-16T09:59:42.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.644+0000 7fc8939b7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c071c80 con 0x7fc88c15da70 2023-10-16T09:59:42.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.644+0000 7fc891753700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc88c15da70 0x7fc88c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.644+0000 7fc890f52700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc88c0ce6a0 0x7fc88c06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53594/0 (socket says 172.21.15.181:53594) 2023-10-16T09:59:42.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.644+0000 7fc890f52700 1 -- 172.21.15.181:0/2111132191 learned_addr learned my addr 172.21.15.181:0/2111132191 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:42.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc891753700 1 -- 172.21.15.181:0/2111132191 >> v1:172.21.15.190:6789/0 conn(0x7fc88c06afb0 legacy=0x7fc88c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc891753700 1 -- 172.21.15.181:0/2111132191 >> v1:172.21.15.181:6789/0 conn(0x7fc88c0ce6a0 legacy=0x7fc88c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc891753700 1 -- 172.21.15.181:0/2111132191 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c071f60 con 0x7fc88c15da70 2023-10-16T09:59:42.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc883fff700 1 -- 172.21.15.181:0/2111132191 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818364531 0 0) 0x7fc88c0cec20 con 0x7fc88c06afb0 2023-10-16T09:59:42.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc891753700 1 --2- 172.21.15.181:0/2111132191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c0ce160 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc888009840 tx=0x7fc888009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1592ade04b607c2d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:42.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc883fff700 1 -- 172.21.15.181:0/2111132191 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888011830 con 0x7fc88c15da70 2023-10-16T09:59:42.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc883fff700 1 -- 172.21.15.181:0/2111132191 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888011990 con 0x7fc88c15da70 2023-10-16T09:59:42.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.645+0000 7fc883fff700 1 -- 172.21.15.181:0/2111132191 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888011d20 con 0x7fc88c15da70 2023-10-16T09:59:42.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.646+0000 7fc8939b7700 1 -- 172.21.15.181:0/2111132191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 msgr2=0x7fc88c0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.646+0000 7fc8939b7700 1 --2- 172.21.15.181:0/2111132191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c0ce160 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc888009840 tx=0x7fc888009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:42.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.646+0000 7fc8939b7700 1 -- 172.21.15.181:0/2111132191 shutdown_connections 2023-10-16T09:59:42.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.646+0000 7fc8939b7700 1 --2- 172.21.15.181:0/2111132191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c0ce160 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.646+0000 7fc8939b7700 1 -- 172.21.15.181:0/2111132191 >> 172.21.15.181:0/2111132191 conn(0x7fc88c0cd010 msgr2=0x7fc88c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:42.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.646+0000 7fc8939b7700 1 -- 172.21.15.181:0/2111132191 shutdown_connections 2023-10-16T09:59:42.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.646+0000 7fc8939b7700 1 -- 172.21.15.181:0/2111132191 wait complete. 2023-10-16T09:59:42.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.648+0000 7fc8939b7700 1 Processor -- start 2023-10-16T09:59:42.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.648+0000 7fc8939b7700 1 -- start start 2023-10-16T09:59:42.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc8939b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88c06afb0 0x7fc88c1f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc8939b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc88c0ce6a0 0x7fc88c160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc8939b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc8939b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c15e6c0 con 0x7fc88c15da70 2023-10-16T09:59:42.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc8939b7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c1f3140 con 0x7fc88c0ce6a0 2023-10-16T09:59:42.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc8939b7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b5730 con 0x7fc88c06afb0 2023-10-16T09:59:42.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42524/0 (socket says 172.21.15.181:42524) 2023-10-16T09:59:42.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 -- 172.21.15.181:0/1762672207 learned_addr learned my addr 172.21.15.181:0/1762672207 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:42.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891753700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88c06afb0 0x7fc88c1f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc890f52700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc88c0ce6a0 0x7fc88c160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:42.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 -- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88c06afb0 msgr2=0x7fc88c1f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88c06afb0 0x7fc88c1f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 -- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc88c0ce6a0 msgr2=0x7fc88c160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc88c0ce6a0 0x7fc88c160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 -- 172.21.15.181:0/1762672207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880094a0 con 0x7fc88c15da70 2023-10-16T09:59:42.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.649+0000 7fc891f54700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c160990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c760 tx=0x7fc88400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:42.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.650+0000 7fc881ffb700 1 -- 172.21.15.181:0/1762672207 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884003e30 con 0x7fc88c15da70 2023-10-16T09:59:42.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.650+0000 7fc881ffb700 1 -- 172.21.15.181:0/1762672207 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc884005070 con 0x7fc88c15da70 2023-10-16T09:59:42.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.650+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c0b5a10 con 0x7fc88c15da70 2023-10-16T09:59:42.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.650+0000 7fc881ffb700 1 -- 172.21.15.181:0/1762672207 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884005400 con 0x7fc88c15da70 2023-10-16T09:59:42.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.650+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c15e350 con 0x7fc88c15da70 2023-10-16T09:59:42.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.651+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c15e800 con 0x7fc88c15da70 2023-10-16T09:59:42.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.651+0000 7fc881ffb700 1 -- 172.21.15.181:0/1762672207 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fc884005560 con 0x7fc88c15da70 2023-10-16T09:59:42.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.652+0000 7fc881ffb700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 0x7fc87006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:42.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.652+0000 7fc891753700 1 -- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 msgr2=0x7fc87006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:42.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.652+0000 7fc881ffb700 1 -- 172.21.15.181:0/1762672207 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fc8840895b0 con 0x7fc88c15da70 2023-10-16T09:59:42.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.652+0000 7fc891753700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 0x7fc87006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:42.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.657+0000 7fc881ffb700 1 -- 172.21.15.181:0/1762672207 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc8840542e0 con 0x7fc88c15da70 2023-10-16T09:59:42.687 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:42.686+0000 7f97cc55b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:59:42.854 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:42.853+0000 7f60d4a83200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T09:59:42.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.853+0000 7fc891753700 1 -- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 msgr2=0x7fc87006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:42.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.853+0000 7fc891753700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 0x7fc87006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:42.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.860+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc88c15de50 con 0x7fc88c15da70 2023-10-16T09:59:42.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.861+0000 7fc881ffb700 1 -- 172.21.15.181:0/1762672207 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc884057900 con 0x7fc88c15da70 2023-10-16T09:59:42.863 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T09:59:42.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 msgr2=0x7fc87006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:42.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 0x7fc87006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 msgr2=0x7fc88c160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:42.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c160990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c760 tx=0x7fc88400ca70 comp rx=0 tx=0).stop 2023-10-16T09:59:42.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 shutdown_connections 2023-10-16T09:59:42.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fc87006b020 0x7fc87006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88c06afb0 0x7fc88c1f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc88c15da70 0x7fc88c160990 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 --2- 172.21.15.181:0/1762672207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc88c0ce6a0 0x7fc88c160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:42.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 >> 172.21.15.181:0/1762672207 conn(0x7fc88c0cd010 msgr2=0x7fc88c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:42.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 shutdown_connections 2023-10-16T09:59:42.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:42.863+0000 7fc8939b7700 1 -- 172.21.15.181:0/1762672207 wait complete. 2023-10-16T09:59:42.877 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T09:59:42.905 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:42.903+0000 7f97cc55b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:59:43.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 Processor -- start 2023-10-16T09:59:43.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 -- start start 2023-10-16T09:59:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc815da70 0x7f7dc80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc80ce6a0 0x7f7dc806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7dc8072760 0x7f7dc806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc80cef00 con 0x7f7dc80ce6a0 2023-10-16T09:59:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc80cec20 con 0x7f7dc8072760 2023-10-16T09:59:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.003+0000 7f7dcdf46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8071c80 con 0x7f7dc815da70 2023-10-16T09:59:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc80ce6a0 0x7f7dc806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dc80ce6a0 0x7f7dc806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42530/0 (socket says 172.21.15.181:42530) 2023-10-16T09:59:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 -- 172.21.15.181:0/2163015860 learned_addr learned my addr 172.21.15.181:0/2163015860 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc77fe700 1 --2- 172.21.15.181:0/2163015860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc815da70 0x7f7dc80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc7fff700 1 --2- 172.21.15.181:0/2163015860 >> v2:172.21.15.190:3300/0 conn(0x7f7dc8072760 0x7f7dc806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 -- 172.21.15.181:0/2163015860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc815da70 msgr2=0x7f7dc80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 --2- 172.21.15.181:0/2163015860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc815da70 0x7f7dc80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 -- 172.21.15.181:0/2163015860 >> v2:172.21.15.190:3300/0 conn(0x7f7dc8072760 msgr2=0x7f7dc806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 --2- 172.21.15.181:0/2163015860 >> v2:172.21.15.190:3300/0 conn(0x7f7dc8072760 0x7f7dc806ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 -- 172.21.15.181:0/2163015860 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8071f60 con 0x7f7dc80ce6a0 2023-10-16T09:59:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc6ffd700 1 --2- 172.21.15.181:0/2163015860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc80ce6a0 0x7f7dc806a8a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8161530 tx=0x7f7dbc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2a71a39a9417d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc67fc700 1 -- 172.21.15.181:0/2163015860 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011830 con 0x7f7dc80ce6a0 2023-10-16T09:59:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc67fc700 1 -- 172.21.15.181:0/2163015860 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc011990 con 0x7f7dc80ce6a0 2023-10-16T09:59:43.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.004+0000 7f7dc67fc700 1 -- 172.21.15.181:0/2163015860 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011d20 con 0x7f7dc80ce6a0 2023-10-16T09:59:43.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2163015860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc80ce6a0 msgr2=0x7f7dc806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2163015860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc80ce6a0 0x7f7dc806a8a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8161530 tx=0x7f7dbc009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2163015860 shutdown_connections 2023-10-16T09:59:43.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2163015860 >> v2:172.21.15.190:3300/0 conn(0x7f7dc8072760 0x7f7dc806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2163015860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc815da70 0x7f7dc80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2163015860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc80ce6a0 0x7f7dc806a8a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2163015860 >> 172.21.15.181:0/2163015860 conn(0x7f7dc80cd010 msgr2=0x7f7dc80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:43.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2163015860 shutdown_connections 2023-10-16T09:59:43.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.005+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2163015860 wait complete. 2023-10-16T09:59:43.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.006+0000 7f7dcdf46700 1 Processor -- start 2023-10-16T09:59:43.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.006+0000 7f7dcdf46700 1 -- start start 2023-10-16T09:59:43.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dcdf46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc8072760 0x7f7dc80c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dcdf46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc80ce6a0 0x7f7dc80aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dcdf46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc815da70 0x7f7dc81f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dcdf46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc80af0d0 con 0x7f7dc815da70 2023-10-16T09:59:43.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dcdf46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc815ebe0 con 0x7f7dc8072760 2023-10-16T09:59:43.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dcdf46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc80c4750 con 0x7f7dc80ce6a0 2023-10-16T09:59:43.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc815da70 0x7f7dc81f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc815da70 0x7f7dc81f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42540/0 (socket says 172.21.15.181:42540) 2023-10-16T09:59:43.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dc7fff700 1 -- 172.21.15.181:0/2640959022 learned_addr learned my addr 172.21.15.181:0/2640959022 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:43.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dc77fe700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc8072760 0x7f7dc80c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dc6ffd700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc80ce6a0 0x7f7dc80aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dc7fff700 1 -- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc80ce6a0 msgr2=0x7f7dc80aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.007+0000 7f7dc7fff700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc80ce6a0 0x7f7dc80aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7dc7fff700 1 -- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc8072760 msgr2=0x7f7dc80c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7dc7fff700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc8072760 0x7f7dc80c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7dc7fff700 1 -- 172.21.15.181:0/2640959022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc009550 con 0x7f7dc815da70 2023-10-16T09:59:43.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7dc6ffd700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc80ce6a0 0x7f7dc80aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:43.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7dc7fff700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc815da70 0x7f7dc81f2940 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000eed0 tx=0x7f7dc000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:43.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7daffff700 1 -- 172.21.15.181:0/2640959022 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0010040 con 0x7f7dc815da70 2023-10-16T09:59:43.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7daffff700 1 -- 172.21.15.181:0/2640959022 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000b9f0 con 0x7f7dc815da70 2023-10-16T09:59:43.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7daffff700 1 -- 172.21.15.181:0/2640959022 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000bda0 con 0x7f7dc815da70 2023-10-16T09:59:43.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc80c4a30 con 0x7f7dc815da70 2023-10-16T09:59:43.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.008+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc80c4ff0 con 0x7f7dc815da70 2023-10-16T09:59:43.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.009+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc815e9b0 con 0x7f7dc815da70 2023-10-16T09:59:43.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.010+0000 7f7daffff700 1 -- 172.21.15.181:0/2640959022 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f7dc0014400 con 0x7f7dc815da70 2023-10-16T09:59:43.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.010+0000 7f7daffff700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 0x7f7db406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.010+0000 7f7dc77fe700 1 -- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 msgr2=0x7f7db406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:43.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.010+0000 7f7daffff700 1 -- 172.21.15.181:0/2640959022 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f7dc0089e80 con 0x7f7dc815da70 2023-10-16T09:59:43.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.010+0000 7f7dc77fe700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 0x7f7db406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:43.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.015+0000 7f7daffff700 1 -- 172.21.15.181:0/2640959022 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7dc815e9b0 con 0x7f7dc815da70 2023-10-16T09:59:43.052 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:43.051+0000 7f97cc55b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:59:43.098 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:43.096+0000 7f60d4a83200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T09:59:43.167 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:43.166+0000 7f97cc55b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:59:43.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.211+0000 7f7dc77fe700 1 -- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 msgr2=0x7f7db406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:43.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.211+0000 7f7dc77fe700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 0x7f7db406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:43.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.214+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7dc815e9b0 con 0x7f7dc815da70 2023-10-16T09:59:43.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.214+0000 7f7daffff700 1 -- 172.21.15.181:0/2640959022 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7dc815e9b0 con 0x7f7dc815da70 2023-10-16T09:59:43.216 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T09:59:43.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 msgr2=0x7f7db406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:43.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 0x7f7db406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc815da70 msgr2=0x7f7dc81f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc815da70 0x7f7dc81f2940 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000eed0 tx=0x7f7dc000ef00 comp rx=0 tx=0).stop 2023-10-16T09:59:43.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 shutdown_connections 2023-10-16T09:59:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f7db406b020 0x7f7db406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc815da70 0x7f7dc81f2940 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dc8072760 0x7f7dc80c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 --2- 172.21.15.181:0/2640959022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dc80ce6a0 0x7f7dc80aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 >> 172.21.15.181:0/2640959022 conn(0x7f7dc80cd010 msgr2=0x7f7dc806abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 shutdown_connections 2023-10-16T09:59:43.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.217+0000 7f7dcdf46700 1 -- 172.21.15.181:0/2640959022 wait complete. 2023-10-16T09:59:43.231 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:43.257 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:43.256+0000 7f60d4a83200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T09:59:43.320 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:43.319+0000 7f97cc55b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:59:43.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.353+0000 7fafd045e700 1 Processor -- start 2023-10-16T09:59:43.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.353+0000 7fafd045e700 1 -- start start 2023-10-16T09:59:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafd045e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fafc80d1e60 0x7fafc80d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafd045e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80b2f00 con 0x7fafc80b3fa0 2023-10-16T09:59:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafd045e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80b31e0 con 0x7fafc815dc20 2023-10-16T09:59:43.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafd045e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc80b3b80 con 0x7fafc80d1e60 2023-10-16T09:59:43.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafce9fb700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fafc80b3fa0 0x7fafc80b43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53628/0 (socket says 172.21.15.181:53628) 2023-10-16T09:59:43.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafce9fb700 1 -- 172.21.15.181:0/3019619282 learned_addr learned my addr 172.21.15.181:0/3019619282 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:43.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafcd9f9700 1 --2- 172.21.15.181:0/3019619282 >> v2:172.21.15.190:3300/0 conn(0x7fafc80d1e60 0x7fafc80d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.354+0000 7fafcd1f8700 1 -- 172.21.15.181:0/3019619282 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402025336 0 0) 0x7fafc80b2f00 con 0x7fafc80b3fa0 2023-10-16T09:59:43.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd1f8700 1 -- 172.21.15.181:0/3019619282 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb40035b0 con 0x7fafc80b3fa0 2023-10-16T09:59:43.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd9f9700 1 -- 172.21.15.181:0/3019619282 >> v1:172.21.15.190:6789/0 conn(0x7fafc815dc20 legacy=0x7fafc80d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd1f8700 1 -- 172.21.15.181:0/3019619282 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443539422 0 0) 0x7fafc80b31e0 con 0x7fafc815dc20 2023-10-16T09:59:43.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd9f9700 1 -- 172.21.15.181:0/3019619282 >> v1:172.21.15.181:6789/0 conn(0x7fafc80b3fa0 legacy=0x7fafc80b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd9f9700 1 -- 172.21.15.181:0/3019619282 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc80b34c0 con 0x7fafc80d1e60 2023-10-16T09:59:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd9f9700 1 --2- 172.21.15.181:0/3019619282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 0x7fafc80d22e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009840 tx=0x7fafb8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd9779609121978 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd1f8700 1 -- 172.21.15.181:0/3019619282 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8011830 con 0x7fafc80d1e60 2023-10-16T09:59:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd1f8700 1 -- 172.21.15.181:0/3019619282 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb8011990 con 0x7fafc80d1e60 2023-10-16T09:59:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.355+0000 7fafcd1f8700 1 -- 172.21.15.181:0/3019619282 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8011d20 con 0x7fafc80d1e60 2023-10-16T09:59:43.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.356+0000 7fafd045e700 1 -- 172.21.15.181:0/3019619282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 msgr2=0x7fafc80d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.356+0000 7fafd045e700 1 --2- 172.21.15.181:0/3019619282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 0x7fafc80d22e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009840 tx=0x7fafb8009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.356+0000 7fafd045e700 1 -- 172.21.15.181:0/3019619282 shutdown_connections 2023-10-16T09:59:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.356+0000 7fafd045e700 1 --2- 172.21.15.181:0/3019619282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 0x7fafc80d22e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.356+0000 7fafd045e700 1 -- 172.21.15.181:0/3019619282 >> 172.21.15.181:0/3019619282 conn(0x7fafc808af90 msgr2=0x7fafc80d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:43.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.356+0000 7fafd045e700 1 -- 172.21.15.181:0/3019619282 shutdown_connections 2023-10-16T09:59:43.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.356+0000 7fafd045e700 1 -- 172.21.15.181:0/3019619282 wait complete. 2023-10-16T09:59:43.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.357+0000 7fafd045e700 1 Processor -- start 2023-10-16T09:59:43.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.357+0000 7fafd045e700 1 -- start start 2023-10-16T09:59:43.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafd045e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafc80b3fa0 0x7fafc8146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafd045e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 0x7fafc815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafd045e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc815dc20 0x7fafc8154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafd045e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc815b730 con 0x7fafc815dc20 2023-10-16T09:59:43.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafd045e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc815bbc0 con 0x7fafc80d1e60 2023-10-16T09:59:43.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafd045e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc815ab20 con 0x7fafc80b3fa0 2023-10-16T09:59:43.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc815dc20 0x7fafc8154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc815dc20 0x7fafc8154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42556/0 (socket says 172.21.15.181:42556) 2023-10-16T09:59:43.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 -- 172.21.15.181:0/938670099 learned_addr learned my addr 172.21.15.181:0/938670099 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:43.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce1fa700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafc80b3fa0 0x7fafc8146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafcd9f9700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 0x7fafc815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 -- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafc80b3fa0 msgr2=0x7fafc8146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafc80b3fa0 0x7fafc8146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 -- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 msgr2=0x7fafc815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 0x7fafc815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 -- 172.21.15.181:0/938670099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80094a0 con 0x7fafc815dc20 2023-10-16T09:59:43.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.358+0000 7fafce9fb700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc815dc20 0x7fafc8154980 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fafc000cb40 tx=0x7fafc000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:43.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.359+0000 7fafbeffd700 1 -- 172.21.15.181:0/938670099 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000a9e0 con 0x7fafc815dc20 2023-10-16T09:59:43.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.359+0000 7fafbeffd700 1 -- 172.21.15.181:0/938670099 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000ab40 con 0x7fafc815dc20 2023-10-16T09:59:43.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.359+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc815ae00 con 0x7fafc815dc20 2023-10-16T09:59:43.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.359+0000 7fafbeffd700 1 -- 172.21.15.181:0/938670099 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0012400 con 0x7fafc815dc20 2023-10-16T09:59:43.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.359+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc815b3c0 con 0x7fafc815dc20 2023-10-16T09:59:43.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.359+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc806bc20 con 0x7fafc815dc20 2023-10-16T09:59:43.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.360+0000 7fafbeffd700 1 -- 172.21.15.181:0/938670099 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fafc001b3b0 con 0x7fafc815dc20 2023-10-16T09:59:43.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.361+0000 7fafbeffd700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 0x7fafac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.361+0000 7fafce1fa700 1 -- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 msgr2=0x7fafac06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:43.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.361+0000 7fafbeffd700 1 -- 172.21.15.181:0/938670099 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fafc00895e0 con 0x7fafc815dc20 2023-10-16T09:59:43.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.361+0000 7fafce1fa700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 0x7fafac06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:43.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.365+0000 7fafbeffd700 1 -- 172.21.15.181:0/938670099 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fafc0054340 con 0x7fafc815dc20 2023-10-16T09:59:43.382 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:43.381+0000 7f60d4a83200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T09:59:43.541 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:43.539+0000 7f60d4a83200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T09:59:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.562+0000 7fafce1fa700 1 -- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 msgr2=0x7fafac06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:43.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.562+0000 7fafce1fa700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 0x7fafac06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:43.583 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:43.582+0000 7f97cc55b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:43.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.607+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafc80d4530 con 0x7fafc815dc20 2023-10-16T09:59:43.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.607+0000 7fafbeffd700 1 -- 172.21.15.181:0/938670099 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fafc0057960 con 0x7fafc815dc20 2023-10-16T09:59:43.609 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2852689416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2852689416},{"type":"v1","addr":"172.21.15.190:6834","nonce":2852689416}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2993219349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2993219349},{"type":"v1","addr":"172.21.15.181:6836","nonce":2993219349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2504788411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2504788411},{"type":"v1","addr":"172.21.15.181:6837","nonce":2504788411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:23.239300+0000","modified":"2023-10-16T09:59:26.271260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1948728404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1948728404},{"type":"v1","addr":"172.21.15.190:6835","nonce":1948728404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T09:59:43.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.609+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 msgr2=0x7fafac06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:43.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.609+0000 7fafd045e700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 0x7fafac06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc815dc20 msgr2=0x7fafc8154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc815dc20 0x7fafc8154980 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fafc000cb40 tx=0x7fafc000ce50 comp rx=0 tx=0).stop 2023-10-16T09:59:43.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 shutdown_connections 2023-10-16T09:59:43.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fafac06b020 0x7fafac06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafc80b3fa0 0x7fafc8146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc815dc20 0x7fafc8154980 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 --2- 172.21.15.181:0/938670099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafc80d1e60 0x7fafc815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 >> 172.21.15.181:0/938670099 conn(0x7fafc808af90 msgr2=0x7fafc80d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:43.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 shutdown_connections 2023-10-16T09:59:43.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.610+0000 7fafd045e700 1 -- 172.21.15.181:0/938670099 wait complete. 2023-10-16T09:59:43.614 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 32 2023-10-16T09:59:43.625 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:43.741 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:43.739+0000 7f97cc55b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:59:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.789+0000 7fed4ea12700 1 Processor -- start 2023-10-16T09:59:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.790+0000 7fed4ea12700 1 -- start start 2023-10-16T09:59:43.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4ea12700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed480cdfb0 0x7fed4815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4ea12700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cc0c0 0x7fed4815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4ea12700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed4806cdf0 0x7fed480cb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4ea12700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed480cabd0 con 0x7fed480cdfb0 2023-10-16T09:59:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4ea12700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed480ca5c0 con 0x7fed4806cdf0 2023-10-16T09:59:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4ea12700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed48066740 con 0x7fed480cc0c0 2023-10-16T09:59:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed480cdfb0 0x7fed4815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed480cdfb0 0x7fed4815d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42562/0 (socket says 172.21.15.181:42562) 2023-10-16T09:59:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 -- 172.21.15.181:0/796084837 learned_addr learned my addr 172.21.15.181:0/796084837 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed477fe700 1 --2- 172.21.15.181:0/796084837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cc0c0 0x7fed4815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4cfaf700 1 --2- 172.21.15.181:0/796084837 >> v2:172.21.15.190:3300/0 conn(0x7fed4806cdf0 0x7fed480cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 -- 172.21.15.181:0/796084837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cc0c0 msgr2=0x7fed4815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 --2- 172.21.15.181:0/796084837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cc0c0 0x7fed4815ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 -- 172.21.15.181:0/796084837 >> v2:172.21.15.190:3300/0 conn(0x7fed4806cdf0 msgr2=0x7fed480cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 --2- 172.21.15.181:0/796084837 >> v2:172.21.15.190:3300/0 conn(0x7fed4806cdf0 0x7fed480cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 -- 172.21.15.181:0/796084837 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48066a20 con 0x7fed480cdfb0 2023-10-16T09:59:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed4cfaf700 1 --2- 172.21.15.181:0/796084837 >> v2:172.21.15.190:3300/0 conn(0x7fed4806cdf0 0x7fed480cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed47fff700 1 --2- 172.21.15.181:0/796084837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cdfb0 0x7fed4815d8b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fed4815f6d0 tx=0x7fed38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66fab33a0e643d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed46ffd700 1 -- 172.21.15.181:0/796084837 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38011830 con 0x7fed480cdfb0 2023-10-16T09:59:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed46ffd700 1 -- 172.21.15.181:0/796084837 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38011990 con 0x7fed480cdfb0 2023-10-16T09:59:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.791+0000 7fed46ffd700 1 -- 172.21.15.181:0/796084837 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38011d20 con 0x7fed480cdfb0 2023-10-16T09:59:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 -- 172.21.15.181:0/796084837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cdfb0 msgr2=0x7fed4815d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 --2- 172.21.15.181:0/796084837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cdfb0 0x7fed4815d8b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fed4815f6d0 tx=0x7fed38009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 -- 172.21.15.181:0/796084837 shutdown_connections 2023-10-16T09:59:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 --2- 172.21.15.181:0/796084837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cdfb0 0x7fed4815d8b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 --2- 172.21.15.181:0/796084837 >> v2:172.21.15.190:3300/0 conn(0x7fed4806cdf0 0x7fed480cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 --2- 172.21.15.181:0/796084837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cc0c0 0x7fed4815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 -- 172.21.15.181:0/796084837 >> 172.21.15.181:0/796084837 conn(0x7fed480b1780 msgr2=0x7fed480b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:43.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 -- 172.21.15.181:0/796084837 shutdown_connections 2023-10-16T09:59:43.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.792+0000 7fed4ea12700 1 -- 172.21.15.181:0/796084837 wait complete. 2023-10-16T09:59:43.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.794+0000 7fed4ea12700 1 Processor -- start 2023-10-16T09:59:43.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.794+0000 7fed4ea12700 1 -- start start 2023-10-16T09:59:43.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed4ea12700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed4806cdf0 0x7fed480c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed4ea12700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cc0c0 0x7fed4815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed4ea12700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cdfb0 0x7fed480c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed4ea12700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480c8910 con 0x7fed480cc0c0 2023-10-16T09:59:43.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed4ea12700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480b32b0 con 0x7fed4806cdf0 2023-10-16T09:59:43.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed4ea12700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed480b3590 con 0x7fed480cdfb0 2023-10-16T09:59:43.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cc0c0 0x7fed4815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cc0c0 0x7fed4815b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42570/0 (socket says 172.21.15.181:42570) 2023-10-16T09:59:43.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 -- 172.21.15.181:0/2767403265 learned_addr learned my addr 172.21.15.181:0/2767403265 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:43.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed47fff700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed4806cdf0 0x7fed480c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed4cfaf700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cdfb0 0x7fed480c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:43.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 -- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cdfb0 msgr2=0x7fed480c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cdfb0 0x7fed480c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 -- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed4806cdf0 msgr2=0x7fed480c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:43.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed4806cdf0 0x7fed480c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:43.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 -- 172.21.15.181:0/2767403265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38009550 con 0x7fed480cc0c0 2023-10-16T09:59:43.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed477fe700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cc0c0 0x7fed4815b6f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00e980 tx=0x7fed3c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:43.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed44ff9700 1 -- 172.21.15.181:0/2767403265 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c009710 con 0x7fed480cc0c0 2023-10-16T09:59:43.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed44ff9700 1 -- 172.21.15.181:0/2767403265 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c00eed0 con 0x7fed480cc0c0 2023-10-16T09:59:43.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.795+0000 7fed44ff9700 1 -- 172.21.15.181:0/2767403265 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00ca90 con 0x7fed480cc0c0 2023-10-16T09:59:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.796+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed480b3870 con 0x7fed480cc0c0 2023-10-16T09:59:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.796+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed480b3e30 con 0x7fed480cc0c0 2023-10-16T09:59:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.796+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48067d20 con 0x7fed480cc0c0 2023-10-16T09:59:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.797+0000 7fed44ff9700 1 -- 172.21.15.181:0/2767403265 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fed3c00cbf0 con 0x7fed480cc0c0 2023-10-16T09:59:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.797+0000 7fed44ff9700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 0x7fed3406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.797+0000 7fed47fff700 1 -- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 msgr2=0x7fed3406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:43.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.797+0000 7fed44ff9700 1 -- 172.21.15.181:0/2767403265 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fed3c089b20 con 0x7fed480cc0c0 2023-10-16T09:59:43.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.797+0000 7fed47fff700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 0x7fed3406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:43.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.802+0000 7fed44ff9700 1 -- 172.21.15.181:0/2767403265 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fed3c0547c0 con 0x7fed480cc0c0 2023-10-16T09:59:43.820 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:43.819+0000 7f60d4a83200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T09:59:43.987 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:43.985+0000 7f60d4a83200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T09:59:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.998+0000 7fed47fff700 1 -- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 msgr2=0x7fed3406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:43.998+0000 7fed47fff700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 0x7fed3406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.003+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed48072510 con 0x7fed480cc0c0 2023-10-16T09:59:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.004+0000 7fed44ff9700 1 -- 172.21.15.181:0/2767403265 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17754 (secure 0 0 0) 0x7fed3c021090 con 0x7fed480cc0c0 2023-10-16T09:59:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:44.006 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":77,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:37.126359+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:21.812039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:22.200991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:6832/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/1081249734":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2658775934":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2227004797":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3531530097":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2146555244":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3441419577":"2023-10-17T09:59:37.126305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 msgr2=0x7fed3406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 0x7fed3406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cc0c0 msgr2=0x7fed4815b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cc0c0 0x7fed4815b6f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00e980 tx=0x7fed3c00ec90 comp rx=0 tx=0).stop 2023-10-16T09:59:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 shutdown_connections 2023-10-16T09:59:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fed3406b020 0x7fed3406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed480cdfb0 0x7fed480c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed4806cdf0 0x7fed480c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 --2- 172.21.15.181:0/2767403265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed480cc0c0 0x7fed4815b6f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 >> 172.21.15.181:0/2767403265 conn(0x7fed480b1780 msgr2=0x7fed4806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 shutdown_connections 2023-10-16T09:59:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.006+0000 7fed4ea12700 1 -- 172.21.15.181:0/2767403265 wait complete. 2023-10-16T09:59:44.020 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T09:59:44.021 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.142+0000 7fd43f686700 1 Processor -- start 2023-10-16T09:59:44.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.143+0000 7fd43f686700 1 -- start start 2023-10-16T09:59:44.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43f686700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815e6d0 0x7fd4380d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43f686700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd43815eab0 0x7fd43815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43f686700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43806ed00 con 0x7fd4380cb0c0 2023-10-16T09:59:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43f686700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4380d02d0 con 0x7fd43815eab0 2023-10-16T09:59:44.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43f686700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43815ad40 con 0x7fd43815e6d0 2023-10-16T09:59:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43d422700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815e6d0 0x7fd4380d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43cc21700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd43815eab0 0x7fd43815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43d422700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815e6d0 0x7fd4380d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:39604/0 (socket says 172.21.15.181:39604) 2023-10-16T09:59:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43d422700 1 -- 172.21.15.181:0/267858936 learned_addr learned my addr 172.21.15.181:0/267858936 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.144+0000 7fd43d422700 1 -- 172.21.15.181:0/267858936 >> v2:172.21.15.190:3300/0 conn(0x7fd43815eab0 msgr2=0x7fd43815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:44.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd43d422700 1 --2- 172.21.15.181:0/267858936 >> v2:172.21.15.190:3300/0 conn(0x7fd43815eab0 0x7fd43815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd43d422700 1 -- 172.21.15.181:0/267858936 >> v1:172.21.15.190:6789/0 conn(0x7fd4380cb0c0 legacy=0x7fd43815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd43d422700 1 -- 172.21.15.181:0/267858936 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43806e0f0 con 0x7fd43815e6d0 2023-10-16T09:59:44.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd43cc21700 1 --2- 172.21.15.181:0/267858936 >> v2:172.21.15.190:3300/0 conn(0x7fd43815eab0 0x7fd43815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:44.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd43d422700 1 --2- 172.21.15.181:0/267858936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815e6d0 0x7fd4380d0fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd434006fd0 tx=0x7fd434009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e54136f823fe88fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:44.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd42ffff700 1 -- 172.21.15.181:0/267858936 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434005e30 con 0x7fd43815e6d0 2023-10-16T09:59:44.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd42ffff700 1 -- 172.21.15.181:0/267858936 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434004030 con 0x7fd43815e6d0 2023-10-16T09:59:44.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.145+0000 7fd42ffff700 1 -- 172.21.15.181:0/267858936 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4340043e0 con 0x7fd43815e6d0 2023-10-16T09:59:44.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 -- 172.21.15.181:0/267858936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815e6d0 msgr2=0x7fd4380d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 --2- 172.21.15.181:0/267858936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815e6d0 0x7fd4380d0fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd434006fd0 tx=0x7fd434009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 -- 172.21.15.181:0/267858936 shutdown_connections 2023-10-16T09:59:44.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 --2- 172.21.15.181:0/267858936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815e6d0 0x7fd4380d0fc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 --2- 172.21.15.181:0/267858936 >> v2:172.21.15.190:3300/0 conn(0x7fd43815eab0 0x7fd43815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 -- 172.21.15.181:0/267858936 >> 172.21.15.181:0/267858936 conn(0x7fd4380d04a0 msgr2=0x7fd4380d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:44.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 -- 172.21.15.181:0/267858936 shutdown_connections 2023-10-16T09:59:44.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.146+0000 7fd43f686700 1 -- 172.21.15.181:0/267858936 wait complete. 2023-10-16T09:59:44.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.148+0000 7fd43f686700 1 Processor -- start 2023-10-16T09:59:44.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.148+0000 7fd43f686700 1 -- start start 2023-10-16T09:59:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43f686700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4380cb0c0 0x7fd4380aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43f686700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43815e6d0 0x7fd438156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43f686700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815eab0 0x7fd4380c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43f686700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380ae1e0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43f686700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380adf00 con 0x7fd43815e6d0 2023-10-16T09:59:44.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43f686700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380c8a50 con 0x7fd43815eab0 2023-10-16T09:59:44.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4380cb0c0 0x7fd4380aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4380cb0c0 0x7fd4380aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42584/0 (socket says 172.21.15.181:42584) 2023-10-16T09:59:44.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 -- 172.21.15.181:0/2373971996 learned_addr learned my addr 172.21.15.181:0/2373971996 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:44.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43cc21700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43815e6d0 0x7fd438156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43dc23700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815eab0 0x7fd4380c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 -- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815eab0 msgr2=0x7fd4380c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815eab0 0x7fd4380c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 -- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43815e6d0 msgr2=0x7fd438156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43815e6d0 0x7fd438156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43d422700 1 -- 172.21.15.181:0/2373971996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4280094a0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.149+0000 7fd43cc21700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43815e6d0 0x7fd438156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.150+0000 7fd43d422700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4380cb0c0 0x7fd4380aecb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.150+0000 7fd42dffb700 1 -- 172.21.15.181:0/2373971996 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434003c70 con 0x7fd4380cb0c0 2023-10-16T09:59:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.150+0000 7fd42dffb700 1 -- 172.21.15.181:0/2373971996 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434003dd0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.150+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd434009500 con 0x7fd4380cb0c0 2023-10-16T09:59:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.150+0000 7fd42dffb700 1 -- 172.21.15.181:0/2373971996 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43401c640 con 0x7fd4380cb0c0 2023-10-16T09:59:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.150+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4380c9010 con 0x7fd4380cb0c0 2023-10-16T09:59:44.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.151+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4380c88b0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.152+0000 7fd42dffb700 1 -- 172.21.15.181:0/2373971996 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd43401c7a0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.152+0000 7fd42dffb700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 0x7fd41c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.152+0000 7fd43cc21700 1 -- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 msgr2=0x7fd41c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:44.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.152+0000 7fd42dffb700 1 -- 172.21.15.181:0/2373971996 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fd43408aba0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.152+0000 7fd43cc21700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 0x7fd41c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:44.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.160+0000 7fd42dffb700 1 -- 172.21.15.181:0/2373971996 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd4340559b0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.265 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T09:59:44.264+0000 7f97cc55b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:59:44.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.353+0000 7fd43cc21700 1 -- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 msgr2=0x7fd41c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:44.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.353+0000 7fd43cc21700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 0x7fd41c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T09:59:44.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.407+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4380c88b0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.408+0000 7fd42dffb700 1 -- 172.21.15.181:0/2373971996 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fd4380c88b0 con 0x7fd4380cb0c0 2023-10-16T09:59:44.410 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:44.410 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2852689416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2852689416},{"type":"v1","addr":"172.21.15.190:6834","nonce":2852689416}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2993219349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2993219349},{"type":"v1","addr":"172.21.15.181:6836","nonce":2993219349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2504788411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2504788411},{"type":"v1","addr":"172.21.15.181:6837","nonce":2504788411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:23.239300+0000","modified":"2023-10-16T09:59:26.271260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1948728404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1948728404},{"type":"v1","addr":"172.21.15.190:6835","nonce":1948728404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T09:59:44.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 msgr2=0x7fd41c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:44.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 0x7fd41c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4380cb0c0 msgr2=0x7fd4380aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4380cb0c0 0x7fd4380aecb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004750 comp rx=0 tx=0).stop 2023-10-16T09:59:44.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 shutdown_connections 2023-10-16T09:59:44.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7fd41c06b020 0x7fd41c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd43815eab0 0x7fd4380c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4380cb0c0 0x7fd4380aecb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 --2- 172.21.15.181:0/2373971996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43815e6d0 0x7fd438156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 >> 172.21.15.181:0/2373971996 conn(0x7fd4380d04a0 msgr2=0x7fd4380c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:44.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 shutdown_connections 2023-10-16T09:59:44.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.410+0000 7fd43f686700 1 -- 172.21.15.181:0/2373971996 wait complete. 2023-10-16T09:59:44.415 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 32 2023-10-16T09:59:44.425 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:44.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.546+0000 7f4083a1a700 1 Processor -- start 2023-10-16T09:59:44.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.546+0000 7f4083a1a700 1 -- start start 2023-10-16T09:59:44.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.547+0000 7f4083a1a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f407c0d1e60 0x7f407c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.547+0000 7f4083a1a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0b2f00 con 0x7f407c15dc20 2023-10-16T09:59:44.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.547+0000 7f4083a1a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0b31e0 con 0x7f407c0b3fa0 2023-10-16T09:59:44.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.547+0000 7f4083a1a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407c0b3b80 con 0x7f407c0d1e60 2023-10-16T09:59:44.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.547+0000 7f40817b6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f407c15dc20 0x7f407c0d41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53672/0 (socket says 172.21.15.181:53672) 2023-10-16T09:59:44.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.547+0000 7f40817b6700 1 -- 172.21.15.181:0/467746059 learned_addr learned my addr 172.21.15.181:0/467746059 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:44.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4080fb5700 1 --2- 172.21.15.181:0/467746059 >> v2:172.21.15.190:3300/0 conn(0x7f407c0d1e60 0x7f407c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053975514 0 0) 0x7f407c0b2f00 con 0x7f407c15dc20 2023-10-16T09:59:44.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40680035b0 con 0x7f407c15dc20 2023-10-16T09:59:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2000999903 0 0) 0x7f40680035b0 con 0x7f407c15dc20 2023-10-16T09:59:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 >> v2:172.21.15.190:3300/0 conn(0x7f407c0d1e60 msgr2=0x7f407c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 --2- 172.21.15.181:0/467746059 >> v2:172.21.15.190:3300/0 conn(0x7f407c0d1e60 0x7f407c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 >> v1:172.21.15.190:6789/0 conn(0x7f407c0b3fa0 legacy=0x7f407c0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0b34c0 con 0x7f407c15dc20 2023-10-16T09:59:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f40780017f0 con 0x7f407c15dc20 2023-10-16T09:59:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 >> v1:172.21.15.181:6789/0 conn(0x7f407c15dc20 legacy=0x7f407c0d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4068003a00 0x7f4068003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4068004850 0x7f4068004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068005210 0x7f406800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0b3b80 con 0x7f4068005210 2023-10-16T09:59:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406800c0b0 con 0x7f4068003a00 2023-10-16T09:59:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406800c270 con 0x7f4068004850 2023-10-16T09:59:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f40817b6700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4068003a00 0x7f4068003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4081fb7700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068005210 0x7f406800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4080fb5700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4068004850 0x7f4068004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f40817b6700 1 -- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4068004850 msgr2=0x7f4068004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:44.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f40817b6700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4068004850 0x7f4068004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f40817b6700 1 -- 172.21.15.181:0/467746059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068005210 msgr2=0x7f406800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:44.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f40817b6700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068005210 0x7f406800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f40817b6700 1 -- 172.21.15.181:0/467746059 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680035b0 con 0x7f4068003a00 2023-10-16T09:59:44.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4081fb7700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068005210 0x7f406800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:44.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.548+0000 7f4081fb7700 1 -- 172.21.15.181:0/467746059 reap_dead start 2023-10-16T09:59:44.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f40817b6700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4068003a00 0x7f4068003e20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f407800c240 tx=0x7f4068005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e567bc2c654aa62a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:44.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407800cde0 con 0x7f4068003a00 2023-10-16T09:59:44.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078007b80 con 0x7f4068003a00 2023-10-16T09:59:44.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4073fff700 1 -- 172.21.15.181:0/467746059 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780072f0 con 0x7f4068003a00 2023-10-16T09:59:44.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4083a1a700 1 -- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4068003a00 msgr2=0x7f4068003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4083a1a700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4068003a00 0x7f4068003e20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f407800c240 tx=0x7f4068005880 comp rx=0 tx=0).stop 2023-10-16T09:59:44.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4083a1a700 1 -- 172.21.15.181:0/467746059 shutdown_connections 2023-10-16T09:59:44.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4083a1a700 1 --2- 172.21.15.181:0/467746059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4068003a00 0x7f4068003e20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4083a1a700 1 -- 172.21.15.181:0/467746059 >> 172.21.15.181:0/467746059 conn(0x7f407c08af90 msgr2=0x7f407c0d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:44.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4083a1a700 1 -- 172.21.15.181:0/467746059 shutdown_connections 2023-10-16T09:59:44.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.549+0000 7f4083a1a700 1 -- 172.21.15.181:0/467746059 wait complete. 2023-10-16T09:59:44.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 Processor -- start 2023-10-16T09:59:44.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 -- start start 2023-10-16T09:59:44.561 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f60d4a83200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T09:59:44.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068003a00 0x7f407c1f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f407c0cc150 0x7f407c142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f407c143200 0x7f407c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c143c60 con 0x7f4068003a00 2023-10-16T09:59:44.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c1436a0 con 0x7f407c143200 2023-10-16T09:59:44.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f4083a1a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c143980 con 0x7f407c0cc150 2023-10-16T09:59:44.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f40817b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068003a00 0x7f407c1f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f40817b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068003a00 0x7f407c1f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42606/0 (socket says 172.21.15.181:42606) 2023-10-16T09:59:44.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.551+0000 7f40817b6700 1 -- 172.21.15.181:0/3995493626 learned_addr learned my addr 172.21.15.181:0/3995493626 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:44.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f4080fb5700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f407c0cc150 0x7f407c142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f4081fb7700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f407c143200 0x7f407c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f40817b6700 1 -- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f407c0cc150 msgr2=0x7f407c142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f40817b6700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f407c0cc150 0x7f407c142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f40817b6700 1 -- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f407c143200 msgr2=0x7f407c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f40817b6700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f407c143200 0x7f407c0b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f40817b6700 1 -- 172.21.15.181:0/3995493626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c1f66d0 con 0x7f4068003a00 2023-10-16T09:59:44.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f40817b6700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068003a00 0x7f407c1f6ed0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4078007d30 tx=0x7f4078001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:44.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780074a0 con 0x7f4068003a00 2023-10-16T09:59:44.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078007600 con 0x7f4068003a00 2023-10-16T09:59:44.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c1f69b0 con 0x7f4068003a00 2023-10-16T09:59:44.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078005a70 con 0x7f4068003a00 2023-10-16T09:59:44.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.552+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c152e70 con 0x7f4068003a00 2023-10-16T09:59:44.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.553+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c153060 con 0x7f4068003a00 2023-10-16T09:59:44.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.554+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 38) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4078005bd0 con 0x7f4068003a00 2023-10-16T09:59:44.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.554+0000 7f4071ffb700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f406006b020 0x7f406006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.554+0000 7f4080fb5700 1 -- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f406006b020 msgr2=0x7f406006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/420719762 2023-10-16T09:59:44.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.554+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f407808a720 con 0x7f4068003a00 2023-10-16T09:59:44.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.554+0000 7f4080fb5700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f406006b020 0x7f406006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T09:59:44.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.558+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f407c153060 con 0x7f4068003a00 2023-10-16T09:59:44.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.576+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 39) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4078055820 con 0x7f4068003a00 2023-10-16T09:59:44.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.576+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f406006b020 msgr2=0x7f406006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T09:59:44.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.576+0000 7f4071ffb700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f406006b020 0x7f406006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.625 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f605026f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.625 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f605026f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.626 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f605026f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.626 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f605026f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.626 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f605026f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.626 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f604d269700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.626 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f604d269700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.627 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f604d269700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.627 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f604d269700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.627 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T09:59:44.623+0000 7f604d269700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T09:59:44.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.812+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f407c1f6530 con 0x7f4068003a00 2023-10-16T09:59:44.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.813+0000 7f4071ffb700 1 -- 172.21.15.181:0/3995493626 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f407c1f6530 con 0x7f4068003a00 2023-10-16T09:59:44.815 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:44.815 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2852689416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2852689416},{"type":"v1","addr":"172.21.15.190:6834","nonce":2852689416}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2993219349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2993219349},{"type":"v1","addr":"172.21.15.181:6836","nonce":2993219349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2504788411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2504788411},{"type":"v1","addr":"172.21.15.181:6837","nonce":2504788411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:23.239300+0000","modified":"2023-10-16T09:59:26.271260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1948728404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1948728404},{"type":"v1","addr":"172.21.15.190:6835","nonce":1948728404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-16T09:59:44.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068003a00 msgr2=0x7f407c1f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068003a00 0x7f407c1f6ed0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4078007d30 tx=0x7f4078001b80 comp rx=0 tx=0).stop 2023-10-16T09:59:44.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 shutdown_connections 2023-10-16T09:59:44.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:6832/420719762,v1:172.21.15.181:6833/420719762] conn(0x7f406006b020 0x7f406006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f407c143200 0x7f407c0b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4068003a00 0x7f407c1f6ed0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 --2- 172.21.15.181:0/3995493626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f407c0cc150 0x7f407c142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 >> 172.21.15.181:0/3995493626 conn(0x7f407c08af90 msgr2=0x7f407c0b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:44.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 shutdown_connections 2023-10-16T09:59:44.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.815+0000 7f4083a1a700 1 -- 172.21.15.181:0/3995493626 wait complete. 2023-10-16T09:59:44.819 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 32 2023-10-16T09:59:44.828 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:44.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.947+0000 7f30a597e700 1 Processor -- start 2023-10-16T09:59:44.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.947+0000 7f30a597e700 1 -- start start 2023-10-16T09:59:44.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.947+0000 7f30a597e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161b60 0x7f30a006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.947+0000 7f30a597e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f30a0161f40 0x7f30a00d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f30a597e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a00d2560 con 0x7f30a0160d20 2023-10-16T09:59:44.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f30a597e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a0162800 con 0x7f30a0161f40 2023-10-16T09:59:44.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f30a597e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00d43e0 con 0x7f30a0161b60 2023-10-16T09:59:44.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f30a0161f40 0x7f30a00d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f30a0161f40 0x7f30a00d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42690/0 (socket says 172.21.15.181:42690) 2023-10-16T09:59:44.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 -- 172.21.15.181:0/1063604648 learned_addr learned my addr 172.21.15.181:0/1063604648 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:44.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309f7fe700 1 --1- 172.21.15.181:0/1063604648 >> v1:172.21.15.181:6789/0 conn(0x7f30a0160d20 0x7f30a00b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53766/0 (socket says 172.21.15.181:53766) 2023-10-16T09:59:44.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309effd700 1 --2- 172.21.15.181:0/1063604648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161b60 0x7f30a006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 -- 172.21.15.181:0/1063604648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161b60 msgr2=0x7f30a006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:44.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 --2- 172.21.15.181:0/1063604648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161b60 0x7f30a006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 -- 172.21.15.181:0/1063604648 >> v1:172.21.15.181:6789/0 conn(0x7f30a0160d20 legacy=0x7f30a00b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 -- 172.21.15.181:0/1063604648 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a006ce20 con 0x7f30a0161f40 2023-10-16T09:59:44.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309e7fc700 1 --2- 172.21.15.181:0/1063604648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0161f40 0x7f30a00d2020 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f30940098f0 tx=0x7f3094009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deab804420399632 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:44.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309dffb700 1 -- 172.21.15.181:0/1063604648 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094011830 con 0x7f30a0161f40 2023-10-16T09:59:44.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309dffb700 1 -- 172.21.15.181:0/1063604648 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3094011990 con 0x7f30a0161f40 2023-10-16T09:59:44.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.948+0000 7f309dffb700 1 -- 172.21.15.181:0/1063604648 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094011d20 con 0x7f30a0161f40 2023-10-16T09:59:44.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 -- 172.21.15.181:0/1063604648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0161f40 msgr2=0x7f30a00d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 --2- 172.21.15.181:0/1063604648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0161f40 0x7f30a00d2020 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f30940098f0 tx=0x7f3094009c00 comp rx=0 tx=0).stop 2023-10-16T09:59:44.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 -- 172.21.15.181:0/1063604648 shutdown_connections 2023-10-16T09:59:44.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 --2- 172.21.15.181:0/1063604648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161b60 0x7f30a006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 --2- 172.21.15.181:0/1063604648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0161f40 0x7f30a00d2020 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 -- 172.21.15.181:0/1063604648 >> 172.21.15.181:0/1063604648 conn(0x7f30a009cba0 msgr2=0x7f30a00d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:44.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 -- 172.21.15.181:0/1063604648 shutdown_connections 2023-10-16T09:59:44.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.949+0000 7f30a597e700 1 -- 172.21.15.181:0/1063604648 wait complete. 2023-10-16T09:59:44.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.950+0000 7f30a597e700 1 Processor -- start 2023-10-16T09:59:44.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.950+0000 7f30a597e700 1 -- start start 2023-10-16T09:59:44.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f30a597e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0160d20 0x7f30a00b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f30a597e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a0161b60 0x7f30a015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f30a597e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161f40 0x7f30a00b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:44.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f30a597e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00b9150 con 0x7f30a0160d20 2023-10-16T09:59:44.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f30a597e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00ccac0 con 0x7f30a0161b60 2023-10-16T09:59:44.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f30a597e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00ccda0 con 0x7f30a0161f40 2023-10-16T09:59:44.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0160d20 0x7f30a00b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0160d20 0x7f30a00b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42692/0 (socket says 172.21.15.181:42692) 2023-10-16T09:59:44.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 -- 172.21.15.181:0/347248962 learned_addr learned my addr 172.21.15.181:0/347248962 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:44.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309e7fc700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a0161b60 0x7f30a015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309f7fe700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161f40 0x7f30a00b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:44.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 -- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161f40 msgr2=0x7f30a00b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161f40 0x7f30a00b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 -- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a0161b60 msgr2=0x7f30a015b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:44.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a0161b60 0x7f30a015b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:44.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.951+0000 7f309effd700 1 -- 172.21.15.181:0/347248962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3094009550 con 0x7f30a0160d20 2023-10-16T09:59:44.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.952+0000 7f309effd700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0160d20 0x7f30a00b7e80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f309000e900 tx=0x7f309000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:44.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.952+0000 7f3087fff700 1 -- 172.21.15.181:0/347248962 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090009650 con 0x7f30a0160d20 2023-10-16T09:59:44.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.952+0000 7f3087fff700 1 -- 172.21.15.181:0/347248962 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309000c850 con 0x7f30a0160d20 2023-10-16T09:59:44.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.952+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a00cd080 con 0x7f30a0160d20 2023-10-16T09:59:44.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.952+0000 7f3087fff700 1 -- 172.21.15.181:0/347248962 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309000cbc0 con 0x7f30a0160d20 2023-10-16T09:59:44.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.952+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a00cd640 con 0x7f30a0160d20 2023-10-16T09:59:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.953+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a004ebb0 con 0x7f30a0160d20 2023-10-16T09:59:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.953+0000 7f3087fff700 1 -- 172.21.15.181:0/347248962 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 39) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f309000cd20 con 0x7f30a0160d20 2023-10-16T09:59:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.953+0000 7f3087fff700 1 -- 172.21.15.181:0/347248962 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f30900898a0 con 0x7f30a0160d20 2023-10-16T09:59:44.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:44.958+0000 7f3087fff700 1 -- 172.21.15.181:0/347248962 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3090054940 con 0x7f30a0160d20 2023-10-16T09:59:45.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.149+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30a006d2a0 con 0x7f30a0160d20 2023-10-16T09:59:45.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.150+0000 7f3087fff700 1 -- 172.21.15.181:0/347248962 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18258 (secure 0 0 0) 0x7f3090017090 con 0x7f30a0160d20 2023-10-16T09:59:45.152 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:45.153 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":78,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:44.566241+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:21.812039+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:22.200991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1343774933":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/1815471563":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/52391112":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/3113822369":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/2394784404":"2023-10-17T09:59:44.566188+0000","172.21.15.181:6833/420719762":"2023-10-17T09:59:44.566188+0000","172.21.15.181:6832/420719762":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/75988000":"2023-10-17T09:59:44.566188+0000","172.21.15.181:6833/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:6832/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/1081249734":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2658775934":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2227004797":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3531530097":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2146555244":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3441419577":"2023-10-17T09:59:37.126305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:45.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0160d20 msgr2=0x7f30a00b7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0160d20 0x7f30a00b7e80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f309000e900 tx=0x7f309000ecc0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 shutdown_connections 2023-10-16T09:59:45.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a0161f40 0x7f30a00b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0160d20 0x7f30a00b7e80 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 --2- 172.21.15.181:0/347248962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a0161b60 0x7f30a015b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 >> 172.21.15.181:0/347248962 conn(0x7f30a009cba0 msgr2=0x7f30a0073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 shutdown_connections 2023-10-16T09:59:45.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.152+0000 7f30a597e700 1 -- 172.21.15.181:0/347248962 wait complete. 2023-10-16T09:59:45.167 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T09:59:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.285+0000 7fbe2853b700 1 Processor -- start 2023-10-16T09:59:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.285+0000 7fbe2853b700 1 -- start start 2023-10-16T09:59:45.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe2853b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe2006d210 0x7fbe200cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe2853b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe200ce4d0 0x7fbe2015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe2853b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe200cf460 con 0x7fbe200cc4c0 2023-10-16T09:59:45.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe2853b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe200d0380 con 0x7fbe200ce4d0 2023-10-16T09:59:45.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe2853b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe200cc940 con 0x7fbe2006d210 2023-10-16T09:59:45.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe26ad8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe200ce4d0 0x7fbe2015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe26ad8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe200ce4d0 0x7fbe2015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42698/0 (socket says 172.21.15.181:42698) 2023-10-16T09:59:45.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe26ad8700 1 -- 172.21.15.181:0/2483846679 learned_addr learned my addr 172.21.15.181:0/2483846679 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:45.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe262d7700 1 --2- 172.21.15.181:0/2483846679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe2006d210 0x7fbe200cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe26ad8700 1 -- 172.21.15.181:0/2483846679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe2006d210 msgr2=0x7fbe200cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:45.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe26ad8700 1 --2- 172.21.15.181:0/2483846679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe2006d210 0x7fbe200cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe26ad8700 1 -- 172.21.15.181:0/2483846679 >> v1:172.21.15.181:6789/0 conn(0x7fbe200cc4c0 legacy=0x7fbe200cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe26ad8700 1 -- 172.21.15.181:0/2483846679 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2006c700 con 0x7fbe200ce4d0 2023-10-16T09:59:45.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.286+0000 7fbe252d5700 1 -- 172.21.15.181:0/2483846679 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997708595 0 0) 0x7fbe200cf460 con 0x7fbe200cc4c0 2023-10-16T09:59:45.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe26ad8700 1 --2- 172.21.15.181:0/2483846679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe200ce4d0 0x7fbe2015d2f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009e50 tx=0x7fbe1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62deed6ca7f0d486 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:45.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe252d5700 1 -- 172.21.15.181:0/2483846679 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800e070 con 0x7fbe200ce4d0 2023-10-16T09:59:45.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe252d5700 1 -- 172.21.15.181:0/2483846679 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1800b910 con 0x7fbe200ce4d0 2023-10-16T09:59:45.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe252d5700 1 -- 172.21.15.181:0/2483846679 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800bcc0 con 0x7fbe200ce4d0 2023-10-16T09:59:45.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 -- 172.21.15.181:0/2483846679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe200ce4d0 msgr2=0x7fbe2015d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 --2- 172.21.15.181:0/2483846679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe200ce4d0 0x7fbe2015d2f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009e50 tx=0x7fbe1800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:45.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 -- 172.21.15.181:0/2483846679 shutdown_connections 2023-10-16T09:59:45.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 --2- 172.21.15.181:0/2483846679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe200ce4d0 0x7fbe2015d2f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 --2- 172.21.15.181:0/2483846679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe2006d210 0x7fbe200cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 -- 172.21.15.181:0/2483846679 >> 172.21.15.181:0/2483846679 conn(0x7fbe2009a9a0 msgr2=0x7fbe200d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:45.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 -- 172.21.15.181:0/2483846679 shutdown_connections 2023-10-16T09:59:45.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.287+0000 7fbe2853b700 1 -- 172.21.15.181:0/2483846679 wait complete. 2023-10-16T09:59:45.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 Processor -- start 2023-10-16T09:59:45.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 -- start start 2023-10-16T09:59:45.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2006d210 0x7fbe201556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe200cc4c0 0x7fbe201517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe200ce4d0 0x7fbe200ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2014f990 con 0x7fbe2006d210 2023-10-16T09:59:45.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20151ce0 con 0x7fbe200ce4d0 2023-10-16T09:59:45.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe2853b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20151f10 con 0x7fbe200cc4c0 2023-10-16T09:59:45.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe262d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2006d210 0x7fbe201556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe262d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2006d210 0x7fbe201556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42702/0 (socket says 172.21.15.181:42702) 2023-10-16T09:59:45.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe262d7700 1 -- 172.21.15.181:0/1937121102 learned_addr learned my addr 172.21.15.181:0/1937121102 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:45.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe26ad8700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe200ce4d0 0x7fbe200ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.289+0000 7fbe25ad6700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe200cc4c0 0x7fbe201517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe262d7700 1 -- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe200cc4c0 msgr2=0x7fbe201517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe262d7700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe200cc4c0 0x7fbe201517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe262d7700 1 -- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe200ce4d0 msgr2=0x7fbe200ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe262d7700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe200ce4d0 0x7fbe200ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe262d7700 1 -- 172.21.15.181:0/1937121102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18009b30 con 0x7fbe2006d210 2023-10-16T09:59:45.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe262d7700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2006d210 0x7fbe201556d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e880 tx=0x7fbe1000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:45.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe16ffd700 1 -- 172.21.15.181:0/1937121102 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10013680 con 0x7fbe2006d210 2023-10-16T09:59:45.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe16ffd700 1 -- 172.21.15.181:0/1937121102 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe10013800 con 0x7fbe2006d210 2023-10-16T09:59:45.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe16ffd700 1 -- 172.21.15.181:0/1937121102 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10013bb0 con 0x7fbe2006d210 2023-10-16T09:59:45.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe201521f0 con 0x7fbe2006d210 2023-10-16T09:59:45.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.290+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe200aea40 con 0x7fbe2006d210 2023-10-16T09:59:45.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.291+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe20152710 con 0x7fbe2006d210 2023-10-16T09:59:45.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.291+0000 7fbe16ffd700 1 -- 172.21.15.181:0/1937121102 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 39) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fbe10013d10 con 0x7fbe2006d210 2023-10-16T09:59:45.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.292+0000 7fbe16ffd700 1 -- 172.21.15.181:0/1937121102 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fbe10089300 con 0x7fbe2006d210 2023-10-16T09:59:45.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.296+0000 7fbe16ffd700 1 -- 172.21.15.181:0/1937121102 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe100543a0 con 0x7fbe2006d210 2023-10-16T09:59:45.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.532+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbe20067d20 con 0x7fbe2006d210 2023-10-16T09:59:45.534 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T09:59:45.532+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T09:59:45.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.534+0000 7fbe16ffd700 1 -- 172.21.15.181:0/1937121102 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbe100579c0 con 0x7fbe2006d210 2023-10-16T09:59:45.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2006d210 msgr2=0x7fbe201556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2006d210 0x7fbe201556d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e880 tx=0x7fbe1000eb90 comp rx=0 tx=0).stop 2023-10-16T09:59:45.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 shutdown_connections 2023-10-16T09:59:45.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe200ce4d0 0x7fbe200ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2006d210 0x7fbe201556d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 --2- 172.21.15.181:0/1937121102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe200cc4c0 0x7fbe201517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 >> 172.21.15.181:0/1937121102 conn(0x7fbe2009a9a0 msgr2=0x7fbe200c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:45.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 shutdown_connections 2023-10-16T09:59:45.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.537+0000 7fbe2853b700 1 -- 172.21.15.181:0/1937121102 wait complete. 2023-10-16T09:59:45.541 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T09:59:45.541 INFO:tasks.ceph.mds.d.smithi190.stderr: -5096> 2023-10-16T09:59:24.832+0000 7f294453d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T09:59:45.551 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T09:59:45.572 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T09:59:45.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.678+0000 7fcb5e3b5700 1 Processor -- start 2023-10-16T09:59:45.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.678+0000 7fcb5e3b5700 1 -- start start 2023-10-16T09:59:45.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb5e3b5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb580cc6e0 0x7fcb58071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb5e3b5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580ce680 0x7fcb580ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb5e3b5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb580ce1f0 con 0x7fcb580b2010 2023-10-16T09:59:45.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb5e3b5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb580ccb90 con 0x7fcb580cc6e0 2023-10-16T09:59:45.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb5e3b5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58072120 con 0x7fcb580ce680 2023-10-16T09:59:45.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb57fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcb580b2010 0x7fcb580b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53808/0 (socket says 172.21.15.181:53808) 2023-10-16T09:59:45.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb57fff700 1 -- 172.21.15.181:0/497236980 learned_addr learned my addr 172.21.15.181:0/497236980 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:45.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb577fe700 1 --2- 172.21.15.181:0/497236980 >> v2:172.21.15.190:3300/0 conn(0x7fcb580cc6e0 0x7fcb58071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb5c952700 1 --2- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580ce680 0x7fcb580ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb56ffd700 1 -- 172.21.15.181:0/497236980 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377892318 0 0) 0x7fcb580ce1f0 con 0x7fcb580b2010 2023-10-16T09:59:45.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.679+0000 7fcb56ffd700 1 -- 172.21.15.181:0/497236980 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb400057c0 con 0x7fcb580b2010 2023-10-16T09:59:45.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb577fe700 1 -- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580ce680 msgr2=0x7fcb580ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:45.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb577fe700 1 --2- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580ce680 0x7fcb580ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb577fe700 1 -- 172.21.15.181:0/497236980 >> v1:172.21.15.181:6789/0 conn(0x7fcb580b2010 legacy=0x7fcb580b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb577fe700 1 -- 172.21.15.181:0/497236980 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb580ccda0 con 0x7fcb580cc6e0 2023-10-16T09:59:45.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb577fe700 1 --2- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 0x7fcb58071be0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009840 tx=0x7fcb48009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d85edaa11565951 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:45.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb56ffd700 1 -- 172.21.15.181:0/497236980 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48011830 con 0x7fcb580cc6e0 2023-10-16T09:59:45.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb56ffd700 1 -- 172.21.15.181:0/497236980 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48011990 con 0x7fcb580cc6e0 2023-10-16T09:59:45.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.680+0000 7fcb56ffd700 1 -- 172.21.15.181:0/497236980 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48011d20 con 0x7fcb580cc6e0 2023-10-16T09:59:45.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 msgr2=0x7fcb58071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 0x7fcb58071be0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009840 tx=0x7fcb48009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:45.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/497236980 shutdown_connections 2023-10-16T09:59:45.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580ce680 0x7fcb580ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/497236980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 0x7fcb58071be0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/497236980 >> 172.21.15.181:0/497236980 conn(0x7fcb580cd010 msgr2=0x7fcb580cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:45.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/497236980 shutdown_connections 2023-10-16T09:59:45.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.681+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/497236980 wait complete. 2023-10-16T09:59:45.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.682+0000 7fcb5e3b5700 1 Processor -- start 2023-10-16T09:59:45.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.682+0000 7fcb5e3b5700 1 -- start start 2023-10-16T09:59:45.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5e3b5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b2010 0x7fcb580af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5e3b5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 0x7fcb580b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5e3b5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb580ce680 0x7fcb580c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5e3b5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580b3090 con 0x7fcb580ce680 2023-10-16T09:59:45.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5e3b5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580ca500 con 0x7fcb580cc6e0 2023-10-16T09:59:45.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5e3b5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580b32a0 con 0x7fcb580b2010 2023-10-16T09:59:45.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb580ce680 0x7fcb580c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb580ce680 0x7fcb580c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42740/0 (socket says 172.21.15.181:42740) 2023-10-16T09:59:45.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 -- 172.21.15.181:0/4083892813 learned_addr learned my addr 172.21.15.181:0/4083892813 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:45.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb577fe700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 0x7fcb580b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb57fff700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b2010 0x7fcb580af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 -- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b2010 msgr2=0x7fcb580af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b2010 0x7fcb580af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 -- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 msgr2=0x7fcb580b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:45.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 0x7fcb580b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:45.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 -- 172.21.15.181:0/4083892813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480094a0 con 0x7fcb580ce680 2023-10-16T09:59:45.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb57fff700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b2010 0x7fcb580af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:45.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb5c952700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb580ce680 0x7fcb580c7a20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000efc0 tx=0x7fcb5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:45.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb54ff9700 1 -- 172.21.15.181:0/4083892813 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50009ea0 con 0x7fcb580ce680 2023-10-16T09:59:45.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.683+0000 7fcb54ff9700 1 -- 172.21.15.181:0/4083892813 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5000f040 con 0x7fcb580ce680 2023-10-16T09:59:45.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.684+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb580b3580 con 0x7fcb580ce680 2023-10-16T09:59:45.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.684+0000 7fcb54ff9700 1 -- 172.21.15.181:0/4083892813 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50014480 con 0x7fcb580ce680 2023-10-16T09:59:45.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.684+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb580b3b40 con 0x7fcb580ce680 2023-10-16T09:59:45.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.684+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb580b6510 con 0x7fcb580ce680 2023-10-16T09:59:45.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.685+0000 7fcb54ff9700 1 -- 172.21.15.181:0/4083892813 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 40) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcb500145e0 con 0x7fcb580ce680 2023-10-16T09:59:45.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.685+0000 7fcb54ff9700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcb4406ad50 0x7fcb4406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:45.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.685+0000 7fcb54ff9700 1 -- 172.21.15.181:0/4083892813 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6907+0+0 (secure 0 0 0) 0x7fcb50089fa0 con 0x7fcb580ce680 2023-10-16T09:59:45.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.685+0000 7fcb57fff700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcb4406ad50 0x7fcb4406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:45.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.686+0000 7fcb57fff700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcb4406ad50 0x7fcb4406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0011e0 tx=0x7fcb4c009040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:45.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.690+0000 7fcb54ff9700 1 -- 172.21.15.181:0/4083892813 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcb500bcfe0 con 0x7fcb580ce680 2023-10-16T09:59:45.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:45.926+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb580b6160 con 0x7fcb580ce680 2023-10-16T09:59:46.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.582+0000 7fcb54ff9700 1 -- 172.21.15.181:0/4083892813 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcb50054ee0 con 0x7fcb580ce680 2023-10-16T09:59:46.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcb4406ad50 msgr2=0x7fcb4406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:46.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcb4406ad50 0x7fcb4406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0011e0 tx=0x7fcb4c009040 comp rx=0 tx=0).stop 2023-10-16T09:59:46.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb580ce680 msgr2=0x7fcb580c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:46.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb580ce680 0x7fcb580c7a20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000efc0 tx=0x7fcb5000c3a0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 shutdown_connections 2023-10-16T09:59:46.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcb4406ad50 0x7fcb4406d210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b2010 0x7fcb580af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.586+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb580ce680 0x7fcb580c7a20 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.587+0000 7fcb5e3b5700 1 --2- 172.21.15.181:0/4083892813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580cc6e0 0x7fcb580b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.587+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 >> 172.21.15.181:0/4083892813 conn(0x7fcb580cd010 msgr2=0x7fcb580cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:46.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.587+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 shutdown_connections 2023-10-16T09:59:46.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.587+0000 7fcb5e3b5700 1 -- 172.21.15.181:0/4083892813 wait complete. 2023-10-16T09:59:46.603 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T09:59:46.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.734+0000 7f24c87ac700 1 Processor -- start 2023-10-16T09:59:46.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.734+0000 7f24c87ac700 1 -- start start 2023-10-16T09:59:46.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c87ac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c0160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:46.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c87ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24c00d03a0 0x7f24c00b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:46.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c87ac700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c006ce60 con 0x7f24c00cff20 2023-10-16T09:59:46.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c87ac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c00d1020 con 0x7f24c00d03a0 2023-10-16T09:59:46.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c87ac700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c006d2f0 con 0x7f24c01608e0 2023-10-16T09:59:46.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c6d49700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24c00d03a0 0x7f24c00b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:46.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c6d49700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24c00d03a0 0x7f24c00b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42766/0 (socket says 172.21.15.181:42766) 2023-10-16T09:59:46.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.735+0000 7f24c6d49700 1 -- 172.21.15.181:0/3538038875 learned_addr learned my addr 172.21.15.181:0/3538038875 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:46.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c6548700 1 --2- 172.21.15.181:0/3538038875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c0160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:46.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c6d49700 1 -- 172.21.15.181:0/3538038875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 msgr2=0x7f24c0160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:46.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c6d49700 1 --2- 172.21.15.181:0/3538038875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c0160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c6d49700 1 -- 172.21.15.181:0/3538038875 >> v1:172.21.15.190:6789/0 conn(0x7f24c00cff20 legacy=0x7f24c00b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:46.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c6d49700 1 -- 172.21.15.181:0/3538038875 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00b2f20 con 0x7f24c00d03a0 2023-10-16T09:59:46.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c6d49700 1 --2- 172.21.15.181:0/3538038875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c00d03a0 0x7f24c00b0f70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009e50 tx=0x7f24b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b4c8cb51033de1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:46.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c5546700 1 -- 172.21.15.181:0/3538038875 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800e070 con 0x7f24c00d03a0 2023-10-16T09:59:46.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c5546700 1 -- 172.21.15.181:0/3538038875 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8003a20 con 0x7f24c00d03a0 2023-10-16T09:59:46.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.736+0000 7f24c5546700 1 -- 172.21.15.181:0/3538038875 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8003dd0 con 0x7f24c00d03a0 2023-10-16T09:59:46.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 -- 172.21.15.181:0/3538038875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c00d03a0 msgr2=0x7f24c00b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:46.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3538038875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c00d03a0 0x7f24c00b0f70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009e50 tx=0x7f24b800b330 comp rx=0 tx=0).stop 2023-10-16T09:59:46.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 -- 172.21.15.181:0/3538038875 shutdown_connections 2023-10-16T09:59:46.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3538038875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c00d03a0 0x7f24c00b0f70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3538038875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c0160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 -- 172.21.15.181:0/3538038875 >> 172.21.15.181:0/3538038875 conn(0x7f24c00d1610 msgr2=0x7f24c00d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:46.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 -- 172.21.15.181:0/3538038875 shutdown_connections 2023-10-16T09:59:46.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.737+0000 7f24c87ac700 1 -- 172.21.15.181:0/3538038875 wait complete. 2023-10-16T09:59:46.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.738+0000 7f24c87ac700 1 Processor -- start 2023-10-16T09:59:46.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.738+0000 7f24c87ac700 1 -- start start 2023-10-16T09:59:46.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00cff20 0x7f24c00b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:46.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c00c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:46.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0159de0 0x7f24c00b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:46.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00b70e0 con 0x7f24c0159de0 2023-10-16T09:59:46.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00b57c0 con 0x7f24c00cff20 2023-10-16T09:59:46.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015a270 con 0x7f24c01608e0 2023-10-16T09:59:46.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0159de0 0x7f24c00b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:46.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0159de0 0x7f24c00b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42772/0 (socket says 172.21.15.181:42772) 2023-10-16T09:59:46.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 -- 172.21.15.181:0/3900070672 learned_addr learned my addr 172.21.15.181:0/3900070672 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:46.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6548700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00cff20 0x7f24c00b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:46.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c5d47700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c00c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:46.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 -- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 msgr2=0x7f24c00c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:46.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c00c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 -- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00cff20 msgr2=0x7f24c00b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00cff20 0x7f24c00b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 -- 172.21.15.181:0/3900070672 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8009b30 con 0x7f24c0159de0 2023-10-16T09:59:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c6d49700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0159de0 0x7f24c00b68b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f24b8015040 tx=0x7f24b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8004440 con 0x7f24c0159de0 2023-10-16T09:59:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b80045a0 con 0x7f24c0159de0 2023-10-16T09:59:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8004950 con 0x7f24c0159de0 2023-10-16T09:59:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c00b4e90 con 0x7f24c0159de0 2023-10-16T09:59:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.739+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c00b5450 con 0x7f24c0159de0 2023-10-16T09:59:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.740+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c00b5610 con 0x7f24c0159de0 2023-10-16T09:59:46.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.741+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 40) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f24b8012430 con 0x7f24c0159de0 2023-10-16T09:59:46.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.741+0000 7f24b6ffd700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f24a406ad50 0x7f24a406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:46.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.741+0000 7f24c6548700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f24a406ad50 0x7f24a406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:46.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.741+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6907+0+0 (secure 0 0 0) 0x7f24b808ab60 con 0x7f24c0159de0 2023-10-16T09:59:46.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.742+0000 7f24c6548700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f24a406ad50 0x7f24a406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24c00b4780 tx=0x7f24b0008040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:46.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.746+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f24b8059530 con 0x7f24c0159de0 2023-10-16T09:59:46.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:46.940+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24c0067d20 con 0x7f24c0159de0 2023-10-16T09:59:47.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.585+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 41) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f24b8055e70 con 0x7f24c0159de0 2023-10-16T09:59:47.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.592+0000 7f24b6ffd700 1 -- 172.21.15.181:0/3900070672 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7f24b8084f60 con 0x7f24c0159de0 2023-10-16T09:59:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.595+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f24a406ad50 msgr2=0x7f24a406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.595+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f24a406ad50 0x7f24a406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24c00b4780 tx=0x7f24b0008040 comp rx=0 tx=0).stop 2023-10-16T09:59:47.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.595+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0159de0 msgr2=0x7f24c00b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:47.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.595+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0159de0 0x7f24c00b68b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f24b8015040 tx=0x7f24b8004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 shutdown_connections 2023-10-16T09:59:47.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f24a406ad50 0x7f24a406d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0159de0 0x7f24c00b68b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00cff20 0x7f24c00b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 --2- 172.21.15.181:0/3900070672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c01608e0 0x7f24c00c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 >> 172.21.15.181:0/3900070672 conn(0x7f24c00d1610 msgr2=0x7f24c00b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:47.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 shutdown_connections 2023-10-16T09:59:47.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.596+0000 7f24c87ac700 1 -- 172.21.15.181:0/3900070672 wait complete. 2023-10-16T09:59:47.600 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T09:59:47.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T09:59:47.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.739+0000 7f94fd275700 1 Processor -- start 2023-10-16T09:59:47.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.739+0000 7f94fd275700 1 -- start start 2023-10-16T09:59:47.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.740+0000 7f94fd275700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94f806afb0 0x7f94f806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.740+0000 7f94fd275700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f8162650 con 0x7f94f80d0fc0 2023-10-16T09:59:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.740+0000 7f94fd275700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f80d0230 con 0x7f94f80ce610 2023-10-16T09:59:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.740+0000 7f94fd275700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f8162190 con 0x7f94f806afb0 2023-10-16T09:59:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.740+0000 7f94f77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94f806afb0 0x7f94f806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.740+0000 7f94f67fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f94f80d0fc0 0x7f94f806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53856/0 (socket says 172.21.15.181:53856) 2023-10-16T09:59:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.740+0000 7f94f67fc700 1 -- 172.21.15.181:0/3408278391 learned_addr learned my addr 172.21.15.181:0/3408278391 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94f77fe700 1 -- 172.21.15.181:0/3408278391 >> v1:172.21.15.190:6789/0 conn(0x7f94f80ce610 legacy=0x7f94f80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94f77fe700 1 -- 172.21.15.181:0/3408278391 >> v1:172.21.15.181:6789/0 conn(0x7f94f80d0fc0 legacy=0x7f94f806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94f77fe700 1 -- 172.21.15.181:0/3408278391 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8162370 con 0x7f94f806afb0 2023-10-16T09:59:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94f77fe700 1 --2- 172.21.15.181:0/3408278391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f806afb0 0x7f94f806d660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009d90 tx=0x7f94f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900098446e19708e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94f5ffb700 1 -- 172.21.15.181:0/3408278391 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f000e070 con 0x7f94f806afb0 2023-10-16T09:59:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94f5ffb700 1 -- 172.21.15.181:0/3408278391 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f000bbd0 con 0x7f94f806afb0 2023-10-16T09:59:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94f5ffb700 1 -- 172.21.15.181:0/3408278391 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0003a20 con 0x7f94f806afb0 2023-10-16T09:59:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94fd275700 1 -- 172.21.15.181:0/3408278391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f806afb0 msgr2=0x7f94f806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94fd275700 1 --2- 172.21.15.181:0/3408278391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f806afb0 0x7f94f806d660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009d90 tx=0x7f94f000b330 comp rx=0 tx=0).stop 2023-10-16T09:59:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94fd275700 1 -- 172.21.15.181:0/3408278391 shutdown_connections 2023-10-16T09:59:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94fd275700 1 --2- 172.21.15.181:0/3408278391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f806afb0 0x7f94f806d660 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94fd275700 1 -- 172.21.15.181:0/3408278391 >> 172.21.15.181:0/3408278391 conn(0x7f94f80d04a0 msgr2=0x7f94f80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.741+0000 7f94fd275700 1 -- 172.21.15.181:0/3408278391 shutdown_connections 2023-10-16T09:59:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.742+0000 7f94fd275700 1 -- 172.21.15.181:0/3408278391 wait complete. 2023-10-16T09:59:47.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.743+0000 7f94fd275700 1 Processor -- start 2023-10-16T09:59:47.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.743+0000 7f94fd275700 1 -- start start 2023-10-16T09:59:47.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94fd275700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94f806afb0 0x7f94f814fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:47.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94fd275700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94f80ce610 0x7f94f81606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:47.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94fd275700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f80d0fc0 0x7f94f8160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:47.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94fd275700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f809adc0 con 0x7f94f80d0fc0 2023-10-16T09:59:47.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94fd275700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f809a220 con 0x7f94f806afb0 2023-10-16T09:59:47.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94fd275700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f809b250 con 0x7f94f80ce610 2023-10-16T09:59:47.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f80d0fc0 0x7f94f8160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:47.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f80d0fc0 0x7f94f8160c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42786/0 (socket says 172.21.15.181:42786) 2023-10-16T09:59:47.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94f77fe700 1 -- 172.21.15.181:0/228409382 learned_addr learned my addr 172.21.15.181:0/228409382 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:47.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94f6ffd700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94f806afb0 0x7f94f814fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:47.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.744+0000 7f94f67fc700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94f80ce610 0x7f94f81606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:47.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94f77fe700 1 -- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94f80ce610 msgr2=0x7f94f81606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:47.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94f77fe700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94f80ce610 0x7f94f81606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94f77fe700 1 -- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94f806afb0 msgr2=0x7f94f814fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:47.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94f77fe700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94f806afb0 0x7f94f814fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:47.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94f77fe700 1 -- 172.21.15.181:0/228409382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f0009a70 con 0x7f94f80d0fc0 2023-10-16T09:59:47.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94f77fe700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f80d0fc0 0x7f94f8160c00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f94f806d530 tx=0x7f94f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:47.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94df7fe700 1 -- 172.21.15.181:0/228409382 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0012630 con 0x7f94f80d0fc0 2023-10-16T09:59:47.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94df7fe700 1 -- 172.21.15.181:0/228409382 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f00127b0 con 0x7f94f80d0fc0 2023-10-16T09:59:47.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f809a490 con 0x7f94f80d0fc0 2023-10-16T09:59:47.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94df7fe700 1 -- 172.21.15.181:0/228409382 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0012b60 con 0x7f94f80d0fc0 2023-10-16T09:59:47.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.745+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f809aa50 con 0x7f94f80d0fc0 2023-10-16T09:59:47.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.746+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f809af30 con 0x7f94f80d0fc0 2023-10-16T09:59:47.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.746+0000 7f94df7fe700 1 -- 172.21.15.181:0/228409382 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 41) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f94f0012cc0 con 0x7f94f80d0fc0 2023-10-16T09:59:47.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.747+0000 7f94df7fe700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f94e406af90 0x7f94e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:47.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.747+0000 7f94df7fe700 1 -- 172.21.15.181:0/228409382 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f94f008c2a0 con 0x7f94f80d0fc0 2023-10-16T09:59:47.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.747+0000 7f94f6ffd700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f94e406af90 0x7f94e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:47.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.747+0000 7f94f6ffd700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f94e406af90 0x7f94e406d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0050c0 tx=0x7f94ec007f10 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:47.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.751+0000 7f94df7fe700 1 -- 172.21.15.181:0/228409382 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f94f0056fe0 con 0x7f94f80d0fc0 2023-10-16T09:59:47.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:47.955+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94f80cbb50 con 0x7f94f80d0fc0 2023-10-16T09:59:48.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.594+0000 7f94df7fe700 1 -- 172.21.15.181:0/228409382 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f94f005a600 con 0x7f94f80d0fc0 2023-10-16T09:59:48.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f94e406af90 msgr2=0x7f94e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f94e406af90 0x7f94e406d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0050c0 tx=0x7f94ec007f10 comp rx=0 tx=0).stop 2023-10-16T09:59:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f80d0fc0 msgr2=0x7f94f8160c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f80d0fc0 0x7f94f8160c00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f94f806d530 tx=0x7f94f0004ad0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 shutdown_connections 2023-10-16T09:59:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f94e406af90 0x7f94e406d450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f80d0fc0 0x7f94f8160c00 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94f806afb0 0x7f94f814fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 --2- 172.21.15.181:0/228409382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94f80ce610 0x7f94f81606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 >> 172.21.15.181:0/228409382 conn(0x7f94f80d04a0 msgr2=0x7f94f80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.597+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 shutdown_connections 2023-10-16T09:59:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.598+0000 7f94fd275700 1 -- 172.21.15.181:0/228409382 wait complete. 2023-10-16T09:59:48.602 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T09:59:48.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T09:59:48.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.744+0000 7f6dbe56a700 1 Processor -- start 2023-10-16T09:59:48.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.744+0000 7f6dbe56a700 1 -- start start 2023-10-16T09:59:48.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.745+0000 7f6dbe56a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6db80ce6c0 0x7f6db80cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:48.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.745+0000 7f6dbe56a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db80b1f70 con 0x7f6db80cc6a0 2023-10-16T09:59:48.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.745+0000 7f6dbe56a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db80b2ae0 con 0x7f6db815f150 2023-10-16T09:59:48.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.745+0000 7f6dbe56a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db80b2dc0 con 0x7f6db80ce6c0 2023-10-16T09:59:48.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.745+0000 7f6db77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6db80ce6c0 0x7f6db80cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:48.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.745+0000 7f6dbcb07700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6db80cc6a0 0x7f6db80ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53872/0 (socket says 172.21.15.181:53872) 2023-10-16T09:59:48.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.745+0000 7f6dbcb07700 1 -- 172.21.15.181:0/1420868875 learned_addr learned my addr 172.21.15.181:0/1420868875 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:48.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db77fe700 1 -- 172.21.15.181:0/1420868875 >> v1:172.21.15.190:6789/0 conn(0x7f6db815f150 legacy=0x7f6db80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db77fe700 1 -- 172.21.15.181:0/1420868875 >> v1:172.21.15.181:6789/0 conn(0x7f6db80cc6a0 legacy=0x7f6db80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db6ffd700 1 -- 172.21.15.181:0/1420868875 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931636153 0 0) 0x7f6db80b1f70 con 0x7f6db80cc6a0 2023-10-16T09:59:48.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db77fe700 1 -- 172.21.15.181:0/1420868875 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80cf9d0 con 0x7f6db80ce6c0 2023-10-16T09:59:48.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db77fe700 1 --2- 172.21.15.181:0/1420868875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80ce6c0 0x7f6db80cc160 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009840 tx=0x7f6da8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15003d4e31713b63 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:48.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db6ffd700 1 -- 172.21.15.181:0/1420868875 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8011830 con 0x7f6db80ce6c0 2023-10-16T09:59:48.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db6ffd700 1 -- 172.21.15.181:0/1420868875 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da8011990 con 0x7f6db80ce6c0 2023-10-16T09:59:48.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6db6ffd700 1 -- 172.21.15.181:0/1420868875 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8011d20 con 0x7f6db80ce6c0 2023-10-16T09:59:48.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6dbe56a700 1 -- 172.21.15.181:0/1420868875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80ce6c0 msgr2=0x7f6db80cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.746+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/1420868875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80ce6c0 0x7f6db80cc160 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009840 tx=0x7f6da8009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:48.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.747+0000 7f6dbe56a700 1 -- 172.21.15.181:0/1420868875 shutdown_connections 2023-10-16T09:59:48.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.747+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/1420868875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80ce6c0 0x7f6db80cc160 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.747+0000 7f6dbe56a700 1 -- 172.21.15.181:0/1420868875 >> 172.21.15.181:0/1420868875 conn(0x7f6db809a930 msgr2=0x7f6db80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:48.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.747+0000 7f6dbe56a700 1 -- 172.21.15.181:0/1420868875 shutdown_connections 2023-10-16T09:59:48.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.747+0000 7f6dbe56a700 1 -- 172.21.15.181:0/1420868875 wait complete. 2023-10-16T09:59:48.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.748+0000 7f6dbe56a700 1 Processor -- start 2023-10-16T09:59:48.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.748+0000 7f6dbe56a700 1 -- start start 2023-10-16T09:59:48.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80cc6a0 0x7f6db80af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:48.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db80ce6c0 0x7f6db80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:48.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db815f150 0x7f6db80ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:48.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80c86d0 con 0x7f6db80cc6a0 2023-10-16T09:59:48.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81f7cc0 con 0x7f6db80ce6c0 2023-10-16T09:59:48.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80c88e0 con 0x7f6db815f150 2023-10-16T09:59:48.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80cc6a0 0x7f6db80af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:48.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80cc6a0 0x7f6db80af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42796/0 (socket says 172.21.15.181:42796) 2023-10-16T09:59:48.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 -- 172.21.15.181:0/3821685206 learned_addr learned my addr 172.21.15.181:0/3821685206 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:48.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db77fe700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db80ce6c0 0x7f6db80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:48.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbcb07700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db815f150 0x7f6db80ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:48.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 -- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db815f150 msgr2=0x7f6db80ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db815f150 0x7f6db80ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 -- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db80ce6c0 msgr2=0x7f6db80c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db80ce6c0 0x7f6db80c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 -- 172.21.15.181:0/3821685206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80094a0 con 0x7f6db80cc6a0 2023-10-16T09:59:48.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db7fff700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80cc6a0 0x7f6db80af760 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c450 tx=0x7f6dac00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:48.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db4ff9700 1 -- 172.21.15.181:0/3821685206 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0052d0 con 0x7f6db80cc6a0 2023-10-16T09:59:48.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db4ff9700 1 -- 172.21.15.181:0/3821685206 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac005430 con 0x7f6db80cc6a0 2023-10-16T09:59:48.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db80c8bc0 con 0x7f6db80cc6a0 2023-10-16T09:59:48.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6db4ff9700 1 -- 172.21.15.181:0/3821685206 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0038c0 con 0x7f6db80cc6a0 2023-10-16T09:59:48.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.749+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db80afca0 con 0x7f6db80cc6a0 2023-10-16T09:59:48.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.750+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db80afe30 con 0x7f6db80cc6a0 2023-10-16T09:59:48.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.751+0000 7f6db4ff9700 1 -- 172.21.15.181:0/3821685206 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 41) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6dac003a20 con 0x7f6db80cc6a0 2023-10-16T09:59:48.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.751+0000 7f6db4ff9700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f6da406af90 0x7f6da406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:48.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.751+0000 7f6db4ff9700 1 -- 172.21.15.181:0/3821685206 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f6dac088c30 con 0x7f6db80cc6a0 2023-10-16T09:59:48.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.751+0000 7f6db77fe700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f6da406af90 0x7f6da406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:48.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.752+0000 7f6db77fe700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f6da406af90 0x7f6da406d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009790 tx=0x7f6da801c040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:48.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.756+0000 7f6db4ff9700 1 -- 172.21.15.181:0/3821685206 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6dac053a10 con 0x7f6db80cc6a0 2023-10-16T09:59:48.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.947+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f6db8067d20 con 0x7f6db80cc6a0 2023-10-16T09:59:48.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.950+0000 7f6db4ff9700 1 -- 172.21.15.181:0/3821685206 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v25) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6dac057030 con 0x7f6db80cc6a0 2023-10-16T09:59:48.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f6da406af90 msgr2=0x7f6da406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f6da406af90 0x7f6da406d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009790 tx=0x7f6da801c040 comp rx=0 tx=0).stop 2023-10-16T09:59:48.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80cc6a0 msgr2=0x7f6db80af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80cc6a0 0x7f6db80af760 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c450 tx=0x7f6dac00c810 comp rx=0 tx=0).stop 2023-10-16T09:59:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 shutdown_connections 2023-10-16T09:59:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f6da406af90 0x7f6da406d450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db815f150 0x7f6db80ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db80cc6a0 0x7f6db80af760 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 --2- 172.21.15.181:0/3821685206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db80ce6c0 0x7f6db80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 >> 172.21.15.181:0/3821685206 conn(0x7f6db809a930 msgr2=0x7f6db80b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 shutdown_connections 2023-10-16T09:59:48.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:48.953+0000 7f6dbe56a700 1 -- 172.21.15.181:0/3821685206 wait complete. 2023-10-16T09:59:48.967 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T09:59:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.088+0000 7f8037b9e700 1 Processor -- start 2023-10-16T09:59:49.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.088+0000 7f8037b9e700 1 -- start start 2023-10-16T09:59:49.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.089+0000 7f8037b9e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80300cc6e0 0x7f8030071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.089+0000 7f8037b9e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300ce1f0 con 0x7f80300b2010 2023-10-16T09:59:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.089+0000 7f8037b9e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300ccda0 con 0x7f80300ce680 2023-10-16T09:59:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.089+0000 7f8037b9e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80300ccb90 con 0x7f80300cc6e0 2023-10-16T09:59:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.089+0000 7f8035139700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80300cc6e0 0x7f8030071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.089+0000 7f803593a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f80300b2010 0x7f80300b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53882/0 (socket says 172.21.15.181:53882) 2023-10-16T09:59:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.089+0000 7f803593a700 1 -- 172.21.15.181:0/1957526594 learned_addr learned my addr 172.21.15.181:0/1957526594 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8035139700 1 -- 172.21.15.181:0/1957526594 >> v1:172.21.15.190:6789/0 conn(0x7f80300ce680 legacy=0x7f80300ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8035139700 1 -- 172.21.15.181:0/1957526594 >> v1:172.21.15.181:6789/0 conn(0x7f80300b2010 legacy=0x7f80300b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8035139700 1 -- 172.21.15.181:0/1957526594 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300727e0 con 0x7f80300cc6e0 2023-10-16T09:59:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8034938700 1 -- 172.21.15.181:0/1957526594 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356357730 0 0) 0x7f80300ce1f0 con 0x7f80300b2010 2023-10-16T09:59:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8035139700 1 --2- 172.21.15.181:0/1957526594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300cc6e0 0x7f8030071be0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f802c009840 tx=0x7f802c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dac298f0d5090 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8034938700 1 -- 172.21.15.181:0/1957526594 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c004230 con 0x7f80300cc6e0 2023-10-16T09:59:49.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8034938700 1 -- 172.21.15.181:0/1957526594 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c004390 con 0x7f80300cc6e0 2023-10-16T09:59:49.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8034938700 1 -- 172.21.15.181:0/1957526594 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0036a0 con 0x7f80300cc6e0 2023-10-16T09:59:49.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8037b9e700 1 -- 172.21.15.181:0/1957526594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300cc6e0 msgr2=0x7f8030071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8037b9e700 1 --2- 172.21.15.181:0/1957526594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300cc6e0 0x7f8030071be0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f802c009840 tx=0x7f802c009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:49.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8037b9e700 1 -- 172.21.15.181:0/1957526594 shutdown_connections 2023-10-16T09:59:49.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8037b9e700 1 --2- 172.21.15.181:0/1957526594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300cc6e0 0x7f8030071be0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.090+0000 7f8037b9e700 1 -- 172.21.15.181:0/1957526594 >> 172.21.15.181:0/1957526594 conn(0x7f80300cd010 msgr2=0x7f80300cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:49.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.091+0000 7f8037b9e700 1 -- 172.21.15.181:0/1957526594 shutdown_connections 2023-10-16T09:59:49.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.091+0000 7f8037b9e700 1 -- 172.21.15.181:0/1957526594 wait complete. 2023-10-16T09:59:49.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 Processor -- start 2023-10-16T09:59:49.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 -- start start 2023-10-16T09:59:49.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300b2010 0x7f8030158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80300cc6e0 0x7f80301f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80300ce680 0x7f80300ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8030159310 con 0x7f80300b2010 2023-10-16T09:59:49.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80300b00c0 con 0x7f80300ce680 2023-10-16T09:59:49.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.092+0000 7f8037b9e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80301f3df0 con 0x7f80300cc6e0 2023-10-16T09:59:49.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300b2010 0x7f8030158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300b2010 0x7f8030158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42810/0 (socket says 172.21.15.181:42810) 2023-10-16T09:59:49.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 -- 172.21.15.181:0/2547434204 learned_addr learned my addr 172.21.15.181:0/2547434204 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:49.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803613b700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80300ce680 0x7f80300ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f8035139700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80300cc6e0 0x7f80301f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 -- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80300cc6e0 msgr2=0x7f80301f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80300cc6e0 0x7f80301f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 -- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80300ce680 msgr2=0x7f80300ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80300ce680 0x7f80300ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 -- 172.21.15.181:0/2547434204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0094a0 con 0x7f80300b2010 2023-10-16T09:59:49.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f803593a700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300b2010 0x7f8030158dd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f802000c550 tx=0x7f802000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020004c80 con 0x7f80300b2010 2023-10-16T09:59:49.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020004e00 con 0x7f80300b2010 2023-10-16T09:59:49.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020007310 con 0x7f80300b2010 2023-10-16T09:59:49.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80301f3830 con 0x7f80300b2010 2023-10-16T09:59:49.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.093+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80300afd50 con 0x7f80300b2010 2023-10-16T09:59:49.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.094+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80300b6510 con 0x7f80300b2010 2023-10-16T09:59:49.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.095+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 41) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8020007470 con 0x7f80300b2010 2023-10-16T09:59:49.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.095+0000 7f80267fc700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f801406af90 0x7f801406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.095+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f80200893a0 con 0x7f80300b2010 2023-10-16T09:59:49.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.095+0000 7f8035139700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f801406af90 0x7f801406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.095+0000 7f8035139700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f801406af90 0x7f801406d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80300b1dc0 tx=0x7f802c013040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.100+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8020054390 con 0x7f80300b2010 2023-10-16T09:59:49.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.175+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8020054760 con 0x7f80300b2010 2023-10-16T09:59:49.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.294+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f80300b4140 con 0x7f80300b2010 2023-10-16T09:59:49.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.298+0000 7f80267fc700 1 -- 172.21.15.181:0/2547434204 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v26) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8020073a60 con 0x7f80300b2010 2023-10-16T09:59:49.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f801406af90 msgr2=0x7f801406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f801406af90 0x7f801406d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80300b1dc0 tx=0x7f802c013040 comp rx=0 tx=0).stop 2023-10-16T09:59:49.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300b2010 msgr2=0x7f8030158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300b2010 0x7f8030158dd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f802000c550 tx=0x7f802000c910 comp rx=0 tx=0).stop 2023-10-16T09:59:49.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 shutdown_connections 2023-10-16T09:59:49.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f801406af90 0x7f801406d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80300ce680 0x7f80300ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80300b2010 0x7f8030158dd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 --2- 172.21.15.181:0/2547434204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80300cc6e0 0x7f80301f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 >> 172.21.15.181:0/2547434204 conn(0x7f80300cd010 msgr2=0x7f80300b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:49.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 shutdown_connections 2023-10-16T09:59:49.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.300+0000 7f8037b9e700 1 -- 172.21.15.181:0/2547434204 wait complete. 2023-10-16T09:59:49.314 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T09:59:49.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.435+0000 7f776277c700 1 Processor -- start 2023-10-16T09:59:49.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.435+0000 7f776277c700 1 -- start start 2023-10-16T09:59:49.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f776277c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f775c0666a0 0x7f775c06f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f776277c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775c0ca300 con 0x7f775c0cdd00 2023-10-16T09:59:49.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f776277c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775c06b6c0 con 0x7f775c0ce0e0 2023-10-16T09:59:49.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f776277c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775c06e5e0 con 0x7f775c0666a0 2023-10-16T09:59:49.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f775b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f775c0666a0 0x7f775c06f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f775bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f775c0cdd00 0x7f775c06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53896/0 (socket says 172.21.15.181:53896) 2023-10-16T09:59:49.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f775b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f775c0666a0 0x7f775c06f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42820/0 (socket says 172.21.15.181:42820) 2023-10-16T09:59:49.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.436+0000 7f775bfff700 1 -- 172.21.15.181:0/3346079023 learned_addr learned my addr 172.21.15.181:0/3346079023 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:49.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775b7fe700 1 -- 172.21.15.181:0/3346079023 >> v1:172.21.15.190:6789/0 conn(0x7f775c0ce0e0 legacy=0x7f775c06afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:49.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775b7fe700 1 -- 172.21.15.181:0/3346079023 >> v1:172.21.15.181:6789/0 conn(0x7f775c0cdd00 legacy=0x7f775c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775b7fe700 1 -- 172.21.15.181:0/3346079023 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c0ca580 con 0x7f775c0666a0 2023-10-16T09:59:49.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775affd700 1 -- 172.21.15.181:0/3346079023 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739734419 0 0) 0x7f775c0ca300 con 0x7f775c0cdd00 2023-10-16T09:59:49.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775b7fe700 1 --2- 172.21.15.181:0/3346079023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c06f5f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f774c009840 tx=0x7f774c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a518397320d123 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775affd700 1 -- 172.21.15.181:0/3346079023 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c004230 con 0x7f775c0666a0 2023-10-16T09:59:49.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775affd700 1 -- 172.21.15.181:0/3346079023 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c004390 con 0x7f775c0666a0 2023-10-16T09:59:49.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f775affd700 1 -- 172.21.15.181:0/3346079023 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0036a0 con 0x7f775c0666a0 2023-10-16T09:59:49.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f776277c700 1 -- 172.21.15.181:0/3346079023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 msgr2=0x7f775c06f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f776277c700 1 --2- 172.21.15.181:0/3346079023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c06f5f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f774c009840 tx=0x7f774c009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:49.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f776277c700 1 -- 172.21.15.181:0/3346079023 shutdown_connections 2023-10-16T09:59:49.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f776277c700 1 --2- 172.21.15.181:0/3346079023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c06f5f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f776277c700 1 -- 172.21.15.181:0/3346079023 >> 172.21.15.181:0/3346079023 conn(0x7f775c0cc730 msgr2=0x7f775c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:49.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f776277c700 1 -- 172.21.15.181:0/3346079023 shutdown_connections 2023-10-16T09:59:49.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.437+0000 7f776277c700 1 -- 172.21.15.181:0/3346079023 wait complete. 2023-10-16T09:59:49.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.439+0000 7f776277c700 1 Processor -- start 2023-10-16T09:59:49.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.439+0000 7f776277c700 1 -- start start 2023-10-16T09:59:49.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f776277c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c1e6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f776277c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f775c0cdd00 0x7f775c14dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f776277c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f775c0ce0e0 0x7f775c1f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f776277c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c0c80b0 con 0x7f775c0666a0 2023-10-16T09:59:49.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f776277c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c0c8390 con 0x7f775c0cdd00 2023-10-16T09:59:49.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f776277c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c0c8670 con 0x7f775c0ce0e0 2023-10-16T09:59:49.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f775bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c1e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f775bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c1e6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42824/0 (socket says 172.21.15.181:42824) 2023-10-16T09:59:49.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f775bfff700 1 -- 172.21.15.181:0/2141344062 learned_addr learned my addr 172.21.15.181:0/2141344062 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:49.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f775b7fe700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f775c0cdd00 0x7f775c14dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.440+0000 7f7760d19700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f775c0ce0e0 0x7f775c1f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f775bfff700 1 -- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f775c0ce0e0 msgr2=0x7f775c1f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f775bfff700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f775c0ce0e0 0x7f775c1f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f775bfff700 1 -- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f775c0cdd00 msgr2=0x7f775c14dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f775bfff700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f775c0cdd00 0x7f775c14dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f775bfff700 1 -- 172.21.15.181:0/2141344062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c0094a0 con 0x7f775c0666a0 2023-10-16T09:59:49.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f775bfff700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c1e6800 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f775000c520 tx=0x7f775000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f7758ff9700 1 -- 172.21.15.181:0/2141344062 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750004c80 con 0x7f775c0666a0 2023-10-16T09:59:49.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f7758ff9700 1 -- 172.21.15.181:0/2141344062 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750004e00 con 0x7f775c0666a0 2023-10-16T09:59:49.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c0c8950 con 0x7f775c0666a0 2023-10-16T09:59:49.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f7758ff9700 1 -- 172.21.15.181:0/2141344062 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750007310 con 0x7f775c0666a0 2023-10-16T09:59:49.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.441+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c0c9150 con 0x7f775c0666a0 2023-10-16T09:59:49.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.442+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c0c8e70 con 0x7f775c0666a0 2023-10-16T09:59:49.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.442+0000 7f7758ff9700 1 -- 172.21.15.181:0/2141344062 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7750007470 con 0x7f775c0666a0 2023-10-16T09:59:49.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.443+0000 7f7758ff9700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f774806b020 0x7f774806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.443+0000 7f7758ff9700 1 -- 172.21.15.181:0/2141344062 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f7750089410 con 0x7f775c0666a0 2023-10-16T09:59:49.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.443+0000 7f775b7fe700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f774806b020 0x7f774806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.443+0000 7f775b7fe700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f774806b020 0x7f774806d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f774c009840 tx=0x7f774c008030 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.447+0000 7f7758ff9700 1 -- 172.21.15.181:0/2141344062 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f775c0c8e70 con 0x7f775c0666a0 2023-10-16T09:59:49.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.641+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f775c0c8e70 con 0x7f775c0666a0 2023-10-16T09:59:49.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.644+0000 7f7758ff9700 1 -- 172.21.15.181:0/2141344062 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f775c0c8e70 con 0x7f775c0666a0 2023-10-16T09:59:49.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f774806b020 msgr2=0x7f774806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f774806b020 0x7f774806d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f774c009840 tx=0x7f774c008030 comp rx=0 tx=0).stop 2023-10-16T09:59:49.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 msgr2=0x7f775c1e6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c1e6800 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f775000c520 tx=0x7f775000c830 comp rx=0 tx=0).stop 2023-10-16T09:59:49.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 shutdown_connections 2023-10-16T09:59:49.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f774806b020 0x7f774806d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f775c0ce0e0 0x7f775c1f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f775c0666a0 0x7f775c1e6800 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 --2- 172.21.15.181:0/2141344062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f775c0cdd00 0x7f775c14dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.646+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 >> 172.21.15.181:0/2141344062 conn(0x7f775c0cc730 msgr2=0x7f775c0ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:49.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.647+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 shutdown_connections 2023-10-16T09:59:49.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.647+0000 7f776277c700 1 -- 172.21.15.181:0/2141344062 wait complete. 2023-10-16T09:59:49.665 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-16T09:59:49.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.783+0000 7f0c1e572700 1 Processor -- start 2023-10-16T09:59:49.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.783+0000 7f0c1e572700 1 -- start start 2023-10-16T09:59:49.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c1e572700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0c180ce7f0 0x7f0c1806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c1e572700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c180d2390 0x7f0c180cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c1e572700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c180cc470 con 0x7f0c180d0180 2023-10-16T09:59:49.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c1e572700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c180d05f0 con 0x7f0c180ce7f0 2023-10-16T09:59:49.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c1e572700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1806ca30 con 0x7f0c180d2390 2023-10-16T09:59:49.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c17fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0c180ce7f0 0x7f0c1806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c1cb0f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0c180d0180 0x7f0c18161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53914/0 (socket says 172.21.15.181:53914) 2023-10-16T09:59:49.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.784+0000 7f0c1cb0f700 1 -- 172.21.15.181:0/196860469 learned_addr learned my addr 172.21.15.181:0/196860469 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:49.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c177fe700 1 --2- 172.21.15.181:0/196860469 >> v2:172.21.15.190:3300/0 conn(0x7f0c180d2390 0x7f0c180cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c17fff700 1 -- 172.21.15.181:0/196860469 >> v2:172.21.15.190:3300/0 conn(0x7f0c180d2390 msgr2=0x7f0c180cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:49.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c17fff700 1 --2- 172.21.15.181:0/196860469 >> v2:172.21.15.190:3300/0 conn(0x7f0c180d2390 0x7f0c180cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c17fff700 1 -- 172.21.15.181:0/196860469 >> v1:172.21.15.181:6789/0 conn(0x7f0c180d0180 legacy=0x7f0c18161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c17fff700 1 -- 172.21.15.181:0/196860469 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1806cd10 con 0x7f0c180ce7f0 2023-10-16T09:59:49.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c16ffd700 1 -- 172.21.15.181:0/196860469 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458857109 0 0) 0x7f0c180cc470 con 0x7f0c180d0180 2023-10-16T09:59:49.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c177fe700 1 --2- 172.21.15.181:0/196860469 >> v2:172.21.15.190:3300/0 conn(0x7f0c180d2390 0x7f0c180cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:49.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c17fff700 1 --2- 172.21.15.181:0/196860469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180ce7f0 0x7f0c1806c100 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0c180cc1e0 tx=0x7f0c08009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970ee7324aa01297 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c16ffd700 1 -- 172.21.15.181:0/196860469 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08005e30 con 0x7f0c180ce7f0 2023-10-16T09:59:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c16ffd700 1 -- 172.21.15.181:0/196860469 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c08006030 con 0x7f0c180ce7f0 2023-10-16T09:59:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c16ffd700 1 -- 172.21.15.181:0/196860469 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080063c0 con 0x7f0c180ce7f0 2023-10-16T09:59:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c1e572700 1 -- 172.21.15.181:0/196860469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180ce7f0 msgr2=0x7f0c1806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.785+0000 7f0c1e572700 1 --2- 172.21.15.181:0/196860469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180ce7f0 0x7f0c1806c100 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0c180cc1e0 tx=0x7f0c08009b00 comp rx=0 tx=0).stop 2023-10-16T09:59:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.786+0000 7f0c1e572700 1 -- 172.21.15.181:0/196860469 shutdown_connections 2023-10-16T09:59:49.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.786+0000 7f0c1e572700 1 --2- 172.21.15.181:0/196860469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180ce7f0 0x7f0c1806c100 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.786+0000 7f0c1e572700 1 --2- 172.21.15.181:0/196860469 >> v2:172.21.15.190:3300/0 conn(0x7f0c180d2390 0x7f0c180cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.786+0000 7f0c1e572700 1 -- 172.21.15.181:0/196860469 >> 172.21.15.181:0/196860469 conn(0x7f0c1809a9a0 msgr2=0x7f0c180d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:49.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.786+0000 7f0c1e572700 1 -- 172.21.15.181:0/196860469 shutdown_connections 2023-10-16T09:59:49.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.786+0000 7f0c1e572700 1 -- 172.21.15.181:0/196860469 wait complete. 2023-10-16T09:59:49.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.787+0000 7f0c1e572700 1 Processor -- start 2023-10-16T09:59:49.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.787+0000 7f0c1e572700 1 -- start start 2023-10-16T09:59:49.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c180ce7f0 0x7f0c1815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c180d0180 0x7f0c180c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180d2390 0x7f0c180c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181f30a0 con 0x7f0c180d2390 2023-10-16T09:59:49.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181f73a0 con 0x7f0c180ce7f0 2023-10-16T09:59:49.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181f3300 con 0x7f0c180d0180 2023-10-16T09:59:49.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180d2390 0x7f0c180c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180d2390 0x7f0c180c78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42842/0 (socket says 172.21.15.181:42842) 2023-10-16T09:59:49.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 -- 172.21.15.181:0/3901920024 learned_addr learned my addr 172.21.15.181:0/3901920024 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:49.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c17fff700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c180ce7f0 0x7f0c1815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c177fe700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c180d0180 0x7f0c180c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 -- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c180d0180 msgr2=0x7f0c180c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c180d0180 0x7f0c180c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 -- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c180ce7f0 msgr2=0x7f0c1815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:49.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c180ce7f0 0x7f0c1815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:49.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 -- 172.21.15.181:0/3901920024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08009430 con 0x7f0c180d2390 2023-10-16T09:59:49.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1cb0f700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180d2390 0x7f0c180c78c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000cba0 tx=0x7f0c1000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c14ff9700 1 -- 172.21.15.181:0/3901920024 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000aad0 con 0x7f0c180d2390 2023-10-16T09:59:49.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c14ff9700 1 -- 172.21.15.181:0/3901920024 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000ac50 con 0x7f0c180d2390 2023-10-16T09:59:49.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c181f35e0 con 0x7f0c180d2390 2023-10-16T09:59:49.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c14ff9700 1 -- 172.21.15.181:0/3901920024 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100070d0 con 0x7f0c180d2390 2023-10-16T09:59:49.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.788+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c181604d0 con 0x7f0c180d2390 2023-10-16T09:59:49.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.789+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c181f3b00 con 0x7f0c180d2390 2023-10-16T09:59:49.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.790+0000 7f0c14ff9700 1 -- 172.21.15.181:0/3901920024 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0c10007230 con 0x7f0c180d2390 2023-10-16T09:59:49.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.790+0000 7f0c14ff9700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f0c0406b020 0x7f0c0406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:49.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.790+0000 7f0c14ff9700 1 -- 172.21.15.181:0/3901920024 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f0c10089670 con 0x7f0c180d2390 2023-10-16T09:59:49.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.790+0000 7f0c17fff700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f0c0406b020 0x7f0c0406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:49.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.791+0000 7f0c17fff700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f0c0406b020 0x7f0c0406d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0c08000c00 tx=0x7f0c08008070 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:49.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:49.795+0000 7f0c14ff9700 1 -- 172.21.15.181:0/3901920024 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0c181f3b00 con 0x7f0c180d2390 2023-10-16T09:59:50.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.063+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f0c181f3b00 con 0x7f0c180d2390 2023-10-16T09:59:50.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.596+0000 7f0c14ff9700 1 -- 172.21.15.181:0/3901920024 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v577) v1 ==== 156+0+0 (secure 0 0 0) 0x7f0c181f3b00 con 0x7f0c180d2390 2023-10-16T09:59:50.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f0c0406b020 msgr2=0x7f0c0406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f0c0406b020 0x7f0c0406d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0c08000c00 tx=0x7f0c08008070 comp rx=0 tx=0).stop 2023-10-16T09:59:50.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180d2390 msgr2=0x7f0c180c78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180d2390 0x7f0c180c78c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000cba0 tx=0x7f0c1000ceb0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 shutdown_connections 2023-10-16T09:59:50.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f0c0406b020 0x7f0c0406d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c180d2390 0x7f0c180c78c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c180ce7f0 0x7f0c1815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 --2- 172.21.15.181:0/3901920024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c180d0180 0x7f0c180c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.599+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 >> 172.21.15.181:0/3901920024 conn(0x7f0c1809a9a0 msgr2=0x7f0c180c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:50.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.600+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 shutdown_connections 2023-10-16T09:59:50.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.600+0000 7f0c1e572700 1 -- 172.21.15.181:0/3901920024 wait complete. 2023-10-16T09:59:50.615 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-16T09:59:50.616 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-16T09:59:50.616 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T09:59:50.617 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:50.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.741+0000 7f7a3748d700 1 Processor -- start 2023-10-16T09:59:50.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.741+0000 7f7a3748d700 1 -- start start 2023-10-16T09:59:50.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a3748d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 0x7f7a300ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a3748d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a300cc510 con 0x7f7a3015b990 2023-10-16T09:59:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a3748d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a300cc7f0 con 0x7f7a300cd2d0 2023-10-16T09:59:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a3748d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300ccad0 con 0x7f7a300d0370 2023-10-16T09:59:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a35229700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7a3015b990 0x7f7a300cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53924/0 (socket says 172.21.15.181:53924) 2023-10-16T09:59:50.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a35229700 1 -- 172.21.15.181:0/2178120515 learned_addr learned my addr 172.21.15.181:0/2178120515 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a34a28700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 0x7f7a300ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358332921 0 0) 0x7f7a300cc510 con 0x7f7a3015b990 2023-10-16T09:59:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a1c0035b0 con 0x7f7a3015b990 2023-10-16T09:59:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743495375 0 0) 0x7f7a300cc7f0 con 0x7f7a300cd2d0 2023-10-16T09:59:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a300cc510 con 0x7f7a300cd2d0 2023-10-16T09:59:50.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991583315 0 0) 0x7f7a1c0035b0 con 0x7f7a3015b990 2023-10-16T09:59:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 msgr2=0x7f7a300ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 0x7f7a300ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 >> v1:172.21.15.190:6789/0 conn(0x7f7a300cd2d0 legacy=0x7f7a300ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300ccdb0 con 0x7f7a3015b990 2023-10-16T09:59:50.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7a200017f0 con 0x7f7a3015b990 2023-10-16T09:59:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 >> v1:172.21.15.181:6789/0 conn(0x7f7a3015b990 legacy=0x7f7a300cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1c003bc0 0x7f7a1c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 0x7f7a1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c004d90 0x7f7a1c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300ccad0 con 0x7f7a1c003bc0 2023-10-16T09:59:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c00bac0 con 0x7f7a1c004d90 2023-10-16T09:59:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c00bc80 con 0x7f7a300d0370 2023-10-16T09:59:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.742+0000 7f7a35229700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1c003bc0 0x7f7a1c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a34a28700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 0x7f7a1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35a2a700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c004d90 0x7f7a1c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35229700 1 -- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 msgr2=0x7f7a1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35229700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a300d0370 0x7f7a1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35229700 1 -- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c004d90 msgr2=0x7f7a1c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35229700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c004d90 0x7f7a1c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35229700 1 -- 172.21.15.181:0/2178120515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0035b0 con 0x7f7a1c003bc0 2023-10-16T09:59:50.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35229700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1c003bc0 0x7f7a1c003fc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c110 tx=0x7f7a1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5476e18743db5f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000cc50 con 0x7f7a1c003bc0 2023-10-16T09:59:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2000cdb0 con 0x7f7a1c003bc0 2023-10-16T09:59:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a27fff700 1 -- 172.21.15.181:0/2178120515 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20007310 con 0x7f7a1c003bc0 2023-10-16T09:59:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a3748d700 1 -- 172.21.15.181:0/2178120515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1c003bc0 msgr2=0x7f7a1c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a3748d700 1 --2- 172.21.15.181:0/2178120515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1c003bc0 0x7f7a1c003fc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c110 tx=0x7f7a1c005880 comp rx=0 tx=0).stop 2023-10-16T09:59:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.743+0000 7f7a35a2a700 1 -- 172.21.15.181:0/2178120515 reap_dead start 2023-10-16T09:59:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.744+0000 7f7a3748d700 1 -- 172.21.15.181:0/2178120515 shutdown_connections 2023-10-16T09:59:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.744+0000 7f7a3748d700 1 -- 172.21.15.181:0/2178120515 >> 172.21.15.181:0/2178120515 conn(0x7f7a3009a9a0 msgr2=0x7f7a300d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.744+0000 7f7a3748d700 1 -- 172.21.15.181:0/2178120515 shutdown_connections 2023-10-16T09:59:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.744+0000 7f7a3748d700 1 -- 172.21.15.181:0/2178120515 wait complete. 2023-10-16T09:59:50.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 Processor -- start 2023-10-16T09:59:50.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 -- start start 2023-10-16T09:59:50.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c003bc0 0x7f7a30158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a30159a50 0x7f7a30159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a3015cdc0 0x7f7a3015d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3015a410 con 0x7f7a30159a50 2023-10-16T09:59:50.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b4f50 con 0x7f7a1c003bc0 2023-10-16T09:59:50.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.745+0000 7f7a3748d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b4340 con 0x7f7a3015cdc0 2023-10-16T09:59:50.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a30159a50 0x7f7a30159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a30159a50 0x7f7a30159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42860/0 (socket says 172.21.15.181:42860) 2023-10-16T09:59:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 -- 172.21.15.181:0/2586704527 learned_addr learned my addr 172.21.15.181:0/2586704527 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a35a2a700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a3015cdc0 0x7f7a3015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a35229700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c003bc0 0x7f7a30158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 -- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a3015cdc0 msgr2=0x7f7a3015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a3015cdc0 0x7f7a3015d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 -- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c003bc0 msgr2=0x7f7a30158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c003bc0 0x7f7a30158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 -- 172.21.15.181:0/2586704527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300b4620 con 0x7f7a30159a50 2023-10-16T09:59:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a34a28700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a30159a50 0x7f7a30159ed0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c011aa0 tx=0x7f7a2c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a25ffb700 1 -- 172.21.15.181:0/2586704527 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011ee0 con 0x7f7a30159a50 2023-10-16T09:59:50.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a25ffb700 1 -- 172.21.15.181:0/2586704527 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c00f940 con 0x7f7a30159a50 2023-10-16T09:59:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.746+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a300b4900 con 0x7f7a30159a50 2023-10-16T09:59:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.747+0000 7f7a25ffb700 1 -- 172.21.15.181:0/2586704527 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00fcf0 con 0x7f7a30159a50 2023-10-16T09:59:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.747+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a30159240 con 0x7f7a30159a50 2023-10-16T09:59:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.748+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a30067d20 con 0x7f7a30159a50 2023-10-16T09:59:50.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.748+0000 7f7a25ffb700 1 -- 172.21.15.181:0/2586704527 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7a2c014440 con 0x7f7a30159a50 2023-10-16T09:59:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.749+0000 7f7a25ffb700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f7a1406b020 0x7f7a1406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.749+0000 7f7a25ffb700 1 -- 172.21.15.181:0/2586704527 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f7a2c089ef0 con 0x7f7a30159a50 2023-10-16T09:59:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.749+0000 7f7a35229700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f7a1406b020 0x7f7a1406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.749+0000 7f7a35229700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f7a1406b020 0x7f7a1406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20007000 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:50.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.753+0000 7f7a25ffb700 1 -- 172.21.15.181:0/2586704527 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a2c054de0 con 0x7f7a30159a50 2023-10-16T09:59:50.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.947+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a300b5230 con 0x7f7a30159a50 2023-10-16T09:59:50.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.948+0000 7f7a25ffb700 1 -- 172.21.15.181:0/2586704527 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+14682 (secure 0 0 0) 0x7f7a2c022020 con 0x7f7a30159a50 2023-10-16T09:59:50.950 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:50.950 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":81,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:48.593532+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1343774933":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/1815471563":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/52391112":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/3113822369":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/2394784404":"2023-10-17T09:59:44.566188+0000","172.21.15.181:6833/420719762":"2023-10-17T09:59:44.566188+0000","172.21.15.190:6835/1948728404":"2023-10-17T09:59:45.533911+0000","172.21.15.190:6833/1948728404":"2023-10-17T09:59:45.533911+0000","172.21.15.181:6832/420719762":"2023-10-17T09:59:44.566188+0000","172.21.15.181:0/75988000":"2023-10-17T09:59:44.566188+0000","172.21.15.181:6833/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:6832/3602430955":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/1081249734":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2658775934":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2227004797":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3531530097":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/2146555244":"2023-10-17T09:59:37.126305+0000","172.21.15.181:0/3441419577":"2023-10-17T09:59:37.126305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f7a1406b020 msgr2=0x7f7a1406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f7a1406b020 0x7f7a1406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20007000 comp rx=0 tx=0).stop 2023-10-16T09:59:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a30159a50 msgr2=0x7f7a30159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a30159a50 0x7f7a30159ed0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c011aa0 tx=0x7f7a2c011db0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 shutdown_connections 2023-10-16T09:59:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f7a1406b020 0x7f7a1406d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a3015cdc0 0x7f7a3015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a1c003bc0 0x7f7a30158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 --2- 172.21.15.181:0/2586704527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a30159a50 0x7f7a30159ed0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 >> 172.21.15.181:0/2586704527 conn(0x7f7a3009a9a0 msgr2=0x7f7a3015ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 shutdown_connections 2023-10-16T09:59:50.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:50.950+0000 7f7a3748d700 1 -- 172.21.15.181:0/2586704527 wait complete. 2023-10-16T09:59:50.964 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:50.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:51.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.086+0000 7fefa1b91700 1 Processor -- start 2023-10-16T09:59:51.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.086+0000 7fefa1b91700 1 -- start start 2023-10-16T09:59:51.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fefa1b91700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef9c15ac20 0x7fef9c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fefa1b91700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c0ceb60 0x7fef9c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fefa1b91700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c0cb340 con 0x7fef9c158360 2023-10-16T09:59:51.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fefa1b91700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c0cf530 con 0x7fef9c15ac20 2023-10-16T09:59:51.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fefa1b91700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15b0d0 con 0x7fef9c0ceb60 2023-10-16T09:59:51.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef9c15ac20 0x7fef9c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fef9c158360 0x7fef9c15ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53944/0 (socket says 172.21.15.181:53944) 2023-10-16T09:59:51.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9b7fe700 1 -- 172.21.15.181:0/4124199681 learned_addr learned my addr 172.21.15.181:0/4124199681 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:51.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9bfff700 1 --2- 172.21.15.181:0/4124199681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c0ceb60 0x7fef9c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9affd700 1 -- 172.21.15.181:0/4124199681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c0ceb60 msgr2=0x7fef9c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:51.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9affd700 1 --2- 172.21.15.181:0/4124199681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c0ceb60 0x7fef9c0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9affd700 1 -- 172.21.15.181:0/4124199681 >> v1:172.21.15.181:6789/0 conn(0x7fef9c158360 legacy=0x7fef9c15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9a7fc700 1 -- 172.21.15.181:0/4124199681 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424620159 0 0) 0x7fef9c0cb340 con 0x7fef9c158360 2023-10-16T09:59:51.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.087+0000 7fef9affd700 1 -- 172.21.15.181:0/4124199681 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c15b3b0 con 0x7fef9c15ac20 2023-10-16T09:59:51.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fef9affd700 1 --2- 172.21.15.181:0/4124199681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c15ac20 0x7fef9c06a8a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04835c66866e0e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:51.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fef9a7fc700 1 -- 172.21.15.181:0/4124199681 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011830 con 0x7fef9c15ac20 2023-10-16T09:59:51.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fef9a7fc700 1 -- 172.21.15.181:0/4124199681 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c011990 con 0x7fef9c15ac20 2023-10-16T09:59:51.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fef9a7fc700 1 -- 172.21.15.181:0/4124199681 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011d20 con 0x7fef9c15ac20 2023-10-16T09:59:51.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 -- 172.21.15.181:0/4124199681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c15ac20 msgr2=0x7fef9c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 --2- 172.21.15.181:0/4124199681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c15ac20 0x7fef9c06a8a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:51.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 -- 172.21.15.181:0/4124199681 shutdown_connections 2023-10-16T09:59:51.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 --2- 172.21.15.181:0/4124199681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c0ceb60 0x7fef9c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 --2- 172.21.15.181:0/4124199681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c15ac20 0x7fef9c06a8a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 -- 172.21.15.181:0/4124199681 >> 172.21.15.181:0/4124199681 conn(0x7fef9c0cc730 msgr2=0x7fef9c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:51.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 -- 172.21.15.181:0/4124199681 shutdown_connections 2023-10-16T09:59:51.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.088+0000 7fefa1b91700 1 -- 172.21.15.181:0/4124199681 wait complete. 2023-10-16T09:59:51.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.089+0000 7fefa1b91700 1 Processor -- start 2023-10-16T09:59:51.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fefa1b91700 1 -- start start 2023-10-16T09:59:51.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fefa1b91700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef9c0ceb60 0x7fef9c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fefa1b91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c158360 0x7fef9c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fefa1b91700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c15ac20 0x7fef9c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fefa1b91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0c72f0 con 0x7fef9c158360 2023-10-16T09:59:51.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fefa1b91700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0c75d0 con 0x7fef9c0ceb60 2023-10-16T09:59:51.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fefa1b91700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0c78b0 con 0x7fef9c15ac20 2023-10-16T09:59:51.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.090+0000 7fef9affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c158360 0x7fef9c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c158360 0x7fef9c0c93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42874/0 (socket says 172.21.15.181:42874) 2023-10-16T09:59:51.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 -- 172.21.15.181:0/1056223278 learned_addr learned my addr 172.21.15.181:0/1056223278 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:51.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9b7fe700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef9c0ceb60 0x7fef9c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9bfff700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c15ac20 0x7fef9c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 -- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c15ac20 msgr2=0x7fef9c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c15ac20 0x7fef9c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 -- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef9c0ceb60 msgr2=0x7fef9c0c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef9c0ceb60 0x7fef9c0c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 -- 172.21.15.181:0/1056223278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0094a0 con 0x7fef9c158360 2023-10-16T09:59:51.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9b7fe700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef9c0ceb60 0x7fef9c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:51.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef9affd700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c158360 0x7fef9c0c93e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0090b0 tx=0x7fef8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:51.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef83fff700 1 -- 172.21.15.181:0/1056223278 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0036a0 con 0x7fef9c158360 2023-10-16T09:59:51.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef83fff700 1 -- 172.21.15.181:0/1056223278 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c003800 con 0x7fef9c158360 2023-10-16T09:59:51.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c0c7b90 con 0x7fef9c158360 2023-10-16T09:59:51.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fef83fff700 1 -- 172.21.15.181:0/1056223278 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c003bb0 con 0x7fef9c158360 2023-10-16T09:59:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.091+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c157630 con 0x7fef9c158360 2023-10-16T09:59:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.092+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0c80b0 con 0x7fef9c158360 2023-10-16T09:59:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.093+0000 7fef83fff700 1 -- 172.21.15.181:0/1056223278 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef8c003d10 con 0x7fef9c158360 2023-10-16T09:59:51.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.093+0000 7fef83fff700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fef8806b020 0x7fef8806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.093+0000 7fef9b7fe700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fef8806b020 0x7fef8806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.093+0000 7fef83fff700 1 -- 172.21.15.181:0/1056223278 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fef8c08ae00 con 0x7fef9c158360 2023-10-16T09:59:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.093+0000 7fef9b7fe700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fef8806b020 0x7fef8806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef90000f60 tx=0x7fef90007ca0 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:51.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.098+0000 7fef83fff700 1 -- 172.21.15.181:0/1056223278 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fef9c0c80b0 con 0x7fef9c158360 2023-10-16T09:59:51.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.288+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef9c0c80b0 con 0x7fef9c158360 2023-10-16T09:59:51.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.288+0000 7fef83fff700 1 -- 172.21.15.181:0/1056223278 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef9c0c80b0 con 0x7fef9c158360 2023-10-16T09:59:51.290 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:51.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fef8806b020 msgr2=0x7fef8806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fef8806b020 0x7fef8806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef90000f60 tx=0x7fef90007ca0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c158360 msgr2=0x7fef9c0c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c158360 0x7fef9c0c93e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0090b0 tx=0x7fef8c004750 comp rx=0 tx=0).stop 2023-10-16T09:59:51.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 shutdown_connections 2023-10-16T09:59:51.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fef8806b020 0x7fef8806d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9c15ac20 0x7fef9c156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef9c0ceb60 0x7fef9c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 --2- 172.21.15.181:0/1056223278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9c158360 0x7fef9c0c93e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.290+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 >> 172.21.15.181:0/1056223278 conn(0x7fef9c0cc730 msgr2=0x7fef9c0af880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:51.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.291+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 shutdown_connections 2023-10-16T09:59:51.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.291+0000 7fefa1b91700 1 -- 172.21.15.181:0/1056223278 wait complete. 2023-10-16T09:59:51.304 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-16T09:59:51.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.422+0000 7f1310975700 1 Processor -- start 2023-10-16T09:59:51.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.422+0000 7f1310975700 1 -- start start 2023-10-16T09:59:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.423+0000 7f1310975700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f130c15dfd0 0x7f130c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.423+0000 7f1310975700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c0af450 0x7f130c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.423+0000 7f1310975700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c06f870 con 0x7f130c06db80 2023-10-16T09:59:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.423+0000 7f1310975700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0d1160 con 0x7f130c15dfd0 2023-10-16T09:59:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.423+0000 7f1310975700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0cc8e0 con 0x7f130c0af450 2023-10-16T09:59:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130ad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f130c06db80 0x7f130c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53962/0 (socket says 172.21.15.181:53962) 2023-10-16T09:59:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130ad9d700 1 -- 172.21.15.181:0/846101741 learned_addr learned my addr 172.21.15.181:0/846101741 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130a59c700 1 --2- 172.21.15.181:0/846101741 >> v2:172.21.15.190:3300/0 conn(0x7f130c15dfd0 0x7f130c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f1309d9b700 1 --2- 172.21.15.181:0/846101741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c0af450 0x7f130c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130959a700 1 -- 172.21.15.181:0/846101741 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270168565 0 0) 0x7f130c06f870 con 0x7f130c06db80 2023-10-16T09:59:51.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130959a700 1 -- 172.21.15.181:0/846101741 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f40035b0 con 0x7f130c06db80 2023-10-16T09:59:51.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f1309d9b700 1 -- 172.21.15.181:0/846101741 >> v2:172.21.15.190:3300/0 conn(0x7f130c15dfd0 msgr2=0x7f130c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:51.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f1309d9b700 1 --2- 172.21.15.181:0/846101741 >> v2:172.21.15.190:3300/0 conn(0x7f130c15dfd0 0x7f130c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f1309d9b700 1 -- 172.21.15.181:0/846101741 >> v1:172.21.15.181:6789/0 conn(0x7f130c06db80 legacy=0x7f130c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f1309d9b700 1 -- 172.21.15.181:0/846101741 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0ccbc0 con 0x7f130c0af450 2023-10-16T09:59:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130a59c700 1 --2- 172.21.15.181:0/846101741 >> v2:172.21.15.190:3300/0 conn(0x7f130c15dfd0 0x7f130c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f1309d9b700 1 --2- 172.21.15.181:0/846101741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c0af450 0x7f130c06d640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009840 tx=0x7f12fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0761939f130d64 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130959a700 1 -- 172.21.15.181:0/846101741 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc011830 con 0x7f130c0af450 2023-10-16T09:59:51.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130959a700 1 -- 172.21.15.181:0/846101741 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc011990 con 0x7f130c0af450 2023-10-16T09:59:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.424+0000 7f130959a700 1 -- 172.21.15.181:0/846101741 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc011d00 con 0x7f130c0af450 2023-10-16T09:59:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 -- 172.21.15.181:0/846101741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c0af450 msgr2=0x7f130c06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 --2- 172.21.15.181:0/846101741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c0af450 0x7f130c06d640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009840 tx=0x7f12fc009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 -- 172.21.15.181:0/846101741 shutdown_connections 2023-10-16T09:59:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 --2- 172.21.15.181:0/846101741 >> v2:172.21.15.190:3300/0 conn(0x7f130c15dfd0 0x7f130c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 --2- 172.21.15.181:0/846101741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c0af450 0x7f130c06d640 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 -- 172.21.15.181:0/846101741 >> 172.21.15.181:0/846101741 conn(0x7f130c0d04a0 msgr2=0x7f130c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:51.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 -- 172.21.15.181:0/846101741 shutdown_connections 2023-10-16T09:59:51.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.425+0000 7f1310975700 1 -- 172.21.15.181:0/846101741 wait complete. 2023-10-16T09:59:51.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.426+0000 7f1310975700 1 Processor -- start 2023-10-16T09:59:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.426+0000 7f1310975700 1 -- start start 2023-10-16T09:59:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f1310975700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130c06db80 0x7f130c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f1310975700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f130c0af450 0x7f130c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f1310975700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c15dfd0 0x7f130c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f1310975700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0b1f40 con 0x7f130c06db80 2023-10-16T09:59:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f1310975700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0b0ea0 con 0x7f130c0af450 2023-10-16T09:59:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f1310975700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0b1180 con 0x7f130c15dfd0 2023-10-16T09:59:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f130a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130c06db80 0x7f130c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f130a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130c06db80 0x7f130c158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42886/0 (socket says 172.21.15.181:42886) 2023-10-16T09:59:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f130a59c700 1 -- 172.21.15.181:0/658042702 learned_addr learned my addr 172.21.15.181:0/658042702 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:51.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f130ad9d700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c15dfd0 0x7f130c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.427+0000 7f1309d9b700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f130c0af450 0x7f130c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f130a59c700 1 -- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c15dfd0 msgr2=0x7f130c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f130a59c700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c15dfd0 0x7f130c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f130a59c700 1 -- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f130c0af450 msgr2=0x7f130c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f130a59c700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f130c0af450 0x7f130c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f130a59c700 1 -- 172.21.15.181:0/658042702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300009430 con 0x7f130c06db80 2023-10-16T09:59:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f130a59c700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130c06db80 0x7f130c158de0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f130000ebe0 tx=0x7f130000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f12faffd700 1 -- 172.21.15.181:0/658042702 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300009740 con 0x7f130c06db80 2023-10-16T09:59:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f12faffd700 1 -- 172.21.15.181:0/658042702 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1300005a20 con 0x7f130c06db80 2023-10-16T09:59:51.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc0094a0 con 0x7f130c06db80 2023-10-16T09:59:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f12faffd700 1 -- 172.21.15.181:0/658042702 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300005db0 con 0x7f130c06db80 2023-10-16T09:59:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.428+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c0b1740 con 0x7f130c06db80 2023-10-16T09:59:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.429+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c0b1d90 con 0x7f130c06db80 2023-10-16T09:59:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.429+0000 7f12faffd700 1 -- 172.21.15.181:0/658042702 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f130000c850 con 0x7f130c06db80 2023-10-16T09:59:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.430+0000 7f12faffd700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f12ec06b020 0x7f12ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.430+0000 7f12faffd700 1 -- 172.21.15.181:0/658042702 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f1300089ad0 con 0x7f130c06db80 2023-10-16T09:59:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.430+0000 7f1309d9b700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f12ec06b020 0x7f12ec06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.430+0000 7f1309d9b700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f12ec06b020 0x7f12ec06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f12fc000c00 tx=0x7f12fc008030 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.434+0000 7f12faffd700 1 -- 172.21.15.181:0/658042702 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1300054890 con 0x7f130c06db80 2023-10-16T09:59:51.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:51.706+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f130c067d20 con 0x7f130c06db80 2023-10-16T09:59:52.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.603+0000 7f12faffd700 1 -- 172.21.15.181:0/658042702 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v579) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1300057eb0 con 0x7f130c06db80 2023-10-16T09:59:52.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f12ec06b020 msgr2=0x7f12ec06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f12ec06b020 0x7f12ec06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f12fc000c00 tx=0x7f12fc008030 comp rx=0 tx=0).stop 2023-10-16T09:59:52.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130c06db80 msgr2=0x7f130c158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130c06db80 0x7f130c158de0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f130000ebe0 tx=0x7f130000eef0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 shutdown_connections 2023-10-16T09:59:52.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f12ec06b020 0x7f12ec06d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f130c15dfd0 0x7f130c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130c06db80 0x7f130c158de0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 --2- 172.21.15.181:0/658042702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f130c0af450 0x7f130c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 >> 172.21.15.181:0/658042702 conn(0x7f130c0d04a0 msgr2=0x7f130c15ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:52.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 shutdown_connections 2023-10-16T09:59:52.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.606+0000 7f1310975700 1 -- 172.21.15.181:0/658042702 wait complete. 2023-10-16T09:59:52.623 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T09:59:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.750+0000 7f2d1b09e700 1 Processor -- start 2023-10-16T09:59:52.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.750+0000 7f2d1b09e700 1 -- start start 2023-10-16T09:59:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d1b09e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d140b2010 0x7f2d140cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d1b09e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d1b09e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d140ce1f0 con 0x7f2d1406f3b0 2023-10-16T09:59:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d1b09e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d140b2930 con 0x7f2d140b2010 2023-10-16T09:59:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d1b09e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140ccda0 con 0x7f2d14071be0 2023-10-16T09:59:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d18e3a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d140b2010 0x7f2d140cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d1963b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2d1406f3b0 0x7f2d1406f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53976/0 (socket says 172.21.15.181:53976) 2023-10-16T09:59:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d1963b700 1 -- 172.21.15.181:0/2543368609 learned_addr learned my addr 172.21.15.181:0/2543368609 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d13fff700 1 --2- 172.21.15.181:0/2543368609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d18e3a700 1 -- 172.21.15.181:0/2543368609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 msgr2=0x7f2d140b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d18e3a700 1 --2- 172.21.15.181:0/2543368609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d18e3a700 1 -- 172.21.15.181:0/2543368609 >> v1:172.21.15.181:6789/0 conn(0x7f2d1406f3b0 legacy=0x7f2d1406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d137fe700 1 -- 172.21.15.181:0/2543368609 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157230402 0 0) 0x7f2d140ce1f0 con 0x7f2d1406f3b0 2023-10-16T09:59:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d18e3a700 1 -- 172.21.15.181:0/2543368609 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140727e0 con 0x7f2d140b2010 2023-10-16T09:59:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.751+0000 7f2d13fff700 1 --2- 172.21.15.181:0/2543368609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d18e3a700 1 --2- 172.21.15.181:0/2543368609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d140cc6e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009840 tx=0x7f2d08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02db73151f9114f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d137fe700 1 -- 172.21.15.181:0/2543368609 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08011830 con 0x7f2d140b2010 2023-10-16T09:59:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d137fe700 1 -- 172.21.15.181:0/2543368609 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d08011990 con 0x7f2d140b2010 2023-10-16T09:59:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d137fe700 1 -- 172.21.15.181:0/2543368609 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08011d20 con 0x7f2d140b2010 2023-10-16T09:59:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 -- 172.21.15.181:0/2543368609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 msgr2=0x7f2d140cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/2543368609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d140cc6e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009840 tx=0x7f2d08009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 -- 172.21.15.181:0/2543368609 shutdown_connections 2023-10-16T09:59:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/2543368609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d140cc6e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/2543368609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 -- 172.21.15.181:0/2543368609 >> 172.21.15.181:0/2543368609 conn(0x7f2d140cd010 msgr2=0x7f2d140cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 -- 172.21.15.181:0/2543368609 shutdown_connections 2023-10-16T09:59:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.752+0000 7f2d1b09e700 1 -- 172.21.15.181:0/2543368609 wait complete. 2023-10-16T09:59:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.753+0000 7f2d1b09e700 1 Processor -- start 2023-10-16T09:59:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1b09e700 1 -- start start 2023-10-16T09:59:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1b09e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d1406f3b0 0x7f2d140ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1b09e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1b09e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d14152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1b09e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140aeb90 con 0x7f2d140b2010 2023-10-16T09:59:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1b09e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140ca450 con 0x7f2d1406f3b0 2023-10-16T09:59:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1b09e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140adf80 con 0x7f2d14071be0 2023-10-16T09:59:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.754+0000 7f2d1963b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d14152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d14152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42904/0 (socket says 172.21.15.181:42904) 2023-10-16T09:59:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 -- 172.21.15.181:0/4241154007 learned_addr learned my addr 172.21.15.181:0/4241154007 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d18e3a700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d1406f3b0 0x7f2d140ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d13fff700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 -- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 msgr2=0x7f2d140c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 -- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d1406f3b0 msgr2=0x7f2d140ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d1406f3b0 0x7f2d140ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 -- 172.21.15.181:0/4241154007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080094a0 con 0x7f2d140b2010 2023-10-16T09:59:52.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d18e3a700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d1406f3b0 0x7f2d140ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:52.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1963b700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d14152ea0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00cba0 tx=0x7f2d0c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:52.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d117fa700 1 -- 172.21.15.181:0/4241154007 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00ab80 con 0x7f2d140b2010 2023-10-16T09:59:52.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d117fa700 1 -- 172.21.15.181:0/4241154007 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c00ad00 con 0x7f2d140b2010 2023-10-16T09:59:52.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d140ae260 con 0x7f2d140b2010 2023-10-16T09:59:52.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d117fa700 1 -- 172.21.15.181:0/4241154007 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c007310 con 0x7f2d140b2010 2023-10-16T09:59:52.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.755+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d140ae820 con 0x7f2d140b2010 2023-10-16T09:59:52.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.756+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d140aecd0 con 0x7f2d140b2010 2023-10-16T09:59:52.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.757+0000 7f2d117fa700 1 -- 172.21.15.181:0/4241154007 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2d0c007470 con 0x7f2d140b2010 2023-10-16T09:59:52.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.757+0000 7f2d117fa700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2cf806b020 0x7f2cf806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:52.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.757+0000 7f2d117fa700 1 -- 172.21.15.181:0/4241154007 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f2d0c089680 con 0x7f2d140b2010 2023-10-16T09:59:52.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.757+0000 7f2d18e3a700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2cf806b020 0x7f2cf806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:52.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.757+0000 7f2d18e3a700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2cf806b020 0x7f2cf806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009b90 tx=0x7f2d08008030 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:52.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.762+0000 7f2d117fa700 1 -- 172.21.15.181:0/4241154007 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d0c054550 con 0x7f2d140b2010 2023-10-16T09:59:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.955+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2d140b6510 con 0x7f2d140b2010 2023-10-16T09:59:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.959+0000 7f2d117fa700 1 -- 172.21.15.181:0/4241154007 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2d0c057b70 con 0x7f2d140b2010 2023-10-16T09:59:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2cf806b020 msgr2=0x7f2cf806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2cf806b020 0x7f2cf806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009b90 tx=0x7f2d08008030 comp rx=0 tx=0).stop 2023-10-16T09:59:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 msgr2=0x7f2d14152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d14152ea0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00cba0 tx=0x7f2d0c00cf60 comp rx=0 tx=0).stop 2023-10-16T09:59:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 shutdown_connections 2023-10-16T09:59:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2cf806b020 0x7f2cf806d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d140b2010 0x7f2d14152ea0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d1406f3b0 0x7f2d140ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 --2- 172.21.15.181:0/4241154007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d14071be0 0x7f2d140c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 >> 172.21.15.181:0/4241154007 conn(0x7f2d140cd010 msgr2=0x7f2d140d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 shutdown_connections 2023-10-16T09:59:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:52.961+0000 7f2d1b09e700 1 -- 172.21.15.181:0/4241154007 wait complete. 2023-10-16T09:59:52.975 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T09:59:52.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:53.016 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T09:59:53.068 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T09:59:53.068 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:53.069 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:53.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:59:53.087 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:53.087 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:53.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:53.107 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T09:59:53.108 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:53.144 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T09:59:53.144 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T09:59:53.145 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T09:59:53.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T09:59:53.166 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T09:59:53.166 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T09:59:53.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.302+0000 7fcf72091700 1 Processor -- start 2023-10-16T09:59:53.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.302+0000 7fcf72091700 1 -- start start 2023-10-16T09:59:53.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf72091700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c15dfd0 0x7fcf6c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf72091700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf6c0af450 0x7fcf6c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf72091700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6c06f870 con 0x7fcf6c06db80 2023-10-16T09:59:53.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf72091700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6c0d1160 con 0x7fcf6c0af450 2023-10-16T09:59:53.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf72091700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0cc8e0 con 0x7fcf6c15dfd0 2023-10-16T09:59:53.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c15dfd0 0x7fcf6c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf6c0af450 0x7fcf6c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c15dfd0 0x7fcf6c0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:39934/0 (socket says 172.21.15.181:39934) 2023-10-16T09:59:53.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6bfff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcf6c06db80 0x7fcf6c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:37932/0 (socket says 172.21.15.181:37932) 2023-10-16T09:59:53.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6b7fe700 1 -- 172.21.15.181:0/1649593314 learned_addr learned my addr 172.21.15.181:0/1649593314 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:53.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6affd700 1 -- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c15dfd0 msgr2=0x7fcf6c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:53.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6affd700 1 --2- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c15dfd0 0x7fcf6c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6affd700 1 -- 172.21.15.181:0/1649593314 >> v1:172.21.15.190:6789/0 conn(0x7fcf6c06db80 legacy=0x7fcf6c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6affd700 1 -- 172.21.15.181:0/1649593314 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c0ccbc0 con 0x7fcf6c0af450 2023-10-16T09:59:53.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6a7fc700 1 -- 172.21.15.181:0/1649593314 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716690238 0 0) 0x7fcf6c06f870 con 0x7fcf6c06db80 2023-10-16T09:59:53.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.303+0000 7fcf6b7fe700 1 --2- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c15dfd0 0x7fcf6c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:53.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf6affd700 1 --2- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c0af450 0x7fcf6c06d640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009840 tx=0x7fcf60009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5af503cbabf3030 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:53.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf6a7fc700 1 -- 172.21.15.181:0/1649593314 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60011830 con 0x7fcf6c0af450 2023-10-16T09:59:53.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf6a7fc700 1 -- 172.21.15.181:0/1649593314 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60011990 con 0x7fcf6c0af450 2023-10-16T09:59:53.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf6a7fc700 1 -- 172.21.15.181:0/1649593314 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60011d20 con 0x7fcf6c0af450 2023-10-16T09:59:53.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 -- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c0af450 msgr2=0x7fcf6c06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 --2- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c0af450 0x7fcf6c06d640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009840 tx=0x7fcf60009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:53.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 -- 172.21.15.181:0/1649593314 shutdown_connections 2023-10-16T09:59:53.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 --2- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c15dfd0 0x7fcf6c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 --2- 172.21.15.181:0/1649593314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c0af450 0x7fcf6c06d640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 -- 172.21.15.181:0/1649593314 >> 172.21.15.181:0/1649593314 conn(0x7fcf6c0d04a0 msgr2=0x7fcf6c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:53.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 -- 172.21.15.181:0/1649593314 shutdown_connections 2023-10-16T09:59:53.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.304+0000 7fcf72091700 1 -- 172.21.15.181:0/1649593314 wait complete. 2023-10-16T09:59:53.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.305+0000 7fcf72091700 1 Processor -- start 2023-10-16T09:59:53.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf72091700 1 -- start start 2023-10-16T09:59:53.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf72091700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c06db80 0x7fcf6c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf72091700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf6c0af450 0x7fcf6c0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf72091700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c15dfd0 0x7fcf6c15e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf72091700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b3d90 con 0x7fcf6c0af450 2023-10-16T09:59:53.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf72091700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b4050 con 0x7fcf6c15dfd0 2023-10-16T09:59:53.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf72091700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b4330 con 0x7fcf6c06db80 2023-10-16T09:59:53.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf6affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf6c0af450 0x7fcf6c0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf6affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf6c0af450 0x7fcf6c0ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42918/0 (socket says 172.21.15.181:42918) 2023-10-16T09:59:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf6b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c06db80 0x7fcf6c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf6affd700 1 -- 172.21.15.181:0/2633900216 learned_addr learned my addr 172.21.15.181:0/2633900216 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.306+0000 7fcf6bfff700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c15dfd0 0x7fcf6c15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf6affd700 1 -- 172.21.15.181:0/2633900216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c06db80 msgr2=0x7fcf6c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf6affd700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c06db80 0x7fcf6c0ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf6affd700 1 -- 172.21.15.181:0/2633900216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c15dfd0 msgr2=0x7fcf6c15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf6affd700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c15dfd0 0x7fcf6c15e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf6affd700 1 -- 172.21.15.181:0/2633900216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c009500 con 0x7fcf6c0af450 2023-10-16T09:59:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf6affd700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf6c0af450 0x7fcf6c0ae7d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009fd0 tx=0x7fcf60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf53fff700 1 -- 172.21.15.181:0/2633900216 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60004510 con 0x7fcf6c0af450 2023-10-16T09:59:53.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf53fff700 1 -- 172.21.15.181:0/2633900216 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6001f070 con 0x7fcf6c0af450 2023-10-16T09:59:53.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf53fff700 1 -- 172.21.15.181:0/2633900216 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf600038e0 con 0x7fcf6c0af450 2023-10-16T09:59:53.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf600094a0 con 0x7fcf6c0af450 2023-10-16T09:59:53.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.307+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6c0b48f0 con 0x7fcf6c0af450 2023-10-16T09:59:53.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.308+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6c1616a0 con 0x7fcf6c0af450 2023-10-16T09:59:53.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.308+0000 7fcf53fff700 1 -- 172.21.15.181:0/2633900216 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf60003a40 con 0x7fcf6c0af450 2023-10-16T09:59:53.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.309+0000 7fcf53fff700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf5806b020 0x7fcf5806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.309+0000 7fcf53fff700 1 -- 172.21.15.181:0/2633900216 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fcf60021070 con 0x7fcf6c0af450 2023-10-16T09:59:53.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.309+0000 7fcf6b7fe700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf5806b020 0x7fcf5806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.309+0000 7fcf6b7fe700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf5806b020 0x7fcf5806d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c006fd0 tx=0x7fcf5c009290 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:53.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.313+0000 7fcf53fff700 1 -- 172.21.15.181:0/2633900216 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf60055e40 con 0x7fcf6c0af450 2023-10-16T09:59:53.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.551+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf6c067d20 con 0x7fcf6c0af450 2023-10-16T09:59:53.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.551+0000 7fcf53fff700 1 -- 172.21.15.181:0/2633900216 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fcf60015090 con 0x7fcf6c0af450 2023-10-16T09:59:53.554 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:53.554 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5693,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2852689416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2852689416},{"type":"v1","addr":"172.21.15.190:6834","nonce":2852689416}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6388,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2852959896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2852959896},{"type":"v1","addr":"172.21.15.190:6835","nonce":2852959896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6456,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2993219349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2993219349},{"type":"v1","addr":"172.21.15.181:6836","nonce":2993219349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2504788411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2504788411},{"type":"v1","addr":"172.21.15.181:6837","nonce":2504788411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-10-16T09:59:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf5806b020 msgr2=0x7fcf5806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf5806b020 0x7fcf5806d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c006fd0 tx=0x7fcf5c009290 comp rx=0 tx=0).stop 2023-10-16T09:59:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf6c0af450 msgr2=0x7fcf6c0ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf6c0af450 0x7fcf6c0ae7d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009fd0 tx=0x7fcf60004750 comp rx=0 tx=0).stop 2023-10-16T09:59:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 shutdown_connections 2023-10-16T09:59:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf5806b020 0x7fcf5806d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf6c06db80 0x7fcf6c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf6c15dfd0 0x7fcf6c15e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 --2- 172.21.15.181:0/2633900216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf6c0af450 0x7fcf6c0ae7d0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.553+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 >> 172.21.15.181:0/2633900216 conn(0x7fcf6c0d04a0 msgr2=0x7fcf6c0cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.554+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 shutdown_connections 2023-10-16T09:59:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.554+0000 7fcf72091700 1 -- 172.21.15.181:0/2633900216 wait complete. 2023-10-16T09:59:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 34 2023-10-16T09:59:53.567 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T09:59:53.567 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T09:59:53.568 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T09:59:53.568 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:53.568 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T09:59:53.569 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T09:59:53.569 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T09:59:53.569 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:53.570 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T09:59:53.570 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T09:59:53.570 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T09:59:53.570 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:53.571 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T09:59:53.571 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T09:59:53.571 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T09:59:53.572 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T09:59:53.577 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T09:59:53.578 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T09:59:53.579 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T09:59:53.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T09:59:53.581 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T09:59:53.582 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T09:59:53.583 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T09:59:53.583 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T09:59:53.620 INFO:tasks.ceph.mds.a:Started 2023-10-16T09:59:53.620 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:53.621 INFO:tasks.ceph.mds.c:Started 2023-10-16T09:59:53.621 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:53.625 INFO:tasks.ceph.mds.b:Started 2023-10-16T09:59:53.625 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:53.626 INFO:tasks.ceph.mds.d:Started 2023-10-16T09:59:53.626 DEBUG:teuthology.parallel:result is None 2023-10-16T09:59:53.627 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T09:59:53.698 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T09:59:53.699 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T09:59:53.714 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T09:59:53.714 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T09:59:53.715 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-10-16T09:59:53.713+0000 7f56f953d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-16T09:59:53.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.752+0000 7fbfcd950700 1 Processor -- start 2023-10-16T09:59:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.752+0000 7fbfcd950700 1 -- start start 2023-10-16T09:59:53.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.752+0000 7fbfcd950700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cfc60 0x7fbfc80d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.752+0000 7fbfcd950700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc815bd50 0x7fbfc80d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.752+0000 7fbfcd950700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc80d24f0 con 0x7fbfc80b05d0 2023-10-16T09:59:53.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.752+0000 7fbfcd950700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc815ccc0 con 0x7fbfc80cfc60 2023-10-16T09:59:53.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.752+0000 7fbfcd950700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80d0580 con 0x7fbfc815bd50 2023-10-16T09:59:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc67fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc815bd50 0x7fbfc80d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc6ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cfc60 0x7fbfc80d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc77fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbfc80b05d0 0x7fbfc815f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:37970/0 (socket says 172.21.15.181:37970) 2023-10-16T09:59:53.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc77fe700 1 -- 172.21.15.181:0/3250036390 learned_addr learned my addr 172.21.15.181:0/3250036390 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc5ffb700 1 -- 172.21.15.181:0/3250036390 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853978618 0 0) 0x7fbfc80d24f0 con 0x7fbfc80b05d0 2023-10-16T09:59:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc5ffb700 1 -- 172.21.15.181:0/3250036390 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb00035b0 con 0x7fbfc80b05d0 2023-10-16T09:59:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc67fc700 1 -- 172.21.15.181:0/3250036390 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cfc60 msgr2=0x7fbfc80d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc67fc700 1 --2- 172.21.15.181:0/3250036390 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cfc60 0x7fbfc80d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc67fc700 1 -- 172.21.15.181:0/3250036390 >> v1:172.21.15.190:6789/0 conn(0x7fbfc80b05d0 legacy=0x7fbfc815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc67fc700 1 -- 172.21.15.181:0/3250036390 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc815c320 con 0x7fbfc815bd50 2023-10-16T09:59:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc6ffd700 1 --2- 172.21.15.181:0/3250036390 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cfc60 0x7fbfc80d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc67fc700 1 --2- 172.21.15.181:0/3250036390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc815bd50 0x7fbfc80d0040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009840 tx=0x7fbfbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f88f05830942bad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc5ffb700 1 -- 172.21.15.181:0/3250036390 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc011830 con 0x7fbfc815bd50 2023-10-16T09:59:53.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc5ffb700 1 -- 172.21.15.181:0/3250036390 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc011990 con 0x7fbfc815bd50 2023-10-16T09:59:53.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.753+0000 7fbfc5ffb700 1 -- 172.21.15.181:0/3250036390 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc011d20 con 0x7fbfc815bd50 2023-10-16T09:59:53.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 -- 172.21.15.181:0/3250036390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc815bd50 msgr2=0x7fbfc80d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 --2- 172.21.15.181:0/3250036390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc815bd50 0x7fbfc80d0040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009840 tx=0x7fbfbc009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:53.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 -- 172.21.15.181:0/3250036390 shutdown_connections 2023-10-16T09:59:53.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 --2- 172.21.15.181:0/3250036390 >> v2:172.21.15.190:3300/0 conn(0x7fbfc80cfc60 0x7fbfc80d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 --2- 172.21.15.181:0/3250036390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc815bd50 0x7fbfc80d0040 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 -- 172.21.15.181:0/3250036390 >> 172.21.15.181:0/3250036390 conn(0x7fbfc809a9a0 msgr2=0x7fbfc80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 -- 172.21.15.181:0/3250036390 shutdown_connections 2023-10-16T09:59:53.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.754+0000 7fbfcd950700 1 -- 172.21.15.181:0/3250036390 wait complete. 2023-10-16T09:59:53.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.755+0000 7fbfcd950700 1 Processor -- start 2023-10-16T09:59:53.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.756+0000 7fbfcd950700 1 -- start start 2023-10-16T09:59:53.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80b05d0 0x7fbfc8153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cfc60 0x7fbfc80af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc815bd50 0x7fbfc80af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80b52c0 con 0x7fbfc80cfc60 2023-10-16T09:59:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8154210 con 0x7fbfc815bd50 2023-10-16T09:59:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80b54a0 con 0x7fbfc80b05d0 2023-10-16T09:59:53.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cfc60 0x7fbfc80af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cfc60 0x7fbfc80af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42954/0 (socket says 172.21.15.181:42954) 2023-10-16T09:59:53.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 -- 172.21.15.181:0/119141200 learned_addr learned my addr 172.21.15.181:0/119141200 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc6ffd700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80b05d0 0x7fbfc8153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc77fe700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc815bd50 0x7fbfc80af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 -- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80b05d0 msgr2=0x7fbfc8153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80b05d0 0x7fbfc8153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 -- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc815bd50 msgr2=0x7fbfc80af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc815bd50 0x7fbfc80af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 -- 172.21.15.181:0/119141200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80094a0 con 0x7fbfc80cfc60 2023-10-16T09:59:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc77fe700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc815bd50 0x7fbfc80af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfc67fc700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cfc60 0x7fbfc80af1e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009fd0 tx=0x7fbfbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfaffff700 1 -- 172.21.15.181:0/119141200 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004590 con 0x7fbfc80cfc60 2023-10-16T09:59:53.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfaffff700 1 -- 172.21.15.181:0/119141200 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc01e070 con 0x7fbfc80cfc60 2023-10-16T09:59:53.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc0094a0 con 0x7fbfc80cfc60 2023-10-16T09:59:53.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfaffff700 1 -- 172.21.15.181:0/119141200 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0038e0 con 0x7fbfc80cfc60 2023-10-16T09:59:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80c9ee0 con 0x7fbfc80cfc60 2023-10-16T09:59:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.758+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc8067d20 con 0x7fbfc80cfc60 2023-10-16T09:59:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.759+0000 7fbfaffff700 1 -- 172.21.15.181:0/119141200 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbfbc003a40 con 0x7fbfc80cfc60 2023-10-16T09:59:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.759+0000 7fbfaffff700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fbfb406b020 0x7fbfb406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.759+0000 7fbfaffff700 1 -- 172.21.15.181:0/119141200 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fbfbc08a6f0 con 0x7fbfc80cfc60 2023-10-16T09:59:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.759+0000 7fbfc6ffd700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fbfb406b020 0x7fbfb406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.760+0000 7fbfc6ffd700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fbfb406b020 0x7fbfb406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009d70 tx=0x7fbfb8009210 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:53.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.764+0000 7fbfaffff700 1 -- 172.21.15.181:0/119141200 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfbc055540 con 0x7fbfc80cfc60 2023-10-16T09:59:53.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:53.977+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbfc80cfa40 con 0x7fbfc80cfc60 2023-10-16T09:59:54.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.984+0000 7fbfaffff700 1 -- 172.21.15.181:0/119141200 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbfbc058b60 con 0x7fbfc80cfc60 2023-10-16T09:59:54.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fbfb406b020 msgr2=0x7fbfb406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:54.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fbfb406b020 0x7fbfb406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009d70 tx=0x7fbfb8009210 comp rx=0 tx=0).stop 2023-10-16T09:59:54.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cfc60 msgr2=0x7fbfc80af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:54.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cfc60 0x7fbfc80af1e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009fd0 tx=0x7fbfbc004750 comp rx=0 tx=0).stop 2023-10-16T09:59:54.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 shutdown_connections 2023-10-16T09:59:54.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fbfb406b020 0x7fbfb406d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:54.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc80b05d0 0x7fbfc8153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:54.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc815bd50 0x7fbfc80af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:54.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 --2- 172.21.15.181:0/119141200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc80cfc60 0x7fbfc80af1e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:54.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 >> 172.21.15.181:0/119141200 conn(0x7fbfc809a9a0 msgr2=0x7fbfc80cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:54.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 shutdown_connections 2023-10-16T09:59:54.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:54.987+0000 7fbfcd950700 1 -- 172.21.15.181:0/119141200 wait complete. 2023-10-16T09:59:54.991 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T09:59:55.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T09:59:55.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.129+0000 7f285bc63700 1 Processor -- start 2023-10-16T09:59:55.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.130+0000 7f285bc63700 1 -- start start 2023-10-16T09:59:55.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.130+0000 7f285bc63700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28540b1ee0 0x7f28540cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.130+0000 7f285bc63700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.130+0000 7f285bc63700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540cc310 con 0x7f285415e7f0 2023-10-16T09:59:55.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.130+0000 7f285bc63700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285415e0f0 con 0x7f28540b1ee0 2023-10-16T09:59:55.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.130+0000 7f285bc63700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540b2800 con 0x7f28540cf2e0 2023-10-16T09:59:55.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28540b1ee0 0x7f28540cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28540b1ee0 0x7f28540cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42964/0 (socket says 172.21.15.181:42964) 2023-10-16T09:59:55.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 -- 172.21.15.181:0/655007015 learned_addr learned my addr 172.21.15.181:0/655007015 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:55.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28591fe700 1 --2- 172.21.15.181:0/655007015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 -- 172.21.15.181:0/655007015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 msgr2=0x7f28540b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 --2- 172.21.15.181:0/655007015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 -- 172.21.15.181:0/655007015 >> v1:172.21.15.190:6789/0 conn(0x7f285415e7f0 legacy=0x7f285406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 -- 172.21.15.181:0/655007015 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28540cf760 con 0x7f28540b1ee0 2023-10-16T09:59:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28599ff700 1 --2- 172.21.15.181:0/655007015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28540b1ee0 0x7f28540cab40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2844006fd0 tx=0x7f2844009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1321d1af5613c082 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28589fd700 1 -- 172.21.15.181:0/655007015 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844005e30 con 0x7f28540b1ee0 2023-10-16T09:59:55.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28589fd700 1 -- 172.21.15.181:0/655007015 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2844006030 con 0x7f28540b1ee0 2023-10-16T09:59:55.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.131+0000 7f28589fd700 1 -- 172.21.15.181:0/655007015 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28440063c0 con 0x7f28540b1ee0 2023-10-16T09:59:55.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 -- 172.21.15.181:0/655007015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28540b1ee0 msgr2=0x7f28540cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 --2- 172.21.15.181:0/655007015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28540b1ee0 0x7f28540cab40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2844006fd0 tx=0x7f2844009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 -- 172.21.15.181:0/655007015 shutdown_connections 2023-10-16T09:59:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 --2- 172.21.15.181:0/655007015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28540b1ee0 0x7f28540cab40 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 --2- 172.21.15.181:0/655007015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 -- 172.21.15.181:0/655007015 >> 172.21.15.181:0/655007015 conn(0x7f28540d0460 msgr2=0x7f28540d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 -- 172.21.15.181:0/655007015 shutdown_connections 2023-10-16T09:59:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.132+0000 7f285bc63700 1 -- 172.21.15.181:0/655007015 wait complete. 2023-10-16T09:59:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.133+0000 7f285bc63700 1 Processor -- start 2023-10-16T09:59:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.133+0000 7f285bc63700 1 -- start start 2023-10-16T09:59:55.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285bc63700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28540b1ee0 0x7f28540c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285bc63700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285bc63700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285415e7f0 0x7f28540ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285bc63700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540adac0 con 0x7f285415e7f0 2023-10-16T09:59:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285bc63700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540adda0 con 0x7f28540b1ee0 2023-10-16T09:59:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285bc63700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540ae080 con 0x7f28540cf2e0 2023-10-16T09:59:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285415e7f0 0x7f28540ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285415e7f0 0x7f28540ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42974/0 (socket says 172.21.15.181:42974) 2023-10-16T09:59:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 -- 172.21.15.181:0/1880406968 learned_addr learned my addr 172.21.15.181:0/1880406968 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f28591fe700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f28599ff700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28540b1ee0 0x7f28540c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 -- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 msgr2=0x7f28540c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 -- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28540b1ee0 msgr2=0x7f28540c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28540b1ee0 0x7f28540c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f285a200700 1 -- 172.21.15.181:0/1880406968 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844009500 con 0x7f285415e7f0 2023-10-16T09:59:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.134+0000 7f28599ff700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28540b1ee0 0x7f28540c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.135+0000 7f285a200700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285415e7f0 0x7f28540ad070 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c7a0 tx=0x7f284c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.135+0000 7f284a7fc700 1 -- 172.21.15.181:0/1880406968 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c003e10 con 0x7f285415e7f0 2023-10-16T09:59:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.135+0000 7f284a7fc700 1 -- 172.21.15.181:0/1880406968 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c005070 con 0x7f285415e7f0 2023-10-16T09:59:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.135+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28540ae360 con 0x7f285415e7f0 2023-10-16T09:59:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.135+0000 7f284a7fc700 1 -- 172.21.15.181:0/1880406968 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c005420 con 0x7f285415e7f0 2023-10-16T09:59:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.135+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28540ae6f0 con 0x7f285415e7f0 2023-10-16T09:59:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.136+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2854067d20 con 0x7f285415e7f0 2023-10-16T09:59:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.136+0000 7f284a7fc700 1 -- 172.21.15.181:0/1880406968 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f284c005580 con 0x7f285415e7f0 2023-10-16T09:59:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.137+0000 7f284a7fc700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f283806b020 0x7f283806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.137+0000 7f284a7fc700 1 -- 172.21.15.181:0/1880406968 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f284c088ce0 con 0x7f285415e7f0 2023-10-16T09:59:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.137+0000 7f28599ff700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f283806b020 0x7f283806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.137+0000 7f28599ff700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f283806b020 0x7f283806d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2844013040 tx=0x7f2844008070 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.141+0000 7f284a7fc700 1 -- 172.21.15.181:0/1880406968 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f284c053e20 con 0x7f285415e7f0 2023-10-16T09:59:55.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.409+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2854160d80 con 0x7f285415e7f0 2023-10-16T09:59:55.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.409+0000 7f284a7fc700 1 -- 172.21.15.181:0/1880406968 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f284c017070 con 0x7f285415e7f0 2023-10-16T09:59:55.411 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:55.411 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T09:59:55.411 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T09:59:55.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T09:59:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T09:59:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T09:59:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T09:59:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T09:59:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T09:59:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T09:59:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T09:59:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T09:59:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T09:59:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T09:59:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T09:59:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T09:59:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T09:59:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T09:59:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T09:59:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T09:59:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T09:59:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T09:59:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T09:59:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T09:59:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T09:59:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T09:59:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T09:59:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T09:59:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T09:59:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T09:59:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T09:59:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T09:59:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T09:59:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T09:59:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T09:59:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:59:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:59:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:59:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T09:59:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T09:59:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.439 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.439 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:59:55.439 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:59:55.439 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:59:55.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T09:59:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T09:59:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T09:59:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T09:59:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T09:59:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T09:59:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T09:59:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T09:59:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:59:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T09:59:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T09:59:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.445 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T09:59:55.445 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T09:59:55.445 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.446 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:59:55.446 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:59:55.446 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.447 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T09:59:55.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T09:59:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T09:59:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T09:59:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T09:59:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T09:59:55.449 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T09:59:55.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:55.450 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T09:59:55.450 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T09:59:55.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.411+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f283806b020 msgr2=0x7f283806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.411+0000 7f285bc63700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f283806b020 0x7f283806d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2844013040 tx=0x7f2844008070 comp rx=0 tx=0).stop 2023-10-16T09:59:55.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.411+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285415e7f0 msgr2=0x7f28540ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.411+0000 7f285bc63700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285415e7f0 0x7f28540ad070 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c7a0 tx=0x7f284c00cb60 comp rx=0 tx=0).stop 2023-10-16T09:59:55.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 shutdown_connections 2023-10-16T09:59:55.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f283806b020 0x7f283806d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285415e7f0 0x7f28540ad070 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28540b1ee0 0x7f28540c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 --2- 172.21.15.181:0/1880406968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28540cf2e0 0x7f28540c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 >> 172.21.15.181:0/1880406968 conn(0x7f28540d0460 msgr2=0x7f285406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:55.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 shutdown_connections 2023-10-16T09:59:55.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.412+0000 7f285bc63700 1 -- 172.21.15.181:0/1880406968 wait complete. 2023-10-16T09:59:55.457 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T09:59:55.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.576+0000 7f2d54043700 1 Processor -- start 2023-10-16T09:59:55.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.577+0000 7f2d54043700 1 -- start start 2023-10-16T09:59:55.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.577+0000 7f2d54043700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 0x7f2d4c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.577+0000 7f2d54043700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c06ed00 con 0x7f2d4c0cb0c0 2023-10-16T09:59:55.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.577+0000 7f2d54043700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0d02d0 con 0x7f2d4c15eab0 2023-10-16T09:59:55.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.577+0000 7f2d54043700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c06e0f0 con 0x7f2d4c15e6d0 2023-10-16T09:59:55.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d525e0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2d4c0cb0c0 0x7f2d4c15a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54060/0 (socket says 172.21.15.181:54060) 2023-10-16T09:59:55.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d525e0700 1 -- 172.21.15.181:0/2461482956 learned_addr learned my addr 172.21.15.181:0/2461482956 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:55.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63211155 0 0) 0x7f2d4c06ed00 con 0x7f2d4c0cb0c0 2023-10-16T09:59:55.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d380035b0 con 0x7f2d4c0cb0c0 2023-10-16T09:59:55.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d51ddf700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 0x7f2d4c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 72242391 0 0) 0x7f2d380035b0 con 0x7f2d4c0cb0c0 2023-10-16T09:59:55.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 msgr2=0x7f2d4c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:55.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 0x7f2d4c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 >> v1:172.21.15.190:6789/0 conn(0x7f2d4c15eab0 legacy=0x7f2d4c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c06e3d0 con 0x7f2d4c0cb0c0 2023-10-16T09:59:55.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f2d440019f0 con 0x7f2d4c0cb0c0 2023-10-16T09:59:55.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 >> v1:172.21.15.181:6789/0 conn(0x7f2d4c0cb0c0 legacy=0x7f2d4c15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38003a30 0x7f2d38003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 0x7f2d38004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c06e0f0 con 0x7f2d38007ad0 2023-10-16T09:59:55.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380053d0 con 0x7f2d38003a30 2023-10-16T09:59:55.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d38005590 con 0x7f2d4c15e6d0 2023-10-16T09:59:55.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d515de700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d51ddf700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38003a30 0x7f2d38003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.578+0000 7f2d525e0700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 0x7f2d38004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d515de700 1 -- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 msgr2=0x7f2d38004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:55.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d515de700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c15e6d0 0x7f2d38004d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d515de700 1 -- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38003a30 msgr2=0x7f2d38003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:55.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d515de700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d38003a30 0x7f2d38003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d515de700 1 -- 172.21.15.181:0/2461482956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d380035b0 con 0x7f2d38007ad0 2023-10-16T09:59:55.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d515de700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d38004850 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2d480017c0 tx=0x7f2d38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de7c9f3415a6867 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d480052d0 con 0x7f2d38007ad0 2023-10-16T09:59:55.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d48005430 con 0x7f2d38007ad0 2023-10-16T09:59:55.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d50ddd700 1 -- 172.21.15.181:0/2461482956 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d480038c0 con 0x7f2d38007ad0 2023-10-16T09:59:55.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d54043700 1 -- 172.21.15.181:0/2461482956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 msgr2=0x7f2d38004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d54043700 1 --2- 172.21.15.181:0/2461482956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d38004850 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2d480017c0 tx=0x7f2d38005880 comp rx=0 tx=0).stop 2023-10-16T09:59:55.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d525e0700 1 -- 172.21.15.181:0/2461482956 reap_dead start 2023-10-16T09:59:55.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d54043700 1 -- 172.21.15.181:0/2461482956 shutdown_connections 2023-10-16T09:59:55.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.579+0000 7f2d54043700 1 -- 172.21.15.181:0/2461482956 >> 172.21.15.181:0/2461482956 conn(0x7f2d4c0d04a0 msgr2=0x7f2d4c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:55.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.580+0000 7f2d54043700 1 -- 172.21.15.181:0/2461482956 shutdown_connections 2023-10-16T09:59:55.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.580+0000 7f2d54043700 1 -- 172.21.15.181:0/2461482956 wait complete. 2023-10-16T09:59:55.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.581+0000 7f2d54043700 1 Processor -- start 2023-10-16T09:59:55.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.581+0000 7f2d54043700 1 -- start start 2023-10-16T09:59:55.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.581+0000 7f2d54043700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d4c0c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d54043700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0b5b60 0x7f2d4c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d54043700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c14e2f0 0x7f2d4c14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d54043700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c14ece0 con 0x7f2d38007ad0 2023-10-16T09:59:55.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d54043700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c60f0 con 0x7f2d4c0b5b60 2023-10-16T09:59:55.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d54043700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c63d0 con 0x7f2d4c14e2f0 2023-10-16T09:59:55.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d4c0c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d4c0c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42990/0 (socket says 172.21.15.181:42990) 2023-10-16T09:59:55.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 -- 172.21.15.181:0/1003923455 learned_addr learned my addr 172.21.15.181:0/1003923455 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:55.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d525e0700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c14e2f0 0x7f2d4c14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d515de700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0b5b60 0x7f2d4c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 -- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c14e2f0 msgr2=0x7f2d4c14e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c14e2f0 0x7f2d4c14e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 -- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0b5b60 msgr2=0x7f2d4c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0b5b60 0x7f2d4c0ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 -- 172.21.15.181:0/1003923455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0c66b0 con 0x7f2d38007ad0 2023-10-16T09:59:55.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d51ddf700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d4c0c5a30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009fd0 tx=0x7f2d3c011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d427fc700 1 -- 172.21.15.181:0/1003923455 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c009610 con 0x7f2d38007ad0 2023-10-16T09:59:55.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d427fc700 1 -- 172.21.15.181:0/1003923455 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c005c30 con 0x7f2d38007ad0 2023-10-16T09:59:55.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d427fc700 1 -- 172.21.15.181:0/1003923455 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00f730 con 0x7f2d38007ad0 2023-10-16T09:59:55.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c0c6990 con 0x7f2d38007ad0 2023-10-16T09:59:55.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.582+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c0b5fe0 con 0x7f2d38007ad0 2023-10-16T09:59:55.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.583+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c067d20 con 0x7f2d38007ad0 2023-10-16T09:59:55.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.584+0000 7f2d427fc700 1 -- 172.21.15.181:0/1003923455 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2d3c00f890 con 0x7f2d38007ad0 2023-10-16T09:59:55.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.584+0000 7f2d427fc700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2d3006b020 0x7f2d3006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.584+0000 7f2d427fc700 1 -- 172.21.15.181:0/1003923455 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f2d3c089c70 con 0x7f2d38007ad0 2023-10-16T09:59:55.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.584+0000 7f2d515de700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2d3006b020 0x7f2d3006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.585+0000 7f2d515de700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2d3006b020 0x7f2d3006d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c1f2900 tx=0x7f2d48009030 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.589+0000 7f2d427fc700 1 -- 172.21.15.181:0/1003923455 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d3c054e40 con 0x7f2d38007ad0 2023-10-16T09:59:55.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.783+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d4c0cb610 con 0x7f2d38007ad0 2023-10-16T09:59:55.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.784+0000 7f2d427fc700 1 -- 172.21.15.181:0/1003923455 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13537 (secure 0 0 0) 0x7f2d3c021020 con 0x7f2d38007ad0 2023-10-16T09:59:55.785 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:55.785 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":86,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:54.982955+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T09:59:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2d3006b020 msgr2=0x7f2d3006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2d3006b020 0x7f2d3006d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c1f2900 tx=0x7f2d48009030 comp rx=0 tx=0).stop 2023-10-16T09:59:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 msgr2=0x7f2d4c0c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d4c0c5a30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009fd0 tx=0x7f2d3c011d30 comp rx=0 tx=0).stop 2023-10-16T09:59:55.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 shutdown_connections 2023-10-16T09:59:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2d3006b020 0x7f2d3006d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d4c14e2f0 0x7f2d4c14e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d38007ad0 0x7f2d4c0c5a30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 --2- 172.21.15.181:0/1003923455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0b5b60 0x7f2d4c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 >> 172.21.15.181:0/1003923455 conn(0x7f2d4c0d04a0 msgr2=0x7f2d4c0ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:55.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 shutdown_connections 2023-10-16T09:59:55.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.787+0000 7f2d54043700 1 -- 172.21.15.181:0/1003923455 wait complete. 2023-10-16T09:59:55.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T09:59:55.800 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T09:59:55.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.924+0000 7f9fc14d8700 1 Processor -- start 2023-10-16T09:59:55.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fc14d8700 1 -- start start 2023-10-16T09:59:55.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fc14d8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca020 0x7f9fbc071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fc14d8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9fbc0ca5f0 0x7f9fbc0718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fc14d8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc0b4ff0 con 0x7f9fbc160f50 2023-10-16T09:59:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fc14d8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc0c66f0 con 0x7f9fbc0ca5f0 2023-10-16T09:59:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fc14d8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0b5800 con 0x7f9fbc0ca020 2023-10-16T09:59:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fbaffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9fbc160f50 0x7f9fbc161670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54076/0 (socket says 172.21.15.181:54076) 2023-10-16T09:59:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.925+0000 7f9fbaffd700 1 -- 172.21.15.181:0/882625974 learned_addr learned my addr 172.21.15.181:0/882625974 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fba7fc700 1 --2- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca020 0x7f9fbc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fbb7fe700 1 --2- 172.21.15.181:0/882625974 >> v2:172.21.15.190:3300/0 conn(0x7f9fbc0ca5f0 0x7f9fbc0718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fb9ffb700 1 -- 172.21.15.181:0/882625974 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223836788 0 0) 0x7f9fbc0b4ff0 con 0x7f9fbc160f50 2023-10-16T09:59:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fb9ffb700 1 -- 172.21.15.181:0/882625974 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa40035b0 con 0x7f9fbc160f50 2023-10-16T09:59:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fb9ffb700 1 -- 172.21.15.181:0/882625974 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 331625657 0 0) 0x7f9fa40035b0 con 0x7f9fbc160f50 2023-10-16T09:59:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fbb7fe700 1 -- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca020 msgr2=0x7f9fbc071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fbb7fe700 1 --2- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca020 0x7f9fbc071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fbb7fe700 1 -- 172.21.15.181:0/882625974 >> v1:172.21.15.181:6789/0 conn(0x7f9fbc160f50 legacy=0x7f9fbc161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fbb7fe700 1 -- 172.21.15.181:0/882625974 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc0b5240 con 0x7f9fbc0ca5f0 2023-10-16T09:59:55.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fba7fc700 1 --2- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca020 0x7f9fbc071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T09:59:55.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fbb7fe700 1 --2- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc0ca5f0 0x7f9fbc0718c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009d90 tx=0x7f9fb400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d6b056b84be490 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fb9ffb700 1 -- 172.21.15.181:0/882625974 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400e070 con 0x7f9fbc0ca5f0 2023-10-16T09:59:55.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fb9ffb700 1 -- 172.21.15.181:0/882625974 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb4003a20 con 0x7f9fbc0ca5f0 2023-10-16T09:59:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.926+0000 7f9fb9ffb700 1 -- 172.21.15.181:0/882625974 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0001670 con 0x7f9fbc0ca5f0 2023-10-16T09:59:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 -- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc0ca5f0 msgr2=0x7f9fbc0718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc0ca5f0 0x7f9fbc0718c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009d90 tx=0x7f9fb400b330 comp rx=0 tx=0).stop 2023-10-16T09:59:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 -- 172.21.15.181:0/882625974 shutdown_connections 2023-10-16T09:59:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc0ca5f0 0x7f9fbc0718c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/882625974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca020 0x7f9fbc071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 -- 172.21.15.181:0/882625974 >> 172.21.15.181:0/882625974 conn(0x7f9fbc06a8a0 msgr2=0x7f9fbc06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:55.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 -- 172.21.15.181:0/882625974 shutdown_connections 2023-10-16T09:59:55.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.927+0000 7f9fc14d8700 1 -- 172.21.15.181:0/882625974 wait complete. 2023-10-16T09:59:55.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.929+0000 7f9fc14d8700 1 Processor -- start 2023-10-16T09:59:55.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.929+0000 7f9fc14d8700 1 -- start start 2023-10-16T09:59:55.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc0ca020 0x7f9fbc0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca5f0 0x7f9fbc145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc160f50 0x7f9fbc145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0c7130 con 0x7f9fbc0ca020 2023-10-16T09:59:55.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0c8e40 con 0x7f9fbc160f50 2023-10-16T09:59:55.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0c8550 con 0x7f9fbc0ca5f0 2023-10-16T09:59:55.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc0ca020 0x7f9fbc0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc0ca020 0x7f9fbc0c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43004/0 (socket says 172.21.15.181:43004) 2023-10-16T09:59:55.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 -- 172.21.15.181:0/370951527 learned_addr learned my addr 172.21.15.181:0/370951527 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:55.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 -- 172.21.15.181:0/370951527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca5f0 msgr2=0x7f9fbc145920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca5f0 0x7f9fbc145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 -- 172.21.15.181:0/370951527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc160f50 msgr2=0x7f9fbc145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:55.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc160f50 0x7f9fbc145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:55.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 -- 172.21.15.181:0/370951527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac0094a0 con 0x7f9fbc0ca020 2023-10-16T09:59:55.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fbaffd700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc0ca020 0x7f9fbc0c6870 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00011e0 tx=0x7f9fb000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fa37fe700 1 -- 172.21.15.181:0/370951527 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0007560 con 0x7f9fbc0ca020 2023-10-16T09:59:55.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fa37fe700 1 -- 172.21.15.181:0/370951527 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb00076c0 con 0x7f9fbc0ca020 2023-10-16T09:59:55.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb4009a70 con 0x7f9fbc0ca020 2023-10-16T09:59:55.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fa37fe700 1 -- 172.21.15.181:0/370951527 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0004a90 con 0x7f9fbc0ca020 2023-10-16T09:59:55.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.930+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc0c87f0 con 0x7f9fbc0ca020 2023-10-16T09:59:55.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.931+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc06b830 con 0x7f9fbc0ca020 2023-10-16T09:59:55.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.932+0000 7f9fa37fe700 1 -- 172.21.15.181:0/370951527 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9fb0004bf0 con 0x7f9fbc0ca020 2023-10-16T09:59:55.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.932+0000 7f9fa37fe700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9fa806b050 0x7f9fa806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.932+0000 7f9fa37fe700 1 -- 172.21.15.181:0/370951527 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f9fb0088b10 con 0x7f9fbc0ca020 2023-10-16T09:59:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.932+0000 7f9fba7fc700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9fa806b050 0x7f9fa806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.933+0000 7f9fba7fc700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9fa806b050 0x7f9fa806d510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc15f010 tx=0x7f9fac006e60 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:55.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:55.937+0000 7f9fa37fe700 1 -- 172.21.15.181:0/370951527 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9fb0053bb0 con 0x7f9fbc0ca020 2023-10-16T09:59:56.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.127+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9fbc04ebb0 con 0x7f9fbc0ca020 2023-10-16T09:59:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.127+0000 7f9fa37fe700 1 -- 172.21.15.181:0/370951527 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9fb0018020 con 0x7f9fbc0ca020 2023-10-16T09:59:56.128 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T09:59:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9fa806b050 msgr2=0x7f9fa806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9fa806b050 0x7f9fa806d510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc15f010 tx=0x7f9fac006e60 comp rx=0 tx=0).stop 2023-10-16T09:59:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc0ca020 msgr2=0x7f9fbc0c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc0ca020 0x7f9fbc0c6870 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00011e0 tx=0x7f9fb000cb00 comp rx=0 tx=0).stop 2023-10-16T09:59:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 shutdown_connections 2023-10-16T09:59:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9fa806b050 0x7f9fa806d510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fbc160f50 0x7f9fbc145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fbc0ca020 0x7f9fbc0c6870 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 --2- 172.21.15.181:0/370951527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fbc0ca5f0 0x7f9fbc145920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 >> 172.21.15.181:0/370951527 conn(0x7f9fbc06a8a0 msgr2=0x7f9fbc0cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 shutdown_connections 2023-10-16T09:59:56.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.129+0000 7f9fc14d8700 1 -- 172.21.15.181:0/370951527 wait complete. 2023-10-16T09:59:56.142 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T09:59:56.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.261+0000 7f323afb9700 1 Processor -- start 2023-10-16T09:59:56.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.261+0000 7f323afb9700 1 -- start start 2023-10-16T09:59:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f323afb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323415dfd0 0x7f32340af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f323afb9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32340af450 0x7f323406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f323afb9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f323406db80 0x7f32340cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f323afb9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323406f870 con 0x7f32340af450 2023-10-16T09:59:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f323afb9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32340d1160 con 0x7f323406db80 2023-10-16T09:59:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f323afb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32340cc8e0 con 0x7f323415dfd0 2023-10-16T09:59:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3233fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32340af450 0x7f323406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3233fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32340af450 0x7f323406d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43014/0 (socket says 172.21.15.181:43014) 2023-10-16T09:59:56.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3233fff700 1 -- 172.21.15.181:0/3397025341 learned_addr learned my addr 172.21.15.181:0/3397025341 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:56.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3238d55700 1 --2- 172.21.15.181:0/3397025341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323415dfd0 0x7f32340af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3233fff700 1 -- 172.21.15.181:0/3397025341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323415dfd0 msgr2=0x7f32340af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:56.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3239556700 1 --2- 172.21.15.181:0/3397025341 >> v2:172.21.15.190:3300/0 conn(0x7f323406db80 0x7f32340cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3233fff700 1 --2- 172.21.15.181:0/3397025341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323415dfd0 0x7f32340af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.262+0000 7f3233fff700 1 -- 172.21.15.181:0/3397025341 >> v2:172.21.15.190:3300/0 conn(0x7f323406db80 msgr2=0x7f32340cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:56.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f3233fff700 1 --2- 172.21.15.181:0/3397025341 >> v2:172.21.15.190:3300/0 conn(0x7f323406db80 0x7f32340cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f3233fff700 1 -- 172.21.15.181:0/3397025341 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32340ccbc0 con 0x7f32340af450 2023-10-16T09:59:56.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f3239556700 1 --2- 172.21.15.181:0/3397025341 >> v2:172.21.15.190:3300/0 conn(0x7f323406db80 0x7f32340cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:56.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f3233fff700 1 --2- 172.21.15.181:0/3397025341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32340af450 0x7f323406d640 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f32340cdfb0 tx=0x7f3220009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3274f63defd19efa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:56.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f32337fe700 1 -- 172.21.15.181:0/3397025341 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000b870 con 0x7f32340af450 2023-10-16T09:59:56.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f32337fe700 1 -- 172.21.15.181:0/3397025341 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322000b9d0 con 0x7f32340af450 2023-10-16T09:59:56.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f32337fe700 1 -- 172.21.15.181:0/3397025341 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000bd60 con 0x7f32340af450 2023-10-16T09:59:56.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 -- 172.21.15.181:0/3397025341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32340af450 msgr2=0x7f323406d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 --2- 172.21.15.181:0/3397025341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32340af450 0x7f323406d640 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f32340cdfb0 tx=0x7f3220009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 -- 172.21.15.181:0/3397025341 shutdown_connections 2023-10-16T09:59:56.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 --2- 172.21.15.181:0/3397025341 >> v2:172.21.15.190:3300/0 conn(0x7f323406db80 0x7f32340cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 --2- 172.21.15.181:0/3397025341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f323415dfd0 0x7f32340af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 --2- 172.21.15.181:0/3397025341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32340af450 0x7f323406d640 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 -- 172.21.15.181:0/3397025341 >> 172.21.15.181:0/3397025341 conn(0x7f32340d04a0 msgr2=0x7f32340d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:56.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 -- 172.21.15.181:0/3397025341 shutdown_connections 2023-10-16T09:59:56.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.263+0000 7f323afb9700 1 -- 172.21.15.181:0/3397025341 wait complete. 2023-10-16T09:59:56.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.265+0000 7f323afb9700 1 Processor -- start 2023-10-16T09:59:56.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.265+0000 7f323afb9700 1 -- start start 2023-10-16T09:59:56.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.265+0000 7f323afb9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f323406db80 0x7f3234148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.265+0000 7f323afb9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32340af450 0x7f3234148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.265+0000 7f323afb9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323415dfd0 0x7f32340c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.265+0000 7f323afb9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32340b4b80 con 0x7f323406db80 2023-10-16T09:59:56.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.265+0000 7f323afb9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323414b9d0 con 0x7f323415dfd0 2023-10-16T09:59:56.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f323afb9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323414b6f0 con 0x7f32340af450 2023-10-16T09:59:56.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f323406db80 0x7f3234148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f323406db80 0x7f3234148970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43018/0 (socket says 172.21.15.181:43018) 2023-10-16T09:59:56.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 -- 172.21.15.181:0/3780353 learned_addr learned my addr 172.21.15.181:0/3780353 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:56.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3239556700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323415dfd0 0x7f32340c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3233fff700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32340af450 0x7f3234148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 -- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32340af450 msgr2=0x7f3234148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32340af450 0x7f3234148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 -- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323415dfd0 msgr2=0x7f32340c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323415dfd0 0x7f32340c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 -- 172.21.15.181:0/3780353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220009550 con 0x7f323406db80 2023-10-16T09:59:56.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3233fff700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32340af450 0x7f3234148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:56.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f3238d55700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f323406db80 0x7f3234148970 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f322800d7d0 tx=0x7f322800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:56.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f32317fa700 1 -- 172.21.15.181:0/3780353 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280125f0 con 0x7f323406db80 2023-10-16T09:59:56.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f32317fa700 1 -- 172.21.15.181:0/3780353 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3228012750 con 0x7f323406db80 2023-10-16T09:59:56.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32340b44c0 con 0x7f323406db80 2023-10-16T09:59:56.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f32317fa700 1 -- 172.21.15.181:0/3780353 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228012b00 con 0x7f323406db80 2023-10-16T09:59:56.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.266+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32340c4510 con 0x7f323406db80 2023-10-16T09:59:56.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.267+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3234067d20 con 0x7f323406db80 2023-10-16T09:59:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.268+0000 7f32317fa700 1 -- 172.21.15.181:0/3780353 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3228012c60 con 0x7f323406db80 2023-10-16T09:59:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.268+0000 7f32317fa700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f321806b020 0x7f321806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.268+0000 7f32317fa700 1 -- 172.21.15.181:0/3780353 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f3228089550 con 0x7f323406db80 2023-10-16T09:59:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.268+0000 7f3233fff700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f321806b020 0x7f321806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.268+0000 7f3233fff700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f321806b020 0x7f321806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c00 tx=0x7f3220008030 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:56.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.272+0000 7f32317fa700 1 -- 172.21.15.181:0/3780353 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3228054690 con 0x7f323406db80 2023-10-16T09:59:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.466+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f32340aefa0 con 0x7f323406db80 2023-10-16T09:59:56.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.467+0000 7f32317fa700 1 -- 172.21.15.181:0/3780353 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3228057cb0 con 0x7f323406db80 2023-10-16T09:59:56.468 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T09:59:56.468 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T09:59:56.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T09:59:56.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T09:59:56.469 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T09:59:56.469 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T09:59:56.469 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T09:59:56.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f321806b020 msgr2=0x7f321806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f321806b020 0x7f321806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3220000c00 tx=0x7f3220008030 comp rx=0 tx=0).stop 2023-10-16T09:59:56.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f323406db80 msgr2=0x7f3234148970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f323406db80 0x7f3234148970 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f322800d7d0 tx=0x7f322800dae0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 shutdown_connections 2023-10-16T09:59:56.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f321806b020 0x7f321806d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f323415dfd0 0x7f32340c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f323406db80 0x7f3234148970 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 --2- 172.21.15.181:0/3780353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32340af450 0x7f3234148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 >> 172.21.15.181:0/3780353 conn(0x7f32340d04a0 msgr2=0x7f32340ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:56.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 shutdown_connections 2023-10-16T09:59:56.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.469+0000 7f323afb9700 1 -- 172.21.15.181:0/3780353 wait complete. 2023-10-16T09:59:56.481 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T09:59:56.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T09:59:56.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.606+0000 7fb744cd6700 1 Processor -- start 2023-10-16T09:59:56.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.606+0000 7fb744cd6700 1 -- start start 2023-10-16T09:59:56.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb744cd6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7400cf5c0 0x7fb7400caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb744cd6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb740160240 con 0x7fb74006a8a0 2023-10-16T09:59:56.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb744cd6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74015ff60 con 0x7fb7400cc4a0 2023-10-16T09:59:56.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb744cd6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74015bd80 con 0x7fb7400cf5c0 2023-10-16T09:59:56.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7400cf5c0 0x7fb7400caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73e59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb74006a8a0 0x7fb74006ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54104/0 (socket says 172.21.15.181:54104) 2023-10-16T09:59:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73e59c700 1 -- 172.21.15.181:0/369764582 learned_addr learned my addr 172.21.15.181:0/369764582 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73d59a700 1 -- 172.21.15.181:0/369764582 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241642634 0 0) 0x7fb740160240 con 0x7fb74006a8a0 2023-10-16T09:59:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73d59a700 1 -- 172.21.15.181:0/369764582 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7280035b0 con 0x7fb74006a8a0 2023-10-16T09:59:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73dd9b700 1 -- 172.21.15.181:0/369764582 >> v1:172.21.15.190:6789/0 conn(0x7fb7400cc4a0 legacy=0x7fb74015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73dd9b700 1 -- 172.21.15.181:0/369764582 >> v1:172.21.15.181:6789/0 conn(0x7fb74006a8a0 legacy=0x7fb74006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73dd9b700 1 -- 172.21.15.181:0/369764582 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400cfc40 con 0x7fb7400cf5c0 2023-10-16T09:59:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73dd9b700 1 --2- 172.21.15.181:0/369764582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cf5c0 0x7fb7400caaf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb734009840 tx=0x7fb734009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526f6c25d88e1024 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73d59a700 1 -- 172.21.15.181:0/369764582 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734011830 con 0x7fb7400cf5c0 2023-10-16T09:59:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73d59a700 1 -- 172.21.15.181:0/369764582 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734011990 con 0x7fb7400cf5c0 2023-10-16T09:59:56.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.607+0000 7fb73d59a700 1 -- 172.21.15.181:0/369764582 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734011d20 con 0x7fb7400cf5c0 2023-10-16T09:59:56.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.608+0000 7fb744cd6700 1 -- 172.21.15.181:0/369764582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cf5c0 msgr2=0x7fb7400caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.608+0000 7fb744cd6700 1 --2- 172.21.15.181:0/369764582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cf5c0 0x7fb7400caaf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb734009840 tx=0x7fb734009b50 comp rx=0 tx=0).stop 2023-10-16T09:59:56.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.608+0000 7fb744cd6700 1 -- 172.21.15.181:0/369764582 shutdown_connections 2023-10-16T09:59:56.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.608+0000 7fb744cd6700 1 --2- 172.21.15.181:0/369764582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cf5c0 0x7fb7400caaf0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.608+0000 7fb744cd6700 1 -- 172.21.15.181:0/369764582 >> 172.21.15.181:0/369764582 conn(0x7fb7400d04a0 msgr2=0x7fb7400d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.608+0000 7fb744cd6700 1 -- 172.21.15.181:0/369764582 shutdown_connections 2023-10-16T09:59:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.608+0000 7fb744cd6700 1 -- 172.21.15.181:0/369764582 wait complete. 2023-10-16T09:59:56.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.609+0000 7fb744cd6700 1 Processor -- start 2023-10-16T09:59:56.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.609+0000 7fb744cd6700 1 -- start start 2023-10-16T09:59:56.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb744cd6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb74006a8a0 0x7fb7400b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb744cd6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cc4a0 0x7fb7400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb744cd6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400cf5c0 0x7fb7401f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb744cd6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b52d0 con 0x7fb7400cc4a0 2023-10-16T09:59:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb744cd6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74006d660 con 0x7fb74006a8a0 2023-10-16T09:59:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb744cd6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb740148160 con 0x7fb7400cf5c0 2023-10-16T09:59:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cc4a0 0x7fb7400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cc4a0 0x7fb7400b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43034/0 (socket says 172.21.15.181:43034) 2023-10-16T09:59:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 -- 172.21.15.181:0/829496037 learned_addr learned my addr 172.21.15.181:0/829496037 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73e59c700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb74006a8a0 0x7fb7400b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73ed9d700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400cf5c0 0x7fb7401f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 -- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400cf5c0 msgr2=0x7fb7401f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400cf5c0 0x7fb7401f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 -- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb74006a8a0 msgr2=0x7fb7400b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:56.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb74006a8a0 0x7fb7400b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:56.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.610+0000 7fb73dd9b700 1 -- 172.21.15.181:0/829496037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340094a0 con 0x7fb7400cc4a0 2023-10-16T09:59:56.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.611+0000 7fb73dd9b700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cc4a0 0x7fb7400b4d90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb734009b90 tx=0x7fb734004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:56.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.611+0000 7fb72effd700 1 -- 172.21.15.181:0/829496037 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340038a0 con 0x7fb7400cc4a0 2023-10-16T09:59:56.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.611+0000 7fb72effd700 1 -- 172.21.15.181:0/829496037 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73401e070 con 0x7fb7400cc4a0 2023-10-16T09:59:56.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.611+0000 7fb72effd700 1 -- 172.21.15.181:0/829496037 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734003c40 con 0x7fb7400cc4a0 2023-10-16T09:59:56.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.611+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74014a7b0 con 0x7fb7400cc4a0 2023-10-16T09:59:56.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.611+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7400b57a0 con 0x7fb7400cc4a0 2023-10-16T09:59:56.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.612+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7400cb730 con 0x7fb7400cc4a0 2023-10-16T09:59:56.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.612+0000 7fb72effd700 1 -- 172.21.15.181:0/829496037 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb73402d450 con 0x7fb7400cc4a0 2023-10-16T09:59:56.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.613+0000 7fb72effd700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb72006b020 0x7fb72006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:56.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.613+0000 7fb73e59c700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb72006b020 0x7fb72006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:56.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.613+0000 7fb72effd700 1 -- 172.21.15.181:0/829496037 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5281+0+0 (secure 0 0 0) 0x7fb73408a550 con 0x7fb7400cc4a0 2023-10-16T09:59:56.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.613+0000 7fb73e59c700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb72006b020 0x7fb72006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb7300010e0 tx=0x7fb730009040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:56.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.617+0000 7fb72effd700 1 -- 172.21.15.181:0/829496037 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb734055720 con 0x7fb7400cc4a0 2023-10-16T09:59:56.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:56.810+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb74006d7d0 con 0x7fb7400cc4a0 2023-10-16T09:59:57.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.628+0000 7fb72effd700 1 -- 172.21.15.181:0/829496037 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb74006d7d0 con 0x7fb7400cc4a0 2023-10-16T09:59:57.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb72006b020 msgr2=0x7fb72006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:57.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb72006b020 0x7fb72006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb7300010e0 tx=0x7fb730009040 comp rx=0 tx=0).stop 2023-10-16T09:59:57.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cc4a0 msgr2=0x7fb7400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:57.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cc4a0 0x7fb7400b4d90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb734009b90 tx=0x7fb734004750 comp rx=0 tx=0).stop 2023-10-16T09:59:57.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 shutdown_connections 2023-10-16T09:59:57.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb72006b020 0x7fb72006d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7400cf5c0 0x7fb7401f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb74006a8a0 0x7fb7400b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 --2- 172.21.15.181:0/829496037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cc4a0 0x7fb7400b4d90 secure :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb734009b90 tx=0x7fb734004750 comp rx=0 tx=0).stop 2023-10-16T09:59:57.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 >> 172.21.15.181:0/829496037 conn(0x7fb7400d04a0 msgr2=0x7fb7400caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:57.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 shutdown_connections 2023-10-16T09:59:57.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.632+0000 7fb744cd6700 1 -- 172.21.15.181:0/829496037 wait complete. 2023-10-16T09:59:57.635 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T09:59:57.651 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T09:59:57.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.786+0000 7ff42678e700 1 Processor -- start 2023-10-16T09:59:57.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.786+0000 7ff42678e700 1 -- start start 2023-10-16T09:59:57.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff42678e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4200b2010 0x7ff4200cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:57.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff42678e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff420071be0 0x7ff4200b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:57.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff42678e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff42006f3b0 0x7ff42006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:57.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff42678e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200ce1f0 con 0x7ff4200b2010 2023-10-16T09:59:57.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff42678e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200b2930 con 0x7ff42006f3b0 2023-10-16T09:59:57.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff42678e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200ccda0 con 0x7ff420071be0 2023-10-16T09:59:57.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4200b2010 0x7ff4200cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:57.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4200b2010 0x7ff4200cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43040/0 (socket says 172.21.15.181:43040) 2023-10-16T09:59:57.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41ffff700 1 -- 172.21.15.181:0/2455814342 learned_addr learned my addr 172.21.15.181:0/2455814342 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:57.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41f7fe700 1 --2- 172.21.15.181:0/2455814342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff420071be0 0x7ff4200b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:57.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff424d2b700 1 --2- 172.21.15.181:0/2455814342 >> v2:172.21.15.190:3300/0 conn(0x7ff42006f3b0 0x7ff42006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:57.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41ffff700 1 -- 172.21.15.181:0/2455814342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff420071be0 msgr2=0x7ff4200b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:57.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41ffff700 1 --2- 172.21.15.181:0/2455814342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff420071be0 0x7ff4200b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41ffff700 1 -- 172.21.15.181:0/2455814342 >> v2:172.21.15.190:3300/0 conn(0x7ff42006f3b0 msgr2=0x7ff42006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:57.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.787+0000 7ff41ffff700 1 --2- 172.21.15.181:0/2455814342 >> v2:172.21.15.190:3300/0 conn(0x7ff42006f3b0 0x7ff42006f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff41ffff700 1 -- 172.21.15.181:0/2455814342 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200727e0 con 0x7ff4200b2010 2023-10-16T09:59:57.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff41f7fe700 1 --2- 172.21.15.181:0/2455814342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff420071be0 0x7ff4200b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:57.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff424d2b700 1 --2- 172.21.15.181:0/2455814342 >> v2:172.21.15.190:3300/0 conn(0x7ff42006f3b0 0x7ff42006f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T09:59:57.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff41ffff700 1 --2- 172.21.15.181:0/2455814342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4200b2010 0x7ff4200cc6e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff420070750 tx=0x7ff414009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d644d7d32ac991 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:57.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff41effd700 1 -- 172.21.15.181:0/2455814342 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41400b870 con 0x7ff4200b2010 2023-10-16T09:59:57.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff41effd700 1 -- 172.21.15.181:0/2455814342 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41400b9d0 con 0x7ff4200b2010 2023-10-16T09:59:57.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff41effd700 1 -- 172.21.15.181:0/2455814342 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41400bd60 con 0x7ff4200b2010 2023-10-16T09:59:57.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 -- 172.21.15.181:0/2455814342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4200b2010 msgr2=0x7ff4200cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:57.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 --2- 172.21.15.181:0/2455814342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4200b2010 0x7ff4200cc6e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff420070750 tx=0x7ff414009bd0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 -- 172.21.15.181:0/2455814342 shutdown_connections 2023-10-16T09:59:57.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 --2- 172.21.15.181:0/2455814342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4200b2010 0x7ff4200cc6e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 --2- 172.21.15.181:0/2455814342 >> v2:172.21.15.190:3300/0 conn(0x7ff42006f3b0 0x7ff42006f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 --2- 172.21.15.181:0/2455814342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff420071be0 0x7ff4200b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 -- 172.21.15.181:0/2455814342 >> 172.21.15.181:0/2455814342 conn(0x7ff4200cd010 msgr2=0x7ff4200cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:57.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 -- 172.21.15.181:0/2455814342 shutdown_connections 2023-10-16T09:59:57.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.788+0000 7ff42678e700 1 -- 172.21.15.181:0/2455814342 wait complete. 2023-10-16T09:59:57.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 Processor -- start 2023-10-16T09:59:57.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 -- start start 2023-10-16T09:59:57.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff42006f3b0 0x7ff420157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:57.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff420071be0 0x7ff4201f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:57.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4200b2010 0x7ff42015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:57.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420157960 con 0x7ff420071be0 2023-10-16T09:59:57.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200b0510 con 0x7ff4200b2010 2023-10-16T09:59:57.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.790+0000 7ff42678e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201f3fd0 con 0x7ff42006f3b0 2023-10-16T09:59:57.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff420071be0 0x7ff4201f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:57.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff420071be0 0x7ff4201f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43048/0 (socket says 172.21.15.181:43048) 2023-10-16T09:59:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 -- 172.21.15.181:0/3922239252 learned_addr learned my addr 172.21.15.181:0/3922239252 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff424d2b700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4200b2010 0x7ff42015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41ffff700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff42006f3b0 0x7ff420157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 -- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff42006f3b0 msgr2=0x7ff420157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff42006f3b0 0x7ff420157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 -- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4200b2010 msgr2=0x7ff42015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4200b2010 0x7ff42015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 -- 172.21.15.181:0/3922239252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414009550 con 0x7ff420071be0 2023-10-16T09:59:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff424d2b700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4200b2010 0x7ff42015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41ffff700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff42006f3b0 0x7ff420157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41f7fe700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff420071be0 0x7ff4201f3a90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c7f0 tx=0x7ff41000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41cff9700 1 -- 172.21.15.181:0/3922239252 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410011800 con 0x7ff420071be0 2023-10-16T09:59:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41cff9700 1 -- 172.21.15.181:0/3922239252 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410011960 con 0x7ff420071be0 2023-10-16T09:59:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4200c7290 con 0x7ff420071be0 2023-10-16T09:59:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff41cff9700 1 -- 172.21.15.181:0/3922239252 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410011d10 con 0x7ff420071be0 2023-10-16T09:59:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.791+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4200afec0 con 0x7ff420071be0 2023-10-16T09:59:57.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.792+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4200d01a0 con 0x7ff420071be0 2023-10-16T09:59:57.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.793+0000 7ff41cff9700 1 -- 172.21.15.181:0/3922239252 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff410024480 con 0x7ff420071be0 2023-10-16T09:59:57.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.793+0000 7ff41cff9700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7ff40c06b020 0x7ff40c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:57.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.793+0000 7ff41cff9700 1 -- 172.21.15.181:0/3922239252 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5651+0+0 (secure 0 0 0) 0x7ff410089c00 con 0x7ff420071be0 2023-10-16T09:59:57.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.793+0000 7ff41ffff700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7ff40c06b020 0x7ff40c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:57.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.794+0000 7ff41ffff700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7ff40c06b020 0x7ff40c06d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff414000c00 tx=0x7ff414008030 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:57.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.797+0000 7ff41cff9700 1 -- 172.21.15.181:0/3922239252 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff410054ba0 con 0x7ff420071be0 2023-10-16T09:59:57.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:57.989+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff420072d70 con 0x7ff420071be0 2023-10-16T09:59:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.635+0000 7ff41cff9700 1 -- 172.21.15.181:0/3922239252 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff41001f0d0 con 0x7ff420071be0 2023-10-16T09:59:58.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7ff40c06b020 msgr2=0x7ff40c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:58.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7ff40c06b020 0x7ff40c06d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff414000c00 tx=0x7ff414008030 comp rx=0 tx=0).stop 2023-10-16T09:59:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff420071be0 msgr2=0x7ff4201f3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff420071be0 0x7ff4201f3a90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c7f0 tx=0x7ff41000cbb0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 shutdown_connections 2023-10-16T09:59:58.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7ff40c06b020 0x7ff40c06d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff42006f3b0 0x7ff420157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4200b2010 0x7ff42015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 --2- 172.21.15.181:0/3922239252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff420071be0 0x7ff4201f3a90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.638+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 >> 172.21.15.181:0/3922239252 conn(0x7ff4200cd010 msgr2=0x7ff4200ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.639+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 shutdown_connections 2023-10-16T09:59:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.639+0000 7ff42678e700 1 -- 172.21.15.181:0/3922239252 wait complete. 2023-10-16T09:59:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T09:59:58.667 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T09:59:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.794+0000 7fb92d263700 1 Processor -- start 2023-10-16T09:59:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.794+0000 7fb92d263700 1 -- start start 2023-10-16T09:59:58.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb92d263700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb9280d01d0 0x7fb9280ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb92d263700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280cf580 con 0x7fb9280d1500 2023-10-16T09:59:58.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb92d263700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280ce970 con 0x7fb92815ea40 2023-10-16T09:59:58.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb92d263700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280d4290 con 0x7fb9280d01d0 2023-10-16T09:59:58.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb9277fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb9280d1500 0x7fb9280d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54134/0 (socket says 172.21.15.181:54134) 2023-10-16T09:59:58.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb9277fe700 1 -- 172.21.15.181:0/3236574995 learned_addr learned my addr 172.21.15.181:0/3236574995 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T09:59:58.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3236574995 >> v2:172.21.15.190:3300/0 conn(0x7fb9280d01d0 0x7fb9280ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572072307 0 0) 0x7fb9280cf580 con 0x7fb9280d1500 2023-10-16T09:59:58.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9100035b0 con 0x7fb9280d1500 2023-10-16T09:59:58.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430918221 0 0) 0x7fb9280ce970 con 0x7fb92815ea40 2023-10-16T09:59:58.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9280cf580 con 0x7fb92815ea40 2023-10-16T09:59:58.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524838428 0 0) 0x7fb9100035b0 con 0x7fb9280d1500 2023-10-16T09:59:58.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 >> v2:172.21.15.190:3300/0 conn(0x7fb9280d01d0 msgr2=0x7fb9280ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 --2- 172.21.15.181:0/3236574995 >> v2:172.21.15.190:3300/0 conn(0x7fb9280d01d0 0x7fb9280ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 >> v1:172.21.15.190:6789/0 conn(0x7fb92815ea40 legacy=0x7fb9280d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280cec50 con 0x7fb9280d1500 2023-10-16T09:59:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fb9200019f0 con 0x7fb9280d1500 2023-10-16T09:59:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 >> v1:172.21.15.181:6789/0 conn(0x7fb9280d1500 legacy=0x7fb9280d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:58.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910003be0 0x7fb910003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.795+0000 7fb925ffb700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910004850 0x7fb910004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb925ffb700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb910005210 0x7fb91000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91000c0e0 con 0x7fb910003be0 2023-10-16T09:59:58.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91000c220 con 0x7fb910005210 2023-10-16T09:59:58.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91000c3e0 con 0x7fb910004850 2023-10-16T09:59:58.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910003be0 0x7fb910003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb9277fe700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb910005210 0x7fb91000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb9267fc700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910004850 0x7fb910004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb926ffd700 1 -- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910004850 msgr2=0x7fb910004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910004850 0x7fb910004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb926ffd700 1 -- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb910005210 msgr2=0x7fb91000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T09:59:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb910005210 0x7fb91000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb926ffd700 1 -- 172.21.15.181:0/3236574995 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100035b0 con 0x7fb910003be0 2023-10-16T09:59:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb9277fe700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb910005210 0x7fb91000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:58.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb9267fc700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910004850 0x7fb910004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T09:59:58.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb9277fe700 1 -- 172.21.15.181:0/3236574995 reap_dead start 2023-10-16T09:59:58.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910003be0 0x7fb910003fe0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009b60 tx=0x7fb910005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927706143e86a135 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:58.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00e6c0 con 0x7fb910003be0 2023-10-16T09:59:58.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c00e820 con 0x7fb910003be0 2023-10-16T09:59:58.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.796+0000 7fb925ffb700 1 -- 172.21.15.181:0/3236574995 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00eb90 con 0x7fb910003be0 2023-10-16T09:59:58.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.797+0000 7fb92d263700 1 -- 172.21.15.181:0/3236574995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910003be0 msgr2=0x7fb910003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:58.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.797+0000 7fb92d263700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910003be0 0x7fb910003fe0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009b60 tx=0x7fb910005880 comp rx=0 tx=0).stop 2023-10-16T09:59:58.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.797+0000 7fb92d263700 1 -- 172.21.15.181:0/3236574995 shutdown_connections 2023-10-16T09:59:58.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.797+0000 7fb92d263700 1 --2- 172.21.15.181:0/3236574995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb910003be0 0x7fb910003fe0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.797+0000 7fb92d263700 1 -- 172.21.15.181:0/3236574995 >> 172.21.15.181:0/3236574995 conn(0x7fb92809cb60 msgr2=0x7fb9280d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:58.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.797+0000 7fb92d263700 1 -- 172.21.15.181:0/3236574995 shutdown_connections 2023-10-16T09:59:58.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.797+0000 7fb92d263700 1 -- 172.21.15.181:0/3236574995 wait complete. 2023-10-16T09:59:58.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.798+0000 7fb92d263700 1 Processor -- start 2023-10-16T09:59:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.798+0000 7fb92d263700 1 -- start start 2023-10-16T09:59:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb92d263700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910003be0 0x7fb9280ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb92d263700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9280ca890 0x7fb9281f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb92d263700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9281f3970 0x7fb9281f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb92d263700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9281f4340 con 0x7fb9281f3970 2023-10-16T09:59:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb92d263700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280cd450 con 0x7fb9280ca890 2023-10-16T09:59:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb92d263700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280cb070 con 0x7fb910003be0 2023-10-16T09:59:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9281f3970 0x7fb9281f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9281f3970 0x7fb9281f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43068/0 (socket says 172.21.15.181:43068) 2023-10-16T09:59:58.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 -- 172.21.15.181:0/3947287958 learned_addr learned my addr 172.21.15.181:0/3947287958 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T09:59:58.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910003be0 0x7fb9280ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9267fc700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9280ca890 0x7fb9281f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 -- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910003be0 msgr2=0x7fb9280ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910003be0 0x7fb9280ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 -- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9280ca890 msgr2=0x7fb9281f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9280ca890 0x7fb9281f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:58.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 -- 172.21.15.181:0/3947287958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280cb350 con 0x7fb9281f3970 2023-10-16T09:59:58.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb9277fe700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9281f3970 0x7fb9281f3e00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb92808c620 tx=0x7fb92000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:58.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb90f7fe700 1 -- 172.21.15.181:0/3947287958 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92000eb00 con 0x7fb9281f3970 2023-10-16T09:59:58.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb90f7fe700 1 -- 172.21.15.181:0/3947287958 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92000ec60 con 0x7fb9281f3970 2023-10-16T09:59:58.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb90f7fe700 1 -- 172.21.15.181:0/3947287958 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920013420 con 0x7fb9281f3970 2023-10-16T09:59:58.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.799+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280cb630 con 0x7fb9281f3970 2023-10-16T09:59:58.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.800+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9280cacf0 con 0x7fb9281f3970 2023-10-16T09:59:58.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.800+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9280c7380 con 0x7fb9281f3970 2023-10-16T09:59:58.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.801+0000 7fb90f7fe700 1 -- 172.21.15.181:0/3947287958 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb920013580 con 0x7fb9281f3970 2023-10-16T09:59:58.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.801+0000 7fb90f7fe700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb91406b020 0x7fb91406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T09:59:58.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.801+0000 7fb90f7fe700 1 -- 172.21.15.181:0/3947287958 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6033+0+0 (secure 0 0 0) 0x7fb920089c50 con 0x7fb9281f3970 2023-10-16T09:59:58.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.802+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb91406b020 0x7fb91406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T09:59:58.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.802+0000 7fb926ffd700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb91406b020 0x7fb91406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0090b0 tx=0x7fb91c01f040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T09:59:58.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:58.806+0000 7fb90f7fe700 1 -- 172.21.15.181:0/3947287958 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb9280c7380 con 0x7fb9281f3970 2023-10-16T09:59:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.042+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb9280c7380 con 0x7fb9281f3970 2023-10-16T09:59:59.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.966+0000 7fb90f7fe700 1 -- 172.21.15.181:0/3947287958 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb9280c7380 con 0x7fb9281f3970 2023-10-16T09:59:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.970+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb91406b020 msgr2=0x7fb91406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.970+0000 7fb92d263700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb91406b020 0x7fb91406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0090b0 tx=0x7fb91c01f040 comp rx=0 tx=0).stop 2023-10-16T09:59:59.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.970+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9281f3970 msgr2=0x7fb9281f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T09:59:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.970+0000 7fb92d263700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9281f3970 0x7fb9281f3e00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb92808c620 tx=0x7fb92000e440 comp rx=0 tx=0).stop 2023-10-16T09:59:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.971+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 shutdown_connections 2023-10-16T09:59:59.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.971+0000 7fb92d263700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fb91406b020 0x7fb91406d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:59.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.971+0000 7fb92d263700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb910003be0 0x7fb9280ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:59.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.971+0000 7fb92d263700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9281f3970 0x7fb9281f3e00 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:59.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.971+0000 7fb92d263700 1 --2- 172.21.15.181:0/3947287958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9280ca890 0x7fb9281f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T09:59:59.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.971+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 >> 172.21.15.181:0/3947287958 conn(0x7fb92809cb60 msgr2=0x7fb928160190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T09:59:59.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.971+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 shutdown_connections 2023-10-16T09:59:59.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T09:59:59.972+0000 7fb92d263700 1 -- 172.21.15.181:0/3947287958 wait complete. 2023-10-16T09:59:59.978 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 10 and data pool 11 2023-10-16T09:59:59.991 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:00.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.118+0000 7fcf521ad700 1 Processor -- start 2023-10-16T10:00:00.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.118+0000 7fcf521ad700 1 -- start start 2023-10-16T10:00:00.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.118+0000 7fcf521ad700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf4c15b7a0 0x7fcf4c15fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.118+0000 7fcf521ad700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf4c070bb0 0x7fcf4c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.118+0000 7fcf521ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4c0cfd20 con 0x7fcf4c0cf8d0 2023-10-16T10:00:00.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.118+0000 7fcf521ad700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4c160300 con 0x7fcf4c070bb0 2023-10-16T10:00:00.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.118+0000 7fcf521ad700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4c071030 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf4c070bb0 0x7fcf4c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcf4c0cf8d0 0x7fcf4c0cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54166/0 (socket says 172.21.15.181:54166) 2023-10-16T10:00:00.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4bfff700 1 -- 172.21.15.181:0/25967210 learned_addr learned my addr 172.21.15.181:0/25967210 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:00.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4b7fe700 1 --2- 172.21.15.181:0/25967210 >> v2:172.21.15.190:3300/0 conn(0x7fcf4c15b7a0 0x7fcf4c15fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4affd700 1 -- 172.21.15.181:0/25967210 >> v2:172.21.15.190:3300/0 conn(0x7fcf4c15b7a0 msgr2=0x7fcf4c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:00.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4affd700 1 --2- 172.21.15.181:0/25967210 >> v2:172.21.15.190:3300/0 conn(0x7fcf4c15b7a0 0x7fcf4c15fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4affd700 1 -- 172.21.15.181:0/25967210 >> v1:172.21.15.181:6789/0 conn(0x7fcf4c0cf8d0 legacy=0x7fcf4c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4a7fc700 1 -- 172.21.15.181:0/25967210 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129727089 0 0) 0x7fcf44001490 con 0x7fcf4c0cf8d0 2023-10-16T10:00:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4affd700 1 -- 172.21.15.181:0/25967210 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c15bb80 con 0x7fcf4c070bb0 2023-10-16T10:00:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4affd700 1 --2- 172.21.15.181:0/25967210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c0cf390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c06ed30 tx=0x7fcf3c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646edb77425b04a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4a7fc700 1 -- 172.21.15.181:0/25967210 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c011830 con 0x7fcf4c070bb0 2023-10-16T10:00:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4a7fc700 1 -- 172.21.15.181:0/25967210 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c011990 con 0x7fcf4c070bb0 2023-10-16T10:00:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.119+0000 7fcf4a7fc700 1 -- 172.21.15.181:0/25967210 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c011d20 con 0x7fcf4c070bb0 2023-10-16T10:00:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 -- 172.21.15.181:0/25967210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c070bb0 msgr2=0x7fcf4c0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 --2- 172.21.15.181:0/25967210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c0cf390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c06ed30 tx=0x7fcf3c009b20 comp rx=0 tx=0).stop 2023-10-16T10:00:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 -- 172.21.15.181:0/25967210 shutdown_connections 2023-10-16T10:00:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 --2- 172.21.15.181:0/25967210 >> v2:172.21.15.190:3300/0 conn(0x7fcf4c15b7a0 0x7fcf4c15fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 --2- 172.21.15.181:0/25967210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c0cf390 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 -- 172.21.15.181:0/25967210 >> 172.21.15.181:0/25967210 conn(0x7fcf4c06aed0 msgr2=0x7fcf4c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 -- 172.21.15.181:0/25967210 shutdown_connections 2023-10-16T10:00:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.120+0000 7fcf521ad700 1 -- 172.21.15.181:0/25967210 wait complete. 2023-10-16T10:00:00.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.121+0000 7fcf521ad700 1 Processor -- start 2023-10-16T10:00:00.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf521ad700 1 -- start start 2023-10-16T10:00:00.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf521ad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c15de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf521ad700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf4c0cf8d0 0x7fcf4c15cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf521ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c15b7a0 0x7fcf4c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf521ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c1f12a0 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf521ad700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c15e6a0 con 0x7fcf4c070bb0 2023-10-16T10:00:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf521ad700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c1f0c50 con 0x7fcf4c0cf8d0 2023-10-16T10:00:00.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c15b7a0 0x7fcf4c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf4bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c15b7a0 0x7fcf4c0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43094/0 (socket says 172.21.15.181:43094) 2023-10-16T10:00:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.122+0000 7fcf4bfff700 1 -- 172.21.15.181:0/2581778753 learned_addr learned my addr 172.21.15.181:0/2581778753 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4affd700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf4c0cf8d0 0x7fcf4c15cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4b7fe700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4bfff700 1 -- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf4c0cf8d0 msgr2=0x7fcf4c15cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4bfff700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf4c0cf8d0 0x7fcf4c15cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4bfff700 1 -- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf4c070bb0 msgr2=0x7fcf4c15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4bfff700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c15de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4bfff700 1 -- 172.21.15.181:0/2581778753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0094a0 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4b7fe700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c15de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf4bfff700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c15b7a0 0x7fcf4c0b3570 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400cca0 tx=0x7fcf4400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf33fff700 1 -- 172.21.15.181:0/2581778753 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4400cea0 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf33fff700 1 -- 172.21.15.181:0/2581778753 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4400ada0 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c1f0f30 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf33fff700 1 -- 172.21.15.181:0/2581778753 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44007310 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.123+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c15e390 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.124+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4c1f1410 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.124+0000 7fcf33fff700 1 -- 172.21.15.181:0/2581778753 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf44007470 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.125+0000 7fcf33fff700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf3806b050 0x7fcf3806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.125+0000 7fcf33fff700 1 -- 172.21.15.181:0/2581778753 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fcf44089210 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.125+0000 7fcf4b7fe700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf3806b050 0x7fcf3806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.125+0000 7fcf4b7fe700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf3806b050 0x7fcf3806d510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009820 tx=0x7fcf40008040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.129+0000 7fcf33fff700 1 -- 172.21.15.181:0/2581778753 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf44054090 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.324+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf4c0cfd20 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.325+0000 7fcf33fff700 1 -- 172.21.15.181:0/2581778753 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+17261 (secure 0 0 0) 0x7fcf4c0cfd20 con 0x7fcf4c15b7a0 2023-10-16T10:00:00.326 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:00.326 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":89,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T09:59:59.043273+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:56.829733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:58.008691+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf3806b050 msgr2=0x7fcf3806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf3806b050 0x7fcf3806d510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009820 tx=0x7fcf40008040 comp rx=0 tx=0).stop 2023-10-16T10:00:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c15b7a0 msgr2=0x7fcf4c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c15b7a0 0x7fcf4c0b3570 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400cca0 tx=0x7fcf4400a340 comp rx=0 tx=0).stop 2023-10-16T10:00:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 shutdown_connections 2023-10-16T10:00:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fcf3806b050 0x7fcf3806d510 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf4c15b7a0 0x7fcf4c0b3570 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf4c070bb0 0x7fcf4c15de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 --2- 172.21.15.181:0/2581778753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf4c0cf8d0 0x7fcf4c15cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 >> 172.21.15.181:0/2581778753 conn(0x7fcf4c06aed0 msgr2=0x7fcf4c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 shutdown_connections 2023-10-16T10:00:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.327+0000 7fcf521ad700 1 -- 172.21.15.181:0/2581778753 wait complete. 2023-10-16T10:00:00.341 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:00.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.460+0000 7f8435612700 1 Processor -- start 2023-10-16T10:00:00.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.460+0000 7f8435612700 1 -- start start 2023-10-16T10:00:00.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f8435612700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8430160d20 0x7f84301611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f8435612700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843006ce20 con 0x7f8430161b60 2023-10-16T10:00:00.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f8435612700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84300d45f0 con 0x7f843006d5f0 2023-10-16T10:00:00.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f8435612700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8430162800 con 0x7f8430160d20 2023-10-16T10:00:00.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f842f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8430160d20 0x7f84301611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f842effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8430161b60 0x7f8430161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54176/0 (socket says 172.21.15.181:54176) 2023-10-16T10:00:00.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f842effd700 1 -- 172.21.15.181:0/3914291476 learned_addr learned my addr 172.21.15.181:0/3914291476 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:00.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f842f7fe700 1 -- 172.21.15.181:0/3914291476 >> v1:172.21.15.190:6789/0 conn(0x7f843006d5f0 legacy=0x7f84300d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:00.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.461+0000 7f842f7fe700 1 -- 172.21.15.181:0/3914291476 >> v1:172.21.15.181:6789/0 conn(0x7f8430161b60 legacy=0x7f8430161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f842dffb700 1 -- 172.21.15.181:0/3914291476 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987889848 0 0) 0x7f843006ce20 con 0x7f8430161b60 2023-10-16T10:00:00.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f842f7fe700 1 -- 172.21.15.181:0/3914291476 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843006d2d0 con 0x7f8430160d20 2023-10-16T10:00:00.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f842f7fe700 1 --2- 172.21.15.181:0/3914291476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301611d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8428009d90 tx=0x7f842800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c37ddcebea9769f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f842dffb700 1 -- 172.21.15.181:0/3914291476 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842800e070 con 0x7f8430160d20 2023-10-16T10:00:00.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f842dffb700 1 -- 172.21.15.181:0/3914291476 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842800bb90 con 0x7f8430160d20 2023-10-16T10:00:00.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f842dffb700 1 -- 172.21.15.181:0/3914291476 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428003a20 con 0x7f8430160d20 2023-10-16T10:00:00.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f8435612700 1 -- 172.21.15.181:0/3914291476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 msgr2=0x7f84301611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f8435612700 1 --2- 172.21.15.181:0/3914291476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301611d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8428009d90 tx=0x7f842800b330 comp rx=0 tx=0).stop 2023-10-16T10:00:00.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f8435612700 1 -- 172.21.15.181:0/3914291476 shutdown_connections 2023-10-16T10:00:00.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f8435612700 1 --2- 172.21.15.181:0/3914291476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301611d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f8435612700 1 -- 172.21.15.181:0/3914291476 >> 172.21.15.181:0/3914291476 conn(0x7f843009cba0 msgr2=0x7f84300d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:00.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f8435612700 1 -- 172.21.15.181:0/3914291476 shutdown_connections 2023-10-16T10:00:00.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.462+0000 7f8435612700 1 -- 172.21.15.181:0/3914291476 wait complete. 2023-10-16T10:00:00.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 Processor -- start 2023-10-16T10:00:00.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 -- start start 2023-10-16T10:00:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843006d5f0 0x7f84301f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8430161b60 0x7f843015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84301f46d0 con 0x7f8430160d20 2023-10-16T10:00:00.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300cac70 con 0x7f843006d5f0 2023-10-16T10:00:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.464+0000 7f8435612700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84300caf50 con 0x7f8430161b60 2023-10-16T10:00:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43106/0 (socket says 172.21.15.181:43106) 2023-10-16T10:00:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 -- 172.21.15.181:0/1334524089 learned_addr learned my addr 172.21.15.181:0/1334524089 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842f7fe700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8430161b60 0x7f843015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842effd700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843006d5f0 0x7f84301f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 -- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8430161b60 msgr2=0x7f843015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8430161b60 0x7f843015fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 -- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843006d5f0 msgr2=0x7f84301f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843006d5f0 0x7f84301f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 -- 172.21.15.181:0/1334524089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428009a70 con 0x7f8430160d20 2023-10-16T10:00:00.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842effd700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843006d5f0 0x7f84301f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:00.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f842e7fc700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301f48d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f842000c4d0 tx=0x7f842000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f84177fe700 1 -- 172.21.15.181:0/1334524089 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420011790 con 0x7f8430160d20 2023-10-16T10:00:00.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f84177fe700 1 -- 172.21.15.181:0/1334524089 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84200118f0 con 0x7f8430160d20 2023-10-16T10:00:00.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84300cb230 con 0x7f8430160d20 2023-10-16T10:00:00.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f84177fe700 1 -- 172.21.15.181:0/1334524089 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420011ca0 con 0x7f8430160d20 2023-10-16T10:00:00.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.465+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84300cb7f0 con 0x7f8430160d20 2023-10-16T10:00:00.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.466+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843004ebb0 con 0x7f8430160d20 2023-10-16T10:00:00.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.467+0000 7f84177fe700 1 -- 172.21.15.181:0/1334524089 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f842001c460 con 0x7f8430160d20 2023-10-16T10:00:00.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.467+0000 7f84177fe700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f841c06b020 0x7f841c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.467+0000 7f84177fe700 1 -- 172.21.15.181:0/1334524089 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f84200896e0 con 0x7f8430160d20 2023-10-16T10:00:00.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.467+0000 7f842effd700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f841c06b020 0x7f841c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.467+0000 7f842effd700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f841c06b020 0x7f841c06d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84300cfc60 tx=0x7f8424009040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.471+0000 7f84177fe700 1 -- 172.21.15.181:0/1334524089 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f84200544d0 con 0x7f8430160d20 2023-10-16T10:00:00.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.669+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84300b5850 con 0x7f8430160d20 2023-10-16T10:00:00.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.670+0000 7f84177fe700 1 -- 172.21.15.181:0/1334524089 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17248 (secure 0 0 0) 0x7f8420057af0 con 0x7f8430160d20 2023-10-16T10:00:00.671 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:00.671 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":90,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:00.637265+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:56.829733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:58.008691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:00.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f841c06b020 msgr2=0x7f841c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f841c06b020 0x7f841c06d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84300cfc60 tx=0x7f8424009040 comp rx=0 tx=0).stop 2023-10-16T10:00:00.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 msgr2=0x7f84301f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301f48d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f842000c4d0 tx=0x7f842000c7e0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 shutdown_connections 2023-10-16T10:00:00.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f841c06b020 0x7f841c06d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8430161b60 0x7f843015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f843006d5f0 0x7f84301f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 --2- 172.21.15.181:0/1334524089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430160d20 0x7f84301f48d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 >> 172.21.15.181:0/1334524089 conn(0x7f843009cba0 msgr2=0x7f84300b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:00.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 shutdown_connections 2023-10-16T10:00:00.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.672+0000 7f8435612700 1 -- 172.21.15.181:0/1334524089 wait complete. 2023-10-16T10:00:00.685 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:00:00.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.808+0000 7f61c405e700 1 Processor -- start 2023-10-16T10:00:00.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.808+0000 7f61c405e700 1 -- start start 2023-10-16T10:00:00.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c405e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc15d9e0 0x7f61bc0d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c405e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61bc0cefa0 0x7f61bc0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c405e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc0d4180 con 0x7f61bc0d00c0 2023-10-16T10:00:00.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c405e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61bc0d20c0 con 0x7f61bc0cefa0 2023-10-16T10:00:00.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c405e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0d1440 con 0x7f61bc15d9e0 2023-10-16T10:00:00.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61bc0cefa0 0x7f61bc0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61bc0cefa0 0x7f61bc0cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43112/0 (socket says 172.21.15.181:43112) 2023-10-16T10:00:00.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 -- 172.21.15.181:0/1259091976 learned_addr learned my addr 172.21.15.181:0/1259091976 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:00.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 -- 172.21.15.181:0/1259091976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc15d9e0 msgr2=0x7f61bc0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:00.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 --2- 172.21.15.181:0/1259091976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc15d9e0 0x7f61bc0d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 -- 172.21.15.181:0/1259091976 >> v1:172.21.15.190:6789/0 conn(0x7f61bc0d00c0 legacy=0x7f61bc0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:00.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 -- 172.21.15.181:0/1259091976 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc0d1720 con 0x7f61bc0cefa0 2023-10-16T10:00:00.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c15f9700 1 --2- 172.21.15.181:0/1259091976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc0cefa0 0x7f61bc0cf420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098f0 tx=0x7f61ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d377820f4ada4d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c0df8700 1 -- 172.21.15.181:0/1259091976 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac004230 con 0x7f61bc0cefa0 2023-10-16T10:00:00.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c0df8700 1 -- 172.21.15.181:0/1259091976 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac004390 con 0x7f61bc0cefa0 2023-10-16T10:00:00.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.809+0000 7f61c0df8700 1 -- 172.21.15.181:0/1259091976 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0036a0 con 0x7f61bc0cefa0 2023-10-16T10:00:00.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 -- 172.21.15.181:0/1259091976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc0cefa0 msgr2=0x7f61bc0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 --2- 172.21.15.181:0/1259091976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc0cefa0 0x7f61bc0cf420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098f0 tx=0x7f61ac009c00 comp rx=0 tx=0).stop 2023-10-16T10:00:00.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 -- 172.21.15.181:0/1259091976 shutdown_connections 2023-10-16T10:00:00.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 --2- 172.21.15.181:0/1259091976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc15d9e0 0x7f61bc0d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 --2- 172.21.15.181:0/1259091976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc0cefa0 0x7f61bc0cf420 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 -- 172.21.15.181:0/1259091976 >> 172.21.15.181:0/1259091976 conn(0x7f61bc09cb40 msgr2=0x7f61bc0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:00.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 -- 172.21.15.181:0/1259091976 shutdown_connections 2023-10-16T10:00:00.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.810+0000 7f61c405e700 1 -- 172.21.15.181:0/1259091976 wait complete. 2023-10-16T10:00:00.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.811+0000 7f61c405e700 1 Processor -- start 2023-10-16T10:00:00.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.811+0000 7f61c405e700 1 -- start start 2023-10-16T10:00:00.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c405e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc0cefa0 0x7f61bc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c405e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc0d00c0 0x7f61bc157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c405e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc15d9e0 0x7f61bc157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c405e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc158a80 con 0x7f61bc15d9e0 2023-10-16T10:00:00.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c405e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc157ea0 con 0x7f61bc0cefa0 2023-10-16T10:00:00.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c405e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0cc780 con 0x7f61bc0d00c0 2023-10-16T10:00:00.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c25fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc15d9e0 0x7f61bc157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c25fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc15d9e0 0x7f61bc157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43122/0 (socket says 172.21.15.181:43122) 2023-10-16T10:00:00.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c25fb700 1 -- 172.21.15.181:0/4100456372 learned_addr learned my addr 172.21.15.181:0/4100456372 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:00.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c1dfa700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc0cefa0 0x7f61bc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c15f9700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc0d00c0 0x7f61bc157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.812+0000 7f61c25fb700 1 -- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc0d00c0 msgr2=0x7f61bc157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61c25fb700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc0d00c0 0x7f61bc157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61c25fb700 1 -- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc0cefa0 msgr2=0x7f61bc0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:00.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61c25fb700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc0cefa0 0x7f61bc0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:00.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61c25fb700 1 -- 172.21.15.181:0/4100456372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac009550 con 0x7f61bc15d9e0 2023-10-16T10:00:00.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61c25fb700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc15d9e0 0x7f61bc157960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c7a0 tx=0x7f61b400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61b27fc700 1 -- 172.21.15.181:0/4100456372 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4003e10 con 0x7f61bc15d9e0 2023-10-16T10:00:00.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61b27fc700 1 -- 172.21.15.181:0/4100456372 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4005070 con 0x7f61bc15d9e0 2023-10-16T10:00:00.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc0cca60 con 0x7f61bc15d9e0 2023-10-16T10:00:00.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61b27fc700 1 -- 172.21.15.181:0/4100456372 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4005420 con 0x7f61bc15d9e0 2023-10-16T10:00:00.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.813+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc0cd020 con 0x7f61bc15d9e0 2023-10-16T10:00:00.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.814+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc067d20 con 0x7f61bc15d9e0 2023-10-16T10:00:00.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.814+0000 7f61b27fc700 1 -- 172.21.15.181:0/4100456372 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f61b4005580 con 0x7f61bc15d9e0 2023-10-16T10:00:00.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.815+0000 7f61b27fc700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f61a006b020 0x7f61a006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:00.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.815+0000 7f61b27fc700 1 -- 172.21.15.181:0/4100456372 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f61b4088d30 con 0x7f61bc15d9e0 2023-10-16T10:00:00.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.815+0000 7f61c1dfa700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f61a006b020 0x7f61a006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:00.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.815+0000 7f61c1dfa700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f61a006b020 0x7f61a006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f61b8006fd0 tx=0x7f61b8008040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:00.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:00.819+0000 7f61b27fc700 1 -- 172.21.15.181:0/4100456372 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61b4053aa0 con 0x7f61bc15d9e0 2023-10-16T10:00:01.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.063+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f61bc0d0e50 con 0x7f61bc15d9e0 2023-10-16T10:00:01.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.980+0000 7f61b27fc700 1 -- 172.21.15.181:0/4100456372 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f61b40570c0 con 0x7f61bc15d9e0 2023-10-16T10:00:01.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.983+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f61a006b020 msgr2=0x7f61a006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:01.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.983+0000 7f61c405e700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f61a006b020 0x7f61a006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f61b8006fd0 tx=0x7f61b8008040 comp rx=0 tx=0).stop 2023-10-16T10:00:01.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.983+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc15d9e0 msgr2=0x7f61bc157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:01.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.983+0000 7f61c405e700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc15d9e0 0x7f61bc157960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c7a0 tx=0x7f61b400cb60 comp rx=0 tx=0).stop 2023-10-16T10:00:01.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 shutdown_connections 2023-10-16T10:00:01.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f61a006b020 0x7f61a006d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:01.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61bc15d9e0 0x7f61bc157960 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:01.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61bc0cefa0 0x7f61bc0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:01.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 --2- 172.21.15.181:0/4100456372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61bc0d00c0 0x7f61bc157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:01.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 >> 172.21.15.181:0/4100456372 conn(0x7f61bc09cb40 msgr2=0x7f61bc15f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:01.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 shutdown_connections 2023-10-16T10:00:01.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:01.984+0000 7f61c405e700 1 -- 172.21.15.181:0/4100456372 wait complete. 2023-10-16T10:00:01.999 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:00:02.000 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:00:02.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.128+0000 7fa9bdab7700 1 Processor -- start 2023-10-16T10:00:02.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.128+0000 7fa9bdab7700 1 -- start start 2023-10-16T10:00:02.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9bdab7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9b8072760 0x7fa9b806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:02.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9bdab7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa9b806ade0 0x7fa9b81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:02.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9bdab7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b806b210 con 0x7fa9b815da70 2023-10-16T10:00:02.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9bdab7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b80cef00 con 0x7fa9b8072760 2023-10-16T10:00:02.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9bdab7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b80cec20 con 0x7fa9b806ade0 2023-10-16T10:00:02.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9b8072760 0x7fa9b806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:02.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9b8072760 0x7fa9b806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43130/0 (socket says 172.21.15.181:43130) 2023-10-16T10:00:02.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 -- 172.21.15.181:0/3724519623 learned_addr learned my addr 172.21.15.181:0/3724519623 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:02.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b7fff700 1 --2- 172.21.15.181:0/3724519623 >> v2:172.21.15.190:3300/0 conn(0x7fa9b806ade0 0x7fa9b81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:02.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 -- 172.21.15.181:0/3724519623 >> v2:172.21.15.190:3300/0 conn(0x7fa9b806ade0 msgr2=0x7fa9b81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:02.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 --2- 172.21.15.181:0/3724519623 >> v2:172.21.15.190:3300/0 conn(0x7fa9b806ade0 0x7fa9b81615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 -- 172.21.15.181:0/3724519623 >> v1:172.21.15.190:6789/0 conn(0x7fa9b815da70 legacy=0x7fa9b80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:02.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 -- 172.21.15.181:0/3724519623 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b8071c80 con 0x7fa9b8072760 2023-10-16T10:00:02.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b6ffd700 1 --2- 172.21.15.181:0/3724519623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8072760 0x7fa9b806a8a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b66b0 tx=0x7fa9a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f004b6b44adef603 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:02.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.129+0000 7fa9b67fc700 1 -- 172.21.15.181:0/3724519623 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8004230 con 0x7fa9b8072760 2023-10-16T10:00:02.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9b67fc700 1 -- 172.21.15.181:0/3724519623 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8004390 con 0x7fa9b8072760 2023-10-16T10:00:02.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9b67fc700 1 -- 172.21.15.181:0/3724519623 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80036a0 con 0x7fa9b8072760 2023-10-16T10:00:02.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 -- 172.21.15.181:0/3724519623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8072760 msgr2=0x7fa9b806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:02.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/3724519623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8072760 0x7fa9b806a8a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80b66b0 tx=0x7fa9a8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 -- 172.21.15.181:0/3724519623 shutdown_connections 2023-10-16T10:00:02.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/3724519623 >> v2:172.21.15.190:3300/0 conn(0x7fa9b806ade0 0x7fa9b81615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/3724519623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b8072760 0x7fa9b806a8a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 -- 172.21.15.181:0/3724519623 >> 172.21.15.181:0/3724519623 conn(0x7fa9b80cd010 msgr2=0x7fa9b80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:02.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 -- 172.21.15.181:0/3724519623 shutdown_connections 2023-10-16T10:00:02.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.130+0000 7fa9bdab7700 1 -- 172.21.15.181:0/3724519623 wait complete. 2023-10-16T10:00:02.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.131+0000 7fa9bdab7700 1 Processor -- start 2023-10-16T10:00:02.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.131+0000 7fa9bdab7700 1 -- start start 2023-10-16T10:00:02.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9bdab7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9b806ade0 0x7fa9b8159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:02.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9bdab7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9b8072760 0x7fa9b80ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:02.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9bdab7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b815da70 0x7fa9b80b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:02.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9bdab7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b815a380 con 0x7fa9b815da70 2023-10-16T10:00:02.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9bdab7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b815a850 con 0x7fa9b806ade0 2023-10-16T10:00:02.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9bdab7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80af3f0 con 0x7fa9b8072760 2023-10-16T10:00:02.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b815da70 0x7fa9b80b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:02.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b815da70 0x7fa9b80b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43136/0 (socket says 172.21.15.181:43136) 2023-10-16T10:00:02.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9b7fff700 1 -- 172.21.15.181:0/959882562 learned_addr learned my addr 172.21.15.181:0/959882562 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:02.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9b77fe700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9b806ade0 0x7fa9b8159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:02.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.132+0000 7fa9b6ffd700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9b8072760 0x7fa9b80ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:02.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9b7fff700 1 -- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9b8072760 msgr2=0x7fa9b80ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:02.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9b7fff700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9b8072760 0x7fa9b80ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9b7fff700 1 -- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9b806ade0 msgr2=0x7fa9b8159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:02.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9b7fff700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9b806ade0 0x7fa9b8159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9b7fff700 1 -- 172.21.15.181:0/959882562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8009550 con 0x7fa9b815da70 2023-10-16T10:00:02.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9b6ffd700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9b8072760 0x7fa9b80ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:02.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9b7fff700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b815da70 0x7fa9b80b1bb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0007e60 tx=0x7fa9b000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:02.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa99ffff700 1 -- 172.21.15.181:0/959882562 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000ed80 con 0x7fa9b815da70 2023-10-16T10:00:02.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa99ffff700 1 -- 172.21.15.181:0/959882562 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b0007e90 con 0x7fa9b815da70 2023-10-16T10:00:02.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b80ad7d0 con 0x7fa9b815da70 2023-10-16T10:00:02.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa99ffff700 1 -- 172.21.15.181:0/959882562 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0014660 con 0x7fa9b815da70 2023-10-16T10:00:02.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.133+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b80aeac0 con 0x7fa9b815da70 2023-10-16T10:00:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.134+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b80b31b0 con 0x7fa9b815da70 2023-10-16T10:00:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.134+0000 7fa99ffff700 1 -- 172.21.15.181:0/959882562 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9b00147c0 con 0x7fa9b815da70 2023-10-16T10:00:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.135+0000 7fa99ffff700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fa9a406b020 0x7fa9a406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.135+0000 7fa99ffff700 1 -- 172.21.15.181:0/959882562 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fa9b0089e50 con 0x7fa9b815da70 2023-10-16T10:00:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.135+0000 7fa9b77fe700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fa9a406b020 0x7fa9a406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.135+0000 7fa9b77fe700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fa9a406b020 0x7fa9a406d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80adbd0 tx=0x7fa9ac007ec0 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.139+0000 7fa99ffff700 1 -- 172.21.15.181:0/959882562 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa9b0054d80 con 0x7fa9b815da70 2023-10-16T10:00:02.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.378+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa9b8071f30 con 0x7fa9b815da70 2023-10-16T10:00:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.986+0000 7fa99ffff700 1 -- 172.21.15.181:0/959882562 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa9b00583a0 con 0x7fa9b815da70 2023-10-16T10:00:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fa9a406b020 msgr2=0x7fa9a406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fa9a406b020 0x7fa9a406d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80adbd0 tx=0x7fa9ac007ec0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b815da70 msgr2=0x7fa9b80b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b815da70 0x7fa9b80b1bb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0007e60 tx=0x7fa9b000e3c0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 shutdown_connections 2023-10-16T10:00:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fa9a406b020 0x7fa9a406d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9b815da70 0x7fa9b80b1bb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9b806ade0 0x7fa9b8159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 --2- 172.21.15.181:0/959882562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9b8072760 0x7fa9b80ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 >> 172.21.15.181:0/959882562 conn(0x7fa9b80cd010 msgr2=0x7fa9b80ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 shutdown_connections 2023-10-16T10:00:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:02.989+0000 7fa9bdab7700 1 -- 172.21.15.181:0/959882562 wait complete. 2023-10-16T10:00:03.005 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:00:03.005 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:03.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.130+0000 7f7393789700 1 Processor -- start 2023-10-16T10:00:03.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.131+0000 7f7393789700 1 -- start start 2023-10-16T10:00:03.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.131+0000 7f7393789700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f738c0ce7f0 0x7f738c06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.131+0000 7f7393789700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f738c0d0350 0x7f738c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.131+0000 7f7393789700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0cc470 con 0x7f738c0d2390 2023-10-16T10:00:03.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.131+0000 7f7393789700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c06ca30 con 0x7f738c0d0350 2023-10-16T10:00:03.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.131+0000 7f7393789700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c06cd10 con 0x7f738c0ce7f0 2023-10-16T10:00:03.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f738c0d0350 0x7f738c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f738c0d0350 0x7f738c0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43142/0 (socket says 172.21.15.181:43142) 2023-10-16T10:00:03.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 -- 172.21.15.181:0/591461799 learned_addr learned my addr 172.21.15.181:0/591461799 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:03.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391525700 1 --2- 172.21.15.181:0/591461799 >> v2:172.21.15.190:3300/0 conn(0x7f738c0ce7f0 0x7f738c06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 -- 172.21.15.181:0/591461799 >> v2:172.21.15.190:3300/0 conn(0x7f738c0ce7f0 msgr2=0x7f738c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:03.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 --2- 172.21.15.181:0/591461799 >> v2:172.21.15.190:3300/0 conn(0x7f738c0ce7f0 0x7f738c06c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 -- 172.21.15.181:0/591461799 >> v1:172.21.15.190:6789/0 conn(0x7f738c0d2390 legacy=0x7f738c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 -- 172.21.15.181:0/591461799 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c06cff0 con 0x7f738c0d0350 2023-10-16T10:00:03.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f7391d26700 1 --2- 172.21.15.181:0/591461799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0cc930 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f738c09af40 tx=0x7f738400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f7d9a424048f09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f737ffff700 1 -- 172.21.15.181:0/591461799 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400e070 con 0x7f738c0d0350 2023-10-16T10:00:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f737ffff700 1 -- 172.21.15.181:0/591461799 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738400b9c0 con 0x7f738c0d0350 2023-10-16T10:00:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.132+0000 7f737ffff700 1 -- 172.21.15.181:0/591461799 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400bd70 con 0x7f738c0d0350 2023-10-16T10:00:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 -- 172.21.15.181:0/591461799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 msgr2=0x7f738c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 --2- 172.21.15.181:0/591461799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0cc930 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f738c09af40 tx=0x7f738400b330 comp rx=0 tx=0).stop 2023-10-16T10:00:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 -- 172.21.15.181:0/591461799 shutdown_connections 2023-10-16T10:00:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 --2- 172.21.15.181:0/591461799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0cc930 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 --2- 172.21.15.181:0/591461799 >> v2:172.21.15.190:3300/0 conn(0x7f738c0ce7f0 0x7f738c06c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 -- 172.21.15.181:0/591461799 >> 172.21.15.181:0/591461799 conn(0x7f738c09a9a0 msgr2=0x7f738c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:03.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 -- 172.21.15.181:0/591461799 shutdown_connections 2023-10-16T10:00:03.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.133+0000 7f7393789700 1 -- 172.21.15.181:0/591461799 wait complete. 2023-10-16T10:00:03.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.134+0000 7f7393789700 1 Processor -- start 2023-10-16T10:00:03.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.134+0000 7f7393789700 1 -- start start 2023-10-16T10:00:03.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7393789700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f738c0ce7f0 0x7f738c14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7393789700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7393789700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f738c0d2390 0x7f738c0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7393789700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c14c070 con 0x7f738c0d0350 2023-10-16T10:00:03.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7393789700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0c51b0 con 0x7f738c0ce7f0 2023-10-16T10:00:03.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7393789700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0c5400 con 0x7f738c0d2390 2023-10-16T10:00:03.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43144/0 (socket says 172.21.15.181:43144) 2023-10-16T10:00:03.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 -- 172.21.15.181:0/52023931 learned_addr learned my addr 172.21.15.181:0/52023931 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:03.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 -- 172.21.15.181:0/52023931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f738c0d2390 msgr2=0x7f738c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f738c0d2390 0x7f738c0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 -- 172.21.15.181:0/52023931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f738c0ce7f0 msgr2=0x7f738c14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f738c0ce7f0 0x7f738c14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 -- 172.21.15.181:0/52023931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384009b30 con 0x7f738c0d0350 2023-10-16T10:00:03.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7390d24700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0ca700 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f738800c510 tx=0x7f738800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f737dffb700 1 -- 172.21.15.181:0/52023931 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880052d0 con 0x7f738c0d0350 2023-10-16T10:00:03.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f737dffb700 1 -- 172.21.15.181:0/52023931 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7388005430 con 0x7f738c0d0350 2023-10-16T10:00:03.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c0c56e0 con 0x7f738c0d0350 2023-10-16T10:00:03.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.135+0000 7f737dffb700 1 -- 172.21.15.181:0/52023931 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880038c0 con 0x7f738c0d0350 2023-10-16T10:00:03.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.136+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c0c5ca0 con 0x7f738c0d0350 2023-10-16T10:00:03.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.137+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c067d20 con 0x7f738c0d0350 2023-10-16T10:00:03.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.137+0000 7f737dffb700 1 -- 172.21.15.181:0/52023931 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7388003a20 con 0x7f738c0d0350 2023-10-16T10:00:03.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.137+0000 7f737dffb700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f737006b020 0x7f737006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.137+0000 7f737dffb700 1 -- 172.21.15.181:0/52023931 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f7388088dd0 con 0x7f738c0d0350 2023-10-16T10:00:03.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.137+0000 7f7391525700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f737006b020 0x7f737006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.138+0000 7f7391525700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f737006b020 0x7f737006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7380006eb0 tx=0x7f7380008040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.142+0000 7f737dffb700 1 -- 172.21.15.181:0/52023931 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7388053bc0 con 0x7f738c0d0350 2023-10-16T10:00:03.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.377+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f738c0cdbf0 con 0x7f738c0d0350 2023-10-16T10:00:03.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.378+0000 7f737dffb700 1 -- 172.21.15.181:0/52023931 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f7388017090 con 0x7f738c0d0350 2023-10-16T10:00:03.379 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:03.379 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3576975018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3576975018},{"type":"v1","addr":"172.21.15.190:6834","nonce":3576975018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/54128065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":54128065},{"type":"v1","addr":"172.21.15.190:6835","nonce":54128065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/509615642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":509615642},{"type":"v1","addr":"172.21.15.181:6836","nonce":509615642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:59.043602+0000","modified":"2023-10-16T10:00:02.379610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2780127849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2780127849},{"type":"v1","addr":"172.21.15.181:6837","nonce":2780127849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:00:03.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f737006b020 msgr2=0x7f737006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f737006b020 0x7f737006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7380006eb0 tx=0x7f7380008040 comp rx=0 tx=0).stop 2023-10-16T10:00:03.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 msgr2=0x7f738c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0ca700 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f738800c510 tx=0x7f738800c8d0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 shutdown_connections 2023-10-16T10:00:03.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f737006b020 0x7f737006d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f738c0d2390 0x7f738c0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f738c0ce7f0 0x7f738c14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 --2- 172.21.15.181:0/52023931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c0d0350 0x7f738c0ca700 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 >> 172.21.15.181:0/52023931 conn(0x7f738c09a9a0 msgr2=0x7f738c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:03.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 shutdown_connections 2023-10-16T10:00:03.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.380+0000 7f7393789700 1 -- 172.21.15.181:0/52023931 wait complete. 2023-10-16T10:00:03.384 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 40 2023-10-16T10:00:03.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:03.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.512+0000 7f9d26322700 1 Processor -- start 2023-10-16T10:00:03.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.512+0000 7f9d26322700 1 -- start start 2023-10-16T10:00:03.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d26322700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d20161b60 0x7f9d2006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d26322700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d20160d20 0x7f9d201611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d26322700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2006ce20 con 0x7f9d20161f40 2023-10-16T10:00:03.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d26322700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d20162800 con 0x7f9d20160d20 2023-10-16T10:00:03.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d26322700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d200d43e0 con 0x7f9d20161b60 2023-10-16T10:00:03.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d248bf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d20160d20 0x7f9d201611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d1f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9d20161f40 0x7f9d200d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54230/0 (socket says 172.21.15.181:54230) 2023-10-16T10:00:03.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d1f7fe700 1 -- 172.21.15.181:0/3365173368 learned_addr learned my addr 172.21.15.181:0/3365173368 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d1ffff700 1 --2- 172.21.15.181:0/3365173368 >> v2:172.21.15.190:3300/0 conn(0x7f9d20161b60 0x7f9d2006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d248bf700 1 -- 172.21.15.181:0/3365173368 >> v2:172.21.15.190:3300/0 conn(0x7f9d20161b60 msgr2=0x7f9d2006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d248bf700 1 --2- 172.21.15.181:0/3365173368 >> v2:172.21.15.190:3300/0 conn(0x7f9d20161b60 0x7f9d2006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d248bf700 1 -- 172.21.15.181:0/3365173368 >> v1:172.21.15.181:6789/0 conn(0x7f9d20161f40 legacy=0x7f9d200d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d248bf700 1 -- 172.21.15.181:0/3365173368 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d200d45f0 con 0x7f9d20160d20 2023-10-16T10:00:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.513+0000 7f9d1effd700 1 -- 172.21.15.181:0/3365173368 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226047812 0 0) 0x7f9d2006ce20 con 0x7f9d20161f40 2023-10-16T10:00:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d1ffff700 1 --2- 172.21.15.181:0/3365173368 >> v2:172.21.15.190:3300/0 conn(0x7f9d20161b60 0x7f9d2006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d248bf700 1 --2- 172.21.15.181:0/3365173368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d201611d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9d20066f30 tx=0x7f9d1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f90c8b896a5446 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d1effd700 1 -- 172.21.15.181:0/3365173368 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800e070 con 0x7f9d20160d20 2023-10-16T10:00:03.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d1effd700 1 -- 172.21.15.181:0/3365173368 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1800bbd0 con 0x7f9d20160d20 2023-10-16T10:00:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d1effd700 1 -- 172.21.15.181:0/3365173368 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18012430 con 0x7f9d20160d20 2023-10-16T10:00:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 -- 172.21.15.181:0/3365173368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 msgr2=0x7f9d201611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 --2- 172.21.15.181:0/3365173368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d201611d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9d20066f30 tx=0x7f9d1800b330 comp rx=0 tx=0).stop 2023-10-16T10:00:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 -- 172.21.15.181:0/3365173368 shutdown_connections 2023-10-16T10:00:03.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 --2- 172.21.15.181:0/3365173368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d201611d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 --2- 172.21.15.181:0/3365173368 >> v2:172.21.15.190:3300/0 conn(0x7f9d20161b60 0x7f9d2006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 -- 172.21.15.181:0/3365173368 >> 172.21.15.181:0/3365173368 conn(0x7f9d2009cba0 msgr2=0x7f9d200d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:03.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 -- 172.21.15.181:0/3365173368 shutdown_connections 2023-10-16T10:00:03.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.514+0000 7f9d26322700 1 -- 172.21.15.181:0/3365173368 wait complete. 2023-10-16T10:00:03.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 Processor -- start 2023-10-16T10:00:03.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 -- start start 2023-10-16T10:00:03.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d200b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d20161b60 0x7f9d20153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d20161f40 0x7f9d2015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201532d0 con 0x7f9d20160d20 2023-10-16T10:00:03.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201526c0 con 0x7f9d20161b60 2023-10-16T10:00:03.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.516+0000 7f9d26322700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201529a0 con 0x7f9d20161f40 2023-10-16T10:00:03.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d200b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d200b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43158/0 (socket says 172.21.15.181:43158) 2023-10-16T10:00:03.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 -- 172.21.15.181:0/4181931981 learned_addr learned my addr 172.21.15.181:0/4181931981 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:03.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 -- 172.21.15.181:0/4181931981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d20161f40 msgr2=0x7f9d2015e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d20161f40 0x7f9d2015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 -- 172.21.15.181:0/4181931981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d20161b60 msgr2=0x7f9d20153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1f7fe700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d20161b60 0x7f9d20153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d20161b60 0x7f9d20153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 -- 172.21.15.181:0/4181931981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18009a70 con 0x7f9d20160d20 2023-10-16T10:00:03.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1ffff700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d200b5aa0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e810 tx=0x7f9d1400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1cff9700 1 -- 172.21.15.181:0/4181931981 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14005c20 con 0x7f9d20160d20 2023-10-16T10:00:03.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d20152c80 con 0x7f9d20160d20 2023-10-16T10:00:03.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1cff9700 1 -- 172.21.15.181:0/4181931981 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d14005d80 con 0x7f9d20160d20 2023-10-16T10:00:03.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d1cff9700 1 -- 172.21.15.181:0/4181931981 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14006230 con 0x7f9d20160d20 2023-10-16T10:00:03.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.517+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d200b5fe0 con 0x7f9d20160d20 2023-10-16T10:00:03.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.518+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d201535b0 con 0x7f9d20160d20 2023-10-16T10:00:03.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.519+0000 7f9d1cff9700 1 -- 172.21.15.181:0/4181931981 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9d14006390 con 0x7f9d20160d20 2023-10-16T10:00:03.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.519+0000 7f9d1cff9700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9d0c06b020 0x7f9d0c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.519+0000 7f9d1cff9700 1 -- 172.21.15.181:0/4181931981 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f9d140895c0 con 0x7f9d20160d20 2023-10-16T10:00:03.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.519+0000 7f9d1f7fe700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9d0c06b020 0x7f9d0c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.519+0000 7f9d1f7fe700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9d0c06b020 0x7f9d0c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d201f6220 tx=0x7f9d10009040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.523+0000 7f9d1cff9700 1 -- 172.21.15.181:0/4181931981 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9d14054220 con 0x7f9d20160d20 2023-10-16T10:00:03.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.719+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d2004ebb0 con 0x7f9d20160d20 2023-10-16T10:00:03.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.720+0000 7f9d1cff9700 1 -- 172.21.15.181:0/4181931981 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+17248 (secure 0 0 0) 0x7f9d14057840 con 0x7f9d20160d20 2023-10-16T10:00:03.721 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:03.721 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":90,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:00.637265+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:56.829733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:58.008691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9d0c06b020 msgr2=0x7f9d0c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9d0c06b020 0x7f9d0c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9d201f6220 tx=0x7f9d10009040 comp rx=0 tx=0).stop 2023-10-16T10:00:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 msgr2=0x7f9d200b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d200b5aa0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e810 tx=0x7f9d1400ebd0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 shutdown_connections 2023-10-16T10:00:03.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f9d0c06b020 0x7f9d0c06d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d20161f40 0x7f9d2015e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d20160d20 0x7f9d200b5aa0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 --2- 172.21.15.181:0/4181931981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d20161b60 0x7f9d20153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 >> 172.21.15.181:0/4181931981 conn(0x7f9d2009cba0 msgr2=0x7f9d20154a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:03.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 shutdown_connections 2023-10-16T10:00:03.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.722+0000 7f9d26322700 1 -- 172.21.15.181:0/4181931981 wait complete. 2023-10-16T10:00:03.735 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:00:03.736 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:00:03.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.856+0000 7fd68bb07700 1 Processor -- start 2023-10-16T10:00:03.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.856+0000 7fd68bb07700 1 -- start start 2023-10-16T10:00:03.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd68bb07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 0x7fd6840ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd68bb07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6840d0370 0x7fd6840d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd68bb07700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6840cc510 con 0x7fd6840cd2d0 2023-10-16T10:00:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd68bb07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840d24f0 con 0x7fd6840d0370 2023-10-16T10:00:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd68bb07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840cc7f0 con 0x7fd68415b990 2023-10-16T10:00:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6890a2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd6840cd2d0 0x7fd6840cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54244/0 (socket says 172.21.15.181:54244) 2023-10-16T10:00:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6890a2700 1 -- 172.21.15.181:0/3457135634 learned_addr learned my addr 172.21.15.181:0/3457135634 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd68a0a4700 1 --2- 172.21.15.181:0/3457135634 >> v2:172.21.15.190:3300/0 conn(0x7fd6840d0370 0x7fd6840d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6898a3700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 0x7fd6840ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081649111 0 0) 0x7fd6840cc510 con 0x7fd6840cd2d0 2023-10-16T10:00:03.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6700035b0 con 0x7fd6840cd2d0 2023-10-16T10:00:03.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833741583 0 0) 0x7fd6700035b0 con 0x7fd6840cd2d0 2023-10-16T10:00:03.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 msgr2=0x7fd6840ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:03.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 0x7fd6840ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 >> v2:172.21.15.190:3300/0 conn(0x7fd6840d0370 msgr2=0x7fd6840d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:03.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 --2- 172.21.15.181:0/3457135634 >> v2:172.21.15.190:3300/0 conn(0x7fd6840d0370 0x7fd6840d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6840ccad0 con 0x7fd6840cd2d0 2023-10-16T10:00:03.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd674001670 con 0x7fd6840cd2d0 2023-10-16T10:00:03.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 >> v1:172.21.15.181:6789/0 conn(0x7fd6840cd2d0 legacy=0x7fd6840cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 0x7fd670003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd670003f80 0x7fd670004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670004d90 0x7fd670005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840d24f0 con 0x7fd670003f80 2023-10-16T10:00:03.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840cc7f0 con 0x7fd670004d90 2023-10-16T10:00:03.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.857+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67000ba40 con 0x7fd68415b990 2023-10-16T10:00:03.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6890a2700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd670003f80 0x7fd670004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd68a0a4700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670004d90 0x7fd670005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6898a3700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 0x7fd670003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6890a2700 1 -- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 msgr2=0x7fd670003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6890a2700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 0x7fd670003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6890a2700 1 -- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670004d90 msgr2=0x7fd670005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:03.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6890a2700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd670004d90 0x7fd670005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6890a2700 1 -- 172.21.15.181:0/3457135634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6700035b0 con 0x7fd670003f80 2023-10-16T10:00:03.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6898a3700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68415b990 0x7fd670003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:03.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6890a2700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd670003f80 0x7fd670004850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c320 tx=0x7fd670005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c289eb6ccf6576 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400ce60 con 0x7fd670003f80 2023-10-16T10:00:03.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd674005960 con 0x7fd670003f80 2023-10-16T10:00:03.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.858+0000 7fd6888a1700 1 -- 172.21.15.181:0/3457135634 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674005d10 con 0x7fd670003f80 2023-10-16T10:00:03.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.859+0000 7fd68bb07700 1 -- 172.21.15.181:0/3457135634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd670003f80 msgr2=0x7fd670004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.859+0000 7fd68bb07700 1 --2- 172.21.15.181:0/3457135634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd670003f80 0x7fd670004850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c320 tx=0x7fd670005880 comp rx=0 tx=0).stop 2023-10-16T10:00:03.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.859+0000 7fd68a0a4700 1 -- 172.21.15.181:0/3457135634 reap_dead start 2023-10-16T10:00:03.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.859+0000 7fd68bb07700 1 -- 172.21.15.181:0/3457135634 shutdown_connections 2023-10-16T10:00:03.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.859+0000 7fd68bb07700 1 -- 172.21.15.181:0/3457135634 >> 172.21.15.181:0/3457135634 conn(0x7fd68409a9a0 msgr2=0x7fd6840d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:03.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.859+0000 7fd68bb07700 1 -- 172.21.15.181:0/3457135634 shutdown_connections 2023-10-16T10:00:03.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.859+0000 7fd68bb07700 1 -- 172.21.15.181:0/3457135634 wait complete. 2023-10-16T10:00:03.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.860+0000 7fd68bb07700 1 Processor -- start 2023-10-16T10:00:03.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.860+0000 7fd68bb07700 1 -- start start 2023-10-16T10:00:03.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.860+0000 7fd68bb07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd670003f80 0x7fd684158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68bb07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd68415e400 0x7fd68415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68bb07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840b60f0 0x7fd6840b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68bb07700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68415edc0 con 0x7fd68415e400 2023-10-16T10:00:03.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68bb07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6841f0d10 con 0x7fd6840b60f0 2023-10-16T10:00:03.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68bb07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840b6ab0 con 0x7fd670003f80 2023-10-16T10:00:03.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd68415e400 0x7fd68415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd68415e400 0x7fd68415e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43178/0 (socket says 172.21.15.181:43178) 2023-10-16T10:00:03.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 -- 172.21.15.181:0/2242933387 learned_addr learned my addr 172.21.15.181:0/2242933387 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:03.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68a0a4700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840b60f0 0x7fd6840b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6898a3700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd670003f80 0x7fd684158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 -- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd670003f80 msgr2=0x7fd684158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd670003f80 0x7fd684158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 -- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840b60f0 msgr2=0x7fd6840b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:03.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840b60f0 0x7fd6840b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:03.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 -- 172.21.15.181:0/2242933387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6841f0fd0 con 0x7fd68415e400 2023-10-16T10:00:03.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68a0a4700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840b60f0 0x7fd6840b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:03.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd6890a2700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd68415e400 0x7fd68415e880 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd674000c00 tx=0x7fd674001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd67a7fc700 1 -- 172.21.15.181:0/2242933387 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674001660 con 0x7fd68415e400 2023-10-16T10:00:03.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd67a7fc700 1 -- 172.21.15.181:0/2242933387 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd674004a80 con 0x7fd68415e400 2023-10-16T10:00:03.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd67a7fc700 1 -- 172.21.15.181:0/2242933387 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674004e30 con 0x7fd68415e400 2023-10-16T10:00:03.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6841f12b0 con 0x7fd68415e400 2023-10-16T10:00:03.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.861+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6841f1870 con 0x7fd68415e400 2023-10-16T10:00:03.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.862+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6840b42a0 con 0x7fd68415e400 2023-10-16T10:00:03.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.863+0000 7fd67a7fc700 1 -- 172.21.15.181:0/2242933387 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd6740070d0 con 0x7fd68415e400 2023-10-16T10:00:03.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.863+0000 7fd67a7fc700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fd66806b020 0x7fd66806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:03.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.863+0000 7fd67a7fc700 1 -- 172.21.15.181:0/2242933387 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fd67408afe0 con 0x7fd68415e400 2023-10-16T10:00:03.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.863+0000 7fd6898a3700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fd66806b020 0x7fd66806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:03.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.863+0000 7fd6898a3700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fd66806b020 0x7fd66806d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6800099b0 tx=0x7fd680008040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:03.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:03.867+0000 7fd67a7fc700 1 -- 172.21.15.181:0/2242933387 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd674056040 con 0x7fd68415e400 2023-10-16T10:00:04.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.057+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 --> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd68415edc0 con 0x7fd66806b020 2023-10-16T10:00:04.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.065+0000 7fd67a7fc700 1 -- 172.21.15.181:0/2242933387 <== mgr.6732 v2:172.21.15.181:6832/752673838 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289562 (secure 0 0 0) 0x7fd68415edc0 con 0x7fd66806b020 2023-10-16T10:00:04.069 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.068+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fd66806b020 msgr2=0x7fd66806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.068+0000 7fd68bb07700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fd66806b020 0x7fd66806d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6800099b0 tx=0x7fd680008040 comp rx=0 tx=0).stop 2023-10-16T10:00:04.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.068+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd68415e400 msgr2=0x7fd68415e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.068+0000 7fd68bb07700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd68415e400 0x7fd68415e880 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd674000c00 tx=0x7fd674001a00 comp rx=0 tx=0).stop 2023-10-16T10:00:04.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.068+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 shutdown_connections 2023-10-16T10:00:04.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.069+0000 7fd68bb07700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fd66806b020 0x7fd66806d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.069+0000 7fd68bb07700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd670003f80 0x7fd684158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.069+0000 7fd68bb07700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840b60f0 0x7fd6840b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.069+0000 7fd68bb07700 1 --2- 172.21.15.181:0/2242933387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd68415e400 0x7fd68415e880 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.069+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 >> 172.21.15.181:0/2242933387 conn(0x7fd68409a9a0 msgr2=0x7fd6840b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:04.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.069+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 shutdown_connections 2023-10-16T10:00:04.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.069+0000 7fd68bb07700 1 -- 172.21.15.181:0/2242933387 wait complete. 2023-10-16T10:00:04.074 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:00:04.081 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T10:00:02.587385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307708,"kb_used_data":9656,"kb_used_omap":12,"kb_used_meta":297907,"kb_avail":754667012,"statfs":{"total":773094113280,"available":772779020288,"internally_reserved":0,"allocated":9887744,"data_stored":5376451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":305057336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.954850"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664385+0000","last_change":"2023-10-16T09:59:59.662665+0000","last_active":"2023-10-16T09:59:59.664385+0000","last_peered":"2023-10-16T09:59:59.664385+0000","last_clean":"2023-10-16T09:59:59.664385+0000","last_became_active":"2023-10-16T09:59:59.662221+0000","last_became_peered":"2023-10-16T09:59:59.662221+0000","last_unstale":"2023-10-16T09:59:59.664385+0000","last_undegraded":"2023-10-16T09:59:59.664385+0000","last_fullsized":"2023-10-16T09:59:59.664385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:55.232049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660741+0000","last_change":"2023-10-16T09:59:59.656725+0000","last_active":"2023-10-16T09:59:59.660741+0000","last_peered":"2023-10-16T09:59:59.660741+0000","last_clean":"2023-10-16T09:59:59.660741+0000","last_became_active":"2023-10-16T09:59:59.656366+0000","last_became_peered":"2023-10-16T09:59:59.656366+0000","last_unstale":"2023-10-16T09:59:59.660741+0000","last_undegraded":"2023-10-16T09:59:59.660741+0000","last_fullsized":"2023-10-16T09:59:59.660741+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:17.421068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664926+0000","last_change":"2023-10-16T09:59:59.661962+0000","last_active":"2023-10-16T09:59:59.664926+0000","last_peered":"2023-10-16T09:59:59.664926+0000","last_clean":"2023-10-16T09:59:59.664926+0000","last_became_active":"2023-10-16T09:59:59.661622+0000","last_became_peered":"2023-10-16T09:59:59.661622+0000","last_unstale":"2023-10-16T09:59:59.664926+0000","last_undegraded":"2023-10-16T09:59:59.664926+0000","last_fullsized":"2023-10-16T09:59:59.664926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:43.735073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.663012+0000","last_change":"2023-10-16T09:59:59.661739+0000","last_active":"2023-10-16T09:59:59.663012+0000","last_peered":"2023-10-16T09:59:59.663012+0000","last_clean":"2023-10-16T09:59:59.663012+0000","last_became_active":"2023-10-16T09:59:59.661086+0000","last_became_peered":"2023-10-16T09:59:59.661086+0000","last_unstale":"2023-10-16T09:59:59.663012+0000","last_undegraded":"2023-10-16T09:59:59.663012+0000","last_fullsized":"2023-10-16T09:59:59.663012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:32:23.604042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660558+0000","last_change":"2023-10-16T09:59:59.655800+0000","last_active":"2023-10-16T09:59:59.660558+0000","last_peered":"2023-10-16T09:59:59.660558+0000","last_clean":"2023-10-16T09:59:59.660558+0000","last_became_active":"2023-10-16T09:59:59.655542+0000","last_became_peered":"2023-10-16T09:59:59.655542+0000","last_unstale":"2023-10-16T09:59:59.660558+0000","last_undegraded":"2023-10-16T09:59:59.660558+0000","last_fullsized":"2023-10-16T09:59:59.660558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:09.692093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658720+0000","last_change":"2023-10-16T09:59:59.652543+0000","last_active":"2023-10-16T09:59:59.658720+0000","last_peered":"2023-10-16T09:59:59.658720+0000","last_clean":"2023-10-16T09:59:59.658720+0000","last_became_active":"2023-10-16T09:59:59.652279+0000","last_became_peered":"2023-10-16T09:59:59.652279+0000","last_unstale":"2023-10-16T09:59:59.658720+0000","last_undegraded":"2023-10-16T09:59:59.658720+0000","last_fullsized":"2023-10-16T09:59:59.658720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:12.553900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664893+0000","last_change":"2023-10-16T09:59:59.664066+0000","last_active":"2023-10-16T09:59:59.664893+0000","last_peered":"2023-10-16T09:59:59.664893+0000","last_clean":"2023-10-16T09:59:59.664893+0000","last_became_active":"2023-10-16T09:59:59.663786+0000","last_became_peered":"2023-10-16T09:59:59.663786+0000","last_unstale":"2023-10-16T09:59:59.664893+0000","last_undegraded":"2023-10-16T09:59:59.664893+0000","last_fullsized":"2023-10-16T09:59:59.664893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:39.249617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664241+0000","last_change":"2023-10-16T09:59:59.661253+0000","last_active":"2023-10-16T09:59:59.664241+0000","last_peered":"2023-10-16T09:59:59.664241+0000","last_clean":"2023-10-16T09:59:59.664241+0000","last_became_active":"2023-10-16T09:59:59.660924+0000","last_became_peered":"2023-10-16T09:59:59.660924+0000","last_unstale":"2023-10-16T09:59:59.664241+0000","last_undegraded":"2023-10-16T09:59:59.664241+0000","last_fullsized":"2023-10-16T09:59:59.664241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:23:18.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666777+0000","last_change":"2023-10-16T09:59:59.663563+0000","last_active":"2023-10-16T09:59:59.666777+0000","last_peered":"2023-10-16T09:59:59.666777+0000","last_clean":"2023-10-16T09:59:59.666777+0000","last_became_active":"2023-10-16T09:59:59.662969+0000","last_became_peered":"2023-10-16T09:59:59.662969+0000","last_unstale":"2023-10-16T09:59:59.666777+0000","last_undegraded":"2023-10-16T09:59:59.666777+0000","last_fullsized":"2023-10-16T09:59:59.666777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:22.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.655537+0000","last_change":"2023-10-16T09:59:59.653517+0000","last_active":"2023-10-16T09:59:59.655537+0000","last_peered":"2023-10-16T09:59:59.655537+0000","last_clean":"2023-10-16T09:59:59.655537+0000","last_became_active":"2023-10-16T09:59:59.653229+0000","last_became_peered":"2023-10-16T09:59:59.653229+0000","last_unstale":"2023-10-16T09:59:59.655537+0000","last_undegraded":"2023-10-16T09:59:59.655537+0000","last_fullsized":"2023-10-16T09:59:59.655537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.818151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665842+0000","last_change":"2023-10-16T09:59:59.664431+0000","last_active":"2023-10-16T09:59:59.665842+0000","last_peered":"2023-10-16T09:59:59.665842+0000","last_clean":"2023-10-16T09:59:59.665842+0000","last_became_active":"2023-10-16T09:59:59.662018+0000","last_became_peered":"2023-10-16T09:59:59.662018+0000","last_unstale":"2023-10-16T09:59:59.665842+0000","last_undegraded":"2023-10-16T09:59:59.665842+0000","last_fullsized":"2023-10-16T09:59:59.665842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:33:11.985316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664387+0000","last_change":"2023-10-16T09:59:59.661877+0000","last_active":"2023-10-16T09:59:59.664387+0000","last_peered":"2023-10-16T09:59:59.664387+0000","last_clean":"2023-10-16T09:59:59.664387+0000","last_became_active":"2023-10-16T09:59:59.661349+0000","last_became_peered":"2023-10-16T09:59:59.661349+0000","last_unstale":"2023-10-16T09:59:59.664387+0000","last_undegraded":"2023-10-16T09:59:59.664387+0000","last_fullsized":"2023-10-16T09:59:59.664387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:42:28.173366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662244+0000","last_change":"2023-10-16T09:59:59.659645+0000","last_active":"2023-10-16T09:59:59.662244+0000","last_peered":"2023-10-16T09:59:59.662244+0000","last_clean":"2023-10-16T09:59:59.662244+0000","last_became_active":"2023-10-16T09:59:59.659341+0000","last_became_peered":"2023-10-16T09:59:59.659341+0000","last_unstale":"2023-10-16T09:59:59.662244+0000","last_undegraded":"2023-10-16T09:59:59.662244+0000","last_fullsized":"2023-10-16T09:59:59.662244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:37.047119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.652173+0000","last_change":"2023-10-16T09:59:59.645777+0000","last_active":"2023-10-16T09:59:59.652173+0000","last_peered":"2023-10-16T09:59:59.652173+0000","last_clean":"2023-10-16T09:59:59.652173+0000","last_became_active":"2023-10-16T09:59:59.645492+0000","last_became_peered":"2023-10-16T09:59:59.645492+0000","last_unstale":"2023-10-16T09:59:59.652173+0000","last_undegraded":"2023-10-16T09:59:59.652173+0000","last_fullsized":"2023-10-16T09:59:59.652173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:04:44.300764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662219+0000","last_change":"2023-10-16T09:59:59.660690+0000","last_active":"2023-10-16T09:59:59.662219+0000","last_peered":"2023-10-16T09:59:59.662219+0000","last_clean":"2023-10-16T09:59:59.662219+0000","last_became_active":"2023-10-16T09:59:59.660297+0000","last_became_peered":"2023-10-16T09:59:59.660297+0000","last_unstale":"2023-10-16T09:59:59.662219+0000","last_undegraded":"2023-10-16T09:59:59.662219+0000","last_fullsized":"2023-10-16T09:59:59.662219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:14.609358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.659703+0000","last_change":"2023-10-16T09:59:59.658267+0000","last_active":"2023-10-16T09:59:59.659703+0000","last_peered":"2023-10-16T09:59:59.659703+0000","last_clean":"2023-10-16T09:59:59.659703+0000","last_became_active":"2023-10-16T09:59:59.657929+0000","last_became_peered":"2023-10-16T09:59:59.657929+0000","last_unstale":"2023-10-16T09:59:59.659703+0000","last_undegraded":"2023-10-16T09:59:59.659703+0000","last_fullsized":"2023-10-16T09:59:59.659703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:17.508223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666760+0000","last_change":"2023-10-16T09:59:59.663569+0000","last_active":"2023-10-16T09:59:59.666760+0000","last_peered":"2023-10-16T09:59:59.666760+0000","last_clean":"2023-10-16T09:59:59.666760+0000","last_became_active":"2023-10-16T09:59:59.663290+0000","last_became_peered":"2023-10-16T09:59:59.663290+0000","last_unstale":"2023-10-16T09:59:59.666760+0000","last_undegraded":"2023-10-16T09:59:59.666760+0000","last_fullsized":"2023-10-16T09:59:59.666760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:03.884054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662489+0000","last_change":"2023-10-16T09:59:59.660241+0000","last_active":"2023-10-16T09:59:59.662489+0000","last_peered":"2023-10-16T09:59:59.662489+0000","last_clean":"2023-10-16T09:59:59.662489+0000","last_became_active":"2023-10-16T09:59:59.659905+0000","last_became_peered":"2023-10-16T09:59:59.659905+0000","last_unstale":"2023-10-16T09:59:59.662489+0000","last_undegraded":"2023-10-16T09:59:59.662489+0000","last_fullsized":"2023-10-16T09:59:59.662489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:28:09.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665295+0000","last_change":"2023-10-16T09:59:59.663580+0000","last_active":"2023-10-16T09:59:59.665295+0000","last_peered":"2023-10-16T09:59:59.665295+0000","last_clean":"2023-10-16T09:59:59.665295+0000","last_became_active":"2023-10-16T09:59:59.663311+0000","last_became_peered":"2023-10-16T09:59:59.663311+0000","last_unstale":"2023-10-16T09:59:59.665295+0000","last_undegraded":"2023-10-16T09:59:59.665295+0000","last_fullsized":"2023-10-16T09:59:59.665295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:08.499979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661463+0000","last_change":"2023-10-16T09:59:59.656725+0000","last_active":"2023-10-16T09:59:59.661463+0000","last_peered":"2023-10-16T09:59:59.661463+0000","last_clean":"2023-10-16T09:59:59.661463+0000","last_became_active":"2023-10-16T09:59:59.656458+0000","last_became_peered":"2023-10-16T09:59:59.656458+0000","last_unstale":"2023-10-16T09:59:59.661463+0000","last_undegraded":"2023-10-16T09:59:59.661463+0000","last_fullsized":"2023-10-16T09:59:59.661463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:19.730450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.655972+0000","last_change":"2023-10-16T09:59:59.652731+0000","last_active":"2023-10-16T09:59:59.655972+0000","last_peered":"2023-10-16T09:59:59.655972+0000","last_clean":"2023-10-16T09:59:59.655972+0000","last_became_active":"2023-10-16T09:59:59.652440+0000","last_became_peered":"2023-10-16T09:59:59.652440+0000","last_unstale":"2023-10-16T09:59:59.655972+0000","last_undegraded":"2023-10-16T09:59:59.655972+0000","last_fullsized":"2023-10-16T09:59:59.655972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:22.187726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.653981+0000","last_change":"2023-10-16T09:59:59.651797+0000","last_active":"2023-10-16T09:59:59.653981+0000","last_peered":"2023-10-16T09:59:59.653981+0000","last_clean":"2023-10-16T09:59:59.653981+0000","last_became_active":"2023-10-16T09:59:59.651456+0000","last_became_peered":"2023-10-16T09:59:59.651456+0000","last_unstale":"2023-10-16T09:59:59.653981+0000","last_undegraded":"2023-10-16T09:59:59.653981+0000","last_fullsized":"2023-10-16T09:59:59.653981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:46.375900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658262+0000","last_change":"2023-10-16T09:59:59.656540+0000","last_active":"2023-10-16T09:59:59.658262+0000","last_peered":"2023-10-16T09:59:59.658262+0000","last_clean":"2023-10-16T09:59:59.658262+0000","last_became_active":"2023-10-16T09:59:59.654971+0000","last_became_peered":"2023-10-16T09:59:59.654971+0000","last_unstale":"2023-10-16T09:59:59.658262+0000","last_undegraded":"2023-10-16T09:59:59.658262+0000","last_fullsized":"2023-10-16T09:59:59.658262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:37.666716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660633+0000","last_change":"2023-10-16T09:59:59.659139+0000","last_active":"2023-10-16T09:59:59.660633+0000","last_peered":"2023-10-16T09:59:59.660633+0000","last_clean":"2023-10-16T09:59:59.660633+0000","last_became_active":"2023-10-16T09:59:59.658876+0000","last_became_peered":"2023-10-16T09:59:59.658876+0000","last_unstale":"2023-10-16T09:59:59.660633+0000","last_undegraded":"2023-10-16T09:59:59.660633+0000","last_fullsized":"2023-10-16T09:59:59.660633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:06.079178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.655513+0000","last_change":"2023-10-16T09:59:59.654098+0000","last_active":"2023-10-16T09:59:59.655513+0000","last_peered":"2023-10-16T09:59:59.655513+0000","last_clean":"2023-10-16T09:59:59.655513+0000","last_became_active":"2023-10-16T09:59:59.653824+0000","last_became_peered":"2023-10-16T09:59:59.653824+0000","last_unstale":"2023-10-16T09:59:59.655513+0000","last_undegraded":"2023-10-16T09:59:59.655513+0000","last_fullsized":"2023-10-16T09:59:59.655513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:14.752732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658269+0000","last_change":"2023-10-16T09:59:59.657200+0000","last_active":"2023-10-16T09:59:59.658269+0000","last_peered":"2023-10-16T09:59:59.658269+0000","last_clean":"2023-10-16T09:59:59.658269+0000","last_became_active":"2023-10-16T09:59:59.656892+0000","last_became_peered":"2023-10-16T09:59:59.656892+0000","last_unstale":"2023-10-16T09:59:59.658269+0000","last_undegraded":"2023-10-16T09:59:59.658269+0000","last_fullsized":"2023-10-16T09:59:59.658269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:34.801362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664905+0000","last_change":"2023-10-16T09:59:59.661235+0000","last_active":"2023-10-16T09:59:59.664905+0000","last_peered":"2023-10-16T09:59:59.664905+0000","last_clean":"2023-10-16T09:59:59.664905+0000","last_became_active":"2023-10-16T09:59:59.660740+0000","last_became_peered":"2023-10-16T09:59:59.660740+0000","last_unstale":"2023-10-16T09:59:59.664905+0000","last_undegraded":"2023-10-16T09:59:59.664905+0000","last_fullsized":"2023-10-16T09:59:59.664905+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:38:36.457869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.654234+0000","last_change":"2023-10-16T09:59:59.651889+0000","last_active":"2023-10-16T09:59:59.654234+0000","last_peered":"2023-10-16T09:59:59.654234+0000","last_clean":"2023-10-16T09:59:59.654234+0000","last_became_active":"2023-10-16T09:59:59.651446+0000","last_became_peered":"2023-10-16T09:59:59.651446+0000","last_unstale":"2023-10-16T09:59:59.654234+0000","last_undegraded":"2023-10-16T09:59:59.654234+0000","last_fullsized":"2023-10-16T09:59:59.654234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:18:34.399822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664638+0000","last_change":"2023-10-16T09:59:59.661388+0000","last_active":"2023-10-16T09:59:59.664638+0000","last_peered":"2023-10-16T09:59:59.664638+0000","last_clean":"2023-10-16T09:59:59.664638+0000","last_became_active":"2023-10-16T09:59:59.661037+0000","last_became_peered":"2023-10-16T09:59:59.661037+0000","last_unstale":"2023-10-16T09:59:59.664638+0000","last_undegraded":"2023-10-16T09:59:59.664638+0000","last_fullsized":"2023-10-16T09:59:59.664638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:09.347465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658019+0000","last_change":"2023-10-16T09:59:59.656922+0000","last_active":"2023-10-16T09:59:59.658019+0000","last_peered":"2023-10-16T09:59:59.658019+0000","last_clean":"2023-10-16T09:59:59.658019+0000","last_became_active":"2023-10-16T09:59:59.656595+0000","last_became_peered":"2023-10-16T09:59:59.656595+0000","last_unstale":"2023-10-16T09:59:59.658019+0000","last_undegraded":"2023-10-16T09:59:59.658019+0000","last_fullsized":"2023-10-16T09:59:59.658019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:47.707940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.656056+0000","last_change":"2023-10-16T09:59:59.650411+0000","last_active":"2023-10-16T09:59:59.656056+0000","last_peered":"2023-10-16T09:59:59.656056+0000","last_clean":"2023-10-16T09:59:59.656056+0000","last_became_active":"2023-10-16T09:59:59.650125+0000","last_became_peered":"2023-10-16T09:59:59.650125+0000","last_unstale":"2023-10-16T09:59:59.656056+0000","last_undegraded":"2023-10-16T09:59:59.656056+0000","last_fullsized":"2023-10-16T09:59:59.656056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:38.709948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666849+0000","last_change":"2023-10-16T09:59:59.665979+0000","last_active":"2023-10-16T09:59:59.666849+0000","last_peered":"2023-10-16T09:59:59.666849+0000","last_clean":"2023-10-16T09:59:59.666849+0000","last_became_active":"2023-10-16T09:59:59.665733+0000","last_became_peered":"2023-10-16T09:59:59.665733+0000","last_unstale":"2023-10-16T09:59:59.666849+0000","last_undegraded":"2023-10-16T09:59:59.666849+0000","last_fullsized":"2023-10-16T09:59:59.666849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:49.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664294+0000","last_change":"2023-10-16T09:59:59.661864+0000","last_active":"2023-10-16T09:59:59.664294+0000","last_peered":"2023-10-16T09:59:59.664294+0000","last_clean":"2023-10-16T09:59:59.664294+0000","last_became_active":"2023-10-16T09:59:59.661606+0000","last_became_peered":"2023-10-16T09:59:59.661606+0000","last_unstale":"2023-10-16T09:59:59.664294+0000","last_undegraded":"2023-10-16T09:59:59.664294+0000","last_fullsized":"2023-10-16T09:59:59.664294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:28.126128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"89'4","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.983167+0000","last_change":"2023-10-16T09:59:58.657015+0000","last_active":"2023-10-16T09:59:59.983167+0000","last_peered":"2023-10-16T09:59:59.983167+0000","last_clean":"2023-10-16T09:59:59.983167+0000","last_became_active":"2023-10-16T09:59:58.656774+0000","last_became_peered":"2023-10-16T09:59:58.656774+0000","last_unstale":"2023-10-16T09:59:59.983167+0000","last_undegraded":"2023-10-16T09:59:59.983167+0000","last_fullsized":"2023-10-16T09:59:59.983167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:45:28.222879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666743+0000","last_change":"2023-10-16T09:59:59.663334+0000","last_active":"2023-10-16T09:59:59.666743+0000","last_peered":"2023-10-16T09:59:59.666743+0000","last_clean":"2023-10-16T09:59:59.666743+0000","last_became_active":"2023-10-16T09:59:59.662887+0000","last_became_peered":"2023-10-16T09:59:59.662887+0000","last_unstale":"2023-10-16T09:59:59.666743+0000","last_undegraded":"2023-10-16T09:59:59.666743+0000","last_fullsized":"2023-10-16T09:59:59.666743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:17.503821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.975923+0000","last_change":"2023-10-16T09:59:58.656288+0000","last_active":"2023-10-16T09:59:59.975923+0000","last_peered":"2023-10-16T09:59:59.975923+0000","last_clean":"2023-10-16T09:59:59.975923+0000","last_became_active":"2023-10-16T09:59:58.655929+0000","last_became_peered":"2023-10-16T09:59:58.655929+0000","last_unstale":"2023-10-16T09:59:59.975923+0000","last_undegraded":"2023-10-16T09:59:59.975923+0000","last_fullsized":"2023-10-16T09:59:59.975923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:56.983631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660573+0000","last_change":"2023-10-16T09:59:59.656921+0000","last_active":"2023-10-16T09:59:59.660573+0000","last_peered":"2023-10-16T09:59:59.660573+0000","last_clean":"2023-10-16T09:59:59.660573+0000","last_became_active":"2023-10-16T09:59:59.656653+0000","last_became_peered":"2023-10-16T09:59:59.656653+0000","last_unstale":"2023-10-16T09:59:59.660573+0000","last_undegraded":"2023-10-16T09:59:59.660573+0000","last_fullsized":"2023-10-16T09:59:59.660573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:48.399295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.979039+0000","last_change":"2023-10-16T09:59:58.656130+0000","last_active":"2023-10-16T09:59:59.979039+0000","last_peered":"2023-10-16T09:59:59.979039+0000","last_clean":"2023-10-16T09:59:59.979039+0000","last_became_active":"2023-10-16T09:59:58.655899+0000","last_became_peered":"2023-10-16T09:59:58.655899+0000","last_unstale":"2023-10-16T09:59:59.979039+0000","last_undegraded":"2023-10-16T09:59:59.979039+0000","last_fullsized":"2023-10-16T09:59:59.979039+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:21.529623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658671+0000","last_change":"2023-10-16T09:59:59.654504+0000","last_active":"2023-10-16T09:59:59.658671+0000","last_peered":"2023-10-16T09:59:59.658671+0000","last_clean":"2023-10-16T09:59:59.658671+0000","last_became_active":"2023-10-16T09:59:59.654113+0000","last_became_peered":"2023-10-16T09:59:59.654113+0000","last_unstale":"2023-10-16T09:59:59.658671+0000","last_undegraded":"2023-10-16T09:59:59.658671+0000","last_fullsized":"2023-10-16T09:59:59.658671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:04:21.462984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.978821+0000","last_change":"2023-10-16T09:59:58.657383+0000","last_active":"2023-10-16T09:59:59.978821+0000","last_peered":"2023-10-16T09:59:59.978821+0000","last_clean":"2023-10-16T09:59:59.978821+0000","last_became_active":"2023-10-16T09:59:58.655813+0000","last_became_peered":"2023-10-16T09:59:58.655813+0000","last_unstale":"2023-10-16T09:59:59.978821+0000","last_undegraded":"2023-10-16T09:59:59.978821+0000","last_fullsized":"2023-10-16T09:59:59.978821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:54.422203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.663531+0000","last_change":"2023-10-16T09:59:59.662073+0000","last_active":"2023-10-16T09:59:59.663531+0000","last_peered":"2023-10-16T09:59:59.663531+0000","last_clean":"2023-10-16T09:59:59.663531+0000","last_became_active":"2023-10-16T09:59:59.661838+0000","last_became_peered":"2023-10-16T09:59:59.661838+0000","last_unstale":"2023-10-16T09:59:59.663531+0000","last_undegraded":"2023-10-16T09:59:59.663531+0000","last_fullsized":"2023-10-16T09:59:59.663531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:59.099478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.976257+0000","last_change":"2023-10-16T09:59:58.646206+0000","last_active":"2023-10-16T09:59:59.976257+0000","last_peered":"2023-10-16T09:59:59.976257+0000","last_clean":"2023-10-16T09:59:59.976257+0000","last_became_active":"2023-10-16T09:59:58.645938+0000","last_became_peered":"2023-10-16T09:59:58.645938+0000","last_unstale":"2023-10-16T09:59:59.976257+0000","last_undegraded":"2023-10-16T09:59:59.976257+0000","last_fullsized":"2023-10-16T09:59:59.976257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:57.676281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.659231+0000","last_change":"2023-10-16T09:59:59.656130+0000","last_active":"2023-10-16T09:59:59.659231+0000","last_peered":"2023-10-16T09:59:59.659231+0000","last_clean":"2023-10-16T09:59:59.659231+0000","last_became_active":"2023-10-16T09:59:59.655753+0000","last_became_peered":"2023-10-16T09:59:59.655753+0000","last_unstale":"2023-10-16T09:59:59.659231+0000","last_undegraded":"2023-10-16T09:59:59.659231+0000","last_fullsized":"2023-10-16T09:59:59.659231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:03.911665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.976774+0000","last_change":"2023-10-16T09:59:58.659019+0000","last_active":"2023-10-16T09:59:59.976774+0000","last_peered":"2023-10-16T09:59:59.976774+0000","last_clean":"2023-10-16T09:59:59.976774+0000","last_became_active":"2023-10-16T09:59:58.657452+0000","last_became_peered":"2023-10-16T09:59:58.657452+0000","last_unstale":"2023-10-16T09:59:59.976774+0000","last_undegraded":"2023-10-16T09:59:59.976774+0000","last_fullsized":"2023-10-16T09:59:59.976774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.403538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.652353+0000","last_change":"2023-10-16T09:59:59.645668+0000","last_active":"2023-10-16T09:59:59.652353+0000","last_peered":"2023-10-16T09:59:59.652353+0000","last_clean":"2023-10-16T09:59:59.652353+0000","last_became_active":"2023-10-16T09:59:59.645414+0000","last_became_peered":"2023-10-16T09:59:59.645414+0000","last_unstale":"2023-10-16T09:59:59.652353+0000","last_undegraded":"2023-10-16T09:59:59.652353+0000","last_fullsized":"2023-10-16T09:59:59.652353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:33.244429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.647450+0000","last_change":"2023-10-16T09:59:58.646272+0000","last_active":"2023-10-16T09:59:59.647450+0000","last_peered":"2023-10-16T09:59:59.647450+0000","last_clean":"2023-10-16T09:59:59.647450+0000","last_became_active":"2023-10-16T09:59:58.645928+0000","last_became_peered":"2023-10-16T09:59:58.645928+0000","last_unstale":"2023-10-16T09:59:59.647450+0000","last_undegraded":"2023-10-16T09:59:59.647450+0000","last_fullsized":"2023-10-16T09:59:59.647450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:29.156017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661515+0000","last_change":"2023-10-16T09:59:59.656201+0000","last_active":"2023-10-16T09:59:59.661515+0000","last_peered":"2023-10-16T09:59:59.661515+0000","last_clean":"2023-10-16T09:59:59.661515+0000","last_became_active":"2023-10-16T09:59:59.655914+0000","last_became_peered":"2023-10-16T09:59:59.655914+0000","last_unstale":"2023-10-16T09:59:59.661515+0000","last_undegraded":"2023-10-16T09:59:59.661515+0000","last_fullsized":"2023-10-16T09:59:59.661515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:22.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.641089+0000","last_change":"2023-10-16T09:59:58.654988+0000","last_active":"2023-10-16T09:59:59.641089+0000","last_peered":"2023-10-16T09:59:59.641089+0000","last_clean":"2023-10-16T09:59:59.641089+0000","last_became_active":"2023-10-16T09:59:58.654236+0000","last_became_peered":"2023-10-16T09:59:58.654236+0000","last_unstale":"2023-10-16T09:59:59.641089+0000","last_undegraded":"2023-10-16T09:59:59.641089+0000","last_fullsized":"2023-10-16T09:59:59.641089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:36.040942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.656020+0000","last_change":"2023-10-16T09:59:59.653532+0000","last_active":"2023-10-16T09:59:59.656020+0000","last_peered":"2023-10-16T09:59:59.656020+0000","last_clean":"2023-10-16T09:59:59.656020+0000","last_became_active":"2023-10-16T09:59:59.653230+0000","last_became_peered":"2023-10-16T09:59:59.653230+0000","last_unstale":"2023-10-16T09:59:59.656020+0000","last_undegraded":"2023-10-16T09:59:59.656020+0000","last_fullsized":"2023-10-16T09:59:59.656020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:22.802365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.644711+0000","last_change":"2023-10-16T09:59:58.645543+0000","last_active":"2023-10-16T09:59:59.644711+0000","last_peered":"2023-10-16T09:59:59.644711+0000","last_clean":"2023-10-16T09:59:59.644711+0000","last_became_active":"2023-10-16T09:59:58.645261+0000","last_became_peered":"2023-10-16T09:59:58.645261+0000","last_unstale":"2023-10-16T09:59:59.644711+0000","last_undegraded":"2023-10-16T09:59:59.644711+0000","last_fullsized":"2023-10-16T09:59:59.644711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:00.997205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664364+0000","last_change":"2023-10-16T09:59:59.662495+0000","last_active":"2023-10-16T09:59:59.664364+0000","last_peered":"2023-10-16T09:59:59.664364+0000","last_clean":"2023-10-16T09:59:59.664364+0000","last_became_active":"2023-10-16T09:59:59.662143+0000","last_became_peered":"2023-10-16T09:59:59.662143+0000","last_unstale":"2023-10-16T09:59:59.664364+0000","last_undegraded":"2023-10-16T09:59:59.664364+0000","last_fullsized":"2023-10-16T09:59:59.664364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:57.910605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.648412+0000","last_change":"2023-10-16T09:59:58.657403+0000","last_active":"2023-10-16T09:59:59.648412+0000","last_peered":"2023-10-16T09:59:59.648412+0000","last_clean":"2023-10-16T09:59:59.648412+0000","last_became_active":"2023-10-16T09:59:58.655954+0000","last_became_peered":"2023-10-16T09:59:58.655954+0000","last_unstale":"2023-10-16T09:59:59.648412+0000","last_undegraded":"2023-10-16T09:59:59.648412+0000","last_fullsized":"2023-10-16T09:59:59.648412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:32.238252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660551+0000","last_change":"2023-10-16T09:59:59.656424+0000","last_active":"2023-10-16T09:59:59.660551+0000","last_peered":"2023-10-16T09:59:59.660551+0000","last_clean":"2023-10-16T09:59:59.660551+0000","last_became_active":"2023-10-16T09:59:59.656134+0000","last_became_peered":"2023-10-16T09:59:59.656134+0000","last_unstale":"2023-10-16T09:59:59.660551+0000","last_undegraded":"2023-10-16T09:59:59.660551+0000","last_fullsized":"2023-10-16T09:59:59.660551+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:29.805756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"89'3","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977979+0000","last_change":"2023-10-16T09:59:58.649178+0000","last_active":"2023-10-16T09:59:59.977979+0000","last_peered":"2023-10-16T09:59:59.977979+0000","last_clean":"2023-10-16T09:59:59.977979+0000","last_became_active":"2023-10-16T09:59:58.648874+0000","last_became_peered":"2023-10-16T09:59:58.648874+0000","last_unstale":"2023-10-16T09:59:59.977979+0000","last_undegraded":"2023-10-16T09:59:59.977979+0000","last_fullsized":"2023-10-16T09:59:59.977979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:03.670419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666785+0000","last_change":"2023-10-16T09:59:59.663731+0000","last_active":"2023-10-16T09:59:59.666785+0000","last_peered":"2023-10-16T09:59:59.666785+0000","last_clean":"2023-10-16T09:59:59.666785+0000","last_became_active":"2023-10-16T09:59:59.663459+0000","last_became_peered":"2023-10-16T09:59:59.663459+0000","last_unstale":"2023-10-16T09:59:59.666785+0000","last_undegraded":"2023-10-16T09:59:59.666785+0000","last_fullsized":"2023-10-16T09:59:59.666785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:55.428380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"89'6","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.982948+0000","last_change":"2023-10-16T09:59:58.656158+0000","last_active":"2023-10-16T09:59:59.982948+0000","last_peered":"2023-10-16T09:59:59.982948+0000","last_clean":"2023-10-16T09:59:59.982948+0000","last_became_active":"2023-10-16T09:59:58.655872+0000","last_became_peered":"2023-10-16T09:59:58.655872+0000","last_unstale":"2023-10-16T09:59:59.982948+0000","last_undegraded":"2023-10-16T09:59:59.982948+0000","last_fullsized":"2023-10-16T09:59:59.982948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:12:47.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.657582+0000","last_change":"2023-10-16T09:59:59.653586+0000","last_active":"2023-10-16T09:59:59.657582+0000","last_peered":"2023-10-16T09:59:59.657582+0000","last_clean":"2023-10-16T09:59:59.657582+0000","last_became_active":"2023-10-16T09:59:59.653285+0000","last_became_peered":"2023-10-16T09:59:59.653285+0000","last_unstale":"2023-10-16T09:59:59.657582+0000","last_undegraded":"2023-10-16T09:59:59.657582+0000","last_fullsized":"2023-10-16T09:59:59.657582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:04.410160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.978327+0000","last_change":"2023-10-16T09:59:58.654580+0000","last_active":"2023-10-16T09:59:59.978327+0000","last_peered":"2023-10-16T09:59:59.978327+0000","last_clean":"2023-10-16T09:59:59.978327+0000","last_became_active":"2023-10-16T09:59:58.654291+0000","last_became_peered":"2023-10-16T09:59:58.654291+0000","last_unstale":"2023-10-16T09:59:59.978327+0000","last_undegraded":"2023-10-16T09:59:59.978327+0000","last_fullsized":"2023-10-16T09:59:59.978327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:13.412420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.654275+0000","last_change":"2023-10-16T09:59:59.645339+0000","last_active":"2023-10-16T09:59:59.654275+0000","last_peered":"2023-10-16T09:59:59.654275+0000","last_clean":"2023-10-16T09:59:59.654275+0000","last_became_active":"2023-10-16T09:59:59.644971+0000","last_became_peered":"2023-10-16T09:59:59.644971+0000","last_unstale":"2023-10-16T09:59:59.654275+0000","last_undegraded":"2023-10-16T09:59:59.654275+0000","last_fullsized":"2023-10-16T09:59:59.654275+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:19.748676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.646919+0000","last_change":"2023-10-16T09:59:58.659186+0000","last_active":"2023-10-16T09:59:59.646919+0000","last_peered":"2023-10-16T09:59:59.646919+0000","last_clean":"2023-10-16T09:59:59.646919+0000","last_became_active":"2023-10-16T09:59:58.658485+0000","last_became_peered":"2023-10-16T09:59:58.658485+0000","last_unstale":"2023-10-16T09:59:59.646919+0000","last_undegraded":"2023-10-16T09:59:59.646919+0000","last_fullsized":"2023-10-16T09:59:59.646919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:07:16.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661509+0000","last_change":"2023-10-16T09:59:59.659210+0000","last_active":"2023-10-16T09:59:59.661509+0000","last_peered":"2023-10-16T09:59:59.661509+0000","last_clean":"2023-10-16T09:59:59.661509+0000","last_became_active":"2023-10-16T09:59:59.658911+0000","last_became_peered":"2023-10-16T09:59:59.658911+0000","last_unstale":"2023-10-16T09:59:59.661509+0000","last_undegraded":"2023-10-16T09:59:59.661509+0000","last_fullsized":"2023-10-16T09:59:59.661509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:03.884054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.640643+0000","last_change":"2023-10-16T09:59:58.651162+0000","last_active":"2023-10-16T09:59:59.640643+0000","last_peered":"2023-10-16T09:59:59.640643+0000","last_clean":"2023-10-16T09:59:59.640643+0000","last_became_active":"2023-10-16T09:59:58.650848+0000","last_became_peered":"2023-10-16T09:59:58.650848+0000","last_unstale":"2023-10-16T09:59:59.640643+0000","last_undegraded":"2023-10-16T09:59:59.640643+0000","last_fullsized":"2023-10-16T09:59:59.640643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:39.765289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666831+0000","last_change":"2023-10-16T09:59:59.664040+0000","last_active":"2023-10-16T09:59:59.666831+0000","last_peered":"2023-10-16T09:59:59.666831+0000","last_clean":"2023-10-16T09:59:59.666831+0000","last_became_active":"2023-10-16T09:59:59.663757+0000","last_became_peered":"2023-10-16T09:59:59.663757+0000","last_unstale":"2023-10-16T09:59:59.666831+0000","last_undegraded":"2023-10-16T09:59:59.666831+0000","last_fullsized":"2023-10-16T09:59:59.666831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:14.609358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.979185+0000","last_change":"2023-10-16T09:59:58.654459+0000","last_active":"2023-10-16T09:59:59.979185+0000","last_peered":"2023-10-16T09:59:59.979185+0000","last_clean":"2023-10-16T09:59:59.979185+0000","last_became_active":"2023-10-16T09:59:58.654077+0000","last_became_peered":"2023-10-16T09:59:58.654077+0000","last_unstale":"2023-10-16T09:59:59.979185+0000","last_undegraded":"2023-10-16T09:59:59.979185+0000","last_fullsized":"2023-10-16T09:59:59.979185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:26.041963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662892+0000","last_change":"2023-10-16T09:59:59.661026+0000","last_active":"2023-10-16T09:59:59.662892+0000","last_peered":"2023-10-16T09:59:59.662892+0000","last_clean":"2023-10-16T09:59:59.662892+0000","last_became_active":"2023-10-16T09:59:59.660749+0000","last_became_peered":"2023-10-16T09:59:59.660749+0000","last_unstale":"2023-10-16T09:59:59.662892+0000","last_undegraded":"2023-10-16T09:59:59.662892+0000","last_fullsized":"2023-10-16T09:59:59.662892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:27.116852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977471+0000","last_change":"2023-10-16T09:59:58.658066+0000","last_active":"2023-10-16T09:59:59.977471+0000","last_peered":"2023-10-16T09:59:59.977471+0000","last_clean":"2023-10-16T09:59:59.977471+0000","last_became_active":"2023-10-16T09:59:58.657777+0000","last_became_peered":"2023-10-16T09:59:58.657777+0000","last_unstale":"2023-10-16T09:59:59.977471+0000","last_undegraded":"2023-10-16T09:59:59.977471+0000","last_fullsized":"2023-10-16T09:59:59.977471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:03.427458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664404+0000","last_change":"2023-10-16T09:59:59.662805+0000","last_active":"2023-10-16T09:59:59.664404+0000","last_peered":"2023-10-16T09:59:59.664404+0000","last_clean":"2023-10-16T09:59:59.664404+0000","last_became_active":"2023-10-16T09:59:59.662533+0000","last_became_peered":"2023-10-16T09:59:59.662533+0000","last_unstale":"2023-10-16T09:59:59.664404+0000","last_undegraded":"2023-10-16T09:59:59.664404+0000","last_fullsized":"2023-10-16T09:59:59.664404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:31:59.669734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977339+0000","last_change":"2023-10-16T09:59:58.658549+0000","last_active":"2023-10-16T09:59:59.977339+0000","last_peered":"2023-10-16T09:59:59.977339+0000","last_clean":"2023-10-16T09:59:59.977339+0000","last_became_active":"2023-10-16T09:59:58.658267+0000","last_became_peered":"2023-10-16T09:59:58.658267+0000","last_unstale":"2023-10-16T09:59:59.977339+0000","last_undegraded":"2023-10-16T09:59:59.977339+0000","last_fullsized":"2023-10-16T09:59:59.977339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:16.407745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662999+0000","last_change":"2023-10-16T09:59:59.661581+0000","last_active":"2023-10-16T09:59:59.662999+0000","last_peered":"2023-10-16T09:59:59.662999+0000","last_clean":"2023-10-16T09:59:59.662999+0000","last_became_active":"2023-10-16T09:59:59.660802+0000","last_became_peered":"2023-10-16T09:59:59.660802+0000","last_unstale":"2023-10-16T09:59:59.662999+0000","last_undegraded":"2023-10-16T09:59:59.662999+0000","last_fullsized":"2023-10-16T09:59:59.662999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:20.291936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.979124+0000","last_change":"2023-10-16T09:59:58.660442+0000","last_active":"2023-10-16T09:59:59.979124+0000","last_peered":"2023-10-16T09:59:59.979124+0000","last_clean":"2023-10-16T09:59:59.979124+0000","last_became_active":"2023-10-16T09:59:58.660191+0000","last_became_peered":"2023-10-16T09:59:58.660191+0000","last_unstale":"2023-10-16T09:59:59.979124+0000","last_undegraded":"2023-10-16T09:59:59.979124+0000","last_fullsized":"2023-10-16T09:59:59.979124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:08.685915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.651081+0000","last_change":"2023-10-16T09:59:59.644870+0000","last_active":"2023-10-16T09:59:59.651081+0000","last_peered":"2023-10-16T09:59:59.651081+0000","last_clean":"2023-10-16T09:59:59.651081+0000","last_became_active":"2023-10-16T09:59:59.644642+0000","last_became_peered":"2023-10-16T09:59:59.644642+0000","last_unstale":"2023-10-16T09:59:59.651081+0000","last_undegraded":"2023-10-16T09:59:59.651081+0000","last_fullsized":"2023-10-16T09:59:59.651081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:09.158237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.648824+0000","last_change":"2023-10-16T09:59:58.655638+0000","last_active":"2023-10-16T09:59:59.648824+0000","last_peered":"2023-10-16T09:59:59.648824+0000","last_clean":"2023-10-16T09:59:59.648824+0000","last_became_active":"2023-10-16T09:59:58.655320+0000","last_became_peered":"2023-10-16T09:59:58.655320+0000","last_unstale":"2023-10-16T09:59:59.648824+0000","last_undegraded":"2023-10-16T09:59:59.648824+0000","last_fullsized":"2023-10-16T09:59:59.648824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:30.303572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.667044+0000","last_change":"2023-10-16T09:59:59.664639+0000","last_active":"2023-10-16T09:59:59.667044+0000","last_peered":"2023-10-16T09:59:59.667044+0000","last_clean":"2023-10-16T09:59:59.667044+0000","last_became_active":"2023-10-16T09:59:59.661787+0000","last_became_peered":"2023-10-16T09:59:59.661787+0000","last_unstale":"2023-10-16T09:59:59.667044+0000","last_undegraded":"2023-10-16T09:59:59.667044+0000","last_fullsized":"2023-10-16T09:59:59.667044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:42.134799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.647327+0000","last_change":"2023-10-16T09:59:58.658728+0000","last_active":"2023-10-16T09:59:59.647327+0000","last_peered":"2023-10-16T09:59:59.647327+0000","last_clean":"2023-10-16T09:59:59.647327+0000","last_became_active":"2023-10-16T09:59:58.657700+0000","last_became_peered":"2023-10-16T09:59:58.657700+0000","last_unstale":"2023-10-16T09:59:59.647327+0000","last_undegraded":"2023-10-16T09:59:59.647327+0000","last_fullsized":"2023-10-16T09:59:59.647327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:16.612088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666609+0000","last_change":"2023-10-16T09:59:59.664049+0000","last_active":"2023-10-16T09:59:59.666609+0000","last_peered":"2023-10-16T09:59:59.666609+0000","last_clean":"2023-10-16T09:59:59.666609+0000","last_became_active":"2023-10-16T09:59:59.661487+0000","last_became_peered":"2023-10-16T09:59:59.661487+0000","last_unstale":"2023-10-16T09:59:59.666609+0000","last_undegraded":"2023-10-16T09:59:59.666609+0000","last_fullsized":"2023-10-16T09:59:59.666609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:36.927647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.640608+0000","last_change":"2023-10-16T09:59:58.656371+0000","last_active":"2023-10-16T09:59:59.640608+0000","last_peered":"2023-10-16T09:59:59.640608+0000","last_clean":"2023-10-16T09:59:59.640608+0000","last_became_active":"2023-10-16T09:59:58.655886+0000","last_became_peered":"2023-10-16T09:59:58.655886+0000","last_unstale":"2023-10-16T09:59:59.640608+0000","last_undegraded":"2023-10-16T09:59:59.640608+0000","last_fullsized":"2023-10-16T09:59:59.640608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:23.052157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664930+0000","last_change":"2023-10-16T09:59:59.661716+0000","last_active":"2023-10-16T09:59:59.664930+0000","last_peered":"2023-10-16T09:59:59.664930+0000","last_clean":"2023-10-16T09:59:59.664930+0000","last_became_active":"2023-10-16T09:59:59.661187+0000","last_became_peered":"2023-10-16T09:59:59.661187+0000","last_unstale":"2023-10-16T09:59:59.664930+0000","last_undegraded":"2023-10-16T09:59:59.664930+0000","last_fullsized":"2023-10-16T09:59:59.664930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:51.489231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.650043+0000","last_change":"2023-10-16T09:59:58.653886+0000","last_active":"2023-10-16T09:59:59.650043+0000","last_peered":"2023-10-16T09:59:59.650043+0000","last_clean":"2023-10-16T09:59:59.650043+0000","last_became_active":"2023-10-16T09:59:58.652708+0000","last_became_peered":"2023-10-16T09:59:58.652708+0000","last_unstale":"2023-10-16T09:59:59.650043+0000","last_undegraded":"2023-10-16T09:59:59.650043+0000","last_fullsized":"2023-10-16T09:59:59.650043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:04:50.890542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662078+0000","last_change":"2023-10-16T09:59:59.653387+0000","last_active":"2023-10-16T09:59:59.662078+0000","last_peered":"2023-10-16T09:59:59.662078+0000","last_clean":"2023-10-16T09:59:59.662078+0000","last_became_active":"2023-10-16T09:59:59.653103+0000","last_became_peered":"2023-10-16T09:59:59.653103+0000","last_unstale":"2023-10-16T09:59:59.662078+0000","last_undegraded":"2023-10-16T09:59:59.662078+0000","last_fullsized":"2023-10-16T09:59:59.662078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:04:44.300764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977270+0000","last_change":"2023-10-16T09:59:58.657889+0000","last_active":"2023-10-16T09:59:59.977270+0000","last_peered":"2023-10-16T09:59:59.977270+0000","last_clean":"2023-10-16T09:59:59.977270+0000","last_became_active":"2023-10-16T09:59:58.657624+0000","last_became_peered":"2023-10-16T09:59:58.657624+0000","last_unstale":"2023-10-16T09:59:59.977270+0000","last_undegraded":"2023-10-16T09:59:59.977270+0000","last_fullsized":"2023-10-16T09:59:59.977270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:38.319429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662778+0000","last_change":"2023-10-16T09:59:59.660572+0000","last_active":"2023-10-16T09:59:59.662778+0000","last_peered":"2023-10-16T09:59:59.662778+0000","last_clean":"2023-10-16T09:59:59.662778+0000","last_became_active":"2023-10-16T09:59:59.660300+0000","last_became_peered":"2023-10-16T09:59:59.660300+0000","last_unstale":"2023-10-16T09:59:59.662778+0000","last_undegraded":"2023-10-16T09:59:59.662778+0000","last_fullsized":"2023-10-16T09:59:59.662778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:45.865340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.641948+0000","last_change":"2023-10-16T09:59:58.650822+0000","last_active":"2023-10-16T09:59:59.641948+0000","last_peered":"2023-10-16T09:59:59.641948+0000","last_clean":"2023-10-16T09:59:59.641948+0000","last_became_active":"2023-10-16T09:59:58.650516+0000","last_became_peered":"2023-10-16T09:59:58.650516+0000","last_unstale":"2023-10-16T09:59:59.641948+0000","last_undegraded":"2023-10-16T09:59:59.641948+0000","last_fullsized":"2023-10-16T09:59:59.641948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:04.816771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664569+0000","last_change":"2023-10-16T09:59:59.663350+0000","last_active":"2023-10-16T09:59:59.664569+0000","last_peered":"2023-10-16T09:59:59.664569+0000","last_clean":"2023-10-16T09:59:59.664569+0000","last_became_active":"2023-10-16T09:59:59.663077+0000","last_became_peered":"2023-10-16T09:59:59.663077+0000","last_unstale":"2023-10-16T09:59:59.664569+0000","last_undegraded":"2023-10-16T09:59:59.664569+0000","last_fullsized":"2023-10-16T09:59:59.664569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:12.329218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.641931+0000","last_change":"2023-10-16T09:59:58.657618+0000","last_active":"2023-10-16T09:59:59.641931+0000","last_peered":"2023-10-16T09:59:59.641931+0000","last_clean":"2023-10-16T09:59:59.641931+0000","last_became_active":"2023-10-16T09:59:58.657319+0000","last_became_peered":"2023-10-16T09:59:58.657319+0000","last_unstale":"2023-10-16T09:59:59.641931+0000","last_undegraded":"2023-10-16T09:59:59.641931+0000","last_fullsized":"2023-10-16T09:59:59.641931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:18.724273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.659637+0000","last_change":"2023-10-16T09:59:59.658842+0000","last_active":"2023-10-16T09:59:59.659637+0000","last_peered":"2023-10-16T09:59:59.659637+0000","last_clean":"2023-10-16T09:59:59.659637+0000","last_became_active":"2023-10-16T09:59:59.658311+0000","last_became_peered":"2023-10-16T09:59:59.658311+0000","last_unstale":"2023-10-16T09:59:59.659637+0000","last_undegraded":"2023-10-16T09:59:59.659637+0000","last_fullsized":"2023-10-16T09:59:59.659637+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:47.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.650172+0000","last_change":"2023-10-16T09:59:58.653705+0000","last_active":"2023-10-16T09:59:59.650172+0000","last_peered":"2023-10-16T09:59:59.650172+0000","last_clean":"2023-10-16T09:59:59.650172+0000","last_became_active":"2023-10-16T09:59:58.653469+0000","last_became_peered":"2023-10-16T09:59:58.653469+0000","last_unstale":"2023-10-16T09:59:59.650172+0000","last_undegraded":"2023-10-16T09:59:59.650172+0000","last_fullsized":"2023-10-16T09:59:59.650172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:37.607585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666862+0000","last_change":"2023-10-16T09:59:59.666154+0000","last_active":"2023-10-16T09:59:59.666862+0000","last_peered":"2023-10-16T09:59:59.666862+0000","last_clean":"2023-10-16T09:59:59.666862+0000","last_became_active":"2023-10-16T09:59:59.665789+0000","last_became_peered":"2023-10-16T09:59:59.665789+0000","last_unstale":"2023-10-16T09:59:59.666862+0000","last_undegraded":"2023-10-16T09:59:59.666862+0000","last_fullsized":"2023-10-16T09:59:59.666862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:39.325606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.640895+0000","last_change":"2023-10-16T09:59:58.654107+0000","last_active":"2023-10-16T09:59:59.640895+0000","last_peered":"2023-10-16T09:59:59.640895+0000","last_clean":"2023-10-16T09:59:59.640895+0000","last_became_active":"2023-10-16T09:59:58.653530+0000","last_became_peered":"2023-10-16T09:59:58.653530+0000","last_unstale":"2023-10-16T09:59:59.640895+0000","last_undegraded":"2023-10-16T09:59:59.640895+0000","last_fullsized":"2023-10-16T09:59:59.640895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:45.499639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661133+0000","last_change":"2023-10-16T09:59:59.659845+0000","last_active":"2023-10-16T09:59:59.661133+0000","last_peered":"2023-10-16T09:59:59.661133+0000","last_clean":"2023-10-16T09:59:59.661133+0000","last_became_active":"2023-10-16T09:59:59.659498+0000","last_became_peered":"2023-10-16T09:59:59.659498+0000","last_unstale":"2023-10-16T09:59:59.661133+0000","last_undegraded":"2023-10-16T09:59:59.661133+0000","last_fullsized":"2023-10-16T09:59:59.661133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:09.158237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.647628+0000","last_change":"2023-10-16T09:59:58.646289+0000","last_active":"2023-10-16T09:59:59.647628+0000","last_peered":"2023-10-16T09:59:59.647628+0000","last_clean":"2023-10-16T09:59:59.647628+0000","last_became_active":"2023-10-16T09:59:58.645543+0000","last_became_peered":"2023-10-16T09:59:58.645543+0000","last_unstale":"2023-10-16T09:59:59.647628+0000","last_undegraded":"2023-10-16T09:59:59.647628+0000","last_fullsized":"2023-10-16T09:59:59.647628+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:25.947204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665024+0000","last_change":"2023-10-16T09:59:59.662410+0000","last_active":"2023-10-16T09:59:59.665024+0000","last_peered":"2023-10-16T09:59:59.665024+0000","last_clean":"2023-10-16T09:59:59.665024+0000","last_became_active":"2023-10-16T09:59:59.662202+0000","last_became_peered":"2023-10-16T09:59:59.662202+0000","last_unstale":"2023-10-16T09:59:59.665024+0000","last_undegraded":"2023-10-16T09:59:59.665024+0000","last_fullsized":"2023-10-16T09:59:59.665024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:36:33.834448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.642967+0000","last_change":"2023-10-16T09:59:58.654363+0000","last_active":"2023-10-16T09:59:59.642967+0000","last_peered":"2023-10-16T09:59:59.642967+0000","last_clean":"2023-10-16T09:59:59.642967+0000","last_became_active":"2023-10-16T09:59:58.654084+0000","last_became_peered":"2023-10-16T09:59:58.654084+0000","last_unstale":"2023-10-16T09:59:59.642967+0000","last_undegraded":"2023-10-16T09:59:59.642967+0000","last_fullsized":"2023-10-16T09:59:59.642967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:21.709240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665334+0000","last_change":"2023-10-16T09:59:59.663161+0000","last_active":"2023-10-16T09:59:59.665334+0000","last_peered":"2023-10-16T09:59:59.665334+0000","last_clean":"2023-10-16T09:59:59.665334+0000","last_became_active":"2023-10-16T09:59:59.662808+0000","last_became_peered":"2023-10-16T09:59:59.662808+0000","last_unstale":"2023-10-16T09:59:59.665334+0000","last_undegraded":"2023-10-16T09:59:59.665334+0000","last_fullsized":"2023-10-16T09:59:59.665334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:41:40.433385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.649493+0000","last_change":"2023-10-16T09:59:58.657388+0000","last_active":"2023-10-16T09:59:59.649493+0000","last_peered":"2023-10-16T09:59:59.649493+0000","last_clean":"2023-10-16T09:59:59.649493+0000","last_became_active":"2023-10-16T09:59:58.655857+0000","last_became_peered":"2023-10-16T09:59:58.655857+0000","last_unstale":"2023-10-16T09:59:59.649493+0000","last_undegraded":"2023-10-16T09:59:59.649493+0000","last_fullsized":"2023-10-16T09:59:59.649493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:45.499639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.667061+0000","last_change":"2023-10-16T09:59:59.664725+0000","last_active":"2023-10-16T09:59:59.667061+0000","last_peered":"2023-10-16T09:59:59.667061+0000","last_clean":"2023-10-16T09:59:59.667061+0000","last_became_active":"2023-10-16T09:59:59.660941+0000","last_became_peered":"2023-10-16T09:59:59.660941+0000","last_unstale":"2023-10-16T09:59:59.667061+0000","last_undegraded":"2023-10-16T09:59:59.667061+0000","last_fullsized":"2023-10-16T09:59:59.667061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:07.563412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.976966+0000","last_change":"2023-10-16T09:59:58.658436+0000","last_active":"2023-10-16T09:59:59.976966+0000","last_peered":"2023-10-16T09:59:59.976966+0000","last_clean":"2023-10-16T09:59:59.976966+0000","last_became_active":"2023-10-16T09:59:58.658173+0000","last_became_peered":"2023-10-16T09:59:58.658173+0000","last_unstale":"2023-10-16T09:59:59.976966+0000","last_undegraded":"2023-10-16T09:59:59.976966+0000","last_fullsized":"2023-10-16T09:59:59.976966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:16.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"79'172","reported_seq":428,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T09:59:58.649970+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:59:58.649970+0000","last_peered":"2023-10-16T09:59:58.649970+0000","last_clean":"2023-10-16T09:59:58.649970+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:59:58.649970+0000","last_undegraded":"2023-10-16T09:59:58.649970+0000","last_fullsized":"2023-10-16T09:59:58.649970+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542338,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":1115757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.25700000000000001},"max":{"1min":0.72899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.80400000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.81200000000000006},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.34399999999999997},"max":{"1min":0.755,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65200000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76200000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.499}]},{"osd":5,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.318,"15min":0.312},"max":{"1min":0.70299999999999996,"5min":1.006,"15min":1.006},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":1.0169999999999999,"15min":1.0489999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68999999999999995,"5min":0.95199999999999996,"15min":1.373},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.85999999999999999,"15min":1.4099999999999999},"last":0.57399999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542337,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":524865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.26100000000000001},"max":{"1min":0.72299999999999998,"5min":0.84899999999999998,"15min":6.194},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":6.1609999999999996},"last":0.58499999999999996}]},{"osd":1,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.28799999999999998},"max":{"1min":0.69299999999999995,"5min":0.78900000000000003,"15min":1.054},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.44400000000000001,"5min":0.39000000000000001,"15min":0.32400000000000001},"max":{"1min":0.72199999999999998,"5min":0.80600000000000005,"15min":0.876},"last":0.56499999999999995}]},{"osd":2,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.61099999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":8.5600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70199999999999996,"5min":0.83699999999999997,"15min":1.1060000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36199999999999999},"max":{"1min":0.69899999999999995,"5min":0.82399999999999995,"15min":1.175},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.624},"min":{"1min":0.40200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79100000000000004,"5min":0.85399999999999998,"15min":6.2249999999999996},"last":0.622}]},{"osd":4,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.499,"15min":0.495},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.30299999999999999},"max":{"1min":0.72399999999999998,"5min":0.74399999999999999,"15min":0.81299999999999994},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.377,"5min":0.33400000000000002,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84699999999999998},"last":0.40999999999999998}]},{"osd":5,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77500000000000002,"5min":0.78800000000000003,"15min":1.1240000000000001},"last":0.435}]},{"osd":7,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.31900000000000001},"max":{"1min":0.754,"5min":0.83499999999999996,"15min":0.879},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542337,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50312,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321528,"statfs":{"total":96636764160,"available":96585244672,"internally_reserved":0,"allocated":1101824,"data_stored":527001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.375,"15min":0.37},"min":{"1min":0.22500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54400000000000004,"5min":1.5,"15min":1.5},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.245,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.60099999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.33600000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55600000000000005,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.44},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.67300000000000004,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.376}]},{"osd":3,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.495,"15min":0.47199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68000000000000005,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.443,"15min":0.436},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.65100000000000002,"5min":1.379,"15min":1.379},"last":0.29399999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85899999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.49399999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":1.26,"15min":1.26},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78700000000000003,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69699999999999995,"15min":0.67000000000000004},"min":{"1min":0.44500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92100000000000004,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.50600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92600000000000005,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.36799999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.443},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.47799999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542338,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":1115141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.373,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.19900000000000001,"15min":0.191},"max":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":5.5090000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40000000000000002,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.23100000000000001},"max":{"1min":0.49199999999999999,"5min":0.64900000000000002,"15min":6.4950000000000001},"last":0.23499999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.40799999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51100000000000001,"5min":0.61799999999999999,"15min":0.69599999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42399999999999999,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57999999999999996,"5min":0.624,"15min":2.7559999999999998},"last":0.31900000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42499999999999999,"15min":0.46300000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.20699999999999999},"max":{"1min":0.55100000000000005,"5min":0.72999999999999998,"15min":8.3260000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47699999999999998,"15min":0.501},"min":{"1min":0.309,"5min":0.309,"15min":0.30099999999999999},"max":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":5.4809999999999999},"last":0.33800000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.624,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":0.88700000000000001,"15min":6.4349999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63100000000000001,"15min":0.66400000000000003},"min":{"1min":0.434,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":0.98799999999999999,"15min":8.2970000000000006},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64200000000000002,"15min":0.67000000000000004},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":5.4160000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64400000000000002,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78000000000000003,"5min":0.96999999999999997,"15min":6.3869999999999996},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.32900000000000001},"max":{"1min":0.75900000000000001,"5min":0.92500000000000004,"15min":8.2189999999999994},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.85699999999999998,"5min":1.0089999999999999,"15min":5.4560000000000004},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66200000000000003,"15min":0.69999999999999996},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":1.0249999999999999,"15min":6.4649999999999999},"last":0.63},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.47399999999999998,"5min":0.38700000000000001,"15min":0.34899999999999998},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":8.266},"last":0.53700000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542337,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1093632,"data_stored":526385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.45300000000000001,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.39800000000000002,"15min":0.39900000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.621,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42599999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.24299999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.214},"max":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.64000000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40699999999999997,"15min":0.40799999999999997},"min":{"1min":0.315,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.5,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.63100000000000001,"5min":0.747,"15min":0.747},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91600000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.625,"15min":0.626},"min":{"1min":0.41999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73799999999999999,"5min":1.091,"15min":1.091},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.94099999999999995,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.90600000000000003,"15min":0.97499999999999998},"last":0.57199999999999995}]},{"osd":6,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.93799999999999994,"15min":0.97899999999999998},"last":0.55700000000000005}]},{"osd":7,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90000000000000002,"5min":0.98299999999999998,"15min":0.98799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77400000000000002,"5min":0.874,"15min":0.998},"last":0.63800000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542338,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335436,"statfs":{"total":96636764160,"available":96599486464,"internally_reserved":0,"allocated":1089536,"data_stored":524885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38400000000000001},"min":{"1min":0.23000000000000001,"5min":0.215,"15min":0.192},"max":{"1min":0.58099999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26900000000000002,"5min":0.185,"15min":0.185},"max":{"1min":0.626,"5min":0.63100000000000001,"15min":1.0780000000000001},"last":0.32500000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.39800000000000002},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.72199999999999998,"15min":2.254},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.246},"max":{"1min":0.69799999999999995,"5min":0.754,"15min":6.2240000000000002},"last":0.28199999999999997}]},{"osd":2,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.25},"max":{"1min":0.60599999999999998,"5min":0.70999999999999996,"15min":2.2909999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.72299999999999998},"last":0.315}]},{"osd":4,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.88700000000000001,"15min":1.24},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.314,"5min":0.311,"15min":0.311},"max":{"1min":0.75600000000000001,"5min":1.0820000000000001,"15min":2.4420000000000002},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.627},"min":{"1min":0.374,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79200000000000004,"5min":0.97599999999999998,"15min":2.4089999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.38600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.996},"last":0.54400000000000004}]},{"osd":6,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63700000000000001,"15min":0.64200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.89100000000000001,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.99099999999999999,"15min":1.1619999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86199999999999999,"5min":1.3,"15min":1.3},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":1.4019999999999999,"15min":2.3740000000000001},"last":0.53400000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542337,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54448,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317392,"statfs":{"total":96636764160,"available":96581009408,"internally_reserved":0,"allocated":1085440,"data_stored":524843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.73699999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58499999999999996},"min":{"1min":0.32900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73699999999999999,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.35599999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72799999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.57499999999999996},"min":{"1min":0.39900000000000002,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.69999999999999996,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72799999999999998,"5min":0.81999999999999995,"15min":0.89100000000000001},"last":0.65800000000000003}]},{"osd":3,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.86499999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73199999999999998,"5min":0.76800000000000002,"15min":0.81000000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.51000000000000001,"15min":0.52000000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71499999999999997,"5min":0.76400000000000001,"15min":0.86899999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65600000000000003,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.76500000000000001,"5min":0.85899999999999999,"15min":0.85999999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":0.80700000000000005,"15min":0.84599999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84599999999999997,"15min":0.95999999999999996},"last":0.53300000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542338,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1069056,"data_stored":517574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40600000000000003,"5min":0.33600000000000002,"15min":0.253},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.92900000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.629},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":0.84199999999999997,"15min":8.0370000000000008},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.98599999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.307},"max":{"1min":0.83499999999999996,"5min":0.92300000000000004,"15min":8.2110000000000003},"last":0.57699999999999996}]},{"osd":2,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.315,"15min":0.29899999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":8.1140000000000008},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.61899999999999999},"min":{"1min":0.379,"5min":0.376,"15min":0.33900000000000002},"max":{"1min":0.79400000000000004,"5min":0.88700000000000001,"15min":8.3000000000000007},"last":0.49299999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.34599999999999997,"15min":0.312},"max":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":1.03},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.64900000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":9.8810000000000002},"last":0.63900000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.496,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.26000000000000001},"max":{"1min":0.69899999999999995,"5min":0.77400000000000002,"15min":1.0449999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":1.0780000000000001},"last":0.625}]},{"osd":6,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.33100000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.78700000000000003,"15min":1.599},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.745,"5min":0.85799999999999998,"15min":1.006},"last":0.60999999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.747,"5min":0.95199999999999996,"15min":1.1339999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":0.90700000000000003,"15min":1.889},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:00:04.088 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:00:04.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.241+0000 7f77dcef8700 1 Processor -- start 2023-10-16T10:00:04.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.241+0000 7f77dcef8700 1 -- start start 2023-10-16T10:00:04.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77dcef8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77d815b990 0x7f77d80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77dcef8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d80cc510 con 0x7f77d80d0370 2023-10-16T10:00:04.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77dcef8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d80cc7f0 con 0x7f77d80cd2d0 2023-10-16T10:00:04.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77dcef8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d80ccad0 con 0x7f77d815b990 2023-10-16T10:00:04.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f77d80d0370 0x7f77d80ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54266/0 (socket says 172.21.15.181:54266) 2023-10-16T10:00:04.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d6d9d700 1 -- 172.21.15.181:0/1747951401 learned_addr learned my addr 172.21.15.181:0/1747951401 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:04.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d659c700 1 --2- 172.21.15.181:0/1747951401 >> v2:172.21.15.190:3300/0 conn(0x7f77d815b990 0x7f77d80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020095436 0 0) 0x7f77d80cc510 con 0x7f77d80d0370 2023-10-16T10:00:04.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c00035b0 con 0x7f77d80d0370 2023-10-16T10:00:04.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 148499173 0 0) 0x7f77c00035b0 con 0x7f77d80d0370 2023-10-16T10:00:04.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 >> v2:172.21.15.190:3300/0 conn(0x7f77d815b990 msgr2=0x7f77d80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:04.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 --2- 172.21.15.181:0/1747951401 >> v2:172.21.15.190:3300/0 conn(0x7f77d815b990 0x7f77d80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 >> v1:172.21.15.190:6789/0 conn(0x7f77d80cd2d0 legacy=0x7f77d80cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80ccdb0 con 0x7f77d80d0370 2023-10-16T10:00:04.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f77d00019f0 con 0x7f77d80d0370 2023-10-16T10:00:04.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 >> v1:172.21.15.181:6789/0 conn(0x7f77d80d0370 legacy=0x7f77d80ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c0004850 0x7f77c0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.242+0000 7f77d559a700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c0005210 0x7f77c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80ccad0 con 0x7f77c0003a00 2023-10-16T10:00:04.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000c0c0 con 0x7f77c0004850 2023-10-16T10:00:04.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000c280 con 0x7f77c0005210 2023-10-16T10:00:04.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d659c700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d5d9b700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c0004850 0x7f77c0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d6d9d700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c0005210 0x7f77c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d659c700 1 -- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c0005210 msgr2=0x7f77c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:04.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d659c700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c0005210 0x7f77c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d659c700 1 -- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c0004850 msgr2=0x7f77c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:04.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d659c700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c0004850 0x7f77c0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d659c700 1 -- 172.21.15.181:0/1747951401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00035b0 con 0x7f77c0003a00 2023-10-16T10:00:04.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d6d9d700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c0005210 0x7f77c000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:04.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d6d9d700 1 -- 172.21.15.181:0/1747951401 reap_dead start 2023-10-16T10:00:04.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d659c700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77c0003e20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009b40 tx=0x7f77c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903881989aa8df0f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:04.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8005a20 con 0x7f77c0003a00 2023-10-16T10:00:04.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8005b80 con 0x7f77c0003a00 2023-10-16T10:00:04.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.243+0000 7f77d559a700 1 -- 172.21.15.181:0/1747951401 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c800e6c0 con 0x7f77c0003a00 2023-10-16T10:00:04.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.244+0000 7f77dcef8700 1 -- 172.21.15.181:0/1747951401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 msgr2=0x7f77c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.244+0000 7f77dcef8700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77c0003e20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009b40 tx=0x7f77c0005880 comp rx=0 tx=0).stop 2023-10-16T10:00:04.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.244+0000 7f77dcef8700 1 -- 172.21.15.181:0/1747951401 shutdown_connections 2023-10-16T10:00:04.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.244+0000 7f77dcef8700 1 --2- 172.21.15.181:0/1747951401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77c0003e20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.244+0000 7f77dcef8700 1 -- 172.21.15.181:0/1747951401 >> 172.21.15.181:0/1747951401 conn(0x7f77d809a9a0 msgr2=0x7f77d80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:04.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.244+0000 7f77dcef8700 1 -- 172.21.15.181:0/1747951401 shutdown_connections 2023-10-16T10:00:04.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.244+0000 7f77dcef8700 1 -- 172.21.15.181:0/1747951401 wait complete. 2023-10-16T10:00:04.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.245+0000 7f77dcef8700 1 Processor -- start 2023-10-16T10:00:04.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.245+0000 7f77dcef8700 1 -- start start 2023-10-16T10:00:04.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77dcef8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77d80b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77dcef8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d81f5650 0x7f77d8159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77dcef8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d80b1b30 0x7f77d80b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77dcef8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d809b480 con 0x7f77c0003a00 2023-10-16T10:00:04.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77dcef8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80b24f0 con 0x7f77d80b1b30 2023-10-16T10:00:04.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77dcef8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80b5370 con 0x7f77d81f5650 2023-10-16T10:00:04.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77d80b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77d80b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43196/0 (socket says 172.21.15.181:43196) 2023-10-16T10:00:04.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 -- 172.21.15.181:0/3818782991 learned_addr learned my addr 172.21.15.181:0/3818782991 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:04.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 -- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d81f5650 msgr2=0x7f77d8159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d81f5650 0x7f77d8159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d6d9d700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d80b1b30 0x7f77d80b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 -- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d80b1b30 msgr2=0x7f77d80b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d80b1b30 0x7f77d80b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 -- 172.21.15.181:0/3818782991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80b4ad0 con 0x7f77c0003a00 2023-10-16T10:00:04.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d6d9d700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d80b1b30 0x7f77d80b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:04.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77d659c700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77d80b42a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000c00 tx=0x7f77c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:04.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.246+0000 7f77c6ffd700 1 -- 172.21.15.181:0/3818782991 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8021070 con 0x7f77c0003a00 2023-10-16T10:00:04.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.247+0000 7f77c6ffd700 1 -- 172.21.15.181:0/3818782991 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8004240 con 0x7f77c0003a00 2023-10-16T10:00:04.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.247+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d80b4db0 con 0x7f77c0003a00 2023-10-16T10:00:04.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.247+0000 7f77c6ffd700 1 -- 172.21.15.181:0/3818782991 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80045d0 con 0x7f77c0003a00 2023-10-16T10:00:04.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.247+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d80c87f0 con 0x7f77c0003a00 2023-10-16T10:00:04.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.247+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d8067d20 con 0x7f77c0003a00 2023-10-16T10:00:04.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.248+0000 7f77c6ffd700 1 -- 172.21.15.181:0/3818782991 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f77c80036a0 con 0x7f77c0003a00 2023-10-16T10:00:04.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.248+0000 7f77c6ffd700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f77b806b020 0x7f77b806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.248+0000 7f77c6ffd700 1 -- 172.21.15.181:0/3818782991 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f77c808aa30 con 0x7f77c0003a00 2023-10-16T10:00:04.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.248+0000 7f77d5d9b700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f77b806b020 0x7f77b806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.249+0000 7f77d5d9b700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f77b806b020 0x7f77b806d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77d80c7af0 tx=0x7f77cc009040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:04.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.253+0000 7f77c6ffd700 1 -- 172.21.15.181:0/3818782991 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f77c80558b0 con 0x7f77c0003a00 2023-10-16T10:00:04.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.443+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 --> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77d809b480 con 0x7f77b806b020 2023-10-16T10:00:04.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.450+0000 7f77c6ffd700 1 -- 172.21.15.181:0/3818782991 <== mgr.6732 v2:172.21.15.181:6832/752673838 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289562 (secure 0 0 0) 0x7f77d809b480 con 0x7f77b806b020 2023-10-16T10:00:04.454 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:04.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f77b806b020 msgr2=0x7f77b806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f77b806b020 0x7f77b806d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77d80c7af0 tx=0x7f77cc009040 comp rx=0 tx=0).stop 2023-10-16T10:00:04.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 msgr2=0x7f77d80b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77d80b42a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000c00 tx=0x7f77c80047c0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 shutdown_connections 2023-10-16T10:00:04.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f77b806b020 0x7f77b806d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77d80b1b30 0x7f77d80b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77c0003a00 0x7f77d80b42a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 --2- 172.21.15.181:0/3818782991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77d81f5650 0x7f77d8159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 >> 172.21.15.181:0/3818782991 conn(0x7f77d809a9a0 msgr2=0x7f77d80cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:04.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 shutdown_connections 2023-10-16T10:00:04.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.454+0000 7f77dcef8700 1 -- 172.21.15.181:0/3818782991 wait complete. 2023-10-16T10:00:04.460 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:00:04.466 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T10:00:02.587385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307708,"kb_used_data":9656,"kb_used_omap":12,"kb_used_meta":297907,"kb_avail":754667012,"statfs":{"total":773094113280,"available":772779020288,"internally_reserved":0,"allocated":9887744,"data_stored":5376451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":305057336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.954850"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664385+0000","last_change":"2023-10-16T09:59:59.662665+0000","last_active":"2023-10-16T09:59:59.664385+0000","last_peered":"2023-10-16T09:59:59.664385+0000","last_clean":"2023-10-16T09:59:59.664385+0000","last_became_active":"2023-10-16T09:59:59.662221+0000","last_became_peered":"2023-10-16T09:59:59.662221+0000","last_unstale":"2023-10-16T09:59:59.664385+0000","last_undegraded":"2023-10-16T09:59:59.664385+0000","last_fullsized":"2023-10-16T09:59:59.664385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:55.232049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660741+0000","last_change":"2023-10-16T09:59:59.656725+0000","last_active":"2023-10-16T09:59:59.660741+0000","last_peered":"2023-10-16T09:59:59.660741+0000","last_clean":"2023-10-16T09:59:59.660741+0000","last_became_active":"2023-10-16T09:59:59.656366+0000","last_became_peered":"2023-10-16T09:59:59.656366+0000","last_unstale":"2023-10-16T09:59:59.660741+0000","last_undegraded":"2023-10-16T09:59:59.660741+0000","last_fullsized":"2023-10-16T09:59:59.660741+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:17.421068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664926+0000","last_change":"2023-10-16T09:59:59.661962+0000","last_active":"2023-10-16T09:59:59.664926+0000","last_peered":"2023-10-16T09:59:59.664926+0000","last_clean":"2023-10-16T09:59:59.664926+0000","last_became_active":"2023-10-16T09:59:59.661622+0000","last_became_peered":"2023-10-16T09:59:59.661622+0000","last_unstale":"2023-10-16T09:59:59.664926+0000","last_undegraded":"2023-10-16T09:59:59.664926+0000","last_fullsized":"2023-10-16T09:59:59.664926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:08:43.735073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.663012+0000","last_change":"2023-10-16T09:59:59.661739+0000","last_active":"2023-10-16T09:59:59.663012+0000","last_peered":"2023-10-16T09:59:59.663012+0000","last_clean":"2023-10-16T09:59:59.663012+0000","last_became_active":"2023-10-16T09:59:59.661086+0000","last_became_peered":"2023-10-16T09:59:59.661086+0000","last_unstale":"2023-10-16T09:59:59.663012+0000","last_undegraded":"2023-10-16T09:59:59.663012+0000","last_fullsized":"2023-10-16T09:59:59.663012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:32:23.604042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660558+0000","last_change":"2023-10-16T09:59:59.655800+0000","last_active":"2023-10-16T09:59:59.660558+0000","last_peered":"2023-10-16T09:59:59.660558+0000","last_clean":"2023-10-16T09:59:59.660558+0000","last_became_active":"2023-10-16T09:59:59.655542+0000","last_became_peered":"2023-10-16T09:59:59.655542+0000","last_unstale":"2023-10-16T09:59:59.660558+0000","last_undegraded":"2023-10-16T09:59:59.660558+0000","last_fullsized":"2023-10-16T09:59:59.660558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:09.692093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658720+0000","last_change":"2023-10-16T09:59:59.652543+0000","last_active":"2023-10-16T09:59:59.658720+0000","last_peered":"2023-10-16T09:59:59.658720+0000","last_clean":"2023-10-16T09:59:59.658720+0000","last_became_active":"2023-10-16T09:59:59.652279+0000","last_became_peered":"2023-10-16T09:59:59.652279+0000","last_unstale":"2023-10-16T09:59:59.658720+0000","last_undegraded":"2023-10-16T09:59:59.658720+0000","last_fullsized":"2023-10-16T09:59:59.658720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:12.553900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664893+0000","last_change":"2023-10-16T09:59:59.664066+0000","last_active":"2023-10-16T09:59:59.664893+0000","last_peered":"2023-10-16T09:59:59.664893+0000","last_clean":"2023-10-16T09:59:59.664893+0000","last_became_active":"2023-10-16T09:59:59.663786+0000","last_became_peered":"2023-10-16T09:59:59.663786+0000","last_unstale":"2023-10-16T09:59:59.664893+0000","last_undegraded":"2023-10-16T09:59:59.664893+0000","last_fullsized":"2023-10-16T09:59:59.664893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:39.249617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664241+0000","last_change":"2023-10-16T09:59:59.661253+0000","last_active":"2023-10-16T09:59:59.664241+0000","last_peered":"2023-10-16T09:59:59.664241+0000","last_clean":"2023-10-16T09:59:59.664241+0000","last_became_active":"2023-10-16T09:59:59.660924+0000","last_became_peered":"2023-10-16T09:59:59.660924+0000","last_unstale":"2023-10-16T09:59:59.664241+0000","last_undegraded":"2023-10-16T09:59:59.664241+0000","last_fullsized":"2023-10-16T09:59:59.664241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:23:18.399960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666777+0000","last_change":"2023-10-16T09:59:59.663563+0000","last_active":"2023-10-16T09:59:59.666777+0000","last_peered":"2023-10-16T09:59:59.666777+0000","last_clean":"2023-10-16T09:59:59.666777+0000","last_became_active":"2023-10-16T09:59:59.662969+0000","last_became_peered":"2023-10-16T09:59:59.662969+0000","last_unstale":"2023-10-16T09:59:59.666777+0000","last_undegraded":"2023-10-16T09:59:59.666777+0000","last_fullsized":"2023-10-16T09:59:59.666777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:22.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.655537+0000","last_change":"2023-10-16T09:59:59.653517+0000","last_active":"2023-10-16T09:59:59.655537+0000","last_peered":"2023-10-16T09:59:59.655537+0000","last_clean":"2023-10-16T09:59:59.655537+0000","last_became_active":"2023-10-16T09:59:59.653229+0000","last_became_peered":"2023-10-16T09:59:59.653229+0000","last_unstale":"2023-10-16T09:59:59.655537+0000","last_undegraded":"2023-10-16T09:59:59.655537+0000","last_fullsized":"2023-10-16T09:59:59.655537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.818151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665842+0000","last_change":"2023-10-16T09:59:59.664431+0000","last_active":"2023-10-16T09:59:59.665842+0000","last_peered":"2023-10-16T09:59:59.665842+0000","last_clean":"2023-10-16T09:59:59.665842+0000","last_became_active":"2023-10-16T09:59:59.662018+0000","last_became_peered":"2023-10-16T09:59:59.662018+0000","last_unstale":"2023-10-16T09:59:59.665842+0000","last_undegraded":"2023-10-16T09:59:59.665842+0000","last_fullsized":"2023-10-16T09:59:59.665842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:33:11.985316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664387+0000","last_change":"2023-10-16T09:59:59.661877+0000","last_active":"2023-10-16T09:59:59.664387+0000","last_peered":"2023-10-16T09:59:59.664387+0000","last_clean":"2023-10-16T09:59:59.664387+0000","last_became_active":"2023-10-16T09:59:59.661349+0000","last_became_peered":"2023-10-16T09:59:59.661349+0000","last_unstale":"2023-10-16T09:59:59.664387+0000","last_undegraded":"2023-10-16T09:59:59.664387+0000","last_fullsized":"2023-10-16T09:59:59.664387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:42:28.173366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662244+0000","last_change":"2023-10-16T09:59:59.659645+0000","last_active":"2023-10-16T09:59:59.662244+0000","last_peered":"2023-10-16T09:59:59.662244+0000","last_clean":"2023-10-16T09:59:59.662244+0000","last_became_active":"2023-10-16T09:59:59.659341+0000","last_became_peered":"2023-10-16T09:59:59.659341+0000","last_unstale":"2023-10-16T09:59:59.662244+0000","last_undegraded":"2023-10-16T09:59:59.662244+0000","last_fullsized":"2023-10-16T09:59:59.662244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:37.047119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.652173+0000","last_change":"2023-10-16T09:59:59.645777+0000","last_active":"2023-10-16T09:59:59.652173+0000","last_peered":"2023-10-16T09:59:59.652173+0000","last_clean":"2023-10-16T09:59:59.652173+0000","last_became_active":"2023-10-16T09:59:59.645492+0000","last_became_peered":"2023-10-16T09:59:59.645492+0000","last_unstale":"2023-10-16T09:59:59.652173+0000","last_undegraded":"2023-10-16T09:59:59.652173+0000","last_fullsized":"2023-10-16T09:59:59.652173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:04:44.300764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662219+0000","last_change":"2023-10-16T09:59:59.660690+0000","last_active":"2023-10-16T09:59:59.662219+0000","last_peered":"2023-10-16T09:59:59.662219+0000","last_clean":"2023-10-16T09:59:59.662219+0000","last_became_active":"2023-10-16T09:59:59.660297+0000","last_became_peered":"2023-10-16T09:59:59.660297+0000","last_unstale":"2023-10-16T09:59:59.662219+0000","last_undegraded":"2023-10-16T09:59:59.662219+0000","last_fullsized":"2023-10-16T09:59:59.662219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:14.609358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.659703+0000","last_change":"2023-10-16T09:59:59.658267+0000","last_active":"2023-10-16T09:59:59.659703+0000","last_peered":"2023-10-16T09:59:59.659703+0000","last_clean":"2023-10-16T09:59:59.659703+0000","last_became_active":"2023-10-16T09:59:59.657929+0000","last_became_peered":"2023-10-16T09:59:59.657929+0000","last_unstale":"2023-10-16T09:59:59.659703+0000","last_undegraded":"2023-10-16T09:59:59.659703+0000","last_fullsized":"2023-10-16T09:59:59.659703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:49:17.508223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666760+0000","last_change":"2023-10-16T09:59:59.663569+0000","last_active":"2023-10-16T09:59:59.666760+0000","last_peered":"2023-10-16T09:59:59.666760+0000","last_clean":"2023-10-16T09:59:59.666760+0000","last_became_active":"2023-10-16T09:59:59.663290+0000","last_became_peered":"2023-10-16T09:59:59.663290+0000","last_unstale":"2023-10-16T09:59:59.666760+0000","last_undegraded":"2023-10-16T09:59:59.666760+0000","last_fullsized":"2023-10-16T09:59:59.666760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:03.884054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662489+0000","last_change":"2023-10-16T09:59:59.660241+0000","last_active":"2023-10-16T09:59:59.662489+0000","last_peered":"2023-10-16T09:59:59.662489+0000","last_clean":"2023-10-16T09:59:59.662489+0000","last_became_active":"2023-10-16T09:59:59.659905+0000","last_became_peered":"2023-10-16T09:59:59.659905+0000","last_unstale":"2023-10-16T09:59:59.662489+0000","last_undegraded":"2023-10-16T09:59:59.662489+0000","last_fullsized":"2023-10-16T09:59:59.662489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:28:09.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665295+0000","last_change":"2023-10-16T09:59:59.663580+0000","last_active":"2023-10-16T09:59:59.665295+0000","last_peered":"2023-10-16T09:59:59.665295+0000","last_clean":"2023-10-16T09:59:59.665295+0000","last_became_active":"2023-10-16T09:59:59.663311+0000","last_became_peered":"2023-10-16T09:59:59.663311+0000","last_unstale":"2023-10-16T09:59:59.665295+0000","last_undegraded":"2023-10-16T09:59:59.665295+0000","last_fullsized":"2023-10-16T09:59:59.665295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:56:08.499979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661463+0000","last_change":"2023-10-16T09:59:59.656725+0000","last_active":"2023-10-16T09:59:59.661463+0000","last_peered":"2023-10-16T09:59:59.661463+0000","last_clean":"2023-10-16T09:59:59.661463+0000","last_became_active":"2023-10-16T09:59:59.656458+0000","last_became_peered":"2023-10-16T09:59:59.656458+0000","last_unstale":"2023-10-16T09:59:59.661463+0000","last_undegraded":"2023-10-16T09:59:59.661463+0000","last_fullsized":"2023-10-16T09:59:59.661463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:19.730450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.655972+0000","last_change":"2023-10-16T09:59:59.652731+0000","last_active":"2023-10-16T09:59:59.655972+0000","last_peered":"2023-10-16T09:59:59.655972+0000","last_clean":"2023-10-16T09:59:59.655972+0000","last_became_active":"2023-10-16T09:59:59.652440+0000","last_became_peered":"2023-10-16T09:59:59.652440+0000","last_unstale":"2023-10-16T09:59:59.655972+0000","last_undegraded":"2023-10-16T09:59:59.655972+0000","last_fullsized":"2023-10-16T09:59:59.655972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:22.187726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.653981+0000","last_change":"2023-10-16T09:59:59.651797+0000","last_active":"2023-10-16T09:59:59.653981+0000","last_peered":"2023-10-16T09:59:59.653981+0000","last_clean":"2023-10-16T09:59:59.653981+0000","last_became_active":"2023-10-16T09:59:59.651456+0000","last_became_peered":"2023-10-16T09:59:59.651456+0000","last_unstale":"2023-10-16T09:59:59.653981+0000","last_undegraded":"2023-10-16T09:59:59.653981+0000","last_fullsized":"2023-10-16T09:59:59.653981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:11:46.375900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658262+0000","last_change":"2023-10-16T09:59:59.656540+0000","last_active":"2023-10-16T09:59:59.658262+0000","last_peered":"2023-10-16T09:59:59.658262+0000","last_clean":"2023-10-16T09:59:59.658262+0000","last_became_active":"2023-10-16T09:59:59.654971+0000","last_became_peered":"2023-10-16T09:59:59.654971+0000","last_unstale":"2023-10-16T09:59:59.658262+0000","last_undegraded":"2023-10-16T09:59:59.658262+0000","last_fullsized":"2023-10-16T09:59:59.658262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:37.666716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660633+0000","last_change":"2023-10-16T09:59:59.659139+0000","last_active":"2023-10-16T09:59:59.660633+0000","last_peered":"2023-10-16T09:59:59.660633+0000","last_clean":"2023-10-16T09:59:59.660633+0000","last_became_active":"2023-10-16T09:59:59.658876+0000","last_became_peered":"2023-10-16T09:59:59.658876+0000","last_unstale":"2023-10-16T09:59:59.660633+0000","last_undegraded":"2023-10-16T09:59:59.660633+0000","last_fullsized":"2023-10-16T09:59:59.660633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:06.079178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.655513+0000","last_change":"2023-10-16T09:59:59.654098+0000","last_active":"2023-10-16T09:59:59.655513+0000","last_peered":"2023-10-16T09:59:59.655513+0000","last_clean":"2023-10-16T09:59:59.655513+0000","last_became_active":"2023-10-16T09:59:59.653824+0000","last_became_peered":"2023-10-16T09:59:59.653824+0000","last_unstale":"2023-10-16T09:59:59.655513+0000","last_undegraded":"2023-10-16T09:59:59.655513+0000","last_fullsized":"2023-10-16T09:59:59.655513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:14.752732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658269+0000","last_change":"2023-10-16T09:59:59.657200+0000","last_active":"2023-10-16T09:59:59.658269+0000","last_peered":"2023-10-16T09:59:59.658269+0000","last_clean":"2023-10-16T09:59:59.658269+0000","last_became_active":"2023-10-16T09:59:59.656892+0000","last_became_peered":"2023-10-16T09:59:59.656892+0000","last_unstale":"2023-10-16T09:59:59.658269+0000","last_undegraded":"2023-10-16T09:59:59.658269+0000","last_fullsized":"2023-10-16T09:59:59.658269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:04:34.801362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664905+0000","last_change":"2023-10-16T09:59:59.661235+0000","last_active":"2023-10-16T09:59:59.664905+0000","last_peered":"2023-10-16T09:59:59.664905+0000","last_clean":"2023-10-16T09:59:59.664905+0000","last_became_active":"2023-10-16T09:59:59.660740+0000","last_became_peered":"2023-10-16T09:59:59.660740+0000","last_unstale":"2023-10-16T09:59:59.664905+0000","last_undegraded":"2023-10-16T09:59:59.664905+0000","last_fullsized":"2023-10-16T09:59:59.664905+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:38:36.457869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.654234+0000","last_change":"2023-10-16T09:59:59.651889+0000","last_active":"2023-10-16T09:59:59.654234+0000","last_peered":"2023-10-16T09:59:59.654234+0000","last_clean":"2023-10-16T09:59:59.654234+0000","last_became_active":"2023-10-16T09:59:59.651446+0000","last_became_peered":"2023-10-16T09:59:59.651446+0000","last_unstale":"2023-10-16T09:59:59.654234+0000","last_undegraded":"2023-10-16T09:59:59.654234+0000","last_fullsized":"2023-10-16T09:59:59.654234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:18:34.399822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664638+0000","last_change":"2023-10-16T09:59:59.661388+0000","last_active":"2023-10-16T09:59:59.664638+0000","last_peered":"2023-10-16T09:59:59.664638+0000","last_clean":"2023-10-16T09:59:59.664638+0000","last_became_active":"2023-10-16T09:59:59.661037+0000","last_became_peered":"2023-10-16T09:59:59.661037+0000","last_unstale":"2023-10-16T09:59:59.664638+0000","last_undegraded":"2023-10-16T09:59:59.664638+0000","last_fullsized":"2023-10-16T09:59:59.664638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:09.347465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658019+0000","last_change":"2023-10-16T09:59:59.656922+0000","last_active":"2023-10-16T09:59:59.658019+0000","last_peered":"2023-10-16T09:59:59.658019+0000","last_clean":"2023-10-16T09:59:59.658019+0000","last_became_active":"2023-10-16T09:59:59.656595+0000","last_became_peered":"2023-10-16T09:59:59.656595+0000","last_unstale":"2023-10-16T09:59:59.658019+0000","last_undegraded":"2023-10-16T09:59:59.658019+0000","last_fullsized":"2023-10-16T09:59:59.658019+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:47.707940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.656056+0000","last_change":"2023-10-16T09:59:59.650411+0000","last_active":"2023-10-16T09:59:59.656056+0000","last_peered":"2023-10-16T09:59:59.656056+0000","last_clean":"2023-10-16T09:59:59.656056+0000","last_became_active":"2023-10-16T09:59:59.650125+0000","last_became_peered":"2023-10-16T09:59:59.650125+0000","last_unstale":"2023-10-16T09:59:59.656056+0000","last_undegraded":"2023-10-16T09:59:59.656056+0000","last_fullsized":"2023-10-16T09:59:59.656056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:01:38.709948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666849+0000","last_change":"2023-10-16T09:59:59.665979+0000","last_active":"2023-10-16T09:59:59.666849+0000","last_peered":"2023-10-16T09:59:59.666849+0000","last_clean":"2023-10-16T09:59:59.666849+0000","last_became_active":"2023-10-16T09:59:59.665733+0000","last_became_peered":"2023-10-16T09:59:59.665733+0000","last_unstale":"2023-10-16T09:59:59.666849+0000","last_undegraded":"2023-10-16T09:59:59.666849+0000","last_fullsized":"2023-10-16T09:59:59.666849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:49.444256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664294+0000","last_change":"2023-10-16T09:59:59.661864+0000","last_active":"2023-10-16T09:59:59.664294+0000","last_peered":"2023-10-16T09:59:59.664294+0000","last_clean":"2023-10-16T09:59:59.664294+0000","last_became_active":"2023-10-16T09:59:59.661606+0000","last_became_peered":"2023-10-16T09:59:59.661606+0000","last_unstale":"2023-10-16T09:59:59.664294+0000","last_undegraded":"2023-10-16T09:59:59.664294+0000","last_fullsized":"2023-10-16T09:59:59.664294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:28.126128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"89'4","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.983167+0000","last_change":"2023-10-16T09:59:58.657015+0000","last_active":"2023-10-16T09:59:59.983167+0000","last_peered":"2023-10-16T09:59:59.983167+0000","last_clean":"2023-10-16T09:59:59.983167+0000","last_became_active":"2023-10-16T09:59:58.656774+0000","last_became_peered":"2023-10-16T09:59:58.656774+0000","last_unstale":"2023-10-16T09:59:59.983167+0000","last_undegraded":"2023-10-16T09:59:59.983167+0000","last_fullsized":"2023-10-16T09:59:59.983167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:45:28.222879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666743+0000","last_change":"2023-10-16T09:59:59.663334+0000","last_active":"2023-10-16T09:59:59.666743+0000","last_peered":"2023-10-16T09:59:59.666743+0000","last_clean":"2023-10-16T09:59:59.666743+0000","last_became_active":"2023-10-16T09:59:59.662887+0000","last_became_peered":"2023-10-16T09:59:59.662887+0000","last_unstale":"2023-10-16T09:59:59.666743+0000","last_undegraded":"2023-10-16T09:59:59.666743+0000","last_fullsized":"2023-10-16T09:59:59.666743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:17.503821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.975923+0000","last_change":"2023-10-16T09:59:58.656288+0000","last_active":"2023-10-16T09:59:59.975923+0000","last_peered":"2023-10-16T09:59:59.975923+0000","last_clean":"2023-10-16T09:59:59.975923+0000","last_became_active":"2023-10-16T09:59:58.655929+0000","last_became_peered":"2023-10-16T09:59:58.655929+0000","last_unstale":"2023-10-16T09:59:59.975923+0000","last_undegraded":"2023-10-16T09:59:59.975923+0000","last_fullsized":"2023-10-16T09:59:59.975923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:56.983631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660573+0000","last_change":"2023-10-16T09:59:59.656921+0000","last_active":"2023-10-16T09:59:59.660573+0000","last_peered":"2023-10-16T09:59:59.660573+0000","last_clean":"2023-10-16T09:59:59.660573+0000","last_became_active":"2023-10-16T09:59:59.656653+0000","last_became_peered":"2023-10-16T09:59:59.656653+0000","last_unstale":"2023-10-16T09:59:59.660573+0000","last_undegraded":"2023-10-16T09:59:59.660573+0000","last_fullsized":"2023-10-16T09:59:59.660573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:48.399295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.979039+0000","last_change":"2023-10-16T09:59:58.656130+0000","last_active":"2023-10-16T09:59:59.979039+0000","last_peered":"2023-10-16T09:59:59.979039+0000","last_clean":"2023-10-16T09:59:59.979039+0000","last_became_active":"2023-10-16T09:59:58.655899+0000","last_became_peered":"2023-10-16T09:59:58.655899+0000","last_unstale":"2023-10-16T09:59:59.979039+0000","last_undegraded":"2023-10-16T09:59:59.979039+0000","last_fullsized":"2023-10-16T09:59:59.979039+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:21.529623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.658671+0000","last_change":"2023-10-16T09:59:59.654504+0000","last_active":"2023-10-16T09:59:59.658671+0000","last_peered":"2023-10-16T09:59:59.658671+0000","last_clean":"2023-10-16T09:59:59.658671+0000","last_became_active":"2023-10-16T09:59:59.654113+0000","last_became_peered":"2023-10-16T09:59:59.654113+0000","last_unstale":"2023-10-16T09:59:59.658671+0000","last_undegraded":"2023-10-16T09:59:59.658671+0000","last_fullsized":"2023-10-16T09:59:59.658671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:04:21.462984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.978821+0000","last_change":"2023-10-16T09:59:58.657383+0000","last_active":"2023-10-16T09:59:59.978821+0000","last_peered":"2023-10-16T09:59:59.978821+0000","last_clean":"2023-10-16T09:59:59.978821+0000","last_became_active":"2023-10-16T09:59:58.655813+0000","last_became_peered":"2023-10-16T09:59:58.655813+0000","last_unstale":"2023-10-16T09:59:59.978821+0000","last_undegraded":"2023-10-16T09:59:59.978821+0000","last_fullsized":"2023-10-16T09:59:59.978821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:54.422203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.663531+0000","last_change":"2023-10-16T09:59:59.662073+0000","last_active":"2023-10-16T09:59:59.663531+0000","last_peered":"2023-10-16T09:59:59.663531+0000","last_clean":"2023-10-16T09:59:59.663531+0000","last_became_active":"2023-10-16T09:59:59.661838+0000","last_became_peered":"2023-10-16T09:59:59.661838+0000","last_unstale":"2023-10-16T09:59:59.663531+0000","last_undegraded":"2023-10-16T09:59:59.663531+0000","last_fullsized":"2023-10-16T09:59:59.663531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:59.099478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.976257+0000","last_change":"2023-10-16T09:59:58.646206+0000","last_active":"2023-10-16T09:59:59.976257+0000","last_peered":"2023-10-16T09:59:59.976257+0000","last_clean":"2023-10-16T09:59:59.976257+0000","last_became_active":"2023-10-16T09:59:58.645938+0000","last_became_peered":"2023-10-16T09:59:58.645938+0000","last_unstale":"2023-10-16T09:59:59.976257+0000","last_undegraded":"2023-10-16T09:59:59.976257+0000","last_fullsized":"2023-10-16T09:59:59.976257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:57.676281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.659231+0000","last_change":"2023-10-16T09:59:59.656130+0000","last_active":"2023-10-16T09:59:59.659231+0000","last_peered":"2023-10-16T09:59:59.659231+0000","last_clean":"2023-10-16T09:59:59.659231+0000","last_became_active":"2023-10-16T09:59:59.655753+0000","last_became_peered":"2023-10-16T09:59:59.655753+0000","last_unstale":"2023-10-16T09:59:59.659231+0000","last_undegraded":"2023-10-16T09:59:59.659231+0000","last_fullsized":"2023-10-16T09:59:59.659231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:03.911665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.976774+0000","last_change":"2023-10-16T09:59:58.659019+0000","last_active":"2023-10-16T09:59:59.976774+0000","last_peered":"2023-10-16T09:59:59.976774+0000","last_clean":"2023-10-16T09:59:59.976774+0000","last_became_active":"2023-10-16T09:59:58.657452+0000","last_became_peered":"2023-10-16T09:59:58.657452+0000","last_unstale":"2023-10-16T09:59:59.976774+0000","last_undegraded":"2023-10-16T09:59:59.976774+0000","last_fullsized":"2023-10-16T09:59:59.976774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:56.403538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.652353+0000","last_change":"2023-10-16T09:59:59.645668+0000","last_active":"2023-10-16T09:59:59.652353+0000","last_peered":"2023-10-16T09:59:59.652353+0000","last_clean":"2023-10-16T09:59:59.652353+0000","last_became_active":"2023-10-16T09:59:59.645414+0000","last_became_peered":"2023-10-16T09:59:59.645414+0000","last_unstale":"2023-10-16T09:59:59.652353+0000","last_undegraded":"2023-10-16T09:59:59.652353+0000","last_fullsized":"2023-10-16T09:59:59.652353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:33.244429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.647450+0000","last_change":"2023-10-16T09:59:58.646272+0000","last_active":"2023-10-16T09:59:59.647450+0000","last_peered":"2023-10-16T09:59:59.647450+0000","last_clean":"2023-10-16T09:59:59.647450+0000","last_became_active":"2023-10-16T09:59:58.645928+0000","last_became_peered":"2023-10-16T09:59:58.645928+0000","last_unstale":"2023-10-16T09:59:59.647450+0000","last_undegraded":"2023-10-16T09:59:59.647450+0000","last_fullsized":"2023-10-16T09:59:59.647450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:29.156017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661515+0000","last_change":"2023-10-16T09:59:59.656201+0000","last_active":"2023-10-16T09:59:59.661515+0000","last_peered":"2023-10-16T09:59:59.661515+0000","last_clean":"2023-10-16T09:59:59.661515+0000","last_became_active":"2023-10-16T09:59:59.655914+0000","last_became_peered":"2023-10-16T09:59:59.655914+0000","last_unstale":"2023-10-16T09:59:59.661515+0000","last_undegraded":"2023-10-16T09:59:59.661515+0000","last_fullsized":"2023-10-16T09:59:59.661515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:22.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.641089+0000","last_change":"2023-10-16T09:59:58.654988+0000","last_active":"2023-10-16T09:59:59.641089+0000","last_peered":"2023-10-16T09:59:59.641089+0000","last_clean":"2023-10-16T09:59:59.641089+0000","last_became_active":"2023-10-16T09:59:58.654236+0000","last_became_peered":"2023-10-16T09:59:58.654236+0000","last_unstale":"2023-10-16T09:59:59.641089+0000","last_undegraded":"2023-10-16T09:59:59.641089+0000","last_fullsized":"2023-10-16T09:59:59.641089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:36.040942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.656020+0000","last_change":"2023-10-16T09:59:59.653532+0000","last_active":"2023-10-16T09:59:59.656020+0000","last_peered":"2023-10-16T09:59:59.656020+0000","last_clean":"2023-10-16T09:59:59.656020+0000","last_became_active":"2023-10-16T09:59:59.653230+0000","last_became_peered":"2023-10-16T09:59:59.653230+0000","last_unstale":"2023-10-16T09:59:59.656020+0000","last_undegraded":"2023-10-16T09:59:59.656020+0000","last_fullsized":"2023-10-16T09:59:59.656020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:22.802365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.644711+0000","last_change":"2023-10-16T09:59:58.645543+0000","last_active":"2023-10-16T09:59:59.644711+0000","last_peered":"2023-10-16T09:59:59.644711+0000","last_clean":"2023-10-16T09:59:59.644711+0000","last_became_active":"2023-10-16T09:59:58.645261+0000","last_became_peered":"2023-10-16T09:59:58.645261+0000","last_unstale":"2023-10-16T09:59:59.644711+0000","last_undegraded":"2023-10-16T09:59:59.644711+0000","last_fullsized":"2023-10-16T09:59:59.644711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:00.997205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664364+0000","last_change":"2023-10-16T09:59:59.662495+0000","last_active":"2023-10-16T09:59:59.664364+0000","last_peered":"2023-10-16T09:59:59.664364+0000","last_clean":"2023-10-16T09:59:59.664364+0000","last_became_active":"2023-10-16T09:59:59.662143+0000","last_became_peered":"2023-10-16T09:59:59.662143+0000","last_unstale":"2023-10-16T09:59:59.664364+0000","last_undegraded":"2023-10-16T09:59:59.664364+0000","last_fullsized":"2023-10-16T09:59:59.664364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:17:57.910605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.648412+0000","last_change":"2023-10-16T09:59:58.657403+0000","last_active":"2023-10-16T09:59:59.648412+0000","last_peered":"2023-10-16T09:59:59.648412+0000","last_clean":"2023-10-16T09:59:59.648412+0000","last_became_active":"2023-10-16T09:59:58.655954+0000","last_became_peered":"2023-10-16T09:59:58.655954+0000","last_unstale":"2023-10-16T09:59:59.648412+0000","last_undegraded":"2023-10-16T09:59:59.648412+0000","last_fullsized":"2023-10-16T09:59:59.648412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:32.238252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.660551+0000","last_change":"2023-10-16T09:59:59.656424+0000","last_active":"2023-10-16T09:59:59.660551+0000","last_peered":"2023-10-16T09:59:59.660551+0000","last_clean":"2023-10-16T09:59:59.660551+0000","last_became_active":"2023-10-16T09:59:59.656134+0000","last_became_peered":"2023-10-16T09:59:59.656134+0000","last_unstale":"2023-10-16T09:59:59.660551+0000","last_undegraded":"2023-10-16T09:59:59.660551+0000","last_fullsized":"2023-10-16T09:59:59.660551+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:29.805756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"89'3","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977979+0000","last_change":"2023-10-16T09:59:58.649178+0000","last_active":"2023-10-16T09:59:59.977979+0000","last_peered":"2023-10-16T09:59:59.977979+0000","last_clean":"2023-10-16T09:59:59.977979+0000","last_became_active":"2023-10-16T09:59:58.648874+0000","last_became_peered":"2023-10-16T09:59:58.648874+0000","last_unstale":"2023-10-16T09:59:59.977979+0000","last_undegraded":"2023-10-16T09:59:59.977979+0000","last_fullsized":"2023-10-16T09:59:59.977979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:03.670419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666785+0000","last_change":"2023-10-16T09:59:59.663731+0000","last_active":"2023-10-16T09:59:59.666785+0000","last_peered":"2023-10-16T09:59:59.666785+0000","last_clean":"2023-10-16T09:59:59.666785+0000","last_became_active":"2023-10-16T09:59:59.663459+0000","last_became_peered":"2023-10-16T09:59:59.663459+0000","last_unstale":"2023-10-16T09:59:59.666785+0000","last_undegraded":"2023-10-16T09:59:59.666785+0000","last_fullsized":"2023-10-16T09:59:59.666785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:55.428380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"89'6","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.982948+0000","last_change":"2023-10-16T09:59:58.656158+0000","last_active":"2023-10-16T09:59:59.982948+0000","last_peered":"2023-10-16T09:59:59.982948+0000","last_clean":"2023-10-16T09:59:59.982948+0000","last_became_active":"2023-10-16T09:59:58.655872+0000","last_became_peered":"2023-10-16T09:59:58.655872+0000","last_unstale":"2023-10-16T09:59:59.982948+0000","last_undegraded":"2023-10-16T09:59:59.982948+0000","last_fullsized":"2023-10-16T09:59:59.982948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:12:47.596299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.657582+0000","last_change":"2023-10-16T09:59:59.653586+0000","last_active":"2023-10-16T09:59:59.657582+0000","last_peered":"2023-10-16T09:59:59.657582+0000","last_clean":"2023-10-16T09:59:59.657582+0000","last_became_active":"2023-10-16T09:59:59.653285+0000","last_became_peered":"2023-10-16T09:59:59.653285+0000","last_unstale":"2023-10-16T09:59:59.657582+0000","last_undegraded":"2023-10-16T09:59:59.657582+0000","last_fullsized":"2023-10-16T09:59:59.657582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:46:04.410160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.978327+0000","last_change":"2023-10-16T09:59:58.654580+0000","last_active":"2023-10-16T09:59:59.978327+0000","last_peered":"2023-10-16T09:59:59.978327+0000","last_clean":"2023-10-16T09:59:59.978327+0000","last_became_active":"2023-10-16T09:59:58.654291+0000","last_became_peered":"2023-10-16T09:59:58.654291+0000","last_unstale":"2023-10-16T09:59:59.978327+0000","last_undegraded":"2023-10-16T09:59:59.978327+0000","last_fullsized":"2023-10-16T09:59:59.978327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:13.412420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.654275+0000","last_change":"2023-10-16T09:59:59.645339+0000","last_active":"2023-10-16T09:59:59.654275+0000","last_peered":"2023-10-16T09:59:59.654275+0000","last_clean":"2023-10-16T09:59:59.654275+0000","last_became_active":"2023-10-16T09:59:59.644971+0000","last_became_peered":"2023-10-16T09:59:59.644971+0000","last_unstale":"2023-10-16T09:59:59.654275+0000","last_undegraded":"2023-10-16T09:59:59.654275+0000","last_fullsized":"2023-10-16T09:59:59.654275+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:19.748676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.646919+0000","last_change":"2023-10-16T09:59:58.659186+0000","last_active":"2023-10-16T09:59:59.646919+0000","last_peered":"2023-10-16T09:59:59.646919+0000","last_clean":"2023-10-16T09:59:59.646919+0000","last_became_active":"2023-10-16T09:59:58.658485+0000","last_became_peered":"2023-10-16T09:59:58.658485+0000","last_unstale":"2023-10-16T09:59:59.646919+0000","last_undegraded":"2023-10-16T09:59:59.646919+0000","last_fullsized":"2023-10-16T09:59:59.646919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:07:16.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661509+0000","last_change":"2023-10-16T09:59:59.659210+0000","last_active":"2023-10-16T09:59:59.661509+0000","last_peered":"2023-10-16T09:59:59.661509+0000","last_clean":"2023-10-16T09:59:59.661509+0000","last_became_active":"2023-10-16T09:59:59.658911+0000","last_became_peered":"2023-10-16T09:59:59.658911+0000","last_unstale":"2023-10-16T09:59:59.661509+0000","last_undegraded":"2023-10-16T09:59:59.661509+0000","last_fullsized":"2023-10-16T09:59:59.661509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:09:03.884054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.640643+0000","last_change":"2023-10-16T09:59:58.651162+0000","last_active":"2023-10-16T09:59:59.640643+0000","last_peered":"2023-10-16T09:59:59.640643+0000","last_clean":"2023-10-16T09:59:59.640643+0000","last_became_active":"2023-10-16T09:59:58.650848+0000","last_became_peered":"2023-10-16T09:59:58.650848+0000","last_unstale":"2023-10-16T09:59:59.640643+0000","last_undegraded":"2023-10-16T09:59:59.640643+0000","last_fullsized":"2023-10-16T09:59:59.640643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:39.765289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666831+0000","last_change":"2023-10-16T09:59:59.664040+0000","last_active":"2023-10-16T09:59:59.666831+0000","last_peered":"2023-10-16T09:59:59.666831+0000","last_clean":"2023-10-16T09:59:59.666831+0000","last_became_active":"2023-10-16T09:59:59.663757+0000","last_became_peered":"2023-10-16T09:59:59.663757+0000","last_unstale":"2023-10-16T09:59:59.666831+0000","last_undegraded":"2023-10-16T09:59:59.666831+0000","last_fullsized":"2023-10-16T09:59:59.666831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:14.609358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.979185+0000","last_change":"2023-10-16T09:59:58.654459+0000","last_active":"2023-10-16T09:59:59.979185+0000","last_peered":"2023-10-16T09:59:59.979185+0000","last_clean":"2023-10-16T09:59:59.979185+0000","last_became_active":"2023-10-16T09:59:58.654077+0000","last_became_peered":"2023-10-16T09:59:58.654077+0000","last_unstale":"2023-10-16T09:59:59.979185+0000","last_undegraded":"2023-10-16T09:59:59.979185+0000","last_fullsized":"2023-10-16T09:59:59.979185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:26.041963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662892+0000","last_change":"2023-10-16T09:59:59.661026+0000","last_active":"2023-10-16T09:59:59.662892+0000","last_peered":"2023-10-16T09:59:59.662892+0000","last_clean":"2023-10-16T09:59:59.662892+0000","last_became_active":"2023-10-16T09:59:59.660749+0000","last_became_peered":"2023-10-16T09:59:59.660749+0000","last_unstale":"2023-10-16T09:59:59.662892+0000","last_undegraded":"2023-10-16T09:59:59.662892+0000","last_fullsized":"2023-10-16T09:59:59.662892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:27.116852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977471+0000","last_change":"2023-10-16T09:59:58.658066+0000","last_active":"2023-10-16T09:59:59.977471+0000","last_peered":"2023-10-16T09:59:59.977471+0000","last_clean":"2023-10-16T09:59:59.977471+0000","last_became_active":"2023-10-16T09:59:58.657777+0000","last_became_peered":"2023-10-16T09:59:58.657777+0000","last_unstale":"2023-10-16T09:59:59.977471+0000","last_undegraded":"2023-10-16T09:59:59.977471+0000","last_fullsized":"2023-10-16T09:59:59.977471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:33:03.427458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664404+0000","last_change":"2023-10-16T09:59:59.662805+0000","last_active":"2023-10-16T09:59:59.664404+0000","last_peered":"2023-10-16T09:59:59.664404+0000","last_clean":"2023-10-16T09:59:59.664404+0000","last_became_active":"2023-10-16T09:59:59.662533+0000","last_became_peered":"2023-10-16T09:59:59.662533+0000","last_unstale":"2023-10-16T09:59:59.664404+0000","last_undegraded":"2023-10-16T09:59:59.664404+0000","last_fullsized":"2023-10-16T09:59:59.664404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:31:59.669734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977339+0000","last_change":"2023-10-16T09:59:58.658549+0000","last_active":"2023-10-16T09:59:59.977339+0000","last_peered":"2023-10-16T09:59:59.977339+0000","last_clean":"2023-10-16T09:59:59.977339+0000","last_became_active":"2023-10-16T09:59:58.658267+0000","last_became_peered":"2023-10-16T09:59:58.658267+0000","last_unstale":"2023-10-16T09:59:59.977339+0000","last_undegraded":"2023-10-16T09:59:59.977339+0000","last_fullsized":"2023-10-16T09:59:59.977339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:16.407745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662999+0000","last_change":"2023-10-16T09:59:59.661581+0000","last_active":"2023-10-16T09:59:59.662999+0000","last_peered":"2023-10-16T09:59:59.662999+0000","last_clean":"2023-10-16T09:59:59.662999+0000","last_became_active":"2023-10-16T09:59:59.660802+0000","last_became_peered":"2023-10-16T09:59:59.660802+0000","last_unstale":"2023-10-16T09:59:59.662999+0000","last_undegraded":"2023-10-16T09:59:59.662999+0000","last_fullsized":"2023-10-16T09:59:59.662999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:20.291936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"89'1","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.979124+0000","last_change":"2023-10-16T09:59:58.660442+0000","last_active":"2023-10-16T09:59:59.979124+0000","last_peered":"2023-10-16T09:59:59.979124+0000","last_clean":"2023-10-16T09:59:59.979124+0000","last_became_active":"2023-10-16T09:59:58.660191+0000","last_became_peered":"2023-10-16T09:59:58.660191+0000","last_unstale":"2023-10-16T09:59:59.979124+0000","last_undegraded":"2023-10-16T09:59:59.979124+0000","last_fullsized":"2023-10-16T09:59:59.979124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:08.685915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.651081+0000","last_change":"2023-10-16T09:59:59.644870+0000","last_active":"2023-10-16T09:59:59.651081+0000","last_peered":"2023-10-16T09:59:59.651081+0000","last_clean":"2023-10-16T09:59:59.651081+0000","last_became_active":"2023-10-16T09:59:59.644642+0000","last_became_peered":"2023-10-16T09:59:59.644642+0000","last_unstale":"2023-10-16T09:59:59.651081+0000","last_undegraded":"2023-10-16T09:59:59.651081+0000","last_fullsized":"2023-10-16T09:59:59.651081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:09.158237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.648824+0000","last_change":"2023-10-16T09:59:58.655638+0000","last_active":"2023-10-16T09:59:59.648824+0000","last_peered":"2023-10-16T09:59:59.648824+0000","last_clean":"2023-10-16T09:59:59.648824+0000","last_became_active":"2023-10-16T09:59:58.655320+0000","last_became_peered":"2023-10-16T09:59:58.655320+0000","last_unstale":"2023-10-16T09:59:59.648824+0000","last_undegraded":"2023-10-16T09:59:59.648824+0000","last_fullsized":"2023-10-16T09:59:59.648824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:30.303572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.667044+0000","last_change":"2023-10-16T09:59:59.664639+0000","last_active":"2023-10-16T09:59:59.667044+0000","last_peered":"2023-10-16T09:59:59.667044+0000","last_clean":"2023-10-16T09:59:59.667044+0000","last_became_active":"2023-10-16T09:59:59.661787+0000","last_became_peered":"2023-10-16T09:59:59.661787+0000","last_unstale":"2023-10-16T09:59:59.667044+0000","last_undegraded":"2023-10-16T09:59:59.667044+0000","last_fullsized":"2023-10-16T09:59:59.667044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:42.134799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.647327+0000","last_change":"2023-10-16T09:59:58.658728+0000","last_active":"2023-10-16T09:59:59.647327+0000","last_peered":"2023-10-16T09:59:59.647327+0000","last_clean":"2023-10-16T09:59:59.647327+0000","last_became_active":"2023-10-16T09:59:58.657700+0000","last_became_peered":"2023-10-16T09:59:58.657700+0000","last_unstale":"2023-10-16T09:59:59.647327+0000","last_undegraded":"2023-10-16T09:59:59.647327+0000","last_fullsized":"2023-10-16T09:59:59.647327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:16.612088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666609+0000","last_change":"2023-10-16T09:59:59.664049+0000","last_active":"2023-10-16T09:59:59.666609+0000","last_peered":"2023-10-16T09:59:59.666609+0000","last_clean":"2023-10-16T09:59:59.666609+0000","last_became_active":"2023-10-16T09:59:59.661487+0000","last_became_peered":"2023-10-16T09:59:59.661487+0000","last_unstale":"2023-10-16T09:59:59.666609+0000","last_undegraded":"2023-10-16T09:59:59.666609+0000","last_fullsized":"2023-10-16T09:59:59.666609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:43:36.927647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.640608+0000","last_change":"2023-10-16T09:59:58.656371+0000","last_active":"2023-10-16T09:59:59.640608+0000","last_peered":"2023-10-16T09:59:59.640608+0000","last_clean":"2023-10-16T09:59:59.640608+0000","last_became_active":"2023-10-16T09:59:58.655886+0000","last_became_peered":"2023-10-16T09:59:58.655886+0000","last_unstale":"2023-10-16T09:59:59.640608+0000","last_undegraded":"2023-10-16T09:59:59.640608+0000","last_fullsized":"2023-10-16T09:59:59.640608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:23.052157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664930+0000","last_change":"2023-10-16T09:59:59.661716+0000","last_active":"2023-10-16T09:59:59.664930+0000","last_peered":"2023-10-16T09:59:59.664930+0000","last_clean":"2023-10-16T09:59:59.664930+0000","last_became_active":"2023-10-16T09:59:59.661187+0000","last_became_peered":"2023-10-16T09:59:59.661187+0000","last_unstale":"2023-10-16T09:59:59.664930+0000","last_undegraded":"2023-10-16T09:59:59.664930+0000","last_fullsized":"2023-10-16T09:59:59.664930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:51.489231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.650043+0000","last_change":"2023-10-16T09:59:58.653886+0000","last_active":"2023-10-16T09:59:59.650043+0000","last_peered":"2023-10-16T09:59:59.650043+0000","last_clean":"2023-10-16T09:59:59.650043+0000","last_became_active":"2023-10-16T09:59:58.652708+0000","last_became_peered":"2023-10-16T09:59:58.652708+0000","last_unstale":"2023-10-16T09:59:59.650043+0000","last_undegraded":"2023-10-16T09:59:59.650043+0000","last_fullsized":"2023-10-16T09:59:59.650043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:04:50.890542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662078+0000","last_change":"2023-10-16T09:59:59.653387+0000","last_active":"2023-10-16T09:59:59.662078+0000","last_peered":"2023-10-16T09:59:59.662078+0000","last_clean":"2023-10-16T09:59:59.662078+0000","last_became_active":"2023-10-16T09:59:59.653103+0000","last_became_peered":"2023-10-16T09:59:59.653103+0000","last_unstale":"2023-10-16T09:59:59.662078+0000","last_undegraded":"2023-10-16T09:59:59.662078+0000","last_fullsized":"2023-10-16T09:59:59.662078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:04:44.300764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.977270+0000","last_change":"2023-10-16T09:59:58.657889+0000","last_active":"2023-10-16T09:59:59.977270+0000","last_peered":"2023-10-16T09:59:59.977270+0000","last_clean":"2023-10-16T09:59:59.977270+0000","last_became_active":"2023-10-16T09:59:58.657624+0000","last_became_peered":"2023-10-16T09:59:58.657624+0000","last_unstale":"2023-10-16T09:59:59.977270+0000","last_undegraded":"2023-10-16T09:59:59.977270+0000","last_fullsized":"2023-10-16T09:59:59.977270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:38.319429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.662778+0000","last_change":"2023-10-16T09:59:59.660572+0000","last_active":"2023-10-16T09:59:59.662778+0000","last_peered":"2023-10-16T09:59:59.662778+0000","last_clean":"2023-10-16T09:59:59.662778+0000","last_became_active":"2023-10-16T09:59:59.660300+0000","last_became_peered":"2023-10-16T09:59:59.660300+0000","last_unstale":"2023-10-16T09:59:59.662778+0000","last_undegraded":"2023-10-16T09:59:59.662778+0000","last_fullsized":"2023-10-16T09:59:59.662778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:45.865340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.641948+0000","last_change":"2023-10-16T09:59:58.650822+0000","last_active":"2023-10-16T09:59:59.641948+0000","last_peered":"2023-10-16T09:59:59.641948+0000","last_clean":"2023-10-16T09:59:59.641948+0000","last_became_active":"2023-10-16T09:59:58.650516+0000","last_became_peered":"2023-10-16T09:59:58.650516+0000","last_unstale":"2023-10-16T09:59:59.641948+0000","last_undegraded":"2023-10-16T09:59:59.641948+0000","last_fullsized":"2023-10-16T09:59:59.641948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:04.816771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.664569+0000","last_change":"2023-10-16T09:59:59.663350+0000","last_active":"2023-10-16T09:59:59.664569+0000","last_peered":"2023-10-16T09:59:59.664569+0000","last_clean":"2023-10-16T09:59:59.664569+0000","last_became_active":"2023-10-16T09:59:59.663077+0000","last_became_peered":"2023-10-16T09:59:59.663077+0000","last_unstale":"2023-10-16T09:59:59.664569+0000","last_undegraded":"2023-10-16T09:59:59.664569+0000","last_fullsized":"2023-10-16T09:59:59.664569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:56:12.329218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.641931+0000","last_change":"2023-10-16T09:59:58.657618+0000","last_active":"2023-10-16T09:59:59.641931+0000","last_peered":"2023-10-16T09:59:59.641931+0000","last_clean":"2023-10-16T09:59:59.641931+0000","last_became_active":"2023-10-16T09:59:58.657319+0000","last_became_peered":"2023-10-16T09:59:58.657319+0000","last_unstale":"2023-10-16T09:59:59.641931+0000","last_undegraded":"2023-10-16T09:59:59.641931+0000","last_fullsized":"2023-10-16T09:59:59.641931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:18.724273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.659637+0000","last_change":"2023-10-16T09:59:59.658842+0000","last_active":"2023-10-16T09:59:59.659637+0000","last_peered":"2023-10-16T09:59:59.659637+0000","last_clean":"2023-10-16T09:59:59.659637+0000","last_became_active":"2023-10-16T09:59:59.658311+0000","last_became_peered":"2023-10-16T09:59:59.658311+0000","last_unstale":"2023-10-16T09:59:59.659637+0000","last_undegraded":"2023-10-16T09:59:59.659637+0000","last_fullsized":"2023-10-16T09:59:59.659637+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:47.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.650172+0000","last_change":"2023-10-16T09:59:58.653705+0000","last_active":"2023-10-16T09:59:59.650172+0000","last_peered":"2023-10-16T09:59:59.650172+0000","last_clean":"2023-10-16T09:59:59.650172+0000","last_became_active":"2023-10-16T09:59:58.653469+0000","last_became_peered":"2023-10-16T09:59:58.653469+0000","last_unstale":"2023-10-16T09:59:59.650172+0000","last_undegraded":"2023-10-16T09:59:59.650172+0000","last_fullsized":"2023-10-16T09:59:59.650172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:37.607585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.666862+0000","last_change":"2023-10-16T09:59:59.666154+0000","last_active":"2023-10-16T09:59:59.666862+0000","last_peered":"2023-10-16T09:59:59.666862+0000","last_clean":"2023-10-16T09:59:59.666862+0000","last_became_active":"2023-10-16T09:59:59.665789+0000","last_became_peered":"2023-10-16T09:59:59.665789+0000","last_unstale":"2023-10-16T09:59:59.666862+0000","last_undegraded":"2023-10-16T09:59:59.666862+0000","last_fullsized":"2023-10-16T09:59:59.666862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:39.325606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.640895+0000","last_change":"2023-10-16T09:59:58.654107+0000","last_active":"2023-10-16T09:59:59.640895+0000","last_peered":"2023-10-16T09:59:59.640895+0000","last_clean":"2023-10-16T09:59:59.640895+0000","last_became_active":"2023-10-16T09:59:58.653530+0000","last_became_peered":"2023-10-16T09:59:58.653530+0000","last_unstale":"2023-10-16T09:59:59.640895+0000","last_undegraded":"2023-10-16T09:59:59.640895+0000","last_fullsized":"2023-10-16T09:59:59.640895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:45.499639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.661133+0000","last_change":"2023-10-16T09:59:59.659845+0000","last_active":"2023-10-16T09:59:59.661133+0000","last_peered":"2023-10-16T09:59:59.661133+0000","last_clean":"2023-10-16T09:59:59.661133+0000","last_became_active":"2023-10-16T09:59:59.659498+0000","last_became_peered":"2023-10-16T09:59:59.659498+0000","last_unstale":"2023-10-16T09:59:59.661133+0000","last_undegraded":"2023-10-16T09:59:59.661133+0000","last_fullsized":"2023-10-16T09:59:59.661133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:09.158237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.647628+0000","last_change":"2023-10-16T09:59:58.646289+0000","last_active":"2023-10-16T09:59:59.647628+0000","last_peered":"2023-10-16T09:59:59.647628+0000","last_clean":"2023-10-16T09:59:59.647628+0000","last_became_active":"2023-10-16T09:59:58.645543+0000","last_became_peered":"2023-10-16T09:59:58.645543+0000","last_unstale":"2023-10-16T09:59:59.647628+0000","last_undegraded":"2023-10-16T09:59:59.647628+0000","last_fullsized":"2023-10-16T09:59:59.647628+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:25.947204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665024+0000","last_change":"2023-10-16T09:59:59.662410+0000","last_active":"2023-10-16T09:59:59.665024+0000","last_peered":"2023-10-16T09:59:59.665024+0000","last_clean":"2023-10-16T09:59:59.665024+0000","last_became_active":"2023-10-16T09:59:59.662202+0000","last_became_peered":"2023-10-16T09:59:59.662202+0000","last_unstale":"2023-10-16T09:59:59.665024+0000","last_undegraded":"2023-10-16T09:59:59.665024+0000","last_fullsized":"2023-10-16T09:59:59.665024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:36:33.834448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.642967+0000","last_change":"2023-10-16T09:59:58.654363+0000","last_active":"2023-10-16T09:59:59.642967+0000","last_peered":"2023-10-16T09:59:59.642967+0000","last_clean":"2023-10-16T09:59:59.642967+0000","last_became_active":"2023-10-16T09:59:58.654084+0000","last_became_peered":"2023-10-16T09:59:58.654084+0000","last_unstale":"2023-10-16T09:59:59.642967+0000","last_undegraded":"2023-10-16T09:59:59.642967+0000","last_fullsized":"2023-10-16T09:59:59.642967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:20:21.709240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.665334+0000","last_change":"2023-10-16T09:59:59.663161+0000","last_active":"2023-10-16T09:59:59.665334+0000","last_peered":"2023-10-16T09:59:59.665334+0000","last_clean":"2023-10-16T09:59:59.665334+0000","last_became_active":"2023-10-16T09:59:59.662808+0000","last_became_peered":"2023-10-16T09:59:59.662808+0000","last_unstale":"2023-10-16T09:59:59.665334+0000","last_undegraded":"2023-10-16T09:59:59.665334+0000","last_fullsized":"2023-10-16T09:59:59.665334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:41:40.433385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.649493+0000","last_change":"2023-10-16T09:59:58.657388+0000","last_active":"2023-10-16T09:59:59.649493+0000","last_peered":"2023-10-16T09:59:59.649493+0000","last_clean":"2023-10-16T09:59:59.649493+0000","last_became_active":"2023-10-16T09:59:58.655857+0000","last_became_peered":"2023-10-16T09:59:58.655857+0000","last_unstale":"2023-10-16T09:59:59.649493+0000","last_undegraded":"2023-10-16T09:59:59.649493+0000","last_fullsized":"2023-10-16T09:59:59.649493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:45.499639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.667061+0000","last_change":"2023-10-16T09:59:59.664725+0000","last_active":"2023-10-16T09:59:59.667061+0000","last_peered":"2023-10-16T09:59:59.667061+0000","last_clean":"2023-10-16T09:59:59.667061+0000","last_became_active":"2023-10-16T09:59:59.660941+0000","last_became_peered":"2023-10-16T09:59:59.660941+0000","last_unstale":"2023-10-16T09:59:59.667061+0000","last_undegraded":"2023-10-16T09:59:59.667061+0000","last_fullsized":"2023-10-16T09:59:59.667061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:58.629700+0000","last_clean_scrub_stamp":"2023-10-16T09:59:58.629700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:07.563412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"89'2","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T09:59:59.976966+0000","last_change":"2023-10-16T09:59:58.658436+0000","last_active":"2023-10-16T09:59:59.976966+0000","last_peered":"2023-10-16T09:59:59.976966+0000","last_clean":"2023-10-16T09:59:59.976966+0000","last_became_active":"2023-10-16T09:59:58.658173+0000","last_became_peered":"2023-10-16T09:59:58.658173+0000","last_unstale":"2023-10-16T09:59:59.976966+0000","last_undegraded":"2023-10-16T09:59:59.976966+0000","last_fullsized":"2023-10-16T09:59:59.976966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:59:57.623523+0000","last_clean_scrub_stamp":"2023-10-16T09:59:57.623523+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:49:16.497643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"79'172","reported_seq":428,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T09:59:58.649970+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T09:59:58.649970+0000","last_peered":"2023-10-16T09:59:58.649970+0000","last_clean":"2023-10-16T09:59:58.649970+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T09:59:58.649970+0000","last_undegraded":"2023-10-16T09:59:58.649970+0000","last_fullsized":"2023-10-16T09:59:58.649970+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542338,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":1115757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.79200000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.25700000000000001},"max":{"1min":0.72899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.72499999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.80400000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.36499999999999999,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.81200000000000006},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.34399999999999997},"max":{"1min":0.755,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65200000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.27400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.76200000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64800000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.499}]},{"osd":5,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.318,"15min":0.312},"max":{"1min":0.70299999999999996,"5min":1.006,"15min":1.006},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.35599999999999998},"max":{"1min":0.79900000000000004,"5min":1.0169999999999999,"15min":1.0489999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Mon Oct 16 09:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68999999999999995,"5min":0.95199999999999996,"15min":1.373},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.39400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.71499999999999997,"5min":0.85999999999999999,"15min":1.4099999999999999},"last":0.57399999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542337,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":524865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56399999999999995,"15min":0.59499999999999997},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.26100000000000001},"max":{"1min":0.72299999999999998,"5min":0.84899999999999998,"15min":6.194},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57699999999999996,"15min":0.60099999999999998},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":6.1609999999999996},"last":0.58499999999999996}]},{"osd":1,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.28799999999999998},"max":{"1min":0.69299999999999995,"5min":0.78900000000000003,"15min":1.054},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.44400000000000001,"5min":0.39000000000000001,"15min":0.32400000000000001},"max":{"1min":0.72199999999999998,"5min":0.80600000000000005,"15min":0.876},"last":0.56499999999999995}]},{"osd":2,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.61099999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.86099999999999999,"15min":8.5600000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70199999999999996,"5min":0.83699999999999997,"15min":1.1060000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.39000000000000001,"5min":0.36499999999999999,"15min":0.36199999999999999},"max":{"1min":0.69899999999999995,"5min":0.82399999999999995,"15min":1.175},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58699999999999997,"15min":0.624},"min":{"1min":0.40200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79100000000000004,"5min":0.85399999999999998,"15min":6.2249999999999996},"last":0.622}]},{"osd":4,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.499,"15min":0.495},"min":{"1min":0.32200000000000001,"5min":0.317,"15min":0.30299999999999999},"max":{"1min":0.72399999999999998,"5min":0.74399999999999999,"15min":0.81299999999999994},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.377,"5min":0.33400000000000002,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.84699999999999998},"last":0.40999999999999998}]},{"osd":5,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77500000000000002,"5min":0.78800000000000003,"15min":1.1240000000000001},"last":0.435}]},{"osd":7,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.57399999999999995},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.31900000000000001},"max":{"1min":0.754,"5min":0.83499999999999996,"15min":0.879},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.50600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542337,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50312,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321528,"statfs":{"total":96636764160,"available":96585244672,"internally_reserved":0,"allocated":1101824,"data_stored":527001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.375,"15min":0.37},"min":{"1min":0.22500000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.54400000000000004,"5min":1.5,"15min":1.5},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.245,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.60099999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.33600000000000002}]},{"osd":2,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55600000000000005,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46300000000000002,"15min":0.44},"min":{"1min":0.28399999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.67300000000000004,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.376}]},{"osd":3,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.495,"15min":0.47199999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68000000000000005,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.443,"15min":0.436},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.65100000000000002,"5min":1.379,"15min":1.379},"last":0.29399999999999998}]},{"osd":4,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85699999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85899999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.49399999999999999}]},{"osd":5,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84999999999999998,"5min":1.26,"15min":1.26},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78700000000000003,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69699999999999995,"15min":0.67000000000000004},"min":{"1min":0.44500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92100000000000004,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64900000000000002},"min":{"1min":0.50600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92600000000000005,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.36799999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90600000000000003,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.443},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.433,"5min":0.42599999999999999,"15min":0.41499999999999998},"max":{"1min":0.86799999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.47799999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542338,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":1115141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.373,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.19900000000000001,"15min":0.191},"max":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":5.5090000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40000000000000002,"15min":0.42899999999999999},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.23100000000000001},"max":{"1min":0.49199999999999999,"5min":0.64900000000000002,"15min":6.4950000000000001},"last":0.23499999999999999}]},{"osd":2,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.40799999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51100000000000001,"5min":0.61799999999999999,"15min":0.69599999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42399999999999999,"15min":0.436},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57999999999999996,"5min":0.624,"15min":2.7559999999999998},"last":0.31900000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42499999999999999,"15min":0.46300000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.20699999999999999},"max":{"1min":0.55100000000000005,"5min":0.72999999999999998,"15min":8.3260000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47699999999999998,"15min":0.501},"min":{"1min":0.309,"5min":0.309,"15min":0.30099999999999999},"max":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":5.4809999999999999},"last":0.33800000000000002}]},{"osd":4,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.624,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.39600000000000002},"max":{"1min":0.84299999999999997,"5min":0.88700000000000001,"15min":6.4349999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63100000000000001,"15min":0.66400000000000003},"min":{"1min":0.434,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":0.98799999999999999,"15min":8.2970000000000006},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64200000000000002,"15min":0.67000000000000004},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":5.4160000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64400000000000002,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78000000000000003,"5min":0.96999999999999997,"15min":6.3869999999999996},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.32900000000000001},"max":{"1min":0.75900000000000001,"5min":0.92500000000000004,"15min":8.2189999999999994},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.85699999999999998,"5min":1.0089999999999999,"15min":5.4560000000000004},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:59:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66200000000000003,"15min":0.69999999999999996},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":1.0249999999999999,"15min":6.4649999999999999},"last":0.63},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.69099999999999995},"min":{"1min":0.47399999999999998,"5min":0.38700000000000001,"15min":0.34899999999999998},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":8.266},"last":0.53700000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542337,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1093632,"data_stored":526385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38200000000000001,"15min":0.374},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.45300000000000001,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.39800000000000002,"15min":0.39900000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.621,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42599999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40999999999999998,"15min":0.40600000000000003},"min":{"1min":0.26400000000000001,"5min":0.245,"15min":0.24299999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.214},"max":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.64000000000000001},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40699999999999997,"15min":0.40799999999999997},"min":{"1min":0.315,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.5,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.63100000000000001,"5min":0.747,"15min":0.747},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91600000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.625,"15min":0.626},"min":{"1min":0.41999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73799999999999999,"5min":1.091,"15min":1.091},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.40699999999999997,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.94099999999999995,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.90600000000000003,"15min":0.97499999999999998},"last":0.57199999999999995}]},{"osd":6,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76000000000000001,"5min":0.93799999999999994,"15min":0.97899999999999998},"last":0.55700000000000005}]},{"osd":7,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90000000000000002,"5min":0.98299999999999998,"15min":0.98799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77400000000000002,"5min":0.874,"15min":0.998},"last":0.63800000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542338,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335436,"statfs":{"total":96636764160,"available":96599486464,"internally_reserved":0,"allocated":1089536,"data_stored":524885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38700000000000001,"15min":0.38400000000000001},"min":{"1min":0.23000000000000001,"5min":0.215,"15min":0.192},"max":{"1min":0.58099999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.26900000000000002,"5min":0.185,"15min":0.185},"max":{"1min":0.626,"5min":0.63100000000000001,"15min":1.0780000000000001},"last":0.32500000000000001}]},{"osd":1,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40100000000000002,"15min":0.39800000000000002},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.72199999999999998,"15min":2.254},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.246},"max":{"1min":0.69799999999999995,"5min":0.754,"15min":6.2240000000000002},"last":0.28199999999999997}]},{"osd":2,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.25},"max":{"1min":0.60599999999999998,"5min":0.70999999999999996,"15min":2.2909999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.28199999999999997},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.72299999999999998},"last":0.315}]},{"osd":4,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.88700000000000001,"15min":1.24},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.314,"5min":0.311,"15min":0.311},"max":{"1min":0.75600000000000001,"5min":1.0820000000000001,"15min":2.4420000000000002},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.627},"min":{"1min":0.374,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79200000000000004,"5min":0.97599999999999998,"15min":2.4089999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.38600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.996},"last":0.54400000000000004}]},{"osd":6,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63700000000000001,"15min":0.64200000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.89100000000000001,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.40200000000000002,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.99099999999999999,"15min":1.1619999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Mon Oct 16 09:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86199999999999999,"5min":1.3,"15min":1.3},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85099999999999998,"5min":1.4019999999999999,"15min":2.3740000000000001},"last":0.53400000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542337,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54448,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317392,"statfs":{"total":96636764160,"available":96581009408,"internally_reserved":0,"allocated":1085440,"data_stored":524843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.73699999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.58499999999999996},"min":{"1min":0.32900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73699999999999999,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57499999999999996,"15min":0.58399999999999996},"min":{"1min":0.35599999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72799999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56000000000000005,"15min":0.57499999999999996},"min":{"1min":0.39900000000000002,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.69999999999999996,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.69899999999999995,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72799999999999998,"5min":0.81999999999999995,"15min":0.89100000000000001},"last":0.65800000000000003}]},{"osd":3,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.86499999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73199999999999998,"5min":0.76800000000000002,"15min":0.81000000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.51000000000000001,"15min":0.52000000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71499999999999997,"5min":0.76400000000000001,"15min":0.86899999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51100000000000001,"15min":0.51400000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65600000000000003,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.76500000000000001,"5min":0.85899999999999999,"15min":0.85999999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Mon Oct 16 09:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.753,"5min":0.80700000000000005,"15min":0.84599999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":0.74099999999999999,"5min":0.84599999999999997,"15min":0.95999999999999996},"last":0.53300000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542338,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1069056,"data_stored":517574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.40600000000000003,"5min":0.33600000000000002,"15min":0.253},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.92900000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.629},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79500000000000004,"5min":0.84199999999999997,"15min":8.0370000000000008},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39100000000000001,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.98599999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.307},"max":{"1min":0.83499999999999996,"5min":0.92300000000000004,"15min":8.2110000000000003},"last":0.57699999999999996}]},{"osd":2,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.42099999999999999,"5min":0.315,"15min":0.29899999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":8.1140000000000008},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.61899999999999999},"min":{"1min":0.379,"5min":0.376,"15min":0.33900000000000002},"max":{"1min":0.79400000000000004,"5min":0.88700000000000001,"15min":8.3000000000000007},"last":0.49299999999999999}]},{"osd":3,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.34599999999999997,"15min":0.312},"max":{"1min":0.83899999999999997,"5min":0.88600000000000001,"15min":1.03},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.64900000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":9.8810000000000002},"last":0.63900000000000001}]},{"osd":4,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.496,"15min":0.499},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.26000000000000001},"max":{"1min":0.69899999999999995,"5min":0.77400000000000002,"15min":1.0449999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":1.0780000000000001},"last":0.625}]},{"osd":6,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.53000000000000003},"min":{"1min":0.33100000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.78700000000000003,"15min":1.599},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.745,"5min":0.85799999999999998,"15min":1.006},"last":0.60999999999999999}]},{"osd":7,"last update":"Mon Oct 16 09:59:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.747,"5min":0.95199999999999996,"15min":1.1339999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":0.90700000000000003,"15min":1.889},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:00:04.473 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:00:04.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:00:04.624 INFO:tasks.ceph.mds.a.smithi181.stderr:2023-10-16T10:00:04.624+0000 7f6ff296b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:00:04.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f593e6700 1 Processor -- start 2023-10-16T10:00:04.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f593e6700 1 -- start start 2023-10-16T10:00:04.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f593e6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 0x7f4f540cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f593e6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f5406d2a0 0x7f4f540cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f593e6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5415ea60 con 0x7f4f54071460 2023-10-16T10:00:04.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f593e6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f540cb390 con 0x7f4f5406d2a0 2023-10-16T10:00:04.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f593e6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54071840 con 0x7f4f5415db30 2023-10-16T10:00:04.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f52ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4f54071460 0x7f4f54159e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54282/0 (socket says 172.21.15.181:54282) 2023-10-16T10:00:04.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.634+0000 7f4f52ffd700 1 -- 172.21.15.181:0/2008229665 learned_addr learned my addr 172.21.15.181:0/2008229665 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:04.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643639128 0 0) 0x7f4f5415ea60 con 0x7f4f54071460 2023-10-16T10:00:04.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f527fc700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 0x7f4f540cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f537fe700 1 --2- 172.21.15.181:0/2008229665 >> v2:172.21.15.190:3300/0 conn(0x7f4f5406d2a0 0x7f4f540cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f3c0035b0 con 0x7f4f54071460 2023-10-16T10:00:04.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272972506 0 0) 0x7f4f3c0035b0 con 0x7f4f54071460 2023-10-16T10:00:04.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 msgr2=0x7f4f540cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:04.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 0x7f4f540cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 >> v2:172.21.15.190:3300/0 conn(0x7f4f5406d2a0 msgr2=0x7f4f540cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:04.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 --2- 172.21.15.181:0/2008229665 >> v2:172.21.15.190:3300/0 conn(0x7f4f5406d2a0 0x7f4f540cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5415e1c0 con 0x7f4f54071460 2023-10-16T10:00:04.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4f48001670 con 0x7f4f54071460 2023-10-16T10:00:04.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 >> v1:172.21.15.181:6789/0 conn(0x7f4f54071460 legacy=0x7f4f54159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f3c003a00 0x7f4f3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 0x7f4f3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c004d90 0x7f4f3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540cb390 con 0x7f4f3c003a00 2023-10-16T10:00:04.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54071840 con 0x7f4f3c004d90 2023-10-16T10:00:04.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c00ba90 con 0x7f4f5415db30 2023-10-16T10:00:04.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f52ffd700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f3c003a00 0x7f4f3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f527fc700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 0x7f4f3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f537fe700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c004d90 0x7f4f3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f52ffd700 1 -- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 msgr2=0x7f4f3c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:04.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f52ffd700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 0x7f4f3c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f52ffd700 1 -- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c004d90 msgr2=0x7f4f3c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:04.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f52ffd700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c004d90 0x7f4f3c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f52ffd700 1 -- 172.21.15.181:0/2008229665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c0035b0 con 0x7f4f3c003a00 2023-10-16T10:00:04.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f537fe700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3c004d90 0x7f4f3c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:04.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.635+0000 7f4f527fc700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f5415db30 0x7f4f3c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:04.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f52ffd700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f3c003a00 0x7f4f3c003e20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c320 tx=0x7f4f3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c20f1317cf26b50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:04.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800ce60 con 0x7f4f3c003a00 2023-10-16T10:00:04.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48005960 con 0x7f4f3c003a00 2023-10-16T10:00:04.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f51ffb700 1 -- 172.21.15.181:0/2008229665 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48005d10 con 0x7f4f3c003a00 2023-10-16T10:00:04.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f593e6700 1 -- 172.21.15.181:0/2008229665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f3c003a00 msgr2=0x7f4f3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f593e6700 1 --2- 172.21.15.181:0/2008229665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f3c003a00 0x7f4f3c003e20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c320 tx=0x7f4f3c005880 comp rx=0 tx=0).stop 2023-10-16T10:00:04.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f537fe700 1 -- 172.21.15.181:0/2008229665 reap_dead start 2023-10-16T10:00:04.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f593e6700 1 -- 172.21.15.181:0/2008229665 shutdown_connections 2023-10-16T10:00:04.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f593e6700 1 -- 172.21.15.181:0/2008229665 >> 172.21.15.181:0/2008229665 conn(0x7f4f5406aed0 msgr2=0x7f4f5406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:04.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f593e6700 1 -- 172.21.15.181:0/2008229665 shutdown_connections 2023-10-16T10:00:04.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.636+0000 7f4f593e6700 1 -- 172.21.15.181:0/2008229665 wait complete. 2023-10-16T10:00:04.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 Processor -- start 2023-10-16T10:00:04.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 -- start start 2023-10-16T10:00:04.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3c003a00 0x7f4f540b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54154140 0x7f4f540c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f54158490 0x7f4f54158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540b20e0 con 0x7f4f54154140 2023-10-16T10:00:04.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f54158e50 con 0x7f4f54158490 2023-10-16T10:00:04.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.638+0000 7f4f593e6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f541545c0 con 0x7f4f3c003a00 2023-10-16T10:00:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54154140 0x7f4f540c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54154140 0x7f4f540c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43218/0 (socket says 172.21.15.181:43218) 2023-10-16T10:00:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 -- 172.21.15.181:0/2389965367 learned_addr learned my addr 172.21.15.181:0/2389965367 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:04.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f52ffd700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3c003a00 0x7f4f540b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f537fe700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f54158490 0x7f4f54158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 -- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3c003a00 msgr2=0x7f4f540b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3c003a00 0x7f4f540b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 -- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f54158490 msgr2=0x7f4f54158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f54158490 0x7f4f54158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 -- 172.21.15.181:0/2389965367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540b0100 con 0x7f4f54154140 2023-10-16T10:00:04.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f537fe700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f54158490 0x7f4f54158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:04.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f527fc700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54154140 0x7f4f540c6fa0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4f440118d0 tx=0x7f4f44011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:04.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f3b7fe700 1 -- 172.21.15.181:0/2389965367 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44004220 con 0x7f4f54154140 2023-10-16T10:00:04.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f3b7fe700 1 -- 172.21.15.181:0/2389965367 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f44004380 con 0x7f4f54154140 2023-10-16T10:00:04.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f540afb40 con 0x7f4f54154140 2023-10-16T10:00:04.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f3b7fe700 1 -- 172.21.15.181:0/2389965367 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440036a0 con 0x7f4f54154140 2023-10-16T10:00:04.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.639+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f540b04d0 con 0x7f4f54154140 2023-10-16T10:00:04.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.640+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f540b0b20 con 0x7f4f54154140 2023-10-16T10:00:04.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.641+0000 7f4f3b7fe700 1 -- 172.21.15.181:0/2389965367 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4f44003820 con 0x7f4f54154140 2023-10-16T10:00:04.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.641+0000 7f4f3b7fe700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f4f4006b020 0x7f4f4006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:04.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.641+0000 7f4f3b7fe700 1 -- 172.21.15.181:0/2389965367 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f4f44089d20 con 0x7f4f54154140 2023-10-16T10:00:04.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.641+0000 7f4f52ffd700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f4f4006b020 0x7f4f4006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:04.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.641+0000 7f4f52ffd700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f4f4006b020 0x7f4f4006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f48008fd0 tx=0x7f4f4800a000 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:04.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.645+0000 7f4f3b7fe700 1 -- 172.21.15.181:0/2389965367 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f44054b10 con 0x7f4f54154140 2023-10-16T10:00:04.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.920+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4f5404ebb0 con 0x7f4f54154140 2023-10-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.923+0000 7f4f3b7fe700 1 -- 172.21.15.181:0/2389965367 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4f440220e0 con 0x7f4f54154140 2023-10-16T10:00:04.924 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T10:00:04.924 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T10:00:04.924 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T10:00:04.925 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T10:00:04.925 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:00:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f4f4006b020 msgr2=0x7f4f4006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f4f4006b020 0x7f4f4006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f48008fd0 tx=0x7f4f4800a000 comp rx=0 tx=0).stop 2023-10-16T10:00:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54154140 msgr2=0x7f4f540c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54154140 0x7f4f540c6fa0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4f440118d0 tx=0x7f4f44011be0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 shutdown_connections 2023-10-16T10:00:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f4f4006b020 0x7f4f4006d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3c003a00 0x7f4f540b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f54158490 0x7f4f54158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 --2- 172.21.15.181:0/2389965367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54154140 0x7f4f540c6fa0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 >> 172.21.15.181:0/2389965367 conn(0x7f4f5406aed0 msgr2=0x7f4f540cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 shutdown_connections 2023-10-16T10:00:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:04.926+0000 7f4f593e6700 1 -- 172.21.15.181:0/2389965367 wait complete. 2023-10-16T10:00:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T10:00:04.939 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3622548700 1 Processor -- start 2023-10-16T10:00:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3622548700 1 -- start start 2023-10-16T10:00:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3622548700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 0x7f361c06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3622548700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f361c0d0350 0x7f361c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3622548700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c0cc470 con 0x7f361c0d2390 2023-10-16T10:00:05.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3622548700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c06ca30 con 0x7f361c0d0350 2023-10-16T10:00:05.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3622548700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c06cd10 con 0x7f361c0ce7f0 2023-10-16T10:00:05.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f3620ae5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f361c0d0350 0x7f361c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.060+0000 7f361b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f361c0d2390 0x7f361c0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54304/0 (socket says 172.21.15.181:54304) 2023-10-16T10:00:05.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f361b7fe700 1 -- 172.21.15.181:0/44451249 learned_addr learned my addr 172.21.15.181:0/44451249 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:05.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f3620ae5700 1 -- 172.21.15.181:0/44451249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 msgr2=0x7f361c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f3620ae5700 1 --2- 172.21.15.181:0/44451249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 0x7f361c06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f3620ae5700 1 -- 172.21.15.181:0/44451249 >> v1:172.21.15.181:6789/0 conn(0x7f361c0d2390 legacy=0x7f361c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f3620ae5700 1 -- 172.21.15.181:0/44451249 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c06cff0 con 0x7f361c0d0350 2023-10-16T10:00:05.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f361affd700 1 -- 172.21.15.181:0/44451249 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500063111 0 0) 0x7f361c0cc470 con 0x7f361c0d2390 2023-10-16T10:00:05.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f3620ae5700 1 --2- 172.21.15.181:0/44451249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d0350 0x7f361c0cc930 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3614009d90 tx=0x7f361400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1122eedfd455fffb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f361affd700 1 -- 172.21.15.181:0/44451249 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400e070 con 0x7f361c0d0350 2023-10-16T10:00:05.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f361affd700 1 -- 172.21.15.181:0/44451249 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361400bbd0 con 0x7f361c0d0350 2023-10-16T10:00:05.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.061+0000 7f361affd700 1 -- 172.21.15.181:0/44451249 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614012430 con 0x7f361c0d0350 2023-10-16T10:00:05.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 -- 172.21.15.181:0/44451249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d0350 msgr2=0x7f361c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 --2- 172.21.15.181:0/44451249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d0350 0x7f361c0cc930 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3614009d90 tx=0x7f361400b330 comp rx=0 tx=0).stop 2023-10-16T10:00:05.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 -- 172.21.15.181:0/44451249 shutdown_connections 2023-10-16T10:00:05.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 --2- 172.21.15.181:0/44451249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d0350 0x7f361c0cc930 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 --2- 172.21.15.181:0/44451249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 0x7f361c06c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 -- 172.21.15.181:0/44451249 >> 172.21.15.181:0/44451249 conn(0x7f361c09a9a0 msgr2=0x7f361c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 -- 172.21.15.181:0/44451249 shutdown_connections 2023-10-16T10:00:05.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.062+0000 7f3622548700 1 -- 172.21.15.181:0/44451249 wait complete. 2023-10-16T10:00:05.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.063+0000 7f3622548700 1 Processor -- start 2023-10-16T10:00:05.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.063+0000 7f3622548700 1 -- start start 2023-10-16T10:00:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 0x7f361c0b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c0d0350 0x7f361c14ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d2390 0x7f361c14d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0b42a0 con 0x7f361c0d2390 2023-10-16T10:00:05.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0b44f0 con 0x7f361c0d0350 2023-10-16T10:00:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0b47d0 con 0x7f361c0ce7f0 2023-10-16T10:00:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d2390 0x7f361c14d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d2390 0x7f361c14d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43234/0 (socket says 172.21.15.181:43234) 2023-10-16T10:00:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 -- 172.21.15.181:0/1121348855 learned_addr learned my addr 172.21.15.181:0/1121348855 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 -- 172.21.15.181:0/1121348855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 msgr2=0x7f361c0b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 0x7f361c0b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 -- 172.21.15.181:0/1121348855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c0d0350 msgr2=0x7f361c14ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f361b7fe700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c0d0350 0x7f361c14ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c0d0350 0x7f361c14ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 -- 172.21.15.181:0/1121348855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009a70 con 0x7f361c0d2390 2023-10-16T10:00:05.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3620ae5700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d2390 0x7f361c14d400 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f361400b370 tx=0x7f3614004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3618ff9700 1 -- 172.21.15.181:0/1121348855 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614003ea0 con 0x7f361c0d2390 2023-10-16T10:00:05.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3618ff9700 1 -- 172.21.15.181:0/1121348855 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3614020070 con 0x7f361c0d2390 2023-10-16T10:00:05.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3618ff9700 1 -- 172.21.15.181:0/1121348855 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614004280 con 0x7f361c0d2390 2023-10-16T10:00:05.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c0b4ab0 con 0x7f361c0d2390 2023-10-16T10:00:05.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.064+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c0b5070 con 0x7f361c0d2390 2023-10-16T10:00:05.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.065+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c067d20 con 0x7f361c0d2390 2023-10-16T10:00:05.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.066+0000 7f3618ff9700 1 -- 172.21.15.181:0/1121348855 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f36140043e0 con 0x7f361c0d2390 2023-10-16T10:00:05.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.066+0000 7f3618ff9700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f35fc06b050 0x7f35fc06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.066+0000 7f361bfff700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f35fc06b050 0x7f35fc06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.066+0000 7f3618ff9700 1 -- 172.21.15.181:0/1121348855 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f3614022070 con 0x7f361c0d2390 2023-10-16T10:00:05.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.067+0000 7f361bfff700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f35fc06b050 0x7f35fc06d510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f360c0096f0 tx=0x7f360c006c70 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.071+0000 7f3618ff9700 1 -- 172.21.15.181:0/1121348855 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3614056610 con 0x7f361c0d2390 2023-10-16T10:00:05.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.305+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f361c0cdbf0 con 0x7f361c0d2390 2023-10-16T10:00:05.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.305+0000 7f3618ff9700 1 -- 172.21.15.181:0/1121348855 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f3614059c30 con 0x7f361c0d2390 2023-10-16T10:00:05.306 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:05.307 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3576975018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3576975018},{"type":"v1","addr":"172.21.15.190:6834","nonce":3576975018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/54128065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":54128065},{"type":"v1","addr":"172.21.15.190:6835","nonce":54128065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/509615642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":509615642},{"type":"v1","addr":"172.21.15.181:6836","nonce":509615642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:59.043602+0000","modified":"2023-10-16T10:00:02.379610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2780127849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2780127849},{"type":"v1","addr":"172.21.15.181:6837","nonce":2780127849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:00:05.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f35fc06b050 msgr2=0x7f35fc06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f35fc06b050 0x7f35fc06d510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f360c0096f0 tx=0x7f360c006c70 comp rx=0 tx=0).stop 2023-10-16T10:00:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d2390 msgr2=0x7f361c14d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d2390 0x7f361c14d400 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f361400b370 tx=0x7f3614004ad0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 shutdown_connections 2023-10-16T10:00:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f35fc06b050 0x7f35fc06d510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0ce7f0 0x7f361c0b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c0d2390 0x7f361c14d400 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 --2- 172.21.15.181:0/1121348855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c0d0350 0x7f361c14ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 >> 172.21.15.181:0/1121348855 conn(0x7f361c09a9a0 msgr2=0x7f361c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 shutdown_connections 2023-10-16T10:00:05.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.307+0000 7f3622548700 1 -- 172.21.15.181:0/1121348855 wait complete. 2023-10-16T10:00:05.311 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 40 2023-10-16T10:00:05.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T09:59:59.043602+0000', 'modified': '2023-10-16T10:00:02.379610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2780127849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 2780127849}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2780127849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:00:05.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:00:05.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T10:00:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.851+0000 7fdad9c68700 1 Processor -- start 2023-10-16T10:00:05.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.851+0000 7fdad9c68700 1 -- start start 2023-10-16T10:00:05.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.852+0000 7fdad9c68700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdad415b7b0 0x7fdad40cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.852+0000 7fdad9c68700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad40665a0 0x7fdad406e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.852+0000 7fdad9c68700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad415f590 con 0x7fdad40b2310 2023-10-16T10:00:05.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.852+0000 7fdad9c68700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad4066a30 con 0x7fdad415b7b0 2023-10-16T10:00:05.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.852+0000 7fdad9c68700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad415f850 con 0x7fdad40665a0 2023-10-16T10:00:05.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.852+0000 7fdad3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdad40b2310 0x7fdad406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54320/0 (socket says 172.21.15.181:54320) 2023-10-16T10:00:05.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.852+0000 7fdad3fff700 1 -- 172.21.15.181:0/1286487222 learned_addr learned my addr 172.21.15.181:0/1286487222 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:05.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558703690 0 0) 0x7fdad415f590 con 0x7fdad40b2310 2023-10-16T10:00:05.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdabc0035b0 con 0x7fdad40b2310 2023-10-16T10:00:05.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad40665a0 0x7fdad406e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad37fe700 1 --2- 172.21.15.181:0/1286487222 >> v2:172.21.15.190:3300/0 conn(0x7fdad415b7b0 0x7fdad40cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 400242458 0 0) 0x7fdabc0035b0 con 0x7fdad40b2310 2023-10-16T10:00:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad40665a0 msgr2=0x7fdad406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdad40665a0 0x7fdad406e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 >> v2:172.21.15.190:3300/0 conn(0x7fdad415b7b0 msgr2=0x7fdad40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 --2- 172.21.15.181:0/1286487222 >> v2:172.21.15.190:3300/0 conn(0x7fdad415b7b0 0x7fdad40cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad415fb30 con 0x7fdad40b2310 2023-10-16T10:00:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fdacc0019f0 con 0x7fdad40b2310 2023-10-16T10:00:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 >> v1:172.21.15.181:6789/0 conn(0x7fdad40b2310 legacy=0x7fdad406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdabc003a00 0x7fdabc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdabc004850 0x7fdabc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc005210 0x7fdabc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4066a30 con 0x7fdabc004850 2023-10-16T10:00:05.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad415f850 con 0x7fdabc003a00 2023-10-16T10:00:05.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc005700 con 0x7fdabc005210 2023-10-16T10:00:05.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdabc004850 0x7fdabc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad37fe700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdabc003a00 0x7fdabc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad3fff700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc005210 0x7fdabc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 -- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc005210 msgr2=0x7fdabc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc005210 0x7fdabc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 -- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdabc003a00 msgr2=0x7fdabc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdabc003a00 0x7fdabc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 -- 172.21.15.181:0/1286487222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0035b0 con 0x7fdabc004850 2023-10-16T10:00:05.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.853+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdabc004850 0x7fdabc004cd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009b30 tx=0x7fdabc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e466d8d8a6c183a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400e6c0 con 0x7fdabc004850 2023-10-16T10:00:05.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400e820 con 0x7fdabc004850 2023-10-16T10:00:05.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad27fc700 1 -- 172.21.15.181:0/1286487222 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400eb90 con 0x7fdabc004850 2023-10-16T10:00:05.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad9c68700 1 -- 172.21.15.181:0/1286487222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdabc004850 msgr2=0x7fdabc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad9c68700 1 --2- 172.21.15.181:0/1286487222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdabc004850 0x7fdabc004cd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009b30 tx=0x7fdabc005880 comp rx=0 tx=0).stop 2023-10-16T10:00:05.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad3fff700 1 -- 172.21.15.181:0/1286487222 reap_dead start 2023-10-16T10:00:05.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad9c68700 1 -- 172.21.15.181:0/1286487222 shutdown_connections 2023-10-16T10:00:05.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad9c68700 1 -- 172.21.15.181:0/1286487222 >> 172.21.15.181:0/1286487222 conn(0x7fdad40b1780 msgr2=0x7fdad40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad9c68700 1 -- 172.21.15.181:0/1286487222 shutdown_connections 2023-10-16T10:00:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.854+0000 7fdad9c68700 1 -- 172.21.15.181:0/1286487222 wait complete. 2023-10-16T10:00:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.856+0000 7fdad9c68700 1 Processor -- start 2023-10-16T10:00:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.856+0000 7fdad9c68700 1 -- start start 2023-10-16T10:00:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad9c68700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc004850 0x7fdad4157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad9c68700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad415d980 0x7fdad415de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad9c68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad41596b0 0x7fdad4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad9c68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad415a070 con 0x7fdad41596b0 2023-10-16T10:00:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad9c68700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40b5560 con 0x7fdad415d980 2023-10-16T10:00:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad9c68700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40b4950 con 0x7fdabc004850 2023-10-16T10:00:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad41596b0 0x7fdad4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad41596b0 0x7fdad4159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43254/0 (socket says 172.21.15.181:43254) 2023-10-16T10:00:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 -- 172.21.15.181:0/2793078926 learned_addr learned my addr 172.21.15.181:0/2793078926 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 -- 172.21.15.181:0/2793078926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc004850 msgr2=0x7fdad4157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc004850 0x7fdad4157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 -- 172.21.15.181:0/2793078926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad415d980 msgr2=0x7fdad415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad415d980 0x7fdad415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 -- 172.21.15.181:0/2793078926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40b4c30 con 0x7fdad41596b0 2023-10-16T10:00:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.857+0000 7fdad3fff700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad41596b0 0x7fdad4159b30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdad40ca960 tx=0x7fdacc007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.858+0000 7fdabbfff700 1 -- 172.21.15.181:0/2793078926 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc017070 con 0x7fdad41596b0 2023-10-16T10:00:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.858+0000 7fdabbfff700 1 -- 172.21.15.181:0/2793078926 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc00ed90 con 0x7fdad41596b0 2023-10-16T10:00:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.858+0000 7fdabbfff700 1 -- 172.21.15.181:0/2793078926 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc013420 con 0x7fdad41596b0 2023-10-16T10:00:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.858+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad40b4f10 con 0x7fdad41596b0 2023-10-16T10:00:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.858+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad40b4620 con 0x7fdad41596b0 2023-10-16T10:00:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.859+0000 7fdabbfff700 1 -- 172.21.15.181:0/2793078926 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdacc013580 con 0x7fdad41596b0 2023-10-16T10:00:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.859+0000 7fdabbfff700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdac006b020 0x7fdac006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.860+0000 7fdabbfff700 1 -- 172.21.15.181:0/2793078926 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fdacc089700 con 0x7fdad41596b0 2023-10-16T10:00:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.860+0000 7fdad37fe700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdac006b020 0x7fdac006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.860+0000 7fdad37fe700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdac006b020 0x7fdac006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006f10 tx=0x7fdac8006e80 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.861+0000 7fdae5e52b80 1 Processor -- start 2023-10-16T10:00:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.861+0000 7fdae5e52b80 1 -- start start 2023-10-16T10:00:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58789b40 0x564e58789f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e587b0f30 0x564e58789280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 0x564e587b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e586fdc20 con 0x564e58789b40 2023-10-16T10:00:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e586e2c10 con 0x564e587b0f30 2023-10-16T10:00:05.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e586e0530 con 0x564e58784f00 2023-10-16T10:00:05.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdad37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58789b40 0x564e58789f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.862+0000 7fdad37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58789b40 0x564e58789f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43258/0 (socket says 172.21.15.181:43258) 2023-10-16T10:00:05.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdad37fe700 1 -- 172.21.15.181:0/582405992 learned_addr learned my addr 172.21.15.181:0/582405992 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:05.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdad37fe700 1 -- 172.21.15.181:0/582405992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 msgr2=0x564e587b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdad37fe700 1 --2- 172.21.15.181:0/582405992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 0x564e587b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdad37fe700 1 -- 172.21.15.181:0/582405992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e587b0f30 msgr2=0x564e58789280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdad37fe700 1 --2- 172.21.15.181:0/582405992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e587b0f30 0x564e58789280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdad37fe700 1 -- 172.21.15.181:0/582405992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e586993a0 con 0x564e58789b40 2023-10-16T10:00:05.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdad37fe700 1 --2- 172.21.15.181:0/582405992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58789b40 0x564e58789f20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006b00 tx=0x7fdac8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b508f257e8ae54 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdab97fa700 1 -- 172.21.15.181:0/582405992 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004560 con 0x564e58789b40 2023-10-16T10:00:05.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdab97fa700 1 -- 172.21.15.181:0/582405992 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac801b070 con 0x564e58789b40 2023-10-16T10:00:05.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.863+0000 7fdab97fa700 1 -- 172.21.15.181:0/582405992 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003ad0 con 0x564e58789b40 2023-10-16T10:00:05.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 -- 172.21.15.181:0/582405992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58789b40 msgr2=0x564e58789f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/582405992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58789b40 0x564e58789f20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006b00 tx=0x7fdac8014000 comp rx=0 tx=0).stop 2023-10-16T10:00:05.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 -- 172.21.15.181:0/582405992 shutdown_connections 2023-10-16T10:00:05.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/582405992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 0x564e587b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/582405992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58789b40 0x564e58789f20 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/582405992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e587b0f30 0x564e58789280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 -- 172.21.15.181:0/582405992 >> 172.21.15.181:0/582405992 conn(0x564e586b6d80 msgr2=0x564e587d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 -- 172.21.15.181:0/582405992 shutdown_connections 2023-10-16T10:00:05.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 -- 172.21.15.181:0/582405992 wait complete. 2023-10-16T10:00:05.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.864+0000 7fdae5e52b80 1 -- start start 2023-10-16T10:00:05.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 1 Processor -- start 2023-10-16T10:00:05.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 0x564e587ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e586dbe20 con 0x564e587b0f30 2023-10-16T10:00:05.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e586786c0 con 0x564e58789b40 2023-10-16T10:00:05.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e58678360 con 0x564e58784f00 2023-10-16T10:00:05.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdae5e52b80 10 client.0 _ll_register_callbacks cb 0x564e584d57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:00:05.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43268/0 (socket says 172.21.15.181:43268) 2023-10-16T10:00:05.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdad3fff700 1 -- 172.21.15.181:0/3602242164 learned_addr learned my addr 172.21.15.181:0/3602242164 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:05.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.865+0000 7fdad37fe700 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 0x564e587ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdad3fff700 1 -- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 msgr2=0x564e587ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdad3fff700 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 0x564e587ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdad3fff700 1 -- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 msgr2=0x564e587b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdad3fff700 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdad3fff700 1 -- 172.21.15.181:0/3602242164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac8004240 con 0x564e587b0f30 2023-10-16T10:00:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdad3fff700 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587aee30 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00a040 tx=0x7fdacc057c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d347e80cdeb0116 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdab27fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:00:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdab27fc700 1 -- 172.21.15.181:0/3602242164 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc087ea0 con 0x564e587b0f30 2023-10-16T10:00:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdab27fc700 1 -- 172.21.15.181:0/3602242164 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fdacc0581b0 con 0x564e587b0f30 2023-10-16T10:00:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e5877c5b0 con 0x564e587b0f30 2023-10-16T10:00:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdab27fc700 1 -- 172.21.15.181:0/3602242164 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdacc05c300 con 0x564e587b0f30 2023-10-16T10:00:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 10 client.7134 fetch_fsmap learned FSMap version 40 2023-10-16T10:00:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x564e58506d90 con 0x564e587b0f30 2023-10-16T10:00:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdab27fc700 1 -- 172.21.15.181:0/3602242164 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564e5877c5b0 con 0x564e587b0f30 2023-10-16T10:00:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdab27fc700 1 -- 172.21.15.181:0/3602242164 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdacc0576a0 con 0x564e587b0f30 2023-10-16T10:00:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 10 client.7134 fetch_fsmap finished waiting for FSMap version 40 2023-10-16T10:00:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 10 client.7134 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:00:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 20 client.7134 populate_metadata read hostname 'smithi181' 2023-10-16T10:00:05.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e587b3200 0x564e587b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 4 client.7134 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:00:05.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.866+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 --> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e586d4e00 con 0x564e587b3200 2023-10-16T10:00:05.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdad37fe700 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e587b3200 0x564e587b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdad37fe700 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e587b3200 0x564e587b53e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e4221573659f67 server_cookie=1a3fa643a87b383c in_seq=0 out_seq=0 2023-10-16T10:00:05.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdab27fc700 10 client.7134 ms_handle_connect on v2:172.21.15.181:6835/2780127849 2023-10-16T10:00:05.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdab27fc700 1 -- 172.21.15.181:0/3602242164 <== mds.0 v2:172.21.15.181:6835/2780127849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564e586d4e00 con 0x564e587b3200 2023-10-16T10:00:05.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdab27fc700 10 client.7134 handle_command_reply: tid=0 2023-10-16T10:00:05.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdae5e52b80 1 client.7134 shutdown 2023-10-16T10:00:05.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 msgr2=0x564e587aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.867+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587aee30 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00a040 tx=0x7fdacc057c00 comp rx=0 tx=0).stop 2023-10-16T10:00:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 shutdown_connections 2023-10-16T10:00:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e587b3200 0x564e587b53e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58784f00 0x564e587ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdab27fc700 0 client.7134 ms_handle_reset on v2:172.21.15.181:6835/2780127849 2023-10-16T10:00:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587aee30 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/3602242164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 >> 172.21.15.181:0/3602242164 conn(0x564e586b6d80 msgr2=0x564e587874b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 shutdown_connections 2023-10-16T10:00:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 1 -- 172.21.15.181:0/3602242164 wait complete. 2023-10-16T10:00:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.868+0000 7fdae5e52b80 20 client.7134 trim_cache size 0 max 16384 2023-10-16T10:00:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.885+0000 7fdae5e52b80 1 Processor -- start 2023-10-16T10:00:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.885+0000 7fdae5e52b80 1 -- start start 2023-10-16T10:00:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58784f00 0x564e58789f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58789b40 0x564e5877d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5855c8d0 con 0x564e587b0f30 2023-10-16T10:00:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5877dda0 con 0x564e58784f00 2023-10-16T10:00:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e58780610 con 0x564e58789b40 2023-10-16T10:00:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43276/0 (socket says 172.21.15.181:43276) 2023-10-16T10:00:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 -- 172.21.15.181:0/156434022 learned_addr learned my addr 172.21.15.181:0/156434022 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad37fe700 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58784f00 0x564e58789f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58789b40 0x564e5877d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 -- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58789b40 msgr2=0x564e5877d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58789b40 0x564e5877d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 -- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58784f00 msgr2=0x564e58789f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58784f00 0x564e58789f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.886+0000 7fdad3fff700 1 -- 172.21.15.181:0/156434022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e584efc10 con 0x564e587b0f30 2023-10-16T10:00:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdad3fff700 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587ca220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdacc056500 tx=0x7fdacc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c46935749cdc30b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdab9ffb700 1 -- 172.21.15.181:0/156434022 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc087ea0 con 0x564e587b0f30 2023-10-16T10:00:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdab9ffb700 1 -- 172.21.15.181:0/156434022 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc0581b0 con 0x564e587b0f30 2023-10-16T10:00:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdab9ffb700 1 -- 172.21.15.181:0/156434022 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc05c300 con 0x564e587b0f30 2023-10-16T10:00:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 -- 172.21.15.181:0/156434022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 msgr2=0x564e587ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587ca220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdacc056500 tx=0x7fdacc01d000 comp rx=0 tx=0).stop 2023-10-16T10:00:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 -- 172.21.15.181:0/156434022 shutdown_connections 2023-10-16T10:00:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e587b0f30 0x564e587ca220 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58784f00 0x564e58789f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/156434022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e58789b40 0x564e5877d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 -- 172.21.15.181:0/156434022 >> 172.21.15.181:0/156434022 conn(0x564e586b6d80 msgr2=0x564e58784440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 -- 172.21.15.181:0/156434022 shutdown_connections 2023-10-16T10:00:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.887+0000 7fdae5e52b80 1 -- 172.21.15.181:0/156434022 wait complete. 2023-10-16T10:00:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.888+0000 7fdae5e52b80 1 -- start start 2023-10-16T10:00:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.888+0000 7fdae5e52b80 1 Processor -- start 2023-10-16T10:00:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58784f00 0x564e587882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e587b0f30 0x564e587ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e58788a20 con 0x564e58784f00 2023-10-16T10:00:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e587aded0 con 0x564e58789b40 2023-10-16T10:00:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e587ebb50 con 0x564e587b0f30 2023-10-16T10:00:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 10 client.0 _ll_register_callbacks cb 0x7fdacc057cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:00:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58784f00 0x564e587882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58784f00 0x564e587882d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43278/0 (socket says 172.21.15.181:43278) 2023-10-16T10:00:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 -- 172.21.15.181:0/2232355841 learned_addr learned my addr 172.21.15.181:0/2232355841 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:05.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad3fff700 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e587b0f30 0x564e587ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 -- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e587b0f30 msgr2=0x564e587ebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e587b0f30 0x564e587ebfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 -- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 msgr2=0x564e587e4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587e4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 -- 172.21.15.181:0/2232355841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e58506d90 con 0x564e58784f00 2023-10-16T10:00:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdad37fe700 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58784f00 0x564e587882d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdac8024040 tx=0x7fdac8009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb14139ff9eee71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdab27fc700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:00:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdab27fc700 1 -- 172.21.15.181:0/2232355841 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8033040 con 0x564e58784f00 2023-10-16T10:00:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdab27fc700 1 -- 172.21.15.181:0/2232355841 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fdac8003a70 con 0x564e58784f00 2023-10-16T10:00:05.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdac801b070 con 0x564e58784f00 2023-10-16T10:00:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdab27fc700 1 -- 172.21.15.181:0/2232355841 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdac8003c70 con 0x564e58784f00 2023-10-16T10:00:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 10 client.7140 fetch_fsmap learned FSMap version 40 2023-10-16T10:00:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.889+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x564e5852d200 con 0x564e58784f00 2023-10-16T10:00:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdab27fc700 1 -- 172.21.15.181:0/2232355841 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac801b070 con 0x564e58784f00 2023-10-16T10:00:05.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdab27fc700 1 -- 172.21.15.181:0/2232355841 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdac803a020 con 0x564e58784f00 2023-10-16T10:00:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdae5e52b80 10 client.7140 fetch_fsmap finished waiting for FSMap version 40 2023-10-16T10:00:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdae5e52b80 10 client.7140 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:00:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdae5e52b80 20 client.7140 populate_metadata read hostname 'smithi181' 2023-10-16T10:00:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e5883d150 0x564e5883d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:05.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdae5e52b80 4 client.7140 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:00:05.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 --> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564e586d4b20 con 0x564e5883d150 2023-10-16T10:00:05.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e5883d150 0x564e5883d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:05.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdab27fc700 10 client.7140 ms_handle_connect on v2:172.21.15.181:6835/2780127849 2023-10-16T10:00:05.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdad2ffd700 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e5883d150 0x564e5883d530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6655dfe78a49d6 server_cookie=e276dd2ec98e755f in_seq=0 out_seq=0 2023-10-16T10:00:05.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdab27fc700 1 -- 172.21.15.181:0/2232355841 <== mds.0 v2:172.21.15.181:6835/2780127849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564e586d4b20 con 0x564e5883d150 2023-10-16T10:00:05.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdab27fc700 10 client.7140 handle_command_reply: tid=0 2023-10-16T10:00:05.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.890+0000 7fdae5e52b80 1 client.7140 shutdown 2023-10-16T10:00:05.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58784f00 msgr2=0x564e587882d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58784f00 0x564e587882d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdac8024040 tx=0x7fdac8009210 comp rx=0 tx=0).stop 2023-10-16T10:00:05.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 shutdown_connections 2023-10-16T10:00:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:6835/2780127849,v1:172.21.15.181:6837/2780127849] conn(0x564e5883d150 0x564e5883d530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564e587b0f30 0x564e587ebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564e58784f00 0x564e587882d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 --2- 172.21.15.181:0/2232355841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564e58789b40 0x564e587e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdab27fc700 0 client.7140 ms_handle_reset on v2:172.21.15.181:6835/2780127849 2023-10-16T10:00:05.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 >> 172.21.15.181:0/2232355841 conn(0x564e586b6d80 msgr2=0x564e5877df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 shutdown_connections 2023-10-16T10:00:05.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 1 -- 172.21.15.181:0/2232355841 wait complete. 2023-10-16T10:00:05.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdae5e52b80 20 client.7140 trim_cache size 0 max 16384 2023-10-16T10:00:05.919 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:00:05.919 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T10:00:05.919 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T10:00:05.919 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 6987, 2023-10-16T10:00:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T10:00:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T10:00:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T10:00:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9234527100000003, 2023-10-16T10:00:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 40, 2023-10-16T10:00:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 90, 2023-10-16T10:00:05.921 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 90, 2023-10-16T10:00:05.921 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 12.191289488000001 2023-10-16T10:00:05.921 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:00:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdac006b020 msgr2=0x7fdac006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdac006b020 0x7fdac006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006f10 tx=0x7fdac8006e80 comp rx=0 tx=0).stop 2023-10-16T10:00:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad41596b0 msgr2=0x7fdad4159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad41596b0 0x7fdad4159b30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdad40ca960 tx=0x7fdacc007c90 comp rx=0 tx=0).stop 2023-10-16T10:00:05.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 shutdown_connections 2023-10-16T10:00:05.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdac006b020 0x7fdac006d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdabc004850 0x7fdad4157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad41596b0 0x7fdad4159b30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 --2- 172.21.15.181:0/2793078926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdad415d980 0x7fdad415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:05.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 >> 172.21.15.181:0/2793078926 conn(0x7fdad40b1780 msgr2=0x7fdad4158950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:05.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 shutdown_connections 2023-10-16T10:00:05.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:05.891+0000 7fdad9c68700 1 -- 172.21.15.181:0/2793078926 wait complete. 2023-10-16T10:00:05.925 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:00:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.046+0000 7f7249cb8700 1 Processor -- start 2023-10-16T10:00:06.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.046+0000 7f7249cb8700 1 -- start start 2023-10-16T10:00:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7249cb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 0x7f72440af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7249cb8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f72440af450 0x7f724406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7249cb8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724406f870 con 0x7f724406db80 2023-10-16T10:00:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7249cb8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72440d1160 con 0x7f72440af450 2023-10-16T10:00:06.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7249cb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72440cc8e0 con 0x7f724415dfd0 2023-10-16T10:00:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7243fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f724406db80 0x7f72440cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54368/0 (socket says 172.21.15.181:54368) 2023-10-16T10:00:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7243fff700 1 -- 172.21.15.181:0/3756998021 learned_addr learned my addr 172.21.15.181:0/3756998021 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72437fe700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 0x7f72440af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f7242ffd700 1 --2- 172.21.15.181:0/3756998021 >> v2:172.21.15.190:3300/0 conn(0x7f72440af450 0x7f724406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316896626 0 0) 0x7f724406f870 con 0x7f724406db80 2023-10-16T10:00:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f722c0035b0 con 0x7f724406db80 2023-10-16T10:00:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1746072304 0 0) 0x7f722c0035b0 con 0x7f724406db80 2023-10-16T10:00:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 msgr2=0x7f72440af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:06.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 0x7f72440af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 >> v2:172.21.15.190:3300/0 conn(0x7f72440af450 msgr2=0x7f724406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 --2- 172.21.15.181:0/3756998021 >> v2:172.21.15.190:3300/0 conn(0x7f72440af450 0x7f724406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72440ccbc0 con 0x7f724406db80 2023-10-16T10:00:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f723c0019f0 con 0x7f724406db80 2023-10-16T10:00:06.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.047+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 >> v1:172.21.15.181:6789/0 conn(0x7f724406db80 legacy=0x7f72440cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f722c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 0x7f722c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f722c004d90 0x7f722c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440d1160 con 0x7f722c003a00 2023-10-16T10:00:06.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440cc8e0 con 0x7f722c004d90 2023-10-16T10:00:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c00ba10 con 0x7f724415dfd0 2023-10-16T10:00:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72437fe700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f722c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f7242ffd700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 0x7f722c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f7243fff700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f722c004d90 0x7f722c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72437fe700 1 -- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 msgr2=0x7f722c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72437fe700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f724415dfd0 0x7f722c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72437fe700 1 -- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f722c004d90 msgr2=0x7f722c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72437fe700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f722c004d90 0x7f722c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72437fe700 1 -- 172.21.15.181:0/3756998021 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c0035b0 con 0x7f722c003a00 2023-10-16T10:00:06.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f7243fff700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f722c004d90 0x7f722c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72437fe700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f722c003e20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7238009b40 tx=0x7f722c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfde9c7f00719f6c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238005a20 con 0x7f722c003a00 2023-10-16T10:00:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7238005b80 con 0x7f722c003a00 2023-10-16T10:00:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.048+0000 7f72427fc700 1 -- 172.21.15.181:0/3756998021 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800e6c0 con 0x7f722c003a00 2023-10-16T10:00:06.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.049+0000 7f7249cb8700 1 -- 172.21.15.181:0/3756998021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 msgr2=0x7f722c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.049+0000 7f7249cb8700 1 --2- 172.21.15.181:0/3756998021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f722c003e20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7238009b40 tx=0x7f722c005880 comp rx=0 tx=0).stop 2023-10-16T10:00:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.049+0000 7f7243fff700 1 -- 172.21.15.181:0/3756998021 reap_dead start 2023-10-16T10:00:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.049+0000 7f7249cb8700 1 -- 172.21.15.181:0/3756998021 shutdown_connections 2023-10-16T10:00:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.049+0000 7f7249cb8700 1 -- 172.21.15.181:0/3756998021 >> 172.21.15.181:0/3756998021 conn(0x7f72440d04a0 msgr2=0x7f72440d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:06.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.049+0000 7f7249cb8700 1 -- 172.21.15.181:0/3756998021 shutdown_connections 2023-10-16T10:00:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.049+0000 7f7249cb8700 1 -- 172.21.15.181:0/3756998021 wait complete. 2023-10-16T10:00:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.050+0000 7f7249cb8700 1 Processor -- start 2023-10-16T10:00:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.050+0000 7f7249cb8700 1 -- start start 2023-10-16T10:00:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7249cb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f72440adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7249cb8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7244157a80 0x7f72440b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7249cb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7244157560 0x7f72440b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7249cb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440b22b0 con 0x7f722c003a00 2023-10-16T10:00:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7249cb8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440b0620 con 0x7f7244157a80 2023-10-16T10:00:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7249cb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72440b5cb0 con 0x7f7244157560 2023-10-16T10:00:06.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f72440adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f72440adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43298/0 (socket says 172.21.15.181:43298) 2023-10-16T10:00:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 -- 172.21.15.181:0/212524944 learned_addr learned my addr 172.21.15.181:0/212524944 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7243fff700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7244157560 0x7f72440b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f7242ffd700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7244157a80 0x7f72440b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 -- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7244157560 msgr2=0x7f72440b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7244157560 0x7f72440b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 -- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7244157a80 msgr2=0x7f72440b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7244157a80 0x7f72440b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 -- 172.21.15.181:0/212524944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72440b5410 con 0x7f722c003a00 2023-10-16T10:00:06.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.051+0000 7f72437fe700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f72440adb20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f723800e4b0 tx=0x7f72380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.052+0000 7f722bfff700 1 -- 172.21.15.181:0/212524944 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238020070 con 0x7f722c003a00 2023-10-16T10:00:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.052+0000 7f722bfff700 1 -- 172.21.15.181:0/212524944 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7238004050 con 0x7f722c003a00 2023-10-16T10:00:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.052+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72440b56f0 con 0x7f722c003a00 2023-10-16T10:00:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.052+0000 7f722bfff700 1 -- 172.21.15.181:0/212524944 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238004400 con 0x7f722c003a00 2023-10-16T10:00:06.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.052+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72440c51c0 con 0x7f722c003a00 2023-10-16T10:00:06.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.052+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72440c5530 con 0x7f722c003a00 2023-10-16T10:00:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.053+0000 7f722bfff700 1 -- 172.21.15.181:0/212524944 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f72380036a0 con 0x7f722c003a00 2023-10-16T10:00:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.053+0000 7f722bfff700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f723006b020 0x7f723006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.053+0000 7f722bfff700 1 -- 172.21.15.181:0/212524944 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f723808a930 con 0x7f722c003a00 2023-10-16T10:00:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.053+0000 7f7242ffd700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f723006b020 0x7f723006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:06.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.054+0000 7f7242ffd700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f723006b020 0x7f723006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7244157ea0 tx=0x7f7234008040 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:06.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.058+0000 7f722bfff700 1 -- 172.21.15.181:0/212524944 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f72380557b0 con 0x7f722c003a00 2023-10-16T10:00:06.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.291+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f72440c5020 con 0x7f722c003a00 2023-10-16T10:00:06.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.292+0000 7f722bfff700 1 -- 172.21.15.181:0/212524944 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7238029060 con 0x7f722c003a00 2023-10-16T10:00:06.293 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:00:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f723006b020 msgr2=0x7f723006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f723006b020 0x7f723006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7244157ea0 tx=0x7f7234008040 comp rx=0 tx=0).stop 2023-10-16T10:00:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 msgr2=0x7f72440adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f72440adb20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f723800e4b0 tx=0x7f72380047c0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 shutdown_connections 2023-10-16T10:00:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f723006b020 0x7f723006d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7244157560 0x7f72440b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f722c003a00 0x7f72440adb20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 --2- 172.21.15.181:0/212524944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7244157a80 0x7f72440b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 >> 172.21.15.181:0/212524944 conn(0x7f72440d04a0 msgr2=0x7f724406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 shutdown_connections 2023-10-16T10:00:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:06.294+0000 7f7249cb8700 1 -- 172.21.15.181:0/212524944 wait complete. 2023-10-16T10:00:06.306 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:00:06.307 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:00:06.307 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T10:00:06.307 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:00:06.308 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:06.308 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:00:06.308 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:00:06.309 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:00:06.309 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:00:06.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.310 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T10:00:06.327 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:00:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:00:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:00:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:06.328 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T10:00:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:06.329 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:00:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:06.329 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:00:06.329 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:06.330 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:00:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:06.330 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:00:06.330 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:00:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41463sec preferred_lft 41463sec 2023-10-16T10:00:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T10:00:06.331 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:06.331 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:00:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:00:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T10:00:06.332 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:06.333 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:00:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:00:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T10:00:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:06.333 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:00:06.333 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:00:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T10:00:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:06.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.335 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:00:06.392 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T10:00:06.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.394 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T10:00:06.452 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T10:00:06.453 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:00:06.453 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:00:06.453 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:00:06.453 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:00:06.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.455 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T10:00:06.455 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T10:00:06.456 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:00:06.456 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:00:06.456 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:00:06.457 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T10:00:06.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.559 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T10:00:06.576 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:00:06.576 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:00:06.576 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:00:06.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.577 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T10:00:06.642 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:00:06.643 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:06.699 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:00:06.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:00:06.700 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:06.756 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T10:00:06.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.757 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:06.841 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:06.842 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:00:06.842 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:00:06.850 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:06.850 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:00:06.850 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:00:06.850 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:00:06.851 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:00:06.878 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:06.878 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:00:06.878 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:00:06.879 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:00:06.879 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:00:06.879 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T10:00:06.879 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T10:00:06.879 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T10:00:06.898 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:00:06.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.899 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:06.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:06.928 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:00:07.015 INFO:teuthology.orchestra.run.smithi181.stdout:client6568 2023-10-16T10:00:07.016 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:00:07.016 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:07.034 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:00:07.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.196+0000 7f29e3c9d700 1 Processor -- start 2023-10-16T10:00:07.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.196+0000 7f29e3c9d700 1 -- start start 2023-10-16T10:00:07.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e3c9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc0b05d0 0x7f29dc0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e3c9d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc0d24f0 con 0x7f29dc15bd50 2023-10-16T10:00:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e3c9d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc15c320 con 0x7f29dc0cfc60 2023-10-16T10:00:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e3c9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc15ccc0 con 0x7f29dc0b05d0 2023-10-16T10:00:07.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e1238700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f29dc15bd50 0x7f29dc0cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54390/0 (socket says 172.21.15.181:54390) 2023-10-16T10:00:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e1238700 1 -- 172.21.15.181:0/3284113663 learned_addr learned my addr 172.21.15.181:0/3284113663 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e223a700 1 --2- 172.21.15.181:0/3284113663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc0b05d0 0x7f29dc0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e0a37700 1 -- 172.21.15.181:0/3284113663 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736599223 0 0) 0x7f29dc0d24f0 con 0x7f29dc15bd50 2023-10-16T10:00:07.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e0a37700 1 -- 172.21.15.181:0/3284113663 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c80035b0 con 0x7f29dc15bd50 2023-10-16T10:00:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e0a37700 1 -- 172.21.15.181:0/3284113663 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786893277 0 0) 0x7f29dc15c320 con 0x7f29dc0cfc60 2023-10-16T10:00:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e0a37700 1 -- 172.21.15.181:0/3284113663 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29dc0d24f0 con 0x7f29dc0cfc60 2023-10-16T10:00:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e223a700 1 -- 172.21.15.181:0/3284113663 >> v1:172.21.15.190:6789/0 conn(0x7f29dc0cfc60 legacy=0x7f29dc0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e223a700 1 -- 172.21.15.181:0/3284113663 >> v1:172.21.15.181:6789/0 conn(0x7f29dc15bd50 legacy=0x7f29dc0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.197+0000 7f29e223a700 1 -- 172.21.15.181:0/3284113663 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc15c600 con 0x7f29dc0b05d0 2023-10-16T10:00:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e223a700 1 --2- 172.21.15.181:0/3284113663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc0b05d0 0x7f29dc0d1fb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009dc0 tx=0x7f29d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9029a366fb024d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e0a37700 1 -- 172.21.15.181:0/3284113663 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400e070 con 0x7f29dc0b05d0 2023-10-16T10:00:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e0a37700 1 -- 172.21.15.181:0/3284113663 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d400bb20 con 0x7f29dc0b05d0 2023-10-16T10:00:07.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e0a37700 1 -- 172.21.15.181:0/3284113663 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4012430 con 0x7f29dc0b05d0 2023-10-16T10:00:07.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e3c9d700 1 -- 172.21.15.181:0/3284113663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc0b05d0 msgr2=0x7f29dc0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/3284113663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc0b05d0 0x7f29dc0d1fb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009dc0 tx=0x7f29d400b330 comp rx=0 tx=0).stop 2023-10-16T10:00:07.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e3c9d700 1 -- 172.21.15.181:0/3284113663 shutdown_connections 2023-10-16T10:00:07.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/3284113663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc0b05d0 0x7f29dc0d1fb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e3c9d700 1 -- 172.21.15.181:0/3284113663 >> 172.21.15.181:0/3284113663 conn(0x7f29dc09a9a0 msgr2=0x7f29dc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e3c9d700 1 -- 172.21.15.181:0/3284113663 shutdown_connections 2023-10-16T10:00:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.198+0000 7f29e3c9d700 1 -- 172.21.15.181:0/3284113663 wait complete. 2023-10-16T10:00:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 Processor -- start 2023-10-16T10:00:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 -- start start 2023-10-16T10:00:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29dc0b05d0 0x7f29dc14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29dc0cfc60 0x7f29dc0b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc15bd50 0x7f29dc159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc14ffd0 con 0x7f29dc0cfc60 2023-10-16T10:00:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc06d3e0 con 0x7f29dc0b05d0 2023-10-16T10:00:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e3c9d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc06d650 con 0x7f29dc15bd50 2023-10-16T10:00:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e1238700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29dc0cfc60 0x7f29dc0b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e1238700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29dc0cfc60 0x7f29dc0b69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43316/0 (socket says 172.21.15.181:43316) 2023-10-16T10:00:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.200+0000 7f29e1238700 1 -- 172.21.15.181:0/406754474 learned_addr learned my addr 172.21.15.181:0/406754474 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e223a700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc15bd50 0x7f29dc159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e1238700 1 -- 172.21.15.181:0/406754474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc15bd50 msgr2=0x7f29dc159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e1238700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc15bd50 0x7f29dc159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e1238700 1 -- 172.21.15.181:0/406754474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29dc0b05d0 msgr2=0x7f29dc14fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e1238700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29dc0b05d0 0x7f29dc14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e1238700 1 -- 172.21.15.181:0/406754474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009a70 con 0x7f29dc0cfc60 2023-10-16T10:00:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e1238700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29dc0cfc60 0x7f29dc0b69c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c4d0 tx=0x7f29cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29d27fc700 1 -- 172.21.15.181:0/406754474 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc005d70 con 0x7f29dc0cfc60 2023-10-16T10:00:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29d27fc700 1 -- 172.21.15.181:0/406754474 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc0070d0 con 0x7f29dc0cfc60 2023-10-16T10:00:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc0b4310 con 0x7f29dc0cfc60 2023-10-16T10:00:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29d27fc700 1 -- 172.21.15.181:0/406754474 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc007480 con 0x7f29dc0cfc60 2023-10-16T10:00:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.201+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc06d930 con 0x7f29dc0cfc60 2023-10-16T10:00:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.202+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc0b4830 con 0x7f29dc0cfc60 2023-10-16T10:00:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.203+0000 7f29d27fc700 1 -- 172.21.15.181:0/406754474 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f29cc0075e0 con 0x7f29dc0cfc60 2023-10-16T10:00:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.203+0000 7f29d27fc700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f29c006b050 0x7f29c006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.203+0000 7f29d27fc700 1 -- 172.21.15.181:0/406754474 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f29cc088ee0 con 0x7f29dc0cfc60 2023-10-16T10:00:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.203+0000 7f29e1a39700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f29c006b050 0x7f29c006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.203+0000 7f29e1a39700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f29c006b050 0x7f29c006d510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29d8007fd0 tx=0x7f29d80079a0 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:07.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.207+0000 7f29d27fc700 1 -- 172.21.15.181:0/406754474 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f29dc0b4830 con 0x7f29dc0cfc60 2023-10-16T10:00:07.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.427+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 --> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f29dc15c8e0 con 0x7f29c006b050 2023-10-16T10:00:07.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.428+0000 7f29d27fc700 1 -- 172.21.15.181:0/406754474 <== mgr.6732 v2:172.21.15.181:6832/752673838 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f29d80050d0 con 0x7f29c006b050 2023-10-16T10:00:07.429 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:00:07.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:07.429 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T10:00:07.430 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:07.430 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:00:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f29c006b050 msgr2=0x7f29c006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f29c006b050 0x7f29c006d510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29d8007fd0 tx=0x7f29d80079a0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29dc0cfc60 msgr2=0x7f29dc0b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29dc0cfc60 0x7f29dc0b69c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c4d0 tx=0x7f29cc00c890 comp rx=0 tx=0).stop 2023-10-16T10:00:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 shutdown_connections 2023-10-16T10:00:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f29c006b050 0x7f29c006d510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29dc15bd50 0x7f29dc159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29dc0b05d0 0x7f29dc14fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 --2- 172.21.15.181:0/406754474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29dc0cfc60 0x7f29dc0b69c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 >> 172.21.15.181:0/406754474 conn(0x7f29dc09a9a0 msgr2=0x7f29dc0d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 shutdown_connections 2023-10-16T10:00:07.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.430+0000 7f29e3c9d700 1 -- 172.21.15.181:0/406754474 wait complete. 2023-10-16T10:00:07.443 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:00:07.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.564+0000 7fe307095700 1 Processor -- start 2023-10-16T10:00:07.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.564+0000 7fe307095700 1 -- start start 2023-10-16T10:00:07.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe307095700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3000ce6c0 0x7fe3000cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe307095700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000cc6a0 0x7fe3000ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe307095700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3000b1f70 con 0x7fe30015f150 2023-10-16T10:00:07.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe307095700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3000b2ae0 con 0x7fe3000ce6c0 2023-10-16T10:00:07.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe307095700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000b2dc0 con 0x7fe3000cc6a0 2023-10-16T10:00:07.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe2fffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3000ce6c0 0x7fe3000cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe304e31700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe30015f150 0x7fe3000cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54402/0 (socket says 172.21.15.181:54402) 2023-10-16T10:00:07.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.565+0000 7fe304e31700 1 -- 172.21.15.181:0/4211129514 learned_addr learned my addr 172.21.15.181:0/4211129514 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:07.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe305632700 1 --2- 172.21.15.181:0/4211129514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000cc6a0 0x7fe3000ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2fffff700 1 -- 172.21.15.181:0/4211129514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000cc6a0 msgr2=0x7fe3000ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:07.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2fffff700 1 --2- 172.21.15.181:0/4211129514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000cc6a0 0x7fe3000ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2fffff700 1 -- 172.21.15.181:0/4211129514 >> v1:172.21.15.181:6789/0 conn(0x7fe30015f150 legacy=0x7fe3000cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2fffff700 1 -- 172.21.15.181:0/4211129514 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3000cf9d0 con 0x7fe3000ce6c0 2023-10-16T10:00:07.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe305632700 1 --2- 172.21.15.181:0/4211129514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000cc6a0 0x7fe3000ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:07.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2fffff700 1 --2- 172.21.15.181:0/4211129514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000ce6c0 0x7fe3000cc160 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009840 tx=0x7fe2ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a542402aac25bb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:07.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2ff7fe700 1 -- 172.21.15.181:0/4211129514 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec011830 con 0x7fe3000ce6c0 2023-10-16T10:00:07.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2ff7fe700 1 -- 172.21.15.181:0/4211129514 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec011990 con 0x7fe3000ce6c0 2023-10-16T10:00:07.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.566+0000 7fe2ff7fe700 1 -- 172.21.15.181:0/4211129514 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec011d20 con 0x7fe3000ce6c0 2023-10-16T10:00:07.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 -- 172.21.15.181:0/4211129514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000ce6c0 msgr2=0x7fe3000cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 --2- 172.21.15.181:0/4211129514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000ce6c0 0x7fe3000cc160 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009840 tx=0x7fe2ec009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:07.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 -- 172.21.15.181:0/4211129514 shutdown_connections 2023-10-16T10:00:07.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 --2- 172.21.15.181:0/4211129514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000cc6a0 0x7fe3000ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 --2- 172.21.15.181:0/4211129514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000ce6c0 0x7fe3000cc160 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 -- 172.21.15.181:0/4211129514 >> 172.21.15.181:0/4211129514 conn(0x7fe30009a930 msgr2=0x7fe3000d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:07.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 -- 172.21.15.181:0/4211129514 shutdown_connections 2023-10-16T10:00:07.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.567+0000 7fe307095700 1 -- 172.21.15.181:0/4211129514 wait complete. 2023-10-16T10:00:07.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.568+0000 7fe307095700 1 Processor -- start 2023-10-16T10:00:07.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.568+0000 7fe307095700 1 -- start start 2023-10-16T10:00:07.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe307095700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000cc6a0 0x7fe3001f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe307095700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000ce6c0 0x7fe3000caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe307095700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30015f150 0x7fe3000ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe307095700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3001f51d0 con 0x7fe3000cc6a0 2023-10-16T10:00:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe307095700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000c9ed0 con 0x7fe30015f150 2023-10-16T10:00:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe307095700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000ca1b0 con 0x7fe3000ce6c0 2023-10-16T10:00:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000cc6a0 0x7fe3001f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000cc6a0 0x7fe3001f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43330/0 (socket says 172.21.15.181:43330) 2023-10-16T10:00:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 -- 172.21.15.181:0/351585647 learned_addr learned my addr 172.21.15.181:0/351585647 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 -- 172.21.15.181:0/351585647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000ce6c0 msgr2=0x7fe3000caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000ce6c0 0x7fe3000caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 -- 172.21.15.181:0/351585647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30015f150 msgr2=0x7fe3000ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30015f150 0x7fe3000ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 -- 172.21.15.181:0/351585647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0094a0 con 0x7fe3000cc6a0 2023-10-16T10:00:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe304e31700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000cc6a0 0x7fe3001f4c90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400c490 tx=0x7fe2f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe2fd7fa700 1 -- 172.21.15.181:0/351585647 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4005d70 con 0x7fe3000cc6a0 2023-10-16T10:00:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe2fd7fa700 1 -- 172.21.15.181:0/351585647 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f40070d0 con 0x7fe3000cc6a0 2023-10-16T10:00:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3000ca490 con 0x7fe3000cc6a0 2023-10-16T10:00:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.569+0000 7fe2fd7fa700 1 -- 172.21.15.181:0/351585647 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4007460 con 0x7fe3000cc6a0 2023-10-16T10:00:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.570+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3000caa50 con 0x7fe3000cc6a0 2023-10-16T10:00:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.571+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3000cac10 con 0x7fe3000cc6a0 2023-10-16T10:00:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.571+0000 7fe2fd7fa700 1 -- 172.21.15.181:0/351585647 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 42) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe2f40075c0 con 0x7fe3000cc6a0 2023-10-16T10:00:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.571+0000 7fe2fd7fa700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fe2e406b020 0x7fe2e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.571+0000 7fe2fd7fa700 1 -- 172.21.15.181:0/351585647 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fe2f4088dd0 con 0x7fe3000cc6a0 2023-10-16T10:00:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.571+0000 7fe2fffff700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fe2e406b020 0x7fe2e406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.572+0000 7fe2fffff700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fe2e406b020 0x7fe2e406d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0090b0 tx=0x7fe2ec010000 comp rx=0 tx=0).ready entity=mgr.6732 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.576+0000 7fe2fd7fa700 1 -- 172.21.15.181:0/351585647 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2f4053bc0 con 0x7fe3000cc6a0 2023-10-16T10:00:07.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:07.824+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe300067d20 con 0x7fe3000cc6a0 2023-10-16T10:00:08.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.002+0000 7fe2fd7fa700 1 -- 172.21.15.181:0/351585647 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v43) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe2f408a0a0 con 0x7fe3000cc6a0 2023-10-16T10:00:08.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fe2e406b020 msgr2=0x7fe2e406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fe2e406b020 0x7fe2e406d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0090b0 tx=0x7fe2ec010000 comp rx=0 tx=0).stop 2023-10-16T10:00:08.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000cc6a0 msgr2=0x7fe3001f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000cc6a0 0x7fe3001f4c90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400c490 tx=0x7fe2f400c850 comp rx=0 tx=0).stop 2023-10-16T10:00:08.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 shutdown_connections 2023-10-16T10:00:08.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fe2e406b020 0x7fe2e406d4e0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0090b0 tx=0x7fe2ec010000 comp rx=0 tx=0).stop 2023-10-16T10:00:08.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30015f150 0x7fe3000ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3000cc6a0 0x7fe3001f4c90 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 --2- 172.21.15.181:0/351585647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3000ce6c0 0x7fe3000caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 >> 172.21.15.181:0/351585647 conn(0x7fe30009a930 msgr2=0x7fe30015db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:08.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.005+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 shutdown_connections 2023-10-16T10:00:08.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.006+0000 7fe307095700 1 -- 172.21.15.181:0/351585647 wait complete. 2023-10-16T10:00:08.022 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:00:08.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.149+0000 7f4911f9c700 1 Processor -- start 2023-10-16T10:00:08.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.149+0000 7f4911f9c700 1 -- start start 2023-10-16T10:00:08.153 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:08.150+0000 7fba372e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:00:08.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f4911f9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f4911f9c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f490c0ce680 0x7f490c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f4911f9c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0ccb60 con 0x7f490c0b2010 2023-10-16T10:00:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f4911f9c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c072120 con 0x7f490c0ce680 2023-10-16T10:00:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f4911f9c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0ce1f0 con 0x7f490c0cc6e0 2023-10-16T10:00:08.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f490bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f490c0ce680 0x7f490c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f490bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f490c0ce680 0x7f490c0ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43368/0 (socket says 172.21.15.181:43368) 2023-10-16T10:00:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f490bfff700 1 -- 172.21.15.181:0/1984481760 learned_addr learned my addr 172.21.15.181:0/1984481760 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.150+0000 7f490affd700 1 --2- 172.21.15.181:0/1984481760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490bfff700 1 -- 172.21.15.181:0/1984481760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 msgr2=0x7f490c071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490bfff700 1 --2- 172.21.15.181:0/1984481760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490bfff700 1 -- 172.21.15.181:0/1984481760 >> v1:172.21.15.190:6789/0 conn(0x7f490c0b2010 legacy=0x7f490c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490bfff700 1 -- 172.21.15.181:0/1984481760 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0ccda0 con 0x7f490c0ce680 2023-10-16T10:00:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490bfff700 1 --2- 172.21.15.181:0/1984481760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0ce680 0x7f490c0ceb10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f49040098f0 tx=0x7f4904009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04daa33548ea17c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490a7fc700 1 -- 172.21.15.181:0/1984481760 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904011830 con 0x7f490c0ce680 2023-10-16T10:00:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490a7fc700 1 -- 172.21.15.181:0/1984481760 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4904011990 con 0x7f490c0ce680 2023-10-16T10:00:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f490a7fc700 1 -- 172.21.15.181:0/1984481760 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904011d20 con 0x7f490c0ce680 2023-10-16T10:00:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f4911f9c700 1 -- 172.21.15.181:0/1984481760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0ce680 msgr2=0x7f490c0ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f4911f9c700 1 --2- 172.21.15.181:0/1984481760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0ce680 0x7f490c0ceb10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f49040098f0 tx=0x7f4904009c00 comp rx=0 tx=0).stop 2023-10-16T10:00:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f4911f9c700 1 -- 172.21.15.181:0/1984481760 shutdown_connections 2023-10-16T10:00:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.151+0000 7f4911f9c700 1 --2- 172.21.15.181:0/1984481760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0ce680 0x7f490c0ceb10 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.152+0000 7f4911f9c700 1 --2- 172.21.15.181:0/1984481760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.152+0000 7f4911f9c700 1 -- 172.21.15.181:0/1984481760 >> 172.21.15.181:0/1984481760 conn(0x7f490c0cd010 msgr2=0x7f490c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:08.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.152+0000 7f4911f9c700 1 -- 172.21.15.181:0/1984481760 shutdown_connections 2023-10-16T10:00:08.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.152+0000 7f4911f9c700 1 -- 172.21.15.181:0/1984481760 wait complete. 2023-10-16T10:00:08.161 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:08.153+0000 7fceaff5a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:00:08.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.153+0000 7f4911f9c700 1 Processor -- start 2023-10-16T10:00:08.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.153+0000 7f4911f9c700 1 -- start start 2023-10-16T10:00:08.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0b2010 0x7f490c0aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c0a8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f490c0ce680 0x7f490c0a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0af330 con 0x7f490c0b2010 2023-10-16T10:00:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0c9710 con 0x7f490c0ce680 2023-10-16T10:00:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0c99f0 con 0x7f490c0cc6e0 2023-10-16T10:00:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0b2010 0x7f490c0aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0b2010 0x7f490c0aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43370/0 (socket says 172.21.15.181:43370) 2023-10-16T10:00:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 -- 172.21.15.181:0/260564222 learned_addr learned my addr 172.21.15.181:0/260564222 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490affd700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c0a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490bfff700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f490c0ce680 0x7f490c0a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 -- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 msgr2=0x7f490c0a8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c0a8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 -- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f490c0ce680 msgr2=0x7f490c0a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f490c0ce680 0x7f490c0a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 -- 172.21.15.181:0/260564222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4904009550 con 0x7f490c0b2010 2023-10-16T10:00:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f490b7fe700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0b2010 0x7f490c0aedf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00cbd0 tx=0x7f48fc00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4903fff700 1 -- 172.21.15.181:0/260564222 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003c00 con 0x7f490c0b2010 2023-10-16T10:00:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4903fff700 1 -- 172.21.15.181:0/260564222 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc003d60 con 0x7f490c0b2010 2023-10-16T10:00:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c0c9cd0 con 0x7f490c0b2010 2023-10-16T10:00:08.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4903fff700 1 -- 172.21.15.181:0/260564222 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00aae0 con 0x7f490c0b2010 2023-10-16T10:00:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.154+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c148520 con 0x7f490c0b2010 2023-10-16T10:00:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.155+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c0b6510 con 0x7f490c0b2010 2023-10-16T10:00:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.156+0000 7f4903fff700 1 -- 172.21.15.181:0/260564222 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 43) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f48fc00ac40 con 0x7f490c0b2010 2023-10-16T10:00:08.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.157+0000 7f4903fff700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 0x7f48ec06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.157+0000 7f490affd700 1 -- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 msgr2=0x7f48ec06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.157+0000 7f4903fff700 1 -- 172.21.15.181:0/260564222 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f48fc08a170 con 0x7f490c0b2010 2023-10-16T10:00:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.157+0000 7f490affd700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 0x7f48ec06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.164+0000 7f4903fff700 1 -- 172.21.15.181:0/260564222 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f48fc055040 con 0x7f490c0b2010 2023-10-16T10:00:08.218 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:08.217+0000 7fba372e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:08.225 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:08.225+0000 7fceaff5a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:08.290 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:08.289+0000 7fba372e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:00:08.299 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:08.299+0000 7fceaff5a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:00:08.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.358+0000 7f490affd700 1 -- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 msgr2=0x7f48ec06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:08.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.358+0000 7f490affd700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 0x7f48ec06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:08.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.364+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f490c0ccb60 con 0x7f490c0b2010 2023-10-16T10:00:08.367 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:08.366+0000 7fba372e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:00:08.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.368+0000 7f4903fff700 1 -- 172.21.15.181:0/260564222 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f490c0ccb60 con 0x7f490c0b2010 2023-10-16T10:00:08.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 msgr2=0x7f48ec06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:08.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 0x7f48ec06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0b2010 msgr2=0x7f490c0aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0b2010 0x7f490c0aedf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00cbd0 tx=0x7f48fc00cee0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 shutdown_connections 2023-10-16T10:00:08.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f48ec06b070 0x7f48ec06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f490c0ce680 0x7f490c0a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f490c0b2010 0x7f490c0aedf0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 --2- 172.21.15.181:0/260564222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f490c0cc6e0 0x7f490c0a8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.370+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 >> 172.21.15.181:0/260564222 conn(0x7f490c0cd010 msgr2=0x7f490c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:08.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.373+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 shutdown_connections 2023-10-16T10:00:08.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.373+0000 7f4911f9c700 1 -- 172.21.15.181:0/260564222 wait complete. 2023-10-16T10:00:08.379 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:08.379+0000 7fceaff5a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:00:08.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:00:08.507 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:08.506+0000 7fba372e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:00:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.511+0000 7fdf79d01700 1 Processor -- start 2023-10-16T10:00:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.511+0000 7fdf79d01700 1 -- start start 2023-10-16T10:00:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf79d01700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf7415da70 0x7fdf740ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf79d01700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf740ce6a0 0x7fdf7406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf79d01700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf740cef00 con 0x7fdf74072760 2023-10-16T10:00:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf79d01700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf740cec20 con 0x7fdf7415da70 2023-10-16T10:00:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf79d01700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74071c80 con 0x7fdf740ce6a0 2023-10-16T10:00:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf737fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf7415da70 0x7fdf740ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf73fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdf74072760 0x7fdf741615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54478/0 (socket says 172.21.15.181:54478) 2023-10-16T10:00:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf737fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf7415da70 0x7fdf740ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43396/0 (socket says 172.21.15.181:43396) 2023-10-16T10:00:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf73fff700 1 -- 172.21.15.181:0/2223991680 learned_addr learned my addr 172.21.15.181:0/2223991680 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf72ffd700 1 --2- 172.21.15.181:0/2223991680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf740ce6a0 0x7fdf7406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf737fe700 1 -- 172.21.15.181:0/2223991680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf740ce6a0 msgr2=0x7fdf7406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf737fe700 1 --2- 172.21.15.181:0/2223991680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf740ce6a0 0x7fdf7406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf737fe700 1 -- 172.21.15.181:0/2223991680 >> v1:172.21.15.181:6789/0 conn(0x7fdf74072760 legacy=0x7fdf741615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf727fc700 1 -- 172.21.15.181:0/2223991680 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442151413 0 0) 0x7fdf740cef00 con 0x7fdf74072760 2023-10-16T10:00:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.512+0000 7fdf737fe700 1 -- 172.21.15.181:0/2223991680 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74071f60 con 0x7fdf7415da70 2023-10-16T10:00:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf737fe700 1 --2- 172.21.15.181:0/2223991680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf7415da70 0x7fdf740ce160 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009840 tx=0x7fdf64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59fa588d1de0e28 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf727fc700 1 -- 172.21.15.181:0/2223991680 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64011830 con 0x7fdf7415da70 2023-10-16T10:00:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf727fc700 1 -- 172.21.15.181:0/2223991680 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64011990 con 0x7fdf7415da70 2023-10-16T10:00:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf727fc700 1 -- 172.21.15.181:0/2223991680 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64011d20 con 0x7fdf7415da70 2023-10-16T10:00:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 -- 172.21.15.181:0/2223991680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf7415da70 msgr2=0x7fdf740ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 --2- 172.21.15.181:0/2223991680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf7415da70 0x7fdf740ce160 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009840 tx=0x7fdf64009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 -- 172.21.15.181:0/2223991680 shutdown_connections 2023-10-16T10:00:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 --2- 172.21.15.181:0/2223991680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf7415da70 0x7fdf740ce160 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 --2- 172.21.15.181:0/2223991680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf740ce6a0 0x7fdf7406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 -- 172.21.15.181:0/2223991680 >> 172.21.15.181:0/2223991680 conn(0x7fdf740cd010 msgr2=0x7fdf740cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 -- 172.21.15.181:0/2223991680 shutdown_connections 2023-10-16T10:00:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.513+0000 7fdf79d01700 1 -- 172.21.15.181:0/2223991680 wait complete. 2023-10-16T10:00:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.514+0000 7fdf79d01700 1 Processor -- start 2023-10-16T10:00:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.515+0000 7fdf79d01700 1 -- start start 2023-10-16T10:00:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.515+0000 7fdf79d01700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf74072760 0x7fdf740b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.515+0000 7fdf79d01700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf740ce6a0 0x7fdf74160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.515+0000 7fdf79d01700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7415da70 0x7fdf740b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.515+0000 7fdf79d01700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741f2ff0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.515+0000 7fdf79d01700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7415fbb0 con 0x7fdf74072760 2023-10-16T10:00:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.515+0000 7fdf79d01700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7415fdc0 con 0x7fdf7415da70 2023-10-16T10:00:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf740ce6a0 0x7fdf74160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf740ce6a0 0x7fdf74160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43402/0 (socket says 172.21.15.181:43402) 2023-10-16T10:00:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 -- 172.21.15.181:0/3417491322 learned_addr learned my addr 172.21.15.181:0/3417491322 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf73fff700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7415da70 0x7fdf740b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf737fe700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf74072760 0x7fdf740b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 -- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7415da70 msgr2=0x7fdf740b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7415da70 0x7fdf740b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 -- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf74072760 msgr2=0x7fdf740b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf74072760 0x7fdf740b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 -- 172.21.15.181:0/3417491322 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640094a0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf72ffd700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf740ce6a0 0x7fdf74160880 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800e870 tx=0x7fdf6800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf5bfff700 1 -- 172.21.15.181:0/3417491322 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf68009590 con 0x7fdf740ce6a0 2023-10-16T10:00:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf5bfff700 1 -- 172.21.15.181:0/3417491322 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf68013680 con 0x7fdf740ce6a0 2023-10-16T10:00:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf741600a0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf5bfff700 1 -- 172.21.15.181:0/3417491322 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf680139f0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.516+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf740c7660 con 0x7fdf740ce6a0 2023-10-16T10:00:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.517+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf740ce1f0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.518+0000 7fdf5bfff700 1 -- 172.21.15.181:0/3417491322 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 43) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fdf68013b50 con 0x7fdf740ce6a0 2023-10-16T10:00:08.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.518+0000 7fdf5bfff700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 0x7fdf6006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.518+0000 7fdf737fe700 1 -- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 msgr2=0x7fdf6006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:08.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.518+0000 7fdf5bfff700 1 -- 172.21.15.181:0/3417491322 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fdf68089730 con 0x7fdf740ce6a0 2023-10-16T10:00:08.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.518+0000 7fdf737fe700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 0x7fdf6006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.523+0000 7fdf5bfff700 1 -- 172.21.15.181:0/3417491322 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdf680545b0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.534 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:08.534+0000 7fceaff5a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:00:08.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.719+0000 7fdf737fe700 1 -- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 msgr2=0x7fdf6006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.719+0000 7fdf737fe700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 0x7fdf6006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:08.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.743+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdf7406f6c0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.746+0000 7fdf5bfff700 1 -- 172.21.15.181:0/3417491322 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdf68057bd0 con 0x7fdf740ce6a0 2023-10-16T10:00:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 msgr2=0x7fdf6006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 0x7fdf6006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf740ce6a0 msgr2=0x7fdf74160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf740ce6a0 0x7fdf74160880 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800e870 tx=0x7fdf6800eb80 comp rx=0 tx=0).stop 2023-10-16T10:00:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 shutdown_connections 2023-10-16T10:00:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7fdf6006b070 0x7fdf6006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7415da70 0x7fdf740b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf74072760 0x7fdf740b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 --2- 172.21.15.181:0/3417491322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf740ce6a0 0x7fdf74160880 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 >> 172.21.15.181:0/3417491322 conn(0x7fdf740cd010 msgr2=0x7fdf7406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 shutdown_connections 2023-10-16T10:00:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.748+0000 7fdf79d01700 1 -- 172.21.15.181:0/3417491322 wait complete. 2023-10-16T10:00:08.762 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-16T10:00:08.832 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:08.830+0000 7fba372e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:00:08.883 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:08.883+0000 7fceaff5a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:00:08.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.887+0000 7f2a197a7700 1 Processor -- start 2023-10-16T10:00:08.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.887+0000 7f2a197a7700 1 -- start start 2023-10-16T10:00:08.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.887+0000 7f2a197a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a140d01d0 0x7f2a140ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.887+0000 7f2a197a7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a1415ea40 0x7f2a140d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.887+0000 7f2a197a7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a140cf580 con 0x7f2a14161530 2023-10-16T10:00:08.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.887+0000 7f2a197a7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a140d4290 con 0x7f2a140d01d0 2023-10-16T10:00:08.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.887+0000 7f2a197a7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a140d0af0 con 0x7f2a1415ea40 2023-10-16T10:00:08.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a140d01d0 0x7f2a140ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a140d01d0 0x7f2a140ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43430/0 (socket says 172.21.15.181:43430) 2023-10-16T10:00:08.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 -- 172.21.15.181:0/2293863542 learned_addr learned my addr 172.21.15.181:0/2293863542 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:08.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a127fc700 1 --2- 172.21.15.181:0/2293863542 >> v2:172.21.15.190:3300/0 conn(0x7f2a1415ea40 0x7f2a140d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 -- 172.21.15.181:0/2293863542 >> v2:172.21.15.190:3300/0 conn(0x7f2a1415ea40 msgr2=0x7f2a140d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:08.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 --2- 172.21.15.181:0/2293863542 >> v2:172.21.15.190:3300/0 conn(0x7f2a1415ea40 0x7f2a140d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 -- 172.21.15.181:0/2293863542 >> v1:172.21.15.190:6789/0 conn(0x7f2a14161530 legacy=0x7f2a141619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 -- 172.21.15.181:0/2293863542 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a140ce970 con 0x7f2a140d01d0 2023-10-16T10:00:08.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a127fc700 1 --2- 172.21.15.181:0/2293863542 >> v2:172.21.15.190:3300/0 conn(0x7f2a1415ea40 0x7f2a140d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:08.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a12ffd700 1 --2- 172.21.15.181:0/2293863542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 0x7f2a140ce2d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2a140d12c0 tx=0x7f2a04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1afe81040d089a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:08.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a11ffb700 1 -- 172.21.15.181:0/2293863542 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400b870 con 0x7f2a140d01d0 2023-10-16T10:00:08.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a11ffb700 1 -- 172.21.15.181:0/2293863542 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0400b9d0 con 0x7f2a140d01d0 2023-10-16T10:00:08.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.888+0000 7f2a11ffb700 1 -- 172.21.15.181:0/2293863542 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400bd60 con 0x7f2a140d01d0 2023-10-16T10:00:08.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 -- 172.21.15.181:0/2293863542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 msgr2=0x7f2a140ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 --2- 172.21.15.181:0/2293863542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 0x7f2a140ce2d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2a140d12c0 tx=0x7f2a04009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 -- 172.21.15.181:0/2293863542 shutdown_connections 2023-10-16T10:00:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 --2- 172.21.15.181:0/2293863542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 0x7f2a140ce2d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 --2- 172.21.15.181:0/2293863542 >> v2:172.21.15.190:3300/0 conn(0x7f2a1415ea40 0x7f2a140d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 -- 172.21.15.181:0/2293863542 >> 172.21.15.181:0/2293863542 conn(0x7f2a1409cb60 msgr2=0x7f2a140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:08.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 -- 172.21.15.181:0/2293863542 shutdown_connections 2023-10-16T10:00:08.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.889+0000 7f2a197a7700 1 -- 172.21.15.181:0/2293863542 wait complete. 2023-10-16T10:00:08.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.890+0000 7f2a197a7700 1 Processor -- start 2023-10-16T10:00:08.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.890+0000 7f2a197a7700 1 -- start start 2023-10-16T10:00:08.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a197a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 0x7f2a140b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a197a7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a1415ea40 0x7f2a140b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a197a7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a14161530 0x7f2a14151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a197a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140c8450 con 0x7f2a140d01d0 2023-10-16T10:00:08.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a197a7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140c77d0 con 0x7f2a14161530 2023-10-16T10:00:08.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a197a7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140c7ab0 con 0x7f2a1415ea40 2023-10-16T10:00:08.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 0x7f2a140b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 0x7f2a140b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43436/0 (socket says 172.21.15.181:43436) 2023-10-16T10:00:08.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 -- 172.21.15.181:0/457713573 learned_addr learned my addr 172.21.15.181:0/457713573 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:08.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a1415ea40 0x7f2a140b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a137fe700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a14161530 0x7f2a14151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:08.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a1415ea40 msgr2=0x7f2a140b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a1415ea40 0x7f2a140b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a14161530 msgr2=0x7f2a14151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:08.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a14161530 0x7f2a14151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:08.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a12ffd700 1 -- 172.21.15.181:0/457713573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009550 con 0x7f2a140d01d0 2023-10-16T10:00:08.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.891+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a1415ea40 0x7f2a140b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:08.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.892+0000 7f2a12ffd700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a140d01d0 0x7f2a140b3a10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2a1408cf30 tx=0x7f2a04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:08.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.892+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04004310 con 0x7f2a140d01d0 2023-10-16T10:00:08.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.892+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0401f070 con 0x7f2a140d01d0 2023-10-16T10:00:08.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.892+0000 7f2a197a7700 1 -- 172.21.15.181:0/457713573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a140c7d90 con 0x7f2a140d01d0 2023-10-16T10:00:08.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.892+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040036a0 con 0x7f2a140d01d0 2023-10-16T10:00:08.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.892+0000 7f2a197a7700 1 -- 172.21.15.181:0/457713573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a140b3f50 con 0x7f2a140d01d0 2023-10-16T10:00:08.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.893+0000 7f2a197a7700 1 -- 172.21.15.181:0/457713573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a140b4110 con 0x7f2a140d01d0 2023-10-16T10:00:08.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.893+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 43) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2a04021030 con 0x7f2a140d01d0 2023-10-16T10:00:08.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.894+0000 7f29fb7fe700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 0x7f2a0006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:08.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.894+0000 7f2a127fc700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 msgr2=0x7f2a0006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:08.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.894+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f2a0408ac20 con 0x7f2a140d01d0 2023-10-16T10:00:08.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.894+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 0x7f2a0006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:08.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:08.898+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2a04055aa0 con 0x7f2a140d01d0 2023-10-16T10:00:08.947 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:08.946+0000 7fba372e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:00:09.005 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:09.005+0000 7fceaff5a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:00:09.021 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:09.020+0000 7fba372e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:00:09.081 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:09.081+0000 7fceaff5a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:00:09.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:09.095+0000 7f2a127fc700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 msgr2=0x7f2a0006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:09.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:09.095+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 0x7f2a0006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:09.101 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:09.100+0000 7fba372e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:00:09.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:09.127+0000 7f2a197a7700 1 -- 172.21.15.181:0/457713573 --> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a140cef30 con 0x7f2a0006b070 2023-10-16T10:00:09.163 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:09.163+0000 7fceaff5a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:00:09.469 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:09.467+0000 7fba372e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:00:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:09.496+0000 7f2a127fc700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 msgr2=0x7f2a0006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:09.496+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 0x7f2a0006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:00:09.533 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:09.533+0000 7fceaff5a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:00:10.036 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:10.035+0000 7fba372e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:00:10.102 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:10.100+0000 7fceaff5a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:00:10.105 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:10.106 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:10.104+0000 7fba372e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:00:10.108 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:10.175 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:10.173+0000 7fceaff5a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:00:10.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:10.296+0000 7f2a127fc700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 msgr2=0x7f2a0006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:10.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:10.296+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 0x7f2a0006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:00:10.742 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:10.741+0000 7fba372e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:10.807 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:10.806+0000 7fceaff5a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:10.819 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:10.818+0000 7fba372e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:00:10.885 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:10.884+0000 7fceaff5a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:00:10.962 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:10.961+0000 7fba372e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:11.029 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:11.028+0000 7fceaff5a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:11.188 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:11.187+0000 7fba372e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:00:11.258 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:11.256+0000 7fceaff5a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:00:11.261 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:11.260+0000 7fba372e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:00:11.332 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:11.330+0000 7fceaff5a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:00:11.526 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:11.524+0000 7fba372e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:00:11.597 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:11.595+0000 7fceaff5a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:00:11.745 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:11.744+0000 7fba372e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:00:11.819 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:11.817+0000 7fceaff5a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:00:11.893 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:11.892+0000 7fba372e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:00:11.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:11.897+0000 7f2a127fc700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 msgr2=0x7f2a0006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/752673838 2023-10-16T10:00:11.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:11.897+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 0x7f2a0006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:00:11.968 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:11.967+0000 7fceaff5a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:00:12.005 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:12.004+0000 7fba372e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:00:12.082 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:12.080+0000 7fceaff5a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:00:12.159 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:12.157+0000 7fba372e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:00:12.237 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:12.236+0000 7fceaff5a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:00:12.422 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:12.421+0000 7fba372e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:12.501 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:12.499+0000 7fceaff5a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:12.580 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:12.579+0000 7fba372e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:00:12.662 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:12.660+0000 7fceaff5a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:00:13.096 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:13.095+0000 7fba372e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:00:13.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:13.164+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 44) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2a04055e70 con 0x7f2a140d01d0 2023-10-16T10:00:13.189 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.187+0000 7fceaff5a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:00:13.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:13.210+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 45) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f2a0405d500 con 0x7f2a140d01d0 2023-10-16T10:00:13.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:13.211+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 msgr2=0x7f2a0006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:13.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:13.211+0000 7f29fb7fe700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/752673838,v1:172.21.15.181:6833/752673838] conn(0x7f2a0006b070 0x7f2a0006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:13.364 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce22f35700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.364 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce22f35700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.365 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce22f35700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.365 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce22f35700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.365 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce22f35700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.365 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce2573a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.365 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce2573a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.366 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce2573a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.366 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce2573a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:13.366 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:13.362+0000 7fce2573a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:14.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.215+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 46) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2a04056740 con 0x7f2a140d01d0 2023-10-16T10:00:14.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.215+0000 7f29fb7fe700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f2a000279c0 0x7f2a00027da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.215+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 --> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a00071070 con 0x7f2a000279c0 2023-10-16T10:00:14.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.216+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f2a000279c0 0x7f2a00027da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.219+0000 7f2a127fc700 1 --2- 172.21.15.181:0/457713573 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f2a000279c0 0x7f2a00027da0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a14160320 tx=0x7f2a08008040 comp rx=0 tx=0).ready entity=mgr.7176 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:14.229 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:14.227+0000 7fce2cf89700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-16T10:00:14.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.227+0000 7f29fb7fe700 1 -- 172.21.15.181:0/457713573 <== mgr.7176 v2:172.21.15.181:6832/2464567114 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f2a00071070 con 0x7f2a000279c0 2023-10-16T10:00:14.231 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-16T10:00:14.234 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:00:14.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:00:14.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.362+0000 7f861b721700 1 Processor -- start 2023-10-16T10:00:14.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.362+0000 7f861b721700 1 -- start start 2023-10-16T10:00:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f861b721700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86140b2010 0x7f86140cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f861b721700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8614071be0 0x7f86140b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f861b721700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f861406f3b0 0x7f861406f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f861b721700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86140ce1f0 con 0x7f8614071be0 2023-10-16T10:00:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f861b721700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8614072090 con 0x7f86140b2010 2023-10-16T10:00:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f861b721700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86140b2930 con 0x7f861406f3b0 2023-10-16T10:00:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f8618cbc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8614071be0 0x7f86140b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f8618cbc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8614071be0 0x7f86140b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43608/0 (socket says 172.21.15.181:43608) 2023-10-16T10:00:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.363+0000 7f8618cbc700 1 -- 172.21.15.181:0/2761559293 learned_addr learned my addr 172.21.15.181:0/2761559293 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f86194bd700 1 --2- 172.21.15.181:0/2761559293 >> v2:172.21.15.190:3300/0 conn(0x7f86140b2010 0x7f86140cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f8619cbe700 1 --2- 172.21.15.181:0/2761559293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f861406f3b0 0x7f861406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f8618cbc700 1 -- 172.21.15.181:0/2761559293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f861406f3b0 msgr2=0x7f861406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f8618cbc700 1 --2- 172.21.15.181:0/2761559293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f861406f3b0 0x7f861406f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f8618cbc700 1 -- 172.21.15.181:0/2761559293 >> v2:172.21.15.190:3300/0 conn(0x7f86140b2010 msgr2=0x7f86140cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:14.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f8618cbc700 1 --2- 172.21.15.181:0/2761559293 >> v2:172.21.15.190:3300/0 conn(0x7f86140b2010 0x7f86140cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f8618cbc700 1 -- 172.21.15.181:0/2761559293 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86140ccda0 con 0x7f8614071be0 2023-10-16T10:00:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f8618cbc700 1 --2- 172.21.15.181:0/2761559293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f86140b23f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f8614070430 tx=0x7f8604009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cb75b6020cdefc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f860bfff700 1 -- 172.21.15.181:0/2761559293 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400b870 con 0x7f8614071be0 2023-10-16T10:00:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f860bfff700 1 -- 172.21.15.181:0/2761559293 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860400b9d0 con 0x7f8614071be0 2023-10-16T10:00:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f860bfff700 1 -- 172.21.15.181:0/2761559293 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400bd60 con 0x7f8614071be0 2023-10-16T10:00:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f861b721700 1 -- 172.21.15.181:0/2761559293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 msgr2=0x7f86140b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.364+0000 7f861b721700 1 --2- 172.21.15.181:0/2761559293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f86140b23f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f8614070430 tx=0x7f8604009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.365+0000 7f861b721700 1 -- 172.21.15.181:0/2761559293 shutdown_connections 2023-10-16T10:00:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.365+0000 7f861b721700 1 --2- 172.21.15.181:0/2761559293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f861406f3b0 0x7f861406f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.365+0000 7f861b721700 1 --2- 172.21.15.181:0/2761559293 >> v2:172.21.15.190:3300/0 conn(0x7f86140b2010 0x7f86140cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.365+0000 7f861b721700 1 --2- 172.21.15.181:0/2761559293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f86140b23f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.365+0000 7f861b721700 1 -- 172.21.15.181:0/2761559293 >> 172.21.15.181:0/2761559293 conn(0x7f86140cd010 msgr2=0x7f86140cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.365+0000 7f861b721700 1 -- 172.21.15.181:0/2761559293 shutdown_connections 2023-10-16T10:00:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.365+0000 7f861b721700 1 -- 172.21.15.181:0/2761559293 wait complete. 2023-10-16T10:00:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.366+0000 7f861b721700 1 Processor -- start 2023-10-16T10:00:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.366+0000 7f861b721700 1 -- start start 2023-10-16T10:00:14.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861406f3b0 0x7f86140c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f8614166d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140b2010 0x7f86140c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86141672b0 con 0x7f8614071be0 2023-10-16T10:00:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614150c30 con 0x7f861406f3b0 2023-10-16T10:00:14.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8614150300 con 0x7f86140b2010 2023-10-16T10:00:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f8614166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f8614166d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43614/0 (socket says 172.21.15.181:43614) 2023-10-16T10:00:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 -- 172.21.15.181:0/77942122 learned_addr learned my addr 172.21.15.181:0/77942122 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f86194bd700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861406f3b0 0x7f86140c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8619cbe700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140b2010 0x7f86140c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 -- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140b2010 msgr2=0x7f86140c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140b2010 0x7f86140c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 -- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861406f3b0 msgr2=0x7f86140c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861406f3b0 0x7f86140c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:14.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 -- 172.21.15.181:0/77942122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604009550 con 0x7f8614071be0 2023-10-16T10:00:14.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8618cbc700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f8614166d70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f8604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:14.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8609ffb700 1 -- 172.21.15.181:0/77942122 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604004520 con 0x7f8614071be0 2023-10-16T10:00:14.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8609ffb700 1 -- 172.21.15.181:0/77942122 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860401e070 con 0x7f8614071be0 2023-10-16T10:00:14.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86141505e0 con 0x7f8614071be0 2023-10-16T10:00:14.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f8609ffb700 1 -- 172.21.15.181:0/77942122 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86040038c0 con 0x7f8614071be0 2023-10-16T10:00:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.367+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86140ca390 con 0x7f8614071be0 2023-10-16T10:00:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.368+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86140b6510 con 0x7f8614071be0 2023-10-16T10:00:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.369+0000 7f8609ffb700 1 -- 172.21.15.181:0/77942122 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 46) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f8604003a20 con 0x7f8614071be0 2023-10-16T10:00:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.370+0000 7f8609ffb700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f85f806ada0 0x7f85f806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.370+0000 7f8609ffb700 1 -- 172.21.15.181:0/77942122 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f8604020070 con 0x7f8614071be0 2023-10-16T10:00:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.370+0000 7f86194bd700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f85f806ada0 0x7f85f806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.370+0000 7f86194bd700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f85f806ada0 0x7f85f806d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86141f6770 tx=0x7f8610008040 comp rx=0 tx=0).ready entity=mgr.7176 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:14.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.374+0000 7f8609ffb700 1 -- 172.21.15.181:0/77942122 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f86040562e0 con 0x7f8614071be0 2023-10-16T10:00:14.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:14.622+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f86140727b0 con 0x7f8614071be0 2023-10-16T10:00:15.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.217+0000 7f8609ffb700 1 -- 172.21.15.181:0/77942122 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v47) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8604059900 con 0x7f8614071be0 2023-10-16T10:00:15.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.221+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f85f806ada0 msgr2=0x7f85f806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.221+0000 7f861b721700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f85f806ada0 0x7f85f806d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86141f6770 tx=0x7f8610008040 comp rx=0 tx=0).stop 2023-10-16T10:00:15.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.221+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 msgr2=0x7f8614166d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.221+0000 7f861b721700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f8614166d70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f8604004750 comp rx=0 tx=0).stop 2023-10-16T10:00:15.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.222+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 shutdown_connections 2023-10-16T10:00:15.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.222+0000 7f861b721700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140b2010 0x7f86140c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.222+0000 7f861b721700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f85f806ada0 0x7f85f806d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.222+0000 7f861b721700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861406f3b0 0x7f86140c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.222+0000 7f861b721700 1 --2- 172.21.15.181:0/77942122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8614071be0 0x7f8614166d70 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.222+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 >> 172.21.15.181:0/77942122 conn(0x7f86140cd010 msgr2=0x7f86140b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:15.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.223+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 shutdown_connections 2023-10-16T10:00:15.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.223+0000 7f861b721700 1 -- 172.21.15.181:0/77942122 wait complete. 2023-10-16T10:00:15.241 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:00:15.241 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:00:15.296 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:00:15.342 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:00:15.342 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:00:15.343 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:15.360 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:00:15.360 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:15.367 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:15.366+0000 7f2881e03200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:00:15.373 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:15.372+0000 7ff3a6f4b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:00:15.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:15.433 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:15.432+0000 7f2881e03200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:15.441 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:15.440+0000 7ff3a6f4b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:15.505 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:15.504+0000 7f2881e03200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:00:15.518 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:15.516+0000 7ff3a6f4b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:00:15.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.551+0000 7fbed130b700 1 Processor -- start 2023-10-16T10:00:15.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbed130b700 1 -- start start 2023-10-16T10:00:15.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbed130b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc161b60 0x7fbecc06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbed130b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbecc161f40 0x7fbecc0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbed130b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbecc160d20 0x7fbecc1611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbed130b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc162800 con 0x7fbecc161f40 2023-10-16T10:00:15.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbed130b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc0d43e0 con 0x7fbecc160d20 2023-10-16T10:00:15.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbed130b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1616f0 con 0x7fbecc161b60 2023-10-16T10:00:15.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.552+0000 7fbeca7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbecc161f40 0x7fbecc0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbecc161f40 0x7fbecc0d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43668/0 (socket says 172.21.15.181:43668) 2023-10-16T10:00:15.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 -- 172.21.15.181:0/3663852596 learned_addr learned my addr 172.21.15.181:0/3663852596 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:15.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbecaffd700 1 --2- 172.21.15.181:0/3663852596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc161b60 0x7fbecc06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbecb7fe700 1 --2- 172.21.15.181:0/3663852596 >> v2:172.21.15.190:3300/0 conn(0x7fbecc160d20 0x7fbecc1611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 -- 172.21.15.181:0/3663852596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc161b60 msgr2=0x7fbecc06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:15.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 --2- 172.21.15.181:0/3663852596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc161b60 0x7fbecc06d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 -- 172.21.15.181:0/3663852596 >> v2:172.21.15.190:3300/0 conn(0x7fbecc160d20 msgr2=0x7fbecc1611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:15.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 --2- 172.21.15.181:0/3663852596 >> v2:172.21.15.190:3300/0 conn(0x7fbecc160d20 0x7fbecc1611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 -- 172.21.15.181:0/3663852596 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0d2560 con 0x7fbecc161f40 2023-10-16T10:00:15.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbeca7fc700 1 --2- 172.21.15.181:0/3663852596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161f40 0x7fbecc0d2020 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0cdde0 tx=0x7fbec0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd050aacd812e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:15.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbec9ffb700 1 -- 172.21.15.181:0/3663852596 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0011830 con 0x7fbecc161f40 2023-10-16T10:00:15.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbec9ffb700 1 -- 172.21.15.181:0/3663852596 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0011990 con 0x7fbecc161f40 2023-10-16T10:00:15.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.553+0000 7fbec9ffb700 1 -- 172.21.15.181:0/3663852596 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0011d20 con 0x7fbecc161f40 2023-10-16T10:00:15.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 -- 172.21.15.181:0/3663852596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161f40 msgr2=0x7fbecc0d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 --2- 172.21.15.181:0/3663852596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161f40 0x7fbecc0d2020 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0cdde0 tx=0x7fbec0009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 -- 172.21.15.181:0/3663852596 shutdown_connections 2023-10-16T10:00:15.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 --2- 172.21.15.181:0/3663852596 >> v2:172.21.15.190:3300/0 conn(0x7fbecc160d20 0x7fbecc1611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 --2- 172.21.15.181:0/3663852596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc161b60 0x7fbecc06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 --2- 172.21.15.181:0/3663852596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161f40 0x7fbecc0d2020 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 -- 172.21.15.181:0/3663852596 >> 172.21.15.181:0/3663852596 conn(0x7fbecc09cba0 msgr2=0x7fbecc0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:15.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 -- 172.21.15.181:0/3663852596 shutdown_connections 2023-10-16T10:00:15.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.554+0000 7fbed130b700 1 -- 172.21.15.181:0/3663852596 wait complete. 2023-10-16T10:00:15.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.555+0000 7fbed130b700 1 Processor -- start 2023-10-16T10:00:15.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.555+0000 7fbed130b700 1 -- start start 2023-10-16T10:00:15.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbed130b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc160d20 0x7fbecc1f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbed130b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161b60 0x7fbecc152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbed130b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbecc161f40 0x7fbecc09e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbed130b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1f4ea0 con 0x7fbecc161b60 2023-10-16T10:00:15.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbed130b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1f3f40 con 0x7fbecc161f40 2023-10-16T10:00:15.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbed130b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1f4220 con 0x7fbecc160d20 2023-10-16T10:00:15.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161b60 0x7fbecc152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161b60 0x7fbecc152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43676/0 (socket says 172.21.15.181:43676) 2023-10-16T10:00:15.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 -- 172.21.15.181:0/2643222372 learned_addr learned my addr 172.21.15.181:0/2643222372 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:15.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbecaffd700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc160d20 0x7fbecc1f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbecb7fe700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbecc161f40 0x7fbecc09e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 -- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc160d20 msgr2=0x7fbecc1f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc160d20 0x7fbecc1f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 -- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbecc161f40 msgr2=0x7fbecc09e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbecc161f40 0x7fbecc09e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.556+0000 7fbeca7fc700 1 -- 172.21.15.181:0/2643222372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec0009550 con 0x7fbecc161b60 2023-10-16T10:00:15.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.557+0000 7fbeca7fc700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161b60 0x7fbecc152850 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009c10 tx=0x7fbec0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:15.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.557+0000 7fbeb37fe700 1 -- 172.21.15.181:0/2643222372 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0004450 con 0x7fbecc161b60 2023-10-16T10:00:15.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.557+0000 7fbeb37fe700 1 -- 172.21.15.181:0/2643222372 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec001e070 con 0x7fbecc161b60 2023-10-16T10:00:15.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.557+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc1f4500 con 0x7fbecc161b60 2023-10-16T10:00:15.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.557+0000 7fbeb37fe700 1 -- 172.21.15.181:0/2643222372 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00038e0 con 0x7fbecc161b60 2023-10-16T10:00:15.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.557+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc1f4ac0 con 0x7fbecc161b60 2023-10-16T10:00:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.558+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc04ebb0 con 0x7fbecc161b60 2023-10-16T10:00:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.559+0000 7fbeb37fe700 1 -- 172.21.15.181:0/2643222372 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbec0003a40 con 0x7fbecc161b60 2023-10-16T10:00:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.559+0000 7fbeb37fe700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 0x7fbeb806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.559+0000 7fbecaffd700 1 -- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 msgr2=0x7fbeb806d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.559+0000 7fbeb37fe700 1 -- 172.21.15.181:0/2643222372 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fbec008a2c0 con 0x7fbecc161b60 2023-10-16T10:00:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.559+0000 7fbecaffd700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 0x7fbeb806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.564+0000 7fbeb37fe700 1 -- 172.21.15.181:0/2643222372 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbec0055330 con 0x7fbecc161b60 2023-10-16T10:00:15.583 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:15.582+0000 7f2881e03200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:00:15.602 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:15.600+0000 7ff3a6f4b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:00:15.727 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:15.726+0000 7f2881e03200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:00:15.749 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:15.747+0000 7ff3a6f4b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:00:15.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.760+0000 7fbecaffd700 1 -- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 msgr2=0x7fbeb806d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:15.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.760+0000 7fbecaffd700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 0x7fbeb806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:15.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.767+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbecc0b61a0 con 0x7fbecc161b60 2023-10-16T10:00:15.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.769+0000 7fbeb37fe700 1 -- 172.21.15.181:0/2643222372 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17753 (secure 0 0 0) 0x7fbec0058950 con 0x7fbecc161b60 2023-10-16T10:00:15.771 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:15.771 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":91,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:13.202252+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:56.829733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:58.008691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1686162648":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/299388856":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/315145191":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/2235576983":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/3806937704":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/4212605490":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6833/752673838":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6832/752673838":"2023-10-17T10:00:13.202216+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:15.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 msgr2=0x7fbeb806d210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:15.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 0x7fbeb806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161b60 msgr2=0x7fbecc152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161b60 0x7fbecc152850 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009c10 tx=0x7fbec0004750 comp rx=0 tx=0).stop 2023-10-16T10:00:15.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 shutdown_connections 2023-10-16T10:00:15.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbecc160d20 0x7fbecc1f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbeb806ad50 0x7fbeb806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbecc161f40 0x7fbecc09e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 --2- 172.21.15.181:0/2643222372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbecc161b60 0x7fbecc152850 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 >> 172.21.15.181:0/2643222372 conn(0x7fbecc09cba0 msgr2=0x7fbecc0b6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:15.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 shutdown_connections 2023-10-16T10:00:15.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.771+0000 7fbed130b700 1 -- 172.21.15.181:0/2643222372 wait complete. 2023-10-16T10:00:15.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:59:56.829733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:59:58.008691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:00:15.791 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:00:15.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.955+0000 7f4c57b53700 1 Processor -- start 2023-10-16T10:00:15.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.955+0000 7f4c57b53700 1 -- start start 2023-10-16T10:00:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c57b53700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c500d01d0 0x7f4c500ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c57b53700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c500d1500 0x7f4c500d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c57b53700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c500cf580 con 0x7f4c5015ea40 2023-10-16T10:00:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c57b53700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500d4290 con 0x7f4c500d01d0 2023-10-16T10:00:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c57b53700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500ce970 con 0x7f4c500d1500 2023-10-16T10:00:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c500d01d0 0x7f4c500ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c500d01d0 0x7f4c500ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43712/0 (socket says 172.21.15.181:43712) 2023-10-16T10:00:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 -- 172.21.15.181:0/2879775810 learned_addr learned my addr 172.21.15.181:0/2879775810 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:15.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c560f0700 1 --2- 172.21.15.181:0/2879775810 >> v2:172.21.15.190:3300/0 conn(0x7f4c500d1500 0x7f4c500d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 -- 172.21.15.181:0/2879775810 >> v2:172.21.15.190:3300/0 conn(0x7f4c500d1500 msgr2=0x7f4c500d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 --2- 172.21.15.181:0/2879775810 >> v2:172.21.15.190:3300/0 conn(0x7f4c500d1500 0x7f4c500d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 -- 172.21.15.181:0/2879775810 >> v1:172.21.15.190:6789/0 conn(0x7f4c5015ea40 legacy=0x7f4c500d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 -- 172.21.15.181:0/2879775810 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500cec50 con 0x7f4c500d01d0 2023-10-16T10:00:15.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c558ef700 1 --2- 172.21.15.181:0/2879775810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500d01d0 0x7f4c500ce2d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4c501625d0 tx=0x7f4c4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d3f0abcd6f650a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c548ed700 1 -- 172.21.15.181:0/2879775810 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c011830 con 0x7f4c500d01d0 2023-10-16T10:00:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c548ed700 1 -- 172.21.15.181:0/2879775810 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c011990 con 0x7f4c500d01d0 2023-10-16T10:00:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.956+0000 7f4c548ed700 1 -- 172.21.15.181:0/2879775810 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c011d20 con 0x7f4c500d01d0 2023-10-16T10:00:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 -- 172.21.15.181:0/2879775810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500d01d0 msgr2=0x7f4c500ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2879775810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500d01d0 0x7f4c500ce2d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4c501625d0 tx=0x7f4c4c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 -- 172.21.15.181:0/2879775810 shutdown_connections 2023-10-16T10:00:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2879775810 >> v2:172.21.15.190:3300/0 conn(0x7f4c500d1500 0x7f4c500d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2879775810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c500d01d0 0x7f4c500ce2d0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 -- 172.21.15.181:0/2879775810 >> 172.21.15.181:0/2879775810 conn(0x7f4c5009cb60 msgr2=0x7f4c500d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 -- 172.21.15.181:0/2879775810 shutdown_connections 2023-10-16T10:00:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.957+0000 7f4c57b53700 1 -- 172.21.15.181:0/2879775810 wait complete. 2023-10-16T10:00:15.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.958+0000 7f4c57b53700 1 Processor -- start 2023-10-16T10:00:15.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.958+0000 7f4c57b53700 1 -- start start 2023-10-16T10:00:15.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c57b53700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c500d01d0 0x7f4c501f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c57b53700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c500d1500 0x7f4c50158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c57b53700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5015ea40 0x7f4c5015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c57b53700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500b9500 con 0x7f4c5015ea40 2023-10-16T10:00:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c57b53700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c500b7bd0 con 0x7f4c500d1500 2023-10-16T10:00:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c57b53700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c501f4520 con 0x7f4c500d01d0 2023-10-16T10:00:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c560f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5015ea40 0x7f4c5015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c560f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5015ea40 0x7f4c5015abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43722/0 (socket says 172.21.15.181:43722) 2023-10-16T10:00:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c560f0700 1 -- 172.21.15.181:0/2546997824 learned_addr learned my addr 172.21.15.181:0/2546997824 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c558ef700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c500d01d0 0x7f4c501f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.959+0000 7f4c550ee700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c500d1500 0x7f4c50158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c560f0700 1 -- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c500d01d0 msgr2=0x7f4c501f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c560f0700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c500d01d0 0x7f4c501f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c560f0700 1 -- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c500d1500 msgr2=0x7f4c50158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c560f0700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c500d1500 0x7f4c50158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c560f0700 1 -- 172.21.15.181:0/2546997824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c009550 con 0x7f4c5015ea40 2023-10-16T10:00:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c558ef700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c500d01d0 0x7f4c501f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c560f0700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5015ea40 0x7f4c5015abe0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009dd0 tx=0x7f4c4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c467fc700 1 -- 172.21.15.181:0/2546997824 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48009c40 con 0x7f4c5015ea40 2023-10-16T10:00:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c467fc700 1 -- 172.21.15.181:0/2546997824 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4800f040 con 0x7f4c5015ea40 2023-10-16T10:00:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c467fc700 1 -- 172.21.15.181:0/2546997824 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800ceb0 con 0x7f4c5015ea40 2023-10-16T10:00:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c500b7610 con 0x7f4c5015ea40 2023-10-16T10:00:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.960+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c500c8af0 con 0x7f4c5015ea40 2023-10-16T10:00:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.961+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5004ebb0 con 0x7f4c5015ea40 2023-10-16T10:00:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.963+0000 7f4c467fc700 1 -- 172.21.15.181:0/2546997824 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 47) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4c48014480 con 0x7f4c5015ea40 2023-10-16T10:00:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.963+0000 7f4c467fc700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 0x7f4c3406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.963+0000 7f4c558ef700 1 -- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 msgr2=0x7f4c3406d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.963+0000 7f4c467fc700 1 -- 172.21.15.181:0/2546997824 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f4c4808a440 con 0x7f4c5015ea40 2023-10-16T10:00:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.963+0000 7f4c558ef700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 0x7f4c3406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:15.967+0000 7f4c467fc700 1 -- 172.21.15.181:0/2546997824 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c480555f0 con 0x7f4c5015ea40 2023-10-16T10:00:16.049 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:16.048+0000 7f2881e03200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:00:16.092 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:16.091+0000 7ff3a6f4b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:00:16.163 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:16.162+0000 7f2881e03200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:00:16.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.162+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c500b9680 con 0x7f4c5015ea40 2023-10-16T10:00:16.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.163+0000 7f4c467fc700 1 -- 172.21.15.181:0/2546997824 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c48018020 con 0x7f4c5015ea40 2023-10-16T10:00:16.165 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:00:16.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.164+0000 7f4c558ef700 1 -- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 msgr2=0x7f4c3406d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:16.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.164+0000 7f4c558ef700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 0x7f4c3406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:16.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 msgr2=0x7f4c3406d210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:16.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 0x7f4c3406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5015ea40 msgr2=0x7f4c5015abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5015ea40 0x7f4c5015abe0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009dd0 tx=0x7f4c4800c3a0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 shutdown_connections 2023-10-16T10:00:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c500d01d0 0x7f4c501f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4c3406ad50 0x7f4c3406d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c5015ea40 0x7f4c5015abe0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 --2- 172.21.15.181:0/2546997824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c500d1500 0x7f4c50158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 >> 172.21.15.181:0/2546997824 conn(0x7f4c5009cb60 msgr2=0x7f4c5015ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:16.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 shutdown_connections 2023-10-16T10:00:16.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.165+0000 7f4c57b53700 1 -- 172.21.15.181:0/2546997824 wait complete. 2023-10-16T10:00:16.181 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:00:16.213 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:16.212+0000 7ff3a6f4b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:00:16.238 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:16.237+0000 7f2881e03200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:00:16.294 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:16.293+0000 7ff3a6f4b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:00:16.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.308+0000 7f4ff5292700 1 Processor -- start 2023-10-16T10:00:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.308+0000 7f4ff5292700 1 -- start start 2023-10-16T10:00:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.308+0000 7f4ff5292700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ff006afb0 0x7f4ff01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.308+0000 7f4ff5292700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff00cef00 con 0x7f4ff015da70 2023-10-16T10:00:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.308+0000 7f4ff5292700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff00cec20 con 0x7f4ff0072760 2023-10-16T10:00:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.308+0000 7f4ff5292700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff0071c80 con 0x7f4ff006afb0 2023-10-16T10:00:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4feeffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4ff015da70 0x7f4ff00ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54806/0 (socket says 172.21.15.181:54806) 2023-10-16T10:00:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4feeffd700 1 -- 172.21.15.181:0/2010392146 learned_addr learned my addr 172.21.15.181:0/2010392146 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fef7fe700 1 --2- 172.21.15.181:0/2010392146 >> v2:172.21.15.190:3300/0 conn(0x7f4ff006afb0 0x7f4ff01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fedffb700 1 -- 172.21.15.181:0/2010392146 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259643534 0 0) 0x7f4ff00cef00 con 0x7f4ff015da70 2023-10-16T10:00:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fedffb700 1 -- 172.21.15.181:0/2010392146 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd80035b0 con 0x7f4ff015da70 2023-10-16T10:00:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fef7fe700 1 -- 172.21.15.181:0/2010392146 >> v1:172.21.15.190:6789/0 conn(0x7f4ff0072760 legacy=0x7f4ff006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fef7fe700 1 -- 172.21.15.181:0/2010392146 >> v1:172.21.15.181:6789/0 conn(0x7f4ff015da70 legacy=0x7f4ff00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fef7fe700 1 -- 172.21.15.181:0/2010392146 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0071f60 con 0x7f4ff006afb0 2023-10-16T10:00:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fef7fe700 1 --2- 172.21.15.181:0/2010392146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff006afb0 0x7f4ff01615f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009d90 tx=0x7f4fe800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30486ef405b7a35b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fedffb700 1 -- 172.21.15.181:0/2010392146 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800e070 con 0x7f4ff006afb0 2023-10-16T10:00:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fedffb700 1 -- 172.21.15.181:0/2010392146 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe800bb20 con 0x7f4ff006afb0 2023-10-16T10:00:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.309+0000 7f4fedffb700 1 -- 172.21.15.181:0/2010392146 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8012430 con 0x7f4ff006afb0 2023-10-16T10:00:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.310+0000 7f4ff5292700 1 -- 172.21.15.181:0/2010392146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff006afb0 msgr2=0x7f4ff01615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.310+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2010392146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff006afb0 0x7f4ff01615f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009d90 tx=0x7f4fe800b330 comp rx=0 tx=0).stop 2023-10-16T10:00:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.310+0000 7f4ff5292700 1 -- 172.21.15.181:0/2010392146 shutdown_connections 2023-10-16T10:00:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.310+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2010392146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff006afb0 0x7f4ff01615f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.310+0000 7f4ff5292700 1 -- 172.21.15.181:0/2010392146 >> 172.21.15.181:0/2010392146 conn(0x7f4ff00cd010 msgr2=0x7f4ff00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.310+0000 7f4ff5292700 1 -- 172.21.15.181:0/2010392146 shutdown_connections 2023-10-16T10:00:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.310+0000 7f4ff5292700 1 -- 172.21.15.181:0/2010392146 wait complete. 2023-10-16T10:00:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.312+0000 7f4ff5292700 1 Processor -- start 2023-10-16T10:00:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.312+0000 7f4ff5292700 1 -- start start 2023-10-16T10:00:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4ff5292700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff006afb0 0x7f4ff00d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4ff5292700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff0072760 0x7f4ff00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4ff5292700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff015da70 0x7f4ff00ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4ff5292700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff01f3350 con 0x7f4ff015da70 2023-10-16T10:00:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4ff5292700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff01f3630 con 0x7f4ff0072760 2023-10-16T10:00:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4ff5292700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff01f3910 con 0x7f4ff006afb0 2023-10-16T10:00:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4fef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff015da70 0x7f4ff00ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4fef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff015da70 0x7f4ff00ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43738/0 (socket says 172.21.15.181:43738) 2023-10-16T10:00:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4fef7fe700 1 -- 172.21.15.181:0/2039848159 learned_addr learned my addr 172.21.15.181:0/2039848159 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4fee7fc700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff0072760 0x7f4ff00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4feeffd700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff006afb0 0x7f4ff00d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4feeffd700 1 -- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff0072760 msgr2=0x7f4ff00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4feeffd700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff0072760 0x7f4ff00ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4feeffd700 1 -- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff015da70 msgr2=0x7f4ff00ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4feeffd700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff015da70 0x7f4ff00ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4feeffd700 1 -- 172.21.15.181:0/2039848159 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8009a70 con 0x7f4ff006afb0 2023-10-16T10:00:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4fee7fc700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff0072760 0x7f4ff00ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4fef7fe700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff015da70 0x7f4ff00ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.313+0000 7f4feeffd700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff006afb0 0x7f4ff00d0d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c490 tx=0x7f4fe000c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.314+0000 7f4fd77fe700 1 -- 172.21.15.181:0/2039848159 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0005d70 con 0x7f4ff006afb0 2023-10-16T10:00:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.314+0000 7f4fd77fe700 1 -- 172.21.15.181:0/2039848159 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00070d0 con 0x7f4ff006afb0 2023-10-16T10:00:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.314+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff01f3bf0 con 0x7f4ff006afb0 2023-10-16T10:00:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.314+0000 7f4fd77fe700 1 -- 172.21.15.181:0/2039848159 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0007440 con 0x7f4ff006afb0 2023-10-16T10:00:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.314+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff01f2fb0 con 0x7f4ff006afb0 2023-10-16T10:00:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.315+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff01f4110 con 0x7f4ff006afb0 2023-10-16T10:00:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.315+0000 7f4fd77fe700 1 -- 172.21.15.181:0/2039848159 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4fe00075a0 con 0x7f4ff006afb0 2023-10-16T10:00:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.315+0000 7f4fd77fe700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 0x7f4fdc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.315+0000 7f4fee7fc700 1 -- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 msgr2=0x7f4fdc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.316+0000 7f4fd77fe700 1 -- 172.21.15.181:0/2039848159 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f4fe0088e50 con 0x7f4ff006afb0 2023-10-16T10:00:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.316+0000 7f4fee7fc700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 0x7f4fdc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:16.326 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:16.318+0000 7f2881e03200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:00:16.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.320+0000 7f4fd77fe700 1 -- 172.21.15.181:0/2039848159 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4ff01f4110 con 0x7f4ff006afb0 2023-10-16T10:00:16.381 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:16.378+0000 7ff3a6f4b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:00:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.517+0000 7f4fee7fc700 1 -- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 msgr2=0x7f4fdc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.517+0000 7f4fee7fc700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 0x7f4fdc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:16.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.521+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ff01f4110 con 0x7f4ff006afb0 2023-10-16T10:00:16.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.521+0000 7f4fd77fe700 1 -- 172.21.15.181:0/2039848159 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fe0017020 con 0x7f4ff006afb0 2023-10-16T10:00:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:00:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 msgr2=0x7f4fdc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 0x7f4fdc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff006afb0 msgr2=0x7f4ff00d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff006afb0 0x7f4ff00d0d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c490 tx=0x7f4fe000c7a0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 shutdown_connections 2023-10-16T10:00:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ff006afb0 0x7f4ff00d0d60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4fdc06b020 0x7f4fdc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff015da70 0x7f4ff00ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 --2- 172.21.15.181:0/2039848159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff0072760 0x7f4ff00ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 >> 172.21.15.181:0/2039848159 conn(0x7f4ff00cd010 msgr2=0x7f4ff015b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 shutdown_connections 2023-10-16T10:00:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.524+0000 7f4ff5292700 1 -- 172.21.15.181:0/2039848159 wait complete. 2023-10-16T10:00:16.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:00:16.687 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:16.686+0000 7f2881e03200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:00:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.703+0000 7f514f80c700 1 Processor -- start 2023-10-16T10:00:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.703+0000 7f514f80c700 1 -- start start 2023-10-16T10:00:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514f80c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480ce6c0 0x7f51480cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514f80c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51480cc6a0 0x7f51480ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514f80c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51480b1f70 con 0x7f514815f150 2023-10-16T10:00:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514f80c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51480b2ae0 con 0x7f51480cc6a0 2023-10-16T10:00:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514f80c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480b2dc0 con 0x7f51480ce6c0 2023-10-16T10:00:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514dda9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51480cc6a0 0x7f51480ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514dda9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51480cc6a0 0x7f51480ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43764/0 (socket says 172.21.15.181:43764) 2023-10-16T10:00:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514dda9700 1 -- 172.21.15.181:0/668370906 learned_addr learned my addr 172.21.15.181:0/668370906 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514cda7700 1 --2- 172.21.15.181:0/668370906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480ce6c0 0x7f51480cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514dda9700 1 -- 172.21.15.181:0/668370906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480ce6c0 msgr2=0x7f51480cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.704+0000 7f514dda9700 1 --2- 172.21.15.181:0/668370906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480ce6c0 0x7f51480cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.705+0000 7f514dda9700 1 -- 172.21.15.181:0/668370906 >> v1:172.21.15.190:6789/0 conn(0x7f514815f150 legacy=0x7f51480cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.705+0000 7f514dda9700 1 -- 172.21.15.181:0/668370906 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480cf9d0 con 0x7f51480cc6a0 2023-10-16T10:00:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.705+0000 7f513ffff700 1 -- 172.21.15.181:0/668370906 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024286725 0 0) 0x7f51480b1f70 con 0x7f514815f150 2023-10-16T10:00:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.705+0000 7f514dda9700 1 --2- 172.21.15.181:0/668370906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480cc6a0 0x7f51480ccb50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5140009e50 tx=0x7f514000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd5faa87f2004a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.705+0000 7f513ffff700 1 -- 172.21.15.181:0/668370906 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514000e070 con 0x7f51480cc6a0 2023-10-16T10:00:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.705+0000 7f513ffff700 1 -- 172.21.15.181:0/668370906 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000b910 con 0x7f51480cc6a0 2023-10-16T10:00:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.705+0000 7f513ffff700 1 -- 172.21.15.181:0/668370906 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514000bcc0 con 0x7f51480cc6a0 2023-10-16T10:00:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 -- 172.21.15.181:0/668370906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480cc6a0 msgr2=0x7f51480ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 --2- 172.21.15.181:0/668370906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480cc6a0 0x7f51480ccb50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5140009e50 tx=0x7f514000b330 comp rx=0 tx=0).stop 2023-10-16T10:00:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 -- 172.21.15.181:0/668370906 shutdown_connections 2023-10-16T10:00:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 --2- 172.21.15.181:0/668370906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480cc6a0 0x7f51480ccb50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 --2- 172.21.15.181:0/668370906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480ce6c0 0x7f51480cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 -- 172.21.15.181:0/668370906 >> 172.21.15.181:0/668370906 conn(0x7f514809a930 msgr2=0x7f51480d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:16.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 -- 172.21.15.181:0/668370906 shutdown_connections 2023-10-16T10:00:16.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.706+0000 7f514f80c700 1 -- 172.21.15.181:0/668370906 wait complete. 2023-10-16T10:00:16.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.707+0000 7f514f80c700 1 Processor -- start 2023-10-16T10:00:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.707+0000 7f514f80c700 1 -- start start 2023-10-16T10:00:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514f80c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480cc6a0 0x7f51480cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514f80c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480ce6c0 0x7f514815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514f80c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514815f150 0x7f5148151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514f80c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480c7910 con 0x7f51480ce6c0 2023-10-16T10:00:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514f80c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514815a970 con 0x7f514815f150 2023-10-16T10:00:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514f80c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514815ac20 con 0x7f51480cc6a0 2023-10-16T10:00:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480ce6c0 0x7f514815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480ce6c0 0x7f514815a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43768/0 (socket says 172.21.15.181:43768) 2023-10-16T10:00:16.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 -- 172.21.15.181:0/467857336 learned_addr learned my addr 172.21.15.181:0/467857336 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:16.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514dda9700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514815f150 0x7f5148151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514d5a8700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480cc6a0 0x7f51480cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:16.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 -- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480cc6a0 msgr2=0x7f51480cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480cc6a0 0x7f51480cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 -- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514815f150 msgr2=0x7f5148151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514815f150 0x7f5148151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 -- 172.21.15.181:0/467857336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140009b30 con 0x7f51480ce6c0 2023-10-16T10:00:16.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514cda7700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480ce6c0 0x7f514815a430 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9b0 tx=0x7f513800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:16.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f513dffb700 1 -- 172.21.15.181:0/467857336 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138009710 con 0x7f51480ce6c0 2023-10-16T10:00:16.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f513dffb700 1 -- 172.21.15.181:0/467857336 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513800ee50 con 0x7f51480ce6c0 2023-10-16T10:00:16.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f513dffb700 1 -- 172.21.15.181:0/467857336 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800ca50 con 0x7f51480ce6c0 2023-10-16T10:00:16.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.708+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514815af00 con 0x7f51480ce6c0 2023-10-16T10:00:16.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.709+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51480c75a0 con 0x7f51480ce6c0 2023-10-16T10:00:16.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.710+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51480c7120 con 0x7f51480ce6c0 2023-10-16T10:00:16.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.711+0000 7f513dffb700 1 -- 172.21.15.181:0/467857336 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f513800cbb0 con 0x7f51480ce6c0 2023-10-16T10:00:16.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.711+0000 7f513dffb700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 0x7f512c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:16.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.711+0000 7f514d5a8700 1 -- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 msgr2=0x7f512c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:16.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.711+0000 7f513dffb700 1 -- 172.21.15.181:0/467857336 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f5138089ac0 con 0x7f51480ce6c0 2023-10-16T10:00:16.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.711+0000 7f514d5a8700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 0x7f512c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:16.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.715+0000 7f513dffb700 1 -- 172.21.15.181:0/467857336 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f51480c7120 con 0x7f51480ce6c0 2023-10-16T10:00:16.768 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:16.766+0000 7ff3a6f4b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:00:16.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.912+0000 7f514d5a8700 1 -- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 msgr2=0x7f512c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:16.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.912+0000 7f514d5a8700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 0x7f512c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:16.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.913+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51480c7120 con 0x7f51480ce6c0 2023-10-16T10:00:16.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.914+0000 7f513dffb700 1 -- 172.21.15.181:0/467857336 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51480c7120 con 0x7f51480ce6c0 2023-10-16T10:00:16.916 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:00:16.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 msgr2=0x7f512c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:16.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 0x7f512c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480ce6c0 msgr2=0x7f514815a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:16.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480ce6c0 0x7f514815a430 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9b0 tx=0x7f513800ecc0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 shutdown_connections 2023-10-16T10:00:16.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51480cc6a0 0x7f51480cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f512c06b020 0x7f512c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f514815f150 0x7f5148151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 --2- 172.21.15.181:0/467857336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51480ce6c0 0x7f514815a430 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:16.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 >> 172.21.15.181:0/467857336 conn(0x7f514809a930 msgr2=0x7f51480cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:16.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 shutdown_connections 2023-10-16T10:00:16.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:16.916+0000 7f514f80c700 1 -- 172.21.15.181:0/467857336 wait complete. 2023-10-16T10:00:16.931 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:00:17.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.052+0000 7f1a389f9700 1 Processor -- start 2023-10-16T10:00:17.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.052+0000 7f1a389f9700 1 -- start start 2023-10-16T10:00:17.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.053+0000 7f1a389f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1a3415e460 0x7f1a340d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.053+0000 7f1a389f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3406ed00 con 0x7f1a3415a0f0 2023-10-16T10:00:17.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.053+0000 7f1a389f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3406e0f0 con 0x7f1a340cb110 2023-10-16T10:00:17.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.053+0000 7f1a389f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3406e3d0 con 0x7f1a3415e460 2023-10-16T10:00:17.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a32d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1a3415a0f0 0x7f1a3415a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54858/0 (socket says 172.21.15.181:54858) 2023-10-16T10:00:17.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a32d9d700 1 -- 172.21.15.181:0/820552507 learned_addr learned my addr 172.21.15.181:0/820552507 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:17.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a3259c700 1 --2- 172.21.15.181:0/820552507 >> v2:172.21.15.190:3300/0 conn(0x7f1a3415e460 0x7f1a340d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a3159a700 1 -- 172.21.15.181:0/820552507 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054569410 0 0) 0x7f1a3406ed00 con 0x7f1a3415a0f0 2023-10-16T10:00:17.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a3159a700 1 -- 172.21.15.181:0/820552507 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a1c0035b0 con 0x7f1a3415a0f0 2023-10-16T10:00:17.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a3259c700 1 -- 172.21.15.181:0/820552507 >> v1:172.21.15.190:6789/0 conn(0x7f1a340cb110 legacy=0x7f1a3415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a3259c700 1 -- 172.21.15.181:0/820552507 >> v1:172.21.15.181:6789/0 conn(0x7f1a3415a0f0 legacy=0x7f1a3415a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a3259c700 1 -- 172.21.15.181:0/820552507 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3406e6b0 con 0x7f1a3415e460 2023-10-16T10:00:17.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.054+0000 7f1a3259c700 1 --2- 172.21.15.181:0/820552507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 0x7f1a340d0fc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009820 tx=0x7f1a24009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2d43cb4ec9272 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:17.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a3159a700 1 -- 172.21.15.181:0/820552507 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24005e30 con 0x7f1a3415e460 2023-10-16T10:00:17.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a3159a700 1 -- 172.21.15.181:0/820552507 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24006030 con 0x7f1a3415e460 2023-10-16T10:00:17.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a3159a700 1 -- 172.21.15.181:0/820552507 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240063c0 con 0x7f1a3415e460 2023-10-16T10:00:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a389f9700 1 -- 172.21.15.181:0/820552507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 msgr2=0x7f1a340d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a389f9700 1 --2- 172.21.15.181:0/820552507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 0x7f1a340d0fc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009820 tx=0x7f1a24009b30 comp rx=0 tx=0).stop 2023-10-16T10:00:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a389f9700 1 -- 172.21.15.181:0/820552507 shutdown_connections 2023-10-16T10:00:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a389f9700 1 --2- 172.21.15.181:0/820552507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 0x7f1a340d0fc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a389f9700 1 -- 172.21.15.181:0/820552507 >> 172.21.15.181:0/820552507 conn(0x7f1a340d04a0 msgr2=0x7f1a340d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:17.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.055+0000 7f1a389f9700 1 -- 172.21.15.181:0/820552507 shutdown_connections 2023-10-16T10:00:17.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.056+0000 7f1a389f9700 1 -- 172.21.15.181:0/820552507 wait complete. 2023-10-16T10:00:17.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.057+0000 7f1a389f9700 1 Processor -- start 2023-10-16T10:00:17.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.057+0000 7f1a389f9700 1 -- start start 2023-10-16T10:00:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a389f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a340cb110 0x7f1a34151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a389f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3415a0f0 0x7f1a340c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a389f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 0x7f1a340c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a389f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340b4470 con 0x7f1a340cb110 2023-10-16T10:00:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a389f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340c4750 con 0x7f1a3415e460 2023-10-16T10:00:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a389f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340c4a30 con 0x7f1a3415a0f0 2023-10-16T10:00:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a340cb110 0x7f1a34151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a3259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a340cb110 0x7f1a34151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43782/0 (socket says 172.21.15.181:43782) 2023-10-16T10:00:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a3259c700 1 -- 172.21.15.181:0/2497225613 learned_addr learned my addr 172.21.15.181:0/2497225613 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.058+0000 7f1a31d9b700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3415a0f0 0x7f1a340c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a3259c700 1 -- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3415a0f0 msgr2=0x7f1a340c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a3259c700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3415a0f0 0x7f1a340c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a3259c700 1 -- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 msgr2=0x7f1a340c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a3259c700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 0x7f1a340c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a3259c700 1 -- 172.21.15.181:0/2497225613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009430 con 0x7f1a340cb110 2023-10-16T10:00:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a31d9b700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3415a0f0 0x7f1a340c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a3259c700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a340cb110 0x7f1a34151ae0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009fd0 tx=0x7f1a24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a22ffd700 1 -- 172.21.15.181:0/2497225613 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24004240 con 0x7f1a340cb110 2023-10-16T10:00:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a22ffd700 1 -- 172.21.15.181:0/2497225613 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a240043e0 con 0x7f1a340cb110 2023-10-16T10:00:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a340b4710 con 0x7f1a340cb110 2023-10-16T10:00:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a22ffd700 1 -- 172.21.15.181:0/2497225613 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240036a0 con 0x7f1a340cb110 2023-10-16T10:00:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.059+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a341f0870 con 0x7f1a340cb110 2023-10-16T10:00:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.060+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3406f170 con 0x7f1a340cb110 2023-10-16T10:00:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.060+0000 7f1a22ffd700 1 -- 172.21.15.181:0/2497225613 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f1a24022070 con 0x7f1a340cb110 2023-10-16T10:00:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.061+0000 7f1a22ffd700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 0x7f1a1406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.061+0000 7f1a31d9b700 1 -- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 msgr2=0x7f1a1406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:17.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.061+0000 7f1a22ffd700 1 -- 172.21.15.181:0/2497225613 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f1a2408b310 con 0x7f1a340cb110 2023-10-16T10:00:17.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.061+0000 7f1a31d9b700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 0x7f1a1406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:17.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.069+0000 7f1a22ffd700 1 -- 172.21.15.181:0/2497225613 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a3406f170 con 0x7f1a340cb110 2023-10-16T10:00:17.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:17.254+0000 7f2881e03200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:00:17.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.262+0000 7f1a31d9b700 1 -- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 msgr2=0x7f1a1406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:17.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.262+0000 7f1a31d9b700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 0x7f1a1406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:17.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.318+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1a3406f170 con 0x7f1a340cb110 2023-10-16T10:00:17.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.319+0000 7f1a22ffd700 1 -- 172.21.15.181:0/2497225613 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v91) v1 ==== 80+0+474 (secure 0 0 0) 0x7f1a3406f170 con 0x7f1a340cb110 2023-10-16T10:00:17.321 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1686162648 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.321 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/299388856 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.321 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/315145191 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.321 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2235576983 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.322 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3806937704 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.322 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/4212605490 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.322 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/752673838 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.322 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/752673838 2023-10-17T10:00:13.202216+0000 2023-10-16T10:00:17.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 msgr2=0x7f1a1406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:17.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 0x7f1a1406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a340cb110 msgr2=0x7f1a34151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a340cb110 0x7f1a34151ae0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009fd0 tx=0x7f1a24004750 comp rx=0 tx=0).stop 2023-10-16T10:00:17.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 shutdown_connections 2023-10-16T10:00:17.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f1a1406b020 0x7f1a1406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a3415e460 0x7f1a340c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a340cb110 0x7f1a34151ae0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 --2- 172.21.15.181:0/2497225613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a3415a0f0 0x7f1a340c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 >> 172.21.15.181:0/2497225613 conn(0x7f1a340d04a0 msgr2=0x7f1a340cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:17.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 shutdown_connections 2023-10-16T10:00:17.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.321+0000 7f1a389f9700 1 -- 172.21.15.181:0/2497225613 wait complete. 2023-10-16T10:00:17.326 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:17.323+0000 7f2881e03200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:00:17.326 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T10:00:17.337 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:00:17.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:17.338 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:17.365 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:17.364+0000 7ff3a6f4b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:00:17.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T10:00:17.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:17.385 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:00:17.439 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:17.437+0000 7ff3a6f4b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:00:17.440 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:00:17.441 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:17.458 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:00:17.458 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:00:17.459 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:17.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:00:17.498 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:00:17.499 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:17.516 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:00:17.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:17.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.660+0000 7f85a2097700 1 Processor -- start 2023-10-16T10:00:17.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.660+0000 7f85a2097700 1 -- start start 2023-10-16T10:00:17.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f85a2097700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f859c15b990 0x7f859c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f85a2097700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cd2d0 0x7f859c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f85a2097700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f859c0cfc60 0x7f859c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f85a2097700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c0d0630 con 0x7f859c15b990 2023-10-16T10:00:17.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f85a2097700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c0d24f0 con 0x7f859c0cfc60 2023-10-16T10:00:17.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f85a2097700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0cc510 con 0x7f859c0cd2d0 2023-10-16T10:00:17.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f859c15b990 0x7f859c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f859c15b990 0x7f859c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43792/0 (socket says 172.21.15.181:43792) 2023-10-16T10:00:17.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 -- 172.21.15.181:0/163958551 learned_addr learned my addr 172.21.15.181:0/163958551 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:17.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859affd700 1 --2- 172.21.15.181:0/163958551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cd2d0 0x7f859c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859bfff700 1 --2- 172.21.15.181:0/163958551 >> v2:172.21.15.190:3300/0 conn(0x7f859c0cfc60 0x7f859c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 -- 172.21.15.181:0/163958551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cd2d0 msgr2=0x7f859c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:17.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 --2- 172.21.15.181:0/163958551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cd2d0 0x7f859c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 -- 172.21.15.181:0/163958551 >> v2:172.21.15.190:3300/0 conn(0x7f859c0cfc60 msgr2=0x7f859c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:17.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 --2- 172.21.15.181:0/163958551 >> v2:172.21.15.190:3300/0 conn(0x7f859c0cfc60 0x7f859c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 -- 172.21.15.181:0/163958551 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c0cc7f0 con 0x7f859c15b990 2023-10-16T10:00:17.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859bfff700 1 --2- 172.21.15.181:0/163958551 >> v2:172.21.15.190:3300/0 conn(0x7f859c0cfc60 0x7f859c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:17.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.661+0000 7f859b7fe700 1 --2- 172.21.15.181:0/163958551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c0ce8f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f859c0cbc60 tx=0x7f858c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da149d2416624897 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f859a7fc700 1 -- 172.21.15.181:0/163958551 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c005e30 con 0x7f859c15b990 2023-10-16T10:00:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f859a7fc700 1 -- 172.21.15.181:0/163958551 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c004030 con 0x7f859c15b990 2023-10-16T10:00:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f859a7fc700 1 -- 172.21.15.181:0/163958551 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c0043e0 con 0x7f859c15b990 2023-10-16T10:00:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 -- 172.21.15.181:0/163958551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 msgr2=0x7f859c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 --2- 172.21.15.181:0/163958551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c0ce8f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f859c0cbc60 tx=0x7f858c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 -- 172.21.15.181:0/163958551 shutdown_connections 2023-10-16T10:00:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 --2- 172.21.15.181:0/163958551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c0ce8f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 --2- 172.21.15.181:0/163958551 >> v2:172.21.15.190:3300/0 conn(0x7f859c0cfc60 0x7f859c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 --2- 172.21.15.181:0/163958551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cd2d0 0x7f859c0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 -- 172.21.15.181:0/163958551 >> 172.21.15.181:0/163958551 conn(0x7f859c09a9a0 msgr2=0x7f859c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:17.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 -- 172.21.15.181:0/163958551 shutdown_connections 2023-10-16T10:00:17.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.662+0000 7f85a2097700 1 -- 172.21.15.181:0/163958551 wait complete. 2023-10-16T10:00:17.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.663+0000 7f85a2097700 1 Processor -- start 2023-10-16T10:00:17.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f85a2097700 1 -- start start 2023-10-16T10:00:17.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f85a2097700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f859c0cd2d0 0x7f859c1f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f85a2097700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cfc60 0x7f859c14f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f85a2097700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f85a2097700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c06c3c0 con 0x7f859c15b990 2023-10-16T10:00:17.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f85a2097700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c06c6a0 con 0x7f859c0cd2d0 2023-10-16T10:00:17.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f85a2097700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c06c980 con 0x7f859c0cfc60 2023-10-16T10:00:17.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.664+0000 7f859bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c14fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43802/0 (socket says 172.21.15.181:43802) 2023-10-16T10:00:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 -- 172.21.15.181:0/903888661 learned_addr learned my addr 172.21.15.181:0/903888661 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859b7fe700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f859c0cd2d0 0x7f859c1f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859affd700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cfc60 0x7f859c14f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 -- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cfc60 msgr2=0x7f859c14f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cfc60 0x7f859c14f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 -- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f859c0cd2d0 msgr2=0x7f859c1f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f859c0cd2d0 0x7f859c1f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 -- 172.21.15.181:0/903888661 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c009500 con 0x7f859c15b990 2023-10-16T10:00:17.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f859bfff700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c14fc10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8594007e70 tx=0x7f859400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f8583fff700 1 -- 172.21.15.181:0/903888661 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400dd20 con 0x7f859c15b990 2023-10-16T10:00:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f8583fff700 1 -- 172.21.15.181:0/903888661 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859400f040 con 0x7f859c15b990 2023-10-16T10:00:17.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c06cc60 con 0x7f859c15b990 2023-10-16T10:00:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f8583fff700 1 -- 172.21.15.181:0/903888661 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400dea0 con 0x7f859c15b990 2023-10-16T10:00:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.665+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c06d220 con 0x7f859c15b990 2023-10-16T10:00:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.666+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c15be90 con 0x7f859c15b990 2023-10-16T10:00:17.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.667+0000 7f8583fff700 1 -- 172.21.15.181:0/903888661 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8594003c60 con 0x7f859c15b990 2023-10-16T10:00:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.667+0000 7f8583fff700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 0x7f858806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.667+0000 7f859b7fe700 1 -- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 msgr2=0x7f858806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.667+0000 7f8583fff700 1 -- 172.21.15.181:0/903888661 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f8594089b40 con 0x7f859c15b990 2023-10-16T10:00:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.667+0000 7f859b7fe700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 0x7f858806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:17.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.671+0000 7f8583fff700 1 -- 172.21.15.181:0/903888661 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8594054630 con 0x7f859c15b990 2023-10-16T10:00:17.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.868+0000 7f859b7fe700 1 -- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 msgr2=0x7f858806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:17.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.868+0000 7f859b7fe700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 0x7f858806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:17.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.912+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f859c067d20 con 0x7f859c15b990 2023-10-16T10:00:17.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.912+0000 7f8583fff700 1 -- 172.21.15.181:0/903888661 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f859401a090 con 0x7f859c15b990 2023-10-16T10:00:17.915 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:17.915 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3576975018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3576975018},{"type":"v1","addr":"172.21.15.190:6834","nonce":3576975018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/54128065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":54128065},{"type":"v1","addr":"172.21.15.190:6835","nonce":54128065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/509615642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":509615642},{"type":"v1","addr":"172.21.15.181:6836","nonce":509615642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:59.043602+0000","modified":"2023-10-16T10:00:02.379610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2780127849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2780127849},{"type":"v1","addr":"172.21.15.181:6837","nonce":2780127849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:00:17.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.914+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 msgr2=0x7f858806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:17.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.914+0000 7f85a2097700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 0x7f858806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.914+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 msgr2=0x7f859c14fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c14fc10 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8594007e70 tx=0x7f859400d3b0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 shutdown_connections 2023-10-16T10:00:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f858806b020 0x7f858806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f859c15b990 0x7f859c14fc10 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f859c0cd2d0 0x7f859c1f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 --2- 172.21.15.181:0/903888661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f859c0cfc60 0x7f859c14f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 >> 172.21.15.181:0/903888661 conn(0x7f859c09a9a0 msgr2=0x7f859c0ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 shutdown_connections 2023-10-16T10:00:17.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:17.915+0000 7f85a2097700 1 -- 172.21.15.181:0/903888661 wait complete. 2023-10-16T10:00:17.920 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 40 2023-10-16T10:00:17.929 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:17.960 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:17.959+0000 7f2881e03200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:18.037 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:18.036+0000 7f2881e03200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:00:18.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.052+0000 7f1811b2f700 1 Processor -- start 2023-10-16T10:00:18.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.053+0000 7f1811b2f700 1 -- start start 2023-10-16T10:00:18.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.053+0000 7f1811b2f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f180c159730 0x7f180c15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.053+0000 7f1811b2f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c0cee70 con 0x7f180c0cc400 2023-10-16T10:00:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.053+0000 7f1811b2f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c159b10 con 0x7f180c0cd5c0 2023-10-16T10:00:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.053+0000 7f1811b2f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c0ce540 con 0x7f180c159730 2023-10-16T10:00:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.053+0000 7f180b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f180c159730 0x7f180c15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f180c0cc400 0x7f180c0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54888/0 (socket says 172.21.15.181:54888) 2023-10-16T10:00:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180affd700 1 -- 172.21.15.181:0/749709680 learned_addr learned my addr 172.21.15.181:0/749709680 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180b7fe700 1 -- 172.21.15.181:0/749709680 >> v1:172.21.15.190:6789/0 conn(0x7f180c0cd5c0 legacy=0x7f180c0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180b7fe700 1 -- 172.21.15.181:0/749709680 >> v1:172.21.15.181:6789/0 conn(0x7f180c0cc400 legacy=0x7f180c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180a7fc700 1 -- 172.21.15.181:0/749709680 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785670922 0 0) 0x7f180c0cee70 con 0x7f180c0cc400 2023-10-16T10:00:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180b7fe700 1 -- 172.21.15.181:0/749709680 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0ce820 con 0x7f180c159730 2023-10-16T10:00:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180b7fe700 1 --2- 172.21.15.181:0/749709680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15cd20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009820 tx=0x7f17fc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1929cccda9e2f903 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180a7fc700 1 -- 172.21.15.181:0/749709680 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc005e30 con 0x7f180c159730 2023-10-16T10:00:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180a7fc700 1 -- 172.21.15.181:0/749709680 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc006030 con 0x7f180c159730 2023-10-16T10:00:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.054+0000 7f180a7fc700 1 -- 172.21.15.181:0/749709680 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0063e0 con 0x7f180c159730 2023-10-16T10:00:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.055+0000 7f1811b2f700 1 -- 172.21.15.181:0/749709680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 msgr2=0x7f180c15cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.055+0000 7f1811b2f700 1 --2- 172.21.15.181:0/749709680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15cd20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009820 tx=0x7f17fc009b30 comp rx=0 tx=0).stop 2023-10-16T10:00:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.055+0000 7f1811b2f700 1 -- 172.21.15.181:0/749709680 shutdown_connections 2023-10-16T10:00:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.055+0000 7f1811b2f700 1 --2- 172.21.15.181:0/749709680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15cd20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.055+0000 7f1811b2f700 1 -- 172.21.15.181:0/749709680 >> 172.21.15.181:0/749709680 conn(0x7f180c0d04a0 msgr2=0x7f180c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.055+0000 7f1811b2f700 1 -- 172.21.15.181:0/749709680 shutdown_connections 2023-10-16T10:00:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.055+0000 7f1811b2f700 1 -- 172.21.15.181:0/749709680 wait complete. 2023-10-16T10:00:18.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.056+0000 7f1811b2f700 1 Processor -- start 2023-10-16T10:00:18.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.056+0000 7f1811b2f700 1 -- start start 2023-10-16T10:00:18.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f1811b2f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c0cc400 0x7f180c1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f1811b2f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0cd5c0 0x7f180c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f1811b2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f1811b2f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c156ee0 con 0x7f180c159730 2023-10-16T10:00:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f1811b2f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0c8650 con 0x7f180c0cd5c0 2023-10-16T10:00:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f1811b2f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0c8930 con 0x7f180c0cc400 2023-10-16T10:00:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43818/0 (socket says 172.21.15.181:43818) 2023-10-16T10:00:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 -- 172.21.15.181:0/2142064947 learned_addr learned my addr 172.21.15.181:0/2142064947 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180affd700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0cd5c0 0x7f180c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180b7fe700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c0cc400 0x7f180c1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 -- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c0cc400 msgr2=0x7f180c1569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c0cc400 0x7f180c1569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 -- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0cd5c0 msgr2=0x7f180c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0cd5c0 0x7f180c0ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.057+0000 7f180bfff700 1 -- 172.21.15.181:0/2142064947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc009430 con 0x7f180c159730 2023-10-16T10:00:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.058+0000 7f180bfff700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15bf80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f180400cbc0 tx=0x7f180400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.058+0000 7f17f3fff700 1 -- 172.21.15.181:0/2142064947 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804003c00 con 0x7f180c159730 2023-10-16T10:00:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.058+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c0c8c10 con 0x7f180c159730 2023-10-16T10:00:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.058+0000 7f17f3fff700 1 -- 172.21.15.181:0/2142064947 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1804003d60 con 0x7f180c159730 2023-10-16T10:00:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.058+0000 7f17f3fff700 1 -- 172.21.15.181:0/2142064947 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180400aae0 con 0x7f180c159730 2023-10-16T10:00:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.058+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c0c91d0 con 0x7f180c159730 2023-10-16T10:00:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.059+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c067d20 con 0x7f180c159730 2023-10-16T10:00:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.059+0000 7f17f3fff700 1 -- 172.21.15.181:0/2142064947 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f180400ac40 con 0x7f180c159730 2023-10-16T10:00:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.060+0000 7f17f3fff700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 0x7f17f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.060+0000 7f180b7fe700 1 -- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 msgr2=0x7f17f806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.060+0000 7f17f3fff700 1 -- 172.21.15.181:0/2142064947 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f180408a4f0 con 0x7f180c159730 2023-10-16T10:00:18.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.060+0000 7f180b7fe700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 0x7f17f806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:18.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.068+0000 7f17f3fff700 1 -- 172.21.15.181:0/2142064947 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1804055190 con 0x7f180c159730 2023-10-16T10:00:18.102 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:18.101+0000 7ff3a6f4b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:18.182 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:18.180+0000 7f2881e03200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:18.183 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:18.182+0000 7ff3a6f4b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:00:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.261+0000 7f180b7fe700 1 -- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 msgr2=0x7f17f806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.261+0000 7f180b7fe700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 0x7f17f806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:18.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.273+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f180c0c3920 con 0x7f180c159730 2023-10-16T10:00:18.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.274+0000 7f17f3fff700 1 -- 172.21.15.181:0/2142064947 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17753 (secure 0 0 0) 0x7f18040587b0 con 0x7f180c159730 2023-10-16T10:00:18.276 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:18.276 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":91,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:13.202252+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:56.829733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:58.008691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1686162648":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/299388856":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/315145191":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/2235576983":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/3806937704":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/4212605490":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6833/752673838":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6832/752673838":"2023-10-17T10:00:13.202216+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:18.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 msgr2=0x7f17f806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:18.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 0x7f17f806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 msgr2=0x7f180c15bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15bf80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f180400cbc0 tx=0x7f180400ced0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 shutdown_connections 2023-10-16T10:00:18.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c0cc400 0x7f180c1569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f17f806b020 0x7f17f806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c159730 0x7f180c15bf80 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 --2- 172.21.15.181:0/2142064947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0cd5c0 0x7f180c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 >> 172.21.15.181:0/2142064947 conn(0x7f180c0d04a0 msgr2=0x7f180c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:18.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 shutdown_connections 2023-10-16T10:00:18.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.276+0000 7f1811b2f700 1 -- 172.21.15.181:0/2142064947 wait complete. 2023-10-16T10:00:18.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T09:59:56.829733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T09:59:58.008691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:00:18.291 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:00:18.336 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:18.334+0000 7ff3a6f4b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:18.410 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:18.408+0000 7f2881e03200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:00:18.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.418+0000 7f993d692700 1 Processor -- start 2023-10-16T10:00:18.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.418+0000 7f993d692700 1 -- start start 2023-10-16T10:00:18.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f993d692700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99381646b0 0x7f9938166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f993d692700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380b1f70 con 0x7f993815f150 2023-10-16T10:00:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f993d692700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380b2ae0 con 0x7f99380ce6c0 2023-10-16T10:00:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f993d692700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99380cd880 con 0x7f99381646b0 2023-10-16T10:00:18.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f9936ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f993815f150 0x7f99380cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54922/0 (socket says 172.21.15.181:54922) 2023-10-16T10:00:18.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f9936ffd700 1 -- 172.21.15.181:0/563743817 learned_addr learned my addr 172.21.15.181:0/563743817 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:18.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f99377fe700 1 --2- 172.21.15.181:0/563743817 >> v2:172.21.15.181:3300/0 conn(0x7f99381646b0 0x7f9938166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f9935ffb700 1 -- 172.21.15.181:0/563743817 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149418724 0 0) 0x7f99380b1f70 con 0x7f993815f150 2023-10-16T10:00:18.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f9935ffb700 1 -- 172.21.15.181:0/563743817 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99200035b0 con 0x7f993815f150 2023-10-16T10:00:18.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.419+0000 7f99377fe700 1 -- 172.21.15.181:0/563743817 >> v1:172.21.15.190:6789/0 conn(0x7f99380ce6c0 legacy=0x7f99380cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f99377fe700 1 -- 172.21.15.181:0/563743817 >> v1:172.21.15.181:6789/0 conn(0x7f993815f150 legacy=0x7f99380cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f99377fe700 1 -- 172.21.15.181:0/563743817 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380b2dc0 con 0x7f99381646b0 2023-10-16T10:00:18.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f99377fe700 1 --2- 172.21.15.181:0/563743817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99381646b0 0x7f9938166aa0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9930009d90 tx=0x7f993000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76edef882290acd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:18.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f9935ffb700 1 -- 172.21.15.181:0/563743817 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000e070 con 0x7f99381646b0 2023-10-16T10:00:18.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f9935ffb700 1 -- 172.21.15.181:0/563743817 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993000bb20 con 0x7f99381646b0 2023-10-16T10:00:18.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f9935ffb700 1 -- 172.21.15.181:0/563743817 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000be90 con 0x7f99381646b0 2023-10-16T10:00:18.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f993d692700 1 -- 172.21.15.181:0/563743817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99381646b0 msgr2=0x7f9938166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f993d692700 1 --2- 172.21.15.181:0/563743817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99381646b0 0x7f9938166aa0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9930009d90 tx=0x7f993000b330 comp rx=0 tx=0).stop 2023-10-16T10:00:18.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f993d692700 1 -- 172.21.15.181:0/563743817 shutdown_connections 2023-10-16T10:00:18.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f993d692700 1 --2- 172.21.15.181:0/563743817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99381646b0 0x7f9938166aa0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f993d692700 1 -- 172.21.15.181:0/563743817 >> 172.21.15.181:0/563743817 conn(0x7f993809a930 msgr2=0x7f99380d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:18.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f993d692700 1 -- 172.21.15.181:0/563743817 shutdown_connections 2023-10-16T10:00:18.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.420+0000 7f993d692700 1 -- 172.21.15.181:0/563743817 wait complete. 2023-10-16T10:00:18.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.422+0000 7f993d692700 1 Processor -- start 2023-10-16T10:00:18.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.422+0000 7f993d692700 1 -- start start 2023-10-16T10:00:18.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.422+0000 7f993d692700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99380ce6c0 0x7f9938163f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.422+0000 7f993d692700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f993815f150 0x7f9938154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.422+0000 7f993d692700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99381646b0 0x7f9938141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.422+0000 7f993d692700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380c7fc0 con 0x7f99380ce6c0 2023-10-16T10:00:18.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f993d692700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381644c0 con 0x7f99381646b0 2023-10-16T10:00:18.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f993d692700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9938143730 con 0x7f993815f150 2023-10-16T10:00:18.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99380ce6c0 0x7f9938163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99380ce6c0 0x7f9938163f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43850/0 (socket says 172.21.15.181:43850) 2023-10-16T10:00:18.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 -- 172.21.15.181:0/2031757736 learned_addr learned my addr 172.21.15.181:0/2031757736 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:18.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f99377fe700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99381646b0 0x7f9938141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f99367fc700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f993815f150 0x7f9938154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 -- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f993815f150 msgr2=0x7f9938154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f993815f150 0x7f9938154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 -- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99381646b0 msgr2=0x7f9938141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99381646b0 0x7f9938141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 -- 172.21.15.181:0/2031757736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930009a70 con 0x7f99380ce6c0 2023-10-16T10:00:18.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f99367fc700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f993815f150 0x7f9938154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:18.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f9936ffd700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99380ce6c0 0x7f9938163f80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f992800c700 tx=0x7f992800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:18.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f991f7fe700 1 -- 172.21.15.181:0/2031757736 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928005d70 con 0x7f99380ce6c0 2023-10-16T10:00:18.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f991f7fe700 1 -- 172.21.15.181:0/2031757736 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99280070d0 con 0x7f99380ce6c0 2023-10-16T10:00:18.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9938141ce0 con 0x7f99380ce6c0 2023-10-16T10:00:18.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f991f7fe700 1 -- 172.21.15.181:0/2031757736 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928007440 con 0x7f99380ce6c0 2023-10-16T10:00:18.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.423+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99380c7720 con 0x7f99380ce6c0 2023-10-16T10:00:18.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.425+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9938067d20 con 0x7f99380ce6c0 2023-10-16T10:00:18.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.425+0000 7f991f7fe700 1 -- 172.21.15.181:0/2031757736 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f99280075a0 con 0x7f99380ce6c0 2023-10-16T10:00:18.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.425+0000 7f991f7fe700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 0x7f992406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.425+0000 7f99367fc700 1 -- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 msgr2=0x7f992406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:18.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.425+0000 7f991f7fe700 1 -- 172.21.15.181:0/2031757736 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f9928088e00 con 0x7f99380ce6c0 2023-10-16T10:00:18.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.425+0000 7f99367fc700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 0x7f992406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:18.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.430+0000 7f991f7fe700 1 -- 172.21.15.181:0/2031757736 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9928053b50 con 0x7f99380ce6c0 2023-10-16T10:00:18.484 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:18.483+0000 7f2881e03200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:00:18.576 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:18.574+0000 7ff3a6f4b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:00:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.626+0000 7f99367fc700 1 -- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 msgr2=0x7f992406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:18.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.626+0000 7f99367fc700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 0x7f992406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:18.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.629+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99380d0890 con 0x7f99380ce6c0 2023-10-16T10:00:18.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.630+0000 7f991f7fe700 1 -- 172.21.15.181:0/2031757736 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99280170a0 con 0x7f99380ce6c0 2023-10-16T10:00:18.632 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:00:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 msgr2=0x7f992406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:18.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 0x7f992406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99380ce6c0 msgr2=0x7f9938163f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99380ce6c0 0x7f9938163f80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f992800c700 tx=0x7f992800ca10 comp rx=0 tx=0).stop 2023-10-16T10:00:18.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 shutdown_connections 2023-10-16T10:00:18.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f992406b020 0x7f992406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99381646b0 0x7f9938141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99380ce6c0 0x7f9938163f80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 --2- 172.21.15.181:0/2031757736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f993815f150 0x7f9938154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 >> 172.21.15.181:0/2031757736 conn(0x7f993809a930 msgr2=0x7f9938158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:18.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.632+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 shutdown_connections 2023-10-16T10:00:18.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.633+0000 7f993d692700 1 -- 172.21.15.181:0/2031757736 wait complete. 2023-10-16T10:00:18.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:00:18.652 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:18.651+0000 7ff3a6f4b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:00:18.748 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:18.747+0000 7f2881e03200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:00:18.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.776+0000 7fd70e66f700 1 Processor -- start 2023-10-16T10:00:18.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.776+0000 7fd70e66f700 1 -- start start 2023-10-16T10:00:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.776+0000 7fd70e66f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd70806bef0 0x7fd7080caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.776+0000 7fd70e66f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70806e4b0 con 0x7fd7080cb0e0 2023-10-16T10:00:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.776+0000 7fd70e66f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70806ed50 con 0x7fd708159e10 2023-10-16T10:00:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.776+0000 7fd70e66f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7080cecb0 con 0x7fd70806bef0 2023-10-16T10:00:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd7077fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd70806bef0 0x7fd7080caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd70cc0c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd7080cb0e0 0x7fd70815db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54942/0 (socket says 172.21.15.181:54942) 2023-10-16T10:00:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd70cc0c700 1 -- 172.21.15.181:0/4248582042 learned_addr learned my addr 172.21.15.181:0/4248582042 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd7077fe700 1 -- 172.21.15.181:0/4248582042 >> v1:172.21.15.190:6789/0 conn(0x7fd708159e10 legacy=0x7fd7080ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd7077fe700 1 -- 172.21.15.181:0/4248582042 >> v1:172.21.15.181:6789/0 conn(0x7fd7080cb0e0 legacy=0x7fd70815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd706ffd700 1 -- 172.21.15.181:0/4248582042 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587308402 0 0) 0x7fd70806e4b0 con 0x7fd7080cb0e0 2023-10-16T10:00:18.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd7077fe700 1 -- 172.21.15.181:0/4248582042 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080ceea0 con 0x7fd70806bef0 2023-10-16T10:00:18.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd7077fe700 1 --2- 172.21.15.181:0/4248582042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd7080caba0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009840 tx=0x7fd6fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a6aa8415719ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:18.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd706ffd700 1 -- 172.21.15.181:0/4248582042 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc010800 con 0x7fd70806bef0 2023-10-16T10:00:18.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd706ffd700 1 -- 172.21.15.181:0/4248582042 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc010960 con 0x7fd70806bef0 2023-10-16T10:00:18.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.777+0000 7fd706ffd700 1 -- 172.21.15.181:0/4248582042 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc010cf0 con 0x7fd70806bef0 2023-10-16T10:00:18.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.778+0000 7fd70e66f700 1 -- 172.21.15.181:0/4248582042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 msgr2=0x7fd7080caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.778+0000 7fd70e66f700 1 --2- 172.21.15.181:0/4248582042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd7080caba0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009840 tx=0x7fd6fc009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:18.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.778+0000 7fd70e66f700 1 -- 172.21.15.181:0/4248582042 shutdown_connections 2023-10-16T10:00:18.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.778+0000 7fd70e66f700 1 --2- 172.21.15.181:0/4248582042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd7080caba0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.778+0000 7fd70e66f700 1 -- 172.21.15.181:0/4248582042 >> 172.21.15.181:0/4248582042 conn(0x7fd7080cc730 msgr2=0x7fd7080ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:18.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.778+0000 7fd70e66f700 1 -- 172.21.15.181:0/4248582042 shutdown_connections 2023-10-16T10:00:18.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.778+0000 7fd70e66f700 1 -- 172.21.15.181:0/4248582042 wait complete. 2023-10-16T10:00:18.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.779+0000 7fd70e66f700 1 Processor -- start 2023-10-16T10:00:18.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.779+0000 7fd70e66f700 1 -- start start 2023-10-16T10:00:18.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd70e66f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd708157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd70e66f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080cb0e0 0x7fd708152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd70e66f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd708159e10 0x7fd7080c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd70e66f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708152c70 con 0x7fd70806bef0 2023-10-16T10:00:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd70e66f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708153680 con 0x7fd7080cb0e0 2023-10-16T10:00:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd70e66f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd708156f20 con 0x7fd708159e10 2023-10-16T10:00:18.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd707fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd708157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd707fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd708157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43866/0 (socket says 172.21.15.181:43866) 2023-10-16T10:00:18.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd7077fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080cb0e0 0x7fd708152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd707fff700 1 -- 172.21.15.181:0/3543730973 learned_addr learned my addr 172.21.15.181:0/3543730973 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:18.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.780+0000 7fd70cc0c700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd708159e10 0x7fd7080c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:18.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd707fff700 1 -- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd708159e10 msgr2=0x7fd7080c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd707fff700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd708159e10 0x7fd7080c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd707fff700 1 -- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080cb0e0 msgr2=0x7fd708152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd707fff700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080cb0e0 0x7fd708152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd707fff700 1 -- 172.21.15.181:0/3543730973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0094a0 con 0x7fd70806bef0 2023-10-16T10:00:18.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd7077fe700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080cb0e0 0x7fd708152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:18.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd707fff700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd708157fc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c450 tx=0x7fd6f800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:18.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd704ff9700 1 -- 172.21.15.181:0/3543730973 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80052d0 con 0x7fd70806bef0 2023-10-16T10:00:18.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd704ff9700 1 -- 172.21.15.181:0/3543730973 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8005430 con 0x7fd70806bef0 2023-10-16T10:00:18.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd708157200 con 0x7fd70806bef0 2023-10-16T10:00:18.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd704ff9700 1 -- 172.21.15.181:0/3543730973 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80038c0 con 0x7fd70806bef0 2023-10-16T10:00:18.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.781+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7081577c0 con 0x7fd70806bef0 2023-10-16T10:00:18.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.782+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7080cd5d0 con 0x7fd70806bef0 2023-10-16T10:00:18.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.782+0000 7fd704ff9700 1 -- 172.21.15.181:0/3543730973 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd6f8003a20 con 0x7fd70806bef0 2023-10-16T10:00:18.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.783+0000 7fd704ff9700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 0x7fd6f406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:18.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.783+0000 7fd7077fe700 1 -- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 msgr2=0x7fd6f406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:18.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.783+0000 7fd704ff9700 1 -- 172.21.15.181:0/3543730973 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fd6f8088e50 con 0x7fd70806bef0 2023-10-16T10:00:18.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.783+0000 7fd7077fe700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 0x7fd6f406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:18.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.788+0000 7fd704ff9700 1 -- 172.21.15.181:0/3543730973 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6f8053a70 con 0x7fd70806bef0 2023-10-16T10:00:18.929 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:18.928+0000 7ff3a6f4b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:00:18.969 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:18.968+0000 7f2881e03200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:00:18.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.984+0000 7fd7077fe700 1 -- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 msgr2=0x7fd6f406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:18.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.984+0000 7fd7077fe700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 0x7fd6f406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:18.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.984+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd708153cf0 con 0x7fd70806bef0 2023-10-16T10:00:18.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.984+0000 7fd704ff9700 1 -- 172.21.15.181:0/3543730973 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6f800ce60 con 0x7fd70806bef0 2023-10-16T10:00:18.987 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:00:18.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 msgr2=0x7fd6f406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:18.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 0x7fd6f406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 msgr2=0x7fd708157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:18.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd708157fc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c450 tx=0x7fd6f800c760 comp rx=0 tx=0).stop 2023-10-16T10:00:18.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 shutdown_connections 2023-10-16T10:00:18.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd708159e10 0x7fd7080c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fd6f406b020 0x7fd6f406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd70806bef0 0x7fd708157fc0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 --2- 172.21.15.181:0/3543730973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080cb0e0 0x7fd708152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:18.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.986+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 >> 172.21.15.181:0/3543730973 conn(0x7fd7080cc730 msgr2=0x7fd70806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:18.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.987+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 shutdown_connections 2023-10-16T10:00:18.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:18.987+0000 7fd70e66f700 1 -- 172.21.15.181:0/3543730973 wait complete. 2023-10-16T10:00:19.000 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:00:19.119 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:19.118+0000 7f2881e03200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:00:19.164 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:19.163+0000 7ff3a6f4b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:00:19.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.166+0000 7f9c8f59e700 1 Processor -- start 2023-10-16T10:00:19.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.166+0000 7f9c8f59e700 1 -- start start 2023-10-16T10:00:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9c9015b770 0x7f9c900ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8f59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c900d0d10 0x7f9c9006b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8f59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9015bb50 con 0x7f9c9006e8c0 2023-10-16T10:00:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8f59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9006f710 con 0x7f9c900d0d10 2023-10-16T10:00:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8f59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c900ccb10 con 0x7f9c9015b770 2023-10-16T10:00:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c900d0d10 0x7f9c9006b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9c9006e8c0 0x7f9c900ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54956/0 (socket says 172.21.15.181:54956) 2023-10-16T10:00:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8ed9d700 1 -- 172.21.15.181:0/4200321732 learned_addr learned my addr 172.21.15.181:0/4200321732 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8e59c700 1 --2- 172.21.15.181:0/4200321732 >> v2:172.21.15.190:3300/0 conn(0x7f9c9015b770 0x7f9c900ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8dd9b700 1 -- 172.21.15.181:0/4200321732 >> v2:172.21.15.190:3300/0 conn(0x7f9c9015b770 msgr2=0x7f9c900ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8dd9b700 1 --2- 172.21.15.181:0/4200321732 >> v2:172.21.15.190:3300/0 conn(0x7f9c9015b770 0x7f9c900ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8dd9b700 1 -- 172.21.15.181:0/4200321732 >> v1:172.21.15.181:6789/0 conn(0x7f9c9006e8c0 legacy=0x7f9c900ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8dd9b700 1 -- 172.21.15.181:0/4200321732 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9006b7f0 con 0x7f9c900d0d10 2023-10-16T10:00:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8d59a700 1 -- 172.21.15.181:0/4200321732 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502986986 0 0) 0x7f9c9015bb50 con 0x7f9c9006e8c0 2023-10-16T10:00:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.167+0000 7f9c8dd9b700 1 --2- 172.21.15.181:0/4200321732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c900d0d10 0x7f9c9006b2b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c9006cae0 tx=0x7f9c84009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b137fdc4e122f42 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:19.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8d59a700 1 -- 172.21.15.181:0/4200321732 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c84011830 con 0x7f9c900d0d10 2023-10-16T10:00:19.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8d59a700 1 -- 172.21.15.181:0/4200321732 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c84011990 con 0x7f9c900d0d10 2023-10-16T10:00:19.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8d59a700 1 -- 172.21.15.181:0/4200321732 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c84011d20 con 0x7f9c900d0d10 2023-10-16T10:00:19.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 -- 172.21.15.181:0/4200321732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c900d0d10 msgr2=0x7f9c9006b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/4200321732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c900d0d10 0x7f9c9006b2b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c9006cae0 tx=0x7f9c84009b20 comp rx=0 tx=0).stop 2023-10-16T10:00:19.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 -- 172.21.15.181:0/4200321732 shutdown_connections 2023-10-16T10:00:19.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/4200321732 >> v2:172.21.15.190:3300/0 conn(0x7f9c9015b770 0x7f9c900ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/4200321732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c900d0d10 0x7f9c9006b2b0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 -- 172.21.15.181:0/4200321732 >> 172.21.15.181:0/4200321732 conn(0x7f9c9006c430 msgr2=0x7f9c900d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:19.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 -- 172.21.15.181:0/4200321732 shutdown_connections 2023-10-16T10:00:19.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.168+0000 7f9c8f59e700 1 -- 172.21.15.181:0/4200321732 wait complete. 2023-10-16T10:00:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 Processor -- start 2023-10-16T10:00:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 -- start start 2023-10-16T10:00:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9006e8c0 0x7f9c900b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c900d0d10 0x7f9c90154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c9015b770 0x7f9c901e1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c901e5a80 con 0x7f9c9015b770 2023-10-16T10:00:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c901e3ef0 con 0x7f9c9006e8c0 2023-10-16T10:00:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.170+0000 7f9c8f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c901e4f00 con 0x7f9c900d0d10 2023-10-16T10:00:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c9015b770 0x7f9c901e1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c9015b770 0x7f9c901e1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43886/0 (socket says 172.21.15.181:43886) 2023-10-16T10:00:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 -- 172.21.15.181:0/2928832787 learned_addr learned my addr 172.21.15.181:0/2928832787 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8e59c700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9006e8c0 0x7f9c900b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8dd9b700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c900d0d10 0x7f9c90154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 -- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c900d0d10 msgr2=0x7f9c90154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c900d0d10 0x7f9c90154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 -- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9006e8c0 msgr2=0x7f9c900b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9006e8c0 0x7f9c900b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 -- 172.21.15.181:0/2928832787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c840094a0 con 0x7f9c9015b770 2023-10-16T10:00:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8ed9d700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c9015b770 0x7f9c901e1920 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9c88002230 tx=0x7f9c8800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c7effd700 1 -- 172.21.15.181:0/2928832787 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800aac0 con 0x7f9c9015b770 2023-10-16T10:00:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c7effd700 1 -- 172.21.15.181:0/2928832787 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8800ac40 con 0x7f9c9015b770 2023-10-16T10:00:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c7effd700 1 -- 172.21.15.181:0/2928832787 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880070d0 con 0x7f9c9015b770 2023-10-16T10:00:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c901e51e0 con 0x7f9c9015b770 2023-10-16T10:00:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.171+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c901e57a0 con 0x7f9c9015b770 2023-10-16T10:00:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.172+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9004f400 con 0x7f9c9015b770 2023-10-16T10:00:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.173+0000 7f9c7effd700 1 -- 172.21.15.181:0/2928832787 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f9c88007250 con 0x7f9c9015b770 2023-10-16T10:00:19.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.173+0000 7f9c7effd700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 0x7f9c7006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.173+0000 7f9c8e59c700 1 -- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 msgr2=0x7f9c7006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:19.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.173+0000 7f9c7effd700 1 -- 172.21.15.181:0/2928832787 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f9c88089440 con 0x7f9c9015b770 2023-10-16T10:00:19.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.173+0000 7f9c8e59c700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 0x7f9c7006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:19.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.182+0000 7f9c7effd700 1 -- 172.21.15.181:0/2928832787 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9c880540f0 con 0x7f9c9015b770 2023-10-16T10:00:19.234 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:19.233+0000 7f2881e03200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:00:19.327 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:19.326+0000 7ff3a6f4b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:00:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.374+0000 7f9c8e59c700 1 -- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 msgr2=0x7f9c7006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.374+0000 7f9c8e59c700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 0x7f9c7006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:19.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.381+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c900ae1f0 con 0x7f9c9015b770 2023-10-16T10:00:19.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.381+0000 7f9c7effd700 1 -- 172.21.15.181:0/2928832787 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c88057710 con 0x7f9c9015b770 2023-10-16T10:00:19.383 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:00:19.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 msgr2=0x7f9c7006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:19.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 0x7f9c7006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c9015b770 msgr2=0x7f9c901e1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c9015b770 0x7f9c901e1920 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9c88002230 tx=0x7f9c8800cf50 comp rx=0 tx=0).stop 2023-10-16T10:00:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 shutdown_connections 2023-10-16T10:00:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f9c7006b020 0x7f9c7006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c9015b770 0x7f9c901e1920 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c9006e8c0 0x7f9c900b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 --2- 172.21.15.181:0/2928832787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c900d0d10 0x7f9c90154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 >> 172.21.15.181:0/2928832787 conn(0x7f9c9006c430 msgr2=0x7f9c900ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 shutdown_connections 2023-10-16T10:00:19.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.384+0000 7f9c8f59e700 1 -- 172.21.15.181:0/2928832787 wait complete. 2023-10-16T10:00:19.390 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:19.387+0000 7f2881e03200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:00:19.402 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:19.456 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:19.454+0000 7ff3a6f4b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:00:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.525+0000 7f4f6e3cd700 1 Processor -- start 2023-10-16T10:00:19.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.525+0000 7f4f6e3cd700 1 -- start start 2023-10-16T10:00:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f6e3cd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f680ce7f0 0x7f4f6806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f6e3cd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d2390 0x7f4f680cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f6e3cd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f680cc470 con 0x7f4f680d0180 2023-10-16T10:00:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f6e3cd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f680d0610 con 0x7f4f680ce7f0 2023-10-16T10:00:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f6e3cd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6806ca30 con 0x7f4f680d2390 2023-10-16T10:00:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f67fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f680ce7f0 0x7f4f6806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f67fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f680ce7f0 0x7f4f6806c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43892/0 (socket says 172.21.15.181:43892) 2023-10-16T10:00:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f67fff700 1 -- 172.21.15.181:0/3311116128 learned_addr learned my addr 172.21.15.181:0/3311116128 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.526+0000 7f4f677fe700 1 --2- 172.21.15.181:0/3311116128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d2390 0x7f4f680cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f67fff700 1 -- 172.21.15.181:0/3311116128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d2390 msgr2=0x7f4f680cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f67fff700 1 --2- 172.21.15.181:0/3311116128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d2390 0x7f4f680cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f67fff700 1 -- 172.21.15.181:0/3311116128 >> v1:172.21.15.190:6789/0 conn(0x7f4f680d0180 legacy=0x7f4f68161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f67fff700 1 -- 172.21.15.181:0/3311116128 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6806cd10 con 0x7f4f680ce7f0 2023-10-16T10:00:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f677fe700 1 --2- 172.21.15.181:0/3311116128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d2390 0x7f4f680cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f67fff700 1 --2- 172.21.15.181:0/3311116128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680ce7f0 0x7f4f6806c100 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4f58006fd0 tx=0x7f4f58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38fb595f17a0b54 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f66ffd700 1 -- 172.21.15.181:0/3311116128 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58005e30 con 0x7f4f680ce7f0 2023-10-16T10:00:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f66ffd700 1 -- 172.21.15.181:0/3311116128 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58004030 con 0x7f4f680ce7f0 2023-10-16T10:00:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f66ffd700 1 -- 172.21.15.181:0/3311116128 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f580043e0 con 0x7f4f680ce7f0 2023-10-16T10:00:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/3311116128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680ce7f0 msgr2=0x7f4f6806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.527+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/3311116128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680ce7f0 0x7f4f6806c100 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4f58006fd0 tx=0x7f4f58009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.528+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/3311116128 shutdown_connections 2023-10-16T10:00:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.528+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/3311116128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680ce7f0 0x7f4f6806c100 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.528+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/3311116128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d2390 0x7f4f680cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.528+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/3311116128 >> 172.21.15.181:0/3311116128 conn(0x7f4f6809a9a0 msgr2=0x7f4f680d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:19.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.528+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/3311116128 shutdown_connections 2023-10-16T10:00:19.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.528+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/3311116128 wait complete. 2023-10-16T10:00:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.529+0000 7f4f6e3cd700 1 Processor -- start 2023-10-16T10:00:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.529+0000 7f4f6e3cd700 1 -- start start 2023-10-16T10:00:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6e3cd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f680ce7f0 0x7f4f6809c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6e3cd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d0180 0x7f4f681f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6e3cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680d2390 0x7f4f680b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6e3cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6809d5a0 con 0x7f4f680d2390 2023-10-16T10:00:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6e3cd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f681f6e60 con 0x7f4f680ce7f0 2023-10-16T10:00:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6e3cd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f681f7db0 con 0x7f4f680d0180 2023-10-16T10:00:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680d2390 0x7f4f680b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680d2390 0x7f4f680b6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43900/0 (socket says 172.21.15.181:43900) 2023-10-16T10:00:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 -- 172.21.15.181:0/357679856 learned_addr learned my addr 172.21.15.181:0/357679856 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f67fff700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f680ce7f0 0x7f4f6809c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f677fe700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d0180 0x7f4f681f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 -- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d0180 msgr2=0x7f4f681f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d0180 0x7f4f681f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 -- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f680ce7f0 msgr2=0x7f4f6809c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f680ce7f0 0x7f4f6809c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 -- 172.21.15.181:0/357679856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f58009500 con 0x7f4f680d2390 2023-10-16T10:00:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6c96a700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680d2390 0x7f4f680b6600 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000cb50 tx=0x7f4f6000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f64ff9700 1 -- 172.21.15.181:0/357679856 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60003c00 con 0x7f4f680d2390 2023-10-16T10:00:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f64ff9700 1 -- 172.21.15.181:0/357679856 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f60003d60 con 0x7f4f680d2390 2023-10-16T10:00:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f64ff9700 1 -- 172.21.15.181:0/357679856 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000aa20 con 0x7f4f680d2390 2023-10-16T10:00:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.530+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6809cfe0 con 0x7f4f680d2390 2023-10-16T10:00:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.531+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f681f7070 con 0x7f4f680d2390 2023-10-16T10:00:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.531+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f680b6b40 con 0x7f4f680d2390 2023-10-16T10:00:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.533+0000 7f4f64ff9700 1 -- 172.21.15.181:0/357679856 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4f6000ab80 con 0x7f4f680d2390 2023-10-16T10:00:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.533+0000 7f4f64ff9700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 0x7f4f5406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.533+0000 7f4f67fff700 1 -- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 msgr2=0x7f4f5406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:19.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.533+0000 7f4f64ff9700 1 -- 172.21.15.181:0/357679856 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f4f6008a2d0 con 0x7f4f680d2390 2023-10-16T10:00:19.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.533+0000 7f4f67fff700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 0x7f4f5406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:19.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.537+0000 7f4f64ff9700 1 -- 172.21.15.181:0/357679856 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f60058ae0 con 0x7f4f680d2390 2023-10-16T10:00:19.622 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:19.620+0000 7ff3a6f4b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:00:19.651 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:19.650+0000 7f2881e03200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:19.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.734+0000 7f4f67fff700 1 -- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 msgr2=0x7f4f5406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:19.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.734+0000 7f4f67fff700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 0x7f4f5406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:19.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.783+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f680b6b40 con 0x7f4f680d2390 2023-10-16T10:00:19.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.783+0000 7f4f64ff9700 1 -- 172.21.15.181:0/357679856 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f4f680b6b40 con 0x7f4f680d2390 2023-10-16T10:00:19.785 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:19.786 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3576975018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3576975018},{"type":"v1","addr":"172.21.15.190:6834","nonce":3576975018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/54128065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":54128065},{"type":"v1","addr":"172.21.15.190:6835","nonce":54128065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/509615642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":509615642},{"type":"v1","addr":"172.21.15.181:6836","nonce":509615642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:59.043602+0000","modified":"2023-10-16T10:00:02.379610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2780127849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2780127849},{"type":"v1","addr":"172.21.15.181:6837","nonce":2780127849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:00:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 msgr2=0x7f4f5406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 0x7f4f5406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680d2390 msgr2=0x7f4f680b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680d2390 0x7f4f680b6600 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000cb50 tx=0x7f4f6000ce60 comp rx=0 tx=0).stop 2023-10-16T10:00:19.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 shutdown_connections 2023-10-16T10:00:19.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f4f5406b020 0x7f4f5406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f680d2390 0x7f4f680b6600 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f680ce7f0 0x7f4f6809c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 --2- 172.21.15.181:0/357679856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f680d0180 0x7f4f681f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 >> 172.21.15.181:0/357679856 conn(0x7f4f6809a9a0 msgr2=0x7f4f68070360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:19.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 shutdown_connections 2023-10-16T10:00:19.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.786+0000 7f4f6e3cd700 1 -- 172.21.15.181:0/357679856 wait complete. 2023-10-16T10:00:19.791 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 40 2023-10-16T10:00:19.800 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:19.809 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:19.808+0000 7f2881e03200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:00:19.901 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:19.899+0000 7ff3a6f4b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:19.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.922+0000 7f8e7048a700 1 Processor -- start 2023-10-16T10:00:19.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.922+0000 7f8e7048a700 1 -- start start 2023-10-16T10:00:19.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.923+0000 7f8e7048a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6815dc20 0x7f8e680d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.923+0000 7f8e7048a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8e680d1e60 0x7f8e680d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.923+0000 7f8e7048a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e680b2f00 con 0x7f8e680ce4b0 2023-10-16T10:00:19.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.923+0000 7f8e7048a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680b3b80 con 0x7f8e680d1e60 2023-10-16T10:00:19.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.923+0000 7f8e7048a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680d2820 con 0x7f8e6815dc20 2023-10-16T10:00:19.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6815dc20 0x7f8e680d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6da25700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8e680d1e60 0x7f8e680d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6815dc20 0x7f8e680d41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:40932/0 (socket says 172.21.15.181:40932) 2023-10-16T10:00:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 -- 172.21.15.181:0/3219173472 learned_addr learned my addr 172.21.15.181:0/3219173472 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6ea27700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8e680ce4b0 0x7f8e680ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:38930/0 (socket says 172.21.15.181:38930) 2023-10-16T10:00:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6da25700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8e680d1e60 0x7f8e680d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:48956/0 (socket says 172.21.15.181:48956) 2023-10-16T10:00:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 -- 172.21.15.181:0/3219173472 >> v2:172.21.15.190:3300/0 conn(0x7f8e680d1e60 msgr2=0x7f8e680d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 --2- 172.21.15.181:0/3219173472 >> v2:172.21.15.190:3300/0 conn(0x7f8e680d1e60 0x7f8e680d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 -- 172.21.15.181:0/3219173472 >> v1:172.21.15.190:6789/0 conn(0x7f8e680ce4b0 legacy=0x7f8e680ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 -- 172.21.15.181:0/3219173472 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e680b31e0 con 0x7f8e6815dc20 2023-10-16T10:00:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6d224700 1 -- 172.21.15.181:0/3219173472 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507744385 0 0) 0x7f8e680b2f00 con 0x7f8e680ce4b0 2023-10-16T10:00:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6da25700 1 --2- 172.21.15.181:0/3219173472 >> v2:172.21.15.190:3300/0 conn(0x7f8e680d1e60 0x7f8e680d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6e226700 1 --2- 172.21.15.181:0/3219173472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6815dc20 0x7f8e680d41c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8e64006fd0 tx=0x7f8e64009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f65c1852573b023 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:19.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6d224700 1 -- 172.21.15.181:0/3219173472 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64005e30 con 0x7f8e6815dc20 2023-10-16T10:00:19.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6d224700 1 -- 172.21.15.181:0/3219173472 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e64004030 con 0x7f8e6815dc20 2023-10-16T10:00:19.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.924+0000 7f8e6d224700 1 -- 172.21.15.181:0/3219173472 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e640043e0 con 0x7f8e6815dc20 2023-10-16T10:00:19.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 -- 172.21.15.181:0/3219173472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6815dc20 msgr2=0x7f8e680d41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 --2- 172.21.15.181:0/3219173472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6815dc20 0x7f8e680d41c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8e64006fd0 tx=0x7f8e64009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 -- 172.21.15.181:0/3219173472 shutdown_connections 2023-10-16T10:00:19.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 --2- 172.21.15.181:0/3219173472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6815dc20 0x7f8e680d41c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 --2- 172.21.15.181:0/3219173472 >> v2:172.21.15.190:3300/0 conn(0x7f8e680d1e60 0x7f8e680d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 -- 172.21.15.181:0/3219173472 >> 172.21.15.181:0/3219173472 conn(0x7f8e6808af90 msgr2=0x7f8e680d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:19.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 -- 172.21.15.181:0/3219173472 shutdown_connections 2023-10-16T10:00:19.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.925+0000 7f8e7048a700 1 -- 172.21.15.181:0/3219173472 wait complete. 2023-10-16T10:00:19.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.926+0000 7f8e7048a700 1 Processor -- start 2023-10-16T10:00:19.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.926+0000 7f8e7048a700 1 -- start start 2023-10-16T10:00:19.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e7048a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e680ce4b0 0x7f8e681e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e7048a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e680d1e60 0x7f8e6815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e7048a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6815dc20 0x7f8e680ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e7048a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68152120 con 0x7f8e680d1e60 2023-10-16T10:00:19.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e7048a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6814e590 con 0x7f8e6815dc20 2023-10-16T10:00:19.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e7048a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e681536f0 con 0x7f8e680ce4b0 2023-10-16T10:00:19.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e680d1e60 0x7f8e6815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e680d1e60 0x7f8e6815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43914/0 (socket says 172.21.15.181:43914) 2023-10-16T10:00:19.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 -- 172.21.15.181:0/382647116 learned_addr learned my addr 172.21.15.181:0/382647116 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:19.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6e226700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e680ce4b0 0x7f8e681e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6ea27700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6815dc20 0x7f8e680ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:19.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 -- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e680ce4b0 msgr2=0x7f8e681e6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e680ce4b0 0x7f8e681e6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 -- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6815dc20 msgr2=0x7f8e680ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:19.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6815dc20 0x7f8e680ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:19.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 -- 172.21.15.181:0/382647116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580094a0 con 0x7f8e680d1e60 2023-10-16T10:00:19.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e6da25700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e680d1e60 0x7f8e6815e000 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800ebe0 tx=0x7f8e5800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:19.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.927+0000 7f8e5effd700 1 -- 172.21.15.181:0/382647116 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800c850 con 0x7f8e680d1e60 2023-10-16T10:00:19.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.928+0000 7f8e5effd700 1 -- 172.21.15.181:0/382647116 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5800c9b0 con 0x7f8e680d1e60 2023-10-16T10:00:19.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.928+0000 7f8e5effd700 1 -- 172.21.15.181:0/382647116 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800cd40 con 0x7f8e680d1e60 2023-10-16T10:00:19.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.928+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e64009500 con 0x7f8e680d1e60 2023-10-16T10:00:19.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.928+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e680cb710 con 0x7f8e680d1e60 2023-10-16T10:00:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.929+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6806c040 con 0x7f8e680d1e60 2023-10-16T10:00:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.929+0000 7f8e5effd700 1 -- 172.21.15.181:0/382647116 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8e58013470 con 0x7f8e680d1e60 2023-10-16T10:00:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.930+0000 7f8e5effd700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 0x7f8e4c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.930+0000 7f8e6e226700 1 -- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 msgr2=0x7f8e4c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.930+0000 7f8e5effd700 1 -- 172.21.15.181:0/382647116 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f8e5808a250 con 0x7f8e680d1e60 2023-10-16T10:00:19.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.930+0000 7f8e6e226700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 0x7f8e4c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:19.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:19.938+0000 7f8e5effd700 1 -- 172.21.15.181:0/382647116 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8e58054fd0 con 0x7f8e680d1e60 2023-10-16T10:00:20.072 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.071+0000 7ff3a6f4b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:00:20.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.131+0000 7f8e6e226700 1 -- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 msgr2=0x7f8e4c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:20.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.131+0000 7f8e6e226700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 0x7f8e4c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:20.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.140+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e6806bd60 con 0x7f8e680d1e60 2023-10-16T10:00:20.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.141+0000 7f8e5effd700 1 -- 172.21.15.181:0/382647116 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17753 (secure 0 0 0) 0x7f8e5800cea0 con 0x7f8e680d1e60 2023-10-16T10:00:20.143 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:20.144 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":91,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:13.202252+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:56.829733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:58.008691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1686162648":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/299388856":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/315145191":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/2235576983":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/3806937704":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/4212605490":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6833/752673838":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6832/752673838":"2023-10-17T10:00:13.202216+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:20.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.143+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 msgr2=0x7f8e4c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:20.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.143+0000 7f8e7048a700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 0x7f8e4c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.143+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e680d1e60 msgr2=0x7f8e6815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.143+0000 7f8e7048a700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e680d1e60 0x7f8e6815e000 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800ebe0 tx=0x7f8e5800eef0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 shutdown_connections 2023-10-16T10:00:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e680ce4b0 0x7f8e681e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7f8e4c06b020 0x7f8e4c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6815dc20 0x7f8e680ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 --2- 172.21.15.181:0/382647116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e680d1e60 0x7f8e6815e000 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 >> 172.21.15.181:0/382647116 conn(0x7f8e6808af90 msgr2=0x7f8e680b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:20.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 shutdown_connections 2023-10-16T10:00:20.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.144+0000 7f8e7048a700 1 -- 172.21.15.181:0/382647116 wait complete. 2023-10-16T10:00:20.158 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:00:20.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:20.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.279+0000 7fbf8ee08700 1 Processor -- start 2023-10-16T10:00:20.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.279+0000 7fbf8ee08700 1 -- start start 2023-10-16T10:00:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.279+0000 7fbf8ee08700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815e1d0 0x7fbf8815e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.279+0000 7fbf8ee08700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf880cb3f0 con 0x7fbf8815ed90 2023-10-16T10:00:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.279+0000 7fbf8ee08700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf880caac0 con 0x7fbf8815a550 2023-10-16T10:00:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.279+0000 7fbf8ee08700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8806ecb0 con 0x7fbf8815e1d0 2023-10-16T10:00:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf87fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbf8815ed90 0x7fbf8806e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55002/0 (socket says 172.21.15.181:55002) 2023-10-16T10:00:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf87fff700 1 -- 172.21.15.181:0/4162051654 learned_addr learned my addr 172.21.15.181:0/4162051654 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf8d3a5700 1 --2- 172.21.15.181:0/4162051654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815e1d0 0x7fbf8815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584763641 0 0) 0x7fbf880cb3f0 con 0x7fbf8815ed90 2023-10-16T10:00:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf780035b0 con 0x7fbf8815ed90 2023-10-16T10:00:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116899081 0 0) 0x7fbf880caac0 con 0x7fbf8815a550 2023-10-16T10:00:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf880cb3f0 con 0x7fbf8815a550 2023-10-16T10:00:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 439550244 0 0) 0x7fbf780035b0 con 0x7fbf8815ed90 2023-10-16T10:00:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf8d3a5700 1 -- 172.21.15.181:0/4162051654 >> v1:172.21.15.190:6789/0 conn(0x7fbf8815a550 legacy=0x7fbf880cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf8d3a5700 1 -- 172.21.15.181:0/4162051654 >> v1:172.21.15.181:6789/0 conn(0x7fbf8815ed90 legacy=0x7fbf8806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf8d3a5700 1 -- 172.21.15.181:0/4162051654 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf880cada0 con 0x7fbf8815e1d0 2023-10-16T10:00:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf8d3a5700 1 --2- 172.21.15.181:0/4162051654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815e1d0 0x7fbf8815e660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009dc0 tx=0x7fbf8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931f771b210da142 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000e070 con 0x7fbf8815e1d0 2023-10-16T10:00:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80003a20 con 0x7fbf8815e1d0 2023-10-16T10:00:20.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.280+0000 7fbf877fe700 1 -- 172.21.15.181:0/4162051654 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74001670 con 0x7fbf8815e1d0 2023-10-16T10:00:20.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.281+0000 7fbf8ee08700 1 -- 172.21.15.181:0/4162051654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815e1d0 msgr2=0x7fbf8815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.281+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/4162051654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815e1d0 0x7fbf8815e660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009dc0 tx=0x7fbf8000b330 comp rx=0 tx=0).stop 2023-10-16T10:00:20.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.281+0000 7fbf8ee08700 1 -- 172.21.15.181:0/4162051654 shutdown_connections 2023-10-16T10:00:20.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.281+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/4162051654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815e1d0 0x7fbf8815e660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.281+0000 7fbf8ee08700 1 -- 172.21.15.181:0/4162051654 >> 172.21.15.181:0/4162051654 conn(0x7fbf880d0460 msgr2=0x7fbf880d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:20.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.281+0000 7fbf8ee08700 1 -- 172.21.15.181:0/4162051654 shutdown_connections 2023-10-16T10:00:20.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.281+0000 7fbf8ee08700 1 -- 172.21.15.181:0/4162051654 wait complete. 2023-10-16T10:00:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.282+0000 7fbf8ee08700 1 Processor -- start 2023-10-16T10:00:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.282+0000 7fbf8ee08700 1 -- start start 2023-10-16T10:00:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8ee08700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815a550 0x7fbf881f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8ee08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf8815e1d0 0x7fbf880c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8ee08700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf8815ed90 0x7fbf880c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8ee08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881f1050 con 0x7fbf8815e1d0 2023-10-16T10:00:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8ee08700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8806ac80 con 0x7fbf8815ed90 2023-10-16T10:00:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8ee08700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8806af60 con 0x7fbf8815a550 2023-10-16T10:00:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf87fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf8815e1d0 0x7fbf880c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8cba4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815a550 0x7fbf881f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf87fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf8815e1d0 0x7fbf880c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43930/0 (socket says 172.21.15.181:43930) 2023-10-16T10:00:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf87fff700 1 -- 172.21.15.181:0/1602505964 learned_addr learned my addr 172.21.15.181:0/1602505964 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.283+0000 7fbf8d3a5700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf8815ed90 0x7fbf880c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf87fff700 1 -- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815a550 msgr2=0x7fbf881f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf87fff700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815a550 0x7fbf881f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf87fff700 1 -- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf8815ed90 msgr2=0x7fbf880c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf87fff700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf8815ed90 0x7fbf880c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf87fff700 1 -- 172.21.15.181:0/1602505964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf80009a70 con 0x7fbf8815e1d0 2023-10-16T10:00:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf8cba4700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815a550 0x7fbf881f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf87fff700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf8815e1d0 0x7fbf880c8e00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fbf740011e0 tx=0x7fbf7400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:20.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf857fa700 1 -- 172.21.15.181:0/1602505964 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74004c80 con 0x7fbf8815e1d0 2023-10-16T10:00:20.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf857fa700 1 -- 172.21.15.181:0/1602505964 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf74004de0 con 0x7fbf8815e1d0 2023-10-16T10:00:20.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8806b240 con 0x7fbf8815e1d0 2023-10-16T10:00:20.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf857fa700 1 -- 172.21.15.181:0/1602505964 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74007310 con 0x7fbf8815e1d0 2023-10-16T10:00:20.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.284+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf881f27a0 con 0x7fbf8815e1d0 2023-10-16T10:00:20.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.285+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf88067d20 con 0x7fbf8815e1d0 2023-10-16T10:00:20.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.285+0000 7fbf857fa700 1 -- 172.21.15.181:0/1602505964 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 48) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fbf74007470 con 0x7fbf8815e1d0 2023-10-16T10:00:20.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.286+0000 7fbf857fa700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 0x7fbf6c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.286+0000 7fbf8cba4700 1 -- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 msgr2=0x7fbf6c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:20.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.286+0000 7fbf857fa700 1 -- 172.21.15.181:0/1602505964 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fbf74089710 con 0x7fbf8815e1d0 2023-10-16T10:00:20.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.286+0000 7fbf8cba4700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 0x7fbf6c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:20.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.290+0000 7fbf857fa700 1 -- 172.21.15.181:0/1602505964 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbf740543b0 con 0x7fbf8815e1d0 2023-10-16T10:00:20.331 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:20.330+0000 7f2881e03200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:00:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.487+0000 7fbf8cba4700 1 -- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 msgr2=0x7fbf6c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2464567114 2023-10-16T10:00:20.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.487+0000 7fbf8cba4700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 0x7fbf6c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:20.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.545+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf880727a0 con 0x7fbf8815e1d0 2023-10-16T10:00:20.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.546+0000 7fbf857fa700 1 -- 172.21.15.181:0/1602505964 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fbf740579d0 con 0x7fbf8815e1d0 2023-10-16T10:00:20.547 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:20.548 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3576975018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3576975018},{"type":"v1","addr":"172.21.15.190:6834","nonce":3576975018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/54128065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":54128065},{"type":"v1","addr":"172.21.15.190:6835","nonce":54128065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/509615642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":509615642},{"type":"v1","addr":"172.21.15.181:6836","nonce":509615642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:59.043602+0000","modified":"2023-10-16T10:00:02.379610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2780127849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2780127849},{"type":"v1","addr":"172.21.15.181:6837","nonce":2780127849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:00:20.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 msgr2=0x7fbf6c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:20.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 0x7fbf6c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf8815e1d0 msgr2=0x7fbf880c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf8815e1d0 0x7fbf880c8e00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fbf740011e0 tx=0x7fbf7400ca30 comp rx=0 tx=0).stop 2023-10-16T10:00:20.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 shutdown_connections 2023-10-16T10:00:20.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf8815a550 0x7fbf881f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:6832/2464567114,v1:172.21.15.181:6833/2464567114] conn(0x7fbf6c06b020 0x7fbf6c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf8815ed90 0x7fbf880c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 --2- 172.21.15.181:0/1602505964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf8815e1d0 0x7fbf880c8e00 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 >> 172.21.15.181:0/1602505964 conn(0x7fbf880d0460 msgr2=0x7fbf8806f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:20.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 shutdown_connections 2023-10-16T10:00:20.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.548+0000 7fbf8ee08700 1 -- 172.21.15.181:0/1602505964 wait complete. 2023-10-16T10:00:20.553 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 40 2023-10-16T10:00:20.562 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:20.637 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.635+0000 7ff3a6f4b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:00:20.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.690+0000 7f45196e4700 1 Processor -- start 2023-10-16T10:00:20.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.690+0000 7f45196e4700 1 -- start start 2023-10-16T10:00:20.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45196e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45140d0370 0x7f45140ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45196e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f45140cd2d0 0x7f45140d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45196e4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45140cc510 con 0x7f451415b990 2023-10-16T10:00:20.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45196e4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140d24f0 con 0x7f45140d0370 2023-10-16T10:00:20.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45196e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140cc7f0 con 0x7f45140cd2d0 2023-10-16T10:00:20.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45140d0370 0x7f45140ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45140d0370 0x7f45140ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43960/0 (socket says 172.21.15.181:43960) 2023-10-16T10:00:20.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 -- 172.21.15.181:0/1986983073 learned_addr learned my addr 172.21.15.181:0/1986983073 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:20.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45137fe700 1 --2- 172.21.15.181:0/1986983073 >> v2:172.21.15.190:3300/0 conn(0x7f45140cd2d0 0x7f45140d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 -- 172.21.15.181:0/1986983073 >> v2:172.21.15.190:3300/0 conn(0x7f45140cd2d0 msgr2=0x7f45140d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:20.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 --2- 172.21.15.181:0/1986983073 >> v2:172.21.15.190:3300/0 conn(0x7f45140cd2d0 0x7f45140d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 -- 172.21.15.181:0/1986983073 >> v1:172.21.15.190:6789/0 conn(0x7f451415b990 legacy=0x7f45140cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:20.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 -- 172.21.15.181:0/1986983073 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140ccad0 con 0x7f45140d0370 2023-10-16T10:00:20.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f45127fc700 1 --2- 172.21.15.181:0/1986983073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140ce8f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f45140cbc60 tx=0x7f4508009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbd95cf487843a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:20.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f4511ffb700 1 -- 172.21.15.181:0/1986983073 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508004230 con 0x7f45140d0370 2023-10-16T10:00:20.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f4511ffb700 1 -- 172.21.15.181:0/1986983073 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4508004390 con 0x7f45140d0370 2023-10-16T10:00:20.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.691+0000 7f4511ffb700 1 -- 172.21.15.181:0/1986983073 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080036a0 con 0x7f45140d0370 2023-10-16T10:00:20.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 -- 172.21.15.181:0/1986983073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 msgr2=0x7f45140ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 --2- 172.21.15.181:0/1986983073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140ce8f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f45140cbc60 tx=0x7f4508009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 -- 172.21.15.181:0/1986983073 shutdown_connections 2023-10-16T10:00:20.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 --2- 172.21.15.181:0/1986983073 >> v2:172.21.15.190:3300/0 conn(0x7f45140cd2d0 0x7f45140d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 --2- 172.21.15.181:0/1986983073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140ce8f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 -- 172.21.15.181:0/1986983073 >> 172.21.15.181:0/1986983073 conn(0x7f451409a9a0 msgr2=0x7f45140d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:20.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 -- 172.21.15.181:0/1986983073 shutdown_connections 2023-10-16T10:00:20.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.692+0000 7f45196e4700 1 -- 172.21.15.181:0/1986983073 wait complete. 2023-10-16T10:00:20.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.693+0000 7f45196e4700 1 Processor -- start 2023-10-16T10:00:20.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45196e4700 1 -- start start 2023-10-16T10:00:20.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45196e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140cd2d0 0x7f45141f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45196e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45196e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451415b990 0x7f45140b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:20.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45196e4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45141f5690 con 0x7f45140d0370 2023-10-16T10:00:20.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45196e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45141482f0 con 0x7f451415b990 2023-10-16T10:00:20.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45196e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45140c8c90 con 0x7f45140cd2d0 2023-10-16T10:00:20.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45127fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:43966/0 (socket says 172.21.15.181:43966) 2023-10-16T10:00:20.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45127fc700 1 -- 172.21.15.181:0/1056063132 learned_addr learned my addr 172.21.15.181:0/1056063132 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:20.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f4512ffd700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140cd2d0 0x7f45141f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.694+0000 7f45137fe700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451415b990 0x7f45140b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:20.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45127fc700 1 -- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140cd2d0 msgr2=0x7f45141f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45127fc700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140cd2d0 0x7f45141f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45127fc700 1 -- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451415b990 msgr2=0x7f45140b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45127fc700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451415b990 0x7f45140b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45127fc700 1 -- 172.21.15.181:0/1056063132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4508009550 con 0x7f45140d0370 2023-10-16T10:00:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45127fc700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140c8750 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f451408bdb0 tx=0x7f4508004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f44fb7fe700 1 -- 172.21.15.181:0/1056063132 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508003d60 con 0x7f45140d0370 2023-10-16T10:00:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f44fb7fe700 1 -- 172.21.15.181:0/1056063132 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4508011630 con 0x7f45140d0370 2023-10-16T10:00:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f44fb7fe700 1 -- 172.21.15.181:0/1056063132 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080119c0 con 0x7f45140d0370 2023-10-16T10:00:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45141476e0 con 0x7f45140d0370 2023-10-16T10:00:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.695+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4514147ca0 con 0x7f45140d0370 2023-10-16T10:00:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.696+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4514067d20 con 0x7f45140d0370 2023-10-16T10:00:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.697+0000 7f44fb7fe700 1 -- 172.21.15.181:0/1056063132 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 49) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4508011b20 con 0x7f45140d0370 2023-10-16T10:00:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.698+0000 7f44fb7fe700 1 -- 172.21.15.181:0/1056063132 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f450808a9e0 con 0x7f45140d0370 2023-10-16T10:00:20.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.703+0000 7f44fb7fe700 1 -- 172.21.15.181:0/1056063132 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4508055990 con 0x7f45140d0370 2023-10-16T10:00:20.726 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff31f731700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.726 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff31f731700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.727 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff31f731700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.727 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff31f731700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.727 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff31f731700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.727 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff322777700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.727 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff322777700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.728 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff322777700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.728 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff322777700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.728 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:20.724+0000 7ff322777700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:20.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.945+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45140cfa40 con 0x7f45140d0370 2023-10-16T10:00:20.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.945+0000 7f44fb7fe700 1 -- 172.21.15.181:0/1056063132 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f4508058fb0 con 0x7f45140d0370 2023-10-16T10:00:20.948 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:20.948 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3576975018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3576975018},{"type":"v1","addr":"172.21.15.190:6834","nonce":3576975018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/54128065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":54128065},{"type":"v1","addr":"172.21.15.190:6835","nonce":54128065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/509615642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":509615642},{"type":"v1","addr":"172.21.15.181:6836","nonce":509615642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T09:59:59.043602+0000","modified":"2023-10-16T10:00:02.379610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2780127849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2780127849},{"type":"v1","addr":"172.21.15.181:6837","nonce":2780127849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-16T10:00:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 msgr2=0x7f45140c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140c8750 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f451408bdb0 tx=0x7f4508004750 comp rx=0 tx=0).stop 2023-10-16T10:00:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 shutdown_connections 2023-10-16T10:00:20.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45140cd2d0 0x7f45141f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f451415b990 0x7f45140b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 --2- 172.21.15.181:0/1056063132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45140d0370 0x7f45140c8750 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 >> 172.21.15.181:0/1056063132 conn(0x7f451409a9a0 msgr2=0x7f45140d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 shutdown_connections 2023-10-16T10:00:20.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:20.948+0000 7f45196e4700 1 -- 172.21.15.181:0/1056063132 wait complete. 2023-10-16T10:00:20.952 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 40 2023-10-16T10:00:20.962 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:21.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.080+0000 7fa169dd8700 1 Processor -- start 2023-10-16T10:00:21.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.080+0000 7fa169dd8700 1 -- start start 2023-10-16T10:00:21.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa169dd8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415dfd0 0x7fa1640af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa169dd8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1640cd470 0x7fa1640cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa169dd8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16406f870 con 0x7fa1640af450 2023-10-16T10:00:21.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa169dd8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1640cde40 con 0x7fa1640cd470 2023-10-16T10:00:21.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa169dd8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640d1160 con 0x7fa16415dfd0 2023-10-16T10:00:21.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa162ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa1640af450 0x7fa16406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55100/0 (socket says 172.21.15.181:55100) 2023-10-16T10:00:21.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa162ffd700 1 -- 172.21.15.181:0/2575335566 learned_addr learned my addr 172.21.15.181:0/2575335566 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:21.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa163fff700 1 --2- 172.21.15.181:0/2575335566 >> v2:172.21.15.190:3300/0 conn(0x7fa1640cd470 0x7fa1640cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa1637fe700 1 --2- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415dfd0 0x7fa1640af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa1627fc700 1 -- 172.21.15.181:0/2575335566 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118890473 0 0) 0x7fa16406f870 con 0x7fa1640af450 2023-10-16T10:00:21.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa1627fc700 1 -- 172.21.15.181:0/2575335566 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c0035b0 con 0x7fa1640af450 2023-10-16T10:00:21.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa163fff700 1 -- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415dfd0 msgr2=0x7fa1640af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:21.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa163fff700 1 --2- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415dfd0 0x7fa1640af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa163fff700 1 -- 172.21.15.181:0/2575335566 >> v1:172.21.15.181:6789/0 conn(0x7fa1640af450 legacy=0x7fa16406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa1627fc700 1 -- 172.21.15.181:0/2575335566 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4254443970 0 0) 0x7fa14c0035b0 con 0x7fa1640af450 2023-10-16T10:00:21.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.081+0000 7fa163fff700 1 -- 172.21.15.181:0/2575335566 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640cc8e0 con 0x7fa1640cd470 2023-10-16T10:00:21.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa163fff700 1 --2- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 0x7fa1640cd900 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009d90 tx=0x7fa15c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f493b17fc6899c03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa1627fc700 1 -- 172.21.15.181:0/2575335566 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00e070 con 0x7fa1640cd470 2023-10-16T10:00:21.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa1627fc700 1 -- 172.21.15.181:0/2575335566 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c003a20 con 0x7fa1640cd470 2023-10-16T10:00:21.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa1627fc700 1 -- 172.21.15.181:0/2575335566 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154001670 con 0x7fa1640cd470 2023-10-16T10:00:21.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 -- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 msgr2=0x7fa1640cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 --2- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 0x7fa1640cd900 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009d90 tx=0x7fa15c00b330 comp rx=0 tx=0).stop 2023-10-16T10:00:21.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 -- 172.21.15.181:0/2575335566 shutdown_connections 2023-10-16T10:00:21.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 --2- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 0x7fa1640cd900 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 --2- 172.21.15.181:0/2575335566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415dfd0 0x7fa1640af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 -- 172.21.15.181:0/2575335566 >> 172.21.15.181:0/2575335566 conn(0x7fa1640d04a0 msgr2=0x7fa1640d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:21.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 -- 172.21.15.181:0/2575335566 shutdown_connections 2023-10-16T10:00:21.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.082+0000 7fa169dd8700 1 -- 172.21.15.181:0/2575335566 wait complete. 2023-10-16T10:00:21.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.084+0000 7fa169dd8700 1 Processor -- start 2023-10-16T10:00:21.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.084+0000 7fa169dd8700 1 -- start start 2023-10-16T10:00:21.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.084+0000 7fa169dd8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1640af450 0x7fa1640aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa169dd8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 0x7fa16415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa169dd8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa16415dfd0 0x7fa164155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa169dd8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415d7d0 con 0x7fa16415dfd0 2023-10-16T10:00:21.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa169dd8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640b04e0 con 0x7fa1640cd470 2023-10-16T10:00:21.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa169dd8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640b07c0 con 0x7fa1640af450 2023-10-16T10:00:21.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa16415dfd0 0x7fa164155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa16415dfd0 0x7fa164155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44030/0 (socket says 172.21.15.181:44030) 2023-10-16T10:00:21.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 -- 172.21.15.181:0/1805909834 learned_addr learned my addr 172.21.15.181:0/1805909834 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:21.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa1637fe700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1640af450 0x7fa1640aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa162ffd700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 0x7fa16415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 -- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1640af450 msgr2=0x7fa1640aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1640af450 0x7fa1640aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 -- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 msgr2=0x7fa16415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 0x7fa16415d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 -- 172.21.15.181:0/1805909834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c009a70 con 0x7fa16415dfd0 2023-10-16T10:00:21.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa163fff700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa16415dfd0 0x7fa164155310 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa1640716a0 tx=0x7fa15c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa14bfff700 1 -- 172.21.15.181:0/1805909834 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004540 con 0x7fa16415dfd0 2023-10-16T10:00:21.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa14bfff700 1 -- 172.21.15.181:0/1805909834 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c0046a0 con 0x7fa16415dfd0 2023-10-16T10:00:21.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1640b0aa0 con 0x7fa16415dfd0 2023-10-16T10:00:21.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa14bfff700 1 -- 172.21.15.181:0/1805909834 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c012430 con 0x7fa16415dfd0 2023-10-16T10:00:21.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.085+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa164155850 con 0x7fa16415dfd0 2023-10-16T10:00:21.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.086+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1640b4820 con 0x7fa16415dfd0 2023-10-16T10:00:21.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.087+0000 7fa14bfff700 1 -- 172.21.15.181:0/1805909834 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 49) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa15c012590 con 0x7fa16415dfd0 2023-10-16T10:00:21.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.087+0000 7fa14bfff700 1 -- 172.21.15.181:0/1805909834 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fa15c08b500 con 0x7fa16415dfd0 2023-10-16T10:00:21.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.091+0000 7fa14bfff700 1 -- 172.21.15.181:0/1805909834 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa1640b4820 con 0x7fa16415dfd0 2023-10-16T10:00:21.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.289+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1640b4820 con 0x7fa16415dfd0 2023-10-16T10:00:21.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.290+0000 7fa14bfff700 1 -- 172.21.15.181:0/1805909834 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+18260 (secure 0 0 0) 0x7fa1640b4820 con 0x7fa16415dfd0 2023-10-16T10:00:21.292 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:21.292 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":92,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:20.650500+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-16T09:59:56.829733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-16T09:59:58.008691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/591701176":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/3458983616":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/395530113":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/3470208380":"2023-10-17T10:00:20.650463+0000","172.21.15.181:6833/2464567114":"2023-10-17T10:00:20.650463+0000","172.21.15.181:6832/2464567114":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/1686162648":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/299388856":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/315145191":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/2235576983":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/3806937704":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/849326393":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/4212605490":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/3680374730":"2023-10-17T10:00:20.650463+0000","172.21.15.181:6833/752673838":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6832/752673838":"2023-10-17T10:00:13.202216+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:21.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa16415dfd0 msgr2=0x7fa164155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa16415dfd0 0x7fa164155310 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa1640716a0 tx=0x7fa15c004ad0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 shutdown_connections 2023-10-16T10:00:21.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1640af450 0x7fa1640aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa16415dfd0 0x7fa164155310 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 --2- 172.21.15.181:0/1805909834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1640cd470 0x7fa16415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 >> 172.21.15.181:0/1805909834 conn(0x7fa1640d04a0 msgr2=0x7fa16406d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:21.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 shutdown_connections 2023-10-16T10:00:21.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.292+0000 7fa169dd8700 1 -- 172.21.15.181:0/1805909834 wait complete. 2023-10-16T10:00:21.306 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:00:21.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.425+0000 7fe5bf046700 1 Processor -- start 2023-10-16T10:00:21.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.425+0000 7fe5bf046700 1 -- start start 2023-10-16T10:00:21.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.426+0000 7fe5bf046700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 0x7fe5b80cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.426+0000 7fe5bf046700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b806b370 con 0x7fe5b815b9f0 2023-10-16T10:00:21.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.426+0000 7fe5bf046700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b80cddd0 con 0x7fe5b80cf980 2023-10-16T10:00:21.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.426+0000 7fe5bf046700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b806b800 con 0x7fe5b80cd950 2023-10-16T10:00:21.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.426+0000 7fe5bcde2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe5b815b9f0 0x7fe5b80cc400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55110/0 (socket says 172.21.15.181:55110) 2023-10-16T10:00:21.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.426+0000 7fe5bcde2700 1 -- 172.21.15.181:0/1549246298 learned_addr learned my addr 172.21.15.181:0/1549246298 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:21.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020531664 0 0) 0x7fe5b806b370 con 0x7fe5b815b9f0 2023-10-16T10:00:21.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a80035b0 con 0x7fe5b815b9f0 2023-10-16T10:00:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b7fff700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 0x7fe5b80cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2437376511 0 0) 0x7fe5a80035b0 con 0x7fe5b815b9f0 2023-10-16T10:00:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 msgr2=0x7fe5b80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 0x7fe5b80cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 >> v1:172.21.15.190:6789/0 conn(0x7fe5b80cf980 legacy=0x7fe5b80ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b80cfdc0 con 0x7fe5b815b9f0 2023-10-16T10:00:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe5ac0017f0 con 0x7fe5b815b9f0 2023-10-16T10:00:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 >> v1:172.21.15.181:6789/0 conn(0x7fe5b815b9f0 legacy=0x7fe5b80cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 0x7fe5a8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a8004060 0x7fe5a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a8004d90 0x7fe5a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b806b800 con 0x7fe5a8004d90 2023-10-16T10:00:21.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a800bbd0 con 0x7fe5a8004060 2023-10-16T10:00:21.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a800bd90 con 0x7fe5b80cd950 2023-10-16T10:00:21.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bd5e3700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a8004d90 0x7fe5a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bcde2700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 0x7fe5a8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b7fff700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a8004060 0x7fe5a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bd5e3700 1 -- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 msgr2=0x7fe5a8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:21.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bd5e3700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5b80cd950 0x7fe5a8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bd5e3700 1 -- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a8004060 msgr2=0x7fe5a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:21.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bd5e3700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a8004060 0x7fe5a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bd5e3700 1 -- 172.21.15.181:0/1549246298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a80035b0 con 0x7fe5a8004d90 2023-10-16T10:00:21.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5b7fff700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a8004060 0x7fe5a8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:21.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.427+0000 7fe5bd5e3700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a8004d90 0x7fe5a8005210 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000c190 tx=0x7fe5a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b068dd4aaa7016 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0001db0 con 0x7fe5a8004d90 2023-10-16T10:00:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b000ccd0 con 0x7fe5a8004d90 2023-10-16T10:00:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5b77fe700 1 -- 172.21.15.181:0/1549246298 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0003c40 con 0x7fe5a8004d90 2023-10-16T10:00:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5bf046700 1 -- 172.21.15.181:0/1549246298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a8004d90 msgr2=0x7fe5a8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5bf046700 1 --2- 172.21.15.181:0/1549246298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a8004d90 0x7fe5a8005210 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000c190 tx=0x7fe5a8005880 comp rx=0 tx=0).stop 2023-10-16T10:00:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5bd5e3700 1 -- 172.21.15.181:0/1549246298 reap_dead start 2023-10-16T10:00:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5bf046700 1 -- 172.21.15.181:0/1549246298 shutdown_connections 2023-10-16T10:00:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5bf046700 1 -- 172.21.15.181:0/1549246298 >> 172.21.15.181:0/1549246298 conn(0x7fe5b80d04a0 msgr2=0x7fe5b80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5bf046700 1 -- 172.21.15.181:0/1549246298 shutdown_connections 2023-10-16T10:00:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.428+0000 7fe5bf046700 1 -- 172.21.15.181:0/1549246298 wait complete. 2023-10-16T10:00:21.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.429+0000 7fe5bf046700 1 Processor -- start 2023-10-16T10:00:21.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.429+0000 7fe5bf046700 1 -- start start 2023-10-16T10:00:21.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bf046700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a8004d90 0x7fe5b81e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bf046700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b80c3600 0x7fe5b80b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bf046700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b80c2f20 0x7fe5b80b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bf046700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b80b3c00 con 0x7fe5b80c3600 2023-10-16T10:00:21.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bf046700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b80b6190 con 0x7fe5b80c2f20 2023-10-16T10:00:21.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bf046700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b80ca460 con 0x7fe5a8004d90 2023-10-16T10:00:21.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b80c3600 0x7fe5b80b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b80c3600 0x7fe5b80b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44042/0 (socket says 172.21.15.181:44042) 2023-10-16T10:00:21.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5b7fff700 1 -- 172.21.15.181:0/3860636457 learned_addr learned my addr 172.21.15.181:0/3860636457 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:21.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bcde2700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a8004d90 0x7fe5b81e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.430+0000 7fe5bd5e3700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b80c2f20 0x7fe5b80b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b7fff700 1 -- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a8004d90 msgr2=0x7fe5b81e5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b7fff700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a8004d90 0x7fe5b81e5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b7fff700 1 -- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b80c2f20 msgr2=0x7fe5b80b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b7fff700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b80c2f20 0x7fe5b80b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b7fff700 1 -- 172.21.15.181:0/3860636457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b80ca740 con 0x7fe5b80c3600 2023-10-16T10:00:21.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b7fff700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b80c3600 0x7fe5b80b5710 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4010890 tx=0x7fe5a4010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4015880 con 0x7fe5b80c3600 2023-10-16T10:00:21.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a400c050 con 0x7fe5b80c3600 2023-10-16T10:00:21.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b80b3e50 con 0x7fe5b80c3600 2023-10-16T10:00:21.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4015c20 con 0x7fe5b80c3600 2023-10-16T10:00:21.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.431+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b80b3510 con 0x7fe5b80c3600 2023-10-16T10:00:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.432+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b8067d20 con 0x7fe5b80c3600 2023-10-16T10:00:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.432+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 49) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe5a4016420 con 0x7fe5b80c3600 2023-10-16T10:00:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.433+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fe5a4089b90 con 0x7fe5b80c3600 2023-10-16T10:00:21.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.437+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe5a4054c30 con 0x7fe5b80c3600 2023-10-16T10:00:21.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.662+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 50) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe5a4015d80 con 0x7fe5b80c3600 2023-10-16T10:00:21.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.663+0000 7fe5b57fa700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fe59c070bc0 0x7fe59c072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.664+0000 7fe5bcde2700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fe59c070bc0 0x7fe59c072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.667+0000 7fe5bcde2700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fe59c070bc0 0x7fe59c072fb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac000ef0 tx=0x7fe5ac009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.681+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe5b80cf620 con 0x7fe5b80c3600 2023-10-16T10:00:21.683 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T10:00:21.682+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:00:21.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.683+0000 7fe5b57fa700 1 -- 172.21.15.181:0/3860636457 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe5a40859a0 con 0x7fe5b80c3600 2023-10-16T10:00:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.686+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fe59c070bc0 msgr2=0x7fe59c072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fe59c070bc0 0x7fe59c072fb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac000ef0 tx=0x7fe5ac009040 comp rx=0 tx=0).stop 2023-10-16T10:00:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b80c3600 msgr2=0x7fe5b80b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b80c3600 0x7fe5b80b5710 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4010890 tx=0x7fe5a4010ba0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 shutdown_connections 2023-10-16T10:00:21.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fe59c070bc0 0x7fe59c072fb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a8004d90 0x7fe5b81e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5b80c2f20 0x7fe5b80b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 --2- 172.21.15.181:0/3860636457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b80c3600 0x7fe5b80b5710 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 >> 172.21.15.181:0/3860636457 conn(0x7fe5b80d04a0 msgr2=0x7fe5b8073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:21.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 shutdown_connections 2023-10-16T10:00:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.687+0000 7fe5bf046700 1 -- 172.21.15.181:0/3860636457 wait complete. 2023-10-16T10:00:21.691 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:00:21.692 INFO:tasks.ceph.mds.a.smithi181.stderr: -3706> 2023-10-16T10:00:04.624+0000 7f6ff296b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:00:21.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:00:21.719 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T10:00:21.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.826+0000 7fb8bd1c2700 1 Processor -- start 2023-10-16T10:00:21.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.826+0000 7fb8bd1c2700 1 -- start start 2023-10-16T10:00:21.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.826+0000 7fb8bd1c2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8b80cb2b0 0x7fb8b806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.826+0000 7fb8bd1c2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb8b80cb690 0x7fb8b80b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.826+0000 7fb8bd1c2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b8070590 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.826+0000 7fb8bd1c2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b8156330 con 0x7fb8b80cb2b0 2023-10-16T10:00:21.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.826+0000 7fb8bd1c2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b806fcf0 con 0x7fb8b80cb690 2023-10-16T10:00:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8b80cb2b0 0x7fb8b806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8b80cb2b0 0x7fb8b806f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44082/0 (socket says 172.21.15.181:44082) 2023-10-16T10:00:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2382470644 learned_addr learned my addr 172.21.15.181:0/2382470644 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b659c700 1 --2- 172.21.15.181:0/2382470644 >> v2:172.21.15.190:3300/0 conn(0x7fb8b80cb690 0x7fb8b80b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2382470644 >> v2:172.21.15.190:3300/0 conn(0x7fb8b80cb690 msgr2=0x7fb8b80b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 --2- 172.21.15.181:0/2382470644 >> v2:172.21.15.190:3300/0 conn(0x7fb8b80cb690 0x7fb8b80b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2382470644 >> v1:172.21.15.190:6789/0 conn(0x7fb8b80b2ba0 legacy=0x7fb8b815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2382470644 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b806ffd0 con 0x7fb8b80cb2b0 2023-10-16T10:00:21.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b659c700 1 --2- 172.21.15.181:0/2382470644 >> v2:172.21.15.190:3300/0 conn(0x7fb8b80cb690 0x7fb8b80b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:21.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b6d9d700 1 --2- 172.21.15.181:0/2382470644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80cb2b0 0x7fb8b806f710 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8159a00 tx=0x7fb8ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a719e04a0b436bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b5d9b700 1 -- 172.21.15.181:0/2382470644 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00b870 con 0x7fb8b80cb2b0 2023-10-16T10:00:21.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b5d9b700 1 -- 172.21.15.181:0/2382470644 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac00b9d0 con 0x7fb8b80cb2b0 2023-10-16T10:00:21.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.827+0000 7fb8b5d9b700 1 -- 172.21.15.181:0/2382470644 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00bd60 con 0x7fb8b80cb2b0 2023-10-16T10:00:21.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2382470644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80cb2b0 msgr2=0x7fb8b806f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2382470644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80cb2b0 0x7fb8b806f710 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8159a00 tx=0x7fb8ac009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2382470644 shutdown_connections 2023-10-16T10:00:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2382470644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80cb2b0 0x7fb8b806f710 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2382470644 >> v2:172.21.15.190:3300/0 conn(0x7fb8b80cb690 0x7fb8b80b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2382470644 >> 172.21.15.181:0/2382470644 conn(0x7fb8b80cd010 msgr2=0x7fb8b80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2382470644 shutdown_connections 2023-10-16T10:00:21.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.828+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2382470644 wait complete. 2023-10-16T10:00:21.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.829+0000 7fb8bd1c2700 1 Processor -- start 2023-10-16T10:00:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.829+0000 7fb8bd1c2700 1 -- start start 2023-10-16T10:00:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8bd1c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80b2ba0 0x7fb8b80c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8bd1c2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b80cb2b0 0x7fb8b81f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8bd1c2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b80cb690 0x7fb8b81f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8bd1c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80ca9b0 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8bd1c2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80c5f40 con 0x7fb8b80cb2b0 2023-10-16T10:00:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8bd1c2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80c6220 con 0x7fb8b80cb690 2023-10-16T10:00:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80b2ba0 0x7fb8b80c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80b2ba0 0x7fb8b80c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44090/0 (socket says 172.21.15.181:44090) 2023-10-16T10:00:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2196592902 learned_addr learned my addr 172.21.15.181:0/2196592902 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b759e700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b80cb690 0x7fb8b81f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b659c700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b80cb2b0 0x7fb8b81f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b80cb690 msgr2=0x7fb8b81f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b80cb690 0x7fb8b81f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b80cb2b0 msgr2=0x7fb8b81f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:21.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b80cb2b0 0x7fb8b81f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:21.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.830+0000 7fb8b6d9d700 1 -- 172.21.15.181:0/2196592902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac009550 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.831+0000 7fb8b6d9d700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80b2ba0 0x7fb8b80c9980 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac000c00 tx=0x7fb8ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.831+0000 7fb89f7fe700 1 -- 172.21.15.181:0/2196592902 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac004540 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.831+0000 7fb89f7fe700 1 -- 172.21.15.181:0/2196592902 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac0036a0 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.831+0000 7fb89f7fe700 1 -- 172.21.15.181:0/2196592902 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac003a30 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.831+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b81564d0 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.831+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b80ca0a0 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.832+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b81566e0 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.832+0000 7fb89f7fe700 1 -- 172.21.15.181:0/2196592902 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 50) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb8ac021070 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.832+0000 7fb89f7fe700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fb8a406ad50 0x7fb8a406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.833+0000 7fb89f7fe700 1 -- 172.21.15.181:0/2196592902 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6907+0+0 (secure 0 0 0) 0x7fb8ac08af50 con 0x7fb8b80b2ba0 2023-10-16T10:00:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.833+0000 7fb8b659c700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fb8a406ad50 0x7fb8a406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:21.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.833+0000 7fb8b659c700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fb8a406ad50 0x7fb8a406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80cbaa0 tx=0x7fb8a8008040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:21.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:21.837+0000 7fb89f7fe700 1 -- 172.21.15.181:0/2196592902 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8ac055fb0 con 0x7fb8b80b2ba0 2023-10-16T10:00:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.075+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb8b806f4f0 con 0x7fb8b80b2ba0 2023-10-16T10:00:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.685+0000 7fb89f7fe700 1 -- 172.21.15.181:0/2196592902 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8ac0150a0 con 0x7fb8b80b2ba0 2023-10-16T10:00:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fb8a406ad50 msgr2=0x7fb8a406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fb8a406ad50 0x7fb8a406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80cbaa0 tx=0x7fb8a8008040 comp rx=0 tx=0).stop 2023-10-16T10:00:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80b2ba0 msgr2=0x7fb8b80c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80b2ba0 0x7fb8b80c9980 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac000c00 tx=0x7fb8ac004750 comp rx=0 tx=0).stop 2023-10-16T10:00:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 shutdown_connections 2023-10-16T10:00:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fb8a406ad50 0x7fb8a406d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8b80cb690 0x7fb8b81f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8b80b2ba0 0x7fb8b80c9980 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 --2- 172.21.15.181:0/2196592902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8b80cb2b0 0x7fb8b81f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 >> 172.21.15.181:0/2196592902 conn(0x7fb8b80cd010 msgr2=0x7fb8b80b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:22.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 shutdown_connections 2023-10-16T10:00:22.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.688+0000 7fb8bd1c2700 1 -- 172.21.15.181:0/2196592902 wait complete. 2023-10-16T10:00:22.705 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:00:22.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.869+0000 7f4c2b068700 1 Processor -- start 2023-10-16T10:00:22.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.869+0000 7f4c2b068700 1 -- start start 2023-10-16T10:00:22.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c2b068700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c240b0f70 0x7f4c240b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:22.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c2b068700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c240ce810 0x7f4c2415f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:22.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c2b068700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240d23c0 con 0x7f4c2415eea0 2023-10-16T10:00:22.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c2b068700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c240cd990 con 0x7f4c240ce810 2023-10-16T10:00:22.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c2b068700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c240cdc70 con 0x7f4c240b0f70 2023-10-16T10:00:22.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c23fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c240b0f70 0x7f4c240b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:22.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c29605700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c240ce810 0x7f4c2415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:22.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c23fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c240b0f70 0x7f4c240b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:41144/0 (socket says 172.21.15.181:41144) 2023-10-16T10:00:22.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c23fff700 1 -- 172.21.15.181:0/2583069575 learned_addr learned my addr 172.21.15.181:0/2583069575 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:22.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c29605700 1 -- 172.21.15.181:0/2583069575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c240b0f70 msgr2=0x7f4c240b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:22.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c29605700 1 --2- 172.21.15.181:0/2583069575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c240b0f70 0x7f4c240b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c29605700 1 -- 172.21.15.181:0/2583069575 >> v1:172.21.15.190:6789/0 conn(0x7f4c2415eea0 legacy=0x7f4c240ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:22.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c29605700 1 -- 172.21.15.181:0/2583069575 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240cce60 con 0x7f4c240ce810 2023-10-16T10:00:22.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.870+0000 7f4c29605700 1 --2- 172.21.15.181:0/2583069575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240ce810 0x7f4c2415f620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009d90 tx=0x7f4c1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa120d2085f33f8a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:22.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c237fe700 1 -- 172.21.15.181:0/2583069575 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00e070 con 0x7f4c240ce810 2023-10-16T10:00:22.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c237fe700 1 -- 172.21.15.181:0/2583069575 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c00bb20 con 0x7f4c240ce810 2023-10-16T10:00:22.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c237fe700 1 -- 172.21.15.181:0/2583069575 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c012430 con 0x7f4c240ce810 2023-10-16T10:00:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 -- 172.21.15.181:0/2583069575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240ce810 msgr2=0x7f4c2415f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:22.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 --2- 172.21.15.181:0/2583069575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240ce810 0x7f4c2415f620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009d90 tx=0x7f4c1c00b330 comp rx=0 tx=0).stop 2023-10-16T10:00:22.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 -- 172.21.15.181:0/2583069575 shutdown_connections 2023-10-16T10:00:22.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 --2- 172.21.15.181:0/2583069575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240ce810 0x7f4c2415f620 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 --2- 172.21.15.181:0/2583069575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c240b0f70 0x7f4c240b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 -- 172.21.15.181:0/2583069575 >> 172.21.15.181:0/2583069575 conn(0x7f4c240d1610 msgr2=0x7f4c240d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:22.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 -- 172.21.15.181:0/2583069575 shutdown_connections 2023-10-16T10:00:22.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.871+0000 7f4c2b068700 1 -- 172.21.15.181:0/2583069575 wait complete. 2023-10-16T10:00:22.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.872+0000 7f4c2b068700 1 Processor -- start 2023-10-16T10:00:22.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.872+0000 7f4c2b068700 1 -- start start 2023-10-16T10:00:22.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c2b068700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240b0f70 0x7f4c241586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:22.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c2b068700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ce810 0x7f4c24155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:22.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c2b068700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2415eea0 0x7f4c2406f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:22.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c2b068700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24159120 con 0x7f4c240ce810 2023-10-16T10:00:22.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c2b068700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c240c6180 con 0x7f4c240b0f70 2023-10-16T10:00:22.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c2b068700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24157930 con 0x7f4c2415eea0 2023-10-16T10:00:22.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c23fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ce810 0x7f4c24155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.873+0000 7f4c23fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ce810 0x7f4c24155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44124/0 (socket says 172.21.15.181:44124) 2023-10-16T10:00:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c23fff700 1 -- 172.21.15.181:0/1914670727 learned_addr learned my addr 172.21.15.181:0/1914670727 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c28e04700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240b0f70 0x7f4c241586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c29605700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2415eea0 0x7f4c2406f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c23fff700 1 -- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2415eea0 msgr2=0x7f4c2406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c23fff700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2415eea0 0x7f4c2406f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c23fff700 1 -- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240b0f70 msgr2=0x7f4c241586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c23fff700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240b0f70 0x7f4c241586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c23fff700 1 -- 172.21.15.181:0/1914670727 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c009a70 con 0x7f4c240ce810 2023-10-16T10:00:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c23fff700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ce810 0x7f4c24155fd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e910 tx=0x7f4c1000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c100096a0 con 0x7f4c240ce810 2023-10-16T10:00:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1000ee60 con 0x7f4c240ce810 2023-10-16T10:00:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c24157c10 con 0x7f4c240ce810 2023-10-16T10:00:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000ca90 con 0x7f4c240ce810 2023-10-16T10:00:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.874+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241581d0 con 0x7f4c240ce810 2023-10-16T10:00:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.875+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c24158540 con 0x7f4c240ce810 2023-10-16T10:00:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.876+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 50) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4c1000cbf0 con 0x7f4c240ce810 2023-10-16T10:00:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.876+0000 7f4c217fa700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f4c0806ad50 0x7f4c0806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.876+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6907+0+0 (secure 0 0 0) 0x7f4c10089680 con 0x7f4c240ce810 2023-10-16T10:00:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.876+0000 7f4c28e04700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f4c0806ad50 0x7f4c0806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.876+0000 7f4c28e04700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f4c0806ad50 0x7f4c0806d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c18007fd0 tx=0x7f4c18009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:22.880+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c10054540 con 0x7f4c240ce810 2023-10-16T10:00:23.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.071+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c24067d20 con 0x7f4c240ce810 2023-10-16T10:00:23.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.675+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 51) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4c10054910 con 0x7f4c240ce810 2023-10-16T10:00:23.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.687+0000 7f4c217fa700 1 -- 172.21.15.181:0/1914670727 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4c10083a80 con 0x7f4c240ce810 2023-10-16T10:00:23.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.691+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f4c0806ad50 msgr2=0x7f4c0806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:23.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.691+0000 7f4c2b068700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f4c0806ad50 0x7f4c0806d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c18007fd0 tx=0x7f4c18009040 comp rx=0 tx=0).stop 2023-10-16T10:00:23.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.691+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ce810 msgr2=0x7f4c24155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:23.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.691+0000 7f4c2b068700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ce810 0x7f4c24155fd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e910 tx=0x7f4c1000ec20 comp rx=0 tx=0).stop 2023-10-16T10:00:23.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.692+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 shutdown_connections 2023-10-16T10:00:23.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.692+0000 7f4c2b068700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f4c0806ad50 0x7f4c0806d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.692+0000 7f4c2b068700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2415eea0 0x7f4c2406f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.692+0000 7f4c2b068700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c240b0f70 0x7f4c241586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.692+0000 7f4c2b068700 1 --2- 172.21.15.181:0/1914670727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ce810 0x7f4c24155fd0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.692+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 >> 172.21.15.181:0/1914670727 conn(0x7f4c240d1610 msgr2=0x7f4c240cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:23.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.692+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 shutdown_connections 2023-10-16T10:00:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.693+0000 7f4c2b068700 1 -- 172.21.15.181:0/1914670727 wait complete. 2023-10-16T10:00:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:00:23.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:00:23.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.832+0000 7f2fe8a20700 1 Processor -- start 2023-10-16T10:00:23.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.832+0000 7f2fe8a20700 1 -- start start 2023-10-16T10:00:23.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe8a20700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2fe40ce610 0x7f2fe406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:23.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe8a20700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2fe40d0fc0 0x7f2fe40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:23.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe8a20700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe40d0060 0x7f2fe406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:23.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe8a20700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4162190 con 0x7f2fe40ce610 2023-10-16T10:00:23.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe8a20700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4162650 con 0x7f2fe40d0fc0 2023-10-16T10:00:23.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe8a20700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4162370 con 0x7f2fe40d0060 2023-10-16T10:00:23.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2fe40ce610 0x7f2fe406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:23.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2fe40ce610 0x7f2fe406d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44130/0 (socket says 172.21.15.181:44130) 2023-10-16T10:00:23.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe259c700 1 -- 172.21.15.181:0/1916808733 learned_addr learned my addr 172.21.15.181:0/1916808733 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:23.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.833+0000 7f2fe2d9d700 1 --2- 172.21.15.181:0/1916808733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe40d0060 0x7f2fe406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:23.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe1d9b700 1 --2- 172.21.15.181:0/1916808733 >> v2:172.21.15.190:3300/0 conn(0x7f2fe40d0fc0 0x7f2fe40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:23.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe259c700 1 -- 172.21.15.181:0/1916808733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe40d0060 msgr2=0x7f2fe406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:23.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe259c700 1 --2- 172.21.15.181:0/1916808733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe40d0060 0x7f2fe406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe259c700 1 -- 172.21.15.181:0/1916808733 >> v2:172.21.15.190:3300/0 conn(0x7f2fe40d0fc0 msgr2=0x7f2fe40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:23.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe259c700 1 --2- 172.21.15.181:0/1916808733 >> v2:172.21.15.190:3300/0 conn(0x7f2fe40d0fc0 0x7f2fe40cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe259c700 1 -- 172.21.15.181:0/1916808733 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe406f0f0 con 0x7f2fe40ce610 2023-10-16T10:00:23.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe2d9d700 1 --2- 172.21.15.181:0/1916808733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe40d0060 0x7f2fe406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:23.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe1d9b700 1 --2- 172.21.15.181:0/1916808733 >> v2:172.21.15.190:3300/0 conn(0x7f2fe40d0fc0 0x7f2fe40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:23.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe259c700 1 --2- 172.21.15.181:0/1916808733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe406d660 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40c7520 tx=0x7f2fd8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5ee23c9734115e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:23.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe159a700 1 -- 172.21.15.181:0/1916808733 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8005e30 con 0x7f2fe40ce610 2023-10-16T10:00:23.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe159a700 1 -- 172.21.15.181:0/1916808733 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8004030 con 0x7f2fe40ce610 2023-10-16T10:00:23.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.834+0000 7f2fe159a700 1 -- 172.21.15.181:0/1916808733 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80043e0 con 0x7f2fe40ce610 2023-10-16T10:00:23.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 -- 172.21.15.181:0/1916808733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 msgr2=0x7f2fe406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:23.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/1916808733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe406d660 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40c7520 tx=0x7f2fd8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 -- 172.21.15.181:0/1916808733 shutdown_connections 2023-10-16T10:00:23.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/1916808733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe406d660 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/1916808733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe40d0060 0x7f2fe406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/1916808733 >> v2:172.21.15.190:3300/0 conn(0x7f2fe40d0fc0 0x7f2fe40cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 -- 172.21.15.181:0/1916808733 >> 172.21.15.181:0/1916808733 conn(0x7f2fe40d04a0 msgr2=0x7f2fe40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:23.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 -- 172.21.15.181:0/1916808733 shutdown_connections 2023-10-16T10:00:23.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.835+0000 7f2fe8a20700 1 -- 172.21.15.181:0/1916808733 wait complete. 2023-10-16T10:00:23.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.836+0000 7f2fe8a20700 1 Processor -- start 2023-10-16T10:00:23.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.836+0000 7f2fe8a20700 1 -- start start 2023-10-16T10:00:23.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe8a20700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe409b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:23.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe8a20700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fe40d0fc0 0x7f2fe409d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:23.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe8a20700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe409dc70 0x7f2fe415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:23.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe8a20700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe409e110 con 0x7f2fe40ce610 2023-10-16T10:00:23.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe8a20700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4160240 con 0x7f2fe40d0fc0 2023-10-16T10:00:23.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe8a20700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4160520 con 0x7f2fe409dc70 2023-10-16T10:00:23.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe409b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:23.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe409b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44136/0 (socket says 172.21.15.181:44136) 2023-10-16T10:00:23.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 -- 172.21.15.181:0/2089630172 learned_addr learned my addr 172.21.15.181:0/2089630172 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:23.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe2d9d700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe409dc70 0x7f2fe415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:23.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 -- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe409dc70 msgr2=0x7f2fe415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:23.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe409dc70 0x7f2fe415f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe1d9b700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fe40d0fc0 0x7f2fe409d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:23.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 -- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fe40d0fc0 msgr2=0x7f2fe409d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:23.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fe40d0fc0 0x7f2fe409d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:23.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 -- 172.21.15.181:0/2089630172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8009500 con 0x7f2fe40ce610 2023-10-16T10:00:23.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe259c700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe409b9b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009c10 tx=0x7f2fd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:23.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fd2ffd700 1 -- 172.21.15.181:0/2089630172 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8003dd0 con 0x7f2fe40ce610 2023-10-16T10:00:23.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fd2ffd700 1 -- 172.21.15.181:0/2089630172 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8011420 con 0x7f2fe40ce610 2023-10-16T10:00:23.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fd2ffd700 1 -- 172.21.15.181:0/2089630172 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80117d0 con 0x7f2fe40ce610 2023-10-16T10:00:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.837+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe4160800 con 0x7f2fe40ce610 2023-10-16T10:00:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.838+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe409bef0 con 0x7f2fe40ce610 2023-10-16T10:00:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.838+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe4160d20 con 0x7f2fe40ce610 2023-10-16T10:00:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.839+0000 7f2fd2ffd700 1 -- 172.21.15.181:0/2089630172 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 51) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2fd8011930 con 0x7f2fe40ce610 2023-10-16T10:00:23.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.839+0000 7f2fd2ffd700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f2fc406af90 0x7f2fc406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:23.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.839+0000 7f2fd2ffd700 1 -- 172.21.15.181:0/2089630172 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f2fd808aa60 con 0x7f2fe40ce610 2023-10-16T10:00:23.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.839+0000 7f2fe1d9b700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f2fc406af90 0x7f2fc406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:23.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.840+0000 7f2fe1d9b700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f2fc406af90 0x7f2fc406d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40b57b0 tx=0x7f2fd4008040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:23.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:23.844+0000 7f2fd2ffd700 1 -- 172.21.15.181:0/2089630172 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2fe4160d20 con 0x7f2fe40ce610 2023-10-16T10:00:24.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.035+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2fe409c0b0 con 0x7f2fe40ce610 2023-10-16T10:00:24.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.970+0000 7f2fd2ffd700 1 -- 172.21.15.181:0/2089630172 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2fd80559d0 con 0x7f2fe40ce610 2023-10-16T10:00:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f2fc406af90 msgr2=0x7f2fc406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f2fc406af90 0x7f2fc406d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40b57b0 tx=0x7f2fd4008040 comp rx=0 tx=0).stop 2023-10-16T10:00:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 msgr2=0x7f2fe409b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe409b9b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009c10 tx=0x7f2fd8004750 comp rx=0 tx=0).stop 2023-10-16T10:00:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 shutdown_connections 2023-10-16T10:00:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f2fc406af90 0x7f2fc406d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fe409dc70 0x7f2fe415f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:24.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.973+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40ce610 0x7f2fe409b9b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.974+0000 7f2fe8a20700 1 --2- 172.21.15.181:0/2089630172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fe40d0fc0 0x7f2fe409d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.974+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 >> 172.21.15.181:0/2089630172 conn(0x7f2fe40d04a0 msgr2=0x7f2fe415d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.974+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 shutdown_connections 2023-10-16T10:00:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:24.974+0000 7f2fe8a20700 1 -- 172.21.15.181:0/2089630172 wait complete. 2023-10-16T10:00:24.977 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T10:00:24.991 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:00:25.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.118+0000 7f3892dff700 1 Processor -- start 2023-10-16T10:00:25.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.118+0000 7f3892dff700 1 -- start start 2023-10-16T10:00:25.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f3892dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f3892dff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f388c0ce6a0 0x7f388c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f3892dff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f388c072760 0x7f388c06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f3892dff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0cef00 con 0x7f388c0ce6a0 2023-10-16T10:00:25.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f3892dff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0cec20 con 0x7f388c072760 2023-10-16T10:00:25.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f3892dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c071c80 con 0x7f388c15da70 2023-10-16T10:00:25.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f388c0ce6a0 0x7f388c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f388c0ce6a0 0x7f388c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44146/0 (socket says 172.21.15.181:44146) 2023-10-16T10:00:25.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 -- 172.21.15.181:0/3633153335 learned_addr learned my addr 172.21.15.181:0/3633153335 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:25.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f3890b9b700 1 --2- 172.21.15.181:0/3633153335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f389139c700 1 --2- 172.21.15.181:0/3633153335 >> v2:172.21.15.190:3300/0 conn(0x7f388c072760 0x7f388c06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 -- 172.21.15.181:0/3633153335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 msgr2=0x7f388c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:25.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 --2- 172.21.15.181:0/3633153335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 -- 172.21.15.181:0/3633153335 >> v2:172.21.15.190:3300/0 conn(0x7f388c072760 msgr2=0x7f388c06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:25.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 --2- 172.21.15.181:0/3633153335 >> v2:172.21.15.190:3300/0 conn(0x7f388c072760 0x7f388c06ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 -- 172.21.15.181:0/3633153335 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c071f60 con 0x7f388c0ce6a0 2023-10-16T10:00:25.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388bfff700 1 --2- 172.21.15.181:0/3633153335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0ce6a0 0x7f388c06a8a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f388c161530 tx=0x7f3878009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca3003370392ae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.119+0000 7f388b7fe700 1 -- 172.21.15.181:0/3633153335 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387800b870 con 0x7f388c0ce6a0 2023-10-16T10:00:25.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f388b7fe700 1 -- 172.21.15.181:0/3633153335 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387800b9d0 con 0x7f388c0ce6a0 2023-10-16T10:00:25.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f388b7fe700 1 -- 172.21.15.181:0/3633153335 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387800bd60 con 0x7f388c0ce6a0 2023-10-16T10:00:25.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 -- 172.21.15.181:0/3633153335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0ce6a0 msgr2=0x7f388c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 --2- 172.21.15.181:0/3633153335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0ce6a0 0x7f388c06a8a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f388c161530 tx=0x7f3878009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 -- 172.21.15.181:0/3633153335 shutdown_connections 2023-10-16T10:00:25.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 --2- 172.21.15.181:0/3633153335 >> v2:172.21.15.190:3300/0 conn(0x7f388c072760 0x7f388c06ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 --2- 172.21.15.181:0/3633153335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 --2- 172.21.15.181:0/3633153335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0ce6a0 0x7f388c06a8a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 -- 172.21.15.181:0/3633153335 >> 172.21.15.181:0/3633153335 conn(0x7f388c0cd010 msgr2=0x7f388c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:25.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 -- 172.21.15.181:0/3633153335 shutdown_connections 2023-10-16T10:00:25.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.120+0000 7f3892dff700 1 -- 172.21.15.181:0/3633153335 wait complete. 2023-10-16T10:00:25.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.121+0000 7f3892dff700 1 Processor -- start 2023-10-16T10:00:25.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.121+0000 7f3892dff700 1 -- start start 2023-10-16T10:00:25.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3892dff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c072760 0x7f388c1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3892dff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c0ce6a0 0x7f388c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3892dff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3892dff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0ad620 con 0x7f388c072760 2023-10-16T10:00:25.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3892dff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0ad900 con 0x7f388c0ce6a0 2023-10-16T10:00:25.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3892dff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0adbe0 con 0x7f388c15da70 2023-10-16T10:00:25.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3890b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c072760 0x7f388c1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3890b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c072760 0x7f388c1519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44152/0 (socket says 172.21.15.181:44152) 2023-10-16T10:00:25.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f3890b9b700 1 -- 172.21.15.181:0/2168568775 learned_addr learned my addr 172.21.15.181:0/2168568775 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:25.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f389139c700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.122+0000 7f388bfff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c0ce6a0 0x7f388c0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3890b9b700 1 -- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 msgr2=0x7f388c0af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3890b9b700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3890b9b700 1 -- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c0ce6a0 msgr2=0x7f388c0aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3890b9b700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c0ce6a0 0x7f388c0aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3890b9b700 1 -- 172.21.15.181:0/2168568775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3878009550 con 0x7f388c072760 2023-10-16T10:00:25.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3890b9b700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c072760 0x7f388c1519b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f388000c7f0 tx=0x7f388000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880004220 con 0x7f388c072760 2023-10-16T10:00:25.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3880004380 con 0x7f388c072760 2023-10-16T10:00:25.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c0adec0 con 0x7f388c072760 2023-10-16T10:00:25.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800036a0 con 0x7f388c072760 2023-10-16T10:00:25.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.123+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c151ef0 con 0x7f388c072760 2023-10-16T10:00:25.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.124+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c06f6c0 con 0x7f388c072760 2023-10-16T10:00:25.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.124+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 51) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3880003800 con 0x7f388c072760 2023-10-16T10:00:25.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.125+0000 7f38897fa700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f387006af90 0x7f387006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.125+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f3880089340 con 0x7f388c072760 2023-10-16T10:00:25.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.125+0000 7f388bfff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f387006af90 0x7f387006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.125+0000 7f388bfff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f387006af90 0x7f387006d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f3878008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.129+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3880054190 con 0x7f388c072760 2023-10-16T10:00:25.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.246+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3880054560 con 0x7f388c072760 2023-10-16T10:00:25.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.327+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f388c0b1ae0 con 0x7f388c072760 2023-10-16T10:00:25.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.331+0000 7f38897fa700 1 -- 172.21.15.181:0/2168568775 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v31) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3880071790 con 0x7f388c072760 2023-10-16T10:00:25.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f387006af90 msgr2=0x7f387006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f387006af90 0x7f387006d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f3878008030 comp rx=0 tx=0).stop 2023-10-16T10:00:25.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c072760 msgr2=0x7f388c1519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c072760 0x7f388c1519b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f388000c7f0 tx=0x7f388000cb00 comp rx=0 tx=0).stop 2023-10-16T10:00:25.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 shutdown_connections 2023-10-16T10:00:25.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f387006af90 0x7f387006d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15da70 0x7f388c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c072760 0x7f388c1519b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 --2- 172.21.15.181:0/2168568775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c0ce6a0 0x7f388c0aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 >> 172.21.15.181:0/2168568775 conn(0x7f388c0cd010 msgr2=0x7f388c06f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:25.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 shutdown_connections 2023-10-16T10:00:25.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.334+0000 7f3892dff700 1 -- 172.21.15.181:0/2168568775 wait complete. 2023-10-16T10:00:25.348 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:00:25.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.469+0000 7f5dbc53a700 1 Processor -- start 2023-10-16T10:00:25.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.469+0000 7f5dbc53a700 1 -- start start 2023-10-16T10:00:25.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5dbc53a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 0x7f5db406be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5dbc53a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db415dfb0 con 0x7f5db415dbd0 2023-10-16T10:00:25.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5dbc53a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40cd3a0 con 0x7f5db40cdd80 2023-10-16T10:00:25.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5dbc53a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cd830 con 0x7f5db40ce200 2023-10-16T10:00:25.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5dba2d6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5db415dbd0 0x7f5db40d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55236/0 (socket says 172.21.15.181:55236) 2023-10-16T10:00:25.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5dba2d6700 1 -- 172.21.15.181:0/805172081 learned_addr learned my addr 172.21.15.181:0/805172081 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:25.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5dbaad7700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 0x7f5db406be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837924392 0 0) 0x7f5db0001110 con 0x7f5db415dbd0 2023-10-16T10:00:25.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da00035b0 con 0x7f5db415dbd0 2023-10-16T10:00:25.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 116416977 0 0) 0x7f5da00035b0 con 0x7f5db415dbd0 2023-10-16T10:00:25.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 msgr2=0x7f5db406be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:25.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 0x7f5db406be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 >> v1:172.21.15.190:6789/0 conn(0x7f5db40cdd80 legacy=0x7f5db406b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40d1fc0 con 0x7f5db415dbd0 2023-10-16T10:00:25.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5db0001800 con 0x7f5db415dbd0 2023-10-16T10:00:25.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 >> v1:172.21.15.181:6789/0 conn(0x7f5db415dbd0 legacy=0x7f5db40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0003a30 0x7f5da0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.470+0000 7f5db92d4700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 0x7f5da0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db92d4700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0004d90 0x7f5da0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cd830 con 0x7f5da0003a30 2023-10-16T10:00:25.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000baf0 con 0x7f5da0004d90 2023-10-16T10:00:25.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000bcb0 con 0x7f5db40ce200 2023-10-16T10:00:25.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0003a30 0x7f5da0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db9ad5700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 0x7f5da0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dbaad7700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0004d90 0x7f5da0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dba2d6700 1 -- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 msgr2=0x7f5da0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:25.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40ce200 0x7f5da0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dba2d6700 1 -- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0004d90 msgr2=0x7f5da0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:25.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0004d90 0x7f5da0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dba2d6700 1 -- 172.21.15.181:0/805172081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00035b0 con 0x7f5da0003a30 2023-10-16T10:00:25.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0003a30 0x7f5da0003e50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c210 tx=0x7f5da0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d123e65a53444cad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0007db0 con 0x7f5da0003a30 2023-10-16T10:00:25.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db000cdb0 con 0x7f5da0003a30 2023-10-16T10:00:25.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.471+0000 7f5db92d4700 1 -- 172.21.15.181:0/805172081 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0005b80 con 0x7f5da0003a30 2023-10-16T10:00:25.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.472+0000 7f5dbc53a700 1 -- 172.21.15.181:0/805172081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0003a30 msgr2=0x7f5da0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.472+0000 7f5dbc53a700 1 --2- 172.21.15.181:0/805172081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5da0003a30 0x7f5da0003e50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c210 tx=0x7f5da0005880 comp rx=0 tx=0).stop 2023-10-16T10:00:25.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.472+0000 7f5dbaad7700 1 -- 172.21.15.181:0/805172081 reap_dead start 2023-10-16T10:00:25.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.472+0000 7f5dbc53a700 1 -- 172.21.15.181:0/805172081 shutdown_connections 2023-10-16T10:00:25.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.472+0000 7f5dbc53a700 1 -- 172.21.15.181:0/805172081 >> 172.21.15.181:0/805172081 conn(0x7f5db409a930 msgr2=0x7f5db40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:25.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.472+0000 7f5dbc53a700 1 -- 172.21.15.181:0/805172081 shutdown_connections 2023-10-16T10:00:25.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.472+0000 7f5dbc53a700 1 -- 172.21.15.181:0/805172081 wait complete. 2023-10-16T10:00:25.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.473+0000 7f5dbc53a700 1 Processor -- start 2023-10-16T10:00:25.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.473+0000 7f5dbc53a700 1 -- start start 2023-10-16T10:00:25.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0003a30 0x7f5db414a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5db40c9910 0x7f5db40cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40c9d90 0x7f5db40cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4158260 con 0x7f5db40c9910 2023-10-16T10:00:25.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40c8870 con 0x7f5da0003a30 2023-10-16T10:00:25.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40c8b50 con 0x7f5db40c9d90 2023-10-16T10:00:25.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5db40c9910 0x7f5db40cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5db40c9910 0x7f5db40cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44172/0 (socket says 172.21.15.181:44172) 2023-10-16T10:00:25.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 -- 172.21.15.181:0/1902675869 learned_addr learned my addr 172.21.15.181:0/1902675869 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:25.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbaad7700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40c9d90 0x7f5db40cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0003a30 0x7f5db414a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 -- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40c9d90 msgr2=0x7f5db40cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40c9d90 0x7f5db40cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 -- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0003a30 msgr2=0x7f5db414a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0003a30 0x7f5db414a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 -- 172.21.15.181:0/1902675869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40c8e30 con 0x7f5db40c9910 2023-10-16T10:00:25.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0003a30 0x7f5db414a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:25.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbaad7700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40c9d90 0x7f5db40cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:25.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5db9ad5700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5db40c9910 0x7f5db40cab30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5da400f820 tx=0x7f5da400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5daaffd700 1 -- 172.21.15.181:0/1902675869 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4005320 con 0x7f5db40c9910 2023-10-16T10:00:25.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5daaffd700 1 -- 172.21.15.181:0/1902675869 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40054a0 con 0x7f5db40c9910 2023-10-16T10:00:25.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db40c9110 con 0x7f5db40c9910 2023-10-16T10:00:25.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5daaffd700 1 -- 172.21.15.181:0/1902675869 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4006aa0 con 0x7f5db40c9910 2023-10-16T10:00:25.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.474+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db414ae40 con 0x7f5db40c9910 2023-10-16T10:00:25.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.475+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db40c9760 con 0x7f5db40c9910 2023-10-16T10:00:25.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.476+0000 7f5daaffd700 1 -- 172.21.15.181:0/1902675869 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5da4006c00 con 0x7f5db40c9910 2023-10-16T10:00:25.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.476+0000 7f5daaffd700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5d9806b020 0x7f5d9806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.476+0000 7f5daaffd700 1 -- 172.21.15.181:0/1902675869 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f5da40891b0 con 0x7f5db40c9910 2023-10-16T10:00:25.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.476+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5d9806b020 0x7f5d9806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.477+0000 7f5dba2d6700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5d9806b020 0x7f5d9806d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db0007000 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.481+0000 7f5daaffd700 1 -- 172.21.15.181:0/1902675869 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5da4053ff0 con 0x7f5db40c9910 2023-10-16T10:00:25.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.678+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f5db4067d20 con 0x7f5db40c9910 2023-10-16T10:00:25.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.681+0000 7f5daaffd700 1 -- 172.21.15.181:0/1902675869 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5da4057610 con 0x7f5db40c9910 2023-10-16T10:00:25.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5d9806b020 msgr2=0x7f5d9806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5d9806b020 0x7f5d9806d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db0007000 comp rx=0 tx=0).stop 2023-10-16T10:00:25.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5db40c9910 msgr2=0x7f5db40cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5db40c9910 0x7f5db40cab30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5da400f820 tx=0x7f5da400fbe0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 shutdown_connections 2023-10-16T10:00:25.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5d9806b020 0x7f5d9806d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5db40c9d90 0x7f5db40cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da0003a30 0x7f5db414a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 --2- 172.21.15.181:0/1902675869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5db40c9910 0x7f5db40cab30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 >> 172.21.15.181:0/1902675869 conn(0x7f5db409a930 msgr2=0x7f5db40cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:25.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 shutdown_connections 2023-10-16T10:00:25.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.684+0000 7f5dbc53a700 1 -- 172.21.15.181:0/1902675869 wait complete. 2023-10-16T10:00:25.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:00:25.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.819+0000 7efc3f2f9700 1 Processor -- start 2023-10-16T10:00:25.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.820+0000 7efc3f2f9700 1 -- start start 2023-10-16T10:00:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.820+0000 7efc3f2f9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc380b2010 0x7efc380cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.820+0000 7efc3f2f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc380ce680 0x7efc380ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.820+0000 7efc3f2f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc380ce1f0 con 0x7efc38071be0 2023-10-16T10:00:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.820+0000 7efc3f2f9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc38072090 con 0x7efc380b2010 2023-10-16T10:00:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.820+0000 7efc3f2f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380ccda0 con 0x7efc380ce680 2023-10-16T10:00:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d095700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc380b2010 0x7efc380cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3c894700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efc38071be0 0x7efc380b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55258/0 (socket says 172.21.15.181:55258) 2023-10-16T10:00:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3c894700 1 -- 172.21.15.181:0/3270101980 learned_addr learned my addr 172.21.15.181:0/3270101980 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d896700 1 --2- 172.21.15.181:0/3270101980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc380ce680 0x7efc380ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d095700 1 -- 172.21.15.181:0/3270101980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc380ce680 msgr2=0x7efc380ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d095700 1 --2- 172.21.15.181:0/3270101980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc380ce680 0x7efc380ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d095700 1 -- 172.21.15.181:0/3270101980 >> v1:172.21.15.181:6789/0 conn(0x7efc38071be0 legacy=0x7efc380b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d095700 1 -- 172.21.15.181:0/3270101980 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380727e0 con 0x7efc380b2010 2023-10-16T10:00:25.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc2ffff700 1 -- 172.21.15.181:0/3270101980 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976489897 0 0) 0x7efc380ce1f0 con 0x7efc38071be0 2023-10-16T10:00:25.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d896700 1 --2- 172.21.15.181:0/3270101980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc380ce680 0x7efc380ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:25.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc3d095700 1 --2- 172.21.15.181:0/3270101980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380b2010 0x7efc380cc6e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7efc28009840 tx=0x7efc28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b00333088414ced server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc2ffff700 1 -- 172.21.15.181:0/3270101980 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28011830 con 0x7efc380b2010 2023-10-16T10:00:25.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc2ffff700 1 -- 172.21.15.181:0/3270101980 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28011990 con 0x7efc380b2010 2023-10-16T10:00:25.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.821+0000 7efc2ffff700 1 -- 172.21.15.181:0/3270101980 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28011d20 con 0x7efc380b2010 2023-10-16T10:00:25.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3270101980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380b2010 msgr2=0x7efc380cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3270101980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380b2010 0x7efc380cc6e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7efc28009840 tx=0x7efc28009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:25.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3270101980 shutdown_connections 2023-10-16T10:00:25.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3270101980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc380ce680 0x7efc380ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3270101980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380b2010 0x7efc380cc6e0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3270101980 >> 172.21.15.181:0/3270101980 conn(0x7efc380cd010 msgr2=0x7efc380cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:25.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3270101980 shutdown_connections 2023-10-16T10:00:25.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.822+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3270101980 wait complete. 2023-10-16T10:00:25.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.823+0000 7efc3f2f9700 1 Processor -- start 2023-10-16T10:00:25.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.823+0000 7efc3f2f9700 1 -- start start 2023-10-16T10:00:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3f2f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc38071be0 0x7efc381f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3f2f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc380b2010 0x7efc380b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3f2f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380ce680 0x7efc3814f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3f2f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc38154390 con 0x7efc380ce680 2023-10-16T10:00:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3f2f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc38155770 con 0x7efc380b2010 2023-10-16T10:00:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3f2f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380b3310 con 0x7efc38071be0 2023-10-16T10:00:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380ce680 0x7efc3814f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380ce680 0x7efc3814f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44188/0 (socket says 172.21.15.181:44188) 2023-10-16T10:00:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 -- 172.21.15.181:0/3099146882 learned_addr learned my addr 172.21.15.181:0/3099146882 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:25.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 -- 172.21.15.181:0/3099146882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc38071be0 msgr2=0x7efc381f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc38071be0 0x7efc381f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 -- 172.21.15.181:0/3099146882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc380b2010 msgr2=0x7efc380b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3c894700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc380b2010 0x7efc380b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc380b2010 0x7efc380b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:25.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 -- 172.21.15.181:0/3099146882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280094a0 con 0x7efc380ce680 2023-10-16T10:00:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc3d896700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380ce680 0x7efc3814f580 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7efc3000cef0 tx=0x7efc3000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.824+0000 7efc2dffb700 1 -- 172.21.15.181:0/3099146882 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000ae50 con 0x7efc380ce680 2023-10-16T10:00:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.825+0000 7efc2dffb700 1 -- 172.21.15.181:0/3099146882 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc30015440 con 0x7efc380ce680 2023-10-16T10:00:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.825+0000 7efc2dffb700 1 -- 172.21.15.181:0/3099146882 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300157f0 con 0x7efc380ce680 2023-10-16T10:00:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.825+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc380b35b0 con 0x7efc380ce680 2023-10-16T10:00:25.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.825+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc380b5c40 con 0x7efc380ce680 2023-10-16T10:00:25.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.826+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc380b6510 con 0x7efc380ce680 2023-10-16T10:00:25.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.826+0000 7efc2dffb700 1 -- 172.21.15.181:0/3099146882 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efc3000d430 con 0x7efc380ce680 2023-10-16T10:00:25.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.826+0000 7efc2dffb700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7efc1c06b020 0x7efc1c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:25.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.826+0000 7efc2dffb700 1 -- 172.21.15.181:0/3099146882 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7efc30089f70 con 0x7efc380ce680 2023-10-16T10:00:25.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.826+0000 7efc3d095700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7efc1c06b020 0x7efc1c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:25.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.827+0000 7efc3d095700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7efc1c06b020 0x7efc1c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc28000c00 tx=0x7efc28010000 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:25.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:25.831+0000 7efc2dffb700 1 -- 172.21.15.181:0/3099146882 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efc30054d10 con 0x7efc380ce680 2023-10-16T10:00:26.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.025+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efc381f4fa0 con 0x7efc380ce680 2023-10-16T10:00:26.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.028+0000 7efc2dffb700 1 -- 172.21.15.181:0/3099146882 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7efc3001a0a0 con 0x7efc380ce680 2023-10-16T10:00:26.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7efc1c06b020 msgr2=0x7efc1c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7efc1c06b020 0x7efc1c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc28000c00 tx=0x7efc28010000 comp rx=0 tx=0).stop 2023-10-16T10:00:26.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380ce680 msgr2=0x7efc3814f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380ce680 0x7efc3814f580 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7efc3000cef0 tx=0x7efc3000a340 comp rx=0 tx=0).stop 2023-10-16T10:00:26.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 shutdown_connections 2023-10-16T10:00:26.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7efc1c06b020 0x7efc1c06d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc38071be0 0x7efc381f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc380ce680 0x7efc3814f580 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 --2- 172.21.15.181:0/3099146882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc380b2010 0x7efc380b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 >> 172.21.15.181:0/3099146882 conn(0x7efc380cd010 msgr2=0x7efc380c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 shutdown_connections 2023-10-16T10:00:26.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.031+0000 7efc3f2f9700 1 -- 172.21.15.181:0/3099146882 wait complete. 2023-10-16T10:00:26.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-16T10:00:26.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.163+0000 7f181ee32700 1 Processor -- start 2023-10-16T10:00:26.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.163+0000 7f181ee32700 1 -- start start 2023-10-16T10:00:26.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181ee32700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f181815e6d0 0x7f18180d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:26.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181ee32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18180cb0c0 0x7f181815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:26.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181ee32700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181806ed00 con 0x7f181815eab0 2023-10-16T10:00:26.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181ee32700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18180d02d0 con 0x7f18180cb0c0 2023-10-16T10:00:26.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181ee32700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181815ad40 con 0x7f181815e6d0 2023-10-16T10:00:26.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181d3cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18180cb0c0 0x7f181815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:26.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181d3cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18180cb0c0 0x7f181815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44196/0 (socket says 172.21.15.181:44196) 2023-10-16T10:00:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.164+0000 7f181d3cf700 1 -- 172.21.15.181:0/3924647204 learned_addr learned my addr 172.21.15.181:0/3924647204 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f181cbce700 1 --2- 172.21.15.181:0/3924647204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f181815e6d0 0x7f18180d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f181d3cf700 1 -- 172.21.15.181:0/3924647204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f181815e6d0 msgr2=0x7f18180d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f181d3cf700 1 --2- 172.21.15.181:0/3924647204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f181815e6d0 0x7f18180d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f181d3cf700 1 -- 172.21.15.181:0/3924647204 >> v1:172.21.15.190:6789/0 conn(0x7f181815eab0 legacy=0x7f181815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f181d3cf700 1 -- 172.21.15.181:0/3924647204 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181806e0f0 con 0x7f18180cb0c0 2023-10-16T10:00:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f181d3cf700 1 --2- 172.21.15.181:0/3924647204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18180cb0c0 0x7f181815a800 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1810009e50 tx=0x7f181000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c45e3adefb73f8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f18177fe700 1 -- 172.21.15.181:0/3924647204 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181000e070 con 0x7f18180cb0c0 2023-10-16T10:00:26.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f18177fe700 1 -- 172.21.15.181:0/3924647204 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181000b910 con 0x7f18180cb0c0 2023-10-16T10:00:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.165+0000 7f18177fe700 1 -- 172.21.15.181:0/3924647204 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181000bcc0 con 0x7f18180cb0c0 2023-10-16T10:00:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 -- 172.21.15.181:0/3924647204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18180cb0c0 msgr2=0x7f181815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 --2- 172.21.15.181:0/3924647204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18180cb0c0 0x7f181815a800 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1810009e50 tx=0x7f181000b330 comp rx=0 tx=0).stop 2023-10-16T10:00:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 -- 172.21.15.181:0/3924647204 shutdown_connections 2023-10-16T10:00:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 --2- 172.21.15.181:0/3924647204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18180cb0c0 0x7f181815a800 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 --2- 172.21.15.181:0/3924647204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f181815e6d0 0x7f18180d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 -- 172.21.15.181:0/3924647204 >> 172.21.15.181:0/3924647204 conn(0x7f18180d04a0 msgr2=0x7f18180d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 -- 172.21.15.181:0/3924647204 shutdown_connections 2023-10-16T10:00:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.166+0000 7f181ee32700 1 -- 172.21.15.181:0/3924647204 wait complete. 2023-10-16T10:00:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.167+0000 7f181ee32700 1 Processor -- start 2023-10-16T10:00:26.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.167+0000 7f181ee32700 1 -- start start 2023-10-16T10:00:26.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181ee32700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18180cb0c0 0x7f1818154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:26.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181ee32700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f181815e6d0 0x7f181814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181ee32700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181815eab0 0x7f181814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181ee32700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b60a0 con 0x7f181815eab0 2023-10-16T10:00:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181ee32700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b5420 con 0x7f181815e6d0 2023-10-16T10:00:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181ee32700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b5700 con 0x7f18180cb0c0 2023-10-16T10:00:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181815eab0 0x7f181814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181815eab0 0x7f181814caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44202/0 (socket says 172.21.15.181:44202) 2023-10-16T10:00:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 -- 172.21.15.181:0/2223534038 learned_addr learned my addr 172.21.15.181:0/2223534038 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181cbce700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18180cb0c0 0x7f1818154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f1817fff700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f181815e6d0 0x7f181814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 -- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18180cb0c0 msgr2=0x7f1818154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18180cb0c0 0x7f1818154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 -- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f181815e6d0 msgr2=0x7f181814c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f181815e6d0 0x7f181814c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 -- 172.21.15.181:0/2223534038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1810009b30 con 0x7f181815eab0 2023-10-16T10:00:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f1817fff700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f181815e6d0 0x7f181814c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181d3cf700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181815eab0 0x7f181814caf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1818071580 tx=0x7f1810004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f18157fa700 1 -- 172.21.15.181:0/2223534038 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100048b0 con 0x7f181815eab0 2023-10-16T10:00:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f18157fa700 1 -- 172.21.15.181:0/2223534038 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18100099b0 con 0x7f181815eab0 2023-10-16T10:00:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18180b59e0 con 0x7f181815eab0 2023-10-16T10:00:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.168+0000 7f18157fa700 1 -- 172.21.15.181:0/2223534038 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181001b670 con 0x7f181815eab0 2023-10-16T10:00:26.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.169+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181814d030 con 0x7f181815eab0 2023-10-16T10:00:26.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.169+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181814aa00 con 0x7f181815eab0 2023-10-16T10:00:26.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.170+0000 7f18157fa700 1 -- 172.21.15.181:0/2223534038 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f181001b7d0 con 0x7f181815eab0 2023-10-16T10:00:26.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.170+0000 7f18157fa700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f17fc06b020 0x7f17fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:26.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.171+0000 7f18157fa700 1 -- 172.21.15.181:0/2223534038 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f181008b5a0 con 0x7f181815eab0 2023-10-16T10:00:26.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.171+0000 7f181cbce700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f17fc06b020 0x7f17fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:26.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.171+0000 7f181cbce700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f17fc06b020 0x7f17fc06d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18181556b0 tx=0x7f180c0091c0 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:26.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.175+0000 7f18157fa700 1 -- 172.21.15.181:0/2223534038 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1810056560 con 0x7f181815eab0 2023-10-16T10:00:26.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.449+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f1818067d20 con 0x7f181815eab0 2023-10-16T10:00:26.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.973+0000 7f18157fa700 1 -- 172.21.15.181:0/2223534038 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v609) v1 ==== 166+0+0 (secure 0 0 0) 0x7f1810059b80 con 0x7f181815eab0 2023-10-16T10:00:26.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f17fc06b020 msgr2=0x7f17fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f17fc06b020 0x7f17fc06d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18181556b0 tx=0x7f180c0091c0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181815eab0 msgr2=0x7f181814caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:26.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181815eab0 0x7f181814caf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1818071580 tx=0x7f1810004ad0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 shutdown_connections 2023-10-16T10:00:26.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f17fc06b020 0x7f17fc06d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18180cb0c0 0x7f1818154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181815eab0 0x7f181814caf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 --2- 172.21.15.181:0/2223534038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f181815e6d0 0x7f181814c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:26.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 >> 172.21.15.181:0/2223534038 conn(0x7f18180d04a0 msgr2=0x7f18180ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:26.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 shutdown_connections 2023-10-16T10:00:26.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:26.976+0000 7f181ee32700 1 -- 172.21.15.181:0/2223534038 wait complete. 2023-10-16T10:00:26.993 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-16T10:00:26.993 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-16T10:00:26.994 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:00:26.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:27.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.119+0000 7fc9aeee8700 1 Processor -- start 2023-10-16T10:00:27.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.119+0000 7fc9aeee8700 1 -- start start 2023-10-16T10:00:27.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9aeee8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9a815dc40 0x7fc9a806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9aeee8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a80c2dc0 0x7fc9a815e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9aeee8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80b2710 con 0x7fc9a806ade0 2023-10-16T10:00:27.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9aeee8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a80c18f0 con 0x7fc9a815dc40 2023-10-16T10:00:27.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9aeee8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80c1bd0 con 0x7fc9a80c2dc0 2023-10-16T10:00:27.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9acc84700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9a815dc40 0x7fc9a806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9acc84700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9a815dc40 0x7fc9a806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44206/0 (socket says 172.21.15.181:44206) 2023-10-16T10:00:27.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9a7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc9a806ade0 0x7fc9a80c26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55284/0 (socket says 172.21.15.181:55284) 2023-10-16T10:00:27.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9acc84700 1 -- 172.21.15.181:0/1074359349 learned_addr learned my addr 172.21.15.181:0/1074359349 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:27.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.120+0000 7fc9ad485700 1 --2- 172.21.15.181:0/1074359349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a80c2dc0 0x7fc9a815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9acc84700 1 -- 172.21.15.181:0/1074359349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a80c2dc0 msgr2=0x7fc9a815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:27.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9acc84700 1 --2- 172.21.15.181:0/1074359349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a80c2dc0 0x7fc9a815e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9acc84700 1 -- 172.21.15.181:0/1074359349 >> v1:172.21.15.181:6789/0 conn(0x7fc9a806ade0 legacy=0x7fc9a80c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9acc84700 1 -- 172.21.15.181:0/1074359349 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a80c1eb0 con 0x7fc9a815dc40 2023-10-16T10:00:27.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9a77fe700 1 -- 172.21.15.181:0/1074359349 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424706578 0 0) 0x7fc994001110 con 0x7fc9a806ade0 2023-10-16T10:00:27.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9ad485700 1 --2- 172.21.15.181:0/1074359349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a80c2dc0 0x7fc9a815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:27.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9acc84700 1 --2- 172.21.15.181:0/1074359349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a806a8a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0098f0 tx=0x7fc99c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b44acf3c65ca138 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9a77fe700 1 -- 172.21.15.181:0/1074359349 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00b8a0 con 0x7fc9a815dc40 2023-10-16T10:00:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9a77fe700 1 -- 172.21.15.181:0/1074359349 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c00ba00 con 0x7fc9a815dc40 2023-10-16T10:00:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9a77fe700 1 -- 172.21.15.181:0/1074359349 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00bd90 con 0x7fc9a815dc40 2023-10-16T10:00:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9aeee8700 1 -- 172.21.15.181:0/1074359349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 msgr2=0x7fc9a806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/1074359349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a806a8a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0098f0 tx=0x7fc99c009c00 comp rx=0 tx=0).stop 2023-10-16T10:00:27.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9aeee8700 1 -- 172.21.15.181:0/1074359349 shutdown_connections 2023-10-16T10:00:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/1074359349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a80c2dc0 0x7fc9a815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/1074359349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a806a8a0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.121+0000 7fc9aeee8700 1 -- 172.21.15.181:0/1074359349 >> 172.21.15.181:0/1074359349 conn(0x7fc9a80b1780 msgr2=0x7fc9a80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.122+0000 7fc9aeee8700 1 -- 172.21.15.181:0/1074359349 shutdown_connections 2023-10-16T10:00:27.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.122+0000 7fc9aeee8700 1 -- 172.21.15.181:0/1074359349 wait complete. 2023-10-16T10:00:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.123+0000 7fc9aeee8700 1 Processor -- start 2023-10-16T10:00:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.123+0000 7fc9aeee8700 1 -- start start 2023-10-16T10:00:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.123+0000 7fc9aeee8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a806ade0 0x7fc9a815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9aeee8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9a80c2dc0 0x7fc9a80b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9aeee8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a809faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9aeee8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a809ef40 con 0x7fc9a815dc40 2023-10-16T10:00:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9aeee8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a815ba40 con 0x7fc9a80c2dc0 2023-10-16T10:00:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9aeee8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a815c8a0 con 0x7fc9a806ade0 2023-10-16T10:00:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a809faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a809faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44216/0 (socket says 172.21.15.181:44216) 2023-10-16T10:00:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 -- 172.21.15.181:0/3169506851 learned_addr learned my addr 172.21.15.181:0/3169506851 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9a7fff700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9a80c2dc0 0x7fc9a80b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9acc84700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a806ade0 0x7fc9a815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 -- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a806ade0 msgr2=0x7fc9a815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a806ade0 0x7fc9a815cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 -- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9a80c2dc0 msgr2=0x7fc9a80b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9a80c2dc0 0x7fc9a80b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 -- 172.21.15.181:0/3169506851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c009550 con 0x7fc9a815dc40 2023-10-16T10:00:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9ad485700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a809faf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0007ef0 tx=0x7fc9a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9a57fa700 1 -- 172.21.15.181:0/3169506851 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a000de60 con 0x7fc9a815dc40 2023-10-16T10:00:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9a57fa700 1 -- 172.21.15.181:0/3169506851 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a000f040 con 0x7fc9a815dc40 2023-10-16T10:00:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9a57fa700 1 -- 172.21.15.181:0/3169506851 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0003cb0 con 0x7fc9a815dc40 2023-10-16T10:00:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a815d270 con 0x7fc9a815dc40 2023-10-16T10:00:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.124+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a815bd40 con 0x7fc9a815dc40 2023-10-16T10:00:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.125+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a815ca30 con 0x7fc9a815dc40 2023-10-16T10:00:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.126+0000 7fc9a57fa700 1 -- 172.21.15.181:0/3169506851 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc9a0004030 con 0x7fc9a815dc40 2023-10-16T10:00:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.126+0000 7fc9a57fa700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fc98c06b020 0x7fc98c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.126+0000 7fc9a57fa700 1 -- 172.21.15.181:0/3169506851 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fc9a0089b30 con 0x7fc9a815dc40 2023-10-16T10:00:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.126+0000 7fc9acc84700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fc98c06b020 0x7fc98c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.127+0000 7fc9acc84700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fc98c06b020 0x7fc98c06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc99c000c00 tx=0x7fc99c008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.131+0000 7fc9a57fa700 1 -- 172.21.15.181:0/3169506851 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc9a0054860 con 0x7fc9a815dc40 2023-10-16T10:00:27.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.325+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9a80cc070 con 0x7fc9a815dc40 2023-10-16T10:00:27.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.326+0000 7fc9a57fa700 1 -- 172.21.15.181:0/3169506851 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+14682 (secure 0 0 0) 0x7fc9a001a020 con 0x7fc9a815dc40 2023-10-16T10:00:27.328 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:27.328 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":95,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:24.970459+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/2780127849":"2023-10-17T10:00:21.683265+0000","172.21.15.181:6835/2780127849":"2023-10-17T10:00:21.683265+0000","172.21.15.181:0/591701176":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/3458983616":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/395530113":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/3470208380":"2023-10-17T10:00:20.650463+0000","172.21.15.181:6833/2464567114":"2023-10-17T10:00:20.650463+0000","172.21.15.181:6832/2464567114":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/1686162648":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/299388856":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/315145191":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/2235576983":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/3806937704":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/849326393":"2023-10-17T10:00:20.650463+0000","172.21.15.181:0/4212605490":"2023-10-17T10:00:13.202216+0000","172.21.15.181:0/3680374730":"2023-10-17T10:00:20.650463+0000","172.21.15.181:6833/752673838":"2023-10-17T10:00:13.202216+0000","172.21.15.181:6832/752673838":"2023-10-17T10:00:13.202216+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.328+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fc98c06b020 msgr2=0x7fc98c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.328+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fc98c06b020 0x7fc98c06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc99c000c00 tx=0x7fc99c008030 comp rx=0 tx=0).stop 2023-10-16T10:00:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.328+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 msgr2=0x7fc9a809faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.328+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a809faf0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0007ef0 tx=0x7fc9a000d3b0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 shutdown_connections 2023-10-16T10:00:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fc98c06b020 0x7fc98c06d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9a806ade0 0x7fc9a815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9a815dc40 0x7fc9a809faf0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 --2- 172.21.15.181:0/3169506851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9a80c2dc0 0x7fc9a80b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 >> 172.21.15.181:0/3169506851 conn(0x7fc9a80b1780 msgr2=0x7fc9a80af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 shutdown_connections 2023-10-16T10:00:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.329+0000 7fc9aeee8700 1 -- 172.21.15.181:0/3169506851 wait complete. 2023-10-16T10:00:27.342 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:00:27.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:00:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.464+0000 7f48b4b0a700 1 Processor -- start 2023-10-16T10:00:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.464+0000 7f48b4b0a700 1 -- start start 2023-10-16T10:00:27.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48b4b0a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48b015f150 0x7f48b00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48b4b0a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48b4b0a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b00b2ae0 con 0x7f48b00b14a0 2023-10-16T10:00:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48b4b0a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b00ce970 con 0x7f48b015f150 2023-10-16T10:00:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48b4b0a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00b2dc0 con 0x7f48b00ce4f0 2023-10-16T10:00:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48ae59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48b015f150 0x7f48b00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48ae59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48b015f150 0x7f48b00cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44220/0 (socket says 172.21.15.181:44220) 2023-10-16T10:00:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48ae59c700 1 -- 172.21.15.181:0/774519369 learned_addr learned my addr 172.21.15.181:0/774519369 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.465+0000 7f48add9b700 1 --2- 172.21.15.181:0/774519369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ae59c700 1 -- 172.21.15.181:0/774519369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 msgr2=0x7f48b00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ae59c700 1 --2- 172.21.15.181:0/774519369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ae59c700 1 -- 172.21.15.181:0/774519369 >> v1:172.21.15.190:6789/0 conn(0x7f48b00b14a0 legacy=0x7f48b00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ae59c700 1 -- 172.21.15.181:0/774519369 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00cf9d0 con 0x7f48b015f150 2023-10-16T10:00:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ae59c700 1 --2- 172.21.15.181:0/774519369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00cdfb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f48a40098f0 tx=0x7f48a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d428bd2a2f2589 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ad59a700 1 -- 172.21.15.181:0/774519369 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4011830 con 0x7f48b015f150 2023-10-16T10:00:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ad59a700 1 -- 172.21.15.181:0/774519369 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a4011990 con 0x7f48b015f150 2023-10-16T10:00:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48ad59a700 1 -- 172.21.15.181:0/774519369 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4011d20 con 0x7f48b015f150 2023-10-16T10:00:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48b4b0a700 1 -- 172.21.15.181:0/774519369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 msgr2=0x7f48b00cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.466+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/774519369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00cdfb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f48a40098f0 tx=0x7f48a4009c00 comp rx=0 tx=0).stop 2023-10-16T10:00:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.467+0000 7f48b4b0a700 1 -- 172.21.15.181:0/774519369 shutdown_connections 2023-10-16T10:00:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.467+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/774519369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00cdfb0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.467+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/774519369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.467+0000 7f48b4b0a700 1 -- 172.21.15.181:0/774519369 >> 172.21.15.181:0/774519369 conn(0x7f48b009a930 msgr2=0x7f48b00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.467+0000 7f48b4b0a700 1 -- 172.21.15.181:0/774519369 shutdown_connections 2023-10-16T10:00:27.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.467+0000 7f48b4b0a700 1 -- 172.21.15.181:0/774519369 wait complete. 2023-10-16T10:00:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.468+0000 7f48b4b0a700 1 Processor -- start 2023-10-16T10:00:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.468+0000 7f48b4b0a700 1 -- start start 2023-10-16T10:00:27.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48b4b0a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b00b14a0 0x7f48b015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48b4b0a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48b4b0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48b4b0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b015dc50 con 0x7f48b015f150 2023-10-16T10:00:27.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48b4b0a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b015dea0 con 0x7f48b00b14a0 2023-10-16T10:00:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48b4b0a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b015e180 con 0x7f48b00ce4f0 2023-10-16T10:00:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44230/0 (socket says 172.21.15.181:44230) 2023-10-16T10:00:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 -- 172.21.15.181:0/1152489293 learned_addr learned my addr 172.21.15.181:0/1152489293 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:27.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48add9b700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48ae59c700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b00b14a0 0x7f48b015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 -- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 msgr2=0x7f48b00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 -- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b00b14a0 msgr2=0x7f48b015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b00b14a0 0x7f48b015ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 -- 172.21.15.181:0/1152489293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4009550 con 0x7f48b015f150 2023-10-16T10:00:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f48aed9d700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00c9620 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f48a800cc50 tx=0x7f48a800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.469+0000 7f489effd700 1 -- 172.21.15.181:0/1152489293 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a800ad80 con 0x7f48b015f150 2023-10-16T10:00:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.470+0000 7f489effd700 1 -- 172.21.15.181:0/1152489293 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a800ce50 con 0x7f48b015f150 2023-10-16T10:00:27.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.470+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b015e460 con 0x7f48b015f150 2023-10-16T10:00:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.470+0000 7f489effd700 1 -- 172.21.15.181:0/1152489293 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8003c40 con 0x7f48b015f150 2023-10-16T10:00:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.470+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b015cf70 con 0x7f48b015f150 2023-10-16T10:00:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.470+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0067d20 con 0x7f48b015f150 2023-10-16T10:00:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.471+0000 7f489effd700 1 -- 172.21.15.181:0/1152489293 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f48a80050d0 con 0x7f48b015f150 2023-10-16T10:00:27.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.471+0000 7f489effd700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f489006b020 0x7f489006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.472+0000 7f489effd700 1 -- 172.21.15.181:0/1152489293 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f48a8089990 con 0x7f48b015f150 2023-10-16T10:00:27.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.472+0000 7f48ae59c700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f489006b020 0x7f489006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.472+0000 7f48ae59c700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f489006b020 0x7f489006d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48a4014040 tx=0x7f48a4008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.476+0000 7f489effd700 1 -- 172.21.15.181:0/1152489293 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f48a80547d0 con 0x7f48b015f150 2023-10-16T10:00:27.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.668+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48b00c8ec0 con 0x7f48b015f150 2023-10-16T10:00:27.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.668+0000 7f489effd700 1 -- 172.21.15.181:0/1152489293 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48a8057df0 con 0x7f48b015f150 2023-10-16T10:00:27.670 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:00:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f489006b020 msgr2=0x7f489006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f489006b020 0x7f489006d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48a4014040 tx=0x7f48a4008030 comp rx=0 tx=0).stop 2023-10-16T10:00:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 msgr2=0x7f48b00c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00c9620 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f48a800cc50 tx=0x7f48a800a340 comp rx=0 tx=0).stop 2023-10-16T10:00:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 shutdown_connections 2023-10-16T10:00:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f489006b020 0x7f489006d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b015f150 0x7f48b00c9620 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48b00b14a0 0x7f48b015ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 --2- 172.21.15.181:0/1152489293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48b00ce4f0 0x7f48b00c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 >> 172.21.15.181:0/1152489293 conn(0x7f48b009a930 msgr2=0x7f48b015ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 shutdown_connections 2023-10-16T10:00:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.670+0000 7f48b4b0a700 1 -- 172.21.15.181:0/1152489293 wait complete. 2023-10-16T10:00:27.684 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-16T10:00:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.804+0000 7f415ecf0700 1 Processor -- start 2023-10-16T10:00:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.804+0000 7f415ecf0700 1 -- start start 2023-10-16T10:00:27.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415ecf0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415ecf0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41580cb0c0 0x7f415815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415ecf0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415806ed00 con 0x7f415815e6d0 2023-10-16T10:00:27.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415ecf0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41580d02d0 con 0x7f41580cb0c0 2023-10-16T10:00:27.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415ecf0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415815ad40 con 0x7f415815eab0 2023-10-16T10:00:27.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41580cb0c0 0x7f415815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41580cb0c0 0x7f415815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44238/0 (socket says 172.21.15.181:44238) 2023-10-16T10:00:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 -- 172.21.15.181:0/2395054753 learned_addr learned my addr 172.21.15.181:0/2395054753 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415ca8c700 1 --1- 172.21.15.181:0/2395054753 >> v1:172.21.15.181:6789/0 conn(0x7f415815e6d0 0x7f415815a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55312/0 (socket says 172.21.15.181:55312) 2023-10-16T10:00:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f4157fff700 1 --2- 172.21.15.181:0/2395054753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 -- 172.21.15.181:0/2395054753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 msgr2=0x7f41580d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:27.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 --2- 172.21.15.181:0/2395054753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 -- 172.21.15.181:0/2395054753 >> v1:172.21.15.181:6789/0 conn(0x7f415815e6d0 legacy=0x7f415815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f41577fe700 1 -- 172.21.15.181:0/2395054753 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284628445 0 0) 0x7f415806ed00 con 0x7f415815e6d0 2023-10-16T10:00:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 -- 172.21.15.181:0/2395054753 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415806e0f0 con 0x7f41580cb0c0 2023-10-16T10:00:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.805+0000 7f415d28d700 1 --2- 172.21.15.181:0/2395054753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41580cb0c0 0x7f415815a800 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f415000d150 tx=0x7f415000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35525b6c8ed0593 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f41577fe700 1 -- 172.21.15.181:0/2395054753 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150010070 con 0x7f41580cb0c0 2023-10-16T10:00:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f41577fe700 1 -- 172.21.15.181:0/2395054753 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415000dd10 con 0x7f41580cb0c0 2023-10-16T10:00:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f41577fe700 1 -- 172.21.15.181:0/2395054753 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415000b310 con 0x7f41580cb0c0 2023-10-16T10:00:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 -- 172.21.15.181:0/2395054753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41580cb0c0 msgr2=0x7f415815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 --2- 172.21.15.181:0/2395054753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41580cb0c0 0x7f415815a800 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f415000d150 tx=0x7f415000d460 comp rx=0 tx=0).stop 2023-10-16T10:00:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 -- 172.21.15.181:0/2395054753 shutdown_connections 2023-10-16T10:00:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 --2- 172.21.15.181:0/2395054753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41580cb0c0 0x7f415815a800 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 --2- 172.21.15.181:0/2395054753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 -- 172.21.15.181:0/2395054753 >> 172.21.15.181:0/2395054753 conn(0x7f41580d04a0 msgr2=0x7f41580d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 -- 172.21.15.181:0/2395054753 shutdown_connections 2023-10-16T10:00:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.806+0000 7f415ecf0700 1 -- 172.21.15.181:0/2395054753 wait complete. 2023-10-16T10:00:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.807+0000 7f415ecf0700 1 Processor -- start 2023-10-16T10:00:27.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.807+0000 7f415ecf0700 1 -- start start 2023-10-16T10:00:27.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f415ecf0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41580cb0c0 0x7f415814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f415ecf0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415815e6d0 0x7f415814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f415ecf0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f415ecf0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415814e270 con 0x7f415815e6d0 2023-10-16T10:00:27.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f415ecf0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41580ca020 con 0x7f41580cb0c0 2023-10-16T10:00:27.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f415ecf0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41580c94e0 con 0x7f415815eab0 2023-10-16T10:00:27.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f4157fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415815e6d0 0x7f415814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.808+0000 7f4157fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415815e6d0 0x7f415814dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44242/0 (socket says 172.21.15.181:44242) 2023-10-16T10:00:27.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f4157fff700 1 -- 172.21.15.181:0/1678642434 learned_addr learned my addr 172.21.15.181:0/1678642434 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f415ca8c700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41580cb0c0 0x7f415814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f415d28d700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f4157fff700 1 -- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 msgr2=0x7f41580c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f4157fff700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f4157fff700 1 -- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41580cb0c0 msgr2=0x7f415814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f4157fff700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41580cb0c0 0x7f415814e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f4157fff700 1 -- 172.21.15.181:0/1678642434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415000bd40 con 0x7f415815e6d0 2023-10-16T10:00:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f415ca8c700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41580cb0c0 0x7f415814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f415d28d700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f4157fff700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415815e6d0 0x7f415814dd30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f414400e9b0 tx=0x7f414400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f41557fa700 1 -- 172.21.15.181:0/1678642434 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144009710 con 0x7f415815e6d0 2023-10-16T10:00:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f41557fa700 1 -- 172.21.15.181:0/1678642434 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414400c850 con 0x7f415815e6d0 2023-10-16T10:00:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41580c97c0 con 0x7f415815e6d0 2023-10-16T10:00:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f41557fa700 1 -- 172.21.15.181:0/1678642434 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400cbc0 con 0x7f415815e6d0 2023-10-16T10:00:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.809+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41580c6d50 con 0x7f415815e6d0 2023-10-16T10:00:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.810+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41580c9950 con 0x7f415815e6d0 2023-10-16T10:00:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.811+0000 7f41557fa700 1 -- 172.21.15.181:0/1678642434 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f414400cd20 con 0x7f415815e6d0 2023-10-16T10:00:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.811+0000 7f41557fa700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f413c06b020 0x7f413c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.811+0000 7f41557fa700 1 -- 172.21.15.181:0/1678642434 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f4144089a10 con 0x7f415815e6d0 2023-10-16T10:00:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.811+0000 7f415ca8c700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f413c06b020 0x7f413c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.811+0000 7f415ca8c700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f413c06b020 0x7f413c06d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41581f2370 tx=0x7f414c009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:27.815+0000 7f41557fa700 1 -- 172.21.15.181:0/1678642434 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4144054850 con 0x7f415815e6d0 2023-10-16T10:00:28.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.089+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f4158067d20 con 0x7f415815e6d0 2023-10-16T10:00:28.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.980+0000 7f41557fa700 1 -- 172.21.15.181:0/1678642434 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v611) v1 ==== 186+0+0 (secure 0 0 0) 0x7f4144021100 con 0x7f415815e6d0 2023-10-16T10:00:28.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f413c06b020 msgr2=0x7f413c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:28.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f413c06b020 0x7f413c06d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41581f2370 tx=0x7f414c009040 comp rx=0 tx=0).stop 2023-10-16T10:00:28.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415815e6d0 msgr2=0x7f415814dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:28.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415815e6d0 0x7f415814dd30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f414400e9b0 tx=0x7f414400ed70 comp rx=0 tx=0).stop 2023-10-16T10:00:28.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 shutdown_connections 2023-10-16T10:00:28.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f413c06b020 0x7f413c06d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:28.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f415815eab0 0x7f41580c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:28.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41580cb0c0 0x7f415814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:28.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 --2- 172.21.15.181:0/1678642434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415815e6d0 0x7f415814dd30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:28.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 >> 172.21.15.181:0/1678642434 conn(0x7f41580d04a0 msgr2=0x7f4158158220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:28.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 shutdown_connections 2023-10-16T10:00:28.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:28.984+0000 7f415ecf0700 1 -- 172.21.15.181:0/1678642434 wait complete. 2023-10-16T10:00:29.001 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:00:29.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.129+0000 7faa26b67700 1 Processor -- start 2023-10-16T10:00:29.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.129+0000 7faa26b67700 1 -- start start 2023-10-16T10:00:29.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa26b67700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faa20072760 0x7faa2006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa26b67700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa2006ade0 0x7faa201615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa26b67700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2006b210 con 0x7faa2015da70 2023-10-16T10:00:29.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa26b67700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200cef00 con 0x7faa2006ade0 2023-10-16T10:00:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa26b67700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200cec20 con 0x7faa20072760 2023-10-16T10:00:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa2006ade0 0x7faa201615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa2006ade0 0x7faa201615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44252/0 (socket says 172.21.15.181:44252) 2023-10-16T10:00:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 -- 172.21.15.181:0/1030309552 learned_addr learned my addr 172.21.15.181:0/1030309552 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:29.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa1ffff700 1 --2- 172.21.15.181:0/1030309552 >> v2:172.21.15.190:3300/0 conn(0x7faa20072760 0x7faa2006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 -- 172.21.15.181:0/1030309552 >> v2:172.21.15.190:3300/0 conn(0x7faa20072760 msgr2=0x7faa2006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 --2- 172.21.15.181:0/1030309552 >> v2:172.21.15.190:3300/0 conn(0x7faa20072760 0x7faa2006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 -- 172.21.15.181:0/1030309552 >> v1:172.21.15.190:6789/0 conn(0x7faa2015da70 legacy=0x7faa200ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 -- 172.21.15.181:0/1030309552 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20071c80 con 0x7faa2006ade0 2023-10-16T10:00:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa1ffff700 1 --2- 172.21.15.181:0/1030309552 >> v2:172.21.15.190:3300/0 conn(0x7faa20072760 0x7faa2006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa25104700 1 --2- 172.21.15.181:0/1030309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2006ade0 0x7faa201615f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7faa2006ef60 tx=0x7faa1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699950c37aa71793 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.130+0000 7faa1f7fe700 1 -- 172.21.15.181:0/1030309552 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800e070 con 0x7faa2006ade0 2023-10-16T10:00:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa1f7fe700 1 -- 172.21.15.181:0/1030309552 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa18003a20 con 0x7faa2006ade0 2023-10-16T10:00:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa1f7fe700 1 -- 172.21.15.181:0/1030309552 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18003dd0 con 0x7faa2006ade0 2023-10-16T10:00:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 -- 172.21.15.181:0/1030309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2006ade0 msgr2=0x7faa201615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 --2- 172.21.15.181:0/1030309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2006ade0 0x7faa201615f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7faa2006ef60 tx=0x7faa1800b330 comp rx=0 tx=0).stop 2023-10-16T10:00:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 -- 172.21.15.181:0/1030309552 shutdown_connections 2023-10-16T10:00:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 --2- 172.21.15.181:0/1030309552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2006ade0 0x7faa201615f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 --2- 172.21.15.181:0/1030309552 >> v2:172.21.15.190:3300/0 conn(0x7faa20072760 0x7faa2006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 -- 172.21.15.181:0/1030309552 >> 172.21.15.181:0/1030309552 conn(0x7faa200cd010 msgr2=0x7faa200cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:29.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 -- 172.21.15.181:0/1030309552 shutdown_connections 2023-10-16T10:00:29.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.131+0000 7faa26b67700 1 -- 172.21.15.181:0/1030309552 wait complete. 2023-10-16T10:00:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.133+0000 7faa26b67700 1 Processor -- start 2023-10-16T10:00:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.133+0000 7faa26b67700 1 -- start start 2023-10-16T10:00:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa26b67700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa2006ade0 0x7faa200af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa26b67700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20072760 0x7faa201f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa26b67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2015da70 0x7faa201607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa26b67700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200adc10 con 0x7faa2015da70 2023-10-16T10:00:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa26b67700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201f6ac0 con 0x7faa20072760 2023-10-16T10:00:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa26b67700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa201f40a0 con 0x7faa2006ade0 2023-10-16T10:00:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2015da70 0x7faa201607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2015da70 0x7faa201607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44258/0 (socket says 172.21.15.181:44258) 2023-10-16T10:00:29.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 -- 172.21.15.181:0/427633671 learned_addr learned my addr 172.21.15.181:0/427633671 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:29.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa1ffff700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20072760 0x7faa201f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa24903700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa2006ade0 0x7faa200af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 -- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa2006ade0 msgr2=0x7faa200af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa2006ade0 0x7faa200af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 -- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20072760 msgr2=0x7faa201f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20072760 0x7faa201f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 -- 172.21.15.181:0/427633671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18009b30 con 0x7faa2015da70 2023-10-16T10:00:29.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa24903700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa2006ade0 0x7faa200af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:29.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa25104700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2015da70 0x7faa201607d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7faa2006b3a0 tx=0x7faa18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.134+0000 7faa1d7fa700 1 -- 172.21.15.181:0/427633671 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18004440 con 0x7faa2015da70 2023-10-16T10:00:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.135+0000 7faa1d7fa700 1 -- 172.21.15.181:0/427633671 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa180045a0 con 0x7faa2015da70 2023-10-16T10:00:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.135+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa20160d10 con 0x7faa2015da70 2023-10-16T10:00:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.135+0000 7faa1d7fa700 1 -- 172.21.15.181:0/427633671 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18004950 con 0x7faa2015da70 2023-10-16T10:00:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.135+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa201f72b0 con 0x7faa2015da70 2023-10-16T10:00:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.135+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa200c7120 con 0x7faa2015da70 2023-10-16T10:00:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.136+0000 7faa1d7fa700 1 -- 172.21.15.181:0/427633671 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa18012430 con 0x7faa2015da70 2023-10-16T10:00:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.136+0000 7faa1d7fa700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7faa0406b020 0x7faa0406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.136+0000 7faa1d7fa700 1 -- 172.21.15.181:0/427633671 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7faa1808ae60 con 0x7faa2015da70 2023-10-16T10:00:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.136+0000 7faa24903700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7faa0406b020 0x7faa0406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.137+0000 7faa24903700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7faa0406b020 0x7faa0406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa140017d0 tx=0x7faa14009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.141+0000 7faa1d7fa700 1 -- 172.21.15.181:0/427633671 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faa18055ba0 con 0x7faa2015da70 2023-10-16T10:00:29.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.334+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa20072210 con 0x7faa2015da70 2023-10-16T10:00:29.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.338+0000 7faa1d7fa700 1 -- 172.21.15.181:0/427633671 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa180591c0 con 0x7faa2015da70 2023-10-16T10:00:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7faa0406b020 msgr2=0x7faa0406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7faa0406b020 0x7faa0406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa140017d0 tx=0x7faa14009040 comp rx=0 tx=0).stop 2023-10-16T10:00:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2015da70 msgr2=0x7faa201607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2015da70 0x7faa201607d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7faa2006b3a0 tx=0x7faa18004ad0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 shutdown_connections 2023-10-16T10:00:29.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7faa0406b020 0x7faa0406d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa2006ade0 0x7faa200af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2015da70 0x7faa201607d0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 --2- 172.21.15.181:0/427633671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa20072760 0x7faa201f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 >> 172.21.15.181:0/427633671 conn(0x7faa200cd010 msgr2=0x7faa200ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 shutdown_connections 2023-10-16T10:00:29.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.341+0000 7faa26b67700 1 -- 172.21.15.181:0/427633671 wait complete. 2023-10-16T10:00:29.355 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:00:29.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:00:29.395 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:00:29.443 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:00:29.443 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:00:29.444 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:29.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:00:29.461 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:00:29.462 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:29.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:00:29.481 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:00:29.482 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:29.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:00:29.519 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:00:29.519 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:29.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:00:29.539 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:00:29.540 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:29.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.677+0000 7fae99abd700 1 Processor -- start 2023-10-16T10:00:29.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.677+0000 7fae99abd700 1 -- start start 2023-10-16T10:00:29.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.677+0000 7fae99abd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 0x7fae940ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.677+0000 7fae99abd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940cc510 con 0x7fae9415b990 2023-10-16T10:00:29.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae99abd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940cc7f0 con 0x7fae940cd2d0 2023-10-16T10:00:29.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae99abd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ccad0 con 0x7fae940d0370 2023-10-16T10:00:29.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae937fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fae9415b990 0x7fae940cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55340/0 (socket says 172.21.15.181:55340) 2023-10-16T10:00:29.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae937fe700 1 -- 172.21.15.181:0/2981195130 learned_addr learned my addr 172.21.15.181:0/2981195130 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:29.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae92ffd700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 0x7fae940ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576506704 0 0) 0x7fae940cc510 con 0x7fae9415b990 2023-10-16T10:00:29.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7c0035b0 con 0x7fae9415b990 2023-10-16T10:00:29.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2744358313 0 0) 0x7fae7c0035b0 con 0x7fae9415b990 2023-10-16T10:00:29.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 msgr2=0x7fae940ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:29.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 0x7fae940ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 >> v1:172.21.15.190:6789/0 conn(0x7fae940cd2d0 legacy=0x7fae940ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940ccdb0 con 0x7fae9415b990 2023-10-16T10:00:29.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fae840017f0 con 0x7fae9415b990 2023-10-16T10:00:29.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 >> v1:172.21.15.181:6789/0 conn(0x7fae9415b990 legacy=0x7fae940cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae7c003a30 0x7fae7c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 0x7fae7c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c004d90 0x7fae7c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ccad0 con 0x7fae7c003a30 2023-10-16T10:00:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00ba10 con 0x7fae7c004d90 2023-10-16T10:00:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00bbd0 con 0x7fae940d0370 2023-10-16T10:00:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae937fe700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae7c003a30 0x7fae7c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae92ffd700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 0x7fae7c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.678+0000 7fae93fff700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c004d90 0x7fae7c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae937fe700 1 -- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 msgr2=0x7fae7c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae937fe700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940d0370 0x7fae7c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae937fe700 1 -- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c004d90 msgr2=0x7fae7c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae937fe700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c004d90 0x7fae7c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae937fe700 1 -- 172.21.15.181:0/2981195130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0035b0 con 0x7fae7c003a30 2023-10-16T10:00:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae93fff700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c004d90 0x7fae7c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae937fe700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae7c003a30 0x7fae7c003e10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fae8400c240 tx=0x7fae7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4c235cd81c8944 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400cd20 con 0x7fae7c003a30 2023-10-16T10:00:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8400ce80 con 0x7fae7c003a30 2023-10-16T10:00:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae927fc700 1 -- 172.21.15.181:0/2981195130 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840072f0 con 0x7fae7c003a30 2023-10-16T10:00:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae99abd700 1 -- 172.21.15.181:0/2981195130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae7c003a30 msgr2=0x7fae7c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae99abd700 1 --2- 172.21.15.181:0/2981195130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae7c003a30 0x7fae7c003e10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fae8400c240 tx=0x7fae7c005880 comp rx=0 tx=0).stop 2023-10-16T10:00:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae93fff700 1 -- 172.21.15.181:0/2981195130 reap_dead start 2023-10-16T10:00:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae99abd700 1 -- 172.21.15.181:0/2981195130 shutdown_connections 2023-10-16T10:00:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae99abd700 1 -- 172.21.15.181:0/2981195130 >> 172.21.15.181:0/2981195130 conn(0x7fae9409a9a0 msgr2=0x7fae940d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae99abd700 1 -- 172.21.15.181:0/2981195130 shutdown_connections 2023-10-16T10:00:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.679+0000 7fae99abd700 1 -- 172.21.15.181:0/2981195130 wait complete. 2023-10-16T10:00:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.681+0000 7fae99abd700 1 Processor -- start 2023-10-16T10:00:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.681+0000 7fae99abd700 1 -- start start 2023-10-16T10:00:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae99abd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c003a30 0x7fae9415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae99abd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae940ca470 0x7fae9406f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae99abd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae9406fee0 0x7fae9415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae99abd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94070360 con 0x7fae940ca470 2023-10-16T10:00:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae99abd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9415e620 con 0x7fae7c003a30 2023-10-16T10:00:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae99abd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940c8ba0 con 0x7fae9406fee0 2023-10-16T10:00:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae92ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae940ca470 0x7fae9406f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae92ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae940ca470 0x7fae9406f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44276/0 (socket says 172.21.15.181:44276) 2023-10-16T10:00:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae92ffd700 1 -- 172.21.15.181:0/3141979042 learned_addr learned my addr 172.21.15.181:0/3141979042 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae93fff700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae9406fee0 0x7fae9415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae937fe700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c003a30 0x7fae9415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae92ffd700 1 -- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae9406fee0 msgr2=0x7fae9415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae92ffd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae9406fee0 0x7fae9415c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae92ffd700 1 -- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c003a30 msgr2=0x7fae9415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.682+0000 7fae92ffd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c003a30 0x7fae9415e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae92ffd700 1 -- 172.21.15.181:0/3141979042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940c8e80 con 0x7fae940ca470 2023-10-16T10:00:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae937fe700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c003a30 0x7fae9415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae92ffd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae940ca470 0x7fae9406f9a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fae88011aa0 tx=0x7fae88011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae7bfff700 1 -- 172.21.15.181:0/3141979042 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8800f980 con 0x7fae940ca470 2023-10-16T10:00:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae7bfff700 1 -- 172.21.15.181:0/3141979042 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8800fae0 con 0x7fae940ca470 2023-10-16T10:00:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae7bfff700 1 -- 172.21.15.181:0/3141979042 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8800fe90 con 0x7fae940ca470 2023-10-16T10:00:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae940c9160 con 0x7fae940ca470 2023-10-16T10:00:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.683+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9415ed10 con 0x7fae940ca470 2023-10-16T10:00:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.684+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae94067d20 con 0x7fae940ca470 2023-10-16T10:00:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.684+0000 7fae7bfff700 1 -- 172.21.15.181:0/3141979042 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fae88014440 con 0x7fae940ca470 2023-10-16T10:00:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.685+0000 7fae7bfff700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fae8006b020 0x7fae8006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.685+0000 7fae7bfff700 1 -- 172.21.15.181:0/3141979042 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7fae88089fc0 con 0x7fae940ca470 2023-10-16T10:00:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.685+0000 7fae937fe700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fae8006b020 0x7fae8006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.685+0000 7fae937fe700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fae8006b020 0x7fae8006d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae84015040 tx=0x7fae84007000 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.689+0000 7fae7bfff700 1 -- 172.21.15.181:0/3141979042 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fae88054eb0 con 0x7fae940ca470 2023-10-16T10:00:29.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.925+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae9406ebc0 con 0x7fae940ca470 2023-10-16T10:00:29.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.926+0000 7fae7bfff700 1 -- 172.21.15.181:0/3141979042 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fae9406ebc0 con 0x7fae940ca470 2023-10-16T10:00:29.928 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:29.928 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3576975018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3576975018},{"type":"v1","addr":"172.21.15.190:6834","nonce":3576975018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/54128065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":54128065},{"type":"v1","addr":"172.21.15.190:6835","nonce":54128065}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/509615642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":509615642},{"type":"v1","addr":"172.21.15.181:6836","nonce":509615642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2839633532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2839633532},{"type":"v1","addr":"172.21.15.181:6837","nonce":2839633532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[]} 2023-10-16T10:00:29.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fae8006b020 msgr2=0x7fae8006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fae8006b020 0x7fae8006d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae84015040 tx=0x7fae84007000 comp rx=0 tx=0).stop 2023-10-16T10:00:29.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae940ca470 msgr2=0x7fae9406f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:29.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae940ca470 0x7fae9406f9a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fae88011aa0 tx=0x7fae88011e60 comp rx=0 tx=0).stop 2023-10-16T10:00:29.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 shutdown_connections 2023-10-16T10:00:29.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fae8006b020 0x7fae8006d4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae9406fee0 0x7fae9415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae7c003a30 0x7fae9415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 --2- 172.21.15.181:0/3141979042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae940ca470 0x7fae9406f9a0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:29.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 >> 172.21.15.181:0/3141979042 conn(0x7fae9409a9a0 msgr2=0x7fae9415a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:29.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 shutdown_connections 2023-10-16T10:00:29.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:29.928+0000 7fae99abd700 1 -- 172.21.15.181:0/3141979042 wait complete. 2023-10-16T10:00:29.933 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 42 2023-10-16T10:00:29.943 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:00:29.943 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:00:29.943 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:00:29.943 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:00:29.944 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:00:29.944 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:00:29.944 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:00:29.945 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:00:29.945 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:00:29.945 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:00:29.946 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:00:29.946 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:00:29.946 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:00:29.946 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:00:29.947 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:00:29.947 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:00:29.952 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:00:29.952 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:00:29.954 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:00:29.954 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:00:29.959 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:00:29.959 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:00:29.960 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:00:29.960 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:00:29.995 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:00:29.995 DEBUG:teuthology.parallel:result is None 2023-10-16T10:00:29.996 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:00:29.996 DEBUG:teuthology.parallel:result is None 2023-10-16T10:00:30.001 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:00:30.002 DEBUG:teuthology.parallel:result is None 2023-10-16T10:00:30.002 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:00:30.003 DEBUG:teuthology.parallel:result is None 2023-10-16T10:00:30.003 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:00:30.074 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T10:00:30.075 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T10:00:30.087 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T10:00:30.088 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T10:00:30.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.129+0000 7f5c0d907700 1 Processor -- start 2023-10-16T10:00:30.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.129+0000 7f5c0d907700 1 -- start start 2023-10-16T10:00:30.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c0d907700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c0815bd50 0x7f5c080d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:30.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c0d907700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c080d24f0 con 0x7f5c080cfc60 2023-10-16T10:00:30.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c0d907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0815c320 con 0x7f5c080b05d0 2023-10-16T10:00:30.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c0d907700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0815ccc0 con 0x7f5c0815bd50 2023-10-16T10:00:30.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c06ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5c080cfc60 0x7f5c080d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55384/0 (socket says 172.21.15.181:55384) 2023-10-16T10:00:30.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c06ffd700 1 -- 172.21.15.181:0/4261451883 learned_addr learned my addr 172.21.15.181:0/4261451883 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c067fc700 1 --2- 172.21.15.181:0/4261451883 >> v2:172.21.15.190:3300/0 conn(0x7f5c0815bd50 0x7f5c080d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c05ffb700 1 -- 172.21.15.181:0/4261451883 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233476292 0 0) 0x7f5c080d24f0 con 0x7f5c080cfc60 2023-10-16T10:00:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c05ffb700 1 -- 172.21.15.181:0/4261451883 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf00035b0 con 0x7f5c080cfc60 2023-10-16T10:00:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c067fc700 1 -- 172.21.15.181:0/4261451883 >> v1:172.21.15.190:6789/0 conn(0x7f5c080b05d0 legacy=0x7f5c080cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c067fc700 1 -- 172.21.15.181:0/4261451883 >> v1:172.21.15.181:6789/0 conn(0x7f5c080cfc60 legacy=0x7f5c080d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c05ffb700 1 -- 172.21.15.181:0/4261451883 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2470830938 0 0) 0x7f5bf00035b0 con 0x7f5c080cfc60 2023-10-16T10:00:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.130+0000 7f5c067fc700 1 -- 172.21.15.181:0/4261451883 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0815c600 con 0x7f5c0815bd50 2023-10-16T10:00:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c067fc700 1 --2- 172.21.15.181:0/4261451883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 0x7f5c080d1fb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009840 tx=0x7f5bf8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0299bb436ddda8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c05ffb700 1 -- 172.21.15.181:0/4261451883 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8011830 con 0x7f5c0815bd50 2023-10-16T10:00:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c05ffb700 1 -- 172.21.15.181:0/4261451883 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8011990 con 0x7f5c0815bd50 2023-10-16T10:00:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c05ffb700 1 -- 172.21.15.181:0/4261451883 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8011d20 con 0x7f5c0815bd50 2023-10-16T10:00:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c0d907700 1 -- 172.21.15.181:0/4261451883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 msgr2=0x7f5c080d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c0d907700 1 --2- 172.21.15.181:0/4261451883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 0x7f5c080d1fb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009840 tx=0x7f5bf8009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c0d907700 1 -- 172.21.15.181:0/4261451883 shutdown_connections 2023-10-16T10:00:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c0d907700 1 --2- 172.21.15.181:0/4261451883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 0x7f5c080d1fb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:30.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c0d907700 1 -- 172.21.15.181:0/4261451883 >> 172.21.15.181:0/4261451883 conn(0x7f5c0809a9a0 msgr2=0x7f5c080d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:30.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c0d907700 1 -- 172.21.15.181:0/4261451883 shutdown_connections 2023-10-16T10:00:30.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.131+0000 7f5c0d907700 1 -- 172.21.15.181:0/4261451883 wait complete. 2023-10-16T10:00:30.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.132+0000 7f5c0d907700 1 Processor -- start 2023-10-16T10:00:30.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c0d907700 1 -- start start 2023-10-16T10:00:30.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c0d907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080b05d0 0x7f5c0806c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:30.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c0d907700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cfc60 0x7f5c0815e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:30.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c0d907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 0x7f5c08157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:30.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c0d907700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08157620 con 0x7f5c080cfc60 2023-10-16T10:00:30.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c0d907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080b4710 con 0x7f5c0815bd50 2023-10-16T10:00:30.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c0d907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08156fd0 con 0x7f5c080b05d0 2023-10-16T10:00:30.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c067fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cfc60 0x7f5c0815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c067fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cfc60 0x7f5c0815e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44314/0 (socket says 172.21.15.181:44314) 2023-10-16T10:00:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.133+0000 7f5c067fc700 1 -- 172.21.15.181:0/108286340 learned_addr learned my addr 172.21.15.181:0/108286340 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c06ffd700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080b05d0 0x7f5c0806c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c077fe700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 0x7f5c08157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c067fc700 1 -- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080b05d0 msgr2=0x7f5c0806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c067fc700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080b05d0 0x7f5c0806c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c067fc700 1 -- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 msgr2=0x7f5c08157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c067fc700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 0x7f5c08157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c067fc700 1 -- 172.21.15.181:0/108286340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf80094a0 con 0x7f5c080cfc60 2023-10-16T10:00:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c067fc700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cfc60 0x7f5c0815e360 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80090b0 tx=0x7f5bf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5beffff700 1 -- 172.21.15.181:0/108286340 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf801a910 con 0x7f5c080cfc60 2023-10-16T10:00:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5beffff700 1 -- 172.21.15.181:0/108286340 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf801aa70 con 0x7f5c080cfc60 2023-10-16T10:00:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c081572b0 con 0x7f5c080cfc60 2023-10-16T10:00:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5beffff700 1 -- 172.21.15.181:0/108286340 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf801ae00 con 0x7f5c080cfc60 2023-10-16T10:00:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.134+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0815e8a0 con 0x7f5c080cfc60 2023-10-16T10:00:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.135+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c08067d20 con 0x7f5c080cfc60 2023-10-16T10:00:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.136+0000 7f5beffff700 1 -- 172.21.15.181:0/108286340 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5bf8022070 con 0x7f5c080cfc60 2023-10-16T10:00:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.136+0000 7f5beffff700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5bf406b020 0x7f5bf406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.136+0000 7f5beffff700 1 -- 172.21.15.181:0/108286340 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6055+0+0 (secure 0 0 0) 0x7f5bf808a940 con 0x7f5c080cfc60 2023-10-16T10:00:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.136+0000 7f5c06ffd700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5bf406b020 0x7f5bf406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.136+0000 7f5c06ffd700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5bf406b020 0x7f5bf406d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc000f60 tx=0x7f5bfc009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:30.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.140+0000 7f5beffff700 1 -- 172.21.15.181:0/108286340 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5bf8055810 con 0x7f5c080cfc60 2023-10-16T10:00:30.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.349+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5c0815b120 con 0x7f5c080cfc60 2023-10-16T10:00:31.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:30.998+0000 7f5beffff700 1 -- 172.21.15.181:0/108286340 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5bf8029080 con 0x7f5c080cfc60 2023-10-16T10:00:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5bf406b020 msgr2=0x7f5bf406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5bf406b020 0x7f5bf406d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc000f60 tx=0x7f5bfc009040 comp rx=0 tx=0).stop 2023-10-16T10:00:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cfc60 msgr2=0x7f5c0815e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cfc60 0x7f5c0815e360 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80090b0 tx=0x7f5bf8004750 comp rx=0 tx=0).stop 2023-10-16T10:00:31.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 shutdown_connections 2023-10-16T10:00:31.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5bf406b020 0x7f5bf406d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c080b05d0 0x7f5c0806c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c0815bd50 0x7f5c08157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 --2- 172.21.15.181:0/108286340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c080cfc60 0x7f5c0815e360 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.001+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 >> 172.21.15.181:0/108286340 conn(0x7f5c0809a9a0 msgr2=0x7f5c080ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:31.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.002+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 shutdown_connections 2023-10-16T10:00:31.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.002+0000 7f5c0d907700 1 -- 172.21.15.181:0/108286340 wait complete. 2023-10-16T10:00:31.005 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T10:00:31.019 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:00:31.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.144+0000 7f21b5d30700 1 Processor -- start 2023-10-16T10:00:31.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.144+0000 7f21b5d30700 1 -- start start 2023-10-16T10:00:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21b5d30700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f21b015dfd0 0x7f21b00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21b5d30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b00af450 0x7f21b006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21b5d30700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21b006db80 0x7f21b00cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21b5d30700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b006f870 con 0x7f21b006db80 2023-10-16T10:00:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21b5d30700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b00d1160 con 0x7f21b015dfd0 2023-10-16T10:00:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21b5d30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00cc8e0 con 0x7f21b00af450 2023-10-16T10:00:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21b006db80 0x7f21b00cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21b006db80 0x7f21b00cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44326/0 (socket says 172.21.15.181:44326) 2023-10-16T10:00:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 -- 172.21.15.181:0/2429278571 learned_addr learned my addr 172.21.15.181:0/2429278571 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21af7fe700 1 --2- 172.21.15.181:0/2429278571 >> v2:172.21.15.190:3300/0 conn(0x7f21b015dfd0 0x7f21b00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21aeffd700 1 --2- 172.21.15.181:0/2429278571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b00af450 0x7f21b006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 -- 172.21.15.181:0/2429278571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b00af450 msgr2=0x7f21b006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 --2- 172.21.15.181:0/2429278571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b00af450 0x7f21b006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 -- 172.21.15.181:0/2429278571 >> v2:172.21.15.190:3300/0 conn(0x7f21b015dfd0 msgr2=0x7f21b00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 --2- 172.21.15.181:0/2429278571 >> v2:172.21.15.190:3300/0 conn(0x7f21b015dfd0 0x7f21b00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 -- 172.21.15.181:0/2429278571 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b00ccbc0 con 0x7f21b006db80 2023-10-16T10:00:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21af7fe700 1 --2- 172.21.15.181:0/2429278571 >> v2:172.21.15.190:3300/0 conn(0x7f21b015dfd0 0x7f21b00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.145+0000 7f21affff700 1 --2- 172.21.15.181:0/2429278571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b006db80 0x7f21b00cd470 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f21b0070340 tx=0x7f21a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8946490ca5b4db77 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21ae7fc700 1 -- 172.21.15.181:0/2429278571 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800e070 con 0x7f21b006db80 2023-10-16T10:00:31.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21ae7fc700 1 -- 172.21.15.181:0/2429278571 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a800b9c0 con 0x7f21b006db80 2023-10-16T10:00:31.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21ae7fc700 1 -- 172.21.15.181:0/2429278571 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800bd70 con 0x7f21b006db80 2023-10-16T10:00:31.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 -- 172.21.15.181:0/2429278571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b006db80 msgr2=0x7f21b00cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2429278571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b006db80 0x7f21b00cd470 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f21b0070340 tx=0x7f21a800b330 comp rx=0 tx=0).stop 2023-10-16T10:00:31.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 -- 172.21.15.181:0/2429278571 shutdown_connections 2023-10-16T10:00:31.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2429278571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b006db80 0x7f21b00cd470 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2429278571 >> v2:172.21.15.190:3300/0 conn(0x7f21b015dfd0 0x7f21b00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2429278571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b00af450 0x7f21b006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 -- 172.21.15.181:0/2429278571 >> 172.21.15.181:0/2429278571 conn(0x7f21b00d04a0 msgr2=0x7f21b00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:31.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 -- 172.21.15.181:0/2429278571 shutdown_connections 2023-10-16T10:00:31.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.146+0000 7f21b5d30700 1 -- 172.21.15.181:0/2429278571 wait complete. 2023-10-16T10:00:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.148+0000 7f21b5d30700 1 Processor -- start 2023-10-16T10:00:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.148+0000 7f21b5d30700 1 -- start start 2023-10-16T10:00:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21b5d30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b006db80 0x7f21b0157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21b5d30700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21b00af450 0x7f21b00ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21b5d30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b015dfd0 0x7f21b015b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21b5d30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00c6450 con 0x7f21b015dfd0 2023-10-16T10:00:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21b5d30700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00c6730 con 0x7f21b00af450 2023-10-16T10:00:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21b5d30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00c6a10 con 0x7f21b006db80 2023-10-16T10:00:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b015dfd0 0x7f21b015b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b015dfd0 0x7f21b015b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44332/0 (socket says 172.21.15.181:44332) 2023-10-16T10:00:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21affff700 1 -- 172.21.15.181:0/2260672469 learned_addr learned my addr 172.21.15.181:0/2260672469 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21aeffd700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21b00af450 0x7f21b00ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21af7fe700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b006db80 0x7f21b0157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.149+0000 7f21affff700 1 -- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b006db80 msgr2=0x7f21b0157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f21affff700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b006db80 0x7f21b0157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f21affff700 1 -- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21b00af450 msgr2=0x7f21b00ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f21affff700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21b00af450 0x7f21b00ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f21affff700 1 -- 172.21.15.181:0/2260672469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8009b30 con 0x7f21b015dfd0 2023-10-16T10:00:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f21affff700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b015dfd0 0x7f21b015b260 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f21a800b370 tx=0x7f21a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f2197fff700 1 -- 172.21.15.181:0/2260672469 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80099b0 con 0x7f21b015dfd0 2023-10-16T10:00:31.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f2197fff700 1 -- 172.21.15.181:0/2260672469 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a8004250 con 0x7f21b015dfd0 2023-10-16T10:00:31.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b00c6cf0 con 0x7f21b015dfd0 2023-10-16T10:00:31.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f2197fff700 1 -- 172.21.15.181:0/2260672469 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80045e0 con 0x7f21b015dfd0 2023-10-16T10:00:31.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.150+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b00aed50 con 0x7f21b015dfd0 2023-10-16T10:00:31.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.151+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b00c7210 con 0x7f21b015dfd0 2023-10-16T10:00:31.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.151+0000 7f2197fff700 1 -- 172.21.15.181:0/2260672469 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f21a8004740 con 0x7f21b015dfd0 2023-10-16T10:00:31.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.152+0000 7f2197fff700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f219c06b020 0x7f219c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.152+0000 7f2197fff700 1 -- 172.21.15.181:0/2260672469 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f21a808aa50 con 0x7f21b015dfd0 2023-10-16T10:00:31.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.152+0000 7f21af7fe700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f219c06b020 0x7f219c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.152+0000 7f21af7fe700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f219c06b020 0x7f219c06d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21b0071f20 tx=0x7f21a4008040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.156+0000 7f2197fff700 1 -- 172.21.15.181:0/2260672469 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21b00c7210 con 0x7f21b015dfd0 2023-10-16T10:00:31.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.423+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f21b00c7210 con 0x7f21b015dfd0 2023-10-16T10:00:31.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.424+0000 7f2197fff700 1 -- 172.21.15.181:0/2260672469 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f21b00c7210 con 0x7f21b015dfd0 2023-10-16T10:00:31.426 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:31.427 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:00:31.427 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T10:00:31.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.427 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T10:00:31.427 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T10:00:31.428 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:00:31.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:00:31.428 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.429 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T10:00:31.429 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T10:00:31.429 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.430 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.430 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:00:31.430 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:00:31.430 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.430 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.431 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T10:00:31.431 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T10:00:31.431 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.432 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:00:31.432 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:00:31.432 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.433 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T10:00:31.433 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T10:00:31.433 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.433 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.434 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:00:31.434 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:00:31.434 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.434 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.435 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T10:00:31.435 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T10:00:31.435 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.435 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.436 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.436 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.436 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.436 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T10:00:31.437 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T10:00:31.437 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.437 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.437 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.437 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.438 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.438 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T10:00:31.438 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T10:00:31.438 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.439 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.439 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.439 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.440 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T10:00:31.440 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T10:00:31.440 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.441 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.441 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.441 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.441 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T10:00:31.442 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T10:00:31.442 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.442 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.442 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.442 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.442 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.443 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T10:00:31.443 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T10:00:31.443 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.443 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.444 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.445 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T10:00:31.445 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T10:00:31.445 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.445 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.445 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.446 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.446 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T10:00:31.447 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T10:00:31.447 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.447 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:00:31.447 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:00:31.447 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.448 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.448 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T10:00:31.448 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T10:00:31.448 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T10:00:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:00:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T10:00:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T10:00:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:00:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:00:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:00:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T10:00:31.452 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T10:00:31.452 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:00:31.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:00:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:00:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T10:00:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T10:00:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:00:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:00:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:00:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:00:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T10:00:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T10:00:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:00:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:00:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:00:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T10:00:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T10:00:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:00:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:00:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:00:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:31.459 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T10:00:31.459 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T10:00:31.459 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:00:31.459 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:00:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:00:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:00:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T10:00:31.461 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:00:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f219c06b020 msgr2=0x7f219c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f219c06b020 0x7f219c06d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21b0071f20 tx=0x7f21a4008040 comp rx=0 tx=0).stop 2023-10-16T10:00:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b015dfd0 msgr2=0x7f21b015b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b015dfd0 0x7f21b015b260 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f21a800b370 tx=0x7f21a8004ad0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 shutdown_connections 2023-10-16T10:00:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f219c06b020 0x7f219c06d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21b006db80 0x7f21b0157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b015dfd0 0x7f21b015b260 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 --2- 172.21.15.181:0/2260672469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21b00af450 0x7f21b00ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 >> 172.21.15.181:0/2260672469 conn(0x7f21b00d04a0 msgr2=0x7f21b00ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 shutdown_connections 2023-10-16T10:00:31.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.426+0000 7f21b5d30700 1 -- 172.21.15.181:0/2260672469 wait complete. 2023-10-16T10:00:31.465 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:31.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.583+0000 7feb1a8b0700 1 Processor -- start 2023-10-16T10:00:31.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.584+0000 7feb1a8b0700 1 -- start start 2023-10-16T10:00:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.584+0000 7feb1a8b0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb1415de50 0x7feb140b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.584+0000 7feb1a8b0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb1406bdb0 0x7feb140cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.584+0000 7feb1a8b0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb14161fc0 con 0x7feb14161ac0 2023-10-16T10:00:31.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.584+0000 7feb1a8b0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb140b4010 con 0x7feb1415de50 2023-10-16T10:00:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.584+0000 7feb1a8b0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb141622a0 con 0x7feb1406bdb0 2023-10-16T10:00:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.584+0000 7feb13fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb1415de50 0x7feb140b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb13fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb1415de50 0x7feb140b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44336/0 (socket says 172.21.15.181:44336) 2023-10-16T10:00:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb13fff700 1 -- 172.21.15.181:0/2773580024 learned_addr learned my addr 172.21.15.181:0/2773580024 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb137fe700 1 --2- 172.21.15.181:0/2773580024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb1406bdb0 0x7feb140cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb13fff700 1 -- 172.21.15.181:0/2773580024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb1406bdb0 msgr2=0x7feb140cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb13fff700 1 --2- 172.21.15.181:0/2773580024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb1406bdb0 0x7feb140cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb13fff700 1 -- 172.21.15.181:0/2773580024 >> v1:172.21.15.190:6789/0 conn(0x7feb14161ac0 legacy=0x7feb140cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb13fff700 1 -- 172.21.15.181:0/2773580024 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14162580 con 0x7feb1415de50 2023-10-16T10:00:31.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb137fe700 1 --2- 172.21.15.181:0/2773580024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb1406bdb0 0x7feb140cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb13fff700 1 --2- 172.21.15.181:0/2773580024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1415de50 0x7feb140b3ad0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7feb040098f0 tx=0x7feb04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe2bec71f2d59a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb12ffd700 1 -- 172.21.15.181:0/2773580024 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0400b8a0 con 0x7feb1415de50 2023-10-16T10:00:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb12ffd700 1 -- 172.21.15.181:0/2773580024 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0400ba00 con 0x7feb1415de50 2023-10-16T10:00:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.585+0000 7feb12ffd700 1 -- 172.21.15.181:0/2773580024 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0400bd90 con 0x7feb1415de50 2023-10-16T10:00:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2773580024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1415de50 msgr2=0x7feb140b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2773580024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1415de50 0x7feb140b3ad0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7feb040098f0 tx=0x7feb04009c00 comp rx=0 tx=0).stop 2023-10-16T10:00:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2773580024 shutdown_connections 2023-10-16T10:00:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2773580024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1415de50 0x7feb140b3ad0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2773580024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb1406bdb0 0x7feb140cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2773580024 >> 172.21.15.181:0/2773580024 conn(0x7feb140665a0 msgr2=0x7feb14066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2773580024 shutdown_connections 2023-10-16T10:00:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.586+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2773580024 wait complete. 2023-10-16T10:00:31.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.587+0000 7feb1a8b0700 1 Processor -- start 2023-10-16T10:00:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.587+0000 7feb1a8b0700 1 -- start start 2023-10-16T10:00:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb1a8b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1406bdb0 0x7feb141f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb1a8b0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb1415de50 0x7feb140cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb1a8b0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb14161ac0 0x7feb140ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb1a8b0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1414cec0 con 0x7feb1406bdb0 2023-10-16T10:00:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb1a8b0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb140c92f0 con 0x7feb1415de50 2023-10-16T10:00:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb1a8b0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb140c95d0 con 0x7feb14161ac0 2023-10-16T10:00:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1406bdb0 0x7feb141f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1406bdb0 0x7feb141f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44342/0 (socket says 172.21.15.181:44342) 2023-10-16T10:00:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 -- 172.21.15.181:0/2961360057 learned_addr learned my addr 172.21.15.181:0/2961360057 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb18e4d700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb14161ac0 0x7feb140ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb137fe700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb1415de50 0x7feb140cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 -- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb14161ac0 msgr2=0x7feb140ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb14161ac0 0x7feb140ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 -- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb1415de50 msgr2=0x7feb140cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb1415de50 0x7feb140cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 -- 172.21.15.181:0/2961360057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04009550 con 0x7feb1406bdb0 2023-10-16T10:00:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb18e4d700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb14161ac0 0x7feb140ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb13fff700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1406bdb0 0x7feb141f3170 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7feb04009c40 tx=0x7feb04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb10ff9700 1 -- 172.21.15.181:0/2961360057 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040036a0 con 0x7feb1406bdb0 2023-10-16T10:00:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb10ff9700 1 -- 172.21.15.181:0/2961360057 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0401f070 con 0x7feb1406bdb0 2023-10-16T10:00:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1415e230 con 0x7feb1406bdb0 2023-10-16T10:00:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.588+0000 7feb10ff9700 1 -- 172.21.15.181:0/2961360057 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04003a60 con 0x7feb1406bdb0 2023-10-16T10:00:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.589+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1415e7f0 con 0x7feb1406bdb0 2023-10-16T10:00:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.589+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb140c4ea0 con 0x7feb1406bdb0 2023-10-16T10:00:31.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.590+0000 7feb10ff9700 1 -- 172.21.15.181:0/2961360057 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb04003bc0 con 0x7feb1406bdb0 2023-10-16T10:00:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.590+0000 7feb10ff9700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feaf406b020 0x7feaf406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.590+0000 7feb137fe700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feaf406b020 0x7feaf406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.590+0000 7feb10ff9700 1 -- 172.21.15.181:0/2961360057 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5281+0+0 (secure 0 0 0) 0x7feb04021070 con 0x7feb1406bdb0 2023-10-16T10:00:31.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.591+0000 7feb137fe700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feaf406b020 0x7feaf406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb140cb110 tx=0x7feafc008040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.595+0000 7feb10ff9700 1 -- 172.21.15.181:0/2961360057 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feb04056070 con 0x7feb1406bdb0 2023-10-16T10:00:31.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.790+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb1406bb90 con 0x7feb1406bdb0 2023-10-16T10:00:31.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.790+0000 7feb10ff9700 1 -- 172.21.15.181:0/2961360057 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13538 (secure 0 0 0) 0x7feb04015090 con 0x7feb1406bdb0 2023-10-16T10:00:31.792 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:31.793 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":99,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:31.012686+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:31.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feaf406b020 msgr2=0x7feaf406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feaf406b020 0x7feaf406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb140cb110 tx=0x7feafc008040 comp rx=0 tx=0).stop 2023-10-16T10:00:31.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1406bdb0 msgr2=0x7feb141f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1406bdb0 0x7feb141f3170 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7feb04009c40 tx=0x7feb04004750 comp rx=0 tx=0).stop 2023-10-16T10:00:31.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 shutdown_connections 2023-10-16T10:00:31.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feaf406b020 0x7feaf406d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb14161ac0 0x7feb140ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb1406bdb0 0x7feb141f3170 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 --2- 172.21.15.181:0/2961360057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb1415de50 0x7feb140cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 >> 172.21.15.181:0/2961360057 conn(0x7feb140665a0 msgr2=0x7feb1406d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:31.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 shutdown_connections 2023-10-16T10:00:31.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.793+0000 7feb1a8b0700 1 -- 172.21.15.181:0/2961360057 wait complete. 2023-10-16T10:00:31.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:00:31.808 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:00:31.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.930+0000 7fa6f8488700 1 Processor -- start 2023-10-16T10:00:31.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.930+0000 7fa6f8488700 1 -- start start 2023-10-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f8488700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa6f00cc6e0 0x7fa6f0071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f8488700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6f00ce680 0x7fa6f00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f8488700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f00ce1f0 con 0x7fa6f00b2010 2023-10-16T10:00:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f8488700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f00ccb90 con 0x7fa6f00ce680 2023-10-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f8488700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f0072120 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6f00ce680 0x7fa6f00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6f00ce680 0x7fa6f00ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44354/0 (socket says 172.21.15.181:44354) 2023-10-16T10:00:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 -- 172.21.15.181:0/42745079 learned_addr learned my addr 172.21.15.181:0/42745079 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f5a23700 1 --2- 172.21.15.181:0/42745079 >> v2:172.21.15.190:3300/0 conn(0x7fa6f00cc6e0 0x7fa6f0071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 -- 172.21.15.181:0/42745079 >> v2:172.21.15.190:3300/0 conn(0x7fa6f00cc6e0 msgr2=0x7fa6f0071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 --2- 172.21.15.181:0/42745079 >> v2:172.21.15.190:3300/0 conn(0x7fa6f00cc6e0 0x7fa6f0071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 -- 172.21.15.181:0/42745079 >> v1:172.21.15.190:6789/0 conn(0x7fa6f00b2010 legacy=0x7fa6f00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 -- 172.21.15.181:0/42745079 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f00ccda0 con 0x7fa6f00ce680 2023-10-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f5a23700 1 --2- 172.21.15.181:0/42745079 >> v2:172.21.15.190:3300/0 conn(0x7fa6f00cc6e0 0x7fa6f0071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f6a25700 1 --2- 172.21.15.181:0/42745079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00ce680 0x7fa6f00ceaf0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa6f015e1c0 tx=0x7fa6e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e58a5c702771e14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f5222700 1 -- 172.21.15.181:0/42745079 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800e070 con 0x7fa6f00ce680 2023-10-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f5222700 1 -- 172.21.15.181:0/42745079 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e800b9c0 con 0x7fa6f00ce680 2023-10-16T10:00:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.931+0000 7fa6f5222700 1 -- 172.21.15.181:0/42745079 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800bd70 con 0x7fa6f00ce680 2023-10-16T10:00:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 -- 172.21.15.181:0/42745079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00ce680 msgr2=0x7fa6f00ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 --2- 172.21.15.181:0/42745079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00ce680 0x7fa6f00ceaf0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa6f015e1c0 tx=0x7fa6e800b330 comp rx=0 tx=0).stop 2023-10-16T10:00:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 -- 172.21.15.181:0/42745079 shutdown_connections 2023-10-16T10:00:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 --2- 172.21.15.181:0/42745079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00ce680 0x7fa6f00ceaf0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 --2- 172.21.15.181:0/42745079 >> v2:172.21.15.190:3300/0 conn(0x7fa6f00cc6e0 0x7fa6f0071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 -- 172.21.15.181:0/42745079 >> 172.21.15.181:0/42745079 conn(0x7fa6f00cd010 msgr2=0x7fa6f00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 -- 172.21.15.181:0/42745079 shutdown_connections 2023-10-16T10:00:31.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.932+0000 7fa6f8488700 1 -- 172.21.15.181:0/42745079 wait complete. 2023-10-16T10:00:31.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.934+0000 7fa6f8488700 1 Processor -- start 2023-10-16T10:00:31.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.934+0000 7fa6f8488700 1 -- start start 2023-10-16T10:00:31.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.934+0000 7fa6f8488700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f00b2010 0x7fa6f0154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.934+0000 7fa6f8488700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00cc6e0 0x7fa6f00c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f8488700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f00ce680 0x7fa6f00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f8488700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f015c320 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f8488700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f0154f40 con 0x7fa6f00ce680 2023-10-16T10:00:31.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f8488700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f015b710 con 0x7fa6f00b2010 2023-10-16T10:00:31.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00cc6e0 0x7fa6f00c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00cc6e0 0x7fa6f00c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44358/0 (socket says 172.21.15.181:44358) 2023-10-16T10:00:31.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 -- 172.21.15.181:0/2933820714 learned_addr learned my addr 172.21.15.181:0/2933820714 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:31.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f6224700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f00b2010 0x7fa6f0154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f6a25700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f00ce680 0x7fa6f00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 -- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f00b2010 msgr2=0x7fa6f0154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f00b2010 0x7fa6f0154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 -- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f00ce680 msgr2=0x7fa6f00c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:31.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f00ce680 0x7fa6f00c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:31.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 -- 172.21.15.181:0/2933820714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e8009b30 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f6224700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f00b2010 0x7fa6f0154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:31.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f6a25700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f00ce680 0x7fa6f00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:31.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f5a23700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00cc6e0 0x7fa6f00c9870 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000e810 tx=0x7fa6e000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6e6ffd700 1 -- 172.21.15.181:0/2933820714 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000c020 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6e6ffd700 1 -- 172.21.15.181:0/2933820714 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e0004250 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f015b9f0 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6e6ffd700 1 -- 172.21.15.181:0/2933820714 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00036a0 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.935+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f015bfb0 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.936+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f015c140 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.937+0000 7fa6e6ffd700 1 -- 172.21.15.181:0/2933820714 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa6e0003800 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.937+0000 7fa6e6ffd700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa6d406b020 0x7fa6d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:31.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.937+0000 7fa6f6224700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa6d406b020 0x7fa6d406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:31.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.937+0000 7fa6e6ffd700 1 -- 172.21.15.181:0/2933820714 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5281+0+0 (secure 0 0 0) 0x7fa6e0089470 con 0x7fa6f00cc6e0 2023-10-16T10:00:31.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.937+0000 7fa6f6224700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa6d406b020 0x7fa6d406d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000f60 tx=0x7fa6ec009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:31.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:31.941+0000 7fa6e6ffd700 1 -- 172.21.15.181:0/2933820714 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6e00544a0 con 0x7fa6f00cc6e0 2023-10-16T10:00:32.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.138+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6f01f0f00 con 0x7fa6f00cc6e0 2023-10-16T10:00:32.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.139+0000 7fa6e6ffd700 1 -- 172.21.15.181:0/2933820714 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6e0057ac0 con 0x7fa6f00cc6e0 2023-10-16T10:00:32.141 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:00:32.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa6d406b020 msgr2=0x7fa6d406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa6d406b020 0x7fa6d406d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec000f60 tx=0x7fa6ec009040 comp rx=0 tx=0).stop 2023-10-16T10:00:32.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00cc6e0 msgr2=0x7fa6f00c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00cc6e0 0x7fa6f00c9870 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000e810 tx=0x7fa6e000ebd0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 shutdown_connections 2023-10-16T10:00:32.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa6d406b020 0x7fa6d406d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f00b2010 0x7fa6f0154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f00ce680 0x7fa6f00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 --2- 172.21.15.181:0/2933820714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f00cc6e0 0x7fa6f00c9870 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 >> 172.21.15.181:0/2933820714 conn(0x7fa6f00cd010 msgr2=0x7fa6f00cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:32.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 shutdown_connections 2023-10-16T10:00:32.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.141+0000 7fa6f8488700 1 -- 172.21.15.181:0/2933820714 wait complete. 2023-10-16T10:00:32.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:00:32.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.273+0000 7f93ae7df700 1 Processor -- start 2023-10-16T10:00:32.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.273+0000 7f93ae7df700 1 -- start start 2023-10-16T10:00:32.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93ae7df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a8159730 0x7f93a815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93ae7df700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93a80cc400 0x7f93a80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93ae7df700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93a80cd3f0 0x7f93a806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93ae7df700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a80cd880 con 0x7f93a80cc400 2023-10-16T10:00:32.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93ae7df700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a80cee70 con 0x7f93a80cd3f0 2023-10-16T10:00:32.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93ae7df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8159b10 con 0x7f93a8159730 2023-10-16T10:00:32.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93a77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93a80cc400 0x7f93a80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93a77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93a80cc400 0x7f93a80cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44366/0 (socket says 172.21.15.181:44366) 2023-10-16T10:00:32.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.274+0000 7f93a77fe700 1 -- 172.21.15.181:0/3711085895 learned_addr learned my addr 172.21.15.181:0/3711085895 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:32.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93acd7c700 1 --2- 172.21.15.181:0/3711085895 >> v2:172.21.15.190:3300/0 conn(0x7f93a80cd3f0 0x7f93a806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a7fff700 1 --2- 172.21.15.181:0/3711085895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a8159730 0x7f93a815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a77fe700 1 -- 172.21.15.181:0/3711085895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a8159730 msgr2=0x7f93a815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:32.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a77fe700 1 --2- 172.21.15.181:0/3711085895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a8159730 0x7f93a815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a77fe700 1 -- 172.21.15.181:0/3711085895 >> v2:172.21.15.190:3300/0 conn(0x7f93a80cd3f0 msgr2=0x7f93a806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:32.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a77fe700 1 --2- 172.21.15.181:0/3711085895 >> v2:172.21.15.190:3300/0 conn(0x7f93a80cd3f0 0x7f93a806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a77fe700 1 -- 172.21.15.181:0/3711085895 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80ce540 con 0x7f93a80cc400 2023-10-16T10:00:32.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a77fe700 1 --2- 172.21.15.181:0/3711085895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a80cc400 0x7f93a80cceb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f93a80cc1f0 tx=0x7f939c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dcca2e654b0093 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:32.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a6ffd700 1 -- 172.21.15.181:0/3711085895 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c011830 con 0x7f93a80cc400 2023-10-16T10:00:32.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a6ffd700 1 -- 172.21.15.181:0/3711085895 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c011990 con 0x7f93a80cc400 2023-10-16T10:00:32.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.275+0000 7f93a6ffd700 1 -- 172.21.15.181:0/3711085895 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c011d20 con 0x7f93a80cc400 2023-10-16T10:00:32.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 -- 172.21.15.181:0/3711085895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a80cc400 msgr2=0x7f93a80cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 --2- 172.21.15.181:0/3711085895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a80cc400 0x7f93a80cceb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f93a80cc1f0 tx=0x7f939c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 -- 172.21.15.181:0/3711085895 shutdown_connections 2023-10-16T10:00:32.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 --2- 172.21.15.181:0/3711085895 >> v2:172.21.15.190:3300/0 conn(0x7f93a80cd3f0 0x7f93a806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 --2- 172.21.15.181:0/3711085895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a8159730 0x7f93a815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 --2- 172.21.15.181:0/3711085895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a80cc400 0x7f93a80cceb0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 -- 172.21.15.181:0/3711085895 >> 172.21.15.181:0/3711085895 conn(0x7f93a80d04a0 msgr2=0x7f93a80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:32.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 -- 172.21.15.181:0/3711085895 shutdown_connections 2023-10-16T10:00:32.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.276+0000 7f93ae7df700 1 -- 172.21.15.181:0/3711085895 wait complete. 2023-10-16T10:00:32.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.277+0000 7f93ae7df700 1 Processor -- start 2023-10-16T10:00:32.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.277+0000 7f93ae7df700 1 -- start start 2023-10-16T10:00:32.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a80cc400 0x7f93a8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a80cd3f0 0x7f93a81507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a8159730 0x7f93a80c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81f1290 con 0x7f93a8159730 2023-10-16T10:00:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80b0730 con 0x7f93a80cc400 2023-10-16T10:00:32.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80b0a10 con 0x7f93a80cd3f0 2023-10-16T10:00:32.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a8159730 0x7f93a80c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a8159730 0x7f93a80c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44374/0 (socket says 172.21.15.181:44374) 2023-10-16T10:00:32.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 -- 172.21.15.181:0/109174675 learned_addr learned my addr 172.21.15.181:0/109174675 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:32.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93a77fe700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a80cd3f0 0x7f93a81507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 -- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a80cd3f0 msgr2=0x7f93a81507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a80cd3f0 0x7f93a81507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93a7fff700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a80cc400 0x7f93a8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 -- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a80cc400 msgr2=0x7f93a8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a80cc400 0x7f93a8150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 -- 172.21.15.181:0/109174675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c009550 con 0x7f93a8159730 2023-10-16T10:00:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93acd7c700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a8159730 0x7f93a80c8ee0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f93a000eed0 tx=0x7f93a000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93a4ff9700 1 -- 172.21.15.181:0/109174675 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0010040 con 0x7f93a8159730 2023-10-16T10:00:32.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93a4ff9700 1 -- 172.21.15.181:0/109174675 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a000bd30 con 0x7f93a8159730 2023-10-16T10:00:32.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a80b0cf0 con 0x7f93a8159730 2023-10-16T10:00:32.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93a4ff9700 1 -- 172.21.15.181:0/109174675 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0009ea0 con 0x7f93a8159730 2023-10-16T10:00:32.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.278+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a80b12b0 con 0x7f93a8159730 2023-10-16T10:00:32.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.279+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a8067d20 con 0x7f93a8159730 2023-10-16T10:00:32.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.281+0000 7f93a4ff9700 1 -- 172.21.15.181:0/109174675 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f93a0014640 con 0x7f93a8159730 2023-10-16T10:00:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.281+0000 7f93a4ff9700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f939406b020 0x7f939406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.281+0000 7f93a4ff9700 1 -- 172.21.15.181:0/109174675 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f93a008a550 con 0x7f93a8159730 2023-10-16T10:00:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.281+0000 7f93a7fff700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f939406b020 0x7f939406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.282+0000 7f93a7fff700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f939406b020 0x7f939406d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93980099f0 tx=0x7f9398008040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:32.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.285+0000 7f93a4ff9700 1 -- 172.21.15.181:0/109174675 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f93a00554e0 con 0x7f93a8159730 2023-10-16T10:00:32.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.480+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f93a80af7b0 con 0x7f93a8159730 2023-10-16T10:00:32.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.481+0000 7f93a4ff9700 1 -- 172.21.15.181:0/109174675 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7f93a0018060 con 0x7f93a8159730 2023-10-16T10:00:32.483 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:32.483 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:00:32.484 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:32.484 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T10:00:32.484 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T10:00:32.484 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:32.485 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:00:32.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f939406b020 msgr2=0x7f939406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f939406b020 0x7f939406d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93980099f0 tx=0x7f9398008040 comp rx=0 tx=0).stop 2023-10-16T10:00:32.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a8159730 msgr2=0x7f93a80c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a8159730 0x7f93a80c8ee0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f93a000eed0 tx=0x7f93a000b390 comp rx=0 tx=0).stop 2023-10-16T10:00:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 shutdown_connections 2023-10-16T10:00:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f939406b020 0x7f939406d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93a8159730 0x7f93a80c8ee0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a80cc400 0x7f93a8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 --2- 172.21.15.181:0/109174675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a80cd3f0 0x7f93a81507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 >> 172.21.15.181:0/109174675 conn(0x7f93a80d04a0 msgr2=0x7f93a80aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 shutdown_connections 2023-10-16T10:00:32.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.483+0000 7f93ae7df700 1 -- 172.21.15.181:0/109174675 wait complete. 2023-10-16T10:00:32.496 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:00:32.497 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:00:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.619+0000 7f78827f2700 1 Processor -- start 2023-10-16T10:00:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.619+0000 7f78827f2700 1 -- start start 2023-10-16T10:00:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f78827f2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cf5c0 0x7f787c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f78827f2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f787c0cc4a0 0x7f787c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f78827f2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c160240 con 0x7f787c06a8a0 2023-10-16T10:00:32.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f78827f2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c15bd80 con 0x7f787c0cc4a0 2023-10-16T10:00:32.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f78827f2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c06b6d0 con 0x7f787c0cf5c0 2023-10-16T10:00:32.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787bfff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f787c06a8a0 0x7f787c06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:39398/0 (socket says 172.21.15.181:39398) 2023-10-16T10:00:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787bfff700 1 -- 172.21.15.181:0/179617762 learned_addr learned my addr 172.21.15.181:0/179617762 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787b7fe700 1 --2- 172.21.15.181:0/179617762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cf5c0 0x7f787c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f7880d8f700 1 --2- 172.21.15.181:0/179617762 >> v2:172.21.15.190:3300/0 conn(0x7f787c0cc4a0 0x7f787c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787b7fe700 1 -- 172.21.15.181:0/179617762 >> v2:172.21.15.190:3300/0 conn(0x7f787c0cc4a0 msgr2=0x7f787c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787b7fe700 1 --2- 172.21.15.181:0/179617762 >> v2:172.21.15.190:3300/0 conn(0x7f787c0cc4a0 0x7f787c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787b7fe700 1 -- 172.21.15.181:0/179617762 >> v1:172.21.15.190:6789/0 conn(0x7f787c06a8a0 legacy=0x7f787c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787b7fe700 1 -- 172.21.15.181:0/179617762 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c15ff60 con 0x7f787c0cf5c0 2023-10-16T10:00:32.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787affd700 1 -- 172.21.15.181:0/179617762 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769361944 0 0) 0x7f787c160240 con 0x7f787c06a8a0 2023-10-16T10:00:32.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.620+0000 7f787b7fe700 1 --2- 172.21.15.181:0/179617762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cf5c0 0x7f787c0caaf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f786c009840 tx=0x7f786c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf445314e047282a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:32.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f787affd700 1 -- 172.21.15.181:0/179617762 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c011830 con 0x7f787c0cf5c0 2023-10-16T10:00:32.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f787affd700 1 -- 172.21.15.181:0/179617762 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c011990 con 0x7f787c0cf5c0 2023-10-16T10:00:32.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f787affd700 1 -- 172.21.15.181:0/179617762 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c011d20 con 0x7f787c0cf5c0 2023-10-16T10:00:32.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 -- 172.21.15.181:0/179617762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cf5c0 msgr2=0x7f787c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 --2- 172.21.15.181:0/179617762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cf5c0 0x7f787c0caaf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f786c009840 tx=0x7f786c009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:32.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 -- 172.21.15.181:0/179617762 shutdown_connections 2023-10-16T10:00:32.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 --2- 172.21.15.181:0/179617762 >> v2:172.21.15.190:3300/0 conn(0x7f787c0cc4a0 0x7f787c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 --2- 172.21.15.181:0/179617762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cf5c0 0x7f787c0caaf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 -- 172.21.15.181:0/179617762 >> 172.21.15.181:0/179617762 conn(0x7f787c0d04a0 msgr2=0x7f787c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:32.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 -- 172.21.15.181:0/179617762 shutdown_connections 2023-10-16T10:00:32.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.621+0000 7f78827f2700 1 -- 172.21.15.181:0/179617762 wait complete. 2023-10-16T10:00:32.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.622+0000 7f78827f2700 1 Processor -- start 2023-10-16T10:00:32.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f78827f2700 1 -- start start 2023-10-16T10:00:32.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f78827f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f787c06a8a0 0x7f787c0c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f78827f2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cc4a0 0x7f787c0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f78827f2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c0cf5c0 0x7f787c1f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f78827f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1f1370 con 0x7f787c06a8a0 2023-10-16T10:00:32.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f78827f2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0c6640 con 0x7f787c0cf5c0 2023-10-16T10:00:32.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f78827f2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0b2800 con 0x7f787c0cc4a0 2023-10-16T10:00:32.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.623+0000 7f787bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f787c06a8a0 0x7f787c0c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f787c06a8a0 0x7f787c0c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44384/0 (socket says 172.21.15.181:44384) 2023-10-16T10:00:32.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 -- 172.21.15.181:0/401708984 learned_addr learned my addr 172.21.15.181:0/401708984 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:32.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787b7fe700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cc4a0 0x7f787c0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f7880d8f700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c0cf5c0 0x7f787c1f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 -- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cc4a0 msgr2=0x7f787c0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cc4a0 0x7f787c0b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 -- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c0cf5c0 msgr2=0x7f787c1f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:32.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c0cf5c0 0x7f787c1f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:32.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 -- 172.21.15.181:0/401708984 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0094a0 con 0x7f787c06a8a0 2023-10-16T10:00:32.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f787bfff700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f787c06a8a0 0x7f787c0c5020 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f786400c4e0 tx=0x7f786400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:32.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f7878ff9700 1 -- 172.21.15.181:0/401708984 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864004cd0 con 0x7f787c06a8a0 2023-10-16T10:00:32.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f7878ff9700 1 -- 172.21.15.181:0/401708984 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7864004e30 con 0x7f787c06a8a0 2023-10-16T10:00:32.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f7878ff9700 1 -- 172.21.15.181:0/401708984 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640072f0 con 0x7f787c06a8a0 2023-10-16T10:00:32.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c0c5600 con 0x7f787c06a8a0 2023-10-16T10:00:32.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.624+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c0c5bc0 con 0x7f787c06a8a0 2023-10-16T10:00:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.625+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c0cdc30 con 0x7f787c06a8a0 2023-10-16T10:00:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.626+0000 7f7878ff9700 1 -- 172.21.15.181:0/401708984 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7864007450 con 0x7f787c06a8a0 2023-10-16T10:00:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.626+0000 7f7878ff9700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f785c06b020 0x7f785c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.626+0000 7f7878ff9700 1 -- 172.21.15.181:0/401708984 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f7864089180 con 0x7f787c06a8a0 2023-10-16T10:00:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.626+0000 7f787b7fe700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f785c06b020 0x7f785c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:32.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.626+0000 7f787b7fe700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f785c06b020 0x7f785c06d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f786c000c00 tx=0x7f786c008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:32.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.631+0000 7f7878ff9700 1 -- 172.21.15.181:0/401708984 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7864054380 con 0x7f787c06a8a0 2023-10-16T10:00:32.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.827+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f787c071e70 con 0x7f787c06a8a0 2023-10-16T10:00:32.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:32.998+0000 7f7878ff9700 1 -- 172.21.15.181:0/401708984 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7f78640579a0 con 0x7f787c06a8a0 2023-10-16T10:00:33.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f785c06b020 msgr2=0x7f785c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:33.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f785c06b020 0x7f785c06d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f786c000c00 tx=0x7f786c008030 comp rx=0 tx=0).stop 2023-10-16T10:00:33.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f787c06a8a0 msgr2=0x7f787c0c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:33.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f787c06a8a0 0x7f787c0c5020 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f786400c4e0 tx=0x7f786400c7f0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 shutdown_connections 2023-10-16T10:00:33.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f785c06b020 0x7f785c06d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f787c0cf5c0 0x7f787c1f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f787c06a8a0 0x7f787c0c5020 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 --2- 172.21.15.181:0/401708984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f787c0cc4a0 0x7f787c0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 >> 172.21.15.181:0/401708984 conn(0x7f787c0d04a0 msgr2=0x7f787c157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:33.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 shutdown_connections 2023-10-16T10:00:33.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.000+0000 7f78827f2700 1 -- 172.21.15.181:0/401708984 wait complete. 2023-10-16T10:00:33.005 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T10:00:33.017 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:00:33.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.143+0000 7f57bddb7700 1 Processor -- start 2023-10-16T10:00:33.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.143+0000 7f57bddb7700 1 -- start start 2023-10-16T10:00:33.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57bddb7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:33.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57bddb7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57b806ade0 0x7f57b81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:33.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57bddb7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b806b230 con 0x7f57b815da70 2023-10-16T10:00:33.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57bddb7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b80cef00 con 0x7f57b806ade0 2023-10-16T10:00:33.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57bddb7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80cec20 con 0x7f57b8072760 2023-10-16T10:00:33.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57b806ade0 0x7f57b81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:33.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57b806ade0 0x7f57b81615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44396/0 (socket says 172.21.15.181:44396) 2023-10-16T10:00:33.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 -- 172.21.15.181:0/3858295182 learned_addr learned my addr 172.21.15.181:0/3858295182 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:33.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b6ffd700 1 --2- 172.21.15.181:0/3858295182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:33.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 -- 172.21.15.181:0/3858295182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 msgr2=0x7f57b806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:33.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 --2- 172.21.15.181:0/3858295182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 -- 172.21.15.181:0/3858295182 >> v1:172.21.15.190:6789/0 conn(0x7f57b815da70 legacy=0x7f57b80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:33.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 -- 172.21.15.181:0/3858295182 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8071c80 con 0x7f57b806ade0 2023-10-16T10:00:33.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.144+0000 7f57b7fff700 1 --2- 172.21.15.181:0/3858295182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b806ade0 0x7f57b81615f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009e50 tx=0x7f57b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38345ebf0bc6d7c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:33.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57b67fc700 1 -- 172.21.15.181:0/3858295182 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000e070 con 0x7f57b806ade0 2023-10-16T10:00:33.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57b67fc700 1 -- 172.21.15.181:0/3858295182 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000b910 con 0x7f57b806ade0 2023-10-16T10:00:33.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57b67fc700 1 -- 172.21.15.181:0/3858295182 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000bcc0 con 0x7f57b806ade0 2023-10-16T10:00:33.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 -- 172.21.15.181:0/3858295182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b806ade0 msgr2=0x7f57b81615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:33.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 --2- 172.21.15.181:0/3858295182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b806ade0 0x7f57b81615f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009e50 tx=0x7f57b000b330 comp rx=0 tx=0).stop 2023-10-16T10:00:33.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 -- 172.21.15.181:0/3858295182 shutdown_connections 2023-10-16T10:00:33.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 --2- 172.21.15.181:0/3858295182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b806ade0 0x7f57b81615f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 --2- 172.21.15.181:0/3858295182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 -- 172.21.15.181:0/3858295182 >> 172.21.15.181:0/3858295182 conn(0x7f57b80cd010 msgr2=0x7f57b80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:33.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 -- 172.21.15.181:0/3858295182 shutdown_connections 2023-10-16T10:00:33.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.145+0000 7f57bddb7700 1 -- 172.21.15.181:0/3858295182 wait complete. 2023-10-16T10:00:33.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.146+0000 7f57bddb7700 1 Processor -- start 2023-10-16T10:00:33.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57bddb7700 1 -- start start 2023-10-16T10:00:33.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57bddb7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b806ade0 0x7f57b80ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:33.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57bddb7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b80b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57bddb7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b815da70 0x7f57b80c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57bddb7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80aeb30 con 0x7f57b815da70 2023-10-16T10:00:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57bddb7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80aee10 con 0x7f57b806ade0 2023-10-16T10:00:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57bddb7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80af0f0 con 0x7f57b8072760 2023-10-16T10:00:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b815da70 0x7f57b80c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b815da70 0x7f57b80c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44402/0 (socket says 172.21.15.181:44402) 2023-10-16T10:00:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.147+0000 7f57b7fff700 1 -- 172.21.15.181:0/1288213917 learned_addr learned my addr 172.21.15.181:0/1288213917 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b6ffd700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b80b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b77fe700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b806ade0 0x7f57b80ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:33.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b7fff700 1 -- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 msgr2=0x7f57b80b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:33.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b7fff700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b80b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b7fff700 1 -- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b806ade0 msgr2=0x7f57b80ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:33.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b7fff700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b806ade0 0x7f57b80ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:33.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b7fff700 1 -- 172.21.15.181:0/1288213917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0009b30 con 0x7f57b815da70 2023-10-16T10:00:33.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57b7fff700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b815da70 0x7f57b80c7660 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f57b806f140 tx=0x7f57b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57affff700 1 -- 172.21.15.181:0/1288213917 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0004800 con 0x7f57b815da70 2023-10-16T10:00:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57affff700 1 -- 172.21.15.181:0/1288213917 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0004960 con 0x7f57b815da70 2023-10-16T10:00:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b80af3d0 con 0x7f57b815da70 2023-10-16T10:00:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57affff700 1 -- 172.21.15.181:0/1288213917 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b001c680 con 0x7f57b815da70 2023-10-16T10:00:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.148+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b80ae820 con 0x7f57b815da70 2023-10-16T10:00:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.149+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b80c4680 con 0x7f57b815da70 2023-10-16T10:00:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.150+0000 7f57affff700 1 -- 172.21.15.181:0/1288213917 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f57b001c7e0 con 0x7f57b815da70 2023-10-16T10:00:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.150+0000 7f57affff700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f579806b020 0x7f579806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.150+0000 7f57affff700 1 -- 172.21.15.181:0/1288213917 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5651+0+0 (secure 0 0 0) 0x7f57b008b3e0 con 0x7f57b815da70 2023-10-16T10:00:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.150+0000 7f57b77fe700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f579806b020 0x7f579806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.150+0000 7f57b77fe700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f579806b020 0x7f579806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57a0007fd0 tx=0x7f57a0009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.155+0000 7f57affff700 1 -- 172.21.15.181:0/1288213917 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f57b0056330 con 0x7f57b815da70 2023-10-16T10:00:33.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:33.345+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f57b806f6c0 con 0x7f57b815da70 2023-10-16T10:00:34.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.007+0000 7f57affff700 1 -- 172.21.15.181:0/1288213917 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7f57b0059950 con 0x7f57b815da70 2023-10-16T10:00:34.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.009+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f579806b020 msgr2=0x7f579806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:34.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.009+0000 7f57bddb7700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f579806b020 0x7f579806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57a0007fd0 tx=0x7f57a0009040 comp rx=0 tx=0).stop 2023-10-16T10:00:34.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.009+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b815da70 msgr2=0x7f57b80c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:34.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.009+0000 7f57bddb7700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b815da70 0x7f57b80c7660 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f57b806f140 tx=0x7f57b0004ad0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 shutdown_connections 2023-10-16T10:00:34.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f579806b020 0x7f579806d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b815da70 0x7f57b80c7660 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b806ade0 0x7f57b80ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 --2- 172.21.15.181:0/1288213917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57b8072760 0x7f57b80b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 >> 172.21.15.181:0/1288213917 conn(0x7f57b80cd010 msgr2=0x7f57b80b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:34.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 shutdown_connections 2023-10-16T10:00:34.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.010+0000 7f57bddb7700 1 -- 172.21.15.181:0/1288213917 wait complete. 2023-10-16T10:00:34.025 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T10:00:34.036 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:00:34.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.159+0000 7f1a01266700 1 Processor -- start 2023-10-16T10:00:34.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.159+0000 7f1a01266700 1 -- start start 2023-10-16T10:00:34.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f1a01266700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:34.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f1a01266700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f19fc0cf890 0x7f19fc0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:34.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f1a01266700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19fc0d0cb0 con 0x7f19fc0ce410 2023-10-16T10:00:34.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f1a01266700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19fc1624b0 con 0x7f19fc0cf890 2023-10-16T10:00:34.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f1a01266700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc0ce880 con 0x7f19fc15d740 2023-10-16T10:00:34.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f19fc0ce410 0x7f19fc0ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55488/0 (socket says 172.21.15.181:55488) 2023-10-16T10:00:34.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fb7fe700 1 -- 172.21.15.181:0/1336516289 learned_addr learned my addr 172.21.15.181:0/1336516289 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:34.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fa7fc700 1 --2- 172.21.15.181:0/1336516289 >> v2:172.21.15.190:3300/0 conn(0x7f19fc0cf890 0x7f19fc0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:34.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19faffd700 1 --2- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:34.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19f9ffb700 1 -- 172.21.15.181:0/1336516289 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617589946 0 0) 0x7f19fc0d0cb0 con 0x7f19fc0ce410 2023-10-16T10:00:34.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19f9ffb700 1 -- 172.21.15.181:0/1336516289 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e40035b0 con 0x7f19fc0ce410 2023-10-16T10:00:34.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fa7fc700 1 -- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 msgr2=0x7f19fc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fa7fc700 1 --2- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fa7fc700 1 -- 172.21.15.181:0/1336516289 >> v1:172.21.15.181:6789/0 conn(0x7f19fc0ce410 legacy=0x7f19fc0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fa7fc700 1 -- 172.21.15.181:0/1336516289 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc0d00a0 con 0x7f19fc0cf890 2023-10-16T10:00:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19faffd700 1 --2- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.160+0000 7f19fa7fc700 1 --2- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 0x7f19fc0cded0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009870 tx=0x7f19f0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43a7bcc0643f307 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f19f9ffb700 1 -- 172.21.15.181:0/1336516289 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0011830 con 0x7f19fc0cf890 2023-10-16T10:00:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f19f9ffb700 1 -- 172.21.15.181:0/1336516289 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f0011990 con 0x7f19fc0cf890 2023-10-16T10:00:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f19f9ffb700 1 -- 172.21.15.181:0/1336516289 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0011d00 con 0x7f19fc0cf890 2023-10-16T10:00:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 -- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 msgr2=0x7f19fc0cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 --2- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 0x7f19fc0cded0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009870 tx=0x7f19f0009b80 comp rx=0 tx=0).stop 2023-10-16T10:00:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 -- 172.21.15.181:0/1336516289 shutdown_connections 2023-10-16T10:00:34.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 --2- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 --2- 172.21.15.181:0/1336516289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 0x7f19fc0cded0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 -- 172.21.15.181:0/1336516289 >> 172.21.15.181:0/1336516289 conn(0x7f19fc09a930 msgr2=0x7f19fc0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:34.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 -- 172.21.15.181:0/1336516289 shutdown_connections 2023-10-16T10:00:34.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.161+0000 7f1a01266700 1 -- 172.21.15.181:0/1336516289 wait complete. 2023-10-16T10:00:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 Processor -- start 2023-10-16T10:00:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 -- start start 2023-10-16T10:00:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19fc0ce410 0x7f19fc14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 0x7f19fc1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc14a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc1f6790 con 0x7f19fc0ce410 2023-10-16T10:00:34.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc14c800 con 0x7f19fc0cf890 2023-10-16T10:00:34.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f1a01266700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc160bf0 con 0x7f19fc15d740 2023-10-16T10:00:34.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f19faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19fc0ce410 0x7f19fc14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:34.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f19faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19fc0ce410 0x7f19fc14c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44412/0 (socket says 172.21.15.181:44412) 2023-10-16T10:00:34.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.163+0000 7f19faffd700 1 -- 172.21.15.181:0/822611197 learned_addr learned my addr 172.21.15.181:0/822611197 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19fa7fc700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 0x7f19fc1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19fb7fe700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc14a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19faffd700 1 -- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 msgr2=0x7f19fc14a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19faffd700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc14a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19faffd700 1 -- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 msgr2=0x7f19fc1606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19faffd700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 0x7f19fc1606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19faffd700 1 -- 172.21.15.181:0/822611197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0094a0 con 0x7f19fc0ce410 2023-10-16T10:00:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19faffd700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19fc0ce410 0x7f19fc14c2c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00ebe0 tx=0x7f19ec00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19e37fe700 1 -- 172.21.15.181:0/822611197 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec009760 con 0x7f19fc0ce410 2023-10-16T10:00:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19e37fe700 1 -- 172.21.15.181:0/822611197 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec00c850 con 0x7f19fc0ce410 2023-10-16T10:00:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f00094a0 con 0x7f19fc0ce410 2023-10-16T10:00:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f19e37fe700 1 -- 172.21.15.181:0/822611197 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00cbe0 con 0x7f19fc0ce410 2023-10-16T10:00:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.164+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc14a7f0 con 0x7f19fc0ce410 2023-10-16T10:00:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.165+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19fc04ebb0 con 0x7f19fc0ce410 2023-10-16T10:00:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.166+0000 7f19e37fe700 1 -- 172.21.15.181:0/822611197 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f19ec00cd40 con 0x7f19fc0ce410 2023-10-16T10:00:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.166+0000 7f19e37fe700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f19e806b020 0x7f19e806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.166+0000 7f19fa7fc700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f19e806b020 0x7f19e806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.166+0000 7f19e37fe700 1 -- 172.21.15.181:0/822611197 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6033+0+0 (secure 0 0 0) 0x7f19ec089f20 con 0x7f19fc0ce410 2023-10-16T10:00:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.167+0000 7f19fa7fc700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f19e806b020 0x7f19e806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19f00090b0 tx=0x7f19f0008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.171+0000 7f19e37fe700 1 -- 172.21.15.181:0/822611197 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f19ec054ee0 con 0x7f19fc0ce410 2023-10-16T10:00:34.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:34.409+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f19fc0d0630 con 0x7f19fc0ce410 2023-10-16T10:00:35.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.022+0000 7f19e37fe700 1 -- 172.21.15.181:0/822611197 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7f19ec058500 con 0x7f19fc0ce410 2023-10-16T10:00:35.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.032+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f19e806b020 msgr2=0x7f19e806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.032+0000 7f1a01266700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f19e806b020 0x7f19e806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19f00090b0 tx=0x7f19f0008030 comp rx=0 tx=0).stop 2023-10-16T10:00:35.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.032+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19fc0ce410 msgr2=0x7f19fc14c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.032+0000 7f1a01266700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19fc0ce410 0x7f19fc14c2c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00ebe0 tx=0x7f19ec00eef0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.034+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 shutdown_connections 2023-10-16T10:00:35.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.034+0000 7f1a01266700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f19e806b020 0x7f19e806d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.034+0000 7f1a01266700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19fc15d740 0x7f19fc14a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.034+0000 7f1a01266700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19fc0ce410 0x7f19fc14c2c0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.034+0000 7f1a01266700 1 --2- 172.21.15.181:0/822611197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19fc0cf890 0x7f19fc1606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.034+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 >> 172.21.15.181:0/822611197 conn(0x7f19fc09a930 msgr2=0x7f19fc15ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:35.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.039+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 shutdown_connections 2023-10-16T10:00:35.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.040+0000 7f1a01266700 1 -- 172.21.15.181:0/822611197 wait complete. 2023-10-16T10:00:35.042 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 12 and data pool 13 2023-10-16T10:00:35.056 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:35.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.178+0000 7f69db0c7700 1 Processor -- start 2023-10-16T10:00:35.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.178+0000 7f69db0c7700 1 -- start start 2023-10-16T10:00:35.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69db0c7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f69d40cf4e0 0x7f69d40665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69db0c7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d4066c60 con 0x7f69d40ca2e0 2023-10-16T10:00:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69db0c7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d406c1a0 con 0x7f69d415db70 2023-10-16T10:00:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69db0c7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d40cda10 con 0x7f69d40cf4e0 2023-10-16T10:00:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69d9664700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f69d40ca2e0 0x7f69d40ca770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55514/0 (socket says 172.21.15.181:55514) 2023-10-16T10:00:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69d9664700 1 -- 172.21.15.181:0/2330249778 learned_addr learned my addr 172.21.15.181:0/2330249778 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69d3fff700 1 --2- 172.21.15.181:0/2330249778 >> v2:172.21.15.190:3300/0 conn(0x7f69d40cf4e0 0x7f69d40665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636547401 0 0) 0x7f69d4066c60 con 0x7f69d40ca2e0 2023-10-16T10:00:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.179+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c40035b0 con 0x7f69d40ca2e0 2023-10-16T10:00:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377943337 0 0) 0x7f69d406c1a0 con 0x7f69d415db70 2023-10-16T10:00:35.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d4066c60 con 0x7f69d415db70 2023-10-16T10:00:35.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 590007548 0 0) 0x7f69c40035b0 con 0x7f69d40ca2e0 2023-10-16T10:00:35.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 >> v2:172.21.15.190:3300/0 conn(0x7f69d40cf4e0 msgr2=0x7f69d40665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:35.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 --2- 172.21.15.181:0/2330249778 >> v2:172.21.15.190:3300/0 conn(0x7f69d40cf4e0 0x7f69d40665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 >> v1:172.21.15.190:6789/0 conn(0x7f69d415db70 legacy=0x7f69d40cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d406b3e0 con 0x7f69d40ca2e0 2023-10-16T10:00:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f69cc001a40 con 0x7f69d40ca2e0 2023-10-16T10:00:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 >> v1:172.21.15.181:6789/0 conn(0x7f69d40ca2e0 legacy=0x7f69d40ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4003b90 0x7f69c4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c4004850 0x7f69c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4005210 0x7f69c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40cda10 con 0x7f69c4004850 2023-10-16T10:00:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c400c140 con 0x7f69c4003b90 2023-10-16T10:00:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c400c300 con 0x7f69c4005210 2023-10-16T10:00:35.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d3fff700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c4004850 0x7f69c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d8e63700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4003b90 0x7f69c4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d9664700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4005210 0x7f69c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d3fff700 1 -- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4005210 msgr2=0x7f69c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:35.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d3fff700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4005210 0x7f69c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d3fff700 1 -- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4003b90 msgr2=0x7f69c4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:35.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d3fff700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4003b90 0x7f69c4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d3fff700 1 -- 172.21.15.181:0/2330249778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40035b0 con 0x7f69c4004850 2023-10-16T10:00:35.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d9664700 1 -- 172.21.15.181:0/2330249778 reap_dead start 2023-10-16T10:00:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d3fff700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c4004850 0x7f69c4004cd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009b60 tx=0x7f69c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0d32f81ae9088f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000e6c0 con 0x7f69c4004850 2023-10-16T10:00:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c000e820 con 0x7f69c4004850 2023-10-16T10:00:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.180+0000 7f69d37fe700 1 -- 172.21.15.181:0/2330249778 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000eb90 con 0x7f69c4004850 2023-10-16T10:00:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.181+0000 7f69db0c7700 1 -- 172.21.15.181:0/2330249778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c4004850 msgr2=0x7f69c4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.181+0000 7f69db0c7700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c4004850 0x7f69c4004cd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009b60 tx=0x7f69c4005880 comp rx=0 tx=0).stop 2023-10-16T10:00:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.181+0000 7f69db0c7700 1 -- 172.21.15.181:0/2330249778 shutdown_connections 2023-10-16T10:00:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.181+0000 7f69db0c7700 1 --2- 172.21.15.181:0/2330249778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69c4004850 0x7f69c4004cd0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.181+0000 7f69db0c7700 1 -- 172.21.15.181:0/2330249778 >> 172.21.15.181:0/2330249778 conn(0x7f69d40b1780 msgr2=0x7f69d40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.181+0000 7f69db0c7700 1 -- 172.21.15.181:0/2330249778 shutdown_connections 2023-10-16T10:00:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.181+0000 7f69db0c7700 1 -- 172.21.15.181:0/2330249778 wait complete. 2023-10-16T10:00:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.183+0000 7f69db0c7700 1 Processor -- start 2023-10-16T10:00:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.183+0000 7f69db0c7700 1 -- start start 2023-10-16T10:00:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69db0c7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4004850 0x7f69d4159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69db0c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69d415df50 0x7f69d40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69db0c7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d409e120 0x7f69d40c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69db0c7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40c7e70 con 0x7f69d415df50 2023-10-16T10:00:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69db0c7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d415e3d0 con 0x7f69c4004850 2023-10-16T10:00:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69db0c7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d415e650 con 0x7f69d409e120 2023-10-16T10:00:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69d415df50 0x7f69d40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69d415df50 0x7f69d40c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44448/0 (socket says 172.21.15.181:44448) 2023-10-16T10:00:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 -- 172.21.15.181:0/501074053 learned_addr learned my addr 172.21.15.181:0/501074053 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d8e63700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4004850 0x7f69d4159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d9664700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d409e120 0x7f69d40c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 -- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d409e120 msgr2=0x7f69d40c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d409e120 0x7f69d40c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 -- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4004850 msgr2=0x7f69d4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4004850 0x7f69d4159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 -- 172.21.15.181:0/501074053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d415e930 con 0x7f69d415df50 2023-10-16T10:00:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d3fff700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69d415df50 0x7f69d40c7930 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f69c000e4b0 tx=0x7f69c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d17fa700 1 -- 172.21.15.181:0/501074053 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000e6c0 con 0x7f69d415df50 2023-10-16T10:00:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d17fa700 1 -- 172.21.15.181:0/501074053 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0003ad0 con 0x7f69d415df50 2023-10-16T10:00:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d415ec10 con 0x7f69d415df50 2023-10-16T10:00:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.184+0000 7f69d17fa700 1 -- 172.21.15.181:0/501074053 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0003e80 con 0x7f69d415df50 2023-10-16T10:00:35.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.185+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d415f1d0 con 0x7f69d415df50 2023-10-16T10:00:35.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.185+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d415f540 con 0x7f69d415df50 2023-10-16T10:00:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.186+0000 7f69d17fa700 1 -- 172.21.15.181:0/501074053 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f69c0013470 con 0x7f69d415df50 2023-10-16T10:00:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.186+0000 7f69d17fa700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f69b806b020 0x7f69b806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.186+0000 7f69d17fa700 1 -- 172.21.15.181:0/501074053 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f69c008b060 con 0x7f69d415df50 2023-10-16T10:00:35.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.186+0000 7f69d8e63700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f69b806b020 0x7f69b806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.187+0000 7f69d8e63700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f69b806b020 0x7f69b806d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f69d40a0d00 tx=0x7f69c8009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.191+0000 7f69d17fa700 1 -- 172.21.15.181:0/501074053 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f69c0056000 con 0x7f69d415df50 2023-10-16T10:00:35.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.382+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69d4067d20 con 0x7f69d415df50 2023-10-16T10:00:35.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.384+0000 7f69d17fa700 1 -- 172.21.15.181:0/501074053 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+17264 (secure 0 0 0) 0x7f69c0059620 con 0x7f69d415df50 2023-10-16T10:00:35.386 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:35.386 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:34.410820+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:00:32.847964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:00:33.365892+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:35.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f69b806b020 msgr2=0x7f69b806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f69b806b020 0x7f69b806d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f69d40a0d00 tx=0x7f69c8009040 comp rx=0 tx=0).stop 2023-10-16T10:00:35.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69d415df50 msgr2=0x7f69d40c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69d415df50 0x7f69d40c7930 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f69c000e4b0 tx=0x7f69c00047c0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 shutdown_connections 2023-10-16T10:00:35.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f69b806b020 0x7f69b806d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d409e120 0x7f69d40c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c4004850 0x7f69d4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 --2- 172.21.15.181:0/501074053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69d415df50 0x7f69d40c7930 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 >> 172.21.15.181:0/501074053 conn(0x7f69d40b1780 msgr2=0x7f69d40cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:35.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 shutdown_connections 2023-10-16T10:00:35.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.386+0000 7f69db0c7700 1 -- 172.21.15.181:0/501074053 wait complete. 2023-10-16T10:00:35.400 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:35.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.518+0000 7f952c2ea700 1 Processor -- start 2023-10-16T10:00:35.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.518+0000 7f952c2ea700 1 -- start start 2023-10-16T10:00:35.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.519+0000 7f952c2ea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f952415b7a0 0x7f952415fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.519+0000 7f952c2ea700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95240cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.519+0000 7f952c2ea700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9524071030 con 0x7f95240cf8d0 2023-10-16T10:00:35.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.519+0000 7f952c2ea700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9524160300 con 0x7f952415b7a0 2023-10-16T10:00:35.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.519+0000 7f952c2ea700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95240cfd60 con 0x7f9524070bb0 2023-10-16T10:00:35.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a086700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f952415b7a0 0x7f952415fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a887700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f95240cf8d0 0x7f95240cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55536/0 (socket says 172.21.15.181:55536) 2023-10-16T10:00:35.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a887700 1 -- 172.21.15.181:0/486944828 learned_addr learned my addr 172.21.15.181:0/486944828 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:35.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f9529885700 1 --2- 172.21.15.181:0/486944828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95240cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f9529084700 1 -- 172.21.15.181:0/486944828 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889343748 0 0) 0x7f9524071030 con 0x7f95240cf8d0 2023-10-16T10:00:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f9529084700 1 -- 172.21.15.181:0/486944828 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95100035b0 con 0x7f95240cf8d0 2023-10-16T10:00:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a086700 1 -- 172.21.15.181:0/486944828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 msgr2=0x7f95240cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a086700 1 --2- 172.21.15.181:0/486944828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95240cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a086700 1 -- 172.21.15.181:0/486944828 >> v1:172.21.15.181:6789/0 conn(0x7f95240cf8d0 legacy=0x7f95240cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a086700 1 -- 172.21.15.181:0/486944828 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952415bb80 con 0x7f952415b7a0 2023-10-16T10:00:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f952a086700 1 --2- 172.21.15.181:0/486944828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415fdc0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9514009840 tx=0x7f9514009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad45708a9eb833f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f9529084700 1 -- 172.21.15.181:0/486944828 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514011830 con 0x7f952415b7a0 2023-10-16T10:00:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f9529084700 1 -- 172.21.15.181:0/486944828 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514011990 con 0x7f952415b7a0 2023-10-16T10:00:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.520+0000 7f9529084700 1 -- 172.21.15.181:0/486944828 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514011d20 con 0x7f952415b7a0 2023-10-16T10:00:35.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 -- 172.21.15.181:0/486944828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 msgr2=0x7f952415fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 --2- 172.21.15.181:0/486944828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415fdc0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9514009840 tx=0x7f9514009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 -- 172.21.15.181:0/486944828 shutdown_connections 2023-10-16T10:00:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 --2- 172.21.15.181:0/486944828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415fdc0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 --2- 172.21.15.181:0/486944828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95240cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 -- 172.21.15.181:0/486944828 >> 172.21.15.181:0/486944828 conn(0x7f952406aed0 msgr2=0x7f95240b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 -- 172.21.15.181:0/486944828 shutdown_connections 2023-10-16T10:00:35.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.521+0000 7f952c2ea700 1 -- 172.21.15.181:0/486944828 wait complete. 2023-10-16T10:00:35.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.522+0000 7f952c2ea700 1 Processor -- start 2023-10-16T10:00:35.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.522+0000 7f952c2ea700 1 -- start start 2023-10-16T10:00:35.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952c2ea700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95241544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952c2ea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95240cf8d0 0x7f95241f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952c2ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952c2ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952415cfd0 con 0x7f952415b7a0 2023-10-16T10:00:35.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952c2ea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952415c3c0 con 0x7f95240cf8d0 2023-10-16T10:00:35.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952c2ea700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952415c6a0 con 0x7f9524070bb0 2023-10-16T10:00:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44464/0 (socket says 172.21.15.181:44464) 2023-10-16T10:00:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 -- 172.21.15.181:0/2401988614 learned_addr learned my addr 172.21.15.181:0/2401988614 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a086700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95241544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 -- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 msgr2=0x7f95241544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95241544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 -- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95240cf8d0 msgr2=0x7f95241f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95240cf8d0 0x7f95241f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 -- 172.21.15.181:0/2401988614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95140094a0 con 0x7f952415b7a0 2023-10-16T10:00:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a086700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95241544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:35.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f952a887700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415ead0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f951c00cb40 tx=0x7f951c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f951affd700 1 -- 172.21.15.181:0/2401988614 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00ab50 con 0x7f952415b7a0 2023-10-16T10:00:35.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.523+0000 7f951affd700 1 -- 172.21.15.181:0/2401988614 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c00acb0 con 0x7f952415b7a0 2023-10-16T10:00:35.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.524+0000 7f951affd700 1 -- 172.21.15.181:0/2401988614 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c012620 con 0x7f952415b7a0 2023-10-16T10:00:35.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.524+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952415c980 con 0x7f952415b7a0 2023-10-16T10:00:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.524+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95241f1760 con 0x7f952415b7a0 2023-10-16T10:00:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.525+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952415f840 con 0x7f952415b7a0 2023-10-16T10:00:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.525+0000 7f951affd700 1 -- 172.21.15.181:0/2401988614 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f951c01b3b0 con 0x7f952415b7a0 2023-10-16T10:00:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.525+0000 7f951affd700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f950806b020 0x7f950806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.525+0000 7f951affd700 1 -- 172.21.15.181:0/2401988614 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f951c089680 con 0x7f952415b7a0 2023-10-16T10:00:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.525+0000 7f952a086700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f950806b020 0x7f950806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.526+0000 7f952a086700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f950806b020 0x7f950806d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95140090b0 tx=0x7f9514008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.530+0000 7f951affd700 1 -- 172.21.15.181:0/2401988614 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f952415f840 con 0x7f952415b7a0 2023-10-16T10:00:35.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.721+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f952415f840 con 0x7f952415b7a0 2023-10-16T10:00:35.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.723+0000 7f951affd700 1 -- 172.21.15.181:0/2401988614 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+17264 (secure 0 0 0) 0x7f952415f840 con 0x7f952415b7a0 2023-10-16T10:00:35.724 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:35.725 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:34.410820+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:00:32.847964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:00:33.365892+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:35.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f950806b020 msgr2=0x7f950806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f950806b020 0x7f950806d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95140090b0 tx=0x7f9514008030 comp rx=0 tx=0).stop 2023-10-16T10:00:35.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 msgr2=0x7f952415ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415ead0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f951c00cb40 tx=0x7f951c00cf00 comp rx=0 tx=0).stop 2023-10-16T10:00:35.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 shutdown_connections 2023-10-16T10:00:35.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f950806b020 0x7f950806d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9524070bb0 0x7f95241544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f952415b7a0 0x7f952415ead0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 --2- 172.21.15.181:0/2401988614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95240cf8d0 0x7f95241f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 >> 172.21.15.181:0/2401988614 conn(0x7f952406aed0 msgr2=0x7f9524158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:35.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 shutdown_connections 2023-10-16T10:00:35.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.725+0000 7f952c2ea700 1 -- 172.21.15.181:0/2401988614 wait complete. 2023-10-16T10:00:35.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:00:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.860+0000 7ff04a5ac700 1 Processor -- start 2023-10-16T10:00:35.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff04a5ac700 1 -- start start 2023-10-16T10:00:35.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff04a5ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff0440b2010 0x7ff0440cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff04a5ac700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0440ce680 0x7ff0440ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff04a5ac700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0440ce1f0 con 0x7ff044071be0 2023-10-16T10:00:35.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff04a5ac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff044072090 con 0x7ff0440b2010 2023-10-16T10:00:35.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff04a5ac700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0440ccda0 con 0x7ff0440ce680 2023-10-16T10:00:35.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff043fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff0440b2010 0x7ff0440cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff0437fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff044071be0 0x7ff0440b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55546/0 (socket says 172.21.15.181:55546) 2023-10-16T10:00:35.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.861+0000 7ff0437fe700 1 -- 172.21.15.181:0/4789167 learned_addr learned my addr 172.21.15.181:0/4789167 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:35.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff048b49700 1 --2- 172.21.15.181:0/4789167 >> v2:172.21.15.190:3300/0 conn(0x7ff0440ce680 0x7ff0440ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff043fff700 1 -- 172.21.15.181:0/4789167 >> v2:172.21.15.190:3300/0 conn(0x7ff0440ce680 msgr2=0x7ff0440ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:35.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff043fff700 1 --2- 172.21.15.181:0/4789167 >> v2:172.21.15.190:3300/0 conn(0x7ff0440ce680 0x7ff0440ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff043fff700 1 -- 172.21.15.181:0/4789167 >> v1:172.21.15.181:6789/0 conn(0x7ff044071be0 legacy=0x7ff0440b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff042ffd700 1 -- 172.21.15.181:0/4789167 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143498382 0 0) 0x7ff0440ce1f0 con 0x7ff044071be0 2023-10-16T10:00:35.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff043fff700 1 -- 172.21.15.181:0/4789167 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0440727e0 con 0x7ff0440b2010 2023-10-16T10:00:35.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff043fff700 1 --2- 172.21.15.181:0/4789167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0440b2010 0x7ff0440cc6e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff044071a30 tx=0x7ff02c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71458ceb5560fc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff042ffd700 1 -- 172.21.15.181:0/4789167 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c011830 con 0x7ff0440b2010 2023-10-16T10:00:35.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff042ffd700 1 -- 172.21.15.181:0/4789167 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c011990 con 0x7ff0440b2010 2023-10-16T10:00:35.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff042ffd700 1 -- 172.21.15.181:0/4789167 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c011d20 con 0x7ff0440b2010 2023-10-16T10:00:35.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff04a5ac700 1 -- 172.21.15.181:0/4789167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0440b2010 msgr2=0x7ff0440cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.862+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/4789167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0440b2010 0x7ff0440cc6e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff044071a30 tx=0x7ff02c009b20 comp rx=0 tx=0).stop 2023-10-16T10:00:35.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.863+0000 7ff04a5ac700 1 -- 172.21.15.181:0/4789167 shutdown_connections 2023-10-16T10:00:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.863+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/4789167 >> v2:172.21.15.190:3300/0 conn(0x7ff0440ce680 0x7ff0440ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.863+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/4789167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0440b2010 0x7ff0440cc6e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.863+0000 7ff04a5ac700 1 -- 172.21.15.181:0/4789167 >> 172.21.15.181:0/4789167 conn(0x7ff0440cd010 msgr2=0x7ff0440cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.863+0000 7ff04a5ac700 1 -- 172.21.15.181:0/4789167 shutdown_connections 2023-10-16T10:00:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.863+0000 7ff04a5ac700 1 -- 172.21.15.181:0/4789167 wait complete. 2023-10-16T10:00:35.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.864+0000 7ff04a5ac700 1 Processor -- start 2023-10-16T10:00:35.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.864+0000 7ff04a5ac700 1 -- start start 2023-10-16T10:00:35.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff044071be0 0x7ff04415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0440b2010 0x7ff0440b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0440ce680 0x7ff0441587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04415c9e0 con 0x7ff044071be0 2023-10-16T10:00:35.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04415ccc0 con 0x7ff0440b2010 2023-10-16T10:00:35.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04415cfa0 con 0x7ff0440ce680 2023-10-16T10:00:35.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff044071be0 0x7ff04415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff044071be0 0x7ff04415ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44474/0 (socket says 172.21.15.181:44474) 2023-10-16T10:00:35.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 -- 172.21.15.181:0/635771580 learned_addr learned my addr 172.21.15.181:0/635771580 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:35.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff0437fe700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0440b2010 0x7ff0440b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff048b49700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0440ce680 0x7ff0441587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 -- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0440ce680 msgr2=0x7ff0441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0440ce680 0x7ff0441587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 -- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0440b2010 msgr2=0x7ff0440b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:35.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0440b2010 0x7ff0440b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:35.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 -- 172.21.15.181:0/635771580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c0094a0 con 0x7ff044071be0 2023-10-16T10:00:35.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff043fff700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff044071be0 0x7ff04415ad00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff02c014040 tx=0x7ff02c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff040ff9700 1 -- 172.21.15.181:0/635771580 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0038a0 con 0x7ff044071be0 2023-10-16T10:00:35.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff040ff9700 1 -- 172.21.15.181:0/635771580 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c01e070 con 0x7ff044071be0 2023-10-16T10:00:35.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff040ff9700 1 -- 172.21.15.181:0/635771580 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c003c40 con 0x7ff044071be0 2023-10-16T10:00:35.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04415d280 con 0x7ff044071be0 2023-10-16T10:00:35.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.865+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04415da80 con 0x7ff044071be0 2023-10-16T10:00:35.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.866+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04415d7a0 con 0x7ff044071be0 2023-10-16T10:00:35.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.867+0000 7ff040ff9700 1 -- 172.21.15.181:0/635771580 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff02c02d450 con 0x7ff044071be0 2023-10-16T10:00:35.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.867+0000 7ff040ff9700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ff02406b020 0x7ff02406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:35.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.867+0000 7ff040ff9700 1 -- 172.21.15.181:0/635771580 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6133+0+0 (secure 0 0 0) 0x7ff02c08aa60 con 0x7ff044071be0 2023-10-16T10:00:35.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.867+0000 7ff0437fe700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ff02406b020 0x7ff02406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:35.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.868+0000 7ff0437fe700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ff02406b020 0x7ff02406d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0440c9ba0 tx=0x7ff034007c90 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:35.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:35.872+0000 7ff040ff9700 1 -- 172.21.15.181:0/635771580 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff04415d7a0 con 0x7ff044071be0 2023-10-16T10:00:36.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:36.114+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff04415d7a0 con 0x7ff044071be0 2023-10-16T10:00:36.999 INFO:tasks.ceph.mds.a.smithi181.stderr:2023-10-16T10:00:36.998+0000 7fa551b34700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:00:37.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.033+0000 7ff040ff9700 1 -- 172.21.15.181:0/635771580 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff04415d7a0 con 0x7ff044071be0 2023-10-16T10:00:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ff02406b020 msgr2=0x7ff02406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ff02406b020 0x7ff02406d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0440c9ba0 tx=0x7ff034007c90 comp rx=0 tx=0).stop 2023-10-16T10:00:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff044071be0 msgr2=0x7ff04415ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:37.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff044071be0 0x7ff04415ad00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff02c014040 tx=0x7ff02c004750 comp rx=0 tx=0).stop 2023-10-16T10:00:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 shutdown_connections 2023-10-16T10:00:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ff02406b020 0x7ff02406d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0440ce680 0x7ff0441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff044071be0 0x7ff04415ad00 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 --2- 172.21.15.181:0/635771580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0440b2010 0x7ff0440b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 >> 172.21.15.181:0/635771580 conn(0x7ff0440cd010 msgr2=0x7ff0440b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 shutdown_connections 2023-10-16T10:00:37.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.037+0000 7ff04a5ac700 1 -- 172.21.15.181:0/635771580 wait complete. 2023-10-16T10:00:37.053 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:00:37.054 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:00:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.181+0000 7ffa8bbf8700 1 Processor -- start 2023-10-16T10:00:37.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.181+0000 7ffa8bbf8700 1 -- start start 2023-10-16T10:00:37.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa8bbf8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 0x7ffa840665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa8bbf8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa8406f1e0 0x7ffa8406f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa8bbf8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8406e4b0 con 0x7ffa8415fee0 2023-10-16T10:00:37.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa8bbf8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8406ed50 con 0x7ffa8406f1e0 2023-10-16T10:00:37.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa8bbf8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa840cb0f0 con 0x7ffa8415b850 2023-10-16T10:00:37.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa89193700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ffa8415fee0 0x7ffa840c3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55560/0 (socket says 172.21.15.181:55560) 2023-10-16T10:00:37.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa89193700 1 -- 172.21.15.181:0/614318724 learned_addr learned my addr 172.21.15.181:0/614318724 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:37.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395354866 0 0) 0x7ffa8406e4b0 con 0x7ffa8415fee0 2023-10-16T10:00:37.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa89994700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 0x7ffa840665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa700035b0 con 0x7ffa8415fee0 2023-10-16T10:00:37.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa8a195700 1 --2- 172.21.15.181:0/614318724 >> v2:172.21.15.190:3300/0 conn(0x7ffa8406f1e0 0x7ffa8406f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1057028914 0 0) 0x7ffa700035b0 con 0x7ffa8415fee0 2023-10-16T10:00:37.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 msgr2=0x7ffa840665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:37.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 0x7ffa840665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 >> v2:172.21.15.190:3300/0 conn(0x7ffa8406f1e0 msgr2=0x7ffa8406f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:37.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 --2- 172.21.15.181:0/614318724 >> v2:172.21.15.190:3300/0 conn(0x7ffa8406f1e0 0x7ffa8406f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa84066b80 con 0x7ffa8415fee0 2023-10-16T10:00:37.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ffa80001670 con 0x7ffa8415fee0 2023-10-16T10:00:37.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 >> v1:172.21.15.181:6789/0 conn(0x7ffa8415fee0 legacy=0x7ffa840c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:37.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.182+0000 7ffa88992700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa70003a00 0x7ffa70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa70004850 0x7ffa70004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 0x7ffa70005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8406ed50 con 0x7ffa70003a00 2023-10-16T10:00:37.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa840cb0f0 con 0x7ffa70004850 2023-10-16T10:00:37.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7000ba10 con 0x7ffa8415b850 2023-10-16T10:00:37.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa8a195700 1 --2- 172.21.15.181:0/614318724 >> v2:172.21.15.190:3300/0 conn(0x7ffa8406f1e0 0x7ffa8406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:37.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89994700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa70003a00 0x7ffa70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89193700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa70004850 0x7ffa70004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa8a195700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 0x7ffa70005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89994700 1 -- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 msgr2=0x7ffa70005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:37.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89994700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa8415b850 0x7ffa70005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89994700 1 -- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa70004850 msgr2=0x7ffa70004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:37.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89994700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa70004850 0x7ffa70004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89994700 1 -- 172.21.15.181:0/614318724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700035b0 con 0x7ffa70003a00 2023-10-16T10:00:37.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa89994700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa70003a00 0x7ffa70003e20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009b60 tx=0x7ffa70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5b218f7486dff7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:37.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400e6c0 con 0x7ffa70003a00 2023-10-16T10:00:37.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7400e820 con 0x7ffa70003a00 2023-10-16T10:00:37.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.183+0000 7ffa88992700 1 -- 172.21.15.181:0/614318724 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400eb90 con 0x7ffa70003a00 2023-10-16T10:00:37.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.184+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/614318724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa70003a00 msgr2=0x7ffa70003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:37.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.184+0000 7ffa8bbf8700 1 --2- 172.21.15.181:0/614318724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa70003a00 0x7ffa70003e20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009b60 tx=0x7ffa70005880 comp rx=0 tx=0).stop 2023-10-16T10:00:37.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.184+0000 7ffa8a195700 1 -- 172.21.15.181:0/614318724 reap_dead start 2023-10-16T10:00:37.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.184+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/614318724 shutdown_connections 2023-10-16T10:00:37.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.184+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/614318724 >> 172.21.15.181:0/614318724 conn(0x7ffa840cc730 msgr2=0x7ffa840ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:37.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.184+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/614318724 shutdown_connections 2023-10-16T10:00:37.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.184+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/614318724 wait complete. 2023-10-16T10:00:37.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.185+0000 7ffa8bbf8700 1 Processor -- start 2023-10-16T10:00:37.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.185+0000 7ffa8bbf8700 1 -- start start 2023-10-16T10:00:37.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa70003a00 0x7ffa841f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa841f3100 0x7ffa8415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa840aee60 0x7ffa840ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa841f4020 con 0x7ffa841f3100 2023-10-16T10:00:37.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa840ae6a0 con 0x7ffa840aee60 2023-10-16T10:00:37.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa841f6bf0 con 0x7ffa70003a00 2023-10-16T10:00:37.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa841f3100 0x7ffa8415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa841f3100 0x7ffa8415dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44496/0 (socket says 172.21.15.181:44496) 2023-10-16T10:00:37.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 -- 172.21.15.181:0/1289424518 learned_addr learned my addr 172.21.15.181:0/1289424518 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:37.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89994700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa70003a00 0x7ffa841f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8a195700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa840aee60 0x7ffa840ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 -- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa70003a00 msgr2=0x7ffa841f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:37.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa70003a00 0x7ffa841f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 -- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa840aee60 msgr2=0x7ffa840ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:37.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa840aee60 0x7ffa840ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:37.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 -- 172.21.15.181:0/1289424518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa841f6e90 con 0x7ffa841f3100 2023-10-16T10:00:37.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89994700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa70003a00 0x7ffa841f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:37.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa89193700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa841f3100 0x7ffa8415dd90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000fb0 tx=0x7ffa8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:37.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa7a7fc700 1 -- 172.21.15.181:0/1289424518 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000c750 con 0x7ffa841f3100 2023-10-16T10:00:37.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa7a7fc700 1 -- 172.21.15.181:0/1289424518 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8000c8b0 con 0x7ffa841f3100 2023-10-16T10:00:37.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa841f7170 con 0x7ffa841f3100 2023-10-16T10:00:37.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa7a7fc700 1 -- 172.21.15.181:0/1289424518 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000cc60 con 0x7ffa841f3100 2023-10-16T10:00:37.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.186+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa840ae940 con 0x7ffa841f3100 2023-10-16T10:00:37.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.187+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8406b090 con 0x7ffa841f3100 2023-10-16T10:00:37.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.188+0000 7ffa7a7fc700 1 -- 172.21.15.181:0/1289424518 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffa8001d460 con 0x7ffa841f3100 2023-10-16T10:00:37.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.188+0000 7ffa7a7fc700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ffa6806b020 0x7ffa6806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:37.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.188+0000 7ffa89994700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ffa6806b020 0x7ffa6806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:37.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.188+0000 7ffa7a7fc700 1 -- 172.21.15.181:0/1289424518 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7ffa80089730 con 0x7ffa841f3100 2023-10-16T10:00:37.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.189+0000 7ffa89994700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ffa6806b020 0x7ffa6806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa74000c00 tx=0x7ffa74008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:37.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.193+0000 7ffa7a7fc700 1 -- 172.21.15.181:0/1289424518 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffa800545b0 con 0x7ffa841f3100 2023-10-16T10:00:37.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:37.429+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffa840cd5d0 con 0x7ffa841f3100 2023-10-16T10:00:38.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.037+0000 7ffa7a7fc700 1 -- 172.21.15.181:0/1289424518 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffa80057bd0 con 0x7ffa841f3100 2023-10-16T10:00:38.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ffa6806b020 msgr2=0x7ffa6806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ffa6806b020 0x7ffa6806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa74000c00 tx=0x7ffa74008030 comp rx=0 tx=0).stop 2023-10-16T10:00:38.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa841f3100 msgr2=0x7ffa8415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa841f3100 0x7ffa8415dd90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000fb0 tx=0x7ffa8000ee80 comp rx=0 tx=0).stop 2023-10-16T10:00:38.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 shutdown_connections 2023-10-16T10:00:38.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7ffa6806b020 0x7ffa6806d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa70003a00 0x7ffa841f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa840aee60 0x7ffa840ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 --2- 172.21.15.181:0/1289424518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa841f3100 0x7ffa8415dd90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.039+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 >> 172.21.15.181:0/1289424518 conn(0x7ffa840cc730 msgr2=0x7ffa840b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:38.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.040+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 shutdown_connections 2023-10-16T10:00:38.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.040+0000 7ffa8bbf8700 1 -- 172.21.15.181:0/1289424518 wait complete. 2023-10-16T10:00:38.055 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:00:38.055 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.177+0000 7f3e08e2a700 1 Processor -- start 2023-10-16T10:00:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.177+0000 7f3e08e2a700 1 -- start start 2023-10-16T10:00:38.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e08e2a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e0415b990 0x7f3e040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e08e2a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3e040d0370 0x7f3e040d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e08e2a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e040cc510 con 0x7f3e040cd2d0 2023-10-16T10:00:38.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e08e2a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e040d24f0 con 0x7f3e040d0370 2023-10-16T10:00:38.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e08e2a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040cc7f0 con 0x7f3e0415b990 2023-10-16T10:00:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e0415b990 0x7f3e040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e0415b990 0x7f3e040ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:41528/0 (socket says 172.21.15.181:41528) 2023-10-16T10:00:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e0259c700 1 -- 172.21.15.181:0/877012680 learned_addr learned my addr 172.21.15.181:0/877012680 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.178+0000 7f3e02d9d700 1 --2- 172.21.15.181:0/877012680 >> v2:172.21.15.190:3300/0 conn(0x7f3e040d0370 0x7f3e040d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0259c700 1 -- 172.21.15.181:0/877012680 >> v2:172.21.15.190:3300/0 conn(0x7f3e040d0370 msgr2=0x7f3e040d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0259c700 1 --2- 172.21.15.181:0/877012680 >> v2:172.21.15.190:3300/0 conn(0x7f3e040d0370 0x7f3e040d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0259c700 1 -- 172.21.15.181:0/877012680 >> v1:172.21.15.190:6789/0 conn(0x7f3e040cd2d0 legacy=0x7f3e040cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0259c700 1 -- 172.21.15.181:0/877012680 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040ccad0 con 0x7f3e0415b990 2023-10-16T10:00:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e02d9d700 1 --2- 172.21.15.181:0/877012680 >> v2:172.21.15.190:3300/0 conn(0x7f3e040d0370 0x7f3e040d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0259c700 1 --2- 172.21.15.181:0/877012680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e0415b990 0x7f3e040ce8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006fd0 tx=0x7f3df8009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae5091499dc985 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0159a700 1 -- 172.21.15.181:0/877012680 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8005e30 con 0x7f3e0415b990 2023-10-16T10:00:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0159a700 1 -- 172.21.15.181:0/877012680 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df8004030 con 0x7f3e0415b990 2023-10-16T10:00:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.179+0000 7f3e0159a700 1 -- 172.21.15.181:0/877012680 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df80043e0 con 0x7f3e0415b990 2023-10-16T10:00:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 -- 172.21.15.181:0/877012680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e0415b990 msgr2=0x7f3e040ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/877012680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e0415b990 0x7f3e040ce8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006fd0 tx=0x7f3df8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 -- 172.21.15.181:0/877012680 shutdown_connections 2023-10-16T10:00:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/877012680 >> v2:172.21.15.190:3300/0 conn(0x7f3e040d0370 0x7f3e040d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/877012680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e0415b990 0x7f3e040ce8f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 -- 172.21.15.181:0/877012680 >> 172.21.15.181:0/877012680 conn(0x7f3e0409a9a0 msgr2=0x7f3e040d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 -- 172.21.15.181:0/877012680 shutdown_connections 2023-10-16T10:00:38.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.180+0000 7f3e08e2a700 1 -- 172.21.15.181:0/877012680 wait complete. 2023-10-16T10:00:38.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.181+0000 7f3e08e2a700 1 Processor -- start 2023-10-16T10:00:38.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.181+0000 7f3e08e2a700 1 -- start start 2023-10-16T10:00:38.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e08e2a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e040cd2d0 0x7f3e0415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e08e2a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040d0370 0x7f3e041f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e08e2a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e0415b990 0x7f3e04155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e08e2a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e041f2390 con 0x7f3e0415b990 2023-10-16T10:00:38.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e08e2a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0415e3e0 con 0x7f3e040d0370 2023-10-16T10:00:38.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e08e2a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e041f4b80 con 0x7f3e040cd2d0 2023-10-16T10:00:38.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e0415b990 0x7f3e04155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e0415b990 0x7f3e04155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44512/0 (socket says 172.21.15.181:44512) 2023-10-16T10:00:38.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 -- 172.21.15.181:0/3948370269 learned_addr learned my addr 172.21.15.181:0/3948370269 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:38.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e0259c700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e040cd2d0 0x7f3e0415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e01d9b700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040d0370 0x7f3e041f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 -- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e040cd2d0 msgr2=0x7f3e0415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e040cd2d0 0x7f3e0415dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 -- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040d0370 msgr2=0x7f3e041f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040d0370 0x7f3e041f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.182+0000 7f3e02d9d700 1 -- 172.21.15.181:0/3948370269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc009a70 con 0x7f3e0415b990 2023-10-16T10:00:38.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.183+0000 7f3e02d9d700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e0415b990 0x7f3e04155bf0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00efc0 tx=0x7f3dfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.183+0000 7f3df2ffd700 1 -- 172.21.15.181:0/3948370269 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00ca30 con 0x7f3e0415b990 2023-10-16T10:00:38.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.183+0000 7f3df2ffd700 1 -- 172.21.15.181:0/3948370269 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc00f040 con 0x7f3e0415b990 2023-10-16T10:00:38.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.183+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df8009500 con 0x7f3e0415b990 2023-10-16T10:00:38.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.183+0000 7f3df2ffd700 1 -- 172.21.15.181:0/3948370269 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00cdf0 con 0x7f3e0415b990 2023-10-16T10:00:38.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.183+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e041f1af0 con 0x7f3e0415b990 2023-10-16T10:00:38.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.184+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04067d20 con 0x7f3e0415b990 2023-10-16T10:00:38.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.184+0000 7f3df2ffd700 1 -- 172.21.15.181:0/3948370269 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3dfc014480 con 0x7f3e0415b990 2023-10-16T10:00:38.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.185+0000 7f3df2ffd700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f3de406b020 0x7f3de406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.185+0000 7f3df2ffd700 1 -- 172.21.15.181:0/3948370269 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f3dfc089ec0 con 0x7f3e0415b990 2023-10-16T10:00:38.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.185+0000 7f3e0259c700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f3de406b020 0x7f3de406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.185+0000 7f3e0259c700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f3de406b020 0x7f3de406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006fd0 tx=0x7f3df8008070 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.189+0000 7f3df2ffd700 1 -- 172.21.15.181:0/3948370269 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3dfc054c30 con 0x7f3e0415b990 2023-10-16T10:00:38.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.427+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e041f4cc0 con 0x7f3e0415b990 2023-10-16T10:00:38.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.428+0000 7f3df2ffd700 1 -- 172.21.15.181:0/3948370269 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f3dfc058250 con 0x7f3e0415b990 2023-10-16T10:00:38.430 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:38.430 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1358211190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1358211190},{"type":"v1","addr":"172.21.15.190:6835","nonce":1358211190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6844,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2394829192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2394829192},{"type":"v1","addr":"172.21.15.190:6834","nonce":2394829192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2217231321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2217231321},{"type":"v1","addr":"172.21.15.181:6836","nonce":2217231321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:00:34.411162+0000","modified":"2023-10-16T10:00:37.431652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1158962128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1158962128},{"type":"v1","addr":"172.21.15.181:6837","nonce":1158962128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:00:38.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f3de406b020 msgr2=0x7f3de406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f3de406b020 0x7f3de406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006fd0 tx=0x7f3df8008070 comp rx=0 tx=0).stop 2023-10-16T10:00:38.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e0415b990 msgr2=0x7f3e04155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e0415b990 0x7f3e04155bf0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00efc0 tx=0x7f3dfc00c3a0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 shutdown_connections 2023-10-16T10:00:38.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f3de406b020 0x7f3de406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e040cd2d0 0x7f3e0415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e0415b990 0x7f3e04155bf0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 --2- 172.21.15.181:0/3948370269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e040d0370 0x7f3e041f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 >> 172.21.15.181:0/3948370269 conn(0x7f3e0409a9a0 msgr2=0x7f3e040cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:38.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 shutdown_connections 2023-10-16T10:00:38.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.430+0000 7f3e08e2a700 1 -- 172.21.15.181:0/3948370269 wait complete. 2023-10-16T10:00:38.434 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 49 2023-10-16T10:00:38.444 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:00:38.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.563+0000 7f49fa022700 1 Processor -- start 2023-10-16T10:00:38.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.563+0000 7f49fa022700 1 -- start start 2023-10-16T10:00:38.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.563+0000 7f49fa022700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f49f40cdb80 0x7f49f40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.563+0000 7f49fa022700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f406f870 con 0x7f49f415dfd0 2023-10-16T10:00:38.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.563+0000 7f49fa022700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f40d1160 con 0x7f49f40af450 2023-10-16T10:00:38.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.563+0000 7f49fa022700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f40cc8e0 con 0x7f49f40cdb80 2023-10-16T10:00:38.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f49f415dfd0 0x7f49f406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55594/0 (socket says 172.21.15.181:55594) 2023-10-16T10:00:38.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f37fe700 1 -- 172.21.15.181:0/2636508198 learned_addr learned my addr 172.21.15.181:0/2636508198 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:38.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f3fff700 1 --2- 172.21.15.181:0/2636508198 >> v2:172.21.15.190:3300/0 conn(0x7f49f40cdb80 0x7f49f40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f27fc700 1 -- 172.21.15.181:0/2636508198 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020013286 0 0) 0x7f49f406f870 con 0x7f49f415dfd0 2023-10-16T10:00:38.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f27fc700 1 -- 172.21.15.181:0/2636508198 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e00035b0 con 0x7f49f415dfd0 2023-10-16T10:00:38.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f3fff700 1 -- 172.21.15.181:0/2636508198 >> v1:172.21.15.190:6789/0 conn(0x7f49f40af450 legacy=0x7f49f40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f3fff700 1 -- 172.21.15.181:0/2636508198 >> v1:172.21.15.181:6789/0 conn(0x7f49f415dfd0 legacy=0x7f49f406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f27fc700 1 -- 172.21.15.181:0/2636508198 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3469825755 0 0) 0x7f49e00035b0 con 0x7f49f415dfd0 2023-10-16T10:00:38.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f3fff700 1 -- 172.21.15.181:0/2636508198 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40ccbc0 con 0x7f49f40cdb80 2023-10-16T10:00:38.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f3fff700 1 --2- 172.21.15.181:0/2636508198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 0x7f49f40af9a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009840 tx=0x7f49ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5494b7240cb5b9c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f27fc700 1 -- 172.21.15.181:0/2636508198 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4001b70 con 0x7f49f40cdb80 2023-10-16T10:00:38.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f27fc700 1 -- 172.21.15.181:0/2636508198 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec004030 con 0x7f49f40cdb80 2023-10-16T10:00:38.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.564+0000 7f49f27fc700 1 -- 172.21.15.181:0/2636508198 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0043a0 con 0x7f49f40cdb80 2023-10-16T10:00:38.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.565+0000 7f49fa022700 1 -- 172.21.15.181:0/2636508198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 msgr2=0x7f49f40af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.565+0000 7f49fa022700 1 --2- 172.21.15.181:0/2636508198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 0x7f49f40af9a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009840 tx=0x7f49ec009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:38.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.565+0000 7f49fa022700 1 -- 172.21.15.181:0/2636508198 shutdown_connections 2023-10-16T10:00:38.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.565+0000 7f49fa022700 1 --2- 172.21.15.181:0/2636508198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 0x7f49f40af9a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.565+0000 7f49fa022700 1 -- 172.21.15.181:0/2636508198 >> 172.21.15.181:0/2636508198 conn(0x7f49f40d04a0 msgr2=0x7f49f40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:38.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.565+0000 7f49fa022700 1 -- 172.21.15.181:0/2636508198 shutdown_connections 2023-10-16T10:00:38.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.565+0000 7f49fa022700 1 -- 172.21.15.181:0/2636508198 wait complete. 2023-10-16T10:00:38.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.566+0000 7f49fa022700 1 Processor -- start 2023-10-16T10:00:38.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.566+0000 7f49fa022700 1 -- start start 2023-10-16T10:00:38.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49fa022700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49f40af450 0x7f49f40af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49fa022700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 0x7f49f41f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49fa022700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f415dfd0 0x7f49f41e7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49fa022700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40afea0 con 0x7f49f415dfd0 2023-10-16T10:00:38.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49fa022700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41e82a0 con 0x7f49f40cdb80 2023-10-16T10:00:38.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49fa022700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f4158bf0 con 0x7f49f40af450 2023-10-16T10:00:38.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49f3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f415dfd0 0x7f49f41e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49f3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f415dfd0 0x7f49f41e7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44526/0 (socket says 172.21.15.181:44526) 2023-10-16T10:00:38.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49f3fff700 1 -- 172.21.15.181:0/3726580094 learned_addr learned my addr 172.21.15.181:0/3726580094 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:38.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49f37fe700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49f40af450 0x7f49f40af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49f2ffd700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 0x7f49f41f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.567+0000 7f49f3fff700 1 -- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49f40af450 msgr2=0x7f49f40af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49f3fff700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49f40af450 0x7f49f40af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49f3fff700 1 -- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 msgr2=0x7f49f41f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49f3fff700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 0x7f49f41f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49f3fff700 1 -- 172.21.15.181:0/3726580094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec0094a0 con 0x7f49f415dfd0 2023-10-16T10:00:38.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49f3fff700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f415dfd0 0x7f49f41e7890 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0090b0 tx=0x7f49ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49ebfff700 1 -- 172.21.15.181:0/3726580094 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec011630 con 0x7f49f415dfd0 2023-10-16T10:00:38.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49ebfff700 1 -- 172.21.15.181:0/3726580094 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec01f070 con 0x7f49f415dfd0 2023-10-16T10:00:38.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49ebfff700 1 -- 172.21.15.181:0/3726580094 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0119f0 con 0x7f49f415dfd0 2023-10-16T10:00:38.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f4158ed0 con 0x7f49f415dfd0 2023-10-16T10:00:38.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.568+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f4159490 con 0x7f49f415dfd0 2023-10-16T10:00:38.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.569+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f40cb050 con 0x7f49f415dfd0 2023-10-16T10:00:38.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.569+0000 7f49ebfff700 1 -- 172.21.15.181:0/3726580094 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f49ec011b50 con 0x7f49f415dfd0 2023-10-16T10:00:38.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.570+0000 7f49ebfff700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f49d406b020 0x7f49d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.570+0000 7f49f37fe700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f49d406b020 0x7f49d406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.570+0000 7f49ebfff700 1 -- 172.21.15.181:0/3726580094 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f49ec021070 con 0x7f49f415dfd0 2023-10-16T10:00:38.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.570+0000 7f49f37fe700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f49d406b020 0x7f49d406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49e4002230 tx=0x7f49e4009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.574+0000 7f49ebfff700 1 -- 172.21.15.181:0/3726580094 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f49ec0bdfe0 con 0x7f49f415dfd0 2023-10-16T10:00:38.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.767+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49f4067d20 con 0x7f49f415dfd0 2023-10-16T10:00:38.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.769+0000 7f49ebfff700 1 -- 172.21.15.181:0/3726580094 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+17251 (secure 0 0 0) 0x7f49ec055ff0 con 0x7f49f415dfd0 2023-10-16T10:00:38.771 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:38.772 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":103,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:00:36.019746+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:00:32.847964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:00:33.365892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:00:38.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f49d406b020 msgr2=0x7f49d406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f49d406b020 0x7f49d406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49e4002230 tx=0x7f49e4009040 comp rx=0 tx=0).stop 2023-10-16T10:00:38.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f415dfd0 msgr2=0x7f49f41e7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f415dfd0 0x7f49f41e7890 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0090b0 tx=0x7f49ec004750 comp rx=0 tx=0).stop 2023-10-16T10:00:38.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 shutdown_connections 2023-10-16T10:00:38.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f49d406b020 0x7f49d406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49f40af450 0x7f49f40af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f415dfd0 0x7f49f41e7890 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 --2- 172.21.15.181:0/3726580094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49f40cdb80 0x7f49f41f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 >> 172.21.15.181:0/3726580094 conn(0x7f49f40d04a0 msgr2=0x7f49f415be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:38.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 shutdown_connections 2023-10-16T10:00:38.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.771+0000 7f49fa022700 1 -- 172.21.15.181:0/3726580094 wait complete. 2023-10-16T10:00:38.785 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:00:38.785 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:00:38.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.904+0000 7f0a8288c700 1 Processor -- start 2023-10-16T10:00:38.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.904+0000 7f0a8288c700 1 -- start start 2023-10-16T10:00:38.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.904+0000 7f0a8288c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 0x7f0a7c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.904+0000 7f0a8288c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c0cee70 con 0x7f0a7c0cd3f0 2023-10-16T10:00:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.904+0000 7f0a8288c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c159b10 con 0x7f0a7c159730 2023-10-16T10:00:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.904+0000 7f0a8288c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0cd880 con 0x7f0a7c15cd20 2023-10-16T10:00:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a80e29700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0a7c0cd3f0 0x7f0a7c0cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55612/0 (socket says 172.21.15.181:55612) 2023-10-16T10:00:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a80e29700 1 -- 172.21.15.181:0/40536306 learned_addr learned my addr 172.21.15.181:0/40536306 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 0x7f0a7c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195813856 0 0) 0x7f0a7c0cee70 con 0x7f0a7c0cd3f0 2023-10-16T10:00:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a640035b0 con 0x7f0a7c0cd3f0 2023-10-16T10:00:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 307979518 0 0) 0x7f0a640035b0 con 0x7f0a7c0cd3f0 2023-10-16T10:00:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 msgr2=0x7f0a7c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 0x7f0a7c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 >> v1:172.21.15.190:6789/0 conn(0x7f0a7c159730 legacy=0x7f0a7c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c0ce540 con 0x7f0a7c0cd3f0 2023-10-16T10:00:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f0a740019f0 con 0x7f0a7c0cd3f0 2023-10-16T10:00:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 >> v1:172.21.15.181:6789/0 conn(0x7f0a7c0cd3f0 legacy=0x7f0a7c0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a64003a30 0x7f0a64003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a64004850 0x7f0a64004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 0x7f0a64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0cd880 con 0x7f0a64004850 2023-10-16T10:00:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6400bac0 con 0x7f0a64003a30 2023-10-16T10:00:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6400bc80 con 0x7f0a7c15cd20 2023-10-16T10:00:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a64004850 0x7f0a64004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7bfff700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a64003a30 0x7f0a64003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a80e29700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 0x7f0a64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7b7fe700 1 -- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 msgr2=0x7f0a64005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.905+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 0x7f0a64005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a7b7fe700 1 -- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a64003a30 msgr2=0x7f0a64003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a64003a30 0x7f0a64003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a7b7fe700 1 -- 172.21.15.181:0/40536306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a640035b0 con 0x7f0a64004850 2023-10-16T10:00:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a80e29700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a7c15cd20 0x7f0a64005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a64004850 0x7f0a64004cd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009b30 tx=0x7f0a64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2549f8d321b68f8a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000e6c0 con 0x7f0a64004850 2023-10-16T10:00:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7000e820 con 0x7f0a64004850 2023-10-16T10:00:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a7affd700 1 -- 172.21.15.181:0/40536306 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000eb90 con 0x7f0a64004850 2023-10-16T10:00:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a8288c700 1 -- 172.21.15.181:0/40536306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a64004850 msgr2=0x7f0a64004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a8288c700 1 --2- 172.21.15.181:0/40536306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a64004850 0x7f0a64004cd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009b30 tx=0x7f0a64005880 comp rx=0 tx=0).stop 2023-10-16T10:00:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a80e29700 1 -- 172.21.15.181:0/40536306 reap_dead start 2023-10-16T10:00:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a8288c700 1 -- 172.21.15.181:0/40536306 shutdown_connections 2023-10-16T10:00:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a8288c700 1 -- 172.21.15.181:0/40536306 >> 172.21.15.181:0/40536306 conn(0x7f0a7c0d04a0 msgr2=0x7f0a7c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a8288c700 1 -- 172.21.15.181:0/40536306 shutdown_connections 2023-10-16T10:00:38.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.906+0000 7f0a8288c700 1 -- 172.21.15.181:0/40536306 wait complete. 2023-10-16T10:00:38.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 Processor -- start 2023-10-16T10:00:38.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 -- start start 2023-10-16T10:00:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a64004850 0x7f0a7c1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a7c0b52b0 0x7f0a7c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a7c15bf80 0x7f0a7c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c15c940 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c0b4e20 con 0x7f0a7c15bf80 2023-10-16T10:00:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.908+0000 7f0a8288c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c06e890 con 0x7f0a64004850 2023-10-16T10:00:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a7c0b52b0 0x7f0a7c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a7c0b52b0 0x7f0a7c0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44542/0 (socket says 172.21.15.181:44542) 2023-10-16T10:00:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 -- 172.21.15.181:0/3721282656 learned_addr learned my addr 172.21.15.181:0/3721282656 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:38.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a80e29700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a7c15bf80 0x7f0a7c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7bfff700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a64004850 0x7f0a7c1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 -- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a64004850 msgr2=0x7f0a7c1f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a64004850 0x7f0a7c1f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 -- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a7c15bf80 msgr2=0x7f0a7c15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:38.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a7c15bf80 0x7f0a7c15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:38.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 -- 172.21.15.181:0/3721282656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c06eb70 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7bfff700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a64004850 0x7f0a7c1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:38.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a80e29700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a7c15bf80 0x7f0a7c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:38.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a7b7fe700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a7c0b52b0 0x7f0a7c0b3d00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000e4b0 tx=0x7f0a700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a78ff9700 1 -- 172.21.15.181:0/3721282656 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000e6c0 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a78ff9700 1 -- 172.21.15.181:0/3721282656 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a70003ad0 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c0b47d0 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a78ff9700 1 -- 172.21.15.181:0/3721282656 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70003e80 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.909+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c06ee50 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.910+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c06f010 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.911+0000 7f0a78ff9700 1 -- 172.21.15.181:0/3721282656 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0a70013470 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.911+0000 7f0a78ff9700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f0a6806b020 0x7f0a6806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:38.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.911+0000 7f0a78ff9700 1 -- 172.21.15.181:0/3721282656 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f0a7008b260 con 0x7f0a7c0b52b0 2023-10-16T10:00:38.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.911+0000 7f0a7bfff700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f0a6806b020 0x7f0a6806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:38.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.911+0000 7f0a7bfff700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f0a6806b020 0x7f0a6806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0056a0 tx=0x7f0a6c009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:38.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:38.915+0000 7f0a78ff9700 1 -- 172.21.15.181:0/3721282656 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0a700560e0 con 0x7f0a7c0b52b0 2023-10-16T10:00:39.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.105+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 --> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a7c15c940 con 0x7f0a6806b020 2023-10-16T10:00:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.114+0000 7f0a78ff9700 1 -- 172.21.15.181:0/3721282656 <== mgr.7254 v2:172.21.15.181:6832/3229300335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289910 (secure 0 0 0) 0x7f0a7c15c940 con 0x7f0a6806b020 2023-10-16T10:00:39.119 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:39.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f0a6806b020 msgr2=0x7f0a6806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f0a6806b020 0x7f0a6806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0056a0 tx=0x7f0a6c009040 comp rx=0 tx=0).stop 2023-10-16T10:00:39.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a7c0b52b0 msgr2=0x7f0a7c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a7c0b52b0 0x7f0a7c0b3d00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000e4b0 tx=0x7f0a700047c0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 shutdown_connections 2023-10-16T10:00:39.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f0a6806b020 0x7f0a6806d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a64004850 0x7f0a7c1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a7c15bf80 0x7f0a7c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.117+0000 7f0a8288c700 1 --2- 172.21.15.181:0/3721282656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a7c0b52b0 0x7f0a7c0b3d00 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.118+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 >> 172.21.15.181:0/3721282656 conn(0x7f0a7c0d04a0 msgr2=0x7f0a7c0ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:39.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.118+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 shutdown_connections 2023-10-16T10:00:39.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.118+0000 7f0a8288c700 1 -- 172.21.15.181:0/3721282656 wait complete. 2023-10-16T10:00:39.124 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:00:39.131 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:00:38.673478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":309100,"kb_used_data":11044,"kb_used_omap":12,"kb_used_meta":297907,"kb_avail":754665620,"statfs":{"total":773094113280,"available":772777594880,"internally_reserved":0,"allocated":11309056,"data_stored":6101856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":305057336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.671764"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195682+0000","last_change":"2023-10-16T10:00:35.028044+0000","last_active":"2023-10-16T10:00:36.195682+0000","last_peered":"2023-10-16T10:00:36.195682+0000","last_clean":"2023-10-16T10:00:36.195682+0000","last_became_active":"2023-10-16T10:00:35.027614+0000","last_became_peered":"2023-10-16T10:00:35.027614+0000","last_unstale":"2023-10-16T10:00:36.195682+0000","last_undegraded":"2023-10-16T10:00:36.195682+0000","last_fullsized":"2023-10-16T10:00:36.195682+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:17.207068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195061+0000","last_change":"2023-10-16T10:00:35.028916+0000","last_active":"2023-10-16T10:00:36.195061+0000","last_peered":"2023-10-16T10:00:36.195061+0000","last_clean":"2023-10-16T10:00:36.195061+0000","last_became_active":"2023-10-16T10:00:35.028609+0000","last_became_peered":"2023-10-16T10:00:35.028609+0000","last_unstale":"2023-10-16T10:00:36.195061+0000","last_undegraded":"2023-10-16T10:00:36.195061+0000","last_fullsized":"2023-10-16T10:00:36.195061+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:14.619379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028875+0000","last_change":"2023-10-16T10:00:35.032100+0000","last_active":"2023-10-16T10:00:36.028875+0000","last_peered":"2023-10-16T10:00:36.028875+0000","last_clean":"2023-10-16T10:00:36.028875+0000","last_became_active":"2023-10-16T10:00:35.031809+0000","last_became_peered":"2023-10-16T10:00:35.031809+0000","last_unstale":"2023-10-16T10:00:36.028875+0000","last_undegraded":"2023-10-16T10:00:36.028875+0000","last_fullsized":"2023-10-16T10:00:36.028875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:20.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195145+0000","last_change":"2023-10-16T10:00:35.035186+0000","last_active":"2023-10-16T10:00:36.195145+0000","last_peered":"2023-10-16T10:00:36.195145+0000","last_clean":"2023-10-16T10:00:36.195145+0000","last_became_active":"2023-10-16T10:00:35.034934+0000","last_became_peered":"2023-10-16T10:00:35.034934+0000","last_unstale":"2023-10-16T10:00:36.195145+0000","last_undegraded":"2023-10-16T10:00:36.195145+0000","last_fullsized":"2023-10-16T10:00:36.195145+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:15.018930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194721+0000","last_change":"2023-10-16T10:00:35.034491+0000","last_active":"2023-10-16T10:00:36.194721+0000","last_peered":"2023-10-16T10:00:36.194721+0000","last_clean":"2023-10-16T10:00:36.194721+0000","last_became_active":"2023-10-16T10:00:35.034223+0000","last_became_peered":"2023-10-16T10:00:35.034223+0000","last_unstale":"2023-10-16T10:00:36.194721+0000","last_undegraded":"2023-10-16T10:00:36.194721+0000","last_fullsized":"2023-10-16T10:00:36.194721+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:42.164790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029568+0000","last_change":"2023-10-16T10:00:35.026863+0000","last_active":"2023-10-16T10:00:36.029568+0000","last_peered":"2023-10-16T10:00:36.029568+0000","last_clean":"2023-10-16T10:00:36.029568+0000","last_became_active":"2023-10-16T10:00:35.026431+0000","last_became_peered":"2023-10-16T10:00:35.026431+0000","last_unstale":"2023-10-16T10:00:36.029568+0000","last_undegraded":"2023-10-16T10:00:36.029568+0000","last_fullsized":"2023-10-16T10:00:36.029568+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:59.182413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194363+0000","last_change":"2023-10-16T10:00:35.025625+0000","last_active":"2023-10-16T10:00:36.194363+0000","last_peered":"2023-10-16T10:00:36.194363+0000","last_clean":"2023-10-16T10:00:36.194363+0000","last_became_active":"2023-10-16T10:00:35.024299+0000","last_became_peered":"2023-10-16T10:00:35.024299+0000","last_unstale":"2023-10-16T10:00:36.194363+0000","last_undegraded":"2023-10-16T10:00:36.194363+0000","last_fullsized":"2023-10-16T10:00:36.194363+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:24.601041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029425+0000","last_change":"2023-10-16T10:00:35.040178+0000","last_active":"2023-10-16T10:00:36.029425+0000","last_peered":"2023-10-16T10:00:36.029425+0000","last_clean":"2023-10-16T10:00:36.029425+0000","last_became_active":"2023-10-16T10:00:35.039829+0000","last_became_peered":"2023-10-16T10:00:35.039829+0000","last_unstale":"2023-10-16T10:00:36.029425+0000","last_undegraded":"2023-10-16T10:00:36.029425+0000","last_fullsized":"2023-10-16T10:00:36.029425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:45.878055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029631+0000","last_change":"2023-10-16T10:00:35.031314+0000","last_active":"2023-10-16T10:00:36.029631+0000","last_peered":"2023-10-16T10:00:36.029631+0000","last_clean":"2023-10-16T10:00:36.029631+0000","last_became_active":"2023-10-16T10:00:35.031041+0000","last_became_peered":"2023-10-16T10:00:35.031041+0000","last_unstale":"2023-10-16T10:00:36.029631+0000","last_undegraded":"2023-10-16T10:00:36.029631+0000","last_fullsized":"2023-10-16T10:00:36.029631+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:07:47.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195415+0000","last_change":"2023-10-16T10:00:35.032610+0000","last_active":"2023-10-16T10:00:36.195415+0000","last_peered":"2023-10-16T10:00:36.195415+0000","last_clean":"2023-10-16T10:00:36.195415+0000","last_became_active":"2023-10-16T10:00:35.032175+0000","last_became_peered":"2023-10-16T10:00:35.032175+0000","last_unstale":"2023-10-16T10:00:36.195415+0000","last_undegraded":"2023-10-16T10:00:36.195415+0000","last_fullsized":"2023-10-16T10:00:36.195415+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:09:49.796392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195299+0000","last_change":"2023-10-16T10:00:35.032540+0000","last_active":"2023-10-16T10:00:36.195299+0000","last_peered":"2023-10-16T10:00:36.195299+0000","last_clean":"2023-10-16T10:00:36.195299+0000","last_became_active":"2023-10-16T10:00:35.032283+0000","last_became_peered":"2023-10-16T10:00:35.032283+0000","last_unstale":"2023-10-16T10:00:36.195299+0000","last_undegraded":"2023-10-16T10:00:36.195299+0000","last_fullsized":"2023-10-16T10:00:36.195299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:55.118437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195595+0000","last_change":"2023-10-16T10:00:35.034687+0000","last_active":"2023-10-16T10:00:36.195595+0000","last_peered":"2023-10-16T10:00:36.195595+0000","last_clean":"2023-10-16T10:00:36.195595+0000","last_became_active":"2023-10-16T10:00:35.033313+0000","last_became_peered":"2023-10-16T10:00:35.033313+0000","last_unstale":"2023-10-16T10:00:36.195595+0000","last_undegraded":"2023-10-16T10:00:36.195595+0000","last_fullsized":"2023-10-16T10:00:36.195595+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:29.215922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028552+0000","last_change":"2023-10-16T10:00:35.022029+0000","last_active":"2023-10-16T10:00:36.028552+0000","last_peered":"2023-10-16T10:00:36.028552+0000","last_clean":"2023-10-16T10:00:36.028552+0000","last_became_active":"2023-10-16T10:00:35.021727+0000","last_became_peered":"2023-10-16T10:00:35.021727+0000","last_unstale":"2023-10-16T10:00:36.028552+0000","last_undegraded":"2023-10-16T10:00:36.028552+0000","last_fullsized":"2023-10-16T10:00:36.028552+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:41.783948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194701+0000","last_change":"2023-10-16T10:00:35.037630+0000","last_active":"2023-10-16T10:00:36.194701+0000","last_peered":"2023-10-16T10:00:36.194701+0000","last_clean":"2023-10-16T10:00:36.194701+0000","last_became_active":"2023-10-16T10:00:35.036988+0000","last_became_peered":"2023-10-16T10:00:35.036988+0000","last_unstale":"2023-10-16T10:00:36.194701+0000","last_undegraded":"2023-10-16T10:00:36.194701+0000","last_fullsized":"2023-10-16T10:00:36.194701+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:46.263256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027246+0000","last_change":"2023-10-16T10:00:35.024763+0000","last_active":"2023-10-16T10:00:36.027246+0000","last_peered":"2023-10-16T10:00:36.027246+0000","last_clean":"2023-10-16T10:00:36.027246+0000","last_became_active":"2023-10-16T10:00:35.024465+0000","last_became_peered":"2023-10-16T10:00:35.024465+0000","last_unstale":"2023-10-16T10:00:36.027246+0000","last_undegraded":"2023-10-16T10:00:36.027246+0000","last_fullsized":"2023-10-16T10:00:36.027246+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.213292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194383+0000","last_change":"2023-10-16T10:00:35.030249+0000","last_active":"2023-10-16T10:00:36.194383+0000","last_peered":"2023-10-16T10:00:36.194383+0000","last_clean":"2023-10-16T10:00:36.194383+0000","last_became_active":"2023-10-16T10:00:35.029727+0000","last_became_peered":"2023-10-16T10:00:35.029727+0000","last_unstale":"2023-10-16T10:00:36.194383+0000","last_undegraded":"2023-10-16T10:00:36.194383+0000","last_fullsized":"2023-10-16T10:00:36.194383+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:42.756758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194911+0000","last_change":"2023-10-16T10:00:35.030845+0000","last_active":"2023-10-16T10:00:36.194911+0000","last_peered":"2023-10-16T10:00:36.194911+0000","last_clean":"2023-10-16T10:00:36.194911+0000","last_became_active":"2023-10-16T10:00:35.030555+0000","last_became_peered":"2023-10-16T10:00:35.030555+0000","last_unstale":"2023-10-16T10:00:36.194911+0000","last_undegraded":"2023-10-16T10:00:36.194911+0000","last_fullsized":"2023-10-16T10:00:36.194911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:53.268872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194784+0000","last_change":"2023-10-16T10:00:35.027864+0000","last_active":"2023-10-16T10:00:36.194784+0000","last_peered":"2023-10-16T10:00:36.194784+0000","last_clean":"2023-10-16T10:00:36.194784+0000","last_became_active":"2023-10-16T10:00:35.027602+0000","last_became_peered":"2023-10-16T10:00:35.027602+0000","last_unstale":"2023-10-16T10:00:36.194784+0000","last_undegraded":"2023-10-16T10:00:36.194784+0000","last_fullsized":"2023-10-16T10:00:36.194784+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:59.230033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195449+0000","last_change":"2023-10-16T10:00:35.031077+0000","last_active":"2023-10-16T10:00:36.195449+0000","last_peered":"2023-10-16T10:00:36.195449+0000","last_clean":"2023-10-16T10:00:36.195449+0000","last_became_active":"2023-10-16T10:00:35.030780+0000","last_became_peered":"2023-10-16T10:00:35.030780+0000","last_unstale":"2023-10-16T10:00:36.195449+0000","last_undegraded":"2023-10-16T10:00:36.195449+0000","last_fullsized":"2023-10-16T10:00:36.195449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:17.795751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195272+0000","last_change":"2023-10-16T10:00:35.031270+0000","last_active":"2023-10-16T10:00:36.195272+0000","last_peered":"2023-10-16T10:00:36.195272+0000","last_clean":"2023-10-16T10:00:36.195272+0000","last_became_active":"2023-10-16T10:00:35.030975+0000","last_became_peered":"2023-10-16T10:00:35.030975+0000","last_unstale":"2023-10-16T10:00:36.195272+0000","last_undegraded":"2023-10-16T10:00:36.195272+0000","last_fullsized":"2023-10-16T10:00:36.195272+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.898298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194650+0000","last_change":"2023-10-16T10:00:35.034123+0000","last_active":"2023-10-16T10:00:36.194650+0000","last_peered":"2023-10-16T10:00:36.194650+0000","last_clean":"2023-10-16T10:00:36.194650+0000","last_became_active":"2023-10-16T10:00:35.033850+0000","last_became_peered":"2023-10-16T10:00:35.033850+0000","last_unstale":"2023-10-16T10:00:36.194650+0000","last_undegraded":"2023-10-16T10:00:36.194650+0000","last_fullsized":"2023-10-16T10:00:36.194650+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:34.255691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194664+0000","last_change":"2023-10-16T10:00:35.028948+0000","last_active":"2023-10-16T10:00:36.194664+0000","last_peered":"2023-10-16T10:00:36.194664+0000","last_clean":"2023-10-16T10:00:36.194664+0000","last_became_active":"2023-10-16T10:00:35.028705+0000","last_became_peered":"2023-10-16T10:00:35.028705+0000","last_unstale":"2023-10-16T10:00:36.194664+0000","last_undegraded":"2023-10-16T10:00:36.194664+0000","last_fullsized":"2023-10-16T10:00:36.194664+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:23.077701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195032+0000","last_change":"2023-10-16T10:00:35.033347+0000","last_active":"2023-10-16T10:00:36.195032+0000","last_peered":"2023-10-16T10:00:36.195032+0000","last_clean":"2023-10-16T10:00:36.195032+0000","last_became_active":"2023-10-16T10:00:35.032806+0000","last_became_peered":"2023-10-16T10:00:35.032806+0000","last_unstale":"2023-10-16T10:00:36.195032+0000","last_undegraded":"2023-10-16T10:00:36.195032+0000","last_fullsized":"2023-10-16T10:00:36.195032+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:15.018930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194656+0000","last_change":"2023-10-16T10:00:35.031979+0000","last_active":"2023-10-16T10:00:36.194656+0000","last_peered":"2023-10-16T10:00:36.194656+0000","last_clean":"2023-10-16T10:00:36.194656+0000","last_became_active":"2023-10-16T10:00:35.031275+0000","last_became_peered":"2023-10-16T10:00:35.031275+0000","last_unstale":"2023-10-16T10:00:36.194656+0000","last_undegraded":"2023-10-16T10:00:36.194656+0000","last_fullsized":"2023-10-16T10:00:36.194656+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:12.030952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194383+0000","last_change":"2023-10-16T10:00:35.029582+0000","last_active":"2023-10-16T10:00:36.194383+0000","last_peered":"2023-10-16T10:00:36.194383+0000","last_clean":"2023-10-16T10:00:36.194383+0000","last_became_active":"2023-10-16T10:00:35.029249+0000","last_became_peered":"2023-10-16T10:00:35.029249+0000","last_unstale":"2023-10-16T10:00:36.194383+0000","last_undegraded":"2023-10-16T10:00:36.194383+0000","last_fullsized":"2023-10-16T10:00:36.194383+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:17.207068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028586+0000","last_change":"2023-10-16T10:00:35.034767+0000","last_active":"2023-10-16T10:00:36.028586+0000","last_peered":"2023-10-16T10:00:36.028586+0000","last_clean":"2023-10-16T10:00:36.028586+0000","last_became_active":"2023-10-16T10:00:35.034459+0000","last_became_peered":"2023-10-16T10:00:35.034459+0000","last_unstale":"2023-10-16T10:00:36.028586+0000","last_undegraded":"2023-10-16T10:00:36.028586+0000","last_fullsized":"2023-10-16T10:00:36.028586+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:51.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195142+0000","last_change":"2023-10-16T10:00:35.032500+0000","last_active":"2023-10-16T10:00:36.195142+0000","last_peered":"2023-10-16T10:00:36.195142+0000","last_clean":"2023-10-16T10:00:36.195142+0000","last_became_active":"2023-10-16T10:00:35.032040+0000","last_became_peered":"2023-10-16T10:00:35.032040+0000","last_unstale":"2023-10-16T10:00:36.195142+0000","last_undegraded":"2023-10-16T10:00:36.195142+0000","last_fullsized":"2023-10-16T10:00:36.195142+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:21:26.664514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195374+0000","last_change":"2023-10-16T10:00:35.029640+0000","last_active":"2023-10-16T10:00:36.195374+0000","last_peered":"2023-10-16T10:00:36.195374+0000","last_clean":"2023-10-16T10:00:36.195374+0000","last_became_active":"2023-10-16T10:00:35.029136+0000","last_became_peered":"2023-10-16T10:00:35.029136+0000","last_unstale":"2023-10-16T10:00:36.195374+0000","last_undegraded":"2023-10-16T10:00:36.195374+0000","last_fullsized":"2023-10-16T10:00:36.195374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:07.768526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194538+0000","last_change":"2023-10-16T10:00:35.033117+0000","last_active":"2023-10-16T10:00:36.194538+0000","last_peered":"2023-10-16T10:00:36.194538+0000","last_clean":"2023-10-16T10:00:36.194538+0000","last_became_active":"2023-10-16T10:00:35.032063+0000","last_became_peered":"2023-10-16T10:00:35.032063+0000","last_unstale":"2023-10-16T10:00:36.194538+0000","last_undegraded":"2023-10-16T10:00:36.194538+0000","last_fullsized":"2023-10-16T10:00:36.194538+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:49.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194794+0000","last_change":"2023-10-16T10:00:35.035940+0000","last_active":"2023-10-16T10:00:36.194794+0000","last_peered":"2023-10-16T10:00:36.194794+0000","last_clean":"2023-10-16T10:00:36.194794+0000","last_became_active":"2023-10-16T10:00:35.031452+0000","last_became_peered":"2023-10-16T10:00:35.031452+0000","last_unstale":"2023-10-16T10:00:36.194794+0000","last_undegraded":"2023-10-16T10:00:36.194794+0000","last_fullsized":"2023-10-16T10:00:36.194794+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:14.079709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194453+0000","last_change":"2023-10-16T10:00:35.032466+0000","last_active":"2023-10-16T10:00:36.194453+0000","last_peered":"2023-10-16T10:00:36.194453+0000","last_clean":"2023-10-16T10:00:36.194453+0000","last_became_active":"2023-10-16T10:00:35.032164+0000","last_became_peered":"2023-10-16T10:00:35.032164+0000","last_unstale":"2023-10-16T10:00:36.194453+0000","last_undegraded":"2023-10-16T10:00:36.194453+0000","last_fullsized":"2023-10-16T10:00:36.194453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:33.587345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194623+0000","last_change":"2023-10-16T10:00:35.039646+0000","last_active":"2023-10-16T10:00:36.194623+0000","last_peered":"2023-10-16T10:00:36.194623+0000","last_clean":"2023-10-16T10:00:36.194623+0000","last_became_active":"2023-10-16T10:00:35.039350+0000","last_became_peered":"2023-10-16T10:00:35.039350+0000","last_unstale":"2023-10-16T10:00:36.194623+0000","last_undegraded":"2023-10-16T10:00:36.194623+0000","last_fullsized":"2023-10-16T10:00:36.194623+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:17.795751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194950+0000","last_change":"2023-10-16T10:00:35.030516+0000","last_active":"2023-10-16T10:00:36.194950+0000","last_peered":"2023-10-16T10:00:36.194950+0000","last_clean":"2023-10-16T10:00:36.194950+0000","last_became_active":"2023-10-16T10:00:35.029991+0000","last_became_peered":"2023-10-16T10:00:35.029991+0000","last_unstale":"2023-10-16T10:00:36.194950+0000","last_undegraded":"2023-10-16T10:00:36.194950+0000","last_fullsized":"2023-10-16T10:00:36.194950+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:47.125673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195465+0000","last_change":"2023-10-16T10:00:34.033033+0000","last_active":"2023-10-16T10:00:36.195465+0000","last_peered":"2023-10-16T10:00:36.195465+0000","last_clean":"2023-10-16T10:00:36.195465+0000","last_became_active":"2023-10-16T10:00:34.032789+0000","last_became_peered":"2023-10-16T10:00:34.032789+0000","last_unstale":"2023-10-16T10:00:36.195465+0000","last_undegraded":"2023-10-16T10:00:36.195465+0000","last_fullsized":"2023-10-16T10:00:36.195465+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:45.257976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195601+0000","last_change":"2023-10-16T10:00:35.031881+0000","last_active":"2023-10-16T10:00:36.195601+0000","last_peered":"2023-10-16T10:00:36.195601+0000","last_clean":"2023-10-16T10:00:36.195601+0000","last_became_active":"2023-10-16T10:00:35.031573+0000","last_became_peered":"2023-10-16T10:00:35.031573+0000","last_unstale":"2023-10-16T10:00:36.195601+0000","last_undegraded":"2023-10-16T10:00:36.195601+0000","last_fullsized":"2023-10-16T10:00:36.195601+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:27.979949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195058+0000","last_change":"2023-10-16T10:00:34.019115+0000","last_active":"2023-10-16T10:00:36.195058+0000","last_peered":"2023-10-16T10:00:36.195058+0000","last_clean":"2023-10-16T10:00:36.195058+0000","last_became_active":"2023-10-16T10:00:34.018749+0000","last_became_peered":"2023-10-16T10:00:34.018749+0000","last_unstale":"2023-10-16T10:00:36.195058+0000","last_undegraded":"2023-10-16T10:00:36.195058+0000","last_fullsized":"2023-10-16T10:00:36.195058+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:08:31.252808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195466+0000","last_change":"2023-10-16T10:00:35.039568+0000","last_active":"2023-10-16T10:00:36.195466+0000","last_peered":"2023-10-16T10:00:36.195466+0000","last_clean":"2023-10-16T10:00:36.195466+0000","last_became_active":"2023-10-16T10:00:35.039260+0000","last_became_peered":"2023-10-16T10:00:35.039260+0000","last_unstale":"2023-10-16T10:00:36.195466+0000","last_undegraded":"2023-10-16T10:00:36.195466+0000","last_fullsized":"2023-10-16T10:00:36.195466+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:44.025934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195538+0000","last_change":"2023-10-16T10:00:34.020596+0000","last_active":"2023-10-16T10:00:36.195538+0000","last_peered":"2023-10-16T10:00:36.195538+0000","last_clean":"2023-10-16T10:00:36.195538+0000","last_became_active":"2023-10-16T10:00:34.020327+0000","last_became_peered":"2023-10-16T10:00:34.020327+0000","last_unstale":"2023-10-16T10:00:36.195538+0000","last_undegraded":"2023-10-16T10:00:36.195538+0000","last_fullsized":"2023-10-16T10:00:36.195538+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:11.025672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028758+0000","last_change":"2023-10-16T10:00:35.021938+0000","last_active":"2023-10-16T10:00:36.028758+0000","last_peered":"2023-10-16T10:00:36.028758+0000","last_clean":"2023-10-16T10:00:36.028758+0000","last_became_active":"2023-10-16T10:00:35.021640+0000","last_became_peered":"2023-10-16T10:00:35.021640+0000","last_unstale":"2023-10-16T10:00:36.028758+0000","last_undegraded":"2023-10-16T10:00:36.028758+0000","last_fullsized":"2023-10-16T10:00:36.028758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:44.669129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195397+0000","last_change":"2023-10-16T10:00:34.024584+0000","last_active":"2023-10-16T10:00:36.195397+0000","last_peered":"2023-10-16T10:00:36.195397+0000","last_clean":"2023-10-16T10:00:36.195397+0000","last_became_active":"2023-10-16T10:00:34.024322+0000","last_became_peered":"2023-10-16T10:00:34.024322+0000","last_unstale":"2023-10-16T10:00:36.195397+0000","last_undegraded":"2023-10-16T10:00:36.195397+0000","last_fullsized":"2023-10-16T10:00:36.195397+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:45.257976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194979+0000","last_change":"2023-10-16T10:00:35.036184+0000","last_active":"2023-10-16T10:00:36.194979+0000","last_peered":"2023-10-16T10:00:36.194979+0000","last_clean":"2023-10-16T10:00:36.194979+0000","last_became_active":"2023-10-16T10:00:35.031821+0000","last_became_peered":"2023-10-16T10:00:35.031821+0000","last_unstale":"2023-10-16T10:00:36.194979+0000","last_undegraded":"2023-10-16T10:00:36.194979+0000","last_fullsized":"2023-10-16T10:00:36.194979+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:21.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194831+0000","last_change":"2023-10-16T10:00:34.018191+0000","last_active":"2023-10-16T10:00:36.194831+0000","last_peered":"2023-10-16T10:00:36.194831+0000","last_clean":"2023-10-16T10:00:36.194831+0000","last_became_active":"2023-10-16T10:00:34.017859+0000","last_became_peered":"2023-10-16T10:00:34.017859+0000","last_unstale":"2023-10-16T10:00:36.194831+0000","last_undegraded":"2023-10-16T10:00:36.194831+0000","last_fullsized":"2023-10-16T10:00:36.194831+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:11.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194844+0000","last_change":"2023-10-16T10:00:35.031066+0000","last_active":"2023-10-16T10:00:36.194844+0000","last_peered":"2023-10-16T10:00:36.194844+0000","last_clean":"2023-10-16T10:00:36.194844+0000","last_became_active":"2023-10-16T10:00:35.029442+0000","last_became_peered":"2023-10-16T10:00:35.029442+0000","last_unstale":"2023-10-16T10:00:36.194844+0000","last_undegraded":"2023-10-16T10:00:36.194844+0000","last_fullsized":"2023-10-16T10:00:36.194844+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:41.123342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194443+0000","last_change":"2023-10-16T10:00:34.019657+0000","last_active":"2023-10-16T10:00:36.194443+0000","last_peered":"2023-10-16T10:00:36.194443+0000","last_clean":"2023-10-16T10:00:36.194443+0000","last_became_active":"2023-10-16T10:00:34.019292+0000","last_became_peered":"2023-10-16T10:00:34.019292+0000","last_unstale":"2023-10-16T10:00:36.194443+0000","last_undegraded":"2023-10-16T10:00:36.194443+0000","last_fullsized":"2023-10-16T10:00:36.194443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:17:35.927298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028859+0000","last_change":"2023-10-16T10:00:35.026951+0000","last_active":"2023-10-16T10:00:36.028859+0000","last_peered":"2023-10-16T10:00:36.028859+0000","last_clean":"2023-10-16T10:00:36.028859+0000","last_became_active":"2023-10-16T10:00:35.026641+0000","last_became_peered":"2023-10-16T10:00:35.026641+0000","last_unstale":"2023-10-16T10:00:36.028859+0000","last_undegraded":"2023-10-16T10:00:36.028859+0000","last_fullsized":"2023-10-16T10:00:36.028859+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:42.215135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195315+0000","last_change":"2023-10-16T10:00:34.024394+0000","last_active":"2023-10-16T10:00:36.195315+0000","last_peered":"2023-10-16T10:00:36.195315+0000","last_clean":"2023-10-16T10:00:36.195315+0000","last_became_active":"2023-10-16T10:00:34.024140+0000","last_became_peered":"2023-10-16T10:00:34.024140+0000","last_unstale":"2023-10-16T10:00:36.195315+0000","last_undegraded":"2023-10-16T10:00:36.195315+0000","last_fullsized":"2023-10-16T10:00:36.195315+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:54.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027299+0000","last_change":"2023-10-16T10:00:35.022448+0000","last_active":"2023-10-16T10:00:36.027299+0000","last_peered":"2023-10-16T10:00:36.027299+0000","last_clean":"2023-10-16T10:00:36.027299+0000","last_became_active":"2023-10-16T10:00:35.022143+0000","last_became_peered":"2023-10-16T10:00:35.022143+0000","last_unstale":"2023-10-16T10:00:36.027299+0000","last_undegraded":"2023-10-16T10:00:36.027299+0000","last_fullsized":"2023-10-16T10:00:36.027299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:44.962479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195584+0000","last_change":"2023-10-16T10:00:34.021987+0000","last_active":"2023-10-16T10:00:36.195584+0000","last_peered":"2023-10-16T10:00:36.195584+0000","last_clean":"2023-10-16T10:00:36.195584+0000","last_became_active":"2023-10-16T10:00:34.021718+0000","last_became_peered":"2023-10-16T10:00:34.021718+0000","last_unstale":"2023-10-16T10:00:36.195584+0000","last_undegraded":"2023-10-16T10:00:36.195584+0000","last_fullsized":"2023-10-16T10:00:36.195584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:34.097110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027558+0000","last_change":"2023-10-16T10:00:35.034809+0000","last_active":"2023-10-16T10:00:36.027558+0000","last_peered":"2023-10-16T10:00:36.027558+0000","last_clean":"2023-10-16T10:00:36.027558+0000","last_became_active":"2023-10-16T10:00:35.034391+0000","last_became_peered":"2023-10-16T10:00:35.034391+0000","last_unstale":"2023-10-16T10:00:36.027558+0000","last_undegraded":"2023-10-16T10:00:36.027558+0000","last_fullsized":"2023-10-16T10:00:36.027558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:42.659658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195429+0000","last_change":"2023-10-16T10:00:34.033022+0000","last_active":"2023-10-16T10:00:36.195429+0000","last_peered":"2023-10-16T10:00:36.195429+0000","last_clean":"2023-10-16T10:00:36.195429+0000","last_became_active":"2023-10-16T10:00:34.032749+0000","last_became_peered":"2023-10-16T10:00:34.032749+0000","last_unstale":"2023-10-16T10:00:36.195429+0000","last_undegraded":"2023-10-16T10:00:36.195429+0000","last_fullsized":"2023-10-16T10:00:36.195429+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:33.250411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029543+0000","last_change":"2023-10-16T10:00:35.027652+0000","last_active":"2023-10-16T10:00:36.029543+0000","last_peered":"2023-10-16T10:00:36.029543+0000","last_clean":"2023-10-16T10:00:36.029543+0000","last_became_active":"2023-10-16T10:00:35.027407+0000","last_became_peered":"2023-10-16T10:00:35.027407+0000","last_unstale":"2023-10-16T10:00:36.029543+0000","last_undegraded":"2023-10-16T10:00:36.029543+0000","last_fullsized":"2023-10-16T10:00:36.029543+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:14.181745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195528+0000","last_change":"2023-10-16T10:00:34.026897+0000","last_active":"2023-10-16T10:00:36.195528+0000","last_peered":"2023-10-16T10:00:36.195528+0000","last_clean":"2023-10-16T10:00:36.195528+0000","last_became_active":"2023-10-16T10:00:34.026673+0000","last_became_peered":"2023-10-16T10:00:34.026673+0000","last_unstale":"2023-10-16T10:00:36.195528+0000","last_undegraded":"2023-10-16T10:00:36.195528+0000","last_fullsized":"2023-10-16T10:00:36.195528+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:23.595761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195406+0000","last_change":"2023-10-16T10:00:35.031886+0000","last_active":"2023-10-16T10:00:36.195406+0000","last_peered":"2023-10-16T10:00:36.195406+0000","last_clean":"2023-10-16T10:00:36.195406+0000","last_became_active":"2023-10-16T10:00:35.031623+0000","last_became_peered":"2023-10-16T10:00:35.031623+0000","last_unstale":"2023-10-16T10:00:36.195406+0000","last_undegraded":"2023-10-16T10:00:36.195406+0000","last_fullsized":"2023-10-16T10:00:36.195406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:41.574638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195121+0000","last_change":"2023-10-16T10:00:34.030892+0000","last_active":"2023-10-16T10:00:36.195121+0000","last_peered":"2023-10-16T10:00:36.195121+0000","last_clean":"2023-10-16T10:00:36.195121+0000","last_became_active":"2023-10-16T10:00:34.030578+0000","last_became_peered":"2023-10-16T10:00:34.030578+0000","last_unstale":"2023-10-16T10:00:36.195121+0000","last_undegraded":"2023-10-16T10:00:36.195121+0000","last_fullsized":"2023-10-16T10:00:36.195121+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.273373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194810+0000","last_change":"2023-10-16T10:00:35.032269+0000","last_active":"2023-10-16T10:00:36.194810+0000","last_peered":"2023-10-16T10:00:36.194810+0000","last_clean":"2023-10-16T10:00:36.194810+0000","last_became_active":"2023-10-16T10:00:35.031666+0000","last_became_peered":"2023-10-16T10:00:35.031666+0000","last_unstale":"2023-10-16T10:00:36.194810+0000","last_undegraded":"2023-10-16T10:00:36.194810+0000","last_fullsized":"2023-10-16T10:00:36.194810+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:05.521100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029723+0000","last_change":"2023-10-16T10:00:34.023919+0000","last_active":"2023-10-16T10:00:36.029723+0000","last_peered":"2023-10-16T10:00:36.029723+0000","last_clean":"2023-10-16T10:00:36.029723+0000","last_became_active":"2023-10-16T10:00:34.023638+0000","last_became_peered":"2023-10-16T10:00:34.023638+0000","last_unstale":"2023-10-16T10:00:36.029723+0000","last_undegraded":"2023-10-16T10:00:36.029723+0000","last_fullsized":"2023-10-16T10:00:36.029723+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:39:10.822350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195506+0000","last_change":"2023-10-16T10:00:35.022990+0000","last_active":"2023-10-16T10:00:36.195506+0000","last_peered":"2023-10-16T10:00:36.195506+0000","last_clean":"2023-10-16T10:00:36.195506+0000","last_became_active":"2023-10-16T10:00:35.022739+0000","last_became_peered":"2023-10-16T10:00:35.022739+0000","last_unstale":"2023-10-16T10:00:36.195506+0000","last_undegraded":"2023-10-16T10:00:36.195506+0000","last_fullsized":"2023-10-16T10:00:36.195506+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:15.018930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194633+0000","last_change":"2023-10-16T10:00:34.023004+0000","last_active":"2023-10-16T10:00:36.194633+0000","last_peered":"2023-10-16T10:00:36.194633+0000","last_clean":"2023-10-16T10:00:36.194633+0000","last_became_active":"2023-10-16T10:00:34.022776+0000","last_became_peered":"2023-10-16T10:00:34.022776+0000","last_unstale":"2023-10-16T10:00:36.194633+0000","last_undegraded":"2023-10-16T10:00:36.194633+0000","last_fullsized":"2023-10-16T10:00:36.194633+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:40.569358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028733+0000","last_change":"2023-10-16T10:00:35.033813+0000","last_active":"2023-10-16T10:00:36.028733+0000","last_peered":"2023-10-16T10:00:36.028733+0000","last_clean":"2023-10-16T10:00:36.028733+0000","last_became_active":"2023-10-16T10:00:35.033322+0000","last_became_peered":"2023-10-16T10:00:35.033322+0000","last_unstale":"2023-10-16T10:00:36.028733+0000","last_undegraded":"2023-10-16T10:00:36.028733+0000","last_fullsized":"2023-10-16T10:00:36.028733+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:06.831142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027374+0000","last_change":"2023-10-16T10:00:34.028095+0000","last_active":"2023-10-16T10:00:36.027374+0000","last_peered":"2023-10-16T10:00:36.027374+0000","last_clean":"2023-10-16T10:00:36.027374+0000","last_became_active":"2023-10-16T10:00:34.027833+0000","last_became_peered":"2023-10-16T10:00:34.027833+0000","last_unstale":"2023-10-16T10:00:36.027374+0000","last_undegraded":"2023-10-16T10:00:36.027374+0000","last_fullsized":"2023-10-16T10:00:36.027374+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:36.102659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194360+0000","last_change":"2023-10-16T10:00:35.037036+0000","last_active":"2023-10-16T10:00:36.194360+0000","last_peered":"2023-10-16T10:00:36.194360+0000","last_clean":"2023-10-16T10:00:36.194360+0000","last_became_active":"2023-10-16T10:00:35.036760+0000","last_became_peered":"2023-10-16T10:00:35.036760+0000","last_unstale":"2023-10-16T10:00:36.194360+0000","last_undegraded":"2023-10-16T10:00:36.194360+0000","last_fullsized":"2023-10-16T10:00:36.194360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:29.215922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029375+0000","last_change":"2023-10-16T10:00:34.024085+0000","last_active":"2023-10-16T10:00:36.029375+0000","last_peered":"2023-10-16T10:00:36.029375+0000","last_clean":"2023-10-16T10:00:36.029375+0000","last_became_active":"2023-10-16T10:00:34.023717+0000","last_became_peered":"2023-10-16T10:00:34.023717+0000","last_unstale":"2023-10-16T10:00:36.029375+0000","last_undegraded":"2023-10-16T10:00:36.029375+0000","last_fullsized":"2023-10-16T10:00:36.029375+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:41.927893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194378+0000","last_change":"2023-10-16T10:00:35.030139+0000","last_active":"2023-10-16T10:00:36.194378+0000","last_peered":"2023-10-16T10:00:36.194378+0000","last_clean":"2023-10-16T10:00:36.194378+0000","last_became_active":"2023-10-16T10:00:35.029815+0000","last_became_peered":"2023-10-16T10:00:35.029815+0000","last_unstale":"2023-10-16T10:00:36.194378+0000","last_undegraded":"2023-10-16T10:00:36.194378+0000","last_fullsized":"2023-10-16T10:00:36.194378+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:27.979949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194190+0000","last_change":"2023-10-16T10:00:34.030630+0000","last_active":"2023-10-16T10:00:36.194190+0000","last_peered":"2023-10-16T10:00:36.194190+0000","last_clean":"2023-10-16T10:00:36.194190+0000","last_became_active":"2023-10-16T10:00:34.030327+0000","last_became_peered":"2023-10-16T10:00:34.030327+0000","last_unstale":"2023-10-16T10:00:36.194190+0000","last_undegraded":"2023-10-16T10:00:36.194190+0000","last_fullsized":"2023-10-16T10:00:36.194190+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:17.802878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028838+0000","last_change":"2023-10-16T10:00:35.026016+0000","last_active":"2023-10-16T10:00:36.028838+0000","last_peered":"2023-10-16T10:00:36.028838+0000","last_clean":"2023-10-16T10:00:36.028838+0000","last_became_active":"2023-10-16T10:00:35.025674+0000","last_became_peered":"2023-10-16T10:00:35.025674+0000","last_unstale":"2023-10-16T10:00:36.028838+0000","last_undegraded":"2023-10-16T10:00:36.028838+0000","last_fullsized":"2023-10-16T10:00:36.028838+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:41.332427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195306+0000","last_change":"2023-10-16T10:00:34.030613+0000","last_active":"2023-10-16T10:00:36.195306+0000","last_peered":"2023-10-16T10:00:36.195306+0000","last_clean":"2023-10-16T10:00:36.195306+0000","last_became_active":"2023-10-16T10:00:34.030308+0000","last_became_peered":"2023-10-16T10:00:34.030308+0000","last_unstale":"2023-10-16T10:00:36.195306+0000","last_undegraded":"2023-10-16T10:00:36.195306+0000","last_fullsized":"2023-10-16T10:00:36.195306+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:58.224753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195587+0000","last_change":"2023-10-16T10:00:35.027988+0000","last_active":"2023-10-16T10:00:36.195587+0000","last_peered":"2023-10-16T10:00:36.195587+0000","last_clean":"2023-10-16T10:00:36.195587+0000","last_became_active":"2023-10-16T10:00:35.027594+0000","last_became_peered":"2023-10-16T10:00:35.027594+0000","last_unstale":"2023-10-16T10:00:36.195587+0000","last_undegraded":"2023-10-16T10:00:36.195587+0000","last_fullsized":"2023-10-16T10:00:36.195587+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:35.102390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195243+0000","last_change":"2023-10-16T10:00:34.022927+0000","last_active":"2023-10-16T10:00:36.195243+0000","last_peered":"2023-10-16T10:00:36.195243+0000","last_clean":"2023-10-16T10:00:36.195243+0000","last_became_active":"2023-10-16T10:00:34.022678+0000","last_became_peered":"2023-10-16T10:00:34.022678+0000","last_unstale":"2023-10-16T10:00:36.195243+0000","last_undegraded":"2023-10-16T10:00:36.195243+0000","last_fullsized":"2023-10-16T10:00:36.195243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:09.291315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195009+0000","last_change":"2023-10-16T10:00:35.039591+0000","last_active":"2023-10-16T10:00:36.195009+0000","last_peered":"2023-10-16T10:00:36.195009+0000","last_clean":"2023-10-16T10:00:36.195009+0000","last_became_active":"2023-10-16T10:00:35.039279+0000","last_became_peered":"2023-10-16T10:00:35.039279+0000","last_unstale":"2023-10-16T10:00:36.195009+0000","last_undegraded":"2023-10-16T10:00:36.195009+0000","last_fullsized":"2023-10-16T10:00:36.195009+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:58:57.667724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195501+0000","last_change":"2023-10-16T10:00:34.024570+0000","last_active":"2023-10-16T10:00:36.195501+0000","last_peered":"2023-10-16T10:00:36.195501+0000","last_clean":"2023-10-16T10:00:36.195501+0000","last_became_active":"2023-10-16T10:00:34.023970+0000","last_became_peered":"2023-10-16T10:00:34.023970+0000","last_unstale":"2023-10-16T10:00:36.195501+0000","last_undegraded":"2023-10-16T10:00:36.195501+0000","last_fullsized":"2023-10-16T10:00:36.195501+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:58.224753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194835+0000","last_change":"2023-10-16T10:00:35.037806+0000","last_active":"2023-10-16T10:00:36.194835+0000","last_peered":"2023-10-16T10:00:36.194835+0000","last_clean":"2023-10-16T10:00:36.194835+0000","last_became_active":"2023-10-16T10:00:35.037366+0000","last_became_peered":"2023-10-16T10:00:35.037366+0000","last_unstale":"2023-10-16T10:00:36.194835+0000","last_undegraded":"2023-10-16T10:00:36.194835+0000","last_fullsized":"2023-10-16T10:00:36.194835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:17.795751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027547+0000","last_change":"2023-10-16T10:00:34.021412+0000","last_active":"2023-10-16T10:00:36.027547+0000","last_peered":"2023-10-16T10:00:36.027547+0000","last_clean":"2023-10-16T10:00:36.027547+0000","last_became_active":"2023-10-16T10:00:34.021132+0000","last_became_peered":"2023-10-16T10:00:34.021132+0000","last_unstale":"2023-10-16T10:00:36.027547+0000","last_undegraded":"2023-10-16T10:00:36.027547+0000","last_fullsized":"2023-10-16T10:00:36.027547+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:43.904441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029517+0000","last_change":"2023-10-16T10:00:35.027880+0000","last_active":"2023-10-16T10:00:36.029517+0000","last_peered":"2023-10-16T10:00:36.029517+0000","last_clean":"2023-10-16T10:00:36.029517+0000","last_became_active":"2023-10-16T10:00:35.027199+0000","last_became_peered":"2023-10-16T10:00:35.027199+0000","last_unstale":"2023-10-16T10:00:36.029517+0000","last_undegraded":"2023-10-16T10:00:36.029517+0000","last_fullsized":"2023-10-16T10:00:36.029517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:24.915071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194997+0000","last_change":"2023-10-16T10:00:34.025139+0000","last_active":"2023-10-16T10:00:36.194997+0000","last_peered":"2023-10-16T10:00:36.194997+0000","last_clean":"2023-10-16T10:00:36.194997+0000","last_became_active":"2023-10-16T10:00:34.024886+0000","last_became_peered":"2023-10-16T10:00:34.024886+0000","last_unstale":"2023-10-16T10:00:36.194997+0000","last_undegraded":"2023-10-16T10:00:36.194997+0000","last_fullsized":"2023-10-16T10:00:36.194997+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:44:43.737230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194797+0000","last_change":"2023-10-16T10:00:35.026459+0000","last_active":"2023-10-16T10:00:36.194797+0000","last_peered":"2023-10-16T10:00:36.194797+0000","last_clean":"2023-10-16T10:00:36.194797+0000","last_became_active":"2023-10-16T10:00:35.026148+0000","last_became_peered":"2023-10-16T10:00:35.026148+0000","last_unstale":"2023-10-16T10:00:36.194797+0000","last_undegraded":"2023-10-16T10:00:36.194797+0000","last_fullsized":"2023-10-16T10:00:36.194797+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:41.574638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194419+0000","last_change":"2023-10-16T10:00:34.030626+0000","last_active":"2023-10-16T10:00:36.194419+0000","last_peered":"2023-10-16T10:00:36.194419+0000","last_clean":"2023-10-16T10:00:36.194419+0000","last_became_active":"2023-10-16T10:00:34.030283+0000","last_became_peered":"2023-10-16T10:00:34.030283+0000","last_unstale":"2023-10-16T10:00:36.194419+0000","last_undegraded":"2023-10-16T10:00:36.194419+0000","last_fullsized":"2023-10-16T10:00:36.194419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:09.291315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029368+0000","last_change":"2023-10-16T10:00:35.033496+0000","last_active":"2023-10-16T10:00:36.029368+0000","last_peered":"2023-10-16T10:00:36.029368+0000","last_clean":"2023-10-16T10:00:36.029368+0000","last_became_active":"2023-10-16T10:00:35.033225+0000","last_became_peered":"2023-10-16T10:00:35.033225+0000","last_unstale":"2023-10-16T10:00:36.029368+0000","last_undegraded":"2023-10-16T10:00:36.029368+0000","last_fullsized":"2023-10-16T10:00:36.029368+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:56:41.501697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027174+0000","last_change":"2023-10-16T10:00:34.023037+0000","last_active":"2023-10-16T10:00:36.027174+0000","last_peered":"2023-10-16T10:00:36.027174+0000","last_clean":"2023-10-16T10:00:36.027174+0000","last_became_active":"2023-10-16T10:00:34.022781+0000","last_became_peered":"2023-10-16T10:00:34.022781+0000","last_unstale":"2023-10-16T10:00:36.027174+0000","last_undegraded":"2023-10-16T10:00:36.027174+0000","last_fullsized":"2023-10-16T10:00:36.027174+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:51.044931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194965+0000","last_change":"2023-10-16T10:00:35.033027+0000","last_active":"2023-10-16T10:00:36.194965+0000","last_peered":"2023-10-16T10:00:36.194965+0000","last_clean":"2023-10-16T10:00:36.194965+0000","last_became_active":"2023-10-16T10:00:35.032717+0000","last_became_peered":"2023-10-16T10:00:35.032717+0000","last_unstale":"2023-10-16T10:00:36.194965+0000","last_undegraded":"2023-10-16T10:00:36.194965+0000","last_fullsized":"2023-10-16T10:00:36.194965+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:58.687087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194483+0000","last_change":"2023-10-16T10:00:34.028275+0000","last_active":"2023-10-16T10:00:36.194483+0000","last_peered":"2023-10-16T10:00:36.194483+0000","last_clean":"2023-10-16T10:00:36.194483+0000","last_became_active":"2023-10-16T10:00:34.028012+0000","last_became_peered":"2023-10-16T10:00:34.028012+0000","last_unstale":"2023-10-16T10:00:36.194483+0000","last_undegraded":"2023-10-16T10:00:36.194483+0000","last_fullsized":"2023-10-16T10:00:36.194483+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:16.790470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028967+0000","last_change":"2023-10-16T10:00:35.029045+0000","last_active":"2023-10-16T10:00:36.028967+0000","last_peered":"2023-10-16T10:00:36.028967+0000","last_clean":"2023-10-16T10:00:36.028967+0000","last_became_active":"2023-10-16T10:00:35.028766+0000","last_became_peered":"2023-10-16T10:00:35.028766+0000","last_unstale":"2023-10-16T10:00:36.028967+0000","last_undegraded":"2023-10-16T10:00:36.028967+0000","last_fullsized":"2023-10-16T10:00:36.028967+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:28.174207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195143+0000","last_change":"2023-10-16T10:00:34.023093+0000","last_active":"2023-10-16T10:00:36.195143+0000","last_peered":"2023-10-16T10:00:36.195143+0000","last_clean":"2023-10-16T10:00:36.195143+0000","last_became_active":"2023-10-16T10:00:34.022828+0000","last_became_peered":"2023-10-16T10:00:34.022828+0000","last_unstale":"2023-10-16T10:00:36.195143+0000","last_undegraded":"2023-10-16T10:00:36.195143+0000","last_fullsized":"2023-10-16T10:00:36.195143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:09:48.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195551+0000","last_change":"2023-10-16T10:00:35.033988+0000","last_active":"2023-10-16T10:00:36.195551+0000","last_peered":"2023-10-16T10:00:36.195551+0000","last_clean":"2023-10-16T10:00:36.195551+0000","last_became_active":"2023-10-16T10:00:35.032936+0000","last_became_peered":"2023-10-16T10:00:35.032936+0000","last_unstale":"2023-10-16T10:00:36.195551+0000","last_undegraded":"2023-10-16T10:00:36.195551+0000","last_fullsized":"2023-10-16T10:00:36.195551+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:38.437167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195372+0000","last_change":"2023-10-16T10:00:34.030314+0000","last_active":"2023-10-16T10:00:36.195372+0000","last_peered":"2023-10-16T10:00:36.195372+0000","last_clean":"2023-10-16T10:00:36.195372+0000","last_became_active":"2023-10-16T10:00:34.030055+0000","last_became_peered":"2023-10-16T10:00:34.030055+0000","last_unstale":"2023-10-16T10:00:36.195372+0000","last_undegraded":"2023-10-16T10:00:36.195372+0000","last_fullsized":"2023-10-16T10:00:36.195372+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:11.025672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195141+0000","last_change":"2023-10-16T10:00:35.032320+0000","last_active":"2023-10-16T10:00:36.195141+0000","last_peered":"2023-10-16T10:00:36.195141+0000","last_clean":"2023-10-16T10:00:36.195141+0000","last_became_active":"2023-10-16T10:00:35.031986+0000","last_became_peered":"2023-10-16T10:00:35.031986+0000","last_unstale":"2023-10-16T10:00:36.195141+0000","last_undegraded":"2023-10-16T10:00:36.195141+0000","last_fullsized":"2023-10-16T10:00:36.195141+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:03.996992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195657+0000","last_change":"2023-10-16T10:00:34.030558+0000","last_active":"2023-10-16T10:00:36.195657+0000","last_peered":"2023-10-16T10:00:36.195657+0000","last_clean":"2023-10-16T10:00:36.195657+0000","last_became_active":"2023-10-16T10:00:34.030249+0000","last_became_peered":"2023-10-16T10:00:34.030249+0000","last_unstale":"2023-10-16T10:00:36.195657+0000","last_undegraded":"2023-10-16T10:00:36.195657+0000","last_fullsized":"2023-10-16T10:00:36.195657+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:40.569358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029498+0000","last_change":"2023-10-16T10:00:35.027926+0000","last_active":"2023-10-16T10:00:36.029498+0000","last_peered":"2023-10-16T10:00:36.029498+0000","last_clean":"2023-10-16T10:00:36.029498+0000","last_became_active":"2023-10-16T10:00:35.027208+0000","last_became_peered":"2023-10-16T10:00:35.027208+0000","last_unstale":"2023-10-16T10:00:36.029498+0000","last_undegraded":"2023-10-16T10:00:36.029498+0000","last_fullsized":"2023-10-16T10:00:36.029498+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:45.826798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195690+0000","last_change":"2023-10-16T10:00:34.028607+0000","last_active":"2023-10-16T10:00:36.195690+0000","last_peered":"2023-10-16T10:00:36.195690+0000","last_clean":"2023-10-16T10:00:36.195690+0000","last_became_active":"2023-10-16T10:00:34.028373+0000","last_became_peered":"2023-10-16T10:00:34.028373+0000","last_unstale":"2023-10-16T10:00:36.195690+0000","last_undegraded":"2023-10-16T10:00:36.195690+0000","last_fullsized":"2023-10-16T10:00:36.195690+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.273373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195725+0000","last_change":"2023-10-16T10:00:35.025215+0000","last_active":"2023-10-16T10:00:36.195725+0000","last_peered":"2023-10-16T10:00:36.195725+0000","last_clean":"2023-10-16T10:00:36.195725+0000","last_became_active":"2023-10-16T10:00:35.024962+0000","last_became_peered":"2023-10-16T10:00:35.024962+0000","last_unstale":"2023-10-16T10:00:36.195725+0000","last_undegraded":"2023-10-16T10:00:36.195725+0000","last_fullsized":"2023-10-16T10:00:36.195725+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.898298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194534+0000","last_change":"2023-10-16T10:00:34.019601+0000","last_active":"2023-10-16T10:00:36.194534+0000","last_peered":"2023-10-16T10:00:36.194534+0000","last_clean":"2023-10-16T10:00:36.194534+0000","last_became_active":"2023-10-16T10:00:34.019191+0000","last_became_peered":"2023-10-16T10:00:34.019191+0000","last_unstale":"2023-10-16T10:00:36.194534+0000","last_undegraded":"2023-10-16T10:00:36.194534+0000","last_fullsized":"2023-10-16T10:00:36.194534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:29.486652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028525+0000","last_change":"2023-10-16T10:00:35.040263+0000","last_active":"2023-10-16T10:00:36.028525+0000","last_peered":"2023-10-16T10:00:36.028525+0000","last_clean":"2023-10-16T10:00:36.028525+0000","last_became_active":"2023-10-16T10:00:35.040019+0000","last_became_peered":"2023-10-16T10:00:35.040019+0000","last_unstale":"2023-10-16T10:00:36.028525+0000","last_undegraded":"2023-10-16T10:00:36.028525+0000","last_fullsized":"2023-10-16T10:00:36.028525+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:40.411278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195228+0000","last_change":"2023-10-16T10:00:34.024325+0000","last_active":"2023-10-16T10:00:36.195228+0000","last_peered":"2023-10-16T10:00:36.195228+0000","last_clean":"2023-10-16T10:00:36.195228+0000","last_became_active":"2023-10-16T10:00:34.023163+0000","last_became_peered":"2023-10-16T10:00:34.023163+0000","last_unstale":"2023-10-16T10:00:36.195228+0000","last_undegraded":"2023-10-16T10:00:36.195228+0000","last_fullsized":"2023-10-16T10:00:36.195228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:58.045684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195232+0000","last_change":"2023-10-16T10:00:35.030312+0000","last_active":"2023-10-16T10:00:36.195232+0000","last_peered":"2023-10-16T10:00:36.195232+0000","last_clean":"2023-10-16T10:00:36.195232+0000","last_became_active":"2023-10-16T10:00:35.030025+0000","last_became_peered":"2023-10-16T10:00:35.030025+0000","last_unstale":"2023-10-16T10:00:36.195232+0000","last_undegraded":"2023-10-16T10:00:36.195232+0000","last_fullsized":"2023-10-16T10:00:36.195232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.898298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194319+0000","last_change":"2023-10-16T10:00:34.030609+0000","last_active":"2023-10-16T10:00:36.194319+0000","last_peered":"2023-10-16T10:00:36.194319+0000","last_clean":"2023-10-16T10:00:36.194319+0000","last_became_active":"2023-10-16T10:00:34.030331+0000","last_became_peered":"2023-10-16T10:00:34.030331+0000","last_unstale":"2023-10-16T10:00:36.194319+0000","last_undegraded":"2023-10-16T10:00:36.194319+0000","last_fullsized":"2023-10-16T10:00:36.194319+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:34:17.143076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194591+0000","last_change":"2023-10-16T10:00:35.029389+0000","last_active":"2023-10-16T10:00:36.194591+0000","last_peered":"2023-10-16T10:00:36.194591+0000","last_clean":"2023-10-16T10:00:36.194591+0000","last_became_active":"2023-10-16T10:00:35.029102+0000","last_became_peered":"2023-10-16T10:00:35.029102+0000","last_unstale":"2023-10-16T10:00:36.194591+0000","last_undegraded":"2023-10-16T10:00:36.194591+0000","last_fullsized":"2023-10-16T10:00:36.194591+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:05.398826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028919+0000","last_change":"2023-10-16T10:00:34.030564+0000","last_active":"2023-10-16T10:00:36.028919+0000","last_peered":"2023-10-16T10:00:36.028919+0000","last_clean":"2023-10-16T10:00:36.028919+0000","last_became_active":"2023-10-16T10:00:34.030220+0000","last_became_peered":"2023-10-16T10:00:34.030220+0000","last_unstale":"2023-10-16T10:00:36.028919+0000","last_undegraded":"2023-10-16T10:00:36.028919+0000","last_fullsized":"2023-10-16T10:00:36.028919+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:56.372390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"93'183","reported_seq":480,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-16T10:00:35.014682+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:00:35.014682+0000","last_peered":"2023-10-16T10:00:35.014682+0000","last_clean":"2023-10-16T10:00:35.014682+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:00:35.014682+0000","last_undegraded":"2023-10-16T10:00:35.014682+0000","last_fullsized":"2023-10-16T10:00:35.014682+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542344,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":1207497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55700000000000005,"15min":0.57999999999999996},"min":{"1min":0.34599999999999997,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.629,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.83099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.442}]},{"osd":1,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.379,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.77000000000000002,"15min":0.80400000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.59899999999999998},"min":{"1min":0.33500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72999999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.753,"5min":0.75900000000000001,"15min":0.81200000000000006},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68100000000000005}]},{"osd":4,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.32100000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":0.73799999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77600000000000002,"5min":1.006,"15min":1.006},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.437,"5min":0.38900000000000001,"15min":0.35599999999999998},"max":{"1min":0.79300000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.95199999999999996,"15min":1.373},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.308},"max":{"1min":0.748,"5min":0.85799999999999998,"15min":1.4099999999999999},"last":0.69799999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1273856,"data_stored":617753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.33800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.63200000000000001,"5min":0.745,"15min":0.83399999999999996},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54700000000000004,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.78900000000000003,"15min":0.81100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]},{"osd":2,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55900000000000005,"15min":0.57899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64500000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.55500000000000005,"15min":0.57899999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.63500000000000001,"5min":0.83699999999999997,"15min":0.84499999999999997},"last":0.57899999999999996}]},{"osd":3,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56899999999999995,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76300000000000001,"5min":0.85399999999999998,"15min":0.85899999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48699999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.311},"max":{"1min":1,"5min":1,"15min":1},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.37,"5min":0.35199999999999998,"15min":0.30199999999999999},"max":{"1min":0.69999999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.78600000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.61199999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.85199999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.83499999999999996,"15min":0.86399999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69499999999999995,"5min":0.83099999999999996,"15min":0.83399999999999996},"last":0.86699999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542343,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50480,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321360,"statfs":{"total":96636764160,"available":96585072640,"internally_reserved":0,"allocated":1273856,"data_stored":617253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.221,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46400000000000002,"5min":1.5,"15min":1.5},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.39700000000000002,"15min":0.40899999999999997},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.42699999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.308}]},{"osd":2,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47699999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45000000000000001,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55000000000000004,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.52400000000000002,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.42899999999999999,"15min":0.437},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.48499999999999999,"5min":1.379,"15min":1.379},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.40500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.40699999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.79700000000000004,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":1.26,"15min":1.26},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63,"15min":0.64800000000000002},"min":{"1min":0.45400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.80800000000000005,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.64800000000000002},"min":{"1min":0.36799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.749,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63600000000000001,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.64400000000000002,"15min":0.66300000000000003},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.41499999999999998},"max":{"1min":0.78300000000000003,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.67600000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542344,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":1200737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.36099999999999999,"15min":0.373},"min":{"1min":0.192,"5min":0.192,"15min":0.191},"max":{"1min":0.53800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.38600000000000001,"15min":0.40000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.48499999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.252}]},{"osd":2,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39400000000000002,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51900000000000002,"5min":0.61799999999999999,"15min":0.626},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.40000000000000002,"15min":0.42299999999999999},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.42999999999999999,"5min":0.624,"15min":0.63600000000000001},"last":0.313}]},{"osd":3,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40899999999999997,"15min":0.432},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58399999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59999999999999998,"5min":0.69299999999999995,"15min":0.69699999999999995},"last":0.33400000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60799999999999998,"15min":0.63},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.628},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.60899999999999999,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.90500000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.627,"15min":0.64700000000000002},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.004,"15min":1.004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.622,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65500000000000003}]},{"osd":6,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.81000000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.46100000000000002,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.63700000000000001,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82099999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.63500000000000001,"15min":0.65900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.91100000000000003,"15min":0.95699999999999996},"last":0.60199999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542343,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36584,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335256,"statfs":{"total":96636764160,"available":96599302144,"internally_reserved":0,"allocated":1265664,"data_stored":617641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.36899999999999999,"15min":0.375},"min":{"1min":0.252,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.45800000000000002,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.373},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.38700000000000001,"15min":0.39900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.51400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51500000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39700000000000002,"15min":0.40600000000000003},"min":{"1min":0.252,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.40500000000000003,"15min":0.42099999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.214},"max":{"1min":0.496,"5min":0.58199999999999996,"15min":0.64000000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.40699999999999997},"min":{"1min":0.25800000000000001,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.44600000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.435},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.44800000000000001,"15min":0.46700000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.747,"15min":0.747},"last":0.47199999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.436,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.747},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.43099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":1.091,"15min":1.091},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.85899999999999999,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.97499999999999998},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.35099999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.64600000000000002},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78500000000000003,"5min":0.93799999999999994,"15min":0.97899999999999998},"last":0.81999999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.624,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73699999999999999,"5min":0.98299999999999998,"15min":0.98799999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83099999999999996,"5min":0.874,"15min":0.998},"last":0.84099999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542344,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1261568,"data_stored":611947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.373,"15min":0.38400000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.192},"max":{"1min":0.52400000000000002,"5min":0.63900000000000001,"15min":0.64900000000000002},"last":0.217},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.28799999999999998,"5min":0.185,"15min":0.185},"max":{"1min":0.50700000000000001,"5min":0.626,"15min":0.63100000000000001},"last":0.26200000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.38400000000000001,"15min":0.39300000000000002},"min":{"1min":0.24399999999999999,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.437,"5min":0.66100000000000003,"15min":0.72199999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.41999999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.433,"15min":0.435},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.54200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.44,"15min":0.45100000000000001},"min":{"1min":0.30199999999999999,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.30599999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.95799999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":0.879,"15min":0.98699999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61799999999999999,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.996},"last":0.60399999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.92400000000000004,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59999999999999998,"15min":0.629},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.86799999999999999,"15min":0.99099999999999999},"last":0.74399999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":1.3,"15min":1.3},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81200000000000006,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.80000000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54624,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317216,"statfs":{"total":96636764160,"available":96580829184,"internally_reserved":0,"allocated":1261568,"data_stored":617059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.58199999999999996},"min":{"1min":0.30099999999999999,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.72399999999999998,"5min":0.76900000000000002,"15min":0.89400000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56799999999999995,"15min":0.58299999999999996},"min":{"1min":0.33300000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.63900000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.42399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.84599999999999997,"15min":0.86599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55000000000000004,"15min":0.57499999999999996},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.73999999999999999,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.54000000000000004}]},{"osd":2,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.755,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.501},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.81999999999999995,"15min":0.89100000000000001},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.42099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.67400000000000004,"5min":0.83199999999999996,"15min":0.86499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57399999999999995,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69399999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56599999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63300000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.73399999999999999,"15min":0.80400000000000005},"last":0.38800000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.82999999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79300000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.78600000000000003,"15min":0.80700000000000005},"last":0.624},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76000000000000001,"5min":0.83999999999999997,"15min":0.86399999999999999},"last":0.65200000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542344,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1265664,"data_stored":611969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.253},"max":{"1min":0.69899999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.378,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":0.84199999999999997,"15min":0.91800000000000004},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.56100000000000005,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":0.58899999999999997,"5min":0.77900000000000003,"15min":0.82799999999999996},"last":0.442},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.315,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.373},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.33900000000000002},"max":{"1min":0.70499999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54600000000000004}]},{"osd":3,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58699999999999997,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":0.88600000000000001,"15min":0.89000000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44500000000000001,"5min":0.39300000000000002,"15min":0.33200000000000002},"max":{"1min":0.77100000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.81499999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.38800000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.5}]},{"osd":7,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.88400000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:00:39.138 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:00:39.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 Processor -- start 2023-10-16T10:00:39.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 -- start start 2023-10-16T10:00:39.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5eac15dfd0 0x7f5eac0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5eac0af450 0x7f5eac06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac06db80 0x7f5eac0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac06f870 con 0x7f5eac0af450 2023-10-16T10:00:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac0d1160 con 0x7f5eac15dfd0 2023-10-16T10:00:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eb2035700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0cc8e0 con 0x7f5eac06db80 2023-10-16T10:00:39.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.290+0000 7f5eaaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5eac0af450 0x7f5eac06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5eac0af450 0x7f5eac06d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44550/0 (socket says 172.21.15.181:44550) 2023-10-16T10:00:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 -- 172.21.15.181:0/1598578175 learned_addr learned my addr 172.21.15.181:0/1598578175 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eabfff700 1 --2- 172.21.15.181:0/1598578175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac06db80 0x7f5eac0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eab7fe700 1 --2- 172.21.15.181:0/1598578175 >> v2:172.21.15.190:3300/0 conn(0x7f5eac15dfd0 0x7f5eac0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 -- 172.21.15.181:0/1598578175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac06db80 msgr2=0x7f5eac0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 --2- 172.21.15.181:0/1598578175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac06db80 0x7f5eac0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 -- 172.21.15.181:0/1598578175 >> v2:172.21.15.190:3300/0 conn(0x7f5eac15dfd0 msgr2=0x7f5eac0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 --2- 172.21.15.181:0/1598578175 >> v2:172.21.15.190:3300/0 conn(0x7f5eac15dfd0 0x7f5eac0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 -- 172.21.15.181:0/1598578175 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac0ccbc0 con 0x7f5eac0af450 2023-10-16T10:00:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaaffd700 1 --2- 172.21.15.181:0/1598578175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac0af450 0x7f5eac06d640 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5eac070340 tx=0x7f5ea0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcb0a0536d53c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaa7fc700 1 -- 172.21.15.181:0/1598578175 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0011830 con 0x7f5eac0af450 2023-10-16T10:00:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaa7fc700 1 -- 172.21.15.181:0/1598578175 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea0011990 con 0x7f5eac0af450 2023-10-16T10:00:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.291+0000 7f5eaa7fc700 1 -- 172.21.15.181:0/1598578175 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0011d20 con 0x7f5eac0af450 2023-10-16T10:00:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 -- 172.21.15.181:0/1598578175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac0af450 msgr2=0x7f5eac06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1598578175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac0af450 0x7f5eac06d640 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5eac070340 tx=0x7f5ea0009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 -- 172.21.15.181:0/1598578175 shutdown_connections 2023-10-16T10:00:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1598578175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac06db80 0x7f5eac0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1598578175 >> v2:172.21.15.190:3300/0 conn(0x7f5eac15dfd0 0x7f5eac0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1598578175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac0af450 0x7f5eac06d640 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 -- 172.21.15.181:0/1598578175 >> 172.21.15.181:0/1598578175 conn(0x7f5eac0d04a0 msgr2=0x7f5eac0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 -- 172.21.15.181:0/1598578175 shutdown_connections 2023-10-16T10:00:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.292+0000 7f5eb2035700 1 -- 172.21.15.181:0/1598578175 wait complete. 2023-10-16T10:00:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.293+0000 7f5eb2035700 1 Processor -- start 2023-10-16T10:00:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.293+0000 7f5eb2035700 1 -- start start 2023-10-16T10:00:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eb2035700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac06db80 0x7f5eac14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eb2035700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac0af450 0x7f5eac0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eb2035700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac15dfd0 0x7f5eac0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eb2035700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0c7c70 con 0x7f5eac06db80 2023-10-16T10:00:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eb2035700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0b1310 con 0x7f5eac15dfd0 2023-10-16T10:00:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eb2035700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0b0d50 con 0x7f5eac0af450 2023-10-16T10:00:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac06db80 0x7f5eac14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac06db80 0x7f5eac14c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44554/0 (socket says 172.21.15.181:44554) 2023-10-16T10:00:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 -- 172.21.15.181:0/1520433992 learned_addr learned my addr 172.21.15.181:0/1520433992 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eaaffd700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac0af450 0x7f5eac0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eabfff700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac15dfd0 0x7f5eac0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 -- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac0af450 msgr2=0x7f5eac0b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac0af450 0x7f5eac0b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 -- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac15dfd0 msgr2=0x7f5eac0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac15dfd0 0x7f5eac0b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 -- 172.21.15.181:0/1520433992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0009550 con 0x7f5eac06db80 2023-10-16T10:00:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.294+0000 7f5eab7fe700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac06db80 0x7f5eac14c6b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00e900 tx=0x7f5e9c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.295+0000 7f5e93fff700 1 -- 172.21.15.181:0/1520433992 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c013680 con 0x7f5eac06db80 2023-10-16T10:00:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.295+0000 7f5e93fff700 1 -- 172.21.15.181:0/1520433992 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0137e0 con 0x7f5eac06db80 2023-10-16T10:00:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.295+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac0b1030 con 0x7f5eac06db80 2023-10-16T10:00:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.295+0000 7f5e93fff700 1 -- 172.21.15.181:0/1520433992 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c013b50 con 0x7f5eac06db80 2023-10-16T10:00:39.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.295+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac0c4350 con 0x7f5eac06db80 2023-10-16T10:00:39.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.295+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac0c4510 con 0x7f5eac06db80 2023-10-16T10:00:39.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.296+0000 7f5e93fff700 1 -- 172.21.15.181:0/1520433992 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5e9c013cb0 con 0x7f5eac06db80 2023-10-16T10:00:39.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.296+0000 7f5e93fff700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5e9806b020 0x7f5e9806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.297+0000 7f5eaaffd700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5e9806b020 0x7f5e9806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.297+0000 7f5e93fff700 1 -- 172.21.15.181:0/1520433992 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f5e9c089c20 con 0x7f5eac06db80 2023-10-16T10:00:39.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.297+0000 7f5eaaffd700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5e9806b020 0x7f5e9806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009840 tx=0x7f5ea0008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:39.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.301+0000 7f5e93fff700 1 -- 172.21.15.181:0/1520433992 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5e9c054a10 con 0x7f5eac06db80 2023-10-16T10:00:39.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.489+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 --> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5eac0cc590 con 0x7f5e9806b020 2023-10-16T10:00:39.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.496+0000 7f5e93fff700 1 -- 172.21.15.181:0/1520433992 <== mgr.7254 v2:172.21.15.181:6832/3229300335 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289910 (secure 0 0 0) 0x7f5ea00117d0 con 0x7f5e9806b020 2023-10-16T10:00:39.504 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:39.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5e9806b020 msgr2=0x7f5e9806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5e9806b020 0x7f5e9806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009840 tx=0x7f5ea0008030 comp rx=0 tx=0).stop 2023-10-16T10:00:39.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac06db80 msgr2=0x7f5eac14c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac06db80 0x7f5eac14c6b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00e900 tx=0x7f5e9c00ec10 comp rx=0 tx=0).stop 2023-10-16T10:00:39.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 shutdown_connections 2023-10-16T10:00:39.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f5e9806b020 0x7f5e9806d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac15dfd0 0x7f5eac0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eac06db80 0x7f5eac14c6b0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 --2- 172.21.15.181:0/1520433992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac0af450 0x7f5eac0b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 >> 172.21.15.181:0/1520433992 conn(0x7f5eac0d04a0 msgr2=0x7f5eac15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:39.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 shutdown_connections 2023-10-16T10:00:39.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.500+0000 7f5eb2035700 1 -- 172.21.15.181:0/1520433992 wait complete. 2023-10-16T10:00:39.513 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:00:39.514 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:00:38.673478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":309100,"kb_used_data":11044,"kb_used_omap":12,"kb_used_meta":297907,"kb_avail":754665620,"statfs":{"total":773094113280,"available":772777594880,"internally_reserved":0,"allocated":11309056,"data_stored":6101856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":305057336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.671764"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195682+0000","last_change":"2023-10-16T10:00:35.028044+0000","last_active":"2023-10-16T10:00:36.195682+0000","last_peered":"2023-10-16T10:00:36.195682+0000","last_clean":"2023-10-16T10:00:36.195682+0000","last_became_active":"2023-10-16T10:00:35.027614+0000","last_became_peered":"2023-10-16T10:00:35.027614+0000","last_unstale":"2023-10-16T10:00:36.195682+0000","last_undegraded":"2023-10-16T10:00:36.195682+0000","last_fullsized":"2023-10-16T10:00:36.195682+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:17.207068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195061+0000","last_change":"2023-10-16T10:00:35.028916+0000","last_active":"2023-10-16T10:00:36.195061+0000","last_peered":"2023-10-16T10:00:36.195061+0000","last_clean":"2023-10-16T10:00:36.195061+0000","last_became_active":"2023-10-16T10:00:35.028609+0000","last_became_peered":"2023-10-16T10:00:35.028609+0000","last_unstale":"2023-10-16T10:00:36.195061+0000","last_undegraded":"2023-10-16T10:00:36.195061+0000","last_fullsized":"2023-10-16T10:00:36.195061+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:14.619379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028875+0000","last_change":"2023-10-16T10:00:35.032100+0000","last_active":"2023-10-16T10:00:36.028875+0000","last_peered":"2023-10-16T10:00:36.028875+0000","last_clean":"2023-10-16T10:00:36.028875+0000","last_became_active":"2023-10-16T10:00:35.031809+0000","last_became_peered":"2023-10-16T10:00:35.031809+0000","last_unstale":"2023-10-16T10:00:36.028875+0000","last_undegraded":"2023-10-16T10:00:36.028875+0000","last_fullsized":"2023-10-16T10:00:36.028875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:03:20.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195145+0000","last_change":"2023-10-16T10:00:35.035186+0000","last_active":"2023-10-16T10:00:36.195145+0000","last_peered":"2023-10-16T10:00:36.195145+0000","last_clean":"2023-10-16T10:00:36.195145+0000","last_became_active":"2023-10-16T10:00:35.034934+0000","last_became_peered":"2023-10-16T10:00:35.034934+0000","last_unstale":"2023-10-16T10:00:36.195145+0000","last_undegraded":"2023-10-16T10:00:36.195145+0000","last_fullsized":"2023-10-16T10:00:36.195145+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:15.018930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194721+0000","last_change":"2023-10-16T10:00:35.034491+0000","last_active":"2023-10-16T10:00:36.194721+0000","last_peered":"2023-10-16T10:00:36.194721+0000","last_clean":"2023-10-16T10:00:36.194721+0000","last_became_active":"2023-10-16T10:00:35.034223+0000","last_became_peered":"2023-10-16T10:00:35.034223+0000","last_unstale":"2023-10-16T10:00:36.194721+0000","last_undegraded":"2023-10-16T10:00:36.194721+0000","last_fullsized":"2023-10-16T10:00:36.194721+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:57:42.164790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029568+0000","last_change":"2023-10-16T10:00:35.026863+0000","last_active":"2023-10-16T10:00:36.029568+0000","last_peered":"2023-10-16T10:00:36.029568+0000","last_clean":"2023-10-16T10:00:36.029568+0000","last_became_active":"2023-10-16T10:00:35.026431+0000","last_became_peered":"2023-10-16T10:00:35.026431+0000","last_unstale":"2023-10-16T10:00:36.029568+0000","last_undegraded":"2023-10-16T10:00:36.029568+0000","last_fullsized":"2023-10-16T10:00:36.029568+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:59.182413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194363+0000","last_change":"2023-10-16T10:00:35.025625+0000","last_active":"2023-10-16T10:00:36.194363+0000","last_peered":"2023-10-16T10:00:36.194363+0000","last_clean":"2023-10-16T10:00:36.194363+0000","last_became_active":"2023-10-16T10:00:35.024299+0000","last_became_peered":"2023-10-16T10:00:35.024299+0000","last_unstale":"2023-10-16T10:00:36.194363+0000","last_undegraded":"2023-10-16T10:00:36.194363+0000","last_fullsized":"2023-10-16T10:00:36.194363+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:24.601041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029425+0000","last_change":"2023-10-16T10:00:35.040178+0000","last_active":"2023-10-16T10:00:36.029425+0000","last_peered":"2023-10-16T10:00:36.029425+0000","last_clean":"2023-10-16T10:00:36.029425+0000","last_became_active":"2023-10-16T10:00:35.039829+0000","last_became_peered":"2023-10-16T10:00:35.039829+0000","last_unstale":"2023-10-16T10:00:36.029425+0000","last_undegraded":"2023-10-16T10:00:36.029425+0000","last_fullsized":"2023-10-16T10:00:36.029425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:45.878055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029631+0000","last_change":"2023-10-16T10:00:35.031314+0000","last_active":"2023-10-16T10:00:36.029631+0000","last_peered":"2023-10-16T10:00:36.029631+0000","last_clean":"2023-10-16T10:00:36.029631+0000","last_became_active":"2023-10-16T10:00:35.031041+0000","last_became_peered":"2023-10-16T10:00:35.031041+0000","last_unstale":"2023-10-16T10:00:36.029631+0000","last_undegraded":"2023-10-16T10:00:36.029631+0000","last_fullsized":"2023-10-16T10:00:36.029631+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:07:47.748355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195415+0000","last_change":"2023-10-16T10:00:35.032610+0000","last_active":"2023-10-16T10:00:36.195415+0000","last_peered":"2023-10-16T10:00:36.195415+0000","last_clean":"2023-10-16T10:00:36.195415+0000","last_became_active":"2023-10-16T10:00:35.032175+0000","last_became_peered":"2023-10-16T10:00:35.032175+0000","last_unstale":"2023-10-16T10:00:36.195415+0000","last_undegraded":"2023-10-16T10:00:36.195415+0000","last_fullsized":"2023-10-16T10:00:36.195415+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:09:49.796392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195299+0000","last_change":"2023-10-16T10:00:35.032540+0000","last_active":"2023-10-16T10:00:36.195299+0000","last_peered":"2023-10-16T10:00:36.195299+0000","last_clean":"2023-10-16T10:00:36.195299+0000","last_became_active":"2023-10-16T10:00:35.032283+0000","last_became_peered":"2023-10-16T10:00:35.032283+0000","last_unstale":"2023-10-16T10:00:36.195299+0000","last_undegraded":"2023-10-16T10:00:36.195299+0000","last_fullsized":"2023-10-16T10:00:36.195299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:55.118437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195595+0000","last_change":"2023-10-16T10:00:35.034687+0000","last_active":"2023-10-16T10:00:36.195595+0000","last_peered":"2023-10-16T10:00:36.195595+0000","last_clean":"2023-10-16T10:00:36.195595+0000","last_became_active":"2023-10-16T10:00:35.033313+0000","last_became_peered":"2023-10-16T10:00:35.033313+0000","last_unstale":"2023-10-16T10:00:36.195595+0000","last_undegraded":"2023-10-16T10:00:36.195595+0000","last_fullsized":"2023-10-16T10:00:36.195595+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:29.215922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028552+0000","last_change":"2023-10-16T10:00:35.022029+0000","last_active":"2023-10-16T10:00:36.028552+0000","last_peered":"2023-10-16T10:00:36.028552+0000","last_clean":"2023-10-16T10:00:36.028552+0000","last_became_active":"2023-10-16T10:00:35.021727+0000","last_became_peered":"2023-10-16T10:00:35.021727+0000","last_unstale":"2023-10-16T10:00:36.028552+0000","last_undegraded":"2023-10-16T10:00:36.028552+0000","last_fullsized":"2023-10-16T10:00:36.028552+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:41.783948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194701+0000","last_change":"2023-10-16T10:00:35.037630+0000","last_active":"2023-10-16T10:00:36.194701+0000","last_peered":"2023-10-16T10:00:36.194701+0000","last_clean":"2023-10-16T10:00:36.194701+0000","last_became_active":"2023-10-16T10:00:35.036988+0000","last_became_peered":"2023-10-16T10:00:35.036988+0000","last_unstale":"2023-10-16T10:00:36.194701+0000","last_undegraded":"2023-10-16T10:00:36.194701+0000","last_fullsized":"2023-10-16T10:00:36.194701+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:46.263256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027246+0000","last_change":"2023-10-16T10:00:35.024763+0000","last_active":"2023-10-16T10:00:36.027246+0000","last_peered":"2023-10-16T10:00:36.027246+0000","last_clean":"2023-10-16T10:00:36.027246+0000","last_became_active":"2023-10-16T10:00:35.024465+0000","last_became_peered":"2023-10-16T10:00:35.024465+0000","last_unstale":"2023-10-16T10:00:36.027246+0000","last_undegraded":"2023-10-16T10:00:36.027246+0000","last_fullsized":"2023-10-16T10:00:36.027246+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:22.213292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194383+0000","last_change":"2023-10-16T10:00:35.030249+0000","last_active":"2023-10-16T10:00:36.194383+0000","last_peered":"2023-10-16T10:00:36.194383+0000","last_clean":"2023-10-16T10:00:36.194383+0000","last_became_active":"2023-10-16T10:00:35.029727+0000","last_became_peered":"2023-10-16T10:00:35.029727+0000","last_unstale":"2023-10-16T10:00:36.194383+0000","last_undegraded":"2023-10-16T10:00:36.194383+0000","last_fullsized":"2023-10-16T10:00:36.194383+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:42.756758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194911+0000","last_change":"2023-10-16T10:00:35.030845+0000","last_active":"2023-10-16T10:00:36.194911+0000","last_peered":"2023-10-16T10:00:36.194911+0000","last_clean":"2023-10-16T10:00:36.194911+0000","last_became_active":"2023-10-16T10:00:35.030555+0000","last_became_peered":"2023-10-16T10:00:35.030555+0000","last_unstale":"2023-10-16T10:00:36.194911+0000","last_undegraded":"2023-10-16T10:00:36.194911+0000","last_fullsized":"2023-10-16T10:00:36.194911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:53.268872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194784+0000","last_change":"2023-10-16T10:00:35.027864+0000","last_active":"2023-10-16T10:00:36.194784+0000","last_peered":"2023-10-16T10:00:36.194784+0000","last_clean":"2023-10-16T10:00:36.194784+0000","last_became_active":"2023-10-16T10:00:35.027602+0000","last_became_peered":"2023-10-16T10:00:35.027602+0000","last_unstale":"2023-10-16T10:00:36.194784+0000","last_undegraded":"2023-10-16T10:00:36.194784+0000","last_fullsized":"2023-10-16T10:00:36.194784+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:59.230033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195449+0000","last_change":"2023-10-16T10:00:35.031077+0000","last_active":"2023-10-16T10:00:36.195449+0000","last_peered":"2023-10-16T10:00:36.195449+0000","last_clean":"2023-10-16T10:00:36.195449+0000","last_became_active":"2023-10-16T10:00:35.030780+0000","last_became_peered":"2023-10-16T10:00:35.030780+0000","last_unstale":"2023-10-16T10:00:36.195449+0000","last_undegraded":"2023-10-16T10:00:36.195449+0000","last_fullsized":"2023-10-16T10:00:36.195449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:17.795751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195272+0000","last_change":"2023-10-16T10:00:35.031270+0000","last_active":"2023-10-16T10:00:36.195272+0000","last_peered":"2023-10-16T10:00:36.195272+0000","last_clean":"2023-10-16T10:00:36.195272+0000","last_became_active":"2023-10-16T10:00:35.030975+0000","last_became_peered":"2023-10-16T10:00:35.030975+0000","last_unstale":"2023-10-16T10:00:36.195272+0000","last_undegraded":"2023-10-16T10:00:36.195272+0000","last_fullsized":"2023-10-16T10:00:36.195272+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.898298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194650+0000","last_change":"2023-10-16T10:00:35.034123+0000","last_active":"2023-10-16T10:00:36.194650+0000","last_peered":"2023-10-16T10:00:36.194650+0000","last_clean":"2023-10-16T10:00:36.194650+0000","last_became_active":"2023-10-16T10:00:35.033850+0000","last_became_peered":"2023-10-16T10:00:35.033850+0000","last_unstale":"2023-10-16T10:00:36.194650+0000","last_undegraded":"2023-10-16T10:00:36.194650+0000","last_fullsized":"2023-10-16T10:00:36.194650+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:34.255691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194664+0000","last_change":"2023-10-16T10:00:35.028948+0000","last_active":"2023-10-16T10:00:36.194664+0000","last_peered":"2023-10-16T10:00:36.194664+0000","last_clean":"2023-10-16T10:00:36.194664+0000","last_became_active":"2023-10-16T10:00:35.028705+0000","last_became_peered":"2023-10-16T10:00:35.028705+0000","last_unstale":"2023-10-16T10:00:36.194664+0000","last_undegraded":"2023-10-16T10:00:36.194664+0000","last_fullsized":"2023-10-16T10:00:36.194664+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:23.077701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195032+0000","last_change":"2023-10-16T10:00:35.033347+0000","last_active":"2023-10-16T10:00:36.195032+0000","last_peered":"2023-10-16T10:00:36.195032+0000","last_clean":"2023-10-16T10:00:36.195032+0000","last_became_active":"2023-10-16T10:00:35.032806+0000","last_became_peered":"2023-10-16T10:00:35.032806+0000","last_unstale":"2023-10-16T10:00:36.195032+0000","last_undegraded":"2023-10-16T10:00:36.195032+0000","last_fullsized":"2023-10-16T10:00:36.195032+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:15.018930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194656+0000","last_change":"2023-10-16T10:00:35.031979+0000","last_active":"2023-10-16T10:00:36.194656+0000","last_peered":"2023-10-16T10:00:36.194656+0000","last_clean":"2023-10-16T10:00:36.194656+0000","last_became_active":"2023-10-16T10:00:35.031275+0000","last_became_peered":"2023-10-16T10:00:35.031275+0000","last_unstale":"2023-10-16T10:00:36.194656+0000","last_undegraded":"2023-10-16T10:00:36.194656+0000","last_fullsized":"2023-10-16T10:00:36.194656+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:12.030952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194383+0000","last_change":"2023-10-16T10:00:35.029582+0000","last_active":"2023-10-16T10:00:36.194383+0000","last_peered":"2023-10-16T10:00:36.194383+0000","last_clean":"2023-10-16T10:00:36.194383+0000","last_became_active":"2023-10-16T10:00:35.029249+0000","last_became_peered":"2023-10-16T10:00:35.029249+0000","last_unstale":"2023-10-16T10:00:36.194383+0000","last_undegraded":"2023-10-16T10:00:36.194383+0000","last_fullsized":"2023-10-16T10:00:36.194383+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:17.207068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028586+0000","last_change":"2023-10-16T10:00:35.034767+0000","last_active":"2023-10-16T10:00:36.028586+0000","last_peered":"2023-10-16T10:00:36.028586+0000","last_clean":"2023-10-16T10:00:36.028586+0000","last_became_active":"2023-10-16T10:00:35.034459+0000","last_became_peered":"2023-10-16T10:00:35.034459+0000","last_unstale":"2023-10-16T10:00:36.028586+0000","last_undegraded":"2023-10-16T10:00:36.028586+0000","last_fullsized":"2023-10-16T10:00:36.028586+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:51.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195142+0000","last_change":"2023-10-16T10:00:35.032500+0000","last_active":"2023-10-16T10:00:36.195142+0000","last_peered":"2023-10-16T10:00:36.195142+0000","last_clean":"2023-10-16T10:00:36.195142+0000","last_became_active":"2023-10-16T10:00:35.032040+0000","last_became_peered":"2023-10-16T10:00:35.032040+0000","last_unstale":"2023-10-16T10:00:36.195142+0000","last_undegraded":"2023-10-16T10:00:36.195142+0000","last_fullsized":"2023-10-16T10:00:36.195142+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:21:26.664514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195374+0000","last_change":"2023-10-16T10:00:35.029640+0000","last_active":"2023-10-16T10:00:36.195374+0000","last_peered":"2023-10-16T10:00:36.195374+0000","last_clean":"2023-10-16T10:00:36.195374+0000","last_became_active":"2023-10-16T10:00:35.029136+0000","last_became_peered":"2023-10-16T10:00:35.029136+0000","last_unstale":"2023-10-16T10:00:36.195374+0000","last_undegraded":"2023-10-16T10:00:36.195374+0000","last_fullsized":"2023-10-16T10:00:36.195374+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:07.768526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194538+0000","last_change":"2023-10-16T10:00:35.033117+0000","last_active":"2023-10-16T10:00:36.194538+0000","last_peered":"2023-10-16T10:00:36.194538+0000","last_clean":"2023-10-16T10:00:36.194538+0000","last_became_active":"2023-10-16T10:00:35.032063+0000","last_became_peered":"2023-10-16T10:00:35.032063+0000","last_unstale":"2023-10-16T10:00:36.194538+0000","last_undegraded":"2023-10-16T10:00:36.194538+0000","last_fullsized":"2023-10-16T10:00:36.194538+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:49.020659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194794+0000","last_change":"2023-10-16T10:00:35.035940+0000","last_active":"2023-10-16T10:00:36.194794+0000","last_peered":"2023-10-16T10:00:36.194794+0000","last_clean":"2023-10-16T10:00:36.194794+0000","last_became_active":"2023-10-16T10:00:35.031452+0000","last_became_peered":"2023-10-16T10:00:35.031452+0000","last_unstale":"2023-10-16T10:00:36.194794+0000","last_undegraded":"2023-10-16T10:00:36.194794+0000","last_fullsized":"2023-10-16T10:00:36.194794+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:02:14.079709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194453+0000","last_change":"2023-10-16T10:00:35.032466+0000","last_active":"2023-10-16T10:00:36.194453+0000","last_peered":"2023-10-16T10:00:36.194453+0000","last_clean":"2023-10-16T10:00:36.194453+0000","last_became_active":"2023-10-16T10:00:35.032164+0000","last_became_peered":"2023-10-16T10:00:35.032164+0000","last_unstale":"2023-10-16T10:00:36.194453+0000","last_undegraded":"2023-10-16T10:00:36.194453+0000","last_fullsized":"2023-10-16T10:00:36.194453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:33.587345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194623+0000","last_change":"2023-10-16T10:00:35.039646+0000","last_active":"2023-10-16T10:00:36.194623+0000","last_peered":"2023-10-16T10:00:36.194623+0000","last_clean":"2023-10-16T10:00:36.194623+0000","last_became_active":"2023-10-16T10:00:35.039350+0000","last_became_peered":"2023-10-16T10:00:35.039350+0000","last_unstale":"2023-10-16T10:00:36.194623+0000","last_undegraded":"2023-10-16T10:00:36.194623+0000","last_fullsized":"2023-10-16T10:00:36.194623+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:17.795751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194950+0000","last_change":"2023-10-16T10:00:35.030516+0000","last_active":"2023-10-16T10:00:36.194950+0000","last_peered":"2023-10-16T10:00:36.194950+0000","last_clean":"2023-10-16T10:00:36.194950+0000","last_became_active":"2023-10-16T10:00:35.029991+0000","last_became_peered":"2023-10-16T10:00:35.029991+0000","last_unstale":"2023-10-16T10:00:36.194950+0000","last_undegraded":"2023-10-16T10:00:36.194950+0000","last_fullsized":"2023-10-16T10:00:36.194950+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:47.125673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195465+0000","last_change":"2023-10-16T10:00:34.033033+0000","last_active":"2023-10-16T10:00:36.195465+0000","last_peered":"2023-10-16T10:00:36.195465+0000","last_clean":"2023-10-16T10:00:36.195465+0000","last_became_active":"2023-10-16T10:00:34.032789+0000","last_became_peered":"2023-10-16T10:00:34.032789+0000","last_unstale":"2023-10-16T10:00:36.195465+0000","last_undegraded":"2023-10-16T10:00:36.195465+0000","last_fullsized":"2023-10-16T10:00:36.195465+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:45.257976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195601+0000","last_change":"2023-10-16T10:00:35.031881+0000","last_active":"2023-10-16T10:00:36.195601+0000","last_peered":"2023-10-16T10:00:36.195601+0000","last_clean":"2023-10-16T10:00:36.195601+0000","last_became_active":"2023-10-16T10:00:35.031573+0000","last_became_peered":"2023-10-16T10:00:35.031573+0000","last_unstale":"2023-10-16T10:00:36.195601+0000","last_undegraded":"2023-10-16T10:00:36.195601+0000","last_fullsized":"2023-10-16T10:00:36.195601+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:27.979949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195058+0000","last_change":"2023-10-16T10:00:34.019115+0000","last_active":"2023-10-16T10:00:36.195058+0000","last_peered":"2023-10-16T10:00:36.195058+0000","last_clean":"2023-10-16T10:00:36.195058+0000","last_became_active":"2023-10-16T10:00:34.018749+0000","last_became_peered":"2023-10-16T10:00:34.018749+0000","last_unstale":"2023-10-16T10:00:36.195058+0000","last_undegraded":"2023-10-16T10:00:36.195058+0000","last_fullsized":"2023-10-16T10:00:36.195058+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:08:31.252808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195466+0000","last_change":"2023-10-16T10:00:35.039568+0000","last_active":"2023-10-16T10:00:36.195466+0000","last_peered":"2023-10-16T10:00:36.195466+0000","last_clean":"2023-10-16T10:00:36.195466+0000","last_became_active":"2023-10-16T10:00:35.039260+0000","last_became_peered":"2023-10-16T10:00:35.039260+0000","last_unstale":"2023-10-16T10:00:36.195466+0000","last_undegraded":"2023-10-16T10:00:36.195466+0000","last_fullsized":"2023-10-16T10:00:36.195466+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:44.025934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195538+0000","last_change":"2023-10-16T10:00:34.020596+0000","last_active":"2023-10-16T10:00:36.195538+0000","last_peered":"2023-10-16T10:00:36.195538+0000","last_clean":"2023-10-16T10:00:36.195538+0000","last_became_active":"2023-10-16T10:00:34.020327+0000","last_became_peered":"2023-10-16T10:00:34.020327+0000","last_unstale":"2023-10-16T10:00:36.195538+0000","last_undegraded":"2023-10-16T10:00:36.195538+0000","last_fullsized":"2023-10-16T10:00:36.195538+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:11.025672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028758+0000","last_change":"2023-10-16T10:00:35.021938+0000","last_active":"2023-10-16T10:00:36.028758+0000","last_peered":"2023-10-16T10:00:36.028758+0000","last_clean":"2023-10-16T10:00:36.028758+0000","last_became_active":"2023-10-16T10:00:35.021640+0000","last_became_peered":"2023-10-16T10:00:35.021640+0000","last_unstale":"2023-10-16T10:00:36.028758+0000","last_undegraded":"2023-10-16T10:00:36.028758+0000","last_fullsized":"2023-10-16T10:00:36.028758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:31:44.669129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195397+0000","last_change":"2023-10-16T10:00:34.024584+0000","last_active":"2023-10-16T10:00:36.195397+0000","last_peered":"2023-10-16T10:00:36.195397+0000","last_clean":"2023-10-16T10:00:36.195397+0000","last_became_active":"2023-10-16T10:00:34.024322+0000","last_became_peered":"2023-10-16T10:00:34.024322+0000","last_unstale":"2023-10-16T10:00:36.195397+0000","last_undegraded":"2023-10-16T10:00:36.195397+0000","last_fullsized":"2023-10-16T10:00:36.195397+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:45.257976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194979+0000","last_change":"2023-10-16T10:00:35.036184+0000","last_active":"2023-10-16T10:00:36.194979+0000","last_peered":"2023-10-16T10:00:36.194979+0000","last_clean":"2023-10-16T10:00:36.194979+0000","last_became_active":"2023-10-16T10:00:35.031821+0000","last_became_peered":"2023-10-16T10:00:35.031821+0000","last_unstale":"2023-10-16T10:00:36.194979+0000","last_undegraded":"2023-10-16T10:00:36.194979+0000","last_fullsized":"2023-10-16T10:00:36.194979+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:21.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194831+0000","last_change":"2023-10-16T10:00:34.018191+0000","last_active":"2023-10-16T10:00:36.194831+0000","last_peered":"2023-10-16T10:00:36.194831+0000","last_clean":"2023-10-16T10:00:36.194831+0000","last_became_active":"2023-10-16T10:00:34.017859+0000","last_became_peered":"2023-10-16T10:00:34.017859+0000","last_unstale":"2023-10-16T10:00:36.194831+0000","last_undegraded":"2023-10-16T10:00:36.194831+0000","last_fullsized":"2023-10-16T10:00:36.194831+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:16:11.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194844+0000","last_change":"2023-10-16T10:00:35.031066+0000","last_active":"2023-10-16T10:00:36.194844+0000","last_peered":"2023-10-16T10:00:36.194844+0000","last_clean":"2023-10-16T10:00:36.194844+0000","last_became_active":"2023-10-16T10:00:35.029442+0000","last_became_peered":"2023-10-16T10:00:35.029442+0000","last_unstale":"2023-10-16T10:00:36.194844+0000","last_undegraded":"2023-10-16T10:00:36.194844+0000","last_fullsized":"2023-10-16T10:00:36.194844+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:41.123342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194443+0000","last_change":"2023-10-16T10:00:34.019657+0000","last_active":"2023-10-16T10:00:36.194443+0000","last_peered":"2023-10-16T10:00:36.194443+0000","last_clean":"2023-10-16T10:00:36.194443+0000","last_became_active":"2023-10-16T10:00:34.019292+0000","last_became_peered":"2023-10-16T10:00:34.019292+0000","last_unstale":"2023-10-16T10:00:36.194443+0000","last_undegraded":"2023-10-16T10:00:36.194443+0000","last_fullsized":"2023-10-16T10:00:36.194443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:17:35.927298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028859+0000","last_change":"2023-10-16T10:00:35.026951+0000","last_active":"2023-10-16T10:00:36.028859+0000","last_peered":"2023-10-16T10:00:36.028859+0000","last_clean":"2023-10-16T10:00:36.028859+0000","last_became_active":"2023-10-16T10:00:35.026641+0000","last_became_peered":"2023-10-16T10:00:35.026641+0000","last_unstale":"2023-10-16T10:00:36.028859+0000","last_undegraded":"2023-10-16T10:00:36.028859+0000","last_fullsized":"2023-10-16T10:00:36.028859+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:42.215135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195315+0000","last_change":"2023-10-16T10:00:34.024394+0000","last_active":"2023-10-16T10:00:36.195315+0000","last_peered":"2023-10-16T10:00:36.195315+0000","last_clean":"2023-10-16T10:00:36.195315+0000","last_became_active":"2023-10-16T10:00:34.024140+0000","last_became_peered":"2023-10-16T10:00:34.024140+0000","last_unstale":"2023-10-16T10:00:36.195315+0000","last_undegraded":"2023-10-16T10:00:36.195315+0000","last_fullsized":"2023-10-16T10:00:36.195315+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:54.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027299+0000","last_change":"2023-10-16T10:00:35.022448+0000","last_active":"2023-10-16T10:00:36.027299+0000","last_peered":"2023-10-16T10:00:36.027299+0000","last_clean":"2023-10-16T10:00:36.027299+0000","last_became_active":"2023-10-16T10:00:35.022143+0000","last_became_peered":"2023-10-16T10:00:35.022143+0000","last_unstale":"2023-10-16T10:00:36.027299+0000","last_undegraded":"2023-10-16T10:00:36.027299+0000","last_fullsized":"2023-10-16T10:00:36.027299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:44.962479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195584+0000","last_change":"2023-10-16T10:00:34.021987+0000","last_active":"2023-10-16T10:00:36.195584+0000","last_peered":"2023-10-16T10:00:36.195584+0000","last_clean":"2023-10-16T10:00:36.195584+0000","last_became_active":"2023-10-16T10:00:34.021718+0000","last_became_peered":"2023-10-16T10:00:34.021718+0000","last_unstale":"2023-10-16T10:00:36.195584+0000","last_undegraded":"2023-10-16T10:00:36.195584+0000","last_fullsized":"2023-10-16T10:00:36.195584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:34.097110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027558+0000","last_change":"2023-10-16T10:00:35.034809+0000","last_active":"2023-10-16T10:00:36.027558+0000","last_peered":"2023-10-16T10:00:36.027558+0000","last_clean":"2023-10-16T10:00:36.027558+0000","last_became_active":"2023-10-16T10:00:35.034391+0000","last_became_peered":"2023-10-16T10:00:35.034391+0000","last_unstale":"2023-10-16T10:00:36.027558+0000","last_undegraded":"2023-10-16T10:00:36.027558+0000","last_fullsized":"2023-10-16T10:00:36.027558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:42.659658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195429+0000","last_change":"2023-10-16T10:00:34.033022+0000","last_active":"2023-10-16T10:00:36.195429+0000","last_peered":"2023-10-16T10:00:36.195429+0000","last_clean":"2023-10-16T10:00:36.195429+0000","last_became_active":"2023-10-16T10:00:34.032749+0000","last_became_peered":"2023-10-16T10:00:34.032749+0000","last_unstale":"2023-10-16T10:00:36.195429+0000","last_undegraded":"2023-10-16T10:00:36.195429+0000","last_fullsized":"2023-10-16T10:00:36.195429+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:01:33.250411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029543+0000","last_change":"2023-10-16T10:00:35.027652+0000","last_active":"2023-10-16T10:00:36.029543+0000","last_peered":"2023-10-16T10:00:36.029543+0000","last_clean":"2023-10-16T10:00:36.029543+0000","last_became_active":"2023-10-16T10:00:35.027407+0000","last_became_peered":"2023-10-16T10:00:35.027407+0000","last_unstale":"2023-10-16T10:00:36.029543+0000","last_undegraded":"2023-10-16T10:00:36.029543+0000","last_fullsized":"2023-10-16T10:00:36.029543+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:14.181745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195528+0000","last_change":"2023-10-16T10:00:34.026897+0000","last_active":"2023-10-16T10:00:36.195528+0000","last_peered":"2023-10-16T10:00:36.195528+0000","last_clean":"2023-10-16T10:00:36.195528+0000","last_became_active":"2023-10-16T10:00:34.026673+0000","last_became_peered":"2023-10-16T10:00:34.026673+0000","last_unstale":"2023-10-16T10:00:36.195528+0000","last_undegraded":"2023-10-16T10:00:36.195528+0000","last_fullsized":"2023-10-16T10:00:36.195528+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:23.595761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195406+0000","last_change":"2023-10-16T10:00:35.031886+0000","last_active":"2023-10-16T10:00:36.195406+0000","last_peered":"2023-10-16T10:00:36.195406+0000","last_clean":"2023-10-16T10:00:36.195406+0000","last_became_active":"2023-10-16T10:00:35.031623+0000","last_became_peered":"2023-10-16T10:00:35.031623+0000","last_unstale":"2023-10-16T10:00:36.195406+0000","last_undegraded":"2023-10-16T10:00:36.195406+0000","last_fullsized":"2023-10-16T10:00:36.195406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:41.574638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195121+0000","last_change":"2023-10-16T10:00:34.030892+0000","last_active":"2023-10-16T10:00:36.195121+0000","last_peered":"2023-10-16T10:00:36.195121+0000","last_clean":"2023-10-16T10:00:36.195121+0000","last_became_active":"2023-10-16T10:00:34.030578+0000","last_became_peered":"2023-10-16T10:00:34.030578+0000","last_unstale":"2023-10-16T10:00:36.195121+0000","last_undegraded":"2023-10-16T10:00:36.195121+0000","last_fullsized":"2023-10-16T10:00:36.195121+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.273373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194810+0000","last_change":"2023-10-16T10:00:35.032269+0000","last_active":"2023-10-16T10:00:36.194810+0000","last_peered":"2023-10-16T10:00:36.194810+0000","last_clean":"2023-10-16T10:00:36.194810+0000","last_became_active":"2023-10-16T10:00:35.031666+0000","last_became_peered":"2023-10-16T10:00:35.031666+0000","last_unstale":"2023-10-16T10:00:36.194810+0000","last_undegraded":"2023-10-16T10:00:36.194810+0000","last_fullsized":"2023-10-16T10:00:36.194810+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:14:05.521100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029723+0000","last_change":"2023-10-16T10:00:34.023919+0000","last_active":"2023-10-16T10:00:36.029723+0000","last_peered":"2023-10-16T10:00:36.029723+0000","last_clean":"2023-10-16T10:00:36.029723+0000","last_became_active":"2023-10-16T10:00:34.023638+0000","last_became_peered":"2023-10-16T10:00:34.023638+0000","last_unstale":"2023-10-16T10:00:36.029723+0000","last_undegraded":"2023-10-16T10:00:36.029723+0000","last_fullsized":"2023-10-16T10:00:36.029723+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:39:10.822350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195506+0000","last_change":"2023-10-16T10:00:35.022990+0000","last_active":"2023-10-16T10:00:36.195506+0000","last_peered":"2023-10-16T10:00:36.195506+0000","last_clean":"2023-10-16T10:00:36.195506+0000","last_became_active":"2023-10-16T10:00:35.022739+0000","last_became_peered":"2023-10-16T10:00:35.022739+0000","last_unstale":"2023-10-16T10:00:36.195506+0000","last_undegraded":"2023-10-16T10:00:36.195506+0000","last_fullsized":"2023-10-16T10:00:36.195506+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:15.018930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194633+0000","last_change":"2023-10-16T10:00:34.023004+0000","last_active":"2023-10-16T10:00:36.194633+0000","last_peered":"2023-10-16T10:00:36.194633+0000","last_clean":"2023-10-16T10:00:36.194633+0000","last_became_active":"2023-10-16T10:00:34.022776+0000","last_became_peered":"2023-10-16T10:00:34.022776+0000","last_unstale":"2023-10-16T10:00:36.194633+0000","last_undegraded":"2023-10-16T10:00:36.194633+0000","last_fullsized":"2023-10-16T10:00:36.194633+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:40.569358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028733+0000","last_change":"2023-10-16T10:00:35.033813+0000","last_active":"2023-10-16T10:00:36.028733+0000","last_peered":"2023-10-16T10:00:36.028733+0000","last_clean":"2023-10-16T10:00:36.028733+0000","last_became_active":"2023-10-16T10:00:35.033322+0000","last_became_peered":"2023-10-16T10:00:35.033322+0000","last_unstale":"2023-10-16T10:00:36.028733+0000","last_undegraded":"2023-10-16T10:00:36.028733+0000","last_fullsized":"2023-10-16T10:00:36.028733+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:06.831142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027374+0000","last_change":"2023-10-16T10:00:34.028095+0000","last_active":"2023-10-16T10:00:36.027374+0000","last_peered":"2023-10-16T10:00:36.027374+0000","last_clean":"2023-10-16T10:00:36.027374+0000","last_became_active":"2023-10-16T10:00:34.027833+0000","last_became_peered":"2023-10-16T10:00:34.027833+0000","last_unstale":"2023-10-16T10:00:36.027374+0000","last_undegraded":"2023-10-16T10:00:36.027374+0000","last_fullsized":"2023-10-16T10:00:36.027374+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:36.102659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194360+0000","last_change":"2023-10-16T10:00:35.037036+0000","last_active":"2023-10-16T10:00:36.194360+0000","last_peered":"2023-10-16T10:00:36.194360+0000","last_clean":"2023-10-16T10:00:36.194360+0000","last_became_active":"2023-10-16T10:00:35.036760+0000","last_became_peered":"2023-10-16T10:00:35.036760+0000","last_unstale":"2023-10-16T10:00:36.194360+0000","last_undegraded":"2023-10-16T10:00:36.194360+0000","last_fullsized":"2023-10-16T10:00:36.194360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:29.215922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029375+0000","last_change":"2023-10-16T10:00:34.024085+0000","last_active":"2023-10-16T10:00:36.029375+0000","last_peered":"2023-10-16T10:00:36.029375+0000","last_clean":"2023-10-16T10:00:36.029375+0000","last_became_active":"2023-10-16T10:00:34.023717+0000","last_became_peered":"2023-10-16T10:00:34.023717+0000","last_unstale":"2023-10-16T10:00:36.029375+0000","last_undegraded":"2023-10-16T10:00:36.029375+0000","last_fullsized":"2023-10-16T10:00:36.029375+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:41.927893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194378+0000","last_change":"2023-10-16T10:00:35.030139+0000","last_active":"2023-10-16T10:00:36.194378+0000","last_peered":"2023-10-16T10:00:36.194378+0000","last_clean":"2023-10-16T10:00:36.194378+0000","last_became_active":"2023-10-16T10:00:35.029815+0000","last_became_peered":"2023-10-16T10:00:35.029815+0000","last_unstale":"2023-10-16T10:00:36.194378+0000","last_undegraded":"2023-10-16T10:00:36.194378+0000","last_fullsized":"2023-10-16T10:00:36.194378+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:27.979949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194190+0000","last_change":"2023-10-16T10:00:34.030630+0000","last_active":"2023-10-16T10:00:36.194190+0000","last_peered":"2023-10-16T10:00:36.194190+0000","last_clean":"2023-10-16T10:00:36.194190+0000","last_became_active":"2023-10-16T10:00:34.030327+0000","last_became_peered":"2023-10-16T10:00:34.030327+0000","last_unstale":"2023-10-16T10:00:36.194190+0000","last_undegraded":"2023-10-16T10:00:36.194190+0000","last_fullsized":"2023-10-16T10:00:36.194190+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:17.802878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028838+0000","last_change":"2023-10-16T10:00:35.026016+0000","last_active":"2023-10-16T10:00:36.028838+0000","last_peered":"2023-10-16T10:00:36.028838+0000","last_clean":"2023-10-16T10:00:36.028838+0000","last_became_active":"2023-10-16T10:00:35.025674+0000","last_became_peered":"2023-10-16T10:00:35.025674+0000","last_unstale":"2023-10-16T10:00:36.028838+0000","last_undegraded":"2023-10-16T10:00:36.028838+0000","last_fullsized":"2023-10-16T10:00:36.028838+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:41.332427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195306+0000","last_change":"2023-10-16T10:00:34.030613+0000","last_active":"2023-10-16T10:00:36.195306+0000","last_peered":"2023-10-16T10:00:36.195306+0000","last_clean":"2023-10-16T10:00:36.195306+0000","last_became_active":"2023-10-16T10:00:34.030308+0000","last_became_peered":"2023-10-16T10:00:34.030308+0000","last_unstale":"2023-10-16T10:00:36.195306+0000","last_undegraded":"2023-10-16T10:00:36.195306+0000","last_fullsized":"2023-10-16T10:00:36.195306+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:58.224753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195587+0000","last_change":"2023-10-16T10:00:35.027988+0000","last_active":"2023-10-16T10:00:36.195587+0000","last_peered":"2023-10-16T10:00:36.195587+0000","last_clean":"2023-10-16T10:00:36.195587+0000","last_became_active":"2023-10-16T10:00:35.027594+0000","last_became_peered":"2023-10-16T10:00:35.027594+0000","last_unstale":"2023-10-16T10:00:36.195587+0000","last_undegraded":"2023-10-16T10:00:36.195587+0000","last_fullsized":"2023-10-16T10:00:36.195587+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:35.102390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195243+0000","last_change":"2023-10-16T10:00:34.022927+0000","last_active":"2023-10-16T10:00:36.195243+0000","last_peered":"2023-10-16T10:00:36.195243+0000","last_clean":"2023-10-16T10:00:36.195243+0000","last_became_active":"2023-10-16T10:00:34.022678+0000","last_became_peered":"2023-10-16T10:00:34.022678+0000","last_unstale":"2023-10-16T10:00:36.195243+0000","last_undegraded":"2023-10-16T10:00:36.195243+0000","last_fullsized":"2023-10-16T10:00:36.195243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:09.291315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195009+0000","last_change":"2023-10-16T10:00:35.039591+0000","last_active":"2023-10-16T10:00:36.195009+0000","last_peered":"2023-10-16T10:00:36.195009+0000","last_clean":"2023-10-16T10:00:36.195009+0000","last_became_active":"2023-10-16T10:00:35.039279+0000","last_became_peered":"2023-10-16T10:00:35.039279+0000","last_unstale":"2023-10-16T10:00:36.195009+0000","last_undegraded":"2023-10-16T10:00:36.195009+0000","last_fullsized":"2023-10-16T10:00:36.195009+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:58:57.667724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195501+0000","last_change":"2023-10-16T10:00:34.024570+0000","last_active":"2023-10-16T10:00:36.195501+0000","last_peered":"2023-10-16T10:00:36.195501+0000","last_clean":"2023-10-16T10:00:36.195501+0000","last_became_active":"2023-10-16T10:00:34.023970+0000","last_became_peered":"2023-10-16T10:00:34.023970+0000","last_unstale":"2023-10-16T10:00:36.195501+0000","last_undegraded":"2023-10-16T10:00:36.195501+0000","last_fullsized":"2023-10-16T10:00:36.195501+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:33:58.224753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194835+0000","last_change":"2023-10-16T10:00:35.037806+0000","last_active":"2023-10-16T10:00:36.194835+0000","last_peered":"2023-10-16T10:00:36.194835+0000","last_clean":"2023-10-16T10:00:36.194835+0000","last_became_active":"2023-10-16T10:00:35.037366+0000","last_became_peered":"2023-10-16T10:00:35.037366+0000","last_unstale":"2023-10-16T10:00:36.194835+0000","last_undegraded":"2023-10-16T10:00:36.194835+0000","last_fullsized":"2023-10-16T10:00:36.194835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:17.795751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027547+0000","last_change":"2023-10-16T10:00:34.021412+0000","last_active":"2023-10-16T10:00:36.027547+0000","last_peered":"2023-10-16T10:00:36.027547+0000","last_clean":"2023-10-16T10:00:36.027547+0000","last_became_active":"2023-10-16T10:00:34.021132+0000","last_became_peered":"2023-10-16T10:00:34.021132+0000","last_unstale":"2023-10-16T10:00:36.027547+0000","last_undegraded":"2023-10-16T10:00:36.027547+0000","last_fullsized":"2023-10-16T10:00:36.027547+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:43.904441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029517+0000","last_change":"2023-10-16T10:00:35.027880+0000","last_active":"2023-10-16T10:00:36.029517+0000","last_peered":"2023-10-16T10:00:36.029517+0000","last_clean":"2023-10-16T10:00:36.029517+0000","last_became_active":"2023-10-16T10:00:35.027199+0000","last_became_peered":"2023-10-16T10:00:35.027199+0000","last_unstale":"2023-10-16T10:00:36.029517+0000","last_undegraded":"2023-10-16T10:00:36.029517+0000","last_fullsized":"2023-10-16T10:00:36.029517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:24.915071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194997+0000","last_change":"2023-10-16T10:00:34.025139+0000","last_active":"2023-10-16T10:00:36.194997+0000","last_peered":"2023-10-16T10:00:36.194997+0000","last_clean":"2023-10-16T10:00:36.194997+0000","last_became_active":"2023-10-16T10:00:34.024886+0000","last_became_peered":"2023-10-16T10:00:34.024886+0000","last_unstale":"2023-10-16T10:00:36.194997+0000","last_undegraded":"2023-10-16T10:00:36.194997+0000","last_fullsized":"2023-10-16T10:00:36.194997+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:44:43.737230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194797+0000","last_change":"2023-10-16T10:00:35.026459+0000","last_active":"2023-10-16T10:00:36.194797+0000","last_peered":"2023-10-16T10:00:36.194797+0000","last_clean":"2023-10-16T10:00:36.194797+0000","last_became_active":"2023-10-16T10:00:35.026148+0000","last_became_peered":"2023-10-16T10:00:35.026148+0000","last_unstale":"2023-10-16T10:00:36.194797+0000","last_undegraded":"2023-10-16T10:00:36.194797+0000","last_fullsized":"2023-10-16T10:00:36.194797+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:41.574638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194419+0000","last_change":"2023-10-16T10:00:34.030626+0000","last_active":"2023-10-16T10:00:36.194419+0000","last_peered":"2023-10-16T10:00:36.194419+0000","last_clean":"2023-10-16T10:00:36.194419+0000","last_became_active":"2023-10-16T10:00:34.030283+0000","last_became_peered":"2023-10-16T10:00:34.030283+0000","last_unstale":"2023-10-16T10:00:36.194419+0000","last_undegraded":"2023-10-16T10:00:36.194419+0000","last_fullsized":"2023-10-16T10:00:36.194419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:09.291315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029368+0000","last_change":"2023-10-16T10:00:35.033496+0000","last_active":"2023-10-16T10:00:36.029368+0000","last_peered":"2023-10-16T10:00:36.029368+0000","last_clean":"2023-10-16T10:00:36.029368+0000","last_became_active":"2023-10-16T10:00:35.033225+0000","last_became_peered":"2023-10-16T10:00:35.033225+0000","last_unstale":"2023-10-16T10:00:36.029368+0000","last_undegraded":"2023-10-16T10:00:36.029368+0000","last_fullsized":"2023-10-16T10:00:36.029368+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:56:41.501697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.027174+0000","last_change":"2023-10-16T10:00:34.023037+0000","last_active":"2023-10-16T10:00:36.027174+0000","last_peered":"2023-10-16T10:00:36.027174+0000","last_clean":"2023-10-16T10:00:36.027174+0000","last_became_active":"2023-10-16T10:00:34.022781+0000","last_became_peered":"2023-10-16T10:00:34.022781+0000","last_unstale":"2023-10-16T10:00:36.027174+0000","last_undegraded":"2023-10-16T10:00:36.027174+0000","last_fullsized":"2023-10-16T10:00:36.027174+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:51.044931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194965+0000","last_change":"2023-10-16T10:00:35.033027+0000","last_active":"2023-10-16T10:00:36.194965+0000","last_peered":"2023-10-16T10:00:36.194965+0000","last_clean":"2023-10-16T10:00:36.194965+0000","last_became_active":"2023-10-16T10:00:35.032717+0000","last_became_peered":"2023-10-16T10:00:35.032717+0000","last_unstale":"2023-10-16T10:00:36.194965+0000","last_undegraded":"2023-10-16T10:00:36.194965+0000","last_fullsized":"2023-10-16T10:00:36.194965+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:57:58.687087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194483+0000","last_change":"2023-10-16T10:00:34.028275+0000","last_active":"2023-10-16T10:00:36.194483+0000","last_peered":"2023-10-16T10:00:36.194483+0000","last_clean":"2023-10-16T10:00:36.194483+0000","last_became_active":"2023-10-16T10:00:34.028012+0000","last_became_peered":"2023-10-16T10:00:34.028012+0000","last_unstale":"2023-10-16T10:00:36.194483+0000","last_undegraded":"2023-10-16T10:00:36.194483+0000","last_fullsized":"2023-10-16T10:00:36.194483+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:16.790470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028967+0000","last_change":"2023-10-16T10:00:35.029045+0000","last_active":"2023-10-16T10:00:36.028967+0000","last_peered":"2023-10-16T10:00:36.028967+0000","last_clean":"2023-10-16T10:00:36.028967+0000","last_became_active":"2023-10-16T10:00:35.028766+0000","last_became_peered":"2023-10-16T10:00:35.028766+0000","last_unstale":"2023-10-16T10:00:36.028967+0000","last_undegraded":"2023-10-16T10:00:36.028967+0000","last_fullsized":"2023-10-16T10:00:36.028967+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:15:28.174207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195143+0000","last_change":"2023-10-16T10:00:34.023093+0000","last_active":"2023-10-16T10:00:36.195143+0000","last_peered":"2023-10-16T10:00:36.195143+0000","last_clean":"2023-10-16T10:00:36.195143+0000","last_became_active":"2023-10-16T10:00:34.022828+0000","last_became_peered":"2023-10-16T10:00:34.022828+0000","last_unstale":"2023-10-16T10:00:36.195143+0000","last_undegraded":"2023-10-16T10:00:36.195143+0000","last_fullsized":"2023-10-16T10:00:36.195143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:09:48.791112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195551+0000","last_change":"2023-10-16T10:00:35.033988+0000","last_active":"2023-10-16T10:00:36.195551+0000","last_peered":"2023-10-16T10:00:36.195551+0000","last_clean":"2023-10-16T10:00:36.195551+0000","last_became_active":"2023-10-16T10:00:35.032936+0000","last_became_peered":"2023-10-16T10:00:35.032936+0000","last_unstale":"2023-10-16T10:00:36.195551+0000","last_undegraded":"2023-10-16T10:00:36.195551+0000","last_fullsized":"2023-10-16T10:00:36.195551+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:38.437167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195372+0000","last_change":"2023-10-16T10:00:34.030314+0000","last_active":"2023-10-16T10:00:36.195372+0000","last_peered":"2023-10-16T10:00:36.195372+0000","last_clean":"2023-10-16T10:00:36.195372+0000","last_became_active":"2023-10-16T10:00:34.030055+0000","last_became_peered":"2023-10-16T10:00:34.030055+0000","last_unstale":"2023-10-16T10:00:36.195372+0000","last_undegraded":"2023-10-16T10:00:36.195372+0000","last_fullsized":"2023-10-16T10:00:36.195372+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:11.025672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195141+0000","last_change":"2023-10-16T10:00:35.032320+0000","last_active":"2023-10-16T10:00:36.195141+0000","last_peered":"2023-10-16T10:00:36.195141+0000","last_clean":"2023-10-16T10:00:36.195141+0000","last_became_active":"2023-10-16T10:00:35.031986+0000","last_became_peered":"2023-10-16T10:00:35.031986+0000","last_unstale":"2023-10-16T10:00:36.195141+0000","last_undegraded":"2023-10-16T10:00:36.195141+0000","last_fullsized":"2023-10-16T10:00:36.195141+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:03.996992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195657+0000","last_change":"2023-10-16T10:00:34.030558+0000","last_active":"2023-10-16T10:00:36.195657+0000","last_peered":"2023-10-16T10:00:36.195657+0000","last_clean":"2023-10-16T10:00:36.195657+0000","last_became_active":"2023-10-16T10:00:34.030249+0000","last_became_peered":"2023-10-16T10:00:34.030249+0000","last_unstale":"2023-10-16T10:00:36.195657+0000","last_undegraded":"2023-10-16T10:00:36.195657+0000","last_fullsized":"2023-10-16T10:00:36.195657+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:40.569358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.029498+0000","last_change":"2023-10-16T10:00:35.027926+0000","last_active":"2023-10-16T10:00:36.029498+0000","last_peered":"2023-10-16T10:00:36.029498+0000","last_clean":"2023-10-16T10:00:36.029498+0000","last_became_active":"2023-10-16T10:00:35.027208+0000","last_became_peered":"2023-10-16T10:00:35.027208+0000","last_unstale":"2023-10-16T10:00:36.029498+0000","last_undegraded":"2023-10-16T10:00:36.029498+0000","last_fullsized":"2023-10-16T10:00:36.029498+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:45.826798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195690+0000","last_change":"2023-10-16T10:00:34.028607+0000","last_active":"2023-10-16T10:00:36.195690+0000","last_peered":"2023-10-16T10:00:36.195690+0000","last_clean":"2023-10-16T10:00:36.195690+0000","last_became_active":"2023-10-16T10:00:34.028373+0000","last_became_peered":"2023-10-16T10:00:34.028373+0000","last_unstale":"2023-10-16T10:00:36.195690+0000","last_undegraded":"2023-10-16T10:00:36.195690+0000","last_fullsized":"2023-10-16T10:00:36.195690+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.273373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195725+0000","last_change":"2023-10-16T10:00:35.025215+0000","last_active":"2023-10-16T10:00:36.195725+0000","last_peered":"2023-10-16T10:00:36.195725+0000","last_clean":"2023-10-16T10:00:36.195725+0000","last_became_active":"2023-10-16T10:00:35.024962+0000","last_became_peered":"2023-10-16T10:00:35.024962+0000","last_unstale":"2023-10-16T10:00:36.195725+0000","last_undegraded":"2023-10-16T10:00:36.195725+0000","last_fullsized":"2023-10-16T10:00:36.195725+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.898298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194534+0000","last_change":"2023-10-16T10:00:34.019601+0000","last_active":"2023-10-16T10:00:36.194534+0000","last_peered":"2023-10-16T10:00:36.194534+0000","last_clean":"2023-10-16T10:00:36.194534+0000","last_became_active":"2023-10-16T10:00:34.019191+0000","last_became_peered":"2023-10-16T10:00:34.019191+0000","last_unstale":"2023-10-16T10:00:36.194534+0000","last_undegraded":"2023-10-16T10:00:36.194534+0000","last_fullsized":"2023-10-16T10:00:36.194534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:29.486652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028525+0000","last_change":"2023-10-16T10:00:35.040263+0000","last_active":"2023-10-16T10:00:36.028525+0000","last_peered":"2023-10-16T10:00:36.028525+0000","last_clean":"2023-10-16T10:00:36.028525+0000","last_became_active":"2023-10-16T10:00:35.040019+0000","last_became_peered":"2023-10-16T10:00:35.040019+0000","last_unstale":"2023-10-16T10:00:36.028525+0000","last_undegraded":"2023-10-16T10:00:36.028525+0000","last_fullsized":"2023-10-16T10:00:36.028525+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:40.411278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195228+0000","last_change":"2023-10-16T10:00:34.024325+0000","last_active":"2023-10-16T10:00:36.195228+0000","last_peered":"2023-10-16T10:00:36.195228+0000","last_clean":"2023-10-16T10:00:36.195228+0000","last_became_active":"2023-10-16T10:00:34.023163+0000","last_became_peered":"2023-10-16T10:00:34.023163+0000","last_unstale":"2023-10-16T10:00:36.195228+0000","last_undegraded":"2023-10-16T10:00:36.195228+0000","last_fullsized":"2023-10-16T10:00:36.195228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:58.045684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.195232+0000","last_change":"2023-10-16T10:00:35.030312+0000","last_active":"2023-10-16T10:00:36.195232+0000","last_peered":"2023-10-16T10:00:36.195232+0000","last_clean":"2023-10-16T10:00:36.195232+0000","last_became_active":"2023-10-16T10:00:35.030025+0000","last_became_peered":"2023-10-16T10:00:35.030025+0000","last_unstale":"2023-10-16T10:00:36.195232+0000","last_undegraded":"2023-10-16T10:00:36.195232+0000","last_fullsized":"2023-10-16T10:00:36.195232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.898298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194319+0000","last_change":"2023-10-16T10:00:34.030609+0000","last_active":"2023-10-16T10:00:36.194319+0000","last_peered":"2023-10-16T10:00:36.194319+0000","last_clean":"2023-10-16T10:00:36.194319+0000","last_became_active":"2023-10-16T10:00:34.030331+0000","last_became_peered":"2023-10-16T10:00:34.030331+0000","last_unstale":"2023-10-16T10:00:36.194319+0000","last_undegraded":"2023-10-16T10:00:36.194319+0000","last_fullsized":"2023-10-16T10:00:36.194319+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:34:17.143076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.194591+0000","last_change":"2023-10-16T10:00:35.029389+0000","last_active":"2023-10-16T10:00:36.194591+0000","last_peered":"2023-10-16T10:00:36.194591+0000","last_clean":"2023-10-16T10:00:36.194591+0000","last_became_active":"2023-10-16T10:00:35.029102+0000","last_became_peered":"2023-10-16T10:00:35.029102+0000","last_unstale":"2023-10-16T10:00:36.194591+0000","last_undegraded":"2023-10-16T10:00:36.194591+0000","last_fullsized":"2023-10-16T10:00:36.194591+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:33.999462+0000","last_clean_scrub_stamp":"2023-10-16T10:00:33.999462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:05.398826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-16T10:00:36.028919+0000","last_change":"2023-10-16T10:00:34.030564+0000","last_active":"2023-10-16T10:00:36.028919+0000","last_peered":"2023-10-16T10:00:36.028919+0000","last_clean":"2023-10-16T10:00:36.028919+0000","last_became_active":"2023-10-16T10:00:34.030220+0000","last_became_peered":"2023-10-16T10:00:34.030220+0000","last_unstale":"2023-10-16T10:00:36.028919+0000","last_undegraded":"2023-10-16T10:00:36.028919+0000","last_fullsized":"2023-10-16T10:00:36.028919+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:00:32.994181+0000","last_clean_scrub_stamp":"2023-10-16T10:00:32.994181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:29:56.372390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"93'183","reported_seq":480,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-16T10:00:35.014682+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:00:35.014682+0000","last_peered":"2023-10-16T10:00:35.014682+0000","last_clean":"2023-10-16T10:00:35.014682+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:00:35.014682+0000","last_undegraded":"2023-10-16T10:00:35.014682+0000","last_fullsized":"2023-10-16T10:00:35.014682+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542344,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":1207497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55700000000000005,"15min":0.57999999999999996},"min":{"1min":0.34599999999999997,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.629,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.26500000000000001},"max":{"1min":0.83099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.442}]},{"osd":1,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.379,"5min":0.32100000000000001,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.77000000000000002,"15min":0.80400000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57799999999999996,"15min":0.59899999999999998},"min":{"1min":0.33500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72999999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.753,"5min":0.75900000000000001,"15min":0.81200000000000006},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.61299999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68100000000000005}]},{"osd":4,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.32100000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":0.73799999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77600000000000002,"5min":1.006,"15min":1.006},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.437,"5min":0.38900000000000001,"15min":0.35599999999999998},"max":{"1min":0.79300000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.95199999999999996,"15min":1.373},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.308},"max":{"1min":0.748,"5min":0.85799999999999998,"15min":1.4099999999999999},"last":0.69799999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1273856,"data_stored":617753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.57099999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.33800000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.63200000000000001,"5min":0.745,"15min":0.83399999999999996},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.54700000000000004,"15min":0.56799999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.78900000000000003,"15min":0.81100000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]},{"osd":2,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55900000000000005,"15min":0.57899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64500000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.55500000000000005,"15min":0.57899999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.63500000000000001,"5min":0.83699999999999997,"15min":0.84499999999999997},"last":0.57899999999999996}]},{"osd":3,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56899999999999995,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76300000000000001,"5min":0.85399999999999998,"15min":0.85899999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48699999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.317,"15min":0.311},"max":{"1min":1,"5min":1,"15min":1},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.37,"5min":0.35199999999999998,"15min":0.30199999999999999},"max":{"1min":0.69999999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.78600000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.61199999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.85199999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.83499999999999996,"15min":0.86399999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69499999999999995,"5min":0.83099999999999996,"15min":0.83399999999999996},"last":0.86699999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542343,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50480,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321360,"statfs":{"total":96636764160,"available":96585072640,"internally_reserved":0,"allocated":1273856,"data_stored":617253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36899999999999999,"15min":0.372},"min":{"1min":0.221,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46400000000000002,"5min":1.5,"15min":1.5},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.39700000000000002,"15min":0.40899999999999997},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.42699999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.308}]},{"osd":2,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.27800000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47699999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45000000000000001,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55000000000000004,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.29099999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.52400000000000002,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.42899999999999999,"15min":0.437},"min":{"1min":0.29399999999999998,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.48499999999999999,"5min":1.379,"15min":1.379},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.40500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.40699999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.79700000000000004,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":1.26,"15min":1.26},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.63,"15min":0.64800000000000002},"min":{"1min":0.45400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76400000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.80800000000000005,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.64800000000000002},"min":{"1min":0.36799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.749,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Mon Oct 16 10:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63600000000000001,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.64400000000000002,"15min":0.66300000000000003},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.41499999999999998},"max":{"1min":0.78300000000000003,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.67600000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542344,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1847296,"data_stored":1200737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.36099999999999999,"15min":0.373},"min":{"1min":0.192,"5min":0.192,"15min":0.191},"max":{"1min":0.53800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.38600000000000001,"15min":0.40000000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.48499999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.252}]},{"osd":2,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39400000000000002,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51900000000000002,"5min":0.61799999999999999,"15min":0.626},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.40000000000000002,"15min":0.42299999999999999},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.42999999999999999,"5min":0.624,"15min":0.63600000000000001},"last":0.313}]},{"osd":3,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.40899999999999997,"15min":0.432},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58399999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59999999999999998,"5min":0.69299999999999995,"15min":0.69699999999999995},"last":0.33400000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60799999999999998,"15min":0.63},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.628},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.60899999999999999,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.90500000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.627,"15min":0.64700000000000002},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34200000000000003},"max":{"1min":0.76600000000000001,"5min":1.004,"15min":1.004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.622,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65500000000000003}]},{"osd":6,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.81000000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.46100000000000002,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:00:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.63700000000000001,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82099999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.63500000000000001,"15min":0.65900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.80700000000000005,"5min":0.91100000000000003,"15min":0.95699999999999996},"last":0.60199999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542343,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36584,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335256,"statfs":{"total":96636764160,"available":96599302144,"internally_reserved":0,"allocated":1265664,"data_stored":617641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.36899999999999999,"15min":0.375},"min":{"1min":0.252,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.45800000000000002,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.373},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.38700000000000001,"15min":0.39900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.51400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51500000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39700000000000002,"15min":0.40600000000000003},"min":{"1min":0.252,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.40500000000000003,"15min":0.42099999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.214},"max":{"1min":0.496,"5min":0.58199999999999996,"15min":0.64000000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.40699999999999997},"min":{"1min":0.25800000000000001,"5min":0.23000000000000001,"15min":0.22900000000000001},"max":{"1min":0.44600000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.435},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.44800000000000001,"15min":0.46700000000000003},"min":{"1min":0.31,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.747,"15min":0.747},"last":0.47199999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.436,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.747},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61199999999999999,"15min":0.625},"min":{"1min":0.43099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":1.091,"15min":1.091},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.85899999999999999,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.63400000000000001},"min":{"1min":0.39800000000000002,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.97499999999999998},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.35099999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.64600000000000002},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78500000000000003,"5min":0.93799999999999994,"15min":0.97899999999999998},"last":0.81999999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:00:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.624,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73699999999999999,"5min":0.98299999999999998,"15min":0.98799999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83099999999999996,"5min":0.874,"15min":0.998},"last":0.84099999999999997}]}]},{"osd":3,"up_from":14,"seq":60129542344,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1261568,"data_stored":611947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.373,"15min":0.38400000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.192},"max":{"1min":0.52400000000000002,"5min":0.63900000000000001,"15min":0.64900000000000002},"last":0.217},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.28799999999999998,"5min":0.185,"15min":0.185},"max":{"1min":0.50700000000000001,"5min":0.626,"15min":0.63100000000000001},"last":0.26200000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.38400000000000001,"15min":0.39300000000000002},"min":{"1min":0.24399999999999999,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.437,"5min":0.66100000000000003,"15min":0.72199999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41299999999999998,"15min":0.41999999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.433,"15min":0.435},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.54200000000000004,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.44,"15min":0.45100000000000001},"min":{"1min":0.30199999999999999,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.30599999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.61199999999999999},"min":{"1min":0.41999999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.95799999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":0.879,"15min":0.98699999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61799999999999999,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.996},"last":0.60399999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.39200000000000002},"max":{"1min":0.92400000000000004,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59999999999999998,"15min":0.629},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.86799999999999999,"15min":0.99099999999999999},"last":0.74399999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":1.3,"15min":1.3},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81200000000000006,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.80000000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54624,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317216,"statfs":{"total":96636764160,"available":96580829184,"internally_reserved":0,"allocated":1261568,"data_stored":617059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.58199999999999996},"min":{"1min":0.30099999999999999,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.72399999999999998,"5min":0.76900000000000002,"15min":0.89400000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.56799999999999995,"15min":0.58299999999999996},"min":{"1min":0.33300000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81299999999999994},"last":0.63900000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.42399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70699999999999996,"5min":0.84599999999999997,"15min":0.86599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55000000000000004,"15min":0.57499999999999996},"min":{"1min":0.35199999999999998,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.73999999999999999,"5min":0.78900000000000003,"15min":0.83799999999999997},"last":0.54000000000000004}]},{"osd":2,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.755,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.501},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69299999999999995,"5min":0.81999999999999995,"15min":0.89100000000000001},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.42099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.67400000000000004,"5min":0.83199999999999996,"15min":0.86499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57399999999999995,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69399999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56599999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63300000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64000000000000001,"5min":0.73399999999999999,"15min":0.80400000000000005},"last":0.38800000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.82999999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79300000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 16 10:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69399999999999995,"5min":0.78600000000000003,"15min":0.80700000000000005},"last":0.624},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76000000000000001,"5min":0.83999999999999997,"15min":0.86399999999999999},"last":0.65200000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542344,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1265664,"data_stored":611969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.253},"max":{"1min":0.69899999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.378,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":0.84199999999999997,"15min":0.91800000000000004},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.56100000000000005,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.28899999999999998,"15min":0.24399999999999999},"max":{"1min":0.58899999999999997,"5min":0.77900000000000003,"15min":0.82799999999999996},"last":0.442},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.307},"max":{"1min":0.67400000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.315,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.373},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.33900000000000002},"max":{"1min":0.70499999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54600000000000004}]},{"osd":3,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.58699999999999997,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":0.88600000000000001,"15min":0.89000000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.44500000000000001,"5min":0.39300000000000002,"15min":0.33200000000000002},"max":{"1min":0.77100000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.81499999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.38800000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83699999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.5}]},{"osd":7,"last update":"Mon Oct 16 10:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.88400000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.50900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:00:39.525 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:00:39.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:00:39.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.679+0000 7f6195b8e700 1 Processor -- start 2023-10-16T10:00:39.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f6195b8e700 1 -- start start 2023-10-16T10:00:39.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f6195b8e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61900cb2b0 0x7f619006f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f6195b8e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6190070590 con 0x7f61900cb690 2023-10-16T10:00:39.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f6195b8e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6190156330 con 0x7f61900b5fd0 2023-10-16T10:00:39.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f6195b8e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619006fcf0 con 0x7f61900cb2b0 2023-10-16T10:00:39.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f618f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61900cb2b0 0x7f619006f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f618f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61900cb2b0 0x7f619006f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44562/0 (socket says 172.21.15.181:44562) 2023-10-16T10:00:39.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f618effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f61900cb690 0x7f61900b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55640/0 (socket says 172.21.15.181:55640) 2023-10-16T10:00:39.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.680+0000 7f618f7fe700 1 -- 172.21.15.181:0/388640302 learned_addr learned my addr 172.21.15.181:0/388640302 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:39.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618f7fe700 1 -- 172.21.15.181:0/388640302 >> v1:172.21.15.190:6789/0 conn(0x7f61900b5fd0 legacy=0x7f619015f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:39.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618e7fc700 1 -- 172.21.15.181:0/388640302 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628539722 0 0) 0x7f6190070590 con 0x7f61900cb690 2023-10-16T10:00:39.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618f7fe700 1 -- 172.21.15.181:0/388640302 >> v1:172.21.15.181:6789/0 conn(0x7f61900cb690 legacy=0x7f61900b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618f7fe700 1 -- 172.21.15.181:0/388640302 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619006ffd0 con 0x7f61900cb2b0 2023-10-16T10:00:39.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618f7fe700 1 --2- 172.21.15.181:0/388640302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb2b0 0x7f619006f710 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f61840098f0 tx=0x7f6184009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f037ce0ffeb0691 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:39.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618e7fc700 1 -- 172.21.15.181:0/388640302 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618400b8a0 con 0x7f61900cb2b0 2023-10-16T10:00:39.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618e7fc700 1 -- 172.21.15.181:0/388640302 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618400ba00 con 0x7f61900cb2b0 2023-10-16T10:00:39.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.681+0000 7f618e7fc700 1 -- 172.21.15.181:0/388640302 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618400bd90 con 0x7f61900cb2b0 2023-10-16T10:00:39.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.682+0000 7f6195b8e700 1 -- 172.21.15.181:0/388640302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb2b0 msgr2=0x7f619006f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.682+0000 7f6195b8e700 1 --2- 172.21.15.181:0/388640302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb2b0 0x7f619006f710 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f61840098f0 tx=0x7f6184009c00 comp rx=0 tx=0).stop 2023-10-16T10:00:39.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.682+0000 7f6195b8e700 1 -- 172.21.15.181:0/388640302 shutdown_connections 2023-10-16T10:00:39.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.682+0000 7f6195b8e700 1 --2- 172.21.15.181:0/388640302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb2b0 0x7f619006f710 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.682+0000 7f6195b8e700 1 -- 172.21.15.181:0/388640302 >> 172.21.15.181:0/388640302 conn(0x7f61900cd010 msgr2=0x7f61900cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:39.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.682+0000 7f6195b8e700 1 -- 172.21.15.181:0/388640302 shutdown_connections 2023-10-16T10:00:39.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.682+0000 7f6195b8e700 1 -- 172.21.15.181:0/388640302 wait complete. 2023-10-16T10:00:39.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.683+0000 7f6195b8e700 1 Processor -- start 2023-10-16T10:00:39.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.683+0000 7f6195b8e700 1 -- start start 2023-10-16T10:00:39.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61900b5fd0 0x7f61901f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61900cb2b0 0x7f61901f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb690 0x7f61900c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61901f6500 con 0x7f61900cb690 2023-10-16T10:00:39.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61900c6070 con 0x7f61900b5fd0 2023-10-16T10:00:39.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61900c6350 con 0x7f61900cb2b0 2023-10-16T10:00:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb690 0x7f61900c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb690 0x7f61900c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44570/0 (socket says 172.21.15.181:44570) 2023-10-16T10:00:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 -- 172.21.15.181:0/2822224060 learned_addr learned my addr 172.21.15.181:0/2822224060 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618f7fe700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61900b5fd0 0x7f61901f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618effd700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61900cb2b0 0x7f61901f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 -- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61900cb2b0 msgr2=0x7f61901f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61900cb2b0 0x7f61901f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 -- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61900b5fd0 msgr2=0x7f61901f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61900b5fd0 0x7f61901f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 -- 172.21.15.181:0/2822224060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184009550 con 0x7f61900cb690 2023-10-16T10:00:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f618ffff700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb690 0x7f61900c5720 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f618800cc50 tx=0x7f618800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6177fff700 1 -- 172.21.15.181:0/2822224060 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800ad80 con 0x7f61900cb690 2023-10-16T10:00:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6177fff700 1 -- 172.21.15.181:0/2822224060 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618800ce50 con 0x7f61900cb690 2023-10-16T10:00:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6177fff700 1 -- 172.21.15.181:0/2822224060 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188003c40 con 0x7f61900cb690 2023-10-16T10:00:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61900c6630 con 0x7f61900cb690 2023-10-16T10:00:39.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.684+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61900c6bf0 con 0x7f61900cb690 2023-10-16T10:00:39.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.685+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619006f4f0 con 0x7f61900cb690 2023-10-16T10:00:39.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.686+0000 7f6177fff700 1 -- 172.21.15.181:0/2822224060 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f61880050d0 con 0x7f61900cb690 2023-10-16T10:00:39.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.686+0000 7f6177fff700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f617c06b020 0x7f617c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:39.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.686+0000 7f618f7fe700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f617c06b020 0x7f617c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:39.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.686+0000 7f6177fff700 1 -- 172.21.15.181:0/2822224060 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f6188089950 con 0x7f61900cb690 2023-10-16T10:00:39.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.687+0000 7f618f7fe700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f617c06b020 0x7f617c06d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f61900cb010 tx=0x7f6184008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:39.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.691+0000 7f6177fff700 1 -- 172.21.15.181:0/2822224060 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61880547d0 con 0x7f61900cb690 2023-10-16T10:00:39.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.961+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f61900c3df0 con 0x7f61900cb690 2023-10-16T10:00:39.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.964+0000 7f6177fff700 1 -- 172.21.15.181:0/2822224060 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6188057df0 con 0x7f61900cb690 2023-10-16T10:00:39.965 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T10:00:39.966 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T10:00:39.966 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T10:00:39.966 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T10:00:39.966 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:00:39.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f617c06b020 msgr2=0x7f617c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f617c06b020 0x7f617c06d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f61900cb010 tx=0x7f6184008030 comp rx=0 tx=0).stop 2023-10-16T10:00:39.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb690 msgr2=0x7f61900c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:39.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb690 0x7f61900c5720 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f618800cc50 tx=0x7f618800a340 comp rx=0 tx=0).stop 2023-10-16T10:00:39.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 shutdown_connections 2023-10-16T10:00:39.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f617c06b020 0x7f617c06d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61900cb690 0x7f61900c5720 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61900b5fd0 0x7f61901f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 --2- 172.21.15.181:0/2822224060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61900cb2b0 0x7f61901f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:39.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 >> 172.21.15.181:0/2822224060 conn(0x7f61900cd010 msgr2=0x7f61900afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:39.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 shutdown_connections 2023-10-16T10:00:39.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:39.966+0000 7f6195b8e700 1 -- 172.21.15.181:0/2822224060 wait complete. 2023-10-16T10:00:39.971 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T10:00:39.980 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:00:40.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.101+0000 7fdcec22d700 1 Processor -- start 2023-10-16T10:00:40.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.101+0000 7fdcec22d700 1 -- start start 2023-10-16T10:00:40.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdcec22d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdce415bac0 0x7fdce4072800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdcec22d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce4072d40 0x7fdce40cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdcec22d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce406fc70 con 0x7fdce40b61a0 2023-10-16T10:00:40.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdcec22d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce4070550 con 0x7fdce415bac0 2023-10-16T10:00:40.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdcec22d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce4070020 con 0x7fdce4072d40 2023-10-16T10:00:40.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdce97c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce4072d40 0x7fdce40cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdce97c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce4072d40 0x7fdce40cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:41602/0 (socket says 172.21.15.181:41602) 2023-10-16T10:00:40.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdce97c8700 1 -- 172.21.15.181:0/3831671361 learned_addr learned my addr 172.21.15.181:0/3831671361 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:40.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdce9fc9700 1 --2- 172.21.15.181:0/3831671361 >> v2:172.21.15.190:3300/0 conn(0x7fdce415bac0 0x7fdce4072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdce97c8700 1 -- 172.21.15.181:0/3831671361 >> v2:172.21.15.190:3300/0 conn(0x7fdce415bac0 msgr2=0x7fdce4072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdce97c8700 1 --2- 172.21.15.181:0/3831671361 >> v2:172.21.15.190:3300/0 conn(0x7fdce415bac0 0x7fdce4072800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.102+0000 7fdce97c8700 1 -- 172.21.15.181:0/3831671361 >> v1:172.21.15.190:6789/0 conn(0x7fdce40b61a0 legacy=0x7fdce40b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdce97c8700 1 -- 172.21.15.181:0/3831671361 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4072370 con 0x7fdce4072d40 2023-10-16T10:00:40.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdce9fc9700 1 --2- 172.21.15.181:0/3831671361 >> v2:172.21.15.190:3300/0 conn(0x7fdce415bac0 0x7fdce4072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:40.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdce97c8700 1 --2- 172.21.15.181:0/3831671361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce4072d40 0x7fdce40cab50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40098f0 tx=0x7fdcd4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1282f8985f123a39 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdce8fc7700 1 -- 172.21.15.181:0/3831671361 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4011830 con 0x7fdce4072d40 2023-10-16T10:00:40.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdce8fc7700 1 -- 172.21.15.181:0/3831671361 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd4011990 con 0x7fdce4072d40 2023-10-16T10:00:40.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdce8fc7700 1 -- 172.21.15.181:0/3831671361 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4011d20 con 0x7fdce4072d40 2023-10-16T10:00:40.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdcec22d700 1 -- 172.21.15.181:0/3831671361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce4072d40 msgr2=0x7fdce40cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdcec22d700 1 --2- 172.21.15.181:0/3831671361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce4072d40 0x7fdce40cab50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40098f0 tx=0x7fdcd4009c00 comp rx=0 tx=0).stop 2023-10-16T10:00:40.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdcec22d700 1 -- 172.21.15.181:0/3831671361 shutdown_connections 2023-10-16T10:00:40.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdcec22d700 1 --2- 172.21.15.181:0/3831671361 >> v2:172.21.15.190:3300/0 conn(0x7fdce415bac0 0x7fdce4072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdcec22d700 1 --2- 172.21.15.181:0/3831671361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce4072d40 0x7fdce40cab50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.103+0000 7fdcec22d700 1 -- 172.21.15.181:0/3831671361 >> 172.21.15.181:0/3831671361 conn(0x7fdce40ccf80 msgr2=0x7fdce40cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.104+0000 7fdcec22d700 1 -- 172.21.15.181:0/3831671361 shutdown_connections 2023-10-16T10:00:40.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.104+0000 7fdcec22d700 1 -- 172.21.15.181:0/3831671361 wait complete. 2023-10-16T10:00:40.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 Processor -- start 2023-10-16T10:00:40.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 -- start start 2023-10-16T10:00:40.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce40b61a0 0x7fdce41f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdce415bac0 0x7fdce414bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce41f47e0 0x7fdce41f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41561b0 con 0x7fdce40b61a0 2023-10-16T10:00:40.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce414c120 con 0x7fdce415bac0 2023-10-16T10:00:40.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.105+0000 7fdcec22d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41f1ab0 con 0x7fdce41f47e0 2023-10-16T10:00:40.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce40b61a0 0x7fdce41f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce40b61a0 0x7fdce41f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44580/0 (socket says 172.21.15.181:44580) 2023-10-16T10:00:40.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 -- 172.21.15.181:0/15183798 learned_addr learned my addr 172.21.15.181:0/15183798 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:40.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdcea7ca700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce41f47e0 0x7fdce41f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce97c8700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdce415bac0 0x7fdce414bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 -- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce41f47e0 msgr2=0x7fdce41f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce41f47e0 0x7fdce41f11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 -- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdce415bac0 msgr2=0x7fdce414bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdce415bac0 0x7fdce414bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 -- 172.21.15.181:0/15183798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce00094a0 con 0x7fdce40b61a0 2023-10-16T10:00:40.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdce9fc9700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce40b61a0 0x7fdce41f23b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fdce000ebe0 tx=0x7fdce0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdcda7fc700 1 -- 172.21.15.181:0/15183798 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000c910 con 0x7fdce40b61a0 2023-10-16T10:00:40.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdcda7fc700 1 -- 172.21.15.181:0/15183798 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce000ca70 con 0x7fdce40b61a0 2023-10-16T10:00:40.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd4009550 con 0x7fdce40b61a0 2023-10-16T10:00:40.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdcda7fc700 1 -- 172.21.15.181:0/15183798 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000cde0 con 0x7fdce40b61a0 2023-10-16T10:00:40.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.106+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce41f2070 con 0x7fdce40b61a0 2023-10-16T10:00:40.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.107+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce404ebb0 con 0x7fdce40b61a0 2023-10-16T10:00:40.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.108+0000 7fdcda7fc700 1 -- 172.21.15.181:0/15183798 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdce0013470 con 0x7fdce40b61a0 2023-10-16T10:00:40.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.108+0000 7fdcda7fc700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fdcc806b020 0x7fdcc806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.108+0000 7fdcda7fc700 1 -- 172.21.15.181:0/15183798 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fdce0089a20 con 0x7fdce40b61a0 2023-10-16T10:00:40.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.108+0000 7fdce97c8700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fdcc806b020 0x7fdcc806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.109+0000 7fdce97c8700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fdcc806b020 0x7fdcc806d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40098f0 tx=0x7fdcd4008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.113+0000 7fdcda7fc700 1 -- 172.21.15.181:0/15183798 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdce00548a0 con 0x7fdce40b61a0 2023-10-16T10:00:40.228 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:40.231 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:00:40.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.352+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdce40c3900 con 0x7fdce40b61a0 2023-10-16T10:00:40.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.353+0000 7fdcda7fc700 1 -- 172.21.15.181:0/15183798 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fdce0017090 con 0x7fdce40b61a0 2023-10-16T10:00:40.355 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:40.355 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1358211190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1358211190},{"type":"v1","addr":"172.21.15.190:6835","nonce":1358211190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6844,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2394829192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2394829192},{"type":"v1","addr":"172.21.15.190:6834","nonce":2394829192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2217231321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2217231321},{"type":"v1","addr":"172.21.15.181:6836","nonce":2217231321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:00:34.411162+0000","modified":"2023-10-16T10:00:37.431652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1158962128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1158962128},{"type":"v1","addr":"172.21.15.181:6837","nonce":1158962128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:00:40.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fdcc806b020 msgr2=0x7fdcc806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fdcc806b020 0x7fdcc806d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40098f0 tx=0x7fdcd4008030 comp rx=0 tx=0).stop 2023-10-16T10:00:40.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce40b61a0 msgr2=0x7fdce41f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce40b61a0 0x7fdce41f23b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fdce000ebe0 tx=0x7fdce0009ec0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 shutdown_connections 2023-10-16T10:00:40.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fdcc806b020 0x7fdcc806d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdce41f47e0 0x7fdce41f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce40b61a0 0x7fdce41f23b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 --2- 172.21.15.181:0/15183798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdce415bac0 0x7fdce414bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 >> 172.21.15.181:0/15183798 conn(0x7fdce40ccf80 msgr2=0x7fdce40b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 shutdown_connections 2023-10-16T10:00:40.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.355+0000 7fdcec22d700 1 -- 172.21.15.181:0/15183798 wait complete. 2023-10-16T10:00:40.360 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 49 2023-10-16T10:00:40.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:00:34.411162+0000', 'modified': '2023-10-16T10:00:37.431652+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7491': {'gid': 7491, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/1158962128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 1158962128}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1158962128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:00:40.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:00:40.773 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T10:00:40.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.895+0000 7fa82f115700 1 Processor -- start 2023-10-16T10:00:40.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.895+0000 7fa82f115700 1 -- start start 2023-10-16T10:00:40.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82f115700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa82806d210 0x7fa8280cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82f115700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa8280cc4c0 0x7fa8280cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82f115700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8280cf460 con 0x7fa8280ce300 2023-10-16T10:00:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82f115700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8280d0380 con 0x7fa82806d210 2023-10-16T10:00:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82f115700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8280cc940 con 0x7fa8280cc4c0 2023-10-16T10:00:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82ceb1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa82806d210 0x7fa8280cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82d6b2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa8280ce300 0x7fa82815d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55670/0 (socket says 172.21.15.181:55670) 2023-10-16T10:00:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82ceb1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa82806d210 0x7fa8280cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44588/0 (socket says 172.21.15.181:44588) 2023-10-16T10:00:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82d6b2700 1 -- 172.21.15.181:0/2644980527 learned_addr learned my addr 172.21.15.181:0/2644980527 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82ceb1700 1 -- 172.21.15.181:0/2644980527 >> v2:172.21.15.190:3300/0 conn(0x7fa8280cc4c0 msgr2=0x7fa8280cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa827fff700 1 --2- 172.21.15.181:0/2644980527 >> v2:172.21.15.190:3300/0 conn(0x7fa8280cc4c0 0x7fa8280cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2644980527 >> v2:172.21.15.190:3300/0 conn(0x7fa8280cc4c0 0x7fa8280cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82ceb1700 1 -- 172.21.15.181:0/2644980527 >> v1:172.21.15.181:6789/0 conn(0x7fa8280ce300 legacy=0x7fa82815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa8277fe700 1 -- 172.21.15.181:0/2644980527 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439870704 0 0) 0x7fa8280cf460 con 0x7fa8280ce300 2023-10-16T10:00:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82ceb1700 1 -- 172.21.15.181:0/2644980527 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82806c700 con 0x7fa82806d210 2023-10-16T10:00:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.896+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2644980527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82806d210 0x7fa8280cd4b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa82806e850 tx=0x7fa81c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3476375da8f434 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa8277fe700 1 -- 172.21.15.181:0/2644980527 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c011830 con 0x7fa82806d210 2023-10-16T10:00:40.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa8277fe700 1 -- 172.21.15.181:0/2644980527 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c011990 con 0x7fa82806d210 2023-10-16T10:00:40.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa8277fe700 1 -- 172.21.15.181:0/2644980527 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c011d20 con 0x7fa82806d210 2023-10-16T10:00:40.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 -- 172.21.15.181:0/2644980527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82806d210 msgr2=0x7fa8280cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 --2- 172.21.15.181:0/2644980527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82806d210 0x7fa8280cd4b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa82806e850 tx=0x7fa81c009b20 comp rx=0 tx=0).stop 2023-10-16T10:00:40.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 -- 172.21.15.181:0/2644980527 shutdown_connections 2023-10-16T10:00:40.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 --2- 172.21.15.181:0/2644980527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82806d210 0x7fa8280cd4b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 --2- 172.21.15.181:0/2644980527 >> v2:172.21.15.190:3300/0 conn(0x7fa8280cc4c0 0x7fa8280cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 -- 172.21.15.181:0/2644980527 >> 172.21.15.181:0/2644980527 conn(0x7fa82809a9a0 msgr2=0x7fa8280d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 -- 172.21.15.181:0/2644980527 shutdown_connections 2023-10-16T10:00:40.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.897+0000 7fa82f115700 1 -- 172.21.15.181:0/2644980527 wait complete. 2023-10-16T10:00:40.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.898+0000 7fa82f115700 1 Processor -- start 2023-10-16T10:00:40.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.898+0000 7fa82f115700 1 -- start start 2023-10-16T10:00:40.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82f115700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa82806d210 0x7fa8281f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82f115700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8280cc4c0 0x7fa82815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82f115700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8280ce300 0x7fa8280c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82f115700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8281f4160 con 0x7fa8280cc4c0 2023-10-16T10:00:40.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82f115700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8281f74e0 con 0x7fa82806d210 2023-10-16T10:00:40.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82f115700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82815f130 con 0x7fa8280ce300 2023-10-16T10:00:40.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8280cc4c0 0x7fa82815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8280cc4c0 0x7fa82815ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44596/0 (socket says 172.21.15.181:44596) 2023-10-16T10:00:40.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa827fff700 1 -- 172.21.15.181:0/2679425037 learned_addr learned my addr 172.21.15.181:0/2679425037 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:40.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa82806d210 0x7fa8281f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.899+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8280ce300 0x7fa8280c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa827fff700 1 -- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8280ce300 msgr2=0x7fa8280c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa827fff700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8280ce300 0x7fa8280c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa827fff700 1 -- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa82806d210 msgr2=0x7fa8281f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa827fff700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa82806d210 0x7fa8281f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa827fff700 1 -- 172.21.15.181:0/2679425037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0094a0 con 0x7fa8280cc4c0 2023-10-16T10:00:40.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa82806d210 0x7fa8281f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:40.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8280ce300 0x7fa8280c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:40.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa827fff700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8280cc4c0 0x7fa82815ebf0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c7f0 tx=0x7fa81400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa8257fa700 1 -- 172.21.15.181:0/2679425037 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814011800 con 0x7fa8280cc4c0 2023-10-16T10:00:40.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa8257fa700 1 -- 172.21.15.181:0/2679425037 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814011960 con 0x7fa8280cc4c0 2023-10-16T10:00:40.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8280ae5c0 con 0x7fa8280cc4c0 2023-10-16T10:00:40.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa8257fa700 1 -- 172.21.15.181:0/2679425037 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814011cf0 con 0x7fa8280cc4c0 2023-10-16T10:00:40.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.900+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8280ca8b0 con 0x7fa8280cc4c0 2023-10-16T10:00:40.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.901+0000 7fa8257fa700 1 -- 172.21.15.181:0/2679425037 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa814018360 con 0x7fa8280cc4c0 2023-10-16T10:00:40.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.902+0000 7fa8257fa700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa80c06b020 0x7fa80c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.902+0000 7fa8257fa700 1 -- 172.21.15.181:0/2679425037 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fa814089990 con 0x7fa8280cc4c0 2023-10-16T10:00:40.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.902+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa80c06b020 0x7fa80c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.902+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa80c06b020 0x7fa80c06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8280d1b40 tx=0x7fa81c008030 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.904+0000 7fa83b2ffb80 1 Processor -- start 2023-10-16T10:00:40.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.904+0000 7fa83b2ffb80 1 -- start start 2023-10-16T10:00:40.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c14a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c14fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad1b5ec20 con 0x555ad1c12630 2023-10-16T10:00:40.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad1b43c10 con 0x555ad1c0d220 2023-10-16T10:00:40.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ad1b41530 con 0x555ad1be8010 2023-10-16T10:00:40.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c14fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c14fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44604/0 (socket says 172.21.15.181:44604) 2023-10-16T10:00:40.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 -- 172.21.15.181:0/2369270735 learned_addr learned my addr 172.21.15.181:0/2369270735 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:40.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c14a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 -- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 msgr2=0x555ad1be86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 -- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 msgr2=0x555ad1c14a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c14a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 -- 172.21.15.181:0/2369270735 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ad1afa3a0 con 0x555ad1c12630 2023-10-16T10:00:40.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:40.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c14a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:40.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa827fff700 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c14fb0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa814060110 tx=0x7fa814058b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5c25b32ff1424c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa8127fc700 1 -- 172.21.15.181:0/2369270735 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814089040 con 0x555ad1c12630 2023-10-16T10:00:40.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa8127fc700 1 -- 172.21.15.181:0/2369270735 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814060640 con 0x555ad1c12630 2023-10-16T10:00:40.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.905+0000 7fa8127fc700 1 -- 172.21.15.181:0/2369270735 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81405c0d0 con 0x555ad1c12630 2023-10-16T10:00:40.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/2369270735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 msgr2=0x555ad1c14fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c14fb0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa814060110 tx=0x7fa814058b10 comp rx=0 tx=0).stop 2023-10-16T10:00:40.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/2369270735 shutdown_connections 2023-10-16T10:00:40.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c14a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/2369270735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c14fb0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/2369270735 >> 172.21.15.181:0/2369270735 conn(0x555ad1b17d80 msgr2=0x555ad1c32ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/2369270735 shutdown_connections 2023-10-16T10:00:40.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.906+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/2369270735 wait complete. 2023-10-16T10:00:40.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.907+0000 7fa83b2ffb80 1 -- start start 2023-10-16T10:00:40.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.907+0000 7fa83b2ffb80 1 Processor -- start 2023-10-16T10:00:40.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.907+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1be8010 0x555ad1c0aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1c12630 0x555ad1c2fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad1b3ce20 con 0x555ad1c0d220 2023-10-16T10:00:40.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad1ad96c0 con 0x555ad1be8010 2023-10-16T10:00:40.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ad1ad9360 con 0x555ad1c12630 2023-10-16T10:00:40.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 10 client.0 _ll_register_callbacks cb 0x555ad19367b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:00:40.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c0b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44610/0 (socket says 172.21.15.181:44610) 2023-10-16T10:00:40.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 -- 172.21.15.181:0/3812384010 learned_addr learned my addr 172.21.15.181:0/3812384010 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:40.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1c12630 0x555ad1c2fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1be8010 0x555ad1c0aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 -- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1c12630 msgr2=0x555ad1c2fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1c12630 0x555ad1c2fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 -- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1be8010 msgr2=0x555ad1c0aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1be8010 0x555ad1c0aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 -- 172.21.15.181:0/3812384010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8280ca8b0 con 0x555ad1c0d220 2023-10-16T10:00:40.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1be8010 0x555ad1c0aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:40.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa827fff700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c0b610 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa814058040 tx=0x7fa81401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf857fa2ad10971 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa8077fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:00:40.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa8077fe700 1 -- 172.21.15.181:0/3812384010 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814089070 con 0x555ad1c0d220 2023-10-16T10:00:40.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa8077fe700 1 -- 172.21.15.181:0/3812384010 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fa81406f030 con 0x555ad1c0d220 2023-10-16T10:00:40.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ad1bdd5b0 con 0x555ad1c0d220 2023-10-16T10:00:40.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa8077fe700 1 -- 172.21.15.181:0/3812384010 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8140586a0 con 0x555ad1c0d220 2023-10-16T10:00:40.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 10 client.7629 fetch_fsmap learned FSMap version 49 2023-10-16T10:00:40.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x555ad1967d90 con 0x555ad1c0d220 2023-10-16T10:00:40.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.908+0000 7fa8077fe700 1 -- 172.21.15.181:0/3812384010 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81407c080 con 0x555ad1c0d220 2023-10-16T10:00:40.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa8077fe700 1 -- 172.21.15.181:0/3812384010 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa814057830 con 0x555ad1c0d220 2023-10-16T10:00:40.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa83b2ffb80 10 client.7629 fetch_fsmap finished waiting for FSMap version 49 2023-10-16T10:00:40.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa83b2ffb80 10 client.7629 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:00:40.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa83b2ffb80 20 client.7629 populate_metadata read hostname 'smithi181' 2023-10-16T10:00:40.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10890 0x555ad1c10c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa83b2ffb80 4 client.7629 mds_command: new command op to 7491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:00:40.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 --> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ad1b35e00 con 0x555ad1c10890 2023-10-16T10:00:40.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10890 0x555ad1c10c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa8077fe700 10 client.7629 ms_handle_connect on v2:172.21.15.181:6835/1158962128 2023-10-16T10:00:40.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.909+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10890 0x555ad1c10c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c81f5421e1406bf server_cookie=b35fbcc8d74a3249 in_seq=0 out_seq=0 2023-10-16T10:00:40.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa8077fe700 1 -- 172.21.15.181:0/3812384010 <== mds.0 v2:172.21.15.181:6835/1158962128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555ad1b35e00 con 0x555ad1c10890 2023-10-16T10:00:40.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa8077fe700 10 client.7629 handle_command_reply: tid=0 2023-10-16T10:00:40.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 client.7629 shutdown 2023-10-16T10:00:40.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 msgr2=0x555ad1c0b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c0b610 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa814058040 tx=0x7fa81401f000 comp rx=0 tx=0).stop 2023-10-16T10:00:40.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 shutdown_connections 2023-10-16T10:00:40.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10890 0x555ad1c10c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1c12630 0x555ad1c2fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1be8010 0x555ad1c0aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa8077fe700 0 client.7629 ms_handle_reset on v2:172.21.15.181:6835/1158962128 2023-10-16T10:00:40.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3812384010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c0b610 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 >> 172.21.15.181:0/3812384010 conn(0x555ad1b17d80 msgr2=0x555ad1be8480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 shutdown_connections 2023-10-16T10:00:40.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3812384010 wait complete. 2023-10-16T10:00:40.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.910+0000 7fa83b2ffb80 20 client.7629 trim_cache size 0 max 16384 2023-10-16T10:00:40.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 Processor -- start 2023-10-16T10:00:40.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 -- start start 2023-10-16T10:00:40.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1c48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c0b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad19bd8d0 con 0x555ad1c12630 2023-10-16T10:00:40.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad1bc74c0 con 0x555ad1c0d220 2023-10-16T10:00:40.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ad1c15580 con 0x555ad1be8010 2023-10-16T10:00:40.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa82d6b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa82d6b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c181c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44620/0 (socket says 172.21.15.181:44620) 2023-10-16T10:00:40.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.928+0000 7fa82d6b2700 1 -- 172.21.15.181:0/499627225 learned_addr learned my addr 172.21.15.181:0/499627225 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:40.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1c48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa827fff700 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c0b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa82d6b2700 1 -- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 msgr2=0x555ad1c48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1c48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa82d6b2700 1 -- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 msgr2=0x555ad1c0b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c0b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa82d6b2700 1 -- 172.21.15.181:0/499627225 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ad1950c10 con 0x555ad1c12630 2023-10-16T10:00:40.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c181c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa820007110 tx=0x7fa820007140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb4146c68843a97 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa812ffd700 1 -- 172.21.15.181:0/499627225 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820014430 con 0x555ad1c12630 2023-10-16T10:00:40.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa812ffd700 1 -- 172.21.15.181:0/499627225 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82000d040 con 0x555ad1c12630 2023-10-16T10:00:40.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.929+0000 7fa812ffd700 1 -- 172.21.15.181:0/499627225 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200147d0 con 0x555ad1c12630 2023-10-16T10:00:40.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/499627225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 msgr2=0x555ad1c181c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c181c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa820007110 tx=0x7fa820007140 comp rx=0 tx=0).stop 2023-10-16T10:00:40.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/499627225 shutdown_connections 2023-10-16T10:00:40.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1c48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c12630 0x555ad1c181c0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/499627225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c0d220 0x555ad1c0b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/499627225 >> 172.21.15.181:0/499627225 conn(0x555ad1b17d80 msgr2=0x555ad1be4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/499627225 shutdown_connections 2023-10-16T10:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/499627225 wait complete. 2023-10-16T10:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 -- start start 2023-10-16T10:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.930+0000 7fa83b2ffb80 1 Processor -- start 2023-10-16T10:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa83b2ffb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c12630 0x555ad1c868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad1c8d690 con 0x555ad1c0d220 2023-10-16T10:00:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ad1c12370 con 0x555ad1c12630 2023-10-16T10:00:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa83b2ffb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ad1be9050 con 0x555ad1be8010 2023-10-16T10:00:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa83b2ffb80 10 client.0 _ll_register_callbacks cb 0x555ad19367b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:00:40.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa827fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c863b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44624/0 (socket says 172.21.15.181:44624) 2023-10-16T10:00:40.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa827fff700 1 -- 172.21.15.181:0/3843788350 learned_addr learned my addr 172.21.15.181:0/3843788350 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:40.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.931+0000 7fa82d6b2700 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c12630 0x555ad1c868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa827fff700 1 -- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 msgr2=0x555ad1be87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa827fff700 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa827fff700 1 -- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c12630 msgr2=0x555ad1c868f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa827fff700 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c12630 0x555ad1c868f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa827fff700 1 -- 172.21.15.181:0/3843788350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa820001f70 con 0x555ad1c0d220 2023-10-16T10:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa827fff700 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c863b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa814054840 tx=0x7fa8140547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0db7e5df1438c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa8077fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa8077fe700 1 -- 172.21.15.181:0/3843788350 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814058a40 con 0x555ad1c0d220 2023-10-16T10:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa8077fe700 1 -- 172.21.15.181:0/3843788350 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fa81406f040 con 0x555ad1c0d220 2023-10-16T10:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa814060640 con 0x555ad1c0d220 2023-10-16T10:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa8077fe700 1 -- 172.21.15.181:0/3843788350 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8140601b0 con 0x555ad1c0d220 2023-10-16T10:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 10 client.7635 fetch_fsmap learned FSMap version 49 2023-10-16T10:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x555ad198e200 con 0x555ad1c0d220 2023-10-16T10:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa8077fe700 1 -- 172.21.15.181:0/3843788350 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814060640 con 0x555ad1c0d220 2023-10-16T10:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa8077fe700 1 -- 172.21.15.181:0/3843788350 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa81405e240 con 0x555ad1c0d220 2023-10-16T10:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 10 client.7635 fetch_fsmap finished waiting for FSMap version 49 2023-10-16T10:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 10 client.7635 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 20 client.7635 populate_metadata read hostname 'smithi181' 2023-10-16T10:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10cc0 0x555ad1c110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 4 client.7635 mds_command: new command op to 7491 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 --> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555ad1b35b20 con 0x555ad1c10cc0 2023-10-16T10:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.932+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10cc0 0x555ad1c110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa8077fe700 10 client.7635 ms_handle_connect on v2:172.21.15.181:6835/1158962128 2023-10-16T10:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa82ceb1700 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10cc0 0x555ad1c110a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9cd3160359879c4 server_cookie=3d14f876e335795a in_seq=0 out_seq=0 2023-10-16T10:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa8077fe700 1 -- 172.21.15.181:0/3843788350 <== mds.0 v2:172.21.15.181:6835/1158962128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555ad1b35b20 con 0x555ad1c10cc0 2023-10-16T10:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa8077fe700 10 client.7635 handle_command_reply: tid=0 2023-10-16T10:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 client.7635 shutdown 2023-10-16T10:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 msgr2=0x555ad1c863b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c863b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa814054840 tx=0x7fa8140547b0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 shutdown_connections 2023-10-16T10:00:40.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:6835/1158962128,v1:172.21.15.181:6837/1158962128] conn(0x555ad1c10cc0 0x555ad1c110a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555ad1be8010 0x555ad1be87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555ad1c12630 0x555ad1c868f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 --2- 172.21.15.181:0/3843788350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ad1c0d220 0x555ad1c863b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa8077fe700 0 client.7635 ms_handle_reset on v2:172.21.15.181:6835/1158962128 2023-10-16T10:00:40.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 >> 172.21.15.181:0/3843788350 conn(0x555ad1b17d80 msgr2=0x555ad1c181c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 shutdown_connections 2023-10-16T10:00:40.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 1 -- 172.21.15.181:0/3843788350 wait complete. 2023-10-16T10:00:40.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.933+0000 7fa83b2ffb80 20 client.7635 trim_cache size 0 max 16384 2023-10-16T10:00:40.959 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:00:40.959 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T10:00:40.959 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T10:00:40.959 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 7491, 2023-10-16T10:00:40.959 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T10:00:40.960 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T10:00:40.960 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T10:00:40.960 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9139077760000003, 2023-10-16T10:00:40.960 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 49, 2023-10-16T10:00:40.961 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 103, 2023-10-16T10:00:40.961 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 103, 2023-10-16T10:00:40.961 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.858539749 2023-10-16T10:00:40.961 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa80c06b020 msgr2=0x7fa80c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa80c06b020 0x7fa80c06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8280d1b40 tx=0x7fa81c008030 comp rx=0 tx=0).stop 2023-10-16T10:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8280cc4c0 msgr2=0x7fa82815ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8280cc4c0 0x7fa82815ebf0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fa81400c7f0 tx=0x7fa81400cbb0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 shutdown_connections 2023-10-16T10:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa80c06b020 0x7fa80c06d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8280ce300 0x7fa8280c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa82806d210 0x7fa8281f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 --2- 172.21.15.181:0/2679425037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8280cc4c0 0x7fa82815ebf0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 >> 172.21.15.181:0/2679425037 conn(0x7fa82809a9a0 msgr2=0x7fa82806edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 shutdown_connections 2023-10-16T10:00:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:40.934+0000 7fa82f115700 1 -- 172.21.15.181:0/2679425037 wait complete. 2023-10-16T10:00:40.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:00:41.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.085+0000 7f61454f1700 1 Processor -- start 2023-10-16T10:00:41.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.085+0000 7f61454f1700 1 -- start start 2023-10-16T10:00:41.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.086+0000 7f61454f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d2390 0x7f61400cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:41.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.086+0000 7f61454f1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61400cc470 con 0x7f61400d0180 2023-10-16T10:00:41.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.086+0000 7f61454f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614006ca30 con 0x7f61400ce7f0 2023-10-16T10:00:41.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.086+0000 7f61454f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61400d0610 con 0x7f61400d2390 2023-10-16T10:00:41.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.086+0000 7f613f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f61400d0180 0x7f61400cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55712/0 (socket says 172.21.15.181:55712) 2023-10-16T10:00:41.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.086+0000 7f613f7fe700 1 -- 172.21.15.181:0/1708032362 learned_addr learned my addr 172.21.15.181:0/1708032362 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:41.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.086+0000 7f613e7fc700 1 --2- 172.21.15.181:0/1708032362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d2390 0x7f61400cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:41.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613dffb700 1 -- 172.21.15.181:0/1708032362 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812667189 0 0) 0x7f61400cc470 con 0x7f61400d0180 2023-10-16T10:00:41.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613dffb700 1 -- 172.21.15.181:0/1708032362 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61280035b0 con 0x7f61400d0180 2023-10-16T10:00:41.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613dffb700 1 -- 172.21.15.181:0/1708032362 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373620082 0 0) 0x7f614006ca30 con 0x7f61400ce7f0 2023-10-16T10:00:41.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613dffb700 1 -- 172.21.15.181:0/1708032362 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61400cc470 con 0x7f61400ce7f0 2023-10-16T10:00:41.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613e7fc700 1 -- 172.21.15.181:0/1708032362 >> v1:172.21.15.190:6789/0 conn(0x7f61400ce7f0 legacy=0x7f614006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:41.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613e7fc700 1 -- 172.21.15.181:0/1708032362 >> v1:172.21.15.181:6789/0 conn(0x7f61400d0180 legacy=0x7f61400cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:41.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613e7fc700 1 -- 172.21.15.181:0/1708032362 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614006cd10 con 0x7f61400d2390 2023-10-16T10:00:41.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613e7fc700 1 --2- 172.21.15.181:0/1708032362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d2390 0x7f61400cfc40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6130009870 tx=0x7f6130009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cabc551079fb68b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:41.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613dffb700 1 -- 172.21.15.181:0/1708032362 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130011830 con 0x7f61400d2390 2023-10-16T10:00:41.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613dffb700 1 -- 172.21.15.181:0/1708032362 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6130011990 con 0x7f61400d2390 2023-10-16T10:00:41.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.087+0000 7f613dffb700 1 -- 172.21.15.181:0/1708032362 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130011d00 con 0x7f61400d2390 2023-10-16T10:00:41.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.088+0000 7f61454f1700 1 -- 172.21.15.181:0/1708032362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d2390 msgr2=0x7f61400cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:41.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.088+0000 7f61454f1700 1 --2- 172.21.15.181:0/1708032362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d2390 0x7f61400cfc40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6130009870 tx=0x7f6130009b80 comp rx=0 tx=0).stop 2023-10-16T10:00:41.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.088+0000 7f61454f1700 1 -- 172.21.15.181:0/1708032362 shutdown_connections 2023-10-16T10:00:41.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.088+0000 7f61454f1700 1 --2- 172.21.15.181:0/1708032362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d2390 0x7f61400cfc40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.088+0000 7f61454f1700 1 -- 172.21.15.181:0/1708032362 >> 172.21.15.181:0/1708032362 conn(0x7f614009a9a0 msgr2=0x7f61400d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:41.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.088+0000 7f61454f1700 1 -- 172.21.15.181:0/1708032362 shutdown_connections 2023-10-16T10:00:41.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.088+0000 7f61454f1700 1 -- 172.21.15.181:0/1708032362 wait complete. 2023-10-16T10:00:41.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.089+0000 7f61454f1700 1 Processor -- start 2023-10-16T10:00:41.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.089+0000 7f61454f1700 1 -- start start 2023-10-16T10:00:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61454f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61400ce7f0 0x7f6140156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61454f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d0180 0x7f6140159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61454f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61400d2390 0x7f61400ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61454f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614015ad10 con 0x7f61400d2390 2023-10-16T10:00:41.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61454f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614015aa30 con 0x7f61400ce7f0 2023-10-16T10:00:41.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61454f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61401f4360 con 0x7f61400d0180 2023-10-16T10:00:41.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61400d2390 0x7f61400ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:41.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61400d2390 0x7f61400ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44640/0 (socket says 172.21.15.181:44640) 2023-10-16T10:00:41.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 -- 172.21.15.181:0/1401233957 learned_addr learned my addr 172.21.15.181:0/1401233957 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:41.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613e7fc700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d0180 0x7f6140159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:41.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613effd700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61400ce7f0 0x7f6140156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:41.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 -- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d0180 msgr2=0x7f6140159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:41.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d0180 0x7f6140159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 -- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61400ce7f0 msgr2=0x7f6140156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:41.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61400ce7f0 0x7f6140156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 -- 172.21.15.181:0/1401233957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300094a0 con 0x7f61400d2390 2023-10-16T10:00:41.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f613f7fe700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61400d2390 0x7f61400ca6a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f613800caf0 tx=0x7f613800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:41.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61277fe700 1 -- 172.21.15.181:0/1401233957 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800aad0 con 0x7f61400d2390 2023-10-16T10:00:41.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61277fe700 1 -- 172.21.15.181:0/1401233957 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613800ac30 con 0x7f61400d2390 2023-10-16T10:00:41.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61277fe700 1 -- 172.21.15.181:0/1401233957 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138012400 con 0x7f61400d2390 2023-10-16T10:00:41.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.090+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61401f4640 con 0x7f61400d2390 2023-10-16T10:00:41.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.091+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6140157120 con 0x7f61400d2390 2023-10-16T10:00:41.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.091+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61401f4b60 con 0x7f61400d2390 2023-10-16T10:00:41.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.092+0000 7f61277fe700 1 -- 172.21.15.181:0/1401233957 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f613801b3b0 con 0x7f61400d2390 2023-10-16T10:00:41.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.092+0000 7f61277fe700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f612c06b020 0x7f612c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:41.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.092+0000 7f61277fe700 1 -- 172.21.15.181:0/1401233957 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f6138089610 con 0x7f61400d2390 2023-10-16T10:00:41.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.092+0000 7f613effd700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f612c06b020 0x7f612c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:41.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.093+0000 7f613effd700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f612c06b020 0x7f612c06d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61400cb7e0 tx=0x7f6134009040 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:41.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.097+0000 7f61277fe700 1 -- 172.21.15.181:0/1401233957 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61401f4b60 con 0x7f61400d2390 2023-10-16T10:00:41.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.335+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61401f4b60 con 0x7f61400d2390 2023-10-16T10:00:41.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.336+0000 7f61277fe700 1 -- 172.21.15.181:0/1401233957 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f61401f4b60 con 0x7f61400d2390 2023-10-16T10:00:41.338 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:00:41.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f612c06b020 msgr2=0x7f612c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:41.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f612c06b020 0x7f612c06d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61400cb7e0 tx=0x7f6134009040 comp rx=0 tx=0).stop 2023-10-16T10:00:41.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61400d2390 msgr2=0x7f61400ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:41.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61400d2390 0x7f61400ca6a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f613800caf0 tx=0x7f613800ceb0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 shutdown_connections 2023-10-16T10:00:41.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f612c06b020 0x7f612c06d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61400d2390 0x7f61400ca6a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61400ce7f0 0x7f6140156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 --2- 172.21.15.181:0/1401233957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61400d0180 0x7f6140159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:41.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 >> 172.21.15.181:0/1401233957 conn(0x7f614009a9a0 msgr2=0x7f614006f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:41.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 shutdown_connections 2023-10-16T10:00:41.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:41.338+0000 7f61454f1700 1 -- 172.21.15.181:0/1401233957 wait complete. 2023-10-16T10:00:41.354 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:00:41.355 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:00:41.355 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T10:00:41.356 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:00:41.356 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:41.357 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:00:41.357 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:00:41.357 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:00:41.358 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:00:41.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.359 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T10:00:41.376 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:00:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:00:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:00:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:41.377 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T10:00:41.378 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:41.378 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:00:41.378 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:41.378 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:00:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:41.379 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:00:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:41.379 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:00:41.379 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:00:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41428sec preferred_lft 41428sec 2023-10-16T10:00:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T10:00:41.380 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:41.380 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:00:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:00:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:00:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T10:00:41.381 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:41.381 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:00:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:00:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T10:00:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:41.382 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:00:41.382 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:00:41.383 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T10:00:41.383 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:00:41.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.384 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:00:41.441 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T10:00:41.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.443 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T10:00:41.502 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T10:00:41.502 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:00:41.502 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:00:41.502 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:00:41.503 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:00:41.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.503 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T10:00:41.504 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T10:00:41.504 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:00:41.505 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:00:41.505 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:00:41.505 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T10:00:41.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.607 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T10:00:41.624 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:00:41.624 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:00:41.625 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:00:41.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.625 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T10:00:41.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:00:41.691 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:41.748 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:00:41.748 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:00:41.749 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:41.804 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T10:00:41.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.806 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:41.889 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:41.890 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:00:41.890 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:00:41.898 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:41.898 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:00:41.898 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:00:41.899 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:00:41.899 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:00:41.973 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:00:41.973 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:00:41.973 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:00:41.973 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:00:41.974 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:00:41.974 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T10:00:41.974 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T10:00:41.974 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T10:00:41.975 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:00:41.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:41.976 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:00:42.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:00:42.003 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:00:42.086 INFO:teuthology.orchestra.run.smithi181.stdout:client6964 2023-10-16T10:00:42.087 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:00:42.087 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:00:42.145 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:00:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.304+0000 7f5725dd6700 1 Processor -- start 2023-10-16T10:00:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.304+0000 7f5725dd6700 1 -- start start 2023-10-16T10:00:42.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.305+0000 7f5725dd6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f572015da60 0x7f57200665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.305+0000 7f5725dd6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5720066ae0 con 0x7f57200b22a0 2023-10-16T10:00:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.305+0000 7f5725dd6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572015e240 con 0x7f57200cf510 2023-10-16T10:00:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.305+0000 7f5725dd6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57200c3660 con 0x7f572015da60 2023-10-16T10:00:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.305+0000 7f571effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f57200b22a0 0x7f57200cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55726/0 (socket says 172.21.15.181:55726) 2023-10-16T10:00:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.305+0000 7f571effd700 1 -- 172.21.15.181:0/1177029017 learned_addr learned my addr 172.21.15.181:0/1177029017 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745460513 0 0) 0x7f5720066ae0 con 0x7f57200b22a0 2023-10-16T10:00:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57080035b0 con 0x7f57200b22a0 2023-10-16T10:00:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571f7fe700 1 --2- 172.21.15.181:0/1177029017 >> v2:172.21.15.190:3300/0 conn(0x7f572015da60 0x7f57200665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4271281926 0 0) 0x7f57080035b0 con 0x7f57200b22a0 2023-10-16T10:00:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 >> v2:172.21.15.190:3300/0 conn(0x7f572015da60 msgr2=0x7f57200665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 --2- 172.21.15.181:0/1177029017 >> v2:172.21.15.190:3300/0 conn(0x7f572015da60 0x7f57200665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 >> v1:172.21.15.190:6789/0 conn(0x7f57200cf510 legacy=0x7f57201615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572015e520 con 0x7f57200b22a0 2023-10-16T10:00:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5710001670 con 0x7f57200b22a0 2023-10-16T10:00:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 >> v1:172.21.15.181:6789/0 conn(0x7f57200b22a0 legacy=0x7f57200cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5708003a00 0x7f5708003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5708004850 0x7f5708004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708005330 0x7f570800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c3660 con 0x7f5708004850 2023-10-16T10:00:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570800c200 con 0x7f5708003a00 2023-10-16T10:00:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570800c3c0 con 0x7f5708005330 2023-10-16T10:00:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571f7fe700 1 --2- 172.21.15.181:0/1177029017 >> v2:172.21.15.190:3300/0 conn(0x7f572015da60 0x7f57200665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:00:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571effd700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5708004850 0x7f5708004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571f7fe700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5708003a00 0x7f5708003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571ffff700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708005330 0x7f570800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571effd700 1 -- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708005330 msgr2=0x7f570800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:42.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571effd700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708005330 0x7f570800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571effd700 1 -- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5708003a00 msgr2=0x7f5708003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:42.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571effd700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5708003a00 0x7f5708003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571ffff700 1 -- 172.21.15.181:0/1177029017 reap_dead start 2023-10-16T10:00:42.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571effd700 1 -- 172.21.15.181:0/1177029017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57080035b0 con 0x7f5708004850 2023-10-16T10:00:42.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.306+0000 7f571effd700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5708004850 0x7f5708004cd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f571000c210 tx=0x7f5708005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7018919efb991e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710007c10 con 0x7f5708004850 2023-10-16T10:00:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571000cdd0 con 0x7f5708004850 2023-10-16T10:00:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f571e7fc700 1 -- 172.21.15.181:0/1177029017 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710005ba0 con 0x7f5708004850 2023-10-16T10:00:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f5725dd6700 1 -- 172.21.15.181:0/1177029017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5708004850 msgr2=0x7f5708004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f5725dd6700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5708004850 0x7f5708004cd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f571000c210 tx=0x7f5708005880 comp rx=0 tx=0).stop 2023-10-16T10:00:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f5725dd6700 1 -- 172.21.15.181:0/1177029017 shutdown_connections 2023-10-16T10:00:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f5725dd6700 1 --2- 172.21.15.181:0/1177029017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5708004850 0x7f5708004cd0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f5725dd6700 1 -- 172.21.15.181:0/1177029017 >> 172.21.15.181:0/1177029017 conn(0x7f57200b1780 msgr2=0x7f57200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f5725dd6700 1 -- 172.21.15.181:0/1177029017 shutdown_connections 2023-10-16T10:00:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.307+0000 7f5725dd6700 1 -- 172.21.15.181:0/1177029017 wait complete. 2023-10-16T10:00:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.308+0000 7f5725dd6700 1 Processor -- start 2023-10-16T10:00:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.308+0000 7f5725dd6700 1 -- start start 2023-10-16T10:00:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f5725dd6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708004850 0x7f57200cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f5725dd6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f572015f0f0 0x7f57200b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f5725dd6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f572015b740 0x7f572015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f5725dd6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572015f580 con 0x7f572015f0f0 2023-10-16T10:00:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f5725dd6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572015fce0 con 0x7f572015b740 2023-10-16T10:00:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f5725dd6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572015ffc0 con 0x7f5708004850 2023-10-16T10:00:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f571effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f572015f0f0 0x7f57200b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f571effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f572015f0f0 0x7f57200b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44662/0 (socket says 172.21.15.181:44662) 2023-10-16T10:00:42.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f571effd700 1 -- 172.21.15.181:0/2353236576 learned_addr learned my addr 172.21.15.181:0/2353236576 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:42.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f571f7fe700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708004850 0x7f57200cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.309+0000 7f571ffff700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f572015b740 0x7f572015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f571effd700 1 -- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708004850 msgr2=0x7f57200cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f571effd700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708004850 0x7f57200cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f571effd700 1 -- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f572015b740 msgr2=0x7f572015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f571effd700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f572015b740 0x7f572015bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f571effd700 1 -- 172.21.15.181:0/2353236576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57201602a0 con 0x7f572015f0f0 2023-10-16T10:00:42.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f571effd700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f572015f0f0 0x7f57200b3fd0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f570800c1d0 tx=0x7f5710001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:42.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f5707fff700 1 -- 172.21.15.181:0/2353236576 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710004c80 con 0x7f572015f0f0 2023-10-16T10:00:42.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f5707fff700 1 -- 172.21.15.181:0/2353236576 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5710004de0 con 0x7f572015f0f0 2023-10-16T10:00:42.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572015c0d0 con 0x7f572015f0f0 2023-10-16T10:00:42.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f5707fff700 1 -- 172.21.15.181:0/2353236576 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710007310 con 0x7f572015f0f0 2023-10-16T10:00:42.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.310+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572015caa0 con 0x7f572015f0f0 2023-10-16T10:00:42.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.311+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5720160490 con 0x7f572015f0f0 2023-10-16T10:00:42.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.311+0000 7f5707fff700 1 -- 172.21.15.181:0/2353236576 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5710007470 con 0x7f572015f0f0 2023-10-16T10:00:42.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.312+0000 7f5707fff700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f570c06b020 0x7f570c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.312+0000 7f5707fff700 1 -- 172.21.15.181:0/2353236576 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f571008a260 con 0x7f572015f0f0 2023-10-16T10:00:42.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.312+0000 7f571f7fe700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f570c06b020 0x7f570c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.312+0000 7f571f7fe700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f570c06b020 0x7f570c06d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5718003fd0 tx=0x7f5718003d00 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:42.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.316+0000 7f5707fff700 1 -- 172.21.15.181:0/2353236576 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f57100550e0 con 0x7f572015f0f0 2023-10-16T10:00:42.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.535+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 --> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f572015fb70 con 0x7f570c06b020 2023-10-16T10:00:42.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.537+0000 7f5707fff700 1 -- 172.21.15.181:0/2353236576 <== mgr.7254 v2:172.21.15.181:6832/3229300335 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f572015fb70 con 0x7f570c06b020 2023-10-16T10:00:42.538 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:00:42.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:00:42.539 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T10:00:42.539 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:00:42.539 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:00:42.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f570c06b020 msgr2=0x7f570c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f570c06b020 0x7f570c06d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5718003fd0 tx=0x7f5718003d00 comp rx=0 tx=0).stop 2023-10-16T10:00:42.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f572015f0f0 msgr2=0x7f57200b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f572015f0f0 0x7f57200b3fd0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f570800c1d0 tx=0x7f5710001a00 comp rx=0 tx=0).stop 2023-10-16T10:00:42.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 shutdown_connections 2023-10-16T10:00:42.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f570c06b020 0x7f570c06d4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5708004850 0x7f57200cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f572015b740 0x7f572015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 --2- 172.21.15.181:0/2353236576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f572015f0f0 0x7f57200b3fd0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 >> 172.21.15.181:0/2353236576 conn(0x7f57200b1780 msgr2=0x7f572006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:42.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 shutdown_connections 2023-10-16T10:00:42.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.539+0000 7f5725dd6700 1 -- 172.21.15.181:0/2353236576 wait complete. 2023-10-16T10:00:42.553 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:00:42.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.673+0000 7feb63efd700 1 Processor -- start 2023-10-16T10:00:42.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.673+0000 7feb63efd700 1 -- start start 2023-10-16T10:00:42.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb63efd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb5c15da70 0x7feb5c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb63efd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb5c0ce6a0 0x7feb5c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb63efd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c0cef00 con 0x7feb5c072760 2023-10-16T10:00:42.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb63efd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c072b40 con 0x7feb5c0ce6a0 2023-10-16T10:00:42.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb63efd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c0cec20 con 0x7feb5c15da70 2023-10-16T10:00:42.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb5c0ce6a0 0x7feb5c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb5c0ce6a0 0x7feb5c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44670/0 (socket says 172.21.15.181:44670) 2023-10-16T10:00:42.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 -- 172.21.15.181:0/3380145335 learned_addr learned my addr 172.21.15.181:0/3380145335 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:42.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61c99700 1 --2- 172.21.15.181:0/3380145335 >> v2:172.21.15.190:3300/0 conn(0x7feb5c15da70 0x7feb5c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 -- 172.21.15.181:0/3380145335 >> v2:172.21.15.190:3300/0 conn(0x7feb5c15da70 msgr2=0x7feb5c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:42.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 --2- 172.21.15.181:0/3380145335 >> v2:172.21.15.190:3300/0 conn(0x7feb5c15da70 0x7feb5c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 -- 172.21.15.181:0/3380145335 >> v1:172.21.15.190:6789/0 conn(0x7feb5c072760 legacy=0x7feb5c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:42.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 -- 172.21.15.181:0/3380145335 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c071c80 con 0x7feb5c0ce6a0 2023-10-16T10:00:42.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61c99700 1 --2- 172.21.15.181:0/3380145335 >> v2:172.21.15.190:3300/0 conn(0x7feb5c15da70 0x7feb5c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:00:42.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb61498700 1 --2- 172.21.15.181:0/3380145335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c0ce6a0 0x7feb5c06a8a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feb5c0675e0 tx=0x7feb4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e6001c3a8b0851 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:42.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.674+0000 7feb60c97700 1 -- 172.21.15.181:0/3380145335 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c004230 con 0x7feb5c0ce6a0 2023-10-16T10:00:42.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb60c97700 1 -- 172.21.15.181:0/3380145335 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c004390 con 0x7feb5c0ce6a0 2023-10-16T10:00:42.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb60c97700 1 -- 172.21.15.181:0/3380145335 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0036a0 con 0x7feb5c0ce6a0 2023-10-16T10:00:42.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 -- 172.21.15.181:0/3380145335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c0ce6a0 msgr2=0x7feb5c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 --2- 172.21.15.181:0/3380145335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c0ce6a0 0x7feb5c06a8a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feb5c0675e0 tx=0x7feb4c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 -- 172.21.15.181:0/3380145335 shutdown_connections 2023-10-16T10:00:42.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 --2- 172.21.15.181:0/3380145335 >> v2:172.21.15.190:3300/0 conn(0x7feb5c15da70 0x7feb5c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 --2- 172.21.15.181:0/3380145335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c0ce6a0 0x7feb5c06a8a0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 -- 172.21.15.181:0/3380145335 >> 172.21.15.181:0/3380145335 conn(0x7feb5c0cd010 msgr2=0x7feb5c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:42.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 -- 172.21.15.181:0/3380145335 shutdown_connections 2023-10-16T10:00:42.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.675+0000 7feb63efd700 1 -- 172.21.15.181:0/3380145335 wait complete. 2023-10-16T10:00:42.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.676+0000 7feb63efd700 1 Processor -- start 2023-10-16T10:00:42.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.676+0000 7feb63efd700 1 -- start start 2023-10-16T10:00:42.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb63efd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c072760 0x7feb5c0ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb63efd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb5c0ce6a0 0x7feb5c154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb63efd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c15da70 0x7feb5c0af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb63efd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c0ae580 con 0x7feb5c15da70 2023-10-16T10:00:42.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb63efd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c154c60 con 0x7feb5c072760 2023-10-16T10:00:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb63efd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c0af6f0 con 0x7feb5c0ce6a0 2023-10-16T10:00:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c15da70 0x7feb5c0af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c15da70 0x7feb5c0af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44678/0 (socket says 172.21.15.181:44678) 2023-10-16T10:00:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 -- 172.21.15.181:0/3975957869 learned_addr learned my addr 172.21.15.181:0/3975957869 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb61c99700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c072760 0x7feb5c0ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb61498700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb5c0ce6a0 0x7feb5c154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 -- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb5c0ce6a0 msgr2=0x7feb5c154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb5c0ce6a0 0x7feb5c154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 -- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c072760 msgr2=0x7feb5c0ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c072760 0x7feb5c0ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.677+0000 7feb6249a700 1 -- 172.21.15.181:0/3975957869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c009550 con 0x7feb5c15da70 2023-10-16T10:00:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.678+0000 7feb6249a700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c15da70 0x7feb5c0af1b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b470 tx=0x7feb5400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.678+0000 7feb527fc700 1 -- 172.21.15.181:0/3975957869 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54011630 con 0x7feb5c15da70 2023-10-16T10:00:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.678+0000 7feb527fc700 1 -- 172.21.15.181:0/3975957869 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb54011790 con 0x7feb5c15da70 2023-10-16T10:00:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.678+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c0ad900 con 0x7feb5c15da70 2023-10-16T10:00:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.678+0000 7feb527fc700 1 -- 172.21.15.181:0/3975957869 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54011b40 con 0x7feb5c15da70 2023-10-16T10:00:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.678+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c0adec0 con 0x7feb5c15da70 2023-10-16T10:00:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.679+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c0cc100 con 0x7feb5c15da70 2023-10-16T10:00:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.679+0000 7feb527fc700 1 -- 172.21.15.181:0/3975957869 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 52) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb54011ca0 con 0x7feb5c15da70 2023-10-16T10:00:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.679+0000 7feb527fc700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feb4006b020 0x7feb4006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.680+0000 7feb527fc700 1 -- 172.21.15.181:0/3975957869 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7feb540893b0 con 0x7feb5c15da70 2023-10-16T10:00:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.680+0000 7feb61c99700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feb4006b020 0x7feb4006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.680+0000 7feb61c99700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feb4006b020 0x7feb4006d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb5800d620 tx=0x7feb58007810 comp rx=0 tx=0).ready entity=mgr.7254 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.684+0000 7feb527fc700 1 -- 172.21.15.181:0/3975957869 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feb54054230 con 0x7feb5c15da70 2023-10-16T10:00:42.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:42.931+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7feb5c06f6c0 con 0x7feb5c15da70 2023-10-16T10:00:43.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.063+0000 7feb527fc700 1 -- 172.21.15.181:0/3975957869 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v53) v1 ==== 92+0+0 (secure 0 0 0) 0x7feb54057850 con 0x7feb5c15da70 2023-10-16T10:00:43.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.066+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feb4006b020 msgr2=0x7feb4006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.066+0000 7feb63efd700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feb4006b020 0x7feb4006d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb5800d620 tx=0x7feb58007810 comp rx=0 tx=0).stop 2023-10-16T10:00:43.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.066+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c15da70 msgr2=0x7feb5c0af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.066+0000 7feb63efd700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c15da70 0x7feb5c0af1b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b470 tx=0x7feb5400b780 comp rx=0 tx=0).stop 2023-10-16T10:00:43.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.066+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 shutdown_connections 2023-10-16T10:00:43.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.066+0000 7feb63efd700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7feb4006b020 0x7feb4006d4e0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb5800d620 tx=0x7feb58007810 comp rx=0 tx=0).stop 2023-10-16T10:00:43.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.067+0000 7feb63efd700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb5c15da70 0x7feb5c0af1b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.067+0000 7feb63efd700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c072760 0x7feb5c0ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.067+0000 7feb63efd700 1 --2- 172.21.15.181:0/3975957869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb5c0ce6a0 0x7feb5c154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.067+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 >> 172.21.15.181:0/3975957869 conn(0x7feb5c0cd010 msgr2=0x7feb5c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:43.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.067+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 shutdown_connections 2023-10-16T10:00:43.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.068+0000 7feb63efd700 1 -- 172.21.15.181:0/3975957869 wait complete. 2023-10-16T10:00:43.086 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:00:43.210 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:43.209+0000 7f6f783c6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:00:43.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.213+0000 7f1f09f8e700 1 Processor -- start 2023-10-16T10:00:43.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.213+0000 7f1f09f8e700 1 -- start start 2023-10-16T10:00:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.213+0000 7f1f09f8e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f0415f150 0x7f1f040cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.213+0000 7f1f09f8e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f040ce4f0 0x7f1f040b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f09f8e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f040b2ae0 con 0x7f1f040b14a0 2023-10-16T10:00:43.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f09f8e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f040ce970 con 0x7f1f040ce4f0 2023-10-16T10:00:43.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f09f8e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040b2dc0 con 0x7f1f0415f150 2023-10-16T10:00:43.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f03fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1f040b14a0 0x7f1f040b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55790/0 (socket says 172.21.15.181:55790) 2023-10-16T10:00:43.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f03fff700 1 -- 172.21.15.181:0/3855344225 learned_addr learned my addr 172.21.15.181:0/3855344225 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:43.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f037fe700 1 --2- 172.21.15.181:0/3855344225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f0415f150 0x7f1f040cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f02ffd700 1 --2- 172.21.15.181:0/3855344225 >> v2:172.21.15.190:3300/0 conn(0x7f1f040ce4f0 0x7f1f040b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f027fc700 1 -- 172.21.15.181:0/3855344225 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149075273 0 0) 0x7f1f040b2ae0 con 0x7f1f040b14a0 2023-10-16T10:00:43.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f027fc700 1 -- 172.21.15.181:0/3855344225 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eec0035b0 con 0x7f1f040b14a0 2023-10-16T10:00:43.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f037fe700 1 -- 172.21.15.181:0/3855344225 >> v2:172.21.15.190:3300/0 conn(0x7f1f040ce4f0 msgr2=0x7f1f040b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:43.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f037fe700 1 --2- 172.21.15.181:0/3855344225 >> v2:172.21.15.190:3300/0 conn(0x7f1f040ce4f0 0x7f1f040b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f037fe700 1 -- 172.21.15.181:0/3855344225 >> v1:172.21.15.181:6789/0 conn(0x7f1f040b14a0 legacy=0x7f1f040b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f037fe700 1 -- 172.21.15.181:0/3855344225 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f040cf9d0 con 0x7f1f0415f150 2023-10-16T10:00:43.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f037fe700 1 --2- 172.21.15.181:0/3855344225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f0415f150 0x7f1f040cdfb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009840 tx=0x7f1ef4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=641c8db90cc85169 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:43.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.214+0000 7f1f027fc700 1 -- 172.21.15.181:0/3855344225 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011830 con 0x7f1f0415f150 2023-10-16T10:00:43.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f027fc700 1 -- 172.21.15.181:0/3855344225 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4011990 con 0x7f1f0415f150 2023-10-16T10:00:43.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f027fc700 1 -- 172.21.15.181:0/3855344225 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011d20 con 0x7f1f0415f150 2023-10-16T10:00:43.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 -- 172.21.15.181:0/3855344225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f0415f150 msgr2=0x7f1f040cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/3855344225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f0415f150 0x7f1f040cdfb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009840 tx=0x7f1ef4009b50 comp rx=0 tx=0).stop 2023-10-16T10:00:43.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 -- 172.21.15.181:0/3855344225 shutdown_connections 2023-10-16T10:00:43.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/3855344225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f0415f150 0x7f1f040cdfb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/3855344225 >> v2:172.21.15.190:3300/0 conn(0x7f1f040ce4f0 0x7f1f040b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 -- 172.21.15.181:0/3855344225 >> 172.21.15.181:0/3855344225 conn(0x7f1f0409a930 msgr2=0x7f1f040d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:43.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 -- 172.21.15.181:0/3855344225 shutdown_connections 2023-10-16T10:00:43.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.215+0000 7f1f09f8e700 1 -- 172.21.15.181:0/3855344225 wait complete. 2023-10-16T10:00:43.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.216+0000 7f1f09f8e700 1 Processor -- start 2023-10-16T10:00:43.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.216+0000 7f1f09f8e700 1 -- start start 2023-10-16T10:00:43.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f09f8e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f040b14a0 0x7f1f040c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f09f8e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f040ce4f0 0x7f1f0415d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f09f8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0415f150 0x7f1f0415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f09f8e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040c55b0 con 0x7f1f0415f150 2023-10-16T10:00:43.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f09f8e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f04159710 con 0x7f1f040ce4f0 2023-10-16T10:00:43.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f09f8e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f041599f0 con 0x7f1f040b14a0 2023-10-16T10:00:43.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0415f150 0x7f1f0415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f02ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f040ce4f0 0x7f1f0415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f037fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f040b14a0 0x7f1f040c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f03fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0415f150 0x7f1f0415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44718/0 (socket says 172.21.15.181:44718) 2023-10-16T10:00:43.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f02ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f040ce4f0 0x7f1f0415d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:49766/0 (socket says 172.21.15.181:49766) 2023-10-16T10:00:43.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f037fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f040b14a0 0x7f1f040c67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:41740/0 (socket says 172.21.15.181:41740) 2023-10-16T10:00:43.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.217+0000 7f1f03fff700 1 -- 172.21.15.181:0/31457249 learned_addr learned my addr 172.21.15.181:0/31457249 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:43.231 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:43.217+0000 7fd3ea0ce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:00:43.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f03fff700 1 -- 172.21.15.181:0/31457249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f040b14a0 msgr2=0x7f1f040c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f03fff700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f040b14a0 0x7f1f040c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f03fff700 1 -- 172.21.15.181:0/31457249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f040ce4f0 msgr2=0x7f1f0415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f03fff700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f040ce4f0 0x7f1f0415d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f03fff700 1 -- 172.21.15.181:0/31457249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40094a0 con 0x7f1f0415f150 2023-10-16T10:00:43.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f037fe700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f040b14a0 0x7f1f040c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:00:43.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f03fff700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0415f150 0x7f1f0415d9a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00cb40 tx=0x7f1efc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1eebfff700 1 -- 172.21.15.181:0/31457249 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00a9e0 con 0x7f1f0415f150 2023-10-16T10:00:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1eebfff700 1 -- 172.21.15.181:0/31457249 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00ab40 con 0x7f1f0415f150 2023-10-16T10:00:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1eebfff700 1 -- 172.21.15.181:0/31457249 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc012400 con 0x7f1f0415f150 2023-10-16T10:00:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f04159cd0 con 0x7f1f0415f150 2023-10-16T10:00:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.218+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f040c6ce0 con 0x7f1f0415f150 2023-10-16T10:00:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.219+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f040c56f0 con 0x7f1f0415f150 2023-10-16T10:00:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.220+0000 7f1eebfff700 1 -- 172.21.15.181:0/31457249 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 53) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f1efc01b3b0 con 0x7f1f0415f150 2023-10-16T10:00:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.220+0000 7f1eebfff700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 0x7f1ef006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.220+0000 7f1f037fe700 1 -- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 msgr2=0x7f1ef006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.220+0000 7f1eebfff700 1 -- 172.21.15.181:0/31457249 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f1efc089550 con 0x7f1f0415f150 2023-10-16T10:00:43.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.220+0000 7f1f037fe700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 0x7f1ef006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:43.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.224+0000 7f1eebfff700 1 -- 172.21.15.181:0/31457249 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1f040c56f0 con 0x7f1f0415f150 2023-10-16T10:00:43.276 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:43.275+0000 7f6f783c6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:43.288 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:43.287+0000 7fd3ea0ce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:43.349 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:43.348+0000 7f6f783c6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:00:43.362 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:43.361+0000 7fd3ea0ce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:00:43.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.421+0000 7f1f037fe700 1 -- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 msgr2=0x7f1ef006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:43.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.421+0000 7f1f037fe700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 0x7f1ef006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:43.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.424+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1f040c56f0 con 0x7f1f0415f150 2023-10-16T10:00:43.426 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:43.425+0000 7f6f783c6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:00:43.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.427+0000 7f1eebfff700 1 -- 172.21.15.181:0/31457249 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1f040c56f0 con 0x7f1f0415f150 2023-10-16T10:00:43.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 msgr2=0x7f1ef006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:43.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 0x7f1ef006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0415f150 msgr2=0x7f1f0415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0415f150 0x7f1f0415d9a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00cb40 tx=0x7f1efc00ce50 comp rx=0 tx=0).stop 2023-10-16T10:00:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 shutdown_connections 2023-10-16T10:00:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f1ef006b070 0x7f1ef006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f040b14a0 0x7f1f040c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0415f150 0x7f1f0415d9a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 --2- 172.21.15.181:0/31457249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f040ce4f0 0x7f1f0415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 >> 172.21.15.181:0/31457249 conn(0x7f1f0409a930 msgr2=0x7f1f0415a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 shutdown_connections 2023-10-16T10:00:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.430+0000 7f1f09f8e700 1 -- 172.21.15.181:0/31457249 wait complete. 2023-10-16T10:00:43.444 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:43.442+0000 7fd3ea0ce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:00:43.445 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:00:43.567 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:43.566+0000 7f6f783c6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:00:43.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.570+0000 7f068c8a8700 1 Processor -- start 2023-10-16T10:00:43.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.570+0000 7f068c8a8700 1 -- start start 2023-10-16T10:00:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068c8a8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f068806afb0 0x7f06881615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068c8a8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06880cef00 con 0x7f068815da70 2023-10-16T10:00:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068c8a8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06880cec20 con 0x7f0688072760 2023-10-16T10:00:43.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068c8a8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0688071c80 con 0x7f068806afb0 2023-10-16T10:00:43.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f0686d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f068806afb0 0x7f06881615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f068815da70 0x7f06880ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55846/0 (socket says 172.21.15.181:55846) 2023-10-16T10:00:43.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068659c700 1 -- 172.21.15.181:0/2994238595 learned_addr learned my addr 172.21.15.181:0/2994238595 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:43.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f0686d9d700 1 -- 172.21.15.181:0/2994238595 >> v1:172.21.15.190:6789/0 conn(0x7f0688072760 legacy=0x7f068806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f0686d9d700 1 -- 172.21.15.181:0/2994238595 >> v1:172.21.15.181:6789/0 conn(0x7f068815da70 legacy=0x7f06880ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f0686d9d700 1 -- 172.21.15.181:0/2994238595 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0688071f60 con 0x7f068806afb0 2023-10-16T10:00:43.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068559a700 1 -- 172.21.15.181:0/2994238595 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107887767 0 0) 0x7f06880cef00 con 0x7f068815da70 2023-10-16T10:00:43.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f0686d9d700 1 --2- 172.21.15.181:0/2994238595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f068806afb0 0x7f06881615f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0680009d90 tx=0x7f068000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9099056f9a060098 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:43.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068559a700 1 -- 172.21.15.181:0/2994238595 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068000e070 con 0x7f068806afb0 2023-10-16T10:00:43.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068559a700 1 -- 172.21.15.181:0/2994238595 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068000bbd0 con 0x7f068806afb0 2023-10-16T10:00:43.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.571+0000 7f068559a700 1 -- 172.21.15.181:0/2994238595 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680012430 con 0x7f068806afb0 2023-10-16T10:00:43.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.572+0000 7f068c8a8700 1 -- 172.21.15.181:0/2994238595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f068806afb0 msgr2=0x7f06881615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.572+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2994238595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f068806afb0 0x7f06881615f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0680009d90 tx=0x7f068000b330 comp rx=0 tx=0).stop 2023-10-16T10:00:43.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.572+0000 7f068c8a8700 1 -- 172.21.15.181:0/2994238595 shutdown_connections 2023-10-16T10:00:43.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.572+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2994238595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f068806afb0 0x7f06881615f0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.572+0000 7f068c8a8700 1 -- 172.21.15.181:0/2994238595 >> 172.21.15.181:0/2994238595 conn(0x7f06880cd010 msgr2=0x7f06880cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:43.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.572+0000 7f068c8a8700 1 -- 172.21.15.181:0/2994238595 shutdown_connections 2023-10-16T10:00:43.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.572+0000 7f068c8a8700 1 -- 172.21.15.181:0/2994238595 wait complete. 2023-10-16T10:00:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.573+0000 7f068c8a8700 1 Processor -- start 2023-10-16T10:00:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068c8a8700 1 -- start start 2023-10-16T10:00:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068c8a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068806afb0 0x7f06880b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068c8a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0688072760 0x7f0688159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068c8a8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f068815da70 0x7f06880b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068c8a8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880b4510 con 0x7f0688072760 2023-10-16T10:00:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068c8a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880b47f0 con 0x7f068806afb0 2023-10-16T10:00:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068c8a8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06880b4ad0 con 0x7f068815da70 2023-10-16T10:00:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f0685d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0688072760 0x7f0688159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f068659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068806afb0 0x7f06880b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f0685d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0688072760 0x7f0688159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44776/0 (socket says 172.21.15.181:44776) 2023-10-16T10:00:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f0685d9b700 1 -- 172.21.15.181:0/2178710974 learned_addr learned my addr 172.21.15.181:0/2178710974 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.574+0000 7f0686d9d700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f068815da70 0x7f06880b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0685d9b700 1 -- 172.21.15.181:0/2178710974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f068815da70 msgr2=0x7f06880b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0685d9b700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f068815da70 0x7f06880b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0685d9b700 1 -- 172.21.15.181:0/2178710974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068806afb0 msgr2=0x7f06880b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0685d9b700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068806afb0 0x7f06880b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0685d9b700 1 -- 172.21.15.181:0/2178710974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0680009a70 con 0x7f0688072760 2023-10-16T10:00:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0685d9b700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0688072760 0x7f0688159ce0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c4e0 tx=0x7f067c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0676ffd700 1 -- 172.21.15.181:0/2178710974 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0052d0 con 0x7f0688072760 2023-10-16T10:00:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0676ffd700 1 -- 172.21.15.181:0/2178710974 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c005430 con 0x7f0688072760 2023-10-16T10:00:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06880b4db0 con 0x7f0688072760 2023-10-16T10:00:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f0676ffd700 1 -- 172.21.15.181:0/2178710974 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0038c0 con 0x7f0688072760 2023-10-16T10:00:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.575+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06880b5af0 con 0x7f0688072760 2023-10-16T10:00:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.576+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06880b52d0 con 0x7f0688072760 2023-10-16T10:00:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.577+0000 7f0676ffd700 1 -- 172.21.15.181:0/2178710974 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 53) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f067c003a20 con 0x7f0688072760 2023-10-16T10:00:43.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.578+0000 7f0676ffd700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 0x7f066806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.578+0000 7f068659c700 1 -- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 msgr2=0x7f066806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:43.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.578+0000 7f0676ffd700 1 -- 172.21.15.181:0/2178710974 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f067c088cf0 con 0x7f0688072760 2023-10-16T10:00:43.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.578+0000 7f068659c700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 0x7f066806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:43.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.582+0000 7f0676ffd700 1 -- 172.21.15.181:0/2178710974 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f06880b52d0 con 0x7f0688072760 2023-10-16T10:00:43.594 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:43.593+0000 7fd3ea0ce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:00:43.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.779+0000 7f068659c700 1 -- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 msgr2=0x7f066806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:43.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.779+0000 7f068659c700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 0x7f066806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:43.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.781+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f06880b52d0 con 0x7f0688072760 2023-10-16T10:00:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.784+0000 7f0676ffd700 1 -- 172.21.15.181:0/2178710974 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7f06880b52d0 con 0x7f0688072760 2023-10-16T10:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 msgr2=0x7f066806d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 0x7f066806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0688072760 msgr2=0x7f0688159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0688072760 0x7f0688159ce0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f067c00c4e0 tx=0x7f067c00c7f0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 shutdown_connections 2023-10-16T10:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7f066806b070 0x7f066806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f068815da70 0x7f06880b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f068806afb0 0x7f06880b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 --2- 172.21.15.181:0/2178710974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0688072760 0x7f0688159ce0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 >> 172.21.15.181:0/2178710974 conn(0x7f06880cd010 msgr2=0x7f06880ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 shutdown_connections 2023-10-16T10:00:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.786+0000 7f068c8a8700 1 -- 172.21.15.181:0/2178710974 wait complete. 2023-10-16T10:00:43.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1m 2023-10-16T10:00:43.891 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:43.890+0000 7f6f783c6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:00:43.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.925+0000 7fa40a98c700 1 Processor -- start 2023-10-16T10:00:43.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.925+0000 7fa40a98c700 1 -- start start 2023-10-16T10:00:43.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.926+0000 7fa40a98c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa40415ea40 0x7fa4040d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.926+0000 7fa40a98c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4040cf580 con 0x7fa4040d01d0 2023-10-16T10:00:43.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.926+0000 7fa40a98c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4040ce970 con 0x7fa4040ce2d0 2023-10-16T10:00:43.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.926+0000 7fa40a98c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040d4290 con 0x7fa40415ea40 2023-10-16T10:00:43.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.926+0000 7fa403fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa4040d01d0 0x7fa4040d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:55896/0 (socket says 172.21.15.181:55896) 2023-10-16T10:00:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.926+0000 7fa403fff700 1 -- 172.21.15.181:0/674440144 learned_addr learned my addr 172.21.15.181:0/674440144 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:00:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.926+0000 7fa408f29700 1 --2- 172.21.15.181:0/674440144 >> v2:172.21.15.181:3300/0 conn(0x7fa40415ea40 0x7fa4040d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.927+0000 7fa408f29700 1 -- 172.21.15.181:0/674440144 >> v1:172.21.15.190:6789/0 conn(0x7fa4040ce2d0 legacy=0x7fa4040d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.927+0000 7fa408f29700 1 -- 172.21.15.181:0/674440144 >> v1:172.21.15.181:6789/0 conn(0x7fa4040d01d0 legacy=0x7fa4040d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.927+0000 7fa408f29700 1 -- 172.21.15.181:0/674440144 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040cec50 con 0x7fa40415ea40 2023-10-16T10:00:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.927+0000 7fa408f29700 1 --2- 172.21.15.181:0/674440144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40415ea40 0x7fa4040d1c10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009d90 tx=0x7fa3fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a9a0bb32498271 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.927+0000 7fa402ffd700 1 -- 172.21.15.181:0/674440144 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00e070 con 0x7fa40415ea40 2023-10-16T10:00:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.927+0000 7fa402ffd700 1 -- 172.21.15.181:0/674440144 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc00bbd0 con 0x7fa40415ea40 2023-10-16T10:00:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.927+0000 7fa402ffd700 1 -- 172.21.15.181:0/674440144 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc012430 con 0x7fa40415ea40 2023-10-16T10:00:43.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.928+0000 7fa40a98c700 1 -- 172.21.15.181:0/674440144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40415ea40 msgr2=0x7fa4040d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.928+0000 7fa40a98c700 1 --2- 172.21.15.181:0/674440144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40415ea40 0x7fa4040d1c10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009d90 tx=0x7fa3fc00b330 comp rx=0 tx=0).stop 2023-10-16T10:00:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.928+0000 7fa40a98c700 1 -- 172.21.15.181:0/674440144 shutdown_connections 2023-10-16T10:00:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.928+0000 7fa40a98c700 1 --2- 172.21.15.181:0/674440144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40415ea40 0x7fa4040d1c10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.928+0000 7fa40a98c700 1 -- 172.21.15.181:0/674440144 >> 172.21.15.181:0/674440144 conn(0x7fa40409cb60 msgr2=0x7fa4040d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.928+0000 7fa40a98c700 1 -- 172.21.15.181:0/674440144 shutdown_connections 2023-10-16T10:00:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.928+0000 7fa40a98c700 1 -- 172.21.15.181:0/674440144 wait complete. 2023-10-16T10:00:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.929+0000 7fa40a98c700 1 Processor -- start 2023-10-16T10:00:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.929+0000 7fa40a98c700 1 -- start start 2023-10-16T10:00:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa40a98c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4040ce2d0 0x7fa4040c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa40a98c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4040d01d0 0x7fa4040b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa40a98c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa40415ea40 0x7fa4040b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa40a98c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040cd4d0 con 0x7fa4040d01d0 2023-10-16T10:00:43.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa40a98c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040b52a0 con 0x7fa4040ce2d0 2023-10-16T10:00:43.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa40a98c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040cd1f0 con 0x7fa40415ea40 2023-10-16T10:00:43.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4040d01d0 0x7fa4040b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4040d01d0 0x7fa4040b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44826/0 (socket says 172.21.15.181:44826) 2023-10-16T10:00:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 -- 172.21.15.181:0/3238269424 learned_addr learned my addr 172.21.15.181:0/3238269424 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4040ce2d0 0x7fa4040c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa408f29700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa40415ea40 0x7fa4040b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa40415ea40 msgr2=0x7fa4040b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa40415ea40 0x7fa4040b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4040ce2d0 msgr2=0x7fa4040c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:43.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4040ce2d0 0x7fa4040c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:43.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.930+0000 7fa4037fe700 1 -- 172.21.15.181:0/3238269424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc009a70 con 0x7fa4040d01d0 2023-10-16T10:00:43.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.931+0000 7fa4037fe700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4040d01d0 0x7fa4040b3750 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c4e0 tx=0x7fa3f800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:43.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.931+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80052d0 con 0x7fa4040d01d0 2023-10-16T10:00:43.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.931+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8005430 con 0x7fa4040d01d0 2023-10-16T10:00:43.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.931+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4040b4be0 con 0x7fa4040d01d0 2023-10-16T10:00:43.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.931+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80038c0 con 0x7fa4040d01d0 2023-10-16T10:00:43.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.931+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4040c7830 con 0x7fa4040d01d0 2023-10-16T10:00:43.940 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:43.932+0000 7fd3ea0ce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:00:43.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.932+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4040b4a40 con 0x7fa4040d01d0 2023-10-16T10:00:43.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.932+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 53) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa3f8003a20 con 0x7fa4040d01d0 2023-10-16T10:00:43.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.933+0000 7fa400ff9700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 0x7fa3f006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:43.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.933+0000 7fa403fff700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 msgr2=0x7fa3f006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:43.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.933+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fa3f8088c10 con 0x7fa4040d01d0 2023-10-16T10:00:43.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.933+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 0x7fa3f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:00:43.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:43.937+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa4040b4a40 con 0x7fa4040d01d0 2023-10-16T10:00:44.007 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:44.006+0000 7f6f783c6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:00:44.055 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:44.053+0000 7fd3ea0ce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:00:44.081 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:44.080+0000 7f6f783c6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:00:44.130 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:44.129+0000 7fd3ea0ce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:00:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:44.134+0000 7fa403fff700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 msgr2=0x7fa3f006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:44.134+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 0x7fa3f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:00:44.160 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:44.159+0000 7f6f783c6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:00:44.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:44.164+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 --> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4040cef30 con 0x7fa3f006b070 2023-10-16T10:00:44.212 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:44.210+0000 7fd3ea0ce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:00:44.526 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:44.525+0000 7f6f783c6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:00:44.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:44.535+0000 7fa403fff700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 msgr2=0x7fa3f006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:44.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:44.535+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 0x7fa3f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:00:44.581 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:44.581+0000 7fd3ea0ce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:00:45.091 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:45.090+0000 7f6f783c6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:00:45.153 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:45.152+0000 7fd3ea0ce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:00:45.161 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:45.160+0000 7f6f783c6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:00:45.223 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:45.223+0000 7fd3ea0ce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:00:45.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:45.336+0000 7fa403fff700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 msgr2=0x7fa3f006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:45.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:45.336+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 0x7fa3f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:00:45.788 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:45.787+0000 7f6f783c6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:45.856 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:45.855+0000 7fd3ea0ce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:45.865 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:45.864+0000 7f6f783c6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:00:45.933 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:45.933+0000 7fd3ea0ce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:00:46.008 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:46.007+0000 7f6f783c6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:46.078 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:46.077+0000 7fd3ea0ce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:00:46.237 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:46.236+0000 7f6f783c6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:00:46.307 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:46.306+0000 7fd3ea0ce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:00:46.310 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:46.309+0000 7f6f783c6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:00:46.381 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:46.381+0000 7fd3ea0ce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:00:46.574 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:46.573+0000 7f6f783c6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:00:46.647 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:46.647+0000 7fd3ea0ce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:00:46.794 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:46.792+0000 7f6f783c6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:00:46.871 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:46.871+0000 7fd3ea0ce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:00:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:46.937+0000 7fa403fff700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 msgr2=0x7fa3f006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3229300335 2023-10-16T10:00:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:46.937+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 0x7fa3f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:00:46.940 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:46.939+0000 7f6f783c6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:00:47.022 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:47.022+0000 7fd3ea0ce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:00:47.053 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:47.052+0000 7f6f783c6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:00:47.137 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:47.136+0000 7fd3ea0ce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:00:47.207 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:47.206+0000 7f6f783c6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:00:47.294 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:47.294+0000 7fd3ea0ce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:00:47.469 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:47.468+0000 7f6f783c6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:47.558 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:47.557+0000 7fd3ea0ce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:00:47.626 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:47.625+0000 7f6f783c6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:00:47.720 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:47.719+0000 7fd3ea0ce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:00:48.143 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:00:48.142+0000 7f6f783c6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:00:48.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:48.212+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 54) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa3f8053e60 con 0x7fa4040d01d0 2023-10-16T10:00:48.252 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.251+0000 7fd3ea0ce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:00:48.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:48.273+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 55) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fa3f8086f50 con 0x7fa4040d01d0 2023-10-16T10:00:48.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:48.273+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 msgr2=0x7fa3f006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:00:48.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:48.273+0000 7fa400ff9700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/3229300335,v1:172.21.15.181:6833/3229300335] conn(0x7fa3f006b070 0x7fa3f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:48.440 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.439+0000 7fd35d0a9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.441 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.439+0000 7fd35d0a9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.441 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.439+0000 7fd35d0a9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.441 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.439+0000 7fd35d0a9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.441 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.439+0000 7fd35d0a9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.441 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.440+0000 7fd35f8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.442 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.440+0000 7fd35f8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.442 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.440+0000 7fd35f8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.442 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.440+0000 7fd35f8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:48.442 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:00:48.440+0000 7fd35f8ae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:00:49.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.281+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 56) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fa3f8054730 con 0x7fa4040d01d0 2023-10-16T10:00:49.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.281+0000 7fa400ff9700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fa3f00279c0 0x7fa3f0027da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:49.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.281+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 --> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3f0071070 con 0x7fa3f00279c0 2023-10-16T10:00:49.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.284+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fa3f00279c0 0x7fa3f0027da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:49.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.285+0000 7fa403fff700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fa3f00279c0 0x7fa3f0027da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005910 tx=0x7fa3f4007f50 comp rx=0 tx=0).ready entity=mgr.6991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:49.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.348+0000 7fa400ff9700 1 -- 172.21.15.181:0/3238269424 <== mgr.6991 v2:172.21.15.181:6832/932006106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fa3f0071070 con 0x7fa3f00279c0 2023-10-16T10:00:49.350 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /bad-path 2023-10-16T10:00:49.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.350+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fa3f00279c0 msgr2=0x7fa3f0027da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:49.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.350+0000 7fa40a98c700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fa3f00279c0 0x7fa3f0027da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005910 tx=0x7fa3f4007f50 comp rx=0 tx=0).stop 2023-10-16T10:00:49.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.350+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4040d01d0 msgr2=0x7fa4040b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:49.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.350+0000 7fa40a98c700 1 --2- 172.21.15.181:0/3238269424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4040d01d0 0x7fa4040b3750 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c4e0 tx=0x7fa3f800c7f0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.350+0000 7fa408f29700 1 -- 172.21.15.181:0/3238269424 reap_dead start 2023-10-16T10:00:49.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.351+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 shutdown_connections 2023-10-16T10:00:49.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.351+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 >> 172.21.15.181:0/3238269424 conn(0x7fa40409cb60 msgr2=0x7fa40406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:49.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.351+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 shutdown_connections 2023-10-16T10:00:49.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.351+0000 7fa40a98c700 1 -- 172.21.15.181:0/3238269424 wait complete. 2023-10-16T10:00:49.363 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-16T10:00:49.363 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-16T10:00:49.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.487+0000 7fc17ea6b700 1 Processor -- start 2023-10-16T10:00:49.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.487+0000 7fc17ea6b700 1 -- start start 2023-10-16T10:00:49.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17ea6b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc17815dc40 0x7fc17806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:49.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17ea6b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1780cabc0 0x7fc1780cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:49.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17ea6b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17806b890 con 0x7fc17806da20 2023-10-16T10:00:49.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17ea6b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17806c260 con 0x7fc17815dc40 2023-10-16T10:00:49.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17ea6b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17806ac80 con 0x7fc1780cabc0 2023-10-16T10:00:49.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17c807700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc17815dc40 0x7fc17806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:49.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17c807700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc17815dc40 0x7fc17806bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44988/0 (socket says 172.21.15.181:44988) 2023-10-16T10:00:49.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17c807700 1 -- 172.21.15.181:0/1370289611 learned_addr learned my addr 172.21.15.181:0/1370289611 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:49.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17c807700 1 -- 172.21.15.181:0/1370289611 >> v2:172.21.15.190:3300/0 conn(0x7fc1780cabc0 msgr2=0x7fc1780cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:49.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17c807700 1 --2- 172.21.15.181:0/1370289611 >> v2:172.21.15.190:3300/0 conn(0x7fc1780cabc0 0x7fc1780cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17c807700 1 -- 172.21.15.181:0/1370289611 >> v1:172.21.15.190:6789/0 conn(0x7fc17806da20 legacy=0x7fc1780ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:00:49.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.488+0000 7fc17c807700 1 -- 172.21.15.181:0/1370289611 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17806af60 con 0x7fc17815dc40 2023-10-16T10:00:49.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17c807700 1 --2- 172.21.15.181:0/1370289611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17815dc40 0x7fc17806bd20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc178072c70 tx=0x7fc16c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eca16fc902fe66f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:49.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc1777fe700 1 -- 172.21.15.181:0/1370289611 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c004230 con 0x7fc17815dc40 2023-10-16T10:00:49.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc1777fe700 1 -- 172.21.15.181:0/1370289611 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c004390 con 0x7fc17815dc40 2023-10-16T10:00:49.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc1777fe700 1 -- 172.21.15.181:0/1370289611 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0036a0 con 0x7fc17815dc40 2023-10-16T10:00:49.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 -- 172.21.15.181:0/1370289611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17815dc40 msgr2=0x7fc17806bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:49.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/1370289611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17815dc40 0x7fc17806bd20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc178072c70 tx=0x7fc16c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 -- 172.21.15.181:0/1370289611 shutdown_connections 2023-10-16T10:00:49.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/1370289611 >> v2:172.21.15.190:3300/0 conn(0x7fc1780cabc0 0x7fc1780cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/1370289611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17815dc40 0x7fc17806bd20 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 -- 172.21.15.181:0/1370289611 >> 172.21.15.181:0/1370289611 conn(0x7fc1780b1950 msgr2=0x7fc1780b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:49.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 -- 172.21.15.181:0/1370289611 shutdown_connections 2023-10-16T10:00:49.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.489+0000 7fc17ea6b700 1 -- 172.21.15.181:0/1370289611 wait complete. 2023-10-16T10:00:49.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.490+0000 7fc17ea6b700 1 Processor -- start 2023-10-16T10:00:49.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.490+0000 7fc17ea6b700 1 -- start start 2023-10-16T10:00:49.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.491+0000 7fc17ea6b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17806da20 0x7fc178159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:49.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.491+0000 7fc17ea6b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1780cabc0 0x7fc1781f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:49.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.491+0000 7fc17ea6b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17814e7f0 0x7fc17814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:49.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.491+0000 7fc17ea6b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781f3000 con 0x7fc17814e7f0 2023-10-16T10:00:49.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.491+0000 7fc17ea6b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780acc90 con 0x7fc17806da20 2023-10-16T10:00:49.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.491+0000 7fc17ea6b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17815a000 con 0x7fc1780cabc0 2023-10-16T10:00:49.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.491+0000 7fc17d008700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17814e7f0 0x7fc17814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:49.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17814e7f0 0x7fc17814bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44998/0 (socket says 172.21.15.181:44998) 2023-10-16T10:00:49.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 -- 172.21.15.181:0/3192298470 learned_addr learned my addr 172.21.15.181:0/3192298470 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:00:49.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17c807700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17806da20 0x7fc178159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:49.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc177fff700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1780cabc0 0x7fc1781f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:49.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 -- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1780cabc0 msgr2=0x7fc1781f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:49.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1780cabc0 0x7fc1781f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 -- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17806da20 msgr2=0x7fc178159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:49.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17806da20 0x7fc178159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 -- 172.21.15.181:0/3192298470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c009550 con 0x7fc17814e7f0 2023-10-16T10:00:49.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17d008700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17814e7f0 0x7fc17814bf00 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc17000b970 tx=0x7fc17000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:49.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc1757fa700 1 -- 172.21.15.181:0/3192298470 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17000c9b0 con 0x7fc17814e7f0 2023-10-16T10:00:49.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc1757fa700 1 -- 172.21.15.181:0/3192298470 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17000cb30 con 0x7fc17814e7f0 2023-10-16T10:00:49.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17815a2e0 con 0x7fc17814e7f0 2023-10-16T10:00:49.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc1757fa700 1 -- 172.21.15.181:0/3192298470 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170014440 con 0x7fc17814e7f0 2023-10-16T10:00:49.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.492+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17815a8a0 con 0x7fc17814e7f0 2023-10-16T10:00:49.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.493+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc178067d20 con 0x7fc17814e7f0 2023-10-16T10:00:49.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.493+0000 7fc1757fa700 1 -- 172.21.15.181:0/3192298470 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 56) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc17001d3b0 con 0x7fc17814e7f0 2023-10-16T10:00:49.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.494+0000 7fc1757fa700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc15c06ada0 0x7fc15c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:00:49.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.494+0000 7fc1757fa700 1 -- 172.21.15.181:0/3192298470 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fc170006e40 con 0x7fc17814e7f0 2023-10-16T10:00:49.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.494+0000 7fc17c807700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc15c06ada0 0x7fc15c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:00:49.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.494+0000 7fc17c807700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc15c06ada0 0x7fc15c06d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009130 tx=0x7fc16c008030 comp rx=0 tx=0).ready entity=mgr.6991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:00:49.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.498+0000 7fc1757fa700 1 -- 172.21.15.181:0/3192298470 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc1700543f0 con 0x7fc17814e7f0 2023-10-16T10:00:49.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.719+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 --> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc17806b520 con 0x7fc15c06ada0 2023-10-16T10:00:49.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.729+0000 7fc1757fa700 1 -- 172.21.15.181:0/3192298470 <== mgr.6991 v2:172.21.15.181:6832/932006106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+302 (secure 0 0 0) 0x7fc16c020070 con 0x7fc15c06ada0 2023-10-16T10:00:49.730 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:00:49.730 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T10:00:49.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc15c06ada0 msgr2=0x7fc15c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc15c06ada0 0x7fc15c06d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009130 tx=0x7fc16c008030 comp rx=0 tx=0).stop 2023-10-16T10:00:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17814e7f0 msgr2=0x7fc17814bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:00:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17814e7f0 0x7fc17814bf00 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc17000b970 tx=0x7fc17000bd30 comp rx=0 tx=0).stop 2023-10-16T10:00:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 shutdown_connections 2023-10-16T10:00:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc15c06ada0 0x7fc15c06d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc17814e7f0 0x7fc17814bf00 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17806da20 0x7fc178159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 --2- 172.21.15.181:0/3192298470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1780cabc0 0x7fc1781f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:00:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 >> 172.21.15.181:0/3192298470 conn(0x7fc1780b1950 msgr2=0x7fc17815fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:00:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.731+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 shutdown_connections 2023-10-16T10:00:49.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:00:49.732+0000 7fc17ea6b700 1 -- 172.21.15.181:0/3192298470 wait complete. 2023-10-16T10:00:49.744 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T10:01:00.359 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:00.357+0000 7fd34f88e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:00.359 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:00.357+0000 7fd34f88e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:00.360 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:00.357+0000 7fd34f88e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:00.360 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:00.357+0000 7fd34f88e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:00.360 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:00.357+0000 7fd34f88e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:10.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:10.302 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:40.364 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:40.368 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:01:49.745 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-16T10:01:49.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.879+0000 7ff9c9688700 1 Processor -- start 2023-10-16T10:01:49.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.879+0000 7ff9c9688700 1 -- start start 2023-10-16T10:01:49.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c9688700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff9c415f150 0x7ff9c40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:49.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c9688700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c40cd170 0x7ff9c40b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:49.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c9688700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c40b2ae0 con 0x7ff9c40ce4f0 2023-10-16T10:01:49.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c9688700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c40ce9a0 con 0x7ff9c415f150 2023-10-16T10:01:49.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c9688700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40b2dc0 con 0x7ff9c40cd170 2023-10-16T10:01:49.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff9c415f150 0x7ff9c40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:49.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c27fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff9c40ce4f0 0x7ff9c40b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56106/0 (socket says 172.21.15.181:56106) 2023-10-16T10:01:49.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c27fc700 1 -- 172.21.15.181:0/1145585583 learned_addr learned my addr 172.21.15.181:0/1145585583 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:49.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.880+0000 7ff9c2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff9c415f150 0x7ff9c40cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45030/0 (socket says 172.21.15.181:45030) 2023-10-16T10:01:49.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c2ffd700 1 -- 172.21.15.181:0/1145585583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c40cd170 msgr2=0x7ff9c40b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:49.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c37fe700 1 --2- 172.21.15.181:0/1145585583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c40cd170 0x7ff9c40b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:49.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c2ffd700 1 --2- 172.21.15.181:0/1145585583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c40cd170 0x7ff9c40b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c1ffb700 1 -- 172.21.15.181:0/1145585583 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191131724 0 0) 0x7ff9c40b2ae0 con 0x7ff9c40ce4f0 2023-10-16T10:01:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c2ffd700 1 -- 172.21.15.181:0/1145585583 >> v1:172.21.15.181:6789/0 conn(0x7ff9c40ce4f0 legacy=0x7ff9c40b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c2ffd700 1 -- 172.21.15.181:0/1145585583 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c40cf9d0 con 0x7ff9c415f150 2023-10-16T10:01:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c37fe700 1 --2- 172.21.15.181:0/1145585583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c40cd170 0x7ff9c40b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c2ffd700 1 --2- 172.21.15.181:0/1145585583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c415f150 0x7ff9c40cdfb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009870 tx=0x7ff9b8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5904df183993687e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:49.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c1ffb700 1 -- 172.21.15.181:0/1145585583 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8011830 con 0x7ff9c415f150 2023-10-16T10:01:49.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c1ffb700 1 -- 172.21.15.181:0/1145585583 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b8011990 con 0x7ff9c415f150 2023-10-16T10:01:49.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.881+0000 7ff9c1ffb700 1 -- 172.21.15.181:0/1145585583 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8011d20 con 0x7ff9c415f150 2023-10-16T10:01:49.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 -- 172.21.15.181:0/1145585583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c415f150 msgr2=0x7ff9c40cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:49.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 --2- 172.21.15.181:0/1145585583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c415f150 0x7ff9c40cdfb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009870 tx=0x7ff9b8009b80 comp rx=0 tx=0).stop 2023-10-16T10:01:49.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 -- 172.21.15.181:0/1145585583 shutdown_connections 2023-10-16T10:01:49.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 --2- 172.21.15.181:0/1145585583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c40cd170 0x7ff9c40b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 --2- 172.21.15.181:0/1145585583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c415f150 0x7ff9c40cdfb0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 -- 172.21.15.181:0/1145585583 >> 172.21.15.181:0/1145585583 conn(0x7ff9c409a930 msgr2=0x7ff9c40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:49.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 -- 172.21.15.181:0/1145585583 shutdown_connections 2023-10-16T10:01:49.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.882+0000 7ff9c9688700 1 -- 172.21.15.181:0/1145585583 wait complete. 2023-10-16T10:01:49.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.883+0000 7ff9c9688700 1 Processor -- start 2023-10-16T10:01:49.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.883+0000 7ff9c9688700 1 -- start start 2023-10-16T10:01:49.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c9688700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c40cd170 0x7ff9c415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:49.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c9688700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9c40ce4f0 0x7ff9c415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:49.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c9688700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c415f150 0x7ff9c415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:49.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c9688700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40c5730 con 0x7ff9c40cd170 2023-10-16T10:01:49.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c9688700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40c5940 con 0x7ff9c40ce4f0 2023-10-16T10:01:49.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c9688700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40c5c20 con 0x7ff9c415f150 2023-10-16T10:01:49.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c40cd170 0x7ff9c415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:49.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c40cd170 0x7ff9c415b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45034/0 (socket says 172.21.15.181:45034) 2023-10-16T10:01:49.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 -- 172.21.15.181:0/596751383 learned_addr learned my addr 172.21.15.181:0/596751383 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:49.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 -- 172.21.15.181:0/596751383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c415f150 msgr2=0x7ff9c415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:49.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c415f150 0x7ff9c415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 -- 172.21.15.181:0/596751383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9c40ce4f0 msgr2=0x7ff9c415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:49.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9c40ce4f0 0x7ff9c415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:49.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 -- 172.21.15.181:0/596751383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80094a0 con 0x7ff9c40cd170 2023-10-16T10:01:49.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.884+0000 7ff9c2ffd700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c40cd170 0x7ff9c415b680 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80090b0 tx=0x7ff9b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:49.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.885+0000 7ff9ab7fe700 1 -- 172.21.15.181:0/596751383 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80038a0 con 0x7ff9c40cd170 2023-10-16T10:01:49.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.885+0000 7ff9ab7fe700 1 -- 172.21.15.181:0/596751383 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b801f070 con 0x7ff9c40cd170 2023-10-16T10:01:49.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.885+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c40c5f00 con 0x7ff9c40cd170 2023-10-16T10:01:49.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.885+0000 7ff9ab7fe700 1 -- 172.21.15.181:0/596751383 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8003c40 con 0x7ff9c40cd170 2023-10-16T10:01:49.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.885+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c40c64c0 con 0x7ff9c40cd170 2023-10-16T10:01:49.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.886+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c4067d20 con 0x7ff9c40cd170 2023-10-16T10:01:49.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.887+0000 7ff9ab7fe700 1 -- 172.21.15.181:0/596751383 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 59) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff9b801a430 con 0x7ff9c40cd170 2023-10-16T10:01:49.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.887+0000 7ff9ab7fe700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7ff9b006b190 0x7ff9b006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:49.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.887+0000 7ff9c27fc700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7ff9b006b190 0x7ff9b006d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:49.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.887+0000 7ff9ab7fe700 1 -- 172.21.15.181:0/596751383 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff9b8021070 con 0x7ff9c40cd170 2023-10-16T10:01:49.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.887+0000 7ff9c27fc700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7ff9b006b190 0x7ff9b006d650 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40011e0 tx=0x7ff9b40079b0 comp rx=0 tx=0).ready entity=mgr.6991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:49.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:49.891+0000 7ff9ab7fe700 1 -- 172.21.15.181:0/596751383 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff9b80565e0 con 0x7ff9c40cd170 2023-10-16T10:01:50.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.109+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 --> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9c40cff90 con 0x7ff9b006b190 2023-10-16T10:01:50.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.120+0000 7ff9ab7fe700 1 -- 172.21.15.181:0/596751383 <== mgr.6991 v2:172.21.15.181:6832/932006106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+303 (secure 0 0 0) 0x7ff9b4004870 con 0x7ff9b006b190 2023-10-16T10:01:50.122 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:50.122 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-16T10:01:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7ff9b006b190 msgr2=0x7ff9b006d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7ff9b006b190 0x7ff9b006d650 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40011e0 tx=0x7ff9b40079b0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c40cd170 msgr2=0x7ff9c415b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c40cd170 0x7ff9c415b680 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80090b0 tx=0x7ff9b8004750 comp rx=0 tx=0).stop 2023-10-16T10:01:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 shutdown_connections 2023-10-16T10:01:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7ff9b006b190 0x7ff9b006d650 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9c415f150 0x7ff9c415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9c40cd170 0x7ff9c415b680 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 --2- 172.21.15.181:0/596751383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9c40ce4f0 0x7ff9c415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 >> 172.21.15.181:0/596751383 conn(0x7ff9c409a930 msgr2=0x7ff9c40b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:50.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 shutdown_connections 2023-10-16T10:01:50.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.122+0000 7ff9c9688700 1 -- 172.21.15.181:0/596751383 wait complete. 2023-10-16T10:01:50.135 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:00:49", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-16T10:01:50.136 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-10-16T10:01:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.256+0000 7f66c03b9700 1 Processor -- start 2023-10-16T10:01:50.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.256+0000 7f66c03b9700 1 -- start start 2023-10-16T10:01:50.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.257+0000 7f66c03b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f66b80cb2b0 0x7f66b806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.257+0000 7f66c03b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f66b80b5fd0 0x7f66b815f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.257+0000 7f66c03b9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b8070590 con 0x7f66b80cb690 2023-10-16T10:01:50.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.257+0000 7f66c03b9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b8156330 con 0x7f66b80cb2b0 2023-10-16T10:01:50.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.257+0000 7f66c03b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b806fcf0 con 0x7f66b80b5fd0 2023-10-16T10:01:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.257+0000 7f66be155700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f66b80cb2b0 0x7f66b806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66bd954700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f66b80cb690 0x7f66b80b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56124/0 (socket says 172.21.15.181:56124) 2023-10-16T10:01:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66bd954700 1 -- 172.21.15.181:0/4258869244 learned_addr learned my addr 172.21.15.181:0/4258869244 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66be956700 1 --2- 172.21.15.181:0/4258869244 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b5fd0 0x7f66b815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66be155700 1 -- 172.21.15.181:0/4258869244 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b5fd0 msgr2=0x7f66b815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:50.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66be155700 1 --2- 172.21.15.181:0/4258869244 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b5fd0 0x7f66b815f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66be155700 1 -- 172.21.15.181:0/4258869244 >> v1:172.21.15.181:6789/0 conn(0x7f66b80cb690 legacy=0x7f66b80b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66be155700 1 -- 172.21.15.181:0/4258869244 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b806ffd0 con 0x7f66b80cb2b0 2023-10-16T10:01:50.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66be956700 1 --2- 172.21.15.181:0/4258869244 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b5fd0 0x7f66b815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:50.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66bd153700 1 -- 172.21.15.181:0/4258869244 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259615692 0 0) 0x7f66b8070590 con 0x7f66b80cb690 2023-10-16T10:01:50.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66be155700 1 --2- 172.21.15.181:0/4258869244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b806f710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f66b80c7610 tx=0x7f66a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42201759893ad6cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66bd153700 1 -- 172.21.15.181:0/4258869244 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8011830 con 0x7f66b80cb2b0 2023-10-16T10:01:50.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66bd153700 1 -- 172.21.15.181:0/4258869244 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a8011990 con 0x7f66b80cb2b0 2023-10-16T10:01:50.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.258+0000 7f66bd153700 1 -- 172.21.15.181:0/4258869244 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8011d20 con 0x7f66b80cb2b0 2023-10-16T10:01:50.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 -- 172.21.15.181:0/4258869244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 msgr2=0x7f66b806f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 --2- 172.21.15.181:0/4258869244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b806f710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f66b80c7610 tx=0x7f66a8009b20 comp rx=0 tx=0).stop 2023-10-16T10:01:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 -- 172.21.15.181:0/4258869244 shutdown_connections 2023-10-16T10:01:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 --2- 172.21.15.181:0/4258869244 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b5fd0 0x7f66b815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 --2- 172.21.15.181:0/4258869244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b806f710 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 -- 172.21.15.181:0/4258869244 >> 172.21.15.181:0/4258869244 conn(0x7f66b80cd010 msgr2=0x7f66b80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 -- 172.21.15.181:0/4258869244 shutdown_connections 2023-10-16T10:01:50.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.259+0000 7f66c03b9700 1 -- 172.21.15.181:0/4258869244 wait complete. 2023-10-16T10:01:50.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.260+0000 7f66c03b9700 1 Processor -- start 2023-10-16T10:01:50.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.260+0000 7f66c03b9700 1 -- start start 2023-10-16T10:01:50.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80b5fd0 0x7f66b81f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b81471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80cb690 0x7f66b81476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b815c930 con 0x7f66b80cb2b0 2023-10-16T10:01:50.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80ced10 con 0x7f66b80b5fd0 2023-10-16T10:01:50.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80ceff0 con 0x7f66b80cb690 2023-10-16T10:01:50.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b81471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b81471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45050/0 (socket says 172.21.15.181:45050) 2023-10-16T10:01:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 -- 172.21.15.181:0/3790510578 learned_addr learned my addr 172.21.15.181:0/3790510578 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66be155700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80b5fd0 0x7f66b81f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 -- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80cb690 msgr2=0x7f66b81476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80cb690 0x7f66b81476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 -- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80b5fd0 msgr2=0x7f66b81f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80b5fd0 0x7f66b81f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 -- 172.21.15.181:0/3790510578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a80094a0 con 0x7f66b80cb2b0 2023-10-16T10:01:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66be155700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80b5fd0 0x7f66b81f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66bd954700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b81471a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f66b400c540 tx=0x7f66b400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66aeffd700 1 -- 172.21.15.181:0/3790510578 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4004c80 con 0x7f66b80cb2b0 2023-10-16T10:01:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66aeffd700 1 -- 172.21.15.181:0/3790510578 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b4004de0 con 0x7f66b80cb2b0 2023-10-16T10:01:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b80cf2d0 con 0x7f66b80cb2b0 2023-10-16T10:01:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66aeffd700 1 -- 172.21.15.181:0/3790510578 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4007310 con 0x7f66b80cb2b0 2023-10-16T10:01:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.261+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b80cfad0 con 0x7f66b80cb2b0 2023-10-16T10:01:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.262+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b80cf7f0 con 0x7f66b80cb2b0 2023-10-16T10:01:50.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.263+0000 7f66aeffd700 1 -- 172.21.15.181:0/3790510578 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 59) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f66b4007470 con 0x7f66b80cb2b0 2023-10-16T10:01:50.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.263+0000 7f66aeffd700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f669c06b190 0x7f669c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.263+0000 7f66be155700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f669c06b190 0x7f669c06d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.263+0000 7f66aeffd700 1 -- 172.21.15.181:0/3790510578 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f66b4089b30 con 0x7f66b80cb2b0 2023-10-16T10:01:50.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.264+0000 7f66be155700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f669c06b190 0x7f669c06d650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f66a80090b0 tx=0x7f66a8008030 comp rx=0 tx=0).ready entity=mgr.6991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.268+0000 7f66aeffd700 1 -- 172.21.15.181:0/3790510578 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f66b80cf7f0 con 0x7f66b80cb2b0 2023-10-16T10:01:50.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.487+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 --> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f66b80b2140 con 0x7f669c06b190 2023-10-16T10:01:50.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.552+0000 7f66aeffd700 1 -- 172.21.15.181:0/3790510578 <== mgr.6991 v2:172.21.15.181:6832/932006106 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f66a8011630 con 0x7f669c06b190 2023-10-16T10:01:50.554 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /bad-path 2023-10-16T10:01:50.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.554+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f669c06b190 msgr2=0x7f669c06d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.554+0000 7f66c03b9700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f669c06b190 0x7f669c06d650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f66a80090b0 tx=0x7f66a8008030 comp rx=0 tx=0).stop 2023-10-16T10:01:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.554+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 msgr2=0x7f66b81471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.554+0000 7f66c03b9700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b81471a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f66b400c540 tx=0x7f66b400c900 comp rx=0 tx=0).stop 2023-10-16T10:01:50.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 shutdown_connections 2023-10-16T10:01:50.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f669c06b190 0x7f669c06d650 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80cb690 0x7f66b81476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80b5fd0 0x7f66b81f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 --2- 172.21.15.181:0/3790510578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b80cb2b0 0x7f66b81471a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 >> 172.21.15.181:0/3790510578 conn(0x7f66b80cd010 msgr2=0x7f66b8071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:50.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 shutdown_connections 2023-10-16T10:01:50.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.555+0000 7f66c03b9700 1 -- 172.21.15.181:0/3790510578 wait complete. 2023-10-16T10:01:50.568 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-16T10:01:50.569 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:01:50.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.689+0000 7f5f4c514700 1 Processor -- start 2023-10-16T10:01:50.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.689+0000 7f5f4c514700 1 -- start start 2023-10-16T10:01:50.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.690+0000 7f5f4c514700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440cc6e0 0x7f5f44071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.690+0000 7f5f4c514700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f440ce680 0x7f5f440ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.690+0000 7f5f4c514700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f440ccb60 con 0x7f5f440b2010 2023-10-16T10:01:50.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.690+0000 7f5f4c514700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f44072120 con 0x7f5f440ce680 2023-10-16T10:01:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.690+0000 7f5f4c514700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440ce1f0 con 0x7f5f440cc6e0 2023-10-16T10:01:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.690+0000 7f5f4aab1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f440ce680 0x7f5f440ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.690+0000 7f5f4a2b0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5f440b2010 0x7f5f440b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56134/0 (socket says 172.21.15.181:56134) 2023-10-16T10:01:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f4a2b0700 1 -- 172.21.15.181:0/3684310554 learned_addr learned my addr 172.21.15.181:0/3684310554 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:50.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f49aaf700 1 --2- 172.21.15.181:0/3684310554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440cc6e0 0x7f5f44071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f4aab1700 1 -- 172.21.15.181:0/3684310554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440cc6e0 msgr2=0x7f5f44071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f4aab1700 1 --2- 172.21.15.181:0/3684310554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440cc6e0 0x7f5f44071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f4aab1700 1 -- 172.21.15.181:0/3684310554 >> v1:172.21.15.181:6789/0 conn(0x7f5f440b2010 legacy=0x7f5f440b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f492ae700 1 -- 172.21.15.181:0/3684310554 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591565149 0 0) 0x7f5f34001110 con 0x7f5f440b2010 2023-10-16T10:01:50.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f4aab1700 1 -- 172.21.15.181:0/3684310554 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440ccda0 con 0x7f5f440ce680 2023-10-16T10:01:50.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f4aab1700 1 --2- 172.21.15.181:0/3684310554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440ce680 0x7f5f440ceb10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009d90 tx=0x7f5f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203d004208c47d72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f492ae700 1 -- 172.21.15.181:0/3684310554 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00e070 con 0x7f5f440ce680 2023-10-16T10:01:50.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f492ae700 1 -- 172.21.15.181:0/3684310554 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c00bbd0 con 0x7f5f440ce680 2023-10-16T10:01:50.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.691+0000 7f5f492ae700 1 -- 172.21.15.181:0/3684310554 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c012430 con 0x7f5f440ce680 2023-10-16T10:01:50.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 -- 172.21.15.181:0/3684310554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440ce680 msgr2=0x7f5f440ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3684310554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440ce680 0x7f5f440ceb10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009d90 tx=0x7f5f3c00b330 comp rx=0 tx=0).stop 2023-10-16T10:01:50.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 -- 172.21.15.181:0/3684310554 shutdown_connections 2023-10-16T10:01:50.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3684310554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440ce680 0x7f5f440ceb10 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3684310554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440cc6e0 0x7f5f44071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 -- 172.21.15.181:0/3684310554 >> 172.21.15.181:0/3684310554 conn(0x7f5f440cd010 msgr2=0x7f5f440cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:50.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 -- 172.21.15.181:0/3684310554 shutdown_connections 2023-10-16T10:01:50.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.692+0000 7f5f4c514700 1 -- 172.21.15.181:0/3684310554 wait complete. 2023-10-16T10:01:50.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.693+0000 7f5f4c514700 1 Processor -- start 2023-10-16T10:01:50.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.693+0000 7f5f4c514700 1 -- start start 2023-10-16T10:01:50.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f440b2010 0x7f5f44158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440cc6e0 0x7f5f440a8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440ce680 0x7f5f440c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440a8dc0 con 0x7f5f440cc6e0 2023-10-16T10:01:50.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440a90a0 con 0x7f5f440b2010 2023-10-16T10:01:50.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440a9380 con 0x7f5f440ce680 2023-10-16T10:01:50.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440cc6e0 0x7f5f440a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440cc6e0 0x7f5f440a8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45064/0 (socket says 172.21.15.181:45064) 2023-10-16T10:01:50.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 -- 172.21.15.181:0/3673315290 learned_addr learned my addr 172.21.15.181:0/3673315290 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:50.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 -- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440ce680 msgr2=0x7f5f440c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440ce680 0x7f5f440c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 -- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f440b2010 msgr2=0x7f5f44158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:50.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4a2b0700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f440b2010 0x7f5f44158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f440b2010 0x7f5f44158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:50.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 -- 172.21.15.181:0/3673315290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c009a70 con 0x7f5f440cc6e0 2023-10-16T10:01:50.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4a2b0700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f440b2010 0x7f5f44158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:50.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f49aaf700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440cc6e0 0x7f5f440a8470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000d800 tx=0x7f5f4000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f400125f0 con 0x7f5f440cc6e0 2023-10-16T10:01:50.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f40012750 con 0x7f5f440cc6e0 2023-10-16T10:01:50.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f440a9660 con 0x7f5f440cc6e0 2023-10-16T10:01:50.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40012b00 con 0x7f5f440cc6e0 2023-10-16T10:01:50.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.694+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f440ac540 con 0x7f5f440cc6e0 2023-10-16T10:01:50.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.695+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f440ac7f0 con 0x7f5f440cc6e0 2023-10-16T10:01:50.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.696+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 59) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5f40012c60 con 0x7f5f440cc6e0 2023-10-16T10:01:50.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.696+0000 7f5f3affd700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f5f2806b190 0x7f5f2806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:50.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.696+0000 7f5f4a2b0700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f5f2806b190 0x7f5f2806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:50.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.696+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5f4008a0b0 con 0x7f5f440cc6e0 2023-10-16T10:01:50.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.697+0000 7f5f4a2b0700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f5f2806b190 0x7f5f2806d650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5f34007fd0 tx=0x7f5f34009040 comp rx=0 tx=0).ready entity=mgr.6991 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:50.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.701+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5f440ac7f0 con 0x7f5f440cc6e0 2023-10-16T10:01:50.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:50.950+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f5f440ac7f0 con 0x7f5f440cc6e0 2023-10-16T10:01:51.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.115+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v60) v1 ==== 93+0+0 (secure 0 0 0) 0x7f5f440ac7f0 con 0x7f5f440cc6e0 2023-10-16T10:01:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.120+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f5f2806b190 msgr2=0x7f5f2806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.120+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f5f2806b190 0x7f5f2806d650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5f34007fd0 tx=0x7f5f34009040 comp rx=0 tx=0).stop 2023-10-16T10:01:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.120+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440cc6e0 msgr2=0x7f5f440a8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f3affd700 1 -- 172.21.15.181:0/3673315290 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f5f40054f40 con 0x7f5f440cc6e0 2023-10-16T10:01:51.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440cc6e0 0x7f5f440a8470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000d800 tx=0x7f5f4000dbc0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 shutdown_connections 2023-10-16T10:01:51.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f5f2806b190 0x7f5f2806d650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f440ce680 0x7f5f440c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f440b2010 0x7f5f44158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 --2- 172.21.15.181:0/3673315290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f440cc6e0 0x7f5f440a8470 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 >> 172.21.15.181:0/3673315290 conn(0x7f5f440cd010 msgr2=0x7f5f440b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:51.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 shutdown_connections 2023-10-16T10:01:51.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.121+0000 7f5f4c514700 1 -- 172.21.15.181:0/3673315290 wait complete. 2023-10-16T10:01:51.135 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:01:51.136 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:01:51.173 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:01:51.215 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:01:51.216 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:01:51.216 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:51.233 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:01:51.233 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:51.264 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:51.262+0000 7f715e782200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:01:51.264 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:51.263+0000 7f52fb5fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:01:51.280 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:51.329 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:51.328+0000 7f52fb5fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:51.331 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:51.330+0000 7f715e782200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:51.401 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:51.400+0000 7f52fb5fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:01:51.406 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:51.405+0000 7f715e782200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:01:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.443+0000 7f367c134700 1 Processor -- start 2023-10-16T10:01:51.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.443+0000 7f367c134700 1 -- start start 2023-10-16T10:01:51.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f367c134700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f367415d650 0x7f3674161430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f367c134700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f367415da30 0x7f36740cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f367c134700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740d1440 con 0x7f36740b39c0 2023-10-16T10:01:51.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f367c134700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3674161a10 con 0x7f367415d650 2023-10-16T10:01:51.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f367c134700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3674161cf0 con 0x7f367415da30 2023-10-16T10:01:51.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f367415d650 0x7f3674161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f367415d650 0x7f3674161430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45140/0 (socket says 172.21.15.181:45140) 2023-10-16T10:01:51.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 -- 172.21.15.181:0/236163472 learned_addr learned my addr 172.21.15.181:0/236163472 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:51.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f36796cf700 1 --2- 172.21.15.181:0/236163472 >> v2:172.21.15.190:3300/0 conn(0x7f367415da30 0x7f36740cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 -- 172.21.15.181:0/236163472 >> v2:172.21.15.190:3300/0 conn(0x7f367415da30 msgr2=0x7f36740cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:51.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 --2- 172.21.15.181:0/236163472 >> v2:172.21.15.190:3300/0 conn(0x7f367415da30 0x7f36740cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 -- 172.21.15.181:0/236163472 >> v1:172.21.15.181:6789/0 conn(0x7f36740b39c0 legacy=0x7f36740cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 -- 172.21.15.181:0/236163472 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674161fd0 con 0x7f367415d650 2023-10-16T10:01:51.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3679ed0700 1 --2- 172.21.15.181:0/236163472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f367415d650 0x7f3674161430 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f36740c83c0 tx=0x7f3670009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb09ef29c644bf40 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3678ece700 1 -- 172.21.15.181:0/236163472 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367000b870 con 0x7f367415d650 2023-10-16T10:01:51.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3678ece700 1 -- 172.21.15.181:0/236163472 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367000b9d0 con 0x7f367415d650 2023-10-16T10:01:51.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.444+0000 7f3678ece700 1 -- 172.21.15.181:0/236163472 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367000bd60 con 0x7f367415d650 2023-10-16T10:01:51.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 -- 172.21.15.181:0/236163472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f367415d650 msgr2=0x7f3674161430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 --2- 172.21.15.181:0/236163472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f367415d650 0x7f3674161430 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f36740c83c0 tx=0x7f3670009bd0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 -- 172.21.15.181:0/236163472 shutdown_connections 2023-10-16T10:01:51.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 --2- 172.21.15.181:0/236163472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f367415d650 0x7f3674161430 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 --2- 172.21.15.181:0/236163472 >> v2:172.21.15.190:3300/0 conn(0x7f367415da30 0x7f36740cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 -- 172.21.15.181:0/236163472 >> 172.21.15.181:0/236163472 conn(0x7f36740665a0 msgr2=0x7f3674066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:51.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 -- 172.21.15.181:0/236163472 shutdown_connections 2023-10-16T10:01:51.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.445+0000 7f367c134700 1 -- 172.21.15.181:0/236163472 wait complete. 2023-10-16T10:01:51.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.446+0000 7f367c134700 1 Processor -- start 2023-10-16T10:01:51.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.446+0000 7f367c134700 1 -- start start 2023-10-16T10:01:51.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f367c134700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36740b39c0 0x7f3674159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f367c134700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f367415d650 0x7f36740b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f367c134700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f367415da30 0x7f36740b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f367c134700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36740c7400 con 0x7f36740b39c0 2023-10-16T10:01:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f367c134700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367415e820 con 0x7f367415d650 2023-10-16T10:01:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f367c134700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36740c89d0 con 0x7f367415da30 2023-10-16T10:01:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f3679ed0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36740b39c0 0x7f3674159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f3679ed0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36740b39c0 0x7f3674159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45146/0 (socket says 172.21.15.181:45146) 2023-10-16T10:01:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f3679ed0700 1 -- 172.21.15.181:0/1279153771 learned_addr learned my addr 172.21.15.181:0/1279153771 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f36796cf700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f367415d650 0x7f36740b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f367a6d1700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f367415da30 0x7f36740b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f3679ed0700 1 -- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f367415da30 msgr2=0x7f36740b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f3679ed0700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f367415da30 0x7f36740b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.447+0000 7f3679ed0700 1 -- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f367415d650 msgr2=0x7f36740b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f3679ed0700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f367415d650 0x7f36740b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f3679ed0700 1 -- 172.21.15.181:0/1279153771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3670009550 con 0x7f36740b39c0 2023-10-16T10:01:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f3679ed0700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36740b39c0 0x7f3674159ce0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f36740cedc0 tx=0x7f3670004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f366a7fc700 1 -- 172.21.15.181:0/1279153771 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670004480 con 0x7f36740b39c0 2023-10-16T10:01:51.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f366a7fc700 1 -- 172.21.15.181:0/1279153771 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36700045e0 con 0x7f36740b39c0 2023-10-16T10:01:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3674160f10 con 0x7f36740b39c0 2023-10-16T10:01:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f366a7fc700 1 -- 172.21.15.181:0/1279153771 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36700038e0 con 0x7f36740b39c0 2023-10-16T10:01:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.448+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36740c7930 con 0x7f36740b39c0 2023-10-16T10:01:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.449+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367415e9b0 con 0x7f36740b39c0 2023-10-16T10:01:51.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.449+0000 7f366a7fc700 1 -- 172.21.15.181:0/1279153771 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f3670021070 con 0x7f36740b39c0 2023-10-16T10:01:51.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.450+0000 7f366a7fc700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 0x7f365806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.450+0000 7f36796cf700 1 -- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 msgr2=0x7f365806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:51.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.450+0000 7f366a7fc700 1 -- 172.21.15.181:0/1279153771 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f367008b910 con 0x7f36740b39c0 2023-10-16T10:01:51.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.450+0000 7f36796cf700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 0x7f365806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:51.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.454+0000 7f366a7fc700 1 -- 172.21.15.181:0/1279153771 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3670056420 con 0x7f36740b39c0 2023-10-16T10:01:51.479 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:51.478+0000 7f52fb5fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:01:51.490 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:51.489+0000 7f715e782200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:01:51.619 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:51.618+0000 7f52fb5fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:01:51.635 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:51.634+0000 7f715e782200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:01:51.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.651+0000 7f36796cf700 1 -- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 msgr2=0x7f365806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:51.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.651+0000 7f36796cf700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 0x7f365806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:51.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.656+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36740c7af0 con 0x7f36740b39c0 2023-10-16T10:01:51.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.657+0000 7f366a7fc700 1 -- 172.21.15.181:0/1279153771 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17807 (secure 0 0 0) 0x7f3670059a40 con 0x7f36740b39c0 2023-10-16T10:01:51.659 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:51.660 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":106,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:01:50.102828+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:00:32.847964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:00:33.365892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1316320487":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/2393260999":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/633839274":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6833/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6832/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1989217127":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1386329415":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/80380984":"2023-10-17T10:00:48.266174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 msgr2=0x7f365806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 0x7f365806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36740b39c0 msgr2=0x7f3674159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36740b39c0 0x7f3674159ce0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f36740cedc0 tx=0x7f3670004750 comp rx=0 tx=0).stop 2023-10-16T10:01:51.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 shutdown_connections 2023-10-16T10:01:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f365806b140 0x7f365806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f367415da30 0x7f36740b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36740b39c0 0x7f3674159ce0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 --2- 172.21.15.181:0/1279153771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f367415d650 0x7f36740b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 >> 172.21.15.181:0/1279153771 conn(0x7f36740665a0 msgr2=0x7f36740cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 shutdown_connections 2023-10-16T10:01:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.660+0000 7f367c134700 1 -- 172.21.15.181:0/1279153771 wait complete. 2023-10-16T10:01:51.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:00:32.847964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:00:33.365892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:01:51.676 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:01:51.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.838+0000 7f8632ab4700 1 Processor -- start 2023-10-16T10:01:51.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.838+0000 7f8632ab4700 1 -- start start 2023-10-16T10:01:51.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8632ab4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f862c0d24e0 0x7f862c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8632ab4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f862c0d0600 0x7f862c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8632ab4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c0d0a50 con 0x7f862c15d9e0 2023-10-16T10:01:51.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8632ab4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c0d20c0 con 0x7f862c0d24e0 2023-10-16T10:01:51.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8632ab4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c0d2960 con 0x7f862c0d0600 2023-10-16T10:01:51.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f862bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f862c0d24e0 0x7f862c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8630850700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f862c15d9e0 0x7f862c0cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56278/0 (socket says 172.21.15.181:56278) 2023-10-16T10:01:51.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8630850700 1 -- 172.21.15.181:0/2784563039 learned_addr learned my addr 172.21.15.181:0/2784563039 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:51.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.839+0000 7f8631051700 1 --2- 172.21.15.181:0/2784563039 >> v2:172.21.15.190:3300/0 conn(0x7f862c0d0600 0x7f862c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862bfff700 1 -- 172.21.15.181:0/2784563039 >> v2:172.21.15.190:3300/0 conn(0x7f862c0d0600 msgr2=0x7f862c0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:51.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862bfff700 1 --2- 172.21.15.181:0/2784563039 >> v2:172.21.15.190:3300/0 conn(0x7f862c0d0600 0x7f862c0b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862bfff700 1 -- 172.21.15.181:0/2784563039 >> v1:172.21.15.181:6789/0 conn(0x7f862c15d9e0 legacy=0x7f862c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862bfff700 1 -- 172.21.15.181:0/2784563039 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0d4180 con 0x7f862c0d24e0 2023-10-16T10:01:51.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862b7fe700 1 -- 172.21.15.181:0/2784563039 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799896251 0 0) 0x7f8620001110 con 0x7f862c15d9e0 2023-10-16T10:01:51.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f8631051700 1 --2- 172.21.15.181:0/2784563039 >> v2:172.21.15.190:3300/0 conn(0x7f862c0d0600 0x7f862c0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:51.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862bfff700 1 --2- 172.21.15.181:0/2784563039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d24e0 0x7f862c0d00c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f862c0d1330 tx=0x7f8618009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542b97cda34966d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862b7fe700 1 -- 172.21.15.181:0/2784563039 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618011830 con 0x7f862c0d24e0 2023-10-16T10:01:51.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862b7fe700 1 -- 172.21.15.181:0/2784563039 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618011990 con 0x7f862c0d24e0 2023-10-16T10:01:51.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f862b7fe700 1 -- 172.21.15.181:0/2784563039 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618011d20 con 0x7f862c0d24e0 2023-10-16T10:01:51.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f8632ab4700 1 -- 172.21.15.181:0/2784563039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d24e0 msgr2=0x7f862c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.840+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2784563039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d24e0 0x7f862c0d00c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f862c0d1330 tx=0x7f8618009b20 comp rx=0 tx=0).stop 2023-10-16T10:01:51.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.841+0000 7f8632ab4700 1 -- 172.21.15.181:0/2784563039 shutdown_connections 2023-10-16T10:01:51.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.841+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2784563039 >> v2:172.21.15.190:3300/0 conn(0x7f862c0d0600 0x7f862c0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.841+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2784563039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d24e0 0x7f862c0d00c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.841+0000 7f8632ab4700 1 -- 172.21.15.181:0/2784563039 >> 172.21.15.181:0/2784563039 conn(0x7f862c09cb40 msgr2=0x7f862c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:51.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.841+0000 7f8632ab4700 1 -- 172.21.15.181:0/2784563039 shutdown_connections 2023-10-16T10:01:51.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.841+0000 7f8632ab4700 1 -- 172.21.15.181:0/2784563039 wait complete. 2023-10-16T10:01:51.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.842+0000 7f8632ab4700 1 Processor -- start 2023-10-16T10:01:51.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.842+0000 7f8632ab4700 1 -- start start 2023-10-16T10:01:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d0600 0x7f862c1f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f862c0d24e0 0x7f862c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f862c15d9e0 0x7f862c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c15a360 con 0x7f862c0d0600 2023-10-16T10:01:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1588c0 con 0x7f862c15d9e0 2023-10-16T10:01:51.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1f3fa0 con 0x7f862c0d24e0 2023-10-16T10:01:51.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d0600 0x7f862c1f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d0600 0x7f862c1f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45206/0 (socket says 172.21.15.181:45206) 2023-10-16T10:01:51.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 -- 172.21.15.181:0/2076055376 learned_addr learned my addr 172.21.15.181:0/2076055376 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:51.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f862bfff700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f862c0d24e0 0x7f862c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8631051700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f862c15d9e0 0x7f862c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:51.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 -- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f862c0d24e0 msgr2=0x7f862c156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f862c0d24e0 0x7f862c156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 -- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f862c15d9e0 msgr2=0x7f862c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:51.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f862c15d9e0 0x7f862c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:51.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 -- 172.21.15.181:0/2076055376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180094a0 con 0x7f862c0d0600 2023-10-16T10:01:51.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8630850700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d0600 0x7f862c1f3a60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f862000c730 tx=0x7f862000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:51.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f86297fa700 1 -- 172.21.15.181:0/2076055376 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8620004c80 con 0x7f862c0d0600 2023-10-16T10:01:51.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f86297fa700 1 -- 172.21.15.181:0/2076055376 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8620004e00 con 0x7f862c0d0600 2023-10-16T10:01:51.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c157cb0 con 0x7f862c0d0600 2023-10-16T10:01:51.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f86297fa700 1 -- 172.21.15.181:0/2076055376 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8620007310 con 0x7f862c0d0600 2023-10-16T10:01:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.843+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c158270 con 0x7f862c0d0600 2023-10-16T10:01:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.844+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862c067d20 con 0x7f862c0d0600 2023-10-16T10:01:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.845+0000 7f86297fa700 1 -- 172.21.15.181:0/2076055376 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8620007470 con 0x7f862c0d0600 2023-10-16T10:01:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.845+0000 7f86297fa700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 0x7f861006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:51.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.845+0000 7f862bfff700 1 -- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 msgr2=0x7f861006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.845+0000 7f86297fa700 1 -- 172.21.15.181:0/2076055376 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f86200897b0 con 0x7f862c0d0600 2023-10-16T10:01:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.846+0000 7f862bfff700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 0x7f861006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:51.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:51.850+0000 7f86297fa700 1 -- 172.21.15.181:0/2076055376 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8620054270 con 0x7f862c0d0600 2023-10-16T10:01:51.942 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:51.941+0000 7f52fb5fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:01:51.973 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:51.971+0000 7f715e782200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:01:52.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.045+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f862c0d0a50 con 0x7f862c0d0600 2023-10-16T10:01:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.045+0000 7f86297fa700 1 -- 172.21.15.181:0/2076055376 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f862c0d0a50 con 0x7f862c0d0600 2023-10-16T10:01:52.047 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:01:52.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.047+0000 7f862bfff700 1 -- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 msgr2=0x7f861006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:52.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.047+0000 7f862bfff700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 0x7f861006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 msgr2=0x7f861006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 0x7f861006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d0600 msgr2=0x7f862c1f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d0600 0x7f862c1f3a60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f862000c730 tx=0x7f862000ca40 comp rx=0 tx=0).stop 2023-10-16T10:01:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 shutdown_connections 2023-10-16T10:01:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f861006b140 0x7f861006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f862c15d9e0 0x7f862c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c0d0600 0x7f862c1f3a60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 --2- 172.21.15.181:0/2076055376 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f862c0d24e0 0x7f862c156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 >> 172.21.15.181:0/2076055376 conn(0x7f862c09cb40 msgr2=0x7f862c15ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 shutdown_connections 2023-10-16T10:01:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.048+0000 7f8632ab4700 1 -- 172.21.15.181:0/2076055376 wait complete. 2023-10-16T10:01:52.058 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:52.057+0000 7f52fb5fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:01:52.063 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:01:52.093 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:52.092+0000 7f715e782200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:01:52.132 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:52.131+0000 7f52fb5fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:01:52.170 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:52.169+0000 7f715e782200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:01:52.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.189+0000 7f933abb4700 1 Processor -- start 2023-10-16T10:01:52.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.189+0000 7f933abb4700 1 -- start start 2023-10-16T10:01:52.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f933abb4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f933415dbd0 0x7f93340d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f933abb4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93340d0610 0x7f93340cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f933abb4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933415dfb0 con 0x7f93340ce2c0 2023-10-16T10:01:52.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f933abb4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93340ce740 con 0x7f93340d0610 2023-10-16T10:01:52.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f933abb4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340cd3a0 con 0x7f933415dbd0 2023-10-16T10:01:52.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9339151700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f93340ce2c0 0x7f9334161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56298/0 (socket says 172.21.15.181:56298) 2023-10-16T10:01:52.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9339151700 1 -- 172.21.15.181:0/3829543910 learned_addr learned my addr 172.21.15.181:0/3829543910 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:52.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9338950700 1 --2- 172.21.15.181:0/3829543910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f933415dbd0 0x7f93340d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9333fff700 1 --2- 172.21.15.181:0/3829543910 >> v2:172.21.15.190:3300/0 conn(0x7f93340d0610 0x7f93340cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f93337fe700 1 -- 172.21.15.181:0/3829543910 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927576747 0 0) 0x7f932c001490 con 0x7f93340ce2c0 2023-10-16T10:01:52.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f93337fe700 1 -- 172.21.15.181:0/3829543910 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93240035b0 con 0x7f93340ce2c0 2023-10-16T10:01:52.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9338950700 1 -- 172.21.15.181:0/3829543910 >> v2:172.21.15.190:3300/0 conn(0x7f93340d0610 msgr2=0x7f93340cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9338950700 1 --2- 172.21.15.181:0/3829543910 >> v2:172.21.15.190:3300/0 conn(0x7f93340d0610 0x7f93340cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9338950700 1 -- 172.21.15.181:0/3829543910 >> v1:172.21.15.181:6789/0 conn(0x7f93340ce2c0 legacy=0x7f9334161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9338950700 1 -- 172.21.15.181:0/3829543910 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340cd830 con 0x7f933415dbd0 2023-10-16T10:01:52.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.190+0000 7f9333fff700 1 --2- 172.21.15.181:0/3829543910 >> v2:172.21.15.190:3300/0 conn(0x7f93340d0610 0x7f93340cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:52.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f9338950700 1 --2- 172.21.15.181:0/3829543910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f933415dbd0 0x7f93340d00d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9328009840 tx=0x7f9328009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9b8255dca5d6766 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f93337fe700 1 -- 172.21.15.181:0/3829543910 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328011830 con 0x7f933415dbd0 2023-10-16T10:01:52.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f93337fe700 1 -- 172.21.15.181:0/3829543910 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9328011990 con 0x7f933415dbd0 2023-10-16T10:01:52.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f93337fe700 1 -- 172.21.15.181:0/3829543910 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328011d20 con 0x7f933415dbd0 2023-10-16T10:01:52.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 -- 172.21.15.181:0/3829543910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f933415dbd0 msgr2=0x7f93340d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 --2- 172.21.15.181:0/3829543910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f933415dbd0 0x7f93340d00d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9328009840 tx=0x7f9328009b50 comp rx=0 tx=0).stop 2023-10-16T10:01:52.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 -- 172.21.15.181:0/3829543910 shutdown_connections 2023-10-16T10:01:52.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 --2- 172.21.15.181:0/3829543910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f933415dbd0 0x7f93340d00d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 --2- 172.21.15.181:0/3829543910 >> v2:172.21.15.190:3300/0 conn(0x7f93340d0610 0x7f93340cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 -- 172.21.15.181:0/3829543910 >> 172.21.15.181:0/3829543910 conn(0x7f933409a930 msgr2=0x7f93340d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 -- 172.21.15.181:0/3829543910 shutdown_connections 2023-10-16T10:01:52.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.191+0000 7f933abb4700 1 -- 172.21.15.181:0/3829543910 wait complete. 2023-10-16T10:01:52.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 Processor -- start 2023-10-16T10:01:52.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 -- start start 2023-10-16T10:01:52.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93340ce2c0 0x7f93341566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93340d0610 0x7f93340b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f933415dbd0 0x7f93340b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341f3250 con 0x7f933415dbd0 2023-10-16T10:01:52.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b4230 con 0x7f93340d0610 2023-10-16T10:01:52.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.193+0000 7f933abb4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b4510 con 0x7f93340ce2c0 2023-10-16T10:01:52.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f933415dbd0 0x7f93340b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f933415dbd0 0x7f93340b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45226/0 (socket says 172.21.15.181:45226) 2023-10-16T10:01:52.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 -- 172.21.15.181:0/1950588313 learned_addr learned my addr 172.21.15.181:0/1950588313 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:52.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9333fff700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93340d0610 0x7f93340b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9338950700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93340ce2c0 0x7f93341566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 -- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93340ce2c0 msgr2=0x7f93341566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93340ce2c0 0x7f93341566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 -- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93340d0610 msgr2=0x7f93340b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93340d0610 0x7f93340b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 -- 172.21.15.181:0/1950588313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200094a0 con 0x7f933415dbd0 2023-10-16T10:01:52.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f9339151700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f933415dbd0 0x7f93340b52d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f932c00cb40 tx=0x7f932c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f93317fa700 1 -- 172.21.15.181:0/1950588313 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00a9e0 con 0x7f933415dbd0 2023-10-16T10:01:52.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f93317fa700 1 -- 172.21.15.181:0/1950588313 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c00ab40 con 0x7f933415dbd0 2023-10-16T10:01:52.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280094a0 con 0x7f933415dbd0 2023-10-16T10:01:52.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f93317fa700 1 -- 172.21.15.181:0/1950588313 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c012400 con 0x7f933415dbd0 2023-10-16T10:01:52.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.194+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340b4ad0 con 0x7f933415dbd0 2023-10-16T10:01:52.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.195+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9334067d20 con 0x7f933415dbd0 2023-10-16T10:01:52.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.196+0000 7f93317fa700 1 -- 172.21.15.181:0/1950588313 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f932c01b3b0 con 0x7f933415dbd0 2023-10-16T10:01:52.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.196+0000 7f93317fa700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 0x7f931806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.196+0000 7f9338950700 1 -- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 msgr2=0x7f931806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:52.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.196+0000 7f93317fa700 1 -- 172.21.15.181:0/1950588313 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f932c089710 con 0x7f933415dbd0 2023-10-16T10:01:52.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.196+0000 7f9338950700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 0x7f931806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:52.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.200+0000 7f93317fa700 1 -- 172.21.15.181:0/1950588313 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f932c0541f0 con 0x7f933415dbd0 2023-10-16T10:01:52.212 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:52.211+0000 7f52fb5fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:01:52.255 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:52.254+0000 7f715e782200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:01:52.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.397+0000 7f9338950700 1 -- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 msgr2=0x7f931806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:52.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.397+0000 7f9338950700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 0x7f931806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:52.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.398+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f933415dfb0 con 0x7f933415dbd0 2023-10-16T10:01:52.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.398+0000 7f93317fa700 1 -- 172.21.15.181:0/1950588313 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f933415dfb0 con 0x7f933415dbd0 2023-10-16T10:01:52.400 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:01:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 msgr2=0x7f931806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 0x7f931806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f933415dbd0 msgr2=0x7f93340b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f933415dbd0 0x7f93340b52d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f932c00cb40 tx=0x7f932c00ce50 comp rx=0 tx=0).stop 2023-10-16T10:01:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 shutdown_connections 2023-10-16T10:01:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f931806b140 0x7f931806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93340ce2c0 0x7f93341566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f933415dbd0 0x7f93340b52d0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 --2- 172.21.15.181:0/1950588313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93340d0610 0x7f93340b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 >> 172.21.15.181:0/1950588313 conn(0x7f933409a930 msgr2=0x7f933415ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 shutdown_connections 2023-10-16T10:01:52.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.401+0000 7f933abb4700 1 -- 172.21.15.181:0/1950588313 wait complete. 2023-10-16T10:01:52.415 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:01:52.579 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:52.578+0000 7f52fb5fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:01:52.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.580+0000 7f2362340700 1 Processor -- start 2023-10-16T10:01:52.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.580+0000 7f2362340700 1 -- start start 2023-10-16T10:01:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.580+0000 7f2362340700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f235c15f150 0x7f235c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f2362340700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f235c0cd170 0x7f235c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f2362340700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c0b2ae0 con 0x7f235c0ce4f0 2023-10-16T10:01:52.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f2362340700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c0ce9a0 con 0x7f235c0cd170 2023-10-16T10:01:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f2362340700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c0b2dc0 con 0x7f235c15f150 2023-10-16T10:01:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f235c0cd170 0x7f235c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f235c0cd170 0x7f235c0b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45268/0 (socket says 172.21.15.181:45268) 2023-10-16T10:01:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f235b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f235c0ce4f0 0x7f235c0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56344/0 (socket says 172.21.15.181:56344) 2023-10-16T10:01:52.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 -- 172.21.15.181:0/1158706141 learned_addr learned my addr 172.21.15.181:0/1158706141 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f235bfff700 1 --2- 172.21.15.181:0/1158706141 >> v2:172.21.15.190:3300/0 conn(0x7f235c15f150 0x7f235c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 -- 172.21.15.181:0/1158706141 >> v2:172.21.15.190:3300/0 conn(0x7f235c15f150 msgr2=0x7f235c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 --2- 172.21.15.181:0/1158706141 >> v2:172.21.15.190:3300/0 conn(0x7f235c15f150 0x7f235c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 -- 172.21.15.181:0/1158706141 >> v1:172.21.15.181:6789/0 conn(0x7f235c0ce4f0 legacy=0x7f235c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 -- 172.21.15.181:0/1158706141 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0cf9d0 con 0x7f235c0cd170 2023-10-16T10:01:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f235affd700 1 -- 172.21.15.181:0/1158706141 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551960361 0 0) 0x7f235c0b2ae0 con 0x7f235c0ce4f0 2023-10-16T10:01:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f23608dd700 1 --2- 172.21.15.181:0/1158706141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0cd170 0x7f235c0b1670 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2354009e50 tx=0x7f235400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448029eee282774d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f235affd700 1 -- 172.21.15.181:0/1158706141 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400e070 con 0x7f235c0cd170 2023-10-16T10:01:52.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f235affd700 1 -- 172.21.15.181:0/1158706141 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235400b8d0 con 0x7f235c0cd170 2023-10-16T10:01:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.581+0000 7f235affd700 1 -- 172.21.15.181:0/1158706141 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400bc80 con 0x7f235c0cd170 2023-10-16T10:01:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 -- 172.21.15.181:0/1158706141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0cd170 msgr2=0x7f235c0b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 --2- 172.21.15.181:0/1158706141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0cd170 0x7f235c0b1670 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2354009e50 tx=0x7f235400b330 comp rx=0 tx=0).stop 2023-10-16T10:01:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 -- 172.21.15.181:0/1158706141 shutdown_connections 2023-10-16T10:01:52.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 --2- 172.21.15.181:0/1158706141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0cd170 0x7f235c0b1670 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 --2- 172.21.15.181:0/1158706141 >> v2:172.21.15.190:3300/0 conn(0x7f235c15f150 0x7f235c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 -- 172.21.15.181:0/1158706141 >> 172.21.15.181:0/1158706141 conn(0x7f235c09a930 msgr2=0x7f235c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 -- 172.21.15.181:0/1158706141 shutdown_connections 2023-10-16T10:01:52.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.582+0000 7f2362340700 1 -- 172.21.15.181:0/1158706141 wait complete. 2023-10-16T10:01:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.583+0000 7f2362340700 1 Processor -- start 2023-10-16T10:01:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f2362340700 1 -- start start 2023-10-16T10:01:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f2362340700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f235c0cd170 0x7f235c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f2362340700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0ce4f0 0x7f235c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f2362340700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f235c15f150 0x7f235c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f2362340700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0c51d0 con 0x7f235c0ce4f0 2023-10-16T10:01:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f2362340700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0c5470 con 0x7f235c15f150 2023-10-16T10:01:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f2362340700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0c5750 con 0x7f235c0cd170 2023-10-16T10:01:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f235b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0ce4f0 0x7f235c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f235b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0ce4f0 0x7f235c0c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45272/0 (socket says 172.21.15.181:45272) 2023-10-16T10:01:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f235b7fe700 1 -- 172.21.15.181:0/952656775 learned_addr learned my addr 172.21.15.181:0/952656775 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f235bfff700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f235c0cd170 0x7f235c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.584+0000 7f23608dd700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f235c15f150 0x7f235c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f235b7fe700 1 -- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f235c0cd170 msgr2=0x7f235c151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f235b7fe700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f235c0cd170 0x7f235c151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f235b7fe700 1 -- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f235c15f150 msgr2=0x7f235c15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f235b7fe700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f235c15f150 0x7f235c15a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f235b7fe700 1 -- 172.21.15.181:0/952656775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2354009b30 con 0x7f235c0ce4f0 2023-10-16T10:01:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f235b7fe700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0ce4f0 0x7f235c0c6ac0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f23500011e0 tx=0x7f235000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f2358ff9700 1 -- 172.21.15.181:0/952656775 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350004c80 con 0x7f235c0ce4f0 2023-10-16T10:01:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c0c5a30 con 0x7f235c0ce4f0 2023-10-16T10:01:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f2358ff9700 1 -- 172.21.15.181:0/952656775 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2350004e00 con 0x7f235c0ce4f0 2023-10-16T10:01:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f2358ff9700 1 -- 172.21.15.181:0/952656775 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350007310 con 0x7f235c0ce4f0 2023-10-16T10:01:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.585+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c151fe0 con 0x7f235c0ce4f0 2023-10-16T10:01:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.586+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c0c5f50 con 0x7f235c0ce4f0 2023-10-16T10:01:52.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.587+0000 7f2358ff9700 1 -- 172.21.15.181:0/952656775 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f2350007470 con 0x7f235c0ce4f0 2023-10-16T10:01:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.587+0000 7f2358ff9700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 0x7f234806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.587+0000 7f235bfff700 1 -- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 msgr2=0x7f234806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.587+0000 7f2358ff9700 1 -- 172.21.15.181:0/952656775 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f235008a030 con 0x7f235c0ce4f0 2023-10-16T10:01:52.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.587+0000 7f235bfff700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 0x7f234806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:52.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.591+0000 7f2358ff9700 1 -- 172.21.15.181:0/952656775 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2350054a60 con 0x7f235c0ce4f0 2023-10-16T10:01:52.646 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:52.645+0000 7f715e782200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:01:52.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.788+0000 7f235bfff700 1 -- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 msgr2=0x7f234806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:52.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.788+0000 7f235bfff700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 0x7f234806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.793+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f235c067d20 con 0x7f235c0ce4f0 2023-10-16T10:01:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.794+0000 7f2358ff9700 1 -- 172.21.15.181:0/952656775 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2350058080 con 0x7f235c0ce4f0 2023-10-16T10:01:52.796 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:01:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 msgr2=0x7f234806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 0x7f234806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0ce4f0 msgr2=0x7f235c0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0ce4f0 0x7f235c0c6ac0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f23500011e0 tx=0x7f235000ca30 comp rx=0 tx=0).stop 2023-10-16T10:01:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 shutdown_connections 2023-10-16T10:01:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f234806b140 0x7f234806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f235c0cd170 0x7f235c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f235c15f150 0x7f235c15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 --2- 172.21.15.181:0/952656775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c0ce4f0 0x7f235c0c6ac0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 >> 172.21.15.181:0/952656775 conn(0x7f235c09a930 msgr2=0x7f235c15d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 shutdown_connections 2023-10-16T10:01:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.796+0000 7f2362340700 1 -- 172.21.15.181:0/952656775 wait complete. 2023-10-16T10:01:52.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:01:52.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.935+0000 7fc357d20700 1 Processor -- start 2023-10-16T10:01:52.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.935+0000 7fc357d20700 1 -- start start 2023-10-16T10:01:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.936+0000 7fc357d20700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc3500cfa60 0x7fc3500d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc357d20700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500d2ac0 con 0x7fc3500d26b0 2023-10-16T10:01:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc357d20700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500d0940 con 0x7fc35015dda0 2023-10-16T10:01:52.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc357d20700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3500d0c20 con 0x7fc3500cfa60 2023-10-16T10:01:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc3562bd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc3500d26b0 0x7fc3500b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56362/0 (socket says 172.21.15.181:56362) 2023-10-16T10:01:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc3562bd700 1 -- 172.21.15.181:0/3860840610 learned_addr learned my addr 172.21.15.181:0/3860840610 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc3552bb700 1 --2- 172.21.15.181:0/3860840610 >> v2:172.21.15.190:3300/0 conn(0x7fc3500cfa60 0x7fc3500d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc354aba700 1 -- 172.21.15.181:0/3860840610 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337878145 0 0) 0x7fc3500d2ac0 con 0x7fc3500d26b0 2023-10-16T10:01:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc354aba700 1 -- 172.21.15.181:0/3860840610 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc33c0035b0 con 0x7fc3500d26b0 2023-10-16T10:01:52.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc3552bb700 1 -- 172.21.15.181:0/3860840610 >> v1:172.21.15.190:6789/0 conn(0x7fc35015dda0 legacy=0x7fc3500d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc3552bb700 1 -- 172.21.15.181:0/3860840610 >> v1:172.21.15.181:6789/0 conn(0x7fc3500d26b0 legacy=0x7fc3500b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc354aba700 1 -- 172.21.15.181:0/3860840610 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3157088685 0 0) 0x7fc33c0035b0 con 0x7fc3500d26b0 2023-10-16T10:01:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc3552bb700 1 -- 172.21.15.181:0/3860840610 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500b2c20 con 0x7fc3500cfa60 2023-10-16T10:01:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc3552bb700 1 --2- 172.21.15.181:0/3860840610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500cfa60 0x7fc3500d2170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc340009840 tx=0x7fc340009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d79cbfb00969e687 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.937+0000 7fc354aba700 1 -- 172.21.15.181:0/3860840610 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480019f0 con 0x7fc3500cfa60 2023-10-16T10:01:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc354aba700 1 -- 172.21.15.181:0/3860840610 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340004030 con 0x7fc3500cfa60 2023-10-16T10:01:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc354aba700 1 -- 172.21.15.181:0/3860840610 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400043c0 con 0x7fc3500cfa60 2023-10-16T10:01:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc357d20700 1 -- 172.21.15.181:0/3860840610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500cfa60 msgr2=0x7fc3500d2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc357d20700 1 --2- 172.21.15.181:0/3860840610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500cfa60 0x7fc3500d2170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc340009840 tx=0x7fc340009b50 comp rx=0 tx=0).stop 2023-10-16T10:01:52.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc357d20700 1 -- 172.21.15.181:0/3860840610 shutdown_connections 2023-10-16T10:01:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc357d20700 1 --2- 172.21.15.181:0/3860840610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500cfa60 0x7fc3500d2170 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc357d20700 1 -- 172.21.15.181:0/3860840610 >> 172.21.15.181:0/3860840610 conn(0x7fc35009cb10 msgr2=0x7fc3500d3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc357d20700 1 -- 172.21.15.181:0/3860840610 shutdown_connections 2023-10-16T10:01:52.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.938+0000 7fc357d20700 1 -- 172.21.15.181:0/3860840610 wait complete. 2023-10-16T10:01:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.939+0000 7fc357d20700 1 Processor -- start 2023-10-16T10:01:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc357d20700 1 -- start start 2023-10-16T10:01:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc357d20700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3500cfa60 0x7fc35015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc357d20700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500d26b0 0x7fc3500ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc357d20700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc35015dda0 0x7fc3500ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc357d20700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35015fa80 con 0x7fc35015dda0 2023-10-16T10:01:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc357d20700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35015ee70 con 0x7fc3500d26b0 2023-10-16T10:01:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc357d20700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35015f150 con 0x7fc3500cfa60 2023-10-16T10:01:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc3562bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc35015dda0 0x7fc3500ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc3562bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc35015dda0 0x7fc3500ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45290/0 (socket says 172.21.15.181:45290) 2023-10-16T10:01:52.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc355abc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3500cfa60 0x7fc35015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc3562bd700 1 -- 172.21.15.181:0/2137722824 learned_addr learned my addr 172.21.15.181:0/2137722824 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:52.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.940+0000 7fc3552bb700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500d26b0 0x7fc3500ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:52.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3562bd700 1 -- 172.21.15.181:0/2137722824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3500cfa60 msgr2=0x7fc35015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3562bd700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3500cfa60 0x7fc35015b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3562bd700 1 -- 172.21.15.181:0/2137722824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500d26b0 msgr2=0x7fc3500ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:52.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3562bd700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500d26b0 0x7fc3500ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:52.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3562bd700 1 -- 172.21.15.181:0/2137722824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400094a0 con 0x7fc35015dda0 2023-10-16T10:01:52.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3562bd700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc35015dda0 0x7fc3500ccff0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc34800cb90 tx=0x7fc34800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:52.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3467fc700 1 -- 172.21.15.181:0/2137722824 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34800aaa0 con 0x7fc35015dda0 2023-10-16T10:01:52.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3467fc700 1 -- 172.21.15.181:0/2137722824 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34800ac00 con 0x7fc35015dda0 2023-10-16T10:01:52.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc3467fc700 1 -- 172.21.15.181:0/2137722824 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480070d0 con 0x7fc35015dda0 2023-10-16T10:01:52.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35015f430 con 0x7fc35015dda0 2023-10-16T10:01:52.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.941+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35015eb60 con 0x7fc35015dda0 2023-10-16T10:01:52.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.943+0000 7fc3467fc700 1 -- 172.21.15.181:0/2137722824 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc348007230 con 0x7fc35015dda0 2023-10-16T10:01:52.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.943+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35004ebb0 con 0x7fc35015dda0 2023-10-16T10:01:52.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.943+0000 7fc3467fc700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 0x7fc33406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:52.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.943+0000 7fc355abc700 1 -- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 msgr2=0x7fc33406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:52.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.943+0000 7fc3467fc700 1 -- 172.21.15.181:0/2137722824 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc348089690 con 0x7fc35015dda0 2023-10-16T10:01:52.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.943+0000 7fc355abc700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 0x7fc33406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:52.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:52.948+0000 7fc3467fc700 1 -- 172.21.15.181:0/2137722824 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc3480540c0 con 0x7fc35015dda0 2023-10-16T10:01:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.144+0000 7fc355abc700 1 -- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 msgr2=0x7fc33406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:53.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.144+0000 7fc355abc700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 0x7fc33406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:53.146 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:53.145+0000 7f52fb5fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:01:53.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.196+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc3500d1f50 con 0x7fc35015dda0 2023-10-16T10:01:53.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.196+0000 7fc3467fc700 1 -- 172.21.15.181:0/2137722824 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v106) v1 ==== 80+0+475 (secure 0 0 0) 0x7fc3480576e0 con 0x7fc35015dda0 2023-10-16T10:01:53.198 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1316320487 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.199 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2393260999 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.199 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/633839274 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.199 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3229300335 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.199 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3229300335 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.199 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1989217127 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.199 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1386329415 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.200 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/80380984 2023-10-17T10:00:48.266174+0000 2023-10-16T10:01:53.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.198+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 msgr2=0x7fc33406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:53.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.198+0000 7fc357d20700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 0x7fc33406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc35015dda0 msgr2=0x7fc3500ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc35015dda0 0x7fc3500ccff0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc34800cb90 tx=0x7fc34800cea0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 shutdown_connections 2023-10-16T10:01:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fc33406b140 0x7fc33406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3500cfa60 0x7fc35015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc35015dda0 0x7fc3500ccff0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 --2- 172.21.15.181:0/2137722824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3500d26b0 0x7fc3500ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 >> 172.21.15.181:0/2137722824 conn(0x7fc35009cb10 msgr2=0x7fc3500b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:53.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 shutdown_connections 2023-10-16T10:01:53.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.199+0000 7fc357d20700 1 -- 172.21.15.181:0/2137722824 wait complete. 2023-10-16T10:01:53.203 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T10:01:53.214 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:01:53.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:53.215 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:01:53.216 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:53.214+0000 7f52fb5fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:01:53.246 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:53.245+0000 7f715e782200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:01:53.263 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T10:01:53.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:01:53.263 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:01:53.320 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:01:53.320 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:53.321 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:53.318+0000 7f715e782200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:01:53.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:01:53.339 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:01:53.339 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:53.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:01:53.380 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:01:53.380 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:01:53.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:01:53.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:53.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.542+0000 7f34014a8700 1 Processor -- start 2023-10-16T10:01:53.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.542+0000 7f34014a8700 1 -- start start 2023-10-16T10:01:53.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f34014a8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161b60 0x7f33fc06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f34014a8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33fc160d20 0x7f33fc1611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f34014a8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33fc06ce20 con 0x7f33fc161f40 2023-10-16T10:01:53.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f34014a8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33fc162800 con 0x7f33fc160d20 2023-10-16T10:01:53.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f34014a8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0d43e0 con 0x7f33fc161b60 2023-10-16T10:01:53.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f33fb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33fc160d20 0x7f33fc1611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f33fa7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f33fc161f40 0x7f33fc0d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56374/0 (socket says 172.21.15.181:56374) 2023-10-16T10:01:53.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f33fa7fc700 1 -- 172.21.15.181:0/1055725506 learned_addr learned my addr 172.21.15.181:0/1055725506 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:53.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f33faffd700 1 --2- 172.21.15.181:0/1055725506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161b60 0x7f33fc06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f33fb7fe700 1 -- 172.21.15.181:0/1055725506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161b60 msgr2=0x7f33fc06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:53.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f33fb7fe700 1 --2- 172.21.15.181:0/1055725506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161b60 0x7f33fc06d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f33fb7fe700 1 -- 172.21.15.181:0/1055725506 >> v1:172.21.15.181:6789/0 conn(0x7f33fc161f40 legacy=0x7f33fc0d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.543+0000 7f33f9ffb700 1 -- 172.21.15.181:0/1055725506 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248570714 0 0) 0x7f33fc06ce20 con 0x7f33fc161f40 2023-10-16T10:01:53.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f33fb7fe700 1 -- 172.21.15.181:0/1055725506 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0d45f0 con 0x7f33fc160d20 2023-10-16T10:01:53.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f33fb7fe700 1 --2- 172.21.15.181:0/1055725506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc1611d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009d90 tx=0x7f33f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185a3f75b89014be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:53.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f33f9ffb700 1 -- 172.21.15.181:0/1055725506 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400e070 con 0x7f33fc160d20 2023-10-16T10:01:53.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f33f9ffb700 1 -- 172.21.15.181:0/1055725506 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f400bb20 con 0x7f33fc160d20 2023-10-16T10:01:53.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f33f9ffb700 1 -- 172.21.15.181:0/1055725506 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4012430 con 0x7f33fc160d20 2023-10-16T10:01:53.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f34014a8700 1 -- 172.21.15.181:0/1055725506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 msgr2=0x7f33fc1611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f34014a8700 1 --2- 172.21.15.181:0/1055725506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc1611d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009d90 tx=0x7f33f400b330 comp rx=0 tx=0).stop 2023-10-16T10:01:53.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f34014a8700 1 -- 172.21.15.181:0/1055725506 shutdown_connections 2023-10-16T10:01:53.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f34014a8700 1 --2- 172.21.15.181:0/1055725506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc1611d0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f34014a8700 1 --2- 172.21.15.181:0/1055725506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161b60 0x7f33fc06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.544+0000 7f34014a8700 1 -- 172.21.15.181:0/1055725506 >> 172.21.15.181:0/1055725506 conn(0x7f33fc09cba0 msgr2=0x7f33fc0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:53.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.545+0000 7f34014a8700 1 -- 172.21.15.181:0/1055725506 shutdown_connections 2023-10-16T10:01:53.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.545+0000 7f34014a8700 1 -- 172.21.15.181:0/1055725506 wait complete. 2023-10-16T10:01:53.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.546+0000 7f34014a8700 1 Processor -- start 2023-10-16T10:01:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.546+0000 7f34014a8700 1 -- start start 2023-10-16T10:01:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33fc161b60 0x7f33fc0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161f40 0x7f33fc0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0c96c0 con 0x7f33fc160d20 2023-10-16T10:01:53.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0c9230 con 0x7f33fc161b60 2023-10-16T10:01:53.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0c8620 con 0x7f33fc161f40 2023-10-16T10:01:53.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc0c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45300/0 (socket says 172.21.15.181:45300) 2023-10-16T10:01:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 -- 172.21.15.181:0/1812107958 learned_addr learned my addr 172.21.15.181:0/1812107958 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33fb7fe700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161f40 0x7f33fc0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33fa7fc700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33fc161b60 0x7f33fc0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 -- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161f40 msgr2=0x7f33fc0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161f40 0x7f33fc0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 -- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33fc161b60 msgr2=0x7f33fc0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33fc161b60 0x7f33fc0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 -- 172.21.15.181:0/1812107958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009a70 con 0x7f33fc160d20 2023-10-16T10:01:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33fa7fc700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33fc161b60 0x7f33fc0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:53.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33faffd700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc0c7c40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f33f000e990 tx=0x7f33f000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33e37fe700 1 -- 172.21.15.181:0/1812107958 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00096f0 con 0x7f33fc160d20 2023-10-16T10:01:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33e37fe700 1 -- 172.21.15.181:0/1812107958 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f000c850 con 0x7f33fc160d20 2023-10-16T10:01:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f33e37fe700 1 -- 172.21.15.181:0/1812107958 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000eee0 con 0x7f33fc160d20 2023-10-16T10:01:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc0c8900 con 0x7f33fc160d20 2023-10-16T10:01:53.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.547+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc0c8ec0 con 0x7f33fc160d20 2023-10-16T10:01:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.548+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc0c9510 con 0x7f33fc160d20 2023-10-16T10:01:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.549+0000 7f33e37fe700 1 -- 172.21.15.181:0/1812107958 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f33f000cbc0 con 0x7f33fc160d20 2023-10-16T10:01:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.550+0000 7f33e37fe700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 0x7f33e806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.550+0000 7f33fa7fc700 1 -- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 msgr2=0x7f33e806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.550+0000 7f33e37fe700 1 -- 172.21.15.181:0/1812107958 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f33f008a150 con 0x7f33fc160d20 2023-10-16T10:01:53.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.550+0000 7f33fa7fc700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 0x7f33e806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:53.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.555+0000 7f33e37fe700 1 -- 172.21.15.181:0/1812107958 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f33f0054b00 con 0x7f33fc160d20 2023-10-16T10:01:53.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.751+0000 7f33fa7fc700 1 -- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 msgr2=0x7f33e806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:53.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.751+0000 7f33fa7fc700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 0x7f33e806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.797+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33fc04ebb0 con 0x7f33fc160d20 2023-10-16T10:01:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.798+0000 7f33e37fe700 1 -- 172.21.15.181:0/1812107958 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f33f0058120 con 0x7f33fc160d20 2023-10-16T10:01:53.800 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:53.800 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1358211190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1358211190},{"type":"v1","addr":"172.21.15.190:6835","nonce":1358211190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6844,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2394829192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2394829192},{"type":"v1","addr":"172.21.15.190:6834","nonce":2394829192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2217231321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2217231321},{"type":"v1","addr":"172.21.15.181:6836","nonce":2217231321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:00:34.411162+0000","modified":"2023-10-16T10:00:37.431652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1158962128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1158962128},{"type":"v1","addr":"172.21.15.181:6837","nonce":1158962128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:01:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 msgr2=0x7f33e806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 0x7f33e806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 msgr2=0x7f33fc0c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc0c7c40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f33f000e990 tx=0x7f33f000eca0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 shutdown_connections 2023-10-16T10:01:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f33e806b140 0x7f33e806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33fc161f40 0x7f33fc0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33fc160d20 0x7f33fc0c7c40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 --2- 172.21.15.181:0/1812107958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33fc161b60 0x7f33fc0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 >> 172.21.15.181:0/1812107958 conn(0x7f33fc09cba0 msgr2=0x7f33fc073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:53.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 shutdown_connections 2023-10-16T10:01:53.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.800+0000 7f34014a8700 1 -- 172.21.15.181:0/1812107958 wait complete. 2023-10-16T10:01:53.805 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 49 2023-10-16T10:01:53.815 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:53.851 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:53.850+0000 7f52fb5fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:53.929 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:53.928+0000 7f52fb5fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:01:53.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.939+0000 7f0d54baa700 1 Processor -- start 2023-10-16T10:01:53.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.940+0000 7f0d54baa700 1 -- start start 2023-10-16T10:01:53.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.940+0000 7f0d54baa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d5006d5f0 0x7f0d500d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.940+0000 7f0d54baa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50160d20 0x7f0d501611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d54baa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d501616f0 con 0x7f0d50161b60 2023-10-16T10:01:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d54baa700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d50162800 con 0x7f0d5006d5f0 2023-10-16T10:01:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d54baa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500d43e0 con 0x7f0d50160d20 2023-10-16T10:01:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50160d20 0x7f0d501611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4e59c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0d50161b60 0x7f0d50161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:40330/0 (socket says 172.21.15.181:40330) 2023-10-16T10:01:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4dd9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d5006d5f0 0x7f0d500d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4e59c700 1 -- 172.21.15.181:0/3722263360 learned_addr learned my addr 172.21.15.181:0/3722263360 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4d59a700 1 -- 172.21.15.181:0/3722263360 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847259170 0 0) 0x7f0d501616f0 con 0x7f0d50161b60 2023-10-16T10:01:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4d59a700 1 -- 172.21.15.181:0/3722263360 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d3c0035b0 con 0x7f0d50161b60 2023-10-16T10:01:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4ed9d700 1 -- 172.21.15.181:0/3722263360 >> v2:172.21.15.190:3300/0 conn(0x7f0d5006d5f0 msgr2=0x7f0d500d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4ed9d700 1 --2- 172.21.15.181:0/3722263360 >> v2:172.21.15.190:3300/0 conn(0x7f0d5006d5f0 0x7f0d500d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4ed9d700 1 -- 172.21.15.181:0/3722263360 >> v1:172.21.15.190:6789/0 conn(0x7f0d50161b60 legacy=0x7f0d50161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4ed9d700 1 -- 172.21.15.181:0/3722263360 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500d2560 con 0x7f0d50160d20 2023-10-16T10:01:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4dd9b700 1 --2- 172.21.15.181:0/3722263360 >> v2:172.21.15.190:3300/0 conn(0x7f0d5006d5f0 0x7f0d500d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4ed9d700 1 --2- 172.21.15.181:0/3722263360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50160d20 0x7f0d501611b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009d90 tx=0x7f0d4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdff2bf19f42ac19 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.941+0000 7f0d4d59a700 1 -- 172.21.15.181:0/3722263360 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800e070 con 0x7f0d50160d20 2023-10-16T10:01:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d4d59a700 1 -- 172.21.15.181:0/3722263360 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4800bb20 con 0x7f0d50160d20 2023-10-16T10:01:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d4d59a700 1 -- 172.21.15.181:0/3722263360 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48012430 con 0x7f0d50160d20 2023-10-16T10:01:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 -- 172.21.15.181:0/3722263360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50160d20 msgr2=0x7f0d501611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3722263360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50160d20 0x7f0d501611b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009d90 tx=0x7f0d4800b330 comp rx=0 tx=0).stop 2023-10-16T10:01:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 -- 172.21.15.181:0/3722263360 shutdown_connections 2023-10-16T10:01:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3722263360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50160d20 0x7f0d501611b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3722263360 >> v2:172.21.15.190:3300/0 conn(0x7f0d5006d5f0 0x7f0d500d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 -- 172.21.15.181:0/3722263360 >> 172.21.15.181:0/3722263360 conn(0x7f0d5009cba0 msgr2=0x7f0d500d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 -- 172.21.15.181:0/3722263360 shutdown_connections 2023-10-16T10:01:53.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.942+0000 7f0d54baa700 1 -- 172.21.15.181:0/3722263360 wait complete. 2023-10-16T10:01:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 Processor -- start 2023-10-16T10:01:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 -- start start 2023-10-16T10:01:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5006d5f0 0x7f0d500b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d50160d20 0x7f0d5015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50161b60 0x7f0d500b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b77d0 con 0x7f0d5006d5f0 2023-10-16T10:01:53.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b9320 con 0x7f0d50160d20 2023-10-16T10:01:53.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d54baa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b9570 con 0x7f0d50161b60 2023-10-16T10:01:53.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d4e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5006d5f0 0x7f0d500b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d4e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5006d5f0 0x7f0d500b8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45316/0 (socket says 172.21.15.181:45316) 2023-10-16T10:01:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d4e59c700 1 -- 172.21.15.181:0/3017527352 learned_addr learned my addr 172.21.15.181:0/3017527352 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d4dd9b700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d50160d20 0x7f0d5015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.944+0000 7f0d4ed9d700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50161b60 0x7f0d500b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d4e59c700 1 -- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50161b60 msgr2=0x7f0d500b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d4e59c700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50161b60 0x7f0d500b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d4e59c700 1 -- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d50160d20 msgr2=0x7f0d5015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d4e59c700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d50160d20 0x7f0d5015beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d4e59c700 1 -- 172.21.15.181:0/3017527352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d400094a0 con 0x7f0d5006d5f0 2023-10-16T10:01:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d4e59c700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5006d5f0 0x7f0d500b8de0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0d38001130 tx=0x7f0d3800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:53.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d46ffd700 1 -- 172.21.15.181:0/3017527352 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38005dc0 con 0x7f0d5006d5f0 2023-10-16T10:01:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d48009a70 con 0x7f0d5006d5f0 2023-10-16T10:01:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d46ffd700 1 -- 172.21.15.181:0/3017527352 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d380070d0 con 0x7f0d5006d5f0 2023-10-16T10:01:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d46ffd700 1 -- 172.21.15.181:0/3017527352 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38007460 con 0x7f0d5006d5f0 2023-10-16T10:01:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.945+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d500b7110 con 0x7f0d5006d5f0 2023-10-16T10:01:53.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.946+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5004ebb0 con 0x7f0d5006d5f0 2023-10-16T10:01:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.946+0000 7f0d46ffd700 1 -- 172.21.15.181:0/3017527352 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f0d380075c0 con 0x7f0d5006d5f0 2023-10-16T10:01:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.947+0000 7f0d46ffd700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 0x7f0d3006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.947+0000 7f0d4dd9b700 1 -- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 msgr2=0x7f0d3006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.947+0000 7f0d46ffd700 1 -- 172.21.15.181:0/3017527352 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f0d38089150 con 0x7f0d5006d5f0 2023-10-16T10:01:53.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.947+0000 7f0d4dd9b700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 0x7f0d3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:53.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:53.951+0000 7f0d46ffd700 1 -- 172.21.15.181:0/3017527352 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0d38053b90 con 0x7f0d5006d5f0 2023-10-16T10:01:53.992 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:53.990+0000 7f715e782200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:54.075 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:54.073+0000 7f715e782200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:01:54.076 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:54.074+0000 7f52fb5fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:54.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.148+0000 7f0d4dd9b700 1 -- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 msgr2=0x7f0d3006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:54.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.148+0000 7f0d4dd9b700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 0x7f0d3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:54.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.158+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d500d0db0 con 0x7f0d5006d5f0 2023-10-16T10:01:54.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.159+0000 7f0d46ffd700 1 -- 172.21.15.181:0/3017527352 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17807 (secure 0 0 0) 0x7f0d380571b0 con 0x7f0d5006d5f0 2023-10-16T10:01:54.161 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:54.162 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":106,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:01:50.102828+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:00:32.847964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:00:33.365892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1316320487":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/2393260999":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/633839274":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6833/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6832/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1989217127":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1386329415":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/80380984":"2023-10-17T10:00:48.266174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:54.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 msgr2=0x7f0d3006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 0x7f0d3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5006d5f0 msgr2=0x7f0d500b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5006d5f0 0x7f0d500b8de0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0d38001130 tx=0x7f0d3800c7b0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 shutdown_connections 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f0d3006b140 0x7f0d3006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d50161b60 0x7f0d500b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5006d5f0 0x7f0d500b8de0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 --2- 172.21.15.181:0/3017527352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d50160d20 0x7f0d5015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 >> 172.21.15.181:0/3017527352 conn(0x7f0d5009cba0 msgr2=0x7f0d500d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 shutdown_connections 2023-10-16T10:01:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.162+0000 7f0d54baa700 1 -- 172.21.15.181:0/3017527352 wait complete. 2023-10-16T10:01:54.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:00:32.847964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:00:33.365892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:01:54.178 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:01:54.230 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:54.228+0000 7f715e782200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:01:54.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 Processor -- start 2023-10-16T10:01:54.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 -- start start 2023-10-16T10:01:54.305 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:54.303+0000 7f52fb5fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:01:54.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18dc0b2010 0x7f18dc0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18dc071be0 0x7f18dc0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc06f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc0ce1f0 con 0x7f18dc071be0 2023-10-16T10:01:54.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc072090 con 0x7f18dc0b2010 2023-10-16T10:01:54.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.303+0000 7f18e16dc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b2930 con 0x7f18dc06f3b0 2023-10-16T10:01:54.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18dc071be0 0x7f18dc0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18dc071be0 0x7f18dc0b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45362/0 (socket says 172.21.15.181:45362) 2023-10-16T10:01:54.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 -- 172.21.15.181:0/4149262823 learned_addr learned my addr 172.21.15.181:0/4149262823 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:54.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18daffd700 1 --2- 172.21.15.181:0/4149262823 >> v2:172.21.15.190:3300/0 conn(0x7f18dc0b2010 0x7f18dc0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18db7fe700 1 --2- 172.21.15.181:0/4149262823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 -- 172.21.15.181:0/4149262823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 msgr2=0x7f18dc06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:54.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 --2- 172.21.15.181:0/4149262823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc06f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 -- 172.21.15.181:0/4149262823 >> v2:172.21.15.190:3300/0 conn(0x7f18dc0b2010 msgr2=0x7f18dc0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:54.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 --2- 172.21.15.181:0/4149262823 >> v2:172.21.15.190:3300/0 conn(0x7f18dc0b2010 0x7f18dc0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 -- 172.21.15.181:0/4149262823 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0ccda0 con 0x7f18dc071be0 2023-10-16T10:01:54.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18da7fc700 1 --2- 172.21.15.181:0/4149262823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc071be0 0x7f18dc0b23f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f18dc070430 tx=0x7f18d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982d5da4340aa6b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18d9ffb700 1 -- 172.21.15.181:0/4149262823 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000f8a0 con 0x7f18dc071be0 2023-10-16T10:01:54.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18d9ffb700 1 -- 172.21.15.181:0/4149262823 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d000fa00 con 0x7f18dc071be0 2023-10-16T10:01:54.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.304+0000 7f18d9ffb700 1 -- 172.21.15.181:0/4149262823 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000fd90 con 0x7f18dc071be0 2023-10-16T10:01:54.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 -- 172.21.15.181:0/4149262823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc071be0 msgr2=0x7f18dc0b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 --2- 172.21.15.181:0/4149262823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc071be0 0x7f18dc0b23f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f18dc070430 tx=0x7f18d0009bd0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 -- 172.21.15.181:0/4149262823 shutdown_connections 2023-10-16T10:01:54.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 --2- 172.21.15.181:0/4149262823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc06f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 --2- 172.21.15.181:0/4149262823 >> v2:172.21.15.190:3300/0 conn(0x7f18dc0b2010 0x7f18dc0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 --2- 172.21.15.181:0/4149262823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc071be0 0x7f18dc0b23f0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 -- 172.21.15.181:0/4149262823 >> 172.21.15.181:0/4149262823 conn(0x7f18dc0cd010 msgr2=0x7f18dc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 -- 172.21.15.181:0/4149262823 shutdown_connections 2023-10-16T10:01:54.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.305+0000 7f18e16dc700 1 -- 172.21.15.181:0/4149262823 wait complete. 2023-10-16T10:01:54.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.306+0000 7f18e16dc700 1 Processor -- start 2023-10-16T10:01:54.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.306+0000 7f18e16dc700 1 -- start start 2023-10-16T10:01:54.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18e16dc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18e16dc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18dc071be0 0x7f18dc14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18e16dc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc0b2010 0x7f18dc1f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18e16dc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1f3430 con 0x7f18dc0b2010 2023-10-16T10:01:54.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18e16dc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc14fa80 con 0x7f18dc071be0 2023-10-16T10:01:54.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18e16dc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc14fd60 con 0x7f18dc06f3b0 2023-10-16T10:01:54.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc0b2010 0x7f18dc1f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc0b2010 0x7f18dc1f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45370/0 (socket says 172.21.15.181:45370) 2023-10-16T10:01:54.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 -- 172.21.15.181:0/243785849 learned_addr learned my addr 172.21.15.181:0/243785849 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:54.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18da7fc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18dc071be0 0x7f18dc14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18daffd700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 -- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 msgr2=0x7f18dc0c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc0c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 -- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18dc071be0 msgr2=0x7f18dc14e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18dc071be0 0x7f18dc14e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18db7fe700 1 -- 172.21.15.181:0/243785849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0009550 con 0x7f18dc0b2010 2023-10-16T10:01:54.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.307+0000 7f18da7fc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18dc071be0 0x7f18dc14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:54.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.308+0000 7f18db7fe700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc0b2010 0x7f18dc1f2650 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f18d400eed0 tx=0x7f18d400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.308+0000 7f18c37fe700 1 -- 172.21.15.181:0/243785849 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4010040 con 0x7f18dc0b2010 2023-10-16T10:01:54.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.308+0000 7f18c37fe700 1 -- 172.21.15.181:0/243785849 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d400b9f0 con 0x7f18dc0b2010 2023-10-16T10:01:54.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.308+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc0c9160 con 0x7f18dc0b2010 2023-10-16T10:01:54.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.308+0000 7f18c37fe700 1 -- 172.21.15.181:0/243785849 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400bda0 con 0x7f18dc0b2010 2023-10-16T10:01:54.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.308+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc0c9720 con 0x7f18dc0b2010 2023-10-16T10:01:54.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.309+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc0b6510 con 0x7f18dc0b2010 2023-10-16T10:01:54.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.309+0000 7f18c37fe700 1 -- 172.21.15.181:0/243785849 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f18d4014400 con 0x7f18dc0b2010 2023-10-16T10:01:54.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.310+0000 7f18c37fe700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 0x7f18c806d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.310+0000 7f18daffd700 1 -- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 msgr2=0x7f18c806d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:54.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.310+0000 7f18c37fe700 1 -- 172.21.15.181:0/243785849 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f18d408a0e0 con 0x7f18dc0b2010 2023-10-16T10:01:54.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.310+0000 7f18daffd700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 0x7f18c806d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:54.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.314+0000 7f18c37fe700 1 -- 172.21.15.181:0/243785849 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f18d4054a00 con 0x7f18dc0b2010 2023-10-16T10:01:54.379 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:54.378+0000 7f52fb5fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:01:54.469 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:54.467+0000 7f715e782200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:01:54.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.511+0000 7f18daffd700 1 -- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 msgr2=0x7f18c806d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:54.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.511+0000 7f18daffd700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 0x7f18c806d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:54.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.514+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18dc0b1760 con 0x7f18dc0b2010 2023-10-16T10:01:54.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.514+0000 7f18c37fe700 1 -- 172.21.15.181:0/243785849 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18d4058020 con 0x7f18dc0b2010 2023-10-16T10:01:54.516 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:01:54.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 msgr2=0x7f18c806d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:54.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 0x7f18c806d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc0b2010 msgr2=0x7f18dc1f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc0b2010 0x7f18dc1f2650 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f18d400eed0 tx=0x7f18d400ef00 comp rx=0 tx=0).stop 2023-10-16T10:01:54.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 shutdown_connections 2023-10-16T10:01:54.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f18c806b170 0x7f18c806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18dc06f3b0 0x7f18dc0c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18dc0b2010 0x7f18dc1f2650 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 --2- 172.21.15.181:0/243785849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18dc071be0 0x7f18dc14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.516+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 >> 172.21.15.181:0/243785849 conn(0x7f18dc0cd010 msgr2=0x7f18dc0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.517+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 shutdown_connections 2023-10-16T10:01:54.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.517+0000 7f18e16dc700 1 -- 172.21.15.181:0/243785849 wait complete. 2023-10-16T10:01:54.531 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:01:54.548 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:54.546+0000 7f715e782200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:01:54.644 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:54.642+0000 7f52fb5fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:01:54.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.659+0000 7f380642c700 1 Processor -- start 2023-10-16T10:01:54.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.660+0000 7f380642c700 1 -- start start 2023-10-16T10:01:54.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f380642c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f38000cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f380642c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f38000ce4f0 0x7f38000b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f380642c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38000b2ae0 con 0x7f38000b14a0 2023-10-16T10:01:54.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f380642c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38000ce970 con 0x7f38000ce4f0 2023-10-16T10:01:54.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f380642c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38000b2dc0 con 0x7f380015f150 2023-10-16T10:01:54.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f38049c9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f38000b14a0 0x7f38000b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56458/0 (socket says 172.21.15.181:56458) 2023-10-16T10:01:54.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f38049c9700 1 -- 172.21.15.181:0/1332301776 learned_addr learned my addr 172.21.15.181:0/1332301776 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:54.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37ff7fe700 1 --2- 172.21.15.181:0/1332301776 >> v2:172.21.15.190:3300/0 conn(0x7f38000ce4f0 0x7f38000b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37fffff700 1 --2- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f38000cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37feffd700 1 -- 172.21.15.181:0/1332301776 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25654608 0 0) 0x7f38000b2ae0 con 0x7f38000b14a0 2023-10-16T10:01:54.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37feffd700 1 -- 172.21.15.181:0/1332301776 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e80035b0 con 0x7f38000b14a0 2023-10-16T10:01:54.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37ff7fe700 1 -- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 msgr2=0x7f38000cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:54.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37ff7fe700 1 --2- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f38000cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37ff7fe700 1 -- 172.21.15.181:0/1332301776 >> v1:172.21.15.181:6789/0 conn(0x7f38000b14a0 legacy=0x7f38000b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37ff7fe700 1 -- 172.21.15.181:0/1332301776 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38000cf9d0 con 0x7f38000ce4f0 2023-10-16T10:01:54.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.661+0000 7f37fffff700 1 --2- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f38000cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:54.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f37ff7fe700 1 --2- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f38000b0f60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009840 tx=0x7f37f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958868e8871fb1a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f37feffd700 1 -- 172.21.15.181:0/1332301776 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4011830 con 0x7f38000ce4f0 2023-10-16T10:01:54.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f37feffd700 1 -- 172.21.15.181:0/1332301776 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f4011990 con 0x7f38000ce4f0 2023-10-16T10:01:54.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f37feffd700 1 -- 172.21.15.181:0/1332301776 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4011d20 con 0x7f38000ce4f0 2023-10-16T10:01:54.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f380642c700 1 -- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 msgr2=0x7f38000b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f380642c700 1 --2- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f38000b0f60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009840 tx=0x7f37f4009b50 comp rx=0 tx=0).stop 2023-10-16T10:01:54.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f380642c700 1 -- 172.21.15.181:0/1332301776 shutdown_connections 2023-10-16T10:01:54.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f380642c700 1 --2- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f38000cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f380642c700 1 --2- 172.21.15.181:0/1332301776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f38000b0f60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.662+0000 7f380642c700 1 -- 172.21.15.181:0/1332301776 >> 172.21.15.181:0/1332301776 conn(0x7f380009a930 msgr2=0x7f38000d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.663+0000 7f380642c700 1 -- 172.21.15.181:0/1332301776 shutdown_connections 2023-10-16T10:01:54.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.663+0000 7f380642c700 1 -- 172.21.15.181:0/1332301776 wait complete. 2023-10-16T10:01:54.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.664+0000 7f380642c700 1 Processor -- start 2023-10-16T10:01:54.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.664+0000 7f380642c700 1 -- start start 2023-10-16T10:01:54.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f380642c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38000b14a0 0x7f38000c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f380642c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f3800151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f380642c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f380015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f380642c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000c8a20 con 0x7f38000b14a0 2023-10-16T10:01:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f380642c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38000c8d00 con 0x7f38000ce4f0 2023-10-16T10:01:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f380642c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38000c8fe0 con 0x7f380015f150 2023-10-16T10:01:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f37fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38000b14a0 0x7f38000c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f37ff7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f3800151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f38049c9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f380015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f37fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38000b14a0 0x7f38000c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45382/0 (socket says 172.21.15.181:45382) 2023-10-16T10:01:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.665+0000 7f37fffff700 1 -- 172.21.15.181:0/2013198021 learned_addr learned my addr 172.21.15.181:0/2013198021 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37ff7fe700 1 -- 172.21.15.181:0/2013198021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 msgr2=0x7f380015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37ff7fe700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f380015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37ff7fe700 1 -- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38000b14a0 msgr2=0x7f38000c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37ff7fe700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38000b14a0 0x7f38000c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37ff7fe700 1 -- 172.21.15.181:0/2013198021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00094a0 con 0x7f38000ce4f0 2023-10-16T10:01:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37ff7fe700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f3800151650 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f37f4016000 tx=0x7f37f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37fcff9700 1 -- 172.21.15.181:0/2013198021 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4004590 con 0x7f38000ce4f0 2023-10-16T10:01:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37fcff9700 1 -- 172.21.15.181:0/2013198021 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f401f070 con 0x7f38000ce4f0 2023-10-16T10:01:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f40094a0 con 0x7f38000ce4f0 2023-10-16T10:01:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f37fcff9700 1 -- 172.21.15.181:0/2013198021 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40038e0 con 0x7f38000ce4f0 2023-10-16T10:01:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.666+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38000c95a0 con 0x7f38000ce4f0 2023-10-16T10:01:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.667+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3800067d20 con 0x7f38000ce4f0 2023-10-16T10:01:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.667+0000 7f37fcff9700 1 -- 172.21.15.181:0/2013198021 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f37f4003a40 con 0x7f38000ce4f0 2023-10-16T10:01:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.668+0000 7f37fcff9700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 0x7f37ec06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.668+0000 7f37fffff700 1 -- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 msgr2=0x7f37ec06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.668+0000 7f37fcff9700 1 -- 172.21.15.181:0/2013198021 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f37f4021070 con 0x7f38000ce4f0 2023-10-16T10:01:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.668+0000 7f37fffff700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 0x7f37ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:54.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.673+0000 7f37fcff9700 1 -- 172.21.15.181:0/2013198021 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f37f4055dc0 con 0x7f38000ce4f0 2023-10-16T10:01:54.829 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:54.827+0000 7f715e782200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:01:54.866 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:54.865+0000 7f52fb5fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:01:54.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.869+0000 7f37fffff700 1 -- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 msgr2=0x7f37ec06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:54.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.869+0000 7f37fffff700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 0x7f37ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:54.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.875+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38000af900 con 0x7f38000ce4f0 2023-10-16T10:01:54.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.875+0000 7f37fcff9700 1 -- 172.21.15.181:0/2013198021 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37f40593e0 con 0x7f38000ce4f0 2023-10-16T10:01:54.877 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:01:54.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 msgr2=0x7f37ec06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 0x7f37ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 msgr2=0x7f3800151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f3800151650 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f37f4016000 tx=0x7f37f4004750 comp rx=0 tx=0).stop 2023-10-16T10:01:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 shutdown_connections 2023-10-16T10:01:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f37ec06b140 0x7f37ec06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f380015f150 0x7f380015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38000b14a0 0x7f38000c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 --2- 172.21.15.181:0/2013198021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38000ce4f0 0x7f3800151650 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 >> 172.21.15.181:0/2013198021 conn(0x7f380009a930 msgr2=0x7f38000ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:54.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 shutdown_connections 2023-10-16T10:01:54.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:54.878+0000 7f380642c700 1 -- 172.21.15.181:0/2013198021 wait complete. 2023-10-16T10:01:54.892 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:01:55.013 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:55.012+0000 7f52fb5fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:01:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.059+0000 7fe776c27700 1 Processor -- start 2023-10-16T10:01:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.059+0000 7fe776c27700 1 -- start start 2023-10-16T10:01:55.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe776c27700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700cb2b0 0x7fe77006f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe776c27700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7700b5fd0 0x7fe77015f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe776c27700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe770070590 con 0x7fe7700cb690 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe776c27700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe770156330 con 0x7fe7700b5fd0 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe776c27700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77006fcf0 con 0x7fe7700cb2b0 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe7751c4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7700b5fd0 0x7fe77015f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe7749c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700cb2b0 0x7fe77006f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe76ffff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe7700cb690 0x7fe7700b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:40410/0 (socket says 172.21.15.181:40410) 2023-10-16T10:01:55.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe76ffff700 1 -- 172.21.15.181:0/3299892658 learned_addr learned my addr 172.21.15.181:0/3299892658 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:55.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe7751c4700 1 -- 172.21.15.181:0/3299892658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700cb2b0 msgr2=0x7fe77006f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:55.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe7751c4700 1 --2- 172.21.15.181:0/3299892658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700cb2b0 0x7fe77006f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe7751c4700 1 -- 172.21.15.181:0/3299892658 >> v1:172.21.15.190:6789/0 conn(0x7fe7700cb690 legacy=0x7fe7700b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.060+0000 7fe7751c4700 1 -- 172.21.15.181:0/3299892658 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77006ffd0 con 0x7fe7700b5fd0 2023-10-16T10:01:55.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe7751c4700 1 --2- 172.21.15.181:0/3299892658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700b5fd0 0x7fe77015f420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae60fa40081cea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe76f7fe700 1 -- 172.21.15.181:0/3299892658 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800e070 con 0x7fe7700b5fd0 2023-10-16T10:01:55.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe76f7fe700 1 -- 172.21.15.181:0/3299892658 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76800bb20 con 0x7fe7700b5fd0 2023-10-16T10:01:55.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe76f7fe700 1 -- 172.21.15.181:0/3299892658 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768012430 con 0x7fe7700b5fd0 2023-10-16T10:01:55.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 -- 172.21.15.181:0/3299892658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700b5fd0 msgr2=0x7fe77015f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 --2- 172.21.15.181:0/3299892658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700b5fd0 0x7fe77015f420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76800b330 comp rx=0 tx=0).stop 2023-10-16T10:01:55.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 -- 172.21.15.181:0/3299892658 shutdown_connections 2023-10-16T10:01:55.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 --2- 172.21.15.181:0/3299892658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700b5fd0 0x7fe77015f420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 --2- 172.21.15.181:0/3299892658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700cb2b0 0x7fe77006f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 -- 172.21.15.181:0/3299892658 >> 172.21.15.181:0/3299892658 conn(0x7fe7700cd010 msgr2=0x7fe7700cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 -- 172.21.15.181:0/3299892658 shutdown_connections 2023-10-16T10:01:55.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.061+0000 7fe776c27700 1 -- 172.21.15.181:0/3299892658 wait complete. 2023-10-16T10:01:55.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 Processor -- start 2023-10-16T10:01:55.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 -- start start 2023-10-16T10:01:55.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700b5fd0 0x7fe7700caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7700cb2b0 0x7fe770154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700cb690 0x7fe7700b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c6a10 con 0x7fe7700cb2b0 2023-10-16T10:01:55.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c60e0 con 0x7fe7700cb690 2023-10-16T10:01:55.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.063+0000 7fe776c27700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c63c0 con 0x7fe7700b5fd0 2023-10-16T10:01:55.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7700cb2b0 0x7fe770154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7700cb2b0 0x7fe770154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45396/0 (socket says 172.21.15.181:45396) 2023-10-16T10:01:55.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 -- 172.21.15.181:0/1081207930 learned_addr learned my addr 172.21.15.181:0/1081207930 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:55.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe7751c4700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700cb690 0x7fe7700b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe7749c3700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700b5fd0 0x7fe7700caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 -- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700b5fd0 msgr2=0x7fe7700caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700b5fd0 0x7fe7700caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 -- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700cb690 msgr2=0x7fe7700b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700cb690 0x7fe7700b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 -- 172.21.15.181:0/1081207930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009a70 con 0x7fe7700cb2b0 2023-10-16T10:01:55.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76ffff700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7700cb2b0 0x7fe770154b70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c480 tx=0x7fe75c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76d7fa700 1 -- 172.21.15.181:0/1081207930 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c005d70 con 0x7fe7700cb2b0 2023-10-16T10:01:55.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76d7fa700 1 -- 172.21.15.181:0/1081207930 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00ce90 con 0x7fe7700cb2b0 2023-10-16T10:01:55.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7700c66a0 con 0x7fe7700cb2b0 2023-10-16T10:01:55.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe76d7fa700 1 -- 172.21.15.181:0/1081207930 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0072d0 con 0x7fe7700cb2b0 2023-10-16T10:01:55.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.064+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770155b90 con 0x7fe7700cb2b0 2023-10-16T10:01:55.075 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:55.065+0000 7f715e782200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:01:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.065+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7700c6b50 con 0x7fe7700cb2b0 2023-10-16T10:01:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.066+0000 7fe76d7fa700 1 -- 172.21.15.181:0/1081207930 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fe75c007430 con 0x7fe7700cb2b0 2023-10-16T10:01:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.066+0000 7fe76d7fa700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 0x7fe75406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.066+0000 7fe7749c3700 1 -- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 msgr2=0x7fe75406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.066+0000 7fe76d7fa700 1 -- 172.21.15.181:0/1081207930 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe75c088fe0 con 0x7fe7700cb2b0 2023-10-16T10:01:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.066+0000 7fe7749c3700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 0x7fe75406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.071+0000 7fe76d7fa700 1 -- 172.21.15.181:0/1081207930 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe75c053a20 con 0x7fe7700cb2b0 2023-10-16T10:01:55.129 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:55.128+0000 7f52fb5fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:01:55.224 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:55.223+0000 7f715e782200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:01:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.267+0000 7fe7749c3700 1 -- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 msgr2=0x7fe75406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:55.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.267+0000 7fe7749c3700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 0x7fe75406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:55.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.270+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7700b5db0 con 0x7fe7700cb2b0 2023-10-16T10:01:55.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.271+0000 7fe76d7fa700 1 -- 172.21.15.181:0/1081207930 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe75c0170a0 con 0x7fe7700cb2b0 2023-10-16T10:01:55.274 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:01:55.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 msgr2=0x7fe75406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:55.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 0x7fe75406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7700cb2b0 msgr2=0x7fe770154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7700cb2b0 0x7fe770154b70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c480 tx=0x7fe75c00c790 comp rx=0 tx=0).stop 2023-10-16T10:01:55.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 shutdown_connections 2023-10-16T10:01:55.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe75406b140 0x7fe75406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7700b5fd0 0x7fe7700caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700cb690 0x7fe7700b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 --2- 172.21.15.181:0/1081207930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7700cb2b0 0x7fe770154b70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 >> 172.21.15.181:0/1081207930 conn(0x7fe7700cd010 msgr2=0x7fe770071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 shutdown_connections 2023-10-16T10:01:55.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.273+0000 7fe776c27700 1 -- 172.21.15.181:0/1081207930 wait complete. 2023-10-16T10:01:55.283 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:55.282+0000 7f52fb5fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:01:55.288 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:55.346 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:55.344+0000 7f715e782200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:01:55.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.410+0000 7f2bfd526700 1 Processor -- start 2023-10-16T10:01:55.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.410+0000 7f2bfd526700 1 -- start start 2023-10-16T10:01:55.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bfd526700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf80cf4e0 0x7f2bf80665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bfd526700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2bf80ca2e0 0x7f2bf80ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bfd526700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf8066c60 con 0x7f2bf815db70 2023-10-16T10:01:55.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bfd526700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf80cda10 con 0x7f2bf80ca2e0 2023-10-16T10:01:55.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bfd526700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf806c1a0 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bf6ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2bf815db70 0x7f2bf80cedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56482/0 (socket says 172.21.15.181:56482) 2023-10-16T10:01:55.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bf6ffd700 1 -- 172.21.15.181:0/3735554684 learned_addr learned my addr 172.21.15.181:0/3735554684 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:55.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bf77fe700 1 --2- 172.21.15.181:0/3735554684 >> v2:172.21.15.181:3300/0 conn(0x7f2bf80ca2e0 0x7f2bf80ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bf67fc700 1 --2- 172.21.15.181:0/3735554684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf80cf4e0 0x7f2bf80665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bf5ffb700 1 -- 172.21.15.181:0/3735554684 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741383804 0 0) 0x7f2bf8066c60 con 0x7f2bf815db70 2023-10-16T10:01:55.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.411+0000 7f2bf5ffb700 1 -- 172.21.15.181:0/3735554684 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be00035b0 con 0x7f2bf815db70 2023-10-16T10:01:55.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf77fe700 1 -- 172.21.15.181:0/3735554684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf80cf4e0 msgr2=0x7f2bf80665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:55.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf77fe700 1 --2- 172.21.15.181:0/3735554684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf80cf4e0 0x7f2bf80665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf77fe700 1 -- 172.21.15.181:0/3735554684 >> v1:172.21.15.181:6789/0 conn(0x7f2bf815db70 legacy=0x7f2bf80cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf77fe700 1 -- 172.21.15.181:0/3735554684 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf806b3e0 con 0x7f2bf80ca2e0 2023-10-16T10:01:55.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf77fe700 1 --2- 172.21.15.181:0/3735554684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80ca2e0 0x7f2bf80ca770 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009d90 tx=0x7f2bf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391e770899b87fd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf5ffb700 1 -- 172.21.15.181:0/3735554684 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000e070 con 0x7f2bf80ca2e0 2023-10-16T10:01:55.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf5ffb700 1 -- 172.21.15.181:0/3735554684 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf000bb20 con 0x7f2bf80ca2e0 2023-10-16T10:01:55.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bf5ffb700 1 -- 172.21.15.181:0/3735554684 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000be90 con 0x7f2bf80ca2e0 2023-10-16T10:01:55.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bfd526700 1 -- 172.21.15.181:0/3735554684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80ca2e0 msgr2=0x7f2bf80ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3735554684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80ca2e0 0x7f2bf80ca770 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009d90 tx=0x7f2bf000b330 comp rx=0 tx=0).stop 2023-10-16T10:01:55.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bfd526700 1 -- 172.21.15.181:0/3735554684 shutdown_connections 2023-10-16T10:01:55.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3735554684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80ca2e0 0x7f2bf80ca770 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3735554684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf80cf4e0 0x7f2bf80665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.412+0000 7f2bfd526700 1 -- 172.21.15.181:0/3735554684 >> 172.21.15.181:0/3735554684 conn(0x7f2bf80b1780 msgr2=0x7f2bf80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.413+0000 7f2bfd526700 1 -- 172.21.15.181:0/3735554684 shutdown_connections 2023-10-16T10:01:55.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.413+0000 7f2bfd526700 1 -- 172.21.15.181:0/3735554684 wait complete. 2023-10-16T10:01:55.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.414+0000 7f2bfd526700 1 Processor -- start 2023-10-16T10:01:55.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.414+0000 7f2bfd526700 1 -- start start 2023-10-16T10:01:55.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf80ca2e0 0x7f2bf814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80cf4e0 0x7f2bf80c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf815db70 0x7f2bf815fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815f950 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815ed40 con 0x7f2bf80ca2e0 2023-10-16T10:01:55.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815f020 con 0x7f2bf815db70 2023-10-16T10:01:55.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80cf4e0 0x7f2bf80c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80cf4e0 0x7f2bf80c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45412/0 (socket says 172.21.15.181:45412) 2023-10-16T10:01:55.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 -- 172.21.15.181:0/3298841889 learned_addr learned my addr 172.21.15.181:0/3298841889 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:55.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf77fe700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf815db70 0x7f2bf815fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf6ffd700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf80ca2e0 0x7f2bf814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 -- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf815db70 msgr2=0x7f2bf815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf815db70 0x7f2bf815fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 -- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf80ca2e0 msgr2=0x7f2bf814f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf80ca2e0 0x7f2bf814f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 -- 172.21.15.181:0/3298841889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0009a70 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bf67fc700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80cf4e0 0x7f2bf80c9510 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e9b0 tx=0x7f2bec00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bdf7fe700 1 -- 172.21.15.181:0/3298841889 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec009710 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bdf7fe700 1 -- 172.21.15.181:0/3298841889 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec00ee50 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf815f300 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bdf7fe700 1 -- 172.21.15.181:0/3298841889 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00ca50 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.415+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf815e960 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.416+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf8067d20 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.417+0000 7f2bdf7fe700 1 -- 172.21.15.181:0/3298841889 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f2bec00cbb0 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.417+0000 7f2bdf7fe700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 0x7f2be406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.417+0000 7f2bf6ffd700 1 -- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 msgr2=0x7f2be406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:55.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.417+0000 7f2bdf7fe700 1 -- 172.21.15.181:0/3298841889 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f2bec089d60 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.417+0000 7f2bf6ffd700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 0x7f2be406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:55.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.422+0000 7f2bdf7fe700 1 -- 172.21.15.181:0/3298841889 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2bec054790 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.512 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:55.510+0000 7f715e782200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:01:55.544 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:55.543+0000 7f52fb5fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:55.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.618+0000 7f2bf6ffd700 1 -- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 msgr2=0x7f2be406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:55.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.618+0000 7f2bf6ffd700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 0x7f2be406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:55.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.666+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bf806b690 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.666+0000 7f2bdf7fe700 1 -- 172.21.15.181:0/3298841889 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f2bec021090 con 0x7f2bf80cf4e0 2023-10-16T10:01:55.668 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:55.669 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1358211190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1358211190},{"type":"v1","addr":"172.21.15.190:6835","nonce":1358211190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6844,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2394829192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2394829192},{"type":"v1","addr":"172.21.15.190:6834","nonce":2394829192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2217231321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2217231321},{"type":"v1","addr":"172.21.15.181:6836","nonce":2217231321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:00:34.411162+0000","modified":"2023-10-16T10:00:37.431652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1158962128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1158962128},{"type":"v1","addr":"172.21.15.181:6837","nonce":1158962128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:01:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 msgr2=0x7f2be406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 0x7f2be406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80cf4e0 msgr2=0x7f2bf80c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80cf4e0 0x7f2bf80c9510 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e9b0 tx=0x7f2bec00ecc0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 shutdown_connections 2023-10-16T10:01:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f2be406b140 0x7f2be406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bf815db70 0x7f2bf815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bf80ca2e0 0x7f2bf814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 --2- 172.21.15.181:0/3298841889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf80cf4e0 0x7f2bf80c9510 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 >> 172.21.15.181:0/3298841889 conn(0x7f2bf80b1780 msgr2=0x7f2bf806e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 shutdown_connections 2023-10-16T10:01:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.670+0000 7f2bfd526700 1 -- 172.21.15.181:0/3298841889 wait complete. 2023-10-16T10:01:55.675 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 49 2023-10-16T10:01:55.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:55.704 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:55.703+0000 7f52fb5fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:01:55.794 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:55.793+0000 7f715e782200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:01:55.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.813+0000 7f40cebd1700 1 Processor -- start 2023-10-16T10:01:55.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.814+0000 7f40cebd1700 1 -- start start 2023-10-16T10:01:55.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.814+0000 7f40cebd1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c80cdb80 0x7f40c80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.814+0000 7f40cebd1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c806f870 con 0x7f40c80af450 2023-10-16T10:01:55.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.814+0000 7f40cebd1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c80d1160 con 0x7f40c815dfd0 2023-10-16T10:01:55.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.814+0000 7f40cebd1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80cc8e0 con 0x7f40c80cdb80 2023-10-16T10:01:55.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.814+0000 7f40c7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f40c80af450 0x7f40c80cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56498/0 (socket says 172.21.15.181:56498) 2023-10-16T10:01:55.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c7fff700 1 -- 172.21.15.181:0/9655764 learned_addr learned my addr 172.21.15.181:0/9655764 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:55.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40cd16e700 1 --2- 172.21.15.181:0/9655764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c80cdb80 0x7f40c80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c77fe700 1 -- 172.21.15.181:0/9655764 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046974101 0 0) 0x7f40c806f870 con 0x7f40c80af450 2023-10-16T10:01:55.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c77fe700 1 -- 172.21.15.181:0/9655764 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40b80035b0 con 0x7f40c80af450 2023-10-16T10:01:55.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c77fe700 1 -- 172.21.15.181:0/9655764 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364804427 0 0) 0x7f40c80d1160 con 0x7f40c815dfd0 2023-10-16T10:01:55.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c77fe700 1 -- 172.21.15.181:0/9655764 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40c806f870 con 0x7f40c815dfd0 2023-10-16T10:01:55.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40cd16e700 1 -- 172.21.15.181:0/9655764 >> v1:172.21.15.190:6789/0 conn(0x7f40c815dfd0 legacy=0x7f40c806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40cd16e700 1 -- 172.21.15.181:0/9655764 >> v1:172.21.15.181:6789/0 conn(0x7f40c80af450 legacy=0x7f40c80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40cd16e700 1 -- 172.21.15.181:0/9655764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c80ccbc0 con 0x7f40c80cdb80 2023-10-16T10:01:55.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40cd16e700 1 --2- 172.21.15.181:0/9655764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c80cdb80 0x7f40c80af9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009dc0 tx=0x7f40c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e1a83ce45395c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c77fe700 1 -- 172.21.15.181:0/9655764 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c000e070 con 0x7f40c80cdb80 2023-10-16T10:01:55.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c77fe700 1 -- 172.21.15.181:0/9655764 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c000bb20 con 0x7f40c80cdb80 2023-10-16T10:01:55.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.815+0000 7f40c77fe700 1 -- 172.21.15.181:0/9655764 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0012430 con 0x7f40c80cdb80 2023-10-16T10:01:55.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.816+0000 7f40cebd1700 1 -- 172.21.15.181:0/9655764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c80cdb80 msgr2=0x7f40c80af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.816+0000 7f40cebd1700 1 --2- 172.21.15.181:0/9655764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c80cdb80 0x7f40c80af9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009dc0 tx=0x7f40c000b330 comp rx=0 tx=0).stop 2023-10-16T10:01:55.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.816+0000 7f40cebd1700 1 -- 172.21.15.181:0/9655764 shutdown_connections 2023-10-16T10:01:55.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.816+0000 7f40cebd1700 1 --2- 172.21.15.181:0/9655764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c80cdb80 0x7f40c80af9a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.816+0000 7f40cebd1700 1 -- 172.21.15.181:0/9655764 >> 172.21.15.181:0/9655764 conn(0x7f40c80d04a0 msgr2=0x7f40c80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:55.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.816+0000 7f40cebd1700 1 -- 172.21.15.181:0/9655764 shutdown_connections 2023-10-16T10:01:55.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.816+0000 7f40cebd1700 1 -- 172.21.15.181:0/9655764 wait complete. 2023-10-16T10:01:55.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.817+0000 7f40cebd1700 1 Processor -- start 2023-10-16T10:01:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.817+0000 7f40cebd1700 1 -- start start 2023-10-16T10:01:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cebd1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c80af450 0x7f40c80b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cebd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c80cdb80 0x7f40c80b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cebd1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c815dfd0 0x7f40c814a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cebd1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8148400 con 0x7f40c80cdb80 2023-10-16T10:01:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cebd1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80b44e0 con 0x7f40c80af450 2023-10-16T10:01:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cebd1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80b47c0 con 0x7f40c815dfd0 2023-10-16T10:01:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c80cdb80 0x7f40c80b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c80cdb80 0x7f40c80b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45428/0 (socket says 172.21.15.181:45428) 2023-10-16T10:01:55.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 -- 172.21.15.181:0/4107392938 learned_addr learned my addr 172.21.15.181:0/4107392938 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:55.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cd16e700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c815dfd0 0x7f40c814a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40cc96d700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c80af450 0x7f40c80b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:55.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 -- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c815dfd0 msgr2=0x7f40c814a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c815dfd0 0x7f40c814a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 -- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c80af450 msgr2=0x7f40c80b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c80af450 0x7f40c80b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.818+0000 7f40c7fff700 1 -- 172.21.15.181:0/4107392938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0009a70 con 0x7f40c80cdb80 2023-10-16T10:01:55.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.819+0000 7f40c7fff700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c80cdb80 0x7f40c80b5580 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f40b400c700 tx=0x7f40b400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.819+0000 7f40c57fa700 1 -- 172.21.15.181:0/4107392938 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4005d70 con 0x7f40c80cdb80 2023-10-16T10:01:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.819+0000 7f40c57fa700 1 -- 172.21.15.181:0/4107392938 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b40070d0 con 0x7f40c80cdb80 2023-10-16T10:01:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.819+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c80b4aa0 con 0x7f40c80cdb80 2023-10-16T10:01:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.819+0000 7f40c57fa700 1 -- 172.21.15.181:0/4107392938 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4007460 con 0x7f40c80cdb80 2023-10-16T10:01:55.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.819+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c80b5060 con 0x7f40c80cdb80 2023-10-16T10:01:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.820+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c80b5250 con 0x7f40c80cdb80 2023-10-16T10:01:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.821+0000 7f40c57fa700 1 -- 172.21.15.181:0/4107392938 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f40b40075c0 con 0x7f40c80cdb80 2023-10-16T10:01:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.821+0000 7f40c57fa700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 0x7f40ac06d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:55.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.821+0000 7f40cc96d700 1 -- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 msgr2=0x7f40ac06d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.821+0000 7f40c57fa700 1 -- 172.21.15.181:0/4107392938 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f40b40890b0 con 0x7f40c80cdb80 2023-10-16T10:01:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.821+0000 7f40cc96d700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 0x7f40ac06d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:55.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:55.829+0000 7f40c57fa700 1 -- 172.21.15.181:0/4107392938 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f40b4053b70 con 0x7f40c80cdb80 2023-10-16T10:01:55.966 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:55.965+0000 7f715e782200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:01:56.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.022+0000 7f40cc96d700 1 -- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 msgr2=0x7f40ac06d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:56.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.022+0000 7f40cc96d700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 0x7f40ac06d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:56.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.030+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40c8067d20 con 0x7f40c80cdb80 2023-10-16T10:01:56.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.031+0000 7f40c57fa700 1 -- 172.21.15.181:0/4107392938 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17807 (secure 0 0 0) 0x7f40b4017090 con 0x7f40c80cdb80 2023-10-16T10:01:56.033 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:56.033 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":106,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:01:50.102828+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:00:32.847964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:00:33.365892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1316320487":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/2393260999":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/633839274":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6833/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6832/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1989217127":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1386329415":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/80380984":"2023-10-17T10:00:48.266174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:56.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 msgr2=0x7f40ac06d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:56.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 0x7f40ac06d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c80cdb80 msgr2=0x7f40c80b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c80cdb80 0x7f40c80b5580 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f40b400c700 tx=0x7f40b400ca10 comp rx=0 tx=0).stop 2023-10-16T10:01:56.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 shutdown_connections 2023-10-16T10:01:56.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7f40ac06b170 0x7f40ac06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40c815dfd0 0x7f40c814a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40c80af450 0x7f40c80b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 --2- 172.21.15.181:0/4107392938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c80cdb80 0x7f40c80b5580 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 >> 172.21.15.181:0/4107392938 conn(0x7f40c80d04a0 msgr2=0x7f40c80aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 shutdown_connections 2023-10-16T10:01:56.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.033+0000 7f40cebd1700 1 -- 172.21.15.181:0/4107392938 wait complete. 2023-10-16T10:01:56.047 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:01:56.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:56.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe30d072700 1 Processor -- start 2023-10-16T10:01:56.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe30d072700 1 -- start start 2023-10-16T10:01:56.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe30d072700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe30815dd00 0x7fe308071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe30d072700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe308071870 0x7fe3081574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe30d072700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe308071df0 con 0x7fe30815e0e0 2023-10-16T10:01:56.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe30d072700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3080720d0 con 0x7fe30815dd00 2023-10-16T10:01:56.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe30d072700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3080723b0 con 0x7fe308071870 2023-10-16T10:01:56.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.169+0000 7fe306d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe30815dd00 0x7fe308071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe30759e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe30815e0e0 0x7fe30815a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56518/0 (socket says 172.21.15.181:56518) 2023-10-16T10:01:56.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe306d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe30815dd00 0x7fe308071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45436/0 (socket says 172.21.15.181:45436) 2023-10-16T10:01:56.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe30759e700 1 -- 172.21.15.181:0/3122845725 learned_addr learned my addr 172.21.15.181:0/3122845725 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:56.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe30659c700 1 --2- 172.21.15.181:0/3122845725 >> v2:172.21.15.190:3300/0 conn(0x7fe308071870 0x7fe3081574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe306d9d700 1 -- 172.21.15.181:0/3122845725 >> v2:172.21.15.190:3300/0 conn(0x7fe308071870 msgr2=0x7fe3081574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:56.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe306d9d700 1 --2- 172.21.15.181:0/3122845725 >> v2:172.21.15.190:3300/0 conn(0x7fe308071870 0x7fe3081574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe306d9d700 1 -- 172.21.15.181:0/3122845725 >> v1:172.21.15.181:6789/0 conn(0x7fe30815e0e0 legacy=0x7fe30815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe305d9b700 1 -- 172.21.15.181:0/3122845725 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033510234 0 0) 0x7fe308071df0 con 0x7fe30815e0e0 2023-10-16T10:01:56.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe306d9d700 1 -- 172.21.15.181:0/3122845725 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308072690 con 0x7fe30815dd00 2023-10-16T10:01:56.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe306d9d700 1 --2- 172.21.15.181:0/3122845725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30815dd00 0x7fe308071330 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe30806ebd0 tx=0x7fe2f8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e06e66d55fac6d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe305d9b700 1 -- 172.21.15.181:0/3122845725 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8011830 con 0x7fe30815dd00 2023-10-16T10:01:56.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe305d9b700 1 -- 172.21.15.181:0/3122845725 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f8011990 con 0x7fe30815dd00 2023-10-16T10:01:56.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.170+0000 7fe305d9b700 1 -- 172.21.15.181:0/3122845725 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8011d20 con 0x7fe30815dd00 2023-10-16T10:01:56.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 -- 172.21.15.181:0/3122845725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30815dd00 msgr2=0x7fe308071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 --2- 172.21.15.181:0/3122845725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30815dd00 0x7fe308071330 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe30806ebd0 tx=0x7fe2f8009b20 comp rx=0 tx=0).stop 2023-10-16T10:01:56.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 -- 172.21.15.181:0/3122845725 shutdown_connections 2023-10-16T10:01:56.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 --2- 172.21.15.181:0/3122845725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30815dd00 0x7fe308071330 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 --2- 172.21.15.181:0/3122845725 >> v2:172.21.15.190:3300/0 conn(0x7fe308071870 0x7fe3081574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 -- 172.21.15.181:0/3122845725 >> 172.21.15.181:0/3122845725 conn(0x7fe30806aed0 msgr2=0x7fe3080b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 -- 172.21.15.181:0/3122845725 shutdown_connections 2023-10-16T10:01:56.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.171+0000 7fe30d072700 1 -- 172.21.15.181:0/3122845725 wait complete. 2023-10-16T10:01:56.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.173+0000 7fe30d072700 1 Processor -- start 2023-10-16T10:01:56.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.173+0000 7fe30d072700 1 -- start start 2023-10-16T10:01:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30d072700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe308071870 0x7fe308145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30d072700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe30815dd00 0x7fe30814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30d072700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30815e0e0 0x7fe30814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30d072700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30814cd30 con 0x7fe308071870 2023-10-16T10:01:56.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30d072700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30814c7e0 con 0x7fe30815e0e0 2023-10-16T10:01:56.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30d072700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30814ca50 con 0x7fe30815dd00 2023-10-16T10:01:56.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe308071870 0x7fe308145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe308071870 0x7fe308145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45442/0 (socket says 172.21.15.181:45442) 2023-10-16T10:01:56.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 -- 172.21.15.181:0/1916303998 learned_addr learned my addr 172.21.15.181:0/1916303998 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:56.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30759e700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30815e0e0 0x7fe30814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30659c700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe30815dd00 0x7fe30814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 -- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe30815dd00 msgr2=0x7fe30814bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe30815dd00 0x7fe30814bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 -- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30815e0e0 msgr2=0x7fe30814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30815e0e0 0x7fe30814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 -- 172.21.15.181:0/1916303998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f80094a0 con 0x7fe308071870 2023-10-16T10:01:56.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30659c700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe30815dd00 0x7fe30814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:56.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe30759e700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30815e0e0 0x7fe30814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:56.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe306d9d700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe308071870 0x7fe308145dc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fe30806edb0 tx=0x7fe2f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.174+0000 7fe2ef7fe700 1 -- 172.21.15.181:0/1916303998 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8004550 con 0x7fe308071870 2023-10-16T10:01:56.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.175+0000 7fe2ef7fe700 1 -- 172.21.15.181:0/1916303998 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f801f070 con 0x7fe308071870 2023-10-16T10:01:56.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.175+0000 7fe2ef7fe700 1 -- 172.21.15.181:0/1916303998 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80038e0 con 0x7fe308071870 2023-10-16T10:01:56.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.175+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3080c3060 con 0x7fe308071870 2023-10-16T10:01:56.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.175+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3080c3620 con 0x7fe308071870 2023-10-16T10:01:56.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.176+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30804ebb0 con 0x7fe308071870 2023-10-16T10:01:56.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.176+0000 7fe2ef7fe700 1 -- 172.21.15.181:0/1916303998 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 60) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fe2f8003a40 con 0x7fe308071870 2023-10-16T10:01:56.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.176+0000 7fe2ef7fe700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 0x7fe2f406d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.176+0000 7fe30659c700 1 -- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 msgr2=0x7fe2f406d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:56.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.176+0000 7fe2ef7fe700 1 -- 172.21.15.181:0/1916303998 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fe2f8021070 con 0x7fe308071870 2023-10-16T10:01:56.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.176+0000 7fe30659c700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 0x7fe2f406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:01:56.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.181+0000 7fe2ef7fe700 1 -- 172.21.15.181:0/1916303998 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2f8055f50 con 0x7fe308071870 2023-10-16T10:01:56.232 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:01:56.231+0000 7f52fb5fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:01:56.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.377+0000 7fe30659c700 1 -- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 msgr2=0x7fe2f406d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/932006106 2023-10-16T10:01:56.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.377+0000 7fe30659c700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 0x7fe2f406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:01:56.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.428+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3080cd700 con 0x7fe308071870 2023-10-16T10:01:56.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.429+0000 7fe2ef7fe700 1 -- 172.21.15.181:0/1916303998 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fe2f8078070 con 0x7fe308071870 2023-10-16T10:01:56.431 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:56.431 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1358211190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1358211190},{"type":"v1","addr":"172.21.15.190:6835","nonce":1358211190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6844,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2394829192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2394829192},{"type":"v1","addr":"172.21.15.190:6834","nonce":2394829192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2217231321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2217231321},{"type":"v1","addr":"172.21.15.181:6836","nonce":2217231321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:00:34.411162+0000","modified":"2023-10-16T10:00:37.431652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1158962128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1158962128},{"type":"v1","addr":"172.21.15.181:6837","nonce":1158962128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:01:56.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 msgr2=0x7fe2f406d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:01:56.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 0x7fe2f406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe308071870 msgr2=0x7fe308145dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe308071870 0x7fe308145dc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fe30806edb0 tx=0x7fe2f8004750 comp rx=0 tx=0).stop 2023-10-16T10:01:56.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 shutdown_connections 2023-10-16T10:01:56.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:6832/932006106,v1:172.21.15.181:6833/932006106] conn(0x7fe2f406b170 0x7fe2f406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe30815e0e0 0x7fe30814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe308071870 0x7fe308145dc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 --2- 172.21.15.181:0/1916303998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe30815dd00 0x7fe30814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 >> 172.21.15.181:0/1916303998 conn(0x7fe30806aed0 msgr2=0x7fe3080b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 shutdown_connections 2023-10-16T10:01:56.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.431+0000 7fe30d072700 1 -- 172.21.15.181:0/1916303998 wait complete. 2023-10-16T10:01:56.436 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 49 2023-10-16T10:01:56.446 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:01:56.524 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.522+0000 7f715e782200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:01:56.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.573+0000 7fc869fa1700 1 Processor -- start 2023-10-16T10:01:56.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.573+0000 7fc869fa1700 1 -- start start 2023-10-16T10:01:56.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.573+0000 7fc869fa1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640ce7f0 0x7fc86406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.573+0000 7fc869fa1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8640d2390 0x7fc8640cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.573+0000 7fc869fa1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8640cc470 con 0x7fc8640d0180 2023-10-16T10:01:56.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.573+0000 7fc869fa1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8640d0630 con 0x7fc8640d2390 2023-10-16T10:01:56.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.573+0000 7fc869fa1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86406ca30 con 0x7fc8640ce7f0 2023-10-16T10:01:56.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8637fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640ce7f0 0x7fc86406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc862ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8640d2390 0x7fc8640cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8637fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640ce7f0 0x7fc86406c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:42498/0 (socket says 172.21.15.181:42498) 2023-10-16T10:01:56.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc863fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc8640d0180 0x7fc864161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:40496/0 (socket says 172.21.15.181:40496) 2023-10-16T10:01:56.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc862ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8640d2390 0x7fc8640cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:50524/0 (socket says 172.21.15.181:50524) 2023-10-16T10:01:56.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8637fe700 1 -- 172.21.15.181:0/2899772090 learned_addr learned my addr 172.21.15.181:0/2899772090 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:56.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc862ffd700 1 -- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640ce7f0 msgr2=0x7fc86406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:56.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc862ffd700 1 --2- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640ce7f0 0x7fc86406c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc862ffd700 1 -- 172.21.15.181:0/2899772090 >> v1:172.21.15.190:6789/0 conn(0x7fc8640d0180 legacy=0x7fc864161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc862ffd700 1 -- 172.21.15.181:0/2899772090 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86406cd10 con 0x7fc8640d2390 2023-10-16T10:01:56.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8627fc700 1 -- 172.21.15.181:0/2899772090 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883654411 0 0) 0x7fc8640cc470 con 0x7fc8640d0180 2023-10-16T10:01:56.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8637fe700 1 --2- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640ce7f0 0x7fc86406c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:56.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc862ffd700 1 --2- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640d2390 0x7fc8640cfc40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc858009840 tx=0x7fc858009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fab4f73b12d639c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8627fc700 1 -- 172.21.15.181:0/2899772090 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858011830 con 0x7fc8640d2390 2023-10-16T10:01:56.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8627fc700 1 -- 172.21.15.181:0/2899772090 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858011990 con 0x7fc8640d2390 2023-10-16T10:01:56.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.574+0000 7fc8627fc700 1 -- 172.21.15.181:0/2899772090 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858011d20 con 0x7fc8640d2390 2023-10-16T10:01:56.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 -- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640d2390 msgr2=0x7fc8640cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 --2- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640d2390 0x7fc8640cfc40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc858009840 tx=0x7fc858009b50 comp rx=0 tx=0).stop 2023-10-16T10:01:56.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 -- 172.21.15.181:0/2899772090 shutdown_connections 2023-10-16T10:01:56.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 --2- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640ce7f0 0x7fc86406c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 --2- 172.21.15.181:0/2899772090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640d2390 0x7fc8640cfc40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 -- 172.21.15.181:0/2899772090 >> 172.21.15.181:0/2899772090 conn(0x7fc86409a9a0 msgr2=0x7fc8640d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 -- 172.21.15.181:0/2899772090 shutdown_connections 2023-10-16T10:01:56.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.575+0000 7fc869fa1700 1 -- 172.21.15.181:0/2899772090 wait complete. 2023-10-16T10:01:56.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.576+0000 7fc869fa1700 1 Processor -- start 2023-10-16T10:01:56.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.577+0000 7fc869fa1700 1 -- start start 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.577+0000 7fc869fa1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640ce7f0 0x7fc8641f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.577+0000 7fc869fa1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8640d0180 0x7fc8641f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.577+0000 7fc869fa1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640d2390 0x7fc86415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.577+0000 7fc869fa1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640ca650 con 0x7fc8640d0180 2023-10-16T10:01:56.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.577+0000 7fc869fa1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86415d770 con 0x7fc8640ce7f0 2023-10-16T10:01:56.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.577+0000 7fc869fa1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640c9d20 con 0x7fc8640d2390 2023-10-16T10:01:56.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8640d0180 0x7fc8641f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8640d0180 0x7fc8641f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45480/0 (socket says 172.21.15.181:45480) 2023-10-16T10:01:56.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 -- 172.21.15.181:0/192841731 learned_addr learned my addr 172.21.15.181:0/192841731 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:56.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc863fff700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640d2390 0x7fc86415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc8637fe700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640ce7f0 0x7fc8641f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 -- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640d2390 msgr2=0x7fc86415caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640d2390 0x7fc86415caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 -- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640ce7f0 msgr2=0x7fc8641f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640ce7f0 0x7fc8641f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 -- 172.21.15.181:0/192841731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc854009500 con 0x7fc8640d0180 2023-10-16T10:01:56.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc862ffd700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8640d0180 0x7fc8641f3d40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc858009fd0 tx=0x7fc858004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc84bfff700 1 -- 172.21.15.181:0/192841731 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8580036a0 con 0x7fc8640d0180 2023-10-16T10:01:56.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc84bfff700 1 -- 172.21.15.181:0/192841731 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85801f070 con 0x7fc8640d0180 2023-10-16T10:01:56.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8580094a0 con 0x7fc8640d0180 2023-10-16T10:01:56.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc84bfff700 1 -- 172.21.15.181:0/192841731 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858003a20 con 0x7fc8640d0180 2023-10-16T10:01:56.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.578+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8640ca2e0 con 0x7fc8640d0180 2023-10-16T10:01:56.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.579+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8640ca800 con 0x7fc8640d0180 2023-10-16T10:01:56.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.580+0000 7fc84bfff700 1 -- 172.21.15.181:0/192841731 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 61) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fc858003b80 con 0x7fc8640d0180 2023-10-16T10:01:56.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.580+0000 7fc84bfff700 1 -- 172.21.15.181:0/192841731 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6936+0+0 (secure 0 0 0) 0x7fc858021070 con 0x7fc8640d0180 2023-10-16T10:01:56.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.585+0000 7fc84bfff700 1 -- 172.21.15.181:0/192841731 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc8640ca800 con 0x7fc8640d0180 2023-10-16T10:01:56.608 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d9fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.609 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d9fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.609 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d9fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.609 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d9fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.609 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d9fae700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.610 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d6f68700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.610 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d6f68700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.610 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d6f68700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.610 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d6f68700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.611 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:01:56.607+0000 7f70d6f68700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:01:56.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.828+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc864067d20 con 0x7fc8640d0180 2023-10-16T10:01:56.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.828+0000 7fc84bfff700 1 -- 172.21.15.181:0/192841731 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fc85802b090 con 0x7fc8640d0180 2023-10-16T10:01:56.830 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:56.831 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1358211190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1358211190},{"type":"v1","addr":"172.21.15.190:6835","nonce":1358211190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6844,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2394829192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2394829192},{"type":"v1","addr":"172.21.15.190:6834","nonce":2394829192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2217231321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2217231321},{"type":"v1","addr":"172.21.15.181:6836","nonce":2217231321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:00:34.411162+0000","modified":"2023-10-16T10:00:37.431652+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1158962128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1158962128},{"type":"v1","addr":"172.21.15.181:6837","nonce":1158962128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-16T10:01:56.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8640d0180 msgr2=0x7fc8641f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8640d0180 0x7fc8641f3d40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc858009fd0 tx=0x7fc858004750 comp rx=0 tx=0).stop 2023-10-16T10:01:56.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 shutdown_connections 2023-10-16T10:01:56.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8640d2390 0x7fc86415caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8640ce7f0 0x7fc8641f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 --2- 172.21.15.181:0/192841731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8640d0180 0x7fc8641f3d40 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 >> 172.21.15.181:0/192841731 conn(0x7fc86409a9a0 msgr2=0x7fc864070640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 shutdown_connections 2023-10-16T10:01:56.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.831+0000 7fc869fa1700 1 -- 172.21.15.181:0/192841731 wait complete. 2023-10-16T10:01:56.835 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 49 2023-10-16T10:01:56.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:01:56.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.963+0000 7facdcbdf700 1 Processor -- start 2023-10-16T10:01:56.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.963+0000 7facdcbdf700 1 -- start start 2023-10-16T10:01:56.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.964+0000 7facdcbdf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facd8161b60 0x7facd806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.964+0000 7facdcbdf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.964+0000 7facdcbdf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd80d2560 con 0x7facd8160d20 2023-10-16T10:01:56.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.964+0000 7facdcbdf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd8162800 con 0x7facd8161b60 2023-10-16T10:01:56.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.964+0000 7facdcbdf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd80d43e0 con 0x7facd8161f40 2023-10-16T10:01:56.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.964+0000 7facd659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facd8161b60 0x7facd806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facd8161b60 0x7facd806d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45536/0 (socket says 172.21.15.181:45536) 2023-10-16T10:01:56.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd659c700 1 -- 172.21.15.181:0/4153107148 learned_addr learned my addr 172.21.15.181:0/4153107148 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:56.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd5d9b700 1 --2- 172.21.15.181:0/4153107148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd659c700 1 -- 172.21.15.181:0/4153107148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 msgr2=0x7facd80d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:56.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd659c700 1 --2- 172.21.15.181:0/4153107148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd659c700 1 -- 172.21.15.181:0/4153107148 >> v1:172.21.15.190:6789/0 conn(0x7facd8160d20 legacy=0x7facd80b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd659c700 1 -- 172.21.15.181:0/4153107148 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd806ce20 con 0x7facd8161b60 2023-10-16T10:01:56.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd659c700 1 --2- 172.21.15.181:0/4153107148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd806d5f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7facc8006fd0 tx=0x7facc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef93f5d177930af1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd559a700 1 -- 172.21.15.181:0/4153107148 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8005e30 con 0x7facd8161b60 2023-10-16T10:01:56.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd559a700 1 -- 172.21.15.181:0/4153107148 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc8004030 con 0x7facd8161b60 2023-10-16T10:01:56.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.965+0000 7facd559a700 1 -- 172.21.15.181:0/4153107148 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc80043e0 con 0x7facd8161b60 2023-10-16T10:01:56.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 -- 172.21.15.181:0/4153107148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 msgr2=0x7facd806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 --2- 172.21.15.181:0/4153107148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd806d5f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7facc8006fd0 tx=0x7facc8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 -- 172.21.15.181:0/4153107148 shutdown_connections 2023-10-16T10:01:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 --2- 172.21.15.181:0/4153107148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd806d5f0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 --2- 172.21.15.181:0/4153107148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 -- 172.21.15.181:0/4153107148 >> 172.21.15.181:0/4153107148 conn(0x7facd809cba0 msgr2=0x7facd80d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:56.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 -- 172.21.15.181:0/4153107148 shutdown_connections 2023-10-16T10:01:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.966+0000 7facdcbdf700 1 -- 172.21.15.181:0/4153107148 wait complete. 2023-10-16T10:01:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.968+0000 7facdcbdf700 1 Processor -- start 2023-10-16T10:01:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.968+0000 7facdcbdf700 1 -- start start 2023-10-16T10:01:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facdcbdf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd8160d20 0x7facd814ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facdcbdf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facdcbdf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facdcbdf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd814f120 con 0x7facd8161b60 2023-10-16T10:01:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facdcbdf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd814e080 con 0x7facd8160d20 2023-10-16T10:01:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facdcbdf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd814e360 con 0x7facd8161f40 2023-10-16T10:01:56.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd80ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45544/0 (socket says 172.21.15.181:45544) 2023-10-16T10:01:56.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 -- 172.21.15.181:0/120803260 learned_addr learned my addr 172.21.15.181:0/120803260 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:56.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd6d9d700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd659c700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd8160d20 0x7facd814ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:56.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 -- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 msgr2=0x7facd80cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 -- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd8160d20 msgr2=0x7facd814ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:56.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd8160d20 0x7facd814ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:56.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd5d9b700 1 -- 172.21.15.181:0/120803260 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8009500 con 0x7facd8161b60 2023-10-16T10:01:56.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.969+0000 7facd6d9d700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:01:56.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.970+0000 7facd5d9b700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd80ceff0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7faccc00e850 tx=0x7faccc00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:56.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.970+0000 7facc6ffd700 1 -- 172.21.15.181:0/120803260 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0095b0 con 0x7facd8161b60 2023-10-16T10:01:56.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.970+0000 7facc6ffd700 1 -- 172.21.15.181:0/120803260 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc013680 con 0x7facd8161b60 2023-10-16T10:01:56.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.970+0000 7facc6ffd700 1 -- 172.21.15.181:0/120803260 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc013a10 con 0x7facd8161b60 2023-10-16T10:01:56.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.970+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd814e640 con 0x7facd8161b60 2023-10-16T10:01:56.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.970+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd814ec00 con 0x7facd8161b60 2023-10-16T10:01:56.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.971+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd804ebb0 con 0x7facd8161b60 2023-10-16T10:01:56.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.971+0000 7facc6ffd700 1 -- 172.21.15.181:0/120803260 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 61) v1 ==== 88940+0+0 (secure 0 0 0) 0x7faccc013b70 con 0x7facd8161b60 2023-10-16T10:01:56.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.972+0000 7facc6ffd700 1 -- 172.21.15.181:0/120803260 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6936+0+0 (secure 0 0 0) 0x7faccc089370 con 0x7facd8161b60 2023-10-16T10:01:56.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:56.976+0000 7facc6ffd700 1 -- 172.21.15.181:0/120803260 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faccc0543a0 con 0x7facd8161b60 2023-10-16T10:01:57.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.172+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facd806d2a0 con 0x7facd8161b60 2023-10-16T10:01:57.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.173+0000 7facc6ffd700 1 -- 172.21.15.181:0/120803260 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+18439 (secure 0 0 0) 0x7faccc0579c0 con 0x7facd8161b60 2023-10-16T10:01:57.175 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:01:57.175 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":107,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:01:56.537344+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:00:32.847964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-16T10:00:33.365892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1805084655":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2616603102":"2023-10-17T10:01:56.537283+0000","172.21.15.181:6832/932006106":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2180020031":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2475032366":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/1044615193":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/517006871":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2950955543":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/929817601":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/1316320487":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/2393260999":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/633839274":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6833/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6832/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1989217127":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1386329415":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6833/932006106":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/80380984":"2023-10-17T10:00:48.266174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:01:57.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 msgr2=0x7facd80ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd80ceff0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7faccc00e850 tx=0x7faccc00eb60 comp rx=0 tx=0).stop 2023-10-16T10:01:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 shutdown_connections 2023-10-16T10:01:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facd8161f40 0x7facd80cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facd8160d20 0x7facd814ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 --2- 172.21.15.181:0/120803260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facd8161b60 0x7facd80ceff0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 >> 172.21.15.181:0/120803260 conn(0x7facd809cba0 msgr2=0x7facd8073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 shutdown_connections 2023-10-16T10:01:57.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.176+0000 7facdcbdf700 1 -- 172.21.15.181:0/120803260 wait complete. 2023-10-16T10:01:57.190 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:01:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.309+0000 7ff3f3682700 1 Processor -- start 2023-10-16T10:01:57.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.309+0000 7ff3f3682700 1 -- start start 2023-10-16T10:01:57.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f3682700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3ec15df60 0x7ff3ec0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f3682700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3ec06a8a0 0x7ff3ec06ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f3682700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0cddf0 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f3682700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec06b290 con 0x7ff3ec15df60 2023-10-16T10:01:57.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f3682700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec0af9c0 con 0x7ff3ec06a8a0 2023-10-16T10:01:57.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f141e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3ec15df60 0x7ff3ec0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f141e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3ec15df60 0x7ff3ec0cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45548/0 (socket says 172.21.15.181:45548) 2023-10-16T10:01:57.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f141e700 1 -- 172.21.15.181:0/3845649361 learned_addr learned my addr 172.21.15.181:0/3845649361 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:57.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f1c1f700 1 --2- 172.21.15.181:0/3845649361 >> v2:172.21.15.190:3300/0 conn(0x7ff3ec06a8a0 0x7ff3ec06ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f141e700 1 -- 172.21.15.181:0/3845649361 >> v2:172.21.15.190:3300/0 conn(0x7ff3ec06a8a0 msgr2=0x7ff3ec06ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f141e700 1 --2- 172.21.15.181:0/3845649361 >> v2:172.21.15.190:3300/0 conn(0x7ff3ec06a8a0 0x7ff3ec06ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.310+0000 7ff3f141e700 1 -- 172.21.15.181:0/3845649361 >> v1:172.21.15.190:6789/0 conn(0x7ff3ec0d0fc0 legacy=0x7ff3ec0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f141e700 1 -- 172.21.15.181:0/3845649361 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0ccf50 con 0x7ff3ec15df60 2023-10-16T10:01:57.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f1c1f700 1 --2- 172.21.15.181:0/3845649361 >> v2:172.21.15.190:3300/0 conn(0x7ff3ec06a8a0 0x7ff3ec06ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:57.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f141e700 1 --2- 172.21.15.181:0/3845649361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec15df60 0x7ff3ec0cae20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec06f9e0 tx=0x7ff3dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63db717ea88438bd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3e3fff700 1 -- 172.21.15.181:0/3845649361 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc005e30 con 0x7ff3ec15df60 2023-10-16T10:01:57.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3e3fff700 1 -- 172.21.15.181:0/3845649361 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc006030 con 0x7ff3ec15df60 2023-10-16T10:01:57.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3e3fff700 1 -- 172.21.15.181:0/3845649361 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0063e0 con 0x7ff3ec15df60 2023-10-16T10:01:57.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f3682700 1 -- 172.21.15.181:0/3845649361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec15df60 msgr2=0x7ff3ec0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f3682700 1 --2- 172.21.15.181:0/3845649361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec15df60 0x7ff3ec0cae20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec06f9e0 tx=0x7ff3dc009bd0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f3682700 1 -- 172.21.15.181:0/3845649361 shutdown_connections 2023-10-16T10:01:57.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f3682700 1 --2- 172.21.15.181:0/3845649361 >> v2:172.21.15.190:3300/0 conn(0x7ff3ec06a8a0 0x7ff3ec06ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f3682700 1 --2- 172.21.15.181:0/3845649361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec15df60 0x7ff3ec0cae20 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f3682700 1 -- 172.21.15.181:0/3845649361 >> 172.21.15.181:0/3845649361 conn(0x7ff3ec0d04a0 msgr2=0x7ff3ec0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.311+0000 7ff3f3682700 1 -- 172.21.15.181:0/3845649361 shutdown_connections 2023-10-16T10:01:57.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.312+0000 7ff3f3682700 1 -- 172.21.15.181:0/3845649361 wait complete. 2023-10-16T10:01:57.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 Processor -- start 2023-10-16T10:01:57.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 -- start start 2023-10-16T10:01:57.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec06a8a0 0x7ff3ec0c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec0d0fc0 0x7ff3ec1f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec15df60 0x7ff3ec1f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cedb0 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cfc10 con 0x7ff3ec06a8a0 2023-10-16T10:01:57.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f3682700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c5f60 con 0x7ff3ec15df60 2023-10-16T10:01:57.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f0c1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec0d0fc0 0x7ff3ec1f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f0c1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec0d0fc0 0x7ff3ec1f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45554/0 (socket says 172.21.15.181:45554) 2023-10-16T10:01:57.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.313+0000 7ff3f0c1d700 1 -- 172.21.15.181:0/334728512 learned_addr learned my addr 172.21.15.181:0/334728512 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:57.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f1c1f700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec15df60 0x7ff3ec1f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f141e700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec06a8a0 0x7ff3ec0c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f0c1d700 1 -- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec15df60 msgr2=0x7ff3ec1f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f0c1d700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec15df60 0x7ff3ec1f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f0c1d700 1 -- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec06a8a0 msgr2=0x7ff3ec0c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f0c1d700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec06a8a0 0x7ff3ec0c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f0c1d700 1 -- 172.21.15.181:0/334728512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc009500 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f1c1f700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec15df60 0x7ff3ec1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:57.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f141e700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec06a8a0 0x7ff3ec0c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:57.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f0c1d700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec0d0fc0 0x7ff3ec1f6300 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c4d0 tx=0x7ff3e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8011790 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e80118f0 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec0cf000 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8011ca0 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.314+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec0cf5c0 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.315+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec067d20 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.315+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 61) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff3e801c460 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.316+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6936+0+0 (secure 0 0 0) 0x7ff3e80893f0 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.320+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff3e8054420 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.548+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 62) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff3e8053ed0 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.548+0000 7ff3e1ffb700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7ff3d0070d60 0x7ff3d0073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.550+0000 7ff3f141e700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7ff3d0070d60 0x7ff3d0073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.551+0000 7ff3f141e700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7ff3d0070d60 0x7ff3d0073150 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec06f9e0 tx=0x7ff3dc008070 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.561+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff3ec0cfd80 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.563 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T10:01:57.561+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:01:57.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.563+0000 7ff3e1ffb700 1 -- 172.21.15.181:0/334728512 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff3e8087330 con 0x7ff3ec0d0fc0 2023-10-16T10:01:57.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7ff3d0070d60 msgr2=0x7ff3d0073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7ff3d0070d60 0x7ff3d0073150 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec06f9e0 tx=0x7ff3dc008070 comp rx=0 tx=0).stop 2023-10-16T10:01:57.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec0d0fc0 msgr2=0x7ff3ec1f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec0d0fc0 0x7ff3ec1f6300 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c4d0 tx=0x7ff3e800c890 comp rx=0 tx=0).stop 2023-10-16T10:01:57.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 shutdown_connections 2023-10-16T10:01:57.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7ff3d0070d60 0x7ff3d0073150 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3ec15df60 0x7ff3ec1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ec06a8a0 0x7ff3ec0c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 --2- 172.21.15.181:0/334728512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ec0d0fc0 0x7ff3ec1f6300 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 >> 172.21.15.181:0/334728512 conn(0x7ff3ec0d04a0 msgr2=0x7ff3ec0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 shutdown_connections 2023-10-16T10:01:57.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.566+0000 7ff3f3682700 1 -- 172.21.15.181:0/334728512 wait complete. 2023-10-16T10:01:57.570 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:01:57.571 INFO:tasks.ceph.mds.a.smithi181.stderr: -8595> 2023-10-16T10:00:36.998+0000 7fa551b34700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:01:57.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:01:57.603 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T10:01:57.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.702+0000 7f5f8e2de700 1 Processor -- start 2023-10-16T10:01:57.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.702+0000 7f5f8e2de700 1 -- start start 2023-10-16T10:01:57.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f8e2de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 0x7f5f8806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f8e2de700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f8806ade0 0x7f5f881615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f8e2de700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8806b230 con 0x7f5f8815da70 2023-10-16T10:01:57.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f8e2de700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880cef00 con 0x7f5f8806ade0 2023-10-16T10:01:57.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f8e2de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880cec20 con 0x7f5f88072760 2023-10-16T10:01:57.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f87fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5f8815da70 0x7f5f880ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56674/0 (socket says 172.21.15.181:56674) 2023-10-16T10:01:57.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f87fff700 1 -- 172.21.15.181:0/3785683730 learned_addr learned my addr 172.21.15.181:0/3785683730 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:57.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f8c87b700 1 --2- 172.21.15.181:0/3785683730 >> v2:172.21.15.190:3300/0 conn(0x7f5f8806ade0 0x7f5f881615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987455223 0 0) 0x7f5f7c001110 con 0x7f5f8815da70 2023-10-16T10:01:57.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f877fe700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 0x7f5f8806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.703+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f700035b0 con 0x7f5f8815da70 2023-10-16T10:01:57.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2918651759 0 0) 0x7f5f700035b0 con 0x7f5f8815da70 2023-10-16T10:01:57.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 msgr2=0x7f5f8806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 0x7f5f8806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 >> v2:172.21.15.190:3300/0 conn(0x7f5f8806ade0 msgr2=0x7f5f881615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 --2- 172.21.15.181:0/3785683730 >> v2:172.21.15.190:3300/0 conn(0x7f5f8806ade0 0x7f5f881615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f88071c80 con 0x7f5f8815da70 2023-10-16T10:01:57.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5f7c001800 con 0x7f5f8815da70 2023-10-16T10:01:57.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 >> v1:172.21.15.181:6789/0 conn(0x7f5f8815da70 legacy=0x7f5f880ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f70003a00 0x7f5f70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f70004850 0x7f5f70004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 0x7f5f70005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880cef00 con 0x7f5f70004850 2023-10-16T10:01:57.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880cec20 con 0x7f5f70003a00 2023-10-16T10:01:57.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7000bac0 con 0x7f5f88072760 2023-10-16T10:01:57.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f877fe700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f70004850 0x7f5f70004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f8c87b700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 0x7f5f70005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f877fe700 1 -- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 msgr2=0x7f5f70005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f877fe700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 0x7f5f70005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f87fff700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f70003a00 0x7f5f70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f877fe700 1 -- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f70003a00 msgr2=0x7f5f70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:57.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f877fe700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f70003a00 0x7f5f70003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f877fe700 1 -- 172.21.15.181:0/3785683730 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700035b0 con 0x7f5f70004850 2023-10-16T10:01:57.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f8c87b700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f88072760 0x7f5f70005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:57.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f87fff700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f70003a00 0x7f5f70003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:01:57.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f877fe700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f70004850 0x7f5f70004cd0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009b60 tx=0x7f5f70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d311aa338d36f661 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.704+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800e6c0 con 0x7f5f70004850 2023-10-16T10:01:57.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7800e820 con 0x7f5f70004850 2023-10-16T10:01:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f86ffd700 1 -- 172.21.15.181:0/3785683730 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800eb90 con 0x7f5f70004850 2023-10-16T10:01:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f8e2de700 1 -- 172.21.15.181:0/3785683730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f70004850 msgr2=0x7f5f70004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f8e2de700 1 --2- 172.21.15.181:0/3785683730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f70004850 0x7f5f70004cd0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009b60 tx=0x7f5f70005880 comp rx=0 tx=0).stop 2023-10-16T10:01:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f8c87b700 1 -- 172.21.15.181:0/3785683730 reap_dead start 2023-10-16T10:01:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f8e2de700 1 -- 172.21.15.181:0/3785683730 shutdown_connections 2023-10-16T10:01:57.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f8e2de700 1 -- 172.21.15.181:0/3785683730 >> 172.21.15.181:0/3785683730 conn(0x7f5f880cd010 msgr2=0x7f5f880cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:57.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.705+0000 7f5f8e2de700 1 -- 172.21.15.181:0/3785683730 shutdown_connections 2023-10-16T10:01:57.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.707+0000 7f5f8e2de700 1 -- 172.21.15.181:0/3785683730 wait complete. 2023-10-16T10:01:57.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.708+0000 7f5f8e2de700 1 Processor -- start 2023-10-16T10:01:57.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.708+0000 7f5f8e2de700 1 -- start start 2023-10-16T10:01:57.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f8e2de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f70004850 0x7f5f880af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f8e2de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f880c9e50 0x7f5f880c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f8e2de700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f880b44d0 0x7f5f880b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f8e2de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880ca5b0 con 0x7f5f880c9e50 2023-10-16T10:01:57.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f8e2de700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880c7de0 con 0x7f5f880b44d0 2023-10-16T10:01:57.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f8e2de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880ca2d0 con 0x7f5f70004850 2023-10-16T10:01:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f880c9e50 0x7f5f880c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f880c9e50 0x7f5f880c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45610/0 (socket says 172.21.15.181:45610) 2023-10-16T10:01:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f877fe700 1 -- 172.21.15.181:0/479933898 learned_addr learned my addr 172.21.15.181:0/479933898 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f8c87b700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f880b44d0 0x7f5f880b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.709+0000 7f5f87fff700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f70004850 0x7f5f880af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f877fe700 1 -- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f70004850 msgr2=0x7f5f880af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f877fe700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f70004850 0x7f5f880af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f877fe700 1 -- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f880b44d0 msgr2=0x7f5f880b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f877fe700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f880b44d0 0x7f5f880b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f877fe700 1 -- 172.21.15.181:0/479933898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880c7790 con 0x7f5f880c9e50 2023-10-16T10:01:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f877fe700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f880c9e50 0x7f5f880c8270 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800e4b0 tx=0x7f5f780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f84ff9700 1 -- 172.21.15.181:0/479933898 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800e6c0 con 0x7f5f880c9e50 2023-10-16T10:01:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f84ff9700 1 -- 172.21.15.181:0/479933898 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780038a0 con 0x7f5f880c9e50 2023-10-16T10:01:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f880c7a70 con 0x7f5f880c9e50 2023-10-16T10:01:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f84ff9700 1 -- 172.21.15.181:0/479933898 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78003c50 con 0x7f5f880c9e50 2023-10-16T10:01:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.710+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f880b4ec0 con 0x7f5f880c9e50 2023-10-16T10:01:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.711+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f880c7f20 con 0x7f5f880c9e50 2023-10-16T10:01:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.711+0000 7f5f84ff9700 1 -- 172.21.15.181:0/479933898 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 62) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5f78013470 con 0x7f5f880c9e50 2023-10-16T10:01:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.712+0000 7f5f84ff9700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5f7406ad50 0x7f5f7406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.712+0000 7f5f84ff9700 1 -- 172.21.15.181:0/479933898 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7022+0+0 (secure 0 0 0) 0x7f5f7808ae70 con 0x7f5f880c9e50 2023-10-16T10:01:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.712+0000 7f5f87fff700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5f7406ad50 0x7f5f7406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:57.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.712+0000 7f5f87fff700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5f7406ad50 0x7f5f7406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001a70 tx=0x7f5f7c009040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:57.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.716+0000 7f5f84ff9700 1 -- 172.21.15.181:0/479933898 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5f78055f10 con 0x7f5f880c9e50 2023-10-16T10:01:57.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:57.953+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5f8806f6c0 con 0x7f5f880c9e50 2023-10-16T10:01:58.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.564+0000 7f5f84ff9700 1 -- 172.21.15.181:0/479933898 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5f78003db0 con 0x7f5f880c9e50 2023-10-16T10:01:58.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5f7406ad50 msgr2=0x7f5f7406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5f7406ad50 0x7f5f7406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001a70 tx=0x7f5f7c009040 comp rx=0 tx=0).stop 2023-10-16T10:01:58.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f880c9e50 msgr2=0x7f5f880c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f880c9e50 0x7f5f880c8270 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800e4b0 tx=0x7f5f780047c0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 shutdown_connections 2023-10-16T10:01:58.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5f7406ad50 0x7f5f7406d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f70004850 0x7f5f880af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f880b44d0 0x7f5f880b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 --2- 172.21.15.181:0/479933898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f880c9e50 0x7f5f880c8270 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 >> 172.21.15.181:0/479933898 conn(0x7f5f880cd010 msgr2=0x7f5f880cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:58.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 shutdown_connections 2023-10-16T10:01:58.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.567+0000 7f5f8e2de700 1 -- 172.21.15.181:0/479933898 wait complete. 2023-10-16T10:01:58.584 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:01:58.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.748+0000 7fd2aa7be700 1 Processor -- start 2023-10-16T10:01:58.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.748+0000 7fd2aa7be700 1 -- start start 2023-10-16T10:01:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2aa7be700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 0x7fd2a406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2aa7be700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a4162650 con 0x7fd2a40ce610 2023-10-16T10:01:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2aa7be700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a40d0230 con 0x7fd2a40d0fc0 2023-10-16T10:01:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2aa7be700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4162190 con 0x7fd2a406afb0 2023-10-16T10:01:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2a3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd2a40ce610 0x7fd2a40cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56714/0 (socket says 172.21.15.181:56714) 2023-10-16T10:01:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2a3fff700 1 -- 172.21.15.181:0/3900059331 learned_addr learned my addr 172.21.15.181:0/3900059331 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2a8d5b700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 0x7fd2a406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866116698 0 0) 0x7fd2a4162650 con 0x7fd2a40ce610 2023-10-16T10:01:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.749+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd28c0035b0 con 0x7fd2a40ce610 2023-10-16T10:01:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2089178918 0 0) 0x7fd28c0035b0 con 0x7fd2a40ce610 2023-10-16T10:01:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 msgr2=0x7fd2a406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 0x7fd2a406d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 >> v1:172.21.15.190:6789/0 conn(0x7fd2a40d0fc0 legacy=0x7fd2a406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4162370 con 0x7fd2a40ce610 2023-10-16T10:01:58.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd2980017f0 con 0x7fd2a40ce610 2023-10-16T10:01:58.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 >> v1:172.21.15.181:6789/0 conn(0x7fd2a40ce610 legacy=0x7fd2a40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 0x7fd28c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c003fb0 0x7fd28c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c004d90 0x7fd28c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4162190 con 0x7fd28c004d90 2023-10-16T10:01:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00baf0 con 0x7fd28c003fb0 2023-10-16T10:01:58.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00bcb0 con 0x7fd2a406afb0 2023-10-16T10:01:58.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a8d5b700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c004d90 0x7fd28c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a3fff700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 0x7fd28c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a37fe700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c003fb0 0x7fd28c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a8d5b700 1 -- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 msgr2=0x7fd28c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a8d5b700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a406afb0 0x7fd28c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a8d5b700 1 -- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c003fb0 msgr2=0x7fd28c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:58.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a8d5b700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c003fb0 0x7fd28c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a8d5b700 1 -- 172.21.15.181:0/3900059331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0035b0 con 0x7fd28c004d90 2023-10-16T10:01:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a8d5b700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c004d90 0x7fd28c005210 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009fc0 tx=0x7fd28c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f28803d5b56448b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00ec30 con 0x7fd28c004d90 2023-10-16T10:01:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c00f040 con 0x7fd28c004d90 2023-10-16T10:01:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.750+0000 7fd2a2ffd700 1 -- 172.21.15.181:0/3900059331 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c014480 con 0x7fd28c004d90 2023-10-16T10:01:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.751+0000 7fd2aa7be700 1 -- 172.21.15.181:0/3900059331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c004d90 msgr2=0x7fd28c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.751+0000 7fd2aa7be700 1 --2- 172.21.15.181:0/3900059331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c004d90 0x7fd28c005210 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009fc0 tx=0x7fd28c005880 comp rx=0 tx=0).stop 2023-10-16T10:01:58.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.751+0000 7fd2a8d5b700 1 -- 172.21.15.181:0/3900059331 reap_dead start 2023-10-16T10:01:58.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.751+0000 7fd2aa7be700 1 -- 172.21.15.181:0/3900059331 shutdown_connections 2023-10-16T10:01:58.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.751+0000 7fd2aa7be700 1 -- 172.21.15.181:0/3900059331 >> 172.21.15.181:0/3900059331 conn(0x7fd2a40d04a0 msgr2=0x7fd2a40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:58.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.751+0000 7fd2aa7be700 1 -- 172.21.15.181:0/3900059331 shutdown_connections 2023-10-16T10:01:58.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.751+0000 7fd2aa7be700 1 -- 172.21.15.181:0/3900059331 wait complete. 2023-10-16T10:01:58.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.753+0000 7fd2aa7be700 1 Processor -- start 2023-10-16T10:01:58.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.753+0000 7fd2aa7be700 1 -- start start 2023-10-16T10:01:58.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2a41f2c80 0x7fd2a41f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a4158790 0x7fd2a41607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a4158c10 0x7fd2a4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a41576f0 con 0x7fd2a4158790 2023-10-16T10:01:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a41579d0 con 0x7fd2a41f2c80 2023-10-16T10:01:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4157cb0 con 0x7fd2a4158c10 2023-10-16T10:01:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a4158790 0x7fd2a41607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a4158790 0x7fd2a41607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45650/0 (socket says 172.21.15.181:45650) 2023-10-16T10:01:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 -- 172.21.15.181:0/2770931631 learned_addr learned my addr 172.21.15.181:0/2770931631 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a3fff700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2a41f2c80 0x7fd2a41f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a8d5b700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a4158c10 0x7fd2a4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 -- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a4158c10 msgr2=0x7fd2a4160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a4158c10 0x7fd2a4160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 -- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2a41f2c80 msgr2=0x7fd2a41f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2a41f2c80 0x7fd2a41f3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 -- 172.21.15.181:0/2770931631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4157f90 con 0x7fd2a4158790 2023-10-16T10:01:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a37fe700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a4158790 0x7fd2a41607c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd29400f820 tx=0x7fd29400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2940052e0 con 0x7fd2a4158790 2023-10-16T10:01:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd294005440 con 0x7fd2a4158790 2023-10-16T10:01:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a4158270 con 0x7fd2a4158790 2023-10-16T10:01:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2940038a0 con 0x7fd2a4158790 2023-10-16T10:01:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.754+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a41458e0 con 0x7fd2a4158790 2023-10-16T10:01:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.755+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a41585e0 con 0x7fd2a4158790 2023-10-16T10:01:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.756+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 62) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd294003a00 con 0x7fd2a4158790 2023-10-16T10:01:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.756+0000 7fd2a0ff9700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd29006ad50 0x7fd29006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.756+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7022+0+0 (secure 0 0 0) 0x7fd294089520 con 0x7fd2a4158790 2023-10-16T10:01:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.756+0000 7fd2a3fff700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd29006ad50 0x7fd29006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.757+0000 7fd2a3fff700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd29006ad50 0x7fd29006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2980011d0 tx=0x7fd298009040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.761+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd2940543f0 con 0x7fd2a4158790 2023-10-16T10:01:58.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:58.953+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2a4067d20 con 0x7fd2a4158790 2023-10-16T10:01:59.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.555+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 63) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd2940547c0 con 0x7fd2a4158790 2023-10-16T10:01:59.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.565+0000 7fd2a0ff9700 1 -- 172.21.15.181:0/2770931631 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd294087370 con 0x7fd2a4158790 2023-10-16T10:01:59.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.569+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd29006ad50 msgr2=0x7fd29006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.569+0000 7fd2aa7be700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd29006ad50 0x7fd29006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2980011d0 tx=0x7fd298009040 comp rx=0 tx=0).stop 2023-10-16T10:01:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.569+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a4158790 msgr2=0x7fd2a41607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.569+0000 7fd2aa7be700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a4158790 0x7fd2a41607c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd29400f820 tx=0x7fd29400fbe0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 shutdown_connections 2023-10-16T10:01:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd29006ad50 0x7fd29006d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a4158c10 0x7fd2a4160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2a41f2c80 0x7fd2a41f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 --2- 172.21.15.181:0/2770931631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a4158790 0x7fd2a41607c0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 >> 172.21.15.181:0/2770931631 conn(0x7fd2a40d04a0 msgr2=0x7fd2a415e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 shutdown_connections 2023-10-16T10:01:59.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.570+0000 7fd2aa7be700 1 -- 172.21.15.181:0/2770931631 wait complete. 2023-10-16T10:01:59.577 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:01:59.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:01:59.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.719+0000 7fa155258700 1 Processor -- start 2023-10-16T10:01:59.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.719+0000 7fa155258700 1 -- start start 2023-10-16T10:01:59.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa155258700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa15006d530 0x7fa1500d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa155258700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15015dd80 con 0x7fa1500ce270 2023-10-16T10:01:59.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa155258700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15006ba60 con 0x7fa1500d01e0 2023-10-16T10:01:59.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa155258700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1500d42a0 con 0x7fa15006d530 2023-10-16T10:01:59.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14f59e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa1500ce270 0x7fa1500ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56738/0 (socket says 172.21.15.181:56738) 2023-10-16T10:01:59.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14f59e700 1 -- 172.21.15.181:0/1526917146 learned_addr learned my addr 172.21.15.181:0/1526917146 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:01:59.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281231265 0 0) 0x7fa15015dd80 con 0x7fa1500ce270 2023-10-16T10:01:59.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14e59c700 1 --2- 172.21.15.181:0/1526917146 >> v2:172.21.15.190:3300/0 conn(0x7fa15006d530 0x7fa1500d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1380035b0 con 0x7fa1500ce270 2023-10-16T10:01:59.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3647038898 0 0) 0x7fa1380035b0 con 0x7fa1500ce270 2023-10-16T10:01:59.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 >> v2:172.21.15.190:3300/0 conn(0x7fa15006d530 msgr2=0x7fa1500d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:59.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 --2- 172.21.15.181:0/1526917146 >> v2:172.21.15.190:3300/0 conn(0x7fa15006d530 0x7fa1500d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 >> v1:172.21.15.190:6789/0 conn(0x7fa1500d01e0 legacy=0x7fa1500d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500d4580 con 0x7fa1500ce270 2023-10-16T10:01:59.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa1480019f0 con 0x7fa1500ce270 2023-10-16T10:01:59.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 >> v1:172.21.15.181:6789/0 conn(0x7fa1500ce270 legacy=0x7fa1500ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138003a00 0x7fa138003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.720+0000 7fa14dd9b700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa138004850 0x7fa138004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14dd9b700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa138005210 0x7fa13800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500d42a0 con 0x7fa138004850 2023-10-16T10:01:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13800c0b0 con 0x7fa138005210 2023-10-16T10:01:59.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13800c270 con 0x7fa138003a00 2023-10-16T10:01:59.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14e59c700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa138004850 0x7fa138004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14f59e700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa138005210 0x7fa13800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14ed9d700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138003a00 0x7fa138003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14e59c700 1 -- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138003a00 msgr2=0x7fa138003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:59.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14e59c700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138003a00 0x7fa138003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14e59c700 1 -- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa138005210 msgr2=0x7fa13800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:01:59.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14e59c700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa138005210 0x7fa13800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14f59e700 1 -- 172.21.15.181:0/1526917146 reap_dead start 2023-10-16T10:01:59.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14e59c700 1 -- 172.21.15.181:0/1526917146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380035b0 con 0x7fa138004850 2023-10-16T10:01:59.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14ed9d700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138003a00 0x7fa138003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:01:59.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14e59c700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa138004850 0x7fa138004cd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa140009b60 tx=0x7fa138005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c12898699c73b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:59.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14000e6c0 con 0x7fa138004850 2023-10-16T10:01:59.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14000e820 con 0x7fa138004850 2023-10-16T10:01:59.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.721+0000 7fa14dd9b700 1 -- 172.21.15.181:0/1526917146 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14000eb90 con 0x7fa138004850 2023-10-16T10:01:59.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.722+0000 7fa155258700 1 -- 172.21.15.181:0/1526917146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa138004850 msgr2=0x7fa138004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.722+0000 7fa155258700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa138004850 0x7fa138004cd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa140009b60 tx=0x7fa138005880 comp rx=0 tx=0).stop 2023-10-16T10:01:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.722+0000 7fa155258700 1 -- 172.21.15.181:0/1526917146 shutdown_connections 2023-10-16T10:01:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.722+0000 7fa155258700 1 --2- 172.21.15.181:0/1526917146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa138004850 0x7fa138004cd0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.722+0000 7fa155258700 1 -- 172.21.15.181:0/1526917146 >> 172.21.15.181:0/1526917146 conn(0x7fa15008af90 msgr2=0x7fa1500d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:01:59.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.722+0000 7fa155258700 1 -- 172.21.15.181:0/1526917146 shutdown_connections 2023-10-16T10:01:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.722+0000 7fa155258700 1 -- 172.21.15.181:0/1526917146 wait complete. 2023-10-16T10:01:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.723+0000 7fa155258700 1 Processor -- start 2023-10-16T10:01:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.723+0000 7fa155258700 1 -- start start 2023-10-16T10:01:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa155258700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138004850 0x7fa1501f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa155258700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1500c9490 0x7fa1501f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa155258700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1500c98f0 0x7fa1501f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa155258700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500cc5e0 con 0x7fa1500c98f0 2023-10-16T10:01:59.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa155258700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500c71d0 con 0x7fa1500c9490 2023-10-16T10:01:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa155258700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500c7490 con 0x7fa138004850 2023-10-16T10:01:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1500c98f0 0x7fa1501f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1500c98f0 0x7fa1501f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45670/0 (socket says 172.21.15.181:45670) 2023-10-16T10:01:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 -- 172.21.15.181:0/3772682197 learned_addr learned my addr 172.21.15.181:0/3772682197 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:01:59.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14ed9d700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138004850 0x7fa1501f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14e59c700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1500c9490 0x7fa1501f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 -- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138004850 msgr2=0x7fa1501f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138004850 0x7fa1501f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 -- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1500c9490 msgr2=0x7fa1501f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:01:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1500c9490 0x7fa1501f6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:01:59.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.724+0000 7fa14f59e700 1 -- 172.21.15.181:0/3772682197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500c7770 con 0x7fa1500c98f0 2023-10-16T10:01:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.725+0000 7fa14f59e700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1500c98f0 0x7fa1501f6c70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fa15008d1a0 tx=0x7fa14800a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.725+0000 7fa1377fe700 1 -- 172.21.15.181:0/3772682197 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14800ac80 con 0x7fa1500c98f0 2023-10-16T10:01:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.725+0000 7fa1377fe700 1 -- 172.21.15.181:0/3772682197 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14800ade0 con 0x7fa1500c98f0 2023-10-16T10:01:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.725+0000 7fa1377fe700 1 -- 172.21.15.181:0/3772682197 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148013630 con 0x7fa1500c98f0 2023-10-16T10:01:59.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.725+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1500c7a50 con 0x7fa1500c98f0 2023-10-16T10:01:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.725+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1501f0990 con 0x7fa1500c98f0 2023-10-16T10:01:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.726+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1500d3ec0 con 0x7fa1500c98f0 2023-10-16T10:01:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.726+0000 7fa1377fe700 1 -- 172.21.15.181:0/3772682197 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 63) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa148013790 con 0x7fa1500c98f0 2023-10-16T10:01:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.726+0000 7fa1377fe700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa13c06af90 0x7fa13c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:01:59.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.727+0000 7fa1377fe700 1 -- 172.21.15.181:0/3772682197 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6584+0+0 (secure 0 0 0) 0x7fa14808a4a0 con 0x7fa1500c98f0 2023-10-16T10:01:59.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.727+0000 7fa14ed9d700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa13c06af90 0x7fa13c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:01:59.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.727+0000 7fa14ed9d700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa13c06af90 0x7fa13c06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1500b9880 tx=0x7fa144009040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:01:59.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.731+0000 7fa1377fe700 1 -- 172.21.15.181:0/3772682197 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa1500d3ec0 con 0x7fa1500c98f0 2023-10-16T10:01:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:01:59.923+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1500c7f70 con 0x7fa1500c98f0 2023-10-16T10:02:00.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.567+0000 7fa1377fe700 1 -- 172.21.15.181:0/3772682197 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa1500c7f70 con 0x7fa1500c98f0 2023-10-16T10:02:00.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa13c06af90 msgr2=0x7fa13c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa13c06af90 0x7fa13c06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1500b9880 tx=0x7fa144009040 comp rx=0 tx=0).stop 2023-10-16T10:02:00.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1500c98f0 msgr2=0x7fa1501f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1500c98f0 0x7fa1501f6c70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fa15008d1a0 tx=0x7fa14800a5f0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 shutdown_connections 2023-10-16T10:02:00.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa13c06af90 0x7fa13c06d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa138004850 0x7fa1501f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1500c98f0 0x7fa1501f6c70 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 --2- 172.21.15.181:0/3772682197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1500c9490 0x7fa1501f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.571+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 >> 172.21.15.181:0/3772682197 conn(0x7fa15008af90 msgr2=0x7fa1500b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:00.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.573+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 shutdown_connections 2023-10-16T10:02:00.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.574+0000 7fa155258700 1 -- 172.21.15.181:0/3772682197 wait complete. 2023-10-16T10:02:00.578 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T10:02:00.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:02:00.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.725+0000 7f38d50cb700 1 Processor -- start 2023-10-16T10:02:00.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.725+0000 7f38d50cb700 1 -- start start 2023-10-16T10:02:00.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38d50cb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d015da70 0x7f38d00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38d50cb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f38d00ce6a0 0x7f38d006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38d50cb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38d0072760 0x7f38d006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38d50cb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00cef00 con 0x7f38d0072760 2023-10-16T10:02:00.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38d50cb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00cec20 con 0x7f38d00ce6a0 2023-10-16T10:02:00.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38d50cb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0071c80 con 0x7f38d015da70 2023-10-16T10:02:00.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38cf59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38d0072760 0x7f38d006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38cf59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38d0072760 0x7f38d006ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45678/0 (socket says 172.21.15.181:45678) 2023-10-16T10:02:00.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38cf59e700 1 -- 172.21.15.181:0/3870137253 learned_addr learned my addr 172.21.15.181:0/3870137253 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:00.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38ce59c700 1 --2- 172.21.15.181:0/3870137253 >> v2:172.21.15.190:3300/0 conn(0x7f38d00ce6a0 0x7f38d006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38ced9d700 1 --2- 172.21.15.181:0/3870137253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d015da70 0x7f38d00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.726+0000 7f38cf59e700 1 -- 172.21.15.181:0/3870137253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d015da70 msgr2=0x7f38d00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:00.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cf59e700 1 --2- 172.21.15.181:0/3870137253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d015da70 0x7f38d00ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cf59e700 1 -- 172.21.15.181:0/3870137253 >> v2:172.21.15.190:3300/0 conn(0x7f38d00ce6a0 msgr2=0x7f38d006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:00.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cf59e700 1 --2- 172.21.15.181:0/3870137253 >> v2:172.21.15.190:3300/0 conn(0x7f38d00ce6a0 0x7f38d006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cf59e700 1 -- 172.21.15.181:0/3870137253 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0071f60 con 0x7f38d0072760 2023-10-16T10:02:00.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cf59e700 1 --2- 172.21.15.181:0/3870137253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d0072760 0x7f38d006ade0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f38d00674f0 tx=0x7f38c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac06bf5d2df8b35 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:00.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cdd9b700 1 -- 172.21.15.181:0/3870137253 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800e070 con 0x7f38d0072760 2023-10-16T10:02:00.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cdd9b700 1 -- 172.21.15.181:0/3870137253 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c800b9c0 con 0x7f38d0072760 2023-10-16T10:02:00.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38cdd9b700 1 -- 172.21.15.181:0/3870137253 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800bd70 con 0x7f38d0072760 2023-10-16T10:02:00.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 -- 172.21.15.181:0/3870137253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d0072760 msgr2=0x7f38d006ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 --2- 172.21.15.181:0/3870137253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d0072760 0x7f38d006ade0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f38d00674f0 tx=0x7f38c800b330 comp rx=0 tx=0).stop 2023-10-16T10:02:00.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 -- 172.21.15.181:0/3870137253 shutdown_connections 2023-10-16T10:02:00.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 --2- 172.21.15.181:0/3870137253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d0072760 0x7f38d006ade0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 --2- 172.21.15.181:0/3870137253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d015da70 0x7f38d00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 --2- 172.21.15.181:0/3870137253 >> v2:172.21.15.190:3300/0 conn(0x7f38d00ce6a0 0x7f38d006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 -- 172.21.15.181:0/3870137253 >> 172.21.15.181:0/3870137253 conn(0x7f38d00cd010 msgr2=0x7f38d00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:00.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 -- 172.21.15.181:0/3870137253 shutdown_connections 2023-10-16T10:02:00.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.727+0000 7f38d50cb700 1 -- 172.21.15.181:0/3870137253 wait complete. 2023-10-16T10:02:00.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.729+0000 7f38d50cb700 1 Processor -- start 2023-10-16T10:02:00.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.729+0000 7f38d50cb700 1 -- start start 2023-10-16T10:02:00.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d0072760 0x7f38d00ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d00ce6a0 0x7f38d00c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d015da70 0x7f38d01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00af700 con 0x7f38d015da70 2023-10-16T10:02:00.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b3ca0 con 0x7f38d00ce6a0 2023-10-16T10:02:00.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00ad970 con 0x7f38d0072760 2023-10-16T10:02:00.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d015da70 0x7f38d01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d015da70 0x7f38d01578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45684/0 (socket says 172.21.15.181:45684) 2023-10-16T10:02:00.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 -- 172.21.15.181:0/1985997087 learned_addr learned my addr 172.21.15.181:0/1985997087 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:00.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38ced9d700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d0072760 0x7f38d00ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38ce59c700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d00ce6a0 0x7f38d00c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 -- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d0072760 msgr2=0x7f38d00ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d0072760 0x7f38d00ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 -- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d00ce6a0 msgr2=0x7f38d00c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d00ce6a0 0x7f38d00c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 -- 172.21.15.181:0/1985997087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009b30 con 0x7f38d015da70 2023-10-16T10:02:00.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38ce59c700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d00ce6a0 0x7f38d00c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:00.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38cf59e700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d015da70 0x7f38d01578a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f38d006f140 tx=0x7f38c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:00.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38b77fe700 1 -- 172.21.15.181:0/1985997087 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004250 con 0x7f38d015da70 2023-10-16T10:02:00.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38b77fe700 1 -- 172.21.15.181:0/1985997087 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c80043d0 con 0x7f38d015da70 2023-10-16T10:02:00.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38b77fe700 1 -- 172.21.15.181:0/1985997087 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004780 con 0x7f38d015da70 2023-10-16T10:02:00.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d00adc50 con 0x7f38d015da70 2023-10-16T10:02:00.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.730+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00ae210 con 0x7f38d015da70 2023-10-16T10:02:00.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.731+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d00ae3d0 con 0x7f38d015da70 2023-10-16T10:02:00.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.732+0000 7f38b77fe700 1 -- 172.21.15.181:0/1985997087 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 63) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f38c801b430 con 0x7f38d015da70 2023-10-16T10:02:00.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.732+0000 7f38b77fe700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f38bc06af90 0x7f38bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:00.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.732+0000 7f38b77fe700 1 -- 172.21.15.181:0/1985997087 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7f38c808a9b0 con 0x7f38d015da70 2023-10-16T10:02:00.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.732+0000 7f38ced9d700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f38bc06af90 0x7f38bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:00.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.733+0000 7f38ced9d700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f38bc06af90 0x7f38bc06d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f38d00cc3f0 tx=0x7f38c4008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:00.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.736+0000 7f38b77fe700 1 -- 172.21.15.181:0/1985997087 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f38c8055840 con 0x7f38d015da70 2023-10-16T10:02:00.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.929+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f38d006f6c0 con 0x7f38d015da70 2023-10-16T10:02:00.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.932+0000 7f38b77fe700 1 -- 172.21.15.181:0/1985997087 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v37) v1 ==== 115+0+0 (secure 0 0 0) 0x7f38c80290b0 con 0x7f38d015da70 2023-10-16T10:02:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f38bc06af90 msgr2=0x7f38bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f38bc06af90 0x7f38bc06d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f38d00cc3f0 tx=0x7f38c4008040 comp rx=0 tx=0).stop 2023-10-16T10:02:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d015da70 msgr2=0x7f38d01578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:00.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d015da70 0x7f38d01578a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f38d006f140 tx=0x7f38c8004ad0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 shutdown_connections 2023-10-16T10:02:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f38bc06af90 0x7f38bc06d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38d0072760 0x7f38d00ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38d015da70 0x7f38d01578a0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 --2- 172.21.15.181:0/1985997087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38d00ce6a0 0x7f38d00c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 >> 172.21.15.181:0/1985997087 conn(0x7f38d00cd010 msgr2=0x7f38d015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 shutdown_connections 2023-10-16T10:02:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:00.935+0000 7f38d50cb700 1 -- 172.21.15.181:0/1985997087 wait complete. 2023-10-16T10:02:00.949 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:02:01.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.069+0000 7f036c6a4700 1 Processor -- start 2023-10-16T10:02:01.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.069+0000 7f036c6a4700 1 -- start start 2023-10-16T10:02:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036c6a4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f0364071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036c6a4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03640ce680 0x7f03640ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036c6a4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03640ccb60 con 0x7f03640b2010 2023-10-16T10:02:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036c6a4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0364072120 con 0x7f03640ce680 2023-10-16T10:02:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036c6a4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03640ce1f0 con 0x7f03640cc6e0 2023-10-16T10:02:01.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036ac41700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03640ce680 0x7f03640ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036a440700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f03640b2010 0x7f03640b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56766/0 (socket says 172.21.15.181:56766) 2023-10-16T10:02:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036ac41700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03640ce680 0x7f03640ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45692/0 (socket says 172.21.15.181:45692) 2023-10-16T10:02:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.070+0000 7f036ac41700 1 -- 172.21.15.181:0/2837349889 learned_addr learned my addr 172.21.15.181:0/2837349889 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f0369c3f700 1 --2- 172.21.15.181:0/2837349889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f0364071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036ac41700 1 -- 172.21.15.181:0/2837349889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 msgr2=0x7f0364071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036ac41700 1 --2- 172.21.15.181:0/2837349889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f0364071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036ac41700 1 -- 172.21.15.181:0/2837349889 >> v1:172.21.15.181:6789/0 conn(0x7f03640b2010 legacy=0x7f03640b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036943e700 1 -- 172.21.15.181:0/2837349889 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837891670 0 0) 0x7f0354001110 con 0x7f03640b2010 2023-10-16T10:02:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036ac41700 1 -- 172.21.15.181:0/2837349889 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640ccda0 con 0x7f03640ce680 2023-10-16T10:02:01.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036ac41700 1 --2- 172.21.15.181:0/2837349889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640ce680 0x7f03640ceb10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f035c009e50 tx=0x7f035c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8651950f92de5a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036943e700 1 -- 172.21.15.181:0/2837349889 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00e070 con 0x7f03640ce680 2023-10-16T10:02:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036943e700 1 -- 172.21.15.181:0/2837349889 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c00b9c0 con 0x7f03640ce680 2023-10-16T10:02:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.071+0000 7f036943e700 1 -- 172.21.15.181:0/2837349889 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00bd70 con 0x7f03640ce680 2023-10-16T10:02:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 -- 172.21.15.181:0/2837349889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640ce680 msgr2=0x7f03640ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 --2- 172.21.15.181:0/2837349889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640ce680 0x7f03640ceb10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f035c009e50 tx=0x7f035c00b330 comp rx=0 tx=0).stop 2023-10-16T10:02:01.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 -- 172.21.15.181:0/2837349889 shutdown_connections 2023-10-16T10:02:01.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 --2- 172.21.15.181:0/2837349889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640ce680 0x7f03640ceb10 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 --2- 172.21.15.181:0/2837349889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f0364071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 -- 172.21.15.181:0/2837349889 >> 172.21.15.181:0/2837349889 conn(0x7f03640cd010 msgr2=0x7f03640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:01.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 -- 172.21.15.181:0/2837349889 shutdown_connections 2023-10-16T10:02:01.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.072+0000 7f036c6a4700 1 -- 172.21.15.181:0/2837349889 wait complete. 2023-10-16T10:02:01.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.073+0000 7f036c6a4700 1 Processor -- start 2023-10-16T10:02:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.073+0000 7f036c6a4700 1 -- start start 2023-10-16T10:02:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640b2010 0x7f0364158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f03640ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03640ce680 0x7f03640aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03641f3e20 con 0x7f03640b2010 2023-10-16T10:02:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03640af360 con 0x7f03640ce680 2023-10-16T10:02:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03641f3060 con 0x7f03640cc6e0 2023-10-16T10:02:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640b2010 0x7f0364158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640b2010 0x7f0364158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45694/0 (socket says 172.21.15.181:45694) 2023-10-16T10:02:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 -- 172.21.15.181:0/3701847198 learned_addr learned my addr 172.21.15.181:0/3701847198 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f0369c3f700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f03640ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036ac41700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03640ce680 0x7f03640aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 -- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 msgr2=0x7f03640ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f03640ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 -- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03640ce680 msgr2=0x7f03640aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03640ce680 0x7f03640aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 -- 172.21.15.181:0/3701847198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c009b30 con 0x7f03640b2010 2023-10-16T10:02:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036ac41700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03640ce680 0x7f03640aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:01.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f0369c3f700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f03640ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036a440700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640b2010 0x7f0364158c40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f035400c550 tx=0x7f035400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354004c80 con 0x7f03640b2010 2023-10-16T10:02:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0354004e00 con 0x7f03640b2010 2023-10-16T10:02:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03641f3340 con 0x7f03640b2010 2023-10-16T10:02:01.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354007310 con 0x7f03640b2010 2023-10-16T10:02:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.074+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03641f3900 con 0x7f03640b2010 2023-10-16T10:02:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.075+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03640b6510 con 0x7f03640b2010 2023-10-16T10:02:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.076+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 63) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0354007470 con 0x7f03640b2010 2023-10-16T10:02:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.076+0000 7f035affd700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f034806af90 0x7f034806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.076+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7f03540892d0 con 0x7f03640b2010 2023-10-16T10:02:01.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.076+0000 7f0369c3f700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f034806af90 0x7f034806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.077+0000 7f0369c3f700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f034806af90 0x7f034806d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.081+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f03540541e0 con 0x7f03640b2010 2023-10-16T10:02:01.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.141+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f035401ad90 con 0x7f03640b2010 2023-10-16T10:02:01.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.276+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f03640ccb60 con 0x7f03640b2010 2023-10-16T10:02:01.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.279+0000 7f035affd700 1 -- 172.21.15.181:0/3701847198 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v38) v1 ==== 120+0+0 (secure 0 0 0) 0x7f03640ccb60 con 0x7f03640b2010 2023-10-16T10:02:01.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f034806af90 msgr2=0x7f034806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f034806af90 0x7f034806d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).stop 2023-10-16T10:02:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640b2010 msgr2=0x7f0364158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640b2010 0x7f0364158c40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f035400c550 tx=0x7f035400c910 comp rx=0 tx=0).stop 2023-10-16T10:02:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 shutdown_connections 2023-10-16T10:02:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f034806af90 0x7f034806d450 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03640ce680 0x7f03640aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03640b2010 0x7f0364158c40 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 --2- 172.21.15.181:0/3701847198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03640cc6e0 0x7f03640ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 >> 172.21.15.181:0/3701847198 conn(0x7f03640cd010 msgr2=0x7f03640b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:01.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 shutdown_connections 2023-10-16T10:02:01.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.281+0000 7f036c6a4700 1 -- 172.21.15.181:0/3701847198 wait complete. 2023-10-16T10:02:01.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:02:01.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.419+0000 7fac9fd07700 1 Processor -- start 2023-10-16T10:02:01.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.419+0000 7fac9fd07700 1 -- start start 2023-10-16T10:02:01.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9fd07700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac9815da70 0x7fac980ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9fd07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac980ce6a0 0x7fac9806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9fd07700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac98072760 0x7fac9806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9fd07700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980cef00 con 0x7fac98072760 2023-10-16T10:02:01.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9fd07700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980cec20 con 0x7fac9815da70 2023-10-16T10:02:01.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9fd07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac98071c80 con 0x7fac980ce6a0 2023-10-16T10:02:01.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac98072760 0x7fac9806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac98072760 0x7fac9806ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45706/0 (socket says 172.21.15.181:45706) 2023-10-16T10:02:01.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 -- 172.21.15.181:0/476260256 learned_addr learned my addr 172.21.15.181:0/476260256 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:01.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 -- 172.21.15.181:0/476260256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac980ce6a0 msgr2=0x7fac9806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9d2a2700 1 --2- 172.21.15.181:0/476260256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac980ce6a0 0x7fac9806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9daa3700 1 --2- 172.21.15.181:0/476260256 >> v2:172.21.15.190:3300/0 conn(0x7fac9815da70 0x7fac980ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 --2- 172.21.15.181:0/476260256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac980ce6a0 0x7fac9806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 -- 172.21.15.181:0/476260256 >> v2:172.21.15.190:3300/0 conn(0x7fac9815da70 msgr2=0x7fac980ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 --2- 172.21.15.181:0/476260256 >> v2:172.21.15.190:3300/0 conn(0x7fac9815da70 0x7fac980ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 -- 172.21.15.181:0/476260256 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98071f60 con 0x7fac98072760 2023-10-16T10:02:01.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9e2a4700 1 --2- 172.21.15.181:0/476260256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9806ade0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fac980675e0 tx=0x7fac9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4dfb6854348ad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9caa1700 1 -- 172.21.15.181:0/476260256 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000e070 con 0x7fac98072760 2023-10-16T10:02:01.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9caa1700 1 -- 172.21.15.181:0/476260256 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9000b9c0 con 0x7fac98072760 2023-10-16T10:02:01.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.420+0000 7fac9caa1700 1 -- 172.21.15.181:0/476260256 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000bd70 con 0x7fac98072760 2023-10-16T10:02:01.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 -- 172.21.15.181:0/476260256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 msgr2=0x7fac9806ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 --2- 172.21.15.181:0/476260256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9806ade0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fac980675e0 tx=0x7fac9000b330 comp rx=0 tx=0).stop 2023-10-16T10:02:01.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 -- 172.21.15.181:0/476260256 shutdown_connections 2023-10-16T10:02:01.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 --2- 172.21.15.181:0/476260256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9806ade0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 --2- 172.21.15.181:0/476260256 >> v2:172.21.15.190:3300/0 conn(0x7fac9815da70 0x7fac980ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 --2- 172.21.15.181:0/476260256 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac980ce6a0 0x7fac9806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 -- 172.21.15.181:0/476260256 >> 172.21.15.181:0/476260256 conn(0x7fac980cd010 msgr2=0x7fac980cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:01.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 -- 172.21.15.181:0/476260256 shutdown_connections 2023-10-16T10:02:01.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.421+0000 7fac9fd07700 1 -- 172.21.15.181:0/476260256 wait complete. 2023-10-16T10:02:01.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.423+0000 7fac9fd07700 1 Processor -- start 2023-10-16T10:02:01.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.423+0000 7fac9fd07700 1 -- start start 2023-10-16T10:02:01.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.423+0000 7fac9fd07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.423+0000 7fac9fd07700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac980ce6a0 0x7fac980ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9fd07700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9815da70 0x7fac981f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9fd07700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980acdb0 con 0x7fac98072760 2023-10-16T10:02:01.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9fd07700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980ad090 con 0x7fac980ce6a0 2023-10-16T10:02:01.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9fd07700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac980ad370 con 0x7fac9815da70 2023-10-16T10:02:01.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9815ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45708/0 (socket says 172.21.15.181:45708) 2023-10-16T10:02:01.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 -- 172.21.15.181:0/1443878590 learned_addr learned my addr 172.21.15.181:0/1443878590 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:01.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 -- 172.21.15.181:0/1443878590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9815da70 msgr2=0x7fac981f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9815da70 0x7fac981f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 -- 172.21.15.181:0/1443878590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac980ce6a0 msgr2=0x7fac980ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac980ce6a0 0x7fac980ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 -- 172.21.15.181:0/1443878590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac90009b30 con 0x7fac98072760 2023-10-16T10:02:01.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9daa3700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9815ff20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c7f0 tx=0x7fac8800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac8e7fc700 1 -- 172.21.15.181:0/1443878590 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88011800 con 0x7fac98072760 2023-10-16T10:02:01.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac8e7fc700 1 -- 172.21.15.181:0/1443878590 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac88011960 con 0x7fac98072760 2023-10-16T10:02:01.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac981f30c0 con 0x7fac98072760 2023-10-16T10:02:01.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac8e7fc700 1 -- 172.21.15.181:0/1443878590 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88011cf0 con 0x7fac98072760 2023-10-16T10:02:01.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.424+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980aedc0 con 0x7fac98072760 2023-10-16T10:02:01.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.425+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac981f32b0 con 0x7fac98072760 2023-10-16T10:02:01.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.426+0000 7fac8e7fc700 1 -- 172.21.15.181:0/1443878590 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fac88018360 con 0x7fac98072760 2023-10-16T10:02:01.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.426+0000 7fac8e7fc700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac7c06b020 0x7fac7c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.426+0000 7fac8e7fc700 1 -- 172.21.15.181:0/1443878590 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7fac88089a00 con 0x7fac98072760 2023-10-16T10:02:01.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.426+0000 7fac9d2a2700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac7c06b020 0x7fac7c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.426+0000 7fac9d2a2700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac7c06b020 0x7fac7c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fac94006ba0 tx=0x7fac94008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.430+0000 7fac8e7fc700 1 -- 172.21.15.181:0/1443878590 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac880546f0 con 0x7fac98072760 2023-10-16T10:02:01.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.629+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fac98158230 con 0x7fac98072760 2023-10-16T10:02:01.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.634+0000 7fac8e7fc700 1 -- 172.21.15.181:0/1443878590 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7fac88057d10 con 0x7fac98072760 2023-10-16T10:02:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac7c06b020 msgr2=0x7fac7c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac7c06b020 0x7fac7c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fac94006ba0 tx=0x7fac94008040 comp rx=0 tx=0).stop 2023-10-16T10:02:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 msgr2=0x7fac9815ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9815ff20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c7f0 tx=0x7fac8800cbb0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 shutdown_connections 2023-10-16T10:02:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac7c06b020 0x7fac7c06d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac9815da70 0x7fac981f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac98072760 0x7fac9815ff20 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 --2- 172.21.15.181:0/1443878590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac980ce6a0 0x7fac980ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 >> 172.21.15.181:0/1443878590 conn(0x7fac980cd010 msgr2=0x7fac98161cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 shutdown_connections 2023-10-16T10:02:01.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.636+0000 7fac9fd07700 1 -- 172.21.15.181:0/1443878590 wait complete. 2023-10-16T10:02:01.650 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-16T10:02:01.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.769+0000 7f1c1f9b4700 1 Processor -- start 2023-10-16T10:02:01.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.769+0000 7f1c1f9b4700 1 -- start start 2023-10-16T10:02:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1f9b4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c18159730 0x7f1c1815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1f9b4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c180cd5c0 0x7f1c180cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1f9b4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c180cee70 con 0x7f1c180cc400 2023-10-16T10:02:01.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1f9b4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c18159b10 con 0x7f1c180cd5c0 2023-10-16T10:02:01.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1f9b4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180ce540 con 0x7f1c18159730 2023-10-16T10:02:01.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1df51700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c180cd5c0 0x7f1c180cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1cf4f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1c180cc400 0x7f1c180cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56794/0 (socket says 172.21.15.181:56794) 2023-10-16T10:02:01.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1cf4f700 1 -- 172.21.15.181:0/4122870148 learned_addr learned my addr 172.21.15.181:0/4122870148 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:01.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1df51700 1 -- 172.21.15.181:0/4122870148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c18159730 msgr2=0x7f1c1815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:01.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1d750700 1 --2- 172.21.15.181:0/4122870148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c18159730 0x7f1c1815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1df51700 1 --2- 172.21.15.181:0/4122870148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c18159730 0x7f1c1815cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c0ffff700 1 -- 172.21.15.181:0/4122870148 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712938800 0 0) 0x7f1c180cee70 con 0x7f1c180cc400 2023-10-16T10:02:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.770+0000 7f1c1df51700 1 -- 172.21.15.181:0/4122870148 >> v1:172.21.15.181:6789/0 conn(0x7f1c180cc400 legacy=0x7f1c180cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1df51700 1 -- 172.21.15.181:0/4122870148 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180ce820 con 0x7f1c180cd5c0 2023-10-16T10:02:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1df51700 1 --2- 172.21.15.181:0/4122870148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c180cd5c0 0x7f1c180cf300 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009d90 tx=0x7f1c1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651abc9e68410f9f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c0ffff700 1 -- 172.21.15.181:0/4122870148 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000e070 con 0x7f1c180cd5c0 2023-10-16T10:02:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c0ffff700 1 -- 172.21.15.181:0/4122870148 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1000bbd0 con 0x7f1c180cd5c0 2023-10-16T10:02:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c0ffff700 1 -- 172.21.15.181:0/4122870148 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10012430 con 0x7f1c180cd5c0 2023-10-16T10:02:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/4122870148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c180cd5c0 msgr2=0x7f1c180cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/4122870148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c180cd5c0 0x7f1c180cf300 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009d90 tx=0x7f1c1000b330 comp rx=0 tx=0).stop 2023-10-16T10:02:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/4122870148 shutdown_connections 2023-10-16T10:02:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/4122870148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c180cd5c0 0x7f1c180cf300 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/4122870148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c18159730 0x7f1c1815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/4122870148 >> 172.21.15.181:0/4122870148 conn(0x7f1c180d04a0 msgr2=0x7f1c180d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/4122870148 shutdown_connections 2023-10-16T10:02:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.771+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/4122870148 wait complete. 2023-10-16T10:02:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.773+0000 7f1c1f9b4700 1 Processor -- start 2023-10-16T10:02:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.773+0000 7f1c1f9b4700 1 -- start start 2023-10-16T10:02:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1f9b4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180cc400 0x7f1c180c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1f9b4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c180cd5c0 0x7f1c180b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1f9b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18159730 0x7f1c180c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1f9b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c181573a0 con 0x7f1c18159730 2023-10-16T10:02:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1f9b4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c18156720 con 0x7f1c180cd5c0 2023-10-16T10:02:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1f9b4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c18156a00 con 0x7f1c180cc400 2023-10-16T10:02:01.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18159730 0x7f1c180c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18159730 0x7f1c180c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45726/0 (socket says 172.21.15.181:45726) 2023-10-16T10:02:01.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 -- 172.21.15.181:0/1128294434 learned_addr learned my addr 172.21.15.181:0/1128294434 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:01.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1d750700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180cc400 0x7f1c180c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1cf4f700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c180cd5c0 0x7f1c180b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 -- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180cc400 msgr2=0x7f1c180c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180cc400 0x7f1c180c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 -- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c180cd5c0 msgr2=0x7f1c180b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:01.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c180cd5c0 0x7f1c180b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:01.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 -- 172.21.15.181:0/1128294434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10009a70 con 0x7f1c18159730 2023-10-16T10:02:01.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.774+0000 7f1c1df51700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18159730 0x7f1c180c5fc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1c10013000 tx=0x7f1c10004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.775+0000 7f1c0dffb700 1 -- 172.21.15.181:0/1128294434 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10003ea0 con 0x7f1c18159730 2023-10-16T10:02:01.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.775+0000 7f1c0dffb700 1 -- 172.21.15.181:0/1128294434 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c10020070 con 0x7f1c18159730 2023-10-16T10:02:01.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.775+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c18156ce0 con 0x7f1c18159730 2023-10-16T10:02:01.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.775+0000 7f1c0dffb700 1 -- 172.21.15.181:0/1128294434 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10004280 con 0x7f1c18159730 2023-10-16T10:02:01.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.775+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c18157e30 con 0x7f1c18159730 2023-10-16T10:02:01.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.776+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c18067d20 con 0x7f1c18159730 2023-10-16T10:02:01.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.776+0000 7f1c0dffb700 1 -- 172.21.15.181:0/1128294434 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1c100043e0 con 0x7f1c18159730 2023-10-16T10:02:01.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.777+0000 7f1c0dffb700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f1bfc06b020 0x7f1bfc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:01.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.777+0000 7f1c0dffb700 1 -- 172.21.15.181:0/1128294434 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7f1c10022070 con 0x7f1c18159730 2023-10-16T10:02:01.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.777+0000 7f1c1d750700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f1bfc06b020 0x7f1bfc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:01.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.777+0000 7f1c1d750700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f1bfc06b020 0x7f1bfc06d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800b840 tx=0x7f1c08008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:01.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:01.781+0000 7f1c0dffb700 1 -- 172.21.15.181:0/1128294434 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1c10056620 con 0x7f1c18159730 2023-10-16T10:02:02.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.051+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f1c180c3920 con 0x7f1c18159730 2023-10-16T10:02:02.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.571+0000 7f1c0dffb700 1 -- 172.21.15.181:0/1128294434 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v672) v1 ==== 183+0+0 (secure 0 0 0) 0x7f1c10060020 con 0x7f1c18159730 2023-10-16T10:02:02.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f1bfc06b020 msgr2=0x7f1bfc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f1bfc06b020 0x7f1bfc06d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800b840 tx=0x7f1c08008040 comp rx=0 tx=0).stop 2023-10-16T10:02:02.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18159730 msgr2=0x7f1c180c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18159730 0x7f1c180c5fc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1c10013000 tx=0x7f1c10004ad0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 shutdown_connections 2023-10-16T10:02:02.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f1bfc06b020 0x7f1bfc06d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c180cc400 0x7f1c180c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c18159730 0x7f1c180c5fc0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 --2- 172.21.15.181:0/1128294434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c180cd5c0 0x7f1c180b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 >> 172.21.15.181:0/1128294434 conn(0x7f1c180d04a0 msgr2=0x7f1c180adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 shutdown_connections 2023-10-16T10:02:02.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.574+0000 7f1c1f9b4700 1 -- 172.21.15.181:0/1128294434 wait complete. 2023-10-16T10:02:02.589 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-16T10:02:02.590 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-16T10:02:02.590 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:02:02.591 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:02.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.712+0000 7f8210284700 1 Processor -- start 2023-10-16T10:02:02.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.712+0000 7f8210284700 1 -- start start 2023-10-16T10:02:02.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f8210284700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f820806cdf0 0x7f820815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f8210284700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82080cabd0 con 0x7f82080cdfb0 2023-10-16T10:02:02.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f8210284700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82080ca5c0 con 0x7f82080cc0c0 2023-10-16T10:02:02.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f8210284700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8208066740 con 0x7f820806cdf0 2023-10-16T10:02:02.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820e020700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f82080cdfb0 0x7f820815d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56808/0 (socket says 172.21.15.181:56808) 2023-10-16T10:02:02.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820e020700 1 -- 172.21.15.181:0/3715548505 learned_addr learned my addr 172.21.15.181:0/3715548505 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:02.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820e821700 1 --2- 172.21.15.181:0/3715548505 >> v2:172.21.15.190:3300/0 conn(0x7f820806cdf0 0x7f820815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711303188 0 0) 0x7f82080cabd0 con 0x7f82080cdfb0 2023-10-16T10:02:02.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f40057c0 con 0x7f82080cdfb0 2023-10-16T10:02:02.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561903373 0 0) 0x7f81f40057c0 con 0x7f82080cdfb0 2023-10-16T10:02:02.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 >> v2:172.21.15.190:3300/0 conn(0x7f820806cdf0 msgr2=0x7f820815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 --2- 172.21.15.181:0/3715548505 >> v2:172.21.15.190:3300/0 conn(0x7f820806cdf0 0x7f820815dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 >> v1:172.21.15.190:6789/0 conn(0x7f82080cc0c0 legacy=0x7f82080cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8208066a20 con 0x7f82080cdfb0 2023-10-16T10:02:02.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f8204001670 con 0x7f82080cdfb0 2023-10-16T10:02:02.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 >> v1:172.21.15.181:6789/0 conn(0x7f82080cdfb0 legacy=0x7f820815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f4005c10 0x7f81f4006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f4006a60 0x7f81f4006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.713+0000 7f820d01e700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4007420 0x7f81f400dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8208066740 con 0x7f81f4005c10 2023-10-16T10:02:02.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f400e2c0 con 0x7f81f4007420 2023-10-16T10:02:02.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f400e480 con 0x7f81f4006a60 2023-10-16T10:02:02.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e020700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f4005c10 0x7f81f4006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e821700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4007420 0x7f81f400dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820d81f700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f4006a60 0x7f81f4006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e020700 1 -- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f4006a60 msgr2=0x7f81f4006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e020700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f4006a60 0x7f81f4006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e020700 1 -- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4007420 msgr2=0x7f81f400dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:02.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e020700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4007420 0x7f81f400dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e020700 1 -- 172.21.15.181:0/3715548505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40057c0 con 0x7f81f4005c10 2023-10-16T10:02:02.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e821700 1 -- 172.21.15.181:0/3715548505 reap_dead start 2023-10-16T10:02:02.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820e020700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f4005c10 0x7f81f4006030 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f820400c2a0 tx=0x7f81f4007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4033b639055cab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204007c10 con 0x7f81f4005c10 2023-10-16T10:02:02.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820400cd80 con 0x7f81f4005c10 2023-10-16T10:02:02.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.714+0000 7f820d01e700 1 -- 172.21.15.181:0/3715548505 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204005ba0 con 0x7f81f4005c10 2023-10-16T10:02:02.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.715+0000 7f8210284700 1 -- 172.21.15.181:0/3715548505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f4005c10 msgr2=0x7f81f4006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.715+0000 7f8210284700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f4005c10 0x7f81f4006030 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f820400c2a0 tx=0x7f81f4007a90 comp rx=0 tx=0).stop 2023-10-16T10:02:02.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.715+0000 7f8210284700 1 -- 172.21.15.181:0/3715548505 shutdown_connections 2023-10-16T10:02:02.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.715+0000 7f8210284700 1 --2- 172.21.15.181:0/3715548505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81f4005c10 0x7f81f4006030 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.715+0000 7f8210284700 1 -- 172.21.15.181:0/3715548505 >> 172.21.15.181:0/3715548505 conn(0x7f82080b1780 msgr2=0x7f82080b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:02.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.715+0000 7f8210284700 1 -- 172.21.15.181:0/3715548505 shutdown_connections 2023-10-16T10:02:02.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.715+0000 7f8210284700 1 -- 172.21.15.181:0/3715548505 wait complete. 2023-10-16T10:02:02.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.716+0000 7f8210284700 1 Processor -- start 2023-10-16T10:02:02.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.716+0000 7f8210284700 1 -- start start 2023-10-16T10:02:02.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4005c10 0x7f8208157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8208158fc0 0x7f82080c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82080c6d60 0x7f82080b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82081f34d0 con 0x7f82080c6d60 2023-10-16T10:02:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8208159560 con 0x7f81f4005c10 2023-10-16T10:02:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82080b2950 con 0x7f8208158fc0 2023-10-16T10:02:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82080c6d60 0x7f82080b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82080c6d60 0x7f82080b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45746/0 (socket says 172.21.15.181:45746) 2023-10-16T10:02:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 -- 172.21.15.181:0/2067085671 learned_addr learned my addr 172.21.15.181:0/2067085671 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e020700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4005c10 0x7f8208157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820d81f700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8208158fc0 0x7f82080c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 -- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8208158fc0 msgr2=0x7f82080c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8208158fc0 0x7f82080c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 -- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4005c10 msgr2=0x7f8208157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4005c10 0x7f8208157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 -- 172.21.15.181:0/2067085671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080b2c30 con 0x7f82080c6d60 2023-10-16T10:02:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820d81f700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8208158fc0 0x7f82080c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f820e821700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82080c6d60 0x7f82080b4ba0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f82080712b0 tx=0x7f8200007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f81feffd700 1 -- 172.21.15.181:0/2067085671 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000e070 con 0x7f82080c6d60 2023-10-16T10:02:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f81feffd700 1 -- 172.21.15.181:0/2067085671 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200003d40 con 0x7f82080c6d60 2023-10-16T10:02:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82080b2f10 con 0x7f82080c6d60 2023-10-16T10:02:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f81feffd700 1 -- 172.21.15.181:0/2067085671 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200011780 con 0x7f82080c6d60 2023-10-16T10:02:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.717+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82080b4670 con 0x7f82080c6d60 2023-10-16T10:02:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.718+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82080b3430 con 0x7f82080c6d60 2023-10-16T10:02:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.719+0000 7f81feffd700 1 -- 172.21.15.181:0/2067085671 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f82000118e0 con 0x7f82080c6d60 2023-10-16T10:02:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.719+0000 7f81feffd700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f81ec06b020 0x7f81ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.719+0000 7f820e020700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f81ec06b020 0x7f81ec06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.719+0000 7f81feffd700 1 -- 172.21.15.181:0/2067085671 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7f8200089f10 con 0x7f82080c6d60 2023-10-16T10:02:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.720+0000 7f820e020700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f81ec06b020 0x7f81ec06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8204000c00 tx=0x7f8204007000 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.724+0000 7f81feffd700 1 -- 172.21.15.181:0/2067085671 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f82080b3430 con 0x7f82080c6d60 2023-10-16T10:02:02.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.917+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8208067d20 con 0x7f82080c6d60 2023-10-16T10:02:02.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.918+0000 7f81feffd700 1 -- 172.21.15.181:0/2067085671 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+14817 (secure 0 0 0) 0x7f8200054b50 con 0x7f82080c6d60 2023-10-16T10:02:02.919 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:02.920 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":110,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:02:00.567089+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/1158962128":"2023-10-17T10:01:57.562816+0000","172.21.15.181:0/1805084655":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2616603102":"2023-10-17T10:01:56.537283+0000","172.21.15.181:6832/932006106":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2180020031":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2475032366":"2023-10-17T10:01:56.537283+0000","172.21.15.181:6835/1158962128":"2023-10-17T10:01:57.562816+0000","172.21.15.181:0/1044615193":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/517006871":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/2950955543":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/929817601":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/1316320487":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/2393260999":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/633839274":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6833/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6832/3229300335":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1989217127":"2023-10-17T10:00:48.266174+0000","172.21.15.181:0/1386329415":"2023-10-17T10:00:48.266174+0000","172.21.15.181:6833/932006106":"2023-10-17T10:01:56.537283+0000","172.21.15.181:0/80380984":"2023-10-17T10:00:48.266174+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:02.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f81ec06b020 msgr2=0x7f81ec06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f81ec06b020 0x7f81ec06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8204000c00 tx=0x7f8204007000 comp rx=0 tx=0).stop 2023-10-16T10:02:02.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82080c6d60 msgr2=0x7f82080b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:02.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82080c6d60 0x7f82080b4ba0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f82080712b0 tx=0x7f8200007f00 comp rx=0 tx=0).stop 2023-10-16T10:02:02.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 shutdown_connections 2023-10-16T10:02:02.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f81ec06b020 0x7f81ec06d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82080c6d60 0x7f82080b4ba0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f4005c10 0x7f8208157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 --2- 172.21.15.181:0/2067085671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8208158fc0 0x7f82080c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:02.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 >> 172.21.15.181:0/2067085671 conn(0x7f82080b1780 msgr2=0x7f82080afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:02.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 shutdown_connections 2023-10-16T10:02:02.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:02.920+0000 7f8210284700 1 -- 172.21.15.181:0/2067085671 wait complete. 2023-10-16T10:02:02.934 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:02:02.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:02:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 Processor -- start 2023-10-16T10:02:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 -- start start 2023-10-16T10:02:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac6015da70 0x7fac600ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac600ce6a0 0x7fac6006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac60072760 0x7fac6006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac600cef00 con 0x7fac600ce6a0 2023-10-16T10:02:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac600cec20 con 0x7fac60072760 2023-10-16T10:02:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.057+0000 7fac668e7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac60071c80 con 0x7fac6015da70 2023-10-16T10:02:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac600ce6a0 0x7fac6006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac600ce6a0 0x7fac6006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45750/0 (socket says 172.21.15.181:45750) 2023-10-16T10:02:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 -- 172.21.15.181:0/3587598152 learned_addr learned my addr 172.21.15.181:0/3587598152 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac64e84700 1 --2- 172.21.15.181:0/3587598152 >> v2:172.21.15.190:3300/0 conn(0x7fac60072760 0x7fac6006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 -- 172.21.15.181:0/3587598152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac6015da70 msgr2=0x7fac600ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5ffff700 1 --2- 172.21.15.181:0/3587598152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac6015da70 0x7fac600ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 --2- 172.21.15.181:0/3587598152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac6015da70 0x7fac600ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 -- 172.21.15.181:0/3587598152 >> v2:172.21.15.190:3300/0 conn(0x7fac60072760 msgr2=0x7fac6006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 --2- 172.21.15.181:0/3587598152 >> v2:172.21.15.190:3300/0 conn(0x7fac60072760 0x7fac6006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 -- 172.21.15.181:0/3587598152 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac60071f60 con 0x7fac600ce6a0 2023-10-16T10:02:03.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac64e84700 1 --2- 172.21.15.181:0/3587598152 >> v2:172.21.15.190:3300/0 conn(0x7fac60072760 0x7fac6006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:03.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5f7fe700 1 --2- 172.21.15.181:0/3587598152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac600ce6a0 0x7fac6006a8a0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fac60161530 tx=0x7fac50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e67e11ab239acb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5effd700 1 -- 172.21.15.181:0/3587598152 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000b870 con 0x7fac600ce6a0 2023-10-16T10:02:03.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5effd700 1 -- 172.21.15.181:0/3587598152 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5000b9d0 con 0x7fac600ce6a0 2023-10-16T10:02:03.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.058+0000 7fac5effd700 1 -- 172.21.15.181:0/3587598152 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000bd60 con 0x7fac600ce6a0 2023-10-16T10:02:03.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 -- 172.21.15.181:0/3587598152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac600ce6a0 msgr2=0x7fac6006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 --2- 172.21.15.181:0/3587598152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac600ce6a0 0x7fac6006a8a0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fac60161530 tx=0x7fac50009bd0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 -- 172.21.15.181:0/3587598152 shutdown_connections 2023-10-16T10:02:03.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 --2- 172.21.15.181:0/3587598152 >> v2:172.21.15.190:3300/0 conn(0x7fac60072760 0x7fac6006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 --2- 172.21.15.181:0/3587598152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac6015da70 0x7fac600ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 --2- 172.21.15.181:0/3587598152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac600ce6a0 0x7fac6006a8a0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 -- 172.21.15.181:0/3587598152 >> 172.21.15.181:0/3587598152 conn(0x7fac600cd010 msgr2=0x7fac600cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:03.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 -- 172.21.15.181:0/3587598152 shutdown_connections 2023-10-16T10:02:03.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.059+0000 7fac668e7700 1 -- 172.21.15.181:0/3587598152 wait complete. 2023-10-16T10:02:03.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.060+0000 7fac668e7700 1 Processor -- start 2023-10-16T10:02:03.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.060+0000 7fac668e7700 1 -- start start 2023-10-16T10:02:03.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac668e7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60072760 0x7fac6015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac668e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac600ce6a0 0x7fac60151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac668e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6015da70 0x7fac600cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac668e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac60152210 con 0x7fac6015da70 2023-10-16T10:02:03.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac668e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac60160870 con 0x7fac600ce6a0 2023-10-16T10:02:03.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac668e7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6015ef20 con 0x7fac60072760 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6015da70 0x7fac600cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6015da70 0x7fac600cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45760/0 (socket says 172.21.15.181:45760) 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 -- 172.21.15.181:0/556259547 learned_addr learned my addr 172.21.15.181:0/556259547 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac5ffff700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60072760 0x7fac6015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac5f7fe700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac600ce6a0 0x7fac60151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 -- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60072760 msgr2=0x7fac6015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60072760 0x7fac6015e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 -- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac600ce6a0 msgr2=0x7fac60151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac600ce6a0 0x7fac60151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 -- 172.21.15.181:0/556259547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac50009550 con 0x7fac6015da70 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.061+0000 7fac64e84700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6015da70 0x7fac600cbdf0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fac5800cd40 tx=0x7fac5800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.062+0000 7fac5cff9700 1 -- 172.21.15.181:0/556259547 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800f480 con 0x7fac6015da70 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.062+0000 7fac5cff9700 1 -- 172.21.15.181:0/556259547 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5800f600 con 0x7fac6015da70 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.062+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac60151c50 con 0x7fac6015da70 2023-10-16T10:02:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.062+0000 7fac5cff9700 1 -- 172.21.15.181:0/556259547 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800f9b0 con 0x7fac6015da70 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.062+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6015fab0 con 0x7fac6015da70 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.063+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6006fe90 con 0x7fac6015da70 2023-10-16T10:02:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.063+0000 7fac5cff9700 1 -- 172.21.15.181:0/556259547 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fac5800b040 con 0x7fac6015da70 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.064+0000 7fac5cff9700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac4c06b020 0x7fac4c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.064+0000 7fac5cff9700 1 -- 172.21.15.181:0/556259547 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7fac5808a180 con 0x7fac6015da70 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.064+0000 7fac5ffff700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac4c06b020 0x7fac4c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.064+0000 7fac5ffff700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac4c06b020 0x7fac4c06d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac54008850 tx=0x7fac54008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.068+0000 7fac5cff9700 1 -- 172.21.15.181:0/556259547 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac58054f20 con 0x7fac6015da70 2023-10-16T10:02:03.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.259+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac60067ca0 con 0x7fac6015da70 2023-10-16T10:02:03.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.260+0000 7fac5cff9700 1 -- 172.21.15.181:0/556259547 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac58058540 con 0x7fac6015da70 2023-10-16T10:02:03.262 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:02:03.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac4c06b020 msgr2=0x7fac4c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac4c06b020 0x7fac4c06d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac54008850 tx=0x7fac54008040 comp rx=0 tx=0).stop 2023-10-16T10:02:03.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6015da70 msgr2=0x7fac600cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6015da70 0x7fac600cbdf0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fac5800cd40 tx=0x7fac5800a560 comp rx=0 tx=0).stop 2023-10-16T10:02:03.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 shutdown_connections 2023-10-16T10:02:03.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac4c06b020 0x7fac4c06d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac60072760 0x7fac6015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6015da70 0x7fac600cbdf0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 --2- 172.21.15.181:0/556259547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac600ce6a0 0x7fac60151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 >> 172.21.15.181:0/556259547 conn(0x7fac600cd010 msgr2=0x7fac600b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:03.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 shutdown_connections 2023-10-16T10:02:03.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.262+0000 7fac668e7700 1 -- 172.21.15.181:0/556259547 wait complete. 2023-10-16T10:02:03.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-16T10:02:03.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.395+0000 7fd8d7da9700 1 Processor -- start 2023-10-16T10:02:03.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.395+0000 7fd8d7da9700 1 -- start start 2023-10-16T10:02:03.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d7da9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d7da9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8d00cb0c0 0x7fd8d015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d7da9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d006ed00 con 0x7fd8d015e6d0 2023-10-16T10:02:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d7da9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d00d02d0 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d7da9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d015ad40 con 0x7fd8d015eab0 2023-10-16T10:02:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d6346700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8d00cb0c0 0x7fd8d015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d5b45700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd8d015e6d0 0x7fd8d015a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56842/0 (socket says 172.21.15.181:56842) 2023-10-16T10:02:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d5b45700 1 -- 172.21.15.181:0/680616095 learned_addr learned my addr 172.21.15.181:0/680616095 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d6346700 1 -- 172.21.15.181:0/680616095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 msgr2=0x7fd8d00d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d5344700 1 --2- 172.21.15.181:0/680616095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d4b43700 1 -- 172.21.15.181:0/680616095 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665008606 0 0) 0x7fd8d006ed00 con 0x7fd8d015e6d0 2023-10-16T10:02:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d6346700 1 --2- 172.21.15.181:0/680616095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d00d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d6346700 1 -- 172.21.15.181:0/680616095 >> v1:172.21.15.181:6789/0 conn(0x7fd8d015e6d0 legacy=0x7fd8d015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d6346700 1 -- 172.21.15.181:0/680616095 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d006e0f0 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.396+0000 7fd8d5344700 1 --2- 172.21.15.181:0/680616095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d00d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d6346700 1 --2- 172.21.15.181:0/680616095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d015a800 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009d90 tx=0x7fd8c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4007cf4e81581ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d4b43700 1 -- 172.21.15.181:0/680616095 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800e070 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d4b43700 1 -- 172.21.15.181:0/680616095 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800bbd0 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d4b43700 1 -- 172.21.15.181:0/680616095 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8012430 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 -- 172.21.15.181:0/680616095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 msgr2=0x7fd8d015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/680616095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d015a800 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009d90 tx=0x7fd8c800b330 comp rx=0 tx=0).stop 2023-10-16T10:02:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 -- 172.21.15.181:0/680616095 shutdown_connections 2023-10-16T10:02:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/680616095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d015a800 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/680616095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d00d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 -- 172.21.15.181:0/680616095 >> 172.21.15.181:0/680616095 conn(0x7fd8d00d04a0 msgr2=0x7fd8d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 -- 172.21.15.181:0/680616095 shutdown_connections 2023-10-16T10:02:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.397+0000 7fd8d7da9700 1 -- 172.21.15.181:0/680616095 wait complete. 2023-10-16T10:02:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 Processor -- start 2023-10-16T10:02:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 -- start start 2023-10-16T10:02:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d015e6d0 0x7fd8d01f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d0158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01f4240 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01f3c80 con 0x7fd8d015e6d0 2023-10-16T10:02:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d7da9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01f3f60 con 0x7fd8d015eab0 2023-10-16T10:02:03.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d5b45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d5b45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d014fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45770/0 (socket says 172.21.15.181:45770) 2023-10-16T10:02:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.399+0000 7fd8d5b45700 1 -- 172.21.15.181:0/4146213594 learned_addr learned my addr 172.21.15.181:0/4146213594 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d5344700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d015e6d0 0x7fd8d01f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d6346700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d0158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d5b45700 1 -- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 msgr2=0x7fd8d0158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d5b45700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d0158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d5b45700 1 -- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d015e6d0 msgr2=0x7fd8d01f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d5b45700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d015e6d0 0x7fd8d01f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d5b45700 1 -- 172.21.15.181:0/4146213594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8009a70 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d5b45700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d014fcc0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000e750 tx=0x7fd8c000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8c67fc700 1 -- 172.21.15.181:0/4146213594 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0005c40 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8c67fc700 1 -- 172.21.15.181:0/4146213594 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0005da0 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d00c76a0 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8c67fc700 1 -- 172.21.15.181:0/4146213594 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00072b0 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.400+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d015dc60 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.401+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d015de50 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.402+0000 7fd8c67fc700 1 -- 172.21.15.181:0/4146213594 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd8c0007410 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.402+0000 7fd8c67fc700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:03.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.402+0000 7fd8d5344700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:03.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.402+0000 7fd8c67fc700 1 -- 172.21.15.181:0/4146213594 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7fd8c008bc80 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.402+0000 7fd8d5344700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd8b406b020 0x7fd8b406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc006fd0 tx=0x7fd8cc008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:03.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.407+0000 7fd8c67fc700 1 -- 172.21.15.181:0/4146213594 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd8d015de50 con 0x7fd8d00cb0c0 2023-10-16T10:02:03.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:03.680+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fd8d015de50 con 0x7fd8d00cb0c0 2023-10-16T10:02:04.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.574+0000 7fd8c67fc700 1 -- 172.21.15.181:0/4146213594 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v674) v1 ==== 151+0+0 (secure 0 0 0) 0x7fd8d015de50 con 0x7fd8d00cb0c0 2023-10-16T10:02:04.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd8b406b020 msgr2=0x7fd8b406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd8b406b020 0x7fd8b406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc006fd0 tx=0x7fd8cc008040 comp rx=0 tx=0).stop 2023-10-16T10:02:04.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 msgr2=0x7fd8d014fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d014fcc0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000e750 tx=0x7fd8c000eb10 comp rx=0 tx=0).stop 2023-10-16T10:02:04.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 shutdown_connections 2023-10-16T10:02:04.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d015eab0 0x7fd8d0158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d00cb0c0 0x7fd8d014fcc0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 --2- 172.21.15.181:0/4146213594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d015e6d0 0x7fd8d01f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 >> 172.21.15.181:0/4146213594 conn(0x7fd8d00d04a0 msgr2=0x7fd8d00aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:04.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 shutdown_connections 2023-10-16T10:02:04.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.577+0000 7fd8d7da9700 1 -- 172.21.15.181:0/4146213594 wait complete. 2023-10-16T10:02:04.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:02:04.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.722+0000 7efdd134c700 1 Processor -- start 2023-10-16T10:02:04.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.722+0000 7efdd134c700 1 -- start start 2023-10-16T10:02:04.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdd134c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdd134c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdcc0cf890 0x7efdcc0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdd134c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdcc0ce410 0x7efdcc0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdd134c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc1624b0 con 0x7efdcc0cf890 2023-10-16T10:02:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdd134c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc0ce880 con 0x7efdcc0ce410 2023-10-16T10:02:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdd134c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0d0cb0 con 0x7efdcc15d740 2023-10-16T10:02:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdcc0cf890 0x7efdcc0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdcc0cf890 0x7efdcc0cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45778/0 (socket says 172.21.15.181:45778) 2023-10-16T10:02:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 -- 172.21.15.181:0/2717871320 learned_addr learned my addr 172.21.15.181:0/2717871320 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdcb7fe700 1 --2- 172.21.15.181:0/2717871320 >> v2:172.21.15.190:3300/0 conn(0x7efdcc0ce410 0x7efdcc0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdcaffd700 1 --2- 172.21.15.181:0/2717871320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 -- 172.21.15.181:0/2717871320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 msgr2=0x7efdcc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 --2- 172.21.15.181:0/2717871320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 -- 172.21.15.181:0/2717871320 >> v2:172.21.15.190:3300/0 conn(0x7efdcc0ce410 msgr2=0x7efdcc0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:04.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 --2- 172.21.15.181:0/2717871320 >> v2:172.21.15.190:3300/0 conn(0x7efdcc0ce410 0x7efdcc0ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 -- 172.21.15.181:0/2717871320 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc0d00a0 con 0x7efdcc0cf890 2023-10-16T10:02:04.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdcb7fe700 1 --2- 172.21.15.181:0/2717871320 >> v2:172.21.15.190:3300/0 conn(0x7efdcc0ce410 0x7efdcc0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:04.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdca7fc700 1 --2- 172.21.15.181:0/2717871320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0cded0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efdcc072870 tx=0x7efdbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e48c4015d5fff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdc9ffb700 1 -- 172.21.15.181:0/2717871320 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00b860 con 0x7efdcc0cf890 2023-10-16T10:02:04.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdc9ffb700 1 -- 172.21.15.181:0/2717871320 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc00b9c0 con 0x7efdcc0cf890 2023-10-16T10:02:04.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.723+0000 7efdc9ffb700 1 -- 172.21.15.181:0/2717871320 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00bd50 con 0x7efdcc0cf890 2023-10-16T10:02:04.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 -- 172.21.15.181:0/2717871320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 msgr2=0x7efdcc0cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 --2- 172.21.15.181:0/2717871320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0cded0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efdcc072870 tx=0x7efdbc009c00 comp rx=0 tx=0).stop 2023-10-16T10:02:04.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 -- 172.21.15.181:0/2717871320 shutdown_connections 2023-10-16T10:02:04.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 --2- 172.21.15.181:0/2717871320 >> v2:172.21.15.190:3300/0 conn(0x7efdcc0ce410 0x7efdcc0ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 --2- 172.21.15.181:0/2717871320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 --2- 172.21.15.181:0/2717871320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0cded0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 -- 172.21.15.181:0/2717871320 >> 172.21.15.181:0/2717871320 conn(0x7efdcc09a930 msgr2=0x7efdcc0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:04.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 -- 172.21.15.181:0/2717871320 shutdown_connections 2023-10-16T10:02:04.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.724+0000 7efdd134c700 1 -- 172.21.15.181:0/2717871320 wait complete. 2023-10-16T10:02:04.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.725+0000 7efdd134c700 1 Processor -- start 2023-10-16T10:02:04.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.725+0000 7efdd134c700 1 -- start start 2023-10-16T10:02:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdd134c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdcc0ce410 0x7efdcc15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdd134c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdd134c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdd134c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc15f7d0 con 0x7efdcc0cf890 2023-10-16T10:02:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdd134c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0b3280 con 0x7efdcc0ce410 2023-10-16T10:02:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdd134c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc15ebc0 con 0x7efdcc15d740 2023-10-16T10:02:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45784/0 (socket says 172.21.15.181:45784) 2023-10-16T10:02:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 -- 172.21.15.181:0/261094745 learned_addr learned my addr 172.21.15.181:0/261094745 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdcaffd700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdcc0ce410 0x7efdcc15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdcb7fe700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 -- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 msgr2=0x7efdcc157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 -- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdcc0ce410 msgr2=0x7efdcc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdcc0ce410 0x7efdcc15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.726+0000 7efdca7fc700 1 -- 172.21.15.181:0/261094745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009550 con 0x7efdcc0cf890 2023-10-16T10:02:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.727+0000 7efdca7fc700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0b2d40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009c40 tx=0x7efdbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.727+0000 7efdb37fe700 1 -- 172.21.15.181:0/261094745 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc004240 con 0x7efdcc0cf890 2023-10-16T10:02:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.727+0000 7efdb37fe700 1 -- 172.21.15.181:0/261094745 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc01f070 con 0x7efdcc0cf890 2023-10-16T10:02:04.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.727+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc15eea0 con 0x7efdcc0cf890 2023-10-16T10:02:04.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.727+0000 7efdb37fe700 1 -- 172.21.15.181:0/261094745 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0036a0 con 0x7efdcc0cf890 2023-10-16T10:02:04.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.727+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc15f460 con 0x7efdcc0cf890 2023-10-16T10:02:04.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.728+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc04ebb0 con 0x7efdcc0cf890 2023-10-16T10:02:04.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.728+0000 7efdb37fe700 1 -- 172.21.15.181:0/261094745 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efdbc021030 con 0x7efdcc0cf890 2023-10-16T10:02:04.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.729+0000 7efdb37fe700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7efdb806b020 0x7efdb806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.729+0000 7efdb37fe700 1 -- 172.21.15.181:0/261094745 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7efdbc08ab10 con 0x7efdcc0cf890 2023-10-16T10:02:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.729+0000 7efdcaffd700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7efdb806b020 0x7efdb806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.729+0000 7efdcaffd700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7efdb806b020 0x7efdb806d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efdc0006b20 tx=0x7efdc0008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.733+0000 7efdb37fe700 1 -- 172.21.15.181:0/261094745 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efdbc0558f0 con 0x7efdcc0cf890 2023-10-16T10:02:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.926+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efdcc0d0630 con 0x7efdcc0cf890 2023-10-16T10:02:04.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.930+0000 7efdb37fe700 1 -- 172.21.15.181:0/261094745 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7efdbc015020 con 0x7efdcc0cf890 2023-10-16T10:02:04.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7efdb806b020 msgr2=0x7efdb806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7efdb806b020 0x7efdb806d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efdc0006b20 tx=0x7efdc0008040 comp rx=0 tx=0).stop 2023-10-16T10:02:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 msgr2=0x7efdcc0b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0b2d40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009c40 tx=0x7efdbc004750 comp rx=0 tx=0).stop 2023-10-16T10:02:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 shutdown_connections 2023-10-16T10:02:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7efdb806b020 0x7efdb806d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdcc15d740 0x7efdcc157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdcc0ce410 0x7efdcc15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 --2- 172.21.15.181:0/261094745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdcc0cf890 0x7efdcc0b2d40 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:04.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 >> 172.21.15.181:0/261094745 conn(0x7efdcc09a930 msgr2=0x7efdcc0af700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:04.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 shutdown_connections 2023-10-16T10:02:04.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:04.932+0000 7efdd134c700 1 -- 172.21.15.181:0/261094745 wait complete. 2023-10-16T10:02:04.946 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:02:04.947 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:02:04.986 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:02:05.040 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:02:05.040 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:02:05.040 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:05.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:02:05.058 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:02:05.058 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:05.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:02:05.079 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:02:05.079 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:05.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:02:05.114 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:02:05.115 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:05.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:02:05.138 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:02:05.138 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:02:05.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.273+0000 7fb57b813700 1 Processor -- start 2023-10-16T10:02:05.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.274+0000 7fb57b813700 1 -- start start 2023-10-16T10:02:05.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.274+0000 7fb57b813700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5740cab40 0x7fb5740cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.274+0000 7fb57b813700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb57415e7f0 0x7fb57406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.274+0000 7fb57b813700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5740cc310 con 0x7fb5740b1ee0 2023-10-16T10:02:05.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.274+0000 7fb57b813700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57415e0f0 con 0x7fb5740cab40 2023-10-16T10:02:05.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.274+0000 7fb57b813700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740cf820 con 0x7fb57415e7f0 2023-10-16T10:02:05.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5740cab40 0x7fb5740cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5740cab40 0x7fb5740cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45792/0 (socket says 172.21.15.181:45792) 2023-10-16T10:02:05.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 -- 172.21.15.181:0/934239311 learned_addr learned my addr 172.21.15.181:0/934239311 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:05.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb579db0700 1 --2- 172.21.15.181:0/934239311 >> v2:172.21.15.190:3300/0 conn(0x7fb57415e7f0 0x7fb57406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 -- 172.21.15.181:0/934239311 >> v2:172.21.15.190:3300/0 conn(0x7fb57415e7f0 msgr2=0x7fb57406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:05.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 --2- 172.21.15.181:0/934239311 >> v2:172.21.15.190:3300/0 conn(0x7fb57415e7f0 0x7fb57406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 -- 172.21.15.181:0/934239311 >> v1:172.21.15.190:6789/0 conn(0x7fb5740b1ee0 legacy=0x7fb5740b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:05.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 -- 172.21.15.181:0/934239311 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740cc590 con 0x7fb5740cab40 2023-10-16T10:02:05.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb578dae700 1 --2- 172.21.15.181:0/934239311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5740cab40 0x7fb5740cf2e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb5700098f0 tx=0x7fb570009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144218712d9d3a18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb56bfff700 1 -- 172.21.15.181:0/934239311 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000b860 con 0x7fb5740cab40 2023-10-16T10:02:05.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb56bfff700 1 -- 172.21.15.181:0/934239311 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57000b9c0 con 0x7fb5740cab40 2023-10-16T10:02:05.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.275+0000 7fb56bfff700 1 -- 172.21.15.181:0/934239311 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000bd50 con 0x7fb5740cab40 2023-10-16T10:02:05.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 -- 172.21.15.181:0/934239311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5740cab40 msgr2=0x7fb5740cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 --2- 172.21.15.181:0/934239311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5740cab40 0x7fb5740cf2e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb5700098f0 tx=0x7fb570009c00 comp rx=0 tx=0).stop 2023-10-16T10:02:05.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 -- 172.21.15.181:0/934239311 shutdown_connections 2023-10-16T10:02:05.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 --2- 172.21.15.181:0/934239311 >> v2:172.21.15.190:3300/0 conn(0x7fb57415e7f0 0x7fb57406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 --2- 172.21.15.181:0/934239311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5740cab40 0x7fb5740cf2e0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 -- 172.21.15.181:0/934239311 >> 172.21.15.181:0/934239311 conn(0x7fb5740d0460 msgr2=0x7fb5740d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 -- 172.21.15.181:0/934239311 shutdown_connections 2023-10-16T10:02:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.276+0000 7fb57b813700 1 -- 172.21.15.181:0/934239311 wait complete. 2023-10-16T10:02:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.277+0000 7fb57b813700 1 Processor -- start 2023-10-16T10:02:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.277+0000 7fb57b813700 1 -- start start 2023-10-16T10:02:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb57b813700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740b1ee0 0x7fb57414f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb57b813700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740cab40 0x7fb5740b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb57b813700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57415e7f0 0x7fb5740b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb57b813700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740ca430 con 0x7fb57415e7f0 2023-10-16T10:02:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb57b813700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57415f840 con 0x7fb5740b1ee0 2023-10-16T10:02:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb57b813700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57415fb20 con 0x7fb5740cab40 2023-10-16T10:02:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57415e7f0 0x7fb5740b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57415e7f0 0x7fb5740b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45800/0 (socket says 172.21.15.181:45800) 2023-10-16T10:02:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 -- 172.21.15.181:0/180840040 learned_addr learned my addr 172.21.15.181:0/180840040 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb5795af700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740b1ee0 0x7fb57414f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 -- 172.21.15.181:0/180840040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740cab40 msgr2=0x7fb5740b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740cab40 0x7fb5740b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 -- 172.21.15.181:0/180840040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740b1ee0 msgr2=0x7fb57414f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740b1ee0 0x7fb57414f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 -- 172.21.15.181:0/180840040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570009550 con 0x7fb57415e7f0 2023-10-16T10:02:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb579db0700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57415e7f0 0x7fb5740b1380 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb56c007e40 tx=0x7fb56c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb569ffb700 1 -- 172.21.15.181:0/180840040 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00eea0 con 0x7fb57415e7f0 2023-10-16T10:02:05.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb569ffb700 1 -- 172.21.15.181:0/180840040 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c014420 con 0x7fb57415e7f0 2023-10-16T10:02:05.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57415fe00 con 0x7fb57415e7f0 2023-10-16T10:02:05.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.278+0000 7fb569ffb700 1 -- 172.21.15.181:0/180840040 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0147b0 con 0x7fb57415e7f0 2023-10-16T10:02:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.279+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5741603c0 con 0x7fb57415e7f0 2023-10-16T10:02:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.280+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb574067d20 con 0x7fb57415e7f0 2023-10-16T10:02:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.280+0000 7fb569ffb700 1 -- 172.21.15.181:0/180840040 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb56c014910 con 0x7fb57415e7f0 2023-10-16T10:02:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.280+0000 7fb569ffb700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb55806b050 0x7fb55806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.280+0000 7fb569ffb700 1 -- 172.21.15.181:0/180840040 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7fb56c08a1b0 con 0x7fb57415e7f0 2023-10-16T10:02:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.280+0000 7fb5795af700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb55806b050 0x7fb55806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.281+0000 7fb5795af700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb55806b050 0x7fb55806d510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb564000fb0 tx=0x7fb564009040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.285+0000 7fb569ffb700 1 -- 172.21.15.181:0/180840040 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb56c054f10 con 0x7fb57415e7f0 2023-10-16T10:02:05.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.525+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5740b0040 con 0x7fb57415e7f0 2023-10-16T10:02:05.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.525+0000 7fb569ffb700 1 -- 172.21.15.181:0/180840040 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fb56c058530 con 0x7fb57415e7f0 2023-10-16T10:02:05.528 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:05.528 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1358211190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1358211190},{"type":"v1","addr":"172.21.15.190:6835","nonce":1358211190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6844,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2394829192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2394829192},{"type":"v1","addr":"172.21.15.190:6834","nonce":2394829192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2217231321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2217231321},{"type":"v1","addr":"172.21.15.181:6836","nonce":2217231321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7920,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/200401990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":200401990},{"type":"v1","addr":"172.21.15.181:6837","nonce":200401990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[]} 2023-10-16T10:02:05.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb55806b050 msgr2=0x7fb55806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb55806b050 0x7fb55806d510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb564000fb0 tx=0x7fb564009040 comp rx=0 tx=0).stop 2023-10-16T10:02:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57415e7f0 msgr2=0x7fb5740b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57415e7f0 0x7fb5740b1380 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb56c007e40 tx=0x7fb56c00e3c0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 shutdown_connections 2023-10-16T10:02:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb55806b050 0x7fb55806d510 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb57415e7f0 0x7fb5740b1380 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5740b1ee0 0x7fb57414f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 --2- 172.21.15.181:0/180840040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5740cab40 0x7fb5740b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 >> 172.21.15.181:0/180840040 conn(0x7fb5740d0460 msgr2=0x7fb5740b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 shutdown_connections 2023-10-16T10:02:05.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.528+0000 7fb57b813700 1 -- 172.21.15.181:0/180840040 wait complete. 2023-10-16T10:02:05.532 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 51 2023-10-16T10:02:05.542 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:02:05.542 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:02:05.542 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:02:05.542 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:05.543 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:02:05.543 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:02:05.543 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:02:05.544 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:05.544 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:02:05.544 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:02:05.544 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:02:05.545 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:05.545 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:02:05.545 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:02:05.545 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:02:05.546 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:02:05.553 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:02:05.553 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:02:05.554 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:02:05.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:02:05.556 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:02:05.556 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:02:05.557 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:02:05.557 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:02:05.595 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:02:05.596 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:05.597 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:02:05.597 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:05.599 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:02:05.600 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:05.600 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:02:05.601 DEBUG:teuthology.parallel:result is None 2023-10-16T10:02:05.601 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:02:05.675 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T10:02:05.675 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T10:02:05.688 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T10:02:05.689 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T10:02:05.689 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-10-16T10:02:05.688+0000 7f02444db700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-16T10:02:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.727+0000 7f7b2fcf5700 1 Processor -- start 2023-10-16T10:02:05.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.727+0000 7f7b2fcf5700 1 -- start start 2023-10-16T10:02:05.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2fcf5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280ce4d0 0x7f7b280cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2fcf5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7b280cc4c0 0x7f7b2815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2fcf5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b280cf460 con 0x7f7b2806d210 2023-10-16T10:02:05.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2fcf5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b280d0380 con 0x7f7b280cc4c0 2023-10-16T10:02:05.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2fcf5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2806c700 con 0x7f7b280ce4d0 2023-10-16T10:02:05.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7b280cc4c0 0x7f7b2815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7b280cc4c0 0x7f7b2815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45830/0 (socket says 172.21.15.181:45830) 2023-10-16T10:02:05.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 -- 172.21.15.181:0/95661530 learned_addr learned my addr 172.21.15.181:0/95661530 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:05.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2d290700 1 --2- 172.21.15.181:0/95661530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280ce4d0 0x7f7b280cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 -- 172.21.15.181:0/95661530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280ce4d0 msgr2=0x7f7b280cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:05.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 --2- 172.21.15.181:0/95661530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280ce4d0 0x7f7b280cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 -- 172.21.15.181:0/95661530 >> v1:172.21.15.190:6789/0 conn(0x7f7b2806d210 legacy=0x7f7b280cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 -- 172.21.15.181:0/95661530 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2806d5f0 con 0x7f7b280cc4c0 2023-10-16T10:02:05.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.728+0000 7f7b2e292700 1 --2- 172.21.15.181:0/95661530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b280cc4c0 0x7f7b2815d2f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009e50 tx=0x7f7b2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f05e4d95c04981 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2ca8f700 1 -- 172.21.15.181:0/95661530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000e070 con 0x7f7b280cc4c0 2023-10-16T10:02:05.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2ca8f700 1 -- 172.21.15.181:0/95661530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2000b910 con 0x7f7b280cc4c0 2023-10-16T10:02:05.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2ca8f700 1 -- 172.21.15.181:0/95661530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000bcc0 con 0x7f7b280cc4c0 2023-10-16T10:02:05.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/95661530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b280cc4c0 msgr2=0x7f7b2815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/95661530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b280cc4c0 0x7f7b2815d2f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009e50 tx=0x7f7b2000b330 comp rx=0 tx=0).stop 2023-10-16T10:02:05.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/95661530 shutdown_connections 2023-10-16T10:02:05.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/95661530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b280cc4c0 0x7f7b2815d2f0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/95661530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280ce4d0 0x7f7b280cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/95661530 >> 172.21.15.181:0/95661530 conn(0x7f7b2809a9a0 msgr2=0x7f7b280d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:05.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/95661530 shutdown_connections 2023-10-16T10:02:05.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.729+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/95661530 wait complete. 2023-10-16T10:02:05.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.731+0000 7f7b2fcf5700 1 Processor -- start 2023-10-16T10:02:05.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.731+0000 7f7b2fcf5700 1 -- start start 2023-10-16T10:02:05.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b2806d210 0x7f7b2815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280cc4c0 0x7f7b280afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b280ce4d0 0x7f7b280ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280c5ed0 con 0x7f7b2806d210 2023-10-16T10:02:05.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280b03d0 con 0x7f7b280ce4d0 2023-10-16T10:02:05.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280c5250 con 0x7f7b280cc4c0 2023-10-16T10:02:05.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b2806d210 0x7f7b2815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b2806d210 0x7f7b2815ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45832/0 (socket says 172.21.15.181:45832) 2023-10-16T10:02:05.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 -- 172.21.15.181:0/2999759644 learned_addr learned my addr 172.21.15.181:0/2999759644 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:05.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2d290700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280cc4c0 0x7f7b280afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2e292700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b280ce4d0 0x7f7b280ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 -- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280cc4c0 msgr2=0x7f7b280afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280cc4c0 0x7f7b280afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 -- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b280ce4d0 msgr2=0x7f7b280ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:05.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b280ce4d0 0x7f7b280ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:05.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 -- 172.21.15.181:0/2999759644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20009b30 con 0x7f7b2806d210 2023-10-16T10:02:05.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2e292700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b280ce4d0 0x7f7b280ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:05.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2da91700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b2806d210 0x7f7b2815ee10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c450 tx=0x7f7b2400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b1e7fc700 1 -- 172.21.15.181:0/2999759644 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b240052d0 con 0x7f7b2806d210 2023-10-16T10:02:05.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b1e7fc700 1 -- 172.21.15.181:0/2999759644 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24005430 con 0x7f7b2806d210 2023-10-16T10:02:05.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b280c5530 con 0x7f7b2806d210 2023-10-16T10:02:05.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b1e7fc700 1 -- 172.21.15.181:0/2999759644 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b240038c0 con 0x7f7b2806d210 2023-10-16T10:02:05.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.732+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b280c5af0 con 0x7f7b2806d210 2023-10-16T10:02:05.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.733+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b28067d20 con 0x7f7b2806d210 2023-10-16T10:02:05.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.734+0000 7f7b1e7fc700 1 -- 172.21.15.181:0/2999759644 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7b24003a20 con 0x7f7b2806d210 2023-10-16T10:02:05.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.734+0000 7f7b1e7fc700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f7b0c06b050 0x7f7b0c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:05.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.734+0000 7f7b1e7fc700 1 -- 172.21.15.181:0/2999759644 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6141+0+0 (secure 0 0 0) 0x7f7b24088c20 con 0x7f7b2806d210 2023-10-16T10:02:05.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.734+0000 7f7b2d290700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f7b0c06b050 0x7f7b0c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:05.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.735+0000 7f7b2d290700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f7b0c06b050 0x7f7b0c06d510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b280cc210 tx=0x7f7b18009210 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:05.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.739+0000 7f7b1e7fc700 1 -- 172.21.15.181:0/2999759644 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7b24053aa0 con 0x7f7b2806d210 2023-10-16T10:02:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:05.945+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7b280b20b0 con 0x7f7b2806d210 2023-10-16T10:02:06.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.589+0000 7f7b1e7fc700 1 -- 172.21.15.181:0/2999759644 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7b24017080 con 0x7f7b2806d210 2023-10-16T10:02:06.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f7b0c06b050 msgr2=0x7f7b0c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f7b0c06b050 0x7f7b0c06d510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b280cc210 tx=0x7f7b18009210 comp rx=0 tx=0).stop 2023-10-16T10:02:06.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b2806d210 msgr2=0x7f7b2815ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b2806d210 0x7f7b2815ee10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c450 tx=0x7f7b2400c760 comp rx=0 tx=0).stop 2023-10-16T10:02:06.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 shutdown_connections 2023-10-16T10:02:06.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f7b0c06b050 0x7f7b0c06d510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b280ce4d0 0x7f7b280ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b2806d210 0x7f7b2815ee10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 --2- 172.21.15.181:0/2999759644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b280cc4c0 0x7f7b280afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 >> 172.21.15.181:0/2999759644 conn(0x7f7b2809a9a0 msgr2=0x7f7b2806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:06.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 shutdown_connections 2023-10-16T10:02:06.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.592+0000 7f7b2fcf5700 1 -- 172.21.15.181:0/2999759644 wait complete. 2023-10-16T10:02:06.597 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T10:02:06.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:02:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.736+0000 7fa387205700 1 Processor -- start 2023-10-16T10:02:06.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.736+0000 7fa387205700 1 -- start start 2023-10-16T10:02:06.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa387205700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3800d2390 0x7fa3800cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa387205700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa387205700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3800cc470 con 0x7fa3800ce7f0 2023-10-16T10:02:06.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa387205700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3800d0610 con 0x7fa3800d2390 2023-10-16T10:02:06.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa387205700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38006c810 con 0x7fa3800d0180 2023-10-16T10:02:06.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3800d2390 0x7fa3800cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa384fa1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa3800ce7f0 0x7fa38006c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56920/0 (socket says 172.21.15.181:56920) 2023-10-16T10:02:06.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa384fa1700 1 -- 172.21.15.181:0/4066442675 learned_addr learned my addr 172.21.15.181:0/4066442675 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:06.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa3857a2700 1 --2- 172.21.15.181:0/4066442675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37ffff700 1 -- 172.21.15.181:0/4066442675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 msgr2=0x7fa3800cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:06.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37ffff700 1 --2- 172.21.15.181:0/4066442675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37ffff700 1 -- 172.21.15.181:0/4066442675 >> v1:172.21.15.181:6789/0 conn(0x7fa3800ce7f0 legacy=0x7fa38006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37f7fe700 1 -- 172.21.15.181:0/4066442675 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79125832 0 0) 0x7fa3800cc470 con 0x7fa3800ce7f0 2023-10-16T10:02:06.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37ffff700 1 -- 172.21.15.181:0/4066442675 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38006ca30 con 0x7fa3800d2390 2023-10-16T10:02:06.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37ffff700 1 --2- 172.21.15.181:0/4066442675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa3800cfc40 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009870 tx=0x7fa36c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d717f2916e0fffae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:06.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37f7fe700 1 -- 172.21.15.181:0/4066442675 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c011830 con 0x7fa3800d2390 2023-10-16T10:02:06.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37f7fe700 1 -- 172.21.15.181:0/4066442675 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c011990 con 0x7fa3800d2390 2023-10-16T10:02:06.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.737+0000 7fa37f7fe700 1 -- 172.21.15.181:0/4066442675 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c011d20 con 0x7fa3800d2390 2023-10-16T10:02:06.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 -- 172.21.15.181:0/4066442675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 msgr2=0x7fa3800cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 --2- 172.21.15.181:0/4066442675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa3800cfc40 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009870 tx=0x7fa36c009b80 comp rx=0 tx=0).stop 2023-10-16T10:02:06.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 -- 172.21.15.181:0/4066442675 shutdown_connections 2023-10-16T10:02:06.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 --2- 172.21.15.181:0/4066442675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 --2- 172.21.15.181:0/4066442675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa3800cfc40 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 -- 172.21.15.181:0/4066442675 >> 172.21.15.181:0/4066442675 conn(0x7fa38009a9a0 msgr2=0x7fa3800d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:06.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 -- 172.21.15.181:0/4066442675 shutdown_connections 2023-10-16T10:02:06.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.738+0000 7fa387205700 1 -- 172.21.15.181:0/4066442675 wait complete. 2023-10-16T10:02:06.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.739+0000 7fa387205700 1 Processor -- start 2023-10-16T10:02:06.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.739+0000 7fa387205700 1 -- start start 2023-10-16T10:02:06.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa387205700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3800ce7f0 0x7fa380150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa387205700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa387205700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa38015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa387205700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38015b1b0 con 0x7fa3800d2390 2023-10-16T10:02:06.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa387205700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800c7e00 con 0x7fa3800ce7f0 2023-10-16T10:02:06.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa387205700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800c80e0 con 0x7fa3800d0180 2023-10-16T10:02:06.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa38015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa38015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45852/0 (socket says 172.21.15.181:45852) 2023-10-16T10:02:06.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 -- 172.21.15.181:0/112814363 learned_addr learned my addr 172.21.15.181:0/112814363 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:06.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa37ffff700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa384fa1700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3800ce7f0 0x7fa380150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 -- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 msgr2=0x7fa3800c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 -- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3800ce7f0 msgr2=0x7fa380150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:06.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3800ce7f0 0x7fa380150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:06.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.740+0000 7fa3857a2700 1 -- 172.21.15.181:0/112814363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0094a0 con 0x7fa3800d2390 2023-10-16T10:02:06.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.741+0000 7fa3857a2700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa38015b920 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa378007e60 tx=0x7fa37800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:06.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.741+0000 7fa37d7fa700 1 -- 172.21.15.181:0/112814363 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800ede0 con 0x7fa3800d2390 2023-10-16T10:02:06.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.741+0000 7fa37d7fa700 1 -- 172.21.15.181:0/112814363 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378014420 con 0x7fa3800d2390 2023-10-16T10:02:06.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.741+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3800c83c0 con 0x7fa3800d2390 2023-10-16T10:02:06.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.741+0000 7fa37d7fa700 1 -- 172.21.15.181:0/112814363 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3780147d0 con 0x7fa3800d2390 2023-10-16T10:02:06.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.741+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3800c7af0 con 0x7fa3800d2390 2023-10-16T10:02:06.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.742+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38015b640 con 0x7fa3800d2390 2023-10-16T10:02:06.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.742+0000 7fa37d7fa700 1 -- 172.21.15.181:0/112814363 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa378014930 con 0x7fa3800d2390 2023-10-16T10:02:06.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.742+0000 7fa37d7fa700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa36406b020 0x7fa36406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:06.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.743+0000 7fa37d7fa700 1 -- 172.21.15.181:0/112814363 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5281+0+0 (secure 0 0 0) 0x7fa378089ce0 con 0x7fa3800d2390 2023-10-16T10:02:06.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.743+0000 7fa384fa1700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa36406b020 0x7fa36406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:06.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.743+0000 7fa384fa1700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa36406b020 0x7fa36406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa374001670 tx=0x7fa374009040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:06.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:06.747+0000 7fa37d7fa700 1 -- 172.21.15.181:0/112814363 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa38015b640 con 0x7fa3800d2390 2023-10-16T10:02:07.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.014+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa38015b640 con 0x7fa3800d2390 2023-10-16T10:02:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.014+0000 7fa37d7fa700 1 -- 172.21.15.181:0/112814363 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa38015b640 con 0x7fa3800d2390 2023-10-16T10:02:07.016 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:07.016 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:02:07.017 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T10:02:07.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.017 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T10:02:07.017 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T10:02:07.017 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.018 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.018 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:02:07.018 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:02:07.018 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.018 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.019 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T10:02:07.019 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T10:02:07.019 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.020 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.020 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:02:07.020 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:02:07.020 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.020 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.021 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T10:02:07.021 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T10:02:07.021 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.022 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.022 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:02:07.022 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:02:07.022 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.022 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.023 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T10:02:07.023 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T10:02:07.023 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.023 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.024 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:02:07.024 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:02:07.024 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.024 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.025 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T10:02:07.025 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T10:02:07.025 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.025 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.026 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.028 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.028 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.029 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T10:02:07.029 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T10:02:07.029 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.030 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.030 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.030 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.030 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.031 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T10:02:07.031 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T10:02:07.031 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.032 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.032 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.033 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T10:02:07.033 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T10:02:07.033 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.033 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.033 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.034 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.034 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.034 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T10:02:07.035 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T10:02:07.035 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.035 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.035 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.036 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T10:02:07.037 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T10:02:07.037 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.037 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.037 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.037 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.037 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.038 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T10:02:07.038 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T10:02:07.039 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.039 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.039 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.039 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.040 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T10:02:07.040 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T10:02:07.040 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.040 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:02:07.041 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:02:07.041 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.041 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.042 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T10:02:07.042 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T10:02:07.042 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.042 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.043 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T10:02:07.043 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:02:07.043 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.044 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T10:02:07.044 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T10:02:07.044 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.044 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.044 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:02:07.044 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:02:07.045 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:02:07.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.045 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T10:02:07.046 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T10:02:07.046 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.046 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.046 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:02:07.046 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:02:07.047 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:02:07.047 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.047 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T10:02:07.048 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T10:02:07.048 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.048 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:02:07.048 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:02:07.048 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:02:07.049 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:02:07.049 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.049 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T10:02:07.050 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T10:02:07.050 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.050 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:02:07.050 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:02:07.050 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:02:07.051 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.051 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.051 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T10:02:07.052 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T10:02:07.052 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.052 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:02:07.052 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:02:07.052 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.053 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:02:07.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:07.053 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T10:02:07.053 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T10:02:07.054 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:02:07.054 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:02:07.054 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:02:07.054 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:02:07.054 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.055 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:07.055 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T10:02:07.055 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:02:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa36406b020 msgr2=0x7fa36406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa36406b020 0x7fa36406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa374001670 tx=0x7fa374009040 comp rx=0 tx=0).stop 2023-10-16T10:02:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 msgr2=0x7fa38015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa38015b920 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa378007e60 tx=0x7fa37800e3c0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 shutdown_connections 2023-10-16T10:02:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa36406b020 0x7fa36406d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3800d2390 0x7fa38015b920 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3800ce7f0 0x7fa380150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 --2- 172.21.15.181:0/112814363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3800d0180 0x7fa3800c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 >> 172.21.15.181:0/112814363 conn(0x7fa38009a9a0 msgr2=0x7fa380070360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 shutdown_connections 2023-10-16T10:02:07.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.016+0000 7fa387205700 1 -- 172.21.15.181:0/112814363 wait complete. 2023-10-16T10:02:07.059 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:07.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 Processor -- start 2023-10-16T10:02:07.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 -- start start 2023-10-16T10:02:07.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effe415dfd0 0x7effe40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe40af450 0x7effe406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7effe406db80 0x7effe40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe406f870 con 0x7effe415dfd0 2023-10-16T10:02:07.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe40d1160 con 0x7effe406db80 2023-10-16T10:02:07.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.178+0000 7effea83d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40cc8e0 con 0x7effe40af450 2023-10-16T10:02:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effe415dfd0 0x7effe40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effe415dfd0 0x7effe40af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45856/0 (socket says 172.21.15.181:45856) 2023-10-16T10:02:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 -- 172.21.15.181:0/1634035203 learned_addr learned my addr 172.21.15.181:0/1634035203 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe8dda700 1 --2- 172.21.15.181:0/1634035203 >> v2:172.21.15.190:3300/0 conn(0x7effe406db80 0x7effe40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe37fe700 1 --2- 172.21.15.181:0/1634035203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe40af450 0x7effe406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 -- 172.21.15.181:0/1634035203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe40af450 msgr2=0x7effe406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 --2- 172.21.15.181:0/1634035203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe40af450 0x7effe406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 -- 172.21.15.181:0/1634035203 >> v2:172.21.15.190:3300/0 conn(0x7effe406db80 msgr2=0x7effe40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 --2- 172.21.15.181:0/1634035203 >> v2:172.21.15.190:3300/0 conn(0x7effe406db80 0x7effe40cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 -- 172.21.15.181:0/1634035203 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40ccbc0 con 0x7effe415dfd0 2023-10-16T10:02:07.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe3fff700 1 --2- 172.21.15.181:0/1634035203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe415dfd0 0x7effe40af9a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7effe40cdfb0 tx=0x7effd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282dbae083d992e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe2ffd700 1 -- 172.21.15.181:0/1634035203 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4005e30 con 0x7effe415dfd0 2023-10-16T10:02:07.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe2ffd700 1 -- 172.21.15.181:0/1634035203 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd4004030 con 0x7effe415dfd0 2023-10-16T10:02:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.179+0000 7effe2ffd700 1 -- 172.21.15.181:0/1634035203 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40043e0 con 0x7effe415dfd0 2023-10-16T10:02:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 -- 172.21.15.181:0/1634035203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe415dfd0 msgr2=0x7effe40af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 --2- 172.21.15.181:0/1634035203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe415dfd0 0x7effe40af9a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7effe40cdfb0 tx=0x7effd4009bd0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 -- 172.21.15.181:0/1634035203 shutdown_connections 2023-10-16T10:02:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 --2- 172.21.15.181:0/1634035203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe415dfd0 0x7effe40af9a0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 --2- 172.21.15.181:0/1634035203 >> v2:172.21.15.190:3300/0 conn(0x7effe406db80 0x7effe40cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 --2- 172.21.15.181:0/1634035203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe40af450 0x7effe406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 -- 172.21.15.181:0/1634035203 >> 172.21.15.181:0/1634035203 conn(0x7effe40d04a0 msgr2=0x7effe40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 -- 172.21.15.181:0/1634035203 shutdown_connections 2023-10-16T10:02:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.180+0000 7effea83d700 1 -- 172.21.15.181:0/1634035203 wait complete. 2023-10-16T10:02:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.181+0000 7effea83d700 1 Processor -- start 2023-10-16T10:02:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.181+0000 7effea83d700 1 -- start start 2023-10-16T10:02:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effea83d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe406db80 0x7effe41578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effea83d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effe40af450 0x7effe40b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effea83d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe415dfd0 0x7effe40b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effea83d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40b27b0 con 0x7effe406db80 2023-10-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effea83d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe4157e30 con 0x7effe40af450 2023-10-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effea83d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40b1ba0 con 0x7effe415dfd0 2023-10-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe406db80 0x7effe41578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe406db80 0x7effe41578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45860/0 (socket says 172.21.15.181:45860) 2023-10-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 -- 172.21.15.181:0/1089224309 learned_addr learned my addr 172.21.15.181:0/1089224309 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe8dda700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe415dfd0 0x7effe40b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe37fe700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effe40af450 0x7effe40b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 -- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe415dfd0 msgr2=0x7effe40b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe415dfd0 0x7effe40b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 -- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effe40af450 msgr2=0x7effe40b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effe40af450 0x7effe40b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 -- 172.21.15.181:0/1089224309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4009500 con 0x7effe406db80 2023-10-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe37fe700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effe40af450 0x7effe40b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:07.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.182+0000 7effe3fff700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe406db80 0x7effe41578f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7effd4000c00 tx=0x7effd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.183+0000 7effe0ff9700 1 -- 172.21.15.181:0/1089224309 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4003d90 con 0x7effe406db80 2023-10-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.183+0000 7effe0ff9700 1 -- 172.21.15.181:0/1089224309 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd4011420 con 0x7effe406db80 2023-10-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.183+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe40b1e80 con 0x7effe406db80 2023-10-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.183+0000 7effe0ff9700 1 -- 172.21.15.181:0/1089224309 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40117b0 con 0x7effe406db80 2023-10-16T10:02:07.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.183+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe40b2440 con 0x7effe406db80 2023-10-16T10:02:07.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.183+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe40cd990 con 0x7effe406db80 2023-10-16T10:02:07.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.184+0000 7effe0ff9700 1 -- 172.21.15.181:0/1089224309 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7effd4011910 con 0x7effe406db80 2023-10-16T10:02:07.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.184+0000 7effe0ff9700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7effd006b020 0x7effd006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.184+0000 7effe0ff9700 1 -- 172.21.15.181:0/1089224309 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5281+0+0 (secure 0 0 0) 0x7effd408a700 con 0x7effe406db80 2023-10-16T10:02:07.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.184+0000 7effe37fe700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7effd006b020 0x7effd006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.185+0000 7effe37fe700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7effd006b020 0x7effd006d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effe415a440 tx=0x7effd8008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.189+0000 7effe0ff9700 1 -- 172.21.15.181:0/1089224309 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7effe40cd990 con 0x7effe406db80 2023-10-16T10:02:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.384+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effe40cd990 con 0x7effe406db80 2023-10-16T10:02:07.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.385+0000 7effe0ff9700 1 -- 172.21.15.181:0/1089224309 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+13547 (secure 0 0 0) 0x7effe40cd990 con 0x7effe406db80 2023-10-16T10:02:07.387 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:07.388 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":114,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:02:06.598583+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:07.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7effd006b020 msgr2=0x7effd006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7effd006b020 0x7effd006d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effe415a440 tx=0x7effd8008040 comp rx=0 tx=0).stop 2023-10-16T10:02:07.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe406db80 msgr2=0x7effe41578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe406db80 0x7effe41578f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7effd4000c00 tx=0x7effd4004750 comp rx=0 tx=0).stop 2023-10-16T10:02:07.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 shutdown_connections 2023-10-16T10:02:07.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7effd006b020 0x7effd006d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effe415dfd0 0x7effe40b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effe406db80 0x7effe41578f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 --2- 172.21.15.181:0/1089224309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effe40af450 0x7effe40b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 >> 172.21.15.181:0/1089224309 conn(0x7effe40d04a0 msgr2=0x7effe40ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 shutdown_connections 2023-10-16T10:02:07.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.387+0000 7effea83d700 1 -- 172.21.15.181:0/1089224309 wait complete. 2023-10-16T10:02:07.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:02:07.401 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:02:07.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.523+0000 7f266bb8e700 1 Processor -- start 2023-10-16T10:02:07.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.523+0000 7f266bb8e700 1 -- start start 2023-10-16T10:02:07.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f266bb8e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2664072760 0x7f266406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f266bb8e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f266406ade0 0x7f26641615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f266bb8e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266406b210 con 0x7f266415da70 2023-10-16T10:02:07.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f266bb8e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640cef00 con 0x7f2664072760 2023-10-16T10:02:07.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f266bb8e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640cec20 con 0x7f266406ade0 2023-10-16T10:02:07.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2664072760 0x7f266406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2664072760 0x7f266406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45868/0 (socket says 172.21.15.181:45868) 2023-10-16T10:02:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 -- 172.21.15.181:0/2356052269 learned_addr learned my addr 172.21.15.181:0/2356052269 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 -- 172.21.15.181:0/2356052269 >> v2:172.21.15.190:3300/0 conn(0x7f266406ade0 msgr2=0x7f26641615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 --2- 172.21.15.181:0/2356052269 >> v2:172.21.15.190:3300/0 conn(0x7f266406ade0 0x7f26641615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 -- 172.21.15.181:0/2356052269 >> v1:172.21.15.190:6789/0 conn(0x7f266415da70 legacy=0x7f26640ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 -- 172.21.15.181:0/2356052269 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2664071c80 con 0x7f2664072760 2023-10-16T10:02:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.524+0000 7f2669129700 1 --2- 172.21.15.181:0/2356052269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664072760 0x7f266406a8a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f26640b66b0 tx=0x7f2660009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189584af9e1ec5bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f2668928700 1 -- 172.21.15.181:0/2356052269 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660004230 con 0x7f2664072760 2023-10-16T10:02:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f2668928700 1 -- 172.21.15.181:0/2356052269 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660004390 con 0x7f2664072760 2023-10-16T10:02:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f2668928700 1 -- 172.21.15.181:0/2356052269 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600036a0 con 0x7f2664072760 2023-10-16T10:02:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 -- 172.21.15.181:0/2356052269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664072760 msgr2=0x7f266406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 --2- 172.21.15.181:0/2356052269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664072760 0x7f266406a8a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f26640b66b0 tx=0x7f2660009bd0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 -- 172.21.15.181:0/2356052269 shutdown_connections 2023-10-16T10:02:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 --2- 172.21.15.181:0/2356052269 >> v2:172.21.15.190:3300/0 conn(0x7f266406ade0 0x7f26641615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 --2- 172.21.15.181:0/2356052269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664072760 0x7f266406a8a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 -- 172.21.15.181:0/2356052269 >> 172.21.15.181:0/2356052269 conn(0x7f26640cd010 msgr2=0x7f26640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 -- 172.21.15.181:0/2356052269 shutdown_connections 2023-10-16T10:02:07.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.525+0000 7f266bb8e700 1 -- 172.21.15.181:0/2356052269 wait complete. 2023-10-16T10:02:07.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.526+0000 7f266bb8e700 1 Processor -- start 2023-10-16T10:02:07.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.526+0000 7f266bb8e700 1 -- start start 2023-10-16T10:02:07.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266bb8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266406ade0 0x7f26640b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266bb8e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2664072760 0x7f26641f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266bb8e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415da70 0x7f26640b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266bb8e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b4a80 con 0x7f266406ade0 2023-10-16T10:02:07.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266bb8e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641f3e80 con 0x7f266415da70 2023-10-16T10:02:07.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266bb8e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b34a0 con 0x7f2664072760 2023-10-16T10:02:07.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266992a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266406ade0 0x7f26640b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266992a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266406ade0 0x7f26640b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45874/0 (socket says 172.21.15.181:45874) 2023-10-16T10:02:07.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266992a700 1 -- 172.21.15.181:0/1056571123 learned_addr learned my addr 172.21.15.181:0/1056571123 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:07.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f2669129700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2664072760 0x7f26641f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.527+0000 7f266a12b700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415da70 0x7f26640b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266992a700 1 -- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2664072760 msgr2=0x7f26641f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266992a700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2664072760 0x7f26641f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266992a700 1 -- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415da70 msgr2=0x7f26640b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266992a700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415da70 0x7f26640b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266992a700 1 -- 172.21.15.181:0/1056571123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660009550 con 0x7f266406ade0 2023-10-16T10:02:07.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266992a700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266406ade0 0x7f26640b4540 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f265400c4a0 tx=0x7f265400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f265a7fc700 1 -- 172.21.15.181:0/1056571123 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540054e0 con 0x7f266406ade0 2023-10-16T10:02:07.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f265a7fc700 1 -- 172.21.15.181:0/1056571123 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654003680 con 0x7f266406ade0 2023-10-16T10:02:07.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640b3780 con 0x7f266406ade0 2023-10-16T10:02:07.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f265a7fc700 1 -- 172.21.15.181:0/1056571123 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654003a30 con 0x7f266406ade0 2023-10-16T10:02:07.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.528+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640b3d40 con 0x7f266406ade0 2023-10-16T10:02:07.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.529+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2664071f40 con 0x7f266406ade0 2023-10-16T10:02:07.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.529+0000 7f265a7fc700 1 -- 172.21.15.181:0/1056571123 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2654003b90 con 0x7f266406ade0 2023-10-16T10:02:07.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.530+0000 7f265a7fc700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f264806b020 0x7f264806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.530+0000 7f2669129700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f264806b020 0x7f264806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.530+0000 7f265a7fc700 1 -- 172.21.15.181:0/1056571123 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f2654088ad0 con 0x7f266406ade0 2023-10-16T10:02:07.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.530+0000 7f2669129700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f264806b020 0x7f264806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26640c6e00 tx=0x7f2660008030 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.534+0000 7f265a7fc700 1 -- 172.21.15.181:0/1056571123 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2664071f40 con 0x7f266406ade0 2023-10-16T10:02:07.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.732+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2664071f10 con 0x7f266406ade0 2023-10-16T10:02:07.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.733+0000 7f265a7fc700 1 -- 172.21.15.181:0/1056571123 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2654017020 con 0x7f266406ade0 2023-10-16T10:02:07.735 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:02:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f264806b020 msgr2=0x7f264806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f264806b020 0x7f264806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26640c6e00 tx=0x7f2660008030 comp rx=0 tx=0).stop 2023-10-16T10:02:07.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266406ade0 msgr2=0x7f26640b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266406ade0 0x7f26640b4540 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f265400c4a0 tx=0x7f265400c860 comp rx=0 tx=0).stop 2023-10-16T10:02:07.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 shutdown_connections 2023-10-16T10:02:07.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f264806b020 0x7f264806d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415da70 0x7f26640b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266406ade0 0x7f26640b4540 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 --2- 172.21.15.181:0/1056571123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2664072760 0x7f26641f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 >> 172.21.15.181:0/1056571123 conn(0x7f26640cd010 msgr2=0x7f26640cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 shutdown_connections 2023-10-16T10:02:07.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.735+0000 7f266bb8e700 1 -- 172.21.15.181:0/1056571123 wait complete. 2023-10-16T10:02:07.749 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:02:07.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.867+0000 7fa3e236a700 1 Processor -- start 2023-10-16T10:02:07.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3e236a700 1 -- start start 2023-10-16T10:02:07.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3e236a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc15dfd0 0x7fa3dc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3e236a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3dc0af450 0x7fa3dc06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3e236a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc06f870 con 0x7fa3dc06db80 2023-10-16T10:02:07.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3e236a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0d1160 con 0x7fa3dc0af450 2023-10-16T10:02:07.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3e236a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0cc8e0 con 0x7fa3dc15dfd0 2023-10-16T10:02:07.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3db7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3dc0af450 0x7fa3dc06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3db7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3dc0af450 0x7fa3dc06d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45884/0 (socket says 172.21.15.181:45884) 2023-10-16T10:02:07.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.868+0000 7fa3db7fe700 1 -- 172.21.15.181:0/1946079844 learned_addr learned my addr 172.21.15.181:0/1946079844 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:07.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/1946079844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc15dfd0 0x7fa3dc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3db7fe700 1 -- 172.21.15.181:0/1946079844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc15dfd0 msgr2=0x7fa3dc0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:07.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/1946079844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc15dfd0 0x7fa3dc0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3db7fe700 1 -- 172.21.15.181:0/1946079844 >> v1:172.21.15.190:6789/0 conn(0x7fa3dc06db80 legacy=0x7fa3dc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3db7fe700 1 -- 172.21.15.181:0/1946079844 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0ccbc0 con 0x7fa3dc0af450 2023-10-16T10:02:07.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/1946079844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc06d640 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0098f0 tx=0x7fa3cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7c48eb0b3db2c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3daffd700 1 -- 172.21.15.181:0/1946079844 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00b8a0 con 0x7fa3dc0af450 2023-10-16T10:02:07.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3daffd700 1 -- 172.21.15.181:0/1946079844 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00ba00 con 0x7fa3dc0af450 2023-10-16T10:02:07.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.869+0000 7fa3daffd700 1 -- 172.21.15.181:0/1946079844 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00bd90 con 0x7fa3dc0af450 2023-10-16T10:02:07.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 -- 172.21.15.181:0/1946079844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 msgr2=0x7fa3dc06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 --2- 172.21.15.181:0/1946079844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc06d640 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0098f0 tx=0x7fa3cc009c00 comp rx=0 tx=0).stop 2023-10-16T10:02:07.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 -- 172.21.15.181:0/1946079844 shutdown_connections 2023-10-16T10:02:07.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 --2- 172.21.15.181:0/1946079844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc15dfd0 0x7fa3dc0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 --2- 172.21.15.181:0/1946079844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc06d640 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 -- 172.21.15.181:0/1946079844 >> 172.21.15.181:0/1946079844 conn(0x7fa3dc0d04a0 msgr2=0x7fa3dc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:07.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 -- 172.21.15.181:0/1946079844 shutdown_connections 2023-10-16T10:02:07.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.870+0000 7fa3e236a700 1 -- 172.21.15.181:0/1946079844 wait complete. 2023-10-16T10:02:07.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.871+0000 7fa3e236a700 1 Processor -- start 2023-10-16T10:02:07.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.871+0000 7fa3e236a700 1 -- start start 2023-10-16T10:02:07.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc06db80 0x7fa3dc154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3dc15dfd0 0x7fa3dc159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0c4970 con 0x7fa3dc0af450 2023-10-16T10:02:07.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0c5180 con 0x7fa3dc15dfd0 2023-10-16T10:02:07.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0c4bc0 con 0x7fa3dc06db80 2023-10-16T10:02:07.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc0c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45892/0 (socket says 172.21.15.181:45892) 2023-10-16T10:02:07.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 -- 172.21.15.181:0/2215493077 learned_addr learned my addr 172.21.15.181:0/2215493077 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:07.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e0907700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3dc15dfd0 0x7fa3dc159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc06db80 0x7fa3dc154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 -- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc06db80 msgr2=0x7fa3dc154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc06db80 0x7fa3dc154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 -- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3dc15dfd0 msgr2=0x7fa3dc159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:07.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3dc15dfd0 0x7fa3dc159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:07.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 -- 172.21.15.181:0/2215493077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009550 con 0x7fa3dc0af450 2023-10-16T10:02:07.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3db7fe700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc0c6630 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009c40 tx=0x7fa3cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3d8ff9700 1 -- 172.21.15.181:0/2215493077 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0036a0 con 0x7fa3dc0af450 2023-10-16T10:02:07.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3d8ff9700 1 -- 172.21.15.181:0/2215493077 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc01f070 con 0x7fa3dc0af450 2023-10-16T10:02:07.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc0c4ea0 con 0x7fa3dc0af450 2023-10-16T10:02:07.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3d8ff9700 1 -- 172.21.15.181:0/2215493077 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc003a60 con 0x7fa3dc0af450 2023-10-16T10:02:07.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.872+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc0ae710 con 0x7fa3dc0af450 2023-10-16T10:02:07.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.873+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc0ae290 con 0x7fa3dc0af450 2023-10-16T10:02:07.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.874+0000 7fa3d8ff9700 1 -- 172.21.15.181:0/2215493077 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa3cc003bc0 con 0x7fa3dc0af450 2023-10-16T10:02:07.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.874+0000 7fa3d8ff9700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa3bc06b020 0x7fa3bc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:07.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.874+0000 7fa3d8ff9700 1 -- 172.21.15.181:0/2215493077 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5281+0+0 (secure 0 0 0) 0x7fa3cc021070 con 0x7fa3dc0af450 2023-10-16T10:02:07.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.874+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa3bc06b020 0x7fa3bc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:07.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.875+0000 7fa3dbfff700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa3bc06b020 0x7fa3bc06d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:07.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:07.879+0000 7fa3d8ff9700 1 -- 172.21.15.181:0/2215493077 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa3dc0ae290 con 0x7fa3dc0af450 2023-10-16T10:02:08.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.071+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa3dc0ae290 con 0x7fa3dc0af450 2023-10-16T10:02:08.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.071+0000 7fa3d8ff9700 1 -- 172.21.15.181:0/2215493077 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa3dc0ae290 con 0x7fa3dc0af450 2023-10-16T10:02:08.073 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:08.074 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:02:08.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:08.074 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T10:02:08.074 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T10:02:08.075 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:08.075 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:02:08.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa3bc06b020 msgr2=0x7fa3bc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa3bc06b020 0x7fa3bc06d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4008040 comp rx=0 tx=0).stop 2023-10-16T10:02:08.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 msgr2=0x7fa3dc0c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc0c6630 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009c40 tx=0x7fa3cc004750 comp rx=0 tx=0).stop 2023-10-16T10:02:08.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 shutdown_connections 2023-10-16T10:02:08.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fa3bc06b020 0x7fa3bc06d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3dc06db80 0x7fa3dc154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3dc15dfd0 0x7fa3dc159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 --2- 172.21.15.181:0/2215493077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3dc0af450 0x7fa3dc0c6630 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 >> 172.21.15.181:0/2215493077 conn(0x7fa3dc0d04a0 msgr2=0x7fa3dc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:08.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 shutdown_connections 2023-10-16T10:02:08.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.073+0000 7fa3e236a700 1 -- 172.21.15.181:0/2215493077 wait complete. 2023-10-16T10:02:08.087 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:02:08.088 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:02:08.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.212+0000 7f0f7d058700 1 Processor -- start 2023-10-16T10:02:08.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.212+0000 7f0f7d058700 1 -- start start 2023-10-16T10:02:08.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7d058700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0f780afa10 0x7f0f780ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7d058700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 0x7f0f780cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7d058700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f780cd850 con 0x7f0f780cd3f0 2023-10-16T10:02:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7d058700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7815b540 con 0x7f0f780afa10 2023-10-16T10:02:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7d058700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7815b820 con 0x7f0f7806d870 2023-10-16T10:02:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7759e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0f780cd3f0 0x7f0f780cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56980/0 (socket says 172.21.15.181:56980) 2023-10-16T10:02:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7759e700 1 -- 172.21.15.181:0/2859450751 learned_addr learned my addr 172.21.15.181:0/2859450751 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:08.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f7659c700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 0x7f0f780cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f76d9d700 1 --2- 172.21.15.181:0/2859450751 >> v2:172.21.15.190:3300/0 conn(0x7f0f780afa10 0x7f0f780ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869039064 0 0) 0x7f0f780cd850 con 0x7f0f780cd3f0 2023-10-16T10:02:08.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f600035b0 con 0x7f0f780cd3f0 2023-10-16T10:02:08.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2475138754 0 0) 0x7f0f600035b0 con 0x7f0f780cd3f0 2023-10-16T10:02:08.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 msgr2=0x7f0f780cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:08.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f75d9b700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 0x7f0f780cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 >> v2:172.21.15.190:3300/0 conn(0x7f0f780afa10 msgr2=0x7f0f780ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:08.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.213+0000 7f0f75d9b700 1 --2- 172.21.15.181:0/2859450751 >> v2:172.21.15.190:3300/0 conn(0x7f0f780afa10 0x7f0f780ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7815bb00 con 0x7f0f780cd3f0 2023-10-16T10:02:08.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f0f700019f0 con 0x7f0f780cd3f0 2023-10-16T10:02:08.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 >> v1:172.21.15.181:6789/0 conn(0x7f0f780cd3f0 legacy=0x7f0f780cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f60003a00 0x7f0f60003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 0x7f0f60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f60004d90 0x7f0f60005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7815b540 con 0x7f0f60004d90 2023-10-16T10:02:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7815b820 con 0x7f0f60003a00 2023-10-16T10:02:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6000ba10 con 0x7f0f7806d870 2023-10-16T10:02:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7759e700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f60004d90 0x7f0f60005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f76d9d700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f60003a00 0x7f0f60003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7659c700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 0x7f0f60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7759e700 1 -- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 msgr2=0x7f0f60004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7759e700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f7806d870 0x7f0f60004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7759e700 1 -- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f60003a00 msgr2=0x7f0f60003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7759e700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f60003a00 0x7f0f60003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7759e700 1 -- 172.21.15.181:0/2859450751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f600035b0 con 0x7f0f60004d90 2023-10-16T10:02:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f7759e700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f60004d90 0x7f0f60005210 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000a8d0 tx=0x7f0f60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4189e868840627c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7000a340 con 0x7f0f60004d90 2023-10-16T10:02:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f70012600 con 0x7f0f60004d90 2023-10-16T10:02:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.214+0000 7f0f75d9b700 1 -- 172.21.15.181:0/2859450751 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f700129b0 con 0x7f0f60004d90 2023-10-16T10:02:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.215+0000 7f0f7d058700 1 -- 172.21.15.181:0/2859450751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f60004d90 msgr2=0x7f0f60005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.215+0000 7f0f7d058700 1 --2- 172.21.15.181:0/2859450751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f60004d90 0x7f0f60005210 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000a8d0 tx=0x7f0f60005880 comp rx=0 tx=0).stop 2023-10-16T10:02:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.215+0000 7f0f7759e700 1 -- 172.21.15.181:0/2859450751 reap_dead start 2023-10-16T10:02:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.215+0000 7f0f7d058700 1 -- 172.21.15.181:0/2859450751 shutdown_connections 2023-10-16T10:02:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.215+0000 7f0f7d058700 1 -- 172.21.15.181:0/2859450751 >> 172.21.15.181:0/2859450751 conn(0x7f0f780d04a0 msgr2=0x7f0f780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.215+0000 7f0f7d058700 1 -- 172.21.15.181:0/2859450751 shutdown_connections 2023-10-16T10:02:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.215+0000 7f0f7d058700 1 -- 172.21.15.181:0/2859450751 wait complete. 2023-10-16T10:02:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.216+0000 7f0f7d058700 1 Processor -- start 2023-10-16T10:02:08.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.216+0000 7f0f7d058700 1 -- start start 2023-10-16T10:02:08.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7d058700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f78150110 0x7f0f780c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7d058700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f78159f70 0x7f0f7815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7d058700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f78151a30 0x7f0f78151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7d058700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78152400 con 0x7f0f78151a30 2023-10-16T10:02:08.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7d058700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7815aa10 con 0x7f0f78159f70 2023-10-16T10:02:08.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7d058700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7815acf0 con 0x7f0f78150110 2023-10-16T10:02:08.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f78151a30 0x7f0f78151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f78151a30 0x7f0f78151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45914/0 (socket says 172.21.15.181:45914) 2023-10-16T10:02:08.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f76d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f78150110 0x7f0f780c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f78159f70 0x7f0f7815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.217+0000 7f0f7759e700 1 -- 172.21.15.181:0/2718677717 learned_addr learned my addr 172.21.15.181:0/2718677717 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:08.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7759e700 1 -- 172.21.15.181:0/2718677717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f78150110 msgr2=0x7f0f780c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7759e700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f78150110 0x7f0f780c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7759e700 1 -- 172.21.15.181:0/2718677717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f78159f70 msgr2=0x7f0f7815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7759e700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f78159f70 0x7f0f7815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7759e700 1 -- 172.21.15.181:0/2718677717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f780c5430 con 0x7f0f78151a30 2023-10-16T10:02:08.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7759e700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f78151a30 0x7f0f78151ec0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0f70015040 tx=0x7f0f70001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:08.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f5f7fe700 1 -- 172.21.15.181:0/2718677717 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7000a340 con 0x7f0f78151a30 2023-10-16T10:02:08.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f5f7fe700 1 -- 172.21.15.181:0/2718677717 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f700019c0 con 0x7f0f78151a30 2023-10-16T10:02:08.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f780c5710 con 0x7f0f78151a30 2023-10-16T10:02:08.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f5f7fe700 1 -- 172.21.15.181:0/2718677717 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70027410 con 0x7f0f78151a30 2023-10-16T10:02:08.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.218+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f78150b30 con 0x7f0f78151a30 2023-10-16T10:02:08.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.219+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f780ccd20 con 0x7f0f78151a30 2023-10-16T10:02:08.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.219+0000 7f0f5f7fe700 1 -- 172.21.15.181:0/2718677717 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0f70019070 con 0x7f0f78151a30 2023-10-16T10:02:08.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.220+0000 7f0f5f7fe700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f0f6406b020 0x7f0f6406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.220+0000 7f0f5f7fe700 1 -- 172.21.15.181:0/2718677717 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5281+0+0 (secure 0 0 0) 0x7f0f7008a850 con 0x7f0f78151a30 2023-10-16T10:02:08.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.220+0000 7f0f76d9d700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f0f6406b020 0x7f0f6406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.220+0000 7f0f76d9d700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f0f6406b020 0x7f0f6406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f7815b470 tx=0x7f0f6c0091c0 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:08.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.224+0000 7f0f5f7fe700 1 -- 172.21.15.181:0/2718677717 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0f70055ae0 con 0x7f0f78151a30 2023-10-16T10:02:08.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.418+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0f780c5c30 con 0x7f0f78151a30 2023-10-16T10:02:08.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.596+0000 7f0f5f7fe700 1 -- 172.21.15.181:0/2718677717 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0f780c5c30 con 0x7f0f78151a30 2023-10-16T10:02:08.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f0f6406b020 msgr2=0x7f0f6406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f0f6406b020 0x7f0f6406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f7815b470 tx=0x7f0f6c0091c0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f78151a30 msgr2=0x7f0f78151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f78151a30 0x7f0f78151ec0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0f70015040 tx=0x7f0f70001360 comp rx=0 tx=0).stop 2023-10-16T10:02:08.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 shutdown_connections 2023-10-16T10:02:08.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f0f6406b020 0x7f0f6406d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f78150110 0x7f0f780c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f78151a30 0x7f0f78151ec0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 --2- 172.21.15.181:0/2718677717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f78159f70 0x7f0f7815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 >> 172.21.15.181:0/2718677717 conn(0x7f0f780d04a0 msgr2=0x7f0f78154a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:08.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 shutdown_connections 2023-10-16T10:02:08.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.599+0000 7f0f7d058700 1 -- 172.21.15.181:0/2718677717 wait complete. 2023-10-16T10:02:08.604 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T10:02:08.617 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:02:08.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.744+0000 7fb7dbcb2700 1 Processor -- start 2023-10-16T10:02:08.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.744+0000 7fb7dbcb2700 1 -- start start 2023-10-16T10:02:08.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7dbcb2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7d4072760 0x7fb7d406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7dbcb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7d406ade0 0x7fb7d41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7dbcb2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d406b210 con 0x7fb7d415da70 2023-10-16T10:02:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7dbcb2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d40cef00 con 0x7fb7d406ade0 2023-10-16T10:02:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7dbcb2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d40cec20 con 0x7fb7d4072760 2023-10-16T10:02:08.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7d406ade0 0x7fb7d41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7d406ade0 0x7fb7d41615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45922/0 (socket says 172.21.15.181:45922) 2023-10-16T10:02:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 -- 172.21.15.181:0/2035840026 learned_addr learned my addr 172.21.15.181:0/2035840026 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 -- 172.21.15.181:0/2035840026 >> v2:172.21.15.190:3300/0 conn(0x7fb7d4072760 msgr2=0x7fb7d406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 --2- 172.21.15.181:0/2035840026 >> v2:172.21.15.190:3300/0 conn(0x7fb7d4072760 0x7fb7d406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 -- 172.21.15.181:0/2035840026 >> v1:172.21.15.190:6789/0 conn(0x7fb7d415da70 legacy=0x7fb7d40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 -- 172.21.15.181:0/2035840026 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d4071c80 con 0x7fb7d406ade0 2023-10-16T10:02:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7da24f700 1 --2- 172.21.15.181:0/2035840026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d41615f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb7d406ef60 tx=0x7fb7cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0b436de8c8ec6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:08.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.745+0000 7fb7d8a4c700 1 -- 172.21.15.181:0/2035840026 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00e070 con 0x7fb7d406ade0 2023-10-16T10:02:08.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7d8a4c700 1 -- 172.21.15.181:0/2035840026 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc003a20 con 0x7fb7d406ade0 2023-10-16T10:02:08.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7d8a4c700 1 -- 172.21.15.181:0/2035840026 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc003dd0 con 0x7fb7d406ade0 2023-10-16T10:02:08.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/2035840026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 msgr2=0x7fb7d41615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/2035840026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d41615f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb7d406ef60 tx=0x7fb7cc00b330 comp rx=0 tx=0).stop 2023-10-16T10:02:08.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/2035840026 shutdown_connections 2023-10-16T10:02:08.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/2035840026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d41615f0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/2035840026 >> v2:172.21.15.190:3300/0 conn(0x7fb7d4072760 0x7fb7d406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/2035840026 >> 172.21.15.181:0/2035840026 conn(0x7fb7d40cd010 msgr2=0x7fb7d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:08.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/2035840026 shutdown_connections 2023-10-16T10:02:08.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.746+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/2035840026 wait complete. 2023-10-16T10:02:08.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.748+0000 7fb7dbcb2700 1 Processor -- start 2023-10-16T10:02:08.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.748+0000 7fb7dbcb2700 1 -- start start 2023-10-16T10:02:08.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d40af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7d4072760 0x7fb7d415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7d415da70 0x7fb7d40c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d40aeef0 con 0x7fb7d406ade0 2023-10-16T10:02:08.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d40ae2e0 con 0x7fb7d4072760 2023-10-16T10:02:08.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d40ae5c0 con 0x7fb7d415da70 2023-10-16T10:02:08.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d40af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d40af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45924/0 (socket says 172.21.15.181:45924) 2023-10-16T10:02:08.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 -- 172.21.15.181:0/439805329 learned_addr learned my addr 172.21.15.181:0/439805329 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:08.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 -- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7d415da70 msgr2=0x7fb7d40c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7d415da70 0x7fb7d40c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 -- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7d4072760 msgr2=0x7fb7d415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:08.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d924d700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7d4072760 0x7fb7d415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7d4072760 0x7fb7d415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:08.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 -- 172.21.15.181:0/439805329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc009b30 con 0x7fb7d406ade0 2023-10-16T10:02:08.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d924d700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7d4072760 0x7fb7d415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:08.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7d9a4e700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d40af380 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400c4a0 tx=0x7fb7c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:08.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7ca7fc700 1 -- 172.21.15.181:0/439805329 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c40054e0 con 0x7fb7d406ade0 2023-10-16T10:02:08.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7ca7fc700 1 -- 172.21.15.181:0/439805329 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c4003680 con 0x7fb7d406ade0 2023-10-16T10:02:08.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d40ae8a0 con 0x7fb7d406ade0 2023-10-16T10:02:08.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7ca7fc700 1 -- 172.21.15.181:0/439805329 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4003a30 con 0x7fb7d406ade0 2023-10-16T10:02:08.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.749+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d415a630 con 0x7fb7d406ade0 2023-10-16T10:02:08.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.750+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d40af1d0 con 0x7fb7d406ade0 2023-10-16T10:02:08.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.751+0000 7fb7ca7fc700 1 -- 172.21.15.181:0/439805329 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb7c4003b90 con 0x7fb7d406ade0 2023-10-16T10:02:08.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.751+0000 7fb7ca7fc700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb7b806b020 0x7fb7b806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:08.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.751+0000 7fb7ca7fc700 1 -- 172.21.15.181:0/439805329 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5651+0+0 (secure 0 0 0) 0x7fb7c40888a0 con 0x7fb7d406ade0 2023-10-16T10:02:08.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.751+0000 7fb7d924d700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb7b806b020 0x7fb7b806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:08.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.752+0000 7fb7d924d700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb7b806b020 0x7fb7b806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7d41f6480 tx=0x7fb7d0008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:08.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.756+0000 7fb7ca7fc700 1 -- 172.21.15.181:0/439805329 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb7c40537f0 con 0x7fb7d406ade0 2023-10-16T10:02:08.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:08.949+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb7d406f6c0 con 0x7fb7d406ade0 2023-10-16T10:02:09.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.598+0000 7fb7ca7fc700 1 -- 172.21.15.181:0/439805329 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb7c4056e10 con 0x7fb7d406ade0 2023-10-16T10:02:09.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.603+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb7b806b020 msgr2=0x7fb7b806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.603+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb7b806b020 0x7fb7b806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7d41f6480 tx=0x7fb7d0008040 comp rx=0 tx=0).stop 2023-10-16T10:02:09.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.603+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 msgr2=0x7fb7d40af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.603+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d40af380 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400c4a0 tx=0x7fb7c400c860 comp rx=0 tx=0).stop 2023-10-16T10:02:09.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.604+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 shutdown_connections 2023-10-16T10:02:09.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.604+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb7b806b020 0x7fb7b806d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.604+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7d415da70 0x7fb7d40c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.604+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7d406ade0 0x7fb7d40af380 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.604+0000 7fb7dbcb2700 1 --2- 172.21.15.181:0/439805329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7d4072760 0x7fb7d415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.604+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 >> 172.21.15.181:0/439805329 conn(0x7fb7d40cd010 msgr2=0x7fb7d40cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:09.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.606+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 shutdown_connections 2023-10-16T10:02:09.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.607+0000 7fb7dbcb2700 1 -- 172.21.15.181:0/439805329 wait complete. 2023-10-16T10:02:09.621 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T10:02:09.634 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:02:09.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.758+0000 7f8561a96700 1 Processor -- start 2023-10-16T10:02:09.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.758+0000 7f8561a96700 1 -- start start 2023-10-16T10:02:09.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f8561a96700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f855c06bef0 0x7f855c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f8561a96700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f8561a96700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855c06e4b0 con 0x7f855c159e10 2023-10-16T10:02:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f8561a96700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855c0cecb0 con 0x7f855c06bef0 2023-10-16T10:02:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f8561a96700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c06ed50 con 0x7f855c0cb0e0 2023-10-16T10:02:09.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f855bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f855affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f855c06bef0 0x7f855c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f855bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c15db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:42960/0 (socket says 172.21.15.181:42960) 2023-10-16T10:02:09.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f855b7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f855c159e10 0x7f855c0ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:40954/0 (socket says 172.21.15.181:40954) 2023-10-16T10:02:09.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.759+0000 7f855bfff700 1 -- 172.21.15.181:0/2800438165 learned_addr learned my addr 172.21.15.181:0/2800438165 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:09.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855affd700 1 -- 172.21.15.181:0/2800438165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 msgr2=0x7f855c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:09.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855affd700 1 --2- 172.21.15.181:0/2800438165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855affd700 1 -- 172.21.15.181:0/2800438165 >> v1:172.21.15.190:6789/0 conn(0x7f855c159e10 legacy=0x7f855c0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855affd700 1 -- 172.21.15.181:0/2800438165 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c0ceea0 con 0x7f855c06bef0 2023-10-16T10:02:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855a7fc700 1 -- 172.21.15.181:0/2800438165 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400219446 0 0) 0x7f855c06e4b0 con 0x7f855c159e10 2023-10-16T10:02:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855affd700 1 --2- 172.21.15.181:0/2800438165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c06bef0 0x7f855c0caba0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f854c009840 tx=0x7f854c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6601af6c5942aa18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855a7fc700 1 -- 172.21.15.181:0/2800438165 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c011830 con 0x7f855c06bef0 2023-10-16T10:02:09.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855a7fc700 1 -- 172.21.15.181:0/2800438165 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c011990 con 0x7f855c06bef0 2023-10-16T10:02:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f855a7fc700 1 -- 172.21.15.181:0/2800438165 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c011d20 con 0x7f855c06bef0 2023-10-16T10:02:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f8561a96700 1 -- 172.21.15.181:0/2800438165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c06bef0 msgr2=0x7f855c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f8561a96700 1 --2- 172.21.15.181:0/2800438165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c06bef0 0x7f855c0caba0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f854c009840 tx=0x7f854c009b50 comp rx=0 tx=0).stop 2023-10-16T10:02:09.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f8561a96700 1 -- 172.21.15.181:0/2800438165 shutdown_connections 2023-10-16T10:02:09.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f8561a96700 1 --2- 172.21.15.181:0/2800438165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f8561a96700 1 --2- 172.21.15.181:0/2800438165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c06bef0 0x7f855c0caba0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.760+0000 7f8561a96700 1 -- 172.21.15.181:0/2800438165 >> 172.21.15.181:0/2800438165 conn(0x7f855c0cc730 msgr2=0x7f855c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:09.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.761+0000 7f8561a96700 1 -- 172.21.15.181:0/2800438165 shutdown_connections 2023-10-16T10:02:09.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.761+0000 7f8561a96700 1 -- 172.21.15.181:0/2800438165 wait complete. 2023-10-16T10:02:09.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.762+0000 7f8561a96700 1 Processor -- start 2023-10-16T10:02:09.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.762+0000 7f8561a96700 1 -- start start 2023-10-16T10:02:09.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c06bef0 0x7f855c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c159e10 0x7f855c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c152280 con 0x7f855c06bef0 2023-10-16T10:02:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c151d80 con 0x7f855c159e10 2023-10-16T10:02:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c156d80 con 0x7f855c0cb0e0 2023-10-16T10:02:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c06bef0 0x7f855c151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c06bef0 0x7f855c151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45938/0 (socket says 172.21.15.181:45938) 2023-10-16T10:02:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 -- 172.21.15.181:0/898637494 learned_addr learned my addr 172.21.15.181:0/898637494 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855affd700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855bfff700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c159e10 0x7f855c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 -- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 msgr2=0x7f855c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 -- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c159e10 msgr2=0x7f855c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c159e10 0x7f855c0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 -- 172.21.15.181:0/898637494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0094a0 con 0x7f855c06bef0 2023-10-16T10:02:09.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855affd700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855bfff700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c159e10 0x7f855c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f855b7fe700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c06bef0 0x7f855c151840 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f855000c550 tx=0x7f855000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8543fff700 1 -- 172.21.15.181:0/898637494 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550004c80 con 0x7f855c06bef0 2023-10-16T10:02:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8543fff700 1 -- 172.21.15.181:0/898637494 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8550004e00 con 0x7f855c06bef0 2023-10-16T10:02:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c157060 con 0x7f855c06bef0 2023-10-16T10:02:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8543fff700 1 -- 172.21.15.181:0/898637494 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550007310 con 0x7f855c06bef0 2023-10-16T10:02:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.763+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c157620 con 0x7f855c06bef0 2023-10-16T10:02:09.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.764+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c0cd5d0 con 0x7f855c06bef0 2023-10-16T10:02:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.765+0000 7f8543fff700 1 -- 172.21.15.181:0/898637494 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8550007470 con 0x7f855c06bef0 2023-10-16T10:02:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.765+0000 7f8543fff700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f854806b020 0x7f854806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.766+0000 7f855affd700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f854806b020 0x7f854806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.766+0000 7f8543fff700 1 -- 172.21.15.181:0/898637494 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6033+0+0 (secure 0 0 0) 0x7f85500895a0 con 0x7f855c06bef0 2023-10-16T10:02:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.766+0000 7f855affd700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f854806b020 0x7f854806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f855c06c300 tx=0x7f854c008030 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:09.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:09.770+0000 7f8543fff700 1 -- 172.21.15.181:0/898637494 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8550054400 con 0x7f855c06bef0 2023-10-16T10:02:10.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.008+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f855c0ca4a0 con 0x7f855c06bef0 2023-10-16T10:02:10.460 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:10.464 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:10.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.620+0000 7f8543fff700 1 -- 172.21.15.181:0/898637494 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8550057a20 con 0x7f855c06bef0 2023-10-16T10:02:10.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.625+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f854806b020 msgr2=0x7f854806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.625+0000 7f8561a96700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f854806b020 0x7f854806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f855c06c300 tx=0x7f854c008030 comp rx=0 tx=0).stop 2023-10-16T10:02:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.625+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c06bef0 msgr2=0x7f855c151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.625+0000 7f8561a96700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c06bef0 0x7f855c151840 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f855000c550 tx=0x7f855000c910 comp rx=0 tx=0).stop 2023-10-16T10:02:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 shutdown_connections 2023-10-16T10:02:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f854806b020 0x7f854806d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f855c159e10 0x7f855c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c06bef0 0x7f855c151840 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 --2- 172.21.15.181:0/898637494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f855c0cb0e0 0x7f855c0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 >> 172.21.15.181:0/898637494 conn(0x7f855c0cc730 msgr2=0x7f855c152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:10.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 shutdown_connections 2023-10-16T10:02:10.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.626+0000 7f8561a96700 1 -- 172.21.15.181:0/898637494 wait complete. 2023-10-16T10:02:10.644 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 14 and data pool 15 2023-10-16T10:02:10.654 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:10.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.774+0000 7fc974507700 1 Processor -- start 2023-10-16T10:02:10.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.774+0000 7fc974507700 1 -- start start 2023-10-16T10:02:10.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.775+0000 7fc974507700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc96c15dfd0 0x7fc96c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.775+0000 7fc974507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c0af450 0x7fc96c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.775+0000 7fc974507700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c06f870 con 0x7fc96c06db80 2023-10-16T10:02:10.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.775+0000 7fc974507700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c0d1160 con 0x7fc96c15dfd0 2023-10-16T10:02:10.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.775+0000 7fc974507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c0cc8e0 con 0x7fc96c0af450 2023-10-16T10:02:10.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.775+0000 7fc9722a3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc96c15dfd0 0x7fc96c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9722a3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc96c15dfd0 0x7fc96c0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45960/0 (socket says 172.21.15.181:45960) 2023-10-16T10:02:10.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc972aa4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc96c06db80 0x7fc96c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57042/0 (socket says 172.21.15.181:57042) 2023-10-16T10:02:10.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9722a3700 1 -- 172.21.15.181:0/667557232 learned_addr learned my addr 172.21.15.181:0/667557232 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:10.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc971aa2700 1 --2- 172.21.15.181:0/667557232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c0af450 0x7fc96c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9722a3700 1 -- 172.21.15.181:0/667557232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c0af450 msgr2=0x7fc96c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:10.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9722a3700 1 --2- 172.21.15.181:0/667557232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c0af450 0x7fc96c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9722a3700 1 -- 172.21.15.181:0/667557232 >> v1:172.21.15.181:6789/0 conn(0x7fc96c06db80 legacy=0x7fc96c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9722a3700 1 -- 172.21.15.181:0/667557232 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c0ccbc0 con 0x7fc96c15dfd0 2023-10-16T10:02:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9712a1700 1 -- 172.21.15.181:0/667557232 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079816985 0 0) 0x7fc96c06f870 con 0x7fc96c06db80 2023-10-16T10:02:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9722a3700 1 --2- 172.21.15.181:0/667557232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c15dfd0 0x7fc96c0af9a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006fd0 tx=0x7fc95c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef22e314fa4d2a80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9712a1700 1 -- 172.21.15.181:0/667557232 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c005e30 con 0x7fc96c15dfd0 2023-10-16T10:02:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9712a1700 1 -- 172.21.15.181:0/667557232 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95c004030 con 0x7fc96c15dfd0 2023-10-16T10:02:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.776+0000 7fc9712a1700 1 -- 172.21.15.181:0/667557232 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0043e0 con 0x7fc96c15dfd0 2023-10-16T10:02:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 -- 172.21.15.181:0/667557232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c15dfd0 msgr2=0x7fc96c0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 --2- 172.21.15.181:0/667557232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c15dfd0 0x7fc96c0af9a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006fd0 tx=0x7fc95c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 -- 172.21.15.181:0/667557232 shutdown_connections 2023-10-16T10:02:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 --2- 172.21.15.181:0/667557232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c15dfd0 0x7fc96c0af9a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 --2- 172.21.15.181:0/667557232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c0af450 0x7fc96c06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 -- 172.21.15.181:0/667557232 >> 172.21.15.181:0/667557232 conn(0x7fc96c0d04a0 msgr2=0x7fc96c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 -- 172.21.15.181:0/667557232 shutdown_connections 2023-10-16T10:02:10.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.777+0000 7fc974507700 1 -- 172.21.15.181:0/667557232 wait complete. 2023-10-16T10:02:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.778+0000 7fc974507700 1 Processor -- start 2023-10-16T10:02:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.778+0000 7fc974507700 1 -- start start 2023-10-16T10:02:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c06db80 0x7fc96c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c0af450 0x7fc96c0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c15dfd0 0x7fc96c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c0b6030 con 0x7fc96c0af450 2023-10-16T10:02:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c0ca370 con 0x7fc96c15dfd0 2023-10-16T10:02:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c0ca090 con 0x7fc96c06db80 2023-10-16T10:02:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c0af450 0x7fc96c0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c0af450 0x7fc96c0b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45968/0 (socket says 172.21.15.181:45968) 2023-10-16T10:02:10.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 -- 172.21.15.181:0/3969437141 learned_addr learned my addr 172.21.15.181:0/3969437141 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:10.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc9722a3700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c06db80 0x7fc96c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc972aa4700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c15dfd0 0x7fc96c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 -- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c06db80 msgr2=0x7fc96c0b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c06db80 0x7fc96c0b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 -- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c15dfd0 msgr2=0x7fc96c0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c15dfd0 0x7fc96c0c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 -- 172.21.15.181:0/3969437141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c009500 con 0x7fc96c0af450 2023-10-16T10:02:10.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc971aa2700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c0af450 0x7fc96c0b52e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc96800e810 tx=0x7fc96800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc962ffd700 1 -- 172.21.15.181:0/3969437141 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968013610 con 0x7fc96c0af450 2023-10-16T10:02:10.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc962ffd700 1 -- 172.21.15.181:0/3969437141 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968013770 con 0x7fc96c0af450 2023-10-16T10:02:10.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c0b59e0 con 0x7fc96c0af450 2023-10-16T10:02:10.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc962ffd700 1 -- 172.21.15.181:0/3969437141 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968013b20 con 0x7fc96c0af450 2023-10-16T10:02:10.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.779+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c1580a0 con 0x7fc96c0af450 2023-10-16T10:02:10.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.780+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96c0c5a10 con 0x7fc96c0af450 2023-10-16T10:02:10.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.781+0000 7fc962ffd700 1 -- 172.21.15.181:0/3969437141 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc968013c80 con 0x7fc96c0af450 2023-10-16T10:02:10.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.781+0000 7fc962ffd700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fc95006b020 0x7fc95006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:10.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.781+0000 7fc9722a3700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fc95006b020 0x7fc95006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:10.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.781+0000 7fc962ffd700 1 -- 172.21.15.181:0/3969437141 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fc968089790 con 0x7fc96c0af450 2023-10-16T10:02:10.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.782+0000 7fc9722a3700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fc95006b020 0x7fc95006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000c00 tx=0x7fc95c008070 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:10.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.786+0000 7fc962ffd700 1 -- 172.21.15.181:0/3969437141 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc9680543f0 con 0x7fc96c0af450 2023-10-16T10:02:10.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.978+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc96c1582c0 con 0x7fc96c0af450 2023-10-16T10:02:10.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.979+0000 7fc962ffd700 1 -- 172.21.15.181:0/3969437141 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+17212 (secure 0 0 0) 0x7fc968057a10 con 0x7fc96c0af450 2023-10-16T10:02:10.981 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:10.982 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:02:10.009991+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:08.438030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:08.968951+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:10.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fc95006b020 msgr2=0x7fc95006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fc95006b020 0x7fc95006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000c00 tx=0x7fc95c008070 comp rx=0 tx=0).stop 2023-10-16T10:02:10.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c0af450 msgr2=0x7fc96c0b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:10.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c0af450 0x7fc96c0b52e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc96800e810 tx=0x7fc96800ebd0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 shutdown_connections 2023-10-16T10:02:10.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fc95006b020 0x7fc95006d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc96c06db80 0x7fc96c0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc96c15dfd0 0x7fc96c0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 --2- 172.21.15.181:0/3969437141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc96c0af450 0x7fc96c0b52e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:10.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 >> 172.21.15.181:0/3969437141 conn(0x7fc96c0d04a0 msgr2=0x7fc96c0add00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:10.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 shutdown_connections 2023-10-16T10:02:10.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:10.982+0000 7fc974507700 1 -- 172.21.15.181:0/3969437141 wait complete. 2023-10-16T10:02:10.996 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:11.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.114+0000 7fee4452b700 1 Processor -- start 2023-10-16T10:02:11.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.114+0000 7fee4452b700 1 -- start start 2023-10-16T10:02:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.115+0000 7fee4452b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee3c15e6d0 0x7fee3c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.115+0000 7fee4452b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee3c15eab0 0x7fee3c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.115+0000 7fee4452b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c0cb0c0 0x7fee3c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.115+0000 7fee4452b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c06ed00 con 0x7fee3c15eab0 2023-10-16T10:02:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.115+0000 7fee4452b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c0d02d0 con 0x7fee3c15e6d0 2023-10-16T10:02:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.115+0000 7fee4452b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c06e0f0 con 0x7fee3c0cb0c0 2023-10-16T10:02:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee3c15eab0 0x7fee3c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee3c15eab0 0x7fee3c15a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45978/0 (socket says 172.21.15.181:45978) 2023-10-16T10:02:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 -- 172.21.15.181:0/4167888333 learned_addr learned my addr 172.21.15.181:0/4167888333 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee422c7700 1 --2- 172.21.15.181:0/4167888333 >> v2:172.21.15.190:3300/0 conn(0x7fee3c15e6d0 0x7fee3c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee42ac8700 1 --2- 172.21.15.181:0/4167888333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c0cb0c0 0x7fee3c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 -- 172.21.15.181:0/4167888333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c0cb0c0 msgr2=0x7fee3c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 --2- 172.21.15.181:0/4167888333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c0cb0c0 0x7fee3c15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 -- 172.21.15.181:0/4167888333 >> v2:172.21.15.190:3300/0 conn(0x7fee3c15e6d0 msgr2=0x7fee3c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:11.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 --2- 172.21.15.181:0/4167888333 >> v2:172.21.15.190:3300/0 conn(0x7fee3c15e6d0 0x7fee3c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 -- 172.21.15.181:0/4167888333 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c06e3d0 con 0x7fee3c15eab0 2023-10-16T10:02:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee42ac8700 1 --2- 172.21.15.181:0/4167888333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c0cb0c0 0x7fee3c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee41ac6700 1 --2- 172.21.15.181:0/4167888333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15eab0 0x7fee3c15a0f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0717b0 tx=0x7fee38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450d80e0dedb6fd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee412c5700 1 -- 172.21.15.181:0/4167888333 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800b870 con 0x7fee3c15eab0 2023-10-16T10:02:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee412c5700 1 -- 172.21.15.181:0/4167888333 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3800b9d0 con 0x7fee3c15eab0 2023-10-16T10:02:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.116+0000 7fee412c5700 1 -- 172.21.15.181:0/4167888333 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800bd60 con 0x7fee3c15eab0 2023-10-16T10:02:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 -- 172.21.15.181:0/4167888333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15eab0 msgr2=0x7fee3c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 --2- 172.21.15.181:0/4167888333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15eab0 0x7fee3c15a0f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0717b0 tx=0x7fee38009bd0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 -- 172.21.15.181:0/4167888333 shutdown_connections 2023-10-16T10:02:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 --2- 172.21.15.181:0/4167888333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c0cb0c0 0x7fee3c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 --2- 172.21.15.181:0/4167888333 >> v2:172.21.15.190:3300/0 conn(0x7fee3c15e6d0 0x7fee3c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 --2- 172.21.15.181:0/4167888333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15eab0 0x7fee3c15a0f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 -- 172.21.15.181:0/4167888333 >> 172.21.15.181:0/4167888333 conn(0x7fee3c0d04a0 msgr2=0x7fee3c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 -- 172.21.15.181:0/4167888333 shutdown_connections 2023-10-16T10:02:11.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.117+0000 7fee4452b700 1 -- 172.21.15.181:0/4167888333 wait complete. 2023-10-16T10:02:11.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.118+0000 7fee4452b700 1 Processor -- start 2023-10-16T10:02:11.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.118+0000 7fee4452b700 1 -- start start 2023-10-16T10:02:11.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee3c0cb0c0 0x7fee3c153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15e6d0 0x7fee3c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c15eab0 0x7fee3c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c153c80 con 0x7fee3c15e6d0 2023-10-16T10:02:11.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0b4840 con 0x7fee3c0cb0c0 2023-10-16T10:02:11.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0b4b20 con 0x7fee3c15eab0 2023-10-16T10:02:11.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15e6d0 0x7fee3c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15e6d0 0x7fee3c15ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45982/0 (socket says 172.21.15.181:45982) 2023-10-16T10:02:11.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 -- 172.21.15.181:0/1053438244 learned_addr learned my addr 172.21.15.181:0/1053438244 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:11.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 -- 172.21.15.181:0/1053438244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c15eab0 msgr2=0x7fee3c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c15eab0 0x7fee3c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 -- 172.21.15.181:0/1053438244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee3c0cb0c0 msgr2=0x7fee3c153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee3c0cb0c0 0x7fee3c153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 -- 172.21.15.181:0/1053438244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38009550 con 0x7fee3c15e6d0 2023-10-16T10:02:11.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee41ac6700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15e6d0 0x7fee3c15ab80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fee38000c00 tx=0x7fee38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee32ffd700 1 -- 172.21.15.181:0/1053438244 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38004540 con 0x7fee3c15e6d0 2023-10-16T10:02:11.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee32ffd700 1 -- 172.21.15.181:0/1053438244 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380036a0 con 0x7fee3c15e6d0 2023-10-16T10:02:11.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee32ffd700 1 -- 172.21.15.181:0/1053438244 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38003a50 con 0x7fee3c15e6d0 2023-10-16T10:02:11.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c0b4e00 con 0x7fee3c15e6d0 2023-10-16T10:02:11.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.119+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c0b53c0 con 0x7fee3c15e6d0 2023-10-16T10:02:11.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.120+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c067d20 con 0x7fee3c15e6d0 2023-10-16T10:02:11.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.121+0000 7fee32ffd700 1 -- 172.21.15.181:0/1053438244 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fee38021070 con 0x7fee3c15e6d0 2023-10-16T10:02:11.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.121+0000 7fee32ffd700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fee2006b020 0x7fee2006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.121+0000 7fee32ffd700 1 -- 172.21.15.181:0/1053438244 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fee3808afc0 con 0x7fee3c15e6d0 2023-10-16T10:02:11.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.121+0000 7fee422c7700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fee2006b020 0x7fee2006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.122+0000 7fee422c7700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fee2006b020 0x7fee2006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006f30 tx=0x7fee2c008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.126+0000 7fee32ffd700 1 -- 172.21.15.181:0/1053438244 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fee380bdfe0 con 0x7fee3c15e6d0 2023-10-16T10:02:11.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.316+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee3c0af320 con 0x7fee3c15e6d0 2023-10-16T10:02:11.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.317+0000 7fee32ffd700 1 -- 172.21.15.181:0/1053438244 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+17212 (secure 0 0 0) 0x7fee38029090 con 0x7fee3c15e6d0 2023-10-16T10:02:11.319 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:11.319 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:02:10.009991+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:08.438030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:08.968951+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:11.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fee2006b020 msgr2=0x7fee2006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fee2006b020 0x7fee2006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006f30 tx=0x7fee2c008040 comp rx=0 tx=0).stop 2023-10-16T10:02:11.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15e6d0 msgr2=0x7fee3c15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15e6d0 0x7fee3c15ab80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fee38000c00 tx=0x7fee38004750 comp rx=0 tx=0).stop 2023-10-16T10:02:11.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 shutdown_connections 2023-10-16T10:02:11.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fee2006b020 0x7fee2006d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee3c15eab0 0x7fee3c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.319+0000 7fee4452b700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee3c0cb0c0 0x7fee3c153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.320+0000 7fee4452b700 1 --2- 172.21.15.181:0/1053438244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee3c15e6d0 0x7fee3c15ab80 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.320+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 >> 172.21.15.181:0/1053438244 conn(0x7fee3c0d04a0 msgr2=0x7fee3c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:11.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.320+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 shutdown_connections 2023-10-16T10:02:11.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.320+0000 7fee4452b700 1 -- 172.21.15.181:0/1053438244 wait complete. 2023-10-16T10:02:11.334 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:02:11.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.456+0000 7fb992e09700 1 Processor -- start 2023-10-16T10:02:11.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.456+0000 7fb992e09700 1 -- start start 2023-10-16T10:02:11.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb992e09700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb98c15dbd0 0x7fb98c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb992e09700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb98c0ce490 0x7fb98c06b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb992e09700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c15dfb0 con 0x7fb98c0d0610 2023-10-16T10:02:11.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb992e09700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c0cd3a0 con 0x7fb98c0ce490 2023-10-16T10:02:11.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb992e09700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c0cd830 con 0x7fb98c15dbd0 2023-10-16T10:02:11.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb9913a6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb98c0ce490 0x7fb98c06b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb98bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb98c0d0610 0x7fb98c0cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57068/0 (socket says 172.21.15.181:57068) 2023-10-16T10:02:11.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.457+0000 7fb98bfff700 1 -- 172.21.15.181:0/723705423 learned_addr learned my addr 172.21.15.181:0/723705423 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:11.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb990ba5700 1 --2- 172.21.15.181:0/723705423 >> v2:172.21.15.190:3300/0 conn(0x7fb98c15dbd0 0x7fb98c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb9913a6700 1 -- 172.21.15.181:0/723705423 >> v2:172.21.15.190:3300/0 conn(0x7fb98c15dbd0 msgr2=0x7fb98c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:11.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb9913a6700 1 --2- 172.21.15.181:0/723705423 >> v2:172.21.15.190:3300/0 conn(0x7fb98c15dbd0 0x7fb98c0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb9913a6700 1 -- 172.21.15.181:0/723705423 >> v1:172.21.15.181:6789/0 conn(0x7fb98c0d0610 legacy=0x7fb98c0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb9913a6700 1 -- 172.21.15.181:0/723705423 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0d1fc0 con 0x7fb98c0ce490 2023-10-16T10:02:11.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb98b7fe700 1 -- 172.21.15.181:0/723705423 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227874256 0 0) 0x7fb978001110 con 0x7fb98c0d0610 2023-10-16T10:02:11.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb9913a6700 1 --2- 172.21.15.181:0/723705423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0ce490 0x7fb98c06b6f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0729b0 tx=0x7fb98400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc6be094eb33209 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb98b7fe700 1 -- 172.21.15.181:0/723705423 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400e070 con 0x7fb98c0ce490 2023-10-16T10:02:11.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb98b7fe700 1 -- 172.21.15.181:0/723705423 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98400bbd0 con 0x7fb98c0ce490 2023-10-16T10:02:11.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.458+0000 7fb98b7fe700 1 -- 172.21.15.181:0/723705423 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984012430 con 0x7fb98c0ce490 2023-10-16T10:02:11.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 -- 172.21.15.181:0/723705423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0ce490 msgr2=0x7fb98c06b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 --2- 172.21.15.181:0/723705423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0ce490 0x7fb98c06b6f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0729b0 tx=0x7fb98400b330 comp rx=0 tx=0).stop 2023-10-16T10:02:11.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 -- 172.21.15.181:0/723705423 shutdown_connections 2023-10-16T10:02:11.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 --2- 172.21.15.181:0/723705423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0ce490 0x7fb98c06b6f0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 --2- 172.21.15.181:0/723705423 >> v2:172.21.15.190:3300/0 conn(0x7fb98c15dbd0 0x7fb98c0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 -- 172.21.15.181:0/723705423 >> 172.21.15.181:0/723705423 conn(0x7fb98c09a930 msgr2=0x7fb98c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 -- 172.21.15.181:0/723705423 shutdown_connections 2023-10-16T10:02:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.459+0000 7fb992e09700 1 -- 172.21.15.181:0/723705423 wait complete. 2023-10-16T10:02:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.460+0000 7fb992e09700 1 Processor -- start 2023-10-16T10:02:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.460+0000 7fb992e09700 1 -- start start 2023-10-16T10:02:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb992e09700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb98c0ce490 0x7fb98c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb992e09700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0d0610 0x7fb98c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb992e09700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb98c15dbd0 0x7fb98c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb992e09700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c15b170 con 0x7fb98c0d0610 2023-10-16T10:02:11.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb992e09700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c0b42c0 con 0x7fb98c0ce490 2023-10-16T10:02:11.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb992e09700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c161130 con 0x7fb98c15dbd0 2023-10-16T10:02:11.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0d0610 0x7fb98c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0d0610 0x7fb98c160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:45996/0 (socket says 172.21.15.181:45996) 2023-10-16T10:02:11.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 -- 172.21.15.181:0/1641009491 learned_addr learned my addr 172.21.15.181:0/1641009491 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:11.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb990ba5700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb98c0ce490 0x7fb98c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb9913a6700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb98c15dbd0 0x7fb98c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 -- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb98c15dbd0 msgr2=0x7fb98c0b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb98c15dbd0 0x7fb98c0b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 -- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb98c0ce490 msgr2=0x7fb98c15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb98c0ce490 0x7fb98c15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 -- 172.21.15.181:0/1641009491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb984009a70 con 0x7fb98c0d0610 2023-10-16T10:02:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb990ba5700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb98c0ce490 0x7fb98c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb98bfff700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0d0610 0x7fb98c160bf0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c550 tx=0x7fb97800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb9897fa700 1 -- 172.21.15.181:0/1641009491 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978004c80 con 0x7fb98c0d0610 2023-10-16T10:02:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb9897fa700 1 -- 172.21.15.181:0/1641009491 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb978004e00 con 0x7fb98c0d0610 2023-10-16T10:02:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c15f120 con 0x7fb98c0d0610 2023-10-16T10:02:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.461+0000 7fb9897fa700 1 -- 172.21.15.181:0/1641009491 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978007310 con 0x7fb98c0d0610 2023-10-16T10:02:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.462+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c15f6e0 con 0x7fb98c0d0610 2023-10-16T10:02:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.463+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c0b4450 con 0x7fb98c0d0610 2023-10-16T10:02:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.463+0000 7fb9897fa700 1 -- 172.21.15.181:0/1641009491 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb978007470 con 0x7fb98c0d0610 2023-10-16T10:02:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.463+0000 7fb9897fa700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb97006b020 0x7fb97006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.464+0000 7fb990ba5700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb97006b020 0x7fb97006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.464+0000 7fb9897fa700 1 -- 172.21.15.181:0/1641009491 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fb978089590 con 0x7fb98c0d0610 2023-10-16T10:02:11.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.464+0000 7fb990ba5700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb97006b020 0x7fb97006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0cdee0 tx=0x7fb980008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:11.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.468+0000 7fb9897fa700 1 -- 172.21.15.181:0/1641009491 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb978054380 con 0x7fb98c0d0610 2023-10-16T10:02:11.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:11.713+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb98c15dfb0 con 0x7fb98c0d0610 2023-10-16T10:02:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.636+0000 7fb9897fa700 1 -- 172.21.15.181:0/1641009491 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb98c15dfb0 con 0x7fb98c0d0610 2023-10-16T10:02:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb97006b020 msgr2=0x7fb97006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb97006b020 0x7fb97006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0cdee0 tx=0x7fb980008040 comp rx=0 tx=0).stop 2023-10-16T10:02:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0d0610 msgr2=0x7fb98c160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0d0610 0x7fb98c160bf0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c550 tx=0x7fb97800c910 comp rx=0 tx=0).stop 2023-10-16T10:02:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 shutdown_connections 2023-10-16T10:02:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fb97006b020 0x7fb97006d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb98c15dbd0 0x7fb98c0b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb98c0ce490 0x7fb98c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 --2- 172.21.15.181:0/1641009491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb98c0d0610 0x7fb98c160bf0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 >> 172.21.15.181:0/1641009491 conn(0x7fb98c09a930 msgr2=0x7fb98c15b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 shutdown_connections 2023-10-16T10:02:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.639+0000 7fb992e09700 1 -- 172.21.15.181:0/1641009491 wait complete. 2023-10-16T10:02:12.656 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:02:12.656 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:02:12.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.785+0000 7fd00cda6700 1 Processor -- start 2023-10-16T10:02:12.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.785+0000 7fd00cda6700 1 -- start start 2023-10-16T10:02:12.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00cda6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd0080cdd00 0x7fd00806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00cda6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0080666a0 0x7fd00806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00cda6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0080ca300 con 0x7fd0080ce0e0 2023-10-16T10:02:12.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00cda6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00806e5e0 con 0x7fd0080666a0 2023-10-16T10:02:12.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00cda6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0080ca580 con 0x7fd0080cdd00 2023-10-16T10:02:12.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0080666a0 0x7fd00806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0080666a0 0x7fd00806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46002/0 (socket says 172.21.15.181:46002) 2023-10-16T10:02:12.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 -- 172.21.15.181:0/4129710453 learned_addr learned my addr 172.21.15.181:0/4129710453 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 -- 172.21.15.181:0/4129710453 >> v2:172.21.15.190:3300/0 conn(0x7fd0080cdd00 msgr2=0x7fd00806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 --2- 172.21.15.181:0/4129710453 >> v2:172.21.15.190:3300/0 conn(0x7fd0080cdd00 0x7fd00806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 -- 172.21.15.181:0/4129710453 >> v1:172.21.15.190:6789/0 conn(0x7fd0080ce0e0 legacy=0x7fd00806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 -- 172.21.15.181:0/4129710453 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080ca860 con 0x7fd0080666a0 2023-10-16T10:02:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd005d9b700 1 --2- 172.21.15.181:0/4129710453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd00806a8a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fd008070b50 tx=0x7fcff8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87b4efc6d12935 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00559a700 1 -- 172.21.15.181:0/4129710453 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8004230 con 0x7fd0080666a0 2023-10-16T10:02:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00559a700 1 -- 172.21.15.181:0/4129710453 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8004390 con 0x7fd0080666a0 2023-10-16T10:02:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.786+0000 7fd00559a700 1 -- 172.21.15.181:0/4129710453 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80036a0 con 0x7fd0080666a0 2023-10-16T10:02:12.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 -- 172.21.15.181:0/4129710453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 msgr2=0x7fd00806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 --2- 172.21.15.181:0/4129710453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd00806a8a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fd008070b50 tx=0x7fcff8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 -- 172.21.15.181:0/4129710453 shutdown_connections 2023-10-16T10:02:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 --2- 172.21.15.181:0/4129710453 >> v2:172.21.15.190:3300/0 conn(0x7fd0080cdd00 0x7fd00806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 --2- 172.21.15.181:0/4129710453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd00806a8a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 -- 172.21.15.181:0/4129710453 >> 172.21.15.181:0/4129710453 conn(0x7fd0080cc730 msgr2=0x7fd0080ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 -- 172.21.15.181:0/4129710453 shutdown_connections 2023-10-16T10:02:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.787+0000 7fd00cda6700 1 -- 172.21.15.181:0/4129710453 wait complete. 2023-10-16T10:02:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.788+0000 7fd00cda6700 1 Processor -- start 2023-10-16T10:02:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.788+0000 7fd00cda6700 1 -- start start 2023-10-16T10:02:12.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00cda6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd008159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00cda6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0080cdd00 0x7fd0081f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00cda6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0080ce0e0 0x7fd0080cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00cda6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0081f3f30 con 0x7fd0080666a0 2023-10-16T10:02:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00cda6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00815ee60 con 0x7fd0080ce0e0 2023-10-16T10:02:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00cda6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080c3ae0 con 0x7fd0080cdd00 2023-10-16T10:02:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd008159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd008159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46006/0 (socket says 172.21.15.181:46006) 2023-10-16T10:02:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 -- 172.21.15.181:0/2313230286 learned_addr learned my addr 172.21.15.181:0/2313230286 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd006d9d700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0080ce0e0 0x7fd0080cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd005d9b700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0080cdd00 0x7fd0081f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 -- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0080cdd00 msgr2=0x7fd0081f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0080cdd00 0x7fd0081f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 -- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0080ce0e0 msgr2=0x7fd0080cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0080ce0e0 0x7fd0080cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.789+0000 7fd00659c700 1 -- 172.21.15.181:0/2313230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff8009550 con 0x7fd0080666a0 2023-10-16T10:02:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.790+0000 7fd00659c700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd008159270 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00b530 tx=0x7fcffc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.790+0000 7fcff6ffd700 1 -- 172.21.15.181:0/2313230286 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc011830 con 0x7fd0080666a0 2023-10-16T10:02:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.790+0000 7fcff6ffd700 1 -- 172.21.15.181:0/2313230286 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc0119b0 con 0x7fd0080666a0 2023-10-16T10:02:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.790+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0080c3dc0 con 0x7fd0080666a0 2023-10-16T10:02:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.790+0000 7fcff6ffd700 1 -- 172.21.15.181:0/2313230286 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc011d60 con 0x7fd0080666a0 2023-10-16T10:02:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.790+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0080c4380 con 0x7fd0080666a0 2023-10-16T10:02:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.791+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0080c45a0 con 0x7fd0080666a0 2023-10-16T10:02:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.791+0000 7fcff6ffd700 1 -- 172.21.15.181:0/2313230286 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcffc01a400 con 0x7fd0080666a0 2023-10-16T10:02:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.791+0000 7fcff6ffd700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fcfe806b020 0x7fcfe806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.791+0000 7fcff6ffd700 1 -- 172.21.15.181:0/2313230286 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fcffc089940 con 0x7fd0080666a0 2023-10-16T10:02:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.792+0000 7fd005d9b700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fcfe806b020 0x7fcfe806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:12.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.792+0000 7fd005d9b700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fcfe806b020 0x7fcfe806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd008070b50 tx=0x7fcff8008030 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:12.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:12.796+0000 7fcff6ffd700 1 -- 172.21.15.181:0/2313230286 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcffc0547c0 con 0x7fd0080666a0 2023-10-16T10:02:13.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.034+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd008072090 con 0x7fd0080666a0 2023-10-16T10:02:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.637+0000 7fcff6ffd700 1 -- 172.21.15.181:0/2313230286 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcffc015020 con 0x7fd0080666a0 2023-10-16T10:02:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fcfe806b020 msgr2=0x7fcfe806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fcfe806b020 0x7fcfe806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd008070b50 tx=0x7fcff8008030 comp rx=0 tx=0).stop 2023-10-16T10:02:13.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 msgr2=0x7fd008159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd008159270 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00b530 tx=0x7fcffc00b8f0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 shutdown_connections 2023-10-16T10:02:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fcfe806b020 0x7fcfe806d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0080ce0e0 0x7fd0080cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0080666a0 0x7fd008159270 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 --2- 172.21.15.181:0/2313230286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0080cdd00 0x7fd0081f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 >> 172.21.15.181:0/2313230286 conn(0x7fd0080cc730 msgr2=0x7fd00806f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 shutdown_connections 2023-10-16T10:02:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.640+0000 7fd00cda6700 1 -- 172.21.15.181:0/2313230286 wait complete. 2023-10-16T10:02:13.655 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:02:13.656 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:02:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.777+0000 7f44c8009700 1 Processor -- start 2023-10-16T10:02:13.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.778+0000 7f44c8009700 1 -- start start 2023-10-16T10:02:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.778+0000 7f44c8009700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c015e6d0 0x7f44c00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.778+0000 7f44c8009700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44c00cb0c0 0x7f44c015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.778+0000 7f44c8009700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c006ed00 con 0x7f44c015eab0 2023-10-16T10:02:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.778+0000 7f44c8009700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00d02d0 con 0x7f44c00cb0c0 2023-10-16T10:02:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.778+0000 7f44c8009700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c015ad40 con 0x7f44c015e6d0 2023-10-16T10:02:13.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c55a4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f44c015eab0 0x7f44c015a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57092/0 (socket says 172.21.15.181:57092) 2023-10-16T10:02:13.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c55a4700 1 -- 172.21.15.181:0/3043135508 learned_addr learned my addr 172.21.15.181:0/3043135508 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:13.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c65a6700 1 --2- 172.21.15.181:0/3043135508 >> v2:172.21.15.190:3300/0 conn(0x7f44c00cb0c0 0x7f44c015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c5da5700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c015e6d0 0x7f44c00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276644366 0 0) 0x7f44c006ed00 con 0x7f44c015eab0 2023-10-16T10:02:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44ac0035b0 con 0x7f44c015eab0 2023-10-16T10:02:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2169124328 0 0) 0x7f44ac0035b0 con 0x7f44c015eab0 2023-10-16T10:02:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c015e6d0 msgr2=0x7f44c00d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:13.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c015e6d0 0x7f44c00d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 >> v2:172.21.15.190:3300/0 conn(0x7f44c00cb0c0 msgr2=0x7f44c015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 --2- 172.21.15.181:0/3043135508 >> v2:172.21.15.190:3300/0 conn(0x7f44c00cb0c0 0x7f44c015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c006e0f0 con 0x7f44c015eab0 2023-10-16T10:02:13.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f44bc001670 con 0x7f44c015eab0 2023-10-16T10:02:13.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 >> v1:172.21.15.181:6789/0 conn(0x7f44c015eab0 legacy=0x7f44c015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac003a00 0x7f44ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac004850 0x7f44ac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac005210 0x7f44ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00d02d0 con 0x7f44ac005210 2023-10-16T10:02:13.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c015ad40 con 0x7f44ac003a00 2023-10-16T10:02:13.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac00bf50 con 0x7f44ac004850 2023-10-16T10:02:13.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c65a6700 1 --2- 172.21.15.181:0/3043135508 >> v2:172.21.15.190:3300/0 conn(0x7f44c00cb0c0 0x7f44c015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:02:13.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c5da5700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c015e6d0 0x7f44c00d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:02:13.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c65a6700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac005210 0x7f44ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c55a4700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac004850 0x7f44ac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.779+0000 7f44c5da5700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac003a00 0x7f44ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c65a6700 1 -- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac004850 msgr2=0x7f44ac004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:13.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c65a6700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac004850 0x7f44ac004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c65a6700 1 -- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac003a00 msgr2=0x7f44ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:13.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c65a6700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac003a00 0x7f44ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c65a6700 1 -- 172.21.15.181:0/3043135508 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac0035b0 con 0x7f44ac005210 2023-10-16T10:02:13.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c65a6700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac005210 0x7f44ac00ba10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f44b800e380 tx=0x7f44ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56253f03f604a21e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:13.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800ee40 con 0x7f44ac005210 2023-10-16T10:02:13.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b800f040 con 0x7f44ac005210 2023-10-16T10:02:13.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c4da3700 1 -- 172.21.15.181:0/3043135508 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8003c40 con 0x7f44ac005210 2023-10-16T10:02:13.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c8009700 1 -- 172.21.15.181:0/3043135508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac005210 msgr2=0x7f44ac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c8009700 1 --2- 172.21.15.181:0/3043135508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac005210 0x7f44ac00ba10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f44b800e380 tx=0x7f44ac005880 comp rx=0 tx=0).stop 2023-10-16T10:02:13.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c65a6700 1 -- 172.21.15.181:0/3043135508 reap_dead start 2023-10-16T10:02:13.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c8009700 1 -- 172.21.15.181:0/3043135508 shutdown_connections 2023-10-16T10:02:13.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.780+0000 7f44c8009700 1 -- 172.21.15.181:0/3043135508 >> 172.21.15.181:0/3043135508 conn(0x7f44c00d04a0 msgr2=0x7f44c00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:13.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.781+0000 7f44c8009700 1 -- 172.21.15.181:0/3043135508 shutdown_connections 2023-10-16T10:02:13.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.781+0000 7f44c8009700 1 -- 172.21.15.181:0/3043135508 wait complete. 2023-10-16T10:02:13.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 Processor -- start 2023-10-16T10:02:13.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 -- start start 2023-10-16T10:02:13.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac005210 0x7f44c00ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c01598b0 0x7f44c014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c00ae9c0 0x7f44c00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00ad650 con 0x7f44c01598b0 2023-10-16T10:02:13.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00ad920 con 0x7f44ac005210 2023-10-16T10:02:13.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.782+0000 7f44c8009700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00adc00 con 0x7f44c00ae9c0 2023-10-16T10:02:13.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c01598b0 0x7f44c014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c01598b0 0x7f44c014e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46026/0 (socket says 172.21.15.181:46026) 2023-10-16T10:02:13.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 -- 172.21.15.181:0/1734304743 learned_addr learned my addr 172.21.15.181:0/1734304743 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:13.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c65a6700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c00ae9c0 0x7f44c00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 -- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c00ae9c0 msgr2=0x7f44c00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c00ae9c0 0x7f44c00c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c5da5700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac005210 0x7f44c00ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 -- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac005210 msgr2=0x7f44c00ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:13.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac005210 0x7f44c00ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:13.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 -- 172.21.15.181:0/1734304743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00adee0 con 0x7f44c01598b0 2023-10-16T10:02:13.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c55a4700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c01598b0 0x7f44c014e7f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0017a0 tx=0x7f44bc0076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:13.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44b67fc700 1 -- 172.21.15.181:0/1734304743 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc007ca0 con 0x7f44c01598b0 2023-10-16T10:02:13.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44b67fc700 1 -- 172.21.15.181:0/1734304743 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc00f8d0 con 0x7f44c01598b0 2023-10-16T10:02:13.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c00ae1c0 con 0x7f44c01598b0 2023-10-16T10:02:13.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44b67fc700 1 -- 172.21.15.181:0/1734304743 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00fc80 con 0x7f44c01598b0 2023-10-16T10:02:13.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.783+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c014ed30 con 0x7f44c01598b0 2023-10-16T10:02:13.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.784+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c00ae6e0 con 0x7f44c01598b0 2023-10-16T10:02:13.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.784+0000 7f44b67fc700 1 -- 172.21.15.181:0/1734304743 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f44bc01d460 con 0x7f44c01598b0 2023-10-16T10:02:13.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.785+0000 7f44b67fc700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f44a406b020 0x7f44a406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:13.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.785+0000 7f44c5da5700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f44a406b020 0x7f44a406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:13.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.785+0000 7f44b67fc700 1 -- 172.21.15.181:0/1734304743 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f44bc089a60 con 0x7f44c01598b0 2023-10-16T10:02:13.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.785+0000 7f44c5da5700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f44a406b020 0x7f44a406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44c00717b0 tx=0x7f44b0008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:13.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:13.790+0000 7f44b67fc700 1 -- 172.21.15.181:0/1734304743 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f44c00ae6e0 con 0x7f44c01598b0 2023-10-16T10:02:14.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.025+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44c00ae6e0 con 0x7f44c01598b0 2023-10-16T10:02:14.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.025+0000 7f44b67fc700 1 -- 172.21.15.181:0/1734304743 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f44c00ae6e0 con 0x7f44c01598b0 2023-10-16T10:02:14.027 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:14.028 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2945191097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2945191097},{"type":"v1","addr":"172.21.15.190:6834","nonce":2945191097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1168488779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1168488779},{"type":"v1","addr":"172.21.15.190:6835","nonce":1168488779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2677039975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2677039975},{"type":"v1","addr":"172.21.15.181:6836","nonce":2677039975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:10.010346+0000","modified":"2023-10-16T10:02:13.035963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1841075189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1841075189},{"type":"v1","addr":"172.21.15.181:6837","nonce":1841075189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:02:14.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f44a406b020 msgr2=0x7f44a406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f44a406b020 0x7f44a406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44c00717b0 tx=0x7f44b0008040 comp rx=0 tx=0).stop 2023-10-16T10:02:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c01598b0 msgr2=0x7f44c014e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c01598b0 0x7f44c014e7f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0017a0 tx=0x7f44bc0076a0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 shutdown_connections 2023-10-16T10:02:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f44a406b020 0x7f44a406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44c00ae9c0 0x7f44c00c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac005210 0x7f44c00ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 --2- 172.21.15.181:0/1734304743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c01598b0 0x7f44c014e7f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.027+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 >> 172.21.15.181:0/1734304743 conn(0x7f44c00d04a0 msgr2=0x7f44c015cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.028+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 shutdown_connections 2023-10-16T10:02:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.028+0000 7f44c8009700 1 -- 172.21.15.181:0/1734304743 wait complete. 2023-10-16T10:02:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 58 2023-10-16T10:02:14.041 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:02:14.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.160+0000 7fac8cb5b700 1 Processor -- start 2023-10-16T10:02:14.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.160+0000 7fac8cb5b700 1 -- start start 2023-10-16T10:02:14.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.161+0000 7fac8cb5b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac880d2390 0x7fac880cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.161+0000 7fac8cb5b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac880cc470 con 0x7fac880d0180 2023-10-16T10:02:14.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.161+0000 7fac8cb5b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8806ca30 con 0x7fac880ce7f0 2023-10-16T10:02:14.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.161+0000 7fac8cb5b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8806c810 con 0x7fac880d2390 2023-10-16T10:02:14.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac86d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fac880d0180 0x7fac880cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57116/0 (socket says 172.21.15.181:57116) 2023-10-16T10:02:14.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac86d9d700 1 -- 172.21.15.181:0/2693662262 learned_addr learned my addr 172.21.15.181:0/2693662262 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:14.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac85d9b700 1 --2- 172.21.15.181:0/2693662262 >> v2:172.21.15.190:3300/0 conn(0x7fac880d2390 0x7fac880cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279734920 0 0) 0x7fac880cc470 con 0x7fac880d0180 2023-10-16T10:02:14.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac740035b0 con 0x7fac880d0180 2023-10-16T10:02:14.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187056263 0 0) 0x7fac740035b0 con 0x7fac880d0180 2023-10-16T10:02:14.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 >> v2:172.21.15.190:3300/0 conn(0x7fac880d2390 msgr2=0x7fac880cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 --2- 172.21.15.181:0/2693662262 >> v2:172.21.15.190:3300/0 conn(0x7fac880d2390 0x7fac880cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 >> v1:172.21.15.190:6789/0 conn(0x7fac880ce7f0 legacy=0x7fac8806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8806cd10 con 0x7fac880d0180 2023-10-16T10:02:14.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fac800019f0 con 0x7fac880d0180 2023-10-16T10:02:14.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 >> v1:172.21.15.181:6789/0 conn(0x7fac880d0180 legacy=0x7fac880cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003a00 0x7fac74003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004400 0x7fac740048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac7400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8806c810 con 0x7fac740050d0 2023-10-16T10:02:14.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7400c030 con 0x7fac74003a00 2023-10-16T10:02:14.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7400c1f0 con 0x7fac74004400 2023-10-16T10:02:14.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac86d9d700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac7400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac85d9b700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004400 0x7fac740048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.162+0000 7fac8659c700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003a00 0x7fac74003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac86d9d700 1 -- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004400 msgr2=0x7fac740048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac86d9d700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac74004400 0x7fac740048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac86d9d700 1 -- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003a00 msgr2=0x7fac74003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac86d9d700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac74003a00 0x7fac74003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac86d9d700 1 -- 172.21.15.181:0/2693662262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740035b0 con 0x7fac740050d0 2023-10-16T10:02:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac86d9d700 1 -- 172.21.15.181:0/2693662262 reap_dead start 2023-10-16T10:02:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac86d9d700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac7400ba10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fac8000a830 tx=0x7fac74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c6275f9b6ff85d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80012600 con 0x7fac740050d0 2023-10-16T10:02:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac80012780 con 0x7fac740050d0 2023-10-16T10:02:14.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8559a700 1 -- 172.21.15.181:0/2693662262 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80012b30 con 0x7fac740050d0 2023-10-16T10:02:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8cb5b700 1 -- 172.21.15.181:0/2693662262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 msgr2=0x7fac7400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac7400ba10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fac8000a830 tx=0x7fac74005880 comp rx=0 tx=0).stop 2023-10-16T10:02:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8cb5b700 1 -- 172.21.15.181:0/2693662262 shutdown_connections 2023-10-16T10:02:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/2693662262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac7400ba10 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.163+0000 7fac8cb5b700 1 -- 172.21.15.181:0/2693662262 >> 172.21.15.181:0/2693662262 conn(0x7fac8809a9a0 msgr2=0x7fac880d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.164+0000 7fac8cb5b700 1 -- 172.21.15.181:0/2693662262 shutdown_connections 2023-10-16T10:02:14.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.164+0000 7fac8cb5b700 1 -- 172.21.15.181:0/2693662262 wait complete. 2023-10-16T10:02:14.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.165+0000 7fac8cb5b700 1 Processor -- start 2023-10-16T10:02:14.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.165+0000 7fac8cb5b700 1 -- start start 2023-10-16T10:02:14.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac881f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac88159900 0x7fac881579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac880b62c0 0x7fac880b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac880b6c90 con 0x7fac740050d0 2023-10-16T10:02:14.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8815a990 con 0x7fac88159900 2023-10-16T10:02:14.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac881f3e40 con 0x7fac880b62c0 2023-10-16T10:02:14.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac881f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac881f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46046/0 (socket says 172.21.15.181:46046) 2023-10-16T10:02:14.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 -- 172.21.15.181:0/62346122 learned_addr learned my addr 172.21.15.181:0/62346122 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:14.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac86d9d700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac880b62c0 0x7fac880b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac85d9b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac88159900 0x7fac881579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 -- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac880b62c0 msgr2=0x7fac880b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac880b62c0 0x7fac880b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 -- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac88159900 msgr2=0x7fac881579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac88159900 0x7fac881579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 -- 172.21.15.181:0/62346122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88159d80 con 0x7fac740050d0 2023-10-16T10:02:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac86d9d700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac880b62c0 0x7fac880b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac85d9b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac88159900 0x7fac881579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8659c700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac881f3900 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fac7000f7a0 tx=0x7fac7000fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac7effd700 1 -- 172.21.15.181:0/62346122 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700052e0 con 0x7fac740050d0 2023-10-16T10:02:14.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac7effd700 1 -- 172.21.15.181:0/62346122 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac70005440 con 0x7fac740050d0 2023-10-16T10:02:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8815a060 con 0x7fac740050d0 2023-10-16T10:02:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac7effd700 1 -- 172.21.15.181:0/62346122 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700055b0 con 0x7fac740050d0 2023-10-16T10:02:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.166+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8815a620 con 0x7fac740050d0 2023-10-16T10:02:14.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.167+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac88067d20 con 0x7fac740050d0 2023-10-16T10:02:14.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.168+0000 7fac7effd700 1 -- 172.21.15.181:0/62346122 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fac70003910 con 0x7fac740050d0 2023-10-16T10:02:14.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.168+0000 7fac7effd700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac6806b020 0x7fac6806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.168+0000 7fac7effd700 1 -- 172.21.15.181:0/62346122 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fac7008a270 con 0x7fac740050d0 2023-10-16T10:02:14.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.168+0000 7fac85d9b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac6806b020 0x7fac6806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.169+0000 7fac85d9b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac6806b020 0x7fac6806d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fac880c5bb0 tx=0x7fac78008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.172+0000 7fac7effd700 1 -- 172.21.15.181:0/62346122 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fac70055140 con 0x7fac740050d0 2023-10-16T10:02:14.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.365+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac880cef40 con 0x7fac740050d0 2023-10-16T10:02:14.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.366+0000 7fac7effd700 1 -- 172.21.15.181:0/62346122 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+17199 (secure 0 0 0) 0x7fac7000dea0 con 0x7fac740050d0 2023-10-16T10:02:14.368 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:14.369 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":118,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:02:11.609218+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:08.438030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:08.968951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:02:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.368+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac6806b020 msgr2=0x7fac6806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac6806b020 0x7fac6806d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fac880c5bb0 tx=0x7fac78008040 comp rx=0 tx=0).stop 2023-10-16T10:02:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 msgr2=0x7fac881f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac881f3900 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fac7000f7a0 tx=0x7fac7000fb60 comp rx=0 tx=0).stop 2023-10-16T10:02:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 shutdown_connections 2023-10-16T10:02:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fac6806b020 0x7fac6806d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac880b62c0 0x7fac880b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac740050d0 0x7fac881f3900 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 --2- 172.21.15.181:0/62346122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac88159900 0x7fac881579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 >> 172.21.15.181:0/62346122 conn(0x7fac8809a9a0 msgr2=0x7fac880cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 shutdown_connections 2023-10-16T10:02:14.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.369+0000 7fac8cb5b700 1 -- 172.21.15.181:0/62346122 wait complete. 2023-10-16T10:02:14.383 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:02:14.383 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:02:14.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.502+0000 7f54e3dd5700 1 Processor -- start 2023-10-16T10:02:14.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.502+0000 7f54e3dd5700 1 -- start start 2023-10-16T10:02:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e3dd5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f54dc0cab40 0x7f54dc164e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e3dd5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc071df0 con 0x7f54dc15e0e0 2023-10-16T10:02:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e3dd5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0720d0 con 0x7f54dc15dd00 2023-10-16T10:02:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e3dd5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc0723b0 con 0x7f54dc0cab40 2023-10-16T10:02:14.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e1370700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f54dc15e0e0 0x7f54dc1574f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57134/0 (socket says 172.21.15.181:57134) 2023-10-16T10:02:14.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e1370700 1 -- 172.21.15.181:0/1888572728 learned_addr learned my addr 172.21.15.181:0/1888572728 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:14.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e2372700 1 --2- 172.21.15.181:0/1888572728 >> v2:172.21.15.190:3300/0 conn(0x7f54dc0cab40 0x7f54dc164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986624974 0 0) 0x7f54dc071df0 con 0x7f54dc15e0e0 2023-10-16T10:02:14.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c80035b0 con 0x7f54dc15e0e0 2023-10-16T10:02:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1511445080 0 0) 0x7f54c80035b0 con 0x7f54dc15e0e0 2023-10-16T10:02:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 >> v2:172.21.15.190:3300/0 conn(0x7f54dc0cab40 msgr2=0x7f54dc164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 --2- 172.21.15.181:0/1888572728 >> v2:172.21.15.190:3300/0 conn(0x7f54dc0cab40 0x7f54dc164e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 >> v1:172.21.15.190:6789/0 conn(0x7f54dc15dd00 legacy=0x7f54dc071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc072690 con 0x7f54dc15e0e0 2023-10-16T10:02:14.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f54cc001670 con 0x7f54dc15e0e0 2023-10-16T10:02:14.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 >> v1:172.21.15.181:6789/0 conn(0x7f54dc15e0e0 legacy=0x7f54dc1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8003a00 0x7f54c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c8004850 0x7f54c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54c8005210 0x7f54c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0723b0 con 0x7f54c8004850 2023-10-16T10:02:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c800c0b0 con 0x7f54c8005210 2023-10-16T10:02:14.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c800c270 con 0x7f54c8003a00 2023-10-16T10:02:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.503+0000 7f54e1370700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c8004850 0x7f54c8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e2372700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54c8005210 0x7f54c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1b71700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8003a00 0x7f54c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1370700 1 -- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8003a00 msgr2=0x7f54c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1370700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8003a00 0x7f54c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1370700 1 -- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54c8005210 msgr2=0x7f54c800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1370700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54c8005210 0x7f54c800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e2372700 1 -- 172.21.15.181:0/1888572728 reap_dead start 2023-10-16T10:02:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1370700 1 -- 172.21.15.181:0/1888572728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80035b0 con 0x7f54c8004850 2023-10-16T10:02:14.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1b71700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8003a00 0x7f54c8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e1370700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c8004850 0x7f54c8004cd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c240 tx=0x7f54c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de8f9acdd1c1357 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc007c10 con 0x7f54c8004850 2023-10-16T10:02:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc00cd80 con 0x7f54c8004850 2023-10-16T10:02:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.504+0000 7f54e0b6f700 1 -- 172.21.15.181:0/1888572728 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc005ba0 con 0x7f54c8004850 2023-10-16T10:02:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.505+0000 7f54e3dd5700 1 -- 172.21.15.181:0/1888572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c8004850 msgr2=0x7f54c8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.505+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c8004850 0x7f54c8004cd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c240 tx=0x7f54c8005880 comp rx=0 tx=0).stop 2023-10-16T10:02:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.505+0000 7f54e3dd5700 1 -- 172.21.15.181:0/1888572728 shutdown_connections 2023-10-16T10:02:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.505+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/1888572728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54c8004850 0x7f54c8004cd0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.505+0000 7f54e3dd5700 1 -- 172.21.15.181:0/1888572728 >> 172.21.15.181:0/1888572728 conn(0x7f54dc06aed0 msgr2=0x7f54dc0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.505+0000 7f54e3dd5700 1 -- 172.21.15.181:0/1888572728 shutdown_connections 2023-10-16T10:02:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.505+0000 7f54e3dd5700 1 -- 172.21.15.181:0/1888572728 wait complete. 2023-10-16T10:02:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.506+0000 7f54e3dd5700 1 Processor -- start 2023-10-16T10:02:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.506+0000 7f54e3dd5700 1 -- start start 2023-10-16T10:02:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8004850 0x7f54dc16a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54dc0adb30 0x7f54dc0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54dc0ad480 0x7f54dc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0ae970 con 0x7f54dc0adb30 2023-10-16T10:02:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0c6770 con 0x7f54dc0ad480 2023-10-16T10:02:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0ae0d0 con 0x7f54c8004850 2023-10-16T10:02:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54dc0adb30 0x7f54dc0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54dc0adb30 0x7f54dc0acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46068/0 (socket says 172.21.15.181:46068) 2023-10-16T10:02:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 -- 172.21.15.181:0/253364858 learned_addr learned my addr 172.21.15.181:0/253364858 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e2372700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54dc0ad480 0x7f54dc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1b71700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8004850 0x7f54dc16a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 -- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8004850 msgr2=0x7f54dc16a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8004850 0x7f54dc16a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 -- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54dc0ad480 msgr2=0x7f54dc15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54dc0ad480 0x7f54dc15c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 -- 172.21.15.181:0/253364858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc0ae3b0 con 0x7f54dc0adb30 2023-10-16T10:02:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1b71700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8004850 0x7f54dc16a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:14.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e1370700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54dc0adb30 0x7f54dc0acf40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c830 tx=0x7f54cc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54d27fc700 1 -- 172.21.15.181:0/253364858 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc001630 con 0x7f54dc0adb30 2023-10-16T10:02:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54d27fc700 1 -- 172.21.15.181:0/253364858 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc0070d0 con 0x7f54dc0adb30 2023-10-16T10:02:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54d27fc700 1 -- 172.21.15.181:0/253364858 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc007480 con 0x7f54dc0adb30 2023-10-16T10:02:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54dc0ae690 con 0x7f54dc0adb30 2023-10-16T10:02:14.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.507+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54dc15cd00 con 0x7f54dc0adb30 2023-10-16T10:02:14.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.508+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54dc0ac880 con 0x7f54dc0adb30 2023-10-16T10:02:14.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.509+0000 7f54d27fc700 1 -- 172.21.15.181:0/253364858 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f54cc0075e0 con 0x7f54dc0adb30 2023-10-16T10:02:14.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.509+0000 7f54d27fc700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f54c006b020 0x7f54c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.509+0000 7f54e1b71700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f54c006b020 0x7f54c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.509+0000 7f54d27fc700 1 -- 172.21.15.181:0/253364858 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f54cc08ab30 con 0x7f54dc0adb30 2023-10-16T10:02:14.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.510+0000 7f54e1b71700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f54c006b020 0x7f54c006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54d8000c00 tx=0x7f54d8009040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.514+0000 7f54d27fc700 1 -- 172.21.15.181:0/253364858 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f54dc0ac880 con 0x7f54dc0adb30 2023-10-16T10:02:14.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.706+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 --> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54dc072970 con 0x7f54c006b020 2023-10-16T10:02:14.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.714+0000 7f54d27fc700 1 -- 172.21.15.181:0/253364858 <== mgr.7779 v2:172.21.15.181:6832/2093458419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289993 (secure 0 0 0) 0x7f54d800d050 con 0x7f54c006b020 2023-10-16T10:02:14.719 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:14.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f54c006b020 msgr2=0x7f54c006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f54c006b020 0x7f54c006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54d8000c00 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54dc0adb30 msgr2=0x7f54dc0acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54dc0adb30 0x7f54dc0acf40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c830 tx=0x7f54cc001a00 comp rx=0 tx=0).stop 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 shutdown_connections 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f54c006b020 0x7f54c006d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c8004850 0x7f54dc16a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54dc0ad480 0x7f54dc15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 --2- 172.21.15.181:0/253364858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54dc0adb30 0x7f54dc0acf40 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 >> 172.21.15.181:0/253364858 conn(0x7f54dc06aed0 msgr2=0x7f54dc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 shutdown_connections 2023-10-16T10:02:14.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.717+0000 7f54e3dd5700 1 -- 172.21.15.181:0/253364858 wait complete. 2023-10-16T10:02:14.724 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:02:14.731 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:02:14.557536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314736,"kb_used_data":12524,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754659984,"statfs":{"total":773094113280,"available":772771823616,"internally_reserved":0,"allocated":12824576,"data_stored":6886944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.959045"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615579+0000","last_change":"2023-10-16T10:02:10.635063+0000","last_active":"2023-10-16T10:02:11.615579+0000","last_peered":"2023-10-16T10:02:11.615579+0000","last_clean":"2023-10-16T10:02:11.615579+0000","last_became_active":"2023-10-16T10:02:10.630042+0000","last_became_peered":"2023-10-16T10:02:10.630042+0000","last_unstale":"2023-10-16T10:02:11.615579+0000","last_undegraded":"2023-10-16T10:02:11.615579+0000","last_fullsized":"2023-10-16T10:02:11.615579+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:37:23.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644619+0000","last_change":"2023-10-16T10:02:10.633784+0000","last_active":"2023-10-16T10:02:10.644619+0000","last_peered":"2023-10-16T10:02:10.644619+0000","last_clean":"2023-10-16T10:02:10.644619+0000","last_became_active":"2023-10-16T10:02:10.633568+0000","last_became_peered":"2023-10-16T10:02:10.633568+0000","last_unstale":"2023-10-16T10:02:10.644619+0000","last_undegraded":"2023-10-16T10:02:10.644619+0000","last_fullsized":"2023-10-16T10:02:10.644619+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:20.315841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615097+0000","last_change":"2023-10-16T10:02:10.634711+0000","last_active":"2023-10-16T10:02:11.615097+0000","last_peered":"2023-10-16T10:02:11.615097+0000","last_clean":"2023-10-16T10:02:11.615097+0000","last_became_active":"2023-10-16T10:02:10.634187+0000","last_became_peered":"2023-10-16T10:02:10.634187+0000","last_unstale":"2023-10-16T10:02:11.615097+0000","last_undegraded":"2023-10-16T10:02:11.615097+0000","last_fullsized":"2023-10-16T10:02:11.615097+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:03.479857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375696+0000","last_change":"2023-10-16T10:02:10.629312+0000","last_active":"2023-10-16T10:02:12.375696+0000","last_peered":"2023-10-16T10:02:12.375696+0000","last_clean":"2023-10-16T10:02:12.375696+0000","last_became_active":"2023-10-16T10:02:10.629052+0000","last_became_peered":"2023-10-16T10:02:10.629052+0000","last_unstale":"2023-10-16T10:02:12.375696+0000","last_undegraded":"2023-10-16T10:02:12.375696+0000","last_fullsized":"2023-10-16T10:02:12.375696+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:00:31.783543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637745+0000","last_change":"2023-10-16T10:02:10.634724+0000","last_active":"2023-10-16T10:02:10.637745+0000","last_peered":"2023-10-16T10:02:10.637745+0000","last_clean":"2023-10-16T10:02:10.637745+0000","last_became_active":"2023-10-16T10:02:10.633039+0000","last_became_peered":"2023-10-16T10:02:10.633039+0000","last_unstale":"2023-10-16T10:02:10.637745+0000","last_undegraded":"2023-10-16T10:02:10.637745+0000","last_fullsized":"2023-10-16T10:02:10.637745+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:53.103175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309414+0000","last_change":"2023-10-16T10:02:10.616124+0000","last_active":"2023-10-16T10:02:12.309414+0000","last_peered":"2023-10-16T10:02:12.309414+0000","last_clean":"2023-10-16T10:02:12.309414+0000","last_became_active":"2023-10-16T10:02:10.615721+0000","last_became_peered":"2023-10-16T10:02:10.615721+0000","last_unstale":"2023-10-16T10:02:12.309414+0000","last_undegraded":"2023-10-16T10:02:12.309414+0000","last_fullsized":"2023-10-16T10:02:12.309414+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:00.899263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309423+0000","last_change":"2023-10-16T10:02:10.616359+0000","last_active":"2023-10-16T10:02:12.309423+0000","last_peered":"2023-10-16T10:02:12.309423+0000","last_clean":"2023-10-16T10:02:12.309423+0000","last_became_active":"2023-10-16T10:02:10.616094+0000","last_became_peered":"2023-10-16T10:02:10.616094+0000","last_unstale":"2023-10-16T10:02:12.309423+0000","last_undegraded":"2023-10-16T10:02:12.309423+0000","last_fullsized":"2023-10-16T10:02:12.309423+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.065635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.638991+0000","last_change":"2023-10-16T10:02:10.632204+0000","last_active":"2023-10-16T10:02:10.638991+0000","last_peered":"2023-10-16T10:02:10.638991+0000","last_clean":"2023-10-16T10:02:10.638991+0000","last_became_active":"2023-10-16T10:02:10.631861+0000","last_became_peered":"2023-10-16T10:02:10.631861+0000","last_unstale":"2023-10-16T10:02:10.638991+0000","last_undegraded":"2023-10-16T10:02:10.638991+0000","last_fullsized":"2023-10-16T10:02:10.638991+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:59:59.196866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637198+0000","last_change":"2023-10-16T10:02:10.633439+0000","last_active":"2023-10-16T10:02:10.637198+0000","last_peered":"2023-10-16T10:02:10.637198+0000","last_clean":"2023-10-16T10:02:10.637198+0000","last_became_active":"2023-10-16T10:02:10.632982+0000","last_became_peered":"2023-10-16T10:02:10.632982+0000","last_unstale":"2023-10-16T10:02:10.637198+0000","last_undegraded":"2023-10-16T10:02:10.637198+0000","last_fullsized":"2023-10-16T10:02:10.637198+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:37.553361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637799+0000","last_change":"2023-10-16T10:02:10.635377+0000","last_active":"2023-10-16T10:02:10.637799+0000","last_peered":"2023-10-16T10:02:10.637799+0000","last_clean":"2023-10-16T10:02:10.637799+0000","last_became_active":"2023-10-16T10:02:10.634910+0000","last_became_peered":"2023-10-16T10:02:10.634910+0000","last_unstale":"2023-10-16T10:02:10.637799+0000","last_undegraded":"2023-10-16T10:02:10.637799+0000","last_fullsized":"2023-10-16T10:02:10.637799+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:20.315841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309639+0000","last_change":"2023-10-16T10:02:10.624373+0000","last_active":"2023-10-16T10:02:12.309639+0000","last_peered":"2023-10-16T10:02:12.309639+0000","last_clean":"2023-10-16T10:02:12.309639+0000","last_became_active":"2023-10-16T10:02:10.623772+0000","last_became_peered":"2023-10-16T10:02:10.623772+0000","last_unstale":"2023-10-16T10:02:12.309639+0000","last_undegraded":"2023-10-16T10:02:12.309639+0000","last_fullsized":"2023-10-16T10:02:12.309639+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:58.074294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637127+0000","last_change":"2023-10-16T10:02:10.634130+0000","last_active":"2023-10-16T10:02:10.637127+0000","last_peered":"2023-10-16T10:02:10.637127+0000","last_clean":"2023-10-16T10:02:10.637127+0000","last_became_active":"2023-10-16T10:02:10.633788+0000","last_became_peered":"2023-10-16T10:02:10.633788+0000","last_unstale":"2023-10-16T10:02:10.637127+0000","last_undegraded":"2023-10-16T10:02:10.637127+0000","last_fullsized":"2023-10-16T10:02:10.637127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:46.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375644+0000","last_change":"2023-10-16T10:02:10.619252+0000","last_active":"2023-10-16T10:02:12.375644+0000","last_peered":"2023-10-16T10:02:12.375644+0000","last_clean":"2023-10-16T10:02:12.375644+0000","last_became_active":"2023-10-16T10:02:10.618992+0000","last_became_peered":"2023-10-16T10:02:10.618992+0000","last_unstale":"2023-10-16T10:02:12.375644+0000","last_undegraded":"2023-10-16T10:02:12.375644+0000","last_fullsized":"2023-10-16T10:02:12.375644+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:13:46.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615889+0000","last_change":"2023-10-16T10:02:10.627401+0000","last_active":"2023-10-16T10:02:11.615889+0000","last_peered":"2023-10-16T10:02:11.615889+0000","last_clean":"2023-10-16T10:02:11.615889+0000","last_became_active":"2023-10-16T10:02:10.626944+0000","last_became_peered":"2023-10-16T10:02:10.626944+0000","last_unstale":"2023-10-16T10:02:11.615889+0000","last_undegraded":"2023-10-16T10:02:11.615889+0000","last_fullsized":"2023-10-16T10:02:11.615889+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:02.457607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375411+0000","last_change":"2023-10-16T10:02:10.631746+0000","last_active":"2023-10-16T10:02:12.375411+0000","last_peered":"2023-10-16T10:02:12.375411+0000","last_clean":"2023-10-16T10:02:12.375411+0000","last_became_active":"2023-10-16T10:02:10.631476+0000","last_became_peered":"2023-10-16T10:02:10.631476+0000","last_unstale":"2023-10-16T10:02:12.375411+0000","last_undegraded":"2023-10-16T10:02:12.375411+0000","last_fullsized":"2023-10-16T10:02:12.375411+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:19:29.735131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309492+0000","last_change":"2023-10-16T10:02:10.635901+0000","last_active":"2023-10-16T10:02:12.309492+0000","last_peered":"2023-10-16T10:02:12.309492+0000","last_clean":"2023-10-16T10:02:12.309492+0000","last_became_active":"2023-10-16T10:02:10.635321+0000","last_became_peered":"2023-10-16T10:02:10.635321+0000","last_unstale":"2023-10-16T10:02:12.309492+0000","last_undegraded":"2023-10-16T10:02:12.309492+0000","last_fullsized":"2023-10-16T10:02:12.309492+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:33.810501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644645+0000","last_change":"2023-10-16T10:02:10.632785+0000","last_active":"2023-10-16T10:02:10.644645+0000","last_peered":"2023-10-16T10:02:10.644645+0000","last_clean":"2023-10-16T10:02:10.644645+0000","last_became_active":"2023-10-16T10:02:10.632548+0000","last_became_peered":"2023-10-16T10:02:10.632548+0000","last_unstale":"2023-10-16T10:02:10.644645+0000","last_undegraded":"2023-10-16T10:02:10.644645+0000","last_fullsized":"2023-10-16T10:02:10.644645+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:41:23.405018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375710+0000","last_change":"2023-10-16T10:02:10.628528+0000","last_active":"2023-10-16T10:02:12.375710+0000","last_peered":"2023-10-16T10:02:12.375710+0000","last_clean":"2023-10-16T10:02:12.375710+0000","last_became_active":"2023-10-16T10:02:10.628274+0000","last_became_peered":"2023-10-16T10:02:10.628274+0000","last_unstale":"2023-10-16T10:02:12.375710+0000","last_undegraded":"2023-10-16T10:02:12.375710+0000","last_fullsized":"2023-10-16T10:02:12.375710+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:20.588202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616063+0000","last_change":"2023-10-16T10:02:10.640628+0000","last_active":"2023-10-16T10:02:11.616063+0000","last_peered":"2023-10-16T10:02:11.616063+0000","last_clean":"2023-10-16T10:02:11.616063+0000","last_became_active":"2023-10-16T10:02:10.636483+0000","last_became_peered":"2023-10-16T10:02:10.636483+0000","last_unstale":"2023-10-16T10:02:11.616063+0000","last_undegraded":"2023-10-16T10:02:11.616063+0000","last_fullsized":"2023-10-16T10:02:11.616063+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:15.644973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637064+0000","last_change":"2023-10-16T10:02:10.633359+0000","last_active":"2023-10-16T10:02:10.637064+0000","last_peered":"2023-10-16T10:02:10.637064+0000","last_clean":"2023-10-16T10:02:10.637064+0000","last_became_active":"2023-10-16T10:02:10.632889+0000","last_became_peered":"2023-10-16T10:02:10.632889+0000","last_unstale":"2023-10-16T10:02:10.637064+0000","last_undegraded":"2023-10-16T10:02:10.637064+0000","last_fullsized":"2023-10-16T10:02:10.637064+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:31.689737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616334+0000","last_change":"2023-10-16T10:02:10.631229+0000","last_active":"2023-10-16T10:02:11.616334+0000","last_peered":"2023-10-16T10:02:11.616334+0000","last_clean":"2023-10-16T10:02:11.616334+0000","last_became_active":"2023-10-16T10:02:10.630748+0000","last_became_peered":"2023-10-16T10:02:10.630748+0000","last_unstale":"2023-10-16T10:02:11.616334+0000","last_undegraded":"2023-10-16T10:02:11.616334+0000","last_fullsized":"2023-10-16T10:02:11.616334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:21.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644637+0000","last_change":"2023-10-16T10:02:10.632557+0000","last_active":"2023-10-16T10:02:10.644637+0000","last_peered":"2023-10-16T10:02:10.644637+0000","last_clean":"2023-10-16T10:02:10.644637+0000","last_became_active":"2023-10-16T10:02:10.632257+0000","last_became_peered":"2023-10-16T10:02:10.632257+0000","last_unstale":"2023-10-16T10:02:10.644637+0000","last_undegraded":"2023-10-16T10:02:10.644637+0000","last_fullsized":"2023-10-16T10:02:10.644637+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:04.618597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644524+0000","last_change":"2023-10-16T10:02:10.632197+0000","last_active":"2023-10-16T10:02:10.644524+0000","last_peered":"2023-10-16T10:02:10.644524+0000","last_clean":"2023-10-16T10:02:10.644524+0000","last_became_active":"2023-10-16T10:02:10.631836+0000","last_became_peered":"2023-10-16T10:02:10.631836+0000","last_unstale":"2023-10-16T10:02:10.644524+0000","last_undegraded":"2023-10-16T10:02:10.644524+0000","last_fullsized":"2023-10-16T10:02:10.644524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:19.116618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309586+0000","last_change":"2023-10-16T10:02:10.632198+0000","last_active":"2023-10-16T10:02:12.309586+0000","last_peered":"2023-10-16T10:02:12.309586+0000","last_clean":"2023-10-16T10:02:12.309586+0000","last_became_active":"2023-10-16T10:02:10.631915+0000","last_became_peered":"2023-10-16T10:02:10.631915+0000","last_unstale":"2023-10-16T10:02:12.309586+0000","last_undegraded":"2023-10-16T10:02:12.309586+0000","last_fullsized":"2023-10-16T10:02:12.309586+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:59.326925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637370+0000","last_change":"2023-10-16T10:02:10.632976+0000","last_active":"2023-10-16T10:02:10.637370+0000","last_peered":"2023-10-16T10:02:10.637370+0000","last_clean":"2023-10-16T10:02:10.637370+0000","last_became_active":"2023-10-16T10:02:10.632681+0000","last_became_peered":"2023-10-16T10:02:10.632681+0000","last_unstale":"2023-10-16T10:02:10.637370+0000","last_undegraded":"2023-10-16T10:02:10.637370+0000","last_fullsized":"2023-10-16T10:02:10.637370+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:47.426245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645126+0000","last_change":"2023-10-16T10:02:10.638584+0000","last_active":"2023-10-16T10:02:10.645126+0000","last_peered":"2023-10-16T10:02:10.645126+0000","last_clean":"2023-10-16T10:02:10.645126+0000","last_became_active":"2023-10-16T10:02:10.636903+0000","last_became_peered":"2023-10-16T10:02:10.636903+0000","last_unstale":"2023-10-16T10:02:10.645126+0000","last_undegraded":"2023-10-16T10:02:10.645126+0000","last_fullsized":"2023-10-16T10:02:10.645126+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:23.253580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309493+0000","last_change":"2023-10-16T10:02:10.627332+0000","last_active":"2023-10-16T10:02:12.309493+0000","last_peered":"2023-10-16T10:02:12.309493+0000","last_clean":"2023-10-16T10:02:12.309493+0000","last_became_active":"2023-10-16T10:02:10.626741+0000","last_became_peered":"2023-10-16T10:02:10.626741+0000","last_unstale":"2023-10-16T10:02:12.309493+0000","last_undegraded":"2023-10-16T10:02:12.309493+0000","last_fullsized":"2023-10-16T10:02:12.309493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:19:29.735131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637814+0000","last_change":"2023-10-16T10:02:10.635323+0000","last_active":"2023-10-16T10:02:10.637814+0000","last_peered":"2023-10-16T10:02:10.637814+0000","last_clean":"2023-10-16T10:02:10.637814+0000","last_became_active":"2023-10-16T10:02:10.632172+0000","last_became_peered":"2023-10-16T10:02:10.632172+0000","last_unstale":"2023-10-16T10:02:10.637814+0000","last_undegraded":"2023-10-16T10:02:10.637814+0000","last_fullsized":"2023-10-16T10:02:10.637814+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:18.531788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644946+0000","last_change":"2023-10-16T10:02:10.638753+0000","last_active":"2023-10-16T10:02:10.644946+0000","last_peered":"2023-10-16T10:02:10.644946+0000","last_clean":"2023-10-16T10:02:10.644946+0000","last_became_active":"2023-10-16T10:02:10.636894+0000","last_became_peered":"2023-10-16T10:02:10.636894+0000","last_unstale":"2023-10-16T10:02:10.644946+0000","last_undegraded":"2023-10-16T10:02:10.644946+0000","last_fullsized":"2023-10-16T10:02:10.644946+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:53.103175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615852+0000","last_change":"2023-10-16T10:02:10.633780+0000","last_active":"2023-10-16T10:02:11.615852+0000","last_peered":"2023-10-16T10:02:11.615852+0000","last_clean":"2023-10-16T10:02:11.615852+0000","last_became_active":"2023-10-16T10:02:10.633489+0000","last_became_peered":"2023-10-16T10:02:10.633489+0000","last_unstale":"2023-10-16T10:02:11.615852+0000","last_undegraded":"2023-10-16T10:02:11.615852+0000","last_fullsized":"2023-10-16T10:02:11.615852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:18.531788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375449+0000","last_change":"2023-10-16T10:02:10.627689+0000","last_active":"2023-10-16T10:02:12.375449+0000","last_peered":"2023-10-16T10:02:12.375449+0000","last_clean":"2023-10-16T10:02:12.375449+0000","last_became_active":"2023-10-16T10:02:10.627472+0000","last_became_peered":"2023-10-16T10:02:10.627472+0000","last_unstale":"2023-10-16T10:02:12.375449+0000","last_undegraded":"2023-10-16T10:02:12.375449+0000","last_fullsized":"2023-10-16T10:02:12.375449+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:39.553440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.614882+0000","last_change":"2023-10-16T10:02:10.639954+0000","last_active":"2023-10-16T10:02:11.614882+0000","last_peered":"2023-10-16T10:02:11.614882+0000","last_clean":"2023-10-16T10:02:11.614882+0000","last_became_active":"2023-10-16T10:02:10.639647+0000","last_became_peered":"2023-10-16T10:02:10.639647+0000","last_unstale":"2023-10-16T10:02:11.614882+0000","last_undegraded":"2023-10-16T10:02:11.614882+0000","last_fullsized":"2023-10-16T10:02:11.614882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:23.346970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616328+0000","last_change":"2023-10-16T10:02:10.626120+0000","last_active":"2023-10-16T10:02:11.616328+0000","last_peered":"2023-10-16T10:02:11.616328+0000","last_clean":"2023-10-16T10:02:11.616328+0000","last_became_active":"2023-10-16T10:02:10.625720+0000","last_became_peered":"2023-10-16T10:02:10.625720+0000","last_unstale":"2023-10-16T10:02:11.616328+0000","last_undegraded":"2023-10-16T10:02:11.616328+0000","last_fullsized":"2023-10-16T10:02:11.616328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:17.100311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"117'4","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645047+0000","last_change":"2023-10-16T10:02:09.619496+0000","last_active":"2023-10-16T10:02:10.645047+0000","last_peered":"2023-10-16T10:02:10.645047+0000","last_clean":"2023-10-16T10:02:10.645047+0000","last_became_active":"2023-10-16T10:02:09.619204+0000","last_became_peered":"2023-10-16T10:02:09.619204+0000","last_unstale":"2023-10-16T10:02:10.645047+0000","last_undegraded":"2023-10-16T10:02:10.645047+0000","last_fullsized":"2023-10-16T10:02:10.645047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:20.420135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375906+0000","last_change":"2023-10-16T10:02:10.627206+0000","last_active":"2023-10-16T10:02:12.375906+0000","last_peered":"2023-10-16T10:02:12.375906+0000","last_clean":"2023-10-16T10:02:12.375906+0000","last_became_active":"2023-10-16T10:02:10.626935+0000","last_became_peered":"2023-10-16T10:02:10.626935+0000","last_unstale":"2023-10-16T10:02:12.375906+0000","last_undegraded":"2023-10-16T10:02:12.375906+0000","last_fullsized":"2023-10-16T10:02:12.375906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:51.595635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309616+0000","last_change":"2023-10-16T10:02:09.617805+0000","last_active":"2023-10-16T10:02:12.309616+0000","last_peered":"2023-10-16T10:02:12.309616+0000","last_clean":"2023-10-16T10:02:12.309616+0000","last_became_active":"2023-10-16T10:02:09.617505+0000","last_became_peered":"2023-10-16T10:02:09.617505+0000","last_unstale":"2023-10-16T10:02:12.309616+0000","last_undegraded":"2023-10-16T10:02:12.309616+0000","last_fullsized":"2023-10-16T10:02:12.309616+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:22.521417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615619+0000","last_change":"2023-10-16T10:02:10.628986+0000","last_active":"2023-10-16T10:02:11.615619+0000","last_peered":"2023-10-16T10:02:11.615619+0000","last_clean":"2023-10-16T10:02:11.615619+0000","last_became_active":"2023-10-16T10:02:10.628481+0000","last_became_peered":"2023-10-16T10:02:10.628481+0000","last_unstale":"2023-10-16T10:02:11.615619+0000","last_undegraded":"2023-10-16T10:02:11.615619+0000","last_fullsized":"2023-10-16T10:02:11.615619+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:25:20.989352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645892+0000","last_change":"2023-10-16T10:02:09.618329+0000","last_active":"2023-10-16T10:02:10.645892+0000","last_peered":"2023-10-16T10:02:10.645892+0000","last_clean":"2023-10-16T10:02:10.645892+0000","last_became_active":"2023-10-16T10:02:09.617391+0000","last_became_peered":"2023-10-16T10:02:09.617391+0000","last_unstale":"2023-10-16T10:02:10.645892+0000","last_undegraded":"2023-10-16T10:02:10.645892+0000","last_fullsized":"2023-10-16T10:02:10.645892+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:17.526510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.627749+0000","last_change":"2023-10-16T10:02:10.626513+0000","last_active":"2023-10-16T10:02:10.627749+0000","last_peered":"2023-10-16T10:02:10.627749+0000","last_clean":"2023-10-16T10:02:10.627749+0000","last_became_active":"2023-10-16T10:02:10.626216+0000","last_became_peered":"2023-10-16T10:02:10.626216+0000","last_unstale":"2023-10-16T10:02:10.627749+0000","last_undegraded":"2023-10-16T10:02:10.627749+0000","last_fullsized":"2023-10-16T10:02:10.627749+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:46.485858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644789+0000","last_change":"2023-10-16T10:02:09.620447+0000","last_active":"2023-10-16T10:02:10.644789+0000","last_peered":"2023-10-16T10:02:10.644789+0000","last_clean":"2023-10-16T10:02:10.644789+0000","last_became_active":"2023-10-16T10:02:09.620142+0000","last_became_peered":"2023-10-16T10:02:09.620142+0000","last_unstale":"2023-10-16T10:02:10.644789+0000","last_undegraded":"2023-10-16T10:02:10.644789+0000","last_fullsized":"2023-10-16T10:02:10.644789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:46.890745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616347+0000","last_change":"2023-10-16T10:02:10.619758+0000","last_active":"2023-10-16T10:02:11.616347+0000","last_peered":"2023-10-16T10:02:11.616347+0000","last_clean":"2023-10-16T10:02:11.616347+0000","last_became_active":"2023-10-16T10:02:10.619437+0000","last_became_peered":"2023-10-16T10:02:10.619437+0000","last_unstale":"2023-10-16T10:02:11.616347+0000","last_undegraded":"2023-10-16T10:02:11.616347+0000","last_fullsized":"2023-10-16T10:02:11.616347+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:37.937239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615432+0000","last_change":"2023-10-16T10:02:09.618119+0000","last_active":"2023-10-16T10:02:11.615432+0000","last_peered":"2023-10-16T10:02:11.615432+0000","last_clean":"2023-10-16T10:02:11.615432+0000","last_became_active":"2023-10-16T10:02:09.617851+0000","last_became_peered":"2023-10-16T10:02:09.617851+0000","last_unstale":"2023-10-16T10:02:11.615432+0000","last_undegraded":"2023-10-16T10:02:11.615432+0000","last_fullsized":"2023-10-16T10:02:11.615432+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:33.775750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309505+0000","last_change":"2023-10-16T10:02:10.624892+0000","last_active":"2023-10-16T10:02:12.309505+0000","last_peered":"2023-10-16T10:02:12.309505+0000","last_clean":"2023-10-16T10:02:12.309505+0000","last_became_active":"2023-10-16T10:02:10.624574+0000","last_became_peered":"2023-10-16T10:02:10.624574+0000","last_unstale":"2023-10-16T10:02:12.309505+0000","last_undegraded":"2023-10-16T10:02:12.309505+0000","last_fullsized":"2023-10-16T10:02:12.309505+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:19.550899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"117'1","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644664+0000","last_change":"2023-10-16T10:02:09.617884+0000","last_active":"2023-10-16T10:02:10.644664+0000","last_peered":"2023-10-16T10:02:10.644664+0000","last_clean":"2023-10-16T10:02:10.644664+0000","last_became_active":"2023-10-16T10:02:09.617629+0000","last_became_peered":"2023-10-16T10:02:09.617629+0000","last_unstale":"2023-10-16T10:02:10.644664+0000","last_undegraded":"2023-10-16T10:02:10.644664+0000","last_fullsized":"2023-10-16T10:02:10.644664+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:14.639695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375448+0000","last_change":"2023-10-16T10:02:10.632282+0000","last_active":"2023-10-16T10:02:12.375448+0000","last_peered":"2023-10-16T10:02:12.375448+0000","last_clean":"2023-10-16T10:02:12.375448+0000","last_became_active":"2023-10-16T10:02:10.632023+0000","last_became_peered":"2023-10-16T10:02:10.632023+0000","last_unstale":"2023-10-16T10:02:12.375448+0000","last_undegraded":"2023-10-16T10:02:12.375448+0000","last_fullsized":"2023-10-16T10:02:12.375448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:59.326925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625352+0000","last_change":"2023-10-16T10:02:09.619478+0000","last_active":"2023-10-16T10:02:10.625352+0000","last_peered":"2023-10-16T10:02:10.625352+0000","last_clean":"2023-10-16T10:02:10.625352+0000","last_became_active":"2023-10-16T10:02:09.619200+0000","last_became_peered":"2023-10-16T10:02:09.619200+0000","last_unstale":"2023-10-16T10:02:10.625352+0000","last_undegraded":"2023-10-16T10:02:10.625352+0000","last_fullsized":"2023-10-16T10:02:10.625352+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:14.639695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.629071+0000","last_change":"2023-10-16T10:02:10.627183+0000","last_active":"2023-10-16T10:02:10.629071+0000","last_peered":"2023-10-16T10:02:10.629071+0000","last_clean":"2023-10-16T10:02:10.629071+0000","last_became_active":"2023-10-16T10:02:10.626898+0000","last_became_peered":"2023-10-16T10:02:10.626898+0000","last_unstale":"2023-10-16T10:02:10.629071+0000","last_undegraded":"2023-10-16T10:02:10.629071+0000","last_fullsized":"2023-10-16T10:02:10.629071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:08.659536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.614941+0000","last_change":"2023-10-16T10:02:09.616819+0000","last_active":"2023-10-16T10:02:11.614941+0000","last_peered":"2023-10-16T10:02:11.614941+0000","last_clean":"2023-10-16T10:02:11.614941+0000","last_became_active":"2023-10-16T10:02:09.616563+0000","last_became_peered":"2023-10-16T10:02:09.616563+0000","last_unstale":"2023-10-16T10:02:11.614941+0000","last_undegraded":"2023-10-16T10:02:11.614941+0000","last_fullsized":"2023-10-16T10:02:11.614941+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:59.508408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644606+0000","last_change":"2023-10-16T10:02:10.633698+0000","last_active":"2023-10-16T10:02:10.644606+0000","last_peered":"2023-10-16T10:02:10.644606+0000","last_clean":"2023-10-16T10:02:10.644606+0000","last_became_active":"2023-10-16T10:02:10.633437+0000","last_became_peered":"2023-10-16T10:02:10.633437+0000","last_unstale":"2023-10-16T10:02:10.644606+0000","last_undegraded":"2023-10-16T10:02:10.644606+0000","last_fullsized":"2023-10-16T10:02:10.644606+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:12.483258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615258+0000","last_change":"2023-10-16T10:02:09.614421+0000","last_active":"2023-10-16T10:02:11.615258+0000","last_peered":"2023-10-16T10:02:11.615258+0000","last_clean":"2023-10-16T10:02:11.615258+0000","last_became_active":"2023-10-16T10:02:09.614181+0000","last_became_peered":"2023-10-16T10:02:09.614181+0000","last_unstale":"2023-10-16T10:02:11.615258+0000","last_undegraded":"2023-10-16T10:02:11.615258+0000","last_fullsized":"2023-10-16T10:02:11.615258+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:20.420135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637028+0000","last_change":"2023-10-16T10:02:10.633257+0000","last_active":"2023-10-16T10:02:10.637028+0000","last_peered":"2023-10-16T10:02:10.637028+0000","last_clean":"2023-10-16T10:02:10.637028+0000","last_became_active":"2023-10-16T10:02:10.632567+0000","last_became_peered":"2023-10-16T10:02:10.632567+0000","last_unstale":"2023-10-16T10:02:10.637028+0000","last_undegraded":"2023-10-16T10:02:10.637028+0000","last_fullsized":"2023-10-16T10:02:10.637028+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:10:26.578599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375816+0000","last_change":"2023-10-16T10:02:09.619611+0000","last_active":"2023-10-16T10:02:12.375816+0000","last_peered":"2023-10-16T10:02:12.375816+0000","last_clean":"2023-10-16T10:02:12.375816+0000","last_became_active":"2023-10-16T10:02:09.619315+0000","last_became_peered":"2023-10-16T10:02:09.619315+0000","last_unstale":"2023-10-16T10:02:12.375816+0000","last_undegraded":"2023-10-16T10:02:12.375816+0000","last_fullsized":"2023-10-16T10:02:12.375816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:18.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637792+0000","last_change":"2023-10-16T10:02:10.634953+0000","last_active":"2023-10-16T10:02:10.637792+0000","last_peered":"2023-10-16T10:02:10.637792+0000","last_clean":"2023-10-16T10:02:10.637792+0000","last_became_active":"2023-10-16T10:02:10.633419+0000","last_became_peered":"2023-10-16T10:02:10.633419+0000","last_unstale":"2023-10-16T10:02:10.637792+0000","last_undegraded":"2023-10-16T10:02:10.637792+0000","last_fullsized":"2023-10-16T10:02:10.637792+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:02.457607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"117'3","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309722+0000","last_change":"2023-10-16T10:02:09.615778+0000","last_active":"2023-10-16T10:02:12.309722+0000","last_peered":"2023-10-16T10:02:12.309722+0000","last_clean":"2023-10-16T10:02:12.309722+0000","last_became_active":"2023-10-16T10:02:09.615343+0000","last_became_peered":"2023-10-16T10:02:09.615343+0000","last_unstale":"2023-10-16T10:02:12.309722+0000","last_undegraded":"2023-10-16T10:02:12.309722+0000","last_fullsized":"2023-10-16T10:02:12.309722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309622+0000","last_change":"2023-10-16T10:02:10.634831+0000","last_active":"2023-10-16T10:02:12.309622+0000","last_peered":"2023-10-16T10:02:12.309622+0000","last_clean":"2023-10-16T10:02:12.309622+0000","last_became_active":"2023-10-16T10:02:10.634468+0000","last_became_peered":"2023-10-16T10:02:10.634468+0000","last_unstale":"2023-10-16T10:02:12.309622+0000","last_undegraded":"2023-10-16T10:02:12.309622+0000","last_fullsized":"2023-10-16T10:02:12.309622+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:39.553440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"117'6","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644564+0000","last_change":"2023-10-16T10:02:09.613487+0000","last_active":"2023-10-16T10:02:10.644564+0000","last_peered":"2023-10-16T10:02:10.644564+0000","last_clean":"2023-10-16T10:02:10.644564+0000","last_became_active":"2023-10-16T10:02:09.613224+0000","last_became_peered":"2023-10-16T10:02:09.613224+0000","last_unstale":"2023-10-16T10:02:10.644564+0000","last_undegraded":"2023-10-16T10:02:10.644564+0000","last_fullsized":"2023-10-16T10:02:10.644564+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:18.051230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375586+0000","last_change":"2023-10-16T10:02:10.617193+0000","last_active":"2023-10-16T10:02:12.375586+0000","last_peered":"2023-10-16T10:02:12.375586+0000","last_clean":"2023-10-16T10:02:12.375586+0000","last_became_active":"2023-10-16T10:02:10.616921+0000","last_became_peered":"2023-10-16T10:02:10.616921+0000","last_unstale":"2023-10-16T10:02:12.375586+0000","last_undegraded":"2023-10-16T10:02:12.375586+0000","last_fullsized":"2023-10-16T10:02:12.375586+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:58.074294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616298+0000","last_change":"2023-10-16T10:02:09.612223+0000","last_active":"2023-10-16T10:02:11.616298+0000","last_peered":"2023-10-16T10:02:11.616298+0000","last_clean":"2023-10-16T10:02:11.616298+0000","last_became_active":"2023-10-16T10:02:09.611964+0000","last_became_peered":"2023-10-16T10:02:09.611964+0000","last_unstale":"2023-10-16T10:02:11.616298+0000","last_undegraded":"2023-10-16T10:02:11.616298+0000","last_fullsized":"2023-10-16T10:02:11.616298+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:52.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615564+0000","last_change":"2023-10-16T10:02:10.628712+0000","last_active":"2023-10-16T10:02:11.615564+0000","last_peered":"2023-10-16T10:02:11.615564+0000","last_clean":"2023-10-16T10:02:11.615564+0000","last_became_active":"2023-10-16T10:02:10.628438+0000","last_became_peered":"2023-10-16T10:02:10.628438+0000","last_unstale":"2023-10-16T10:02:11.615564+0000","last_undegraded":"2023-10-16T10:02:11.615564+0000","last_fullsized":"2023-10-16T10:02:11.615564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:32.976285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616080+0000","last_change":"2023-10-16T10:02:09.616584+0000","last_active":"2023-10-16T10:02:11.616080+0000","last_peered":"2023-10-16T10:02:11.616080+0000","last_clean":"2023-10-16T10:02:11.616080+0000","last_became_active":"2023-10-16T10:02:09.616291+0000","last_became_peered":"2023-10-16T10:02:09.616291+0000","last_unstale":"2023-10-16T10:02:11.616080+0000","last_undegraded":"2023-10-16T10:02:11.616080+0000","last_fullsized":"2023-10-16T10:02:11.616080+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:11.477980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.636962+0000","last_change":"2023-10-16T10:02:10.632497+0000","last_active":"2023-10-16T10:02:10.636962+0000","last_peered":"2023-10-16T10:02:10.636962+0000","last_clean":"2023-10-16T10:02:10.636962+0000","last_became_active":"2023-10-16T10:02:10.632194+0000","last_became_peered":"2023-10-16T10:02:10.632194+0000","last_unstale":"2023-10-16T10:02:10.636962+0000","last_undegraded":"2023-10-16T10:02:10.636962+0000","last_fullsized":"2023-10-16T10:02:10.636962+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:14.868185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.621611+0000","last_change":"2023-10-16T10:02:09.615236+0000","last_active":"2023-10-16T10:02:10.621611+0000","last_peered":"2023-10-16T10:02:10.621611+0000","last_clean":"2023-10-16T10:02:10.621611+0000","last_became_active":"2023-10-16T10:02:09.614973+0000","last_became_peered":"2023-10-16T10:02:09.614973+0000","last_unstale":"2023-10-16T10:02:10.621611+0000","last_undegraded":"2023-10-16T10:02:10.621611+0000","last_fullsized":"2023-10-16T10:02:10.621611+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:35.252905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615808+0000","last_change":"2023-10-16T10:02:10.625544+0000","last_active":"2023-10-16T10:02:11.615808+0000","last_peered":"2023-10-16T10:02:11.615808+0000","last_clean":"2023-10-16T10:02:11.615808+0000","last_became_active":"2023-10-16T10:02:10.625050+0000","last_became_peered":"2023-10-16T10:02:10.625050+0000","last_unstale":"2023-10-16T10:02:11.615808+0000","last_undegraded":"2023-10-16T10:02:11.615808+0000","last_fullsized":"2023-10-16T10:02:11.615808+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:47.426245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"117'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615036+0000","last_change":"2023-10-16T10:02:09.615479+0000","last_active":"2023-10-16T10:02:11.615036+0000","last_peered":"2023-10-16T10:02:11.615036+0000","last_clean":"2023-10-16T10:02:11.615036+0000","last_became_active":"2023-10-16T10:02:09.615154+0000","last_became_peered":"2023-10-16T10:02:09.615154+0000","last_unstale":"2023-10-16T10:02:11.615036+0000","last_undegraded":"2023-10-16T10:02:11.615036+0000","last_fullsized":"2023-10-16T10:02:11.615036+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:29:48.775082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.626964+0000","last_change":"2023-10-16T10:02:10.624108+0000","last_active":"2023-10-16T10:02:10.626964+0000","last_peered":"2023-10-16T10:02:10.626964+0000","last_clean":"2023-10-16T10:02:10.626964+0000","last_became_active":"2023-10-16T10:02:10.623839+0000","last_became_peered":"2023-10-16T10:02:10.623839+0000","last_unstale":"2023-10-16T10:02:10.626964+0000","last_undegraded":"2023-10-16T10:02:10.626964+0000","last_fullsized":"2023-10-16T10:02:10.626964+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:23.253580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645803+0000","last_change":"2023-10-16T10:02:09.616793+0000","last_active":"2023-10-16T10:02:10.645803+0000","last_peered":"2023-10-16T10:02:10.645803+0000","last_clean":"2023-10-16T10:02:10.645803+0000","last_became_active":"2023-10-16T10:02:09.616506+0000","last_became_peered":"2023-10-16T10:02:09.616506+0000","last_unstale":"2023-10-16T10:02:10.645803+0000","last_undegraded":"2023-10-16T10:02:10.645803+0000","last_fullsized":"2023-10-16T10:02:10.645803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:05.085570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616501+0000","last_change":"2023-10-16T10:02:10.624019+0000","last_active":"2023-10-16T10:02:11.616501+0000","last_peered":"2023-10-16T10:02:11.616501+0000","last_clean":"2023-10-16T10:02:11.616501+0000","last_became_active":"2023-10-16T10:02:10.623719+0000","last_became_peered":"2023-10-16T10:02:10.623719+0000","last_unstale":"2023-10-16T10:02:11.616501+0000","last_undegraded":"2023-10-16T10:02:11.616501+0000","last_fullsized":"2023-10-16T10:02:11.616501+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:33:56.707067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"117'1","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637768+0000","last_change":"2023-10-16T10:02:09.612346+0000","last_active":"2023-10-16T10:02:10.637768+0000","last_peered":"2023-10-16T10:02:10.637768+0000","last_clean":"2023-10-16T10:02:10.637768+0000","last_became_active":"2023-10-16T10:02:09.609794+0000","last_became_peered":"2023-10-16T10:02:09.609794+0000","last_unstale":"2023-10-16T10:02:10.637768+0000","last_undegraded":"2023-10-16T10:02:10.637768+0000","last_fullsized":"2023-10-16T10:02:10.637768+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:08.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309780+0000","last_change":"2023-10-16T10:02:10.624552+0000","last_active":"2023-10-16T10:02:12.309780+0000","last_peered":"2023-10-16T10:02:12.309780+0000","last_clean":"2023-10-16T10:02:12.309780+0000","last_became_active":"2023-10-16T10:02:10.624113+0000","last_became_peered":"2023-10-16T10:02:10.624113+0000","last_unstale":"2023-10-16T10:02:12.309780+0000","last_undegraded":"2023-10-16T10:02:12.309780+0000","last_fullsized":"2023-10-16T10:02:12.309780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:13:46.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625326+0000","last_change":"2023-10-16T10:02:09.618634+0000","last_active":"2023-10-16T10:02:10.625326+0000","last_peered":"2023-10-16T10:02:10.625326+0000","last_clean":"2023-10-16T10:02:10.625326+0000","last_became_active":"2023-10-16T10:02:09.617097+0000","last_became_peered":"2023-10-16T10:02:09.617097+0000","last_unstale":"2023-10-16T10:02:10.625326+0000","last_undegraded":"2023-10-16T10:02:10.625326+0000","last_fullsized":"2023-10-16T10:02:10.625326+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:53.698172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615987+0000","last_change":"2023-10-16T10:02:10.642134+0000","last_active":"2023-10-16T10:02:11.615987+0000","last_peered":"2023-10-16T10:02:11.615987+0000","last_clean":"2023-10-16T10:02:11.615987+0000","last_became_active":"2023-10-16T10:02:10.641870+0000","last_became_peered":"2023-10-16T10:02:10.641870+0000","last_unstale":"2023-10-16T10:02:11.615987+0000","last_undegraded":"2023-10-16T10:02:11.615987+0000","last_fullsized":"2023-10-16T10:02:11.615987+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:54.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.648855+0000","last_change":"2023-10-16T10:02:09.620224+0000","last_active":"2023-10-16T10:02:10.648855+0000","last_peered":"2023-10-16T10:02:10.648855+0000","last_clean":"2023-10-16T10:02:10.648855+0000","last_became_active":"2023-10-16T10:02:09.619975+0000","last_became_peered":"2023-10-16T10:02:09.619975+0000","last_unstale":"2023-10-16T10:02:10.648855+0000","last_undegraded":"2023-10-16T10:02:10.648855+0000","last_fullsized":"2023-10-16T10:02:10.648855+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:20.471392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616218+0000","last_change":"2023-10-16T10:02:10.632929+0000","last_active":"2023-10-16T10:02:11.616218+0000","last_peered":"2023-10-16T10:02:11.616218+0000","last_clean":"2023-10-16T10:02:11.616218+0000","last_became_active":"2023-10-16T10:02:10.632633+0000","last_became_peered":"2023-10-16T10:02:10.632633+0000","last_unstale":"2023-10-16T10:02:11.616218+0000","last_undegraded":"2023-10-16T10:02:11.616218+0000","last_fullsized":"2023-10-16T10:02:11.616218+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:21.476670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616457+0000","last_change":"2023-10-16T10:02:09.616535+0000","last_active":"2023-10-16T10:02:11.616457+0000","last_peered":"2023-10-16T10:02:11.616457+0000","last_clean":"2023-10-16T10:02:11.616457+0000","last_became_active":"2023-10-16T10:02:09.616191+0000","last_became_peered":"2023-10-16T10:02:09.616191+0000","last_unstale":"2023-10-16T10:02:11.616457+0000","last_undegraded":"2023-10-16T10:02:11.616457+0000","last_fullsized":"2023-10-16T10:02:11.616457+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:58:14.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615932+0000","last_change":"2023-10-16T10:02:10.641130+0000","last_active":"2023-10-16T10:02:11.615932+0000","last_peered":"2023-10-16T10:02:11.615932+0000","last_clean":"2023-10-16T10:02:11.615932+0000","last_became_active":"2023-10-16T10:02:10.640872+0000","last_became_peered":"2023-10-16T10:02:10.640872+0000","last_unstale":"2023-10-16T10:02:11.615932+0000","last_undegraded":"2023-10-16T10:02:11.615932+0000","last_fullsized":"2023-10-16T10:02:11.615932+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:47.896023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616384+0000","last_change":"2023-10-16T10:02:09.608800+0000","last_active":"2023-10-16T10:02:11.616384+0000","last_peered":"2023-10-16T10:02:11.616384+0000","last_clean":"2023-10-16T10:02:11.616384+0000","last_became_active":"2023-10-16T10:02:09.608505+0000","last_became_peered":"2023-10-16T10:02:09.608505+0000","last_unstale":"2023-10-16T10:02:11.616384+0000","last_undegraded":"2023-10-16T10:02:11.616384+0000","last_fullsized":"2023-10-16T10:02:11.616384+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:30.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637850+0000","last_change":"2023-10-16T10:02:10.635709+0000","last_active":"2023-10-16T10:02:10.637850+0000","last_peered":"2023-10-16T10:02:10.637850+0000","last_clean":"2023-10-16T10:02:10.637850+0000","last_became_active":"2023-10-16T10:02:10.635408+0000","last_became_peered":"2023-10-16T10:02:10.635408+0000","last_unstale":"2023-10-16T10:02:10.637850+0000","last_undegraded":"2023-10-16T10:02:10.637850+0000","last_fullsized":"2023-10-16T10:02:10.637850+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:36.017941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.624937+0000","last_change":"2023-10-16T10:02:09.617864+0000","last_active":"2023-10-16T10:02:10.624937+0000","last_peered":"2023-10-16T10:02:10.624937+0000","last_clean":"2023-10-16T10:02:10.624937+0000","last_became_active":"2023-10-16T10:02:09.617054+0000","last_became_peered":"2023-10-16T10:02:09.617054+0000","last_unstale":"2023-10-16T10:02:10.624937+0000","last_undegraded":"2023-10-16T10:02:10.624937+0000","last_fullsized":"2023-10-16T10:02:10.624937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:46.420967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644586+0000","last_change":"2023-10-16T10:02:10.633202+0000","last_active":"2023-10-16T10:02:10.644586+0000","last_peered":"2023-10-16T10:02:10.644586+0000","last_clean":"2023-10-16T10:02:10.644586+0000","last_became_active":"2023-10-16T10:02:10.632931+0000","last_became_peered":"2023-10-16T10:02:10.632931+0000","last_unstale":"2023-10-16T10:02:10.644586+0000","last_undegraded":"2023-10-16T10:02:10.644586+0000","last_fullsized":"2023-10-16T10:02:10.644586+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:36.017941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616617+0000","last_change":"2023-10-16T10:02:09.613235+0000","last_active":"2023-10-16T10:02:11.616617+0000","last_peered":"2023-10-16T10:02:11.616617+0000","last_clean":"2023-10-16T10:02:11.616617+0000","last_became_active":"2023-10-16T10:02:09.612868+0000","last_became_peered":"2023-10-16T10:02:09.612868+0000","last_unstale":"2023-10-16T10:02:11.616617+0000","last_undegraded":"2023-10-16T10:02:11.616617+0000","last_fullsized":"2023-10-16T10:02:11.616617+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:48.368824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615174+0000","last_change":"2023-10-16T10:02:10.630321+0000","last_active":"2023-10-16T10:02:11.615174+0000","last_peered":"2023-10-16T10:02:11.615174+0000","last_clean":"2023-10-16T10:02:11.615174+0000","last_became_active":"2023-10-16T10:02:10.629740+0000","last_became_peered":"2023-10-16T10:02:10.629740+0000","last_unstale":"2023-10-16T10:02:11.615174+0000","last_undegraded":"2023-10-16T10:02:11.615174+0000","last_fullsized":"2023-10-16T10:02:11.615174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:19:24.035481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625454+0000","last_change":"2023-10-16T10:02:09.620358+0000","last_active":"2023-10-16T10:02:10.625454+0000","last_peered":"2023-10-16T10:02:10.625454+0000","last_clean":"2023-10-16T10:02:10.625454+0000","last_became_active":"2023-10-16T10:02:09.620082+0000","last_became_peered":"2023-10-16T10:02:09.620082+0000","last_unstale":"2023-10-16T10:02:10.625454+0000","last_undegraded":"2023-10-16T10:02:10.625454+0000","last_fullsized":"2023-10-16T10:02:10.625454+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:08.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615181+0000","last_change":"2023-10-16T10:02:10.629172+0000","last_active":"2023-10-16T10:02:11.615181+0000","last_peered":"2023-10-16T10:02:11.615181+0000","last_clean":"2023-10-16T10:02:11.615181+0000","last_became_active":"2023-10-16T10:02:10.628736+0000","last_became_peered":"2023-10-16T10:02:10.628736+0000","last_unstale":"2023-10-16T10:02:11.615181+0000","last_undegraded":"2023-10-16T10:02:11.615181+0000","last_fullsized":"2023-10-16T10:02:11.615181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:19.949318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625151+0000","last_change":"2023-10-16T10:02:09.617405+0000","last_active":"2023-10-16T10:02:10.625151+0000","last_peered":"2023-10-16T10:02:10.625151+0000","last_clean":"2023-10-16T10:02:10.625151+0000","last_became_active":"2023-10-16T10:02:09.617110+0000","last_became_peered":"2023-10-16T10:02:09.617110+0000","last_unstale":"2023-10-16T10:02:10.625151+0000","last_undegraded":"2023-10-16T10:02:10.625151+0000","last_fullsized":"2023-10-16T10:02:10.625151+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:01.452329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.635195+0000","last_change":"2023-10-16T10:02:10.632632+0000","last_active":"2023-10-16T10:02:10.635195+0000","last_peered":"2023-10-16T10:02:10.635195+0000","last_clean":"2023-10-16T10:02:10.635195+0000","last_became_active":"2023-10-16T10:02:10.632308+0000","last_became_peered":"2023-10-16T10:02:10.632308+0000","last_unstale":"2023-10-16T10:02:10.635195+0000","last_undegraded":"2023-10-16T10:02:10.635195+0000","last_fullsized":"2023-10-16T10:02:10.635195+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:12.483258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625257+0000","last_change":"2023-10-16T10:02:09.613112+0000","last_active":"2023-10-16T10:02:10.625257+0000","last_peered":"2023-10-16T10:02:10.625257+0000","last_clean":"2023-10-16T10:02:10.625257+0000","last_became_active":"2023-10-16T10:02:09.612781+0000","last_became_peered":"2023-10-16T10:02:09.612781+0000","last_unstale":"2023-10-16T10:02:10.625257+0000","last_undegraded":"2023-10-16T10:02:10.625257+0000","last_fullsized":"2023-10-16T10:02:10.625257+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:53.698172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615999+0000","last_change":"2023-10-16T10:02:10.627207+0000","last_active":"2023-10-16T10:02:11.615999+0000","last_peered":"2023-10-16T10:02:11.615999+0000","last_clean":"2023-10-16T10:02:11.615999+0000","last_became_active":"2023-10-16T10:02:10.626653+0000","last_became_peered":"2023-10-16T10:02:10.626653+0000","last_unstale":"2023-10-16T10:02:11.615999+0000","last_undegraded":"2023-10-16T10:02:11.615999+0000","last_fullsized":"2023-10-16T10:02:11.615999+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:09.650834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.626050+0000","last_change":"2023-10-16T10:02:09.618711+0000","last_active":"2023-10-16T10:02:10.626050+0000","last_peered":"2023-10-16T10:02:10.626050+0000","last_clean":"2023-10-16T10:02:10.626050+0000","last_became_active":"2023-10-16T10:02:09.617471+0000","last_became_peered":"2023-10-16T10:02:09.617471+0000","last_unstale":"2023-10-16T10:02:10.626050+0000","last_undegraded":"2023-10-16T10:02:10.626050+0000","last_fullsized":"2023-10-16T10:02:10.626050+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:16.095034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.627836+0000","last_change":"2023-10-16T10:02:10.626946+0000","last_active":"2023-10-16T10:02:10.627836+0000","last_peered":"2023-10-16T10:02:10.627836+0000","last_clean":"2023-10-16T10:02:10.627836+0000","last_became_active":"2023-10-16T10:02:10.626666+0000","last_became_peered":"2023-10-16T10:02:10.626666+0000","last_unstale":"2023-10-16T10:02:10.627836+0000","last_undegraded":"2023-10-16T10:02:10.627836+0000","last_fullsized":"2023-10-16T10:02:10.627836+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:30.329398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625144+0000","last_change":"2023-10-16T10:02:09.617146+0000","last_active":"2023-10-16T10:02:10.625144+0000","last_peered":"2023-10-16T10:02:10.625144+0000","last_clean":"2023-10-16T10:02:10.625144+0000","last_became_active":"2023-10-16T10:02:09.616879+0000","last_became_peered":"2023-10-16T10:02:09.616879+0000","last_unstale":"2023-10-16T10:02:10.625144+0000","last_undegraded":"2023-10-16T10:02:10.625144+0000","last_fullsized":"2023-10-16T10:02:10.625144+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:46.890745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615790+0000","last_change":"2023-10-16T10:02:10.626300+0000","last_active":"2023-10-16T10:02:11.615790+0000","last_peered":"2023-10-16T10:02:11.615790+0000","last_clean":"2023-10-16T10:02:11.615790+0000","last_became_active":"2023-10-16T10:02:10.626028+0000","last_became_peered":"2023-10-16T10:02:10.626028+0000","last_unstale":"2023-10-16T10:02:11.615790+0000","last_undegraded":"2023-10-16T10:02:11.615790+0000","last_fullsized":"2023-10-16T10:02:11.615790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:20.315841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375679+0000","last_change":"2023-10-16T10:02:09.616244+0000","last_active":"2023-10-16T10:02:12.375679+0000","last_peered":"2023-10-16T10:02:12.375679+0000","last_clean":"2023-10-16T10:02:12.375679+0000","last_became_active":"2023-10-16T10:02:09.615975+0000","last_became_peered":"2023-10-16T10:02:09.615975+0000","last_unstale":"2023-10-16T10:02:12.375679+0000","last_undegraded":"2023-10-16T10:02:12.375679+0000","last_fullsized":"2023-10-16T10:02:12.375679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:59.893985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637352+0000","last_change":"2023-10-16T10:02:10.632923+0000","last_active":"2023-10-16T10:02:10.637352+0000","last_peered":"2023-10-16T10:02:10.637352+0000","last_clean":"2023-10-16T10:02:10.637352+0000","last_became_active":"2023-10-16T10:02:10.632639+0000","last_became_peered":"2023-10-16T10:02:10.632639+0000","last_unstale":"2023-10-16T10:02:10.637352+0000","last_undegraded":"2023-10-16T10:02:10.637352+0000","last_fullsized":"2023-10-16T10:02:10.637352+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:59:59.196866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616022+0000","last_change":"2023-10-16T10:02:09.619788+0000","last_active":"2023-10-16T10:02:11.616022+0000","last_peered":"2023-10-16T10:02:11.616022+0000","last_clean":"2023-10-16T10:02:11.616022+0000","last_became_active":"2023-10-16T10:02:09.619400+0000","last_became_peered":"2023-10-16T10:02:09.619400+0000","last_unstale":"2023-10-16T10:02:11.616022+0000","last_undegraded":"2023-10-16T10:02:11.616022+0000","last_fullsized":"2023-10-16T10:02:11.616022+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:22.248303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616407+0000","last_change":"2023-10-16T10:02:10.640498+0000","last_active":"2023-10-16T10:02:11.616407+0000","last_peered":"2023-10-16T10:02:11.616407+0000","last_clean":"2023-10-16T10:02:11.616407+0000","last_became_active":"2023-10-16T10:02:10.640190+0000","last_became_peered":"2023-10-16T10:02:10.640190+0000","last_unstale":"2023-10-16T10:02:11.616407+0000","last_undegraded":"2023-10-16T10:02:11.616407+0000","last_fullsized":"2023-10-16T10:02:11.616407+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:05:00.513685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309625+0000","last_change":"2023-10-16T10:02:09.614931+0000","last_active":"2023-10-16T10:02:12.309625+0000","last_peered":"2023-10-16T10:02:12.309625+0000","last_clean":"2023-10-16T10:02:12.309625+0000","last_became_active":"2023-10-16T10:02:09.614701+0000","last_became_peered":"2023-10-16T10:02:09.614701+0000","last_unstale":"2023-10-16T10:02:12.309625+0000","last_undegraded":"2023-10-16T10:02:12.309625+0000","last_fullsized":"2023-10-16T10:02:12.309625+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:41:54.091049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"108'205","reported_seq":574,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.614336+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:02:10.614336+0000","last_peered":"2023-10-16T10:02:10.614336+0000","last_clean":"2023-10-16T10:02:10.614336+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:02:10.614336+0000","last_undegraded":"2023-10-16T10:02:10.614336+0000","last_fullsized":"2023-10-16T10:02:10.614336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542362,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2043904,"data_stored":1300632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.76500000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.27200000000000002},"max":{"1min":0.76000000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55200000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.57399999999999995},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.74399999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.71299999999999997,"5min":1.788,"15min":1.788},"last":0.749}]},{"osd":2,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.48099999999999998,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.68100000000000005,"5min":0.77000000000000002,"15min":0.80400000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.81200000000000006},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.60199999999999998,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.34399999999999997},"max":{"1min":0.72299999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47399999999999998,"15min":0.46899999999999997},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":0.68400000000000005,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57799999999999996}]},{"osd":5,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76200000000000001,"5min":1.006,"15min":1.006},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.38900000000000001,"15min":0.35599999999999998},"max":{"1min":0.72299999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.95199999999999996,"15min":1.373},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":1.4099999999999999},"last":0.50600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542361,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":710804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.371,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":0.69799999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.57899999999999996},"min":{"1min":0.438,"5min":0.33600000000000002,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.745,"15min":0.83399999999999996},"last":0.60999999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.72999999999999998,"5min":0.78900000000000003,"15min":0.79800000000000004},"last":0.625},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.58599999999999997},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.68999999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59399999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71399999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55400000000000005,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.70599999999999996,"5min":0.83699999999999997,"15min":0.84499999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":1.03,"15min":1.03},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56699999999999995,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.35499999999999998,"15min":0.34599999999999997},"max":{"1min":0.70299999999999996,"5min":0.85399999999999998,"15min":0.85899999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":1,"15min":1},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.85199999999999998,"5min":1.056,"15min":1.056},"last":0.67300000000000004}]},{"osd":7,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.83299999999999996,"5min":0.83499999999999996,"15min":0.86399999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66300000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542361,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50656,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321184,"statfs":{"total":96636764160,"available":96584892416,"internally_reserved":0,"allocated":1454080,"data_stored":716876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36899999999999999,"15min":0.373},"min":{"1min":0.26400000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46500000000000002,"5min":1.5,"15min":1.5},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.40999999999999998},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.58199999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.35399999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.26300000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58099999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.442},"min":{"1min":0.26900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.622,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.372}]},{"osd":3,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.309,"5min":0.253,"15min":0.253},"max":{"1min":0.64900000000000002,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.56299999999999994,"5min":1.379,"15min":1.379},"last":0.38900000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96899999999999997,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.49199999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61599999999999999,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94999999999999996,"5min":1.26,"15min":1.26},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.624,"15min":0.65100000000000002},"min":{"1min":0.45800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.623,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89500000000000002,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98299999999999998,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.51000000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542362,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2039808,"data_stored":1301102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.192,"15min":0.191},"max":{"1min":0.59999999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38200000000000001,"15min":0.39900000000000002},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.55700000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.40799999999999997},"min":{"1min":0.29199999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.495,"5min":0.61299999999999999,"15min":0.626},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.26300000000000001,"15min":0.26100000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.371}]},{"osd":3,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40899999999999997,"15min":0.434},"min":{"1min":0.27600000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64800000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.48199999999999998},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.69699999999999995},"last":0.42499999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.63200000000000001},"min":{"1min":0.41799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.89600000000000002,"5min":0.91500000000000004,"15min":0.98799999999999999},"last":0.622}]},{"osd":5,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.625,"15min":0.65000000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34200000000000003},"max":{"1min":0.89100000000000001,"5min":1.004,"15min":1.004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77800000000000002,"5min":0.88700000000000001,"15min":0.96999999999999997},"last":0.63900000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.63900000000000001},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.92400000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.46600000000000003,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":0.91900000000000004,"15min":1.0089999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63200000000000001,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.94499999999999995,"15min":1.0249999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.59899999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542361,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1458176,"data_stored":716910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.377},"min":{"1min":0.27100000000000002,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.52600000000000002,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.377},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53300000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.26300000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.40600000000000003},"min":{"1min":0.27300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.56100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.42399999999999999},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.64000000000000001},"last":0.33300000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.40799999999999997},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59199999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.315},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.46800000000000003},"min":{"1min":0.33400000000000002,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.625,"5min":0.747,"15min":0.747},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.61499999999999999},"min":{"1min":0.47299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.874,"5min":0.91600000000000004,"15min":0.93999999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":1.091,"15min":1.091},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.52200000000000002,"5min":0.39700000000000002,"15min":0.38200000000000001},"max":{"1min":1.01,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60799999999999998,"15min":0.63500000000000001},"min":{"1min":0.49299999999999999,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.54700000000000004,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86199999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.62,"15min":0.64800000000000002},"min":{"1min":0.505,"5min":0.37,"15min":0.37},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.97899999999999998},"last":0.57099999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.624,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.98799999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.623,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.998},"last":0.58199999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542362,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1458176,"data_stored":711786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38800000000000001},"min":{"1min":0.217,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.64900000000000002},"last":0.215},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.40200000000000002,"15min":0.41899999999999998},"min":{"1min":0.26200000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63100000000000001},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.379,"15min":0.39600000000000002},"min":{"1min":0.27000000000000002,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.497,"5min":0.66100000000000003,"15min":0.72199999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.28299999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63500000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.437},"min":{"1min":0.308,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.58599999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59899999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.61499999999999999},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95799999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.442,"5min":0.311,"15min":0.311},"max":{"1min":0.83099999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59799999999999998,"15min":0.625},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.98699999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.996},"last":0.57499999999999996}]},{"osd":6,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.499},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60399999999999998,"15min":0.63500000000000001},"min":{"1min":0.432,"5min":0.373,"15min":0.373},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.99099999999999999},"last":0.47999999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.64400000000000002},"min":{"1min":0.41299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":1.3,"15min":1.3},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64600000000000002,"15min":0.66300000000000003},"min":{"1min":0.47599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91800000000000004,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.54500000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542361,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1462272,"data_stored":711888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.89400000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73499999999999999,"5min":0.76800000000000002,"15min":0.81299999999999994},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.378,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71399999999999997,"5min":0.84599999999999997,"15min":0.86599999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.57699999999999996},"min":{"1min":0.44900000000000001,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.83799999999999997},"last":0.56100000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.58099999999999996},"min":{"1min":0.40400000000000003,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.71199999999999997,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.60099999999999998},"min":{"1min":0.36899999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68400000000000005,"5min":0.72799999999999998,"15min":0.89100000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70199999999999996,"5min":0.83199999999999996,"15min":0.86499999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.441,"5min":0.378,"15min":0.378},"max":{"1min":0.755,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.5,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.36399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.80400000000000005},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.82999999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77600000000000002,"5min":0.80100000000000005,"15min":0.85899999999999999},"last":0.63}]},{"osd":7,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.746,"5min":0.78600000000000003,"15min":0.80700000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.84599999999999997},"last":0.64300000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542362,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1458176,"data_stored":716946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.253},"max":{"1min":0.79600000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.27700000000000002},"max":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.91800000000000004},"last":0.45200000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.82799999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.307},"max":{"1min":0.80300000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.72099999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.33900000000000002},"max":{"1min":0.70199999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76100000000000001,"5min":0.88600000000000001,"15min":0.89000000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.624}]},{"osd":6,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70899999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.433,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71699999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:02:14.738 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:02:14.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.890+0000 7f84e3d96700 1 Processor -- start 2023-10-16T10:02:14.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.891+0000 7f84e3d96700 1 -- start start 2023-10-16T10:02:14.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.891+0000 7f84e3d96700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f84dc0cdfb0 0x7f84dc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.891+0000 7f84e3d96700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc0cabd0 con 0x7f84dc0cc0c0 2023-10-16T10:02:14.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.891+0000 7f84e3d96700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc0ca5c0 con 0x7f84dc06cdf0 2023-10-16T10:02:14.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.891+0000 7f84e3d96700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84dc066740 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.891+0000 7f84e1b32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f84dc0cdfb0 0x7f84dc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e1331700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f84dc0cc0c0 0x7f84dc15ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57154/0 (socket says 172.21.15.181:57154) 2023-10-16T10:02:14.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e1331700 1 -- 172.21.15.181:0/2387225648 learned_addr learned my addr 172.21.15.181:0/2387225648 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:14.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e1b32700 1 -- 172.21.15.181:0/2387225648 >> v1:172.21.15.190:6789/0 conn(0x7f84dc06cdf0 legacy=0x7f84dc0cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:14.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e1b32700 1 -- 172.21.15.181:0/2387225648 >> v1:172.21.15.181:6789/0 conn(0x7f84dc0cc0c0 legacy=0x7f84dc15ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e0b30700 1 -- 172.21.15.181:0/2387225648 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100225291 0 0) 0x7f84dc0cabd0 con 0x7f84dc0cc0c0 2023-10-16T10:02:14.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e1b32700 1 -- 172.21.15.181:0/2387225648 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc066a20 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e1b32700 1 --2- 172.21.15.181:0/2387225648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc15d8b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f84d800bae0 tx=0x7f84d800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b082913b56122ff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e0b30700 1 -- 172.21.15.181:0/2387225648 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d800dcf0 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e0b30700 1 -- 172.21.15.181:0/2387225648 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d800de50 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.892+0000 7f84e0b30700 1 -- 172.21.15.181:0/2387225648 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8004250 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.893+0000 7f84e3d96700 1 -- 172.21.15.181:0/2387225648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 msgr2=0x7f84dc15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.893+0000 7f84e3d96700 1 --2- 172.21.15.181:0/2387225648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc15d8b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f84d800bae0 tx=0x7f84d800bdf0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.893+0000 7f84e3d96700 1 -- 172.21.15.181:0/2387225648 shutdown_connections 2023-10-16T10:02:14.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.893+0000 7f84e3d96700 1 --2- 172.21.15.181:0/2387225648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc15d8b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.893+0000 7f84e3d96700 1 -- 172.21.15.181:0/2387225648 >> 172.21.15.181:0/2387225648 conn(0x7f84dc0b1780 msgr2=0x7f84dc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:14.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.893+0000 7f84e3d96700 1 -- 172.21.15.181:0/2387225648 shutdown_connections 2023-10-16T10:02:14.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.893+0000 7f84e3d96700 1 -- 172.21.15.181:0/2387225648 wait complete. 2023-10-16T10:02:14.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.894+0000 7f84e3d96700 1 Processor -- start 2023-10-16T10:02:14.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.894+0000 7f84e3d96700 1 -- start start 2023-10-16T10:02:14.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84dc06cdf0 0x7f84dc0b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84dc0cc0c0 0x7f84dc159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc0c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc0ad650 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc159ea0 con 0x7f84dc06cdf0 2023-10-16T10:02:14.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc0c4d70 con 0x7f84dc0cc0c0 2023-10-16T10:02:14.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc0c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc0c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46084/0 (socket says 172.21.15.181:46084) 2023-10-16T10:02:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 -- 172.21.15.181:0/1379695993 learned_addr learned my addr 172.21.15.181:0/1379695993 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e1331700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84dc0cc0c0 0x7f84dc159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e1b32700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84dc06cdf0 0x7f84dc0b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 -- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84dc0cc0c0 msgr2=0x7f84dc159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84dc0cc0c0 0x7f84dc159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 -- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84dc06cdf0 msgr2=0x7f84dc0b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:14.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84dc06cdf0 0x7f84dc0b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:14.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 -- 172.21.15.181:0/1379695993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d800b740 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e2333700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc0c4430 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f84d400b970 tx=0x7f84d400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84d27fc700 1 -- 172.21.15.181:0/1379695993 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d400c9b0 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84d27fc700 1 -- 172.21.15.181:0/1379695993 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d400cb10 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc0c5050 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84d27fc700 1 -- 172.21.15.181:0/1379695993 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4014440 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.895+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc0c5610 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.896+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc0b2f10 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.897+0000 7f84d27fc700 1 -- 172.21.15.181:0/1379695993 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f84d401d3b0 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.897+0000 7f84d27fc700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f84c006b020 0x7f84c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:14.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.897+0000 7f84d27fc700 1 -- 172.21.15.181:0/1379695993 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f84d4089500 con 0x7f84dc0cdfb0 2023-10-16T10:02:14.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.897+0000 7f84e1b32700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f84c006b020 0x7f84c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:14.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.898+0000 7f84e1b32700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f84c006b020 0x7f84c006d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84d800b2c0 tx=0x7f84d801b040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:14.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:14.902+0000 7f84d27fc700 1 -- 172.21.15.181:0/1379695993 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f84d4054380 con 0x7f84dc0cdfb0 2023-10-16T10:02:15.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.090+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 --> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84dc067d20 con 0x7f84c006b020 2023-10-16T10:02:15.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.098+0000 7f84d27fc700 1 -- 172.21.15.181:0/1379695993 <== mgr.7779 v2:172.21.15.181:6832/2093458419 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289993 (secure 0 0 0) 0x7f84dc067d20 con 0x7f84c006b020 2023-10-16T10:02:15.104 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:15.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.101+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f84c006b020 msgr2=0x7f84c006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.101+0000 7f84e3d96700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f84c006b020 0x7f84c006d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84d800b2c0 tx=0x7f84d801b040 comp rx=0 tx=0).stop 2023-10-16T10:02:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.101+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 msgr2=0x7f84dc0c4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.101+0000 7f84e3d96700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc0c4430 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f84d400b970 tx=0x7f84d400bd30 comp rx=0 tx=0).stop 2023-10-16T10:02:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.101+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 shutdown_connections 2023-10-16T10:02:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.101+0000 7f84e3d96700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f84c006b020 0x7f84c006d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.102+0000 7f84e3d96700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84dc0cdfb0 0x7f84dc0c4430 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.102+0000 7f84e3d96700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84dc06cdf0 0x7f84dc0b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.102+0000 7f84e3d96700 1 --2- 172.21.15.181:0/1379695993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84dc0cc0c0 0x7f84dc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.102+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 >> 172.21.15.181:0/1379695993 conn(0x7f84dc0b1780 msgr2=0x7f84dc0c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:15.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.102+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 shutdown_connections 2023-10-16T10:02:15.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.102+0000 7f84e3d96700 1 -- 172.21.15.181:0/1379695993 wait complete. 2023-10-16T10:02:15.110 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:02:15.115 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:02:14.557536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314736,"kb_used_data":12524,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754659984,"statfs":{"total":773094113280,"available":772771823616,"internally_reserved":0,"allocated":12824576,"data_stored":6886944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.959045"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615579+0000","last_change":"2023-10-16T10:02:10.635063+0000","last_active":"2023-10-16T10:02:11.615579+0000","last_peered":"2023-10-16T10:02:11.615579+0000","last_clean":"2023-10-16T10:02:11.615579+0000","last_became_active":"2023-10-16T10:02:10.630042+0000","last_became_peered":"2023-10-16T10:02:10.630042+0000","last_unstale":"2023-10-16T10:02:11.615579+0000","last_undegraded":"2023-10-16T10:02:11.615579+0000","last_fullsized":"2023-10-16T10:02:11.615579+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:37:23.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644619+0000","last_change":"2023-10-16T10:02:10.633784+0000","last_active":"2023-10-16T10:02:10.644619+0000","last_peered":"2023-10-16T10:02:10.644619+0000","last_clean":"2023-10-16T10:02:10.644619+0000","last_became_active":"2023-10-16T10:02:10.633568+0000","last_became_peered":"2023-10-16T10:02:10.633568+0000","last_unstale":"2023-10-16T10:02:10.644619+0000","last_undegraded":"2023-10-16T10:02:10.644619+0000","last_fullsized":"2023-10-16T10:02:10.644619+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:20.315841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615097+0000","last_change":"2023-10-16T10:02:10.634711+0000","last_active":"2023-10-16T10:02:11.615097+0000","last_peered":"2023-10-16T10:02:11.615097+0000","last_clean":"2023-10-16T10:02:11.615097+0000","last_became_active":"2023-10-16T10:02:10.634187+0000","last_became_peered":"2023-10-16T10:02:10.634187+0000","last_unstale":"2023-10-16T10:02:11.615097+0000","last_undegraded":"2023-10-16T10:02:11.615097+0000","last_fullsized":"2023-10-16T10:02:11.615097+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:03.479857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375696+0000","last_change":"2023-10-16T10:02:10.629312+0000","last_active":"2023-10-16T10:02:12.375696+0000","last_peered":"2023-10-16T10:02:12.375696+0000","last_clean":"2023-10-16T10:02:12.375696+0000","last_became_active":"2023-10-16T10:02:10.629052+0000","last_became_peered":"2023-10-16T10:02:10.629052+0000","last_unstale":"2023-10-16T10:02:12.375696+0000","last_undegraded":"2023-10-16T10:02:12.375696+0000","last_fullsized":"2023-10-16T10:02:12.375696+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:00:31.783543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637745+0000","last_change":"2023-10-16T10:02:10.634724+0000","last_active":"2023-10-16T10:02:10.637745+0000","last_peered":"2023-10-16T10:02:10.637745+0000","last_clean":"2023-10-16T10:02:10.637745+0000","last_became_active":"2023-10-16T10:02:10.633039+0000","last_became_peered":"2023-10-16T10:02:10.633039+0000","last_unstale":"2023-10-16T10:02:10.637745+0000","last_undegraded":"2023-10-16T10:02:10.637745+0000","last_fullsized":"2023-10-16T10:02:10.637745+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:53.103175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309414+0000","last_change":"2023-10-16T10:02:10.616124+0000","last_active":"2023-10-16T10:02:12.309414+0000","last_peered":"2023-10-16T10:02:12.309414+0000","last_clean":"2023-10-16T10:02:12.309414+0000","last_became_active":"2023-10-16T10:02:10.615721+0000","last_became_peered":"2023-10-16T10:02:10.615721+0000","last_unstale":"2023-10-16T10:02:12.309414+0000","last_undegraded":"2023-10-16T10:02:12.309414+0000","last_fullsized":"2023-10-16T10:02:12.309414+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:00.899263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309423+0000","last_change":"2023-10-16T10:02:10.616359+0000","last_active":"2023-10-16T10:02:12.309423+0000","last_peered":"2023-10-16T10:02:12.309423+0000","last_clean":"2023-10-16T10:02:12.309423+0000","last_became_active":"2023-10-16T10:02:10.616094+0000","last_became_peered":"2023-10-16T10:02:10.616094+0000","last_unstale":"2023-10-16T10:02:12.309423+0000","last_undegraded":"2023-10-16T10:02:12.309423+0000","last_fullsized":"2023-10-16T10:02:12.309423+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:12.065635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.638991+0000","last_change":"2023-10-16T10:02:10.632204+0000","last_active":"2023-10-16T10:02:10.638991+0000","last_peered":"2023-10-16T10:02:10.638991+0000","last_clean":"2023-10-16T10:02:10.638991+0000","last_became_active":"2023-10-16T10:02:10.631861+0000","last_became_peered":"2023-10-16T10:02:10.631861+0000","last_unstale":"2023-10-16T10:02:10.638991+0000","last_undegraded":"2023-10-16T10:02:10.638991+0000","last_fullsized":"2023-10-16T10:02:10.638991+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:59:59.196866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637198+0000","last_change":"2023-10-16T10:02:10.633439+0000","last_active":"2023-10-16T10:02:10.637198+0000","last_peered":"2023-10-16T10:02:10.637198+0000","last_clean":"2023-10-16T10:02:10.637198+0000","last_became_active":"2023-10-16T10:02:10.632982+0000","last_became_peered":"2023-10-16T10:02:10.632982+0000","last_unstale":"2023-10-16T10:02:10.637198+0000","last_undegraded":"2023-10-16T10:02:10.637198+0000","last_fullsized":"2023-10-16T10:02:10.637198+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:37.553361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637799+0000","last_change":"2023-10-16T10:02:10.635377+0000","last_active":"2023-10-16T10:02:10.637799+0000","last_peered":"2023-10-16T10:02:10.637799+0000","last_clean":"2023-10-16T10:02:10.637799+0000","last_became_active":"2023-10-16T10:02:10.634910+0000","last_became_peered":"2023-10-16T10:02:10.634910+0000","last_unstale":"2023-10-16T10:02:10.637799+0000","last_undegraded":"2023-10-16T10:02:10.637799+0000","last_fullsized":"2023-10-16T10:02:10.637799+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:20.315841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309639+0000","last_change":"2023-10-16T10:02:10.624373+0000","last_active":"2023-10-16T10:02:12.309639+0000","last_peered":"2023-10-16T10:02:12.309639+0000","last_clean":"2023-10-16T10:02:12.309639+0000","last_became_active":"2023-10-16T10:02:10.623772+0000","last_became_peered":"2023-10-16T10:02:10.623772+0000","last_unstale":"2023-10-16T10:02:12.309639+0000","last_undegraded":"2023-10-16T10:02:12.309639+0000","last_fullsized":"2023-10-16T10:02:12.309639+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:58.074294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637127+0000","last_change":"2023-10-16T10:02:10.634130+0000","last_active":"2023-10-16T10:02:10.637127+0000","last_peered":"2023-10-16T10:02:10.637127+0000","last_clean":"2023-10-16T10:02:10.637127+0000","last_became_active":"2023-10-16T10:02:10.633788+0000","last_became_peered":"2023-10-16T10:02:10.633788+0000","last_unstale":"2023-10-16T10:02:10.637127+0000","last_undegraded":"2023-10-16T10:02:10.637127+0000","last_fullsized":"2023-10-16T10:02:10.637127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:46.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375644+0000","last_change":"2023-10-16T10:02:10.619252+0000","last_active":"2023-10-16T10:02:12.375644+0000","last_peered":"2023-10-16T10:02:12.375644+0000","last_clean":"2023-10-16T10:02:12.375644+0000","last_became_active":"2023-10-16T10:02:10.618992+0000","last_became_peered":"2023-10-16T10:02:10.618992+0000","last_unstale":"2023-10-16T10:02:12.375644+0000","last_undegraded":"2023-10-16T10:02:12.375644+0000","last_fullsized":"2023-10-16T10:02:12.375644+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:13:46.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615889+0000","last_change":"2023-10-16T10:02:10.627401+0000","last_active":"2023-10-16T10:02:11.615889+0000","last_peered":"2023-10-16T10:02:11.615889+0000","last_clean":"2023-10-16T10:02:11.615889+0000","last_became_active":"2023-10-16T10:02:10.626944+0000","last_became_peered":"2023-10-16T10:02:10.626944+0000","last_unstale":"2023-10-16T10:02:11.615889+0000","last_undegraded":"2023-10-16T10:02:11.615889+0000","last_fullsized":"2023-10-16T10:02:11.615889+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:02.457607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375411+0000","last_change":"2023-10-16T10:02:10.631746+0000","last_active":"2023-10-16T10:02:12.375411+0000","last_peered":"2023-10-16T10:02:12.375411+0000","last_clean":"2023-10-16T10:02:12.375411+0000","last_became_active":"2023-10-16T10:02:10.631476+0000","last_became_peered":"2023-10-16T10:02:10.631476+0000","last_unstale":"2023-10-16T10:02:12.375411+0000","last_undegraded":"2023-10-16T10:02:12.375411+0000","last_fullsized":"2023-10-16T10:02:12.375411+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:19:29.735131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309492+0000","last_change":"2023-10-16T10:02:10.635901+0000","last_active":"2023-10-16T10:02:12.309492+0000","last_peered":"2023-10-16T10:02:12.309492+0000","last_clean":"2023-10-16T10:02:12.309492+0000","last_became_active":"2023-10-16T10:02:10.635321+0000","last_became_peered":"2023-10-16T10:02:10.635321+0000","last_unstale":"2023-10-16T10:02:12.309492+0000","last_undegraded":"2023-10-16T10:02:12.309492+0000","last_fullsized":"2023-10-16T10:02:12.309492+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:33.810501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644645+0000","last_change":"2023-10-16T10:02:10.632785+0000","last_active":"2023-10-16T10:02:10.644645+0000","last_peered":"2023-10-16T10:02:10.644645+0000","last_clean":"2023-10-16T10:02:10.644645+0000","last_became_active":"2023-10-16T10:02:10.632548+0000","last_became_peered":"2023-10-16T10:02:10.632548+0000","last_unstale":"2023-10-16T10:02:10.644645+0000","last_undegraded":"2023-10-16T10:02:10.644645+0000","last_fullsized":"2023-10-16T10:02:10.644645+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:41:23.405018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375710+0000","last_change":"2023-10-16T10:02:10.628528+0000","last_active":"2023-10-16T10:02:12.375710+0000","last_peered":"2023-10-16T10:02:12.375710+0000","last_clean":"2023-10-16T10:02:12.375710+0000","last_became_active":"2023-10-16T10:02:10.628274+0000","last_became_peered":"2023-10-16T10:02:10.628274+0000","last_unstale":"2023-10-16T10:02:12.375710+0000","last_undegraded":"2023-10-16T10:02:12.375710+0000","last_fullsized":"2023-10-16T10:02:12.375710+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:33:20.588202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616063+0000","last_change":"2023-10-16T10:02:10.640628+0000","last_active":"2023-10-16T10:02:11.616063+0000","last_peered":"2023-10-16T10:02:11.616063+0000","last_clean":"2023-10-16T10:02:11.616063+0000","last_became_active":"2023-10-16T10:02:10.636483+0000","last_became_peered":"2023-10-16T10:02:10.636483+0000","last_unstale":"2023-10-16T10:02:11.616063+0000","last_undegraded":"2023-10-16T10:02:11.616063+0000","last_fullsized":"2023-10-16T10:02:11.616063+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:15.644973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637064+0000","last_change":"2023-10-16T10:02:10.633359+0000","last_active":"2023-10-16T10:02:10.637064+0000","last_peered":"2023-10-16T10:02:10.637064+0000","last_clean":"2023-10-16T10:02:10.637064+0000","last_became_active":"2023-10-16T10:02:10.632889+0000","last_became_peered":"2023-10-16T10:02:10.632889+0000","last_unstale":"2023-10-16T10:02:10.637064+0000","last_undegraded":"2023-10-16T10:02:10.637064+0000","last_fullsized":"2023-10-16T10:02:10.637064+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:31.689737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616334+0000","last_change":"2023-10-16T10:02:10.631229+0000","last_active":"2023-10-16T10:02:11.616334+0000","last_peered":"2023-10-16T10:02:11.616334+0000","last_clean":"2023-10-16T10:02:11.616334+0000","last_became_active":"2023-10-16T10:02:10.630748+0000","last_became_peered":"2023-10-16T10:02:10.630748+0000","last_unstale":"2023-10-16T10:02:11.616334+0000","last_undegraded":"2023-10-16T10:02:11.616334+0000","last_fullsized":"2023-10-16T10:02:11.616334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:21.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644637+0000","last_change":"2023-10-16T10:02:10.632557+0000","last_active":"2023-10-16T10:02:10.644637+0000","last_peered":"2023-10-16T10:02:10.644637+0000","last_clean":"2023-10-16T10:02:10.644637+0000","last_became_active":"2023-10-16T10:02:10.632257+0000","last_became_peered":"2023-10-16T10:02:10.632257+0000","last_unstale":"2023-10-16T10:02:10.644637+0000","last_undegraded":"2023-10-16T10:02:10.644637+0000","last_fullsized":"2023-10-16T10:02:10.644637+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:04.618597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644524+0000","last_change":"2023-10-16T10:02:10.632197+0000","last_active":"2023-10-16T10:02:10.644524+0000","last_peered":"2023-10-16T10:02:10.644524+0000","last_clean":"2023-10-16T10:02:10.644524+0000","last_became_active":"2023-10-16T10:02:10.631836+0000","last_became_peered":"2023-10-16T10:02:10.631836+0000","last_unstale":"2023-10-16T10:02:10.644524+0000","last_undegraded":"2023-10-16T10:02:10.644524+0000","last_fullsized":"2023-10-16T10:02:10.644524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:58:19.116618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309586+0000","last_change":"2023-10-16T10:02:10.632198+0000","last_active":"2023-10-16T10:02:12.309586+0000","last_peered":"2023-10-16T10:02:12.309586+0000","last_clean":"2023-10-16T10:02:12.309586+0000","last_became_active":"2023-10-16T10:02:10.631915+0000","last_became_peered":"2023-10-16T10:02:10.631915+0000","last_unstale":"2023-10-16T10:02:12.309586+0000","last_undegraded":"2023-10-16T10:02:12.309586+0000","last_fullsized":"2023-10-16T10:02:12.309586+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:59.326925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637370+0000","last_change":"2023-10-16T10:02:10.632976+0000","last_active":"2023-10-16T10:02:10.637370+0000","last_peered":"2023-10-16T10:02:10.637370+0000","last_clean":"2023-10-16T10:02:10.637370+0000","last_became_active":"2023-10-16T10:02:10.632681+0000","last_became_peered":"2023-10-16T10:02:10.632681+0000","last_unstale":"2023-10-16T10:02:10.637370+0000","last_undegraded":"2023-10-16T10:02:10.637370+0000","last_fullsized":"2023-10-16T10:02:10.637370+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:47.426245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645126+0000","last_change":"2023-10-16T10:02:10.638584+0000","last_active":"2023-10-16T10:02:10.645126+0000","last_peered":"2023-10-16T10:02:10.645126+0000","last_clean":"2023-10-16T10:02:10.645126+0000","last_became_active":"2023-10-16T10:02:10.636903+0000","last_became_peered":"2023-10-16T10:02:10.636903+0000","last_unstale":"2023-10-16T10:02:10.645126+0000","last_undegraded":"2023-10-16T10:02:10.645126+0000","last_fullsized":"2023-10-16T10:02:10.645126+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:23.253580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309493+0000","last_change":"2023-10-16T10:02:10.627332+0000","last_active":"2023-10-16T10:02:12.309493+0000","last_peered":"2023-10-16T10:02:12.309493+0000","last_clean":"2023-10-16T10:02:12.309493+0000","last_became_active":"2023-10-16T10:02:10.626741+0000","last_became_peered":"2023-10-16T10:02:10.626741+0000","last_unstale":"2023-10-16T10:02:12.309493+0000","last_undegraded":"2023-10-16T10:02:12.309493+0000","last_fullsized":"2023-10-16T10:02:12.309493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:19:29.735131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637814+0000","last_change":"2023-10-16T10:02:10.635323+0000","last_active":"2023-10-16T10:02:10.637814+0000","last_peered":"2023-10-16T10:02:10.637814+0000","last_clean":"2023-10-16T10:02:10.637814+0000","last_became_active":"2023-10-16T10:02:10.632172+0000","last_became_peered":"2023-10-16T10:02:10.632172+0000","last_unstale":"2023-10-16T10:02:10.637814+0000","last_undegraded":"2023-10-16T10:02:10.637814+0000","last_fullsized":"2023-10-16T10:02:10.637814+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:18.531788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644946+0000","last_change":"2023-10-16T10:02:10.638753+0000","last_active":"2023-10-16T10:02:10.644946+0000","last_peered":"2023-10-16T10:02:10.644946+0000","last_clean":"2023-10-16T10:02:10.644946+0000","last_became_active":"2023-10-16T10:02:10.636894+0000","last_became_peered":"2023-10-16T10:02:10.636894+0000","last_unstale":"2023-10-16T10:02:10.644946+0000","last_undegraded":"2023-10-16T10:02:10.644946+0000","last_fullsized":"2023-10-16T10:02:10.644946+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:53.103175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615852+0000","last_change":"2023-10-16T10:02:10.633780+0000","last_active":"2023-10-16T10:02:11.615852+0000","last_peered":"2023-10-16T10:02:11.615852+0000","last_clean":"2023-10-16T10:02:11.615852+0000","last_became_active":"2023-10-16T10:02:10.633489+0000","last_became_peered":"2023-10-16T10:02:10.633489+0000","last_unstale":"2023-10-16T10:02:11.615852+0000","last_undegraded":"2023-10-16T10:02:11.615852+0000","last_fullsized":"2023-10-16T10:02:11.615852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:18.531788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375449+0000","last_change":"2023-10-16T10:02:10.627689+0000","last_active":"2023-10-16T10:02:12.375449+0000","last_peered":"2023-10-16T10:02:12.375449+0000","last_clean":"2023-10-16T10:02:12.375449+0000","last_became_active":"2023-10-16T10:02:10.627472+0000","last_became_peered":"2023-10-16T10:02:10.627472+0000","last_unstale":"2023-10-16T10:02:12.375449+0000","last_undegraded":"2023-10-16T10:02:12.375449+0000","last_fullsized":"2023-10-16T10:02:12.375449+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:39.553440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.614882+0000","last_change":"2023-10-16T10:02:10.639954+0000","last_active":"2023-10-16T10:02:11.614882+0000","last_peered":"2023-10-16T10:02:11.614882+0000","last_clean":"2023-10-16T10:02:11.614882+0000","last_became_active":"2023-10-16T10:02:10.639647+0000","last_became_peered":"2023-10-16T10:02:10.639647+0000","last_unstale":"2023-10-16T10:02:11.614882+0000","last_undegraded":"2023-10-16T10:02:11.614882+0000","last_fullsized":"2023-10-16T10:02:11.614882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:09:23.346970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616328+0000","last_change":"2023-10-16T10:02:10.626120+0000","last_active":"2023-10-16T10:02:11.616328+0000","last_peered":"2023-10-16T10:02:11.616328+0000","last_clean":"2023-10-16T10:02:11.616328+0000","last_became_active":"2023-10-16T10:02:10.625720+0000","last_became_peered":"2023-10-16T10:02:10.625720+0000","last_unstale":"2023-10-16T10:02:11.616328+0000","last_undegraded":"2023-10-16T10:02:11.616328+0000","last_fullsized":"2023-10-16T10:02:11.616328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:17.100311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"117'4","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645047+0000","last_change":"2023-10-16T10:02:09.619496+0000","last_active":"2023-10-16T10:02:10.645047+0000","last_peered":"2023-10-16T10:02:10.645047+0000","last_clean":"2023-10-16T10:02:10.645047+0000","last_became_active":"2023-10-16T10:02:09.619204+0000","last_became_peered":"2023-10-16T10:02:09.619204+0000","last_unstale":"2023-10-16T10:02:10.645047+0000","last_undegraded":"2023-10-16T10:02:10.645047+0000","last_fullsized":"2023-10-16T10:02:10.645047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:20.420135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375906+0000","last_change":"2023-10-16T10:02:10.627206+0000","last_active":"2023-10-16T10:02:12.375906+0000","last_peered":"2023-10-16T10:02:12.375906+0000","last_clean":"2023-10-16T10:02:12.375906+0000","last_became_active":"2023-10-16T10:02:10.626935+0000","last_became_peered":"2023-10-16T10:02:10.626935+0000","last_unstale":"2023-10-16T10:02:12.375906+0000","last_undegraded":"2023-10-16T10:02:12.375906+0000","last_fullsized":"2023-10-16T10:02:12.375906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:51.595635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309616+0000","last_change":"2023-10-16T10:02:09.617805+0000","last_active":"2023-10-16T10:02:12.309616+0000","last_peered":"2023-10-16T10:02:12.309616+0000","last_clean":"2023-10-16T10:02:12.309616+0000","last_became_active":"2023-10-16T10:02:09.617505+0000","last_became_peered":"2023-10-16T10:02:09.617505+0000","last_unstale":"2023-10-16T10:02:12.309616+0000","last_undegraded":"2023-10-16T10:02:12.309616+0000","last_fullsized":"2023-10-16T10:02:12.309616+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:22.521417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615619+0000","last_change":"2023-10-16T10:02:10.628986+0000","last_active":"2023-10-16T10:02:11.615619+0000","last_peered":"2023-10-16T10:02:11.615619+0000","last_clean":"2023-10-16T10:02:11.615619+0000","last_became_active":"2023-10-16T10:02:10.628481+0000","last_became_peered":"2023-10-16T10:02:10.628481+0000","last_unstale":"2023-10-16T10:02:11.615619+0000","last_undegraded":"2023-10-16T10:02:11.615619+0000","last_fullsized":"2023-10-16T10:02:11.615619+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:25:20.989352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645892+0000","last_change":"2023-10-16T10:02:09.618329+0000","last_active":"2023-10-16T10:02:10.645892+0000","last_peered":"2023-10-16T10:02:10.645892+0000","last_clean":"2023-10-16T10:02:10.645892+0000","last_became_active":"2023-10-16T10:02:09.617391+0000","last_became_peered":"2023-10-16T10:02:09.617391+0000","last_unstale":"2023-10-16T10:02:10.645892+0000","last_undegraded":"2023-10-16T10:02:10.645892+0000","last_fullsized":"2023-10-16T10:02:10.645892+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:17.526510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.627749+0000","last_change":"2023-10-16T10:02:10.626513+0000","last_active":"2023-10-16T10:02:10.627749+0000","last_peered":"2023-10-16T10:02:10.627749+0000","last_clean":"2023-10-16T10:02:10.627749+0000","last_became_active":"2023-10-16T10:02:10.626216+0000","last_became_peered":"2023-10-16T10:02:10.626216+0000","last_unstale":"2023-10-16T10:02:10.627749+0000","last_undegraded":"2023-10-16T10:02:10.627749+0000","last_fullsized":"2023-10-16T10:02:10.627749+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:46.485858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644789+0000","last_change":"2023-10-16T10:02:09.620447+0000","last_active":"2023-10-16T10:02:10.644789+0000","last_peered":"2023-10-16T10:02:10.644789+0000","last_clean":"2023-10-16T10:02:10.644789+0000","last_became_active":"2023-10-16T10:02:09.620142+0000","last_became_peered":"2023-10-16T10:02:09.620142+0000","last_unstale":"2023-10-16T10:02:10.644789+0000","last_undegraded":"2023-10-16T10:02:10.644789+0000","last_fullsized":"2023-10-16T10:02:10.644789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:46.890745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616347+0000","last_change":"2023-10-16T10:02:10.619758+0000","last_active":"2023-10-16T10:02:11.616347+0000","last_peered":"2023-10-16T10:02:11.616347+0000","last_clean":"2023-10-16T10:02:11.616347+0000","last_became_active":"2023-10-16T10:02:10.619437+0000","last_became_peered":"2023-10-16T10:02:10.619437+0000","last_unstale":"2023-10-16T10:02:11.616347+0000","last_undegraded":"2023-10-16T10:02:11.616347+0000","last_fullsized":"2023-10-16T10:02:11.616347+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:37.937239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615432+0000","last_change":"2023-10-16T10:02:09.618119+0000","last_active":"2023-10-16T10:02:11.615432+0000","last_peered":"2023-10-16T10:02:11.615432+0000","last_clean":"2023-10-16T10:02:11.615432+0000","last_became_active":"2023-10-16T10:02:09.617851+0000","last_became_peered":"2023-10-16T10:02:09.617851+0000","last_unstale":"2023-10-16T10:02:11.615432+0000","last_undegraded":"2023-10-16T10:02:11.615432+0000","last_fullsized":"2023-10-16T10:02:11.615432+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:33.775750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309505+0000","last_change":"2023-10-16T10:02:10.624892+0000","last_active":"2023-10-16T10:02:12.309505+0000","last_peered":"2023-10-16T10:02:12.309505+0000","last_clean":"2023-10-16T10:02:12.309505+0000","last_became_active":"2023-10-16T10:02:10.624574+0000","last_became_peered":"2023-10-16T10:02:10.624574+0000","last_unstale":"2023-10-16T10:02:12.309505+0000","last_undegraded":"2023-10-16T10:02:12.309505+0000","last_fullsized":"2023-10-16T10:02:12.309505+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:19.550899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"117'1","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644664+0000","last_change":"2023-10-16T10:02:09.617884+0000","last_active":"2023-10-16T10:02:10.644664+0000","last_peered":"2023-10-16T10:02:10.644664+0000","last_clean":"2023-10-16T10:02:10.644664+0000","last_became_active":"2023-10-16T10:02:09.617629+0000","last_became_peered":"2023-10-16T10:02:09.617629+0000","last_unstale":"2023-10-16T10:02:10.644664+0000","last_undegraded":"2023-10-16T10:02:10.644664+0000","last_fullsized":"2023-10-16T10:02:10.644664+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:14.639695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375448+0000","last_change":"2023-10-16T10:02:10.632282+0000","last_active":"2023-10-16T10:02:12.375448+0000","last_peered":"2023-10-16T10:02:12.375448+0000","last_clean":"2023-10-16T10:02:12.375448+0000","last_became_active":"2023-10-16T10:02:10.632023+0000","last_became_peered":"2023-10-16T10:02:10.632023+0000","last_unstale":"2023-10-16T10:02:12.375448+0000","last_undegraded":"2023-10-16T10:02:12.375448+0000","last_fullsized":"2023-10-16T10:02:12.375448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:59.326925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625352+0000","last_change":"2023-10-16T10:02:09.619478+0000","last_active":"2023-10-16T10:02:10.625352+0000","last_peered":"2023-10-16T10:02:10.625352+0000","last_clean":"2023-10-16T10:02:10.625352+0000","last_became_active":"2023-10-16T10:02:09.619200+0000","last_became_peered":"2023-10-16T10:02:09.619200+0000","last_unstale":"2023-10-16T10:02:10.625352+0000","last_undegraded":"2023-10-16T10:02:10.625352+0000","last_fullsized":"2023-10-16T10:02:10.625352+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:14.639695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.629071+0000","last_change":"2023-10-16T10:02:10.627183+0000","last_active":"2023-10-16T10:02:10.629071+0000","last_peered":"2023-10-16T10:02:10.629071+0000","last_clean":"2023-10-16T10:02:10.629071+0000","last_became_active":"2023-10-16T10:02:10.626898+0000","last_became_peered":"2023-10-16T10:02:10.626898+0000","last_unstale":"2023-10-16T10:02:10.629071+0000","last_undegraded":"2023-10-16T10:02:10.629071+0000","last_fullsized":"2023-10-16T10:02:10.629071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:08.659536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.614941+0000","last_change":"2023-10-16T10:02:09.616819+0000","last_active":"2023-10-16T10:02:11.614941+0000","last_peered":"2023-10-16T10:02:11.614941+0000","last_clean":"2023-10-16T10:02:11.614941+0000","last_became_active":"2023-10-16T10:02:09.616563+0000","last_became_peered":"2023-10-16T10:02:09.616563+0000","last_unstale":"2023-10-16T10:02:11.614941+0000","last_undegraded":"2023-10-16T10:02:11.614941+0000","last_fullsized":"2023-10-16T10:02:11.614941+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:59.508408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644606+0000","last_change":"2023-10-16T10:02:10.633698+0000","last_active":"2023-10-16T10:02:10.644606+0000","last_peered":"2023-10-16T10:02:10.644606+0000","last_clean":"2023-10-16T10:02:10.644606+0000","last_became_active":"2023-10-16T10:02:10.633437+0000","last_became_peered":"2023-10-16T10:02:10.633437+0000","last_unstale":"2023-10-16T10:02:10.644606+0000","last_undegraded":"2023-10-16T10:02:10.644606+0000","last_fullsized":"2023-10-16T10:02:10.644606+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:12.483258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615258+0000","last_change":"2023-10-16T10:02:09.614421+0000","last_active":"2023-10-16T10:02:11.615258+0000","last_peered":"2023-10-16T10:02:11.615258+0000","last_clean":"2023-10-16T10:02:11.615258+0000","last_became_active":"2023-10-16T10:02:09.614181+0000","last_became_peered":"2023-10-16T10:02:09.614181+0000","last_unstale":"2023-10-16T10:02:11.615258+0000","last_undegraded":"2023-10-16T10:02:11.615258+0000","last_fullsized":"2023-10-16T10:02:11.615258+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:13:20.420135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637028+0000","last_change":"2023-10-16T10:02:10.633257+0000","last_active":"2023-10-16T10:02:10.637028+0000","last_peered":"2023-10-16T10:02:10.637028+0000","last_clean":"2023-10-16T10:02:10.637028+0000","last_became_active":"2023-10-16T10:02:10.632567+0000","last_became_peered":"2023-10-16T10:02:10.632567+0000","last_unstale":"2023-10-16T10:02:10.637028+0000","last_undegraded":"2023-10-16T10:02:10.637028+0000","last_fullsized":"2023-10-16T10:02:10.637028+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:10:26.578599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375816+0000","last_change":"2023-10-16T10:02:09.619611+0000","last_active":"2023-10-16T10:02:12.375816+0000","last_peered":"2023-10-16T10:02:12.375816+0000","last_clean":"2023-10-16T10:02:12.375816+0000","last_became_active":"2023-10-16T10:02:09.619315+0000","last_became_peered":"2023-10-16T10:02:09.619315+0000","last_unstale":"2023-10-16T10:02:12.375816+0000","last_undegraded":"2023-10-16T10:02:12.375816+0000","last_fullsized":"2023-10-16T10:02:12.375816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:18.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637792+0000","last_change":"2023-10-16T10:02:10.634953+0000","last_active":"2023-10-16T10:02:10.637792+0000","last_peered":"2023-10-16T10:02:10.637792+0000","last_clean":"2023-10-16T10:02:10.637792+0000","last_became_active":"2023-10-16T10:02:10.633419+0000","last_became_peered":"2023-10-16T10:02:10.633419+0000","last_unstale":"2023-10-16T10:02:10.637792+0000","last_undegraded":"2023-10-16T10:02:10.637792+0000","last_fullsized":"2023-10-16T10:02:10.637792+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:02.457607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"117'3","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309722+0000","last_change":"2023-10-16T10:02:09.615778+0000","last_active":"2023-10-16T10:02:12.309722+0000","last_peered":"2023-10-16T10:02:12.309722+0000","last_clean":"2023-10-16T10:02:12.309722+0000","last_became_active":"2023-10-16T10:02:09.615343+0000","last_became_peered":"2023-10-16T10:02:09.615343+0000","last_unstale":"2023-10-16T10:02:12.309722+0000","last_undegraded":"2023-10-16T10:02:12.309722+0000","last_fullsized":"2023-10-16T10:02:12.309722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:01.175348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309622+0000","last_change":"2023-10-16T10:02:10.634831+0000","last_active":"2023-10-16T10:02:12.309622+0000","last_peered":"2023-10-16T10:02:12.309622+0000","last_clean":"2023-10-16T10:02:12.309622+0000","last_became_active":"2023-10-16T10:02:10.634468+0000","last_became_peered":"2023-10-16T10:02:10.634468+0000","last_unstale":"2023-10-16T10:02:12.309622+0000","last_undegraded":"2023-10-16T10:02:12.309622+0000","last_fullsized":"2023-10-16T10:02:12.309622+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:39.553440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"117'6","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644564+0000","last_change":"2023-10-16T10:02:09.613487+0000","last_active":"2023-10-16T10:02:10.644564+0000","last_peered":"2023-10-16T10:02:10.644564+0000","last_clean":"2023-10-16T10:02:10.644564+0000","last_became_active":"2023-10-16T10:02:09.613224+0000","last_became_peered":"2023-10-16T10:02:09.613224+0000","last_unstale":"2023-10-16T10:02:10.644564+0000","last_undegraded":"2023-10-16T10:02:10.644564+0000","last_fullsized":"2023-10-16T10:02:10.644564+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:18.051230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375586+0000","last_change":"2023-10-16T10:02:10.617193+0000","last_active":"2023-10-16T10:02:12.375586+0000","last_peered":"2023-10-16T10:02:12.375586+0000","last_clean":"2023-10-16T10:02:12.375586+0000","last_became_active":"2023-10-16T10:02:10.616921+0000","last_became_peered":"2023-10-16T10:02:10.616921+0000","last_unstale":"2023-10-16T10:02:12.375586+0000","last_undegraded":"2023-10-16T10:02:12.375586+0000","last_fullsized":"2023-10-16T10:02:12.375586+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:58.074294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616298+0000","last_change":"2023-10-16T10:02:09.612223+0000","last_active":"2023-10-16T10:02:11.616298+0000","last_peered":"2023-10-16T10:02:11.616298+0000","last_clean":"2023-10-16T10:02:11.616298+0000","last_became_active":"2023-10-16T10:02:09.611964+0000","last_became_peered":"2023-10-16T10:02:09.611964+0000","last_unstale":"2023-10-16T10:02:11.616298+0000","last_undegraded":"2023-10-16T10:02:11.616298+0000","last_fullsized":"2023-10-16T10:02:11.616298+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:52.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615564+0000","last_change":"2023-10-16T10:02:10.628712+0000","last_active":"2023-10-16T10:02:11.615564+0000","last_peered":"2023-10-16T10:02:11.615564+0000","last_clean":"2023-10-16T10:02:11.615564+0000","last_became_active":"2023-10-16T10:02:10.628438+0000","last_became_peered":"2023-10-16T10:02:10.628438+0000","last_unstale":"2023-10-16T10:02:11.615564+0000","last_undegraded":"2023-10-16T10:02:11.615564+0000","last_fullsized":"2023-10-16T10:02:11.615564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:31:32.976285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616080+0000","last_change":"2023-10-16T10:02:09.616584+0000","last_active":"2023-10-16T10:02:11.616080+0000","last_peered":"2023-10-16T10:02:11.616080+0000","last_clean":"2023-10-16T10:02:11.616080+0000","last_became_active":"2023-10-16T10:02:09.616291+0000","last_became_peered":"2023-10-16T10:02:09.616291+0000","last_unstale":"2023-10-16T10:02:11.616080+0000","last_undegraded":"2023-10-16T10:02:11.616080+0000","last_fullsized":"2023-10-16T10:02:11.616080+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:11.477980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.636962+0000","last_change":"2023-10-16T10:02:10.632497+0000","last_active":"2023-10-16T10:02:10.636962+0000","last_peered":"2023-10-16T10:02:10.636962+0000","last_clean":"2023-10-16T10:02:10.636962+0000","last_became_active":"2023-10-16T10:02:10.632194+0000","last_became_peered":"2023-10-16T10:02:10.632194+0000","last_unstale":"2023-10-16T10:02:10.636962+0000","last_undegraded":"2023-10-16T10:02:10.636962+0000","last_fullsized":"2023-10-16T10:02:10.636962+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:14.868185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.621611+0000","last_change":"2023-10-16T10:02:09.615236+0000","last_active":"2023-10-16T10:02:10.621611+0000","last_peered":"2023-10-16T10:02:10.621611+0000","last_clean":"2023-10-16T10:02:10.621611+0000","last_became_active":"2023-10-16T10:02:09.614973+0000","last_became_peered":"2023-10-16T10:02:09.614973+0000","last_unstale":"2023-10-16T10:02:10.621611+0000","last_undegraded":"2023-10-16T10:02:10.621611+0000","last_fullsized":"2023-10-16T10:02:10.621611+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:35.252905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615808+0000","last_change":"2023-10-16T10:02:10.625544+0000","last_active":"2023-10-16T10:02:11.615808+0000","last_peered":"2023-10-16T10:02:11.615808+0000","last_clean":"2023-10-16T10:02:11.615808+0000","last_became_active":"2023-10-16T10:02:10.625050+0000","last_became_peered":"2023-10-16T10:02:10.625050+0000","last_unstale":"2023-10-16T10:02:11.615808+0000","last_undegraded":"2023-10-16T10:02:11.615808+0000","last_fullsized":"2023-10-16T10:02:11.615808+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:47.426245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"117'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615036+0000","last_change":"2023-10-16T10:02:09.615479+0000","last_active":"2023-10-16T10:02:11.615036+0000","last_peered":"2023-10-16T10:02:11.615036+0000","last_clean":"2023-10-16T10:02:11.615036+0000","last_became_active":"2023-10-16T10:02:09.615154+0000","last_became_peered":"2023-10-16T10:02:09.615154+0000","last_unstale":"2023-10-16T10:02:11.615036+0000","last_undegraded":"2023-10-16T10:02:11.615036+0000","last_fullsized":"2023-10-16T10:02:11.615036+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:29:48.775082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.626964+0000","last_change":"2023-10-16T10:02:10.624108+0000","last_active":"2023-10-16T10:02:10.626964+0000","last_peered":"2023-10-16T10:02:10.626964+0000","last_clean":"2023-10-16T10:02:10.626964+0000","last_became_active":"2023-10-16T10:02:10.623839+0000","last_became_peered":"2023-10-16T10:02:10.623839+0000","last_unstale":"2023-10-16T10:02:10.626964+0000","last_undegraded":"2023-10-16T10:02:10.626964+0000","last_fullsized":"2023-10-16T10:02:10.626964+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:23.253580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.645803+0000","last_change":"2023-10-16T10:02:09.616793+0000","last_active":"2023-10-16T10:02:10.645803+0000","last_peered":"2023-10-16T10:02:10.645803+0000","last_clean":"2023-10-16T10:02:10.645803+0000","last_became_active":"2023-10-16T10:02:09.616506+0000","last_became_peered":"2023-10-16T10:02:09.616506+0000","last_unstale":"2023-10-16T10:02:10.645803+0000","last_undegraded":"2023-10-16T10:02:10.645803+0000","last_fullsized":"2023-10-16T10:02:10.645803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:05.085570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616501+0000","last_change":"2023-10-16T10:02:10.624019+0000","last_active":"2023-10-16T10:02:11.616501+0000","last_peered":"2023-10-16T10:02:11.616501+0000","last_clean":"2023-10-16T10:02:11.616501+0000","last_became_active":"2023-10-16T10:02:10.623719+0000","last_became_peered":"2023-10-16T10:02:10.623719+0000","last_unstale":"2023-10-16T10:02:11.616501+0000","last_undegraded":"2023-10-16T10:02:11.616501+0000","last_fullsized":"2023-10-16T10:02:11.616501+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:33:56.707067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"117'1","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637768+0000","last_change":"2023-10-16T10:02:09.612346+0000","last_active":"2023-10-16T10:02:10.637768+0000","last_peered":"2023-10-16T10:02:10.637768+0000","last_clean":"2023-10-16T10:02:10.637768+0000","last_became_active":"2023-10-16T10:02:09.609794+0000","last_became_peered":"2023-10-16T10:02:09.609794+0000","last_unstale":"2023-10-16T10:02:10.637768+0000","last_undegraded":"2023-10-16T10:02:10.637768+0000","last_fullsized":"2023-10-16T10:02:10.637768+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:08.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309780+0000","last_change":"2023-10-16T10:02:10.624552+0000","last_active":"2023-10-16T10:02:12.309780+0000","last_peered":"2023-10-16T10:02:12.309780+0000","last_clean":"2023-10-16T10:02:12.309780+0000","last_became_active":"2023-10-16T10:02:10.624113+0000","last_became_peered":"2023-10-16T10:02:10.624113+0000","last_unstale":"2023-10-16T10:02:12.309780+0000","last_undegraded":"2023-10-16T10:02:12.309780+0000","last_fullsized":"2023-10-16T10:02:12.309780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:13:46.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625326+0000","last_change":"2023-10-16T10:02:09.618634+0000","last_active":"2023-10-16T10:02:10.625326+0000","last_peered":"2023-10-16T10:02:10.625326+0000","last_clean":"2023-10-16T10:02:10.625326+0000","last_became_active":"2023-10-16T10:02:09.617097+0000","last_became_peered":"2023-10-16T10:02:09.617097+0000","last_unstale":"2023-10-16T10:02:10.625326+0000","last_undegraded":"2023-10-16T10:02:10.625326+0000","last_fullsized":"2023-10-16T10:02:10.625326+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:53.698172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615987+0000","last_change":"2023-10-16T10:02:10.642134+0000","last_active":"2023-10-16T10:02:11.615987+0000","last_peered":"2023-10-16T10:02:11.615987+0000","last_clean":"2023-10-16T10:02:11.615987+0000","last_became_active":"2023-10-16T10:02:10.641870+0000","last_became_peered":"2023-10-16T10:02:10.641870+0000","last_unstale":"2023-10-16T10:02:11.615987+0000","last_undegraded":"2023-10-16T10:02:11.615987+0000","last_fullsized":"2023-10-16T10:02:11.615987+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:54.703450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"117'2","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.648855+0000","last_change":"2023-10-16T10:02:09.620224+0000","last_active":"2023-10-16T10:02:10.648855+0000","last_peered":"2023-10-16T10:02:10.648855+0000","last_clean":"2023-10-16T10:02:10.648855+0000","last_became_active":"2023-10-16T10:02:09.619975+0000","last_became_peered":"2023-10-16T10:02:09.619975+0000","last_unstale":"2023-10-16T10:02:10.648855+0000","last_undegraded":"2023-10-16T10:02:10.648855+0000","last_fullsized":"2023-10-16T10:02:10.648855+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:20.471392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616218+0000","last_change":"2023-10-16T10:02:10.632929+0000","last_active":"2023-10-16T10:02:11.616218+0000","last_peered":"2023-10-16T10:02:11.616218+0000","last_clean":"2023-10-16T10:02:11.616218+0000","last_became_active":"2023-10-16T10:02:10.632633+0000","last_became_peered":"2023-10-16T10:02:10.632633+0000","last_unstale":"2023-10-16T10:02:11.616218+0000","last_undegraded":"2023-10-16T10:02:11.616218+0000","last_fullsized":"2023-10-16T10:02:11.616218+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:21.476670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616457+0000","last_change":"2023-10-16T10:02:09.616535+0000","last_active":"2023-10-16T10:02:11.616457+0000","last_peered":"2023-10-16T10:02:11.616457+0000","last_clean":"2023-10-16T10:02:11.616457+0000","last_became_active":"2023-10-16T10:02:09.616191+0000","last_became_peered":"2023-10-16T10:02:09.616191+0000","last_unstale":"2023-10-16T10:02:11.616457+0000","last_undegraded":"2023-10-16T10:02:11.616457+0000","last_fullsized":"2023-10-16T10:02:11.616457+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:58:14.335738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615932+0000","last_change":"2023-10-16T10:02:10.641130+0000","last_active":"2023-10-16T10:02:11.615932+0000","last_peered":"2023-10-16T10:02:11.615932+0000","last_clean":"2023-10-16T10:02:11.615932+0000","last_became_active":"2023-10-16T10:02:10.640872+0000","last_became_peered":"2023-10-16T10:02:10.640872+0000","last_unstale":"2023-10-16T10:02:11.615932+0000","last_undegraded":"2023-10-16T10:02:11.615932+0000","last_fullsized":"2023-10-16T10:02:11.615932+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:47.896023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616384+0000","last_change":"2023-10-16T10:02:09.608800+0000","last_active":"2023-10-16T10:02:11.616384+0000","last_peered":"2023-10-16T10:02:11.616384+0000","last_clean":"2023-10-16T10:02:11.616384+0000","last_became_active":"2023-10-16T10:02:09.608505+0000","last_became_peered":"2023-10-16T10:02:09.608505+0000","last_unstale":"2023-10-16T10:02:11.616384+0000","last_undegraded":"2023-10-16T10:02:11.616384+0000","last_fullsized":"2023-10-16T10:02:11.616384+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:21:30.634629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637850+0000","last_change":"2023-10-16T10:02:10.635709+0000","last_active":"2023-10-16T10:02:10.637850+0000","last_peered":"2023-10-16T10:02:10.637850+0000","last_clean":"2023-10-16T10:02:10.637850+0000","last_became_active":"2023-10-16T10:02:10.635408+0000","last_became_peered":"2023-10-16T10:02:10.635408+0000","last_unstale":"2023-10-16T10:02:10.637850+0000","last_undegraded":"2023-10-16T10:02:10.637850+0000","last_fullsized":"2023-10-16T10:02:10.637850+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:36.017941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.624937+0000","last_change":"2023-10-16T10:02:09.617864+0000","last_active":"2023-10-16T10:02:10.624937+0000","last_peered":"2023-10-16T10:02:10.624937+0000","last_clean":"2023-10-16T10:02:10.624937+0000","last_became_active":"2023-10-16T10:02:09.617054+0000","last_became_peered":"2023-10-16T10:02:09.617054+0000","last_unstale":"2023-10-16T10:02:10.624937+0000","last_undegraded":"2023-10-16T10:02:10.624937+0000","last_fullsized":"2023-10-16T10:02:10.624937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:46.420967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.644586+0000","last_change":"2023-10-16T10:02:10.633202+0000","last_active":"2023-10-16T10:02:10.644586+0000","last_peered":"2023-10-16T10:02:10.644586+0000","last_clean":"2023-10-16T10:02:10.644586+0000","last_became_active":"2023-10-16T10:02:10.632931+0000","last_became_peered":"2023-10-16T10:02:10.632931+0000","last_unstale":"2023-10-16T10:02:10.644586+0000","last_undegraded":"2023-10-16T10:02:10.644586+0000","last_fullsized":"2023-10-16T10:02:10.644586+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:36.017941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616617+0000","last_change":"2023-10-16T10:02:09.613235+0000","last_active":"2023-10-16T10:02:11.616617+0000","last_peered":"2023-10-16T10:02:11.616617+0000","last_clean":"2023-10-16T10:02:11.616617+0000","last_became_active":"2023-10-16T10:02:09.612868+0000","last_became_peered":"2023-10-16T10:02:09.612868+0000","last_unstale":"2023-10-16T10:02:11.616617+0000","last_undegraded":"2023-10-16T10:02:11.616617+0000","last_fullsized":"2023-10-16T10:02:11.616617+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:58:48.368824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615174+0000","last_change":"2023-10-16T10:02:10.630321+0000","last_active":"2023-10-16T10:02:11.615174+0000","last_peered":"2023-10-16T10:02:11.615174+0000","last_clean":"2023-10-16T10:02:11.615174+0000","last_became_active":"2023-10-16T10:02:10.629740+0000","last_became_peered":"2023-10-16T10:02:10.629740+0000","last_unstale":"2023-10-16T10:02:11.615174+0000","last_undegraded":"2023-10-16T10:02:11.615174+0000","last_fullsized":"2023-10-16T10:02:11.615174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:19:24.035481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625454+0000","last_change":"2023-10-16T10:02:09.620358+0000","last_active":"2023-10-16T10:02:10.625454+0000","last_peered":"2023-10-16T10:02:10.625454+0000","last_clean":"2023-10-16T10:02:10.625454+0000","last_became_active":"2023-10-16T10:02:09.620082+0000","last_became_peered":"2023-10-16T10:02:09.620082+0000","last_unstale":"2023-10-16T10:02:10.625454+0000","last_undegraded":"2023-10-16T10:02:10.625454+0000","last_fullsized":"2023-10-16T10:02:10.625454+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:08.645556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615181+0000","last_change":"2023-10-16T10:02:10.629172+0000","last_active":"2023-10-16T10:02:11.615181+0000","last_peered":"2023-10-16T10:02:11.615181+0000","last_clean":"2023-10-16T10:02:11.615181+0000","last_became_active":"2023-10-16T10:02:10.628736+0000","last_became_peered":"2023-10-16T10:02:10.628736+0000","last_unstale":"2023-10-16T10:02:11.615181+0000","last_undegraded":"2023-10-16T10:02:11.615181+0000","last_fullsized":"2023-10-16T10:02:11.615181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:19.949318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625151+0000","last_change":"2023-10-16T10:02:09.617405+0000","last_active":"2023-10-16T10:02:10.625151+0000","last_peered":"2023-10-16T10:02:10.625151+0000","last_clean":"2023-10-16T10:02:10.625151+0000","last_became_active":"2023-10-16T10:02:09.617110+0000","last_became_peered":"2023-10-16T10:02:09.617110+0000","last_unstale":"2023-10-16T10:02:10.625151+0000","last_undegraded":"2023-10-16T10:02:10.625151+0000","last_fullsized":"2023-10-16T10:02:10.625151+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:01.452329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.635195+0000","last_change":"2023-10-16T10:02:10.632632+0000","last_active":"2023-10-16T10:02:10.635195+0000","last_peered":"2023-10-16T10:02:10.635195+0000","last_clean":"2023-10-16T10:02:10.635195+0000","last_became_active":"2023-10-16T10:02:10.632308+0000","last_became_peered":"2023-10-16T10:02:10.632308+0000","last_unstale":"2023-10-16T10:02:10.635195+0000","last_undegraded":"2023-10-16T10:02:10.635195+0000","last_fullsized":"2023-10-16T10:02:10.635195+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:39:12.483258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625257+0000","last_change":"2023-10-16T10:02:09.613112+0000","last_active":"2023-10-16T10:02:10.625257+0000","last_peered":"2023-10-16T10:02:10.625257+0000","last_clean":"2023-10-16T10:02:10.625257+0000","last_became_active":"2023-10-16T10:02:09.612781+0000","last_became_peered":"2023-10-16T10:02:09.612781+0000","last_unstale":"2023-10-16T10:02:10.625257+0000","last_undegraded":"2023-10-16T10:02:10.625257+0000","last_fullsized":"2023-10-16T10:02:10.625257+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:53.698172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615999+0000","last_change":"2023-10-16T10:02:10.627207+0000","last_active":"2023-10-16T10:02:11.615999+0000","last_peered":"2023-10-16T10:02:11.615999+0000","last_clean":"2023-10-16T10:02:11.615999+0000","last_became_active":"2023-10-16T10:02:10.626653+0000","last_became_peered":"2023-10-16T10:02:10.626653+0000","last_unstale":"2023-10-16T10:02:11.615999+0000","last_undegraded":"2023-10-16T10:02:11.615999+0000","last_fullsized":"2023-10-16T10:02:11.615999+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:09.650834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.626050+0000","last_change":"2023-10-16T10:02:09.618711+0000","last_active":"2023-10-16T10:02:10.626050+0000","last_peered":"2023-10-16T10:02:10.626050+0000","last_clean":"2023-10-16T10:02:10.626050+0000","last_became_active":"2023-10-16T10:02:09.617471+0000","last_became_peered":"2023-10-16T10:02:09.617471+0000","last_unstale":"2023-10-16T10:02:10.626050+0000","last_undegraded":"2023-10-16T10:02:10.626050+0000","last_fullsized":"2023-10-16T10:02:10.626050+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:16.095034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.627836+0000","last_change":"2023-10-16T10:02:10.626946+0000","last_active":"2023-10-16T10:02:10.627836+0000","last_peered":"2023-10-16T10:02:10.627836+0000","last_clean":"2023-10-16T10:02:10.627836+0000","last_became_active":"2023-10-16T10:02:10.626666+0000","last_became_peered":"2023-10-16T10:02:10.626666+0000","last_unstale":"2023-10-16T10:02:10.627836+0000","last_undegraded":"2023-10-16T10:02:10.627836+0000","last_fullsized":"2023-10-16T10:02:10.627836+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:30.329398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.625144+0000","last_change":"2023-10-16T10:02:09.617146+0000","last_active":"2023-10-16T10:02:10.625144+0000","last_peered":"2023-10-16T10:02:10.625144+0000","last_clean":"2023-10-16T10:02:10.625144+0000","last_became_active":"2023-10-16T10:02:09.616879+0000","last_became_peered":"2023-10-16T10:02:09.616879+0000","last_unstale":"2023-10-16T10:02:10.625144+0000","last_undegraded":"2023-10-16T10:02:10.625144+0000","last_fullsized":"2023-10-16T10:02:10.625144+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:46.890745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.615790+0000","last_change":"2023-10-16T10:02:10.626300+0000","last_active":"2023-10-16T10:02:11.615790+0000","last_peered":"2023-10-16T10:02:11.615790+0000","last_clean":"2023-10-16T10:02:11.615790+0000","last_became_active":"2023-10-16T10:02:10.626028+0000","last_became_peered":"2023-10-16T10:02:10.626028+0000","last_unstale":"2023-10-16T10:02:11.615790+0000","last_undegraded":"2023-10-16T10:02:11.615790+0000","last_fullsized":"2023-10-16T10:02:11.615790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:20.315841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.375679+0000","last_change":"2023-10-16T10:02:09.616244+0000","last_active":"2023-10-16T10:02:12.375679+0000","last_peered":"2023-10-16T10:02:12.375679+0000","last_clean":"2023-10-16T10:02:12.375679+0000","last_became_active":"2023-10-16T10:02:09.615975+0000","last_became_peered":"2023-10-16T10:02:09.615975+0000","last_unstale":"2023-10-16T10:02:12.375679+0000","last_undegraded":"2023-10-16T10:02:12.375679+0000","last_fullsized":"2023-10-16T10:02:12.375679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:59.893985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.637352+0000","last_change":"2023-10-16T10:02:10.632923+0000","last_active":"2023-10-16T10:02:10.637352+0000","last_peered":"2023-10-16T10:02:10.637352+0000","last_clean":"2023-10-16T10:02:10.637352+0000","last_became_active":"2023-10-16T10:02:10.632639+0000","last_became_peered":"2023-10-16T10:02:10.632639+0000","last_unstale":"2023-10-16T10:02:10.637352+0000","last_undegraded":"2023-10-16T10:02:10.637352+0000","last_fullsized":"2023-10-16T10:02:10.637352+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:59:59.196866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616022+0000","last_change":"2023-10-16T10:02:09.619788+0000","last_active":"2023-10-16T10:02:11.616022+0000","last_peered":"2023-10-16T10:02:11.616022+0000","last_clean":"2023-10-16T10:02:11.616022+0000","last_became_active":"2023-10-16T10:02:09.619400+0000","last_became_peered":"2023-10-16T10:02:09.619400+0000","last_unstale":"2023-10-16T10:02:11.616022+0000","last_undegraded":"2023-10-16T10:02:11.616022+0000","last_fullsized":"2023-10-16T10:02:11.616022+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:22.248303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:11.616407+0000","last_change":"2023-10-16T10:02:10.640498+0000","last_active":"2023-10-16T10:02:11.616407+0000","last_peered":"2023-10-16T10:02:11.616407+0000","last_clean":"2023-10-16T10:02:11.616407+0000","last_became_active":"2023-10-16T10:02:10.640190+0000","last_became_peered":"2023-10-16T10:02:10.640190+0000","last_unstale":"2023-10-16T10:02:11.616407+0000","last_undegraded":"2023-10-16T10:02:11.616407+0000","last_fullsized":"2023-10-16T10:02:11.616407+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:09.598076+0000","last_clean_scrub_stamp":"2023-10-16T10:02:09.598076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:05:00.513685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-16T10:02:12.309625+0000","last_change":"2023-10-16T10:02:09.614931+0000","last_active":"2023-10-16T10:02:12.309625+0000","last_peered":"2023-10-16T10:02:12.309625+0000","last_clean":"2023-10-16T10:02:12.309625+0000","last_became_active":"2023-10-16T10:02:09.614701+0000","last_became_peered":"2023-10-16T10:02:09.614701+0000","last_unstale":"2023-10-16T10:02:12.309625+0000","last_undegraded":"2023-10-16T10:02:12.309625+0000","last_fullsized":"2023-10-16T10:02:12.309625+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:02:08.592799+0000","last_clean_scrub_stamp":"2023-10-16T10:02:08.592799+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:41:54.091049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"108'205","reported_seq":574,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-16T10:02:10.614336+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:02:10.614336+0000","last_peered":"2023-10-16T10:02:10.614336+0000","last_clean":"2023-10-16T10:02:10.614336+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:02:10.614336+0000","last_undegraded":"2023-10-16T10:02:10.614336+0000","last_fullsized":"2023-10-16T10:02:10.614336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542362,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2043904,"data_stored":1300632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.26100000000000001,"15min":0.248},"max":{"1min":0.76500000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.27200000000000002},"max":{"1min":0.76000000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55200000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.57399999999999995},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.29199999999999998},"max":{"1min":0.74399999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.71299999999999997,"5min":1.788,"15min":1.788},"last":0.749}]},{"osd":2,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.48099999999999998,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.68100000000000005,"5min":0.77000000000000002,"15min":0.80400000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.35499999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.81200000000000006},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.60199999999999998,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.34399999999999997},"max":{"1min":0.72299999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47399999999999998,"15min":0.46899999999999997},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59399999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":0.68400000000000005,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57799999999999996}]},{"osd":5,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76200000000000001,"5min":1.006,"15min":1.006},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.38900000000000001,"15min":0.35599999999999998},"max":{"1min":0.72299999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.745,"5min":0.95199999999999996,"15min":1.373},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":1.4099999999999999},"last":0.50600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542361,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":710804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.371,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":0.69799999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.57899999999999996},"min":{"1min":0.438,"5min":0.33600000000000002,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.745,"15min":0.83399999999999996},"last":0.60999999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.72999999999999998,"5min":0.78900000000000003,"15min":0.79800000000000004},"last":0.625},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.58599999999999997},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.68999999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59399999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55700000000000005,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71399999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55400000000000005,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.70599999999999996,"5min":0.83699999999999997,"15min":0.84499999999999997},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57999999999999996,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":1.03,"15min":1.03},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56699999999999995,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.35499999999999998,"15min":0.34599999999999997},"max":{"1min":0.70299999999999996,"5min":0.85399999999999998,"15min":0.85899999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":1,"15min":1},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.36299999999999999,"5min":0.35199999999999998,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.85199999999999998,"5min":1.056,"15min":1.056},"last":0.67300000000000004}]},{"osd":7,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.83299999999999996,"5min":0.83499999999999996,"15min":0.86399999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66300000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542361,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50656,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321184,"statfs":{"total":96636764160,"available":96584892416,"internally_reserved":0,"allocated":1454080,"data_stored":716876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36899999999999999,"15min":0.373},"min":{"1min":0.26400000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46500000000000002,"5min":1.5,"15min":1.5},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.40999999999999998},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.19600000000000001},"max":{"1min":0.58199999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.35399999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.26300000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58099999999999996,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.442},"min":{"1min":0.26900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.622,"5min":3.8730000000000002,"15min":3.8730000000000002},"last":0.372}]},{"osd":3,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.47499999999999998},"min":{"1min":0.309,"5min":0.253,"15min":0.253},"max":{"1min":0.64900000000000002,"5min":4.6740000000000004,"15min":4.6740000000000004},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.25700000000000001},"max":{"1min":0.56299999999999994,"5min":1.379,"15min":1.379},"last":0.38900000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.96899999999999997,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.318,"15min":0.318},"max":{"1min":0.91900000000000004,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.49199999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61599999999999999,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94999999999999996,"5min":1.26,"15min":1.26},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.624,"15min":0.65100000000000002},"min":{"1min":0.45800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.623,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89500000000000002,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98299999999999998,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.51000000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542362,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2039808,"data_stored":1301102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.192,"15min":0.191},"max":{"1min":0.59999999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38200000000000001,"15min":0.39900000000000002},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.55700000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.40799999999999997},"min":{"1min":0.29199999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.495,"5min":0.61299999999999999,"15min":0.626},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.26300000000000001,"15min":0.26100000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.371}]},{"osd":3,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40899999999999997,"15min":0.434},"min":{"1min":0.27600000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64800000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.48199999999999998},"min":{"1min":0.33400000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.69699999999999995},"last":0.42499999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.63200000000000001},"min":{"1min":0.41799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60899999999999999,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.372,"15min":0.36699999999999999},"max":{"1min":0.89600000000000002,"5min":0.91500000000000004,"15min":0.98799999999999999},"last":0.622}]},{"osd":5,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.625,"15min":0.65000000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34200000000000003},"max":{"1min":0.89100000000000001,"5min":1.004,"15min":1.004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77800000000000002,"5min":0.88700000000000001,"15min":0.96999999999999997},"last":0.63900000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.63900000000000001},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.92400000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.46600000000000003,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":0.91900000000000004,"15min":1.0089999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Mon Oct 16 10:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63200000000000001,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85299999999999998,"5min":0.94499999999999995,"15min":1.0249999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63300000000000001,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.95699999999999996},"last":0.59899999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542361,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1458176,"data_stored":716910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36599999999999999,"15min":0.377},"min":{"1min":0.27100000000000002,"5min":0.20799999999999999,"15min":0.183},"max":{"1min":0.52600000000000002,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.377},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.40200000000000002},"min":{"1min":0.27200000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.53300000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.26300000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.40600000000000003},"min":{"1min":0.27300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.56100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.42399999999999999},"min":{"1min":0.316,"5min":0.27800000000000002,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.64000000000000001},"last":0.33300000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.40799999999999997},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59199999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.315},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.46800000000000003},"min":{"1min":0.33400000000000002,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.625,"5min":0.747,"15min":0.747},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.61499999999999999},"min":{"1min":0.47299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.874,"5min":0.91600000000000004,"15min":0.93999999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":1.091,"15min":1.091},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.52200000000000002,"5min":0.39700000000000002,"15min":0.38200000000000001},"max":{"1min":1.01,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60799999999999998,"15min":0.63500000000000001},"min":{"1min":0.49299999999999999,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.54700000000000004,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86199999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.62,"15min":0.64800000000000002},"min":{"1min":0.505,"5min":0.37,"15min":0.37},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.97899999999999998},"last":0.57099999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.624,"15min":0.65200000000000002},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.98799999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.623,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.998},"last":0.58199999999999996}]}]},{"osd":3,"up_from":14,"seq":60129542362,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1458176,"data_stored":711786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38800000000000001},"min":{"1min":0.217,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.64900000000000002},"last":0.215},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.40200000000000002,"15min":0.41899999999999998},"min":{"1min":0.26200000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.56899999999999995,"5min":0.626,"15min":0.63100000000000001},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.379,"15min":0.39600000000000002},"min":{"1min":0.27000000000000002,"5min":0.23100000000000001,"15min":0.219},"max":{"1min":0.497,"5min":0.66100000000000003,"15min":0.72199999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.42299999999999999},"min":{"1min":0.28299999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63500000000000001,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.437},"min":{"1min":0.308,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.58599999999999997,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59899999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59499999999999997,"15min":0.61499999999999999},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95799999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.442,"5min":0.311,"15min":0.311},"max":{"1min":0.83099999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.44900000000000001}]},{"osd":5,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59799999999999998,"15min":0.625},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.98699999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.996},"last":0.57499999999999996}]},{"osd":6,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.96199999999999997,"15min":0.98399999999999999},"last":0.499},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60399999999999998,"15min":0.63500000000000001},"min":{"1min":0.432,"5min":0.373,"15min":0.373},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.99099999999999999},"last":0.47999999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.64400000000000002},"min":{"1min":0.41299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84899999999999998,"5min":1.3,"15min":1.3},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64600000000000002,"15min":0.66300000000000003},"min":{"1min":0.47599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91800000000000004,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.54500000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542361,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1462272,"data_stored":711888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.20100000000000001,"15min":0.186},"max":{"1min":0.68799999999999994,"5min":0.76900000000000002,"15min":0.89400000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73499999999999999,"5min":0.76800000000000002,"15min":0.81299999999999994},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.378,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71399999999999997,"5min":0.84599999999999997,"15min":0.86599999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.57699999999999996},"min":{"1min":0.44900000000000001,"5min":0.26700000000000002,"15min":0.248},"max":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.83799999999999997},"last":0.56100000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.58099999999999996},"min":{"1min":0.40400000000000003,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.71199999999999997,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.60099999999999998},"min":{"1min":0.36899999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68400000000000005,"5min":0.72799999999999998,"15min":0.89100000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70199999999999996,"5min":0.83199999999999996,"15min":0.86499999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.441,"5min":0.378,"15min":0.378},"max":{"1min":0.755,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.5,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.36399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.80400000000000005},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.82999999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77600000000000002,"5min":0.80100000000000005,"15min":0.85899999999999999},"last":0.63}]},{"osd":7,"last update":"Mon Oct 16 10:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.746,"5min":0.78600000000000003,"15min":0.80700000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57199999999999995,"15min":0.58999999999999997},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.84599999999999997},"last":0.64300000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542362,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1458176,"data_stored":716946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.253},"max":{"1min":0.79600000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.27700000000000002},"max":{"1min":0.78400000000000003,"5min":0.84199999999999997,"15min":0.91800000000000004},"last":0.45200000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55800000000000005,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73199999999999998,"5min":0.77900000000000003,"15min":0.82799999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.307},"max":{"1min":0.80300000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.72099999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.33900000000000002},"max":{"1min":0.70199999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.47599999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76100000000000001,"5min":0.88600000000000001,"15min":0.89000000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.52300000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.624}]},{"osd":6,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70899999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.69899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.433,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71699999999999997,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:02:15.122 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:02:15.122 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:02:15.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.281+0000 7f5ce52c5700 1 Processor -- start 2023-10-16T10:02:15.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.281+0000 7f5ce52c5700 1 -- start start 2023-10-16T10:02:15.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5ce52c5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5ce00d01d0 0x7f5ce00ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5ce52c5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce00cf580 con 0x7f5ce00d1500 2023-10-16T10:02:15.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5ce52c5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce00ce970 con 0x7f5ce015ea40 2023-10-16T10:02:15.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5ce52c5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce00d4290 con 0x7f5ce00d01d0 2023-10-16T10:02:15.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cdf7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5ce00d1500 0x7f5ce00d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57166/0 (socket says 172.21.15.181:57166) 2023-10-16T10:02:15.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cdf7fe700 1 -- 172.21.15.181:0/3658261599 learned_addr learned my addr 172.21.15.181:0/3658261599 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:15.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cdeffd700 1 --2- 172.21.15.181:0/3658261599 >> v2:172.21.15.190:3300/0 conn(0x7f5ce00d01d0 0x7f5ce00ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892122107 0 0) 0x7f5ce00cf580 con 0x7f5ce00d1500 2023-10-16T10:02:15.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc80035b0 con 0x7f5ce00d1500 2023-10-16T10:02:15.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1104442683 0 0) 0x7f5cc80035b0 con 0x7f5ce00d1500 2023-10-16T10:02:15.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 >> v2:172.21.15.190:3300/0 conn(0x7f5ce00d01d0 msgr2=0x7f5ce00ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 --2- 172.21.15.181:0/3658261599 >> v2:172.21.15.190:3300/0 conn(0x7f5ce00d01d0 0x7f5ce00ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 >> v1:172.21.15.190:6789/0 conn(0x7f5ce015ea40 legacy=0x7f5ce00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce00cec50 con 0x7f5ce00d1500 2023-10-16T10:02:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5cd80019f0 con 0x7f5ce00d1500 2023-10-16T10:02:15.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 >> v1:172.21.15.181:6789/0 conn(0x7f5ce00d1500 legacy=0x7f5ce00d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8003a00 0x7f5cc8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cc8004850 0x7f5cc8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cc8005210 0x7f5cc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc800c080 con 0x7f5cc8004850 2023-10-16T10:02:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc800c1f0 con 0x7f5cc8003a00 2023-10-16T10:02:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.282+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc800c3b0 con 0x7f5cc8005210 2023-10-16T10:02:15.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cde7fc700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cc8004850 0x7f5cc8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cdeffd700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8003a00 0x7f5cc8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cdf7fe700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cc8005210 0x7f5cc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cde7fc700 1 -- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cc8005210 msgr2=0x7f5cc800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cde7fc700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cc8005210 0x7f5cc800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cde7fc700 1 -- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8003a00 msgr2=0x7f5cc8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cde7fc700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8003a00 0x7f5cc8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cdf7fe700 1 -- 172.21.15.181:0/3658261599 reap_dead start 2023-10-16T10:02:15.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cde7fc700 1 -- 172.21.15.181:0/3658261599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80035b0 con 0x7f5cc8004850 2023-10-16T10:02:15.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cde7fc700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cc8004850 0x7f5cc8004cd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00017c0 tx=0x7f5cc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d04955edc0042a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00052d0 con 0x7f5cc8004850 2023-10-16T10:02:15.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0005430 con 0x7f5cc8004850 2023-10-16T10:02:15.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.283+0000 7f5cddffb700 1 -- 172.21.15.181:0/3658261599 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00038c0 con 0x7f5cc8004850 2023-10-16T10:02:15.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.284+0000 7f5ce52c5700 1 -- 172.21.15.181:0/3658261599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cc8004850 msgr2=0x7f5cc8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.284+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cc8004850 0x7f5cc8004cd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00017c0 tx=0x7f5cc8005880 comp rx=0 tx=0).stop 2023-10-16T10:02:15.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.284+0000 7f5ce52c5700 1 -- 172.21.15.181:0/3658261599 shutdown_connections 2023-10-16T10:02:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.284+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/3658261599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cc8004850 0x7f5cc8004cd0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.284+0000 7f5ce52c5700 1 -- 172.21.15.181:0/3658261599 >> 172.21.15.181:0/3658261599 conn(0x7f5ce009cb60 msgr2=0x7f5ce00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.284+0000 7f5ce52c5700 1 -- 172.21.15.181:0/3658261599 shutdown_connections 2023-10-16T10:02:15.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.284+0000 7f5ce52c5700 1 -- 172.21.15.181:0/3658261599 wait complete. 2023-10-16T10:02:15.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.285+0000 7f5ce52c5700 1 Processor -- start 2023-10-16T10:02:15.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.285+0000 7f5ce52c5700 1 -- start start 2023-10-16T10:02:15.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5ce52c5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8004850 0x7f5ce01f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5ce52c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce01f2b00 0x7f5ce00b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5ce52c5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ce00b8110 0x7f5ce00cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5ce52c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00b92f0 con 0x7f5ce01f2b00 2023-10-16T10:02:15.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5ce52c5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00b8a50 con 0x7f5cc8004850 2023-10-16T10:02:15.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5ce52c5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00b8d30 con 0x7f5ce00b8110 2023-10-16T10:02:15.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce01f2b00 0x7f5ce00b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce01f2b00 0x7f5ce00b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46102/0 (socket says 172.21.15.181:46102) 2023-10-16T10:02:15.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 -- 172.21.15.181:0/1926024815 learned_addr learned my addr 172.21.15.181:0/1926024815 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:15.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cdeffd700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8004850 0x7f5ce01f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cdf7fe700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ce00b8110 0x7f5ce00cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 -- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ce00b8110 msgr2=0x7f5ce00cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ce00b8110 0x7f5ce00cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 -- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8004850 msgr2=0x7f5ce01f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8004850 0x7f5ce01f25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 -- 172.21.15.181:0/1926024815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce00b9010 con 0x7f5ce01f2b00 2023-10-16T10:02:15.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cdeffd700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8004850 0x7f5ce01f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:15.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cdf7fe700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ce00b8110 0x7f5ce00cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:15.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cde7fc700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce01f2b00 0x7f5ce00b7bd0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005040 tx=0x7f5cd0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cc77fe700 1 -- 172.21.15.181:0/1926024815 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0001290 con 0x7f5ce01f2b00 2023-10-16T10:02:15.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cc77fe700 1 -- 172.21.15.181:0/1926024815 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0007040 con 0x7f5ce01f2b00 2023-10-16T10:02:15.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce00b6610 con 0x7f5ce01f2b00 2023-10-16T10:02:15.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.286+0000 7f5cc77fe700 1 -- 172.21.15.181:0/1926024815 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00073f0 con 0x7f5ce01f2b00 2023-10-16T10:02:15.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.287+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce00b8770 con 0x7f5ce01f2b00 2023-10-16T10:02:15.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.287+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce004ebb0 con 0x7f5ce01f2b00 2023-10-16T10:02:15.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.288+0000 7f5cc77fe700 1 -- 172.21.15.181:0/1926024815 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5cd0007550 con 0x7f5ce01f2b00 2023-10-16T10:02:15.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.288+0000 7f5cc77fe700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5ccc06b020 0x7f5ccc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.288+0000 7f5cc77fe700 1 -- 172.21.15.181:0/1926024815 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f5cd008a250 con 0x7f5ce01f2b00 2023-10-16T10:02:15.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.288+0000 7f5cdeffd700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5ccc06b020 0x7f5ccc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.289+0000 7f5cdeffd700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5ccc06b020 0x7f5ccc06d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00697f0 tx=0x7f5cd4008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.293+0000 7f5cc77fe700 1 -- 172.21.15.181:0/1926024815 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5cd00551f0 con 0x7f5ce01f2b00 2023-10-16T10:02:15.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.564+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5ce015e820 con 0x7f5ce01f2b00 2023-10-16T10:02:15.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.567+0000 7f5cc77fe700 1 -- 172.21.15.181:0/1926024815 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5cd00210b0 con 0x7f5ce01f2b00 2023-10-16T10:02:15.569 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T10:02:15.569 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T10:02:15.569 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T10:02:15.569 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T10:02:15.569 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:02:15.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5ccc06b020 msgr2=0x7f5ccc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5ccc06b020 0x7f5ccc06d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00697f0 tx=0x7f5cd4008040 comp rx=0 tx=0).stop 2023-10-16T10:02:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce01f2b00 msgr2=0x7f5ce00b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce01f2b00 0x7f5ce00b7bd0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005040 tx=0x7f5cd0001120 comp rx=0 tx=0).stop 2023-10-16T10:02:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 shutdown_connections 2023-10-16T10:02:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f5ccc06b020 0x7f5ccc06d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ce00b8110 0x7f5ce00cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc8004850 0x7f5ce01f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 --2- 172.21.15.181:0/1926024815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce01f2b00 0x7f5ce00b7bd0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 >> 172.21.15.181:0/1926024815 conn(0x7f5ce009cb60 msgr2=0x7f5ce015acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 shutdown_connections 2023-10-16T10:02:15.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.569+0000 7f5ce52c5700 1 -- 172.21.15.181:0/1926024815 wait complete. 2023-10-16T10:02:15.574 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T10:02:15.583 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:02:15.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.703+0000 7f6cfe8d9700 1 Processor -- start 2023-10-16T10:02:15.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.703+0000 7f6cfe8d9700 1 -- start start 2023-10-16T10:02:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cfe8d9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 0x7f6cf80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cfe8d9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf806ed00 con 0x7f6cf80cb0c0 2023-10-16T10:02:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cfe8d9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80d02d0 con 0x7f6cf815eab0 2023-10-16T10:02:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cfe8d9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf806e0f0 con 0x7f6cf815e6d0 2023-10-16T10:02:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cfce76700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6cf80cb0c0 0x7f6cf815a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57190/0 (socket says 172.21.15.181:57190) 2023-10-16T10:02:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cfce76700 1 -- 172.21.15.181:0/97245467 learned_addr learned my addr 172.21.15.181:0/97245467 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf7fff700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 0x7f6cf80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723135253 0 0) 0x7f6cf806ed00 con 0x7f6cf80cb0c0 2023-10-16T10:02:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce00035b0 con 0x7f6cf80cb0c0 2023-10-16T10:02:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2916820129 0 0) 0x7f6ce00035b0 con 0x7f6cf80cb0c0 2023-10-16T10:02:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 msgr2=0x7f6cf80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 0x7f6cf80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 >> v1:172.21.15.190:6789/0 conn(0x7f6cf815eab0 legacy=0x7f6cf815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf806e3d0 con 0x7f6cf80cb0c0 2023-10-16T10:02:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.704+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f6cf00019f0 con 0x7f6cf80cb0c0 2023-10-16T10:02:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 >> v1:172.21.15.181:6789/0 conn(0x7f6cf80cb0c0 legacy=0x7f6cf815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce0003a30 0x7f6ce0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 0x7f6ce0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce0004d90 0x7f6ce0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf806e0f0 con 0x7f6ce0004d90 2023-10-16T10:02:15.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce000bac0 con 0x7f6ce0003a30 2023-10-16T10:02:15.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce000bc80 con 0x7f6cf815e6d0 2023-10-16T10:02:15.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cfce76700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce0004d90 0x7f6ce0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf7fff700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce0003a30 0x7f6ce0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf77fe700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 0x7f6ce0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cfce76700 1 -- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 msgr2=0x7f6ce0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cfce76700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf815e6d0 0x7f6ce0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cfce76700 1 -- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce0003a30 msgr2=0x7f6ce0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:15.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cfce76700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce0003a30 0x7f6ce0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cfce76700 1 -- 172.21.15.181:0/97245467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00035b0 con 0x7f6ce0004d90 2023-10-16T10:02:15.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cfce76700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce0004d90 0x7f6ce0005210 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000a800 tx=0x7f6ce0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c070b1914b95b67 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf000a2f0 con 0x7f6ce0004d90 2023-10-16T10:02:15.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0012600 con 0x7f6ce0004d90 2023-10-16T10:02:15.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.705+0000 7f6cf6ffd700 1 -- 172.21.15.181:0/97245467 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00129b0 con 0x7f6ce0004d90 2023-10-16T10:02:15.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.706+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/97245467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce0004d90 msgr2=0x7f6ce0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.706+0000 7f6cfe8d9700 1 --2- 172.21.15.181:0/97245467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce0004d90 0x7f6ce0005210 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000a800 tx=0x7f6ce0005880 comp rx=0 tx=0).stop 2023-10-16T10:02:15.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.706+0000 7f6cfce76700 1 -- 172.21.15.181:0/97245467 reap_dead start 2023-10-16T10:02:15.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.706+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/97245467 shutdown_connections 2023-10-16T10:02:15.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.706+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/97245467 >> 172.21.15.181:0/97245467 conn(0x7f6cf80d04a0 msgr2=0x7f6cf80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:15.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.706+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/97245467 shutdown_connections 2023-10-16T10:02:15.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.706+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/97245467 wait complete. 2023-10-16T10:02:15.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.707+0000 7f6cfe8d9700 1 Processor -- start 2023-10-16T10:02:15.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.707+0000 7f6cfe8d9700 1 -- start start 2023-10-16T10:02:15.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cfe8d9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cf80b2060 0x7f6cf80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cfe8d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cf80b3a40 0x7f6cf806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cfe8d9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf80c9620 0x7f6cf806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cfe8d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80c9190 con 0x7f6cf80b3a40 2023-10-16T10:02:15.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cfe8d9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80b3fa0 con 0x7f6cf80b2060 2023-10-16T10:02:15.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cfe8d9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80c8b40 con 0x7f6cf80c9620 2023-10-16T10:02:15.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cf80b3a40 0x7f6cf806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cf80b3a40 0x7f6cf806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46122/0 (socket says 172.21.15.181:46122) 2023-10-16T10:02:15.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 -- 172.21.15.181:0/1244518837 learned_addr learned my addr 172.21.15.181:0/1244518837 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:15.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cfce76700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf80c9620 0x7f6cf806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf7fff700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cf80b2060 0x7f6cf80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 -- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf80c9620 msgr2=0x7f6cf806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf80c9620 0x7f6cf806ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 -- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cf80b2060 msgr2=0x7f6cf80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cf80b2060 0x7f6cf80b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 -- 172.21.15.181:0/1244518837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80c8e20 con 0x7f6cf80b3a40 2023-10-16T10:02:15.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf77fe700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cf80b3a40 0x7f6cf806a8a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800f7f0 tx=0x7f6ce800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.708+0000 7f6cf4ff9700 1 -- 172.21.15.181:0/1244518837 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80052e0 con 0x7f6cf80b3a40 2023-10-16T10:02:15.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.709+0000 7f6cf4ff9700 1 -- 172.21.15.181:0/1244518837 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce8005440 con 0x7f6cf80b3a40 2023-10-16T10:02:15.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.709+0000 7f6cf4ff9700 1 -- 172.21.15.181:0/1244518837 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80038c0 con 0x7f6cf80b3a40 2023-10-16T10:02:15.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.709+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf80b4280 con 0x7f6cf80b3a40 2023-10-16T10:02:15.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.709+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf80b2440 con 0x7f6cf80b3a40 2023-10-16T10:02:15.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.710+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf80c8fb0 con 0x7f6cf80b3a40 2023-10-16T10:02:15.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.710+0000 7f6cf4ff9700 1 -- 172.21.15.181:0/1244518837 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6ce8003a20 con 0x7f6cf80b3a40 2023-10-16T10:02:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.710+0000 7f6cf4ff9700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f6ce406b020 0x7f6ce406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.711+0000 7f6cf4ff9700 1 -- 172.21.15.181:0/1244518837 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f6ce80896f0 con 0x7f6cf80b3a40 2023-10-16T10:02:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.711+0000 7f6cf7fff700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f6ce406b020 0x7f6ce406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.711+0000 7f6cf7fff700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f6ce406b020 0x7f6ce406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009b80 tx=0x7f6cec008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:15.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.715+0000 7f6cf4ff9700 1 -- 172.21.15.181:0/1244518837 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6ce8054570 con 0x7f6cf80b3a40 2023-10-16T10:02:15.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.949+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cf8067d20 con 0x7f6cf80b3a40 2023-10-16T10:02:15.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.950+0000 7f6cf4ff9700 1 -- 172.21.15.181:0/1244518837 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f6ce8057b90 con 0x7f6cf80b3a40 2023-10-16T10:02:15.952 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:15.952 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2945191097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2945191097},{"type":"v1","addr":"172.21.15.190:6834","nonce":2945191097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1168488779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1168488779},{"type":"v1","addr":"172.21.15.190:6835","nonce":1168488779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2677039975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2677039975},{"type":"v1","addr":"172.21.15.181:6836","nonce":2677039975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:10.010346+0000","modified":"2023-10-16T10:02:13.035963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1841075189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1841075189},{"type":"v1","addr":"172.21.15.181:6837","nonce":1841075189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:02:15.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f6ce406b020 msgr2=0x7f6ce406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f6ce406b020 0x7f6ce406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009b80 tx=0x7f6cec008040 comp rx=0 tx=0).stop 2023-10-16T10:02:15.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cf80b3a40 msgr2=0x7f6cf806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:15.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cf80b3a40 0x7f6cf806a8a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800f7f0 tx=0x7f6ce800fbb0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 shutdown_connections 2023-10-16T10:02:15.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f6ce406b020 0x7f6ce406d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6cf80c9620 0x7f6cf806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6cf80b2060 0x7f6cf80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.952+0000 7f6cfe8d9700 1 --2- 172.21.15.181:0/1244518837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cf80b3a40 0x7f6cf806a8a0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:15.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.953+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 >> 172.21.15.181:0/1244518837 conn(0x7f6cf80d04a0 msgr2=0x7f6cf80ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:15.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.953+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 shutdown_connections 2023-10-16T10:02:15.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:15.953+0000 7f6cfe8d9700 1 -- 172.21.15.181:0/1244518837 wait complete. 2023-10-16T10:02:15.958 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 58 2023-10-16T10:02:16.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:02:10.010346+0000', 'modified': '2023-10-16T10:02:13.035963+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8022}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8022': {'gid': 8022, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/1841075189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 1841075189}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1841075189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:02:16.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:02:16.370 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T10:02:16.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.493+0000 7f749f7b0700 1 Processor -- start 2023-10-16T10:02:16.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.493+0000 7f749f7b0700 1 -- start start 2023-10-16T10:02:16.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749f7b0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74980d0fc0 0x7f74980cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749f7b0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980cddf0 con 0x7f749806a8a0 2023-10-16T10:02:16.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749f7b0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980ccf50 con 0x7f749815df60 2023-10-16T10:02:16.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749f7b0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980af9c0 con 0x7f74980d0fc0 2023-10-16T10:02:16.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749cd4b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74980d0fc0 0x7f74980cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749dd4d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f749806a8a0 0x7f749806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57210/0 (socket says 172.21.15.181:57210) 2023-10-16T10:02:16.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749dd4d700 1 -- 172.21.15.181:0/2928005468 learned_addr learned my addr 172.21.15.181:0/2928005468 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:16.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749cd4b700 1 -- 172.21.15.181:0/2928005468 >> v1:172.21.15.190:6789/0 conn(0x7f749815df60 legacy=0x7f74980cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749cd4b700 1 -- 172.21.15.181:0/2928005468 >> v1:172.21.15.181:6789/0 conn(0x7f749806a8a0 legacy=0x7f749806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749cd4b700 1 -- 172.21.15.181:0/2928005468 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980cc600 con 0x7f74980d0fc0 2023-10-16T10:02:16.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f748ffff700 1 -- 172.21.15.181:0/2928005468 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158204468 0 0) 0x7f74980cddf0 con 0x7f749806a8a0 2023-10-16T10:02:16.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f749cd4b700 1 --2- 172.21.15.181:0/2928005468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980cae20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7494009840 tx=0x7f7494009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4b55b35aa564e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f748ffff700 1 -- 172.21.15.181:0/2928005468 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494004230 con 0x7f74980d0fc0 2023-10-16T10:02:16.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f748ffff700 1 -- 172.21.15.181:0/2928005468 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7494004390 con 0x7f74980d0fc0 2023-10-16T10:02:16.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.494+0000 7f748ffff700 1 -- 172.21.15.181:0/2928005468 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74940036a0 con 0x7f74980d0fc0 2023-10-16T10:02:16.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.495+0000 7f749f7b0700 1 -- 172.21.15.181:0/2928005468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 msgr2=0x7f74980cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.495+0000 7f749f7b0700 1 --2- 172.21.15.181:0/2928005468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980cae20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7494009840 tx=0x7f7494009b50 comp rx=0 tx=0).stop 2023-10-16T10:02:16.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.495+0000 7f749f7b0700 1 -- 172.21.15.181:0/2928005468 shutdown_connections 2023-10-16T10:02:16.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.495+0000 7f749f7b0700 1 --2- 172.21.15.181:0/2928005468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980cae20 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.495+0000 7f749f7b0700 1 -- 172.21.15.181:0/2928005468 >> 172.21.15.181:0/2928005468 conn(0x7f74980d04a0 msgr2=0x7f74980d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.495+0000 7f749f7b0700 1 -- 172.21.15.181:0/2928005468 shutdown_connections 2023-10-16T10:02:16.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.495+0000 7f749f7b0700 1 -- 172.21.15.181:0/2928005468 wait complete. 2023-10-16T10:02:16.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.496+0000 7f749f7b0700 1 Processor -- start 2023-10-16T10:02:16.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.496+0000 7f749f7b0700 1 -- start start 2023-10-16T10:02:16.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749f7b0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f749806a8a0 0x7f74980b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749f7b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749f7b0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815df60 0x7f74980c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749f7b0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980b6220 con 0x7f74980d0fc0 2023-10-16T10:02:16.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749f7b0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980c59e0 con 0x7f749806a8a0 2023-10-16T10:02:16.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749f7b0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980c4dd0 con 0x7f749815df60 2023-10-16T10:02:16.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46136/0 (socket says 172.21.15.181:46136) 2023-10-16T10:02:16.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 -- 172.21.15.181:0/984588065 learned_addr learned my addr 172.21.15.181:0/984588065 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:16.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749d54c700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f749806a8a0 0x7f74980b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749dd4d700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815df60 0x7f74980c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 -- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815df60 msgr2=0x7f74980c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815df60 0x7f74980c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 -- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f749806a8a0 msgr2=0x7f74980b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f749806a8a0 0x7f74980b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.497+0000 7f749cd4b700 1 -- 172.21.15.181:0/984588065 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940094a0 con 0x7f74980d0fc0 2023-10-16T10:02:16.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.498+0000 7f749cd4b700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980c5c10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f7494009b90 tx=0x7f7494004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.498+0000 7f748dffb700 1 -- 172.21.15.181:0/984588065 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494011630 con 0x7f74980d0fc0 2023-10-16T10:02:16.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.498+0000 7f748dffb700 1 -- 172.21.15.181:0/984588065 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749401e070 con 0x7f74980d0fc0 2023-10-16T10:02:16.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.498+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74980c50b0 con 0x7f74980d0fc0 2023-10-16T10:02:16.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.498+0000 7f748dffb700 1 -- 172.21.15.181:0/984588065 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74940119b0 con 0x7f74980d0fc0 2023-10-16T10:02:16.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.498+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74980c5670 con 0x7f74980d0fc0 2023-10-16T10:02:16.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.499+0000 7f748dffb700 1 -- 172.21.15.181:0/984588065 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7494011b10 con 0x7f74980d0fc0 2023-10-16T10:02:16.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.500+0000 7f748dffb700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f747c06b020 0x7f747c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.500+0000 7f748dffb700 1 -- 172.21.15.181:0/984588065 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f749408ace0 con 0x7f74980d0fc0 2023-10-16T10:02:16.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.500+0000 7f749d54c700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f747c06b020 0x7f747c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.500+0000 7f749d54c700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f747c06b020 0x7f747c06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f74981e87e0 tx=0x7f7488008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.502+0000 7f74ab99ab80 1 Processor -- start 2023-10-16T10:02:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.502+0000 7f74ab99ab80 1 -- start start 2023-10-16T10:02:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862762a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862762fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x5558627366c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558626acc20 con 0x555862736010 2023-10-16T10:02:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555862691c10 con 0x55586275b220 2023-10-16T10:02:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55586268f530 con 0x555862760630 2023-10-16T10:02:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x5558627366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x5558627366c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46146/0 (socket says 172.21.15.181:46146) 2023-10-16T10:02:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 -- 172.21.15.181:0/384065909 learned_addr learned my addr 172.21.15.181:0/384065909 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749cd4b700 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862762fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749d54c700 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862762a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 -- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 msgr2=0x555862762fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862762fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 -- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 msgr2=0x555862762a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862762a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 -- 172.21.15.181:0/384065909 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558626483a0 con 0x555862736010 2023-10-16T10:02:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749d54c700 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862762a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f749dd4d700 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x5558627366c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f7490002200 tx=0x7f749000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a998ffd8d128793c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f7482ffd700 1 -- 172.21.15.181:0/384065909 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74900070d0 con 0x555862736010 2023-10-16T10:02:16.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f7482ffd700 1 -- 172.21.15.181:0/384065909 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7490007230 con 0x555862736010 2023-10-16T10:02:16.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.503+0000 7f7482ffd700 1 -- 172.21.15.181:0/384065909 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74900075a0 con 0x555862736010 2023-10-16T10:02:16.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 -- 172.21.15.181:0/384065909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 msgr2=0x5558627366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x5558627366c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f7490002200 tx=0x7f749000d430 comp rx=0 tx=0).stop 2023-10-16T10:02:16.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 -- 172.21.15.181:0/384065909 shutdown_connections 2023-10-16T10:02:16.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x5558627366c0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862762a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/384065909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862762fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 -- 172.21.15.181:0/384065909 >> 172.21.15.181:0/384065909 conn(0x555862665d80 msgr2=0x555862780ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 -- 172.21.15.181:0/384065909 shutdown_connections 2023-10-16T10:02:16.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.504+0000 7f74ab99ab80 1 -- 172.21.15.181:0/384065909 wait complete. 2023-10-16T10:02:16.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.505+0000 7f74ab99ab80 1 -- start start 2023-10-16T10:02:16.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.505+0000 7f74ab99ab80 1 Processor -- start 2023-10-16T10:02:16.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x55586275a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55586275b220 0x55586278d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862760630 0x55586275c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55586268ae20 con 0x555862760630 2023-10-16T10:02:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558626276c0 con 0x555862736010 2023-10-16T10:02:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555862627360 con 0x55586275b220 2023-10-16T10:02:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 10 client.0 _ll_register_callbacks cb 0x5558624847b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:02:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862760630 0x55586275c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862760630 0x55586275c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46152/0 (socket says 172.21.15.181:46152) 2023-10-16T10:02:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 -- 172.21.15.181:0/2439641722 learned_addr learned my addr 172.21.15.181:0/2439641722 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749cd4b700 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55586275b220 0x55586278d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749d54c700 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x55586275a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 -- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55586275b220 msgr2=0x55586278d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55586275b220 0x55586278d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 -- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 msgr2=0x55586275a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x55586275a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 -- 172.21.15.181:0/2439641722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7490001e30 con 0x555862760630 2023-10-16T10:02:16.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f749dd4d700 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862760630 0x55586275c080 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x555862739fa0 tx=0x7f7490001480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb73645a3bd9b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f7477fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:02:16.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f7477fff700 1 -- 172.21.15.181:0/2439641722 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000dd80 con 0x555862760630 2023-10-16T10:02:16.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f7477fff700 1 -- 172.21.15.181:0/2439641722 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f749001f470 con 0x555862760630 2023-10-16T10:02:16.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55586272b5b0 con 0x555862760630 2023-10-16T10:02:16.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f7477fff700 1 -- 172.21.15.181:0/2439641722 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f749000dee0 con 0x555862760630 2023-10-16T10:02:16.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 10 client.8166 fetch_fsmap learned FSMap version 58 2023-10-16T10:02:16.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.506+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x5558624b5d90 con 0x555862760630 2023-10-16T10:02:16.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f7477fff700 1 -- 172.21.15.181:0/2439641722 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55586272b5b0 con 0x555862760630 2023-10-16T10:02:16.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f7477fff700 1 -- 172.21.15.181:0/2439641722 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f74900283f0 con 0x555862760630 2023-10-16T10:02:16.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f74ab99ab80 10 client.8166 fetch_fsmap finished waiting for FSMap version 58 2023-10-16T10:02:16.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f74ab99ab80 10 client.8166 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:02:16.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f74ab99ab80 20 client.8166 populate_metadata read hostname 'smithi181' 2023-10-16T10:02:16.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586275ebe0 0x55586278e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f74ab99ab80 4 client.8166 mds_command: new command op to 8022 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:02:16.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 --> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555862683e00 con 0x55586275ebe0 2023-10-16T10:02:16.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f749d54c700 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586275ebe0 0x55586278e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f7477fff700 10 client.8166 ms_handle_connect on v2:172.21.15.181:6835/1841075189 2023-10-16T10:02:16.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.507+0000 7f749d54c700 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586275ebe0 0x55586278e570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51e4a8c5aeb3fae server_cookie=eaee7cfd72e99ce5 in_seq=0 out_seq=0 2023-10-16T10:02:16.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f7477fff700 1 -- 172.21.15.181:0/2439641722 <== mds.0 v2:172.21.15.181:6835/1841075189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555862683e00 con 0x55586275ebe0 2023-10-16T10:02:16.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f7477fff700 10 client.8166 handle_command_reply: tid=0 2023-10-16T10:02:16.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 client.8166 shutdown 2023-10-16T10:02:16.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862760630 msgr2=0x55586275c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862760630 0x55586275c080 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x555862739fa0 tx=0x7f7490001480 comp rx=0 tx=0).stop 2023-10-16T10:02:16.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 shutdown_connections 2023-10-16T10:02:16.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862760630 0x55586275c080 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x55586275a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586275ebe0 0x55586278e570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2439641722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55586275b220 0x55586278d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f7477fff700 0 client.8166 ms_handle_reset on v2:172.21.15.181:6835/1841075189 2023-10-16T10:02:16.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 >> 172.21.15.181:0/2439641722 conn(0x555862665d80 msgr2=0x55586277e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 shutdown_connections 2023-10-16T10:02:16.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2439641722 wait complete. 2023-10-16T10:02:16.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.508+0000 7f74ab99ab80 20 client.8166 trim_cache size 0 max 16384 2023-10-16T10:02:16.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 Processor -- start 2023-10-16T10:02:16.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 -- start start 2023-10-16T10:02:16.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x55586272c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862796ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862738bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55586250b8d0 con 0x555862736010 2023-10-16T10:02:16.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55586272f630 con 0x55586275b220 2023-10-16T10:02:16.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55586275dab0 con 0x555862760630 2023-10-16T10:02:16.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f749d54c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x55586272c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f749d54c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x55586272c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46156/0 (socket says 172.21.15.181:46156) 2023-10-16T10:02:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.526+0000 7f749d54c700 1 -- 172.21.15.181:0/1795731120 learned_addr learned my addr 172.21.15.181:0/1795731120 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749cd4b700 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862796ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749dd4d700 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862738bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749d54c700 1 -- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 msgr2=0x555862738bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749d54c700 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862738bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749d54c700 1 -- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 msgr2=0x555862796ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749d54c700 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862796ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749d54c700 1 -- 172.21.15.181:0/1795731120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55586249ec10 con 0x555862736010 2023-10-16T10:02:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749cd4b700 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862796ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f749d54c700 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x55586272c9d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f74880114c0 tx=0x7f7488013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bae144a34b46b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f74837fe700 1 -- 172.21.15.181:0/1795731120 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488011580 con 0x555862736010 2023-10-16T10:02:16.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f74837fe700 1 -- 172.21.15.181:0/1795731120 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7488019040 con 0x555862736010 2023-10-16T10:02:16.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f74837fe700 1 -- 172.21.15.181:0/1795731120 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488011ee0 con 0x555862736010 2023-10-16T10:02:16.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f74ab99ab80 1 -- 172.21.15.181:0/1795731120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 msgr2=0x55586272c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.527+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x55586272c9d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f74880114c0 tx=0x7f7488013000 comp rx=0 tx=0).stop 2023-10-16T10:02:16.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 -- 172.21.15.181:0/1795731120 shutdown_connections 2023-10-16T10:02:16.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862738bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555862736010 0x55586272c9d0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/1795731120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55586275b220 0x555862796ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 -- 172.21.15.181:0/1795731120 >> 172.21.15.181:0/1795731120 conn(0x555862665d80 msgr2=0x55586275a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 -- 172.21.15.181:0/1795731120 shutdown_connections 2023-10-16T10:02:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 -- 172.21.15.181:0/1795731120 wait complete. 2023-10-16T10:02:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 -- start start 2023-10-16T10:02:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.528+0000 7f74ab99ab80 1 Processor -- start 2023-10-16T10:02:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x555862796ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55586275b220 0x5558627de890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f74ab99ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862737910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55586275fce0 con 0x55586275b220 2023-10-16T10:02:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558627970e0 con 0x555862736010 2023-10-16T10:02:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f74ab99ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555862737480 con 0x555862760630 2023-10-16T10:02:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f74ab99ab80 10 client.0 _ll_register_callbacks cb 0x7f7490002050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:02:16.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55586275b220 0x5558627de890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55586275b220 0x5558627de890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46164/0 (socket says 172.21.15.181:46164) 2023-10-16T10:02:16.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 -- 172.21.15.181:0/2558918613 learned_addr learned my addr 172.21.15.181:0/2558918613 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:16.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749d54c700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x555862796ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749dd4d700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862737910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 -- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 msgr2=0x555862737910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862737910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 -- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 msgr2=0x555862796ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x555862796ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749cd4b700 1 -- 172.21.15.181:0/2558918613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7488024af0 con 0x55586275b220 2023-10-16T10:02:16.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.529+0000 7f749dd4d700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862737910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:16.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f749cd4b700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55586275b220 0x5558627de890 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7494003fd0 tx=0x7f74940598a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5192f245c6753482 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f7477fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:02:16.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f7477fff700 1 -- 172.21.15.181:0/2558918613 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494055f30 con 0x55586275b220 2023-10-16T10:02:16.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f7477fff700 1 -- 172.21.15.181:0/2558918613 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f7494056090 con 0x55586275b220 2023-10-16T10:02:16.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7490007230 con 0x55586275b220 2023-10-16T10:02:16.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f7477fff700 1 -- 172.21.15.181:0/2558918613 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7494071020 con 0x55586275b220 2023-10-16T10:02:16.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 10 client.8172 fetch_fsmap learned FSMap version 58 2023-10-16T10:02:16.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x5558624dc200 con 0x55586275b220 2023-10-16T10:02:16.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f7477fff700 1 -- 172.21.15.181:0/2558918613 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7490007230 con 0x55586275b220 2023-10-16T10:02:16.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f7477fff700 1 -- 172.21.15.181:0/2558918613 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7494059a80 con 0x55586275b220 2023-10-16T10:02:16.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 10 client.8172 fetch_fsmap finished waiting for FSMap version 58 2023-10-16T10:02:16.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 10 client.8172 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T10:02:16.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 20 client.8172 populate_metadata read hostname 'smithi181' 2023-10-16T10:02:16.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586278c980 0x55586278cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 4 client.8172 mds_command: new command op to 8022 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:02:16.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 --> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555862683b20 con 0x55586278c980 2023-10-16T10:02:16.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.530+0000 7f749d54c700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586278c980 0x55586278cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f7477fff700 10 client.8172 ms_handle_connect on v2:172.21.15.181:6835/1841075189 2023-10-16T10:02:16.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f749d54c700 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586278c980 0x55586278cd60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f830a9baa4ef91c6 server_cookie=ecef783b07c275a in_seq=0 out_seq=0 2023-10-16T10:02:16.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f7477fff700 1 -- 172.21.15.181:0/2558918613 <== mds.0 v2:172.21.15.181:6835/1841075189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555862683b20 con 0x55586278c980 2023-10-16T10:02:16.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f7477fff700 10 client.8172 handle_command_reply: tid=0 2023-10-16T10:02:16.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 client.8172 shutdown 2023-10-16T10:02:16.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55586275b220 msgr2=0x5558627de890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55586275b220 0x5558627de890 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7494003fd0 tx=0x7f74940598a0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 shutdown_connections 2023-10-16T10:02:16.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:6835/1841075189,v1:172.21.15.181:6837/1841075189] conn(0x55586278c980 0x55586278cd60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555862760630 0x555862737910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555862736010 0x555862796ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 --2- 172.21.15.181:0/2558918613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55586275b220 0x5558627de890 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f7477fff700 0 client.8172 ms_handle_reset on v2:172.21.15.181:6835/1841075189 2023-10-16T10:02:16.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 >> 172.21.15.181:0/2558918613 conn(0x555862665d80 msgr2=0x55586272cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 shutdown_connections 2023-10-16T10:02:16.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 1 -- 172.21.15.181:0/2558918613 wait complete. 2023-10-16T10:02:16.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.531+0000 7f74ab99ab80 20 client.8172 trim_cache size 0 max 16384 2023-10-16T10:02:16.561 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:02:16.561 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T10:02:16.561 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T10:02:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 8022, 2023-10-16T10:02:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T10:02:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T10:02:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T10:02:16.563 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9093830709999997, 2023-10-16T10:02:16.563 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 58, 2023-10-16T10:02:16.563 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 118, 2023-10-16T10:02:16.564 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 118, 2023-10-16T10:02:16.564 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.855713908 2023-10-16T10:02:16.564 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:02:16.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f747c06b020 msgr2=0x7f747c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f747c06b020 0x7f747c06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f74981e87e0 tx=0x7f7488008040 comp rx=0 tx=0).stop 2023-10-16T10:02:16.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 msgr2=0x7f74980c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980c5c10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f7494009b90 tx=0x7f7494004750 comp rx=0 tx=0).stop 2023-10-16T10:02:16.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 shutdown_connections 2023-10-16T10:02:16.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f747c06b020 0x7f747c06d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815df60 0x7f74980c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f749806a8a0 0x7f74980b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 --2- 172.21.15.181:0/984588065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980d0fc0 0x7f74980c5c10 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 >> 172.21.15.181:0/984588065 conn(0x7f74980d04a0 msgr2=0x7f74980ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 shutdown_connections 2023-10-16T10:02:16.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.532+0000 7f749f7b0700 1 -- 172.21.15.181:0/984588065 wait complete. 2023-10-16T10:02:16.570 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:02:16.593 INFO:tasks.ceph.mds.a.smithi181.stderr:2023-10-16T10:02:16.592+0000 7f2af794e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:02:16.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.690+0000 7fdc12cf8700 1 Processor -- start 2023-10-16T10:02:16.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc12cf8700 1 -- start start 2023-10-16T10:02:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc12cf8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc0c15dc40 0x7fdc0c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc12cf8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdc0c06ade0 0x7fdc0c0c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc12cf8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0c0b2710 con 0x7fdc0c0c2bf0 2023-10-16T10:02:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc12cf8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0c0c3080 con 0x7fdc0c15dc40 2023-10-16T10:02:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc12cf8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0c0c18f0 con 0x7fdc0c06ade0 2023-10-16T10:02:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc10a94700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc0c15dc40 0x7fdc0c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.691+0000 7fdc10a94700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc0c15dc40 0x7fdc0c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46170/0 (socket says 172.21.15.181:46170) 2023-10-16T10:02:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc10a94700 1 -- 172.21.15.181:0/1884851509 learned_addr learned my addr 172.21.15.181:0/1884851509 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:16.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc0bfff700 1 --2- 172.21.15.181:0/1884851509 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c06ade0 0x7fdc0c0c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc10a94700 1 -- 172.21.15.181:0/1884851509 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c06ade0 msgr2=0x7fdc0c0c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc10a94700 1 --2- 172.21.15.181:0/1884851509 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c06ade0 0x7fdc0c0c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc10a94700 1 -- 172.21.15.181:0/1884851509 >> v1:172.21.15.190:6789/0 conn(0x7fdc0c0c2bf0 legacy=0x7fdc0c15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:16.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc10a94700 1 -- 172.21.15.181:0/1884851509 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0c1bd0 con 0x7fdc0c15dc40 2023-10-16T10:02:16.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc10a94700 1 --2- 172.21.15.181:0/1884851509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c06a8a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c06c700 tx=0x7fdc00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba5acff456e3226 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc0b7fe700 1 -- 172.21.15.181:0/1884851509 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000b870 con 0x7fdc0c15dc40 2023-10-16T10:02:16.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc0b7fe700 1 -- 172.21.15.181:0/1884851509 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0000b9d0 con 0x7fdc0c15dc40 2023-10-16T10:02:16.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.692+0000 7fdc0b7fe700 1 -- 172.21.15.181:0/1884851509 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000bd60 con 0x7fdc0c15dc40 2023-10-16T10:02:16.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1884851509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c15dc40 msgr2=0x7fdc0c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1884851509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c06a8a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c06c700 tx=0x7fdc00009bd0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1884851509 shutdown_connections 2023-10-16T10:02:16.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1884851509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c06a8a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1884851509 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c06ade0 0x7fdc0c0c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1884851509 >> 172.21.15.181:0/1884851509 conn(0x7fdc0c0b1780 msgr2=0x7fdc0c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1884851509 shutdown_connections 2023-10-16T10:02:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.693+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1884851509 wait complete. 2023-10-16T10:02:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.694+0000 7fdc12cf8700 1 Processor -- start 2023-10-16T10:02:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.694+0000 7fdc12cf8700 1 -- start start 2023-10-16T10:02:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc12cf8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c06ade0 0x7fdc0c071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc12cf8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0c2bf0 0x7fdc0c06e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc12cf8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc12cf8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0c92c0 con 0x7fdc0c06ade0 2023-10-16T10:02:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc12cf8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c1571c0 con 0x7fdc0c15dc40 2023-10-16T10:02:16.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc12cf8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0c9810 con 0x7fdc0c0c2bf0 2023-10-16T10:02:16.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c06ade0 0x7fdc0c071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c06ade0 0x7fdc0c071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46176/0 (socket says 172.21.15.181:46176) 2023-10-16T10:02:16.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 -- 172.21.15.181:0/1032775577 learned_addr learned my addr 172.21.15.181:0/1032775577 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc11295700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc0bfff700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0c2bf0 0x7fdc0c06e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 -- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0c2bf0 msgr2=0x7fdc0c06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0c2bf0 0x7fdc0c06e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 -- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15dc40 msgr2=0x7fdc0c0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c0b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 -- 172.21.15.181:0/1032775577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00009550 con 0x7fdc0c06ade0 2023-10-16T10:02:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc0bfff700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0c2bf0 0x7fdc0c06e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc11295700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:16.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.695+0000 7fdc10a94700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c06ade0 0x7fdc0c071f00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.696+0000 7fdc097fa700 1 -- 172.21.15.181:0/1032775577 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00004520 con 0x7fdc0c06ade0 2023-10-16T10:02:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.696+0000 7fdc097fa700 1 -- 172.21.15.181:0/1032775577 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0001e070 con 0x7fdc0c06ade0 2023-10-16T10:02:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.696+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c157460 con 0x7fdc0c06ade0 2023-10-16T10:02:16.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.696+0000 7fdc097fa700 1 -- 172.21.15.181:0/1032775577 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000038a0 con 0x7fdc0c06ade0 2023-10-16T10:02:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.696+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c155f50 con 0x7fdc0c06ade0 2023-10-16T10:02:16.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.697+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0c067d20 con 0x7fdc0c06ade0 2023-10-16T10:02:16.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.697+0000 7fdc097fa700 1 -- 172.21.15.181:0/1032775577 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdc00003a00 con 0x7fdc0c06ade0 2023-10-16T10:02:16.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.698+0000 7fdc097fa700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fdbf006b020 0x7fdbf006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:16.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.698+0000 7fdc097fa700 1 -- 172.21.15.181:0/1032775577 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fdc00020070 con 0x7fdc0c06ade0 2023-10-16T10:02:16.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.698+0000 7fdc0bfff700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fdbf006b020 0x7fdbf006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:16.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.698+0000 7fdc0bfff700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fdbf006b020 0x7fdbf006d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c159600 tx=0x7fdbf8008040 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:16.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.702+0000 7fdc097fa700 1 -- 172.21.15.181:0/1032775577 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdc000562d0 con 0x7fdc0c06ade0 2023-10-16T10:02:16.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.941+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc0c072510 con 0x7fdc0c06ade0 2023-10-16T10:02:16.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.942+0000 7fdc097fa700 1 -- 172.21.15.181:0/1032775577 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc000598f0 con 0x7fdc0c06ade0 2023-10-16T10:02:16.944 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:02:16.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fdbf006b020 msgr2=0x7fdbf006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fdbf006b020 0x7fdbf006d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c159600 tx=0x7fdbf8008040 comp rx=0 tx=0).stop 2023-10-16T10:02:16.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c06ade0 msgr2=0x7fdc0c071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c06ade0 0x7fdc0c071f00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00004750 comp rx=0 tx=0).stop 2023-10-16T10:02:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 shutdown_connections 2023-10-16T10:02:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fdbf006b020 0x7fdbf006d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15dc40 0x7fdc0c0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0c06ade0 0x7fdc0c071f00 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 --2- 172.21.15.181:0/1032775577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0c2bf0 0x7fdc0c06e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 >> 172.21.15.181:0/1032775577 conn(0x7fdc0c0b1780 msgr2=0x7fdc0c0b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 shutdown_connections 2023-10-16T10:02:16.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:16.944+0000 7fdc12cf8700 1 -- 172.21.15.181:0/1032775577 wait complete. 2023-10-16T10:02:16.958 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:02:16.959 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:02:16.960 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T10:02:16.960 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:02:16.961 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:16.961 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:02:16.962 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:02:16.963 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:02:16.963 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:02:16.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:16.965 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T10:02:17.016 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:02:17.016 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:02:17.017 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:02:17.017 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:17.017 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T10:02:17.017 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:17.018 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:02:17.018 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:17.018 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:02:17.018 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:17.018 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:02:17.019 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:17.019 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:02:17.019 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:17.019 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:02:17.019 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41333sec preferred_lft 41333sec 2023-10-16T10:02:17.020 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T10:02:17.020 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:17.020 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:02:17.020 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:02:17.021 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:02:17.021 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:17.022 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T10:02:17.022 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:17.022 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:02:17.022 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:02:17.023 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T10:02:17.023 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:17.023 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:02:17.023 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:02:17.023 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T10:02:17.024 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:02:17.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.025 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:02:17.081 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T10:02:17.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.083 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T10:02:17.143 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T10:02:17.144 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:02:17.144 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:02:17.144 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:02:17.145 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:02:17.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.148 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T10:02:17.148 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T10:02:17.149 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:02:17.149 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:02:17.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:02:17.151 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T10:02:17.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.249 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T10:02:17.267 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:02:17.268 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:02:17.268 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:02:17.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.269 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T10:02:17.334 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:02:17.335 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:17.391 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:02:17.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:02:17.392 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:17.447 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T10:02:17.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.448 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:02:17.532 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:02:17.533 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:02:17.533 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:02:17.542 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:02:17.542 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:02:17.542 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:02:17.543 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:02:17.543 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:02:17.616 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:02:17.616 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:02:17.617 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:02:17.617 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:02:17.617 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:02:17.617 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T10:02:17.617 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T10:02:17.618 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T10:02:17.619 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:02:17.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:02:17.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:17.647 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:02:17.727 INFO:teuthology.orchestra.run.smithi181.stdout:client7354 2023-10-16T10:02:17.728 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:02:17.729 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:02:17.786 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:02:17.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.945+0000 7f668a849700 1 Processor -- start 2023-10-16T10:02:17.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.945+0000 7f668a849700 1 -- start start 2023-10-16T10:02:17.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.945+0000 7f668a849700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f668415df60 0x7f66840cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.945+0000 7f668a849700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66840cddf0 con 0x7f66840d0fc0 2023-10-16T10:02:17.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.945+0000 7f668a849700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66840ccf50 con 0x7f668406a8a0 2023-10-16T10:02:17.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.945+0000 7f668a849700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840af9c0 con 0x7f668415df60 2023-10-16T10:02:17.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f66837fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f66840d0fc0 0x7f66840cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57268/0 (socket says 172.21.15.181:57268) 2023-10-16T10:02:17.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f66837fe700 1 -- 172.21.15.181:0/3221424621 learned_addr learned my addr 172.21.15.181:0/3221424621 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:17.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6683fff700 1 --2- 172.21.15.181:0/3221424621 >> v2:172.21.15.190:3300/0 conn(0x7f668415df60 0x7f66840cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851063124 0 0) 0x7f66840cddf0 con 0x7f66840d0fc0 2023-10-16T10:02:17.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c0035b0 con 0x7f66840d0fc0 2023-10-16T10:02:17.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2685253330 0 0) 0x7f666c0035b0 con 0x7f66840d0fc0 2023-10-16T10:02:17.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 >> v2:172.21.15.190:3300/0 conn(0x7f668415df60 msgr2=0x7f66840cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:17.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 --2- 172.21.15.181:0/3221424621 >> v2:172.21.15.190:3300/0 conn(0x7f668415df60 0x7f66840cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 >> v1:172.21.15.190:6789/0 conn(0x7f668406a8a0 legacy=0x7f668406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840cc600 con 0x7f66840d0fc0 2023-10-16T10:02:17.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f6678001670 con 0x7f66840d0fc0 2023-10-16T10:02:17.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 >> v1:172.21.15.181:6789/0 conn(0x7f66840d0fc0 legacy=0x7f66840cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f666c003a00 0x7f666c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f666c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f666c005210 0x7f666c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c00c080 con 0x7f666c004850 2023-10-16T10:02:17.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c00c1f0 con 0x7f666c005210 2023-10-16T10:02:17.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c00c3b0 con 0x7f666c003a00 2023-10-16T10:02:17.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f66837fe700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f666c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6688de6700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f666c005210 0x7f666c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.946+0000 7f6683fff700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f666c003a00 0x7f666c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f66837fe700 1 -- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f666c003a00 msgr2=0x7f666c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:17.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f66837fe700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f666c003a00 0x7f666c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f66837fe700 1 -- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f666c005210 msgr2=0x7f666c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:17.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f66837fe700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f666c005210 0x7f666c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f66837fe700 1 -- 172.21.15.181:0/3221424621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0035b0 con 0x7f666c004850 2023-10-16T10:02:17.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f6688de6700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f666c005210 0x7f666c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:02:17.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f6688de6700 1 -- 172.21.15.181:0/3221424621 reap_dead start 2023-10-16T10:02:17.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f66837fe700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f666c004cd0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f667800c240 tx=0x7f666c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60e7bd8c99343c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678007c10 con 0x7f666c004850 2023-10-16T10:02:17.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667800cde0 con 0x7f666c004850 2023-10-16T10:02:17.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f6682ffd700 1 -- 172.21.15.181:0/3221424621 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678005ba0 con 0x7f666c004850 2023-10-16T10:02:17.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f668a849700 1 -- 172.21.15.181:0/3221424621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 msgr2=0x7f666c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f668a849700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f666c004cd0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f667800c240 tx=0x7f666c005880 comp rx=0 tx=0).stop 2023-10-16T10:02:17.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f668a849700 1 -- 172.21.15.181:0/3221424621 shutdown_connections 2023-10-16T10:02:17.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f668a849700 1 --2- 172.21.15.181:0/3221424621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f666c004cd0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f668a849700 1 -- 172.21.15.181:0/3221424621 >> 172.21.15.181:0/3221424621 conn(0x7f66840d04a0 msgr2=0x7f66840d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:17.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.947+0000 7f668a849700 1 -- 172.21.15.181:0/3221424621 shutdown_connections 2023-10-16T10:02:17.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.948+0000 7f668a849700 1 -- 172.21.15.181:0/3221424621 wait complete. 2023-10-16T10:02:17.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.949+0000 7f668a849700 1 Processor -- start 2023-10-16T10:02:17.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.949+0000 7f668a849700 1 -- start start 2023-10-16T10:02:17.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f668a849700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f668406dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f668a849700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66840b32f0 0x7f6684147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f668a849700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66840b3770 0x7f6684148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f668a849700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b3cd0 con 0x7f666c004850 2023-10-16T10:02:17.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f668a849700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b09d0 con 0x7f66840b3770 2023-10-16T10:02:17.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f668a849700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b00a0 con 0x7f66840b32f0 2023-10-16T10:02:17.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f668406dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f668406dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46200/0 (socket says 172.21.15.181:46200) 2023-10-16T10:02:17.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 -- 172.21.15.181:0/357571411 learned_addr learned my addr 172.21.15.181:0/357571411 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:17.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f66837fe700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66840b32f0 0x7f6684147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6688de6700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66840b3770 0x7f6684148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 -- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66840b32f0 msgr2=0x7f6684147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66840b32f0 0x7f6684147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 -- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66840b3770 msgr2=0x7f6684148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:17.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66840b3770 0x7f6684148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:17.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 -- 172.21.15.181:0/357571411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840b0380 con 0x7f666c004850 2023-10-16T10:02:17.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6683fff700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f668406dc70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f66740116d0 tx=0x7f66740119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6680ff9700 1 -- 172.21.15.181:0/357571411 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674015870 con 0x7f666c004850 2023-10-16T10:02:17.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6680ff9700 1 -- 172.21.15.181:0/357571411 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66740159d0 con 0x7f666c004850 2023-10-16T10:02:17.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66840b0660 con 0x7f666c004850 2023-10-16T10:02:17.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.950+0000 7f6680ff9700 1 -- 172.21.15.181:0/357571411 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674015d40 con 0x7f666c004850 2023-10-16T10:02:17.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.951+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668406e390 con 0x7f666c004850 2023-10-16T10:02:17.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.952+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66840b01e0 con 0x7f666c004850 2023-10-16T10:02:17.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.952+0000 7f6680ff9700 1 -- 172.21.15.181:0/357571411 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f66740143e0 con 0x7f666c004850 2023-10-16T10:02:17.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.952+0000 7f6680ff9700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f667006b020 0x7f667006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:17.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.952+0000 7f6680ff9700 1 -- 172.21.15.181:0/357571411 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f6674089e80 con 0x7f666c004850 2023-10-16T10:02:17.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.952+0000 7f66837fe700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f667006b020 0x7f667006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:17.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.953+0000 7f66837fe700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f667006b020 0x7f667006d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6678001930 tx=0x7f6678007000 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:17.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:17.957+0000 7f6680ff9700 1 -- 172.21.15.181:0/357571411 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f66840b01e0 con 0x7f666c004850 2023-10-16T10:02:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.177+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 --> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f66840cd1c0 con 0x7f667006b020 2023-10-16T10:02:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.178+0000 7f6680ff9700 1 -- 172.21.15.181:0/357571411 <== mgr.7779 v2:172.21.15.181:6832/2093458419 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6678007b30 con 0x7f667006b020 2023-10-16T10:02:18.180 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:02:18.180 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:02:18.181 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T10:02:18.181 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:02:18.181 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:02:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.180+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f667006b020 msgr2=0x7f667006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.180+0000 7f668a849700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f667006b020 0x7f667006d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6678001930 tx=0x7f6678007000 comp rx=0 tx=0).stop 2023-10-16T10:02:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.180+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 msgr2=0x7f668406dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.180+0000 7f668a849700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f668406dc70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f66740116d0 tx=0x7f66740119e0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 shutdown_connections 2023-10-16T10:02:18.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f667006b020 0x7f667006d4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66840b3770 0x7f6684148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f666c004850 0x7f668406dc70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 --2- 172.21.15.181:0/357571411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66840b32f0 0x7f6684147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 >> 172.21.15.181:0/357571411 conn(0x7f66840d04a0 msgr2=0x7f668406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:18.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 shutdown_connections 2023-10-16T10:02:18.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.181+0000 7f668a849700 1 -- 172.21.15.181:0/357571411 wait complete. 2023-10-16T10:02:18.194 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:02:18.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.316+0000 7fd23e596700 1 Processor -- start 2023-10-16T10:02:18.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.316+0000 7fd23e596700 1 -- start start 2023-10-16T10:02:18.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd23e596700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd23815da70 0x7fd2380ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd23e596700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2380cef00 con 0x7fd23806afb0 2023-10-16T10:02:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd23e596700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2380cec20 con 0x7fd2380ce6a0 2023-10-16T10:02:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd23e596700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd238071c80 con 0x7fd23815da70 2023-10-16T10:02:18.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd237fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd23815da70 0x7fd2380ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd23cb33700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd23806afb0 0x7fd2381615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57290/0 (socket says 172.21.15.181:57290) 2023-10-16T10:02:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd23cb33700 1 -- 172.21.15.181:0/3815805478 learned_addr learned my addr 172.21.15.181:0/3815805478 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd237fff700 1 -- 172.21.15.181:0/3815805478 >> v1:172.21.15.190:6789/0 conn(0x7fd2380ce6a0 legacy=0x7fd23806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd237fff700 1 -- 172.21.15.181:0/3815805478 >> v1:172.21.15.181:6789/0 conn(0x7fd23806afb0 legacy=0x7fd2381615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd237fff700 1 -- 172.21.15.181:0/3815805478 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238071f60 con 0x7fd23815da70 2023-10-16T10:02:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd237fff700 1 --2- 172.21.15.181:0/3815805478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd23815da70 0x7fd2380ce160 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009840 tx=0x7fd22c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727348ca13942608 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd236ffd700 1 -- 172.21.15.181:0/3815805478 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c004230 con 0x7fd23815da70 2023-10-16T10:02:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd236ffd700 1 -- 172.21.15.181:0/3815805478 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c004390 con 0x7fd23815da70 2023-10-16T10:02:18.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.317+0000 7fd236ffd700 1 -- 172.21.15.181:0/3815805478 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0036a0 con 0x7fd23815da70 2023-10-16T10:02:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.318+0000 7fd23e596700 1 -- 172.21.15.181:0/3815805478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd23815da70 msgr2=0x7fd2380ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.318+0000 7fd23e596700 1 --2- 172.21.15.181:0/3815805478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd23815da70 0x7fd2380ce160 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009840 tx=0x7fd22c009b50 comp rx=0 tx=0).stop 2023-10-16T10:02:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.318+0000 7fd23e596700 1 -- 172.21.15.181:0/3815805478 shutdown_connections 2023-10-16T10:02:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.318+0000 7fd23e596700 1 --2- 172.21.15.181:0/3815805478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd23815da70 0x7fd2380ce160 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.318+0000 7fd23e596700 1 -- 172.21.15.181:0/3815805478 >> 172.21.15.181:0/3815805478 conn(0x7fd2380cd010 msgr2=0x7fd2380cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.318+0000 7fd23e596700 1 -- 172.21.15.181:0/3815805478 shutdown_connections 2023-10-16T10:02:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.318+0000 7fd23e596700 1 -- 172.21.15.181:0/3815805478 wait complete. 2023-10-16T10:02:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.319+0000 7fd23e596700 1 Processor -- start 2023-10-16T10:02:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.319+0000 7fd23e596700 1 -- start start 2023-10-16T10:02:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd23e596700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd23806afb0 0x7fd2380b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd23e596700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2380ce6a0 0x7fd2380ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd23e596700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd23815da70 0x7fd23814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd23e596700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2380adef0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd23e596700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2380aecf0 con 0x7fd23815da70 2023-10-16T10:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd23e596700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23814ff60 con 0x7fd23806afb0 2023-10-16T10:02:18.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd2377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2380ce6a0 0x7fd2380ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd2377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2380ce6a0 0x7fd2380ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46216/0 (socket says 172.21.15.181:46216) 2023-10-16T10:02:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd2377fe700 1 -- 172.21.15.181:0/2322953581 learned_addr learned my addr 172.21.15.181:0/2322953581 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd23cb33700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd23815da70 0x7fd23814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.320+0000 7fd237fff700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd23806afb0 0x7fd2380b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd2377fe700 1 -- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd23806afb0 msgr2=0x7fd2380b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd2377fe700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd23806afb0 0x7fd2380b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd2377fe700 1 -- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd23815da70 msgr2=0x7fd23814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd2377fe700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd23815da70 0x7fd23814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd2377fe700 1 -- 172.21.15.181:0/2322953581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c0094a0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd237fff700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd23806afb0 0x7fd2380b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd2377fe700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2380ce6a0 0x7fd2380ad9b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c4d0 tx=0x7fd22800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd234ff9700 1 -- 172.21.15.181:0/2322953581 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228011790 con 0x7fd2380ce6a0 2023-10-16T10:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd234ff9700 1 -- 172.21.15.181:0/2322953581 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2280118f0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2380ae0e0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd234ff9700 1 -- 172.21.15.181:0/2322953581 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228011ca0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.321+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2380ae6a0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.322+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23806f6c0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.322+0000 7fd234ff9700 1 -- 172.21.15.181:0/2322953581 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 64) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd22801c460 con 0x7fd2380ce6a0 2023-10-16T10:02:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.323+0000 7fd234ff9700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd22406b020 0x7fd22406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:18.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.323+0000 7fd234ff9700 1 -- 172.21.15.181:0/2322953581 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fd2280897e0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.323+0000 7fd237fff700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd22406b020 0x7fd22406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.323+0000 7fd237fff700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd22406b020 0x7fd22406d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009b90 tx=0x7fd22c008030 comp rx=0 tx=0).ready entity=mgr.7779 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:18.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.327+0000 7fd234ff9700 1 -- 172.21.15.181:0/2322953581 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd2280545d0 con 0x7fd2380ce6a0 2023-10-16T10:02:18.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:18.577+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd23806acb0 con 0x7fd2380ce6a0 2023-10-16T10:02:19.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.018+0000 7fd234ff9700 1 -- 172.21.15.181:0/2322953581 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v65) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd228057bf0 con 0x7fd2380ce6a0 2023-10-16T10:02:19.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd22406b020 msgr2=0x7fd22406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd22406b020 0x7fd22406d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009b90 tx=0x7fd22c008030 comp rx=0 tx=0).stop 2023-10-16T10:02:19.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2380ce6a0 msgr2=0x7fd2380ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2380ce6a0 0x7fd2380ad9b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c4d0 tx=0x7fd22800c890 comp rx=0 tx=0).stop 2023-10-16T10:02:19.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 shutdown_connections 2023-10-16T10:02:19.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd22406b020 0x7fd22406d4e0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009b90 tx=0x7fd22c008030 comp rx=0 tx=0).stop 2023-10-16T10:02:19.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd23806afb0 0x7fd2380b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd23815da70 0x7fd23814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 --2- 172.21.15.181:0/2322953581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2380ce6a0 0x7fd2380ad9b0 secure :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c4d0 tx=0x7fd22800c890 comp rx=0 tx=0).stop 2023-10-16T10:02:19.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.021+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 >> 172.21.15.181:0/2322953581 conn(0x7fd2380cd010 msgr2=0x7fd2380cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.022+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 shutdown_connections 2023-10-16T10:02:19.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.023+0000 7fd23e596700 1 -- 172.21.15.181:0/2322953581 wait complete. 2023-10-16T10:02:19.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:02:19.165 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:19.164+0000 7f05cb680200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:02:19.166 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:19.165+0000 7faa8c6fe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:02:19.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.166+0000 7faf574ee700 1 Processor -- start 2023-10-16T10:02:19.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.166+0000 7faf574ee700 1 -- start start 2023-10-16T10:02:19.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf574ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf500cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf574ee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf50071be0 0x7faf500b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf574ee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500ce1f0 con 0x7faf5006f3b0 2023-10-16T10:02:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf574ee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500b2930 con 0x7faf50071be0 2023-10-16T10:02:19.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf574ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf500ccda0 con 0x7faf500b2010 2023-10-16T10:02:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf54a89700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf50071be0 0x7faf500b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf54a89700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf50071be0 0x7faf500b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46250/0 (socket says 172.21.15.181:46250) 2023-10-16T10:02:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf54a89700 1 -- 172.21.15.181:0/2216688698 learned_addr learned my addr 172.21.15.181:0/2216688698 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:19.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf5528a700 1 --2- 172.21.15.181:0/2216688698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf500cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf54a89700 1 -- 172.21.15.181:0/2216688698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 msgr2=0x7faf500cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf54a89700 1 --2- 172.21.15.181:0/2216688698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf500cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf54a89700 1 -- 172.21.15.181:0/2216688698 >> v1:172.21.15.181:6789/0 conn(0x7faf5006f3b0 legacy=0x7faf5006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.167+0000 7faf54a89700 1 -- 172.21.15.181:0/2216688698 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500727e0 con 0x7faf50071be0 2023-10-16T10:02:19.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf54a89700 1 --2- 172.21.15.181:0/2216688698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf50071be0 0x7faf500b23f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7faf400098f0 tx=0x7faf40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668719a3452d8387 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf47fff700 1 -- 172.21.15.181:0/2216688698 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40011830 con 0x7faf50071be0 2023-10-16T10:02:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf47fff700 1 -- 172.21.15.181:0/2216688698 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40011990 con 0x7faf50071be0 2023-10-16T10:02:19.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf47fff700 1 -- 172.21.15.181:0/2216688698 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40011d20 con 0x7faf50071be0 2023-10-16T10:02:19.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf574ee700 1 -- 172.21.15.181:0/2216688698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf50071be0 msgr2=0x7faf500b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf574ee700 1 --2- 172.21.15.181:0/2216688698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf50071be0 0x7faf500b23f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7faf400098f0 tx=0x7faf40009c00 comp rx=0 tx=0).stop 2023-10-16T10:02:19.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf574ee700 1 -- 172.21.15.181:0/2216688698 shutdown_connections 2023-10-16T10:02:19.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf574ee700 1 --2- 172.21.15.181:0/2216688698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf500cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf574ee700 1 --2- 172.21.15.181:0/2216688698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf50071be0 0x7faf500b23f0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.168+0000 7faf574ee700 1 -- 172.21.15.181:0/2216688698 >> 172.21.15.181:0/2216688698 conn(0x7faf500cd010 msgr2=0x7faf500cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.169+0000 7faf574ee700 1 -- 172.21.15.181:0/2216688698 shutdown_connections 2023-10-16T10:02:19.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.169+0000 7faf574ee700 1 -- 172.21.15.181:0/2216688698 wait complete. 2023-10-16T10:02:19.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.170+0000 7faf574ee700 1 Processor -- start 2023-10-16T10:02:19.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.170+0000 7faf574ee700 1 -- start start 2023-10-16T10:02:19.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf574ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5006f3b0 0x7faf5014d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf574ee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf50071be0 0x7faf500c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf574ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf50152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf574ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500c7af0 con 0x7faf5006f3b0 2023-10-16T10:02:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf574ee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500c7dd0 con 0x7faf50071be0 2023-10-16T10:02:19.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf574ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf500c80b0 con 0x7faf500b2010 2023-10-16T10:02:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf5528a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5006f3b0 0x7faf5014d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf5528a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5006f3b0 0x7faf5014d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46254/0 (socket says 172.21.15.181:46254) 2023-10-16T10:02:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.171+0000 7faf5528a700 1 -- 172.21.15.181:0/1174072617 learned_addr learned my addr 172.21.15.181:0/1174072617 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf54a89700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf50071be0 0x7faf500c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf55a8b700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf50152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf5528a700 1 -- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 msgr2=0x7faf50152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf5528a700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf50152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf5528a700 1 -- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf50071be0 msgr2=0x7faf500c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf5528a700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf50071be0 0x7faf500c1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf5528a700 1 -- 172.21.15.181:0/1174072617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40009550 con 0x7faf5006f3b0 2023-10-16T10:02:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf5528a700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5006f3b0 0x7faf5014d440 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e9b0 tx=0x7faf4c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf45ffb700 1 -- 172.21.15.181:0/1174072617 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c009710 con 0x7faf5006f3b0 2023-10-16T10:02:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf45ffb700 1 -- 172.21.15.181:0/1174072617 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c00ee50 con 0x7faf5006f3b0 2023-10-16T10:02:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf500c8390 con 0x7faf5006f3b0 2023-10-16T10:02:19.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf45ffb700 1 -- 172.21.15.181:0/1174072617 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00ca50 con 0x7faf5006f3b0 2023-10-16T10:02:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.172+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf500c2450 con 0x7faf5006f3b0 2023-10-16T10:02:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.173+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf500c89e0 con 0x7faf5006f3b0 2023-10-16T10:02:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.174+0000 7faf45ffb700 1 -- 172.21.15.181:0/1174072617 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 65) v1 ==== 89348+0+0 (secure 0 0 0) 0x7faf4c00cbb0 con 0x7faf5006f3b0 2023-10-16T10:02:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.174+0000 7faf45ffb700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 0x7faf3406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.174+0000 7faf54a89700 1 -- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 msgr2=0x7faf3406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.174+0000 7faf45ffb700 1 -- 172.21.15.181:0/1174072617 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7faf4c089890 con 0x7faf5006f3b0 2023-10-16T10:02:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.174+0000 7faf54a89700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 0x7faf3406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:02:19.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.179+0000 7faf45ffb700 1 -- 172.21.15.181:0/1174072617 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faf4c054710 con 0x7faf5006f3b0 2023-10-16T10:02:19.232 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:19.231+0000 7f05cb680200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:19.237 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:19.235+0000 7faa8c6fe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:19.304 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:19.303+0000 7f05cb680200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:02:19.310 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:19.309+0000 7faa8c6fe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:02:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.375+0000 7faf54a89700 1 -- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 msgr2=0x7faf3406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:19.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.375+0000 7faf54a89700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 0x7faf3406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:02:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.379+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7faf500d13e0 con 0x7faf5006f3b0 2023-10-16T10:02:19.382 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:19.381+0000 7f05cb680200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:02:19.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.382+0000 7faf45ffb700 1 -- 172.21.15.181:0/1174072617 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7faf4c0210d0 con 0x7faf5006f3b0 2023-10-16T10:02:19.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 msgr2=0x7faf3406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:02:19.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 0x7faf3406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5006f3b0 msgr2=0x7faf5014d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5006f3b0 0x7faf5014d440 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e9b0 tx=0x7faf4c00ecc0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 shutdown_connections 2023-10-16T10:02:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7faf3406b070 0x7faf3406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf500b2010 0x7faf50152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5006f3b0 0x7faf5014d440 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 --2- 172.21.15.181:0/1174072617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf50071be0 0x7faf500c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 >> 172.21.15.181:0/1174072617 conn(0x7faf500cd010 msgr2=0x7faf500b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 shutdown_connections 2023-10-16T10:02:19.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.384+0000 7faf574ee700 1 -- 172.21.15.181:0/1174072617 wait complete. 2023-10-16T10:02:19.390 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:19.389+0000 7faa8c6fe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:02:19.399 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:02:19.522 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:19.521+0000 7f05cb680200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:02:19.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.524+0000 7f3b0ea31700 1 Processor -- start 2023-10-16T10:02:19.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.524+0000 7f3b0ea31700 1 -- start start 2023-10-16T10:02:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0ea31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0ea31700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b080ca300 con 0x7f3b080cdd00 2023-10-16T10:02:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0ea31700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0806b6c0 con 0x7f3b080666a0 2023-10-16T10:02:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0ea31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0806e5e0 con 0x7f3b080cc050 2023-10-16T10:02:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b07fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3b080cdd00 0x7f3b0806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57386/0 (socket says 172.21.15.181:57386) 2023-10-16T10:02:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b07fff700 1 -- 172.21.15.181:0/605443566 learned_addr learned my addr 172.21.15.181:0/605443566 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0cfce700 1 --2- 172.21.15.181:0/605443566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b06ffd700 1 -- 172.21.15.181:0/605443566 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889098169 0 0) 0x7f3b080ca300 con 0x7f3b080cdd00 2023-10-16T10:02:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b06ffd700 1 -- 172.21.15.181:0/605443566 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af00035b0 con 0x7f3b080cdd00 2023-10-16T10:02:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b06ffd700 1 -- 172.21.15.181:0/605443566 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571521340 0 0) 0x7f3b0806b6c0 con 0x7f3b080666a0 2023-10-16T10:02:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b06ffd700 1 -- 172.21.15.181:0/605443566 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b080ca300 con 0x7f3b080666a0 2023-10-16T10:02:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0cfce700 1 -- 172.21.15.181:0/605443566 >> v1:172.21.15.190:6789/0 conn(0x7f3b080666a0 legacy=0x7f3b0806afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0cfce700 1 -- 172.21.15.181:0/605443566 >> v1:172.21.15.181:6789/0 conn(0x7f3b080cdd00 legacy=0x7f3b0806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.525+0000 7f3b0cfce700 1 -- 172.21.15.181:0/605443566 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080ca580 con 0x7f3b080cc050 2023-10-16T10:02:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0cfce700 1 --2- 172.21.15.181:0/605443566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0806f5f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009dc0 tx=0x7f3b0000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df93e854d0a81d65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b06ffd700 1 -- 172.21.15.181:0/605443566 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000e070 con 0x7f3b080cc050 2023-10-16T10:02:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b06ffd700 1 -- 172.21.15.181:0/605443566 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0000bb20 con 0x7f3b080cc050 2023-10-16T10:02:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b06ffd700 1 -- 172.21.15.181:0/605443566 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00012430 con 0x7f3b080cc050 2023-10-16T10:02:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0ea31700 1 -- 172.21.15.181:0/605443566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 msgr2=0x7f3b0806f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/605443566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0806f5f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009dc0 tx=0x7f3b0000b330 comp rx=0 tx=0).stop 2023-10-16T10:02:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0ea31700 1 -- 172.21.15.181:0/605443566 shutdown_connections 2023-10-16T10:02:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/605443566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0806f5f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0ea31700 1 -- 172.21.15.181:0/605443566 >> 172.21.15.181:0/605443566 conn(0x7f3b080cc730 msgr2=0x7f3b080ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0ea31700 1 -- 172.21.15.181:0/605443566 shutdown_connections 2023-10-16T10:02:19.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.526+0000 7f3b0ea31700 1 -- 172.21.15.181:0/605443566 wait complete. 2023-10-16T10:02:19.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.528+0000 7f3b0ea31700 1 Processor -- start 2023-10-16T10:02:19.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.528+0000 7f3b0ea31700 1 -- start start 2023-10-16T10:02:19.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b0ea31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b080666a0 0x7f3b08147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b0ea31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b0ea31700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080cdd00 0x7f3b0815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b0ea31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0815b870 con 0x7f3b080666a0 2023-10-16T10:02:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b0ea31700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b081e8670 con 0x7f3b080cdd00 2023-10-16T10:02:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b0ea31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b081e8950 con 0x7f3b080cc050 2023-10-16T10:02:19.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b080666a0 0x7f3b08147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b080666a0 0x7f3b08147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46314/0 (socket says 172.21.15.181:46314) 2023-10-16T10:02:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 -- 172.21.15.181:0/3154873845 learned_addr learned my addr 172.21.15.181:0/3154873845 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b077fe700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b0cfce700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080cdd00 0x7f3b0815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 -- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 msgr2=0x7f3b0815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 -- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080cdd00 msgr2=0x7f3b0815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080cdd00 0x7f3b0815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.529+0000 7f3b07fff700 1 -- 172.21.15.181:0/3154873845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00009a70 con 0x7f3b080666a0 2023-10-16T10:02:19.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.530+0000 7f3b07fff700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b080666a0 0x7f3b08147c80 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c700 tx=0x7f3af800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.530+0000 7f3b04ff9700 1 -- 172.21.15.181:0/3154873845 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8005d70 con 0x7f3b080666a0 2023-10-16T10:02:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.530+0000 7f3b04ff9700 1 -- 172.21.15.181:0/3154873845 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af80070d0 con 0x7f3b080666a0 2023-10-16T10:02:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.530+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b081e8c30 con 0x7f3b080666a0 2023-10-16T10:02:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.530+0000 7f3b04ff9700 1 -- 172.21.15.181:0/3154873845 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8007460 con 0x7f3b080666a0 2023-10-16T10:02:19.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.530+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b081e91f0 con 0x7f3b080666a0 2023-10-16T10:02:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.531+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b081483c0 con 0x7f3b080666a0 2023-10-16T10:02:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.531+0000 7f3b04ff9700 1 -- 172.21.15.181:0/3154873845 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 65) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f3af80075c0 con 0x7f3b080666a0 2023-10-16T10:02:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.532+0000 7f3b04ff9700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 0x7f3af406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.532+0000 7f3b077fe700 1 -- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 msgr2=0x7f3af406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.532+0000 7f3b04ff9700 1 -- 172.21.15.181:0/3154873845 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f3af8088d10 con 0x7f3b080666a0 2023-10-16T10:02:19.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.532+0000 7f3b077fe700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 0x7f3af406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:02:19.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.540+0000 7f3b04ff9700 1 -- 172.21.15.181:0/3154873845 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b081483c0 con 0x7f3b080666a0 2023-10-16T10:02:19.546 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:19.544+0000 7faa8c6fe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:02:19.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.733+0000 7f3b077fe700 1 -- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 msgr2=0x7f3af406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:19.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.733+0000 7f3b077fe700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 0x7f3af406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:02:19.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.740+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f3b081483c0 con 0x7f3b080666a0 2023-10-16T10:02:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.743+0000 7f3b04ff9700 1 -- 172.21.15.181:0/3154873845 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3b081483c0 con 0x7f3b080666a0 2023-10-16T10:02:19.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 msgr2=0x7f3af406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:02:19.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 0x7f3af406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b080666a0 msgr2=0x7f3b08147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b080666a0 0x7f3b08147c80 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c700 tx=0x7f3af800ca10 comp rx=0 tx=0).stop 2023-10-16T10:02:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 shutdown_connections 2023-10-16T10:02:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7f3af406b070 0x7f3af406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080cdd00 0x7f3b0815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b080666a0 0x7f3b08147c80 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 --2- 172.21.15.181:0/3154873845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b080cc050 0x7f3b0815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 >> 172.21.15.181:0/3154873845 conn(0x7f3b080cc730 msgr2=0x7f3b080ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 shutdown_connections 2023-10-16T10:02:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.745+0000 7f3b0ea31700 1 -- 172.21.15.181:0/3154873845 wait complete. 2023-10-16T10:02:19.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:19.760 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T10:02:19.781 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T10:02:19.847 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:19.846+0000 7f05cb680200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:02:19.888 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:19.886+0000 7faa8c6fe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:02:19.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.946+0000 7fda004b8700 1 Processor -- start 2023-10-16T10:02:19.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.946+0000 7fda004b8700 1 -- start start 2023-10-16T10:02:19.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fda004b8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9f80ce6c0 0x7fd9f80cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fda004b8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f80b1f70 con 0x7fd9f815f150 2023-10-16T10:02:19.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fda004b8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f80b2ae0 con 0x7fd9f80cc6a0 2023-10-16T10:02:19.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fda004b8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f80b2dc0 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fd9fe254700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd9f815f150 0x7fd9f80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57446/0 (socket says 172.21.15.181:57446) 2023-10-16T10:02:19.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fd9fe254700 1 -- 172.21.15.181:0/961622420 learned_addr learned my addr 172.21.15.181:0/961622420 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:19.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fd9fda53700 1 --2- 172.21.15.181:0/961622420 >> v2:172.21.15.190:3300/0 conn(0x7fd9f80ce6c0 0x7fd9f80cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.947+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862095442 0 0) 0x7fd9f80b1f70 con 0x7fd9f815f150 2023-10-16T10:02:19.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e40035b0 con 0x7fd9f815f150 2023-10-16T10:02:19.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168910595 0 0) 0x7fd9f80b2ae0 con 0x7fd9f80cc6a0 2023-10-16T10:02:19.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f80b1f70 con 0x7fd9f80cc6a0 2023-10-16T10:02:19.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fda53700 1 -- 172.21.15.181:0/961622420 >> v1:172.21.15.190:6789/0 conn(0x7fd9f80cc6a0 legacy=0x7fd9f80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fda53700 1 -- 172.21.15.181:0/961622420 >> v1:172.21.15.181:6789/0 conn(0x7fd9f815f150 legacy=0x7fd9f80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342136413 0 0) 0x7fd9e40035b0 con 0x7fd9f815f150 2023-10-16T10:02:19.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fda53700 1 -- 172.21.15.181:0/961622420 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80cf9d0 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fda53700 1 --2- 172.21.15.181:0/961622420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f80cc160 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009840 tx=0x7fd9f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c172b95f20088961 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4011830 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4011990 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.948+0000 7fd9fd252700 1 -- 172.21.15.181:0/961622420 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4011d20 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.949+0000 7fda004b8700 1 -- 172.21.15.181:0/961622420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f80ce6c0 msgr2=0x7fd9f80cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.949+0000 7fda004b8700 1 --2- 172.21.15.181:0/961622420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f80cc160 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009840 tx=0x7fd9f4009b50 comp rx=0 tx=0).stop 2023-10-16T10:02:19.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.949+0000 7fda004b8700 1 -- 172.21.15.181:0/961622420 shutdown_connections 2023-10-16T10:02:19.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.949+0000 7fda004b8700 1 --2- 172.21.15.181:0/961622420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f80cc160 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.949+0000 7fda004b8700 1 -- 172.21.15.181:0/961622420 >> 172.21.15.181:0/961622420 conn(0x7fd9f809a930 msgr2=0x7fd9f80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:19.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.949+0000 7fda004b8700 1 -- 172.21.15.181:0/961622420 shutdown_connections 2023-10-16T10:02:19.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.949+0000 7fda004b8700 1 -- 172.21.15.181:0/961622420 wait complete. 2023-10-16T10:02:19.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.950+0000 7fda004b8700 1 Processor -- start 2023-10-16T10:02:19.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.950+0000 7fda004b8700 1 -- start start 2023-10-16T10:02:19.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fda004b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f80cc6a0 0x7fd9f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fda004b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f81f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fda004b8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f815f150 0x7fd9f806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fda004b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f806da20 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fda004b8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80c5440 con 0x7fd9f815f150 2023-10-16T10:02:19.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fda004b8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f81f4290 con 0x7fd9f80cc6a0 2023-10-16T10:02:19.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f81f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f81f07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46376/0 (socket says 172.21.15.181:46376) 2023-10-16T10:02:19.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 -- 172.21.15.181:0/391720397 learned_addr learned my addr 172.21.15.181:0/391720397 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:19.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fea55700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f815f150 0x7fd9f806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f80cc6a0 0x7fd9f806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:19.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f80cc6a0 msgr2=0x7fd9f806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9f80cc6a0 0x7fd9f806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f815f150 msgr2=0x7fd9f806b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:19.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9f815f150 0x7fd9f806b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:19.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 -- 172.21.15.181:0/391720397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40094a0 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.951+0000 7fd9fda53700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f81f07a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:19.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.952+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f401a6e0 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.952+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f401a840 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.952+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f401abd0 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.952+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f80ca960 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.952+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f80caf20 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.953+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f80b0700 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.953+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 65) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fd9f4022070 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.953+0000 7fd9eeffd700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 0x7fd9dc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:19.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.954+0000 7fd9fe254700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 msgr2=0x7fd9dc06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:19.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.954+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6133+0+0 (secure 0 0 0) 0x7fd9f408a4f0 con 0x7fd9f80ce6c0 2023-10-16T10:02:19.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.954+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 0x7fd9dc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:02:19.964 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:19.961+0000 7f05cb680200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:02:19.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:19.958+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd9f4055370 con 0x7fd9f80ce6c0 2023-10-16T10:02:20.009 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:20.008+0000 7faa8c6fe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:02:20.036 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:20.035+0000 7f05cb680200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:02:20.085 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:20.084+0000 7faa8c6fe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:02:20.115 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:20.114+0000 7f05cb680200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:02:20.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:20.155+0000 7fd9fe254700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 msgr2=0x7fd9dc06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:20.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:20.155+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 0x7fd9dc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:02:20.167 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:20.166+0000 7faa8c6fe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:02:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:20.183+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 --> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9f80cff90 con 0x7fd9dc06b070 2023-10-16T10:02:20.483 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:20.482+0000 7f05cb680200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:02:20.537 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:20.535+0000 7faa8c6fe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:02:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:20.556+0000 7fd9fe254700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 msgr2=0x7fd9dc06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:20.556+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 0x7fd9dc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:02:21.048 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:21.047+0000 7f05cb680200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:02:21.106 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:21.105+0000 7faa8c6fe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:02:21.117 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:21.116+0000 7f05cb680200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:02:21.176 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:21.174+0000 7faa8c6fe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:02:21.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:21.357+0000 7fd9fe254700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 msgr2=0x7fd9dc06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:21.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:21.357+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 0x7fd9dc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:02:21.748 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:21.746+0000 7f05cb680200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:21.806 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:21.805+0000 7faa8c6fe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:21.825 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:21.823+0000 7f05cb680200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:02:21.884 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:21.882+0000 7faa8c6fe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:02:21.969 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:21.967+0000 7f05cb680200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:22.028 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:22.026+0000 7faa8c6fe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:02:22.196 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:22.194+0000 7f05cb680200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:02:22.258 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:22.255+0000 7faa8c6fe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:02:22.269 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:22.267+0000 7f05cb680200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:02:22.330 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:22.329+0000 7faa8c6fe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:02:22.534 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:22.532+0000 7f05cb680200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:02:22.595 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:22.594+0000 7faa8c6fe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:02:22.752 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:22.750+0000 7f05cb680200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:02:22.817 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:22.815+0000 7faa8c6fe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:02:22.900 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:22.898+0000 7f05cb680200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:02:22.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:22.958+0000 7fd9fe254700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 msgr2=0x7fd9dc06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2093458419 2023-10-16T10:02:22.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:22.958+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 0x7fd9dc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:02:22.966 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:22.965+0000 7faa8c6fe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:02:23.012 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:23.010+0000 7f05cb680200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:02:23.079 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:23.078+0000 7faa8c6fe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:02:23.167 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:23.166+0000 7f05cb680200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:02:23.236 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:23.234+0000 7faa8c6fe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:02:23.430 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:23.428+0000 7f05cb680200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:23.501 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:23.498+0000 7faa8c6fe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:02:23.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:23.588+0000 7f05cb680200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:02:23.660 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:23.659+0000 7faa8c6fe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:02:24.111 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:02:24.110+0000 7f05cb680200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:02:24.189 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.188+0000 7faa8c6fe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:02:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:24.209+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 66) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fd9f401ad30 con 0x7fd9f80ce6c0 2023-10-16T10:02:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:24.209+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 msgr2=0x7fd9dc06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:02:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:24.209+0000 7fd9eeffd700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/2093458419,v1:172.21.15.181:6833/2093458419] conn(0x7fd9dc06b070 0x7fd9dc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:24.380 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.379+0000 7fa9ff6d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.381 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.379+0000 7fa9ff6d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.381 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.379+0000 7fa9ff6d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.381 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.379+0000 7fa9ff6d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.381 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.379+0000 7fa9ff6d9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.382 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.380+0000 7faa026df700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.382 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.380+0000 7faa026df700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.382 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.380+0000 7faa026df700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.382 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.380+0000 7faa026df700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:24.382 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:02:24.380+0000 7faa026df700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:02:25.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.214+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 67) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd9f405f0f0 con 0x7fd9f80ce6c0 2023-10-16T10:02:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.214+0000 7fd9eeffd700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fd9dc035910 0x7fd9dc075530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.214+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 --> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9dc071070 con 0x7fd9dc035910 2023-10-16T10:02:25.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.216+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fd9dc035910 0x7fd9dc075530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.218+0000 7fd9fe254700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fd9dc035910 0x7fd9dc075530 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8007fd0 tx=0x7fd9e8009040 comp rx=0 tx=0).ready entity=mgr.8208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:25.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.285+0000 7fd9eeffd700 1 -- 172.21.15.181:0/391720397 <== mgr.8208 v2:172.21.15.181:6832/716040864 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd9dc071070 con 0x7fd9dc035910 2023-10-16T10:02:25.286 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1 2023-10-16T10:02:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fd9dc035910 msgr2=0x7fd9dc075530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fd9dc035910 0x7fd9dc075530 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8007fd0 tx=0x7fd9e8009040 comp rx=0 tx=0).stop 2023-10-16T10:02:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9f80ce6c0 msgr2=0x7fd9f81f07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 --2- 172.21.15.181:0/391720397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9f80ce6c0 0x7fd9f81f07a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4004750 comp rx=0 tx=0).stop 2023-10-16T10:02:25.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fd9fea55700 1 -- 172.21.15.181:0/391720397 reap_dead start 2023-10-16T10:02:25.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 shutdown_connections 2023-10-16T10:02:25.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 >> 172.21.15.181:0/391720397 conn(0x7fd9f809a930 msgr2=0x7fd9f815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:25.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 shutdown_connections 2023-10-16T10:02:25.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.287+0000 7fda004b8700 1 -- 172.21.15.181:0/391720397 wait complete. 2023-10-16T10:02:25.301 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T10:02:25.302 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-10_03_00 in ~35s... 2023-10-16T10:02:25.302 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-16T10:02:25.303 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T10:02:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.426+0000 7f86b9b91700 1 Processor -- start 2023-10-16T10:02:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.426+0000 7f86b9b91700 1 -- start start 2023-10-16T10:02:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.427+0000 7f86b9b91700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb2b0 0x7f86b406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.427+0000 7f86b9b91700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86b40cb690 0x7f86b40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.427+0000 7f86b9b91700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b4070590 con 0x7f86b40b2ba0 2023-10-16T10:02:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.427+0000 7f86b9b91700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b4156330 con 0x7f86b40cb690 2023-10-16T10:02:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.427+0000 7f86b9b91700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b406fcf0 con 0x7f86b40cb2b0 2023-10-16T10:02:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.427+0000 7f86b37fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb2b0 0x7f86b406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.427+0000 7f86b2ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86b40cb690 0x7f86b40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b3fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f86b40b2ba0 0x7f86b415f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:41616/0 (socket says 172.21.15.181:41616) 2023-10-16T10:02:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b3fff700 1 -- 172.21.15.181:0/1910859760 learned_addr learned my addr 172.21.15.181:0/1910859760 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:02:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b2ffd700 1 -- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb2b0 msgr2=0x7f86b406f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:02:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b2ffd700 1 --2- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb2b0 0x7f86b406f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b2ffd700 1 -- 172.21.15.181:0/1910859760 >> v1:172.21.15.190:6789/0 conn(0x7f86b40b2ba0 legacy=0x7f86b415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b2ffd700 1 -- 172.21.15.181:0/1910859760 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b406ffd0 con 0x7f86b40cb690 2023-10-16T10:02:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b27fc700 1 -- 172.21.15.181:0/1910859760 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773887005 0 0) 0x7f86b4070590 con 0x7f86b40b2ba0 2023-10-16T10:02:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b37fe700 1 --2- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb2b0 0x7f86b406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:02:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b2ffd700 1 --2- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb690 0x7f86b40b2660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009840 tx=0x7f86a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ccda336ae2da35 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b27fc700 1 -- 172.21.15.181:0/1910859760 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4011830 con 0x7f86b40cb690 2023-10-16T10:02:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b27fc700 1 -- 172.21.15.181:0/1910859760 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a4011990 con 0x7f86b40cb690 2023-10-16T10:02:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.428+0000 7f86b27fc700 1 -- 172.21.15.181:0/1910859760 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4011d20 con 0x7f86b40cb690 2023-10-16T10:02:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 -- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb690 msgr2=0x7f86b40b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 --2- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb690 0x7f86b40b2660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009840 tx=0x7f86a4009b50 comp rx=0 tx=0).stop 2023-10-16T10:02:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 -- 172.21.15.181:0/1910859760 shutdown_connections 2023-10-16T10:02:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 --2- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb2b0 0x7f86b406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 --2- 172.21.15.181:0/1910859760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb690 0x7f86b40b2660 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 -- 172.21.15.181:0/1910859760 >> 172.21.15.181:0/1910859760 conn(0x7f86b40cd010 msgr2=0x7f86b40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 -- 172.21.15.181:0/1910859760 shutdown_connections 2023-10-16T10:02:25.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.429+0000 7f86b9b91700 1 -- 172.21.15.181:0/1910859760 wait complete. 2023-10-16T10:02:25.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.430+0000 7f86b9b91700 1 Processor -- start 2023-10-16T10:02:25.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.430+0000 7f86b9b91700 1 -- start start 2023-10-16T10:02:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b9b91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b40b2ba0 0x7f86b41f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b9b91700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb2b0 0x7f86b41506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b9b91700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb690 0x7f86b40b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b9b91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b41f2c00 con 0x7f86b40b2ba0 2023-10-16T10:02:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b9b91700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b414f9a0 con 0x7f86b40cb2b0 2023-10-16T10:02:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b9b91700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b4150240 con 0x7f86b40cb690 2023-10-16T10:02:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b40b2ba0 0x7f86b41f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b40b2ba0 0x7f86b41f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46600/0 (socket says 172.21.15.181:46600) 2023-10-16T10:02:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 -- 172.21.15.181:0/671443678 learned_addr learned my addr 172.21.15.181:0/671443678 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:02:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b2ffd700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb2b0 0x7f86b41506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b3fff700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb690 0x7f86b40b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 -- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb690 msgr2=0x7f86b40b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb690 0x7f86b40b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 -- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb2b0 msgr2=0x7f86b41506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb2b0 0x7f86b41506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.431+0000 7f86b37fe700 1 -- 172.21.15.181:0/671443678 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0094a0 con 0x7f86b40b2ba0 2023-10-16T10:02:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.432+0000 7f86b37fe700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b40b2ba0 0x7f86b41f26c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f869c00ebe0 tx=0x7f869c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.432+0000 7f86abfff700 1 -- 172.21.15.181:0/671443678 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c00c850 con 0x7f86b40b2ba0 2023-10-16T10:02:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.432+0000 7f86abfff700 1 -- 172.21.15.181:0/671443678 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c00c9b0 con 0x7f86b40b2ba0 2023-10-16T10:02:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.432+0000 7f86abfff700 1 -- 172.21.15.181:0/671443678 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c00cd40 con 0x7f86b40b2ba0 2023-10-16T10:02:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.432+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a40094a0 con 0x7f86b40b2ba0 2023-10-16T10:02:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.432+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b414fbf0 con 0x7f86b40b2ba0 2023-10-16T10:02:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.433+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b4150520 con 0x7f86b40b2ba0 2023-10-16T10:02:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.433+0000 7f86abfff700 1 -- 172.21.15.181:0/671443678 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 67) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f869c013470 con 0x7f86b40b2ba0 2023-10-16T10:02:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.434+0000 7f86abfff700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f869406ada0 0x7f869406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:02:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.434+0000 7f86abfff700 1 -- 172.21.15.181:0/671443678 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f869c0895d0 con 0x7f86b40b2ba0 2023-10-16T10:02:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.434+0000 7f86b2ffd700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f869406ada0 0x7f869406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:02:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.434+0000 7f86b2ffd700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f869406ada0 0x7f869406d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86a40090b0 tx=0x7f86a401d040 comp rx=0 tx=0).ready entity=mgr.8208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:02:25.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.438+0000 7f86abfff700 1 -- 172.21.15.181:0/671443678 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f869c0546c0 con 0x7f86b40b2ba0 2023-10-16T10:02:25.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.658+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 --> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86b40b2140 con 0x7f869406ada0 2023-10-16T10:02:25.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.666+0000 7f86abfff700 1 -- 172.21.15.181:0/671443678 <== mgr.8208 v2:172.21.15.181:6832/716040864 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f86a4011630 con 0x7f869406ada0 2023-10-16T10:02:25.667 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:02:25.668 INFO:teuthology.orchestra.run.smithi181.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-16T10:02:25.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f869406ada0 msgr2=0x7f869406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f869406ada0 0x7f869406d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86a40090b0 tx=0x7f86a401d040 comp rx=0 tx=0).stop 2023-10-16T10:02:25.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b40b2ba0 msgr2=0x7f86b41f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:02:25.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b40b2ba0 0x7f86b41f26c0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f869c00ebe0 tx=0x7f869c00eef0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 shutdown_connections 2023-10-16T10:02:25.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f869406ada0 0x7f869406d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86b40cb690 0x7f86b40b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b40b2ba0 0x7f86b41f26c0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 --2- 172.21.15.181:0/671443678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b40cb2b0 0x7f86b41506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:02:25.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 >> 172.21.15.181:0/671443678 conn(0x7f86b40cd010 msgr2=0x7f86b40b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:02:25.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 shutdown_connections 2023-10-16T10:02:25.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:02:25.669+0000 7f86b9b91700 1 -- 172.21.15.181:0/671443678 wait complete. 2023-10-16T10:02:25.682 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-16T10:02:25.683 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{}]} 2023-10-16T10:02:25.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:25.683 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:25.701 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:25.701 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:25.701 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:26.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:26.703 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:26.722 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:26.722 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:26.723 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:27.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:27.725 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:27.747 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:27.747 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:27.747 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:28.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:28.749 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:28.771 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:28.771 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:28.772 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:29.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:29.773 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:29.795 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:29.795 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:29.796 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:30.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:30.798 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:30.820 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:30.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:30.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:31.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:31.822 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:31.844 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:31.845 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:31.845 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:32.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:32.847 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:32.869 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:32.869 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:32.870 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:33.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:33.871 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:33.893 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:33.893 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:33.894 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:34.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:34.896 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:34.918 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:34.918 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:34.919 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:35.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:35.920 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:35.942 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:35.943 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:35.943 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:36.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:36.945 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:36.967 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:36.967 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:36.967 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:37.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:37.969 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:37.991 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:37.991 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:37.992 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:38.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:38.994 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:39.016 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:39.016 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:39.017 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:40.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:40.018 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:40.040 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:40.041 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:40.041 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:40.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:40.536 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:02:41.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:41.042 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:41.064 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:41.065 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:41.065 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:42.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:42.067 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:42.089 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:42.089 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:42.090 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:43.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:43.091 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:43.114 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:43.114 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:43.114 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:44.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:44.116 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:44.138 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:44.138 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:44.139 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:45.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:45.140 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:45.162 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:45.162 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:45.163 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:46.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:46.165 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:46.187 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:46.188 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:46.188 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:47.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:47.190 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:47.212 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:47.212 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:47.213 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:48.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:48.215 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:48.236 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:48.237 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:48.237 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:49.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:49.310 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:49.335 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:49.336 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:49.336 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:50.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:50.339 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:50.362 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:50.362 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:50.362 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:51.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:51.364 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:51.386 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:51.387 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:51.387 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:52.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:52.389 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:52.410 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:52.411 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:52.411 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:53.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:53.413 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:53.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:53.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:53.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:54.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:54.442 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:54.463 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:54.463 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:54.464 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:55.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:55.465 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:55.487 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:55.487 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:55.488 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:56.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:56.489 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:56.511 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:56.512 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:56.512 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:57.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:57.514 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:57.536 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:57.536 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:57.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:58.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:58.539 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:58.561 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:58.561 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:58.561 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:02:59.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:02:59.563 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:02:59.585 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T10:02:59.585 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T10:02:59.585 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:00.295 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:00.293+0000 7fa9f1ebe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.295 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:00.293+0000 7fa9f1ebe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.295 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:00.293+0000 7fa9f1ebe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.296 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:00.293+0000 7fa9f1ebe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.296 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:00.293+0000 7fa9f1ebe700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:00.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:00.587 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:00.608 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:00.609 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:00.610 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-10_03_00_UTC'} 2023-10-16T10:03:00.610 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T10:03:00.610 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-10_03_00_UTC'] 2023-10-16T10:03:00.611 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697450545.3021216, now = 1697450580.611045, timo = 35 2023-10-16T10:03:01.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T10:03:01.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.746+0000 7fbccc98f700 1 Processor -- start 2023-10-16T10:03:01.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.746+0000 7fbccc98f700 1 -- start start 2023-10-16T10:03:01.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbccc98f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbcc80cc5e0 0x7fbcc8071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:01.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbccc98f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc815db30 0x7fbcc80cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:01.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbccc98f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc815df60 con 0x7fbcc8159e50 2023-10-16T10:03:01.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbccc98f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc80cb390 con 0x7fbcc815db30 2023-10-16T10:03:01.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbccc98f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc815ea60 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbcc6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc815db30 0x7fbcc80cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:01.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbcc6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc815db30 0x7fbcc80cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46642/0 (socket says 172.21.15.181:46642) 2023-10-16T10:03:01.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.747+0000 7fbcc6d9d700 1 -- 172.21.15.181:0/542386033 learned_addr learned my addr 172.21.15.181:0/542386033 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:01.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc6d9d700 1 -- 172.21.15.181:0/542386033 >> v2:172.21.15.190:3300/0 conn(0x7fbcc80cc5e0 msgr2=0x7fbcc8071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:01.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc6d9d700 1 --2- 172.21.15.181:0/542386033 >> v2:172.21.15.190:3300/0 conn(0x7fbcc80cc5e0 0x7fbcc8071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:01.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc6d9d700 1 -- 172.21.15.181:0/542386033 >> v1:172.21.15.190:6789/0 conn(0x7fbcc8159e50 legacy=0x7fbcc80cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:01.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc6d9d700 1 -- 172.21.15.181:0/542386033 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc815e1c0 con 0x7fbcc815db30 2023-10-16T10:03:01.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc6d9d700 1 --2- 172.21.15.181:0/542386033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc815db30 0x7fbcc80cae50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80cccc0 tx=0x7fbcc000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9cd666e7101125 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:01.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc559a700 1 -- 172.21.15.181:0/542386033 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0010070 con 0x7fbcc815db30 2023-10-16T10:03:01.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc559a700 1 -- 172.21.15.181:0/542386033 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc000dd50 con 0x7fbcc815db30 2023-10-16T10:03:01.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbcc559a700 1 -- 172.21.15.181:0/542386033 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000b310 con 0x7fbcc815db30 2023-10-16T10:03:01.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbccc98f700 1 -- 172.21.15.181:0/542386033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc815db30 msgr2=0x7fbcc80cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:01.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.748+0000 7fbccc98f700 1 --2- 172.21.15.181:0/542386033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc815db30 0x7fbcc80cae50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80cccc0 tx=0x7fbcc000d430 comp rx=0 tx=0).stop 2023-10-16T10:03:01.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.749+0000 7fbccc98f700 1 -- 172.21.15.181:0/542386033 shutdown_connections 2023-10-16T10:03:01.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.749+0000 7fbccc98f700 1 --2- 172.21.15.181:0/542386033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc815db30 0x7fbcc80cae50 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:01.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.749+0000 7fbccc98f700 1 --2- 172.21.15.181:0/542386033 >> v2:172.21.15.190:3300/0 conn(0x7fbcc80cc5e0 0x7fbcc8071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:01.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.749+0000 7fbccc98f700 1 -- 172.21.15.181:0/542386033 >> 172.21.15.181:0/542386033 conn(0x7fbcc806aed0 msgr2=0x7fbcc806cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:01.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.749+0000 7fbccc98f700 1 -- 172.21.15.181:0/542386033 shutdown_connections 2023-10-16T10:03:01.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.749+0000 7fbccc98f700 1 -- 172.21.15.181:0/542386033 wait complete. 2023-10-16T10:03:01.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.750+0000 7fbccc98f700 1 Processor -- start 2023-10-16T10:03:01.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.750+0000 7fbccc98f700 1 -- start start 2023-10-16T10:03:01.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbccc98f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc80cc5e0 0x7fbcc815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:01.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbccc98f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcc8159e50 0x7fbcc8154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:01.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbccc98f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcc815db30 0x7fbcc80b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:01.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbccc98f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc8155140 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbccc98f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80b2b20 con 0x7fbcc815db30 2023-10-16T10:03:01.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbccc98f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc80b2e00 con 0x7fbcc8159e50 2023-10-16T10:03:01.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc80cc5e0 0x7fbcc815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:01.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc80cc5e0 0x7fbcc815c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46644/0 (socket says 172.21.15.181:46644) 2023-10-16T10:03:01.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 -- 172.21.15.181:0/2610808036 learned_addr learned my addr 172.21.15.181:0/2610808036 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:01.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc5d9b700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcc8159e50 0x7fbcc8154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:01.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 -- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcc8159e50 msgr2=0x7fbcc8154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:01.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcc8159e50 0x7fbcc8154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:01.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc6d9d700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcc815db30 0x7fbcc80b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:01.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 -- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcc815db30 msgr2=0x7fbcc80b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:01.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcc815db30 0x7fbcc80b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:01.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 -- 172.21.15.181:0/2610808036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc000bd40 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc6d9d700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcc815db30 0x7fbcc80b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:01.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.751+0000 7fbcc659c700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc80cc5e0 0x7fbcc815c550 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c4a0 tx=0x7fbcbc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.752+0000 7fbcb6ffd700 1 -- 172.21.15.181:0/2610808036 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0054e0 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.752+0000 7fbcb6ffd700 1 -- 172.21.15.181:0/2610808036 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc003680 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.752+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc80b30e0 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.752+0000 7fbcb6ffd700 1 -- 172.21.15.181:0/2610808036 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc003a30 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.752+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc80c6130 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.752+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc80b3600 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.753+0000 7fbcb6ffd700 1 -- 172.21.15.181:0/2610808036 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 70) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fbcbc003b90 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.754+0000 7fbcb6ffd700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fbca806b190 0x7fbca806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.754+0000 7fbcb6ffd700 1 -- 172.21.15.181:0/2610808036 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fbcbc089630 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.754+0000 7fbcc5d9b700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fbca806b190 0x7fbca806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:01.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.754+0000 7fbcc5d9b700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fbca806b190 0x7fbca806d650 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006fd0 tx=0x7fbcb8008040 comp rx=0 tx=0).ready entity=mgr.8208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:01.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.758+0000 7fbcb6ffd700 1 -- 172.21.15.181:0/2610808036 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbcbc0541f0 con 0x7fbcc80cc5e0 2023-10-16T10:03:01.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:01.974+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 --> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcc80b2c90 con 0x7fbca806b190 2023-10-16T10:03:02.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.047+0000 7fbcb6ffd700 1 -- 172.21.15.181:0/2610808036 <== mgr.8208 v2:172.21.15.181:6832/716040864 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbcc80b2c90 con 0x7fbca806b190 2023-10-16T10:03:02.049 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path snap_test_dir1 2023-10-16T10:03:02.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.049+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fbca806b190 msgr2=0x7fbca806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:02.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.049+0000 7fbccc98f700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fbca806b190 0x7fbca806d650 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006fd0 tx=0x7fbcb8008040 comp rx=0 tx=0).stop 2023-10-16T10:03:02.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.049+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc80cc5e0 msgr2=0x7fbcc815c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:02.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.049+0000 7fbccc98f700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc80cc5e0 0x7fbcc815c550 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c4a0 tx=0x7fbcbc00c860 comp rx=0 tx=0).stop 2023-10-16T10:03:02.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 shutdown_connections 2023-10-16T10:03:02.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fbca806b190 0x7fbca806d650 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcc815db30 0x7fbcc80b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc80cc5e0 0x7fbcc815c550 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 --2- 172.21.15.181:0/2610808036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcc8159e50 0x7fbcc8154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 >> 172.21.15.181:0/2610808036 conn(0x7fbcc806aed0 msgr2=0x7fbcc81558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:02.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 shutdown_connections 2023-10-16T10:03:02.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.050+0000 7fbccc98f700 1 -- 172.21.15.181:0/2610808036 wait complete. 2023-10-16T10:03:02.063 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T10:03:02.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:02.064 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T10:03:02.081 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:02.081 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_03_00_UTC 2023-10-16T10:03:02.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:02.082 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_03_00_UTC) 2023-10-16T10:03:02.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:02.144 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T10:03:02.202 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:03:02.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.362+0000 7fc5dbcdd700 1 Processor -- start 2023-10-16T10:03:02.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.363+0000 7fc5dbcdd700 1 -- start start 2023-10-16T10:03:02.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.363+0000 7fc5dbcdd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc5d40cdd80 0x7fc5d40ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.363+0000 7fc5dbcdd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d415dfb0 con 0x7fc5d406b6f0 2023-10-16T10:03:02.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.363+0000 7fc5dbcdd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d40cd3a0 con 0x7fc5d415dbd0 2023-10-16T10:03:02.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.363+0000 7fc5dbcdd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d40ce740 con 0x7fc5d40cdd80 2023-10-16T10:03:02.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.363+0000 7fc5da27a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc5d406b6f0 0x7fc5d406bba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57734/0 (socket says 172.21.15.181:57734) 2023-10-16T10:03:02.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5da27a700 1 -- 172.21.15.181:0/1079398260 learned_addr learned my addr 172.21.15.181:0/1079398260 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:02.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9278700 1 --2- 172.21.15.181:0/1079398260 >> v2:172.21.15.190:3300/0 conn(0x7fc5d40cdd80 0x7fc5d40ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250434882 0 0) 0x7fc5cc001490 con 0x7fc5d406b6f0 2023-10-16T10:03:02.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c00035b0 con 0x7fc5d406b6f0 2023-10-16T10:03:02.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337166236 0 0) 0x7fc5c00035b0 con 0x7fc5d406b6f0 2023-10-16T10:03:02.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 >> v2:172.21.15.190:3300/0 conn(0x7fc5d40cdd80 msgr2=0x7fc5d40ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:02.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 --2- 172.21.15.181:0/1079398260 >> v2:172.21.15.190:3300/0 conn(0x7fc5d40cdd80 0x7fc5d40ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 >> v1:172.21.15.190:6789/0 conn(0x7fc5d415dbd0 legacy=0x7fc5d40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:02.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d40cd830 con 0x7fc5d406b6f0 2023-10-16T10:03:02.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc5cc001b80 con 0x7fc5d406b6f0 2023-10-16T10:03:02.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 >> v1:172.21.15.181:6789/0 conn(0x7fc5d406b6f0 legacy=0x7fc5d406bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:02.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5c0003a00 0x7fc5c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5c0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c0005210 0x7fc5c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40ce740 con 0x7fc5c0004850 2023-10-16T10:03:02.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000c0b0 con 0x7fc5c0005210 2023-10-16T10:03:02.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000c270 con 0x7fc5c0003a00 2023-10-16T10:03:02.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9278700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5c0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5da27a700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c0005210 0x7fc5c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9a79700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5c0003a00 0x7fc5c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9278700 1 -- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5c0003a00 msgr2=0x7fc5c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:02.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9278700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5c0003a00 0x7fc5c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9278700 1 -- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c0005210 msgr2=0x7fc5c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:02.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9278700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c0005210 0x7fc5c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5da27a700 1 -- 172.21.15.181:0/1079398260 reap_dead start 2023-10-16T10:03:02.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.364+0000 7fc5d9278700 1 -- 172.21.15.181:0/1079398260 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c00035b0 con 0x7fc5c0004850 2023-10-16T10:03:02.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5d9278700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5c0004cd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009b30 tx=0x7fc5c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bebea670b2b5f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:02.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000e6c0 con 0x7fc5c0004850 2023-10-16T10:03:02.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d000e820 con 0x7fc5c0004850 2023-10-16T10:03:02.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5d8a77700 1 -- 172.21.15.181:0/1079398260 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000eb90 con 0x7fc5c0004850 2023-10-16T10:03:02.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/1079398260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 msgr2=0x7fc5c0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:02.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5c0004cd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009b30 tx=0x7fc5c0005880 comp rx=0 tx=0).stop 2023-10-16T10:03:02.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/1079398260 shutdown_connections 2023-10-16T10:03:02.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/1079398260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5c0004cd0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/1079398260 >> 172.21.15.181:0/1079398260 conn(0x7fc5d409a930 msgr2=0x7fc5d40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:02.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/1079398260 shutdown_connections 2023-10-16T10:03:02.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.365+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/1079398260 wait complete. 2023-10-16T10:03:02.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 Processor -- start 2023-10-16T10:03:02.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 -- start start 2023-10-16T10:03:02.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5d4154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d40c7940 0x7fc5d41600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d40c7dc0 0x7fc5d4160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40c86d0 con 0x7fc5c0004850 2023-10-16T10:03:02.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40c7520 con 0x7fc5d40c7dc0 2023-10-16T10:03:02.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5dbcdd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d415e310 con 0x7fc5d40c7940 2023-10-16T10:03:02.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5d9a79700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5d4154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5d9a79700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5d4154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46664/0 (socket says 172.21.15.181:46664) 2023-10-16T10:03:02.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.367+0000 7fc5d9a79700 1 -- 172.21.15.181:0/956883791 learned_addr learned my addr 172.21.15.181:0/956883791 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:02.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5d9278700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d40c7940 0x7fc5d41600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5da27a700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d40c7dc0 0x7fc5d4160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5d9a79700 1 -- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d40c7940 msgr2=0x7fc5d41600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:02.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5d9a79700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d40c7940 0x7fc5d41600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5d9a79700 1 -- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d40c7dc0 msgr2=0x7fc5d4160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:02.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5d9a79700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d40c7dc0 0x7fc5d4160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:02.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5d9a79700 1 -- 172.21.15.181:0/956883791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d40c6c80 con 0x7fc5c0004850 2023-10-16T10:03:02.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5d9a79700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5d4154ef0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400e580 tx=0x7fc5c400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:02.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5ca7fc700 1 -- 172.21.15.181:0/956883791 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40052e0 con 0x7fc5c0004850 2023-10-16T10:03:02.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5ca7fc700 1 -- 172.21.15.181:0/956883791 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4005440 con 0x7fc5c0004850 2023-10-16T10:03:02.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d40c6f60 con 0x7fc5c0004850 2023-10-16T10:03:02.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5ca7fc700 1 -- 172.21.15.181:0/956883791 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40038a0 con 0x7fc5c0004850 2023-10-16T10:03:02.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.368+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d4158c20 con 0x7fc5c0004850 2023-10-16T10:03:02.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.369+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d415e5f0 con 0x7fc5c0004850 2023-10-16T10:03:02.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.369+0000 7fc5ca7fc700 1 -- 172.21.15.181:0/956883791 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 70) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fc5c4003a00 con 0x7fc5c0004850 2023-10-16T10:03:02.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.370+0000 7fc5ca7fc700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc5b806b190 0x7fc5b806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:02.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.370+0000 7fc5ca7fc700 1 -- 172.21.15.181:0/956883791 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fc5c40893a0 con 0x7fc5c0004850 2023-10-16T10:03:02.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.370+0000 7fc5d9278700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc5b806b190 0x7fc5b806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:02.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.370+0000 7fc5d9278700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc5b806b190 0x7fc5b806d650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00090b0 tx=0x7fc5d001e040 comp rx=0 tx=0).ready entity=mgr.8208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:02.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.374+0000 7fc5ca7fc700 1 -- 172.21.15.181:0/956883791 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc5c4053f30 con 0x7fc5c0004850 2023-10-16T10:03:02.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:02.621+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc5d4158db0 con 0x7fc5c0004850 2023-10-16T10:03:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.051+0000 7fc5ca7fc700 1 -- 172.21.15.181:0/956883791 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v71) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc5c4003cb0 con 0x7fc5c0004850 2023-10-16T10:03:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.054+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc5b806b190 msgr2=0x7fc5b806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.054+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc5b806b190 0x7fc5b806d650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00090b0 tx=0x7fc5d001e040 comp rx=0 tx=0).stop 2023-10-16T10:03:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.054+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 msgr2=0x7fc5d4154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.054+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5d4154ef0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400e580 tx=0x7fc5c400e940 comp rx=0 tx=0).stop 2023-10-16T10:03:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 shutdown_connections 2023-10-16T10:03:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc5b806b190 0x7fc5b806d650 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d40c7dc0 0x7fc5d4160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5c0004850 0x7fc5d4154ef0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 --2- 172.21.15.181:0/956883791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d40c7940 0x7fc5d41600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 >> 172.21.15.181:0/956883791 conn(0x7fc5d409a930 msgr2=0x7fc5d41615e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 shutdown_connections 2023-10-16T10:03:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.055+0000 7fc5dbcdd700 1 -- 172.21.15.181:0/956883791 wait complete. 2023-10-16T10:03:03.075 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:03:03.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:03.119 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:03.158 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:03:03.159 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:03.160 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:03.177 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:03.177 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:03.201 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:03.200+0000 7fee20cac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:03:03.205 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:03.204+0000 7fb3131b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:03:03.222 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:03.267 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:03.266+0000 7fee20cac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:03.273 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:03.272+0000 7fb3131b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:03.340 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:03.339+0000 7fee20cac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:03:03.349 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:03.347+0000 7fb3131b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:03:03.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.384+0000 7f89b82b0700 1 Processor -- start 2023-10-16T10:03:03.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.385+0000 7f89b82b0700 1 -- start start 2023-10-16T10:03:03.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.385+0000 7f89b82b0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89b015b990 0x7f89b00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.385+0000 7f89b82b0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.385+0000 7f89b82b0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b00cc510 con 0x7f89b00cd2d0 2023-10-16T10:03:03.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.385+0000 7f89b82b0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b00d24f0 con 0x7f89b015b990 2023-10-16T10:03:03.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.385+0000 7f89b82b0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00cc7f0 con 0x7f89b00d0370 2023-10-16T10:03:03.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b584b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f89b00cd2d0 0x7f89b00cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57816/0 (socket says 172.21.15.181:57816) 2023-10-16T10:03:03.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b584b700 1 -- 172.21.15.181:0/2113813988 learned_addr learned my addr 172.21.15.181:0/2113813988 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:03.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b604c700 1 --2- 172.21.15.181:0/2113813988 >> v2:172.21.15.190:3300/0 conn(0x7f89b015b990 0x7f89b00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b684d700 1 --2- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b504a700 1 -- 172.21.15.181:0/2113813988 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506360301 0 0) 0x7f89b00cc510 con 0x7f89b00cd2d0 2023-10-16T10:03:03.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b504a700 1 -- 172.21.15.181:0/2113813988 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f899c0035b0 con 0x7f89b00cd2d0 2023-10-16T10:03:03.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b504a700 1 -- 172.21.15.181:0/2113813988 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2655422190 0 0) 0x7f899c0035b0 con 0x7f89b00cd2d0 2023-10-16T10:03:03.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b604c700 1 -- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 msgr2=0x7f89b00d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:03.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b604c700 1 --2- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b604c700 1 -- 172.21.15.181:0/2113813988 >> v1:172.21.15.181:6789/0 conn(0x7f89b00cd2d0 legacy=0x7f89b00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b604c700 1 -- 172.21.15.181:0/2113813988 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00ccad0 con 0x7f89b015b990 2023-10-16T10:03:03.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b604c700 1 --2- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b015b990 0x7f89b00ce8f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009820 tx=0x7f89ac009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155c72adfb6887dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:03.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b504a700 1 -- 172.21.15.181:0/2113813988 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0001670 con 0x7f89b015b990 2023-10-16T10:03:03.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b504a700 1 -- 172.21.15.181:0/2113813988 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac005c30 con 0x7f89b015b990 2023-10-16T10:03:03.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.386+0000 7f89b504a700 1 -- 172.21.15.181:0/2113813988 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac004030 con 0x7f89b015b990 2023-10-16T10:03:03.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 -- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b015b990 msgr2=0x7f89b00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b015b990 0x7f89b00ce8f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009820 tx=0x7f89ac009b30 comp rx=0 tx=0).stop 2023-10-16T10:03:03.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 -- 172.21.15.181:0/2113813988 shutdown_connections 2023-10-16T10:03:03.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2113813988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b015b990 0x7f89b00ce8f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 -- 172.21.15.181:0/2113813988 >> 172.21.15.181:0/2113813988 conn(0x7f89b009a9a0 msgr2=0x7f89b00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:03.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 -- 172.21.15.181:0/2113813988 shutdown_connections 2023-10-16T10:03:03.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.387+0000 7f89b82b0700 1 -- 172.21.15.181:0/2113813988 wait complete. 2023-10-16T10:03:03.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.388+0000 7f89b82b0700 1 Processor -- start 2023-10-16T10:03:03.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.388+0000 7f89b82b0700 1 -- start start 2023-10-16T10:03:03.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b82b0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b00cd2d0 0x7f89b01439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b82b0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b82b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b015b990 0x7f89b00c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b82b0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00b2a60 con 0x7f89b015b990 2023-10-16T10:03:03.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b82b0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00b1de0 con 0x7f89b00cd2d0 2023-10-16T10:03:03.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b82b0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00b20c0 con 0x7f89b00d0370 2023-10-16T10:03:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b015b990 0x7f89b00c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b015b990 0x7f89b00c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46748/0 (socket says 172.21.15.181:46748) 2023-10-16T10:03:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 -- 172.21.15.181:0/2569017803 learned_addr learned my addr 172.21.15.181:0/2569017803 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b604c700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b00cd2d0 0x7f89b01439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b584b700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 -- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 msgr2=0x7f89b00c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 -- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b00cd2d0 msgr2=0x7f89b01439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b00cd2d0 0x7f89b01439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.389+0000 7f89b684d700 1 -- 172.21.15.181:0/2569017803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac009430 con 0x7f89b015b990 2023-10-16T10:03:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.390+0000 7f89b684d700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b015b990 0x7f89b00c8c90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f89a800ef90 tx=0x7f89a8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.390+0000 7f89a6ffd700 1 -- 172.21.15.181:0/2569017803 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8009b20 con 0x7f89b015b990 2023-10-16T10:03:03.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.390+0000 7f89a6ffd700 1 -- 172.21.15.181:0/2569017803 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a800f040 con 0x7f89b015b990 2023-10-16T10:03:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.390+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b00b23a0 con 0x7f89b015b990 2023-10-16T10:03:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.390+0000 7f89a6ffd700 1 -- 172.21.15.181:0/2569017803 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a800cc70 con 0x7f89b015b990 2023-10-16T10:03:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.390+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b0143ee0 con 0x7f89b015b990 2023-10-16T10:03:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.391+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b00b2ba0 con 0x7f89b015b990 2023-10-16T10:03:03.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.392+0000 7f89a6ffd700 1 -- 172.21.15.181:0/2569017803 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f89a8014480 con 0x7f89b015b990 2023-10-16T10:03:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.392+0000 7f89a6ffd700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 0x7f899406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.392+0000 7f89b604c700 1 -- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 msgr2=0x7f899406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.392+0000 7f89a6ffd700 1 -- 172.21.15.181:0/2569017803 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f89a808a810 con 0x7f89b015b990 2023-10-16T10:03:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.392+0000 7f89b604c700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 0x7f899406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:03.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.397+0000 7f89a6ffd700 1 -- 172.21.15.181:0/2569017803 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f89b00b2ba0 con 0x7f89b015b990 2023-10-16T10:03:03.418 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:03.417+0000 7fee20cac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:03:03.432 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:03.431+0000 7fb3131b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:03:03.558 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:03.557+0000 7fee20cac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:03:03.576 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:03.575+0000 7fb3131b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:03:03.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.593+0000 7f89b604c700 1 -- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 msgr2=0x7f899406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:03.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.593+0000 7f89b604c700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 0x7f899406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:03.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.597+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89b00b2ba0 con 0x7f89b015b990 2023-10-16T10:03:03.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.598+0000 7f89a6ffd700 1 -- 172.21.15.181:0/2569017803 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+17708 (secure 0 0 0) 0x7f89b00b2ba0 con 0x7f89b015b990 2023-10-16T10:03:03.600 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:03.601 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":119,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:02:24.201123+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:08.438030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:08.968951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:6832/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1068945898":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1396028129":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/2917390541":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1347286988":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1199722576":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/3331655182":"2023-10-17T10:02:24.201085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:03.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 msgr2=0x7f899406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:03.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 0x7f899406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b015b990 msgr2=0x7f89b00c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b015b990 0x7f89b00c8c90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f89a800ef90 tx=0x7f89a8009f60 comp rx=0 tx=0).stop 2023-10-16T10:03:03.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 shutdown_connections 2023-10-16T10:03:03.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f899406b140 0x7f899406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b015b990 0x7f89b00c8c90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b00cd2d0 0x7f89b01439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 --2- 172.21.15.181:0/2569017803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b00d0370 0x7f89b00c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 >> 172.21.15.181:0/2569017803 conn(0x7f89b009a9a0 msgr2=0x7f89b015abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:03.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 shutdown_connections 2023-10-16T10:03:03.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.601+0000 7f89b82b0700 1 -- 172.21.15.181:0/2569017803 wait complete. 2023-10-16T10:03:03.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:02:08.438030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:02:08.968951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:03.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:03.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.779+0000 7f8db1dc1700 1 Processor -- start 2023-10-16T10:03:03.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.779+0000 7f8db1dc1700 1 -- start start 2023-10-16T10:03:03.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.779+0000 7f8db1dc1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8dac0ce6c0 0x7f8dac0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.779+0000 7f8db1dc1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0cc6a0 0x7f8dac0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.779+0000 7f8db1dc1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dac0b1f70 con 0x7f8dac15f150 2023-10-16T10:03:03.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.779+0000 7f8db1dc1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac0b2ae0 con 0x7f8dac0ce6c0 2023-10-16T10:03:03.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.779+0000 7f8db1dc1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0b2dc0 con 0x7f8dac0cc6a0 2023-10-16T10:03:03.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8dab7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8dac15f150 0x7f8dac0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57848/0 (socket says 172.21.15.181:57848) 2023-10-16T10:03:03.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8dab7fe700 1 -- 172.21.15.181:0/1314960064 learned_addr learned my addr 172.21.15.181:0/1314960064 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:03.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daaffd700 1 --2- 172.21.15.181:0/1314960064 >> v2:172.21.15.190:3300/0 conn(0x7f8dac0ce6c0 0x7f8dac0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8dabfff700 1 --2- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0cc6a0 0x7f8dac0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daa7fc700 1 -- 172.21.15.181:0/1314960064 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223088228 0 0) 0x7f8dac0b1f70 con 0x7f8dac15f150 2023-10-16T10:03:03.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daa7fc700 1 -- 172.21.15.181:0/1314960064 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d940035b0 con 0x7f8dac15f150 2023-10-16T10:03:03.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daa7fc700 1 -- 172.21.15.181:0/1314960064 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2449601468 0 0) 0x7f8d940035b0 con 0x7f8dac15f150 2023-10-16T10:03:03.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daaffd700 1 -- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0cc6a0 msgr2=0x7f8dac0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:03.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daaffd700 1 --2- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0cc6a0 0x7f8dac0ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daaffd700 1 -- 172.21.15.181:0/1314960064 >> v1:172.21.15.181:6789/0 conn(0x7f8dac15f150 legacy=0x7f8dac0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daaffd700 1 -- 172.21.15.181:0/1314960064 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0cf9d0 con 0x7f8dac0ce6c0 2023-10-16T10:03:03.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8dabfff700 1 --2- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0cc6a0 0x7f8dac0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:03.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.780+0000 7f8daaffd700 1 --2- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0ce6c0 0x7f8dac0cc160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009840 tx=0x7f8da0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927142ce49ef5cf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:03.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8daa7fc700 1 -- 172.21.15.181:0/1314960064 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c001670 con 0x7f8dac0ce6c0 2023-10-16T10:03:03.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8daa7fc700 1 -- 172.21.15.181:0/1314960064 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da0004030 con 0x7f8dac0ce6c0 2023-10-16T10:03:03.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8daa7fc700 1 -- 172.21.15.181:0/1314960064 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da00043c0 con 0x7f8dac0ce6c0 2023-10-16T10:03:03.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 -- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0ce6c0 msgr2=0x7f8dac0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0ce6c0 0x7f8dac0cc160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009840 tx=0x7f8da0009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:03.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 -- 172.21.15.181:0/1314960064 shutdown_connections 2023-10-16T10:03:03.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0cc6a0 0x7f8dac0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/1314960064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0ce6c0 0x7f8dac0cc160 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 -- 172.21.15.181:0/1314960064 >> 172.21.15.181:0/1314960064 conn(0x7f8dac09a930 msgr2=0x7f8dac0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:03.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 -- 172.21.15.181:0/1314960064 shutdown_connections 2023-10-16T10:03:03.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.781+0000 7f8db1dc1700 1 -- 172.21.15.181:0/1314960064 wait complete. 2023-10-16T10:03:03.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.783+0000 7f8db1dc1700 1 Processor -- start 2023-10-16T10:03:03.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.783+0000 7f8db1dc1700 1 -- start start 2023-10-16T10:03:03.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8db1dc1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0cc6a0 0x7f8dac151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8db1dc1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0ce6c0 0x7f8dac0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8db1dc1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac15f150 0x7f8dac0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8db1dc1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac152de0 con 0x7f8dac15f150 2023-10-16T10:03:03.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8db1dc1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac152ad0 con 0x7f8dac0cc6a0 2023-10-16T10:03:03.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8db1dc1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0c6dc0 con 0x7f8dac0ce6c0 2023-10-16T10:03:03.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac15f150 0x7f8dac0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac15f150 0x7f8dac0c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46780/0 (socket says 172.21.15.181:46780) 2023-10-16T10:03:03.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 -- 172.21.15.181:0/689580345 learned_addr learned my addr 172.21.15.181:0/689580345 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:03.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dab7fe700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0cc6a0 0x7f8dac151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8daaffd700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0ce6c0 0x7f8dac0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:03.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 -- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0ce6c0 msgr2=0x7f8dac0c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0ce6c0 0x7f8dac0c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 -- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0cc6a0 msgr2=0x7f8dac151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:03.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0cc6a0 0x7f8dac151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:03.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 -- 172.21.15.181:0/689580345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4009a70 con 0x7f8dac15f150 2023-10-16T10:03:03.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dab7fe700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0cc6a0 0x7f8dac151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:03.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8dabfff700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac15f150 0x7f8dac0c9e20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8da400efc0 tx=0x7f8da400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:03.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.785+0000 7f8d93fff700 1 -- 172.21.15.181:0/689580345 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400ca90 con 0x7f8dac15f150 2023-10-16T10:03:03.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.786+0000 7f8d93fff700 1 -- 172.21.15.181:0/689580345 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400f040 con 0x7f8dac15f150 2023-10-16T10:03:03.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.786+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da00094a0 con 0x7f8dac15f150 2023-10-16T10:03:03.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.786+0000 7f8d93fff700 1 -- 172.21.15.181:0/689580345 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400ce10 con 0x7f8dac15f150 2023-10-16T10:03:03.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.786+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac0c6ae0 con 0x7f8dac15f150 2023-10-16T10:03:03.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.786+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac067d20 con 0x7f8dac15f150 2023-10-16T10:03:03.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.787+0000 7f8d93fff700 1 -- 172.21.15.181:0/689580345 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8da4014480 con 0x7f8dac15f150 2023-10-16T10:03:03.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.787+0000 7f8d93fff700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 0x7f8d9806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:03.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.787+0000 7f8dab7fe700 1 -- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 msgr2=0x7f8d9806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:03.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.787+0000 7f8d93fff700 1 -- 172.21.15.181:0/689580345 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f8da408a3e0 con 0x7f8dac15f150 2023-10-16T10:03:03.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.788+0000 7f8dab7fe700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 0x7f8d9806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:03.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.796+0000 7f8d93fff700 1 -- 172.21.15.181:0/689580345 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8da4054db0 con 0x7f8dac15f150 2023-10-16T10:03:03.882 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:03.881+0000 7fee20cac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:03:03.921 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:03.919+0000 7fb3131b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:03:03.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.989+0000 7f8dab7fe700 1 -- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 msgr2=0x7f8d9806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:03.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.989+0000 7f8dab7fe700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 0x7f8d9806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:03.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.995+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dac0d2430 con 0x7f8dac15f150 2023-10-16T10:03:03.997 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:03.996+0000 7fee20cac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:03:03.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.996+0000 7f8d93fff700 1 -- 172.21.15.181:0/689580345 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8da4009d70 con 0x7f8dac15f150 2023-10-16T10:03:03.998 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:04.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 msgr2=0x7f8d9806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:04.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 0x7f8d9806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac15f150 msgr2=0x7f8dac0c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac15f150 0x7f8dac0c9e20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8da400efc0 tx=0x7f8da400c3a0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 shutdown_connections 2023-10-16T10:03:04.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f8d9806b140 0x7f8d9806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dac15f150 0x7f8dac0c9e20 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac0cc6a0 0x7f8dac151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 --2- 172.21.15.181:0/689580345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0ce6c0 0x7f8dac0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 >> 172.21.15.181:0/689580345 conn(0x7f8dac09a930 msgr2=0x7f8dac0d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:04.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 shutdown_connections 2023-10-16T10:03:04.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:03.998+0000 7f8db1dc1700 1 -- 172.21.15.181:0/689580345 wait complete. 2023-10-16T10:03:04.015 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:03:04.042 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:04.041+0000 7fb3131b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:03:04.071 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:04.070+0000 7fee20cac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:03:04.119 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:04.117+0000 7fb3131b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:03:04.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.140+0000 7f73b6877700 1 Processor -- start 2023-10-16T10:03:04.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.140+0000 7f73b6877700 1 -- start start 2023-10-16T10:03:04.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73b6877700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73b00d01d0 0x7f73b00ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73b6877700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73b015ea40 0x7f73b00d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73b6877700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b00cf580 con 0x7f73b0161530 2023-10-16T10:03:04.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73b6877700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b00d4290 con 0x7f73b00d01d0 2023-10-16T10:03:04.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73b6877700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b00d0af0 con 0x7f73b015ea40 2023-10-16T10:03:04.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73affff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73b00d01d0 0x7f73b00ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73b4e14700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f73b0161530 0x7f73b01619c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57866/0 (socket says 172.21.15.181:57866) 2023-10-16T10:03:04.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73b4e14700 1 -- 172.21.15.181:0/1317516386 learned_addr learned my addr 172.21.15.181:0/1317516386 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:04.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73af7fe700 1 --2- 172.21.15.181:0/1317516386 >> v2:172.21.15.190:3300/0 conn(0x7f73b015ea40 0x7f73b00d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73affff700 1 -- 172.21.15.181:0/1317516386 >> v2:172.21.15.190:3300/0 conn(0x7f73b015ea40 msgr2=0x7f73b00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:04.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73affff700 1 --2- 172.21.15.181:0/1317516386 >> v2:172.21.15.190:3300/0 conn(0x7f73b015ea40 0x7f73b00d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73affff700 1 -- 172.21.15.181:0/1317516386 >> v1:172.21.15.181:6789/0 conn(0x7f73b0161530 legacy=0x7f73b01619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73affff700 1 -- 172.21.15.181:0/1317516386 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b00ce970 con 0x7f73b00d01d0 2023-10-16T10:03:04.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.141+0000 7f73aeffd700 1 -- 172.21.15.181:0/1317516386 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584121596 0 0) 0x7f73b00cf580 con 0x7f73b0161530 2023-10-16T10:03:04.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73affff700 1 --2- 172.21.15.181:0/1317516386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b00d01d0 0x7f73b00ce2d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f73b00d12c0 tx=0x7f73a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafed76795e79b21 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:04.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73aeffd700 1 -- 172.21.15.181:0/1317516386 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0011830 con 0x7f73b00d01d0 2023-10-16T10:03:04.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73aeffd700 1 -- 172.21.15.181:0/1317516386 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0011990 con 0x7f73b00d01d0 2023-10-16T10:03:04.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73aeffd700 1 -- 172.21.15.181:0/1317516386 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0011d20 con 0x7f73b00d01d0 2023-10-16T10:03:04.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73b6877700 1 -- 172.21.15.181:0/1317516386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b00d01d0 msgr2=0x7f73b00ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73b6877700 1 --2- 172.21.15.181:0/1317516386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b00d01d0 0x7f73b00ce2d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f73b00d12c0 tx=0x7f73a0009b20 comp rx=0 tx=0).stop 2023-10-16T10:03:04.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73b6877700 1 -- 172.21.15.181:0/1317516386 shutdown_connections 2023-10-16T10:03:04.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73b6877700 1 --2- 172.21.15.181:0/1317516386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b00d01d0 0x7f73b00ce2d0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73b6877700 1 --2- 172.21.15.181:0/1317516386 >> v2:172.21.15.190:3300/0 conn(0x7f73b015ea40 0x7f73b00d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73b6877700 1 -- 172.21.15.181:0/1317516386 >> 172.21.15.181:0/1317516386 conn(0x7f73b009cb60 msgr2=0x7f73b00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:04.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.142+0000 7f73b6877700 1 -- 172.21.15.181:0/1317516386 shutdown_connections 2023-10-16T10:03:04.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.143+0000 7f73b6877700 1 -- 172.21.15.181:0/1317516386 wait complete. 2023-10-16T10:03:04.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.144+0000 7f73b6877700 1 Processor -- start 2023-10-16T10:03:04.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.144+0000 7f73b6877700 1 -- start start 2023-10-16T10:03:04.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b6877700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73b00d01d0 0x7f73b01f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b6877700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b015ea40 0x7f73b00b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b6877700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73b0161530 0x7f73b00b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b6877700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b01f42b0 con 0x7f73b015ea40 2023-10-16T10:03:04.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b6877700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00b9300 con 0x7f73b00d01d0 2023-10-16T10:03:04.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b6877700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00b8930 con 0x7f73b0161530 2023-10-16T10:03:04.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b015ea40 0x7f73b00b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b015ea40 0x7f73b00b8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46794/0 (socket says 172.21.15.181:46794) 2023-10-16T10:03:04.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 -- 172.21.15.181:0/14357938 learned_addr learned my addr 172.21.15.181:0/14357938 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:04.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73affff700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73b00d01d0 0x7f73b01f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b4e14700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73b0161530 0x7f73b00b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 -- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73b0161530 msgr2=0x7f73b00b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73b0161530 0x7f73b00b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 -- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73b00d01d0 msgr2=0x7f73b01f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73b00d01d0 0x7f73b01f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 -- 172.21.15.181:0/14357938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a00094a0 con 0x7f73b015ea40 2023-10-16T10:03:04.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73affff700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73b00d01d0 0x7f73b01f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:04.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73af7fe700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b015ea40 0x7f73b00b8dc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f73a400ea10 tx=0x7f73a400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:04.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73acff9700 1 -- 172.21.15.181:0/14357938 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4009710 con 0x7f73b015ea40 2023-10-16T10:03:04.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73acff9700 1 -- 172.21.15.181:0/14357938 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400eeb0 con 0x7f73b015ea40 2023-10-16T10:03:04.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b00b7d60 con 0x7f73b015ea40 2023-10-16T10:03:04.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.145+0000 7f73acff9700 1 -- 172.21.15.181:0/14357938 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400ca50 con 0x7f73b015ea40 2023-10-16T10:03:04.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.146+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b00b82e0 con 0x7f73b015ea40 2023-10-16T10:03:04.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.147+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b004ebb0 con 0x7f73b015ea40 2023-10-16T10:03:04.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.148+0000 7f73acff9700 1 -- 172.21.15.181:0/14357938 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f73a400cbb0 con 0x7f73b015ea40 2023-10-16T10:03:04.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.148+0000 7f73acff9700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 0x7f739c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.148+0000 7f73affff700 1 -- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 msgr2=0x7f739c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:04.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.148+0000 7f73acff9700 1 -- 172.21.15.181:0/14357938 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f73a408a060 con 0x7f73b015ea40 2023-10-16T10:03:04.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.148+0000 7f73affff700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 0x7f739c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:04.160 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:04.150+0000 7fee20cac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:03:04.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.152+0000 7f73acff9700 1 -- 172.21.15.181:0/14357938 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f73a4054ab0 con 0x7f73b015ea40 2023-10-16T10:03:04.204 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:04.202+0000 7fb3131b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:03:04.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.349+0000 7f73affff700 1 -- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 msgr2=0x7f739c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:04.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.349+0000 7f73affff700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 0x7f739c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:04.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.352+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73b00b8a70 con 0x7f73b015ea40 2023-10-16T10:03:04.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.353+0000 7f73acff9700 1 -- 172.21.15.181:0/14357938 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73a40580d0 con 0x7f73b015ea40 2023-10-16T10:03:04.355 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:03:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 msgr2=0x7f739c06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 0x7f739c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b015ea40 msgr2=0x7f73b00b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b015ea40 0x7f73b00b8dc0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f73a400ea10 tx=0x7f73a400ed20 comp rx=0 tx=0).stop 2023-10-16T10:03:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 shutdown_connections 2023-10-16T10:03:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f739c06b140 0x7f739c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73b0161530 0x7f73b00b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73b00d01d0 0x7f73b01f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 --2- 172.21.15.181:0/14357938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b015ea40 0x7f73b00b8dc0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 >> 172.21.15.181:0/14357938 conn(0x7f73b009cb60 msgr2=0x7f73b01605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 shutdown_connections 2023-10-16T10:03:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.355+0000 7f73b6877700 1 -- 172.21.15.181:0/14357938 wait complete. 2023-10-16T10:03:04.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:03:04.519 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:04.518+0000 7fee20cac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:03:04.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.533+0000 7f429a6f7700 1 Processor -- start 2023-10-16T10:03:04.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.533+0000 7f429a6f7700 1 -- start start 2023-10-16T10:03:04.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.534+0000 7f429a6f7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4294159e10 0x7f42940ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.534+0000 7f429a6f7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429406e4b0 con 0x7f429406bef0 2023-10-16T10:03:04.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.534+0000 7f429a6f7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429406ed50 con 0x7f429415a1f0 2023-10-16T10:03:04.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.534+0000 7f429a6f7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42940ceea0 con 0x7f4294159e10 2023-10-16T10:03:04.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.534+0000 7f4293fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4294159e10 0x7f42940ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.534+0000 7f42937fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f429406bef0 0x7f42940caba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57924/0 (socket says 172.21.15.181:57924) 2023-10-16T10:03:04.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.534+0000 7f42937fe700 1 -- 172.21.15.181:0/2271257925 learned_addr learned my addr 172.21.15.181:0/2271257925 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:04.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.535+0000 7f4293fff700 1 -- 172.21.15.181:0/2271257925 >> v1:172.21.15.190:6789/0 conn(0x7f429415a1f0 legacy=0x7f429415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.535+0000 7f4293fff700 1 -- 172.21.15.181:0/2271257925 >> v1:172.21.15.181:6789/0 conn(0x7f429406bef0 legacy=0x7f42940caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.535+0000 7f4293fff700 1 -- 172.21.15.181:0/2271257925 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429406e700 con 0x7f4294159e10 2023-10-16T10:03:04.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.535+0000 7f4293fff700 1 --2- 172.21.15.181:0/2271257925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940ce5a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4284009840 tx=0x7f4284009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1015557390e52009 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:04.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.535+0000 7f4292ffd700 1 -- 172.21.15.181:0/2271257925 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284011830 con 0x7f4294159e10 2023-10-16T10:03:04.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.535+0000 7f4292ffd700 1 -- 172.21.15.181:0/2271257925 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4284011990 con 0x7f4294159e10 2023-10-16T10:03:04.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.535+0000 7f4292ffd700 1 -- 172.21.15.181:0/2271257925 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284011d20 con 0x7f4294159e10 2023-10-16T10:03:04.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.536+0000 7f429a6f7700 1 -- 172.21.15.181:0/2271257925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 msgr2=0x7f42940ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.536+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2271257925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940ce5a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4284009840 tx=0x7f4284009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:04.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.536+0000 7f429a6f7700 1 -- 172.21.15.181:0/2271257925 shutdown_connections 2023-10-16T10:03:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.536+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2271257925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940ce5a0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.536+0000 7f429a6f7700 1 -- 172.21.15.181:0/2271257925 >> 172.21.15.181:0/2271257925 conn(0x7f42940cc730 msgr2=0x7f42940ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.536+0000 7f429a6f7700 1 -- 172.21.15.181:0/2271257925 shutdown_connections 2023-10-16T10:03:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.536+0000 7f429a6f7700 1 -- 172.21.15.181:0/2271257925 wait complete. 2023-10-16T10:03:04.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.537+0000 7f429a6f7700 1 Processor -- start 2023-10-16T10:03:04.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.537+0000 7f429a6f7700 1 -- start start 2023-10-16T10:03:04.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f429a6f7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429406bef0 0x7f42940c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f429a6f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f429a6f7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429415a1f0 0x7f429415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f429a6f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429415d380 con 0x7f4294159e10 2023-10-16T10:03:04.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f429a6f7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429414d390 con 0x7f429415a1f0 2023-10-16T10:03:04.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f429a6f7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42940bdf90 con 0x7f429406bef0 2023-10-16T10:03:04.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46852/0 (socket says 172.21.15.181:46852) 2023-10-16T10:03:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 -- 172.21.15.181:0/2493738055 learned_addr learned my addr 172.21.15.181:0/2493738055 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f4293fff700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429406bef0 0x7f42940c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f4298c94700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429415a1f0 0x7f429415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 -- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429406bef0 msgr2=0x7f42940c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429406bef0 0x7f42940c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 -- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429415a1f0 msgr2=0x7f429415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429415a1f0 0x7f429415b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 -- 172.21.15.181:0/2493738055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42840094a0 con 0x7f4294159e10 2023-10-16T10:03:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.538+0000 7f42937fe700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940c65f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f428800c500 tx=0x7f428800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.539+0000 7f4290ff9700 1 -- 172.21.15.181:0/2493738055 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288005d70 con 0x7f4294159e10 2023-10-16T10:03:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.539+0000 7f4290ff9700 1 -- 172.21.15.181:0/2493738055 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42880070d0 con 0x7f4294159e10 2023-10-16T10:03:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.539+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429415d0a0 con 0x7f4294159e10 2023-10-16T10:03:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.539+0000 7f4290ff9700 1 -- 172.21.15.181:0/2493738055 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288007440 con 0x7f4294159e10 2023-10-16T10:03:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.539+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429414d5a0 con 0x7f4294159e10 2023-10-16T10:03:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.540+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42940be800 con 0x7f4294159e10 2023-10-16T10:03:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.540+0000 7f4290ff9700 1 -- 172.21.15.181:0/2493738055 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f42880075a0 con 0x7f4294159e10 2023-10-16T10:03:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.541+0000 7f4290ff9700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 0x7f428006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.541+0000 7f4293fff700 1 -- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 msgr2=0x7f428006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.541+0000 7f4290ff9700 1 -- 172.21.15.181:0/2493738055 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f4288088ee0 con 0x7f4294159e10 2023-10-16T10:03:04.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.541+0000 7f4293fff700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 0x7f428006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:04.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.545+0000 7f4290ff9700 1 -- 172.21.15.181:0/2493738055 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f42940be800 con 0x7f4294159e10 2023-10-16T10:03:04.592 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:04.590+0000 7fb3131b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:03:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.742+0000 7f4293fff700 1 -- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 msgr2=0x7f428006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.742+0000 7f4293fff700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 0x7f428006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.745+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42940be800 con 0x7f4294159e10 2023-10-16T10:03:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.745+0000 7f4290ff9700 1 -- 172.21.15.181:0/2493738055 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42940be800 con 0x7f4294159e10 2023-10-16T10:03:04.747 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:03:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 msgr2=0x7f428006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 0x7f428006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 msgr2=0x7f42940c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940c65f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f428800c500 tx=0x7f428800c810 comp rx=0 tx=0).stop 2023-10-16T10:03:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 shutdown_connections 2023-10-16T10:03:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f428006b140 0x7f428006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429406bef0 0x7f42940c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429415a1f0 0x7f429415b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 --2- 172.21.15.181:0/2493738055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4294159e10 0x7f42940c65f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 >> 172.21.15.181:0/2493738055 conn(0x7f42940cc730 msgr2=0x7f4294149b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 shutdown_connections 2023-10-16T10:03:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.747+0000 7f429a6f7700 1 -- 172.21.15.181:0/2493738055 wait complete. 2023-10-16T10:03:04.762 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:03:04.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.886+0000 7fc2c2567700 1 Processor -- start 2023-10-16T10:03:04.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.886+0000 7fc2c2567700 1 -- start start 2023-10-16T10:03:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.886+0000 7fc2c2567700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc15dd50 0x7fc2bc0d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c2567700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc2bc0d1e80 0x7fc2bc0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c2567700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2bc0ce440 0x7fc2bc0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c2567700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc0b3c00 con 0x7fc2bc0ce440 2023-10-16T10:03:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c2567700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc15e130 con 0x7fc2bc0d1e80 2023-10-16T10:03:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c2567700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0d2840 con 0x7fc2bc15dd50 2023-10-16T10:03:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2bc0ce440 0x7fc2bc0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2bc0ce440 0x7fc2bc0ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46864/0 (socket says 172.21.15.181:46864) 2023-10-16T10:03:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 -- 172.21.15.181:0/3702896357 learned_addr learned my addr 172.21.15.181:0/3702896357 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2bb7fe700 1 --2- 172.21.15.181:0/3702896357 >> v2:172.21.15.190:3300/0 conn(0x7fc2bc0d1e80 0x7fc2bc0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2bbfff700 1 --2- 172.21.15.181:0/3702896357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc15dd50 0x7fc2bc0d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 -- 172.21.15.181:0/3702896357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc15dd50 msgr2=0x7fc2bc0d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 --2- 172.21.15.181:0/3702896357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc15dd50 0x7fc2bc0d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 -- 172.21.15.181:0/3702896357 >> v2:172.21.15.190:3300/0 conn(0x7fc2bc0d1e80 msgr2=0x7fc2bc0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 --2- 172.21.15.181:0/3702896357 >> v2:172.21.15.190:3300/0 conn(0x7fc2bc0d1e80 0x7fc2bc0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 -- 172.21.15.181:0/3702896357 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc0b0410 con 0x7fc2bc0ce440 2023-10-16T10:03:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2bbfff700 1 --2- 172.21.15.181:0/3702896357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc15dd50 0x7fc2bc0d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2c0b04700 1 --2- 172.21.15.181:0/3702896357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc0ce440 0x7fc2bc0ce8a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc06b790 tx=0x7fc2b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0f6fb660a31cf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2baffd700 1 -- 172.21.15.181:0/3702896357 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400e070 con 0x7fc2bc0ce440 2023-10-16T10:03:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2baffd700 1 -- 172.21.15.181:0/3702896357 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b400b9c0 con 0x7fc2bc0ce440 2023-10-16T10:03:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.887+0000 7fc2baffd700 1 -- 172.21.15.181:0/3702896357 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400bd70 con 0x7fc2bc0ce440 2023-10-16T10:03:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 -- 172.21.15.181:0/3702896357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc0ce440 msgr2=0x7fc2bc0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 --2- 172.21.15.181:0/3702896357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc0ce440 0x7fc2bc0ce8a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc06b790 tx=0x7fc2b400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 -- 172.21.15.181:0/3702896357 shutdown_connections 2023-10-16T10:03:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 --2- 172.21.15.181:0/3702896357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc0ce440 0x7fc2bc0ce8a0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 --2- 172.21.15.181:0/3702896357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc15dd50 0x7fc2bc0d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 --2- 172.21.15.181:0/3702896357 >> v2:172.21.15.190:3300/0 conn(0x7fc2bc0d1e80 0x7fc2bc0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 -- 172.21.15.181:0/3702896357 >> 172.21.15.181:0/3702896357 conn(0x7fc2bc08af90 msgr2=0x7fc2bc0d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 -- 172.21.15.181:0/3702896357 shutdown_connections 2023-10-16T10:03:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.888+0000 7fc2c2567700 1 -- 172.21.15.181:0/3702896357 wait complete. 2023-10-16T10:03:04.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.889+0000 7fc2c2567700 1 Processor -- start 2023-10-16T10:03:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c2567700 1 -- start start 2023-10-16T10:03:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c2567700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2bc0ce440 0x7fc2bc15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c2567700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc0d1e80 0x7fc2bc1f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c2567700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc15dd50 0x7fc2bc0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c2567700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0cac60 con 0x7fc2bc15dd50 2023-10-16T10:03:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c2567700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0caf40 con 0x7fc2bc0ce440 2023-10-16T10:03:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c2567700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0cb220 con 0x7fc2bc0d1e80 2023-10-16T10:03:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c0b04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc15dd50 0x7fc2bc0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c0b04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc15dd50 0x7fc2bc0cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46868/0 (socket says 172.21.15.181:46868) 2023-10-16T10:03:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2c0b04700 1 -- 172.21.15.181:0/1232958927 learned_addr learned my addr 172.21.15.181:0/1232958927 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2bb7fe700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc0d1e80 0x7fc2bc1f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.890+0000 7fc2bbfff700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2bc0ce440 0x7fc2bc15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c0b04700 1 -- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc0d1e80 msgr2=0x7fc2bc1f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c0b04700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc0d1e80 0x7fc2bc1f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c0b04700 1 -- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2bc0ce440 msgr2=0x7fc2bc15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c0b04700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2bc0ce440 0x7fc2bc15b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c0b04700 1 -- 172.21.15.181:0/1232958927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b4009b30 con 0x7fc2bc15dd50 2023-10-16T10:03:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c0b04700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc15dd50 0x7fc2bc0cbd00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc08d0d0 tx=0x7fc2b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2b8ff9700 1 -- 172.21.15.181:0/1232958927 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4004880 con 0x7fc2bc15dd50 2023-10-16T10:03:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2bc0cb500 con 0x7fc2bc15dd50 2023-10-16T10:03:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.891+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc0cd3b0 con 0x7fc2bc15dd50 2023-10-16T10:03:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.892+0000 7fc2b8ff9700 1 -- 172.21.15.181:0/1232958927 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b40092e0 con 0x7fc2bc15dd50 2023-10-16T10:03:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.892+0000 7fc2b8ff9700 1 -- 172.21.15.181:0/1232958927 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4025650 con 0x7fc2bc15dd50 2023-10-16T10:03:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.892+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c0052f0 con 0x7fc2bc15dd50 2023-10-16T10:03:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.893+0000 7fc2b8ff9700 1 -- 172.21.15.181:0/1232958927 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc2b40257b0 con 0x7fc2bc15dd50 2023-10-16T10:03:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.893+0000 7fc2b8ff9700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 0x7fc2a806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.893+0000 7fc2bbfff700 1 -- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 msgr2=0x7fc2a806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.894+0000 7fc2b8ff9700 1 -- 172.21.15.181:0/1232958927 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7fc2b408ba30 con 0x7fc2bc15dd50 2023-10-16T10:03:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.894+0000 7fc2bbfff700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 0x7fc2a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:04.898+0000 7fc2b8ff9700 1 -- 172.21.15.181:0/1232958927 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc2b4056520 con 0x7fc2bc15dd50 2023-10-16T10:03:05.087 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:05.086+0000 7fee20cac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:03:05.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.095+0000 7fc2bbfff700 1 -- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 msgr2=0x7fc2a806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:05.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.095+0000 7fc2bbfff700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 0x7fc2a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.149+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc29c0055a0 con 0x7fc2bc15dd50 2023-10-16T10:03:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.149+0000 7fc2b8ff9700 1 -- 172.21.15.181:0/1232958927 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v119) v1 ==== 80+0+478 (secure 0 0 0) 0x7fc2b402a1f0 con 0x7fc2bc15dd50 2023-10-16T10:03:05.151 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/2093458419 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.151 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/2093458419 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.152 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1068945898 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.152 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1396028129 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.152 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2917390541 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.152 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1347286988 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.152 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1199722576 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.152 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3331655182 2023-10-17T10:02:24.201085+0000 2023-10-16T10:03:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 msgr2=0x7fc2a806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 0x7fc2a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc15dd50 msgr2=0x7fc2bc0cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc15dd50 0x7fc2bc0cbd00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc08d0d0 tx=0x7fc2b4004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 shutdown_connections 2023-10-16T10:03:05.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc2a806b140 0x7fc2a806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2bc15dd50 0x7fc2bc0cbd00 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2bc0ce440 0x7fc2bc15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 --2- 172.21.15.181:0/1232958927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2bc0d1e80 0x7fc2bc1f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 >> 172.21.15.181:0/1232958927 conn(0x7fc2bc08af90 msgr2=0x7fc2bc0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:05.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 shutdown_connections 2023-10-16T10:03:05.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.153+0000 7fc2c2567700 1 -- 172.21.15.181:0/1232958927 wait complete. 2023-10-16T10:03:05.158 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:05.155+0000 7fee20cac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:03:05.158 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T10:03:05.168 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:03:05.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:05.169 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:03:05.188 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:05.186+0000 7fb3131b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:03:05.211 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T10:03:05.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:05.212 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:03:05.262 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:05.261+0000 7fb3131b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:03:05.267 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:05.268 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:05.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:05.286 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:05.287 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:05.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:03:05.324 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:05.325 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:05.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:05.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:05.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.486+0000 7f0eda049700 1 Processor -- start 2023-10-16T10:03:05.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.486+0000 7f0eda049700 1 -- start start 2023-10-16T10:03:05.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.486+0000 7f0eda049700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ed40af450 0x7f0ed40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.486+0000 7f0eda049700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40cd470 0x7f0ed40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.486+0000 7f0eda049700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed406f870 con 0x7f0ed415dfd0 2023-10-16T10:03:05.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.486+0000 7f0eda049700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed40d1160 con 0x7f0ed40af450 2023-10-16T10:03:05.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.486+0000 7f0eda049700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40cc8e0 con 0x7f0ed40cd470 2023-10-16T10:03:05.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0ed415dfd0 0x7f0ed406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57952/0 (socket says 172.21.15.181:57952) 2023-10-16T10:03:05.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed37fe700 1 -- 172.21.15.181:0/393115136 learned_addr learned my addr 172.21.15.181:0/393115136 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:05.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed2ffd700 1 --2- 172.21.15.181:0/393115136 >> v2:172.21.15.190:3300/0 conn(0x7f0ed40af450 0x7f0ed40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed3fff700 1 --2- 172.21.15.181:0/393115136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40cd470 0x7f0ed40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed27fc700 1 -- 172.21.15.181:0/393115136 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906902127 0 0) 0x7f0ed406f870 con 0x7f0ed415dfd0 2023-10-16T10:03:05.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed27fc700 1 -- 172.21.15.181:0/393115136 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ebc0035b0 con 0x7f0ed415dfd0 2023-10-16T10:03:05.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed3fff700 1 -- 172.21.15.181:0/393115136 >> v2:172.21.15.190:3300/0 conn(0x7f0ed40af450 msgr2=0x7f0ed40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:05.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed3fff700 1 --2- 172.21.15.181:0/393115136 >> v2:172.21.15.190:3300/0 conn(0x7f0ed40af450 0x7f0ed40af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed3fff700 1 -- 172.21.15.181:0/393115136 >> v1:172.21.15.181:6789/0 conn(0x7f0ed415dfd0 legacy=0x7f0ed406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed3fff700 1 -- 172.21.15.181:0/393115136 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed40ccbc0 con 0x7f0ed40cd470 2023-10-16T10:03:05.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed2ffd700 1 --2- 172.21.15.181:0/393115136 >> v2:172.21.15.190:3300/0 conn(0x7f0ed40af450 0x7f0ed40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:05.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed3fff700 1 --2- 172.21.15.181:0/393115136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40cd470 0x7f0ed40cd920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009d90 tx=0x7f0ecc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af3d90032896ff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:05.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed27fc700 1 -- 172.21.15.181:0/393115136 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc00e070 con 0x7f0ed40cd470 2023-10-16T10:03:05.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed27fc700 1 -- 172.21.15.181:0/393115136 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ecc00bb20 con 0x7f0ed40cd470 2023-10-16T10:03:05.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.487+0000 7f0ed27fc700 1 -- 172.21.15.181:0/393115136 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc012430 con 0x7f0ed40cd470 2023-10-16T10:03:05.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 -- 172.21.15.181:0/393115136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40cd470 msgr2=0x7f0ed40cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 --2- 172.21.15.181:0/393115136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40cd470 0x7f0ed40cd920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009d90 tx=0x7f0ecc00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:05.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 -- 172.21.15.181:0/393115136 shutdown_connections 2023-10-16T10:03:05.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 --2- 172.21.15.181:0/393115136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40cd470 0x7f0ed40cd920 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 --2- 172.21.15.181:0/393115136 >> v2:172.21.15.190:3300/0 conn(0x7f0ed40af450 0x7f0ed40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 -- 172.21.15.181:0/393115136 >> 172.21.15.181:0/393115136 conn(0x7f0ed40d04a0 msgr2=0x7f0ed40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:05.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 -- 172.21.15.181:0/393115136 shutdown_connections 2023-10-16T10:03:05.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.488+0000 7f0eda049700 1 -- 172.21.15.181:0/393115136 wait complete. 2023-10-16T10:03:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.489+0000 7f0eda049700 1 Processor -- start 2023-10-16T10:03:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.489+0000 7f0eda049700 1 -- start start 2023-10-16T10:03:05.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0eda049700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40af450 0x7f0ed4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0eda049700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ed40cd470 0x7f0ed4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0eda049700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed415dfd0 0x7f0ed415e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0eda049700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed415a1e0 con 0x7f0ed415dfd0 2023-10-16T10:03:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0eda049700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40b1f60 con 0x7f0ed40cd470 2023-10-16T10:03:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0eda049700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40b10b0 con 0x7f0ed40af450 2023-10-16T10:03:05.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0ed3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed415dfd0 0x7f0ed415e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0ed37fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40af450 0x7f0ed4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0ed2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ed40cd470 0x7f0ed4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0ed3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed415dfd0 0x7f0ed415e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46884/0 (socket says 172.21.15.181:46884) 2023-10-16T10:03:05.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.490+0000 7f0ed3fff700 1 -- 172.21.15.181:0/898828846 learned_addr learned my addr 172.21.15.181:0/898828846 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:05.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ed37fe700 1 -- 172.21.15.181:0/898828846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ed40cd470 msgr2=0x7f0ed4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ed37fe700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ed40cd470 0x7f0ed4159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ed37fe700 1 -- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed415dfd0 msgr2=0x7f0ed415e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ed37fe700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed415dfd0 0x7f0ed415e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ed37fe700 1 -- 172.21.15.181:0/898828846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80094a0 con 0x7f0ed40af450 2023-10-16T10:03:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ed3fff700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed415dfd0 0x7f0ed415e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ed37fe700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40af450 0x7f0ed4157a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001360 tx=0x7f0ec400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ebbfff700 1 -- 172.21.15.181:0/898828846 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40072d0 con 0x7f0ed40af450 2023-10-16T10:03:05.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ebbfff700 1 -- 172.21.15.181:0/898828846 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4007430 con 0x7f0ed40af450 2023-10-16T10:03:05.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0ebbfff700 1 -- 172.21.15.181:0/898828846 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4005830 con 0x7f0ed40af450 2023-10-16T10:03:05.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc009a70 con 0x7f0ed40af450 2023-10-16T10:03:05.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.491+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed40b1630 con 0x7f0ed40af450 2023-10-16T10:03:05.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.492+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed4067d20 con 0x7f0ed40af450 2023-10-16T10:03:05.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.492+0000 7f0ebbfff700 1 -- 172.21.15.181:0/898828846 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f0ec4005ba0 con 0x7f0ed40af450 2023-10-16T10:03:05.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.493+0000 7f0ebbfff700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 0x7f0ec006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.493+0000 7f0ed2ffd700 1 -- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 msgr2=0x7f0ec006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:05.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.493+0000 7f0ebbfff700 1 -- 172.21.15.181:0/898828846 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f0ec4089270 con 0x7f0ed40af450 2023-10-16T10:03:05.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.493+0000 7f0ed2ffd700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 0x7f0ec006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:05.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.497+0000 7f0ebbfff700 1 -- 172.21.15.181:0/898828846 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0ec4053cd0 con 0x7f0ed40af450 2023-10-16T10:03:05.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.694+0000 7f0ed2ffd700 1 -- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 msgr2=0x7f0ec006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:05.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.694+0000 7f0ed2ffd700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 0x7f0ec006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:05.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.742+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ed40caec0 con 0x7f0ed40af450 2023-10-16T10:03:05.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.743+0000 7f0ebbfff700 1 -- 172.21.15.181:0/898828846 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f0ec4007da0 con 0x7f0ed40af450 2023-10-16T10:03:05.745 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:05.745 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2945191097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2945191097},{"type":"v1","addr":"172.21.15.190:6834","nonce":2945191097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1168488779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1168488779},{"type":"v1","addr":"172.21.15.190:6835","nonce":1168488779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2677039975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2677039975},{"type":"v1","addr":"172.21.15.181:6836","nonce":2677039975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:10.010346+0000","modified":"2023-10-16T10:02:13.035963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1841075189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1841075189},{"type":"v1","addr":"172.21.15.181:6837","nonce":1841075189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:03:05.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 msgr2=0x7f0ec006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:05.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 0x7f0ec006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40af450 msgr2=0x7f0ed4157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40af450 0x7f0ed4157a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001360 tx=0x7f0ec400c6e0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 shutdown_connections 2023-10-16T10:03:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f0ec006b140 0x7f0ec006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ed40af450 0x7f0ed4157a40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed415dfd0 0x7f0ed415e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 --2- 172.21.15.181:0/898828846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ed40cd470 0x7f0ed4159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 >> 172.21.15.181:0/898828846 conn(0x7f0ed40d04a0 msgr2=0x7f0ed406d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.745+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 shutdown_connections 2023-10-16T10:03:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.746+0000 7f0eda049700 1 -- 172.21.15.181:0/898828846 wait complete. 2023-10-16T10:03:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 58 2023-10-16T10:03:05.760 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:05.793 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:05.792+0000 7fee20cac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:05.870 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:05.869+0000 7fee20cac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:03:05.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.883+0000 7f3519fee700 1 Processor -- start 2023-10-16T10:03:05.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.884+0000 7f3519fee700 1 -- start start 2023-10-16T10:03:05.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.884+0000 7f3519fee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f351415dfd0 0x7f35140af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.884+0000 7f3519fee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351406f870 con 0x7f35140cd470 2023-10-16T10:03:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.884+0000 7f3519fee700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35140d1160 con 0x7f35140af450 2023-10-16T10:03:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.884+0000 7f3519fee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35140cc8e0 con 0x7f351415dfd0 2023-10-16T10:03:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35137fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f351415dfd0 0x7f35140af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f3513fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f35140cd470 0x7f351406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57972/0 (socket says 172.21.15.181:57972) 2023-10-16T10:03:05.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f3513fff700 1 -- 172.21.15.181:0/687060116 learned_addr learned my addr 172.21.15.181:0/687060116 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35137fe700 1 -- 172.21.15.181:0/687060116 >> v1:172.21.15.190:6789/0 conn(0x7f35140af450 legacy=0x7f351406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35137fe700 1 -- 172.21.15.181:0/687060116 >> v1:172.21.15.181:6789/0 conn(0x7f35140cd470 legacy=0x7f351406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35127fc700 1 -- 172.21.15.181:0/687060116 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203449800 0 0) 0x7f351406f870 con 0x7f35140cd470 2023-10-16T10:03:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35137fe700 1 -- 172.21.15.181:0/687060116 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35140ccbc0 con 0x7f351415dfd0 2023-10-16T10:03:05.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35137fe700 1 --2- 172.21.15.181:0/687060116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140af9a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3504009820 tx=0x7f3504009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40402eedea9261a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35127fc700 1 -- 172.21.15.181:0/687060116 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504005e30 con 0x7f351415dfd0 2023-10-16T10:03:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35127fc700 1 -- 172.21.15.181:0/687060116 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3504006030 con 0x7f351415dfd0 2023-10-16T10:03:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.885+0000 7f35127fc700 1 -- 172.21.15.181:0/687060116 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040063e0 con 0x7f351415dfd0 2023-10-16T10:03:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.886+0000 7f3519fee700 1 -- 172.21.15.181:0/687060116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 msgr2=0x7f35140af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.886+0000 7f3519fee700 1 --2- 172.21.15.181:0/687060116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140af9a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3504009820 tx=0x7f3504009b30 comp rx=0 tx=0).stop 2023-10-16T10:03:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.886+0000 7f3519fee700 1 -- 172.21.15.181:0/687060116 shutdown_connections 2023-10-16T10:03:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.886+0000 7f3519fee700 1 --2- 172.21.15.181:0/687060116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140af9a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.886+0000 7f3519fee700 1 -- 172.21.15.181:0/687060116 >> 172.21.15.181:0/687060116 conn(0x7f35140d04a0 msgr2=0x7f35140d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.886+0000 7f3519fee700 1 -- 172.21.15.181:0/687060116 shutdown_connections 2023-10-16T10:03:05.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.886+0000 7f3519fee700 1 -- 172.21.15.181:0/687060116 wait complete. 2023-10-16T10:03:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.887+0000 7f3519fee700 1 Processor -- start 2023-10-16T10:03:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.887+0000 7f3519fee700 1 -- start start 2023-10-16T10:03:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3519fee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35140af450 0x7f35141597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3519fee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35140cd470 0x7f35140b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3519fee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3519fee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140c4be0 con 0x7f351415dfd0 2023-10-16T10:03:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3519fee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140c3fd0 con 0x7f35140cd470 2023-10-16T10:03:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3519fee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35140c42b0 con 0x7f35140af450 2023-10-16T10:03:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46900/0 (socket says 172.21.15.181:46900) 2023-10-16T10:03:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 -- 172.21.15.181:0/3830906768 learned_addr learned my addr 172.21.15.181:0/3830906768 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3512ffd700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35140cd470 0x7f35140b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f35137fe700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35140af450 0x7f35141597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 -- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35140af450 msgr2=0x7f35141597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35140af450 0x7f35141597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 -- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35140cd470 msgr2=0x7f35140b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35140cd470 0x7f35140b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 -- 172.21.15.181:0/3830906768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3504009430 con 0x7f351415dfd0 2023-10-16T10:03:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.888+0000 7f3513fff700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140b5a00 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f350c00cca0 tx=0x7f350c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.889+0000 7f350bfff700 1 -- 172.21.15.181:0/3830906768 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00ab70 con 0x7f351415dfd0 2023-10-16T10:03:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.889+0000 7f350bfff700 1 -- 172.21.15.181:0/3830906768 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c00acd0 con 0x7f351415dfd0 2023-10-16T10:03:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.889+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35140c4590 con 0x7f351415dfd0 2023-10-16T10:03:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.889+0000 7f350bfff700 1 -- 172.21.15.181:0/3830906768 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c007310 con 0x7f351415dfd0 2023-10-16T10:03:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.889+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35140c5070 con 0x7f351415dfd0 2023-10-16T10:03:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.890+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35140c4d50 con 0x7f351415dfd0 2023-10-16T10:03:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.891+0000 7f350bfff700 1 -- 172.21.15.181:0/3830906768 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f350c007470 con 0x7f351415dfd0 2023-10-16T10:03:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.892+0000 7f350bfff700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 0x7f34f406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.892+0000 7f35137fe700 1 -- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 msgr2=0x7f34f406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.892+0000 7f350bfff700 1 -- 172.21.15.181:0/3830906768 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6477+0+0 (secure 0 0 0) 0x7f350c089d90 con 0x7f351415dfd0 2023-10-16T10:03:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.892+0000 7f35137fe700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 0x7f34f406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:05.895+0000 7f350bfff700 1 -- 172.21.15.181:0/3830906768 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f350c0548c0 con 0x7f351415dfd0 2023-10-16T10:03:05.929 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:05.927+0000 7fb3131b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:06.009 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:06.008+0000 7fb3131b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:03:06.014 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:06.013+0000 7fee20cac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:06.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.093+0000 7f35137fe700 1 -- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 msgr2=0x7f34f406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:06.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.093+0000 7f35137fe700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 0x7f34f406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:06.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.099+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3514067d20 con 0x7f351415dfd0 2023-10-16T10:03:06.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.100+0000 7f350bfff700 1 -- 172.21.15.181:0/3830906768 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17820 (secure 0 0 0) 0x7f350c018090 con 0x7f351415dfd0 2023-10-16T10:03:06.103 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:06.103 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":120,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:06.053558+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:08.438030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:08.968951+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:6832/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1068945898":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1396028129":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/2917390541":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1347286988":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1199722576":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/3331655182":"2023-10-17T10:02:24.201085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:06.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 msgr2=0x7f34f406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:06.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 0x7f34f406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 msgr2=0x7f35140b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140b5a00 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f350c00cca0 tx=0x7f350c00a060 comp rx=0 tx=0).stop 2023-10-16T10:03:06.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 shutdown_connections 2023-10-16T10:03:06.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f34f406b140 0x7f34f406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35140af450 0x7f35141597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351415dfd0 0x7f35140b5a00 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 --2- 172.21.15.181:0/3830906768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35140cd470 0x7f35140b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 >> 172.21.15.181:0/3830906768 conn(0x7f35140d04a0 msgr2=0x7f351415d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:06.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 shutdown_connections 2023-10-16T10:03:06.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.103+0000 7f3519fee700 1 -- 172.21.15.181:0/3830906768 wait complete. 2023-10-16T10:03:06.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:02:08.438030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:02:08.968951+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 120, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:06.118 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:06.168 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:06.167+0000 7fb3131b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:06.251 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:06.250+0000 7fee20cac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:03:06.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.252+0000 7f348b324700 1 Processor -- start 2023-10-16T10:03:06.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.252+0000 7f348b324700 1 -- start start 2023-10-16T10:03:06.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f348b324700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34840cb2b0 0x7f348406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f348b324700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3484070590 con 0x7f34840cb690 2023-10-16T10:03:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f348b324700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3484156330 con 0x7f34840b5fd0 2023-10-16T10:03:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f348b324700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348406fcf0 con 0x7f34840cb2b0 2023-10-16T10:03:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f34890c0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34840cb2b0 0x7f348406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f34888bf700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f34840cb690 0x7f34840b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58028/0 (socket says 172.21.15.181:58028) 2023-10-16T10:03:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f34888bf700 1 -- 172.21.15.181:0/3015694285 learned_addr learned my addr 172.21.15.181:0/3015694285 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.253+0000 7f34890c0700 1 -- 172.21.15.181:0/3015694285 >> v1:172.21.15.190:6789/0 conn(0x7f34840b5fd0 legacy=0x7f348415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f34890c0700 1 -- 172.21.15.181:0/3015694285 >> v1:172.21.15.181:6789/0 conn(0x7f34840cb690 legacy=0x7f34840b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f34890c0700 1 -- 172.21.15.181:0/3015694285 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348406ffd0 con 0x7f34840cb2b0 2023-10-16T10:03:06.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f347bfff700 1 -- 172.21.15.181:0/3015694285 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442377190 0 0) 0x7f3484070590 con 0x7f34840cb690 2023-10-16T10:03:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f34890c0700 1 --2- 172.21.15.181:0/3015694285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840cb2b0 0x7f348406f710 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3480009840 tx=0x7f3480009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16769134aa947104 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f347bfff700 1 -- 172.21.15.181:0/3015694285 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480011830 con 0x7f34840cb2b0 2023-10-16T10:03:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f347bfff700 1 -- 172.21.15.181:0/3015694285 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480011990 con 0x7f34840cb2b0 2023-10-16T10:03:06.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f347bfff700 1 -- 172.21.15.181:0/3015694285 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480011d20 con 0x7f34840cb2b0 2023-10-16T10:03:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f348b324700 1 -- 172.21.15.181:0/3015694285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840cb2b0 msgr2=0x7f348406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f348b324700 1 --2- 172.21.15.181:0/3015694285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840cb2b0 0x7f348406f710 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3480009840 tx=0x7f3480009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:06.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f348b324700 1 -- 172.21.15.181:0/3015694285 shutdown_connections 2023-10-16T10:03:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f348b324700 1 --2- 172.21.15.181:0/3015694285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840cb2b0 0x7f348406f710 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.254+0000 7f348b324700 1 -- 172.21.15.181:0/3015694285 >> 172.21.15.181:0/3015694285 conn(0x7f34840cd010 msgr2=0x7f34840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.255+0000 7f348b324700 1 -- 172.21.15.181:0/3015694285 shutdown_connections 2023-10-16T10:03:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.255+0000 7f348b324700 1 -- 172.21.15.181:0/3015694285 wait complete. 2023-10-16T10:03:06.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.256+0000 7f348b324700 1 Processor -- start 2023-10-16T10:03:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.256+0000 7f348b324700 1 -- start start 2023-10-16T10:03:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f348b324700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840b5fd0 0x7f34841f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f348b324700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34840cb2b0 0x7f348415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f348b324700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34840cb690 0x7f34840c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f348b324700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34841f4b20 con 0x7f34840b5fd0 2023-10-16T10:03:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f348b324700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34841f44f0 con 0x7f34840cb690 2023-10-16T10:03:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f348b324700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348415bcd0 con 0x7f34840cb2b0 2023-10-16T10:03:06.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840b5fd0 0x7f34841f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840b5fd0 0x7f34841f4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46956/0 (socket says 172.21.15.181:46956) 2023-10-16T10:03:06.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 -- 172.21.15.181:0/3371351986 learned_addr learned my addr 172.21.15.181:0/3371351986 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:06.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34888bf700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34840cb2b0 0x7f348415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34898c1700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34840cb690 0x7f34840c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 -- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34840cb2b0 msgr2=0x7f348415b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34840cb2b0 0x7f348415b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 -- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34840cb690 msgr2=0x7f34840c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34840cb690 0x7f34840c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.257+0000 7f34890c0700 1 -- 172.21.15.181:0/3371351986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800094a0 con 0x7f34840b5fd0 2023-10-16T10:03:06.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f34888bf700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34840cb2b0 0x7f348415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:06.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f34898c1700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34840cb690 0x7f34840c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:06.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f34890c0700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840b5fd0 0x7f34841f4fb0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f34800090b0 tx=0x7f3480004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:06.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f3479ffb700 1 -- 172.21.15.181:0/3371351986 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348001a980 con 0x7f34840b5fd0 2023-10-16T10:03:06.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f3479ffb700 1 -- 172.21.15.181:0/3371351986 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348001ab00 con 0x7f34840b5fd0 2023-10-16T10:03:06.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f3479ffb700 1 -- 172.21.15.181:0/3371351986 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480024420 con 0x7f34840b5fd0 2023-10-16T10:03:06.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840c4ad0 con 0x7f34840b5fd0 2023-10-16T10:03:06.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.258+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34841f47b0 con 0x7f34840b5fd0 2023-10-16T10:03:06.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.259+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348406f4f0 con 0x7f34840b5fd0 2023-10-16T10:03:06.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.260+0000 7f3479ffb700 1 -- 172.21.15.181:0/3371351986 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f3480022070 con 0x7f34840b5fd0 2023-10-16T10:03:06.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.260+0000 7f3479ffb700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 0x7f346806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.260+0000 7f34888bf700 1 -- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 msgr2=0x7f346806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:06.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.260+0000 7f3479ffb700 1 -- 172.21.15.181:0/3371351986 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f348008b070 con 0x7f34840b5fd0 2023-10-16T10:03:06.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.260+0000 7f34888bf700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 0x7f346806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:06.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.264+0000 7f3479ffb700 1 -- 172.21.15.181:0/3371351986 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3480055b10 con 0x7f34840b5fd0 2023-10-16T10:03:06.324 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:06.323+0000 7fee20cac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:03:06.413 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:06.412+0000 7fb3131b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:03:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.461+0000 7f34888bf700 1 -- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 msgr2=0x7f346806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.461+0000 7f34888bf700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 0x7f346806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:06.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.463+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34840c3df0 con 0x7f34840b5fd0 2023-10-16T10:03:06.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.464+0000 7f3479ffb700 1 -- 172.21.15.181:0/3371351986 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34800290a0 con 0x7f34840b5fd0 2023-10-16T10:03:06.466 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:06.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 msgr2=0x7f346806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:06.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 0x7f346806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840b5fd0 msgr2=0x7f34841f4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840b5fd0 0x7f34841f4fb0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f34800090b0 tx=0x7f3480004750 comp rx=0 tx=0).stop 2023-10-16T10:03:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 shutdown_connections 2023-10-16T10:03:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f346806b140 0x7f346806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34840cb690 0x7f34840c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34840b5fd0 0x7f34841f4fb0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 --2- 172.21.15.181:0/3371351986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34840cb2b0 0x7f348415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 >> 172.21.15.181:0/3371351986 conn(0x7f34840cd010 msgr2=0x7f34840afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:06.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.466+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 shutdown_connections 2023-10-16T10:03:06.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.467+0000 7f348b324700 1 -- 172.21.15.181:0/3371351986 wait complete. 2023-10-16T10:03:06.481 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:03:06.491 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:06.490+0000 7fb3131b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:03:06.589 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:06.588+0000 7fee20cac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:03:06.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.610+0000 7fe9fa576700 1 Processor -- start 2023-10-16T10:03:06.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.610+0000 7fe9fa576700 1 -- start start 2023-10-16T10:03:06.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9fa576700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9fa576700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe9f4071be0 0x7fe9f40b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9fa576700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9f406f3b0 0x7fe9f406f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9fa576700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f40ce1f0 con 0x7fe9f406f3b0 2023-10-16T10:03:06.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9fa576700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f40b2930 con 0x7fe9f4071be0 2023-10-16T10:03:06.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9fa576700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40ccda0 con 0x7fe9f40b2010 2023-10-16T10:03:06.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9f8b13700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9f406f3b0 0x7fe9f406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9f8b13700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9f406f3b0 0x7fe9f406f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46970/0 (socket says 172.21.15.181:46970) 2023-10-16T10:03:06.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9f8b13700 1 -- 172.21.15.181:0/1972773990 learned_addr learned my addr 172.21.15.181:0/1972773990 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:06.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9f3fff700 1 --2- 172.21.15.181:0/1972773990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9f37fe700 1 --2- 172.21.15.181:0/1972773990 >> v2:172.21.15.190:3300/0 conn(0x7fe9f4071be0 0x7fe9f40b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.611+0000 7fe9f8b13700 1 -- 172.21.15.181:0/1972773990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 msgr2=0x7fe9f40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:06.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f8b13700 1 --2- 172.21.15.181:0/1972773990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f8b13700 1 -- 172.21.15.181:0/1972773990 >> v2:172.21.15.190:3300/0 conn(0x7fe9f4071be0 msgr2=0x7fe9f40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:06.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f8b13700 1 --2- 172.21.15.181:0/1972773990 >> v2:172.21.15.190:3300/0 conn(0x7fe9f4071be0 0x7fe9f40b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f8b13700 1 -- 172.21.15.181:0/1972773990 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40727e0 con 0x7fe9f406f3b0 2023-10-16T10:03:06.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f8b13700 1 --2- 172.21.15.181:0/1972773990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f406f3b0 0x7fe9f406f840 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4067600 tx=0x7fe9ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527cba23ab4f53f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:06.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f2ffd700 1 -- 172.21.15.181:0/1972773990 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00e070 con 0x7fe9f406f3b0 2023-10-16T10:03:06.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f2ffd700 1 -- 172.21.15.181:0/1972773990 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec00b910 con 0x7fe9f406f3b0 2023-10-16T10:03:06.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.612+0000 7fe9f2ffd700 1 -- 172.21.15.181:0/1972773990 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00bcc0 con 0x7fe9f406f3b0 2023-10-16T10:03:06.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 -- 172.21.15.181:0/1972773990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f406f3b0 msgr2=0x7fe9f406f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 --2- 172.21.15.181:0/1972773990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f406f3b0 0x7fe9f406f840 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4067600 tx=0x7fe9ec00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:06.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 -- 172.21.15.181:0/1972773990 shutdown_connections 2023-10-16T10:03:06.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 --2- 172.21.15.181:0/1972773990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f406f3b0 0x7fe9f406f840 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 --2- 172.21.15.181:0/1972773990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 --2- 172.21.15.181:0/1972773990 >> v2:172.21.15.190:3300/0 conn(0x7fe9f4071be0 0x7fe9f40b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 -- 172.21.15.181:0/1972773990 >> 172.21.15.181:0/1972773990 conn(0x7fe9f40cd010 msgr2=0x7fe9f40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:06.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 -- 172.21.15.181:0/1972773990 shutdown_connections 2023-10-16T10:03:06.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.613+0000 7fe9fa576700 1 -- 172.21.15.181:0/1972773990 wait complete. 2023-10-16T10:03:06.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.614+0000 7fe9fa576700 1 Processor -- start 2023-10-16T10:03:06.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.614+0000 7fe9fa576700 1 -- start start 2023-10-16T10:03:06.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9fa576700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f406f3b0 0x7fe9f4158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9fa576700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4071be0 0x7fe9f40b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9fa576700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9fa576700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40b4ca0 con 0x7fe9f4071be0 2023-10-16T10:03:06.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9fa576700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40b4f80 con 0x7fe9f406f3b0 2023-10-16T10:03:06.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9fa576700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f415c760 con 0x7fe9f40b2010 2023-10-16T10:03:06.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4071be0 0x7fe9f40b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4071be0 0x7fe9f40b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46972/0 (socket says 172.21.15.181:46972) 2023-10-16T10:03:06.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 -- 172.21.15.181:0/4290353275 learned_addr learned my addr 172.21.15.181:0/4290353275 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:06.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f3fff700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f406f3b0 0x7fe9f4158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f8b13700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:06.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 -- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 msgr2=0x7fe9f40c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 -- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f406f3b0 msgr2=0x7fe9f4158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f406f3b0 0x7fe9f4158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.615+0000 7fe9f37fe700 1 -- 172.21.15.181:0/4290353275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec009b30 con 0x7fe9f4071be0 2023-10-16T10:03:06.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.616+0000 7fe9f37fe700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4071be0 0x7fe9f40b4200 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e870 tx=0x7fe9e800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:06.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.616+0000 7fe9f0ff9700 1 -- 172.21.15.181:0/4290353275 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8013680 con 0x7fe9f4071be0 2023-10-16T10:03:06.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.616+0000 7fe9f0ff9700 1 -- 172.21.15.181:0/4290353275 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e80137e0 con 0x7fe9f4071be0 2023-10-16T10:03:06.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.616+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f415ca40 con 0x7fe9f4071be0 2023-10-16T10:03:06.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.616+0000 7fe9f0ff9700 1 -- 172.21.15.181:0/4290353275 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8013b90 con 0x7fe9f4071be0 2023-10-16T10:03:06.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.616+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f40c7930 con 0x7fe9f4071be0 2023-10-16T10:03:06.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.617+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f415c5c0 con 0x7fe9f4071be0 2023-10-16T10:03:06.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.618+0000 7fe9f0ff9700 1 -- 172.21.15.181:0/4290353275 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fe9e8013cf0 con 0x7fe9f4071be0 2023-10-16T10:03:06.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.619+0000 7fe9f0ff9700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 0x7fe9e006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:06.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.619+0000 7fe9f3fff700 1 -- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 msgr2=0x7fe9e006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:06.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.619+0000 7fe9f0ff9700 1 -- 172.21.15.181:0/4290353275 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fe9e808a030 con 0x7fe9f4071be0 2023-10-16T10:03:06.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.619+0000 7fe9f3fff700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 0x7fe9e006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:06.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.622+0000 7fe9f0ff9700 1 -- 172.21.15.181:0/4290353275 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe9e8054bb0 con 0x7fe9f4071be0 2023-10-16T10:03:06.772 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:06.771+0000 7fb3131b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:03:06.812 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:06.811+0000 7fee20cac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:03:06.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.820+0000 7fe9f3fff700 1 -- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 msgr2=0x7fe9e006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:06.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.820+0000 7fe9f3fff700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 0x7fe9e006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:06.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.824+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9f415c5c0 con 0x7fe9f4071be0 2023-10-16T10:03:06.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.825+0000 7fe9f0ff9700 1 -- 172.21.15.181:0/4290353275 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9e8017020 con 0x7fe9f4071be0 2023-10-16T10:03:06.827 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:03:06.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 msgr2=0x7fe9e006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:06.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 0x7fe9e006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4071be0 msgr2=0x7fe9f40b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4071be0 0x7fe9f40b4200 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e870 tx=0x7fe9e800eb80 comp rx=0 tx=0).stop 2023-10-16T10:03:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 shutdown_connections 2023-10-16T10:03:06.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fe9e006b140 0x7fe9e006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9f40b2010 0x7fe9f40c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9f406f3b0 0x7fe9f4158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.827+0000 7fe9fa576700 1 --2- 172.21.15.181:0/4290353275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4071be0 0x7fe9f40b4200 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:06.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.828+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 >> 172.21.15.181:0/4290353275 conn(0x7fe9f40cd010 msgr2=0x7fe9f40ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:06.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.828+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 shutdown_connections 2023-10-16T10:03:06.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:06.828+0000 7fe9fa576700 1 -- 172.21.15.181:0/4290353275 wait complete. 2023-10-16T10:03:06.846 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:03:06.961 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:06.960+0000 7fee20cac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:03:07.012 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:07.010+0000 7fb3131b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:03:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.012+0000 7fc9b2722700 1 Processor -- start 2023-10-16T10:03:07.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.012+0000 7fc9b2722700 1 -- start start 2023-10-16T10:03:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9b2722700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9ac072760 0x7fc9ac06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9b2722700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac06b210 con 0x7fc9ac15da70 2023-10-16T10:03:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9b2722700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac0cef00 con 0x7fc9ac06ade0 2023-10-16T10:03:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9b2722700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac0cec20 con 0x7fc9ac072760 2023-10-16T10:03:07.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9abfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc9ac15da70 0x7fc9ac0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58060/0 (socket says 172.21.15.181:58060) 2023-10-16T10:03:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9abfff700 1 -- 172.21.15.181:0/2988657289 learned_addr learned my addr 172.21.15.181:0/2988657289 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9ab7fe700 1 --2- 172.21.15.181:0/2988657289 >> v2:172.21.15.190:3300/0 conn(0x7fc9ac072760 0x7fc9ac06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9aaffd700 1 -- 172.21.15.181:0/2988657289 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163178526 0 0) 0x7fc9ac06b210 con 0x7fc9ac15da70 2023-10-16T10:03:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9aaffd700 1 -- 172.21.15.181:0/2988657289 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9940035b0 con 0x7fc9ac15da70 2023-10-16T10:03:07.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9aaffd700 1 -- 172.21.15.181:0/2988657289 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010509057 0 0) 0x7fc9ac0cef00 con 0x7fc9ac06ade0 2023-10-16T10:03:07.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9aaffd700 1 -- 172.21.15.181:0/2988657289 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ac06b210 con 0x7fc9ac06ade0 2023-10-16T10:03:07.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9ab7fe700 1 -- 172.21.15.181:0/2988657289 >> v1:172.21.15.190:6789/0 conn(0x7fc9ac06ade0 legacy=0x7fc9ac1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9ab7fe700 1 -- 172.21.15.181:0/2988657289 >> v1:172.21.15.181:6789/0 conn(0x7fc9ac15da70 legacy=0x7fc9ac0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9ab7fe700 1 -- 172.21.15.181:0/2988657289 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac071c80 con 0x7fc9ac072760 2023-10-16T10:03:07.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9ab7fe700 1 --2- 172.21.15.181:0/2988657289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac072760 0x7fc9ac06a8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009840 tx=0x7fc9a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45664b58e8215edd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:07.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.013+0000 7fc9aaffd700 1 -- 172.21.15.181:0/2988657289 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0011830 con 0x7fc9ac072760 2023-10-16T10:03:07.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9aaffd700 1 -- 172.21.15.181:0/2988657289 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0011990 con 0x7fc9ac072760 2023-10-16T10:03:07.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9aaffd700 1 -- 172.21.15.181:0/2988657289 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0011d00 con 0x7fc9ac072760 2023-10-16T10:03:07.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9b2722700 1 -- 172.21.15.181:0/2988657289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac072760 msgr2=0x7fc9ac06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9b2722700 1 --2- 172.21.15.181:0/2988657289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac072760 0x7fc9ac06a8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009840 tx=0x7fc9a0009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:07.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9b2722700 1 -- 172.21.15.181:0/2988657289 shutdown_connections 2023-10-16T10:03:07.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9b2722700 1 --2- 172.21.15.181:0/2988657289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac072760 0x7fc9ac06a8a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9b2722700 1 -- 172.21.15.181:0/2988657289 >> 172.21.15.181:0/2988657289 conn(0x7fc9ac0cd010 msgr2=0x7fc9ac0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:07.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9b2722700 1 -- 172.21.15.181:0/2988657289 shutdown_connections 2023-10-16T10:03:07.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.014+0000 7fc9b2722700 1 -- 172.21.15.181:0/2988657289 wait complete. 2023-10-16T10:03:07.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.015+0000 7fc9b2722700 1 Processor -- start 2023-10-16T10:03:07.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b2722700 1 -- start start 2023-10-16T10:03:07.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b2722700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9ac06ade0 0x7fc9ac0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b2722700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ac072760 0x7fc9ac154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b2722700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac15da70 0x7fc9ac0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b2722700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0b6020 con 0x7fc9ac06ade0 2023-10-16T10:03:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b2722700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0b4b60 con 0x7fc9ac15da70 2023-10-16T10:03:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b2722700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0b42c0 con 0x7fc9ac072760 2023-10-16T10:03:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.016+0000 7fc9b0cbf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac15da70 0x7fc9ac0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac15da70 0x7fc9ac0adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:52042/0 (socket says 172.21.15.181:52042) 2023-10-16T10:03:07.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9ac06ade0 0x7fc9ac0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 -- 172.21.15.181:0/444325159 learned_addr learned my addr 172.21.15.181:0/444325159 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9ab7fe700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ac072760 0x7fc9ac154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 -- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ac072760 msgr2=0x7fc9ac154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ac072760 0x7fc9ac154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 -- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9ac06ade0 msgr2=0x7fc9ac0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9ac06ade0 0x7fc9ac0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 -- 172.21.15.181:0/444325159 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a00094a0 con 0x7fc9ac15da70 2023-10-16T10:03:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9ab7fe700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ac072760 0x7fc9ac154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b0cbf700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac15da70 0x7fc9ac0adfc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400cbc0 tx=0x7fc9a400ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:07.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9a8ff9700 1 -- 172.21.15.181:0/444325159 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4003c00 con 0x7fc9ac15da70 2023-10-16T10:03:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9a8ff9700 1 -- 172.21.15.181:0/444325159 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4003d60 con 0x7fc9ac15da70 2023-10-16T10:03:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac0b45a0 con 0x7fc9ac15da70 2023-10-16T10:03:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9a8ff9700 1 -- 172.21.15.181:0/444325159 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400aaa0 con 0x7fc9ac15da70 2023-10-16T10:03:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.017+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac15e9e0 con 0x7fc9ac15da70 2023-10-16T10:03:07.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.018+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac06f6c0 con 0x7fc9ac15da70 2023-10-16T10:03:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.019+0000 7fc9a8ff9700 1 -- 172.21.15.181:0/444325159 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc9a400ac00 con 0x7fc9ac15da70 2023-10-16T10:03:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.019+0000 7fc9a8ff9700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 0x7fc99806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.019+0000 7fc9abfff700 1 -- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 msgr2=0x7fc99806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.019+0000 7fc9a8ff9700 1 -- 172.21.15.181:0/444325159 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fc9a4089cf0 con 0x7fc9ac15da70 2023-10-16T10:03:07.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.019+0000 7fc9abfff700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 0x7fc99806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:07.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.028+0000 7fc9a8ff9700 1 -- 172.21.15.181:0/444325159 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc9a408e070 con 0x7fc9ac15da70 2023-10-16T10:03:07.081 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:07.080+0000 7fee20cac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:03:07.172 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:07.171+0000 7fb3131b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:03:07.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.220+0000 7fc9abfff700 1 -- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 msgr2=0x7fc99806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:07.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.220+0000 7fc9abfff700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 0x7fc99806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:07.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.229+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9ac0668b0 con 0x7fc9ac15da70 2023-10-16T10:03:07.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.230+0000 7fc9a8ff9700 1 -- 172.21.15.181:0/444325159 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9a4054710 con 0x7fc9ac15da70 2023-10-16T10:03:07.231 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:03:07.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 msgr2=0x7fc99806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:07.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 0x7fc99806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac15da70 msgr2=0x7fc9ac0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac15da70 0x7fc9ac0adfc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400cbc0 tx=0x7fc9a400ced0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 shutdown_connections 2023-10-16T10:03:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc99806b140 0x7fc99806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9ac15da70 0x7fc9ac0adfc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9ac06ade0 0x7fc9ac0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 --2- 172.21.15.181:0/444325159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9ac072760 0x7fc9ac154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 >> 172.21.15.181:0/444325159 conn(0x7fc9ac0cd010 msgr2=0x7fc9ac0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:07.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 shutdown_connections 2023-10-16T10:03:07.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.232+0000 7fc9b2722700 1 -- 172.21.15.181:0/444325159 wait complete. 2023-10-16T10:03:07.236 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:07.235+0000 7fee20cac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:03:07.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:07.294 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:07.292+0000 7fb3131b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:03:07.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.368+0000 7f56a1f29700 1 Processor -- start 2023-10-16T10:03:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.368+0000 7f56a1f29700 1 -- start start 2023-10-16T10:03:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f56a1f29700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f569c0d1e60 0x7f569c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f56a1f29700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f569c0b3fa0 0x7f569c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f56a1f29700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c0b2f00 con 0x7f569c15dc20 2023-10-16T10:03:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f56a1f29700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c0b3b80 con 0x7f569c0b3fa0 2023-10-16T10:03:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f56a1f29700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c0d2820 con 0x7f569c0d1e60 2023-10-16T10:03:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f569c0b3fa0 0x7f569c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f569c15dc20 0x7f569c0d41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58076/0 (socket says 172.21.15.181:58076) 2023-10-16T10:03:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569b7fe700 1 -- 172.21.15.181:0/1487120171 learned_addr learned my addr 172.21.15.181:0/1487120171 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569affd700 1 --2- 172.21.15.181:0/1487120171 >> v2:172.21.15.190:3300/0 conn(0x7f569c0d1e60 0x7f569c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569a7fc700 1 -- 172.21.15.181:0/1487120171 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908335341 0 0) 0x7f569c0b2f00 con 0x7f569c15dc20 2023-10-16T10:03:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569a7fc700 1 -- 172.21.15.181:0/1487120171 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56840035b0 con 0x7f569c15dc20 2023-10-16T10:03:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569bfff700 1 -- 172.21.15.181:0/1487120171 >> v2:172.21.15.190:3300/0 conn(0x7f569c0d1e60 msgr2=0x7f569c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569bfff700 1 --2- 172.21.15.181:0/1487120171 >> v2:172.21.15.190:3300/0 conn(0x7f569c0d1e60 0x7f569c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569bfff700 1 -- 172.21.15.181:0/1487120171 >> v1:172.21.15.181:6789/0 conn(0x7f569c15dc20 legacy=0x7f569c0d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569bfff700 1 -- 172.21.15.181:0/1487120171 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0b31e0 con 0x7f569c0b3fa0 2023-10-16T10:03:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.369+0000 7f569bfff700 1 --2- 172.21.15.181:0/1487120171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0b43d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f569c161750 tx=0x7f569400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577fdc4fb8b7135 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f569a7fc700 1 -- 172.21.15.181:0/1487120171 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569400e070 con 0x7f569c0b3fa0 2023-10-16T10:03:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f569a7fc700 1 -- 172.21.15.181:0/1487120171 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569400bb20 con 0x7f569c0b3fa0 2023-10-16T10:03:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f569a7fc700 1 -- 172.21.15.181:0/1487120171 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569400be90 con 0x7f569c0b3fa0 2023-10-16T10:03:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 -- 172.21.15.181:0/1487120171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 msgr2=0x7f569c0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 --2- 172.21.15.181:0/1487120171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0b43d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f569c161750 tx=0x7f569400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 -- 172.21.15.181:0/1487120171 shutdown_connections 2023-10-16T10:03:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 --2- 172.21.15.181:0/1487120171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0b43d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 --2- 172.21.15.181:0/1487120171 >> v2:172.21.15.190:3300/0 conn(0x7f569c0d1e60 0x7f569c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 -- 172.21.15.181:0/1487120171 >> 172.21.15.181:0/1487120171 conn(0x7f569c08af90 msgr2=0x7f569c0d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 -- 172.21.15.181:0/1487120171 shutdown_connections 2023-10-16T10:03:07.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.370+0000 7f56a1f29700 1 -- 172.21.15.181:0/1487120171 wait complete. 2023-10-16T10:03:07.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.371+0000 7f56a1f29700 1 Processor -- start 2023-10-16T10:03:07.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f56a1f29700 1 -- start start 2023-10-16T10:03:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f56a1f29700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f56a1f29700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f569c0d1e60 0x7f569c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f56a1f29700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f569c15dc20 0x7f569c1f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f56a1f29700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0ca6c0 con 0x7f569c0b3fa0 2023-10-16T10:03:07.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f56a1f29700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0cbcc0 con 0x7f569c15dc20 2023-10-16T10:03:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f56a1f29700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c15b130 con 0x7f569c0d1e60 2023-10-16T10:03:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47004/0 (socket says 172.21.15.181:47004) 2023-10-16T10:03:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f569b7fe700 1 -- 172.21.15.181:0/3044178277 learned_addr learned my addr 172.21.15.181:0/3044178277 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:07.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f569bfff700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f569c15dc20 0x7f569c1f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.372+0000 7f569affd700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f569c0d1e60 0x7f569c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f569b7fe700 1 -- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f569c0d1e60 msgr2=0x7f569c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f569b7fe700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f569c0d1e60 0x7f569c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f569b7fe700 1 -- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f569c15dc20 msgr2=0x7f569c1f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f569b7fe700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f569c15dc20 0x7f569c1f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f569b7fe700 1 -- 172.21.15.181:0/3044178277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5694009a70 con 0x7f569c0b3fa0 2023-10-16T10:03:07.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f569b7fe700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0cb780 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c500 tx=0x7f568c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:07.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f5683fff700 1 -- 172.21.15.181:0/3044178277 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c0074e0 con 0x7f569c0b3fa0 2023-10-16T10:03:07.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f5683fff700 1 -- 172.21.15.181:0/3044178277 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c007660 con 0x7f569c0b3fa0 2023-10-16T10:03:07.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c0ca3e0 con 0x7f569c0b3fa0 2023-10-16T10:03:07.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f5683fff700 1 -- 172.21.15.181:0/3044178277 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c005a70 con 0x7f569c0b3fa0 2023-10-16T10:03:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.373+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c0a1280 con 0x7f569c0b3fa0 2023-10-16T10:03:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.374+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c0cbe60 con 0x7f569c0b3fa0 2023-10-16T10:03:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.375+0000 7f5683fff700 1 -- 172.21.15.181:0/3044178277 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f568c005bd0 con 0x7f569c0b3fa0 2023-10-16T10:03:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.375+0000 7f5683fff700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 0x7f568806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.375+0000 7f569affd700 1 -- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 msgr2=0x7f568806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.375+0000 7f5683fff700 1 -- 172.21.15.181:0/3044178277 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f568c0890a0 con 0x7f569c0b3fa0 2023-10-16T10:03:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.375+0000 7f569affd700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 0x7f568806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:07.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.380+0000 7f5683fff700 1 -- 172.21.15.181:0/3044178277 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f568c053ab0 con 0x7f569c0b3fa0 2023-10-16T10:03:07.459 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:07.457+0000 7fb3131b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:03:07.498 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:07.497+0000 7fee20cac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.576+0000 7f569affd700 1 -- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 msgr2=0x7f568806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.576+0000 7f569affd700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 0x7f568806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:07.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.621+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f569c15d250 con 0x7f569c0b3fa0 2023-10-16T10:03:07.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.621+0000 7f5683fff700 1 -- 172.21.15.181:0/3044178277 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f568c017090 con 0x7f569c0b3fa0 2023-10-16T10:03:07.624 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:07.624 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2945191097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2945191097},{"type":"v1","addr":"172.21.15.190:6834","nonce":2945191097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1168488779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1168488779},{"type":"v1","addr":"172.21.15.190:6835","nonce":1168488779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2677039975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2677039975},{"type":"v1","addr":"172.21.15.181:6836","nonce":2677039975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:10.010346+0000","modified":"2023-10-16T10:02:13.035963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1841075189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1841075189},{"type":"v1","addr":"172.21.15.181:6837","nonce":1841075189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:03:07.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 msgr2=0x7f568806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:07.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 0x7f568806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 msgr2=0x7f569c0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0cb780 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c500 tx=0x7f568c00c810 comp rx=0 tx=0).stop 2023-10-16T10:03:07.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 shutdown_connections 2023-10-16T10:03:07.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f568806b140 0x7f568806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f569c15dc20 0x7f569c1f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569c0b3fa0 0x7f569c0cb780 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 --2- 172.21.15.181:0/3044178277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f569c0d1e60 0x7f569c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 >> 172.21.15.181:0/3044178277 conn(0x7f569c08af90 msgr2=0x7f569c0b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:07.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 shutdown_connections 2023-10-16T10:03:07.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.624+0000 7f56a1f29700 1 -- 172.21.15.181:0/3044178277 wait complete. 2023-10-16T10:03:07.628 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 58 2023-10-16T10:03:07.638 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:07.657 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:07.656+0000 7fee20cac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:03:07.731 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:07.729+0000 7fb3131b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:07.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.760+0000 7f135e00b700 1 Processor -- start 2023-10-16T10:03:07.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f135e00b700 1 -- start start 2023-10-16T10:03:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f135e00b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580ce7f0 0x7f135806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f135e00b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13580d2390 0x7f13580cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f135e00b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f13580d0180 0x7f1358161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f135e00b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13580d0630 con 0x7f13580d2390 2023-10-16T10:03:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f135e00b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13580cc470 con 0x7f13580d0180 2023-10-16T10:03:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f135e00b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135806ca30 con 0x7f13580ce7f0 2023-10-16T10:03:07.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f1356ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13580d2390 0x7f13580cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f1356ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13580d2390 0x7f13580cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47018/0 (socket says 172.21.15.181:47018) 2023-10-16T10:03:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.761+0000 7f1356ffd700 1 -- 172.21.15.181:0/2008847731 learned_addr learned my addr 172.21.15.181:0/2008847731 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f13577fe700 1 --2- 172.21.15.181:0/2008847731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580ce7f0 0x7f135806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f1357fff700 1 --2- 172.21.15.181:0/2008847731 >> v2:172.21.15.190:3300/0 conn(0x7f13580d0180 0x7f1358161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f1356ffd700 1 -- 172.21.15.181:0/2008847731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580ce7f0 msgr2=0x7f135806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f1356ffd700 1 --2- 172.21.15.181:0/2008847731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580ce7f0 0x7f135806c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f1356ffd700 1 -- 172.21.15.181:0/2008847731 >> v2:172.21.15.190:3300/0 conn(0x7f13580d0180 msgr2=0x7f1358161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f1356ffd700 1 --2- 172.21.15.181:0/2008847731 >> v2:172.21.15.190:3300/0 conn(0x7f13580d0180 0x7f1358161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f1356ffd700 1 -- 172.21.15.181:0/2008847731 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135806cd10 con 0x7f13580d2390 2023-10-16T10:03:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f1356ffd700 1 --2- 172.21.15.181:0/2008847731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d2390 0x7f13580cfc40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f13580c5180 tx=0x7f1340009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38922f6728bfa49 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f13567fc700 1 -- 172.21.15.181:0/2008847731 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340011830 con 0x7f13580d2390 2023-10-16T10:03:07.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f13567fc700 1 -- 172.21.15.181:0/2008847731 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1340011990 con 0x7f13580d2390 2023-10-16T10:03:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.762+0000 7f13567fc700 1 -- 172.21.15.181:0/2008847731 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340011d20 con 0x7f13580d2390 2023-10-16T10:03:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 -- 172.21.15.181:0/2008847731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d2390 msgr2=0x7f13580cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 --2- 172.21.15.181:0/2008847731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d2390 0x7f13580cfc40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f13580c5180 tx=0x7f1340009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 -- 172.21.15.181:0/2008847731 shutdown_connections 2023-10-16T10:03:07.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 --2- 172.21.15.181:0/2008847731 >> v2:172.21.15.190:3300/0 conn(0x7f13580d0180 0x7f1358161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 --2- 172.21.15.181:0/2008847731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580ce7f0 0x7f135806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 --2- 172.21.15.181:0/2008847731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d2390 0x7f13580cfc40 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 -- 172.21.15.181:0/2008847731 >> 172.21.15.181:0/2008847731 conn(0x7f135809a9a0 msgr2=0x7f13580d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 -- 172.21.15.181:0/2008847731 shutdown_connections 2023-10-16T10:03:07.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.763+0000 7f135e00b700 1 -- 172.21.15.181:0/2008847731 wait complete. 2023-10-16T10:03:07.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.764+0000 7f135e00b700 1 Processor -- start 2023-10-16T10:03:07.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.764+0000 7f135e00b700 1 -- start start 2023-10-16T10:03:07.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f135e00b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13580ce7f0 0x7f13580aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f135e00b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d0180 0x7f13581f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f135e00b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580d2390 0x7f13580c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f135e00b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13581f40f0 con 0x7f13580d0180 2023-10-16T10:03:07.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f135e00b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13580b0590 con 0x7f13580ce7f0 2023-10-16T10:03:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f135e00b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13580b0830 con 0x7f13580d2390 2023-10-16T10:03:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f1356ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d0180 0x7f13581f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f1356ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d0180 0x7f13581f42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47022/0 (socket says 172.21.15.181:47022) 2023-10-16T10:03:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f1356ffd700 1 -- 172.21.15.181:0/954646521 learned_addr learned my addr 172.21.15.181:0/954646521 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:07.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f13577fe700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13580ce7f0 0x7f13580aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f1357fff700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580d2390 0x7f13580c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.765+0000 7f1356ffd700 1 -- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580d2390 msgr2=0x7f13580c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f1356ffd700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580d2390 0x7f13580c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f1356ffd700 1 -- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13580ce7f0 msgr2=0x7f13580aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f1356ffd700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13580ce7f0 0x7f13580aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f1356ffd700 1 -- 172.21.15.181:0/954646521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1340009550 con 0x7f13580d0180 2023-10-16T10:03:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f1356ffd700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d0180 0x7f13581f42a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f134ffff700 1 -- 172.21.15.181:0/954646521 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340004240 con 0x7f13580d0180 2023-10-16T10:03:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f134ffff700 1 -- 172.21.15.181:0/954646521 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134001f070 con 0x7f13580d0180 2023-10-16T10:03:07.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13580b0b10 con 0x7f13580d0180 2023-10-16T10:03:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f134ffff700 1 -- 172.21.15.181:0/954646521 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400036a0 con 0x7f13580d0180 2023-10-16T10:03:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.766+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13580b10d0 con 0x7f13580d0180 2023-10-16T10:03:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.767+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1358067d20 con 0x7f13580d0180 2023-10-16T10:03:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.768+0000 7f134ffff700 1 -- 172.21.15.181:0/954646521 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f1340021030 con 0x7f13580d0180 2023-10-16T10:03:07.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.768+0000 7f134ffff700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 0x7f133806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:07.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.768+0000 7f13577fe700 1 -- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 msgr2=0x7f133806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:07.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.768+0000 7f134ffff700 1 -- 172.21.15.181:0/954646521 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f134008ae20 con 0x7f13580d0180 2023-10-16T10:03:07.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.769+0000 7f13577fe700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 0x7f133806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:07.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.773+0000 7f134ffff700 1 -- 172.21.15.181:0/954646521 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f13400558c0 con 0x7f13580d0180 2023-10-16T10:03:07.901 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:07.900+0000 7fb3131b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:03:07.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.970+0000 7f13577fe700 1 -- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 msgr2=0x7f133806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:07.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.970+0000 7f13577fe700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 0x7f133806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:07.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.972+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13580cef40 con 0x7f13580d0180 2023-10-16T10:03:07.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.973+0000 7f134ffff700 1 -- 172.21.15.181:0/954646521 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17820 (secure 0 0 0) 0x7f1340015020 con 0x7f13580d0180 2023-10-16T10:03:07.975 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:07.975 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":120,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:06.053558+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:08.438030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:08.968951+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:6832/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1068945898":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1396028129":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/2917390541":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1347286988":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1199722576":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/3331655182":"2023-10-17T10:02:24.201085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 msgr2=0x7f133806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 0x7f133806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d0180 msgr2=0x7f13581f42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d0180 0x7f13581f42a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1340000c00 tx=0x7f1340004750 comp rx=0 tx=0).stop 2023-10-16T10:03:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 shutdown_connections 2023-10-16T10:03:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7f133806b140 0x7f133806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13580d2390 0x7f13580c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13580ce7f0 0x7f13580aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 --2- 172.21.15.181:0/954646521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13580d0180 0x7f13581f42a0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:07.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 >> 172.21.15.181:0/954646521 conn(0x7f135809a9a0 msgr2=0x7f13580c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:07.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 shutdown_connections 2023-10-16T10:03:07.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:07.975+0000 7f135e00b700 1 -- 172.21.15.181:0/954646521 wait complete. 2023-10-16T10:03:07.989 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:03:07.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.111+0000 7fc1c290e700 1 Processor -- start 2023-10-16T10:03:08.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.111+0000 7fc1c290e700 1 -- start start 2023-10-16T10:03:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c290e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c290e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1bc160d20 0x7fc1bc1611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c290e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1bc1616f0 con 0x7fc1bc161b60 2023-10-16T10:03:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c290e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc162800 con 0x7fc1bc160d20 2023-10-16T10:03:08.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c290e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0d43e0 con 0x7fc1bc06d5f0 2023-10-16T10:03:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c0eab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1bc160d20 0x7fc1bc1611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c0eab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1bc160d20 0x7fc1bc1611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47034/0 (socket says 172.21.15.181:47034) 2023-10-16T10:03:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1c0eab700 1 -- 172.21.15.181:0/1386359077 learned_addr learned my addr 172.21.15.181:0/1386359077 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.112+0000 7fc1bb7fe700 1 --2- 172.21.15.181:0/1386359077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1c0eab700 1 -- 172.21.15.181:0/1386359077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 msgr2=0x7fc1bc0d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1c0eab700 1 --2- 172.21.15.181:0/1386359077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1c0eab700 1 -- 172.21.15.181:0/1386359077 >> v1:172.21.15.190:6789/0 conn(0x7fc1bc161b60 legacy=0x7fc1bc161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1c0eab700 1 -- 172.21.15.181:0/1386359077 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0d2560 con 0x7fc1bc160d20 2023-10-16T10:03:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1baffd700 1 -- 172.21.15.181:0/1386359077 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237159585 0 0) 0x7fc1bc1616f0 con 0x7fc1bc161b60 2023-10-16T10:03:08.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1c0eab700 1 --2- 172.21.15.181:0/1386359077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1611b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009e50 tx=0x7fc1b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8268d8d7d947461a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1baffd700 1 -- 172.21.15.181:0/1386359077 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e070 con 0x7fc1bc160d20 2023-10-16T10:03:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1baffd700 1 -- 172.21.15.181:0/1386359077 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b400b910 con 0x7fc1bc160d20 2023-10-16T10:03:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.113+0000 7fc1baffd700 1 -- 172.21.15.181:0/1386359077 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400bcc0 con 0x7fc1bc160d20 2023-10-16T10:03:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 -- 172.21.15.181:0/1386359077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 msgr2=0x7fc1bc1611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 --2- 172.21.15.181:0/1386359077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1611b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009e50 tx=0x7fc1b400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 -- 172.21.15.181:0/1386359077 shutdown_connections 2023-10-16T10:03:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 --2- 172.21.15.181:0/1386359077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1611b0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 --2- 172.21.15.181:0/1386359077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 -- 172.21.15.181:0/1386359077 >> 172.21.15.181:0/1386359077 conn(0x7fc1bc09cba0 msgr2=0x7fc1bc0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:08.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 -- 172.21.15.181:0/1386359077 shutdown_connections 2023-10-16T10:03:08.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.114+0000 7fc1c290e700 1 -- 172.21.15.181:0/1386359077 wait complete. 2023-10-16T10:03:08.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.115+0000 7fc1c290e700 1 Processor -- start 2023-10-16T10:03:08.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.115+0000 7fc1c290e700 1 -- start start 2023-10-16T10:03:08.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1c290e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1c290e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1c290e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc161b60 0x7fc1bc0ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1c290e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0ca8d0 con 0x7fc1bc160d20 2023-10-16T10:03:08.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1c290e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0cabb0 con 0x7fc1bc161b60 2023-10-16T10:03:08.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1c290e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0a0ce0 con 0x7fc1bc06d5f0 2023-10-16T10:03:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47038/0 (socket says 172.21.15.181:47038) 2023-10-16T10:03:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/2789556698 learned_addr learned my addr 172.21.15.181:0/2789556698 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bbfff700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1c0eab700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc161b60 0x7fc1bc0ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 msgr2=0x7fc1bc0a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc161b60 msgr2=0x7fc1bc0ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc161b60 0x7fc1bc0ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 -- 172.21.15.181:0/2789556698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009b30 con 0x7fc1bc160d20 2023-10-16T10:03:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.116+0000 7fc1bb7fe700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1f2d40 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000e8a0 tx=0x7fc1b000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.117+0000 7fc1b8ff9700 1 -- 172.21.15.181:0/2789556698 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0009590 con 0x7fc1bc160d20 2023-10-16T10:03:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.117+0000 7fc1b8ff9700 1 -- 172.21.15.181:0/2789556698 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b0013680 con 0x7fc1bc160d20 2023-10-16T10:03:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.117+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc0a0fc0 con 0x7fc1bc160d20 2023-10-16T10:03:08.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.117+0000 7fc1b8ff9700 1 -- 172.21.15.181:0/2789556698 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00139f0 con 0x7fc1bc160d20 2023-10-16T10:03:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.117+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc0a09d0 con 0x7fc1bc160d20 2023-10-16T10:03:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.118+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc04ebb0 con 0x7fc1bc160d20 2023-10-16T10:03:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.118+0000 7fc1b8ff9700 1 -- 172.21.15.181:0/2789556698 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 71) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fc1b0013b50 con 0x7fc1bc160d20 2023-10-16T10:03:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.118+0000 7fc1b8ff9700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 0x7fc1a806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.119+0000 7fc1bbfff700 1 -- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 msgr2=0x7fc1a806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.119+0000 7fc1b8ff9700 1 -- 172.21.15.181:0/2789556698 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fc1b0089cb0 con 0x7fc1bc160d20 2023-10-16T10:03:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.119+0000 7fc1bbfff700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 0x7fc1a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:08.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.123+0000 7fc1b8ff9700 1 -- 172.21.15.181:0/2789556698 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc1b00545b0 con 0x7fc1bc160d20 2023-10-16T10:03:08.182 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:08.181+0000 7fee20cac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:03:08.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.320+0000 7fc1bbfff700 1 -- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 msgr2=0x7fc1a806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/716040864 2023-10-16T10:03:08.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.320+0000 7fc1bbfff700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 0x7fc1a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:08.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.372+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1bc0cacf0 con 0x7fc1bc160d20 2023-10-16T10:03:08.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.373+0000 7fc1b8ff9700 1 -- 172.21.15.181:0/2789556698 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fc1b0021090 con 0x7fc1bc160d20 2023-10-16T10:03:08.375 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:08.375 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2945191097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2945191097},{"type":"v1","addr":"172.21.15.190:6834","nonce":2945191097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1168488779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1168488779},{"type":"v1","addr":"172.21.15.190:6835","nonce":1168488779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2677039975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2677039975},{"type":"v1","addr":"172.21.15.181:6836","nonce":2677039975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:10.010346+0000","modified":"2023-10-16T10:02:13.035963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1841075189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1841075189},{"type":"v1","addr":"172.21.15.181:6837","nonce":1841075189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:03:08.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 msgr2=0x7fc1a806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:08.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 0x7fc1a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 msgr2=0x7fc1bc1f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1f2d40 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000e8a0 tx=0x7fc1b000ebb0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 shutdown_connections 2023-10-16T10:03:08.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:6832/716040864,v1:172.21.15.181:6833/716040864] conn(0x7fc1a806b140 0x7fc1a806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc06d5f0 0x7fc1bc0a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc161b60 0x7fc1bc0ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 --2- 172.21.15.181:0/2789556698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1bc160d20 0x7fc1bc1f2d40 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 >> 172.21.15.181:0/2789556698 conn(0x7fc1bc09cba0 msgr2=0x7fc1bc15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:08.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 shutdown_connections 2023-10-16T10:03:08.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.375+0000 7fc1c290e700 1 -- 172.21.15.181:0/2789556698 wait complete. 2023-10-16T10:03:08.380 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 58 2023-10-16T10:03:08.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:08.461 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.459+0000 7fb3131b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:03:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.517+0000 7f5f4f110700 1 Processor -- start 2023-10-16T10:03:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.517+0000 7f5f4f110700 1 -- start start 2023-10-16T10:03:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4f110700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f4815eab0 0x7f5f480d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4f110700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4f110700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4806ed00 con 0x7f5f4815e6d0 2023-10-16T10:03:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4f110700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f480d02d0 con 0x7f5f4815eab0 2023-10-16T10:03:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4f110700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815ad40 con 0x7f5f480cb0c0 2023-10-16T10:03:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4ceac700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5f4815e6d0 0x7f5f4815a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58150/0 (socket says 172.21.15.181:58150) 2023-10-16T10:03:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4ceac700 1 -- 172.21.15.181:0/3958210006 learned_addr learned my addr 172.21.15.181:0/3958210006 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f47fff700 1 --2- 172.21.15.181:0/3958210006 >> v2:172.21.15.190:3300/0 conn(0x7f5f4815eab0 0x7f5f480d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4d6ad700 1 --2- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f477fe700 1 -- 172.21.15.181:0/3958210006 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746616278 0 0) 0x7f5f4806ed00 con 0x7f5f4815e6d0 2023-10-16T10:03:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f477fe700 1 -- 172.21.15.181:0/3958210006 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f380035b0 con 0x7f5f4815e6d0 2023-10-16T10:03:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f47fff700 1 -- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 msgr2=0x7f5f4815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f47fff700 1 --2- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f47fff700 1 -- 172.21.15.181:0/3958210006 >> v1:172.21.15.181:6789/0 conn(0x7f5f4815e6d0 legacy=0x7f5f4815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f47fff700 1 -- 172.21.15.181:0/3958210006 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4806e0f0 con 0x7f5f4815eab0 2023-10-16T10:03:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.518+0000 7f5f4d6ad700 1 --2- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f47fff700 1 --2- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815eab0 0x7f5f480d0fc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009840 tx=0x7f5f34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6e30d378d6b987 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f477fe700 1 -- 172.21.15.181:0/3958210006 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34011830 con 0x7f5f4815eab0 2023-10-16T10:03:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f477fe700 1 -- 172.21.15.181:0/3958210006 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34011990 con 0x7f5f4815eab0 2023-10-16T10:03:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f477fe700 1 -- 172.21.15.181:0/3958210006 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34011d20 con 0x7f5f4815eab0 2023-10-16T10:03:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f4f110700 1 -- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815eab0 msgr2=0x7f5f480d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f4f110700 1 --2- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815eab0 0x7f5f480d0fc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009840 tx=0x7f5f34009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f4f110700 1 -- 172.21.15.181:0/3958210006 shutdown_connections 2023-10-16T10:03:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f4f110700 1 --2- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f4f110700 1 --2- 172.21.15.181:0/3958210006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815eab0 0x7f5f480d0fc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.519+0000 7f5f4f110700 1 -- 172.21.15.181:0/3958210006 >> 172.21.15.181:0/3958210006 conn(0x7f5f480d04a0 msgr2=0x7f5f480d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.520+0000 7f5f4f110700 1 -- 172.21.15.181:0/3958210006 shutdown_connections 2023-10-16T10:03:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.520+0000 7f5f4f110700 1 -- 172.21.15.181:0/3958210006 wait complete. 2023-10-16T10:03:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.521+0000 7f5f4f110700 1 Processor -- start 2023-10-16T10:03:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.521+0000 7f5f4f110700 1 -- start start 2023-10-16T10:03:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4f110700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4f110700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815e6d0 0x7f5f480b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4f110700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f4815eab0 0x7f5f4814e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4f110700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815d690 con 0x7f5f4815eab0 2023-10-16T10:03:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4f110700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480c6a20 con 0x7f5f4815e6d0 2023-10-16T10:03:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4f110700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480c6d00 con 0x7f5f480cb0c0 2023-10-16T10:03:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4d6ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f4815eab0 0x7f5f4814e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815e6d0 0x7f5f480b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4ceac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4d6ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f4815eab0 0x7f5f4814e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47088/0 (socket says 172.21.15.181:47088) 2023-10-16T10:03:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f47fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815e6d0 0x7f5f480b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:52136/0 (socket says 172.21.15.181:52136) 2023-10-16T10:03:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f4d6ad700 1 -- 172.21.15.181:0/3505591913 learned_addr learned my addr 172.21.15.181:0/3505591913 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f47fff700 1 -- 172.21.15.181:0/3505591913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 msgr2=0x7f5f4815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f47fff700 1 --2- 172.21.15.181:0/3505591913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f47fff700 1 -- 172.21.15.181:0/3505591913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f4815eab0 msgr2=0x7f5f4814e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f47fff700 1 --2- 172.21.15.181:0/3505591913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f4815eab0 0x7f5f4814e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.522+0000 7f5f47fff700 1 -- 172.21.15.181:0/3505591913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f40009a70 con 0x7f5f4815e6d0 2023-10-16T10:03:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.523+0000 7f5f47fff700 1 --2- 172.21.15.181:0/3505591913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815e6d0 0x7f5f480b58b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5f34016000 tx=0x7f5f34004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.523+0000 7f5f457fa700 1 -- 172.21.15.181:0/3505591913 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34004510 con 0x7f5f4815e6d0 2023-10-16T10:03:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.523+0000 7f5f457fa700 1 -- 172.21.15.181:0/3505591913 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f340036a0 con 0x7f5f4815e6d0 2023-10-16T10:03:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.523+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f340094a0 con 0x7f5f4815e6d0 2023-10-16T10:03:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.523+0000 7f5f457fa700 1 -- 172.21.15.181:0/3505591913 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34003a50 con 0x7f5f4815e6d0 2023-10-16T10:03:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.523+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f480c4fc0 con 0x7f5f4815e6d0 2023-10-16T10:03:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.524+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f48067d20 con 0x7f5f4815e6d0 2023-10-16T10:03:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.524+0000 7f5f457fa700 1 -- 172.21.15.181:0/3505591913 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 72) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f5f34003bb0 con 0x7f5f4815e6d0 2023-10-16T10:03:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.525+0000 7f5f457fa700 1 -- 172.21.15.181:0/3505591913 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6935+0+0 (secure 0 0 0) 0x7f5f3408a7d0 con 0x7f5f4815e6d0 2023-10-16T10:03:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.529+0000 7f5f457fa700 1 -- 172.21.15.181:0/3505591913 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5f34055780 con 0x7f5f4815e6d0 2023-10-16T10:03:08.543 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28e9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.544 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28e9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.544 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28e9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.544 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28e9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.544 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28e9dc700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.545 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28b996700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.545 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28b996700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.545 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28b996700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.545 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28b996700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.545 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:08.542+0000 7fb28b996700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:08.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.771+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f480cb610 con 0x7f5f4815e6d0 2023-10-16T10:03:08.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.771+0000 7f5f457fa700 1 -- 172.21.15.181:0/3505591913 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f5f34003e60 con 0x7f5f4815e6d0 2023-10-16T10:03:08.773 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:08.774 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2945191097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2945191097},{"type":"v1","addr":"172.21.15.190:6834","nonce":2945191097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1168488779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1168488779},{"type":"v1","addr":"172.21.15.190:6835","nonce":1168488779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2677039975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2677039975},{"type":"v1","addr":"172.21.15.181:6836","nonce":2677039975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:02:10.010346+0000","modified":"2023-10-16T10:02:13.035963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1841075189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1841075189},{"type":"v1","addr":"172.21.15.181:6837","nonce":1841075189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-16T10:03:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.773+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815e6d0 msgr2=0x7f5f480b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.773+0000 7f5f4f110700 1 --2- 172.21.15.181:0/3505591913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815e6d0 0x7f5f480b58b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5f34016000 tx=0x7f5f34004750 comp rx=0 tx=0).stop 2023-10-16T10:03:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.774+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 shutdown_connections 2023-10-16T10:03:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.774+0000 7f5f4f110700 1 --2- 172.21.15.181:0/3505591913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f480cb0c0 0x7f5f4815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.774+0000 7f5f4f110700 1 --2- 172.21.15.181:0/3505591913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f4815eab0 0x7f5f4814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.774+0000 7f5f4f110700 1 --2- 172.21.15.181:0/3505591913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f4815e6d0 0x7f5f480b58b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.774+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 >> 172.21.15.181:0/3505591913 conn(0x7f5f480d04a0 msgr2=0x7f5f480aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.774+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 shutdown_connections 2023-10-16T10:03:08.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.774+0000 7f5f4f110700 1 -- 172.21.15.181:0/3505591913 wait complete. 2023-10-16T10:03:08.778 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 58 2023-10-16T10:03:08.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:08.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.907+0000 7fe817bf5700 1 Processor -- start 2023-10-16T10:03:08.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.907+0000 7fe817bf5700 1 -- start start 2023-10-16T10:03:08.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe817bf5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810161f60 0x7fe81006ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe817bf5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe810162340 0x7fe8100cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe817bf5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81006b600 con 0x7fe81006bfd0 2023-10-16T10:03:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe817bf5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8100ca0a0 con 0x7fe810162340 2023-10-16T10:03:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe817bf5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8100b2660 con 0x7fe810161f60 2023-10-16T10:03:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe815190700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe81006bfd0 0x7fe81006cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58212/0 (socket says 172.21.15.181:58212) 2023-10-16T10:03:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe815190700 1 -- 172.21.15.181:0/2981054102 learned_addr learned my addr 172.21.15.181:0/2981054102 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe815991700 1 --2- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810161f60 0x7fe81006ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe816192700 1 --2- 172.21.15.181:0/2981054102 >> v2:172.21.15.190:3300/0 conn(0x7fe810162340 0x7fe8100cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe81498f700 1 -- 172.21.15.181:0/2981054102 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640154587 0 0) 0x7fe81006b600 con 0x7fe81006bfd0 2023-10-16T10:03:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe81498f700 1 -- 172.21.15.181:0/2981054102 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7fc0035b0 con 0x7fe81006bfd0 2023-10-16T10:03:08.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe816192700 1 -- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810161f60 msgr2=0x7fe81006ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:08.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe816192700 1 --2- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810161f60 0x7fe81006ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe816192700 1 -- 172.21.15.181:0/2981054102 >> v1:172.21.15.181:6789/0 conn(0x7fe81006bfd0 legacy=0x7fe81006cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.908+0000 7fe816192700 1 -- 172.21.15.181:0/2981054102 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810066850 con 0x7fe810162340 2023-10-16T10:03:08.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe816192700 1 --2- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe810162340 0x7fe8100cedd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe808009d90 tx=0x7fe80800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55559d21fa292ff5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:08.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe81498f700 1 -- 172.21.15.181:0/2981054102 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800e070 con 0x7fe810162340 2023-10-16T10:03:08.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe81498f700 1 -- 172.21.15.181:0/2981054102 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80800bb20 con 0x7fe810162340 2023-10-16T10:03:08.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe81498f700 1 -- 172.21.15.181:0/2981054102 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808012430 con 0x7fe810162340 2023-10-16T10:03:08.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 -- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe810162340 msgr2=0x7fe8100cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 --2- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe810162340 0x7fe8100cedd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe808009d90 tx=0x7fe80800b330 comp rx=0 tx=0).stop 2023-10-16T10:03:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 -- 172.21.15.181:0/2981054102 shutdown_connections 2023-10-16T10:03:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 --2- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe810162340 0x7fe8100cedd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 --2- 172.21.15.181:0/2981054102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810161f60 0x7fe81006ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 -- 172.21.15.181:0/2981054102 >> 172.21.15.181:0/2981054102 conn(0x7fe8100b1780 msgr2=0x7fe8100b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:08.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 -- 172.21.15.181:0/2981054102 shutdown_connections 2023-10-16T10:03:08.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.909+0000 7fe817bf5700 1 -- 172.21.15.181:0/2981054102 wait complete. 2023-10-16T10:03:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.910+0000 7fe817bf5700 1 Processor -- start 2023-10-16T10:03:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe817bf5700 1 -- start start 2023-10-16T10:03:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe817bf5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe81006bfd0 0x7fe8100b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe817bf5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe810161f60 0x7fe810159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe817bf5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810162340 0x7fe8100b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe817bf5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8100b2d40 con 0x7fe810161f60 2023-10-16T10:03:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe817bf5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81015a0e0 con 0x7fe81006bfd0 2023-10-16T10:03:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe817bf5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8100c4f90 con 0x7fe810162340 2023-10-16T10:03:08.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe815190700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe810161f60 0x7fe810159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe815190700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe810161f60 0x7fe810159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47138/0 (socket says 172.21.15.181:47138) 2023-10-16T10:03:08.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe815190700 1 -- 172.21.15.181:0/1431259680 learned_addr learned my addr 172.21.15.181:0/1431259680 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:08.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe816192700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810162340 0x7fe8100b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.911+0000 7fe815991700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe81006bfd0 0x7fe8100b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe815190700 1 -- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810162340 msgr2=0x7fe8100b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe815190700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810162340 0x7fe8100b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe815190700 1 -- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe81006bfd0 msgr2=0x7fe8100b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe815190700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe81006bfd0 0x7fe8100b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:08.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe815190700 1 -- 172.21.15.181:0/1431259680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009a70 con 0x7fe810161f60 2023-10-16T10:03:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe815190700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe810161f60 0x7fe810159490 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe800005910 tx=0x7fe80000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe8067fc700 1 -- 172.21.15.181:0/1431259680 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800005d70 con 0x7fe810161f60 2023-10-16T10:03:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe8067fc700 1 -- 172.21.15.181:0/1431259680 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8000070d0 con 0x7fe810161f60 2023-10-16T10:03:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8100c41a0 con 0x7fe810161f60 2023-10-16T10:03:08.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe8067fc700 1 -- 172.21.15.181:0/1431259680 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800007460 con 0x7fe810161f60 2023-10-16T10:03:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.912+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8100c4760 con 0x7fe810161f60 2023-10-16T10:03:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.913+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81006bdf0 con 0x7fe810161f60 2023-10-16T10:03:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.913+0000 7fe8067fc700 1 -- 172.21.15.181:0/1431259680 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 72) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe8000075c0 con 0x7fe810161f60 2023-10-16T10:03:08.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.914+0000 7fe8067fc700 1 -- 172.21.15.181:0/1431259680 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6935+0+0 (secure 0 0 0) 0x7fe800088950 con 0x7fe810161f60 2023-10-16T10:03:08.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:08.918+0000 7fe8067fc700 1 -- 172.21.15.181:0/1431259680 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe800053980 con 0x7fe810161f60 2023-10-16T10:03:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.118+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe81006b230 con 0x7fe810161f60 2023-10-16T10:03:09.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.119+0000 7fe8067fc700 1 -- 172.21.15.181:0/1431259680 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+18407 (secure 0 0 0) 0x7fe800056fa0 con 0x7fe810161f60 2023-10-16T10:03:09.121 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:09.122 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":121,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:08.473276+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:02:08.438030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-16T10:02:08.968951+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3393593592":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/535816161":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/1900611548":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/3011175383":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/431381719":"2023-10-17T10:03:08.473205+0000","172.21.15.181:6833/716040864":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/2599688587":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/862922257":"2023-10-17T10:03:08.473205+0000","172.21.15.181:6833/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:6832/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:6832/716040864":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/1068945898":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1396028129":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/3417874540":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/2917390541":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1347286988":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1199722576":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/3331655182":"2023-10-17T10:02:24.201085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.121+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe810161f60 msgr2=0x7fe810159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.121+0000 7fe817bf5700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe810161f60 0x7fe810159490 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe800005910 tx=0x7fe80000c9e0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.122+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 shutdown_connections 2023-10-16T10:03:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.122+0000 7fe817bf5700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe810162340 0x7fe8100b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.122+0000 7fe817bf5700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe81006bfd0 0x7fe8100b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.122+0000 7fe817bf5700 1 --2- 172.21.15.181:0/1431259680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe810161f60 0x7fe810159490 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.122+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 >> 172.21.15.181:0/1431259680 conn(0x7fe8100b1780 msgr2=0x7fe8100ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:09.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.122+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 shutdown_connections 2023-10-16T10:03:09.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.122+0000 7fe817bf5700 1 -- 172.21.15.181:0/1431259680 wait complete. 2023-10-16T10:03:09.136 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:03:09.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.256+0000 7f08c253b700 1 Processor -- start 2023-10-16T10:03:09.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.256+0000 7f08c253b700 1 -- start start 2023-10-16T10:03:09.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.256+0000 7f08c253b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc0cd4b0 0x7f08bc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.256+0000 7f08c253b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc06c700 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.256+0000 7f08c253b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0cf460 con 0x7f08bc06d210 2023-10-16T10:03:09.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.256+0000 7f08c253b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0d0380 con 0x7f08bc0cd4b0 2023-10-16T10:03:09.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08bb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f08bc0ce4d0 0x7f08bc15d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58222/0 (socket says 172.21.15.181:58222) 2023-10-16T10:03:09.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08bb7fe700 1 -- 172.21.15.181:0/3701893151 learned_addr learned my addr 172.21.15.181:0/3701893151 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:09.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08c0ad8700 1 --2- 172.21.15.181:0/3701893151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc0cd4b0 0x7f08bc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759190315 0 0) 0x7f08bc06c700 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a40035b0 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368324573 0 0) 0x7f08bc0cf460 con 0x7f08bc06d210 2023-10-16T10:03:09.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08bc06c700 con 0x7f08bc06d210 2023-10-16T10:03:09.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2855327767 0 0) 0x7f08a40035b0 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08c0ad8700 1 -- 172.21.15.181:0/3701893151 >> v1:172.21.15.190:6789/0 conn(0x7f08bc06d210 legacy=0x7f08bc0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08c0ad8700 1 -- 172.21.15.181:0/3701893151 >> v1:172.21.15.181:6789/0 conn(0x7f08bc0ce4d0 legacy=0x7f08bc15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08c0ad8700 1 -- 172.21.15.181:0/3701893151 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc06d5f0 con 0x7f08bc0cd4b0 2023-10-16T10:03:09.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08c0ad8700 1 --2- 172.21.15.181:0/3701893151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc0cd4b0 0x7f08bc0cc4c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009dc0 tx=0x7f08b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e6ed6ca840cb53 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:09.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b400e070 con 0x7f08bc0cd4b0 2023-10-16T10:03:09.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b4003a20 con 0x7f08bc0cd4b0 2023-10-16T10:03:09.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.257+0000 7f08baffd700 1 -- 172.21.15.181:0/3701893151 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0001670 con 0x7f08bc0cd4b0 2023-10-16T10:03:09.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.258+0000 7f08c253b700 1 -- 172.21.15.181:0/3701893151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc0cd4b0 msgr2=0x7f08bc0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.258+0000 7f08c253b700 1 --2- 172.21.15.181:0/3701893151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc0cd4b0 0x7f08bc0cc4c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009dc0 tx=0x7f08b400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:09.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.258+0000 7f08c253b700 1 -- 172.21.15.181:0/3701893151 shutdown_connections 2023-10-16T10:03:09.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.258+0000 7f08c253b700 1 --2- 172.21.15.181:0/3701893151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc0cd4b0 0x7f08bc0cc4c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.258+0000 7f08c253b700 1 -- 172.21.15.181:0/3701893151 >> 172.21.15.181:0/3701893151 conn(0x7f08bc09a9a0 msgr2=0x7f08bc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:09.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.258+0000 7f08c253b700 1 -- 172.21.15.181:0/3701893151 shutdown_connections 2023-10-16T10:03:09.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.258+0000 7f08c253b700 1 -- 172.21.15.181:0/3701893151 wait complete. 2023-10-16T10:03:09.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.259+0000 7f08c253b700 1 Processor -- start 2023-10-16T10:03:09.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.259+0000 7f08c253b700 1 -- start start 2023-10-16T10:03:09.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c253b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc06d210 0x7f08bc09dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c253b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08bc0cd4b0 0x7f08bc1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c253b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc0ce4d0 0x7f08bc0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c253b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15f530 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c253b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc09f170 con 0x7f08bc0cd4b0 2023-10-16T10:03:09.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c253b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc09e560 con 0x7f08bc06d210 2023-10-16T10:03:09.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c0ad8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc0ce4d0 0x7f08bc0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c0ad8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc0ce4d0 0x7f08bc0c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47152/0 (socket says 172.21.15.181:47152) 2023-10-16T10:03:09.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08c0ad8700 1 -- 172.21.15.181:0/2298925280 learned_addr learned my addr 172.21.15.181:0/2298925280 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:09.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08bbfff700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc06d210 0x7f08bc09dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.260+0000 7f08bb7fe700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08bc0cd4b0 0x7f08bc1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c0ad8700 1 -- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc06d210 msgr2=0x7f08bc09dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c0ad8700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc06d210 0x7f08bc09dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c0ad8700 1 -- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08bc0cd4b0 msgr2=0x7f08bc1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c0ad8700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08bc0cd4b0 0x7f08bc1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c0ad8700 1 -- 172.21.15.181:0/2298925280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009a70 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c0ad8700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc0ce4d0 0x7f08bc0c95b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f08b400b370 tx=0x7f08b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:09.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40044c0 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b4004620 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc09e840 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b401d470 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.261+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc09ee00 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.262+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc067d20 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.262+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 72) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f08b401d5d0 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.263+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6935+0+0 (secure 0 0 0) 0x7f08b408aa90 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.267+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f08b4055ac0 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.484+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 73) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f08b4055570 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.485+0000 7f08b8ff9700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f08a8070d60 0x7f08a8073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.486+0000 7f08bbfff700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f08a8070d60 0x7f08a8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.487+0000 7f08bbfff700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f08a8070d60 0x7f08a8073150 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f08ac001220 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:09.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.508+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f08bc067290 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.510 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T10:03:09.509+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:03:09.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.510+0000 7f08b8ff9700 1 -- 172.21.15.181:0/2298925280 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f08b406e950 con 0x7f08bc0ce4d0 2023-10-16T10:03:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.513+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f08a8070d60 msgr2=0x7f08a8073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.513+0000 7f08c253b700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f08a8070d60 0x7f08a8073150 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f08ac001220 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2023-10-16T10:03:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.513+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc0ce4d0 msgr2=0x7f08bc0c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.513+0000 7f08c253b700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc0ce4d0 0x7f08bc0c95b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f08b400b370 tx=0x7f08b4004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 shutdown_connections 2023-10-16T10:03:09.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f08a8070d60 0x7f08a8073150 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08bc06d210 0x7f08bc09dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08bc0ce4d0 0x7f08bc0c95b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 --2- 172.21.15.181:0/2298925280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08bc0cd4b0 0x7f08bc1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 >> 172.21.15.181:0/2298925280 conn(0x7f08bc09a9a0 msgr2=0x7f08bc06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 shutdown_connections 2023-10-16T10:03:09.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.514+0000 7f08c253b700 1 -- 172.21.15.181:0/2298925280 wait complete. 2023-10-16T10:03:09.518 INFO:tasks.ceph.mds.a.smithi181.stderr: -9649> 2023-10-16T10:02:16.592+0000 7f2af794e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:03:09.519 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:03:09.530 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:03:09.552 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T10:03:09.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.652+0000 7fcf61d3c700 1 Processor -- start 2023-10-16T10:03:09.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.652+0000 7fcf61d3c700 1 -- start start 2023-10-16T10:03:09.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf61d3c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf5c0b2010 0x7fcf5c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf61d3c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf61d3c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c0ce1f0 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf61d3c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c0b2930 con 0x7fcf5c0b2010 2023-10-16T10:03:09.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf61d3c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0ccda0 con 0x7fcf5c071be0 2023-10-16T10:03:09.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf5c0b2010 0x7fcf5c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf5c0b2010 0x7fcf5c0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:52240/0 (socket says 172.21.15.181:52240) 2023-10-16T10:03:09.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5bfff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcf5c06f3b0 0x7fcf5c06f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:42212/0 (socket says 172.21.15.181:42212) 2023-10-16T10:03:09.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c0b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:44220/0 (socket says 172.21.15.181:44220) 2023-10-16T10:03:09.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/3434359798 learned_addr learned my addr 172.21.15.181:0/3434359798 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:09.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 msgr2=0x7fcf5c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:09.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5b7fe700 1 --2- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/3434359798 >> v1:172.21.15.190:6789/0 conn(0x7fcf5c06f3b0 legacy=0x7fcf5c06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/3434359798 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0727e0 con 0x7fcf5c0b2010 2023-10-16T10:03:09.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.653+0000 7fcf5affd700 1 --2- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:09.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf5b7fe700 1 --2- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 0x7fcf5c0cc6e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009e50 tx=0x7fcf5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6354b2c91124e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:09.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf5a7fc700 1 -- 172.21.15.181:0/3434359798 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400e070 con 0x7fcf5c0b2010 2023-10-16T10:03:09.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf5a7fc700 1 -- 172.21.15.181:0/3434359798 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5400b910 con 0x7fcf5c0b2010 2023-10-16T10:03:09.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf5a7fc700 1 -- 172.21.15.181:0/3434359798 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400bcc0 con 0x7fcf5c0b2010 2023-10-16T10:03:09.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 -- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 msgr2=0x7fcf5c0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 0x7fcf5c0cc6e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009e50 tx=0x7fcf5400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:09.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 -- 172.21.15.181:0/3434359798 shutdown_connections 2023-10-16T10:03:09.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 0x7fcf5c0cc6e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/3434359798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 -- 172.21.15.181:0/3434359798 >> 172.21.15.181:0/3434359798 conn(0x7fcf5c0cd010 msgr2=0x7fcf5c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:09.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 -- 172.21.15.181:0/3434359798 shutdown_connections 2023-10-16T10:03:09.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.654+0000 7fcf61d3c700 1 -- 172.21.15.181:0/3434359798 wait complete. 2023-10-16T10:03:09.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.656+0000 7fcf61d3c700 1 Processor -- start 2023-10-16T10:03:09.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.656+0000 7fcf61d3c700 1 -- start start 2023-10-16T10:03:09.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf61d3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5c06f3b0 0x7fcf5c14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf61d3c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf61d3c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 0x7fcf5c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf61d3c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1f0a70 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf61d3c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1f4f80 con 0x7fcf5c0b2010 2023-10-16T10:03:09.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf61d3c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0a8510 con 0x7fcf5c071be0 2023-10-16T10:03:09.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5c06f3b0 0x7fcf5c14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5c06f3b0 0x7fcf5c14f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47196/0 (socket says 172.21.15.181:47196) 2023-10-16T10:03:09.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/837090614 learned_addr learned my addr 172.21.15.181:0/837090614 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:09.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5affd700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5bfff700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 0x7fcf5c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 msgr2=0x7fcf5c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 msgr2=0x7fcf5c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:09.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 0x7fcf5c06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:09.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.657+0000 7fcf5b7fe700 1 -- 172.21.15.181:0/837090614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440094a0 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.658+0000 7fcf5b7fe700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5c06f3b0 0x7fcf5c14f3a0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b370 tx=0x7fcf54004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:09.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.658+0000 7fcf53fff700 1 -- 172.21.15.181:0/837090614 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf540048c0 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.658+0000 7fcf53fff700 1 -- 172.21.15.181:0/837090614 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf540092e0 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.658+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf54009b30 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.658+0000 7fcf53fff700 1 -- 172.21.15.181:0/837090614 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5401c680 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.658+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c0a8ad0 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.659+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c0b6510 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.659+0000 7fcf53fff700 1 -- 172.21.15.181:0/837090614 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 73) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcf5401c7e0 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.659+0000 7fcf53fff700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fcf4006ad50 0x7fcf4006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:09.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.660+0000 7fcf53fff700 1 -- 172.21.15.181:0/837090614 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7021+0+0 (secure 0 0 0) 0x7fcf5408b0e0 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.660+0000 7fcf5affd700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fcf4006ad50 0x7fcf4006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:09.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.660+0000 7fcf5affd700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fcf4006ad50 0x7fcf4006d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c159950 tx=0x7fcf44009210 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:09.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.664+0000 7fcf53fff700 1 -- 172.21.15.181:0/837090614 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf54055f30 con 0x7fcf5c06f3b0 2023-10-16T10:03:09.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:09.902+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcf5c0b1070 con 0x7fcf5c06f3b0 2023-10-16T10:03:10.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.513+0000 7fcf53fff700 1 -- 172.21.15.181:0/837090614 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcf54059550 con 0x7fcf5c06f3b0 2023-10-16T10:03:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fcf4006ad50 msgr2=0x7fcf4006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fcf4006ad50 0x7fcf4006d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c159950 tx=0x7fcf44009210 comp rx=0 tx=0).stop 2023-10-16T10:03:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5c06f3b0 msgr2=0x7fcf5c14f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5c06f3b0 0x7fcf5c14f3a0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b370 tx=0x7fcf54004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 shutdown_connections 2023-10-16T10:03:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf5c0b2010 0x7fcf5c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5c06f3b0 0x7fcf5c14f3a0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fcf4006ad50 0x7fcf4006d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 --2- 172.21.15.181:0/837090614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf5c071be0 0x7fcf5c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 >> 172.21.15.181:0/837090614 conn(0x7fcf5c0cd010 msgr2=0x7fcf5c0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 shutdown_connections 2023-10-16T10:03:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.517+0000 7fcf61d3c700 1 -- 172.21.15.181:0/837090614 wait complete. 2023-10-16T10:03:10.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:03:10.597 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:10.620 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:10.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.713+0000 7f31b60f9700 1 Processor -- start 2023-10-16T10:03:10.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.713+0000 7f31b60f9700 1 -- start start 2023-10-16T10:03:10.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31b60f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 0x7f31b00b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31b60f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31b00d0ec0 0x7f31b00b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31b60f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b00cab60 con 0x7f31b00b1c10 2023-10-16T10:03:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31b60f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b00cfaf0 con 0x7f31b00d0ec0 2023-10-16T10:03:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31b60f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00cfdd0 con 0x7f31b015dd30 2023-10-16T10:03:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31affff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f31b00b1c10 0x7f31b006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58306/0 (socket says 172.21.15.181:58306) 2023-10-16T10:03:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31affff700 1 -- 172.21.15.181:0/2140469375 learned_addr learned my addr 172.21.15.181:0/2140469375 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251968809 0 0) 0x7f31b00cab60 con 0x7f31b00b1c10 2023-10-16T10:03:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31980035b0 con 0x7f31b00b1c10 2023-10-16T10:03:10.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31af7fe700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 0x7f31b00b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.714+0000 7f31aeffd700 1 --2- 172.21.15.181:0/2140469375 >> v2:172.21.15.190:3300/0 conn(0x7f31b00d0ec0 0x7f31b00b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1078843834 0 0) 0x7f31980035b0 con 0x7f31b00b1c10 2023-10-16T10:03:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 msgr2=0x7f31b00b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 0x7f31b00b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 >> v2:172.21.15.190:3300/0 conn(0x7f31b00d0ec0 msgr2=0x7f31b00b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 --2- 172.21.15.181:0/2140469375 >> v2:172.21.15.190:3300/0 conn(0x7f31b00d0ec0 0x7f31b00b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00d00b0 con 0x7f31b00b1c10 2023-10-16T10:03:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f31a80019f0 con 0x7f31b00b1c10 2023-10-16T10:03:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 >> v1:172.21.15.181:6789/0 conn(0x7f31b00b1c10 legacy=0x7f31b006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 0x7f3198003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3198003f80 0x7f3198004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3198004d90 0x7f3198005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00cfaf0 con 0x7f3198004d90 2023-10-16T10:03:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00cfdd0 con 0x7f3198003f80 2023-10-16T10:03:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319800baf0 con 0x7f31b015dd30 2023-10-16T10:03:10.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31affff700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3198004d90 0x7f3198005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31af7fe700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 0x7f3198003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31aeffd700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3198003f80 0x7f3198004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31affff700 1 -- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 msgr2=0x7f3198003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:10.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31affff700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b015dd30 0x7f3198003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31affff700 1 -- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3198003f80 msgr2=0x7f3198004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:10.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31affff700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3198003f80 0x7f3198004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31affff700 1 -- 172.21.15.181:0/2140469375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980035b0 con 0x7f3198004d90 2023-10-16T10:03:10.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31affff700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3198004d90 0x7f3198005210 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f31b0071a10 tx=0x7f3198005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88898946d9031979 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:10.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a800a340 con 0x7f3198004d90 2023-10-16T10:03:10.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8012600 con 0x7f3198004d90 2023-10-16T10:03:10.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.715+0000 7f31ae7fc700 1 -- 172.21.15.181:0/2140469375 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a80129b0 con 0x7f3198004d90 2023-10-16T10:03:10.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.716+0000 7f31b60f9700 1 -- 172.21.15.181:0/2140469375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3198004d90 msgr2=0x7f3198005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:10.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.716+0000 7f31b60f9700 1 --2- 172.21.15.181:0/2140469375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3198004d90 0x7f3198005210 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f31b0071a10 tx=0x7f3198005880 comp rx=0 tx=0).stop 2023-10-16T10:03:10.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.716+0000 7f31affff700 1 -- 172.21.15.181:0/2140469375 reap_dead start 2023-10-16T10:03:10.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.716+0000 7f31b60f9700 1 -- 172.21.15.181:0/2140469375 shutdown_connections 2023-10-16T10:03:10.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.716+0000 7f31b60f9700 1 -- 172.21.15.181:0/2140469375 >> 172.21.15.181:0/2140469375 conn(0x7f31b0099a00 msgr2=0x7f31b00d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:10.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.716+0000 7f31b60f9700 1 -- 172.21.15.181:0/2140469375 shutdown_connections 2023-10-16T10:03:10.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.716+0000 7f31b60f9700 1 -- 172.21.15.181:0/2140469375 wait complete. 2023-10-16T10:03:10.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.717+0000 7f31b60f9700 1 Processor -- start 2023-10-16T10:03:10.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.717+0000 7f31b60f9700 1 -- start start 2023-10-16T10:03:10.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31b60f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31b0157840 0x7f31b006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31b60f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b00b61d0 0x7f31b006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31b60f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00b3090 0x7f31b00c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31b60f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00c75e0 con 0x7f31b0157840 2023-10-16T10:03:10.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31b60f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00b3590 con 0x7f31b00b3090 2023-10-16T10:03:10.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31b60f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00c7830 con 0x7f31b00b61d0 2023-10-16T10:03:10.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31b0157840 0x7f31b006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31b0157840 0x7f31b006ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47236/0 (socket says 172.21.15.181:47236) 2023-10-16T10:03:10.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31af7fe700 1 -- 172.21.15.181:0/3030951050 learned_addr learned my addr 172.21.15.181:0/3030951050 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:10.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31affff700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00b3090 0x7f31b00c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.718+0000 7f31aeffd700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b00b61d0 0x7f31b006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31af7fe700 1 -- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b00b61d0 msgr2=0x7f31b006f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:10.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31af7fe700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b00b61d0 0x7f31b006f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31af7fe700 1 -- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00b3090 msgr2=0x7f31b00c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:10.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31af7fe700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00b3090 0x7f31b00c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:10.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31af7fe700 1 -- 172.21.15.181:0/3030951050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00c7b10 con 0x7f31b0157840 2023-10-16T10:03:10.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31af7fe700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31b0157840 0x7f31b006ed20 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f31a4011aa0 tx=0x7f31a4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:10.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4011ed0 con 0x7f31b0157840 2023-10-16T10:03:10.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a400f940 con 0x7f31b0157840 2023-10-16T10:03:10.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a400fcf0 con 0x7f31b0157840 2023-10-16T10:03:10.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b00c7df0 con 0x7f31b0157840 2023-10-16T10:03:10.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.719+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b0157e00 con 0x7f31b0157840 2023-10-16T10:03:10.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.720+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b00c8160 con 0x7f31b0157840 2023-10-16T10:03:10.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.720+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 73) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f31a4014440 con 0x7f31b0157840 2023-10-16T10:03:10.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.721+0000 7f3197fff700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f319c06ad50 0x7f319c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:10.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.721+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f31a4089d00 con 0x7f31b0157840 2023-10-16T10:03:10.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.721+0000 7f31aeffd700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f319c06ad50 0x7f319c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:10.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.721+0000 7f31aeffd700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f319c06ad50 0x7f319c06d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31b01f6460 tx=0x7f31a0009210 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:10.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.725+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31a4054c90 con 0x7f31b0157840 2023-10-16T10:03:10.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:10.916+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f31b0067d20 con 0x7f31b0157840 2023-10-16T10:03:11.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.078+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f31a4090080 con 0x7f31b0157840 2023-10-16T10:03:11.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.517+0000 7f3197fff700 1 -- 172.21.15.181:0/3030951050 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 181+0+0 (secure 0 0 0) 0x7f31a4085d20 con 0x7f31b0157840 2023-10-16T10:03:11.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f319c06ad50 msgr2=0x7f319c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:11.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f319c06ad50 0x7f319c06d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31b01f6460 tx=0x7f31a0009210 comp rx=0 tx=0).stop 2023-10-16T10:03:11.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31b0157840 msgr2=0x7f31b006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:11.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31b0157840 0x7f31b006ed20 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f31a4011aa0 tx=0x7f31a4011db0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 shutdown_connections 2023-10-16T10:03:11.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31b00b3090 0x7f31b00c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31b0157840 0x7f31b006ed20 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f319c06ad50 0x7f319c06d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 --2- 172.21.15.181:0/3030951050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31b00b61d0 0x7f31b006f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 >> 172.21.15.181:0/3030951050 conn(0x7f31b0099a00 msgr2=0x7f31b006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:11.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 shutdown_connections 2023-10-16T10:03:11.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.520+0000 7f31b60f9700 1 -- 172.21.15.181:0/3030951050 wait complete. 2023-10-16T10:03:11.524 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:03:11.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:03:11.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.700+0000 7fa4df942700 1 Processor -- start 2023-10-16T10:03:11.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.700+0000 7fa4df942700 1 -- start start 2023-10-16T10:03:11.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4df942700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa4d80d0fc0 0x7fa4d806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:11.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4df942700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:11.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4df942700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d8162650 con 0x7fa4d80ce610 2023-10-16T10:03:11.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4df942700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d8162190 con 0x7fa4d80d0fc0 2023-10-16T10:03:11.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4df942700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80d0230 con 0x7fa4d80ce9f0 2023-10-16T10:03:11.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4dcedd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa4d80d0fc0 0x7fa4d806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:11.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4dcedd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa4d80d0fc0 0x7fa4d806d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47246/0 (socket says 172.21.15.181:47246) 2023-10-16T10:03:11.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4dcedd700 1 -- 172.21.15.181:0/1621010906 learned_addr learned my addr 172.21.15.181:0/1621010906 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:11.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4dcedd700 1 -- 172.21.15.181:0/1621010906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 msgr2=0x7fa4d806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:11.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4dcedd700 1 --2- 172.21.15.181:0/1621010906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4dcedd700 1 -- 172.21.15.181:0/1621010906 >> v1:172.21.15.190:6789/0 conn(0x7fa4d80ce610 legacy=0x7fa4d80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:11.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.701+0000 7fa4dcedd700 1 -- 172.21.15.181:0/1621010906 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8162370 con 0x7fa4d80d0fc0 2023-10-16T10:03:11.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4dcedd700 1 --2- 172.21.15.181:0/1621010906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d806d660 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098f0 tx=0x7fa4d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb3bca0857d76a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:11.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4cffff700 1 -- 172.21.15.181:0/1621010906 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4004230 con 0x7fa4d80d0fc0 2023-10-16T10:03:11.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4cffff700 1 -- 172.21.15.181:0/1621010906 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4004390 con 0x7fa4d80d0fc0 2023-10-16T10:03:11.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4cffff700 1 -- 172.21.15.181:0/1621010906 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40036a0 con 0x7fa4d80d0fc0 2023-10-16T10:03:11.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 -- 172.21.15.181:0/1621010906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80d0fc0 msgr2=0x7fa4d806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:11.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 --2- 172.21.15.181:0/1621010906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d806d660 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098f0 tx=0x7fa4d4009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:11.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 -- 172.21.15.181:0/1621010906 shutdown_connections 2023-10-16T10:03:11.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 --2- 172.21.15.181:0/1621010906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 --2- 172.21.15.181:0/1621010906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d806d660 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 -- 172.21.15.181:0/1621010906 >> 172.21.15.181:0/1621010906 conn(0x7fa4d80d04a0 msgr2=0x7fa4d80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:11.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 -- 172.21.15.181:0/1621010906 shutdown_connections 2023-10-16T10:03:11.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.702+0000 7fa4df942700 1 -- 172.21.15.181:0/1621010906 wait complete. 2023-10-16T10:03:11.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 Processor -- start 2023-10-16T10:03:11.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 -- start start 2023-10-16T10:03:11.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80ce610 0x7fa4d81f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:11.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:11.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d809c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:11.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d81f4150 con 0x7fa4d80ce610 2023-10-16T10:03:11.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d809a080 con 0x7fa4d80d0fc0 2023-10-16T10:03:11.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4df942700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d815aad0 con 0x7fa4d80ce9f0 2023-10-16T10:03:11.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4dd6de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80ce610 0x7fa4d81f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:11.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4dd6de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80ce610 0x7fa4d81f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47250/0 (socket says 172.21.15.181:47250) 2023-10-16T10:03:11.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.704+0000 7fa4dd6de700 1 -- 172.21.15.181:0/807799148 learned_addr learned my addr 172.21.15.181:0/807799148 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:11.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dcedd700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:11.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4ddedf700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d809c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:11.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dd6de700 1 -- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 msgr2=0x7fa4d815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:11.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dd6de700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d815a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dd6de700 1 -- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4d80d0fc0 msgr2=0x7fa4d809c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:11.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dd6de700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d809c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:11.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dd6de700 1 -- 172.21.15.181:0/807799148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4009550 con 0x7fa4d80ce610 2023-10-16T10:03:11.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dcedd700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:11.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4ddedf700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d809c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:11.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4dd6de700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80ce610 0x7fa4d81f3c10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800c520 tx=0x7fa4c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:11.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4cdffb700 1 -- 172.21.15.181:0/807799148 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c80054e0 con 0x7fa4d80ce610 2023-10-16T10:03:11.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4cdffb700 1 -- 172.21.15.181:0/807799148 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c80036d0 con 0x7fa4d80ce610 2023-10-16T10:03:11.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d809a2d0 con 0x7fa4d80ce610 2023-10-16T10:03:11.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4cdffb700 1 -- 172.21.15.181:0/807799148 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8003a80 con 0x7fa4d80ce610 2023-10-16T10:03:11.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.705+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d809a890 con 0x7fa4d80ce610 2023-10-16T10:03:11.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.706+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d8067d20 con 0x7fa4d80ce610 2023-10-16T10:03:11.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.707+0000 7fa4cdffb700 1 -- 172.21.15.181:0/807799148 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa4c8003be0 con 0x7fa4d80ce610 2023-10-16T10:03:11.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.707+0000 7fa4cdffb700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa4bc06af90 0x7fa4bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:11.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.707+0000 7fa4cdffb700 1 -- 172.21.15.181:0/807799148 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6583+0+0 (secure 0 0 0) 0x7fa4c8088dc0 con 0x7fa4d80ce610 2023-10-16T10:03:11.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.707+0000 7fa4dcedd700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa4bc06af90 0x7fa4bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:11.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.707+0000 7fa4dcedd700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa4bc06af90 0x7fa4bc06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80d13f0 tx=0x7fa4d4008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:11.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.711+0000 7fa4cdffb700 1 -- 172.21.15.181:0/807799148 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa4c8053ba0 con 0x7fa4d80ce610 2023-10-16T10:03:11.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:11.905+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4d8066fb0 con 0x7fa4d80ce610 2023-10-16T10:03:12.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.526+0000 7fa4cdffb700 1 -- 172.21.15.181:0/807799148 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa4c8017020 con 0x7fa4d80ce610 2023-10-16T10:03:12.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa4bc06af90 msgr2=0x7fa4bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:12.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa4bc06af90 0x7fa4bc06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80d13f0 tx=0x7fa4d4008030 comp rx=0 tx=0).stop 2023-10-16T10:03:12.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80ce610 msgr2=0x7fa4d81f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:12.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80ce610 0x7fa4d81f3c10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800c520 tx=0x7fa4c800c8e0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 shutdown_connections 2023-10-16T10:03:12.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4d80d0fc0 0x7fa4d809c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4d80ce610 0x7fa4d81f3c10 secure :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800c520 tx=0x7fa4c800c8e0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa4bc06af90 0x7fa4bc06d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 --2- 172.21.15.181:0/807799148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4d80ce9f0 0x7fa4d815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 >> 172.21.15.181:0/807799148 conn(0x7fa4d80d04a0 msgr2=0x7fa4d80b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:12.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 shutdown_connections 2023-10-16T10:03:12.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.530+0000 7fa4df942700 1 -- 172.21.15.181:0/807799148 wait complete. 2023-10-16T10:03:12.537 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T10:03:12.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:03:12.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d89ace700 1 Processor -- start 2023-10-16T10:03:12.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d89ace700 1 -- start start 2023-10-16T10:03:12.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d89ace700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d8415f150 0x7f2d840cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:12.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d89ace700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d840cd170 0x7f2d840b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:12.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d89ace700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d840b2ae0 con 0x7f2d840ce4f0 2023-10-16T10:03:12.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d89ace700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d840ce970 con 0x7f2d840cd170 2023-10-16T10:03:12.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d89ace700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840b2dc0 con 0x7f2d8415f150 2023-10-16T10:03:12.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.671+0000 7f2d83fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d840cd170 0x7f2d840b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:12.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d83fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d840cd170 0x7f2d840b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47262/0 (socket says 172.21.15.181:47262) 2023-10-16T10:03:12.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d83fff700 1 -- 172.21.15.181:0/646245401 learned_addr learned my addr 172.21.15.181:0/646245401 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:12.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d837fe700 1 --2- 172.21.15.181:0/646245401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d8415f150 0x7f2d840cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:12.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d83fff700 1 -- 172.21.15.181:0/646245401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d8415f150 msgr2=0x7f2d840cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:12.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d83fff700 1 --2- 172.21.15.181:0/646245401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d8415f150 0x7f2d840cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d83fff700 1 -- 172.21.15.181:0/646245401 >> v1:172.21.15.190:6789/0 conn(0x7f2d840ce4f0 legacy=0x7f2d840b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d83fff700 1 -- 172.21.15.181:0/646245401 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840cf9d0 con 0x7f2d840cd170 2023-10-16T10:03:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d83fff700 1 --2- 172.21.15.181:0/646245401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840cd170 0x7f2d840b1670 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009e50 tx=0x7f2d7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a761a8d2dba13d3a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d827fc700 1 -- 172.21.15.181:0/646245401 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400e070 con 0x7f2d840cd170 2023-10-16T10:03:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d827fc700 1 -- 172.21.15.181:0/646245401 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74003a20 con 0x7f2d840cd170 2023-10-16T10:03:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.672+0000 7f2d827fc700 1 -- 172.21.15.181:0/646245401 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74003dd0 con 0x7f2d840cd170 2023-10-16T10:03:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 -- 172.21.15.181:0/646245401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840cd170 msgr2=0x7f2d840b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 --2- 172.21.15.181:0/646245401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840cd170 0x7f2d840b1670 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009e50 tx=0x7f2d7400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 -- 172.21.15.181:0/646245401 shutdown_connections 2023-10-16T10:03:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 --2- 172.21.15.181:0/646245401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840cd170 0x7f2d840b1670 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 --2- 172.21.15.181:0/646245401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d8415f150 0x7f2d840cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 -- 172.21.15.181:0/646245401 >> 172.21.15.181:0/646245401 conn(0x7f2d8409a930 msgr2=0x7f2d840d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 -- 172.21.15.181:0/646245401 shutdown_connections 2023-10-16T10:03:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.673+0000 7f2d89ace700 1 -- 172.21.15.181:0/646245401 wait complete. 2023-10-16T10:03:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.674+0000 7f2d89ace700 1 Processor -- start 2023-10-16T10:03:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.674+0000 7f2d89ace700 1 -- start start 2023-10-16T10:03:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d89ace700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d840cd170 0x7f2d84151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d89ace700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840ce4f0 0x7f2d84156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d89ace700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8415f150 0x7f2d8415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d89ace700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841f2020 con 0x7f2d840ce4f0 2023-10-16T10:03:12.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d89ace700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841f2850 con 0x7f2d8415f150 2023-10-16T10:03:12.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d89ace700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841568b0 con 0x7f2d840cd170 2023-10-16T10:03:12.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840ce4f0 0x7f2d84156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:12.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840ce4f0 0x7f2d84156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47266/0 (socket says 172.21.15.181:47266) 2023-10-16T10:03:12.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 -- 172.21.15.181:0/226739089 learned_addr learned my addr 172.21.15.181:0/226739089 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d837fe700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d840cd170 0x7f2d84151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d83fff700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8415f150 0x7f2d8415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 -- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d840cd170 msgr2=0x7f2d84151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d840cd170 0x7f2d84151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 -- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8415f150 msgr2=0x7f2d8415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8415f150 0x7f2d8415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 -- 172.21.15.181:0/226739089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d74009b30 con 0x7f2d840ce4f0 2023-10-16T10:03:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d837fe700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d840cd170 0x7f2d84151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d83fff700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8415f150 0x7f2d8415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d82ffd700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840ce4f0 0x7f2d84156370 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c500 tx=0x7f2d6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d7bfff700 1 -- 172.21.15.181:0/226739089 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c011790 con 0x7f2d840ce4f0 2023-10-16T10:03:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d7bfff700 1 -- 172.21.15.181:0/226739089 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0118f0 con 0x7f2d840ce4f0 2023-10-16T10:03:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d7bfff700 1 -- 172.21.15.181:0/226739089 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c011ca0 con 0x7f2d840ce4f0 2023-10-16T10:03:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.675+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d84156b80 con 0x7f2d840ce4f0 2023-10-16T10:03:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.676+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d841f24e0 con 0x7f2d840ce4f0 2023-10-16T10:03:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.676+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d84067d20 con 0x7f2d840ce4f0 2023-10-16T10:03:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.677+0000 7f2d7bfff700 1 -- 172.21.15.181:0/226739089 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2d6c01c460 con 0x7f2d840ce4f0 2023-10-16T10:03:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.677+0000 7f2d7bfff700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f2d6406af90 0x7f2d6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.677+0000 7f2d837fe700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f2d6406af90 0x7f2d6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.677+0000 7f2d7bfff700 1 -- 172.21.15.181:0/226739089 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f2d6c089600 con 0x7f2d840ce4f0 2023-10-16T10:03:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.678+0000 7f2d837fe700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f2d6406af90 0x7f2d6406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006fd0 tx=0x7f2d7c008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:12.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.682+0000 7f2d7bfff700 1 -- 172.21.15.181:0/226739089 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d6c0544f0 con 0x7f2d840ce4f0 2023-10-16T10:03:12.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.875+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f2d8406efb0 con 0x7f2d840ce4f0 2023-10-16T10:03:12.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.880+0000 7f2d7bfff700 1 -- 172.21.15.181:0/226739089 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v43) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2d6c057b10 con 0x7f2d840ce4f0 2023-10-16T10:03:12.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f2d6406af90 msgr2=0x7f2d6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:12.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f2d6406af90 0x7f2d6406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006fd0 tx=0x7f2d7c008040 comp rx=0 tx=0).stop 2023-10-16T10:03:12.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840ce4f0 msgr2=0x7f2d84156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:12.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840ce4f0 0x7f2d84156370 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c500 tx=0x7f2d6c00c8c0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 shutdown_connections 2023-10-16T10:03:12.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f2d6406af90 0x7f2d6406d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d840cd170 0x7f2d84151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d8415f150 0x7f2d8415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 --2- 172.21.15.181:0/226739089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840ce4f0 0x7f2d84156370 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:12.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.882+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 >> 172.21.15.181:0/226739089 conn(0x7f2d8409a930 msgr2=0x7f2d841e6550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:12.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.883+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 shutdown_connections 2023-10-16T10:03:12.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:12.883+0000 7f2d89ace700 1 -- 172.21.15.181:0/226739089 wait complete. 2023-10-16T10:03:12.897 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:03:13.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07f0fe4700 1 Processor -- start 2023-10-16T10:03:13.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07f0fe4700 1 -- start start 2023-10-16T10:03:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07f0fe4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec0b2010 0x7f07ec0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07f0fe4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07ec0ce680 0x7f07ec0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07f0fe4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ec0ce1f0 con 0x7f07ec071be0 2023-10-16T10:03:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07f0fe4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ec0ccda0 con 0x7f07ec0ce680 2023-10-16T10:03:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07f0fe4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0727e0 con 0x7f07ec0b2010 2023-10-16T10:03:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07ec0ce680 0x7f07ec0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07ec0ce680 0x7f07ec0ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47276/0 (socket says 172.21.15.181:47276) 2023-10-16T10:03:13.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 -- 172.21.15.181:0/3284585224 learned_addr learned my addr 172.21.15.181:0/3284585224 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 -- 172.21.15.181:0/3284585224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec0b2010 msgr2=0x7f07ec0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 --2- 172.21.15.181:0/3284585224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec0b2010 0x7f07ec0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 -- 172.21.15.181:0/3284585224 >> v1:172.21.15.190:6789/0 conn(0x7f07ec071be0 legacy=0x7f07ec0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 -- 172.21.15.181:0/3284585224 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec072ac0 con 0x7f07ec0ce680 2023-10-16T10:03:13.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07ead9d700 1 --2- 172.21.15.181:0/3284585224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0ceb10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f07e400d150 tx=0x7f07e400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b40ac8c98ff7c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07e959a700 1 -- 172.21.15.181:0/3284585224 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4010070 con 0x7f07ec0ce680 2023-10-16T10:03:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07e959a700 1 -- 172.21.15.181:0/3284585224 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e400dd80 con 0x7f07ec0ce680 2023-10-16T10:03:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.024+0000 7f07e959a700 1 -- 172.21.15.181:0/3284585224 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e400b310 con 0x7f07ec0ce680 2023-10-16T10:03:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 -- 172.21.15.181:0/3284585224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 msgr2=0x7f07ec0ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/3284585224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0ceb10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f07e400d150 tx=0x7f07e400d460 comp rx=0 tx=0).stop 2023-10-16T10:03:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 -- 172.21.15.181:0/3284585224 shutdown_connections 2023-10-16T10:03:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/3284585224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0ceb10 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/3284585224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec0b2010 0x7f07ec0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 -- 172.21.15.181:0/3284585224 >> 172.21.15.181:0/3284585224 conn(0x7f07ec0cd010 msgr2=0x7f07ec0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:13.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 -- 172.21.15.181:0/3284585224 shutdown_connections 2023-10-16T10:03:13.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.025+0000 7f07f0fe4700 1 -- 172.21.15.181:0/3284585224 wait complete. 2023-10-16T10:03:13.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.026+0000 7f07f0fe4700 1 Processor -- start 2023-10-16T10:03:13.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.026+0000 7f07f0fe4700 1 -- start start 2023-10-16T10:03:13.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07f0fe4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec071be0 0x7f07ec15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07f0fe4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ec0b2010 0x7f07ec14f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07f0fe4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07f0fe4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0ad6a0 con 0x7f07ec0ce680 2023-10-16T10:03:13.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07f0fe4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0ad980 con 0x7f07ec0b2010 2023-10-16T10:03:13.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07f0fe4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0adc60 con 0x7f07ec071be0 2023-10-16T10:03:13.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47282/0 (socket says 172.21.15.181:47282) 2023-10-16T10:03:13.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 -- 172.21.15.181:0/4083466288 learned_addr learned my addr 172.21.15.181:0/4083466288 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:13.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07e9d9b700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ec0b2010 0x7f07ec14f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ea59c700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec071be0 0x7f07ec15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 -- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec071be0 msgr2=0x7f07ec15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec071be0 0x7f07ec15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 -- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ec0b2010 msgr2=0x7f07ec14f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ec0b2010 0x7f07ec14f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07ead9d700 1 -- 172.21.15.181:0/4083466288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e400bd40 con 0x7f07ec0ce680 2023-10-16T10:03:13.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.027+0000 7f07e9d9b700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ec0b2010 0x7f07ec14f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:13.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.028+0000 7f07ead9d700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0c7f70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f07ec070460 tx=0x7f07e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.028+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4009a10 con 0x7f07ec0ce680 2023-10-16T10:03:13.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.028+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e4009b90 con 0x7f07ec0ce680 2023-10-16T10:03:13.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.028+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec0adf40 con 0x7f07ec0ce680 2023-10-16T10:03:13.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.028+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4004030 con 0x7f07ec0ce680 2023-10-16T10:03:13.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.028+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec14fda0 con 0x7f07ec0ce680 2023-10-16T10:03:13.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.029+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec0ae460 con 0x7f07ec0ce680 2023-10-16T10:03:13.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.029+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f07e401b070 con 0x7f07ec0ce680 2023-10-16T10:03:13.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.030+0000 7f07daffd700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f07cc06af90 0x7f07cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.030+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f07e408b320 con 0x7f07ec0ce680 2023-10-16T10:03:13.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.030+0000 7f07ea59c700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f07cc06af90 0x7f07cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.030+0000 7f07ea59c700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f07cc06af90 0x7f07cc06d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006fd0 tx=0x7f07dc008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.034+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f07e4056320 con 0x7f07ec0ce680 2023-10-16T10:03:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.086+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f07e40566f0 con 0x7f07ec0ce680 2023-10-16T10:03:13.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.230+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f07ec0ae460 con 0x7f07ec0ce680 2023-10-16T10:03:13.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.233+0000 7f07daffd700 1 -- 172.21.15.181:0/4083466288 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v44) v1 ==== 120+0+0 (secure 0 0 0) 0x7f07ec0ae460 con 0x7f07ec0ce680 2023-10-16T10:03:13.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f07cc06af90 msgr2=0x7f07cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f07cc06af90 0x7f07cc06d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07dc006fd0 tx=0x7f07dc008040 comp rx=0 tx=0).stop 2023-10-16T10:03:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 msgr2=0x7f07ec0c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0c7f70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f07ec070460 tx=0x7f07e4004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 shutdown_connections 2023-10-16T10:03:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f07cc06af90 0x7f07cc06d450 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07ec071be0 0x7f07ec15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ec0ce680 0x7f07ec0c7f70 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 --2- 172.21.15.181:0/4083466288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07ec0b2010 0x7f07ec14f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.235+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 >> 172.21.15.181:0/4083466288 conn(0x7f07ec0cd010 msgr2=0x7f07ec0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:13.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.236+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 shutdown_connections 2023-10-16T10:03:13.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.236+0000 7f07f0fe4700 1 -- 172.21.15.181:0/4083466288 wait complete. 2023-10-16T10:03:13.249 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:03:13.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.370+0000 7f978eeff700 1 Processor -- start 2023-10-16T10:03:13.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.370+0000 7f978eeff700 1 -- start start 2023-10-16T10:03:13.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978eeff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f97880b2010 0x7f97880cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978eeff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978eeff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97880ce1f0 con 0x7f9788071be0 2023-10-16T10:03:13.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978eeff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9788072090 con 0x7f97880b2010 2023-10-16T10:03:13.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978eeff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97880ccda0 con 0x7f97880ce680 2023-10-16T10:03:13.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978cc9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f97880b2010 0x7f97880cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978cc9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f97880b2010 0x7f97880cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47286/0 (socket says 172.21.15.181:47286) 2023-10-16T10:03:13.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978cc9b700 1 -- 172.21.15.181:0/3482298863 learned_addr learned my addr 172.21.15.181:0/3482298863 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:13.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.371+0000 7f978d49c700 1 --2- 172.21.15.181:0/3482298863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978cc9b700 1 -- 172.21.15.181:0/3482298863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 msgr2=0x7f97880ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978cc9b700 1 --2- 172.21.15.181:0/3482298863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978cc9b700 1 -- 172.21.15.181:0/3482298863 >> v1:172.21.15.190:6789/0 conn(0x7f9788071be0 legacy=0x7f97880b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978cc9b700 1 -- 172.21.15.181:0/3482298863 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97880727e0 con 0x7f97880b2010 2023-10-16T10:03:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978cc9b700 1 --2- 172.21.15.181:0/3482298863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f97880cc6e0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f977c0098f0 tx=0x7f977c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194cc66999bcc67 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f97877fe700 1 -- 172.21.15.181:0/3482298863 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00b8a0 con 0x7f97880b2010 2023-10-16T10:03:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f97877fe700 1 -- 172.21.15.181:0/3482298863 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c00ba00 con 0x7f97880b2010 2023-10-16T10:03:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f97877fe700 1 -- 172.21.15.181:0/3482298863 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00bd90 con 0x7f97880b2010 2023-10-16T10:03:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978eeff700 1 -- 172.21.15.181:0/3482298863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 msgr2=0x7f97880cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978eeff700 1 --2- 172.21.15.181:0/3482298863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f97880cc6e0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f977c0098f0 tx=0x7f977c009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978eeff700 1 -- 172.21.15.181:0/3482298863 shutdown_connections 2023-10-16T10:03:13.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978eeff700 1 --2- 172.21.15.181:0/3482298863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978eeff700 1 --2- 172.21.15.181:0/3482298863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f97880cc6e0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.372+0000 7f978eeff700 1 -- 172.21.15.181:0/3482298863 >> 172.21.15.181:0/3482298863 conn(0x7f97880cd010 msgr2=0x7f97880cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.373+0000 7f978eeff700 1 -- 172.21.15.181:0/3482298863 shutdown_connections 2023-10-16T10:03:13.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.373+0000 7f978eeff700 1 -- 172.21.15.181:0/3482298863 wait complete. 2023-10-16T10:03:13.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.374+0000 7f978eeff700 1 Processor -- start 2023-10-16T10:03:13.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.374+0000 7f978eeff700 1 -- start start 2023-10-16T10:03:13.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f978eeff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788071be0 0x7f978814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f978eeff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f978815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f978eeff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f978eeff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9788152390 con 0x7f97880b2010 2023-10-16T10:03:13.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f978eeff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97881500f0 con 0x7f9788071be0 2023-10-16T10:03:13.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f978eeff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97881519f0 con 0x7f97880ce680 2023-10-16T10:03:13.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f9787fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f978815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f9787fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f978815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47294/0 (socket says 172.21.15.181:47294) 2023-10-16T10:03:13.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.375+0000 7f9787fff700 1 -- 172.21.15.181:0/1815272720 learned_addr learned my addr 172.21.15.181:0/1815272720 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:13.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f978cc9b700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788071be0 0x7f978814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f978d49c700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f9787fff700 1 -- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 msgr2=0x7f97880c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f9787fff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f9787fff700 1 -- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788071be0 msgr2=0x7f978814fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f9787fff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788071be0 0x7f978814fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f9787fff700 1 -- 172.21.15.181:0/1815272720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c009550 con 0x7f97880b2010 2023-10-16T10:03:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f978d49c700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f9787fff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f978815ceb0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f977400c430 tx=0x7f977400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f97857fa700 1 -- 172.21.15.181:0/1815272720 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740052d0 con 0x7f97880b2010 2023-10-16T10:03:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f97857fa700 1 -- 172.21.15.181:0/1815272720 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9774005450 con 0x7f97880b2010 2023-10-16T10:03:13.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9788151cd0 con 0x7f97880b2010 2023-10-16T10:03:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f97857fa700 1 -- 172.21.15.181:0/1815272720 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774006aa0 con 0x7f97880b2010 2023-10-16T10:03:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.376+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978815d3f0 con 0x7f97880b2010 2023-10-16T10:03:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.377+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97881524d0 con 0x7f97880b2010 2023-10-16T10:03:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.377+0000 7f97857fa700 1 -- 172.21.15.181:0/1815272720 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9774003680 con 0x7f97880b2010 2023-10-16T10:03:13.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.378+0000 7f97857fa700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f976c06b020 0x7f976c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.378+0000 7f97857fa700 1 -- 172.21.15.181:0/1815272720 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f9774088b70 con 0x7f97880b2010 2023-10-16T10:03:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.378+0000 7f978cc9b700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f976c06b020 0x7f976c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.378+0000 7f978cc9b700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f976c06b020 0x7f976c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f977c014040 tx=0x7f977c008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.382+0000 7f97857fa700 1 -- 172.21.15.181:0/1815272720 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f97881524d0 con 0x7f97880b2010 2023-10-16T10:03:13.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.576+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f97881524d0 con 0x7f97880b2010 2023-10-16T10:03:13.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.579+0000 7f97857fa700 1 -- 172.21.15.181:0/1815272720 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f97881524d0 con 0x7f97880b2010 2023-10-16T10:03:13.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f976c06b020 msgr2=0x7f976c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f976c06b020 0x7f976c06d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f977c014040 tx=0x7f977c008030 comp rx=0 tx=0).stop 2023-10-16T10:03:13.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 msgr2=0x7f978815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f978815ceb0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f977400c430 tx=0x7f977400c7f0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 shutdown_connections 2023-10-16T10:03:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f976c06b020 0x7f976c06d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97880ce680 0x7f97880c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788071be0 0x7f978814fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 --2- 172.21.15.181:0/1815272720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97880b2010 0x7f978815ceb0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.581+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 >> 172.21.15.181:0/1815272720 conn(0x7f97880cd010 msgr2=0x7f97880afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.582+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 shutdown_connections 2023-10-16T10:03:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.582+0000 7f978eeff700 1 -- 172.21.15.181:0/1815272720 wait complete. 2023-10-16T10:03:13.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-16T10:03:13.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.715+0000 7f8a74cf3700 1 Processor -- start 2023-10-16T10:03:13.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.715+0000 7f8a74cf3700 1 -- start start 2023-10-16T10:03:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a74cf3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 0x7f8a700d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a74cf3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8a700cb0c0 0x7f8a7015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a74cf3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7006ed00 con 0x7f8a7015eab0 2023-10-16T10:03:13.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a74cf3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700d02d0 con 0x7f8a700cb0c0 2023-10-16T10:03:13.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a74cf3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7015ad40 con 0x7f8a7015e6d0 2023-10-16T10:03:13.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8a7015eab0 0x7f8a7015a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58378/0 (socket says 172.21.15.181:58378) 2023-10-16T10:03:13.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6dd9b700 1 -- 172.21.15.181:0/3763471253 learned_addr learned my addr 172.21.15.181:0/3763471253 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:13.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/3763471253 >> v2:172.21.15.190:3300/0 conn(0x7f8a700cb0c0 0x7f8a7015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6e59c700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 0x7f8a700d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074590569 0 0) 0x7f8a7006ed00 con 0x7f8a7015eab0 2023-10-16T10:03:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a580035b0 con 0x7f8a7015eab0 2023-10-16T10:03:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590542826 0 0) 0x7f8a580035b0 con 0x7f8a7015eab0 2023-10-16T10:03:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 msgr2=0x7f8a700d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 0x7f8a700d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 >> v2:172.21.15.190:3300/0 conn(0x7f8a700cb0c0 msgr2=0x7f8a7015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 --2- 172.21.15.181:0/3763471253 >> v2:172.21.15.190:3300/0 conn(0x7f8a700cb0c0 0x7f8a7015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7006e0f0 con 0x7f8a7015eab0 2023-10-16T10:03:13.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.716+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f8a64001670 con 0x7f8a7015eab0 2023-10-16T10:03:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 >> v1:172.21.15.181:6789/0 conn(0x7f8a7015eab0 legacy=0x7f8a7015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58003a00 0x7f8a58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 0x7f8a58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700d02d0 con 0x7f8a58004d90 2023-10-16T10:03:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7015ad40 con 0x7f8a58003a00 2023-10-16T10:03:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5800ba10 con 0x7f8a7015e6d0 2023-10-16T10:03:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/3763471253 >> v2:172.21.15.190:3300/0 conn(0x7f8a700cb0c0 0x7f8a7015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6dd9b700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 0x7f8a58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6e59c700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58003a00 0x7f8a58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 msgr2=0x7f8a58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7015e6d0 0x7f8a58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58003a00 msgr2=0x7f8a58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:13.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58003a00 0x7f8a58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/3763471253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580035b0 con 0x7f8a58004d90 2023-10-16T10:03:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e350 tx=0x7f8a58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f92ce5ea00ff8c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800ee10 con 0x7f8a58004d90 2023-10-16T10:03:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6800f040 con 0x7f8a58004d90 2023-10-16T10:03:13.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.717+0000 7f8a6d59a700 1 -- 172.21.15.181:0/3763471253 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68003c40 con 0x7f8a58004d90 2023-10-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.718+0000 7f8a74cf3700 1 -- 172.21.15.181:0/3763471253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a58004d90 msgr2=0x7f8a58005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.718+0000 7f8a74cf3700 1 --2- 172.21.15.181:0/3763471253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a58004d90 0x7f8a58005210 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e350 tx=0x7f8a58005880 comp rx=0 tx=0).stop 2023-10-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.718+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/3763471253 reap_dead start 2023-10-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.718+0000 7f8a74cf3700 1 -- 172.21.15.181:0/3763471253 shutdown_connections 2023-10-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.718+0000 7f8a74cf3700 1 -- 172.21.15.181:0/3763471253 >> 172.21.15.181:0/3763471253 conn(0x7f8a700d04a0 msgr2=0x7f8a700d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.718+0000 7f8a74cf3700 1 -- 172.21.15.181:0/3763471253 shutdown_connections 2023-10-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.718+0000 7f8a74cf3700 1 -- 172.21.15.181:0/3763471253 wait complete. 2023-10-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.719+0000 7f8a74cf3700 1 Processor -- start 2023-10-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.719+0000 7f8a74cf3700 1 -- start start 2023-10-16T10:03:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a74cf3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58004d90 0x7f8a700add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a74cf3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a70154a10 0x7f8a700c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a74cf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a700c9780 0x7f8a700c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a74cf3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ad8e0 con 0x7f8a700c9780 2023-10-16T10:03:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a74cf3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700c9c90 con 0x7f8a58004d90 2023-10-16T10:03:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a74cf3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700ad290 con 0x7f8a70154a10 2023-10-16T10:03:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a6ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a700c9780 0x7f8a700c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a6ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a700c9780 0x7f8a700c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47316/0 (socket says 172.21.15.181:47316) 2023-10-16T10:03:13.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/346128148 learned_addr learned my addr 172.21.15.181:0/346128148 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a6e59c700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58004d90 0x7f8a700add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.720+0000 7f8a6dd9b700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a70154a10 0x7f8a700c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a70154a10 msgr2=0x7f8a700c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a70154a10 0x7f8a700c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58004d90 msgr2=0x7f8a700add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58004d90 0x7f8a700add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a6ed9d700 1 -- 172.21.15.181:0/346128148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700ad570 con 0x7f8a700c9780 2023-10-16T10:03:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a6ed9d700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a700c9780 0x7f8a700c5900 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8a68004000 tx=0x7f8a68009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a5effd700 1 -- 172.21.15.181:0/346128148 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68004440 con 0x7f8a700c9780 2023-10-16T10:03:13.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a5effd700 1 -- 172.21.15.181:0/346128148 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a680045c0 con 0x7f8a700c9780 2023-10-16T10:03:13.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a5effd700 1 -- 172.21.15.181:0/346128148 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68004970 con 0x7f8a700c9780 2023-10-16T10:03:13.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a700ad8e0 con 0x7f8a700c9780 2023-10-16T10:03:13.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.721+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a700c8bc0 con 0x7f8a700c9780 2023-10-16T10:03:13.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.722+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a700ada70 con 0x7f8a700c9780 2023-10-16T10:03:13.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.722+0000 7f8a5effd700 1 -- 172.21.15.181:0/346128148 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8a68025020 con 0x7f8a700c9780 2023-10-16T10:03:13.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.723+0000 7f8a5effd700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f8a5006b020 0x7f8a5006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:13.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.723+0000 7f8a5effd700 1 -- 172.21.15.181:0/346128148 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f8a6808b950 con 0x7f8a700c9780 2023-10-16T10:03:13.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.723+0000 7f8a6e59c700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f8a5006b020 0x7f8a5006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:13.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.723+0000 7f8a6e59c700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f8a5006b020 0x7f8a5006d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009bc0 tx=0x7f8a600091c0 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:13.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:13.727+0000 7f8a5effd700 1 -- 172.21.15.181:0/346128148 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a680568d0 con 0x7f8a700c9780 2023-10-16T10:03:14.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.000+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f8a700ad700 con 0x7f8a700c9780 2023-10-16T10:03:14.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.531+0000 7f8a5effd700 1 -- 172.21.15.181:0/346128148 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v722) v1 ==== 148+0+0 (secure 0 0 0) 0x7f8a68059ef0 con 0x7f8a700c9780 2023-10-16T10:03:14.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.533+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f8a5006b020 msgr2=0x7f8a5006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f8a5006b020 0x7f8a5006d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009bc0 tx=0x7f8a600091c0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a700c9780 msgr2=0x7f8a700c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a700c9780 0x7f8a700c5900 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8a68004000 tx=0x7f8a68009750 comp rx=0 tx=0).stop 2023-10-16T10:03:14.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 shutdown_connections 2023-10-16T10:03:14.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a700c9780 0x7f8a700c5900 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a58004d90 0x7f8a700add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f8a5006b020 0x7f8a5006d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 --2- 172.21.15.181:0/346128148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a70154a10 0x7f8a700c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 >> 172.21.15.181:0/346128148 conn(0x7f8a700d04a0 msgr2=0x7f8a700ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:14.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 shutdown_connections 2023-10-16T10:03:14.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.534+0000 7f8a74cf3700 1 -- 172.21.15.181:0/346128148 wait complete. 2023-10-16T10:03:14.549 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-16T10:03:14.549 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:03:14.550 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:14.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.674+0000 7fd56236c700 1 Processor -- start 2023-10-16T10:03:14.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.674+0000 7fd56236c700 1 -- start start 2023-10-16T10:03:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd56236c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd55c15dfd0 0x7fd55c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd56236c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 0x7fd55c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd56236c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55c06f870 con 0x7fd55c0af450 2023-10-16T10:03:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd56236c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c0cde40 con 0x7fd55c15dfd0 2023-10-16T10:03:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd56236c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0d1160 con 0x7fd55c0cd470 2023-10-16T10:03:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd55b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd55c0af450 0x7fd55c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58398/0 (socket says 172.21.15.181:58398) 2023-10-16T10:03:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd55b7fe700 1 -- 172.21.15.181:0/565254236 learned_addr learned my addr 172.21.15.181:0/565254236 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939744165 0 0) 0x7fd55c06f870 con 0x7fd55c0af450 2023-10-16T10:03:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5440035b0 con 0x7fd55c0af450 2023-10-16T10:03:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd560909700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 0x7fd55c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.675+0000 7fd55bfff700 1 --2- 172.21.15.181:0/565254236 >> v2:172.21.15.190:3300/0 conn(0x7fd55c15dfd0 0x7fd55c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767011485 0 0) 0x7fd5440035b0 con 0x7fd55c0af450 2023-10-16T10:03:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 msgr2=0x7fd55c0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 0x7fd55c0cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 >> v2:172.21.15.190:3300/0 conn(0x7fd55c15dfd0 msgr2=0x7fd55c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 --2- 172.21.15.181:0/565254236 >> v2:172.21.15.190:3300/0 conn(0x7fd55c15dfd0 0x7fd55c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0cc8e0 con 0x7fd55c0af450 2023-10-16T10:03:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd54c001670 con 0x7fd55c0af450 2023-10-16T10:03:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 >> v1:172.21.15.181:6789/0 conn(0x7fd55c0af450 legacy=0x7fd55c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd544003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 0x7fd544004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd544004d90 0x7fd544005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0cde40 con 0x7fd544003a00 2023-10-16T10:03:14.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0d1160 con 0x7fd544004d90 2023-10-16T10:03:14.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54400bba0 con 0x7fd55c0cd470 2023-10-16T10:03:14.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55bfff700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd544003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55b7fe700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 0x7fd544004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd560909700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd544004d90 0x7fd544005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55bfff700 1 -- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 msgr2=0x7fd544004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55bfff700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0cd470 0x7fd544004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55bfff700 1 -- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd544004d90 msgr2=0x7fd544005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55bfff700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd544004d90 0x7fd544005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55bfff700 1 -- 172.21.15.181:0/565254236 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440035b0 con 0x7fd544003a00 2023-10-16T10:03:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55bfff700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd544003e20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd550009880 tx=0x7fd544005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d24d330a4e3483 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000c070 con 0x7fd544003a00 2023-10-16T10:03:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550005c20 con 0x7fd544003a00 2023-10-16T10:03:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.676+0000 7fd55affd700 1 -- 172.21.15.181:0/565254236 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550006050 con 0x7fd544003a00 2023-10-16T10:03:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.677+0000 7fd56236c700 1 -- 172.21.15.181:0/565254236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 msgr2=0x7fd544003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.677+0000 7fd56236c700 1 --2- 172.21.15.181:0/565254236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd544003e20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd550009880 tx=0x7fd544005880 comp rx=0 tx=0).stop 2023-10-16T10:03:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.677+0000 7fd560909700 1 -- 172.21.15.181:0/565254236 reap_dead start 2023-10-16T10:03:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.677+0000 7fd56236c700 1 -- 172.21.15.181:0/565254236 shutdown_connections 2023-10-16T10:03:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.677+0000 7fd56236c700 1 -- 172.21.15.181:0/565254236 >> 172.21.15.181:0/565254236 conn(0x7fd55c0d04a0 msgr2=0x7fd55c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.677+0000 7fd56236c700 1 -- 172.21.15.181:0/565254236 shutdown_connections 2023-10-16T10:03:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.677+0000 7fd56236c700 1 -- 172.21.15.181:0/565254236 wait complete. 2023-10-16T10:03:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.678+0000 7fd56236c700 1 Processor -- start 2023-10-16T10:03:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.678+0000 7fd56236c700 1 -- start start 2023-10-16T10:03:14.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd56236c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd55c1e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd56236c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55c0aff00 0x7fd55c0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd56236c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0b21c0 0x7fd55c0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd56236c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0b08c0 con 0x7fd544003a00 2023-10-16T10:03:14.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd56236c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0c7cb0 con 0x7fd55c0aff00 2023-10-16T10:03:14.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd56236c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0c7f50 con 0x7fd55c0b21c0 2023-10-16T10:03:14.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd55c1e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd55c1e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47330/0 (socket says 172.21.15.181:47330) 2023-10-16T10:03:14.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 -- 172.21.15.181:0/1151937806 learned_addr learned my addr 172.21.15.181:0/1151937806 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:14.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd560909700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0b21c0 0x7fd55c0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55b7fe700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55c0aff00 0x7fd55c0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 -- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0b21c0 msgr2=0x7fd55c0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0b21c0 0x7fd55c0b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 -- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55c0aff00 msgr2=0x7fd55c0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55c0aff00 0x7fd55c0b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.679+0000 7fd55bfff700 1 -- 172.21.15.181:0/1151937806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0c8230 con 0x7fd544003a00 2023-10-16T10:03:14.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.680+0000 7fd55bfff700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd55c1e7d20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd54400baf0 tx=0x7fd5500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:14.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.680+0000 7fd558ff9700 1 -- 172.21.15.181:0/1151937806 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550006420 con 0x7fd544003a00 2023-10-16T10:03:14.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.680+0000 7fd558ff9700 1 -- 172.21.15.181:0/1151937806 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5500065a0 con 0x7fd544003a00 2023-10-16T10:03:14.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.680+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c0c8510 con 0x7fd544003a00 2023-10-16T10:03:14.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.680+0000 7fd558ff9700 1 -- 172.21.15.181:0/1151937806 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550006950 con 0x7fd544003a00 2023-10-16T10:03:14.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.680+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c1e8260 con 0x7fd544003a00 2023-10-16T10:03:14.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.681+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c067d20 con 0x7fd544003a00 2023-10-16T10:03:14.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.681+0000 7fd558ff9700 1 -- 172.21.15.181:0/1151937806 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd550013430 con 0x7fd544003a00 2023-10-16T10:03:14.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.682+0000 7fd558ff9700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd54806b020 0x7fd54806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:14.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.682+0000 7fd55b7fe700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd54806b020 0x7fd54806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:14.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.682+0000 7fd558ff9700 1 -- 172.21.15.181:0/1151937806 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd55008a170 con 0x7fd544003a00 2023-10-16T10:03:14.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.682+0000 7fd55b7fe700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd54806b020 0x7fd54806d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd54c000c00 tx=0x7fd54c009040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:14.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.686+0000 7fd558ff9700 1 -- 172.21.15.181:0/1151937806 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd550054ec0 con 0x7fd544003a00 2023-10-16T10:03:14.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.879+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd55c0ccf20 con 0x7fd544003a00 2023-10-16T10:03:14.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.879+0000 7fd558ff9700 1 -- 172.21.15.181:0/1151937806 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+14864 (secure 0 0 0) 0x7fd5500584e0 con 0x7fd544003a00 2023-10-16T10:03:14.880 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:14.881 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":124,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:12.526510+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/1841075189":"2023-10-17T10:03:09.510172+0000","172.21.15.181:6835/1841075189":"2023-10-17T10:03:09.510172+0000","172.21.15.181:0/3393593592":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/535816161":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/1900611548":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/3011175383":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/431381719":"2023-10-17T10:03:08.473205+0000","172.21.15.181:6833/716040864":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/2599688587":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/862922257":"2023-10-17T10:03:08.473205+0000","172.21.15.181:6833/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:6832/2093458419":"2023-10-17T10:02:24.201085+0000","172.21.15.181:6832/716040864":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/1068945898":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1396028129":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/3417874540":"2023-10-17T10:03:08.473205+0000","172.21.15.181:0/2917390541":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1347286988":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/1199722576":"2023-10-17T10:02:24.201085+0000","172.21.15.181:0/3331655182":"2023-10-17T10:02:24.201085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:14.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd54806b020 msgr2=0x7fd54806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd54806b020 0x7fd54806d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd54c000c00 tx=0x7fd54c009040 comp rx=0 tx=0).stop 2023-10-16T10:03:14.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 msgr2=0x7fd55c1e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:14.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd55c1e7d20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd54400baf0 tx=0x7fd5500047c0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 shutdown_connections 2023-10-16T10:03:14.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd54806b020 0x7fd54806d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd55c0b21c0 0x7fd55c0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd544003a00 0x7fd55c1e7d20 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 --2- 172.21.15.181:0/1151937806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55c0aff00 0x7fd55c0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:14.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 >> 172.21.15.181:0/1151937806 conn(0x7fd55c0d04a0 msgr2=0x7fd55c15cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:14.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 shutdown_connections 2023-10-16T10:03:14.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:14.882+0000 7fd56236c700 1 -- 172.21.15.181:0/1151937806 wait complete. 2023-10-16T10:03:14.895 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:14.895 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:15.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.018+0000 7f2614b4a700 1 Processor -- start 2023-10-16T10:03:15.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.018+0000 7f2614b4a700 1 -- start start 2023-10-16T10:03:15.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f2614b4a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2610159e10 0x7f26100ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f2614b4a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f261006bef0 0x7f26100caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f2614b4a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261006e4b0 con 0x7f26100cb0e0 2023-10-16T10:03:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f2614b4a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261006ed50 con 0x7f261006bef0 2023-10-16T10:03:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f2614b4a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26100ceea0 con 0x7f2610159e10 2023-10-16T10:03:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f261006bef0 0x7f26100caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f26100cb0e0 0x7f261015db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58420/0 (socket says 172.21.15.181:58420) 2023-10-16T10:03:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f261006bef0 0x7f26100caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47340/0 (socket says 172.21.15.181:47340) 2023-10-16T10:03:15.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260ed9d700 1 -- 172.21.15.181:0/3780229491 learned_addr learned my addr 172.21.15.181:0/3780229491 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:15.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260dd9b700 1 -- 172.21.15.181:0/3780229491 >> v2:172.21.15.190:3300/0 conn(0x7f2610159e10 msgr2=0x7f26100ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:15.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260dd9b700 1 --2- 172.21.15.181:0/3780229491 >> v2:172.21.15.190:3300/0 conn(0x7f2610159e10 0x7f26100ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260dd9b700 1 -- 172.21.15.181:0/3780229491 >> v1:172.21.15.181:6789/0 conn(0x7f26100cb0e0 legacy=0x7f261015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260dd9b700 1 -- 172.21.15.181:0/3780229491 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261006e700 con 0x7f261006bef0 2023-10-16T10:03:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260d59a700 1 -- 172.21.15.181:0/3780229491 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846807776 0 0) 0x7f261006e4b0 con 0x7f26100cb0e0 2023-10-16T10:03:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260dd9b700 1 --2- 172.21.15.181:0/3780229491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f26100caba0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2610067360 tx=0x7f2604009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973c549916bd25e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260d59a700 1 -- 172.21.15.181:0/3780229491 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604011830 con 0x7f261006bef0 2023-10-16T10:03:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260d59a700 1 -- 172.21.15.181:0/3780229491 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2604011990 con 0x7f261006bef0 2023-10-16T10:03:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.019+0000 7f260d59a700 1 -- 172.21.15.181:0/3780229491 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604011d20 con 0x7f261006bef0 2023-10-16T10:03:15.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 -- 172.21.15.181:0/3780229491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 msgr2=0x7f26100caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 --2- 172.21.15.181:0/3780229491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f26100caba0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2610067360 tx=0x7f2604009b20 comp rx=0 tx=0).stop 2023-10-16T10:03:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 -- 172.21.15.181:0/3780229491 shutdown_connections 2023-10-16T10:03:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 --2- 172.21.15.181:0/3780229491 >> v2:172.21.15.190:3300/0 conn(0x7f2610159e10 0x7f26100ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 --2- 172.21.15.181:0/3780229491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f26100caba0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 -- 172.21.15.181:0/3780229491 >> 172.21.15.181:0/3780229491 conn(0x7f26100cc730 msgr2=0x7f26100ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 -- 172.21.15.181:0/3780229491 shutdown_connections 2023-10-16T10:03:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.020+0000 7f2614b4a700 1 -- 172.21.15.181:0/3780229491 wait complete. 2023-10-16T10:03:15.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.021+0000 7f2614b4a700 1 Processor -- start 2023-10-16T10:03:15.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.021+0000 7f2614b4a700 1 -- start start 2023-10-16T10:03:15.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f2614b4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f261015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f2614b4a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26100cb0e0 0x7f261015c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f2614b4a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2610159e10 0x7f26101509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f2614b4a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26100c8840 con 0x7f261006bef0 2023-10-16T10:03:15.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f2614b4a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26100c8b20 con 0x7f26100cb0e0 2023-10-16T10:03:15.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f2614b4a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26100c8e00 con 0x7f2610159e10 2023-10-16T10:03:15.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f260e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f261015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f260e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f261015bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47344/0 (socket says 172.21.15.181:47344) 2023-10-16T10:03:15.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f260e59c700 1 -- 172.21.15.181:0/2815079975 learned_addr learned my addr 172.21.15.181:0/2815079975 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:15.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f260ed9d700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2610159e10 0x7f26101509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.022+0000 7f260dd9b700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26100cb0e0 0x7f261015c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f260e59c700 1 -- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2610159e10 msgr2=0x7f26101509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f260e59c700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2610159e10 0x7f26101509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f260e59c700 1 -- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26100cb0e0 msgr2=0x7f261015c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f260e59c700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26100cb0e0 0x7f261015c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f260e59c700 1 -- 172.21.15.181:0/2815079975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040094a0 con 0x7f261006bef0 2023-10-16T10:03:15.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f260e59c700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f261015bb90 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f260000b4a0 tx=0x7f260000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:15.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f25feffd700 1 -- 172.21.15.181:0/2815079975 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600011830 con 0x7f261006bef0 2023-10-16T10:03:15.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f25feffd700 1 -- 172.21.15.181:0/2815079975 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600011990 con 0x7f261006bef0 2023-10-16T10:03:15.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26100c90e0 con 0x7f261006bef0 2023-10-16T10:03:15.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f25feffd700 1 -- 172.21.15.181:0/2815079975 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600011d40 con 0x7f261006bef0 2023-10-16T10:03:15.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.023+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26100c98c0 con 0x7f261006bef0 2023-10-16T10:03:15.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.024+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26100c9a80 con 0x7f261006bef0 2023-10-16T10:03:15.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.024+0000 7f25feffd700 1 -- 172.21.15.181:0/2815079975 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f260001a400 con 0x7f261006bef0 2023-10-16T10:03:15.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.025+0000 7f25feffd700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f25f006b020 0x7f25f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.025+0000 7f25feffd700 1 -- 172.21.15.181:0/2815079975 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f2600089900 con 0x7f261006bef0 2023-10-16T10:03:15.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.025+0000 7f260dd9b700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f25f006b020 0x7f25f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.025+0000 7f260dd9b700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f25f006b020 0x7f25f006d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f261006c300 tx=0x7f2604008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:15.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.029+0000 7f25feffd700 1 -- 172.21.15.181:0/2815079975 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f26000546e0 con 0x7f261006bef0 2023-10-16T10:03:15.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.223+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26100c9730 con 0x7f261006bef0 2023-10-16T10:03:15.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.223+0000 7f25feffd700 1 -- 172.21.15.181:0/2815079975 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26000150a0 con 0x7f261006bef0 2023-10-16T10:03:15.224 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:15.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f25f006b020 msgr2=0x7f25f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f25f006b020 0x7f25f006d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f261006c300 tx=0x7f2604008030 comp rx=0 tx=0).stop 2023-10-16T10:03:15.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 msgr2=0x7f261015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f261015bb90 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f260000b4a0 tx=0x7f260000b860 comp rx=0 tx=0).stop 2023-10-16T10:03:15.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 shutdown_connections 2023-10-16T10:03:15.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f25f006b020 0x7f25f006d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2610159e10 0x7f26101509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f261006bef0 0x7f261015bb90 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 --2- 172.21.15.181:0/2815079975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26100cb0e0 0x7f261015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.225+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 >> 172.21.15.181:0/2815079975 conn(0x7f26100cc730 msgr2=0x7f2610155450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:15.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.226+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 shutdown_connections 2023-10-16T10:03:15.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.226+0000 7f2614b4a700 1 -- 172.21.15.181:0/2815079975 wait complete. 2023-10-16T10:03:15.238 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-16T10:03:15.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.359+0000 7f24a1b7b700 1 Processor -- start 2023-10-16T10:03:15.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.359+0000 7f24a1b7b700 1 -- start start 2023-10-16T10:03:15.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f24a1b7b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f249c0d0370 0x7f249c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f24a1b7b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f249c0cd2d0 0x7f249c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f24a1b7b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c0cc510 con 0x7f249c15b990 2023-10-16T10:03:15.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f24a1b7b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c0d24f0 con 0x7f249c0d0370 2023-10-16T10:03:15.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f24a1b7b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c0cc7f0 con 0x7f249c0cd2d0 2023-10-16T10:03:15.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f249c0d0370 0x7f249c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f249c15b990 0x7f249c0cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58430/0 (socket says 172.21.15.181:58430) 2023-10-16T10:03:15.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249b7fe700 1 -- 172.21.15.181:0/4001092954 learned_addr learned my addr 172.21.15.181:0/4001092954 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:15.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249bfff700 1 --2- 172.21.15.181:0/4001092954 >> v2:172.21.15.190:3300/0 conn(0x7f249c0cd2d0 0x7f249c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249affd700 1 -- 172.21.15.181:0/4001092954 >> v2:172.21.15.190:3300/0 conn(0x7f249c0cd2d0 msgr2=0x7f249c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:15.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249affd700 1 --2- 172.21.15.181:0/4001092954 >> v2:172.21.15.190:3300/0 conn(0x7f249c0cd2d0 0x7f249c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249affd700 1 -- 172.21.15.181:0/4001092954 >> v1:172.21.15.181:6789/0 conn(0x7f249c15b990 legacy=0x7f249c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249affd700 1 -- 172.21.15.181:0/4001092954 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c0ccad0 con 0x7f249c0d0370 2023-10-16T10:03:15.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249a7fc700 1 -- 172.21.15.181:0/4001092954 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834988795 0 0) 0x7f249c0cc510 con 0x7f249c15b990 2023-10-16T10:03:15.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249bfff700 1 --2- 172.21.15.181:0/4001092954 >> v2:172.21.15.190:3300/0 conn(0x7f249c0cd2d0 0x7f249c0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:15.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249affd700 1 --2- 172.21.15.181:0/4001092954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c0d0370 0x7f249c0ce8f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f249c0cbc60 tx=0x7f248c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c01174dc7650ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:15.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249a7fc700 1 -- 172.21.15.181:0/4001092954 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c011830 con 0x7f249c0d0370 2023-10-16T10:03:15.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249a7fc700 1 -- 172.21.15.181:0/4001092954 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c011990 con 0x7f249c0d0370 2023-10-16T10:03:15.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.360+0000 7f249a7fc700 1 -- 172.21.15.181:0/4001092954 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c011d20 con 0x7f249c0d0370 2023-10-16T10:03:15.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 -- 172.21.15.181:0/4001092954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c0d0370 msgr2=0x7f249c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/4001092954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c0d0370 0x7f249c0ce8f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f249c0cbc60 tx=0x7f248c009b20 comp rx=0 tx=0).stop 2023-10-16T10:03:15.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 -- 172.21.15.181:0/4001092954 shutdown_connections 2023-10-16T10:03:15.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/4001092954 >> v2:172.21.15.190:3300/0 conn(0x7f249c0cd2d0 0x7f249c0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/4001092954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c0d0370 0x7f249c0ce8f0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 -- 172.21.15.181:0/4001092954 >> 172.21.15.181:0/4001092954 conn(0x7f249c09a9a0 msgr2=0x7f249c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:15.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 -- 172.21.15.181:0/4001092954 shutdown_connections 2023-10-16T10:03:15.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.361+0000 7f24a1b7b700 1 -- 172.21.15.181:0/4001092954 wait complete. 2023-10-16T10:03:15.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.362+0000 7f24a1b7b700 1 Processor -- start 2023-10-16T10:03:15.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.362+0000 7f24a1b7b700 1 -- start start 2023-10-16T10:03:15.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f24a1b7b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f249c0cd2d0 0x7f249c158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f24a1b7b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f249c0d0370 0x7f249c0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f24a1b7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c15b990 0x7f249c15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f24a1b7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0b5170 con 0x7f249c15b990 2023-10-16T10:03:15.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f24a1b7b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0b42a0 con 0x7f249c0d0370 2023-10-16T10:03:15.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f24a1b7b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0b44f0 con 0x7f249c0cd2d0 2023-10-16T10:03:15.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f249bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c15b990 0x7f249c15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f249bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c15b990 0x7f249c15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47362/0 (socket says 172.21.15.181:47362) 2023-10-16T10:03:15.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f249bfff700 1 -- 172.21.15.181:0/1276903832 learned_addr learned my addr 172.21.15.181:0/1276903832 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:15.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f249b7fe700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f249c0cd2d0 0x7f249c158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.363+0000 7f249affd700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f249c0d0370 0x7f249c0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f249bfff700 1 -- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f249c0cd2d0 msgr2=0x7f249c158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f249bfff700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f249c0cd2d0 0x7f249c158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f249bfff700 1 -- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f249c0d0370 msgr2=0x7f249c0c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:15.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f249bfff700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f249c0d0370 0x7f249c0c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:15.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f249bfff700 1 -- 172.21.15.181:0/1276903832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0094a0 con 0x7f249c15b990 2023-10-16T10:03:15.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f249b7fe700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f249c0cd2d0 0x7f249c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:15.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f249bfff700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c15b990 0x7f249c15d600 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f2494007e70 tx=0x7f249400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:15.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f2483fff700 1 -- 172.21.15.181:0/1276903832 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400de40 con 0x7f249c15b990 2023-10-16T10:03:15.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f2483fff700 1 -- 172.21.15.181:0/1276903832 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249400f040 con 0x7f249c15b990 2023-10-16T10:03:15.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c0b47d0 con 0x7f249c15b990 2023-10-16T10:03:15.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f2483fff700 1 -- 172.21.15.181:0/1276903832 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494003c60 con 0x7f249c15b990 2023-10-16T10:03:15.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.364+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c0b4d90 con 0x7f249c15b990 2023-10-16T10:03:15.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.365+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c067d20 con 0x7f249c15b990 2023-10-16T10:03:15.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.365+0000 7f2483fff700 1 -- 172.21.15.181:0/1276903832 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2494004030 con 0x7f249c15b990 2023-10-16T10:03:15.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.366+0000 7f2483fff700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f248806b050 0x7f248806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:15.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.366+0000 7f249b7fe700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f248806b050 0x7f248806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:15.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.366+0000 7f2483fff700 1 -- 172.21.15.181:0/1276903832 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f2494089ac0 con 0x7f249c15b990 2023-10-16T10:03:15.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.366+0000 7f249b7fe700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f248806b050 0x7f248806d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2490007fd0 tx=0x7f2490009040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:15.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.370+0000 7f2483fff700 1 -- 172.21.15.181:0/1276903832 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2494054810 con 0x7f249c15b990 2023-10-16T10:03:15.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:15.640+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7f249c0cfa40 con 0x7f249c15b990 2023-10-16T10:03:16.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.537+0000 7f2483fff700 1 -- 172.21.15.181:0/1276903832 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v724) v1 ==== 163+0+0 (secure 0 0 0) 0x7f249401a0e0 con 0x7f249c15b990 2023-10-16T10:03:16.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f248806b050 msgr2=0x7f248806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f248806b050 0x7f248806d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2490007fd0 tx=0x7f2490009040 comp rx=0 tx=0).stop 2023-10-16T10:03:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c15b990 msgr2=0x7f249c15d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c15b990 0x7f249c15d600 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f2494007e70 tx=0x7f249400d3b0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 shutdown_connections 2023-10-16T10:03:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f248806b050 0x7f248806d510 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f249c0cd2d0 0x7f249c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f249c15b990 0x7f249c15d600 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 --2- 172.21.15.181:0/1276903832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f249c0d0370 0x7f249c0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 >> 172.21.15.181:0/1276903832 conn(0x7f249c09a9a0 msgr2=0x7f249c0d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:16.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 shutdown_connections 2023-10-16T10:03:16.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.540+0000 7f24a1b7b700 1 -- 172.21.15.181:0/1276903832 wait complete. 2023-10-16T10:03:16.556 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:03:16.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.684+0000 7f6388a7c700 1 Processor -- start 2023-10-16T10:03:16.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.684+0000 7f6388a7c700 1 -- start start 2023-10-16T10:03:16.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.685+0000 7f6388a7c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 0x7f63840ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.685+0000 7f6388a7c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63840cef00 con 0x7f638406afb0 2023-10-16T10:03:16.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.685+0000 7f6388a7c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63840cec20 con 0x7f63840ce6a0 2023-10-16T10:03:16.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.685+0000 7f6388a7c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6384071c80 con 0x7f638415da70 2023-10-16T10:03:16.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.685+0000 7f6382d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f638406afb0 0x7f63841615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58446/0 (socket says 172.21.15.181:58446) 2023-10-16T10:03:16.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.685+0000 7f6382d9d700 1 -- 172.21.15.181:0/2579124294 learned_addr learned my addr 172.21.15.181:0/2579124294 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:16.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136995124 0 0) 0x7f63840cef00 con 0x7f638406afb0 2023-10-16T10:03:16.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 0x7f63840ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f636c0035b0 con 0x7f638406afb0 2023-10-16T10:03:16.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2440857834 0 0) 0x7f636c0035b0 con 0x7f638406afb0 2023-10-16T10:03:16.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 msgr2=0x7f63840ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:16.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 0x7f63840ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 >> v1:172.21.15.190:6789/0 conn(0x7f63840ce6a0 legacy=0x7f638406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384071f60 con 0x7f638406afb0 2023-10-16T10:03:16.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f637c0019f0 con 0x7f638406afb0 2023-10-16T10:03:16.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 >> v1:172.21.15.181:6789/0 conn(0x7f638406afb0 legacy=0x7f63841615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f636c003a30 0x7f636c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 0x7f636c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636c004d90 0x7f636c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6384071c80 con 0x7f636c003a30 2023-10-16T10:03:16.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c00bbd0 con 0x7f636c004d90 2023-10-16T10:03:16.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c00bd90 con 0x7f638415da70 2023-10-16T10:03:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f636c003a30 0x7f636c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f6381d9b700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 0x7f636c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f6382d9d700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636c004d90 0x7f636c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 -- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 msgr2=0x7f636c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:16.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 0x7f636c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 -- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636c004d90 msgr2=0x7f636c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f636c004d90 0x7f636c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 -- 172.21.15.181:0/2579124294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c0035b0 con 0x7f636c003a30 2023-10-16T10:03:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f6381d9b700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f638415da70 0x7f636c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:16.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.686+0000 7f638259c700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f636c003a30 0x7f636c003e50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6378009b60 tx=0x7f636c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b2f712d22a88b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637800e6c0 con 0x7f636c003a30 2023-10-16T10:03:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637800e820 con 0x7f636c003a30 2023-10-16T10:03:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f638159a700 1 -- 172.21.15.181:0/2579124294 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637800eb90 con 0x7f636c003a30 2023-10-16T10:03:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f6388a7c700 1 -- 172.21.15.181:0/2579124294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f636c003a30 msgr2=0x7f636c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f6388a7c700 1 --2- 172.21.15.181:0/2579124294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f636c003a30 0x7f636c003e50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6378009b60 tx=0x7f636c005880 comp rx=0 tx=0).stop 2023-10-16T10:03:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f6382d9d700 1 -- 172.21.15.181:0/2579124294 reap_dead start 2023-10-16T10:03:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f6388a7c700 1 -- 172.21.15.181:0/2579124294 shutdown_connections 2023-10-16T10:03:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f6388a7c700 1 -- 172.21.15.181:0/2579124294 >> 172.21.15.181:0/2579124294 conn(0x7f63840cd010 msgr2=0x7f63840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f6388a7c700 1 -- 172.21.15.181:0/2579124294 shutdown_connections 2023-10-16T10:03:16.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.687+0000 7f6388a7c700 1 -- 172.21.15.181:0/2579124294 wait complete. 2023-10-16T10:03:16.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 Processor -- start 2023-10-16T10:03:16.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 -- start start 2023-10-16T10:03:16.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f636c003a30 0x7f63840c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638415e9e0 0x7f638415ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840b4150 0x7f63840b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638415f3a0 con 0x7f63840b4150 2023-10-16T10:03:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840c8410 con 0x7f638415e9e0 2023-10-16T10:03:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6388a7c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63840c7800 con 0x7f636c003a30 2023-10-16T10:03:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6382d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840b4150 0x7f63840b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6382d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840b4150 0x7f63840b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47382/0 (socket says 172.21.15.181:47382) 2023-10-16T10:03:16.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6382d9d700 1 -- 172.21.15.181:0/1894277373 learned_addr learned my addr 172.21.15.181:0/1894277373 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:16.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f638259c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f636c003a30 0x7f63840c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.689+0000 7f6381d9b700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638415e9e0 0x7f638415ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6382d9d700 1 -- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f636c003a30 msgr2=0x7f63840c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6382d9d700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f636c003a30 0x7f63840c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6382d9d700 1 -- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638415e9e0 msgr2=0x7f638415ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6382d9d700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638415e9e0 0x7f638415ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6382d9d700 1 -- 172.21.15.181:0/1894277373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63840c7ae0 con 0x7f63840b4150 2023-10-16T10:03:16.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6382d9d700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840b4150 0x7f63840b45d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f638406f690 tx=0x7f637c007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:16.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6372ffd700 1 -- 172.21.15.181:0/1894277373 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c00ea90 con 0x7f63840b4150 2023-10-16T10:03:16.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6372ffd700 1 -- 172.21.15.181:0/1894277373 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c00ebf0 con 0x7f63840b4150 2023-10-16T10:03:16.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63840c7dc0 con 0x7f63840b4150 2023-10-16T10:03:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6372ffd700 1 -- 172.21.15.181:0/1894277373 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c013420 con 0x7f63840b4150 2023-10-16T10:03:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.690+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63840b4b10 con 0x7f63840b4150 2023-10-16T10:03:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.691+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63840c8550 con 0x7f63840b4150 2023-10-16T10:03:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.691+0000 7f6372ffd700 1 -- 172.21.15.181:0/1894277373 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f637c013580 con 0x7f63840b4150 2023-10-16T10:03:16.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.692+0000 7f6372ffd700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f636406b020 0x7f636406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.692+0000 7f638259c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f636406b020 0x7f636406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.692+0000 7f6372ffd700 1 -- 172.21.15.181:0/1894277373 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f637c089e00 con 0x7f63840b4150 2023-10-16T10:03:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.692+0000 7f638259c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f636406b020 0x7f636406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f63780090b0 tx=0x7f637801f040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:16.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.696+0000 7f6372ffd700 1 -- 172.21.15.181:0/1894277373 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f637c058660 con 0x7f63840b4150 2023-10-16T10:03:16.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.894+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f638406f6c0 con 0x7f63840b4150 2023-10-16T10:03:16.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.897+0000 7f6372ffd700 1 -- 172.21.15.181:0/1894277373 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f637c0581f0 con 0x7f63840b4150 2023-10-16T10:03:16.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f636406b020 msgr2=0x7f636406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f636406b020 0x7f636406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f63780090b0 tx=0x7f637801f040 comp rx=0 tx=0).stop 2023-10-16T10:03:16.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840b4150 msgr2=0x7f63840b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:16.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840b4150 0x7f63840b45d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f638406f690 tx=0x7f637c007c90 comp rx=0 tx=0).stop 2023-10-16T10:03:16.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 shutdown_connections 2023-10-16T10:03:16.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f636406b020 0x7f636406d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f636c003a30 0x7f63840c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840b4150 0x7f63840b45d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 --2- 172.21.15.181:0/1894277373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f638415e9e0 0x7f638415ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 >> 172.21.15.181:0/1894277373 conn(0x7f63840cd010 msgr2=0x7f638406f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 shutdown_connections 2023-10-16T10:03:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:16.899+0000 7f6388a7c700 1 -- 172.21.15.181:0/1894277373 wait complete. 2023-10-16T10:03:16.912 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:03:16.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:16.950 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:17.000 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:03:17.000 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:17.001 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:17.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:03:17.019 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:17.019 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:17.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:17.040 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:17.041 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:17.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:03:17.076 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:17.076 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:17.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:17.100 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:03:17.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:17.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.235+0000 7f087cb6c700 1 Processor -- start 2023-10-16T10:03:17.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.235+0000 7f087cb6c700 1 -- start start 2023-10-16T10:03:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087cb6c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f08780cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087cb6c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f087806e010 0x7f087806e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087cb6c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0878066c90 con 0x7f08780b22a0 2023-10-16T10:03:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087cb6c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087806e9a0 con 0x7f087806e010 2023-10-16T10:03:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087cb6c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0878066740 con 0x7f087815b8b0 2023-10-16T10:03:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f0875d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f08780b22a0 0x7f087815feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58464/0 (socket says 172.21.15.181:58464) 2023-10-16T10:03:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f0875d9b700 1 -- 172.21.15.181:0/1629729106 learned_addr learned my addr 172.21.15.181:0/1629729106 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035738667 0 0) 0x7f0868001110 con 0x7f08780b22a0 2023-10-16T10:03:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f0876d9d700 1 --2- 172.21.15.181:0/1629729106 >> v2:172.21.15.190:3300/0 conn(0x7f087806e010 0x7f087806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087659c700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f08780cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.236+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08600035b0 con 0x7f08780b22a0 2023-10-16T10:03:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2205550753 0 0) 0x7f08600035b0 con 0x7f08780b22a0 2023-10-16T10:03:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 msgr2=0x7f08780cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f08780cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 >> v2:172.21.15.190:3300/0 conn(0x7f087806e010 msgr2=0x7f087806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:17.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 --2- 172.21.15.181:0/1629729106 >> v2:172.21.15.190:3300/0 conn(0x7f087806e010 0x7f087806e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780669b0 con 0x7f08780b22a0 2023-10-16T10:03:17.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f0868001800 con 0x7f08780b22a0 2023-10-16T10:03:17.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 >> v1:172.21.15.181:6789/0 conn(0x7f08780b22a0 legacy=0x7f087815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f0860003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0860003f80 0x7f0860004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0860004d90 0x7f0860005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087806e9a0 con 0x7f0860004d90 2023-10-16T10:03:17.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0878066740 con 0x7f0860003f80 2023-10-16T10:03:17.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086000ba10 con 0x7f087815b8b0 2023-10-16T10:03:17.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f0876d9d700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0860004d90 0x7f0860005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087659c700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f0860003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f0875d9b700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0860003f80 0x7f0860004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087659c700 1 -- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0860003f80 msgr2=0x7f0860004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:17.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087659c700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0860003f80 0x7f0860004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087659c700 1 -- 172.21.15.181:0/1629729106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0860004d90 msgr2=0x7f0860005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:17.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087659c700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0860004d90 0x7f0860005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.237+0000 7f087659c700 1 -- 172.21.15.181:0/1629729106 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08600035b0 con 0x7f087815b8b0 2023-10-16T10:03:17.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087659c700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f0860003a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f086c009b60 tx=0x7f0860005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e701337a63b40e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:17.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00e6c0 con 0x7f087815b8b0 2023-10-16T10:03:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c00e820 con 0x7f087815b8b0 2023-10-16T10:03:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087559a700 1 -- 172.21.15.181:0/1629729106 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00eb90 con 0x7f087815b8b0 2023-10-16T10:03:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087cb6c700 1 -- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 msgr2=0x7f0860003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087cb6c700 1 --2- 172.21.15.181:0/1629729106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f0860003a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f086c009b60 tx=0x7f0860005880 comp rx=0 tx=0).stop 2023-10-16T10:03:17.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f0876d9d700 1 -- 172.21.15.181:0/1629729106 reap_dead start 2023-10-16T10:03:17.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087cb6c700 1 -- 172.21.15.181:0/1629729106 shutdown_connections 2023-10-16T10:03:17.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087cb6c700 1 -- 172.21.15.181:0/1629729106 >> 172.21.15.181:0/1629729106 conn(0x7f08780b1780 msgr2=0x7f08780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:17.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087cb6c700 1 -- 172.21.15.181:0/1629729106 shutdown_connections 2023-10-16T10:03:17.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.238+0000 7f087cb6c700 1 -- 172.21.15.181:0/1629729106 wait complete. 2023-10-16T10:03:17.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.240+0000 7f087cb6c700 1 Processor -- start 2023-10-16T10:03:17.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.240+0000 7f087cb6c700 1 -- start start 2023-10-16T10:03:17.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f087cb6c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f087815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f087cb6c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f087815ec10 0x7f087815f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f087cb6c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780b48d0 0x7f08780b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f087cb6c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780b5290 con 0x7f087815ec10 2023-10-16T10:03:17.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f087cb6c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08781f1d70 con 0x7f08780b48d0 2023-10-16T10:03:17.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f087cb6c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08781f1160 con 0x7f087815b8b0 2023-10-16T10:03:17.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f0875d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f087815ec10 0x7f087815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f0875d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f087815ec10 0x7f087815f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47400/0 (socket says 172.21.15.181:47400) 2023-10-16T10:03:17.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f0875d9b700 1 -- 172.21.15.181:0/1164614177 learned_addr learned my addr 172.21.15.181:0/1164614177 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:17.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f087659c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f087815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.241+0000 7f0876d9d700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780b48d0 0x7f08780b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0875d9b700 1 -- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 msgr2=0x7f087815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0875d9b700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f087815aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0875d9b700 1 -- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780b48d0 msgr2=0x7f08780b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0875d9b700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780b48d0 0x7f08780b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0875d9b700 1 -- 172.21.15.181:0/1164614177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08781f1440 con 0x7f087815ec10 2023-10-16T10:03:17.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0876d9d700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780b48d0 0x7f08780b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:17.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f087659c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f087815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:17.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0875d9b700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f087815ec10 0x7f087815f090 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0868007310 tx=0x7f0868007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:17.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0866ffd700 1 -- 172.21.15.181:0/1164614177 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800f660 con 0x7f087815ec10 2023-10-16T10:03:17.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0866ffd700 1 -- 172.21.15.181:0/1164614177 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086800f7e0 con 0x7f087815ec10 2023-10-16T10:03:17.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08781f1720 con 0x7f087815ec10 2023-10-16T10:03:17.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f0866ffd700 1 -- 172.21.15.181:0/1164614177 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800fb90 con 0x7f087815ec10 2023-10-16T10:03:17.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.242+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087815b010 con 0x7f087815ec10 2023-10-16T10:03:17.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.243+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0878067d20 con 0x7f087815ec10 2023-10-16T10:03:17.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.243+0000 7f0866ffd700 1 -- 172.21.15.181:0/1164614177 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f086800fcf0 con 0x7f087815ec10 2023-10-16T10:03:17.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.244+0000 7f0866ffd700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f085806b020 0x7f085806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.244+0000 7f0866ffd700 1 -- 172.21.15.181:0/1164614177 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f08680897f0 con 0x7f087815ec10 2023-10-16T10:03:17.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.244+0000 7f087659c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f085806b020 0x7f085806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.244+0000 7f087659c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f085806b020 0x7f085806d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f086c0090b0 tx=0x7f086c008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:17.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.248+0000 7f0866ffd700 1 -- 172.21.15.181:0/1164614177 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0868054540 con 0x7f087815ec10 2023-10-16T10:03:17.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.483+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0878066c90 con 0x7f087815ec10 2023-10-16T10:03:17.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.484+0000 7f0866ffd700 1 -- 172.21.15.181:0/1164614177 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f0878066c90 con 0x7f087815ec10 2023-10-16T10:03:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:17.485 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2945191097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2945191097},{"type":"v1","addr":"172.21.15.190:6834","nonce":2945191097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1168488779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1168488779},{"type":"v1","addr":"172.21.15.190:6835","nonce":1168488779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2677039975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2677039975},{"type":"v1","addr":"172.21.15.181:6836","nonce":2677039975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1408985574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1408985574},{"type":"v1","addr":"172.21.15.181:6837","nonce":1408985574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2023-10-16T10:03:17.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f085806b020 msgr2=0x7f085806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f085806b020 0x7f085806d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f086c0090b0 tx=0x7f086c008030 comp rx=0 tx=0).stop 2023-10-16T10:03:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f087815ec10 msgr2=0x7f087815f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f087815ec10 0x7f087815f090 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0868007310 tx=0x7f0868007620 comp rx=0 tx=0).stop 2023-10-16T10:03:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 shutdown_connections 2023-10-16T10:03:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f085806b020 0x7f085806d4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f087815b8b0 0x7f087815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08780b48d0 0x7f08780b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 --2- 172.21.15.181:0/1164614177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f087815ec10 0x7f087815f090 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 >> 172.21.15.181:0/1164614177 conn(0x7f08780b1780 msgr2=0x7f08780c6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 shutdown_connections 2023-10-16T10:03:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.486+0000 7f087cb6c700 1 -- 172.21.15.181:0/1164614177 wait complete. 2023-10-16T10:03:17.490 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 60 2023-10-16T10:03:17.499 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:03:17.500 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:03:17.500 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:03:17.500 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:17.501 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:03:17.501 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:03:17.501 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:03:17.501 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:17.502 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:03:17.502 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:03:17.502 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:03:17.503 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:17.503 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:03:17.503 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:03:17.504 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:03:17.504 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:17.509 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:03:17.509 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:03:17.510 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:03:17.511 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:03:17.513 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:03:17.514 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:03:17.515 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:03:17.515 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:03:17.552 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:03:17.553 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:17.553 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:03:17.554 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:17.556 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:03:17.557 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:17.557 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:03:17.558 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:17.558 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:03:17.632 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T10:03:17.632 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T10:03:17.644 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T10:03:17.644 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T10:03:17.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.684+0000 7f2656567700 1 Processor -- start 2023-10-16T10:03:17.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.684+0000 7f2656567700 1 -- start start 2023-10-16T10:03:17.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f2656567700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015e460 0x7f26500d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f2656567700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26500cb110 0x7f265015e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f2656567700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f265015a0f0 0x7f265015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f2656567700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265006ed00 con 0x7f26500cb110 2023-10-16T10:03:17.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f2656567700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265015ed80 con 0x7f265015a0f0 2023-10-16T10:03:17.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f2656567700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265006e0f0 con 0x7f265015e460 2023-10-16T10:03:17.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f264f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26500cb110 0x7f265015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f264f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26500cb110 0x7f265015e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47432/0 (socket says 172.21.15.181:47432) 2023-10-16T10:03:17.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.685+0000 7f264f7fe700 1 -- 172.21.15.181:0/3212762418 learned_addr learned my addr 172.21.15.181:0/3212762418 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:17.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f2654b04700 1 --2- 172.21.15.181:0/3212762418 >> v2:172.21.15.190:3300/0 conn(0x7f265015a0f0 0x7f265015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264ffff700 1 --2- 172.21.15.181:0/3212762418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015e460 0x7f26500d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264f7fe700 1 -- 172.21.15.181:0/3212762418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015e460 msgr2=0x7f26500d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:17.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264f7fe700 1 --2- 172.21.15.181:0/3212762418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015e460 0x7f26500d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264f7fe700 1 -- 172.21.15.181:0/3212762418 >> v2:172.21.15.190:3300/0 conn(0x7f265015a0f0 msgr2=0x7f265015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:17.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264f7fe700 1 --2- 172.21.15.181:0/3212762418 >> v2:172.21.15.190:3300/0 conn(0x7f265015a0f0 0x7f265015a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264f7fe700 1 -- 172.21.15.181:0/3212762418 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265006e3d0 con 0x7f26500cb110 2023-10-16T10:03:17.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f2654b04700 1 --2- 172.21.15.181:0/3212762418 >> v2:172.21.15.190:3300/0 conn(0x7f265015a0f0 0x7f265015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:17.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264f7fe700 1 --2- 172.21.15.181:0/3212762418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f265015e840 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2650071b00 tx=0x7f263800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bed88a2ff98083 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:17.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264effd700 1 -- 172.21.15.181:0/3212762418 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263800dd30 con 0x7f26500cb110 2023-10-16T10:03:17.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264effd700 1 -- 172.21.15.181:0/3212762418 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263800de90 con 0x7f26500cb110 2023-10-16T10:03:17.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f264effd700 1 -- 172.21.15.181:0/3212762418 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638010690 con 0x7f26500cb110 2023-10-16T10:03:17.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f2656567700 1 -- 172.21.15.181:0/3212762418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 msgr2=0x7f265015e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.686+0000 7f2656567700 1 --2- 172.21.15.181:0/3212762418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f265015e840 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f2650071b00 tx=0x7f263800be70 comp rx=0 tx=0).stop 2023-10-16T10:03:17.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.687+0000 7f2656567700 1 -- 172.21.15.181:0/3212762418 shutdown_connections 2023-10-16T10:03:17.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.687+0000 7f2656567700 1 --2- 172.21.15.181:0/3212762418 >> v2:172.21.15.190:3300/0 conn(0x7f265015a0f0 0x7f265015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.687+0000 7f2656567700 1 --2- 172.21.15.181:0/3212762418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015e460 0x7f26500d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.687+0000 7f2656567700 1 --2- 172.21.15.181:0/3212762418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f265015e840 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.687+0000 7f2656567700 1 -- 172.21.15.181:0/3212762418 >> 172.21.15.181:0/3212762418 conn(0x7f26500d04a0 msgr2=0x7f26500d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:17.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.687+0000 7f2656567700 1 -- 172.21.15.181:0/3212762418 shutdown_connections 2023-10-16T10:03:17.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.687+0000 7f2656567700 1 -- 172.21.15.181:0/3212762418 wait complete. 2023-10-16T10:03:17.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.688+0000 7f2656567700 1 Processor -- start 2023-10-16T10:03:17.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.688+0000 7f2656567700 1 -- start start 2023-10-16T10:03:17.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f2656567700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f26500aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f2656567700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015a0f0 0x7f26500af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f2656567700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f265015e460 0x7f2650152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f2656567700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26501efc50 con 0x7f26500cb110 2023-10-16T10:03:17.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f2656567700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26501ef970 con 0x7f265015e460 2023-10-16T10:03:17.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f2656567700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26501eff80 con 0x7f265015a0f0 2023-10-16T10:03:17.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f264ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f26500aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f264ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f26500aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47438/0 (socket says 172.21.15.181:47438) 2023-10-16T10:03:17.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f264ffff700 1 -- 172.21.15.181:0/2075677658 learned_addr learned my addr 172.21.15.181:0/2075677658 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:17.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f2654b04700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f265015e460 0x7f2650152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.689+0000 7f264f7fe700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015a0f0 0x7f26500af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264ffff700 1 -- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015a0f0 msgr2=0x7f26500af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264ffff700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015a0f0 0x7f26500af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264ffff700 1 -- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f265015e460 msgr2=0x7f2650152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:17.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264ffff700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f265015e460 0x7f2650152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:17.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264ffff700 1 -- 172.21.15.181:0/2075677658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263800b7f0 con 0x7f26500cb110 2023-10-16T10:03:17.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f2654b04700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f265015e460 0x7f2650152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:17.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264ffff700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f26500aebf0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f264000e730 tx=0x7f264000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:17.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264cff9700 1 -- 172.21.15.181:0/2075677658 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640013610 con 0x7f26500cb110 2023-10-16T10:03:17.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264cff9700 1 -- 172.21.15.181:0/2075677658 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2640013770 con 0x7f26500cb110 2023-10-16T10:03:17.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26501f0260 con 0x7f26500cb110 2023-10-16T10:03:17.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f264cff9700 1 -- 172.21.15.181:0/2075677658 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640013b20 con 0x7f26500cb110 2023-10-16T10:03:17.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.690+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26501525b0 con 0x7f26500cb110 2023-10-16T10:03:17.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.691+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26501efde0 con 0x7f26500cb110 2023-10-16T10:03:17.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.692+0000 7f264cff9700 1 -- 172.21.15.181:0/2075677658 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2640013c80 con 0x7f26500cb110 2023-10-16T10:03:17.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.692+0000 7f264cff9700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f263006b020 0x7f263006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:17.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.692+0000 7f264cff9700 1 -- 172.21.15.181:0/2075677658 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f2640089330 con 0x7f26500cb110 2023-10-16T10:03:17.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.692+0000 7f264f7fe700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f263006b020 0x7f263006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:17.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.692+0000 7f264f7fe700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f263006b020 0x7f263006d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f263800beb0 tx=0x7f2638012000 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:17.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.696+0000 7f264cff9700 1 -- 172.21.15.181:0/2075677658 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f26501efde0 con 0x7f26500cb110 2023-10-16T10:03:17.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:17.907+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f26501f08b0 con 0x7f26500cb110 2023-10-16T10:03:18.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.549+0000 7f264cff9700 1 -- 172.21.15.181:0/2075677658 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2640054000 con 0x7f26500cb110 2023-10-16T10:03:18.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f263006b020 msgr2=0x7f263006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:18.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f263006b020 0x7f263006d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f263800beb0 tx=0x7f2638012000 comp rx=0 tx=0).stop 2023-10-16T10:03:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 msgr2=0x7f26500aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f26500aebf0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f264000e730 tx=0x7f264000ea40 comp rx=0 tx=0).stop 2023-10-16T10:03:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 shutdown_connections 2023-10-16T10:03:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f265015e460 0x7f2650152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500cb110 0x7f26500aebf0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f263006b020 0x7f263006d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 --2- 172.21.15.181:0/2075677658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f265015a0f0 0x7f26500af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 >> 172.21.15.181:0/2075677658 conn(0x7f26500d04a0 msgr2=0x7f26500adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 shutdown_connections 2023-10-16T10:03:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.553+0000 7f2656567700 1 -- 172.21.15.181:0/2075677658 wait complete. 2023-10-16T10:03:18.557 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T10:03:18.570 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:03:18.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.694+0000 7fda613eb700 1 Processor -- start 2023-10-16T10:03:18.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.694+0000 7fda613eb700 1 -- start start 2023-10-16T10:03:18.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.695+0000 7fda613eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0d0370 0x7fda5c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:18.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.695+0000 7fda613eb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda5c0cd2d0 0x7fda5c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:18.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.695+0000 7fda613eb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c0cc510 con 0x7fda5c15b990 2023-10-16T10:03:18.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.695+0000 7fda613eb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0d24f0 con 0x7fda5c0cd2d0 2023-10-16T10:03:18.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.695+0000 7fda613eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0cc7f0 con 0x7fda5c0d0370 2023-10-16T10:03:18.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda5c0cd2d0 0x7fda5c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:18.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5a7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0d0370 0x7fda5c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:18.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5affd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fda5c15b990 0x7fda5c0cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:42468/0 (socket says 172.21.15.181:42468) 2023-10-16T10:03:18.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda5c0cd2d0 0x7fda5c0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:52498/0 (socket says 172.21.15.181:52498) 2023-10-16T10:03:18.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5affd700 1 -- 172.21.15.181:0/2586495845 learned_addr learned my addr 172.21.15.181:0/2586495845 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:18.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5b7fe700 1 -- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0d0370 msgr2=0x7fda5c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:18.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5b7fe700 1 --2- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0d0370 0x7fda5c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5b7fe700 1 -- 172.21.15.181:0/2586495845 >> v1:172.21.15.190:6789/0 conn(0x7fda5c15b990 legacy=0x7fda5c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:18.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda59ffb700 1 -- 172.21.15.181:0/2586495845 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932377734 0 0) 0x7fda5c0cc510 con 0x7fda5c15b990 2023-10-16T10:03:18.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5b7fe700 1 -- 172.21.15.181:0/2586495845 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0ccad0 con 0x7fda5c0cd2d0 2023-10-16T10:03:18.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5a7fc700 1 --2- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0d0370 0x7fda5c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:18.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda5b7fe700 1 --2- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 0x7fda5c0d1fb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda54009d90 tx=0x7fda5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9633ef057e554db4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:18.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.696+0000 7fda59ffb700 1 -- 172.21.15.181:0/2586495845 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400e070 con 0x7fda5c0cd2d0 2023-10-16T10:03:18.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda59ffb700 1 -- 172.21.15.181:0/2586495845 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5400bb20 con 0x7fda5c0cd2d0 2023-10-16T10:03:18.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda59ffb700 1 -- 172.21.15.181:0/2586495845 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54012430 con 0x7fda5c0cd2d0 2023-10-16T10:03:18.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 -- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 msgr2=0x7fda5c0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:18.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 --2- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 0x7fda5c0d1fb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fda54009d90 tx=0x7fda5400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:18.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 -- 172.21.15.181:0/2586495845 shutdown_connections 2023-10-16T10:03:18.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 --2- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 0x7fda5c0d1fb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 --2- 172.21.15.181:0/2586495845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c0d0370 0x7fda5c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 -- 172.21.15.181:0/2586495845 >> 172.21.15.181:0/2586495845 conn(0x7fda5c09a9a0 msgr2=0x7fda5c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:18.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 -- 172.21.15.181:0/2586495845 shutdown_connections 2023-10-16T10:03:18.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.697+0000 7fda613eb700 1 -- 172.21.15.181:0/2586495845 wait complete. 2023-10-16T10:03:18.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.698+0000 7fda613eb700 1 Processor -- start 2023-10-16T10:03:18.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.698+0000 7fda613eb700 1 -- start start 2023-10-16T10:03:18.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda613eb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 0x7fda5c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:18.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda613eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c0d0370 0x7fda5c15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:18.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda613eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c15b990 0x7fda5c06f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:18.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda613eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15efc0 con 0x7fda5c0d0370 2023-10-16T10:03:18.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda613eb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c06bbe0 con 0x7fda5c0cd2d0 2023-10-16T10:03:18.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda613eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c15e720 con 0x7fda5c15b990 2023-10-16T10:03:18.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda5a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c0d0370 0x7fda5c15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:18.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda5a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c0d0370 0x7fda5c15ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47454/0 (socket says 172.21.15.181:47454) 2023-10-16T10:03:18.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda5a7fc700 1 -- 172.21.15.181:0/4259018183 learned_addr learned my addr 172.21.15.181:0/4259018183 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:18.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda5affd700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 0x7fda5c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:18.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.699+0000 7fda5b7fe700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c15b990 0x7fda5c06f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:18.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda5a7fc700 1 -- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c15b990 msgr2=0x7fda5c06f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:18.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda5a7fc700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c15b990 0x7fda5c06f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda5a7fc700 1 -- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 msgr2=0x7fda5c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:18.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda5a7fc700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 0x7fda5c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda5a7fc700 1 -- 172.21.15.181:0/4259018183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0094a0 con 0x7fda5c0d0370 2023-10-16T10:03:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda5a7fc700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c0d0370 0x7fda5c15ddb0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00ebe0 tx=0x7fda4c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda437fe700 1 -- 172.21.15.181:0/4259018183 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c009760 con 0x7fda5c0d0370 2023-10-16T10:03:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda437fe700 1 -- 172.21.15.181:0/4259018183 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00c850 con 0x7fda5c0d0370 2023-10-16T10:03:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54009a70 con 0x7fda5c0d0370 2023-10-16T10:03:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda437fe700 1 -- 172.21.15.181:0/4259018183 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00cbe0 con 0x7fda5c0d0370 2023-10-16T10:03:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.700+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c15ece0 con 0x7fda5c0d0370 2023-10-16T10:03:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.701+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c06b9b0 con 0x7fda5c0d0370 2023-10-16T10:03:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.701+0000 7fda437fe700 1 -- 172.21.15.181:0/4259018183 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fda4c00cd40 con 0x7fda5c0d0370 2023-10-16T10:03:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.702+0000 7fda437fe700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fda4806b020 0x7fda4806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.702+0000 7fda437fe700 1 -- 172.21.15.181:0/4259018183 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5309+0+0 (secure 0 0 0) 0x7fda4c089590 con 0x7fda5c0d0370 2023-10-16T10:03:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.702+0000 7fda5affd700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fda4806b020 0x7fda4806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.702+0000 7fda5affd700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fda4806b020 0x7fda4806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fda50000fb0 tx=0x7fda50000ed0 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.706+0000 7fda437fe700 1 -- 172.21.15.181:0/4259018183 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fda5c06b9b0 con 0x7fda5c0d0370 2023-10-16T10:03:18.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.977+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fda5c06b9b0 con 0x7fda5c0d0370 2023-10-16T10:03:18.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.977+0000 7fda437fe700 1 -- 172.21.15.181:0/4259018183 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fda5c06b9b0 con 0x7fda5c0d0370 2023-10-16T10:03:18.979 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:18.979 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:03:18.979 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T10:03:18.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.980 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T10:03:18.980 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T10:03:18.980 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.980 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:18.980 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:18.981 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:18.981 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.981 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.981 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T10:03:18.982 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T10:03:18.982 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.982 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:18.982 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:18.982 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:18.983 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.983 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.983 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T10:03:18.984 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T10:03:18.984 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:18.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:18.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:18.985 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.985 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.985 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T10:03:18.986 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T10:03:18.986 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.986 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:18.986 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:18.986 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:18.987 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.987 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.987 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T10:03:18.987 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T10:03:18.988 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.988 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:18.988 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:18.988 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.988 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.989 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T10:03:18.989 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T10:03:18.989 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.989 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:18.990 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:18.990 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.990 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.993 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T10:03:18.994 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T10:03:18.994 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.994 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:18.994 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:18.994 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.995 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.995 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T10:03:18.995 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T10:03:18.995 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.996 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:18.996 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:18.996 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.997 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T10:03:18.997 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T10:03:18.997 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.997 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:18.998 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:18.998 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:18.998 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:18.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:18.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:18.999 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T10:03:18.999 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T10:03:18.999 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:18.999 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:18.999 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:18.999 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:19.000 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.000 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T10:03:19.000 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T10:03:19.001 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.001 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:19.001 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:19.001 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:19.001 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.002 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T10:03:19.002 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T10:03:19.002 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.002 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:19.003 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:19.003 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:19.003 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.003 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T10:03:19.004 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T10:03:19.004 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.004 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:19.004 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T10:03:19.004 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:03:19.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.005 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T10:03:19.005 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T10:03:19.005 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.006 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:19.006 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:03:19.006 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:03:19.006 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:03:19.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.007 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T10:03:19.007 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T10:03:19.007 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:19.008 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:03:19.008 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:03:19.008 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:03:19.008 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.009 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T10:03:19.009 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T10:03:19.009 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.009 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:19.009 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:03:19.010 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:03:19.010 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:03:19.010 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.011 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T10:03:19.011 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T10:03:19.011 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.011 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:03:19.011 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:19.012 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:19.012 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:19.012 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.012 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T10:03:19.013 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T10:03:19.013 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.013 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:03:19.013 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:03:19.013 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:19.014 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:19.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:19.014 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T10:03:19.014 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T10:03:19.014 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:19.015 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:03:19.015 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:03:19.015 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:19.015 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:19.016 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T10:03:19.016 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:03:19.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fda4806b020 msgr2=0x7fda4806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fda4806b020 0x7fda4806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fda50000fb0 tx=0x7fda50000ed0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c0d0370 msgr2=0x7fda5c15ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c0d0370 0x7fda5c15ddb0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00ebe0 tx=0x7fda4c00eef0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 shutdown_connections 2023-10-16T10:03:19.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fda4806b020 0x7fda4806d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5c15b990 0x7fda5c06f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5c0cd2d0 0x7fda5c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 --2- 172.21.15.181:0/4259018183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda5c0d0370 0x7fda5c15ddb0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 >> 172.21.15.181:0/4259018183 conn(0x7fda5c09a9a0 msgr2=0x7fda5c0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:19.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 shutdown_connections 2023-10-16T10:03:19.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:18.980+0000 7fda613eb700 1 -- 172.21.15.181:0/4259018183 wait complete. 2023-10-16T10:03:19.020 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.140+0000 7fee58de9700 1 Processor -- start 2023-10-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.140+0000 7fee58de9700 1 -- start start 2023-10-16T10:03:19.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee58de9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee5415b990 0x7fee540ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee58de9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee540d0370 0x7fee540d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee58de9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee540cc510 con 0x7fee540cd2d0 2023-10-16T10:03:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee58de9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee540d24f0 con 0x7fee5415b990 2023-10-16T10:03:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee58de9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee540cc7f0 con 0x7fee540d0370 2023-10-16T10:03:19.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee5259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee5415b990 0x7fee540ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee5259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee5415b990 0x7fee540ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47460/0 (socket says 172.21.15.181:47460) 2023-10-16T10:03:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.141+0000 7fee5259c700 1 -- 172.21.15.181:0/2844248692 learned_addr learned my addr 172.21.15.181:0/2844248692 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee52d9d700 1 --2- 172.21.15.181:0/2844248692 >> v2:172.21.15.190:3300/0 conn(0x7fee540d0370 0x7fee540d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5259c700 1 -- 172.21.15.181:0/2844248692 >> v2:172.21.15.190:3300/0 conn(0x7fee540d0370 msgr2=0x7fee540d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5259c700 1 --2- 172.21.15.181:0/2844248692 >> v2:172.21.15.190:3300/0 conn(0x7fee540d0370 0x7fee540d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5259c700 1 -- 172.21.15.181:0/2844248692 >> v1:172.21.15.190:6789/0 conn(0x7fee540cd2d0 legacy=0x7fee540cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:19.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5259c700 1 -- 172.21.15.181:0/2844248692 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540ccad0 con 0x7fee5415b990 2023-10-16T10:03:19.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5259c700 1 --2- 172.21.15.181:0/2844248692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee5415b990 0x7fee540ce8f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fee5409ae40 tx=0x7fee48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c832ed21ee30850 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5159a700 1 -- 172.21.15.181:0/2844248692 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48005e30 con 0x7fee5415b990 2023-10-16T10:03:19.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5159a700 1 -- 172.21.15.181:0/2844248692 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48006030 con 0x7fee5415b990 2023-10-16T10:03:19.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee5159a700 1 -- 172.21.15.181:0/2844248692 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee480063e0 con 0x7fee5415b990 2023-10-16T10:03:19.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee58de9700 1 -- 172.21.15.181:0/2844248692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee5415b990 msgr2=0x7fee540ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.142+0000 7fee58de9700 1 --2- 172.21.15.181:0/2844248692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee5415b990 0x7fee540ce8f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fee5409ae40 tx=0x7fee48009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.143+0000 7fee58de9700 1 -- 172.21.15.181:0/2844248692 shutdown_connections 2023-10-16T10:03:19.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.143+0000 7fee58de9700 1 --2- 172.21.15.181:0/2844248692 >> v2:172.21.15.190:3300/0 conn(0x7fee540d0370 0x7fee540d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.143+0000 7fee58de9700 1 --2- 172.21.15.181:0/2844248692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee5415b990 0x7fee540ce8f0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.143+0000 7fee58de9700 1 -- 172.21.15.181:0/2844248692 >> 172.21.15.181:0/2844248692 conn(0x7fee5409a9a0 msgr2=0x7fee540d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:19.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.143+0000 7fee58de9700 1 -- 172.21.15.181:0/2844248692 shutdown_connections 2023-10-16T10:03:19.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.143+0000 7fee58de9700 1 -- 172.21.15.181:0/2844248692 wait complete. 2023-10-16T10:03:19.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.144+0000 7fee58de9700 1 Processor -- start 2023-10-16T10:03:19.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.144+0000 7fee58de9700 1 -- start start 2023-10-16T10:03:19.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee58de9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee540cd2d0 0x7fee5414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee58de9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee540d0370 0x7fee540cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee58de9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee5415b990 0x7fee541f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee58de9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee541f5460 con 0x7fee540d0370 2023-10-16T10:03:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee58de9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee540c8950 con 0x7fee5415b990 2023-10-16T10:03:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee58de9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee541f4bc0 con 0x7fee540cd2d0 2023-10-16T10:03:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee51d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee540d0370 0x7fee540cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee51d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee540d0370 0x7fee540cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47468/0 (socket says 172.21.15.181:47468) 2023-10-16T10:03:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee51d9b700 1 -- 172.21.15.181:0/2120386696 learned_addr learned my addr 172.21.15.181:0/2120386696 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.145+0000 7fee52d9d700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee5415b990 0x7fee541f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee5259c700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee540cd2d0 0x7fee5414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee51d9b700 1 -- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee540cd2d0 msgr2=0x7fee5414fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee51d9b700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee540cd2d0 0x7fee5414fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee51d9b700 1 -- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee5415b990 msgr2=0x7fee541f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee51d9b700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee5415b990 0x7fee541f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee51d9b700 1 -- 172.21.15.181:0/2120386696 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009500 con 0x7fee540d0370 2023-10-16T10:03:19.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee51d9b700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee540d0370 0x7fee540cb4a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c4d0 tx=0x7fee4400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee42ffd700 1 -- 172.21.15.181:0/2120386696 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44011790 con 0x7fee540d0370 2023-10-16T10:03:19.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee42ffd700 1 -- 172.21.15.181:0/2120386696 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee440118f0 con 0x7fee540d0370 2023-10-16T10:03:19.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee541f4ea0 con 0x7fee540d0370 2023-10-16T10:03:19.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee42ffd700 1 -- 172.21.15.181:0/2120386696 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44011ca0 con 0x7fee540d0370 2023-10-16T10:03:19.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.146+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee540c6710 con 0x7fee540d0370 2023-10-16T10:03:19.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.147+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee540c6570 con 0x7fee540d0370 2023-10-16T10:03:19.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.147+0000 7fee42ffd700 1 -- 172.21.15.181:0/2120386696 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fee4401c460 con 0x7fee540d0370 2023-10-16T10:03:19.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.148+0000 7fee42ffd700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fee3406b020 0x7fee3406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.148+0000 7fee42ffd700 1 -- 172.21.15.181:0/2120386696 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5309+0+0 (secure 0 0 0) 0x7fee440892f0 con 0x7fee540d0370 2023-10-16T10:03:19.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.148+0000 7fee5259c700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fee3406b020 0x7fee3406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.148+0000 7fee5259c700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fee3406b020 0x7fee3406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fee5409ae40 tx=0x7fee48008070 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.153+0000 7fee42ffd700 1 -- 172.21.15.181:0/2120386696 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fee540c6570 con 0x7fee540d0370 2023-10-16T10:03:19.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.344+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee540c6570 con 0x7fee540d0370 2023-10-16T10:03:19.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.345+0000 7fee42ffd700 1 -- 172.21.15.181:0/2120386696 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13592 (secure 0 0 0) 0x7fee540c6570 con 0x7fee540d0370 2023-10-16T10:03:19.346 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:19.346 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":128,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:18.557469+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:19.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.347+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fee3406b020 msgr2=0x7fee3406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.347+0000 7fee58de9700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fee3406b020 0x7fee3406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fee5409ae40 tx=0x7fee48008070 comp rx=0 tx=0).stop 2023-10-16T10:03:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.347+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee540d0370 msgr2=0x7fee540cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.347+0000 7fee58de9700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee540d0370 0x7fee540cb4a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c4d0 tx=0x7fee4400c7e0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 shutdown_connections 2023-10-16T10:03:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fee3406b020 0x7fee3406d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee540cd2d0 0x7fee5414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee5415b990 0x7fee541f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 --2- 172.21.15.181:0/2120386696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee540d0370 0x7fee540cb4a0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 >> 172.21.15.181:0/2120386696 conn(0x7fee5409a9a0 msgr2=0x7fee5415d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 shutdown_connections 2023-10-16T10:03:19.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.348+0000 7fee58de9700 1 -- 172.21.15.181:0/2120386696 wait complete. 2023-10-16T10:03:19.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:19.361 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:19.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.484+0000 7f070823b700 1 Processor -- start 2023-10-16T10:03:19.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.484+0000 7f070823b700 1 -- start start 2023-10-16T10:03:19.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f070823b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07000b2010 0x7f07000cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f070823b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07000ce1f0 con 0x7f0700071be0 2023-10-16T10:03:19.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f070823b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0700072090 con 0x7f07000ce680 2023-10-16T10:03:19.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f070823b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07000ccda0 con 0x7f07000b2010 2023-10-16T10:03:19.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0705fd7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07000b2010 0x7f07000cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f07057d6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0700071be0 0x7f07000b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58554/0 (socket says 172.21.15.181:58554) 2023-10-16T10:03:19.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f07057d6700 1 -- 172.21.15.181:0/4242222567 learned_addr learned my addr 172.21.15.181:0/4242222567 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:19.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0705fd7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07000b2010 0x7f07000cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47474/0 (socket says 172.21.15.181:47474) 2023-10-16T10:03:19.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0705fd7700 1 -- 172.21.15.181:0/4242222567 >> v1:172.21.15.190:6789/0 conn(0x7f07000ce680 legacy=0x7f07000ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0705fd7700 1 -- 172.21.15.181:0/4242222567 >> v1:172.21.15.181:6789/0 conn(0x7f0700071be0 legacy=0x7f07000b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0705fd7700 1 -- 172.21.15.181:0/4242222567 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000727e0 con 0x7f07000b2010 2023-10-16T10:03:19.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0704fd5700 1 -- 172.21.15.181:0/4242222567 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454464809 0 0) 0x7f07000ce1f0 con 0x7f0700071be0 2023-10-16T10:03:19.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0705fd7700 1 --2- 172.21.15.181:0/4242222567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000cc6e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00bae0 tx=0x7f06fc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c97951fb524e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0704fd5700 1 -- 172.21.15.181:0/4242222567 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00dd30 con 0x7f07000b2010 2023-10-16T10:03:19.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0704fd5700 1 -- 172.21.15.181:0/4242222567 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06fc00de90 con 0x7f07000b2010 2023-10-16T10:03:19.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.485+0000 7f0704fd5700 1 -- 172.21.15.181:0/4242222567 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc010690 con 0x7f07000b2010 2023-10-16T10:03:19.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.486+0000 7f070823b700 1 -- 172.21.15.181:0/4242222567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 msgr2=0x7f07000cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.486+0000 7f070823b700 1 --2- 172.21.15.181:0/4242222567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000cc6e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00bae0 tx=0x7f06fc00bdf0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.486+0000 7f070823b700 1 -- 172.21.15.181:0/4242222567 shutdown_connections 2023-10-16T10:03:19.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.486+0000 7f070823b700 1 --2- 172.21.15.181:0/4242222567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000cc6e0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.486+0000 7f070823b700 1 -- 172.21.15.181:0/4242222567 >> 172.21.15.181:0/4242222567 conn(0x7f07000cd010 msgr2=0x7f07000cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:19.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.486+0000 7f070823b700 1 -- 172.21.15.181:0/4242222567 shutdown_connections 2023-10-16T10:03:19.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.486+0000 7f070823b700 1 -- 172.21.15.181:0/4242222567 wait complete. 2023-10-16T10:03:19.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.487+0000 7f070823b700 1 Processor -- start 2023-10-16T10:03:19.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.487+0000 7f070823b700 1 -- start start 2023-10-16T10:03:19.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f070823b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0700071be0 0x7f07000b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f070823b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f070823b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07000ce680 0x7f0700148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f070823b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000af1d0 con 0x7f07000b2010 2023-10-16T10:03:19.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f070823b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b42f0 con 0x7f07000ce680 2023-10-16T10:03:19.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f070823b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b45d0 con 0x7f0700071be0 2023-10-16T10:03:19.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47482/0 (socket says 172.21.15.181:47482) 2023-10-16T10:03:19.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 -- 172.21.15.181:0/1156752043 learned_addr learned my addr 172.21.15.181:0/1156752043 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:19.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07067d8700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07000ce680 0x7f0700148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f0705fd7700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0700071be0 0x7f07000b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 -- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0700071be0 msgr2=0x7f07000b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0700071be0 0x7f07000b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 -- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07000ce680 msgr2=0x7f0700148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07000ce680 0x7f0700148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 -- 172.21.15.181:0/1156752043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc00b740 con 0x7f07000b2010 2023-10-16T10:03:19.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.488+0000 7f07057d6700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000aec90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c520 tx=0x7f06f000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.489+0000 7f06f67fc700 1 -- 172.21.15.181:0/1156752043 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0004c80 con 0x7f07000b2010 2023-10-16T10:03:19.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.489+0000 7f06f67fc700 1 -- 172.21.15.181:0/1156752043 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f0004de0 con 0x7f07000b2010 2023-10-16T10:03:19.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.489+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07000b48b0 con 0x7f07000b2010 2023-10-16T10:03:19.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.489+0000 7f06f67fc700 1 -- 172.21.15.181:0/1156752043 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0007310 con 0x7f07000b2010 2023-10-16T10:03:19.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.489+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07001542e0 con 0x7f07000b2010 2023-10-16T10:03:19.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.490+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07000ac910 con 0x7f07000b2010 2023-10-16T10:03:19.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.490+0000 7f06f67fc700 1 -- 172.21.15.181:0/1156752043 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f06f0007470 con 0x7f07000b2010 2023-10-16T10:03:19.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.491+0000 7f06f67fc700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f06e406b020 0x7f06e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.491+0000 7f06f67fc700 1 -- 172.21.15.181:0/1156752043 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5309+0+0 (secure 0 0 0) 0x7f06f00890a0 con 0x7f07000b2010 2023-10-16T10:03:19.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.491+0000 7f0705fd7700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f06e406b020 0x7f06e406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.491+0000 7f0705fd7700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f06e406b020 0x7f06e406d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00b2c0 tx=0x7f06fc01b040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.495+0000 7f06f67fc700 1 -- 172.21.15.181:0/1156752043 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f07000ac910 con 0x7f07000b2010 2023-10-16T10:03:19.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.690+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07000ac910 con 0x7f07000b2010 2023-10-16T10:03:19.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.691+0000 7f06f67fc700 1 -- 172.21.15.181:0/1156752043 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07000ac910 con 0x7f07000b2010 2023-10-16T10:03:19.691 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:19.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f06e406b020 msgr2=0x7f06e406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f06e406b020 0x7f06e406d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00b2c0 tx=0x7f06fc01b040 comp rx=0 tx=0).stop 2023-10-16T10:03:19.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 msgr2=0x7f07000aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000aec90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f06f000c520 tx=0x7f06f000c830 comp rx=0 tx=0).stop 2023-10-16T10:03:19.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 shutdown_connections 2023-10-16T10:03:19.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f06e406b020 0x7f06e406d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0700071be0 0x7f07000b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07000ce680 0x7f0700148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 --2- 172.21.15.181:0/1156752043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07000b2010 0x7f07000aec90 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 >> 172.21.15.181:0/1156752043 conn(0x7f07000cd010 msgr2=0x7f07000cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:19.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 shutdown_connections 2023-10-16T10:03:19.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.693+0000 7f070823b700 1 -- 172.21.15.181:0/1156752043 wait complete. 2023-10-16T10:03:19.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:03:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.825+0000 7f2539160700 1 Processor -- start 2023-10-16T10:03:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.825+0000 7f2539160700 1 -- start start 2023-10-16T10:03:19.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.826+0000 7f2539160700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25340cdb80 0x7f25340af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.826+0000 7f2539160700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253406f870 con 0x7f253415dfd0 2023-10-16T10:03:19.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.826+0000 7f2539160700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340d1160 con 0x7f25340af450 2023-10-16T10:03:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.826+0000 7f2539160700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25340cc8e0 con 0x7f25340cdb80 2023-10-16T10:03:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.826+0000 7f253359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25340cdb80 0x7f25340af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.826+0000 7f2532d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f253415dfd0 0x7f253406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58568/0 (socket says 172.21.15.181:58568) 2023-10-16T10:03:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.826+0000 7f2532d9d700 1 -- 172.21.15.181:0/2909315541 learned_addr learned my addr 172.21.15.181:0/2909315541 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f2531d9b700 1 -- 172.21.15.181:0/2909315541 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589481871 0 0) 0x7f253406f870 con 0x7f253415dfd0 2023-10-16T10:03:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f2531d9b700 1 -- 172.21.15.181:0/2909315541 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f251c0035b0 con 0x7f253415dfd0 2023-10-16T10:03:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f253359e700 1 -- 172.21.15.181:0/2909315541 >> v1:172.21.15.190:6789/0 conn(0x7f25340af450 legacy=0x7f25340cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f253359e700 1 -- 172.21.15.181:0/2909315541 >> v1:172.21.15.181:6789/0 conn(0x7f253415dfd0 legacy=0x7f253406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f253359e700 1 -- 172.21.15.181:0/2909315541 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25340ccbc0 con 0x7f25340cdb80 2023-10-16T10:03:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f253359e700 1 --2- 172.21.15.181:0/2909315541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340af9a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f252c009d90 tx=0x7f252c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a464b3628ce9f7ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f2531d9b700 1 -- 172.21.15.181:0/2909315541 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00e070 con 0x7f25340cdb80 2023-10-16T10:03:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f2531d9b700 1 -- 172.21.15.181:0/2909315541 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c00bae0 con 0x7f25340cdb80 2023-10-16T10:03:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f2531d9b700 1 -- 172.21.15.181:0/2909315541 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c00be90 con 0x7f25340cdb80 2023-10-16T10:03:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f2539160700 1 -- 172.21.15.181:0/2909315541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 msgr2=0x7f25340af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.827+0000 7f2539160700 1 --2- 172.21.15.181:0/2909315541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340af9a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f252c009d90 tx=0x7f252c00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.828+0000 7f2539160700 1 -- 172.21.15.181:0/2909315541 shutdown_connections 2023-10-16T10:03:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.828+0000 7f2539160700 1 --2- 172.21.15.181:0/2909315541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340af9a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.828+0000 7f2539160700 1 -- 172.21.15.181:0/2909315541 >> 172.21.15.181:0/2909315541 conn(0x7f25340d04a0 msgr2=0x7f25340d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.828+0000 7f2539160700 1 -- 172.21.15.181:0/2909315541 shutdown_connections 2023-10-16T10:03:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.828+0000 7f2539160700 1 -- 172.21.15.181:0/2909315541 wait complete. 2023-10-16T10:03:19.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.829+0000 7f2539160700 1 Processor -- start 2023-10-16T10:03:19.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.829+0000 7f2539160700 1 -- start start 2023-10-16T10:03:19.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2539160700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25340af450 0x7f25340aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2539160700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2539160700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f253415dfd0 0x7f25341f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2539160700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25341f71d0 con 0x7f25340cdb80 2023-10-16T10:03:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2539160700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25341f6ef0 con 0x7f253415dfd0 2023-10-16T10:03:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2539160700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340b0a40 con 0x7f25340af450 2023-10-16T10:03:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47498/0 (socket says 172.21.15.181:47498) 2023-10-16T10:03:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 -- 172.21.15.181:0/2556988314 learned_addr learned my addr 172.21.15.181:0/2556988314 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253359e700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f253415dfd0 0x7f25341f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2532d9d700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25340af450 0x7f25340aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 -- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25340af450 msgr2=0x7f25340aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25340af450 0x7f25340aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 -- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f253415dfd0 msgr2=0x7f25341f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f253415dfd0 0x7f25341f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 -- 172.21.15.181:0/2556988314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c009a70 con 0x7f25340cdb80 2023-10-16T10:03:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f2532d9d700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25340af450 0x7f25340aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.830+0000 7f253259c700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340b57a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f252400c4d0 tx=0x7f252400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.831+0000 7f251b7fe700 1 -- 172.21.15.181:0/2556988314 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524011790 con 0x7f25340cdb80 2023-10-16T10:03:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.831+0000 7f251b7fe700 1 -- 172.21.15.181:0/2556988314 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25240118f0 con 0x7f25340cdb80 2023-10-16T10:03:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.831+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25340b0d20 con 0x7f25340cdb80 2023-10-16T10:03:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.831+0000 7f251b7fe700 1 -- 172.21.15.181:0/2556988314 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524011ca0 con 0x7f25340cdb80 2023-10-16T10:03:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.831+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25340b5ce0 con 0x7f25340cdb80 2023-10-16T10:03:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.831+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25340b1240 con 0x7f25340cdb80 2023-10-16T10:03:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.832+0000 7f251b7fe700 1 -- 172.21.15.181:0/2556988314 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f252401c460 con 0x7f25340cdb80 2023-10-16T10:03:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.832+0000 7f251b7fe700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f252006b020 0x7f252006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.833+0000 7f251b7fe700 1 -- 172.21.15.181:0/2556988314 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5309+0+0 (secure 0 0 0) 0x7f2524089410 con 0x7f25340cdb80 2023-10-16T10:03:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.833+0000 7f2532d9d700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f252006b020 0x7f252006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.833+0000 7f2532d9d700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f252006b020 0x7f252006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2528001360 tx=0x7f2528009040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:19.837+0000 7f251b7fe700 1 -- 172.21.15.181:0/2556988314 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f25340b1240 con 0x7f25340cdb80 2023-10-16T10:03:20.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.030+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25340b1240 con 0x7f25340cdb80 2023-10-16T10:03:20.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.030+0000 7f251b7fe700 1 -- 172.21.15.181:0/2556988314 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+65 (secure 0 0 0) 0x7f25340b1240 con 0x7f25340cdb80 2023-10-16T10:03:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:20.032 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:03:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T10:03:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T10:03:20.033 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:20.033 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:03:20.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.032+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f252006b020 msgr2=0x7f252006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.032+0000 7f2539160700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f252006b020 0x7f252006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2528001360 tx=0x7f2528009040 comp rx=0 tx=0).stop 2023-10-16T10:03:20.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.032+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 msgr2=0x7f25340b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.032+0000 7f2539160700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340b57a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f252400c4d0 tx=0x7f252400c890 comp rx=0 tx=0).stop 2023-10-16T10:03:20.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 shutdown_connections 2023-10-16T10:03:20.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f252006b020 0x7f252006d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25340af450 0x7f25340aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f253415dfd0 0x7f25341f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 --2- 172.21.15.181:0/2556988314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25340cdb80 0x7f25340b57a0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 >> 172.21.15.181:0/2556988314 conn(0x7f25340d04a0 msgr2=0x7f25340cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:20.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 shutdown_connections 2023-10-16T10:03:20.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.033+0000 7f2539160700 1 -- 172.21.15.181:0/2556988314 wait complete. 2023-10-16T10:03:20.045 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:03:20.046 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:03:20.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 Processor -- start 2023-10-16T10:03:20.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 -- start start 2023-10-16T10:03:20.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1dbc0cfc30 0x7f1dbc06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dbc15dff0 0x7f1dbc0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc0d1d50 0x7f1dbc0d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dbc06bf70 con 0x7f1dbc15dff0 2023-10-16T10:03:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dbc0d0550 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.170+0000 7f1dc14d1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc06c210 con 0x7f1dbc0d1d50 2023-10-16T10:03:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dbc15dff0 0x7f1dbc0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dbc15dff0 0x7f1dbc0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47504/0 (socket says 172.21.15.181:47504) 2023-10-16T10:03:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 -- 172.21.15.181:0/3281033966 learned_addr learned my addr 172.21.15.181:0/3281033966 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dbaffd700 1 --2- 172.21.15.181:0/3281033966 >> v2:172.21.15.190:3300/0 conn(0x7f1dbc0cfc30 0x7f1dbc06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dbb7fe700 1 --2- 172.21.15.181:0/3281033966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc0d1d50 0x7f1dbc0d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 -- 172.21.15.181:0/3281033966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc0d1d50 msgr2=0x7f1dbc0d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 --2- 172.21.15.181:0/3281033966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc0d1d50 0x7f1dbc0d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 -- 172.21.15.181:0/3281033966 >> v2:172.21.15.190:3300/0 conn(0x7f1dbc0cfc30 msgr2=0x7f1dbc06d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 --2- 172.21.15.181:0/3281033966 >> v2:172.21.15.190:3300/0 conn(0x7f1dbc0cfc30 0x7f1dbc06d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 -- 172.21.15.181:0/3281033966 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc06c4f0 con 0x7f1dbc15dff0 2023-10-16T10:03:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1dba7fc700 1 --2- 172.21.15.181:0/3281033966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc15dff0 0x7f1dbc0d0010 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc071490 tx=0x7f1dac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c3581ed8900c4e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1db9ffb700 1 -- 172.21.15.181:0/3281033966 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac011830 con 0x7f1dbc15dff0 2023-10-16T10:03:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1db9ffb700 1 -- 172.21.15.181:0/3281033966 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac011990 con 0x7f1dbc15dff0 2023-10-16T10:03:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.171+0000 7f1db9ffb700 1 -- 172.21.15.181:0/3281033966 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac011d20 con 0x7f1dbc15dff0 2023-10-16T10:03:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 -- 172.21.15.181:0/3281033966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc15dff0 msgr2=0x7f1dbc0d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/3281033966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc15dff0 0x7f1dbc0d0010 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc071490 tx=0x7f1dac009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 -- 172.21.15.181:0/3281033966 shutdown_connections 2023-10-16T10:03:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/3281033966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc0d1d50 0x7f1dbc0d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/3281033966 >> v2:172.21.15.190:3300/0 conn(0x7f1dbc0cfc30 0x7f1dbc06d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/3281033966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc15dff0 0x7f1dbc0d0010 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 -- 172.21.15.181:0/3281033966 >> 172.21.15.181:0/3281033966 conn(0x7f1dbc09a970 msgr2=0x7f1dbc0d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 -- 172.21.15.181:0/3281033966 shutdown_connections 2023-10-16T10:03:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.172+0000 7f1dc14d1700 1 -- 172.21.15.181:0/3281033966 wait complete. 2023-10-16T10:03:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.173+0000 7f1dc14d1700 1 Processor -- start 2023-10-16T10:03:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.174+0000 7f1dc14d1700 1 -- start start 2023-10-16T10:03:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.174+0000 7f1dc14d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc0cfc30 0x7f1dbc09d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.174+0000 7f1dc14d1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dbc0d1d50 0x7f1dbc09b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.174+0000 7f1dc14d1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc15dff0 0x7f1dbc14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.174+0000 7f1dc14d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc09d550 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.174+0000 7f1dc14d1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc149850 con 0x7f1dbc0d1d50 2023-10-16T10:03:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.174+0000 7f1dc14d1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc149b30 con 0x7f1dbc15dff0 2023-10-16T10:03:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc0cfc30 0x7f1dbc09d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc0cfc30 0x7f1dbc09d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47510/0 (socket says 172.21.15.181:47510) 2023-10-16T10:03:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 -- 172.21.15.181:0/1423572717 learned_addr learned my addr 172.21.15.181:0/1423572717 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 -- 172.21.15.181:0/1423572717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc15dff0 msgr2=0x7f1dbc14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc15dff0 0x7f1dbc14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 -- 172.21.15.181:0/1423572717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dbc0d1d50 msgr2=0x7f1dbc09b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dba7fc700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dbc0d1d50 0x7f1dbc09b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dbc0d1d50 0x7f1dbc09b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 -- 172.21.15.181:0/1423572717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac009550 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dbaffd700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc0cfc30 0x7f1dbc09d010 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e7f0 tx=0x7f1db000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1da37fe700 1 -- 172.21.15.181:0/1423572717 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0013680 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1da37fe700 1 -- 172.21.15.181:0/1423572717 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db00137e0 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1da37fe700 1 -- 172.21.15.181:0/1423572717 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db0013b90 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dbc149e10 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.175+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dbc14a3d0 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.176+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dbc14a560 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.176+0000 7f1da37fe700 1 -- 172.21.15.181:0/1423572717 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1db0013cf0 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.177+0000 7f1da37fe700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f1da806b020 0x7f1da806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.177+0000 7f1da37fe700 1 -- 172.21.15.181:0/1423572717 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5309+0+0 (secure 0 0 0) 0x7f1db0089480 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.177+0000 7f1dba7fc700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f1da806b020 0x7f1da806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.177+0000 7f1dba7fc700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f1da806b020 0x7f1da806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc071490 tx=0x7f1dac008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:20.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.181+0000 7f1da37fe700 1 -- 172.21.15.181:0/1423572717 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1db00545b0 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.376+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1dbc04ebb0 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.557+0000 7f1da37fe700 1 -- 172.21.15.181:0/1423572717 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1db0057bd0 con 0x7f1dbc0cfc30 2023-10-16T10:03:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f1da806b020 msgr2=0x7f1da806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f1da806b020 0x7f1da806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc071490 tx=0x7f1dac008030 comp rx=0 tx=0).stop 2023-10-16T10:03:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc0cfc30 msgr2=0x7f1dbc09d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc0cfc30 0x7f1dbc09d010 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1db000e7f0 tx=0x7f1db000ebb0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 shutdown_connections 2023-10-16T10:03:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f1da806b020 0x7f1da806d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dbc15dff0 0x7f1dbc14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dbc0cfc30 0x7f1dbc09d010 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 --2- 172.21.15.181:0/1423572717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dbc0d1d50 0x7f1dbc09b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 >> 172.21.15.181:0/1423572717 conn(0x7f1dbc09a970 msgr2=0x7f1dbc15d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 shutdown_connections 2023-10-16T10:03:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.560+0000 7f1dc14d1700 1 -- 172.21.15.181:0/1423572717 wait complete. 2023-10-16T10:03:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T10:03:20.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:03:20.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.703+0000 7f7084cbd700 1 Processor -- start 2023-10-16T10:03:20.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.703+0000 7f7084cbd700 1 -- start start 2023-10-16T10:03:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f7084cbd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7080072760 0x7f708006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f7084cbd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708006b210 con 0x7f708006ade0 2023-10-16T10:03:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f7084cbd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70800cef00 con 0x7f708015da70 2023-10-16T10:03:20.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f7084cbd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70800cec20 con 0x7f7080072760 2023-10-16T10:03:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f707dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7080072760 0x7f708006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f707ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f708006ade0 0x7f70801615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58598/0 (socket says 172.21.15.181:58598) 2023-10-16T10:03:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f707ed9d700 1 -- 172.21.15.181:0/733687073 learned_addr learned my addr 172.21.15.181:0/733687073 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f707dd9b700 1 -- 172.21.15.181:0/733687073 >> v1:172.21.15.190:6789/0 conn(0x7f708015da70 legacy=0x7f70800ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f707dd9b700 1 -- 172.21.15.181:0/733687073 >> v1:172.21.15.181:6789/0 conn(0x7f708006ade0 legacy=0x7f70801615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f707dd9b700 1 -- 172.21.15.181:0/733687073 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7080071c80 con 0x7f7080072760 2023-10-16T10:03:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.704+0000 7f707dd9b700 1 --2- 172.21.15.181:0/733687073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7080072760 0x7f708006a8a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f7074009840 tx=0x7f7074009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32da9a62905c0f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f707d59a700 1 -- 172.21.15.181:0/733687073 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074011830 con 0x7f7080072760 2023-10-16T10:03:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f707d59a700 1 -- 172.21.15.181:0/733687073 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7074011990 con 0x7f7080072760 2023-10-16T10:03:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f707d59a700 1 -- 172.21.15.181:0/733687073 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074011d20 con 0x7f7080072760 2023-10-16T10:03:20.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f7084cbd700 1 -- 172.21.15.181:0/733687073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7080072760 msgr2=0x7f708006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f7084cbd700 1 --2- 172.21.15.181:0/733687073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7080072760 0x7f708006a8a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f7074009840 tx=0x7f7074009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:20.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f7084cbd700 1 -- 172.21.15.181:0/733687073 shutdown_connections 2023-10-16T10:03:20.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f7084cbd700 1 --2- 172.21.15.181:0/733687073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7080072760 0x7f708006a8a0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f7084cbd700 1 -- 172.21.15.181:0/733687073 >> 172.21.15.181:0/733687073 conn(0x7f70800cd010 msgr2=0x7f70800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:20.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f7084cbd700 1 -- 172.21.15.181:0/733687073 shutdown_connections 2023-10-16T10:03:20.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.705+0000 7f7084cbd700 1 -- 172.21.15.181:0/733687073 wait complete. 2023-10-16T10:03:20.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.706+0000 7f7084cbd700 1 Processor -- start 2023-10-16T10:03:20.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f7084cbd700 1 -- start start 2023-10-16T10:03:20.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f7084cbd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f708006ade0 0x7f70800c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f7084cbd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7080072760 0x7f70800c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f7084cbd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f708015da70 0x7f708014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f7084cbd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c90e0 con 0x7f708006ade0 2023-10-16T10:03:20.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f7084cbd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c8040 con 0x7f708015da70 2023-10-16T10:03:20.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f7084cbd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c8320 con 0x7f7080072760 2023-10-16T10:03:20.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f707e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f708006ade0 0x7f70800c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f707e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f708006ade0 0x7f70800c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47522/0 (socket says 172.21.15.181:47522) 2023-10-16T10:03:20.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f707e59c700 1 -- 172.21.15.181:0/2321980542 learned_addr learned my addr 172.21.15.181:0/2321980542 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:20.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f707ed9d700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f708015da70 0x7f708014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.707+0000 7f707dd9b700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7080072760 0x7f70800c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f707e59c700 1 -- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7080072760 msgr2=0x7f70800c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f707e59c700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7080072760 0x7f70800c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f707e59c700 1 -- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f708015da70 msgr2=0x7f708014d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:20.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f707e59c700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f708015da70 0x7f708014d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f707e59c700 1 -- 172.21.15.181:0/2321980542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70740094a0 con 0x7f708006ade0 2023-10-16T10:03:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f707e59c700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f708006ade0 0x7f70800c9a70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f707000c4e0 tx=0x7f707000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f706effd700 1 -- 172.21.15.181:0/2321980542 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70700052d0 con 0x7f708006ade0 2023-10-16T10:03:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f706effd700 1 -- 172.21.15.181:0/2321980542 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070005430 con 0x7f708006ade0 2023-10-16T10:03:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70800c8600 con 0x7f708006ade0 2023-10-16T10:03:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f706effd700 1 -- 172.21.15.181:0/2321980542 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70700038c0 con 0x7f708006ade0 2023-10-16T10:03:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.708+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70800c8bc0 con 0x7f708006ade0 2023-10-16T10:03:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.709+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70800c8db0 con 0x7f708006ade0 2023-10-16T10:03:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.710+0000 7f706effd700 1 -- 172.21.15.181:0/2321980542 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7070003a20 con 0x7f708006ade0 2023-10-16T10:03:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.710+0000 7f706effd700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f706006b020 0x7f706006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:20.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.710+0000 7f706effd700 1 -- 172.21.15.181:0/2321980542 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f7070088ad0 con 0x7f708006ade0 2023-10-16T10:03:20.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.710+0000 7f707dd9b700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f706006b020 0x7f706006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:20.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.710+0000 7f707dd9b700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f706006b020 0x7f706006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70740090b0 tx=0x7f7074008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:20.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.715+0000 7f706effd700 1 -- 172.21.15.181:0/2321980542 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7070053aa0 con 0x7f708006ade0 2023-10-16T10:03:20.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:20.904+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f708006f690 con 0x7f708006ade0 2023-10-16T10:03:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.567+0000 7f706effd700 1 -- 172.21.15.181:0/2321980542 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) v1 ==== 170+0+0 (secure 0 0 0) 0x7f70700570c0 con 0x7f708006ade0 2023-10-16T10:03:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f706006b020 msgr2=0x7f706006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f706006b020 0x7f706006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70740090b0 tx=0x7f7074008030 comp rx=0 tx=0).stop 2023-10-16T10:03:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f708006ade0 msgr2=0x7f70800c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f708006ade0 0x7f70800c9a70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f707000c4e0 tx=0x7f707000c7f0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 shutdown_connections 2023-10-16T10:03:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f708015da70 0x7f708014d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f708006ade0 0x7f70800c9a70 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f706006b020 0x7f706006d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 --2- 172.21.15.181:0/2321980542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7080072760 0x7f70800c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 >> 172.21.15.181:0/2321980542 conn(0x7f70800cd010 msgr2=0x7f70800cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 shutdown_connections 2023-10-16T10:03:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.570+0000 7f7084cbd700 1 -- 172.21.15.181:0/2321980542 wait complete. 2023-10-16T10:03:21.573 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T10:03:21.596 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:03:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.721+0000 7f31d9a45700 1 Processor -- start 2023-10-16T10:03:21.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.721+0000 7f31d9a45700 1 -- start start 2023-10-16T10:03:21.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d9a45700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31d4159e10 0x7f31d40ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:21.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d9a45700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31d406bef0 0x7f31d40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:21.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d9a45700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d406e4b0 con 0x7f31d40cb0e0 2023-10-16T10:03:21.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d9a45700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d406ed50 con 0x7f31d406bef0 2023-10-16T10:03:21.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d9a45700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d40ceea0 con 0x7f31d4159e10 2023-10-16T10:03:21.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31d406bef0 0x7f31d40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:21.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31d406bef0 0x7f31d40caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47534/0 (socket says 172.21.15.181:47534) 2023-10-16T10:03:21.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d27fc700 1 -- 172.21.15.181:0/1565820180 learned_addr learned my addr 172.21.15.181:0/1565820180 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:21.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d2ffd700 1 --2- 172.21.15.181:0/1565820180 >> v2:172.21.15.190:3300/0 conn(0x7f31d4159e10 0x7f31d40ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:21.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d27fc700 1 -- 172.21.15.181:0/1565820180 >> v2:172.21.15.190:3300/0 conn(0x7f31d4159e10 msgr2=0x7f31d40ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:21.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d27fc700 1 --2- 172.21.15.181:0/1565820180 >> v2:172.21.15.190:3300/0 conn(0x7f31d4159e10 0x7f31d40ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.722+0000 7f31d27fc700 1 -- 172.21.15.181:0/1565820180 >> v1:172.21.15.190:6789/0 conn(0x7f31d40cb0e0 legacy=0x7f31d415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:21.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d27fc700 1 -- 172.21.15.181:0/1565820180 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d406e700 con 0x7f31d406bef0 2023-10-16T10:03:21.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d27fc700 1 --2- 172.21.15.181:0/1565820180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d406bef0 0x7f31d40caba0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f31d4067360 tx=0x7f31c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699abd0cb579bfae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:21.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d1ffb700 1 -- 172.21.15.181:0/1565820180 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4010a90 con 0x7f31d406bef0 2023-10-16T10:03:21.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d1ffb700 1 -- 172.21.15.181:0/1565820180 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c4010bf0 con 0x7f31d406bef0 2023-10-16T10:03:21.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d1ffb700 1 -- 172.21.15.181:0/1565820180 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4018440 con 0x7f31d406bef0 2023-10-16T10:03:21.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d9a45700 1 -- 172.21.15.181:0/1565820180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d406bef0 msgr2=0x7f31d40caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:21.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d9a45700 1 --2- 172.21.15.181:0/1565820180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d406bef0 0x7f31d40caba0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f31d4067360 tx=0x7f31c4009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:21.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d9a45700 1 -- 172.21.15.181:0/1565820180 shutdown_connections 2023-10-16T10:03:21.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d9a45700 1 --2- 172.21.15.181:0/1565820180 >> v2:172.21.15.190:3300/0 conn(0x7f31d4159e10 0x7f31d40ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d9a45700 1 --2- 172.21.15.181:0/1565820180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d406bef0 0x7f31d40caba0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.723+0000 7f31d9a45700 1 -- 172.21.15.181:0/1565820180 >> 172.21.15.181:0/1565820180 conn(0x7f31d40cc730 msgr2=0x7f31d40ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:21.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.724+0000 7f31d9a45700 1 -- 172.21.15.181:0/1565820180 shutdown_connections 2023-10-16T10:03:21.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.724+0000 7f31d9a45700 1 -- 172.21.15.181:0/1565820180 wait complete. 2023-10-16T10:03:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.725+0000 7f31d9a45700 1 Processor -- start 2023-10-16T10:03:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.725+0000 7f31d9a45700 1 -- start start 2023-10-16T10:03:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d9a45700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d406bef0 0x7f31d4149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d9a45700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d40cb0e0 0x7f31d40cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d9a45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d4159e10 0x7f31d4158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d9a45700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d414a470 con 0x7f31d4159e10 2023-10-16T10:03:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d9a45700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4157670 con 0x7f31d40cb0e0 2023-10-16T10:03:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d9a45700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4157950 con 0x7f31d406bef0 2023-10-16T10:03:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d4159e10 0x7f31d4158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d4159e10 0x7f31d4158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47540/0 (socket says 172.21.15.181:47540) 2023-10-16T10:03:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d37fe700 1 -- 172.21.15.181:0/2308014472 learned_addr learned my addr 172.21.15.181:0/2308014472 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d27fc700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d40cb0e0 0x7f31d40cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.726+0000 7f31d2ffd700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d406bef0 0x7f31d4149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d37fe700 1 -- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d406bef0 msgr2=0x7f31d4149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d37fe700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d406bef0 0x7f31d4149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d37fe700 1 -- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d40cb0e0 msgr2=0x7f31d40cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d37fe700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d40cb0e0 0x7f31d40cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d37fe700 1 -- 172.21.15.181:0/2308014472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4009550 con 0x7f31d4159e10 2023-10-16T10:03:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d37fe700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d4159e10 0x7f31d4158710 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00cc50 tx=0x7f31cc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31bbfff700 1 -- 172.21.15.181:0/2308014472 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc003c00 con 0x7f31d4159e10 2023-10-16T10:03:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31bbfff700 1 -- 172.21.15.181:0/2308014472 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc003d60 con 0x7f31d4159e10 2023-10-16T10:03:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d4157c30 con 0x7f31d4159e10 2023-10-16T10:03:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31bbfff700 1 -- 172.21.15.181:0/2308014472 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00aab0 con 0x7f31d4159e10 2023-10-16T10:03:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.727+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d41581f0 con 0x7f31d4159e10 2023-10-16T10:03:21.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.728+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d4066c10 con 0x7f31d4159e10 2023-10-16T10:03:21.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.728+0000 7f31bbfff700 1 -- 172.21.15.181:0/2308014472 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31cc00ac10 con 0x7f31d4159e10 2023-10-16T10:03:21.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.729+0000 7f31bbfff700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f31c006b020 0x7f31c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:21.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.729+0000 7f31bbfff700 1 -- 172.21.15.181:0/2308014472 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6061+0+0 (secure 0 0 0) 0x7f31cc089800 con 0x7f31d4159e10 2023-10-16T10:03:21.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.729+0000 7f31d2ffd700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f31c006b020 0x7f31c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:21.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.729+0000 7f31d2ffd700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f31c006b020 0x7f31c006d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31d40c72a0 tx=0x7f31c8008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:21.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.733+0000 7f31bbfff700 1 -- 172.21.15.181:0/2308014472 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31cc0546d0 con 0x7f31d4159e10 2023-10-16T10:03:21.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:21.970+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f31d40cd5d0 con 0x7f31d4159e10 2023-10-16T10:03:22.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.577+0000 7f31bbfff700 1 -- 172.21.15.181:0/2308014472 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7f31cc057cf0 con 0x7f31d4159e10 2023-10-16T10:03:22.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.584+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f31c006b020 msgr2=0x7f31c006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.584+0000 7f31d9a45700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f31c006b020 0x7f31c006d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31d40c72a0 tx=0x7f31c8008040 comp rx=0 tx=0).stop 2023-10-16T10:03:22.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.584+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d4159e10 msgr2=0x7f31d4158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.584+0000 7f31d9a45700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d4159e10 0x7f31d4158710 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00cc50 tx=0x7f31cc00cf60 comp rx=0 tx=0).stop 2023-10-16T10:03:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.585+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 shutdown_connections 2023-10-16T10:03:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.585+0000 7f31d9a45700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f31c006b020 0x7f31c006d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.585+0000 7f31d9a45700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d406bef0 0x7f31d4149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.585+0000 7f31d9a45700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31d4159e10 0x7f31d4158710 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.585+0000 7f31d9a45700 1 --2- 172.21.15.181:0/2308014472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d40cb0e0 0x7f31d40cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.585+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 >> 172.21.15.181:0/2308014472 conn(0x7f31d40cc730 msgr2=0x7f31d4153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:22.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.585+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 shutdown_connections 2023-10-16T10:03:22.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.588+0000 7f31d9a45700 1 -- 172.21.15.181:0/2308014472 wait complete. 2023-10-16T10:03:22.599 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 16 and data pool 17 2023-10-16T10:03:22.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:22.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.730+0000 7fb38e72f700 1 Processor -- start 2023-10-16T10:03:22.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.730+0000 7fb38e72f700 1 -- start start 2023-10-16T10:03:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb38e72f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3880d0370 0x7fb3880ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb38e72f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3880cd2d0 0x7fb3880d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb38e72f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3880cc510 con 0x7fb38815b990 2023-10-16T10:03:22.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb38e72f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880d24f0 con 0x7fb3880d0370 2023-10-16T10:03:22.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb38e72f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880cc7f0 con 0x7fb3880cd2d0 2023-10-16T10:03:22.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3880d0370 0x7fb3880ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:22.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3880d0370 0x7fb3880ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47546/0 (socket says 172.21.15.181:47546) 2023-10-16T10:03:22.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb387fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb38815b990 0x7fb3880cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58622/0 (socket says 172.21.15.181:58622) 2023-10-16T10:03:22.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 -- 172.21.15.181:0/1295643413 learned_addr learned my addr 172.21.15.181:0/1295643413 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:22.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 -- 172.21.15.181:0/1295643413 >> v2:172.21.15.190:3300/0 conn(0x7fb3880cd2d0 msgr2=0x7fb3880d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:22.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb38cccc700 1 --2- 172.21.15.181:0/1295643413 >> v2:172.21.15.190:3300/0 conn(0x7fb3880cd2d0 0x7fb3880d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:22.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb386ffd700 1 -- 172.21.15.181:0/1295643413 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623221339 0 0) 0x7fb3880cc510 con 0x7fb38815b990 2023-10-16T10:03:22.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 --2- 172.21.15.181:0/1295643413 >> v2:172.21.15.190:3300/0 conn(0x7fb3880cd2d0 0x7fb3880d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 -- 172.21.15.181:0/1295643413 >> v1:172.21.15.181:6789/0 conn(0x7fb38815b990 legacy=0x7fb3880cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 -- 172.21.15.181:0/1295643413 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880ccad0 con 0x7fb3880d0370 2023-10-16T10:03:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb38cccc700 1 --2- 172.21.15.181:0/1295643413 >> v2:172.21.15.190:3300/0 conn(0x7fb3880cd2d0 0x7fb3880d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb3877fe700 1 --2- 172.21.15.181:0/1295643413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880d0370 0x7fb3880ce8f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb3880cbc60 tx=0x7fb37c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7ba8937906093b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb386ffd700 1 -- 172.21.15.181:0/1295643413 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00b870 con 0x7fb3880d0370 2023-10-16T10:03:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb386ffd700 1 -- 172.21.15.181:0/1295643413 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00b9d0 con 0x7fb3880d0370 2023-10-16T10:03:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.731+0000 7fb386ffd700 1 -- 172.21.15.181:0/1295643413 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00bd60 con 0x7fb3880d0370 2023-10-16T10:03:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 -- 172.21.15.181:0/1295643413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880d0370 msgr2=0x7fb3880ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1295643413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880d0370 0x7fb3880ce8f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb3880cbc60 tx=0x7fb37c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 -- 172.21.15.181:0/1295643413 shutdown_connections 2023-10-16T10:03:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1295643413 >> v2:172.21.15.190:3300/0 conn(0x7fb3880cd2d0 0x7fb3880d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1295643413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880d0370 0x7fb3880ce8f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 -- 172.21.15.181:0/1295643413 >> 172.21.15.181:0/1295643413 conn(0x7fb38809a9a0 msgr2=0x7fb3880d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 -- 172.21.15.181:0/1295643413 shutdown_connections 2023-10-16T10:03:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.732+0000 7fb38e72f700 1 -- 172.21.15.181:0/1295643413 wait complete. 2023-10-16T10:03:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.733+0000 7fb38e72f700 1 Processor -- start 2023-10-16T10:03:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.733+0000 7fb38e72f700 1 -- start start 2023-10-16T10:03:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb38e72f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880cd2d0 0x7fb388157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb38e72f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3880d0370 0x7fb3880c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb38e72f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb38815b990 0x7fb38815e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb38e72f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880cb140 con 0x7fb3880cd2d0 2023-10-16T10:03:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb38e72f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880b5100 con 0x7fb3880d0370 2023-10-16T10:03:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb38e72f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880ca8a0 con 0x7fb38815b990 2023-10-16T10:03:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb387fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880cd2d0 0x7fb388157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb387fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880cd2d0 0x7fb388157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47550/0 (socket says 172.21.15.181:47550) 2023-10-16T10:03:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.734+0000 7fb387fff700 1 -- 172.21.15.181:0/1614286848 learned_addr learned my addr 172.21.15.181:0/1614286848 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb3877fe700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3880d0370 0x7fb3880c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb38cccc700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb38815b990 0x7fb38815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb387fff700 1 -- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb38815b990 msgr2=0x7fb38815e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb387fff700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb38815b990 0x7fb38815e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb387fff700 1 -- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3880d0370 msgr2=0x7fb3880c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb387fff700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3880d0370 0x7fb3880c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb387fff700 1 -- 172.21.15.181:0/1614286848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c009550 con 0x7fb3880cd2d0 2023-10-16T10:03:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb3877fe700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3880d0370 0x7fb3880c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb387fff700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880cd2d0 0x7fb388157ad0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c520 tx=0x7fb37800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb384ff9700 1 -- 172.21.15.181:0/1614286848 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3780074e0 con 0x7fb3880cd2d0 2023-10-16T10:03:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb384ff9700 1 -- 172.21.15.181:0/1614286848 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb378007640 con 0x7fb3880cd2d0 2023-10-16T10:03:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3880cab80 con 0x7fb3880cd2d0 2023-10-16T10:03:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb384ff9700 1 -- 172.21.15.181:0/1614286848 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378005a50 con 0x7fb3880cd2d0 2023-10-16T10:03:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.735+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38815c240 con 0x7fb3880cd2d0 2023-10-16T10:03:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.736+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38815c0a0 con 0x7fb3880cd2d0 2023-10-16T10:03:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.736+0000 7fb384ff9700 1 -- 172.21.15.181:0/1614286848 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb378005bb0 con 0x7fb3880cd2d0 2023-10-16T10:03:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.737+0000 7fb384ff9700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fb37406b020 0x7fb37406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.737+0000 7fb384ff9700 1 -- 172.21.15.181:0/1614286848 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fb378088f70 con 0x7fb3880cd2d0 2023-10-16T10:03:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.737+0000 7fb3877fe700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fb37406b020 0x7fb37406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.737+0000 7fb3877fe700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fb37406b020 0x7fb37406d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb3881f3fe0 tx=0x7fb37c008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.741+0000 7fb384ff9700 1 -- 172.21.15.181:0/1614286848 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb38815c0a0 con 0x7fb3880cd2d0 2023-10-16T10:03:22.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.936+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb38815c0a0 con 0x7fb3880cd2d0 2023-10-16T10:03:22.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.937+0000 7fb384ff9700 1 -- 172.21.15.181:0/1614286848 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+17265 (secure 0 0 0) 0x7fb38815c0a0 con 0x7fb3880cd2d0 2023-10-16T10:03:22.938 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:22.938 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":131,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:21.971467+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:03:20.395061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:03:20.923916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:22.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fb37406b020 msgr2=0x7fb37406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fb37406b020 0x7fb37406d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb3881f3fe0 tx=0x7fb37c008030 comp rx=0 tx=0).stop 2023-10-16T10:03:22.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880cd2d0 msgr2=0x7fb388157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:22.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880cd2d0 0x7fb388157ad0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c520 tx=0x7fb37800c8e0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 shutdown_connections 2023-10-16T10:03:22.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fb37406b020 0x7fb37406d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb38815b990 0x7fb38815e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3880cd2d0 0x7fb388157ad0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 --2- 172.21.15.181:0/1614286848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3880d0370 0x7fb3880c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:22.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 >> 172.21.15.181:0/1614286848 conn(0x7fb38809a9a0 msgr2=0x7fb3880cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:22.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 shutdown_connections 2023-10-16T10:03:22.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:22.939+0000 7fb38e72f700 1 -- 172.21.15.181:0/1614286848 wait complete. 2023-10-16T10:03:22.952 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:23.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.071+0000 7f04afd9b700 1 Processor -- start 2023-10-16T10:03:23.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.072+0000 7f04afd9b700 1 -- start start 2023-10-16T10:03:23.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.072+0000 7f04afd9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04a815eab0 0x7f04a80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.072+0000 7f04afd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04a80cb0c0 0x7f04a815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.072+0000 7f04afd9b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a806ed00 con 0x7f04a815e6d0 2023-10-16T10:03:23.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.072+0000 7f04afd9b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a80d02d0 con 0x7f04a80cb0c0 2023-10-16T10:03:23.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.072+0000 7f04afd9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a815ad40 con 0x7f04a815eab0 2023-10-16T10:03:23.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04a80cb0c0 0x7f04a815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04a80cb0c0 0x7f04a815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47562/0 (socket says 172.21.15.181:47562) 2023-10-16T10:03:23.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 -- 172.21.15.181:0/981300812 learned_addr learned my addr 172.21.15.181:0/981300812 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:23.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ad336700 1 --2- 172.21.15.181:0/981300812 >> v2:172.21.15.190:3300/0 conn(0x7f04a815eab0 0x7f04a80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 -- 172.21.15.181:0/981300812 >> v2:172.21.15.190:3300/0 conn(0x7f04a815eab0 msgr2=0x7f04a80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:23.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 --2- 172.21.15.181:0/981300812 >> v2:172.21.15.190:3300/0 conn(0x7f04a815eab0 0x7f04a80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 -- 172.21.15.181:0/981300812 >> v1:172.21.15.190:6789/0 conn(0x7f04a815e6d0 legacy=0x7f04a815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:23.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 -- 172.21.15.181:0/981300812 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a806e0f0 con 0x7f04a80cb0c0 2023-10-16T10:03:23.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04ae338700 1 --2- 172.21.15.181:0/981300812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a80cb0c0 0x7f04a815a800 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f04a80714b0 tx=0x7f04a000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1c0ec9fbb116a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:23.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04acb35700 1 -- 172.21.15.181:0/981300812 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0010070 con 0x7f04a80cb0c0 2023-10-16T10:03:23.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04acb35700 1 -- 172.21.15.181:0/981300812 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a000dd50 con 0x7f04a80cb0c0 2023-10-16T10:03:23.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.073+0000 7f04acb35700 1 -- 172.21.15.181:0/981300812 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000b310 con 0x7f04a80cb0c0 2023-10-16T10:03:23.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 -- 172.21.15.181:0/981300812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a80cb0c0 msgr2=0x7f04a815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 --2- 172.21.15.181:0/981300812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a80cb0c0 0x7f04a815a800 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f04a80714b0 tx=0x7f04a000d430 comp rx=0 tx=0).stop 2023-10-16T10:03:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 -- 172.21.15.181:0/981300812 shutdown_connections 2023-10-16T10:03:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 --2- 172.21.15.181:0/981300812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a80cb0c0 0x7f04a815a800 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 --2- 172.21.15.181:0/981300812 >> v2:172.21.15.190:3300/0 conn(0x7f04a815eab0 0x7f04a80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 -- 172.21.15.181:0/981300812 >> 172.21.15.181:0/981300812 conn(0x7f04a80d04a0 msgr2=0x7f04a80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 -- 172.21.15.181:0/981300812 shutdown_connections 2023-10-16T10:03:23.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.074+0000 7f04afd9b700 1 -- 172.21.15.181:0/981300812 wait complete. 2023-10-16T10:03:23.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.075+0000 7f04afd9b700 1 Processor -- start 2023-10-16T10:03:23.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.075+0000 7f04afd9b700 1 -- start start 2023-10-16T10:03:23.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a80cb0c0 0x7f04a814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815e6d0 0x7f04a814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a815eab0 0x7f04a80c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80b62c0 con 0x7f04a815e6d0 2023-10-16T10:03:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c5a90 con 0x7f04a815eab0 2023-10-16T10:03:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c51a0 con 0x7f04a80cb0c0 2023-10-16T10:03:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815e6d0 0x7f04a814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815e6d0 0x7f04a814df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47566/0 (socket says 172.21.15.181:47566) 2023-10-16T10:03:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 -- 172.21.15.181:0/3672835030 learned_addr learned my addr 172.21.15.181:0/3672835030 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04adb37700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a80cb0c0 0x7f04a814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ae338700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a815eab0 0x7f04a80c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 -- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a80cb0c0 msgr2=0x7f04a814e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a80cb0c0 0x7f04a814e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 -- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a815eab0 msgr2=0x7f04a80c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a815eab0 0x7f04a80c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 -- 172.21.15.181:0/3672835030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a000bd40 con 0x7f04a815e6d0 2023-10-16T10:03:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04adb37700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a80cb0c0 0x7f04a814e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ae338700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a815eab0 0x7f04a80c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04ad336700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815e6d0 0x7f04a814df60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f04a400e810 tx=0x7f04a400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f049e7fc700 1 -- 172.21.15.181:0/3672835030 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4013610 con 0x7f04a815e6d0 2023-10-16T10:03:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f049e7fc700 1 -- 172.21.15.181:0/3672835030 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a4013770 con 0x7f04a815e6d0 2023-10-16T10:03:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a80b5fe0 con 0x7f04a815e6d0 2023-10-16T10:03:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f049e7fc700 1 -- 172.21.15.181:0/3672835030 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4013b20 con 0x7f04a815e6d0 2023-10-16T10:03:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.076+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a80c5720 con 0x7f04a815e6d0 2023-10-16T10:03:23.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.077+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a8067d20 con 0x7f04a815e6d0 2023-10-16T10:03:23.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.078+0000 7f049e7fc700 1 -- 172.21.15.181:0/3672835030 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f04a4013c80 con 0x7f04a815e6d0 2023-10-16T10:03:23.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.078+0000 7f049e7fc700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f048c06b020 0x7f048c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.078+0000 7f049e7fc700 1 -- 172.21.15.181:0/3672835030 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f04a4089730 con 0x7f04a815e6d0 2023-10-16T10:03:23.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.078+0000 7f04adb37700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f048c06b020 0x7f048c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.079+0000 7f04adb37700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f048c06b020 0x7f048c06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04a81f2400 tx=0x7f0498008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:23.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.083+0000 7f049e7fc700 1 -- 172.21.15.181:0/3672835030 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f04a4054360 con 0x7f04a815e6d0 2023-10-16T10:03:23.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.275+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04a80cb610 con 0x7f04a815e6d0 2023-10-16T10:03:23.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.276+0000 7f049e7fc700 1 -- 172.21.15.181:0/3672835030 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+17265 (secure 0 0 0) 0x7f04a4057980 con 0x7f04a815e6d0 2023-10-16T10:03:23.277 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:23.277 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":131,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:21.971467+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:03:20.395061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:03:20.923916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:23.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f048c06b020 msgr2=0x7f048c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f048c06b020 0x7f048c06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04a81f2400 tx=0x7f0498008040 comp rx=0 tx=0).stop 2023-10-16T10:03:23.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815e6d0 msgr2=0x7f04a814df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815e6d0 0x7f04a814df60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f04a400e810 tx=0x7f04a400ebd0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 shutdown_connections 2023-10-16T10:03:23.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f048c06b020 0x7f048c06d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04a80cb0c0 0x7f04a814e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a815eab0 0x7f04a80c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 --2- 172.21.15.181:0/3672835030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a815e6d0 0x7f04a814df60 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 >> 172.21.15.181:0/3672835030 conn(0x7f04a80d04a0 msgr2=0x7f04a80adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:23.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 shutdown_connections 2023-10-16T10:03:23.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.278+0000 7f04afd9b700 1 -- 172.21.15.181:0/3672835030 wait complete. 2023-10-16T10:03:23.291 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:03:23.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.414+0000 7f127521e700 1 Processor -- start 2023-10-16T10:03:23.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.414+0000 7f127521e700 1 -- start start 2023-10-16T10:03:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f127521e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12700b2010 0x7f12700cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f127521e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1270071be0 0x7f12700b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f127521e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f127006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f127521e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12700ce1f0 con 0x7f12700b2010 2023-10-16T10:03:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f127521e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1270072090 con 0x7f1270071be0 2023-10-16T10:03:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f127521e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12700b2930 con 0x7f127006f3b0 2023-10-16T10:03:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12700b2010 0x7f12700cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12700b2010 0x7f12700cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47572/0 (socket says 172.21.15.181:47572) 2023-10-16T10:03:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 -- 172.21.15.181:0/3980479326 learned_addr learned my addr 172.21.15.181:0/3980479326 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126f59e700 1 --2- 172.21.15.181:0/3980479326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f127006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126e59c700 1 --2- 172.21.15.181:0/3980479326 >> v2:172.21.15.190:3300/0 conn(0x7f1270071be0 0x7f12700b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 -- 172.21.15.181:0/3980479326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 msgr2=0x7f127006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 --2- 172.21.15.181:0/3980479326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f127006f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 -- 172.21.15.181:0/3980479326 >> v2:172.21.15.190:3300/0 conn(0x7f1270071be0 msgr2=0x7f12700b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:23.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 --2- 172.21.15.181:0/3980479326 >> v2:172.21.15.190:3300/0 conn(0x7f1270071be0 0x7f12700b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 -- 172.21.15.181:0/3980479326 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700ccda0 con 0x7f12700b2010 2023-10-16T10:03:23.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126f59e700 1 --2- 172.21.15.181:0/3980479326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f127006f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:23.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126e59c700 1 --2- 172.21.15.181:0/3980479326 >> v2:172.21.15.190:3300/0 conn(0x7f1270071be0 0x7f12700b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:23.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126ed9d700 1 --2- 172.21.15.181:0/3980479326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f12700cc6e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f127015e1c0 tx=0x7f1260009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dc342182578e8d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:23.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126dd9b700 1 -- 172.21.15.181:0/3980479326 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126000b870 con 0x7f12700b2010 2023-10-16T10:03:23.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126dd9b700 1 -- 172.21.15.181:0/3980479326 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126000b9d0 con 0x7f12700b2010 2023-10-16T10:03:23.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.415+0000 7f126dd9b700 1 -- 172.21.15.181:0/3980479326 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126000bd60 con 0x7f12700b2010 2023-10-16T10:03:23.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 -- 172.21.15.181:0/3980479326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 msgr2=0x7f12700cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 --2- 172.21.15.181:0/3980479326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f12700cc6e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f127015e1c0 tx=0x7f1260009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 -- 172.21.15.181:0/3980479326 shutdown_connections 2023-10-16T10:03:23.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 --2- 172.21.15.181:0/3980479326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f12700cc6e0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 --2- 172.21.15.181:0/3980479326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f127006f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 --2- 172.21.15.181:0/3980479326 >> v2:172.21.15.190:3300/0 conn(0x7f1270071be0 0x7f12700b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 -- 172.21.15.181:0/3980479326 >> 172.21.15.181:0/3980479326 conn(0x7f12700cd010 msgr2=0x7f12700cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:23.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 -- 172.21.15.181:0/3980479326 shutdown_connections 2023-10-16T10:03:23.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.416+0000 7f127521e700 1 -- 172.21.15.181:0/3980479326 wait complete. 2023-10-16T10:03:23.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 Processor -- start 2023-10-16T10:03:23.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 -- start start 2023-10-16T10:03:23.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f1270152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1270071be0 0x7f1270148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f1270154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270148dc0 con 0x7f12700b2010 2023-10-16T10:03:23.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127015c110 con 0x7f1270071be0 2023-10-16T10:03:23.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f127521e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127015c3f0 con 0x7f127006f3b0 2023-10-16T10:03:23.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f126f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f1270154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f126f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f1270154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47582/0 (socket says 172.21.15.181:47582) 2023-10-16T10:03:23.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f126f59e700 1 -- 172.21.15.181:0/598758966 learned_addr learned my addr 172.21.15.181:0/598758966 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:23.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f126ed9d700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f1270152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.418+0000 7f126e59c700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1270071be0 0x7f1270148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f126f59e700 1 -- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 msgr2=0x7f1270152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f126f59e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f1270152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f126f59e700 1 -- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1270071be0 msgr2=0x7f1270148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:23.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f126f59e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1270071be0 0x7f1270148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:23.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f126f59e700 1 -- 172.21.15.181:0/598758966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1260009550 con 0x7f12700b2010 2023-10-16T10:03:23.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f126f59e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f1270154460 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1268007db0 tx=0x7f126800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:23.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f12577fe700 1 -- 172.21.15.181:0/598758966 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800dd20 con 0x7f12700b2010 2023-10-16T10:03:23.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f12577fe700 1 -- 172.21.15.181:0/598758966 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126800f040 con 0x7f12700b2010 2023-10-16T10:03:23.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127015c6d0 con 0x7f12700b2010 2023-10-16T10:03:23.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f12577fe700 1 -- 172.21.15.181:0/598758966 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800dea0 con 0x7f12700b2010 2023-10-16T10:03:23.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.419+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127015cc90 con 0x7f12700b2010 2023-10-16T10:03:23.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.420+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1270072aa0 con 0x7f12700b2010 2023-10-16T10:03:23.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.420+0000 7f12577fe700 1 -- 172.21.15.181:0/598758966 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1268003c60 con 0x7f12700b2010 2023-10-16T10:03:23.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.421+0000 7f12577fe700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f125c06b020 0x7f125c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:23.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.421+0000 7f12577fe700 1 -- 172.21.15.181:0/598758966 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f1268089970 con 0x7f12700b2010 2023-10-16T10:03:23.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.421+0000 7f126ed9d700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f125c06b020 0x7f125c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:23.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.421+0000 7f126ed9d700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f125c06b020 0x7f125c06d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1260014040 tx=0x7f1260008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:23.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.425+0000 7f12577fe700 1 -- 172.21.15.181:0/598758966 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1270072aa0 con 0x7f12700b2010 2023-10-16T10:03:23.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:23.669+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f127015ceb0 con 0x7f12700b2010 2023-10-16T10:03:24.557 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-10-16T10:03:24.556+0000 7f27a4493700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:03:24.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.589+0000 7f12577fe700 1 -- 172.21.15.181:0/598758966 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f126801a0b0 con 0x7f12700b2010 2023-10-16T10:03:24.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.593+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f125c06b020 msgr2=0x7f125c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:24.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.593+0000 7f127521e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f125c06b020 0x7f125c06d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1260014040 tx=0x7f1260008030 comp rx=0 tx=0).stop 2023-10-16T10:03:24.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.593+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 msgr2=0x7f1270154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:24.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f1270154460 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1268007db0 tx=0x7f126800d3b0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 shutdown_connections 2023-10-16T10:03:24.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f125c06b020 0x7f125c06d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f127006f3b0 0x7f1270152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12700b2010 0x7f1270154460 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 --2- 172.21.15.181:0/598758966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1270071be0 0x7f1270148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 >> 172.21.15.181:0/598758966 conn(0x7f12700cd010 msgr2=0x7f12700b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:24.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 shutdown_connections 2023-10-16T10:03:24.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.594+0000 7f127521e700 1 -- 172.21.15.181:0/598758966 wait complete. 2023-10-16T10:03:24.610 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:03:24.611 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:03:24.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.744+0000 7f3dfce2d700 1 Processor -- start 2023-10-16T10:03:24.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.744+0000 7f3dfce2d700 1 -- start start 2023-10-16T10:03:24.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3dfce2d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3df80cdd00 0x7f3df806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:24.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3dfce2d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3df80666a0 0x7f3df806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:24.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3dfce2d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 0x7f3df806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:24.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3dfce2d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df80ca300 con 0x7f3df80cdd00 2023-10-16T10:03:24.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3dfce2d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df806e5e0 con 0x7f3df80666a0 2023-10-16T10:03:24.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3dfce2d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80ca580 con 0x7f3df80ce0e0 2023-10-16T10:03:24.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3df80cdd00 0x7f3df806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:24.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3df80cdd00 0x7f3df806f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47586/0 (socket says 172.21.15.181:47586) 2023-10-16T10:03:24.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 -- 172.21.15.181:0/300253429 learned_addr learned my addr 172.21.15.181:0/300253429 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:24.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 -- 172.21.15.181:0/300253429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 msgr2=0x7f3df806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:24.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 --2- 172.21.15.181:0/300253429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 0x7f3df806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 -- 172.21.15.181:0/300253429 >> v2:172.21.15.190:3300/0 conn(0x7f3df80666a0 msgr2=0x7f3df806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:24.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 --2- 172.21.15.181:0/300253429 >> v2:172.21.15.190:3300/0 conn(0x7f3df80666a0 0x7f3df806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 -- 172.21.15.181:0/300253429 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80ca860 con 0x7f3df80cdd00 2023-10-16T10:03:24.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df659c700 1 --2- 172.21.15.181:0/300253429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80cdd00 0x7f3df806f5f0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3df80cc6e0 tx=0x7f3de8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c652df2fbd11ab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:24.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df559a700 1 -- 172.21.15.181:0/300253429 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800b870 con 0x7f3df80cdd00 2023-10-16T10:03:24.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df559a700 1 -- 172.21.15.181:0/300253429 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de800b9d0 con 0x7f3df80cdd00 2023-10-16T10:03:24.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.745+0000 7f3df559a700 1 -- 172.21.15.181:0/300253429 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800bd60 con 0x7f3df80cdd00 2023-10-16T10:03:24.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 -- 172.21.15.181:0/300253429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80cdd00 msgr2=0x7f3df806f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:24.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/300253429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80cdd00 0x7f3df806f5f0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3df80cc6e0 tx=0x7f3de8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 -- 172.21.15.181:0/300253429 shutdown_connections 2023-10-16T10:03:24.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/300253429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80cdd00 0x7f3df806f5f0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/300253429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 0x7f3df806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/300253429 >> v2:172.21.15.190:3300/0 conn(0x7f3df80666a0 0x7f3df806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 -- 172.21.15.181:0/300253429 >> 172.21.15.181:0/300253429 conn(0x7f3df80cc730 msgr2=0x7f3df80ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:24.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 -- 172.21.15.181:0/300253429 shutdown_connections 2023-10-16T10:03:24.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.746+0000 7f3dfce2d700 1 -- 172.21.15.181:0/300253429 wait complete. 2023-10-16T10:03:24.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.748+0000 7f3dfce2d700 1 Processor -- start 2023-10-16T10:03:24.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.748+0000 7f3dfce2d700 1 -- start start 2023-10-16T10:03:24.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80666a0 0x7f3df8157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:24.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df80cdd00 0x7f3df80c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:24.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 0x7f3df81f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:24.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df8158470 con 0x7f3df80666a0 2023-10-16T10:03:24.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df8160350 con 0x7f3df80cdd00 2023-10-16T10:03:24.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df815fd90 con 0x7f3df80ce0e0 2023-10-16T10:03:24.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80666a0 0x7f3df8157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:24.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80666a0 0x7f3df8157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47592/0 (socket says 172.21.15.181:47592) 2023-10-16T10:03:24.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 -- 172.21.15.181:0/2489493267 learned_addr learned my addr 172.21.15.181:0/2489493267 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:24.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 -- 172.21.15.181:0/2489493267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 msgr2=0x7f3df81f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:24.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 0x7f3df81f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 -- 172.21.15.181:0/2489493267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df80cdd00 msgr2=0x7f3df80c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:24.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df80cdd00 0x7f3df80c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:24.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 -- 172.21.15.181:0/2489493267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009550 con 0x7f3df80666a0 2023-10-16T10:03:24.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3df659c700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80666a0 0x7f3df8157f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3df8070f70 tx=0x7f3de8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:24.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3de6ffd700 1 -- 172.21.15.181:0/2489493267 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8004310 con 0x7f3df80666a0 2023-10-16T10:03:24.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3de6ffd700 1 -- 172.21.15.181:0/2489493267 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de801f070 con 0x7f3df80666a0 2023-10-16T10:03:24.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df8160070 con 0x7f3df80666a0 2023-10-16T10:03:24.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3de6ffd700 1 -- 172.21.15.181:0/2489493267 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80036a0 con 0x7f3df80666a0 2023-10-16T10:03:24.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.749+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df81f6960 con 0x7f3df80666a0 2023-10-16T10:03:24.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.750+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df81f6b20 con 0x7f3df80666a0 2023-10-16T10:03:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.751+0000 7f3de6ffd700 1 -- 172.21.15.181:0/2489493267 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3de8021030 con 0x7f3df80666a0 2023-10-16T10:03:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.751+0000 7f3de6ffd700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f3dd806b020 0x7f3dd806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.751+0000 7f3de6ffd700 1 -- 172.21.15.181:0/2489493267 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f3de808ae00 con 0x7f3df80666a0 2023-10-16T10:03:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.751+0000 7f3df5d9b700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f3dd806b020 0x7f3dd806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:24.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.752+0000 7f3df5d9b700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f3dd806b020 0x7f3dd806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3df815c200 tx=0x7f3dec006d20 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:24.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.756+0000 7f3de6ffd700 1 -- 172.21.15.181:0/2489493267 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3de8055d00 con 0x7f3df80666a0 2023-10-16T10:03:24.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:24.995+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3df8071940 con 0x7f3df80666a0 2023-10-16T10:03:25.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.591+0000 7f3de6ffd700 1 -- 172.21.15.181:0/2489493267 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3de80150c0 con 0x7f3df80666a0 2023-10-16T10:03:25.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f3dd806b020 msgr2=0x7f3dd806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f3dd806b020 0x7f3dd806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3df815c200 tx=0x7f3dec006d20 comp rx=0 tx=0).stop 2023-10-16T10:03:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80666a0 msgr2=0x7f3df8157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80666a0 0x7f3df8157f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3df8070f70 tx=0x7f3de8004750 comp rx=0 tx=0).stop 2023-10-16T10:03:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 shutdown_connections 2023-10-16T10:03:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f3dd806b020 0x7f3dd806d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3df80ce0e0 0x7f3df81f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df80666a0 0x7f3df8157f30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 --2- 172.21.15.181:0/2489493267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df80cdd00 0x7f3df80c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 >> 172.21.15.181:0/2489493267 conn(0x7f3df80cc730 msgr2=0x7f3df8161280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 shutdown_connections 2023-10-16T10:03:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.594+0000 7f3dfce2d700 1 -- 172.21.15.181:0/2489493267 wait complete. 2023-10-16T10:03:25.610 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:03:25.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:25.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.732+0000 7f4146d40700 1 Processor -- start 2023-10-16T10:03:25.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.733+0000 7f4146d40700 1 -- start start 2023-10-16T10:03:25.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.733+0000 7f4146d40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414015b990 0x7f41400ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:25.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.733+0000 7f4146d40700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f41400d0370 0x7f41400d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:25.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.733+0000 7f4146d40700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41400cc510 con 0x7f41400cd2d0 2023-10-16T10:03:25.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.733+0000 7f4146d40700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41400d24f0 con 0x7f41400d0370 2023-10-16T10:03:25.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.733+0000 7f4146d40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400cc7f0 con 0x7f414015b990 2023-10-16T10:03:25.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414015b990 0x7f41400ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:25.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414015b990 0x7f41400ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:44626/0 (socket says 172.21.15.181:44626) 2023-10-16T10:03:25.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f41452dd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f41400d0370 0x7f41400d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:25.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 -- 172.21.15.181:0/3009565764 learned_addr learned my addr 172.21.15.181:0/3009565764 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:25.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 -- 172.21.15.181:0/3009565764 >> v2:172.21.15.190:3300/0 conn(0x7f41400d0370 msgr2=0x7f41400d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:25.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 --2- 172.21.15.181:0/3009565764 >> v2:172.21.15.190:3300/0 conn(0x7f41400d0370 0x7f41400d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 -- 172.21.15.181:0/3009565764 >> v1:172.21.15.190:6789/0 conn(0x7f41400cd2d0 legacy=0x7f41400cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 -- 172.21.15.181:0/3009565764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41400ccad0 con 0x7f414015b990 2023-10-16T10:03:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f41452dd700 1 --2- 172.21.15.181:0/3009565764 >> v2:172.21.15.190:3300/0 conn(0x7f41400d0370 0x7f41400d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f4144adc700 1 --2- 172.21.15.181:0/3009565764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414015b990 0x7f41400ce8f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4134006fd0 tx=0x7f4134009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f742d23aaaeeabc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.734+0000 7f413f7fe700 1 -- 172.21.15.181:0/3009565764 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134005e30 con 0x7f414015b990 2023-10-16T10:03:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f413f7fe700 1 -- 172.21.15.181:0/3009565764 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4134004030 con 0x7f414015b990 2023-10-16T10:03:25.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f413f7fe700 1 -- 172.21.15.181:0/3009565764 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340043e0 con 0x7f414015b990 2023-10-16T10:03:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 -- 172.21.15.181:0/3009565764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414015b990 msgr2=0x7f41400ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 --2- 172.21.15.181:0/3009565764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414015b990 0x7f41400ce8f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4134006fd0 tx=0x7f4134009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 -- 172.21.15.181:0/3009565764 shutdown_connections 2023-10-16T10:03:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 --2- 172.21.15.181:0/3009565764 >> v2:172.21.15.190:3300/0 conn(0x7f41400d0370 0x7f41400d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 --2- 172.21.15.181:0/3009565764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f414015b990 0x7f41400ce8f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 -- 172.21.15.181:0/3009565764 >> 172.21.15.181:0/3009565764 conn(0x7f414009a9a0 msgr2=0x7f41400d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 -- 172.21.15.181:0/3009565764 shutdown_connections 2023-10-16T10:03:25.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.735+0000 7f4146d40700 1 -- 172.21.15.181:0/3009565764 wait complete. 2023-10-16T10:03:25.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.737+0000 7f4146d40700 1 Processor -- start 2023-10-16T10:03:25.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.737+0000 7f4146d40700 1 -- start start 2023-10-16T10:03:25.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4146d40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41400cd2d0 0x7f414015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:25.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4146d40700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400d0370 0x7f41401f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:25.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4146d40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414015b990 0x7f4140155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:25.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4146d40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401f2390 con 0x7f414015b990 2023-10-16T10:03:25.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4146d40700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414015e3e0 con 0x7f41400d0370 2023-10-16T10:03:25.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4146d40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41401f4b80 con 0x7f41400cd2d0 2023-10-16T10:03:25.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414015b990 0x7f4140155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:25.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414015b990 0x7f4140155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47610/0 (socket says 172.21.15.181:47610) 2023-10-16T10:03:25.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 -- 172.21.15.181:0/3189279682 learned_addr learned my addr 172.21.15.181:0/3189279682 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:25.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4144adc700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41400cd2d0 0x7f414015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:25.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f413ffff700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400d0370 0x7f41401f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:25.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 -- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41400cd2d0 msgr2=0x7f414015dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41400cd2d0 0x7f414015dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 -- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400d0370 msgr2=0x7f41401f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400d0370 0x7f41401f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 -- 172.21.15.181:0/3189279682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4138009a70 con 0x7f414015b990 2023-10-16T10:03:25.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f4144adc700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41400cd2d0 0x7f414015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:25.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f413ffff700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400d0370 0x7f41401f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:25.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f41452dd700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414015b990 0x7f4140155bf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f413800efc0 tx=0x7f413800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:25.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f413d7fa700 1 -- 172.21.15.181:0/3189279682 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413800cae0 con 0x7f414015b990 2023-10-16T10:03:25.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.738+0000 7f413d7fa700 1 -- 172.21.15.181:0/3189279682 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413800f040 con 0x7f414015b990 2023-10-16T10:03:25.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.739+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4134009500 con 0x7f414015b990 2023-10-16T10:03:25.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.739+0000 7f413d7fa700 1 -- 172.21.15.181:0/3189279682 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413800cea0 con 0x7f414015b990 2023-10-16T10:03:25.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.739+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401f1af0 con 0x7f414015b990 2023-10-16T10:03:25.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.739+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4140067d20 con 0x7f414015b990 2023-10-16T10:03:25.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.740+0000 7f413d7fa700 1 -- 172.21.15.181:0/3189279682 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4138014480 con 0x7f414015b990 2023-10-16T10:03:25.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.740+0000 7f413d7fa700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f412406b020 0x7f412406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:25.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.740+0000 7f413d7fa700 1 -- 172.21.15.181:0/3189279682 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f413808a0c0 con 0x7f414015b990 2023-10-16T10:03:25.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.740+0000 7f4144adc700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f412406b020 0x7f412406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:25.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.741+0000 7f4144adc700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f412406b020 0x7f412406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4134006fd0 tx=0x7f4134008070 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:25.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.745+0000 7f413d7fa700 1 -- 172.21.15.181:0/3189279682 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4138054d70 con 0x7f414015b990 2023-10-16T10:03:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.979+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41401f4cc0 con 0x7f414015b990 2023-10-16T10:03:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.980+0000 7f413d7fa700 1 -- 172.21.15.181:0/3189279682 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f4138058390 con 0x7f414015b990 2023-10-16T10:03:25.981 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:25.981 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2508916691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2508916691},{"type":"v1","addr":"172.21.15.181:6836","nonce":2508916691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7687,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1303363384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1303363384},{"type":"v1","addr":"172.21.15.190:6834","nonce":1303363384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/300739418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":300739418},{"type":"v1","addr":"172.21.15.181:6837","nonce":300739418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:03:21.971792+0000","modified":"2023-10-16T10:03:24.995974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1716409851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1716409851},{"type":"v1","addr":"172.21.15.190:6835","nonce":1716409851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:03:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f412406b020 msgr2=0x7f412406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f412406b020 0x7f412406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4134006fd0 tx=0x7f4134008070 comp rx=0 tx=0).stop 2023-10-16T10:03:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414015b990 msgr2=0x7f4140155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414015b990 0x7f4140155bf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f413800efc0 tx=0x7f413800c3a0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 shutdown_connections 2023-10-16T10:03:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f412406b020 0x7f412406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41400cd2d0 0x7f414015dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f414015b990 0x7f4140155bf0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 --2- 172.21.15.181:0/3189279682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41400d0370 0x7f41401f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 >> 172.21.15.181:0/3189279682 conn(0x7f414009a9a0 msgr2=0x7f41400cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 shutdown_connections 2023-10-16T10:03:25.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:25.982+0000 7f4146d40700 1 -- 172.21.15.181:0/3189279682 wait complete. 2023-10-16T10:03:25.986 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2023-10-16T10:03:25.995 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.114+0000 7fa83e273700 1 Processor -- start 2023-10-16T10:03:26.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.114+0000 7fa83e273700 1 -- start start 2023-10-16T10:03:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.115+0000 7fa83e273700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83815dfd0 0x7fa8380af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.115+0000 7fa83e273700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa8380af450 0x7fa83806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.115+0000 7fa83e273700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83806f870 con 0x7fa83806db80 2023-10-16T10:03:26.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.115+0000 7fa83e273700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380d1160 con 0x7fa8380af450 2023-10-16T10:03:26.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.115+0000 7fa83e273700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380cc8e0 con 0x7fa83815dfd0 2023-10-16T10:03:26.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.115+0000 7fa83c810700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa83806db80 0x7fa8380cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58696/0 (socket says 172.21.15.181:58696) 2023-10-16T10:03:26.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.115+0000 7fa83c810700 1 -- 172.21.15.181:0/11015363 learned_addr learned my addr 172.21.15.181:0/11015363 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:26.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa8377fe700 1 --2- 172.21.15.181:0/11015363 >> v2:172.21.15.190:3300/0 conn(0x7fa8380af450 0x7fa83806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa837fff700 1 --2- 172.21.15.181:0/11015363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83815dfd0 0x7fa8380af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa836ffd700 1 -- 172.21.15.181:0/11015363 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912135802 0 0) 0x7fa83806f870 con 0x7fa83806db80 2023-10-16T10:03:26.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa836ffd700 1 -- 172.21.15.181:0/11015363 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8200035b0 con 0x7fa83806db80 2023-10-16T10:03:26.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa837fff700 1 -- 172.21.15.181:0/11015363 >> v2:172.21.15.190:3300/0 conn(0x7fa8380af450 msgr2=0x7fa83806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:26.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa837fff700 1 --2- 172.21.15.181:0/11015363 >> v2:172.21.15.190:3300/0 conn(0x7fa8380af450 0x7fa83806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa837fff700 1 -- 172.21.15.181:0/11015363 >> v1:172.21.15.181:6789/0 conn(0x7fa83806db80 legacy=0x7fa8380cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa837fff700 1 -- 172.21.15.181:0/11015363 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8380ccbc0 con 0x7fa83815dfd0 2023-10-16T10:03:26.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa837fff700 1 --2- 172.21.15.181:0/11015363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83815dfd0 0x7fa8380af9a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009820 tx=0x7fa82c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35020cfcdf3df1ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa836ffd700 1 -- 172.21.15.181:0/11015363 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c005e30 con 0x7fa83815dfd0 2023-10-16T10:03:26.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa836ffd700 1 -- 172.21.15.181:0/11015363 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c006030 con 0x7fa83815dfd0 2023-10-16T10:03:26.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.116+0000 7fa836ffd700 1 -- 172.21.15.181:0/11015363 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0063c0 con 0x7fa83815dfd0 2023-10-16T10:03:26.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 -- 172.21.15.181:0/11015363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83815dfd0 msgr2=0x7fa8380af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 --2- 172.21.15.181:0/11015363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83815dfd0 0x7fa8380af9a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009820 tx=0x7fa82c009b30 comp rx=0 tx=0).stop 2023-10-16T10:03:26.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 -- 172.21.15.181:0/11015363 shutdown_connections 2023-10-16T10:03:26.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 --2- 172.21.15.181:0/11015363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83815dfd0 0x7fa8380af9a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 --2- 172.21.15.181:0/11015363 >> v2:172.21.15.190:3300/0 conn(0x7fa8380af450 0x7fa83806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 -- 172.21.15.181:0/11015363 >> 172.21.15.181:0/11015363 conn(0x7fa8380d04a0 msgr2=0x7fa8380d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:26.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 -- 172.21.15.181:0/11015363 shutdown_connections 2023-10-16T10:03:26.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.117+0000 7fa83e273700 1 -- 172.21.15.181:0/11015363 wait complete. 2023-10-16T10:03:26.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.118+0000 7fa83e273700 1 Processor -- start 2023-10-16T10:03:26.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.118+0000 7fa83e273700 1 -- start start 2023-10-16T10:03:26.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83e273700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83806db80 0x7fa838158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83e273700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8380af450 0x7fa8380b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83e273700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83815dfd0 0x7fa8381f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83e273700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380b2d40 con 0x7fa83806db80 2023-10-16T10:03:26.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83e273700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381f6000 con 0x7fa83815dfd0 2023-10-16T10:03:26.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83e273700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380c7570 con 0x7fa8380af450 2023-10-16T10:03:26.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83806db80 0x7fa838158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83806db80 0x7fa838158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47622/0 (socket says 172.21.15.181:47622) 2023-10-16T10:03:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 -- 172.21.15.181:0/280640979 learned_addr learned my addr 172.21.15.181:0/280640979 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa8377fe700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8380af450 0x7fa8380b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83c810700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83815dfd0 0x7fa8381f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 -- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8380af450 msgr2=0x7fa8380b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8380af450 0x7fa8380b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 -- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83815dfd0 msgr2=0x7fa8381f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83815dfd0 0x7fa8381f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 -- 172.21.15.181:0/280640979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c009430 con 0x7fa83806db80 2023-10-16T10:03:26.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa83c810700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83815dfd0 0x7fa8381f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:26.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa8377fe700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8380af450 0x7fa8380b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:26.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.119+0000 7fa837fff700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83806db80 0x7fa838158520 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.120+0000 7fa834ff9700 1 -- 172.21.15.181:0/280640979 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0065b0 con 0x7fa83806db80 2023-10-16T10:03:26.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.120+0000 7fa834ff9700 1 -- 172.21.15.181:0/280640979 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c006710 con 0x7fa83806db80 2023-10-16T10:03:26.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.120+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8380c7850 con 0x7fa83806db80 2023-10-16T10:03:26.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.120+0000 7fa834ff9700 1 -- 172.21.15.181:0/280640979 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0036a0 con 0x7fa83806db80 2023-10-16T10:03:26.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.120+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8380ca0e0 con 0x7fa83806db80 2023-10-16T10:03:26.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.121+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8380c9c60 con 0x7fa83806db80 2023-10-16T10:03:26.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.121+0000 7fa834ff9700 1 -- 172.21.15.181:0/280640979 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa82c022070 con 0x7fa83806db80 2023-10-16T10:03:26.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.121+0000 7fa834ff9700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa82406b020 0x7fa82406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.121+0000 7fa834ff9700 1 -- 172.21.15.181:0/280640979 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fa82c08afe0 con 0x7fa83806db80 2023-10-16T10:03:26.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.122+0000 7fa8377fe700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa82406b020 0x7fa82406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.122+0000 7fa8377fe700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa82406b020 0x7fa82406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa828009b60 tx=0x7fa828008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.126+0000 7fa834ff9700 1 -- 172.21.15.181:0/280640979 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa8380c9c60 con 0x7fa83806db80 2023-10-16T10:03:26.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.317+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8380c9c60 con 0x7fa83806db80 2023-10-16T10:03:26.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.319+0000 7fa834ff9700 1 -- 172.21.15.181:0/280640979 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+17252 (secure 0 0 0) 0x7fa8380c9c60 con 0x7fa83806db80 2023-10-16T10:03:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:26.321 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":132,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:23.577542+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:03:20.395061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:03:20.923916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:26.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa82406b020 msgr2=0x7fa82406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa82406b020 0x7fa82406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa828009b60 tx=0x7fa828008040 comp rx=0 tx=0).stop 2023-10-16T10:03:26.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83806db80 msgr2=0x7fa838158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83806db80 0x7fa838158520 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c004750 comp rx=0 tx=0).stop 2023-10-16T10:03:26.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 shutdown_connections 2023-10-16T10:03:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa83815dfd0 0x7fa8381f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83806db80 0x7fa838158520 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa82406b020 0x7fa82406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 --2- 172.21.15.181:0/280640979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8380af450 0x7fa8380b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 >> 172.21.15.181:0/280640979 conn(0x7fa8380d04a0 msgr2=0x7fa8380cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 shutdown_connections 2023-10-16T10:03:26.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.321+0000 7fa83e273700 1 -- 172.21.15.181:0/280640979 wait complete. 2023-10-16T10:03:26.334 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:03:26.335 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:03:26.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.455+0000 7f38ed4f6700 1 Processor -- start 2023-10-16T10:03:26.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.455+0000 7f38ed4f6700 1 -- start start 2023-10-16T10:03:26.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38ed4f6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f38e80af450 0x7f38e80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38ed4f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38e80cd470 0x7f38e80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38ed4f6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e806f870 con 0x7f38e815dfd0 2023-10-16T10:03:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38ed4f6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e80d1160 con 0x7f38e80cd470 2023-10-16T10:03:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38ed4f6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e80cc8e0 con 0x7f38e80af450 2023-10-16T10:03:26.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38e80cd470 0x7f38e80cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38e80cd470 0x7f38e80cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47632/0 (socket says 172.21.15.181:47632) 2023-10-16T10:03:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 -- 172.21.15.181:0/914631240 learned_addr learned my addr 172.21.15.181:0/914631240 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 -- 172.21.15.181:0/914631240 >> v2:172.21.15.190:3300/0 conn(0x7f38e80af450 msgr2=0x7f38e80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:26.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e67fc700 1 --2- 172.21.15.181:0/914631240 >> v2:172.21.15.190:3300/0 conn(0x7f38e80af450 0x7f38e80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 --2- 172.21.15.181:0/914631240 >> v2:172.21.15.190:3300/0 conn(0x7f38e80af450 0x7f38e80af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 -- 172.21.15.181:0/914631240 >> v1:172.21.15.190:6789/0 conn(0x7f38e815dfd0 legacy=0x7f38e806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 -- 172.21.15.181:0/914631240 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80ccbc0 con 0x7f38e80cd470 2023-10-16T10:03:26.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.456+0000 7f38e77fe700 1 --2- 172.21.15.181:0/914631240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80cd920 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f38e80b3070 tx=0x7f38d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73542298e40809c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38e5ffb700 1 -- 172.21.15.181:0/914631240 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8004230 con 0x7f38e80cd470 2023-10-16T10:03:26.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38e5ffb700 1 -- 172.21.15.181:0/914631240 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8004390 con 0x7f38e80cd470 2023-10-16T10:03:26.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38e5ffb700 1 -- 172.21.15.181:0/914631240 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d80036a0 con 0x7f38e80cd470 2023-10-16T10:03:26.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 -- 172.21.15.181:0/914631240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 msgr2=0x7f38e80cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/914631240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80cd920 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f38e80b3070 tx=0x7f38d8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 -- 172.21.15.181:0/914631240 shutdown_connections 2023-10-16T10:03:26.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/914631240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80cd920 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/914631240 >> v2:172.21.15.190:3300/0 conn(0x7f38e80af450 0x7f38e80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 -- 172.21.15.181:0/914631240 >> 172.21.15.181:0/914631240 conn(0x7f38e80d04a0 msgr2=0x7f38e80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:26.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 -- 172.21.15.181:0/914631240 shutdown_connections 2023-10-16T10:03:26.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.457+0000 7f38ed4f6700 1 -- 172.21.15.181:0/914631240 wait complete. 2023-10-16T10:03:26.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 Processor -- start 2023-10-16T10:03:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 -- start start 2023-10-16T10:03:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38e80af450 0x7f38e81f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38e815dfd0 0x7f38e80c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e81f61e0 con 0x7f38e80cd470 2023-10-16T10:03:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80b4d10 con 0x7f38e815dfd0 2023-10-16T10:03:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38ed4f6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80c7100 con 0x7f38e80af450 2023-10-16T10:03:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38e67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38e67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47636/0 (socket says 172.21.15.181:47636) 2023-10-16T10:03:26.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.459+0000 7f38e67fc700 1 -- 172.21.15.181:0/25374994 learned_addr learned my addr 172.21.15.181:0/25374994 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:26.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e77fe700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38e815dfd0 0x7f38e80c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e6ffd700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38e80af450 0x7f38e81f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e67fc700 1 -- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38e80af450 msgr2=0x7f38e81f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e67fc700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38e80af450 0x7f38e81f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e67fc700 1 -- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38e815dfd0 msgr2=0x7f38e80c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e67fc700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38e815dfd0 0x7f38e80c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e67fc700 1 -- 172.21.15.181:0/25374994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8009550 con 0x7f38e80cd470 2023-10-16T10:03:26.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38e67fc700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80c63b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c7f0 tx=0x7f38d000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38df7fe700 1 -- 172.21.15.181:0/25374994 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011800 con 0x7f38e80cd470 2023-10-16T10:03:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38df7fe700 1 -- 172.21.15.181:0/25374994 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0011960 con 0x7f38e80cd470 2023-10-16T10:03:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e80c6e30 con 0x7f38e80cd470 2023-10-16T10:03:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38df7fe700 1 -- 172.21.15.181:0/25374994 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011cf0 con 0x7f38e80cd470 2023-10-16T10:03:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.460+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e80b46c0 con 0x7f38e80cd470 2023-10-16T10:03:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.461+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e8067d20 con 0x7f38e80cd470 2023-10-16T10:03:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.461+0000 7f38df7fe700 1 -- 172.21.15.181:0/25374994 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f38d0018360 con 0x7f38e80cd470 2023-10-16T10:03:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.462+0000 7f38df7fe700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f38c806b020 0x7f38c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.462+0000 7f38df7fe700 1 -- 172.21.15.181:0/25374994 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f38d00899c0 con 0x7f38e80cd470 2023-10-16T10:03:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.462+0000 7f38e6ffd700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f38c806b020 0x7f38c806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.462+0000 7f38e6ffd700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f38c806b020 0x7f38c806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009f20 tx=0x7f38e00094d0 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.466+0000 7f38df7fe700 1 -- 172.21.15.181:0/25374994 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f38d0054690 con 0x7f38e80cd470 2023-10-16T10:03:26.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.655+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 --> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38e80cc590 con 0x7f38c806b020 2023-10-16T10:03:26.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.663+0000 7f38df7fe700 1 -- 172.21.15.181:0/25374994 <== mgr.8304 v2:172.21.15.181:6832/3280351398 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289936 (secure 0 0 0) 0x7f38e000cc00 con 0x7f38c806b020 2023-10-16T10:03:26.667 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f38c806b020 msgr2=0x7f38c806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f38c806b020 0x7f38c806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009f20 tx=0x7f38e00094d0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 msgr2=0x7f38e80c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80c63b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c7f0 tx=0x7f38d000cbb0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 shutdown_connections 2023-10-16T10:03:26.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f38c806b020 0x7f38c806d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38e80af450 0x7f38e81f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38e815dfd0 0x7f38e80c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 --2- 172.21.15.181:0/25374994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38e80cd470 0x7f38e80c63b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.666+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 >> 172.21.15.181:0/25374994 conn(0x7f38e80d04a0 msgr2=0x7f38e80cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.667+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 shutdown_connections 2023-10-16T10:03:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.667+0000 7f38ed4f6700 1 -- 172.21.15.181:0/25374994 wait complete. 2023-10-16T10:03:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:03:26.679 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:03:26.491600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":334212,"kb_used_data":13956,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754640508,"statfs":{"total":773094113280,"available":772751880192,"internally_reserved":0,"allocated":14290944,"data_stored":7649396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":327798328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.932734"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594777+0000","last_change":"2023-10-16T10:03:22.588516+0000","last_active":"2023-10-16T10:03:22.594777+0000","last_peered":"2023-10-16T10:03:22.594777+0000","last_clean":"2023-10-16T10:03:22.594777+0000","last_became_active":"2023-10-16T10:03:22.587709+0000","last_became_peered":"2023-10-16T10:03:22.587709+0000","last_unstale":"2023-10-16T10:03:22.594777+0000","last_undegraded":"2023-10-16T10:03:22.594777+0000","last_fullsized":"2023-10-16T10:03:22.594777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:58.233371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588892+0000","last_change":"2023-10-16T10:03:22.585643+0000","last_active":"2023-10-16T10:03:23.588892+0000","last_peered":"2023-10-16T10:03:23.588892+0000","last_clean":"2023-10-16T10:03:23.588892+0000","last_became_active":"2023-10-16T10:03:22.585142+0000","last_became_peered":"2023-10-16T10:03:22.585142+0000","last_unstale":"2023-10-16T10:03:23.588892+0000","last_undegraded":"2023-10-16T10:03:23.588892+0000","last_fullsized":"2023-10-16T10:03:23.588892+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:01.994296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591080+0000","last_change":"2023-10-16T10:03:22.595113+0000","last_active":"2023-10-16T10:03:23.591080+0000","last_peered":"2023-10-16T10:03:23.591080+0000","last_clean":"2023-10-16T10:03:23.591080+0000","last_became_active":"2023-10-16T10:03:22.594786+0000","last_became_peered":"2023-10-16T10:03:22.594786+0000","last_unstale":"2023-10-16T10:03:23.591080+0000","last_undegraded":"2023-10-16T10:03:23.591080+0000","last_fullsized":"2023-10-16T10:03:23.591080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:17.357998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596638+0000","last_change":"2023-10-16T10:03:22.587988+0000","last_active":"2023-10-16T10:03:22.596638+0000","last_peered":"2023-10-16T10:03:22.596638+0000","last_clean":"2023-10-16T10:03:22.596638+0000","last_became_active":"2023-10-16T10:03:22.587737+0000","last_became_peered":"2023-10-16T10:03:22.587737+0000","last_unstale":"2023-10-16T10:03:22.596638+0000","last_undegraded":"2023-10-16T10:03:22.596638+0000","last_fullsized":"2023-10-16T10:03:22.596638+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:49.377260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677790+0000","last_change":"2023-10-16T10:03:22.589948+0000","last_active":"2023-10-16T10:03:23.677790+0000","last_peered":"2023-10-16T10:03:23.677790+0000","last_clean":"2023-10-16T10:03:23.677790+0000","last_became_active":"2023-10-16T10:03:22.589657+0000","last_became_peered":"2023-10-16T10:03:22.589657+0000","last_unstale":"2023-10-16T10:03:23.677790+0000","last_undegraded":"2023-10-16T10:03:23.677790+0000","last_fullsized":"2023-10-16T10:03:23.677790+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:47.180693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586771+0000","last_change":"2023-10-16T10:03:22.582717+0000","last_active":"2023-10-16T10:03:23.586771+0000","last_peered":"2023-10-16T10:03:23.586771+0000","last_clean":"2023-10-16T10:03:23.586771+0000","last_became_active":"2023-10-16T10:03:22.582457+0000","last_became_peered":"2023-10-16T10:03:22.582457+0000","last_unstale":"2023-10-16T10:03:23.586771+0000","last_undegraded":"2023-10-16T10:03:23.586771+0000","last_fullsized":"2023-10-16T10:03:23.586771+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:49.377260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590773+0000","last_change":"2023-10-16T10:03:22.583166+0000","last_active":"2023-10-16T10:03:23.590773+0000","last_peered":"2023-10-16T10:03:23.590773+0000","last_clean":"2023-10-16T10:03:23.590773+0000","last_became_active":"2023-10-16T10:03:22.582710+0000","last_became_peered":"2023-10-16T10:03:22.582710+0000","last_unstale":"2023-10-16T10:03:23.590773+0000","last_undegraded":"2023-10-16T10:03:23.590773+0000","last_fullsized":"2023-10-16T10:03:23.590773+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:47.266900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590636+0000","last_change":"2023-10-16T10:03:22.594197+0000","last_active":"2023-10-16T10:03:23.590636+0000","last_peered":"2023-10-16T10:03:23.590636+0000","last_clean":"2023-10-16T10:03:23.590636+0000","last_became_active":"2023-10-16T10:03:22.593731+0000","last_became_peered":"2023-10-16T10:03:22.593731+0000","last_unstale":"2023-10-16T10:03:23.590636+0000","last_undegraded":"2023-10-16T10:03:23.590636+0000","last_fullsized":"2023-10-16T10:03:23.590636+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:39.069792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594757+0000","last_change":"2023-10-16T10:03:22.588000+0000","last_active":"2023-10-16T10:03:22.594757+0000","last_peered":"2023-10-16T10:03:22.594757+0000","last_clean":"2023-10-16T10:03:22.594757+0000","last_became_active":"2023-10-16T10:03:22.587385+0000","last_became_peered":"2023-10-16T10:03:22.587385+0000","last_unstale":"2023-10-16T10:03:22.594757+0000","last_undegraded":"2023-10-16T10:03:22.594757+0000","last_fullsized":"2023-10-16T10:03:22.594757+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:14.140877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594512+0000","last_change":"2023-10-16T10:03:22.587449+0000","last_active":"2023-10-16T10:03:22.594512+0000","last_peered":"2023-10-16T10:03:22.594512+0000","last_clean":"2023-10-16T10:03:22.594512+0000","last_became_active":"2023-10-16T10:03:22.586557+0000","last_became_peered":"2023-10-16T10:03:22.586557+0000","last_unstale":"2023-10-16T10:03:22.594512+0000","last_undegraded":"2023-10-16T10:03:22.594512+0000","last_fullsized":"2023-10-16T10:03:22.594512+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:26.494959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588773+0000","last_change":"2023-10-16T10:03:22.579908+0000","last_active":"2023-10-16T10:03:23.588773+0000","last_peered":"2023-10-16T10:03:23.588773+0000","last_clean":"2023-10-16T10:03:23.588773+0000","last_became_active":"2023-10-16T10:03:22.579603+0000","last_became_peered":"2023-10-16T10:03:22.579603+0000","last_unstale":"2023-10-16T10:03:23.588773+0000","last_undegraded":"2023-10-16T10:03:23.588773+0000","last_fullsized":"2023-10-16T10:03:23.588773+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:33.324720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596620+0000","last_change":"2023-10-16T10:03:22.587092+0000","last_active":"2023-10-16T10:03:22.596620+0000","last_peered":"2023-10-16T10:03:22.596620+0000","last_clean":"2023-10-16T10:03:22.596620+0000","last_became_active":"2023-10-16T10:03:22.586823+0000","last_became_peered":"2023-10-16T10:03:22.586823+0000","last_unstale":"2023-10-16T10:03:22.596620+0000","last_undegraded":"2023-10-16T10:03:22.596620+0000","last_fullsized":"2023-10-16T10:03:22.596620+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:46.360051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590974+0000","last_change":"2023-10-16T10:03:22.591961+0000","last_active":"2023-10-16T10:03:23.590974+0000","last_peered":"2023-10-16T10:03:23.590974+0000","last_clean":"2023-10-16T10:03:23.590974+0000","last_became_active":"2023-10-16T10:03:22.591126+0000","last_became_peered":"2023-10-16T10:03:22.591126+0000","last_unstale":"2023-10-16T10:03:23.590974+0000","last_undegraded":"2023-10-16T10:03:23.590974+0000","last_fullsized":"2023-10-16T10:03:23.590974+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:18.574504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590749+0000","last_change":"2023-10-16T10:03:22.598184+0000","last_active":"2023-10-16T10:03:23.590749+0000","last_peered":"2023-10-16T10:03:23.590749+0000","last_clean":"2023-10-16T10:03:23.590749+0000","last_became_active":"2023-10-16T10:03:22.597903+0000","last_became_peered":"2023-10-16T10:03:22.597903+0000","last_unstale":"2023-10-16T10:03:23.590749+0000","last_undegraded":"2023-10-16T10:03:23.590749+0000","last_fullsized":"2023-10-16T10:03:23.590749+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:23.829971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586829+0000","last_change":"2023-10-16T10:03:22.580762+0000","last_active":"2023-10-16T10:03:23.586829+0000","last_peered":"2023-10-16T10:03:23.586829+0000","last_clean":"2023-10-16T10:03:23.586829+0000","last_became_active":"2023-10-16T10:03:22.580463+0000","last_became_peered":"2023-10-16T10:03:22.580463+0000","last_unstale":"2023-10-16T10:03:23.586829+0000","last_undegraded":"2023-10-16T10:03:23.586829+0000","last_fullsized":"2023-10-16T10:03:23.586829+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:35.486946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.598306+0000","last_change":"2023-10-16T10:03:22.595486+0000","last_active":"2023-10-16T10:03:22.598306+0000","last_peered":"2023-10-16T10:03:22.598306+0000","last_clean":"2023-10-16T10:03:22.598306+0000","last_became_active":"2023-10-16T10:03:22.595032+0000","last_became_peered":"2023-10-16T10:03:22.595032+0000","last_unstale":"2023-10-16T10:03:22.598306+0000","last_undegraded":"2023-10-16T10:03:22.598306+0000","last_fullsized":"2023-10-16T10:03:22.598306+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:04:35.394523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588806+0000","last_change":"2023-10-16T10:03:22.579943+0000","last_active":"2023-10-16T10:03:23.588806+0000","last_peered":"2023-10-16T10:03:23.588806+0000","last_clean":"2023-10-16T10:03:23.588806+0000","last_became_active":"2023-10-16T10:03:22.579647+0000","last_became_peered":"2023-10-16T10:03:22.579647+0000","last_unstale":"2023-10-16T10:03:23.588806+0000","last_undegraded":"2023-10-16T10:03:23.588806+0000","last_fullsized":"2023-10-16T10:03:23.588806+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:10.511924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.583018+0000","last_change":"2023-10-16T10:03:22.582433+0000","last_active":"2023-10-16T10:03:22.583018+0000","last_peered":"2023-10-16T10:03:22.583018+0000","last_clean":"2023-10-16T10:03:22.583018+0000","last_became_active":"2023-10-16T10:03:22.582075+0000","last_became_peered":"2023-10-16T10:03:22.582075+0000","last_unstale":"2023-10-16T10:03:22.583018+0000","last_undegraded":"2023-10-16T10:03:22.583018+0000","last_fullsized":"2023-10-16T10:03:22.583018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:27.301267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591326+0000","last_change":"2023-10-16T10:03:22.591369+0000","last_active":"2023-10-16T10:03:23.591326+0000","last_peered":"2023-10-16T10:03:23.591326+0000","last_clean":"2023-10-16T10:03:23.591326+0000","last_became_active":"2023-10-16T10:03:22.591094+0000","last_became_peered":"2023-10-16T10:03:22.591094+0000","last_unstale":"2023-10-16T10:03:23.591326+0000","last_undegraded":"2023-10-16T10:03:23.591326+0000","last_fullsized":"2023-10-16T10:03:23.591326+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:49.994100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588862+0000","last_change":"2023-10-16T10:03:22.592099+0000","last_active":"2023-10-16T10:03:23.588862+0000","last_peered":"2023-10-16T10:03:23.588862+0000","last_clean":"2023-10-16T10:03:23.588862+0000","last_became_active":"2023-10-16T10:03:22.591815+0000","last_became_peered":"2023-10-16T10:03:22.591815+0000","last_unstale":"2023-10-16T10:03:23.588862+0000","last_undegraded":"2023-10-16T10:03:23.588862+0000","last_fullsized":"2023-10-16T10:03:23.588862+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:13.453159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588841+0000","last_change":"2023-10-16T10:03:22.586396+0000","last_active":"2023-10-16T10:03:23.588841+0000","last_peered":"2023-10-16T10:03:23.588841+0000","last_clean":"2023-10-16T10:03:23.588841+0000","last_became_active":"2023-10-16T10:03:22.586024+0000","last_became_peered":"2023-10-16T10:03:22.586024+0000","last_unstale":"2023-10-16T10:03:23.588841+0000","last_undegraded":"2023-10-16T10:03:23.588841+0000","last_fullsized":"2023-10-16T10:03:23.588841+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:44:56.446872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589520+0000","last_change":"2023-10-16T10:03:22.584011+0000","last_active":"2023-10-16T10:03:22.589520+0000","last_peered":"2023-10-16T10:03:22.589520+0000","last_clean":"2023-10-16T10:03:22.589520+0000","last_became_active":"2023-10-16T10:03:22.583768+0000","last_became_peered":"2023-10-16T10:03:22.583768+0000","last_unstale":"2023-10-16T10:03:22.589520+0000","last_undegraded":"2023-10-16T10:03:22.589520+0000","last_fullsized":"2023-10-16T10:03:22.589520+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:23.829971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588461+0000","last_change":"2023-10-16T10:03:22.588560+0000","last_active":"2023-10-16T10:03:23.588461+0000","last_peered":"2023-10-16T10:03:23.588461+0000","last_clean":"2023-10-16T10:03:23.588461+0000","last_became_active":"2023-10-16T10:03:22.588269+0000","last_became_peered":"2023-10-16T10:03:22.588269+0000","last_unstale":"2023-10-16T10:03:23.588461+0000","last_undegraded":"2023-10-16T10:03:23.588461+0000","last_fullsized":"2023-10-16T10:03:23.588461+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:15.522323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594226+0000","last_change":"2023-10-16T10:03:22.592213+0000","last_active":"2023-10-16T10:03:22.594226+0000","last_peered":"2023-10-16T10:03:22.594226+0000","last_clean":"2023-10-16T10:03:22.594226+0000","last_became_active":"2023-10-16T10:03:22.591948+0000","last_became_peered":"2023-10-16T10:03:22.591948+0000","last_unstale":"2023-10-16T10:03:22.594226+0000","last_undegraded":"2023-10-16T10:03:22.594226+0000","last_fullsized":"2023-10-16T10:03:22.594226+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:49.642486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588951+0000","last_change":"2023-10-16T10:03:22.582511+0000","last_active":"2023-10-16T10:03:23.588951+0000","last_peered":"2023-10-16T10:03:23.588951+0000","last_clean":"2023-10-16T10:03:23.588951+0000","last_became_active":"2023-10-16T10:03:22.582200+0000","last_became_peered":"2023-10-16T10:03:22.582200+0000","last_unstale":"2023-10-16T10:03:23.588951+0000","last_undegraded":"2023-10-16T10:03:23.588951+0000","last_fullsized":"2023-10-16T10:03:23.588951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:20.241725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.593984+0000","last_change":"2023-10-16T10:03:22.587275+0000","last_active":"2023-10-16T10:03:22.593984+0000","last_peered":"2023-10-16T10:03:22.593984+0000","last_clean":"2023-10-16T10:03:22.593984+0000","last_became_active":"2023-10-16T10:03:22.586967+0000","last_became_peered":"2023-10-16T10:03:22.586967+0000","last_unstale":"2023-10-16T10:03:22.593984+0000","last_undegraded":"2023-10-16T10:03:22.593984+0000","last_fullsized":"2023-10-16T10:03:22.593984+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:09.469320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587046+0000","last_change":"2023-10-16T10:03:22.583046+0000","last_active":"2023-10-16T10:03:23.587046+0000","last_peered":"2023-10-16T10:03:23.587046+0000","last_clean":"2023-10-16T10:03:23.587046+0000","last_became_active":"2023-10-16T10:03:22.582801+0000","last_became_peered":"2023-10-16T10:03:22.582801+0000","last_unstale":"2023-10-16T10:03:23.587046+0000","last_undegraded":"2023-10-16T10:03:23.587046+0000","last_fullsized":"2023-10-16T10:03:23.587046+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:07.056578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589804+0000","last_change":"2023-10-16T10:03:22.588449+0000","last_active":"2023-10-16T10:03:22.589804+0000","last_peered":"2023-10-16T10:03:22.589804+0000","last_clean":"2023-10-16T10:03:22.589804+0000","last_became_active":"2023-10-16T10:03:22.588166+0000","last_became_peered":"2023-10-16T10:03:22.588166+0000","last_unstale":"2023-10-16T10:03:22.589804+0000","last_undegraded":"2023-10-16T10:03:22.589804+0000","last_fullsized":"2023-10-16T10:03:22.589804+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:13.231204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588928+0000","last_change":"2023-10-16T10:03:22.596965+0000","last_active":"2023-10-16T10:03:23.588928+0000","last_peered":"2023-10-16T10:03:23.588928+0000","last_clean":"2023-10-16T10:03:23.588928+0000","last_became_active":"2023-10-16T10:03:22.596493+0000","last_became_peered":"2023-10-16T10:03:22.596493+0000","last_unstale":"2023-10-16T10:03:23.588928+0000","last_undegraded":"2023-10-16T10:03:23.588928+0000","last_fullsized":"2023-10-16T10:03:23.588928+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:55.997051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591087+0000","last_change":"2023-10-16T10:03:22.595265+0000","last_active":"2023-10-16T10:03:23.591087+0000","last_peered":"2023-10-16T10:03:23.591087+0000","last_clean":"2023-10-16T10:03:23.591087+0000","last_became_active":"2023-10-16T10:03:22.594990+0000","last_became_peered":"2023-10-16T10:03:22.594990+0000","last_unstale":"2023-10-16T10:03:23.591087+0000","last_undegraded":"2023-10-16T10:03:23.591087+0000","last_fullsized":"2023-10-16T10:03:23.591087+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:25.935592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591662+0000","last_change":"2023-10-16T10:03:22.582245+0000","last_active":"2023-10-16T10:03:22.591662+0000","last_peered":"2023-10-16T10:03:22.591662+0000","last_clean":"2023-10-16T10:03:22.591662+0000","last_became_active":"2023-10-16T10:03:22.581981+0000","last_became_peered":"2023-10-16T10:03:22.581981+0000","last_unstale":"2023-10-16T10:03:22.591662+0000","last_undegraded":"2023-10-16T10:03:22.591662+0000","last_fullsized":"2023-10-16T10:03:22.591662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:05.704094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586723+0000","last_change":"2023-10-16T10:03:22.581258+0000","last_active":"2023-10-16T10:03:23.586723+0000","last_peered":"2023-10-16T10:03:23.586723+0000","last_clean":"2023-10-16T10:03:23.586723+0000","last_became_active":"2023-10-16T10:03:22.580924+0000","last_became_peered":"2023-10-16T10:03:22.580924+0000","last_unstale":"2023-10-16T10:03:23.586723+0000","last_undegraded":"2023-10-16T10:03:23.586723+0000","last_fullsized":"2023-10-16T10:03:23.586723+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:48.591325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587226+0000","last_change":"2023-10-16T10:03:22.587072+0000","last_active":"2023-10-16T10:03:23.587226+0000","last_peered":"2023-10-16T10:03:23.587226+0000","last_clean":"2023-10-16T10:03:23.587226+0000","last_became_active":"2023-10-16T10:03:22.586853+0000","last_became_peered":"2023-10-16T10:03:22.586853+0000","last_unstale":"2023-10-16T10:03:23.587226+0000","last_undegraded":"2023-10-16T10:03:23.587226+0000","last_fullsized":"2023-10-16T10:03:23.587226+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:05.704094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"131'4","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591528+0000","last_change":"2023-10-16T10:03:21.584969+0000","last_active":"2023-10-16T10:03:23.591528+0000","last_peered":"2023-10-16T10:03:23.591528+0000","last_clean":"2023-10-16T10:03:23.591528+0000","last_became_active":"2023-10-16T10:03:21.583402+0000","last_became_peered":"2023-10-16T10:03:21.583402+0000","last_unstale":"2023-10-16T10:03:23.591528+0000","last_undegraded":"2023-10-16T10:03:23.591528+0000","last_fullsized":"2023-10-16T10:03:23.591528+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:57.163494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.597580+0000","last_change":"2023-10-16T10:03:22.593817+0000","last_active":"2023-10-16T10:03:22.597580+0000","last_peered":"2023-10-16T10:03:22.597580+0000","last_clean":"2023-10-16T10:03:22.597580+0000","last_became_active":"2023-10-16T10:03:22.593536+0000","last_became_peered":"2023-10-16T10:03:22.593536+0000","last_unstale":"2023-10-16T10:03:22.597580+0000","last_undegraded":"2023-10-16T10:03:22.597580+0000","last_fullsized":"2023-10-16T10:03:22.597580+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:01.334353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"131'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591237+0000","last_change":"2023-10-16T10:03:21.589233+0000","last_active":"2023-10-16T10:03:23.591237+0000","last_peered":"2023-10-16T10:03:23.591237+0000","last_clean":"2023-10-16T10:03:23.591237+0000","last_became_active":"2023-10-16T10:03:21.588904+0000","last_became_peered":"2023-10-16T10:03:21.588904+0000","last_unstale":"2023-10-16T10:03:23.591237+0000","last_undegraded":"2023-10-16T10:03:23.591237+0000","last_fullsized":"2023-10-16T10:03:23.591237+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:07.058246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587138+0000","last_change":"2023-10-16T10:03:22.580778+0000","last_active":"2023-10-16T10:03:23.587138+0000","last_peered":"2023-10-16T10:03:23.587138+0000","last_clean":"2023-10-16T10:03:23.587138+0000","last_became_active":"2023-10-16T10:03:22.580451+0000","last_became_peered":"2023-10-16T10:03:22.580451+0000","last_unstale":"2023-10-16T10:03:23.587138+0000","last_undegraded":"2023-10-16T10:03:23.587138+0000","last_fullsized":"2023-10-16T10:03:23.587138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.864772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677810+0000","last_change":"2023-10-16T10:03:21.588947+0000","last_active":"2023-10-16T10:03:23.677810+0000","last_peered":"2023-10-16T10:03:23.677810+0000","last_clean":"2023-10-16T10:03:23.677810+0000","last_became_active":"2023-10-16T10:03:21.588675+0000","last_became_peered":"2023-10-16T10:03:21.588675+0000","last_unstale":"2023-10-16T10:03:23.677810+0000","last_undegraded":"2023-10-16T10:03:23.677810+0000","last_fullsized":"2023-10-16T10:03:23.677810+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:43.064912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591417+0000","last_change":"2023-10-16T10:03:22.592680+0000","last_active":"2023-10-16T10:03:23.591417+0000","last_peered":"2023-10-16T10:03:23.591417+0000","last_clean":"2023-10-16T10:03:23.591417+0000","last_became_active":"2023-10-16T10:03:22.592383+0000","last_became_peered":"2023-10-16T10:03:22.592383+0000","last_unstale":"2023-10-16T10:03:23.591417+0000","last_undegraded":"2023-10-16T10:03:23.591417+0000","last_fullsized":"2023-10-16T10:03:23.591417+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:41:39.434739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677721+0000","last_change":"2023-10-16T10:03:21.579770+0000","last_active":"2023-10-16T10:03:23.677721+0000","last_peered":"2023-10-16T10:03:23.677721+0000","last_clean":"2023-10-16T10:03:23.677721+0000","last_became_active":"2023-10-16T10:03:21.579493+0000","last_became_peered":"2023-10-16T10:03:21.579493+0000","last_unstale":"2023-10-16T10:03:23.677721+0000","last_undegraded":"2023-10-16T10:03:23.677721+0000","last_fullsized":"2023-10-16T10:03:23.677721+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:48.890166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591415+0000","last_change":"2023-10-16T10:03:22.582323+0000","last_active":"2023-10-16T10:03:22.591415+0000","last_peered":"2023-10-16T10:03:22.591415+0000","last_clean":"2023-10-16T10:03:22.591415+0000","last_became_active":"2023-10-16T10:03:22.582062+0000","last_became_peered":"2023-10-16T10:03:22.582062+0000","last_unstale":"2023-10-16T10:03:22.591415+0000","last_undegraded":"2023-10-16T10:03:22.591415+0000","last_fullsized":"2023-10-16T10:03:22.591415+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:37.628308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"131'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590726+0000","last_change":"2023-10-16T10:03:21.587933+0000","last_active":"2023-10-16T10:03:23.590726+0000","last_peered":"2023-10-16T10:03:23.590726+0000","last_clean":"2023-10-16T10:03:23.590726+0000","last_became_active":"2023-10-16T10:03:21.587648+0000","last_became_peered":"2023-10-16T10:03:21.587648+0000","last_unstale":"2023-10-16T10:03:23.590726+0000","last_undegraded":"2023-10-16T10:03:23.590726+0000","last_fullsized":"2023-10-16T10:03:23.590726+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:18.440663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.589007+0000","last_change":"2023-10-16T10:03:22.586053+0000","last_active":"2023-10-16T10:03:23.589007+0000","last_peered":"2023-10-16T10:03:23.589007+0000","last_clean":"2023-10-16T10:03:23.589007+0000","last_became_active":"2023-10-16T10:03:22.585237+0000","last_became_peered":"2023-10-16T10:03:22.585237+0000","last_unstale":"2023-10-16T10:03:23.589007+0000","last_undegraded":"2023-10-16T10:03:23.589007+0000","last_fullsized":"2023-10-16T10:03:23.589007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:53.304341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"131'1","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.595632+0000","last_change":"2023-10-16T10:03:21.587344+0000","last_active":"2023-10-16T10:03:22.595632+0000","last_peered":"2023-10-16T10:03:22.595632+0000","last_clean":"2023-10-16T10:03:22.595632+0000","last_became_active":"2023-10-16T10:03:21.586995+0000","last_became_peered":"2023-10-16T10:03:21.586995+0000","last_unstale":"2023-10-16T10:03:22.595632+0000","last_undegraded":"2023-10-16T10:03:22.595632+0000","last_fullsized":"2023-10-16T10:03:22.595632+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:28.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591528+0000","last_change":"2023-10-16T10:03:22.582908+0000","last_active":"2023-10-16T10:03:22.591528+0000","last_peered":"2023-10-16T10:03:22.591528+0000","last_clean":"2023-10-16T10:03:22.591528+0000","last_became_active":"2023-10-16T10:03:22.581688+0000","last_became_peered":"2023-10-16T10:03:22.581688+0000","last_unstale":"2023-10-16T10:03:22.591528+0000","last_undegraded":"2023-10-16T10:03:22.591528+0000","last_fullsized":"2023-10-16T10:03:22.591528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:55.535256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588493+0000","last_change":"2023-10-16T10:03:21.584214+0000","last_active":"2023-10-16T10:03:23.588493+0000","last_peered":"2023-10-16T10:03:23.588493+0000","last_clean":"2023-10-16T10:03:23.588493+0000","last_became_active":"2023-10-16T10:03:21.583916+0000","last_became_peered":"2023-10-16T10:03:21.583916+0000","last_unstale":"2023-10-16T10:03:23.588493+0000","last_undegraded":"2023-10-16T10:03:23.588493+0000","last_fullsized":"2023-10-16T10:03:23.588493+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:29.336207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591318+0000","last_change":"2023-10-16T10:03:22.582983+0000","last_active":"2023-10-16T10:03:23.591318+0000","last_peered":"2023-10-16T10:03:23.591318+0000","last_clean":"2023-10-16T10:03:23.591318+0000","last_became_active":"2023-10-16T10:03:22.582341+0000","last_became_peered":"2023-10-16T10:03:22.582341+0000","last_unstale":"2023-10-16T10:03:23.591318+0000","last_undegraded":"2023-10-16T10:03:23.591318+0000","last_fullsized":"2023-10-16T10:03:23.591318+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:35.573344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578624+0000","last_change":"2023-10-16T10:03:21.592508+0000","last_active":"2023-10-16T10:03:22.578624+0000","last_peered":"2023-10-16T10:03:22.578624+0000","last_clean":"2023-10-16T10:03:22.578624+0000","last_became_active":"2023-10-16T10:03:21.592231+0000","last_became_peered":"2023-10-16T10:03:21.592231+0000","last_unstale":"2023-10-16T10:03:22.578624+0000","last_undegraded":"2023-10-16T10:03:22.578624+0000","last_fullsized":"2023-10-16T10:03:22.578624+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:48:54.635819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591565+0000","last_change":"2023-10-16T10:03:22.586534+0000","last_active":"2023-10-16T10:03:23.591565+0000","last_peered":"2023-10-16T10:03:23.591565+0000","last_clean":"2023-10-16T10:03:23.591565+0000","last_became_active":"2023-10-16T10:03:22.586245+0000","last_became_peered":"2023-10-16T10:03:22.586245+0000","last_unstale":"2023-10-16T10:03:23.591565+0000","last_undegraded":"2023-10-16T10:03:23.591565+0000","last_fullsized":"2023-10-16T10:03:23.591565+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:30.872243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677749+0000","last_change":"2023-10-16T10:03:21.584838+0000","last_active":"2023-10-16T10:03:23.677749+0000","last_peered":"2023-10-16T10:03:23.677749+0000","last_clean":"2023-10-16T10:03:23.677749+0000","last_became_active":"2023-10-16T10:03:21.584336+0000","last_became_peered":"2023-10-16T10:03:21.584336+0000","last_unstale":"2023-10-16T10:03:23.677749+0000","last_undegraded":"2023-10-16T10:03:23.677749+0000","last_fullsized":"2023-10-16T10:03:23.677749+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:26.293942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.597077+0000","last_change":"2023-10-16T10:03:22.593020+0000","last_active":"2023-10-16T10:03:22.597077+0000","last_peered":"2023-10-16T10:03:22.597077+0000","last_clean":"2023-10-16T10:03:22.597077+0000","last_became_active":"2023-10-16T10:03:22.592763+0000","last_became_peered":"2023-10-16T10:03:22.592763+0000","last_unstale":"2023-10-16T10:03:22.597077+0000","last_undegraded":"2023-10-16T10:03:22.597077+0000","last_fullsized":"2023-10-16T10:03:22.597077+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:43.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578285+0000","last_change":"2023-10-16T10:03:21.586835+0000","last_active":"2023-10-16T10:03:22.578285+0000","last_peered":"2023-10-16T10:03:22.578285+0000","last_clean":"2023-10-16T10:03:22.578285+0000","last_became_active":"2023-10-16T10:03:21.586569+0000","last_became_peered":"2023-10-16T10:03:21.586569+0000","last_unstale":"2023-10-16T10:03:22.578285+0000","last_undegraded":"2023-10-16T10:03:22.578285+0000","last_fullsized":"2023-10-16T10:03:22.578285+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:16.350674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589175+0000","last_change":"2023-10-16T10:03:22.585921+0000","last_active":"2023-10-16T10:03:22.589175+0000","last_peered":"2023-10-16T10:03:22.589175+0000","last_clean":"2023-10-16T10:03:22.589175+0000","last_became_active":"2023-10-16T10:03:22.585642+0000","last_became_peered":"2023-10-16T10:03:22.585642+0000","last_unstale":"2023-10-16T10:03:22.589175+0000","last_undegraded":"2023-10-16T10:03:22.589175+0000","last_fullsized":"2023-10-16T10:03:22.589175+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:44.243861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"131'3","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.600113+0000","last_change":"2023-10-16T10:03:21.589534+0000","last_active":"2023-10-16T10:03:22.600113+0000","last_peered":"2023-10-16T10:03:22.600113+0000","last_clean":"2023-10-16T10:03:22.600113+0000","last_became_active":"2023-10-16T10:03:21.589249+0000","last_became_peered":"2023-10-16T10:03:21.589249+0000","last_unstale":"2023-10-16T10:03:22.600113+0000","last_undegraded":"2023-10-16T10:03:22.600113+0000","last_fullsized":"2023-10-16T10:03:22.600113+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:39.709740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591170+0000","last_change":"2023-10-16T10:03:22.596384+0000","last_active":"2023-10-16T10:03:23.591170+0000","last_peered":"2023-10-16T10:03:23.591170+0000","last_clean":"2023-10-16T10:03:23.591170+0000","last_became_active":"2023-10-16T10:03:22.594202+0000","last_became_peered":"2023-10-16T10:03:22.594202+0000","last_unstale":"2023-10-16T10:03:23.591170+0000","last_undegraded":"2023-10-16T10:03:23.591170+0000","last_fullsized":"2023-10-16T10:03:23.591170+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:32.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"131'6","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.602828+0000","last_change":"2023-10-16T10:03:21.592585+0000","last_active":"2023-10-16T10:03:22.602828+0000","last_peered":"2023-10-16T10:03:22.602828+0000","last_clean":"2023-10-16T10:03:22.602828+0000","last_became_active":"2023-10-16T10:03:21.592189+0000","last_became_peered":"2023-10-16T10:03:21.592189+0000","last_unstale":"2023-10-16T10:03:22.602828+0000","last_undegraded":"2023-10-16T10:03:22.602828+0000","last_fullsized":"2023-10-16T10:03:22.602828+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:06.049254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589501+0000","last_change":"2023-10-16T10:03:22.582114+0000","last_active":"2023-10-16T10:03:22.589501+0000","last_peered":"2023-10-16T10:03:22.589501+0000","last_clean":"2023-10-16T10:03:22.589501+0000","last_became_active":"2023-10-16T10:03:22.581862+0000","last_became_peered":"2023-10-16T10:03:22.581862+0000","last_unstale":"2023-10-16T10:03:22.589501+0000","last_undegraded":"2023-10-16T10:03:22.589501+0000","last_fullsized":"2023-10-16T10:03:22.589501+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:15.351038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586953+0000","last_change":"2023-10-16T10:03:21.586742+0000","last_active":"2023-10-16T10:03:23.586953+0000","last_peered":"2023-10-16T10:03:23.586953+0000","last_clean":"2023-10-16T10:03:23.586953+0000","last_became_active":"2023-10-16T10:03:21.586485+0000","last_became_peered":"2023-10-16T10:03:21.586485+0000","last_unstale":"2023-10-16T10:03:23.586953+0000","last_undegraded":"2023-10-16T10:03:23.586953+0000","last_fullsized":"2023-10-16T10:03:23.586953+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:39.709740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594629+0000","last_change":"2023-10-16T10:03:22.592239+0000","last_active":"2023-10-16T10:03:22.594629+0000","last_peered":"2023-10-16T10:03:22.594629+0000","last_clean":"2023-10-16T10:03:22.594629+0000","last_became_active":"2023-10-16T10:03:22.591965+0000","last_became_peered":"2023-10-16T10:03:22.591965+0000","last_unstale":"2023-10-16T10:03:22.594629+0000","last_undegraded":"2023-10-16T10:03:22.594629+0000","last_fullsized":"2023-10-16T10:03:22.594629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:39.069792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.572990+0000","last_change":"2023-10-16T10:03:21.585684+0000","last_active":"2023-10-16T10:03:22.572990+0000","last_peered":"2023-10-16T10:03:22.572990+0000","last_clean":"2023-10-16T10:03:22.572990+0000","last_became_active":"2023-10-16T10:03:21.585337+0000","last_became_peered":"2023-10-16T10:03:21.585337+0000","last_unstale":"2023-10-16T10:03:22.572990+0000","last_undegraded":"2023-10-16T10:03:22.572990+0000","last_fullsized":"2023-10-16T10:03:22.572990+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:38.220519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589839+0000","last_change":"2023-10-16T10:03:22.585877+0000","last_active":"2023-10-16T10:03:22.589839+0000","last_peered":"2023-10-16T10:03:22.589839+0000","last_clean":"2023-10-16T10:03:22.589839+0000","last_became_active":"2023-10-16T10:03:22.585606+0000","last_became_peered":"2023-10-16T10:03:22.585606+0000","last_unstale":"2023-10-16T10:03:22.589839+0000","last_undegraded":"2023-10-16T10:03:22.589839+0000","last_fullsized":"2023-10-16T10:03:22.589839+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:48.130668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"131'8","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677539+0000","last_change":"2023-10-16T10:03:21.587389+0000","last_active":"2023-10-16T10:03:23.677539+0000","last_peered":"2023-10-16T10:03:23.677539+0000","last_clean":"2023-10-16T10:03:23.677539+0000","last_became_active":"2023-10-16T10:03:21.587083+0000","last_became_peered":"2023-10-16T10:03:21.587083+0000","last_unstale":"2023-10-16T10:03:23.677539+0000","last_undegraded":"2023-10-16T10:03:23.677539+0000","last_fullsized":"2023-10-16T10:03:23.677539+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:00.327029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677897+0000","last_change":"2023-10-16T10:03:22.595487+0000","last_active":"2023-10-16T10:03:23.677897+0000","last_peered":"2023-10-16T10:03:23.677897+0000","last_clean":"2023-10-16T10:03:23.677897+0000","last_became_active":"2023-10-16T10:03:22.595186+0000","last_became_peered":"2023-10-16T10:03:22.595186+0000","last_unstale":"2023-10-16T10:03:23.677897+0000","last_undegraded":"2023-10-16T10:03:23.677897+0000","last_fullsized":"2023-10-16T10:03:23.677897+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:37.628308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"131'1","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.600261+0000","last_change":"2023-10-16T10:03:21.589801+0000","last_active":"2023-10-16T10:03:22.600261+0000","last_peered":"2023-10-16T10:03:22.600261+0000","last_clean":"2023-10-16T10:03:22.600261+0000","last_became_active":"2023-10-16T10:03:21.589537+0000","last_became_peered":"2023-10-16T10:03:21.589537+0000","last_unstale":"2023-10-16T10:03:22.600261+0000","last_undegraded":"2023-10-16T10:03:22.600261+0000","last_fullsized":"2023-10-16T10:03:22.600261+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:30.503826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587176+0000","last_change":"2023-10-16T10:03:22.594096+0000","last_active":"2023-10-16T10:03:23.587176+0000","last_peered":"2023-10-16T10:03:23.587176+0000","last_clean":"2023-10-16T10:03:23.587176+0000","last_became_active":"2023-10-16T10:03:22.593826+0000","last_became_peered":"2023-10-16T10:03:22.593826+0000","last_unstale":"2023-10-16T10:03:23.587176+0000","last_undegraded":"2023-10-16T10:03:23.587176+0000","last_fullsized":"2023-10-16T10:03:23.587176+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:26.494959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677873+0000","last_change":"2023-10-16T10:03:21.579011+0000","last_active":"2023-10-16T10:03:23.677873+0000","last_peered":"2023-10-16T10:03:23.677873+0000","last_clean":"2023-10-16T10:03:23.677873+0000","last_became_active":"2023-10-16T10:03:21.578747+0000","last_became_peered":"2023-10-16T10:03:21.578747+0000","last_unstale":"2023-10-16T10:03:23.677873+0000","last_undegraded":"2023-10-16T10:03:23.677873+0000","last_fullsized":"2023-10-16T10:03:23.677873+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:12.781127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591214+0000","last_change":"2023-10-16T10:03:22.598884+0000","last_active":"2023-10-16T10:03:23.591214+0000","last_peered":"2023-10-16T10:03:23.591214+0000","last_clean":"2023-10-16T10:03:23.591214+0000","last_became_active":"2023-10-16T10:03:22.598597+0000","last_became_peered":"2023-10-16T10:03:22.598597+0000","last_unstale":"2023-10-16T10:03:23.591214+0000","last_undegraded":"2023-10-16T10:03:23.591214+0000","last_fullsized":"2023-10-16T10:03:23.591214+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:07.391180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"131'1","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596720+0000","last_change":"2023-10-16T10:03:21.584346+0000","last_active":"2023-10-16T10:03:22.596720+0000","last_peered":"2023-10-16T10:03:22.596720+0000","last_clean":"2023-10-16T10:03:22.596720+0000","last_became_active":"2023-10-16T10:03:21.583862+0000","last_became_peered":"2023-10-16T10:03:21.583862+0000","last_unstale":"2023-10-16T10:03:22.596720+0000","last_undegraded":"2023-10-16T10:03:22.596720+0000","last_fullsized":"2023-10-16T10:03:22.596720+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:12.781127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591472+0000","last_change":"2023-10-16T10:03:22.597792+0000","last_active":"2023-10-16T10:03:23.591472+0000","last_peered":"2023-10-16T10:03:23.591472+0000","last_clean":"2023-10-16T10:03:23.591472+0000","last_became_active":"2023-10-16T10:03:22.597489+0000","last_became_peered":"2023-10-16T10:03:22.597489+0000","last_unstale":"2023-10-16T10:03:23.591472+0000","last_undegraded":"2023-10-16T10:03:23.591472+0000","last_fullsized":"2023-10-16T10:03:23.591472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:51.200376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590936+0000","last_change":"2023-10-16T10:03:21.591458+0000","last_active":"2023-10-16T10:03:23.590936+0000","last_peered":"2023-10-16T10:03:23.590936+0000","last_clean":"2023-10-16T10:03:23.590936+0000","last_became_active":"2023-10-16T10:03:21.591159+0000","last_became_peered":"2023-10-16T10:03:21.591159+0000","last_unstale":"2023-10-16T10:03:23.590936+0000","last_undegraded":"2023-10-16T10:03:23.590936+0000","last_fullsized":"2023-10-16T10:03:23.590936+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:02.548562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591589+0000","last_change":"2023-10-16T10:03:22.591974+0000","last_active":"2023-10-16T10:03:23.591589+0000","last_peered":"2023-10-16T10:03:23.591589+0000","last_clean":"2023-10-16T10:03:23.591589+0000","last_became_active":"2023-10-16T10:03:22.591671+0000","last_became_peered":"2023-10-16T10:03:22.591671+0000","last_unstale":"2023-10-16T10:03:23.591589+0000","last_undegraded":"2023-10-16T10:03:23.591589+0000","last_fullsized":"2023-10-16T10:03:23.591589+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:13.996226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587039+0000","last_change":"2023-10-16T10:03:21.588926+0000","last_active":"2023-10-16T10:03:23.587039+0000","last_peered":"2023-10-16T10:03:23.587039+0000","last_clean":"2023-10-16T10:03:23.587039+0000","last_became_active":"2023-10-16T10:03:21.588670+0000","last_became_peered":"2023-10-16T10:03:21.588670+0000","last_unstale":"2023-10-16T10:03:23.587039+0000","last_undegraded":"2023-10-16T10:03:23.587039+0000","last_fullsized":"2023-10-16T10:03:23.587039+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:04:34.387199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.598235+0000","last_change":"2023-10-16T10:03:22.595087+0000","last_active":"2023-10-16T10:03:22.598235+0000","last_peered":"2023-10-16T10:03:22.598235+0000","last_clean":"2023-10-16T10:03:22.598235+0000","last_became_active":"2023-10-16T10:03:22.594827+0000","last_became_peered":"2023-10-16T10:03:22.594827+0000","last_unstale":"2023-10-16T10:03:22.598235+0000","last_undegraded":"2023-10-16T10:03:22.598235+0000","last_fullsized":"2023-10-16T10:03:22.598235+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:47.180693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677602+0000","last_change":"2023-10-16T10:03:21.584284+0000","last_active":"2023-10-16T10:03:23.677602+0000","last_peered":"2023-10-16T10:03:23.677602+0000","last_clean":"2023-10-16T10:03:23.677602+0000","last_became_active":"2023-10-16T10:03:21.583749+0000","last_became_peered":"2023-10-16T10:03:21.583749+0000","last_unstale":"2023-10-16T10:03:23.677602+0000","last_undegraded":"2023-10-16T10:03:23.677602+0000","last_fullsized":"2023-10-16T10:03:23.677602+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:07.659994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590788+0000","last_change":"2023-10-16T10:03:22.596574+0000","last_active":"2023-10-16T10:03:23.590788+0000","last_peered":"2023-10-16T10:03:23.590788+0000","last_clean":"2023-10-16T10:03:23.590788+0000","last_became_active":"2023-10-16T10:03:22.596286+0000","last_became_peered":"2023-10-16T10:03:22.596286+0000","last_unstale":"2023-10-16T10:03:23.590788+0000","last_undegraded":"2023-10-16T10:03:23.590788+0000","last_fullsized":"2023-10-16T10:03:23.590788+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:29.366054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586765+0000","last_change":"2023-10-16T10:03:21.588529+0000","last_active":"2023-10-16T10:03:23.586765+0000","last_peered":"2023-10-16T10:03:23.586765+0000","last_clean":"2023-10-16T10:03:23.586765+0000","last_became_active":"2023-10-16T10:03:21.588264+0000","last_became_peered":"2023-10-16T10:03:21.588264+0000","last_unstale":"2023-10-16T10:03:23.586765+0000","last_undegraded":"2023-10-16T10:03:23.586765+0000","last_fullsized":"2023-10-16T10:03:23.586765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:25.710723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.588605+0000","last_change":"2023-10-16T10:03:22.582928+0000","last_active":"2023-10-16T10:03:22.588605+0000","last_peered":"2023-10-16T10:03:22.588605+0000","last_clean":"2023-10-16T10:03:22.588605+0000","last_became_active":"2023-10-16T10:03:22.582671+0000","last_became_peered":"2023-10-16T10:03:22.582671+0000","last_unstale":"2023-10-16T10:03:22.588605+0000","last_undegraded":"2023-10-16T10:03:22.588605+0000","last_fullsized":"2023-10-16T10:03:22.588605+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:34.692527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578786+0000","last_change":"2023-10-16T10:03:21.586147+0000","last_active":"2023-10-16T10:03:22.578786+0000","last_peered":"2023-10-16T10:03:22.578786+0000","last_clean":"2023-10-16T10:03:22.578786+0000","last_became_active":"2023-10-16T10:03:21.585611+0000","last_became_peered":"2023-10-16T10:03:21.585611+0000","last_unstale":"2023-10-16T10:03:22.578786+0000","last_undegraded":"2023-10-16T10:03:22.578786+0000","last_fullsized":"2023-10-16T10:03:22.578786+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:11.852190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589643+0000","last_change":"2023-10-16T10:03:22.588355+0000","last_active":"2023-10-16T10:03:22.589643+0000","last_peered":"2023-10-16T10:03:22.589643+0000","last_clean":"2023-10-16T10:03:22.589643+0000","last_became_active":"2023-10-16T10:03:22.587985+0000","last_became_peered":"2023-10-16T10:03:22.587985+0000","last_unstale":"2023-10-16T10:03:22.589643+0000","last_undegraded":"2023-10-16T10:03:22.589643+0000","last_fullsized":"2023-10-16T10:03:22.589643+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:54.469586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"131'2","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.597766+0000","last_change":"2023-10-16T10:03:21.589473+0000","last_active":"2023-10-16T10:03:22.597766+0000","last_peered":"2023-10-16T10:03:22.597766+0000","last_clean":"2023-10-16T10:03:22.597766+0000","last_became_active":"2023-10-16T10:03:21.589166+0000","last_became_peered":"2023-10-16T10:03:21.589166+0000","last_unstale":"2023-10-16T10:03:22.597766+0000","last_undegraded":"2023-10-16T10:03:22.597766+0000","last_fullsized":"2023-10-16T10:03:22.597766+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:47.376383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594797+0000","last_change":"2023-10-16T10:03:22.588681+0000","last_active":"2023-10-16T10:03:22.594797+0000","last_peered":"2023-10-16T10:03:22.594797+0000","last_clean":"2023-10-16T10:03:22.594797+0000","last_became_active":"2023-10-16T10:03:22.588248+0000","last_became_peered":"2023-10-16T10:03:22.588248+0000","last_unstale":"2023-10-16T10:03:22.594797+0000","last_undegraded":"2023-10-16T10:03:22.594797+0000","last_fullsized":"2023-10-16T10:03:22.594797+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:31.251884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.573587+0000","last_change":"2023-10-16T10:03:21.582990+0000","last_active":"2023-10-16T10:03:22.573587+0000","last_peered":"2023-10-16T10:03:22.573587+0000","last_clean":"2023-10-16T10:03:22.573587+0000","last_became_active":"2023-10-16T10:03:21.582715+0000","last_became_peered":"2023-10-16T10:03:21.582715+0000","last_unstale":"2023-10-16T10:03:22.573587+0000","last_undegraded":"2023-10-16T10:03:22.573587+0000","last_fullsized":"2023-10-16T10:03:22.573587+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:07.659994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596061+0000","last_change":"2023-10-16T10:03:22.580715+0000","last_active":"2023-10-16T10:03:22.596061+0000","last_peered":"2023-10-16T10:03:22.596061+0000","last_clean":"2023-10-16T10:03:22.596061+0000","last_became_active":"2023-10-16T10:03:22.580434+0000","last_became_peered":"2023-10-16T10:03:22.580434+0000","last_unstale":"2023-10-16T10:03:22.596061+0000","last_undegraded":"2023-10-16T10:03:22.596061+0000","last_fullsized":"2023-10-16T10:03:22.596061+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:35.486946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.574097+0000","last_change":"2023-10-16T10:03:21.585852+0000","last_active":"2023-10-16T10:03:22.574097+0000","last_peered":"2023-10-16T10:03:22.574097+0000","last_clean":"2023-10-16T10:03:22.574097+0000","last_became_active":"2023-10-16T10:03:21.585568+0000","last_became_peered":"2023-10-16T10:03:21.585568+0000","last_unstale":"2023-10-16T10:03:22.574097+0000","last_undegraded":"2023-10-16T10:03:22.574097+0000","last_fullsized":"2023-10-16T10:03:22.574097+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:47.584001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.582155+0000","last_change":"2023-10-16T10:03:22.580507+0000","last_active":"2023-10-16T10:03:22.582155+0000","last_peered":"2023-10-16T10:03:22.582155+0000","last_clean":"2023-10-16T10:03:22.582155+0000","last_became_active":"2023-10-16T10:03:22.580214+0000","last_became_peered":"2023-10-16T10:03:22.580214+0000","last_unstale":"2023-10-16T10:03:22.582155+0000","last_undegraded":"2023-10-16T10:03:22.582155+0000","last_fullsized":"2023-10-16T10:03:22.582155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:49.897491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578247+0000","last_change":"2023-10-16T10:03:21.591639+0000","last_active":"2023-10-16T10:03:22.578247+0000","last_peered":"2023-10-16T10:03:22.578247+0000","last_clean":"2023-10-16T10:03:22.578247+0000","last_became_active":"2023-10-16T10:03:21.591454+0000","last_became_peered":"2023-10-16T10:03:21.591454+0000","last_unstale":"2023-10-16T10:03:22.578247+0000","last_undegraded":"2023-10-16T10:03:22.578247+0000","last_fullsized":"2023-10-16T10:03:22.578247+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:17.567179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588519+0000","last_change":"2023-10-16T10:03:22.585824+0000","last_active":"2023-10-16T10:03:23.588519+0000","last_peered":"2023-10-16T10:03:23.588519+0000","last_clean":"2023-10-16T10:03:23.588519+0000","last_became_active":"2023-10-16T10:03:22.585564+0000","last_became_peered":"2023-10-16T10:03:22.585564+0000","last_unstale":"2023-10-16T10:03:23.588519+0000","last_undegraded":"2023-10-16T10:03:23.588519+0000","last_fullsized":"2023-10-16T10:03:23.588519+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:06.355104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590823+0000","last_change":"2023-10-16T10:03:21.582814+0000","last_active":"2023-10-16T10:03:23.590823+0000","last_peered":"2023-10-16T10:03:23.590823+0000","last_clean":"2023-10-16T10:03:23.590823+0000","last_became_active":"2023-10-16T10:03:21.582505+0000","last_became_peered":"2023-10-16T10:03:21.582505+0000","last_unstale":"2023-10-16T10:03:23.590823+0000","last_undegraded":"2023-10-16T10:03:23.590823+0000","last_fullsized":"2023-10-16T10:03:23.590823+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:10.999669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.589088+0000","last_change":"2023-10-16T10:03:22.596780+0000","last_active":"2023-10-16T10:03:23.589088+0000","last_peered":"2023-10-16T10:03:23.589088+0000","last_clean":"2023-10-16T10:03:23.589088+0000","last_became_active":"2023-10-16T10:03:22.596099+0000","last_became_peered":"2023-10-16T10:03:22.596099+0000","last_unstale":"2023-10-16T10:03:23.589088+0000","last_undegraded":"2023-10-16T10:03:23.589088+0000","last_fullsized":"2023-10-16T10:03:23.589088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:11.895956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588501+0000","last_change":"2023-10-16T10:03:21.581587+0000","last_active":"2023-10-16T10:03:23.588501+0000","last_peered":"2023-10-16T10:03:23.588501+0000","last_clean":"2023-10-16T10:03:23.588501+0000","last_became_active":"2023-10-16T10:03:21.581286+0000","last_became_peered":"2023-10-16T10:03:21.581286+0000","last_unstale":"2023-10-16T10:03:23.588501+0000","last_undegraded":"2023-10-16T10:03:23.588501+0000","last_fullsized":"2023-10-16T10:03:23.588501+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:01.674797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590941+0000","last_change":"2023-10-16T10:03:22.598230+0000","last_active":"2023-10-16T10:03:23.590941+0000","last_peered":"2023-10-16T10:03:23.590941+0000","last_clean":"2023-10-16T10:03:23.590941+0000","last_became_active":"2023-10-16T10:03:22.597939+0000","last_became_peered":"2023-10-16T10:03:22.597939+0000","last_unstale":"2023-10-16T10:03:23.590941+0000","last_undegraded":"2023-10-16T10:03:23.590941+0000","last_fullsized":"2023-10-16T10:03:23.590941+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:48.383707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677597+0000","last_change":"2023-10-16T10:03:21.584858+0000","last_active":"2023-10-16T10:03:23.677597+0000","last_peered":"2023-10-16T10:03:23.677597+0000","last_clean":"2023-10-16T10:03:23.677597+0000","last_became_active":"2023-10-16T10:03:21.584583+0000","last_became_peered":"2023-10-16T10:03:21.584583+0000","last_unstale":"2023-10-16T10:03:23.677597+0000","last_undegraded":"2023-10-16T10:03:23.677597+0000","last_fullsized":"2023-10-16T10:03:23.677597+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:25.710723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591642+0000","last_change":"2023-10-16T10:03:22.580706+0000","last_active":"2023-10-16T10:03:22.591642+0000","last_peered":"2023-10-16T10:03:22.591642+0000","last_clean":"2023-10-16T10:03:22.591642+0000","last_became_active":"2023-10-16T10:03:22.580442+0000","last_became_peered":"2023-10-16T10:03:22.580442+0000","last_unstale":"2023-10-16T10:03:22.591642+0000","last_undegraded":"2023-10-16T10:03:22.591642+0000","last_fullsized":"2023-10-16T10:03:22.591642+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.864772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578802+0000","last_change":"2023-10-16T10:03:21.589131+0000","last_active":"2023-10-16T10:03:22.578802+0000","last_peered":"2023-10-16T10:03:22.578802+0000","last_clean":"2023-10-16T10:03:22.578802+0000","last_became_active":"2023-10-16T10:03:21.588513+0000","last_became_peered":"2023-10-16T10:03:21.588513+0000","last_unstale":"2023-10-16T10:03:22.578802+0000","last_undegraded":"2023-10-16T10:03:22.578802+0000","last_fullsized":"2023-10-16T10:03:22.578802+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:23.018562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589753+0000","last_change":"2023-10-16T10:03:22.588425+0000","last_active":"2023-10-16T10:03:22.589753+0000","last_peered":"2023-10-16T10:03:22.589753+0000","last_clean":"2023-10-16T10:03:22.589753+0000","last_became_active":"2023-10-16T10:03:22.588123+0000","last_became_peered":"2023-10-16T10:03:22.588123+0000","last_unstale":"2023-10-16T10:03:22.589753+0000","last_undegraded":"2023-10-16T10:03:22.589753+0000","last_fullsized":"2023-10-16T10:03:22.589753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:51.752847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591270+0000","last_change":"2023-10-16T10:03:21.591467+0000","last_active":"2023-10-16T10:03:23.591270+0000","last_peered":"2023-10-16T10:03:23.591270+0000","last_clean":"2023-10-16T10:03:23.591270+0000","last_became_active":"2023-10-16T10:03:21.591179+0000","last_became_peered":"2023-10-16T10:03:21.591179+0000","last_unstale":"2023-10-16T10:03:23.591270+0000","last_undegraded":"2023-10-16T10:03:23.591270+0000","last_fullsized":"2023-10-16T10:03:23.591270+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:45.651540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"122'227","reported_seq":665,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589922+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:03:22.589922+0000","last_peered":"2023-10-16T10:03:22.589922+0000","last_clean":"2023-10-16T10:03:22.589922+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:03:22.589922+0000","last_undegraded":"2023-10-16T10:03:22.589922+0000","last_fullsized":"2023-10-16T10:03:22.589922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542375,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33372,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338468,"statfs":{"total":96636764160,"available":96602591232,"internally_reserved":0,"allocated":2236416,"data_stored":1401496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.248},"max":{"1min":0.67400000000000004,"5min":0.79200000000000004,"15min":0.90000000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.27200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54300000000000004}]},{"osd":1,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.84699999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.749,"5min":1.788,"15min":1.788},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55700000000000005,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.80400000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.438}]},{"osd":3,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.35199999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.505},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.96399999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.46700000000000003},"min":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.26500000000000001},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.89200000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48699999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":1.1779999999999999},"last":0.35099999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.006},"last":0.374},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.80500000000000005,"15min":1.0169999999999999},"last":0.627}]},{"osd":6,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.36399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":1.373},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.38,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.81399999999999995,"5min":0.83799999999999997,"15min":1.4099999999999999},"last":0.42399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542374,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1626112,"data_stored":803830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.45100000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.79800000000000004},"last":0.438},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.504}]},{"osd":2,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.57499999999999996},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.86099999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.36299999999999999},"max":{"1min":0.70999999999999996,"5min":0.76100000000000001,"15min":0.84499999999999997},"last":0.36899999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":1.03,"15min":1.03},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.58199999999999996,"15min":0.59699999999999998},"min":{"1min":0.44800000000000001,"5min":0.35499999999999998,"15min":0.34599999999999997},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.85899999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.746,"5min":1,"15min":1},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35199999999999998,"15min":0.33300000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51200000000000001},"min":{"1min":0.378,"5min":0.314,"15min":0.314},"max":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.78100000000000003,"5min":1.056,"15min":1.056},"last":0.66000000000000003}]},{"osd":7,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.86399999999999999},"last":0.623},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55300000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542374,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50840,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321000,"statfs":{"total":96636764160,"available":96584704000,"internally_reserved":0,"allocated":1642496,"data_stored":810636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36199999999999999,"15min":0.36699999999999999},"min":{"1min":0.248,"5min":0.221,"15min":0.20100000000000001},"max":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":1.5},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39500000000000002,"15min":0.40400000000000003},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.60099999999999998,"15min":0.80000000000000004},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.377,"15min":0.39200000000000002},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.56699999999999995,"5min":0.63800000000000001,"15min":1.1040000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.41599999999999998,"15min":0.437},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.441,"5min":0.67300000000000004,"15min":3.8730000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.46800000000000003},"min":{"1min":0.32200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.53100000000000003,"5min":0.68000000000000005,"15min":4.6740000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.30299999999999999,"5min":0.29399999999999998,"15min":0.25700000000000001},"max":{"1min":0.55100000000000005,"5min":0.65100000000000002,"15min":1.379},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.379,"15min":0.33100000000000002},"max":{"1min":0.94199999999999995,"5min":0.96899999999999997,"15min":1.2490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.64500000000000002},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80900000000000005,"5min":0.92100000000000004,"15min":5.4119999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84399999999999997,"5min":0.94999999999999996,"15min":1.26},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.40200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.85399999999999998,"15min":1.3560000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64600000000000002,"15min":0.66600000000000004},"min":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":0.92100000000000004,"15min":6.2140000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.47199999999999998,"5min":0.36799999999999999,"15min":0.36199999999999999},"max":{"1min":0.877,"5min":0.92600000000000005,"15min":1.3420000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.65600000000000003},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.39800000000000002},"max":{"1min":0.91000000000000003,"5min":0.98299999999999998,"15min":1.4079999999999999},"last":0.752},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64000000000000001,"15min":0.66400000000000003},"min":{"1min":0.45200000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84999999999999998,"5min":0.93799999999999994,"15min":1.4510000000000001},"last":0.77900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542375,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33364,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338476,"statfs":{"total":96636764160,"available":96602599424,"internally_reserved":0,"allocated":2236416,"data_stored":1395852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.22800000000000001,"5min":0.192,"15min":0.191},"max":{"1min":0.52800000000000002,"5min":0.59999999999999998,"15min":0.65400000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.60999999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.28899999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39000000000000001,"15min":0.40600000000000003},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54400000000000004,"5min":0.61299999999999999,"15min":0.626},"last":0.308},{"interface":"front","average":{"1min":0.375,"5min":0.39700000000000002,"15min":0.41899999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.53000000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.33400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40600000000000003,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55600000000000005,"5min":0.64800000000000002,"15min":0.72999999999999998},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45900000000000002,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59999999999999998,"5min":0.69299999999999995,"15min":0.69699999999999995},"last":0.375}]},{"osd":4,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83199999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.63200000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.98799999999999999},"last":0.48199999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34200000000000003},"max":{"1min":0.92200000000000004,"5min":1.004,"15min":1.004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83499999999999996,"5min":0.88700000000000001,"15min":0.96999999999999997},"last":0.621}]},{"osd":6,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.85499999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.66200000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.86899999999999999,"15min":1.0089999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.65700000000000003},"min":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83899999999999997,"5min":0.94499999999999995,"15min":1.0249999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.66000000000000003},"min":{"1min":0.432,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.95699999999999996},"last":0.503}]}]},{"osd":2,"up_from":14,"seq":60129542374,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54988,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316852,"statfs":{"total":96636764160,"available":96580456448,"internally_reserved":0,"allocated":1634304,"data_stored":810512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.371,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.183},"max":{"1min":0.53100000000000003,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.26300000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.49199999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40100000000000002,"15min":0.40699999999999997},"min":{"1min":0.28499999999999998,"5min":0.252,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.42399999999999999},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.214},"max":{"1min":0.58399999999999996,"5min":0.60699999999999998,"15min":0.64000000000000001},"last":0.33600000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.40699999999999997},"min":{"1min":0.252,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.55500000000000005,"5min":0.59199999999999997,"15min":0.61899999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.46600000000000003},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.61699999999999999,"5min":0.747,"15min":0.747},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.81599999999999995,"5min":0.91600000000000004,"15min":0.93999999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.36799999999999999},"max":{"1min":0.82699999999999996,"5min":0.88200000000000001,"15min":1.091},"last":0.60999999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":1.01,"15min":4.3609999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.44900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.879,"15min":1.0720000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63,"15min":0.64500000000000002},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.90500000000000003,"15min":0.97899999999999998},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79400000000000004,"5min":0.96499999999999997,"15min":0.98799999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.38500000000000001},"max":{"1min":0.79100000000000004,"5min":0.98799999999999999,"15min":0.998},"last":0.64100000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542375,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36944,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334896,"statfs":{"total":96636764160,"available":96598933504,"internally_reserved":0,"allocated":1638400,"data_stored":805442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.372,"15min":0.38600000000000001},"min":{"1min":0.215,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":0.55000000000000004,"5min":0.60299999999999998,"15min":0.64900000000000002},"last":0.313},{"interface":"front","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.41699999999999998},"min":{"1min":0.28999999999999998,"5min":0.26200000000000001,"15min":0.185},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.63100000000000001},"last":0.33900000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38100000000000001,"15min":0.39500000000000002},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.219},"max":{"1min":0.51500000000000001,"5min":0.61399999999999999,"15min":0.72199999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.40500000000000003,"15min":0.42099999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52800000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.76200000000000001,"5min":0.95699999999999996,"15min":0.95799999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.44900000000000001,"5min":0.314,"15min":0.311},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.0820000000000001},"last":0.45800000000000002}]},{"osd":5,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.98699999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.35599999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.996},"last":0.41499999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.626,"15min":0.64200000000000002},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90000000000000002,"5min":0.92400000000000004,"15min":0.98399999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.99099999999999999},"last":0.42399999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.3},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91800000000000004,"15min":1.4019999999999999},"last":0.52100000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59144,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312696,"statfs":{"total":96636764160,"available":96576200704,"internally_reserved":0,"allocated":1634304,"data_stored":810512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.186},"max":{"1min":0.67400000000000004,"5min":0.76900000000000002,"15min":0.89400000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38400000000000001,"5min":0.248,"15min":0.23899999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.81299999999999994},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.29899999999999999,"15min":0.26300000000000001},"max":{"1min":0.75700000000000001,"5min":0.84599999999999997,"15min":0.86599999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.248},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.83799999999999997},"last":0.52500000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.76000000000000001,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.59699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.27900000000000003},"max":{"1min":0.72299999999999998,"5min":0.72799999999999998,"15min":0.89100000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.314},"max":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.86499999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60499999999999998},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76700000000000002,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.496,"15min":0.51200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65700000000000003,"5min":0.77800000000000002,"15min":0.80400000000000005},"last":0.36899999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48799999999999999,"15min":0.50800000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.82999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.56100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.85899999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53900000000000003,"15min":0.55600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.80700000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80400000000000005,"5min":0.83299999999999996,"15min":0.84599999999999997},"last":0.5}]}]},{"osd":5,"up_from":14,"seq":60129542375,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1642496,"data_stored":811116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57799999999999996,"15min":0.58699999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77000000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56399999999999995,"15min":0.59099999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.27700000000000002},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.91800000000000004},"last":0.57399999999999995}]},{"osd":1,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.82799999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70199999999999996}]},{"osd":2,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56499999999999995,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.35599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.88700000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.88600000000000001,"15min":0.89000000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48799999999999999,"15min":0.48899999999999999},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.624},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85599999999999998,"5min":0.86099999999999999,"15min":0.86799999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.311},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88600000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:03:26.686 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:03:26.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.838+0000 7f2690303700 1 Processor -- start 2023-10-16T10:03:26.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.838+0000 7f2690303700 1 -- start start 2023-10-16T10:03:26.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f2690303700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f268815b990 0x7f26880ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f2690303700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26880d0370 0x7f26880d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f2690303700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26880cc510 con 0x7f26880cd2d0 2023-10-16T10:03:26.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f2690303700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26880d24f0 con 0x7f26880d0370 2023-10-16T10:03:26.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f2690303700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26880cc7f0 con 0x7f268815b990 2023-10-16T10:03:26.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f268e8a0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26880d0370 0x7f26880d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f268d89e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f26880cd2d0 0x7f26880cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58720/0 (socket says 172.21.15.181:58720) 2023-10-16T10:03:26.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f268e8a0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26880d0370 0x7f26880d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47646/0 (socket says 172.21.15.181:47646) 2023-10-16T10:03:26.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.839+0000 7f268d89e700 1 -- 172.21.15.181:0/1615085079 learned_addr learned my addr 172.21.15.181:0/1615085079 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:26.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268e09f700 1 --2- 172.21.15.181:0/1615085079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f268815b990 0x7f26880ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268e8a0700 1 -- 172.21.15.181:0/1615085079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f268815b990 msgr2=0x7f26880ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:26.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268e8a0700 1 --2- 172.21.15.181:0/1615085079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f268815b990 0x7f26880ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268e8a0700 1 -- 172.21.15.181:0/1615085079 >> v1:172.21.15.181:6789/0 conn(0x7f26880cd2d0 legacy=0x7f26880cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268e8a0700 1 -- 172.21.15.181:0/1615085079 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880ccad0 con 0x7f26880d0370 2023-10-16T10:03:26.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268d09d700 1 -- 172.21.15.181:0/1615085079 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421650249 0 0) 0x7f26880cc510 con 0x7f26880cd2d0 2023-10-16T10:03:26.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268e09f700 1 --2- 172.21.15.181:0/1615085079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f268815b990 0x7f26880ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:26.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268e8a0700 1 --2- 172.21.15.181:0/1615085079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26880d0370 0x7f26880d1fb0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f2680009d90 tx=0x7f268000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8902713fdaaa92ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268d09d700 1 -- 172.21.15.181:0/1615085079 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268000e070 con 0x7f26880d0370 2023-10-16T10:03:26.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268d09d700 1 -- 172.21.15.181:0/1615085079 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268000bbd0 con 0x7f26880d0370 2023-10-16T10:03:26.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f268d09d700 1 -- 172.21.15.181:0/1615085079 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680012430 con 0x7f26880d0370 2023-10-16T10:03:26.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f2690303700 1 -- 172.21.15.181:0/1615085079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26880d0370 msgr2=0x7f26880d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.840+0000 7f2690303700 1 --2- 172.21.15.181:0/1615085079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26880d0370 0x7f26880d1fb0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f2680009d90 tx=0x7f268000b330 comp rx=0 tx=0).stop 2023-10-16T10:03:26.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.841+0000 7f2690303700 1 -- 172.21.15.181:0/1615085079 shutdown_connections 2023-10-16T10:03:26.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.841+0000 7f2690303700 1 --2- 172.21.15.181:0/1615085079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26880d0370 0x7f26880d1fb0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.841+0000 7f2690303700 1 --2- 172.21.15.181:0/1615085079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f268815b990 0x7f26880ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.841+0000 7f2690303700 1 -- 172.21.15.181:0/1615085079 >> 172.21.15.181:0/1615085079 conn(0x7f268809a9a0 msgr2=0x7f26880d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:26.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.841+0000 7f2690303700 1 -- 172.21.15.181:0/1615085079 shutdown_connections 2023-10-16T10:03:26.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.841+0000 7f2690303700 1 -- 172.21.15.181:0/1615085079 wait complete. 2023-10-16T10:03:26.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 Processor -- start 2023-10-16T10:03:26.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 -- start start 2023-10-16T10:03:26.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26880cd2d0 0x7f2688158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26880d0370 0x7f268806ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268815b990 0x7f26880ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268815e5c0 con 0x7f268815b990 2023-10-16T10:03:26.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268806d950 con 0x7f26880cd2d0 2023-10-16T10:03:26.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.842+0000 7f2690303700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268815dc20 con 0x7f26880d0370 2023-10-16T10:03:26.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268815b990 0x7f26880ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268815b990 0x7f26880ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47652/0 (socket says 172.21.15.181:47652) 2023-10-16T10:03:26.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 -- 172.21.15.181:0/4105368517 learned_addr learned my addr 172.21.15.181:0/4105368517 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:26.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268d89e700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26880d0370 0x7f268806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e09f700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26880cd2d0 0x7f2688158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 -- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26880d0370 msgr2=0x7f268806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26880d0370 0x7f268806ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 -- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26880cd2d0 msgr2=0x7f2688158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:26.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26880cd2d0 0x7f2688158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:26.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 -- 172.21.15.181:0/4105368517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680009a70 con 0x7f268815b990 2023-10-16T10:03:26.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268d89e700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26880d0370 0x7f268806ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:26.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f268e8a0700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268815b990 0x7f26880ca920 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2680013000 tx=0x7f2680004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f267effd700 1 -- 172.21.15.181:0/4105368517 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680003ea0 con 0x7f268815b990 2023-10-16T10:03:26.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f267effd700 1 -- 172.21.15.181:0/4105368517 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2680020070 con 0x7f268815b990 2023-10-16T10:03:26.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268815df00 con 0x7f268815b990 2023-10-16T10:03:26.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f267effd700 1 -- 172.21.15.181:0/4105368517 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680004280 con 0x7f268815b990 2023-10-16T10:03:26.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.843+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26881592f0 con 0x7f268815b990 2023-10-16T10:03:26.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.844+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268815e700 con 0x7f268815b990 2023-10-16T10:03:26.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.845+0000 7f267effd700 1 -- 172.21.15.181:0/4105368517 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f26800043e0 con 0x7f268815b990 2023-10-16T10:03:26.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.845+0000 7f267effd700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f266c06b020 0x7f266c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:26.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.845+0000 7f267effd700 1 -- 172.21.15.181:0/4105368517 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f2680022070 con 0x7f268815b990 2023-10-16T10:03:26.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.845+0000 7f268e09f700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f266c06b020 0x7f266c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:26.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.846+0000 7f268e09f700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f266c06b020 0x7f266c06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f267800a970 tx=0x7f2678008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:26.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:26.850+0000 7f267effd700 1 -- 172.21.15.181:0/4105368517 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f268815e700 con 0x7f268815b990 2023-10-16T10:03:27.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.039+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 --> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26880cc200 con 0x7f266c06b020 2023-10-16T10:03:27.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.047+0000 7f267effd700 1 -- 172.21.15.181:0/4105368517 <== mgr.8304 v2:172.21.15.181:6832/3280351398 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289936 (secure 0 0 0) 0x7f267800ae40 con 0x7f266c06b020 2023-10-16T10:03:27.051 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:27.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f266c06b020 msgr2=0x7f266c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f266c06b020 0x7f266c06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f267800a970 tx=0x7f2678008040 comp rx=0 tx=0).stop 2023-10-16T10:03:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268815b990 msgr2=0x7f26880ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268815b990 0x7f26880ca920 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2680013000 tx=0x7f2680004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 shutdown_connections 2023-10-16T10:03:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f268815b990 0x7f26880ca920 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26880cd2d0 0x7f2688158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f266c06b020 0x7f266c06d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 --2- 172.21.15.181:0/4105368517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26880d0370 0x7f268806ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 >> 172.21.15.181:0/4105368517 conn(0x7f268809a9a0 msgr2=0x7f2688070290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 shutdown_connections 2023-10-16T10:03:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.050+0000 7f2690303700 1 -- 172.21.15.181:0/4105368517 wait complete. 2023-10-16T10:03:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:03:27.063 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:03:26.491600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":334212,"kb_used_data":13956,"kb_used_omap":12,"kb_used_meta":320115,"kb_avail":754640508,"statfs":{"total":773094113280,"available":772751880192,"internally_reserved":0,"allocated":14290944,"data_stored":7649396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":327798328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.932734"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594777+0000","last_change":"2023-10-16T10:03:22.588516+0000","last_active":"2023-10-16T10:03:22.594777+0000","last_peered":"2023-10-16T10:03:22.594777+0000","last_clean":"2023-10-16T10:03:22.594777+0000","last_became_active":"2023-10-16T10:03:22.587709+0000","last_became_peered":"2023-10-16T10:03:22.587709+0000","last_unstale":"2023-10-16T10:03:22.594777+0000","last_undegraded":"2023-10-16T10:03:22.594777+0000","last_fullsized":"2023-10-16T10:03:22.594777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:58:58.233371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588892+0000","last_change":"2023-10-16T10:03:22.585643+0000","last_active":"2023-10-16T10:03:23.588892+0000","last_peered":"2023-10-16T10:03:23.588892+0000","last_clean":"2023-10-16T10:03:23.588892+0000","last_became_active":"2023-10-16T10:03:22.585142+0000","last_became_peered":"2023-10-16T10:03:22.585142+0000","last_unstale":"2023-10-16T10:03:23.588892+0000","last_undegraded":"2023-10-16T10:03:23.588892+0000","last_fullsized":"2023-10-16T10:03:23.588892+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:01.994296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591080+0000","last_change":"2023-10-16T10:03:22.595113+0000","last_active":"2023-10-16T10:03:23.591080+0000","last_peered":"2023-10-16T10:03:23.591080+0000","last_clean":"2023-10-16T10:03:23.591080+0000","last_became_active":"2023-10-16T10:03:22.594786+0000","last_became_peered":"2023-10-16T10:03:22.594786+0000","last_unstale":"2023-10-16T10:03:23.591080+0000","last_undegraded":"2023-10-16T10:03:23.591080+0000","last_fullsized":"2023-10-16T10:03:23.591080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:17.357998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596638+0000","last_change":"2023-10-16T10:03:22.587988+0000","last_active":"2023-10-16T10:03:22.596638+0000","last_peered":"2023-10-16T10:03:22.596638+0000","last_clean":"2023-10-16T10:03:22.596638+0000","last_became_active":"2023-10-16T10:03:22.587737+0000","last_became_peered":"2023-10-16T10:03:22.587737+0000","last_unstale":"2023-10-16T10:03:22.596638+0000","last_undegraded":"2023-10-16T10:03:22.596638+0000","last_fullsized":"2023-10-16T10:03:22.596638+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:49.377260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677790+0000","last_change":"2023-10-16T10:03:22.589948+0000","last_active":"2023-10-16T10:03:23.677790+0000","last_peered":"2023-10-16T10:03:23.677790+0000","last_clean":"2023-10-16T10:03:23.677790+0000","last_became_active":"2023-10-16T10:03:22.589657+0000","last_became_peered":"2023-10-16T10:03:22.589657+0000","last_unstale":"2023-10-16T10:03:23.677790+0000","last_undegraded":"2023-10-16T10:03:23.677790+0000","last_fullsized":"2023-10-16T10:03:23.677790+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:47.180693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586771+0000","last_change":"2023-10-16T10:03:22.582717+0000","last_active":"2023-10-16T10:03:23.586771+0000","last_peered":"2023-10-16T10:03:23.586771+0000","last_clean":"2023-10-16T10:03:23.586771+0000","last_became_active":"2023-10-16T10:03:22.582457+0000","last_became_peered":"2023-10-16T10:03:22.582457+0000","last_unstale":"2023-10-16T10:03:23.586771+0000","last_undegraded":"2023-10-16T10:03:23.586771+0000","last_fullsized":"2023-10-16T10:03:23.586771+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:49.377260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590773+0000","last_change":"2023-10-16T10:03:22.583166+0000","last_active":"2023-10-16T10:03:23.590773+0000","last_peered":"2023-10-16T10:03:23.590773+0000","last_clean":"2023-10-16T10:03:23.590773+0000","last_became_active":"2023-10-16T10:03:22.582710+0000","last_became_peered":"2023-10-16T10:03:22.582710+0000","last_unstale":"2023-10-16T10:03:23.590773+0000","last_undegraded":"2023-10-16T10:03:23.590773+0000","last_fullsized":"2023-10-16T10:03:23.590773+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:47.266900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590636+0000","last_change":"2023-10-16T10:03:22.594197+0000","last_active":"2023-10-16T10:03:23.590636+0000","last_peered":"2023-10-16T10:03:23.590636+0000","last_clean":"2023-10-16T10:03:23.590636+0000","last_became_active":"2023-10-16T10:03:22.593731+0000","last_became_peered":"2023-10-16T10:03:22.593731+0000","last_unstale":"2023-10-16T10:03:23.590636+0000","last_undegraded":"2023-10-16T10:03:23.590636+0000","last_fullsized":"2023-10-16T10:03:23.590636+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:39.069792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594757+0000","last_change":"2023-10-16T10:03:22.588000+0000","last_active":"2023-10-16T10:03:22.594757+0000","last_peered":"2023-10-16T10:03:22.594757+0000","last_clean":"2023-10-16T10:03:22.594757+0000","last_became_active":"2023-10-16T10:03:22.587385+0000","last_became_peered":"2023-10-16T10:03:22.587385+0000","last_unstale":"2023-10-16T10:03:22.594757+0000","last_undegraded":"2023-10-16T10:03:22.594757+0000","last_fullsized":"2023-10-16T10:03:22.594757+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:18:14.140877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594512+0000","last_change":"2023-10-16T10:03:22.587449+0000","last_active":"2023-10-16T10:03:22.594512+0000","last_peered":"2023-10-16T10:03:22.594512+0000","last_clean":"2023-10-16T10:03:22.594512+0000","last_became_active":"2023-10-16T10:03:22.586557+0000","last_became_peered":"2023-10-16T10:03:22.586557+0000","last_unstale":"2023-10-16T10:03:22.594512+0000","last_undegraded":"2023-10-16T10:03:22.594512+0000","last_fullsized":"2023-10-16T10:03:22.594512+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:26.494959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588773+0000","last_change":"2023-10-16T10:03:22.579908+0000","last_active":"2023-10-16T10:03:23.588773+0000","last_peered":"2023-10-16T10:03:23.588773+0000","last_clean":"2023-10-16T10:03:23.588773+0000","last_became_active":"2023-10-16T10:03:22.579603+0000","last_became_peered":"2023-10-16T10:03:22.579603+0000","last_unstale":"2023-10-16T10:03:23.588773+0000","last_undegraded":"2023-10-16T10:03:23.588773+0000","last_fullsized":"2023-10-16T10:03:23.588773+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:33.324720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596620+0000","last_change":"2023-10-16T10:03:22.587092+0000","last_active":"2023-10-16T10:03:22.596620+0000","last_peered":"2023-10-16T10:03:22.596620+0000","last_clean":"2023-10-16T10:03:22.596620+0000","last_became_active":"2023-10-16T10:03:22.586823+0000","last_became_peered":"2023-10-16T10:03:22.586823+0000","last_unstale":"2023-10-16T10:03:22.596620+0000","last_undegraded":"2023-10-16T10:03:22.596620+0000","last_fullsized":"2023-10-16T10:03:22.596620+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:46.360051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590974+0000","last_change":"2023-10-16T10:03:22.591961+0000","last_active":"2023-10-16T10:03:23.590974+0000","last_peered":"2023-10-16T10:03:23.590974+0000","last_clean":"2023-10-16T10:03:23.590974+0000","last_became_active":"2023-10-16T10:03:22.591126+0000","last_became_peered":"2023-10-16T10:03:22.591126+0000","last_unstale":"2023-10-16T10:03:23.590974+0000","last_undegraded":"2023-10-16T10:03:23.590974+0000","last_fullsized":"2023-10-16T10:03:23.590974+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:18.574504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590749+0000","last_change":"2023-10-16T10:03:22.598184+0000","last_active":"2023-10-16T10:03:23.590749+0000","last_peered":"2023-10-16T10:03:23.590749+0000","last_clean":"2023-10-16T10:03:23.590749+0000","last_became_active":"2023-10-16T10:03:22.597903+0000","last_became_peered":"2023-10-16T10:03:22.597903+0000","last_unstale":"2023-10-16T10:03:23.590749+0000","last_undegraded":"2023-10-16T10:03:23.590749+0000","last_fullsized":"2023-10-16T10:03:23.590749+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:23.829971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586829+0000","last_change":"2023-10-16T10:03:22.580762+0000","last_active":"2023-10-16T10:03:23.586829+0000","last_peered":"2023-10-16T10:03:23.586829+0000","last_clean":"2023-10-16T10:03:23.586829+0000","last_became_active":"2023-10-16T10:03:22.580463+0000","last_became_peered":"2023-10-16T10:03:22.580463+0000","last_unstale":"2023-10-16T10:03:23.586829+0000","last_undegraded":"2023-10-16T10:03:23.586829+0000","last_fullsized":"2023-10-16T10:03:23.586829+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:35.486946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.598306+0000","last_change":"2023-10-16T10:03:22.595486+0000","last_active":"2023-10-16T10:03:22.598306+0000","last_peered":"2023-10-16T10:03:22.598306+0000","last_clean":"2023-10-16T10:03:22.598306+0000","last_became_active":"2023-10-16T10:03:22.595032+0000","last_became_peered":"2023-10-16T10:03:22.595032+0000","last_unstale":"2023-10-16T10:03:22.598306+0000","last_undegraded":"2023-10-16T10:03:22.598306+0000","last_fullsized":"2023-10-16T10:03:22.598306+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:04:35.394523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588806+0000","last_change":"2023-10-16T10:03:22.579943+0000","last_active":"2023-10-16T10:03:23.588806+0000","last_peered":"2023-10-16T10:03:23.588806+0000","last_clean":"2023-10-16T10:03:23.588806+0000","last_became_active":"2023-10-16T10:03:22.579647+0000","last_became_peered":"2023-10-16T10:03:22.579647+0000","last_unstale":"2023-10-16T10:03:23.588806+0000","last_undegraded":"2023-10-16T10:03:23.588806+0000","last_fullsized":"2023-10-16T10:03:23.588806+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:10.511924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.583018+0000","last_change":"2023-10-16T10:03:22.582433+0000","last_active":"2023-10-16T10:03:22.583018+0000","last_peered":"2023-10-16T10:03:22.583018+0000","last_clean":"2023-10-16T10:03:22.583018+0000","last_became_active":"2023-10-16T10:03:22.582075+0000","last_became_peered":"2023-10-16T10:03:22.582075+0000","last_unstale":"2023-10-16T10:03:22.583018+0000","last_undegraded":"2023-10-16T10:03:22.583018+0000","last_fullsized":"2023-10-16T10:03:22.583018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:27.301267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591326+0000","last_change":"2023-10-16T10:03:22.591369+0000","last_active":"2023-10-16T10:03:23.591326+0000","last_peered":"2023-10-16T10:03:23.591326+0000","last_clean":"2023-10-16T10:03:23.591326+0000","last_became_active":"2023-10-16T10:03:22.591094+0000","last_became_peered":"2023-10-16T10:03:22.591094+0000","last_unstale":"2023-10-16T10:03:23.591326+0000","last_undegraded":"2023-10-16T10:03:23.591326+0000","last_fullsized":"2023-10-16T10:03:23.591326+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:49.994100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588862+0000","last_change":"2023-10-16T10:03:22.592099+0000","last_active":"2023-10-16T10:03:23.588862+0000","last_peered":"2023-10-16T10:03:23.588862+0000","last_clean":"2023-10-16T10:03:23.588862+0000","last_became_active":"2023-10-16T10:03:22.591815+0000","last_became_peered":"2023-10-16T10:03:22.591815+0000","last_unstale":"2023-10-16T10:03:23.588862+0000","last_undegraded":"2023-10-16T10:03:23.588862+0000","last_fullsized":"2023-10-16T10:03:23.588862+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:31:13.453159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588841+0000","last_change":"2023-10-16T10:03:22.586396+0000","last_active":"2023-10-16T10:03:23.588841+0000","last_peered":"2023-10-16T10:03:23.588841+0000","last_clean":"2023-10-16T10:03:23.588841+0000","last_became_active":"2023-10-16T10:03:22.586024+0000","last_became_peered":"2023-10-16T10:03:22.586024+0000","last_unstale":"2023-10-16T10:03:23.588841+0000","last_undegraded":"2023-10-16T10:03:23.588841+0000","last_fullsized":"2023-10-16T10:03:23.588841+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:44:56.446872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589520+0000","last_change":"2023-10-16T10:03:22.584011+0000","last_active":"2023-10-16T10:03:22.589520+0000","last_peered":"2023-10-16T10:03:22.589520+0000","last_clean":"2023-10-16T10:03:22.589520+0000","last_became_active":"2023-10-16T10:03:22.583768+0000","last_became_peered":"2023-10-16T10:03:22.583768+0000","last_unstale":"2023-10-16T10:03:22.589520+0000","last_undegraded":"2023-10-16T10:03:22.589520+0000","last_fullsized":"2023-10-16T10:03:22.589520+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:23.829971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588461+0000","last_change":"2023-10-16T10:03:22.588560+0000","last_active":"2023-10-16T10:03:23.588461+0000","last_peered":"2023-10-16T10:03:23.588461+0000","last_clean":"2023-10-16T10:03:23.588461+0000","last_became_active":"2023-10-16T10:03:22.588269+0000","last_became_peered":"2023-10-16T10:03:22.588269+0000","last_unstale":"2023-10-16T10:03:23.588461+0000","last_undegraded":"2023-10-16T10:03:23.588461+0000","last_fullsized":"2023-10-16T10:03:23.588461+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:10:15.522323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594226+0000","last_change":"2023-10-16T10:03:22.592213+0000","last_active":"2023-10-16T10:03:22.594226+0000","last_peered":"2023-10-16T10:03:22.594226+0000","last_clean":"2023-10-16T10:03:22.594226+0000","last_became_active":"2023-10-16T10:03:22.591948+0000","last_became_peered":"2023-10-16T10:03:22.591948+0000","last_unstale":"2023-10-16T10:03:22.594226+0000","last_undegraded":"2023-10-16T10:03:22.594226+0000","last_fullsized":"2023-10-16T10:03:22.594226+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:49.642486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588951+0000","last_change":"2023-10-16T10:03:22.582511+0000","last_active":"2023-10-16T10:03:23.588951+0000","last_peered":"2023-10-16T10:03:23.588951+0000","last_clean":"2023-10-16T10:03:23.588951+0000","last_became_active":"2023-10-16T10:03:22.582200+0000","last_became_peered":"2023-10-16T10:03:22.582200+0000","last_unstale":"2023-10-16T10:03:23.588951+0000","last_undegraded":"2023-10-16T10:03:23.588951+0000","last_fullsized":"2023-10-16T10:03:23.588951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:20.241725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.593984+0000","last_change":"2023-10-16T10:03:22.587275+0000","last_active":"2023-10-16T10:03:22.593984+0000","last_peered":"2023-10-16T10:03:22.593984+0000","last_clean":"2023-10-16T10:03:22.593984+0000","last_became_active":"2023-10-16T10:03:22.586967+0000","last_became_peered":"2023-10-16T10:03:22.586967+0000","last_unstale":"2023-10-16T10:03:22.593984+0000","last_undegraded":"2023-10-16T10:03:22.593984+0000","last_fullsized":"2023-10-16T10:03:22.593984+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:09.469320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587046+0000","last_change":"2023-10-16T10:03:22.583046+0000","last_active":"2023-10-16T10:03:23.587046+0000","last_peered":"2023-10-16T10:03:23.587046+0000","last_clean":"2023-10-16T10:03:23.587046+0000","last_became_active":"2023-10-16T10:03:22.582801+0000","last_became_peered":"2023-10-16T10:03:22.582801+0000","last_unstale":"2023-10-16T10:03:23.587046+0000","last_undegraded":"2023-10-16T10:03:23.587046+0000","last_fullsized":"2023-10-16T10:03:23.587046+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:07.056578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589804+0000","last_change":"2023-10-16T10:03:22.588449+0000","last_active":"2023-10-16T10:03:22.589804+0000","last_peered":"2023-10-16T10:03:22.589804+0000","last_clean":"2023-10-16T10:03:22.589804+0000","last_became_active":"2023-10-16T10:03:22.588166+0000","last_became_peered":"2023-10-16T10:03:22.588166+0000","last_unstale":"2023-10-16T10:03:22.589804+0000","last_undegraded":"2023-10-16T10:03:22.589804+0000","last_fullsized":"2023-10-16T10:03:22.589804+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:13.231204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588928+0000","last_change":"2023-10-16T10:03:22.596965+0000","last_active":"2023-10-16T10:03:23.588928+0000","last_peered":"2023-10-16T10:03:23.588928+0000","last_clean":"2023-10-16T10:03:23.588928+0000","last_became_active":"2023-10-16T10:03:22.596493+0000","last_became_peered":"2023-10-16T10:03:22.596493+0000","last_unstale":"2023-10-16T10:03:23.588928+0000","last_undegraded":"2023-10-16T10:03:23.588928+0000","last_fullsized":"2023-10-16T10:03:23.588928+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:55.997051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591087+0000","last_change":"2023-10-16T10:03:22.595265+0000","last_active":"2023-10-16T10:03:23.591087+0000","last_peered":"2023-10-16T10:03:23.591087+0000","last_clean":"2023-10-16T10:03:23.591087+0000","last_became_active":"2023-10-16T10:03:22.594990+0000","last_became_peered":"2023-10-16T10:03:22.594990+0000","last_unstale":"2023-10-16T10:03:23.591087+0000","last_undegraded":"2023-10-16T10:03:23.591087+0000","last_fullsized":"2023-10-16T10:03:23.591087+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:03:25.935592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591662+0000","last_change":"2023-10-16T10:03:22.582245+0000","last_active":"2023-10-16T10:03:22.591662+0000","last_peered":"2023-10-16T10:03:22.591662+0000","last_clean":"2023-10-16T10:03:22.591662+0000","last_became_active":"2023-10-16T10:03:22.581981+0000","last_became_peered":"2023-10-16T10:03:22.581981+0000","last_unstale":"2023-10-16T10:03:22.591662+0000","last_undegraded":"2023-10-16T10:03:22.591662+0000","last_fullsized":"2023-10-16T10:03:22.591662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:05.704094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586723+0000","last_change":"2023-10-16T10:03:22.581258+0000","last_active":"2023-10-16T10:03:23.586723+0000","last_peered":"2023-10-16T10:03:23.586723+0000","last_clean":"2023-10-16T10:03:23.586723+0000","last_became_active":"2023-10-16T10:03:22.580924+0000","last_became_peered":"2023-10-16T10:03:22.580924+0000","last_unstale":"2023-10-16T10:03:23.586723+0000","last_undegraded":"2023-10-16T10:03:23.586723+0000","last_fullsized":"2023-10-16T10:03:23.586723+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:48.591325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587226+0000","last_change":"2023-10-16T10:03:22.587072+0000","last_active":"2023-10-16T10:03:23.587226+0000","last_peered":"2023-10-16T10:03:23.587226+0000","last_clean":"2023-10-16T10:03:23.587226+0000","last_became_active":"2023-10-16T10:03:22.586853+0000","last_became_peered":"2023-10-16T10:03:22.586853+0000","last_unstale":"2023-10-16T10:03:23.587226+0000","last_undegraded":"2023-10-16T10:03:23.587226+0000","last_fullsized":"2023-10-16T10:03:23.587226+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:05.704094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"131'4","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591528+0000","last_change":"2023-10-16T10:03:21.584969+0000","last_active":"2023-10-16T10:03:23.591528+0000","last_peered":"2023-10-16T10:03:23.591528+0000","last_clean":"2023-10-16T10:03:23.591528+0000","last_became_active":"2023-10-16T10:03:21.583402+0000","last_became_peered":"2023-10-16T10:03:21.583402+0000","last_unstale":"2023-10-16T10:03:23.591528+0000","last_undegraded":"2023-10-16T10:03:23.591528+0000","last_fullsized":"2023-10-16T10:03:23.591528+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:07:57.163494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.597580+0000","last_change":"2023-10-16T10:03:22.593817+0000","last_active":"2023-10-16T10:03:22.597580+0000","last_peered":"2023-10-16T10:03:22.597580+0000","last_clean":"2023-10-16T10:03:22.597580+0000","last_became_active":"2023-10-16T10:03:22.593536+0000","last_became_peered":"2023-10-16T10:03:22.593536+0000","last_unstale":"2023-10-16T10:03:22.597580+0000","last_undegraded":"2023-10-16T10:03:22.597580+0000","last_fullsized":"2023-10-16T10:03:22.597580+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:01.334353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"131'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591237+0000","last_change":"2023-10-16T10:03:21.589233+0000","last_active":"2023-10-16T10:03:23.591237+0000","last_peered":"2023-10-16T10:03:23.591237+0000","last_clean":"2023-10-16T10:03:23.591237+0000","last_became_active":"2023-10-16T10:03:21.588904+0000","last_became_peered":"2023-10-16T10:03:21.588904+0000","last_unstale":"2023-10-16T10:03:23.591237+0000","last_undegraded":"2023-10-16T10:03:23.591237+0000","last_fullsized":"2023-10-16T10:03:23.591237+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:07.058246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587138+0000","last_change":"2023-10-16T10:03:22.580778+0000","last_active":"2023-10-16T10:03:23.587138+0000","last_peered":"2023-10-16T10:03:23.587138+0000","last_clean":"2023-10-16T10:03:23.587138+0000","last_became_active":"2023-10-16T10:03:22.580451+0000","last_became_peered":"2023-10-16T10:03:22.580451+0000","last_unstale":"2023-10-16T10:03:23.587138+0000","last_undegraded":"2023-10-16T10:03:23.587138+0000","last_fullsized":"2023-10-16T10:03:23.587138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.864772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677810+0000","last_change":"2023-10-16T10:03:21.588947+0000","last_active":"2023-10-16T10:03:23.677810+0000","last_peered":"2023-10-16T10:03:23.677810+0000","last_clean":"2023-10-16T10:03:23.677810+0000","last_became_active":"2023-10-16T10:03:21.588675+0000","last_became_peered":"2023-10-16T10:03:21.588675+0000","last_unstale":"2023-10-16T10:03:23.677810+0000","last_undegraded":"2023-10-16T10:03:23.677810+0000","last_fullsized":"2023-10-16T10:03:23.677810+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:43.064912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591417+0000","last_change":"2023-10-16T10:03:22.592680+0000","last_active":"2023-10-16T10:03:23.591417+0000","last_peered":"2023-10-16T10:03:23.591417+0000","last_clean":"2023-10-16T10:03:23.591417+0000","last_became_active":"2023-10-16T10:03:22.592383+0000","last_became_peered":"2023-10-16T10:03:22.592383+0000","last_unstale":"2023-10-16T10:03:23.591417+0000","last_undegraded":"2023-10-16T10:03:23.591417+0000","last_fullsized":"2023-10-16T10:03:23.591417+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:41:39.434739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677721+0000","last_change":"2023-10-16T10:03:21.579770+0000","last_active":"2023-10-16T10:03:23.677721+0000","last_peered":"2023-10-16T10:03:23.677721+0000","last_clean":"2023-10-16T10:03:23.677721+0000","last_became_active":"2023-10-16T10:03:21.579493+0000","last_became_peered":"2023-10-16T10:03:21.579493+0000","last_unstale":"2023-10-16T10:03:23.677721+0000","last_undegraded":"2023-10-16T10:03:23.677721+0000","last_fullsized":"2023-10-16T10:03:23.677721+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:48.890166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591415+0000","last_change":"2023-10-16T10:03:22.582323+0000","last_active":"2023-10-16T10:03:22.591415+0000","last_peered":"2023-10-16T10:03:22.591415+0000","last_clean":"2023-10-16T10:03:22.591415+0000","last_became_active":"2023-10-16T10:03:22.582062+0000","last_became_peered":"2023-10-16T10:03:22.582062+0000","last_unstale":"2023-10-16T10:03:22.591415+0000","last_undegraded":"2023-10-16T10:03:22.591415+0000","last_fullsized":"2023-10-16T10:03:22.591415+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:37.628308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"131'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590726+0000","last_change":"2023-10-16T10:03:21.587933+0000","last_active":"2023-10-16T10:03:23.590726+0000","last_peered":"2023-10-16T10:03:23.590726+0000","last_clean":"2023-10-16T10:03:23.590726+0000","last_became_active":"2023-10-16T10:03:21.587648+0000","last_became_peered":"2023-10-16T10:03:21.587648+0000","last_unstale":"2023-10-16T10:03:23.590726+0000","last_undegraded":"2023-10-16T10:03:23.590726+0000","last_fullsized":"2023-10-16T10:03:23.590726+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:18.440663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.589007+0000","last_change":"2023-10-16T10:03:22.586053+0000","last_active":"2023-10-16T10:03:23.589007+0000","last_peered":"2023-10-16T10:03:23.589007+0000","last_clean":"2023-10-16T10:03:23.589007+0000","last_became_active":"2023-10-16T10:03:22.585237+0000","last_became_peered":"2023-10-16T10:03:22.585237+0000","last_unstale":"2023-10-16T10:03:23.589007+0000","last_undegraded":"2023-10-16T10:03:23.589007+0000","last_fullsized":"2023-10-16T10:03:23.589007+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:53.304341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"131'1","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.595632+0000","last_change":"2023-10-16T10:03:21.587344+0000","last_active":"2023-10-16T10:03:22.595632+0000","last_peered":"2023-10-16T10:03:22.595632+0000","last_clean":"2023-10-16T10:03:22.595632+0000","last_became_active":"2023-10-16T10:03:21.586995+0000","last_became_peered":"2023-10-16T10:03:21.586995+0000","last_unstale":"2023-10-16T10:03:22.595632+0000","last_undegraded":"2023-10-16T10:03:22.595632+0000","last_fullsized":"2023-10-16T10:03:22.595632+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:28.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591528+0000","last_change":"2023-10-16T10:03:22.582908+0000","last_active":"2023-10-16T10:03:22.591528+0000","last_peered":"2023-10-16T10:03:22.591528+0000","last_clean":"2023-10-16T10:03:22.591528+0000","last_became_active":"2023-10-16T10:03:22.581688+0000","last_became_peered":"2023-10-16T10:03:22.581688+0000","last_unstale":"2023-10-16T10:03:22.591528+0000","last_undegraded":"2023-10-16T10:03:22.591528+0000","last_fullsized":"2023-10-16T10:03:22.591528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:55.535256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588493+0000","last_change":"2023-10-16T10:03:21.584214+0000","last_active":"2023-10-16T10:03:23.588493+0000","last_peered":"2023-10-16T10:03:23.588493+0000","last_clean":"2023-10-16T10:03:23.588493+0000","last_became_active":"2023-10-16T10:03:21.583916+0000","last_became_peered":"2023-10-16T10:03:21.583916+0000","last_unstale":"2023-10-16T10:03:23.588493+0000","last_undegraded":"2023-10-16T10:03:23.588493+0000","last_fullsized":"2023-10-16T10:03:23.588493+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:11:29.336207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591318+0000","last_change":"2023-10-16T10:03:22.582983+0000","last_active":"2023-10-16T10:03:23.591318+0000","last_peered":"2023-10-16T10:03:23.591318+0000","last_clean":"2023-10-16T10:03:23.591318+0000","last_became_active":"2023-10-16T10:03:22.582341+0000","last_became_peered":"2023-10-16T10:03:22.582341+0000","last_unstale":"2023-10-16T10:03:23.591318+0000","last_undegraded":"2023-10-16T10:03:23.591318+0000","last_fullsized":"2023-10-16T10:03:23.591318+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:35.573344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578624+0000","last_change":"2023-10-16T10:03:21.592508+0000","last_active":"2023-10-16T10:03:22.578624+0000","last_peered":"2023-10-16T10:03:22.578624+0000","last_clean":"2023-10-16T10:03:22.578624+0000","last_became_active":"2023-10-16T10:03:21.592231+0000","last_became_peered":"2023-10-16T10:03:21.592231+0000","last_unstale":"2023-10-16T10:03:22.578624+0000","last_undegraded":"2023-10-16T10:03:22.578624+0000","last_fullsized":"2023-10-16T10:03:22.578624+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:48:54.635819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591565+0000","last_change":"2023-10-16T10:03:22.586534+0000","last_active":"2023-10-16T10:03:23.591565+0000","last_peered":"2023-10-16T10:03:23.591565+0000","last_clean":"2023-10-16T10:03:23.591565+0000","last_became_active":"2023-10-16T10:03:22.586245+0000","last_became_peered":"2023-10-16T10:03:22.586245+0000","last_unstale":"2023-10-16T10:03:23.591565+0000","last_undegraded":"2023-10-16T10:03:23.591565+0000","last_fullsized":"2023-10-16T10:03:23.591565+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:30.872243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677749+0000","last_change":"2023-10-16T10:03:21.584838+0000","last_active":"2023-10-16T10:03:23.677749+0000","last_peered":"2023-10-16T10:03:23.677749+0000","last_clean":"2023-10-16T10:03:23.677749+0000","last_became_active":"2023-10-16T10:03:21.584336+0000","last_became_peered":"2023-10-16T10:03:21.584336+0000","last_unstale":"2023-10-16T10:03:23.677749+0000","last_undegraded":"2023-10-16T10:03:23.677749+0000","last_fullsized":"2023-10-16T10:03:23.677749+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:26.293942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.597077+0000","last_change":"2023-10-16T10:03:22.593020+0000","last_active":"2023-10-16T10:03:22.597077+0000","last_peered":"2023-10-16T10:03:22.597077+0000","last_clean":"2023-10-16T10:03:22.597077+0000","last_became_active":"2023-10-16T10:03:22.592763+0000","last_became_peered":"2023-10-16T10:03:22.592763+0000","last_unstale":"2023-10-16T10:03:22.597077+0000","last_undegraded":"2023-10-16T10:03:22.597077+0000","last_fullsized":"2023-10-16T10:03:22.597077+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:43.600158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578285+0000","last_change":"2023-10-16T10:03:21.586835+0000","last_active":"2023-10-16T10:03:22.578285+0000","last_peered":"2023-10-16T10:03:22.578285+0000","last_clean":"2023-10-16T10:03:22.578285+0000","last_became_active":"2023-10-16T10:03:21.586569+0000","last_became_peered":"2023-10-16T10:03:21.586569+0000","last_unstale":"2023-10-16T10:03:22.578285+0000","last_undegraded":"2023-10-16T10:03:22.578285+0000","last_fullsized":"2023-10-16T10:03:22.578285+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:16.350674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589175+0000","last_change":"2023-10-16T10:03:22.585921+0000","last_active":"2023-10-16T10:03:22.589175+0000","last_peered":"2023-10-16T10:03:22.589175+0000","last_clean":"2023-10-16T10:03:22.589175+0000","last_became_active":"2023-10-16T10:03:22.585642+0000","last_became_peered":"2023-10-16T10:03:22.585642+0000","last_unstale":"2023-10-16T10:03:22.589175+0000","last_undegraded":"2023-10-16T10:03:22.589175+0000","last_fullsized":"2023-10-16T10:03:22.589175+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:05:44.243861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"131'3","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.600113+0000","last_change":"2023-10-16T10:03:21.589534+0000","last_active":"2023-10-16T10:03:22.600113+0000","last_peered":"2023-10-16T10:03:22.600113+0000","last_clean":"2023-10-16T10:03:22.600113+0000","last_became_active":"2023-10-16T10:03:21.589249+0000","last_became_peered":"2023-10-16T10:03:21.589249+0000","last_unstale":"2023-10-16T10:03:22.600113+0000","last_undegraded":"2023-10-16T10:03:22.600113+0000","last_fullsized":"2023-10-16T10:03:22.600113+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:39.709740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591170+0000","last_change":"2023-10-16T10:03:22.596384+0000","last_active":"2023-10-16T10:03:23.591170+0000","last_peered":"2023-10-16T10:03:23.591170+0000","last_clean":"2023-10-16T10:03:23.591170+0000","last_became_active":"2023-10-16T10:03:22.594202+0000","last_became_peered":"2023-10-16T10:03:22.594202+0000","last_unstale":"2023-10-16T10:03:23.591170+0000","last_undegraded":"2023-10-16T10:03:23.591170+0000","last_fullsized":"2023-10-16T10:03:23.591170+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:32.157094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"131'6","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.602828+0000","last_change":"2023-10-16T10:03:21.592585+0000","last_active":"2023-10-16T10:03:22.602828+0000","last_peered":"2023-10-16T10:03:22.602828+0000","last_clean":"2023-10-16T10:03:22.602828+0000","last_became_active":"2023-10-16T10:03:21.592189+0000","last_became_peered":"2023-10-16T10:03:21.592189+0000","last_unstale":"2023-10-16T10:03:22.602828+0000","last_undegraded":"2023-10-16T10:03:22.602828+0000","last_fullsized":"2023-10-16T10:03:22.602828+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:06.049254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589501+0000","last_change":"2023-10-16T10:03:22.582114+0000","last_active":"2023-10-16T10:03:22.589501+0000","last_peered":"2023-10-16T10:03:22.589501+0000","last_clean":"2023-10-16T10:03:22.589501+0000","last_became_active":"2023-10-16T10:03:22.581862+0000","last_became_peered":"2023-10-16T10:03:22.581862+0000","last_unstale":"2023-10-16T10:03:22.589501+0000","last_undegraded":"2023-10-16T10:03:22.589501+0000","last_fullsized":"2023-10-16T10:03:22.589501+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:15.351038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586953+0000","last_change":"2023-10-16T10:03:21.586742+0000","last_active":"2023-10-16T10:03:23.586953+0000","last_peered":"2023-10-16T10:03:23.586953+0000","last_clean":"2023-10-16T10:03:23.586953+0000","last_became_active":"2023-10-16T10:03:21.586485+0000","last_became_peered":"2023-10-16T10:03:21.586485+0000","last_unstale":"2023-10-16T10:03:23.586953+0000","last_undegraded":"2023-10-16T10:03:23.586953+0000","last_fullsized":"2023-10-16T10:03:23.586953+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:39.709740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594629+0000","last_change":"2023-10-16T10:03:22.592239+0000","last_active":"2023-10-16T10:03:22.594629+0000","last_peered":"2023-10-16T10:03:22.594629+0000","last_clean":"2023-10-16T10:03:22.594629+0000","last_became_active":"2023-10-16T10:03:22.591965+0000","last_became_peered":"2023-10-16T10:03:22.591965+0000","last_unstale":"2023-10-16T10:03:22.594629+0000","last_undegraded":"2023-10-16T10:03:22.594629+0000","last_fullsized":"2023-10-16T10:03:22.594629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:39.069792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.572990+0000","last_change":"2023-10-16T10:03:21.585684+0000","last_active":"2023-10-16T10:03:22.572990+0000","last_peered":"2023-10-16T10:03:22.572990+0000","last_clean":"2023-10-16T10:03:22.572990+0000","last_became_active":"2023-10-16T10:03:21.585337+0000","last_became_peered":"2023-10-16T10:03:21.585337+0000","last_unstale":"2023-10-16T10:03:22.572990+0000","last_undegraded":"2023-10-16T10:03:22.572990+0000","last_fullsized":"2023-10-16T10:03:22.572990+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:38.220519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589839+0000","last_change":"2023-10-16T10:03:22.585877+0000","last_active":"2023-10-16T10:03:22.589839+0000","last_peered":"2023-10-16T10:03:22.589839+0000","last_clean":"2023-10-16T10:03:22.589839+0000","last_became_active":"2023-10-16T10:03:22.585606+0000","last_became_peered":"2023-10-16T10:03:22.585606+0000","last_unstale":"2023-10-16T10:03:22.589839+0000","last_undegraded":"2023-10-16T10:03:22.589839+0000","last_fullsized":"2023-10-16T10:03:22.589839+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:48.130668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"131'8","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677539+0000","last_change":"2023-10-16T10:03:21.587389+0000","last_active":"2023-10-16T10:03:23.677539+0000","last_peered":"2023-10-16T10:03:23.677539+0000","last_clean":"2023-10-16T10:03:23.677539+0000","last_became_active":"2023-10-16T10:03:21.587083+0000","last_became_peered":"2023-10-16T10:03:21.587083+0000","last_unstale":"2023-10-16T10:03:23.677539+0000","last_undegraded":"2023-10-16T10:03:23.677539+0000","last_fullsized":"2023-10-16T10:03:23.677539+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:00.327029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677897+0000","last_change":"2023-10-16T10:03:22.595487+0000","last_active":"2023-10-16T10:03:23.677897+0000","last_peered":"2023-10-16T10:03:23.677897+0000","last_clean":"2023-10-16T10:03:23.677897+0000","last_became_active":"2023-10-16T10:03:22.595186+0000","last_became_peered":"2023-10-16T10:03:22.595186+0000","last_unstale":"2023-10-16T10:03:23.677897+0000","last_undegraded":"2023-10-16T10:03:23.677897+0000","last_fullsized":"2023-10-16T10:03:23.677897+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:37.628308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"131'1","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.600261+0000","last_change":"2023-10-16T10:03:21.589801+0000","last_active":"2023-10-16T10:03:22.600261+0000","last_peered":"2023-10-16T10:03:22.600261+0000","last_clean":"2023-10-16T10:03:22.600261+0000","last_became_active":"2023-10-16T10:03:21.589537+0000","last_became_peered":"2023-10-16T10:03:21.589537+0000","last_unstale":"2023-10-16T10:03:22.600261+0000","last_undegraded":"2023-10-16T10:03:22.600261+0000","last_fullsized":"2023-10-16T10:03:22.600261+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:30.503826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587176+0000","last_change":"2023-10-16T10:03:22.594096+0000","last_active":"2023-10-16T10:03:23.587176+0000","last_peered":"2023-10-16T10:03:23.587176+0000","last_clean":"2023-10-16T10:03:23.587176+0000","last_became_active":"2023-10-16T10:03:22.593826+0000","last_became_peered":"2023-10-16T10:03:22.593826+0000","last_unstale":"2023-10-16T10:03:23.587176+0000","last_undegraded":"2023-10-16T10:03:23.587176+0000","last_fullsized":"2023-10-16T10:03:23.587176+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:26.494959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677873+0000","last_change":"2023-10-16T10:03:21.579011+0000","last_active":"2023-10-16T10:03:23.677873+0000","last_peered":"2023-10-16T10:03:23.677873+0000","last_clean":"2023-10-16T10:03:23.677873+0000","last_became_active":"2023-10-16T10:03:21.578747+0000","last_became_peered":"2023-10-16T10:03:21.578747+0000","last_unstale":"2023-10-16T10:03:23.677873+0000","last_undegraded":"2023-10-16T10:03:23.677873+0000","last_fullsized":"2023-10-16T10:03:23.677873+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:12.781127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591214+0000","last_change":"2023-10-16T10:03:22.598884+0000","last_active":"2023-10-16T10:03:23.591214+0000","last_peered":"2023-10-16T10:03:23.591214+0000","last_clean":"2023-10-16T10:03:23.591214+0000","last_became_active":"2023-10-16T10:03:22.598597+0000","last_became_peered":"2023-10-16T10:03:22.598597+0000","last_unstale":"2023-10-16T10:03:23.591214+0000","last_undegraded":"2023-10-16T10:03:23.591214+0000","last_fullsized":"2023-10-16T10:03:23.591214+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:07.391180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"131'1","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596720+0000","last_change":"2023-10-16T10:03:21.584346+0000","last_active":"2023-10-16T10:03:22.596720+0000","last_peered":"2023-10-16T10:03:22.596720+0000","last_clean":"2023-10-16T10:03:22.596720+0000","last_became_active":"2023-10-16T10:03:21.583862+0000","last_became_peered":"2023-10-16T10:03:21.583862+0000","last_unstale":"2023-10-16T10:03:22.596720+0000","last_undegraded":"2023-10-16T10:03:22.596720+0000","last_fullsized":"2023-10-16T10:03:22.596720+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:12.781127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591472+0000","last_change":"2023-10-16T10:03:22.597792+0000","last_active":"2023-10-16T10:03:23.591472+0000","last_peered":"2023-10-16T10:03:23.591472+0000","last_clean":"2023-10-16T10:03:23.591472+0000","last_became_active":"2023-10-16T10:03:22.597489+0000","last_became_peered":"2023-10-16T10:03:22.597489+0000","last_unstale":"2023-10-16T10:03:23.591472+0000","last_undegraded":"2023-10-16T10:03:23.591472+0000","last_fullsized":"2023-10-16T10:03:23.591472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:51.200376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590936+0000","last_change":"2023-10-16T10:03:21.591458+0000","last_active":"2023-10-16T10:03:23.590936+0000","last_peered":"2023-10-16T10:03:23.590936+0000","last_clean":"2023-10-16T10:03:23.590936+0000","last_became_active":"2023-10-16T10:03:21.591159+0000","last_became_peered":"2023-10-16T10:03:21.591159+0000","last_unstale":"2023-10-16T10:03:23.590936+0000","last_undegraded":"2023-10-16T10:03:23.590936+0000","last_fullsized":"2023-10-16T10:03:23.590936+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:02.548562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591589+0000","last_change":"2023-10-16T10:03:22.591974+0000","last_active":"2023-10-16T10:03:23.591589+0000","last_peered":"2023-10-16T10:03:23.591589+0000","last_clean":"2023-10-16T10:03:23.591589+0000","last_became_active":"2023-10-16T10:03:22.591671+0000","last_became_peered":"2023-10-16T10:03:22.591671+0000","last_unstale":"2023-10-16T10:03:23.591589+0000","last_undegraded":"2023-10-16T10:03:23.591589+0000","last_fullsized":"2023-10-16T10:03:23.591589+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:13.996226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"131'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.587039+0000","last_change":"2023-10-16T10:03:21.588926+0000","last_active":"2023-10-16T10:03:23.587039+0000","last_peered":"2023-10-16T10:03:23.587039+0000","last_clean":"2023-10-16T10:03:23.587039+0000","last_became_active":"2023-10-16T10:03:21.588670+0000","last_became_peered":"2023-10-16T10:03:21.588670+0000","last_unstale":"2023-10-16T10:03:23.587039+0000","last_undegraded":"2023-10-16T10:03:23.587039+0000","last_fullsized":"2023-10-16T10:03:23.587039+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:04:34.387199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.598235+0000","last_change":"2023-10-16T10:03:22.595087+0000","last_active":"2023-10-16T10:03:22.598235+0000","last_peered":"2023-10-16T10:03:22.598235+0000","last_clean":"2023-10-16T10:03:22.598235+0000","last_became_active":"2023-10-16T10:03:22.594827+0000","last_became_peered":"2023-10-16T10:03:22.594827+0000","last_unstale":"2023-10-16T10:03:22.598235+0000","last_undegraded":"2023-10-16T10:03:22.598235+0000","last_fullsized":"2023-10-16T10:03:22.598235+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:47.180693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677602+0000","last_change":"2023-10-16T10:03:21.584284+0000","last_active":"2023-10-16T10:03:23.677602+0000","last_peered":"2023-10-16T10:03:23.677602+0000","last_clean":"2023-10-16T10:03:23.677602+0000","last_became_active":"2023-10-16T10:03:21.583749+0000","last_became_peered":"2023-10-16T10:03:21.583749+0000","last_unstale":"2023-10-16T10:03:23.677602+0000","last_undegraded":"2023-10-16T10:03:23.677602+0000","last_fullsized":"2023-10-16T10:03:23.677602+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:07.659994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590788+0000","last_change":"2023-10-16T10:03:22.596574+0000","last_active":"2023-10-16T10:03:23.590788+0000","last_peered":"2023-10-16T10:03:23.590788+0000","last_clean":"2023-10-16T10:03:23.590788+0000","last_became_active":"2023-10-16T10:03:22.596286+0000","last_became_peered":"2023-10-16T10:03:22.596286+0000","last_unstale":"2023-10-16T10:03:23.590788+0000","last_undegraded":"2023-10-16T10:03:23.590788+0000","last_fullsized":"2023-10-16T10:03:23.590788+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:18:29.366054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.586765+0000","last_change":"2023-10-16T10:03:21.588529+0000","last_active":"2023-10-16T10:03:23.586765+0000","last_peered":"2023-10-16T10:03:23.586765+0000","last_clean":"2023-10-16T10:03:23.586765+0000","last_became_active":"2023-10-16T10:03:21.588264+0000","last_became_peered":"2023-10-16T10:03:21.588264+0000","last_unstale":"2023-10-16T10:03:23.586765+0000","last_undegraded":"2023-10-16T10:03:23.586765+0000","last_fullsized":"2023-10-16T10:03:23.586765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:25.710723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.588605+0000","last_change":"2023-10-16T10:03:22.582928+0000","last_active":"2023-10-16T10:03:22.588605+0000","last_peered":"2023-10-16T10:03:22.588605+0000","last_clean":"2023-10-16T10:03:22.588605+0000","last_became_active":"2023-10-16T10:03:22.582671+0000","last_became_peered":"2023-10-16T10:03:22.582671+0000","last_unstale":"2023-10-16T10:03:22.588605+0000","last_undegraded":"2023-10-16T10:03:22.588605+0000","last_fullsized":"2023-10-16T10:03:22.588605+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:11:34.692527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578786+0000","last_change":"2023-10-16T10:03:21.586147+0000","last_active":"2023-10-16T10:03:22.578786+0000","last_peered":"2023-10-16T10:03:22.578786+0000","last_clean":"2023-10-16T10:03:22.578786+0000","last_became_active":"2023-10-16T10:03:21.585611+0000","last_became_peered":"2023-10-16T10:03:21.585611+0000","last_unstale":"2023-10-16T10:03:22.578786+0000","last_undegraded":"2023-10-16T10:03:22.578786+0000","last_fullsized":"2023-10-16T10:03:22.578786+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:11.852190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589643+0000","last_change":"2023-10-16T10:03:22.588355+0000","last_active":"2023-10-16T10:03:22.589643+0000","last_peered":"2023-10-16T10:03:22.589643+0000","last_clean":"2023-10-16T10:03:22.589643+0000","last_became_active":"2023-10-16T10:03:22.587985+0000","last_became_peered":"2023-10-16T10:03:22.587985+0000","last_unstale":"2023-10-16T10:03:22.589643+0000","last_undegraded":"2023-10-16T10:03:22.589643+0000","last_fullsized":"2023-10-16T10:03:22.589643+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:54.469586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"131'2","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.597766+0000","last_change":"2023-10-16T10:03:21.589473+0000","last_active":"2023-10-16T10:03:22.597766+0000","last_peered":"2023-10-16T10:03:22.597766+0000","last_clean":"2023-10-16T10:03:22.597766+0000","last_became_active":"2023-10-16T10:03:21.589166+0000","last_became_peered":"2023-10-16T10:03:21.589166+0000","last_unstale":"2023-10-16T10:03:22.597766+0000","last_undegraded":"2023-10-16T10:03:22.597766+0000","last_fullsized":"2023-10-16T10:03:22.597766+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:47.376383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.594797+0000","last_change":"2023-10-16T10:03:22.588681+0000","last_active":"2023-10-16T10:03:22.594797+0000","last_peered":"2023-10-16T10:03:22.594797+0000","last_clean":"2023-10-16T10:03:22.594797+0000","last_became_active":"2023-10-16T10:03:22.588248+0000","last_became_peered":"2023-10-16T10:03:22.588248+0000","last_unstale":"2023-10-16T10:03:22.594797+0000","last_undegraded":"2023-10-16T10:03:22.594797+0000","last_fullsized":"2023-10-16T10:03:22.594797+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:31.251884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.573587+0000","last_change":"2023-10-16T10:03:21.582990+0000","last_active":"2023-10-16T10:03:22.573587+0000","last_peered":"2023-10-16T10:03:22.573587+0000","last_clean":"2023-10-16T10:03:22.573587+0000","last_became_active":"2023-10-16T10:03:21.582715+0000","last_became_peered":"2023-10-16T10:03:21.582715+0000","last_unstale":"2023-10-16T10:03:22.573587+0000","last_undegraded":"2023-10-16T10:03:22.573587+0000","last_fullsized":"2023-10-16T10:03:22.573587+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:07.659994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.596061+0000","last_change":"2023-10-16T10:03:22.580715+0000","last_active":"2023-10-16T10:03:22.596061+0000","last_peered":"2023-10-16T10:03:22.596061+0000","last_clean":"2023-10-16T10:03:22.596061+0000","last_became_active":"2023-10-16T10:03:22.580434+0000","last_became_peered":"2023-10-16T10:03:22.580434+0000","last_unstale":"2023-10-16T10:03:22.596061+0000","last_undegraded":"2023-10-16T10:03:22.596061+0000","last_fullsized":"2023-10-16T10:03:22.596061+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:35.486946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.574097+0000","last_change":"2023-10-16T10:03:21.585852+0000","last_active":"2023-10-16T10:03:22.574097+0000","last_peered":"2023-10-16T10:03:22.574097+0000","last_clean":"2023-10-16T10:03:22.574097+0000","last_became_active":"2023-10-16T10:03:21.585568+0000","last_became_peered":"2023-10-16T10:03:21.585568+0000","last_unstale":"2023-10-16T10:03:22.574097+0000","last_undegraded":"2023-10-16T10:03:22.574097+0000","last_fullsized":"2023-10-16T10:03:22.574097+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:47.584001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.582155+0000","last_change":"2023-10-16T10:03:22.580507+0000","last_active":"2023-10-16T10:03:22.582155+0000","last_peered":"2023-10-16T10:03:22.582155+0000","last_clean":"2023-10-16T10:03:22.582155+0000","last_became_active":"2023-10-16T10:03:22.580214+0000","last_became_peered":"2023-10-16T10:03:22.580214+0000","last_unstale":"2023-10-16T10:03:22.582155+0000","last_undegraded":"2023-10-16T10:03:22.582155+0000","last_fullsized":"2023-10-16T10:03:22.582155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:49.897491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578247+0000","last_change":"2023-10-16T10:03:21.591639+0000","last_active":"2023-10-16T10:03:22.578247+0000","last_peered":"2023-10-16T10:03:22.578247+0000","last_clean":"2023-10-16T10:03:22.578247+0000","last_became_active":"2023-10-16T10:03:21.591454+0000","last_became_peered":"2023-10-16T10:03:21.591454+0000","last_unstale":"2023-10-16T10:03:22.578247+0000","last_undegraded":"2023-10-16T10:03:22.578247+0000","last_fullsized":"2023-10-16T10:03:22.578247+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:17.567179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588519+0000","last_change":"2023-10-16T10:03:22.585824+0000","last_active":"2023-10-16T10:03:23.588519+0000","last_peered":"2023-10-16T10:03:23.588519+0000","last_clean":"2023-10-16T10:03:23.588519+0000","last_became_active":"2023-10-16T10:03:22.585564+0000","last_became_peered":"2023-10-16T10:03:22.585564+0000","last_unstale":"2023-10-16T10:03:23.588519+0000","last_undegraded":"2023-10-16T10:03:23.588519+0000","last_fullsized":"2023-10-16T10:03:23.588519+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:06.355104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590823+0000","last_change":"2023-10-16T10:03:21.582814+0000","last_active":"2023-10-16T10:03:23.590823+0000","last_peered":"2023-10-16T10:03:23.590823+0000","last_clean":"2023-10-16T10:03:23.590823+0000","last_became_active":"2023-10-16T10:03:21.582505+0000","last_became_peered":"2023-10-16T10:03:21.582505+0000","last_unstale":"2023-10-16T10:03:23.590823+0000","last_undegraded":"2023-10-16T10:03:23.590823+0000","last_fullsized":"2023-10-16T10:03:23.590823+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:10.999669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.589088+0000","last_change":"2023-10-16T10:03:22.596780+0000","last_active":"2023-10-16T10:03:23.589088+0000","last_peered":"2023-10-16T10:03:23.589088+0000","last_clean":"2023-10-16T10:03:23.589088+0000","last_became_active":"2023-10-16T10:03:22.596099+0000","last_became_peered":"2023-10-16T10:03:22.596099+0000","last_unstale":"2023-10-16T10:03:23.589088+0000","last_undegraded":"2023-10-16T10:03:23.589088+0000","last_fullsized":"2023-10-16T10:03:23.589088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:11.895956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.588501+0000","last_change":"2023-10-16T10:03:21.581587+0000","last_active":"2023-10-16T10:03:23.588501+0000","last_peered":"2023-10-16T10:03:23.588501+0000","last_clean":"2023-10-16T10:03:23.588501+0000","last_became_active":"2023-10-16T10:03:21.581286+0000","last_became_peered":"2023-10-16T10:03:21.581286+0000","last_unstale":"2023-10-16T10:03:23.588501+0000","last_undegraded":"2023-10-16T10:03:23.588501+0000","last_fullsized":"2023-10-16T10:03:23.588501+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:54:01.674797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.590941+0000","last_change":"2023-10-16T10:03:22.598230+0000","last_active":"2023-10-16T10:03:23.590941+0000","last_peered":"2023-10-16T10:03:23.590941+0000","last_clean":"2023-10-16T10:03:23.590941+0000","last_became_active":"2023-10-16T10:03:22.597939+0000","last_became_peered":"2023-10-16T10:03:22.597939+0000","last_unstale":"2023-10-16T10:03:23.590941+0000","last_undegraded":"2023-10-16T10:03:23.590941+0000","last_fullsized":"2023-10-16T10:03:23.590941+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:48.383707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.677597+0000","last_change":"2023-10-16T10:03:21.584858+0000","last_active":"2023-10-16T10:03:23.677597+0000","last_peered":"2023-10-16T10:03:23.677597+0000","last_clean":"2023-10-16T10:03:23.677597+0000","last_became_active":"2023-10-16T10:03:21.584583+0000","last_became_peered":"2023-10-16T10:03:21.584583+0000","last_unstale":"2023-10-16T10:03:23.677597+0000","last_undegraded":"2023-10-16T10:03:23.677597+0000","last_fullsized":"2023-10-16T10:03:23.677597+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:25.710723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.591642+0000","last_change":"2023-10-16T10:03:22.580706+0000","last_active":"2023-10-16T10:03:22.591642+0000","last_peered":"2023-10-16T10:03:22.591642+0000","last_clean":"2023-10-16T10:03:22.591642+0000","last_became_active":"2023-10-16T10:03:22.580442+0000","last_became_peered":"2023-10-16T10:03:22.580442+0000","last_unstale":"2023-10-16T10:03:22.591642+0000","last_undegraded":"2023-10-16T10:03:22.591642+0000","last_fullsized":"2023-10-16T10:03:22.591642+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:00:09.864772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.578802+0000","last_change":"2023-10-16T10:03:21.589131+0000","last_active":"2023-10-16T10:03:22.578802+0000","last_peered":"2023-10-16T10:03:22.578802+0000","last_clean":"2023-10-16T10:03:22.578802+0000","last_became_active":"2023-10-16T10:03:21.588513+0000","last_became_peered":"2023-10-16T10:03:21.588513+0000","last_unstale":"2023-10-16T10:03:22.578802+0000","last_undegraded":"2023-10-16T10:03:22.578802+0000","last_fullsized":"2023-10-16T10:03:22.578802+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:23.018562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589753+0000","last_change":"2023-10-16T10:03:22.588425+0000","last_active":"2023-10-16T10:03:22.589753+0000","last_peered":"2023-10-16T10:03:22.589753+0000","last_clean":"2023-10-16T10:03:22.589753+0000","last_became_active":"2023-10-16T10:03:22.588123+0000","last_became_peered":"2023-10-16T10:03:22.588123+0000","last_unstale":"2023-10-16T10:03:22.589753+0000","last_undegraded":"2023-10-16T10:03:22.589753+0000","last_fullsized":"2023-10-16T10:03:22.589753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:21.558328+0000","last_clean_scrub_stamp":"2023-10-16T10:03:21.558328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:51.752847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-16T10:03:23.591270+0000","last_change":"2023-10-16T10:03:21.591467+0000","last_active":"2023-10-16T10:03:23.591270+0000","last_peered":"2023-10-16T10:03:23.591270+0000","last_clean":"2023-10-16T10:03:23.591270+0000","last_became_active":"2023-10-16T10:03:21.591179+0000","last_became_peered":"2023-10-16T10:03:21.591179+0000","last_unstale":"2023-10-16T10:03:23.591270+0000","last_undegraded":"2023-10-16T10:03:23.591270+0000","last_fullsized":"2023-10-16T10:03:23.591270+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:03:20.551003+0000","last_clean_scrub_stamp":"2023-10-16T10:03:20.551003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:10:45.651540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"122'227","reported_seq":665,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-16T10:03:22.589922+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:03:22.589922+0000","last_peered":"2023-10-16T10:03:22.589922+0000","last_clean":"2023-10-16T10:03:22.589922+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:03:22.589922+0000","last_undegraded":"2023-10-16T10:03:22.589922+0000","last_fullsized":"2023-10-16T10:03:22.589922+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542375,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33372,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338468,"statfs":{"total":96636764160,"available":96602591232,"internally_reserved":0,"allocated":2236416,"data_stored":1401496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.248},"max":{"1min":0.67400000000000004,"5min":0.79200000000000004,"15min":0.90000000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55400000000000005,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.27200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54300000000000004}]},{"osd":1,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.84699999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.749,"5min":1.788,"15min":1.788},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55700000000000005,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.308},"max":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.80400000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.438}]},{"osd":3,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.35199999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.505},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.61499999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.96399999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46000000000000002,"15min":0.46700000000000003},"min":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.26500000000000001},"max":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.89200000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48699999999999999,"15min":0.505},"min":{"1min":0.34000000000000002,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":1.1779999999999999},"last":0.35099999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.006},"last":0.374},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.80500000000000005,"15min":1.0169999999999999},"last":0.627}]},{"osd":6,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.56100000000000005},"min":{"1min":0.379,"5min":0.36399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":1.373},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.38,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.81399999999999995,"5min":0.83799999999999997,"15min":1.4099999999999999},"last":0.42399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542374,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1626112,"data_stored":803830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.45100000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.79800000000000004},"last":0.438},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.504}]},{"osd":2,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.57499999999999996},"min":{"1min":0.35699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.86099999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.36299999999999999},"max":{"1min":0.70999999999999996,"5min":0.76100000000000001,"15min":0.84499999999999997},"last":0.36899999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78800000000000003,"5min":1.03,"15min":1.03},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.58199999999999996,"15min":0.59699999999999998},"min":{"1min":0.44800000000000001,"5min":0.35499999999999998,"15min":0.34599999999999997},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.85899999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.746,"5min":1,"15min":1},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35199999999999998,"15min":0.33300000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51200000000000001},"min":{"1min":0.378,"5min":0.314,"15min":0.314},"max":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.78100000000000003,"5min":1.056,"15min":1.056},"last":0.66000000000000003}]},{"osd":7,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.86399999999999999},"last":0.623},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55300000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542374,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50840,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321000,"statfs":{"total":96636764160,"available":96584704000,"internally_reserved":0,"allocated":1642496,"data_stored":810636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36199999999999999,"15min":0.36699999999999999},"min":{"1min":0.248,"5min":0.221,"15min":0.20100000000000001},"max":{"1min":0.49099999999999999,"5min":0.54400000000000004,"15min":1.5},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39500000000000002,"15min":0.40400000000000003},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.19600000000000001},"max":{"1min":0.48199999999999998,"5min":0.60099999999999998,"15min":0.80000000000000004},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.377,"15min":0.39200000000000002},"min":{"1min":0.223,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.56699999999999995,"5min":0.63800000000000001,"15min":1.1040000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.41599999999999998,"15min":0.437},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.441,"5min":0.67300000000000004,"15min":3.8730000000000002},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.46800000000000003},"min":{"1min":0.32200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.53100000000000003,"5min":0.68000000000000005,"15min":4.6740000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.30299999999999999,"5min":0.29399999999999998,"15min":0.25700000000000001},"max":{"1min":0.55100000000000005,"5min":0.65100000000000002,"15min":1.379},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.379,"15min":0.33100000000000002},"max":{"1min":0.94199999999999995,"5min":0.96899999999999997,"15min":1.2490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.64500000000000002},"min":{"1min":0.41099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80900000000000005,"5min":0.92100000000000004,"15min":5.4119999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.624,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84399999999999997,"5min":0.94999999999999996,"15min":1.26},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.40200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.753,"5min":0.85399999999999998,"15min":1.3560000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64600000000000002,"15min":0.66600000000000004},"min":{"1min":0.45400000000000001,"5min":0.44500000000000001,"15min":0.374},"max":{"1min":0.79700000000000004,"5min":0.92100000000000004,"15min":6.2140000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.47199999999999998,"5min":0.36799999999999999,"15min":0.36199999999999999},"max":{"1min":0.877,"5min":0.92600000000000005,"15min":1.3420000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.65600000000000003},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.39800000000000002},"max":{"1min":0.91000000000000003,"5min":0.98299999999999998,"15min":1.4079999999999999},"last":0.752},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64000000000000001,"15min":0.66400000000000003},"min":{"1min":0.45200000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84999999999999998,"5min":0.93799999999999994,"15min":1.4510000000000001},"last":0.77900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542375,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33364,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338476,"statfs":{"total":96636764160,"available":96602599424,"internally_reserved":0,"allocated":2236416,"data_stored":1395852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.22800000000000001,"5min":0.192,"15min":0.191},"max":{"1min":0.52800000000000002,"5min":0.59999999999999998,"15min":0.65400000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.38300000000000001,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.60999999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.28899999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39000000000000001,"15min":0.40600000000000003},"min":{"1min":0.26700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54400000000000004,"5min":0.61299999999999999,"15min":0.626},"last":0.308},{"interface":"front","average":{"1min":0.375,"5min":0.39700000000000002,"15min":0.41899999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.53000000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.33400000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40600000000000003,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55600000000000005,"5min":0.64800000000000002,"15min":0.72999999999999998},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45900000000000002,"15min":0.47899999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.59999999999999998,"5min":0.69299999999999995,"15min":0.69699999999999995},"last":0.375}]},{"osd":4,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60199999999999998,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83199999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.63200000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.98799999999999999},"last":0.48199999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.64600000000000002},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34200000000000003},"max":{"1min":0.92200000000000004,"5min":1.004,"15min":1.004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61399999999999999,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83499999999999996,"5min":0.88700000000000001,"15min":0.96999999999999997},"last":0.621}]},{"osd":6,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.85499999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.66200000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81699999999999995,"5min":0.86899999999999999,"15min":1.0089999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.65700000000000003},"min":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83899999999999997,"5min":0.94499999999999995,"15min":1.0249999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.66000000000000003},"min":{"1min":0.432,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.95699999999999996},"last":0.503}]}]},{"osd":2,"up_from":14,"seq":60129542374,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54988,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316852,"statfs":{"total":96636764160,"available":96580456448,"internally_reserved":0,"allocated":1634304,"data_stored":810512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.371,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.183},"max":{"1min":0.53100000000000003,"5min":0.56299999999999994,"15min":0.64900000000000002},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.26300000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.49199999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40100000000000002,"15min":0.40699999999999997},"min":{"1min":0.28499999999999998,"5min":0.252,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.42399999999999999},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.214},"max":{"1min":0.58399999999999996,"5min":0.60699999999999998,"15min":0.64000000000000001},"last":0.33600000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.40699999999999997},"min":{"1min":0.252,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.55500000000000005,"5min":0.59199999999999997,"15min":0.61899999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.46600000000000003},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.27800000000000002},"max":{"1min":0.61699999999999999,"5min":0.747,"15min":0.747},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.81599999999999995,"5min":0.91600000000000004,"15min":0.93999999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.42299999999999999,"5min":0.41999999999999998,"15min":0.36799999999999999},"max":{"1min":0.82699999999999996,"5min":0.88200000000000001,"15min":1.091},"last":0.60999999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":1.01,"15min":4.3609999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61599999999999999,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.44900000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.879,"15min":1.0720000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63,"15min":0.64500000000000002},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.90500000000000003,"15min":0.97899999999999998},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Oct 16 10:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.48499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79400000000000004,"5min":0.96499999999999997,"15min":0.98799999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.39800000000000002,"15min":0.38500000000000001},"max":{"1min":0.79100000000000004,"5min":0.98799999999999999,"15min":0.998},"last":0.64100000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542375,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36944,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334896,"statfs":{"total":96636764160,"available":96598933504,"internally_reserved":0,"allocated":1638400,"data_stored":805442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.372,"15min":0.38600000000000001},"min":{"1min":0.215,"5min":0.214,"15min":0.20200000000000001},"max":{"1min":0.55000000000000004,"5min":0.60299999999999998,"15min":0.64900000000000002},"last":0.313},{"interface":"front","average":{"1min":0.378,"5min":0.40400000000000003,"15min":0.41699999999999998},"min":{"1min":0.28999999999999998,"5min":0.26200000000000001,"15min":0.185},"max":{"1min":0.48299999999999998,"5min":0.626,"15min":0.63100000000000001},"last":0.33900000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.38100000000000001,"15min":0.39500000000000002},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.219},"max":{"1min":0.51500000000000001,"5min":0.61399999999999999,"15min":0.72199999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.40500000000000003,"15min":0.42099999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52800000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.44,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.76200000000000001,"5min":0.95699999999999996,"15min":0.95799999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.44900000000000001,"5min":0.314,"15min":0.311},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.0820000000000001},"last":0.45800000000000002}]},{"osd":5,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.35099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.98699999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.35599999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.996},"last":0.41499999999999998}]},{"osd":6,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.626,"15min":0.64200000000000002},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90000000000000002,"5min":0.92400000000000004,"15min":0.98399999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.99099999999999999},"last":0.42399999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.3},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.66000000000000003},"min":{"1min":0.48599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91800000000000004,"15min":1.4019999999999999},"last":0.52100000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59144,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312696,"statfs":{"total":96636764160,"available":96576200704,"internally_reserved":0,"allocated":1634304,"data_stored":810512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56200000000000006,"15min":0.57599999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.186},"max":{"1min":0.67400000000000004,"5min":0.76900000000000002,"15min":0.89400000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38400000000000001,"5min":0.248,"15min":0.23899999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.81299999999999994},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56899999999999995,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.29899999999999999,"15min":0.26300000000000001},"max":{"1min":0.75700000000000001,"5min":0.84599999999999997,"15min":0.86599999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.248},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.83799999999999997},"last":0.52500000000000002}]},{"osd":2,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.29799999999999999},"max":{"1min":0.76000000000000001,"5min":0.81899999999999995,"15min":0.85399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.59699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.27900000000000003},"max":{"1min":0.72299999999999998,"5min":0.72799999999999998,"15min":0.89100000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.314},"max":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.86499999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.60499999999999998},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.76700000000000002,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47899999999999998,"15min":0.48399999999999999},"min":{"1min":0.32500000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.496,"15min":0.51200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65700000000000003,"5min":0.77800000000000002,"15min":0.80400000000000005},"last":0.36899999999999999}]},{"osd":6,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48799999999999999,"15min":0.50800000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.82999999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54500000000000004,"15min":0.56100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.85899999999999999},"last":0.48199999999999998}]},{"osd":7,"last update":"Mon Oct 16 10:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53900000000000003,"15min":0.55600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.80700000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80400000000000005,"5min":0.83299999999999996,"15min":0.84599999999999997},"last":0.5}]}]},{"osd":5,"up_from":14,"seq":60129542375,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1642496,"data_stored":811116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57799999999999996,"15min":0.58699999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77000000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56399999999999995,"15min":0.59099999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.27700000000000002},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.91800000000000004},"last":0.57399999999999995}]},{"osd":1,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.58199999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28899999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.82799999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59599999999999997},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70199999999999996}]},{"osd":2,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56499999999999995,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.32300000000000001,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.35599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.88700000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.88600000000000001,"15min":0.89000000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75800000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48799999999999999,"15min":0.48899999999999999},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.73599999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.624},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85599999999999998,"5min":0.86099999999999999,"15min":0.86799999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.311},"max":{"1min":0.81299999999999994,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.35699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 10:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.433,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88600000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:03:27.069 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:03:27.069 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:03:27.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.229+0000 7f89037e9700 1 Processor -- start 2023-10-16T10:03:27.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.230+0000 7f89037e9700 1 -- start start 2023-10-16T10:03:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.230+0000 7f89037e9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 0x7f88fc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.230+0000 7f89037e9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88fc0b2ae0 con 0x7f88fc0ce4f0 2023-10-16T10:03:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.230+0000 7f89037e9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88fc0b2dc0 con 0x7f88fc0cd170 2023-10-16T10:03:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.230+0000 7f89037e9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0ce970 con 0x7f88fc15f150 2023-10-16T10:03:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.230+0000 7f8900d84700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f88fc0ce4f0 0x7f88fc0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58736/0 (socket says 172.21.15.181:58736) 2023-10-16T10:03:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8900d84700 1 -- 172.21.15.181:0/776906826 learned_addr learned my addr 172.21.15.181:0/776906826 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901585700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 0x7f88fc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476905932 0 0) 0x7f88fc0b2ae0 con 0x7f88fc0ce4f0 2023-10-16T10:03:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e80035b0 con 0x7f88fc0ce4f0 2023-10-16T10:03:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706205736 0 0) 0x7f88fc0b2dc0 con 0x7f88fc0cd170 2023-10-16T10:03:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88fc0b2ae0 con 0x7f88fc0cd170 2023-10-16T10:03:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 488288873 0 0) 0x7f88e80035b0 con 0x7f88fc0ce4f0 2023-10-16T10:03:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 msgr2=0x7f88fc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 0x7f88fc0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 >> v1:172.21.15.190:6789/0 conn(0x7f88fc0cd170 legacy=0x7f88fc0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0cf9d0 con 0x7f88fc0ce4f0 2023-10-16T10:03:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f88ec001670 con 0x7f88fc0ce4f0 2023-10-16T10:03:27.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 >> v1:172.21.15.181:6789/0 conn(0x7f88fc0ce4f0 legacy=0x7f88fc0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88e8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 0x7f88e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8004d90 0x7f88e8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0ce970 con 0x7f88e8003bc0 2023-10-16T10:03:27.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e800bac0 con 0x7f88e8004d90 2023-10-16T10:03:27.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e800bc30 con 0x7f88fc15f150 2023-10-16T10:03:27.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901585700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88e8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8900d84700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 0x7f88e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901d86700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8004d90 0x7f88e8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901585700 1 -- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 msgr2=0x7f88e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:27.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901585700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15f150 0x7f88e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901585700 1 -- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8004d90 msgr2=0x7f88e8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:27.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901585700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8004d90 0x7f88e8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.231+0000 7f8901585700 1 -- 172.21.15.181:0/776906826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80035b0 con 0x7f88e8003bc0 2023-10-16T10:03:27.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f8901585700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88e8003fc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009b60 tx=0x7f88e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceee530dc5a28bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:27.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800e6c0 con 0x7f88e8003bc0 2023-10-16T10:03:27.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f800e820 con 0x7f88e8003bc0 2023-10-16T10:03:27.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f88f3fff700 1 -- 172.21.15.181:0/776906826 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800eb90 con 0x7f88e8003bc0 2023-10-16T10:03:27.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f89037e9700 1 -- 172.21.15.181:0/776906826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 msgr2=0x7f88e8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f89037e9700 1 --2- 172.21.15.181:0/776906826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88e8003fc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009b60 tx=0x7f88e8005880 comp rx=0 tx=0).stop 2023-10-16T10:03:27.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f8901d86700 1 -- 172.21.15.181:0/776906826 reap_dead start 2023-10-16T10:03:27.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f89037e9700 1 -- 172.21.15.181:0/776906826 shutdown_connections 2023-10-16T10:03:27.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f89037e9700 1 -- 172.21.15.181:0/776906826 >> 172.21.15.181:0/776906826 conn(0x7f88fc09a930 msgr2=0x7f88fc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:27.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f89037e9700 1 -- 172.21.15.181:0/776906826 shutdown_connections 2023-10-16T10:03:27.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.232+0000 7f89037e9700 1 -- 172.21.15.181:0/776906826 wait complete. 2023-10-16T10:03:27.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 Processor -- start 2023-10-16T10:03:27.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 -- start start 2023-10-16T10:03:27.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88fc0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc157c50 0x7f88fc14a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15dda0 0x7f88fc15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0c8d00 con 0x7f88e8003bc0 2023-10-16T10:03:27.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc15e790 con 0x7f88fc157c50 2023-10-16T10:03:27.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f89037e9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0c86d0 con 0x7f88fc15dda0 2023-10-16T10:03:27.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f8901585700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88fc0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f8901585700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88fc0cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47668/0 (socket says 172.21.15.181:47668) 2023-10-16T10:03:27.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.234+0000 7f8901585700 1 -- 172.21.15.181:0/614209032 learned_addr learned my addr 172.21.15.181:0/614209032 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:27.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901d86700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15dda0 0x7f88fc15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8900d84700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc157c50 0x7f88fc14a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901585700 1 -- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15dda0 msgr2=0x7f88fc15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901585700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15dda0 0x7f88fc15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901585700 1 -- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc157c50 msgr2=0x7f88fc14a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901585700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc157c50 0x7f88fc14a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901585700 1 -- 172.21.15.181:0/614209032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc1583f0 con 0x7f88e8003bc0 2023-10-16T10:03:27.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8900d84700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc157c50 0x7f88fc14a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:27.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901d86700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15dda0 0x7f88fc15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:27.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f8901585700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88fc0cb1b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f88f80090b0 tx=0x7f88f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:27.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f88f1ffb700 1 -- 172.21.15.181:0/614209032 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800e6c0 con 0x7f88e8003bc0 2023-10-16T10:03:27.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f88f1ffb700 1 -- 172.21.15.181:0/614209032 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f8003ad0 con 0x7f88e8003bc0 2023-10-16T10:03:27.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc1586d0 con 0x7f88e8003bc0 2023-10-16T10:03:27.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f88f1ffb700 1 -- 172.21.15.181:0/614209032 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f80045d0 con 0x7f88e8003bc0 2023-10-16T10:03:27.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.235+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc14af00 con 0x7f88e8003bc0 2023-10-16T10:03:27.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.236+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc067d20 con 0x7f88e8003bc0 2023-10-16T10:03:27.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.237+0000 7f88f1ffb700 1 -- 172.21.15.181:0/614209032 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f88f8013470 con 0x7f88e8003bc0 2023-10-16T10:03:27.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.237+0000 7f88f1ffb700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f88e006b020 0x7f88e006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.237+0000 7f88f1ffb700 1 -- 172.21.15.181:0/614209032 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f88f808b300 con 0x7f88e8003bc0 2023-10-16T10:03:27.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.237+0000 7f8900d84700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f88e006b020 0x7f88e006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.238+0000 7f8900d84700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f88e006b020 0x7f88e006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88fc1523f0 tx=0x7f88ec009040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:27.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.242+0000 7f88f1ffb700 1 -- 172.21.15.181:0/614209032 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88f8056280 con 0x7f88e8003bc0 2023-10-16T10:03:27.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.512+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f88fc0b1520 con 0x7f88e8003bc0 2023-10-16T10:03:27.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.514+0000 7f88f1ffb700 1 -- 172.21.15.181:0/614209032 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f88f80598a0 con 0x7f88e8003bc0 2023-10-16T10:03:27.515 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T10:03:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T10:03:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T10:03:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T10:03:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:03:27.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f88e006b020 msgr2=0x7f88e006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f88e006b020 0x7f88e006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88fc1523f0 tx=0x7f88ec009040 comp rx=0 tx=0).stop 2023-10-16T10:03:27.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 msgr2=0x7f88fc0cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88fc0cb1b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f88f80090b0 tx=0x7f88f80047c0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 shutdown_connections 2023-10-16T10:03:27.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f88e006b020 0x7f88e006d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88fc15dda0 0x7f88fc15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e8003bc0 0x7f88fc0cb1b0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 --2- 172.21.15.181:0/614209032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc157c50 0x7f88fc14a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 >> 172.21.15.181:0/614209032 conn(0x7f88fc09a930 msgr2=0x7f88fc070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 shutdown_connections 2023-10-16T10:03:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.517+0000 7f89037e9700 1 -- 172.21.15.181:0/614209032 wait complete. 2023-10-16T10:03:27.521 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T10:03:27.530 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.650+0000 7fe2c2ae1700 1 Processor -- start 2023-10-16T10:03:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.651+0000 7fe2c2ae1700 1 -- start start 2023-10-16T10:03:27.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.651+0000 7fe2c2ae1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 0x7fe2bc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.651+0000 7fe2c2ae1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc06f870 con 0x7fe2bc15dfd0 2023-10-16T10:03:27.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.651+0000 7fe2c2ae1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0d1160 con 0x7fe2bc0af450 2023-10-16T10:03:27.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.651+0000 7fe2c2ae1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0cc8e0 con 0x7fe2bc0cdb80 2023-10-16T10:03:27.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.651+0000 7fe2c087d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe2bc15dfd0 0x7fe2bc06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58754/0 (socket says 172.21.15.181:58754) 2023-10-16T10:03:27.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.651+0000 7fe2c087d700 1 -- 172.21.15.181:0/3499151578 learned_addr learned my addr 172.21.15.181:0/3499151578 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:27.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c107e700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 0x7fe2bc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583165352 0 0) 0x7fe2bc06f870 con 0x7fe2bc15dfd0 2023-10-16T10:03:27.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ac0035b0 con 0x7fe2bc15dfd0 2023-10-16T10:03:27.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 846607912 0 0) 0x7fe2ac0035b0 con 0x7fe2bc15dfd0 2023-10-16T10:03:27.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 msgr2=0x7fe2bc0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:27.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 0x7fe2bc0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 >> v1:172.21.15.190:6789/0 conn(0x7fe2bc0af450 legacy=0x7fe2bc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0ccbc0 con 0x7fe2bc15dfd0 2023-10-16T10:03:27.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe2b00017f0 con 0x7fe2bc15dfd0 2023-10-16T10:03:27.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 >> v1:172.21.15.181:6789/0 conn(0x7fe2bc15dfd0 legacy=0x7fe2bc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2ac003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 0x7fe2ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2ac004d90 0x7fe2ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0cc8e0 con 0x7fe2ac003a30 2023-10-16T10:03:27.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac00bb70 con 0x7fe2ac004d90 2023-10-16T10:03:27.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac00bd30 con 0x7fe2bc0cdb80 2023-10-16T10:03:27.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c087d700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2ac003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c107e700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2ac004d90 0x7fe2ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2bbfff700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 0x7fe2ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c087d700 1 -- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 msgr2=0x7fe2ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:27.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c087d700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc0cdb80 0x7fe2ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c087d700 1 -- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2ac004d90 msgr2=0x7fe2ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:27.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c087d700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2ac004d90 0x7fe2ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.652+0000 7fe2c087d700 1 -- 172.21.15.181:0/3499151578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0035b0 con 0x7fe2ac003a30 2023-10-16T10:03:27.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c087d700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2ac003e50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c240 tx=0x7fe2ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0ac834a993cd6c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:27.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000cd20 con 0x7fe2ac003a30 2023-10-16T10:03:27.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000ce80 con 0x7fe2ac003a30 2023-10-16T10:03:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2bb7fe700 1 -- 172.21.15.181:0/3499151578 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00072f0 con 0x7fe2ac003a30 2023-10-16T10:03:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/3499151578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 msgr2=0x7fe2ac003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c2ae1700 1 --2- 172.21.15.181:0/3499151578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2ac003e50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c240 tx=0x7fe2ac005880 comp rx=0 tx=0).stop 2023-10-16T10:03:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c107e700 1 -- 172.21.15.181:0/3499151578 reap_dead start 2023-10-16T10:03:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/3499151578 shutdown_connections 2023-10-16T10:03:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/3499151578 >> 172.21.15.181:0/3499151578 conn(0x7fe2bc0d04a0 msgr2=0x7fe2bc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/3499151578 shutdown_connections 2023-10-16T10:03:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.653+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/3499151578 wait complete. 2023-10-16T10:03:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.654+0000 7fe2c2ae1700 1 Processor -- start 2023-10-16T10:03:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.654+0000 7fe2c2ae1700 1 -- start start 2023-10-16T10:03:27.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c2ae1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2bc0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c2ae1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2bc1f4150 0x7fe2bc0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c2ae1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc1ed560 0x7fe2bc0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c2ae1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1f3d30 con 0x7fe2ac003a30 2023-10-16T10:03:27.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c2ae1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1f36d0 con 0x7fe2bc1f4150 2023-10-16T10:03:27.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c2ae1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0c9cc0 con 0x7fe2bc1ed560 2023-10-16T10:03:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c087d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2bc0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c087d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2bc0c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47688/0 (socket says 172.21.15.181:47688) 2023-10-16T10:03:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.655+0000 7fe2c087d700 1 -- 172.21.15.181:0/937774066 learned_addr learned my addr 172.21.15.181:0/937774066 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c107e700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc1ed560 0x7fe2bc0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2bbfff700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2bc1f4150 0x7fe2bc0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c087d700 1 -- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc1ed560 msgr2=0x7fe2bc0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c087d700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc1ed560 0x7fe2bc0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c087d700 1 -- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2bc1f4150 msgr2=0x7fe2bc0c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c087d700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2bc1f4150 0x7fe2bc0c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c087d700 1 -- 172.21.15.181:0/937774066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0c9f60 con 0x7fe2ac003a30 2023-10-16T10:03:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2bbfff700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2bc1f4150 0x7fe2bc0c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c087d700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2bc0c42f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00bb40 tx=0x7fe2b0001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2b97fa700 1 -- 172.21.15.181:0/937774066 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0005a30 con 0x7fe2ac003a30 2023-10-16T10:03:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2b97fa700 1 -- 172.21.15.181:0/937774066 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0005bb0 con 0x7fe2ac003a30 2023-10-16T10:03:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2b97fa700 1 -- 172.21.15.181:0/937774066 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00017c0 con 0x7fe2ac003a30 2023-10-16T10:03:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc0ca240 con 0x7fe2ac003a30 2023-10-16T10:03:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.656+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc1f45d0 con 0x7fe2ac003a30 2023-10-16T10:03:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.657+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc067d20 con 0x7fe2ac003a30 2023-10-16T10:03:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.657+0000 7fe2b97fa700 1 -- 172.21.15.181:0/937774066 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe2b0019070 con 0x7fe2ac003a30 2023-10-16T10:03:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.658+0000 7fe2b97fa700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe2a006b020 0x7fe2a006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.658+0000 7fe2b97fa700 1 -- 172.21.15.181:0/937774066 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fe2b008a7b0 con 0x7fe2ac003a30 2023-10-16T10:03:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.658+0000 7fe2bbfff700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe2a006b020 0x7fe2a006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.658+0000 7fe2bbfff700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe2a006b020 0x7fe2a006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8009040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.662+0000 7fe2b97fa700 1 -- 172.21.15.181:0/937774066 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2b0055590 con 0x7fe2ac003a30 2023-10-16T10:03:27.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.897+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2bc0b3050 con 0x7fe2ac003a30 2023-10-16T10:03:27.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.897+0000 7fe2b97fa700 1 -- 172.21.15.181:0/937774066 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fe2b0000f70 con 0x7fe2ac003a30 2023-10-16T10:03:27.898 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:27.899 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2508916691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2508916691},{"type":"v1","addr":"172.21.15.181:6836","nonce":2508916691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7687,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1303363384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1303363384},{"type":"v1","addr":"172.21.15.190:6834","nonce":1303363384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/300739418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":300739418},{"type":"v1","addr":"172.21.15.181:6837","nonce":300739418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:03:21.971792+0000","modified":"2023-10-16T10:03:24.995974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1716409851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1716409851},{"type":"v1","addr":"172.21.15.190:6835","nonce":1716409851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:03:27.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe2a006b020 msgr2=0x7fe2a006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe2a006b020 0x7fe2a006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8009040 comp rx=0 tx=0).stop 2023-10-16T10:03:27.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 msgr2=0x7fe2bc0c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:27.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2bc0c42f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00bb40 tx=0x7fe2b0001b80 comp rx=0 tx=0).stop 2023-10-16T10:03:27.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 shutdown_connections 2023-10-16T10:03:27.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe2a006b020 0x7fe2a006d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2bc1ed560 0x7fe2bc0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac003a30 0x7fe2bc0c42f0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 --2- 172.21.15.181:0/937774066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2bc1f4150 0x7fe2bc0c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 >> 172.21.15.181:0/937774066 conn(0x7fe2bc0d04a0 msgr2=0x7fe2bc0adf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 shutdown_connections 2023-10-16T10:03:27.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:27.899+0000 7fe2c2ae1700 1 -- 172.21.15.181:0/937774066 wait complete. 2023-10-16T10:03:27.903 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2023-10-16T10:03:28.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:03:21.971792+0000', 'modified': '2023-10-16T10:03:24.995974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8556': {'gid': 8556, 'name': 'b', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/1716409851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 1716409851}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1716409851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:03:28.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:03:28.316 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T10:03:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.440+0000 7f6054b7c700 1 Processor -- start 2023-10-16T10:03:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.440+0000 7f6054b7c700 1 -- start start 2023-10-16T10:03:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f6054b7c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f605015df60 0x7f60500cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f6054b7c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60500cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f6054b7c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f605006a8a0 0x7f605006ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f6054b7c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60500af9c0 con 0x7f605006a8a0 2023-10-16T10:03:28.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f6054b7c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605006b270 con 0x7f605015df60 2023-10-16T10:03:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f6054b7c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60500ccf50 con 0x7f60500d0fc0 2023-10-16T10:03:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f604ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f605006a8a0 0x7f605006ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f604ed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f605006a8a0 0x7f605006ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47700/0 (socket says 172.21.15.181:47700) 2023-10-16T10:03:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f604ed9d700 1 -- 172.21.15.181:0/2867650550 learned_addr learned my addr 172.21.15.181:0/2867650550 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f604dd9b700 1 --2- 172.21.15.181:0/2867650550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60500cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f604e59c700 1 --2- 172.21.15.181:0/2867650550 >> v2:172.21.15.190:3300/0 conn(0x7f605015df60 0x7f60500cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f604ed9d700 1 -- 172.21.15.181:0/2867650550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 msgr2=0x7f60500cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.441+0000 7f604ed9d700 1 --2- 172.21.15.181:0/2867650550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60500cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604ed9d700 1 -- 172.21.15.181:0/2867650550 >> v2:172.21.15.190:3300/0 conn(0x7f605015df60 msgr2=0x7f60500cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604ed9d700 1 --2- 172.21.15.181:0/2867650550 >> v2:172.21.15.190:3300/0 conn(0x7f605015df60 0x7f60500cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604ed9d700 1 -- 172.21.15.181:0/2867650550 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60500cc600 con 0x7f605006a8a0 2023-10-16T10:03:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604e59c700 1 --2- 172.21.15.181:0/2867650550 >> v2:172.21.15.190:3300/0 conn(0x7f605015df60 0x7f60500cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604ed9d700 1 --2- 172.21.15.181:0/2867650550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f605006ad30 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6050072040 tx=0x7f6048009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166fda0738e73556 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604d59a700 1 -- 172.21.15.181:0/2867650550 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604800b870 con 0x7f605006a8a0 2023-10-16T10:03:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604d59a700 1 -- 172.21.15.181:0/2867650550 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604800b9d0 con 0x7f605006a8a0 2023-10-16T10:03:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f604d59a700 1 -- 172.21.15.181:0/2867650550 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604800bd60 con 0x7f605006a8a0 2023-10-16T10:03:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 -- 172.21.15.181:0/2867650550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 msgr2=0x7f605006ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 --2- 172.21.15.181:0/2867650550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f605006ad30 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6050072040 tx=0x7f6048009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 -- 172.21.15.181:0/2867650550 shutdown_connections 2023-10-16T10:03:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 --2- 172.21.15.181:0/2867650550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f605006ad30 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 --2- 172.21.15.181:0/2867650550 >> v2:172.21.15.190:3300/0 conn(0x7f605015df60 0x7f60500cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 --2- 172.21.15.181:0/2867650550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60500cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 -- 172.21.15.181:0/2867650550 >> 172.21.15.181:0/2867650550 conn(0x7f60500d04a0 msgr2=0x7f60500d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 -- 172.21.15.181:0/2867650550 shutdown_connections 2023-10-16T10:03:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.442+0000 7f6054b7c700 1 -- 172.21.15.181:0/2867650550 wait complete. 2023-10-16T10:03:28.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.444+0000 7f6054b7c700 1 Processor -- start 2023-10-16T10:03:28.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.444+0000 7f6054b7c700 1 -- start start 2023-10-16T10:03:28.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f60500c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60501e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f605015df60 0x7f60501e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60500c5c00 con 0x7f605006a8a0 2023-10-16T10:03:28.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501f39d0 con 0x7f605015df60 2023-10-16T10:03:28.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60501f3cb0 con 0x7f60500d0fc0 2023-10-16T10:03:28.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f60500c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f60500c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47702/0 (socket says 172.21.15.181:47702) 2023-10-16T10:03:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 -- 172.21.15.181:0/996122526 learned_addr learned my addr 172.21.15.181:0/996122526 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604dd9b700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60501e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604ed9d700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f605015df60 0x7f60501e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 -- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 msgr2=0x7f60501e77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60501e77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 -- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f605015df60 msgr2=0x7f60501e7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f605015df60 0x7f60501e7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 -- 172.21.15.181:0/996122526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6048009550 con 0x7f605006a8a0 2023-10-16T10:03:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f604e59c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f60500c56c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f604000cec0 tx=0x7f604000a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6046ffd700 1 -- 172.21.15.181:0/996122526 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604000e650 con 0x7f605006a8a0 2023-10-16T10:03:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6046ffd700 1 -- 172.21.15.181:0/996122526 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604000e7b0 con 0x7f605006a8a0 2023-10-16T10:03:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60501f3f90 con 0x7f605006a8a0 2023-10-16T10:03:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6046ffd700 1 -- 172.21.15.181:0/996122526 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604000eb40 con 0x7f605006a8a0 2023-10-16T10:03:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.445+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60501f36b0 con 0x7f605006a8a0 2023-10-16T10:03:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.447+0000 7f6046ffd700 1 -- 172.21.15.181:0/996122526 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f604000eca0 con 0x7f605006a8a0 2023-10-16T10:03:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.447+0000 7f6046ffd700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f603006b020 0x7f603006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.447+0000 7f6046ffd700 1 -- 172.21.15.181:0/996122526 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f6040012070 con 0x7f605006a8a0 2023-10-16T10:03:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.447+0000 7f604dd9b700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f603006b020 0x7f603006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.448+0000 7f604dd9b700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f603006b020 0x7f603006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60500c6f50 tx=0x7f6038008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.449+0000 7f6060d66b80 1 Processor -- start 2023-10-16T10:03:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.449+0000 7f6060d66b80 1 -- start start 2023-10-16T10:03:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce04a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce04fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdd86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f6060d66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6cd4ec20 con 0x557a6cdd8010 2023-10-16T10:03:28.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6cd33c10 con 0x557a6ce02630 2023-10-16T10:03:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a6cd31530 con 0x557a6cdfd220 2023-10-16T10:03:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdd86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdd86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47714/0 (socket says 172.21.15.181:47714) 2023-10-16T10:03:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 -- 172.21.15.181:0/1491831315 learned_addr learned my addr 172.21.15.181:0/1491831315 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604e59c700 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce04a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604dd9b700 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce04fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 -- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 msgr2=0x557a6ce04a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce04a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 -- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 msgr2=0x557a6ce04fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce04fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 -- 172.21.15.181:0/1491831315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a6ccea3a0 con 0x557a6cdd8010 2023-10-16T10:03:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604e59c700 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce04a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.450+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdd86c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f60500716a0 tx=0x7f6048010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0cbd6dcca0d7d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f602ffff700 1 -- 172.21.15.181:0/1491831315 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048020070 con 0x557a6cdd8010 2023-10-16T10:03:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f602ffff700 1 -- 172.21.15.181:0/1491831315 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604800b820 con 0x557a6cdd8010 2023-10-16T10:03:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f602ffff700 1 -- 172.21.15.181:0/1491831315 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604800bd60 con 0x557a6cdd8010 2023-10-16T10:03:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 -- 172.21.15.181:0/1491831315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 msgr2=0x557a6cdd86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdd86c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f60500716a0 tx=0x7f6048010000 comp rx=0 tx=0).stop 2023-10-16T10:03:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 -- 172.21.15.181:0/1491831315 shutdown_connections 2023-10-16T10:03:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce04a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdd86c0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1491831315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce04fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 -- 172.21.15.181:0/1491831315 >> 172.21.15.181:0/1491831315 conn(0x557a6cd07d80 msgr2=0x557a6ce22ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 -- 172.21.15.181:0/1491831315 shutdown_connections 2023-10-16T10:03:28.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.451+0000 7f6060d66b80 1 -- 172.21.15.181:0/1491831315 wait complete. 2023-10-16T10:03:28.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.452+0000 7f6060d66b80 1 -- start start 2023-10-16T10:03:28.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.452+0000 7f6060d66b80 1 Processor -- start 2023-10-16T10:03:28.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdfd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce2f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f6060d66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6cd2ce20 con 0x557a6cdd8010 2023-10-16T10:03:28.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6ccc96c0 con 0x557a6ce02630 2023-10-16T10:03:28.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a6ccc9360 con 0x557a6cdfd220 2023-10-16T10:03:28.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f6060d66b80 10 client.0 _ll_register_callbacks cb 0x557a6cb267b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:03:28.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdfd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdfd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47716/0 (socket says 172.21.15.181:47716) 2023-10-16T10:03:28.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 -- 172.21.15.181:0/1831246498 learned_addr learned my addr 172.21.15.181:0/1831246498 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce2f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604dd9b700 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 -- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 msgr2=0x557a6ce07620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce07620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 -- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 msgr2=0x557a6ce2f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce2f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 -- 172.21.15.181:0/1831246498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60500cc600 con 0x557a6cdd8010 2023-10-16T10:03:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f604e59c700 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdfd600 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6040058ca0 tx=0x7f604000aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6572b08b037fd97c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.453+0000 7f602d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:03:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f602d7fa700 1 -- 172.21.15.181:0/1831246498 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040058400 con 0x557a6cdd8010 2023-10-16T10:03:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f602d7fa700 1 -- 172.21.15.181:0/1831246498 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f6040058d10 con 0x557a6cdd8010 2023-10-16T10:03:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a6cdcd5b0 con 0x557a6cdd8010 2023-10-16T10:03:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f602d7fa700 1 -- 172.21.15.181:0/1831246498 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f604005ce60 con 0x557a6cdd8010 2023-10-16T10:03:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f6060d66b80 10 client.8688 fetch_fsmap learned FSMap version 67 2023-10-16T10:03:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x557a6cb57d90 con 0x557a6cdd8010 2023-10-16T10:03:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f602d7fa700 1 -- 172.21.15.181:0/1831246498 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a6cdcd5b0 con 0x557a6cdd8010 2023-10-16T10:03:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.454+0000 7f602d7fa700 1 -- 172.21.15.181:0/1831246498 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6040060b80 con 0x557a6cdd8010 2023-10-16T10:03:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f6060d66b80 10 client.8688 fetch_fsmap finished waiting for FSMap version 67 2023-10-16T10:03:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f6060d66b80 10 client.8688 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T10:03:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f6060d66b80 20 client.8688 populate_metadata read hostname 'smithi181' 2023-10-16T10:03:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce07b60 0x557a6ce00a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f6060d66b80 4 client.8688 mds_command: new command op to 8556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:03:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 --> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a6cd25e00 con 0x557a6ce07b60 2023-10-16T10:03:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f604dd9b700 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce07b60 0x557a6ce00a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f602d7fa700 10 client.8688 ms_handle_connect on v2:172.21.15.190:6833/1716409851 2023-10-16T10:03:28.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.455+0000 7f604dd9b700 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce07b60 0x557a6ce00a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847e1d4fd5767aed server_cookie=90ed44834ea778b7 in_seq=0 out_seq=0 2023-10-16T10:03:28.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f602d7fa700 1 -- 172.21.15.181:0/1831246498 <== mds.0 v2:172.21.15.190:6833/1716409851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a6cd25e00 con 0x557a6ce07b60 2023-10-16T10:03:28.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f602d7fa700 10 client.8688 handle_command_reply: tid=0 2023-10-16T10:03:28.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 client.8688 shutdown 2023-10-16T10:03:28.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 msgr2=0x557a6cdfd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdfd600 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6040058ca0 tx=0x7f604000aec0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 shutdown_connections 2023-10-16T10:03:28.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6ce02630 0x557a6ce2f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6cdd8010 0x557a6cdfd600 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce07b60 0x557a6ce00a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1831246498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6ce07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f602d7fa700 0 client.8688 ms_handle_reset on v2:172.21.15.190:6833/1716409851 2023-10-16T10:03:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 >> 172.21.15.181:0/1831246498 conn(0x557a6cd07d80 msgr2=0x557a6ce22f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 shutdown_connections 2023-10-16T10:03:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 1 -- 172.21.15.181:0/1831246498 wait complete. 2023-10-16T10:03:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.456+0000 7f6060d66b80 20 client.8688 trim_cache size 0 max 16384 2023-10-16T10:03:28.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 Processor -- start 2023-10-16T10:03:28.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 -- start start 2023-10-16T10:03:28.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdd8010 0x557a6ce20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6cdd4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6cbad8d0 con 0x557a6ce02630 2023-10-16T10:03:28.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6ce024c0 con 0x557a6cdd8010 2023-10-16T10:03:28.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.473+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a6cdb7420 con 0x557a6cdfd220 2023-10-16T10:03:28.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47726/0 (socket says 172.21.15.181:47726) 2023-10-16T10:03:28.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 -- 172.21.15.181:0/1730305830 learned_addr learned my addr 172.21.15.181:0/1730305830 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604dd9b700 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6cdd4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604e59c700 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdd8010 0x557a6ce20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 -- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 msgr2=0x557a6cdd4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6cdd4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 -- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdd8010 msgr2=0x557a6ce20430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdd8010 0x557a6ce20430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 -- 172.21.15.181:0/1730305830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a6cb40c10 con 0x557a6ce02630 2023-10-16T10:03:28.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd5040 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x557a6cddbfa0 tx=0x7f6048011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b58b004c9bd7fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f6044ff9700 1 -- 172.21.15.181:0/1730305830 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048004040 con 0x557a6ce02630 2023-10-16T10:03:28.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f6044ff9700 1 -- 172.21.15.181:0/1730305830 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60480041a0 con 0x557a6ce02630 2023-10-16T10:03:28.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.474+0000 7f6044ff9700 1 -- 172.21.15.181:0/1730305830 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048004550 con 0x557a6ce02630 2023-10-16T10:03:28.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 -- 172.21.15.181:0/1730305830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 msgr2=0x557a6cdd5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd5040 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x557a6cddbfa0 tx=0x7f6048011000 comp rx=0 tx=0).stop 2023-10-16T10:03:28.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 -- 172.21.15.181:0/1730305830 shutdown_connections 2023-10-16T10:03:28.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd5040 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdd8010 0x557a6ce20430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1730305830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdfd220 0x557a6cdd4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 -- 172.21.15.181:0/1730305830 >> 172.21.15.181:0/1730305830 conn(0x557a6cd07d80 msgr2=0x557a6ce22f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 -- 172.21.15.181:0/1730305830 shutdown_connections 2023-10-16T10:03:28.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 -- 172.21.15.181:0/1730305830 wait complete. 2023-10-16T10:03:28.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 -- start start 2023-10-16T10:03:28.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.475+0000 7f6060d66b80 1 Processor -- start 2023-10-16T10:03:28.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdd8010 0x557a6ce04470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdfd220 0x557a6ce701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f6060d66b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f6060d66b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6cdd9380 con 0x557a6ce02630 2023-10-16T10:03:28.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6ce07580 con 0x557a6cdfd220 2023-10-16T10:03:28.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f6060d66b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a6cdd83f0 con 0x557a6cdd8010 2023-10-16T10:03:28.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f6060d66b80 10 client.0 _ll_register_callbacks cb 0x7f6040058850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:03:28.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47734/0 (socket says 172.21.15.181:47734) 2023-10-16T10:03:28.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f604ed9d700 1 -- 172.21.15.181:0/1898068745 learned_addr learned my addr 172.21.15.181:0/1898068745 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f604dd9b700 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdfd220 0x557a6ce701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.476+0000 7f604e59c700 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdd8010 0x557a6ce04470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f604ed9d700 1 -- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdd8010 msgr2=0x557a6ce04470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdd8010 0x557a6ce04470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f604ed9d700 1 -- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdfd220 msgr2=0x557a6ce701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdfd220 0x557a6ce701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f604ed9d700 1 -- 172.21.15.181:0/1898068745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a6ccea3a0 con 0x557a6ce02630 2023-10-16T10:03:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f604ed9d700 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd8e40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f604800b010 tx=0x7f60480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47b2e2134394081 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f602d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:03:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f602d7fa700 1 -- 172.21.15.181:0/1898068745 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60480036a0 con 0x557a6ce02630 2023-10-16T10:03:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f602d7fa700 1 -- 172.21.15.181:0/1898068745 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f6048003800 con 0x557a6ce02630 2023-10-16T10:03:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f604800b820 con 0x557a6ce02630 2023-10-16T10:03:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f602d7fa700 1 -- 172.21.15.181:0/1898068745 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60480039e0 con 0x557a6ce02630 2023-10-16T10:03:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 10 client.8694 fetch_fsmap learned FSMap version 67 2023-10-16T10:03:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x557a6cb7e200 con 0x557a6ce02630 2023-10-16T10:03:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f602d7fa700 1 -- 172.21.15.181:0/1898068745 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604800b820 con 0x557a6ce02630 2023-10-16T10:03:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f602d7fa700 1 -- 172.21.15.181:0/1898068745 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6048003b40 con 0x557a6ce02630 2023-10-16T10:03:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 10 client.8694 fetch_fsmap finished waiting for FSMap version 67 2023-10-16T10:03:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 10 client.8694 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T10:03:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 20 client.8694 populate_metadata read hostname 'smithi181' 2023-10-16T10:03:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce06dd0 0x557a6cdd4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 4 client.8694 mds_command: new command op to 8556 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:03:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 --> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557a6cd25b20 con 0x557a6ce06dd0 2023-10-16T10:03:28.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.477+0000 7f604e59c700 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce06dd0 0x557a6cdd4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f602d7fa700 10 client.8694 ms_handle_connect on v2:172.21.15.190:6833/1716409851 2023-10-16T10:03:28.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f604e59c700 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce06dd0 0x557a6cdd4a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17cc5f487600903 server_cookie=fb86a4891b21c063 in_seq=0 out_seq=0 2023-10-16T10:03:28.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f602d7fa700 1 -- 172.21.15.181:0/1898068745 <== mds.0 v2:172.21.15.190:6833/1716409851 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557a6cd25b20 con 0x557a6ce06dd0 2023-10-16T10:03:28.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f602d7fa700 10 client.8694 handle_command_reply: tid=0 2023-10-16T10:03:28.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 client.8694 shutdown 2023-10-16T10:03:28.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 msgr2=0x557a6cdd8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd8e40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f604800b010 tx=0x7f60480047c0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 shutdown_connections 2023-10-16T10:03:28.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:6833/1716409851,v1:172.21.15.190:6835/1716409851] conn(0x557a6ce06dd0 0x557a6cdd4a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557a6cdd8010 0x557a6ce04470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557a6ce02630 0x557a6cdd8e40 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 --2- 172.21.15.181:0/1898068745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a6cdfd220 0x557a6ce701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f602d7fa700 0 client.8694 ms_handle_reset on v2:172.21.15.190:6833/1716409851 2023-10-16T10:03:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 >> 172.21.15.181:0/1898068745 conn(0x557a6cd07d80 msgr2=0x557a6ce22f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 shutdown_connections 2023-10-16T10:03:28.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 1 -- 172.21.15.181:0/1898068745 wait complete. 2023-10-16T10:03:28.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.478+0000 7f6060d66b80 20 client.8694 trim_cache size 0 max 16384 2023-10-16T10:03:28.501 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:03:28.501 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T10:03:28.502 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T10:03:28.502 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 8556, 2023-10-16T10:03:28.502 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T10:03:28.502 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T10:03:28.502 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T10:03:28.503 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9001605000000001, 2023-10-16T10:03:28.503 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 67, 2023-10-16T10:03:28.503 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 132, 2023-10-16T10:03:28.503 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 132, 2023-10-16T10:03:28.503 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.833922257999999 2023-10-16T10:03:28.504 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:03:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f603006b020 msgr2=0x7f603006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f603006b020 0x7f603006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60500c6f50 tx=0x7f6038008040 comp rx=0 tx=0).stop 2023-10-16T10:03:28.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 msgr2=0x7f60500c56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f60500c56c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f604000cec0 tx=0x7f604000a5d0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 shutdown_connections 2023-10-16T10:03:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f605015df60 0x7f60501e7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605006a8a0 0x7f60500c56c0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f603006b020 0x7f603006d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 --2- 172.21.15.181:0/996122526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60500d0fc0 0x7f60501e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 >> 172.21.15.181:0/996122526 conn(0x7f60500d04a0 msgr2=0x7f605015a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 shutdown_connections 2023-10-16T10:03:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.479+0000 7f6054b7c700 1 -- 172.21.15.181:0/996122526 wait complete. 2023-10-16T10:03:28.508 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:03:28.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 Processor -- start 2023-10-16T10:03:28.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 -- start start 2023-10-16T10:03:28.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f583c15e6d0 0x7f583c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 0x7f583c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f583c0cb0c0 0x7f583c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c06ed00 con 0x7f583c15e6d0 2023-10-16T10:03:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c0d02d0 con 0x7f583c0cb0c0 2023-10-16T10:03:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f58421b2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c06e0f0 con 0x7f583c15eab0 2023-10-16T10:03:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.629+0000 7f583b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f583c15e6d0 0x7f583c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f583c15e6d0 0x7f583c0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47738/0 (socket says 172.21.15.181:47738) 2023-10-16T10:03:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 -- 172.21.15.181:0/4207917514 learned_addr learned my addr 172.21.15.181:0/4207917514 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 -- 172.21.15.181:0/4207917514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 msgr2=0x7f583c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 --2- 172.21.15.181:0/4207917514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 0x7f583c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 -- 172.21.15.181:0/4207917514 >> v2:172.21.15.190:3300/0 conn(0x7f583c0cb0c0 msgr2=0x7f583c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 --2- 172.21.15.181:0/4207917514 >> v2:172.21.15.190:3300/0 conn(0x7f583c0cb0c0 0x7f583c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 -- 172.21.15.181:0/4207917514 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c06e3d0 con 0x7f583c15e6d0 2023-10-16T10:03:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583b7fe700 1 --2- 172.21.15.181:0/4207917514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c15e6d0 0x7f583c0d0fc0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f583c0705c0 tx=0x7f582c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeca7561a099932a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583a7fc700 1 -- 172.21.15.181:0/4207917514 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c005e30 con 0x7f583c15e6d0 2023-10-16T10:03:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583a7fc700 1 -- 172.21.15.181:0/4207917514 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c004030 con 0x7f583c15e6d0 2023-10-16T10:03:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.630+0000 7f583a7fc700 1 -- 172.21.15.181:0/4207917514 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0043e0 con 0x7f583c15e6d0 2023-10-16T10:03:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 -- 172.21.15.181:0/4207917514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c15e6d0 msgr2=0x7f583c0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 --2- 172.21.15.181:0/4207917514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c15e6d0 0x7f583c0d0fc0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f583c0705c0 tx=0x7f582c009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 -- 172.21.15.181:0/4207917514 shutdown_connections 2023-10-16T10:03:28.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 --2- 172.21.15.181:0/4207917514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c15e6d0 0x7f583c0d0fc0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 --2- 172.21.15.181:0/4207917514 >> v2:172.21.15.190:3300/0 conn(0x7f583c0cb0c0 0x7f583c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 --2- 172.21.15.181:0/4207917514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 0x7f583c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 -- 172.21.15.181:0/4207917514 >> 172.21.15.181:0/4207917514 conn(0x7f583c0d04a0 msgr2=0x7f583c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 -- 172.21.15.181:0/4207917514 shutdown_connections 2023-10-16T10:03:28.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.631+0000 7f58421b2700 1 -- 172.21.15.181:0/4207917514 wait complete. 2023-10-16T10:03:28.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.632+0000 7f58421b2700 1 Processor -- start 2023-10-16T10:03:28.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.632+0000 7f58421b2700 1 -- start start 2023-10-16T10:03:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f58421b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c0cb0c0 0x7f583c1f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f58421b2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c15e6d0 0x7f583c0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f58421b2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 0x7f583c1f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f58421b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1f74d0 con 0x7f583c0cb0c0 2023-10-16T10:03:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f58421b2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1f43f0 con 0x7f583c15e6d0 2023-10-16T10:03:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f58421b2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1f6e80 con 0x7f583c15eab0 2023-10-16T10:03:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c0cb0c0 0x7f583c1f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c0cb0c0 0x7f583c1f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47744/0 (socket says 172.21.15.181:47744) 2023-10-16T10:03:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 -- 172.21.15.181:0/1996998742 learned_addr learned my addr 172.21.15.181:0/1996998742 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:28.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 -- 172.21.15.181:0/1996998742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 msgr2=0x7f583c1f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 0x7f583c1f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 -- 172.21.15.181:0/1996998742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c15e6d0 msgr2=0x7f583c0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583affd700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c15e6d0 0x7f583c0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c15e6d0 0x7f583c0b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 -- 172.21.15.181:0/1996998742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c009500 con 0x7f583c0cb0c0 2023-10-16T10:03:28.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f583b7fe700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c0cb0c0 0x7f583c1f7960 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f582c009130 tx=0x7f582c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f5833fff700 1 -- 172.21.15.181:0/1996998742 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c01d630 con 0x7f583c0cb0c0 2023-10-16T10:03:28.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f5833fff700 1 -- 172.21.15.181:0/1996998742 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c01d7b0 con 0x7f583c0cb0c0 2023-10-16T10:03:28.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c1f7160 con 0x7f583c0cb0c0 2023-10-16T10:03:28.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.633+0000 7f5833fff700 1 -- 172.21.15.181:0/1996998742 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c01db60 con 0x7f583c0cb0c0 2023-10-16T10:03:28.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.634+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c1f46d0 con 0x7f583c0cb0c0 2023-10-16T10:03:28.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.634+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c067d20 con 0x7f583c0cb0c0 2023-10-16T10:03:28.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.635+0000 7f5833fff700 1 -- 172.21.15.181:0/1996998742 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f582c01dcc0 con 0x7f583c0cb0c0 2023-10-16T10:03:28.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.635+0000 7f5833fff700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f581c06b020 0x7f581c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:28.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.635+0000 7f5833fff700 1 -- 172.21.15.181:0/1996998742 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f582c08af70 con 0x7f583c0cb0c0 2023-10-16T10:03:28.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.635+0000 7f583affd700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f581c06b020 0x7f581c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:28.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.636+0000 7f583affd700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f581c06b020 0x7f581c06d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:28.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.640+0000 7f5833fff700 1 -- 172.21.15.181:0/1996998742 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f582c0bdfe0 con 0x7f583c0cb0c0 2023-10-16T10:03:28.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.875+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f583c0cb610 con 0x7f583c0cb0c0 2023-10-16T10:03:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.876+0000 7f5833fff700 1 -- 172.21.15.181:0/1996998742 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f582c016060 con 0x7f583c0cb0c0 2023-10-16T10:03:28.878 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:03:28.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f581c06b020 msgr2=0x7f581c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f581c06b020 0x7f581c06d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).stop 2023-10-16T10:03:28.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c0cb0c0 msgr2=0x7f583c1f7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:28.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c0cb0c0 0x7f583c1f7960 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f582c009130 tx=0x7f582c004750 comp rx=0 tx=0).stop 2023-10-16T10:03:28.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 shutdown_connections 2023-10-16T10:03:28.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f581c06b020 0x7f581c06d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f583c15eab0 0x7f583c1f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f583c0cb0c0 0x7f583c1f7960 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 --2- 172.21.15.181:0/1996998742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f583c15e6d0 0x7f583c0b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:28.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 >> 172.21.15.181:0/1996998742 conn(0x7f583c0d04a0 msgr2=0x7f583c0af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:28.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 shutdown_connections 2023-10-16T10:03:28.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:28.878+0000 7f58421b2700 1 -- 172.21.15.181:0/1996998742 wait complete. 2023-10-16T10:03:28.892 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:03:28.892 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:03:28.892 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T10:03:28.893 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:03:28.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:03:28.893 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:03:28.894 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:03:28.894 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:03:28.894 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:03:28.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:28.895 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T10:03:28.912 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:03:28.913 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:03:28.913 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:03:28.913 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:03:28.913 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T10:03:28.913 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:03:28.914 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:03:28.914 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:03:28.914 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:03:28.914 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:03:28.914 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:03:28.915 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T10:03:28.915 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:03:28.915 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T10:03:28.915 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:03:28.915 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41261sec preferred_lft 41261sec 2023-10-16T10:03:28.916 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T10:03:28.916 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:03:28.916 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:03:28.916 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:03:28.916 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:03:28.917 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:03:28.917 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T10:03:28.917 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:03:28.917 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:03:28.917 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:03:28.917 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T10:03:28.918 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:03:28.918 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:03:28.918 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:03:28.918 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T10:03:28.919 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:03:28.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:28.919 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:03:28.979 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T10:03:28.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:28.981 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T10:03:29.039 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T10:03:29.040 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:03:29.040 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:03:29.040 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:03:29.040 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:03:29.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:29.041 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T10:03:29.041 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T10:03:29.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:03:29.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:03:29.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:03:29.043 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T10:03:29.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:29.145 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T10:03:29.161 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:03:29.162 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:03:29.162 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:03:29.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:29.163 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T10:03:29.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:03:29.228 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:03:29.285 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:03:29.285 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:03:29.286 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:03:29.342 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T10:03:29.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:29.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:03:29.427 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:03:29.428 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:03:29.428 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:03:29.436 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:03:29.436 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:03:29.437 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:03:29.437 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:03:29.437 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:03:29.464 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:03:29.464 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:03:29.464 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:03:29.465 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:03:29.465 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:03:29.465 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T10:03:29.465 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T10:03:29.465 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T10:03:29.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:03:29.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:29.467 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:03:29.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:29.498 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:03:29.586 INFO:teuthology.orchestra.run.smithi181.stdout:client8703 2023-10-16T10:03:29.586 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:29.587 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:29.603 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:03:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.762+0000 7f65cd8ea700 1 Processor -- start 2023-10-16T10:03:29.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.762+0000 7f65cd8ea700 1 -- start start 2023-10-16T10:03:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65cd8ea700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 0x7f65c80d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65cd8ea700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f65c80d26b0 0x7f65c80b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65cd8ea700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80d0940 con 0x7f65c815dda0 2023-10-16T10:03:29.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65cd8ea700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80d2b20 con 0x7f65c80d26b0 2023-10-16T10:03:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65cd8ea700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80d0c20 con 0x7f65c80cfa60 2023-10-16T10:03:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65c6ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f65c815dda0 0x7f65c80d0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58834/0 (socket says 172.21.15.181:58834) 2023-10-16T10:03:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65c6ffd700 1 -- 172.21.15.181:0/120525293 learned_addr learned my addr 172.21.15.181:0/120525293 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359860616 0 0) 0x7f65c80d0940 con 0x7f65c815dda0 2023-10-16T10:03:29.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b00035b0 con 0x7f65c815dda0 2023-10-16T10:03:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65c77fe700 1 --2- 172.21.15.181:0/120525293 >> v2:172.21.15.190:3300/0 conn(0x7f65c80d26b0 0x7f65c80b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.763+0000 7f65c67fc700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 0x7f65c80d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2831428322 0 0) 0x7f65b00035b0 con 0x7f65c815dda0 2023-10-16T10:03:29.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 msgr2=0x7f65c80d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 0x7f65c80d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 >> v2:172.21.15.190:3300/0 conn(0x7f65c80d26b0 msgr2=0x7f65c80b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 --2- 172.21.15.181:0/120525293 >> v2:172.21.15.190:3300/0 conn(0x7f65c80d26b0 0x7f65c80b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80b2c20 con 0x7f65c815dda0 2023-10-16T10:03:29.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f65bc0017f0 con 0x7f65c815dda0 2023-10-16T10:03:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 >> v1:172.21.15.181:6789/0 conn(0x7f65c815dda0 legacy=0x7f65c80d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 0x7f65b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65b0004d90 0x7f65b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80d2b20 con 0x7f65b0003a00 2023-10-16T10:03:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80d0c20 con 0x7f65b0004d90 2023-10-16T10:03:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b000ba10 con 0x7f65c80cfa60 2023-10-16T10:03:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c6ffd700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c77fe700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65b0004d90 0x7f65b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c67fc700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 0x7f65b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c6ffd700 1 -- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 msgr2=0x7f65b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c6ffd700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 0x7f65b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c6ffd700 1 -- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65b0004d90 msgr2=0x7f65b0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c6ffd700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65b0004d90 0x7f65b0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c6ffd700 1 -- 172.21.15.181:0/120525293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00035b0 con 0x7f65b0003a00 2023-10-16T10:03:29.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c67fc700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c80cfa60 0x7f65b0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c77fe700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65b0004d90 0x7f65b0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c6ffd700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65b0003e20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c270 tx=0x7f65b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2342b27754053b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00cdd0 con 0x7f65b0003a00 2023-10-16T10:03:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc007bd0 con 0x7f65b0003a00 2023-10-16T10:03:29.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.764+0000 7f65c5ffb700 1 -- 172.21.15.181:0/120525293 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc007310 con 0x7f65b0003a00 2023-10-16T10:03:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.765+0000 7f65cd8ea700 1 -- 172.21.15.181:0/120525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 msgr2=0x7f65b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.765+0000 7f65cd8ea700 1 --2- 172.21.15.181:0/120525293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65b0003e20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c270 tx=0x7f65b0005880 comp rx=0 tx=0).stop 2023-10-16T10:03:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.765+0000 7f65c77fe700 1 -- 172.21.15.181:0/120525293 reap_dead start 2023-10-16T10:03:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.765+0000 7f65cd8ea700 1 -- 172.21.15.181:0/120525293 shutdown_connections 2023-10-16T10:03:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.765+0000 7f65cd8ea700 1 -- 172.21.15.181:0/120525293 >> 172.21.15.181:0/120525293 conn(0x7f65c809cb10 msgr2=0x7f65c80d3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:29.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.765+0000 7f65cd8ea700 1 -- 172.21.15.181:0/120525293 shutdown_connections 2023-10-16T10:03:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.765+0000 7f65cd8ea700 1 -- 172.21.15.181:0/120525293 wait complete. 2023-10-16T10:03:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.766+0000 7f65cd8ea700 1 Processor -- start 2023-10-16T10:03:29.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.766+0000 7f65cd8ea700 1 -- start start 2023-10-16T10:03:29.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65cd8ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65c80c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65cd8ea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65c80c9aa0 0x7f65c80b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65cd8ea700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c8160e40 0x7f65c80b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65cd8ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c814a7d0 con 0x7f65b0003a00 2023-10-16T10:03:29.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65cd8ea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80c84c0 con 0x7f65c80c9aa0 2023-10-16T10:03:29.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65cd8ea700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80c87a0 con 0x7f65c8160e40 2023-10-16T10:03:29.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65c80c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65c80c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47768/0 (socket says 172.21.15.181:47768) 2023-10-16T10:03:29.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c6ffd700 1 -- 172.21.15.181:0/2253715127 learned_addr learned my addr 172.21.15.181:0/2253715127 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:29.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c77fe700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c8160e40 0x7f65c80b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c67fc700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65c80c9aa0 0x7f65c80b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c6ffd700 1 -- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c8160e40 msgr2=0x7f65c80b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c6ffd700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c8160e40 0x7f65c80b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c6ffd700 1 -- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65c80c9aa0 msgr2=0x7f65c80b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.767+0000 7f65c6ffd700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65c80c9aa0 0x7f65c80b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.768+0000 7f65c6ffd700 1 -- 172.21.15.181:0/2253715127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80c8a80 con 0x7f65b0003a00 2023-10-16T10:03:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.768+0000 7f65c6ffd700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65c80c9560 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f65bc001ab0 tx=0x7f65bc001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.768+0000 7f65affff700 1 -- 172.21.15.181:0/2253715127 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc005e70 con 0x7f65b0003a00 2023-10-16T10:03:29.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.768+0000 7f65affff700 1 -- 172.21.15.181:0/2253715127 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc0070d0 con 0x7f65b0003a00 2023-10-16T10:03:29.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.768+0000 7f65affff700 1 -- 172.21.15.181:0/2253715127 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc007460 con 0x7f65b0003a00 2023-10-16T10:03:29.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.768+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c814a7d0 con 0x7f65b0003a00 2023-10-16T10:03:29.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.768+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c80c9040 con 0x7f65b0003a00 2023-10-16T10:03:29.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.769+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c80c91d0 con 0x7f65b0003a00 2023-10-16T10:03:29.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.769+0000 7f65affff700 1 -- 172.21.15.181:0/2253715127 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f65bc019030 con 0x7f65b0003a00 2023-10-16T10:03:29.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.770+0000 7f65affff700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f65b406b020 0x7f65b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:29.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.770+0000 7f65affff700 1 -- 172.21.15.181:0/2253715127 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f65bc08add0 con 0x7f65b0003a00 2023-10-16T10:03:29.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.770+0000 7f65c67fc700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f65b406b020 0x7f65b406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:29.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.770+0000 7f65c67fc700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f65b406b020 0x7f65b406d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65c80c9ea0 tx=0x7f65b8008040 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:29.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.774+0000 7f65affff700 1 -- 172.21.15.181:0/2253715127 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65bc055ce0 con 0x7f65b0003a00 2023-10-16T10:03:29.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.992+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 --> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f65c80b31e0 con 0x7f65b406b020 2023-10-16T10:03:29.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.994+0000 7f65affff700 1 -- 172.21.15.181:0/2253715127 <== mgr.8304 v2:172.21.15.181:6832/3280351398 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f65b8009810 con 0x7f65b406b020 2023-10-16T10:03:29.996 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:03:29.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:29.996 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T10:03:29.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:29.996 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:03:29.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f65b406b020 msgr2=0x7f65b406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:29.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f65b406b020 0x7f65b406d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f65c80c9ea0 tx=0x7f65b8008040 comp rx=0 tx=0).stop 2023-10-16T10:03:29.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 msgr2=0x7f65c80c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:29.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65c80c9560 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f65bc001ab0 tx=0x7f65bc001160 comp rx=0 tx=0).stop 2023-10-16T10:03:29.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 shutdown_connections 2023-10-16T10:03:29.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f65b406b020 0x7f65b406d4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65c8160e40 0x7f65c80b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:29.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65b0003a00 0x7f65c80c9560 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 --2- 172.21.15.181:0/2253715127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65c80c9aa0 0x7f65c80b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 >> 172.21.15.181:0/2253715127 conn(0x7f65c809cb10 msgr2=0x7f65c806d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:30.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 shutdown_connections 2023-10-16T10:03:30.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:29.996+0000 7f65cd8ea700 1 -- 172.21.15.181:0/2253715127 wait complete. 2023-10-16T10:03:30.010 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:03:30.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.130+0000 7faa0dae8700 1 Processor -- start 2023-10-16T10:03:30.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.130+0000 7faa0dae8700 1 -- start start 2023-10-16T10:03:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.131+0000 7faa0dae8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa080cdd80 0x7faa080ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.131+0000 7faa0dae8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0815dfb0 con 0x7faa0806b6f0 2023-10-16T10:03:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.131+0000 7faa0dae8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa080cd3a0 con 0x7faa0815dbd0 2023-10-16T10:03:30.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.131+0000 7faa0dae8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa080ce740 con 0x7faa080cdd80 2023-10-16T10:03:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.131+0000 7faa06ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa080cdd80 0x7faa080ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.131+0000 7faa07fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faa0806b6f0 0x7faa0806bba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58858/0 (socket says 172.21.15.181:58858) 2023-10-16T10:03:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.131+0000 7faa07fff700 1 -- 172.21.15.181:0/3739256209 learned_addr learned my addr 172.21.15.181:0/3739256209 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa06ffd700 1 -- 172.21.15.181:0/3739256209 >> v1:172.21.15.190:6789/0 conn(0x7faa0815dbd0 legacy=0x7faa080d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa06ffd700 1 -- 172.21.15.181:0/3739256209 >> v1:172.21.15.181:6789/0 conn(0x7faa0806b6f0 legacy=0x7faa0806bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa067fc700 1 -- 172.21.15.181:0/3739256209 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581746514 0 0) 0x7faa00001490 con 0x7faa0806b6f0 2023-10-16T10:03:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa06ffd700 1 -- 172.21.15.181:0/3739256209 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa080cd830 con 0x7faa080cdd80 2023-10-16T10:03:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa06ffd700 1 --2- 172.21.15.181:0/3739256209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa080cdd80 0x7faa080ce200 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef368a760ea47e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa067fc700 1 -- 172.21.15.181:0/3739256209 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004230 con 0x7faa080cdd80 2023-10-16T10:03:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa067fc700 1 -- 172.21.15.181:0/3739256209 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8004390 con 0x7faa080cdd80 2023-10-16T10:03:30.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa067fc700 1 -- 172.21.15.181:0/3739256209 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80036a0 con 0x7faa080cdd80 2023-10-16T10:03:30.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa0dae8700 1 -- 172.21.15.181:0/3739256209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa080cdd80 msgr2=0x7faa080ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa0dae8700 1 --2- 172.21.15.181:0/3739256209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa080cdd80 0x7faa080ce200 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:30.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa0dae8700 1 -- 172.21.15.181:0/3739256209 shutdown_connections 2023-10-16T10:03:30.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa0dae8700 1 --2- 172.21.15.181:0/3739256209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa080cdd80 0x7faa080ce200 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa0dae8700 1 -- 172.21.15.181:0/3739256209 >> 172.21.15.181:0/3739256209 conn(0x7faa0809a930 msgr2=0x7faa080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:30.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa0dae8700 1 -- 172.21.15.181:0/3739256209 shutdown_connections 2023-10-16T10:03:30.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.132+0000 7faa0dae8700 1 -- 172.21.15.181:0/3739256209 wait complete. 2023-10-16T10:03:30.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.134+0000 7faa0dae8700 1 Processor -- start 2023-10-16T10:03:30.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.134+0000 7faa0dae8700 1 -- start start 2023-10-16T10:03:30.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.134+0000 7faa0dae8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0806b6f0 0x7faa0814b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.134+0000 7faa0dae8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa080cdd80 0x7faa0809ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.134+0000 7faa0dae8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa0815dbd0 0x7faa08160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.134+0000 7faa0dae8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080c4f70 con 0x7faa0806b6f0 2023-10-16T10:03:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa0dae8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080c5230 con 0x7faa080cdd80 2023-10-16T10:03:30.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa0dae8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa080c5510 con 0x7faa0815dbd0 2023-10-16T10:03:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0806b6f0 0x7faa0814b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0806b6f0 0x7faa0814b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47782/0 (socket says 172.21.15.181:47782) 2023-10-16T10:03:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 -- 172.21.15.181:0/1082245095 learned_addr learned my addr 172.21.15.181:0/1082245095 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa07fff700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa0815dbd0 0x7faa08160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 -- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa0815dbd0 msgr2=0x7faa08160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa06ffd700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa080cdd80 0x7faa0809ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa0815dbd0 0x7faa08160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 -- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa080cdd80 msgr2=0x7faa0809ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa080cdd80 0x7faa0809ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 -- 172.21.15.181:0/1082245095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80094a0 con 0x7faa0806b6f0 2023-10-16T10:03:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa077fe700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0806b6f0 0x7faa0814b480 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c4d0 tx=0x7fa9fc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7fa9effff700 1 -- 172.21.15.181:0/1082245095 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc011790 con 0x7faa0806b6f0 2023-10-16T10:03:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7fa9effff700 1 -- 172.21.15.181:0/1082245095 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc0118f0 con 0x7faa0806b6f0 2023-10-16T10:03:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa080c57f0 con 0x7faa0806b6f0 2023-10-16T10:03:30.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7fa9effff700 1 -- 172.21.15.181:0/1082245095 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc011ca0 con 0x7faa0806b6f0 2023-10-16T10:03:30.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.135+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0809ef60 con 0x7faa0806b6f0 2023-10-16T10:03:30.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.136+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa08067d20 con 0x7faa0806b6f0 2023-10-16T10:03:30.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.137+0000 7fa9effff700 1 -- 172.21.15.181:0/1082245095 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9fc01c460 con 0x7faa0806b6f0 2023-10-16T10:03:30.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.137+0000 7fa9effff700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa9f406b020 0x7fa9f406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.137+0000 7fa9effff700 1 -- 172.21.15.181:0/1082245095 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fa9fc089770 con 0x7faa0806b6f0 2023-10-16T10:03:30.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.137+0000 7faa06ffd700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa9f406b020 0x7fa9f406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.138+0000 7faa06ffd700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa9f406b020 0x7fa9f406d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8008030 comp rx=0 tx=0).ready entity=mgr.8304 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:30.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.142+0000 7fa9effff700 1 -- 172.21.15.181:0/1082245095 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa9fc0545d0 con 0x7faa0806b6f0 2023-10-16T10:03:30.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.388+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7faa080ccd40 con 0x7faa0806b6f0 2023-10-16T10:03:30.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.616+0000 7fa9effff700 1 -- 172.21.15.181:0/1082245095 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v76) v1 ==== 92+0+0 (secure 0 0 0) 0x7fa9fc057bf0 con 0x7faa0806b6f0 2023-10-16T10:03:30.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa9f406b020 msgr2=0x7fa9f406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa9f406b020 0x7fa9f406d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8008030 comp rx=0 tx=0).stop 2023-10-16T10:03:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0806b6f0 msgr2=0x7faa0814b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0806b6f0 0x7faa0814b480 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00c4d0 tx=0x7fa9fc00c890 comp rx=0 tx=0).stop 2023-10-16T10:03:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 shutdown_connections 2023-10-16T10:03:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fa9f406b020 0x7fa9f406d4e0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8008030 comp rx=0 tx=0).stop 2023-10-16T10:03:30.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa0815dbd0 0x7faa08160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0806b6f0 0x7faa0814b480 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 --2- 172.21.15.181:0/1082245095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa080cdd80 0x7faa0809ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 >> 172.21.15.181:0/1082245095 conn(0x7faa0809a930 msgr2=0x7faa080d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:30.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.620+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 shutdown_connections 2023-10-16T10:03:30.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.621+0000 7faa0dae8700 1 -- 172.21.15.181:0/1082245095 wait complete. 2023-10-16T10:03:30.638 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:03:30.767 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:30.765+0000 7fd7540ed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:03:30.768 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:30.766+0000 7eff9e42d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:03:30.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.767+0000 7f771dc88700 1 Processor -- start 2023-10-16T10:03:30.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.767+0000 7f771dc88700 1 -- start start 2023-10-16T10:03:30.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f771dc88700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f771806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f771dc88700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f771806ade0 0x7f77181615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f771dc88700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771806b230 con 0x7f771815da70 2023-10-16T10:03:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f771dc88700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180cef00 con 0x7f771806ade0 2023-10-16T10:03:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f771dc88700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77180cec20 con 0x7f7718072760 2023-10-16T10:03:30.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f771806ade0 0x7f77181615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f771806ade0 0x7f77181615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47820/0 (socket says 172.21.15.181:47820) 2023-10-16T10:03:30.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 -- 172.21.15.181:0/2506871875 learned_addr learned my addr 172.21.15.181:0/2506871875 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:30.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7716ffd700 1 --2- 172.21.15.181:0/2506871875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f771806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 -- 172.21.15.181:0/2506871875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 msgr2=0x7f771806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:30.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 --2- 172.21.15.181:0/2506871875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f771806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 -- 172.21.15.181:0/2506871875 >> v1:172.21.15.190:6789/0 conn(0x7f771815da70 legacy=0x7f77180ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 -- 172.21.15.181:0/2506871875 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718071c80 con 0x7f771806ade0 2023-10-16T10:03:30.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.768+0000 7f7717fff700 1 --2- 172.21.15.181:0/2506871875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77181615f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7708009e50 tx=0x7f770800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c10691062be61b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:30.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f77167fc700 1 -- 172.21.15.181:0/2506871875 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800e070 con 0x7f771806ade0 2023-10-16T10:03:30.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f77167fc700 1 -- 172.21.15.181:0/2506871875 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800b910 con 0x7f771806ade0 2023-10-16T10:03:30.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f77167fc700 1 -- 172.21.15.181:0/2506871875 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800bcc0 con 0x7f771806ade0 2023-10-16T10:03:30.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 -- 172.21.15.181:0/2506871875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 msgr2=0x7f77181615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 --2- 172.21.15.181:0/2506871875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77181615f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7708009e50 tx=0x7f770800b330 comp rx=0 tx=0).stop 2023-10-16T10:03:30.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 -- 172.21.15.181:0/2506871875 shutdown_connections 2023-10-16T10:03:30.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 --2- 172.21.15.181:0/2506871875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77181615f0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 --2- 172.21.15.181:0/2506871875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f771806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 -- 172.21.15.181:0/2506871875 >> 172.21.15.181:0/2506871875 conn(0x7f77180cd010 msgr2=0x7f77180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:30.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 -- 172.21.15.181:0/2506871875 shutdown_connections 2023-10-16T10:03:30.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.769+0000 7f771dc88700 1 -- 172.21.15.181:0/2506871875 wait complete. 2023-10-16T10:03:30.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.770+0000 7f771dc88700 1 Processor -- start 2023-10-16T10:03:30.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.771+0000 7f771dc88700 1 -- start start 2023-10-16T10:03:30.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.771+0000 7f771dc88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77180ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.771+0000 7f771dc88700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f77180b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.771+0000 7f771dc88700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815da70 0x7f77180c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.771+0000 7f771dc88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180aeb30 con 0x7f771806ade0 2023-10-16T10:03:30.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.771+0000 7f771dc88700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180aee10 con 0x7f771815da70 2023-10-16T10:03:30.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.771+0000 7f771dc88700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77180af0f0 con 0x7f7718072760 2023-10-16T10:03:30.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77180ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77180ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47822/0 (socket says 172.21.15.181:47822) 2023-10-16T10:03:30.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 -- 172.21.15.181:0/2791366724 learned_addr learned my addr 172.21.15.181:0/2791366724 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:30.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f7716ffd700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f77180b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f7717fff700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815da70 0x7f77180c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:30.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 -- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 msgr2=0x7f77180b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f77180b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 -- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815da70 msgr2=0x7f77180c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815da70 0x7f77180c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 -- 172.21.15.181:0/2791366724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708009b30 con 0x7f771806ade0 2023-10-16T10:03:30.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f7716ffd700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f77180b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:30.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f77177fe700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77180ae2e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f771000c450 tx=0x7f771000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:30.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f76fffff700 1 -- 172.21.15.181:0/2791366724 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77100052d0 con 0x7f771806ade0 2023-10-16T10:03:30.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f76fffff700 1 -- 172.21.15.181:0/2791366724 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7710005430 con 0x7f771806ade0 2023-10-16T10:03:30.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f76fffff700 1 -- 172.21.15.181:0/2791366724 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77100038c0 con 0x7f771806ade0 2023-10-16T10:03:30.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77180af3d0 con 0x7f771806ade0 2023-10-16T10:03:30.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.772+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77180ae820 con 0x7f771806ade0 2023-10-16T10:03:30.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.773+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77180c46d0 con 0x7f771806ade0 2023-10-16T10:03:30.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.774+0000 7f76fffff700 1 -- 172.21.15.181:0/2791366724 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 76) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7710003a20 con 0x7f771806ade0 2023-10-16T10:03:30.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.774+0000 7f76fffff700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 0x7f770406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:30.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.774+0000 7f7716ffd700 1 -- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 msgr2=0x7f770406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:30.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.774+0000 7f76fffff700 1 -- 172.21.15.181:0/2791366724 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f7710088b80 con 0x7f771806ade0 2023-10-16T10:03:30.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.774+0000 7f7716ffd700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 0x7f770406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:30.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.779+0000 7f76fffff700 1 -- 172.21.15.181:0/2791366724 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f77100538c0 con 0x7f771806ade0 2023-10-16T10:03:30.834 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:30.832+0000 7fd7540ed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:30.837 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:30.836+0000 7eff9e42d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:30.905 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:30.904+0000 7fd7540ed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:03:30.911 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:30.910+0000 7eff9e42d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:03:30.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.975+0000 7f7716ffd700 1 -- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 msgr2=0x7f770406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:30.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.975+0000 7f7716ffd700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 0x7f770406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:30.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.978+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f771806b230 con 0x7f771806ade0 2023-10-16T10:03:30.983 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:30.982+0000 7fd7540ed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:03:30.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.983+0000 7f76fffff700 1 -- 172.21.15.181:0/2791366724 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f771806b230 con 0x7f771806ade0 2023-10-16T10:03:30.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 msgr2=0x7f770406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:30.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 0x7f770406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 msgr2=0x7f77180ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:30.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77180ae2e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f771000c450 tx=0x7f771000c760 comp rx=0 tx=0).stop 2023-10-16T10:03:30.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 shutdown_connections 2023-10-16T10:03:30.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f771815da70 0x7f77180c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f771806ade0 0x7f77180ae2e0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7f770406b070 0x7f770406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 --2- 172.21.15.181:0/2791366724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7718072760 0x7f77180b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:30.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 >> 172.21.15.181:0/2791366724 conn(0x7f77180cd010 msgr2=0x7f77180b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:30.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 shutdown_connections 2023-10-16T10:03:30.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:30.986+0000 7f771dc88700 1 -- 172.21.15.181:0/2791366724 wait complete. 2023-10-16T10:03:30.991 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:30.989+0000 7eff9e42d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:03:31.001 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:03:31.125 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:31.124+0000 7fd7540ed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:03:31.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.125+0000 7fd17a76b700 1 Processor -- start 2023-10-16T10:03:31.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.125+0000 7fd17a76b700 1 -- start start 2023-10-16T10:03:31.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd17a76b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd174072760 0x7fd17406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd17a76b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17406b210 con 0x7fd17406ade0 2023-10-16T10:03:31.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd17a76b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1740cef00 con 0x7fd17415da70 2023-10-16T10:03:31.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd17a76b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1740cec20 con 0x7fd174072760 2023-10-16T10:03:31.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd178d08700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd17406ade0 0x7fd1741615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58954/0 (socket says 172.21.15.181:58954) 2023-10-16T10:03:31.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd178d08700 1 -- 172.21.15.181:0/364918273 learned_addr learned my addr 172.21.15.181:0/364918273 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:31.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd1737fe700 1 --2- 172.21.15.181:0/364918273 >> v2:172.21.15.190:3300/0 conn(0x7fd174072760 0x7fd17406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd172ffd700 1 -- 172.21.15.181:0/364918273 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036189231 0 0) 0x7fd17406b210 con 0x7fd17406ade0 2023-10-16T10:03:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd172ffd700 1 -- 172.21.15.181:0/364918273 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1600035b0 con 0x7fd17406ade0 2023-10-16T10:03:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd1737fe700 1 -- 172.21.15.181:0/364918273 >> v1:172.21.15.190:6789/0 conn(0x7fd17415da70 legacy=0x7fd1740ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd1737fe700 1 -- 172.21.15.181:0/364918273 >> v1:172.21.15.181:6789/0 conn(0x7fd17406ade0 legacy=0x7fd1741615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.126+0000 7fd1737fe700 1 -- 172.21.15.181:0/364918273 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174071c80 con 0x7fd174072760 2023-10-16T10:03:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd1737fe700 1 --2- 172.21.15.181:0/364918273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd17406a8a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd164009840 tx=0x7fd164009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b54b5073f208ef3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd172ffd700 1 -- 172.21.15.181:0/364918273 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164011830 con 0x7fd174072760 2023-10-16T10:03:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd172ffd700 1 -- 172.21.15.181:0/364918273 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd164011990 con 0x7fd174072760 2023-10-16T10:03:31.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd172ffd700 1 -- 172.21.15.181:0/364918273 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164011d00 con 0x7fd174072760 2023-10-16T10:03:31.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd17a76b700 1 -- 172.21.15.181:0/364918273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 msgr2=0x7fd17406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd17a76b700 1 --2- 172.21.15.181:0/364918273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd17406a8a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd164009840 tx=0x7fd164009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:31.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd17a76b700 1 -- 172.21.15.181:0/364918273 shutdown_connections 2023-10-16T10:03:31.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd17a76b700 1 --2- 172.21.15.181:0/364918273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd17406a8a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd17a76b700 1 -- 172.21.15.181:0/364918273 >> 172.21.15.181:0/364918273 conn(0x7fd1740cd010 msgr2=0x7fd1740cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:31.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd17a76b700 1 -- 172.21.15.181:0/364918273 shutdown_connections 2023-10-16T10:03:31.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.127+0000 7fd17a76b700 1 -- 172.21.15.181:0/364918273 wait complete. 2023-10-16T10:03:31.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.129+0000 7fd17a76b700 1 Processor -- start 2023-10-16T10:03:31.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.129+0000 7fd17a76b700 1 -- start start 2023-10-16T10:03:31.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd17a76b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd17406ade0 0x7fd174154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd17a76b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd1740c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd17a76b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd17415da70 0x7fd1740b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd17a76b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c87e0 con 0x7fd17406ade0 2023-10-16T10:03:31.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd17a76b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c7f40 con 0x7fd174072760 2023-10-16T10:03:31.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd17a76b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c8220 con 0x7fd17415da70 2023-10-16T10:03:31.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd173fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd17406ade0 0x7fd174154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd173fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd17406ade0 0x7fd174154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47880/0 (socket says 172.21.15.181:47880) 2023-10-16T10:03:31.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd1737fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd1740c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd173fff700 1 -- 172.21.15.181:0/4248161023 learned_addr learned my addr 172.21.15.181:0/4248161023 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:31.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.130+0000 7fd178d08700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd17415da70 0x7fd1740b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd1737fe700 1 -- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd17415da70 msgr2=0x7fd1740b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd1737fe700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd17415da70 0x7fd1740b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd1737fe700 1 -- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd17406ade0 msgr2=0x7fd174154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd1737fe700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd17406ade0 0x7fd174154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd1737fe700 1 -- 172.21.15.181:0/4248161023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1640094a0 con 0x7fd174072760 2023-10-16T10:03:31.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd1737fe700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd1740c7660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd1740b66b0 tx=0x7fd164004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:31.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd170ff9700 1 -- 172.21.15.181:0/4248161023 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1640045d0 con 0x7fd174072760 2023-10-16T10:03:31.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd170ff9700 1 -- 172.21.15.181:0/4248161023 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16401f070 con 0x7fd174072760 2023-10-16T10:03:31.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1740c8500 con 0x7fd174072760 2023-10-16T10:03:31.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd170ff9700 1 -- 172.21.15.181:0/4248161023 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1640038c0 con 0x7fd174072760 2023-10-16T10:03:31.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.131+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd174155020 con 0x7fd174072760 2023-10-16T10:03:31.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.132+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17415eb70 con 0x7fd174072760 2023-10-16T10:03:31.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.132+0000 7fd170ff9700 1 -- 172.21.15.181:0/4248161023 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 76) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fd164003a20 con 0x7fd174072760 2023-10-16T10:03:31.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.133+0000 7fd170ff9700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 0x7fd15406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.133+0000 7fd173fff700 1 -- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 msgr2=0x7fd15406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:31.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.133+0000 7fd170ff9700 1 -- 172.21.15.181:0/4248161023 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fd164021070 con 0x7fd174072760 2023-10-16T10:03:31.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.133+0000 7fd173fff700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 0x7fd15406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:31.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.141+0000 7fd170ff9700 1 -- 172.21.15.181:0/4248161023 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd17415eb70 con 0x7fd174072760 2023-10-16T10:03:31.146 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:31.143+0000 7eff9e42d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:03:31.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.334+0000 7fd173fff700 1 -- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 msgr2=0x7fd15406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:31.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.334+0000 7fd173fff700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 0x7fd15406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:31.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.343+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fd17415eb70 con 0x7fd174072760 2023-10-16T10:03:31.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.349+0000 7fd170ff9700 1 -- 172.21.15.181:0/4248161023 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd17415eb70 con 0x7fd174072760 2023-10-16T10:03:31.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.351+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 msgr2=0x7fd15406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:31.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 0x7fd15406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 msgr2=0x7fd1740c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd1740c7660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd1740b66b0 tx=0x7fd164004750 comp rx=0 tx=0).stop 2023-10-16T10:03:31.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 shutdown_connections 2023-10-16T10:03:31.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fd15406b070 0x7fd15406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd17415da70 0x7fd1740b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd17406ade0 0x7fd174154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 --2- 172.21.15.181:0/4248161023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd174072760 0x7fd1740c7660 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 >> 172.21.15.181:0/4248161023 conn(0x7fd1740cd010 msgr2=0x7fd17415bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:31.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 shutdown_connections 2023-10-16T10:03:31.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.352+0000 7fd17a76b700 1 -- 172.21.15.181:0/4248161023 wait complete. 2023-10-16T10:03:31.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:31.366 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/minutes) 2023-10-16T10:03:31.391 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/minutes 1m 2023-10-16T10:03:31.453 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:31.452+0000 7fd7540ed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:03:31.485 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:31.483+0000 7eff9e42d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:03:31.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.556+0000 7fe40ddb6700 1 Processor -- start 2023-10-16T10:03:31.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.556+0000 7fe40ddb6700 1 -- start start 2023-10-16T10:03:31.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe40ddb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe40815f150 0x7fe4080cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe40ddb6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4080ce4f0 0x7fe4080b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe40ddb6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4080b2ae0 con 0x7fe4080b14a0 2023-10-16T10:03:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe40ddb6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4080ce970 con 0x7fe4080ce4f0 2023-10-16T10:03:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe40ddb6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080b2dc0 con 0x7fe40815f150 2023-10-16T10:03:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4080ce4f0 0x7fe4080b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4080ce4f0 0x7fe4080b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:47926/0 (socket says 172.21.15.181:47926) 2023-10-16T10:03:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 -- 172.21.15.181:0/2418458031 learned_addr learned my addr 172.21.15.181:0/2418458031 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe4077fe700 1 --2- 172.21.15.181:0/2418458031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe40815f150 0x7fe4080cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 -- 172.21.15.181:0/2418458031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe40815f150 msgr2=0x7fe4080cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 --2- 172.21.15.181:0/2418458031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe40815f150 0x7fe4080cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 -- 172.21.15.181:0/2418458031 >> v1:172.21.15.190:6789/0 conn(0x7fe4080b14a0 legacy=0x7fe4080b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 -- 172.21.15.181:0/2418458031 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4080cf9d0 con 0x7fe4080ce4f0 2023-10-16T10:03:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.557+0000 7fe406ffd700 1 --2- 172.21.15.181:0/2418458031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4080ce4f0 0x7fe4080b0f60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0098f0 tx=0x7fe3fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7c4eae0c327b47 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe4067fc700 1 -- 172.21.15.181:0/2418458031 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc011830 con 0x7fe4080ce4f0 2023-10-16T10:03:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe4067fc700 1 -- 172.21.15.181:0/2418458031 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc011990 con 0x7fe4080ce4f0 2023-10-16T10:03:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe4067fc700 1 -- 172.21.15.181:0/2418458031 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc011d20 con 0x7fe4080ce4f0 2023-10-16T10:03:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 -- 172.21.15.181:0/2418458031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4080ce4f0 msgr2=0x7fe4080b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 --2- 172.21.15.181:0/2418458031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4080ce4f0 0x7fe4080b0f60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0098f0 tx=0x7fe3fc009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 -- 172.21.15.181:0/2418458031 shutdown_connections 2023-10-16T10:03:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 --2- 172.21.15.181:0/2418458031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe40815f150 0x7fe4080cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 --2- 172.21.15.181:0/2418458031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4080ce4f0 0x7fe4080b0f60 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 -- 172.21.15.181:0/2418458031 >> 172.21.15.181:0/2418458031 conn(0x7fe40809a930 msgr2=0x7fe4080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 -- 172.21.15.181:0/2418458031 shutdown_connections 2023-10-16T10:03:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.558+0000 7fe40ddb6700 1 -- 172.21.15.181:0/2418458031 wait complete. 2023-10-16T10:03:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.560+0000 7fe40ddb6700 1 Processor -- start 2023-10-16T10:03:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.560+0000 7fe40ddb6700 1 -- start start 2023-10-16T10:03:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe40ddb6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4080b14a0 0x7fe408151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe40ddb6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4080ce4f0 0x7fe40815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe40ddb6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40815f150 0x7fe40815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe40ddb6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080c51a0 con 0x7fe40815f150 2023-10-16T10:03:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe40ddb6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4081e5f90 con 0x7fe4080b14a0 2023-10-16T10:03:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe40ddb6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080ca330 con 0x7fe4080ce4f0 2023-10-16T10:03:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4080b14a0 0x7fe408151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe407fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40815f150 0x7fe40815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4080b14a0 0x7fe408151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:52980/0 (socket says 172.21.15.181:52980) 2023-10-16T10:03:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 -- 172.21.15.181:0/371230002 learned_addr learned my addr 172.21.15.181:0/371230002 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4080ce4f0 0x7fe40815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4080ce4f0 msgr2=0x7fe40815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4080ce4f0 0x7fe40815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40815f150 msgr2=0x7fe40815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40815f150 0x7fe40815da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe4077fe700 1 -- 172.21.15.181:0/371230002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc009550 con 0x7fe4080b14a0 2023-10-16T10:03:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.561+0000 7fe407fff700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40815f150 0x7fe40815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.562+0000 7fe4077fe700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4080b14a0 0x7fe408151c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800efc0 tx=0x7fe3f800c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.562+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800cb50 con 0x7fe4080b14a0 2023-10-16T10:03:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.562+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f800f040 con 0x7fe4080b14a0 2023-10-16T10:03:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.562+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4080ca610 con 0x7fe4080b14a0 2023-10-16T10:03:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.562+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8014480 con 0x7fe4080b14a0 2023-10-16T10:03:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.562+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4080cabd0 con 0x7fe4080b14a0 2023-10-16T10:03:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.563+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4080c5410 con 0x7fe4080b14a0 2023-10-16T10:03:31.575 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:31.568+0000 7fd7540ed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:03:31.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.564+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 76) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fe3f80145e0 con 0x7fe4080b14a0 2023-10-16T10:03:31.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.564+0000 7fe3effff700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 0x7fe3f406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:31.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.564+0000 7fe406ffd700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 msgr2=0x7fe3f406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.564+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fe3f808a480 con 0x7fe4080b14a0 2023-10-16T10:03:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.564+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 0x7fe3f406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.569+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe3f80551d0 con 0x7fe4080b14a0 2023-10-16T10:03:31.607 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:31.605+0000 7eff9e42d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:03:31.643 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:31.642+0000 7fd7540ed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:03:31.682 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:31.681+0000 7eff9e42d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:03:31.723 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:31.722+0000 7fd7540ed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:03:31.763 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:31.762+0000 7eff9e42d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:03:31.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.765+0000 7fe406ffd700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 msgr2=0x7fe3f406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:31.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.765+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 0x7fe3f406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:31.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:31.794+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 --> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4080cff90 con 0x7fe3f406b070 2023-10-16T10:03:32.090 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:32.089+0000 7fd7540ed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:03:32.133 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:32.132+0000 7eff9e42d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:03:32.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:32.166+0000 7fe406ffd700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 msgr2=0x7fe3f406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:32.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:32.166+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 0x7fe3f406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:03:32.660 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:32.659+0000 7fd7540ed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:03:32.702 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:32.700+0000 7eff9e42d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:03:32.729 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:32.728+0000 7fd7540ed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:03:32.771 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:32.769+0000 7eff9e42d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:03:32.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:32.967+0000 7fe406ffd700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 msgr2=0x7fe3f406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:32.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:32.967+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 0x7fe3f406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:03:33.358 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:33.357+0000 7fd7540ed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:33.401 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:33.400+0000 7eff9e42d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:33.435 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:33.434+0000 7fd7540ed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:03:33.479 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:33.477+0000 7eff9e42d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:03:33.579 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:33.577+0000 7fd7540ed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:33.622 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:33.621+0000 7eff9e42d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:33.805 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:33.804+0000 7fd7540ed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:03:33.851 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:33.849+0000 7eff9e42d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:03:33.878 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:33.877+0000 7fd7540ed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:03:33.925 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:33.923+0000 7eff9e42d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:03:34.142 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:34.141+0000 7fd7540ed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:03:34.192 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:34.190+0000 7eff9e42d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:03:34.365 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:34.363+0000 7fd7540ed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:03:34.415 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:34.413+0000 7eff9e42d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:03:34.513 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:34.512+0000 7fd7540ed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:03:34.564 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:34.563+0000 7eff9e42d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:03:34.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:34.568+0000 7fe406ffd700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 msgr2=0x7fe3f406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3280351398 2023-10-16T10:03:34.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:34.568+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 0x7fe3f406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:03:34.626 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:34.625+0000 7fd7540ed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:03:34.678 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:34.677+0000 7eff9e42d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:03:34.781 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:34.780+0000 7fd7540ed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:03:34.834 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:34.832+0000 7eff9e42d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:03:35.040 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:35.039+0000 7fd7540ed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:35.096 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.095+0000 7eff9e42d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:35.199 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:35.197+0000 7fd7540ed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:03:35.259 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.258+0000 7eff9e42d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:03:35.720 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:35.719+0000 7fd7540ed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:03:35.789 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.788+0000 7eff9e42d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:03:35.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:35.789+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 77) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fe3f80887d0 con 0x7fe4080b14a0 2023-10-16T10:03:35.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:35.811+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mgrmap(e 78) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fe3f808a740 con 0x7fe4080b14a0 2023-10-16T10:03:35.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:35.811+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 msgr2=0x7fe3f406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:35.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:35.811+0000 7fe3effff700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3280351398,v1:172.21.15.181:6833/3280351398] conn(0x7fe3f406b070 0x7fe3f406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:35.977 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff10c07700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.978 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff10c07700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.978 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff10c07700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.978 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff10c07700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.978 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff10c07700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.979 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff13c0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.979 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff13c0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.979 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff13c0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.979 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff13c0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:35.980 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:35.976+0000 7eff13c0d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:36.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.814+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mon.1 v2:172.21.15.190:3300/0 9 ==== mgrmap(e 79) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fe3f80439d0 con 0x7fe4080b14a0 2023-10-16T10:03:36.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.814+0000 7fe3effff700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe3f40279c0 0x7fe3f4027da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:36.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.814+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3f4071070 con 0x7fe3f40279c0 2023-10-16T10:03:36.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.815+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe3f40279c0 0x7fe3f4027da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:36.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.819+0000 7fe406ffd700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe3f40279c0 0x7fe3f4027da0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc000c00 tx=0x7fe3fc008030 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:36.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.887+0000 7fe3effff700 1 -- 172.21.15.181:0/371230002 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7fe3f4071070 con 0x7fe3f40279c0 2023-10-16T10:03:36.890 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1/minutes 2023-10-16T10:03:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe3f40279c0 msgr2=0x7fe3f4027da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe3f40279c0 0x7fe3f4027da0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc000c00 tx=0x7fe3fc008030 comp rx=0 tx=0).stop 2023-10-16T10:03:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4080b14a0 msgr2=0x7fe408151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:36.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 --2- 172.21.15.181:0/371230002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4080b14a0 0x7fe408151c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800efc0 tx=0x7fe3f800c0c0 comp rx=0 tx=0).stop 2023-10-16T10:03:36.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe407fff700 1 -- 172.21.15.181:0/371230002 reap_dead start 2023-10-16T10:03:36.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 shutdown_connections 2023-10-16T10:03:36.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 >> 172.21.15.181:0/371230002 conn(0x7fe40809a930 msgr2=0x7fe4080b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:36.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 shutdown_connections 2023-10-16T10:03:36.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:36.890+0000 7fe40ddb6700 1 -- 172.21.15.181:0/371230002 wait complete. 2023-10-16T10:03:36.905 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/minutes 2023-10-16T10:03:36.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:36.906 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/hourly) 2023-10-16T10:03:36.926 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/hourly 1h 2023-10-16T10:03:37.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.088+0000 7f27666ae700 1 Processor -- start 2023-10-16T10:03:37.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.088+0000 7f27666ae700 1 -- start start 2023-10-16T10:03:37.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f27666ae700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f27600b2010 0x7f27600cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f27666ae700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2760071be0 0x7f27600b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f27666ae700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27600ce1f0 con 0x7f276006f3b0 2023-10-16T10:03:37.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f27666ae700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27600b2930 con 0x7f27600b2010 2023-10-16T10:03:37.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f27666ae700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600ccda0 con 0x7f2760071be0 2023-10-16T10:03:37.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f275ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f27600b2010 0x7f27600cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2760071be0 0x7f27600b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f2764c4b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f276006f3b0 0x7f276006f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:43154/0 (socket says 172.21.15.181:43154) 2023-10-16T10:03:37.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f275ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f27600b2010 0x7f27600cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:53180/0 (socket says 172.21.15.181:53180) 2023-10-16T10:03:37.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2760071be0 0x7f27600b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:45156/0 (socket says 172.21.15.181:45156) 2023-10-16T10:03:37.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.089+0000 7f275ffff700 1 -- 172.21.15.181:0/2763365682 learned_addr learned my addr 172.21.15.181:0/2763365682 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:37.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275f7fe700 1 -- 172.21.15.181:0/2763365682 >> v2:172.21.15.190:3300/0 conn(0x7f27600b2010 msgr2=0x7f27600cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:37.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275f7fe700 1 --2- 172.21.15.181:0/2763365682 >> v2:172.21.15.190:3300/0 conn(0x7f27600b2010 0x7f27600cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275f7fe700 1 -- 172.21.15.181:0/2763365682 >> v1:172.21.15.190:6789/0 conn(0x7f276006f3b0 legacy=0x7f276006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275f7fe700 1 -- 172.21.15.181:0/2763365682 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600727e0 con 0x7f2760071be0 2023-10-16T10:03:37.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275f7fe700 1 --2- 172.21.15.181:0/2763365682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2760071be0 0x7f27600b23f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2750009840 tx=0x7f2750009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec199619575fb21 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:37.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275effd700 1 -- 172.21.15.181:0/2763365682 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750011830 con 0x7f2760071be0 2023-10-16T10:03:37.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275effd700 1 -- 172.21.15.181:0/2763365682 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750011990 con 0x7f2760071be0 2023-10-16T10:03:37.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.090+0000 7f275effd700 1 -- 172.21.15.181:0/2763365682 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750011d20 con 0x7f2760071be0 2023-10-16T10:03:37.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 -- 172.21.15.181:0/2763365682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2760071be0 msgr2=0x7f27600b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 --2- 172.21.15.181:0/2763365682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2760071be0 0x7f27600b23f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2750009840 tx=0x7f2750009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:37.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 -- 172.21.15.181:0/2763365682 shutdown_connections 2023-10-16T10:03:37.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 --2- 172.21.15.181:0/2763365682 >> v2:172.21.15.190:3300/0 conn(0x7f27600b2010 0x7f27600cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 --2- 172.21.15.181:0/2763365682 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2760071be0 0x7f27600b23f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 -- 172.21.15.181:0/2763365682 >> 172.21.15.181:0/2763365682 conn(0x7f27600cd010 msgr2=0x7f27600cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:37.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 -- 172.21.15.181:0/2763365682 shutdown_connections 2023-10-16T10:03:37.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.091+0000 7f27666ae700 1 -- 172.21.15.181:0/2763365682 wait complete. 2023-10-16T10:03:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.092+0000 7f27666ae700 1 Processor -- start 2023-10-16T10:03:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.092+0000 7f27666ae700 1 -- start start 2023-10-16T10:03:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f27666ae700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f276006f3b0 0x7f27600c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f27666ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2760071be0 0x7f276014fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f27666ae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27600b2010 0x7f276006f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f27666ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601535a0 con 0x7f2760071be0 2023-10-16T10:03:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f27666ae700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601f7940 con 0x7f27600b2010 2023-10-16T10:03:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f27666ae700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27601f6a70 con 0x7f276006f3b0 2023-10-16T10:03:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2760071be0 0x7f276014fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2760071be0 0x7f276014fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48136/0 (socket says 172.21.15.181:48136) 2023-10-16T10:03:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 -- 172.21.15.181:0/3152262137 learned_addr learned my addr 172.21.15.181:0/3152262137 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275ffff700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f276006f3b0 0x7f27600c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f2764c4b700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27600b2010 0x7f276006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 -- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f276006f3b0 msgr2=0x7f27600c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f276006f3b0 0x7f27600c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 -- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27600b2010 msgr2=0x7f276006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27600b2010 0x7f276006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 -- 172.21.15.181:0/3152262137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500094a0 con 0x7f2760071be0 2023-10-16T10:03:37.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275ffff700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f276006f3b0 0x7f27600c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:37.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f2764c4b700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27600b2010 0x7f276006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:37.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.093+0000 7f275f7fe700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2760071be0 0x7f276014fab0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f2750004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:37.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.094+0000 7f275cff9700 1 -- 172.21.15.181:0/3152262137 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500038a0 con 0x7f2760071be0 2023-10-16T10:03:37.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.094+0000 7f275cff9700 1 -- 172.21.15.181:0/3152262137 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275001e070 con 0x7f2760071be0 2023-10-16T10:03:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.094+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276014fff0 con 0x7f2760071be0 2023-10-16T10:03:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.094+0000 7f275cff9700 1 -- 172.21.15.181:0/3152262137 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750003c40 con 0x7f2760071be0 2023-10-16T10:03:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.094+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27601f6fa0 con 0x7f2760071be0 2023-10-16T10:03:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.095+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27600b6510 con 0x7f2760071be0 2023-10-16T10:03:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.095+0000 7f275cff9700 1 -- 172.21.15.181:0/3152262137 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 79) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f275001a420 con 0x7f2760071be0 2023-10-16T10:03:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.096+0000 7f275cff9700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f274c06ada0 0x7f274c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.096+0000 7f275cff9700 1 -- 172.21.15.181:0/3152262137 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f275008a610 con 0x7f2760071be0 2023-10-16T10:03:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.096+0000 7f275ffff700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f274c06ada0 0x7f274c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.096+0000 7f275ffff700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f274c06ada0 0x7f274c06d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2754009b50 tx=0x7f2754008040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.100+0000 7f275cff9700 1 -- 172.21.15.181:0/3152262137 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f27500556e0 con 0x7f2760071be0 2023-10-16T10:03:37.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.321+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/hourly", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f2760072da0 con 0x7f274c06ada0 2023-10-16T10:03:37.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.385+0000 7f275cff9700 1 -- 172.21.15.181:0/3152262137 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f27540097c0 con 0x7f274c06ada0 2023-10-16T10:03:37.388 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1/hourly 2023-10-16T10:03:37.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.387+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f274c06ada0 msgr2=0x7f274c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.387+0000 7f27666ae700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f274c06ada0 0x7f274c06d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2754009b50 tx=0x7f2754008040 comp rx=0 tx=0).stop 2023-10-16T10:03:37.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2760071be0 msgr2=0x7f276014fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2760071be0 0x7f276014fab0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f2750004750 comp rx=0 tx=0).stop 2023-10-16T10:03:37.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 shutdown_connections 2023-10-16T10:03:37.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f274c06ada0 0x7f274c06d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f276006f3b0 0x7f27600c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27600b2010 0x7f276006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 --2- 172.21.15.181:0/3152262137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2760071be0 0x7f276014fab0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 >> 172.21.15.181:0/3152262137 conn(0x7f27600cd010 msgr2=0x7f27600b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:37.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 shutdown_connections 2023-10-16T10:03:37.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.388+0000 7f27666ae700 1 -- 172.21.15.181:0/3152262137 wait complete. 2023-10-16T10:03:37.402 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/hourly 2023-10-16T10:03:37.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:37.403 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/daily) 2023-10-16T10:03:37.421 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/daily 1d 2023-10-16T10:03:37.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.584+0000 7ff2c8b99700 1 Processor -- start 2023-10-16T10:03:37.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.584+0000 7ff2c8b99700 1 -- start start 2023-10-16T10:03:37.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c8b99700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2c406bef0 0x7ff2c40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c8b99700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2c40cb0e0 0x7ff2c415db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c8b99700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c406e4b0 con 0x7ff2c4159e10 2023-10-16T10:03:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c8b99700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c40cecb0 con 0x7ff2c40cb0e0 2023-10-16T10:03:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c8b99700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c406ed50 con 0x7ff2c406bef0 2023-10-16T10:03:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2c40cb0e0 0x7ff2c415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff2c4159e10 0x7ff2c40ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59222/0 (socket says 172.21.15.181:59222) 2023-10-16T10:03:37.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c259c700 1 -- 172.21.15.181:0/2338339163 learned_addr learned my addr 172.21.15.181:0/2338339163 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:37.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c1d9b700 1 --2- 172.21.15.181:0/2338339163 >> v2:172.21.15.190:3300/0 conn(0x7ff2c406bef0 0x7ff2c40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c159a700 1 -- 172.21.15.181:0/2338339163 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304301912 0 0) 0x7ff2c406e4b0 con 0x7ff2c4159e10 2023-10-16T10:03:37.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c159a700 1 -- 172.21.15.181:0/2338339163 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ac0035b0 con 0x7ff2c4159e10 2023-10-16T10:03:37.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c1d9b700 1 -- 172.21.15.181:0/2338339163 >> v2:172.21.15.181:3300/0 conn(0x7ff2c40cb0e0 msgr2=0x7ff2c415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:37.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c1d9b700 1 --2- 172.21.15.181:0/2338339163 >> v2:172.21.15.181:3300/0 conn(0x7ff2c40cb0e0 0x7ff2c415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c1d9b700 1 -- 172.21.15.181:0/2338339163 >> v1:172.21.15.181:6789/0 conn(0x7ff2c4159e10 legacy=0x7ff2c40ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c1d9b700 1 -- 172.21.15.181:0/2338339163 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c40ceea0 con 0x7ff2c406bef0 2023-10-16T10:03:37.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.585+0000 7ff2c2d9d700 1 --2- 172.21.15.181:0/2338339163 >> v2:172.21.15.181:3300/0 conn(0x7ff2c40cb0e0 0x7ff2c415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:37.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c1d9b700 1 --2- 172.21.15.181:0/2338339163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c406bef0 0x7ff2c40caba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40cc700 tx=0x7ff2b8009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f58123340c71632 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:37.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c159a700 1 -- 172.21.15.181:0/2338339163 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8011830 con 0x7ff2c406bef0 2023-10-16T10:03:37.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c159a700 1 -- 172.21.15.181:0/2338339163 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b8011990 con 0x7ff2c406bef0 2023-10-16T10:03:37.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c159a700 1 -- 172.21.15.181:0/2338339163 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8011d20 con 0x7ff2c406bef0 2023-10-16T10:03:37.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2338339163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c406bef0 msgr2=0x7ff2c40caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2338339163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c406bef0 0x7ff2c40caba0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40cc700 tx=0x7ff2b8009b20 comp rx=0 tx=0).stop 2023-10-16T10:03:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2338339163 shutdown_connections 2023-10-16T10:03:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2338339163 >> v2:172.21.15.181:3300/0 conn(0x7ff2c40cb0e0 0x7ff2c415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2338339163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c406bef0 0x7ff2c40caba0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2338339163 >> 172.21.15.181:0/2338339163 conn(0x7ff2c40cc730 msgr2=0x7ff2c40ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:37.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.586+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2338339163 shutdown_connections 2023-10-16T10:03:37.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.587+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2338339163 wait complete. 2023-10-16T10:03:37.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.588+0000 7ff2c8b99700 1 Processor -- start 2023-10-16T10:03:37.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.588+0000 7ff2c8b99700 1 -- start start 2023-10-16T10:03:37.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c8b99700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c406bef0 0x7ff2c4146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c8b99700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2c40cb0e0 0x7ff2c4147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c8b99700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c4159e10 0x7ff2c40c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c8b99700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41f1070 con 0x7ff2c406bef0 2023-10-16T10:03:37.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c8b99700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4149c70 con 0x7ff2c4159e10 2023-10-16T10:03:37.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c8b99700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4145fc0 con 0x7ff2c40cb0e0 2023-10-16T10:03:37.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c406bef0 0x7ff2c4146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c406bef0 0x7ff2c4146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48150/0 (socket says 172.21.15.181:48150) 2023-10-16T10:03:37.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 -- 172.21.15.181:0/2737662411 learned_addr learned my addr 172.21.15.181:0/2737662411 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:37.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c1d9b700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2c40cb0e0 0x7ff2c4147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c2d9d700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c4159e10 0x7ff2c40c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 -- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2c40cb0e0 msgr2=0x7ff2c4147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2c40cb0e0 0x7ff2c4147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 -- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c4159e10 msgr2=0x7ff2c40c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c4159e10 0x7ff2c40c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 -- 172.21.15.181:0/2737662411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc009a70 con 0x7ff2c406bef0 2023-10-16T10:03:37.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.589+0000 7ff2c259c700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c406bef0 0x7ff2c4146580 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4001eb0 tx=0x7ff2b400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:37.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.590+0000 7ff2b2ffd700 1 -- 172.21.15.181:0/2737662411 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400cea0 con 0x7ff2c406bef0 2023-10-16T10:03:37.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.590+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b80094a0 con 0x7ff2c406bef0 2023-10-16T10:03:37.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.590+0000 7ff2b2ffd700 1 -- 172.21.15.181:0/2737662411 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b4005d70 con 0x7ff2c406bef0 2023-10-16T10:03:37.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.590+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c40c8220 con 0x7ff2c406bef0 2023-10-16T10:03:37.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.590+0000 7ff2b2ffd700 1 -- 172.21.15.181:0/2737662411 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40072d0 con 0x7ff2c406bef0 2023-10-16T10:03:37.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.591+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c40cd5d0 con 0x7ff2c406bef0 2023-10-16T10:03:37.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.591+0000 7ff2b2ffd700 1 -- 172.21.15.181:0/2737662411 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 79) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff2b4007430 con 0x7ff2c406bef0 2023-10-16T10:03:37.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.592+0000 7ff2b2ffd700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ff2a406ada0 0x7ff2a406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:37.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.592+0000 7ff2c1d9b700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ff2a406ada0 0x7ff2a406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:37.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.592+0000 7ff2b2ffd700 1 -- 172.21.15.181:0/2737662411 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7ff2b4088890 con 0x7ff2c406bef0 2023-10-16T10:03:37.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.592+0000 7ff2c1d9b700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ff2a406ada0 0x7ff2a406d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80090b0 tx=0x7ff2b8008030 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:37.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.596+0000 7ff2b2ffd700 1 -- 172.21.15.181:0/2737662411 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff2b4053990 con 0x7ff2c406bef0 2023-10-16T10:03:37.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.816+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/daily", "snap_schedule": "1d", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2c4146130 con 0x7ff2a406ada0 2023-10-16T10:03:37.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.887+0000 7ff2b2ffd700 1 -- 172.21.15.181:0/2737662411 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff2c4146130 con 0x7ff2a406ada0 2023-10-16T10:03:37.889 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1/daily 2023-10-16T10:03:37.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ff2a406ada0 msgr2=0x7ff2a406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ff2a406ada0 0x7ff2a406d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80090b0 tx=0x7ff2b8008030 comp rx=0 tx=0).stop 2023-10-16T10:03:37.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c406bef0 msgr2=0x7ff2c4146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:37.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c406bef0 0x7ff2c4146580 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4001eb0 tx=0x7ff2b400c7b0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 shutdown_connections 2023-10-16T10:03:37.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ff2a406ada0 0x7ff2a406d260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2c4159e10 0x7ff2c40c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c406bef0 0x7ff2c4146580 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 --2- 172.21.15.181:0/2737662411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2c40cb0e0 0x7ff2c4147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:37.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 >> 172.21.15.181:0/2737662411 conn(0x7ff2c40cc730 msgr2=0x7ff2c40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:37.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 shutdown_connections 2023-10-16T10:03:37.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:37.889+0000 7ff2c8b99700 1 -- 172.21.15.181:0/2737662411 wait complete. 2023-10-16T10:03:37.903 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/daily 2023-10-16T10:03:37.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:37.904 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/weekly) 2023-10-16T10:03:37.922 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/weekly 1w 2023-10-16T10:03:38.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.085+0000 7fd1c2974700 1 Processor -- start 2023-10-16T10:03:38.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.085+0000 7fd1c2974700 1 -- start start 2023-10-16T10:03:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1c2974700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 0x7fd1bc06f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1c2974700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1bc0b5fd0 0x7fd1bc15f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1c2974700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1bc070590 con 0x7fd1bc0cb2b0 2023-10-16T10:03:38.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1c2974700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1bc156330 con 0x7fd1bc0b5fd0 2023-10-16T10:03:38.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1c2974700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc06fcf0 con 0x7fd1bc0cb690 2023-10-16T10:03:38.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1bbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd1bc0cb2b0 0x7fd1bc0b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59254/0 (socket says 172.21.15.181:59254) 2023-10-16T10:03:38.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1bbfff700 1 -- 172.21.15.181:0/1318791042 learned_addr learned my addr 172.21.15.181:0/1318791042 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:38.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1bb7fe700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 0x7fd1bc06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/1318791042 >> v2:172.21.15.190:3300/0 conn(0x7fd1bc0b5fd0 0x7fd1bc15f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590306165 0 0) 0x7fd1bc070590 con 0x7fd1bc0cb2b0 2023-10-16T10:03:38.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.086+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a80035b0 con 0x7fd1bc0cb2b0 2023-10-16T10:03:38.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3407020295 0 0) 0x7fd1a80035b0 con 0x7fd1bc0cb2b0 2023-10-16T10:03:38.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 msgr2=0x7fd1bc06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:38.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 0x7fd1bc06f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 >> v2:172.21.15.190:3300/0 conn(0x7fd1bc0b5fd0 msgr2=0x7fd1bc15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:38.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 --2- 172.21.15.181:0/1318791042 >> v2:172.21.15.190:3300/0 conn(0x7fd1bc0b5fd0 0x7fd1bc15f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc06ffd0 con 0x7fd1bc0cb2b0 2023-10-16T10:03:38.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd1a4001670 con 0x7fd1bc0cb2b0 2023-10-16T10:03:38.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 >> v1:172.21.15.181:6789/0 conn(0x7fd1bc0cb2b0 legacy=0x7fd1bc0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8003a00 0x7fd1a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 0x7fd1a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a8004d90 0x7fd1a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc156330 con 0x7fd1a8004d90 2023-10-16T10:03:38.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc06fcf0 con 0x7fd1a8003a00 2023-10-16T10:03:38.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a800ba40 con 0x7fd1bc0cb690 2023-10-16T10:03:38.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a8004d90 0x7fd1a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1bbfff700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8003a00 0x7fd1a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1bb7fe700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 0x7fd1a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1c0f11700 1 -- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 msgr2=0x7fd1a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc0cb690 0x7fd1a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1c0f11700 1 -- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8003a00 msgr2=0x7fd1a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8003a00 0x7fd1a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1c0f11700 1 -- 172.21.15.181:0/1318791042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80035b0 con 0x7fd1a8004d90 2023-10-16T10:03:38.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a8004d90 0x7fd1a8005210 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009fc0 tx=0x7fd1a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578b437091c5706a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:38.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400ec30 con 0x7fd1a8004d90 2023-10-16T10:03:38.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400f040 con 0x7fd1a8004d90 2023-10-16T10:03:38.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.087+0000 7fd1baffd700 1 -- 172.21.15.181:0/1318791042 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4014480 con 0x7fd1a8004d90 2023-10-16T10:03:38.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.088+0000 7fd1c2974700 1 -- 172.21.15.181:0/1318791042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a8004d90 msgr2=0x7fd1a8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.088+0000 7fd1c2974700 1 --2- 172.21.15.181:0/1318791042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a8004d90 0x7fd1a8005210 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4009fc0 tx=0x7fd1a8005880 comp rx=0 tx=0).stop 2023-10-16T10:03:38.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.088+0000 7fd1c0f11700 1 -- 172.21.15.181:0/1318791042 reap_dead start 2023-10-16T10:03:38.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.088+0000 7fd1c2974700 1 -- 172.21.15.181:0/1318791042 shutdown_connections 2023-10-16T10:03:38.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.088+0000 7fd1c2974700 1 -- 172.21.15.181:0/1318791042 >> 172.21.15.181:0/1318791042 conn(0x7fd1bc0cd010 msgr2=0x7fd1bc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:38.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.088+0000 7fd1c2974700 1 -- 172.21.15.181:0/1318791042 shutdown_connections 2023-10-16T10:03:38.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.088+0000 7fd1c2974700 1 -- 172.21.15.181:0/1318791042 wait complete. 2023-10-16T10:03:38.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.090+0000 7fd1c2974700 1 Processor -- start 2023-10-16T10:03:38.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.090+0000 7fd1c2974700 1 -- start start 2023-10-16T10:03:38.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8004d90 0x7fd1bc1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc15b1c0 0x7fd1bc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1bc0c5000 0x7fd1bc0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc15c960 con 0x7fd1bc0c5000 2023-10-16T10:03:38.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc15b640 con 0x7fd1a8004d90 2023-10-16T10:03:38.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc15c700 con 0x7fd1bc15b1c0 2023-10-16T10:03:38.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1bc0c5000 0x7fd1bc0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1bc0c5000 0x7fd1bc0c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48192/0 (socket says 172.21.15.181:48192) 2023-10-16T10:03:38.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 -- 172.21.15.181:0/3478620905 learned_addr learned my addr 172.21.15.181:0/3478620905 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:38.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1bbfff700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8004d90 0x7fd1bc1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1bb7fe700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc15b1c0 0x7fd1bc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 -- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc15b1c0 msgr2=0x7fd1bc15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc15b1c0 0x7fd1bc15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 -- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8004d90 msgr2=0x7fd1bc1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8004d90 0x7fd1bc1f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 -- 172.21.15.181:0/3478620905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc0c4240 con 0x7fd1bc0c5000 2023-10-16T10:03:38.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1bb7fe700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc15b1c0 0x7fd1bc15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:38.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c0f11700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1bc0c5000 0x7fd1bc0c58c0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4016000 tx=0x7fd1b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:38.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1b8ff9700 1 -- 172.21.15.181:0/3478620905 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007000 con 0x7fd1bc0c5000 2023-10-16T10:03:38.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1b8ff9700 1 -- 172.21.15.181:0/3478620905 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4004450 con 0x7fd1bc0c5000 2023-10-16T10:03:38.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc0c4520 con 0x7fd1bc0c5000 2023-10-16T10:03:38.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1b8ff9700 1 -- 172.21.15.181:0/3478620905 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4004800 con 0x7fd1bc0c5000 2023-10-16T10:03:38.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.091+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1bc0c4ae0 con 0x7fd1bc0c5000 2023-10-16T10:03:38.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.092+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1bc0af4e0 con 0x7fd1bc0c5000 2023-10-16T10:03:38.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.093+0000 7fd1b8ff9700 1 -- 172.21.15.181:0/3478620905 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 79) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd1b4014480 con 0x7fd1bc0c5000 2023-10-16T10:03:38.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.093+0000 7fd1b8ff9700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fd19c06ada0 0x7fd19c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.093+0000 7fd1b8ff9700 1 -- 172.21.15.181:0/3478620905 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7fd1b408a840 con 0x7fd1bc0c5000 2023-10-16T10:03:38.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.093+0000 7fd1bbfff700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fd19c06ada0 0x7fd19c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.094+0000 7fd1bbfff700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fd19c06ada0 0x7fd19c06d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4007980 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:38.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.098+0000 7fd1b8ff9700 1 -- 172.21.15.181:0/3478620905 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd1b40559b0 con 0x7fd1bc0c5000 2023-10-16T10:03:38.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.313+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/weekly", "snap_schedule": "1w", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1bc0b2140 con 0x7fd19c06ada0 2023-10-16T10:03:38.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.378+0000 7fd1b8ff9700 1 -- 172.21.15.181:0/3478620905 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7fd1a4007e00 con 0x7fd19c06ada0 2023-10-16T10:03:38.380 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1/weekly 2023-10-16T10:03:38.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fd19c06ada0 msgr2=0x7fd19c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fd19c06ada0 0x7fd19c06d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4007980 comp rx=0 tx=0).stop 2023-10-16T10:03:38.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1bc0c5000 msgr2=0x7fd1bc0c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1bc0c5000 0x7fd1bc0c58c0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4016000 tx=0x7fd1b4009750 comp rx=0 tx=0).stop 2023-10-16T10:03:38.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 shutdown_connections 2023-10-16T10:03:38.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fd19c06ada0 0x7fd19c06d260 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1bc0c5000 0x7fd1bc0c58c0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8004d90 0x7fd1bc1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.380+0000 7fd1c2974700 1 --2- 172.21.15.181:0/3478620905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1bc15b1c0 0x7fd1bc15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.381+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 >> 172.21.15.181:0/3478620905 conn(0x7fd1bc0cd010 msgr2=0x7fd1bc15e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:38.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.381+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 shutdown_connections 2023-10-16T10:03:38.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.381+0000 7fd1c2974700 1 -- 172.21.15.181:0/3478620905 wait complete. 2023-10-16T10:03:38.394 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/weekly 2023-10-16T10:03:38.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:38.395 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/monthly) 2023-10-16T10:03:38.414 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/monthly 1M 2023-10-16T10:03:38.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.576+0000 7ffb77a61700 1 Processor -- start 2023-10-16T10:03:38.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.576+0000 7ffb77a61700 1 -- start start 2023-10-16T10:03:38.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb77a61700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb7015d740 0x7ffb70161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb77a61700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb700ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb77a61700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb700d0cb0 con 0x7ffb700cf890 2023-10-16T10:03:38.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb77a61700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb701624b0 con 0x7ffb7015d740 2023-10-16T10:03:38.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb77a61700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700d00a0 con 0x7ffb700ce5e0 2023-10-16T10:03:38.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb7015d740 0x7ffb70161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb7015d740 0x7ffb70161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48196/0 (socket says 172.21.15.181:48196) 2023-10-16T10:03:38.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 -- 172.21.15.181:0/2994766497 learned_addr learned my addr 172.21.15.181:0/2994766497 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:38.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 -- 172.21.15.181:0/2994766497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 msgr2=0x7ffb700ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:38.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb75ffe700 1 --2- 172.21.15.181:0/2994766497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb700ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 --2- 172.21.15.181:0/2994766497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb700ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 -- 172.21.15.181:0/2994766497 >> v1:172.21.15.190:6789/0 conn(0x7ffb700cf890 legacy=0x7ffb700cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:38.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 -- 172.21.15.181:0/2994766497 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700d0380 con 0x7ffb7015d740 2023-10-16T10:03:38.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb75ffe700 1 --2- 172.21.15.181:0/2994766497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb700ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:38.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb757fd700 1 --2- 172.21.15.181:0/2994766497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7015d740 0x7ffb70161f70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d076e52419a30eba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:38.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb67fff700 1 -- 172.21.15.181:0/2994766497 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00b8a0 con 0x7ffb7015d740 2023-10-16T10:03:38.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb67fff700 1 -- 172.21.15.181:0/2994766497 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c00ba00 con 0x7ffb7015d740 2023-10-16T10:03:38.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.577+0000 7ffb67fff700 1 -- 172.21.15.181:0/2994766497 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00bd90 con 0x7ffb7015d740 2023-10-16T10:03:38.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 -- 172.21.15.181:0/2994766497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7015d740 msgr2=0x7ffb70161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 --2- 172.21.15.181:0/2994766497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7015d740 0x7ffb70161f70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:38.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 -- 172.21.15.181:0/2994766497 shutdown_connections 2023-10-16T10:03:38.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 --2- 172.21.15.181:0/2994766497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb700ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 --2- 172.21.15.181:0/2994766497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7015d740 0x7ffb70161f70 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 -- 172.21.15.181:0/2994766497 >> 172.21.15.181:0/2994766497 conn(0x7ffb7009a930 msgr2=0x7ffb700d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:38.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 -- 172.21.15.181:0/2994766497 shutdown_connections 2023-10-16T10:03:38.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.578+0000 7ffb77a61700 1 -- 172.21.15.181:0/2994766497 wait complete. 2023-10-16T10:03:38.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 Processor -- start 2023-10-16T10:03:38.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 -- start start 2023-10-16T10:03:38.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb7006f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb700cf890 0x7ffb700c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7015d740 0x7ffb700c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c87c0 con 0x7ffb700cf890 2023-10-16T10:03:38.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c8a60 con 0x7ffb7015d740 2023-10-16T10:03:38.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.580+0000 7ffb77a61700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700c8d40 con 0x7ffb700ce5e0 2023-10-16T10:03:38.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb700cf890 0x7ffb700c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb757fd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb7006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb700cf890 0x7ffb700c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48204/0 (socket says 172.21.15.181:48204) 2023-10-16T10:03:38.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 -- 172.21.15.181:0/1712214802 learned_addr learned my addr 172.21.15.181:0/1712214802 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:38.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb75ffe700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7015d740 0x7ffb700c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 -- 172.21.15.181:0/1712214802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 msgr2=0x7ffb7006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb7006f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 -- 172.21.15.181:0/1712214802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7015d740 msgr2=0x7ffb700c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7015d740 0x7ffb700c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 -- 172.21.15.181:0/1712214802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c009550 con 0x7ffb700cf890 2023-10-16T10:03:38.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb74ffc700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb700cf890 0x7ffb700c6ef0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c4e0 tx=0x7ffb6000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:38.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600052d0 con 0x7ffb700cf890 2023-10-16T10:03:38.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60005430 con 0x7ffb700cf890 2023-10-16T10:03:38.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600038c0 con 0x7ffb700cf890 2023-10-16T10:03:38.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.581+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb700c9020 con 0x7ffb700cf890 2023-10-16T10:03:38.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.582+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb70151aa0 con 0x7ffb700cf890 2023-10-16T10:03:38.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.583+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb700c9540 con 0x7ffb700cf890 2023-10-16T10:03:38.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.583+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 79) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ffb60003a20 con 0x7ffb700cf890 2023-10-16T10:03:38.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.583+0000 7ffb65ffb700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ffb5406ada0 0x7ffb5406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:38.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.583+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7ffb60088900 con 0x7ffb700cf890 2023-10-16T10:03:38.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.583+0000 7ffb757fd700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ffb5406ada0 0x7ffb5406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:38.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.584+0000 7ffb757fd700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ffb5406ada0 0x7ffb5406d260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c014040 tx=0x7ffb6c008030 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:38.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.588+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb700c9540 con 0x7ffb700cf890 2023-10-16T10:03:38.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.808+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/monthly", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb700d0940 con 0x7ffb5406ada0 2023-10-16T10:03:38.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.818+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 80) v1 ==== 89349+0+0 (secure 0 0 0) 0x7ffb60053da0 con 0x7ffb700cf890 2023-10-16T10:03:38.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.881+0000 7ffb65ffb700 1 -- 172.21.15.181:0/1712214802 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7ffb6c00bbf0 con 0x7ffb5406ada0 2023-10-16T10:03:38.884 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1/monthly 2023-10-16T10:03:38.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ffb5406ada0 msgr2=0x7ffb5406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ffb5406ada0 0x7ffb5406d260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c014040 tx=0x7ffb6c008030 comp rx=0 tx=0).stop 2023-10-16T10:03:38.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb700cf890 msgr2=0x7ffb700c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:38.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb700cf890 0x7ffb700c6ef0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c4e0 tx=0x7ffb6000c7f0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 shutdown_connections 2023-10-16T10:03:38.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7ffb5406ada0 0x7ffb5406d260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb700ce5e0 0x7ffb7006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7015d740 0x7ffb700c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 --2- 172.21.15.181:0/1712214802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb700cf890 0x7ffb700c6ef0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:38.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 >> 172.21.15.181:0/1712214802 conn(0x7ffb7009a930 msgr2=0x7ffb700cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:38.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 shutdown_connections 2023-10-16T10:03:38.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:38.884+0000 7ffb77a61700 1 -- 172.21.15.181:0/1712214802 wait complete. 2023-10-16T10:03:38.897 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/monthly 2023-10-16T10:03:38.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:38.898 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/yearly) 2023-10-16T10:03:38.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/yearly 1Y 2023-10-16T10:03:39.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.078+0000 7fe8dbfd4700 1 Processor -- start 2023-10-16T10:03:39.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.079+0000 7fe8dbfd4700 1 -- start start 2023-10-16T10:03:39.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.079+0000 7fe8dbfd4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8d40cc6e0 0x7fe8d4071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.079+0000 7fe8dbfd4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 0x7fe8d40ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.079+0000 7fe8dbfd4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d40ce1f0 con 0x7fe8d40b2010 2023-10-16T10:03:39.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.079+0000 7fe8dbfd4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d40ccb90 con 0x7fe8d40cc6e0 2023-10-16T10:03:39.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.079+0000 7fe8dbfd4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4072120 con 0x7fe8d40ce680 2023-10-16T10:03:39.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe8d40b2010 0x7fe8d40b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59288/0 (socket says 172.21.15.181:59288) 2023-10-16T10:03:39.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 -- 172.21.15.181:0/3785217036 learned_addr learned my addr 172.21.15.181:0/3785217036 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:39.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8da571700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 0x7fe8d40ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332932644 0 0) 0x7fe8d40ce1f0 con 0x7fe8d40b2010 2023-10-16T10:03:39.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d956f700 1 --2- 172.21.15.181:0/3785217036 >> v2:172.21.15.190:3300/0 conn(0x7fe8d40cc6e0 0x7fe8d4071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c00057c0 con 0x7fe8d40b2010 2023-10-16T10:03:39.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1272308936 0 0) 0x7fe8c00057c0 con 0x7fe8d40b2010 2023-10-16T10:03:39.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 msgr2=0x7fe8d40ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:39.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 0x7fe8d40ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 >> v2:172.21.15.190:3300/0 conn(0x7fe8d40cc6e0 msgr2=0x7fe8d4071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:39.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 --2- 172.21.15.181:0/3785217036 >> v2:172.21.15.190:3300/0 conn(0x7fe8d40cc6e0 0x7fe8d4071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40ccda0 con 0x7fe8d40b2010 2023-10-16T10:03:39.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe8c4001670 con 0x7fe8d40b2010 2023-10-16T10:03:39.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 >> v1:172.21.15.181:6789/0 conn(0x7fe8d40b2010 legacy=0x7fe8d40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8c0006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8c0006a60 0x7fe8c0006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 0x7fe8c0007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40ccb90 con 0x7fe8c0005c10 2023-10-16T10:03:39.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4072120 con 0x7fe8c0006a60 2023-10-16T10:03:39.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c000dc20 con 0x7fe8d40ce680 2023-10-16T10:03:39.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8c0006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d956f700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8c0006a60 0x7fe8c0006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8da571700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 0x7fe8c0007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 -- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 msgr2=0x7fe8c0007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:39.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 0x7fe8c0007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 -- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8c0006a60 msgr2=0x7fe8c0006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:39.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8c0006a60 0x7fe8c0006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8d9d70700 1 -- 172.21.15.181:0/3785217036 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00057c0 con 0x7fe8c0005c10 2023-10-16T10:03:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.080+0000 7fe8da571700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d40ce680 0x7fe8c0007420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8d9d70700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8c0006030 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c370 tx=0x7fe8c0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19be7f24ec210bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4007c60 con 0x7fe8c0005c10 2023-10-16T10:03:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4005960 con 0x7fe8c0005c10 2023-10-16T10:03:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8d8d6e700 1 -- 172.21.15.181:0/3785217036 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4005d10 con 0x7fe8c0005c10 2023-10-16T10:03:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/3785217036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 msgr2=0x7fe8c0006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8dbfd4700 1 --2- 172.21.15.181:0/3785217036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8c0006030 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c370 tx=0x7fe8c0007a90 comp rx=0 tx=0).stop 2023-10-16T10:03:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8da571700 1 -- 172.21.15.181:0/3785217036 reap_dead start 2023-10-16T10:03:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/3785217036 shutdown_connections 2023-10-16T10:03:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/3785217036 >> 172.21.15.181:0/3785217036 conn(0x7fe8d40cd010 msgr2=0x7fe8d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/3785217036 shutdown_connections 2023-10-16T10:03:39.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.081+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/3785217036 wait complete. 2023-10-16T10:03:39.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.083+0000 7fe8dbfd4700 1 Processor -- start 2023-10-16T10:03:39.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.083+0000 7fe8dbfd4700 1 -- start start 2023-10-16T10:03:39.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8dbfd4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8d40c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8dbfd4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d414e750 0x7fe8d40c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8dbfd4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d41f2190 0x7fe8d41f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8dbfd4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41f2b60 con 0x7fe8c0005c10 2023-10-16T10:03:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8dbfd4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40ca520 con 0x7fe8d41f2190 2023-10-16T10:03:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8dbfd4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415b150 con 0x7fe8d414e750 2023-10-16T10:03:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8d40c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8d40c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48224/0 (socket says 172.21.15.181:48224) 2023-10-16T10:03:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 -- 172.21.15.181:0/91241000 learned_addr learned my addr 172.21.15.181:0/91241000 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8da571700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d41f2190 0x7fe8d41f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d956f700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d414e750 0x7fe8d40c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 -- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d414e750 msgr2=0x7fe8d40c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d414e750 0x7fe8d40c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 -- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d41f2190 msgr2=0x7fe8d41f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d41f2190 0x7fe8d41f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 -- 172.21.15.181:0/91241000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d415dbe0 con 0x7fe8c0005c10 2023-10-16T10:03:39.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8da571700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d41f2190 0x7fe8d41f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:39.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d956f700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d414e750 0x7fe8d40c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:39.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.084+0000 7fe8d9d70700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8d40c9aa0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00068b0 tx=0x7fe8c4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.085+0000 7fe8ca7fc700 1 -- 172.21.15.181:0/91241000 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4004a70 con 0x7fe8c0005c10 2023-10-16T10:03:39.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.085+0000 7fe8ca7fc700 1 -- 172.21.15.181:0/91241000 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4004bd0 con 0x7fe8c0005c10 2023-10-16T10:03:39.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.085+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d40c81c0 con 0x7fe8c0005c10 2023-10-16T10:03:39.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.085+0000 7fe8ca7fc700 1 -- 172.21.15.181:0/91241000 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c400cec0 con 0x7fe8c0005c10 2023-10-16T10:03:39.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.085+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d415ae70 con 0x7fe8c0005c10 2023-10-16T10:03:39.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.086+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d40b6510 con 0x7fe8c0005c10 2023-10-16T10:03:39.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.086+0000 7fe8ca7fc700 1 -- 172.21.15.181:0/91241000 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 80) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fe8c40070d0 con 0x7fe8c0005c10 2023-10-16T10:03:39.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.086+0000 7fe8ca7fc700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe8b806b070 0x7fe8b806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.086+0000 7fe8ca7fc700 1 -- 172.21.15.181:0/91241000 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7fe8c408af90 con 0x7fe8c0005c10 2023-10-16T10:03:39.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.086+0000 7fe8d956f700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe8b806b070 0x7fe8b806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.087+0000 7fe8d956f700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe8b806b070 0x7fe8b806d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe8d414eb70 tx=0x7fe8d0008040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.091+0000 7fe8ca7fc700 1 -- 172.21.15.181:0/91241000 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe8c40bdfe0 con 0x7fe8c0005c10 2023-10-16T10:03:39.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.310+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/yearly", "snap_schedule": "1Y", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8d40c9fe0 con 0x7fe8b806b070 2023-10-16T10:03:39.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.374+0000 7fe8ca7fc700 1 -- 172.21.15.181:0/91241000 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7fe8d0009790 con 0x7fe8b806b070 2023-10-16T10:03:39.376 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1/yearly 2023-10-16T10:03:39.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe8b806b070 msgr2=0x7fe8b806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe8b806b070 0x7fe8b806d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe8d414eb70 tx=0x7fe8d0008040 comp rx=0 tx=0).stop 2023-10-16T10:03:39.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 msgr2=0x7fe8d40c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8d40c9aa0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00068b0 tx=0x7fe8c4000fe0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 shutdown_connections 2023-10-16T10:03:39.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fe8b806b070 0x7fe8b806d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8d41f2190 0x7fe8d41f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.376+0000 7fe8dbfd4700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8c0005c10 0x7fe8d40c9aa0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.377+0000 7fe8dbfd4700 1 --2- 172.21.15.181:0/91241000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8d414e750 0x7fe8d40c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.377+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 >> 172.21.15.181:0/91241000 conn(0x7fe8d40cd010 msgr2=0x7fe8d40c6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:39.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.377+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 shutdown_connections 2023-10-16T10:03:39.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.377+0000 7fe8dbfd4700 1 -- 172.21.15.181:0/91241000 wait complete. 2023-10-16T10:03:39.390 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/yearly 2023-10-16T10:03:39.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:39.391 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/bad_period_spec) 2023-10-16T10:03:39.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1X 2023-10-16T10:03:39.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.570+0000 7fee326c8700 1 Processor -- start 2023-10-16T10:03:39.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.570+0000 7fee326c8700 1 -- start start 2023-10-16T10:03:39.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee326c8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee2c15f150 0x7fee2c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee326c8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee2c0cd170 0x7fee2c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee326c8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2c0b2ae0 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee326c8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0ce9a0 con 0x7fee2c15f150 2023-10-16T10:03:39.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee326c8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0b2dc0 con 0x7fee2c0cd170 2023-10-16T10:03:39.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee2c15f150 0x7fee2c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fee2c0ce4f0 0x7fee2c0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59308/0 (socket says 172.21.15.181:59308) 2023-10-16T10:03:39.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2b7fe700 1 -- 172.21.15.181:0/3390158612 learned_addr learned my addr 172.21.15.181:0/3390158612 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:39.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee30c65700 1 --2- 172.21.15.181:0/3390158612 >> v2:172.21.15.190:3300/0 conn(0x7fee2c0cd170 0x7fee2c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2bfff700 1 -- 172.21.15.181:0/3390158612 >> v2:172.21.15.190:3300/0 conn(0x7fee2c0cd170 msgr2=0x7fee2c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:39.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2bfff700 1 --2- 172.21.15.181:0/3390158612 >> v2:172.21.15.190:3300/0 conn(0x7fee2c0cd170 0x7fee2c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2bfff700 1 -- 172.21.15.181:0/3390158612 >> v1:172.21.15.181:6789/0 conn(0x7fee2c0ce4f0 legacy=0x7fee2c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2bfff700 1 -- 172.21.15.181:0/3390158612 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c0cf9d0 con 0x7fee2c15f150 2023-10-16T10:03:39.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee2affd700 1 -- 172.21.15.181:0/3390158612 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130404883 0 0) 0x7fee2c0b2ae0 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.571+0000 7fee30c65700 1 --2- 172.21.15.181:0/3390158612 >> v2:172.21.15.190:3300/0 conn(0x7fee2c0cd170 0x7fee2c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:39.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee2bfff700 1 --2- 172.21.15.181:0/3390158612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c15f150 0x7fee2c0cdfb0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fee20009840 tx=0x7fee20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7abf1a41473198 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee2affd700 1 -- 172.21.15.181:0/3390158612 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20011830 con 0x7fee2c15f150 2023-10-16T10:03:39.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee2affd700 1 -- 172.21.15.181:0/3390158612 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20011990 con 0x7fee2c15f150 2023-10-16T10:03:39.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee2affd700 1 -- 172.21.15.181:0/3390158612 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20011d20 con 0x7fee2c15f150 2023-10-16T10:03:39.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 -- 172.21.15.181:0/3390158612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c15f150 msgr2=0x7fee2c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 --2- 172.21.15.181:0/3390158612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c15f150 0x7fee2c0cdfb0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fee20009840 tx=0x7fee20009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 -- 172.21.15.181:0/3390158612 shutdown_connections 2023-10-16T10:03:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 --2- 172.21.15.181:0/3390158612 >> v2:172.21.15.190:3300/0 conn(0x7fee2c0cd170 0x7fee2c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 --2- 172.21.15.181:0/3390158612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c15f150 0x7fee2c0cdfb0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 -- 172.21.15.181:0/3390158612 >> 172.21.15.181:0/3390158612 conn(0x7fee2c09a930 msgr2=0x7fee2c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 -- 172.21.15.181:0/3390158612 shutdown_connections 2023-10-16T10:03:39.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.572+0000 7fee326c8700 1 -- 172.21.15.181:0/3390158612 wait complete. 2023-10-16T10:03:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.574+0000 7fee326c8700 1 Processor -- start 2023-10-16T10:03:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.574+0000 7fee326c8700 1 -- start start 2023-10-16T10:03:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.574+0000 7fee326c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee2c0cd170 0x7fee2c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.574+0000 7fee326c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c0ce4f0 0x7fee2c1e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.574+0000 7fee326c8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c15f150 0x7fee2c1e5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee326c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0cb170 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee326c8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0c98a0 con 0x7fee2c15f150 2023-10-16T10:03:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee326c8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0c9b80 con 0x7fee2c0cd170 2023-10-16T10:03:39.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c0ce4f0 0x7fee2c1e53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c0ce4f0 0x7fee2c1e53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48234/0 (socket says 172.21.15.181:48234) 2023-10-16T10:03:39.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2674181446 learned_addr learned my addr 172.21.15.181:0/2674181446 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:39.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2674181446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee2c0cd170 msgr2=0x7fee2c151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee30c65700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c15f150 0x7fee2c1e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2bfff700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee2c0cd170 0x7fee2c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee2c0cd170 0x7fee2c151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2674181446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c15f150 msgr2=0x7fee2c1e5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c15f150 0x7fee2c1e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 -- 172.21.15.181:0/2674181446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200094a0 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee2b7fe700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee2c0ce4f0 0x7fee2c1e53c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0011e0 tx=0x7fee1c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee28ff9700 1 -- 172.21.15.181:0/2674181446 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c004c80 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee28ff9700 1 -- 172.21.15.181:0/2674181446 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c004e00 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee326c8700 1 -- 172.21.15.181:0/2674181446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c0c9e60 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee28ff9700 1 -- 172.21.15.181:0/2674181446 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c007310 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.575+0000 7fee326c8700 1 -- 172.21.15.181:0/2674181446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c1f1ee0 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.576+0000 7fee326c8700 1 -- 172.21.15.181:0/2674181446 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c0ca380 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.577+0000 7fee28ff9700 1 -- 172.21.15.181:0/2674181446 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 80) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fee1c007470 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.577+0000 7fee28ff9700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fee1806b070 0x7fee1806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.577+0000 7fee28ff9700 1 -- 172.21.15.181:0/2674181446 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7fee1c089880 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.577+0000 7fee2bfff700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fee1806b070 0x7fee1806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.578+0000 7fee2bfff700 1 --2- 172.21.15.181:0/2674181446 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fee1806b070 0x7fee1806d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0d2110 tx=0x7fee20008030 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.582+0000 7fee28ff9700 1 -- 172.21.15.181:0/2674181446 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fee2c0ca380 con 0x7fee2c0ce4f0 2023-10-16T10:03:39.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.800+0000 7fee326c8700 1 -- 172.21.15.181:0/2674181446 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1X", "target": ["mon-mgr", ""]}) v1 -- 0x7fee2c0c9a10 con 0x7fee1806b070 2023-10-16T10:03:39.802 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:39.801+0000 7eff1b45c700 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-16T10:03:39.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.801+0000 7fee28ff9700 1 -- 172.21.15.181:0/2674181446 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - multiplier should be one of h,d,w,M,Y) v1 ==== 74+0+0 (secure 0 0 0) 0x7fee2c0c9a10 con 0x7fee1806b070 2023-10-16T10:03:39.805 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-16T10:03:39.808 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:03:39.809 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1MM 2023-10-16T10:03:39.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.931+0000 7f6ace89a700 1 Processor -- start 2023-10-16T10:03:39.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.931+0000 7f6ace89a700 1 -- start start 2023-10-16T10:03:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ace89a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6ac80cdd80 0x7f6ac80ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ace89a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac806b6f0 0x7f6ac806bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ace89a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac815dfb0 con 0x7f6ac815dbd0 2023-10-16T10:03:39.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ace89a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac80ce740 con 0x7f6ac80cdd80 2023-10-16T10:03:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ace89a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80cd3a0 con 0x7f6ac806b6f0 2023-10-16T10:03:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ac7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6ac815dbd0 0x7f6ac80d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59318/0 (socket says 172.21.15.181:59318) 2023-10-16T10:03:39.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ac7fff700 1 -- 172.21.15.181:0/2853363401 learned_addr learned my addr 172.21.15.181:0/2853363401 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ac77fe700 1 --2- 172.21.15.181:0/2853363401 >> v2:172.21.15.190:3300/0 conn(0x7f6ac80cdd80 0x7f6ac80ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6acce37700 1 --2- 172.21.15.181:0/2853363401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac806b6f0 0x7f6ac806bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.932+0000 7f6ac6ffd700 1 -- 172.21.15.181:0/2853363401 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509572690 0 0) 0x7f6abc001110 con 0x7f6ac815dbd0 2023-10-16T10:03:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6ac6ffd700 1 -- 172.21.15.181:0/2853363401 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab00035b0 con 0x7f6ac815dbd0 2023-10-16T10:03:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6acce37700 1 -- 172.21.15.181:0/2853363401 >> v2:172.21.15.190:3300/0 conn(0x7f6ac80cdd80 msgr2=0x7f6ac80ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6acce37700 1 --2- 172.21.15.181:0/2853363401 >> v2:172.21.15.190:3300/0 conn(0x7f6ac80cdd80 0x7f6ac80ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6acce37700 1 -- 172.21.15.181:0/2853363401 >> v1:172.21.15.181:6789/0 conn(0x7f6ac815dbd0 legacy=0x7f6ac80d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6acce37700 1 -- 172.21.15.181:0/2853363401 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac80cd830 con 0x7f6ac806b6f0 2023-10-16T10:03:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6ac77fe700 1 --2- 172.21.15.181:0/2853363401 >> v2:172.21.15.190:3300/0 conn(0x7f6ac80cdd80 0x7f6ac80ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:39.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6acce37700 1 --2- 172.21.15.181:0/2853363401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac806b6f0 0x7f6ac806bb70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009d90 tx=0x7f6ac000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dfb5b672a6f775c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6ac6ffd700 1 -- 172.21.15.181:0/2853363401 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000e070 con 0x7f6ac806b6f0 2023-10-16T10:03:39.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6ac6ffd700 1 -- 172.21.15.181:0/2853363401 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac000bb20 con 0x7f6ac806b6f0 2023-10-16T10:03:39.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.933+0000 7f6ac6ffd700 1 -- 172.21.15.181:0/2853363401 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0012430 con 0x7f6ac806b6f0 2023-10-16T10:03:39.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 -- 172.21.15.181:0/2853363401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac806b6f0 msgr2=0x7f6ac806bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 --2- 172.21.15.181:0/2853363401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac806b6f0 0x7f6ac806bb70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009d90 tx=0x7f6ac000b330 comp rx=0 tx=0).stop 2023-10-16T10:03:39.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 -- 172.21.15.181:0/2853363401 shutdown_connections 2023-10-16T10:03:39.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 --2- 172.21.15.181:0/2853363401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac806b6f0 0x7f6ac806bb70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 --2- 172.21.15.181:0/2853363401 >> v2:172.21.15.190:3300/0 conn(0x7f6ac80cdd80 0x7f6ac80ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 -- 172.21.15.181:0/2853363401 >> 172.21.15.181:0/2853363401 conn(0x7f6ac809a930 msgr2=0x7f6ac80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:39.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 -- 172.21.15.181:0/2853363401 shutdown_connections 2023-10-16T10:03:39.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.934+0000 7f6ace89a700 1 -- 172.21.15.181:0/2853363401 wait complete. 2023-10-16T10:03:39.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.935+0000 7f6ace89a700 1 Processor -- start 2023-10-16T10:03:39.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.935+0000 7f6ace89a700 1 -- start start 2023-10-16T10:03:39.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ace89a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806b6f0 0x7f6ac809b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ace89a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ac80cdd80 0x7f6ac809ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ace89a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac815dbd0 0x7f6ac815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ace89a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac8160380 con 0x7f6ac806b6f0 2023-10-16T10:03:39.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ace89a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac815e150 con 0x7f6ac80cdd80 2023-10-16T10:03:39.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ace89a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac81605b0 con 0x7f6ac815dbd0 2023-10-16T10:03:39.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806b6f0 0x7f6ac809b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806b6f0 0x7f6ac809b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48246/0 (socket says 172.21.15.181:48246) 2023-10-16T10:03:39.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 -- 172.21.15.181:0/1423531566 learned_addr learned my addr 172.21.15.181:0/1423531566 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:39.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6acce37700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac815dbd0 0x7f6ac815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac77fe700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ac80cdd80 0x7f6ac809ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 -- 172.21.15.181:0/1423531566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac815dbd0 msgr2=0x7f6ac815fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ac815dbd0 0x7f6ac815fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 -- 172.21.15.181:0/1423531566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ac80cdd80 msgr2=0x7f6ac809ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:39.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ac80cdd80 0x7f6ac809ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:39.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 -- 172.21.15.181:0/1423531566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab80094a0 con 0x7f6ac806b6f0 2023-10-16T10:03:39.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac7fff700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806b6f0 0x7f6ac809b3a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6abc007fd0 tx=0x7f6abc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc005d70 con 0x7f6ac806b6f0 2023-10-16T10:03:39.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc0070d0 con 0x7f6ac806b6f0 2023-10-16T10:03:39.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ace89a700 1 -- 172.21.15.181:0/1423531566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac0009a70 con 0x7f6ac806b6f0 2023-10-16T10:03:39.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.936+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc007460 con 0x7f6ac806b6f0 2023-10-16T10:03:39.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.937+0000 7f6ace89a700 1 -- 172.21.15.181:0/1423531566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac8160b70 con 0x7f6ac806b6f0 2023-10-16T10:03:39.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.937+0000 7f6ace89a700 1 -- 172.21.15.181:0/1423531566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac8067d20 con 0x7f6ac806b6f0 2023-10-16T10:03:39.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.938+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 80) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f6abc0075c0 con 0x7f6ac806b6f0 2023-10-16T10:03:39.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.938+0000 7f6ac4ff9700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f6ab406b070 0x7f6ab406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:39.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.938+0000 7f6ac77fe700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f6ab406b070 0x7f6ab406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:39.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.938+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f6abc088e10 con 0x7f6ac806b6f0 2023-10-16T10:03:39.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.939+0000 7f6ac77fe700 1 --2- 172.21.15.181:0/1423531566 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f6ab406b070 0x7f6ab406d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8161560 tx=0x7f6ab8008040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:39.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:39.943+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6abc053b20 con 0x7f6ac806b6f0 2023-10-16T10:03:40.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.025+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 81) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f6abc053ef0 con 0x7f6ac806b6f0 2023-10-16T10:03:40.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.165+0000 7f6ace89a700 1 -- 172.21.15.181:0/1423531566 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1MM", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ac80cc790 con 0x7f6ab406b070 2023-10-16T10:03:40.167 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:40.166+0000 7eff1b45c700 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:40.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.166+0000 7f6ac4ff9700 1 -- 172.21.15.181:0/1423531566 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6ab8009e10 con 0x7f6ab406b070 2023-10-16T10:03:40.169 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:40.172 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:03:40.173 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1 2023-10-16T10:03:40.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.296+0000 7f2a5834c700 1 Processor -- start 2023-10-16T10:03:40.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.296+0000 7f2a5834c700 1 -- start start 2023-10-16T10:03:40.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a5834c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a5834c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a500ce4f0 0x7f2a500b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a5834c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a500b14a0 0x7f2a500b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a5834c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a500ce970 con 0x7f2a500b14a0 2023-10-16T10:03:40.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a5834c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a500b2ae0 con 0x7f2a500ce4f0 2023-10-16T10:03:40.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a5834c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500b2dc0 con 0x7f2a5015f150 2023-10-16T10:03:40.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a500b14a0 0x7f2a500b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a500b14a0 0x7f2a500b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48256/0 (socket says 172.21.15.181:48256) 2023-10-16T10:03:40.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 -- 172.21.15.181:0/296208607 learned_addr learned my addr 172.21.15.181:0/296208607 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:40.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a560e8700 1 --2- 172.21.15.181:0/296208607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a558e7700 1 --2- 172.21.15.181:0/296208607 >> v2:172.21.15.190:3300/0 conn(0x7f2a500ce4f0 0x7f2a500b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 -- 172.21.15.181:0/296208607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 msgr2=0x7f2a500cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:40.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 --2- 172.21.15.181:0/296208607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 -- 172.21.15.181:0/296208607 >> v2:172.21.15.190:3300/0 conn(0x7f2a500ce4f0 msgr2=0x7f2a500b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:40.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 --2- 172.21.15.181:0/296208607 >> v2:172.21.15.190:3300/0 conn(0x7f2a500ce4f0 0x7f2a500b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 -- 172.21.15.181:0/296208607 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500cf9d0 con 0x7f2a500b14a0 2023-10-16T10:03:40.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a568e9700 1 --2- 172.21.15.181:0/296208607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 0x7f2a500b1930 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2a50071730 tx=0x7f2a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7d0451ab5d4b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:40.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.297+0000 7f2a550e6700 1 -- 172.21.15.181:0/296208607 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800e070 con 0x7f2a500b14a0 2023-10-16T10:03:40.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a550e6700 1 -- 172.21.15.181:0/296208607 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4800b9c0 con 0x7f2a500b14a0 2023-10-16T10:03:40.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a550e6700 1 -- 172.21.15.181:0/296208607 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800bd70 con 0x7f2a500b14a0 2023-10-16T10:03:40.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 -- 172.21.15.181:0/296208607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 msgr2=0x7f2a500b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:40.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 --2- 172.21.15.181:0/296208607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 0x7f2a500b1930 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2a50071730 tx=0x7f2a4800b330 comp rx=0 tx=0).stop 2023-10-16T10:03:40.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 -- 172.21.15.181:0/296208607 shutdown_connections 2023-10-16T10:03:40.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 --2- 172.21.15.181:0/296208607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 0x7f2a500b1930 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 --2- 172.21.15.181:0/296208607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 --2- 172.21.15.181:0/296208607 >> v2:172.21.15.190:3300/0 conn(0x7f2a500ce4f0 0x7f2a500b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 -- 172.21.15.181:0/296208607 >> 172.21.15.181:0/296208607 conn(0x7f2a5009a930 msgr2=0x7f2a500d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:40.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 -- 172.21.15.181:0/296208607 shutdown_connections 2023-10-16T10:03:40.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.298+0000 7f2a5834c700 1 -- 172.21.15.181:0/296208607 wait complete. 2023-10-16T10:03:40.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.299+0000 7f2a5834c700 1 Processor -- start 2023-10-16T10:03:40.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.299+0000 7f2a5834c700 1 -- start start 2023-10-16T10:03:40.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a5834c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 0x7f2a500caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a5834c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a500ce4f0 0x7f2a501559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a5834c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a5834c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5006cbb0 con 0x7f2a500b14a0 2023-10-16T10:03:40.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a5834c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500c56f0 con 0x7f2a500ce4f0 2023-10-16T10:03:40.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a5834c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500caf40 con 0x7f2a5015f150 2023-10-16T10:03:40.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 0x7f2a500caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 0x7f2a500caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48260/0 (socket says 172.21.15.181:48260) 2023-10-16T10:03:40.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 -- 172.21.15.181:0/228294233 learned_addr learned my addr 172.21.15.181:0/228294233 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:40.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a558e7700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a500ce4f0 0x7f2a501559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a568e9700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 -- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 msgr2=0x7f2a500cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:40.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 -- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a500ce4f0 msgr2=0x7f2a501559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:40.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a500ce4f0 0x7f2a501559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a560e8700 1 -- 172.21.15.181:0/228294233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48009b30 con 0x7f2a500b14a0 2023-10-16T10:03:40.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a558e7700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a500ce4f0 0x7f2a501559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:40.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.300+0000 7f2a568e9700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a5015f150 0x7f2a500cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:40.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.301+0000 7f2a560e8700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a500b14a0 0x7f2a500caa00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00d800 tx=0x7f2a4c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:40.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.301+0000 7f2a46ffd700 1 -- 172.21.15.181:0/228294233 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0125f0 con 0x7f2a500b14a0 2023-10-16T10:03:40.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.301+0000 7f2a46ffd700 1 -- 172.21.15.181:0/228294233 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c012750 con 0x7f2a500b14a0 2023-10-16T10:03:40.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.301+0000 7f2a46ffd700 1 -- 172.21.15.181:0/228294233 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c012b00 con 0x7f2a500b14a0 2023-10-16T10:03:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.301+0000 7f2a5834c700 1 -- 172.21.15.181:0/228294233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a500c5130 con 0x7f2a500b14a0 2023-10-16T10:03:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.301+0000 7f2a5834c700 1 -- 172.21.15.181:0/228294233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5006c4f0 con 0x7f2a500b14a0 2023-10-16T10:03:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.302+0000 7f2a5834c700 1 -- 172.21.15.181:0/228294233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a50067d20 con 0x7f2a500b14a0 2023-10-16T10:03:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.302+0000 7f2a46ffd700 1 -- 172.21.15.181:0/228294233 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 81) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f2a4c012c60 con 0x7f2a500b14a0 2023-10-16T10:03:40.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.302+0000 7f2a46ffd700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2a3406b100 0x7f2a3406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.303+0000 7f2a46ffd700 1 -- 172.21.15.181:0/228294233 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f2a4c08a050 con 0x7f2a500b14a0 2023-10-16T10:03:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.303+0000 7f2a558e7700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2a3406b100 0x7f2a3406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.303+0000 7f2a558e7700 1 --2- 172.21.15.181:0/228294233 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2a3406b100 0x7f2a3406d5c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a500b0310 tx=0x7f2a40008040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:40.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.307+0000 7f2a46ffd700 1 -- 172.21.15.181:0/228294233 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2a4c0bcfe0 con 0x7f2a500b14a0 2023-10-16T10:03:40.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.525+0000 7f2a5834c700 1 -- 172.21.15.181:0/228294233 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a500cff90 con 0x7f2a3406b100 2023-10-16T10:03:40.527 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:40.526+0000 7eff1b45c700 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:40.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.526+0000 7f2a46ffd700 1 -- 172.21.15.181:0/228294233 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2a40009e10 con 0x7f2a3406b100 2023-10-16T10:03:40.529 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:40.533 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:03:40.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec M 2023-10-16T10:03:40.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.655+0000 7f865b92b700 1 Processor -- start 2023-10-16T10:03:40.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.655+0000 7f865b92b700 1 -- start start 2023-10-16T10:03:40.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f865b92b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 0x7f86540d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f865b92b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86540ce490 0x7f865406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f865b92b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865415dfb0 con 0x7f86540d0610 2023-10-16T10:03:40.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f865b92b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86540cd3a0 con 0x7f86540ce490 2023-10-16T10:03:40.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f865b92b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86540cd830 con 0x7f865415dbd0 2023-10-16T10:03:40.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86540ce490 0x7f865406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86540ce490 0x7f865406b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48268/0 (socket says 172.21.15.181:48268) 2023-10-16T10:03:40.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 -- 172.21.15.181:0/1008760899 learned_addr learned my addr 172.21.15.181:0/1008760899 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:40.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f86596c7700 1 --2- 172.21.15.181:0/1008760899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 0x7f86540d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 -- 172.21.15.181:0/1008760899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 msgr2=0x7f86540d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:40.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 --2- 172.21.15.181:0/1008760899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 0x7f86540d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 -- 172.21.15.181:0/1008760899 >> v1:172.21.15.190:6789/0 conn(0x7f86540d0610 legacy=0x7f86540cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:40.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 -- 172.21.15.181:0/1008760899 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540d1fc0 con 0x7f86540ce490 2023-10-16T10:03:40.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f8659ec8700 1 --2- 172.21.15.181:0/1008760899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 0x7f865406b6f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f864c009e50 tx=0x7f864c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9819e12b35b6c0a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:40.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f864bfff700 1 -- 172.21.15.181:0/1008760899 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00e070 con 0x7f86540ce490 2023-10-16T10:03:40.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f864bfff700 1 -- 172.21.15.181:0/1008760899 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c00b9c0 con 0x7f86540ce490 2023-10-16T10:03:40.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.656+0000 7f864bfff700 1 -- 172.21.15.181:0/1008760899 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00bd70 con 0x7f86540ce490 2023-10-16T10:03:40.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 -- 172.21.15.181:0/1008760899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 msgr2=0x7f865406b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:40.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 --2- 172.21.15.181:0/1008760899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 0x7f865406b6f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f864c009e50 tx=0x7f864c00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:40.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 -- 172.21.15.181:0/1008760899 shutdown_connections 2023-10-16T10:03:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 --2- 172.21.15.181:0/1008760899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 0x7f865406b6f0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 --2- 172.21.15.181:0/1008760899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 0x7f86540d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 -- 172.21.15.181:0/1008760899 >> 172.21.15.181:0/1008760899 conn(0x7f865409a930 msgr2=0x7f86540d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 -- 172.21.15.181:0/1008760899 shutdown_connections 2023-10-16T10:03:40.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.657+0000 7f865b92b700 1 -- 172.21.15.181:0/1008760899 wait complete. 2023-10-16T10:03:40.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.658+0000 7f865b92b700 1 Processor -- start 2023-10-16T10:03:40.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.658+0000 7f865b92b700 1 -- start start 2023-10-16T10:03:40.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f865b92b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 0x7f8654144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f865b92b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86540d0610 0x7f8654160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f865b92b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 0x7f8654160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f865b92b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654161410 con 0x7f86540ce490 2023-10-16T10:03:40.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f865b92b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541566c0 con 0x7f86540d0610 2023-10-16T10:03:40.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f865b92b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8654145460 con 0x7f865415dbd0 2023-10-16T10:03:40.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 0x7f8654144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 0x7f8654144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48270/0 (socket says 172.21.15.181:48270) 2023-10-16T10:03:40.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 -- 172.21.15.181:0/753854927 learned_addr learned my addr 172.21.15.181:0/753854927 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:40.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f8659ec8700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 0x7f8654160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f8658ec6700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86540d0610 0x7f8654160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 -- 172.21.15.181:0/753854927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 msgr2=0x7f8654160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:40.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f865415dbd0 0x7f8654160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 -- 172.21.15.181:0/753854927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86540d0610 msgr2=0x7f8654160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:40.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86540d0610 0x7f8654160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:40.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 -- 172.21.15.181:0/753854927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009b30 con 0x7f86540ce490 2023-10-16T10:03:40.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.659+0000 7f86596c7700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86540ce490 0x7f8654144f20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f864400d800 tx=0x7f864400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:40.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.660+0000 7f8649ffb700 1 -- 172.21.15.181:0/753854927 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440125f0 con 0x7f86540ce490 2023-10-16T10:03:40.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.660+0000 7f8649ffb700 1 -- 172.21.15.181:0/753854927 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644012750 con 0x7f86540ce490 2023-10-16T10:03:40.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.660+0000 7f8649ffb700 1 -- 172.21.15.181:0/753854927 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644012b00 con 0x7f86540ce490 2023-10-16T10:03:40.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.660+0000 7f865b92b700 1 -- 172.21.15.181:0/753854927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8654160b70 con 0x7f86540ce490 2023-10-16T10:03:40.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.660+0000 7f865b92b700 1 -- 172.21.15.181:0/753854927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8654161130 con 0x7f86540ce490 2023-10-16T10:03:40.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.661+0000 7f865b92b700 1 -- 172.21.15.181:0/753854927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8654156520 con 0x7f86540ce490 2023-10-16T10:03:40.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.661+0000 7f8649ffb700 1 -- 172.21.15.181:0/753854927 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 81) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f8644012c60 con 0x7f86540ce490 2023-10-16T10:03:40.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.661+0000 7f8649ffb700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f863806b100 0x7f863806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:40.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.662+0000 7f8649ffb700 1 -- 172.21.15.181:0/753854927 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f8644089cf0 con 0x7f86540ce490 2023-10-16T10:03:40.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.662+0000 7f8658ec6700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f863806b100 0x7f863806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:40.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.662+0000 7f8658ec6700 1 --2- 172.21.15.181:0/753854927 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f863806b100 0x7f863806d5c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f865414b7d0 tx=0x7f8650009040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:40.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.666+0000 7f8649ffb700 1 -- 172.21.15.181:0/753854927 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8654156520 con 0x7f86540ce490 2023-10-16T10:03:40.693 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:40.695 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:03:40.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.889+0000 7f865b92b700 1 -- 172.21.15.181:0/753854927 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "M", "target": ["mon-mgr", ""]}) v1 -- 0x7f86540cca70 con 0x7f863806b100 2023-10-16T10:03:40.892 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:40.890+0000 7eff1b45c700 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:40.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:40.891+0000 7f8649ffb700 1 -- 172.21.15.181:0/753854927 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8650001560 con 0x7f863806b100 2023-10-16T10:03:40.894 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:40.897 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:03:40.898 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec -1m 2023-10-16T10:03:41.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.021+0000 7f795c2ab700 1 Processor -- start 2023-10-16T10:03:41.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.021+0000 7f795c2ab700 1 -- start start 2023-10-16T10:03:41.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.021+0000 7f795c2ab700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7954161f70 0x7f79540cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.021+0000 7f795c2ab700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79540ce410 0x7f79540ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.021+0000 7f795c2ab700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79540ce860 con 0x7f795415d740 2023-10-16T10:03:41.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.021+0000 7f795c2ab700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79540d0cb0 con 0x7f7954161f70 2023-10-16T10:03:41.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.021+0000 7f795c2ab700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7954162400 con 0x7f79540ce410 2023-10-16T10:03:41.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f7959846700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7954161f70 0x7f79540cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f795a848700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79540ce410 0x7f79540ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f795a047700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f795415d740 0x7f79540cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:43298/0 (socket says 172.21.15.181:43298) 2023-10-16T10:03:41.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f795a848700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79540ce410 0x7f79540ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:45308/0 (socket says 172.21.15.181:45308) 2023-10-16T10:03:41.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f7959846700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7954161f70 0x7f79540cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:53330/0 (socket says 172.21.15.181:53330) 2023-10-16T10:03:41.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f795a047700 1 -- 172.21.15.181:0/2494328068 learned_addr learned my addr 172.21.15.181:0/2494328068 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:41.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f7959846700 1 -- 172.21.15.181:0/2494328068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79540ce410 msgr2=0x7f79540ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:41.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f7959846700 1 --2- 172.21.15.181:0/2494328068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79540ce410 0x7f79540ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f7959846700 1 -- 172.21.15.181:0/2494328068 >> v1:172.21.15.190:6789/0 conn(0x7f795415d740 legacy=0x7f79540cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f7959846700 1 -- 172.21.15.181:0/2494328068 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540d00a0 con 0x7f7954161f70 2023-10-16T10:03:41.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.022+0000 7f7959846700 1 --2- 172.21.15.181:0/2494328068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7954161f70 0x7f79540cded0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7950009840 tx=0x7f7950009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db5438cb7f6bbd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:41.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.028+0000 7f7959045700 1 -- 172.21.15.181:0/2494328068 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950011830 con 0x7f7954161f70 2023-10-16T10:03:41.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.028+0000 7f7959045700 1 -- 172.21.15.181:0/2494328068 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7950011990 con 0x7f7954161f70 2023-10-16T10:03:41.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.028+0000 7f7959045700 1 -- 172.21.15.181:0/2494328068 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950011d20 con 0x7f7954161f70 2023-10-16T10:03:41.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 -- 172.21.15.181:0/2494328068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7954161f70 msgr2=0x7f79540cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 --2- 172.21.15.181:0/2494328068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7954161f70 0x7f79540cded0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7950009840 tx=0x7f7950009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:41.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 -- 172.21.15.181:0/2494328068 shutdown_connections 2023-10-16T10:03:41.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 --2- 172.21.15.181:0/2494328068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79540ce410 0x7f79540ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 --2- 172.21.15.181:0/2494328068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7954161f70 0x7f79540cded0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 -- 172.21.15.181:0/2494328068 >> 172.21.15.181:0/2494328068 conn(0x7f795409a930 msgr2=0x7f79540d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:41.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 -- 172.21.15.181:0/2494328068 shutdown_connections 2023-10-16T10:03:41.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.029+0000 7f795c2ab700 1 -- 172.21.15.181:0/2494328068 wait complete. 2023-10-16T10:03:41.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.031+0000 7f795c2ab700 1 Processor -- start 2023-10-16T10:03:41.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.031+0000 7f795c2ab700 1 -- start start 2023-10-16T10:03:41.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795c2ab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79540ce410 0x7f79540c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795c2ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f795415d740 0x7f7954070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795c2ab700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7954161f70 0x7f79540cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795c2ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79540c90f0 con 0x7f795415d740 2023-10-16T10:03:41.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795c2ab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79540cdd70 con 0x7f79540ce410 2023-10-16T10:03:41.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795c2ab700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79540c87c0 con 0x7f7954161f70 2023-10-16T10:03:41.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f795415d740 0x7f7954070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f795415d740 0x7f7954070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48286/0 (socket says 172.21.15.181:48286) 2023-10-16T10:03:41.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 -- 172.21.15.181:0/2559838088 learned_addr learned my addr 172.21.15.181:0/2559838088 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:41.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795a848700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7954161f70 0x7f79540cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f795a047700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79540ce410 0x7f79540c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 -- 172.21.15.181:0/2559838088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7954161f70 msgr2=0x7f79540cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7954161f70 0x7f79540cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 -- 172.21.15.181:0/2559838088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79540ce410 msgr2=0x7f79540c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79540ce410 0x7f79540c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 -- 172.21.15.181:0/2559838088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500094a0 con 0x7f795415d740 2023-10-16T10:03:41.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f7959846700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f795415d740 0x7f7954070180 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7950014040 tx=0x7f7950004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:41.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.032+0000 7f794affd700 1 -- 172.21.15.181:0/2559838088 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500036a0 con 0x7f795415d740 2023-10-16T10:03:41.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.033+0000 7f794affd700 1 -- 172.21.15.181:0/2559838088 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795001f070 con 0x7f795415d740 2023-10-16T10:03:41.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.033+0000 7f794affd700 1 -- 172.21.15.181:0/2559838088 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950003a00 con 0x7f795415d740 2023-10-16T10:03:41.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.033+0000 7f795c2ab700 1 -- 172.21.15.181:0/2559838088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79540c8aa0 con 0x7f795415d740 2023-10-16T10:03:41.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.033+0000 7f795c2ab700 1 -- 172.21.15.181:0/2559838088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7954151b50 con 0x7f795415d740 2023-10-16T10:03:41.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.034+0000 7f795c2ab700 1 -- 172.21.15.181:0/2559838088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795404ebb0 con 0x7f795415d740 2023-10-16T10:03:41.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.034+0000 7f794affd700 1 -- 172.21.15.181:0/2559838088 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 81) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f7950003b60 con 0x7f795415d740 2023-10-16T10:03:41.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.034+0000 7f794affd700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f793806b100 0x7f793806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.034+0000 7f794affd700 1 -- 172.21.15.181:0/2559838088 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f7950021070 con 0x7f795415d740 2023-10-16T10:03:41.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.034+0000 7f795a047700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f793806b100 0x7f793806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.035+0000 7f795a047700 1 --2- 172.21.15.181:0/2559838088 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f793806b100 0x7f793806d5c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7944005910 tx=0x7f7944009040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:41.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.039+0000 7f794affd700 1 -- 172.21.15.181:0/2559838088 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f79500560e0 con 0x7f795415d740 2023-10-16T10:03:41.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.257+0000 7f795c2ab700 1 -- 172.21.15.181:0/2559838088 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "-1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f79540d0380 con 0x7f793806b100 2023-10-16T10:03:41.259 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:41.258+0000 7eff1b45c700 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-16T10:03:41.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.258+0000 7f794affd700 1 -- 172.21.15.181:0/2559838088 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc.) v1 ==== 96+0+0 (secure 0 0 0) 0x7f7944001340 con 0x7f793806b100 2023-10-16T10:03:41.261 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-16T10:03:41.264 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:03:41.265 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec '' 2023-10-16T10:03:41.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.388+0000 7fbfc7049700 1 Processor -- start 2023-10-16T10:03:41.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.388+0000 7fbfc7049700 1 -- start start 2023-10-16T10:03:41.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.388+0000 7fbfc7049700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfc015da70 0x7fbfc00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.388+0000 7fbfc7049700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfc006afb0 0x7fbfc01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.388+0000 7fbfc7049700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc00cef00 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.388+0000 7fbfc7049700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00cec20 con 0x7fbfc006afb0 2023-10-16T10:03:41.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.388+0000 7fbfc7049700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc0071c80 con 0x7fbfc015da70 2023-10-16T10:03:41.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc55e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfc006afb0 0x7fbfc01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfbffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbfc00ce6a0 0x7fbfc006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59370/0 (socket says 172.21.15.181:59370) 2023-10-16T10:03:41.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfbffff700 1 -- 172.21.15.181:0/1540936295 learned_addr learned my addr 172.21.15.181:0/1540936295 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:41.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc4de5700 1 --2- 172.21.15.181:0/1540936295 >> v2:172.21.15.190:3300/0 conn(0x7fbfc015da70 0x7fbfc00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc55e6700 1 -- 172.21.15.181:0/1540936295 >> v2:172.21.15.190:3300/0 conn(0x7fbfc015da70 msgr2=0x7fbfc00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:41.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc55e6700 1 --2- 172.21.15.181:0/1540936295 >> v2:172.21.15.190:3300/0 conn(0x7fbfc015da70 0x7fbfc00ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc55e6700 1 -- 172.21.15.181:0/1540936295 >> v1:172.21.15.181:6789/0 conn(0x7fbfc00ce6a0 legacy=0x7fbfc006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfbf7fe700 1 -- 172.21.15.181:0/1540936295 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103329278 0 0) 0x7fbfc00cef00 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc55e6700 1 -- 172.21.15.181:0/1540936295 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc0071f60 con 0x7fbfc006afb0 2023-10-16T10:03:41.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc4de5700 1 --2- 172.21.15.181:0/1540936295 >> v2:172.21.15.190:3300/0 conn(0x7fbfc015da70 0x7fbfc00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:41.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfc55e6700 1 --2- 172.21.15.181:0/1540936295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc006afb0 0x7fbfc01615f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00675e0 tx=0x7fbfb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11338b8859791a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:41.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfbf7fe700 1 -- 172.21.15.181:0/1540936295 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800e070 con 0x7fbfc006afb0 2023-10-16T10:03:41.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfbf7fe700 1 -- 172.21.15.181:0/1540936295 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800bbd0 con 0x7fbfc006afb0 2023-10-16T10:03:41.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.389+0000 7fbfbf7fe700 1 -- 172.21.15.181:0/1540936295 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8012430 con 0x7fbfc006afb0 2023-10-16T10:03:41.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 -- 172.21.15.181:0/1540936295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc006afb0 msgr2=0x7fbfc01615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 --2- 172.21.15.181:0/1540936295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc006afb0 0x7fbfc01615f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00675e0 tx=0x7fbfb800b330 comp rx=0 tx=0).stop 2023-10-16T10:03:41.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 -- 172.21.15.181:0/1540936295 shutdown_connections 2023-10-16T10:03:41.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 --2- 172.21.15.181:0/1540936295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc006afb0 0x7fbfc01615f0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 --2- 172.21.15.181:0/1540936295 >> v2:172.21.15.190:3300/0 conn(0x7fbfc015da70 0x7fbfc00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 -- 172.21.15.181:0/1540936295 >> 172.21.15.181:0/1540936295 conn(0x7fbfc00cd010 msgr2=0x7fbfc00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:41.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 -- 172.21.15.181:0/1540936295 shutdown_connections 2023-10-16T10:03:41.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.390+0000 7fbfc7049700 1 -- 172.21.15.181:0/1540936295 wait complete. 2023-10-16T10:03:41.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.391+0000 7fbfc7049700 1 Processor -- start 2023-10-16T10:03:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.391+0000 7fbfc7049700 1 -- start start 2023-10-16T10:03:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.392+0000 7fbfc7049700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc006afb0 0x7fbfc00b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.392+0000 7fbfc7049700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc00ce6a0 0x7fbfc00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc7049700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc015da70 0x7fbfc0159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc7049700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01f7050 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc7049700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01f66b0 con 0x7fbfc006afb0 2023-10-16T10:03:41.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc7049700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01f6990 con 0x7fbfc015da70 2023-10-16T10:03:41.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc00ce6a0 0x7fbfc00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc00ce6a0 0x7fbfc00adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48298/0 (socket says 172.21.15.181:48298) 2023-10-16T10:03:41.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 -- 172.21.15.181:0/235026092 learned_addr learned my addr 172.21.15.181:0/235026092 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:41.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc4de5700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc006afb0 0x7fbfc00b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc55e6700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc015da70 0x7fbfc0159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 -- 172.21.15.181:0/235026092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc015da70 msgr2=0x7fbfc0159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfc015da70 0x7fbfc0159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 -- 172.21.15.181:0/235026092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc006afb0 msgr2=0x7fbfc00b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfc006afb0 0x7fbfc00b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 -- 172.21.15.181:0/235026092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8009a70 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbffff700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc00ce6a0 0x7fbfc00adaa0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c550 tx=0x7fbfac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbd7fa700 1 -- 172.21.15.181:0/235026092 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004c80 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbd7fa700 1 -- 172.21.15.181:0/235026092 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac004e00 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc7049700 1 -- 172.21.15.181:0/235026092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc01f6c70 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfbd7fa700 1 -- 172.21.15.181:0/235026092 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac007310 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.393+0000 7fbfc7049700 1 -- 172.21.15.181:0/235026092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc015a4e0 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.394+0000 7fbfc7049700 1 -- 172.21.15.181:0/235026092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc01f7190 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.395+0000 7fbfbd7fa700 1 -- 172.21.15.181:0/235026092 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 81) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fbfac007470 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.395+0000 7fbfbd7fa700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fbfa406b100 0x7fbfa406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.395+0000 7fbfc4de5700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fbfa406b100 0x7fbfa406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.395+0000 7fbfbd7fa700 1 -- 172.21.15.181:0/235026092 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7fbfac089970 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.396+0000 7fbfc4de5700 1 --2- 172.21.15.181:0/235026092 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fbfa406b100 0x7fbfa406d5c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbfc006a910 tx=0x7fbfb4008040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:41.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.400+0000 7fbfbd7fa700 1 -- 172.21.15.181:0/235026092 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfc01f7190 con 0x7fbfc00ce6a0 2023-10-16T10:03:41.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.615+0000 7fbfc7049700 1 -- 172.21.15.181:0/235026092 --> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfc01f6820 con 0x7fbfa406b100 2023-10-16T10:03:41.617 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:41.616+0000 7eff1b45c700 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:41.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:41.616+0000 7fbfbd7fa700 1 -- 172.21.15.181:0/235026092 <== mgr.8727 v2:172.21.15.181:6832/3791133934 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbfc01f6820 con 0x7fbfa406b100 2023-10-16T10:03:41.619 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-16T10:03:41.622 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T10:03:41.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.623 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/minutes) 2023-10-16T10:03:41.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.642 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/hourly) 2023-10-16T10:03:41.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.701 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/daily) 2023-10-16T10:03:41.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.759 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/weekly) 2023-10-16T10:03:41.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.817 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/monthly) 2023-10-16T10:03:41.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.876 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/yearly) 2023-10-16T10:03:41.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:41.935 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/bad_period_spec) 2023-10-16T10:03:41.993 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T10:03:42.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.155+0000 7f09a5958700 1 Processor -- start 2023-10-16T10:03:42.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.155+0000 7f09a5958700 1 -- start start 2023-10-16T10:03:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f09a5958700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09a015dce0 0x7f09a00c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:42.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f09a5958700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a00cd9a0 con 0x7f09a00af090 2023-10-16T10:03:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f09a5958700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a00ce540 con 0x7f09a00aec90 2023-10-16T10:03:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f09a5958700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a006d9e0 con 0x7f09a015dce0 2023-10-16T10:03:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09a015dce0 0x7f09a00c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:42.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09a015dce0 0x7f09a00c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48304/0 (socket says 172.21.15.181:48304) 2023-10-16T10:03:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f09a00af090 0x7f09a006de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59386/0 (socket says 172.21.15.181:59386) 2023-10-16T10:03:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099effd700 1 -- 172.21.15.181:0/2189948642 learned_addr learned my addr 172.21.15.181:0/2189948642 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099effd700 1 -- 172.21.15.181:0/2189948642 >> v1:172.21.15.190:6789/0 conn(0x7f09a00aec90 legacy=0x7f09a00c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099effd700 1 -- 172.21.15.181:0/2189948642 >> v1:172.21.15.181:6789/0 conn(0x7f09a00af090 legacy=0x7f09a006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:42.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099effd700 1 -- 172.21.15.181:0/2189948642 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a00ce790 con 0x7f09a015dce0 2023-10-16T10:03:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099dffb700 1 -- 172.21.15.181:0/2189948642 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523961111 0 0) 0x7f09a00cd9a0 con 0x7f09a00af090 2023-10-16T10:03:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099effd700 1 --2- 172.21.15.181:0/2189948642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a015dce0 0x7f09a00c4b40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f09900098f0 tx=0x7f0990009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc29b062b9537a69 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099dffb700 1 -- 172.21.15.181:0/2189948642 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990004230 con 0x7f09a015dce0 2023-10-16T10:03:42.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099dffb700 1 -- 172.21.15.181:0/2189948642 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990004390 con 0x7f09a015dce0 2023-10-16T10:03:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.156+0000 7f099dffb700 1 -- 172.21.15.181:0/2189948642 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09900036a0 con 0x7f09a015dce0 2023-10-16T10:03:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.157+0000 7f09a5958700 1 -- 172.21.15.181:0/2189948642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a015dce0 msgr2=0x7f09a00c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.157+0000 7f09a5958700 1 --2- 172.21.15.181:0/2189948642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a015dce0 0x7f09a00c4b40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f09900098f0 tx=0x7f0990009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.157+0000 7f09a5958700 1 -- 172.21.15.181:0/2189948642 shutdown_connections 2023-10-16T10:03:42.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.157+0000 7f09a5958700 1 --2- 172.21.15.181:0/2189948642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a015dce0 0x7f09a00c4b40 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.157+0000 7f09a5958700 1 -- 172.21.15.181:0/2189948642 >> 172.21.15.181:0/2189948642 conn(0x7f09a00cc740 msgr2=0x7f09a00ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.157+0000 7f09a5958700 1 -- 172.21.15.181:0/2189948642 shutdown_connections 2023-10-16T10:03:42.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.157+0000 7f09a5958700 1 -- 172.21.15.181:0/2189948642 wait complete. 2023-10-16T10:03:42.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.158+0000 7f09a5958700 1 Processor -- start 2023-10-16T10:03:42.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.159+0000 7f09a5958700 1 -- start start 2023-10-16T10:03:42.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.159+0000 7f09a5958700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a00aec90 0x7f09a014f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:42.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f09a5958700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09a00af090 0x7f09a00ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:42.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f09a5958700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09a015dce0 0x7f09a00adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:42.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f09a5958700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a00c6d90 con 0x7f09a00aec90 2023-10-16T10:03:42.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f09a5958700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a0160bc0 con 0x7f09a015dce0 2023-10-16T10:03:42.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f09a5958700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a0160ea0 con 0x7f09a00af090 2023-10-16T10:03:42.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a00aec90 0x7f09a014f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:42.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a00aec90 0x7f09a014f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48310/0 (socket says 172.21.15.181:48310) 2023-10-16T10:03:42.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 -- 172.21.15.181:0/3688893369 learned_addr learned my addr 172.21.15.181:0/3688893369 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:42.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099e7fc700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09a00af090 0x7f09a00ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:42.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099f7fe700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09a015dce0 0x7f09a00adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:42.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 -- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09a00af090 msgr2=0x7f09a00ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09a00af090 0x7f09a00ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 -- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09a015dce0 msgr2=0x7f09a00adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09a015dce0 0x7f09a00adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 -- 172.21.15.181:0/3688893369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990009550 con 0x7f09a00aec90 2023-10-16T10:03:42.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099e7fc700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09a00af090 0x7f09a00ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099f7fe700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09a015dce0 0x7f09a00adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f099effd700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a00aec90 0x7f09a014f680 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f0990004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f0987fff700 1 -- 172.21.15.181:0/3688893369 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990003db0 con 0x7f09a00aec90 2023-10-16T10:03:42.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f0987fff700 1 -- 172.21.15.181:0/3688893369 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099001f070 con 0x7f09a00aec90 2023-10-16T10:03:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f0987fff700 1 -- 172.21.15.181:0/3688893369 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099000b8a0 con 0x7f09a00aec90 2023-10-16T10:03:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a00c6fe0 con 0x7f09a00aec90 2023-10-16T10:03:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.160+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a00cf3f0 con 0x7f09a00aec90 2023-10-16T10:03:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.161+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a00cf580 con 0x7f09a00aec90 2023-10-16T10:03:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.162+0000 7f0987fff700 1 -- 172.21.15.181:0/3688893369 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 81) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f0990021030 con 0x7f09a00aec90 2023-10-16T10:03:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.162+0000 7f0987fff700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f098c06b100 0x7f098c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.162+0000 7f0987fff700 1 -- 172.21.15.181:0/3688893369 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f099008b380 con 0x7f09a00aec90 2023-10-16T10:03:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.162+0000 7f099e7fc700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f098c06b100 0x7f098c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.163+0000 7f099e7fc700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f098c06b100 0x7f098c06d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0994006fd0 tx=0x7f0994008040 comp rx=0 tx=0).ready entity=mgr.8727 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.167+0000 7f0987fff700 1 -- 172.21.15.181:0/3688893369 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0990055f20 con 0x7f09a00aec90 2023-10-16T10:03:42.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:42.416+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f09a00cf580 con 0x7f09a00aec90 2023-10-16T10:03:43.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.033+0000 7f0987fff700 1 -- 172.21.15.181:0/3688893369 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v82) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0990059540 con 0x7f09a00aec90 2023-10-16T10:03:43.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f098c06b100 msgr2=0x7f098c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f098c06b100 0x7f098c06d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0994006fd0 tx=0x7f0994008040 comp rx=0 tx=0).stop 2023-10-16T10:03:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a00aec90 msgr2=0x7f09a014f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a00aec90 0x7f09a014f680 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f0990004750 comp rx=0 tx=0).stop 2023-10-16T10:03:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 shutdown_connections 2023-10-16T10:03:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f098c06b100 0x7f098c06d5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09a015dce0 0x7f09a00adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a00aec90 0x7f09a014f680 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 --2- 172.21.15.181:0/3688893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09a00af090 0x7f09a00ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 >> 172.21.15.181:0/3688893369 conn(0x7f09a00cc740 msgr2=0x7f09a00af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:43.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.038+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 shutdown_connections 2023-10-16T10:03:43.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.039+0000 7f09a5958700 1 -- 172.21.15.181:0/3688893369 wait complete. 2023-10-16T10:03:43.057 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:03:43.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:43.103 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:43.148 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:03:43.149 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:43.149 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:43.167 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:43.168 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:43.185 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:43.184+0000 7f9a6d36e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:03:43.191 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:43.189+0000 7fc064740200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:03:43.208 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:43.251 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:43.250+0000 7f9a6d36e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:43.260 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:43.258+0000 7fc064740200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:43.324 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:43.323+0000 7f9a6d36e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:03:43.337 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:43.335+0000 7fc064740200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:03:43.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.372+0000 7f880e238700 1 Processor -- start 2023-10-16T10:03:43.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.372+0000 7f880e238700 1 -- start start 2023-10-16T10:03:43.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f880e238700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f880806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f880e238700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8808160d20 0x7f88081611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f880e238700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880806ce20 con 0x7f8808161f40 2023-10-16T10:03:43.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f880e238700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8808162800 con 0x7f8808160d20 2023-10-16T10:03:43.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f880e238700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88080d43e0 con 0x7f8808161b60 2023-10-16T10:03:43.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f8807fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8808160d20 0x7f88081611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f8807fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8808160d20 0x7f88081611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48380/0 (socket says 172.21.15.181:48380) 2023-10-16T10:03:43.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f8807fff700 1 -- 172.21.15.181:0/3271568309 learned_addr learned my addr 172.21.15.181:0/3271568309 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:43.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.373+0000 7f88077fe700 1 --2- 172.21.15.181:0/3271568309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f880806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f8807fff700 1 -- 172.21.15.181:0/3271568309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 msgr2=0x7f880806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:43.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f8807fff700 1 --2- 172.21.15.181:0/3271568309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f880806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f8807fff700 1 -- 172.21.15.181:0/3271568309 >> v1:172.21.15.190:6789/0 conn(0x7f8808161f40 legacy=0x7f88080d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f8807fff700 1 -- 172.21.15.181:0/3271568309 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080d45f0 con 0x7f8808160d20 2023-10-16T10:03:43.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f88067fc700 1 -- 172.21.15.181:0/3271568309 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609567808 0 0) 0x7f880806ce20 con 0x7f8808161f40 2023-10-16T10:03:43.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f8807fff700 1 --2- 172.21.15.181:0/3271568309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808160d20 0x7f88081611d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8800009e50 tx=0x7f880000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8832a6c74a8794 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:43.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f88067fc700 1 -- 172.21.15.181:0/3271568309 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880000e070 con 0x7f8808160d20 2023-10-16T10:03:43.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f88067fc700 1 -- 172.21.15.181:0/3271568309 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880000b910 con 0x7f8808160d20 2023-10-16T10:03:43.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f88067fc700 1 -- 172.21.15.181:0/3271568309 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880000bcc0 con 0x7f8808160d20 2023-10-16T10:03:43.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f880e238700 1 -- 172.21.15.181:0/3271568309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808160d20 msgr2=0x7f88081611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.374+0000 7f880e238700 1 --2- 172.21.15.181:0/3271568309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808160d20 0x7f88081611d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8800009e50 tx=0x7f880000b330 comp rx=0 tx=0).stop 2023-10-16T10:03:43.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.375+0000 7f880e238700 1 -- 172.21.15.181:0/3271568309 shutdown_connections 2023-10-16T10:03:43.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.375+0000 7f880e238700 1 --2- 172.21.15.181:0/3271568309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808160d20 0x7f88081611d0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.375+0000 7f880e238700 1 --2- 172.21.15.181:0/3271568309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f880806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.375+0000 7f880e238700 1 -- 172.21.15.181:0/3271568309 >> 172.21.15.181:0/3271568309 conn(0x7f880809cba0 msgr2=0x7f88080d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:43.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.375+0000 7f880e238700 1 -- 172.21.15.181:0/3271568309 shutdown_connections 2023-10-16T10:03:43.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.375+0000 7f880e238700 1 -- 172.21.15.181:0/3271568309 wait complete. 2023-10-16T10:03:43.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.376+0000 7f880e238700 1 Processor -- start 2023-10-16T10:03:43.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.376+0000 7f880e238700 1 -- start start 2023-10-16T10:03:43.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f880e238700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808160d20 0x7f88080c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f880e238700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f8808160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f880e238700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808161f40 0x7f88080b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f880e238700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8808160890 con 0x7f8808161f40 2023-10-16T10:03:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f880e238700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88080cc460 con 0x7f8808160d20 2023-10-16T10:03:43.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f880e238700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88081606a0 con 0x7f8808161b60 2023-10-16T10:03:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f8807fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808161f40 0x7f88080b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f8807fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808161f40 0x7f88080b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48386/0 (socket says 172.21.15.181:48386) 2023-10-16T10:03:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f8807fff700 1 -- 172.21.15.181:0/843596618 learned_addr learned my addr 172.21.15.181:0/843596618 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f88077fe700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808160d20 0x7f88080c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f8806ffd700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f8808160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f88077fe700 1 -- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 msgr2=0x7f8808160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f88077fe700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f8808160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f88077fe700 1 -- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808161f40 msgr2=0x7f88080b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f88077fe700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808161f40 0x7f88080b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f88077fe700 1 -- 172.21.15.181:0/843596618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800009b30 con 0x7f8808160d20 2023-10-16T10:03:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f8807fff700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808161f40 0x7f88080b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.377+0000 7f88077fe700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808160d20 0x7f88080c9560 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f87f800e8a0 tx=0x7f87f800ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.378+0000 7f87effff700 1 -- 172.21.15.181:0/843596618 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8013680 con 0x7f8808160d20 2023-10-16T10:03:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.378+0000 7f87effff700 1 -- 172.21.15.181:0/843596618 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f80137e0 con 0x7f8808160d20 2023-10-16T10:03:43.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.378+0000 7f87effff700 1 -- 172.21.15.181:0/843596618 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8013b50 con 0x7f8808160d20 2023-10-16T10:03:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.378+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88080cbac0 con 0x7f8808160d20 2023-10-16T10:03:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.378+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88080cc080 con 0x7f8808160d20 2023-10-16T10:03:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.379+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88080d2520 con 0x7f8808160d20 2023-10-16T10:03:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.379+0000 7f87effff700 1 -- 172.21.15.181:0/843596618 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f87f8013cb0 con 0x7f8808160d20 2023-10-16T10:03:43.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.380+0000 7f87effff700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 0x7f87f406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.380+0000 7f8806ffd700 1 -- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 msgr2=0x7f87f406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:43.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.380+0000 7f87effff700 1 -- 172.21.15.181:0/843596618 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f87f8089a70 con 0x7f8808160d20 2023-10-16T10:03:43.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.380+0000 7f8806ffd700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 0x7f87f406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:43.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.385+0000 7f87effff700 1 -- 172.21.15.181:0/843596618 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f87f8054580 con 0x7f8808160d20 2023-10-16T10:03:43.402 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:43.401+0000 7f9a6d36e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:03:43.421 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:43.419+0000 7fc064740200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:03:43.542 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:43.541+0000 7f9a6d36e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:03:43.566 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:43.565+0000 7fc064740200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:03:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.581+0000 7f8806ffd700 1 -- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 msgr2=0x7f87f406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.581+0000 7f8806ffd700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 0x7f87f406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:43.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.585+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f880804ebb0 con 0x7f8808160d20 2023-10-16T10:03:43.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.586+0000 7f87effff700 1 -- 172.21.15.181:0/843596618 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17760 (secure 0 0 0) 0x7f87f8021090 con 0x7f8808160d20 2023-10-16T10:03:43.588 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:43.588 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:35.802255+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:03:20.395061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:03:20.923916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/240608457":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/3359421143":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6833/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6832/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/2638619292":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1179812996":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1546779767":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1171463515":"2023-10-17T10:03:35.802223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 msgr2=0x7f87f406d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 0x7f87f406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808160d20 msgr2=0x7f88080c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808160d20 0x7f88080c9560 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f87f800e8a0 tx=0x7f87f800ebb0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 shutdown_connections 2023-10-16T10:03:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f87f406b0e0 0x7f87f406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8808161f40 0x7f88080b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808160d20 0x7f88080c9560 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 --2- 172.21.15.181:0/843596618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8808161b60 0x7f8808160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 >> 172.21.15.181:0/843596618 conn(0x7f880809cba0 msgr2=0x7f88080ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 shutdown_connections 2023-10-16T10:03:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.589+0000 7f880e238700 1 -- 172.21.15.181:0/843596618 wait complete. 2023-10-16T10:03:43.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:03:20.395061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:03:20.923916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:43.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:43.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.767+0000 7f624d66a700 1 Processor -- start 2023-10-16T10:03:43.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.767+0000 7f624d66a700 1 -- start start 2023-10-16T10:03:43.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.767+0000 7f624d66a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.767+0000 7f624d66a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f624806b6f0 0x7f624806bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.767+0000 7f624d66a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624815dfb0 con 0x7f624815dbd0 2023-10-16T10:03:43.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.767+0000 7f624d66a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62480ce740 con 0x7f624806b6f0 2023-10-16T10:03:43.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.767+0000 7f624d66a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480cd3a0 con 0x7f62480cdd80 2023-10-16T10:03:43.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f62477fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f624806b6f0 0x7f624806bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f6246ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f624815dbd0 0x7f62480d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59494/0 (socket says 172.21.15.181:59494) 2023-10-16T10:03:43.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f6246ffd700 1 -- 172.21.15.181:0/1794388277 learned_addr learned my addr 172.21.15.181:0/1794388277 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:43.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f62467fc700 1 --2- 172.21.15.181:0/1794388277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f62477fe700 1 -- 172.21.15.181:0/1794388277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 msgr2=0x7f62480ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:43.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f62477fe700 1 --2- 172.21.15.181:0/1794388277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f62477fe700 1 -- 172.21.15.181:0/1794388277 >> v1:172.21.15.181:6789/0 conn(0x7f624815dbd0 legacy=0x7f62480d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f6245ffb700 1 -- 172.21.15.181:0/1794388277 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212403036 0 0) 0x7f6238001110 con 0x7f624815dbd0 2023-10-16T10:03:43.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f62477fe700 1 -- 172.21.15.181:0/1794388277 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62480cd830 con 0x7f624806b6f0 2023-10-16T10:03:43.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f62477fe700 1 --2- 172.21.15.181:0/1794388277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f624806bb70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6240009d90 tx=0x7f624000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9980c8054958b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:43.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f6245ffb700 1 -- 172.21.15.181:0/1794388277 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000e070 con 0x7f624806b6f0 2023-10-16T10:03:43.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f6245ffb700 1 -- 172.21.15.181:0/1794388277 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624000bb20 con 0x7f624806b6f0 2023-10-16T10:03:43.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.768+0000 7f6245ffb700 1 -- 172.21.15.181:0/1794388277 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240012430 con 0x7f624806b6f0 2023-10-16T10:03:43.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 -- 172.21.15.181:0/1794388277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 msgr2=0x7f624806bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 --2- 172.21.15.181:0/1794388277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f624806bb70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6240009d90 tx=0x7f624000b330 comp rx=0 tx=0).stop 2023-10-16T10:03:43.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 -- 172.21.15.181:0/1794388277 shutdown_connections 2023-10-16T10:03:43.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 --2- 172.21.15.181:0/1794388277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f624806bb70 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 --2- 172.21.15.181:0/1794388277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 -- 172.21.15.181:0/1794388277 >> 172.21.15.181:0/1794388277 conn(0x7f624809a930 msgr2=0x7f62480d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:43.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 -- 172.21.15.181:0/1794388277 shutdown_connections 2023-10-16T10:03:43.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.769+0000 7f624d66a700 1 -- 172.21.15.181:0/1794388277 wait complete. 2023-10-16T10:03:43.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.771+0000 7f624d66a700 1 Processor -- start 2023-10-16T10:03:43.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.771+0000 7f624d66a700 1 -- start start 2023-10-16T10:03:43.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f624d66a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f6248160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f624d66a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f624d66a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f624815dbd0 0x7f62480b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f624d66a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480c9720 con 0x7f624806b6f0 2023-10-16T10:03:43.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f624d66a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624809f380 con 0x7f624815dbd0 2023-10-16T10:03:43.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f624d66a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480c9970 con 0x7f62480cdd80 2023-10-16T10:03:43.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f6246ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f6248160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f6246ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f6248160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48422/0 (socket says 172.21.15.181:48422) 2023-10-16T10:03:43.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.772+0000 7f6246ffd700 1 -- 172.21.15.181:0/1973021154 learned_addr learned my addr 172.21.15.181:0/1973021154 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:43.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f62467fc700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f62477fe700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f624815dbd0 0x7f62480b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:43.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f6246ffd700 1 -- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 msgr2=0x7f62480caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f6246ffd700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f6246ffd700 1 -- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f624815dbd0 msgr2=0x7f62480b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f6246ffd700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f624815dbd0 0x7f62480b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f6246ffd700 1 -- 172.21.15.181:0/1973021154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240009a70 con 0x7f624806b6f0 2023-10-16T10:03:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f6246ffd700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f6248160ac0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f623800c730 tx=0x7f623800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f622f7fe700 1 -- 172.21.15.181:0/1973021154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238004c80 con 0x7f624806b6f0 2023-10-16T10:03:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f622f7fe700 1 -- 172.21.15.181:0/1973021154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238004e00 con 0x7f624806b6f0 2023-10-16T10:03:43.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62480c9c50 con 0x7f624806b6f0 2023-10-16T10:03:43.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f622f7fe700 1 -- 172.21.15.181:0/1973021154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238007310 con 0x7f624806b6f0 2023-10-16T10:03:43.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.773+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62480ca210 con 0x7f624806b6f0 2023-10-16T10:03:43.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.774+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6248067d20 con 0x7f624806b6f0 2023-10-16T10:03:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.775+0000 7f622f7fe700 1 -- 172.21.15.181:0/1973021154 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f6238007470 con 0x7f624806b6f0 2023-10-16T10:03:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.775+0000 7f622f7fe700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 0x7f623406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.775+0000 7f62467fc700 1 -- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 msgr2=0x7f623406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.775+0000 7f622f7fe700 1 -- 172.21.15.181:0/1973021154 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f62380896d0 con 0x7f624806b6f0 2023-10-16T10:03:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.775+0000 7f62467fc700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 0x7f623406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.779+0000 7f622f7fe700 1 -- 172.21.15.181:0/1973021154 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f62380541e0 con 0x7f624806b6f0 2023-10-16T10:03:43.868 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:43.867+0000 7f9a6d36e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:03:43.909 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:43.908+0000 7fc064740200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:03:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.976+0000 7f62467fc700 1 -- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 msgr2=0x7f623406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.976+0000 7f62467fc700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 0x7f623406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.976+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f624815dfb0 con 0x7f624806b6f0 2023-10-16T10:03:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.976+0000 7f622f7fe700 1 -- 172.21.15.181:0/1973021154 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f624815dfb0 con 0x7f624806b6f0 2023-10-16T10:03:43.979 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.978+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 msgr2=0x7f623406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.978+0000 7f624d66a700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 0x7f623406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.978+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 msgr2=0x7f6248160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.978+0000 7f624d66a700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f6248160ac0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f623800c730 tx=0x7f623800ca40 comp rx=0 tx=0).stop 2023-10-16T10:03:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 shutdown_connections 2023-10-16T10:03:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f623406b0b0 0x7f623406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f624815dbd0 0x7f62480b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f624806b6f0 0x7f6248160ac0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 --2- 172.21.15.181:0/1973021154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62480cdd80 0x7f62480caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 >> 172.21.15.181:0/1973021154 conn(0x7f624809a930 msgr2=0x7f6248070380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 shutdown_connections 2023-10-16T10:03:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:43.979+0000 7f624d66a700 1 -- 172.21.15.181:0/1973021154 wait complete. 2023-10-16T10:03:43.985 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:43.984+0000 7f9a6d36e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:03:43.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:03:44.031 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:44.029+0000 7fc064740200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:03:44.059 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:44.058+0000 7f9a6d36e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:03:44.109 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:44.107+0000 7fc064740200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:03:44.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.119+0000 7f7db738f700 1 Processor -- start 2023-10-16T10:03:44.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.120+0000 7f7db738f700 1 -- start start 2023-10-16T10:03:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db738f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7db0072760 0x7f7db006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db738f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7db006ade0 0x7f7db01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db738f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db006b210 con 0x7f7db015da70 2023-10-16T10:03:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db738f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00cef00 con 0x7f7db006ade0 2023-10-16T10:03:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db738f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00cec20 con 0x7f7db0072760 2023-10-16T10:03:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db592c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7db006ade0 0x7f7db01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db512b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7db015da70 0x7f7db00ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59510/0 (socket says 172.21.15.181:59510) 2023-10-16T10:03:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db512b700 1 -- 172.21.15.181:0/2217528602 learned_addr learned my addr 172.21.15.181:0/2217528602 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.121+0000 7f7db492a700 1 --2- 172.21.15.181:0/2217528602 >> v2:172.21.15.190:3300/0 conn(0x7f7db0072760 0x7f7db006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7db592c700 1 -- 172.21.15.181:0/2217528602 >> v2:172.21.15.190:3300/0 conn(0x7f7db0072760 msgr2=0x7f7db006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7db592c700 1 --2- 172.21.15.181:0/2217528602 >> v2:172.21.15.190:3300/0 conn(0x7f7db0072760 0x7f7db006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7db592c700 1 -- 172.21.15.181:0/2217528602 >> v1:172.21.15.181:6789/0 conn(0x7f7db015da70 legacy=0x7f7db00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7da7fff700 1 -- 172.21.15.181:0/2217528602 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035397739 0 0) 0x7f7db006b210 con 0x7f7db015da70 2023-10-16T10:03:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7db592c700 1 -- 172.21.15.181:0/2217528602 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db0071c80 con 0x7f7db006ade0 2023-10-16T10:03:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7db592c700 1 --2- 172.21.15.181:0/2217528602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006ade0 0x7f7db01615f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7db006ef60 tx=0x7f7da800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcfe86d820a9efa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7da7fff700 1 -- 172.21.15.181:0/2217528602 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da800e070 con 0x7f7db006ade0 2023-10-16T10:03:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7da7fff700 1 -- 172.21.15.181:0/2217528602 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da800bb20 con 0x7f7db006ade0 2023-10-16T10:03:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.122+0000 7f7da7fff700 1 -- 172.21.15.181:0/2217528602 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8012430 con 0x7f7db006ade0 2023-10-16T10:03:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 -- 172.21.15.181:0/2217528602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006ade0 msgr2=0x7f7db01615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 --2- 172.21.15.181:0/2217528602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006ade0 0x7f7db01615f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7db006ef60 tx=0x7f7da800b330 comp rx=0 tx=0).stop 2023-10-16T10:03:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 -- 172.21.15.181:0/2217528602 shutdown_connections 2023-10-16T10:03:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 --2- 172.21.15.181:0/2217528602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006ade0 0x7f7db01615f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 --2- 172.21.15.181:0/2217528602 >> v2:172.21.15.190:3300/0 conn(0x7f7db0072760 0x7f7db006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 -- 172.21.15.181:0/2217528602 >> 172.21.15.181:0/2217528602 conn(0x7f7db00cd010 msgr2=0x7f7db00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 -- 172.21.15.181:0/2217528602 shutdown_connections 2023-10-16T10:03:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.123+0000 7f7db738f700 1 -- 172.21.15.181:0/2217528602 wait complete. 2023-10-16T10:03:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.124+0000 7f7db738f700 1 Processor -- start 2023-10-16T10:03:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.124+0000 7f7db738f700 1 -- start start 2023-10-16T10:03:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db738f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db006ade0 0x7f7db00c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db738f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0072760 0x7f7db01f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db738f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db015da70 0x7f7db00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db738f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0151270 con 0x7f7db015da70 2023-10-16T10:03:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db738f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00c6300 con 0x7f7db006ade0 2023-10-16T10:03:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db738f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00c5d20 con 0x7f7db0072760 2023-10-16T10:03:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db592c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db015da70 0x7f7db00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db592c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db015da70 0x7f7db00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48442/0 (socket says 172.21.15.181:48442) 2023-10-16T10:03:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db592c700 1 -- 172.21.15.181:0/2530811958 learned_addr learned my addr 172.21.15.181:0/2530811958 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db492a700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0072760 0x7f7db01f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.125+0000 7f7db512b700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db006ade0 0x7f7db00c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db592c700 1 -- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0072760 msgr2=0x7f7db01f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db592c700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0072760 0x7f7db01f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db592c700 1 -- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db006ade0 msgr2=0x7f7db00c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db592c700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db006ade0 0x7f7db00c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db592c700 1 -- 172.21.15.181:0/2530811958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8009a70 con 0x7f7db015da70 2023-10-16T10:03:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db592c700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db015da70 0x7f7db00b4610 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7da800b680 tx=0x7f7da8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7da5ffb700 1 -- 172.21.15.181:0/2530811958 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8012430 con 0x7f7db015da70 2023-10-16T10:03:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7da5ffb700 1 -- 172.21.15.181:0/2530811958 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da8020070 con 0x7f7db015da70 2023-10-16T10:03:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db00b4e80 con 0x7f7db015da70 2023-10-16T10:03:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7da5ffb700 1 -- 172.21.15.181:0/2530811958 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8004280 con 0x7f7db015da70 2023-10-16T10:03:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.126+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db01f30b0 con 0x7f7db015da70 2023-10-16T10:03:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.127+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db00c6440 con 0x7f7db015da70 2023-10-16T10:03:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.128+0000 7f7da5ffb700 1 -- 172.21.15.181:0/2530811958 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f7da80043e0 con 0x7f7db015da70 2023-10-16T10:03:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.128+0000 7f7da5ffb700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 0x7f7d9406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.128+0000 7f7db512b700 1 -- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 msgr2=0x7f7d9406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:44.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.128+0000 7f7da5ffb700 1 -- 172.21.15.181:0/2530811958 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f7da8022070 con 0x7f7db015da70 2023-10-16T10:03:44.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.128+0000 7f7db512b700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 0x7f7d9406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:44.140 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:44.138+0000 7f9a6d36e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:03:44.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.133+0000 7f7da5ffb700 1 -- 172.21.15.181:0/2530811958 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7da8056710 con 0x7f7db015da70 2023-10-16T10:03:44.195 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:44.193+0000 7fc064740200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:03:44.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.329+0000 7f7db512b700 1 -- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 msgr2=0x7f7d9406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.329+0000 7f7db512b700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 0x7f7d9406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:44.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.330+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7db00afff0 con 0x7f7db015da70 2023-10-16T10:03:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.330+0000 7f7da5ffb700 1 -- 172.21.15.181:0/2530811958 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7da8059d30 con 0x7f7db015da70 2023-10-16T10:03:44.332 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:03:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 msgr2=0x7f7d9406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 0x7f7d9406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db015da70 msgr2=0x7f7db00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db015da70 0x7f7db00b4610 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7da800b680 tx=0x7f7da8004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 shutdown_connections 2023-10-16T10:03:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d9406b0b0 0x7f7d9406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db015da70 0x7f7db00b4610 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db006ade0 0x7f7db00c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 --2- 172.21.15.181:0/2530811958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db0072760 0x7f7db01f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 >> 172.21.15.181:0/2530811958 conn(0x7f7db00cd010 msgr2=0x7f7db00ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 shutdown_connections 2023-10-16T10:03:44.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.333+0000 7f7db738f700 1 -- 172.21.15.181:0/2530811958 wait complete. 2023-10-16T10:03:44.348 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:03:44.509 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:44.508+0000 7f9a6d36e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:03:44.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.514+0000 7fa52e53b700 1 Processor -- start 2023-10-16T10:03:44.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.514+0000 7fa52e53b700 1 -- start start 2023-10-16T10:03:44.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52e53b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa5280d01d0 0x7fa5280ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52e53b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52e53b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa528161530 0x7fa5281619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52e53b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5280d4290 con 0x7fa528161530 2023-10-16T10:03:44.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52e53b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5280cf580 con 0x7fa5280d01d0 2023-10-16T10:03:44.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52e53b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280d0af0 con 0x7fa52815ea40 2023-10-16T10:03:44.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa528161530 0x7fa5281619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa528161530 0x7fa5281619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48486/0 (socket says 172.21.15.181:48486) 2023-10-16T10:03:44.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 -- 172.21.15.181:0/20502346 learned_addr learned my addr 172.21.15.181:0/20502346 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:44.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa5277fe700 1 --2- 172.21.15.181:0/20502346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa527fff700 1 --2- 172.21.15.181:0/20502346 >> v2:172.21.15.190:3300/0 conn(0x7fa5280d01d0 0x7fa5280ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 -- 172.21.15.181:0/20502346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 msgr2=0x7fa5280d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:44.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 --2- 172.21.15.181:0/20502346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 -- 172.21.15.181:0/20502346 >> v2:172.21.15.190:3300/0 conn(0x7fa5280d01d0 msgr2=0x7fa5280ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:44.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 --2- 172.21.15.181:0/20502346 >> v2:172.21.15.190:3300/0 conn(0x7fa5280d01d0 0x7fa5280ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 -- 172.21.15.181:0/20502346 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5280ce970 con 0x7fa528161530 2023-10-16T10:03:44.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa52cad8700 1 --2- 172.21.15.181:0/20502346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa5281619c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa52808e050 tx=0x7fa52000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a323954a5dd2939 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa526ffd700 1 -- 172.21.15.181:0/20502346 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000e070 con 0x7fa528161530 2023-10-16T10:03:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa526ffd700 1 -- 172.21.15.181:0/20502346 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52000b910 con 0x7fa528161530 2023-10-16T10:03:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.515+0000 7fa526ffd700 1 -- 172.21.15.181:0/20502346 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52000bcc0 con 0x7fa528161530 2023-10-16T10:03:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 -- 172.21.15.181:0/20502346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 msgr2=0x7fa5281619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 --2- 172.21.15.181:0/20502346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa5281619c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa52808e050 tx=0x7fa52000b330 comp rx=0 tx=0).stop 2023-10-16T10:03:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 -- 172.21.15.181:0/20502346 shutdown_connections 2023-10-16T10:03:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 --2- 172.21.15.181:0/20502346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa5281619c0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 --2- 172.21.15.181:0/20502346 >> v2:172.21.15.190:3300/0 conn(0x7fa5280d01d0 0x7fa5280ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 --2- 172.21.15.181:0/20502346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 -- 172.21.15.181:0/20502346 >> 172.21.15.181:0/20502346 conn(0x7fa52809cb60 msgr2=0x7fa5280d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:44.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 -- 172.21.15.181:0/20502346 shutdown_connections 2023-10-16T10:03:44.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.516+0000 7fa52e53b700 1 -- 172.21.15.181:0/20502346 wait complete. 2023-10-16T10:03:44.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.517+0000 7fa52e53b700 1 Processor -- start 2023-10-16T10:03:44.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52e53b700 1 -- start start 2023-10-16T10:03:44.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52e53b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5280d01d0 0x7fa5280c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52e53b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52e53b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa52815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52e53b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280b5210 con 0x7fa528161530 2023-10-16T10:03:44.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52e53b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280b54e0 con 0x7fa5280d01d0 2023-10-16T10:03:44.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52e53b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5280b57c0 con 0x7fa52815ea40 2023-10-16T10:03:44.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52cad8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa52815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.518+0000 7fa52cad8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa52815ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48492/0 (socket says 172.21.15.181:48492) 2023-10-16T10:03:44.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52cad8700 1 -- 172.21.15.181:0/475564803 learned_addr learned my addr 172.21.15.181:0/475564803 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:44.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa527fff700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5280d01d0 0x7fa5280c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa5277fe700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52cad8700 1 -- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 msgr2=0x7fa5280b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52cad8700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52cad8700 1 -- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5280d01d0 msgr2=0x7fa5280c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52cad8700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5280d01d0 0x7fa5280c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52cad8700 1 -- 172.21.15.181:0/475564803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520009b30 con 0x7fa528161530 2023-10-16T10:03:44.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa527fff700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5280d01d0 0x7fa5280c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:44.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa5277fe700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:44.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52cad8700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa52815ac40 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa52808cf30 tx=0x7fa520004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:44.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa524ff9700 1 -- 172.21.15.181:0/475564803 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520004450 con 0x7fa528161530 2023-10-16T10:03:44.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa524ff9700 1 -- 172.21.15.181:0/475564803 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5200045d0 con 0x7fa528161530 2023-10-16T10:03:44.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5280b5aa0 con 0x7fa528161530 2023-10-16T10:03:44.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa524ff9700 1 -- 172.21.15.181:0/475564803 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5200099b0 con 0x7fa528161530 2023-10-16T10:03:44.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.519+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52815b180 con 0x7fa528161530 2023-10-16T10:03:44.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.520+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52804ebb0 con 0x7fa528161530 2023-10-16T10:03:44.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.521+0000 7fa524ff9700 1 -- 172.21.15.181:0/475564803 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fa52001c410 con 0x7fa528161530 2023-10-16T10:03:44.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.521+0000 7fa524ff9700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 0x7fa51406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.521+0000 7fa527fff700 1 -- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 msgr2=0x7fa51406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:44.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.521+0000 7fa524ff9700 1 -- 172.21.15.181:0/475564803 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7fa52008bb70 con 0x7fa528161530 2023-10-16T10:03:44.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.521+0000 7fa527fff700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 0x7fa51406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:44.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.526+0000 7fa524ff9700 1 -- 172.21.15.181:0/475564803 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa520056600 con 0x7fa528161530 2023-10-16T10:03:44.584 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:44.582+0000 7fc064740200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:03:44.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.722+0000 7fa527fff700 1 -- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 msgr2=0x7fa51406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:44.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.722+0000 7fa527fff700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 0x7fa51406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:44.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.725+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5280d0e00 con 0x7fa528161530 2023-10-16T10:03:44.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.726+0000 7fa524ff9700 1 -- 172.21.15.181:0/475564803 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5200200a0 con 0x7fa528161530 2023-10-16T10:03:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:03:44.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 msgr2=0x7fa51406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:44.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 0x7fa51406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 msgr2=0x7fa52815ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa52815ac40 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa52808cf30 tx=0x7fa520004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 shutdown_connections 2023-10-16T10:03:44.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fa51406b0b0 0x7fa51406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa528161530 0x7fa52815ac40 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5280d01d0 0x7fa5280c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 --2- 172.21.15.181:0/475564803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa52815ea40 0x7fa5280b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 >> 172.21.15.181:0/475564803 conn(0x7fa52809cb60 msgr2=0x7fa52815f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:44.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 shutdown_connections 2023-10-16T10:03:44.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.728+0000 7fa52e53b700 1 -- 172.21.15.181:0/475564803 wait complete. 2023-10-16T10:03:44.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T10:03:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.863+0000 7eff6ceae700 1 Processor -- start 2023-10-16T10:03:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.864+0000 7eff6ceae700 1 -- start start 2023-10-16T10:03:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.864+0000 7eff6ceae700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff680cfa60 0x7eff680d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.864+0000 7eff6ceae700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff680d26b0 0x7eff680b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff6ceae700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff680d2b00 con 0x7eff6815dda0 2023-10-16T10:03:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff6ceae700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff680d0940 con 0x7eff680cfa60 2023-10-16T10:03:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff6ceae700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680d0c20 con 0x7eff680d26b0 2023-10-16T10:03:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff65d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff680cfa60 0x7eff680d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff66d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff680d26b0 0x7eff680b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff6659c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7eff6815dda0 0x7eff680d0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:43518/0 (socket says 172.21.15.181:43518) 2023-10-16T10:03:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff65d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff680cfa60 0x7eff680d2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:53550/0 (socket says 172.21.15.181:53550) 2023-10-16T10:03:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff6659c700 1 -- 172.21.15.181:0/2111362897 learned_addr learned my addr 172.21.15.181:0/2111362897 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff66d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff680d26b0 0x7eff680b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:45528/0 (socket says 172.21.15.181:45528) 2023-10-16T10:03:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff6559a700 1 -- 172.21.15.181:0/2111362897 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185023818 0 0) 0x7eff58001160 con 0x7eff6815dda0 2023-10-16T10:03:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff66d9d700 1 -- 172.21.15.181:0/2111362897 >> v2:172.21.15.190:3300/0 conn(0x7eff680cfa60 msgr2=0x7eff680d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff66d9d700 1 --2- 172.21.15.181:0/2111362897 >> v2:172.21.15.190:3300/0 conn(0x7eff680cfa60 0x7eff680d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff66d9d700 1 -- 172.21.15.181:0/2111362897 >> v1:172.21.15.190:6789/0 conn(0x7eff6815dda0 legacy=0x7eff680d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff66d9d700 1 -- 172.21.15.181:0/2111362897 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680b2c20 con 0x7eff680d26b0 2023-10-16T10:03:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff65d9b700 1 --2- 172.21.15.181:0/2111362897 >> v2:172.21.15.190:3300/0 conn(0x7eff680cfa60 0x7eff680d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.865+0000 7eff66d9d700 1 --2- 172.21.15.181:0/2111362897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff680d26b0 0x7eff680b3700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff60009d90 tx=0x7eff6000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576b8d7b4e7e7878 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6559a700 1 -- 172.21.15.181:0/2111362897 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000e070 con 0x7eff680d26b0 2023-10-16T10:03:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6559a700 1 -- 172.21.15.181:0/2111362897 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6000bb20 con 0x7eff680d26b0 2023-10-16T10:03:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6559a700 1 -- 172.21.15.181:0/2111362897 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60012430 con 0x7eff680d26b0 2023-10-16T10:03:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 -- 172.21.15.181:0/2111362897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff680d26b0 msgr2=0x7eff680b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 --2- 172.21.15.181:0/2111362897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff680d26b0 0x7eff680b3700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff60009d90 tx=0x7eff6000b330 comp rx=0 tx=0).stop 2023-10-16T10:03:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 -- 172.21.15.181:0/2111362897 shutdown_connections 2023-10-16T10:03:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 --2- 172.21.15.181:0/2111362897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff680d26b0 0x7eff680b3700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 --2- 172.21.15.181:0/2111362897 >> v2:172.21.15.190:3300/0 conn(0x7eff680cfa60 0x7eff680d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 -- 172.21.15.181:0/2111362897 >> 172.21.15.181:0/2111362897 conn(0x7eff6809cb10 msgr2=0x7eff680d3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:44.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 -- 172.21.15.181:0/2111362897 shutdown_connections 2023-10-16T10:03:44.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.866+0000 7eff6ceae700 1 -- 172.21.15.181:0/2111362897 wait complete. 2023-10-16T10:03:44.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 Processor -- start 2023-10-16T10:03:44.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 -- start start 2023-10-16T10:03:44.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff680cfa60 0x7eff680cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff680d26b0 0x7eff68160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff6815dda0 0x7eff68157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff680cb850 con 0x7eff680d26b0 2023-10-16T10:03:44.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff680cc5c0 con 0x7eff680cfa60 2023-10-16T10:03:44.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6ceae700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff68158f40 con 0x7eff6815dda0 2023-10-16T10:03:44.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff65d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff680d26b0 0x7eff68160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff6659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff680cfa60 0x7eff680cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff65d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff680d26b0 0x7eff68160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48506/0 (socket says 172.21.15.181:48506) 2023-10-16T10:03:44.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff65d9b700 1 -- 172.21.15.181:0/1078617760 learned_addr learned my addr 172.21.15.181:0/1078617760 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:44.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.868+0000 7eff66d9d700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff6815dda0 0x7eff68157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:44.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6659c700 1 -- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff6815dda0 msgr2=0x7eff68157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6659c700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff6815dda0 0x7eff68157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6659c700 1 -- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff680d26b0 msgr2=0x7eff68160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:44.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6659c700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff680d26b0 0x7eff68160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:44.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6659c700 1 -- 172.21.15.181:0/1078617760 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c0094a0 con 0x7eff680cfa60 2023-10-16T10:03:44.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff66d9d700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff6815dda0 0x7eff68157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:44.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6659c700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff680cfa60 0x7eff680cb310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7eff5800c490 tx=0x7eff5800c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:44.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff56ffd700 1 -- 172.21.15.181:0/1078617760 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58004cd0 con 0x7eff680cfa60 2023-10-16T10:03:44.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff56ffd700 1 -- 172.21.15.181:0/1078617760 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff58004e50 con 0x7eff680cfa60 2023-10-16T10:03:44.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff56ffd700 1 -- 172.21.15.181:0/1078617760 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58007310 con 0x7eff680cfa60 2023-10-16T10:03:44.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff60009a70 con 0x7eff680cfa60 2023-10-16T10:03:44.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.869+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff680cbd20 con 0x7eff680cfa60 2023-10-16T10:03:44.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.870+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6804ebb0 con 0x7eff680cfa60 2023-10-16T10:03:44.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.871+0000 7eff56ffd700 1 -- 172.21.15.181:0/1078617760 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7eff58007470 con 0x7eff680cfa60 2023-10-16T10:03:44.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.871+0000 7eff56ffd700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 0x7eff4806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:44.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.871+0000 7eff65d9b700 1 -- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 msgr2=0x7eff4806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:44.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.871+0000 7eff56ffd700 1 -- 172.21.15.181:0/1078617760 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7eff580894e0 con 0x7eff680cfa60 2023-10-16T10:03:44.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.871+0000 7eff65d9b700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 0x7eff4806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:44.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:44.878+0000 7eff56ffd700 1 -- 172.21.15.181:0/1078617760 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7eff58054080 con 0x7eff680cfa60 2023-10-16T10:03:45.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.072+0000 7eff65d9b700 1 -- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 msgr2=0x7eff4806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:45.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.072+0000 7eff65d9b700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 0x7eff4806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:45.079 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:45.078+0000 7f9a6d36e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:03:45.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.130+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff680d2b00 con 0x7eff680cfa60 2023-10-16T10:03:45.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.130+0000 7eff56ffd700 1 -- 172.21.15.181:0/1078617760 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v133) v1 ==== 80+0+477 (secure 0 0 0) 0x7eff680d2b00 con 0x7eff680cfa60 2023-10-16T10:03:45.132 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/240608457 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.133 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3359421143 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.133 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3280351398 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.133 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3280351398 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.133 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2638619292 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.133 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1179812996 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.134 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1546779767 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.134 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1171463515 2023-10-17T10:03:35.802223+0000 2023-10-16T10:03:45.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 msgr2=0x7eff4806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:45.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 0x7eff4806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff680cfa60 msgr2=0x7eff680cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff680cfa60 0x7eff680cb310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7eff5800c490 tx=0x7eff5800c7a0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 shutdown_connections 2023-10-16T10:03:45.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7eff4806b0b0 0x7eff4806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff6815dda0 0x7eff68157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff680cfa60 0x7eff680cb310 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 --2- 172.21.15.181:0/1078617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff680d26b0 0x7eff68160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 >> 172.21.15.181:0/1078617760 conn(0x7eff6809cb10 msgr2=0x7eff6806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 shutdown_connections 2023-10-16T10:03:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.133+0000 7eff6ceae700 1 -- 172.21.15.181:0/1078617760 wait complete. 2023-10-16T10:03:45.138 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T10:03:45.148 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:45.147+0000 7f9a6d36e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:03:45.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T10:03:45.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:45.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T10:03:45.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T10:03:45.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:03:45.193 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T10:03:45.194 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:45.190+0000 7fc064740200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:03:45.248 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:45.249 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:45.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:45.266 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:45.267 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:45.268 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:45.264+0000 7fc064740200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:03:45.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:03:45.284 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:45.285 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:45.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:45.323 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:45.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.445+0000 7f7d35675700 1 Processor -- start 2023-10-16T10:03:45.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.445+0000 7f7d35675700 1 -- start start 2023-10-16T10:03:45.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d35675700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d35675700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300cc510 con 0x7f7d300d0370 2023-10-16T10:03:45.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d35675700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300cc7f0 con 0x7f7d300cd2d0 2023-10-16T10:03:45.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d35675700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300ccad0 con 0x7f7d3015b990 2023-10-16T10:03:45.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7d300d0370 0x7f7d300ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59596/0 (socket says 172.21.15.181:59596) 2023-10-16T10:03:45.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2f7fe700 1 -- 172.21.15.181:0/1444651731 learned_addr learned my addr 172.21.15.181:0/1444651731 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:45.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2effd700 1 --2- 172.21.15.181:0/1444651731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2dffb700 1 -- 172.21.15.181:0/1444651731 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636737378 0 0) 0x7f7d300cc510 con 0x7f7d300d0370 2023-10-16T10:03:45.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2dffb700 1 -- 172.21.15.181:0/1444651731 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d180035b0 con 0x7f7d300d0370 2023-10-16T10:03:45.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2dffb700 1 -- 172.21.15.181:0/1444651731 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869725257 0 0) 0x7f7d300cc7f0 con 0x7f7d300cd2d0 2023-10-16T10:03:45.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2dffb700 1 -- 172.21.15.181:0/1444651731 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d300cc510 con 0x7f7d300cd2d0 2023-10-16T10:03:45.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2effd700 1 -- 172.21.15.181:0/1444651731 >> v1:172.21.15.190:6789/0 conn(0x7f7d300cd2d0 legacy=0x7f7d300cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2effd700 1 -- 172.21.15.181:0/1444651731 >> v1:172.21.15.181:6789/0 conn(0x7f7d300d0370 legacy=0x7f7d300ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.446+0000 7f7d2effd700 1 -- 172.21.15.181:0/1444651731 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300ccdb0 con 0x7f7d3015b990 2023-10-16T10:03:45.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d2effd700 1 --2- 172.21.15.181:0/1444651731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300ce8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009850 tx=0x7f7d20009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aaf45f6e777fb6c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:45.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d2dffb700 1 -- 172.21.15.181:0/1444651731 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20005e30 con 0x7f7d3015b990 2023-10-16T10:03:45.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d2dffb700 1 -- 172.21.15.181:0/1444651731 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20006030 con 0x7f7d3015b990 2023-10-16T10:03:45.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d2dffb700 1 -- 172.21.15.181:0/1444651731 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200063e0 con 0x7f7d3015b990 2023-10-16T10:03:45.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d35675700 1 -- 172.21.15.181:0/1444651731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 msgr2=0x7f7d300ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d35675700 1 --2- 172.21.15.181:0/1444651731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300ce8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009850 tx=0x7f7d20009b60 comp rx=0 tx=0).stop 2023-10-16T10:03:45.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d35675700 1 -- 172.21.15.181:0/1444651731 shutdown_connections 2023-10-16T10:03:45.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d35675700 1 --2- 172.21.15.181:0/1444651731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300ce8f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d35675700 1 -- 172.21.15.181:0/1444651731 >> 172.21.15.181:0/1444651731 conn(0x7f7d3009a9a0 msgr2=0x7f7d300d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:45.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d35675700 1 -- 172.21.15.181:0/1444651731 shutdown_connections 2023-10-16T10:03:45.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.447+0000 7f7d35675700 1 -- 172.21.15.181:0/1444651731 wait complete. 2023-10-16T10:03:45.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.449+0000 7f7d35675700 1 Processor -- start 2023-10-16T10:03:45.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.449+0000 7f7d35675700 1 -- start start 2023-10-16T10:03:45.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.449+0000 7f7d35675700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d300cd2d0 0x7f7d301f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.449+0000 7f7d35675700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d300d0370 0x7f7d300b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.449+0000 7f7d35675700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d35675700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b74a0 con 0x7f7d300cd2d0 2023-10-16T10:03:45.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d35675700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b6b70 con 0x7f7d300d0370 2023-10-16T10:03:45.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d35675700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b6e50 con 0x7f7d3015b990 2023-10-16T10:03:45.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d300cd2d0 0x7f7d301f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d300cd2d0 0x7f7d301f07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48520/0 (socket says 172.21.15.181:48520) 2023-10-16T10:03:45.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 -- 172.21.15.181:0/2947700957 learned_addr learned my addr 172.21.15.181:0/2947700957 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:45.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2e7fc700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d300d0370 0x7f7d300b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2f7fe700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 -- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 msgr2=0x7f7d300b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 -- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d300d0370 msgr2=0x7f7d300b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d300d0370 0x7f7d300b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 -- 172.21.15.181:0/2947700957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009430 con 0x7f7d300cd2d0 2023-10-16T10:03:45.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d2effd700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d300cd2d0 0x7f7d301f07a0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7d20000c00 tx=0x7f7d20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:45.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d177fe700 1 -- 172.21.15.181:0/2947700957 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20006790 con 0x7f7d300cd2d0 2023-10-16T10:03:45.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d300b7130 con 0x7f7d300cd2d0 2023-10-16T10:03:45.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d300b47e0 con 0x7f7d300cd2d0 2023-10-16T10:03:45.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d177fe700 1 -- 172.21.15.181:0/2947700957 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20020070 con 0x7f7d300cd2d0 2023-10-16T10:03:45.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.450+0000 7f7d177fe700 1 -- 172.21.15.181:0/2947700957 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20025430 con 0x7f7d300cd2d0 2023-10-16T10:03:45.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.451+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d100052f0 con 0x7f7d300cd2d0 2023-10-16T10:03:45.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.452+0000 7f7d177fe700 1 -- 172.21.15.181:0/2947700957 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f7d20022030 con 0x7f7d300cd2d0 2023-10-16T10:03:45.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.452+0000 7f7d177fe700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 0x7f7d1c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.452+0000 7f7d2e7fc700 1 -- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 msgr2=0x7f7d1c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:45.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.452+0000 7f7d177fe700 1 -- 172.21.15.181:0/2947700957 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f7d2008b150 con 0x7f7d300cd2d0 2023-10-16T10:03:45.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.452+0000 7f7d2e7fc700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 0x7f7d1c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:45.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.457+0000 7f7d177fe700 1 -- 172.21.15.181:0/2947700957 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7d20055be0 con 0x7f7d300cd2d0 2023-10-16T10:03:45.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.653+0000 7f7d2e7fc700 1 -- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 msgr2=0x7f7d1c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:45.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.653+0000 7f7d2e7fc700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 0x7f7d1c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:45.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.705+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d100055a0 con 0x7f7d300cd2d0 2023-10-16T10:03:45.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.706+0000 7f7d177fe700 1 -- 172.21.15.181:0/2947700957 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f7d20059200 con 0x7f7d300cd2d0 2023-10-16T10:03:45.708 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:45.708 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2508916691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2508916691},{"type":"v1","addr":"172.21.15.181:6836","nonce":2508916691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7687,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1303363384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1303363384},{"type":"v1","addr":"172.21.15.190:6834","nonce":1303363384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/300739418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":300739418},{"type":"v1","addr":"172.21.15.181:6837","nonce":300739418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:03:21.971792+0000","modified":"2023-10-16T10:03:24.995974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1716409851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1716409851},{"type":"v1","addr":"172.21.15.190:6835","nonce":1716409851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:03:45.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 msgr2=0x7f7d1c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:45.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 0x7f7d1c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d300cd2d0 msgr2=0x7f7d301f07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d300cd2d0 0x7f7d301f07a0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7d20000c00 tx=0x7f7d20004750 comp rx=0 tx=0).stop 2023-10-16T10:03:45.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 shutdown_connections 2023-10-16T10:03:45.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f7d1c06b0b0 0x7f7d1c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d3015b990 0x7f7d300b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d300cd2d0 0x7f7d301f07a0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 --2- 172.21.15.181:0/2947700957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d300d0370 0x7f7d300b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 >> 172.21.15.181:0/2947700957 conn(0x7f7d3009a9a0 msgr2=0x7f7d300afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:45.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 shutdown_connections 2023-10-16T10:03:45.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.708+0000 7f7d35675700 1 -- 172.21.15.181:0/2947700957 wait complete. 2023-10-16T10:03:45.713 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2023-10-16T10:03:45.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:45.787 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:45.786+0000 7f9a6d36e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:45.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.849+0000 7f4c22ad8700 1 Processor -- start 2023-10-16T10:03:45.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.849+0000 7f4c22ad8700 1 -- start start 2023-10-16T10:03:45.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c22ad8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c1c0cdb80 0x7f4c1c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c22ad8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c06f870 con 0x7f4c1c0af450 2023-10-16T10:03:45.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c22ad8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c0d1160 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c22ad8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0cc8e0 con 0x7f4c1c0cdb80 2023-10-16T10:03:45.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c21075700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c1c0cdb80 0x7f4c1c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c1bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4c1c0af450 0x7f4c1c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59608/0 (socket says 172.21.15.181:59608) 2023-10-16T10:03:45.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c1bfff700 1 -- 172.21.15.181:0/2940358496 learned_addr learned my addr 172.21.15.181:0/2940358496 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:45.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c1b7fe700 1 -- 172.21.15.181:0/2940358496 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089863424 0 0) 0x7f4c1c06f870 con 0x7f4c1c0af450 2023-10-16T10:03:45.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c1b7fe700 1 -- 172.21.15.181:0/2940358496 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c0c0035b0 con 0x7f4c1c0af450 2023-10-16T10:03:45.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c21075700 1 -- 172.21.15.181:0/2940358496 >> v1:172.21.15.190:6789/0 conn(0x7f4c1c15dfd0 legacy=0x7f4c1c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c21075700 1 -- 172.21.15.181:0/2940358496 >> v1:172.21.15.181:6789/0 conn(0x7f4c1c0af450 legacy=0x7f4c1c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c21075700 1 -- 172.21.15.181:0/2940358496 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0ccbc0 con 0x7f4c1c0cdb80 2023-10-16T10:03:45.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c21075700 1 --2- 172.21.15.181:0/2940358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c0cdb80 0x7f4c1c0af9a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d90 tx=0x7f4c1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab32114a1b7152d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:45.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c1b7fe700 1 -- 172.21.15.181:0/2940358496 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400e070 con 0x7f4c1c0cdb80 2023-10-16T10:03:45.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c1b7fe700 1 -- 172.21.15.181:0/2940358496 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1400bb20 con 0x7f4c1c0cdb80 2023-10-16T10:03:45.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.850+0000 7f4c1b7fe700 1 -- 172.21.15.181:0/2940358496 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14012430 con 0x7f4c1c0cdb80 2023-10-16T10:03:45.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.851+0000 7f4c22ad8700 1 -- 172.21.15.181:0/2940358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c0cdb80 msgr2=0x7f4c1c0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.851+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/2940358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c0cdb80 0x7f4c1c0af9a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d90 tx=0x7f4c1400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:45.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.851+0000 7f4c22ad8700 1 -- 172.21.15.181:0/2940358496 shutdown_connections 2023-10-16T10:03:45.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.851+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/2940358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c0cdb80 0x7f4c1c0af9a0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.851+0000 7f4c22ad8700 1 -- 172.21.15.181:0/2940358496 >> 172.21.15.181:0/2940358496 conn(0x7f4c1c0d04a0 msgr2=0x7f4c1c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:45.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.851+0000 7f4c22ad8700 1 -- 172.21.15.181:0/2940358496 shutdown_connections 2023-10-16T10:03:45.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.851+0000 7f4c22ad8700 1 -- 172.21.15.181:0/2940358496 wait complete. 2023-10-16T10:03:45.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.852+0000 7f4c22ad8700 1 Processor -- start 2023-10-16T10:03:45.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.852+0000 7f4c22ad8700 1 -- start start 2023-10-16T10:03:45.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c22ad8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c1c0af450 0x7f4c1c1f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c22ad8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c1c0cdb80 0x7f4c1c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c22ad8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c15dfd0 0x7f4c1c0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c22ad8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0ad780 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c22ad8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c85b0 con 0x7f4c1c0af450 2023-10-16T10:03:45.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c22ad8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1f35c0 con 0x7f4c1c0cdb80 2023-10-16T10:03:45.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c21075700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c15dfd0 0x7f4c1c0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c21075700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c15dfd0 0x7f4c1c0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48540/0 (socket says 172.21.15.181:48540) 2023-10-16T10:03:45.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c20874700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c1c0af450 0x7f4c1c1f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c21075700 1 -- 172.21.15.181:0/1983871732 learned_addr learned my addr 172.21.15.181:0/1983871732 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:45.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.853+0000 7f4c1bfff700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c1c0cdb80 0x7f4c1c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:45.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c21075700 1 -- 172.21.15.181:0/1983871732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c1c0cdb80 msgr2=0x7f4c1c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c21075700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c1c0cdb80 0x7f4c1c0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c21075700 1 -- 172.21.15.181:0/1983871732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c1c0af450 msgr2=0x7f4c1c1f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c21075700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c1c0af450 0x7f4c1c1f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c21075700 1 -- 172.21.15.181:0/1983871732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14009a70 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c21075700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c15dfd0 0x7f4c1c0ae340 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4c14013000 tx=0x7f4c14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c197fa700 1 -- 172.21.15.181:0/1983871732 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14012430 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c197fa700 1 -- 172.21.15.181:0/1983871732 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14020070 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c1f3860 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c197fa700 1 -- 172.21.15.181:0/1983871732 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14004280 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.854+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1f3e20 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.855+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c067d20 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.855+0000 7f4c197fa700 1 -- 172.21.15.181:0/1983871732 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f4c140043e0 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.856+0000 7f4c197fa700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 0x7f4c0006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.856+0000 7f4c20874700 1 -- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 msgr2=0x7f4c0006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.856+0000 7f4c197fa700 1 -- 172.21.15.181:0/1983871732 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f4c14022070 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.856+0000 7f4c20874700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 0x7f4c0006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:45.866 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:45.864+0000 7f9a6d36e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:03:45.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:45.860+0000 7f4c197fa700 1 -- 172.21.15.181:0/1983871732 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c140567c0 con 0x7f4c1c15dfd0 2023-10-16T10:03:45.943 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:45.941+0000 7fc064740200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:46.009 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:46.008+0000 7f9a6d36e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:46.023 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:46.022+0000 7fc064740200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:03:46.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.057+0000 7f4c20874700 1 -- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 msgr2=0x7f4c0006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:46.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.057+0000 7f4c20874700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 0x7f4c0006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:46.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.062+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c1c0b6100 con 0x7f4c1c15dfd0 2023-10-16T10:03:46.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.064+0000 7f4c197fa700 1 -- 172.21.15.181:0/1983871732 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17760 (secure 0 0 0) 0x7f4c14059de0 con 0x7f4c1c15dfd0 2023-10-16T10:03:46.066 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:46.067 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:35.802255+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:03:20.395061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:03:20.923916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/240608457":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/3359421143":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6833/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6832/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/2638619292":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1179812996":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1546779767":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1171463515":"2023-10-17T10:03:35.802223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:46.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 msgr2=0x7f4c0006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:46.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 0x7f4c0006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c15dfd0 msgr2=0x7f4c1c0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c15dfd0 0x7f4c1c0ae340 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4c14013000 tx=0x7f4c14004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 shutdown_connections 2023-10-16T10:03:46.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f4c0006b0b0 0x7f4c0006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1c15dfd0 0x7f4c1c0ae340 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c1c0af450 0x7f4c1c1f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 --2- 172.21.15.181:0/1983871732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c1c0cdb80 0x7f4c1c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 >> 172.21.15.181:0/1983871732 conn(0x7f4c1c0d04a0 msgr2=0x7f4c1c15cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:46.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 shutdown_connections 2023-10-16T10:03:46.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.067+0000 7f4c22ad8700 1 -- 172.21.15.181:0/1983871732 wait complete. 2023-10-16T10:03:46.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:03:20.395061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:03:20.923916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:46.083 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:46.181 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:46.179+0000 7fc064740200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:03:46.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.208+0000 7fc5e3548700 1 Processor -- start 2023-10-16T10:03:46.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.208+0000 7fc5e3548700 1 -- start start 2023-10-16T10:03:46.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e3548700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5dc15d9e0 0x7fc5dc0d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e3548700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e3548700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5dc0d4180 con 0x7fc5dc0d00c0 2023-10-16T10:03:46.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e3548700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5dc0d20c0 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e3548700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0d1440 con 0x7fc5dc0cefa0 2023-10-16T10:03:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e12e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5dc15d9e0 0x7fc5dc0d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e12e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5dc15d9e0 0x7fc5dc0d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48590/0 (socket says 172.21.15.181:48590) 2023-10-16T10:03:46.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e12e4700 1 -- 172.21.15.181:0/1164848563 learned_addr learned my addr 172.21.15.181:0/1164848563 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e0ae3700 1 --2- 172.21.15.181:0/1164848563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e12e4700 1 -- 172.21.15.181:0/1164848563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 msgr2=0x7fc5dc0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e12e4700 1 --2- 172.21.15.181:0/1164848563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.209+0000 7fc5e12e4700 1 -- 172.21.15.181:0/1164848563 >> v1:172.21.15.190:6789/0 conn(0x7fc5dc0d00c0 legacy=0x7fc5dc0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e12e4700 1 -- 172.21.15.181:0/1164848563 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc0d1720 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e12e4700 1 --2- 172.21.15.181:0/1164848563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc0d24e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80098f0 tx=0x7fc5d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435fc767fcbb27c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5d3fff700 1 -- 172.21.15.181:0/1164848563 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8011830 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5d3fff700 1 -- 172.21.15.181:0/1164848563 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d8011990 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5d3fff700 1 -- 172.21.15.181:0/1164848563 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8011d20 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e3548700 1 -- 172.21.15.181:0/1164848563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 msgr2=0x7fc5dc0d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e3548700 1 --2- 172.21.15.181:0/1164848563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc0d24e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80098f0 tx=0x7fc5d8009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:46.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e3548700 1 -- 172.21.15.181:0/1164848563 shutdown_connections 2023-10-16T10:03:46.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e3548700 1 --2- 172.21.15.181:0/1164848563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc0d24e0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e3548700 1 --2- 172.21.15.181:0/1164848563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.210+0000 7fc5e3548700 1 -- 172.21.15.181:0/1164848563 >> 172.21.15.181:0/1164848563 conn(0x7fc5dc09cb40 msgr2=0x7fc5dc0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.211+0000 7fc5e3548700 1 -- 172.21.15.181:0/1164848563 shutdown_connections 2023-10-16T10:03:46.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.211+0000 7fc5e3548700 1 -- 172.21.15.181:0/1164848563 wait complete. 2023-10-16T10:03:46.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.212+0000 7fc5e3548700 1 Processor -- start 2023-10-16T10:03:46.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.212+0000 7fc5e3548700 1 -- start start 2023-10-16T10:03:46.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e3548700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e3548700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc0d00c0 0x7fc5dc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e3548700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc1f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e3548700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc1f69c0 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e3548700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0c9e30 con 0x7fc5dc0d00c0 2023-10-16T10:03:46.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e3548700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0c7fb0 con 0x7fc5dc0cefa0 2023-10-16T10:03:46.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e1ae5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc1f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e12e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e1ae5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc1f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48600/0 (socket says 172.21.15.181:48600) 2023-10-16T10:03:46.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e1ae5700 1 -- 172.21.15.181:0/949999350 learned_addr learned my addr 172.21.15.181:0/949999350 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:46.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.213+0000 7fc5e0ae3700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc0d00c0 0x7fc5dc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e1ae5700 1 -- 172.21.15.181:0/949999350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 msgr2=0x7fc5dc0c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e1ae5700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e1ae5700 1 -- 172.21.15.181:0/949999350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc0d00c0 msgr2=0x7fc5dc0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e1ae5700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc0d00c0 0x7fc5dc0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e1ae5700 1 -- 172.21.15.181:0/949999350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8009550 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e1ae5700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc1f32a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400cc50 tx=0x7fc5d400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:46.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5d1ffb700 1 -- 172.21.15.181:0/949999350 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4003c00 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5d1ffb700 1 -- 172.21.15.181:0/949999350 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d4003d60 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5dc0c7710 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5d1ffb700 1 -- 172.21.15.181:0/949999350 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400aab0 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.214+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc0c7cd0 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.215+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5dc067d20 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.216+0000 7fc5d1ffb700 1 -- 172.21.15.181:0/949999350 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fc5d400ac10 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.216+0000 7fc5d1ffb700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 0x7fc5c006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.216+0000 7fc5e12e4700 1 -- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 msgr2=0x7fc5c006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:46.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.216+0000 7fc5d1ffb700 1 -- 172.21.15.181:0/949999350 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7fc5d4089b60 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.217+0000 7fc5e12e4700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 0x7fc5c006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:46.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.225+0000 7fc5d1ffb700 1 -- 172.21.15.181:0/949999350 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc5d4054700 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.237 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:46.237+0000 7f9a6d36e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:03:46.311 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:46.310+0000 7f9a6d36e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:03:46.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.418+0000 7fc5e12e4700 1 -- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 msgr2=0x7fc5c006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.418+0000 7fc5e12e4700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 0x7fc5c006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:46.422 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:46.421+0000 7fc064740200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:03:46.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.422+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5dc0d0e50 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.423+0000 7fc5d1ffb700 1 -- 172.21.15.181:0/949999350 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5d4057d20 con 0x7fc5dc15d9e0 2023-10-16T10:03:46.425 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 msgr2=0x7fc5c006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:46.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 0x7fc5c006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 msgr2=0x7fc5dc1f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc1f32a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400cc50 tx=0x7fc5d400cf60 comp rx=0 tx=0).stop 2023-10-16T10:03:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 shutdown_connections 2023-10-16T10:03:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fc5c006b0b0 0x7fc5c006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5dc0cefa0 0x7fc5dc0c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5dc15d9e0 0x7fc5dc1f32a0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 --2- 172.21.15.181:0/949999350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5dc0d00c0 0x7fc5dc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.425+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 >> 172.21.15.181:0/949999350 conn(0x7fc5dc09cb40 msgr2=0x7fc5dc15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:46.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.426+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 shutdown_connections 2023-10-16T10:03:46.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.426+0000 7fc5e3548700 1 -- 172.21.15.181:0/949999350 wait complete. 2023-10-16T10:03:46.439 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T10:03:46.503 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:46.502+0000 7fc064740200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:03:46.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.565+0000 7f2f43057700 1 Processor -- start 2023-10-16T10:03:46.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.565+0000 7f2f43057700 1 -- start start 2023-10-16T10:03:46.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f43057700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c0d24e0 0x7f2f3c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f43057700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f3c0d0600 0x7f2f3c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f43057700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c0d2960 con 0x7f2f3c15d9e0 2023-10-16T10:03:46.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f43057700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c0d20c0 con 0x7f2f3c0d0600 2023-10-16T10:03:46.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f43057700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0d0a90 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f40df3700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2f3c15d9e0 0x7f2f3c0cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59684/0 (socket says 172.21.15.181:59684) 2023-10-16T10:03:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f40df3700 1 -- 172.21.15.181:0/148293585 learned_addr learned my addr 172.21.15.181:0/148293585 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f415f4700 1 --2- 172.21.15.181:0/148293585 >> v2:172.21.15.190:3300/0 conn(0x7f2f3c0d0600 0x7f2f3c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3bfff700 1 --2- 172.21.15.181:0/148293585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c0d24e0 0x7f2f3c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3b7fe700 1 -- 172.21.15.181:0/148293585 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50914524 0 0) 0x7f2f3c0d2960 con 0x7f2f3c15d9e0 2023-10-16T10:03:46.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3b7fe700 1 -- 172.21.15.181:0/148293585 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c0035b0 con 0x7f2f3c15d9e0 2023-10-16T10:03:46.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3bfff700 1 -- 172.21.15.181:0/148293585 >> v2:172.21.15.190:3300/0 conn(0x7f2f3c0d0600 msgr2=0x7f2f3c0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:46.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3bfff700 1 --2- 172.21.15.181:0/148293585 >> v2:172.21.15.190:3300/0 conn(0x7f2f3c0d0600 0x7f2f3c0b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3bfff700 1 -- 172.21.15.181:0/148293585 >> v1:172.21.15.181:6789/0 conn(0x7f2f3c15d9e0 legacy=0x7f2f3c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3bfff700 1 -- 172.21.15.181:0/148293585 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0d4180 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f415f4700 1 --2- 172.21.15.181:0/148293585 >> v2:172.21.15.190:3300/0 conn(0x7f2f3c0d0600 0x7f2f3c0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:46.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.566+0000 7f2f3bfff700 1 --2- 172.21.15.181:0/148293585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c0d24e0 0x7f2f3c0d00c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009840 tx=0x7f2f28009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e307d83423ead60b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:46.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f3b7fe700 1 -- 172.21.15.181:0/148293585 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28011830 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f3b7fe700 1 -- 172.21.15.181:0/148293585 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28011990 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f3b7fe700 1 -- 172.21.15.181:0/148293585 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28011d00 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 -- 172.21.15.181:0/148293585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c0d24e0 msgr2=0x7f2f3c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 --2- 172.21.15.181:0/148293585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c0d24e0 0x7f2f3c0d00c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009840 tx=0x7f2f28009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:46.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 -- 172.21.15.181:0/148293585 shutdown_connections 2023-10-16T10:03:46.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 --2- 172.21.15.181:0/148293585 >> v2:172.21.15.190:3300/0 conn(0x7f2f3c0d0600 0x7f2f3c0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 --2- 172.21.15.181:0/148293585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c0d24e0 0x7f2f3c0d00c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 -- 172.21.15.181:0/148293585 >> 172.21.15.181:0/148293585 conn(0x7f2f3c09cb40 msgr2=0x7f2f3c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:46.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 -- 172.21.15.181:0/148293585 shutdown_connections 2023-10-16T10:03:46.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.567+0000 7f2f43057700 1 -- 172.21.15.181:0/148293585 wait complete. 2023-10-16T10:03:46.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.569+0000 7f2f43057700 1 Processor -- start 2023-10-16T10:03:46.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.569+0000 7f2f43057700 1 -- start start 2023-10-16T10:03:46.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.569+0000 7f2f43057700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3c0d0600 0x7f2f3c1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.569+0000 7f2f43057700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f3c0d24e0 0x7f2f3c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.569+0000 7f2f43057700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c15d9e0 0x7f2f3c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f43057700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1572f0 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f43057700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1575d0 con 0x7f2f3c0d0600 2023-10-16T10:03:46.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f43057700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1578b0 con 0x7f2f3c15d9e0 2023-10-16T10:03:46.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f3c0d24e0 0x7f2f3c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f3c0d24e0 0x7f2f3c158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48612/0 (socket says 172.21.15.181:48612) 2023-10-16T10:03:46.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 -- 172.21.15.181:0/1110954120 learned_addr learned my addr 172.21.15.181:0/1110954120 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:46.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f40df3700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3c0d0600 0x7f2f3c1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f415f4700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c15d9e0 0x7f2f3c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 -- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c15d9e0 msgr2=0x7f2f3c15b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c15d9e0 0x7f2f3c15b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 -- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3c0d0600 msgr2=0x7f2f3c1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3c0d0600 0x7f2f3c1f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 -- 172.21.15.181:0/1110954120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f34009a70 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f3bfff700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f3c0d24e0 0x7f2f3c158830 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2f28000c00 tx=0x7f2f28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:46.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f397fa700 1 -- 172.21.15.181:0/1110954120 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28004550 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f397fa700 1 -- 172.21.15.181:0/1110954120 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2801f070 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f280094a0 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f397fa700 1 -- 172.21.15.181:0/1110954120 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280038a0 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.570+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c157e70 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.571+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c067d20 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.572+0000 7f2f397fa700 1 -- 172.21.15.181:0/1110954120 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f2f28003a00 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.572+0000 7f2f397fa700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 0x7f2f2006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.573+0000 7f2f40df3700 1 -- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 msgr2=0x7f2f2006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:46.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.573+0000 7f2f397fa700 1 -- 172.21.15.181:0/1110954120 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f2f28021070 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.573+0000 7f2f40df3700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 0x7f2f2006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:46.583 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:46.575+0000 7f9a6d36e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:03:46.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.577+0000 7f2f397fa700 1 -- 172.21.15.181:0/1110954120 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2f28055f30 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.774+0000 7f2f40df3700 1 -- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 msgr2=0x7f2f2006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:46.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.774+0000 7f2f40df3700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 0x7f2f2006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:46.780 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:46.778+0000 7fc064740200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:03:46.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.779+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f3c06c3c0 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.779+0000 7f2f397fa700 1 -- 172.21.15.181:0/1110954120 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f28003cb0 con 0x7f2f3c0d24e0 2023-10-16T10:03:46.781 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T10:03:46.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 msgr2=0x7f2f2006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:46.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 0x7f2f2006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f3c0d24e0 msgr2=0x7f2f3c158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f3c0d24e0 0x7f2f3c158830 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2f28000c00 tx=0x7f2f28004750 comp rx=0 tx=0).stop 2023-10-16T10:03:46.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 shutdown_connections 2023-10-16T10:03:46.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f2f2006b0b0 0x7f2f2006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f3c15d9e0 0x7f2f3c15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f3c0d0600 0x7f2f3c1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 --2- 172.21.15.181:0/1110954120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f3c0d24e0 0x7f2f3c158830 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 >> 172.21.15.181:0/1110954120 conn(0x7f2f3c09cb40 msgr2=0x7f2f3c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:46.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 shutdown_connections 2023-10-16T10:03:46.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.781+0000 7f2f43057700 1 -- 172.21.15.181:0/1110954120 wait complete. 2023-10-16T10:03:46.795 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T10:03:46.800 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:46.799+0000 7f9a6d36e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:03:46.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.921+0000 7faffbc1c700 1 Processor -- start 2023-10-16T10:03:46.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.921+0000 7faffbc1c700 1 -- start start 2023-10-16T10:03:46.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faffbc1c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faffbc1c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faff406ade0 0x7faff41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faffbc1c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff406b230 con 0x7faff415da70 2023-10-16T10:03:46.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faffbc1c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff40cef00 con 0x7faff406ade0 2023-10-16T10:03:46.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faffbc1c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff40cec20 con 0x7faff4072760 2023-10-16T10:03:46.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faffa1b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faff406ade0 0x7faff41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faff99b8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faff415da70 0x7faff40ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59700/0 (socket says 172.21.15.181:59700) 2023-10-16T10:03:46.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faff99b8700 1 -- 172.21.15.181:0/1278044383 learned_addr learned my addr 172.21.15.181:0/1278044383 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:46.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.922+0000 7faff91b7700 1 --2- 172.21.15.181:0/1278044383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faffa1b9700 1 -- 172.21.15.181:0/1278044383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 msgr2=0x7faff406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:46.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faffa1b9700 1 --2- 172.21.15.181:0/1278044383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faffa1b9700 1 -- 172.21.15.181:0/1278044383 >> v1:172.21.15.181:6789/0 conn(0x7faff415da70 legacy=0x7faff40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faffa1b9700 1 -- 172.21.15.181:0/1278044383 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4071c80 con 0x7faff406ade0 2023-10-16T10:03:46.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faff89b6700 1 -- 172.21.15.181:0/1278044383 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204066501 0 0) 0x7faff0001110 con 0x7faff415da70 2023-10-16T10:03:46.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faffa1b9700 1 --2- 172.21.15.181:0/1278044383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff41615f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fafec009d90 tx=0x7fafec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db92db529eb250b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:46.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faff89b6700 1 -- 172.21.15.181:0/1278044383 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00e070 con 0x7faff406ade0 2023-10-16T10:03:46.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faff89b6700 1 -- 172.21.15.181:0/1278044383 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec00bb20 con 0x7faff406ade0 2023-10-16T10:03:46.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faff89b6700 1 -- 172.21.15.181:0/1278044383 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec012430 con 0x7faff406ade0 2023-10-16T10:03:46.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faffbc1c700 1 -- 172.21.15.181:0/1278044383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 msgr2=0x7faff41615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.923+0000 7faffbc1c700 1 --2- 172.21.15.181:0/1278044383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff41615f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fafec009d90 tx=0x7fafec00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:46.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.924+0000 7faffbc1c700 1 -- 172.21.15.181:0/1278044383 shutdown_connections 2023-10-16T10:03:46.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.924+0000 7faffbc1c700 1 --2- 172.21.15.181:0/1278044383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff41615f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.924+0000 7faffbc1c700 1 --2- 172.21.15.181:0/1278044383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.924+0000 7faffbc1c700 1 -- 172.21.15.181:0/1278044383 >> 172.21.15.181:0/1278044383 conn(0x7faff40cd010 msgr2=0x7faff40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:46.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.924+0000 7faffbc1c700 1 -- 172.21.15.181:0/1278044383 shutdown_connections 2023-10-16T10:03:46.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.924+0000 7faffbc1c700 1 -- 172.21.15.181:0/1278044383 wait complete. 2023-10-16T10:03:46.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.925+0000 7faffbc1c700 1 Processor -- start 2023-10-16T10:03:46.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.925+0000 7faffbc1c700 1 -- start start 2023-10-16T10:03:46.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.925+0000 7faffbc1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff40b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.925+0000 7faffbc1c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff40c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faffbc1c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faff415da70 0x7faff40c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faffbc1c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40c5650 con 0x7faff406ade0 2023-10-16T10:03:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faffbc1c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40c58f0 con 0x7faff415da70 2023-10-16T10:03:46.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faffbc1c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff40c5bd0 con 0x7faff4072760 2023-10-16T10:03:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff40b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff40b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48628/0 (socket says 172.21.15.181:48628) 2023-10-16T10:03:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 -- 172.21.15.181:0/2234804901 learned_addr learned my addr 172.21.15.181:0/2234804901 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff91b7700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff40c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faffa1b9700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faff415da70 0x7faff40c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 -- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 msgr2=0x7faff40c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff40c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 -- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faff415da70 msgr2=0x7faff40c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faff415da70 0x7faff40c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:46.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 -- 172.21.15.181:0/2234804901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec009a70 con 0x7faff406ade0 2023-10-16T10:03:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faff99b8700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff40b5c70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7faff000c730 tx=0x7faff000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7fafea7fc700 1 -- 172.21.15.181:0/2234804901 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0004c80 con 0x7faff406ade0 2023-10-16T10:03:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7fafea7fc700 1 -- 172.21.15.181:0/2234804901 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff0004e00 con 0x7faff406ade0 2023-10-16T10:03:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff40c5eb0 con 0x7faff406ade0 2023-10-16T10:03:46.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7fafea7fc700 1 -- 172.21.15.181:0/2234804901 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0007310 con 0x7faff406ade0 2023-10-16T10:03:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.926+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff40c6470 con 0x7faff406ade0 2023-10-16T10:03:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.927+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff40c6690 con 0x7faff406ade0 2023-10-16T10:03:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.928+0000 7fafea7fc700 1 -- 172.21.15.181:0/2234804901 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7faff0007470 con 0x7faff406ade0 2023-10-16T10:03:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.929+0000 7fafea7fc700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 0x7fafd806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:46.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.929+0000 7faff91b7700 1 -- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 msgr2=0x7fafd806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:46.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.929+0000 7fafea7fc700 1 -- 172.21.15.181:0/2234804901 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7faff00896a0 con 0x7faff406ade0 2023-10-16T10:03:46.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.929+0000 7faff91b7700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 0x7fafd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:46.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:46.934+0000 7fafea7fc700 1 -- 172.21.15.181:0/2234804901 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faff00541c0 con 0x7faff406ade0 2023-10-16T10:03:46.950 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:46.949+0000 7f9a6d36e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:03:47.019 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:47.018+0000 7fc064740200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:03:47.066 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:47.065+0000 7f9a6d36e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:03:47.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.130+0000 7faff91b7700 1 -- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 msgr2=0x7fafd806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:47.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.130+0000 7faff91b7700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 0x7fafd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:47.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.131+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faff406b230 con 0x7faff406ade0 2023-10-16T10:03:47.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.131+0000 7fafea7fc700 1 -- 172.21.15.181:0/2234804901 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7faff406b230 con 0x7faff406ade0 2023-10-16T10:03:47.133 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T10:03:47.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 msgr2=0x7fafd806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 0x7fafd806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 msgr2=0x7faff40b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff40b5c70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7faff000c730 tx=0x7faff000ca40 comp rx=0 tx=0).stop 2023-10-16T10:03:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 shutdown_connections 2023-10-16T10:03:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7fafd806b0b0 0x7fafd806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faff415da70 0x7faff40c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faff406ade0 0x7faff40b5c70 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 --2- 172.21.15.181:0/2234804901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faff4072760 0x7faff40c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 >> 172.21.15.181:0/2234804901 conn(0x7faff40cd010 msgr2=0x7faff40b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 shutdown_connections 2023-10-16T10:03:47.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.134+0000 7faffbc1c700 1 -- 172.21.15.181:0/2234804901 wait complete. 2023-10-16T10:03:47.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:47.179 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:47.177+0000 7fc064740200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:03:47.222 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:47.221+0000 7f9a6d36e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:03:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.270+0000 7f84c8e3b700 1 Processor -- start 2023-10-16T10:03:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.270+0000 7f84c8e3b700 1 -- start start 2023-10-16T10:03:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c8e3b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c8e3b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f84c4161f40 0x7f84c40d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c8e3b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c40d2560 con 0x7f84c4160d20 2023-10-16T10:03:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c8e3b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c4162800 con 0x7f84c4161f40 2023-10-16T10:03:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c8e3b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40d43e0 con 0x7f84c4161b60 2023-10-16T10:03:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c2d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f84c4160d20 0x7f84c40b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59720/0 (socket says 172.21.15.181:59720) 2023-10-16T10:03:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c2d9d700 1 -- 172.21.15.181:0/2342928302 learned_addr learned my addr 172.21.15.181:0/2342928302 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c259c700 1 --2- 172.21.15.181:0/2342928302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c1d9b700 1 --2- 172.21.15.181:0/2342928302 >> v2:172.21.15.190:3300/0 conn(0x7f84c4161f40 0x7f84c40d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c159a700 1 -- 172.21.15.181:0/2342928302 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776577786 0 0) 0x7f84c40d2560 con 0x7f84c4160d20 2023-10-16T10:03:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.271+0000 7f84c159a700 1 -- 172.21.15.181:0/2342928302 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ac0035b0 con 0x7f84c4160d20 2023-10-16T10:03:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c159a700 1 -- 172.21.15.181:0/2342928302 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4258138457 0 0) 0x7f84ac0035b0 con 0x7f84c4160d20 2023-10-16T10:03:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c259c700 1 -- 172.21.15.181:0/2342928302 >> v2:172.21.15.190:3300/0 conn(0x7f84c4161f40 msgr2=0x7f84c40d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c259c700 1 --2- 172.21.15.181:0/2342928302 >> v2:172.21.15.190:3300/0 conn(0x7f84c4161f40 0x7f84c40d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c259c700 1 -- 172.21.15.181:0/2342928302 >> v1:172.21.15.181:6789/0 conn(0x7f84c4160d20 legacy=0x7f84c40b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c259c700 1 -- 172.21.15.181:0/2342928302 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c406ce20 con 0x7f84c4161b60 2023-10-16T10:03:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c259c700 1 --2- 172.21.15.181:0/2342928302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c406d5f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009820 tx=0x7f84b4009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551f423719eca6cf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c159a700 1 -- 172.21.15.181:0/2342928302 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0019f0 con 0x7f84c4161b60 2023-10-16T10:03:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c159a700 1 -- 172.21.15.181:0/2342928302 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b4005c30 con 0x7f84c4161b60 2023-10-16T10:03:47.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.272+0000 7f84c159a700 1 -- 172.21.15.181:0/2342928302 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4004030 con 0x7f84c4161b60 2023-10-16T10:03:47.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2342928302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 msgr2=0x7f84c406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2342928302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c406d5f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009820 tx=0x7f84b4009b30 comp rx=0 tx=0).stop 2023-10-16T10:03:47.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2342928302 shutdown_connections 2023-10-16T10:03:47.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2342928302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c406d5f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2342928302 >> v2:172.21.15.190:3300/0 conn(0x7f84c4161f40 0x7f84c40d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2342928302 >> 172.21.15.181:0/2342928302 conn(0x7f84c409cba0 msgr2=0x7f84c40d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:47.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2342928302 shutdown_connections 2023-10-16T10:03:47.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.273+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2342928302 wait complete. 2023-10-16T10:03:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.274+0000 7f84c8e3b700 1 Processor -- start 2023-10-16T10:03:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.275+0000 7f84c8e3b700 1 -- start start 2023-10-16T10:03:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.275+0000 7f84c8e3b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c4160d20 0x7f84c40c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.275+0000 7f84c8e3b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.275+0000 7f84c8e3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4161f40 0x7f84c40c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.275+0000 7f84c8e3b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4150210 con 0x7f84c4161f40 2023-10-16T10:03:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.275+0000 7f84c8e3b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c415e2a0 con 0x7f84c4160d20 2023-10-16T10:03:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.275+0000 7f84c8e3b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40c7a30 con 0x7f84c4161b60 2023-10-16T10:03:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4161f40 0x7f84c40c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4161f40 0x7f84c40c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48648/0 (socket says 172.21.15.181:48648) 2023-10-16T10:03:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 -- 172.21.15.181:0/2923731 learned_addr learned my addr 172.21.15.181:0/2923731 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c1d9b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c259c700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c4160d20 0x7f84c40c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 -- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 msgr2=0x7f84c414fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c414fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 -- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c4160d20 msgr2=0x7f84c40c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c4160d20 0x7f84c40c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 -- 172.21.15.181:0/2923731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4009430 con 0x7f84c4161f40 2023-10-16T10:03:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c1d9b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c2d9d700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4161f40 0x7f84c40c9c40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00cc10 tx=0x7f84bc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84b2ffd700 1 -- 172.21.15.181:0/2923731 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00a9e0 con 0x7f84c4161f40 2023-10-16T10:03:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84b2ffd700 1 -- 172.21.15.181:0/2923731 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc00ab60 con 0x7f84c4161f40 2023-10-16T10:03:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c40c7d10 con 0x7f84c4161f40 2023-10-16T10:03:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84b2ffd700 1 -- 172.21.15.181:0/2923731 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0070d0 con 0x7f84c4161f40 2023-10-16T10:03:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.276+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c40c82d0 con 0x7f84c4161f40 2023-10-16T10:03:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.277+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c404ebb0 con 0x7f84c4161f40 2023-10-16T10:03:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.278+0000 7f84b2ffd700 1 -- 172.21.15.181:0/2923731 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f84bc007230 con 0x7f84c4161f40 2023-10-16T10:03:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.278+0000 7f84b2ffd700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 0x7f84a406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.278+0000 7f84c259c700 1 -- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 msgr2=0x7f84a406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:47.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.278+0000 7f84b2ffd700 1 -- 172.21.15.181:0/2923731 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f84bc089a10 con 0x7f84c4161f40 2023-10-16T10:03:47.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.278+0000 7f84c259c700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 0x7f84a406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:47.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.283+0000 7f84b2ffd700 1 -- 172.21.15.181:0/2923731 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f84bc054490 con 0x7f84c4161f40 2023-10-16T10:03:47.302 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:47.300+0000 7fc064740200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:03:47.464 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:47.462+0000 7fc064740200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:03:47.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.479+0000 7f84c259c700 1 -- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 msgr2=0x7f84a406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:47.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.479+0000 7f84c259c700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 0x7f84a406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:47.483 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:47.482+0000 7f9a6d36e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:47.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.528+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c40d0db0 con 0x7f84c4161f40 2023-10-16T10:03:47.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.529+0000 7f84b2ffd700 1 -- 172.21.15.181:0/2923731 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f84bc018070 con 0x7f84c4161f40 2023-10-16T10:03:47.531 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:47.531 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2508916691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2508916691},{"type":"v1","addr":"172.21.15.181:6836","nonce":2508916691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7687,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1303363384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1303363384},{"type":"v1","addr":"172.21.15.190:6834","nonce":1303363384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/300739418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":300739418},{"type":"v1","addr":"172.21.15.181:6837","nonce":300739418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:03:21.971792+0000","modified":"2023-10-16T10:03:24.995974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1716409851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1716409851},{"type":"v1","addr":"172.21.15.190:6835","nonce":1716409851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:03:47.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 msgr2=0x7f84a406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:47.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 0x7f84a406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4161f40 msgr2=0x7f84c40c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4161f40 0x7f84c40c9c40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00cc10 tx=0x7f84bc00cf20 comp rx=0 tx=0).stop 2023-10-16T10:03:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 shutdown_connections 2023-10-16T10:03:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f84a406b0b0 0x7f84a406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84c4161f40 0x7f84c40c9c40 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c4160d20 0x7f84c40c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 --2- 172.21.15.181:0/2923731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c4161b60 0x7f84c414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 >> 172.21.15.181:0/2923731 conn(0x7f84c409cba0 msgr2=0x7f84c40b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 shutdown_connections 2023-10-16T10:03:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.531+0000 7f84c8e3b700 1 -- 172.21.15.181:0/2923731 wait complete. 2023-10-16T10:03:47.536 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2023-10-16T10:03:47.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:47.642 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:47.641+0000 7f9a6d36e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:03:47.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.710+0000 7f341fc11700 1 Processor -- start 2023-10-16T10:03:47.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.710+0000 7f341fc11700 1 -- start start 2023-10-16T10:03:47.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341fc11700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34180d2390 0x7f34180cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341fc11700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d0180 0x7f34180cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341fc11700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34180cc470 con 0x7f34180ce7f0 2023-10-16T10:03:47.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341fc11700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34180d0610 con 0x7f34180d2390 2023-10-16T10:03:47.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341fc11700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341806c810 con 0x7f34180d0180 2023-10-16T10:03:47.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d9ad700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f34180ce7f0 0x7f341806c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59732/0 (socket says 172.21.15.181:59732) 2023-10-16T10:03:47.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d1ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f34180d2390 0x7f34180cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d9ad700 1 -- 172.21.15.181:0/998595788 learned_addr learned my addr 172.21.15.181:0/998595788 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:47.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341e1ae700 1 --2- 172.21.15.181:0/998595788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d0180 0x7f34180cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341c9ab700 1 -- 172.21.15.181:0/998595788 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211750149 0 0) 0x7f34180cc470 con 0x7f34180ce7f0 2023-10-16T10:03:47.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341c9ab700 1 -- 172.21.15.181:0/998595788 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34040035b0 con 0x7f34180ce7f0 2023-10-16T10:03:47.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d1ac700 1 -- 172.21.15.181:0/998595788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d0180 msgr2=0x7f34180cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:47.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d1ac700 1 --2- 172.21.15.181:0/998595788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d0180 0x7f34180cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d1ac700 1 -- 172.21.15.181:0/998595788 >> v1:172.21.15.181:6789/0 conn(0x7f34180ce7f0 legacy=0x7f341806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d1ac700 1 -- 172.21.15.181:0/998595788 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341806ca30 con 0x7f34180d2390 2023-10-16T10:03:47.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341d1ac700 1 --2- 172.21.15.181:0/998595788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180d2390 0x7f34180cfc40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3408009870 tx=0x7f3408009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f93f86578e271f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:47.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341c9ab700 1 -- 172.21.15.181:0/998595788 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408011830 con 0x7f34180d2390 2023-10-16T10:03:47.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341c9ab700 1 -- 172.21.15.181:0/998595788 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3408011990 con 0x7f34180d2390 2023-10-16T10:03:47.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.711+0000 7f341c9ab700 1 -- 172.21.15.181:0/998595788 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408011d20 con 0x7f34180d2390 2023-10-16T10:03:47.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 -- 172.21.15.181:0/998595788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180d2390 msgr2=0x7f34180cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 --2- 172.21.15.181:0/998595788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180d2390 0x7f34180cfc40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3408009870 tx=0x7f3408009b80 comp rx=0 tx=0).stop 2023-10-16T10:03:47.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 -- 172.21.15.181:0/998595788 shutdown_connections 2023-10-16T10:03:47.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 --2- 172.21.15.181:0/998595788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d0180 0x7f34180cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 --2- 172.21.15.181:0/998595788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180d2390 0x7f34180cfc40 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 -- 172.21.15.181:0/998595788 >> 172.21.15.181:0/998595788 conn(0x7f341809a9a0 msgr2=0x7f34180d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:47.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 -- 172.21.15.181:0/998595788 shutdown_connections 2023-10-16T10:03:47.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.712+0000 7f341fc11700 1 -- 172.21.15.181:0/998595788 wait complete. 2023-10-16T10:03:47.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.713+0000 7f341fc11700 1 Processor -- start 2023-10-16T10:03:47.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.713+0000 7f341fc11700 1 -- start start 2023-10-16T10:03:47.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341fc11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180ce7f0 0x7f34180c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341fc11700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34180d0180 0x7f34180b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341fc11700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d2390 0x7f34180b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341fc11700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418157b30 con 0x7f34180ce7f0 2023-10-16T10:03:47.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341fc11700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418157e10 con 0x7f34180d0180 2023-10-16T10:03:47.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341fc11700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34181580f0 con 0x7f34180d2390 2023-10-16T10:03:47.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180ce7f0 0x7f34180c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180ce7f0 0x7f34180c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48660/0 (socket says 172.21.15.181:48660) 2023-10-16T10:03:47.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 -- 172.21.15.181:0/1193718709 learned_addr learned my addr 172.21.15.181:0/1193718709 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:47.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341e1ae700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d2390 0x7f34180b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d1ac700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34180d0180 0x7f34180b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:47.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 -- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d2390 msgr2=0x7f34180b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d2390 0x7f34180b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 -- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34180d0180 msgr2=0x7f34180b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34180d0180 0x7f34180b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d9ad700 1 -- 172.21.15.181:0/1193718709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080094a0 con 0x7f34180ce7f0 2023-10-16T10:03:47.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.714+0000 7f341d1ac700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34180d0180 0x7f34180b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:47.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.715+0000 7f341d9ad700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180ce7f0 0x7f34180c7930 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f3414001260 tx=0x7f341400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:47.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.715+0000 7f340e7fc700 1 -- 172.21.15.181:0/1193718709 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140074e0 con 0x7f34180ce7f0 2023-10-16T10:03:47.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.715+0000 7f340e7fc700 1 -- 172.21.15.181:0/1193718709 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414007640 con 0x7f34180ce7f0 2023-10-16T10:03:47.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.715+0000 7f340e7fc700 1 -- 172.21.15.181:0/1193718709 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414005a50 con 0x7f34180ce7f0 2023-10-16T10:03:47.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.715+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34181583d0 con 0x7f34180ce7f0 2023-10-16T10:03:47.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.715+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341815b9f0 con 0x7f34180ce7f0 2023-10-16T10:03:47.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.716+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3418067d20 con 0x7f34180ce7f0 2023-10-16T10:03:47.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.717+0000 7f340e7fc700 1 -- 172.21.15.181:0/1193718709 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f3414005bb0 con 0x7f34180ce7f0 2023-10-16T10:03:47.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.717+0000 7f340e7fc700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 0x7f33fc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:47.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.717+0000 7f341d1ac700 1 -- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 msgr2=0x7f33fc06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:47.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.717+0000 7f340e7fc700 1 -- 172.21.15.181:0/1193718709 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f3414089030 con 0x7f34180ce7f0 2023-10-16T10:03:47.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.717+0000 7f341d1ac700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 0x7f33fc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:47.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.722+0000 7f340e7fc700 1 -- 172.21.15.181:0/1193718709 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3414053bd0 con 0x7f34180ce7f0 2023-10-16T10:03:47.746 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:47.744+0000 7fc064740200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:03:47.915 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:47.913+0000 7fc064740200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:03:47.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.918+0000 7f341d1ac700 1 -- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 msgr2=0x7f33fc06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:47.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.918+0000 7f341d1ac700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 0x7f33fc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:47.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.922+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34180cef40 con 0x7f34180ce7f0 2023-10-16T10:03:47.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.924+0000 7f340e7fc700 1 -- 172.21.15.181:0/1193718709 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17760 (secure 0 0 0) 0x7f3414017090 con 0x7f34180ce7f0 2023-10-16T10:03:47.926 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:47.926 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:35.802255+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:03:20.395061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:03:20.923916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/240608457":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/3359421143":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6833/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6832/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/2638619292":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1179812996":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1546779767":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1171463515":"2023-10-17T10:03:35.802223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:47.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 msgr2=0x7f33fc06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:47.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 0x7f33fc06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180ce7f0 msgr2=0x7f34180c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:47.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180ce7f0 0x7f34180c7930 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f3414001260 tx=0x7f341400c7b0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 shutdown_connections 2023-10-16T10:03:47.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f33fc06b0b0 0x7f33fc06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34180d2390 0x7f34180b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34180ce7f0 0x7f34180c7930 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 --2- 172.21.15.181:0/1193718709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34180d0180 0x7f34180b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:47.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 >> 172.21.15.181:0/1193718709 conn(0x7f341809a9a0 msgr2=0x7f34180c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:47.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 shutdown_connections 2023-10-16T10:03:47.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:47.926+0000 7f341fc11700 1 -- 172.21.15.181:0/1193718709 wait complete. 2023-10-16T10:03:47.941 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T10:03:47.941 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:48.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.066+0000 7f31ee75a700 1 Processor -- start 2023-10-16T10:03:48.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.066+0000 7f31ee75a700 1 -- start start 2023-10-16T10:03:48.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.066+0000 7f31ee75a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31e815eab0 0x7f31e80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.066+0000 7f31ee75a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e806ed00 con 0x7f31e80cb0c0 2023-10-16T10:03:48.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.066+0000 7f31ee75a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80d02d0 con 0x7f31e815e6d0 2023-10-16T10:03:48.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.066+0000 7f31ee75a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e806e0f0 con 0x7f31e815eab0 2023-10-16T10:03:48.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31eccf7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f31e80cb0c0 0x7f31e815a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59752/0 (socket says 172.21.15.181:59752) 2023-10-16T10:03:48.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31eccf7700 1 -- 172.21.15.181:0/692784452 learned_addr learned my addr 172.21.15.181:0/692784452 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:48.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e77fe700 1 --2- 172.21.15.181:0/692784452 >> v2:172.21.15.190:3300/0 conn(0x7f31e815eab0 0x7f31e80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e6ffd700 1 -- 172.21.15.181:0/692784452 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918006472 0 0) 0x7f31e806ed00 con 0x7f31e80cb0c0 2023-10-16T10:03:48.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e6ffd700 1 -- 172.21.15.181:0/692784452 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d00035b0 con 0x7f31e80cb0c0 2023-10-16T10:03:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e6ffd700 1 -- 172.21.15.181:0/692784452 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065366916 0 0) 0x7f31e80d02d0 con 0x7f31e815e6d0 2023-10-16T10:03:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e6ffd700 1 -- 172.21.15.181:0/692784452 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31e806ed00 con 0x7f31e815e6d0 2023-10-16T10:03:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e77fe700 1 -- 172.21.15.181:0/692784452 >> v1:172.21.15.190:6789/0 conn(0x7f31e815e6d0 legacy=0x7f31e815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e77fe700 1 -- 172.21.15.181:0/692784452 >> v1:172.21.15.181:6789/0 conn(0x7f31e80cb0c0 legacy=0x7f31e815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e77fe700 1 -- 172.21.15.181:0/692784452 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e806e3d0 con 0x7f31e815eab0 2023-10-16T10:03:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e77fe700 1 --2- 172.21.15.181:0/692784452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815eab0 0x7f31e80d0fc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009840 tx=0x7f31dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35253f776c3f38db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e6ffd700 1 -- 172.21.15.181:0/692784452 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc011830 con 0x7f31e815eab0 2023-10-16T10:03:48.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e6ffd700 1 -- 172.21.15.181:0/692784452 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc011990 con 0x7f31e815eab0 2023-10-16T10:03:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.067+0000 7f31e6ffd700 1 -- 172.21.15.181:0/692784452 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc011d20 con 0x7f31e815eab0 2023-10-16T10:03:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.068+0000 7f31ee75a700 1 -- 172.21.15.181:0/692784452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815eab0 msgr2=0x7f31e80d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.068+0000 7f31ee75a700 1 --2- 172.21.15.181:0/692784452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815eab0 0x7f31e80d0fc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009840 tx=0x7f31dc009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:48.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.068+0000 7f31ee75a700 1 -- 172.21.15.181:0/692784452 shutdown_connections 2023-10-16T10:03:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.068+0000 7f31ee75a700 1 --2- 172.21.15.181:0/692784452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815eab0 0x7f31e80d0fc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.068+0000 7f31ee75a700 1 -- 172.21.15.181:0/692784452 >> 172.21.15.181:0/692784452 conn(0x7f31e80d04a0 msgr2=0x7f31e80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.068+0000 7f31ee75a700 1 -- 172.21.15.181:0/692784452 shutdown_connections 2023-10-16T10:03:48.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.068+0000 7f31ee75a700 1 -- 172.21.15.181:0/692784452 wait complete. 2023-10-16T10:03:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.069+0000 7f31ee75a700 1 Processor -- start 2023-10-16T10:03:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31ee75a700 1 -- start start 2023-10-16T10:03:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31ee75a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e80cb0c0 0x7f31e80ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31ee75a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815e6d0 0x7f31e80c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31ee75a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31e815eab0 0x7f31e81509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31ee75a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80c7ab0 con 0x7f31e815eab0 2023-10-16T10:03:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31ee75a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8150f10 con 0x7f31e815e6d0 2023-10-16T10:03:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31ee75a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80c7cc0 con 0x7f31e80cb0c0 2023-10-16T10:03:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31eccf7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31e815eab0 0x7f31e81509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31eccf7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31e815eab0 0x7f31e81509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48680/0 (socket says 172.21.15.181:48680) 2023-10-16T10:03:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31e77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815e6d0 0x7f31e80c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31eccf7700 1 -- 172.21.15.181:0/1045169603 learned_addr learned my addr 172.21.15.181:0/1045169603 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:48.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.070+0000 7f31e7fff700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e80cb0c0 0x7f31e80ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31eccf7700 1 -- 172.21.15.181:0/1045169603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e80cb0c0 msgr2=0x7f31e80ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31eccf7700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e80cb0c0 0x7f31e80ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31eccf7700 1 -- 172.21.15.181:0/1045169603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815e6d0 msgr2=0x7f31e80c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31eccf7700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815e6d0 0x7f31e80c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31eccf7700 1 -- 172.21.15.181:0/1045169603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc0094a0 con 0x7f31e815eab0 2023-10-16T10:03:48.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31eccf7700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31e815eab0 0x7f31e81509d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f31e000caa0 tx=0x7f31e000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:48.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31e4ff9700 1 -- 172.21.15.181:0/1045169603 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e000a9e0 con 0x7f31e815eab0 2023-10-16T10:03:48.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31e4ff9700 1 -- 172.21.15.181:0/1045169603 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e000ab60 con 0x7f31e815eab0 2023-10-16T10:03:48.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e80c7fa0 con 0x7f31e815eab0 2023-10-16T10:03:48.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31e4ff9700 1 -- 172.21.15.181:0/1045169603 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0012400 con 0x7f31e815eab0 2023-10-16T10:03:48.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.071+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e80c8560 con 0x7f31e815eab0 2023-10-16T10:03:48.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.072+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e8067d20 con 0x7f31e815eab0 2023-10-16T10:03:48.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.073+0000 7f31e4ff9700 1 -- 172.21.15.181:0/1045169603 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f31e001b3b0 con 0x7f31e815eab0 2023-10-16T10:03:48.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.073+0000 7f31e4ff9700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 0x7f31d406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.073+0000 7f31e7fff700 1 -- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 msgr2=0x7f31d406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:48.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.073+0000 7f31e4ff9700 1 -- 172.21.15.181:0/1045169603 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f31e0089780 con 0x7f31e815eab0 2023-10-16T10:03:48.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.073+0000 7f31e7fff700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 0x7f31d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:48.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.077+0000 7f31e4ff9700 1 -- 172.21.15.181:0/1045169603 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31e00543e0 con 0x7f31e815eab0 2023-10-16T10:03:48.169 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:03:48.168+0000 7f9a6d36e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:03:48.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.274+0000 7f31e7fff700 1 -- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 msgr2=0x7f31d406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:48.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.274+0000 7f31e7fff700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 0x7f31d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:03:48.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.323+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31e80c43c0 con 0x7f31e815eab0 2023-10-16T10:03:48.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.324+0000 7f31e4ff9700 1 -- 172.21.15.181:0/1045169603 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f31e0057a00 con 0x7f31e815eab0 2023-10-16T10:03:48.326 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:48.326 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2508916691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2508916691},{"type":"v1","addr":"172.21.15.181:6836","nonce":2508916691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7687,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1303363384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1303363384},{"type":"v1","addr":"172.21.15.190:6834","nonce":1303363384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/300739418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":300739418},{"type":"v1","addr":"172.21.15.181:6837","nonce":300739418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:03:21.971792+0000","modified":"2023-10-16T10:03:24.995974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1716409851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1716409851},{"type":"v1","addr":"172.21.15.190:6835","nonce":1716409851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:03:48.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 msgr2=0x7f31d406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:48.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 0x7f31d406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31e815eab0 msgr2=0x7f31e81509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31e815eab0 0x7f31e81509d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f31e000caa0 tx=0x7f31e000cdb0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 shutdown_connections 2023-10-16T10:03:48.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d406b0b0 0x7f31d406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e80cb0c0 0x7f31e80ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31e815eab0 0x7f31e81509d0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 --2- 172.21.15.181:0/1045169603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e815e6d0 0x7f31e80c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 >> 172.21.15.181:0/1045169603 conn(0x7f31e80d04a0 msgr2=0x7f31e80cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:48.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 shutdown_connections 2023-10-16T10:03:48.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.326+0000 7f31ee75a700 1 -- 172.21.15.181:0/1045169603 wait complete. 2023-10-16T10:03:48.331 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2023-10-16T10:03:48.341 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.462+0000 7f31f2e0a700 1 Processor -- start 2023-10-16T10:03:48.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.462+0000 7f31f2e0a700 1 -- start start 2023-10-16T10:03:48.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f2e0a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31ec161b60 0x7f31ec06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f2e0a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec1611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f2e0a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec06ce20 con 0x7f31ec161f40 2023-10-16T10:03:48.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f2e0a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ec162800 con 0x7f31ec161b60 2023-10-16T10:03:48.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f2e0a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0d43e0 con 0x7f31ec160d20 2023-10-16T10:03:48.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f0ba6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31ec161b60 0x7f31ec06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f13a7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec1611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f0ba6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31ec161b60 0x7f31ec06d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:53736/0 (socket says 172.21.15.181:53736) 2023-10-16T10:03:48.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f0ba6700 1 -- 172.21.15.181:0/1206185572 learned_addr learned my addr 172.21.15.181:0/1206185572 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:48.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31ebfff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f31ec161f40 0x7f31ec0d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:43706/0 (socket says 172.21.15.181:43706) 2023-10-16T10:03:48.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31eb7fe700 1 -- 172.21.15.181:0/1206185572 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645688235 0 0) 0x7f31ec06ce20 con 0x7f31ec161f40 2023-10-16T10:03:48.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f13a7700 1 -- 172.21.15.181:0/1206185572 >> v2:172.21.15.190:3300/0 conn(0x7f31ec161b60 msgr2=0x7f31ec06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:48.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f13a7700 1 --2- 172.21.15.181:0/1206185572 >> v2:172.21.15.190:3300/0 conn(0x7f31ec161b60 0x7f31ec06d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f13a7700 1 -- 172.21.15.181:0/1206185572 >> v1:172.21.15.190:6789/0 conn(0x7f31ec161f40 legacy=0x7f31ec0d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.463+0000 7f31f13a7700 1 -- 172.21.15.181:0/1206185572 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0d45f0 con 0x7f31ec160d20 2023-10-16T10:03:48.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f0ba6700 1 --2- 172.21.15.181:0/1206185572 >> v2:172.21.15.190:3300/0 conn(0x7f31ec161b60 0x7f31ec06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:48.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f13a7700 1 --2- 172.21.15.181:0/1206185572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec1611d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009d90 tx=0x7f31e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8cfa3ff8d07e15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:48.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31eb7fe700 1 -- 172.21.15.181:0/1206185572 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400e070 con 0x7f31ec160d20 2023-10-16T10:03:48.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31eb7fe700 1 -- 172.21.15.181:0/1206185572 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400bb20 con 0x7f31ec160d20 2023-10-16T10:03:48.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31eb7fe700 1 -- 172.21.15.181:0/1206185572 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4012430 con 0x7f31ec160d20 2023-10-16T10:03:48.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 -- 172.21.15.181:0/1206185572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 msgr2=0x7f31ec1611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/1206185572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec1611d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009d90 tx=0x7f31e400b330 comp rx=0 tx=0).stop 2023-10-16T10:03:48.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 -- 172.21.15.181:0/1206185572 shutdown_connections 2023-10-16T10:03:48.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/1206185572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec1611d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/1206185572 >> v2:172.21.15.190:3300/0 conn(0x7f31ec161b60 0x7f31ec06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 -- 172.21.15.181:0/1206185572 >> 172.21.15.181:0/1206185572 conn(0x7f31ec09cba0 msgr2=0x7f31ec0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:48.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 -- 172.21.15.181:0/1206185572 shutdown_connections 2023-10-16T10:03:48.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.464+0000 7f31f2e0a700 1 -- 172.21.15.181:0/1206185572 wait complete. 2023-10-16T10:03:48.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 Processor -- start 2023-10-16T10:03:48.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 -- start start 2023-10-16T10:03:48.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ec161b60 0x7f31ec0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec161f40 0x7f31ec1f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0ca0d0 con 0x7f31ec161b60 2023-10-16T10:03:48.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0c7d40 con 0x7f31ec161f40 2023-10-16T10:03:48.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.466+0000 7f31f2e0a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0c88c0 con 0x7f31ec160d20 2023-10-16T10:03:48.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ec161b60 0x7f31ec0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ec161b60 0x7f31ec0c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48694/0 (socket says 172.21.15.181:48694) 2023-10-16T10:03:48.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 -- 172.21.15.181:0/3154273799 learned_addr learned my addr 172.21.15.181:0/3154273799 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:48.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31f0ba6700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31f13a7700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec161f40 0x7f31ec1f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 -- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 msgr2=0x7f31ec0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 -- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec161f40 msgr2=0x7f31ec1f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec161f40 0x7f31ec1f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 -- 172.21.15.181:0/3154273799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0009500 con 0x7f31ec161b60 2023-10-16T10:03:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31f13a7700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec161f40 0x7f31ec1f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31ebfff700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ec161b60 0x7f31ec0c9b90 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c580 tx=0x7f31d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:48.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8004c80 con 0x7f31ec161b60 2023-10-16T10:03:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8004e00 con 0x7f31ec161b60 2023-10-16T10:03:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e4009a70 con 0x7f31ec161b60 2023-10-16T10:03:48.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8007310 con 0x7f31ec161b60 2023-10-16T10:03:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.467+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec0c8270 con 0x7f31ec161b60 2023-10-16T10:03:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.468+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec04ebb0 con 0x7f31ec161b60 2023-10-16T10:03:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.469+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 82) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f31d8007470 con 0x7f31ec161b60 2023-10-16T10:03:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.469+0000 7f31e97fa700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d006b0b0 0x7f31d006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.469+0000 7f31f0ba6700 1 -- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d006b0b0 msgr2=0x7f31d006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3791133934 2023-10-16T10:03:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.469+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f31d8089970 con 0x7f31ec161b60 2023-10-16T10:03:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.469+0000 7f31f0ba6700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d006b0b0 0x7f31d006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:03:48.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.473+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31d8054400 con 0x7f31ec161b60 2023-10-16T10:03:48.483 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.481+0000 7fc064740200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:03:48.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.501+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 83) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f31d80547d0 con 0x7f31ec161b60 2023-10-16T10:03:48.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.501+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d006b0b0 msgr2=0x7f31d006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:03:48.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.501+0000 7f31e97fa700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d006b0b0 0x7f31d006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.566 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdff6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.567 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdff6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.567 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdff6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.567 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdff6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.567 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdff6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.568 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdcf26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.568 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdcf26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.568 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdcf26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.569 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdcf26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.569 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:03:48.565+0000 7fbfdcf26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:03:48.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.731+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31ec0c8a00 con 0x7f31ec161b60 2023-10-16T10:03:48.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.732+0000 7f31e97fa700 1 -- 172.21.15.181:0/3154273799 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f31d80851f0 con 0x7f31ec161b60 2023-10-16T10:03:48.734 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:48.734 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2508916691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2508916691},{"type":"v1","addr":"172.21.15.181:6836","nonce":2508916691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7687,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1303363384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1303363384},{"type":"v1","addr":"172.21.15.190:6834","nonce":1303363384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/300739418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":300739418},{"type":"v1","addr":"172.21.15.181:6837","nonce":300739418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:03:21.971792+0000","modified":"2023-10-16T10:03:24.995974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/1716409851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1716409851},{"type":"v1","addr":"172.21.15.190:6835","nonce":1716409851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-16T10:03:48.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ec161b60 msgr2=0x7f31ec0c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ec161b60 0x7f31ec0c9b90 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c580 tx=0x7f31d800c890 comp rx=0 tx=0).stop 2023-10-16T10:03:48.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 shutdown_connections 2023-10-16T10:03:48.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:6832/3791133934,v1:172.21.15.181:6833/3791133934] conn(0x7f31d006b0b0 0x7f31d006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31ec160d20 0x7f31ec0b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec161f40 0x7f31ec1f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 --2- 172.21.15.181:0/3154273799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ec161b60 0x7f31ec0c9b90 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 >> 172.21.15.181:0/3154273799 conn(0x7f31ec09cba0 msgr2=0x7f31ec160070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:48.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 shutdown_connections 2023-10-16T10:03:48.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.734+0000 7f31f2e0a700 1 -- 172.21.15.181:0/3154273799 wait complete. 2023-10-16T10:03:48.738 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 67 2023-10-16T10:03:48.748 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:48.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.866+0000 7f4296d79700 1 Processor -- start 2023-10-16T10:03:48.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.867+0000 7f4296d79700 1 -- start start 2023-10-16T10:03:48.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.867+0000 7f4296d79700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f429015db70 0x7f42900665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.867+0000 7f4296d79700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4290066c60 con 0x7f42900ca2e0 2023-10-16T10:03:48.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.867+0000 7f4296d79700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429006c1a0 con 0x7f42900cedd0 2023-10-16T10:03:48.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.867+0000 7f4296d79700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900cda10 con 0x7f429015db70 2023-10-16T10:03:48.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.867+0000 7f4294b15700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f429015db70 0x7f42900665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.867+0000 7f4294b15700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f429015db70 0x7f42900665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48768/0 (socket says 172.21.15.181:48768) 2023-10-16T10:03:48.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f4294b15700 1 -- 172.21.15.181:0/2574083818 learned_addr learned my addr 172.21.15.181:0/2574083818 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:48.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f4294b15700 1 -- 172.21.15.181:0/2574083818 >> v1:172.21.15.190:6789/0 conn(0x7f42900cedd0 legacy=0x7f42900cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f4294b15700 1 -- 172.21.15.181:0/2574083818 >> v1:172.21.15.181:6789/0 conn(0x7f42900ca2e0 legacy=0x7f42900ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f4294b15700 1 -- 172.21.15.181:0/2574083818 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429006b3e0 con 0x7f429015db70 2023-10-16T10:03:48.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f4294b15700 1 --2- 172.21.15.181:0/2574083818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f429015db70 0x7f42900665a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f42840098f0 tx=0x7f4284009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93831ba8bb0fdcb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:48.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f428f7fe700 1 -- 172.21.15.181:0/2574083818 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400b8a0 con 0x7f429015db70 2023-10-16T10:03:48.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f428f7fe700 1 -- 172.21.15.181:0/2574083818 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428400ba00 con 0x7f429015db70 2023-10-16T10:03:48.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.868+0000 7f428f7fe700 1 -- 172.21.15.181:0/2574083818 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400bd90 con 0x7f429015db70 2023-10-16T10:03:48.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.869+0000 7f4296d79700 1 -- 172.21.15.181:0/2574083818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f429015db70 msgr2=0x7f42900665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.869+0000 7f4296d79700 1 --2- 172.21.15.181:0/2574083818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f429015db70 0x7f42900665a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f42840098f0 tx=0x7f4284009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:48.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.869+0000 7f4296d79700 1 -- 172.21.15.181:0/2574083818 shutdown_connections 2023-10-16T10:03:48.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.869+0000 7f4296d79700 1 --2- 172.21.15.181:0/2574083818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f429015db70 0x7f42900665a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.869+0000 7f4296d79700 1 -- 172.21.15.181:0/2574083818 >> 172.21.15.181:0/2574083818 conn(0x7f42900b1780 msgr2=0x7f42900b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:48.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.869+0000 7f4296d79700 1 -- 172.21.15.181:0/2574083818 shutdown_connections 2023-10-16T10:03:48.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.869+0000 7f4296d79700 1 -- 172.21.15.181:0/2574083818 wait complete. 2023-10-16T10:03:48.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.870+0000 7f4296d79700 1 Processor -- start 2023-10-16T10:03:48.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.870+0000 7f4296d79700 1 -- start start 2023-10-16T10:03:48.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42900ca2e0 0x7f42900a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42900cedd0 0x7f42901f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429015db70 0x7f429015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:48.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42900c8f60 con 0x7f42900ca2e0 2023-10-16T10:03:48.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42900c9840 con 0x7f42900cedd0 2023-10-16T10:03:48.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42900c83e0 con 0x7f429015db70 2023-10-16T10:03:48.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42900ca2e0 0x7f42900a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42900ca2e0 0x7f42900a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48774/0 (socket says 172.21.15.181:48774) 2023-10-16T10:03:48.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 -- 172.21.15.181:0/2937057596 learned_addr learned my addr 172.21.15.181:0/2937057596 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:48.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4295316700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429015db70 0x7f429015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f428ffff700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42900cedd0 0x7f42901f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:48.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 -- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429015db70 msgr2=0x7f429015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429015db70 0x7f429015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 -- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42900cedd0 msgr2=0x7f42901f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:48.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42900cedd0 0x7f42901f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:48.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 -- 172.21.15.181:0/2937057596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284009550 con 0x7f42900ca2e0 2023-10-16T10:03:48.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f428ffff700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42900cedd0 0x7f42901f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:48.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4294b15700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42900ca2e0 0x7f42900a04a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f4284014040 tx=0x7f4284004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:48.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f428d7fa700 1 -- 172.21.15.181:0/2937057596 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42840036a0 con 0x7f42900ca2e0 2023-10-16T10:03:48.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f428d7fa700 1 -- 172.21.15.181:0/2937057596 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428401e070 con 0x7f42900ca2e0 2023-10-16T10:03:48.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f428d7fa700 1 -- 172.21.15.181:0/2937057596 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284003a60 con 0x7f42900ca2e0 2023-10-16T10:03:48.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42900c86c0 con 0x7f42900ca2e0 2023-10-16T10:03:48.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.871+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42900c8c80 con 0x7f42900ca2e0 2023-10-16T10:03:48.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.872+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4290067d20 con 0x7f42900ca2e0 2023-10-16T10:03:48.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.873+0000 7f428d7fa700 1 -- 172.21.15.181:0/2937057596 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 83) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4284003bc0 con 0x7f42900ca2e0 2023-10-16T10:03:48.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.873+0000 7f428d7fa700 1 -- 172.21.15.181:0/2937057596 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f4284020070 con 0x7f42900ca2e0 2023-10-16T10:03:48.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:48.878+0000 7f428d7fa700 1 -- 172.21.15.181:0/2937057596 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4284055a50 con 0x7f42900ca2e0 2023-10-16T10:03:49.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.091+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f429009de10 con 0x7f42900ca2e0 2023-10-16T10:03:49.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.093+0000 7f428d7fa700 1 -- 172.21.15.181:0/2937057596 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+18332 (secure 0 0 0) 0x7f4284015090 con 0x7f42900ca2e0 2023-10-16T10:03:49.095 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:49.095 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":134,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:48.495181+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:03:20.395061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-16T10:03:20.923916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/3791133934":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/2235318164":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/1126035840":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/2621214072":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/2702413404":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/2541635143":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/662310189":"2023-10-17T10:03:48.495129+0000","172.21.15.181:6832/3791133934":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/240608457":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1855739676":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/3359421143":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6833/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6832/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/2638619292":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1179812996":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1546779767":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1171463515":"2023-10-17T10:03:35.802223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42900ca2e0 msgr2=0x7f42900a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42900ca2e0 0x7f42900a04a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f4284014040 tx=0x7f4284004750 comp rx=0 tx=0).stop 2023-10-16T10:03:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 shutdown_connections 2023-10-16T10:03:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f429015db70 0x7f429015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42900ca2e0 0x7f42900a04a0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 --2- 172.21.15.181:0/2937057596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42900cedd0 0x7f42901f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 >> 172.21.15.181:0/2937057596 conn(0x7f42900b1780 msgr2=0x7f429015ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 shutdown_connections 2023-10-16T10:03:49.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.095+0000 7f4296d79700 1 -- 172.21.15.181:0/2937057596 wait complete. 2023-10-16T10:03:49.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T10:03:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.228+0000 7f89ee213700 1 Processor -- start 2023-10-16T10:03:49.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.228+0000 7f89ee213700 1 -- start start 2023-10-16T10:03:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89ee213700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e80c2cb0 0x7f89e80c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89ee213700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89e815ded0 0x7f89e806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89ee213700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e815b410 con 0x7f89e815daf0 2023-10-16T10:03:49.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89ee213700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e80c1f50 con 0x7f89e815ded0 2023-10-16T10:03:49.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89ee213700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80cfa70 con 0x7f89e80c2cb0 2023-10-16T10:03:49.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89e815ded0 0x7f89e806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89e815ded0 0x7f89e806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48784/0 (socket says 172.21.15.181:48784) 2023-10-16T10:03:49.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 -- 172.21.15.181:0/2457990196 learned_addr learned my addr 172.21.15.181:0/2457990196 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:49.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e6ffd700 1 --2- 172.21.15.181:0/2457990196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e80c2cb0 0x7f89e80c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 -- 172.21.15.181:0/2457990196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e80c2cb0 msgr2=0x7f89e80c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:49.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 --2- 172.21.15.181:0/2457990196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e80c2cb0 0x7f89e80c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 -- 172.21.15.181:0/2457990196 >> v1:172.21.15.181:6789/0 conn(0x7f89e815daf0 legacy=0x7f89e80c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 -- 172.21.15.181:0/2457990196 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80c2a80 con 0x7f89e815ded0 2023-10-16T10:03:49.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.229+0000 7f89e7fff700 1 --2- 172.21.15.181:0/2457990196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e815ded0 0x7f89e806a8a0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e80 tx=0x7f89e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38833838fbb428be server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:49.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89e67fc700 1 -- 172.21.15.181:0/2457990196 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000b8d0 con 0x7f89e815ded0 2023-10-16T10:03:49.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89e67fc700 1 -- 172.21.15.181:0/2457990196 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000ba30 con 0x7f89e815ded0 2023-10-16T10:03:49.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89e67fc700 1 -- 172.21.15.181:0/2457990196 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000bde0 con 0x7f89e815ded0 2023-10-16T10:03:49.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 -- 172.21.15.181:0/2457990196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e815ded0 msgr2=0x7f89e806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 --2- 172.21.15.181:0/2457990196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e815ded0 0x7f89e806a8a0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e80 tx=0x7f89e000b330 comp rx=0 tx=0).stop 2023-10-16T10:03:49.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 -- 172.21.15.181:0/2457990196 shutdown_connections 2023-10-16T10:03:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 --2- 172.21.15.181:0/2457990196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e815ded0 0x7f89e806a8a0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 --2- 172.21.15.181:0/2457990196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e80c2cb0 0x7f89e80c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 -- 172.21.15.181:0/2457990196 >> 172.21.15.181:0/2457990196 conn(0x7f89e80b1780 msgr2=0x7f89e80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 -- 172.21.15.181:0/2457990196 shutdown_connections 2023-10-16T10:03:49.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.230+0000 7f89ee213700 1 -- 172.21.15.181:0/2457990196 wait complete. 2023-10-16T10:03:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.231+0000 7f89ee213700 1 Processor -- start 2023-10-16T10:03:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.231+0000 7f89ee213700 1 -- start start 2023-10-16T10:03:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89ee213700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e80c2cb0 0x7f89e80c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89ee213700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89e815daf0 0x7f89e80ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89ee213700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e815ded0 0x7f89e80c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89ee213700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80aeeb0 con 0x7f89e80c2cb0 2023-10-16T10:03:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89ee213700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c3f10 con 0x7f89e815daf0 2023-10-16T10:03:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89ee213700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c36b0 con 0x7f89e815ded0 2023-10-16T10:03:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e80c2cb0 0x7f89e80c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e80c2cb0 0x7f89e80c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48786/0 (socket says 172.21.15.181:48786) 2023-10-16T10:03:49.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89e77fe700 1 -- 172.21.15.181:0/1528097007 learned_addr learned my addr 172.21.15.181:0/1528097007 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:49.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89e7fff700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e815ded0 0x7f89e80c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.232+0000 7f89e6ffd700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89e815daf0 0x7f89e80ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89e77fe700 1 -- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e815ded0 msgr2=0x7f89e80c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89e77fe700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e815ded0 0x7f89e80c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89e77fe700 1 -- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89e815daf0 msgr2=0x7f89e80ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89e77fe700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89e815daf0 0x7f89e80ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89e77fe700 1 -- 172.21.15.181:0/1528097007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e0009b30 con 0x7f89e80c2cb0 2023-10-16T10:03:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89e6ffd700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89e815daf0 0x7f89e80ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:49.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89e77fe700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e80c2cb0 0x7f89e80c6410 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c4e0 tx=0x7f89dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89cffff700 1 -- 172.21.15.181:0/1528097007 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0052d0 con 0x7f89e80c2cb0 2023-10-16T10:03:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89cffff700 1 -- 172.21.15.181:0/1528097007 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc005430 con 0x7f89e80c2cb0 2023-10-16T10:03:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e80c3950 con 0x7f89e80c2cb0 2023-10-16T10:03:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89cffff700 1 -- 172.21.15.181:0/1528097007 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0038c0 con 0x7f89e80c2cb0 2023-10-16T10:03:49.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.233+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e81e4390 con 0x7f89e80c2cb0 2023-10-16T10:03:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.234+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e80af460 con 0x7f89e80c2cb0 2023-10-16T10:03:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.234+0000 7f89cffff700 1 -- 172.21.15.181:0/1528097007 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 83) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f89dc003a20 con 0x7f89e80c2cb0 2023-10-16T10:03:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.235+0000 7f89cffff700 1 -- 172.21.15.181:0/1528097007 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6892+0+0 (secure 0 0 0) 0x7f89dc088980 con 0x7f89e80c2cb0 2023-10-16T10:03:49.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.239+0000 7f89cffff700 1 -- 172.21.15.181:0/1528097007 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f89dc0539e0 con 0x7f89e80c2cb0 2023-10-16T10:03:49.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.475+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f89e81e41f0 con 0x7f89e80c2cb0 2023-10-16T10:03:49.477 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T10:03:49.476+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T10:03:49.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.477+0000 7f89cffff700 1 -- 172.21.15.181:0/1528097007 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) v1 ==== 161+0+0 (secure 0 0 0) 0x7f89e81e41f0 con 0x7f89e80c2cb0 2023-10-16T10:03:49.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e80c2cb0 msgr2=0x7f89e80c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e80c2cb0 0x7f89e80c6410 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c4e0 tx=0x7f89dc00c7f0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 shutdown_connections 2023-10-16T10:03:49.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89e815ded0 0x7f89e80c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e80c2cb0 0x7f89e80c6410 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 --2- 172.21.15.181:0/1528097007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89e815daf0 0x7f89e80ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 >> 172.21.15.181:0/1528097007 conn(0x7f89e80b1780 msgr2=0x7f89e806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:49.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 shutdown_connections 2023-10-16T10:03:49.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.480+0000 7f89ee213700 1 -- 172.21.15.181:0/1528097007 wait complete. 2023-10-16T10:03:49.484 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T10:03:49.485 INFO:tasks.ceph.mds.b.smithi190.stderr: -9814> 2023-10-16T10:03:24.556+0000 7f27a4493700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:03:49.494 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T10:03:49.522 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T10:03:49.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.620+0000 7f24fb861700 1 Processor -- start 2023-10-16T10:03:49.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.620+0000 7f24fb861700 1 -- start start 2023-10-16T10:03:49.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24fb861700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24f415b850 0x7f24f40665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24fb861700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24f406f1e0 0x7f24f406f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24fb861700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f406e4b0 con 0x7f24f415fee0 2023-10-16T10:03:49.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24fb861700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f406ed50 con 0x7f24f415b850 2023-10-16T10:03:49.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24fb861700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f40cb0f0 con 0x7f24f406f1e0 2023-10-16T10:03:49.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24f95fd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24f415b850 0x7f24f40665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24f95fd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24f415b850 0x7f24f40665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48814/0 (socket says 172.21.15.181:48814) 2023-10-16T10:03:49.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24f95fd700 1 -- 172.21.15.181:0/2659919258 learned_addr learned my addr 172.21.15.181:0/2659919258 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:49.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24f9dfe700 1 --2- 172.21.15.181:0/2659919258 >> v2:172.21.15.190:3300/0 conn(0x7f24f406f1e0 0x7f24f406f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24f95fd700 1 -- 172.21.15.181:0/2659919258 >> v2:172.21.15.190:3300/0 conn(0x7f24f406f1e0 msgr2=0x7f24f406f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:49.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24f95fd700 1 --2- 172.21.15.181:0/2659919258 >> v2:172.21.15.190:3300/0 conn(0x7f24f406f1e0 0x7f24f406f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.621+0000 7f24f95fd700 1 -- 172.21.15.181:0/2659919258 >> v1:172.21.15.190:6789/0 conn(0x7f24f415fee0 legacy=0x7f24f40c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:49.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24f95fd700 1 -- 172.21.15.181:0/2659919258 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4066b80 con 0x7f24f415b850 2023-10-16T10:03:49.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24f95fd700 1 --2- 172.21.15.181:0/2659919258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f40665a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f24f406c3b0 tx=0x7f24e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d14863b08c7f27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:49.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24ebfff700 1 -- 172.21.15.181:0/2659919258 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4004230 con 0x7f24f415b850 2023-10-16T10:03:49.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24ebfff700 1 -- 172.21.15.181:0/2659919258 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e4004390 con 0x7f24f415b850 2023-10-16T10:03:49.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24ebfff700 1 -- 172.21.15.181:0/2659919258 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e40036a0 con 0x7f24f415b850 2023-10-16T10:03:49.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24fb861700 1 -- 172.21.15.181:0/2659919258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 msgr2=0x7f24f40665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24fb861700 1 --2- 172.21.15.181:0/2659919258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f40665a0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f24f406c3b0 tx=0x7f24e4009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24fb861700 1 -- 172.21.15.181:0/2659919258 shutdown_connections 2023-10-16T10:03:49.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24fb861700 1 --2- 172.21.15.181:0/2659919258 >> v2:172.21.15.190:3300/0 conn(0x7f24f406f1e0 0x7f24f406f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24fb861700 1 --2- 172.21.15.181:0/2659919258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f40665a0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24fb861700 1 -- 172.21.15.181:0/2659919258 >> 172.21.15.181:0/2659919258 conn(0x7f24f40cc730 msgr2=0x7f24f40ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:49.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.622+0000 7f24fb861700 1 -- 172.21.15.181:0/2659919258 shutdown_connections 2023-10-16T10:03:49.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.623+0000 7f24fb861700 1 -- 172.21.15.181:0/2659919258 wait complete. 2023-10-16T10:03:49.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 Processor -- start 2023-10-16T10:03:49.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 -- start start 2023-10-16T10:03:49.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f406f1e0 0x7f24f414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f41f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f415fee0 0x7f24f41f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41520f0 con 0x7f24f415b850 2023-10-16T10:03:49.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41f1ff0 con 0x7f24f415fee0 2023-10-16T10:03:49.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.624+0000 7f24fb861700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41f22d0 con 0x7f24f406f1e0 2023-10-16T10:03:49.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f41f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f41f14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48822/0 (socket says 172.21.15.181:48822) 2023-10-16T10:03:49.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 -- 172.21.15.181:0/1427274451 learned_addr learned my addr 172.21.15.181:0/1427274451 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f95fd700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f406f1e0 0x7f24f414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f9dfe700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f415fee0 0x7f24f41f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 -- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f406f1e0 msgr2=0x7f24f414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f406f1e0 0x7f24f414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 -- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f415fee0 msgr2=0x7f24f41f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:49.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f415fee0 0x7f24f41f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:49.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 -- 172.21.15.181:0/1427274451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4009550 con 0x7f24f415b850 2023-10-16T10:03:49.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f9dfe700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f415fee0 0x7f24f41f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:49.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24f8dfc700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f41f14d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c4d0 tx=0x7f24f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:49.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24e9ffb700 1 -- 172.21.15.181:0/1427274451 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0011790 con 0x7f24f415b850 2023-10-16T10:03:49.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24e9ffb700 1 -- 172.21.15.181:0/1427274451 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f00118f0 con 0x7f24f415b850 2023-10-16T10:03:49.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f4151d10 con 0x7f24f415b850 2023-10-16T10:03:49.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24e9ffb700 1 -- 172.21.15.181:0/1427274451 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0011ca0 con 0x7f24f415b850 2023-10-16T10:03:49.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.625+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f41f25b0 con 0x7f24f415b850 2023-10-16T10:03:49.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.626+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f40cd5d0 con 0x7f24f415b850 2023-10-16T10:03:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.627+0000 7f24e9ffb700 1 -- 172.21.15.181:0/1427274451 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 84) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f24f001c460 con 0x7f24f415b850 2023-10-16T10:03:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.627+0000 7f24e9ffb700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f24d806ad50 0x7f24d806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.627+0000 7f24e9ffb700 1 -- 172.21.15.181:0/1427274451 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6978+0+0 (secure 0 0 0) 0x7f24f008a000 con 0x7f24f415b850 2023-10-16T10:03:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.627+0000 7f24f95fd700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f24d806ad50 0x7f24d806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:49.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.628+0000 7f24f95fd700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f24d806ad50 0x7f24d806d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009130 tx=0x7f24e4008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:49.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.632+0000 7f24e9ffb700 1 -- 172.21.15.181:0/1427274451 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f24f0054fb0 con 0x7f24f415b850 2023-10-16T10:03:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:49.871+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f24f415d0c0 con 0x7f24f415b850 2023-10-16T10:03:50.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.504+0000 7f24e9ffb700 1 -- 172.21.15.181:0/1427274451 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7f24f00585d0 con 0x7f24f415b850 2023-10-16T10:03:50.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f24d806ad50 msgr2=0x7f24d806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:50.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f24d806ad50 0x7f24d806d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009130 tx=0x7f24e4008030 comp rx=0 tx=0).stop 2023-10-16T10:03:50.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 msgr2=0x7f24f41f14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:50.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f41f14d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c4d0 tx=0x7f24f000c890 comp rx=0 tx=0).stop 2023-10-16T10:03:50.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 shutdown_connections 2023-10-16T10:03:50.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f24d806ad50 0x7f24d806d210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:50.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24f406f1e0 0x7f24f414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:50.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f415fee0 0x7f24f41f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:50.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 --2- 172.21.15.181:0/1427274451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24f415b850 0x7f24f41f14d0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:50.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 >> 172.21.15.181:0/1427274451 conn(0x7f24f40cc730 msgr2=0x7f24f415e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:50.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 shutdown_connections 2023-10-16T10:03:50.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.508+0000 7f24fb861700 1 -- 172.21.15.181:0/1427274451 wait complete. 2023-10-16T10:03:50.525 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T10:03:50.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.657+0000 7fd508a7d700 1 Processor -- start 2023-10-16T10:03:50.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.657+0000 7fd508a7d700 1 -- start start 2023-10-16T10:03:50.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd508a7d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5040b03e0 0x7fd504166950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:50.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd508a7d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5040d23c0 con 0x7fd5040b0f70 2023-10-16T10:03:50.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd508a7d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5040cd990 con 0x7fd50415eea0 2023-10-16T10:03:50.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd508a7d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5040b07f0 con 0x7fd5040b03e0 2023-10-16T10:03:50.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd502d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5040b03e0 0x7fd504166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:50.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd501d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd5040b0f70 0x7fd5040ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:59924/0 (socket says 172.21.15.181:59924) 2023-10-16T10:03:50.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd501d9b700 1 -- 172.21.15.181:0/2656894401 learned_addr learned my addr 172.21.15.181:0/2656894401 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:50.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd502d9d700 1 -- 172.21.15.181:0/2656894401 >> v1:172.21.15.190:6789/0 conn(0x7fd50415eea0 legacy=0x7fd5040ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:50.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd502d9d700 1 -- 172.21.15.181:0/2656894401 >> v1:172.21.15.181:6789/0 conn(0x7fd5040b0f70 legacy=0x7fd5040ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:50.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd502d9d700 1 -- 172.21.15.181:0/2656894401 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040cdc70 con 0x7fd5040b03e0 2023-10-16T10:03:50.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd50159a700 1 -- 172.21.15.181:0/2656894401 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259178481 0 0) 0x7fd5040d23c0 con 0x7fd5040b0f70 2023-10-16T10:03:50.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.658+0000 7fd502d9d700 1 --2- 172.21.15.181:0/2656894401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040b03e0 0x7fd504166950 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009d90 tx=0x7fd4fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec52b23b211f7a99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:50.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd50159a700 1 -- 172.21.15.181:0/2656894401 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00e070 con 0x7fd5040b03e0 2023-10-16T10:03:50.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd50159a700 1 -- 172.21.15.181:0/2656894401 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc00bbd0 con 0x7fd5040b03e0 2023-10-16T10:03:50.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd50159a700 1 -- 172.21.15.181:0/2656894401 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc012430 con 0x7fd5040b03e0 2023-10-16T10:03:50.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd508a7d700 1 -- 172.21.15.181:0/2656894401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040b03e0 msgr2=0x7fd504166950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:50.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd508a7d700 1 --2- 172.21.15.181:0/2656894401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040b03e0 0x7fd504166950 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009d90 tx=0x7fd4fc00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:50.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd508a7d700 1 -- 172.21.15.181:0/2656894401 shutdown_connections 2023-10-16T10:03:50.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd508a7d700 1 --2- 172.21.15.181:0/2656894401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040b03e0 0x7fd504166950 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:50.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd508a7d700 1 -- 172.21.15.181:0/2656894401 >> 172.21.15.181:0/2656894401 conn(0x7fd5040d1610 msgr2=0x7fd5040d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:50.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd508a7d700 1 -- 172.21.15.181:0/2656894401 shutdown_connections 2023-10-16T10:03:50.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.659+0000 7fd508a7d700 1 -- 172.21.15.181:0/2656894401 wait complete. 2023-10-16T10:03:50.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 Processor -- start 2023-10-16T10:03:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 -- start start 2023-10-16T10:03:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5040b03e0 0x7fd504147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5040b0f70 0x7fd5040ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50415eea0 0x7fd5040caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:50.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd504148270 con 0x7fd50415eea0 2023-10-16T10:03:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040cb0d0 con 0x7fd5040b0f70 2023-10-16T10:03:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd508a7d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040cb3b0 con 0x7fd5040b03e0 2023-10-16T10:03:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd502d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50415eea0 0x7fd5040caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd502d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50415eea0 0x7fd5040caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48854/0 (socket says 172.21.15.181:48854) 2023-10-16T10:03:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd502d9d700 1 -- 172.21.15.181:0/3423360093 learned_addr learned my addr 172.21.15.181:0/3423360093 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:50.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd501d9b700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5040b0f70 0x7fd5040ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.661+0000 7fd50259c700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5040b03e0 0x7fd504147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd502d9d700 1 -- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5040b03e0 msgr2=0x7fd504147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd502d9d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5040b03e0 0x7fd504147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd502d9d700 1 -- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5040b0f70 msgr2=0x7fd5040ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:50.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd502d9d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5040b0f70 0x7fd5040ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd502d9d700 1 -- 172.21.15.181:0/3423360093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc009a70 con 0x7fd50415eea0 2023-10-16T10:03:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd502d9d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50415eea0 0x7fd5040caa90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc014040 tx=0x7fd4fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd4f2ffd700 1 -- 172.21.15.181:0/3423360093 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc01b810 con 0x7fd50415eea0 2023-10-16T10:03:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd4f2ffd700 1 -- 172.21.15.181:0/3423360093 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc020070 con 0x7fd50415eea0 2023-10-16T10:03:50.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd4f2ffd700 1 -- 172.21.15.181:0/3423360093 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc01bbb0 con 0x7fd50415eea0 2023-10-16T10:03:50.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5040b4950 con 0x7fd50415eea0 2023-10-16T10:03:50.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.662+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5040cb690 con 0x7fd50415eea0 2023-10-16T10:03:50.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.663+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd504067d20 con 0x7fd50415eea0 2023-10-16T10:03:50.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.664+0000 7fd4f2ffd700 1 -- 172.21.15.181:0/3423360093 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 84) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd4fc01bd10 con 0x7fd50415eea0 2023-10-16T10:03:50.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.664+0000 7fd4f2ffd700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd4e406ad50 0x7fd4e406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:50.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.664+0000 7fd4f2ffd700 1 -- 172.21.15.181:0/3423360093 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6978+0+0 (secure 0 0 0) 0x7fd4fc022070 con 0x7fd50415eea0 2023-10-16T10:03:50.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.664+0000 7fd50259c700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd4e406ad50 0x7fd4e406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:50.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.665+0000 7fd50259c700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd4e406ad50 0x7fd4e406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4007fd0 tx=0x7fd4f4009040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:50.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.669+0000 7fd4f2ffd700 1 -- 172.21.15.181:0/3423360093 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd4fc056680 con 0x7fd50415eea0 2023-10-16T10:03:50.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:50.860+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd50406eb60 con 0x7fd50415eea0 2023-10-16T10:03:51.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.510+0000 7fd4f2ffd700 1 -- 172.21.15.181:0/3423360093 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd4fc02c0d0 con 0x7fd50415eea0 2023-10-16T10:03:51.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.514+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd4e406ad50 msgr2=0x7fd4e406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:51.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.514+0000 7fd508a7d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd4e406ad50 0x7fd4e406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4007fd0 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2023-10-16T10:03:51.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.514+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50415eea0 msgr2=0x7fd5040caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:51.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.514+0000 7fd508a7d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50415eea0 0x7fd5040caa90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc014040 tx=0x7fd4fc004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.516+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 shutdown_connections 2023-10-16T10:03:51.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.516+0000 7fd508a7d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd4e406ad50 0x7fd4e406d210 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4007fd0 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2023-10-16T10:03:51.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.516+0000 7fd508a7d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5040b03e0 0x7fd504147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.516+0000 7fd508a7d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd50415eea0 0x7fd5040caa90 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.516+0000 7fd508a7d700 1 --2- 172.21.15.181:0/3423360093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5040b0f70 0x7fd5040ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.516+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 >> 172.21.15.181:0/3423360093 conn(0x7fd5040d1610 msgr2=0x7fd50406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:51.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.518+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 shutdown_connections 2023-10-16T10:03:51.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.518+0000 7fd508a7d700 1 -- 172.21.15.181:0/3423360093 wait complete. 2023-10-16T10:03:51.521 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T10:03:51.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T10:03:51.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.659+0000 7f4f0e0a4700 1 Processor -- start 2023-10-16T10:03:51.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.659+0000 7f4f0e0a4700 1 -- start start 2023-10-16T10:03:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f0e0a4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f080ce610 0x7f4f0806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f0e0a4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f080d0fc0 0x7f4f080cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f0e0a4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080d0060 0x7f4f0806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f0e0a4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f08162190 con 0x7f4f080d0fc0 2023-10-16T10:03:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f0e0a4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f08162650 con 0x7f4f080ce610 2023-10-16T10:03:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f0e0a4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08162370 con 0x7f4f080d0060 2023-10-16T10:03:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f080d0fc0 0x7f4f080cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f080d0fc0 0x7f4f080cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48860/0 (socket says 172.21.15.181:48860) 2023-10-16T10:03:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 -- 172.21.15.181:0/2945174444 learned_addr learned my addr 172.21.15.181:0/2945174444 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:51.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 -- 172.21.15.181:0/2945174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080d0060 msgr2=0x7f4f0806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f077fe700 1 --2- 172.21.15.181:0/2945174444 >> v2:172.21.15.190:3300/0 conn(0x7f4f080ce610 0x7f4f0806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f07fff700 1 --2- 172.21.15.181:0/2945174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080d0060 0x7f4f0806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 --2- 172.21.15.181:0/2945174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080d0060 0x7f4f0806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 -- 172.21.15.181:0/2945174444 >> v2:172.21.15.190:3300/0 conn(0x7f4f080ce610 msgr2=0x7f4f0806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:51.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 --2- 172.21.15.181:0/2945174444 >> v2:172.21.15.190:3300/0 conn(0x7f4f080ce610 0x7f4f0806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f06ffd700 1 -- 172.21.15.181:0/2945174444 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0806f0f0 con 0x7f4f080d0fc0 2023-10-16T10:03:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.660+0000 7f4f077fe700 1 --2- 172.21.15.181:0/2945174444 >> v2:172.21.15.190:3300/0 conn(0x7f4f080ce610 0x7f4f0806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f06ffd700 1 --2- 172.21.15.181:0/2945174444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080d0fc0 0x7f4f080cfb20 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f4f080cdf30 tx=0x7f4ef8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d267f25c032a98b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f067fc700 1 -- 172.21.15.181:0/2945174444 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800b870 con 0x7f4f080d0fc0 2023-10-16T10:03:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f067fc700 1 -- 172.21.15.181:0/2945174444 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef800b9d0 con 0x7f4f080d0fc0 2023-10-16T10:03:51.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f067fc700 1 -- 172.21.15.181:0/2945174444 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800bd60 con 0x7f4f080d0fc0 2023-10-16T10:03:51.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/2945174444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080d0fc0 msgr2=0x7f4f080cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:51.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/2945174444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080d0fc0 0x7f4f080cfb20 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f4f080cdf30 tx=0x7f4ef8009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/2945174444 shutdown_connections 2023-10-16T10:03:51.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/2945174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080d0060 0x7f4f0806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/2945174444 >> v2:172.21.15.190:3300/0 conn(0x7f4f080ce610 0x7f4f0806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/2945174444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080d0fc0 0x7f4f080cfb20 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/2945174444 >> 172.21.15.181:0/2945174444 conn(0x7f4f080d04a0 msgr2=0x7f4f080d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:51.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/2945174444 shutdown_connections 2023-10-16T10:03:51.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.661+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/2945174444 wait complete. 2023-10-16T10:03:51.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 Processor -- start 2023-10-16T10:03:51.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 -- start start 2023-10-16T10:03:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080ce610 0x7f4f0815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f080d0fc0 0x7f4f0809a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080cb5c0 0x7f4f0809b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:51.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080c6ab0 con 0x7f4f080ce610 2023-10-16T10:03:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0815bd90 con 0x7f4f080d0fc0 2023-10-16T10:03:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.663+0000 7f4f0e0a4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0809a6c0 con 0x7f4f080cb5c0 2023-10-16T10:03:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080ce610 0x7f4f0815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080ce610 0x7f4f0815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48866/0 (socket says 172.21.15.181:48866) 2023-10-16T10:03:51.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 -- 172.21.15.181:0/885583344 learned_addr learned my addr 172.21.15.181:0/885583344 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:51.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f07fff700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080cb5c0 0x7f4f0809b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:51.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f06ffd700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f080d0fc0 0x7f4f0809a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:51.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 -- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080cb5c0 msgr2=0x7f4f0809b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:51.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080cb5c0 0x7f4f0809b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 -- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f080d0fc0 msgr2=0x7f4f0809a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:51.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f080d0fc0 0x7f4f0809a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:51.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 -- 172.21.15.181:0/885583344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8009550 con 0x7f4f080ce610 2023-10-16T10:03:51.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f07fff700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080cb5c0 0x7f4f0809b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:51.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f077fe700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080ce610 0x7f4f0815b850 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f4efc006fd0 tx=0x7f4efc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:51.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4eeffff700 1 -- 172.21.15.181:0/885583344 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc005c20 con 0x7f4f080ce610 2023-10-16T10:03:51.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4eeffff700 1 -- 172.21.15.181:0/885583344 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc005d80 con 0x7f4f080ce610 2023-10-16T10:03:51.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0809a8f0 con 0x7f4f080ce610 2023-10-16T10:03:51.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4eeffff700 1 -- 172.21.15.181:0/885583344 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc011840 con 0x7f4f080ce610 2023-10-16T10:03:51.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.664+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0809aeb0 con 0x7f4f080ce610 2023-10-16T10:03:51.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.665+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f08067d20 con 0x7f4f080ce610 2023-10-16T10:03:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.666+0000 7f4eeffff700 1 -- 172.21.15.181:0/885583344 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 85) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4efc0119a0 con 0x7f4f080ce610 2023-10-16T10:03:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.666+0000 7f4eeffff700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4ef406af90 0x7f4ef406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.666+0000 7f4eeffff700 1 -- 172.21.15.181:0/885583344 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6540+0+0 (secure 0 0 0) 0x7f4efc089a30 con 0x7f4f080ce610 2023-10-16T10:03:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.666+0000 7f4f06ffd700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4ef406af90 0x7f4ef406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:51.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.666+0000 7f4f06ffd700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4ef406af90 0x7f4ef406d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f080d13f0 tx=0x7f4ef8008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:51.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.670+0000 7f4eeffff700 1 -- 172.21.15.181:0/885583344 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4efc0546a0 con 0x7f4f080ce610 2023-10-16T10:03:51.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:51.864+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4f080b2d80 con 0x7f4f080ce610 2023-10-16T10:03:52.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.511+0000 7f4eeffff700 1 -- 172.21.15.181:0/885583344 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4efc057cc0 con 0x7f4f080ce610 2023-10-16T10:03:52.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4ef406af90 msgr2=0x7f4ef406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:52.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4ef406af90 0x7f4ef406d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f080d13f0 tx=0x7f4ef8008030 comp rx=0 tx=0).stop 2023-10-16T10:03:52.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080ce610 msgr2=0x7f4f0815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:52.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080ce610 0x7f4f0815b850 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f4efc006fd0 tx=0x7f4efc00cae0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 shutdown_connections 2023-10-16T10:03:52.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4ef406af90 0x7f4ef406d450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f080cb5c0 0x7f4f0809b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f080ce610 0x7f4f0815b850 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 --2- 172.21.15.181:0/885583344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f080d0fc0 0x7f4f0809a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.513+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 >> 172.21.15.181:0/885583344 conn(0x7f4f080d04a0 msgr2=0x7f4f0815da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:52.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.514+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 shutdown_connections 2023-10-16T10:03:52.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.514+0000 7f4f0e0a4700 1 -- 172.21.15.181:0/885583344 wait complete. 2023-10-16T10:03:52.518 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T10:03:52.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T10:03:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.654+0000 7f094aca2700 1 Processor -- start 2023-10-16T10:03:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.654+0000 7f094aca2700 1 -- start start 2023-10-16T10:03:52.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f094aca2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0944072760 0x7f094406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:52.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f094aca2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f094406ade0 0x7f09441615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:52.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f094aca2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094406b210 con 0x7f094415da70 2023-10-16T10:03:52.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f094aca2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09440cef00 con 0x7f0944072760 2023-10-16T10:03:52.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f094aca2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09440cec20 con 0x7f094406ade0 2023-10-16T10:03:52.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f0943fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0944072760 0x7f094406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:52.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f0943fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0944072760 0x7f094406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48874/0 (socket says 172.21.15.181:48874) 2023-10-16T10:03:52.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f0943fff700 1 -- 172.21.15.181:0/2065320389 learned_addr learned my addr 172.21.15.181:0/2065320389 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:52.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f0943fff700 1 -- 172.21.15.181:0/2065320389 >> v2:172.21.15.190:3300/0 conn(0x7f094406ade0 msgr2=0x7f09441615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:52.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f094923f700 1 --2- 172.21.15.181:0/2065320389 >> v2:172.21.15.190:3300/0 conn(0x7f094406ade0 0x7f09441615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:52.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f0943fff700 1 --2- 172.21.15.181:0/2065320389 >> v2:172.21.15.190:3300/0 conn(0x7f094406ade0 0x7f09441615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f0943fff700 1 -- 172.21.15.181:0/2065320389 >> v1:172.21.15.190:6789/0 conn(0x7f094415da70 legacy=0x7f09440ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:52.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f0943fff700 1 -- 172.21.15.181:0/2065320389 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944071c80 con 0x7f0944072760 2023-10-16T10:03:52.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.655+0000 7f094923f700 1 --2- 172.21.15.181:0/2065320389 >> v2:172.21.15.190:3300/0 conn(0x7f094406ade0 0x7f09441615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:52.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f0943fff700 1 --2- 172.21.15.181:0/2065320389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0944072760 0x7f094406a8a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f09440b66b0 tx=0x7f0930009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcbdaba59dc0e2b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:52.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f09437fe700 1 -- 172.21.15.181:0/2065320389 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930004230 con 0x7f0944072760 2023-10-16T10:03:52.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f09437fe700 1 -- 172.21.15.181:0/2065320389 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930004390 con 0x7f0944072760 2023-10-16T10:03:52.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f09437fe700 1 -- 172.21.15.181:0/2065320389 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09300036a0 con 0x7f0944072760 2023-10-16T10:03:52.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 -- 172.21.15.181:0/2065320389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0944072760 msgr2=0x7f094406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:52.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 --2- 172.21.15.181:0/2065320389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0944072760 0x7f094406a8a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f09440b66b0 tx=0x7f0930009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 -- 172.21.15.181:0/2065320389 shutdown_connections 2023-10-16T10:03:52.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 --2- 172.21.15.181:0/2065320389 >> v2:172.21.15.190:3300/0 conn(0x7f094406ade0 0x7f09441615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 --2- 172.21.15.181:0/2065320389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0944072760 0x7f094406a8a0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 -- 172.21.15.181:0/2065320389 >> 172.21.15.181:0/2065320389 conn(0x7f09440cd010 msgr2=0x7f09440cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 -- 172.21.15.181:0/2065320389 shutdown_connections 2023-10-16T10:03:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.656+0000 7f094aca2700 1 -- 172.21.15.181:0/2065320389 wait complete. 2023-10-16T10:03:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.657+0000 7f094aca2700 1 Processor -- start 2023-10-16T10:03:52.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.657+0000 7f094aca2700 1 -- start start 2023-10-16T10:03:52.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.658+0000 7f094aca2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f094406ade0 0x7f09441f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:52.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.658+0000 7f094aca2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0944072760 0x7f09441f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:52.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.658+0000 7f094aca2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094415da70 0x7f09440b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:52.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.658+0000 7f094aca2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441f45e0 con 0x7f094406ade0 2023-10-16T10:03:52.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.658+0000 7f094aca2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441f72b0 con 0x7f0944072760 2023-10-16T10:03:52.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.658+0000 7f094aca2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09441f7590 con 0x7f094415da70 2023-10-16T10:03:52.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.658+0000 7f0948a3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f094406ade0 0x7f09441f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f094406ade0 0x7f09441f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48878/0 (socket says 172.21.15.181:48878) 2023-10-16T10:03:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 -- 172.21.15.181:0/3505201249 learned_addr learned my addr 172.21.15.181:0/3505201249 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0943fff700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0944072760 0x7f09441f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f094923f700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094415da70 0x7f09440b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:52.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 -- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094415da70 msgr2=0x7f09440b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094415da70 0x7f09440b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 -- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0944072760 msgr2=0x7f09441f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0944072760 0x7f09441f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 -- 172.21.15.181:0/3505201249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009550 con 0x7f094406ade0 2023-10-16T10:03:52.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0943fff700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0944072760 0x7f09441f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f0948a3e700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f094406ade0 0x7f09441f6bf0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f093800c4d0 tx=0x7f093800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f09417fa700 1 -- 172.21.15.181:0/3505201249 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938011790 con 0x7f094406ade0 2023-10-16T10:03:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f09417fa700 1 -- 172.21.15.181:0/3505201249 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09380118f0 con 0x7f094406ade0 2023-10-16T10:03:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f09417fa700 1 -- 172.21.15.181:0/3505201249 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938011ca0 con 0x7f094406ade0 2023-10-16T10:03:52.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09441f7870 con 0x7f094406ade0 2023-10-16T10:03:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.659+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09440b20f0 con 0x7f094406ade0 2023-10-16T10:03:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.660+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09440b22b0 con 0x7f094406ade0 2023-10-16T10:03:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.661+0000 7f09417fa700 1 -- 172.21.15.181:0/3505201249 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 85) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f093801c460 con 0x7f094406ade0 2023-10-16T10:03:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.661+0000 7f09417fa700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:52.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.661+0000 7f09417fa700 1 -- 172.21.15.181:0/3505201249 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f0938089570 con 0x7f094406ade0 2023-10-16T10:03:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.661+0000 7f0943fff700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.661+0000 7f0943fff700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f092806af90 0x7f092806d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0930009130 tx=0x7f0930008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:52.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.665+0000 7f09417fa700 1 -- 172.21.15.181:0/3505201249 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0938054520 con 0x7f094406ade0 2023-10-16T10:03:52.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.863+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f0944074db0 con 0x7f094406ade0 2023-10-16T10:03:52.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.869+0000 7f09417fa700 1 -- 172.21.15.181:0/3505201249 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v49) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0938057b40 con 0x7f094406ade0 2023-10-16T10:03:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f092806af90 msgr2=0x7f092806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:52.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f092806af90 0x7f092806d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0930009130 tx=0x7f0930008030 comp rx=0 tx=0).stop 2023-10-16T10:03:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f094406ade0 msgr2=0x7f09441f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f094406ade0 0x7f09441f6bf0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f093800c4d0 tx=0x7f093800c890 comp rx=0 tx=0).stop 2023-10-16T10:03:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 shutdown_connections 2023-10-16T10:03:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094415da70 0x7f09440b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f094406ade0 0x7f09441f6bf0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.871+0000 7f094aca2700 1 --2- 172.21.15.181:0/3505201249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0944072760 0x7f09441f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.872+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 >> 172.21.15.181:0/3505201249 conn(0x7f09440cd010 msgr2=0x7f09440cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.872+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 shutdown_connections 2023-10-16T10:03:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:52.872+0000 7f094aca2700 1 -- 172.21.15.181:0/3505201249 wait complete. 2023-10-16T10:03:52.886 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T10:03:53.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.007+0000 7f3adb995700 1 Processor -- start 2023-10-16T10:03:53.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.007+0000 7f3adb995700 1 -- start start 2023-10-16T10:03:53.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3adb995700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ad4072760 0x7f3ad406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3adb995700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad406ade0 0x7f3ad41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3adb995700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad406b210 con 0x7f3ad415da70 2023-10-16T10:03:53.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3adb995700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad40cef00 con 0x7f3ad4072760 2023-10-16T10:03:53.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3adb995700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40cec20 con 0x7f3ad406ade0 2023-10-16T10:03:53.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ad4072760 0x7f3ad406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ad4072760 0x7f3ad406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48888/0 (socket says 172.21.15.181:48888) 2023-10-16T10:03:53.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 -- 172.21.15.181:0/2361801571 learned_addr learned my addr 172.21.15.181:0/2361801571 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:53.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad9f32700 1 --2- 172.21.15.181:0/2361801571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad406ade0 0x7f3ad41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 -- 172.21.15.181:0/2361801571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad406ade0 msgr2=0x7f3ad41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:53.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 --2- 172.21.15.181:0/2361801571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad406ade0 0x7f3ad41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 -- 172.21.15.181:0/2361801571 >> v1:172.21.15.190:6789/0 conn(0x7f3ad415da70 legacy=0x7f3ad40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 -- 172.21.15.181:0/2361801571 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad4071c80 con 0x7f3ad4072760 2023-10-16T10:03:53.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.008+0000 7f3ad8f30700 1 --2- 172.21.15.181:0/2361801571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad4072760 0x7f3ad406a8a0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40098f0 tx=0x7f3ac4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780e440780f1440f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3acbfff700 1 -- 172.21.15.181:0/2361801571 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4011830 con 0x7f3ad4072760 2023-10-16T10:03:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3acbfff700 1 -- 172.21.15.181:0/2361801571 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4011990 con 0x7f3ad4072760 2023-10-16T10:03:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3acbfff700 1 -- 172.21.15.181:0/2361801571 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4011d20 con 0x7f3ad4072760 2023-10-16T10:03:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 -- 172.21.15.181:0/2361801571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad4072760 msgr2=0x7f3ad406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 --2- 172.21.15.181:0/2361801571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad4072760 0x7f3ad406a8a0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40098f0 tx=0x7f3ac4009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 -- 172.21.15.181:0/2361801571 shutdown_connections 2023-10-16T10:03:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 --2- 172.21.15.181:0/2361801571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad406ade0 0x7f3ad41615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 --2- 172.21.15.181:0/2361801571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad4072760 0x7f3ad406a8a0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 -- 172.21.15.181:0/2361801571 >> 172.21.15.181:0/2361801571 conn(0x7f3ad40cd010 msgr2=0x7f3ad40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 -- 172.21.15.181:0/2361801571 shutdown_connections 2023-10-16T10:03:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.009+0000 7f3adb995700 1 -- 172.21.15.181:0/2361801571 wait complete. 2023-10-16T10:03:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.010+0000 7f3adb995700 1 Processor -- start 2023-10-16T10:03:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3adb995700 1 -- start start 2023-10-16T10:03:53.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3adb995700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad406ade0 0x7f3ad41f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3adb995700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad4072760 0x7f3ad41f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3adb995700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad415da70 0x7f3ad40ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3adb995700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad41f4440 con 0x7f3ad406ade0 2023-10-16T10:03:53.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3adb995700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40adac0 con 0x7f3ad415da70 2023-10-16T10:03:53.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3adb995700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40aca20 con 0x7f3ad4072760 2023-10-16T10:03:53.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3ad9731700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad406ade0 0x7f3ad41f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3ad9731700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad406ade0 0x7f3ad41f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48894/0 (socket says 172.21.15.181:48894) 2023-10-16T10:03:53.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3ad9731700 1 -- 172.21.15.181:0/3708604174 learned_addr learned my addr 172.21.15.181:0/3708604174 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:53.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.011+0000 7f3ad9f32700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad415da70 0x7f3ad40ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad8f30700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad4072760 0x7f3ad41f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad9731700 1 -- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad4072760 msgr2=0x7f3ad41f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad9731700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad4072760 0x7f3ad41f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad9731700 1 -- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad415da70 msgr2=0x7f3ad40ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad9731700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad415da70 0x7f3ad40ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad9731700 1 -- 172.21.15.181:0/3708604174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4009550 con 0x7f3ad406ade0 2023-10-16T10:03:53.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad9f32700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad415da70 0x7f3ad40ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:53.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ad9731700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad406ade0 0x7f3ad41f3f00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c450 tx=0x7f3ad000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ac9ffb700 1 -- 172.21.15.181:0/3708604174 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00052d0 con 0x7f3ad406ade0 2023-10-16T10:03:53.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ac9ffb700 1 -- 172.21.15.181:0/3708604174 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0005430 con 0x7f3ad406ade0 2023-10-16T10:03:53.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad40acd00 con 0x7f3ad406ade0 2023-10-16T10:03:53.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3ac9ffb700 1 -- 172.21.15.181:0/3708604174 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00038c0 con 0x7f3ad406ade0 2023-10-16T10:03:53.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.012+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad40ad2c0 con 0x7f3ad406ade0 2023-10-16T10:03:53.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.013+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad40b1ee0 con 0x7f3ad406ade0 2023-10-16T10:03:53.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.013+0000 7f3ac9ffb700 1 -- 172.21.15.181:0/3708604174 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 85) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3ad0003a20 con 0x7f3ad406ade0 2023-10-16T10:03:53.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.014+0000 7f3ac9ffb700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f3ab806af90 0x7f3ab806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.014+0000 7f3ac9ffb700 1 -- 172.21.15.181:0/3708604174 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f3ad0088ab0 con 0x7f3ad406ade0 2023-10-16T10:03:53.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.014+0000 7f3ad8f30700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f3ab806af90 0x7f3ab806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.014+0000 7f3ad8f30700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f3ab806af90 0x7f3ab806d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009c40 tx=0x7f3ac4008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.018+0000 7f3ac9ffb700 1 -- 172.21.15.181:0/3708604174 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3ad0053a60 con 0x7f3ad406ade0 2023-10-16T10:03:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.213+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f3ad40b1c00 con 0x7f3ad406ade0 2023-10-16T10:03:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.219+0000 7f3ac9ffb700 1 -- 172.21.15.181:0/3708604174 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v50) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3ad0057080 con 0x7f3ad406ade0 2023-10-16T10:03:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f3ab806af90 msgr2=0x7f3ab806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f3ab806af90 0x7f3ab806d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009c40 tx=0x7f3ac4008030 comp rx=0 tx=0).stop 2023-10-16T10:03:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad406ade0 msgr2=0x7f3ad41f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad406ade0 0x7f3ad41f3f00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c450 tx=0x7f3ad000c810 comp rx=0 tx=0).stop 2023-10-16T10:03:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 shutdown_connections 2023-10-16T10:03:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f3ab806af90 0x7f3ab806d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ad415da70 0x7f3ad40ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ad406ade0 0x7f3ad41f3f00 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 --2- 172.21.15.181:0/3708604174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ad4072760 0x7f3ad41f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 >> 172.21.15.181:0/3708604174 conn(0x7f3ad40cd010 msgr2=0x7f3ad40d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:53.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 shutdown_connections 2023-10-16T10:03:53.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.221+0000 7f3adb995700 1 -- 172.21.15.181:0/3708604174 wait complete. 2023-10-16T10:03:53.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T10:03:53.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.356+0000 7f44aaa62700 1 Processor -- start 2023-10-16T10:03:53.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.356+0000 7f44aaa62700 1 -- start start 2023-10-16T10:03:53.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44aaa62700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44a40ce2d0 0x7f44a415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44aaa62700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44aaa62700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a415ef80 con 0x7f44a40d01d0 2023-10-16T10:03:53.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44aaa62700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a40d4290 con 0x7f44a40ce2d0 2023-10-16T10:03:53.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44aaa62700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40ce750 con 0x7f44a40d1500 2023-10-16T10:03:53.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44a40ce2d0 0x7f44a415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44a40ce2d0 0x7f44a415ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48900/0 (socket says 172.21.15.181:48900) 2023-10-16T10:03:53.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a37fe700 1 -- 172.21.15.181:0/2510492970 learned_addr learned my addr 172.21.15.181:0/2510492970 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:53.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a8fff700 1 --2- 172.21.15.181:0/2510492970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a37fe700 1 -- 172.21.15.181:0/2510492970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 msgr2=0x7f44a40d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:53.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a37fe700 1 --2- 172.21.15.181:0/2510492970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a37fe700 1 -- 172.21.15.181:0/2510492970 >> v1:172.21.15.190:6789/0 conn(0x7f44a40d01d0 legacy=0x7f44a40d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:53.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.357+0000 7f44a37fe700 1 -- 172.21.15.181:0/2510492970 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40cf580 con 0x7f44a40ce2d0 2023-10-16T10:03:53.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44a37fe700 1 --2- 172.21.15.181:0/2510492970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40ce2d0 0x7f44a415ea40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4498009920 tx=0x7f4498009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917d464a2930902b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44a2ffd700 1 -- 172.21.15.181:0/2510492970 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498004230 con 0x7f44a40ce2d0 2023-10-16T10:03:53.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44a2ffd700 1 -- 172.21.15.181:0/2510492970 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4498004390 con 0x7f44a40ce2d0 2023-10-16T10:03:53.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44a2ffd700 1 -- 172.21.15.181:0/2510492970 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498009e70 con 0x7f44a40ce2d0 2023-10-16T10:03:53.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 -- 172.21.15.181:0/2510492970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40ce2d0 msgr2=0x7f44a415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 --2- 172.21.15.181:0/2510492970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40ce2d0 0x7f44a415ea40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4498009920 tx=0x7f4498009c30 comp rx=0 tx=0).stop 2023-10-16T10:03:53.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 -- 172.21.15.181:0/2510492970 shutdown_connections 2023-10-16T10:03:53.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 --2- 172.21.15.181:0/2510492970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 --2- 172.21.15.181:0/2510492970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40ce2d0 0x7f44a415ea40 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 -- 172.21.15.181:0/2510492970 >> 172.21.15.181:0/2510492970 conn(0x7f44a409cb60 msgr2=0x7f44a40d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:53.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 -- 172.21.15.181:0/2510492970 shutdown_connections 2023-10-16T10:03:53.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.358+0000 7f44aaa62700 1 -- 172.21.15.181:0/2510492970 wait complete. 2023-10-16T10:03:53.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.360+0000 7f44aaa62700 1 Processor -- start 2023-10-16T10:03:53.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.360+0000 7f44aaa62700 1 -- start start 2023-10-16T10:03:53.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.360+0000 7f44aaa62700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44a40ce2d0 0x7f44a4151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44aaa62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40d01d0 0x7f44a40b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44aaa62700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44aaa62700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41523a0 con 0x7f44a40d01d0 2023-10-16T10:03:53.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44aaa62700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40b0f80 con 0x7f44a40ce2d0 2023-10-16T10:03:53.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44aaa62700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40b1260 con 0x7f44a40d1500 2023-10-16T10:03:53.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40d01d0 0x7f44a40b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40d01d0 0x7f44a40b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48904/0 (socket says 172.21.15.181:48904) 2023-10-16T10:03:53.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 -- 172.21.15.181:0/4196194317 learned_addr learned my addr 172.21.15.181:0/4196194317 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:53.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a8fff700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a3fff700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44a40ce2d0 0x7f44a4151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 -- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 msgr2=0x7f44a40ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 -- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44a40ce2d0 msgr2=0x7f44a4151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44a40ce2d0 0x7f44a4151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 -- 172.21.15.181:0/4196194317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4498009550 con 0x7f44a40d01d0 2023-10-16T10:03:53.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a37fe700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40d01d0 0x7f44a40b06f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f44a408cf30 tx=0x7f4498004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44980036a0 con 0x7f44a40d01d0 2023-10-16T10:03:53.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4498003800 con 0x7f44a40d01d0 2023-10-16T10:03:53.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a40b1540 con 0x7f44a40d01d0 2023-10-16T10:03:53.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498003b70 con 0x7f44a40d01d0 2023-10-16T10:03:53.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.361+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a40b1b00 con 0x7f44a40d01d0 2023-10-16T10:03:53.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.362+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a404ebb0 con 0x7f44a40d01d0 2023-10-16T10:03:53.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.363+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 85) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4498003cd0 con 0x7f44a40d01d0 2023-10-16T10:03:53.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.363+0000 7f44a0ff9700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f449006af90 0x7f449006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.363+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f449808af40 con 0x7f44a40d01d0 2023-10-16T10:03:53.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.363+0000 7f44a3fff700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f449006af90 0x7f449006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.363+0000 7f44a3fff700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f449006af90 0x7f449006d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4494004f30 tx=0x7f4494009040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.367+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f44980bdfe0 con 0x7f44a40d01d0 2023-10-16T10:03:53.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.526+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f44980562c0 con 0x7f44a40d01d0 2023-10-16T10:03:53.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.563+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f44a415ef80 con 0x7f44a40d01d0 2023-10-16T10:03:53.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.567+0000 7f44a0ff9700 1 -- 172.21.15.181:0/4196194317 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f44a415ef80 con 0x7f44a40d01d0 2023-10-16T10:03:53.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f449006af90 msgr2=0x7f449006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f449006af90 0x7f449006d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4494004f30 tx=0x7f4494009040 comp rx=0 tx=0).stop 2023-10-16T10:03:53.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40d01d0 msgr2=0x7f44a40b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40d01d0 0x7f44a40b06f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f44a408cf30 tx=0x7f4498004750 comp rx=0 tx=0).stop 2023-10-16T10:03:53.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 shutdown_connections 2023-10-16T10:03:53.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f449006af90 0x7f449006d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44a40d1500 0x7f44a40ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44a40ce2d0 0x7f44a4151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 --2- 172.21.15.181:0/4196194317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44a40d01d0 0x7f44a40b06f0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 >> 172.21.15.181:0/4196194317 conn(0x7f44a409cb60 msgr2=0x7f44a415f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 shutdown_connections 2023-10-16T10:03:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.569+0000 7f44aaa62700 1 -- 172.21.15.181:0/4196194317 wait complete. 2023-10-16T10:03:53.583 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-16T10:03:53.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.703+0000 7efe250e7700 1 Processor -- start 2023-10-16T10:03:53.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.703+0000 7efe250e7700 1 -- start start 2023-10-16T10:03:53.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe250e7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe2015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe250e7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efe2006cdf0 0x7efe200cb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe250e7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe200cabd0 con 0x7efe200cdfb0 2023-10-16T10:03:53.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe250e7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe200ca5c0 con 0x7efe2006cdf0 2023-10-16T10:03:53.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe250e7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe20066740 con 0x7efe200cc0c0 2023-10-16T10:03:53.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe2015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efe2006cdf0 0x7efe200cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe2015dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:45940/0 (socket says 172.21.15.181:45940) 2023-10-16T10:03:53.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 -- 172.21.15.181:0/3310821684 learned_addr learned my addr 172.21.15.181:0/3310821684 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:53.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1ed9d700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7efe200cdfb0 0x7efe2015d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:43932/0 (socket says 172.21.15.181:43932) 2023-10-16T10:03:53.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 -- 172.21.15.181:0/3310821684 >> v2:172.21.15.190:3300/0 conn(0x7efe2006cdf0 msgr2=0x7efe200cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:53.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 --2- 172.21.15.181:0/3310821684 >> v2:172.21.15.190:3300/0 conn(0x7efe2006cdf0 0x7efe200cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 -- 172.21.15.181:0/3310821684 >> v1:172.21.15.190:6789/0 conn(0x7efe200cdfb0 legacy=0x7efe2015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 -- 172.21.15.181:0/3310821684 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20066a20 con 0x7efe200cc0c0 2023-10-16T10:03:53.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.704+0000 7efe1e59c700 1 --2- 172.21.15.181:0/3310821684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe2015dfc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efe1400bb90 tx=0x7efe1400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc2111794da0021 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe1dd9b700 1 -- 172.21.15.181:0/3310821684 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1400dd30 con 0x7efe200cc0c0 2023-10-16T10:03:53.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe1dd9b700 1 -- 172.21.15.181:0/3310821684 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1400de90 con 0x7efe200cc0c0 2023-10-16T10:03:53.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe1dd9b700 1 -- 172.21.15.181:0/3310821684 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14010690 con 0x7efe200cc0c0 2023-10-16T10:03:53.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 -- 172.21.15.181:0/3310821684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 msgr2=0x7efe2015dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 --2- 172.21.15.181:0/3310821684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe2015dfc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efe1400bb90 tx=0x7efe1400bea0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 -- 172.21.15.181:0/3310821684 shutdown_connections 2023-10-16T10:03:53.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 --2- 172.21.15.181:0/3310821684 >> v2:172.21.15.190:3300/0 conn(0x7efe2006cdf0 0x7efe200cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 --2- 172.21.15.181:0/3310821684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe2015dfc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 -- 172.21.15.181:0/3310821684 >> 172.21.15.181:0/3310821684 conn(0x7efe200b1780 msgr2=0x7efe200b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 -- 172.21.15.181:0/3310821684 shutdown_connections 2023-10-16T10:03:53.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.705+0000 7efe250e7700 1 -- 172.21.15.181:0/3310821684 wait complete. 2023-10-16T10:03:53.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 Processor -- start 2023-10-16T10:03:53.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 -- start start 2023-10-16T10:03:53.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe2006cdf0 0x7efe200b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe200b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe200cdfb0 0x7efe200b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200a08b0 con 0x7efe200cdfb0 2023-10-16T10:03:53.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2009f030 con 0x7efe2006cdf0 2023-10-16T10:03:53.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.707+0000 7efe250e7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2009f310 con 0x7efe200cc0c0 2023-10-16T10:03:53.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe200cdfb0 0x7efe200b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe200cdfb0 0x7efe200b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48922/0 (socket says 172.21.15.181:48922) 2023-10-16T10:03:53.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 -- 172.21.15.181:0/529786801 learned_addr learned my addr 172.21.15.181:0/529786801 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:53.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1e59c700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe200b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1ed9d700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe2006cdf0 0x7efe200b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 -- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 msgr2=0x7efe200b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe200b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 -- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe2006cdf0 msgr2=0x7efe200b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:53.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe2006cdf0 0x7efe200b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:53.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 -- 172.21.15.181:0/529786801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1400b7f0 con 0x7efe200cdfb0 2023-10-16T10:03:53.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1ed9d700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe2006cdf0 0x7efe200b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:53.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe1f59e700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe200cdfb0 0x7efe200b3030 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7efe1800efc0 tx=0x7efe1800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe077fe700 1 -- 172.21.15.181:0/529786801 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1800cb80 con 0x7efe200cdfb0 2023-10-16T10:03:53.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe077fe700 1 -- 172.21.15.181:0/529786801 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1800f040 con 0x7efe200cdfb0 2023-10-16T10:03:53.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe077fe700 1 -- 172.21.15.181:0/529786801 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18014480 con 0x7efe200cdfb0 2023-10-16T10:03:53.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2009f5f0 con 0x7efe200cdfb0 2023-10-16T10:03:53.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.708+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2009fbb0 con 0x7efe200cdfb0 2023-10-16T10:03:53.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.709+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe20067d20 con 0x7efe200cdfb0 2023-10-16T10:03:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.710+0000 7efe077fe700 1 -- 172.21.15.181:0/529786801 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efe180145e0 con 0x7efe200cdfb0 2023-10-16T10:03:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.710+0000 7efe077fe700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efe0c06b020 0x7efe0c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.710+0000 7efe077fe700 1 -- 172.21.15.181:0/529786801 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7efe1808a220 con 0x7efe200cdfb0 2023-10-16T10:03:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.710+0000 7efe1ed9d700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efe0c06b020 0x7efe0c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:53.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.711+0000 7efe1ed9d700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efe0c06b020 0x7efe0c06d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efe10000f60 tx=0x7efe10007dd0 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:53.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.715+0000 7efe077fe700 1 -- 172.21.15.181:0/529786801 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efe18054fa0 con 0x7efe200cdfb0 2023-10-16T10:03:53.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:53.990+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7efe20072510 con 0x7efe200cdfb0 2023-10-16T10:03:54.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.523+0000 7efe077fe700 1 -- 172.21.15.181:0/529786801 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v758) v1 ==== 160+0+0 (secure 0 0 0) 0x7efe180585c0 con 0x7efe200cdfb0 2023-10-16T10:03:54.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efe0c06b020 msgr2=0x7efe0c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efe0c06b020 0x7efe0c06d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efe10000f60 tx=0x7efe10007dd0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe200cdfb0 msgr2=0x7efe200b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe200cdfb0 0x7efe200b3030 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7efe1800efc0 tx=0x7efe1800c0c0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 shutdown_connections 2023-10-16T10:03:54.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efe0c06b020 0x7efe0c06d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe200cdfb0 0x7efe200b3030 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe2006cdf0 0x7efe200b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 --2- 172.21.15.181:0/529786801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe200cc0c0 0x7efe200b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 >> 172.21.15.181:0/529786801 conn(0x7efe200b1780 msgr2=0x7efe2006e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:54.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 shutdown_connections 2023-10-16T10:03:54.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.526+0000 7efe250e7700 1 -- 172.21.15.181:0/529786801 wait complete. 2023-10-16T10:03:54.543 INFO:tasks.cephfs_test_runner:test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2023-10-16T10:03:54.543 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-16T10:03:54.543 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T10:03:54.544 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:54.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.668+0000 7f43c7c30700 1 Processor -- start 2023-10-16T10:03:54.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.669+0000 7f43c7c30700 1 -- start start 2023-10-16T10:03:54.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.669+0000 7f43c7c30700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f43c00cbea0 0x7f43c00d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.669+0000 7f43c7c30700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c015fc90 con 0x7f43c015f1e0 2023-10-16T10:03:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.669+0000 7f43c7c30700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c015ff70 con 0x7f43c00ce760 2023-10-16T10:03:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.669+0000 7f43c7c30700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c0160250 con 0x7f43c00cbea0 2023-10-16T10:03:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c59cc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f43c015f1e0 0x7f43c00ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60004/0 (socket says 172.21.15.181:60004) 2023-10-16T10:03:54.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c59cc700 1 -- 172.21.15.181:0/1428178058 learned_addr learned my addr 172.21.15.181:0/1428178058 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c61cd700 1 --2- 172.21.15.181:0/1428178058 >> v2:172.21.15.190:3300/0 conn(0x7f43c00cbea0 0x7f43c00d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212381993 0 0) 0x7f43c015fc90 con 0x7f43c015f1e0 2023-10-16T10:03:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43ac0035b0 con 0x7f43c015f1e0 2023-10-16T10:03:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 947252004 0 0) 0x7f43ac0035b0 con 0x7f43c015f1e0 2023-10-16T10:03:54.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 >> v2:172.21.15.190:3300/0 conn(0x7f43c00cbea0 msgr2=0x7f43c00d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 --2- 172.21.15.181:0/1428178058 >> v2:172.21.15.190:3300/0 conn(0x7f43c00cbea0 0x7f43c00d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 >> v1:172.21.15.190:6789/0 conn(0x7f43c00ce760 legacy=0x7f43c00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0160530 con 0x7f43c015f1e0 2023-10-16T10:03:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f43b0001670 con 0x7f43c015f1e0 2023-10-16T10:03:54.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 >> v1:172.21.15.181:6789/0 conn(0x7f43c015f1e0 legacy=0x7f43c00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac003a00 0x7f43ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43ac004850 0x7f43ac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43ac005210 0x7f43ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0160250 con 0x7f43ac004850 2023-10-16T10:03:54.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac00c1b0 con 0x7f43ac003a00 2023-10-16T10:03:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac00c370 con 0x7f43ac005210 2023-10-16T10:03:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c51cb700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43ac004850 0x7f43ac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c59cc700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac003a00 0x7f43ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.670+0000 7f43c61cd700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43ac005210 0x7f43ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c51cb700 1 -- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43ac005210 msgr2=0x7f43ac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:54.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c51cb700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43ac005210 0x7f43ac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c51cb700 1 -- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac003a00 msgr2=0x7f43ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:54.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c51cb700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac003a00 0x7f43ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c51cb700 1 -- 172.21.15.181:0/1428178058 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0035b0 con 0x7f43ac004850 2023-10-16T10:03:54.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c61cd700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43ac005210 0x7f43ac00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:54.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c61cd700 1 -- 172.21.15.181:0/1428178058 reap_dead start 2023-10-16T10:03:54.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c51cb700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43ac004850 0x7f43ac004cd0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f43c00d2670 tx=0x7f43ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cd7e885528801a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:54.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0052d0 con 0x7f43ac004850 2023-10-16T10:03:54.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc005430 con 0x7f43ac004850 2023-10-16T10:03:54.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c49ca700 1 -- 172.21.15.181:0/1428178058 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0038c0 con 0x7f43ac004850 2023-10-16T10:03:54.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c7c30700 1 -- 172.21.15.181:0/1428178058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43ac004850 msgr2=0x7f43ac004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c7c30700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43ac004850 0x7f43ac004cd0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f43c00d2670 tx=0x7f43ac005880 comp rx=0 tx=0).stop 2023-10-16T10:03:54.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c7c30700 1 -- 172.21.15.181:0/1428178058 shutdown_connections 2023-10-16T10:03:54.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c7c30700 1 --2- 172.21.15.181:0/1428178058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43ac004850 0x7f43ac004cd0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.671+0000 7f43c7c30700 1 -- 172.21.15.181:0/1428178058 >> 172.21.15.181:0/1428178058 conn(0x7f43c00665a0 msgr2=0x7f43c0066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.672+0000 7f43c7c30700 1 -- 172.21.15.181:0/1428178058 shutdown_connections 2023-10-16T10:03:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.672+0000 7f43c7c30700 1 -- 172.21.15.181:0/1428178058 wait complete. 2023-10-16T10:03:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 Processor -- start 2023-10-16T10:03:54.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 -- start start 2023-10-16T10:03:54.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac004850 0x7f43c00ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43c0157170 0x7f43c00cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c01f4ab0 0x7f43c01f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01564b0 con 0x7f43c01f4ab0 2023-10-16T10:03:54.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01f5470 con 0x7f43ac004850 2023-10-16T10:03:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.673+0000 7f43c7c30700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0156ce0 con 0x7f43c0157170 2023-10-16T10:03:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c01f4ab0 0x7f43c01f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c01f4ab0 0x7f43c01f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48942/0 (socket says 172.21.15.181:48942) 2023-10-16T10:03:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 -- 172.21.15.181:0/3960169583 learned_addr learned my addr 172.21.15.181:0/3960169583 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c59cc700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac004850 0x7f43c00ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c51cb700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43c0157170 0x7f43c00cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 -- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43c0157170 msgr2=0x7f43c00cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43c0157170 0x7f43c00cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 -- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac004850 msgr2=0x7f43c00ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac004850 0x7f43c00ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 -- 172.21.15.181:0/3960169583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c00cb550 con 0x7f43c01f4ab0 2023-10-16T10:03:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c61cd700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c01f4ab0 0x7f43c01f4f30 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f43c00cdea0 tx=0x7f43b80104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43b67fc700 1 -- 172.21.15.181:0/3960169583 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b801b070 con 0x7f43c01f4ab0 2023-10-16T10:03:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43b67fc700 1 -- 172.21.15.181:0/3960169583 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b800f040 con 0x7f43c01f4ab0 2023-10-16T10:03:54.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c00b1360 con 0x7f43c01f4ab0 2023-10-16T10:03:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43b67fc700 1 -- 172.21.15.181:0/3960169583 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b800e050 con 0x7f43c01f4ab0 2023-10-16T10:03:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.674+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c0156970 con 0x7f43c01f4ab0 2023-10-16T10:03:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.675+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c0156b00 con 0x7f43c01f4ab0 2023-10-16T10:03:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.675+0000 7f43b67fc700 1 -- 172.21.15.181:0/3960169583 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f43b8015440 con 0x7f43c01f4ab0 2023-10-16T10:03:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.676+0000 7f43b67fc700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f43a406b020 0x7f43a406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.676+0000 7f43b67fc700 1 -- 172.21.15.181:0/3960169583 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f43b80899d0 con 0x7f43c01f4ab0 2023-10-16T10:03:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.676+0000 7f43c59cc700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f43a406b020 0x7f43a406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.676+0000 7f43c59cc700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f43a406b020 0x7f43a406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43b0001c90 tx=0x7f43b0001b20 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.680+0000 7f43b67fc700 1 -- 172.21.15.181:0/3960169583 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f43b80546c0 con 0x7f43c01f4ab0 2023-10-16T10:03:54.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.874+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c00cf3f0 con 0x7f43c01f4ab0 2023-10-16T10:03:54.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.875+0000 7f43b67fc700 1 -- 172.21.15.181:0/3960169583 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+14804 (secure 0 0 0) 0x7f43b8057ce0 con 0x7f43c01f4ab0 2023-10-16T10:03:54.877 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:54.877 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":137,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:52.511519+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/1716409851":"2023-10-17T10:03:49.477261+0000","172.21.15.181:6833/3791133934":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/2235318164":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/1126035840":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/2621214072":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/2702413404":"2023-10-17T10:03:48.495129+0000","172.21.15.190:6833/1716409851":"2023-10-17T10:03:49.477261+0000","172.21.15.181:0/2541635143":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/662310189":"2023-10-17T10:03:48.495129+0000","172.21.15.181:6832/3791133934":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/240608457":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1855739676":"2023-10-17T10:03:48.495129+0000","172.21.15.181:0/3359421143":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6833/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:6832/3280351398":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/2638619292":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1179812996":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1546779767":"2023-10-17T10:03:35.802223+0000","172.21.15.181:0/1171463515":"2023-10-17T10:03:35.802223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:54.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f43a406b020 msgr2=0x7f43a406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f43a406b020 0x7f43a406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43b0001c90 tx=0x7f43b0001b20 comp rx=0 tx=0).stop 2023-10-16T10:03:54.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c01f4ab0 msgr2=0x7f43c01f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c01f4ab0 0x7f43c01f4f30 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f43c00cdea0 tx=0x7f43b80104a0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 shutdown_connections 2023-10-16T10:03:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f43a406b020 0x7f43a406d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c01f4ab0 0x7f43c01f4f30 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43ac004850 0x7f43c00ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 --2- 172.21.15.181:0/3960169583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43c0157170 0x7f43c00cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 >> 172.21.15.181:0/3960169583 conn(0x7f43c00665a0 msgr2=0x7f43c015d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 shutdown_connections 2023-10-16T10:03:54.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:54.877+0000 7f43c7c30700 1 -- 172.21.15.181:0/3960169583 wait complete. 2023-10-16T10:03:54.891 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:54.892 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:55.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.013+0000 7f9c02166700 1 Processor -- start 2023-10-16T10:03:55.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.014+0000 7f9c02166700 1 -- start start 2023-10-16T10:03:55.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.014+0000 7f9c02166700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bfc159e10 0x7f9bfc0ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.014+0000 7f9c02166700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9bfc15a1f0 0x7f9bfc15db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.014+0000 7f9c02166700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc06e4b0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.014+0000 7f9c02166700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc06ed50 con 0x7f9bfc159e10 2023-10-16T10:03:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.014+0000 7f9c02166700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc0ceea0 con 0x7f9bfc15a1f0 2023-10-16T10:03:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bfc159e10 0x7f9bfc0ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bfc159e10 0x7f9bfc0ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48946/0 (socket says 172.21.15.181:48946) 2023-10-16T10:03:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/2457068163 learned_addr learned my addr 172.21.15.181:0/2457068163 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfbfff700 1 --2- 172.21.15.181:0/2457068163 >> v2:172.21.15.190:3300/0 conn(0x7f9bfc15a1f0 0x7f9bfc15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/2457068163 >> v2:172.21.15.190:3300/0 conn(0x7f9bfc15a1f0 msgr2=0x7f9bfc15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 --2- 172.21.15.181:0/2457068163 >> v2:172.21.15.190:3300/0 conn(0x7f9bfc15a1f0 0x7f9bfc15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/2457068163 >> v1:172.21.15.190:6789/0 conn(0x7f9bfc06bef0 legacy=0x7f9bfc0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:55.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/2457068163 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc06e700 con 0x7f9bfc159e10 2023-10-16T10:03:55.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfbfff700 1 --2- 172.21.15.181:0/2457068163 >> v2:172.21.15.190:3300/0 conn(0x7f9bfc15a1f0 0x7f9bfc15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:55.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfb7fe700 1 --2- 172.21.15.181:0/2457068163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc159e10 0x7f9bfc0ce5a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc072c10 tx=0x7f9bf000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a1504800dba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:55.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfa7fc700 1 -- 172.21.15.181:0/2457068163 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000dcf0 con 0x7f9bfc159e10 2023-10-16T10:03:55.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfa7fc700 1 -- 172.21.15.181:0/2457068163 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf000de50 con 0x7f9bfc159e10 2023-10-16T10:03:55.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.015+0000 7f9bfa7fc700 1 -- 172.21.15.181:0/2457068163 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0004250 con 0x7f9bfc159e10 2023-10-16T10:03:55.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 -- 172.21.15.181:0/2457068163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc159e10 msgr2=0x7f9bfc0ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 --2- 172.21.15.181:0/2457068163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc159e10 0x7f9bfc0ce5a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc072c10 tx=0x7f9bf000be70 comp rx=0 tx=0).stop 2023-10-16T10:03:55.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 -- 172.21.15.181:0/2457068163 shutdown_connections 2023-10-16T10:03:55.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 --2- 172.21.15.181:0/2457068163 >> v2:172.21.15.190:3300/0 conn(0x7f9bfc15a1f0 0x7f9bfc15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 --2- 172.21.15.181:0/2457068163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc159e10 0x7f9bfc0ce5a0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 -- 172.21.15.181:0/2457068163 >> 172.21.15.181:0/2457068163 conn(0x7f9bfc0cc730 msgr2=0x7f9bfc0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:55.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 -- 172.21.15.181:0/2457068163 shutdown_connections 2023-10-16T10:03:55.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.016+0000 7f9c02166700 1 -- 172.21.15.181:0/2457068163 wait complete. 2023-10-16T10:03:55.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.017+0000 7f9c02166700 1 Processor -- start 2023-10-16T10:03:55.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.017+0000 7f9c02166700 1 -- start start 2023-10-16T10:03:55.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9c02166700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc06bef0 0x7f9bfc158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9c02166700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bfc159e10 0x7f9bfc0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9c02166700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bfc15a1f0 0x7f9bfc0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9c02166700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc159460 con 0x7f9bfc06bef0 2023-10-16T10:03:55.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9c02166700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0c6fd0 con 0x7f9bfc159e10 2023-10-16T10:03:55.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9c02166700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc14b510 con 0x7f9bfc15a1f0 2023-10-16T10:03:55.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc06bef0 0x7f9bfc158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc06bef0 0x7f9bfc158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48950/0 (socket says 172.21.15.181:48950) 2023-10-16T10:03:55.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/76231801 learned_addr learned my addr 172.21.15.181:0/76231801 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:55.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfaffd700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bfc159e10 0x7f9bfc0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfbfff700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bfc15a1f0 0x7f9bfc0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bfc15a1f0 msgr2=0x7f9bfc0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bfc15a1f0 0x7f9bfc0c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bfc159e10 msgr2=0x7f9bfc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bfc159e10 0x7f9bfc0cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 -- 172.21.15.181:0/76231801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf000b7f0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9bfb7fe700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc06bef0 0x7f9bfc158df0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0000c00 tx=0x7f9bf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:55.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9be3fff700 1 -- 172.21.15.181:0/76231801 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0004230 con 0x7f9bfc06bef0 2023-10-16T10:03:55.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.018+0000 7f9be3fff700 1 -- 172.21.15.181:0/76231801 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0010470 con 0x7f9bfc06bef0 2023-10-16T10:03:55.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.019+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc14b7f0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.019+0000 7f9be3fff700 1 -- 172.21.15.181:0/76231801 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00107e0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.019+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc14bdb0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.019+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc159630 con 0x7f9bfc06bef0 2023-10-16T10:03:55.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.020+0000 7f9be3fff700 1 -- 172.21.15.181:0/76231801 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9bf00193f0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.020+0000 7f9be3fff700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f9be806b020 0x7f9be806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.020+0000 7f9be3fff700 1 -- 172.21.15.181:0/76231801 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f9bf008b770 con 0x7f9bfc06bef0 2023-10-16T10:03:55.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.020+0000 7f9bfaffd700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f9be806b020 0x7f9be806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.021+0000 7f9bfaffd700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f9be806b020 0x7f9be806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc1f6990 tx=0x7f9bec007ec0 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:55.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.025+0000 7f9be3fff700 1 -- 172.21.15.181:0/76231801 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9bf00564f0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.215+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bfc0ceab0 con 0x7f9bfc06bef0 2023-10-16T10:03:55.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.215+0000 7f9be3fff700 1 -- 172.21.15.181:0/76231801 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bf0059b10 con 0x7f9bfc06bef0 2023-10-16T10:03:55.217 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:55.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f9be806b020 msgr2=0x7f9be806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f9be806b020 0x7f9be806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc1f6990 tx=0x7f9bec007ec0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc06bef0 msgr2=0x7f9bfc158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc06bef0 0x7f9bfc158df0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0000c00 tx=0x7f9bf0004750 comp rx=0 tx=0).stop 2023-10-16T10:03:55.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 shutdown_connections 2023-10-16T10:03:55.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f9be806b020 0x7f9be806d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bfc15a1f0 0x7f9bfc0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bfc06bef0 0x7f9bfc158df0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 --2- 172.21.15.181:0/76231801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bfc159e10 0x7f9bfc0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 >> 172.21.15.181:0/76231801 conn(0x7f9bfc0cc730 msgr2=0x7f9bfc15a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:55.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 shutdown_connections 2023-10-16T10:03:55.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.217+0000 7f9c02166700 1 -- 172.21.15.181:0/76231801 wait complete. 2023-10-16T10:03:55.231 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-16T10:03:55.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.350+0000 7f5c45914700 1 Processor -- start 2023-10-16T10:03:55.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.350+0000 7f5c45914700 1 -- start start 2023-10-16T10:03:55.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c45914700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c40159730 0x7f5c4015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c45914700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c400cc400 0x7f5c400cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c45914700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c400d0310 con 0x7f5c400cd3f0 2023-10-16T10:03:55.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c45914700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400cd880 con 0x7f5c400cc400 2023-10-16T10:03:55.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c45914700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400cee70 con 0x7f5c40159730 2023-10-16T10:03:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c3e7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c400cc400 0x7f5c400cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c3e7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c400cc400 0x7f5c400cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48962/0 (socket says 172.21.15.181:48962) 2023-10-16T10:03:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c3e7fc700 1 -- 172.21.15.181:0/2054285416 learned_addr learned my addr 172.21.15.181:0/2054285416 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.351+0000 7f5c3effd700 1 --2- 172.21.15.181:0/2054285416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c40159730 0x7f5c4015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3e7fc700 1 -- 172.21.15.181:0/2054285416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c40159730 msgr2=0x7f5c4015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3e7fc700 1 --2- 172.21.15.181:0/2054285416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c40159730 0x7f5c4015cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3e7fc700 1 -- 172.21.15.181:0/2054285416 >> v1:172.21.15.190:6789/0 conn(0x7f5c400cd3f0 legacy=0x7f5c4006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3e7fc700 1 -- 172.21.15.181:0/2054285416 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c40159b10 con 0x7f5c400cc400 2023-10-16T10:03:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3e7fc700 1 --2- 172.21.15.181:0/2054285416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c400cceb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5c300098f0 tx=0x7f5c30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7151e021a7e92c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3dffb700 1 -- 172.21.15.181:0/2054285416 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30004230 con 0x7f5c400cc400 2023-10-16T10:03:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3dffb700 1 -- 172.21.15.181:0/2054285416 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c30004390 con 0x7f5c400cc400 2023-10-16T10:03:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c3dffb700 1 -- 172.21.15.181:0/2054285416 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300036a0 con 0x7f5c400cc400 2023-10-16T10:03:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c45914700 1 -- 172.21.15.181:0/2054285416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 msgr2=0x7f5c400cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.352+0000 7f5c45914700 1 --2- 172.21.15.181:0/2054285416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c400cceb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5c300098f0 tx=0x7f5c30009c00 comp rx=0 tx=0).stop 2023-10-16T10:03:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.353+0000 7f5c45914700 1 -- 172.21.15.181:0/2054285416 shutdown_connections 2023-10-16T10:03:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.353+0000 7f5c45914700 1 --2- 172.21.15.181:0/2054285416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c40159730 0x7f5c4015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.353+0000 7f5c45914700 1 --2- 172.21.15.181:0/2054285416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c400cceb0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.353+0000 7f5c45914700 1 -- 172.21.15.181:0/2054285416 >> 172.21.15.181:0/2054285416 conn(0x7f5c400d04a0 msgr2=0x7f5c400d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.353+0000 7f5c45914700 1 -- 172.21.15.181:0/2054285416 shutdown_connections 2023-10-16T10:03:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.353+0000 7f5c45914700 1 -- 172.21.15.181:0/2054285416 wait complete. 2023-10-16T10:03:55.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.354+0000 7f5c45914700 1 Processor -- start 2023-10-16T10:03:55.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.354+0000 7f5c45914700 1 -- start start 2023-10-16T10:03:55.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c45914700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c401f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c45914700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c400cd3f0 0x7f5c40154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c45914700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40159730 0x7f5c400ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c45914700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401f2150 con 0x7f5c400cc400 2023-10-16T10:03:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c45914700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40154fc0 con 0x7f5c40159730 2023-10-16T10:03:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c45914700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400c4200 con 0x7f5c400cd3f0 2023-10-16T10:03:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c3effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c401f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c3effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c401f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48964/0 (socket says 172.21.15.181:48964) 2023-10-16T10:03:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.355+0000 7f5c3effd700 1 -- 172.21.15.181:0/2369975708 learned_addr learned my addr 172.21.15.181:0/2369975708 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3e7fc700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c400cd3f0 0x7f5c40154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3effd700 1 -- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c400cd3f0 msgr2=0x7f5c40154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3f7fe700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40159730 0x7f5c400ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3effd700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c400cd3f0 0x7f5c40154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3effd700 1 -- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40159730 msgr2=0x7f5c400ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3effd700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40159730 0x7f5c400ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3effd700 1 -- 172.21.15.181:0/2369975708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c30009550 con 0x7f5c400cc400 2023-10-16T10:03:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c3effd700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c401f11b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400d7d0 tx=0x7f5c3400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c27fff700 1 -- 172.21.15.181:0/2369975708 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340125f0 con 0x7f5c400cc400 2023-10-16T10:03:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c27fff700 1 -- 172.21.15.181:0/2369975708 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34012750 con 0x7f5c400cc400 2023-10-16T10:03:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c400ca8f0 con 0x7f5c400cc400 2023-10-16T10:03:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c27fff700 1 -- 172.21.15.181:0/2369975708 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34012b00 con 0x7f5c400cc400 2023-10-16T10:03:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.356+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c401f1b90 con 0x7f5c400cc400 2023-10-16T10:03:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.357+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c400c44e0 con 0x7f5c400cc400 2023-10-16T10:03:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.357+0000 7f5c27fff700 1 -- 172.21.15.181:0/2369975708 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c34012c60 con 0x7f5c400cc400 2023-10-16T10:03:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.358+0000 7f5c27fff700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f5c2c06b020 0x7f5c2c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.358+0000 7f5c27fff700 1 -- 172.21.15.181:0/2369975708 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f5c340899d0 con 0x7f5c400cc400 2023-10-16T10:03:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.358+0000 7f5c3e7fc700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f5c2c06b020 0x7f5c2c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.358+0000 7f5c3e7fc700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f5c2c06b020 0x7f5c2c06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c4009e460 tx=0x7f5c30008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.362+0000 7f5c27fff700 1 -- 172.21.15.181:0/2369975708 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5c340547d0 con 0x7f5c400cc400 2023-10-16T10:03:55.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:55.632+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f5c400c3fd0 con 0x7f5c400cc400 2023-10-16T10:03:56.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.529+0000 7f5c27fff700 1 -- 172.21.15.181:0/2369975708 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v760) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5c400c3fd0 con 0x7f5c400cc400 2023-10-16T10:03:56.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.532+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f5c2c06b020 msgr2=0x7f5c2c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.532+0000 7f5c45914700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f5c2c06b020 0x7f5c2c06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c4009e460 tx=0x7f5c30008030 comp rx=0 tx=0).stop 2023-10-16T10:03:56.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.532+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 msgr2=0x7f5c401f11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.532+0000 7f5c45914700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c401f11b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400d7d0 tx=0x7f5c3400db90 comp rx=0 tx=0).stop 2023-10-16T10:03:56.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 shutdown_connections 2023-10-16T10:03:56.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f5c2c06b020 0x7f5c2c06d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c40159730 0x7f5c400ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c400cc400 0x7f5c401f11b0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 --2- 172.21.15.181:0/2369975708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c400cd3f0 0x7f5c40154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 >> 172.21.15.181:0/2369975708 conn(0x7f5c400d04a0 msgr2=0x7f5c401589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:56.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 shutdown_connections 2023-10-16T10:03:56.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.533+0000 7f5c45914700 1 -- 172.21.15.181:0/2369975708 wait complete. 2023-10-16T10:03:56.550 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T10:03:56.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.678+0000 7fd6f71ac700 1 Processor -- start 2023-10-16T10:03:56.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.678+0000 7fd6f71ac700 1 -- start start 2023-10-16T10:03:56.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f71ac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f00b2010 0x7fd6f00cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:56.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f71ac700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6f0071be0 0x7fd6f00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:56.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f71ac700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00ce1f0 con 0x7fd6f006f3b0 2023-10-16T10:03:56.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f71ac700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f00b2930 con 0x7fd6f0071be0 2023-10-16T10:03:56.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f71ac700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00ccda0 con 0x7fd6f00b2010 2023-10-16T10:03:56.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6effff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6f0071be0 0x7fd6f00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:56.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f4f48700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f00b2010 0x7fd6f00cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:56.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f5749700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd6f006f3b0 0x7fd6f006f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:43994/0 (socket says 172.21.15.181:43994) 2023-10-16T10:03:56.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f5749700 1 -- 172.21.15.181:0/663228880 learned_addr learned my addr 172.21.15.181:0/663228880 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:56.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.679+0000 7fd6f4f48700 1 --2- 172.21.15.181:0/663228880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f00b2010 0x7fd6f00cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:45998/0 (socket says 172.21.15.181:45998) 2023-10-16T10:03:56.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6f4f48700 1 -- 172.21.15.181:0/663228880 >> v2:172.21.15.190:3300/0 conn(0x7fd6f0071be0 msgr2=0x7fd6f00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:56.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6f4f48700 1 --2- 172.21.15.181:0/663228880 >> v2:172.21.15.190:3300/0 conn(0x7fd6f0071be0 0x7fd6f00b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6f4f48700 1 -- 172.21.15.181:0/663228880 >> v1:172.21.15.190:6789/0 conn(0x7fd6f006f3b0 legacy=0x7fd6f006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6f4f48700 1 -- 172.21.15.181:0/663228880 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00727e0 con 0x7fd6f00b2010 2023-10-16T10:03:56.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6effff700 1 --2- 172.21.15.181:0/663228880 >> v2:172.21.15.190:3300/0 conn(0x7fd6f0071be0 0x7fd6f00b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:56.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6f4f48700 1 --2- 172.21.15.181:0/663228880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f00b2010 0x7fd6f00cc6e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009840 tx=0x7fd6e4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22ead54c432c9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:56.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6ef7fe700 1 -- 172.21.15.181:0/663228880 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4011830 con 0x7fd6f00b2010 2023-10-16T10:03:56.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6ef7fe700 1 -- 172.21.15.181:0/663228880 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4011990 con 0x7fd6f00b2010 2023-10-16T10:03:56.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6ef7fe700 1 -- 172.21.15.181:0/663228880 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4011d20 con 0x7fd6f00b2010 2023-10-16T10:03:56.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6f71ac700 1 -- 172.21.15.181:0/663228880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f00b2010 msgr2=0x7fd6f00cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.680+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/663228880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f00b2010 0x7fd6f00cc6e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009840 tx=0x7fd6e4009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:56.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.681+0000 7fd6f71ac700 1 -- 172.21.15.181:0/663228880 shutdown_connections 2023-10-16T10:03:56.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.681+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/663228880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f00b2010 0x7fd6f00cc6e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.681+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/663228880 >> v2:172.21.15.190:3300/0 conn(0x7fd6f0071be0 0x7fd6f00b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.681+0000 7fd6f71ac700 1 -- 172.21.15.181:0/663228880 >> 172.21.15.181:0/663228880 conn(0x7fd6f00cd010 msgr2=0x7fd6f00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:56.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.681+0000 7fd6f71ac700 1 -- 172.21.15.181:0/663228880 shutdown_connections 2023-10-16T10:03:56.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.681+0000 7fd6f71ac700 1 -- 172.21.15.181:0/663228880 wait complete. 2023-10-16T10:03:56.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.682+0000 7fd6f71ac700 1 Processor -- start 2023-10-16T10:03:56.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.682+0000 7fd6f71ac700 1 -- start start 2023-10-16T10:03:56.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f71ac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f006f3b0 0x7fd6f014c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:56.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f71ac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f0071be0 0x7fd6f00c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:56.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f71ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b2010 0x7fd6f00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:56.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f71ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00cff60 con 0x7fd6f00b2010 2023-10-16T10:03:56.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f71ac700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f015c050 con 0x7fd6f006f3b0 2023-10-16T10:03:56.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f71ac700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f015c330 con 0x7fd6f0071be0 2023-10-16T10:03:56.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b2010 0x7fd6f00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:56.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b2010 0x7fd6f00c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48982/0 (socket says 172.21.15.181:48982) 2023-10-16T10:03:56.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 -- 172.21.15.181:0/4258409656 learned_addr learned my addr 172.21.15.181:0/4258409656 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:56.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f4f48700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f006f3b0 0x7fd6f014c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:56.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6effff700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f0071be0 0x7fd6f00c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:56.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 -- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f0071be0 msgr2=0x7fd6f00c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f0071be0 0x7fd6f00c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 -- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f006f3b0 msgr2=0x7fd6f014c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f006f3b0 0x7fd6f014c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 -- 172.21.15.181:0/4258409656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0094a0 con 0x7fd6f00b2010 2023-10-16T10:03:56.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f4f48700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f006f3b0 0x7fd6f014c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:56.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f5749700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b2010 0x7fd6f00c9db0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800ca80 tx=0x7fd6e800ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:56.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6ed7fa700 1 -- 172.21.15.181:0/4258409656 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800aaa0 con 0x7fd6f00b2010 2023-10-16T10:03:56.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6ed7fa700 1 -- 172.21.15.181:0/4258409656 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800ac00 con 0x7fd6f00b2010 2023-10-16T10:03:56.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e40094a0 con 0x7fd6f00b2010 2023-10-16T10:03:56.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.683+0000 7fd6ed7fa700 1 -- 172.21.15.181:0/4258409656 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8012400 con 0x7fd6f00b2010 2023-10-16T10:03:56.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.684+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f015c8f0 con 0x7fd6f00b2010 2023-10-16T10:03:56.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.684+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f01f21a0 con 0x7fd6f00b2010 2023-10-16T10:03:56.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.685+0000 7fd6ed7fa700 1 -- 172.21.15.181:0/4258409656 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd6e801b3b0 con 0x7fd6f00b2010 2023-10-16T10:03:56.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.685+0000 7fd6ed7fa700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd6d406b020 0x7fd6d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:56.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.685+0000 7fd6ed7fa700 1 -- 172.21.15.181:0/4258409656 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7fd6e80894c0 con 0x7fd6f00b2010 2023-10-16T10:03:56.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.685+0000 7fd6f4f48700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd6d406b020 0x7fd6d406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:56.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.686+0000 7fd6f4f48700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd6d406b020 0x7fd6d406d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fd0 tx=0x7fd6e4008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:56.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.690+0000 7fd6ed7fa700 1 -- 172.21.15.181:0/4258409656 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6f01f21a0 con 0x7fd6f00b2010 2023-10-16T10:03:56.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.882+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd6f01f21a0 con 0x7fd6f00b2010 2023-10-16T10:03:56.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.885+0000 7fd6ed7fa700 1 -- 172.21.15.181:0/4258409656 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6f01f21a0 con 0x7fd6f00b2010 2023-10-16T10:03:56.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd6d406b020 msgr2=0x7fd6d406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd6d406b020 0x7fd6d406d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fd0 tx=0x7fd6e4008030 comp rx=0 tx=0).stop 2023-10-16T10:03:56.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b2010 msgr2=0x7fd6f00c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:56.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b2010 0x7fd6f00c9db0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800ca80 tx=0x7fd6e800ce40 comp rx=0 tx=0).stop 2023-10-16T10:03:56.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 shutdown_connections 2023-10-16T10:03:56.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd6d406b020 0x7fd6d406d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b2010 0x7fd6f00c9db0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f006f3b0 0x7fd6f014c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 --2- 172.21.15.181:0/4258409656 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f0071be0 0x7fd6f00c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:56.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 >> 172.21.15.181:0/4258409656 conn(0x7fd6f00cd010 msgr2=0x7fd6f00b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:56.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 shutdown_connections 2023-10-16T10:03:56.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:56.887+0000 7fd6f71ac700 1 -- 172.21.15.181:0/4258409656 wait complete. 2023-10-16T10:03:56.901 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T10:03:56.902 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:56.942 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T10:03:56.992 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T10:03:56.992 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:56.993 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:57.010 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:03:57.010 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:57.011 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:57.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:57.031 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:03:57.031 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:57.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T10:03:57.067 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T10:03:57.068 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:03:57.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T10:03:57.089 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T10:03:57.090 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:03:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.225+0000 7f8d4d94b700 1 Processor -- start 2023-10-16T10:03:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d4d94b700 1 -- start start 2023-10-16T10:03:57.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d4d94b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8d480d2390 0x7f8d480cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d4d94b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d4d94b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480cc470 con 0x7f8d480ce7f0 2023-10-16T10:03:57.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d4d94b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480d0610 con 0x7f8d480d2390 2023-10-16T10:03:57.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d4d94b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4806c810 con 0x7f8d480d0180 2023-10-16T10:03:57.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d467fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8d480d2390 0x7f8d480cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d46ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8d480ce7f0 0x7f8d4806c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60064/0 (socket says 172.21.15.181:60064) 2023-10-16T10:03:57.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.226+0000 7f8d46ffd700 1 -- 172.21.15.181:0/1097537652 learned_addr learned my addr 172.21.15.181:0/1097537652 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:57.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d477fe700 1 --2- 172.21.15.181:0/1097537652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d467fc700 1 -- 172.21.15.181:0/1097537652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 msgr2=0x7f8d480cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:57.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d467fc700 1 --2- 172.21.15.181:0/1097537652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d467fc700 1 -- 172.21.15.181:0/1097537652 >> v1:172.21.15.181:6789/0 conn(0x7f8d480ce7f0 legacy=0x7f8d4806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d467fc700 1 -- 172.21.15.181:0/1097537652 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4806ca30 con 0x7f8d480d2390 2023-10-16T10:03:57.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d45ffb700 1 -- 172.21.15.181:0/1097537652 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023214661 0 0) 0x7f8d480cc470 con 0x7f8d480ce7f0 2023-10-16T10:03:57.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d467fc700 1 --2- 172.21.15.181:0/1097537652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480d2390 0x7f8d480cfc40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009870 tx=0x7f8d38009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49092d98cefd8918 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:57.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d45ffb700 1 -- 172.21.15.181:0/1097537652 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38011830 con 0x7f8d480d2390 2023-10-16T10:03:57.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d45ffb700 1 -- 172.21.15.181:0/1097537652 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d38011990 con 0x7f8d480d2390 2023-10-16T10:03:57.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.227+0000 7f8d45ffb700 1 -- 172.21.15.181:0/1097537652 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38011d20 con 0x7f8d480d2390 2023-10-16T10:03:57.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 -- 172.21.15.181:0/1097537652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480d2390 msgr2=0x7f8d480cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/1097537652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480d2390 0x7f8d480cfc40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009870 tx=0x7f8d38009b80 comp rx=0 tx=0).stop 2023-10-16T10:03:57.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 -- 172.21.15.181:0/1097537652 shutdown_connections 2023-10-16T10:03:57.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/1097537652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/1097537652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480d2390 0x7f8d480cfc40 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 -- 172.21.15.181:0/1097537652 >> 172.21.15.181:0/1097537652 conn(0x7f8d4809a9a0 msgr2=0x7f8d480d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:57.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 -- 172.21.15.181:0/1097537652 shutdown_connections 2023-10-16T10:03:57.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.228+0000 7f8d4d94b700 1 -- 172.21.15.181:0/1097537652 wait complete. 2023-10-16T10:03:57.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.229+0000 7f8d4d94b700 1 Processor -- start 2023-10-16T10:03:57.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.229+0000 7f8d4d94b700 1 -- start start 2023-10-16T10:03:57.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d4d94b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480ce7f0 0x7f8d48149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d4d94b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d4d94b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d480d2390 0x7f8d48070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d4d94b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4814a300 con 0x7f8d480ce7f0 2023-10-16T10:03:57.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d4d94b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4814b8c0 con 0x7f8d480d2390 2023-10-16T10:03:57.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d4d94b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4814bba0 con 0x7f8d480d0180 2023-10-16T10:03:57.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480ce7f0 0x7f8d48149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480ce7f0 0x7f8d48149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:48992/0 (socket says 172.21.15.181:48992) 2023-10-16T10:03:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 -- 172.21.15.181:0/2920990842 learned_addr learned my addr 172.21.15.181:0/2920990842 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d477fe700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d480d2390 0x7f8d48070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d467fc700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 -- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 msgr2=0x7f8d480701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 -- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d480d2390 msgr2=0x7f8d48070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d480d2390 0x7f8d48070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 -- 172.21.15.181:0/2920990842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380094a0 con 0x7f8d480ce7f0 2023-10-16T10:03:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d467fc700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:57.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d477fe700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d480d2390 0x7f8d48070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:03:57.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d46ffd700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480ce7f0 0x7f8d48149dc0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c4a0 tx=0x7f8d3c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:57.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d2ffff700 1 -- 172.21.15.181:0/2920990842 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0074e0 con 0x7f8d480ce7f0 2023-10-16T10:03:57.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d2ffff700 1 -- 172.21.15.181:0/2920990842 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c007640 con 0x7f8d480ce7f0 2023-10-16T10:03:57.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4814be80 con 0x7f8d480ce7f0 2023-10-16T10:03:57.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.230+0000 7f8d2ffff700 1 -- 172.21.15.181:0/2920990842 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c005a70 con 0x7f8d480ce7f0 2023-10-16T10:03:57.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.231+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4814c440 con 0x7f8d480ce7f0 2023-10-16T10:03:57.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.231+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d48067d20 con 0x7f8d480ce7f0 2023-10-16T10:03:57.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.232+0000 7f8d2ffff700 1 -- 172.21.15.181:0/2920990842 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8d3c005bd0 con 0x7f8d480ce7f0 2023-10-16T10:03:57.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.232+0000 7f8d2ffff700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8d3406b020 0x7f8d3406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.232+0000 7f8d2ffff700 1 -- 172.21.15.181:0/2920990842 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f8d3c088e10 con 0x7f8d480ce7f0 2023-10-16T10:03:57.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.232+0000 7f8d467fc700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8d3406b020 0x7f8d3406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.233+0000 7f8d467fc700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8d3406b020 0x7f8d3406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000c00 tx=0x7f8d38008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:57.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.237+0000 7f8d2ffff700 1 -- 172.21.15.181:0/2920990842 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8d3c053c10 con 0x7f8d480ce7f0 2023-10-16T10:03:57.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.471+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d480cef40 con 0x7f8d480ce7f0 2023-10-16T10:03:57.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.471+0000 7f8d2ffff700 1 -- 172.21.15.181:0/2920990842 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f8d3c017020 con 0x7f8d480ce7f0 2023-10-16T10:03:57.473 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:57.474 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7672,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2508916691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2508916691},{"type":"v1","addr":"172.21.15.181:6836","nonce":2508916691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7687,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1303363384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1303363384},{"type":"v1","addr":"172.21.15.190:6834","nonce":1303363384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8065,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/221469978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":221469978},{"type":"v1","addr":"172.21.15.190:6835","nonce":221469978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/300739418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":300739418},{"type":"v1","addr":"172.21.15.181:6837","nonce":300739418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[]} 2023-10-16T10:03:57.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8d3406b020 msgr2=0x7f8d3406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8d3406b020 0x7f8d3406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000c00 tx=0x7f8d38008030 comp rx=0 tx=0).stop 2023-10-16T10:03:57.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480ce7f0 msgr2=0x7f8d48149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480ce7f0 0x7f8d48149dc0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c4a0 tx=0x7f8d3c00c860 comp rx=0 tx=0).stop 2023-10-16T10:03:57.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 shutdown_connections 2023-10-16T10:03:57.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8d3406b020 0x7f8d3406d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d480d2390 0x7f8d48070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d480ce7f0 0x7f8d48149dc0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 --2- 172.21.15.181:0/2920990842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d480d0180 0x7f8d480701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.473+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 >> 172.21.15.181:0/2920990842 conn(0x7f8d4809a9a0 msgr2=0x7f8d480c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:57.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.474+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 shutdown_connections 2023-10-16T10:03:57.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.474+0000 7f8d4d94b700 1 -- 172.21.15.181:0/2920990842 wait complete. 2023-10-16T10:03:57.478 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 69 2023-10-16T10:03:57.488 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T10:03:57.488 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T10:03:57.488 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T10:03:57.488 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:57.489 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T10:03:57.489 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T10:03:57.489 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T10:03:57.490 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:57.490 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T10:03:57.490 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T10:03:57.490 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T10:03:57.491 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:57.491 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T10:03:57.491 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T10:03:57.491 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T10:03:57.492 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T10:03:57.497 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T10:03:57.497 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T10:03:57.498 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T10:03:57.499 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T10:03:57.502 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T10:03:57.502 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T10:03:57.503 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T10:03:57.507 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T10:03:57.540 INFO:tasks.ceph.mds.a:Started 2023-10-16T10:03:57.541 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:57.542 INFO:tasks.ceph.mds.c:Started 2023-10-16T10:03:57.542 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:57.544 INFO:tasks.ceph.mds.b:Started 2023-10-16T10:03:57.545 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:57.545 INFO:tasks.ceph.mds.d:Started 2023-10-16T10:03:57.546 DEBUG:teuthology.parallel:result is None 2023-10-16T10:03:57.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T10:03:57.619 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T10:03:57.620 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T10:03:57.632 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T10:03:57.633 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T10:03:57.633 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-10-16T10:03:57.631+0000 7f500eb1c700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-16T10:03:57.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.673+0000 7fd8f3aaa700 1 Processor -- start 2023-10-16T10:03:57.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.673+0000 7fd8f3aaa700 1 -- start start 2023-10-16T10:03:57.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f3aaa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8ec0cd4b0 0x7fd8ec0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f3aaa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec06c700 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f3aaa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec0cf460 con 0x7fd8ec06d210 2023-10-16T10:03:57.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f3aaa700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0d0380 con 0x7fd8ec0cd4b0 2023-10-16T10:03:57.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f1045700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd8ec0ce4d0 0x7fd8ec15d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60106/0 (socket says 172.21.15.181:60106) 2023-10-16T10:03:57.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f1045700 1 -- 172.21.15.181:0/3243594087 learned_addr learned my addr 172.21.15.181:0/3243594087 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:57.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f2047700 1 --2- 172.21.15.181:0/3243594087 >> v2:172.21.15.190:3300/0 conn(0x7fd8ec0cd4b0 0x7fd8ec0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548002397 0 0) 0x7fd8ec06c700 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d40035b0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409261312 0 0) 0x7fd8ec0cf460 con 0x7fd8ec06d210 2023-10-16T10:03:57.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8ec06c700 con 0x7fd8ec06d210 2023-10-16T10:03:57.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f2047700 1 -- 172.21.15.181:0/3243594087 >> v1:172.21.15.190:6789/0 conn(0x7fd8ec06d210 legacy=0x7fd8ec0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.674+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1926852802 0 0) 0x7fd8d40035b0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f2047700 1 -- 172.21.15.181:0/3243594087 >> v1:172.21.15.181:6789/0 conn(0x7fd8ec0ce4d0 legacy=0x7fd8ec15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f2047700 1 -- 172.21.15.181:0/3243594087 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec06d5f0 con 0x7fd8ec0cd4b0 2023-10-16T10:03:57.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f2047700 1 --2- 172.21.15.181:0/3243594087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec0cd4b0 0x7fd8ec0cc4c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009d90 tx=0x7fd8dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e4e5a557b0d708 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:57.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc00e070 con 0x7fd8ec0cd4b0 2023-10-16T10:03:57.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc003a20 con 0x7fd8ec0cd4b0 2023-10-16T10:03:57.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f0844700 1 -- 172.21.15.181:0/3243594087 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0001670 con 0x7fd8ec0cd4b0 2023-10-16T10:03:57.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3243594087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec0cd4b0 msgr2=0x7fd8ec0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3243594087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec0cd4b0 0x7fd8ec0cc4c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009d90 tx=0x7fd8dc00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:57.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3243594087 shutdown_connections 2023-10-16T10:03:57.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3243594087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec0cd4b0 0x7fd8ec0cc4c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.675+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3243594087 >> 172.21.15.181:0/3243594087 conn(0x7fd8ec09a9a0 msgr2=0x7fd8ec0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:57.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.676+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3243594087 shutdown_connections 2023-10-16T10:03:57.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.676+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3243594087 wait complete. 2023-10-16T10:03:57.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 Processor -- start 2023-10-16T10:03:57.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 -- start start 2023-10-16T10:03:57.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec06d210 0x7fd8ec15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ec0cd4b0 0x7fd8ec0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8ec0ce4d0 0x7fd8ec15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec15c190 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0c6ef0 con 0x7fd8ec06d210 2023-10-16T10:03:57.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f3aaa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0c71d0 con 0x7fd8ec0cd4b0 2023-10-16T10:03:57.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.677+0000 7fd8f2047700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8ec0ce4d0 0x7fd8ec15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8ec0ce4d0 0x7fd8ec15ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49036/0 (socket says 172.21.15.181:49036) 2023-10-16T10:03:57.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 -- 172.21.15.181:0/3530264918 learned_addr learned my addr 172.21.15.181:0/3530264918 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:57.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f1846700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec06d210 0x7fd8ec15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f1045700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ec0cd4b0 0x7fd8ec0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 -- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ec0cd4b0 msgr2=0x7fd8ec0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ec0cd4b0 0x7fd8ec0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 -- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec06d210 msgr2=0x7fd8ec15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:57.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec06d210 0x7fd8ec15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:57.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 -- 172.21.15.181:0/3530264918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc009a70 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f2047700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8ec0ce4d0 0x7fd8ec15ce20 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00b370 tx=0x7fd8dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:57.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8ea7fc700 1 -- 172.21.15.181:0/3530264918 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0044c0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8ea7fc700 1 -- 172.21.15.181:0/3530264918 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc004620 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec0c74b0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8ea7fc700 1 -- 172.21.15.181:0/3530264918 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc012430 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.678+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec0c6be0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.679+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec0c79d0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.680+0000 7fd8ea7fc700 1 -- 172.21.15.181:0/3530264918 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd8dc012590 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.680+0000 7fd8ea7fc700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd8d806b020 0x7fd8d806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:57.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.680+0000 7fd8ea7fc700 1 -- 172.21.15.181:0/3530264918 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6126+0+0 (secure 0 0 0) 0x7fd8dc08ad90 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.680+0000 7fd8f1846700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd8d806b020 0x7fd8d806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:57.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.680+0000 7fd8f1846700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd8d806b020 0x7fd8d806d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40011e0 tx=0x7fd8e4009040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:57.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.684+0000 7fd8ea7fc700 1 -- 172.21.15.181:0/3530264918 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd8ec0c79d0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:57.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:57.892+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd8ec0c79d0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:58.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.545+0000 7fd8ea7fc700 1 -- 172.21.15.181:0/3530264918 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd8ec0c79d0 con 0x7fd8ec0ce4d0 2023-10-16T10:03:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd8d806b020 msgr2=0x7fd8d806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:58.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd8d806b020 0x7fd8d806d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40011e0 tx=0x7fd8e4009040 comp rx=0 tx=0).stop 2023-10-16T10:03:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8ec0ce4d0 msgr2=0x7fd8ec15ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8ec0ce4d0 0x7fd8ec15ce20 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00b370 tx=0x7fd8dc004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 shutdown_connections 2023-10-16T10:03:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd8d806b020 0x7fd8d806d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8ec0ce4d0 0x7fd8ec15ce20 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8ec06d210 0x7fd8ec15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 --2- 172.21.15.181:0/3530264918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8ec0cd4b0 0x7fd8ec0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.548+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 >> 172.21.15.181:0/3530264918 conn(0x7fd8ec09a9a0 msgr2=0x7fd8ec0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.549+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 shutdown_connections 2023-10-16T10:03:58.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.549+0000 7fd8f3aaa700 1 -- 172.21.15.181:0/3530264918 wait complete. 2023-10-16T10:03:58.553 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T10:03:58.566 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T10:03:58.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.691+0000 7f99bc57a700 1 Processor -- start 2023-10-16T10:03:58.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.691+0000 7f99bc57a700 1 -- start start 2023-10-16T10:03:58.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.692+0000 7f99bc57a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99b415dfd0 0x7f99b40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:58.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.692+0000 7f99bc57a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b40af450 0x7f99b406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:58.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.692+0000 7f99bc57a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b406f870 con 0x7f99b406db80 2023-10-16T10:03:58.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.692+0000 7f99bc57a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b40d1160 con 0x7f99b415dfd0 2023-10-16T10:03:58.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.692+0000 7f99bc57a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40cc8e0 con 0x7f99b40af450 2023-10-16T10:03:58.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99b9b15700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b40af450 0x7f99b406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:58.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99bab17700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f99b406db80 0x7f99b40cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:44066/0 (socket says 172.21.15.181:44066) 2023-10-16T10:03:58.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99ba316700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99b415dfd0 0x7f99b40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:58.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99bab17700 1 -- 172.21.15.181:0/1289889079 learned_addr learned my addr 172.21.15.181:0/1289889079 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:58.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99b9314700 1 -- 172.21.15.181:0/1289889079 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836776693 0 0) 0x7f99b406f870 con 0x7f99b406db80 2023-10-16T10:03:58.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99b9314700 1 -- 172.21.15.181:0/1289889079 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a00035b0 con 0x7f99b406db80 2023-10-16T10:03:58.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99b9b15700 1 -- 172.21.15.181:0/1289889079 >> v2:172.21.15.190:3300/0 conn(0x7f99b415dfd0 msgr2=0x7f99b40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:58.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99b9b15700 1 --2- 172.21.15.181:0/1289889079 >> v2:172.21.15.190:3300/0 conn(0x7f99b415dfd0 0x7f99b40af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99b9b15700 1 -- 172.21.15.181:0/1289889079 >> v1:172.21.15.190:6789/0 conn(0x7f99b406db80 legacy=0x7f99b40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:58.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99b9b15700 1 -- 172.21.15.181:0/1289889079 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40ccbc0 con 0x7f99b40af450 2023-10-16T10:03:58.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.693+0000 7f99ba316700 1 --2- 172.21.15.181:0/1289889079 >> v2:172.21.15.190:3300/0 conn(0x7f99b415dfd0 0x7f99b40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:03:58.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99b9b15700 1 --2- 172.21.15.181:0/1289889079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b40af450 0x7f99b406d640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009840 tx=0x7f99b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6cec657274f9c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:58.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99b9314700 1 -- 172.21.15.181:0/1289889079 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0011830 con 0x7f99b40af450 2023-10-16T10:03:58.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99b9314700 1 -- 172.21.15.181:0/1289889079 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b0011990 con 0x7f99b40af450 2023-10-16T10:03:58.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99b9314700 1 -- 172.21.15.181:0/1289889079 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0011d00 con 0x7f99b40af450 2023-10-16T10:03:58.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 -- 172.21.15.181:0/1289889079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b40af450 msgr2=0x7f99b406d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:58.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 --2- 172.21.15.181:0/1289889079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b40af450 0x7f99b406d640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009840 tx=0x7f99b0009b50 comp rx=0 tx=0).stop 2023-10-16T10:03:58.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 -- 172.21.15.181:0/1289889079 shutdown_connections 2023-10-16T10:03:58.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 --2- 172.21.15.181:0/1289889079 >> v2:172.21.15.190:3300/0 conn(0x7f99b415dfd0 0x7f99b40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 --2- 172.21.15.181:0/1289889079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b40af450 0x7f99b406d640 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 -- 172.21.15.181:0/1289889079 >> 172.21.15.181:0/1289889079 conn(0x7f99b40d04a0 msgr2=0x7f99b40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:58.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 -- 172.21.15.181:0/1289889079 shutdown_connections 2023-10-16T10:03:58.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.694+0000 7f99bc57a700 1 -- 172.21.15.181:0/1289889079 wait complete. 2023-10-16T10:03:58.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 Processor -- start 2023-10-16T10:03:58.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 -- start start 2023-10-16T10:03:58.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b406db80 0x7f99b4158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:58.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99b40af450 0x7f99b415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:58.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99b415dfd0 0x7f99b415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:58.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40b1f40 con 0x7f99b415dfd0 2023-10-16T10:03:58.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40b0ea0 con 0x7f99b40af450 2023-10-16T10:03:58.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bc57a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40b1180 con 0x7f99b406db80 2023-10-16T10:03:58.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bab17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99b415dfd0 0x7f99b415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bab17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99b415dfd0 0x7f99b415d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49052/0 (socket says 172.21.15.181:49052) 2023-10-16T10:03:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99bab17700 1 -- 172.21.15.181:0/3960845510 learned_addr learned my addr 172.21.15.181:0/3960845510 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.696+0000 7f99b9b15700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99b40af450 0x7f99b415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99ba316700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b406db80 0x7f99b4158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bab17700 1 -- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b406db80 msgr2=0x7f99b4158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:58.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bab17700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b406db80 0x7f99b4158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bab17700 1 -- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99b40af450 msgr2=0x7f99b415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:58.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bab17700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99b40af450 0x7f99b415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:58.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bab17700 1 -- 172.21.15.181:0/3960845510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4009430 con 0x7f99b415dfd0 2023-10-16T10:03:58.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bab17700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99b415dfd0 0x7f99b415d340 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00cb40 tx=0x7f99ac00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:58.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99aaffd700 1 -- 172.21.15.181:0/3960845510 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00a9e0 con 0x7f99b415dfd0 2023-10-16T10:03:58.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99aaffd700 1 -- 172.21.15.181:0/3960845510 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac00ab40 con 0x7f99b415dfd0 2023-10-16T10:03:58.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b00094a0 con 0x7f99b415dfd0 2023-10-16T10:03:58.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99aaffd700 1 -- 172.21.15.181:0/3960845510 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac012400 con 0x7f99b415dfd0 2023-10-16T10:03:58.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.697+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b40b1740 con 0x7f99b415dfd0 2023-10-16T10:03:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.698+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b4067d20 con 0x7f99b415dfd0 2023-10-16T10:03:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.698+0000 7f99aaffd700 1 -- 172.21.15.181:0/3960845510 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f99ac01b3b0 con 0x7f99b415dfd0 2023-10-16T10:03:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.699+0000 7f99aaffd700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f999806b020 0x7f999806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.699+0000 7f99aaffd700 1 -- 172.21.15.181:0/3960845510 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5309+0+0 (secure 0 0 0) 0x7f99ac089150 con 0x7f99b415dfd0 2023-10-16T10:03:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.699+0000 7f99ba316700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f999806b020 0x7f999806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:58.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.699+0000 7f99ba316700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f999806b020 0x7f999806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f99b41e88c0 tx=0x7f99a40091c0 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:58.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.703+0000 7f99aaffd700 1 -- 172.21.15.181:0/3960845510 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f99ac054330 con 0x7f99b415dfd0 2023-10-16T10:03:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.973+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f99b40cb050 con 0x7f99b415dfd0 2023-10-16T10:03:58.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.974+0000 7f99aaffd700 1 -- 172.21.15.181:0/3960845510 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f99ac057950 con 0x7f99b415dfd0 2023-10-16T10:03:58.976 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:58.976 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:03:58.976 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T10:03:58.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.977 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T10:03:58.977 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T10:03:58.977 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.977 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:58.978 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:58.978 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:58.978 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.978 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.979 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T10:03:58.979 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T10:03:58.979 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.980 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:58.980 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:58.980 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:58.980 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.980 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.981 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T10:03:58.981 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T10:03:58.981 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.981 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:58.982 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:58.982 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:58.982 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.982 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.983 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T10:03:58.983 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T10:03:58.983 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.983 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:58.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:58.984 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:58.984 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.984 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.985 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T10:03:58.985 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T10:03:58.985 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.985 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.985 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.986 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.986 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.986 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T10:03:58.987 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T10:03:58.987 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.987 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.987 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.987 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.988 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.988 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T10:03:58.988 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T10:03:58.988 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.989 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.989 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.989 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.989 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.990 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T10:03:58.990 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T10:03:58.990 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.991 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.991 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.991 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.992 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T10:03:58.992 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T10:03:58.992 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.992 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.992 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.993 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.993 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.993 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T10:03:58.993 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T10:03:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.994 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.995 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T10:03:58.995 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T10:03:58.995 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.995 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.996 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.996 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.996 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T10:03:58.997 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T10:03:58.997 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.997 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T10:03:58.997 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T10:03:58.997 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:58.998 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:58.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:58.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:58.998 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T10:03:58.998 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T10:03:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T10:03:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T10:03:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T10:03:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T10:03:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:03:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:03:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:03:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T10:03:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T10:03:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:03:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:03:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:03:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T10:03:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T10:03:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T10:03:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T10:03:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T10:03:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T10:03:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T10:03:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T10:03:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:03:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T10:03:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T10:03:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T10:03:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T10:03:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:03:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:03:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T10:03:59.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:03:59.010 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T10:03:59.010 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T10:03:59.011 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T10:03:59.011 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T10:03:59.011 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T10:03:59.011 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T10:03:59.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.012 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:03:59.012 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T10:03:59.012 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:03:59.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f999806b020 msgr2=0x7f999806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f999806b020 0x7f999806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f99b41e88c0 tx=0x7f99a40091c0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99b415dfd0 msgr2=0x7f99b415d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99b415dfd0 0x7f99b415d340 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00cb40 tx=0x7f99ac00ce50 comp rx=0 tx=0).stop 2023-10-16T10:03:59.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 shutdown_connections 2023-10-16T10:03:59.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f999806b020 0x7f999806d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99b406db80 0x7f99b4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99b415dfd0 0x7f99b415d340 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 --2- 172.21.15.181:0/3960845510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99b40af450 0x7f99b415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 >> 172.21.15.181:0/3960845510 conn(0x7f99b40d04a0 msgr2=0x7f99b415ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:59.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 shutdown_connections 2023-10-16T10:03:59.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:58.976+0000 7f99bc57a700 1 -- 172.21.15.181:0/3960845510 wait complete. 2023-10-16T10:03:59.016 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:03:59.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.136+0000 7fb1ab1cf700 1 Processor -- start 2023-10-16T10:03:59.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.136+0000 7fb1ab1cf700 1 -- start start 2023-10-16T10:03:59.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.136+0000 7fb1ab1cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1a40cdb80 0x7fb1a40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.136+0000 7fb1ab1cf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a406f870 con 0x7fb1a40af450 2023-10-16T10:03:59.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.136+0000 7fb1ab1cf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40d1160 con 0x7fb1a415dfd0 2023-10-16T10:03:59.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.136+0000 7fb1ab1cf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a40cc8e0 con 0x7fb1a40cdb80 2023-10-16T10:03:59.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a976c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1a40cdb80 0x7fb1a40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a976c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1a40cdb80 0x7fb1a40af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49056/0 (socket says 172.21.15.181:49056) 2023-10-16T10:03:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a976c700 1 -- 172.21.15.181:0/3682813293 learned_addr learned my addr 172.21.15.181:0/3682813293 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a3fff700 1 --1- 172.21.15.181:0/3682813293 >> v1:172.21.15.181:6789/0 conn(0x7fb1a40af450 0x7fb1a40cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60134/0 (socket says 172.21.15.181:60134) 2023-10-16T10:03:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a976c700 1 -- 172.21.15.181:0/3682813293 >> v1:172.21.15.190:6789/0 conn(0x7fb1a415dfd0 legacy=0x7fb1a406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a976c700 1 -- 172.21.15.181:0/3682813293 >> v1:172.21.15.181:6789/0 conn(0x7fb1a40af450 legacy=0x7fb1a40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a976c700 1 -- 172.21.15.181:0/3682813293 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40ccbc0 con 0x7fb1a40cdb80 2023-10-16T10:03:59.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a976c700 1 --2- 172.21.15.181:0/3682813293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a40cdb80 0x7fb1a40af9a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009e50 tx=0x7fb19c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24688f8407aab0cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a37fe700 1 -- 172.21.15.181:0/3682813293 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00e070 con 0x7fb1a40cdb80 2023-10-16T10:03:59.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a37fe700 1 -- 172.21.15.181:0/3682813293 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00b9c0 con 0x7fb1a40cdb80 2023-10-16T10:03:59.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.137+0000 7fb1a37fe700 1 -- 172.21.15.181:0/3682813293 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00bd70 con 0x7fb1a40cdb80 2023-10-16T10:03:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.138+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3682813293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a40cdb80 msgr2=0x7fb1a40af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.138+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3682813293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a40cdb80 0x7fb1a40af9a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009e50 tx=0x7fb19c00b330 comp rx=0 tx=0).stop 2023-10-16T10:03:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.138+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3682813293 shutdown_connections 2023-10-16T10:03:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.138+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3682813293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a40cdb80 0x7fb1a40af9a0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.138+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3682813293 >> 172.21.15.181:0/3682813293 conn(0x7fb1a40d04a0 msgr2=0x7fb1a40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:59.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.138+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3682813293 shutdown_connections 2023-10-16T10:03:59.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.138+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3682813293 wait complete. 2023-10-16T10:03:59.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.139+0000 7fb1ab1cf700 1 Processor -- start 2023-10-16T10:03:59.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.139+0000 7fb1ab1cf700 1 -- start start 2023-10-16T10:03:59.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1ab1cf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40af450 0x7fb1a4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1ab1cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40cdb80 0x7fb1a40b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1ab1cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a415dfd0 0x7fb1a40b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1ab1cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c85c0 con 0x7fb1a415dfd0 2023-10-16T10:03:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1ab1cf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c8810 con 0x7fb1a40af450 2023-10-16T10:03:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1ab1cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c8af0 con 0x7fb1a40cdb80 2023-10-16T10:03:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a415dfd0 0x7fb1a40b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a415dfd0 0x7fb1a40b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49066/0 (socket says 172.21.15.181:49066) 2023-10-16T10:03:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 -- 172.21.15.181:0/3626565926 learned_addr learned my addr 172.21.15.181:0/3626565926 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:59.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a3fff700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40cdb80 0x7fb1a40b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a8f6b700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40af450 0x7fb1a4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 -- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40cdb80 msgr2=0x7fb1a40b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40cdb80 0x7fb1a40b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 -- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40af450 msgr2=0x7fb1a4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40af450 0x7fb1a4158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 -- 172.21.15.181:0/3626565926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c009b30 con 0x7fb1a415dfd0 2023-10-16T10:03:59.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a3fff700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40cdb80 0x7fb1a40b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:59.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a8f6b700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40af450 0x7fb1a4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:03:59.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.140+0000 7fb1a976c700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a415dfd0 0x7fb1a40b5840 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb19c012000 tx=0x7fb19c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.141+0000 7fb1a17fa700 1 -- 172.21.15.181:0/3626565926 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0099b0 con 0x7fb1a415dfd0 2023-10-16T10:03:59.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.141+0000 7fb1a17fa700 1 -- 172.21.15.181:0/3626565926 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c0092e0 con 0x7fb1a415dfd0 2023-10-16T10:03:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.141+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a40c8dd0 con 0x7fb1a415dfd0 2023-10-16T10:03:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.141+0000 7fb1a17fa700 1 -- 172.21.15.181:0/3626565926 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c01c640 con 0x7fb1a415dfd0 2023-10-16T10:03:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.141+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a40c9390 con 0x7fb1a415dfd0 2023-10-16T10:03:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.142+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a4067d20 con 0x7fb1a415dfd0 2023-10-16T10:03:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.142+0000 7fb1a17fa700 1 -- 172.21.15.181:0/3626565926 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb19c01c7a0 con 0x7fb1a415dfd0 2023-10-16T10:03:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.143+0000 7fb1a17fa700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fb18806b020 0x7fb18806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.143+0000 7fb1a8f6b700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fb18806b020 0x7fb18806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.143+0000 7fb1a17fa700 1 -- 172.21.15.181:0/3626565926 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5309+0+0 (secure 0 0 0) 0x7fb19c08b520 con 0x7fb1a415dfd0 2023-10-16T10:03:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.143+0000 7fb1a8f6b700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fb18806b020 0x7fb18806d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40c7520 tx=0x7fb198009040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.147+0000 7fb1a17fa700 1 -- 172.21.15.181:0/3626565926 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb19c0565d0 con 0x7fb1a415dfd0 2023-10-16T10:03:59.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.340+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1a40c9550 con 0x7fb1a415dfd0 2023-10-16T10:03:59.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.341+0000 7fb1a17fa700 1 -- 172.21.15.181:0/3626565926 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+13592 (secure 0 0 0) 0x7fb19c059bf0 con 0x7fb1a415dfd0 2023-10-16T10:03:59.343 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:03:59.343 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":141,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:03:58.560123+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:03:59.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.343+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fb18806b020 msgr2=0x7fb18806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.343+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fb18806b020 0x7fb18806d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40c7520 tx=0x7fb198009040 comp rx=0 tx=0).stop 2023-10-16T10:03:59.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.343+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a415dfd0 msgr2=0x7fb1a40b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.343+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a415dfd0 0x7fb1a40b5840 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb19c012000 tx=0x7fb19c004ad0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 shutdown_connections 2023-10-16T10:03:59.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fb18806b020 0x7fb18806d4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a415dfd0 0x7fb1a40b5840 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40af450 0x7fb1a4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 --2- 172.21.15.181:0/3626565926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40cdb80 0x7fb1a40b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 >> 172.21.15.181:0/3626565926 conn(0x7fb1a40d04a0 msgr2=0x7fb1a40cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:59.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 shutdown_connections 2023-10-16T10:03:59.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.344+0000 7fb1ab1cf700 1 -- 172.21.15.181:0/3626565926 wait complete. 2023-10-16T10:03:59.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T10:03:59.358 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T10:03:59.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.479+0000 7fbe875df700 1 Processor -- start 2023-10-16T10:03:59.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.479+0000 7fbe875df700 1 -- start start 2023-10-16T10:03:59.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe875df700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbe80072760 0x7fbe8006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe875df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 0x7fbe801615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe875df700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8006b210 con 0x7fbe8015da70 2023-10-16T10:03:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe875df700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe800cef00 con 0x7fbe80072760 2023-10-16T10:03:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe875df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800cec20 con 0x7fbe8006ade0 2023-10-16T10:03:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe8537b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbe8015da70 0x7fbe800ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60148/0 (socket says 172.21.15.181:60148) 2023-10-16T10:03:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe8537b700 1 -- 172.21.15.181:0/1187285208 learned_addr learned my addr 172.21.15.181:0/1187285208 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:03:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596759953 0 0) 0x7fbe8006b210 con 0x7fbe8015da70 2023-10-16T10:03:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe6c0035b0 con 0x7fbe8015da70 2023-10-16T10:03:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe84b7a700 1 --2- 172.21.15.181:0/1187285208 >> v2:172.21.15.190:3300/0 conn(0x7fbe80072760 0x7fbe8006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.480+0000 7fbe85b7c700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 0x7fbe801615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070279166 0 0) 0x7fbe6c0035b0 con 0x7fbe8015da70 2023-10-16T10:03:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 msgr2=0x7fbe801615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 0x7fbe801615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 >> v2:172.21.15.190:3300/0 conn(0x7fbe80072760 msgr2=0x7fbe8006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 --2- 172.21.15.181:0/1187285208 >> v2:172.21.15.190:3300/0 conn(0x7fbe80072760 0x7fbe8006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80071c80 con 0x7fbe8015da70 2023-10-16T10:03:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fbe7c001670 con 0x7fbe8015da70 2023-10-16T10:03:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 >> v1:172.21.15.181:6789/0 conn(0x7fbe8015da70 legacy=0x7fbe800ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe6c003a00 0x7fbe6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 0x7fbe6c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe6c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800cef00 con 0x7fbe6c004d90 2023-10-16T10:03:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800cec20 con 0x7fbe6c003a00 2023-10-16T10:03:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c00bb70 con 0x7fbe8006ade0 2023-10-16T10:03:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe85b7c700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe6c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe8537b700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe6c003a00 0x7fbe6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe84b7a700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 0x7fbe6c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe85b7c700 1 -- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 msgr2=0x7fbe6c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe85b7c700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe8006ade0 0x7fbe6c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe85b7c700 1 -- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe6c003a00 msgr2=0x7fbe6c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe85b7c700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe6c003a00 0x7fbe6c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe85b7c700 1 -- 172.21.15.181:0/1187285208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0035b0 con 0x7fbe6c004d90 2023-10-16T10:03:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe85b7c700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe6c005210 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fbe78007360 tx=0x7fbe6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2089f3927f850486 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78007e80 con 0x7fbe6c004d90 2023-10-16T10:03:59.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78014400 con 0x7fbe6c004d90 2023-10-16T10:03:59.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.481+0000 7fbe77fff700 1 -- 172.21.15.181:0/1187285208 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78014790 con 0x7fbe6c004d90 2023-10-16T10:03:59.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.482+0000 7fbe875df700 1 -- 172.21.15.181:0/1187285208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 msgr2=0x7fbe6c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.482+0000 7fbe875df700 1 --2- 172.21.15.181:0/1187285208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe6c005210 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fbe78007360 tx=0x7fbe6c005880 comp rx=0 tx=0).stop 2023-10-16T10:03:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.482+0000 7fbe85b7c700 1 -- 172.21.15.181:0/1187285208 reap_dead start 2023-10-16T10:03:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.482+0000 7fbe875df700 1 -- 172.21.15.181:0/1187285208 shutdown_connections 2023-10-16T10:03:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.482+0000 7fbe875df700 1 -- 172.21.15.181:0/1187285208 >> 172.21.15.181:0/1187285208 conn(0x7fbe800cd010 msgr2=0x7fbe800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.482+0000 7fbe875df700 1 -- 172.21.15.181:0/1187285208 shutdown_connections 2023-10-16T10:03:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.482+0000 7fbe875df700 1 -- 172.21.15.181:0/1187285208 wait complete. 2023-10-16T10:03:59.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.483+0000 7fbe875df700 1 Processor -- start 2023-10-16T10:03:59.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.483+0000 7fbe875df700 1 -- start start 2023-10-16T10:03:59.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe875df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe8015af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe875df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe800b5350 0x7fbe800c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe875df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe8006fb90 0x7fbe80070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe875df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800b6310 con 0x7fbe6c004d90 2023-10-16T10:03:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe875df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80070550 con 0x7fbe8006fb90 2023-10-16T10:03:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe875df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800c41f0 con 0x7fbe800b5350 2023-10-16T10:03:59.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe8015af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe8015af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49082/0 (socket says 172.21.15.181:49082) 2023-10-16T10:03:59.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 -- 172.21.15.181:0/3901072629 learned_addr learned my addr 172.21.15.181:0/3901072629 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:59.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe85b7c700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe8006fb90 0x7fbe80070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe84b7a700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe800b5350 0x7fbe800c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 -- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe800b5350 msgr2=0x7fbe800c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe800b5350 0x7fbe800c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 -- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe8006fb90 msgr2=0x7fbe80070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe8006fb90 0x7fbe80070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 -- 172.21.15.181:0/3901072629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800c4490 con 0x7fbe6c004d90 2023-10-16T10:03:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.484+0000 7fbe8537b700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe8015af90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c000c00 tx=0x7fbe7c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.485+0000 7fbe75ffb700 1 -- 172.21.15.181:0/3901072629 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00f690 con 0x7fbe6c004d90 2023-10-16T10:03:59.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.485+0000 7fbe75ffb700 1 -- 172.21.15.181:0/3901072629 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c00f810 con 0x7fbe6c004d90 2023-10-16T10:03:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.485+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe800c4770 con 0x7fbe6c004d90 2023-10-16T10:03:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.485+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe800c4d30 con 0x7fbe6c004d90 2023-10-16T10:03:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.485+0000 7fbe75ffb700 1 -- 172.21.15.181:0/3901072629 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00fbc0 con 0x7fbe6c004d90 2023-10-16T10:03:59.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.486+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe600052f0 con 0x7fbe6c004d90 2023-10-16T10:03:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.486+0000 7fbe75ffb700 1 -- 172.21.15.181:0/3901072629 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbe7c00fd20 con 0x7fbe6c004d90 2023-10-16T10:03:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.487+0000 7fbe75ffb700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fbe6406b020 0x7fbe6406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.487+0000 7fbe75ffb700 1 -- 172.21.15.181:0/3901072629 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5309+0+0 (secure 0 0 0) 0x7fbe7c089560 con 0x7fbe6c004d90 2023-10-16T10:03:59.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.487+0000 7fbe84b7a700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fbe6406b020 0x7fbe6406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.487+0000 7fbe84b7a700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fbe6406b020 0x7fbe6406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe80161420 tx=0x7fbe70008040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.491+0000 7fbe75ffb700 1 -- 172.21.15.181:0/3901072629 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe7c054690 con 0x7fbe6c004d90 2023-10-16T10:03:59.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.690+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe60005b20 con 0x7fbe6c004d90 2023-10-16T10:03:59.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.691+0000 7fbe75ffb700 1 -- 172.21.15.181:0/3901072629 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe7c057cb0 con 0x7fbe6c004d90 2023-10-16T10:03:59.693 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T10:03:59.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fbe6406b020 msgr2=0x7fbe6406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fbe6406b020 0x7fbe6406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe80161420 tx=0x7fbe70008040 comp rx=0 tx=0).stop 2023-10-16T10:03:59.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 msgr2=0x7fbe8015af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe8015af90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c000c00 tx=0x7fbe7c0075f0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 shutdown_connections 2023-10-16T10:03:59.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fbe6406b020 0x7fbe6406d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe8006fb90 0x7fbe80070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe6c004d90 0x7fbe8015af90 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 --2- 172.21.15.181:0/3901072629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe800b5350 0x7fbe800c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 >> 172.21.15.181:0/3901072629 conn(0x7fbe800cd010 msgr2=0x7fbe800b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:59.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 shutdown_connections 2023-10-16T10:03:59.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.693+0000 7fbe875df700 1 -- 172.21.15.181:0/3901072629 wait complete. 2023-10-16T10:03:59.707 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T10:03:59.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.825+0000 7f4a28948700 1 Processor -- start 2023-10-16T10:03:59.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.825+0000 7f4a28948700 1 -- start start 2023-10-16T10:03:59.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.826+0000 7f4a28948700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a240d0370 0x7f4a240ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.826+0000 7f4a28948700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a240cd2d0 0x7f4a240d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.826+0000 7f4a28948700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240cc510 con 0x7f4a2415b990 2023-10-16T10:03:59.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.826+0000 7f4a28948700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240d24f0 con 0x7f4a240d0370 2023-10-16T10:03:59.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.826+0000 7f4a28948700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240cc7f0 con 0x7f4a240cd2d0 2023-10-16T10:03:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a240d0370 0x7f4a240ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a240d0370 0x7f4a240ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49092/0 (socket says 172.21.15.181:49092) 2023-10-16T10:03:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 -- 172.21.15.181:0/577286519 learned_addr learned my addr 172.21.15.181:0/577286519 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:59.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a22d9d700 1 --2- 172.21.15.181:0/577286519 >> v2:172.21.15.190:3300/0 conn(0x7f4a240cd2d0 0x7f4a240d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 -- 172.21.15.181:0/577286519 >> v2:172.21.15.190:3300/0 conn(0x7f4a240cd2d0 msgr2=0x7f4a240d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:03:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 --2- 172.21.15.181:0/577286519 >> v2:172.21.15.190:3300/0 conn(0x7f4a240cd2d0 0x7f4a240d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 -- 172.21.15.181:0/577286519 >> v1:172.21.15.190:6789/0 conn(0x7f4a2415b990 legacy=0x7f4a240cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 -- 172.21.15.181:0/577286519 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240ccad0 con 0x7f4a240d0370 2023-10-16T10:03:59.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a21d9b700 1 --2- 172.21.15.181:0/577286519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a240ce8f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4a240cbc60 tx=0x7f4a14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9080b933a587ab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a2159a700 1 -- 172.21.15.181:0/577286519 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400b870 con 0x7f4a240d0370 2023-10-16T10:03:59.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a2159a700 1 -- 172.21.15.181:0/577286519 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400b9d0 con 0x7f4a240d0370 2023-10-16T10:03:59.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.827+0000 7f4a2159a700 1 -- 172.21.15.181:0/577286519 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400bd60 con 0x7f4a240d0370 2023-10-16T10:03:59.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 -- 172.21.15.181:0/577286519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 msgr2=0x7f4a240ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 --2- 172.21.15.181:0/577286519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a240ce8f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4a240cbc60 tx=0x7f4a14009bd0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 -- 172.21.15.181:0/577286519 shutdown_connections 2023-10-16T10:03:59.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 --2- 172.21.15.181:0/577286519 >> v2:172.21.15.190:3300/0 conn(0x7f4a240cd2d0 0x7f4a240d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 --2- 172.21.15.181:0/577286519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a240ce8f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 -- 172.21.15.181:0/577286519 >> 172.21.15.181:0/577286519 conn(0x7f4a2409a9a0 msgr2=0x7f4a240d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:03:59.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 -- 172.21.15.181:0/577286519 shutdown_connections 2023-10-16T10:03:59.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.828+0000 7f4a28948700 1 -- 172.21.15.181:0/577286519 wait complete. 2023-10-16T10:03:59.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.829+0000 7f4a28948700 1 Processor -- start 2023-10-16T10:03:59.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.829+0000 7f4a28948700 1 -- start start 2023-10-16T10:03:59.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a28948700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240cd2d0 0x7f4a240c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a28948700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a2406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a28948700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a2415b990 0x7f4a241573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a28948700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c4750 con 0x7f4a240d0370 2023-10-16T10:03:59.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a28948700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2406bef0 con 0x7f4a2415b990 2023-10-16T10:03:59.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a28948700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2406c9a0 con 0x7f4a240cd2d0 2023-10-16T10:03:59.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a2406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a2406b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49096/0 (socket says 172.21.15.181:49096) 2023-10-16T10:03:59.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 -- 172.21.15.181:0/4029077963 learned_addr learned my addr 172.21.15.181:0/4029077963 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:03:59.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a22d9d700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a2415b990 0x7f4a241573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a2259c700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240cd2d0 0x7f4a240c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 -- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240cd2d0 msgr2=0x7f4a240c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240cd2d0 0x7f4a240c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 -- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a2415b990 msgr2=0x7f4a241573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:03:59.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a2415b990 0x7f4a241573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:03:59.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.830+0000 7f4a21d9b700 1 -- 172.21.15.181:0/4029077963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009550 con 0x7f4a240d0370 2023-10-16T10:03:59.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.831+0000 7f4a21d9b700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a2406b9b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4a14014040 tx=0x7f4a14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.831+0000 7f4a12ffd700 1 -- 172.21.15.181:0/4029077963 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14004520 con 0x7f4a240d0370 2023-10-16T10:03:59.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.831+0000 7f4a12ffd700 1 -- 172.21.15.181:0/4029077963 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1401e070 con 0x7f4a240d0370 2023-10-16T10:03:59.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.831+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2406c100 con 0x7f4a240d0370 2023-10-16T10:03:59.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.831+0000 7f4a12ffd700 1 -- 172.21.15.181:0/4029077963 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140038a0 con 0x7f4a240d0370 2023-10-16T10:03:59.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.831+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a240b5a40 con 0x7f4a240d0370 2023-10-16T10:03:59.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.832+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2406c4f0 con 0x7f4a240d0370 2023-10-16T10:03:59.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.832+0000 7f4a12ffd700 1 -- 172.21.15.181:0/4029077963 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4a14003a00 con 0x7f4a240d0370 2023-10-16T10:03:59.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.832+0000 7f4a12ffd700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4a0406b020 0x7f4a0406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:03:59.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.833+0000 7f4a12ffd700 1 -- 172.21.15.181:0/4029077963 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5309+0+0 (secure 0 0 0) 0x7f4a14020070 con 0x7f4a240d0370 2023-10-16T10:03:59.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.833+0000 7f4a2259c700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4a0406b020 0x7f4a0406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:03:59.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.833+0000 7f4a2259c700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4a0406b020 0x7f4a0406d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a240b72a0 tx=0x7f4a18009040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:03:59.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:03:59.837+0000 7f4a12ffd700 1 -- 172.21.15.181:0/4029077963 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4a14055c90 con 0x7f4a240d0370 2023-10-16T10:04:00.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.031+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4a24067d20 con 0x7f4a240d0370 2023-10-16T10:04:00.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.032+0000 7f4a12ffd700 1 -- 172.21.15.181:0/4029077963 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4a14015020 con 0x7f4a240d0370 2023-10-16T10:04:00.034 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:00.034 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:04:00.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:04:00.034 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T10:04:00.035 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T10:04:00.035 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:04:00.035 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:04:00.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4a0406b020 msgr2=0x7f4a0406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4a0406b020 0x7f4a0406d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a240b72a0 tx=0x7f4a18009040 comp rx=0 tx=0).stop 2023-10-16T10:04:00.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 msgr2=0x7f4a2406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a2406b9b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4a14014040 tx=0x7f4a14004750 comp rx=0 tx=0).stop 2023-10-16T10:04:00.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 shutdown_connections 2023-10-16T10:04:00.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f4a0406b020 0x7f4a0406d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240cd2d0 0x7f4a240c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a2415b990 0x7f4a241573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 --2- 172.21.15.181:0/4029077963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a240d0370 0x7f4a2406b9b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 >> 172.21.15.181:0/4029077963 conn(0x7f4a2409a9a0 msgr2=0x7f4a240cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:00.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 shutdown_connections 2023-10-16T10:04:00.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.034+0000 7f4a28948700 1 -- 172.21.15.181:0/4029077963 wait complete. 2023-10-16T10:04:00.048 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T10:04:00.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T10:04:00.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.170+0000 7f7903713700 1 Processor -- start 2023-10-16T10:04:00.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.170+0000 7f7903713700 1 -- start start 2023-10-16T10:04:00.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f7903713700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f78fc15e610 0x7f78fc15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f7903713700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f78fc0cf040 0x7f78fc0cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f7903713700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc06ed90 con 0x7f78fc0665a0 2023-10-16T10:04:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f7903713700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc15f260 con 0x7f78fc15e610 2023-10-16T10:04:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f7903713700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc0ceb40 con 0x7f78fc0cf040 2023-10-16T10:04:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f79014af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f78fc15e610 0x7f78fc15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f79014af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f78fc15e610 0x7f78fc15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49104/0 (socket says 172.21.15.181:49104) 2023-10-16T10:04:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f79014af700 1 -- 172.21.15.181:0/241952633 learned_addr learned my addr 172.21.15.181:0/241952633 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.171+0000 7f7901cb0700 1 --2- 172.21.15.181:0/241952633 >> v2:172.21.15.190:3300/0 conn(0x7f78fc0cf040 0x7f78fc0cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f79014af700 1 -- 172.21.15.181:0/241952633 >> v2:172.21.15.190:3300/0 conn(0x7f78fc0cf040 msgr2=0x7f78fc0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f79014af700 1 --2- 172.21.15.181:0/241952633 >> v2:172.21.15.190:3300/0 conn(0x7f78fc0cf040 0x7f78fc0cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f79014af700 1 -- 172.21.15.181:0/241952633 >> v1:172.21.15.190:6789/0 conn(0x7f78fc0665a0 legacy=0x7f78fc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f79014af700 1 -- 172.21.15.181:0/241952633 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc06e7d0 con 0x7f78fc15e610 2023-10-16T10:04:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f79014af700 1 --2- 172.21.15.181:0/241952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc15e9f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f78fc158830 tx=0x7f78ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd01cf0f2fb75b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f78f3fff700 1 -- 172.21.15.181:0/241952633 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec011830 con 0x7f78fc15e610 2023-10-16T10:04:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f78f3fff700 1 -- 172.21.15.181:0/241952633 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec011990 con 0x7f78fc15e610 2023-10-16T10:04:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f78f3fff700 1 -- 172.21.15.181:0/241952633 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec011d20 con 0x7f78fc15e610 2023-10-16T10:04:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f7903713700 1 -- 172.21.15.181:0/241952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 msgr2=0x7f78fc15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f7903713700 1 --2- 172.21.15.181:0/241952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc15e9f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f78fc158830 tx=0x7f78ec009bd0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f7903713700 1 -- 172.21.15.181:0/241952633 shutdown_connections 2023-10-16T10:04:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f7903713700 1 --2- 172.21.15.181:0/241952633 >> v2:172.21.15.190:3300/0 conn(0x7f78fc0cf040 0x7f78fc0cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f7903713700 1 --2- 172.21.15.181:0/241952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc15e9f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.172+0000 7f7903713700 1 -- 172.21.15.181:0/241952633 >> 172.21.15.181:0/241952633 conn(0x7f78fc0b1780 msgr2=0x7f78fc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:00.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.173+0000 7f7903713700 1 -- 172.21.15.181:0/241952633 shutdown_connections 2023-10-16T10:04:00.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.173+0000 7f7903713700 1 -- 172.21.15.181:0/241952633 wait complete. 2023-10-16T10:04:00.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 Processor -- start 2023-10-16T10:04:00.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 -- start start 2023-10-16T10:04:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78fc0665a0 0x7f78fc158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78fc0cf040 0x7f78fc157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc06a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c7fe0 con 0x7f78fc15e610 2023-10-16T10:04:00.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1580e0 con 0x7f78fc0665a0 2023-10-16T10:04:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7903713700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c7a20 con 0x7f78fc0cf040 2023-10-16T10:04:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7901cb0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc06a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7901cb0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc06a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49112/0 (socket says 172.21.15.181:49112) 2023-10-16T10:04:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.174+0000 7f7901cb0700 1 -- 172.21.15.181:0/2987219432 learned_addr learned my addr 172.21.15.181:0/2987219432 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f79014af700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78fc0665a0 0x7f78fc158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7900cae700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78fc0cf040 0x7f78fc157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7901cb0700 1 -- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78fc0cf040 msgr2=0x7f78fc157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7901cb0700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78fc0cf040 0x7f78fc157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7901cb0700 1 -- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78fc0665a0 msgr2=0x7f78fc158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7901cb0700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78fc0665a0 0x7f78fc158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7901cb0700 1 -- 172.21.15.181:0/2987219432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009550 con 0x7f78fc15e610 2023-10-16T10:04:00.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f79014af700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78fc0665a0 0x7f78fc158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7900cae700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78fc0cf040 0x7f78fc157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7901cb0700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc06a9a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f78f400efc0 tx=0x7f78f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f78f1ffb700 1 -- 172.21.15.181:0/2987219432 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f400cd40 con 0x7f78fc15e610 2023-10-16T10:04:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f78f1ffb700 1 -- 172.21.15.181:0/2987219432 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f400f040 con 0x7f78fc15e610 2023-10-16T10:04:00.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0c7d00 con 0x7f78fc15e610 2023-10-16T10:04:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f78f1ffb700 1 -- 172.21.15.181:0/2987219432 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40146c0 con 0x7f78fc15e610 2023-10-16T10:04:00.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.175+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0c8aa0 con 0x7f78fc15e610 2023-10-16T10:04:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.176+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc0c8620 con 0x7f78fc15e610 2023-10-16T10:04:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.177+0000 7f78f1ffb700 1 -- 172.21.15.181:0/2987219432 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f78f4014820 con 0x7f78fc15e610 2023-10-16T10:04:00.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.177+0000 7f78f1ffb700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f78e006b020 0x7f78e006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.177+0000 7f78f1ffb700 1 -- 172.21.15.181:0/2987219432 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5309+0+0 (secure 0 0 0) 0x7f78f408a310 con 0x7f78fc15e610 2023-10-16T10:04:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.177+0000 7f79014af700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f78e006b020 0x7f78e006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.177+0000 7f79014af700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f78e006b020 0x7f78e006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78ec014040 tx=0x7f78ec008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:00.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.181+0000 7f78f1ffb700 1 -- 172.21.15.181:0/2987219432 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f78fc0c8620 con 0x7f78fc15e610 2023-10-16T10:04:00.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.372+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f78fc0c8620 con 0x7f78fc15e610 2023-10-16T10:04:00.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.545+0000 7f78f1ffb700 1 -- 172.21.15.181:0/2987219432 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) v1 ==== 138+0+0 (secure 0 0 0) 0x7f78fc0c8620 con 0x7f78fc15e610 2023-10-16T10:04:00.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f78e006b020 msgr2=0x7f78e006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f78e006b020 0x7f78e006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78ec014040 tx=0x7f78ec008030 comp rx=0 tx=0).stop 2023-10-16T10:04:00.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 msgr2=0x7f78fc06a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc06a9a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f78f400efc0 tx=0x7f78f400c3a0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 shutdown_connections 2023-10-16T10:04:00.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f78e006b020 0x7f78e006d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78fc15e610 0x7f78fc06a9a0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78fc0665a0 0x7f78fc158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 --2- 172.21.15.181:0/2987219432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78fc0cf040 0x7f78fc157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 >> 172.21.15.181:0/2987219432 conn(0x7f78fc0b1780 msgr2=0x7f78fc0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:00.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 shutdown_connections 2023-10-16T10:04:00.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.549+0000 7f7903713700 1 -- 172.21.15.181:0/2987219432 wait complete. 2023-10-16T10:04:00.554 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T10:04:00.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T10:04:00.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.691+0000 7f97a415d700 1 Processor -- start 2023-10-16T10:04:00.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.692+0000 7f97a415d700 1 -- start start 2023-10-16T10:04:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.692+0000 7f97a415d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 0x7f979c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.692+0000 7f97a415d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c15dfb0 con 0x7f979c0d0610 2023-10-16T10:04:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.692+0000 7f97a415d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c0cd3a0 con 0x7f979c0ce490 2023-10-16T10:04:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.692+0000 7f97a415d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0cd830 con 0x7f979c15dbd0 2023-10-16T10:04:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a16f8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f979c0d0610 0x7f979c0cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60194/0 (socket says 172.21.15.181:60194) 2023-10-16T10:04:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a16f8700 1 -- 172.21.15.181:0/1027302485 learned_addr learned my addr 172.21.15.181:0/1027302485 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914518736 0 0) 0x7f9798001110 con 0x7f979c0d0610 2023-10-16T10:04:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97880035b0 con 0x7f979c0d0610 2023-10-16T10:04:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a1ef9700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 0x7f979c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4206088487 0 0) 0x7f97880035b0 con 0x7f979c0d0610 2023-10-16T10:04:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 msgr2=0x7f979c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 0x7f979c0d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 >> v1:172.21.15.190:6789/0 conn(0x7f979c0ce490 legacy=0x7f979c06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0d1fc0 con 0x7f979c0d0610 2023-10-16T10:04:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f9798001800 con 0x7f979c0d0610 2023-10-16T10:04:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 >> v1:172.21.15.181:6789/0 conn(0x7f979c0d0610 legacy=0x7f979c0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788003a30 0x7f9788003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f9788004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 0x7f9788005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0cd830 con 0x7f9788004850 2023-10-16T10:04:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978800bb40 con 0x7f9788003a30 2023-10-16T10:04:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978800bd00 con 0x7f979c15dbd0 2023-10-16T10:04:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f9788004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a1ef9700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788003a30 0x7f9788003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.693+0000 7f97a26fa700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 0x7f9788005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a16f8700 1 -- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 msgr2=0x7f9788005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 0x7f9788005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a16f8700 1 -- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788003a30 msgr2=0x7f9788003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9788003a30 0x7f9788003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a16f8700 1 -- 172.21.15.181:0/1027302485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97880035b0 con 0x7f9788004850 2023-10-16T10:04:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a26fa700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c15dbd0 0x7f9788005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f9788004cd0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f979800c210 tx=0x7f9788005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19e9989606d362a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798007db0 con 0x7f9788004850 2023-10-16T10:04:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979800cdb0 con 0x7f9788004850 2023-10-16T10:04:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a0ef7700 1 -- 172.21.15.181:0/1027302485 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798005b80 con 0x7f9788004850 2023-10-16T10:04:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a415d700 1 -- 172.21.15.181:0/1027302485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 msgr2=0x7f9788004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a415d700 1 --2- 172.21.15.181:0/1027302485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f9788004cd0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f979800c210 tx=0x7f9788005880 comp rx=0 tx=0).stop 2023-10-16T10:04:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a26fa700 1 -- 172.21.15.181:0/1027302485 reap_dead start 2023-10-16T10:04:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a415d700 1 -- 172.21.15.181:0/1027302485 shutdown_connections 2023-10-16T10:04:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a415d700 1 -- 172.21.15.181:0/1027302485 >> 172.21.15.181:0/1027302485 conn(0x7f979c09a930 msgr2=0x7f979c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a415d700 1 -- 172.21.15.181:0/1027302485 shutdown_connections 2023-10-16T10:04:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.694+0000 7f97a415d700 1 -- 172.21.15.181:0/1027302485 wait complete. 2023-10-16T10:04:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 Processor -- start 2023-10-16T10:04:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 -- start start 2023-10-16T10:04:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f979c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c06c740 0x7f979c06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c15f3f0 0x7f979c15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c09dec0 con 0x7f9788004850 2023-10-16T10:04:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c06d100 con 0x7f979c15f3f0 2023-10-16T10:04:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a415d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c09db40 con 0x7f979c06c740 2023-10-16T10:04:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a1ef9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f979c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a1ef9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f979c158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49128/0 (socket says 172.21.15.181:49128) 2023-10-16T10:04:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.696+0000 7f97a1ef9700 1 -- 172.21.15.181:0/1366983638 learned_addr learned my addr 172.21.15.181:0/1366983638 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c06c740 0x7f979c06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a26fa700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c15f3f0 0x7f979c15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a1ef9700 1 -- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c06c740 msgr2=0x7f979c06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a1ef9700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c06c740 0x7f979c06cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a1ef9700 1 -- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c15f3f0 msgr2=0x7f979c15f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a1ef9700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c15f3f0 0x7f979c15f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a1ef9700 1 -- 172.21.15.181:0/1366983638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c160160 con 0x7f9788004850 2023-10-16T10:04:00.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c06c740 0x7f979c06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a1ef9700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f979c158c60 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f978c0116c0 tx=0x7f978c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97927fc700 1 -- 172.21.15.181:0/1366983638 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c00c070 con 0x7f9788004850 2023-10-16T10:04:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97927fc700 1 -- 172.21.15.181:0/1366983638 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c015870 con 0x7f9788004850 2023-10-16T10:04:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c160440 con 0x7f9788004850 2023-10-16T10:04:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97927fc700 1 -- 172.21.15.181:0/1366983638 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c015c20 con 0x7f9788004850 2023-10-16T10:04:00.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.697+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c1f3030 con 0x7f9788004850 2023-10-16T10:04:00.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.698+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c160960 con 0x7f9788004850 2023-10-16T10:04:00.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.699+0000 7f97927fc700 1 -- 172.21.15.181:0/1366983638 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f978c013400 con 0x7f9788004850 2023-10-16T10:04:00.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.699+0000 7f97927fc700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f978006b020 0x7f978006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:00.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.699+0000 7f97927fc700 1 -- 172.21.15.181:0/1366983638 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f978c089a60 con 0x7f9788004850 2023-10-16T10:04:00.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.699+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f978006b020 0x7f978006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.699+0000 7f97a16f8700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f978006b020 0x7f978006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f979c15cf70 tx=0x7f9798007000 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.704+0000 7f97927fc700 1 -- 172.21.15.181:0/1366983638 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f978c0549a0 con 0x7f9788004850 2023-10-16T10:04:00.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:00.896+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f979c067d20 con 0x7f9788004850 2023-10-16T10:04:01.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.548+0000 7f97927fc700 1 -- 172.21.15.181:0/1366983638 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) v1 ==== 170+0+0 (secure 0 0 0) 0x7f978c0230d0 con 0x7f9788004850 2023-10-16T10:04:01.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f978006b020 msgr2=0x7f978006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:01.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f978006b020 0x7f978006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f979c15cf70 tx=0x7f9798007000 comp rx=0 tx=0).stop 2023-10-16T10:04:01.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 msgr2=0x7f979c158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:01.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f979c158c60 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f978c0116c0 tx=0x7f978c011a80 comp rx=0 tx=0).stop 2023-10-16T10:04:01.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 shutdown_connections 2023-10-16T10:04:01.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f978006b020 0x7f978006d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f979c15f3f0 0x7f979c15f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9788004850 0x7f979c158c60 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 --2- 172.21.15.181:0/1366983638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f979c06c740 0x7f979c06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.552+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 >> 172.21.15.181:0/1366983638 conn(0x7f979c09a930 msgr2=0x7f979c15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:01.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.553+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 shutdown_connections 2023-10-16T10:04:01.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.553+0000 7f97a415d700 1 -- 172.21.15.181:0/1366983638 wait complete. 2023-10-16T10:04:01.557 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T10:04:01.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T10:04:01.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.705+0000 7f40d0c12700 1 Processor -- start 2023-10-16T10:04:01.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.705+0000 7f40d0c12700 1 -- start start 2023-10-16T10:04:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40d0c12700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40cc15f150 0x7f40cc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40d0c12700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f40cc0ce4f0 0x7f40cc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:01.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40d0c12700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0b2ae0 con 0x7f40cc0b14a0 2023-10-16T10:04:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40d0c12700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc0ce9a0 con 0x7f40cc15f150 2023-10-16T10:04:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40d0c12700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc0b2dc0 con 0x7f40cc0ce4f0 2023-10-16T10:04:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40ca59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40cc15f150 0x7f40cc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:01.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40ca59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40cc15f150 0x7f40cc0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49136/0 (socket says 172.21.15.181:49136) 2023-10-16T10:04:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.706+0000 7f40ca59c700 1 -- 172.21.15.181:0/948530718 learned_addr learned my addr 172.21.15.181:0/948530718 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40c9d9b700 1 --2- 172.21.15.181:0/948530718 >> v2:172.21.15.190:3300/0 conn(0x7f40cc0ce4f0 0x7f40cc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40ca59c700 1 -- 172.21.15.181:0/948530718 >> v2:172.21.15.190:3300/0 conn(0x7f40cc0ce4f0 msgr2=0x7f40cc0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40ca59c700 1 --2- 172.21.15.181:0/948530718 >> v2:172.21.15.190:3300/0 conn(0x7f40cc0ce4f0 0x7f40cc0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40ca59c700 1 -- 172.21.15.181:0/948530718 >> v1:172.21.15.190:6789/0 conn(0x7f40cc0b14a0 legacy=0x7f40cc0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40ca59c700 1 -- 172.21.15.181:0/948530718 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0cf9d0 con 0x7f40cc15f150 2023-10-16T10:04:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40c9d9b700 1 --2- 172.21.15.181:0/948530718 >> v2:172.21.15.190:3300/0 conn(0x7f40cc0ce4f0 0x7f40cc0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40ca59c700 1 --2- 172.21.15.181:0/948530718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0cdfb0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f40c00098f0 tx=0x7f40c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5042c2b55d240e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40c959a700 1 -- 172.21.15.181:0/948530718 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0004230 con 0x7f40cc15f150 2023-10-16T10:04:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40c959a700 1 -- 172.21.15.181:0/948530718 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c0004390 con 0x7f40cc15f150 2023-10-16T10:04:01.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.707+0000 7f40c959a700 1 -- 172.21.15.181:0/948530718 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c00036a0 con 0x7f40cc15f150 2023-10-16T10:04:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 -- 172.21.15.181:0/948530718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 msgr2=0x7f40cc0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 --2- 172.21.15.181:0/948530718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0cdfb0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f40c00098f0 tx=0x7f40c0009c00 comp rx=0 tx=0).stop 2023-10-16T10:04:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 -- 172.21.15.181:0/948530718 shutdown_connections 2023-10-16T10:04:01.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 --2- 172.21.15.181:0/948530718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0cdfb0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 --2- 172.21.15.181:0/948530718 >> v2:172.21.15.190:3300/0 conn(0x7f40cc0ce4f0 0x7f40cc0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 -- 172.21.15.181:0/948530718 >> 172.21.15.181:0/948530718 conn(0x7f40cc09a930 msgr2=0x7f40cc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 -- 172.21.15.181:0/948530718 shutdown_connections 2023-10-16T10:04:01.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.708+0000 7f40d0c12700 1 -- 172.21.15.181:0/948530718 wait complete. 2023-10-16T10:04:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.709+0000 7f40d0c12700 1 Processor -- start 2023-10-16T10:04:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.709+0000 7f40d0c12700 1 -- start start 2023-10-16T10:04:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40d0c12700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40cc0b14a0 0x7f40cc0b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40d0c12700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40cc0ce4f0 0x7f40cc15aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:01.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40d0c12700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40d0c12700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0b4900 con 0x7f40cc15f150 2023-10-16T10:04:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40d0c12700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0b5d20 con 0x7f40cc0b14a0 2023-10-16T10:04:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40d0c12700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0b4620 con 0x7f40cc0ce4f0 2023-10-16T10:04:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:01.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49146/0 (socket says 172.21.15.181:49146) 2023-10-16T10:04:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 -- 172.21.15.181:0/446250741 learned_addr learned my addr 172.21.15.181:0/446250741 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40ca59c700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40cc0b14a0 0x7f40cc0b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 -- 172.21.15.181:0/446250741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40cc0ce4f0 msgr2=0x7f40cc15aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40cc0ce4f0 0x7f40cc15aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 -- 172.21.15.181:0/446250741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40cc0b14a0 msgr2=0x7f40cc0b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40cc0b14a0 0x7f40cc0b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 -- 172.21.15.181:0/446250741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0009550 con 0x7f40cc15f150 2023-10-16T10:04:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40cad9d700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0b3ab0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f40c400b970 tx=0x7f40c400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40baffd700 1 -- 172.21.15.181:0/446250741 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c400c9b0 con 0x7f40cc15f150 2023-10-16T10:04:01.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40baffd700 1 -- 172.21.15.181:0/446250741 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c400cb10 con 0x7f40cc15f150 2023-10-16T10:04:01.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc0b5fc0 con 0x7f40cc15f150 2023-10-16T10:04:01.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.710+0000 7f40baffd700 1 -- 172.21.15.181:0/446250741 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4014440 con 0x7f40cc15f150 2023-10-16T10:04:01.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.711+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc0b6580 con 0x7f40cc15f150 2023-10-16T10:04:01.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.711+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc0b6770 con 0x7f40cc15f150 2023-10-16T10:04:01.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.712+0000 7f40baffd700 1 -- 172.21.15.181:0/446250741 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f40c4003a00 con 0x7f40cc15f150 2023-10-16T10:04:01.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.713+0000 7f40baffd700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f40ac06b020 0x7f40ac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:01.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.713+0000 7f40baffd700 1 -- 172.21.15.181:0/446250741 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6061+0+0 (secure 0 0 0) 0x7f40c40896f0 con 0x7f40cc15f150 2023-10-16T10:04:01.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.713+0000 7f40ca59c700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f40ac06b020 0x7f40ac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:01.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.713+0000 7f40ca59c700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f40ac06b020 0x7f40ac06d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009840 tx=0x7f40c0008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:01.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.717+0000 7f40baffd700 1 -- 172.21.15.181:0/446250741 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f40c40545a0 con 0x7f40cc15f150 2023-10-16T10:04:01.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:01.952+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f40cc067d20 con 0x7f40cc15f150 2023-10-16T10:04:02.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.579+0000 7f40baffd700 1 -- 172.21.15.181:0/446250741 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v72) v1 ==== 174+0+0 (secure 0 0 0) 0x7f40c4003cb0 con 0x7f40cc15f150 2023-10-16T10:04:02.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.586+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f40ac06b020 msgr2=0x7f40ac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.586+0000 7f40d0c12700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f40ac06b020 0x7f40ac06d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009840 tx=0x7f40c0008030 comp rx=0 tx=0).stop 2023-10-16T10:04:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.586+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 msgr2=0x7f40cc0b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.586+0000 7f40d0c12700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0b3ab0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f40c400b970 tx=0x7f40c400bd30 comp rx=0 tx=0).stop 2023-10-16T10:04:02.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 shutdown_connections 2023-10-16T10:04:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f40ac06b020 0x7f40ac06d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40cc15f150 0x7f40cc0b3ab0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40cc0b14a0 0x7f40cc0b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 --2- 172.21.15.181:0/446250741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40cc0ce4f0 0x7f40cc15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 >> 172.21.15.181:0/446250741 conn(0x7f40cc09a930 msgr2=0x7f40cc0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 shutdown_connections 2023-10-16T10:04:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.587+0000 7f40d0c12700 1 -- 172.21.15.181:0/446250741 wait complete. 2023-10-16T10:04:02.592 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 18 and data pool 19 2023-10-16T10:04:02.607 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:02.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.726+0000 7f1dd9f50700 1 Processor -- start 2023-10-16T10:04:02.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.726+0000 7f1dd9f50700 1 -- start start 2023-10-16T10:04:02.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd9f50700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd415dfd0 0x7f1dd40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd9f50700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1dd40af450 0x7f1dd406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd9f50700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dd406db80 0x7f1dd40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd9f50700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd406f870 con 0x7f1dd406db80 2023-10-16T10:04:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd9f50700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd40d1160 con 0x7f1dd40af450 2023-10-16T10:04:02.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd9f50700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40cc8e0 con 0x7f1dd415dfd0 2023-10-16T10:04:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dd406db80 0x7f1dd40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dd406db80 0x7f1dd40cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49154/0 (socket says 172.21.15.181:49154) 2023-10-16T10:04:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 -- 172.21.15.181:0/2803489747 learned_addr learned my addr 172.21.15.181:0/2803489747 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd37fe700 1 --2- 172.21.15.181:0/2803489747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd415dfd0 0x7f1dd40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd2ffd700 1 --2- 172.21.15.181:0/2803489747 >> v2:172.21.15.190:3300/0 conn(0x7f1dd40af450 0x7f1dd406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 -- 172.21.15.181:0/2803489747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd415dfd0 msgr2=0x7f1dd40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 --2- 172.21.15.181:0/2803489747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd415dfd0 0x7f1dd40af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 -- 172.21.15.181:0/2803489747 >> v2:172.21.15.190:3300/0 conn(0x7f1dd40af450 msgr2=0x7f1dd406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 --2- 172.21.15.181:0/2803489747 >> v2:172.21.15.190:3300/0 conn(0x7f1dd40af450 0x7f1dd406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.727+0000 7f1dd3fff700 1 -- 172.21.15.181:0/2803489747 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40ccbc0 con 0x7f1dd406db80 2023-10-16T10:04:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd3fff700 1 --2- 172.21.15.181:0/2803489747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40cd470 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40705e0 tx=0x7f1dcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fce62546e1c799 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd27fc700 1 -- 172.21.15.181:0/2803489747 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00e070 con 0x7f1dd406db80 2023-10-16T10:04:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd27fc700 1 -- 172.21.15.181:0/2803489747 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc00b9c0 con 0x7f1dd406db80 2023-10-16T10:04:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd27fc700 1 -- 172.21.15.181:0/2803489747 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00bd70 con 0x7f1dd406db80 2023-10-16T10:04:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 -- 172.21.15.181:0/2803489747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 msgr2=0x7f1dd40cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/2803489747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40cd470 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40705e0 tx=0x7f1dcc00b330 comp rx=0 tx=0).stop 2023-10-16T10:04:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 -- 172.21.15.181:0/2803489747 shutdown_connections 2023-10-16T10:04:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/2803489747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40cd470 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/2803489747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd415dfd0 0x7f1dd40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/2803489747 >> v2:172.21.15.190:3300/0 conn(0x7f1dd40af450 0x7f1dd406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 -- 172.21.15.181:0/2803489747 >> 172.21.15.181:0/2803489747 conn(0x7f1dd40d04a0 msgr2=0x7f1dd40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:02.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 -- 172.21.15.181:0/2803489747 shutdown_connections 2023-10-16T10:04:02.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.728+0000 7f1dd9f50700 1 -- 172.21.15.181:0/2803489747 wait complete. 2023-10-16T10:04:02.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.730+0000 7f1dd9f50700 1 Processor -- start 2023-10-16T10:04:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.730+0000 7f1dd9f50700 1 -- start start 2023-10-16T10:04:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.730+0000 7f1dd9f50700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.730+0000 7f1dd9f50700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd40af450 0x7f1dd40ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.730+0000 7f1dd9f50700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dd415dfd0 0x7f1dd40adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.730+0000 7f1dd9f50700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40c81a0 con 0x7f1dd406db80 2023-10-16T10:04:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd9f50700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40c8480 con 0x7f1dd415dfd0 2023-10-16T10:04:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd9f50700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40c8760 con 0x7f1dd40af450 2023-10-16T10:04:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49156/0 (socket says 172.21.15.181:49156) 2023-10-16T10:04:02.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 -- 172.21.15.181:0/3285619699 learned_addr learned my addr 172.21.15.181:0/3285619699 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd2ffd700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd40af450 0x7f1dd40ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd3fff700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dd415dfd0 0x7f1dd40adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 -- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd40af450 msgr2=0x7f1dd40ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd40af450 0x7f1dd40ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 -- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dd415dfd0 msgr2=0x7f1dd40adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dd415dfd0 0x7f1dd40adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 -- 172.21.15.181:0/3285619699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc009b30 con 0x7f1dd406db80 2023-10-16T10:04:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd3fff700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dd415dfd0 0x7f1dd40adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd37fe700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40b2570 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00d7d0 tx=0x7f1dbc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:02.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dcbfff700 1 -- 172.21.15.181:0/3285619699 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0125f0 con 0x7f1dd406db80 2023-10-16T10:04:02.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dcbfff700 1 -- 172.21.15.181:0/3285619699 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc012750 con 0x7f1dd406db80 2023-10-16T10:04:02.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd40c8a40 con 0x7f1dd406db80 2023-10-16T10:04:02.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dcbfff700 1 -- 172.21.15.181:0/3285619699 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc012b00 con 0x7f1dd406db80 2023-10-16T10:04:02.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.731+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd40c9000 con 0x7f1dd406db80 2023-10-16T10:04:02.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.732+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd40c91c0 con 0x7f1dd406db80 2023-10-16T10:04:02.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.733+0000 7f1dcbfff700 1 -- 172.21.15.181:0/3285619699 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1dbc012c60 con 0x7f1dd406db80 2023-10-16T10:04:02.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.733+0000 7f1dcbfff700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f1db406b020 0x7f1db406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:02.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.733+0000 7f1dcbfff700 1 -- 172.21.15.181:0/3285619699 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f1dbc0898c0 con 0x7f1dd406db80 2023-10-16T10:04:02.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.733+0000 7f1dd2ffd700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f1db406b020 0x7f1db406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:02.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.734+0000 7f1dd2ffd700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f1db406b020 0x7f1db406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40ae2b0 tx=0x7f1dc4008040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:02.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.739+0000 7f1dcbfff700 1 -- 172.21.15.181:0/3285619699 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1dbc054690 con 0x7f1dd406db80 2023-10-16T10:04:02.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.932+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dd4067d20 con 0x7f1dd406db80 2023-10-16T10:04:02.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.934+0000 7f1dcbfff700 1 -- 172.21.15.181:0/3285619699 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17317 (secure 0 0 0) 0x7f1dbc057cb0 con 0x7f1dd406db80 2023-10-16T10:04:02.936 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:02.936 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:04:01.954095+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:00.393030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:00.916899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f1db406b020 msgr2=0x7f1db406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f1db406b020 0x7f1db406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40ae2b0 tx=0x7f1dc4008040 comp rx=0 tx=0).stop 2023-10-16T10:04:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 msgr2=0x7f1dd40b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40b2570 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00d7d0 tx=0x7f1dbc00dae0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 shutdown_connections 2023-10-16T10:04:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f1db406b020 0x7f1db406d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dd415dfd0 0x7f1dd40adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dd406db80 0x7f1dd40b2570 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 --2- 172.21.15.181:0/3285619699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dd40af450 0x7f1dd40ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:02.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 >> 172.21.15.181:0/3285619699 conn(0x7f1dd40d04a0 msgr2=0x7f1dd40aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:02.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 shutdown_connections 2023-10-16T10:04:02.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:02.936+0000 7f1dd9f50700 1 -- 172.21.15.181:0/3285619699 wait complete. 2023-10-16T10:04:02.950 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:03.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.070+0000 7f54b866a700 1 Processor -- start 2023-10-16T10:04:03.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.070+0000 7f54b866a700 1 -- start start 2023-10-16T10:04:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b866a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f54b0159730 0x7f54b015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b866a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b866a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b00d0310 con 0x7f54b00cc860 2023-10-16T10:04:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b866a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b00cee70 con 0x7f54b0159730 2023-10-16T10:04:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b866a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0159b10 con 0x7f54b00cc400 2023-10-16T10:04:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f54b0159730 0x7f54b015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f54b0159730 0x7f54b015cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49164/0 (socket says 172.21.15.181:49164) 2023-10-16T10:04:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 -- 172.21.15.181:0/3736688953 learned_addr learned my addr 172.21.15.181:0/3736688953 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b5c05700 1 --2- 172.21.15.181:0/3736688953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 -- 172.21.15.181:0/3736688953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 msgr2=0x7f54b00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 --2- 172.21.15.181:0/3736688953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 -- 172.21.15.181:0/3736688953 >> v1:172.21.15.190:6789/0 conn(0x7f54b00cc860 legacy=0x7f54b006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 -- 172.21.15.181:0/3736688953 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00ce540 con 0x7f54b0159730 2023-10-16T10:04:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b6406700 1 --2- 172.21.15.181:0/3736688953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b015cd20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006fd0 tx=0x7f54ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1a8b057ed37360 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.071+0000 7f54b5404700 1 -- 172.21.15.181:0/3736688953 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac005e30 con 0x7f54b0159730 2023-10-16T10:04:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b5404700 1 -- 172.21.15.181:0/3736688953 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac004030 con 0x7f54b0159730 2023-10-16T10:04:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b5404700 1 -- 172.21.15.181:0/3736688953 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0043e0 con 0x7f54b0159730 2023-10-16T10:04:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 -- 172.21.15.181:0/3736688953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 msgr2=0x7f54b015cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 --2- 172.21.15.181:0/3736688953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b015cd20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f54ac006fd0 tx=0x7f54ac009bd0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 -- 172.21.15.181:0/3736688953 shutdown_connections 2023-10-16T10:04:03.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 --2- 172.21.15.181:0/3736688953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b015cd20 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 --2- 172.21.15.181:0/3736688953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 -- 172.21.15.181:0/3736688953 >> 172.21.15.181:0/3736688953 conn(0x7f54b00d04a0 msgr2=0x7f54b00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:03.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 -- 172.21.15.181:0/3736688953 shutdown_connections 2023-10-16T10:04:03.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.072+0000 7f54b866a700 1 -- 172.21.15.181:0/3736688953 wait complete. 2023-10-16T10:04:03.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.073+0000 7f54b866a700 1 Processor -- start 2023-10-16T10:04:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.073+0000 7f54b866a700 1 -- start start 2023-10-16T10:04:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b866a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b866a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b00cc860 0x7f54b006d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b866a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b866a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0151210 con 0x7f54b0159730 2023-10-16T10:04:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b866a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00b55b0 con 0x7f54b00cc860 2023-10-16T10:04:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b866a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01f3990 con 0x7f54b00cc400 2023-10-16T10:04:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b006db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49172/0 (socket says 172.21.15.181:49172) 2023-10-16T10:04:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 -- 172.21.15.181:0/492512060 learned_addr learned my addr 172.21.15.181:0/492512060 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6406700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b5c05700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b00cc860 0x7f54b006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 -- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 msgr2=0x7f54b00b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 -- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b00cc860 msgr2=0x7f54b006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b00cc860 0x7f54b006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.074+0000 7f54b6c07700 1 -- 172.21.15.181:0/492512060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac009500 con 0x7f54b0159730 2023-10-16T10:04:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.075+0000 7f54b6c07700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b006db80 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f54a800cb90 tx=0x7f54a800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.075+0000 7f54a6ffd700 1 -- 172.21.15.181:0/492512060 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8003c00 con 0x7f54b0159730 2023-10-16T10:04:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.075+0000 7f54a6ffd700 1 -- 172.21.15.181:0/492512060 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8003d60 con 0x7f54b0159730 2023-10-16T10:04:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.075+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b01f3c70 con 0x7f54b0159730 2023-10-16T10:04:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.075+0000 7f54a6ffd700 1 -- 172.21.15.181:0/492512060 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800aae0 con 0x7f54b0159730 2023-10-16T10:04:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.075+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b00b5160 con 0x7f54b0159730 2023-10-16T10:04:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.076+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0067d20 con 0x7f54b0159730 2023-10-16T10:04:03.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.076+0000 7f54a6ffd700 1 -- 172.21.15.181:0/492512060 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f54a800ac40 con 0x7f54b0159730 2023-10-16T10:04:03.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.077+0000 7f54a6ffd700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f549406b020 0x7f549406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.077+0000 7f54a6ffd700 1 -- 172.21.15.181:0/492512060 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f54a8089890 con 0x7f54b0159730 2023-10-16T10:04:03.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.077+0000 7f54b6406700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f549406b020 0x7f549406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.077+0000 7f54b6406700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f549406b020 0x7f549406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54ac000c00 tx=0x7f54ac008070 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.081+0000 7f54a6ffd700 1 -- 172.21.15.181:0/492512060 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f54a8054660 con 0x7f54b0159730 2023-10-16T10:04:03.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.277+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54b00c3920 con 0x7f54b0159730 2023-10-16T10:04:03.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.278+0000 7f54a6ffd700 1 -- 172.21.15.181:0/492512060 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+17317 (secure 0 0 0) 0x7f54a8057c80 con 0x7f54b0159730 2023-10-16T10:04:03.279 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:03.279 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:04:01.954095+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:00.393030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:00.916899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f549406b020 msgr2=0x7f549406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f549406b020 0x7f549406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54ac000c00 tx=0x7f54ac008070 comp rx=0 tx=0).stop 2023-10-16T10:04:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 msgr2=0x7f54b006db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b006db80 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f54a800cb90 tx=0x7f54a800cea0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 shutdown_connections 2023-10-16T10:04:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f549406b020 0x7f549406d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54b00cc400 0x7f54b00b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54b0159730 0x7f54b006db80 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 --2- 172.21.15.181:0/492512060 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b00cc860 0x7f54b006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 >> 172.21.15.181:0/492512060 conn(0x7f54b00d04a0 msgr2=0x7f54b00aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 shutdown_connections 2023-10-16T10:04:03.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.280+0000 7f54b866a700 1 -- 172.21.15.181:0/492512060 wait complete. 2023-10-16T10:04:03.293 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T10:04:03.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.416+0000 7feb6f330700 1 Processor -- start 2023-10-16T10:04:03.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.416+0000 7feb6f330700 1 -- start start 2023-10-16T10:04:03.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6f330700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb6815f150 0x7feb680cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6f330700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb680ce4f0 0x7feb680b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6f330700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb680b2ae0 con 0x7feb680b14a0 2023-10-16T10:04:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6f330700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb680ce970 con 0x7feb680ce4f0 2023-10-16T10:04:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6f330700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb680b2dc0 con 0x7feb6815f150 2023-10-16T10:04:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb6815f150 0x7feb680cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb6815f150 0x7feb680cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:46204/0 (socket says 172.21.15.181:46204) 2023-10-16T10:04:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6c8cb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb680ce4f0 0x7feb680b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 -- 172.21.15.181:0/2236424719 learned_addr learned my addr 172.21.15.181:0/2236424719 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:03.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 -- 172.21.15.181:0/2236424719 >> v2:172.21.15.190:3300/0 conn(0x7feb680ce4f0 msgr2=0x7feb680b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:03.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 --2- 172.21.15.181:0/2236424719 >> v2:172.21.15.190:3300/0 conn(0x7feb680ce4f0 0x7feb680b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 -- 172.21.15.181:0/2236424719 >> v1:172.21.15.190:6789/0 conn(0x7feb680b14a0 legacy=0x7feb680b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 -- 172.21.15.181:0/2236424719 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb680cf9d0 con 0x7feb6815f150 2023-10-16T10:04:03.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.417+0000 7feb6d0cc700 1 --2- 172.21.15.181:0/2236424719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb6815f150 0x7feb680cdfb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feb580098f0 tx=0x7feb58009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805bcd9494beb9a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb5ffff700 1 -- 172.21.15.181:0/2236424719 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58011830 con 0x7feb6815f150 2023-10-16T10:04:03.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb5ffff700 1 -- 172.21.15.181:0/2236424719 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb58011990 con 0x7feb6815f150 2023-10-16T10:04:03.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb5ffff700 1 -- 172.21.15.181:0/2236424719 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58011d20 con 0x7feb6815f150 2023-10-16T10:04:03.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 -- 172.21.15.181:0/2236424719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb6815f150 msgr2=0x7feb680cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 --2- 172.21.15.181:0/2236424719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb6815f150 0x7feb680cdfb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feb580098f0 tx=0x7feb58009c00 comp rx=0 tx=0).stop 2023-10-16T10:04:03.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 -- 172.21.15.181:0/2236424719 shutdown_connections 2023-10-16T10:04:03.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 --2- 172.21.15.181:0/2236424719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb6815f150 0x7feb680cdfb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 --2- 172.21.15.181:0/2236424719 >> v2:172.21.15.190:3300/0 conn(0x7feb680ce4f0 0x7feb680b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 -- 172.21.15.181:0/2236424719 >> 172.21.15.181:0/2236424719 conn(0x7feb6809a930 msgr2=0x7feb680d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:03.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 -- 172.21.15.181:0/2236424719 shutdown_connections 2023-10-16T10:04:03.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.418+0000 7feb6f330700 1 -- 172.21.15.181:0/2236424719 wait complete. 2023-10-16T10:04:03.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.420+0000 7feb6f330700 1 Processor -- start 2023-10-16T10:04:03.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.420+0000 7feb6f330700 1 -- start start 2023-10-16T10:04:03.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6f330700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb680b14a0 0x7feb680c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6f330700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb680ce4f0 0x7feb68158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6f330700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb6815f150 0x7feb680c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6f330700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb680c90a0 con 0x7feb6815f150 2023-10-16T10:04:03.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6f330700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb680c9380 con 0x7feb680b14a0 2023-10-16T10:04:03.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6f330700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb680c9660 con 0x7feb680ce4f0 2023-10-16T10:04:03.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb6815f150 0x7feb680c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb6815f150 0x7feb680c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49188/0 (socket says 172.21.15.181:49188) 2023-10-16T10:04:03.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 -- 172.21.15.181:0/1785202551 learned_addr learned my addr 172.21.15.181:0/1785202551 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:03.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6c8cb700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb680ce4f0 0x7feb68158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d0cc700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb680b14a0 0x7feb680c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 -- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb680ce4f0 msgr2=0x7feb68158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb680ce4f0 0x7feb68158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 -- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb680b14a0 msgr2=0x7feb680c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:03.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb680b14a0 0x7feb680c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:03.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 -- 172.21.15.181:0/1785202551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58009550 con 0x7feb6815f150 2023-10-16T10:04:03.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb6d8cd700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb6815f150 0x7feb680c5130 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7feb6000cb50 tx=0x7feb6000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.421+0000 7feb5dffb700 1 -- 172.21.15.181:0/1785202551 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60003c00 con 0x7feb6815f150 2023-10-16T10:04:03.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.422+0000 7feb5dffb700 1 -- 172.21.15.181:0/1785202551 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb60003d60 con 0x7feb6815f150 2023-10-16T10:04:03.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.422+0000 7feb5dffb700 1 -- 172.21.15.181:0/1785202551 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6000aa20 con 0x7feb6815f150 2023-10-16T10:04:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.422+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb680c9940 con 0x7feb6815f150 2023-10-16T10:04:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.422+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb680c8d30 con 0x7feb6815f150 2023-10-16T10:04:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.423+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb680c9e60 con 0x7feb6815f150 2023-10-16T10:04:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.423+0000 7feb5dffb700 1 -- 172.21.15.181:0/1785202551 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb6000ab80 con 0x7feb6815f150 2023-10-16T10:04:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.423+0000 7feb5dffb700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7feb4c06b020 0x7feb4c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.424+0000 7feb5dffb700 1 -- 172.21.15.181:0/1785202551 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6161+0+0 (secure 0 0 0) 0x7feb60089960 con 0x7feb6815f150 2023-10-16T10:04:03.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.424+0000 7feb6d0cc700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7feb4c06b020 0x7feb4c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:03.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.424+0000 7feb6d0cc700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7feb4c06b020 0x7feb4c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb58000c00 tx=0x7feb5801d040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:03.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.428+0000 7feb5dffb700 1 -- 172.21.15.181:0/1785202551 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feb600546a0 con 0x7feb6815f150 2023-10-16T10:04:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:03.670+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feb68067d20 con 0x7feb6815f150 2023-10-16T10:04:04.553 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-10-16T10:04:04.552+0000 7f5008b10700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T10:04:04.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.593+0000 7feb5dffb700 1 -- 172.21.15.181:0/1785202551 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7feb60057cc0 con 0x7feb6815f150 2023-10-16T10:04:04.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7feb4c06b020 msgr2=0x7feb4c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7feb4c06b020 0x7feb4c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb58000c00 tx=0x7feb5801d040 comp rx=0 tx=0).stop 2023-10-16T10:04:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb6815f150 msgr2=0x7feb680c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb6815f150 0x7feb680c5130 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7feb6000cb50 tx=0x7feb6000ce60 comp rx=0 tx=0).stop 2023-10-16T10:04:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 shutdown_connections 2023-10-16T10:04:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7feb4c06b020 0x7feb4c06d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb6815f150 0x7feb680c5130 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb680b14a0 0x7feb680c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 --2- 172.21.15.181:0/1785202551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb680ce4f0 0x7feb68158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.596+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 >> 172.21.15.181:0/1785202551 conn(0x7feb6809a930 msgr2=0x7feb6815ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:04.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.597+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 shutdown_connections 2023-10-16T10:04:04.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.597+0000 7feb6f330700 1 -- 172.21.15.181:0/1785202551 wait complete. 2023-10-16T10:04:04.612 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T10:04:04.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T10:04:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.742+0000 7f88e702e700 1 Processor -- start 2023-10-16T10:04:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.742+0000 7f88e702e700 1 -- start start 2023-10-16T10:04:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88e702e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88e0072760 0x7f88e006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88e702e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88e702e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e006b210 con 0x7f88e015da70 2023-10-16T10:04:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88e702e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e00cef00 con 0x7f88e0072760 2023-10-16T10:04:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88e702e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00cec20 con 0x7f88e006ade0 2023-10-16T10:04:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88e0072760 0x7f88e006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88e0072760 0x7f88e006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49194/0 (socket says 172.21.15.181:49194) 2023-10-16T10:04:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 -- 172.21.15.181:0/379409746 learned_addr learned my addr 172.21.15.181:0/379409746 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 -- 172.21.15.181:0/379409746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 msgr2=0x7f88e01615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88e55cb700 1 --2- 172.21.15.181:0/379409746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 --2- 172.21.15.181:0/379409746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e01615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 -- 172.21.15.181:0/379409746 >> v1:172.21.15.181:6789/0 conn(0x7f88e015da70 legacy=0x7f88e00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 -- 172.21.15.181:0/379409746 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e0071c80 con 0x7f88e0072760 2023-10-16T10:04:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88e55cb700 1 --2- 172.21.15.181:0/379409746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e01615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88dffff700 1 --2- 172.21.15.181:0/379409746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e006a8a0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0098f0 tx=0x7f88cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21760d6094f8e504 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88df7fe700 1 -- 172.21.15.181:0/379409746 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00b8a0 con 0x7f88e0072760 2023-10-16T10:04:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88df7fe700 1 -- 172.21.15.181:0/379409746 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc00ba00 con 0x7f88e0072760 2023-10-16T10:04:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.743+0000 7f88df7fe700 1 -- 172.21.15.181:0/379409746 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00bd90 con 0x7f88e0072760 2023-10-16T10:04:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 -- 172.21.15.181:0/379409746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 msgr2=0x7f88e006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 --2- 172.21.15.181:0/379409746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e006a8a0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0098f0 tx=0x7f88cc009c00 comp rx=0 tx=0).stop 2023-10-16T10:04:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 -- 172.21.15.181:0/379409746 shutdown_connections 2023-10-16T10:04:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 --2- 172.21.15.181:0/379409746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e01615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 --2- 172.21.15.181:0/379409746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e006a8a0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 -- 172.21.15.181:0/379409746 >> 172.21.15.181:0/379409746 conn(0x7f88e00cd010 msgr2=0x7f88e00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 -- 172.21.15.181:0/379409746 shutdown_connections 2023-10-16T10:04:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.744+0000 7f88e702e700 1 -- 172.21.15.181:0/379409746 wait complete. 2023-10-16T10:04:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.745+0000 7f88e702e700 1 Processor -- start 2023-10-16T10:04:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.745+0000 7f88e702e700 1 -- start start 2023-10-16T10:04:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e702e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e0157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e702e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e01602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e702e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e015da70 0x7f88e0160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e702e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0158270 con 0x7f88e0072760 2023-10-16T10:04:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e702e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e01f3450 con 0x7f88e015da70 2023-10-16T10:04:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e702e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e01f3710 con 0x7f88e006ade0 2023-10-16T10:04:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e01602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e01602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49200/0 (socket says 172.21.15.181:49200) 2023-10-16T10:04:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 -- 172.21.15.181:0/3847842329 learned_addr learned my addr 172.21.15.181:0/3847842329 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e4dca700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e0157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e55cb700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e015da70 0x7f88e0160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 -- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 msgr2=0x7f88e0157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e0157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 -- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e015da70 msgr2=0x7f88e0160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e015da70 0x7f88e0160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88dffff700 1 -- 172.21.15.181:0/3847842329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc009550 con 0x7f88e0072760 2023-10-16T10:04:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.746+0000 7f88e4dca700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:04.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.747+0000 7f88dffff700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e01602c0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f88cc000c00 tx=0x7f88cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:04.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.747+0000 7f88dd7fa700 1 -- 172.21.15.181:0/3847842329 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0038a0 con 0x7f88e0072760 2023-10-16T10:04:04.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.747+0000 7f88dd7fa700 1 -- 172.21.15.181:0/3847842329 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc003a00 con 0x7f88e0072760 2023-10-16T10:04:04.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.747+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e01f39f0 con 0x7f88e0072760 2023-10-16T10:04:04.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.747+0000 7f88dd7fa700 1 -- 172.21.15.181:0/3847842329 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc003db0 con 0x7f88e0072760 2023-10-16T10:04:04.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.747+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e01f3fb0 con 0x7f88e0072760 2023-10-16T10:04:04.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.748+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e00ae0a0 con 0x7f88e0072760 2023-10-16T10:04:04.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.748+0000 7f88dd7fa700 1 -- 172.21.15.181:0/3847842329 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f88cc021070 con 0x7f88e0072760 2023-10-16T10:04:04.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.749+0000 7f88dd7fa700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f88c406b020 0x7f88c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:04.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.749+0000 7f88dd7fa700 1 -- 172.21.15.181:0/3847842329 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f88cc08b070 con 0x7f88e0072760 2023-10-16T10:04:04.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.749+0000 7f88e4dca700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f88c406b020 0x7f88c406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.749+0000 7f88e4dca700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f88c406b020 0x7f88c406d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88e00b3030 tx=0x7f88d4009040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.753+0000 7f88dd7fa700 1 -- 172.21.15.181:0/3847842329 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88cc055ff0 con 0x7f88e0072760 2023-10-16T10:04:04.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:04.992+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f88e0072210 con 0x7f88e0072760 2023-10-16T10:04:05.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.595+0000 7f88dd7fa700 1 -- 172.21.15.181:0/3847842329 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88cc059610 con 0x7f88e0072760 2023-10-16T10:04:05.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.597+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f88c406b020 msgr2=0x7f88c406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.597+0000 7f88e702e700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f88c406b020 0x7f88c406d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88e00b3030 tx=0x7f88d4009040 comp rx=0 tx=0).stop 2023-10-16T10:04:05.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.597+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 msgr2=0x7f88e01602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.597+0000 7f88e702e700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e01602c0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f88cc000c00 tx=0x7f88cc004750 comp rx=0 tx=0).stop 2023-10-16T10:04:05.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 shutdown_connections 2023-10-16T10:04:05.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f88c406b020 0x7f88c406d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88e006ade0 0x7f88e0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e015da70 0x7f88e0160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 --2- 172.21.15.181:0/3847842329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e0072760 0x7f88e01602c0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 >> 172.21.15.181:0/3847842329 conn(0x7f88e00cd010 msgr2=0x7f88e00cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:05.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 shutdown_connections 2023-10-16T10:04:05.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.598+0000 7f88e702e700 1 -- 172.21.15.181:0/3847842329 wait complete. 2023-10-16T10:04:05.613 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T10:04:05.614 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:05.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.742+0000 7f38a1cf4700 1 Processor -- start 2023-10-16T10:04:05.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.742+0000 7f38a1cf4700 1 -- start start 2023-10-16T10:04:05.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.742+0000 7f38a1cf4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c0af450 0x7f389c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.742+0000 7f38a1cf4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c06f870 con 0x7f389c0cd470 2023-10-16T10:04:05.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.742+0000 7f38a1cf4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c0d1160 con 0x7f389c15dfd0 2023-10-16T10:04:05.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.742+0000 7f38a1cf4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0cc8e0 con 0x7f389c0af450 2023-10-16T10:04:05.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f389c0cd470 0x7f389c06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60288/0 (socket says 172.21.15.181:60288) 2023-10-16T10:04:05.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389bfff700 1 -- 172.21.15.181:0/62300194 learned_addr learned my addr 172.21.15.181:0/62300194 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:05.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389affd700 1 --2- 172.21.15.181:0/62300194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c0af450 0x7f389c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389a7fc700 1 -- 172.21.15.181:0/62300194 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820998830 0 0) 0x7f389c06f870 con 0x7f389c0cd470 2023-10-16T10:04:05.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389a7fc700 1 -- 172.21.15.181:0/62300194 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38880035b0 con 0x7f389c0cd470 2023-10-16T10:04:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389a7fc700 1 -- 172.21.15.181:0/62300194 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535733390 0 0) 0x7f389c0d1160 con 0x7f389c15dfd0 2023-10-16T10:04:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389a7fc700 1 -- 172.21.15.181:0/62300194 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f389c06f870 con 0x7f389c15dfd0 2023-10-16T10:04:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389affd700 1 -- 172.21.15.181:0/62300194 >> v1:172.21.15.190:6789/0 conn(0x7f389c15dfd0 legacy=0x7f389c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389affd700 1 -- 172.21.15.181:0/62300194 >> v1:172.21.15.181:6789/0 conn(0x7f389c0cd470 legacy=0x7f389c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389affd700 1 -- 172.21.15.181:0/62300194 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c0ccbc0 con 0x7f389c0af450 2023-10-16T10:04:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389affd700 1 --2- 172.21.15.181:0/62300194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c0af450 0x7f389c0af9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3884009870 tx=0x7f3884009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65250174ac72b92b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389a7fc700 1 -- 172.21.15.181:0/62300194 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884011830 con 0x7f389c0af450 2023-10-16T10:04:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389a7fc700 1 -- 172.21.15.181:0/62300194 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3884011990 con 0x7f389c0af450 2023-10-16T10:04:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.743+0000 7f389a7fc700 1 -- 172.21.15.181:0/62300194 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884011d00 con 0x7f389c0af450 2023-10-16T10:04:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.744+0000 7f38a1cf4700 1 -- 172.21.15.181:0/62300194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c0af450 msgr2=0x7f389c0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.744+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/62300194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c0af450 0x7f389c0af9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3884009870 tx=0x7f3884009b80 comp rx=0 tx=0).stop 2023-10-16T10:04:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.744+0000 7f38a1cf4700 1 -- 172.21.15.181:0/62300194 shutdown_connections 2023-10-16T10:04:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.744+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/62300194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c0af450 0x7f389c0af9a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.744+0000 7f38a1cf4700 1 -- 172.21.15.181:0/62300194 >> 172.21.15.181:0/62300194 conn(0x7f389c0d04a0 msgr2=0x7f389c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.744+0000 7f38a1cf4700 1 -- 172.21.15.181:0/62300194 shutdown_connections 2023-10-16T10:04:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.744+0000 7f38a1cf4700 1 -- 172.21.15.181:0/62300194 wait complete. 2023-10-16T10:04:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.745+0000 7f38a1cf4700 1 Processor -- start 2023-10-16T10:04:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.745+0000 7f38a1cf4700 1 -- start start 2023-10-16T10:04:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f38a1cf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c0af450 0x7f389c0c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f38a1cf4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389c0cd470 0x7f389c0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f38a1cf4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c15dfd0 0x7f389c1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f38a1cf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c40f0 con 0x7f389c0af450 2023-10-16T10:04:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f38a1cf4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0b0440 con 0x7f389c0cd470 2023-10-16T10:04:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f38a1cf4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0b26c0 con 0x7f389c15dfd0 2023-10-16T10:04:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f389b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c0af450 0x7f389c0c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f389b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c0af450 0x7f389c0c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49212/0 (socket says 172.21.15.181:49212) 2023-10-16T10:04:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f389b7fe700 1 -- 172.21.15.181:0/868824115 learned_addr learned my addr 172.21.15.181:0/868824115 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f389affd700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389c0cd470 0x7f389c0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f389bfff700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c15dfd0 0x7f389c1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.746+0000 7f389b7fe700 1 -- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c15dfd0 msgr2=0x7f389c1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f389b7fe700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c15dfd0 0x7f389c1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f389b7fe700 1 -- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389c0cd470 msgr2=0x7f389c0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f389b7fe700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389c0cd470 0x7f389c0aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f389b7fe700 1 -- 172.21.15.181:0/868824115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38840094a0 con 0x7f389c0af450 2023-10-16T10:04:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f389affd700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389c0cd470 0x7f389c0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f389bfff700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c15dfd0 0x7f389c1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f389b7fe700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c0af450 0x7f389c0c6a50 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c410 tx=0x7f388c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f3893fff700 1 -- 172.21.15.181:0/868824115 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0052d0 con 0x7f389c0af450 2023-10-16T10:04:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f3893fff700 1 -- 172.21.15.181:0/868824115 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c005450 con 0x7f389c0af450 2023-10-16T10:04:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f3893fff700 1 -- 172.21.15.181:0/868824115 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c003910 con 0x7f389c0af450 2023-10-16T10:04:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c0b29a0 con 0x7f389c0af450 2023-10-16T10:04:05.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.747+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c0c4320 con 0x7f389c0af450 2023-10-16T10:04:05.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.748+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c067d20 con 0x7f389c0af450 2023-10-16T10:04:05.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.748+0000 7f3893fff700 1 -- 172.21.15.181:0/868824115 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f388c003a70 con 0x7f389c0af450 2023-10-16T10:04:05.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.749+0000 7f3893fff700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f387c06b020 0x7f387c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:05.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.749+0000 7f3893fff700 1 -- 172.21.15.181:0/868824115 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f388c088ea0 con 0x7f389c0af450 2023-10-16T10:04:05.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.749+0000 7f389affd700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f387c06b020 0x7f387c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:05.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.749+0000 7f389affd700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f387c06b020 0x7f387c06d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38840090b0 tx=0x7f3884008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:05.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.753+0000 7f3893fff700 1 -- 172.21.15.181:0/868824115 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f388c053ca0 con 0x7f389c0af450 2023-10-16T10:04:05.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.988+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f389c06d640 con 0x7f389c0af450 2023-10-16T10:04:05.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.989+0000 7f3893fff700 1 -- 172.21.15.181:0/868824115 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f388c017020 con 0x7f389c0af450 2023-10-16T10:04:05.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:05.990 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1229456263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1229456263},{"type":"v1","addr":"172.21.15.190:6834","nonce":1229456263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3176781358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3176781358},{"type":"v1","addr":"172.21.15.181:6836","nonce":3176781358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2996369280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2996369280},{"type":"v1","addr":"172.21.15.181:6837","nonce":2996369280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:01.954427+0000","modified":"2023-10-16T10:04:04.993634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8146":{"gid":8146,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3854101486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3854101486},{"type":"v1","addr":"172.21.15.190:6835","nonce":3854101486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T10:04:05.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f387c06b020 msgr2=0x7f387c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f387c06b020 0x7f387c06d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38840090b0 tx=0x7f3884008030 comp rx=0 tx=0).stop 2023-10-16T10:04:05.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c0af450 msgr2=0x7f389c0c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:05.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c0af450 0x7f389c0c6a50 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c410 tx=0x7f388c00c7d0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 shutdown_connections 2023-10-16T10:04:05.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f387c06b020 0x7f387c06d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f389c15dfd0 0x7f389c1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f389c0af450 0x7f389c0c6a50 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 --2- 172.21.15.181:0/868824115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f389c0cd470 0x7f389c0aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.991+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 >> 172.21.15.181:0/868824115 conn(0x7f389c0d04a0 msgr2=0x7f389c0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.992+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 shutdown_connections 2023-10-16T10:04:05.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:05.992+0000 7f38a1cf4700 1 -- 172.21.15.181:0/868824115 wait complete. 2023-10-16T10:04:05.995 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 76 2023-10-16T10:04:06.005 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T10:04:06.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.125+0000 7f0ec9902700 1 Processor -- start 2023-10-16T10:04:06.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.125+0000 7f0ec9902700 1 -- start start 2023-10-16T10:04:06.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.125+0000 7f0ec9902700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ec40cab40 0x7f0ec4164e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.125+0000 7f0ec9902700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec4071df0 con 0x7f0ec415dd00 2023-10-16T10:04:06.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.125+0000 7f0ec9902700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec40720d0 con 0x7f0ec415e0e0 2023-10-16T10:04:06.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.125+0000 7f0ec9902700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec40723b0 con 0x7f0ec40cab40 2023-10-16T10:04:06.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0ec415dd00 0x7f0ec4071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60298/0 (socket says 172.21.15.181:60298) 2023-10-16T10:04:06.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec2ffd700 1 -- 172.21.15.181:0/4023502497 learned_addr learned my addr 172.21.15.181:0/4023502497 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:06.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033770180 0 0) 0x7f0ec4071df0 con 0x7f0ec415dd00 2023-10-16T10:04:06.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eac0035b0 con 0x7f0ec415dd00 2023-10-16T10:04:06.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec37fe700 1 --2- 172.21.15.181:0/4023502497 >> v2:172.21.15.190:3300/0 conn(0x7f0ec40cab40 0x7f0ec4164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1770790710 0 0) 0x7f0eac0035b0 con 0x7f0ec415dd00 2023-10-16T10:04:06.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 >> v2:172.21.15.190:3300/0 conn(0x7f0ec40cab40 msgr2=0x7f0ec4164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 --2- 172.21.15.181:0/4023502497 >> v2:172.21.15.190:3300/0 conn(0x7f0ec40cab40 0x7f0ec4164e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 >> v1:172.21.15.190:6789/0 conn(0x7f0ec415e0e0 legacy=0x7f0ec41574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4072690 con 0x7f0ec415dd00 2023-10-16T10:04:06.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f0eb4001670 con 0x7f0ec415dd00 2023-10-16T10:04:06.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 >> v1:172.21.15.181:6789/0 conn(0x7f0ec415dd00 legacy=0x7f0ec4071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eac003a00 0x7f0eac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0eac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eac005210 0x7f0eac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40723b0 con 0x7f0eac004850 2023-10-16T10:04:06.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac00c0b0 con 0x7f0eac003a00 2023-10-16T10:04:06.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac00c270 con 0x7f0eac005210 2023-10-16T10:04:06.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec27fc700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0eac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec2ffd700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eac003a00 0x7f0eac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.126+0000 7f0ec37fe700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eac005210 0x7f0eac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec27fc700 1 -- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eac005210 msgr2=0x7f0eac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec27fc700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eac005210 0x7f0eac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec27fc700 1 -- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eac003a00 msgr2=0x7f0eac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec27fc700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eac003a00 0x7f0eac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec27fc700 1 -- 172.21.15.181:0/4023502497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0035b0 con 0x7f0eac004850 2023-10-16T10:04:06.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec37fe700 1 -- 172.21.15.181:0/4023502497 reap_dead start 2023-10-16T10:04:06.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec2ffd700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eac003a00 0x7f0eac003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:06.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec27fc700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0eac004cd0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80017c0 tx=0x7f0eac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810e4d1d1f9a714e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb80052d0 con 0x7f0eac004850 2023-10-16T10:04:06.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb8005430 con 0x7f0eac004850 2023-10-16T10:04:06.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec1ffb700 1 -- 172.21.15.181:0/4023502497 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb80038c0 con 0x7f0eac004850 2023-10-16T10:04:06.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec9902700 1 -- 172.21.15.181:0/4023502497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 msgr2=0x7f0eac004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec9902700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0eac004cd0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80017c0 tx=0x7f0eac005880 comp rx=0 tx=0).stop 2023-10-16T10:04:06.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec9902700 1 -- 172.21.15.181:0/4023502497 shutdown_connections 2023-10-16T10:04:06.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec9902700 1 --2- 172.21.15.181:0/4023502497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0eac004cd0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec9902700 1 -- 172.21.15.181:0/4023502497 >> 172.21.15.181:0/4023502497 conn(0x7f0ec406aed0 msgr2=0x7f0ec40b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec9902700 1 -- 172.21.15.181:0/4023502497 shutdown_connections 2023-10-16T10:04:06.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.127+0000 7f0ec9902700 1 -- 172.21.15.181:0/4023502497 wait complete. 2023-10-16T10:04:06.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.129+0000 7f0ec9902700 1 Processor -- start 2023-10-16T10:04:06.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.129+0000 7f0ec9902700 1 -- start start 2023-10-16T10:04:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.129+0000 7f0ec9902700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0ec414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec9902700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ec415c9f0 0x7f0ec415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec9902700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ec40c39b0 0x7f0ec40c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec9902700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40c4fc0 con 0x7f0eac004850 2023-10-16T10:04:06.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec9902700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec414e350 con 0x7f0ec40c39b0 2023-10-16T10:04:06.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec9902700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec415d3b0 con 0x7f0ec415c9f0 2023-10-16T10:04:06.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0ec414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0ec414d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49232/0 (socket says 172.21.15.181:49232) 2023-10-16T10:04:06.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 -- 172.21.15.181:0/2714689657 learned_addr learned my addr 172.21.15.181:0/2714689657 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:06.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec27fc700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ec415c9f0 0x7f0ec415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec37fe700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ec40c39b0 0x7f0ec40c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 -- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ec415c9f0 msgr2=0x7f0ec415ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ec415c9f0 0x7f0ec415ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 -- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ec40c39b0 msgr2=0x7f0ec40c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ec40c39b0 0x7f0ec40c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 -- 172.21.15.181:0/2714689657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec40c4720 con 0x7f0eac004850 2023-10-16T10:04:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec2ffd700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0ec414d810 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e740 tx=0x7f0eb400eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0eabfff700 1 -- 172.21.15.181:0/2714689657 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4012590 con 0x7f0eac004850 2023-10-16T10:04:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0eabfff700 1 -- 172.21.15.181:0/2714689657 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb40126f0 con 0x7f0eac004850 2023-10-16T10:04:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec40c4a00 con 0x7f0eac004850 2023-10-16T10:04:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0eabfff700 1 -- 172.21.15.181:0/2714689657 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4012a80 con 0x7f0eac004850 2023-10-16T10:04:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.130+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec414e560 con 0x7f0eac004850 2023-10-16T10:04:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.131+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec404ebb0 con 0x7f0eac004850 2023-10-16T10:04:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.132+0000 7f0eabfff700 1 -- 172.21.15.181:0/2714689657 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0eb4012be0 con 0x7f0eac004850 2023-10-16T10:04:06.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.132+0000 7f0eabfff700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f0eb006b020 0x7f0eb006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.132+0000 7f0ec27fc700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f0eb006b020 0x7f0eb006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.132+0000 7f0eabfff700 1 -- 172.21.15.181:0/2714689657 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f0eb40896a0 con 0x7f0eac004850 2023-10-16T10:04:06.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.133+0000 7f0ec27fc700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f0eb006b020 0x7f0eb006d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8005040 tx=0x7f0eb8009030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.137+0000 7f0eabfff700 1 -- 172.21.15.181:0/2714689657 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0eb4054500 con 0x7f0eac004850 2023-10-16T10:04:06.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.332+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ec40cd700 con 0x7f0eac004850 2023-10-16T10:04:06.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.333+0000 7f0eabfff700 1 -- 172.21.15.181:0/2714689657 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17304 (secure 0 0 0) 0x7f0eb4057b20 con 0x7f0eac004850 2023-10-16T10:04:06.334 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:06.335 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":145,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T10:04:03.565540+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:00.393030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:00.916899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T10:04:06.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f0eb006b020 msgr2=0x7f0eb006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f0eb006b020 0x7f0eb006d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8005040 tx=0x7f0eb8009030 comp rx=0 tx=0).stop 2023-10-16T10:04:06.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 msgr2=0x7f0ec414d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0ec414d810 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e740 tx=0x7f0eb400eb00 comp rx=0 tx=0).stop 2023-10-16T10:04:06.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 shutdown_connections 2023-10-16T10:04:06.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f0eb006b020 0x7f0eb006d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ec40c39b0 0x7f0ec40c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eac004850 0x7f0ec414d810 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 --2- 172.21.15.181:0/2714689657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ec415c9f0 0x7f0ec415ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 >> 172.21.15.181:0/2714689657 conn(0x7f0ec406aed0 msgr2=0x7f0ec40ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 shutdown_connections 2023-10-16T10:04:06.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.336+0000 7f0ec9902700 1 -- 172.21.15.181:0/2714689657 wait complete. 2023-10-16T10:04:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T10:04:06.350 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:04:06.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.470+0000 7f8e4a577700 1 Processor -- start 2023-10-16T10:04:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.470+0000 7f8e4a577700 1 -- start start 2023-10-16T10:04:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e4a577700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8e4415dfd0 0x7f8e440af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e4a577700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e440cd470 0x7f8e440cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e4a577700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4406f870 con 0x7f8e440af450 2023-10-16T10:04:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e4a577700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440cde40 con 0x7f8e440cd470 2023-10-16T10:04:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e4a577700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440d1160 con 0x7f8e4415dfd0 2023-10-16T10:04:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e440cd470 0x7f8e440cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e440cd470 0x7f8e440cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49244/0 (socket says 172.21.15.181:49244) 2023-10-16T10:04:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 -- 172.21.15.181:0/532880454 learned_addr learned my addr 172.21.15.181:0/532880454 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 -- 172.21.15.181:0/532880454 >> v2:172.21.15.190:3300/0 conn(0x7f8e4415dfd0 msgr2=0x7f8e440af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e43fff700 1 --2- 172.21.15.181:0/532880454 >> v2:172.21.15.190:3300/0 conn(0x7f8e4415dfd0 0x7f8e440af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 --2- 172.21.15.181:0/532880454 >> v2:172.21.15.190:3300/0 conn(0x7f8e4415dfd0 0x7f8e440af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 -- 172.21.15.181:0/532880454 >> v1:172.21.15.190:6789/0 conn(0x7f8e440af450 legacy=0x7f8e4406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 -- 172.21.15.181:0/532880454 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440cc8e0 con 0x7f8e440cd470 2023-10-16T10:04:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e48b14700 1 --2- 172.21.15.181:0/532880454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e440cd900 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8e44070340 tx=0x7f8e3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9823895229c5bba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.471+0000 7f8e42ffd700 1 -- 172.21.15.181:0/532880454 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00e070 con 0x7f8e440cd470 2023-10-16T10:04:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e42ffd700 1 -- 172.21.15.181:0/532880454 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c003a20 con 0x7f8e440cd470 2023-10-16T10:04:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e42ffd700 1 -- 172.21.15.181:0/532880454 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c003dd0 con 0x7f8e440cd470 2023-10-16T10:04:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 -- 172.21.15.181:0/532880454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 msgr2=0x7f8e440cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 --2- 172.21.15.181:0/532880454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e440cd900 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8e44070340 tx=0x7f8e3c00b330 comp rx=0 tx=0).stop 2023-10-16T10:04:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 -- 172.21.15.181:0/532880454 shutdown_connections 2023-10-16T10:04:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 --2- 172.21.15.181:0/532880454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e440cd900 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 --2- 172.21.15.181:0/532880454 >> v2:172.21.15.190:3300/0 conn(0x7f8e4415dfd0 0x7f8e440af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 -- 172.21.15.181:0/532880454 >> 172.21.15.181:0/532880454 conn(0x7f8e440d04a0 msgr2=0x7f8e440d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 -- 172.21.15.181:0/532880454 shutdown_connections 2023-10-16T10:04:06.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.472+0000 7f8e4a577700 1 -- 172.21.15.181:0/532880454 wait complete. 2023-10-16T10:04:06.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.473+0000 7f8e4a577700 1 Processor -- start 2023-10-16T10:04:06.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.473+0000 7f8e4a577700 1 -- start start 2023-10-16T10:04:06.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e4a577700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e440af450 0x7f8e440c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e4a577700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e441f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e4a577700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4415dfd0 0x7f8e441f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e4a577700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b5e00 con 0x7f8e440cd470 2023-10-16T10:04:06.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e4a577700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b5280 con 0x7f8e4415dfd0 2023-10-16T10:04:06.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e4a577700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b5560 con 0x7f8e440af450 2023-10-16T10:04:06.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e441f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e441f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49248/0 (socket says 172.21.15.181:49248) 2023-10-16T10:04:06.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e437fe700 1 -- 172.21.15.181:0/1245652960 learned_addr learned my addr 172.21.15.181:0/1245652960 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:06.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e43fff700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e440af450 0x7f8e440c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.474+0000 7f8e48b14700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4415dfd0 0x7f8e441f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e437fe700 1 -- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e440af450 msgr2=0x7f8e440c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e437fe700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e440af450 0x7f8e440c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e437fe700 1 -- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4415dfd0 msgr2=0x7f8e441f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e437fe700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4415dfd0 0x7f8e441f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e437fe700 1 -- 172.21.15.181:0/1245652960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c009b30 con 0x7f8e440cd470 2023-10-16T10:04:06.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e437fe700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e441f2990 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400b470 tx=0x7f8e3400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e40ff9700 1 -- 172.21.15.181:0/1245652960 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011830 con 0x7f8e440cd470 2023-10-16T10:04:06.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e40ff9700 1 -- 172.21.15.181:0/1245652960 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e340119b0 con 0x7f8e440cd470 2023-10-16T10:04:06.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e440b5840 con 0x7f8e440cd470 2023-10-16T10:04:06.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e40ff9700 1 -- 172.21.15.181:0/1245652960 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011d60 con 0x7f8e440cd470 2023-10-16T10:04:06.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.475+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e440c7bb0 con 0x7f8e440cd470 2023-10-16T10:04:06.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.476+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e440c7dd0 con 0x7f8e440cd470 2023-10-16T10:04:06.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.476+0000 7f8e40ff9700 1 -- 172.21.15.181:0/1245652960 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8e3401a410 con 0x7f8e440cd470 2023-10-16T10:04:06.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.477+0000 7f8e40ff9700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8e3006b050 0x7f8e3006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.477+0000 7f8e43fff700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8e3006b050 0x7f8e3006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.477+0000 7f8e40ff9700 1 -- 172.21.15.181:0/1245652960 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f8e34089220 con 0x7f8e440cd470 2023-10-16T10:04:06.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.477+0000 7f8e43fff700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8e3006b050 0x7f8e3006d510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800ab40 tx=0x7f8e38008040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.481+0000 7f8e40ff9700 1 -- 172.21.15.181:0/1245652960 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8e34054080 con 0x7f8e440cd470 2023-10-16T10:04:06.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.671+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 --> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e440ccf50 con 0x7f8e3006b050 2023-10-16T10:04:06.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.680+0000 7f8e40ff9700 1 -- 172.21.15.181:0/1245652960 <== mgr.8877 v2:172.21.15.181:6832/3796559700 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289825 (secure 0 0 0) 0x7f8e3800c040 con 0x7f8e3006b050 2023-10-16T10:04:06.684 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:06.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8e3006b050 msgr2=0x7f8e3006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8e3006b050 0x7f8e3006d510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800ab40 tx=0x7f8e38008040 comp rx=0 tx=0).stop 2023-10-16T10:04:06.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 msgr2=0x7f8e441f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e441f2990 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400b470 tx=0x7f8e3400b780 comp rx=0 tx=0).stop 2023-10-16T10:04:06.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 shutdown_connections 2023-10-16T10:04:06.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f8e3006b050 0x7f8e3006d510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e440af450 0x7f8e440c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e4415dfd0 0x7f8e441f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 --2- 172.21.15.181:0/1245652960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e440cd470 0x7f8e441f2990 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 >> 172.21.15.181:0/1245652960 conn(0x7f8e440d04a0 msgr2=0x7f8e4406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 shutdown_connections 2023-10-16T10:04:06.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.683+0000 7f8e4a577700 1 -- 172.21.15.181:0/1245652960 wait complete. 2023-10-16T10:04:06.689 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:04:06.696 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:04:06.513580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":357720,"kb_used_data":15244,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754617000,"statfs":{"total":773094113280,"available":772727808000,"internally_reserved":0,"allocated":15609856,"data_stored":8325644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":350539320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.963131"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570429+0000","last_change":"2023-10-16T10:04:02.586793+0000","last_active":"2023-10-16T10:04:03.570429+0000","last_peered":"2023-10-16T10:04:03.570429+0000","last_clean":"2023-10-16T10:04:03.570429+0000","last_became_active":"2023-10-16T10:04:02.586517+0000","last_became_peered":"2023-10-16T10:04:02.586517+0000","last_unstale":"2023-10-16T10:04:03.570429+0000","last_undegraded":"2023-10-16T10:04:03.570429+0000","last_fullsized":"2023-10-16T10:04:03.570429+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:05.924440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580354+0000","last_change":"2023-10-16T10:04:02.577541+0000","last_active":"2023-10-16T10:04:02.580354+0000","last_peered":"2023-10-16T10:04:02.580354+0000","last_clean":"2023-10-16T10:04:02.580354+0000","last_became_active":"2023-10-16T10:04:02.577226+0000","last_became_peered":"2023-10-16T10:04:02.577226+0000","last_unstale":"2023-10-16T10:04:02.580354+0000","last_undegraded":"2023-10-16T10:04:02.580354+0000","last_fullsized":"2023-10-16T10:04:02.580354+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:12.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600358+0000","last_change":"2023-10-16T10:04:02.591204+0000","last_active":"2023-10-16T10:04:02.600358+0000","last_peered":"2023-10-16T10:04:02.600358+0000","last_clean":"2023-10-16T10:04:02.600358+0000","last_became_active":"2023-10-16T10:04:02.590822+0000","last_became_peered":"2023-10-16T10:04:02.590822+0000","last_unstale":"2023-10-16T10:04:02.600358+0000","last_undegraded":"2023-10-16T10:04:02.600358+0000","last_fullsized":"2023-10-16T10:04:02.600358+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:52.848362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570336+0000","last_change":"2023-10-16T10:04:02.584772+0000","last_active":"2023-10-16T10:04:03.570336+0000","last_peered":"2023-10-16T10:04:03.570336+0000","last_clean":"2023-10-16T10:04:03.570336+0000","last_became_active":"2023-10-16T10:04:02.584518+0000","last_became_peered":"2023-10-16T10:04:02.584518+0000","last_unstale":"2023-10-16T10:04:03.570336+0000","last_undegraded":"2023-10-16T10:04:03.570336+0000","last_fullsized":"2023-10-16T10:04:03.570336+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:51.995841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.598329+0000","last_change":"2023-10-16T10:04:02.590182+0000","last_active":"2023-10-16T10:04:02.598329+0000","last_peered":"2023-10-16T10:04:02.598329+0000","last_clean":"2023-10-16T10:04:02.598329+0000","last_became_active":"2023-10-16T10:04:02.589724+0000","last_became_peered":"2023-10-16T10:04:02.589724+0000","last_unstale":"2023-10-16T10:04:02.598329+0000","last_undegraded":"2023-10-16T10:04:02.598329+0000","last_fullsized":"2023-10-16T10:04:02.598329+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:34.550162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.590948+0000","last_change":"2023-10-16T10:04:02.585659+0000","last_active":"2023-10-16T10:04:02.590948+0000","last_peered":"2023-10-16T10:04:02.590948+0000","last_clean":"2023-10-16T10:04:02.590948+0000","last_became_active":"2023-10-16T10:04:02.585406+0000","last_became_peered":"2023-10-16T10:04:02.585406+0000","last_unstale":"2023-10-16T10:04:02.590948+0000","last_undegraded":"2023-10-16T10:04:02.590948+0000","last_fullsized":"2023-10-16T10:04:02.590948+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:04.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602496+0000","last_change":"2023-10-16T10:04:02.601662+0000","last_active":"2023-10-16T10:04:02.602496+0000","last_peered":"2023-10-16T10:04:02.602496+0000","last_clean":"2023-10-16T10:04:02.602496+0000","last_became_active":"2023-10-16T10:04:02.601280+0000","last_became_peered":"2023-10-16T10:04:02.601280+0000","last_unstale":"2023-10-16T10:04:02.602496+0000","last_undegraded":"2023-10-16T10:04:02.602496+0000","last_fullsized":"2023-10-16T10:04:02.602496+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:47.426033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600612+0000","last_change":"2023-10-16T10:04:02.592492+0000","last_active":"2023-10-16T10:04:02.600612+0000","last_peered":"2023-10-16T10:04:02.600612+0000","last_clean":"2023-10-16T10:04:02.600612+0000","last_became_active":"2023-10-16T10:04:02.592203+0000","last_became_peered":"2023-10-16T10:04:02.592203+0000","last_unstale":"2023-10-16T10:04:02.600612+0000","last_undegraded":"2023-10-16T10:04:02.600612+0000","last_fullsized":"2023-10-16T10:04:02.600612+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:20:50.023393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600915+0000","last_change":"2023-10-16T10:04:02.595183+0000","last_active":"2023-10-16T10:04:02.600915+0000","last_peered":"2023-10-16T10:04:02.600915+0000","last_clean":"2023-10-16T10:04:02.600915+0000","last_became_active":"2023-10-16T10:04:02.594931+0000","last_became_peered":"2023-10-16T10:04:02.594931+0000","last_unstale":"2023-10-16T10:04:02.600915+0000","last_undegraded":"2023-10-16T10:04:02.600915+0000","last_fullsized":"2023-10-16T10:04:02.600915+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:31.502539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580333+0000","last_change":"2023-10-16T10:04:02.577509+0000","last_active":"2023-10-16T10:04:02.580333+0000","last_peered":"2023-10-16T10:04:02.580333+0000","last_clean":"2023-10-16T10:04:02.580333+0000","last_became_active":"2023-10-16T10:04:02.577214+0000","last_became_peered":"2023-10-16T10:04:02.577214+0000","last_unstale":"2023-10-16T10:04:02.580333+0000","last_undegraded":"2023-10-16T10:04:02.580333+0000","last_fullsized":"2023-10-16T10:04:02.580333+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:27.255747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.574415+0000","last_change":"2023-10-16T10:04:02.573890+0000","last_active":"2023-10-16T10:04:02.574415+0000","last_peered":"2023-10-16T10:04:02.574415+0000","last_clean":"2023-10-16T10:04:02.574415+0000","last_became_active":"2023-10-16T10:04:02.573611+0000","last_became_peered":"2023-10-16T10:04:02.573611+0000","last_unstale":"2023-10-16T10:04:02.574415+0000","last_undegraded":"2023-10-16T10:04:02.574415+0000","last_fullsized":"2023-10-16T10:04:02.574415+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:43:00.869962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600941+0000","last_change":"2023-10-16T10:04:02.595098+0000","last_active":"2023-10-16T10:04:02.600941+0000","last_peered":"2023-10-16T10:04:02.600941+0000","last_clean":"2023-10-16T10:04:02.600941+0000","last_became_active":"2023-10-16T10:04:02.594835+0000","last_became_peered":"2023-10-16T10:04:02.594835+0000","last_unstale":"2023-10-16T10:04:02.600941+0000","last_undegraded":"2023-10-16T10:04:02.600941+0000","last_fullsized":"2023-10-16T10:04:02.600941+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:51.276024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570391+0000","last_change":"2023-10-16T10:04:02.586779+0000","last_active":"2023-10-16T10:04:03.570391+0000","last_peered":"2023-10-16T10:04:03.570391+0000","last_clean":"2023-10-16T10:04:03.570391+0000","last_became_active":"2023-10-16T10:04:02.586510+0000","last_became_peered":"2023-10-16T10:04:02.586510+0000","last_unstale":"2023-10-16T10:04:03.570391+0000","last_undegraded":"2023-10-16T10:04:03.570391+0000","last_fullsized":"2023-10-16T10:04:03.570391+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:27.255747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.582331+0000","last_change":"2023-10-16T10:04:02.581315+0000","last_active":"2023-10-16T10:04:02.582331+0000","last_peered":"2023-10-16T10:04:02.582331+0000","last_clean":"2023-10-16T10:04:02.582331+0000","last_became_active":"2023-10-16T10:04:02.581064+0000","last_became_peered":"2023-10-16T10:04:02.581064+0000","last_unstale":"2023-10-16T10:04:02.582331+0000","last_undegraded":"2023-10-16T10:04:02.582331+0000","last_fullsized":"2023-10-16T10:04:02.582331+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:38.159666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569990+0000","last_change":"2023-10-16T10:04:02.601424+0000","last_active":"2023-10-16T10:04:03.569990+0000","last_peered":"2023-10-16T10:04:03.569990+0000","last_clean":"2023-10-16T10:04:03.569990+0000","last_became_active":"2023-10-16T10:04:02.601030+0000","last_became_peered":"2023-10-16T10:04:02.601030+0000","last_unstale":"2023-10-16T10:04:03.569990+0000","last_undegraded":"2023-10-16T10:04:03.569990+0000","last_fullsized":"2023-10-16T10:04:03.569990+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:40.278099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.568975+0000","last_change":"2023-10-16T10:04:02.578748+0000","last_active":"2023-10-16T10:04:03.568975+0000","last_peered":"2023-10-16T10:04:03.568975+0000","last_clean":"2023-10-16T10:04:03.568975+0000","last_became_active":"2023-10-16T10:04:02.578453+0000","last_became_peered":"2023-10-16T10:04:02.578453+0000","last_unstale":"2023-10-16T10:04:03.568975+0000","last_undegraded":"2023-10-16T10:04:03.568975+0000","last_fullsized":"2023-10-16T10:04:03.568975+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:53.062282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.586613+0000","last_change":"2023-10-16T10:04:02.583197+0000","last_active":"2023-10-16T10:04:02.586613+0000","last_peered":"2023-10-16T10:04:02.586613+0000","last_clean":"2023-10-16T10:04:02.586613+0000","last_became_active":"2023-10-16T10:04:02.582753+0000","last_became_peered":"2023-10-16T10:04:02.582753+0000","last_unstale":"2023-10-16T10:04:02.586613+0000","last_undegraded":"2023-10-16T10:04:02.586613+0000","last_fullsized":"2023-10-16T10:04:02.586613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:47.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569380+0000","last_change":"2023-10-16T10:04:02.580972+0000","last_active":"2023-10-16T10:04:03.569380+0000","last_peered":"2023-10-16T10:04:03.569380+0000","last_clean":"2023-10-16T10:04:03.569380+0000","last_became_active":"2023-10-16T10:04:02.580623+0000","last_became_peered":"2023-10-16T10:04:02.580623+0000","last_unstale":"2023-10-16T10:04:03.569380+0000","last_undegraded":"2023-10-16T10:04:03.569380+0000","last_fullsized":"2023-10-16T10:04:03.569380+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:10.846791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.581487+0000","last_change":"2023-10-16T10:04:02.577488+0000","last_active":"2023-10-16T10:04:02.581487+0000","last_peered":"2023-10-16T10:04:02.581487+0000","last_clean":"2023-10-16T10:04:02.581487+0000","last_became_active":"2023-10-16T10:04:02.577223+0000","last_became_peered":"2023-10-16T10:04:02.577223+0000","last_unstale":"2023-10-16T10:04:02.581487+0000","last_undegraded":"2023-10-16T10:04:02.581487+0000","last_fullsized":"2023-10-16T10:04:02.581487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:28.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569421+0000","last_change":"2023-10-16T10:04:02.572783+0000","last_active":"2023-10-16T10:04:03.569421+0000","last_peered":"2023-10-16T10:04:03.569421+0000","last_clean":"2023-10-16T10:04:03.569421+0000","last_became_active":"2023-10-16T10:04:02.572478+0000","last_became_peered":"2023-10-16T10:04:02.572478+0000","last_unstale":"2023-10-16T10:04:03.569421+0000","last_undegraded":"2023-10-16T10:04:03.569421+0000","last_fullsized":"2023-10-16T10:04:03.569421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:12:24.129273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.598306+0000","last_change":"2023-10-16T10:04:02.590000+0000","last_active":"2023-10-16T10:04:02.598306+0000","last_peered":"2023-10-16T10:04:02.598306+0000","last_clean":"2023-10-16T10:04:02.598306+0000","last_became_active":"2023-10-16T10:04:02.589358+0000","last_became_peered":"2023-10-16T10:04:02.589358+0000","last_unstale":"2023-10-16T10:04:02.598306+0000","last_undegraded":"2023-10-16T10:04:02.598306+0000","last_fullsized":"2023-10-16T10:04:02.598306+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:26.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.573612+0000","last_change":"2023-10-16T10:04:02.573052+0000","last_active":"2023-10-16T10:04:02.573612+0000","last_peered":"2023-10-16T10:04:02.573612+0000","last_clean":"2023-10-16T10:04:02.573612+0000","last_became_active":"2023-10-16T10:04:02.572786+0000","last_became_peered":"2023-10-16T10:04:02.572786+0000","last_unstale":"2023-10-16T10:04:02.573612+0000","last_undegraded":"2023-10-16T10:04:02.573612+0000","last_fullsized":"2023-10-16T10:04:02.573612+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:11.534908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600546+0000","last_change":"2023-10-16T10:04:02.594335+0000","last_active":"2023-10-16T10:04:02.600546+0000","last_peered":"2023-10-16T10:04:02.600546+0000","last_clean":"2023-10-16T10:04:02.600546+0000","last_became_active":"2023-10-16T10:04:02.593661+0000","last_became_peered":"2023-10-16T10:04:02.593661+0000","last_unstale":"2023-10-16T10:04:02.600546+0000","last_undegraded":"2023-10-16T10:04:02.600546+0000","last_fullsized":"2023-10-16T10:04:02.600546+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:25.759600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.589231+0000","last_change":"2023-10-16T10:04:02.588856+0000","last_active":"2023-10-16T10:04:02.589231+0000","last_peered":"2023-10-16T10:04:02.589231+0000","last_clean":"2023-10-16T10:04:02.589231+0000","last_became_active":"2023-10-16T10:04:02.588520+0000","last_became_peered":"2023-10-16T10:04:02.588520+0000","last_unstale":"2023-10-16T10:04:02.589231+0000","last_undegraded":"2023-10-16T10:04:02.589231+0000","last_fullsized":"2023-10-16T10:04:02.589231+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:48.153915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.587990+0000","last_change":"2023-10-16T10:04:02.586644+0000","last_active":"2023-10-16T10:04:02.587990+0000","last_peered":"2023-10-16T10:04:02.587990+0000","last_clean":"2023-10-16T10:04:02.587990+0000","last_became_active":"2023-10-16T10:04:02.586364+0000","last_became_peered":"2023-10-16T10:04:02.586364+0000","last_unstale":"2023-10-16T10:04:02.587990+0000","last_undegraded":"2023-10-16T10:04:02.587990+0000","last_fullsized":"2023-10-16T10:04:02.587990+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:56.291708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569575+0000","last_change":"2023-10-16T10:04:02.581466+0000","last_active":"2023-10-16T10:04:03.569575+0000","last_peered":"2023-10-16T10:04:03.569575+0000","last_clean":"2023-10-16T10:04:03.569575+0000","last_became_active":"2023-10-16T10:04:02.581191+0000","last_became_peered":"2023-10-16T10:04:02.581191+0000","last_unstale":"2023-10-16T10:04:03.569575+0000","last_undegraded":"2023-10-16T10:04:03.569575+0000","last_fullsized":"2023-10-16T10:04:03.569575+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:38:05.858233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570362+0000","last_change":"2023-10-16T10:04:02.581351+0000","last_active":"2023-10-16T10:04:03.570362+0000","last_peered":"2023-10-16T10:04:03.570362+0000","last_clean":"2023-10-16T10:04:03.570362+0000","last_became_active":"2023-10-16T10:04:02.581065+0000","last_became_peered":"2023-10-16T10:04:02.581065+0000","last_unstale":"2023-10-16T10:04:03.570362+0000","last_undegraded":"2023-10-16T10:04:03.570362+0000","last_fullsized":"2023-10-16T10:04:03.570362+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:26.647712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570182+0000","last_change":"2023-10-16T10:04:02.587434+0000","last_active":"2023-10-16T10:04:03.570182+0000","last_peered":"2023-10-16T10:04:03.570182+0000","last_clean":"2023-10-16T10:04:03.570182+0000","last_became_active":"2023-10-16T10:04:02.586671+0000","last_became_peered":"2023-10-16T10:04:02.586671+0000","last_unstale":"2023-10-16T10:04:03.570182+0000","last_undegraded":"2023-10-16T10:04:03.570182+0000","last_fullsized":"2023-10-16T10:04:03.570182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:53.220052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570216+0000","last_change":"2023-10-16T10:04:02.601653+0000","last_active":"2023-10-16T10:04:03.570216+0000","last_peered":"2023-10-16T10:04:03.570216+0000","last_clean":"2023-10-16T10:04:03.570216+0000","last_became_active":"2023-10-16T10:04:02.601068+0000","last_became_peered":"2023-10-16T10:04:02.601068+0000","last_unstale":"2023-10-16T10:04:03.570216+0000","last_undegraded":"2023-10-16T10:04:03.570216+0000","last_fullsized":"2023-10-16T10:04:03.570216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:59.436835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570180+0000","last_change":"2023-10-16T10:04:02.596511+0000","last_active":"2023-10-16T10:04:03.570180+0000","last_peered":"2023-10-16T10:04:03.570180+0000","last_clean":"2023-10-16T10:04:03.570180+0000","last_became_active":"2023-10-16T10:04:02.596205+0000","last_became_peered":"2023-10-16T10:04:02.596205+0000","last_unstale":"2023-10-16T10:04:03.570180+0000","last_undegraded":"2023-10-16T10:04:03.570180+0000","last_fullsized":"2023-10-16T10:04:03.570180+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:59.614714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.597723+0000","last_change":"2023-10-16T10:04:02.591025+0000","last_active":"2023-10-16T10:04:02.597723+0000","last_peered":"2023-10-16T10:04:02.597723+0000","last_clean":"2023-10-16T10:04:02.597723+0000","last_became_active":"2023-10-16T10:04:02.589290+0000","last_became_peered":"2023-10-16T10:04:02.589290+0000","last_unstale":"2023-10-16T10:04:02.597723+0000","last_undegraded":"2023-10-16T10:04:02.597723+0000","last_fullsized":"2023-10-16T10:04:02.597723+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:07.783459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580188+0000","last_change":"2023-10-16T10:04:02.577386+0000","last_active":"2023-10-16T10:04:02.580188+0000","last_peered":"2023-10-16T10:04:02.580188+0000","last_clean":"2023-10-16T10:04:02.580188+0000","last_became_active":"2023-10-16T10:04:02.577118+0000","last_became_peered":"2023-10-16T10:04:02.577118+0000","last_unstale":"2023-10-16T10:04:02.580188+0000","last_undegraded":"2023-10-16T10:04:02.580188+0000","last_fullsized":"2023-10-16T10:04:02.580188+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:51:30.341763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569555+0000","last_change":"2023-10-16T10:04:02.586215+0000","last_active":"2023-10-16T10:04:03.569555+0000","last_peered":"2023-10-16T10:04:03.569555+0000","last_clean":"2023-10-16T10:04:03.569555+0000","last_became_active":"2023-10-16T10:04:02.585929+0000","last_became_peered":"2023-10-16T10:04:02.585929+0000","last_unstale":"2023-10-16T10:04:03.569555+0000","last_undegraded":"2023-10-16T10:04:03.569555+0000","last_fullsized":"2023-10-16T10:04:03.569555+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:53:49.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"144'4","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600836+0000","last_change":"2023-10-16T10:04:01.572025+0000","last_active":"2023-10-16T10:04:02.600836+0000","last_peered":"2023-10-16T10:04:02.600836+0000","last_clean":"2023-10-16T10:04:02.600836+0000","last_became_active":"2023-10-16T10:04:01.571745+0000","last_became_peered":"2023-10-16T10:04:01.571745+0000","last_unstale":"2023-10-16T10:04:02.600836+0000","last_undegraded":"2023-10-16T10:04:02.600836+0000","last_fullsized":"2023-10-16T10:04:02.600836+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:58.997435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600518+0000","last_change":"2023-10-16T10:04:02.591439+0000","last_active":"2023-10-16T10:04:02.600518+0000","last_peered":"2023-10-16T10:04:02.600518+0000","last_clean":"2023-10-16T10:04:02.600518+0000","last_became_active":"2023-10-16T10:04:02.590017+0000","last_became_peered":"2023-10-16T10:04:02.590017+0000","last_unstale":"2023-10-16T10:04:02.600518+0000","last_undegraded":"2023-10-16T10:04:02.600518+0000","last_fullsized":"2023-10-16T10:04:02.600518+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:21:21.684230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"144'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570401+0000","last_change":"2023-10-16T10:04:01.574227+0000","last_active":"2023-10-16T10:04:03.570401+0000","last_peered":"2023-10-16T10:04:03.570401+0000","last_clean":"2023-10-16T10:04:03.570401+0000","last_became_active":"2023-10-16T10:04:01.573945+0000","last_became_peered":"2023-10-16T10:04:01.573945+0000","last_unstale":"2023-10-16T10:04:03.570401+0000","last_undegraded":"2023-10-16T10:04:03.570401+0000","last_fullsized":"2023-10-16T10:04:03.570401+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:23.228033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596695+0000","last_change":"2023-10-16T10:04:02.588489+0000","last_active":"2023-10-16T10:04:02.596695+0000","last_peered":"2023-10-16T10:04:02.596695+0000","last_clean":"2023-10-16T10:04:02.596695+0000","last_became_active":"2023-10-16T10:04:02.588205+0000","last_became_peered":"2023-10-16T10:04:02.588205+0000","last_unstale":"2023-10-16T10:04:02.596695+0000","last_undegraded":"2023-10-16T10:04:02.596695+0000","last_fullsized":"2023-10-16T10:04:02.596695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:31.566338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602658+0000","last_change":"2023-10-16T10:04:01.571927+0000","last_active":"2023-10-16T10:04:02.602658+0000","last_peered":"2023-10-16T10:04:02.602658+0000","last_clean":"2023-10-16T10:04:02.602658+0000","last_became_active":"2023-10-16T10:04:01.571650+0000","last_became_peered":"2023-10-16T10:04:01.571650+0000","last_unstale":"2023-10-16T10:04:02.602658+0000","last_undegraded":"2023-10-16T10:04:02.602658+0000","last_fullsized":"2023-10-16T10:04:02.602658+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:10.211904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.594244+0000","last_change":"2023-10-16T10:04:02.585094+0000","last_active":"2023-10-16T10:04:02.594244+0000","last_peered":"2023-10-16T10:04:02.594244+0000","last_clean":"2023-10-16T10:04:02.594244+0000","last_became_active":"2023-10-16T10:04:02.583916+0000","last_became_peered":"2023-10-16T10:04:02.583916+0000","last_unstale":"2023-10-16T10:04:02.594244+0000","last_undegraded":"2023-10-16T10:04:02.594244+0000","last_fullsized":"2023-10-16T10:04:02.594244+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:18.435506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602233+0000","last_change":"2023-10-16T10:04:01.574345+0000","last_active":"2023-10-16T10:04:02.602233+0000","last_peered":"2023-10-16T10:04:02.602233+0000","last_clean":"2023-10-16T10:04:02.602233+0000","last_became_active":"2023-10-16T10:04:01.574016+0000","last_became_peered":"2023-10-16T10:04:01.574016+0000","last_unstale":"2023-10-16T10:04:02.602233+0000","last_undegraded":"2023-10-16T10:04:02.602233+0000","last_fullsized":"2023-10-16T10:04:02.602233+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:48.534444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569374+0000","last_change":"2023-10-16T10:04:02.579189+0000","last_active":"2023-10-16T10:04:03.569374+0000","last_peered":"2023-10-16T10:04:03.569374+0000","last_clean":"2023-10-16T10:04:03.569374+0000","last_became_active":"2023-10-16T10:04:02.578892+0000","last_became_peered":"2023-10-16T10:04:02.578892+0000","last_unstale":"2023-10-16T10:04:03.569374+0000","last_undegraded":"2023-10-16T10:04:03.569374+0000","last_fullsized":"2023-10-16T10:04:03.569374+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:57:46.029828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"144'1","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.595802+0000","last_change":"2023-10-16T10:04:01.569217+0000","last_active":"2023-10-16T10:04:02.595802+0000","last_peered":"2023-10-16T10:04:02.595802+0000","last_clean":"2023-10-16T10:04:02.595802+0000","last_became_active":"2023-10-16T10:04:01.568809+0000","last_became_peered":"2023-10-16T10:04:01.568809+0000","last_unstale":"2023-10-16T10:04:02.595802+0000","last_undegraded":"2023-10-16T10:04:02.595802+0000","last_fullsized":"2023-10-16T10:04:02.595802+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:39.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.586491+0000","last_change":"2023-10-16T10:04:02.582996+0000","last_active":"2023-10-16T10:04:02.586491+0000","last_peered":"2023-10-16T10:04:02.586491+0000","last_clean":"2023-10-16T10:04:02.586491+0000","last_became_active":"2023-10-16T10:04:02.582587+0000","last_became_peered":"2023-10-16T10:04:02.582587+0000","last_unstale":"2023-10-16T10:04:02.586491+0000","last_undegraded":"2023-10-16T10:04:02.586491+0000","last_fullsized":"2023-10-16T10:04:02.586491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:21.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"144'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569128+0000","last_change":"2023-10-16T10:04:01.572373+0000","last_active":"2023-10-16T10:04:03.569128+0000","last_peered":"2023-10-16T10:04:03.569128+0000","last_clean":"2023-10-16T10:04:03.569128+0000","last_became_active":"2023-10-16T10:04:01.571927+0000","last_became_peered":"2023-10-16T10:04:01.571927+0000","last_unstale":"2023-10-16T10:04:03.569128+0000","last_undegraded":"2023-10-16T10:04:03.569128+0000","last_fullsized":"2023-10-16T10:04:03.569128+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:15.857118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600634+0000","last_change":"2023-10-16T10:04:02.594497+0000","last_active":"2023-10-16T10:04:02.600634+0000","last_peered":"2023-10-16T10:04:02.600634+0000","last_clean":"2023-10-16T10:04:02.600634+0000","last_became_active":"2023-10-16T10:04:02.594219+0000","last_became_peered":"2023-10-16T10:04:02.594219+0000","last_unstale":"2023-10-16T10:04:02.600634+0000","last_undegraded":"2023-10-16T10:04:02.600634+0000","last_fullsized":"2023-10-16T10:04:02.600634+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:38.385014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.573045+0000","last_change":"2023-10-16T10:04:01.563983+0000","last_active":"2023-10-16T10:04:02.573045+0000","last_peered":"2023-10-16T10:04:02.573045+0000","last_clean":"2023-10-16T10:04:02.573045+0000","last_became_active":"2023-10-16T10:04:01.563678+0000","last_became_peered":"2023-10-16T10:04:01.563678+0000","last_unstale":"2023-10-16T10:04:02.573045+0000","last_undegraded":"2023-10-16T10:04:02.573045+0000","last_fullsized":"2023-10-16T10:04:02.573045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:14.618053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.582416+0000","last_change":"2023-10-16T10:04:02.579387+0000","last_active":"2023-10-16T10:04:02.582416+0000","last_peered":"2023-10-16T10:04:02.582416+0000","last_clean":"2023-10-16T10:04:02.582416+0000","last_became_active":"2023-10-16T10:04:02.579135+0000","last_became_peered":"2023-10-16T10:04:02.579135+0000","last_unstale":"2023-10-16T10:04:02.582416+0000","last_undegraded":"2023-10-16T10:04:02.582416+0000","last_fullsized":"2023-10-16T10:04:02.582416+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:59.436835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567867+0000","last_change":"2023-10-16T10:04:01.570323+0000","last_active":"2023-10-16T10:04:02.567867+0000","last_peered":"2023-10-16T10:04:02.567867+0000","last_clean":"2023-10-16T10:04:02.567867+0000","last_became_active":"2023-10-16T10:04:01.570060+0000","last_became_peered":"2023-10-16T10:04:01.570060+0000","last_unstale":"2023-10-16T10:04:02.567867+0000","last_undegraded":"2023-10-16T10:04:02.567867+0000","last_fullsized":"2023-10-16T10:04:02.567867+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:04.919764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570474+0000","last_change":"2023-10-16T10:04:02.583940+0000","last_active":"2023-10-16T10:04:03.570474+0000","last_peered":"2023-10-16T10:04:03.570474+0000","last_clean":"2023-10-16T10:04:03.570474+0000","last_became_active":"2023-10-16T10:04:02.583689+0000","last_became_peered":"2023-10-16T10:04:02.583689+0000","last_unstale":"2023-10-16T10:04:03.570474+0000","last_undegraded":"2023-10-16T10:04:03.570474+0000","last_fullsized":"2023-10-16T10:04:03.570474+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:12.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570681+0000","last_change":"2023-10-16T10:04:01.565644+0000","last_active":"2023-10-16T10:04:03.570681+0000","last_peered":"2023-10-16T10:04:03.570681+0000","last_clean":"2023-10-16T10:04:03.570681+0000","last_became_active":"2023-10-16T10:04:01.565251+0000","last_became_peered":"2023-10-16T10:04:01.565251+0000","last_unstale":"2023-10-16T10:04:03.570681+0000","last_undegraded":"2023-10-16T10:04:03.570681+0000","last_fullsized":"2023-10-16T10:04:03.570681+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:30.737019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570502+0000","last_change":"2023-10-16T10:04:02.593619+0000","last_active":"2023-10-16T10:04:03.570502+0000","last_peered":"2023-10-16T10:04:03.570502+0000","last_clean":"2023-10-16T10:04:03.570502+0000","last_became_active":"2023-10-16T10:04:02.593349+0000","last_became_peered":"2023-10-16T10:04:02.593349+0000","last_unstale":"2023-10-16T10:04:03.570502+0000","last_undegraded":"2023-10-16T10:04:03.570502+0000","last_fullsized":"2023-10-16T10:04:03.570502+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:19.058639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569643+0000","last_change":"2023-10-16T10:04:01.565268+0000","last_active":"2023-10-16T10:04:03.569643+0000","last_peered":"2023-10-16T10:04:03.569643+0000","last_clean":"2023-10-16T10:04:03.569643+0000","last_became_active":"2023-10-16T10:04:01.563155+0000","last_became_peered":"2023-10-16T10:04:01.563155+0000","last_unstale":"2023-10-16T10:04:03.569643+0000","last_undegraded":"2023-10-16T10:04:03.569643+0000","last_fullsized":"2023-10-16T10:04:03.569643+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.594220+0000","last_change":"2023-10-16T10:04:02.584773+0000","last_active":"2023-10-16T10:04:02.594220+0000","last_peered":"2023-10-16T10:04:02.594220+0000","last_clean":"2023-10-16T10:04:02.594220+0000","last_became_active":"2023-10-16T10:04:02.584475+0000","last_became_peered":"2023-10-16T10:04:02.584475+0000","last_unstale":"2023-10-16T10:04:02.594220+0000","last_undegraded":"2023-10-16T10:04:02.594220+0000","last_fullsized":"2023-10-16T10:04:02.594220+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:29:22.042904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"144'3","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570131+0000","last_change":"2023-10-16T10:04:01.559977+0000","last_active":"2023-10-16T10:04:03.570131+0000","last_peered":"2023-10-16T10:04:03.570131+0000","last_clean":"2023-10-16T10:04:03.570131+0000","last_became_active":"2023-10-16T10:04:01.559666+0000","last_became_peered":"2023-10-16T10:04:01.559666+0000","last_unstale":"2023-10-16T10:04:03.570131+0000","last_undegraded":"2023-10-16T10:04:03.570131+0000","last_fullsized":"2023-10-16T10:04:03.570131+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:47.049742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.599787+0000","last_change":"2023-10-16T10:04:02.598274+0000","last_active":"2023-10-16T10:04:02.599787+0000","last_peered":"2023-10-16T10:04:02.599787+0000","last_clean":"2023-10-16T10:04:02.599787+0000","last_became_active":"2023-10-16T10:04:02.597438+0000","last_became_peered":"2023-10-16T10:04:02.597438+0000","last_unstale":"2023-10-16T10:04:02.599787+0000","last_undegraded":"2023-10-16T10:04:02.599787+0000","last_fullsized":"2023-10-16T10:04:02.599787+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:21.548633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"144'6","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.605524+0000","last_change":"2023-10-16T10:04:01.565485+0000","last_active":"2023-10-16T10:04:02.605524+0000","last_peered":"2023-10-16T10:04:02.605524+0000","last_clean":"2023-10-16T10:04:02.605524+0000","last_became_active":"2023-10-16T10:04:01.565202+0000","last_became_peered":"2023-10-16T10:04:01.565202+0000","last_unstale":"2023-10-16T10:04:02.605524+0000","last_undegraded":"2023-10-16T10:04:02.605524+0000","last_fullsized":"2023-10-16T10:04:02.605524+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:22.727966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570758+0000","last_change":"2023-10-16T10:04:02.583014+0000","last_active":"2023-10-16T10:04:03.570758+0000","last_peered":"2023-10-16T10:04:03.570758+0000","last_clean":"2023-10-16T10:04:03.570758+0000","last_became_active":"2023-10-16T10:04:02.582696+0000","last_became_peered":"2023-10-16T10:04:02.582696+0000","last_unstale":"2023-10-16T10:04:03.570758+0000","last_undegraded":"2023-10-16T10:04:03.570758+0000","last_fullsized":"2023-10-16T10:04:03.570758+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:10.861091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570218+0000","last_change":"2023-10-16T10:04:01.569192+0000","last_active":"2023-10-16T10:04:03.570218+0000","last_peered":"2023-10-16T10:04:03.570218+0000","last_clean":"2023-10-16T10:04:03.570218+0000","last_became_active":"2023-10-16T10:04:01.568893+0000","last_became_peered":"2023-10-16T10:04:01.568893+0000","last_unstale":"2023-10-16T10:04:03.570218+0000","last_undegraded":"2023-10-16T10:04:03.570218+0000","last_fullsized":"2023-10-16T10:04:03.570218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:42.540059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602689+0000","last_change":"2023-10-16T10:04:02.602370+0000","last_active":"2023-10-16T10:04:02.602689+0000","last_peered":"2023-10-16T10:04:02.602689+0000","last_clean":"2023-10-16T10:04:02.602689+0000","last_became_active":"2023-10-16T10:04:02.601714+0000","last_became_peered":"2023-10-16T10:04:02.601714+0000","last_unstale":"2023-10-16T10:04:02.602689+0000","last_undegraded":"2023-10-16T10:04:02.602689+0000","last_fullsized":"2023-10-16T10:04:02.602689+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:59.614714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567733+0000","last_change":"2023-10-16T10:04:01.563990+0000","last_active":"2023-10-16T10:04:02.567733+0000","last_peered":"2023-10-16T10:04:02.567733+0000","last_clean":"2023-10-16T10:04:02.567733+0000","last_became_active":"2023-10-16T10:04:01.563729+0000","last_became_peered":"2023-10-16T10:04:01.563729+0000","last_unstale":"2023-10-16T10:04:02.567733+0000","last_undegraded":"2023-10-16T10:04:02.567733+0000","last_fullsized":"2023-10-16T10:04:02.567733+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:50.991166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600617+0000","last_change":"2023-10-16T10:04:02.594492+0000","last_active":"2023-10-16T10:04:02.600617+0000","last_peered":"2023-10-16T10:04:02.600617+0000","last_clean":"2023-10-16T10:04:02.600617+0000","last_became_active":"2023-10-16T10:04:02.593952+0000","last_became_peered":"2023-10-16T10:04:02.593952+0000","last_unstale":"2023-10-16T10:04:02.600617+0000","last_undegraded":"2023-10-16T10:04:02.600617+0000","last_fullsized":"2023-10-16T10:04:02.600617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:04.014734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"144'8","reported_seq":19,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.603667+0000","last_change":"2023-10-16T10:04:01.566320+0000","last_active":"2023-10-16T10:04:02.603667+0000","last_peered":"2023-10-16T10:04:02.603667+0000","last_clean":"2023-10-16T10:04:02.603667+0000","last_became_active":"2023-10-16T10:04:01.566038+0000","last_became_peered":"2023-10-16T10:04:01.566038+0000","last_unstale":"2023-10-16T10:04:02.603667+0000","last_undegraded":"2023-10-16T10:04:02.603667+0000","last_fullsized":"2023-10-16T10:04:02.603667+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:11.532626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596513+0000","last_change":"2023-10-16T10:04:02.590480+0000","last_active":"2023-10-16T10:04:02.596513+0000","last_peered":"2023-10-16T10:04:02.596513+0000","last_clean":"2023-10-16T10:04:02.596513+0000","last_became_active":"2023-10-16T10:04:02.590172+0000","last_became_peered":"2023-10-16T10:04:02.590172+0000","last_unstale":"2023-10-16T10:04:02.596513+0000","last_undegraded":"2023-10-16T10:04:02.596513+0000","last_fullsized":"2023-10-16T10:04:02.596513+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:11.499998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"144'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570781+0000","last_change":"2023-10-16T10:04:01.558524+0000","last_active":"2023-10-16T10:04:03.570781+0000","last_peered":"2023-10-16T10:04:03.570781+0000","last_clean":"2023-10-16T10:04:03.570781+0000","last_became_active":"2023-10-16T10:04:01.558229+0000","last_became_peered":"2023-10-16T10:04:01.558229+0000","last_unstale":"2023-10-16T10:04:03.570781+0000","last_undegraded":"2023-10-16T10:04:03.570781+0000","last_fullsized":"2023-10-16T10:04:03.570781+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:28.626659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570629+0000","last_change":"2023-10-16T10:04:02.592746+0000","last_active":"2023-10-16T10:04:03.570629+0000","last_peered":"2023-10-16T10:04:03.570629+0000","last_clean":"2023-10-16T10:04:03.570629+0000","last_became_active":"2023-10-16T10:04:02.592471+0000","last_became_peered":"2023-10-16T10:04:02.592471+0000","last_unstale":"2023-10-16T10:04:03.570629+0000","last_undegraded":"2023-10-16T10:04:03.570629+0000","last_fullsized":"2023-10-16T10:04:03.570629+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:03.818818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.603468+0000","last_change":"2023-10-16T10:04:01.573876+0000","last_active":"2023-10-16T10:04:02.603468+0000","last_peered":"2023-10-16T10:04:02.603468+0000","last_clean":"2023-10-16T10:04:02.603468+0000","last_became_active":"2023-10-16T10:04:01.573519+0000","last_became_peered":"2023-10-16T10:04:01.573519+0000","last_unstale":"2023-10-16T10:04:02.603468+0000","last_undegraded":"2023-10-16T10:04:02.603468+0000","last_fullsized":"2023-10-16T10:04:02.603468+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:35.652882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569440+0000","last_change":"2023-10-16T10:04:02.579600+0000","last_active":"2023-10-16T10:04:03.569440+0000","last_peered":"2023-10-16T10:04:03.569440+0000","last_clean":"2023-10-16T10:04:03.569440+0000","last_became_active":"2023-10-16T10:04:02.579271+0000","last_became_peered":"2023-10-16T10:04:02.579271+0000","last_unstale":"2023-10-16T10:04:03.569440+0000","last_undegraded":"2023-10-16T10:04:03.569440+0000","last_fullsized":"2023-10-16T10:04:03.569440+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:37.043627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"144'1","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.599738+0000","last_change":"2023-10-16T10:04:01.571111+0000","last_active":"2023-10-16T10:04:02.599738+0000","last_peered":"2023-10-16T10:04:02.599738+0000","last_clean":"2023-10-16T10:04:02.599738+0000","last_became_active":"2023-10-16T10:04:01.570735+0000","last_became_peered":"2023-10-16T10:04:01.570735+0000","last_unstale":"2023-10-16T10:04:02.599738+0000","last_undegraded":"2023-10-16T10:04:02.599738+0000","last_fullsized":"2023-10-16T10:04:02.599738+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:32:20.936832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.582301+0000","last_change":"2023-10-16T10:04:02.580443+0000","last_active":"2023-10-16T10:04:02.582301+0000","last_peered":"2023-10-16T10:04:02.582301+0000","last_clean":"2023-10-16T10:04:02.582301+0000","last_became_active":"2023-10-16T10:04:02.579622+0000","last_became_peered":"2023-10-16T10:04:02.579622+0000","last_unstale":"2023-10-16T10:04:02.582301+0000","last_undegraded":"2023-10-16T10:04:02.582301+0000","last_fullsized":"2023-10-16T10:04:02.582301+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:10.861091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570171+0000","last_change":"2023-10-16T10:04:01.574381+0000","last_active":"2023-10-16T10:04:03.570171+0000","last_peered":"2023-10-16T10:04:03.570171+0000","last_clean":"2023-10-16T10:04:03.570171+0000","last_became_active":"2023-10-16T10:04:01.574038+0000","last_became_peered":"2023-10-16T10:04:01.574038+0000","last_unstale":"2023-10-16T10:04:03.570171+0000","last_undegraded":"2023-10-16T10:04:03.570171+0000","last_fullsized":"2023-10-16T10:04:03.570171+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:13.676699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570132+0000","last_change":"2023-10-16T10:04:02.581856+0000","last_active":"2023-10-16T10:04:03.570132+0000","last_peered":"2023-10-16T10:04:03.570132+0000","last_clean":"2023-10-16T10:04:03.570132+0000","last_became_active":"2023-10-16T10:04:02.581545+0000","last_became_peered":"2023-10-16T10:04:02.581545+0000","last_unstale":"2023-10-16T10:04:03.570132+0000","last_undegraded":"2023-10-16T10:04:03.570132+0000","last_fullsized":"2023-10-16T10:04:03.570132+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:38.159666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"144'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570341+0000","last_change":"2023-10-16T10:04:01.571931+0000","last_active":"2023-10-16T10:04:03.570341+0000","last_peered":"2023-10-16T10:04:03.570341+0000","last_clean":"2023-10-16T10:04:03.570341+0000","last_became_active":"2023-10-16T10:04:01.571610+0000","last_became_peered":"2023-10-16T10:04:01.571610+0000","last_unstale":"2023-10-16T10:04:03.570341+0000","last_undegraded":"2023-10-16T10:04:03.570341+0000","last_fullsized":"2023-10-16T10:04:03.570341+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:52.006116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.599949+0000","last_change":"2023-10-16T10:04:02.599276+0000","last_active":"2023-10-16T10:04:02.599949+0000","last_peered":"2023-10-16T10:04:02.599949+0000","last_clean":"2023-10-16T10:04:02.599949+0000","last_became_active":"2023-10-16T10:04:02.599015+0000","last_became_peered":"2023-10-16T10:04:02.599015+0000","last_unstale":"2023-10-16T10:04:02.599949+0000","last_undegraded":"2023-10-16T10:04:02.599949+0000","last_fullsized":"2023-10-16T10:04:02.599949+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:29.891777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569209+0000","last_change":"2023-10-16T10:04:01.563456+0000","last_active":"2023-10-16T10:04:03.569209+0000","last_peered":"2023-10-16T10:04:03.569209+0000","last_clean":"2023-10-16T10:04:03.569209+0000","last_became_active":"2023-10-16T10:04:01.563166+0000","last_became_peered":"2023-10-16T10:04:01.563166+0000","last_unstale":"2023-10-16T10:04:03.569209+0000","last_undegraded":"2023-10-16T10:04:03.569209+0000","last_fullsized":"2023-10-16T10:04:03.569209+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:54.583503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.575538+0000","last_change":"2023-10-16T10:04:02.573255+0000","last_active":"2023-10-16T10:04:02.575538+0000","last_peered":"2023-10-16T10:04:02.575538+0000","last_clean":"2023-10-16T10:04:02.575538+0000","last_became_active":"2023-10-16T10:04:02.572998+0000","last_became_peered":"2023-10-16T10:04:02.572998+0000","last_unstale":"2023-10-16T10:04:02.575538+0000","last_undegraded":"2023-10-16T10:04:02.575538+0000","last_fullsized":"2023-10-16T10:04:02.575538+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:12.537301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.568926+0000","last_change":"2023-10-16T10:04:01.558099+0000","last_active":"2023-10-16T10:04:03.568926+0000","last_peered":"2023-10-16T10:04:03.568926+0000","last_clean":"2023-10-16T10:04:03.568926+0000","last_became_active":"2023-10-16T10:04:01.557452+0000","last_became_peered":"2023-10-16T10:04:01.557452+0000","last_unstale":"2023-10-16T10:04:03.568926+0000","last_undegraded":"2023-10-16T10:04:03.568926+0000","last_fullsized":"2023-10-16T10:04:03.568926+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:07.596628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.597745+0000","last_change":"2023-10-16T10:04:02.591073+0000","last_active":"2023-10-16T10:04:02.597745+0000","last_peered":"2023-10-16T10:04:02.597745+0000","last_clean":"2023-10-16T10:04:02.597745+0000","last_became_active":"2023-10-16T10:04:02.589611+0000","last_became_peered":"2023-10-16T10:04:02.589611+0000","last_unstale":"2023-10-16T10:04:02.597745+0000","last_undegraded":"2023-10-16T10:04:02.597745+0000","last_fullsized":"2023-10-16T10:04:02.597745+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:08.601304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.571103+0000","last_change":"2023-10-16T10:04:01.573306+0000","last_active":"2023-10-16T10:04:02.571103+0000","last_peered":"2023-10-16T10:04:02.571103+0000","last_clean":"2023-10-16T10:04:02.571103+0000","last_became_active":"2023-10-16T10:04:01.573013+0000","last_became_peered":"2023-10-16T10:04:01.573013+0000","last_unstale":"2023-10-16T10:04:02.571103+0000","last_undegraded":"2023-10-16T10:04:02.571103+0000","last_fullsized":"2023-10-16T10:04:02.571103+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:29.820650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.593396+0000","last_change":"2023-10-16T10:04:02.587800+0000","last_active":"2023-10-16T10:04:02.593396+0000","last_peered":"2023-10-16T10:04:02.593396+0000","last_clean":"2023-10-16T10:04:02.593396+0000","last_became_active":"2023-10-16T10:04:02.587519+0000","last_became_peered":"2023-10-16T10:04:02.587519+0000","last_unstale":"2023-10-16T10:04:02.593396+0000","last_undegraded":"2023-10-16T10:04:02.593396+0000","last_fullsized":"2023-10-16T10:04:02.593396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:02.256227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"144'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570627+0000","last_change":"2023-10-16T10:04:01.570740+0000","last_active":"2023-10-16T10:04:03.570627+0000","last_peered":"2023-10-16T10:04:03.570627+0000","last_clean":"2023-10-16T10:04:03.570627+0000","last_became_active":"2023-10-16T10:04:01.570244+0000","last_became_peered":"2023-10-16T10:04:01.570244+0000","last_unstale":"2023-10-16T10:04:03.570627+0000","last_undegraded":"2023-10-16T10:04:03.570627+0000","last_fullsized":"2023-10-16T10:04:03.570627+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:20:49.018718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570603+0000","last_change":"2023-10-16T10:04:02.597569+0000","last_active":"2023-10-16T10:04:03.570603+0000","last_peered":"2023-10-16T10:04:03.570603+0000","last_clean":"2023-10-16T10:04:03.570603+0000","last_became_active":"2023-10-16T10:04:02.597303+0000","last_became_peered":"2023-10-16T10:04:02.597303+0000","last_unstale":"2023-10-16T10:04:03.570603+0000","last_undegraded":"2023-10-16T10:04:03.570603+0000","last_fullsized":"2023-10-16T10:04:03.570603+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:15.562192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.579504+0000","last_change":"2023-10-16T10:04:01.564207+0000","last_active":"2023-10-16T10:04:02.579504+0000","last_peered":"2023-10-16T10:04:02.579504+0000","last_clean":"2023-10-16T10:04:02.579504+0000","last_became_active":"2023-10-16T10:04:01.563896+0000","last_became_peered":"2023-10-16T10:04:01.563896+0000","last_unstale":"2023-10-16T10:04:02.579504+0000","last_undegraded":"2023-10-16T10:04:02.579504+0000","last_fullsized":"2023-10-16T10:04:02.579504+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:33.202190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.585036+0000","last_change":"2023-10-16T10:04:02.584569+0000","last_active":"2023-10-16T10:04:02.585036+0000","last_peered":"2023-10-16T10:04:02.585036+0000","last_clean":"2023-10-16T10:04:02.585036+0000","last_became_active":"2023-10-16T10:04:02.584243+0000","last_became_peered":"2023-10-16T10:04:02.584243+0000","last_unstale":"2023-10-16T10:04:02.585036+0000","last_undegraded":"2023-10-16T10:04:02.585036+0000","last_fullsized":"2023-10-16T10:04:02.585036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:20.554648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567899+0000","last_change":"2023-10-16T10:04:01.569580+0000","last_active":"2023-10-16T10:04:02.567899+0000","last_peered":"2023-10-16T10:04:02.567899+0000","last_clean":"2023-10-16T10:04:02.567899+0000","last_became_active":"2023-10-16T10:04:01.569301+0000","last_became_peered":"2023-10-16T10:04:01.569301+0000","last_unstale":"2023-10-16T10:04:02.567899+0000","last_undegraded":"2023-10-16T10:04:02.567899+0000","last_fullsized":"2023-10-16T10:04:02.567899+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:25.643036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.581875+0000","last_change":"2023-10-16T10:04:02.579642+0000","last_active":"2023-10-16T10:04:02.581875+0000","last_peered":"2023-10-16T10:04:02.581875+0000","last_clean":"2023-10-16T10:04:02.581875+0000","last_became_active":"2023-10-16T10:04:02.579146+0000","last_became_peered":"2023-10-16T10:04:02.579146+0000","last_unstale":"2023-10-16T10:04:02.581875+0000","last_undegraded":"2023-10-16T10:04:02.581875+0000","last_fullsized":"2023-10-16T10:04:02.581875+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:18.125405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.571865+0000","last_change":"2023-10-16T10:04:01.573867+0000","last_active":"2023-10-16T10:04:02.571865+0000","last_peered":"2023-10-16T10:04:02.571865+0000","last_clean":"2023-10-16T10:04:02.571865+0000","last_became_active":"2023-10-16T10:04:01.573583+0000","last_became_peered":"2023-10-16T10:04:01.573583+0000","last_unstale":"2023-10-16T10:04:02.571865+0000","last_undegraded":"2023-10-16T10:04:02.571865+0000","last_fullsized":"2023-10-16T10:04:02.571865+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:20.483076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596664+0000","last_change":"2023-10-16T10:04:02.591697+0000","last_active":"2023-10-16T10:04:02.596664+0000","last_peered":"2023-10-16T10:04:02.596664+0000","last_clean":"2023-10-16T10:04:02.596664+0000","last_became_active":"2023-10-16T10:04:02.591460+0000","last_became_peered":"2023-10-16T10:04:02.591460+0000","last_unstale":"2023-10-16T10:04:02.596664+0000","last_undegraded":"2023-10-16T10:04:02.596664+0000","last_fullsized":"2023-10-16T10:04:02.596664+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:03.848893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567941+0000","last_change":"2023-10-16T10:04:01.569004+0000","last_active":"2023-10-16T10:04:02.567941+0000","last_peered":"2023-10-16T10:04:02.567941+0000","last_clean":"2023-10-16T10:04:02.567941+0000","last_became_active":"2023-10-16T10:04:01.568471+0000","last_became_peered":"2023-10-16T10:04:01.568471+0000","last_unstale":"2023-10-16T10:04:02.567941+0000","last_undegraded":"2023-10-16T10:04:02.567941+0000","last_fullsized":"2023-10-16T10:04:02.567941+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:14.557516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570309+0000","last_change":"2023-10-16T10:04:02.585338+0000","last_active":"2023-10-16T10:04:03.570309+0000","last_peered":"2023-10-16T10:04:03.570309+0000","last_clean":"2023-10-16T10:04:03.570309+0000","last_became_active":"2023-10-16T10:04:02.585067+0000","last_became_peered":"2023-10-16T10:04:02.585067+0000","last_unstale":"2023-10-16T10:04:03.570309+0000","last_undegraded":"2023-10-16T10:04:03.570309+0000","last_fullsized":"2023-10-16T10:04:03.570309+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:55.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580429+0000","last_change":"2023-10-16T10:04:01.570145+0000","last_active":"2023-10-16T10:04:02.580429+0000","last_peered":"2023-10-16T10:04:02.580429+0000","last_clean":"2023-10-16T10:04:02.580429+0000","last_became_active":"2023-10-16T10:04:01.569872+0000","last_became_peered":"2023-10-16T10:04:01.569872+0000","last_unstale":"2023-10-16T10:04:02.580429+0000","last_undegraded":"2023-10-16T10:04:02.580429+0000","last_fullsized":"2023-10-16T10:04:02.580429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:51.435313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.568976+0000","last_change":"2023-10-16T10:04:02.593222+0000","last_active":"2023-10-16T10:04:03.568976+0000","last_peered":"2023-10-16T10:04:03.568976+0000","last_clean":"2023-10-16T10:04:03.568976+0000","last_became_active":"2023-10-16T10:04:02.592903+0000","last_became_peered":"2023-10-16T10:04:02.592903+0000","last_unstale":"2023-10-16T10:04:03.568976+0000","last_undegraded":"2023-10-16T10:04:03.568976+0000","last_fullsized":"2023-10-16T10:04:03.568976+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:57.459012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569015+0000","last_change":"2023-10-16T10:04:01.563914+0000","last_active":"2023-10-16T10:04:03.569015+0000","last_peered":"2023-10-16T10:04:03.569015+0000","last_clean":"2023-10-16T10:04:03.569015+0000","last_became_active":"2023-10-16T10:04:01.563619+0000","last_became_peered":"2023-10-16T10:04:01.563619+0000","last_unstale":"2023-10-16T10:04:03.569015+0000","last_undegraded":"2023-10-16T10:04:03.569015+0000","last_fullsized":"2023-10-16T10:04:03.569015+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:10.342856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.592966+0000","last_change":"2023-10-16T10:04:02.592274+0000","last_active":"2023-10-16T10:04:02.592966+0000","last_peered":"2023-10-16T10:04:02.592966+0000","last_clean":"2023-10-16T10:04:02.592966+0000","last_became_active":"2023-10-16T10:04:02.591914+0000","last_became_peered":"2023-10-16T10:04:02.591914+0000","last_unstale":"2023-10-16T10:04:02.592966+0000","last_undegraded":"2023-10-16T10:04:02.592966+0000","last_fullsized":"2023-10-16T10:04:02.592966+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:43.544734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.571237+0000","last_change":"2023-10-16T10:04:01.570131+0000","last_active":"2023-10-16T10:04:02.571237+0000","last_peered":"2023-10-16T10:04:02.571237+0000","last_clean":"2023-10-16T10:04:02.571237+0000","last_became_active":"2023-10-16T10:04:01.569792+0000","last_became_peered":"2023-10-16T10:04:01.569792+0000","last_unstale":"2023-10-16T10:04:02.571237+0000","last_undegraded":"2023-10-16T10:04:02.571237+0000","last_fullsized":"2023-10-16T10:04:02.571237+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:35.625437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596642+0000","last_change":"2023-10-16T10:04:02.590688+0000","last_active":"2023-10-16T10:04:02.596642+0000","last_peered":"2023-10-16T10:04:02.596642+0000","last_clean":"2023-10-16T10:04:02.596642+0000","last_became_active":"2023-10-16T10:04:02.590413+0000","last_became_peered":"2023-10-16T10:04:02.590413+0000","last_unstale":"2023-10-16T10:04:02.596642+0000","last_undegraded":"2023-10-16T10:04:02.596642+0000","last_fullsized":"2023-10-16T10:04:02.596642+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:39.919315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600175+0000","last_change":"2023-10-16T10:04:01.571838+0000","last_active":"2023-10-16T10:04:02.600175+0000","last_peered":"2023-10-16T10:04:02.600175+0000","last_clean":"2023-10-16T10:04:02.600175+0000","last_became_active":"2023-10-16T10:04:01.571558+0000","last_became_peered":"2023-10-16T10:04:01.571558+0000","last_unstale":"2023-10-16T10:04:02.600175+0000","last_undegraded":"2023-10-16T10:04:02.600175+0000","last_fullsized":"2023-10-16T10:04:02.600175+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:47.369337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"135'249","reported_seq":753,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-16T10:04:01.555050+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:04:01.555050+0000","last_peered":"2023-10-16T10:04:01.555050+0000","last_clean":"2023-10-16T10:04:01.555050+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:04:01.555050+0000","last_undegraded":"2023-10-16T10:04:01.555050+0000","last_fullsized":"2023-10-16T10:04:01.555050+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542382,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33540,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338300,"statfs":{"total":96636764160,"available":96602419200,"internally_reserved":0,"allocated":2408448,"data_stored":1482597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55400000000000005,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.248},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.27200000000000002},"max":{"1min":0.627,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":1,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.58399999999999996},"min":{"1min":0.42299999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":1.788,"15min":1.788},"last":0.59599999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.308},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.33500000000000002,"15min":0.32600000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.70799999999999996}]},{"osd":3,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.41999999999999998,"5min":0.36499999999999999,"15min":0.35199999999999998},"max":{"1min":0.76400000000000001,"5min":0.874,"15min":0.874},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60199999999999998,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.626}]},{"osd":4,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.46600000000000003},"min":{"1min":0.33400000000000002,"5min":0.27400000000000002,"15min":0.26500000000000001},"max":{"1min":0.67500000000000004,"5min":0.76200000000000001,"15min":0.89200000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.5,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.374,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.64400000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.56200000000000006},"min":{"1min":0.38400000000000001,"5min":0.36399999999999999,"15min":0.30399999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.373},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.80900000000000005,"5min":0.83799999999999997,"15min":1.4099999999999999},"last":0.71999999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542381,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1802240,"data_stored":891591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":0.71599999999999997,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.57999999999999996},"min":{"1min":0.45100000000000001,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.55100000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.753,"5min":0.754,"15min":0.79800000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.58799999999999997},"min":{"1min":0.41299999999999998,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.72599999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.86099999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.84499999999999997},"last":0.501}]},{"osd":3,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77600000000000002,"5min":1.03,"15min":1.03},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.34599999999999997},"max":{"1min":0.73399999999999999,"5min":0.79800000000000004,"15min":0.85899999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.752,"5min":1,"15min":1},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.33300000000000002},"max":{"1min":0.82699999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.42799999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.81699999999999995,"5min":1.056,"15min":1.056},"last":0.73899999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.33200000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80000000000000004,"5min":0.83299999999999996,"15min":0.86399999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.625}]}]},{"osd":1,"up_from":14,"seq":60129542381,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50984,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320856,"statfs":{"total":96636764160,"available":96584556544,"internally_reserved":0,"allocated":1789952,"data_stored":889979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.221,"15min":0.20100000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":1.5},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.40300000000000002},"min":{"1min":0.253,"5min":0.20599999999999999,"15min":0.19600000000000001},"max":{"1min":0.58399999999999996,"5min":0.60099999999999998,"15min":0.80000000000000004},"last":0.34300000000000003}]},{"osd":2,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.36899999999999999,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":1.1040000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.436},"min":{"1min":0.307,"5min":0.25,"15min":0.25},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":3.8730000000000002},"last":0.39900000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.436,"15min":0.46400000000000002},"min":{"1min":0.28899999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.64900000000000002,"5min":0.68000000000000005,"15min":4.6740000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.434},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":1.379},"last":0.38200000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.61399999999999999},"min":{"1min":0.45200000000000001,"5min":0.379,"15min":0.33100000000000002},"max":{"1min":0.86899999999999999,"5min":0.96899999999999997,"15min":1.2490000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":5.4119999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61199999999999999,"15min":0.628},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.94999999999999996,"15min":1.26},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.3560000000000001},"last":0.628}]},{"osd":6,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.374},"max":{"1min":1.026,"5min":1.026,"15min":6.2140000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.48499999999999999,"5min":0.36799999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":0.92600000000000005,"15min":1.3420000000000001},"last":0.57699999999999996}]},{"osd":7,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.98299999999999998,"15min":1.4079999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.65900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542382,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1480347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.192,"15min":0.191},"max":{"1min":0.53600000000000003,"5min":0.59999999999999998,"15min":0.65400000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.374,"15min":0.39700000000000002},"min":{"1min":0.24299999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.64900000000000002},"last":0.313}]},{"osd":2,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.40500000000000003},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.626},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.41899999999999998},"min":{"1min":0.28599999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.60799999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.43099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.72999999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65000000000000002,"5min":0.66500000000000004,"15min":0.69699999999999995},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80500000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.63},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78200000000000003,"5min":0.94599999999999995,"15min":0.98799999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61099999999999999,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.38500000000000001,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":1.004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.96999999999999997},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.63400000000000001},"min":{"1min":0.48299999999999998,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.83999999999999997,"5min":0.92400000000000004,"15min":0.92500000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83199999999999996,"5min":0.85999999999999999,"15min":1.0089999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61799999999999999,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81200000000000006,"5min":0.876,"15min":1.0249999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.65700000000000003},"min":{"1min":0.503,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.86099999999999999,"5min":0.90100000000000002,"15min":0.95699999999999996},"last":0.5}]}]},{"osd":2,"up_from":14,"seq":60129542381,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1810432,"data_stored":897347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36099999999999999,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.23100000000000001,"15min":0.183},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.64900000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.379,"15min":0.39900000000000002},"min":{"1min":0.27000000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.55600000000000005,"5min":0.621,"15min":0.63400000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.25800000000000001,"5min":0.252,"15min":0.245},"max":{"1min":0.55300000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40600000000000003,"15min":0.42299999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.214},"max":{"1min":0.622,"5min":0.622,"15min":0.64000000000000001},"last":0.32500000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.51800000000000002,"5min":0.59199999999999997,"15min":0.61899999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.43099999999999999,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.60599999999999998,"5min":0.63400000000000001,"15min":0.747},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.91600000000000004,"15min":0.93999999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.628},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":0.88200000000000001,"15min":1.091},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":3.4009999999999998,"5min":3.4009999999999998,"15min":4.3609999999999998},"last":0.372},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.872,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.44600000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.0720000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.624,"15min":0.64500000000000002},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.97899999999999998},"last":0.38700000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.4289999999999998,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38500000000000001},"max":{"1min":0.94899999999999995,"5min":0.98799999999999999,"15min":0.998},"last":0.51000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37116,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334724,"statfs":{"total":96636764160,"available":96598757376,"internally_reserved":0,"allocated":1810432,"data_stored":896891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35999999999999999,"15min":0.38300000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.64900000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.41499999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.185},"max":{"1min":0.45700000000000002,"5min":0.626,"15min":0.63100000000000001},"last":0.33800000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.36299999999999999,"15min":0.39200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":0.54100000000000004,"15min":0.72199999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39200000000000002,"15min":0.41899999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48099999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41099999999999998,"15min":0.433},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.26000000000000001},"max":{"1min":0.46800000000000003,"5min":0.60599999999999998,"15min":0.70999999999999996},"last":0.318},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42899999999999999,"15min":0.44800000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54900000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.60699999999999998},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.74299999999999999,"5min":0.95699999999999996,"15min":0.95799999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.621},"min":{"1min":0.318,"5min":0.314,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.88700000000000001,"15min":1.0820000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.436,"5min":0.374,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.93700000000000006,"15min":0.98699999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60899999999999999,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":0.94799999999999995,"15min":0.996},"last":0.624}]},{"osd":6,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61299999999999999,"15min":0.63900000000000001},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.755,"5min":0.92400000000000004,"15min":0.98399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59899999999999998,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.96799999999999997,"15min":0.99099999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60799999999999998,"15min":0.63600000000000001},"min":{"1min":0.42499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":1.3},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82099999999999995,"5min":0.91800000000000004,"15min":1.4019999999999999},"last":0.52200000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542381,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59300,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312540,"statfs":{"total":96636764160,"available":96576040960,"internally_reserved":0,"allocated":1794048,"data_stored":890049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.27600000000000002,"15min":0.186},"max":{"1min":0.71799999999999997,"5min":0.73699999999999999,"15min":0.89400000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.23899999999999999},"max":{"1min":0.745,"5min":0.78100000000000003,"15min":0.81299999999999994},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.57799999999999996},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.26300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86599999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.248},"max":{"1min":0.69499999999999995,"5min":0.77900000000000003,"15min":0.83799999999999997},"last":0.60799999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.34000000000000002,"15min":0.29799999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.81899999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55300000000000005,"15min":0.58999999999999997},"min":{"1min":0.41199999999999998,"5min":0.29399999999999998,"15min":0.27900000000000003},"max":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.82799999999999996},"last":0.48999999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.38100000000000001,"15min":0.314},"max":{"1min":0.78000000000000003,"5min":0.81200000000000006,"15min":0.83299999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59899999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.68799999999999994,"5min":0.77300000000000002,"15min":0.88300000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66500000000000004,"5min":0.73299999999999998,"15min":0.76800000000000002},"last":0.44},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.747,"5min":0.77800000000000002,"15min":0.80400000000000005},"last":0.502}]},{"osd":6,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.82999999999999996},"last":0.38},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.56000000000000005},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73699999999999999,"5min":0.79300000000000004,"15min":0.85899999999999999},"last":0.53800000000000003}]},{"osd":7,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52900000000000003,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.71099999999999997,"5min":0.753,"15min":0.80700000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.58699999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.83299999999999996,"15min":0.84599999999999997},"last":0.47499999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37112,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334728,"statfs":{"total":96636764160,"available":96598761472,"internally_reserved":0,"allocated":1810432,"data_stored":896843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69899999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55100000000000005,"15min":0.58799999999999997},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.79100000000000004,"15min":0.82799999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.92300000000000004},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.32300000000000001,"15min":0.315},"max":{"1min":0.70899999999999996,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.58599999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.73999999999999999,"5min":0.87,"15min":0.88700000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.89000000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59399999999999997,"15min":0.61499999999999999},"min":{"1min":0.46000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.627}]},{"osd":4,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.32900000000000001,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78400000000000003,"5min":0.86099999999999999,"15min":0.86799999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.311},"max":{"1min":0.71999999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82999999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70199999999999996}]},{"osd":7,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.89400000000000002,"5min":1.006,"15min":1.006},"last":0.752},{"interface":"front","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.35599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.874,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:04:06.703 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T10:04:06.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.858+0000 7fe7e85da700 1 Processor -- start 2023-10-16T10:04:06.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.858+0000 7fe7e85da700 1 -- start start 2023-10-16T10:04:06.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e85da700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 0x7fe7e00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e85da700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e006ed00 con 0x7fe7e00cb0c0 2023-10-16T10:04:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e85da700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e00d02d0 con 0x7fe7e015e6d0 2023-10-16T10:04:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e85da700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e006e0f0 con 0x7fe7e015eab0 2023-10-16T10:04:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e6b77700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe7e00cb0c0 0x7fe7e015a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60334/0 (socket says 172.21.15.181:60334) 2023-10-16T10:04:06.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e6b77700 1 -- 172.21.15.181:0/1761732489 learned_addr learned my addr 172.21.15.181:0/1761732489 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 0x7fe7e00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230260608 0 0) 0x7fe7e006ed00 con 0x7fe7e00cb0c0 2023-10-16T10:04:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7cc0035b0 con 0x7fe7e00cb0c0 2023-10-16T10:04:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 575117240 0 0) 0x7fe7cc0035b0 con 0x7fe7e00cb0c0 2023-10-16T10:04:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 msgr2=0x7fe7e00d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 0x7fe7e00d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 >> v1:172.21.15.190:6789/0 conn(0x7fe7e015e6d0 legacy=0x7fe7e015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e006e3d0 con 0x7fe7e00cb0c0 2023-10-16T10:04:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe7d80019f0 con 0x7fe7e00cb0c0 2023-10-16T10:04:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 >> v1:172.21.15.181:6789/0 conn(0x7fe7e00cb0c0 legacy=0x7fe7e015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 0x7fe7cc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7cc004d90 0x7fe7cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e006e0f0 con 0x7fe7cc003fb0 2023-10-16T10:04:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc00dd00 con 0x7fe7cc004d90 2023-10-16T10:04:06.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.859+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc00dec0 con 0x7fe7e015eab0 2023-10-16T10:04:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e6376700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 0x7fe7cc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e6b77700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7cc004d90 0x7fe7cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5b75700 1 -- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 msgr2=0x7fe7cc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 0x7fe7cc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5b75700 1 -- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7cc004d90 msgr2=0x7fe7cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7cc004d90 0x7fe7cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5b75700 1 -- 172.21.15.181:0/1761732489 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0035b0 con 0x7fe7cc003fb0 2023-10-16T10:04:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e6376700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015eab0 0x7fe7cc003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:06.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e6b77700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7cc004d90 0x7fe7cc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7cc004850 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009b60 tx=0x7fe7cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6498a40d3013c230 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000e6c0 con 0x7fe7cc003fb0 2023-10-16T10:04:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d000e820 con 0x7fe7cc003fb0 2023-10-16T10:04:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e5374700 1 -- 172.21.15.181:0/1761732489 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000eb90 con 0x7fe7cc003fb0 2023-10-16T10:04:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e85da700 1 -- 172.21.15.181:0/1761732489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 msgr2=0x7fe7cc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.860+0000 7fe7e85da700 1 --2- 172.21.15.181:0/1761732489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7cc004850 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009b60 tx=0x7fe7cc005880 comp rx=0 tx=0).stop 2023-10-16T10:04:06.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.861+0000 7fe7e6b77700 1 -- 172.21.15.181:0/1761732489 reap_dead start 2023-10-16T10:04:06.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.861+0000 7fe7e85da700 1 -- 172.21.15.181:0/1761732489 shutdown_connections 2023-10-16T10:04:06.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.861+0000 7fe7e85da700 1 -- 172.21.15.181:0/1761732489 >> 172.21.15.181:0/1761732489 conn(0x7fe7e00d04a0 msgr2=0x7fe7e00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:06.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.861+0000 7fe7e85da700 1 -- 172.21.15.181:0/1761732489 shutdown_connections 2023-10-16T10:04:06.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.861+0000 7fe7e85da700 1 -- 172.21.15.181:0/1761732489 wait complete. 2023-10-16T10:04:06.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.862+0000 7fe7e85da700 1 Processor -- start 2023-10-16T10:04:06.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.862+0000 7fe7e85da700 1 -- start start 2023-10-16T10:04:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7e00c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015af60 0x7fe7e00c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e00adfc0 0x7fe7e00ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e015d060 con 0x7fe7cc003fb0 2023-10-16T10:04:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00ae9b0 con 0x7fe7e00adfc0 2023-10-16T10:04:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00c7bc0 con 0x7fe7e015af60 2023-10-16T10:04:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7e00c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7e00c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49266/0 (socket says 172.21.15.181:49266) 2023-10-16T10:04:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 -- 172.21.15.181:0/3614355808 learned_addr learned my addr 172.21.15.181:0/3614355808 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6b77700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e00adfc0 0x7fe7e00ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015af60 0x7fe7e00c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 -- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015af60 msgr2=0x7fe7e00c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015af60 0x7fe7e00c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 -- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e00adfc0 msgr2=0x7fe7e00ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e00adfc0 0x7fe7e00ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 -- 172.21.15.181:0/3614355808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e015c770 con 0x7fe7cc003fb0 2023-10-16T10:04:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6b77700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e00adfc0 0x7fe7e00ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e6376700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7e00c9e50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e7b0 tx=0x7fe7dc00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7d6ffd700 1 -- 172.21.15.181:0/3614355808 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0052e0 con 0x7fe7cc003fb0 2023-10-16T10:04:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7d6ffd700 1 -- 172.21.15.181:0/3614355808 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc005440 con 0x7fe7cc003fb0 2023-10-16T10:04:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e01efbd0 con 0x7fe7cc003fb0 2023-10-16T10:04:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7d6ffd700 1 -- 172.21.15.181:0/3614355808 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc003910 con 0x7fe7cc003fb0 2023-10-16T10:04:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.863+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e01ef8f0 con 0x7fe7cc003fb0 2023-10-16T10:04:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.864+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e015b340 con 0x7fe7cc003fb0 2023-10-16T10:04:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.865+0000 7fe7d6ffd700 1 -- 172.21.15.181:0/3614355808 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe7dc003a70 con 0x7fe7cc003fb0 2023-10-16T10:04:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.865+0000 7fe7d6ffd700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fe7c406b020 0x7fe7c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.865+0000 7fe7d6ffd700 1 -- 172.21.15.181:0/3614355808 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fe7dc0896a0 con 0x7fe7cc003fb0 2023-10-16T10:04:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.865+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fe7c406b020 0x7fe7c406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.866+0000 7fe7e5b75700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fe7c406b020 0x7fe7c406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00090b0 tx=0x7fe7d0008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:06.870+0000 7fe7d6ffd700 1 -- 172.21.15.181:0/3614355808 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe7dc0543d0 con 0x7fe7cc003fb0 2023-10-16T10:04:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.055+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 --> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7e006e990 con 0x7fe7c406b020 2023-10-16T10:04:07.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.063+0000 7fe7d6ffd700 1 -- 172.21.15.181:0/3614355808 <== mgr.8877 v2:172.21.15.181:6832/3796559700 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289825 (secure 0 0 0) 0x7fe7d0022040 con 0x7fe7c406b020 2023-10-16T10:04:07.067 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:07.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fe7c406b020 msgr2=0x7fe7c406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fe7c406b020 0x7fe7c406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00090b0 tx=0x7fe7d0008030 comp rx=0 tx=0).stop 2023-10-16T10:04:07.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 msgr2=0x7fe7e00c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7e00c9e50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e7b0 tx=0x7fe7dc00eb70 comp rx=0 tx=0).stop 2023-10-16T10:04:07.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 shutdown_connections 2023-10-16T10:04:07.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fe7c406b020 0x7fe7c406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7e00adfc0 0x7fe7e00ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7cc003fb0 0x7fe7e00c9e50 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 --2- 172.21.15.181:0/3614355808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e015af60 0x7fe7e00c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.066+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 >> 172.21.15.181:0/3614355808 conn(0x7fe7e00d04a0 msgr2=0x7fe7e0158350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.067+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 shutdown_connections 2023-10-16T10:04:07.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.067+0000 7fe7e85da700 1 -- 172.21.15.181:0/3614355808 wait complete. 2023-10-16T10:04:07.072 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T10:04:07.079 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T10:04:06.513580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":357720,"kb_used_data":15244,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754617000,"statfs":{"total":773094113280,"available":772727808000,"internally_reserved":0,"allocated":15609856,"data_stored":8325644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":350539320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.963131"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570429+0000","last_change":"2023-10-16T10:04:02.586793+0000","last_active":"2023-10-16T10:04:03.570429+0000","last_peered":"2023-10-16T10:04:03.570429+0000","last_clean":"2023-10-16T10:04:03.570429+0000","last_became_active":"2023-10-16T10:04:02.586517+0000","last_became_peered":"2023-10-16T10:04:02.586517+0000","last_unstale":"2023-10-16T10:04:03.570429+0000","last_undegraded":"2023-10-16T10:04:03.570429+0000","last_fullsized":"2023-10-16T10:04:03.570429+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:05.924440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580354+0000","last_change":"2023-10-16T10:04:02.577541+0000","last_active":"2023-10-16T10:04:02.580354+0000","last_peered":"2023-10-16T10:04:02.580354+0000","last_clean":"2023-10-16T10:04:02.580354+0000","last_became_active":"2023-10-16T10:04:02.577226+0000","last_became_peered":"2023-10-16T10:04:02.577226+0000","last_unstale":"2023-10-16T10:04:02.580354+0000","last_undegraded":"2023-10-16T10:04:02.580354+0000","last_fullsized":"2023-10-16T10:04:02.580354+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:12.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600358+0000","last_change":"2023-10-16T10:04:02.591204+0000","last_active":"2023-10-16T10:04:02.600358+0000","last_peered":"2023-10-16T10:04:02.600358+0000","last_clean":"2023-10-16T10:04:02.600358+0000","last_became_active":"2023-10-16T10:04:02.590822+0000","last_became_peered":"2023-10-16T10:04:02.590822+0000","last_unstale":"2023-10-16T10:04:02.600358+0000","last_undegraded":"2023-10-16T10:04:02.600358+0000","last_fullsized":"2023-10-16T10:04:02.600358+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:52.848362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570336+0000","last_change":"2023-10-16T10:04:02.584772+0000","last_active":"2023-10-16T10:04:03.570336+0000","last_peered":"2023-10-16T10:04:03.570336+0000","last_clean":"2023-10-16T10:04:03.570336+0000","last_became_active":"2023-10-16T10:04:02.584518+0000","last_became_peered":"2023-10-16T10:04:02.584518+0000","last_unstale":"2023-10-16T10:04:03.570336+0000","last_undegraded":"2023-10-16T10:04:03.570336+0000","last_fullsized":"2023-10-16T10:04:03.570336+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:51.995841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.598329+0000","last_change":"2023-10-16T10:04:02.590182+0000","last_active":"2023-10-16T10:04:02.598329+0000","last_peered":"2023-10-16T10:04:02.598329+0000","last_clean":"2023-10-16T10:04:02.598329+0000","last_became_active":"2023-10-16T10:04:02.589724+0000","last_became_peered":"2023-10-16T10:04:02.589724+0000","last_unstale":"2023-10-16T10:04:02.598329+0000","last_undegraded":"2023-10-16T10:04:02.598329+0000","last_fullsized":"2023-10-16T10:04:02.598329+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:53:34.550162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.590948+0000","last_change":"2023-10-16T10:04:02.585659+0000","last_active":"2023-10-16T10:04:02.590948+0000","last_peered":"2023-10-16T10:04:02.590948+0000","last_clean":"2023-10-16T10:04:02.590948+0000","last_became_active":"2023-10-16T10:04:02.585406+0000","last_became_peered":"2023-10-16T10:04:02.585406+0000","last_unstale":"2023-10-16T10:04:02.590948+0000","last_undegraded":"2023-10-16T10:04:02.590948+0000","last_fullsized":"2023-10-16T10:04:02.590948+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:04.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602496+0000","last_change":"2023-10-16T10:04:02.601662+0000","last_active":"2023-10-16T10:04:02.602496+0000","last_peered":"2023-10-16T10:04:02.602496+0000","last_clean":"2023-10-16T10:04:02.602496+0000","last_became_active":"2023-10-16T10:04:02.601280+0000","last_became_peered":"2023-10-16T10:04:02.601280+0000","last_unstale":"2023-10-16T10:04:02.602496+0000","last_undegraded":"2023-10-16T10:04:02.602496+0000","last_fullsized":"2023-10-16T10:04:02.602496+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:47.426033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600612+0000","last_change":"2023-10-16T10:04:02.592492+0000","last_active":"2023-10-16T10:04:02.600612+0000","last_peered":"2023-10-16T10:04:02.600612+0000","last_clean":"2023-10-16T10:04:02.600612+0000","last_became_active":"2023-10-16T10:04:02.592203+0000","last_became_peered":"2023-10-16T10:04:02.592203+0000","last_unstale":"2023-10-16T10:04:02.600612+0000","last_undegraded":"2023-10-16T10:04:02.600612+0000","last_fullsized":"2023-10-16T10:04:02.600612+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:20:50.023393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600915+0000","last_change":"2023-10-16T10:04:02.595183+0000","last_active":"2023-10-16T10:04:02.600915+0000","last_peered":"2023-10-16T10:04:02.600915+0000","last_clean":"2023-10-16T10:04:02.600915+0000","last_became_active":"2023-10-16T10:04:02.594931+0000","last_became_peered":"2023-10-16T10:04:02.594931+0000","last_unstale":"2023-10-16T10:04:02.600915+0000","last_undegraded":"2023-10-16T10:04:02.600915+0000","last_fullsized":"2023-10-16T10:04:02.600915+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:31.502539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580333+0000","last_change":"2023-10-16T10:04:02.577509+0000","last_active":"2023-10-16T10:04:02.580333+0000","last_peered":"2023-10-16T10:04:02.580333+0000","last_clean":"2023-10-16T10:04:02.580333+0000","last_became_active":"2023-10-16T10:04:02.577214+0000","last_became_peered":"2023-10-16T10:04:02.577214+0000","last_unstale":"2023-10-16T10:04:02.580333+0000","last_undegraded":"2023-10-16T10:04:02.580333+0000","last_fullsized":"2023-10-16T10:04:02.580333+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:27.255747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.574415+0000","last_change":"2023-10-16T10:04:02.573890+0000","last_active":"2023-10-16T10:04:02.574415+0000","last_peered":"2023-10-16T10:04:02.574415+0000","last_clean":"2023-10-16T10:04:02.574415+0000","last_became_active":"2023-10-16T10:04:02.573611+0000","last_became_peered":"2023-10-16T10:04:02.573611+0000","last_unstale":"2023-10-16T10:04:02.574415+0000","last_undegraded":"2023-10-16T10:04:02.574415+0000","last_fullsized":"2023-10-16T10:04:02.574415+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:43:00.869962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600941+0000","last_change":"2023-10-16T10:04:02.595098+0000","last_active":"2023-10-16T10:04:02.600941+0000","last_peered":"2023-10-16T10:04:02.600941+0000","last_clean":"2023-10-16T10:04:02.600941+0000","last_became_active":"2023-10-16T10:04:02.594835+0000","last_became_peered":"2023-10-16T10:04:02.594835+0000","last_unstale":"2023-10-16T10:04:02.600941+0000","last_undegraded":"2023-10-16T10:04:02.600941+0000","last_fullsized":"2023-10-16T10:04:02.600941+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:43:51.276024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570391+0000","last_change":"2023-10-16T10:04:02.586779+0000","last_active":"2023-10-16T10:04:03.570391+0000","last_peered":"2023-10-16T10:04:03.570391+0000","last_clean":"2023-10-16T10:04:03.570391+0000","last_became_active":"2023-10-16T10:04:02.586510+0000","last_became_peered":"2023-10-16T10:04:02.586510+0000","last_unstale":"2023-10-16T10:04:03.570391+0000","last_undegraded":"2023-10-16T10:04:03.570391+0000","last_fullsized":"2023-10-16T10:04:03.570391+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:27.255747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.582331+0000","last_change":"2023-10-16T10:04:02.581315+0000","last_active":"2023-10-16T10:04:02.582331+0000","last_peered":"2023-10-16T10:04:02.582331+0000","last_clean":"2023-10-16T10:04:02.582331+0000","last_became_active":"2023-10-16T10:04:02.581064+0000","last_became_peered":"2023-10-16T10:04:02.581064+0000","last_unstale":"2023-10-16T10:04:02.582331+0000","last_undegraded":"2023-10-16T10:04:02.582331+0000","last_fullsized":"2023-10-16T10:04:02.582331+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:38.159666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569990+0000","last_change":"2023-10-16T10:04:02.601424+0000","last_active":"2023-10-16T10:04:03.569990+0000","last_peered":"2023-10-16T10:04:03.569990+0000","last_clean":"2023-10-16T10:04:03.569990+0000","last_became_active":"2023-10-16T10:04:02.601030+0000","last_became_peered":"2023-10-16T10:04:02.601030+0000","last_unstale":"2023-10-16T10:04:03.569990+0000","last_undegraded":"2023-10-16T10:04:03.569990+0000","last_fullsized":"2023-10-16T10:04:03.569990+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:40.278099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.568975+0000","last_change":"2023-10-16T10:04:02.578748+0000","last_active":"2023-10-16T10:04:03.568975+0000","last_peered":"2023-10-16T10:04:03.568975+0000","last_clean":"2023-10-16T10:04:03.568975+0000","last_became_active":"2023-10-16T10:04:02.578453+0000","last_became_peered":"2023-10-16T10:04:02.578453+0000","last_unstale":"2023-10-16T10:04:03.568975+0000","last_undegraded":"2023-10-16T10:04:03.568975+0000","last_fullsized":"2023-10-16T10:04:03.568975+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:53.062282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.586613+0000","last_change":"2023-10-16T10:04:02.583197+0000","last_active":"2023-10-16T10:04:02.586613+0000","last_peered":"2023-10-16T10:04:02.586613+0000","last_clean":"2023-10-16T10:04:02.586613+0000","last_became_active":"2023-10-16T10:04:02.582753+0000","last_became_peered":"2023-10-16T10:04:02.582753+0000","last_unstale":"2023-10-16T10:04:02.586613+0000","last_undegraded":"2023-10-16T10:04:02.586613+0000","last_fullsized":"2023-10-16T10:04:02.586613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:47.052983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569380+0000","last_change":"2023-10-16T10:04:02.580972+0000","last_active":"2023-10-16T10:04:03.569380+0000","last_peered":"2023-10-16T10:04:03.569380+0000","last_clean":"2023-10-16T10:04:03.569380+0000","last_became_active":"2023-10-16T10:04:02.580623+0000","last_became_peered":"2023-10-16T10:04:02.580623+0000","last_unstale":"2023-10-16T10:04:03.569380+0000","last_undegraded":"2023-10-16T10:04:03.569380+0000","last_fullsized":"2023-10-16T10:04:03.569380+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:10.846791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.581487+0000","last_change":"2023-10-16T10:04:02.577488+0000","last_active":"2023-10-16T10:04:02.581487+0000","last_peered":"2023-10-16T10:04:02.581487+0000","last_clean":"2023-10-16T10:04:02.581487+0000","last_became_active":"2023-10-16T10:04:02.577223+0000","last_became_peered":"2023-10-16T10:04:02.577223+0000","last_unstale":"2023-10-16T10:04:02.581487+0000","last_undegraded":"2023-10-16T10:04:02.581487+0000","last_fullsized":"2023-10-16T10:04:02.581487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:28.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569421+0000","last_change":"2023-10-16T10:04:02.572783+0000","last_active":"2023-10-16T10:04:03.569421+0000","last_peered":"2023-10-16T10:04:03.569421+0000","last_clean":"2023-10-16T10:04:03.569421+0000","last_became_active":"2023-10-16T10:04:02.572478+0000","last_became_peered":"2023-10-16T10:04:02.572478+0000","last_unstale":"2023-10-16T10:04:03.569421+0000","last_undegraded":"2023-10-16T10:04:03.569421+0000","last_fullsized":"2023-10-16T10:04:03.569421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:12:24.129273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.598306+0000","last_change":"2023-10-16T10:04:02.590000+0000","last_active":"2023-10-16T10:04:02.598306+0000","last_peered":"2023-10-16T10:04:02.598306+0000","last_clean":"2023-10-16T10:04:02.598306+0000","last_became_active":"2023-10-16T10:04:02.589358+0000","last_became_peered":"2023-10-16T10:04:02.589358+0000","last_unstale":"2023-10-16T10:04:02.598306+0000","last_undegraded":"2023-10-16T10:04:02.598306+0000","last_fullsized":"2023-10-16T10:04:02.598306+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:26.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.573612+0000","last_change":"2023-10-16T10:04:02.573052+0000","last_active":"2023-10-16T10:04:02.573612+0000","last_peered":"2023-10-16T10:04:02.573612+0000","last_clean":"2023-10-16T10:04:02.573612+0000","last_became_active":"2023-10-16T10:04:02.572786+0000","last_became_peered":"2023-10-16T10:04:02.572786+0000","last_unstale":"2023-10-16T10:04:02.573612+0000","last_undegraded":"2023-10-16T10:04:02.573612+0000","last_fullsized":"2023-10-16T10:04:02.573612+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:11.534908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600546+0000","last_change":"2023-10-16T10:04:02.594335+0000","last_active":"2023-10-16T10:04:02.600546+0000","last_peered":"2023-10-16T10:04:02.600546+0000","last_clean":"2023-10-16T10:04:02.600546+0000","last_became_active":"2023-10-16T10:04:02.593661+0000","last_became_peered":"2023-10-16T10:04:02.593661+0000","last_unstale":"2023-10-16T10:04:02.600546+0000","last_undegraded":"2023-10-16T10:04:02.600546+0000","last_fullsized":"2023-10-16T10:04:02.600546+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:07:25.759600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.589231+0000","last_change":"2023-10-16T10:04:02.588856+0000","last_active":"2023-10-16T10:04:02.589231+0000","last_peered":"2023-10-16T10:04:02.589231+0000","last_clean":"2023-10-16T10:04:02.589231+0000","last_became_active":"2023-10-16T10:04:02.588520+0000","last_became_peered":"2023-10-16T10:04:02.588520+0000","last_unstale":"2023-10-16T10:04:02.589231+0000","last_undegraded":"2023-10-16T10:04:02.589231+0000","last_fullsized":"2023-10-16T10:04:02.589231+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:48.153915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.587990+0000","last_change":"2023-10-16T10:04:02.586644+0000","last_active":"2023-10-16T10:04:02.587990+0000","last_peered":"2023-10-16T10:04:02.587990+0000","last_clean":"2023-10-16T10:04:02.587990+0000","last_became_active":"2023-10-16T10:04:02.586364+0000","last_became_peered":"2023-10-16T10:04:02.586364+0000","last_unstale":"2023-10-16T10:04:02.587990+0000","last_undegraded":"2023-10-16T10:04:02.587990+0000","last_fullsized":"2023-10-16T10:04:02.587990+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:56:56.291708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569575+0000","last_change":"2023-10-16T10:04:02.581466+0000","last_active":"2023-10-16T10:04:03.569575+0000","last_peered":"2023-10-16T10:04:03.569575+0000","last_clean":"2023-10-16T10:04:03.569575+0000","last_became_active":"2023-10-16T10:04:02.581191+0000","last_became_peered":"2023-10-16T10:04:02.581191+0000","last_unstale":"2023-10-16T10:04:03.569575+0000","last_undegraded":"2023-10-16T10:04:03.569575+0000","last_fullsized":"2023-10-16T10:04:03.569575+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:38:05.858233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570362+0000","last_change":"2023-10-16T10:04:02.581351+0000","last_active":"2023-10-16T10:04:03.570362+0000","last_peered":"2023-10-16T10:04:03.570362+0000","last_clean":"2023-10-16T10:04:03.570362+0000","last_became_active":"2023-10-16T10:04:02.581065+0000","last_became_peered":"2023-10-16T10:04:02.581065+0000","last_unstale":"2023-10-16T10:04:03.570362+0000","last_undegraded":"2023-10-16T10:04:03.570362+0000","last_fullsized":"2023-10-16T10:04:03.570362+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:26.647712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570182+0000","last_change":"2023-10-16T10:04:02.587434+0000","last_active":"2023-10-16T10:04:03.570182+0000","last_peered":"2023-10-16T10:04:03.570182+0000","last_clean":"2023-10-16T10:04:03.570182+0000","last_became_active":"2023-10-16T10:04:02.586671+0000","last_became_peered":"2023-10-16T10:04:02.586671+0000","last_unstale":"2023-10-16T10:04:03.570182+0000","last_undegraded":"2023-10-16T10:04:03.570182+0000","last_fullsized":"2023-10-16T10:04:03.570182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:26:53.220052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570216+0000","last_change":"2023-10-16T10:04:02.601653+0000","last_active":"2023-10-16T10:04:03.570216+0000","last_peered":"2023-10-16T10:04:03.570216+0000","last_clean":"2023-10-16T10:04:03.570216+0000","last_became_active":"2023-10-16T10:04:02.601068+0000","last_became_peered":"2023-10-16T10:04:02.601068+0000","last_unstale":"2023-10-16T10:04:03.570216+0000","last_undegraded":"2023-10-16T10:04:03.570216+0000","last_fullsized":"2023-10-16T10:04:03.570216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:59.436835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570180+0000","last_change":"2023-10-16T10:04:02.596511+0000","last_active":"2023-10-16T10:04:03.570180+0000","last_peered":"2023-10-16T10:04:03.570180+0000","last_clean":"2023-10-16T10:04:03.570180+0000","last_became_active":"2023-10-16T10:04:02.596205+0000","last_became_peered":"2023-10-16T10:04:02.596205+0000","last_unstale":"2023-10-16T10:04:03.570180+0000","last_undegraded":"2023-10-16T10:04:03.570180+0000","last_fullsized":"2023-10-16T10:04:03.570180+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:59.614714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.597723+0000","last_change":"2023-10-16T10:04:02.591025+0000","last_active":"2023-10-16T10:04:02.597723+0000","last_peered":"2023-10-16T10:04:02.597723+0000","last_clean":"2023-10-16T10:04:02.597723+0000","last_became_active":"2023-10-16T10:04:02.589290+0000","last_became_peered":"2023-10-16T10:04:02.589290+0000","last_unstale":"2023-10-16T10:04:02.597723+0000","last_undegraded":"2023-10-16T10:04:02.597723+0000","last_fullsized":"2023-10-16T10:04:02.597723+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:07.783459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580188+0000","last_change":"2023-10-16T10:04:02.577386+0000","last_active":"2023-10-16T10:04:02.580188+0000","last_peered":"2023-10-16T10:04:02.580188+0000","last_clean":"2023-10-16T10:04:02.580188+0000","last_became_active":"2023-10-16T10:04:02.577118+0000","last_became_peered":"2023-10-16T10:04:02.577118+0000","last_unstale":"2023-10-16T10:04:02.580188+0000","last_undegraded":"2023-10-16T10:04:02.580188+0000","last_fullsized":"2023-10-16T10:04:02.580188+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:51:30.341763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569555+0000","last_change":"2023-10-16T10:04:02.586215+0000","last_active":"2023-10-16T10:04:03.569555+0000","last_peered":"2023-10-16T10:04:03.569555+0000","last_clean":"2023-10-16T10:04:03.569555+0000","last_became_active":"2023-10-16T10:04:02.585929+0000","last_became_peered":"2023-10-16T10:04:02.585929+0000","last_unstale":"2023-10-16T10:04:03.569555+0000","last_undegraded":"2023-10-16T10:04:03.569555+0000","last_fullsized":"2023-10-16T10:04:03.569555+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:53:49.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"144'4","reported_seq":15,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600836+0000","last_change":"2023-10-16T10:04:01.572025+0000","last_active":"2023-10-16T10:04:02.600836+0000","last_peered":"2023-10-16T10:04:02.600836+0000","last_clean":"2023-10-16T10:04:02.600836+0000","last_became_active":"2023-10-16T10:04:01.571745+0000","last_became_peered":"2023-10-16T10:04:01.571745+0000","last_unstale":"2023-10-16T10:04:02.600836+0000","last_undegraded":"2023-10-16T10:04:02.600836+0000","last_fullsized":"2023-10-16T10:04:02.600836+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:58.997435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600518+0000","last_change":"2023-10-16T10:04:02.591439+0000","last_active":"2023-10-16T10:04:02.600518+0000","last_peered":"2023-10-16T10:04:02.600518+0000","last_clean":"2023-10-16T10:04:02.600518+0000","last_became_active":"2023-10-16T10:04:02.590017+0000","last_became_peered":"2023-10-16T10:04:02.590017+0000","last_unstale":"2023-10-16T10:04:02.600518+0000","last_undegraded":"2023-10-16T10:04:02.600518+0000","last_fullsized":"2023-10-16T10:04:02.600518+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:21:21.684230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"144'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570401+0000","last_change":"2023-10-16T10:04:01.574227+0000","last_active":"2023-10-16T10:04:03.570401+0000","last_peered":"2023-10-16T10:04:03.570401+0000","last_clean":"2023-10-16T10:04:03.570401+0000","last_became_active":"2023-10-16T10:04:01.573945+0000","last_became_peered":"2023-10-16T10:04:01.573945+0000","last_unstale":"2023-10-16T10:04:03.570401+0000","last_undegraded":"2023-10-16T10:04:03.570401+0000","last_fullsized":"2023-10-16T10:04:03.570401+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:06:23.228033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596695+0000","last_change":"2023-10-16T10:04:02.588489+0000","last_active":"2023-10-16T10:04:02.596695+0000","last_peered":"2023-10-16T10:04:02.596695+0000","last_clean":"2023-10-16T10:04:02.596695+0000","last_became_active":"2023-10-16T10:04:02.588205+0000","last_became_peered":"2023-10-16T10:04:02.588205+0000","last_unstale":"2023-10-16T10:04:02.596695+0000","last_undegraded":"2023-10-16T10:04:02.596695+0000","last_fullsized":"2023-10-16T10:04:02.596695+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:31.566338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602658+0000","last_change":"2023-10-16T10:04:01.571927+0000","last_active":"2023-10-16T10:04:02.602658+0000","last_peered":"2023-10-16T10:04:02.602658+0000","last_clean":"2023-10-16T10:04:02.602658+0000","last_became_active":"2023-10-16T10:04:01.571650+0000","last_became_peered":"2023-10-16T10:04:01.571650+0000","last_unstale":"2023-10-16T10:04:02.602658+0000","last_undegraded":"2023-10-16T10:04:02.602658+0000","last_fullsized":"2023-10-16T10:04:02.602658+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:10.211904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.594244+0000","last_change":"2023-10-16T10:04:02.585094+0000","last_active":"2023-10-16T10:04:02.594244+0000","last_peered":"2023-10-16T10:04:02.594244+0000","last_clean":"2023-10-16T10:04:02.594244+0000","last_became_active":"2023-10-16T10:04:02.583916+0000","last_became_peered":"2023-10-16T10:04:02.583916+0000","last_unstale":"2023-10-16T10:04:02.594244+0000","last_undegraded":"2023-10-16T10:04:02.594244+0000","last_fullsized":"2023-10-16T10:04:02.594244+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:40:18.435506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602233+0000","last_change":"2023-10-16T10:04:01.574345+0000","last_active":"2023-10-16T10:04:02.602233+0000","last_peered":"2023-10-16T10:04:02.602233+0000","last_clean":"2023-10-16T10:04:02.602233+0000","last_became_active":"2023-10-16T10:04:01.574016+0000","last_became_peered":"2023-10-16T10:04:01.574016+0000","last_unstale":"2023-10-16T10:04:02.602233+0000","last_undegraded":"2023-10-16T10:04:02.602233+0000","last_fullsized":"2023-10-16T10:04:02.602233+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:48.534444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569374+0000","last_change":"2023-10-16T10:04:02.579189+0000","last_active":"2023-10-16T10:04:03.569374+0000","last_peered":"2023-10-16T10:04:03.569374+0000","last_clean":"2023-10-16T10:04:03.569374+0000","last_became_active":"2023-10-16T10:04:02.578892+0000","last_became_peered":"2023-10-16T10:04:02.578892+0000","last_unstale":"2023-10-16T10:04:03.569374+0000","last_undegraded":"2023-10-16T10:04:03.569374+0000","last_fullsized":"2023-10-16T10:04:03.569374+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:57:46.029828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"144'1","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.595802+0000","last_change":"2023-10-16T10:04:01.569217+0000","last_active":"2023-10-16T10:04:02.595802+0000","last_peered":"2023-10-16T10:04:02.595802+0000","last_clean":"2023-10-16T10:04:02.595802+0000","last_became_active":"2023-10-16T10:04:01.568809+0000","last_became_peered":"2023-10-16T10:04:01.568809+0000","last_unstale":"2023-10-16T10:04:02.595802+0000","last_undegraded":"2023-10-16T10:04:02.595802+0000","last_fullsized":"2023-10-16T10:04:02.595802+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:42:39.273423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.586491+0000","last_change":"2023-10-16T10:04:02.582996+0000","last_active":"2023-10-16T10:04:02.586491+0000","last_peered":"2023-10-16T10:04:02.586491+0000","last_clean":"2023-10-16T10:04:02.586491+0000","last_became_active":"2023-10-16T10:04:02.582587+0000","last_became_peered":"2023-10-16T10:04:02.582587+0000","last_unstale":"2023-10-16T10:04:02.586491+0000","last_undegraded":"2023-10-16T10:04:02.586491+0000","last_fullsized":"2023-10-16T10:04:02.586491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:21.658733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"144'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569128+0000","last_change":"2023-10-16T10:04:01.572373+0000","last_active":"2023-10-16T10:04:03.569128+0000","last_peered":"2023-10-16T10:04:03.569128+0000","last_clean":"2023-10-16T10:04:03.569128+0000","last_became_active":"2023-10-16T10:04:01.571927+0000","last_became_peered":"2023-10-16T10:04:01.571927+0000","last_unstale":"2023-10-16T10:04:03.569128+0000","last_undegraded":"2023-10-16T10:04:03.569128+0000","last_fullsized":"2023-10-16T10:04:03.569128+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:15.857118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600634+0000","last_change":"2023-10-16T10:04:02.594497+0000","last_active":"2023-10-16T10:04:02.600634+0000","last_peered":"2023-10-16T10:04:02.600634+0000","last_clean":"2023-10-16T10:04:02.600634+0000","last_became_active":"2023-10-16T10:04:02.594219+0000","last_became_peered":"2023-10-16T10:04:02.594219+0000","last_unstale":"2023-10-16T10:04:02.600634+0000","last_undegraded":"2023-10-16T10:04:02.600634+0000","last_fullsized":"2023-10-16T10:04:02.600634+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:15:38.385014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.573045+0000","last_change":"2023-10-16T10:04:01.563983+0000","last_active":"2023-10-16T10:04:02.573045+0000","last_peered":"2023-10-16T10:04:02.573045+0000","last_clean":"2023-10-16T10:04:02.573045+0000","last_became_active":"2023-10-16T10:04:01.563678+0000","last_became_peered":"2023-10-16T10:04:01.563678+0000","last_unstale":"2023-10-16T10:04:02.573045+0000","last_undegraded":"2023-10-16T10:04:02.573045+0000","last_fullsized":"2023-10-16T10:04:02.573045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:14.618053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.582416+0000","last_change":"2023-10-16T10:04:02.579387+0000","last_active":"2023-10-16T10:04:02.582416+0000","last_peered":"2023-10-16T10:04:02.582416+0000","last_clean":"2023-10-16T10:04:02.582416+0000","last_became_active":"2023-10-16T10:04:02.579135+0000","last_became_peered":"2023-10-16T10:04:02.579135+0000","last_unstale":"2023-10-16T10:04:02.582416+0000","last_undegraded":"2023-10-16T10:04:02.582416+0000","last_fullsized":"2023-10-16T10:04:02.582416+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:59.436835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567867+0000","last_change":"2023-10-16T10:04:01.570323+0000","last_active":"2023-10-16T10:04:02.567867+0000","last_peered":"2023-10-16T10:04:02.567867+0000","last_clean":"2023-10-16T10:04:02.567867+0000","last_became_active":"2023-10-16T10:04:01.570060+0000","last_became_peered":"2023-10-16T10:04:01.570060+0000","last_unstale":"2023-10-16T10:04:02.567867+0000","last_undegraded":"2023-10-16T10:04:02.567867+0000","last_fullsized":"2023-10-16T10:04:02.567867+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:04:04.919764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570474+0000","last_change":"2023-10-16T10:04:02.583940+0000","last_active":"2023-10-16T10:04:03.570474+0000","last_peered":"2023-10-16T10:04:03.570474+0000","last_clean":"2023-10-16T10:04:03.570474+0000","last_became_active":"2023-10-16T10:04:02.583689+0000","last_became_peered":"2023-10-16T10:04:02.583689+0000","last_unstale":"2023-10-16T10:04:03.570474+0000","last_undegraded":"2023-10-16T10:04:03.570474+0000","last_fullsized":"2023-10-16T10:04:03.570474+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:51:12.145942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570681+0000","last_change":"2023-10-16T10:04:01.565644+0000","last_active":"2023-10-16T10:04:03.570681+0000","last_peered":"2023-10-16T10:04:03.570681+0000","last_clean":"2023-10-16T10:04:03.570681+0000","last_became_active":"2023-10-16T10:04:01.565251+0000","last_became_peered":"2023-10-16T10:04:01.565251+0000","last_unstale":"2023-10-16T10:04:03.570681+0000","last_undegraded":"2023-10-16T10:04:03.570681+0000","last_fullsized":"2023-10-16T10:04:03.570681+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:30.737019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570502+0000","last_change":"2023-10-16T10:04:02.593619+0000","last_active":"2023-10-16T10:04:03.570502+0000","last_peered":"2023-10-16T10:04:03.570502+0000","last_clean":"2023-10-16T10:04:03.570502+0000","last_became_active":"2023-10-16T10:04:02.593349+0000","last_became_peered":"2023-10-16T10:04:02.593349+0000","last_unstale":"2023-10-16T10:04:03.570502+0000","last_undegraded":"2023-10-16T10:04:03.570502+0000","last_fullsized":"2023-10-16T10:04:03.570502+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:19.058639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569643+0000","last_change":"2023-10-16T10:04:01.565268+0000","last_active":"2023-10-16T10:04:03.569643+0000","last_peered":"2023-10-16T10:04:03.569643+0000","last_clean":"2023-10-16T10:04:03.569643+0000","last_became_active":"2023-10-16T10:04:01.563155+0000","last_became_peered":"2023-10-16T10:04:01.563155+0000","last_unstale":"2023-10-16T10:04:03.569643+0000","last_undegraded":"2023-10-16T10:04:03.569643+0000","last_fullsized":"2023-10-16T10:04:03.569643+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:11.671806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.594220+0000","last_change":"2023-10-16T10:04:02.584773+0000","last_active":"2023-10-16T10:04:02.594220+0000","last_peered":"2023-10-16T10:04:02.594220+0000","last_clean":"2023-10-16T10:04:02.594220+0000","last_became_active":"2023-10-16T10:04:02.584475+0000","last_became_peered":"2023-10-16T10:04:02.584475+0000","last_unstale":"2023-10-16T10:04:02.594220+0000","last_undegraded":"2023-10-16T10:04:02.594220+0000","last_fullsized":"2023-10-16T10:04:02.594220+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:29:22.042904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"144'3","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570131+0000","last_change":"2023-10-16T10:04:01.559977+0000","last_active":"2023-10-16T10:04:03.570131+0000","last_peered":"2023-10-16T10:04:03.570131+0000","last_clean":"2023-10-16T10:04:03.570131+0000","last_became_active":"2023-10-16T10:04:01.559666+0000","last_became_peered":"2023-10-16T10:04:01.559666+0000","last_unstale":"2023-10-16T10:04:03.570131+0000","last_undegraded":"2023-10-16T10:04:03.570131+0000","last_fullsized":"2023-10-16T10:04:03.570131+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:47.049742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.599787+0000","last_change":"2023-10-16T10:04:02.598274+0000","last_active":"2023-10-16T10:04:02.599787+0000","last_peered":"2023-10-16T10:04:02.599787+0000","last_clean":"2023-10-16T10:04:02.599787+0000","last_became_active":"2023-10-16T10:04:02.597438+0000","last_became_peered":"2023-10-16T10:04:02.597438+0000","last_unstale":"2023-10-16T10:04:02.599787+0000","last_undegraded":"2023-10-16T10:04:02.599787+0000","last_fullsized":"2023-10-16T10:04:02.599787+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:21.548633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"144'6","reported_seq":17,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.605524+0000","last_change":"2023-10-16T10:04:01.565485+0000","last_active":"2023-10-16T10:04:02.605524+0000","last_peered":"2023-10-16T10:04:02.605524+0000","last_clean":"2023-10-16T10:04:02.605524+0000","last_became_active":"2023-10-16T10:04:01.565202+0000","last_became_peered":"2023-10-16T10:04:01.565202+0000","last_unstale":"2023-10-16T10:04:02.605524+0000","last_undegraded":"2023-10-16T10:04:02.605524+0000","last_fullsized":"2023-10-16T10:04:02.605524+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:02:22.727966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570758+0000","last_change":"2023-10-16T10:04:02.583014+0000","last_active":"2023-10-16T10:04:03.570758+0000","last_peered":"2023-10-16T10:04:03.570758+0000","last_clean":"2023-10-16T10:04:03.570758+0000","last_became_active":"2023-10-16T10:04:02.582696+0000","last_became_peered":"2023-10-16T10:04:02.582696+0000","last_unstale":"2023-10-16T10:04:03.570758+0000","last_undegraded":"2023-10-16T10:04:03.570758+0000","last_fullsized":"2023-10-16T10:04:03.570758+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:10.861091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570218+0000","last_change":"2023-10-16T10:04:01.569192+0000","last_active":"2023-10-16T10:04:03.570218+0000","last_peered":"2023-10-16T10:04:03.570218+0000","last_clean":"2023-10-16T10:04:03.570218+0000","last_became_active":"2023-10-16T10:04:01.568893+0000","last_became_peered":"2023-10-16T10:04:01.568893+0000","last_unstale":"2023-10-16T10:04:03.570218+0000","last_undegraded":"2023-10-16T10:04:03.570218+0000","last_fullsized":"2023-10-16T10:04:03.570218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:42.540059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.602689+0000","last_change":"2023-10-16T10:04:02.602370+0000","last_active":"2023-10-16T10:04:02.602689+0000","last_peered":"2023-10-16T10:04:02.602689+0000","last_clean":"2023-10-16T10:04:02.602689+0000","last_became_active":"2023-10-16T10:04:02.601714+0000","last_became_peered":"2023-10-16T10:04:02.601714+0000","last_unstale":"2023-10-16T10:04:02.602689+0000","last_undegraded":"2023-10-16T10:04:02.602689+0000","last_fullsized":"2023-10-16T10:04:02.602689+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:59.614714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567733+0000","last_change":"2023-10-16T10:04:01.563990+0000","last_active":"2023-10-16T10:04:02.567733+0000","last_peered":"2023-10-16T10:04:02.567733+0000","last_clean":"2023-10-16T10:04:02.567733+0000","last_became_active":"2023-10-16T10:04:01.563729+0000","last_became_peered":"2023-10-16T10:04:01.563729+0000","last_unstale":"2023-10-16T10:04:02.567733+0000","last_undegraded":"2023-10-16T10:04:02.567733+0000","last_fullsized":"2023-10-16T10:04:02.567733+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:09:50.991166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600617+0000","last_change":"2023-10-16T10:04:02.594492+0000","last_active":"2023-10-16T10:04:02.600617+0000","last_peered":"2023-10-16T10:04:02.600617+0000","last_clean":"2023-10-16T10:04:02.600617+0000","last_became_active":"2023-10-16T10:04:02.593952+0000","last_became_peered":"2023-10-16T10:04:02.593952+0000","last_unstale":"2023-10-16T10:04:02.600617+0000","last_undegraded":"2023-10-16T10:04:02.600617+0000","last_fullsized":"2023-10-16T10:04:02.600617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:53:04.014734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"144'8","reported_seq":19,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.603667+0000","last_change":"2023-10-16T10:04:01.566320+0000","last_active":"2023-10-16T10:04:02.603667+0000","last_peered":"2023-10-16T10:04:02.603667+0000","last_clean":"2023-10-16T10:04:02.603667+0000","last_became_active":"2023-10-16T10:04:01.566038+0000","last_became_peered":"2023-10-16T10:04:01.566038+0000","last_unstale":"2023-10-16T10:04:02.603667+0000","last_undegraded":"2023-10-16T10:04:02.603667+0000","last_fullsized":"2023-10-16T10:04:02.603667+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:11.532626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596513+0000","last_change":"2023-10-16T10:04:02.590480+0000","last_active":"2023-10-16T10:04:02.596513+0000","last_peered":"2023-10-16T10:04:02.596513+0000","last_clean":"2023-10-16T10:04:02.596513+0000","last_became_active":"2023-10-16T10:04:02.590172+0000","last_became_peered":"2023-10-16T10:04:02.590172+0000","last_unstale":"2023-10-16T10:04:02.596513+0000","last_undegraded":"2023-10-16T10:04:02.596513+0000","last_fullsized":"2023-10-16T10:04:02.596513+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:47:11.499998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"144'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570781+0000","last_change":"2023-10-16T10:04:01.558524+0000","last_active":"2023-10-16T10:04:03.570781+0000","last_peered":"2023-10-16T10:04:03.570781+0000","last_clean":"2023-10-16T10:04:03.570781+0000","last_became_active":"2023-10-16T10:04:01.558229+0000","last_became_peered":"2023-10-16T10:04:01.558229+0000","last_unstale":"2023-10-16T10:04:03.570781+0000","last_undegraded":"2023-10-16T10:04:03.570781+0000","last_fullsized":"2023-10-16T10:04:03.570781+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:03:28.626659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570629+0000","last_change":"2023-10-16T10:04:02.592746+0000","last_active":"2023-10-16T10:04:03.570629+0000","last_peered":"2023-10-16T10:04:03.570629+0000","last_clean":"2023-10-16T10:04:03.570629+0000","last_became_active":"2023-10-16T10:04:02.592471+0000","last_became_peered":"2023-10-16T10:04:02.592471+0000","last_unstale":"2023-10-16T10:04:03.570629+0000","last_undegraded":"2023-10-16T10:04:03.570629+0000","last_fullsized":"2023-10-16T10:04:03.570629+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:03.818818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.603468+0000","last_change":"2023-10-16T10:04:01.573876+0000","last_active":"2023-10-16T10:04:02.603468+0000","last_peered":"2023-10-16T10:04:02.603468+0000","last_clean":"2023-10-16T10:04:02.603468+0000","last_became_active":"2023-10-16T10:04:01.573519+0000","last_became_peered":"2023-10-16T10:04:01.573519+0000","last_unstale":"2023-10-16T10:04:02.603468+0000","last_undegraded":"2023-10-16T10:04:02.603468+0000","last_fullsized":"2023-10-16T10:04:02.603468+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:39:35.652882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569440+0000","last_change":"2023-10-16T10:04:02.579600+0000","last_active":"2023-10-16T10:04:03.569440+0000","last_peered":"2023-10-16T10:04:03.569440+0000","last_clean":"2023-10-16T10:04:03.569440+0000","last_became_active":"2023-10-16T10:04:02.579271+0000","last_became_peered":"2023-10-16T10:04:02.579271+0000","last_unstale":"2023-10-16T10:04:03.569440+0000","last_undegraded":"2023-10-16T10:04:03.569440+0000","last_fullsized":"2023-10-16T10:04:03.569440+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:52:37.043627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"144'1","reported_seq":12,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.599738+0000","last_change":"2023-10-16T10:04:01.571111+0000","last_active":"2023-10-16T10:04:02.599738+0000","last_peered":"2023-10-16T10:04:02.599738+0000","last_clean":"2023-10-16T10:04:02.599738+0000","last_became_active":"2023-10-16T10:04:01.570735+0000","last_became_peered":"2023-10-16T10:04:01.570735+0000","last_unstale":"2023-10-16T10:04:02.599738+0000","last_undegraded":"2023-10-16T10:04:02.599738+0000","last_fullsized":"2023-10-16T10:04:02.599738+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:32:20.936832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.582301+0000","last_change":"2023-10-16T10:04:02.580443+0000","last_active":"2023-10-16T10:04:02.582301+0000","last_peered":"2023-10-16T10:04:02.582301+0000","last_clean":"2023-10-16T10:04:02.582301+0000","last_became_active":"2023-10-16T10:04:02.579622+0000","last_became_peered":"2023-10-16T10:04:02.579622+0000","last_unstale":"2023-10-16T10:04:02.582301+0000","last_undegraded":"2023-10-16T10:04:02.582301+0000","last_fullsized":"2023-10-16T10:04:02.582301+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:54:10.861091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570171+0000","last_change":"2023-10-16T10:04:01.574381+0000","last_active":"2023-10-16T10:04:03.570171+0000","last_peered":"2023-10-16T10:04:03.570171+0000","last_clean":"2023-10-16T10:04:03.570171+0000","last_became_active":"2023-10-16T10:04:01.574038+0000","last_became_peered":"2023-10-16T10:04:01.574038+0000","last_unstale":"2023-10-16T10:04:03.570171+0000","last_undegraded":"2023-10-16T10:04:03.570171+0000","last_fullsized":"2023-10-16T10:04:03.570171+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:13.676699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570132+0000","last_change":"2023-10-16T10:04:02.581856+0000","last_active":"2023-10-16T10:04:03.570132+0000","last_peered":"2023-10-16T10:04:03.570132+0000","last_clean":"2023-10-16T10:04:03.570132+0000","last_became_active":"2023-10-16T10:04:02.581545+0000","last_became_peered":"2023-10-16T10:04:02.581545+0000","last_unstale":"2023-10-16T10:04:03.570132+0000","last_undegraded":"2023-10-16T10:04:03.570132+0000","last_fullsized":"2023-10-16T10:04:03.570132+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:38.159666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"144'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570341+0000","last_change":"2023-10-16T10:04:01.571931+0000","last_active":"2023-10-16T10:04:03.570341+0000","last_peered":"2023-10-16T10:04:03.570341+0000","last_clean":"2023-10-16T10:04:03.570341+0000","last_became_active":"2023-10-16T10:04:01.571610+0000","last_became_peered":"2023-10-16T10:04:01.571610+0000","last_unstale":"2023-10-16T10:04:03.570341+0000","last_undegraded":"2023-10-16T10:04:03.570341+0000","last_fullsized":"2023-10-16T10:04:03.570341+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:52.006116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.599949+0000","last_change":"2023-10-16T10:04:02.599276+0000","last_active":"2023-10-16T10:04:02.599949+0000","last_peered":"2023-10-16T10:04:02.599949+0000","last_clean":"2023-10-16T10:04:02.599949+0000","last_became_active":"2023-10-16T10:04:02.599015+0000","last_became_peered":"2023-10-16T10:04:02.599015+0000","last_unstale":"2023-10-16T10:04:02.599949+0000","last_undegraded":"2023-10-16T10:04:02.599949+0000","last_fullsized":"2023-10-16T10:04:02.599949+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:50:29.891777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569209+0000","last_change":"2023-10-16T10:04:01.563456+0000","last_active":"2023-10-16T10:04:03.569209+0000","last_peered":"2023-10-16T10:04:03.569209+0000","last_clean":"2023-10-16T10:04:03.569209+0000","last_became_active":"2023-10-16T10:04:01.563166+0000","last_became_peered":"2023-10-16T10:04:01.563166+0000","last_unstale":"2023-10-16T10:04:03.569209+0000","last_undegraded":"2023-10-16T10:04:03.569209+0000","last_fullsized":"2023-10-16T10:04:03.569209+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:54.583503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.575538+0000","last_change":"2023-10-16T10:04:02.573255+0000","last_active":"2023-10-16T10:04:02.575538+0000","last_peered":"2023-10-16T10:04:02.575538+0000","last_clean":"2023-10-16T10:04:02.575538+0000","last_became_active":"2023-10-16T10:04:02.572998+0000","last_became_peered":"2023-10-16T10:04:02.572998+0000","last_unstale":"2023-10-16T10:04:02.575538+0000","last_undegraded":"2023-10-16T10:04:02.575538+0000","last_fullsized":"2023-10-16T10:04:02.575538+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:12.537301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.568926+0000","last_change":"2023-10-16T10:04:01.558099+0000","last_active":"2023-10-16T10:04:03.568926+0000","last_peered":"2023-10-16T10:04:03.568926+0000","last_clean":"2023-10-16T10:04:03.568926+0000","last_became_active":"2023-10-16T10:04:01.557452+0000","last_became_peered":"2023-10-16T10:04:01.557452+0000","last_unstale":"2023-10-16T10:04:03.568926+0000","last_undegraded":"2023-10-16T10:04:03.568926+0000","last_fullsized":"2023-10-16T10:04:03.568926+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:07.596628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.597745+0000","last_change":"2023-10-16T10:04:02.591073+0000","last_active":"2023-10-16T10:04:02.597745+0000","last_peered":"2023-10-16T10:04:02.597745+0000","last_clean":"2023-10-16T10:04:02.597745+0000","last_became_active":"2023-10-16T10:04:02.589611+0000","last_became_peered":"2023-10-16T10:04:02.589611+0000","last_unstale":"2023-10-16T10:04:02.597745+0000","last_undegraded":"2023-10-16T10:04:02.597745+0000","last_fullsized":"2023-10-16T10:04:02.597745+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:25:08.601304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.571103+0000","last_change":"2023-10-16T10:04:01.573306+0000","last_active":"2023-10-16T10:04:02.571103+0000","last_peered":"2023-10-16T10:04:02.571103+0000","last_clean":"2023-10-16T10:04:02.571103+0000","last_became_active":"2023-10-16T10:04:01.573013+0000","last_became_peered":"2023-10-16T10:04:01.573013+0000","last_unstale":"2023-10-16T10:04:02.571103+0000","last_undegraded":"2023-10-16T10:04:02.571103+0000","last_fullsized":"2023-10-16T10:04:02.571103+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:29.820650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.593396+0000","last_change":"2023-10-16T10:04:02.587800+0000","last_active":"2023-10-16T10:04:02.593396+0000","last_peered":"2023-10-16T10:04:02.593396+0000","last_clean":"2023-10-16T10:04:02.593396+0000","last_became_active":"2023-10-16T10:04:02.587519+0000","last_became_peered":"2023-10-16T10:04:02.587519+0000","last_unstale":"2023-10-16T10:04:02.593396+0000","last_undegraded":"2023-10-16T10:04:02.593396+0000","last_fullsized":"2023-10-16T10:04:02.593396+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:02.256227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"144'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570627+0000","last_change":"2023-10-16T10:04:01.570740+0000","last_active":"2023-10-16T10:04:03.570627+0000","last_peered":"2023-10-16T10:04:03.570627+0000","last_clean":"2023-10-16T10:04:03.570627+0000","last_became_active":"2023-10-16T10:04:01.570244+0000","last_became_peered":"2023-10-16T10:04:01.570244+0000","last_unstale":"2023-10-16T10:04:03.570627+0000","last_undegraded":"2023-10-16T10:04:03.570627+0000","last_fullsized":"2023-10-16T10:04:03.570627+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:20:49.018718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570603+0000","last_change":"2023-10-16T10:04:02.597569+0000","last_active":"2023-10-16T10:04:03.570603+0000","last_peered":"2023-10-16T10:04:03.570603+0000","last_clean":"2023-10-16T10:04:03.570603+0000","last_became_active":"2023-10-16T10:04:02.597303+0000","last_became_peered":"2023-10-16T10:04:02.597303+0000","last_unstale":"2023-10-16T10:04:03.570603+0000","last_undegraded":"2023-10-16T10:04:03.570603+0000","last_fullsized":"2023-10-16T10:04:03.570603+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:15.562192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.579504+0000","last_change":"2023-10-16T10:04:01.564207+0000","last_active":"2023-10-16T10:04:02.579504+0000","last_peered":"2023-10-16T10:04:02.579504+0000","last_clean":"2023-10-16T10:04:02.579504+0000","last_became_active":"2023-10-16T10:04:01.563896+0000","last_became_peered":"2023-10-16T10:04:01.563896+0000","last_unstale":"2023-10-16T10:04:02.579504+0000","last_undegraded":"2023-10-16T10:04:02.579504+0000","last_fullsized":"2023-10-16T10:04:02.579504+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:33.202190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.585036+0000","last_change":"2023-10-16T10:04:02.584569+0000","last_active":"2023-10-16T10:04:02.585036+0000","last_peered":"2023-10-16T10:04:02.585036+0000","last_clean":"2023-10-16T10:04:02.585036+0000","last_became_active":"2023-10-16T10:04:02.584243+0000","last_became_peered":"2023-10-16T10:04:02.584243+0000","last_unstale":"2023-10-16T10:04:02.585036+0000","last_undegraded":"2023-10-16T10:04:02.585036+0000","last_fullsized":"2023-10-16T10:04:02.585036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:20.554648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567899+0000","last_change":"2023-10-16T10:04:01.569580+0000","last_active":"2023-10-16T10:04:02.567899+0000","last_peered":"2023-10-16T10:04:02.567899+0000","last_clean":"2023-10-16T10:04:02.567899+0000","last_became_active":"2023-10-16T10:04:01.569301+0000","last_became_peered":"2023-10-16T10:04:01.569301+0000","last_unstale":"2023-10-16T10:04:02.567899+0000","last_undegraded":"2023-10-16T10:04:02.567899+0000","last_fullsized":"2023-10-16T10:04:02.567899+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:11:25.643036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.581875+0000","last_change":"2023-10-16T10:04:02.579642+0000","last_active":"2023-10-16T10:04:02.581875+0000","last_peered":"2023-10-16T10:04:02.581875+0000","last_clean":"2023-10-16T10:04:02.581875+0000","last_became_active":"2023-10-16T10:04:02.579146+0000","last_became_peered":"2023-10-16T10:04:02.579146+0000","last_unstale":"2023-10-16T10:04:02.581875+0000","last_undegraded":"2023-10-16T10:04:02.581875+0000","last_fullsized":"2023-10-16T10:04:02.581875+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:18.125405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.571865+0000","last_change":"2023-10-16T10:04:01.573867+0000","last_active":"2023-10-16T10:04:02.571865+0000","last_peered":"2023-10-16T10:04:02.571865+0000","last_clean":"2023-10-16T10:04:02.571865+0000","last_became_active":"2023-10-16T10:04:01.573583+0000","last_became_peered":"2023-10-16T10:04:01.573583+0000","last_unstale":"2023-10-16T10:04:02.571865+0000","last_undegraded":"2023-10-16T10:04:02.571865+0000","last_fullsized":"2023-10-16T10:04:02.571865+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:15:20.483076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596664+0000","last_change":"2023-10-16T10:04:02.591697+0000","last_active":"2023-10-16T10:04:02.596664+0000","last_peered":"2023-10-16T10:04:02.596664+0000","last_clean":"2023-10-16T10:04:02.596664+0000","last_became_active":"2023-10-16T10:04:02.591460+0000","last_became_peered":"2023-10-16T10:04:02.591460+0000","last_unstale":"2023-10-16T10:04:02.596664+0000","last_undegraded":"2023-10-16T10:04:02.596664+0000","last_fullsized":"2023-10-16T10:04:02.596664+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:03.848893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.567941+0000","last_change":"2023-10-16T10:04:01.569004+0000","last_active":"2023-10-16T10:04:02.567941+0000","last_peered":"2023-10-16T10:04:02.567941+0000","last_clean":"2023-10-16T10:04:02.567941+0000","last_became_active":"2023-10-16T10:04:01.568471+0000","last_became_peered":"2023-10-16T10:04:01.568471+0000","last_unstale":"2023-10-16T10:04:02.567941+0000","last_undegraded":"2023-10-16T10:04:02.567941+0000","last_fullsized":"2023-10-16T10:04:02.567941+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:14.557516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.570309+0000","last_change":"2023-10-16T10:04:02.585338+0000","last_active":"2023-10-16T10:04:03.570309+0000","last_peered":"2023-10-16T10:04:03.570309+0000","last_clean":"2023-10-16T10:04:03.570309+0000","last_became_active":"2023-10-16T10:04:02.585067+0000","last_became_peered":"2023-10-16T10:04:02.585067+0000","last_unstale":"2023-10-16T10:04:03.570309+0000","last_undegraded":"2023-10-16T10:04:03.570309+0000","last_fullsized":"2023-10-16T10:04:03.570309+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:55.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.580429+0000","last_change":"2023-10-16T10:04:01.570145+0000","last_active":"2023-10-16T10:04:02.580429+0000","last_peered":"2023-10-16T10:04:02.580429+0000","last_clean":"2023-10-16T10:04:02.580429+0000","last_became_active":"2023-10-16T10:04:01.569872+0000","last_became_peered":"2023-10-16T10:04:01.569872+0000","last_unstale":"2023-10-16T10:04:02.580429+0000","last_undegraded":"2023-10-16T10:04:02.580429+0000","last_fullsized":"2023-10-16T10:04:02.580429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:51.435313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.568976+0000","last_change":"2023-10-16T10:04:02.593222+0000","last_active":"2023-10-16T10:04:03.568976+0000","last_peered":"2023-10-16T10:04:03.568976+0000","last_clean":"2023-10-16T10:04:03.568976+0000","last_became_active":"2023-10-16T10:04:02.592903+0000","last_became_peered":"2023-10-16T10:04:02.592903+0000","last_unstale":"2023-10-16T10:04:03.568976+0000","last_undegraded":"2023-10-16T10:04:03.568976+0000","last_fullsized":"2023-10-16T10:04:03.568976+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:57.459012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-16T10:04:03.569015+0000","last_change":"2023-10-16T10:04:01.563914+0000","last_active":"2023-10-16T10:04:03.569015+0000","last_peered":"2023-10-16T10:04:03.569015+0000","last_clean":"2023-10-16T10:04:03.569015+0000","last_became_active":"2023-10-16T10:04:01.563619+0000","last_became_peered":"2023-10-16T10:04:01.563619+0000","last_unstale":"2023-10-16T10:04:03.569015+0000","last_undegraded":"2023-10-16T10:04:03.569015+0000","last_fullsized":"2023-10-16T10:04:03.569015+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:30:10.342856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.592966+0000","last_change":"2023-10-16T10:04:02.592274+0000","last_active":"2023-10-16T10:04:02.592966+0000","last_peered":"2023-10-16T10:04:02.592966+0000","last_clean":"2023-10-16T10:04:02.592966+0000","last_became_active":"2023-10-16T10:04:02.591914+0000","last_became_peered":"2023-10-16T10:04:02.591914+0000","last_unstale":"2023-10-16T10:04:02.592966+0000","last_undegraded":"2023-10-16T10:04:02.592966+0000","last_fullsized":"2023-10-16T10:04:02.592966+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:50:43.544734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.571237+0000","last_change":"2023-10-16T10:04:01.570131+0000","last_active":"2023-10-16T10:04:02.571237+0000","last_peered":"2023-10-16T10:04:02.571237+0000","last_clean":"2023-10-16T10:04:02.571237+0000","last_became_active":"2023-10-16T10:04:01.569792+0000","last_became_peered":"2023-10-16T10:04:01.569792+0000","last_unstale":"2023-10-16T10:04:02.571237+0000","last_undegraded":"2023-10-16T10:04:02.571237+0000","last_fullsized":"2023-10-16T10:04:02.571237+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:35.625437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.596642+0000","last_change":"2023-10-16T10:04:02.590688+0000","last_active":"2023-10-16T10:04:02.596642+0000","last_peered":"2023-10-16T10:04:02.596642+0000","last_clean":"2023-10-16T10:04:02.596642+0000","last_became_active":"2023-10-16T10:04:02.590413+0000","last_became_peered":"2023-10-16T10:04:02.590413+0000","last_unstale":"2023-10-16T10:04:02.596642+0000","last_undegraded":"2023-10-16T10:04:02.596642+0000","last_fullsized":"2023-10-16T10:04:02.596642+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:01.547175+0000","last_clean_scrub_stamp":"2023-10-16T10:04:01.547175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:39.919315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"144'2","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-16T10:04:02.600175+0000","last_change":"2023-10-16T10:04:01.571838+0000","last_active":"2023-10-16T10:04:02.600175+0000","last_peered":"2023-10-16T10:04:02.600175+0000","last_clean":"2023-10-16T10:04:02.600175+0000","last_became_active":"2023-10-16T10:04:01.571558+0000","last_became_peered":"2023-10-16T10:04:01.571558+0000","last_unstale":"2023-10-16T10:04:02.600175+0000","last_undegraded":"2023-10-16T10:04:02.600175+0000","last_fullsized":"2023-10-16T10:04:02.600175+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T10:04:00.542500+0000","last_clean_scrub_stamp":"2023-10-16T10:04:00.542500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:42:47.369337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"135'249","reported_seq":753,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-16T10:04:01.555050+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T10:04:01.555050+0000","last_peered":"2023-10-16T10:04:01.555050+0000","last_clean":"2023-10-16T10:04:01.555050+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T10:04:01.555050+0000","last_undegraded":"2023-10-16T10:04:01.555050+0000","last_fullsized":"2023-10-16T10:04:01.555050+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542382,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33540,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338300,"statfs":{"total":96636764160,"available":96602419200,"internally_reserved":0,"allocated":2408448,"data_stored":1482597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55400000000000005,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.248},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.27200000000000002},"max":{"1min":0.627,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":1,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57199999999999995,"15min":0.58399999999999996},"min":{"1min":0.42299999999999999,"5min":0.35999999999999999,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":1.788,"15min":1.788},"last":0.59599999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.308},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.33500000000000002,"15min":0.32600000000000001},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.70799999999999996}]},{"osd":3,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.58399999999999996},"min":{"1min":0.41999999999999998,"5min":0.36499999999999999,"15min":0.35199999999999998},"max":{"1min":0.76400000000000001,"5min":0.874,"15min":0.874},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60199999999999998,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.626}]},{"osd":4,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.46600000000000003},"min":{"1min":0.33400000000000002,"5min":0.27400000000000002,"15min":0.26500000000000001},"max":{"1min":0.67500000000000004,"5min":0.76200000000000001,"15min":0.89200000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.5,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.27500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.374,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.64400000000000002}]},{"osd":6,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.56200000000000006},"min":{"1min":0.38400000000000001,"5min":0.36399999999999999,"15min":0.30399999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.373},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.308},"max":{"1min":0.80900000000000005,"5min":0.83799999999999997,"15min":1.4099999999999999},"last":0.71999999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542381,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1802240,"data_stored":891591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.26100000000000001},"max":{"1min":0.71599999999999997,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.57999999999999996},"min":{"1min":0.45100000000000001,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.83399999999999996},"last":0.55100000000000005}]},{"osd":1,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.753,"5min":0.754,"15min":0.79800000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.58799999999999997},"min":{"1min":0.41299999999999998,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.72599999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.86099999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.57899999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.84499999999999997},"last":0.501}]},{"osd":3,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.59199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77600000000000002,"5min":1.03,"15min":1.03},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.34599999999999997},"max":{"1min":0.73399999999999999,"5min":0.79800000000000004,"15min":0.85899999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.311},"max":{"1min":0.752,"5min":1,"15min":1},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53500000000000003,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.33300000000000002},"max":{"1min":0.82699999999999996,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.42799999999999999,"5min":0.36099999999999999,"15min":0.34999999999999998},"max":{"1min":0.81699999999999995,"5min":1.056,"15min":1.056},"last":0.73899999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.33200000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80000000000000004,"5min":0.83299999999999996,"15min":0.86399999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.625}]}]},{"osd":1,"up_from":14,"seq":60129542381,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50984,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320856,"statfs":{"total":96636764160,"available":96584556544,"internally_reserved":0,"allocated":1789952,"data_stored":889979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.221,"15min":0.20100000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":1.5},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.40300000000000002},"min":{"1min":0.253,"5min":0.20599999999999999,"15min":0.19600000000000001},"max":{"1min":0.58399999999999996,"5min":0.60099999999999998,"15min":0.80000000000000004},"last":0.34300000000000003}]},{"osd":2,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.36899999999999999,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":1.1040000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.436},"min":{"1min":0.307,"5min":0.25,"15min":0.25},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":3.8730000000000002},"last":0.39900000000000002}]},{"osd":3,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.436,"15min":0.46400000000000002},"min":{"1min":0.28899999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.64900000000000002,"5min":0.68000000000000005,"15min":4.6740000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.434},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.25700000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":1.379},"last":0.38200000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.61399999999999999},"min":{"1min":0.45200000000000001,"5min":0.379,"15min":0.33100000000000002},"max":{"1min":0.86899999999999999,"5min":0.96899999999999997,"15min":1.2490000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":5.4119999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61199999999999999,"15min":0.628},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.94999999999999996,"15min":1.26},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.3560000000000001},"last":0.628}]},{"osd":6,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.374},"max":{"1min":1.026,"5min":1.026,"15min":6.2140000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.48499999999999999,"5min":0.36799999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":0.92600000000000005,"15min":1.3420000000000001},"last":0.57699999999999996}]},{"osd":7,"last update":"Mon Oct 16 10:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":0.98299999999999998,"15min":1.4079999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.41899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.65900000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542382,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1480347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.23200000000000001,"5min":0.192,"15min":0.191},"max":{"1min":0.53600000000000003,"5min":0.59999999999999998,"15min":0.65400000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.374,"15min":0.39700000000000002},"min":{"1min":0.24299999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.64900000000000002},"last":0.313}]},{"osd":2,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.40500000000000003},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.626},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39300000000000002,"15min":0.41899999999999998},"min":{"1min":0.28599999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.60799999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.43099999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.72999999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65000000000000002,"5min":0.66500000000000004,"15min":0.69699999999999995},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58999999999999997,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80500000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.63},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78200000000000003,"5min":0.94599999999999995,"15min":0.98799999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61099999999999999,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.38500000000000001,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.92200000000000004,"15min":1.004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.96999999999999997},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.63400000000000001},"min":{"1min":0.48299999999999998,"5min":0.34399999999999997,"15min":0.32900000000000001},"max":{"1min":0.83999999999999997,"5min":0.92400000000000004,"15min":0.92500000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.66100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83199999999999996,"5min":0.85999999999999999,"15min":1.0089999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61799999999999999,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81200000000000006,"5min":0.876,"15min":1.0249999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.65700000000000003},"min":{"1min":0.503,"5min":0.373,"15min":0.34899999999999998},"max":{"1min":0.86099999999999999,"5min":0.90100000000000002,"15min":0.95699999999999996},"last":0.5}]}]},{"osd":2,"up_from":14,"seq":60129542381,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1810432,"data_stored":897347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36099999999999999,"15min":0.377},"min":{"1min":0.23799999999999999,"5min":0.23100000000000001,"15min":0.183},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.64900000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.379,"15min":0.39900000000000002},"min":{"1min":0.27000000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.55600000000000005,"5min":0.621,"15min":0.63400000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.40799999999999997},"min":{"1min":0.25800000000000001,"5min":0.252,"15min":0.245},"max":{"1min":0.55300000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40600000000000003,"15min":0.42299999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.214},"max":{"1min":0.622,"5min":0.622,"15min":0.64000000000000001},"last":0.32500000000000001}]},{"osd":3,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.23000000000000001},"max":{"1min":0.51800000000000002,"5min":0.59199999999999997,"15min":0.61899999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.43099999999999999,"15min":0.46500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.60599999999999998,"5min":0.63400000000000001,"15min":0.747},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.91600000000000004,"15min":0.93999999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.628},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":0.88200000000000001,"15min":1.091},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":3.4009999999999998,"5min":3.4009999999999998,"15min":4.3609999999999998},"last":0.372},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.63100000000000001},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.35099999999999998},"max":{"1min":0.872,"5min":0.92100000000000004,"15min":0.97499999999999998},"last":0.44600000000000001}]},{"osd":6,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61699999999999999,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.0720000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.624,"15min":0.64500000000000002},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.90500000000000003,"15min":0.97899999999999998},"last":0.38700000000000001}]},{"osd":7,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.4289999999999998,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.38500000000000001},"max":{"1min":0.94899999999999995,"5min":0.98799999999999999,"15min":0.998},"last":0.51000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37116,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334724,"statfs":{"total":96636764160,"available":96598757376,"internally_reserved":0,"allocated":1810432,"data_stored":896891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35999999999999999,"15min":0.38300000000000001},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.64900000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.41499999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.185},"max":{"1min":0.45700000000000002,"5min":0.626,"15min":0.63100000000000001},"last":0.33800000000000002}]},{"osd":1,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.36299999999999999,"15min":0.39200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":0.54100000000000004,"15min":0.72199999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39200000000000002,"15min":0.41899999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48099999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41099999999999998,"15min":0.433},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.26000000000000001},"max":{"1min":0.46800000000000003,"5min":0.60599999999999998,"15min":0.70999999999999996},"last":0.318},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.42899999999999999,"15min":0.44800000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54900000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.60699999999999998},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.74299999999999999,"5min":0.95699999999999996,"15min":0.95799999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.621},"min":{"1min":0.318,"5min":0.314,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.88700000000000001,"15min":1.0820000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.436,"5min":0.374,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.93700000000000006,"15min":0.98699999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60899999999999999,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.38600000000000001,"15min":0.35599999999999998},"max":{"1min":0.78600000000000003,"5min":0.94799999999999995,"15min":0.996},"last":0.624}]},{"osd":6,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61299999999999999,"15min":0.63900000000000001},"min":{"1min":0.42999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.755,"5min":0.92400000000000004,"15min":0.98399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59899999999999998,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.96799999999999997,"15min":0.99099999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60799999999999998,"15min":0.63600000000000001},"min":{"1min":0.42499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.91800000000000004,"15min":1.3},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63400000000000001,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82099999999999995,"5min":0.91800000000000004,"15min":1.4019999999999999},"last":0.52200000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542381,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59300,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312540,"statfs":{"total":96636764160,"available":96576040960,"internally_reserved":0,"allocated":1794048,"data_stored":890049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.27600000000000002,"15min":0.186},"max":{"1min":0.71799999999999997,"5min":0.73699999999999999,"15min":0.89400000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.57799999999999996},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.23899999999999999},"max":{"1min":0.745,"5min":0.78100000000000003,"15min":0.81299999999999994},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.57799999999999996},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.26300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86599999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.248},"max":{"1min":0.69499999999999995,"5min":0.77900000000000003,"15min":0.83799999999999997},"last":0.60799999999999998}]},{"osd":2,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.41699999999999998,"5min":0.34000000000000002,"15min":0.29799999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.81899999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55300000000000005,"15min":0.58999999999999997},"min":{"1min":0.41199999999999998,"5min":0.29399999999999998,"15min":0.27900000000000003},"max":{"1min":0.70799999999999996,"5min":0.72799999999999998,"15min":0.82799999999999996},"last":0.48999999999999999}]},{"osd":3,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54600000000000004,"15min":0.57699999999999996},"min":{"1min":0.41099999999999998,"5min":0.38100000000000001,"15min":0.314},"max":{"1min":0.78000000000000003,"5min":0.81200000000000006,"15min":0.83299999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59899999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.68799999999999994,"5min":0.77300000000000002,"15min":0.88300000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47399999999999998,"15min":0.48199999999999998},"min":{"1min":0.29699999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66500000000000004,"5min":0.73299999999999998,"15min":0.76800000000000002},"last":0.44},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.747,"5min":0.77800000000000002,"15min":0.80400000000000005},"last":0.502}]},{"osd":6,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.82999999999999996},"last":0.38},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.56000000000000005},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73699999999999999,"5min":0.79300000000000004,"15min":0.85899999999999999},"last":0.53800000000000003}]},{"osd":7,"last update":"Mon Oct 16 10:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52900000000000003,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.317,"15min":0.30399999999999999},"max":{"1min":0.71099999999999997,"5min":0.753,"15min":0.80700000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.58699999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.68799999999999994,"5min":0.83299999999999996,"15min":0.84599999999999997},"last":0.47499999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542382,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37112,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334728,"statfs":{"total":96636764160,"available":96598761472,"internally_reserved":0,"allocated":1810432,"data_stored":896843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69899999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55100000000000005,"15min":0.58799999999999997},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.79100000000000004,"15min":0.82799999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":0.83499999999999996,"15min":0.92300000000000004},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.32300000000000001,"15min":0.315},"max":{"1min":0.70899999999999996,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.58599999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.73999999999999999,"5min":0.87,"15min":0.88700000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.89000000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59399999999999997,"15min":0.61499999999999999},"min":{"1min":0.46000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.627}]},{"osd":4,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.32900000000000001,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78400000000000003,"5min":0.86099999999999999,"15min":0.86799999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.311},"max":{"1min":0.71999999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.36399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82999999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70199999999999996}]},{"osd":7,"last update":"Mon Oct 16 10:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.89400000000000002,"5min":1.006,"15min":1.006},"last":0.752},{"interface":"front","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.35599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.874,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T10:04:07.086 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T10:04:07.087 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T10:04:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.246+0000 7f93294b9700 1 Processor -- start 2023-10-16T10:04:07.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.246+0000 7f93294b9700 1 -- start start 2023-10-16T10:04:07.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f93294b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93240b2010 0x7f93240cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f93294b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9324071be0 0x7f93240b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f93294b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932406f3b0 0x7f932406f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f93294b9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93240ce1f0 con 0x7f93240b2010 2023-10-16T10:04:07.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f93294b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9324072090 con 0x7f9324071be0 2023-10-16T10:04:07.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f93294b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93240b2930 con 0x7f932406f3b0 2023-10-16T10:04:07.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f9322ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93240b2010 0x7f93240cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f9322ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93240b2010 0x7f93240cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49274/0 (socket says 172.21.15.181:49274) 2023-10-16T10:04:07.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.247+0000 7f9322ffd700 1 -- 172.21.15.181:0/893383321 learned_addr learned my addr 172.21.15.181:0/893383321 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:07.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9322ffd700 1 -- 172.21.15.181:0/893383321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932406f3b0 msgr2=0x7f932406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:07.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93237fe700 1 --2- 172.21.15.181:0/893383321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932406f3b0 0x7f932406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93227fc700 1 --2- 172.21.15.181:0/893383321 >> v2:172.21.15.190:3300/0 conn(0x7f9324071be0 0x7f93240b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9322ffd700 1 --2- 172.21.15.181:0/893383321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932406f3b0 0x7f932406f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9322ffd700 1 -- 172.21.15.181:0/893383321 >> v2:172.21.15.190:3300/0 conn(0x7f9324071be0 msgr2=0x7f93240b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:07.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9322ffd700 1 --2- 172.21.15.181:0/893383321 >> v2:172.21.15.190:3300/0 conn(0x7f9324071be0 0x7f93240b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9322ffd700 1 -- 172.21.15.181:0/893383321 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240ccda0 con 0x7f93240b2010 2023-10-16T10:04:07.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9322ffd700 1 --2- 172.21.15.181:0/893383321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240cc6e0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f932415e1c0 tx=0x7f931800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d1bf2c23751ec0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9321ffb700 1 -- 172.21.15.181:0/893383321 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931800dd30 con 0x7f93240b2010 2023-10-16T10:04:07.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9321ffb700 1 -- 172.21.15.181:0/893383321 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931800de90 con 0x7f93240b2010 2023-10-16T10:04:07.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f9321ffb700 1 -- 172.21.15.181:0/893383321 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318010690 con 0x7f93240b2010 2023-10-16T10:04:07.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93294b9700 1 -- 172.21.15.181:0/893383321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 msgr2=0x7f93240cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93294b9700 1 --2- 172.21.15.181:0/893383321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240cc6e0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f932415e1c0 tx=0x7f931800be70 comp rx=0 tx=0).stop 2023-10-16T10:04:07.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93294b9700 1 -- 172.21.15.181:0/893383321 shutdown_connections 2023-10-16T10:04:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93294b9700 1 --2- 172.21.15.181:0/893383321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240cc6e0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93294b9700 1 --2- 172.21.15.181:0/893383321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f932406f3b0 0x7f932406f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93294b9700 1 --2- 172.21.15.181:0/893383321 >> v2:172.21.15.190:3300/0 conn(0x7f9324071be0 0x7f93240b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.248+0000 7f93294b9700 1 -- 172.21.15.181:0/893383321 >> 172.21.15.181:0/893383321 conn(0x7f93240cd010 msgr2=0x7f93240cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.249+0000 7f93294b9700 1 -- 172.21.15.181:0/893383321 shutdown_connections 2023-10-16T10:04:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.249+0000 7f93294b9700 1 -- 172.21.15.181:0/893383321 wait complete. 2023-10-16T10:04:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.250+0000 7f93294b9700 1 Processor -- start 2023-10-16T10:04:07.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.250+0000 7f93294b9700 1 -- start start 2023-10-16T10:04:07.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f932406f3b0 0x7f93240ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9324071be0 0x7f9324152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93240acdc0 con 0x7f93240b2010 2023-10-16T10:04:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93240ca300 con 0x7f932406f3b0 2023-10-16T10:04:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9324153b60 con 0x7f9324071be0 2023-10-16T10:04:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49284/0 (socket says 172.21.15.181:49284) 2023-10-16T10:04:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 -- 172.21.15.181:0/3284359958 learned_addr learned my addr 172.21.15.181:0/3284359958 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93227fc700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9324071be0 0x7f9324152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f9322ffd700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f932406f3b0 0x7f93240ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 -- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9324071be0 msgr2=0x7f9324152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9324071be0 0x7f9324152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 -- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f932406f3b0 msgr2=0x7f93240ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f932406f3b0 0x7f93240ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 -- 172.21.15.181:0/3284359958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931800b7f0 con 0x7f93240b2010 2023-10-16T10:04:07.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93237fe700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240c7660 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f931c00cb80 tx=0x7f931c00ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f930b7fe700 1 -- 172.21.15.181:0/3284359958 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00a820 con 0x7f93240b2010 2023-10-16T10:04:07.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f930b7fe700 1 -- 172.21.15.181:0/3284359958 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c00a980 con 0x7f93240b2010 2023-10-16T10:04:07.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9324153e40 con 0x7f93240b2010 2023-10-16T10:04:07.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f930b7fe700 1 -- 172.21.15.181:0/3284359958 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00ad70 con 0x7f93240b2010 2023-10-16T10:04:07.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.251+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9324154400 con 0x7f93240b2010 2023-10-16T10:04:07.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.252+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93240b6510 con 0x7f93240b2010 2023-10-16T10:04:07.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.253+0000 7f930b7fe700 1 -- 172.21.15.181:0/3284359958 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f931c011410 con 0x7f93240b2010 2023-10-16T10:04:07.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.253+0000 7f930b7fe700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f931006b020 0x7f931006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.253+0000 7f930b7fe700 1 -- 172.21.15.181:0/3284359958 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f931c0894b0 con 0x7f93240b2010 2023-10-16T10:04:07.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.253+0000 7f9322ffd700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f931006b020 0x7f931006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.254+0000 7f9322ffd700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f931006b020 0x7f931006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f931800beb0 tx=0x7f931801c040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.258+0000 7f930b7fe700 1 -- 172.21.15.181:0/3284359958 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f931c054200 con 0x7f93240b2010 2023-10-16T10:04:07.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.526+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f93240b0900 con 0x7f93240b2010 2023-10-16T10:04:07.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.528+0000 7f930b7fe700 1 -- 172.21.15.181:0/3284359958 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7f931c057820 con 0x7f93240b2010 2023-10-16T10:04:07.529 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T10:04:07.529 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T10:04:07.529 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T10:04:07.530 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T10:04:07.530 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T10:04:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f931006b020 msgr2=0x7f931006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f931006b020 0x7f931006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f931800beb0 tx=0x7f931801c040 comp rx=0 tx=0).stop 2023-10-16T10:04:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 msgr2=0x7f93240c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240c7660 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f931c00cb80 tx=0x7f931c00ce90 comp rx=0 tx=0).stop 2023-10-16T10:04:07.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 shutdown_connections 2023-10-16T10:04:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7f931006b020 0x7f931006d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93240b2010 0x7f93240c7660 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f932406f3b0 0x7f93240ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 --2- 172.21.15.181:0/3284359958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9324071be0 0x7f9324152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 >> 172.21.15.181:0/3284359958 conn(0x7f93240cd010 msgr2=0x7f93240affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 shutdown_connections 2023-10-16T10:04:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.531+0000 7f93294b9700 1 -- 172.21.15.181:0/3284359958 wait complete. 2023-10-16T10:04:07.534 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T10:04:07.544 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T10:04:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.664+0000 7fddd0be5700 1 Processor -- start 2023-10-16T10:04:07.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.664+0000 7fddd0be5700 1 -- start start 2023-10-16T10:04:07.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddd0be5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 0x7fddcc0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddd0be5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc06ed00 con 0x7fddcc15e6d0 2023-10-16T10:04:07.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddd0be5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc0d02d0 con 0x7fddcc0cb0c0 2023-10-16T10:04:07.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddd0be5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc06e0f0 con 0x7fddcc15eab0 2023-10-16T10:04:07.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddca59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fddcc15e6d0 0x7fddcc15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60366/0 (socket says 172.21.15.181:60366) 2023-10-16T10:04:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddca59c700 1 -- 172.21.15.181:0/3598448630 learned_addr learned my addr 172.21.15.181:0/3598448630 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc9d9b700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 0x7fddcc0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196582000 0 0) 0x7fddcc06ed00 con 0x7fddcc15e6d0 2023-10-16T10:04:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb40035b0 con 0x7fddcc15e6d0 2023-10-16T10:04:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2775235728 0 0) 0x7fddb40035b0 con 0x7fddcc15e6d0 2023-10-16T10:04:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 msgr2=0x7fddcc0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 0x7fddcc0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 >> v1:172.21.15.190:6789/0 conn(0x7fddcc0cb0c0 legacy=0x7fddcc15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc06e3d0 con 0x7fddcc15e6d0 2023-10-16T10:04:07.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.666+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fddbc0017f0 con 0x7fddcc15e6d0 2023-10-16T10:04:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 >> v1:172.21.15.181:6789/0 conn(0x7fddcc15e6d0 legacy=0x7fddcc15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddb4003a30 0x7fddb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddb40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 0x7fddb4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc06e0f0 con 0x7fddb40043e0 2023-10-16T10:04:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400ba10 con 0x7fddb4003a30 2023-10-16T10:04:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400dde0 con 0x7fddcc15eab0 2023-10-16T10:04:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc9d9b700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddb40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddca59c700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddb4003a30 0x7fddb4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddcad9d700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 0x7fddb4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc9d9b700 1 -- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 msgr2=0x7fddb4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc9d9b700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15eab0 0x7fddb4005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc9d9b700 1 -- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddb4003a30 msgr2=0x7fddb4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc9d9b700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddb4003a30 0x7fddb4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc9d9b700 1 -- 172.21.15.181:0/3598448630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40035b0 con 0x7fddb40043e0 2023-10-16T10:04:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc9d9b700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddb40048b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009b60 tx=0x7fddb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e6aa5b89dbd014 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000e6c0 con 0x7fddb40043e0 2023-10-16T10:04:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc000e820 con 0x7fddb40043e0 2023-10-16T10:04:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.667+0000 7fddc959a700 1 -- 172.21.15.181:0/3598448630 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000eb90 con 0x7fddb40043e0 2023-10-16T10:04:07.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.668+0000 7fddd0be5700 1 -- 172.21.15.181:0/3598448630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 msgr2=0x7fddb40048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.668+0000 7fddd0be5700 1 --2- 172.21.15.181:0/3598448630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddb40048b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009b60 tx=0x7fddb4005880 comp rx=0 tx=0).stop 2023-10-16T10:04:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.668+0000 7fddcad9d700 1 -- 172.21.15.181:0/3598448630 reap_dead start 2023-10-16T10:04:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.668+0000 7fddd0be5700 1 -- 172.21.15.181:0/3598448630 shutdown_connections 2023-10-16T10:04:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.668+0000 7fddd0be5700 1 -- 172.21.15.181:0/3598448630 >> 172.21.15.181:0/3598448630 conn(0x7fddcc0d04a0 msgr2=0x7fddcc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.668+0000 7fddd0be5700 1 -- 172.21.15.181:0/3598448630 shutdown_connections 2023-10-16T10:04:07.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.668+0000 7fddd0be5700 1 -- 172.21.15.181:0/3598448630 wait complete. 2023-10-16T10:04:07.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.670+0000 7fddd0be5700 1 Processor -- start 2023-10-16T10:04:07.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.670+0000 7fddd0be5700 1 -- start start 2023-10-16T10:04:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddcc152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddcc0b32f0 0x7fddcc1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15c550 0x7fddcc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc15cf10 con 0x7fddb40043e0 2023-10-16T10:04:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc154c00 con 0x7fddcc0b32f0 2023-10-16T10:04:07.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc0b3770 con 0x7fddcc15c550 2023-10-16T10:04:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddcc152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddcc152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49298/0 (socket says 172.21.15.181:49298) 2023-10-16T10:04:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 -- 172.21.15.181:0/3116008697 learned_addr learned my addr 172.21.15.181:0/3116008697 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:07.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 -- 172.21.15.181:0/3116008697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15c550 msgr2=0x7fddcc15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15c550 0x7fddcc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 -- 172.21.15.181:0/3116008697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddcc0b32f0 msgr2=0x7fddcc1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddcc0b32f0 0x7fddcc1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 -- 172.21.15.181:0/3116008697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc0b39f0 con 0x7fddb40043e0 2023-10-16T10:04:07.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddca59c700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddcc152b80 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fddbc001920 tx=0x7fddbc00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddbaffd700 1 -- 172.21.15.181:0/3116008697 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00fe80 con 0x7fddb40043e0 2023-10-16T10:04:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddbaffd700 1 -- 172.21.15.181:0/3116008697 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc0072e0 con 0x7fddb40043e0 2023-10-16T10:04:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc0b3cd0 con 0x7fddb40043e0 2023-10-16T10:04:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddbaffd700 1 -- 172.21.15.181:0/3116008697 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc007670 con 0x7fddb40043e0 2023-10-16T10:04:07.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.671+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddcc0b4290 con 0x7fddb40043e0 2023-10-16T10:04:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.672+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddcc067d20 con 0x7fddb40043e0 2023-10-16T10:04:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.673+0000 7fddbaffd700 1 -- 172.21.15.181:0/3116008697 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fddbc005830 con 0x7fddb40043e0 2023-10-16T10:04:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.673+0000 7fddbaffd700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fddac06b020 0x7fddac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.673+0000 7fddbaffd700 1 -- 172.21.15.181:0/3116008697 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fddbc0898d0 con 0x7fddb40043e0 2023-10-16T10:04:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.673+0000 7fddc9d9b700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fddac06b020 0x7fddac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:07.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.673+0000 7fddc9d9b700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fddac06b020 0x7fddac06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000c00 tx=0x7fddc0013000 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:07.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.677+0000 7fddbaffd700 1 -- 172.21.15.181:0/3116008697 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fddbc054730 con 0x7fddb40043e0 2023-10-16T10:04:07.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.910+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddcc0cb610 con 0x7fddb40043e0 2023-10-16T10:04:07.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.910+0000 7fddbaffd700 1 -- 172.21.15.181:0/3116008697 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fddbc057d50 con 0x7fddb40043e0 2023-10-16T10:04:07.912 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:07.912 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1229456263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1229456263},{"type":"v1","addr":"172.21.15.190:6834","nonce":1229456263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3176781358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3176781358},{"type":"v1","addr":"172.21.15.181:6836","nonce":3176781358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2996369280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2996369280},{"type":"v1","addr":"172.21.15.181:6837","nonce":2996369280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:01.954427+0000","modified":"2023-10-16T10:04:04.993634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8146":{"gid":8146,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3854101486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3854101486},{"type":"v1","addr":"172.21.15.190:6835","nonce":3854101486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T10:04:07.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fddac06b020 msgr2=0x7fddac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fddac06b020 0x7fddac06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000c00 tx=0x7fddc0013000 comp rx=0 tx=0).stop 2023-10-16T10:04:07.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 msgr2=0x7fddcc152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:07.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddcc152b80 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fddbc001920 tx=0x7fddbc00fc40 comp rx=0 tx=0).stop 2023-10-16T10:04:07.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 shutdown_connections 2023-10-16T10:04:07.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fddac06b020 0x7fddac06d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddcc15c550 0x7fddcc15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddb40043e0 0x7fddcc152b80 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.912+0000 7fddd0be5700 1 --2- 172.21.15.181:0/3116008697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddcc0b32f0 0x7fddcc1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:07.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.913+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 >> 172.21.15.181:0/3116008697 conn(0x7fddcc0d04a0 msgr2=0x7fddcc0c6580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:07.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.913+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 shutdown_connections 2023-10-16T10:04:07.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:07.913+0000 7fddd0be5700 1 -- 172.21.15.181:0/3116008697 wait complete. 2023-10-16T10:04:07.916 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 76 2023-10-16T10:04:08.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T10:04:01.954427+0000', 'modified': '2023-10-16T10:04:04.993634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8146}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8146': {'gid': 8146, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/3854101486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 3854101486}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 3854101486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T10:04:08.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T10:04:08.330 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-16T10:04:08.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.454+0000 7fcd77ac5700 1 Processor -- start 2023-10-16T10:04:08.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.454+0000 7fcd77ac5700 1 -- start start 2023-10-16T10:04:08.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd77ac5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcd700ce4d0 0x7fcd700cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd77ac5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700cc4c0 0x7fcd7015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd77ac5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700cf460 con 0x7fcd7006d210 2023-10-16T10:04:08.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd77ac5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd700d0380 con 0x7fcd700ce4d0 2023-10-16T10:04:08.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd77ac5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7006c700 con 0x7fcd700cc4c0 2023-10-16T10:04:08.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700cc4c0 0x7fcd7015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd75060700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcd700ce4d0 0x7fcd700cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700cc4c0 0x7fcd7015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:46336/0 (socket says 172.21.15.181:46336) 2023-10-16T10:04:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 -- 172.21.15.181:0/3985197728 learned_addr learned my addr 172.21.15.181:0/3985197728 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd75861700 1 --1- 172.21.15.181:0/3985197728 >> v1:172.21.15.190:6789/0 conn(0x7fcd7006d210 0x7fcd700cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:44326/0 (socket says 172.21.15.181:44326) 2023-10-16T10:04:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 -- 172.21.15.181:0/3985197728 >> v2:172.21.15.190:3300/0 conn(0x7fcd700ce4d0 msgr2=0x7fcd700cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 --2- 172.21.15.181:0/3985197728 >> v2:172.21.15.190:3300/0 conn(0x7fcd700ce4d0 0x7fcd700cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 -- 172.21.15.181:0/3985197728 >> v1:172.21.15.190:6789/0 conn(0x7fcd7006d210 legacy=0x7fcd700cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 -- 172.21.15.181:0/3985197728 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7006d5f0 con 0x7fcd700cc4c0 2023-10-16T10:04:08.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.455+0000 7fcd76062700 1 --2- 172.21.15.181:0/3985197728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700cc4c0 0x7fcd7015d2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009e50 tx=0x7fcd6800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3483c9b69e82e9a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd7485f700 1 -- 172.21.15.181:0/3985197728 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800e070 con 0x7fcd700cc4c0 2023-10-16T10:04:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd7485f700 1 -- 172.21.15.181:0/3985197728 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800b910 con 0x7fcd700cc4c0 2023-10-16T10:04:08.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd7485f700 1 -- 172.21.15.181:0/3985197728 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800bcc0 con 0x7fcd700cc4c0 2023-10-16T10:04:08.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 -- 172.21.15.181:0/3985197728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700cc4c0 msgr2=0x7fcd7015d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/3985197728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700cc4c0 0x7fcd7015d2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009e50 tx=0x7fcd6800b330 comp rx=0 tx=0).stop 2023-10-16T10:04:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 -- 172.21.15.181:0/3985197728 shutdown_connections 2023-10-16T10:04:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/3985197728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700cc4c0 0x7fcd7015d2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/3985197728 >> v2:172.21.15.190:3300/0 conn(0x7fcd700ce4d0 0x7fcd700cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 -- 172.21.15.181:0/3985197728 >> 172.21.15.181:0/3985197728 conn(0x7fcd7009a9a0 msgr2=0x7fcd700d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 -- 172.21.15.181:0/3985197728 shutdown_connections 2023-10-16T10:04:08.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.456+0000 7fcd77ac5700 1 -- 172.21.15.181:0/3985197728 wait complete. 2023-10-16T10:04:08.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.457+0000 7fcd77ac5700 1 Processor -- start 2023-10-16T10:04:08.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd77ac5700 1 -- start start 2023-10-16T10:04:08.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd77ac5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd7006d210 0x7fcd700b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd77ac5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd700cc4c0 0x7fcd701506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd77ac5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700ce4d0 0x7fcd7009d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd77ac5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b4c50 con 0x7fcd700cc4c0 2023-10-16T10:04:08.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd77ac5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b4ea0 con 0x7fcd7006d210 2023-10-16T10:04:08.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd77ac5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b5180 con 0x7fcd700ce4d0 2023-10-16T10:04:08.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd75060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd700cc4c0 0x7fcd701506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd75060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd700cc4c0 0x7fcd701506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49314/0 (socket says 172.21.15.181:49314) 2023-10-16T10:04:08.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd75060700 1 -- 172.21.15.181:0/488281449 learned_addr learned my addr 172.21.15.181:0/488281449 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:08.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.458+0000 7fcd75861700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd7006d210 0x7fcd700b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd76062700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700ce4d0 0x7fcd7009d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd75060700 1 -- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700ce4d0 msgr2=0x7fcd7009d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd75060700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700ce4d0 0x7fcd7009d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd75060700 1 -- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd7006d210 msgr2=0x7fcd700b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd75060700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd7006d210 0x7fcd700b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd75060700 1 -- 172.21.15.181:0/488281449 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009b30 con 0x7fcd700cc4c0 2023-10-16T10:04:08.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd75861700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd7006d210 0x7fcd700b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:08.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd75060700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd700cc4c0 0x7fcd701506c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9d0 tx=0x7fcd6c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd667fc700 1 -- 172.21.15.181:0/488281449 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c009730 con 0x7fcd700cc4c0 2023-10-16T10:04:08.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd667fc700 1 -- 172.21.15.181:0/488281449 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00c850 con 0x7fcd700cc4c0 2023-10-16T10:04:08.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd700b5460 con 0x7fcd700cc4c0 2023-10-16T10:04:08.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd667fc700 1 -- 172.21.15.181:0/488281449 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00cbe0 con 0x7fcd700cc4c0 2023-10-16T10:04:08.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.459+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70150c00 con 0x7fcd700cc4c0 2023-10-16T10:04:08.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.460+0000 7fcd667fc700 1 -- 172.21.15.181:0/488281449 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcd6c00cd40 con 0x7fcd700cc4c0 2023-10-16T10:04:08.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.461+0000 7fcd667fc700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fcd5406b050 0x7fcd5406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.461+0000 7fcd667fc700 1 -- 172.21.15.181:0/488281449 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fcd6c089a40 con 0x7fcd700cc4c0 2023-10-16T10:04:08.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.461+0000 7fcd75861700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fcd5406b050 0x7fcd5406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.461+0000 7fcd75861700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fcd5406b050 0x7fcd5406d510 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000f60 tx=0x7fcd60007df0 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.463+0000 7fcd83cafb80 1 Processor -- start 2023-10-16T10:04:08.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.463+0000 7fcd83cafb80 1 -- start start 2023-10-16T10:04:08.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c444f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c44466c20 con 0x564c4451a630 2023-10-16T10:04:08.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4444bc10 con 0x564c44515220 2023-10-16T10:04:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c44449530 con 0x564c444f0010 2023-10-16T10:04:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49322/0 (socket says 172.21.15.181:49322) 2023-10-16T10:04:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 -- 172.21.15.181:0/3074178480 learned_addr learned my addr 172.21.15.181:0/3074178480 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75861700 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd76062700 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c444f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 -- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 msgr2=0x564c444f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c444f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 -- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 msgr2=0x564c4451ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 -- 172.21.15.181:0/3074178480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c444023a0 con 0x564c4451a630 2023-10-16T10:04:08.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd75060700 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451cfb0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c060330 tx=0x7fcd6c058d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961892f1f64ffdc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd537fe700 1 -- 172.21.15.181:0/3074178480 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c060860 con 0x564c4451a630 2023-10-16T10:04:08.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd537fe700 1 -- 172.21.15.181:0/3074178480 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c054c70 con 0x564c4451a630 2023-10-16T10:04:08.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.464+0000 7fcd537fe700 1 -- 172.21.15.181:0/3074178480 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c058a00 con 0x564c4451a630 2023-10-16T10:04:08.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 -- 172.21.15.181:0/3074178480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 msgr2=0x564c4451cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451cfb0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c060330 tx=0x7fcd6c058d90 comp rx=0 tx=0).stop 2023-10-16T10:04:08.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 -- 172.21.15.181:0/3074178480 shutdown_connections 2023-10-16T10:04:08.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c444f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/3074178480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451cfb0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 -- 172.21.15.181:0/3074178480 >> 172.21.15.181:0/3074178480 conn(0x564c4441fd80 msgr2=0x564c4453aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 -- 172.21.15.181:0/3074178480 shutdown_connections 2023-10-16T10:04:08.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.465+0000 7fcd83cafb80 1 -- 172.21.15.181:0/3074178480 wait complete. 2023-10-16T10:04:08.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.466+0000 7fcd83cafb80 1 -- start start 2023-10-16T10:04:08.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.466+0000 7fcd83cafb80 1 Processor -- start 2023-10-16T10:04:08.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c445477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c44444e20 con 0x564c4451a630 2023-10-16T10:04:08.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c443e16c0 con 0x564c44515220 2023-10-16T10:04:08.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c443e1360 con 0x564c444f0010 2023-10-16T10:04:08.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd83cafb80 10 client.0 _ll_register_callbacks cb 0x564c4423e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:04:08.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49330/0 (socket says 172.21.15.181:49330) 2023-10-16T10:04:08.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 -- 172.21.15.181:0/77324520 learned_addr learned my addr 172.21.15.181:0/77324520 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:08.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd75861700 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c445477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd75060700 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 -- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 msgr2=0x564c445477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c445477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 -- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 msgr2=0x564c4451ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.467+0000 7fcd76062700 1 -- 172.21.15.181:0/77324520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70150c00 con 0x564c4451a630 2023-10-16T10:04:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd76062700 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451bfc0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800b680 tx=0x7fcd68004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37871f6c3224571f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:04:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd50ff9700 1 -- 172.21.15.181:0/77324520 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68003ea0 con 0x564c4451a630 2023-10-16T10:04:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd50ff9700 1 -- 172.21.15.181:0/77324520 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fcd68004450 con 0x564c4451a630 2023-10-16T10:04:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c444e55b0 con 0x564c4451a630 2023-10-16T10:04:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd50ff9700 1 -- 172.21.15.181:0/77324520 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd68004610 con 0x564c4451a630 2023-10-16T10:04:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 10 client.9252 fetch_fsmap learned FSMap version 76 2023-10-16T10:04:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x564c4426fd90 con 0x564c4451a630 2023-10-16T10:04:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd50ff9700 1 -- 172.21.15.181:0/77324520 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c444e55b0 con 0x564c4451a630 2023-10-16T10:04:08.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd50ff9700 1 -- 172.21.15.181:0/77324520 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd6800b050 con 0x564c4451a630 2023-10-16T10:04:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 10 client.9252 fetch_fsmap finished waiting for FSMap version 76 2023-10-16T10:04:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 10 client.9252 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T10:04:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 20 client.9252 populate_metadata read hostname 'smithi181' 2023-10-16T10:04:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c4454acb0 0x564c44554610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 4 client.9252 mds_command: new command op to 8146 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T10:04:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 --> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c4443de00 con 0x564c4454acb0 2023-10-16T10:04:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.468+0000 7fcd75861700 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c4454acb0 0x564c44554610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.469+0000 7fcd50ff9700 10 client.9252 ms_handle_connect on v2:172.21.15.190:6833/3854101486 2023-10-16T10:04:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.469+0000 7fcd75861700 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c4454acb0 0x564c44554610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f201042e798648a server_cookie=3bb553f60c61e7a7 in_seq=0 out_seq=0 2023-10-16T10:04:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.469+0000 7fcd50ff9700 1 -- 172.21.15.181:0/77324520 <== mds.0 v2:172.21.15.190:6833/3854101486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c4443de00 con 0x564c4454acb0 2023-10-16T10:04:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.469+0000 7fcd50ff9700 10 client.9252 handle_command_reply: tid=0 2023-10-16T10:04:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.469+0000 7fcd83cafb80 1 client.9252 shutdown 2023-10-16T10:04:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 msgr2=0x564c4451bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451bfc0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800b680 tx=0x7fcd68004b40 comp rx=0 tx=0).stop 2023-10-16T10:04:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 shutdown_connections 2023-10-16T10:04:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c4454acb0 0x564c44554610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c444f0010 0x564c445477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd50ff9700 0 client.9252 ms_handle_reset on v2:172.21.15.190:6833/3854101486 2023-10-16T10:04:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c4451a630 0x564c4451bfc0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/77324520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c44515220 0x564c4451ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 >> 172.21.15.181:0/77324520 conn(0x564c4441fd80 msgr2=0x564c44537a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 shutdown_connections 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 1 -- 172.21.15.181:0/77324520 wait complete. 2023-10-16T10:04:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.470+0000 7fcd83cafb80 20 client.9252 trim_cache size 0 max 16384 2023-10-16T10:04:08.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 Processor -- start 2023-10-16T10:04:08.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 -- start start 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c44558630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44558b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c444eca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c442c58d0 con 0x564c444f0010 2023-10-16T10:04:08.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c445148d0 con 0x564c4451a630 2023-10-16T10:04:08.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c444f3130 con 0x564c44515220 2023-10-16T10:04:08.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd75861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c44558630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd75861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c44558630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49334/0 (socket says 172.21.15.181:49334) 2023-10-16T10:04:08.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.488+0000 7fcd75861700 1 -- 172.21.15.181:0/2664567315 learned_addr learned my addr 172.21.15.181:0/2664567315 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:08.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75060700 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44558b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd76062700 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c444eca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75861700 1 -- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 msgr2=0x564c44558b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75861700 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44558b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75861700 1 -- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 msgr2=0x564c444eca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75861700 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c444eca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75861700 1 -- 172.21.15.181:0/2664567315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c44258c10 con 0x564c444f0010 2023-10-16T10:04:08.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75060700 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44558b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:08.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd75861700 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c44558630 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fcd600059f0 tx=0x7fcd60012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a159938a7a66f695 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd53fff700 1 -- 172.21.15.181:0/2664567315 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60015d30 con 0x564c444f0010 2023-10-16T10:04:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd53fff700 1 -- 172.21.15.181:0/2664567315 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd60015e90 con 0x564c444f0010 2023-10-16T10:04:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd53fff700 1 -- 172.21.15.181:0/2664567315 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60027b00 con 0x564c444f0010 2023-10-16T10:04:08.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2664567315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 msgr2=0x564c44558630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.489+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c44558630 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fcd600059f0 tx=0x7fcd60012000 comp rx=0 tx=0).stop 2023-10-16T10:04:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2664567315 shutdown_connections 2023-10-16T10:04:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c444eca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c44558630 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2664567315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44558b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2664567315 >> 172.21.15.181:0/2664567315 conn(0x564c4441fd80 msgr2=0x564c44515600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2664567315 shutdown_connections 2023-10-16T10:04:08.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2664567315 wait complete. 2023-10-16T10:04:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 -- start start 2023-10-16T10:04:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.490+0000 7fcd83cafb80 1 Processor -- start 2023-10-16T10:04:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c4459bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44532d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd83cafb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c4459ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c444ed440 con 0x564c444f0010 2023-10-16T10:04:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c444ed900 con 0x564c4451a630 2023-10-16T10:04:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd83cafb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c44533280 con 0x564c44515220 2023-10-16T10:04:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd83cafb80 10 client.0 _ll_register_callbacks cb 0x564c4423e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T10:04:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c4459bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c4459bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49340/0 (socket says 172.21.15.181:49340) 2023-10-16T10:04:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 -- 172.21.15.181:0/2109806231 learned_addr learned my addr 172.21.15.181:0/2109806231 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75060700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44532d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd76062700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c4459ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 -- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 msgr2=0x564c44532d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44532d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 -- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 msgr2=0x564c4459ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c4459ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75861700 1 -- 172.21.15.181:0/2109806231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6001d040 con 0x564c444f0010 2023-10-16T10:04:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.491+0000 7fcd75060700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44532d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd75861700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c4459bbd0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005e30 tx=0x7fcd60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ea110c84641869 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T10:04:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd50ff9700 1 -- 172.21.15.181:0/2109806231 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600272c0 con 0x564c444f0010 2023-10-16T10:04:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd50ff9700 1 -- 172.21.15.181:0/2109806231 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fcd60027420 con 0x564c444f0010 2023-10-16T10:04:08.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd6c054c70 con 0x564c444f0010 2023-10-16T10:04:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd50ff9700 1 -- 172.21.15.181:0/2109806231 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd60030450 con 0x564c444f0010 2023-10-16T10:04:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 10 client.9258 fetch_fsmap learned FSMap version 76 2023-10-16T10:04:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x564c44296200 con 0x564c444f0010 2023-10-16T10:04:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd50ff9700 1 -- 172.21.15.181:0/2109806231 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c054c70 con 0x564c444f0010 2023-10-16T10:04:08.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd50ff9700 1 -- 172.21.15.181:0/2109806231 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd600305b0 con 0x564c444f0010 2023-10-16T10:04:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 10 client.9258 fetch_fsmap finished waiting for FSMap version 76 2023-10-16T10:04:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 10 client.9258 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T10:04:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 20 client.9258 populate_metadata read hostname 'smithi181' 2023-10-16T10:04:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c44519440 0x564c44519820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 4 client.9258 mds_command: new command op to 8146 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T10:04:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 --> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c4443db20 con 0x564c44519440 2023-10-16T10:04:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.492+0000 7fcd75060700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c44519440 0x564c44519820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd50ff9700 10 client.9258 ms_handle_connect on v2:172.21.15.190:6833/3854101486 2023-10-16T10:04:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd75060700 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c44519440 0x564c44519820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3076b4e6c34cb7 server_cookie=c3cccd316636bc7e in_seq=0 out_seq=0 2023-10-16T10:04:08.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd50ff9700 1 -- 172.21.15.181:0/2109806231 <== mds.0 v2:172.21.15.190:6833/3854101486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x564c4443db20 con 0x564c44519440 2023-10-16T10:04:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd50ff9700 10 client.9258 handle_command_reply: tid=0 2023-10-16T10:04:08.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 client.9258 shutdown 2023-10-16T10:04:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 msgr2=0x564c4459bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c4459bbd0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005e30 tx=0x7fcd60000bc0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 shutdown_connections 2023-10-16T10:04:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:6833/3854101486,v1:172.21.15.190:6835/3854101486] conn(0x564c44519440 0x564c44519820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c4451a630 0x564c4459ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd50ff9700 0 client.9258 ms_handle_reset on v2:172.21.15.190:6833/3854101486 2023-10-16T10:04:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564c444f0010 0x564c4459bbd0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 --2- 172.21.15.181:0/2109806231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c44515220 0x564c44532d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 >> 172.21.15.181:0/2109806231 conn(0x564c4441fd80 msgr2=0x564c445471e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 shutdown_connections 2023-10-16T10:04:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 1 -- 172.21.15.181:0/2109806231 wait complete. 2023-10-16T10:04:08.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.493+0000 7fcd83cafb80 20 client.9258 trim_cache size 0 max 16384 2023-10-16T10:04:08.527 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T10:04:08.528 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T10:04:08.528 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T10:04:08.528 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 8146, 2023-10-16T10:04:08.528 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T10:04:08.529 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T10:04:08.529 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T10:04:08.529 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.911299241, 2023-10-16T10:04:08.529 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 76, 2023-10-16T10:04:08.529 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 145, 2023-10-16T10:04:08.529 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 145, 2023-10-16T10:04:08.530 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.859810406999999 2023-10-16T10:04:08.530 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T10:04:08.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fcd5406b050 msgr2=0x7fcd5406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fcd5406b050 0x7fcd5406d510 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000f60 tx=0x7fcd60007df0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd700cc4c0 msgr2=0x7fcd701506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd700cc4c0 0x7fcd701506c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9d0 tx=0x7fcd6c00ed90 comp rx=0 tx=0).stop 2023-10-16T10:04:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 shutdown_connections 2023-10-16T10:04:08.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fcd5406b050 0x7fcd5406d510 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd700ce4d0 0x7fcd7009d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd7006d210 0x7fcd700b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 --2- 172.21.15.181:0/488281449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd700cc4c0 0x7fcd701506c0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 >> 172.21.15.181:0/488281449 conn(0x7fcd7009a9a0 msgr2=0x7fcd700c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 shutdown_connections 2023-10-16T10:04:08.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.494+0000 7fcd77ac5700 1 -- 172.21.15.181:0/488281449 wait complete. 2023-10-16T10:04:08.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T10:04:08.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.658+0000 7fd64e334700 1 Processor -- start 2023-10-16T10:04:08.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.658+0000 7fd64e334700 1 -- start start 2023-10-16T10:04:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd64e334700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd64815dc40 0x7fd64806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd64e334700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6480b2710 con 0x7fd64806ade0 2023-10-16T10:04:08.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd64e334700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6480c18f0 con 0x7fd6480c2dc0 2023-10-16T10:04:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd64e334700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480c1bd0 con 0x7fd64815dc40 2023-10-16T10:04:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd6477fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd64806ade0 0x7fd6480c26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60428/0 (socket says 172.21.15.181:60428) 2023-10-16T10:04:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd6477fe700 1 -- 172.21.15.181:0/2560652652 learned_addr learned my addr 172.21.15.181:0/2560652652 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:08.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd647fff700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd64815dc40 0x7fd64806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354841917 0 0) 0x7fd63c001110 con 0x7fd64806ade0 2023-10-16T10:04:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6300035b0 con 0x7fd64806ade0 2023-10-16T10:04:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886562986 0 0) 0x7fd6480c18f0 con 0x7fd6480c2dc0 2023-10-16T10:04:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63c001110 con 0x7fd6480c2dc0 2023-10-16T10:04:08.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3106433517 0 0) 0x7fd6300035b0 con 0x7fd64806ade0 2023-10-16T10:04:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd64815dc40 msgr2=0x7fd64806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd64815dc40 0x7fd64806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 >> v1:172.21.15.190:6789/0 conn(0x7fd6480c2dc0 legacy=0x7fd64815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480c1eb0 con 0x7fd64806ade0 2023-10-16T10:04:08.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd63c001800 con 0x7fd64806ade0 2023-10-16T10:04:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 >> v1:172.21.15.181:6789/0 conn(0x7fd64806ade0 legacy=0x7fd6480c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630003bc0 0x7fd630003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630004850 0x7fd630004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd630005210 0x7fd63000cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480c1bd0 con 0x7fd630004850 2023-10-16T10:04:08.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63000d140 con 0x7fd630003bc0 2023-10-16T10:04:08.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63000d300 con 0x7fd630005210 2023-10-16T10:04:08.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd647fff700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd64815dc40 0x7fd64806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:08.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.659+0000 7fd6477fe700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630004850 0x7fd630004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd64c8d1700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd630005210 0x7fd63000cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd647fff700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630003bc0 0x7fd630003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd6477fe700 1 -- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd630005210 msgr2=0x7fd63000cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd6477fe700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd630005210 0x7fd63000cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd6477fe700 1 -- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630003bc0 msgr2=0x7fd630003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:08.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd6477fe700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630003bc0 0x7fd630003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd6477fe700 1 -- 172.21.15.181:0/2560652652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6300035b0 con 0x7fd630004850 2023-10-16T10:04:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd647fff700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630003bc0 0x7fd630003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd6477fe700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630004850 0x7fd630004cd0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c0e0 tx=0x7fd630005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd981846a2672fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00cc80 con 0x7fd630004850 2023-10-16T10:04:08.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c00cde0 con 0x7fd630004850 2023-10-16T10:04:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd646ffd700 1 -- 172.21.15.181:0/2560652652 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c005ba0 con 0x7fd630004850 2023-10-16T10:04:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd64e334700 1 -- 172.21.15.181:0/2560652652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630004850 msgr2=0x7fd630004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd64e334700 1 --2- 172.21.15.181:0/2560652652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd630004850 0x7fd630004cd0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c0e0 tx=0x7fd630005880 comp rx=0 tx=0).stop 2023-10-16T10:04:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.660+0000 7fd64c8d1700 1 -- 172.21.15.181:0/2560652652 reap_dead start 2023-10-16T10:04:08.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.661+0000 7fd64e334700 1 -- 172.21.15.181:0/2560652652 shutdown_connections 2023-10-16T10:04:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.661+0000 7fd64e334700 1 -- 172.21.15.181:0/2560652652 >> 172.21.15.181:0/2560652652 conn(0x7fd6480b1780 msgr2=0x7fd6480b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.661+0000 7fd64e334700 1 -- 172.21.15.181:0/2560652652 shutdown_connections 2023-10-16T10:04:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.661+0000 7fd64e334700 1 -- 172.21.15.181:0/2560652652 wait complete. 2023-10-16T10:04:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.662+0000 7fd64e334700 1 Processor -- start 2023-10-16T10:04:08.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.662+0000 7fd64e334700 1 -- start start 2023-10-16T10:04:08.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630004850 0x7fd6480c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6480c6d60 0x7fd6480c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6480c95a0 0x7fd648155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480c9a20 con 0x7fd6480c6d60 2023-10-16T10:04:08.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480c72d0 con 0x7fd630004850 2023-10-16T10:04:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480c75b0 con 0x7fd6480c95a0 2023-10-16T10:04:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6480c6d60 0x7fd6480c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6480c6d60 0x7fd6480c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49362/0 (socket says 172.21.15.181:49362) 2023-10-16T10:04:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 -- 172.21.15.181:0/1883794835 learned_addr learned my addr 172.21.15.181:0/1883794835 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:08.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd647fff700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630004850 0x7fd6480c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64c8d1700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6480c95a0 0x7fd648155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 -- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6480c95a0 msgr2=0x7fd648155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6480c95a0 0x7fd648155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 -- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630004850 msgr2=0x7fd6480c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630004850 0x7fd6480c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 -- 172.21.15.181:0/1883794835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480c7890 con 0x7fd6480c6d60 2023-10-16T10:04:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd6477fe700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6480c6d60 0x7fd6480c9060 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c6d0 tx=0x7fd63c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd644ff9700 1 -- 172.21.15.181:0/1883794835 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c021070 con 0x7fd6480c6d60 2023-10-16T10:04:08.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd644ff9700 1 -- 172.21.15.181:0/1883794835 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c000ed0 con 0x7fd6480c6d60 2023-10-16T10:04:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6480c7b70 con 0x7fd6480c6d60 2023-10-16T10:04:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd644ff9700 1 -- 172.21.15.181:0/1883794835 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0070d0 con 0x7fd6480c6d60 2023-10-16T10:04:08.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.663+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd648155fc0 con 0x7fd6480c6d60 2023-10-16T10:04:08.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.664+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd648067d20 con 0x7fd6480c6d60 2023-10-16T10:04:08.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.665+0000 7fd644ff9700 1 -- 172.21.15.181:0/1883794835 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd63c007230 con 0x7fd6480c6d60 2023-10-16T10:04:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.665+0000 7fd644ff9700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd63406b020 0x7fd63406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.666+0000 7fd644ff9700 1 -- 172.21.15.181:0/1883794835 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fd63c08c0a0 con 0x7fd6480c6d60 2023-10-16T10:04:08.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.666+0000 7fd647fff700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd63406b020 0x7fd63406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:08.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.666+0000 7fd647fff700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd63406b020 0x7fd63406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd6480cac10 tx=0x7fd638008040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:08.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.669+0000 7fd644ff9700 1 -- 172.21.15.181:0/1883794835 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd63c056fe0 con 0x7fd6480c6d60 2023-10-16T10:04:08.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.908+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd648072510 con 0x7fd6480c6d60 2023-10-16T10:04:08.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.908+0000 7fd644ff9700 1 -- 172.21.15.181:0/1883794835 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd63c05a600 con 0x7fd6480c6d60 2023-10-16T10:04:08.909 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T10:04:08.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd63406b020 msgr2=0x7fd63406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd63406b020 0x7fd63406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd6480cac10 tx=0x7fd638008040 comp rx=0 tx=0).stop 2023-10-16T10:04:08.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6480c6d60 msgr2=0x7fd6480c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6480c6d60 0x7fd6480c9060 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c6d0 tx=0x7fd63c001ac0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 shutdown_connections 2023-10-16T10:04:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fd63406b020 0x7fd63406d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6480c95a0 0x7fd648155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630004850 0x7fd6480c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 --2- 172.21.15.181:0/1883794835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6480c6d60 0x7fd6480c9060 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 >> 172.21.15.181:0/1883794835 conn(0x7fd6480b1780 msgr2=0x7fd6480cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 shutdown_connections 2023-10-16T10:04:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:08.910+0000 7fd64e334700 1 -- 172.21.15.181:0/1883794835 wait complete. 2023-10-16T10:04:08.923 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T10:04:08.924 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T10:04:08.924 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T10:04:08.924 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T10:04:08.925 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:08.925 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T10:04:08.925 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T10:04:08.926 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T10:04:08.926 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:04:08.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:08.927 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T10:04:08.944 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T10:04:08.944 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T10:04:08.944 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T10:04:08.945 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:08.945 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T10:04:08.945 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:08.945 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:04:08.945 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:08.946 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41221sec preferred_lft 41221sec 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T10:04:08.947 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:08.948 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T10:04:08.949 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T10:04:08.950 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T10:04:08.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:08.951 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T10:04:09.011 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T10:04:09.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:09.012 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T10:04:09.071 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T10:04:09.071 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T10:04:09.071 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T10:04:09.072 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T10:04:09.072 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T10:04:09.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:09.073 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T10:04:09.073 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T10:04:09.073 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T10:04:09.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T10:04:09.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T10:04:09.074 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T10:04:09.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:09.176 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T10:04:09.193 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T10:04:09.193 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T10:04:09.193 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:04:09.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:09.194 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T10:04:09.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:04:09.260 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:09.316 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T10:04:09.316 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T10:04:09.317 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:09.372 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T10:04:09.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:09.373 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:04:09.457 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:04:09.457 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:04:09.458 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:04:09.466 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:04:09.466 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:04:09.466 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:04:09.467 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:04:09.467 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:04:09.541 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T10:04:09.541 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T10:04:09.541 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T10:04:09.541 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T10:04:09.542 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T10:04:09.542 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T10:04:09.542 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T10:04:09.542 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T10:04:09.543 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T10:04:09.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:09.544 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T10:04:09.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:09.572 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T10:04:09.659 INFO:teuthology.orchestra.run.smithi181.stdout:client7445 2023-10-16T10:04:09.659 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T10:04:09.660 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T10:04:09.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T10:04:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.837+0000 7ff31501d700 1 Processor -- start 2023-10-16T10:04:09.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.837+0000 7ff31501d700 1 -- start start 2023-10-16T10:04:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.837+0000 7ff31501d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff31015b9f0 0x7ff3100cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.837+0000 7ff31501d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31006b370 con 0x7ff3100cc400 2023-10-16T10:04:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.837+0000 7ff31501d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31006aa40 con 0x7ff3100cfb50 2023-10-16T10:04:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff31501d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31006b800 con 0x7ff31015b9f0 2023-10-16T10:04:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30e59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff3100cc400 0x7ff3100cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60450/0 (socket says 172.21.15.181:60450) 2023-10-16T10:04:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30e59c700 1 -- 172.21.15.181:0/4242896885 learned_addr learned my addr 172.21.15.181:0/4242896885 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617082760 0 0) 0x7ff31006b370 con 0x7ff3100cc400 2023-10-16T10:04:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f80035b0 con 0x7ff3100cc400 2023-10-16T10:04:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30ed9d700 1 --2- 172.21.15.181:0/4242896885 >> v2:172.21.15.190:3300/0 conn(0x7ff31015b9f0 0x7ff3100cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962739343 0 0) 0x7ff2f80035b0 con 0x7ff3100cc400 2023-10-16T10:04:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 >> v2:172.21.15.190:3300/0 conn(0x7ff31015b9f0 msgr2=0x7ff3100cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 --2- 172.21.15.181:0/4242896885 >> v2:172.21.15.190:3300/0 conn(0x7ff31015b9f0 0x7ff3100cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 >> v1:172.21.15.190:6789/0 conn(0x7ff3100cfb50 legacy=0x7ff3100ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31006ad20 con 0x7ff3100cc400 2023-10-16T10:04:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff304001670 con 0x7ff3100cc400 2023-10-16T10:04:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 >> v1:172.21.15.181:6789/0 conn(0x7ff3100cc400 legacy=0x7ff3100cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8003a00 0x7ff2f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2f8004850 0x7ff2f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2f8005330 0x7ff2f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31006b800 con 0x7ff2f8004850 2023-10-16T10:04:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800c200 con 0x7ff2f8005330 2023-10-16T10:04:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f800c3c0 con 0x7ff2f8003a00 2023-10-16T10:04:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30e59c700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2f8004850 0x7ff2f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30f59e700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2f8005330 0x7ff2f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.838+0000 7ff30ed9d700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8003a00 0x7ff2f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30e59c700 1 -- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8003a00 msgr2=0x7ff2f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30e59c700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8003a00 0x7ff2f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30e59c700 1 -- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2f8005330 msgr2=0x7ff2f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30e59c700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2f8005330 0x7ff2f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30e59c700 1 -- 172.21.15.181:0/4242896885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80035b0 con 0x7ff2f8004850 2023-10-16T10:04:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30f59e700 1 -- 172.21.15.181:0/4242896885 reap_dead start 2023-10-16T10:04:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30ed9d700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8003a00 0x7ff2f8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30e59c700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2f8004850 0x7ff2f8004cd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c270 tx=0x7ff2f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae41d2620dea20f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304007c10 con 0x7ff2f8004850 2023-10-16T10:04:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30400cdb0 con 0x7ff2f8004850 2023-10-16T10:04:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff30dd9b700 1 -- 172.21.15.181:0/4242896885 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304005ba0 con 0x7ff2f8004850 2023-10-16T10:04:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff31501d700 1 -- 172.21.15.181:0/4242896885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2f8004850 msgr2=0x7ff2f8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff31501d700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2f8004850 0x7ff2f8004cd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c270 tx=0x7ff2f8005880 comp rx=0 tx=0).stop 2023-10-16T10:04:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff31501d700 1 -- 172.21.15.181:0/4242896885 shutdown_connections 2023-10-16T10:04:09.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff31501d700 1 --2- 172.21.15.181:0/4242896885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2f8004850 0x7ff2f8004cd0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff31501d700 1 -- 172.21.15.181:0/4242896885 >> 172.21.15.181:0/4242896885 conn(0x7ff3100d04a0 msgr2=0x7ff3100d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:09.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff31501d700 1 -- 172.21.15.181:0/4242896885 shutdown_connections 2023-10-16T10:04:09.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.839+0000 7ff31501d700 1 -- 172.21.15.181:0/4242896885 wait complete. 2023-10-16T10:04:09.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.841+0000 7ff31501d700 1 Processor -- start 2023-10-16T10:04:09.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.841+0000 7ff31501d700 1 -- start start 2023-10-16T10:04:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8004850 0x7ff31006e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3100c4410 0x7ff3100c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3100c60f0 0x7ff3100c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100c7e80 con 0x7ff3100c60f0 2023-10-16T10:04:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100c6ab0 con 0x7ff3100c4410 2023-10-16T10:04:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100ca520 con 0x7ff2f8004850 2023-10-16T10:04:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3100c60f0 0x7ff3100c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3100c60f0 0x7ff3100c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49388/0 (socket says 172.21.15.181:49388) 2023-10-16T10:04:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 -- 172.21.15.181:0/1735825107 learned_addr learned my addr 172.21.15.181:0/1735825107 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30ed9d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8004850 0x7ff31006e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30e59c700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3100c4410 0x7ff3100c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 -- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8004850 msgr2=0x7ff31006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8004850 0x7ff31006e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 -- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3100c4410 msgr2=0x7ff3100c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3100c4410 0x7ff3100c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 -- 172.21.15.181:0/1735825107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31006e980 con 0x7ff3100c60f0 2023-10-16T10:04:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30e59c700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3100c4410 0x7ff3100c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff30f59e700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3100c60f0 0x7ff3100c6570 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff3100cb3c0 tx=0x7ff30800d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff2f77fe700 1 -- 172.21.15.181:0/1735825107 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308003cd0 con 0x7ff3100c60f0 2023-10-16T10:04:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff2f77fe700 1 -- 172.21.15.181:0/1735825107 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308003e30 con 0x7ff3100c60f0 2023-10-16T10:04:09.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31006ec60 con 0x7ff3100c60f0 2023-10-16T10:04:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff2f77fe700 1 -- 172.21.15.181:0/1735825107 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30800da10 con 0x7ff3100c60f0 2023-10-16T10:04:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.842+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31006e550 con 0x7ff3100c60f0 2023-10-16T10:04:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.843+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3100c7fc0 con 0x7ff3100c60f0 2023-10-16T10:04:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.844+0000 7ff2f77fe700 1 -- 172.21.15.181:0/1735825107 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff30800db70 con 0x7ff3100c60f0 2023-10-16T10:04:09.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.844+0000 7ff2f77fe700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.844+0000 7ff2f77fe700 1 -- 172.21.15.181:0/1735825107 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7ff308089ae0 con 0x7ff3100c60f0 2023-10-16T10:04:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.844+0000 7ff30ed9d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.845+0000 7ff30ed9d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff3100697f0 tx=0x7ff300008040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:09.849+0000 7ff2f77fe700 1 -- 172.21.15.181:0/1735825107 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff3100c7fc0 con 0x7ff3100c60f0 2023-10-16T10:04:10.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.069+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 --> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3100c7c00 con 0x7ff2fc06b020 2023-10-16T10:04:10.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.071+0000 7ff2f77fe700 1 -- 172.21.15.181:0/1735825107 <== mgr.8877 v2:172.21.15.181:6832/3796559700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff3000094e0 con 0x7ff2fc06b020 2023-10-16T10:04:10.072 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T10:04:10.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T10:04:10.072 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T10:04:10.072 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T10:04:10.072 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T10:04:10.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff2fc06b020 msgr2=0x7ff2fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff3100697f0 tx=0x7ff300008040 comp rx=0 tx=0).stop 2023-10-16T10:04:10.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3100c60f0 msgr2=0x7ff3100c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3100c60f0 0x7ff3100c6570 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff3100cb3c0 tx=0x7ff30800d150 comp rx=0 tx=0).stop 2023-10-16T10:04:10.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 shutdown_connections 2023-10-16T10:04:10.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2f8004850 0x7ff31006e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3100c60f0 0x7ff3100c6570 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 --2- 172.21.15.181:0/1735825107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3100c4410 0x7ff3100c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 >> 172.21.15.181:0/1735825107 conn(0x7ff3100d04a0 msgr2=0x7ff3100afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 shutdown_connections 2023-10-16T10:04:10.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.073+0000 7ff31501d700 1 -- 172.21.15.181:0/1735825107 wait complete. 2023-10-16T10:04:10.086 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T10:04:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.207+0000 7fc75de4a700 1 Processor -- start 2023-10-16T10:04:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.207+0000 7fc75de4a700 1 -- start start 2023-10-16T10:04:10.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.208+0000 7fc75de4a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 0x7fc75806f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.208+0000 7fc75de4a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc758070590 con 0x7fc7580cb2b0 2023-10-16T10:04:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.208+0000 7fc7577fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc7580cb2b0 0x7fc7580b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60470/0 (socket says 172.21.15.181:60470) 2023-10-16T10:04:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.208+0000 7fc7577fe700 1 -- 172.21.15.181:0/1496545232 learned_addr learned my addr 172.21.15.181:0/1496545232 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.208+0000 7fc75de4a700 1 -- 172.21.15.181:0/1496545232 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc758156330 con 0x7fc7580b5fd0 2023-10-16T10:04:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.208+0000 7fc75de4a700 1 -- 172.21.15.181:0/1496545232 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75806fcf0 con 0x7fc7580cb690 2023-10-16T10:04:10.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 0x7fc75806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229158465 0 0) 0x7fc758070590 con 0x7fc7580cb2b0 2023-10-16T10:04:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7440035b0 con 0x7fc7580cb2b0 2023-10-16T10:04:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850735221 0 0) 0x7fc7440035b0 con 0x7fc7580cb2b0 2023-10-16T10:04:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 msgr2=0x7fc75806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:10.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 0x7fc75806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 >> v1:172.21.15.190:6789/0 conn(0x7fc7580b5fd0 legacy=0x7fc75815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75806ffd0 con 0x7fc7580cb2b0 2023-10-16T10:04:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc740001670 con 0x7fc7580cb2b0 2023-10-16T10:04:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 >> v1:172.21.15.181:6789/0 conn(0x7fc7580cb2b0 legacy=0x7fc7580b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 0x7fc744003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc744004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc744004d90 0x7fc744005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75806fcf0 con 0x7fc744003fb0 2023-10-16T10:04:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74400bba0 con 0x7fc744004d90 2023-10-16T10:04:10.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74400bd60 con 0x7fc7580cb690 2023-10-16T10:04:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc744004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc7577fe700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 0x7fc744003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc757fff700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc744004d90 0x7fc744005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc756ffd700 1 -- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 msgr2=0x7fc744003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:10.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580cb690 0x7fc744003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc756ffd700 1 -- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc744004d90 msgr2=0x7fc744005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:10.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc744004d90 0x7fc744005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc756ffd700 1 -- 172.21.15.181:0/1496545232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440035b0 con 0x7fc744003fb0 2023-10-16T10:04:10.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.209+0000 7fc757fff700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc744004d90 0x7fc744005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:10.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc744004850 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc748009b60 tx=0x7fc744005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32f8323fbdb0f94 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800e6c0 con 0x7fc744003fb0 2023-10-16T10:04:10.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800e820 con 0x7fc744003fb0 2023-10-16T10:04:10.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc7567fc700 1 -- 172.21.15.181:0/1496545232 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800eb90 con 0x7fc744003fb0 2023-10-16T10:04:10.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc75de4a700 1 -- 172.21.15.181:0/1496545232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 msgr2=0x7fc744004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc75de4a700 1 --2- 172.21.15.181:0/1496545232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc744004850 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc748009b60 tx=0x7fc744005880 comp rx=0 tx=0).stop 2023-10-16T10:04:10.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc757fff700 1 -- 172.21.15.181:0/1496545232 reap_dead start 2023-10-16T10:04:10.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc75de4a700 1 -- 172.21.15.181:0/1496545232 shutdown_connections 2023-10-16T10:04:10.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc75de4a700 1 -- 172.21.15.181:0/1496545232 >> 172.21.15.181:0/1496545232 conn(0x7fc7580cd010 msgr2=0x7fc7580cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc75de4a700 1 -- 172.21.15.181:0/1496545232 shutdown_connections 2023-10-16T10:04:10.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.210+0000 7fc75de4a700 1 -- 172.21.15.181:0/1496545232 wait complete. 2023-10-16T10:04:10.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.212+0000 7fc75de4a700 1 Processor -- start 2023-10-16T10:04:10.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.212+0000 7fc75de4a700 1 -- start start 2023-10-16T10:04:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.212+0000 7fc75de4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc7580c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.212+0000 7fc75de4a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580c6810 0x7fc7580c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.212+0000 7fc75de4a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758154b50 0x7fc75815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc75de4a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580c7ba0 con 0x7fc744003fb0 2023-10-16T10:04:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc75de4a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581578d0 con 0x7fc758154b50 2023-10-16T10:04:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc75de4a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc758157bb0 con 0x7fc7580c6810 2023-10-16T10:04:10.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc7580c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc7580c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49404/0 (socket says 172.21.15.181:49404) 2023-10-16T10:04:10.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 -- 172.21.15.181:0/1489473416 learned_addr learned my addr 172.21.15.181:0/1489473416 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:10.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc757fff700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758154b50 0x7fc75815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580c6810 0x7fc7580c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 -- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580c6810 msgr2=0x7fc7580c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580c6810 0x7fc7580c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 -- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758154b50 msgr2=0x7fc75815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758154b50 0x7fc75815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 -- 172.21.15.181:0/1489473416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758157e90 con 0x7fc744003fb0 2023-10-16T10:04:10.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580c6810 0x7fc7580c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T10:04:10.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc7577fe700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc7580c7660 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc740007290 tx=0x7fc740007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc74ffff700 1 -- 172.21.15.181:0/1489473416 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc740007830 con 0x7fc744003fb0 2023-10-16T10:04:10.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc74ffff700 1 -- 172.21.15.181:0/1489473416 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74000f8d0 con 0x7fc744003fb0 2023-10-16T10:04:10.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc758158170 con 0x7fc744003fb0 2023-10-16T10:04:10.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc74ffff700 1 -- 172.21.15.181:0/1489473416 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74000fc80 con 0x7fc744003fb0 2023-10-16T10:04:10.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.213+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7580c4100 con 0x7fc744003fb0 2023-10-16T10:04:10.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.214+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc758158690 con 0x7fc744003fb0 2023-10-16T10:04:10.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.215+0000 7fc74ffff700 1 -- 172.21.15.181:0/1489473416 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 86) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc74001d460 con 0x7fc744003fb0 2023-10-16T10:04:10.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.215+0000 7fc74ffff700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fc73806b020 0x7fc73806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.215+0000 7fc74ffff700 1 -- 172.21.15.181:0/1489473416 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fc7400898e0 con 0x7fc744003fb0 2023-10-16T10:04:10.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.215+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fc73806b020 0x7fc73806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.216+0000 7fc756ffd700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fc73806b020 0x7fc73806d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7580c6c50 tx=0x7fc748008030 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.220+0000 7fc74ffff700 1 -- 172.21.15.181:0/1489473416 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc758158690 con 0x7fc744003fb0 2023-10-16T10:04:10.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.468+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc7580b2d40 con 0x7fc744003fb0 2023-10-16T10:04:10.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.623+0000 7fc74ffff700 1 -- 172.21.15.181:0/1489473416 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v87) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc740054740 con 0x7fc744003fb0 2023-10-16T10:04:10.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.626+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fc73806b020 msgr2=0x7fc73806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.626+0000 7fc75de4a700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fc73806b020 0x7fc73806d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7580c6c50 tx=0x7fc748008030 comp rx=0 tx=0).stop 2023-10-16T10:04:10.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.626+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 msgr2=0x7fc7580c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.626+0000 7fc75de4a700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc7580c7660 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc740007290 tx=0x7fc740007650 comp rx=0 tx=0).stop 2023-10-16T10:04:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.627+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 shutdown_connections 2023-10-16T10:04:10.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.627+0000 7fc75de4a700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fc73806b020 0x7fc73806d4e0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc7580c6c50 tx=0x7fc748008030 comp rx=0 tx=0).stop 2023-10-16T10:04:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.627+0000 7fc75de4a700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758154b50 0x7fc75815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.627+0000 7fc75de4a700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc744003fb0 0x7fc7580c7660 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.627+0000 7fc75de4a700 1 --2- 172.21.15.181:0/1489473416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7580c6810 0x7fc7580c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.627+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 >> 172.21.15.181:0/1489473416 conn(0x7fc7580cd010 msgr2=0x7fc7580b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.627+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 shutdown_connections 2023-10-16T10:04:10.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.628+0000 7fc75de4a700 1 -- 172.21.15.181:0/1489473416 wait complete. 2023-10-16T10:04:10.643 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T10:04:10.765 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:04:10.768 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:04:10.770 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:10.769+0000 7f3d6cab2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:10.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.773+0000 7efda1947700 1 Processor -- start 2023-10-16T10:04:10.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.773+0000 7efda1947700 1 -- start start 2023-10-16T10:04:10.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.773+0000 7efda1947700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efd9c15da70 0x7efd9c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.773+0000 7efda1947700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd9c0ce6a0 0x7efd9c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.773+0000 7efda1947700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9c0cef00 con 0x7efd9c072760 2023-10-16T10:04:10.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.773+0000 7efda1947700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9c072b40 con 0x7efd9c0ce6a0 2023-10-16T10:04:10.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.773+0000 7efda1947700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9c0cec20 con 0x7efd9c15da70 2023-10-16T10:04:10.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd9c0ce6a0 0x7efd9c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efd9c072760 0x7efd9c1615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60518/0 (socket says 172.21.15.181:60518) 2023-10-16T10:04:10.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9b7fe700 1 -- 172.21.15.181:0/2129706042 learned_addr learned my addr 172.21.15.181:0/2129706042 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:10.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9affd700 1 --2- 172.21.15.181:0/2129706042 >> v2:172.21.15.190:3300/0 conn(0x7efd9c15da70 0x7efd9c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9a7fc700 1 -- 172.21.15.181:0/2129706042 >> v2:172.21.15.190:3300/0 conn(0x7efd9c15da70 msgr2=0x7efd9c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:10.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9a7fc700 1 --2- 172.21.15.181:0/2129706042 >> v2:172.21.15.190:3300/0 conn(0x7efd9c15da70 0x7efd9c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9a7fc700 1 -- 172.21.15.181:0/2129706042 >> v1:172.21.15.181:6789/0 conn(0x7efd9c072760 legacy=0x7efd9c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9a7fc700 1 -- 172.21.15.181:0/2129706042 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c071c80 con 0x7efd9c0ce6a0 2023-10-16T10:04:10.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd9a7fc700 1 --2- 172.21.15.181:0/2129706042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c06a8a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0675e0 tx=0x7efd90009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6266bf8505e275d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd99ffb700 1 -- 172.21.15.181:0/2129706042 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90011830 con 0x7efd9c0ce6a0 2023-10-16T10:04:10.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd99ffb700 1 -- 172.21.15.181:0/2129706042 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd90011990 con 0x7efd9c0ce6a0 2023-10-16T10:04:10.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.774+0000 7efd99ffb700 1 -- 172.21.15.181:0/2129706042 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90011d20 con 0x7efd9c0ce6a0 2023-10-16T10:04:10.779 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:10.775+0000 7fbeef959200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T10:04:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 -- 172.21.15.181:0/2129706042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 msgr2=0x7efd9c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 --2- 172.21.15.181:0/2129706042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c06a8a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0675e0 tx=0x7efd90009b20 comp rx=0 tx=0).stop 2023-10-16T10:04:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 -- 172.21.15.181:0/2129706042 shutdown_connections 2023-10-16T10:04:10.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 --2- 172.21.15.181:0/2129706042 >> v2:172.21.15.190:3300/0 conn(0x7efd9c15da70 0x7efd9c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 --2- 172.21.15.181:0/2129706042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c06a8a0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 -- 172.21.15.181:0/2129706042 >> 172.21.15.181:0/2129706042 conn(0x7efd9c0cd010 msgr2=0x7efd9c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 -- 172.21.15.181:0/2129706042 shutdown_connections 2023-10-16T10:04:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.775+0000 7efda1947700 1 -- 172.21.15.181:0/2129706042 wait complete. 2023-10-16T10:04:10.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.776+0000 7efda1947700 1 Processor -- start 2023-10-16T10:04:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.777+0000 7efda1947700 1 -- start start 2023-10-16T10:04:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.777+0000 7efda1947700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd9c072760 0x7efd9c0c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.777+0000 7efda1947700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.777+0000 7efda1947700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd9c15da70 0x7efd9c0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.777+0000 7efda1947700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c0c8a50 con 0x7efd9c0ce6a0 2023-10-16T10:04:10.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.777+0000 7efda1947700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c1f7230 con 0x7efd9c072760 2023-10-16T10:04:10.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.777+0000 7efda1947700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c0c79b0 con 0x7efd9c15da70 2023-10-16T10:04:10.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd9c072760 0x7efd9c0c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd9c072760 0x7efd9c0c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:54492/0 (socket says 172.21.15.181:54492) 2023-10-16T10:04:10.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c0b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49444/0 (socket says 172.21.15.181:49444) 2023-10-16T10:04:10.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 -- 172.21.15.181:0/1673540459 learned_addr learned my addr 172.21.15.181:0/1673540459 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:10.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9b7fe700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd9c15da70 0x7efd9c0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:10.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 -- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd9c15da70 msgr2=0x7efd9c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd9c15da70 0x7efd9c0b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 -- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 msgr2=0x7efd9c0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:10.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c0b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 -- 172.21.15.181:0/1673540459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900094a0 con 0x7efd9c072760 2023-10-16T10:04:10.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9a7fc700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c0b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:10.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd9affd700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd9c072760 0x7efd9c0c5ff0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e9b0 tx=0x7efd8c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:10.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd83fff700 1 -- 172.21.15.181:0/1673540459 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c009710 con 0x7efd9c072760 2023-10-16T10:04:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd83fff700 1 -- 172.21.15.181:0/1673540459 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c00ee50 con 0x7efd9c072760 2023-10-16T10:04:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efd83fff700 1 -- 172.21.15.181:0/1673540459 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00ca50 con 0x7efd9c072760 2023-10-16T10:04:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.778+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c0c7c90 con 0x7efd9c072760 2023-10-16T10:04:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.779+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c0c8250 con 0x7efd9c072760 2023-10-16T10:04:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.780+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c0c88a0 con 0x7efd9c072760 2023-10-16T10:04:10.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.780+0000 7efd83fff700 1 -- 172.21.15.181:0/1673540459 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 87) v1 ==== 89348+0+0 (secure 0 0 0) 0x7efd8c00cbb0 con 0x7efd9c072760 2023-10-16T10:04:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.781+0000 7efd83fff700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 0x7efd8806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.781+0000 7efd9a7fc700 1 -- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 msgr2=0x7efd8806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.781+0000 7efd83fff700 1 -- 172.21.15.181:0/1673540459 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7efd8c089960 con 0x7efd9c072760 2023-10-16T10:04:10.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.781+0000 7efd9a7fc700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 0x7efd8806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:10.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.786+0000 7efd83fff700 1 -- 172.21.15.181:0/1673540459 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efd8c0547c0 con 0x7efd9c072760 2023-10-16T10:04:10.840 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:10.839+0000 7f3d6cab2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:10.846 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:10.846+0000 7fbeef959200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:10.912 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:10.911+0000 7f3d6cab2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:10.920 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:10.919+0000 7fbeef959200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T10:04:10.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.982+0000 7efd9a7fc700 1 -- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 msgr2=0x7efd8806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:10.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.982+0000 7efd9a7fc700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 0x7efd8806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:10.990 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:10.989+0000 7f3d6cab2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:10.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.990+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7efd9c06f6c0 con 0x7efd9c072760 2023-10-16T10:04:10.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.995+0000 7efd83fff700 1 -- 172.21.15.181:0/1673540459 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7efd8c0210d0 con 0x7efd9c072760 2023-10-16T10:04:10.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 msgr2=0x7efd8806d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:10.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 0x7efd8806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:10.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd9c072760 msgr2=0x7efd9c0c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd9c072760 0x7efd9c0c5ff0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e9b0 tx=0x7efd8c00ecc0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 shutdown_connections 2023-10-16T10:04:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7efd8806b070 0x7efd8806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd9c15da70 0x7efd9c0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd9c072760 0x7efd9c0c5ff0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 --2- 172.21.15.181:0/1673540459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd9c0ce6a0 0x7efd9c0b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 >> 172.21.15.181:0/1673540459 conn(0x7efd9c0cd010 msgr2=0x7efd9c0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 shutdown_connections 2023-10-16T10:04:11.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:10.998+0000 7efda1947700 1 -- 172.21.15.181:0/1673540459 wait complete. 2023-10-16T10:04:11.002 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:11.001+0000 7fbeef959200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T10:04:11.011 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T10:04:11.132 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:11.131+0000 7f3d6cab2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:11.150 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:11.150+0000 7fbeef959200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T10:04:11.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.177+0000 7fbab1711700 1 Processor -- start 2023-10-16T10:04:11.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.177+0000 7fbab1711700 1 -- start start 2023-10-16T10:04:11.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbab1711700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbab1711700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbaac0d1500 0x7fbaac0d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbab1711700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaac15ef80 con 0x7fbaac0d01d0 2023-10-16T10:04:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbab1711700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaac0d4290 con 0x7fbaac0d1500 2023-10-16T10:04:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbab1711700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac0ce750 con 0x7fbaac0ce2d0 2023-10-16T10:04:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaab7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbaac0d1500 0x7fbaac0d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaaffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbaac0d01d0 0x7fbaac0d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:44506/0 (socket says 172.21.15.181:44506) 2023-10-16T10:04:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaaffd700 1 -- 172.21.15.181:0/3623902813 learned_addr learned my addr 172.21.15.181:0/3623902813 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaa7fc700 1 -- 172.21.15.181:0/3623902813 >> v2:172.21.15.190:3300/0 conn(0x7fbaac0d1500 msgr2=0x7fbaac0d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaa7fc700 1 --2- 172.21.15.181:0/3623902813 >> v2:172.21.15.190:3300/0 conn(0x7fbaac0d1500 0x7fbaac0d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaa7fc700 1 -- 172.21.15.181:0/3623902813 >> v1:172.21.15.190:6789/0 conn(0x7fbaac0d01d0 legacy=0x7fbaac0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaa7fc700 1 -- 172.21.15.181:0/3623902813 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac0cf580 con 0x7fbaac0ce2d0 2023-10-16T10:04:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaab7fe700 1 --2- 172.21.15.181:0/3623902813 >> v2:172.21.15.190:3300/0 conn(0x7fbaac0d1500 0x7fbaac0d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.179+0000 7fbaaa7fc700 1 --2- 172.21.15.181:0/3623902813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac15ea40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009870 tx=0x7fbaa0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dded35da894a0a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.180+0000 7fbaa9ffb700 1 -- 172.21.15.181:0/3623902813 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0011830 con 0x7fbaac0ce2d0 2023-10-16T10:04:11.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.180+0000 7fbaa9ffb700 1 -- 172.21.15.181:0/3623902813 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa0011990 con 0x7fbaac0ce2d0 2023-10-16T10:04:11.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.180+0000 7fbaa9ffb700 1 -- 172.21.15.181:0/3623902813 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0011d20 con 0x7fbaac0ce2d0 2023-10-16T10:04:11.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.180+0000 7fbab1711700 1 -- 172.21.15.181:0/3623902813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 msgr2=0x7fbaac15ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.180+0000 7fbab1711700 1 --2- 172.21.15.181:0/3623902813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac15ea40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009870 tx=0x7fbaa0009b80 comp rx=0 tx=0).stop 2023-10-16T10:04:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.181+0000 7fbab1711700 1 -- 172.21.15.181:0/3623902813 shutdown_connections 2023-10-16T10:04:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.181+0000 7fbab1711700 1 --2- 172.21.15.181:0/3623902813 >> v2:172.21.15.190:3300/0 conn(0x7fbaac0d1500 0x7fbaac0d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.181+0000 7fbab1711700 1 --2- 172.21.15.181:0/3623902813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac15ea40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.181+0000 7fbab1711700 1 -- 172.21.15.181:0/3623902813 >> 172.21.15.181:0/3623902813 conn(0x7fbaac09cb60 msgr2=0x7fbaac0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.181+0000 7fbab1711700 1 -- 172.21.15.181:0/3623902813 shutdown_connections 2023-10-16T10:04:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.181+0000 7fbab1711700 1 -- 172.21.15.181:0/3623902813 wait complete. 2023-10-16T10:04:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.183+0000 7fbab1711700 1 Processor -- start 2023-10-16T10:04:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.183+0000 7fbab1711700 1 -- start start 2023-10-16T10:04:11.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbab1711700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac0b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbab1711700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaac0d01d0 0x7fbaac0b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbab1711700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0d1500 0x7fbaac0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbab1711700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac0b7450 con 0x7fbaac0d1500 2023-10-16T10:04:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbab1711700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac0b7100 con 0x7fbaac0d01d0 2023-10-16T10:04:11.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbab1711700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac0cb350 con 0x7fbaac0ce2d0 2023-10-16T10:04:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0d1500 0x7fbaac0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0d1500 0x7fbaac0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49496/0 (socket says 172.21.15.181:49496) 2023-10-16T10:04:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 -- 172.21.15.181:0/711064042 learned_addr learned my addr 172.21.15.181:0/711064042 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaaaffd700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac0b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaaa7fc700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaac0d01d0 0x7fbaac0b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 -- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 msgr2=0x7fbaac0b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac0b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 -- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaac0d01d0 msgr2=0x7fbaac0b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaac0d01d0 0x7fbaac0b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.184+0000 7fbaab7fe700 1 -- 172.21.15.181:0/711064042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa4009a70 con 0x7fbaac0d1500 2023-10-16T10:04:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.185+0000 7fbaab7fe700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0d1500 0x7fbaac0cc7a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009e60 tx=0x7fbaa400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.185+0000 7fba937fe700 1 -- 172.21.15.181:0/711064042 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4009cd0 con 0x7fbaac0d1500 2023-10-16T10:04:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.185+0000 7fba937fe700 1 -- 172.21.15.181:0/711064042 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa400f040 con 0x7fbaac0d1500 2023-10-16T10:04:11.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.185+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa00094a0 con 0x7fbaac0d1500 2023-10-16T10:04:11.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.185+0000 7fba937fe700 1 -- 172.21.15.181:0/711064042 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400ccb0 con 0x7fbaac0d1500 2023-10-16T10:04:11.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.185+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaac0cd340 con 0x7fbaac0d1500 2023-10-16T10:04:11.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.186+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaac04ebb0 con 0x7fbaac0d1500 2023-10-16T10:04:11.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.186+0000 7fba937fe700 1 -- 172.21.15.181:0/711064042 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 87) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fbaa4014480 con 0x7fbaac0d1500 2023-10-16T10:04:11.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.187+0000 7fba937fe700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 0x7fba9806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.187+0000 7fbaaaffd700 1 -- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 msgr2=0x7fba9806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:11.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.187+0000 7fba937fe700 1 -- 172.21.15.181:0/711064042 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7fbaa4089f90 con 0x7fbaac0d1500 2023-10-16T10:04:11.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.187+0000 7fbaaaffd700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 0x7fba9806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:11.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.191+0000 7fba937fe700 1 -- 172.21.15.181:0/711064042 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbaa40bcfe0 con 0x7fbaac0d1500 2023-10-16T10:04:11.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.388+0000 7fbaaaffd700 1 -- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 msgr2=0x7fba9806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:11.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.388+0000 7fbaaaffd700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 0x7fba9806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:11.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.391+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fbaac0d3cb0 con 0x7fbaac0d1500 2023-10-16T10:04:11.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.397+0000 7fba937fe700 1 -- 172.21.15.181:0/711064042 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbaa4054ce0 con 0x7fbaac0d1500 2023-10-16T10:04:11.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 msgr2=0x7fba9806d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:11.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 0x7fba9806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0d1500 msgr2=0x7fbaac0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0d1500 0x7fbaac0cc7a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009e60 tx=0x7fbaa400c3a0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 shutdown_connections 2023-10-16T10:04:11.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7fba9806b070 0x7fba9806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaac0ce2d0 0x7fbaac0b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0d1500 0x7fbaac0cc7a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 --2- 172.21.15.181:0/711064042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaac0d01d0 0x7fbaac0b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 >> 172.21.15.181:0/711064042 conn(0x7fbaac09cb60 msgr2=0x7fbaac15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 shutdown_connections 2023-10-16T10:04:11.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.399+0000 7fbab1711700 1 -- 172.21.15.181:0/711064042 wait complete. 2023-10-16T10:04:11.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T10:04:11.413 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T10:04:11.433 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1m 2023-10-16T10:04:11.458 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:11.457+0000 7f3d6cab2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:11.498 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:11.498+0000 7fbeef959200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T10:04:11.573 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:11.572+0000 7f3d6cab2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.602+0000 7ff345502700 1 Processor -- start 2023-10-16T10:04:11.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.602+0000 7ff345502700 1 -- start start 2023-10-16T10:04:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff345502700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400d01d0 0x7ff3400ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff345502700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3400d1500 0x7ff3400d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff345502700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3400cf580 con 0x7ff34015ea40 2023-10-16T10:04:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff345502700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3400d4290 con 0x7ff3400d1500 2023-10-16T10:04:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff345502700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400ce970 con 0x7ff3400d01d0 2023-10-16T10:04:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3400d1500 0x7ff3400d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff34015ea40 0x7ff3400d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:60616/0 (socket says 172.21.15.181:60616) 2023-10-16T10:04:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3400d1500 0x7ff3400d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49540/0 (socket says 172.21.15.181:49540) 2023-10-16T10:04:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33e7fc700 1 -- 172.21.15.181:0/619432781 learned_addr learned my addr 172.21.15.181:0/619432781 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T10:04:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33effd700 1 --2- 172.21.15.181:0/619432781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400d01d0 0x7ff3400ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33f7fe700 1 -- 172.21.15.181:0/619432781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400d01d0 msgr2=0x7ff3400ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33f7fe700 1 --2- 172.21.15.181:0/619432781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400d01d0 0x7ff3400ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33f7fe700 1 -- 172.21.15.181:0/619432781 >> v1:172.21.15.181:6789/0 conn(0x7ff34015ea40 legacy=0x7ff3400d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33f7fe700 1 -- 172.21.15.181:0/619432781 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400cec50 con 0x7ff3400d1500 2023-10-16T10:04:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33dffb700 1 -- 172.21.15.181:0/619432781 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231698912 0 0) 0x7ff3400cf580 con 0x7ff34015ea40 2023-10-16T10:04:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33f7fe700 1 --2- 172.21.15.181:0/619432781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff3400d1990 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7ff338009dc0 tx=0x7ff33800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bff38b2b2038ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33dffb700 1 -- 172.21.15.181:0/619432781 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33800e070 con 0x7ff3400d1500 2023-10-16T10:04:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33dffb700 1 -- 172.21.15.181:0/619432781 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33800bae0 con 0x7ff3400d1500 2023-10-16T10:04:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.603+0000 7ff33dffb700 1 -- 172.21.15.181:0/619432781 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33800be90 con 0x7ff3400d1500 2023-10-16T10:04:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 -- 172.21.15.181:0/619432781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 msgr2=0x7ff3400d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 --2- 172.21.15.181:0/619432781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff3400d1990 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7ff338009dc0 tx=0x7ff33800b330 comp rx=0 tx=0).stop 2023-10-16T10:04:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 -- 172.21.15.181:0/619432781 shutdown_connections 2023-10-16T10:04:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 --2- 172.21.15.181:0/619432781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff3400d1990 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 --2- 172.21.15.181:0/619432781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3400d01d0 0x7ff3400ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 -- 172.21.15.181:0/619432781 >> 172.21.15.181:0/619432781 conn(0x7ff34009cb60 msgr2=0x7ff3400d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 -- 172.21.15.181:0/619432781 shutdown_connections 2023-10-16T10:04:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.604+0000 7ff345502700 1 -- 172.21.15.181:0/619432781 wait complete. 2023-10-16T10:04:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.605+0000 7ff345502700 1 Processor -- start 2023-10-16T10:04:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.605+0000 7ff345502700 1 -- start start 2023-10-16T10:04:11.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff345502700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3400d01d0 0x7ff340144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff345502700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff340147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff345502700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff34015ea40 0x7ff340146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff345502700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340147970 con 0x7ff3400d1500 2023-10-16T10:04:11.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff345502700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401f7700 con 0x7ff3400d01d0 2023-10-16T10:04:11.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff345502700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401f6af0 con 0x7ff34015ea40 2023-10-16T10:04:11.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff33e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff340147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff33e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff340147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49544/0 (socket says 172.21.15.181:49544) 2023-10-16T10:04:11.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff33e7fc700 1 -- 172.21.15.181:0/3814861252 learned_addr learned my addr 172.21.15.181:0/3814861252 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:11.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff33f7fe700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff34015ea40 0x7ff340146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.606+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3400d01d0 0x7ff340144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff33e7fc700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff34015ea40 msgr2=0x7ff340146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff33e7fc700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff34015ea40 0x7ff340146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff33e7fc700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3400d01d0 msgr2=0x7ff340144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff33e7fc700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3400d01d0 0x7ff340144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:11.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff33e7fc700 1 -- 172.21.15.181:0/3814861252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338009a70 con 0x7ff3400d1500 2023-10-16T10:04:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff33f7fe700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff34015ea40 0x7ff340146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff33e7fc700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff340147430 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff3300011e0 tx=0x7ff33000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330004c80 con 0x7ff3400d1500 2023-10-16T10:04:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330004e00 con 0x7ff3400d1500 2023-10-16T10:04:11.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3401f6dd0 con 0x7ff3400d1500 2023-10-16T10:04:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330007310 con 0x7ff3400d1500 2023-10-16T10:04:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.607+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3401f7390 con 0x7ff3400d1500 2023-10-16T10:04:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.608+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34004ebb0 con 0x7ff3400d1500 2023-10-16T10:04:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.608+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 87) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff330007470 con 0x7ff3400d1500 2023-10-16T10:04:11.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.609+0000 7ff3277fe700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 0x7ff32c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.609+0000 7ff33effd700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 msgr2=0x7ff32c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.609+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6161+0+0 (secure 0 0 0) 0x7ff3300895f0 con 0x7ff3400d1500 2023-10-16T10:04:11.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.609+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 0x7ff32c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T10:04:11.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.613+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff330054450 con 0x7ff3400d1500 2023-10-16T10:04:11.622 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:11.621+0000 7fbeef959200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T10:04:11.648 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:11.647+0000 7f3d6cab2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:11.698 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:11.697+0000 7fbeef959200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T10:04:11.729 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:11.728+0000 7f3d6cab2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:11.780 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:11.780+0000 7fbeef959200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T10:04:11.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.810+0000 7ff33effd700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 msgr2=0x7ff32c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:11.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.810+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 0x7ff32c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T10:04:11.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:11.841+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 --> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3400cf210 con 0x7ff32c06b070 2023-10-16T10:04:12.097 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:12.096+0000 7f3d6cab2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:12.152 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:12.151+0000 7fbeef959200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T10:04:12.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:12.211+0000 7ff33effd700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 msgr2=0x7ff32c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:12.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:12.211+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 0x7ff32c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T10:04:12.667 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:12.666+0000 7f3d6cab2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:12.718 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:12.718+0000 7fbeef959200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T10:04:12.737 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:12.736+0000 7f3d6cab2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:12.788 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:12.787+0000 7fbeef959200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T10:04:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:13.012+0000 7ff33effd700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 msgr2=0x7ff32c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:13.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:13.012+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 0x7ff32c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T10:04:13.365 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:13.364+0000 7f3d6cab2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:13.415 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:13.415+0000 7fbeef959200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:13.443 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:13.442+0000 7f3d6cab2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:13.494 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:13.493+0000 7fbeef959200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T10:04:13.586 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:13.585+0000 7f3d6cab2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:13.639 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:13.638+0000 7fbeef959200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T10:04:13.814 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:13.813+0000 7f3d6cab2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:13.868 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:13.867+0000 7fbeef959200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T10:04:13.888 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:13.887+0000 7f3d6cab2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:13.941 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:13.941+0000 7fbeef959200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T10:04:14.155 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:14.154+0000 7f3d6cab2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:14.209 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:14.209+0000 7fbeef959200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T10:04:14.378 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:14.377+0000 7f3d6cab2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:14.433 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:14.432+0000 7fbeef959200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T10:04:14.527 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:14.526+0000 7f3d6cab2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:14.583 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:14.582+0000 7fbeef959200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T10:04:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:14.613+0000 7ff33effd700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 msgr2=0x7ff32c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3796559700 2023-10-16T10:04:14.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:14.613+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 0x7ff32c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T10:04:14.641 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:14.640+0000 7f3d6cab2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:14.696 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:14.696+0000 7fbeef959200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T10:04:14.797 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:14.796+0000 7f3d6cab2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:14.854 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:14.853+0000 7fbeef959200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T10:04:15.060 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:15.059+0000 7f3d6cab2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:15.118 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.117+0000 7fbeef959200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T10:04:15.219 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:15.218+0000 7f3d6cab2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:15.280 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.280+0000 7fbeef959200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T10:04:15.747 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T10:04:15.746+0000 7f3d6cab2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:15.809 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.809+0000 7fbeef959200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T10:04:15.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:15.816+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 88) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff330054820 con 0x7ff3400d1500 2023-10-16T10:04:15.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:15.828+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 89) v1 ==== 88957+0+0 (secure 0 0 0) 0x7ff330087910 con 0x7ff3400d1500 2023-10-16T10:04:15.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:15.829+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 msgr2=0x7ff32c06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T10:04:15.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:15.829+0000 7ff3277fe700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/3796559700,v1:172.21.15.181:6833/3796559700] conn(0x7ff32c06b070 0x7ff32c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:15.981 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe6593a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.981 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe6593a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.981 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe6593a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.981 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe6593a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.982 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe62934700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.982 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe6593a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.982 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe62934700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.982 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe62934700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.983 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe62934700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:15.983 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:04:15.980+0000 7fbe62934700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:04:16.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.833+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 90) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff33006f8e0 con 0x7ff3400d1500 2023-10-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.834+0000 7ff3277fe700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff32c0279c0 0x7ff32c027da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:16.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.834+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 --> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff32c071070 con 0x7ff32c0279c0 2023-10-16T10:04:16.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.836+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff32c0279c0 0x7ff32c027da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:16.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.838+0000 7ff33effd700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff32c0279c0 0x7ff32c027da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff334009680 tx=0x7ff334009210 comp rx=0 tx=0).ready entity=mgr.9294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:16.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.898+0000 7ff3277fe700 1 -- 172.21.15.181:0/3814861252 <== mgr.9294 v2:172.21.15.181:6832/1123221928 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff32c071070 con 0x7ff32c0279c0 2023-10-16T10:04:16.899 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1 2023-10-16T10:04:16.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.900+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff32c0279c0 msgr2=0x7ff32c027da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.900+0000 7ff345502700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff32c0279c0 0x7ff32c027da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff334009680 tx=0x7ff334009210 comp rx=0 tx=0).stop 2023-10-16T10:04:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.900+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 msgr2=0x7ff340147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.900+0000 7ff345502700 1 --2- 172.21.15.181:0/3814861252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3400d1500 0x7ff340147430 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff3300011e0 tx=0x7ff33000ca30 comp rx=0 tx=0).stop 2023-10-16T10:04:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.900+0000 7ff33f7fe700 1 -- 172.21.15.181:0/3814861252 reap_dead start 2023-10-16T10:04:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.900+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 shutdown_connections 2023-10-16T10:04:16.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.900+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 >> 172.21.15.181:0/3814861252 conn(0x7ff34009cb60 msgr2=0x7ff3400d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:16.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.901+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 shutdown_connections 2023-10-16T10:04:16.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:16.901+0000 7ff345502700 1 -- 172.21.15.181:0/3814861252 wait complete. 2023-10-16T10:04:16.914 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-16T10:04:16.914 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-10-16T10:04:17.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.082+0000 7f63c1bb2700 1 Processor -- start 2023-10-16T10:04:17.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.082+0000 7f63c1bb2700 1 -- start start 2023-10-16T10:04:17.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.082+0000 7f63c1bb2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc15eea0 0x7f63bc0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.082+0000 7f63c1bb2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63bc0ce810 0x7f63bc15f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63c1bb2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63bc0d23c0 con 0x7f63bc0b0f70 2023-10-16T10:04:17.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63c1bb2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63bc0cd990 con 0x7f63bc0ce810 2023-10-16T10:04:17.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63c1bb2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0cdc70 con 0x7f63bc15eea0 2023-10-16T10:04:17.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc15eea0 0x7f63bc0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc15eea0 0x7f63bc0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:46762/0 (socket says 172.21.15.181:46762) 2023-10-16T10:04:17.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 -- 172.21.15.181:0/3313730057 learned_addr learned my addr 172.21.15.181:0/3313730057 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:17.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bbfff700 1 --2- 172.21.15.181:0/3313730057 >> v2:172.21.15.190:3300/0 conn(0x7f63bc0ce810 0x7f63bc15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 -- 172.21.15.181:0/3313730057 >> v2:172.21.15.190:3300/0 conn(0x7f63bc0ce810 msgr2=0x7f63bc15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:17.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 --2- 172.21.15.181:0/3313730057 >> v2:172.21.15.190:3300/0 conn(0x7f63bc0ce810 0x7f63bc15f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 -- 172.21.15.181:0/3313730057 >> v1:172.21.15.190:6789/0 conn(0x7f63bc0b0f70 legacy=0x7f63bc0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 -- 172.21.15.181:0/3313730057 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0cce60 con 0x7f63bc15eea0 2023-10-16T10:04:17.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63bb7fe700 1 --2- 172.21.15.181:0/3313730057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc15eea0 0x7f63bc0b03e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0098f0 tx=0x7f63ac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2263c664ff5daec5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.083+0000 7f63ba7fc700 1 -- 172.21.15.181:0/3313730057 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac011830 con 0x7f63bc15eea0 2023-10-16T10:04:17.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63ba7fc700 1 -- 172.21.15.181:0/3313730057 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac011990 con 0x7f63bc15eea0 2023-10-16T10:04:17.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63ba7fc700 1 -- 172.21.15.181:0/3313730057 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac011d20 con 0x7f63bc15eea0 2023-10-16T10:04:17.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 -- 172.21.15.181:0/3313730057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc15eea0 msgr2=0x7f63bc0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/3313730057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc15eea0 0x7f63bc0b03e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0098f0 tx=0x7f63ac009c00 comp rx=0 tx=0).stop 2023-10-16T10:04:17.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 -- 172.21.15.181:0/3313730057 shutdown_connections 2023-10-16T10:04:17.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/3313730057 >> v2:172.21.15.190:3300/0 conn(0x7f63bc0ce810 0x7f63bc15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/3313730057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc15eea0 0x7f63bc0b03e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 -- 172.21.15.181:0/3313730057 >> 172.21.15.181:0/3313730057 conn(0x7f63bc0d1610 msgr2=0x7f63bc0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 -- 172.21.15.181:0/3313730057 shutdown_connections 2023-10-16T10:04:17.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.084+0000 7f63c1bb2700 1 -- 172.21.15.181:0/3313730057 wait complete. 2023-10-16T10:04:17.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.085+0000 7f63c1bb2700 1 Processor -- start 2023-10-16T10:04:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63c1bb2700 1 -- start start 2023-10-16T10:04:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63c1bb2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b0f70 0x7f63bc148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63c1bb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc0ce810 0x7f63bc0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63c1bb2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc15eea0 0x7f63bc0c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63c1bb2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0ca2a0 con 0x7f63bc0ce810 2023-10-16T10:04:17.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63c1bb2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0ca580 con 0x7f63bc15eea0 2023-10-16T10:04:17.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63c1bb2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc0ca860 con 0x7f63bc0b0f70 2023-10-16T10:04:17.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc0ce810 0x7f63bc0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63baffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc0ce810 0x7f63bc0c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49744/0 (socket says 172.21.15.181:49744) 2023-10-16T10:04:17.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.086+0000 7f63baffd700 1 -- 172.21.15.181:0/12995461 learned_addr learned my addr 172.21.15.181:0/12995461 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:17.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63bb7fe700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b0f70 0x7f63bc148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63bbfff700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc15eea0 0x7f63bc0c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63baffd700 1 -- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b0f70 msgr2=0x7f63bc148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63baffd700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b0f70 0x7f63bc148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63baffd700 1 -- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc15eea0 msgr2=0x7f63bc0c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63baffd700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc15eea0 0x7f63bc0c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63baffd700 1 -- 172.21.15.181:0/12995461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac009550 con 0x7f63bc0ce810 2023-10-16T10:04:17.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63baffd700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc0ce810 0x7f63bc0c9730 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c450 tx=0x7f63b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63a3fff700 1 -- 172.21.15.181:0/12995461 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00052d0 con 0x7f63bc0ce810 2023-10-16T10:04:17.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63a3fff700 1 -- 172.21.15.181:0/12995461 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b0005430 con 0x7f63bc0ce810 2023-10-16T10:04:17.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63a3fff700 1 -- 172.21.15.181:0/12995461 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00038c0 con 0x7f63bc0ce810 2023-10-16T10:04:17.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc0cab40 con 0x7f63bc0ce810 2023-10-16T10:04:17.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.087+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc148ca0 con 0x7f63bc0ce810 2023-10-16T10:04:17.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.088+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc0cb060 con 0x7f63bc0ce810 2023-10-16T10:04:17.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.088+0000 7f63a3fff700 1 -- 172.21.15.181:0/12995461 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 90) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f63b0003a20 con 0x7f63bc0ce810 2023-10-16T10:04:17.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.089+0000 7f63a3fff700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f63a806ada0 0x7f63a806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.089+0000 7f63a3fff700 1 -- 172.21.15.181:0/12995461 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6505+0+0 (secure 0 0 0) 0x7f63b00888d0 con 0x7f63bc0ce810 2023-10-16T10:04:17.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.089+0000 7f63bb7fe700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f63a806ada0 0x7f63a806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.089+0000 7f63bb7fe700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f63a806ada0 0x7f63a806d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009c40 tx=0x7f63ac008030 comp rx=0 tx=0).ready entity=mgr.9294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.094+0000 7f63a3fff700 1 -- 172.21.15.181:0/12995461 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63bc0cb060 con 0x7f63bc0ce810 2023-10-16T10:04:17.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.314+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 --> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f63bc0ca6f0 con 0x7f63a806ada0 2023-10-16T10:04:17.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.373+0000 7f63a3fff700 1 -- 172.21.15.181:0/12995461 <== mgr.9294 v2:172.21.15.181:6832/1123221928 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f63bc0ca6f0 con 0x7f63a806ada0 2023-10-16T10:04:17.375 INFO:teuthology.orchestra.run.smithi181.stdout:Retention added to path /snap_test_dir1 2023-10-16T10:04:17.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f63a806ada0 msgr2=0x7f63a806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f63a806ada0 0x7f63a806d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009c40 tx=0x7f63ac008030 comp rx=0 tx=0).stop 2023-10-16T10:04:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc0ce810 msgr2=0x7f63bc0c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc0ce810 0x7f63bc0c9730 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c450 tx=0x7f63b000c810 comp rx=0 tx=0).stop 2023-10-16T10:04:17.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 shutdown_connections 2023-10-16T10:04:17.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f63a806ada0 0x7f63a806d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0b0f70 0x7f63bc148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc15eea0 0x7f63bc0c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 --2- 172.21.15.181:0/12995461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc0ce810 0x7f63bc0c9730 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 >> 172.21.15.181:0/12995461 conn(0x7f63bc0d1610 msgr2=0x7f63bc0af930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 shutdown_connections 2023-10-16T10:04:17.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.376+0000 7f63c1bb2700 1 -- 172.21.15.181:0/12995461 wait complete. 2023-10-16T10:04:17.389 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-16T10:04:17.390 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-16T10:04:17.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-10-16T10:04:17.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.516+0000 7fa0f5f18700 1 Processor -- start 2023-10-16T10:04:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.516+0000 7fa0f5f18700 1 -- start start 2023-10-16T10:04:17.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0f5f18700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa0f00cdde0 0x7fa0f015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0f5f18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f006a8a0 0x7fa0f006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0f5f18700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f00b1e20 con 0x7fa0f015e520 2023-10-16T10:04:17.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0f5f18700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f006b290 con 0x7fa0f00cdde0 2023-10-16T10:04:17.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0f5f18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00cd0f0 con 0x7fa0f006a8a0 2023-10-16T10:04:17.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0effff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f006a8a0 0x7fa0f006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0eeffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa0f00cdde0 0x7fa0f015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0effff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f006a8a0 0x7fa0f006ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:46776/0 (socket says 172.21.15.181:46776) 2023-10-16T10:04:17.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.517+0000 7fa0effff700 1 -- 172.21.15.181:0/2622883223 learned_addr learned my addr 172.21.15.181:0/2622883223 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:17.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0eeffd700 1 -- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f006a8a0 msgr2=0x7fa0f006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T10:04:17.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0eeffd700 1 --2- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f006a8a0 0x7fa0f006ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0eeffd700 1 -- 172.21.15.181:0/2622883223 >> v1:172.21.15.190:6789/0 conn(0x7fa0f015e520 legacy=0x7fa0f00b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0eeffd700 1 -- 172.21.15.181:0/2622883223 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00cd300 con 0x7fa0f00cdde0 2023-10-16T10:04:17.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0ee7fc700 1 -- 172.21.15.181:0/2622883223 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201518843 0 0) 0x7fa0f00b1e20 con 0x7fa0f015e520 2023-10-16T10:04:17.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0effff700 1 --2- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f006a8a0 0x7fa0f006ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T10:04:17.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0eeffd700 1 --2- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f015e900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009840 tx=0x7fa0e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cff99648d621a21 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0ee7fc700 1 -- 172.21.15.181:0/2622883223 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0011830 con 0x7fa0f00cdde0 2023-10-16T10:04:17.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0ee7fc700 1 -- 172.21.15.181:0/2622883223 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e0011990 con 0x7fa0f00cdde0 2023-10-16T10:04:17.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0ee7fc700 1 -- 172.21.15.181:0/2622883223 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0011d20 con 0x7fa0f00cdde0 2023-10-16T10:04:17.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0f5f18700 1 -- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 msgr2=0x7fa0f015e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.518+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f015e900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009840 tx=0x7fa0e0009b50 comp rx=0 tx=0).stop 2023-10-16T10:04:17.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.519+0000 7fa0f5f18700 1 -- 172.21.15.181:0/2622883223 shutdown_connections 2023-10-16T10:04:17.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.519+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f006a8a0 0x7fa0f006ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.519+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/2622883223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f015e900 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.519+0000 7fa0f5f18700 1 -- 172.21.15.181:0/2622883223 >> 172.21.15.181:0/2622883223 conn(0x7fa0f006c430 msgr2=0x7fa0f0099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.519+0000 7fa0f5f18700 1 -- 172.21.15.181:0/2622883223 shutdown_connections 2023-10-16T10:04:17.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.519+0000 7fa0f5f18700 1 -- 172.21.15.181:0/2622883223 wait complete. 2023-10-16T10:04:17.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.520+0000 7fa0f5f18700 1 Processor -- start 2023-10-16T10:04:17.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.520+0000 7fa0f5f18700 1 -- start start 2023-10-16T10:04:17.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f006a8a0 0x7fa0f00b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f00c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f015e520 0x7fa0f015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0160900 con 0x7fa0f006a8a0 2023-10-16T10:04:17.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f014e750 con 0x7fa0f00cdde0 2023-10-16T10:04:17.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00c4b20 con 0x7fa0f015e520 2023-10-16T10:04:17.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f006a8a0 0x7fa0f00b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f006a8a0 0x7fa0f00b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49754/0 (socket says 172.21.15.181:49754) 2023-10-16T10:04:17.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 -- 172.21.15.181:0/609137051 learned_addr learned my addr 172.21.15.181:0/609137051 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T10:04:17.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0eeffd700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f00c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0effff700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f015e520 0x7fa0f015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 -- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f015e520 msgr2=0x7fa0f015f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f015e520 0x7fa0f015f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 -- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 msgr2=0x7fa0f00c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f00c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 -- 172.21.15.181:0/609137051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e8009b30 con 0x7fa0f006a8a0 2023-10-16T10:04:17.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0eeffd700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f00c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T10:04:17.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0ef7fe700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f006a8a0 0x7fa0f00b0020 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80011e0 tx=0x7fa0d800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0e7fff700 1 -- 172.21.15.181:0/609137051 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80054e0 con 0x7fa0f006a8a0 2023-10-16T10:04:17.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0e7fff700 1 -- 172.21.15.181:0/609137051 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d8003680 con 0x7fa0f006a8a0 2023-10-16T10:04:17.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e00094a0 con 0x7fa0f006a8a0 2023-10-16T10:04:17.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0e7fff700 1 -- 172.21.15.181:0/609137051 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8003a30 con 0x7fa0f006a8a0 2023-10-16T10:04:17.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.521+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f00b0e10 con 0x7fa0f006a8a0 2023-10-16T10:04:17.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.522+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f00cbd20 con 0x7fa0f006a8a0 2023-10-16T10:04:17.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.523+0000 7fa0e7fff700 1 -- 172.21.15.181:0/609137051 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 90) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fa0d8003b90 con 0x7fa0f006a8a0 2023-10-16T10:04:17.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.523+0000 7fa0e7fff700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa0d006add0 0x7fa0d006d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T10:04:17.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.523+0000 7fa0e7fff700 1 -- 172.21.15.181:0/609137051 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6505+0+0 (secure 0 0 0) 0x7fa0d8088af0 con 0x7fa0f006a8a0 2023-10-16T10:04:17.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.523+0000 7fa0eeffd700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa0d006add0 0x7fa0d006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T10:04:17.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.524+0000 7fa0eeffd700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa0d006add0 0x7fa0d006d290 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00090b0 tx=0x7fa0e0008030 comp rx=0 tx=0).ready entity=mgr.9294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T10:04:17.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.528+0000 7fa0e7fff700 1 -- 172.21.15.181:0/609137051 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa0d8053b40 con 0x7fa0f006a8a0 2023-10-16T10:04:17.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.747+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 --> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0f00c4ce0 con 0x7fa0d006add0 2023-10-16T10:04:17.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.756+0000 7fa0e7fff700 1 -- 172.21.15.181:0/609137051 <== mgr.9294 v2:172.21.15.181:6832/1123221928 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7fa0f00c4ce0 con 0x7fa0d006add0 2023-10-16T10:04:17.757 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T10:04:17.757 INFO:teuthology.orchestra.run.smithi181.stdout:{"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-16T10:04:17.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa0d006add0 msgr2=0x7fa0d006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa0d006add0 0x7fa0d006d290 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00090b0 tx=0x7fa0e0008030 comp rx=0 tx=0).stop 2023-10-16T10:04:17.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f006a8a0 msgr2=0x7fa0f00b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T10:04:17.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f006a8a0 0x7fa0f00b0020 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80011e0 tx=0x7fa0d800cae0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 shutdown_connections 2023-10-16T10:04:17.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa0d006add0 0x7fa0d006d290 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f015e520 0x7fa0f015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f006a8a0 0x7fa0f00b0020 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 --2- 172.21.15.181:0/609137051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f00cdde0 0x7fa0f00c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T10:04:17.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 >> 172.21.15.181:0/609137051 conn(0x7fa0f006c430 msgr2=0x7fa0f00cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T10:04:17.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 shutdown_connections 2023-10-16T10:04:17.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T10:04:17.758+0000 7fa0f5f18700 1 -- 172.21.15.181:0/609137051 wait complete. 2023-10-16T10:04:17.771 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-16T10:04:17.771 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1m'], 'retention': [{'n': 55}]} 2023-10-16T10:04:40.847 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:04:40.850 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:00.385 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:05:00.385+0000 7fbe54918700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:00.386 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:05:00.385+0000 7fbe54918700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:00.386 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:05:00.385+0000 7fbe54918700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:00.386 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:05:00.385+0000 7fbe54918700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:00.387 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T10:05:00.385+0000 7fbe54918700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T10:05:10.916 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:10.920 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:40.981 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:05:40.985 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:06:11.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:06:11.074 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:06:41.142 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:06:41.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:11.213 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:11.217 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:41.290 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:07:41.294 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:11.353 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:11.356 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:41.462 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:08:41.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:09:11.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:09:11.529 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:09:41.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:09:41.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:11.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:11.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:41.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:10:41.849 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:11.935 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:11.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:42.018 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:11:42.022 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:12.082 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:12.086 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:42.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:12:42.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:12.231 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:12.235 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:42.321 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:13:42.324 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:12.394 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:12.398 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:42.466 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:14:42.470 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:12.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:12.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:42.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:15:42.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:12.722 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:12.725 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:42.794 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:16:42.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:12.866 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:12.870 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:42.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:17:42.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:13.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:13.005 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:43.082 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:18:43.086 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:13.179 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:13.182 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:43.242 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:19:43.245 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:13.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:13.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:43.411 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:20:43.414 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:13.475 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:13.478 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:43.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:21:43.550 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:13.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:13.673 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:43.748 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:22:43.751 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:13.821 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:13.824 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:43.891 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:23:43.894 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:13.971 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:13.976 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:44.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:24:44.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:14.167 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:14.170 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:44.245 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:25:44.248 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:14.317 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:14.320 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:44.391 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:26:44.394 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:14.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:14.482 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:44.549 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:27:44.552 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:14.631 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:14.635 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:44.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:28:44.704 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:14.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:14.788 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:44.853 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:29:44.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:14.942 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:14.945 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:45.013 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:30:45.016 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:15.087 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:15.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:45.184 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:31:45.188 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:15.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:15.270 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:45.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:32:45.350 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:15.419 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:15.423 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:45.488 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:33:45.491 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:15.568 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:15.571 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:45.649 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:34:45.652 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:15.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:15.719 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:45.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:35:45.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:15.856 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:15.859 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:45.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:36:45.931 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:15.998 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:16.001 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:46.088 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:37:46.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:16.159 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:16.163 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:46.231 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:38:46.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:16.412 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:16.417 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:46.484 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:39:46.488 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:16.563 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:16.567 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:46.661 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:40:46.665 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:16.750 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:16.754 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:46.816 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:41:46.820 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:16.879 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:16.882 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:46.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:42:46.994 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:17.140 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:17.143 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:47.283 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:43:47.287 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:17.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:17.425 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:47.561 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:44:47.564 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:17.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:17.702 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:47.795 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:45:47.799 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:17.925 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:17.929 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:48.081 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:46:48.085 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:18.227 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:18.233 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:48.366 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:47:48.369 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:18.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:18.499 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:48.625 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:48:48.629 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:18.756 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:18.760 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:48.852 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:49:48.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:19.008 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:19.011 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:49.162 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:50:49.166 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:19.297 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:19.300 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:49.428 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:51:49.432 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:19.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:19.558 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:49.679 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:52:49.683 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:19.779 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:19.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:49.932 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:53:49.935 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:20.082 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:20.085 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:50.140 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:54:50.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:20.273 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:20.277 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:50.426 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:55:50.430 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:20.584 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:20.587 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:50.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:56:50.718 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:20.842 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:20.845 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:50.987 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:57:50.991 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:21.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:21.104 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:51.215 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:58:51.218 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:21.372 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:21.375 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:51.502 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T10:59:51.505 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:15.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:15.773 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:00:15.800 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_06_00_UTC 2023-10-16T11:00:15.800 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_07_00_UTC 2023-10-16T11:00:15.800 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_08_00_UTC 2023-10-16T11:00:15.800 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_09_00_UTC 2023-10-16T11:00:15.800 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_10_00_UTC 2023-10-16T11:00:15.801 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_11_00_UTC 2023-10-16T11:00:15.801 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_12_00_UTC 2023-10-16T11:00:15.801 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_13_00_UTC 2023-10-16T11:00:15.801 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_14_00_UTC 2023-10-16T11:00:15.801 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_15_00_UTC 2023-10-16T11:00:15.801 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_16_00_UTC 2023-10-16T11:00:15.802 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_17_00_UTC 2023-10-16T11:00:15.802 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_18_00_UTC 2023-10-16T11:00:15.802 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_19_00_UTC 2023-10-16T11:00:15.802 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_20_00_UTC 2023-10-16T11:00:15.802 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_21_00_UTC 2023-10-16T11:00:15.803 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_22_00_UTC 2023-10-16T11:00:15.803 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_23_00_UTC 2023-10-16T11:00:15.803 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_24_00_UTC 2023-10-16T11:00:15.803 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_25_00_UTC 2023-10-16T11:00:15.803 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_26_00_UTC 2023-10-16T11:00:15.803 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_27_00_UTC 2023-10-16T11:00:15.804 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_28_00_UTC 2023-10-16T11:00:15.804 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_29_00_UTC 2023-10-16T11:00:15.804 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_30_00_UTC 2023-10-16T11:00:15.804 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_31_00_UTC 2023-10-16T11:00:15.805 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_32_00_UTC 2023-10-16T11:00:15.805 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_33_00_UTC 2023-10-16T11:00:15.805 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_34_00_UTC 2023-10-16T11:00:15.805 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_35_00_UTC 2023-10-16T11:00:15.805 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_36_00_UTC 2023-10-16T11:00:15.806 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_37_00_UTC 2023-10-16T11:00:15.806 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_38_00_UTC 2023-10-16T11:00:15.806 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_39_00_UTC 2023-10-16T11:00:15.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_40_00_UTC 2023-10-16T11:00:15.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_41_00_UTC 2023-10-16T11:00:15.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_42_00_UTC 2023-10-16T11:00:15.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_43_00_UTC 2023-10-16T11:00:15.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_44_00_UTC 2023-10-16T11:00:15.807 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_45_00_UTC 2023-10-16T11:00:15.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_46_00_UTC 2023-10-16T11:00:15.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_47_00_UTC 2023-10-16T11:00:15.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_48_00_UTC 2023-10-16T11:00:15.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_49_00_UTC 2023-10-16T11:00:15.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_50_00_UTC 2023-10-16T11:00:15.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_51_00_UTC 2023-10-16T11:00:15.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_52_00_UTC 2023-10-16T11:00:15.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_53_00_UTC 2023-10-16T11:00:15.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_54_00_UTC 2023-10-16T11:00:15.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_55_00_UTC 2023-10-16T11:00:15.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_56_00_UTC 2023-10-16T11:00:15.809 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_57_00_UTC 2023-10-16T11:00:15.810 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_58_00_UTC 2023-10-16T11:00:15.810 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_59_00_UTC 2023-10-16T11:00:15.810 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_00_00_UTC 2023-10-16T11:00:15.810 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-10_06_00_UTC', 'scheduled-2023-10-16-10_07_00_UTC', 'scheduled-2023-10-16-10_08_00_UTC', 'scheduled-2023-10-16-10_09_00_UTC', 'scheduled-2023-10-16-10_10_00_UTC', 'scheduled-2023-10-16-10_11_00_UTC', 'scheduled-2023-10-16-10_12_00_UTC', 'scheduled-2023-10-16-10_13_00_UTC', 'scheduled-2023-10-16-10_14_00_UTC', 'scheduled-2023-10-16-10_15_00_UTC', 'scheduled-2023-10-16-10_16_00_UTC', 'scheduled-2023-10-16-10_17_00_UTC', 'scheduled-2023-10-16-10_18_00_UTC', 'scheduled-2023-10-16-10_19_00_UTC', 'scheduled-2023-10-16-10_20_00_UTC', 'scheduled-2023-10-16-10_21_00_UTC', 'scheduled-2023-10-16-10_22_00_UTC', 'scheduled-2023-10-16-10_23_00_UTC', 'scheduled-2023-10-16-10_24_00_UTC', 'scheduled-2023-10-16-10_25_00_UTC', 'scheduled-2023-10-16-10_26_00_UTC', 'scheduled-2023-10-16-10_27_00_UTC', 'scheduled-2023-10-16-10_28_00_UTC', 'scheduled-2023-10-16-10_29_00_UTC', 'scheduled-2023-10-16-10_30_00_UTC', 'scheduled-2023-10-16-10_31_00_UTC', 'scheduled-2023-10-16-10_32_00_UTC', 'scheduled-2023-10-16-10_33_00_UTC', 'scheduled-2023-10-16-10_34_00_UTC', 'scheduled-2023-10-16-10_35_00_UTC', 'scheduled-2023-10-16-10_36_00_UTC', 'scheduled-2023-10-16-10_37_00_UTC', 'scheduled-2023-10-16-10_38_00_UTC', 'scheduled-2023-10-16-10_39_00_UTC', 'scheduled-2023-10-16-10_40_00_UTC', 'scheduled-2023-10-16-10_41_00_UTC', 'scheduled-2023-10-16-10_42_00_UTC', 'scheduled-2023-10-16-10_43_00_UTC', 'scheduled-2023-10-16-10_44_00_UTC', 'scheduled-2023-10-16-10_45_00_UTC', 'scheduled-2023-10-16-10_46_00_UTC', 'scheduled-2023-10-16-10_47_00_UTC', 'scheduled-2023-10-16-10_48_00_UTC', 'scheduled-2023-10-16-10_49_00_UTC', 'scheduled-2023-10-16-10_50_00_UTC', 'scheduled-2023-10-16-10_51_00_UTC', 'scheduled-2023-10-16-10_52_00_UTC', 'scheduled-2023-10-16-10_53_00_UTC', 'scheduled-2023-10-16-10_54_00_UTC', 'scheduled-2023-10-16-10_55_00_UTC', 'scheduled-2023-10-16-10_56_00_UTC', 'scheduled-2023-10-16-10_57_00_UTC', 'scheduled-2023-10-16-10_58_00_UTC', 'scheduled-2023-10-16-10_59_00_UTC', 'scheduled-2023-10-16-11_00_00_UTC'] 2023-10-16T11:00:15.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-10-16T11:00:15.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.972+0000 7fbae6dca700 1 Processor -- start 2023-10-16T11:00:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.973+0000 7fbae6dca700 1 -- start start 2023-10-16T11:00:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.973+0000 7fbae6dca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbae00d01d0 0x7fbae00ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.973+0000 7fbae6dca700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbae015ea40 0x7fbae00d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.973+0000 7fbae6dca700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae00cf580 con 0x7fbae0161530 2023-10-16T11:00:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.973+0000 7fbae6dca700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae00d4290 con 0x7fbae00d01d0 2023-10-16T11:00:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.973+0000 7fbae6dca700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae00d0af0 con 0x7fbae015ea40 2023-10-16T11:00:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.973+0000 7fbae4b66700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbae00d01d0 0x7fbae00ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbae4b66700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbae00d01d0 0x7fbae00ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49862/0 (socket says 172.21.15.181:49862) 2023-10-16T11:00:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbae4b66700 1 -- 172.21.15.181:0/1635642219 learned_addr learned my addr 172.21.15.181:0/1635642219 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbadffff700 1 --2- 172.21.15.181:0/1635642219 >> v2:172.21.15.190:3300/0 conn(0x7fbae015ea40 0x7fbae00d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbae4b66700 1 -- 172.21.15.181:0/1635642219 >> v2:172.21.15.190:3300/0 conn(0x7fbae015ea40 msgr2=0x7fbae00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbae4b66700 1 --2- 172.21.15.181:0/1635642219 >> v2:172.21.15.190:3300/0 conn(0x7fbae015ea40 0x7fbae00d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbae4b66700 1 -- 172.21.15.181:0/1635642219 >> v1:172.21.15.190:6789/0 conn(0x7fbae0161530 legacy=0x7fbae01619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbae4b66700 1 -- 172.21.15.181:0/1635642219 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00ce970 con 0x7fbae00d01d0 2023-10-16T11:00:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbae4b66700 1 --2- 172.21.15.181:0/1635642219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae00d01d0 0x7fbae00ce2d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbae00d12c0 tx=0x7fbad4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359363086848b436 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbadf7fe700 1 -- 172.21.15.181:0/1635642219 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400b870 con 0x7fbae00d01d0 2023-10-16T11:00:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbadf7fe700 1 -- 172.21.15.181:0/1635642219 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad400b9d0 con 0x7fbae00d01d0 2023-10-16T11:00:15.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.974+0000 7fbadf7fe700 1 -- 172.21.15.181:0/1635642219 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400bd60 con 0x7fbae00d01d0 2023-10-16T11:00:15.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 -- 172.21.15.181:0/1635642219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae00d01d0 msgr2=0x7fbae00ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:15.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1635642219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae00d01d0 0x7fbae00ce2d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbae00d12c0 tx=0x7fbad4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:15.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 -- 172.21.15.181:0/1635642219 shutdown_connections 2023-10-16T11:00:15.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1635642219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae00d01d0 0x7fbae00ce2d0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:15.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1635642219 >> v2:172.21.15.190:3300/0 conn(0x7fbae015ea40 0x7fbae00d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:15.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 -- 172.21.15.181:0/1635642219 >> 172.21.15.181:0/1635642219 conn(0x7fbae009cb60 msgr2=0x7fbae00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:15.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 -- 172.21.15.181:0/1635642219 shutdown_connections 2023-10-16T11:00:15.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.975+0000 7fbae6dca700 1 -- 172.21.15.181:0/1635642219 wait complete. 2023-10-16T11:00:15.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.976+0000 7fbae6dca700 1 Processor -- start 2023-10-16T11:00:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.976+0000 7fbae6dca700 1 -- start start 2023-10-16T11:00:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae6dca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae00d01d0 0x7fbae00b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae6dca700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae015ea40 0x7fbae00c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae6dca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae0161530 0x7fbae00c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae6dca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00b5f00 con 0x7fbae0161530 2023-10-16T11:00:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae6dca700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00cc380 con 0x7fbae00d01d0 2023-10-16T11:00:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae6dca700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00cd460 con 0x7fbae015ea40 2023-10-16T11:00:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae0161530 0x7fbae00c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae0161530 0x7fbae00c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49872/0 (socket says 172.21.15.181:49872) 2023-10-16T11:00:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 -- 172.21.15.181:0/1275727357 learned_addr learned my addr 172.21.15.181:0/1275727357 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae4b66700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae00d01d0 0x7fbae00b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbadffff700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae015ea40 0x7fbae00c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 -- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae015ea40 msgr2=0x7fbae00c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae015ea40 0x7fbae00c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 -- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae00d01d0 msgr2=0x7fbae00b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae00d01d0 0x7fbae00b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 -- 172.21.15.181:0/1275727357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad4009550 con 0x7fbae0161530 2023-10-16T11:00:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.977+0000 7fbae5367700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae0161530 0x7fbae00c7830 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbad800cb50 tx=0x7fbad800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.978+0000 7fbadd7fa700 1 -- 172.21.15.181:0/1275727357 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8003c00 con 0x7fbae0161530 2023-10-16T11:00:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.978+0000 7fbadd7fa700 1 -- 172.21.15.181:0/1275727357 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad8003d60 con 0x7fbae0161530 2023-10-16T11:00:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.978+0000 7fbadd7fa700 1 -- 172.21.15.181:0/1275727357 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad800aa20 con 0x7fbae0161530 2023-10-16T11:00:15.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.978+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae00cba90 con 0x7fbae0161530 2023-10-16T11:00:15.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.978+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae00cbd30 con 0x7fbae0161530 2023-10-16T11:00:15.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.979+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae00cc4f0 con 0x7fbae0161530 2023-10-16T11:00:15.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.979+0000 7fbadd7fa700 1 -- 172.21.15.181:0/1275727357 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 94) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fbad800ab80 con 0x7fbae0161530 2023-10-16T11:00:15.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.979+0000 7fbadd7fa700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fbac406b070 0x7fbac406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:15.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.979+0000 7fbadd7fa700 1 -- 172.21.15.181:0/1275727357 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6607+0+0 (secure 0 0 0) 0x7fbad8089b00 con 0x7fbae0161530 2023-10-16T11:00:15.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.979+0000 7fbae4b66700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fbac406b070 0x7fbac406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:15.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.980+0000 7fbae4b66700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fbac406b070 0x7fbac406d530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad4008030 comp rx=0 tx=0).ready entity=mgr.9294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:15.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:15.984+0000 7fbadd7fa700 1 -- 172.21.15.181:0/1275727357 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbad8054720 con 0x7fbae0161530 2023-10-16T11:00:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.201+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 --> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbae00cef30 con 0x7fbac406b070 2023-10-16T11:00:16.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.210+0000 7fbadd7fa700 1 -- 172.21.15.181:0/1275727357 <== mgr.9294 v2:172.21.15.181:6832/1123221928 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+373 (secure 0 0 0) 0x7fbad400bb40 con 0x7fbac406b070 2023-10-16T11:00:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:16.211 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:04:16", "first": "2023-10-16T10:05:00", "last": "2023-10-16T11:00:00", "last_pruned": "2023-10-16T11:00:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-16T11:00:16.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fbac406b070 msgr2=0x7fbac406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:16.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fbac406b070 0x7fbac406d530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad4008030 comp rx=0 tx=0).stop 2023-10-16T11:00:16.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae0161530 msgr2=0x7fbae00c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:16.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae0161530 0x7fbae00c7830 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbad800cb50 tx=0x7fbad800ce60 comp rx=0 tx=0).stop 2023-10-16T11:00:16.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 shutdown_connections 2023-10-16T11:00:16.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fbac406b070 0x7fbac406d530 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae0161530 0x7fbae00c7830 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbae00d01d0 0x7fbae00b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 --2- 172.21.15.181:0/1275727357 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbae015ea40 0x7fbae00c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 >> 172.21.15.181:0/1275727357 conn(0x7fbae009cb60 msgr2=0x7fbae006d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:16.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 shutdown_connections 2023-10-16T11:00:16.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.212+0000 7fbae6dca700 1 -- 172.21.15.181:0/1275727357 wait complete. 2023-10-16T11:00:16.224 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T10:04:16", "first": "2023-10-16T10:05:00", "last": "2023-10-16T11:00:00", "last_pruned": "2023-10-16T11:00:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-16T11:00:16.225 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1m', 'retention': {'n': 55}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T10:04:16', 'first': '2023-10-16T10:05:00', 'last': '2023-10-16T11:00:00', 'last_pruned': '2023-10-16T11:00:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-16T11:00:16.225 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-16T11:00:16.225 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-16T11:00:16.226 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-10-16T11:00:16.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.347+0000 7fd558ecb700 1 Processor -- start 2023-10-16T11:00:16.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.347+0000 7fd558ecb700 1 -- start start 2023-10-16T11:00:16.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.348+0000 7fd558ecb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd55415f150 0x7fd5540cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:16.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.348+0000 7fd558ecb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5540ce4f0 0x7fd5540b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:16.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.348+0000 7fd558ecb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540b14a0 0x7fd5540b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:16.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.348+0000 7fd558ecb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5540ce9a0 con 0x7fd5540ce4f0 2023-10-16T11:00:16.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.348+0000 7fd558ecb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5540b2ae0 con 0x7fd55415f150 2023-10-16T11:00:16.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.348+0000 7fd558ecb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540b2dc0 con 0x7fd5540b14a0 2023-10-16T11:00:16.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.348+0000 7fd551d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5540ce4f0 0x7fd5540b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:16.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5540ce4f0 0x7fd5540b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49878/0 (socket says 172.21.15.181:49878) 2023-10-16T11:00:16.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 -- 172.21.15.181:0/4237298148 learned_addr learned my addr 172.21.15.181:0/4237298148 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:16.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd552d9d700 1 --2- 172.21.15.181:0/4237298148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540b14a0 0x7fd5540b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:16.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd55259c700 1 --2- 172.21.15.181:0/4237298148 >> v2:172.21.15.190:3300/0 conn(0x7fd55415f150 0x7fd5540cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:16.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 -- 172.21.15.181:0/4237298148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540b14a0 msgr2=0x7fd5540b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:16.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 --2- 172.21.15.181:0/4237298148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540b14a0 0x7fd5540b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 -- 172.21.15.181:0/4237298148 >> v2:172.21.15.190:3300/0 conn(0x7fd55415f150 msgr2=0x7fd5540cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:16.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 --2- 172.21.15.181:0/4237298148 >> v2:172.21.15.190:3300/0 conn(0x7fd55415f150 0x7fd5540cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 -- 172.21.15.181:0/4237298148 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540cf9d0 con 0x7fd5540ce4f0 2023-10-16T11:00:16.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd55259c700 1 --2- 172.21.15.181:0/4237298148 >> v2:172.21.15.190:3300/0 conn(0x7fd55415f150 0x7fd5540cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:16.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd552d9d700 1 --2- 172.21.15.181:0/4237298148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540b14a0 0x7fd5540b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:16.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd551d9b700 1 --2- 172.21.15.181:0/4237298148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540ce4f0 0x7fd5540b0f60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fd5480098f0 tx=0x7fd548009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12437ea451511d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:16.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd55159a700 1 -- 172.21.15.181:0/4237298148 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800b860 con 0x7fd5540ce4f0 2023-10-16T11:00:16.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd55159a700 1 -- 172.21.15.181:0/4237298148 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54800b9c0 con 0x7fd5540ce4f0 2023-10-16T11:00:16.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd55159a700 1 -- 172.21.15.181:0/4237298148 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800bd50 con 0x7fd5540ce4f0 2023-10-16T11:00:16.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd558ecb700 1 -- 172.21.15.181:0/4237298148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540ce4f0 msgr2=0x7fd5540b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:16.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.349+0000 7fd558ecb700 1 --2- 172.21.15.181:0/4237298148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540ce4f0 0x7fd5540b0f60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fd5480098f0 tx=0x7fd548009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:16.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.350+0000 7fd558ecb700 1 -- 172.21.15.181:0/4237298148 shutdown_connections 2023-10-16T11:00:16.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.350+0000 7fd558ecb700 1 --2- 172.21.15.181:0/4237298148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540b14a0 0x7fd5540b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.350+0000 7fd558ecb700 1 --2- 172.21.15.181:0/4237298148 >> v2:172.21.15.190:3300/0 conn(0x7fd55415f150 0x7fd5540cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.350+0000 7fd558ecb700 1 --2- 172.21.15.181:0/4237298148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540ce4f0 0x7fd5540b0f60 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.350+0000 7fd558ecb700 1 -- 172.21.15.181:0/4237298148 >> 172.21.15.181:0/4237298148 conn(0x7fd55409a930 msgr2=0x7fd5540d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:16.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.350+0000 7fd558ecb700 1 -- 172.21.15.181:0/4237298148 shutdown_connections 2023-10-16T11:00:16.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.350+0000 7fd558ecb700 1 -- 172.21.15.181:0/4237298148 wait complete. 2023-10-16T11:00:16.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.351+0000 7fd558ecb700 1 Processor -- start 2023-10-16T11:00:16.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.351+0000 7fd558ecb700 1 -- start start 2023-10-16T11:00:16.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd558ecb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b14a0 0x7fd55415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:16.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd558ecb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540ce4f0 0x7fd55415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:16.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd558ecb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55415f150 0x7fd55414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:16.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd558ecb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55406ba40 con 0x7fd5540b14a0 2023-10-16T11:00:16.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd558ecb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55415dcc0 con 0x7fd55415f150 2023-10-16T11:00:16.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd558ecb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd554158f70 con 0x7fd5540ce4f0 2023-10-16T11:00:16.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b14a0 0x7fd55415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:16.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b14a0 0x7fd55415bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49882/0 (socket says 172.21.15.181:49882) 2023-10-16T11:00:16.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 -- 172.21.15.181:0/668114412 learned_addr learned my addr 172.21.15.181:0/668114412 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:16.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd551d9b700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540ce4f0 0x7fd55415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:16.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd552d9d700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55415f150 0x7fd55414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:16.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 -- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540ce4f0 msgr2=0x7fd55415d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:16.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540ce4f0 0x7fd55415d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 -- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55415f150 msgr2=0x7fd55414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:16.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55415f150 0x7fd55414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 -- 172.21.15.181:0/668114412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd548009550 con 0x7fd5540b14a0 2023-10-16T11:00:16.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd552d9d700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55415f150 0x7fd55414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:16.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd55259c700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b14a0 0x7fd55415bf00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fd54400c780 tx=0x7fd54400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:16.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd542ffd700 1 -- 172.21.15.181:0/668114412 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544011800 con 0x7fd5540b14a0 2023-10-16T11:00:16.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd542ffd700 1 -- 172.21.15.181:0/668114412 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544011960 con 0x7fd5540b14a0 2023-10-16T11:00:16.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd542ffd700 1 -- 172.21.15.181:0/668114412 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544011cd0 con 0x7fd5540b14a0 2023-10-16T11:00:16.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.352+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55415df10 con 0x7fd5540b14a0 2023-10-16T11:00:16.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.353+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55415e4d0 con 0x7fd5540b14a0 2023-10-16T11:00:16.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.353+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd554067d20 con 0x7fd5540b14a0 2023-10-16T11:00:16.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.354+0000 7fd542ffd700 1 -- 172.21.15.181:0/668114412 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 94) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fd544021350 con 0x7fd5540b14a0 2023-10-16T11:00:16.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.354+0000 7fd542ffd700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd53406b070 0x7fd53406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:16.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.355+0000 7fd551d9b700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd53406b070 0x7fd53406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:16.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.355+0000 7fd542ffd700 1 -- 172.21.15.181:0/668114412 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6607+0+0 (secure 0 0 0) 0x7fd544089af0 con 0x7fd5540b14a0 2023-10-16T11:00:16.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.355+0000 7fd551d9b700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd53406b070 0x7fd53406d530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd548008030 comp rx=0 tx=0).ready entity=mgr.9294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:16.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.359+0000 7fd542ffd700 1 -- 172.21.15.181:0/668114412 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd544054690 con 0x7fd5540b14a0 2023-10-16T11:00:16.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.580+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 --> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5540cff90 con 0x7fd53406b070 2023-10-16T11:00:16.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.659+0000 7fd542ffd700 1 -- 172.21.15.181:0/668114412 <== mgr.9294 v2:172.21.15.181:6832/1123221928 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7fd54800bce0 con 0x7fd53406b070 2023-10-16T11:00:16.660 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1 2023-10-16T11:00:16.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd53406b070 msgr2=0x7fd53406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:16.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd53406b070 0x7fd53406d530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd548008030 comp rx=0 tx=0).stop 2023-10-16T11:00:16.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b14a0 msgr2=0x7fd55415bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:16.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b14a0 0x7fd55415bf00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fd54400c780 tx=0x7fd54400ca90 comp rx=0 tx=0).stop 2023-10-16T11:00:16.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 shutdown_connections 2023-10-16T11:00:16.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd53406b070 0x7fd53406d530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd55415f150 0x7fd55414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b14a0 0x7fd55415bf00 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 --2- 172.21.15.181:0/668114412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5540ce4f0 0x7fd55415d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:16.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 >> 172.21.15.181:0/668114412 conn(0x7fd55409a930 msgr2=0x7fd5540700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:16.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 shutdown_connections 2023-10-16T11:00:16.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:16.662+0000 7fd558ecb700 1 -- 172.21.15.181:0/668114412 wait complete. 2023-10-16T11:00:16.674 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-16T11:00:16.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:16.675 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:00:16.695 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_06_00_UTC 2023-10-16T11:00:16.695 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_07_00_UTC 2023-10-16T11:00:16.695 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_08_00_UTC 2023-10-16T11:00:16.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_09_00_UTC 2023-10-16T11:00:16.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_10_00_UTC 2023-10-16T11:00:16.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_11_00_UTC 2023-10-16T11:00:16.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_12_00_UTC 2023-10-16T11:00:16.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_13_00_UTC 2023-10-16T11:00:16.696 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_14_00_UTC 2023-10-16T11:00:16.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_15_00_UTC 2023-10-16T11:00:16.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_16_00_UTC 2023-10-16T11:00:16.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_17_00_UTC 2023-10-16T11:00:16.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_18_00_UTC 2023-10-16T11:00:16.697 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_19_00_UTC 2023-10-16T11:00:16.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_20_00_UTC 2023-10-16T11:00:16.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_21_00_UTC 2023-10-16T11:00:16.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_22_00_UTC 2023-10-16T11:00:16.698 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_23_00_UTC 2023-10-16T11:00:16.699 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_24_00_UTC 2023-10-16T11:00:16.699 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_25_00_UTC 2023-10-16T11:00:16.699 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_26_00_UTC 2023-10-16T11:00:16.699 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_27_00_UTC 2023-10-16T11:00:16.699 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_28_00_UTC 2023-10-16T11:00:16.699 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_29_00_UTC 2023-10-16T11:00:16.700 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_30_00_UTC 2023-10-16T11:00:16.700 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_31_00_UTC 2023-10-16T11:00:16.700 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_32_00_UTC 2023-10-16T11:00:16.700 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_33_00_UTC 2023-10-16T11:00:16.700 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_34_00_UTC 2023-10-16T11:00:16.701 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_35_00_UTC 2023-10-16T11:00:16.701 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_36_00_UTC 2023-10-16T11:00:16.701 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_37_00_UTC 2023-10-16T11:00:16.701 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_38_00_UTC 2023-10-16T11:00:16.701 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_39_00_UTC 2023-10-16T11:00:16.701 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_40_00_UTC 2023-10-16T11:00:16.702 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_41_00_UTC 2023-10-16T11:00:16.702 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_42_00_UTC 2023-10-16T11:00:16.702 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_43_00_UTC 2023-10-16T11:00:16.702 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_44_00_UTC 2023-10-16T11:00:16.702 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_45_00_UTC 2023-10-16T11:00:16.703 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_46_00_UTC 2023-10-16T11:00:16.703 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_47_00_UTC 2023-10-16T11:00:16.703 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_48_00_UTC 2023-10-16T11:00:16.703 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_49_00_UTC 2023-10-16T11:00:16.704 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_50_00_UTC 2023-10-16T11:00:16.704 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_51_00_UTC 2023-10-16T11:00:16.704 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_52_00_UTC 2023-10-16T11:00:16.704 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_53_00_UTC 2023-10-16T11:00:16.704 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_54_00_UTC 2023-10-16T11:00:16.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_55_00_UTC 2023-10-16T11:00:16.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_56_00_UTC 2023-10-16T11:00:16.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_57_00_UTC 2023-10-16T11:00:16.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_58_00_UTC 2023-10-16T11:00:16.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-10_59_00_UTC 2023-10-16T11:00:16.705 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_00_00_UTC 2023-10-16T11:00:16.706 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_06_00_UTC 2023-10-16T11:00:16.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:16.706 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_06_00_UTC) 2023-10-16T11:00:16.758 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_07_00_UTC 2023-10-16T11:00:16.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:16.759 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_07_00_UTC) 2023-10-16T11:00:16.821 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_08_00_UTC 2023-10-16T11:00:16.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:16.822 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_08_00_UTC) 2023-10-16T11:00:16.883 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_09_00_UTC 2023-10-16T11:00:16.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:16.884 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_09_00_UTC) 2023-10-16T11:00:16.945 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_10_00_UTC 2023-10-16T11:00:16.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:16.946 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_10_00_UTC) 2023-10-16T11:00:17.007 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_11_00_UTC 2023-10-16T11:00:17.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.008 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_11_00_UTC) 2023-10-16T11:00:17.069 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_12_00_UTC 2023-10-16T11:00:17.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.070 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_12_00_UTC) 2023-10-16T11:00:17.131 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_13_00_UTC 2023-10-16T11:00:17.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.132 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_13_00_UTC) 2023-10-16T11:00:17.195 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_14_00_UTC 2023-10-16T11:00:17.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.195 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_14_00_UTC) 2023-10-16T11:00:17.257 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_15_00_UTC 2023-10-16T11:00:17.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.258 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_15_00_UTC) 2023-10-16T11:00:17.319 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_16_00_UTC 2023-10-16T11:00:17.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.320 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_16_00_UTC) 2023-10-16T11:00:17.381 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_17_00_UTC 2023-10-16T11:00:17.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.382 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_17_00_UTC) 2023-10-16T11:00:17.443 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_18_00_UTC 2023-10-16T11:00:17.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.444 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_18_00_UTC) 2023-10-16T11:00:17.506 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_19_00_UTC 2023-10-16T11:00:17.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.506 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_19_00_UTC) 2023-10-16T11:00:17.569 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_20_00_UTC 2023-10-16T11:00:17.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.570 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_20_00_UTC) 2023-10-16T11:00:17.632 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_21_00_UTC 2023-10-16T11:00:17.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.633 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_21_00_UTC) 2023-10-16T11:00:17.696 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_22_00_UTC 2023-10-16T11:00:17.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.697 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_22_00_UTC) 2023-10-16T11:00:17.761 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_23_00_UTC 2023-10-16T11:00:17.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.763 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_23_00_UTC) 2023-10-16T11:00:17.836 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_24_00_UTC 2023-10-16T11:00:17.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.836 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_24_00_UTC) 2023-10-16T11:00:17.909 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_25_00_UTC 2023-10-16T11:00:17.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.910 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_25_00_UTC) 2023-10-16T11:00:17.982 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_26_00_UTC 2023-10-16T11:00:17.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:17.982 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_26_00_UTC) 2023-10-16T11:00:18.044 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_27_00_UTC 2023-10-16T11:00:18.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.045 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_27_00_UTC) 2023-10-16T11:00:18.106 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_28_00_UTC 2023-10-16T11:00:18.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.106 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_28_00_UTC) 2023-10-16T11:00:18.168 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_29_00_UTC 2023-10-16T11:00:18.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.169 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_29_00_UTC) 2023-10-16T11:00:18.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_30_00_UTC 2023-10-16T11:00:18.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.231 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_30_00_UTC) 2023-10-16T11:00:18.292 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_31_00_UTC 2023-10-16T11:00:18.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.292 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_31_00_UTC) 2023-10-16T11:00:18.353 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_32_00_UTC 2023-10-16T11:00:18.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.354 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_32_00_UTC) 2023-10-16T11:00:18.415 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_33_00_UTC 2023-10-16T11:00:18.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.416 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_33_00_UTC) 2023-10-16T11:00:18.477 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_34_00_UTC 2023-10-16T11:00:18.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.477 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_34_00_UTC) 2023-10-16T11:00:18.539 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_35_00_UTC 2023-10-16T11:00:18.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.540 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_35_00_UTC) 2023-10-16T11:00:18.600 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_36_00_UTC 2023-10-16T11:00:18.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.601 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_36_00_UTC) 2023-10-16T11:00:18.661 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_37_00_UTC 2023-10-16T11:00:18.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.661 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_37_00_UTC) 2023-10-16T11:00:18.724 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_38_00_UTC 2023-10-16T11:00:18.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.724 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_38_00_UTC) 2023-10-16T11:00:18.793 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_39_00_UTC 2023-10-16T11:00:18.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.793 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_39_00_UTC) 2023-10-16T11:00:18.855 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_40_00_UTC 2023-10-16T11:00:18.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.855 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_40_00_UTC) 2023-10-16T11:00:18.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_41_00_UTC 2023-10-16T11:00:18.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.917 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_41_00_UTC) 2023-10-16T11:00:18.978 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_42_00_UTC 2023-10-16T11:00:18.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:18.979 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_42_00_UTC) 2023-10-16T11:00:19.041 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_43_00_UTC 2023-10-16T11:00:19.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.041 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_43_00_UTC) 2023-10-16T11:00:19.103 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_44_00_UTC 2023-10-16T11:00:19.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.103 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_44_00_UTC) 2023-10-16T11:00:19.165 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_45_00_UTC 2023-10-16T11:00:19.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.166 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_45_00_UTC) 2023-10-16T11:00:19.228 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_46_00_UTC 2023-10-16T11:00:19.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.229 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_46_00_UTC) 2023-10-16T11:00:19.290 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_47_00_UTC 2023-10-16T11:00:19.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.291 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_47_00_UTC) 2023-10-16T11:00:19.353 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_48_00_UTC 2023-10-16T11:00:19.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.354 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_48_00_UTC) 2023-10-16T11:00:19.417 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_49_00_UTC 2023-10-16T11:00:19.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.417 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_49_00_UTC) 2023-10-16T11:00:19.480 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_50_00_UTC 2023-10-16T11:00:19.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.480 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_50_00_UTC) 2023-10-16T11:00:19.542 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_51_00_UTC 2023-10-16T11:00:19.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.543 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_51_00_UTC) 2023-10-16T11:00:19.605 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_52_00_UTC 2023-10-16T11:00:19.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.606 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_52_00_UTC) 2023-10-16T11:00:19.667 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_53_00_UTC 2023-10-16T11:00:19.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.668 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_53_00_UTC) 2023-10-16T11:00:19.730 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_54_00_UTC 2023-10-16T11:00:19.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.731 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_54_00_UTC) 2023-10-16T11:00:19.816 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_55_00_UTC 2023-10-16T11:00:19.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.817 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_55_00_UTC) 2023-10-16T11:00:19.860 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_56_00_UTC 2023-10-16T11:00:19.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.861 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_56_00_UTC) 2023-10-16T11:00:19.922 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_57_00_UTC 2023-10-16T11:00:19.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.923 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_57_00_UTC) 2023-10-16T11:00:19.984 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_58_00_UTC 2023-10-16T11:00:19.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:19.985 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_58_00_UTC) 2023-10-16T11:00:20.046 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-10_59_00_UTC 2023-10-16T11:00:20.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:20.047 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-10_59_00_UTC) 2023-10-16T11:00:20.107 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_00_00_UTC 2023-10-16T11:00:20.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:20.109 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_00_00_UTC) 2023-10-16T11:00:20.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:20.169 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:00:20.229 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:00:20.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.390+0000 7f2dd1a9a700 1 Processor -- start 2023-10-16T11:00:20.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.390+0000 7f2dd1a9a700 1 -- start start 2023-10-16T11:00:20.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dd1a9a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2dcc0d01d0 0x7f2dcc0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:20.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dd1a9a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2dcc15ea40 0x7f2dcc0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:20.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dd1a9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc161530 0x7f2dcc1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:20.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dd1a9a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0d4290 con 0x7f2dcc15ea40 2023-10-16T11:00:20.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dd1a9a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0cf580 con 0x7f2dcc0d01d0 2023-10-16T11:00:20.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dd1a9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0d0af0 con 0x7f2dcc161530 2023-10-16T11:00:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dcaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2dcc15ea40 0x7f2dcc0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dcaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2dcc15ea40 0x7f2dcc0d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49890/0 (socket says 172.21.15.181:49890) 2023-10-16T11:00:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.391+0000 7f2dcaffd700 1 -- 172.21.15.181:0/2396624198 learned_addr learned my addr 172.21.15.181:0/2396624198 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcbfff700 1 --2- 172.21.15.181:0/2396624198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc161530 0x7f2dcc1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcb7fe700 1 --2- 172.21.15.181:0/2396624198 >> v2:172.21.15.190:3300/0 conn(0x7f2dcc0d01d0 0x7f2dcc0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:20.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcaffd700 1 -- 172.21.15.181:0/2396624198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc161530 msgr2=0x7f2dcc1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:20.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcaffd700 1 --2- 172.21.15.181:0/2396624198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc161530 0x7f2dcc1619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcaffd700 1 -- 172.21.15.181:0/2396624198 >> v2:172.21.15.190:3300/0 conn(0x7f2dcc0d01d0 msgr2=0x7f2dcc0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:20.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcaffd700 1 --2- 172.21.15.181:0/2396624198 >> v2:172.21.15.190:3300/0 conn(0x7f2dcc0d01d0 0x7f2dcc0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcaffd700 1 -- 172.21.15.181:0/2396624198 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0ce970 con 0x7f2dcc15ea40 2023-10-16T11:00:20.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dcaffd700 1 --2- 172.21.15.181:0/2396624198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0d05b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc08e050 tx=0x7f2db4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796a59f184c65918 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dca7fc700 1 -- 172.21.15.181:0/2396624198 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4011830 con 0x7f2dcc15ea40 2023-10-16T11:00:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dca7fc700 1 -- 172.21.15.181:0/2396624198 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4011990 con 0x7f2dcc15ea40 2023-10-16T11:00:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.392+0000 7f2dca7fc700 1 -- 172.21.15.181:0/2396624198 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4011d20 con 0x7f2dcc15ea40 2023-10-16T11:00:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/2396624198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 msgr2=0x7f2dcc0d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/2396624198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0d05b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc08e050 tx=0x7f2db4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/2396624198 shutdown_connections 2023-10-16T11:00:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/2396624198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc161530 0x7f2dcc1619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/2396624198 >> v2:172.21.15.190:3300/0 conn(0x7f2dcc0d01d0 0x7f2dcc0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/2396624198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0d05b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/2396624198 >> 172.21.15.181:0/2396624198 conn(0x7f2dcc09cb60 msgr2=0x7f2dcc0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/2396624198 shutdown_connections 2023-10-16T11:00:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.393+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/2396624198 wait complete. 2023-10-16T11:00:20.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.394+0000 7f2dd1a9a700 1 Processor -- start 2023-10-16T11:00:20.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.394+0000 7f2dd1a9a700 1 -- start start 2023-10-16T11:00:20.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dd1a9a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc0d01d0 0x7f2dcc147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:20.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dd1a9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dd1a9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dcc161530 0x7f2dcc147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dd1a9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0cc9a0 con 0x7f2dcc15ea40 2023-10-16T11:00:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dd1a9a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0cadd0 con 0x7f2dcc161530 2023-10-16T11:00:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dd1a9a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0b2190 con 0x7f2dcc0d01d0 2023-10-16T11:00:20.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dcaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dcaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49898/0 (socket says 172.21.15.181:49898) 2023-10-16T11:00:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dcaffd700 1 -- 172.21.15.181:0/3767008208 learned_addr learned my addr 172.21.15.181:0/3767008208 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dcb7fe700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc0d01d0 0x7f2dcc147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.395+0000 7f2dcbfff700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dcc161530 0x7f2dcc147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dcaffd700 1 -- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc0d01d0 msgr2=0x7f2dcc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:20.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dcaffd700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc0d01d0 0x7f2dcc147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dcaffd700 1 -- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dcc161530 msgr2=0x7f2dcc147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dcaffd700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dcc161530 0x7f2dcc147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dcaffd700 1 -- 172.21.15.181:0/3767008208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4009550 con 0x7f2dcc15ea40 2023-10-16T11:00:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dcaffd700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0ca890 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009c10 tx=0x7f2db4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:20.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dc3fff700 1 -- 172.21.15.181:0/3767008208 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4004450 con 0x7f2dcc15ea40 2023-10-16T11:00:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dc3fff700 1 -- 172.21.15.181:0/3767008208 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db401e070 con 0x7f2dcc15ea40 2023-10-16T11:00:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc0b2470 con 0x7f2dcc15ea40 2023-10-16T11:00:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dc3fff700 1 -- 172.21.15.181:0/3767008208 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40038e0 con 0x7f2dcc15ea40 2023-10-16T11:00:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.396+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc0cc220 con 0x7f2dcc15ea40 2023-10-16T11:00:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.397+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc158bd0 con 0x7f2dcc15ea40 2023-10-16T11:00:20.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.397+0000 7f2dc3fff700 1 -- 172.21.15.181:0/3767008208 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 94) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f2db4003a40 con 0x7f2dcc15ea40 2023-10-16T11:00:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.398+0000 7f2dc3fff700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f2dac06b070 0x7f2dac06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.398+0000 7f2dc3fff700 1 -- 172.21.15.181:0/3767008208 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f2db408a780 con 0x7f2dcc15ea40 2023-10-16T11:00:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.398+0000 7f2dcb7fe700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f2dac06b070 0x7f2dac06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:20.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.398+0000 7f2dcb7fe700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f2dac06b070 0x7f2dac06d530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4011200 tx=0x7f2dc400d040 comp rx=0 tx=0).ready entity=mgr.9294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:20.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.402+0000 7f2dc3fff700 1 -- 172.21.15.181:0/3767008208 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2db4055400 con 0x7f2dcc15ea40 2023-10-16T11:00:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.652+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2dcc04ebb0 con 0x7f2dcc15ea40 2023-10-16T11:00:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.787+0000 7f2dc3fff700 1 -- 172.21.15.181:0/3767008208 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v95) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2db4058a20 con 0x7f2dcc15ea40 2023-10-16T11:00:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.791+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f2dac06b070 msgr2=0x7f2dac06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.791+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f2dac06b070 0x7f2dac06d530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4011200 tx=0x7f2dc400d040 comp rx=0 tx=0).stop 2023-10-16T11:00:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.791+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 msgr2=0x7f2dcc0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.791+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0ca890 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009c10 tx=0x7f2db4004750 comp rx=0 tx=0).stop 2023-10-16T11:00:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.792+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 shutdown_connections 2023-10-16T11:00:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.792+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f2dac06b070 0x7f2dac06d530 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.792+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2dcc0d01d0 0x7f2dcc147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.792+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2dcc161530 0x7f2dcc147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.792+0000 7f2dd1a9a700 1 --2- 172.21.15.181:0/3767008208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dcc15ea40 0x7f2dcc0ca890 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.792+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 >> 172.21.15.181:0/3767008208 conn(0x7f2dcc09cb60 msgr2=0x7f2dcc15f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:20.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.793+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 shutdown_connections 2023-10-16T11:00:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:20.793+0000 7f2dd1a9a700 1 -- 172.21.15.181:0/3767008208 wait complete. 2023-10-16T11:00:20.825 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:00:20.825 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:00:20.877 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:00:20.924 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:00:20.925 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:00:20.925 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:20.942 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:00:20.942 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:20.961 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:20.961+0000 7f76c1d77200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:00:20.964 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:20.963+0000 7f52d6f76200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:00:20.978 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:21.034 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.033+0000 7f52d6f76200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:00:21.047 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:21.046+0000 7f76c1d77200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:00:21.107 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.106+0000 7f52d6f76200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:00:21.120 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:21.120+0000 7f76c1d77200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:00:21.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 Processor -- start 2023-10-16T11:00:21.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 -- start start 2023-10-16T11:00:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c15b990 0x7f529c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f529c0cd2d0 0x7f529c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f529c0cfc60 0x7f529c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c0d0630 con 0x7f529c0cd2d0 2023-10-16T11:00:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c0d24f0 con 0x7f529c0cfc60 2023-10-16T11:00:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.141+0000 7f52a15ed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0cc510 con 0x7f529c15b990 2023-10-16T11:00:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f529c0cd2d0 0x7f529c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f529c0cd2d0 0x7f529c0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49948/0 (socket says 172.21.15.181:49948) 2023-10-16T11:00:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 -- 172.21.15.181:0/2739735257 learned_addr learned my addr 172.21.15.181:0/2739735257 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529b7fe700 1 --2- 172.21.15.181:0/2739735257 >> v2:172.21.15.190:3300/0 conn(0x7f529c0cfc60 0x7f529c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529affd700 1 --2- 172.21.15.181:0/2739735257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c15b990 0x7f529c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 -- 172.21.15.181:0/2739735257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c15b990 msgr2=0x7f529c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 --2- 172.21.15.181:0/2739735257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c15b990 0x7f529c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 -- 172.21.15.181:0/2739735257 >> v2:172.21.15.190:3300/0 conn(0x7f529c0cfc60 msgr2=0x7f529c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 --2- 172.21.15.181:0/2739735257 >> v2:172.21.15.190:3300/0 conn(0x7f529c0cfc60 0x7f529c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 -- 172.21.15.181:0/2739735257 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0cc7f0 con 0x7f529c0cd2d0 2023-10-16T11:00:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f529a7fc700 1 --2- 172.21.15.181:0/2739735257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0d1fb0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f529c0cbc60 tx=0x7f5290009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441697ecbe473c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f5299ffb700 1 -- 172.21.15.181:0/2739735257 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290011830 con 0x7f529c0cd2d0 2023-10-16T11:00:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f5299ffb700 1 -- 172.21.15.181:0/2739735257 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290011990 con 0x7f529c0cd2d0 2023-10-16T11:00:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.142+0000 7f5299ffb700 1 -- 172.21.15.181:0/2739735257 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290011d20 con 0x7f529c0cd2d0 2023-10-16T11:00:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 -- 172.21.15.181:0/2739735257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 msgr2=0x7f529c0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 --2- 172.21.15.181:0/2739735257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0d1fb0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f529c0cbc60 tx=0x7f5290009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 -- 172.21.15.181:0/2739735257 shutdown_connections 2023-10-16T11:00:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 --2- 172.21.15.181:0/2739735257 >> v2:172.21.15.190:3300/0 conn(0x7f529c0cfc60 0x7f529c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 --2- 172.21.15.181:0/2739735257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c15b990 0x7f529c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 --2- 172.21.15.181:0/2739735257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0d1fb0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 -- 172.21.15.181:0/2739735257 >> 172.21.15.181:0/2739735257 conn(0x7f529c09a9a0 msgr2=0x7f529c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:21.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 -- 172.21.15.181:0/2739735257 shutdown_connections 2023-10-16T11:00:21.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.143+0000 7f52a15ed700 1 -- 172.21.15.181:0/2739735257 wait complete. 2023-10-16T11:00:21.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.144+0000 7f52a15ed700 1 Processor -- start 2023-10-16T11:00:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.144+0000 7f52a15ed700 1 -- start start 2023-10-16T11:00:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f52a15ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f52a15ed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0cfc60 0x7f529c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f52a15ed700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c15b990 0x7f529c151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f52a15ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c15e6e0 con 0x7f529c0cd2d0 2023-10-16T11:00:21.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f52a15ed700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c150760 con 0x7f529c15b990 2023-10-16T11:00:21.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f52a15ed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0c9650 con 0x7f529c0cfc60 2023-10-16T11:00:21.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f529affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f529affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49952/0 (socket says 172.21.15.181:49952) 2023-10-16T11:00:21.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f529affd700 1 -- 172.21.15.181:0/822999046 learned_addr learned my addr 172.21.15.181:0/822999046 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:21.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f529b7fe700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c15b990 0x7f529c151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.145+0000 7f529a7fc700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0cfc60 0x7f529c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f529affd700 1 -- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0cfc60 msgr2=0x7f529c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f529affd700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0cfc60 0x7f529c0703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f529affd700 1 -- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c15b990 msgr2=0x7f529c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f529affd700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c15b990 0x7f529c151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f529affd700 1 -- 172.21.15.181:0/822999046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290009550 con 0x7f529c0cd2d0 2023-10-16T11:00:21.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f529affd700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0c9110 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f528c00e900 tx=0x7f528c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:21.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f52837fe700 1 -- 172.21.15.181:0/822999046 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c013680 con 0x7f529c0cd2d0 2023-10-16T11:00:21.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f52837fe700 1 -- 172.21.15.181:0/822999046 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c0137e0 con 0x7f529c0cd2d0 2023-10-16T11:00:21.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c15e400 con 0x7f529c0cd2d0 2023-10-16T11:00:21.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f52837fe700 1 -- 172.21.15.181:0/822999046 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c013b50 con 0x7f529c0cd2d0 2023-10-16T11:00:21.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.146+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c150ce0 con 0x7f529c0cd2d0 2023-10-16T11:00:21.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.147+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c150ea0 con 0x7f529c0cd2d0 2023-10-16T11:00:21.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.147+0000 7f52837fe700 1 -- 172.21.15.181:0/822999046 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f528c013cb0 con 0x7f529c0cd2d0 2023-10-16T11:00:21.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.148+0000 7f52837fe700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 0x7f528806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.148+0000 7f529a7fc700 1 -- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 msgr2=0x7f528806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:21.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.148+0000 7f52837fe700 1 -- 172.21.15.181:0/822999046 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f528c089d90 con 0x7f529c0cd2d0 2023-10-16T11:00:21.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.148+0000 7f529a7fc700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 0x7f528806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:21.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.152+0000 7f52837fe700 1 -- 172.21.15.181:0/822999046 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f528c054900 con 0x7f529c0cd2d0 2023-10-16T11:00:21.185 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.184+0000 7f52d6f76200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:00:21.204 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:21.203+0000 7f76c1d77200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:00:21.327 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.326+0000 7f52d6f76200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:00:21.347 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:21.347+0000 7f76c1d77200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:00:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.349+0000 7f529a7fc700 1 -- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 msgr2=0x7f528806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.349+0000 7f529a7fc700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 0x7f528806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:21.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.353+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f529c1505a0 con 0x7f529c0cd2d0 2023-10-16T11:00:21.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.355+0000 7f52837fe700 1 -- 172.21.15.181:0/822999046 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+18020 (secure 0 0 0) 0x7f528c017090 con 0x7f529c0cd2d0 2023-10-16T11:00:21.355 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:21.356 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":154,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:20.779478+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:00.393030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:00.916899+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":105,"snap_epoch":154,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.13","mappings":[{"from":2,"to":5}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1561933478":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1604670603":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6833/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6832/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1093898558":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/40763561":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/2364682800":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1727687969":"2023-10-17T10:04:15.822177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":47},{"begin":59,"length":47}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":34,"length":16},{"begin":90,"length":16}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:21.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 msgr2=0x7f528806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:21.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 0x7f528806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 msgr2=0x7f529c0c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0c9110 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f528c00e900 tx=0x7f528c00ec10 comp rx=0 tx=0).stop 2023-10-16T11:00:21.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 shutdown_connections 2023-10-16T11:00:21.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f528806b020 0x7f528806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f529c15b990 0x7f529c151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f529c0cd2d0 0x7f529c0c9110 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 --2- 172.21.15.181:0/822999046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f529c0cfc60 0x7f529c0703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 >> 172.21.15.181:0/822999046 conn(0x7f529c09a9a0 msgr2=0x7f529c0b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:21.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 shutdown_connections 2023-10-16T11:00:21.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.357+0000 7f52a15ed700 1 -- 172.21.15.181:0/822999046 wait complete. 2023-10-16T11:00:21.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:04:00.393030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:04:00.916899+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 105, 'snap_epoch': 154, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:00:21.371 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:00:21.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.545+0000 7fedd0ab6700 1 Processor -- start 2023-10-16T11:00:21.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.545+0000 7fedd0ab6700 1 -- start start 2023-10-16T11:00:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.547+0000 7fedd0ab6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fedcc0b11e0 0x7fedcc0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.547+0000 7fedd0ab6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 0x7fedcc0c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.547+0000 7fedd0ab6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0cab60 con 0x7fedcc0b15c0 2023-10-16T11:00:21.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.547+0000 7fedd0ab6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0ce6f0 con 0x7fedcc0b11e0 2023-10-16T11:00:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.547+0000 7fedd0ab6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0cdd20 con 0x7fedcc0c2490 2023-10-16T11:00:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.547+0000 7fedc9d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fedcc0b15c0 0x7fedcc0c9e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32830/0 (socket says 172.21.15.181:32830) 2023-10-16T11:00:21.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.547+0000 7fedc9d9b700 1 -- 172.21.15.181:0/2082493343 learned_addr learned my addr 172.21.15.181:0/2082493343 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:21.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedcad9d700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 0x7fedcc0c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034853393 0 0) 0x7fedc0001110 con 0x7fedcc0b15c0 2023-10-16T11:00:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb40035b0 con 0x7fedcc0b15c0 2023-10-16T11:00:21.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedca59c700 1 --2- 172.21.15.181:0/2082493343 >> v2:172.21.15.181:3300/0 conn(0x7fedcc0b11e0 0x7fedcc0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 963777235 0 0) 0x7fedb40035b0 con 0x7fedcc0b15c0 2023-10-16T11:00:21.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 msgr2=0x7fedcc0c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 0x7fedcc0c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 >> v2:172.21.15.181:3300/0 conn(0x7fedcc0b11e0 msgr2=0x7fedcc0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 --2- 172.21.15.181:0/2082493343 >> v2:172.21.15.181:3300/0 conn(0x7fedcc0b11e0 0x7fedcc0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0c2e20 con 0x7fedcc0b15c0 2023-10-16T11:00:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fedc0001800 con 0x7fedcc0b15c0 2023-10-16T11:00:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 >> v1:172.21.15.181:6789/0 conn(0x7fedcc0b15c0 legacy=0x7fedcc0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb4003a00 0x7fedb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedb40043f0 0x7fedb40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 0x7fedb4005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ce6f0 con 0x7fedb40043f0 2023-10-16T11:00:21.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0cdd20 con 0x7fedb4003a00 2023-10-16T11:00:21.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.548+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb400ba60 con 0x7fedcc0c2490 2023-10-16T11:00:21.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedca59c700 1 --2- 172.21.15.181:0/2082493343 >> v2:172.21.15.181:3300/0 conn(0x7fedcc0b11e0 0x7fedcc0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:21.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedc9d9b700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedb40043f0 0x7fedb40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedca59c700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb4003a00 0x7fedb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedcad9d700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 0x7fedb4005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedca59c700 1 -- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 msgr2=0x7fedb4005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:21.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedca59c700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc0c2490 0x7fedb4005050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedca59c700 1 -- 172.21.15.181:0/2082493343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedb40043f0 msgr2=0x7fedb40048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:21.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedca59c700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedb40043f0 0x7fedb40048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.549+0000 7fedca59c700 1 -- 172.21.15.181:0/2082493343 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0c2e20 con 0x7fedb4003a00 2023-10-16T11:00:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.550+0000 7fedca59c700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb4003a00 0x7fedb4003e20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009950 tx=0x7fedb4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd2830813e731927 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.550+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00c070 con 0x7fedb4003a00 2023-10-16T11:00:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.550+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc013610 con 0x7fedb4003a00 2023-10-16T11:00:21.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.550+0000 7fedc959a700 1 -- 172.21.15.181:0/2082493343 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc013980 con 0x7fedb4003a00 2023-10-16T11:00:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.551+0000 7fedd0ab6700 1 -- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb4003a00 msgr2=0x7fedb4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.551+0000 7fedd0ab6700 1 --2- 172.21.15.181:0/2082493343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb4003a00 0x7fedb4003e20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009950 tx=0x7fedb4005880 comp rx=0 tx=0).stop 2023-10-16T11:00:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.551+0000 7fedcad9d700 1 -- 172.21.15.181:0/2082493343 reap_dead start 2023-10-16T11:00:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.551+0000 7fedd0ab6700 1 -- 172.21.15.181:0/2082493343 shutdown_connections 2023-10-16T11:00:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.551+0000 7fedd0ab6700 1 -- 172.21.15.181:0/2082493343 >> 172.21.15.181:0/2082493343 conn(0x7fedcc0cbf70 msgr2=0x7fedcc0cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:21.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.552+0000 7fedd0ab6700 1 -- 172.21.15.181:0/2082493343 shutdown_connections 2023-10-16T11:00:21.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.553+0000 7fedd0ab6700 1 -- 172.21.15.181:0/2082493343 wait complete. 2023-10-16T11:00:21.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.555+0000 7fedd0ab6700 1 Processor -- start 2023-10-16T11:00:21.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.555+0000 7fedd0ab6700 1 -- start start 2023-10-16T11:00:21.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedd0ab6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc15fb90 0x7fedcc165900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedd0ab6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedcc165e40 0x7fedcc0721a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedd0ab6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0726e0 0x7fedcc0c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedd0ab6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1f8270 con 0x7fedcc0726e0 2023-10-16T11:00:21.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedd0ab6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1662c0 con 0x7fedcc165e40 2023-10-16T11:00:21.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedd0ab6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1f75f0 con 0x7fedcc15fb90 2023-10-16T11:00:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedc9d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedcc165e40 0x7fedcc0721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.556+0000 7fedcad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0726e0 0x7fedcc0c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedcad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0726e0 0x7fedcc0c2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49996/0 (socket says 172.21.15.181:49996) 2023-10-16T11:00:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedcad9d700 1 -- 172.21.15.181:0/1922676907 learned_addr learned my addr 172.21.15.181:0/1922676907 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedca59c700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc15fb90 0x7fedcc165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedcad9d700 1 -- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc15fb90 msgr2=0x7fedcc165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedcad9d700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc15fb90 0x7fedcc165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedcad9d700 1 -- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedcc165e40 msgr2=0x7fedcc0721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedcad9d700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedcc165e40 0x7fedcc0721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedcad9d700 1 -- 172.21.15.181:0/1922676907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc1f78d0 con 0x7fedcc0726e0 2023-10-16T11:00:21.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.557+0000 7fedc9d9b700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedcc165e40 0x7fedcc0721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.558+0000 7fedcad9d700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0726e0 0x7fedcc0c2fb0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0aef00 tx=0x7fedc40114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.558+0000 7fedbaffd700 1 -- 172.21.15.181:0/1922676907 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400e070 con 0x7fedcc0726e0 2023-10-16T11:00:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.558+0000 7fedbaffd700 1 -- 172.21.15.181:0/1922676907 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4003c30 con 0x7fedcc0726e0 2023-10-16T11:00:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.558+0000 7fedbaffd700 1 -- 172.21.15.181:0/1922676907 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4003de0 con 0x7fedcc0726e0 2023-10-16T11:00:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.558+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc1f7bb0 con 0x7fedcc0726e0 2023-10-16T11:00:21.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.558+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc072b90 con 0x7fedcc0726e0 2023-10-16T11:00:21.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.559+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc1f83b0 con 0x7fedcc0726e0 2023-10-16T11:00:21.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.569+0000 7fedbaffd700 1 -- 172.21.15.181:0/1922676907 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fedc4011a10 con 0x7fedcc0726e0 2023-10-16T11:00:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.570+0000 7fedbaffd700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.570+0000 7fedca59c700 1 -- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 msgr2=0x7fedac06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.570+0000 7fedbaffd700 1 -- 172.21.15.181:0/1922676907 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fedc408ab80 con 0x7fedcc0726e0 2023-10-16T11:00:21.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.570+0000 7fedca59c700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:21.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.570+0000 7fedbaffd700 1 -- 172.21.15.181:0/1922676907 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fedc40b70a0 con 0x7fedcc0726e0 2023-10-16T11:00:21.651 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:21.654 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:21.685 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.684+0000 7f52d6f76200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:00:21.734 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:21.734+0000 7f76c1d77200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:00:21.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.771+0000 7fedca59c700 1 -- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 msgr2=0x7fedac06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:21.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.771+0000 7fedca59c700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:21.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.796+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fedcc1f83b0 con 0x7fedcc0726e0 2023-10-16T11:00:21.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.796+0000 7fedbaffd700 1 -- 172.21.15.181:0/1922676907 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7fedcc1f83b0 con 0x7fedcc0726e0 2023-10-16T11:00:21.797 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:00:21.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 msgr2=0x7fedac06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:21.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0726e0 msgr2=0x7fedcc0c2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0726e0 0x7fedcc0c2fb0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0aef00 tx=0x7fedc40114b0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 shutdown_connections 2023-10-16T11:00:21.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedcc15fb90 0x7fedcc165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0726e0 0x7fedcc0c2fb0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 --2- 172.21.15.181:0/1922676907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedcc165e40 0x7fedcc0721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 >> 172.21.15.181:0/1922676907 conn(0x7fedcc0cbf70 msgr2=0x7fedcc164600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:21.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.800+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 shutdown_connections 2023-10-16T11:00:21.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.801+0000 7fedd0ab6700 1 -- 172.21.15.181:0/1922676907 wait complete. 2023-10-16T11:00:21.810 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.808+0000 7f52d6f76200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:00:21.820 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:00:21.861 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:21.860+0000 7f76c1d77200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:00:21.893 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.892+0000 7f52d6f76200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:00:21.938 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:21.938+0000 7f76c1d77200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:00:21.973 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:21.972+0000 7f52d6f76200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:00:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.989+0000 7f3f5f787700 1 Processor -- start 2023-10-16T11:00:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.990+0000 7f3f5f787700 1 -- start start 2023-10-16T11:00:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5f787700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f5815b9c0 0x7f3f580ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5f787700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3f580b4e00 0x7f3f580b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5f787700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5815bda0 con 0x7f3f580b52b0 2023-10-16T11:00:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5f787700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5815f570 con 0x7f3f5815b9c0 2023-10-16T11:00:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5f787700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5815f830 con 0x7f3f580b4e00 2023-10-16T11:00:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5d523700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f5815b9c0 0x7f3f580ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5cd22700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3f580b4e00 0x7f3f580b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5d523700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f5815b9c0 0x7f3f580ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50002/0 (socket says 172.21.15.181:50002) 2023-10-16T11:00:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5d523700 1 -- 172.21.15.181:0/1845933469 learned_addr learned my addr 172.21.15.181:0/1845933469 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5d523700 1 -- 172.21.15.181:0/1845933469 >> v2:172.21.15.190:3300/0 conn(0x7f3f580b4e00 msgr2=0x7f3f580b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5d523700 1 --2- 172.21.15.181:0/1845933469 >> v2:172.21.15.190:3300/0 conn(0x7f3f580b4e00 0x7f3f580b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5d523700 1 -- 172.21.15.181:0/1845933469 >> v1:172.21.15.181:6789/0 conn(0x7f3f580b52b0 legacy=0x7f3f5815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5d523700 1 -- 172.21.15.181:0/1845933469 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5815fb10 con 0x7f3f5815b9c0 2023-10-16T11:00:21.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.991+0000 7f3f5cd22700 1 --2- 172.21.15.181:0/1845933469 >> v2:172.21.15.190:3300/0 conn(0x7f3f580b4e00 0x7f3f580b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.992+0000 7f3f5d523700 1 --2- 172.21.15.181:0/1845933469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f5815b9c0 0x7f3f580ca010 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f3f5815f330 tx=0x7f3f54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ce5e0e9dc8f2af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.992+0000 7f3f4ffff700 1 -- 172.21.15.181:0/1845933469 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54011830 con 0x7f3f5815b9c0 2023-10-16T11:00:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.992+0000 7f3f4ffff700 1 -- 172.21.15.181:0/1845933469 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f54011990 con 0x7f3f5815b9c0 2023-10-16T11:00:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.992+0000 7f3f4ffff700 1 -- 172.21.15.181:0/1845933469 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54011d20 con 0x7f3f5815b9c0 2023-10-16T11:00:21.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 -- 172.21.15.181:0/1845933469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f5815b9c0 msgr2=0x7f3f580ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:21.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 --2- 172.21.15.181:0/1845933469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f5815b9c0 0x7f3f580ca010 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f3f5815f330 tx=0x7f3f54009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 -- 172.21.15.181:0/1845933469 shutdown_connections 2023-10-16T11:00:21.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 --2- 172.21.15.181:0/1845933469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f5815b9c0 0x7f3f580ca010 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 --2- 172.21.15.181:0/1845933469 >> v2:172.21.15.190:3300/0 conn(0x7f3f580b4e00 0x7f3f580b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:21.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 -- 172.21.15.181:0/1845933469 >> 172.21.15.181:0/1845933469 conn(0x7f3f580cbf70 msgr2=0x7f3f580cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 -- 172.21.15.181:0/1845933469 shutdown_connections 2023-10-16T11:00:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.993+0000 7f3f5f787700 1 -- 172.21.15.181:0/1845933469 wait complete. 2023-10-16T11:00:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.994+0000 7f3f5f787700 1 Processor -- start 2023-10-16T11:00:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.994+0000 7f3f5f787700 1 -- start start 2023-10-16T11:00:22.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5f787700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580b4e00 0x7f3f580c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5f787700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f580b52b0 0x7f3f581f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5f787700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5815b9c0 0x7f3f581eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5f787700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f581ef900 con 0x7f3f580b4e00 2023-10-16T11:00:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5f787700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f581efbe0 con 0x7f3f5815b9c0 2023-10-16T11:00:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5f787700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f581f26d0 con 0x7f3f580b52b0 2023-10-16T11:00:22.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5d523700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580b4e00 0x7f3f580c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5d523700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580b4e00 0x7f3f580c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50008/0 (socket says 172.21.15.181:50008) 2023-10-16T11:00:22.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5d523700 1 -- 172.21.15.181:0/3804205429 learned_addr learned my addr 172.21.15.181:0/3804205429 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:22.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5dd24700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5815b9c0 0x7f3f581eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5cd22700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f580b52b0 0x7f3f581f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5dd24700 1 -- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f580b52b0 msgr2=0x7f3f581f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5dd24700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f580b52b0 0x7f3f581f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5dd24700 1 -- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580b4e00 msgr2=0x7f3f580c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5dd24700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580b4e00 0x7f3f580c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5dd24700 1 -- 172.21.15.181:0/3804205429 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480094a0 con 0x7f3f5815b9c0 2023-10-16T11:00:22.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.995+0000 7f3f5d523700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580b4e00 0x7f3f580c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:22.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.996+0000 7f3f5dd24700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5815b9c0 0x7f3f581eeff0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000cb90 tx=0x7f3f5000cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:22.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.996+0000 7f3f4dffb700 1 -- 172.21.15.181:0/3804205429 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50003c00 con 0x7f3f5815b9c0 2023-10-16T11:00:22.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.996+0000 7f3f4dffb700 1 -- 172.21.15.181:0/3804205429 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f50003d60 con 0x7f3f5815b9c0 2023-10-16T11:00:22.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.996+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f54009550 con 0x7f3f5815b9c0 2023-10-16T11:00:22.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.996+0000 7f3f4dffb700 1 -- 172.21.15.181:0/3804205429 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000aa80 con 0x7f3f5815b9c0 2023-10-16T11:00:22.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.996+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f581efec0 con 0x7f3f5815b9c0 2023-10-16T11:00:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.997+0000 7f3f4dffb700 1 -- 172.21.15.181:0/3804205429 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f3f5000abe0 con 0x7f3f5815b9c0 2023-10-16T11:00:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.997+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5804ebb0 con 0x7f3f5815b9c0 2023-10-16T11:00:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.998+0000 7f3f4dffb700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 0x7f3f3c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.998+0000 7f3f5d523700 1 -- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 msgr2=0x7f3f3c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:22.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.998+0000 7f3f4dffb700 1 -- 172.21.15.181:0/3804205429 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f3f50089a50 con 0x7f3f5815b9c0 2023-10-16T11:00:22.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:21.998+0000 7f3f5d523700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 0x7f3f3c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:22.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.002+0000 7f3f4dffb700 1 -- 172.21.15.181:0/3804205429 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3f50054620 con 0x7f3f5815b9c0 2023-10-16T11:00:22.039 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:22.038+0000 7f76c1d77200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:00:22.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.199+0000 7f3f5d523700 1 -- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 msgr2=0x7f3f3c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:22.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.199+0000 7f3f5d523700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 0x7f3f3c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:22.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.217+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f580cce30 con 0x7f3f5815b9c0 2023-10-16T11:00:22.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.218+0000 7f3f4dffb700 1 -- 172.21.15.181:0/3804205429 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f5005e030 con 0x7f3f5815b9c0 2023-10-16T11:00:22.218 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:00:22.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 msgr2=0x7f3f3c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:22.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 0x7f3f3c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5815b9c0 msgr2=0x7f3f581eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5815b9c0 0x7f3f581eeff0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000cb90 tx=0x7f3f5000cea0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 shutdown_connections 2023-10-16T11:00:22.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f3f3c06b020 0x7f3f3c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5815b9c0 0x7f3f581eeff0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f580b4e00 0x7f3f580c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 --2- 172.21.15.181:0/3804205429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f580b52b0 0x7f3f581f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 >> 172.21.15.181:0/3804205429 conn(0x7f3f580cbf70 msgr2=0x7f3f580cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:22.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 shutdown_connections 2023-10-16T11:00:22.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.220+0000 7f3f5f787700 1 -- 172.21.15.181:0/3804205429 wait complete. 2023-10-16T11:00:22.233 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:00:22.347 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:22.346+0000 7f52d6f76200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:00:22.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.357+0000 7fcb00d62700 1 Processor -- start 2023-10-16T11:00:22.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.357+0000 7fcb00d62700 1 -- start start 2023-10-16T11:00:22.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcb00d62700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcb00d62700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcafc0ce4f0 0x7fcafc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcb00d62700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcafc0b2ae0 con 0x7fcafc0b14a0 2023-10-16T11:00:22.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcb00d62700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcafc0ce970 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcb00d62700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0b2dc0 con 0x7fcafc15f150 2023-10-16T11:00:22.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcaf9d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcafc0ce4f0 0x7fcafc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcafad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcafc0b14a0 0x7fcafc0b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32882/0 (socket says 172.21.15.181:32882) 2023-10-16T11:00:22.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcafad9d700 1 -- 172.21.15.181:0/1971152291 learned_addr learned my addr 172.21.15.181:0/1971152291 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:22.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcaf9d9b700 1 --2- 172.21.15.181:0/1971152291 >> v2:172.21.15.181:3300/0 conn(0x7fcafc0ce4f0 0x7fcafc0b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50032/0 (socket says 172.21.15.181:50032) 2023-10-16T11:00:22.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.358+0000 7fcafa59c700 1 --2- 172.21.15.181:0/1971152291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf9d9b700 1 -- 172.21.15.181:0/1971152291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 msgr2=0x7fcafc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:22.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf9d9b700 1 --2- 172.21.15.181:0/1971152291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf9d9b700 1 -- 172.21.15.181:0/1971152291 >> v1:172.21.15.181:6789/0 conn(0x7fcafc0b14a0 legacy=0x7fcafc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf959a700 1 -- 172.21.15.181:0/1971152291 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587013934 0 0) 0x7fcafc0b2ae0 con 0x7fcafc0b14a0 2023-10-16T11:00:22.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf9d9b700 1 -- 172.21.15.181:0/1971152291 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0cf9d0 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf9d9b700 1 --2- 172.21.15.181:0/1971152291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0b0f60 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009840 tx=0x7fcaf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29a7369e9b80328 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:22.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf959a700 1 -- 172.21.15.181:0/1971152291 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0011830 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf959a700 1 -- 172.21.15.181:0/1971152291 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf0011990 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcaf959a700 1 -- 172.21.15.181:0/1971152291 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0011d20 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcb00d62700 1 -- 172.21.15.181:0/1971152291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 msgr2=0x7fcafc0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.359+0000 7fcb00d62700 1 --2- 172.21.15.181:0/1971152291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0b0f60 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009840 tx=0x7fcaf0009b50 comp rx=0 tx=0).stop 2023-10-16T11:00:22.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.360+0000 7fcb00d62700 1 -- 172.21.15.181:0/1971152291 shutdown_connections 2023-10-16T11:00:22.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.360+0000 7fcb00d62700 1 --2- 172.21.15.181:0/1971152291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.360+0000 7fcb00d62700 1 --2- 172.21.15.181:0/1971152291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0b0f60 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.360+0000 7fcb00d62700 1 -- 172.21.15.181:0/1971152291 >> 172.21.15.181:0/1971152291 conn(0x7fcafc09a930 msgr2=0x7fcafc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:22.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.360+0000 7fcb00d62700 1 -- 172.21.15.181:0/1971152291 shutdown_connections 2023-10-16T11:00:22.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.360+0000 7fcb00d62700 1 -- 172.21.15.181:0/1971152291 wait complete. 2023-10-16T11:00:22.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.361+0000 7fcb00d62700 1 Processor -- start 2023-10-16T11:00:22.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.361+0000 7fcb00d62700 1 -- start start 2023-10-16T11:00:22.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcb00d62700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcafc0b14a0 0x7fcafc151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcb00d62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcb00d62700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcb00d62700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0c9bc0 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcb00d62700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc151960 con 0x7fcafc0b14a0 2023-10-16T11:00:22.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcb00d62700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0c8fb0 con 0x7fcafc15f150 2023-10-16T11:00:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50040/0 (socket says 172.21.15.181:50040) 2023-10-16T11:00:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 -- 172.21.15.181:0/913372578 learned_addr learned my addr 172.21.15.181:0/913372578 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcafa59c700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcafc0b14a0 0x7fcafc151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcafad9d700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 -- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 msgr2=0x7fcafc0ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 -- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcafc0b14a0 msgr2=0x7fcafc151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcafc0b14a0 0x7fcafc151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 -- 172.21.15.181:0/913372578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00094a0 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaf9d9b700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0c86d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00090b0 tx=0x7fcaf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.362+0000 7fcaeaffd700 1 -- 172.21.15.181:0/913372578 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00036a0 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.363+0000 7fcaeaffd700 1 -- 172.21.15.181:0/913372578 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf001f070 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.363+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc0c9290 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.363+0000 7fcaeaffd700 1 -- 172.21.15.181:0/913372578 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0003a60 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.363+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc0c9850 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.364+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcafc0c9d30 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.364+0000 7fcaeaffd700 1 -- 172.21.15.181:0/913372578 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fcaf0003bc0 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.365+0000 7fcaeaffd700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 0x7fcadc06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.365+0000 7fcafa59c700 1 -- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 msgr2=0x7fcadc06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:22.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.365+0000 7fcaeaffd700 1 -- 172.21.15.181:0/913372578 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fcaf0021070 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.365+0000 7fcafa59c700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 0x7fcadc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:22.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.373+0000 7fcaeaffd700 1 -- 172.21.15.181:0/913372578 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcaf0055f10 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.446 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:22.444+0000 7f76c1d77200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:00:22.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.566+0000 7fcafa59c700 1 -- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 msgr2=0x7fcadc06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:22.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.566+0000 7fcafa59c700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 0x7fcadc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:22.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.589+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcafc067d20 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.589+0000 7fcaeaffd700 1 -- 172.21.15.181:0/913372578 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcaf0059530 con 0x7fcafc0ce4f0 2023-10-16T11:00:22.591 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:00:22.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 msgr2=0x7fcadc06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:22.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 0x7fcadc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 msgr2=0x7fcafc0c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0c86d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00090b0 tx=0x7fcaf0004750 comp rx=0 tx=0).stop 2023-10-16T11:00:22.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 shutdown_connections 2023-10-16T11:00:22.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fcadc06b050 0x7fcadc06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcafc15f150 0x7fcafc0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcafc0b14a0 0x7fcafc151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 --2- 172.21.15.181:0/913372578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcafc0ce4f0 0x7fcafc0c86d0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 >> 172.21.15.181:0/913372578 conn(0x7fcafc09a930 msgr2=0x7fcafc0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:22.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 shutdown_connections 2023-10-16T11:00:22.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.591+0000 7fcb00d62700 1 -- 172.21.15.181:0/913372578 wait complete. 2023-10-16T11:00:22.606 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:00:22.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.727+0000 7fc708198700 1 Processor -- start 2023-10-16T11:00:22.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.727+0000 7fc708198700 1 -- start start 2023-10-16T11:00:22.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.728+0000 7fc708198700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc7000cfc60 0x7fc7000d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.728+0000 7fc708198700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000b05d0 0x7fc7000cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.728+0000 7fc708198700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7000d24f0 con 0x7fc70015bd50 2023-10-16T11:00:22.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.728+0000 7fc708198700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70015ccc0 con 0x7fc7000cfc60 2023-10-16T11:00:22.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.728+0000 7fc708198700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70015c320 con 0x7fc7000b05d0 2023-10-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000b05d0 0x7fc7000cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc705f34700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc7000cfc60 0x7fc7000d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000b05d0 0x7fc7000cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:47078/0 (socket says 172.21.15.181:47078) 2023-10-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc705f34700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc7000cfc60 0x7fc7000d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:55098/0 (socket says 172.21.15.181:55098) 2023-10-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 -- 172.21.15.181:0/521452804 learned_addr learned my addr 172.21.15.181:0/521452804 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 -- 172.21.15.181:0/521452804 >> v2:172.21.15.190:3300/0 conn(0x7fc7000cfc60 msgr2=0x7fc7000d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 --2- 172.21.15.181:0/521452804 >> v2:172.21.15.190:3300/0 conn(0x7fc7000cfc60 0x7fc7000d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 -- 172.21.15.181:0/521452804 >> v1:172.21.15.190:6789/0 conn(0x7fc70015bd50 legacy=0x7fc7000d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 -- 172.21.15.181:0/521452804 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70015c600 con 0x7fc7000b05d0 2023-10-16T11:00:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc706735700 1 --2- 172.21.15.181:0/521452804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000b05d0 0x7fc7000cc200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009e50 tx=0x7fc6f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2471de4e4dacd53a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:22.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc704f32700 1 -- 172.21.15.181:0/521452804 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800e070 con 0x7fc7000b05d0 2023-10-16T11:00:22.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc704f32700 1 -- 172.21.15.181:0/521452804 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f800b910 con 0x7fc7000b05d0 2023-10-16T11:00:22.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.729+0000 7fc704f32700 1 -- 172.21.15.181:0/521452804 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800bcc0 con 0x7fc7000b05d0 2023-10-16T11:00:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 -- 172.21.15.181:0/521452804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000b05d0 msgr2=0x7fc7000cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 --2- 172.21.15.181:0/521452804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000b05d0 0x7fc7000cc200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009e50 tx=0x7fc6f800b330 comp rx=0 tx=0).stop 2023-10-16T11:00:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 -- 172.21.15.181:0/521452804 shutdown_connections 2023-10-16T11:00:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 --2- 172.21.15.181:0/521452804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000b05d0 0x7fc7000cc200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 --2- 172.21.15.181:0/521452804 >> v2:172.21.15.190:3300/0 conn(0x7fc7000cfc60 0x7fc7000d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 -- 172.21.15.181:0/521452804 >> 172.21.15.181:0/521452804 conn(0x7fc70009a9a0 msgr2=0x7fc7000d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 -- 172.21.15.181:0/521452804 shutdown_connections 2023-10-16T11:00:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.730+0000 7fc708198700 1 -- 172.21.15.181:0/521452804 wait complete. 2023-10-16T11:00:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.731+0000 7fc708198700 1 Processor -- start 2023-10-16T11:00:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.731+0000 7fc708198700 1 -- start start 2023-10-16T11:00:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc708198700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7000b05d0 0x7fc700159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc708198700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000cfc60 0x7fc7001f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc708198700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70015bd50 0x7fc7001f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc708198700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc700158dc0 con 0x7fc70015bd50 2023-10-16T11:00:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc708198700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7001f5920 con 0x7fc7000b05d0 2023-10-16T11:00:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc708198700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7001581b0 con 0x7fc7000cfc60 2023-10-16T11:00:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc705f34700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7000b05d0 0x7fc700159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc705f34700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7000b05d0 0x7fc700159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:55104/0 (socket says 172.21.15.181:55104) 2023-10-16T11:00:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc705f34700 1 -- 172.21.15.181:0/352344422 learned_addr learned my addr 172.21.15.181:0/352344422 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc706735700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70015bd50 0x7fc7001f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.732+0000 7fc705733700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000cfc60 0x7fc7001f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc705f34700 1 -- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000cfc60 msgr2=0x7fc7001f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc705f34700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000cfc60 0x7fc7001f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc705f34700 1 -- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70015bd50 msgr2=0x7fc7001f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc705f34700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70015bd50 0x7fc7001f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc705f34700 1 -- 172.21.15.181:0/352344422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8009b30 con 0x7fc7000b05d0 2023-10-16T11:00:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc705f34700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7000b05d0 0x7fc700159640 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e9b0 tx=0x7fc6fc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc6f67fc700 1 -- 172.21.15.181:0/352344422 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc009710 con 0x7fc7000b05d0 2023-10-16T11:00:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc6f67fc700 1 -- 172.21.15.181:0/352344422 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc00ee50 con 0x7fc7000b05d0 2023-10-16T11:00:22.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc700158490 con 0x7fc7000b05d0 2023-10-16T11:00:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc6f67fc700 1 -- 172.21.15.181:0/352344422 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00ca50 con 0x7fc7000b05d0 2023-10-16T11:00:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.733+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc700158a50 con 0x7fc7000b05d0 2023-10-16T11:00:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.734+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7001590a0 con 0x7fc7000b05d0 2023-10-16T11:00:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.735+0000 7fc6f67fc700 1 -- 172.21.15.181:0/352344422 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fc6fc00cbb0 con 0x7fc7000b05d0 2023-10-16T11:00:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.735+0000 7fc6f67fc700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 0x7fc6e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.735+0000 7fc705733700 1 -- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 msgr2=0x7fc6e406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.735+0000 7fc705733700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 0x7fc6e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.743+0000 7fc6f67fc700 1 -- 172.21.15.181:0/352344422 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fc6fc0592c0 con 0x7fc7000b05d0 2023-10-16T11:00:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.743+0000 7fc6f67fc700 1 -- 172.21.15.181:0/352344422 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc6fc054f00 con 0x7fc7000b05d0 2023-10-16T11:00:22.929 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:22.928+0000 7f52d6f76200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:00:22.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.936+0000 7fc705733700 1 -- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 msgr2=0x7fc6e406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:22.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.936+0000 7fc705733700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 0x7fc6e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:22.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.992+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc700067d20 con 0x7fc7000b05d0 2023-10-16T11:00:22.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.992+0000 7fc6f67fc700 1 -- 172.21.15.181:0/352344422 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v155) v1 ==== 80+0+476 (secure 0 0 0) 0x7fc6fc021080 con 0x7fc7000b05d0 2023-10-16T11:00:22.994 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1561933478 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.994 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1604670603 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.994 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3796559700 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.994 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3796559700 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.995 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1093898558 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.995 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/40763561 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.995 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2364682800 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.995 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1727687969 2023-10-17T10:04:15.822177+0000 2023-10-16T11:00:22.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.994+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 msgr2=0x7fc6e406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.994+0000 7fc708198700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 0x7fc6e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.994+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7000b05d0 msgr2=0x7fc700159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.994+0000 7fc708198700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7000b05d0 0x7fc700159640 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e9b0 tx=0x7fc6fc00ecc0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 shutdown_connections 2023-10-16T11:00:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fc6e406b020 0x7fc6e406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70015bd50 0x7fc7001f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7000b05d0 0x7fc700159640 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 --2- 172.21.15.181:0/352344422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7000cfc60 0x7fc7001f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 >> 172.21.15.181:0/352344422 conn(0x7fc70009a9a0 msgr2=0x7fc7000aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 shutdown_connections 2023-10-16T11:00:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:22.995+0000 7fc708198700 1 -- 172.21.15.181:0/352344422 wait complete. 2023-10-16T11:00:22.999 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:22.998+0000 7f52d6f76200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:00:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T11:00:23.010 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:00:23.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:23.010 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:00:23.036 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:23.035+0000 7f76c1d77200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:00:23.054 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:00:23.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:23.055 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:00:23.111 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:00:23.111 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:23.112 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:23.111+0000 7f76c1d77200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:00:23.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:00:23.129 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:00:23.130 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:23.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:00:23.147 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:00:23.147 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:23.186 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:00:23.186 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:00:23.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.308+0000 7f68b307e700 1 Processor -- start 2023-10-16T11:00:23.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.308+0000 7f68b307e700 1 -- start start 2023-10-16T11:00:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b307e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f68ac161b60 0x7f68ac06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b307e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68ac161f40 0x7f68ac0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b307e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac0d2560 con 0x7f68ac160d20 2023-10-16T11:00:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b307e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac162800 con 0x7f68ac161b60 2023-10-16T11:00:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b307e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0d43e0 con 0x7f68ac161f40 2023-10-16T11:00:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b0e1a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f68ac161b60 0x7f68ac06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b161b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f68ac160d20 0x7f68ac0b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32914/0 (socket says 172.21.15.181:32914) 2023-10-16T11:00:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.309+0000 7f68b161b700 1 -- 172.21.15.181:0/2325971964 learned_addr learned my addr 172.21.15.181:0/2325971964 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68abfff700 1 --2- 172.21.15.181:0/2325971964 >> v2:172.21.15.190:3300/0 conn(0x7f68ac161f40 0x7f68ac0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68b0e1a700 1 -- 172.21.15.181:0/2325971964 >> v2:172.21.15.190:3300/0 conn(0x7f68ac161f40 msgr2=0x7f68ac0d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68b0e1a700 1 --2- 172.21.15.181:0/2325971964 >> v2:172.21.15.190:3300/0 conn(0x7f68ac161f40 0x7f68ac0d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68b0e1a700 1 -- 172.21.15.181:0/2325971964 >> v1:172.21.15.181:6789/0 conn(0x7f68ac160d20 legacy=0x7f68ac0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68b0e1a700 1 -- 172.21.15.181:0/2325971964 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac06ce20 con 0x7f68ac161b60 2023-10-16T11:00:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68ab7fe700 1 -- 172.21.15.181:0/2325971964 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868364775 0 0) 0x7f68ac0d2560 con 0x7f68ac160d20 2023-10-16T11:00:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68b0e1a700 1 --2- 172.21.15.181:0/2325971964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac161b60 0x7f68ac06d5f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f68ac08ddd0 tx=0x7f68a0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e046bf8f08a0e9b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:23.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68ab7fe700 1 -- 172.21.15.181:0/2325971964 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0005e30 con 0x7f68ac161b60 2023-10-16T11:00:23.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68ab7fe700 1 -- 172.21.15.181:0/2325971964 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a0006030 con 0x7f68ac161b60 2023-10-16T11:00:23.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.310+0000 7f68ab7fe700 1 -- 172.21.15.181:0/2325971964 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a00063c0 con 0x7f68ac161b60 2023-10-16T11:00:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 -- 172.21.15.181:0/2325971964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac161b60 msgr2=0x7f68ac06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 --2- 172.21.15.181:0/2325971964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac161b60 0x7f68ac06d5f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f68ac08ddd0 tx=0x7f68a0009b00 comp rx=0 tx=0).stop 2023-10-16T11:00:23.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 -- 172.21.15.181:0/2325971964 shutdown_connections 2023-10-16T11:00:23.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 --2- 172.21.15.181:0/2325971964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac161b60 0x7f68ac06d5f0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 --2- 172.21.15.181:0/2325971964 >> v2:172.21.15.190:3300/0 conn(0x7f68ac161f40 0x7f68ac0d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 -- 172.21.15.181:0/2325971964 >> 172.21.15.181:0/2325971964 conn(0x7f68ac09cba0 msgr2=0x7f68ac0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:23.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 -- 172.21.15.181:0/2325971964 shutdown_connections 2023-10-16T11:00:23.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.311+0000 7f68b307e700 1 -- 172.21.15.181:0/2325971964 wait complete. 2023-10-16T11:00:23.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.312+0000 7f68b307e700 1 Processor -- start 2023-10-16T11:00:23.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.312+0000 7f68b307e700 1 -- start start 2023-10-16T11:00:23.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b307e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac160d20 0x7f68ac0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b307e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac161b60 0x7f68ac1f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b307e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68ac161f40 0x7f68ac0b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b307e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0b6960 con 0x7f68ac160d20 2023-10-16T11:00:23.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b307e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac1f6d40 con 0x7f68ac161b60 2023-10-16T11:00:23.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b307e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0b8fd0 con 0x7f68ac161f40 2023-10-16T11:00:23.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac160d20 0x7f68ac0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac160d20 0x7f68ac0b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50070/0 (socket says 172.21.15.181:50070) 2023-10-16T11:00:23.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 -- 172.21.15.181:0/2361579603 learned_addr learned my addr 172.21.15.181:0/2361579603 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:23.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68abfff700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac161b60 0x7f68ac1f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b161b700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68ac161f40 0x7f68ac0b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 -- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68ac161f40 msgr2=0x7f68ac0b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68ac161f40 0x7f68ac0b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 -- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac161b60 msgr2=0x7f68ac1f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac161b60 0x7f68ac1f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.313+0000 7f68b0e1a700 1 -- 172.21.15.181:0/2361579603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a0009430 con 0x7f68ac160d20 2023-10-16T11:00:23.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.314+0000 7f68b0e1a700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac160d20 0x7f68ac0b6420 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f68a0000c00 tx=0x7f68a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:23.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.314+0000 7f68a97fa700 1 -- 172.21.15.181:0/2361579603 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a00068b0 con 0x7f68ac160d20 2023-10-16T11:00:23.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.314+0000 7f68a97fa700 1 -- 172.21.15.181:0/2361579603 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a0020070 con 0x7f68ac160d20 2023-10-16T11:00:23.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.314+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac0b92b0 con 0x7f68ac160d20 2023-10-16T11:00:23.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.314+0000 7f68a97fa700 1 -- 172.21.15.181:0/2361579603 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0004280 con 0x7f68ac160d20 2023-10-16T11:00:23.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.314+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac0b9640 con 0x7f68ac160d20 2023-10-16T11:00:23.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.315+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac04ebb0 con 0x7f68ac160d20 2023-10-16T11:00:23.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.315+0000 7f68a97fa700 1 -- 172.21.15.181:0/2361579603 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f68a0022030 con 0x7f68ac160d20 2023-10-16T11:00:23.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.316+0000 7f68a97fa700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 0x7f689006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.316+0000 7f68abfff700 1 -- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 msgr2=0x7f689006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:23.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.316+0000 7f68a97fa700 1 -- 172.21.15.181:0/2361579603 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f68a008aef0 con 0x7f68ac160d20 2023-10-16T11:00:23.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.316+0000 7f68abfff700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 0x7f689006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:23.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.321+0000 7f68a97fa700 1 -- 172.21.15.181:0/2361579603 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f68a0055af0 con 0x7f68ac160d20 2023-10-16T11:00:23.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.517+0000 7f68abfff700 1 -- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 msgr2=0x7f689006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:23.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.517+0000 7f68abfff700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 0x7f689006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:23.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.564+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68ac0d45c0 con 0x7f68ac160d20 2023-10-16T11:00:23.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.564+0000 7f68a97fa700 1 -- 172.21.15.181:0/2361579603 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f68a0059110 con 0x7f68ac160d20 2023-10-16T11:00:23.566 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:23.566 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1229456263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1229456263},{"type":"v1","addr":"172.21.15.190:6834","nonce":1229456263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3176781358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3176781358},{"type":"v1","addr":"172.21.15.181:6836","nonce":3176781358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2996369280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2996369280},{"type":"v1","addr":"172.21.15.181:6837","nonce":2996369280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:01.954427+0000","modified":"2023-10-16T10:04:04.993634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8146":{"gid":8146,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3854101486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3854101486},{"type":"v1","addr":"172.21.15.190:6835","nonce":3854101486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:00:23.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 msgr2=0x7f689006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:23.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 0x7f689006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac160d20 msgr2=0x7f68ac0b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac160d20 0x7f68ac0b6420 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f68a0000c00 tx=0x7f68a0004750 comp rx=0 tx=0).stop 2023-10-16T11:00:23.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 shutdown_connections 2023-10-16T11:00:23.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f689006b020 0x7f689006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68ac161f40 0x7f68ac0b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac160d20 0x7f68ac0b6420 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 --2- 172.21.15.181:0/2361579603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68ac161b60 0x7f68ac1f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.566+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 >> 172.21.15.181:0/2361579603 conn(0x7f68ac09cba0 msgr2=0x7f68ac15f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:23.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.567+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 shutdown_connections 2023-10-16T11:00:23.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.567+0000 7f68b307e700 1 -- 172.21.15.181:0/2361579603 wait complete. 2023-10-16T11:00:23.571 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 76 2023-10-16T11:00:23.581 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:23.640 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:23.639+0000 7f52d6f76200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.705+0000 7fb69c108700 1 Processor -- start 2023-10-16T11:00:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.705+0000 7fb69c108700 1 -- start start 2023-10-16T11:00:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb69c108700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6940d2390 0x7fb6940cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb69c108700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6940cc470 con 0x7fb6940ce7f0 2023-10-16T11:00:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb69c108700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69406ca30 con 0x7fb6940d0180 2023-10-16T11:00:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb69c108700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69406c810 con 0x7fb6940d2390 2023-10-16T11:00:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb699ea4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb6940ce7f0 0x7fb69406c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32926/0 (socket says 172.21.15.181:32926) 2023-10-16T11:00:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb699ea4700 1 -- 172.21.15.181:0/4119928623 learned_addr learned my addr 172.21.15.181:0/4119928623 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb6996a3700 1 --2- 172.21.15.181:0/4119928623 >> v2:172.21.15.190:3300/0 conn(0x7fb6940d2390 0x7fb6940cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb698ea2700 1 -- 172.21.15.181:0/4119928623 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675561603 0 0) 0x7fb6940cc470 con 0x7fb6940ce7f0 2023-10-16T11:00:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb698ea2700 1 -- 172.21.15.181:0/4119928623 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6800035b0 con 0x7fb6940ce7f0 2023-10-16T11:00:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb6996a3700 1 -- 172.21.15.181:0/4119928623 >> v1:172.21.15.190:6789/0 conn(0x7fb6940d0180 legacy=0x7fb6940cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb698ea2700 1 -- 172.21.15.181:0/4119928623 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891344551 0 0) 0x7fb69406ca30 con 0x7fb6940d0180 2023-10-16T11:00:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.706+0000 7fb6996a3700 1 -- 172.21.15.181:0/4119928623 >> v1:172.21.15.181:6789/0 conn(0x7fb6940ce7f0 legacy=0x7fb69406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb6996a3700 1 -- 172.21.15.181:0/4119928623 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69406cd10 con 0x7fb6940d2390 2023-10-16T11:00:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb6996a3700 1 --2- 172.21.15.181:0/4119928623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d2390 0x7fb6940cfc40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb684009840 tx=0x7fb684009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9334e7c0a019320 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb698ea2700 1 -- 172.21.15.181:0/4119928623 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684011830 con 0x7fb6940d2390 2023-10-16T11:00:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb698ea2700 1 -- 172.21.15.181:0/4119928623 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb684011990 con 0x7fb6940d2390 2023-10-16T11:00:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb698ea2700 1 -- 172.21.15.181:0/4119928623 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684011d20 con 0x7fb6940d2390 2023-10-16T11:00:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb69c108700 1 -- 172.21.15.181:0/4119928623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d2390 msgr2=0x7fb6940cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb69c108700 1 --2- 172.21.15.181:0/4119928623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d2390 0x7fb6940cfc40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb684009840 tx=0x7fb684009b50 comp rx=0 tx=0).stop 2023-10-16T11:00:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb69c108700 1 -- 172.21.15.181:0/4119928623 shutdown_connections 2023-10-16T11:00:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb69c108700 1 --2- 172.21.15.181:0/4119928623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d2390 0x7fb6940cfc40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.707+0000 7fb69c108700 1 -- 172.21.15.181:0/4119928623 >> 172.21.15.181:0/4119928623 conn(0x7fb69409a9a0 msgr2=0x7fb6940d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.708+0000 7fb69c108700 1 -- 172.21.15.181:0/4119928623 shutdown_connections 2023-10-16T11:00:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.708+0000 7fb69c108700 1 -- 172.21.15.181:0/4119928623 wait complete. 2023-10-16T11:00:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.709+0000 7fb69c108700 1 Processor -- start 2023-10-16T11:00:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.709+0000 7fb69c108700 1 -- start start 2023-10-16T11:00:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb69c108700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6940ce7f0 0x7fb6941564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb69c108700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d0180 0x7fb69415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb69c108700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6940d2390 0x7fb6940b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb69c108700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69415c8a0 con 0x7fb6940ce7f0 2023-10-16T11:00:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb69c108700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69415ba40 con 0x7fb6940d0180 2023-10-16T11:00:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb69c108700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69415bc90 con 0x7fb6940d2390 2023-10-16T11:00:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6940ce7f0 0x7fb6941564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6940ce7f0 0x7fb6941564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50086/0 (socket says 172.21.15.181:50086) 2023-10-16T11:00:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 -- 172.21.15.181:0/3344946321 learned_addr learned my addr 172.21.15.181:0/3344946321 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb6996a3700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d0180 0x7fb69415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb69a6a5700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6940d2390 0x7fb6940b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 -- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6940d2390 msgr2=0x7fb6940b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6940d2390 0x7fb6940b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 -- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d0180 msgr2=0x7fb69415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d0180 0x7fb69415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb699ea4700 1 -- 172.21.15.181:0/3344946321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6840094a0 con 0x7fb6940ce7f0 2023-10-16T11:00:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.710+0000 7fb6996a3700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d0180 0x7fb69415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.711+0000 7fb699ea4700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6940ce7f0 0x7fb6941564d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c4d0 tx=0x7fb69000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.711+0000 7fb68a7fc700 1 -- 172.21.15.181:0/3344946321 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900074e0 con 0x7fb6940ce7f0 2023-10-16T11:00:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.711+0000 7fb68a7fc700 1 -- 172.21.15.181:0/3344946321 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb690007640 con 0x7fb6940ce7f0 2023-10-16T11:00:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.711+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69415bf70 con 0x7fb6940ce7f0 2023-10-16T11:00:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.711+0000 7fb68a7fc700 1 -- 172.21.15.181:0/3344946321 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690005a70 con 0x7fb6940ce7f0 2023-10-16T11:00:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.711+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69415c530 con 0x7fb6940ce7f0 2023-10-16T11:00:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.712+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6940cb630 con 0x7fb6940ce7f0 2023-10-16T11:00:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.712+0000 7fb68a7fc700 1 -- 172.21.15.181:0/3344946321 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fb690005bd0 con 0x7fb6940ce7f0 2023-10-16T11:00:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.713+0000 7fb68a7fc700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 0x7fb67806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.713+0000 7fb6996a3700 1 -- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 msgr2=0x7fb67806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.713+0000 7fb68a7fc700 1 -- 172.21.15.181:0/3344946321 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fb6900890c0 con 0x7fb6940ce7f0 2023-10-16T11:00:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.713+0000 7fb6996a3700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 0x7fb67806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:23.722 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:23.717+0000 7f52d6f76200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:00:23.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.717+0000 7fb68a7fc700 1 -- 172.21.15.181:0/3344946321 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb690053c30 con 0x7fb6940ce7f0 2023-10-16T11:00:23.774 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:23.772+0000 7f76c1d77200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:23.855 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:23.854+0000 7f76c1d77200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:00:23.863 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:23.862+0000 7f52d6f76200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:23.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.914+0000 7fb6996a3700 1 -- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 msgr2=0x7fb67806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:23.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.914+0000 7fb6996a3700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 0x7fb67806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:23.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.920+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb694067d20 con 0x7fb6940ce7f0 2023-10-16T11:00:23.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.922+0000 7fb68a7fc700 1 -- 172.21.15.181:0/3344946321 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+18019 (secure 0 0 0) 0x7fb690057250 con 0x7fb6940ce7f0 2023-10-16T11:00:23.924 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:23.925 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":155,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:21.787794+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:00.393030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:00.916899+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":155,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.13","mappings":[{"from":2,"to":5}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1561933478":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1604670603":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6833/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6832/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1093898558":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/40763561":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/2364682800":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1727687969":"2023-10-17T10:04:15.822177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":50,"length":7},{"begin":106,"length":7}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:23.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 msgr2=0x7fb67806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:23.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 0x7fb67806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6940ce7f0 msgr2=0x7fb6941564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:23.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6940ce7f0 0x7fb6941564d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c4d0 tx=0x7fb69000c890 comp rx=0 tx=0).stop 2023-10-16T11:00:23.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 shutdown_connections 2023-10-16T11:00:23.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fb67806b020 0x7fb67806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6940d2390 0x7fb6940b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6940ce7f0 0x7fb6941564d0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 --2- 172.21.15.181:0/3344946321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6940d0180 0x7fb69415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:23.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 >> 172.21.15.181:0/3344946321 conn(0x7fb69409a9a0 msgr2=0x7fb69406edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:23.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 shutdown_connections 2023-10-16T11:00:23.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:23.925+0000 7fb69c108700 1 -- 172.21.15.181:0/3344946321 wait complete. 2023-10-16T11:00:23.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T10:04:00.393030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T10:04:00.916899+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 155, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:00:23.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:00:24.024 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:24.022+0000 7f76c1d77200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:24.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.078+0000 7f1e504a9700 1 Processor -- start 2023-10-16T11:00:24.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.078+0000 7f1e504a9700 1 -- start start 2023-10-16T11:00:24.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e504a9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e481646b0 0x7f1e48166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e504a9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e480b1f70 con 0x7f1e4815f150 2023-10-16T11:00:24.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e504a9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e480b2ae0 con 0x7f1e480ce6c0 2023-10-16T11:00:24.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e504a9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e480cd880 con 0x7f1e481646b0 2023-10-16T11:00:24.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4ea46700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e481646b0 0x7f1e48166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4e245700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1e4815f150 0x7f1e480cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32968/0 (socket says 172.21.15.181:32968) 2023-10-16T11:00:24.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4e245700 1 -- 172.21.15.181:0/3936512016 learned_addr learned my addr 172.21.15.181:0/3936512016 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:24.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4ea46700 1 -- 172.21.15.181:0/3936512016 >> v1:172.21.15.190:6789/0 conn(0x7f1e480ce6c0 legacy=0x7f1e480cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4ea46700 1 -- 172.21.15.181:0/3936512016 >> v1:172.21.15.181:6789/0 conn(0x7f1e4815f150 legacy=0x7f1e480cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4ea46700 1 -- 172.21.15.181:0/3936512016 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e480b2dc0 con 0x7f1e481646b0 2023-10-16T11:00:24.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4ea46700 1 --2- 172.21.15.181:0/3936512016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48166aa0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009d90 tx=0x7f1e4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3737e29d90c845 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:24.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4d243700 1 -- 172.21.15.181:0/3936512016 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000e070 con 0x7f1e481646b0 2023-10-16T11:00:24.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4d243700 1 -- 172.21.15.181:0/3936512016 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4000bad0 con 0x7f1e481646b0 2023-10-16T11:00:24.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.079+0000 7f1e4d243700 1 -- 172.21.15.181:0/3936512016 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000be80 con 0x7f1e481646b0 2023-10-16T11:00:24.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.080+0000 7f1e504a9700 1 -- 172.21.15.181:0/3936512016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 msgr2=0x7f1e48166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.080+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3936512016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48166aa0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009d90 tx=0x7f1e4000b330 comp rx=0 tx=0).stop 2023-10-16T11:00:24.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.080+0000 7f1e504a9700 1 -- 172.21.15.181:0/3936512016 shutdown_connections 2023-10-16T11:00:24.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.080+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3936512016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48166aa0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.080+0000 7f1e504a9700 1 -- 172.21.15.181:0/3936512016 >> 172.21.15.181:0/3936512016 conn(0x7f1e4809a930 msgr2=0x7f1e480d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:24.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.080+0000 7f1e504a9700 1 -- 172.21.15.181:0/3936512016 shutdown_connections 2023-10-16T11:00:24.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.080+0000 7f1e504a9700 1 -- 172.21.15.181:0/3936512016 wait complete. 2023-10-16T11:00:24.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.081+0000 7f1e504a9700 1 Processor -- start 2023-10-16T11:00:24.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.081+0000 7f1e504a9700 1 -- start start 2023-10-16T11:00:24.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e504a9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e480ce6c0 0x7f1e4814fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e504a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e4815f150 0x7f1e48154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e504a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e504a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e481542b0 con 0x7f1e481646b0 2023-10-16T11:00:24.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e504a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4806d210 con 0x7f1e4815f150 2023-10-16T11:00:24.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e504a9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4806c600 con 0x7f1e480ce6c0 2023-10-16T11:00:24.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e4ea46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e4ea46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48163940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50132/0 (socket says 172.21.15.181:50132) 2023-10-16T11:00:24.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e4ea46700 1 -- 172.21.15.181:0/3951842278 learned_addr learned my addr 172.21.15.181:0/3951842278 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:24.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e4da44700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e4815f150 0x7f1e48154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.082+0000 7f1e4e245700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e480ce6c0 0x7f1e4814fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e4ea46700 1 -- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e480ce6c0 msgr2=0x7f1e4814fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e4ea46700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e480ce6c0 0x7f1e4814fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e4ea46700 1 -- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e4815f150 msgr2=0x7f1e48154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e4ea46700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e4815f150 0x7f1e48154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e4ea46700 1 -- 172.21.15.181:0/3951842278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40009a70 con 0x7f1e481646b0 2023-10-16T11:00:24.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e4da44700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e4815f150 0x7f1e48154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:24.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e4ea46700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48163940 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806b3a0 tx=0x7f1e40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:24.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e3effd700 1 -- 172.21.15.181:0/3951842278 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40004970 con 0x7f1e481646b0 2023-10-16T11:00:24.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e3effd700 1 -- 172.21.15.181:0/3951842278 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e40007080 con 0x7f1e481646b0 2023-10-16T11:00:24.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4806c8e0 con 0x7f1e481646b0 2023-10-16T11:00:24.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e3effd700 1 -- 172.21.15.181:0/3951842278 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40012870 con 0x7f1e481646b0 2023-10-16T11:00:24.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.083+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4806cea0 con 0x7f1e481646b0 2023-10-16T11:00:24.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.084+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4806d3a0 con 0x7f1e481646b0 2023-10-16T11:00:24.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.084+0000 7f1e3effd700 1 -- 172.21.15.181:0/3951842278 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f1e400129d0 con 0x7f1e481646b0 2023-10-16T11:00:24.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.085+0000 7f1e3effd700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 0x7f1e2c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.085+0000 7f1e4e245700 1 -- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 msgr2=0x7f1e2c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:24.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.085+0000 7f1e3effd700 1 -- 172.21.15.181:0/3951842278 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f1e4008bd50 con 0x7f1e481646b0 2023-10-16T11:00:24.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.085+0000 7f1e4e245700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 0x7f1e2c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:24.095 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:24.087+0000 7f52d6f76200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:00:24.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.089+0000 7f1e3effd700 1 -- 172.21.15.181:0/3951842278 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1e40056820 con 0x7f1e481646b0 2023-10-16T11:00:24.162 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:24.161+0000 7f52d6f76200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:00:24.262 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:24.260+0000 7f76c1d77200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:00:24.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.286+0000 7f1e4e245700 1 -- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 msgr2=0x7f1e2c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:24.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.286+0000 7f1e4e245700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 0x7f1e2c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:24.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.288+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e48067d20 con 0x7f1e481646b0 2023-10-16T11:00:24.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.288+0000 7f1e3effd700 1 -- 172.21.15.181:0/3951842278 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e40059e40 con 0x7f1e481646b0 2023-10-16T11:00:24.290 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:00:24.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 msgr2=0x7f1e2c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:24.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 0x7f1e2c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 msgr2=0x7f1e48163940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48163940 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806b3a0 tx=0x7f1e40004ad0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 shutdown_connections 2023-10-16T11:00:24.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f1e2c06b020 0x7f1e2c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e480ce6c0 0x7f1e4814fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e481646b0 0x7f1e48163940 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 --2- 172.21.15.181:0/3951842278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e4815f150 0x7f1e48154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 >> 172.21.15.181:0/3951842278 conn(0x7f1e4809a930 msgr2=0x7f1e4815e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:24.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 shutdown_connections 2023-10-16T11:00:24.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.290+0000 7f1e504a9700 1 -- 172.21.15.181:0/3951842278 wait complete. 2023-10-16T11:00:24.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:00:24.345 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:24.344+0000 7f76c1d77200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:00:24.428 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:24.427+0000 7f52d6f76200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:00:24.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.435+0000 7fd9ecd40700 1 Processor -- start 2023-10-16T11:00:24.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.435+0000 7fd9ecd40700 1 -- start start 2023-10-16T11:00:24.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9ecd40700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9e806afb0 0x7fd9e81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9ecd40700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e80cef00 con 0x7fd9e815da70 2023-10-16T11:00:24.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9ecd40700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e80cec20 con 0x7fd9e8072760 2023-10-16T11:00:24.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9ecd40700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e8071c80 con 0x7fd9e806afb0 2023-10-16T11:00:24.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd9e815da70 0x7fd9e80ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32984/0 (socket says 172.21.15.181:32984) 2023-10-16T11:00:24.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e659c700 1 -- 172.21.15.181:0/2139506468 learned_addr learned my addr 172.21.15.181:0/2139506468 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:24.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e6d9d700 1 --2- 172.21.15.181:0/2139506468 >> v2:172.21.15.190:3300/0 conn(0x7fd9e806afb0 0x7fd9e81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e559a700 1 -- 172.21.15.181:0/2139506468 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187364005 0 0) 0x7fd9e80cef00 con 0x7fd9e815da70 2023-10-16T11:00:24.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e559a700 1 -- 172.21.15.181:0/2139506468 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d00035b0 con 0x7fd9e815da70 2023-10-16T11:00:24.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e6d9d700 1 -- 172.21.15.181:0/2139506468 >> v1:172.21.15.190:6789/0 conn(0x7fd9e8072760 legacy=0x7fd9e806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e559a700 1 -- 172.21.15.181:0/2139506468 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918984759 0 0) 0x7fd9e80cec20 con 0x7fd9e8072760 2023-10-16T11:00:24.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e6d9d700 1 -- 172.21.15.181:0/2139506468 >> v1:172.21.15.181:6789/0 conn(0x7fd9e815da70 legacy=0x7fd9e80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.436+0000 7fd9e6d9d700 1 -- 172.21.15.181:0/2139506468 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8071f60 con 0x7fd9e806afb0 2023-10-16T11:00:24.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.437+0000 7fd9e6d9d700 1 --2- 172.21.15.181:0/2139506468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e806afb0 0x7fd9e81615f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009d90 tx=0x7fd9e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510130df065b9726 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:24.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.437+0000 7fd9e559a700 1 -- 172.21.15.181:0/2139506468 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000e070 con 0x7fd9e806afb0 2023-10-16T11:00:24.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.437+0000 7fd9e559a700 1 -- 172.21.15.181:0/2139506468 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e000bb20 con 0x7fd9e806afb0 2023-10-16T11:00:24.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.437+0000 7fd9e559a700 1 -- 172.21.15.181:0/2139506468 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000bed0 con 0x7fd9e806afb0 2023-10-16T11:00:24.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.437+0000 7fd9ecd40700 1 -- 172.21.15.181:0/2139506468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e806afb0 msgr2=0x7fd9e81615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.437+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/2139506468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e806afb0 0x7fd9e81615f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009d90 tx=0x7fd9e000b330 comp rx=0 tx=0).stop 2023-10-16T11:00:24.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.438+0000 7fd9ecd40700 1 -- 172.21.15.181:0/2139506468 shutdown_connections 2023-10-16T11:00:24.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.438+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/2139506468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e806afb0 0x7fd9e81615f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.438+0000 7fd9ecd40700 1 -- 172.21.15.181:0/2139506468 >> 172.21.15.181:0/2139506468 conn(0x7fd9e80cd010 msgr2=0x7fd9e80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:24.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.438+0000 7fd9ecd40700 1 -- 172.21.15.181:0/2139506468 shutdown_connections 2023-10-16T11:00:24.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.438+0000 7fd9ecd40700 1 -- 172.21.15.181:0/2139506468 wait complete. 2023-10-16T11:00:24.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.439+0000 7fd9ecd40700 1 Processor -- start 2023-10-16T11:00:24.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.439+0000 7fd9ecd40700 1 -- start start 2023-10-16T11:00:24.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9ecd40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9e806afb0 0x7fd9e81f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9ecd40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8072760 0x7fd9e80ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9ecd40700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e815da70 0x7fd9e80ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9ecd40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80c41e0 con 0x7fd9e8072760 2023-10-16T11:00:24.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9ecd40700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80b4f00 con 0x7fd9e815da70 2023-10-16T11:00:24.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9ecd40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80b51e0 con 0x7fd9e806afb0 2023-10-16T11:00:24.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8072760 0x7fd9e80ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8072760 0x7fd9e80ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50146/0 (socket says 172.21.15.181:50146) 2023-10-16T11:00:24.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 -- 172.21.15.181:0/3345490932 learned_addr learned my addr 172.21.15.181:0/3345490932 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:24.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e659c700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9e806afb0 0x7fd9e81f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e6d9d700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e815da70 0x7fd9e80ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 -- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9e806afb0 msgr2=0x7fd9e81f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9e806afb0 0x7fd9e81f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 -- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e815da70 msgr2=0x7fd9e80ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e815da70 0x7fd9e80ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 -- 172.21.15.181:0/3345490932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e0009a70 con 0x7fd9e8072760 2023-10-16T11:00:24.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9e5d9b700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8072760 0x7fd9e80ad290 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0011e0 tx=0x7fd9dc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:24.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.440+0000 7fd9d6ffd700 1 -- 172.21.15.181:0/3345490932 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0054e0 con 0x7fd9e8072760 2023-10-16T11:00:24.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.441+0000 7fd9d6ffd700 1 -- 172.21.15.181:0/3345490932 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc003680 con 0x7fd9e8072760 2023-10-16T11:00:24.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.441+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e80b54c0 con 0x7fd9e8072760 2023-10-16T11:00:24.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.441+0000 7fd9d6ffd700 1 -- 172.21.15.181:0/3345490932 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc003a30 con 0x7fd9e8072760 2023-10-16T11:00:24.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.441+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e80b3630 con 0x7fd9e8072760 2023-10-16T11:00:24.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.442+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e80b31b0 con 0x7fd9e8072760 2023-10-16T11:00:24.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.442+0000 7fd9d6ffd700 1 -- 172.21.15.181:0/3345490932 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd9dc003b90 con 0x7fd9e8072760 2023-10-16T11:00:24.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.442+0000 7fd9d6ffd700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 0x7fd9c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.443+0000 7fd9e659c700 1 -- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 msgr2=0x7fd9c806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:24.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.443+0000 7fd9d6ffd700 1 -- 172.21.15.181:0/3345490932 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fd9dc088f10 con 0x7fd9e8072760 2023-10-16T11:00:24.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.443+0000 7fd9e659c700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 0x7fd9c806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:24.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.447+0000 7fd9d6ffd700 1 -- 172.21.15.181:0/3345490932 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd9e80b31b0 con 0x7fd9e8072760 2023-10-16T11:00:24.628 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:24.626+0000 7f76c1d77200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:00:24.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.644+0000 7fd9e659c700 1 -- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 msgr2=0x7fd9c806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:24.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.644+0000 7fd9e659c700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 0x7fd9c806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:24.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.649+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9e80b31b0 con 0x7fd9e8072760 2023-10-16T11:00:24.651 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:24.650+0000 7f52d6f76200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:00:24.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.650+0000 7fd9d6ffd700 1 -- 172.21.15.181:0/3345490932 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9e80b31b0 con 0x7fd9e8072760 2023-10-16T11:00:24.652 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:00:24.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 msgr2=0x7fd9c806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:24.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 0x7fd9c806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8072760 msgr2=0x7fd9e80ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8072760 0x7fd9e80ad290 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0011e0 tx=0x7fd9dc00ca30 comp rx=0 tx=0).stop 2023-10-16T11:00:24.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 shutdown_connections 2023-10-16T11:00:24.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fd9c806b020 0x7fd9c806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9e806afb0 0x7fd9e81f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9e815da70 0x7fd9e80ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 --2- 172.21.15.181:0/3345490932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9e8072760 0x7fd9e80ad290 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 >> 172.21.15.181:0/3345490932 conn(0x7fd9e80cd010 msgr2=0x7fd9e815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:24.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 shutdown_connections 2023-10-16T11:00:24.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.652+0000 7fd9ecd40700 1 -- 172.21.15.181:0/3345490932 wait complete. 2023-10-16T11:00:24.666 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:00:24.802 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:24.801+0000 7f52d6f76200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:00:24.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.833+0000 7fa9255de700 1 Processor -- start 2023-10-16T11:00:24.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.833+0000 7fa9255de700 1 -- start start 2023-10-16T11:00:24.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa9255de700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9200d01d0 0x7fa9200ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa9255de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92015ea40 0x7fa9200d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa9255de700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9200cf580 con 0x7fa920161530 2023-10-16T11:00:24.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa9255de700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9200d4290 con 0x7fa9200d01d0 2023-10-16T11:00:24.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa9255de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9200d0af0 con 0x7fa92015ea40 2023-10-16T11:00:24.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9200d01d0 0x7fa9200ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9200d01d0 0x7fa9200ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50154/0 (socket says 172.21.15.181:50154) 2023-10-16T11:00:24.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 -- 172.21.15.181:0/1784495080 learned_addr learned my addr 172.21.15.181:0/1784495080 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:24.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91e7fc700 1 --2- 172.21.15.181:0/1784495080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92015ea40 0x7fa9200d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 -- 172.21.15.181:0/1784495080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92015ea40 msgr2=0x7fa9200d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:24.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 --2- 172.21.15.181:0/1784495080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92015ea40 0x7fa9200d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 -- 172.21.15.181:0/1784495080 >> v1:172.21.15.190:6789/0 conn(0x7fa920161530 legacy=0x7fa9201619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 -- 172.21.15.181:0/1784495080 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9200ce970 con 0x7fa9200d01d0 2023-10-16T11:00:24.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91dffb700 1 -- 172.21.15.181:0/1784495080 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365048146 0 0) 0x7fa9200cf580 con 0x7fa920161530 2023-10-16T11:00:24.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91effd700 1 --2- 172.21.15.181:0/1784495080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200d01d0 0x7fa9200ce2d0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098f0 tx=0x7fa910009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955a2a73cfcc0646 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:24.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91dffb700 1 -- 172.21.15.181:0/1784495080 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011830 con 0x7fa9200d01d0 2023-10-16T11:00:24.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91dffb700 1 -- 172.21.15.181:0/1784495080 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910011990 con 0x7fa9200d01d0 2023-10-16T11:00:24.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.834+0000 7fa91dffb700 1 -- 172.21.15.181:0/1784495080 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011d20 con 0x7fa9200d01d0 2023-10-16T11:00:24.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 -- 172.21.15.181:0/1784495080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200d01d0 msgr2=0x7fa9200ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 --2- 172.21.15.181:0/1784495080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200d01d0 0x7fa9200ce2d0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa9100098f0 tx=0x7fa910009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:24.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 -- 172.21.15.181:0/1784495080 shutdown_connections 2023-10-16T11:00:24.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 --2- 172.21.15.181:0/1784495080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200d01d0 0x7fa9200ce2d0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 --2- 172.21.15.181:0/1784495080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa92015ea40 0x7fa9200d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 -- 172.21.15.181:0/1784495080 >> 172.21.15.181:0/1784495080 conn(0x7fa92009cb60 msgr2=0x7fa9200d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:24.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 -- 172.21.15.181:0/1784495080 shutdown_connections 2023-10-16T11:00:24.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.835+0000 7fa9255de700 1 -- 172.21.15.181:0/1784495080 wait complete. 2023-10-16T11:00:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.836+0000 7fa9255de700 1 Processor -- start 2023-10-16T11:00:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.836+0000 7fa9255de700 1 -- start start 2023-10-16T11:00:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa9255de700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9200d01d0 0x7fa9201f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa9255de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015ea40 0x7fa9200b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa9255de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa920161530 0x7fa9200b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa9255de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92014ba70 con 0x7fa92015ea40 2023-10-16T11:00:24.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa9255de700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92014bcc0 con 0x7fa9200d01d0 2023-10-16T11:00:24.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa9255de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92014bfa0 con 0x7fa920161530 2023-10-16T11:00:24.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa91e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015ea40 0x7fa9200b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa91effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9200d01d0 0x7fa9201f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa91effd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9200d01d0 0x7fa9201f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:55210/0 (socket says 172.21.15.181:55210) 2023-10-16T11:00:24.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa91effd700 1 -- 172.21.15.181:0/741353578 learned_addr learned my addr 172.21.15.181:0/741353578 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:24.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa91e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015ea40 0x7fa9200b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50160/0 (socket says 172.21.15.181:50160) 2023-10-16T11:00:24.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.837+0000 7fa91f7fe700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa920161530 0x7fa9200b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:24.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa91effd700 1 -- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa920161530 msgr2=0x7fa9200b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa91effd700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa920161530 0x7fa9200b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa91effd700 1 -- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015ea40 msgr2=0x7fa9200b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:24.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa91effd700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015ea40 0x7fa9200b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:24.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa91effd700 1 -- 172.21.15.181:0/741353578 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910009550 con 0x7fa9200d01d0 2023-10-16T11:00:24.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa91e7fc700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015ea40 0x7fa9200b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa91effd700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9200d01d0 0x7fa9201f6090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa910009c40 tx=0x7fa910004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa9077fe700 1 -- 172.21.15.181:0/741353578 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100044d0 con 0x7fa9200d01d0 2023-10-16T11:00:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa9077fe700 1 -- 172.21.15.181:0/741353578 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91001e070 con 0x7fa9200d01d0 2023-10-16T11:00:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa9077fe700 1 -- 172.21.15.181:0/741353578 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100038e0 con 0x7fa9200d01d0 2023-10-16T11:00:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92014c280 con 0x7fa9200d01d0 2023-10-16T11:00:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.838+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9201f65d0 con 0x7fa9200d01d0 2023-10-16T11:00:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.839+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92004ebb0 con 0x7fa9200d01d0 2023-10-16T11:00:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.839+0000 7fa9077fe700 1 -- 172.21.15.181:0/741353578 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fa910003a40 con 0x7fa9200d01d0 2023-10-16T11:00:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.840+0000 7fa9077fe700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 0x7fa90c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.840+0000 7fa91e7fc700 1 -- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 msgr2=0x7fa90c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.840+0000 7fa9077fe700 1 -- 172.21.15.181:0/741353578 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fa91008a9b0 con 0x7fa9200d01d0 2023-10-16T11:00:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.840+0000 7fa91e7fc700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 0x7fa90c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:24.844+0000 7fa9077fe700 1 -- 172.21.15.181:0/741353578 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa910055590 con 0x7fa9200d01d0 2023-10-16T11:00:24.867 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:24.865+0000 7f76c1d77200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:00:24.917 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:24.916+0000 7f52d6f76200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:00:25.021 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:25.019+0000 7f76c1d77200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:00:25.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.041+0000 7fa91e7fc700 1 -- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 msgr2=0x7fa90c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:25.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.041+0000 7fa91e7fc700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 0x7fa90c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:25.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.042+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9200d0e00 con 0x7fa9200d01d0 2023-10-16T11:00:25.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.043+0000 7fa9077fe700 1 -- 172.21.15.181:0/741353578 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9100200a0 con 0x7fa9200d01d0 2023-10-16T11:00:25.044 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:00:25.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 msgr2=0x7fa90c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:25.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 0x7fa90c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9200d01d0 msgr2=0x7fa9201f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9200d01d0 0x7fa9201f6090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa910009c40 tx=0x7fa910004750 comp rx=0 tx=0).stop 2023-10-16T11:00:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 shutdown_connections 2023-10-16T11:00:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7fa90c06b020 0x7fa90c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa920161530 0x7fa9200b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9200d01d0 0x7fa9201f6090 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 --2- 172.21.15.181:0/741353578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015ea40 0x7fa9200b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 >> 172.21.15.181:0/741353578 conn(0x7fa92009cb60 msgr2=0x7fa920161940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:25.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 shutdown_connections 2023-10-16T11:00:25.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.045+0000 7fa9255de700 1 -- 172.21.15.181:0/741353578 wait complete. 2023-10-16T11:00:25.059 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:00:25.073 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:25.072+0000 7f52d6f76200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:00:25.142 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:25.140+0000 7f76c1d77200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:00:25.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.180+0000 7ff3aa883700 1 Processor -- start 2023-10-16T11:00:25.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.180+0000 7ff3aa883700 1 -- start start 2023-10-16T11:00:25.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3aa883700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3a415dfd0 0x7ff3a40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3aa883700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a40cd470 0x7ff3a40cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3aa883700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a406f870 con 0x7ff3a40af450 2023-10-16T11:00:25.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3aa883700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a40cde40 con 0x7ff3a415dfd0 2023-10-16T11:00:25.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3aa883700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40d1160 con 0x7ff3a40cd470 2023-10-16T11:00:25.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3a8e20700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a40cd470 0x7ff3a40cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3a3fff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3a415dfd0 0x7ff3a40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3a8e20700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a40cd470 0x7ff3a40cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:47200/0 (socket says 172.21.15.181:47200) 2023-10-16T11:00:25.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.181+0000 7ff3a37fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff3a40af450 0x7ff3a406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:45190/0 (socket says 172.21.15.181:45190) 2023-10-16T11:00:25.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a8e20700 1 -- 172.21.15.181:0/3535827375 learned_addr learned my addr 172.21.15.181:0/3535827375 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:25.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a8e20700 1 -- 172.21.15.181:0/3535827375 >> v2:172.21.15.190:3300/0 conn(0x7ff3a415dfd0 msgr2=0x7ff3a40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:25.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a8e20700 1 --2- 172.21.15.181:0/3535827375 >> v2:172.21.15.190:3300/0 conn(0x7ff3a415dfd0 0x7ff3a40af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a8e20700 1 -- 172.21.15.181:0/3535827375 >> v1:172.21.15.190:6789/0 conn(0x7ff3a40af450 legacy=0x7ff3a406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a8e20700 1 -- 172.21.15.181:0/3535827375 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40cc8e0 con 0x7ff3a40cd470 2023-10-16T11:00:25.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a3fff700 1 --2- 172.21.15.181:0/3535827375 >> v2:172.21.15.190:3300/0 conn(0x7ff3a415dfd0 0x7ff3a40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:25.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a8e20700 1 --2- 172.21.15.181:0/3535827375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a40cd470 0x7ff3a40cd900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009e50 tx=0x7ff39c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd99c02bfbcbe6fc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:25.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a2ffd700 1 -- 172.21.15.181:0/3535827375 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00e070 con 0x7ff3a40cd470 2023-10-16T11:00:25.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a2ffd700 1 -- 172.21.15.181:0/3535827375 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c00b910 con 0x7ff3a40cd470 2023-10-16T11:00:25.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.182+0000 7ff3a2ffd700 1 -- 172.21.15.181:0/3535827375 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00bcc0 con 0x7ff3a40cd470 2023-10-16T11:00:25.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 -- 172.21.15.181:0/3535827375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a40cd470 msgr2=0x7ff3a40cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 --2- 172.21.15.181:0/3535827375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a40cd470 0x7ff3a40cd900 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009e50 tx=0x7ff39c00b330 comp rx=0 tx=0).stop 2023-10-16T11:00:25.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 -- 172.21.15.181:0/3535827375 shutdown_connections 2023-10-16T11:00:25.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 --2- 172.21.15.181:0/3535827375 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a40cd470 0x7ff3a40cd900 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 --2- 172.21.15.181:0/3535827375 >> v2:172.21.15.190:3300/0 conn(0x7ff3a415dfd0 0x7ff3a40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 -- 172.21.15.181:0/3535827375 >> 172.21.15.181:0/3535827375 conn(0x7ff3a40d04a0 msgr2=0x7ff3a40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:25.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 -- 172.21.15.181:0/3535827375 shutdown_connections 2023-10-16T11:00:25.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.183+0000 7ff3aa883700 1 -- 172.21.15.181:0/3535827375 wait complete. 2023-10-16T11:00:25.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.184+0000 7ff3aa883700 1 Processor -- start 2023-10-16T11:00:25.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.185+0000 7ff3aa883700 1 -- start start 2023-10-16T11:00:25.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.185+0000 7ff3aa883700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a40af450 0x7ff3a40c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.185+0000 7ff3aa883700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40cd470 0x7ff3a40c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.185+0000 7ff3aa883700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a415dfd0 0x7ff3a40ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3aa883700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c68a0 con 0x7ff3a40cd470 2023-10-16T11:00:25.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3aa883700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c7200 con 0x7ff3a40af450 2023-10-16T11:00:25.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3aa883700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c4b40 con 0x7ff3a415dfd0 2023-10-16T11:00:25.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40cd470 0x7ff3a40c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40cd470 0x7ff3a40c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50178/0 (socket says 172.21.15.181:50178) 2023-10-16T11:00:25.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 -- 172.21.15.181:0/4011608977 learned_addr learned my addr 172.21.15.181:0/4011608977 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:25.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a3fff700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a40af450 0x7ff3a40c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a8e20700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a415dfd0 0x7ff3a40ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 -- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a415dfd0 msgr2=0x7ff3a40ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a415dfd0 0x7ff3a40ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 -- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a40af450 msgr2=0x7ff3a40c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a40af450 0x7ff3a40c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 -- 172.21.15.181:0/4011608977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394009430 con 0x7ff3a40cd470 2023-10-16T11:00:25.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a37fe700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40cd470 0x7ff3a40c4600 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c530 tx=0x7ff39800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:25.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a0ff9700 1 -- 172.21.15.181:0/4011608977 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39800cea0 con 0x7ff3a40cd470 2023-10-16T11:00:25.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a0ff9700 1 -- 172.21.15.181:0/4011608977 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff398005cb0 con 0x7ff3a40cd470 2023-10-16T11:00:25.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c009b30 con 0x7ff3a40cd470 2023-10-16T11:00:25.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3a0ff9700 1 -- 172.21.15.181:0/4011608977 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980072d0 con 0x7ff3a40cd470 2023-10-16T11:00:25.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.186+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a40c6200 con 0x7ff3a40cd470 2023-10-16T11:00:25.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.187+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a40ccba0 con 0x7ff3a40cd470 2023-10-16T11:00:25.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.188+0000 7ff3a0ff9700 1 -- 172.21.15.181:0/4011608977 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff398007430 con 0x7ff3a40cd470 2023-10-16T11:00:25.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.188+0000 7ff3a0ff9700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 0x7ff39006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.188+0000 7ff3a3fff700 1 -- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 msgr2=0x7ff39006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:25.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.188+0000 7ff3a0ff9700 1 -- 172.21.15.181:0/4011608977 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7ff398089050 con 0x7ff3a40cd470 2023-10-16T11:00:25.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.188+0000 7ff3a3fff700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 0x7ff39006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:25.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.192+0000 7ff3a0ff9700 1 -- 172.21.15.181:0/4011608977 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff398053cd0 con 0x7ff3a40cd470 2023-10-16T11:00:25.305 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:25.304+0000 7f76c1d77200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:00:25.336 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:25.335+0000 7f52d6f76200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:00:25.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.389+0000 7ff3a3fff700 1 -- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 msgr2=0x7ff39006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:25.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.389+0000 7ff3a3fff700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 0x7ff39006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.439+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3a40c69e0 con 0x7ff3a40cd470 2023-10-16T11:00:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.440+0000 7ff3a0ff9700 1 -- 172.21.15.181:0/4011608977 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4927 (secure 0 0 0) 0x7ff3a40c69e0 con 0x7ff3a40cd470 2023-10-16T11:00:25.442 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:25.442 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1229456263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1229456263},{"type":"v1","addr":"172.21.15.190:6834","nonce":1229456263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3176781358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3176781358},{"type":"v1","addr":"172.21.15.181:6836","nonce":3176781358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2996369280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2996369280},{"type":"v1","addr":"172.21.15.181:6837","nonce":2996369280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:01.954427+0000","modified":"2023-10-16T10:04:04.993634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8146":{"gid":8146,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3854101486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3854101486},{"type":"v1","addr":"172.21.15.190:6835","nonce":3854101486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:00:25.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 msgr2=0x7ff39006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:25.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 0x7ff39006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40cd470 msgr2=0x7ff3a40c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40cd470 0x7ff3a40c4600 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c530 tx=0x7ff39800c840 comp rx=0 tx=0).stop 2023-10-16T11:00:25.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 shutdown_connections 2023-10-16T11:00:25.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7ff39006b020 0x7ff39006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a415dfd0 0x7ff3a40ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a40af450 0x7ff3a40c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 --2- 172.21.15.181:0/4011608977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3a40cd470 0x7ff3a40c4600 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.442+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 >> 172.21.15.181:0/4011608977 conn(0x7ff3a40d04a0 msgr2=0x7ff3a406d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:25.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.443+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 shutdown_connections 2023-10-16T11:00:25.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.443+0000 7ff3aa883700 1 -- 172.21.15.181:0/4011608977 wait complete. 2023-10-16T11:00:25.447 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 76 2023-10-16T11:00:25.457 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:25.496 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:25.495+0000 7f52d6f76200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:00:25.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.579+0000 7f88b9703700 1 Processor -- start 2023-10-16T11:00:25.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.579+0000 7f88b9703700 1 -- start start 2023-10-16T11:00:25.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b9703700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b415b990 0x7f88b40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b9703700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88b40d0370 0x7f88b40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b9703700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b40cc510 con 0x7f88b40cd2d0 2023-10-16T11:00:25.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b9703700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b40d24f0 con 0x7f88b40d0370 2023-10-16T11:00:25.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b9703700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b40cc7f0 con 0x7f88b415b990 2023-10-16T11:00:25.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b27fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f88b40cd2d0 0x7f88b40cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33034/0 (socket says 172.21.15.181:33034) 2023-10-16T11:00:25.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b27fc700 1 -- 172.21.15.181:0/2883661713 learned_addr learned my addr 172.21.15.181:0/2883661713 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:25.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88b40d0370 0x7f88b40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b2ffd700 1 --2- 172.21.15.181:0/2883661713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b415b990 0x7f88b40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b1ffb700 1 -- 172.21.15.181:0/2883661713 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783879273 0 0) 0x7f88b40cc510 con 0x7f88b40cd2d0 2023-10-16T11:00:25.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.580+0000 7f88b1ffb700 1 -- 172.21.15.181:0/2883661713 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f889c0035b0 con 0x7f88b40cd2d0 2023-10-16T11:00:25.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b37fe700 1 -- 172.21.15.181:0/2883661713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b415b990 msgr2=0x7f88b40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:25.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b37fe700 1 --2- 172.21.15.181:0/2883661713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b415b990 0x7f88b40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b37fe700 1 -- 172.21.15.181:0/2883661713 >> v1:172.21.15.181:6789/0 conn(0x7f88b40cd2d0 legacy=0x7f88b40cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b37fe700 1 -- 172.21.15.181:0/2883661713 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b40ccad0 con 0x7f88b40d0370 2023-10-16T11:00:25.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b37fe700 1 --2- 172.21.15.181:0/2883661713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b40d0370 0x7f88b40d1fb0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009d90 tx=0x7f88ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a0ffb4b165a792 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:25.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b1ffb700 1 -- 172.21.15.181:0/2883661713 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00e070 con 0x7f88b40d0370 2023-10-16T11:00:25.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b1ffb700 1 -- 172.21.15.181:0/2883661713 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac00bb20 con 0x7f88b40d0370 2023-10-16T11:00:25.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b1ffb700 1 -- 172.21.15.181:0/2883661713 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac012430 con 0x7f88b40d0370 2023-10-16T11:00:25.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b9703700 1 -- 172.21.15.181:0/2883661713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b40d0370 msgr2=0x7f88b40d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b9703700 1 --2- 172.21.15.181:0/2883661713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b40d0370 0x7f88b40d1fb0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009d90 tx=0x7f88ac00b330 comp rx=0 tx=0).stop 2023-10-16T11:00:25.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b9703700 1 -- 172.21.15.181:0/2883661713 shutdown_connections 2023-10-16T11:00:25.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b9703700 1 --2- 172.21.15.181:0/2883661713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b40d0370 0x7f88b40d1fb0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b9703700 1 --2- 172.21.15.181:0/2883661713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b415b990 0x7f88b40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b9703700 1 -- 172.21.15.181:0/2883661713 >> 172.21.15.181:0/2883661713 conn(0x7f88b409a9a0 msgr2=0x7f88b40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:25.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.581+0000 7f88b9703700 1 -- 172.21.15.181:0/2883661713 shutdown_connections 2023-10-16T11:00:25.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.582+0000 7f88b9703700 1 -- 172.21.15.181:0/2883661713 wait complete. 2023-10-16T11:00:25.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.583+0000 7f88b9703700 1 Processor -- start 2023-10-16T11:00:25.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.583+0000 7f88b9703700 1 -- start start 2023-10-16T11:00:25.590 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:25.583+0000 7f76c1d77200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:00:25.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b9703700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b40cd2d0 0x7f88b40af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b9703700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b40d0370 0x7f88b40af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b9703700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b415b990 0x7f88b40b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b9703700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b415ef90 con 0x7f88b415b990 2023-10-16T11:00:25.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b9703700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b415def0 con 0x7f88b40d0370 2023-10-16T11:00:25.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b9703700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b415e1d0 con 0x7f88b40cd2d0 2023-10-16T11:00:25.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b415b990 0x7f88b40b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b415b990 0x7f88b40b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50196/0 (socket says 172.21.15.181:50196) 2023-10-16T11:00:25.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b37fe700 1 -- 172.21.15.181:0/3143076792 learned_addr learned my addr 172.21.15.181:0/3143076792 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:25.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b2ffd700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b40cd2d0 0x7f88b40af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.584+0000 7f88b27fc700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b40d0370 0x7f88b40af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b37fe700 1 -- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b40cd2d0 msgr2=0x7f88b40af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b37fe700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b40cd2d0 0x7f88b40af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b37fe700 1 -- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b40d0370 msgr2=0x7f88b40af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b37fe700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b40d0370 0x7f88b40af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b37fe700 1 -- 172.21.15.181:0/3143076792 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac009a70 con 0x7f88b415b990 2023-10-16T11:00:25.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b2ffd700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b40cd2d0 0x7f88b40af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:25.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b37fe700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b415b990 0x7f88b40b0b30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f88ac014040 tx=0x7f88ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f889b7fe700 1 -- 172.21.15.181:0/3143076792 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0129f0 con 0x7f88b415b990 2023-10-16T11:00:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f889b7fe700 1 -- 172.21.15.181:0/3143076792 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac020070 con 0x7f88b415b990 2023-10-16T11:00:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b415e4b0 con 0x7f88b415b990 2023-10-16T11:00:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f889b7fe700 1 -- 172.21.15.181:0/3143076792 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac012db0 con 0x7f88b415b990 2023-10-16T11:00:25.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.585+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b415ea70 con 0x7f88b415b990 2023-10-16T11:00:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.586+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b4067d20 con 0x7f88b415b990 2023-10-16T11:00:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.587+0000 7f889b7fe700 1 -- 172.21.15.181:0/3143076792 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f88ac004040 con 0x7f88b415b990 2023-10-16T11:00:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.588+0000 7f889b7fe700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 0x7f88a006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.588+0000 7f88b2ffd700 1 -- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 msgr2=0x7f88a006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:25.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.588+0000 7f889b7fe700 1 -- 172.21.15.181:0/3143076792 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f88ac022070 con 0x7f88b415b990 2023-10-16T11:00:25.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.588+0000 7f88b2ffd700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 0x7f88a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:25.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.591+0000 7f889b7fe700 1 -- 172.21.15.181:0/3143076792 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88ac056930 con 0x7f88b415b990 2023-10-16T11:00:25.756 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:25.754+0000 7f76c1d77200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:00:25.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.789+0000 7f88b2ffd700 1 -- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 msgr2=0x7f88a006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:25.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.789+0000 7f88b2ffd700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 0x7f88a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.793+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b40d1d90 con 0x7f88b415b990 2023-10-16T11:00:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.794+0000 7f889b7fe700 1 -- 172.21.15.181:0/3143076792 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+18019 (secure 0 0 0) 0x7f88ac02c020 con 0x7f88b415b990 2023-10-16T11:00:25.796 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:25.796 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":155,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:21.787794+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:00.393030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:00.916899+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":155,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.13","mappings":[{"from":2,"to":5}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1561933478":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1604670603":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6833/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6832/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1093898558":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/40763561":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/2364682800":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1727687969":"2023-10-17T10:04:15.822177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":50,"length":7},{"begin":106,"length":7}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:25.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 msgr2=0x7f88a006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:25.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 0x7f88a006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b415b990 msgr2=0x7f88b40b0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b415b990 0x7f88b40b0b30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f88ac014040 tx=0x7f88ac004ad0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 shutdown_connections 2023-10-16T11:00:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f88a006b020 0x7f88a006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88b40cd2d0 0x7f88b40af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b415b990 0x7f88b40b0b30 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 --2- 172.21.15.181:0/3143076792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b40d0370 0x7f88b40af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.796+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 >> 172.21.15.181:0/3143076792 conn(0x7f88b409a9a0 msgr2=0x7f88b40ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.797+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 shutdown_connections 2023-10-16T11:00:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.797+0000 7f88b9703700 1 -- 172.21.15.181:0/3143076792 wait complete. 2023-10-16T11:00:25.811 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:00:25.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:00:25.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.932+0000 7f023310f700 1 Processor -- start 2023-10-16T11:00:25.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f023310f700 1 -- start start 2023-10-16T11:00:25.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f023310f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f022c15b990 0x7f022c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f023310f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f022c0cd2d0 0x7f022c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f023310f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f023310f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c0d0630 con 0x7f022c0cd2d0 2023-10-16T11:00:25.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f023310f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c0d24f0 con 0x7f022c15b990 2023-10-16T11:00:25.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f023310f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0cc510 con 0x7f022c0cfc60 2023-10-16T11:00:25.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.933+0000 7f022bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f022c0cd2d0 0x7f022c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f022c0cd2d0 0x7f022c0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50204/0 (socket says 172.21.15.181:50204) 2023-10-16T11:00:25.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 -- 172.21.15.181:0/1097622530 learned_addr learned my addr 172.21.15.181:0/1097622530 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:25.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f02316ac700 1 --2- 172.21.15.181:0/1097622530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f0230eab700 1 --2- 172.21.15.181:0/1097622530 >> v2:172.21.15.190:3300/0 conn(0x7f022c15b990 0x7f022c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 -- 172.21.15.181:0/1097622530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 msgr2=0x7f022c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:25.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 --2- 172.21.15.181:0/1097622530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 -- 172.21.15.181:0/1097622530 >> v2:172.21.15.190:3300/0 conn(0x7f022c15b990 msgr2=0x7f022c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:25.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 --2- 172.21.15.181:0/1097622530 >> v2:172.21.15.190:3300/0 conn(0x7f022c15b990 0x7f022c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 -- 172.21.15.181:0/1097622530 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0cc7f0 con 0x7f022c0cd2d0 2023-10-16T11:00:25.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022bfff700 1 --2- 172.21.15.181:0/1097622530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c0d1fb0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f022c073000 tx=0x7f0218009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13edfecf8c302c93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:25.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022b7fe700 1 -- 172.21.15.181:0/1097622530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218011830 con 0x7f022c0cd2d0 2023-10-16T11:00:25.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022b7fe700 1 -- 172.21.15.181:0/1097622530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0218011990 con 0x7f022c0cd2d0 2023-10-16T11:00:25.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.934+0000 7f022b7fe700 1 -- 172.21.15.181:0/1097622530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218011d20 con 0x7f022c0cd2d0 2023-10-16T11:00:25.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 -- 172.21.15.181:0/1097622530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 msgr2=0x7f022c0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 --2- 172.21.15.181:0/1097622530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c0d1fb0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f022c073000 tx=0x7f0218009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 -- 172.21.15.181:0/1097622530 shutdown_connections 2023-10-16T11:00:25.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 --2- 172.21.15.181:0/1097622530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 --2- 172.21.15.181:0/1097622530 >> v2:172.21.15.190:3300/0 conn(0x7f022c15b990 0x7f022c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 --2- 172.21.15.181:0/1097622530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c0d1fb0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 -- 172.21.15.181:0/1097622530 >> 172.21.15.181:0/1097622530 conn(0x7f022c09a9a0 msgr2=0x7f022c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:25.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 -- 172.21.15.181:0/1097622530 shutdown_connections 2023-10-16T11:00:25.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.935+0000 7f023310f700 1 -- 172.21.15.181:0/1097622530 wait complete. 2023-10-16T11:00:25.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.936+0000 7f023310f700 1 Processor -- start 2023-10-16T11:00:25.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.936+0000 7f023310f700 1 -- start start 2023-10-16T11:00:25.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f023310f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f023310f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f023310f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022c15b990 0x7f022c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f023310f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b51f0 con 0x7f022c0cd2d0 2023-10-16T11:00:25.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f023310f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b54d0 con 0x7f022c15b990 2023-10-16T11:00:25.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f023310f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b57b0 con 0x7f022c0cfc60 2023-10-16T11:00:25.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c14d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50208/0 (socket says 172.21.15.181:50208) 2023-10-16T11:00:25.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 -- 172.21.15.181:0/1923169523 learned_addr learned my addr 172.21.15.181:0/1923169523 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:25.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f022bfff700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f02316ac700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022c15b990 0x7f022c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:25.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 -- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 msgr2=0x7f022c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 -- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022c15b990 msgr2=0x7f022c0cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:25.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022c15b990 0x7f022c0cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:25.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 -- 172.21.15.181:0/1923169523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218009550 con 0x7f022c0cd2d0 2023-10-16T11:00:25.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.937+0000 7f0230eab700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c14d990 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f022000e900 tx=0x7f022000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:25.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.938+0000 7f02297fa700 1 -- 172.21.15.181:0/1923169523 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220013680 con 0x7f022c0cd2d0 2023-10-16T11:00:25.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.938+0000 7f02297fa700 1 -- 172.21.15.181:0/1923169523 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02200137e0 con 0x7f022c0cd2d0 2023-10-16T11:00:25.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.938+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c0b5a90 con 0x7f022c0cd2d0 2023-10-16T11:00:25.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.938+0000 7f02297fa700 1 -- 172.21.15.181:0/1923169523 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220013b50 con 0x7f022c0cd2d0 2023-10-16T11:00:25.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.938+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c14ded0 con 0x7f022c0cd2d0 2023-10-16T11:00:25.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.939+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c14e060 con 0x7f022c0cd2d0 2023-10-16T11:00:25.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.940+0000 7f02297fa700 1 -- 172.21.15.181:0/1923169523 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 95) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f0220013cb0 con 0x7f022c0cd2d0 2023-10-16T11:00:25.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.940+0000 7f02297fa700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 0x7f021006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:25.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.940+0000 7f022bfff700 1 -- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 msgr2=0x7f021006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:25.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.940+0000 7f02297fa700 1 -- 172.21.15.181:0/1923169523 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f0220089c80 con 0x7f022c0cd2d0 2023-10-16T11:00:25.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.940+0000 7f022bfff700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 0x7f021006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:25.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:25.944+0000 7f02297fa700 1 -- 172.21.15.181:0/1923169523 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f022c14e060 con 0x7f022c0cd2d0 2023-10-16T11:00:26.026 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:26.025+0000 7f52d6f76200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:00:26.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.141+0000 7f022bfff700 1 -- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 msgr2=0x7f021006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1123221928 2023-10-16T11:00:26.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.141+0000 7f022bfff700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 0x7f021006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.190+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f022c14e060 con 0x7f022c0cd2d0 2023-10-16T11:00:26.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.190+0000 7f02297fa700 1 -- 172.21.15.181:0/1923169523 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f022c14e060 con 0x7f022c0cd2d0 2023-10-16T11:00:26.192 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:26.192 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1229456263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1229456263},{"type":"v1","addr":"172.21.15.190:6834","nonce":1229456263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3176781358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3176781358},{"type":"v1","addr":"172.21.15.181:6836","nonce":3176781358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2996369280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2996369280},{"type":"v1","addr":"172.21.15.181:6837","nonce":2996369280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:01.954427+0000","modified":"2023-10-16T10:04:04.993634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8146":{"gid":8146,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3854101486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3854101486},{"type":"v1","addr":"172.21.15.190:6835","nonce":3854101486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:00:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.192+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 msgr2=0x7f021006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.192+0000 7f023310f700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 0x7f021006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.192+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 msgr2=0x7f022c14d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.192+0000 7f023310f700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c14d990 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f022000e900 tx=0x7f022000ec10 comp rx=0 tx=0).stop 2023-10-16T11:00:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 shutdown_connections 2023-10-16T11:00:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:6832/1123221928,v1:172.21.15.181:6833/1123221928] conn(0x7f021006b020 0x7f021006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022c15b990 0x7f022c0cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f022c0cd2d0 0x7f022c14d990 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 --2- 172.21.15.181:0/1923169523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f022c0cfc60 0x7f022c0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 >> 172.21.15.181:0/1923169523 conn(0x7f022c09a9a0 msgr2=0x7f022c0d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 shutdown_connections 2023-10-16T11:00:26.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.193+0000 7f023310f700 1 -- 172.21.15.181:0/1923169523 wait complete. 2023-10-16T11:00:26.197 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 76 2023-10-16T11:00:26.207 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:00:26.319 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.317+0000 7f76c1d77200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:00:26.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.329+0000 7f9f9abc5700 1 Processor -- start 2023-10-16T11:00:26.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.329+0000 7f9f9abc5700 1 -- start start 2023-10-16T11:00:26.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f9abc5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f9406d5f0 0x7f9f940d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f9abc5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f940d2560 con 0x7f9f94161b60 2023-10-16T11:00:26.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f9abc5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9406ce20 con 0x7f9f94160d20 2023-10-16T11:00:26.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f9abc5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f94162800 con 0x7f9f9406d5f0 2023-10-16T11:00:26.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f98961700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9f94161b60 0x7f9f94161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33064/0 (socket says 172.21.15.181:33064) 2023-10-16T11:00:26.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f98961700 1 -- 172.21.15.181:0/3708920411 learned_addr learned my addr 172.21.15.181:0/3708920411 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f93fff700 1 --2- 172.21.15.181:0/3708920411 >> v2:172.21.15.190:3300/0 conn(0x7f9f9406d5f0 0x7f9f940d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f937fe700 1 -- 172.21.15.181:0/3708920411 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961653205 0 0) 0x7f9f940d2560 con 0x7f9f94161b60 2023-10-16T11:00:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f937fe700 1 -- 172.21.15.181:0/3708920411 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f7c0035b0 con 0x7f9f94161b60 2023-10-16T11:00:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f937fe700 1 -- 172.21.15.181:0/3708920411 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882072363 0 0) 0x7f9f9406ce20 con 0x7f9f94160d20 2023-10-16T11:00:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f937fe700 1 -- 172.21.15.181:0/3708920411 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f940d2560 con 0x7f9f94160d20 2023-10-16T11:00:26.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f93fff700 1 -- 172.21.15.181:0/3708920411 >> v1:172.21.15.190:6789/0 conn(0x7f9f94160d20 legacy=0x7f9f940b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f93fff700 1 -- 172.21.15.181:0/3708920411 >> v1:172.21.15.181:6789/0 conn(0x7f9f94161b60 legacy=0x7f9f94161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.330+0000 7f9f93fff700 1 -- 172.21.15.181:0/3708920411 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f940d45f0 con 0x7f9f9406d5f0 2023-10-16T11:00:26.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.331+0000 7f9f93fff700 1 --2- 172.21.15.181:0/3708920411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9406d5f0 0x7f9f940d2020 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009840 tx=0x7f9f88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87267655bfae4b63 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:26.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.331+0000 7f9f937fe700 1 -- 172.21.15.181:0/3708920411 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88011830 con 0x7f9f9406d5f0 2023-10-16T11:00:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.331+0000 7f9f937fe700 1 -- 172.21.15.181:0/3708920411 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f88011990 con 0x7f9f9406d5f0 2023-10-16T11:00:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.331+0000 7f9f937fe700 1 -- 172.21.15.181:0/3708920411 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88011d20 con 0x7f9f9406d5f0 2023-10-16T11:00:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.331+0000 7f9f9abc5700 1 -- 172.21.15.181:0/3708920411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9406d5f0 msgr2=0x7f9f940d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.331+0000 7f9f9abc5700 1 --2- 172.21.15.181:0/3708920411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9406d5f0 0x7f9f940d2020 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009840 tx=0x7f9f88009b50 comp rx=0 tx=0).stop 2023-10-16T11:00:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.332+0000 7f9f9abc5700 1 -- 172.21.15.181:0/3708920411 shutdown_connections 2023-10-16T11:00:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.332+0000 7f9f9abc5700 1 --2- 172.21.15.181:0/3708920411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f9406d5f0 0x7f9f940d2020 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.332+0000 7f9f9abc5700 1 -- 172.21.15.181:0/3708920411 >> 172.21.15.181:0/3708920411 conn(0x7f9f9409cba0 msgr2=0x7f9f940d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:26.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.332+0000 7f9f9abc5700 1 -- 172.21.15.181:0/3708920411 shutdown_connections 2023-10-16T11:00:26.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.332+0000 7f9f9abc5700 1 -- 172.21.15.181:0/3708920411 wait complete. 2023-10-16T11:00:26.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.333+0000 7f9f9abc5700 1 Processor -- start 2023-10-16T11:00:26.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.333+0000 7f9f9abc5700 1 -- start start 2023-10-16T11:00:26.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f9abc5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9406d5f0 0x7f9f940cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f9abc5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f94160d20 0x7f9f940b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f9abc5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f94161b60 0x7f9f940b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f9abc5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b6df0 con 0x7f9f94160d20 2023-10-16T11:00:26.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f9abc5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9414f500 con 0x7f9f94161b60 2023-10-16T11:00:26.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f9abc5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941f7460 con 0x7f9f9406d5f0 2023-10-16T11:00:26.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f94160d20 0x7f9f940b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f94160d20 0x7f9f940b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50228/0 (socket says 172.21.15.181:50228) 2023-10-16T11:00:26.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f93fff700 1 -- 172.21.15.181:0/40820003 learned_addr learned my addr 172.21.15.181:0/40820003 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:26.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f98961700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9406d5f0 0x7f9f940cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.334+0000 7f9f99162700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f94161b60 0x7f9f940b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.335+0000 7f9f93fff700 1 -- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9406d5f0 msgr2=0x7f9f940cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.335+0000 7f9f93fff700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9406d5f0 0x7f9f940cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.335+0000 7f9f93fff700 1 -- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f94161b60 msgr2=0x7f9f940b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.335+0000 7f9f93fff700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f94161b60 0x7f9f940b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.335+0000 7f9f93fff700 1 -- 172.21.15.181:0/40820003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880094a0 con 0x7f9f94160d20 2023-10-16T11:00:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.335+0000 7f9f93fff700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f94160d20 0x7f9f940b4450 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9f88014040 tx=0x7f9f88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:26.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.341+0000 7f9f917fa700 1 -- 172.21.15.181:0/40820003 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8801a6e0 con 0x7f9f94160d20 2023-10-16T11:00:26.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.342+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f940b64c0 con 0x7f9f94160d20 2023-10-16T11:00:26.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.342+0000 7f9f917fa700 1 -- 172.21.15.181:0/40820003 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8801a840 con 0x7f9f94160d20 2023-10-16T11:00:26.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.342+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f940b6a80 con 0x7f9f94160d20 2023-10-16T11:00:26.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.342+0000 7f9f917fa700 1 -- 172.21.15.181:0/40820003 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8801abb0 con 0x7f9f94160d20 2023-10-16T11:00:26.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.343+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f740052f0 con 0x7f9f94160d20 2023-10-16T11:00:26.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.343+0000 7f9f917fa700 1 -- 172.21.15.181:0/40820003 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 96) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f9f88022070 con 0x7f9f94160d20 2023-10-16T11:00:26.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.344+0000 7f9f917fa700 1 -- 172.21.15.181:0/40820003 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6951+0+0 (secure 0 0 0) 0x7f9f8808a440 con 0x7f9f94160d20 2023-10-16T11:00:26.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.351+0000 7f9f917fa700 1 -- 172.21.15.181:0/40820003 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9f88055630 con 0x7f9f94160d20 2023-10-16T11:00:26.411 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.409+0000 7f763d5a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.411 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.409+0000 7f763d5a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.412 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.409+0000 7f763d5a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.412 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.409+0000 7f763d5a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.412 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.409+0000 7f763d5a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.412 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.410+0000 7f763a55d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.412 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.410+0000 7f763a55d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.412 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.410+0000 7f763a55d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.413 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.410+0000 7f763a55d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.413 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:26.410+0000 7f763a55d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:00:26.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.602+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f74005b20 con 0x7f9f94160d20 2023-10-16T11:00:26.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.602+0000 7f9f917fa700 1 -- 172.21.15.181:0/40820003 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f9f88015090 con 0x7f9f94160d20 2023-10-16T11:00:26.604 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:26.604 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1229456263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1229456263},{"type":"v1","addr":"172.21.15.190:6834","nonce":1229456263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3176781358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3176781358},{"type":"v1","addr":"172.21.15.181:6836","nonce":3176781358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2996369280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2996369280},{"type":"v1","addr":"172.21.15.181:6837","nonce":2996369280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T10:04:01.954427+0000","modified":"2023-10-16T10:04:04.993634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8146":{"gid":8146,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3854101486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3854101486},{"type":"v1","addr":"172.21.15.190:6835","nonce":3854101486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-16T11:00:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f94160d20 msgr2=0x7f9f940b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f94160d20 0x7f9f940b4450 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9f88014040 tx=0x7f9f88004750 comp rx=0 tx=0).stop 2023-10-16T11:00:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 shutdown_connections 2023-10-16T11:00:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f9406d5f0 0x7f9f940cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f94161b60 0x7f9f940b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 --2- 172.21.15.181:0/40820003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f94160d20 0x7f9f940b4450 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 >> 172.21.15.181:0/40820003 conn(0x7f9f9409cba0 msgr2=0x7f9f940d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 shutdown_connections 2023-10-16T11:00:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.604+0000 7f9f9abc5700 1 -- 172.21.15.181:0/40820003 wait complete. 2023-10-16T11:00:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 76 2023-10-16T11:00:26.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:26.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.738+0000 7f810e978700 1 Processor -- start 2023-10-16T11:00:26.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.738+0000 7f810e978700 1 -- start start 2023-10-16T11:00:26.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810e978700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080cdb80 0x7f81080af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810e978700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810806f870 con 0x7f81080af450 2023-10-16T11:00:26.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810e978700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81080d1160 con 0x7f810815dfd0 2023-10-16T11:00:26.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810e978700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81080cc8e0 con 0x7f81080cdb80 2023-10-16T11:00:26.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f80fffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f81080af450 0x7f81080cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33148/0 (socket says 172.21.15.181:33148) 2023-10-16T11:00:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f80fffff700 1 -- 172.21.15.181:0/3927563890 learned_addr learned my addr 172.21.15.181:0/3927563890 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810cf15700 1 --2- 172.21.15.181:0/3927563890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080cdb80 0x7f81080af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317292924 0 0) 0x7f810806f870 con 0x7f81080af450 2023-10-16T11:00:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f40035b0 con 0x7f81080af450 2023-10-16T11:00:26.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376003576 0 0) 0x7f81080d1160 con 0x7f810815dfd0 2023-10-16T11:00:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f810806f870 con 0x7f810815dfd0 2023-10-16T11:00:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757102363 0 0) 0x7f80f40035b0 con 0x7f81080af450 2023-10-16T11:00:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810cf15700 1 -- 172.21.15.181:0/3927563890 >> v1:172.21.15.190:6789/0 conn(0x7f810815dfd0 legacy=0x7f810806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810cf15700 1 -- 172.21.15.181:0/3927563890 >> v1:172.21.15.181:6789/0 conn(0x7f81080af450 legacy=0x7f81080cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810cf15700 1 -- 172.21.15.181:0/3927563890 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81080ccbc0 con 0x7f81080cdb80 2023-10-16T11:00:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.739+0000 7f810cf15700 1 --2- 172.21.15.181:0/3927563890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080cdb80 0x7f81080af9a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8100009dc0 tx=0x7f810000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4a6f66188770eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810000e070 con 0x7f81080cdb80 2023-10-16T11:00:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8100003a20 con 0x7f81080cdb80 2023-10-16T11:00:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f81077fe700 1 -- 172.21.15.181:0/3927563890 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0001670 con 0x7f81080cdb80 2023-10-16T11:00:26.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f810e978700 1 -- 172.21.15.181:0/3927563890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080cdb80 msgr2=0x7f81080af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f810e978700 1 --2- 172.21.15.181:0/3927563890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080cdb80 0x7f81080af9a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8100009dc0 tx=0x7f810000b330 comp rx=0 tx=0).stop 2023-10-16T11:00:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f810e978700 1 -- 172.21.15.181:0/3927563890 shutdown_connections 2023-10-16T11:00:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f810e978700 1 --2- 172.21.15.181:0/3927563890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080cdb80 0x7f81080af9a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f810e978700 1 -- 172.21.15.181:0/3927563890 >> 172.21.15.181:0/3927563890 conn(0x7f81080d04a0 msgr2=0x7f81080d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:26.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f810e978700 1 -- 172.21.15.181:0/3927563890 shutdown_connections 2023-10-16T11:00:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.740+0000 7f810e978700 1 -- 172.21.15.181:0/3927563890 wait complete. 2023-10-16T11:00:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.741+0000 7f810e978700 1 Processor -- start 2023-10-16T11:00:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.741+0000 7f810e978700 1 -- start start 2023-10-16T11:00:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f810e978700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080af450 0x7f81080b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f810e978700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81080cdb80 0x7f81080b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f810e978700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f810815dfd0 0x7f8108146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:26.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f810e978700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8108145520 con 0x7f81080cdb80 2023-10-16T11:00:26.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f810e978700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81080c5d30 con 0x7f810815dfd0 2023-10-16T11:00:26.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f810e978700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8108144c50 con 0x7f81080af450 2023-10-16T11:00:26.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f80fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81080cdb80 0x7f81080b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f80fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81080cdb80 0x7f81080b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50308/0 (socket says 172.21.15.181:50308) 2023-10-16T11:00:26.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f80fffff700 1 -- 172.21.15.181:0/3221623761 learned_addr learned my addr 172.21.15.181:0/3221623761 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:26.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f810cf15700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f810815dfd0 0x7f8108146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.742+0000 7f8107fff700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080af450 0x7f81080b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:26.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f80fffff700 1 -- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080af450 msgr2=0x7f81080b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f80fffff700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080af450 0x7f81080b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f80fffff700 1 -- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f810815dfd0 msgr2=0x7f8108146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f80fffff700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f810815dfd0 0x7f8108146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f80fffff700 1 -- 172.21.15.181:0/3221623761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8100009a70 con 0x7f81080cdb80 2023-10-16T11:00:26.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f80fffff700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81080cdb80 0x7f81080b2ad0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f80f000c750 tx=0x7f80f000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f81057fa700 1 -- 172.21.15.181:0/3221623761 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0004c80 con 0x7f81080cdb80 2023-10-16T11:00:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f81057fa700 1 -- 172.21.15.181:0/3221623761 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f0004e00 con 0x7f81080cdb80 2023-10-16T11:00:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f81057fa700 1 -- 172.21.15.181:0/3221623761 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0007310 con 0x7f81080cdb80 2023-10-16T11:00:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81080aff00 con 0x7f81080cdb80 2023-10-16T11:00:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.743+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8108144e60 con 0x7f81080cdb80 2023-10-16T11:00:26.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.744+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8108146ac0 con 0x7f81080cdb80 2023-10-16T11:00:26.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.744+0000 7f81057fa700 1 -- 172.21.15.181:0/3221623761 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 96) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f80f0007470 con 0x7f81080cdb80 2023-10-16T11:00:26.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.745+0000 7f81057fa700 1 -- 172.21.15.181:0/3221623761 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6951+0+0 (secure 0 0 0) 0x7f80f0089270 con 0x7f81080cdb80 2023-10-16T11:00:26.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.749+0000 7f81057fa700 1 -- 172.21.15.181:0/3221623761 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f80f0054290 con 0x7f81080cdb80 2023-10-16T11:00:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.945+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8108067d20 con 0x7f81080cdb80 2023-10-16T11:00:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.946+0000 7f81057fa700 1 -- 172.21.15.181:0/3221623761 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+18458 (secure 0 0 0) 0x7f80f00578b0 con 0x7f81080cdb80 2023-10-16T11:00:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:26.948 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":156,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:26.333140+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-16T10:04:00.393030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-16T10:04:00.916899+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":155,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.13","mappings":[{"from":2,"to":5}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/292833217":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/3382478479":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/4069049155":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/4130286653":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/3490595853":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/1561933478":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1604670603":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6833/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6833/1123221928":"2023-10-17T11:00:26.333089+0000","172.21.15.181:6832/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6832/1123221928":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/1093898558":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/2314316702":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/40763561":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/2364682800":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1727687969":"2023-10-17T10:04:15.822177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:26.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81080cdb80 msgr2=0x7f81080b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:26.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81080cdb80 0x7f81080b2ad0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f80f000c750 tx=0x7f80f000ca60 comp rx=0 tx=0).stop 2023-10-16T11:00:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 shutdown_connections 2023-10-16T11:00:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81080af450 0x7f81080b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f810815dfd0 0x7f8108146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 --2- 172.21.15.181:0/3221623761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81080cdb80 0x7f81080b2ad0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 >> 172.21.15.181:0/3221623761 conn(0x7f81080d04a0 msgr2=0x7f810815cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 shutdown_connections 2023-10-16T11:00:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:26.948+0000 7f810e978700 1 -- 172.21.15.181:0/3221623761 wait complete. 2023-10-16T11:00:26.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:00:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.082+0000 7fb45c7e3700 1 Processor -- start 2023-10-16T11:00:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.082+0000 7fb45c7e3700 1 -- start start 2023-10-16T11:00:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45c7e3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb45415b9f0 0x7fb4540cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45c7e3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45406b370 con 0x7fb4540cc400 2023-10-16T11:00:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45c7e3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45406aa40 con 0x7fb4540cfb50 2023-10-16T11:00:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45c7e3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45406b800 con 0x7fb45415b9f0 2023-10-16T11:00:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb459d7e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb4540cc400 0x7fb4540cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33160/0 (socket says 172.21.15.181:33160) 2023-10-16T11:00:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb459d7e700 1 -- 172.21.15.181:0/1461702465 learned_addr learned my addr 172.21.15.181:0/1461702465 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356780337 0 0) 0x7fb45406b370 con 0x7fb4540cc400 2023-10-16T11:00:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4400035b0 con 0x7fb4540cc400 2023-10-16T11:00:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45a57f700 1 --2- 172.21.15.181:0/1461702465 >> v2:172.21.15.190:3300/0 conn(0x7fb45415b9f0 0x7fb4540cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1446150611 0 0) 0x7fb4400035b0 con 0x7fb4540cc400 2023-10-16T11:00:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 >> v2:172.21.15.190:3300/0 conn(0x7fb45415b9f0 msgr2=0x7fb4540cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 --2- 172.21.15.181:0/1461702465 >> v2:172.21.15.190:3300/0 conn(0x7fb45415b9f0 0x7fb4540cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 >> v1:172.21.15.190:6789/0 conn(0x7fb4540cfb50 legacy=0x7fb4540ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45406ad20 con 0x7fb4540cc400 2023-10-16T11:00:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fb444001670 con 0x7fb4540cc400 2023-10-16T11:00:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45a57f700 1 --2- 172.21.15.181:0/1461702465 >> v2:172.21.15.190:3300/0 conn(0x7fb45415b9f0 0x7fb4540cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 >> v1:172.21.15.181:6789/0 conn(0x7fb4540cc400 legacy=0x7fb4540cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440003a00 0x7fb440003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb440004850 0x7fb440004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb440005330 0x7fb44000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45406b800 con 0x7fb440005330 2023-10-16T11:00:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44000c1e0 con 0x7fb440003a00 2023-10-16T11:00:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.083+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44000c3a0 con 0x7fb440004850 2023-10-16T11:00:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb440005330 0x7fb44000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45a57f700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440003a00 0x7fb440003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb459d7e700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb440004850 0x7fb440004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 -- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb440004850 msgr2=0x7fb440004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb440004850 0x7fb440004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 -- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440003a00 msgr2=0x7fb440003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440003a00 0x7fb440003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 -- 172.21.15.181:0/1461702465 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400035b0 con 0x7fb440005330 2023-10-16T11:00:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 -- 172.21.15.181:0/1461702465 reap_dead start 2023-10-16T11:00:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45ad80700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb440005330 0x7fb44000ba10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0049b0 tx=0x7fb440005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfc906388e66053 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c016070 con 0x7fb440005330 2023-10-16T11:00:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c00c9f0 con 0x7fb440005330 2023-10-16T11:00:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.084+0000 7fb45957d700 1 -- 172.21.15.181:0/1461702465 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004450 con 0x7fb440005330 2023-10-16T11:00:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.085+0000 7fb45c7e3700 1 -- 172.21.15.181:0/1461702465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb440005330 msgr2=0x7fb44000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.085+0000 7fb45c7e3700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb440005330 0x7fb44000ba10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0049b0 tx=0x7fb440005880 comp rx=0 tx=0).stop 2023-10-16T11:00:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.085+0000 7fb45c7e3700 1 -- 172.21.15.181:0/1461702465 shutdown_connections 2023-10-16T11:00:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.085+0000 7fb45c7e3700 1 --2- 172.21.15.181:0/1461702465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb440005330 0x7fb44000ba10 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.085+0000 7fb45c7e3700 1 -- 172.21.15.181:0/1461702465 >> 172.21.15.181:0/1461702465 conn(0x7fb4540d04a0 msgr2=0x7fb4540d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.085+0000 7fb45c7e3700 1 -- 172.21.15.181:0/1461702465 shutdown_connections 2023-10-16T11:00:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.085+0000 7fb45c7e3700 1 -- 172.21.15.181:0/1461702465 wait complete. 2023-10-16T11:00:27.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.086+0000 7fb45c7e3700 1 Processor -- start 2023-10-16T11:00:27.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.086+0000 7fb45c7e3700 1 -- start start 2023-10-16T11:00:27.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440005330 0x7fb4540b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4540c2bd0 0x7fb4540c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540c4480 0x7fb45406dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540c4900 con 0x7fb4540c4480 2023-10-16T11:00:27.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540b62e0 con 0x7fb440005330 2023-10-16T11:00:27.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb454158450 con 0x7fb4540c2bd0 2023-10-16T11:00:27.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540c4480 0x7fb45406dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540c4480 0x7fb45406dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50328/0 (socket says 172.21.15.181:50328) 2023-10-16T11:00:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 -- 172.21.15.181:0/721916064 learned_addr learned my addr 172.21.15.181:0/721916064 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 -- 172.21.15.181:0/721916064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4540c2bd0 msgr2=0x7fb4540c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4540c2bd0 0x7fb4540c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 -- 172.21.15.181:0/721916064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440005330 msgr2=0x7fb4540b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45a57f700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440005330 0x7fb4540b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440005330 0x7fb4540b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 -- 172.21.15.181:0/721916064 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb454158730 con 0x7fb4540c4480 2023-10-16T11:00:27.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45ad80700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540c4480 0x7fb45406dfe0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fb4540cb3c0 tx=0x7fb44c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:27.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb44affd700 1 -- 172.21.15.181:0/721916064 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c016040 con 0x7fb4540c4480 2023-10-16T11:00:27.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb44affd700 1 -- 172.21.15.181:0/721916064 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c0129f0 con 0x7fb4540c4480 2023-10-16T11:00:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45406d7e0 con 0x7fb4540c4480 2023-10-16T11:00:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb44affd700 1 -- 172.21.15.181:0/721916064 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c012d80 con 0x7fb4540c4480 2023-10-16T11:00:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.087+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4540c6bb0 con 0x7fb4540c4480 2023-10-16T11:00:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.088+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45406d640 con 0x7fb4540c4480 2023-10-16T11:00:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.089+0000 7fb44affd700 1 -- 172.21.15.181:0/721916064 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 96) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fb44c029460 con 0x7fb4540c4480 2023-10-16T11:00:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.089+0000 7fb44affd700 1 -- 172.21.15.181:0/721916064 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6951+0+0 (secure 0 0 0) 0x7fb44c08a920 con 0x7fb4540c4480 2023-10-16T11:00:27.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.093+0000 7fb44affd700 1 -- 172.21.15.181:0/721916064 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb45406d640 con 0x7fb4540c4480 2023-10-16T11:00:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.329+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb45406d640 con 0x7fb4540c4480 2023-10-16T11:00:27.331 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:00:27.330+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:00:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.331+0000 7fb44affd700 1 -- 172.21.15.181:0/721916064 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb45406d640 con 0x7fb4540c4480 2023-10-16T11:00:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.334+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540c4480 msgr2=0x7fb45406dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.334+0000 7fb45c7e3700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540c4480 0x7fb45406dfe0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fb4540cb3c0 tx=0x7fb44c003b00 comp rx=0 tx=0).stop 2023-10-16T11:00:27.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.334+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 shutdown_connections 2023-10-16T11:00:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.334+0000 7fb45c7e3700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4540c4480 0x7fb45406dfe0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.334+0000 7fb45c7e3700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb440005330 0x7fb4540b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.334+0000 7fb45c7e3700 1 --2- 172.21.15.181:0/721916064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4540c2bd0 0x7fb4540c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.334+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 >> 172.21.15.181:0/721916064 conn(0x7fb4540d04a0 msgr2=0x7fb45415a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:27.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.335+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 shutdown_connections 2023-10-16T11:00:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.335+0000 7fb45c7e3700 1 -- 172.21.15.181:0/721916064 wait complete. 2023-10-16T11:00:27.338 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:00:27.353 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:00:27.385 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:00:27.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.482+0000 7f5e807e3700 1 Processor -- start 2023-10-16T11:00:27.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.482+0000 7f5e807e3700 1 -- start start 2023-10-16T11:00:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e807e3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e7815f150 0x7f5e780cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e807e3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e780b2ae0 con 0x7f5e780ce4f0 2023-10-16T11:00:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e807e3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e780b2dc0 con 0x7f5e780cd170 2023-10-16T11:00:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e807e3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e780ce9a0 con 0x7f5e7815f150 2023-10-16T11:00:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e7dd7e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5e780ce4f0 0x7f5e780b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33198/0 (socket says 172.21.15.181:33198) 2023-10-16T11:00:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e7dd7e700 1 -- 172.21.15.181:0/38013602 learned_addr learned my addr 172.21.15.181:0/38013602 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:27.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e7e57f700 1 --2- 172.21.15.181:0/38013602 >> v2:172.21.15.190:3300/0 conn(0x7f5e7815f150 0x7f5e780cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183779566 0 0) 0x7f5e780b2ae0 con 0x7f5e780ce4f0 2023-10-16T11:00:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.483+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e640035b0 con 0x7f5e780ce4f0 2023-10-16T11:00:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103809983 0 0) 0x7f5e780b2dc0 con 0x7f5e780cd170 2023-10-16T11:00:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e780b2ae0 con 0x7f5e780cd170 2023-10-16T11:00:27.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7e57f700 1 -- 172.21.15.181:0/38013602 >> v1:172.21.15.190:6789/0 conn(0x7f5e780cd170 legacy=0x7f5e780b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7e57f700 1 -- 172.21.15.181:0/38013602 >> v1:172.21.15.181:6789/0 conn(0x7f5e780ce4f0 legacy=0x7f5e780b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3043061073 0 0) 0x7f5e640035b0 con 0x7f5e780ce4f0 2023-10-16T11:00:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7e57f700 1 -- 172.21.15.181:0/38013602 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e780cf9d0 con 0x7f5e7815f150 2023-10-16T11:00:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7e57f700 1 --2- 172.21.15.181:0/38013602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e780cdfb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009870 tx=0x7f5e68009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad09a713f59101 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74001670 con 0x7f5e7815f150 2023-10-16T11:00:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e68004030 con 0x7f5e7815f150 2023-10-16T11:00:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e7d57d700 1 -- 172.21.15.181:0/38013602 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e680043c0 con 0x7f5e7815f150 2023-10-16T11:00:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e807e3700 1 -- 172.21.15.181:0/38013602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 msgr2=0x7f5e780cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.484+0000 7f5e807e3700 1 --2- 172.21.15.181:0/38013602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e780cdfb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009870 tx=0x7f5e68009b80 comp rx=0 tx=0).stop 2023-10-16T11:00:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.485+0000 7f5e807e3700 1 -- 172.21.15.181:0/38013602 shutdown_connections 2023-10-16T11:00:27.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.485+0000 7f5e807e3700 1 --2- 172.21.15.181:0/38013602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e780cdfb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.485+0000 7f5e807e3700 1 -- 172.21.15.181:0/38013602 >> 172.21.15.181:0/38013602 conn(0x7f5e7809a930 msgr2=0x7f5e780d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:27.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.485+0000 7f5e807e3700 1 -- 172.21.15.181:0/38013602 shutdown_connections 2023-10-16T11:00:27.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.485+0000 7f5e807e3700 1 -- 172.21.15.181:0/38013602 wait complete. 2023-10-16T11:00:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 Processor -- start 2023-10-16T11:00:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 -- start start 2023-10-16T11:00:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780cd170 0x7f5e780caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce4f0 0x7f5e781578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e7806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7815c600 con 0x7f5e780cd170 2023-10-16T11:00:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780cb5d0 con 0x7f5e7815f150 2023-10-16T11:00:27.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.486+0000 7f5e807e3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7806c4f0 con 0x7f5e780ce4f0 2023-10-16T11:00:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780cd170 0x7f5e780caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780cd170 0x7f5e780caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50358/0 (socket says 172.21.15.181:50358) 2023-10-16T11:00:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 -- 172.21.15.181:0/400625659 learned_addr learned my addr 172.21.15.181:0/400625659 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7ed80700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e7806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7dd7e700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce4f0 0x7f5e781578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 -- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce4f0 msgr2=0x7f5e781578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce4f0 0x7f5e781578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 -- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 msgr2=0x7f5e7806d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e7806d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:27.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 -- 172.21.15.181:0/400625659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680094a0 con 0x7f5e780cd170 2023-10-16T11:00:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7dd7e700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce4f0 0x7f5e781578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7ed80700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e7806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e7e57f700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780cd170 0x7f5e780caed0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5e7806b3a0 tx=0x7f5e68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e6effd700 1 -- 172.21.15.181:0/400625659 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68011830 con 0x7f5e780cd170 2023-10-16T11:00:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e6effd700 1 -- 172.21.15.181:0/400625659 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6801e070 con 0x7f5e780cd170 2023-10-16T11:00:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e6effd700 1 -- 172.21.15.181:0/400625659 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68011bd0 con 0x7f5e780cd170 2023-10-16T11:00:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7806c7d0 con 0x7f5e780cd170 2023-10-16T11:00:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.487+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7806cd90 con 0x7f5e780cd170 2023-10-16T11:00:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.488+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7806cf80 con 0x7f5e780cd170 2023-10-16T11:00:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.489+0000 7f5e6effd700 1 -- 172.21.15.181:0/400625659 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 97) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5e68011d30 con 0x7f5e780cd170 2023-10-16T11:00:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.489+0000 7f5e6effd700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5e5c06ad50 0x7f5e5c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.489+0000 7f5e6effd700 1 -- 172.21.15.181:0/400625659 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 7081+0+0 (secure 0 0 0) 0x7f5e6808a7d0 con 0x7f5e780cd170 2023-10-16T11:00:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.489+0000 7f5e7dd7e700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5e5c06ad50 0x7f5e5c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.489+0000 7f5e7dd7e700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5e5c06ad50 0x7f5e5c06d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e781e63e0 tx=0x7f5e74009040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.493+0000 7f5e6effd700 1 -- 172.21.15.181:0/400625659 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5e68055660 con 0x7f5e780cd170 2023-10-16T11:00:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:27.730+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5e78067d20 con 0x7f5e780cd170 2023-10-16T11:00:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.355+0000 7f5e6effd700 1 -- 172.21.15.181:0/400625659 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5e68058c80 con 0x7f5e780cd170 2023-10-16T11:00:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5e5c06ad50 msgr2=0x7f5e5c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5e5c06ad50 0x7f5e5c06d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e781e63e0 tx=0x7f5e74009040 comp rx=0 tx=0).stop 2023-10-16T11:00:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780cd170 msgr2=0x7f5e780caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780cd170 0x7f5e780caed0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5e7806b3a0 tx=0x7f5e68004750 comp rx=0 tx=0).stop 2023-10-16T11:00:28.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 shutdown_connections 2023-10-16T11:00:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5e5c06ad50 0x7f5e5c06d210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7815f150 0x7f5e7806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e780cd170 0x7f5e780caed0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 --2- 172.21.15.181:0/400625659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e780ce4f0 0x7f5e781578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 >> 172.21.15.181:0/400625659 conn(0x7f5e7809a930 msgr2=0x7f5e781584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 shutdown_connections 2023-10-16T11:00:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.359+0000 7f5e807e3700 1 -- 172.21.15.181:0/400625659 wait complete. 2023-10-16T11:00:28.375 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:00:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.506+0000 7fd437d04700 1 Processor -- start 2023-10-16T11:00:28.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.507+0000 7fd437d04700 1 -- start start 2023-10-16T11:00:28.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.507+0000 7fd437d04700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300afa10 0x7fd4300ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:28.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.507+0000 7fd437d04700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4300cd5c0 0x7fd4300cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:28.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.507+0000 7fd437d04700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43015b540 con 0x7fd43006d870 2023-10-16T11:00:28.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.507+0000 7fd437d04700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43015b820 con 0x7fd4300cd5c0 2023-10-16T11:00:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.507+0000 7fd437d04700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43015bb00 con 0x7fd4300afa10 2023-10-16T11:00:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd4362a1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4300cd5c0 0x7fd4300cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd435aa0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300afa10 0x7fd4300ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd43529f700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd43006d870 0x7fd4300cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:45404/0 (socket says 172.21.15.181:45404) 2023-10-16T11:00:28.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd43529f700 1 -- 172.21.15.181:0/514356345 learned_addr learned my addr 172.21.15.181:0/514356345 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd4362a1700 1 -- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300afa10 msgr2=0x7fd4300ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd4362a1700 1 --2- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300afa10 0x7fd4300ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd4362a1700 1 -- 172.21.15.181:0/514356345 >> v1:172.21.15.190:6789/0 conn(0x7fd43006d870 legacy=0x7fd4300cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd4362a1700 1 -- 172.21.15.181:0/514356345 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43015bde0 con 0x7fd4300cd5c0 2023-10-16T11:00:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd435aa0700 1 --2- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300afa10 0x7fd4300ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:28.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd4362a1700 1 --2- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4300cd5c0 0x7fd4300cb8b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd428009d90 tx=0x7fd42800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427347e1b31c7671 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.508+0000 7fd434a9e700 1 -- 172.21.15.181:0/514356345 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800e070 con 0x7fd4300cd5c0 2023-10-16T11:00:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd434a9e700 1 -- 172.21.15.181:0/514356345 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42800bb20 con 0x7fd4300cd5c0 2023-10-16T11:00:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd434a9e700 1 -- 172.21.15.181:0/514356345 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428012430 con 0x7fd4300cd5c0 2023-10-16T11:00:28.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 -- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4300cd5c0 msgr2=0x7fd4300cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 --2- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4300cd5c0 0x7fd4300cb8b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd428009d90 tx=0x7fd42800b330 comp rx=0 tx=0).stop 2023-10-16T11:00:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 -- 172.21.15.181:0/514356345 shutdown_connections 2023-10-16T11:00:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 --2- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4300cd5c0 0x7fd4300cb8b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 --2- 172.21.15.181:0/514356345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300afa10 0x7fd4300ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 -- 172.21.15.181:0/514356345 >> 172.21.15.181:0/514356345 conn(0x7fd4300d04a0 msgr2=0x7fd4300d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 -- 172.21.15.181:0/514356345 shutdown_connections 2023-10-16T11:00:28.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.509+0000 7fd437d04700 1 -- 172.21.15.181:0/514356345 wait complete. 2023-10-16T11:00:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.510+0000 7fd437d04700 1 Processor -- start 2023-10-16T11:00:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.510+0000 7fd437d04700 1 -- start start 2023-10-16T11:00:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd437d04700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43006d870 0x7fd43015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd437d04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4300afa10 0x7fd4300c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd437d04700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300cd5c0 0x7fd43014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd437d04700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4300b3900 con 0x7fd4300afa10 2023-10-16T11:00:28.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd437d04700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43014ff00 con 0x7fd43006d870 2023-10-16T11:00:28.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd437d04700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43015d380 con 0x7fd4300cd5c0 2023-10-16T11:00:28.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd43529f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4300afa10 0x7fd4300c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:28.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd43529f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4300afa10 0x7fd4300c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50392/0 (socket says 172.21.15.181:50392) 2023-10-16T11:00:28.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd43529f700 1 -- 172.21.15.181:0/2166291438 learned_addr learned my addr 172.21.15.181:0/2166291438 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:28.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd435aa0700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43006d870 0x7fd43015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:28.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.511+0000 7fd4362a1700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300cd5c0 0x7fd43014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:28.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd43529f700 1 -- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300cd5c0 msgr2=0x7fd43014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:28.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd43529f700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300cd5c0 0x7fd43014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd43529f700 1 -- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43006d870 msgr2=0x7fd43015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:28.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd43529f700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43006d870 0x7fd43015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd43529f700 1 -- 172.21.15.181:0/2166291438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420009430 con 0x7fd4300afa10 2023-10-16T11:00:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd43529f700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4300afa10 0x7fd4300c45a0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00e940 tx=0x7fd42c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c005d70 con 0x7fd4300afa10 2023-10-16T11:00:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c00ee20 con 0x7fd4300afa10 2023-10-16T11:00:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0018b0 con 0x7fd4300afa10 2023-10-16T11:00:28.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd428009a70 con 0x7fd4300afa10 2023-10-16T11:00:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.512+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4300b3240 con 0x7fd4300afa10 2023-10-16T11:00:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.513+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4300ccd20 con 0x7fd4300afa10 2023-10-16T11:00:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.513+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 97) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd42c001a10 con 0x7fd4300afa10 2023-10-16T11:00:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.514+0000 7fd4267fc700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fd41406ad50 0x7fd41406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:28.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.514+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 7081+0+0 (secure 0 0 0) 0x7fd42c08b9c0 con 0x7fd4300afa10 2023-10-16T11:00:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.514+0000 7fd435aa0700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fd41406ad50 0x7fd41406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.514+0000 7fd435aa0700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fd41406ad50 0x7fd41406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd43015f4d0 tx=0x7fd420008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:28.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.518+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd42c056910 con 0x7fd4300afa10 2023-10-16T11:00:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.709+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4300b30a0 con 0x7fd4300afa10 2023-10-16T11:00:28.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:28.807+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 98) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd42c056ce0 con 0x7fd4300afa10 2023-10-16T11:00:29.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.359+0000 7fd4267fc700 1 -- 172.21.15.181:0/2166291438 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd42c0796c0 con 0x7fd4300afa10 2023-10-16T11:00:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fd41406ad50 msgr2=0x7fd41406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fd41406ad50 0x7fd41406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd43015f4d0 tx=0x7fd420008040 comp rx=0 tx=0).stop 2023-10-16T11:00:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4300afa10 msgr2=0x7fd4300c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4300afa10 0x7fd4300c45a0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00e940 tx=0x7fd42c00ed00 comp rx=0 tx=0).stop 2023-10-16T11:00:29.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 shutdown_connections 2023-10-16T11:00:29.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fd41406ad50 0x7fd41406d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4300cd5c0 0x7fd43014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd43006d870 0x7fd43015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 --2- 172.21.15.181:0/2166291438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4300afa10 0x7fd4300c45a0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 >> 172.21.15.181:0/2166291438 conn(0x7fd4300d04a0 msgr2=0x7fd4300afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 shutdown_connections 2023-10-16T11:00:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.362+0000 7fd437d04700 1 -- 172.21.15.181:0/2166291438 wait complete. 2023-10-16T11:00:29.366 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:00:29.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:00:29.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.505+0000 7fbeeee8a700 1 Processor -- start 2023-10-16T11:00:29.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.505+0000 7fbeeee8a700 1 -- start start 2023-10-16T11:00:29.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbeeee8a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 0x7fbee80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbeeee8a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee806ce60 con 0x7fbee81608e0 2023-10-16T11:00:29.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbeeee8a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee806d2f0 con 0x7fbee80b0f70 2023-10-16T11:00:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbeeee8a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80d1020 con 0x7fbee8160cc0 2023-10-16T11:00:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbeecc26700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbee81608e0 0x7fbee80cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33244/0 (socket says 172.21.15.181:33244) 2023-10-16T11:00:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbeecc26700 1 -- 172.21.15.181:0/194824282 learned_addr learned my addr 172.21.15.181:0/194824282 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177157257 0 0) 0x7fbee806ce60 con 0x7fbee81608e0 2023-10-16T11:00:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbee7fff700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 0x7fbee80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.506+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed80035b0 con 0x7fbee81608e0 2023-10-16T11:00:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 296468854 0 0) 0x7fbed80035b0 con 0x7fbee81608e0 2023-10-16T11:00:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 msgr2=0x7fbee80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 0x7fbee80b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 >> v1:172.21.15.190:6789/0 conn(0x7fbee80b0f70 legacy=0x7fbee80b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee806c250 con 0x7fbee81608e0 2023-10-16T11:00:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fbedc001670 con 0x7fbee81608e0 2023-10-16T11:00:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 >> v1:172.21.15.181:6789/0 conn(0x7fbee81608e0 legacy=0x7fbee80cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8003a30 0x7fbed8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbed8004850 0x7fbed8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 0x7fbed8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed800bac0 con 0x7fbed8004850 2023-10-16T11:00:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed800bc00 con 0x7fbed8003a30 2023-10-16T11:00:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed800bdc0 con 0x7fbee8160cc0 2023-10-16T11:00:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee7fff700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbed8004850 0x7fbed8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbeecc26700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8003a30 0x7fbed8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbeed427700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 0x7fbed8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee7fff700 1 -- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 msgr2=0x7fbed8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:29.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee7fff700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee8160cc0 0x7fbed8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee7fff700 1 -- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8003a30 msgr2=0x7fbed8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:29.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee7fff700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8003a30 0x7fbed8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee7fff700 1 -- 172.21.15.181:0/194824282 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed80035b0 con 0x7fbed8004850 2023-10-16T11:00:29.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee7fff700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbed8004850 0x7fbed8004cd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009b30 tx=0x7fbed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d475bc1da9621f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:29.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed400e6c0 con 0x7fbed8004850 2023-10-16T11:00:29.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed400e820 con 0x7fbed8004850 2023-10-16T11:00:29.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.507+0000 7fbee77fe700 1 -- 172.21.15.181:0/194824282 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed400eb90 con 0x7fbed8004850 2023-10-16T11:00:29.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.508+0000 7fbeeee8a700 1 -- 172.21.15.181:0/194824282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbed8004850 msgr2=0x7fbed8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.508+0000 7fbeeee8a700 1 --2- 172.21.15.181:0/194824282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbed8004850 0x7fbed8004cd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009b30 tx=0x7fbed8005880 comp rx=0 tx=0).stop 2023-10-16T11:00:29.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.508+0000 7fbeed427700 1 -- 172.21.15.181:0/194824282 reap_dead start 2023-10-16T11:00:29.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.508+0000 7fbeeee8a700 1 -- 172.21.15.181:0/194824282 shutdown_connections 2023-10-16T11:00:29.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.508+0000 7fbeeee8a700 1 -- 172.21.15.181:0/194824282 >> 172.21.15.181:0/194824282 conn(0x7fbee80d1610 msgr2=0x7fbee80d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:29.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.508+0000 7fbeeee8a700 1 -- 172.21.15.181:0/194824282 shutdown_connections 2023-10-16T11:00:29.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.508+0000 7fbeeee8a700 1 -- 172.21.15.181:0/194824282 wait complete. 2023-10-16T11:00:29.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.509+0000 7fbeeee8a700 1 Processor -- start 2023-10-16T11:00:29.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.510+0000 7fbeeee8a700 1 -- start start 2023-10-16T11:00:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8004850 0x7fbee8156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee815ac40 0x7fbee80af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee81f7930 0x7fbee80afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee81f7510 con 0x7fbee81f7930 2023-10-16T11:00:29.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee815b470 con 0x7fbed8004850 2023-10-16T11:00:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee815b190 con 0x7fbee815ac40 2023-10-16T11:00:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee81f7930 0x7fbee80afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee81f7930 0x7fbee80afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50414/0 (socket says 172.21.15.181:50414) 2023-10-16T11:00:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 -- 172.21.15.181:0/1236897586 learned_addr learned my addr 172.21.15.181:0/1236897586 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:29.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbee7fff700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee815ac40 0x7fbee80af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeecc26700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8004850 0x7fbee8156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 -- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee815ac40 msgr2=0x7fbee80af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee815ac40 0x7fbee80af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 -- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8004850 msgr2=0x7fbee8156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8004850 0x7fbee8156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 -- 172.21.15.181:0/1236897586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee81f6e50 con 0x7fbee81f7930 2023-10-16T11:00:29.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeed427700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee81f7930 0x7fbee80afa10 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fbee8071520 tx=0x7fbee000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:29.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbee57fa700 1 -- 172.21.15.181:0/1236897586 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0003c10 con 0x7fbee81f7930 2023-10-16T11:00:29.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbee57fa700 1 -- 172.21.15.181:0/1236897586 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee0003d90 con 0x7fbee81f7930 2023-10-16T11:00:29.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee81f7130 con 0x7fbee81f7930 2023-10-16T11:00:29.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbee57fa700 1 -- 172.21.15.181:0/1236897586 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee000d960 con 0x7fbee81f7930 2023-10-16T11:00:29.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.511+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee8159c40 con 0x7fbee81f7930 2023-10-16T11:00:29.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.512+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee8067d20 con 0x7fbee81f7930 2023-10-16T11:00:29.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.513+0000 7fbee57fa700 1 -- 172.21.15.181:0/1236897586 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 98) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbee000dac0 con 0x7fbee81f7930 2023-10-16T11:00:29.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.513+0000 7fbee57fa700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fbecc06af90 0x7fbecc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.513+0000 7fbee57fa700 1 -- 172.21.15.181:0/1236897586 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fbee0089ac0 con 0x7fbee81f7930 2023-10-16T11:00:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.513+0000 7fbeecc26700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fbecc06af90 0x7fbecc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.514+0000 7fbeecc26700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fbecc06af90 0x7fbecc06d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbedc000c00 tx=0x7fbedc007980 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:29.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.518+0000 7fbee57fa700 1 -- 172.21.15.181:0/1236897586 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbee0054800 con 0x7fbee81f7930 2023-10-16T11:00:29.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.711+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbee80ceb70 con 0x7fbee81f7930 2023-10-16T11:00:29.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.756+0000 7fbee57fa700 1 -- 172.21.15.181:0/1236897586 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbee0057e20 con 0x7fbee81f7930 2023-10-16T11:00:29.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fbecc06af90 msgr2=0x7fbecc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fbecc06af90 0x7fbecc06d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbedc000c00 tx=0x7fbedc007980 comp rx=0 tx=0).stop 2023-10-16T11:00:29.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee81f7930 msgr2=0x7fbee80afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee81f7930 0x7fbee80afa10 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fbee8071520 tx=0x7fbee000d150 comp rx=0 tx=0).stop 2023-10-16T11:00:29.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 shutdown_connections 2023-10-16T11:00:29.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fbecc06af90 0x7fbecc06d450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbee81f7930 0x7fbee80afa10 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbed8004850 0x7fbee8156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 --2- 172.21.15.181:0/1236897586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbee815ac40 0x7fbee80af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 >> 172.21.15.181:0/1236897586 conn(0x7fbee80d1610 msgr2=0x7fbee80ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 shutdown_connections 2023-10-16T11:00:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.759+0000 7fbeeee8a700 1 -- 172.21.15.181:0/1236897586 wait complete. 2023-10-16T11:00:29.763 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:00:29.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:00:29.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.897+0000 7f1545e31700 1 Processor -- start 2023-10-16T11:00:29.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f1545e31700 1 -- start start 2023-10-16T11:00:29.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f1545e31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015b850 0x7f15400665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f1545e31700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f154015fee0 0x7f15400c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f1545e31700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15400c3d40 0x7f154006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f1545e31700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154006e4b0 con 0x7f15400c3d40 2023-10-16T11:00:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f1545e31700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154006ed50 con 0x7f154015fee0 2023-10-16T11:00:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f1545e31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15400cb0f0 con 0x7f154015b850 2023-10-16T11:00:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f153ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15400c3d40 0x7f154006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f153ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15400c3d40 0x7f154006f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50422/0 (socket says 172.21.15.181:50422) 2023-10-16T11:00:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.898+0000 7f153ffff700 1 -- 172.21.15.181:0/1847204619 learned_addr learned my addr 172.21.15.181:0/1847204619 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153f7fe700 1 --2- 172.21.15.181:0/1847204619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015b850 0x7f15400665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153ffff700 1 -- 172.21.15.181:0/1847204619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015b850 msgr2=0x7f15400665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153ffff700 1 --2- 172.21.15.181:0/1847204619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015b850 0x7f15400665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153ffff700 1 -- 172.21.15.181:0/1847204619 >> v2:172.21.15.190:3300/0 conn(0x7f154015fee0 msgr2=0x7f15400c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:29.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153effd700 1 --2- 172.21.15.181:0/1847204619 >> v2:172.21.15.190:3300/0 conn(0x7f154015fee0 0x7f15400c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153ffff700 1 --2- 172.21.15.181:0/1847204619 >> v2:172.21.15.190:3300/0 conn(0x7f154015fee0 0x7f15400c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153ffff700 1 -- 172.21.15.181:0/1847204619 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540066b80 con 0x7f15400c3d40 2023-10-16T11:00:29.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153f7fe700 1 --2- 172.21.15.181:0/1847204619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015b850 0x7f15400665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153effd700 1 --2- 172.21.15.181:0/1847204619 >> v2:172.21.15.190:3300/0 conn(0x7f154015fee0 0x7f15400c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153ffff700 1 --2- 172.21.15.181:0/1847204619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154006f1e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1540067270 tx=0x7f153800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c6ffcbfe6bb009 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153e7fc700 1 -- 172.21.15.181:0/1847204619 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800e070 con 0x7f15400c3d40 2023-10-16T11:00:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153e7fc700 1 -- 172.21.15.181:0/1847204619 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153800b9c0 con 0x7f15400c3d40 2023-10-16T11:00:29.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.899+0000 7f153e7fc700 1 -- 172.21.15.181:0/1847204619 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800bd70 con 0x7f15400c3d40 2023-10-16T11:00:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 -- 172.21.15.181:0/1847204619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 msgr2=0x7f154006f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 --2- 172.21.15.181:0/1847204619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154006f1e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1540067270 tx=0x7f153800b330 comp rx=0 tx=0).stop 2023-10-16T11:00:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 -- 172.21.15.181:0/1847204619 shutdown_connections 2023-10-16T11:00:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 --2- 172.21.15.181:0/1847204619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154006f1e0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 --2- 172.21.15.181:0/1847204619 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015b850 0x7f15400665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 --2- 172.21.15.181:0/1847204619 >> v2:172.21.15.190:3300/0 conn(0x7f154015fee0 0x7f15400c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 -- 172.21.15.181:0/1847204619 >> 172.21.15.181:0/1847204619 conn(0x7f15400cc730 msgr2=0x7f15400ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 -- 172.21.15.181:0/1847204619 shutdown_connections 2023-10-16T11:00:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.900+0000 7f1545e31700 1 -- 172.21.15.181:0/1847204619 wait complete. 2023-10-16T11:00:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.901+0000 7f1545e31700 1 Processor -- start 2023-10-16T11:00:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.901+0000 7f1545e31700 1 -- start start 2023-10-16T11:00:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1545e31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1545e31700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f154015b850 0x7f15400cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1545e31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015fee0 0x7f15401f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1545e31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154015dc50 con 0x7f15400c3d40 2023-10-16T11:00:29.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1545e31700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154015df30 con 0x7f154015b850 2023-10-16T11:00:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1545e31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154015e210 con 0x7f154015fee0 2023-10-16T11:00:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154015d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50424/0 (socket says 172.21.15.181:50424) 2023-10-16T11:00:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 -- 172.21.15.181:0/3161001921 learned_addr learned my addr 172.21.15.181:0/3161001921 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153effd700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f154015b850 0x7f15400cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153ffff700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015fee0 0x7f15401f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 -- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015fee0 msgr2=0x7f15401f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015fee0 0x7f15401f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 -- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f154015b850 msgr2=0x7f15400cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f154015b850 0x7f15400cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 -- 172.21.15.181:0/3161001921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538009b30 con 0x7f15400c3d40 2023-10-16T11:00:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f153f7fe700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154015d640 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f153400c750 tx=0x7f153400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1527fff700 1 -- 172.21.15.181:0/3161001921 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534011800 con 0x7f15400c3d40 2023-10-16T11:00:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.902+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154015e4f0 con 0x7f15400c3d40 2023-10-16T11:00:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.903+0000 7f1527fff700 1 -- 172.21.15.181:0/3161001921 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1534011960 con 0x7f15400c3d40 2023-10-16T11:00:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.903+0000 7f1527fff700 1 -- 172.21.15.181:0/3161001921 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534011cd0 con 0x7f15400c3d40 2023-10-16T11:00:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.903+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15400cf7f0 con 0x7f15400c3d40 2023-10-16T11:00:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.904+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15400acb60 con 0x7f15400c3d40 2023-10-16T11:00:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.904+0000 7f1527fff700 1 -- 172.21.15.181:0/3161001921 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 98) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1534021350 con 0x7f15400c3d40 2023-10-16T11:00:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.904+0000 7f1527fff700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f152c06af90 0x7f152c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.904+0000 7f1527fff700 1 -- 172.21.15.181:0/3161001921 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f1534089820 con 0x7f15400c3d40 2023-10-16T11:00:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.904+0000 7f153effd700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f152c06af90 0x7f152c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.905+0000 7f153effd700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f152c06af90 0x7f152c06d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f154015db80 tx=0x7f1530008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:29.909+0000 7f1527fff700 1 -- 172.21.15.181:0/3161001921 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f15400acb60 con 0x7f15400c3d40 2023-10-16T11:00:30.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.101+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f15400acb60 con 0x7f15400c3d40 2023-10-16T11:00:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.104+0000 7f1527fff700 1 -- 172.21.15.181:0/3161001921 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v55) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1534054580 con 0x7f15400c3d40 2023-10-16T11:00:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f152c06af90 msgr2=0x7f152c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f152c06af90 0x7f152c06d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f154015db80 tx=0x7f1530008040 comp rx=0 tx=0).stop 2023-10-16T11:00:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 msgr2=0x7f154015d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154015d640 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f153400c750 tx=0x7f153400ca60 comp rx=0 tx=0).stop 2023-10-16T11:00:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 shutdown_connections 2023-10-16T11:00:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f152c06af90 0x7f152c06d450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f154015fee0 0x7f15401f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15400c3d40 0x7f154015d640 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 --2- 172.21.15.181:0/3161001921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f154015b850 0x7f15400cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 >> 172.21.15.181:0/3161001921 conn(0x7f15400cc730 msgr2=0x7f15400af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 shutdown_connections 2023-10-16T11:00:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.107+0000 7f1545e31700 1 -- 172.21.15.181:0/3161001921 wait complete. 2023-10-16T11:00:30.121 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:00:30.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.251+0000 7f18ee832700 1 Processor -- start 2023-10-16T11:00:30.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.251+0000 7f18ee832700 1 -- start start 2023-10-16T11:00:30.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18ee832700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18e815dbd0 0x7f18e80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18ee832700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18e80d0610 0x7f18e80cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18ee832700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e8161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18ee832700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e80ce740 con 0x7f18e815dbd0 2023-10-16T11:00:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18ee832700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e815dfb0 con 0x7f18e80d0610 2023-10-16T11:00:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18ee832700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e80cd3a0 con 0x7f18e80ce2c0 2023-10-16T11:00:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18e7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18e815dbd0 0x7f18e80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18e7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18e815dbd0 0x7f18e80d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50432/0 (socket says 172.21.15.181:50432) 2023-10-16T11:00:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.252+0000 7f18e7fff700 1 -- 172.21.15.181:0/1557875692 learned_addr learned my addr 172.21.15.181:0/1557875692 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e77fe700 1 --2- 172.21.15.181:0/1557875692 >> v2:172.21.15.190:3300/0 conn(0x7f18e80d0610 0x7f18e80cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18ecdcf700 1 --2- 172.21.15.181:0/1557875692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e8161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e7fff700 1 -- 172.21.15.181:0/1557875692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 msgr2=0x7f18e8161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e7fff700 1 --2- 172.21.15.181:0/1557875692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e8161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e7fff700 1 -- 172.21.15.181:0/1557875692 >> v2:172.21.15.190:3300/0 conn(0x7f18e80d0610 msgr2=0x7f18e80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:30.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e7fff700 1 --2- 172.21.15.181:0/1557875692 >> v2:172.21.15.190:3300/0 conn(0x7f18e80d0610 0x7f18e80cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e7fff700 1 -- 172.21.15.181:0/1557875692 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80cd830 con 0x7f18e815dbd0 2023-10-16T11:00:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e7fff700 1 --2- 172.21.15.181:0/1557875692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e815dbd0 0x7f18e80d00d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f18e80cb990 tx=0x7f18dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1aabbc2f989080 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e6ffd700 1 -- 172.21.15.181:0/1557875692 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00b870 con 0x7f18e815dbd0 2023-10-16T11:00:30.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e6ffd700 1 -- 172.21.15.181:0/1557875692 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00b9d0 con 0x7f18e815dbd0 2023-10-16T11:00:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.253+0000 7f18e6ffd700 1 -- 172.21.15.181:0/1557875692 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00bd60 con 0x7f18e815dbd0 2023-10-16T11:00:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 -- 172.21.15.181:0/1557875692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e815dbd0 msgr2=0x7f18e80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 --2- 172.21.15.181:0/1557875692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e815dbd0 0x7f18e80d00d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f18e80cb990 tx=0x7f18dc009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 -- 172.21.15.181:0/1557875692 shutdown_connections 2023-10-16T11:00:30.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 --2- 172.21.15.181:0/1557875692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e815dbd0 0x7f18e80d00d0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 --2- 172.21.15.181:0/1557875692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e8161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 --2- 172.21.15.181:0/1557875692 >> v2:172.21.15.190:3300/0 conn(0x7f18e80d0610 0x7f18e80cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 -- 172.21.15.181:0/1557875692 >> 172.21.15.181:0/1557875692 conn(0x7f18e809a930 msgr2=0x7f18e80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:30.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 -- 172.21.15.181:0/1557875692 shutdown_connections 2023-10-16T11:00:30.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.254+0000 7f18ee832700 1 -- 172.21.15.181:0/1557875692 wait complete. 2023-10-16T11:00:30.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.255+0000 7f18ee832700 1 Processor -- start 2023-10-16T11:00:30.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.255+0000 7f18ee832700 1 -- start start 2023-10-16T11:00:30.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ee832700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e81f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ee832700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e80d0610 0x7f18e80b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ee832700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18e815dbd0 0x7f18e80b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ee832700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e81f3780 con 0x7f18e80d0610 2023-10-16T11:00:30.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ee832700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e81f7220 con 0x7f18e815dbd0 2023-10-16T11:00:30.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ee832700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e81f7500 con 0x7f18e80ce2c0 2023-10-16T11:00:30.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e80d0610 0x7f18e80b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e80d0610 0x7f18e80b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50440/0 (socket says 172.21.15.181:50440) 2023-10-16T11:00:30.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 -- 172.21.15.181:0/1941825790 learned_addr learned my addr 172.21.15.181:0/1941825790 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:30.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e7fff700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e81f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ecdcf700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18e815dbd0 0x7f18e80b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 -- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 msgr2=0x7f18e81f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e81f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 -- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18e815dbd0 msgr2=0x7f18e80b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18e815dbd0 0x7f18e80b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 -- 172.21.15.181:0/1941825790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009550 con 0x7f18e80d0610 2023-10-16T11:00:30.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ecdcf700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18e815dbd0 0x7f18e80b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:30.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e77fe700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e80d0610 0x7f18e80b6520 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e7e0 tx=0x7f18d800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e4ff9700 1 -- 172.21.15.181:0/1941825790 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8013610 con 0x7f18e80d0610 2023-10-16T11:00:30.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18e4ff9700 1 -- 172.21.15.181:0/1941825790 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8013770 con 0x7f18e80d0610 2023-10-16T11:00:30.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.256+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e81f77e0 con 0x7f18e80d0610 2023-10-16T11:00:30.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.257+0000 7f18e4ff9700 1 -- 172.21.15.181:0/1941825790 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8013ae0 con 0x7f18e80d0610 2023-10-16T11:00:30.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.257+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e80caee0 con 0x7f18e80d0610 2023-10-16T11:00:30.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.257+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e8067d20 con 0x7f18e80d0610 2023-10-16T11:00:30.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.258+0000 7f18e4ff9700 1 -- 172.21.15.181:0/1941825790 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 98) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f18d8013c40 con 0x7f18e80d0610 2023-10-16T11:00:30.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.258+0000 7f18e4ff9700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f18d406af90 0x7f18d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.258+0000 7f18e4ff9700 1 -- 172.21.15.181:0/1941825790 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f18d8089540 con 0x7f18e80d0610 2023-10-16T11:00:30.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.258+0000 7f18e7fff700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f18d406af90 0x7f18d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.259+0000 7f18e7fff700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f18d406af90 0x7f18d406d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c00 tx=0x7f18dc008030 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.263+0000 7f18e4ff9700 1 -- 172.21.15.181:0/1941825790 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f18d8054320 con 0x7f18e80d0610 2023-10-16T11:00:30.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.453+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f18e80cebb0 con 0x7f18e80d0610 2023-10-16T11:00:30.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.457+0000 7f18e4ff9700 1 -- 172.21.15.181:0/1941825790 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v56) v1 ==== 120+0+0 (secure 0 0 0) 0x7f18d8057940 con 0x7f18e80d0610 2023-10-16T11:00:30.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f18d406af90 msgr2=0x7f18d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f18d406af90 0x7f18d406d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c00 tx=0x7f18dc008030 comp rx=0 tx=0).stop 2023-10-16T11:00:30.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e80d0610 msgr2=0x7f18e80b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e80d0610 0x7f18e80b6520 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e7e0 tx=0x7f18d800eaf0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 shutdown_connections 2023-10-16T11:00:30.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f18d406af90 0x7f18d406d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18e80ce2c0 0x7f18e81f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18e815dbd0 0x7f18e80b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 --2- 172.21.15.181:0/1941825790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e80d0610 0x7f18e80b6520 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 >> 172.21.15.181:0/1941825790 conn(0x7f18e809a930 msgr2=0x7f18e8161de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:30.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 shutdown_connections 2023-10-16T11:00:30.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.459+0000 7f18ee832700 1 -- 172.21.15.181:0/1941825790 wait complete. 2023-10-16T11:00:30.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:00:30.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.594+0000 7f61f729d700 1 Processor -- start 2023-10-16T11:00:30.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.594+0000 7f61f729d700 1 -- start start 2023-10-16T11:00:30.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f729d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61f00cdd00 0x7f61f006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f729d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f00ca300 con 0x7f61f00666a0 2023-10-16T11:00:30.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f729d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f006b6c0 con 0x7f61f00ce0e0 2023-10-16T11:00:30.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f729d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f006e5e0 con 0x7f61f00cdd00 2023-10-16T11:00:30.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f5039700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61f00cdd00 0x7f61f006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f4838700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f61f00666a0 0x7f61f006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33292/0 (socket says 172.21.15.181:33292) 2023-10-16T11:00:30.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f4838700 1 -- 172.21.15.181:0/3358680988 learned_addr learned my addr 172.21.15.181:0/3358680988 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:30.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f5039700 1 -- 172.21.15.181:0/3358680988 >> v1:172.21.15.190:6789/0 conn(0x7f61f00ce0e0 legacy=0x7f61f006afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:30.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f5039700 1 -- 172.21.15.181:0/3358680988 >> v1:172.21.15.181:6789/0 conn(0x7f61f00666a0 legacy=0x7f61f006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f5039700 1 -- 172.21.15.181:0/3358680988 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00ca580 con 0x7f61f00cdd00 2023-10-16T11:00:30.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61e7fff700 1 -- 172.21.15.181:0/3358680988 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228507219 0 0) 0x7f61f00ca300 con 0x7f61f00666a0 2023-10-16T11:00:30.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61f5039700 1 --2- 172.21.15.181:0/3358680988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f006f5f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009840 tx=0x7f61ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56ebed85daf5f80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61e7fff700 1 -- 172.21.15.181:0/3358680988 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec004230 con 0x7f61f00cdd00 2023-10-16T11:00:30.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61e7fff700 1 -- 172.21.15.181:0/3358680988 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec004390 con 0x7f61f00cdd00 2023-10-16T11:00:30.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.595+0000 7f61e7fff700 1 -- 172.21.15.181:0/3358680988 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0036a0 con 0x7f61f00cdd00 2023-10-16T11:00:30.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.596+0000 7f61f729d700 1 -- 172.21.15.181:0/3358680988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 msgr2=0x7f61f006f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.596+0000 7f61f729d700 1 --2- 172.21.15.181:0/3358680988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f006f5f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009840 tx=0x7f61ec009b50 comp rx=0 tx=0).stop 2023-10-16T11:00:30.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.596+0000 7f61f729d700 1 -- 172.21.15.181:0/3358680988 shutdown_connections 2023-10-16T11:00:30.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.596+0000 7f61f729d700 1 --2- 172.21.15.181:0/3358680988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f006f5f0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.596+0000 7f61f729d700 1 -- 172.21.15.181:0/3358680988 >> 172.21.15.181:0/3358680988 conn(0x7f61f00cc730 msgr2=0x7f61f00ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:30.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.596+0000 7f61f729d700 1 -- 172.21.15.181:0/3358680988 shutdown_connections 2023-10-16T11:00:30.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.596+0000 7f61f729d700 1 -- 172.21.15.181:0/3358680988 wait complete. 2023-10-16T11:00:30.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.597+0000 7f61f729d700 1 Processor -- start 2023-10-16T11:00:30.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.597+0000 7f61f729d700 1 -- start start 2023-10-16T11:00:30.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f729d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f00666a0 0x7f61f00c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f729d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f01f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f729d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f00ce0e0 0x7f61f01f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f729d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f01e8e30 con 0x7f61f00cdd00 2023-10-16T11:00:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f729d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f01f64d0 con 0x7f61f00666a0 2023-10-16T11:00:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f729d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f01f33f0 con 0x7f61f00ce0e0 2023-10-16T11:00:30.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f01f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f01f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50452/0 (socket says 172.21.15.181:50452) 2023-10-16T11:00:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 -- 172.21.15.181:0/4291504215 learned_addr learned my addr 172.21.15.181:0/4291504215 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f5039700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f00666a0 0x7f61f00c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f583a700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f00ce0e0 0x7f61f01f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 -- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f00ce0e0 msgr2=0x7f61f01f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f00ce0e0 0x7f61f01f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 -- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f00666a0 msgr2=0x7f61f00c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f00666a0 0x7f61f00c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f4838700 1 -- 172.21.15.181:0/4291504215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0094a0 con 0x7f61f00cdd00 2023-10-16T11:00:30.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.598+0000 7f61f583a700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f00ce0e0 0x7f61f01f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.599+0000 7f61f4838700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f01f2eb0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c520 tx=0x7f61e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.599+0000 7f61e5ffb700 1 -- 172.21.15.181:0/4291504215 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0004c80 con 0x7f61f00cdd00 2023-10-16T11:00:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.599+0000 7f61e5ffb700 1 -- 172.21.15.181:0/4291504215 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e0004de0 con 0x7f61f00cdd00 2023-10-16T11:00:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.599+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f01f36c0 con 0x7f61f00cdd00 2023-10-16T11:00:30.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.599+0000 7f61e5ffb700 1 -- 172.21.15.181:0/4291504215 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e00072d0 con 0x7f61f00cdd00 2023-10-16T11:00:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.599+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f01f3c80 con 0x7f61f00cdd00 2023-10-16T11:00:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.600+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f01615a0 con 0x7f61f00cdd00 2023-10-16T11:00:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.600+0000 7f61e5ffb700 1 -- 172.21.15.181:0/4291504215 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 98) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f61e0007430 con 0x7f61f00cdd00 2023-10-16T11:00:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.601+0000 7f61e5ffb700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f61d406af90 0x7f61d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.601+0000 7f61e5ffb700 1 -- 172.21.15.181:0/4291504215 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f61e0089570 con 0x7f61f00cdd00 2023-10-16T11:00:30.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.601+0000 7f61f5039700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f61d406af90 0x7f61d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.601+0000 7f61f5039700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f61d406af90 0x7f61d406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61ec000c00 tx=0x7f61ec01c040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.605+0000 7f61e5ffb700 1 -- 172.21.15.181:0/4291504215 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f61e0054460 con 0x7f61f00cdd00 2023-10-16T11:00:30.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.799+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f61f0071960 con 0x7f61f00cdd00 2023-10-16T11:00:30.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.803+0000 7f61e5ffb700 1 -- 172.21.15.181:0/4291504215 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61e0057a80 con 0x7f61f00cdd00 2023-10-16T11:00:30.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f61d406af90 msgr2=0x7f61d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f61d406af90 0x7f61d406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61ec000c00 tx=0x7f61ec01c040 comp rx=0 tx=0).stop 2023-10-16T11:00:30.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 msgr2=0x7f61f01f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f01f2eb0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c520 tx=0x7f61e000c8e0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 shutdown_connections 2023-10-16T11:00:30.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f61d406af90 0x7f61d406d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61f00ce0e0 0x7f61f01f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61f00666a0 0x7f61f00c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 --2- 172.21.15.181:0/4291504215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f00cdd00 0x7f61f01f2eb0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 >> 172.21.15.181:0/4291504215 conn(0x7f61f00cc730 msgr2=0x7f61f00ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 shutdown_connections 2023-10-16T11:00:30.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.805+0000 7f61f729d700 1 -- 172.21.15.181:0/4291504215 wait complete. 2023-10-16T11:00:30.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-16T11:00:30.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.938+0000 7f6dbbe31700 1 Processor -- start 2023-10-16T11:00:30.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.938+0000 7f6dbbe31700 1 -- start start 2023-10-16T11:00:30.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6dbbe31700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6db415dfd0 0x7f6db40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6dbbe31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db40af450 0x7f6db406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6dbbe31700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db406f870 con 0x7f6db406db80 2023-10-16T11:00:30.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6dbbe31700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40d1160 con 0x7f6db415dfd0 2023-10-16T11:00:30.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6dbbe31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40cc8e0 con 0x7f6db40af450 2023-10-16T11:00:30.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6db9bcd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6db415dfd0 0x7f6db40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6dba3ce700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f6db406db80 0x7f6db40cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:45482/0 (socket says 172.21.15.181:45482) 2023-10-16T11:00:30.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6db93cc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db40af450 0x7f6db406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.939+0000 7f6dba3ce700 1 -- 172.21.15.181:0/158841362 learned_addr learned my addr 172.21.15.181:0/158841362 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:30.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db8bcb700 1 -- 172.21.15.181:0/158841362 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982612641 0 0) 0x7f6db406f870 con 0x7f6db406db80 2023-10-16T11:00:30.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db8bcb700 1 -- 172.21.15.181:0/158841362 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da00035b0 con 0x7f6db406db80 2023-10-16T11:00:30.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db9bcd700 1 -- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db40af450 msgr2=0x7f6db406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:30.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db9bcd700 1 --2- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db40af450 0x7f6db406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db9bcd700 1 -- 172.21.15.181:0/158841362 >> v1:172.21.15.190:6789/0 conn(0x7f6db406db80 legacy=0x7f6db40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db9bcd700 1 -- 172.21.15.181:0/158841362 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40ccbc0 con 0x7f6db415dfd0 2023-10-16T11:00:30.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db93cc700 1 --2- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db40af450 0x7f6db406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:30.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db9bcd700 1 --2- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db415dfd0 0x7f6db40af9a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009820 tx=0x7f6da4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2383435c6fa8434 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db8bcb700 1 -- 172.21.15.181:0/158841362 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4005e30 con 0x7f6db415dfd0 2023-10-16T11:00:30.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db8bcb700 1 -- 172.21.15.181:0/158841362 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da4006030 con 0x7f6db415dfd0 2023-10-16T11:00:30.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.940+0000 7f6db8bcb700 1 -- 172.21.15.181:0/158841362 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da40063c0 con 0x7f6db415dfd0 2023-10-16T11:00:30.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 -- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db415dfd0 msgr2=0x7f6db40af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db415dfd0 0x7f6db40af9a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009820 tx=0x7f6da4009b30 comp rx=0 tx=0).stop 2023-10-16T11:00:30.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 -- 172.21.15.181:0/158841362 shutdown_connections 2023-10-16T11:00:30.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db415dfd0 0x7f6db40af9a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/158841362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db40af450 0x7f6db406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 -- 172.21.15.181:0/158841362 >> 172.21.15.181:0/158841362 conn(0x7f6db40d04a0 msgr2=0x7f6db40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:30.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 -- 172.21.15.181:0/158841362 shutdown_connections 2023-10-16T11:00:30.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.941+0000 7f6dbbe31700 1 -- 172.21.15.181:0/158841362 wait complete. 2023-10-16T11:00:30.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.942+0000 7f6dbbe31700 1 Processor -- start 2023-10-16T11:00:30.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.942+0000 7f6dbbe31700 1 -- start start 2023-10-16T11:00:30.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dbbe31700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db406db80 0x7f6db4153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dbbe31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db40af450 0x7f6db40aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dbbe31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db415dfd0 0x7f6db40b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dbbe31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db406f850 con 0x7f6db40af450 2023-10-16T11:00:30.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dbbe31700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b2d90 con 0x7f6db406db80 2023-10-16T11:00:30.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dbbe31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db406d7e0 con 0x7f6db415dfd0 2023-10-16T11:00:30.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db40af450 0x7f6db40aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db40af450 0x7f6db40aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50466/0 (socket says 172.21.15.181:50466) 2023-10-16T11:00:30.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 -- 172.21.15.181:0/3771631092 learned_addr learned my addr 172.21.15.181:0/3771631092 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:30.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dba3ce700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db415dfd0 0x7f6db40b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db9bcd700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db406db80 0x7f6db4153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 -- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db415dfd0 msgr2=0x7f6db40b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db415dfd0 0x7f6db40b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 -- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db406db80 msgr2=0x7f6db4153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:30.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db406db80 0x7f6db4153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:30.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 -- 172.21.15.181:0/3771631092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db00094a0 con 0x7f6db40af450 2023-10-16T11:00:30.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6db93cc700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db40af450 0x7f6db40aff00 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6db000ebe0 tx=0x7f6db000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000c850 con 0x7f6db40af450 2023-10-16T11:00:30.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db000c9b0 con 0x7f6db40af450 2023-10-16T11:00:30.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da4009430 con 0x7f6db40af450 2023-10-16T11:00:30.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.943+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000cd40 con 0x7f6db40af450 2023-10-16T11:00:30.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.944+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db40c9fb0 con 0x7f6db40af450 2023-10-16T11:00:30.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.944+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db40ca320 con 0x7f6db40af450 2023-10-16T11:00:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.945+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 98) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6db0013470 con 0x7f6db40af450 2023-10-16T11:00:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.946+0000 7f6daa7fc700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6d9806af90 0x7f6d9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.946+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f6db008a150 con 0x7f6db40af450 2023-10-16T11:00:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.946+0000 7f6db9bcd700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6d9806af90 0x7f6d9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.946+0000 7f6db9bcd700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6d9806af90 0x7f6d9806d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009b70 tx=0x7f6da4008070 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:30.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:30.949+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6db0054fd0 con 0x7f6db40af450 2023-10-16T11:00:31.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.118+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6db00553a0 con 0x7f6db40af450 2023-10-16T11:00:31.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.223+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f6db40cf880 con 0x7f6db40af450 2023-10-16T11:00:31.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.817+0000 7f6daa7fc700 1 -- 172.21.15.181:0/3771631092 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2485) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6db003cd50 con 0x7f6db40af450 2023-10-16T11:00:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.819+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6d9806af90 msgr2=0x7f6d9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:31.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.819+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6d9806af90 0x7f6d9806d450 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009b70 tx=0x7f6da4008070 comp rx=0 tx=0).stop 2023-10-16T11:00:31.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.819+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db40af450 msgr2=0x7f6db40aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:31.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.819+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db40af450 0x7f6db40aff00 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6db000ebe0 tx=0x7f6db000eef0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 shutdown_connections 2023-10-16T11:00:31.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6d9806af90 0x7f6d9806d450 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6db415dfd0 0x7f6db40b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6db406db80 0x7f6db4153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 --2- 172.21.15.181:0/3771631092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6db40af450 0x7f6db40aff00 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 >> 172.21.15.181:0/3771631092 conn(0x7f6db40d04a0 msgr2=0x7f6db40aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:31.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 shutdown_connections 2023-10-16T11:00:31.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.820+0000 7f6dbbe31700 1 -- 172.21.15.181:0/3771631092 wait complete. 2023-10-16T11:00:31.834 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-16T11:00:31.835 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-16T11:00:31.835 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T11:00:31.836 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:31.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.958+0000 7f9d32715700 1 Processor -- start 2023-10-16T11:00:31.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.958+0000 7f9d32715700 1 -- start start 2023-10-16T11:00:31.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d32715700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d2c0d0350 0x7f9d2c1688a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:31.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d32715700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0cc470 con 0x7f9d2c0ce7f0 2023-10-16T11:00:31.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d32715700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c06ca30 con 0x7f9d2c0d2390 2023-10-16T11:00:31.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d32715700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c06c810 con 0x7f9d2c0d0350 2023-10-16T11:00:31.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d30cb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d2c0d0350 0x7f9d2c1688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:31.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d2bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9d2c0ce7f0 0x7f9d2c06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33318/0 (socket says 172.21.15.181:33318) 2023-10-16T11:00:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d30cb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d2c0d0350 0x7f9d2c1688a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50476/0 (socket says 172.21.15.181:50476) 2023-10-16T11:00:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d2bfff700 1 -- 172.21.15.181:0/4085759515 learned_addr learned my addr 172.21.15.181:0/4085759515 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d30cb2700 1 -- 172.21.15.181:0/4085759515 >> v1:172.21.15.190:6789/0 conn(0x7f9d2c0d2390 legacy=0x7f9d2c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d30cb2700 1 -- 172.21.15.181:0/4085759515 >> v1:172.21.15.181:6789/0 conn(0x7f9d2c0ce7f0 legacy=0x7f9d2c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d30cb2700 1 -- 172.21.15.181:0/4085759515 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c06cd10 con 0x7f9d2c0d0350 2023-10-16T11:00:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d2affd700 1 -- 172.21.15.181:0/4085759515 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287303964 0 0) 0x7f9d2c0cc470 con 0x7f9d2c0ce7f0 2023-10-16T11:00:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d30cb2700 1 --2- 172.21.15.181:0/4085759515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c1688a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d90 tx=0x7f9d2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350c959ec63456ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d2affd700 1 -- 172.21.15.181:0/4085759515 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400e070 con 0x7f9d2c0d0350 2023-10-16T11:00:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d2affd700 1 -- 172.21.15.181:0/4085759515 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2400bb90 con 0x7f9d2c0d0350 2023-10-16T11:00:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.959+0000 7f9d2affd700 1 -- 172.21.15.181:0/4085759515 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003a20 con 0x7f9d2c0d0350 2023-10-16T11:00:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.960+0000 7f9d32715700 1 -- 172.21.15.181:0/4085759515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 msgr2=0x7f9d2c1688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.960+0000 7f9d32715700 1 --2- 172.21.15.181:0/4085759515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c1688a0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d90 tx=0x7f9d2400b330 comp rx=0 tx=0).stop 2023-10-16T11:00:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.960+0000 7f9d32715700 1 -- 172.21.15.181:0/4085759515 shutdown_connections 2023-10-16T11:00:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.960+0000 7f9d32715700 1 --2- 172.21.15.181:0/4085759515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c1688a0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.960+0000 7f9d32715700 1 -- 172.21.15.181:0/4085759515 >> 172.21.15.181:0/4085759515 conn(0x7f9d2c09a9a0 msgr2=0x7f9d2c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.960+0000 7f9d32715700 1 -- 172.21.15.181:0/4085759515 shutdown_connections 2023-10-16T11:00:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.960+0000 7f9d32715700 1 -- 172.21.15.181:0/4085759515 wait complete. 2023-10-16T11:00:31.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.961+0000 7f9d32715700 1 Processor -- start 2023-10-16T11:00:31.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.961+0000 7f9d32715700 1 -- start start 2023-10-16T11:00:31.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d32715700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d2c0ce7f0 0x7f9d2c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:31.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d32715700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c15eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d32715700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d2c0d2390 0x7f9d2c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d32715700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c6750 con 0x7f9d2c0d0350 2023-10-16T11:00:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d32715700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c6400 con 0x7f9d2c0d2390 2023-10-16T11:00:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d32715700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0ae850 con 0x7f9d2c0ce7f0 2023-10-16T11:00:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c15eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c15eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50480/0 (socket says 172.21.15.181:50480) 2023-10-16T11:00:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d2b7fe700 1 -- 172.21.15.181:0/3025628082 learned_addr learned my addr 172.21.15.181:0/3025628082 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d2bfff700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d2c0ce7f0 0x7f9d2c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.962+0000 7f9d30cb2700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d2c0d2390 0x7f9d2c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d2b7fe700 1 -- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d2c0ce7f0 msgr2=0x7f9d2c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d2b7fe700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d2c0ce7f0 0x7f9d2c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d2b7fe700 1 -- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d2c0d2390 msgr2=0x7f9d2c0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d2b7fe700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d2c0d2390 0x7f9d2c0c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d2b7fe700 1 -- 172.21.15.181:0/3025628082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24009a70 con 0x7f9d2c0d0350 2023-10-16T11:00:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d2b7fe700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c15eb80 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00b4d0 tx=0x7f9d1c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:31.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d28ff9700 1 -- 172.21.15.181:0/3025628082 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00bea0 con 0x7f9d2c0d0350 2023-10-16T11:00:31.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d28ff9700 1 -- 172.21.15.181:0/3025628082 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c011830 con 0x7f9d2c0d0350 2023-10-16T11:00:31.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c0aeb30 con 0x7f9d2c0d0350 2023-10-16T11:00:31.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d28ff9700 1 -- 172.21.15.181:0/3025628082 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c011be0 con 0x7f9d2c0d0350 2023-10-16T11:00:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.963+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c15f0c0 con 0x7f9d2c0d0350 2023-10-16T11:00:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.964+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c067d20 con 0x7f9d2c0d0350 2023-10-16T11:00:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.964+0000 7f9d28ff9700 1 -- 172.21.15.181:0/3025628082 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9d1c011d40 con 0x7f9d2c0d0350 2023-10-16T11:00:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.965+0000 7f9d28ff9700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f9d1806b020 0x7f9d1806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.965+0000 7f9d2bfff700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f9d1806b020 0x7f9d1806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.965+0000 7f9d28ff9700 1 -- 172.21.15.181:0/3025628082 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f9d1c0897a0 con 0x7f9d2c0d0350 2023-10-16T11:00:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.965+0000 7f9d2bfff700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f9d1806b020 0x7f9d1806d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d200010e0 tx=0x7f9d20009040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:31.969+0000 7f9d28ff9700 1 -- 172.21.15.181:0/3025628082 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9d1c054580 con 0x7f9d2c0d0350 2023-10-16T11:00:32.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.162+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d2c161180 con 0x7f9d2c0d0350 2023-10-16T11:00:32.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.163+0000 7f9d28ff9700 1 -- 172.21.15.181:0/3025628082 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+14881 (secure 0 0 0) 0x7f9d1c057ba0 con 0x7f9d2c0d0350 2023-10-16T11:00:32.165 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:32.165 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":159,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:29.754744+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/3854101486":"2023-10-17T11:00:27.331395+0000","172.21.15.190:6833/3854101486":"2023-10-17T11:00:27.331395+0000","172.21.15.181:0/292833217":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/3382478479":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/4069049155":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/4130286653":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/3490595853":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/1561933478":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1604670603":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6833/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6833/1123221928":"2023-10-17T11:00:26.333089+0000","172.21.15.181:6832/3796559700":"2023-10-17T10:04:15.822177+0000","172.21.15.181:6832/1123221928":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/1093898558":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/2314316702":"2023-10-17T11:00:26.333089+0000","172.21.15.181:0/40763561":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/2364682800":"2023-10-17T10:04:15.822177+0000","172.21.15.181:0/1727687969":"2023-10-17T10:04:15.822177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:32.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f9d1806b020 msgr2=0x7f9d1806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f9d1806b020 0x7f9d1806d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d200010e0 tx=0x7f9d20009040 comp rx=0 tx=0).stop 2023-10-16T11:00:32.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 msgr2=0x7f9d2c15eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c15eb80 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00b4d0 tx=0x7f9d1c00b7e0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 shutdown_connections 2023-10-16T11:00:32.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f9d1806b020 0x7f9d1806d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d2c0ce7f0 0x7f9d2c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d2c0d2390 0x7f9d2c0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 --2- 172.21.15.181:0/3025628082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c0d0350 0x7f9d2c15eb80 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 >> 172.21.15.181:0/3025628082 conn(0x7f9d2c09a9a0 msgr2=0x7f9d2c0c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:32.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 shutdown_connections 2023-10-16T11:00:32.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.165+0000 7f9d32715700 1 -- 172.21.15.181:0/3025628082 wait complete. 2023-10-16T11:00:32.179 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:00:32.179 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:00:32.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.301+0000 7fe4a7f98700 1 Processor -- start 2023-10-16T11:00:32.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.301+0000 7fe4a7f98700 1 -- start start 2023-10-16T11:00:32.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a7f98700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4a00665a0 0x7fe4a015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a7f98700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a0160420 con 0x7fe4a015b850 2023-10-16T11:00:32.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a7f98700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a006e4b0 con 0x7fe4a006f1e0 2023-10-16T11:00:32.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a7f98700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a006ed50 con 0x7fe4a00665a0 2023-10-16T11:00:32.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a5533700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4a00665a0 0x7fe4a015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a5d34700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe4a015b850 0x7fe4a00c3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33332/0 (socket says 172.21.15.181:33332) 2023-10-16T11:00:32.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a5d34700 1 -- 172.21.15.181:0/3819866502 learned_addr learned my addr 172.21.15.181:0/3819866502 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:32.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a5533700 1 -- 172.21.15.181:0/3819866502 >> v1:172.21.15.190:6789/0 conn(0x7fe4a006f1e0 legacy=0x7fe4a00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:32.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a5533700 1 -- 172.21.15.181:0/3819866502 >> v1:172.21.15.181:6789/0 conn(0x7fe4a015b850 legacy=0x7fe4a00c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a5533700 1 -- 172.21.15.181:0/3819866502 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a00cb0f0 con 0x7fe4a00665a0 2023-10-16T11:00:32.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.302+0000 7fe4a4d32700 1 -- 172.21.15.181:0/3819866502 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452045879 0 0) 0x7fe490001110 con 0x7fe4a015b850 2023-10-16T11:00:32.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a5533700 1 --2- 172.21.15.181:0/3819866502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a00665a0 0x7fe4a015fee0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009840 tx=0x7fe49c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01c0dde03380221 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:32.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a4d32700 1 -- 172.21.15.181:0/3819866502 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c004230 con 0x7fe4a00665a0 2023-10-16T11:00:32.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a4d32700 1 -- 172.21.15.181:0/3819866502 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c004390 con 0x7fe4a00665a0 2023-10-16T11:00:32.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a4d32700 1 -- 172.21.15.181:0/3819866502 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0036a0 con 0x7fe4a00665a0 2023-10-16T11:00:32.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3819866502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a00665a0 msgr2=0x7fe4a015fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3819866502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a00665a0 0x7fe4a015fee0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009840 tx=0x7fe49c009b50 comp rx=0 tx=0).stop 2023-10-16T11:00:32.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3819866502 shutdown_connections 2023-10-16T11:00:32.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3819866502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a00665a0 0x7fe4a015fee0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3819866502 >> 172.21.15.181:0/3819866502 conn(0x7fe4a00cc730 msgr2=0x7fe4a00ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:32.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3819866502 shutdown_connections 2023-10-16T11:00:32.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.303+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3819866502 wait complete. 2023-10-16T11:00:32.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 Processor -- start 2023-10-16T11:00:32.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 -- start start 2023-10-16T11:00:32.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4a00665a0 0x7fe4a0154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a006f1e0 0x7fe4a00ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4a015b850 0x7fe4a00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00cf840 con 0x7fe4a006f1e0 2023-10-16T11:00:32.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00ac920 con 0x7fe4a015b850 2023-10-16T11:00:32.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a7f98700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00acc00 con 0x7fe4a00665a0 2023-10-16T11:00:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a5533700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a006f1e0 0x7fe4a00ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a5533700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a006f1e0 0x7fe4a00ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50494/0 (socket says 172.21.15.181:50494) 2023-10-16T11:00:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.305+0000 7fe4a5533700 1 -- 172.21.15.181:0/3022046422 learned_addr learned my addr 172.21.15.181:0/3022046422 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a6535700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4a015b850 0x7fe4a00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a5d34700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4a00665a0 0x7fe4a0154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a5533700 1 -- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4a00665a0 msgr2=0x7fe4a0154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a5533700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4a00665a0 0x7fe4a0154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a5533700 1 -- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4a015b850 msgr2=0x7fe4a00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a5533700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4a015b850 0x7fe4a00b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a5533700 1 -- 172.21.15.181:0/3022046422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c0094a0 con 0x7fe4a006f1e0 2023-10-16T11:00:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a5533700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a006f1e0 0x7fe4a00ad6e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0090b0 tx=0x7fe49c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4967fc700 1 -- 172.21.15.181:0/3022046422 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c011830 con 0x7fe4a006f1e0 2023-10-16T11:00:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4967fc700 1 -- 172.21.15.181:0/3022046422 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c01e070 con 0x7fe4a006f1e0 2023-10-16T11:00:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a00acee0 con 0x7fe4a006f1e0 2023-10-16T11:00:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4967fc700 1 -- 172.21.15.181:0/3022046422 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c011bb0 con 0x7fe4a006f1e0 2023-10-16T11:00:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.306+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a01545a0 con 0x7fe4a006f1e0 2023-10-16T11:00:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.307+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a0154730 con 0x7fe4a006f1e0 2023-10-16T11:00:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.307+0000 7fe4967fc700 1 -- 172.21.15.181:0/3022046422 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe49c011d10 con 0x7fe4a006f1e0 2023-10-16T11:00:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.308+0000 7fe4967fc700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fe48406b020 0x7fe48406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.308+0000 7fe4967fc700 1 -- 172.21.15.181:0/3022046422 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fe49c08ac90 con 0x7fe4a006f1e0 2023-10-16T11:00:32.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.308+0000 7fe4a5d34700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fe48406b020 0x7fe48406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.308+0000 7fe4a5d34700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fe48406b020 0x7fe48406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe490000f60 tx=0x7fe490007c90 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:32.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.312+0000 7fe4967fc700 1 -- 172.21.15.181:0/3022046422 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe49c055af0 con 0x7fe4a006f1e0 2023-10-16T11:00:32.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.504+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4a00cd5d0 con 0x7fe4a006f1e0 2023-10-16T11:00:32.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.505+0000 7fe4967fc700 1 -- 172.21.15.181:0/3022046422 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe49c020020 con 0x7fe4a006f1e0 2023-10-16T11:00:32.507 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:00:32.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fe48406b020 msgr2=0x7fe48406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fe48406b020 0x7fe48406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe490000f60 tx=0x7fe490007c90 comp rx=0 tx=0).stop 2023-10-16T11:00:32.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a006f1e0 msgr2=0x7fe4a00ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a006f1e0 0x7fe4a00ad6e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0090b0 tx=0x7fe49c004750 comp rx=0 tx=0).stop 2023-10-16T11:00:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 shutdown_connections 2023-10-16T11:00:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fe48406b020 0x7fe48406d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4a00665a0 0x7fe4a0154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4a015b850 0x7fe4a00b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 --2- 172.21.15.181:0/3022046422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a006f1e0 0x7fe4a00ad6e0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 >> 172.21.15.181:0/3022046422 conn(0x7fe4a00cc730 msgr2=0x7fe4a00cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:32.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 shutdown_connections 2023-10-16T11:00:32.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.507+0000 7fe4a7f98700 1 -- 172.21.15.181:0/3022046422 wait complete. 2023-10-16T11:00:32.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-16T11:00:32.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.638+0000 7fede3b1d700 1 Processor -- start 2023-10-16T11:00:32.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.639+0000 7fede3b1d700 1 -- start start 2023-10-16T11:00:32.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.639+0000 7fede3b1d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc15dfd0 0x7feddc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.639+0000 7fede3b1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feddc0af450 0x7feddc06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.639+0000 7fede3b1d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc06f870 con 0x7feddc06db80 2023-10-16T11:00:32.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.639+0000 7fede3b1d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc0d1160 con 0x7feddc0af450 2023-10-16T11:00:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.639+0000 7fede3b1d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0cc8e0 con 0x7feddc15dfd0 2023-10-16T11:00:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feddc0af450 0x7feddc06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feddc0af450 0x7feddc06d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50500/0 (socket says 172.21.15.181:50500) 2023-10-16T11:00:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 -- 172.21.15.181:0/4239646677 learned_addr learned my addr 172.21.15.181:0/4239646677 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:32.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede18b9700 1 --2- 172.21.15.181:0/4239646677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc15dfd0 0x7feddc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 -- 172.21.15.181:0/4239646677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc15dfd0 msgr2=0x7feddc0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:32.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 --2- 172.21.15.181:0/4239646677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc15dfd0 0x7feddc0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 -- 172.21.15.181:0/4239646677 >> v1:172.21.15.190:6789/0 conn(0x7feddc06db80 legacy=0x7feddc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 -- 172.21.15.181:0/4239646677 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0ccbc0 con 0x7feddc0af450 2023-10-16T11:00:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede10b8700 1 --2- 172.21.15.181:0/4239646677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc06d640 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fedd80098f0 tx=0x7fedd8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd7ab82511e2d32 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede08b7700 1 -- 172.21.15.181:0/4239646677 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8011830 con 0x7feddc0af450 2023-10-16T11:00:32.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede08b7700 1 -- 172.21.15.181:0/4239646677 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8011990 con 0x7feddc0af450 2023-10-16T11:00:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.640+0000 7fede08b7700 1 -- 172.21.15.181:0/4239646677 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8011d20 con 0x7feddc0af450 2023-10-16T11:00:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 -- 172.21.15.181:0/4239646677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 msgr2=0x7feddc06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 --2- 172.21.15.181:0/4239646677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc06d640 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fedd80098f0 tx=0x7fedd8009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 -- 172.21.15.181:0/4239646677 shutdown_connections 2023-10-16T11:00:32.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 --2- 172.21.15.181:0/4239646677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc15dfd0 0x7feddc0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 --2- 172.21.15.181:0/4239646677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc06d640 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 -- 172.21.15.181:0/4239646677 >> 172.21.15.181:0/4239646677 conn(0x7feddc0d04a0 msgr2=0x7feddc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 -- 172.21.15.181:0/4239646677 shutdown_connections 2023-10-16T11:00:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.641+0000 7fede3b1d700 1 -- 172.21.15.181:0/4239646677 wait complete. 2023-10-16T11:00:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.642+0000 7fede3b1d700 1 Processor -- start 2023-10-16T11:00:32.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.642+0000 7fede3b1d700 1 -- start start 2023-10-16T11:00:32.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc06db80 0x7feddc0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc15dfd0 0x7feddc0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0b0560 con 0x7feddc0af450 2023-10-16T11:00:32.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1f36a0 con 0x7feddc15dfd0 2023-10-16T11:00:32.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0affa0 con 0x7feddc06db80 2023-10-16T11:00:32.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50508/0 (socket says 172.21.15.181:50508) 2023-10-16T11:00:32.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 -- 172.21.15.181:0/2825586985 learned_addr learned my addr 172.21.15.181:0/2825586985 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:32.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 -- 172.21.15.181:0/2825586985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc06db80 msgr2=0x7feddc0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc06db80 0x7feddc0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 -- 172.21.15.181:0/2825586985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc15dfd0 msgr2=0x7feddc0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:32.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc15dfd0 0x7feddc0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:32.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 -- 172.21.15.181:0/2825586985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8009550 con 0x7feddc0af450 2023-10-16T11:00:32.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede10b8700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc147940 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009c40 tx=0x7fedd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:32.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fedd27fc700 1 -- 172.21.15.181:0/2825586985 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004400 con 0x7feddc0af450 2023-10-16T11:00:32.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fedd27fc700 1 -- 172.21.15.181:0/2825586985 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd801f070 con 0x7feddc0af450 2023-10-16T11:00:32.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc0b0280 con 0x7feddc0af450 2023-10-16T11:00:32.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fedd27fc700 1 -- 172.21.15.181:0/2825586985 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd80038e0 con 0x7feddc0af450 2023-10-16T11:00:32.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.643+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc0c4450 con 0x7feddc0af450 2023-10-16T11:00:32.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.644+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc067ab0 con 0x7feddc0af450 2023-10-16T11:00:32.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.645+0000 7fedd27fc700 1 -- 172.21.15.181:0/2825586985 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fedd8021030 con 0x7feddc0af450 2023-10-16T11:00:32.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.645+0000 7fedd27fc700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fedc006b020 0x7fedc006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:32.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.645+0000 7fedd27fc700 1 -- 172.21.15.181:0/2825586985 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fedd808abd0 con 0x7feddc0af450 2023-10-16T11:00:32.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.645+0000 7fede18b9700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fedc006b020 0x7fedc006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:32.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.646+0000 7fede18b9700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fedc006b020 0x7fedc006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fedcc006ac0 tx=0x7fedcc006a50 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:32.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.650+0000 7fedd27fc700 1 -- 172.21.15.181:0/2825586985 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fedd8055ae0 con 0x7feddc0af450 2023-10-16T11:00:32.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:32.927+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7feddc067d20 con 0x7feddc0af450 2023-10-16T11:00:33.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.825+0000 7fedd27fc700 1 -- 172.21.15.181:0/2825586985 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2487) v1 ==== 177+0+0 (secure 0 0 0) 0x7fedd8015020 con 0x7feddc0af450 2023-10-16T11:00:33.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fedc006b020 msgr2=0x7fedc006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:33.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fedc006b020 0x7fedc006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fedcc006ac0 tx=0x7fedcc006a50 comp rx=0 tx=0).stop 2023-10-16T11:00:33.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 msgr2=0x7feddc147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:33.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc147940 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009c40 tx=0x7fedd8004750 comp rx=0 tx=0).stop 2023-10-16T11:00:33.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 shutdown_connections 2023-10-16T11:00:33.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fedc006b020 0x7fedc006d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feddc06db80 0x7feddc0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feddc15dfd0 0x7feddc0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 --2- 172.21.15.181:0/2825586985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feddc0af450 0x7feddc147940 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.828+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 >> 172.21.15.181:0/2825586985 conn(0x7feddc0d04a0 msgr2=0x7feddc0cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.829+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 shutdown_connections 2023-10-16T11:00:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.829+0000 7fede3b1d700 1 -- 172.21.15.181:0/2825586985 wait complete. 2023-10-16T11:00:33.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:00:33.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.973+0000 7f749d829700 1 Processor -- start 2023-10-16T11:00:33.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.973+0000 7f749d829700 1 -- start start 2023-10-16T11:00:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f749d829700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f7498161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:33.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f749d829700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74980cf890 0x7f74980cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f749d829700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74980ce410 0x7f74980ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f749d829700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74981624b0 con 0x7f74980ce410 2023-10-16T11:00:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f749d829700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980ce880 con 0x7f74980cf890 2023-10-16T11:00:33.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f749d829700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980d0cb0 con 0x7f749815d740 2023-10-16T11:00:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74980ce410 0x7f74980ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74980ce410 0x7f74980ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50518/0 (socket says 172.21.15.181:50518) 2023-10-16T11:00:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 -- 172.21.15.181:0/3487979367 learned_addr learned my addr 172.21.15.181:0/3487979367 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:33.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f7496ffd700 1 --2- 172.21.15.181:0/3487979367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f7498161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 -- 172.21.15.181:0/3487979367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 msgr2=0x7f7498161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 --2- 172.21.15.181:0/3487979367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f7498161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74967fc700 1 --2- 172.21.15.181:0/3487979367 >> v2:172.21.15.190:3300/0 conn(0x7f74980cf890 0x7f74980cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:33.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 -- 172.21.15.181:0/3487979367 >> v2:172.21.15.190:3300/0 conn(0x7f74980cf890 msgr2=0x7f74980cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 --2- 172.21.15.181:0/3487979367 >> v2:172.21.15.190:3300/0 conn(0x7f74980cf890 0x7f74980cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 -- 172.21.15.181:0/3487979367 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980d00a0 con 0x7f74980ce410 2023-10-16T11:00:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74967fc700 1 --2- 172.21.15.181:0/3487979367 >> v2:172.21.15.190:3300/0 conn(0x7f74980cf890 0x7f74980cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.974+0000 7f74977fe700 1 --2- 172.21.15.181:0/3487979367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980ce410 0x7f74980ccf00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7498071760 tx=0x7f749000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe1ea3fd22feca3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:33.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f7495ffb700 1 -- 172.21.15.181:0/3487979367 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000e070 con 0x7f74980ce410 2023-10-16T11:00:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f7495ffb700 1 -- 172.21.15.181:0/3487979367 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749000b980 con 0x7f74980ce410 2023-10-16T11:00:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f7495ffb700 1 -- 172.21.15.181:0/3487979367 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000bd30 con 0x7f74980ce410 2023-10-16T11:00:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 -- 172.21.15.181:0/3487979367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980ce410 msgr2=0x7f74980ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 --2- 172.21.15.181:0/3487979367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980ce410 0x7f74980ccf00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7498071760 tx=0x7f749000b330 comp rx=0 tx=0).stop 2023-10-16T11:00:33.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 -- 172.21.15.181:0/3487979367 shutdown_connections 2023-10-16T11:00:33.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 --2- 172.21.15.181:0/3487979367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980ce410 0x7f74980ccf00 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 --2- 172.21.15.181:0/3487979367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f7498161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 --2- 172.21.15.181:0/3487979367 >> v2:172.21.15.190:3300/0 conn(0x7f74980cf890 0x7f74980cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 -- 172.21.15.181:0/3487979367 >> 172.21.15.181:0/3487979367 conn(0x7f749809a930 msgr2=0x7f74980d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:33.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 -- 172.21.15.181:0/3487979367 shutdown_connections 2023-10-16T11:00:33.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.975+0000 7f749d829700 1 -- 172.21.15.181:0/3487979367 wait complete. 2023-10-16T11:00:33.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.976+0000 7f749d829700 1 Processor -- start 2023-10-16T11:00:33.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f749d829700 1 -- start start 2023-10-16T11:00:33.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f749d829700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74980ce410 0x7f74980b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:33.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f749d829700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980cf890 0x7f7498151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:33.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f749d829700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f74980b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:33.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f749d829700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980b17c0 con 0x7f74980cf890 2023-10-16T11:00:33.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f749d829700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980b1aa0 con 0x7f74980ce410 2023-10-16T11:00:33.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f749d829700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980b1d80 con 0x7f749815d740 2023-10-16T11:00:33.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f74967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980cf890 0x7f7498151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:33.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f74967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980cf890 0x7f7498151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50522/0 (socket says 172.21.15.181:50522) 2023-10-16T11:00:33.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f74967fc700 1 -- 172.21.15.181:0/787740969 learned_addr learned my addr 172.21.15.181:0/787740969 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:33.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f7496ffd700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74980ce410 0x7f74980b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:33.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.977+0000 7f74977fe700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f74980b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:33.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f74967fc700 1 -- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 msgr2=0x7f74980b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:33.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f74967fc700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f74980b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f74967fc700 1 -- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74980ce410 msgr2=0x7f74980b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:33.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f74967fc700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74980ce410 0x7f74980b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:33.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f74967fc700 1 -- 172.21.15.181:0/787740969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7490009b30 con 0x7f74980cf890 2023-10-16T11:00:33.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f74967fc700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980cf890 0x7f7498151bf0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f748800c740 tx=0x7f748800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:33.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f747ffff700 1 -- 172.21.15.181:0/787740969 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488011800 con 0x7f74980cf890 2023-10-16T11:00:33.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f747ffff700 1 -- 172.21.15.181:0/787740969 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7488011960 con 0x7f74980cf890 2023-10-16T11:00:33.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74980b2060 con 0x7f74980cf890 2023-10-16T11:00:33.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f747ffff700 1 -- 172.21.15.181:0/787740969 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488011cd0 con 0x7f74980cf890 2023-10-16T11:00:33.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.978+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74980b14d0 con 0x7f74980cf890 2023-10-16T11:00:33.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.979+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74980b2580 con 0x7f74980cf890 2023-10-16T11:00:33.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.979+0000 7f747ffff700 1 -- 172.21.15.181:0/787740969 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7488021350 con 0x7f74980cf890 2023-10-16T11:00:33.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.980+0000 7f747ffff700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f748406b020 0x7f748406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:33.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.980+0000 7f747ffff700 1 -- 172.21.15.181:0/787740969 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f7488089810 con 0x7f74980cf890 2023-10-16T11:00:33.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.980+0000 7f7496ffd700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f748406b020 0x7f748406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:33.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.980+0000 7f7496ffd700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f748406b020 0x7f748406d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f748c006fd0 tx=0x7f748c008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:33.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:33.984+0000 7f747ffff700 1 -- 172.21.15.181:0/787740969 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f74980b2580 con 0x7f74980cf890 2023-10-16T11:00:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.176+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f74980b2580 con 0x7f74980cf890 2023-10-16T11:00:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.180+0000 7f747ffff700 1 -- 172.21.15.181:0/787740969 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f74980b2580 con 0x7f74980cf890 2023-10-16T11:00:34.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f748406b020 msgr2=0x7f748406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:34.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f748406b020 0x7f748406d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f748c006fd0 tx=0x7f748c008040 comp rx=0 tx=0).stop 2023-10-16T11:00:34.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980cf890 msgr2=0x7f7498151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980cf890 0x7f7498151bf0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f748800c740 tx=0x7f748800ca50 comp rx=0 tx=0).stop 2023-10-16T11:00:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 shutdown_connections 2023-10-16T11:00:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f748406b020 0x7f748406d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f749815d740 0x7f74980b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74980ce410 0x7f74980b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 --2- 172.21.15.181:0/787740969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74980cf890 0x7f7498151bf0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 >> 172.21.15.181:0/787740969 conn(0x7f749809a930 msgr2=0x7f74980ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 shutdown_connections 2023-10-16T11:00:34.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.182+0000 7f749d829700 1 -- 172.21.15.181:0/787740969 wait complete. 2023-10-16T11:00:34.196 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:00:34.196 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:00:34.241 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:00:34.291 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:00:34.291 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:00:34.291 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:34.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:00:34.309 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:00:34.309 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:34.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:00:34.329 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:00:34.329 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:34.366 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:00:34.367 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:00:34.367 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:34.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:00:34.388 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:00:34.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:00:34.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.524+0000 7f526e8cf700 1 Processor -- start 2023-10-16T11:00:34.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.524+0000 7f526e8cf700 1 -- start start 2023-10-16T11:00:34.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f526e8cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f526815b990 0x7f52680ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:34.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f526e8cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cd2d0 0x7f52680d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:34.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f526e8cf700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52680cfc60 0x7f52680d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:34.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f526e8cf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52680d0630 con 0x7f526815b990 2023-10-16T11:00:34.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f526e8cf700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52680d24f0 con 0x7f52680cfc60 2023-10-16T11:00:34.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f526e8cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680cc510 con 0x7f52680cd2d0 2023-10-16T11:00:34.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f5267fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f526815b990 0x7f52680ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:34.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f5267fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f526815b990 0x7f52680ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50528/0 (socket says 172.21.15.181:50528) 2023-10-16T11:00:34.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f5267fff700 1 -- 172.21.15.181:0/3524717504 learned_addr learned my addr 172.21.15.181:0/3524717504 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:34.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f52677fe700 1 --2- 172.21.15.181:0/3524717504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cd2d0 0x7f52680d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:34.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.525+0000 7f526ce6c700 1 --2- 172.21.15.181:0/3524717504 >> v2:172.21.15.190:3300/0 conn(0x7f52680cfc60 0x7f52680d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:34.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5267fff700 1 -- 172.21.15.181:0/3524717504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cd2d0 msgr2=0x7f52680d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:34.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5267fff700 1 --2- 172.21.15.181:0/3524717504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cd2d0 0x7f52680d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5267fff700 1 -- 172.21.15.181:0/3524717504 >> v2:172.21.15.190:3300/0 conn(0x7f52680cfc60 msgr2=0x7f52680d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:34.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5267fff700 1 --2- 172.21.15.181:0/3524717504 >> v2:172.21.15.190:3300/0 conn(0x7f52680cfc60 0x7f52680d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5267fff700 1 -- 172.21.15.181:0/3524717504 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52680cc7f0 con 0x7f526815b990 2023-10-16T11:00:34.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526ce6c700 1 --2- 172.21.15.181:0/3524717504 >> v2:172.21.15.190:3300/0 conn(0x7f52680cfc60 0x7f52680d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:34.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5267fff700 1 --2- 172.21.15.181:0/3524717504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680ce8f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f52680cbc60 tx=0x7f525c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f90a14d25d5a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:34.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5266ffd700 1 -- 172.21.15.181:0/3524717504 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c005e30 con 0x7f526815b990 2023-10-16T11:00:34.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5266ffd700 1 -- 172.21.15.181:0/3524717504 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c004030 con 0x7f526815b990 2023-10-16T11:00:34.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f5266ffd700 1 -- 172.21.15.181:0/3524717504 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0043e0 con 0x7f526815b990 2023-10-16T11:00:34.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526e8cf700 1 -- 172.21.15.181:0/3524717504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 msgr2=0x7f52680ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:34.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526e8cf700 1 --2- 172.21.15.181:0/3524717504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680ce8f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f52680cbc60 tx=0x7f525c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526e8cf700 1 -- 172.21.15.181:0/3524717504 shutdown_connections 2023-10-16T11:00:34.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526e8cf700 1 --2- 172.21.15.181:0/3524717504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680ce8f0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526e8cf700 1 --2- 172.21.15.181:0/3524717504 >> v2:172.21.15.190:3300/0 conn(0x7f52680cfc60 0x7f52680d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526e8cf700 1 --2- 172.21.15.181:0/3524717504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cd2d0 0x7f52680d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.526+0000 7f526e8cf700 1 -- 172.21.15.181:0/3524717504 >> 172.21.15.181:0/3524717504 conn(0x7f526809a9a0 msgr2=0x7f52680d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:34.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.527+0000 7f526e8cf700 1 -- 172.21.15.181:0/3524717504 shutdown_connections 2023-10-16T11:00:34.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.527+0000 7f526e8cf700 1 -- 172.21.15.181:0/3524717504 wait complete. 2023-10-16T11:00:34.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.528+0000 7f526e8cf700 1 Processor -- start 2023-10-16T11:00:34.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.528+0000 7f526e8cf700 1 -- start start 2023-10-16T11:00:34.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52680cd2d0 0x7f526815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:34.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cfc60 0x7f52680cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:34.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:34.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680cba30 con 0x7f526815b990 2023-10-16T11:00:34.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681f5330 con 0x7f52680cd2d0 2023-10-16T11:00:34.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52681f4720 con 0x7f52680cfc60 2023-10-16T11:00:34.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:34.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50536/0 (socket says 172.21.15.181:50536) 2023-10-16T11:00:34.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 -- 172.21.15.181:0/1980653133 learned_addr learned my addr 172.21.15.181:0/1980653133 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:34.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f5267fff700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52680cd2d0 0x7f526815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:34.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f52677fe700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cfc60 0x7f52680cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:34.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 -- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cfc60 msgr2=0x7f52680cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:34.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cfc60 0x7f52680cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 -- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52680cd2d0 msgr2=0x7f526815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:34.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52680cd2d0 0x7f526815eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 -- 172.21.15.181:0/1980653133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c009500 con 0x7f526815b990 2023-10-16T11:00:34.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526ce6c700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680cb4f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5260007e70 tx=0x7f526000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:34.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f5264ff9700 1 -- 172.21.15.181:0/1980653133 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000dd20 con 0x7f526815b990 2023-10-16T11:00:34.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f5264ff9700 1 -- 172.21.15.181:0/1980653133 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000f040 con 0x7f526815b990 2023-10-16T11:00:34.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52681f4a00 con 0x7f526815b990 2023-10-16T11:00:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f5264ff9700 1 -- 172.21.15.181:0/1980653133 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000dea0 con 0x7f526815b990 2023-10-16T11:00:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.529+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52681f4fc0 con 0x7f526815b990 2023-10-16T11:00:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.530+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5268067d20 con 0x7f526815b990 2023-10-16T11:00:34.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.531+0000 7f5264ff9700 1 -- 172.21.15.181:0/1980653133 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5260003c60 con 0x7f526815b990 2023-10-16T11:00:34.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.531+0000 7f5264ff9700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f525406b020 0x7f525406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:34.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.531+0000 7f5264ff9700 1 -- 172.21.15.181:0/1980653133 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f5260089a30 con 0x7f526815b990 2023-10-16T11:00:34.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.531+0000 7f5267fff700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f525406b020 0x7f525406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:34.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.532+0000 7f5267fff700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f525406b020 0x7f525406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f52680cbc60 tx=0x7f525c008070 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:34.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.536+0000 7f5264ff9700 1 -- 172.21.15.181:0/1980653133 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5260054670 con 0x7f526815b990 2023-10-16T11:00:34.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.775+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52681f5470 con 0x7f526815b990 2023-10-16T11:00:34.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.775+0000 7f5264ff9700 1 -- 172.21.15.181:0/1980653133 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f5260057c90 con 0x7f526815b990 2023-10-16T11:00:34.778 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:34.778 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7349,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1229456263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1229456263},{"type":"v1","addr":"172.21.15.190:6834","nonce":1229456263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8134,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3176781358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3176781358},{"type":"v1","addr":"172.21.15.181:6836","nonce":3176781358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":8137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2996369280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2996369280},{"type":"v1","addr":"172.21.15.181:6837","nonce":2996369280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/300920705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":300920705},{"type":"v1","addr":"172.21.15.190:6835","nonce":300920705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[]} 2023-10-16T11:00:34.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f525406b020 msgr2=0x7f525406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:34.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f525406b020 0x7f525406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f52680cbc60 tx=0x7f525c008070 comp rx=0 tx=0).stop 2023-10-16T11:00:34.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 msgr2=0x7f52680cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:34.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680cb4f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5260007e70 tx=0x7f526000d3b0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 shutdown_connections 2023-10-16T11:00:34.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f525406b020 0x7f525406d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526815b990 0x7f52680cb4f0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52680cd2d0 0x7f526815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 --2- 172.21.15.181:0/1980653133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52680cfc60 0x7f52680cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:34.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 >> 172.21.15.181:0/1980653133 conn(0x7f526809a9a0 msgr2=0x7f52680afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:34.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 shutdown_connections 2023-10-16T11:00:34.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:34.778+0000 7f526e8cf700 1 -- 172.21.15.181:0/1980653133 wait complete. 2023-10-16T11:00:34.782 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 78 2023-10-16T11:00:34.792 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:00:34.792 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:00:34.792 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:00:34.792 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:00:34.793 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:00:34.793 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:00:34.793 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:00:34.794 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:00:34.794 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:00:34.794 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:00:34.794 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:00:34.795 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:00:34.795 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:00:34.795 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:00:34.796 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:00:34.796 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:00:34.801 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:00:34.802 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:00:34.803 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:00:34.803 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:00:34.807 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:00:34.807 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:00:34.843 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:00:34.844 DEBUG:teuthology.parallel:result is None 2023-10-16T11:00:34.844 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:00:34.845 DEBUG:teuthology.parallel:result is None 2023-10-16T11:00:34.849 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:00:34.850 DEBUG:teuthology.parallel:result is None 2023-10-16T11:00:34.921 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:00:34.922 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:00:34.933 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:00:38.787 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-16T11:00:40.806 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:00:40.807 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:00:40.810 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:00:40.810 DEBUG:teuthology.parallel:result is None 2023-10-16T11:00:40.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:00:40.900 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.947+0000 7f4bc6db4700 1 Processor -- start 2023-10-16T11:00:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.947+0000 7f4bc6db4700 1 -- start start 2023-10-16T11:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.947+0000 7f4bc6db4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc015b9f0 0x7f4bc00cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.947+0000 7f4bc6db4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bc00cc400 0x7f4bc00cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:40.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.947+0000 7f4bc6db4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc006b370 con 0x7f4bc00cf980 2023-10-16T11:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.947+0000 7f4bc6db4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc006b800 con 0x7f4bc00cc400 2023-10-16T11:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.947+0000 7f4bc6db4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00cfdc0 con 0x7f4bc015b9f0 2023-10-16T11:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bc00cc400 0x7f4bc00cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bc5351700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4bc00cf980 0x7f4bc00ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33418/0 (socket says 172.21.15.181:33418) 2023-10-16T11:00:40.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bc5351700 1 -- 172.21.15.181:0/798416103 learned_addr learned my addr 172.21.15.181:0/798416103 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bc4b50700 1 --2- 172.21.15.181:0/798416103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc015b9f0 0x7f4bc00cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbffff700 1 -- 172.21.15.181:0/798416103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc015b9f0 msgr2=0x7f4bc00cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbffff700 1 --2- 172.21.15.181:0/798416103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc015b9f0 0x7f4bc00cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbffff700 1 -- 172.21.15.181:0/798416103 >> v1:172.21.15.181:6789/0 conn(0x7f4bc00cf980 legacy=0x7f4bc00ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbffff700 1 -- 172.21.15.181:0/798416103 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc006aa40 con 0x7f4bc00cc400 2023-10-16T11:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbf7fe700 1 -- 172.21.15.181:0/798416103 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062918017 0 0) 0x7f4bc006b370 con 0x7f4bc00cf980 2023-10-16T11:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbffff700 1 --2- 172.21.15.181:0/798416103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cc400 0x7f4bc00cf440 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009840 tx=0x7f4bac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c8e61ff7d646eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbf7fe700 1 -- 172.21.15.181:0/798416103 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac011830 con 0x7f4bc00cc400 2023-10-16T11:00:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbf7fe700 1 -- 172.21.15.181:0/798416103 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac011990 con 0x7f4bc00cc400 2023-10-16T11:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.948+0000 7f4bbf7fe700 1 -- 172.21.15.181:0/798416103 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac011d20 con 0x7f4bc00cc400 2023-10-16T11:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 -- 172.21.15.181:0/798416103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cc400 msgr2=0x7f4bc00cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/798416103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cc400 0x7f4bc00cf440 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009840 tx=0x7f4bac009b50 comp rx=0 tx=0).stop 2023-10-16T11:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 -- 172.21.15.181:0/798416103 shutdown_connections 2023-10-16T11:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/798416103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc015b9f0 0x7f4bc00cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/798416103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cc400 0x7f4bc00cf440 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 -- 172.21.15.181:0/798416103 >> 172.21.15.181:0/798416103 conn(0x7f4bc00d04a0 msgr2=0x7f4bc00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 -- 172.21.15.181:0/798416103 shutdown_connections 2023-10-16T11:00:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.949+0000 7f4bc6db4700 1 -- 172.21.15.181:0/798416103 wait complete. 2023-10-16T11:00:40.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.950+0000 7f4bc6db4700 1 Processor -- start 2023-10-16T11:00:40.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.950+0000 7f4bc6db4700 1 -- start start 2023-10-16T11:00:40.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc6db4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc00cc400 0x7f4bc00b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:40.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc6db4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cf980 0x7f4bc01e4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:40.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc6db4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc015b9f0 0x7f4bc00b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc6db4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b63c0 con 0x7f4bc00cf980 2023-10-16T11:00:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc6db4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b4da0 con 0x7f4bc015b9f0 2023-10-16T11:00:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc6db4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc014af00 con 0x7f4bc00cc400 2023-10-16T11:00:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cf980 0x7f4bc01e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:40.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cf980 0x7f4bc01e4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50576/0 (socket says 172.21.15.181:50576) 2023-10-16T11:00:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 -- 172.21.15.181:0/4241261373 learned_addr learned my addr 172.21.15.181:0/4241261373 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc5351700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc015b9f0 0x7f4bc00b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bc4b50700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc00cc400 0x7f4bc00b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 -- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc00cc400 msgr2=0x7f4bc00b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:40.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc00cc400 0x7f4bc00b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 -- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc015b9f0 msgr2=0x7f4bc00b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc015b9f0 0x7f4bc00b5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.951+0000 7f4bbffff700 1 -- 172.21.15.181:0/4241261373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac0094a0 con 0x7f4bc00cf980 2023-10-16T11:00:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.952+0000 7f4bbffff700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cf980 0x7f4bc01e4230 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0090b0 tx=0x7f4bac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:40.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.952+0000 7f4bbd7fa700 1 -- 172.21.15.181:0/4241261373 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0036a0 con 0x7f4bc00cf980 2023-10-16T11:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.952+0000 7f4bbd7fa700 1 -- 172.21.15.181:0/4241261373 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac01f070 con 0x7f4bc00cf980 2023-10-16T11:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.952+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc00b4470 con 0x7f4bc00cf980 2023-10-16T11:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.952+0000 7f4bbd7fa700 1 -- 172.21.15.181:0/4241261373 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac003a60 con 0x7f4bc00cf980 2023-10-16T11:00:40.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.952+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00b4a30 con 0x7f4bc00cf980 2023-10-16T11:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.953+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0067d20 con 0x7f4bc00cf980 2023-10-16T11:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.953+0000 7f4bbd7fa700 1 -- 172.21.15.181:0/4241261373 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4bac003bc0 con 0x7f4bc00cf980 2023-10-16T11:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.954+0000 7f4bbd7fa700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.954+0000 7f4bbd7fa700 1 -- 172.21.15.181:0/4241261373 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f4bac021070 con 0x7f4bc00cf980 2023-10-16T11:00:40.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.954+0000 7f4bc4b50700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.954+0000 7f4bc4b50700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4ba406b020 0x7f4ba406d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006fd0 tx=0x7f4bb4008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:40.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:40.958+0000 7f4bbd7fa700 1 -- 172.21.15.181:0/4241261373 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4bac055f80 con 0x7f4bc00cf980 2023-10-16T11:00:41.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:41.163+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4bc00b4ee0 con 0x7f4bc00cf980 2023-10-16T11:00:42.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.851+0000 7f4bbd7fa700 1 -- 172.21.15.181:0/4241261373 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v164) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4bac015080 con 0x7f4bc00cf980 2023-10-16T11:00:42.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4ba406b020 msgr2=0x7f4ba406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:42.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4ba406b020 0x7f4ba406d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006fd0 tx=0x7f4bb4008040 comp rx=0 tx=0).stop 2023-10-16T11:00:42.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cf980 msgr2=0x7f4bc01e4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:42.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cf980 0x7f4bc01e4230 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0090b0 tx=0x7f4bac004750 comp rx=0 tx=0).stop 2023-10-16T11:00:42.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 shutdown_connections 2023-10-16T11:00:42.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:42.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc00cc400 0x7f4bc00b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:42.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc015b9f0 0x7f4bc00b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:42.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 --2- 172.21.15.181:0/4241261373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00cf980 0x7f4bc01e4230 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:42.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 >> 172.21.15.181:0/4241261373 conn(0x7f4bc00d04a0 msgr2=0x7f4bc015a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:42.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 shutdown_connections 2023-10-16T11:00:42.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.854+0000 7f4bc6db4700 1 -- 172.21.15.181:0/4241261373 wait complete. 2023-10-16T11:00:42.859 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T11:00:42.871 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:00:42.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.997+0000 7f485fd16700 1 Processor -- start 2023-10-16T11:00:42.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.997+0000 7f485fd16700 1 -- start start 2023-10-16T11:00:43.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485fd16700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f485815b990 0x7f48580ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485fd16700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 0x7f48580d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485fd16700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48580cfc60 0x7f48580d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485fd16700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48580d0630 con 0x7f48580cfc60 2023-10-16T11:00:43.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485fd16700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48580d24f0 con 0x7f485815b990 2023-10-16T11:00:43.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485fd16700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580cc510 con 0x7f48580cd2d0 2023-10-16T11:00:43.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485e2b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48580cfc60 0x7f48580d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485e2b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48580cfc60 0x7f48580d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50584/0 (socket says 172.21.15.181:50584) 2023-10-16T11:00:43.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.998+0000 7f485e2b3700 1 -- 172.21.15.181:0/1932140240 learned_addr learned my addr 172.21.15.181:0/1932140240 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:43.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485e2b3700 1 -- 172.21.15.181:0/1932140240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 msgr2=0x7f48580d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:43.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485e2b3700 1 --2- 172.21.15.181:0/1932140240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 0x7f48580d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485e2b3700 1 -- 172.21.15.181:0/1932140240 >> v2:172.21.15.190:3300/0 conn(0x7f485815b990 msgr2=0x7f48580ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:43.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485e2b3700 1 --2- 172.21.15.181:0/1932140240 >> v2:172.21.15.190:3300/0 conn(0x7f485815b990 0x7f48580ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485e2b3700 1 -- 172.21.15.181:0/1932140240 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48580cc7f0 con 0x7f48580cfc60 2023-10-16T11:00:43.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485e2b3700 1 --2- 172.21.15.181:0/1932140240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f48580d00f0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4858073000 tx=0x7f485000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee94379f4f527977 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485cab0700 1 -- 172.21.15.181:0/1932140240 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000e070 con 0x7f48580cfc60 2023-10-16T11:00:43.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485cab0700 1 -- 172.21.15.181:0/1932140240 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485000b9c0 con 0x7f48580cfc60 2023-10-16T11:00:43.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485cab0700 1 -- 172.21.15.181:0/1932140240 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000bd70 con 0x7f48580cfc60 2023-10-16T11:00:43.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 -- 172.21.15.181:0/1932140240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 msgr2=0x7f48580d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 --2- 172.21.15.181:0/1932140240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f48580d00f0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4858073000 tx=0x7f485000b330 comp rx=0 tx=0).stop 2023-10-16T11:00:43.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 -- 172.21.15.181:0/1932140240 shutdown_connections 2023-10-16T11:00:43.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 --2- 172.21.15.181:0/1932140240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f48580d00f0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 --2- 172.21.15.181:0/1932140240 >> v2:172.21.15.190:3300/0 conn(0x7f485815b990 0x7f48580ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 --2- 172.21.15.181:0/1932140240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 0x7f48580d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 -- 172.21.15.181:0/1932140240 >> 172.21.15.181:0/1932140240 conn(0x7f485809a9a0 msgr2=0x7f48580d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 -- 172.21.15.181:0/1932140240 shutdown_connections 2023-10-16T11:00:43.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:42.999+0000 7f485fd16700 1 -- 172.21.15.181:0/1932140240 wait complete. 2023-10-16T11:00:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.001+0000 7f485fd16700 1 Processor -- start 2023-10-16T11:00:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.001+0000 7f485fd16700 1 -- start start 2023-10-16T11:00:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485fd16700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 0x7f48580c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485fd16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f4858159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485fd16700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485815b990 0x7f48581598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485fd16700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48580b2650 con 0x7f48580cfc60 2023-10-16T11:00:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485fd16700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48580b1d20 con 0x7f485815b990 2023-10-16T11:00:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485fd16700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580b2000 con 0x7f48580cd2d0 2023-10-16T11:00:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f4858159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f4858159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50588/0 (socket says 172.21.15.181:50588) 2023-10-16T11:00:43.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 -- 172.21.15.181:0/2276285815 learned_addr learned my addr 172.21.15.181:0/2276285815 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485dab2700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 0x7f48580c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485e2b3700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485815b990 0x7f48581598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 -- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 msgr2=0x7f48580c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 0x7f48580c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 -- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485815b990 msgr2=0x7f48581598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485815b990 0x7f48581598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485d2b1700 1 -- 172.21.15.181:0/2276285815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4850009b30 con 0x7f48580cfc60 2023-10-16T11:00:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.002+0000 7f485e2b3700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485815b990 0x7f48581598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.003+0000 7f485d2b1700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f4858159380 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f485400b4a0 tx=0x7f485400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.003+0000 7f484e7fc700 1 -- 172.21.15.181:0/2276285815 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854011830 con 0x7f48580cfc60 2023-10-16T11:00:43.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.003+0000 7f484e7fc700 1 -- 172.21.15.181:0/2276285815 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854011990 con 0x7f48580cfc60 2023-10-16T11:00:43.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.003+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48580b22e0 con 0x7f48580cfc60 2023-10-16T11:00:43.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.003+0000 7f484e7fc700 1 -- 172.21.15.181:0/2276285815 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854011d40 con 0x7f48580cfc60 2023-10-16T11:00:43.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.003+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48580b1940 con 0x7f48580cfc60 2023-10-16T11:00:43.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.004+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48580b2800 con 0x7f48580cfc60 2023-10-16T11:00:43.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.004+0000 7f484e7fc700 1 -- 172.21.15.181:0/2276285815 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f485401a400 con 0x7f48580cfc60 2023-10-16T11:00:43.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.004+0000 7f484e7fc700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f483c06b020 0x7f483c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.004+0000 7f484e7fc700 1 -- 172.21.15.181:0/2276285815 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f4854089560 con 0x7f48580cfc60 2023-10-16T11:00:43.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.004+0000 7f485dab2700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f483c06b020 0x7f483c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.005+0000 7f485dab2700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f483c06b020 0x7f483c06d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48580c8370 tx=0x7f48480090e0 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.009+0000 7f484e7fc700 1 -- 172.21.15.181:0/2276285815 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f48580b2800 con 0x7f48580cfc60 2023-10-16T11:00:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.280+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4858067d20 con 0x7f48580cfc60 2023-10-16T11:00:43.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.280+0000 7f484e7fc700 1 -- 172.21.15.181:0/2276285815 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v41) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4854054670 con 0x7f48580cfc60 2023-10-16T11:00:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:43.283 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:00:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T11:00:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T11:00:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T11:00:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:00:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:00:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T11:00:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T11:00:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:00:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:00:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T11:00:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T11:00:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:00:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:00:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T11:00:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T11:00:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:00:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:00:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T11:00:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T11:00:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T11:00:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T11:00:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T11:00:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T11:00:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T11:00:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T11:00:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T11:00:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T11:00:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T11:00:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T11:00:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T11:00:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T11:00:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T11:00:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T11:00:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:00:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:00:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T11:00:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T11:00:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T11:00:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:00:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T11:00:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T11:00:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:00:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:00:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:00:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T11:00:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T11:00:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:00:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:00:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:00:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T11:00:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T11:00:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:00:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:00:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:00:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:00:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T11:00:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T11:00:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:00:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:00:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:00:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T11:00:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T11:00:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:00:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:00:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:00:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T11:00:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T11:00:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:00:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:00:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:00:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:00:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T11:00:43.316 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:00:43.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f483c06b020 msgr2=0x7f483c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f483c06b020 0x7f483c06d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48580c8370 tx=0x7f48480090e0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 msgr2=0x7f4858159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f4858159380 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f485400b4a0 tx=0x7f485400b7b0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 shutdown_connections 2023-10-16T11:00:43.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f483c06b020 0x7f483c06d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48580cd2d0 0x7f48580c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485815b990 0x7f48581598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 --2- 172.21.15.181:0/2276285815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48580cfc60 0x7f4858159380 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 >> 172.21.15.181:0/2276285815 conn(0x7f485809a9a0 msgr2=0x7f48580ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:43.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 shutdown_connections 2023-10-16T11:00:43.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.283+0000 7f485fd16700 1 -- 172.21.15.181:0/2276285815 wait complete. 2023-10-16T11:00:43.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:43.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.440+0000 7f2189ba7700 1 Processor -- start 2023-10-16T11:00:43.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.440+0000 7f2189ba7700 1 -- start start 2023-10-16T11:00:43.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2189ba7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218415dfd0 0x7f21840af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2189ba7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f21840af450 0x7f218406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2189ba7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f218406db80 0x7f21840cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2189ba7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218406f870 con 0x7f218406db80 2023-10-16T11:00:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2189ba7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21840d1160 con 0x7f21840af450 2023-10-16T11:00:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2189ba7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21840cc8e0 con 0x7f218415dfd0 2023-10-16T11:00:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2183fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f218406db80 0x7f21840cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2183fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f218406db80 0x7f21840cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50598/0 (socket says 172.21.15.181:50598) 2023-10-16T11:00:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.441+0000 7f2183fff700 1 -- 172.21.15.181:0/4271044434 learned_addr learned my addr 172.21.15.181:0/4271044434 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2183fff700 1 -- 172.21.15.181:0/4271044434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218415dfd0 msgr2=0x7f21840af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f21837fe700 1 --2- 172.21.15.181:0/4271044434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218415dfd0 0x7f21840af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2182ffd700 1 --2- 172.21.15.181:0/4271044434 >> v2:172.21.15.190:3300/0 conn(0x7f21840af450 0x7f218406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2183fff700 1 --2- 172.21.15.181:0/4271044434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218415dfd0 0x7f21840af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2183fff700 1 -- 172.21.15.181:0/4271044434 >> v2:172.21.15.190:3300/0 conn(0x7f21840af450 msgr2=0x7f218406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2183fff700 1 --2- 172.21.15.181:0/4271044434 >> v2:172.21.15.190:3300/0 conn(0x7f21840af450 0x7f218406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2183fff700 1 -- 172.21.15.181:0/4271044434 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21840ccbc0 con 0x7f218406db80 2023-10-16T11:00:43.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f21837fe700 1 --2- 172.21.15.181:0/4271044434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218415dfd0 0x7f21840af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:43.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2182ffd700 1 --2- 172.21.15.181:0/4271044434 >> v2:172.21.15.190:3300/0 conn(0x7f21840af450 0x7f218406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:43.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2183fff700 1 --2- 172.21.15.181:0/4271044434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218406db80 0x7f21840cd470 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f21840705e0 tx=0x7f217c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98632fee4e354e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f21827fc700 1 -- 172.21.15.181:0/4271044434 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00e070 con 0x7f218406db80 2023-10-16T11:00:43.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f21827fc700 1 -- 172.21.15.181:0/4271044434 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c00b9c0 con 0x7f218406db80 2023-10-16T11:00:43.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f21827fc700 1 -- 172.21.15.181:0/4271044434 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00bd70 con 0x7f218406db80 2023-10-16T11:00:43.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2189ba7700 1 -- 172.21.15.181:0/4271044434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218406db80 msgr2=0x7f21840cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.442+0000 7f2189ba7700 1 --2- 172.21.15.181:0/4271044434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218406db80 0x7f21840cd470 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f21840705e0 tx=0x7f217c00b330 comp rx=0 tx=0).stop 2023-10-16T11:00:43.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.443+0000 7f2189ba7700 1 -- 172.21.15.181:0/4271044434 shutdown_connections 2023-10-16T11:00:43.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.443+0000 7f2189ba7700 1 --2- 172.21.15.181:0/4271044434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218406db80 0x7f21840cd470 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.443+0000 7f2189ba7700 1 --2- 172.21.15.181:0/4271044434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218415dfd0 0x7f21840af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.443+0000 7f2189ba7700 1 --2- 172.21.15.181:0/4271044434 >> v2:172.21.15.190:3300/0 conn(0x7f21840af450 0x7f218406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.443+0000 7f2189ba7700 1 -- 172.21.15.181:0/4271044434 >> 172.21.15.181:0/4271044434 conn(0x7f21840d04a0 msgr2=0x7f21840d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:43.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.443+0000 7f2189ba7700 1 -- 172.21.15.181:0/4271044434 shutdown_connections 2023-10-16T11:00:43.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.443+0000 7f2189ba7700 1 -- 172.21.15.181:0/4271044434 wait complete. 2023-10-16T11:00:43.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.444+0000 7f2189ba7700 1 Processor -- start 2023-10-16T11:00:43.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.444+0000 7f2189ba7700 1 -- start start 2023-10-16T11:00:43.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2189ba7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218406db80 0x7f2184157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2189ba7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840af450 0x7f21840b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2189ba7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f218415dfd0 0x7f21840b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2189ba7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218415f460 con 0x7f21840af450 2023-10-16T11:00:43.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2189ba7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218415b370 con 0x7f218415dfd0 2023-10-16T11:00:43.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2189ba7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218415ee50 con 0x7f218406db80 2023-10-16T11:00:43.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840af450 0x7f21840b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840af450 0x7f21840b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50602/0 (socket says 172.21.15.181:50602) 2023-10-16T11:00:43.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 -- 172.21.15.181:0/1794722876 learned_addr learned my addr 172.21.15.181:0/1794722876 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:43.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f21837fe700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218406db80 0x7f2184157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2183fff700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f218415dfd0 0x7f21840b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 -- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218406db80 msgr2=0x7f2184157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218406db80 0x7f2184157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 -- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f218415dfd0 msgr2=0x7f21840b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f218415dfd0 0x7f21840b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 -- 172.21.15.181:0/1794722876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c009b30 con 0x7f21840af450 2023-10-16T11:00:43.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2182ffd700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840af450 0x7f21840b54c0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f217400c740 tx=0x7f217400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f216bfff700 1 -- 172.21.15.181:0/1794722876 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174011800 con 0x7f21840af450 2023-10-16T11:00:43.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f216bfff700 1 -- 172.21.15.181:0/1794722876 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174011960 con 0x7f21840af450 2023-10-16T11:00:43.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218415b610 con 0x7f21840af450 2023-10-16T11:00:43.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.445+0000 7f216bfff700 1 -- 172.21.15.181:0/1794722876 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174011cd0 con 0x7f21840af450 2023-10-16T11:00:43.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.446+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2184157f80 con 0x7f21840af450 2023-10-16T11:00:43.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.446+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2184067d20 con 0x7f21840af450 2023-10-16T11:00:43.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.447+0000 7f216bfff700 1 -- 172.21.15.181:0/1794722876 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2174021350 con 0x7f21840af450 2023-10-16T11:00:43.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.447+0000 7f216bfff700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f217006b020 0x7f217006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.447+0000 7f216bfff700 1 -- 172.21.15.181:0/1794722876 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f21740894e0 con 0x7f21840af450 2023-10-16T11:00:43.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.447+0000 7f21837fe700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f217006b020 0x7f217006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.448+0000 7f21837fe700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f217006b020 0x7f217006d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21841f61b0 tx=0x7f2178008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.452+0000 7f216bfff700 1 -- 172.21.15.181:0/1794722876 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21740544d0 con 0x7f21840af450 2023-10-16T11:00:43.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.644+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f218406cc30 con 0x7f21840af450 2023-10-16T11:00:43.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.645+0000 7f216bfff700 1 -- 172.21.15.181:0/1794722876 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+13663 (secure 0 0 0) 0x7f2174057af0 con 0x7f21840af450 2023-10-16T11:00:43.647 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:43.647 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":164,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:42.850886+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:43.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f217006b020 msgr2=0x7f217006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f217006b020 0x7f217006d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21841f61b0 tx=0x7f2178008040 comp rx=0 tx=0).stop 2023-10-16T11:00:43.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840af450 msgr2=0x7f21840b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840af450 0x7f21840b54c0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f217400c740 tx=0x7f217400ca50 comp rx=0 tx=0).stop 2023-10-16T11:00:43.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 shutdown_connections 2023-10-16T11:00:43.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f217006b020 0x7f217006d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f218406db80 0x7f2184157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f218415dfd0 0x7f21840b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 --2- 172.21.15.181:0/1794722876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840af450 0x7f21840b54c0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.647+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 >> 172.21.15.181:0/1794722876 conn(0x7f21840d04a0 msgr2=0x7f218415c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:43.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.648+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 shutdown_connections 2023-10-16T11:00:43.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.648+0000 7f2189ba7700 1 -- 172.21.15.181:0/1794722876 wait complete. 2023-10-16T11:00:43.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:00:43.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:00:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.783+0000 7fadcde83700 1 Processor -- start 2023-10-16T11:00:43.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.783+0000 7fadcde83700 1 -- start start 2023-10-16T11:00:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadcde83700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadc815da70 0x7fadc80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadcde83700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc80ce6a0 0x7fadc806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadcde83700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc80cef00 con 0x7fadc8072760 2023-10-16T11:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadcde83700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc80cec20 con 0x7fadc815da70 2023-10-16T11:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadcde83700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8071c80 con 0x7fadc80ce6a0 2023-10-16T11:00:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadc815da70 0x7fadc80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadc815da70 0x7fadc80ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50608/0 (socket says 172.21.15.181:50608) 2023-10-16T11:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc77fe700 1 -- 172.21.15.181:0/3219545945 learned_addr learned my addr 172.21.15.181:0/3219545945 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc6ffd700 1 --2- 172.21.15.181:0/3219545945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc80ce6a0 0x7fadc806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc77fe700 1 -- 172.21.15.181:0/3219545945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc80ce6a0 msgr2=0x7fadc806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc77fe700 1 --2- 172.21.15.181:0/3219545945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc80ce6a0 0x7fadc806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc77fe700 1 -- 172.21.15.181:0/3219545945 >> v1:172.21.15.190:6789/0 conn(0x7fadc8072760 legacy=0x7fadc81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.784+0000 7fadc77fe700 1 -- 172.21.15.181:0/3219545945 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8071f60 con 0x7fadc815da70 2023-10-16T11:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadc77fe700 1 --2- 172.21.15.181:0/3219545945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc815da70 0x7fadc80ce160 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098f0 tx=0x7fadbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc07342c0a06ebd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadc67fc700 1 -- 172.21.15.181:0/3219545945 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc004230 con 0x7fadc815da70 2023-10-16T11:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadc67fc700 1 -- 172.21.15.181:0/3219545945 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc004390 con 0x7fadc815da70 2023-10-16T11:00:43.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadc67fc700 1 -- 172.21.15.181:0/3219545945 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0036a0 con 0x7fadc815da70 2023-10-16T11:00:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 -- 172.21.15.181:0/3219545945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc815da70 msgr2=0x7fadc80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 --2- 172.21.15.181:0/3219545945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc815da70 0x7fadc80ce160 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098f0 tx=0x7fadbc009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 -- 172.21.15.181:0/3219545945 shutdown_connections 2023-10-16T11:00:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 --2- 172.21.15.181:0/3219545945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc815da70 0x7fadc80ce160 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 --2- 172.21.15.181:0/3219545945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc80ce6a0 0x7fadc806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 -- 172.21.15.181:0/3219545945 >> 172.21.15.181:0/3219545945 conn(0x7fadc80cd010 msgr2=0x7fadc80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:43.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 -- 172.21.15.181:0/3219545945 shutdown_connections 2023-10-16T11:00:43.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.785+0000 7fadcde83700 1 -- 172.21.15.181:0/3219545945 wait complete. 2023-10-16T11:00:43.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.787+0000 7fadcde83700 1 Processor -- start 2023-10-16T11:00:43.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.787+0000 7fadcde83700 1 -- start start 2023-10-16T11:00:43.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadcde83700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8072760 0x7fadc8151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadcde83700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80ce6a0 0x7fadc80c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadcde83700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc815da70 0x7fadc80ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadcde83700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80b3470 con 0x7fadc80ce6a0 2023-10-16T11:00:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadcde83700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8151ff0 con 0x7fadc815da70 2023-10-16T11:00:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadcde83700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80b49d0 con 0x7fadc8072760 2023-10-16T11:00:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80ce6a0 0x7fadc80c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80ce6a0 0x7fadc80c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50614/0 (socket says 172.21.15.181:50614) 2023-10-16T11:00:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 -- 172.21.15.181:0/3826341662 learned_addr learned my addr 172.21.15.181:0/3826341662 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc77fe700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8072760 0x7fadc8151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc7fff700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc815da70 0x7fadc80ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 -- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8072760 msgr2=0x7fadc8151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8072760 0x7fadc8151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 -- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc815da70 msgr2=0x7fadc80ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc815da70 0x7fadc80ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc6ffd700 1 -- 172.21.15.181:0/3826341662 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc009550 con 0x7fadc80ce6a0 2023-10-16T11:00:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc7fff700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc815da70 0x7fadc80ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.788+0000 7fadc77fe700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8072760 0x7fadc8151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:43.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.789+0000 7fadc6ffd700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80ce6a0 0x7fadc80c9070 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d800 tx=0x7fadb800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.789+0000 7fadaffff700 1 -- 172.21.15.181:0/3826341662 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb80125f0 con 0x7fadc80ce6a0 2023-10-16T11:00:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.789+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc80b4c70 con 0x7fadc80ce6a0 2023-10-16T11:00:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.789+0000 7fadaffff700 1 -- 172.21.15.181:0/3826341662 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8012750 con 0x7fadc80ce6a0 2023-10-16T11:00:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.789+0000 7fadaffff700 1 -- 172.21.15.181:0/3826341662 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8012b00 con 0x7fadc80ce6a0 2023-10-16T11:00:43.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.789+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc80b5230 con 0x7fadc80ce6a0 2023-10-16T11:00:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.790+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc806f6c0 con 0x7fadc80ce6a0 2023-10-16T11:00:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.790+0000 7fadaffff700 1 -- 172.21.15.181:0/3826341662 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fadb8012c60 con 0x7fadc80ce6a0 2023-10-16T11:00:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.791+0000 7fadaffff700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fadb406b050 0x7fadb406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.791+0000 7fadaffff700 1 -- 172.21.15.181:0/3826341662 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fadb8089a70 con 0x7fadc80ce6a0 2023-10-16T11:00:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.791+0000 7fadc77fe700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fadb406b050 0x7fadb406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:43.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.791+0000 7fadc77fe700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fadb406b050 0x7fadb406d510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009130 tx=0x7fadbc008030 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:43.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.795+0000 7fadaffff700 1 -- 172.21.15.181:0/3826341662 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fadb8054af0 con 0x7fadc80ce6a0 2023-10-16T11:00:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.986+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadc80cd5d0 con 0x7fadc80ce6a0 2023-10-16T11:00:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.986+0000 7fadaffff700 1 -- 172.21.15.181:0/3826341662 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadb8058110 con 0x7fadc80ce6a0 2023-10-16T11:00:43.989 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:00:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fadb406b050 msgr2=0x7fadb406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fadb406b050 0x7fadb406d510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009130 tx=0x7fadbc008030 comp rx=0 tx=0).stop 2023-10-16T11:00:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80ce6a0 msgr2=0x7fadc80c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80ce6a0 0x7fadc80c9070 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fadb800d800 tx=0x7fadb800dbc0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 shutdown_connections 2023-10-16T11:00:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fadb406b050 0x7fadb406d510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadc8072760 0x7fadc8151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadc815da70 0x7fadc80ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 --2- 172.21.15.181:0/3826341662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc80ce6a0 0x7fadc80c9070 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 >> 172.21.15.181:0/3826341662 conn(0x7fadc80cd010 msgr2=0x7fadc80cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 shutdown_connections 2023-10-16T11:00:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:43.989+0000 7fadcde83700 1 -- 172.21.15.181:0/3826341662 wait complete. 2023-10-16T11:00:44.003 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:00:44.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.121+0000 7f6826ab9700 1 Processor -- start 2023-10-16T11:00:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.121+0000 7f6826ab9700 1 -- start start 2023-10-16T11:00:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f6826ab9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f682015eab0 0x7f68200d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f6826ab9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68200cb0c0 0x7f682015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f6826ab9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682006ed00 con 0x7f682015e6d0 2023-10-16T11:00:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f6826ab9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68200d02d0 con 0x7f682015eab0 2023-10-16T11:00:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f6826ab9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682015ad40 con 0x7f68200cb0c0 2023-10-16T11:00:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f682015eab0 0x7f68200d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f682015eab0 0x7f68200d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50622/0 (socket says 172.21.15.181:50622) 2023-10-16T11:00:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 -- 172.21.15.181:0/2993750445 learned_addr learned my addr 172.21.15.181:0/2993750445 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f6825056700 1 --2- 172.21.15.181:0/2993750445 >> v2:172.21.15.190:3300/0 conn(0x7f68200cb0c0 0x7f682015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 -- 172.21.15.181:0/2993750445 >> v2:172.21.15.190:3300/0 conn(0x7f68200cb0c0 msgr2=0x7f682015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 --2- 172.21.15.181:0/2993750445 >> v2:172.21.15.190:3300/0 conn(0x7f68200cb0c0 0x7f682015a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 -- 172.21.15.181:0/2993750445 >> v1:172.21.15.190:6789/0 conn(0x7f682015e6d0 legacy=0x7f682015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 -- 172.21.15.181:0/2993750445 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682006e0f0 con 0x7f682015eab0 2023-10-16T11:00:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681ffff700 1 --2- 172.21.15.181:0/2993750445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f68200d0fc0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f6820071650 tx=0x7f680c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c14ddfacbd9acf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681f7fe700 1 -- 172.21.15.181:0/2993750445 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c00dcf0 con 0x7f682015eab0 2023-10-16T11:00:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681f7fe700 1 -- 172.21.15.181:0/2993750445 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c00de50 con 0x7f682015eab0 2023-10-16T11:00:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.122+0000 7f681f7fe700 1 -- 172.21.15.181:0/2993750445 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c004250 con 0x7f682015eab0 2023-10-16T11:00:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 -- 172.21.15.181:0/2993750445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 msgr2=0x7f68200d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 --2- 172.21.15.181:0/2993750445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f68200d0fc0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f6820071650 tx=0x7f680c00be70 comp rx=0 tx=0).stop 2023-10-16T11:00:44.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 -- 172.21.15.181:0/2993750445 shutdown_connections 2023-10-16T11:00:44.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 --2- 172.21.15.181:0/2993750445 >> v2:172.21.15.190:3300/0 conn(0x7f68200cb0c0 0x7f682015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 --2- 172.21.15.181:0/2993750445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f68200d0fc0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 -- 172.21.15.181:0/2993750445 >> 172.21.15.181:0/2993750445 conn(0x7f68200d04a0 msgr2=0x7f68200d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:44.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 -- 172.21.15.181:0/2993750445 shutdown_connections 2023-10-16T11:00:44.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.123+0000 7f6826ab9700 1 -- 172.21.15.181:0/2993750445 wait complete. 2023-10-16T11:00:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.124+0000 7f6826ab9700 1 Processor -- start 2023-10-16T11:00:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.124+0000 7f6826ab9700 1 -- start start 2023-10-16T11:00:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6826ab9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200cb0c0 0x7f682015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6826ab9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682015e6d0 0x7f6820159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6826ab9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f682015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6826ab9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682014e190 con 0x7f682015eab0 2023-10-16T11:00:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6826ab9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682015d2a0 con 0x7f682015e6d0 2023-10-16T11:00:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6826ab9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200c6b50 con 0x7f68200cb0c0 2023-10-16T11:00:44.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6825056700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f682015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6825056700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f682015a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50630/0 (socket says 172.21.15.181:50630) 2023-10-16T11:00:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6825056700 1 -- 172.21.15.181:0/3314271246 learned_addr learned my addr 172.21.15.181:0/3314271246 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f6824855700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200cb0c0 0x7f682015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.125+0000 7f681ffff700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682015e6d0 0x7f6820159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6825056700 1 -- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200cb0c0 msgr2=0x7f682015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6825056700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200cb0c0 0x7f682015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6825056700 1 -- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682015e6d0 msgr2=0x7f6820159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6825056700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682015e6d0 0x7f6820159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6825056700 1 -- 172.21.15.181:0/3314271246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c00b7f0 con 0x7f682015eab0 2023-10-16T11:00:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6825056700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f682015a290 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f681800cc30 tx=0x7f681800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f681d7fa700 1 -- 172.21.15.181:0/3314271246 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818003a20 con 0x7f682015eab0 2023-10-16T11:00:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f681d7fa700 1 -- 172.21.15.181:0/3314271246 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6818003b80 con 0x7f682015eab0 2023-10-16T11:00:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68200c6230 con 0x7f682015eab0 2023-10-16T11:00:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f681d7fa700 1 -- 172.21.15.181:0/3314271246 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800a9f0 con 0x7f682015eab0 2023-10-16T11:00:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.126+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68200c6500 con 0x7f682015eab0 2023-10-16T11:00:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.127+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68200c6cc0 con 0x7f682015eab0 2023-10-16T11:00:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.127+0000 7f681d7fa700 1 -- 172.21.15.181:0/3314271246 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f681800ab50 con 0x7f682015eab0 2023-10-16T11:00:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.128+0000 7f681d7fa700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f680406b020 0x7f680406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.128+0000 7f6824855700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f680406b020 0x7f680406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.128+0000 7f681d7fa700 1 -- 172.21.15.181:0/3314271246 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f68180896c0 con 0x7f682015eab0 2023-10-16T11:00:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.128+0000 7f6824855700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f680406b020 0x7f680406d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f682015c660 tx=0x7f68140090e0 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.132+0000 7f681d7fa700 1 -- 172.21.15.181:0/3314271246 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f68180546b0 con 0x7f682015eab0 2023-10-16T11:00:44.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.328+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6820067d20 con 0x7f682015eab0 2023-10-16T11:00:44.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.328+0000 7f681d7fa700 1 -- 172.21.15.181:0/3314271246 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v164) v1 ==== 84+0+65 (secure 0 0 0) 0x7f681801d020 con 0x7f682015eab0 2023-10-16T11:00:44.330 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:44.330 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:00:44.331 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:44.331 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:00:44.331 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:00:44.331 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:44.331 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:00:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.330+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f680406b020 msgr2=0x7f680406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.330+0000 7f6826ab9700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f680406b020 0x7f680406d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f682015c660 tx=0x7f68140090e0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.330+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 msgr2=0x7f682015a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.330+0000 7f6826ab9700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f682015a290 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f681800cc30 tx=0x7f681800cf40 comp rx=0 tx=0).stop 2023-10-16T11:00:44.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 shutdown_connections 2023-10-16T11:00:44.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f680406b020 0x7f680406d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200cb0c0 0x7f682015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f682015eab0 0x7f682015a290 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 --2- 172.21.15.181:0/3314271246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f682015e6d0 0x7f6820159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 >> 172.21.15.181:0/3314271246 conn(0x7f68200d04a0 msgr2=0x7f68200aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:44.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 shutdown_connections 2023-10-16T11:00:44.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.331+0000 7f6826ab9700 1 -- 172.21.15.181:0/3314271246 wait complete. 2023-10-16T11:00:44.344 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:00:44.345 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:00:44.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.469+0000 7fb5945bb700 1 Processor -- start 2023-10-16T11:00:44.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.469+0000 7fb5945bb700 1 -- start start 2023-10-16T11:00:44.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb5945bb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb58c0caaf0 0x7fb58c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb5945bb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c160240 con 0x7fb58c06a8a0 2023-10-16T11:00:44.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb5945bb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c15ff60 con 0x7fb58c0cf5c0 2023-10-16T11:00:44.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb5945bb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58c15bd80 con 0x7fb58c0caaf0 2023-10-16T11:00:44.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb592b58700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb58c0caaf0 0x7fb58c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb592b58700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb58c0caaf0 0x7fb58c15c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50638/0 (socket says 172.21.15.181:50638) 2023-10-16T11:00:44.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb592357700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb58c06a8a0 0x7fb58c06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33480/0 (socket says 172.21.15.181:33480) 2023-10-16T11:00:44.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb592b58700 1 -- 172.21.15.181:0/1791997290 learned_addr learned my addr 172.21.15.181:0/1791997290 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:44.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb592b58700 1 -- 172.21.15.181:0/1791997290 >> v1:172.21.15.190:6789/0 conn(0x7fb58c0cf5c0 legacy=0x7fb58c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:44.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb592b58700 1 -- 172.21.15.181:0/1791997290 >> v1:172.21.15.181:6789/0 conn(0x7fb58c06a8a0 legacy=0x7fb58c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb591355700 1 -- 172.21.15.181:0/1791997290 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882940135 0 0) 0x7fb58c160240 con 0x7fb58c06a8a0 2023-10-16T11:00:44.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.470+0000 7fb592b58700 1 -- 172.21.15.181:0/1791997290 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c0cfc40 con 0x7fb58c0caaf0 2023-10-16T11:00:44.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb592b58700 1 --2- 172.21.15.181:0/1791997290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c15c1a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb584009e50 tx=0x7fb58400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff252eb7c7d4ca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:44.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb591355700 1 -- 172.21.15.181:0/1791997290 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58400e070 con 0x7fb58c0caaf0 2023-10-16T11:00:44.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb591355700 1 -- 172.21.15.181:0/1791997290 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb584003a20 con 0x7fb58c0caaf0 2023-10-16T11:00:44.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb591355700 1 -- 172.21.15.181:0/1791997290 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584003dd0 con 0x7fb58c0caaf0 2023-10-16T11:00:44.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb5945bb700 1 -- 172.21.15.181:0/1791997290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 msgr2=0x7fb58c15c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb5945bb700 1 --2- 172.21.15.181:0/1791997290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c15c1a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb584009e50 tx=0x7fb58400b330 comp rx=0 tx=0).stop 2023-10-16T11:00:44.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb5945bb700 1 -- 172.21.15.181:0/1791997290 shutdown_connections 2023-10-16T11:00:44.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb5945bb700 1 --2- 172.21.15.181:0/1791997290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c15c1a0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb5945bb700 1 -- 172.21.15.181:0/1791997290 >> 172.21.15.181:0/1791997290 conn(0x7fb58c0d04a0 msgr2=0x7fb58c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:44.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb5945bb700 1 -- 172.21.15.181:0/1791997290 shutdown_connections 2023-10-16T11:00:44.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.471+0000 7fb5945bb700 1 -- 172.21.15.181:0/1791997290 wait complete. 2023-10-16T11:00:44.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 Processor -- start 2023-10-16T11:00:44.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 -- start start 2023-10-16T11:00:44.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06a8a0 0x7fb58c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c0ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0cf5c0 0x7fb58c0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0af810 con 0x7fb58c0caaf0 2023-10-16T11:00:44.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0aef70 con 0x7fb58c0cf5c0 2023-10-16T11:00:44.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb5945bb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0af250 con 0x7fb58c06a8a0 2023-10-16T11:00:44.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb591b56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c0ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb591b56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c0ae400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50642/0 (socket says 172.21.15.181:50642) 2023-10-16T11:00:44.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.473+0000 7fb591b56700 1 -- 172.21.15.181:0/845763121 learned_addr learned my addr 172.21.15.181:0/845763121 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:44.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb592b58700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0cf5c0 0x7fb58c0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb592357700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06a8a0 0x7fb58c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb591b56700 1 -- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06a8a0 msgr2=0x7fb58c156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb591b56700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06a8a0 0x7fb58c156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb591b56700 1 -- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0cf5c0 msgr2=0x7fb58c0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb591b56700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0cf5c0 0x7fb58c0ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb591b56700 1 -- 172.21.15.181:0/845763121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584009b30 con 0x7fb58c0caaf0 2023-10-16T11:00:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb591b56700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c0ae400 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb58800b4a0 tx=0x7fb58800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb582ffd700 1 -- 172.21.15.181:0/845763121 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588011830 con 0x7fb58c0caaf0 2023-10-16T11:00:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb582ffd700 1 -- 172.21.15.181:0/845763121 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb588011990 con 0x7fb58c0caaf0 2023-10-16T11:00:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c0af530 con 0x7fb58c0caaf0 2023-10-16T11:00:44.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb582ffd700 1 -- 172.21.15.181:0/845763121 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588011d40 con 0x7fb58c0caaf0 2023-10-16T11:00:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.474+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c157e80 con 0x7fb58c0caaf0 2023-10-16T11:00:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.475+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c0c3d10 con 0x7fb58c0caaf0 2023-10-16T11:00:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.475+0000 7fb582ffd700 1 -- 172.21.15.181:0/845763121 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb58801a400 con 0x7fb58c0caaf0 2023-10-16T11:00:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.476+0000 7fb582ffd700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb57006b020 0x7fb57006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:44.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.476+0000 7fb592357700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb57006b020 0x7fb57006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.476+0000 7fb582ffd700 1 -- 172.21.15.181:0/845763121 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fb588089260 con 0x7fb58c0caaf0 2023-10-16T11:00:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.476+0000 7fb592357700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb57006b020 0x7fb57006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001020 tx=0x7fb57c009040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:44.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.480+0000 7fb582ffd700 1 -- 172.21.15.181:0/845763121 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb5880542d0 con 0x7fb58c0caaf0 2023-10-16T11:00:44.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.672+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb58c0cfef0 con 0x7fb58c0caaf0 2023-10-16T11:00:44.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.865+0000 7fb582ffd700 1 -- 172.21.15.181:0/845763121 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v165) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb5880578f0 con 0x7fb58c0caaf0 2023-10-16T11:00:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb57006b020 msgr2=0x7fb57006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb57006b020 0x7fb57006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001020 tx=0x7fb57c009040 comp rx=0 tx=0).stop 2023-10-16T11:00:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 msgr2=0x7fb58c0ae400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c0ae400 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb58800b4a0 tx=0x7fb58800b7b0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 shutdown_connections 2023-10-16T11:00:44.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb57006b020 0x7fb57006d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb58c06a8a0 0x7fb58c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb58c0cf5c0 0x7fb58c0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 --2- 172.21.15.181:0/845763121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb58c0caaf0 0x7fb58c0ae400 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 >> 172.21.15.181:0/845763121 conn(0x7fb58c0d04a0 msgr2=0x7fb58c159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 shutdown_connections 2023-10-16T11:00:44.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:44.868+0000 7fb5945bb700 1 -- 172.21.15.181:0/845763121 wait complete. 2023-10-16T11:00:44.872 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T11:00:44.885 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:00:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.011+0000 7f359e23b700 1 Processor -- start 2023-10-16T11:00:45.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.011+0000 7f359e23b700 1 -- start start 2023-10-16T11:00:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f359e23b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35980b2010 0x7f35980cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f359e23b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3598071be0 0x7f35980b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:45.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f359e23b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f359806f3b0 0x7f359806f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f359e23b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35980ce1f0 con 0x7f35980b2010 2023-10-16T11:00:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f359e23b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3598072090 con 0x7f3598071be0 2023-10-16T11:00:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f359e23b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35980b2930 con 0x7f359806f3b0 2023-10-16T11:00:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35980b2010 0x7f35980cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:45.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35980b2010 0x7f35980cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50648/0 (socket says 172.21.15.181:50648) 2023-10-16T11:00:45.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 -- 172.21.15.181:0/1810141760 learned_addr learned my addr 172.21.15.181:0/1810141760 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:45.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f3596ffd700 1 --2- 172.21.15.181:0/1810141760 >> v2:172.21.15.190:3300/0 conn(0x7f3598071be0 0x7f35980b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:45.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f3597fff700 1 --2- 172.21.15.181:0/1810141760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f359806f3b0 0x7f359806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:45.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 -- 172.21.15.181:0/1810141760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f359806f3b0 msgr2=0x7f359806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:45.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 --2- 172.21.15.181:0/1810141760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f359806f3b0 0x7f359806f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 -- 172.21.15.181:0/1810141760 >> v2:172.21.15.190:3300/0 conn(0x7f3598071be0 msgr2=0x7f35980b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:45.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 --2- 172.21.15.181:0/1810141760 >> v2:172.21.15.190:3300/0 conn(0x7f3598071be0 0x7f35980b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.012+0000 7f35977fe700 1 -- 172.21.15.181:0/1810141760 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35980ccda0 con 0x7f35980b2010 2023-10-16T11:00:45.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f35977fe700 1 --2- 172.21.15.181:0/1810141760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35980b2010 0x7f35980cc6e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f359815e1c0 tx=0x7f358c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c71bb076b9752af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:45.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f35967fc700 1 -- 172.21.15.181:0/1810141760 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00b870 con 0x7f35980b2010 2023-10-16T11:00:45.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f35967fc700 1 -- 172.21.15.181:0/1810141760 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00b9d0 con 0x7f35980b2010 2023-10-16T11:00:45.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f35967fc700 1 -- 172.21.15.181:0/1810141760 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00bd60 con 0x7f35980b2010 2023-10-16T11:00:45.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 -- 172.21.15.181:0/1810141760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35980b2010 msgr2=0x7f35980cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:45.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 --2- 172.21.15.181:0/1810141760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35980b2010 0x7f35980cc6e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f359815e1c0 tx=0x7f358c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 -- 172.21.15.181:0/1810141760 shutdown_connections 2023-10-16T11:00:45.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 --2- 172.21.15.181:0/1810141760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35980b2010 0x7f35980cc6e0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 --2- 172.21.15.181:0/1810141760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f359806f3b0 0x7f359806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 --2- 172.21.15.181:0/1810141760 >> v2:172.21.15.190:3300/0 conn(0x7f3598071be0 0x7f35980b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 -- 172.21.15.181:0/1810141760 >> 172.21.15.181:0/1810141760 conn(0x7f35980cd010 msgr2=0x7f35980cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:45.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 -- 172.21.15.181:0/1810141760 shutdown_connections 2023-10-16T11:00:45.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.013+0000 7f359e23b700 1 -- 172.21.15.181:0/1810141760 wait complete. 2023-10-16T11:00:45.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.015+0000 7f359e23b700 1 Processor -- start 2023-10-16T11:00:45.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.015+0000 7f359e23b700 1 -- start start 2023-10-16T11:00:45.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.015+0000 7f359e23b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359806f3b0 0x7f359814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:45.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.015+0000 7f359e23b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3598071be0 0x7f35981f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.015+0000 7f359e23b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35980b2010 0x7f359806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.015+0000 7f359e23b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981f41d0 con 0x7f3598071be0 2023-10-16T11:00:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f359e23b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981f48c0 con 0x7f359806f3b0 2023-10-16T11:00:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f359e23b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35980ad9e0 con 0x7f35980b2010 2023-10-16T11:00:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3598071be0 0x7f35981f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3598071be0 0x7f35981f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50656/0 (socket says 172.21.15.181:50656) 2023-10-16T11:00:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 -- 172.21.15.181:0/3112572848 learned_addr learned my addr 172.21.15.181:0/3112572848 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f35977fe700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359806f3b0 0x7f359814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3597fff700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35980b2010 0x7f359806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 -- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35980b2010 msgr2=0x7f359806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:45.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35980b2010 0x7f359806f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 -- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359806f3b0 msgr2=0x7f359814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:45.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359806f3b0 0x7f359814fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 -- 172.21.15.181:0/3112572848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c009550 con 0x7f3598071be0 2023-10-16T11:00:45.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3597fff700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35980b2010 0x7f359806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:45.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f3596ffd700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3598071be0 0x7f35981f4380 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f358800d830 tx=0x7f358800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:45.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f357ffff700 1 -- 172.21.15.181:0/3112572848 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880125f0 con 0x7f3598071be0 2023-10-16T11:00:45.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f357ffff700 1 -- 172.21.15.181:0/3112572848 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3588012750 con 0x7f3598071be0 2023-10-16T11:00:45.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35980adcc0 con 0x7f3598071be0 2023-10-16T11:00:45.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f357ffff700 1 -- 172.21.15.181:0/3112572848 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588012b00 con 0x7f3598071be0 2023-10-16T11:00:45.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.016+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35980ae280 con 0x7f3598071be0 2023-10-16T11:00:45.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.017+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35980ae5f0 con 0x7f3598071be0 2023-10-16T11:00:45.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.018+0000 7f357ffff700 1 -- 172.21.15.181:0/3112572848 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3588012c60 con 0x7f3598071be0 2023-10-16T11:00:45.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.018+0000 7f357ffff700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f358406b020 0x7f358406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:45.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.018+0000 7f35977fe700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f358406b020 0x7f358406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:45.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.018+0000 7f357ffff700 1 -- 172.21.15.181:0/3112572848 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5723+0+0 (secure 0 0 0) 0x7f3588089a30 con 0x7f3598071be0 2023-10-16T11:00:45.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.019+0000 7f35977fe700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f358406b020 0x7f358406d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f358c014040 tx=0x7f358c008030 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:45.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.023+0000 7f357ffff700 1 -- 172.21.15.181:0/3112572848 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f35880549d0 con 0x7f3598071be0 2023-10-16T11:00:45.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.214+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f35980ae410 con 0x7f3598071be0 2023-10-16T11:00:45.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-16T11:00:45.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.867+0000 7f357ffff700 1 -- 172.21.15.181:0/3112572848 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v166) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3588057ff0 con 0x7f3598071be0 2023-10-16T11:00:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f358406b020 msgr2=0x7f358406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f358406b020 0x7f358406d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f358c014040 tx=0x7f358c008030 comp rx=0 tx=0).stop 2023-10-16T11:00:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3598071be0 msgr2=0x7f35981f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3598071be0 0x7f35981f4380 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f358800d830 tx=0x7f358800db40 comp rx=0 tx=0).stop 2023-10-16T11:00:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 shutdown_connections 2023-10-16T11:00:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f358406b020 0x7f358406d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35980b2010 0x7f359806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f359806f3b0 0x7f359814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 --2- 172.21.15.181:0/3112572848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3598071be0 0x7f35981f4380 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.871+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 >> 172.21.15.181:0/3112572848 conn(0x7f35980cd010 msgr2=0x7f35980b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.872+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 shutdown_connections 2023-10-16T11:00:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:45.872+0000 7f359e23b700 1 -- 172.21.15.181:0/3112572848 wait complete. 2023-10-16T11:00:45.880 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T11:00:45.898 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:00:46.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.024+0000 7f649d0f3700 1 Processor -- start 2023-10-16T11:00:46.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.024+0000 7f649d0f3700 1 -- start start 2023-10-16T11:00:46.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649d0f3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6498072760 0x7f649806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:46.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649d0f3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649806ade0 0x7f64981615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:46.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649d0f3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649806b210 con 0x7f649815da70 2023-10-16T11:00:46.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649d0f3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64980cef00 con 0x7f6498072760 2023-10-16T11:00:46.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649d0f3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980cec20 con 0x7f649806ade0 2023-10-16T11:00:46.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6498072760 0x7f649806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:46.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6498072760 0x7f649806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50664/0 (socket says 172.21.15.181:50664) 2023-10-16T11:00:46.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 -- 172.21.15.181:0/2840737464 learned_addr learned my addr 172.21.15.181:0/2840737464 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649759e700 1 --2- 172.21.15.181:0/2840737464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649806ade0 0x7f64981615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 -- 172.21.15.181:0/2840737464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649806ade0 msgr2=0x7f64981615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 --2- 172.21.15.181:0/2840737464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649806ade0 0x7f64981615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 -- 172.21.15.181:0/2840737464 >> v1:172.21.15.190:6789/0 conn(0x7f649815da70 legacy=0x7f64980ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 -- 172.21.15.181:0/2840737464 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6498071c80 con 0x7f6498072760 2023-10-16T11:00:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f649659c700 1 --2- 172.21.15.181:0/2840737464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6498072760 0x7f649806a8a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f648c0098f0 tx=0x7f648c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a302a9817d30f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f6495d9b700 1 -- 172.21.15.181:0/2840737464 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c011830 con 0x7f6498072760 2023-10-16T11:00:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f6495d9b700 1 -- 172.21.15.181:0/2840737464 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c011990 con 0x7f6498072760 2023-10-16T11:00:46.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.025+0000 7f6495d9b700 1 -- 172.21.15.181:0/2840737464 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c011d20 con 0x7f6498072760 2023-10-16T11:00:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 -- 172.21.15.181:0/2840737464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6498072760 msgr2=0x7f649806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 --2- 172.21.15.181:0/2840737464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6498072760 0x7f649806a8a0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f648c0098f0 tx=0x7f648c009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 -- 172.21.15.181:0/2840737464 shutdown_connections 2023-10-16T11:00:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 --2- 172.21.15.181:0/2840737464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649806ade0 0x7f64981615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 --2- 172.21.15.181:0/2840737464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6498072760 0x7f649806a8a0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 -- 172.21.15.181:0/2840737464 >> 172.21.15.181:0/2840737464 conn(0x7f64980cd010 msgr2=0x7f64980cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:46.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 -- 172.21.15.181:0/2840737464 shutdown_connections 2023-10-16T11:00:46.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.026+0000 7f649d0f3700 1 -- 172.21.15.181:0/2840737464 wait complete. 2023-10-16T11:00:46.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.027+0000 7f649d0f3700 1 Processor -- start 2023-10-16T11:00:46.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.027+0000 7f649d0f3700 1 -- start start 2023-10-16T11:00:46.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649d0f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649806ade0 0x7f64981f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:46.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649d0f3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498072760 0x7f64981f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:46.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649d0f3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649815da70 0x7f64980ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:46.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649d0f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64981f4440 con 0x7f649806ade0 2023-10-16T11:00:46.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649d0f3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980adac0 con 0x7f6498072760 2023-10-16T11:00:46.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649d0f3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980aca20 con 0x7f649815da70 2023-10-16T11:00:46.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f6496d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649806ade0 0x7f64981f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:46.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f6496d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649806ade0 0x7f64981f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50668/0 (socket says 172.21.15.181:50668) 2023-10-16T11:00:46.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f6496d9d700 1 -- 172.21.15.181:0/3152472385 learned_addr learned my addr 172.21.15.181:0/3152472385 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:46.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649659c700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498072760 0x7f64981f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:46.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.028+0000 7f649759e700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649815da70 0x7f64980ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:46.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f6496d9d700 1 -- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649815da70 msgr2=0x7f64980ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:46.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f6496d9d700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649815da70 0x7f64980ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f6496d9d700 1 -- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498072760 msgr2=0x7f64981f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:46.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f6496d9d700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498072760 0x7f64981f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f6496d9d700 1 -- 172.21.15.181:0/3152472385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c009550 con 0x7f649806ade0 2023-10-16T11:00:46.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f649759e700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649815da70 0x7f64980ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:46.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f6496d9d700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649806ade0 0x7f64981f3f00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f648800c450 tx=0x7f648800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:46.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f647f7fe700 1 -- 172.21.15.181:0/3152472385 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64880052d0 con 0x7f649806ade0 2023-10-16T11:00:46.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f647f7fe700 1 -- 172.21.15.181:0/3152472385 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6488005430 con 0x7f649806ade0 2023-10-16T11:00:46.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64980acd00 con 0x7f649806ade0 2023-10-16T11:00:46.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f647f7fe700 1 -- 172.21.15.181:0/3152472385 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64880038c0 con 0x7f649806ade0 2023-10-16T11:00:46.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.029+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64980ad2c0 con 0x7f649806ade0 2023-10-16T11:00:46.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.030+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64980b1ee0 con 0x7f649806ade0 2023-10-16T11:00:46.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.031+0000 7f647f7fe700 1 -- 172.21.15.181:0/3152472385 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6488003a20 con 0x7f649806ade0 2023-10-16T11:00:46.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.031+0000 7f647f7fe700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f648406b020 0x7f648406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:46.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.031+0000 7f647f7fe700 1 -- 172.21.15.181:0/3152472385 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f6488088c00 con 0x7f649806ade0 2023-10-16T11:00:46.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.031+0000 7f649659c700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f648406b020 0x7f648406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:46.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.031+0000 7f649659c700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f648406b020 0x7f648406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f648c009c40 tx=0x7f648c008030 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:46.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.036+0000 7f647f7fe700 1 -- 172.21.15.181:0/3152472385 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f648808f080 con 0x7f649806ade0 2023-10-16T11:00:46.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.274+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f64980b1c00 con 0x7f649806ade0 2023-10-16T11:00:46.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.890+0000 7f647f7fe700 1 -- 172.21.15.181:0/3152472385 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v81) v1 ==== 174+0+0 (secure 0 0 0) 0x7f64880170c0 con 0x7f649806ade0 2023-10-16T11:00:46.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.899+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f648406b020 msgr2=0x7f648406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:46.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.899+0000 7f649d0f3700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f648406b020 0x7f648406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f648c009c40 tx=0x7f648c008030 comp rx=0 tx=0).stop 2023-10-16T11:00:46.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.899+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649806ade0 msgr2=0x7f64981f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:46.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.899+0000 7f649d0f3700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649806ade0 0x7f64981f3f00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f648800c450 tx=0x7f648800c760 comp rx=0 tx=0).stop 2023-10-16T11:00:46.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 shutdown_connections 2023-10-16T11:00:46.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f648406b020 0x7f648406d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f649815da70 0x7f64980ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649806ade0 0x7f64981f3f00 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 --2- 172.21.15.181:0/3152472385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498072760 0x7f64981f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:46.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 >> 172.21.15.181:0/3152472385 conn(0x7f64980cd010 msgr2=0x7f64980d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:46.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 shutdown_connections 2023-10-16T11:00:46.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:46.901+0000 7f649d0f3700 1 -- 172.21.15.181:0/3152472385 wait complete. 2023-10-16T11:00:46.911 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 20 and data pool 21 2023-10-16T11:00:46.921 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:47.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.040+0000 7f71872ba700 1 Processor -- start 2023-10-16T11:00:47.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.040+0000 7f71872ba700 1 -- start start 2023-10-16T11:00:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.041+0000 7f71872ba700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f718015dfd0 0x7f71800af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.041+0000 7f71872ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 0x7f71800cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.041+0000 7f71872ba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718006f870 con 0x7f71800af450 2023-10-16T11:00:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.041+0000 7f71872ba700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800cde40 con 0x7f718015dfd0 2023-10-16T11:00:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.041+0000 7f71872ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800d1160 con 0x7f71800cd470 2023-10-16T11:00:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.041+0000 7f7184855700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f71800af450 0x7f718006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33536/0 (socket says 172.21.15.181:33536) 2023-10-16T11:00:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.041+0000 7f7184855700 1 -- 172.21.15.181:0/2486376596 learned_addr learned my addr 172.21.15.181:0/2486376596 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185056700 1 --2- 172.21.15.181:0/2486376596 >> v2:172.21.15.190:3300/0 conn(0x7f718015dfd0 0x7f71800af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185857700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 0x7f71800cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638168908 0 0) 0x7f718006f870 con 0x7f71800af450 2023-10-16T11:00:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f716c0035b0 con 0x7f71800af450 2023-10-16T11:00:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248114879 0 0) 0x7f716c0035b0 con 0x7f71800af450 2023-10-16T11:00:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 msgr2=0x7f71800cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 0x7f71800cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 >> v2:172.21.15.190:3300/0 conn(0x7f718015dfd0 msgr2=0x7f71800af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 --2- 172.21.15.181:0/2486376596 >> v2:172.21.15.190:3300/0 conn(0x7f718015dfd0 0x7f71800af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800cc8e0 con 0x7f71800af450 2023-10-16T11:00:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f717c001670 con 0x7f71800af450 2023-10-16T11:00:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 >> v1:172.21.15.181:6789/0 conn(0x7f71800af450 legacy=0x7f718006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c003a00 0x7f716c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c004850 0x7f716c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 0x7f716c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800cde40 con 0x7f716c003a00 2023-10-16T11:00:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800d1160 con 0x7f716c004850 2023-10-16T11:00:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c00ba40 con 0x7f71800cd470 2023-10-16T11:00:47.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185056700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c003a00 0x7f716c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185857700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 0x7f716c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7184855700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c004850 0x7f716c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185056700 1 -- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 msgr2=0x7f716c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185056700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800cd470 0x7f716c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185056700 1 -- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c004850 msgr2=0x7f716c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185056700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c004850 0x7f716c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.042+0000 7f7185056700 1 -- 172.21.15.181:0/2486376596 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0035b0 con 0x7f716c003a00 2023-10-16T11:00:47.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f7185056700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c003a00 0x7f716c003e20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7170009b40 tx=0x7f716c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798a45e163bc5c99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170005a20 con 0x7f716c003a00 2023-10-16T11:00:47.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170005b80 con 0x7f716c003a00 2023-10-16T11:00:47.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f7177fff700 1 -- 172.21.15.181:0/2486376596 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000e660 con 0x7f716c003a00 2023-10-16T11:00:47.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f71872ba700 1 -- 172.21.15.181:0/2486376596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c003a00 msgr2=0x7f716c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f71872ba700 1 --2- 172.21.15.181:0/2486376596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f716c003a00 0x7f716c003e20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7170009b40 tx=0x7f716c005880 comp rx=0 tx=0).stop 2023-10-16T11:00:47.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f7185857700 1 -- 172.21.15.181:0/2486376596 reap_dead start 2023-10-16T11:00:47.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f71872ba700 1 -- 172.21.15.181:0/2486376596 shutdown_connections 2023-10-16T11:00:47.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f71872ba700 1 -- 172.21.15.181:0/2486376596 >> 172.21.15.181:0/2486376596 conn(0x7f71800d04a0 msgr2=0x7f71800d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:47.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f71872ba700 1 -- 172.21.15.181:0/2486376596 shutdown_connections 2023-10-16T11:00:47.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.043+0000 7f71872ba700 1 -- 172.21.15.181:0/2486376596 wait complete. 2023-10-16T11:00:47.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.045+0000 7f71872ba700 1 Processor -- start 2023-10-16T11:00:47.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.045+0000 7f71872ba700 1 -- start start 2023-10-16T11:00:47.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f71872ba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c003a00 0x7f71801562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f71872ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71800b0910 0x7f71800c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f71872ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800c94b0 0x7f718015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f71872ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b0da0 con 0x7f71800b0910 2023-10-16T11:00:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f71872ba700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b1080 con 0x7f716c003a00 2023-10-16T11:00:47.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f71872ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b1360 con 0x7f71800c94b0 2023-10-16T11:00:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71800b0910 0x7f71800c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71800b0910 0x7f71800c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50702/0 (socket says 172.21.15.181:50702) 2023-10-16T11:00:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 -- 172.21.15.181:0/3403096148 learned_addr learned my addr 172.21.15.181:0/3403096148 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 -- 172.21.15.181:0/3403096148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800c94b0 msgr2=0x7f718015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800c94b0 0x7f718015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 -- 172.21.15.181:0/3403096148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c003a00 msgr2=0x7f71801562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c003a00 0x7f71801562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 -- 172.21.15.181:0/3403096148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800b1640 con 0x7f71800b0910 2023-10-16T11:00:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7184855700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71800b0910 0x7f71800c8f70 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f717c000fb0 tx=0x7f717c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.046+0000 7f7175ffb700 1 -- 172.21.15.181:0/3403096148 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00f600 con 0x7f71800b0910 2023-10-16T11:00:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.047+0000 7f7175ffb700 1 -- 172.21.15.181:0/3403096148 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00f760 con 0x7f71800b0910 2023-10-16T11:00:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.047+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71800b1920 con 0x7f71800b0910 2023-10-16T11:00:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.047+0000 7f7175ffb700 1 -- 172.21.15.181:0/3403096148 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00fb10 con 0x7f71800b0910 2023-10-16T11:00:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.047+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718015c570 con 0x7f71800b0910 2023-10-16T11:00:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.048+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71800b1b40 con 0x7f71800b0910 2023-10-16T11:00:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.048+0000 7f7175ffb700 1 -- 172.21.15.181:0/3403096148 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f717c00fc70 con 0x7f71800b0910 2023-10-16T11:00:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.048+0000 7f7175ffb700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f716406b020 0x7f716406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.049+0000 7f7175ffb700 1 -- 172.21.15.181:0/3403096148 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f717c089880 con 0x7f71800b0910 2023-10-16T11:00:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.049+0000 7f7185056700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f716406b020 0x7f716406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.049+0000 7f7185056700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f716406b020 0x7f716406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f717000a010 tx=0x7f7170013000 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.053+0000 7f7175ffb700 1 -- 172.21.15.181:0/3403096148 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f717c054630 con 0x7f71800b0910 2023-10-16T11:00:47.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.246+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7180067d20 con 0x7f71800b0910 2023-10-16T11:00:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.247+0000 7f7175ffb700 1 -- 172.21.15.181:0/3403096148 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+17388 (secure 0 0 0) 0x7f717c057c50 con 0x7f71800b0910 2023-10-16T11:00:47.249 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:47.249 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":167,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:46.275906+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:00:44.693955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:00:45.236607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:47.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f716406b020 msgr2=0x7f716406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f716406b020 0x7f716406d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f717000a010 tx=0x7f7170013000 comp rx=0 tx=0).stop 2023-10-16T11:00:47.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71800b0910 msgr2=0x7f71800c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71800b0910 0x7f71800c8f70 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f717c000fb0 tx=0x7f717c007570 comp rx=0 tx=0).stop 2023-10-16T11:00:47.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 shutdown_connections 2023-10-16T11:00:47.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f716406b020 0x7f716406d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71800c94b0 0x7f718015c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c003a00 0x7f71801562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 --2- 172.21.15.181:0/3403096148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71800b0910 0x7f71800c8f70 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 >> 172.21.15.181:0/3403096148 conn(0x7f71800d04a0 msgr2=0x7f718006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:47.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 shutdown_connections 2023-10-16T11:00:47.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.249+0000 7f71872ba700 1 -- 172.21.15.181:0/3403096148 wait complete. 2023-10-16T11:00:47.263 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:47.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.382+0000 7f7e78aa5700 1 Processor -- start 2023-10-16T11:00:47.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.382+0000 7f7e78aa5700 1 -- start start 2023-10-16T11:00:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e78aa5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 0x7f7e740af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e78aa5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e740af450 0x7f7e7406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e78aa5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7406f870 con 0x7f7e7406db80 2023-10-16T11:00:47.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e78aa5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e740d1160 con 0x7f7e740af450 2023-10-16T11:00:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e78aa5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740cc8e0 con 0x7f7e7415dfd0 2023-10-16T11:00:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e72d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e7406db80 0x7f7e740cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33556/0 (socket says 172.21.15.181:33556) 2023-10-16T11:00:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e72d9d700 1 -- 172.21.15.181:0/1638625740 learned_addr learned my addr 172.21.15.181:0/1638625740 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e7259c700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 0x7f7e740af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e71d9b700 1 --2- 172.21.15.181:0/1638625740 >> v2:172.21.15.190:3300/0 conn(0x7f7e740af450 0x7f7e7406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.383+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971980056 0 0) 0x7f7e7406f870 con 0x7f7e7406db80 2023-10-16T11:00:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e5c0035b0 con 0x7f7e7406db80 2023-10-16T11:00:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3650484142 0 0) 0x7f7e5c0035b0 con 0x7f7e7406db80 2023-10-16T11:00:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 msgr2=0x7f7e740af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 0x7f7e740af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 >> v2:172.21.15.190:3300/0 conn(0x7f7e740af450 msgr2=0x7f7e7406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 --2- 172.21.15.181:0/1638625740 >> v2:172.21.15.190:3300/0 conn(0x7f7e740af450 0x7f7e7406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740ccbc0 con 0x7f7e7406db80 2023-10-16T11:00:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7e6c0019f0 con 0x7f7e7406db80 2023-10-16T11:00:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 >> v1:172.21.15.181:6789/0 conn(0x7f7e7406db80 legacy=0x7f7e740cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 0x7f7e5c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c003f80 0x7f7e5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c004d90 0x7f7e5c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740d1160 con 0x7f7e5c003f80 2023-10-16T11:00:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740cc8e0 con 0x7f7e5c004d90 2023-10-16T11:00:47.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c00ba60 con 0x7f7e7415dfd0 2023-10-16T11:00:47.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e71d9b700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c003f80 0x7f7e5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7259c700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 0x7f7e5c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e72d9d700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c004d90 0x7f7e5c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e71d9b700 1 -- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 msgr2=0x7f7e5c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e71d9b700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e7415dfd0 0x7f7e5c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e71d9b700 1 -- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c004d90 msgr2=0x7f7e5c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e71d9b700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e5c004d90 0x7f7e5c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e71d9b700 1 -- 172.21.15.181:0/1638625740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0035b0 con 0x7f7e5c003f80 2023-10-16T11:00:47.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e71d9b700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c003f80 0x7f7e5c004850 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009b90 tx=0x7f7e5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38c09116f220d74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.384+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400e6c0 con 0x7f7e5c003f80 2023-10-16T11:00:47.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6400e820 con 0x7f7e5c003f80 2023-10-16T11:00:47.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e7159a700 1 -- 172.21.15.181:0/1638625740 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400eb90 con 0x7f7e5c003f80 2023-10-16T11:00:47.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e78aa5700 1 -- 172.21.15.181:0/1638625740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c003f80 msgr2=0x7f7e5c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e78aa5700 1 --2- 172.21.15.181:0/1638625740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c003f80 0x7f7e5c004850 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009b90 tx=0x7f7e5c005880 comp rx=0 tx=0).stop 2023-10-16T11:00:47.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e72d9d700 1 -- 172.21.15.181:0/1638625740 reap_dead start 2023-10-16T11:00:47.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e78aa5700 1 -- 172.21.15.181:0/1638625740 shutdown_connections 2023-10-16T11:00:47.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e78aa5700 1 -- 172.21.15.181:0/1638625740 >> 172.21.15.181:0/1638625740 conn(0x7f7e740d04a0 msgr2=0x7f7e740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:47.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e78aa5700 1 -- 172.21.15.181:0/1638625740 shutdown_connections 2023-10-16T11:00:47.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.385+0000 7f7e78aa5700 1 -- 172.21.15.181:0/1638625740 wait complete. 2023-10-16T11:00:47.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.386+0000 7f7e78aa5700 1 Processor -- start 2023-10-16T11:00:47.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e78aa5700 1 -- start start 2023-10-16T11:00:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e78aa5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c003f80 0x7f7e740c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e78aa5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e740b4350 0x7f7e7415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e78aa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e740b5e40 0x7f7e7415bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e78aa5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740b3f30 con 0x7f7e740b5e40 2023-10-16T11:00:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e78aa5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7415cde0 con 0x7f7e740b4350 2023-10-16T11:00:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e78aa5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740c65e0 con 0x7f7e5c003f80 2023-10-16T11:00:47.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e72d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e740b5e40 0x7f7e7415bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e72d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e740b5e40 0x7f7e7415bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50720/0 (socket says 172.21.15.181:50720) 2023-10-16T11:00:47.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e72d9d700 1 -- 172.21.15.181:0/2263382842 learned_addr learned my addr 172.21.15.181:0/2263382842 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:47.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e71d9b700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e740b4350 0x7f7e7415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.387+0000 7f7e7259c700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c003f80 0x7f7e740c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e72d9d700 1 -- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c003f80 msgr2=0x7f7e740c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e72d9d700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c003f80 0x7f7e740c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e72d9d700 1 -- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e740b4350 msgr2=0x7f7e7415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e72d9d700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e740b4350 0x7f7e7415b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e72d9d700 1 -- 172.21.15.181:0/2263382842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740c51a0 con 0x7f7e740b5e40 2023-10-16T11:00:47.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e72d9d700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e740b5e40 0x7f7e7415bb90 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7e740cca20 tx=0x7f7e6c007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e62ffd700 1 -- 172.21.15.181:0/2263382842 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c017070 con 0x7f7e740b5e40 2023-10-16T11:00:47.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e62ffd700 1 -- 172.21.15.181:0/2263382842 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c00ebc0 con 0x7f7e740b5e40 2023-10-16T11:00:47.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e740c6e00 con 0x7f7e740b5e40 2023-10-16T11:00:47.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e62ffd700 1 -- 172.21.15.181:0/2263382842 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00ed70 con 0x7f7e740b5e40 2023-10-16T11:00:47.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.388+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e740c6840 con 0x7f7e740b5e40 2023-10-16T11:00:47.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.389+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e740b4070 con 0x7f7e740b5e40 2023-10-16T11:00:47.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.389+0000 7f7e62ffd700 1 -- 172.21.15.181:0/2263382842 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7e6c013420 con 0x7f7e740b5e40 2023-10-16T11:00:47.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.390+0000 7f7e62ffd700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f7e5406b020 0x7f7e5406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.390+0000 7f7e7259c700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f7e5406b020 0x7f7e5406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.390+0000 7f7e62ffd700 1 -- 172.21.15.181:0/2263382842 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f7e6c0894d0 con 0x7f7e740b5e40 2023-10-16T11:00:47.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.390+0000 7f7e7259c700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f7e5406b020 0x7f7e5406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e740add00 tx=0x7f7e680091c0 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.394+0000 7f7e62ffd700 1 -- 172.21.15.181:0/2263382842 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e740b4070 con 0x7f7e740b5e40 2023-10-16T11:00:47.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.590+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e740b4070 con 0x7f7e740b5e40 2023-10-16T11:00:47.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.591+0000 7f7e62ffd700 1 -- 172.21.15.181:0/2263382842 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+17388 (secure 0 0 0) 0x7f7e740b4070 con 0x7f7e740b5e40 2023-10-16T11:00:47.593 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:47.593 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":167,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:46.275906+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:00:44.693955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:00:45.236607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:47.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f7e5406b020 msgr2=0x7f7e5406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f7e5406b020 0x7f7e5406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e740add00 tx=0x7f7e680091c0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e740b5e40 msgr2=0x7f7e7415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e740b5e40 0x7f7e7415bb90 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7e740cca20 tx=0x7f7e6c007c90 comp rx=0 tx=0).stop 2023-10-16T11:00:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 shutdown_connections 2023-10-16T11:00:47.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f7e5406b020 0x7f7e5406d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e5c003f80 0x7f7e740c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e740b5e40 0x7f7e7415bb90 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 --2- 172.21.15.181:0/2263382842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e740b4350 0x7f7e7415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.593+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 >> 172.21.15.181:0/2263382842 conn(0x7f7e740d04a0 msgr2=0x7f7e740ad370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:47.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.594+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 shutdown_connections 2023-10-16T11:00:47.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.594+0000 7f7e78aa5700 1 -- 172.21.15.181:0/2263382842 wait complete. 2023-10-16T11:00:47.607 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:00:47.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.728+0000 7f4f76455700 1 Processor -- start 2023-10-16T11:00:47.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.728+0000 7f4f76455700 1 -- start start 2023-10-16T11:00:47.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f76455700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f70159e10 0x7f4f700ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f76455700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7006bef0 0x7f4f700caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f76455700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7006e4b0 con 0x7f4f700cb0e0 2023-10-16T11:00:47.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f76455700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7006ed50 con 0x7f4f70159e10 2023-10-16T11:00:47.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f76455700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700ceea0 con 0x7f4f7006bef0 2023-10-16T11:00:47.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f70159e10 0x7f4f700ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f70159e10 0x7f4f700ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50726/0 (socket says 172.21.15.181:50726) 2023-10-16T11:00:47.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6ffff700 1 -- 172.21.15.181:0/4229241824 learned_addr learned my addr 172.21.15.181:0/4229241824 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:47.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f749f2700 1 --1- 172.21.15.181:0/4229241824 >> v1:172.21.15.181:6789/0 conn(0x7f4f700cb0e0 0x7f4f7015db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33576/0 (socket says 172.21.15.181:33576) 2023-10-16T11:00:47.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6f7fe700 1 --2- 172.21.15.181:0/4229241824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7006bef0 0x7f4f700caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6ffff700 1 -- 172.21.15.181:0/4229241824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7006bef0 msgr2=0x7f4f700caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:47.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6ffff700 1 --2- 172.21.15.181:0/4229241824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7006bef0 0x7f4f700caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6ffff700 1 -- 172.21.15.181:0/4229241824 >> v1:172.21.15.181:6789/0 conn(0x7f4f700cb0e0 legacy=0x7f4f7015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6ffff700 1 -- 172.21.15.181:0/4229241824 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7006e700 con 0x7f4f70159e10 2023-10-16T11:00:47.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6effd700 1 -- 172.21.15.181:0/4229241824 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477794192 0 0) 0x7f4f7006e4b0 con 0x7f4f700cb0e0 2023-10-16T11:00:47.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.729+0000 7f4f6f7fe700 1 --2- 172.21.15.181:0/4229241824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7006bef0 0x7f4f700caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:47.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f6ffff700 1 --2- 172.21.15.181:0/4229241824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f70159e10 0x7f4f700ce5a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4f640098f0 tx=0x7f4f64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa20e1c8013037c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f6effd700 1 -- 172.21.15.181:0/4229241824 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6400b8a0 con 0x7f4f70159e10 2023-10-16T11:00:47.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f6effd700 1 -- 172.21.15.181:0/4229241824 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6400ba00 con 0x7f4f70159e10 2023-10-16T11:00:47.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f6effd700 1 -- 172.21.15.181:0/4229241824 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6400bd90 con 0x7f4f70159e10 2023-10-16T11:00:47.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 -- 172.21.15.181:0/4229241824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f70159e10 msgr2=0x7f4f700ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 --2- 172.21.15.181:0/4229241824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f70159e10 0x7f4f700ce5a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4f640098f0 tx=0x7f4f64009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:47.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 -- 172.21.15.181:0/4229241824 shutdown_connections 2023-10-16T11:00:47.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 --2- 172.21.15.181:0/4229241824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f70159e10 0x7f4f700ce5a0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 --2- 172.21.15.181:0/4229241824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7006bef0 0x7f4f700caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 -- 172.21.15.181:0/4229241824 >> 172.21.15.181:0/4229241824 conn(0x7f4f700cc730 msgr2=0x7f4f700ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:47.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 -- 172.21.15.181:0/4229241824 shutdown_connections 2023-10-16T11:00:47.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.730+0000 7f4f76455700 1 -- 172.21.15.181:0/4229241824 wait complete. 2023-10-16T11:00:47.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.732+0000 7f4f76455700 1 Processor -- start 2023-10-16T11:00:47.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.732+0000 7f4f76455700 1 -- start start 2023-10-16T11:00:47.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.732+0000 7f4f76455700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7006bef0 0x7f4f70156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.732+0000 7f4f76455700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f700cb0e0 0x7f4f7015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f76455700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f70159e10 0x7f4f70154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f76455700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f70157330 con 0x7f4f7006bef0 2023-10-16T11:00:47.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f76455700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f70156a90 con 0x7f4f70159e10 2023-10-16T11:00:47.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f76455700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f70156d70 con 0x7f4f700cb0e0 2023-10-16T11:00:47.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7006bef0 0x7f4f70156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7006bef0 0x7f4f70156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50734/0 (socket says 172.21.15.181:50734) 2023-10-16T11:00:47.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 -- 172.21.15.181:0/1680758227 learned_addr learned my addr 172.21.15.181:0/1680758227 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:47.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 -- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f700cb0e0 msgr2=0x7f4f7015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f700cb0e0 0x7f4f7015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 -- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f70159e10 msgr2=0x7f4f70154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:47.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f749f2700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f70159e10 0x7f4f70154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f70159e10 0x7f4f70154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:47.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 -- 172.21.15.181:0/1680758227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f64009550 con 0x7f4f7006bef0 2023-10-16T11:00:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f749f2700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f70159e10 0x7f4f70154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6ffff700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7006bef0 0x7f4f70156260 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f4f64000c00 tx=0x7f4f64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6cff9700 1 -- 172.21.15.181:0/1680758227 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f640038a0 con 0x7f4f7006bef0 2023-10-16T11:00:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6cff9700 1 -- 172.21.15.181:0/1680758227 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6401e070 con 0x7f4f7006bef0 2023-10-16T11:00:47.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f6cff9700 1 -- 172.21.15.181:0/1680758227 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64003c20 con 0x7f4f7006bef0 2023-10-16T11:00:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f70157050 con 0x7f4f7006bef0 2023-10-16T11:00:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.733+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f701567a0 con 0x7f4f7006bef0 2023-10-16T11:00:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.734+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f700ac880 con 0x7f4f7006bef0 2023-10-16T11:00:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.735+0000 7f4f6cff9700 1 -- 172.21.15.181:0/1680758227 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4f64011420 con 0x7f4f7006bef0 2023-10-16T11:00:47.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.735+0000 7f4f6cff9700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4f5c06b020 0x7f4f5c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.735+0000 7f4f6f7fe700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4f5c06b020 0x7f4f5c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.735+0000 7f4f6cff9700 1 -- 172.21.15.181:0/1680758227 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f4f64020070 con 0x7f4f7006bef0 2023-10-16T11:00:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.736+0000 7f4f6f7fe700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4f5c06b020 0x7f4f5c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4f7015d3a0 tx=0x7f4f60008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.740+0000 7f4f6cff9700 1 -- 172.21.15.181:0/1680758227 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f700ac880 con 0x7f4f7006bef0 2023-10-16T11:00:47.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:47.980+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4f700ac880 con 0x7f4f7006bef0 2023-10-16T11:00:48.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.899+0000 7f4f6cff9700 1 -- 172.21.15.181:0/1680758227 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4f700ac880 con 0x7f4f7006bef0 2023-10-16T11:00:48.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4f5c06b020 msgr2=0x7f4f5c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:48.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4f5c06b020 0x7f4f5c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4f7015d3a0 tx=0x7f4f60008040 comp rx=0 tx=0).stop 2023-10-16T11:00:48.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7006bef0 msgr2=0x7f4f70156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:48.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7006bef0 0x7f4f70156260 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f4f64000c00 tx=0x7f4f64004750 comp rx=0 tx=0).stop 2023-10-16T11:00:48.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 shutdown_connections 2023-10-16T11:00:48.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f4f5c06b020 0x7f4f5c06d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:48.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f70159e10 0x7f4f70154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:48.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7006bef0 0x7f4f70156260 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:48.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 --2- 172.21.15.181:0/1680758227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f700cb0e0 0x7f4f7015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:48.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 >> 172.21.15.181:0/1680758227 conn(0x7f4f700cc730 msgr2=0x7f4f7006f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:48.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 shutdown_connections 2023-10-16T11:00:48.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:48.902+0000 7f4f76455700 1 -- 172.21.15.181:0/1680758227 wait complete. 2023-10-16T11:00:48.919 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:00:48.920 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:00:49.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.047+0000 7f1180ca7700 1 Processor -- start 2023-10-16T11:00:49.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.047+0000 7f1180ca7700 1 -- start start 2023-10-16T11:00:49.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1180ca7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c15da70 0x7f117c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:49.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1180ca7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f117c0ce6a0 0x7f117c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:49.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1180ca7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c0cef00 con 0x7f117c072760 2023-10-16T11:00:49.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1180ca7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c0cec20 con 0x7f117c0ce6a0 2023-10-16T11:00:49.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1180ca7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c071c80 con 0x7f117c15da70 2023-10-16T11:00:49.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1179d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f117c0ce6a0 0x7f117c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:49.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1179d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f117c0ce6a0 0x7f117c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50742/0 (socket says 172.21.15.181:50742) 2023-10-16T11:00:49.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f1179d9b700 1 -- 172.21.15.181:0/1887334435 learned_addr learned my addr 172.21.15.181:0/1887334435 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:49.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.048+0000 7f117ad9d700 1 --1- 172.21.15.181:0/1887334435 >> v1:172.21.15.181:6789/0 conn(0x7f117c072760 0x7f117c1615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33590/0 (socket says 172.21.15.181:33590) 2023-10-16T11:00:49.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f117a59c700 1 --2- 172.21.15.181:0/1887334435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c15da70 0x7f117c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:49.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1179d9b700 1 -- 172.21.15.181:0/1887334435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c15da70 msgr2=0x7f117c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:49.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1179d9b700 1 --2- 172.21.15.181:0/1887334435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c15da70 0x7f117c0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1179d9b700 1 -- 172.21.15.181:0/1887334435 >> v1:172.21.15.181:6789/0 conn(0x7f117c072760 legacy=0x7f117c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:49.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1179d9b700 1 -- 172.21.15.181:0/1887334435 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c071f60 con 0x7f117c0ce6a0 2023-10-16T11:00:49.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f117959a700 1 -- 172.21.15.181:0/1887334435 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969384765 0 0) 0x7f117c0cef00 con 0x7f117c072760 2023-10-16T11:00:49.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1179d9b700 1 --2- 172.21.15.181:0/1887334435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c0ce6a0 0x7f117c06a8a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f116c0098f0 tx=0x7f116c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad904fde04986f65 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:49.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f117959a700 1 -- 172.21.15.181:0/1887334435 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00b8a0 con 0x7f117c0ce6a0 2023-10-16T11:00:49.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f117959a700 1 -- 172.21.15.181:0/1887334435 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c00ba00 con 0x7f117c0ce6a0 2023-10-16T11:00:49.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f117959a700 1 -- 172.21.15.181:0/1887334435 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00bd90 con 0x7f117c0ce6a0 2023-10-16T11:00:49.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1180ca7700 1 -- 172.21.15.181:0/1887334435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c0ce6a0 msgr2=0x7f117c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:49.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1180ca7700 1 --2- 172.21.15.181:0/1887334435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c0ce6a0 0x7f117c06a8a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f116c0098f0 tx=0x7f116c009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:49.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1180ca7700 1 -- 172.21.15.181:0/1887334435 shutdown_connections 2023-10-16T11:00:49.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1180ca7700 1 --2- 172.21.15.181:0/1887334435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c15da70 0x7f117c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1180ca7700 1 --2- 172.21.15.181:0/1887334435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c0ce6a0 0x7f117c06a8a0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.049+0000 7f1180ca7700 1 -- 172.21.15.181:0/1887334435 >> 172.21.15.181:0/1887334435 conn(0x7f117c0cd010 msgr2=0x7f117c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:49.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.050+0000 7f1180ca7700 1 -- 172.21.15.181:0/1887334435 shutdown_connections 2023-10-16T11:00:49.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.050+0000 7f1180ca7700 1 -- 172.21.15.181:0/1887334435 wait complete. 2023-10-16T11:00:49.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.051+0000 7f1180ca7700 1 Processor -- start 2023-10-16T11:00:49.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.051+0000 7f1180ca7700 1 -- start start 2023-10-16T11:00:49.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f117c072760 0x7f117c0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:49.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c0ce6a0 0x7f117c0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:49.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c15da70 0x7f117c151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:49.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0aea50 con 0x7f117c15da70 2023-10-16T11:00:49.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0aed30 con 0x7f117c072760 2023-10-16T11:00:49.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0af010 con 0x7f117c0ce6a0 2023-10-16T11:00:49.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c15da70 0x7f117c151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:49.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c15da70 0x7f117c151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50750/0 (socket says 172.21.15.181:50750) 2023-10-16T11:00:49.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 -- 172.21.15.181:0/626856438 learned_addr learned my addr 172.21.15.181:0/626856438 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:49.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117a59c700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f117c072760 0x7f117c0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:49.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1179d9b700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c0ce6a0 0x7f117c0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:49.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 -- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c0ce6a0 msgr2=0x7f117c0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:49.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c0ce6a0 0x7f117c0c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 -- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f117c072760 msgr2=0x7f117c0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:49.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f117c072760 0x7f117c0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 -- 172.21.15.181:0/626856438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c009550 con 0x7f117c15da70 2023-10-16T11:00:49.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1179d9b700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c0ce6a0 0x7f117c0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:49.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117a59c700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f117c072760 0x7f117c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:49.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f117ad9d700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c15da70 0x7f117c151580 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f117400cba0 tx=0x7f117400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:49.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f116affd700 1 -- 172.21.15.181:0/626856438 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400ab80 con 0x7f117c15da70 2023-10-16T11:00:49.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f116affd700 1 -- 172.21.15.181:0/626856438 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400ad00 con 0x7f117c15da70 2023-10-16T11:00:49.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f116affd700 1 -- 172.21.15.181:0/626856438 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174007310 con 0x7f117c15da70 2023-10-16T11:00:49.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c160370 con 0x7f117c15da70 2023-10-16T11:00:49.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.052+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c0c9b20 con 0x7f117c15da70 2023-10-16T11:00:49.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.053+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c06f6c0 con 0x7f117c15da70 2023-10-16T11:00:49.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.054+0000 7f116affd700 1 -- 172.21.15.181:0/626856438 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1174007470 con 0x7f117c15da70 2023-10-16T11:00:49.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.054+0000 7f116affd700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f115c06b020 0x7f115c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:49.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.054+0000 7f116affd700 1 -- 172.21.15.181:0/626856438 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f1174089340 con 0x7f117c15da70 2023-10-16T11:00:49.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.054+0000 7f117a59c700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f115c06b020 0x7f115c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:49.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.055+0000 7f117a59c700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f115c06b020 0x7f115c06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f117c0ca660 tx=0x7f1170008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:49.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.058+0000 7f116affd700 1 -- 172.21.15.181:0/626856438 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1174054180 con 0x7f117c15da70 2023-10-16T11:00:49.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.297+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f117c0aeb90 con 0x7f117c15da70 2023-10-16T11:00:49.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.904+0000 7f116affd700 1 -- 172.21.15.181:0/626856438 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) v1 ==== 122+0+0 (secure 0 0 0) 0x7f117c0aeb90 con 0x7f117c15da70 2023-10-16T11:00:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f115c06b020 msgr2=0x7f115c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f115c06b020 0x7f115c06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f117c0ca660 tx=0x7f1170008040 comp rx=0 tx=0).stop 2023-10-16T11:00:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c15da70 msgr2=0x7f117c151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:49.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c15da70 0x7f117c151580 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f117400cba0 tx=0x7f117400cf60 comp rx=0 tx=0).stop 2023-10-16T11:00:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 shutdown_connections 2023-10-16T11:00:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f115c06b020 0x7f115c06d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f117c15da70 0x7f117c151580 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f117c072760 0x7f117c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 --2- 172.21.15.181:0/626856438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f117c0ce6a0 0x7f117c0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:49.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 >> 172.21.15.181:0/626856438 conn(0x7f117c0cd010 msgr2=0x7f117c073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:49.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 shutdown_connections 2023-10-16T11:00:49.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:49.908+0000 7f1180ca7700 1 -- 172.21.15.181:0/626856438 wait complete. 2023-10-16T11:00:49.926 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:00:49.926 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:00:50.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.053+0000 7fc96e0fd700 1 Processor -- start 2023-10-16T11:00:50.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.053+0000 7fc96e0fd700 1 -- start start 2023-10-16T11:00:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.053+0000 7fc96e0fd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc96815dfd0 0x7fc9680af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.053+0000 7fc96e0fd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 0x7fc96806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.053+0000 7fc96e0fd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96806f870 con 0x7fc96806db80 2023-10-16T11:00:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.053+0000 7fc96e0fd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680d1160 con 0x7fc96815dfd0 2023-10-16T11:00:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc96e0fd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680cc8e0 con 0x7fc9680af450 2023-10-16T11:00:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc967fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc96806db80 0x7fc9680cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33606/0 (socket says 172.21.15.181:33606) 2023-10-16T11:00:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc967fff700 1 -- 172.21.15.181:0/2820839453 learned_addr learned my addr 172.21.15.181:0/2820839453 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:50.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9677fe700 1 --2- 172.21.15.181:0/2820839453 >> v2:172.21.15.190:3300/0 conn(0x7fc96815dfd0 0x7fc9680af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc966ffd700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 0x7fc96806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347218164 0 0) 0x7fc96806f870 con 0x7fc96806db80 2023-10-16T11:00:50.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9540035b0 con 0x7fc96806db80 2023-10-16T11:00:50.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 836022957 0 0) 0x7fc9540035b0 con 0x7fc96806db80 2023-10-16T11:00:50.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 msgr2=0x7fc96806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 0x7fc96806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 >> v2:172.21.15.190:3300/0 conn(0x7fc96815dfd0 msgr2=0x7fc9680af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 --2- 172.21.15.181:0/2820839453 >> v2:172.21.15.190:3300/0 conn(0x7fc96815dfd0 0x7fc9680af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680ccbc0 con 0x7fc96806db80 2023-10-16T11:00:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc9600019f0 con 0x7fc96806db80 2023-10-16T11:00:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 >> v1:172.21.15.181:6789/0 conn(0x7fc96806db80 legacy=0x7fc9680cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 0x7fc954003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc954004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954004d90 0x7fc954005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680d1160 con 0x7fc954003f80 2023-10-16T11:00:50.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680cc8e0 con 0x7fc954004d90 2023-10-16T11:00:50.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95400ba60 con 0x7fc9680af450 2023-10-16T11:00:50.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc966ffd700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc954004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.054+0000 7fc9677fe700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 0x7fc954003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc967fff700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954004d90 0x7fc954005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc966ffd700 1 -- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 msgr2=0x7fc954003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc966ffd700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680af450 0x7fc954003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc966ffd700 1 -- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954004d90 msgr2=0x7fc954005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc966ffd700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954004d90 0x7fc954005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc966ffd700 1 -- 172.21.15.181:0/2820839453 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540035b0 con 0x7fc954003f80 2023-10-16T11:00:50.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc967fff700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc954004d90 0x7fc954005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:50.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc966ffd700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc954004850 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fc950009b90 tx=0x7fc954005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5e6f83f05e8096 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000e6c0 con 0x7fc954003f80 2023-10-16T11:00:50.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95000e820 con 0x7fc954003f80 2023-10-16T11:00:50.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc9667fc700 1 -- 172.21.15.181:0/2820839453 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000eb90 con 0x7fc954003f80 2023-10-16T11:00:50.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc96e0fd700 1 -- 172.21.15.181:0/2820839453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 msgr2=0x7fc954004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc96e0fd700 1 --2- 172.21.15.181:0/2820839453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc954004850 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fc950009b90 tx=0x7fc954005880 comp rx=0 tx=0).stop 2023-10-16T11:00:50.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc967fff700 1 -- 172.21.15.181:0/2820839453 reap_dead start 2023-10-16T11:00:50.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc96e0fd700 1 -- 172.21.15.181:0/2820839453 shutdown_connections 2023-10-16T11:00:50.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.055+0000 7fc96e0fd700 1 -- 172.21.15.181:0/2820839453 >> 172.21.15.181:0/2820839453 conn(0x7fc9680d04a0 msgr2=0x7fc9680d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:50.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.056+0000 7fc96e0fd700 1 -- 172.21.15.181:0/2820839453 shutdown_connections 2023-10-16T11:00:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.056+0000 7fc96e0fd700 1 -- 172.21.15.181:0/2820839453 wait complete. 2023-10-16T11:00:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.057+0000 7fc96e0fd700 1 Processor -- start 2023-10-16T11:00:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.057+0000 7fc96e0fd700 1 -- start start 2023-10-16T11:00:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc9680c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680c6ba0 0x7fc96815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc968154ac0 0x7fc968154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc968155490 con 0x7fc954003f80 2023-10-16T11:00:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680b4ea0 con 0x7fc968154ac0 2023-10-16T11:00:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680b5180 con 0x7fc9680c6ba0 2023-10-16T11:00:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc9680c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc9680c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50768/0 (socket says 172.21.15.181:50768) 2023-10-16T11:00:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 -- 172.21.15.181:0/452892183 learned_addr learned my addr 172.21.15.181:0/452892183 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc967fff700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc968154ac0 0x7fc968154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 -- 172.21.15.181:0/452892183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680c6ba0 msgr2=0x7fc96815c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680c6ba0 0x7fc96815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 -- 172.21.15.181:0/452892183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc968154ac0 msgr2=0x7fc968154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc968154ac0 0x7fc968154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 -- 172.21.15.181:0/452892183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680b5460 con 0x7fc954003f80 2023-10-16T11:00:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc9677fe700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc9680c5a30 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fc958011a70 tx=0x7fc958011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc95ffff700 1 -- 172.21.15.181:0/452892183 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958005a20 con 0x7fc954003f80 2023-10-16T11:00:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc95ffff700 1 -- 172.21.15.181:0/452892183 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958005b80 con 0x7fc954003f80 2023-10-16T11:00:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9680b5740 con 0x7fc954003f80 2023-10-16T11:00:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc95ffff700 1 -- 172.21.15.181:0/452892183 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800fa00 con 0x7fc954003f80 2023-10-16T11:00:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.058+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9680c5f70 con 0x7fc954003f80 2023-10-16T11:00:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.059+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc968067d20 con 0x7fc954003f80 2023-10-16T11:00:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.060+0000 7fc95ffff700 1 -- 172.21.15.181:0/452892183 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc95800fb60 con 0x7fc954003f80 2023-10-16T11:00:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.060+0000 7fc95ffff700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc94806b020 0x7fc94806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.060+0000 7fc95ffff700 1 -- 172.21.15.181:0/452892183 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fc958089dd0 con 0x7fc954003f80 2023-10-16T11:00:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.060+0000 7fc966ffd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc94806b020 0x7fc94806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.061+0000 7fc966ffd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc94806b020 0x7fc94806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9680caef0 tx=0x7fc950013000 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.065+0000 7fc95ffff700 1 -- 172.21.15.181:0/452892183 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc958054c10 con 0x7fc954003f80 2023-10-16T11:00:50.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.300+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9680b5d90 con 0x7fc954003f80 2023-10-16T11:00:50.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.300+0000 7fc95ffff700 1 -- 172.21.15.181:0/452892183 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4929 (secure 0 0 0) 0x7fc95808b090 con 0x7fc954003f80 2023-10-16T11:00:50.303 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:50.303 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1460537182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1460537182},{"type":"v1","addr":"172.21.15.190:6833","nonce":1460537182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3626960628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3626960628},{"type":"v1","addr":"172.21.15.190:6835","nonce":3626960628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1772227629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1772227629},{"type":"v1","addr":"172.21.15.181:6836","nonce":1772227629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:00:46.276243+0000","modified":"2023-10-16T11:00:49.298812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9651},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9651":{"gid":9651,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6837/2931073086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2931073086},{"type":"v1","addr":"172.21.15.181:6837","nonce":2931073086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:00:50.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc94806b020 msgr2=0x7fc94806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc94806b020 0x7fc94806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc9680caef0 tx=0x7fc950013000 comp rx=0 tx=0).stop 2023-10-16T11:00:50.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 msgr2=0x7fc9680c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc9680c5a30 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fc958011a70 tx=0x7fc958011e30 comp rx=0 tx=0).stop 2023-10-16T11:00:50.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 shutdown_connections 2023-10-16T11:00:50.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc94806b020 0x7fc94806d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc968154ac0 0x7fc968154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc954003f80 0x7fc9680c5a30 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.302+0000 7fc96e0fd700 1 --2- 172.21.15.181:0/452892183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9680c6ba0 0x7fc96815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.303+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 >> 172.21.15.181:0/452892183 conn(0x7fc9680d04a0 msgr2=0x7fc9680ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:50.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.303+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 shutdown_connections 2023-10-16T11:00:50.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.303+0000 7fc96e0fd700 1 -- 172.21.15.181:0/452892183 wait complete. 2023-10-16T11:00:50.307 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2023-10-16T11:00:50.317 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:00:50.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.435+0000 7fb402903700 1 Processor -- start 2023-10-16T11:00:50.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.435+0000 7fb402903700 1 -- start start 2023-10-16T11:00:50.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb402903700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3fc0af450 0x7fb3fc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb402903700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3fc0cd470 0x7fb3fc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb402903700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc06f870 con 0x7fb3fc15dfd0 2023-10-16T11:00:50.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb402903700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0d1160 con 0x7fb3fc0cd470 2023-10-16T11:00:50.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb402903700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3fc0cc8e0 con 0x7fb3fc0af450 2023-10-16T11:00:50.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3fc0cd470 0x7fb3fc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3fc0cd470 0x7fb3fc0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50780/0 (socket says 172.21.15.181:50780) 2023-10-16T11:00:50.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 -- 172.21.15.181:0/3437296478 learned_addr learned my addr 172.21.15.181:0/3437296478 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:50.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb3fb7fe700 1 --2- 172.21.15.181:0/3437296478 >> v2:172.21.15.190:3300/0 conn(0x7fb3fc0af450 0x7fb3fc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 -- 172.21.15.181:0/3437296478 >> v2:172.21.15.190:3300/0 conn(0x7fb3fc0af450 msgr2=0x7fb3fc0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 --2- 172.21.15.181:0/3437296478 >> v2:172.21.15.190:3300/0 conn(0x7fb3fc0af450 0x7fb3fc0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 -- 172.21.15.181:0/3437296478 >> v1:172.21.15.190:6789/0 conn(0x7fb3fc15dfd0 legacy=0x7fb3fc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 -- 172.21.15.181:0/3437296478 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0ccbc0 con 0x7fb3fc0cd470 2023-10-16T11:00:50.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb3fb7fe700 1 --2- 172.21.15.181:0/3437296478 >> v2:172.21.15.190:3300/0 conn(0x7fb3fc0af450 0x7fb3fc0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:50.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb400ea0700 1 --2- 172.21.15.181:0/3437296478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0cd470 0x7fb3fc0cd920 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0b3070 tx=0x7fb3f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fe83994eed01d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.436+0000 7fb3faffd700 1 -- 172.21.15.181:0/3437296478 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400e070 con 0x7fb3fc0cd470 2023-10-16T11:00:50.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb3faffd700 1 -- 172.21.15.181:0/3437296478 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f400b9c0 con 0x7fb3fc0cd470 2023-10-16T11:00:50.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb3faffd700 1 -- 172.21.15.181:0/3437296478 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400bd70 con 0x7fb3fc0cd470 2023-10-16T11:00:50.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 -- 172.21.15.181:0/3437296478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0cd470 msgr2=0x7fb3fc0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 --2- 172.21.15.181:0/3437296478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0cd470 0x7fb3fc0cd920 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0b3070 tx=0x7fb3f400b330 comp rx=0 tx=0).stop 2023-10-16T11:00:50.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 -- 172.21.15.181:0/3437296478 shutdown_connections 2023-10-16T11:00:50.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 --2- 172.21.15.181:0/3437296478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0cd470 0x7fb3fc0cd920 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 --2- 172.21.15.181:0/3437296478 >> v2:172.21.15.190:3300/0 conn(0x7fb3fc0af450 0x7fb3fc0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 -- 172.21.15.181:0/3437296478 >> 172.21.15.181:0/3437296478 conn(0x7fb3fc0d04a0 msgr2=0x7fb3fc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:50.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 -- 172.21.15.181:0/3437296478 shutdown_connections 2023-10-16T11:00:50.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.437+0000 7fb402903700 1 -- 172.21.15.181:0/3437296478 wait complete. 2023-10-16T11:00:50.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.438+0000 7fb402903700 1 Processor -- start 2023-10-16T11:00:50.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.438+0000 7fb402903700 1 -- start start 2023-10-16T11:00:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb402903700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0af450 0x7fb3fc0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb402903700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc0cd470 0x7fb3fc155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb402903700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc15dfd0 0x7fb3fc0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb402903700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b6150 con 0x7fb3fc0af450 2023-10-16T11:00:50.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb402903700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc15fcf0 con 0x7fb3fc0cd470 2023-10-16T11:00:50.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb402903700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0b2c70 con 0x7fb3fc15dfd0 2023-10-16T11:00:50.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb3fbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0af450 0x7fb3fc0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb3fbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0af450 0x7fb3fc0b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50786/0 (socket says 172.21.15.181:50786) 2023-10-16T11:00:50.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.439+0000 7fb3fbfff700 1 -- 172.21.15.181:0/917801017 learned_addr learned my addr 172.21.15.181:0/917801017 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:50.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb400ea0700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc15dfd0 0x7fb3fc0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3fb7fe700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc0cd470 0x7fb3fc155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3fbfff700 1 -- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc15dfd0 msgr2=0x7fb3fc0b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3fbfff700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc15dfd0 0x7fb3fc0b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3fbfff700 1 -- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc0cd470 msgr2=0x7fb3fc155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3fbfff700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc0cd470 0x7fb3fc155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3fbfff700 1 -- 172.21.15.181:0/917801017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009b30 con 0x7fb3fc0af450 2023-10-16T11:00:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3fbfff700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0af450 0x7fb3fc0b0bb0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c460 tx=0x7fb3f000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3f8ff9700 1 -- 172.21.15.181:0/917801017 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00052d0 con 0x7fb3fc0af450 2023-10-16T11:00:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3f8ff9700 1 -- 172.21.15.181:0/917801017 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f0005450 con 0x7fb3fc0af450 2023-10-16T11:00:50.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb3f8ff9700 1 -- 172.21.15.181:0/917801017 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0006af0 con 0x7fb3fc0af450 2023-10-16T11:00:50.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc15f6a0 con 0x7fb3fc0af450 2023-10-16T11:00:50.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.440+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc155fd0 con 0x7fb3fc0af450 2023-10-16T11:00:50.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.441+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc067d20 con 0x7fb3fc0af450 2023-10-16T11:00:50.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.442+0000 7fb3f8ff9700 1 -- 172.21.15.181:0/917801017 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb3f00036d0 con 0x7fb3fc0af450 2023-10-16T11:00:50.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.442+0000 7fb3f8ff9700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb3e806b020 0x7fb3e806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.442+0000 7fb3f8ff9700 1 -- 172.21.15.181:0/917801017 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fb3f0088b00 con 0x7fb3fc0af450 2023-10-16T11:00:50.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.442+0000 7fb3fb7fe700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb3e806b020 0x7fb3e806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.442+0000 7fb3fb7fe700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb3e806b020 0x7fb3e806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc09e460 tx=0x7fb3ec008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.446+0000 7fb3f8ff9700 1 -- 172.21.15.181:0/917801017 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb3f0053940 con 0x7fb3fc0af450 2023-10-16T11:00:50.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.641+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3fc0cffe0 con 0x7fb3fc0af450 2023-10-16T11:00:50.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.642+0000 7fb3f8ff9700 1 -- 172.21.15.181:0/917801017 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+17375 (secure 0 0 0) 0x7fb3f0056f60 con 0x7fb3fc0af450 2023-10-16T11:00:50.644 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:50.644 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":168,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:00:47.879964+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:00:44.693955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:00:45.236607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:00:50.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.644+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb3e806b020 msgr2=0x7fb3e806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.644+0000 7fb402903700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb3e806b020 0x7fb3e806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc09e460 tx=0x7fb3ec008040 comp rx=0 tx=0).stop 2023-10-16T11:00:50.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.644+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0af450 msgr2=0x7fb3fc0b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.644+0000 7fb402903700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0af450 0x7fb3fc0b0bb0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c460 tx=0x7fb3f000c820 comp rx=0 tx=0).stop 2023-10-16T11:00:50.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 shutdown_connections 2023-10-16T11:00:50.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb3e806b020 0x7fb3e806d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3fc15dfd0 0x7fb3fc0b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc0af450 0x7fb3fc0b0bb0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 --2- 172.21.15.181:0/917801017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3fc0cd470 0x7fb3fc155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 >> 172.21.15.181:0/917801017 conn(0x7fb3fc0d04a0 msgr2=0x7fb3fc0cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:50.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 shutdown_connections 2023-10-16T11:00:50.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.645+0000 7fb402903700 1 -- 172.21.15.181:0/917801017 wait complete. 2023-10-16T11:00:50.659 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:00:50.659 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:00:50.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.778+0000 7f460d983700 1 Processor -- start 2023-10-16T11:00:50.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.778+0000 7f460d983700 1 -- start start 2023-10-16T11:00:50.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f460d983700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f460815fdc0 0x7f4608070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f460d983700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460815bb80 con 0x7f460815b7a0 2023-10-16T11:00:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f460d983700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080718a0 con 0x7f46080cfaa0 2023-10-16T11:00:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f460d983700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080710f0 con 0x7f460815fdc0 2023-10-16T11:00:50.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4606ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f460815b7a0 0x7f46080cf390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33636/0 (socket says 172.21.15.181:33636) 2023-10-16T11:00:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4606ffd700 1 -- 172.21.15.181:0/3830552218 learned_addr learned my addr 172.21.15.181:0/3830552218 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f46077fe700 1 --2- 172.21.15.181:0/3830552218 >> v2:172.21.15.190:3300/0 conn(0x7f460815fdc0 0x7f4608070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912227691 0 0) 0x7f460815bb80 con 0x7f460815b7a0 2023-10-16T11:00:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f00035b0 con 0x7f460815b7a0 2023-10-16T11:00:50.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 683217005 0 0) 0x7f45f00035b0 con 0x7f460815b7a0 2023-10-16T11:00:50.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 >> v2:172.21.15.190:3300/0 conn(0x7f460815fdc0 msgr2=0x7f4608070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 --2- 172.21.15.181:0/3830552218 >> v2:172.21.15.190:3300/0 conn(0x7f460815fdc0 0x7f4608070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 >> v1:172.21.15.190:6789/0 conn(0x7f46080cfaa0 legacy=0x7f46080ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080712e0 con 0x7f460815b7a0 2023-10-16T11:00:50.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f45f8001670 con 0x7f460815b7a0 2023-10-16T11:00:50.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 >> v1:172.21.15.181:6789/0 conn(0x7f460815b7a0 legacy=0x7f46080cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45f0003a00 0x7f45f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45f0004850 0x7f45f0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f45f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f000c080 con 0x7f45f0005210 2023-10-16T11:00:50.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f000c1f0 con 0x7f45f0004850 2023-10-16T11:00:50.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f000c3b0 con 0x7f45f0003a00 2023-10-16T11:00:50.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.779+0000 7f46077fe700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f45f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f4606ffd700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45f0003a00 0x7f45f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46067fc700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45f0004850 0x7f45f0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46077fe700 1 -- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45f0003a00 msgr2=0x7f45f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46077fe700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45f0003a00 0x7f45f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46077fe700 1 -- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45f0004850 msgr2=0x7f45f0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:50.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46077fe700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45f0004850 0x7f45f0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46077fe700 1 -- 172.21.15.181:0/3830552218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f00035b0 con 0x7f45f0005210 2023-10-16T11:00:50.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46077fe700 1 -- 172.21.15.181:0/3830552218 reap_dead start 2023-10-16T11:00:50.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f46077fe700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f45f000ba10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4600009fc0 tx=0x7f45f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a438634053f08f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600003a20 con 0x7f45f0005210 2023-10-16T11:00:50.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460000f040 con 0x7f45f0005210 2023-10-16T11:00:50.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f4605ffb700 1 -- 172.21.15.181:0/3830552218 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600003da0 con 0x7f45f0005210 2023-10-16T11:00:50.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f460d983700 1 -- 172.21.15.181:0/3830552218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 msgr2=0x7f45f000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.780+0000 7f460d983700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f45f000ba10 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4600009fc0 tx=0x7f45f0005880 comp rx=0 tx=0).stop 2023-10-16T11:00:50.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.781+0000 7f460d983700 1 -- 172.21.15.181:0/3830552218 shutdown_connections 2023-10-16T11:00:50.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.781+0000 7f460d983700 1 --2- 172.21.15.181:0/3830552218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f45f000ba10 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.781+0000 7f460d983700 1 -- 172.21.15.181:0/3830552218 >> 172.21.15.181:0/3830552218 conn(0x7f460806aed0 msgr2=0x7f46080b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:50.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.781+0000 7f460d983700 1 -- 172.21.15.181:0/3830552218 shutdown_connections 2023-10-16T11:00:50.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.781+0000 7f460d983700 1 -- 172.21.15.181:0/3830552218 wait complete. 2023-10-16T11:00:50.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.782+0000 7f460d983700 1 Processor -- start 2023-10-16T11:00:50.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.782+0000 7f460d983700 1 -- start start 2023-10-16T11:00:50.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f4608159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46080c5440 0x7f46080adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46080ae130 0x7f46080b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080aed50 con 0x7f45f0005210 2023-10-16T11:00:50.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460815a200 con 0x7f46080c5440 2023-10-16T11:00:50.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080ae790 con 0x7f46080ae130 2023-10-16T11:00:50.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f4608159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f4608159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50802/0 (socket says 172.21.15.181:50802) 2023-10-16T11:00:50.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 -- 172.21.15.181:0/1143984576 learned_addr learned my addr 172.21.15.181:0/1143984576 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:50.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f46067fc700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46080c5440 0x7f46080adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 -- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46080ae130 msgr2=0x7f46080b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f46077fe700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46080ae130 0x7f46080b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46080ae130 0x7f46080b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 -- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46080c5440 msgr2=0x7f46080adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46080c5440 0x7f46080adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 -- 172.21.15.181:0/1143984576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080aea70 con 0x7f45f0005210 2023-10-16T11:00:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f4606ffd700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f4608159cc0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f45f8007290 tx=0x7f45f80075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f45effff700 1 -- 172.21.15.181:0/1143984576 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800f660 con 0x7f45f0005210 2023-10-16T11:00:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f45effff700 1 -- 172.21.15.181:0/1143984576 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800f7c0 con 0x7f45f0005210 2023-10-16T11:00:50.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46080c4c40 con 0x7f45f0005210 2023-10-16T11:00:50.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f45effff700 1 -- 172.21.15.181:0/1143984576 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800fb70 con 0x7f45f0005210 2023-10-16T11:00:50.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.783+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460815ea20 con 0x7f45f0005210 2023-10-16T11:00:50.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.784+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46080c5160 con 0x7f45f0005210 2023-10-16T11:00:50.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.785+0000 7f45effff700 1 -- 172.21.15.181:0/1143984576 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f45f800fcd0 con 0x7f45f0005210 2023-10-16T11:00:50.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.785+0000 7f45effff700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f45f406b020 0x7f45f406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.785+0000 7f45effff700 1 -- 172.21.15.181:0/1143984576 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f45f8089920 con 0x7f45f0005210 2023-10-16T11:00:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.785+0000 7f46067fc700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f45f406b020 0x7f45f406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.786+0000 7f46067fc700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f45f406b020 0x7f45f406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007fd0 tx=0x7f45fc009040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:50.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.790+0000 7f45effff700 1 -- 172.21.15.181:0/1143984576 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f46080c5160 con 0x7f45f0005210 2023-10-16T11:00:50.837 INFO:tasks.ceph.mds.c.smithi181.stderr:2023-10-16T11:00:50.836+0000 7f9aa3b42700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:00:50.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.982+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 --> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46080c4ac0 con 0x7f45f406b020 2023-10-16T11:00:50.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.990+0000 7f45effff700 1 -- 172.21.15.181:0/1143984576 <== mgr.9405 v2:172.21.15.181:6832/551790976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290270 (secure 0 0 0) 0x7f45fc0051d0 con 0x7f45f406b020 2023-10-16T11:00:50.995 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:50.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f45f406b020 msgr2=0x7f45f406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f45f406b020 0x7f45f406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007fd0 tx=0x7f45fc009040 comp rx=0 tx=0).stop 2023-10-16T11:00:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 msgr2=0x7f4608159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f4608159cc0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f45f8007290 tx=0x7f45f80075a0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 shutdown_connections 2023-10-16T11:00:50.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f45f406b020 0x7f45f406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46080ae130 0x7f46080b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f0005210 0x7f4608159cc0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.993+0000 7f460d983700 1 --2- 172.21.15.181:0/1143984576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46080c5440 0x7f46080adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:50.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.994+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 >> 172.21.15.181:0/1143984576 conn(0x7f460806aed0 msgr2=0x7f46080caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:50.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.994+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 shutdown_connections 2023-10-16T11:00:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:50.994+0000 7f460d983700 1 -- 172.21.15.181:0/1143984576 wait complete. 2023-10-16T11:00:51.000 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:00:51.008 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-16T11:00:50.354359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420904,"kb_used_data":17384,"kb_used_omap":12,"kb_used_meta":403379,"kb_avail":754553816,"statfs":{"total":773094113280,"available":772663107584,"internally_reserved":0,"allocated":17801216,"data_stored":9545521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":413060664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.486938"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904234+0000","last_change":"2023-10-16T11:00:46.901158+0000","last_active":"2023-10-16T11:00:46.904234+0000","last_peered":"2023-10-16T11:00:46.904234+0000","last_clean":"2023-10-16T11:00:46.904234+0000","last_became_active":"2023-10-16T11:00:46.900871+0000","last_became_peered":"2023-10-16T11:00:46.900871+0000","last_unstale":"2023-10-16T11:00:46.904234+0000","last_undegraded":"2023-10-16T11:00:46.904234+0000","last_fullsized":"2023-10-16T11:00:46.904234+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:54.690802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896026+0000","last_change":"2023-10-16T11:00:46.892431+0000","last_active":"2023-10-16T11:00:46.896026+0000","last_peered":"2023-10-16T11:00:46.896026+0000","last_clean":"2023-10-16T11:00:46.896026+0000","last_became_active":"2023-10-16T11:00:46.892174+0000","last_became_peered":"2023-10-16T11:00:46.892174+0000","last_unstale":"2023-10-16T11:00:46.896026+0000","last_undegraded":"2023-10-16T11:00:46.896026+0000","last_fullsized":"2023-10-16T11:00:46.896026+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:53:26.575660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911400+0000","last_change":"2023-10-16T11:00:46.907081+0000","last_active":"2023-10-16T11:00:46.911400+0000","last_peered":"2023-10-16T11:00:46.911400+0000","last_clean":"2023-10-16T11:00:46.911400+0000","last_became_active":"2023-10-16T11:00:46.906465+0000","last_became_peered":"2023-10-16T11:00:46.906465+0000","last_unstale":"2023-10-16T11:00:46.911400+0000","last_undegraded":"2023-10-16T11:00:46.911400+0000","last_fullsized":"2023-10-16T11:00:46.911400+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:39.003786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909893+0000","last_change":"2023-10-16T11:00:46.907933+0000","last_active":"2023-10-16T11:00:46.909893+0000","last_peered":"2023-10-16T11:00:46.909893+0000","last_clean":"2023-10-16T11:00:46.909893+0000","last_became_active":"2023-10-16T11:00:46.907478+0000","last_became_peered":"2023-10-16T11:00:46.907478+0000","last_unstale":"2023-10-16T11:00:46.909893+0000","last_undegraded":"2023-10-16T11:00:46.909893+0000","last_fullsized":"2023-10-16T11:00:46.909893+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:23:42.144165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.900537+0000","last_change":"2023-10-16T11:00:46.899238+0000","last_active":"2023-10-16T11:00:46.900537+0000","last_peered":"2023-10-16T11:00:46.900537+0000","last_clean":"2023-10-16T11:00:46.900537+0000","last_became_active":"2023-10-16T11:00:46.898563+0000","last_became_peered":"2023-10-16T11:00:46.898563+0000","last_unstale":"2023-10-16T11:00:46.900537+0000","last_undegraded":"2023-10-16T11:00:46.900537+0000","last_fullsized":"2023-10-16T11:00:46.900537+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:20.907400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909547+0000","last_change":"2023-10-16T11:00:46.908382+0000","last_active":"2023-10-16T11:00:46.909547+0000","last_peered":"2023-10-16T11:00:46.909547+0000","last_clean":"2023-10-16T11:00:46.909547+0000","last_became_active":"2023-10-16T11:00:46.908137+0000","last_became_peered":"2023-10-16T11:00:46.908137+0000","last_unstale":"2023-10-16T11:00:46.909547+0000","last_undegraded":"2023-10-16T11:00:46.909547+0000","last_fullsized":"2023-10-16T11:00:46.909547+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:40:39.263959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.899097+0000","last_change":"2023-10-16T11:00:46.895357+0000","last_active":"2023-10-16T11:00:46.899097+0000","last_peered":"2023-10-16T11:00:46.899097+0000","last_clean":"2023-10-16T11:00:46.899097+0000","last_became_active":"2023-10-16T11:00:46.895095+0000","last_became_peered":"2023-10-16T11:00:46.895095+0000","last_unstale":"2023-10-16T11:00:46.899097+0000","last_undegraded":"2023-10-16T11:00:46.899097+0000","last_fullsized":"2023-10-16T11:00:46.899097+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:52.739724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909237+0000","last_change":"2023-10-16T11:00:46.905806+0000","last_active":"2023-10-16T11:00:46.909237+0000","last_peered":"2023-10-16T11:00:46.909237+0000","last_clean":"2023-10-16T11:00:46.909237+0000","last_became_active":"2023-10-16T11:00:46.904841+0000","last_became_peered":"2023-10-16T11:00:46.904841+0000","last_unstale":"2023-10-16T11:00:46.909237+0000","last_undegraded":"2023-10-16T11:00:46.909237+0000","last_fullsized":"2023-10-16T11:00:46.909237+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:20:21.136039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.901502+0000","last_change":"2023-10-16T11:00:46.900758+0000","last_active":"2023-10-16T11:00:46.901502+0000","last_peered":"2023-10-16T11:00:46.901502+0000","last_clean":"2023-10-16T11:00:46.901502+0000","last_became_active":"2023-10-16T11:00:46.900368+0000","last_became_peered":"2023-10-16T11:00:46.900368+0000","last_unstale":"2023-10-16T11:00:46.901502+0000","last_undegraded":"2023-10-16T11:00:46.901502+0000","last_fullsized":"2023-10-16T11:00:46.901502+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:11.995275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.908609+0000","last_change":"2023-10-16T11:00:46.900799+0000","last_active":"2023-10-16T11:00:46.908609+0000","last_peered":"2023-10-16T11:00:46.908609+0000","last_clean":"2023-10-16T11:00:46.908609+0000","last_became_active":"2023-10-16T11:00:46.900432+0000","last_became_peered":"2023-10-16T11:00:46.900432+0000","last_unstale":"2023-10-16T11:00:46.908609+0000","last_undegraded":"2023-10-16T11:00:46.908609+0000","last_fullsized":"2023-10-16T11:00:46.908609+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:19.452012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909221+0000","last_change":"2023-10-16T11:00:46.905440+0000","last_active":"2023-10-16T11:00:46.909221+0000","last_peered":"2023-10-16T11:00:46.909221+0000","last_clean":"2023-10-16T11:00:46.909221+0000","last_became_active":"2023-10-16T11:00:46.905020+0000","last_became_peered":"2023-10-16T11:00:46.905020+0000","last_unstale":"2023-10-16T11:00:46.909221+0000","last_undegraded":"2023-10-16T11:00:46.909221+0000","last_fullsized":"2023-10-16T11:00:46.909221+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:18.477635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906052+0000","last_change":"2023-10-16T11:00:46.901198+0000","last_active":"2023-10-16T11:00:46.906052+0000","last_peered":"2023-10-16T11:00:46.906052+0000","last_clean":"2023-10-16T11:00:46.906052+0000","last_became_active":"2023-10-16T11:00:46.900880+0000","last_became_peered":"2023-10-16T11:00:46.900880+0000","last_unstale":"2023-10-16T11:00:46.906052+0000","last_undegraded":"2023-10-16T11:00:46.906052+0000","last_fullsized":"2023-10-16T11:00:46.906052+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:24.781899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.900560+0000","last_change":"2023-10-16T11:00:46.899449+0000","last_active":"2023-10-16T11:00:46.900560+0000","last_peered":"2023-10-16T11:00:46.900560+0000","last_clean":"2023-10-16T11:00:46.900560+0000","last_became_active":"2023-10-16T11:00:46.898952+0000","last_became_peered":"2023-10-16T11:00:46.898952+0000","last_unstale":"2023-10-16T11:00:46.900560+0000","last_undegraded":"2023-10-16T11:00:46.900560+0000","last_fullsized":"2023-10-16T11:00:46.900560+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:21.092522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905221+0000","last_change":"2023-10-16T11:00:46.903295+0000","last_active":"2023-10-16T11:00:46.905221+0000","last_peered":"2023-10-16T11:00:46.905221+0000","last_clean":"2023-10-16T11:00:46.905221+0000","last_became_active":"2023-10-16T11:00:46.903058+0000","last_became_peered":"2023-10-16T11:00:46.903058+0000","last_unstale":"2023-10-16T11:00:46.905221+0000","last_undegraded":"2023-10-16T11:00:46.905221+0000","last_fullsized":"2023-10-16T11:00:46.905221+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:02:01.876573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905588+0000","last_change":"2023-10-16T11:00:46.898600+0000","last_active":"2023-10-16T11:00:46.905588+0000","last_peered":"2023-10-16T11:00:46.905588+0000","last_clean":"2023-10-16T11:00:46.905588+0000","last_became_active":"2023-10-16T11:00:46.898239+0000","last_became_peered":"2023-10-16T11:00:46.898239+0000","last_unstale":"2023-10-16T11:00:46.905588+0000","last_undegraded":"2023-10-16T11:00:46.905588+0000","last_fullsized":"2023-10-16T11:00:46.905588+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:27.914080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.903172+0000","last_change":"2023-10-16T11:00:46.902225+0000","last_active":"2023-10-16T11:00:46.903172+0000","last_peered":"2023-10-16T11:00:46.903172+0000","last_clean":"2023-10-16T11:00:46.903172+0000","last_became_active":"2023-10-16T11:00:46.901724+0000","last_became_peered":"2023-10-16T11:00:46.901724+0000","last_unstale":"2023-10-16T11:00:46.903172+0000","last_undegraded":"2023-10-16T11:00:46.903172+0000","last_fullsized":"2023-10-16T11:00:46.903172+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:08.155992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905756+0000","last_change":"2023-10-16T11:00:46.898206+0000","last_active":"2023-10-16T11:00:46.905756+0000","last_peered":"2023-10-16T11:00:46.905756+0000","last_clean":"2023-10-16T11:00:46.905756+0000","last_became_active":"2023-10-16T11:00:46.897920+0000","last_became_peered":"2023-10-16T11:00:46.897920+0000","last_unstale":"2023-10-16T11:00:46.905756+0000","last_undegraded":"2023-10-16T11:00:46.905756+0000","last_fullsized":"2023-10-16T11:00:46.905756+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:12:31.132132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911382+0000","last_change":"2023-10-16T11:00:46.905164+0000","last_active":"2023-10-16T11:00:46.911382+0000","last_peered":"2023-10-16T11:00:46.911382+0000","last_clean":"2023-10-16T11:00:46.911382+0000","last_became_active":"2023-10-16T11:00:46.904837+0000","last_became_peered":"2023-10-16T11:00:46.904837+0000","last_unstale":"2023-10-16T11:00:46.911382+0000","last_undegraded":"2023-10-16T11:00:46.911382+0000","last_fullsized":"2023-10-16T11:00:46.911382+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:52.739724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904089+0000","last_change":"2023-10-16T11:00:46.899601+0000","last_active":"2023-10-16T11:00:46.904089+0000","last_peered":"2023-10-16T11:00:46.904089+0000","last_clean":"2023-10-16T11:00:46.904089+0000","last_became_active":"2023-10-16T11:00:46.898827+0000","last_became_peered":"2023-10-16T11:00:46.898827+0000","last_unstale":"2023-10-16T11:00:46.904089+0000","last_undegraded":"2023-10-16T11:00:46.904089+0000","last_fullsized":"2023-10-16T11:00:46.904089+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:32.446652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889460+0000","last_change":"2023-10-16T11:00:46.888871+0000","last_active":"2023-10-16T11:00:46.889460+0000","last_peered":"2023-10-16T11:00:46.889460+0000","last_clean":"2023-10-16T11:00:46.889460+0000","last_became_active":"2023-10-16T11:00:46.888509+0000","last_became_peered":"2023-10-16T11:00:46.888509+0000","last_unstale":"2023-10-16T11:00:46.889460+0000","last_undegraded":"2023-10-16T11:00:46.889460+0000","last_fullsized":"2023-10-16T11:00:46.889460+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:20.907400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.901396+0000","last_change":"2023-10-16T11:00:46.897885+0000","last_active":"2023-10-16T11:00:46.901396+0000","last_peered":"2023-10-16T11:00:46.901396+0000","last_clean":"2023-10-16T11:00:46.901396+0000","last_became_active":"2023-10-16T11:00:46.897596+0000","last_became_peered":"2023-10-16T11:00:46.897596+0000","last_unstale":"2023-10-16T11:00:46.901396+0000","last_undegraded":"2023-10-16T11:00:46.901396+0000","last_fullsized":"2023-10-16T11:00:46.901396+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:10.179682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904299+0000","last_change":"2023-10-16T11:00:46.897693+0000","last_active":"2023-10-16T11:00:46.904299+0000","last_peered":"2023-10-16T11:00:46.904299+0000","last_clean":"2023-10-16T11:00:46.904299+0000","last_became_active":"2023-10-16T11:00:46.897322+0000","last_became_peered":"2023-10-16T11:00:46.897322+0000","last_unstale":"2023-10-16T11:00:46.904299+0000","last_undegraded":"2023-10-16T11:00:46.904299+0000","last_fullsized":"2023-10-16T11:00:46.904299+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:02:43.364536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909962+0000","last_change":"2023-10-16T11:00:46.909284+0000","last_active":"2023-10-16T11:00:46.909962+0000","last_peered":"2023-10-16T11:00:46.909962+0000","last_clean":"2023-10-16T11:00:46.909962+0000","last_became_active":"2023-10-16T11:00:46.909021+0000","last_became_peered":"2023-10-16T11:00:46.909021+0000","last_unstale":"2023-10-16T11:00:46.909962+0000","last_undegraded":"2023-10-16T11:00:46.909962+0000","last_fullsized":"2023-10-16T11:00:46.909962+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:04.360171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906251+0000","last_change":"2023-10-16T11:00:46.898817+0000","last_active":"2023-10-16T11:00:46.906251+0000","last_peered":"2023-10-16T11:00:46.906251+0000","last_clean":"2023-10-16T11:00:46.906251+0000","last_became_active":"2023-10-16T11:00:46.898071+0000","last_became_peered":"2023-10-16T11:00:46.898071+0000","last_unstale":"2023-10-16T11:00:46.906251+0000","last_undegraded":"2023-10-16T11:00:46.906251+0000","last_fullsized":"2023-10-16T11:00:46.906251+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:33.787439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.901009+0000","last_change":"2023-10-16T11:00:46.899886+0000","last_active":"2023-10-16T11:00:46.901009+0000","last_peered":"2023-10-16T11:00:46.901009+0000","last_clean":"2023-10-16T11:00:46.901009+0000","last_became_active":"2023-10-16T11:00:46.899619+0000","last_became_peered":"2023-10-16T11:00:46.899619+0000","last_unstale":"2023-10-16T11:00:46.901009+0000","last_undegraded":"2023-10-16T11:00:46.901009+0000","last_fullsized":"2023-10-16T11:00:46.901009+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:09.653311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904265+0000","last_change":"2023-10-16T11:00:46.899553+0000","last_active":"2023-10-16T11:00:46.904265+0000","last_peered":"2023-10-16T11:00:46.904265+0000","last_clean":"2023-10-16T11:00:46.904265+0000","last_became_active":"2023-10-16T11:00:46.897016+0000","last_became_peered":"2023-10-16T11:00:46.897016+0000","last_unstale":"2023-10-16T11:00:46.904265+0000","last_undegraded":"2023-10-16T11:00:46.904265+0000","last_fullsized":"2023-10-16T11:00:46.904265+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:32.401288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.907258+0000","last_change":"2023-10-16T11:00:46.905993+0000","last_active":"2023-10-16T11:00:46.907258+0000","last_peered":"2023-10-16T11:00:46.907258+0000","last_clean":"2023-10-16T11:00:46.907258+0000","last_became_active":"2023-10-16T11:00:46.905539+0000","last_became_peered":"2023-10-16T11:00:46.905539+0000","last_unstale":"2023-10-16T11:00:46.907258+0000","last_undegraded":"2023-10-16T11:00:46.907258+0000","last_fullsized":"2023-10-16T11:00:46.907258+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:29.686971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.915978+0000","last_change":"2023-10-16T11:00:46.907147+0000","last_active":"2023-10-16T11:00:46.915978+0000","last_peered":"2023-10-16T11:00:46.915978+0000","last_clean":"2023-10-16T11:00:46.915978+0000","last_became_active":"2023-10-16T11:00:46.906870+0000","last_became_peered":"2023-10-16T11:00:46.906870+0000","last_unstale":"2023-10-16T11:00:46.915978+0000","last_undegraded":"2023-10-16T11:00:46.915978+0000","last_fullsized":"2023-10-16T11:00:46.915978+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:41.397200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909595+0000","last_change":"2023-10-16T11:00:46.908701+0000","last_active":"2023-10-16T11:00:46.909595+0000","last_peered":"2023-10-16T11:00:46.909595+0000","last_clean":"2023-10-16T11:00:46.909595+0000","last_became_active":"2023-10-16T11:00:46.908105+0000","last_became_peered":"2023-10-16T11:00:46.908105+0000","last_unstale":"2023-10-16T11:00:46.909595+0000","last_undegraded":"2023-10-16T11:00:46.909595+0000","last_fullsized":"2023-10-16T11:00:46.909595+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:13.819254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905203+0000","last_change":"2023-10-16T11:00:46.902285+0000","last_active":"2023-10-16T11:00:46.905203+0000","last_peered":"2023-10-16T11:00:46.905203+0000","last_clean":"2023-10-16T11:00:46.905203+0000","last_became_active":"2023-10-16T11:00:46.899319+0000","last_became_peered":"2023-10-16T11:00:46.899319+0000","last_unstale":"2023-10-16T11:00:46.905203+0000","last_undegraded":"2023-10-16T11:00:46.905203+0000","last_fullsized":"2023-10-16T11:00:46.905203+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:35.500097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906833+0000","last_change":"2023-10-16T11:00:46.905796+0000","last_active":"2023-10-16T11:00:46.906833+0000","last_peered":"2023-10-16T11:00:46.906833+0000","last_clean":"2023-10-16T11:00:46.906833+0000","last_became_active":"2023-10-16T11:00:46.905511+0000","last_became_peered":"2023-10-16T11:00:46.905511+0000","last_unstale":"2023-10-16T11:00:46.906833+0000","last_undegraded":"2023-10-16T11:00:46.906833+0000","last_fullsized":"2023-10-16T11:00:46.906833+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:47.131768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889435+0000","last_change":"2023-10-16T11:00:46.888001+0000","last_active":"2023-10-16T11:00:46.889435+0000","last_peered":"2023-10-16T11:00:46.889435+0000","last_clean":"2023-10-16T11:00:46.889435+0000","last_became_active":"2023-10-16T11:00:46.887721+0000","last_became_peered":"2023-10-16T11:00:46.887721+0000","last_unstale":"2023-10-16T11:00:46.889435+0000","last_undegraded":"2023-10-16T11:00:46.889435+0000","last_fullsized":"2023-10-16T11:00:46.889435+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:58.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.908648+0000","last_change":"2023-10-16T11:00:46.906734+0000","last_active":"2023-10-16T11:00:46.908648+0000","last_peered":"2023-10-16T11:00:46.908648+0000","last_clean":"2023-10-16T11:00:46.908648+0000","last_became_active":"2023-10-16T11:00:46.906368+0000","last_became_peered":"2023-10-16T11:00:46.906368+0000","last_unstale":"2023-10-16T11:00:46.908648+0000","last_undegraded":"2023-10-16T11:00:46.908648+0000","last_fullsized":"2023-10-16T11:00:46.908648+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:31.142626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"167'4","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.913403+0000","last_change":"2023-10-16T11:00:45.890693+0000","last_active":"2023-10-16T11:00:46.913403+0000","last_peered":"2023-10-16T11:00:46.913403+0000","last_clean":"2023-10-16T11:00:46.913403+0000","last_became_active":"2023-10-16T11:00:45.890406+0000","last_became_peered":"2023-10-16T11:00:45.890406+0000","last_unstale":"2023-10-16T11:00:46.913403+0000","last_undegraded":"2023-10-16T11:00:46.913403+0000","last_fullsized":"2023-10-16T11:00:46.913403+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:02.407707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.895577+0000","last_change":"2023-10-16T11:00:46.892500+0000","last_active":"2023-10-16T11:00:46.895577+0000","last_peered":"2023-10-16T11:00:46.895577+0000","last_clean":"2023-10-16T11:00:46.895577+0000","last_became_active":"2023-10-16T11:00:46.892261+0000","last_became_peered":"2023-10-16T11:00:46.892261+0000","last_unstale":"2023-10-16T11:00:46.895577+0000","last_undegraded":"2023-10-16T11:00:46.895577+0000","last_fullsized":"2023-10-16T11:00:46.895577+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:41.036049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911587+0000","last_change":"2023-10-16T11:00:45.885964+0000","last_active":"2023-10-16T11:00:46.911587+0000","last_peered":"2023-10-16T11:00:46.911587+0000","last_clean":"2023-10-16T11:00:46.911587+0000","last_became_active":"2023-10-16T11:00:45.885695+0000","last_became_peered":"2023-10-16T11:00:45.885695+0000","last_unstale":"2023-10-16T11:00:46.911587+0000","last_undegraded":"2023-10-16T11:00:46.911587+0000","last_fullsized":"2023-10-16T11:00:46.911587+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:26.907080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904143+0000","last_change":"2023-10-16T11:00:46.902256+0000","last_active":"2023-10-16T11:00:46.904143+0000","last_peered":"2023-10-16T11:00:46.904143+0000","last_clean":"2023-10-16T11:00:46.904143+0000","last_became_active":"2023-10-16T11:00:46.901953+0000","last_became_peered":"2023-10-16T11:00:46.901953+0000","last_unstale":"2023-10-16T11:00:46.904143+0000","last_undegraded":"2023-10-16T11:00:46.904143+0000","last_fullsized":"2023-10-16T11:00:46.904143+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:32:06.946008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.916460+0000","last_change":"2023-10-16T11:00:45.887292+0000","last_active":"2023-10-16T11:00:46.916460+0000","last_peered":"2023-10-16T11:00:46.916460+0000","last_clean":"2023-10-16T11:00:46.916460+0000","last_became_active":"2023-10-16T11:00:45.887063+0000","last_became_peered":"2023-10-16T11:00:45.887063+0000","last_unstale":"2023-10-16T11:00:46.916460+0000","last_undegraded":"2023-10-16T11:00:46.916460+0000","last_fullsized":"2023-10-16T11:00:46.916460+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:05.942415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906304+0000","last_change":"2023-10-16T11:00:46.901435+0000","last_active":"2023-10-16T11:00:46.906304+0000","last_peered":"2023-10-16T11:00:46.906304+0000","last_clean":"2023-10-16T11:00:46.906304+0000","last_became_active":"2023-10-16T11:00:46.901174+0000","last_became_peered":"2023-10-16T11:00:46.901174+0000","last_unstale":"2023-10-16T11:00:46.906304+0000","last_undegraded":"2023-10-16T11:00:46.906304+0000","last_fullsized":"2023-10-16T11:00:46.906304+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:40.738233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.910306+0000","last_change":"2023-10-16T11:00:45.885277+0000","last_active":"2023-10-16T11:00:46.910306+0000","last_peered":"2023-10-16T11:00:46.910306+0000","last_clean":"2023-10-16T11:00:46.910306+0000","last_became_active":"2023-10-16T11:00:45.884990+0000","last_became_peered":"2023-10-16T11:00:45.884990+0000","last_unstale":"2023-10-16T11:00:46.910306+0000","last_undegraded":"2023-10-16T11:00:46.910306+0000","last_fullsized":"2023-10-16T11:00:46.910306+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:49.599246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906930+0000","last_change":"2023-10-16T11:00:46.906304+0000","last_active":"2023-10-16T11:00:46.906930+0000","last_peered":"2023-10-16T11:00:46.906930+0000","last_clean":"2023-10-16T11:00:46.906930+0000","last_became_active":"2023-10-16T11:00:46.905969+0000","last_became_peered":"2023-10-16T11:00:46.905969+0000","last_unstale":"2023-10-16T11:00:46.906930+0000","last_undegraded":"2023-10-16T11:00:46.906930+0000","last_fullsized":"2023-10-16T11:00:46.906930+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:54:50.605587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.910635+0000","last_change":"2023-10-16T11:00:45.884820+0000","last_active":"2023-10-16T11:00:46.910635+0000","last_peered":"2023-10-16T11:00:46.910635+0000","last_clean":"2023-10-16T11:00:46.910635+0000","last_became_active":"2023-10-16T11:00:45.884526+0000","last_became_peered":"2023-10-16T11:00:45.884526+0000","last_unstale":"2023-10-16T11:00:46.910635+0000","last_undegraded":"2023-10-16T11:00:46.910635+0000","last_fullsized":"2023-10-16T11:00:46.910635+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:20.085523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909548+0000","last_change":"2023-10-16T11:00:46.906607+0000","last_active":"2023-10-16T11:00:46.909548+0000","last_peered":"2023-10-16T11:00:46.909548+0000","last_clean":"2023-10-16T11:00:46.909548+0000","last_became_active":"2023-10-16T11:00:46.906310+0000","last_became_peered":"2023-10-16T11:00:46.906310+0000","last_unstale":"2023-10-16T11:00:46.909548+0000","last_undegraded":"2023-10-16T11:00:46.909548+0000","last_fullsized":"2023-10-16T11:00:46.909548+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:18.176542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911914+0000","last_change":"2023-10-16T11:00:45.886046+0000","last_active":"2023-10-16T11:00:46.911914+0000","last_peered":"2023-10-16T11:00:46.911914+0000","last_clean":"2023-10-16T11:00:46.911914+0000","last_became_active":"2023-10-16T11:00:45.885765+0000","last_became_peered":"2023-10-16T11:00:45.885765+0000","last_unstale":"2023-10-16T11:00:46.911914+0000","last_undegraded":"2023-10-16T11:00:46.911914+0000","last_fullsized":"2023-10-16T11:00:46.911914+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:38.533483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889791+0000","last_change":"2023-10-16T11:00:46.889006+0000","last_active":"2023-10-16T11:00:46.889791+0000","last_peered":"2023-10-16T11:00:46.889791+0000","last_clean":"2023-10-16T11:00:46.889791+0000","last_became_active":"2023-10-16T11:00:46.888733+0000","last_became_peered":"2023-10-16T11:00:46.888733+0000","last_unstale":"2023-10-16T11:00:46.889791+0000","last_undegraded":"2023-10-16T11:00:46.889791+0000","last_fullsized":"2023-10-16T11:00:46.889791+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:22.471105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.885686+0000","last_change":"2023-10-16T11:00:45.888656+0000","last_active":"2023-10-16T11:00:46.885686+0000","last_peered":"2023-10-16T11:00:46.885686+0000","last_clean":"2023-10-16T11:00:46.885686+0000","last_became_active":"2023-10-16T11:00:45.888292+0000","last_became_peered":"2023-10-16T11:00:45.888292+0000","last_unstale":"2023-10-16T11:00:46.885686+0000","last_undegraded":"2023-10-16T11:00:46.885686+0000","last_fullsized":"2023-10-16T11:00:46.885686+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:38.533483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.902793+0000","last_change":"2023-10-16T11:00:46.901209+0000","last_active":"2023-10-16T11:00:46.902793+0000","last_peered":"2023-10-16T11:00:46.902793+0000","last_clean":"2023-10-16T11:00:46.902793+0000","last_became_active":"2023-10-16T11:00:46.900210+0000","last_became_peered":"2023-10-16T11:00:46.900210+0000","last_unstale":"2023-10-16T11:00:46.902793+0000","last_undegraded":"2023-10-16T11:00:46.902793+0000","last_fullsized":"2023-10-16T11:00:46.902793+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:33.966810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.892944+0000","last_change":"2023-10-16T11:00:45.885738+0000","last_active":"2023-10-16T11:00:46.892944+0000","last_peered":"2023-10-16T11:00:46.892944+0000","last_clean":"2023-10-16T11:00:46.892944+0000","last_became_active":"2023-10-16T11:00:45.885297+0000","last_became_peered":"2023-10-16T11:00:45.885297+0000","last_unstale":"2023-10-16T11:00:46.892944+0000","last_undegraded":"2023-10-16T11:00:46.892944+0000","last_fullsized":"2023-10-16T11:00:46.892944+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:08.936192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891978+0000","last_change":"2023-10-16T11:00:46.887561+0000","last_active":"2023-10-16T11:00:46.891978+0000","last_peered":"2023-10-16T11:00:46.891978+0000","last_clean":"2023-10-16T11:00:46.891978+0000","last_became_active":"2023-10-16T11:00:46.887158+0000","last_became_peered":"2023-10-16T11:00:46.887158+0000","last_unstale":"2023-10-16T11:00:46.891978+0000","last_undegraded":"2023-10-16T11:00:46.891978+0000","last_fullsized":"2023-10-16T11:00:46.891978+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:02.044252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.886076+0000","last_change":"2023-10-16T11:00:45.884257+0000","last_active":"2023-10-16T11:00:46.886076+0000","last_peered":"2023-10-16T11:00:46.886076+0000","last_clean":"2023-10-16T11:00:46.886076+0000","last_became_active":"2023-10-16T11:00:45.883979+0000","last_became_peered":"2023-10-16T11:00:45.883979+0000","last_unstale":"2023-10-16T11:00:46.886076+0000","last_undegraded":"2023-10-16T11:00:46.886076+0000","last_fullsized":"2023-10-16T11:00:46.886076+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:23.774900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905776+0000","last_change":"2023-10-16T11:00:46.898435+0000","last_active":"2023-10-16T11:00:46.905776+0000","last_peered":"2023-10-16T11:00:46.905776+0000","last_clean":"2023-10-16T11:00:46.905776+0000","last_became_active":"2023-10-16T11:00:46.898104+0000","last_became_peered":"2023-10-16T11:00:46.898104+0000","last_unstale":"2023-10-16T11:00:46.905776+0000","last_undegraded":"2023-10-16T11:00:46.905776+0000","last_fullsized":"2023-10-16T11:00:46.905776+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:10:57.310617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891179+0000","last_change":"2023-10-16T11:00:45.888648+0000","last_active":"2023-10-16T11:00:46.891179+0000","last_peered":"2023-10-16T11:00:46.891179+0000","last_clean":"2023-10-16T11:00:46.891179+0000","last_became_active":"2023-10-16T11:00:45.888357+0000","last_became_peered":"2023-10-16T11:00:45.888357+0000","last_unstale":"2023-10-16T11:00:46.891179+0000","last_undegraded":"2023-10-16T11:00:46.891179+0000","last_fullsized":"2023-10-16T11:00:46.891179+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:49:57.153979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.892417+0000","last_change":"2023-10-16T11:00:46.890350+0000","last_active":"2023-10-16T11:00:46.892417+0000","last_peered":"2023-10-16T11:00:46.892417+0000","last_clean":"2023-10-16T11:00:46.892417+0000","last_became_active":"2023-10-16T11:00:46.889954+0000","last_became_peered":"2023-10-16T11:00:46.889954+0000","last_unstale":"2023-10-16T11:00:46.892417+0000","last_undegraded":"2023-10-16T11:00:46.892417+0000","last_fullsized":"2023-10-16T11:00:46.892417+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:53.183222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"167'6","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.917722+0000","last_change":"2023-10-16T11:00:45.887197+0000","last_active":"2023-10-16T11:00:46.917722+0000","last_peered":"2023-10-16T11:00:46.917722+0000","last_clean":"2023-10-16T11:00:46.917722+0000","last_became_active":"2023-10-16T11:00:45.886934+0000","last_became_peered":"2023-10-16T11:00:45.886934+0000","last_unstale":"2023-10-16T11:00:46.917722+0000","last_undegraded":"2023-10-16T11:00:46.917722+0000","last_fullsized":"2023-10-16T11:00:46.917722+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:57.775529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911356+0000","last_change":"2023-10-16T11:00:46.905195+0000","last_active":"2023-10-16T11:00:46.911356+0000","last_peered":"2023-10-16T11:00:46.911356+0000","last_clean":"2023-10-16T11:00:46.911356+0000","last_became_active":"2023-10-16T11:00:46.904879+0000","last_became_peered":"2023-10-16T11:00:46.904879+0000","last_unstale":"2023-10-16T11:00:46.911356+0000","last_undegraded":"2023-10-16T11:00:46.911356+0000","last_fullsized":"2023-10-16T11:00:46.911356+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:29.686971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.916488+0000","last_change":"2023-10-16T11:00:45.889424+0000","last_active":"2023-10-16T11:00:46.916488+0000","last_peered":"2023-10-16T11:00:46.916488+0000","last_clean":"2023-10-16T11:00:46.916488+0000","last_became_active":"2023-10-16T11:00:45.889120+0000","last_became_peered":"2023-10-16T11:00:45.889120+0000","last_unstale":"2023-10-16T11:00:46.916488+0000","last_undegraded":"2023-10-16T11:00:46.916488+0000","last_fullsized":"2023-10-16T11:00:46.916488+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:48.971019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891906+0000","last_change":"2023-10-16T11:00:46.890015+0000","last_active":"2023-10-16T11:00:46.891906+0000","last_peered":"2023-10-16T11:00:46.891906+0000","last_clean":"2023-10-16T11:00:46.891906+0000","last_became_active":"2023-10-16T11:00:46.889757+0000","last_became_peered":"2023-10-16T11:00:46.889757+0000","last_unstale":"2023-10-16T11:00:46.891906+0000","last_undegraded":"2023-10-16T11:00:46.891906+0000","last_fullsized":"2023-10-16T11:00:46.891906+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.664803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896659+0000","last_change":"2023-10-16T11:00:45.888150+0000","last_active":"2023-10-16T11:00:46.896659+0000","last_peered":"2023-10-16T11:00:46.896659+0000","last_clean":"2023-10-16T11:00:46.896659+0000","last_became_active":"2023-10-16T11:00:45.887892+0000","last_became_peered":"2023-10-16T11:00:45.887892+0000","last_unstale":"2023-10-16T11:00:46.896659+0000","last_undegraded":"2023-10-16T11:00:46.896659+0000","last_fullsized":"2023-10-16T11:00:46.896659+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:20.085523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.893310+0000","last_change":"2023-10-16T11:00:46.891811+0000","last_active":"2023-10-16T11:00:46.893310+0000","last_peered":"2023-10-16T11:00:46.893310+0000","last_clean":"2023-10-16T11:00:46.893310+0000","last_became_active":"2023-10-16T11:00:46.891322+0000","last_became_peered":"2023-10-16T11:00:46.891322+0000","last_unstale":"2023-10-16T11:00:46.893310+0000","last_undegraded":"2023-10-16T11:00:46.893310+0000","last_fullsized":"2023-10-16T11:00:46.893310+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:53.183222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896905+0000","last_change":"2023-10-16T11:00:45.885246+0000","last_active":"2023-10-16T11:00:46.896905+0000","last_peered":"2023-10-16T11:00:46.896905+0000","last_clean":"2023-10-16T11:00:46.896905+0000","last_became_active":"2023-10-16T11:00:45.884985+0000","last_became_peered":"2023-10-16T11:00:45.884985+0000","last_unstale":"2023-10-16T11:00:46.896905+0000","last_undegraded":"2023-10-16T11:00:46.896905+0000","last_fullsized":"2023-10-16T11:00:46.896905+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:52.696700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911178+0000","last_change":"2023-10-16T11:00:46.905063+0000","last_active":"2023-10-16T11:00:46.911178+0000","last_peered":"2023-10-16T11:00:46.911178+0000","last_clean":"2023-10-16T11:00:46.911178+0000","last_became_active":"2023-10-16T11:00:46.904452+0000","last_became_peered":"2023-10-16T11:00:46.904452+0000","last_unstale":"2023-10-16T11:00:46.911178+0000","last_undegraded":"2023-10-16T11:00:46.911178+0000","last_fullsized":"2023-10-16T11:00:46.911178+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:53.183222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"167'8","reported_seq":19,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.912395+0000","last_change":"2023-10-16T11:00:45.889231+0000","last_active":"2023-10-16T11:00:46.912395+0000","last_peered":"2023-10-16T11:00:46.912395+0000","last_clean":"2023-10-16T11:00:46.912395+0000","last_became_active":"2023-10-16T11:00:45.888928+0000","last_became_peered":"2023-10-16T11:00:45.888928+0000","last_unstale":"2023-10-16T11:00:46.912395+0000","last_undegraded":"2023-10-16T11:00:46.912395+0000","last_fullsized":"2023-10-16T11:00:46.912395+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:01.037253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.908631+0000","last_change":"2023-10-16T11:00:46.905961+0000","last_active":"2023-10-16T11:00:46.908631+0000","last_peered":"2023-10-16T11:00:46.908631+0000","last_clean":"2023-10-16T11:00:46.908631+0000","last_became_active":"2023-10-16T11:00:46.905654+0000","last_became_peered":"2023-10-16T11:00:46.905654+0000","last_unstale":"2023-10-16T11:00:46.908631+0000","last_undegraded":"2023-10-16T11:00:46.908631+0000","last_fullsized":"2023-10-16T11:00:46.908631+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:55.391317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.912036+0000","last_change":"2023-10-16T11:00:45.889667+0000","last_active":"2023-10-16T11:00:46.912036+0000","last_peered":"2023-10-16T11:00:46.912036+0000","last_clean":"2023-10-16T11:00:46.912036+0000","last_became_active":"2023-10-16T11:00:45.889357+0000","last_became_peered":"2023-10-16T11:00:45.889357+0000","last_unstale":"2023-10-16T11:00:46.912036+0000","last_undegraded":"2023-10-16T11:00:46.912036+0000","last_fullsized":"2023-10-16T11:00:46.912036+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:48.971019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.897403+0000","last_change":"2023-10-16T11:00:46.893546+0000","last_active":"2023-10-16T11:00:46.897403+0000","last_peered":"2023-10-16T11:00:46.897403+0000","last_clean":"2023-10-16T11:00:46.897403+0000","last_became_active":"2023-10-16T11:00:46.892620+0000","last_became_peered":"2023-10-16T11:00:46.892620+0000","last_unstale":"2023-10-16T11:00:46.897403+0000","last_undegraded":"2023-10-16T11:00:46.897403+0000","last_fullsized":"2023-10-16T11:00:46.897403+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:49.978018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.907292+0000","last_change":"2023-10-16T11:00:45.892515+0000","last_active":"2023-10-16T11:00:46.907292+0000","last_peered":"2023-10-16T11:00:46.907292+0000","last_clean":"2023-10-16T11:00:46.907292+0000","last_became_active":"2023-10-16T11:00:45.892170+0000","last_became_peered":"2023-10-16T11:00:45.892170+0000","last_unstale":"2023-10-16T11:00:46.907292+0000","last_undegraded":"2023-10-16T11:00:46.907292+0000","last_fullsized":"2023-10-16T11:00:46.907292+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:31:55.521169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911418+0000","last_change":"2023-10-16T11:00:46.907142+0000","last_active":"2023-10-16T11:00:46.911418+0000","last_peered":"2023-10-16T11:00:46.911418+0000","last_clean":"2023-10-16T11:00:46.911418+0000","last_became_active":"2023-10-16T11:00:46.906790+0000","last_became_peered":"2023-10-16T11:00:46.906790+0000","last_unstale":"2023-10-16T11:00:46.911418+0000","last_undegraded":"2023-10-16T11:00:46.911418+0000","last_fullsized":"2023-10-16T11:00:46.911418+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:08.155992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.910712+0000","last_change":"2023-10-16T11:00:45.888904+0000","last_active":"2023-10-16T11:00:46.910712+0000","last_peered":"2023-10-16T11:00:46.910712+0000","last_clean":"2023-10-16T11:00:46.910712+0000","last_became_active":"2023-10-16T11:00:45.888581+0000","last_became_peered":"2023-10-16T11:00:45.888581+0000","last_unstale":"2023-10-16T11:00:46.910712+0000","last_undegraded":"2023-10-16T11:00:46.910712+0000","last_fullsized":"2023-10-16T11:00:46.910712+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:32.959811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909574+0000","last_change":"2023-10-16T11:00:46.908448+0000","last_active":"2023-10-16T11:00:46.909574+0000","last_peered":"2023-10-16T11:00:46.909574+0000","last_clean":"2023-10-16T11:00:46.909574+0000","last_became_active":"2023-10-16T11:00:46.908159+0000","last_became_peered":"2023-10-16T11:00:46.908159+0000","last_unstale":"2023-10-16T11:00:46.909574+0000","last_undegraded":"2023-10-16T11:00:46.909574+0000","last_fullsized":"2023-10-16T11:00:46.909574+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.530601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.885677+0000","last_change":"2023-10-16T11:00:45.886341+0000","last_active":"2023-10-16T11:00:46.885677+0000","last_peered":"2023-10-16T11:00:46.885677+0000","last_clean":"2023-10-16T11:00:46.885677+0000","last_became_active":"2023-10-16T11:00:45.886073+0000","last_became_peered":"2023-10-16T11:00:45.886073+0000","last_unstale":"2023-10-16T11:00:46.885677+0000","last_undegraded":"2023-10-16T11:00:46.885677+0000","last_fullsized":"2023-10-16T11:00:46.885677+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:15.359818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904427+0000","last_change":"2023-10-16T11:00:46.899575+0000","last_active":"2023-10-16T11:00:46.904427+0000","last_peered":"2023-10-16T11:00:46.904427+0000","last_clean":"2023-10-16T11:00:46.904427+0000","last_became_active":"2023-10-16T11:00:46.898969+0000","last_became_peered":"2023-10-16T11:00:46.898969+0000","last_unstale":"2023-10-16T11:00:46.904427+0000","last_undegraded":"2023-10-16T11:00:46.904427+0000","last_fullsized":"2023-10-16T11:00:46.904427+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:09.091226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.916045+0000","last_change":"2023-10-16T11:00:45.894799+0000","last_active":"2023-10-16T11:00:46.916045+0000","last_peered":"2023-10-16T11:00:46.916045+0000","last_clean":"2023-10-16T11:00:46.916045+0000","last_became_active":"2023-10-16T11:00:45.894571+0000","last_became_peered":"2023-10-16T11:00:45.894571+0000","last_unstale":"2023-10-16T11:00:46.916045+0000","last_undegraded":"2023-10-16T11:00:46.916045+0000","last_fullsized":"2023-10-16T11:00:46.916045+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:52.176222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905773+0000","last_change":"2023-10-16T11:00:46.898436+0000","last_active":"2023-10-16T11:00:46.905773+0000","last_peered":"2023-10-16T11:00:46.905773+0000","last_clean":"2023-10-16T11:00:46.905773+0000","last_became_active":"2023-10-16T11:00:46.898171+0000","last_became_peered":"2023-10-16T11:00:46.898171+0000","last_unstale":"2023-10-16T11:00:46.905773+0000","last_undegraded":"2023-10-16T11:00:46.905773+0000","last_fullsized":"2023-10-16T11:00:46.905773+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:21:57.437473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889145+0000","last_change":"2023-10-16T11:00:45.888767+0000","last_active":"2023-10-16T11:00:46.889145+0000","last_peered":"2023-10-16T11:00:46.889145+0000","last_clean":"2023-10-16T11:00:46.889145+0000","last_became_active":"2023-10-16T11:00:45.888470+0000","last_became_peered":"2023-10-16T11:00:45.888470+0000","last_unstale":"2023-10-16T11:00:46.889145+0000","last_undegraded":"2023-10-16T11:00:46.889145+0000","last_fullsized":"2023-10-16T11:00:46.889145+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:15.359818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905163+0000","last_change":"2023-10-16T11:00:46.902037+0000","last_active":"2023-10-16T11:00:46.905163+0000","last_peered":"2023-10-16T11:00:46.905163+0000","last_clean":"2023-10-16T11:00:46.905163+0000","last_became_active":"2023-10-16T11:00:46.899414+0000","last_became_peered":"2023-10-16T11:00:46.899414+0000","last_unstale":"2023-10-16T11:00:46.905163+0000","last_undegraded":"2023-10-16T11:00:46.905163+0000","last_fullsized":"2023-10-16T11:00:46.905163+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:47.612853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889557+0000","last_change":"2023-10-16T11:00:45.887113+0000","last_active":"2023-10-16T11:00:46.889557+0000","last_peered":"2023-10-16T11:00:46.889557+0000","last_clean":"2023-10-16T11:00:46.889557+0000","last_became_active":"2023-10-16T11:00:45.886814+0000","last_became_peered":"2023-10-16T11:00:45.886814+0000","last_unstale":"2023-10-16T11:00:46.889557+0000","last_undegraded":"2023-10-16T11:00:46.889557+0000","last_fullsized":"2023-10-16T11:00:46.889557+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:44.525092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.915441+0000","last_change":"2023-10-16T11:00:46.898804+0000","last_active":"2023-10-16T11:00:46.915441+0000","last_peered":"2023-10-16T11:00:46.915441+0000","last_clean":"2023-10-16T11:00:46.915441+0000","last_became_active":"2023-10-16T11:00:46.898130+0000","last_became_peered":"2023-10-16T11:00:46.898130+0000","last_unstale":"2023-10-16T11:00:46.915441+0000","last_undegraded":"2023-10-16T11:00:46.915441+0000","last_fullsized":"2023-10-16T11:00:46.915441+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:39.003786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.894453+0000","last_change":"2023-10-16T11:00:45.888182+0000","last_active":"2023-10-16T11:00:46.894453+0000","last_peered":"2023-10-16T11:00:46.894453+0000","last_clean":"2023-10-16T11:00:46.894453+0000","last_became_active":"2023-10-16T11:00:45.887627+0000","last_became_peered":"2023-10-16T11:00:45.887627+0000","last_unstale":"2023-10-16T11:00:46.894453+0000","last_undegraded":"2023-10-16T11:00:46.894453+0000","last_fullsized":"2023-10-16T11:00:46.894453+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:40.390200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.902841+0000","last_change":"2023-10-16T11:00:46.900783+0000","last_active":"2023-10-16T11:00:46.902841+0000","last_peered":"2023-10-16T11:00:46.902841+0000","last_clean":"2023-10-16T11:00:46.902841+0000","last_became_active":"2023-10-16T11:00:46.899994+0000","last_became_peered":"2023-10-16T11:00:46.899994+0000","last_unstale":"2023-10-16T11:00:46.902841+0000","last_undegraded":"2023-10-16T11:00:46.902841+0000","last_fullsized":"2023-10-16T11:00:46.902841+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:09.653311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.912320+0000","last_change":"2023-10-16T11:00:45.892736+0000","last_active":"2023-10-16T11:00:46.912320+0000","last_peered":"2023-10-16T11:00:46.912320+0000","last_clean":"2023-10-16T11:00:46.912320+0000","last_became_active":"2023-10-16T11:00:45.892430+0000","last_became_peered":"2023-10-16T11:00:45.892430+0000","last_unstale":"2023-10-16T11:00:46.912320+0000","last_undegraded":"2023-10-16T11:00:46.912320+0000","last_fullsized":"2023-10-16T11:00:46.912320+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:59.311120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911241+0000","last_change":"2023-10-16T11:00:46.906117+0000","last_active":"2023-10-16T11:00:46.911241+0000","last_peered":"2023-10-16T11:00:46.911241+0000","last_clean":"2023-10-16T11:00:46.911241+0000","last_became_active":"2023-10-16T11:00:46.904628+0000","last_became_peered":"2023-10-16T11:00:46.904628+0000","last_unstale":"2023-10-16T11:00:46.911241+0000","last_undegraded":"2023-10-16T11:00:46.911241+0000","last_fullsized":"2023-10-16T11:00:46.911241+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:58.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896843+0000","last_change":"2023-10-16T11:00:45.884542+0000","last_active":"2023-10-16T11:00:46.896843+0000","last_peered":"2023-10-16T11:00:46.896843+0000","last_clean":"2023-10-16T11:00:46.896843+0000","last_became_active":"2023-10-16T11:00:45.884284+0000","last_became_peered":"2023-10-16T11:00:45.884284+0000","last_unstale":"2023-10-16T11:00:46.896843+0000","last_undegraded":"2023-10-16T11:00:46.896843+0000","last_fullsized":"2023-10-16T11:00:46.896843+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:35.551193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.898564+0000","last_change":"2023-10-16T11:00:46.897763+0000","last_active":"2023-10-16T11:00:46.898564+0000","last_peered":"2023-10-16T11:00:46.898564+0000","last_clean":"2023-10-16T11:00:46.898564+0000","last_became_active":"2023-10-16T11:00:46.897407+0000","last_became_peered":"2023-10-16T11:00:46.897407+0000","last_unstale":"2023-10-16T11:00:46.898564+0000","last_undegraded":"2023-10-16T11:00:46.898564+0000","last_fullsized":"2023-10-16T11:00:46.898564+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:48.456233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.897163+0000","last_change":"2023-10-16T11:00:45.889633+0000","last_active":"2023-10-16T11:00:46.897163+0000","last_peered":"2023-10-16T11:00:46.897163+0000","last_clean":"2023-10-16T11:00:46.897163+0000","last_became_active":"2023-10-16T11:00:45.889343+0000","last_became_peered":"2023-10-16T11:00:45.889343+0000","last_unstale":"2023-10-16T11:00:46.897163+0000","last_undegraded":"2023-10-16T11:00:46.897163+0000","last_fullsized":"2023-10-16T11:00:46.897163+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:20.127327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904208+0000","last_change":"2023-10-16T11:00:46.902735+0000","last_active":"2023-10-16T11:00:46.904208+0000","last_peered":"2023-10-16T11:00:46.904208+0000","last_clean":"2023-10-16T11:00:46.904208+0000","last_became_active":"2023-10-16T11:00:46.902290+0000","last_became_peered":"2023-10-16T11:00:46.902290+0000","last_unstale":"2023-10-16T11:00:46.904208+0000","last_undegraded":"2023-10-16T11:00:46.904208+0000","last_fullsized":"2023-10-16T11:00:46.904208+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:23.999858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896622+0000","last_change":"2023-10-16T11:00:45.884269+0000","last_active":"2023-10-16T11:00:46.896622+0000","last_peered":"2023-10-16T11:00:46.896622+0000","last_clean":"2023-10-16T11:00:46.896622+0000","last_became_active":"2023-10-16T11:00:45.884003+0000","last_became_peered":"2023-10-16T11:00:45.884003+0000","last_unstale":"2023-10-16T11:00:46.896622+0000","last_undegraded":"2023-10-16T11:00:46.896622+0000","last_fullsized":"2023-10-16T11:00:46.896622+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:03.949532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.899426+0000","last_change":"2023-10-16T11:00:46.897844+0000","last_active":"2023-10-16T11:00:46.899426+0000","last_peered":"2023-10-16T11:00:46.899426+0000","last_clean":"2023-10-16T11:00:46.899426+0000","last_became_active":"2023-10-16T11:00:46.897213+0000","last_became_peered":"2023-10-16T11:00:46.897213+0000","last_unstale":"2023-10-16T11:00:46.899426+0000","last_undegraded":"2023-10-16T11:00:46.899426+0000","last_fullsized":"2023-10-16T11:00:46.899426+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:41.397200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889329+0000","last_change":"2023-10-16T11:00:45.884337+0000","last_active":"2023-10-16T11:00:46.889329+0000","last_peered":"2023-10-16T11:00:46.889329+0000","last_clean":"2023-10-16T11:00:46.889329+0000","last_became_active":"2023-10-16T11:00:45.882283+0000","last_became_peered":"2023-10-16T11:00:45.882283+0000","last_unstale":"2023-10-16T11:00:46.889329+0000","last_undegraded":"2023-10-16T11:00:46.889329+0000","last_fullsized":"2023-10-16T11:00:46.889329+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:08.646311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906586+0000","last_change":"2023-10-16T11:00:46.905823+0000","last_active":"2023-10-16T11:00:46.906586+0000","last_peered":"2023-10-16T11:00:46.906586+0000","last_clean":"2023-10-16T11:00:46.906586+0000","last_became_active":"2023-10-16T11:00:46.905505+0000","last_became_peered":"2023-10-16T11:00:46.905505+0000","last_unstale":"2023-10-16T11:00:46.906586+0000","last_undegraded":"2023-10-16T11:00:46.906586+0000","last_fullsized":"2023-10-16T11:00:46.906586+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:58.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889097+0000","last_change":"2023-10-16T11:00:45.892408+0000","last_active":"2023-10-16T11:00:46.889097+0000","last_peered":"2023-10-16T11:00:46.889097+0000","last_clean":"2023-10-16T11:00:46.889097+0000","last_became_active":"2023-10-16T11:00:45.892125+0000","last_became_peered":"2023-10-16T11:00:45.892125+0000","last_unstale":"2023-10-16T11:00:46.889097+0000","last_undegraded":"2023-10-16T11:00:46.889097+0000","last_fullsized":"2023-10-16T11:00:46.889097+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:10.988275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906278+0000","last_change":"2023-10-16T11:00:46.898987+0000","last_active":"2023-10-16T11:00:46.906278+0000","last_peered":"2023-10-16T11:00:46.906278+0000","last_clean":"2023-10-16T11:00:46.906278+0000","last_became_active":"2023-10-16T11:00:46.898472+0000","last_became_peered":"2023-10-16T11:00:46.898472+0000","last_unstale":"2023-10-16T11:00:46.906278+0000","last_undegraded":"2023-10-16T11:00:46.906278+0000","last_fullsized":"2023-10-16T11:00:46.906278+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:41.036049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.895598+0000","last_change":"2023-10-16T11:00:45.892198+0000","last_active":"2023-10-16T11:00:46.895598+0000","last_peered":"2023-10-16T11:00:46.895598+0000","last_clean":"2023-10-16T11:00:46.895598+0000","last_became_active":"2023-10-16T11:00:45.891907+0000","last_became_peered":"2023-10-16T11:00:45.891907+0000","last_unstale":"2023-10-16T11:00:46.895598+0000","last_undegraded":"2023-10-16T11:00:46.895598+0000","last_fullsized":"2023-10-16T11:00:46.895598+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:08.646311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891945+0000","last_change":"2023-10-16T11:00:46.887240+0000","last_active":"2023-10-16T11:00:46.891945+0000","last_peered":"2023-10-16T11:00:46.891945+0000","last_clean":"2023-10-16T11:00:46.891945+0000","last_became_active":"2023-10-16T11:00:46.886908+0000","last_became_peered":"2023-10-16T11:00:46.886908+0000","last_unstale":"2023-10-16T11:00:46.891945+0000","last_undegraded":"2023-10-16T11:00:46.891945+0000","last_fullsized":"2023-10-16T11:00:46.891945+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:29.686971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.885184+0000","last_change":"2023-10-16T11:00:45.883424+0000","last_active":"2023-10-16T11:00:46.885184+0000","last_peered":"2023-10-16T11:00:46.885184+0000","last_clean":"2023-10-16T11:00:46.885184+0000","last_became_active":"2023-10-16T11:00:45.883148+0000","last_became_peered":"2023-10-16T11:00:45.883148+0000","last_unstale":"2023-10-16T11:00:46.885184+0000","last_undegraded":"2023-10-16T11:00:46.885184+0000","last_fullsized":"2023-10-16T11:00:46.885184+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:07.026016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904851+0000","last_change":"2023-10-16T11:00:46.898366+0000","last_active":"2023-10-16T11:00:46.904851+0000","last_peered":"2023-10-16T11:00:46.904851+0000","last_clean":"2023-10-16T11:00:46.904851+0000","last_became_active":"2023-10-16T11:00:46.898059+0000","last_became_peered":"2023-10-16T11:00:46.898059+0000","last_unstale":"2023-10-16T11:00:46.904851+0000","last_undegraded":"2023-10-16T11:00:46.904851+0000","last_fullsized":"2023-10-16T11:00:46.904851+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:08.155992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"167'3","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909297+0000","last_change":"2023-10-16T11:00:45.892089+0000","last_active":"2023-10-16T11:00:46.909297+0000","last_peered":"2023-10-16T11:00:46.909297+0000","last_clean":"2023-10-16T11:00:46.909297+0000","last_became_active":"2023-10-16T11:00:45.891821+0000","last_became_peered":"2023-10-16T11:00:45.891821+0000","last_unstale":"2023-10-16T11:00:46.909297+0000","last_undegraded":"2023-10-16T11:00:46.909297+0000","last_fullsized":"2023-10-16T11:00:46.909297+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:01.037253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"157'271","reported_seq":882,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-16T11:00:45.882158+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:00:45.882158+0000","last_peered":"2023-10-16T11:00:45.882158+0000","last_clean":"2023-10-16T11:00:45.882158+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:00:45.882158+0000","last_undegraded":"2023-10-16T11:00:45.882158+0000","last_fullsized":"2023-10-16T11:00:45.882158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543061,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":2682880,"data_stored":1640864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76700000000000002,"5min":0.78600000000000003,"15min":0.83499999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.754,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.378}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.89200000000000002},"last":0.54900000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.81100000000000005,"15min":0.88100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69799999999999995,"5min":0.81399999999999995,"15min":0.83899999999999997},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.85199999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.86699999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.876},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.308},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.85999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.89300000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.92400000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.621}]}]},{"osd":6,"up_from":14,"seq":60129543060,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.871,"15min":0.871},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.32600000000000001},"max":{"1min":0.75700000000000001,"5min":0.83399999999999996,"15min":0.86899999999999999},"last":0.61899999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.41699999999999998,"15min":0.36799999999999999},"max":{"1min":0.70599999999999996,"5min":0.82999999999999996,"15min":0.83699999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.629},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74099999999999999,"5min":0.83299999999999996,"15min":0.89200000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.623},"min":{"1min":0.51400000000000001,"5min":0.44800000000000001,"15min":0.40200000000000002},"max":{"1min":0.76200000000000001,"5min":0.82499999999999996,"15min":0.86099999999999999},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.44900000000000001,"15min":0.44500000000000001},"max":{"1min":0.79600000000000004,"5min":0.86499999999999999,"15min":0.874},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.442,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53000000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.621,"5min":0.72799999999999998,"15min":0.91100000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.90200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.35499999999999998},"max":{"1min":0.84599999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.34599999999999997},"max":{"1min":0.876,"5min":0.876,"15min":0.89600000000000002},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129543060,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55412,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316428,"statfs":{"total":96636764160,"available":96580022272,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.24199999999999999},"max":{"1min":0.59099999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42899999999999999,"15min":0.42199999999999999},"min":{"1min":0.32000000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56599999999999995,"5min":0.64400000000000002,"15min":0.68000000000000005},"last":0.44}]},{"osd":2,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.35199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.745},"last":0.34599999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.376},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.29899999999999999},"max":{"1min":0.63600000000000001,"5min":0.66300000000000003,"15min":0.73399999999999999},"last":0.39300000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":1.004,"15min":1.044},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.39500000000000002},"max":{"1min":0.89600000000000002,"5min":0.94399999999999995,"15min":1.0409999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.43099999999999999},"max":{"1min":0.86199999999999999,"5min":1.038,"15min":1.038},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.56699999999999995,"5min":0.45100000000000001,"15min":0.43099999999999999},"max":{"1min":1.0149999999999999,"5min":1.038,"15min":1.038},"last":0.66800000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.55000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":1.0169999999999999,"15min":1.105},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":0.94599999999999995,"5min":0.94699999999999995,"15min":1.091},"last":0.65200000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.1299999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.53500000000000003,"5min":0.47299999999999998,"15min":0.45700000000000002},"max":{"1min":0.96299999999999997,"5min":1.034,"15min":1.034},"last":0.70299999999999996}]}]},{"osd":0,"up_from":14,"seq":60129543061,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56024,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315816,"statfs":{"total":96636764160,"available":96579395584,"internally_reserved":0,"allocated":2691072,"data_stored":1641858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.22900000000000001},"max":{"1min":0.54500000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.218},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.69199999999999995},"last":0.83299999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.64200000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.86799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.72599999999999998},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66000000000000003,"5min":0.71599999999999997,"15min":0.74099999999999999},"last":0.90100000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.35899999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.0900000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.39700000000000002},"max":{"1min":0.79000000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.999,"15min":1.0820000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.67000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.48399999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.98699999999999999},"last":0.81999999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.746}]}]},{"osd":2,"up_from":14,"seq":60129543060,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59572,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312268,"statfs":{"total":96636764160,"available":96575762432,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.39700000000000002,"15min":0.38500000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.375},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.78900000000000003},"last":0.28000000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46100000000000002,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.26200000000000001},"max":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.67900000000000005},"last":0.35299999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.624,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.502,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.751},"last":0.40699999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.97999999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.42599999999999999},"max":{"1min":0.88300000000000001,"5min":0.98699999999999999,"15min":1.022},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86199999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68400000000000005,"15min":0.66900000000000004},"min":{"1min":0.45700000000000002,"5min":0.40999999999999998,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.48299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68799999999999994,"15min":0.67000000000000004},"min":{"1min":0.47299999999999998,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89800000000000002,"5min":1.159,"15min":1.159},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68999999999999995,"15min":0.67900000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.437},"max":{"1min":0.90200000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.49199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":1.121,"15min":1.121},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93200000000000005,"5min":1.095,"15min":1.095},"last":0.504}]}]},{"osd":3,"up_from":14,"seq":60129543061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2084864,"data_stored":1049934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.441},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59399999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52300000000000002,"5min":0.621,"15min":0.76400000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.441},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.66600000000000004},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28499999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.64700000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.28499999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.96999999999999997},"last":0.76800000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.92400000000000004,"15min":0.98499999999999999},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81399999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.0029999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.0349999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.995},"last":0.70499999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95599999999999996,"15min":0.95699999999999996},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94099999999999995,"5min":0.96599999999999997,"15min":0.98399999999999999},"last":0.66000000000000003}]}]},{"osd":4,"up_from":14,"seq":60129543060,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51252,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320588,"statfs":{"total":96636764160,"available":96584282112,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72999999999999998,"5min":0.83799999999999997,"15min":0.90100000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.32300000000000001},"max":{"1min":0.77500000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.622}]},{"osd":1,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.88400000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.68799999999999994,"5min":0.77400000000000002,"15min":0.80800000000000005},"last":0.39300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.34200000000000003},"max":{"1min":0.72499999999999998,"5min":0.86199999999999999,"15min":1.077},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41099999999999998},"max":{"1min":0.72099999999999997,"5min":0.81100000000000005,"15min":0.97899999999999998},"last":0.54600000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.84099999999999997},"last":0.499},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.48299999999999998,"5min":0.432,"15min":0.41099999999999998},"max":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.61799999999999999,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.86899999999999999,"15min":1.03},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71299999999999997,"5min":0.89100000000000001,"15min":0.96099999999999997},"last":0.66100000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.69099999999999995,"5min":0.88,"15min":0.88},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69299999999999995,"5min":0.83999999999999997,"15min":1.133},"last":0.72999999999999998}]}]},{"osd":5,"up_from":14,"seq":60129543061,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.437},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.79500000000000004,"5min":0.82599999999999996,"15min":0.83599999999999997},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.876},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.623,"15min":0.621},"min":{"1min":0.432,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.872,"15min":0.90400000000000003},"last":0.497}]},{"osd":2,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.34100000000000003,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.871,"15min":0.871},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.871},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.42799999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.86099999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.315},"max":{"1min":0.71099999999999997,"5min":0.753,"15min":0.78700000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.40799999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38300000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.33000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.84899999999999998},"last":0.63300000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63,"15min":0.623},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.36099999999999999},"max":{"1min":0.81999999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:00:51.014 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:00:51.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.167+0000 7fa39e7d9700 1 Processor -- start 2023-10-16T11:00:51.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.167+0000 7fa39e7d9700 1 -- start start 2023-10-16T11:00:51.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa39e7d9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa39815dfd0 0x7fa3980af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa39e7d9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa3980af450 0x7fa39806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa39e7d9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39806f870 con 0x7fa39806db80 2023-10-16T11:00:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa39e7d9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3980d1160 con 0x7fa39815dfd0 2023-10-16T11:00:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa39e7d9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3980cc8e0 con 0x7fa3980af450 2023-10-16T11:00:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa39815dfd0 0x7fa3980af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa39815dfd0 0x7fa3980af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50810/0 (socket says 172.21.15.181:50810) 2023-10-16T11:00:51.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 -- 172.21.15.181:0/3483685201 learned_addr learned my addr 172.21.15.181:0/3483685201 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 -- 172.21.15.181:0/3483685201 >> v2:172.21.15.190:3300/0 conn(0x7fa3980af450 msgr2=0x7fa39806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa3977fe700 1 --2- 172.21.15.181:0/3483685201 >> v2:172.21.15.190:3300/0 conn(0x7fa3980af450 0x7fa39806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 --2- 172.21.15.181:0/3483685201 >> v2:172.21.15.190:3300/0 conn(0x7fa3980af450 0x7fa39806d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 -- 172.21.15.181:0/3483685201 >> v1:172.21.15.190:6789/0 conn(0x7fa39806db80 legacy=0x7fa3980cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 -- 172.21.15.181:0/3483685201 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3980ccbc0 con 0x7fa39815dfd0 2023-10-16T11:00:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa397fff700 1 --2- 172.21.15.181:0/3483685201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa3980af9a0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa3980b3070 tx=0x7fa388009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c880e8f9eee596 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa396ffd700 1 -- 172.21.15.181:0/3483685201 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388005e30 con 0x7fa39815dfd0 2023-10-16T11:00:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa396ffd700 1 -- 172.21.15.181:0/3483685201 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa388006030 con 0x7fa39815dfd0 2023-10-16T11:00:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.168+0000 7fa396ffd700 1 -- 172.21.15.181:0/3483685201 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3880063e0 con 0x7fa39815dfd0 2023-10-16T11:00:51.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 -- 172.21.15.181:0/3483685201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 msgr2=0x7fa3980af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/3483685201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa3980af9a0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa3980b3070 tx=0x7fa388009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 -- 172.21.15.181:0/3483685201 shutdown_connections 2023-10-16T11:00:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/3483685201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa3980af9a0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/3483685201 >> v2:172.21.15.190:3300/0 conn(0x7fa3980af450 0x7fa39806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 -- 172.21.15.181:0/3483685201 >> 172.21.15.181:0/3483685201 conn(0x7fa3980d04a0 msgr2=0x7fa3980d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:51.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 -- 172.21.15.181:0/3483685201 shutdown_connections 2023-10-16T11:00:51.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.169+0000 7fa39e7d9700 1 -- 172.21.15.181:0/3483685201 wait complete. 2023-10-16T11:00:51.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.170+0000 7fa39e7d9700 1 Processor -- start 2023-10-16T11:00:51.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.171+0000 7fa39e7d9700 1 -- start start 2023-10-16T11:00:51.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa39806db80 0x7fa3980c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3980af450 0x7fa3980ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa39815a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980ca8e0 con 0x7fa39815dfd0 2023-10-16T11:00:51.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980c7b50 con 0x7fa3980af450 2023-10-16T11:00:51.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980c7e30 con 0x7fa39806db80 2023-10-16T11:00:51.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa39815a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa39815a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50820/0 (socket says 172.21.15.181:50820) 2023-10-16T11:00:51.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 -- 172.21.15.181:0/2044674457 learned_addr learned my addr 172.21.15.181:0/2044674457 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:51.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa3977fe700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3980af450 0x7fa3980ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa397fff700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa39806db80 0x7fa3980c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 -- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa39806db80 msgr2=0x7fa3980c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa39806db80 0x7fa3980c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 -- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3980af450 msgr2=0x7fa3980ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3980af450 0x7fa3980ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 -- 172.21.15.181:0/2044674457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388009500 con 0x7fa39815dfd0 2023-10-16T11:00:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa3977fe700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3980af450 0x7fa3980ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa397fff700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa39806db80 0x7fa3980c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39cd76700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa39815a7b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c7a0 tx=0x7fa39000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa394ff9700 1 -- 172.21.15.181:0/2044674457 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390003e10 con 0x7fa39815dfd0 2023-10-16T11:00:51.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa394ff9700 1 -- 172.21.15.181:0/2044674457 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa390005070 con 0x7fa39815dfd0 2023-10-16T11:00:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3980c8110 con 0x7fa39815dfd0 2023-10-16T11:00:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa394ff9700 1 -- 172.21.15.181:0/2044674457 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390005420 con 0x7fa39815dfd0 2023-10-16T11:00:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.172+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3980c86d0 con 0x7fa39815dfd0 2023-10-16T11:00:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.173+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980c88f0 con 0x7fa39815dfd0 2023-10-16T11:00:51.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.174+0000 7fa394ff9700 1 -- 172.21.15.181:0/2044674457 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa390005580 con 0x7fa39815dfd0 2023-10-16T11:00:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.174+0000 7fa394ff9700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fa38406b020 0x7fa38406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.174+0000 7fa394ff9700 1 -- 172.21.15.181:0/2044674457 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fa390088fe0 con 0x7fa39815dfd0 2023-10-16T11:00:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.174+0000 7fa397fff700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fa38406b020 0x7fa38406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.175+0000 7fa397fff700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fa38406b020 0x7fa38406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3980b3070 tx=0x7fa388008070 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:51.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.179+0000 7fa394ff9700 1 -- 172.21.15.181:0/2044674457 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa390053e20 con 0x7fa39815dfd0 2023-10-16T11:00:51.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.365+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 --> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3980cc590 con 0x7fa38406b020 2023-10-16T11:00:51.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.373+0000 7fa394ff9700 1 -- 172.21.15.181:0/2044674457 <== mgr.9405 v2:172.21.15.181:6832/551790976 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290270 (secure 0 0 0) 0x7fa388006c00 con 0x7fa38406b020 2023-10-16T11:00:51.377 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fa38406b020 msgr2=0x7fa38406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fa38406b020 0x7fa38406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3980b3070 tx=0x7fa388008070 comp rx=0 tx=0).stop 2023-10-16T11:00:51.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 msgr2=0x7fa39815a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa39815a7b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c7a0 tx=0x7fa39000cb60 comp rx=0 tx=0).stop 2023-10-16T11:00:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 shutdown_connections 2023-10-16T11:00:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fa38406b020 0x7fa38406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa39806db80 0x7fa3980c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa39815dfd0 0x7fa39815a7b0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 --2- 172.21.15.181:0/2044674457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3980af450 0x7fa3980ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 >> 172.21.15.181:0/2044674457 conn(0x7fa3980d04a0 msgr2=0x7fa3980ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 shutdown_connections 2023-10-16T11:00:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.376+0000 7fa39e7d9700 1 -- 172.21.15.181:0/2044674457 wait complete. 2023-10-16T11:00:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:00:51.390 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-16T11:00:50.354359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":420904,"kb_used_data":17384,"kb_used_omap":12,"kb_used_meta":403379,"kb_avail":754553816,"statfs":{"total":773094113280,"available":772663107584,"internally_reserved":0,"allocated":17801216,"data_stored":9545521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":413060664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.486938"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904234+0000","last_change":"2023-10-16T11:00:46.901158+0000","last_active":"2023-10-16T11:00:46.904234+0000","last_peered":"2023-10-16T11:00:46.904234+0000","last_clean":"2023-10-16T11:00:46.904234+0000","last_became_active":"2023-10-16T11:00:46.900871+0000","last_became_peered":"2023-10-16T11:00:46.900871+0000","last_unstale":"2023-10-16T11:00:46.904234+0000","last_undegraded":"2023-10-16T11:00:46.904234+0000","last_fullsized":"2023-10-16T11:00:46.904234+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:54.690802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896026+0000","last_change":"2023-10-16T11:00:46.892431+0000","last_active":"2023-10-16T11:00:46.896026+0000","last_peered":"2023-10-16T11:00:46.896026+0000","last_clean":"2023-10-16T11:00:46.896026+0000","last_became_active":"2023-10-16T11:00:46.892174+0000","last_became_peered":"2023-10-16T11:00:46.892174+0000","last_unstale":"2023-10-16T11:00:46.896026+0000","last_undegraded":"2023-10-16T11:00:46.896026+0000","last_fullsized":"2023-10-16T11:00:46.896026+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:53:26.575660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911400+0000","last_change":"2023-10-16T11:00:46.907081+0000","last_active":"2023-10-16T11:00:46.911400+0000","last_peered":"2023-10-16T11:00:46.911400+0000","last_clean":"2023-10-16T11:00:46.911400+0000","last_became_active":"2023-10-16T11:00:46.906465+0000","last_became_peered":"2023-10-16T11:00:46.906465+0000","last_unstale":"2023-10-16T11:00:46.911400+0000","last_undegraded":"2023-10-16T11:00:46.911400+0000","last_fullsized":"2023-10-16T11:00:46.911400+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:39.003786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909893+0000","last_change":"2023-10-16T11:00:46.907933+0000","last_active":"2023-10-16T11:00:46.909893+0000","last_peered":"2023-10-16T11:00:46.909893+0000","last_clean":"2023-10-16T11:00:46.909893+0000","last_became_active":"2023-10-16T11:00:46.907478+0000","last_became_peered":"2023-10-16T11:00:46.907478+0000","last_unstale":"2023-10-16T11:00:46.909893+0000","last_undegraded":"2023-10-16T11:00:46.909893+0000","last_fullsized":"2023-10-16T11:00:46.909893+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:23:42.144165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.900537+0000","last_change":"2023-10-16T11:00:46.899238+0000","last_active":"2023-10-16T11:00:46.900537+0000","last_peered":"2023-10-16T11:00:46.900537+0000","last_clean":"2023-10-16T11:00:46.900537+0000","last_became_active":"2023-10-16T11:00:46.898563+0000","last_became_peered":"2023-10-16T11:00:46.898563+0000","last_unstale":"2023-10-16T11:00:46.900537+0000","last_undegraded":"2023-10-16T11:00:46.900537+0000","last_fullsized":"2023-10-16T11:00:46.900537+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:20.907400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909547+0000","last_change":"2023-10-16T11:00:46.908382+0000","last_active":"2023-10-16T11:00:46.909547+0000","last_peered":"2023-10-16T11:00:46.909547+0000","last_clean":"2023-10-16T11:00:46.909547+0000","last_became_active":"2023-10-16T11:00:46.908137+0000","last_became_peered":"2023-10-16T11:00:46.908137+0000","last_unstale":"2023-10-16T11:00:46.909547+0000","last_undegraded":"2023-10-16T11:00:46.909547+0000","last_fullsized":"2023-10-16T11:00:46.909547+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:40:39.263959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.899097+0000","last_change":"2023-10-16T11:00:46.895357+0000","last_active":"2023-10-16T11:00:46.899097+0000","last_peered":"2023-10-16T11:00:46.899097+0000","last_clean":"2023-10-16T11:00:46.899097+0000","last_became_active":"2023-10-16T11:00:46.895095+0000","last_became_peered":"2023-10-16T11:00:46.895095+0000","last_unstale":"2023-10-16T11:00:46.899097+0000","last_undegraded":"2023-10-16T11:00:46.899097+0000","last_fullsized":"2023-10-16T11:00:46.899097+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:52.739724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909237+0000","last_change":"2023-10-16T11:00:46.905806+0000","last_active":"2023-10-16T11:00:46.909237+0000","last_peered":"2023-10-16T11:00:46.909237+0000","last_clean":"2023-10-16T11:00:46.909237+0000","last_became_active":"2023-10-16T11:00:46.904841+0000","last_became_peered":"2023-10-16T11:00:46.904841+0000","last_unstale":"2023-10-16T11:00:46.909237+0000","last_undegraded":"2023-10-16T11:00:46.909237+0000","last_fullsized":"2023-10-16T11:00:46.909237+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:20:21.136039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.901502+0000","last_change":"2023-10-16T11:00:46.900758+0000","last_active":"2023-10-16T11:00:46.901502+0000","last_peered":"2023-10-16T11:00:46.901502+0000","last_clean":"2023-10-16T11:00:46.901502+0000","last_became_active":"2023-10-16T11:00:46.900368+0000","last_became_peered":"2023-10-16T11:00:46.900368+0000","last_unstale":"2023-10-16T11:00:46.901502+0000","last_undegraded":"2023-10-16T11:00:46.901502+0000","last_fullsized":"2023-10-16T11:00:46.901502+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:11.995275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.908609+0000","last_change":"2023-10-16T11:00:46.900799+0000","last_active":"2023-10-16T11:00:46.908609+0000","last_peered":"2023-10-16T11:00:46.908609+0000","last_clean":"2023-10-16T11:00:46.908609+0000","last_became_active":"2023-10-16T11:00:46.900432+0000","last_became_peered":"2023-10-16T11:00:46.900432+0000","last_unstale":"2023-10-16T11:00:46.908609+0000","last_undegraded":"2023-10-16T11:00:46.908609+0000","last_fullsized":"2023-10-16T11:00:46.908609+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:19.452012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909221+0000","last_change":"2023-10-16T11:00:46.905440+0000","last_active":"2023-10-16T11:00:46.909221+0000","last_peered":"2023-10-16T11:00:46.909221+0000","last_clean":"2023-10-16T11:00:46.909221+0000","last_became_active":"2023-10-16T11:00:46.905020+0000","last_became_peered":"2023-10-16T11:00:46.905020+0000","last_unstale":"2023-10-16T11:00:46.909221+0000","last_undegraded":"2023-10-16T11:00:46.909221+0000","last_fullsized":"2023-10-16T11:00:46.909221+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:18.477635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906052+0000","last_change":"2023-10-16T11:00:46.901198+0000","last_active":"2023-10-16T11:00:46.906052+0000","last_peered":"2023-10-16T11:00:46.906052+0000","last_clean":"2023-10-16T11:00:46.906052+0000","last_became_active":"2023-10-16T11:00:46.900880+0000","last_became_peered":"2023-10-16T11:00:46.900880+0000","last_unstale":"2023-10-16T11:00:46.906052+0000","last_undegraded":"2023-10-16T11:00:46.906052+0000","last_fullsized":"2023-10-16T11:00:46.906052+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:24.781899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.900560+0000","last_change":"2023-10-16T11:00:46.899449+0000","last_active":"2023-10-16T11:00:46.900560+0000","last_peered":"2023-10-16T11:00:46.900560+0000","last_clean":"2023-10-16T11:00:46.900560+0000","last_became_active":"2023-10-16T11:00:46.898952+0000","last_became_peered":"2023-10-16T11:00:46.898952+0000","last_unstale":"2023-10-16T11:00:46.900560+0000","last_undegraded":"2023-10-16T11:00:46.900560+0000","last_fullsized":"2023-10-16T11:00:46.900560+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:21.092522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905221+0000","last_change":"2023-10-16T11:00:46.903295+0000","last_active":"2023-10-16T11:00:46.905221+0000","last_peered":"2023-10-16T11:00:46.905221+0000","last_clean":"2023-10-16T11:00:46.905221+0000","last_became_active":"2023-10-16T11:00:46.903058+0000","last_became_peered":"2023-10-16T11:00:46.903058+0000","last_unstale":"2023-10-16T11:00:46.905221+0000","last_undegraded":"2023-10-16T11:00:46.905221+0000","last_fullsized":"2023-10-16T11:00:46.905221+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:02:01.876573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905588+0000","last_change":"2023-10-16T11:00:46.898600+0000","last_active":"2023-10-16T11:00:46.905588+0000","last_peered":"2023-10-16T11:00:46.905588+0000","last_clean":"2023-10-16T11:00:46.905588+0000","last_became_active":"2023-10-16T11:00:46.898239+0000","last_became_peered":"2023-10-16T11:00:46.898239+0000","last_unstale":"2023-10-16T11:00:46.905588+0000","last_undegraded":"2023-10-16T11:00:46.905588+0000","last_fullsized":"2023-10-16T11:00:46.905588+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:27.914080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.903172+0000","last_change":"2023-10-16T11:00:46.902225+0000","last_active":"2023-10-16T11:00:46.903172+0000","last_peered":"2023-10-16T11:00:46.903172+0000","last_clean":"2023-10-16T11:00:46.903172+0000","last_became_active":"2023-10-16T11:00:46.901724+0000","last_became_peered":"2023-10-16T11:00:46.901724+0000","last_unstale":"2023-10-16T11:00:46.903172+0000","last_undegraded":"2023-10-16T11:00:46.903172+0000","last_fullsized":"2023-10-16T11:00:46.903172+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:08.155992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905756+0000","last_change":"2023-10-16T11:00:46.898206+0000","last_active":"2023-10-16T11:00:46.905756+0000","last_peered":"2023-10-16T11:00:46.905756+0000","last_clean":"2023-10-16T11:00:46.905756+0000","last_became_active":"2023-10-16T11:00:46.897920+0000","last_became_peered":"2023-10-16T11:00:46.897920+0000","last_unstale":"2023-10-16T11:00:46.905756+0000","last_undegraded":"2023-10-16T11:00:46.905756+0000","last_fullsized":"2023-10-16T11:00:46.905756+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:12:31.132132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911382+0000","last_change":"2023-10-16T11:00:46.905164+0000","last_active":"2023-10-16T11:00:46.911382+0000","last_peered":"2023-10-16T11:00:46.911382+0000","last_clean":"2023-10-16T11:00:46.911382+0000","last_became_active":"2023-10-16T11:00:46.904837+0000","last_became_peered":"2023-10-16T11:00:46.904837+0000","last_unstale":"2023-10-16T11:00:46.911382+0000","last_undegraded":"2023-10-16T11:00:46.911382+0000","last_fullsized":"2023-10-16T11:00:46.911382+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:52.739724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904089+0000","last_change":"2023-10-16T11:00:46.899601+0000","last_active":"2023-10-16T11:00:46.904089+0000","last_peered":"2023-10-16T11:00:46.904089+0000","last_clean":"2023-10-16T11:00:46.904089+0000","last_became_active":"2023-10-16T11:00:46.898827+0000","last_became_peered":"2023-10-16T11:00:46.898827+0000","last_unstale":"2023-10-16T11:00:46.904089+0000","last_undegraded":"2023-10-16T11:00:46.904089+0000","last_fullsized":"2023-10-16T11:00:46.904089+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:32.446652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889460+0000","last_change":"2023-10-16T11:00:46.888871+0000","last_active":"2023-10-16T11:00:46.889460+0000","last_peered":"2023-10-16T11:00:46.889460+0000","last_clean":"2023-10-16T11:00:46.889460+0000","last_became_active":"2023-10-16T11:00:46.888509+0000","last_became_peered":"2023-10-16T11:00:46.888509+0000","last_unstale":"2023-10-16T11:00:46.889460+0000","last_undegraded":"2023-10-16T11:00:46.889460+0000","last_fullsized":"2023-10-16T11:00:46.889460+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:20.907400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.901396+0000","last_change":"2023-10-16T11:00:46.897885+0000","last_active":"2023-10-16T11:00:46.901396+0000","last_peered":"2023-10-16T11:00:46.901396+0000","last_clean":"2023-10-16T11:00:46.901396+0000","last_became_active":"2023-10-16T11:00:46.897596+0000","last_became_peered":"2023-10-16T11:00:46.897596+0000","last_unstale":"2023-10-16T11:00:46.901396+0000","last_undegraded":"2023-10-16T11:00:46.901396+0000","last_fullsized":"2023-10-16T11:00:46.901396+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:10.179682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904299+0000","last_change":"2023-10-16T11:00:46.897693+0000","last_active":"2023-10-16T11:00:46.904299+0000","last_peered":"2023-10-16T11:00:46.904299+0000","last_clean":"2023-10-16T11:00:46.904299+0000","last_became_active":"2023-10-16T11:00:46.897322+0000","last_became_peered":"2023-10-16T11:00:46.897322+0000","last_unstale":"2023-10-16T11:00:46.904299+0000","last_undegraded":"2023-10-16T11:00:46.904299+0000","last_fullsized":"2023-10-16T11:00:46.904299+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:02:43.364536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909962+0000","last_change":"2023-10-16T11:00:46.909284+0000","last_active":"2023-10-16T11:00:46.909962+0000","last_peered":"2023-10-16T11:00:46.909962+0000","last_clean":"2023-10-16T11:00:46.909962+0000","last_became_active":"2023-10-16T11:00:46.909021+0000","last_became_peered":"2023-10-16T11:00:46.909021+0000","last_unstale":"2023-10-16T11:00:46.909962+0000","last_undegraded":"2023-10-16T11:00:46.909962+0000","last_fullsized":"2023-10-16T11:00:46.909962+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:04.360171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906251+0000","last_change":"2023-10-16T11:00:46.898817+0000","last_active":"2023-10-16T11:00:46.906251+0000","last_peered":"2023-10-16T11:00:46.906251+0000","last_clean":"2023-10-16T11:00:46.906251+0000","last_became_active":"2023-10-16T11:00:46.898071+0000","last_became_peered":"2023-10-16T11:00:46.898071+0000","last_unstale":"2023-10-16T11:00:46.906251+0000","last_undegraded":"2023-10-16T11:00:46.906251+0000","last_fullsized":"2023-10-16T11:00:46.906251+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:34:33.787439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.901009+0000","last_change":"2023-10-16T11:00:46.899886+0000","last_active":"2023-10-16T11:00:46.901009+0000","last_peered":"2023-10-16T11:00:46.901009+0000","last_clean":"2023-10-16T11:00:46.901009+0000","last_became_active":"2023-10-16T11:00:46.899619+0000","last_became_peered":"2023-10-16T11:00:46.899619+0000","last_unstale":"2023-10-16T11:00:46.901009+0000","last_undegraded":"2023-10-16T11:00:46.901009+0000","last_fullsized":"2023-10-16T11:00:46.901009+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:09.653311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904265+0000","last_change":"2023-10-16T11:00:46.899553+0000","last_active":"2023-10-16T11:00:46.904265+0000","last_peered":"2023-10-16T11:00:46.904265+0000","last_clean":"2023-10-16T11:00:46.904265+0000","last_became_active":"2023-10-16T11:00:46.897016+0000","last_became_peered":"2023-10-16T11:00:46.897016+0000","last_unstale":"2023-10-16T11:00:46.904265+0000","last_undegraded":"2023-10-16T11:00:46.904265+0000","last_fullsized":"2023-10-16T11:00:46.904265+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:32.401288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.907258+0000","last_change":"2023-10-16T11:00:46.905993+0000","last_active":"2023-10-16T11:00:46.907258+0000","last_peered":"2023-10-16T11:00:46.907258+0000","last_clean":"2023-10-16T11:00:46.907258+0000","last_became_active":"2023-10-16T11:00:46.905539+0000","last_became_peered":"2023-10-16T11:00:46.905539+0000","last_unstale":"2023-10-16T11:00:46.907258+0000","last_undegraded":"2023-10-16T11:00:46.907258+0000","last_fullsized":"2023-10-16T11:00:46.907258+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:29.686971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.915978+0000","last_change":"2023-10-16T11:00:46.907147+0000","last_active":"2023-10-16T11:00:46.915978+0000","last_peered":"2023-10-16T11:00:46.915978+0000","last_clean":"2023-10-16T11:00:46.915978+0000","last_became_active":"2023-10-16T11:00:46.906870+0000","last_became_peered":"2023-10-16T11:00:46.906870+0000","last_unstale":"2023-10-16T11:00:46.915978+0000","last_undegraded":"2023-10-16T11:00:46.915978+0000","last_fullsized":"2023-10-16T11:00:46.915978+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:41.397200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909595+0000","last_change":"2023-10-16T11:00:46.908701+0000","last_active":"2023-10-16T11:00:46.909595+0000","last_peered":"2023-10-16T11:00:46.909595+0000","last_clean":"2023-10-16T11:00:46.909595+0000","last_became_active":"2023-10-16T11:00:46.908105+0000","last_became_peered":"2023-10-16T11:00:46.908105+0000","last_unstale":"2023-10-16T11:00:46.909595+0000","last_undegraded":"2023-10-16T11:00:46.909595+0000","last_fullsized":"2023-10-16T11:00:46.909595+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:13.819254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905203+0000","last_change":"2023-10-16T11:00:46.902285+0000","last_active":"2023-10-16T11:00:46.905203+0000","last_peered":"2023-10-16T11:00:46.905203+0000","last_clean":"2023-10-16T11:00:46.905203+0000","last_became_active":"2023-10-16T11:00:46.899319+0000","last_became_peered":"2023-10-16T11:00:46.899319+0000","last_unstale":"2023-10-16T11:00:46.905203+0000","last_undegraded":"2023-10-16T11:00:46.905203+0000","last_fullsized":"2023-10-16T11:00:46.905203+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:35.500097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906833+0000","last_change":"2023-10-16T11:00:46.905796+0000","last_active":"2023-10-16T11:00:46.906833+0000","last_peered":"2023-10-16T11:00:46.906833+0000","last_clean":"2023-10-16T11:00:46.906833+0000","last_became_active":"2023-10-16T11:00:46.905511+0000","last_became_peered":"2023-10-16T11:00:46.905511+0000","last_unstale":"2023-10-16T11:00:46.906833+0000","last_undegraded":"2023-10-16T11:00:46.906833+0000","last_fullsized":"2023-10-16T11:00:46.906833+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:26:47.131768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889435+0000","last_change":"2023-10-16T11:00:46.888001+0000","last_active":"2023-10-16T11:00:46.889435+0000","last_peered":"2023-10-16T11:00:46.889435+0000","last_clean":"2023-10-16T11:00:46.889435+0000","last_became_active":"2023-10-16T11:00:46.887721+0000","last_became_peered":"2023-10-16T11:00:46.887721+0000","last_unstale":"2023-10-16T11:00:46.889435+0000","last_undegraded":"2023-10-16T11:00:46.889435+0000","last_fullsized":"2023-10-16T11:00:46.889435+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:58.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.908648+0000","last_change":"2023-10-16T11:00:46.906734+0000","last_active":"2023-10-16T11:00:46.908648+0000","last_peered":"2023-10-16T11:00:46.908648+0000","last_clean":"2023-10-16T11:00:46.908648+0000","last_became_active":"2023-10-16T11:00:46.906368+0000","last_became_peered":"2023-10-16T11:00:46.906368+0000","last_unstale":"2023-10-16T11:00:46.908648+0000","last_undegraded":"2023-10-16T11:00:46.908648+0000","last_fullsized":"2023-10-16T11:00:46.908648+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:23:31.142626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"167'4","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.913403+0000","last_change":"2023-10-16T11:00:45.890693+0000","last_active":"2023-10-16T11:00:46.913403+0000","last_peered":"2023-10-16T11:00:46.913403+0000","last_clean":"2023-10-16T11:00:46.913403+0000","last_became_active":"2023-10-16T11:00:45.890406+0000","last_became_peered":"2023-10-16T11:00:45.890406+0000","last_unstale":"2023-10-16T11:00:46.913403+0000","last_undegraded":"2023-10-16T11:00:46.913403+0000","last_fullsized":"2023-10-16T11:00:46.913403+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:02.407707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.895577+0000","last_change":"2023-10-16T11:00:46.892500+0000","last_active":"2023-10-16T11:00:46.895577+0000","last_peered":"2023-10-16T11:00:46.895577+0000","last_clean":"2023-10-16T11:00:46.895577+0000","last_became_active":"2023-10-16T11:00:46.892261+0000","last_became_peered":"2023-10-16T11:00:46.892261+0000","last_unstale":"2023-10-16T11:00:46.895577+0000","last_undegraded":"2023-10-16T11:00:46.895577+0000","last_fullsized":"2023-10-16T11:00:46.895577+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:41.036049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911587+0000","last_change":"2023-10-16T11:00:45.885964+0000","last_active":"2023-10-16T11:00:46.911587+0000","last_peered":"2023-10-16T11:00:46.911587+0000","last_clean":"2023-10-16T11:00:46.911587+0000","last_became_active":"2023-10-16T11:00:45.885695+0000","last_became_peered":"2023-10-16T11:00:45.885695+0000","last_unstale":"2023-10-16T11:00:46.911587+0000","last_undegraded":"2023-10-16T11:00:46.911587+0000","last_fullsized":"2023-10-16T11:00:46.911587+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:26.907080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904143+0000","last_change":"2023-10-16T11:00:46.902256+0000","last_active":"2023-10-16T11:00:46.904143+0000","last_peered":"2023-10-16T11:00:46.904143+0000","last_clean":"2023-10-16T11:00:46.904143+0000","last_became_active":"2023-10-16T11:00:46.901953+0000","last_became_peered":"2023-10-16T11:00:46.901953+0000","last_unstale":"2023-10-16T11:00:46.904143+0000","last_undegraded":"2023-10-16T11:00:46.904143+0000","last_fullsized":"2023-10-16T11:00:46.904143+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:32:06.946008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.916460+0000","last_change":"2023-10-16T11:00:45.887292+0000","last_active":"2023-10-16T11:00:46.916460+0000","last_peered":"2023-10-16T11:00:46.916460+0000","last_clean":"2023-10-16T11:00:46.916460+0000","last_became_active":"2023-10-16T11:00:45.887063+0000","last_became_peered":"2023-10-16T11:00:45.887063+0000","last_unstale":"2023-10-16T11:00:46.916460+0000","last_undegraded":"2023-10-16T11:00:46.916460+0000","last_fullsized":"2023-10-16T11:00:46.916460+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:05.942415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906304+0000","last_change":"2023-10-16T11:00:46.901435+0000","last_active":"2023-10-16T11:00:46.906304+0000","last_peered":"2023-10-16T11:00:46.906304+0000","last_clean":"2023-10-16T11:00:46.906304+0000","last_became_active":"2023-10-16T11:00:46.901174+0000","last_became_peered":"2023-10-16T11:00:46.901174+0000","last_unstale":"2023-10-16T11:00:46.906304+0000","last_undegraded":"2023-10-16T11:00:46.906304+0000","last_fullsized":"2023-10-16T11:00:46.906304+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:26:40.738233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.910306+0000","last_change":"2023-10-16T11:00:45.885277+0000","last_active":"2023-10-16T11:00:46.910306+0000","last_peered":"2023-10-16T11:00:46.910306+0000","last_clean":"2023-10-16T11:00:46.910306+0000","last_became_active":"2023-10-16T11:00:45.884990+0000","last_became_peered":"2023-10-16T11:00:45.884990+0000","last_unstale":"2023-10-16T11:00:46.910306+0000","last_undegraded":"2023-10-16T11:00:46.910306+0000","last_fullsized":"2023-10-16T11:00:46.910306+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:49.599246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906930+0000","last_change":"2023-10-16T11:00:46.906304+0000","last_active":"2023-10-16T11:00:46.906930+0000","last_peered":"2023-10-16T11:00:46.906930+0000","last_clean":"2023-10-16T11:00:46.906930+0000","last_became_active":"2023-10-16T11:00:46.905969+0000","last_became_peered":"2023-10-16T11:00:46.905969+0000","last_unstale":"2023-10-16T11:00:46.906930+0000","last_undegraded":"2023-10-16T11:00:46.906930+0000","last_fullsized":"2023-10-16T11:00:46.906930+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:54:50.605587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.910635+0000","last_change":"2023-10-16T11:00:45.884820+0000","last_active":"2023-10-16T11:00:46.910635+0000","last_peered":"2023-10-16T11:00:46.910635+0000","last_clean":"2023-10-16T11:00:46.910635+0000","last_became_active":"2023-10-16T11:00:45.884526+0000","last_became_peered":"2023-10-16T11:00:45.884526+0000","last_unstale":"2023-10-16T11:00:46.910635+0000","last_undegraded":"2023-10-16T11:00:46.910635+0000","last_fullsized":"2023-10-16T11:00:46.910635+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:20.085523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909548+0000","last_change":"2023-10-16T11:00:46.906607+0000","last_active":"2023-10-16T11:00:46.909548+0000","last_peered":"2023-10-16T11:00:46.909548+0000","last_clean":"2023-10-16T11:00:46.909548+0000","last_became_active":"2023-10-16T11:00:46.906310+0000","last_became_peered":"2023-10-16T11:00:46.906310+0000","last_unstale":"2023-10-16T11:00:46.909548+0000","last_undegraded":"2023-10-16T11:00:46.909548+0000","last_fullsized":"2023-10-16T11:00:46.909548+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:18.176542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911914+0000","last_change":"2023-10-16T11:00:45.886046+0000","last_active":"2023-10-16T11:00:46.911914+0000","last_peered":"2023-10-16T11:00:46.911914+0000","last_clean":"2023-10-16T11:00:46.911914+0000","last_became_active":"2023-10-16T11:00:45.885765+0000","last_became_peered":"2023-10-16T11:00:45.885765+0000","last_unstale":"2023-10-16T11:00:46.911914+0000","last_undegraded":"2023-10-16T11:00:46.911914+0000","last_fullsized":"2023-10-16T11:00:46.911914+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:38.533483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889791+0000","last_change":"2023-10-16T11:00:46.889006+0000","last_active":"2023-10-16T11:00:46.889791+0000","last_peered":"2023-10-16T11:00:46.889791+0000","last_clean":"2023-10-16T11:00:46.889791+0000","last_became_active":"2023-10-16T11:00:46.888733+0000","last_became_peered":"2023-10-16T11:00:46.888733+0000","last_unstale":"2023-10-16T11:00:46.889791+0000","last_undegraded":"2023-10-16T11:00:46.889791+0000","last_fullsized":"2023-10-16T11:00:46.889791+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:22.471105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.885686+0000","last_change":"2023-10-16T11:00:45.888656+0000","last_active":"2023-10-16T11:00:46.885686+0000","last_peered":"2023-10-16T11:00:46.885686+0000","last_clean":"2023-10-16T11:00:46.885686+0000","last_became_active":"2023-10-16T11:00:45.888292+0000","last_became_peered":"2023-10-16T11:00:45.888292+0000","last_unstale":"2023-10-16T11:00:46.885686+0000","last_undegraded":"2023-10-16T11:00:46.885686+0000","last_fullsized":"2023-10-16T11:00:46.885686+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:38.533483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.902793+0000","last_change":"2023-10-16T11:00:46.901209+0000","last_active":"2023-10-16T11:00:46.902793+0000","last_peered":"2023-10-16T11:00:46.902793+0000","last_clean":"2023-10-16T11:00:46.902793+0000","last_became_active":"2023-10-16T11:00:46.900210+0000","last_became_peered":"2023-10-16T11:00:46.900210+0000","last_unstale":"2023-10-16T11:00:46.902793+0000","last_undegraded":"2023-10-16T11:00:46.902793+0000","last_fullsized":"2023-10-16T11:00:46.902793+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:33.966810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.892944+0000","last_change":"2023-10-16T11:00:45.885738+0000","last_active":"2023-10-16T11:00:46.892944+0000","last_peered":"2023-10-16T11:00:46.892944+0000","last_clean":"2023-10-16T11:00:46.892944+0000","last_became_active":"2023-10-16T11:00:45.885297+0000","last_became_peered":"2023-10-16T11:00:45.885297+0000","last_unstale":"2023-10-16T11:00:46.892944+0000","last_undegraded":"2023-10-16T11:00:46.892944+0000","last_fullsized":"2023-10-16T11:00:46.892944+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:08.936192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891978+0000","last_change":"2023-10-16T11:00:46.887561+0000","last_active":"2023-10-16T11:00:46.891978+0000","last_peered":"2023-10-16T11:00:46.891978+0000","last_clean":"2023-10-16T11:00:46.891978+0000","last_became_active":"2023-10-16T11:00:46.887158+0000","last_became_peered":"2023-10-16T11:00:46.887158+0000","last_unstale":"2023-10-16T11:00:46.891978+0000","last_undegraded":"2023-10-16T11:00:46.891978+0000","last_fullsized":"2023-10-16T11:00:46.891978+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:02.044252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.886076+0000","last_change":"2023-10-16T11:00:45.884257+0000","last_active":"2023-10-16T11:00:46.886076+0000","last_peered":"2023-10-16T11:00:46.886076+0000","last_clean":"2023-10-16T11:00:46.886076+0000","last_became_active":"2023-10-16T11:00:45.883979+0000","last_became_peered":"2023-10-16T11:00:45.883979+0000","last_unstale":"2023-10-16T11:00:46.886076+0000","last_undegraded":"2023-10-16T11:00:46.886076+0000","last_fullsized":"2023-10-16T11:00:46.886076+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:23.774900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905776+0000","last_change":"2023-10-16T11:00:46.898435+0000","last_active":"2023-10-16T11:00:46.905776+0000","last_peered":"2023-10-16T11:00:46.905776+0000","last_clean":"2023-10-16T11:00:46.905776+0000","last_became_active":"2023-10-16T11:00:46.898104+0000","last_became_peered":"2023-10-16T11:00:46.898104+0000","last_unstale":"2023-10-16T11:00:46.905776+0000","last_undegraded":"2023-10-16T11:00:46.905776+0000","last_fullsized":"2023-10-16T11:00:46.905776+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:10:57.310617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891179+0000","last_change":"2023-10-16T11:00:45.888648+0000","last_active":"2023-10-16T11:00:46.891179+0000","last_peered":"2023-10-16T11:00:46.891179+0000","last_clean":"2023-10-16T11:00:46.891179+0000","last_became_active":"2023-10-16T11:00:45.888357+0000","last_became_peered":"2023-10-16T11:00:45.888357+0000","last_unstale":"2023-10-16T11:00:46.891179+0000","last_undegraded":"2023-10-16T11:00:46.891179+0000","last_fullsized":"2023-10-16T11:00:46.891179+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:49:57.153979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.892417+0000","last_change":"2023-10-16T11:00:46.890350+0000","last_active":"2023-10-16T11:00:46.892417+0000","last_peered":"2023-10-16T11:00:46.892417+0000","last_clean":"2023-10-16T11:00:46.892417+0000","last_became_active":"2023-10-16T11:00:46.889954+0000","last_became_peered":"2023-10-16T11:00:46.889954+0000","last_unstale":"2023-10-16T11:00:46.892417+0000","last_undegraded":"2023-10-16T11:00:46.892417+0000","last_fullsized":"2023-10-16T11:00:46.892417+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:53.183222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"167'6","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.917722+0000","last_change":"2023-10-16T11:00:45.887197+0000","last_active":"2023-10-16T11:00:46.917722+0000","last_peered":"2023-10-16T11:00:46.917722+0000","last_clean":"2023-10-16T11:00:46.917722+0000","last_became_active":"2023-10-16T11:00:45.886934+0000","last_became_peered":"2023-10-16T11:00:45.886934+0000","last_unstale":"2023-10-16T11:00:46.917722+0000","last_undegraded":"2023-10-16T11:00:46.917722+0000","last_fullsized":"2023-10-16T11:00:46.917722+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:57.775529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911356+0000","last_change":"2023-10-16T11:00:46.905195+0000","last_active":"2023-10-16T11:00:46.911356+0000","last_peered":"2023-10-16T11:00:46.911356+0000","last_clean":"2023-10-16T11:00:46.911356+0000","last_became_active":"2023-10-16T11:00:46.904879+0000","last_became_peered":"2023-10-16T11:00:46.904879+0000","last_unstale":"2023-10-16T11:00:46.911356+0000","last_undegraded":"2023-10-16T11:00:46.911356+0000","last_fullsized":"2023-10-16T11:00:46.911356+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:29.686971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.916488+0000","last_change":"2023-10-16T11:00:45.889424+0000","last_active":"2023-10-16T11:00:46.916488+0000","last_peered":"2023-10-16T11:00:46.916488+0000","last_clean":"2023-10-16T11:00:46.916488+0000","last_became_active":"2023-10-16T11:00:45.889120+0000","last_became_peered":"2023-10-16T11:00:45.889120+0000","last_unstale":"2023-10-16T11:00:46.916488+0000","last_undegraded":"2023-10-16T11:00:46.916488+0000","last_fullsized":"2023-10-16T11:00:46.916488+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:48.971019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891906+0000","last_change":"2023-10-16T11:00:46.890015+0000","last_active":"2023-10-16T11:00:46.891906+0000","last_peered":"2023-10-16T11:00:46.891906+0000","last_clean":"2023-10-16T11:00:46.891906+0000","last_became_active":"2023-10-16T11:00:46.889757+0000","last_became_peered":"2023-10-16T11:00:46.889757+0000","last_unstale":"2023-10-16T11:00:46.891906+0000","last_undegraded":"2023-10-16T11:00:46.891906+0000","last_fullsized":"2023-10-16T11:00:46.891906+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:51.664803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896659+0000","last_change":"2023-10-16T11:00:45.888150+0000","last_active":"2023-10-16T11:00:46.896659+0000","last_peered":"2023-10-16T11:00:46.896659+0000","last_clean":"2023-10-16T11:00:46.896659+0000","last_became_active":"2023-10-16T11:00:45.887892+0000","last_became_peered":"2023-10-16T11:00:45.887892+0000","last_unstale":"2023-10-16T11:00:46.896659+0000","last_undegraded":"2023-10-16T11:00:46.896659+0000","last_fullsized":"2023-10-16T11:00:46.896659+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:20.085523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.893310+0000","last_change":"2023-10-16T11:00:46.891811+0000","last_active":"2023-10-16T11:00:46.893310+0000","last_peered":"2023-10-16T11:00:46.893310+0000","last_clean":"2023-10-16T11:00:46.893310+0000","last_became_active":"2023-10-16T11:00:46.891322+0000","last_became_peered":"2023-10-16T11:00:46.891322+0000","last_unstale":"2023-10-16T11:00:46.893310+0000","last_undegraded":"2023-10-16T11:00:46.893310+0000","last_fullsized":"2023-10-16T11:00:46.893310+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:53.183222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896905+0000","last_change":"2023-10-16T11:00:45.885246+0000","last_active":"2023-10-16T11:00:46.896905+0000","last_peered":"2023-10-16T11:00:46.896905+0000","last_clean":"2023-10-16T11:00:46.896905+0000","last_became_active":"2023-10-16T11:00:45.884985+0000","last_became_peered":"2023-10-16T11:00:45.884985+0000","last_unstale":"2023-10-16T11:00:46.896905+0000","last_undegraded":"2023-10-16T11:00:46.896905+0000","last_fullsized":"2023-10-16T11:00:46.896905+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:52.696700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911178+0000","last_change":"2023-10-16T11:00:46.905063+0000","last_active":"2023-10-16T11:00:46.911178+0000","last_peered":"2023-10-16T11:00:46.911178+0000","last_clean":"2023-10-16T11:00:46.911178+0000","last_became_active":"2023-10-16T11:00:46.904452+0000","last_became_peered":"2023-10-16T11:00:46.904452+0000","last_unstale":"2023-10-16T11:00:46.911178+0000","last_undegraded":"2023-10-16T11:00:46.911178+0000","last_fullsized":"2023-10-16T11:00:46.911178+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:53.183222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"167'8","reported_seq":19,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.912395+0000","last_change":"2023-10-16T11:00:45.889231+0000","last_active":"2023-10-16T11:00:46.912395+0000","last_peered":"2023-10-16T11:00:46.912395+0000","last_clean":"2023-10-16T11:00:46.912395+0000","last_became_active":"2023-10-16T11:00:45.888928+0000","last_became_peered":"2023-10-16T11:00:45.888928+0000","last_unstale":"2023-10-16T11:00:46.912395+0000","last_undegraded":"2023-10-16T11:00:46.912395+0000","last_fullsized":"2023-10-16T11:00:46.912395+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:01.037253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.908631+0000","last_change":"2023-10-16T11:00:46.905961+0000","last_active":"2023-10-16T11:00:46.908631+0000","last_peered":"2023-10-16T11:00:46.908631+0000","last_clean":"2023-10-16T11:00:46.908631+0000","last_became_active":"2023-10-16T11:00:46.905654+0000","last_became_peered":"2023-10-16T11:00:46.905654+0000","last_unstale":"2023-10-16T11:00:46.908631+0000","last_undegraded":"2023-10-16T11:00:46.908631+0000","last_fullsized":"2023-10-16T11:00:46.908631+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:55.391317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.912036+0000","last_change":"2023-10-16T11:00:45.889667+0000","last_active":"2023-10-16T11:00:46.912036+0000","last_peered":"2023-10-16T11:00:46.912036+0000","last_clean":"2023-10-16T11:00:46.912036+0000","last_became_active":"2023-10-16T11:00:45.889357+0000","last_became_peered":"2023-10-16T11:00:45.889357+0000","last_unstale":"2023-10-16T11:00:46.912036+0000","last_undegraded":"2023-10-16T11:00:46.912036+0000","last_fullsized":"2023-10-16T11:00:46.912036+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:48.971019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.897403+0000","last_change":"2023-10-16T11:00:46.893546+0000","last_active":"2023-10-16T11:00:46.897403+0000","last_peered":"2023-10-16T11:00:46.897403+0000","last_clean":"2023-10-16T11:00:46.897403+0000","last_became_active":"2023-10-16T11:00:46.892620+0000","last_became_peered":"2023-10-16T11:00:46.892620+0000","last_unstale":"2023-10-16T11:00:46.897403+0000","last_undegraded":"2023-10-16T11:00:46.897403+0000","last_fullsized":"2023-10-16T11:00:46.897403+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:47:49.978018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.907292+0000","last_change":"2023-10-16T11:00:45.892515+0000","last_active":"2023-10-16T11:00:46.907292+0000","last_peered":"2023-10-16T11:00:46.907292+0000","last_clean":"2023-10-16T11:00:46.907292+0000","last_became_active":"2023-10-16T11:00:45.892170+0000","last_became_peered":"2023-10-16T11:00:45.892170+0000","last_unstale":"2023-10-16T11:00:46.907292+0000","last_undegraded":"2023-10-16T11:00:46.907292+0000","last_fullsized":"2023-10-16T11:00:46.907292+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:31:55.521169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911418+0000","last_change":"2023-10-16T11:00:46.907142+0000","last_active":"2023-10-16T11:00:46.911418+0000","last_peered":"2023-10-16T11:00:46.911418+0000","last_clean":"2023-10-16T11:00:46.911418+0000","last_became_active":"2023-10-16T11:00:46.906790+0000","last_became_peered":"2023-10-16T11:00:46.906790+0000","last_unstale":"2023-10-16T11:00:46.911418+0000","last_undegraded":"2023-10-16T11:00:46.911418+0000","last_fullsized":"2023-10-16T11:00:46.911418+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:08.155992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"167'1","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.910712+0000","last_change":"2023-10-16T11:00:45.888904+0000","last_active":"2023-10-16T11:00:46.910712+0000","last_peered":"2023-10-16T11:00:46.910712+0000","last_clean":"2023-10-16T11:00:46.910712+0000","last_became_active":"2023-10-16T11:00:45.888581+0000","last_became_peered":"2023-10-16T11:00:45.888581+0000","last_unstale":"2023-10-16T11:00:46.910712+0000","last_undegraded":"2023-10-16T11:00:46.910712+0000","last_fullsized":"2023-10-16T11:00:46.910712+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:26:32.959811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909574+0000","last_change":"2023-10-16T11:00:46.908448+0000","last_active":"2023-10-16T11:00:46.909574+0000","last_peered":"2023-10-16T11:00:46.909574+0000","last_clean":"2023-10-16T11:00:46.909574+0000","last_became_active":"2023-10-16T11:00:46.908159+0000","last_became_peered":"2023-10-16T11:00:46.908159+0000","last_unstale":"2023-10-16T11:00:46.909574+0000","last_undegraded":"2023-10-16T11:00:46.909574+0000","last_fullsized":"2023-10-16T11:00:46.909574+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:30:19.530601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.885677+0000","last_change":"2023-10-16T11:00:45.886341+0000","last_active":"2023-10-16T11:00:46.885677+0000","last_peered":"2023-10-16T11:00:46.885677+0000","last_clean":"2023-10-16T11:00:46.885677+0000","last_became_active":"2023-10-16T11:00:45.886073+0000","last_became_peered":"2023-10-16T11:00:45.886073+0000","last_unstale":"2023-10-16T11:00:46.885677+0000","last_undegraded":"2023-10-16T11:00:46.885677+0000","last_fullsized":"2023-10-16T11:00:46.885677+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:15.359818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904427+0000","last_change":"2023-10-16T11:00:46.899575+0000","last_active":"2023-10-16T11:00:46.904427+0000","last_peered":"2023-10-16T11:00:46.904427+0000","last_clean":"2023-10-16T11:00:46.904427+0000","last_became_active":"2023-10-16T11:00:46.898969+0000","last_became_peered":"2023-10-16T11:00:46.898969+0000","last_unstale":"2023-10-16T11:00:46.904427+0000","last_undegraded":"2023-10-16T11:00:46.904427+0000","last_fullsized":"2023-10-16T11:00:46.904427+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:09.091226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.916045+0000","last_change":"2023-10-16T11:00:45.894799+0000","last_active":"2023-10-16T11:00:46.916045+0000","last_peered":"2023-10-16T11:00:46.916045+0000","last_clean":"2023-10-16T11:00:46.916045+0000","last_became_active":"2023-10-16T11:00:45.894571+0000","last_became_peered":"2023-10-16T11:00:45.894571+0000","last_unstale":"2023-10-16T11:00:46.916045+0000","last_undegraded":"2023-10-16T11:00:46.916045+0000","last_fullsized":"2023-10-16T11:00:46.916045+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:52.176222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905773+0000","last_change":"2023-10-16T11:00:46.898436+0000","last_active":"2023-10-16T11:00:46.905773+0000","last_peered":"2023-10-16T11:00:46.905773+0000","last_clean":"2023-10-16T11:00:46.905773+0000","last_became_active":"2023-10-16T11:00:46.898171+0000","last_became_peered":"2023-10-16T11:00:46.898171+0000","last_unstale":"2023-10-16T11:00:46.905773+0000","last_undegraded":"2023-10-16T11:00:46.905773+0000","last_fullsized":"2023-10-16T11:00:46.905773+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:21:57.437473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889145+0000","last_change":"2023-10-16T11:00:45.888767+0000","last_active":"2023-10-16T11:00:46.889145+0000","last_peered":"2023-10-16T11:00:46.889145+0000","last_clean":"2023-10-16T11:00:46.889145+0000","last_became_active":"2023-10-16T11:00:45.888470+0000","last_became_peered":"2023-10-16T11:00:45.888470+0000","last_unstale":"2023-10-16T11:00:46.889145+0000","last_undegraded":"2023-10-16T11:00:46.889145+0000","last_fullsized":"2023-10-16T11:00:46.889145+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:15.359818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.905163+0000","last_change":"2023-10-16T11:00:46.902037+0000","last_active":"2023-10-16T11:00:46.905163+0000","last_peered":"2023-10-16T11:00:46.905163+0000","last_clean":"2023-10-16T11:00:46.905163+0000","last_became_active":"2023-10-16T11:00:46.899414+0000","last_became_peered":"2023-10-16T11:00:46.899414+0000","last_unstale":"2023-10-16T11:00:46.905163+0000","last_undegraded":"2023-10-16T11:00:46.905163+0000","last_fullsized":"2023-10-16T11:00:46.905163+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:30:47.612853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889557+0000","last_change":"2023-10-16T11:00:45.887113+0000","last_active":"2023-10-16T11:00:46.889557+0000","last_peered":"2023-10-16T11:00:46.889557+0000","last_clean":"2023-10-16T11:00:46.889557+0000","last_became_active":"2023-10-16T11:00:45.886814+0000","last_became_peered":"2023-10-16T11:00:45.886814+0000","last_unstale":"2023-10-16T11:00:46.889557+0000","last_undegraded":"2023-10-16T11:00:46.889557+0000","last_fullsized":"2023-10-16T11:00:46.889557+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:44.525092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.915441+0000","last_change":"2023-10-16T11:00:46.898804+0000","last_active":"2023-10-16T11:00:46.915441+0000","last_peered":"2023-10-16T11:00:46.915441+0000","last_clean":"2023-10-16T11:00:46.915441+0000","last_became_active":"2023-10-16T11:00:46.898130+0000","last_became_peered":"2023-10-16T11:00:46.898130+0000","last_unstale":"2023-10-16T11:00:46.915441+0000","last_undegraded":"2023-10-16T11:00:46.915441+0000","last_fullsized":"2023-10-16T11:00:46.915441+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:39.003786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.894453+0000","last_change":"2023-10-16T11:00:45.888182+0000","last_active":"2023-10-16T11:00:46.894453+0000","last_peered":"2023-10-16T11:00:46.894453+0000","last_clean":"2023-10-16T11:00:46.894453+0000","last_became_active":"2023-10-16T11:00:45.887627+0000","last_became_peered":"2023-10-16T11:00:45.887627+0000","last_unstale":"2023-10-16T11:00:46.894453+0000","last_undegraded":"2023-10-16T11:00:46.894453+0000","last_fullsized":"2023-10-16T11:00:46.894453+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:40.390200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.902841+0000","last_change":"2023-10-16T11:00:46.900783+0000","last_active":"2023-10-16T11:00:46.902841+0000","last_peered":"2023-10-16T11:00:46.902841+0000","last_clean":"2023-10-16T11:00:46.902841+0000","last_became_active":"2023-10-16T11:00:46.899994+0000","last_became_peered":"2023-10-16T11:00:46.899994+0000","last_unstale":"2023-10-16T11:00:46.902841+0000","last_undegraded":"2023-10-16T11:00:46.902841+0000","last_fullsized":"2023-10-16T11:00:46.902841+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:09.653311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"167'2","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.912320+0000","last_change":"2023-10-16T11:00:45.892736+0000","last_active":"2023-10-16T11:00:46.912320+0000","last_peered":"2023-10-16T11:00:46.912320+0000","last_clean":"2023-10-16T11:00:46.912320+0000","last_became_active":"2023-10-16T11:00:45.892430+0000","last_became_peered":"2023-10-16T11:00:45.892430+0000","last_unstale":"2023-10-16T11:00:46.912320+0000","last_undegraded":"2023-10-16T11:00:46.912320+0000","last_fullsized":"2023-10-16T11:00:46.912320+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:59.311120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.911241+0000","last_change":"2023-10-16T11:00:46.906117+0000","last_active":"2023-10-16T11:00:46.911241+0000","last_peered":"2023-10-16T11:00:46.911241+0000","last_clean":"2023-10-16T11:00:46.911241+0000","last_became_active":"2023-10-16T11:00:46.904628+0000","last_became_peered":"2023-10-16T11:00:46.904628+0000","last_unstale":"2023-10-16T11:00:46.911241+0000","last_undegraded":"2023-10-16T11:00:46.911241+0000","last_fullsized":"2023-10-16T11:00:46.911241+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:58.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896843+0000","last_change":"2023-10-16T11:00:45.884542+0000","last_active":"2023-10-16T11:00:46.896843+0000","last_peered":"2023-10-16T11:00:46.896843+0000","last_clean":"2023-10-16T11:00:46.896843+0000","last_became_active":"2023-10-16T11:00:45.884284+0000","last_became_peered":"2023-10-16T11:00:45.884284+0000","last_unstale":"2023-10-16T11:00:46.896843+0000","last_undegraded":"2023-10-16T11:00:46.896843+0000","last_fullsized":"2023-10-16T11:00:46.896843+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:02:35.551193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.898564+0000","last_change":"2023-10-16T11:00:46.897763+0000","last_active":"2023-10-16T11:00:46.898564+0000","last_peered":"2023-10-16T11:00:46.898564+0000","last_clean":"2023-10-16T11:00:46.898564+0000","last_became_active":"2023-10-16T11:00:46.897407+0000","last_became_peered":"2023-10-16T11:00:46.897407+0000","last_unstale":"2023-10-16T11:00:46.898564+0000","last_undegraded":"2023-10-16T11:00:46.898564+0000","last_fullsized":"2023-10-16T11:00:46.898564+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:48.456233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.897163+0000","last_change":"2023-10-16T11:00:45.889633+0000","last_active":"2023-10-16T11:00:46.897163+0000","last_peered":"2023-10-16T11:00:46.897163+0000","last_clean":"2023-10-16T11:00:46.897163+0000","last_became_active":"2023-10-16T11:00:45.889343+0000","last_became_peered":"2023-10-16T11:00:45.889343+0000","last_unstale":"2023-10-16T11:00:46.897163+0000","last_undegraded":"2023-10-16T11:00:46.897163+0000","last_fullsized":"2023-10-16T11:00:46.897163+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:20.127327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904208+0000","last_change":"2023-10-16T11:00:46.902735+0000","last_active":"2023-10-16T11:00:46.904208+0000","last_peered":"2023-10-16T11:00:46.904208+0000","last_clean":"2023-10-16T11:00:46.904208+0000","last_became_active":"2023-10-16T11:00:46.902290+0000","last_became_peered":"2023-10-16T11:00:46.902290+0000","last_unstale":"2023-10-16T11:00:46.904208+0000","last_undegraded":"2023-10-16T11:00:46.904208+0000","last_fullsized":"2023-10-16T11:00:46.904208+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:23.999858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.896622+0000","last_change":"2023-10-16T11:00:45.884269+0000","last_active":"2023-10-16T11:00:46.896622+0000","last_peered":"2023-10-16T11:00:46.896622+0000","last_clean":"2023-10-16T11:00:46.896622+0000","last_became_active":"2023-10-16T11:00:45.884003+0000","last_became_peered":"2023-10-16T11:00:45.884003+0000","last_unstale":"2023-10-16T11:00:46.896622+0000","last_undegraded":"2023-10-16T11:00:46.896622+0000","last_fullsized":"2023-10-16T11:00:46.896622+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:03.949532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.899426+0000","last_change":"2023-10-16T11:00:46.897844+0000","last_active":"2023-10-16T11:00:46.899426+0000","last_peered":"2023-10-16T11:00:46.899426+0000","last_clean":"2023-10-16T11:00:46.899426+0000","last_became_active":"2023-10-16T11:00:46.897213+0000","last_became_peered":"2023-10-16T11:00:46.897213+0000","last_unstale":"2023-10-16T11:00:46.899426+0000","last_undegraded":"2023-10-16T11:00:46.899426+0000","last_fullsized":"2023-10-16T11:00:46.899426+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:17:41.397200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889329+0000","last_change":"2023-10-16T11:00:45.884337+0000","last_active":"2023-10-16T11:00:46.889329+0000","last_peered":"2023-10-16T11:00:46.889329+0000","last_clean":"2023-10-16T11:00:46.889329+0000","last_became_active":"2023-10-16T11:00:45.882283+0000","last_became_peered":"2023-10-16T11:00:45.882283+0000","last_unstale":"2023-10-16T11:00:46.889329+0000","last_undegraded":"2023-10-16T11:00:46.889329+0000","last_fullsized":"2023-10-16T11:00:46.889329+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:08.646311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906586+0000","last_change":"2023-10-16T11:00:46.905823+0000","last_active":"2023-10-16T11:00:46.906586+0000","last_peered":"2023-10-16T11:00:46.906586+0000","last_clean":"2023-10-16T11:00:46.906586+0000","last_became_active":"2023-10-16T11:00:46.905505+0000","last_became_peered":"2023-10-16T11:00:46.905505+0000","last_unstale":"2023-10-16T11:00:46.906586+0000","last_undegraded":"2023-10-16T11:00:46.906586+0000","last_fullsized":"2023-10-16T11:00:46.906586+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:45:58.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.889097+0000","last_change":"2023-10-16T11:00:45.892408+0000","last_active":"2023-10-16T11:00:46.889097+0000","last_peered":"2023-10-16T11:00:46.889097+0000","last_clean":"2023-10-16T11:00:46.889097+0000","last_became_active":"2023-10-16T11:00:45.892125+0000","last_became_peered":"2023-10-16T11:00:45.892125+0000","last_unstale":"2023-10-16T11:00:46.889097+0000","last_undegraded":"2023-10-16T11:00:46.889097+0000","last_fullsized":"2023-10-16T11:00:46.889097+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:10.988275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.906278+0000","last_change":"2023-10-16T11:00:46.898987+0000","last_active":"2023-10-16T11:00:46.906278+0000","last_peered":"2023-10-16T11:00:46.906278+0000","last_clean":"2023-10-16T11:00:46.906278+0000","last_became_active":"2023-10-16T11:00:46.898472+0000","last_became_peered":"2023-10-16T11:00:46.898472+0000","last_unstale":"2023-10-16T11:00:46.906278+0000","last_undegraded":"2023-10-16T11:00:46.906278+0000","last_fullsized":"2023-10-16T11:00:46.906278+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:42:41.036049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.895598+0000","last_change":"2023-10-16T11:00:45.892198+0000","last_active":"2023-10-16T11:00:46.895598+0000","last_peered":"2023-10-16T11:00:46.895598+0000","last_clean":"2023-10-16T11:00:46.895598+0000","last_became_active":"2023-10-16T11:00:45.891907+0000","last_became_peered":"2023-10-16T11:00:45.891907+0000","last_unstale":"2023-10-16T11:00:46.895598+0000","last_undegraded":"2023-10-16T11:00:46.895598+0000","last_fullsized":"2023-10-16T11:00:46.895598+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:08.646311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.891945+0000","last_change":"2023-10-16T11:00:46.887240+0000","last_active":"2023-10-16T11:00:46.891945+0000","last_peered":"2023-10-16T11:00:46.891945+0000","last_clean":"2023-10-16T11:00:46.891945+0000","last_became_active":"2023-10-16T11:00:46.886908+0000","last_became_peered":"2023-10-16T11:00:46.886908+0000","last_unstale":"2023-10-16T11:00:46.891945+0000","last_undegraded":"2023-10-16T11:00:46.891945+0000","last_fullsized":"2023-10-16T11:00:46.891945+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:29.686971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.885184+0000","last_change":"2023-10-16T11:00:45.883424+0000","last_active":"2023-10-16T11:00:46.885184+0000","last_peered":"2023-10-16T11:00:46.885184+0000","last_clean":"2023-10-16T11:00:46.885184+0000","last_became_active":"2023-10-16T11:00:45.883148+0000","last_became_peered":"2023-10-16T11:00:45.883148+0000","last_unstale":"2023-10-16T11:00:46.885184+0000","last_undegraded":"2023-10-16T11:00:46.885184+0000","last_fullsized":"2023-10-16T11:00:46.885184+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:07.026016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.904851+0000","last_change":"2023-10-16T11:00:46.898366+0000","last_active":"2023-10-16T11:00:46.904851+0000","last_peered":"2023-10-16T11:00:46.904851+0000","last_clean":"2023-10-16T11:00:46.904851+0000","last_became_active":"2023-10-16T11:00:46.898059+0000","last_became_peered":"2023-10-16T11:00:46.898059+0000","last_unstale":"2023-10-16T11:00:46.904851+0000","last_undegraded":"2023-10-16T11:00:46.904851+0000","last_fullsized":"2023-10-16T11:00:46.904851+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:45.866457+0000","last_clean_scrub_stamp":"2023-10-16T11:00:45.866457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:08.155992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"167'3","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-16T11:00:46.909297+0000","last_change":"2023-10-16T11:00:45.892089+0000","last_active":"2023-10-16T11:00:46.909297+0000","last_peered":"2023-10-16T11:00:46.909297+0000","last_clean":"2023-10-16T11:00:46.909297+0000","last_became_active":"2023-10-16T11:00:45.891821+0000","last_became_peered":"2023-10-16T11:00:45.891821+0000","last_unstale":"2023-10-16T11:00:46.909297+0000","last_undegraded":"2023-10-16T11:00:46.909297+0000","last_fullsized":"2023-10-16T11:00:46.909297+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:00:44.859458+0000","last_clean_scrub_stamp":"2023-10-16T11:00:44.859458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:01.037253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"157'271","reported_seq":882,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-16T11:00:45.882158+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:00:45.882158+0000","last_peered":"2023-10-16T11:00:45.882158+0000","last_clean":"2023-10-16T11:00:45.882158+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:00:45.882158+0000","last_undegraded":"2023-10-16T11:00:45.882158+0000","last_fullsized":"2023-10-16T11:00:45.882158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543061,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":2682880,"data_stored":1640864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76700000000000002,"5min":0.78600000000000003,"15min":0.83499999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.754,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.378}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.89200000000000002},"last":0.54900000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.81100000000000005,"15min":0.88100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69799999999999995,"5min":0.81399999999999995,"15min":0.83899999999999997},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.85199999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.86699999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.876},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.308},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.85999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.89300000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.92400000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.621}]}]},{"osd":6,"up_from":14,"seq":60129543060,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.78600000000000003,"5min":0.871,"15min":0.871},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.32600000000000001},"max":{"1min":0.75700000000000001,"5min":0.83399999999999996,"15min":0.86899999999999999},"last":0.61899999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.41699999999999998,"15min":0.36799999999999999},"max":{"1min":0.70599999999999996,"5min":0.82999999999999996,"15min":0.83699999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.623,"15min":0.629},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74099999999999999,"5min":0.83299999999999996,"15min":0.89200000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80600000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.623},"min":{"1min":0.51400000000000001,"5min":0.44800000000000001,"15min":0.40200000000000002},"max":{"1min":0.76200000000000001,"5min":0.82499999999999996,"15min":0.86099999999999999},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.44900000000000001,"15min":0.44500000000000001},"max":{"1min":0.79600000000000004,"5min":0.86499999999999999,"15min":0.874},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.442,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53000000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.314},"max":{"1min":0.621,"5min":0.72799999999999998,"15min":0.91100000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.90200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.35499999999999998},"max":{"1min":0.84599999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:00:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.34599999999999997},"max":{"1min":0.876,"5min":0.876,"15min":0.89600000000000002},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129543060,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55412,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316428,"statfs":{"total":96636764160,"available":96580022272,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.24199999999999999},"max":{"1min":0.59099999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.42899999999999999,"15min":0.42199999999999999},"min":{"1min":0.32000000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56599999999999995,"5min":0.64400000000000002,"15min":0.68000000000000005},"last":0.44}]},{"osd":2,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.25800000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.35199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.745},"last":0.34599999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.376},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.29899999999999999},"max":{"1min":0.63600000000000001,"5min":0.66300000000000003,"15min":0.73399999999999999},"last":0.39300000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":1.004,"15min":1.044},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.39500000000000002},"max":{"1min":0.89600000000000002,"5min":0.94399999999999995,"15min":1.0409999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.43099999999999999},"max":{"1min":0.86199999999999999,"5min":1.038,"15min":1.038},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.56699999999999995,"5min":0.45100000000000001,"15min":0.43099999999999999},"max":{"1min":1.0149999999999999,"5min":1.038,"15min":1.038},"last":0.66800000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.55000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":1.0169999999999999,"15min":1.105},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":0.94599999999999995,"5min":0.94699999999999995,"15min":1.091},"last":0.65200000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:00:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.1299999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.53500000000000003,"5min":0.47299999999999998,"15min":0.45700000000000002},"max":{"1min":0.96299999999999997,"5min":1.034,"15min":1.034},"last":0.70299999999999996}]}]},{"osd":0,"up_from":14,"seq":60129543061,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56024,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315816,"statfs":{"total":96636764160,"available":96579395584,"internally_reserved":0,"allocated":2691072,"data_stored":1641858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.22900000000000001},"max":{"1min":0.54500000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.218},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.69199999999999995},"last":0.83299999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.64200000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.86799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.72599999999999998},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66000000000000003,"5min":0.71599999999999997,"15min":0.74099999999999999},"last":0.90100000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.35899999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.0900000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.39700000000000002},"max":{"1min":0.79000000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.999,"15min":1.0820000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.67000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.48399999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.98699999999999999},"last":0.81999999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.746}]}]},{"osd":2,"up_from":14,"seq":60129543060,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59572,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312268,"statfs":{"total":96636764160,"available":96575762432,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.39700000000000002,"15min":0.38500000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.375},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.78900000000000003},"last":0.28000000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46100000000000002,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.26200000000000001},"max":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.67900000000000005},"last":0.35299999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.624,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.502,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.751},"last":0.40699999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.97999999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.42599999999999999},"max":{"1min":0.88300000000000001,"5min":0.98699999999999999,"15min":1.022},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86199999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68400000000000005,"15min":0.66900000000000004},"min":{"1min":0.45700000000000002,"5min":0.40999999999999998,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.48299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68799999999999994,"15min":0.67000000000000004},"min":{"1min":0.47299999999999998,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89800000000000002,"5min":1.159,"15min":1.159},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68999999999999995,"15min":0.67900000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.437},"max":{"1min":0.90200000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.49199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":1.121,"15min":1.121},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93200000000000005,"5min":1.095,"15min":1.095},"last":0.504}]}]},{"osd":3,"up_from":14,"seq":60129543061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2084864,"data_stored":1049934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.441},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59399999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.60299999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52300000000000002,"5min":0.621,"15min":0.76400000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.441},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.66600000000000004},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28499999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.64700000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.28499999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.96999999999999997},"last":0.76800000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.92400000000000004,"15min":0.98499999999999999},"last":0.80700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81399999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.0029999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.0349999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.995},"last":0.70499999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95599999999999996,"15min":0.95699999999999996},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94099999999999995,"5min":0.96599999999999997,"15min":0.98399999999999999},"last":0.66000000000000003}]}]},{"osd":4,"up_from":14,"seq":60129543060,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51252,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320588,"statfs":{"total":96636764160,"available":96584282112,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72999999999999998,"5min":0.83799999999999997,"15min":0.90100000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.32300000000000001},"max":{"1min":0.77500000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.622}]},{"osd":1,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.88400000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.68799999999999994,"5min":0.77400000000000002,"15min":0.80800000000000005},"last":0.39300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.34200000000000003},"max":{"1min":0.72499999999999998,"5min":0.86199999999999999,"15min":1.077},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41099999999999998},"max":{"1min":0.72099999999999997,"5min":0.81100000000000005,"15min":0.97899999999999998},"last":0.54600000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.84099999999999997},"last":0.499},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.48299999999999998,"5min":0.432,"15min":0.41099999999999998},"max":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63800000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.61799999999999999,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.86899999999999999,"15min":1.03},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71299999999999997,"5min":0.89100000000000001,"15min":0.96099999999999997},"last":0.66100000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.69099999999999995,"5min":0.88,"15min":0.88},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69299999999999995,"5min":0.83999999999999997,"15min":1.133},"last":0.72999999999999998}]}]},{"osd":5,"up_from":14,"seq":60129543061,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2068480,"data_stored":1042573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.437},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.79500000000000004,"5min":0.82599999999999996,"15min":0.83599999999999997},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.876},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.623,"15min":0.621},"min":{"1min":0.432,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.872,"15min":0.90400000000000003},"last":0.497}]},{"osd":2,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.34100000000000003,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.871,"15min":0.871},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.871},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.64200000000000002,"15min":0.64300000000000002},"min":{"1min":0.42799999999999999,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.86099999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.315},"max":{"1min":0.71099999999999997,"5min":0.753,"15min":0.78700000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.40799999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38300000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.36199999999999999,"5min":0.35199999999999998,"15min":0.34000000000000002},"max":{"1min":0.76800000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.33000000000000002},"max":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.84899999999999998},"last":0.63300000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63,"15min":0.623},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.36099999999999999},"max":{"1min":0.81999999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:00:51.396 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:00:51.396 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:00:51.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.556+0000 7ffb2e598700 1 Processor -- start 2023-10-16T11:00:51.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.556+0000 7ffb2e598700 1 -- start start 2023-10-16T11:00:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.557+0000 7ffb2e598700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 0x7ffb280cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.557+0000 7ffb2e598700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb28071be0 0x7ffb280b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.557+0000 7ffb2e598700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb280ce1f0 con 0x7ffb2806f3b0 2023-10-16T11:00:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.557+0000 7ffb2e598700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb280b2930 con 0x7ffb28071be0 2023-10-16T11:00:51.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.557+0000 7ffb2e598700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280ccda0 con 0x7ffb280b2010 2023-10-16T11:00:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.557+0000 7ffb2cb35700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ffb2806f3b0 0x7ffb2806f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33674/0 (socket says 172.21.15.181:33674) 2023-10-16T11:00:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.557+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1451817813 learned_addr learned my addr 172.21.15.181:0/1451817813 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb27fff700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 0x7ffb280cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb277fe700 1 --2- 172.21.15.181:0/1451817813 >> v2:172.21.15.190:3300/0 conn(0x7ffb28071be0 0x7ffb280b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994041 0 0) 0x7ffb280ce1f0 con 0x7ffb2806f3b0 2023-10-16T11:00:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb140035b0 con 0x7ffb2806f3b0 2023-10-16T11:00:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 775289345 0 0) 0x7ffb140035b0 con 0x7ffb2806f3b0 2023-10-16T11:00:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 msgr2=0x7ffb280cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 0x7ffb280cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 >> v2:172.21.15.190:3300/0 conn(0x7ffb28071be0 msgr2=0x7ffb280b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 --2- 172.21.15.181:0/1451817813 >> v2:172.21.15.190:3300/0 conn(0x7ffb28071be0 0x7ffb280b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb280727e0 con 0x7ffb2806f3b0 2023-10-16T11:00:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ffb200019f0 con 0x7ffb2806f3b0 2023-10-16T11:00:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 >> v1:172.21.15.181:6789/0 conn(0x7ffb2806f3b0 legacy=0x7ffb2806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 0x7ffb14003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb14003f80 0x7ffb14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb14004d90 0x7ffb14005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280b2930 con 0x7ffb14004d90 2023-10-16T11:00:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280ccda0 con 0x7ffb14003f80 2023-10-16T11:00:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1400ba10 con 0x7ffb280b2010 2023-10-16T11:00:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb2cb35700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb14004d90 0x7ffb14005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb277fe700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb14003f80 0x7ffb14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb27fff700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 0x7ffb14003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 msgr2=0x7ffb14003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb2cb35700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb280b2010 0x7ffb14003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb14003f80 msgr2=0x7ffb14004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb2cb35700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb14003f80 0x7ffb14004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.558+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1451817813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140035b0 con 0x7ffb14004d90 2023-10-16T11:00:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2cb35700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb14004d90 0x7ffb14005210 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000a8d0 tx=0x7ffb14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622024684fed6af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:51.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2000a340 con 0x7ffb14004d90 2023-10-16T11:00:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20012600 con 0x7ffb14004d90 2023-10-16T11:00:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb26ffd700 1 -- 172.21.15.181:0/1451817813 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200129b0 con 0x7ffb14004d90 2023-10-16T11:00:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2e598700 1 -- 172.21.15.181:0/1451817813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb14004d90 msgr2=0x7ffb14005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2e598700 1 --2- 172.21.15.181:0/1451817813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb14004d90 0x7ffb14005210 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000a8d0 tx=0x7ffb14005880 comp rx=0 tx=0).stop 2023-10-16T11:00:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1451817813 reap_dead start 2023-10-16T11:00:51.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2e598700 1 -- 172.21.15.181:0/1451817813 shutdown_connections 2023-10-16T11:00:51.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2e598700 1 -- 172.21.15.181:0/1451817813 >> 172.21.15.181:0/1451817813 conn(0x7ffb280cd010 msgr2=0x7ffb280cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:51.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2e598700 1 -- 172.21.15.181:0/1451817813 shutdown_connections 2023-10-16T11:00:51.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.559+0000 7ffb2e598700 1 -- 172.21.15.181:0/1451817813 wait complete. 2023-10-16T11:00:51.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.560+0000 7ffb2e598700 1 Processor -- start 2023-10-16T11:00:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.561+0000 7ffb2e598700 1 -- start start 2023-10-16T11:00:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.561+0000 7ffb2e598700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb281491b0 0x7ffb280afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.561+0000 7ffb2e598700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb28149590 0x7ffb280b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.561+0000 7ffb2e598700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb281f64b0 0x7ffb281f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.561+0000 7ffb2e598700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281f6e70 con 0x7ffb281f64b0 2023-10-16T11:00:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.561+0000 7ffb2e598700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2815aa60 con 0x7ffb28149590 2023-10-16T11:00:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.561+0000 7ffb2e598700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280adda0 con 0x7ffb281491b0 2023-10-16T11:00:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb281f64b0 0x7ffb281f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb281f64b0 0x7ffb281f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50840/0 (socket says 172.21.15.181:50840) 2023-10-16T11:00:51.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1775371046 learned_addr learned my addr 172.21.15.181:0/1775371046 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb27fff700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb281491b0 0x7ffb280afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb277fe700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb28149590 0x7ffb280b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb281491b0 msgr2=0x7ffb280afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb281491b0 0x7ffb280afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb28149590 msgr2=0x7ffb280b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb28149590 0x7ffb280b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 -- 172.21.15.181:0/1775371046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28149d40 con 0x7ffb281f64b0 2023-10-16T11:00:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb27fff700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb281491b0 0x7ffb280afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb277fe700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb28149590 0x7ffb280b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2cb35700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb281f64b0 0x7ffb281f6930 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ffb20012000 tx=0x7ffb20001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:51.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb24ff9700 1 -- 172.21.15.181:0/1775371046 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2000a340 con 0x7ffb281f64b0 2023-10-16T11:00:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb24ff9700 1 -- 172.21.15.181:0/1775371046 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb200019c0 con 0x7ffb281f64b0 2023-10-16T11:00:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb281f7090 con 0x7ffb281f64b0 2023-10-16T11:00:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb24ff9700 1 -- 172.21.15.181:0/1775371046 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20027410 con 0x7ffb281f64b0 2023-10-16T11:00:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.562+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb280ad750 con 0x7ffb281f64b0 2023-10-16T11:00:51.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.563+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb280adee0 con 0x7ffb281f64b0 2023-10-16T11:00:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.564+0000 7ffb24ff9700 1 -- 172.21.15.181:0/1775371046 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffb20019070 con 0x7ffb281f64b0 2023-10-16T11:00:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.564+0000 7ffb24ff9700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ffb0806b020 0x7ffb0806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.564+0000 7ffb24ff9700 1 -- 172.21.15.181:0/1775371046 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7ffb2008ac60 con 0x7ffb281f64b0 2023-10-16T11:00:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.564+0000 7ffb27fff700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ffb0806b020 0x7ffb0806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.564+0000 7ffb27fff700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ffb0806b020 0x7ffb0806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009b70 tx=0x7ffb10008040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:51.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.568+0000 7ffb24ff9700 1 -- 172.21.15.181:0/1775371046 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb20055a20 con 0x7ffb281f64b0 2023-10-16T11:00:51.719 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:51.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:00:51.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.843+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffb280b6510 con 0x7ffb281f64b0 2023-10-16T11:00:51.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.846+0000 7ffb24ff9700 1 -- 172.21.15.181:0/1775371046 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v43) v1 ==== 190+0+147 (secure 0 0 0) 0x7ffb20059040 con 0x7ffb281f64b0 2023-10-16T11:00:51.848 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T11:00:51.848 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T11:00:51.848 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T11:00:51.848 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T11:00:51.848 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:00:51.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ffb0806b020 msgr2=0x7ffb0806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ffb0806b020 0x7ffb0806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009b70 tx=0x7ffb10008040 comp rx=0 tx=0).stop 2023-10-16T11:00:51.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb281f64b0 msgr2=0x7ffb281f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb281f64b0 0x7ffb281f6930 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ffb20012000 tx=0x7ffb20001360 comp rx=0 tx=0).stop 2023-10-16T11:00:51.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 shutdown_connections 2023-10-16T11:00:51.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ffb0806b020 0x7ffb0806d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb281491b0 0x7ffb280afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb281f64b0 0x7ffb281f6930 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 --2- 172.21.15.181:0/1775371046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb28149590 0x7ffb280b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 >> 172.21.15.181:0/1775371046 conn(0x7ffb280cd010 msgr2=0x7ffb280b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 shutdown_connections 2023-10-16T11:00:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.848+0000 7ffb2e598700 1 -- 172.21.15.181:0/1775371046 wait complete. 2023-10-16T11:00:51.852 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T11:00:51.862 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:00:51.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.982+0000 7fc9a4317700 1 Processor -- start 2023-10-16T11:00:51.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.982+0000 7fc9a4317700 1 -- start start 2023-10-16T11:00:51.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a4317700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc99c15b990 0x7fc99c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a4317700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c0cc510 con 0x7fc99c0d0370 2023-10-16T11:00:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a4317700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c0cc7f0 con 0x7fc99c0cd2d0 2023-10-16T11:00:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a4317700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99c0ccad0 con 0x7fc99c15b990 2023-10-16T11:00:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a28b4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc99c0d0370 0x7fc99c0ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33694/0 (socket says 172.21.15.181:33694) 2023-10-16T11:00:51.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a28b4700 1 -- 172.21.15.181:0/2054346172 learned_addr learned my addr 172.21.15.181:0/2054346172 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a20b3700 1 --2- 172.21.15.181:0/2054346172 >> v2:172.21.15.190:3300/0 conn(0x7fc99c15b990 0x7fc99c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369789991 0 0) 0x7fc99c0cc510 con 0x7fc99c0d0370 2023-10-16T11:00:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9880035b0 con 0x7fc99c0d0370 2023-10-16T11:00:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017386702 0 0) 0x7fc9880035b0 con 0x7fc99c0d0370 2023-10-16T11:00:51.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 >> v2:172.21.15.190:3300/0 conn(0x7fc99c15b990 msgr2=0x7fc99c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 --2- 172.21.15.181:0/2054346172 >> v2:172.21.15.190:3300/0 conn(0x7fc99c15b990 0x7fc99c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 >> v1:172.21.15.190:6789/0 conn(0x7fc99c0cd2d0 legacy=0x7fc99c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0ccdb0 con 0x7fc99c0d0370 2023-10-16T11:00:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc9940019f0 con 0x7fc99c0d0370 2023-10-16T11:00:51.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.983+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 >> v1:172.21.15.181:6789/0 conn(0x7fc99c0d0370 legacy=0x7fc99c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988003a00 0x7fc988003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc988004850 0x7fc988004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc988005210 0x7fc98800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0ccad0 con 0x7fc988005210 2023-10-16T11:00:51.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800c0c0 con 0x7fc988004850 2023-10-16T11:00:51.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800c280 con 0x7fc988003a00 2023-10-16T11:00:51.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc988005210 0x7fc98800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a18b2700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc988004850 0x7fc988004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a20b3700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988003a00 0x7fc988003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:51.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 -- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988003a00 msgr2=0x7fc988003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988003a00 0x7fc988003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 -- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc988004850 msgr2=0x7fc988004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:51.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc988004850 0x7fc988004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 -- 172.21.15.181:0/2054346172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880035b0 con 0x7fc988005210 2023-10-16T11:00:51.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 -- 172.21.15.181:0/2054346172 reap_dead start 2023-10-16T11:00:51.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a28b4700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc988005210 0x7fc98800ba10 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fc99400a800 tx=0x7fc988005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260d09502b894b74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:51.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99400a2f0 con 0x7fc988005210 2023-10-16T11:00:51.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994012620 con 0x7fc988005210 2023-10-16T11:00:51.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.984+0000 7fc9a10b1700 1 -- 172.21.15.181:0/2054346172 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940129d0 con 0x7fc988005210 2023-10-16T11:00:51.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.985+0000 7fc9a4317700 1 -- 172.21.15.181:0/2054346172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc988005210 msgr2=0x7fc98800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:51.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.985+0000 7fc9a4317700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc988005210 0x7fc98800ba10 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fc99400a800 tx=0x7fc988005880 comp rx=0 tx=0).stop 2023-10-16T11:00:51.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.985+0000 7fc9a4317700 1 -- 172.21.15.181:0/2054346172 shutdown_connections 2023-10-16T11:00:51.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.985+0000 7fc9a4317700 1 --2- 172.21.15.181:0/2054346172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc988005210 0x7fc98800ba10 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:51.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.985+0000 7fc9a4317700 1 -- 172.21.15.181:0/2054346172 >> 172.21.15.181:0/2054346172 conn(0x7fc99c09a9a0 msgr2=0x7fc99c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:51.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.985+0000 7fc9a4317700 1 -- 172.21.15.181:0/2054346172 shutdown_connections 2023-10-16T11:00:51.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.985+0000 7fc9a4317700 1 -- 172.21.15.181:0/2054346172 wait complete. 2023-10-16T11:00:51.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.986+0000 7fc9a4317700 1 Processor -- start 2023-10-16T11:00:51.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.986+0000 7fc9a4317700 1 -- start start 2023-10-16T11:00:51.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a4317700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988005210 0x7fc99c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:51.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a4317700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99c0b41d0 0x7fc99c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a4317700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc99c0b4650 0x7fc99c0b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a4317700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c1f4890 con 0x7fc99c0b4650 2023-10-16T11:00:52.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a4317700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0cb270 con 0x7fc99c0b41d0 2023-10-16T11:00:52.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a4317700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0b6f00 con 0x7fc988005210 2023-10-16T11:00:52.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc99c0b4650 0x7fc99c0b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc99c0b4650 0x7fc99c0b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50860/0 (socket says 172.21.15.181:50860) 2023-10-16T11:00:52.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 -- 172.21.15.181:0/3324522885 learned_addr learned my addr 172.21.15.181:0/3324522885 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a18b2700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99c0b41d0 0x7fc99c0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a20b3700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988005210 0x7fc99c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 -- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988005210 msgr2=0x7fc99c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988005210 0x7fc99c0cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 -- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99c0b41d0 msgr2=0x7fc99c0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99c0b41d0 0x7fc99c0b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 -- 172.21.15.181:0/3324522885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c1f3950 con 0x7fc99c0b4650 2023-10-16T11:00:52.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc9a28b4700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc99c0b4650 0x7fc99c0b3030 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0cd780 tx=0x7fc994001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.987+0000 7fc992ffd700 1 -- 172.21.15.181:0/3324522885 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994025070 con 0x7fc99c0b4650 2023-10-16T11:00:52.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.988+0000 7fc992ffd700 1 -- 172.21.15.181:0/3324522885 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9940129a0 con 0x7fc99c0b4650 2023-10-16T11:00:52.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.988+0000 7fc992ffd700 1 -- 172.21.15.181:0/3324522885 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994012d30 con 0x7fc99c0b4650 2023-10-16T11:00:52.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.988+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c1f3c10 con 0x7fc99c0b4650 2023-10-16T11:00:52.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.988+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c1f41d0 con 0x7fc99c0b4650 2023-10-16T11:00:52.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.988+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c067d20 con 0x7fc99c0b4650 2023-10-16T11:00:52.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.989+0000 7fc992ffd700 1 -- 172.21.15.181:0/3324522885 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc99401e430 con 0x7fc99c0b4650 2023-10-16T11:00:52.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.989+0000 7fc992ffd700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc98006b020 0x7fc98006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.989+0000 7fc992ffd700 1 -- 172.21.15.181:0/3324522885 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fc99408b600 con 0x7fc99c0b4650 2023-10-16T11:00:52.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.989+0000 7fc9a20b3700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc98006b020 0x7fc98006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.990+0000 7fc9a20b3700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc98006b020 0x7fc98006d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc998009bc0 tx=0x7fc9980091c0 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:51.994+0000 7fc992ffd700 1 -- 172.21.15.181:0/3324522885 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc9940563c0 con 0x7fc99c0b4650 2023-10-16T11:00:52.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.232+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc99c0cecb0 con 0x7fc99c0b4650 2023-10-16T11:00:52.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.233+0000 7fc992ffd700 1 -- 172.21.15.181:0/3324522885 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4929 (secure 0 0 0) 0x7fc9940599e0 con 0x7fc99c0b4650 2023-10-16T11:00:52.235 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:00:52.235 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1460537182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1460537182},{"type":"v1","addr":"172.21.15.190:6833","nonce":1460537182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3626960628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3626960628},{"type":"v1","addr":"172.21.15.190:6835","nonce":3626960628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1772227629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1772227629},{"type":"v1","addr":"172.21.15.181:6836","nonce":1772227629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:00:46.276243+0000","modified":"2023-10-16T11:00:49.298812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9651},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9651":{"gid":9651,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6837/2931073086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2931073086},{"type":"v1","addr":"172.21.15.181:6837","nonce":2931073086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:00:52.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc98006b020 msgr2=0x7fc98006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc98006b020 0x7fc98006d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc998009bc0 tx=0x7fc9980091c0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc99c0b4650 msgr2=0x7fc99c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc99c0b4650 0x7fc99c0b3030 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0cd780 tx=0x7fc994001d80 comp rx=0 tx=0).stop 2023-10-16T11:00:52.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 shutdown_connections 2023-10-16T11:00:52.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fc98006b020 0x7fc98006d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc988005210 0x7fc99c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc99c0b4650 0x7fc99c0b3030 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 --2- 172.21.15.181:0/3324522885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc99c0b41d0 0x7fc99c0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 >> 172.21.15.181:0/3324522885 conn(0x7fc99c09a9a0 msgr2=0x7fc99c0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 shutdown_connections 2023-10-16T11:00:52.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.235+0000 7fc9a4317700 1 -- 172.21.15.181:0/3324522885 wait complete. 2023-10-16T11:00:52.239 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2023-10-16T11:00:52.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:00:46.276243+0000', 'modified': '2023-10-16T11:00:49.298812+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9651}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9651': {'gid': 9651, 'name': 'c', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.181:6837/2931073086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 2931073086}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2931073086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:00:52.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:00:52.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T11:00:52.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.776+0000 7fb32d075700 1 Processor -- start 2023-10-16T11:00:52.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.776+0000 7fb32d075700 1 -- start start 2023-10-16T11:00:52.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32d075700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32815df60 0x7fb3280cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32d075700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3280d0fc0 0x7fb3280cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32d075700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3280ccf50 con 0x7fb32806a8a0 2023-10-16T11:00:52.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32d075700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3280af9c0 con 0x7fb3280d0fc0 2023-10-16T11:00:52.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32d075700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280cc600 con 0x7fb32815df60 2023-10-16T11:00:52.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3280d0fc0 0x7fb3280cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3280d0fc0 0x7fb3280cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50864/0 (socket says 172.21.15.181:50864) 2023-10-16T11:00:52.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.777+0000 7fb32659c700 1 -- 172.21.15.181:0/1913075769 learned_addr learned my addr 172.21.15.181:0/1913075769 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb326d9d700 1 --2- 172.21.15.181:0/1913075769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32815df60 0x7fb3280cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb32659c700 1 -- 172.21.15.181:0/1913075769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32815df60 msgr2=0x7fb3280cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb32659c700 1 --2- 172.21.15.181:0/1913075769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32815df60 0x7fb3280cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb32659c700 1 -- 172.21.15.181:0/1913075769 >> v1:172.21.15.190:6789/0 conn(0x7fb32806a8a0 legacy=0x7fb32806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb32659c700 1 -- 172.21.15.181:0/1913075769 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3280cc8a0 con 0x7fb3280d0fc0 2023-10-16T11:00:52.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb32659c700 1 --2- 172.21.15.181:0/1913075769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3280d0fc0 0x7fb3280cd6e0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb3180098f0 tx=0x7fb318009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b085825b5b550 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb325d9b700 1 -- 172.21.15.181:0/1913075769 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318011830 con 0x7fb3280d0fc0 2023-10-16T11:00:52.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb325d9b700 1 -- 172.21.15.181:0/1913075769 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318011990 con 0x7fb3280d0fc0 2023-10-16T11:00:52.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.778+0000 7fb325d9b700 1 -- 172.21.15.181:0/1913075769 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318011d20 con 0x7fb3280d0fc0 2023-10-16T11:00:52.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 -- 172.21.15.181:0/1913075769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3280d0fc0 msgr2=0x7fb3280cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 --2- 172.21.15.181:0/1913075769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3280d0fc0 0x7fb3280cd6e0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb3180098f0 tx=0x7fb318009c00 comp rx=0 tx=0).stop 2023-10-16T11:00:52.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 -- 172.21.15.181:0/1913075769 shutdown_connections 2023-10-16T11:00:52.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 --2- 172.21.15.181:0/1913075769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32815df60 0x7fb3280cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 --2- 172.21.15.181:0/1913075769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3280d0fc0 0x7fb3280cd6e0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 -- 172.21.15.181:0/1913075769 >> 172.21.15.181:0/1913075769 conn(0x7fb3280d04a0 msgr2=0x7fb3280d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 -- 172.21.15.181:0/1913075769 shutdown_connections 2023-10-16T11:00:52.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.779+0000 7fb32d075700 1 -- 172.21.15.181:0/1913075769 wait complete. 2023-10-16T11:00:52.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.780+0000 7fb32d075700 1 Processor -- start 2023-10-16T11:00:52.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.780+0000 7fb32d075700 1 -- start start 2023-10-16T11:00:52.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32d075700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32806a8a0 0x7fb328157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32d075700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3280d0fc0 0x7fb3280b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32d075700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb32815df60 0x7fb3280c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32d075700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280b1670 con 0x7fb32815df60 2023-10-16T11:00:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32d075700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280b4e80 con 0x7fb3280d0fc0 2023-10-16T11:00:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32d075700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281581f0 con 0x7fb32806a8a0 2023-10-16T11:00:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb32815df60 0x7fb3280c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb32815df60 0x7fb3280c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50874/0 (socket says 172.21.15.181:50874) 2023-10-16T11:00:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 -- 172.21.15.181:0/1185282109 learned_addr learned my addr 172.21.15.181:0/1185282109 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb326d9d700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32806a8a0 0x7fb328157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32659c700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3280d0fc0 0x7fb3280b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 -- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32806a8a0 msgr2=0x7fb328157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32806a8a0 0x7fb328157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 -- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3280d0fc0 msgr2=0x7fb3280b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3280d0fc0 0x7fb3280b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 -- 172.21.15.181:0/1185282109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb318009550 con 0x7fb32815df60 2023-10-16T11:00:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32659c700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3280d0fc0 0x7fb3280b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:52.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.781+0000 7fb32759e700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb32815df60 0x7fb3280c8890 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fb32000cb50 tx=0x7fb32000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.782+0000 7fb30f7fe700 1 -- 172.21.15.181:0/1185282109 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320003c00 con 0x7fb32815df60 2023-10-16T11:00:52.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.782+0000 7fb30f7fe700 1 -- 172.21.15.181:0/1185282109 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb320003d60 con 0x7fb32815df60 2023-10-16T11:00:52.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.782+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3280b44e0 con 0x7fb32815df60 2023-10-16T11:00:52.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.782+0000 7fb30f7fe700 1 -- 172.21.15.181:0/1185282109 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32000aac0 con 0x7fb32815df60 2023-10-16T11:00:52.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.782+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3280b4aa0 con 0x7fb32815df60 2023-10-16T11:00:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.783+0000 7fb30f7fe700 1 -- 172.21.15.181:0/1185282109 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb32000ac20 con 0x7fb32815df60 2023-10-16T11:00:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.784+0000 7fb30f7fe700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb31406b020 0x7fb31406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.784+0000 7fb30f7fe700 1 -- 172.21.15.181:0/1185282109 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fb320089870 con 0x7fb32815df60 2023-10-16T11:00:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.784+0000 7fb326d9d700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb31406b020 0x7fb31406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.784+0000 7fb326d9d700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb31406b020 0x7fb31406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009bc0 tx=0x7fb31c0091c0 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.786+0000 7fb33925fb80 1 Processor -- start 2023-10-16T11:00:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.786+0000 7fb33925fb80 1 -- start start 2023-10-16T11:00:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.786+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a863a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.786+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a863fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a837010 0x55853a8376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb33925fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a7adc20 con 0x55853a861630 2023-10-16T11:00:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a792c10 con 0x55853a837010 2023-10-16T11:00:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55853a790530 con 0x55853a85c220 2023-10-16T11:00:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a863fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a863fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50878/0 (socket says 172.21.15.181:50878) 2023-10-16T11:00:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 -- 172.21.15.181:0/3589721815 learned_addr learned my addr 172.21.15.181:0/3589721815 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb326d9d700 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a863a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32759e700 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a837010 0x55853a8376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 -- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 msgr2=0x55853a863a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a863a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 -- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a837010 msgr2=0x55853a8376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a837010 0x55853a8376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 -- 172.21.15.181:0/3589721815 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55853a7493a0 con 0x55853a861630 2023-10-16T11:00:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb32659c700 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a863fb0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fb318013000 tx=0x7fb318010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda0af0bca09314b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb30cff9700 1 -- 172.21.15.181:0/3589721815 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318011d00 con 0x55853a861630 2023-10-16T11:00:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb30cff9700 1 -- 172.21.15.181:0/3589721815 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318004040 con 0x55853a861630 2023-10-16T11:00:52.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.787+0000 7fb30cff9700 1 -- 172.21.15.181:0/3589721815 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3180043d0 con 0x55853a861630 2023-10-16T11:00:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 -- 172.21.15.181:0/3589721815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 msgr2=0x55853a863fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a863fb0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fb318013000 tx=0x7fb318010000 comp rx=0 tx=0).stop 2023-10-16T11:00:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 -- 172.21.15.181:0/3589721815 shutdown_connections 2023-10-16T11:00:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a863a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a837010 0x55853a8376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3589721815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a863fb0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 -- 172.21.15.181:0/3589721815 >> 172.21.15.181:0/3589721815 conn(0x55853a766d80 msgr2=0x55853a881ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 -- 172.21.15.181:0/3589721815 shutdown_connections 2023-10-16T11:00:52.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.788+0000 7fb33925fb80 1 -- 172.21.15.181:0/3589721815 wait complete. 2023-10-16T11:00:52.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 -- start start 2023-10-16T11:00:52.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 Processor -- start 2023-10-16T11:00:52.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a837010 0x55853a85b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a85c220 0x55853a85df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a861630 0x55853a866bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a78be20 con 0x55853a837010 2023-10-16T11:00:52.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a7286c0 con 0x55853a85c220 2023-10-16T11:00:52.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.789+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55853a728360 con 0x55853a861630 2023-10-16T11:00:52.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb33925fb80 10 client.0 _ll_register_callbacks cb 0x55853a5857b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:00:52.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a837010 0x55853a85b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a837010 0x55853a85b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50884/0 (socket says 172.21.15.181:50884) 2023-10-16T11:00:52.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 -- 172.21.15.181:0/3272613276 learned_addr learned my addr 172.21.15.181:0/3272613276 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb32659c700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a85c220 0x55853a85df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb32759e700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a861630 0x55853a866bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 -- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a861630 msgr2=0x55853a866bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a861630 0x55853a866bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 -- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a85c220 msgr2=0x55853a85df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a85c220 0x55853a85df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 -- 172.21.15.181:0/3272613276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3280cc8a0 con 0x55853a837010 2023-10-16T11:00:52.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb32759e700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a861630 0x55853a866bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:52.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb326d9d700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a837010 0x55853a85b5b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fb31c006080 tx=0x7fb31c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c3979e17210b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb305ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:00:52.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb305ffb700 1 -- 172.21.15.181:0/3272613276 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0068d0 con 0x55853a837010 2023-10-16T11:00:52.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb305ffb700 1 -- 172.21.15.181:0/3272613276 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fb31c014440 con 0x55853a837010 2023-10-16T11:00:52.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55853a82c5b0 con 0x55853a837010 2023-10-16T11:00:52.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb305ffb700 1 -- 172.21.15.181:0/3272613276 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb31c014640 con 0x55853a837010 2023-10-16T11:00:52.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb33925fb80 10 client.9798 fetch_fsmap learned FSMap version 85 2023-10-16T11:00:52.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55853a5b6d90 con 0x55853a837010 2023-10-16T11:00:52.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.790+0000 7fb305ffb700 1 -- 172.21.15.181:0/3272613276 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55853a82c5b0 con 0x55853a837010 2023-10-16T11:00:52.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb305ffb700 1 -- 172.21.15.181:0/3272613276 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 85) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb31c028020 con 0x55853a837010 2023-10-16T11:00:52.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb33925fb80 10 client.9798 fetch_fsmap finished waiting for FSMap version 85 2023-10-16T11:00:52.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb33925fb80 10 client.9798 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:00:52.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb33925fb80 20 client.9798 populate_metadata read hostname 'smithi181' 2023-10-16T11:00:52.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8670f0 0x55853a85cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb33925fb80 4 client.9798 mds_command: new command op to 9651 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:00:52.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 --> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55853a784e00 con 0x55853a8670f0 2023-10-16T11:00:52.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb32659c700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8670f0 0x55853a85cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb32659c700 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8670f0 0x55853a85cd10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18a43e772b988a9 server_cookie=47b998f9b4ef34ca in_seq=0 out_seq=0 2023-10-16T11:00:52.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.791+0000 7fb305ffb700 10 client.9798 ms_handle_connect on v2:172.21.15.181:6835/2931073086 2023-10-16T11:00:52.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb305ffb700 1 -- 172.21.15.181:0/3272613276 <== mds.0 v2:172.21.15.181:6835/2931073086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55853a784e00 con 0x55853a8670f0 2023-10-16T11:00:52.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb305ffb700 10 client.9798 handle_command_reply: tid=0 2023-10-16T11:00:52.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 client.9798 shutdown 2023-10-16T11:00:52.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a837010 msgr2=0x55853a85b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a837010 0x55853a85b5b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fb31c006080 tx=0x7fb31c004750 comp rx=0 tx=0).stop 2023-10-16T11:00:52.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 shutdown_connections 2023-10-16T11:00:52.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8670f0 0x55853a85cd10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a861630 0x55853a866bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a837010 0x55853a85b5b0 secure :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fb31c006080 tx=0x7fb31c004750 comp rx=0 tx=0).stop 2023-10-16T11:00:52.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 --2- 172.21.15.181:0/3272613276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a85c220 0x55853a85df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb305ffb700 0 client.9798 ms_handle_reset on v2:172.21.15.181:6835/2931073086 2023-10-16T11:00:52.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 >> 172.21.15.181:0/3272613276 conn(0x55853a766d80 msgr2=0x55853a87f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 shutdown_connections 2023-10-16T11:00:52.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 1 -- 172.21.15.181:0/3272613276 wait complete. 2023-10-16T11:00:52.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.792+0000 7fb33925fb80 20 client.9798 trim_cache size 0 max 16384 2023-10-16T11:00:52.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 Processor -- start 2023-10-16T11:00:52.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 -- start start 2023-10-16T11:00:52.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a85c220 0x55853a860e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a861630 0x55853a878420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a88cee0 0x55853a878960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a60c8d0 con 0x55853a85c220 2023-10-16T11:00:52.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a80f3c0 con 0x55853a861630 2023-10-16T11:00:52.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55853a88d880 con 0x55853a88cee0 2023-10-16T11:00:52.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb326d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a85c220 0x55853a860e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb326d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a85c220 0x55853a860e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50892/0 (socket says 172.21.15.181:50892) 2023-10-16T11:00:52.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.810+0000 7fb326d9d700 1 -- 172.21.15.181:0/2691331082 learned_addr learned my addr 172.21.15.181:0/2691331082 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb32659c700 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a861630 0x55853a878420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb32759e700 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a88cee0 0x55853a878960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb326d9d700 1 -- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a88cee0 msgr2=0x55853a878960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb326d9d700 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a88cee0 0x55853a878960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb326d9d700 1 -- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a861630 msgr2=0x55853a878420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb326d9d700 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a861630 0x55853a878420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb326d9d700 1 -- 172.21.15.181:0/2691331082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55853a59fc10 con 0x55853a85c220 2023-10-16T11:00:52.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb32659c700 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a861630 0x55853a878420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:52.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb326d9d700 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a85c220 0x55853a860e30 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0092b0 tx=0x7fb31c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683c0834d98eefd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb30d7fa700 1 -- 172.21.15.181:0/2691331082 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c014440 con 0x55853a85c220 2023-10-16T11:00:52.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb30d7fa700 1 -- 172.21.15.181:0/2691331082 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c0145e0 con 0x55853a85c220 2023-10-16T11:00:52.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb30d7fa700 1 -- 172.21.15.181:0/2691331082 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c014c90 con 0x55853a85c220 2023-10-16T11:00:52.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb33925fb80 1 -- 172.21.15.181:0/2691331082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a85c220 msgr2=0x55853a860e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.811+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a85c220 0x55853a860e30 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0092b0 tx=0x7fb31c024000 comp rx=0 tx=0).stop 2023-10-16T11:00:52.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 -- 172.21.15.181:0/2691331082 shutdown_connections 2023-10-16T11:00:52.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a88cee0 0x55853a878960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a85c220 0x55853a860e30 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2691331082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a861630 0x55853a878420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 -- 172.21.15.181:0/2691331082 >> 172.21.15.181:0/2691331082 conn(0x55853a766d80 msgr2=0x55853a8a07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 -- 172.21.15.181:0/2691331082 shutdown_connections 2023-10-16T11:00:52.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 -- 172.21.15.181:0/2691331082 wait complete. 2023-10-16T11:00:52.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 -- start start 2023-10-16T11:00:52.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.812+0000 7fb33925fb80 1 Processor -- start 2023-10-16T11:00:52.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a865dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a8373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb33925fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a88cee0 0x55853a837930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb33925fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a8e6da0 con 0x55853a861630 2023-10-16T11:00:52.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853a85aa60 con 0x55853a88cee0 2023-10-16T11:00:52.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb33925fb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55853a865020 con 0x55853a85c220 2023-10-16T11:00:52.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb33925fb80 10 client.0 _ll_register_callbacks cb 0x7fb31c011be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:00:52.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a8373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a8373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50900/0 (socket says 172.21.15.181:50900) 2023-10-16T11:00:52.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 -- 172.21.15.181:0/2177629086 learned_addr learned my addr 172.21.15.181:0/2177629086 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32759e700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a88cee0 0x55853a837930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb326d9d700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a865dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 -- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 msgr2=0x55853a865dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a865dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 -- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a88cee0 msgr2=0x55853a837930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a88cee0 0x55853a837930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 -- 172.21.15.181:0/2177629086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55853a5b6d90 con 0x55853a861630 2023-10-16T11:00:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb326d9d700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a865dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:52.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb32659c700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a8373f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb318000c00 tx=0x7fb31800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bc1305b41c0227 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.813+0000 7fb305ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:00:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 1 -- 172.21.15.181:0/2177629086 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31802a070 con 0x55853a861630 2023-10-16T11:00:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 1 -- 172.21.15.181:0/2177629086 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fb318034a00 con 0x55853a861630 2023-10-16T11:00:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb318004040 con 0x55853a861630 2023-10-16T11:00:52.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 1 -- 172.21.15.181:0/2177629086 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3180045c0 con 0x55853a861630 2023-10-16T11:00:52.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 10 client.9804 fetch_fsmap learned FSMap version 85 2023-10-16T11:00:52.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55853a5dd200 con 0x55853a861630 2023-10-16T11:00:52.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 1 -- 172.21.15.181:0/2177629086 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318004040 con 0x55853a861630 2023-10-16T11:00:52.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 1 -- 172.21.15.181:0/2177629086 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 85) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb318034bc0 con 0x55853a861630 2023-10-16T11:00:52.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 10 client.9804 fetch_fsmap finished waiting for FSMap version 85 2023-10-16T11:00:52.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 10 client.9804 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:00:52.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 20 client.9804 populate_metadata read hostname 'smithi181' 2023-10-16T11:00:52.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8e5ed0 0x55853a8e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 4 client.9804 mds_command: new command op to 9651 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:00:52.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 --> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55853a784b20 con 0x55853a8e5ed0 2023-10-16T11:00:52.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb326d9d700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8e5ed0 0x55853a8e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 10 client.9804 ms_handle_connect on v2:172.21.15.181:6835/2931073086 2023-10-16T11:00:52.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb326d9d700 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8e5ed0 0x55853a8e62b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7b835a43ed11a5 server_cookie=ada6df4716809ed6 in_seq=0 out_seq=0 2023-10-16T11:00:52.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 1 -- 172.21.15.181:0/2177629086 <== mds.0 v2:172.21.15.181:6835/2931073086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55853a784b20 con 0x55853a8e5ed0 2023-10-16T11:00:52.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb305ffb700 10 client.9804 handle_command_reply: tid=0 2023-10-16T11:00:52.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.814+0000 7fb33925fb80 1 client.9804 shutdown 2023-10-16T11:00:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 msgr2=0x55853a8373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a8373f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb318000c00 tx=0x7fb31800e000 comp rx=0 tx=0).stop 2023-10-16T11:00:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 shutdown_connections 2023-10-16T11:00:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:6835/2931073086,v1:172.21.15.181:6837/2931073086] conn(0x55853a8e5ed0 0x55853a8e62b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55853a85c220 0x55853a865dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb305ffb700 0 client.9804 ms_handle_reset on v2:172.21.15.181:6835/2931073086 2023-10-16T11:00:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55853a88cee0 0x55853a837930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 --2- 172.21.15.181:0/2177629086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55853a861630 0x55853a8373f0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 >> 172.21.15.181:0/2177629086 conn(0x55853a766d80 msgr2=0x55853a839dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 shutdown_connections 2023-10-16T11:00:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 1 -- 172.21.15.181:0/2177629086 wait complete. 2023-10-16T11:00:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb33925fb80 20 client.9804 trim_cache size 0 max 16384 2023-10-16T11:00:52.837 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:00:52.837 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:00:52.838 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:00:52.838 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 9651, 2023-10-16T11:00:52.838 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:00:52.838 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:00:52.838 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T11:00:52.839 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9238844799999999, 2023-10-16T11:00:52.839 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 85, 2023-10-16T11:00:52.839 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 168, 2023-10-16T11:00:52.839 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 168, 2023-10-16T11:00:52.839 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 17.893923135000001 2023-10-16T11:00:52.840 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:00:52.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb31406b020 msgr2=0x7fb31406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb31406b020 0x7fb31406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009bc0 tx=0x7fb31c0091c0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb32815df60 msgr2=0x7fb3280c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb32815df60 0x7fb3280c8890 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fb32000cb50 tx=0x7fb32000ce60 comp rx=0 tx=0).stop 2023-10-16T11:00:52.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 shutdown_connections 2023-10-16T11:00:52.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fb31406b020 0x7fb31406d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb32806a8a0 0x7fb328157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb32815df60 0x7fb3280c8890 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 --2- 172.21.15.181:0/1185282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3280d0fc0 0x7fb3280b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 >> 172.21.15.181:0/1185282109 conn(0x7fb3280d04a0 msgr2=0x7fb32806d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 shutdown_connections 2023-10-16T11:00:52.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.815+0000 7fb32d075700 1 -- 172.21.15.181:0/1185282109 wait complete. 2023-10-16T11:00:52.844 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:00:52.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.966+0000 7f5a3fe7b700 1 Processor -- start 2023-10-16T11:00:52.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.967+0000 7f5a3fe7b700 1 -- start start 2023-10-16T11:00:52.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.967+0000 7f5a3fe7b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5a380ce7f0 0x7f5a3806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.967+0000 7f5a3fe7b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5a380d0350 0x7f5a380cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.967+0000 7f5a3fe7b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a380cc470 con 0x7f5a380d2390 2023-10-16T11:00:52.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.967+0000 7f5a3fe7b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3806ca30 con 0x7f5a380ce7f0 2023-10-16T11:00:52.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.967+0000 7f5a3fe7b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3806cd10 con 0x7f5a380d0350 2023-10-16T11:00:52.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.967+0000 7f5a3dc17700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5a380ce7f0 0x7f5a3806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3dc17700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5a380ce7f0 0x7f5a3806c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50906/0 (socket says 172.21.15.181:50906) 2023-10-16T11:00:52.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2357253262 learned_addr learned my addr 172.21.15.181:0/2357253262 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3d416700 1 --1- 172.21.15.181:0/2357253262 >> v1:172.21.15.181:6789/0 conn(0x7f5a380d2390 0x7f5a380cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33754/0 (socket says 172.21.15.181:33754) 2023-10-16T11:00:52.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3e418700 1 --2- 172.21.15.181:0/2357253262 >> v2:172.21.15.190:3300/0 conn(0x7f5a380d0350 0x7f5a380cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2357253262 >> v2:172.21.15.190:3300/0 conn(0x7f5a380d0350 msgr2=0x7f5a380cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:52.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3dc17700 1 --2- 172.21.15.181:0/2357253262 >> v2:172.21.15.190:3300/0 conn(0x7f5a380d0350 0x7f5a380cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2357253262 >> v1:172.21.15.181:6789/0 conn(0x7f5a380d2390 legacy=0x7f5a380cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2357253262 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3806cff0 con 0x7f5a380ce7f0 2023-10-16T11:00:52.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3cc15700 1 -- 172.21.15.181:0/2357253262 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71379011 0 0) 0x7f5a380cc470 con 0x7f5a380d2390 2023-10-16T11:00:52.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3e418700 1 --2- 172.21.15.181:0/2357253262 >> v2:172.21.15.190:3300/0 conn(0x7f5a380d0350 0x7f5a380cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:00:52.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3dc17700 1 --2- 172.21.15.181:0/2357253262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a3806c100 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f5a38072b80 tx=0x7f5a28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abec792a345e41ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3cc15700 1 -- 172.21.15.181:0/2357253262 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28005e30 con 0x7f5a380ce7f0 2023-10-16T11:00:52.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3cc15700 1 -- 172.21.15.181:0/2357253262 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a28004030 con 0x7f5a380ce7f0 2023-10-16T11:00:52.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.968+0000 7f5a3cc15700 1 -- 172.21.15.181:0/2357253262 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a280043e0 con 0x7f5a380ce7f0 2023-10-16T11:00:52.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2357253262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 msgr2=0x7f5a3806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2357253262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a3806c100 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f5a38072b80 tx=0x7f5a28009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2357253262 shutdown_connections 2023-10-16T11:00:52.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2357253262 >> v2:172.21.15.190:3300/0 conn(0x7f5a380d0350 0x7f5a380cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2357253262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a3806c100 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2357253262 >> 172.21.15.181:0/2357253262 conn(0x7f5a3809a9a0 msgr2=0x7f5a380d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:52.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2357253262 shutdown_connections 2023-10-16T11:00:52.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.969+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2357253262 wait complete. 2023-10-16T11:00:52.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.970+0000 7f5a3fe7b700 1 Processor -- start 2023-10-16T11:00:52.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.970+0000 7f5a3fe7b700 1 -- start start 2023-10-16T11:00:52.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a380c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a380d0350 0x7f5a380ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a380d2390 0x7f5a380c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a381f3c70 con 0x7f5a380ce7f0 2023-10-16T11:00:52.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380c8020 con 0x7f5a380d2390 2023-10-16T11:00:52.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a381f30f0 con 0x7f5a380d0350 2023-10-16T11:00:52.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a380c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a380c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50912/0 (socket says 172.21.15.181:50912) 2023-10-16T11:00:52.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2565648635 learned_addr learned my addr 172.21.15.181:0/2565648635 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:52.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a380d0350 msgr2=0x7f5a380ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3d416700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a380d0350 0x7f5a380ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3e418700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a380d2390 0x7f5a380c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a380d0350 0x7f5a380ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a380d2390 msgr2=0x7f5a380c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:52.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a380d2390 0x7f5a380c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:52.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 -- 172.21.15.181:0/2565648635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a28009500 con 0x7f5a380ce7f0 2023-10-16T11:00:52.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3dc17700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a380c51b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009c10 tx=0x7f5a28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a2e7fc700 1 -- 172.21.15.181:0/2565648635 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28003dd0 con 0x7f5a380ce7f0 2023-10-16T11:00:52.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a2e7fc700 1 -- 172.21.15.181:0/2565648635 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a28011420 con 0x7f5a380ce7f0 2023-10-16T11:00:52.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a381f33d0 con 0x7f5a380ce7f0 2023-10-16T11:00:52.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a2e7fc700 1 -- 172.21.15.181:0/2565648635 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a280117b0 con 0x7f5a380ce7f0 2023-10-16T11:00:52.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.971+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a381f3990 con 0x7f5a380ce7f0 2023-10-16T11:00:52.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.972+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a38067d20 con 0x7f5a380ce7f0 2023-10-16T11:00:52.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.973+0000 7f5a2e7fc700 1 -- 172.21.15.181:0/2565648635 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5a28011910 con 0x7f5a380ce7f0 2023-10-16T11:00:52.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.973+0000 7f5a2e7fc700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5a1c06b020 0x7f5a1c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:52.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.973+0000 7f5a2e7fc700 1 -- 172.21.15.181:0/2565648635 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f5a2808ab60 con 0x7f5a380ce7f0 2023-10-16T11:00:52.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.973+0000 7f5a3d416700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5a1c06b020 0x7f5a1c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:52.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.974+0000 7f5a3d416700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5a1c06b020 0x7f5a1c06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5a3815f330 tx=0x7f5a34009040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:52.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:52.978+0000 7f5a2e7fc700 1 -- 172.21.15.181:0/2565648635 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5a280559a0 con 0x7f5a380ce7f0 2023-10-16T11:00:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.213+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a380b2930 con 0x7f5a380ce7f0 2023-10-16T11:00:53.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.213+0000 7f5a2e7fc700 1 -- 172.21.15.181:0/2565648635 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5a28058fc0 con 0x7f5a380ce7f0 2023-10-16T11:00:53.215 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:00:53.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.215+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5a1c06b020 msgr2=0x7f5a1c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:53.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.215+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5a1c06b020 0x7f5a1c06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5a3815f330 tx=0x7f5a34009040 comp rx=0 tx=0).stop 2023-10-16T11:00:53.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.215+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 msgr2=0x7f5a380c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:53.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.215+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a380c51b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009c10 tx=0x7f5a28004750 comp rx=0 tx=0).stop 2023-10-16T11:00:53.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 shutdown_connections 2023-10-16T11:00:53.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f5a1c06b020 0x7f5a1c06d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:53.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a380d2390 0x7f5a380c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a380ce7f0 0x7f5a380c51b0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 --2- 172.21.15.181:0/2565648635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a380d0350 0x7f5a380ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 >> 172.21.15.181:0/2565648635 conn(0x7f5a3809a9a0 msgr2=0x7f5a380c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 shutdown_connections 2023-10-16T11:00:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:53.216+0000 7f5a3fe7b700 1 -- 172.21.15.181:0/2565648635 wait complete. 2023-10-16T11:00:53.229 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:00:53.230 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:00:53.230 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T11:00:53.230 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:00:53.231 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:00:53.231 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:00:53.231 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:00:53.231 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:00:53.232 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:00:53.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.233 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T11:00:53.250 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:00:53.251 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:00:53.251 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:00:53.251 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:00:53.251 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T11:00:53.251 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:00:53.252 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:00:53.252 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:00:53.252 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:00:53.252 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:00:53.252 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:00:53.253 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T11:00:53.253 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:00:53.253 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T11:00:53.253 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:00:53.253 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 37817sec preferred_lft 37817sec 2023-10-16T11:00:53.254 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T11:00:53.254 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:00:53.254 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:00:53.254 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:00:53.254 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:00:53.255 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:00:53.255 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T11:00:53.255 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:00:53.255 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:00:53.255 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:00:53.255 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T11:00:53.256 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:00:53.256 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:00:53.256 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:00:53.256 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T11:00:53.256 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:00:53.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.257 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:00:53.315 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T11:00:53.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.317 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T11:00:53.375 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T11:00:53.375 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:00:53.375 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:00:53.376 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:00:53.376 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:00:53.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.377 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T11:00:53.378 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T11:00:53.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:00:53.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:00:53.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:00:53.379 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T11:00:53.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.482 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:00:53.498 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:00:53.499 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:00:53.499 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:00:53.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.500 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T11:00:53.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:00:53.573 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:00:53.631 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:00:53.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:00:53.632 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:00:53.691 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T11:00:53.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.692 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:00:53.781 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:00:53.782 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:00:53.782 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:00:53.792 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:00:53.792 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:00:53.793 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:00:53.793 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:00:53.795 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:00:53.864 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:00:53.865 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:00:53.865 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:00:53.865 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:00:53.865 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:00:53.866 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T11:00:53.866 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T11:00:53.866 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T11:00:53.867 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:00:53.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.868 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:00:53.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:53.897 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:00:53.977 INFO:teuthology.orchestra.run.smithi181.stdout:client7805 2023-10-16T11:00:53.978 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:00:53.978 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:00:54.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:00:54.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.198+0000 7f7965c66700 1 Processor -- start 2023-10-16T11:00:54.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.198+0000 7f7965c66700 1 -- start start 2023-10-16T11:00:54.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.198+0000 7f7965c66700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f796015dd50 0x7f79600d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.198+0000 7f7965c66700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79600d1e80 0x7f79600d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f7965c66700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79600b0410 con 0x7f79600ce440 2023-10-16T11:00:54.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f7965c66700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79600b3c00 con 0x7f79600d1e80 2023-10-16T11:00:54.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f7965c66700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796015e130 con 0x7f796015dd50 2023-10-16T11:00:54.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79600d1e80 0x7f79600d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f79600ce440 0x7f79600ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33770/0 (socket says 172.21.15.181:33770) 2023-10-16T11:00:54.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795ffff700 1 -- 172.21.15.181:0/2117078794 learned_addr learned my addr 172.21.15.181:0/2117078794 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:54.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795f7fe700 1 --2- 172.21.15.181:0/2117078794 >> v2:172.21.15.190:3300/0 conn(0x7f796015dd50 0x7f79600d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795effd700 1 -- 172.21.15.181:0/2117078794 >> v2:172.21.15.190:3300/0 conn(0x7f796015dd50 msgr2=0x7f79600d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:54.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795effd700 1 --2- 172.21.15.181:0/2117078794 >> v2:172.21.15.190:3300/0 conn(0x7f796015dd50 0x7f79600d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795effd700 1 -- 172.21.15.181:0/2117078794 >> v1:172.21.15.181:6789/0 conn(0x7f79600ce440 legacy=0x7f79600ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795effd700 1 -- 172.21.15.181:0/2117078794 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79600b2ca0 con 0x7f79600d1e80 2023-10-16T11:00:54.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795e7fc700 1 -- 172.21.15.181:0/2117078794 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994909251 0 0) 0x7f79600b0410 con 0x7f79600ce440 2023-10-16T11:00:54.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795effd700 1 --2- 172.21.15.181:0/2117078794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600d1e80 0x7f79600d2300 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f796008b770 tx=0x7f7954009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368e1e2c7e86399f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:54.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795e7fc700 1 -- 172.21.15.181:0/2117078794 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954011830 con 0x7f79600d1e80 2023-10-16T11:00:54.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795e7fc700 1 -- 172.21.15.181:0/2117078794 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7954011990 con 0x7f79600d1e80 2023-10-16T11:00:54.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.199+0000 7f795e7fc700 1 -- 172.21.15.181:0/2117078794 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954011d20 con 0x7f79600d1e80 2023-10-16T11:00:54.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 -- 172.21.15.181:0/2117078794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600d1e80 msgr2=0x7f79600d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 --2- 172.21.15.181:0/2117078794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600d1e80 0x7f79600d2300 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f796008b770 tx=0x7f7954009b20 comp rx=0 tx=0).stop 2023-10-16T11:00:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 -- 172.21.15.181:0/2117078794 shutdown_connections 2023-10-16T11:00:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 --2- 172.21.15.181:0/2117078794 >> v2:172.21.15.190:3300/0 conn(0x7f796015dd50 0x7f79600d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 --2- 172.21.15.181:0/2117078794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600d1e80 0x7f79600d2300 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 -- 172.21.15.181:0/2117078794 >> 172.21.15.181:0/2117078794 conn(0x7f796008af90 msgr2=0x7f79600d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 -- 172.21.15.181:0/2117078794 shutdown_connections 2023-10-16T11:00:54.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.200+0000 7f7965c66700 1 -- 172.21.15.181:0/2117078794 wait complete. 2023-10-16T11:00:54.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.202+0000 7f7965c66700 1 Processor -- start 2023-10-16T11:00:54.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.202+0000 7f7965c66700 1 -- start start 2023-10-16T11:00:54.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7965c66700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600ce440 0x7f79600c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7965c66700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79600d1e80 0x7f79600b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7965c66700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796015dd50 0x7f79600b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7965c66700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600b1e50 con 0x7f79600ce440 2023-10-16T11:00:54.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7965c66700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600b11d0 con 0x7f79600d1e80 2023-10-16T11:00:54.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7965c66700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79600b14b0 con 0x7f796015dd50 2023-10-16T11:00:54.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600ce440 0x7f79600c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600ce440 0x7f79600c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50928/0 (socket says 172.21.15.181:50928) 2023-10-16T11:00:54.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 -- 172.21.15.181:0/2438670668 learned_addr learned my addr 172.21.15.181:0/2438670668 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:54.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795effd700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79600d1e80 0x7f79600b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795ffff700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796015dd50 0x7f79600b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 -- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796015dd50 msgr2=0x7f79600b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796015dd50 0x7f79600b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 -- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79600d1e80 msgr2=0x7f79600b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79600d1e80 0x7f79600b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 -- 172.21.15.181:0/2438670668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540094a0 con 0x7f79600ce440 2023-10-16T11:00:54.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f795f7fe700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600ce440 0x7f79600c9640 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f795000d7d0 tx=0x7f795000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:54.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7947fff700 1 -- 172.21.15.181:0/2438670668 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500125f0 con 0x7f79600ce440 2023-10-16T11:00:54.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7947fff700 1 -- 172.21.15.181:0/2438670668 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7950012750 con 0x7f79600ce440 2023-10-16T11:00:54.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79600b1790 con 0x7f79600ce440 2023-10-16T11:00:54.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.203+0000 7f7947fff700 1 -- 172.21.15.181:0/2438670668 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950012b00 con 0x7f79600ce440 2023-10-16T11:00:54.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.204+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79600c9b80 con 0x7f79600ce440 2023-10-16T11:00:54.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.205+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79600d1c60 con 0x7f79600ce440 2023-10-16T11:00:54.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.205+0000 7f7947fff700 1 -- 172.21.15.181:0/2438670668 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7950012c60 con 0x7f79600ce440 2023-10-16T11:00:54.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.205+0000 7f7947fff700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f794c06b050 0x7f794c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.205+0000 7f7947fff700 1 -- 172.21.15.181:0/2438670668 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f79500898b0 con 0x7f79600ce440 2023-10-16T11:00:54.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.205+0000 7f795effd700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f794c06b050 0x7f794c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.206+0000 7f795effd700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f794c06b050 0x7f794c06d510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79540090b0 tx=0x7f7954008030 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:54.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.210+0000 7f7947fff700 1 -- 172.21.15.181:0/2438670668 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7950054660 con 0x7f79600ce440 2023-10-16T11:00:54.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.451+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 --> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f79600b3260 con 0x7f794c06b050 2023-10-16T11:00:54.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.452+0000 7f7947fff700 1 -- 172.21.15.181:0/2438670668 <== mgr.9405 v2:172.21.15.181:6832/551790976 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7954011630 con 0x7f794c06b050 2023-10-16T11:00:54.454 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:00:54.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:00:54.455 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T11:00:54.455 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:00:54.455 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:00:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f794c06b050 msgr2=0x7f794c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f794c06b050 0x7f794c06d510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79540090b0 tx=0x7f7954008030 comp rx=0 tx=0).stop 2023-10-16T11:00:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600ce440 msgr2=0x7f79600c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600ce440 0x7f79600c9640 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f795000d7d0 tx=0x7f795000dae0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 shutdown_connections 2023-10-16T11:00:54.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f794c06b050 0x7f794c06d510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796015dd50 0x7f79600b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79600ce440 0x7f79600c9640 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 --2- 172.21.15.181:0/2438670668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79600d1e80 0x7f79600b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.455+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 >> 172.21.15.181:0/2438670668 conn(0x7f796008af90 msgr2=0x7f796006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:54.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.456+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 shutdown_connections 2023-10-16T11:00:54.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.456+0000 7f7965c66700 1 -- 172.21.15.181:0/2438670668 wait complete. 2023-10-16T11:00:54.470 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:00:54.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.596+0000 7fccb6c50700 1 Processor -- start 2023-10-16T11:00:54.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.596+0000 7fccb6c50700 1 -- start start 2023-10-16T11:00:54.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.596+0000 7fccb6c50700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccb015ea40 0x7fccb00d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb6c50700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb00cf580 con 0x7fccb00d01d0 2023-10-16T11:00:54.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb6c50700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb00ce970 con 0x7fccb00ce2d0 2023-10-16T11:00:54.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb6c50700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb00d4290 con 0x7fccb015ea40 2023-10-16T11:00:54.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb51ed700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccb015ea40 0x7fccb00d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb49ec700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fccb00d01d0 0x7fccb00d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33782/0 (socket says 172.21.15.181:33782) 2023-10-16T11:00:54.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb51ed700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccb015ea40 0x7fccb00d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50936/0 (socket says 172.21.15.181:50936) 2023-10-16T11:00:54.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb49ec700 1 -- 172.21.15.181:0/2531331567 learned_addr learned my addr 172.21.15.181:0/2531331567 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:54.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb51ed700 1 -- 172.21.15.181:0/2531331567 >> v1:172.21.15.190:6789/0 conn(0x7fccb00ce2d0 legacy=0x7fccb00d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:54.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb51ed700 1 -- 172.21.15.181:0/2531331567 >> v1:172.21.15.181:6789/0 conn(0x7fccb00d01d0 legacy=0x7fccb00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccaf7fe700 1 -- 172.21.15.181:0/2531331567 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127928914 0 0) 0x7fccb00cf580 con 0x7fccb00d01d0 2023-10-16T11:00:54.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb51ed700 1 -- 172.21.15.181:0/2531331567 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00cec50 con 0x7fccb015ea40 2023-10-16T11:00:54.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.597+0000 7fccb51ed700 1 --2- 172.21.15.181:0/2531331567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb015ea40 0x7fccb00d1c10 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009d90 tx=0x7fcca800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee4194bc6e2edf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:54.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccaf7fe700 1 -- 172.21.15.181:0/2531331567 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca800e070 con 0x7fccb015ea40 2023-10-16T11:00:54.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccaf7fe700 1 -- 172.21.15.181:0/2531331567 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca800bb90 con 0x7fccb015ea40 2023-10-16T11:00:54.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccaf7fe700 1 -- 172.21.15.181:0/2531331567 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8003a20 con 0x7fccb015ea40 2023-10-16T11:00:54.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccb6c50700 1 -- 172.21.15.181:0/2531331567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb015ea40 msgr2=0x7fccb00d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccb6c50700 1 --2- 172.21.15.181:0/2531331567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb015ea40 0x7fccb00d1c10 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009d90 tx=0x7fcca800b330 comp rx=0 tx=0).stop 2023-10-16T11:00:54.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccb6c50700 1 -- 172.21.15.181:0/2531331567 shutdown_connections 2023-10-16T11:00:54.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccb6c50700 1 --2- 172.21.15.181:0/2531331567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb015ea40 0x7fccb00d1c10 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccb6c50700 1 -- 172.21.15.181:0/2531331567 >> 172.21.15.181:0/2531331567 conn(0x7fccb009cb60 msgr2=0x7fccb00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:54.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccb6c50700 1 -- 172.21.15.181:0/2531331567 shutdown_connections 2023-10-16T11:00:54.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.598+0000 7fccb6c50700 1 -- 172.21.15.181:0/2531331567 wait complete. 2023-10-16T11:00:54.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.600+0000 7fccb6c50700 1 Processor -- start 2023-10-16T11:00:54.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.600+0000 7fccb6c50700 1 -- start start 2023-10-16T11:00:54.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccb00ce2d0 0x7fccb0152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb00d01d0 0x7fccb01f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb015ea40 0x7fccb01f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb01f7ac0 con 0x7fccb00d01d0 2023-10-16T11:00:54.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0153290 con 0x7fccb015ea40 2023-10-16T11:00:54.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00c8200 con 0x7fccb00ce2d0 2023-10-16T11:00:54.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb00d01d0 0x7fccb01f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb00d01d0 0x7fccb01f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50942/0 (socket says 172.21.15.181:50942) 2023-10-16T11:00:54.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 -- 172.21.15.181:0/1475361713 learned_addr learned my addr 172.21.15.181:0/1475361713 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:54.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb49ec700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccb00ce2d0 0x7fccb0152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb51ed700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb015ea40 0x7fccb01f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 -- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccb00ce2d0 msgr2=0x7fccb0152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccb00ce2d0 0x7fccb0152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 -- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb015ea40 msgr2=0x7fccb01f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:54.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb015ea40 0x7fccb01f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:54.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 -- 172.21.15.181:0/1475361713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009a70 con 0x7fccb00d01d0 2023-10-16T11:00:54.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb49ec700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccb00ce2d0 0x7fccb0152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:54.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb51ed700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb015ea40 0x7fccb01f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:00:54.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccaffff700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb00d01d0 0x7fccb01f6c60 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b4a0 tx=0x7fcc9c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:54.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccad7fa700 1 -- 172.21.15.181:0/1475361713 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c011830 con 0x7fccb00d01d0 2023-10-16T11:00:54.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccad7fa700 1 -- 172.21.15.181:0/1475361713 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c011990 con 0x7fccb00d01d0 2023-10-16T11:00:54.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb00c84e0 con 0x7fccb00d01d0 2023-10-16T11:00:54.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccad7fa700 1 -- 172.21.15.181:0/1475361713 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c011d40 con 0x7fccb00d01d0 2023-10-16T11:00:54.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.601+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb00c8aa0 con 0x7fccb00d01d0 2023-10-16T11:00:54.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.603+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb00b30a0 con 0x7fccb00d01d0 2023-10-16T11:00:54.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.603+0000 7fccad7fa700 1 -- 172.21.15.181:0/1475361713 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 99) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcc9c01a400 con 0x7fccb00d01d0 2023-10-16T11:00:54.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.603+0000 7fccad7fa700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fcc9406b020 0x7fcc9406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:54.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.603+0000 7fccad7fa700 1 -- 172.21.15.181:0/1475361713 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fcc9c0895d0 con 0x7fccb00d01d0 2023-10-16T11:00:54.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.603+0000 7fccb49ec700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fcc9406b020 0x7fcc9406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:54.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.604+0000 7fccb49ec700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fcc9406b020 0x7fcc9406d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcca4000f60 tx=0x7fcca4009040 comp rx=0 tx=0).ready entity=mgr.9405 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:54.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.608+0000 7fccad7fa700 1 -- 172.21.15.181:0/1475361713 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcc9c054410 con 0x7fccb00d01d0 2023-10-16T11:00:54.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:54.861+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fccb00d2eb0 con 0x7fccb00d01d0 2023-10-16T11:00:55.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.766+0000 7fccad7fa700 1 -- 172.21.15.181:0/1475361713 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v100) v1 ==== 92+0+0 (secure 0 0 0) 0x7fcc9c057a30 con 0x7fccb00d01d0 2023-10-16T11:00:55.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fcc9406b020 msgr2=0x7fcc9406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:55.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fcc9406b020 0x7fcc9406d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcca4000f60 tx=0x7fcca4009040 comp rx=0 tx=0).stop 2023-10-16T11:00:55.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb00d01d0 msgr2=0x7fccb01f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:55.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb00d01d0 0x7fccb01f6c60 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b4a0 tx=0x7fcc9c00b860 comp rx=0 tx=0).stop 2023-10-16T11:00:55.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 shutdown_connections 2023-10-16T11:00:55.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7fcc9406b020 0x7fcc9406d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:55.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccb00ce2d0 0x7fccb0152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:55.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb015ea40 0x7fccb01f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:55.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 --2- 172.21.15.181:0/1475361713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb00d01d0 0x7fccb01f6c60 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:55.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.770+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 >> 172.21.15.181:0/1475361713 conn(0x7fccb009cb60 msgr2=0x7fccb015dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:55.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.771+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 shutdown_connections 2023-10-16T11:00:55.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.772+0000 7fccb6c50700 1 -- 172.21.15.181:0/1475361713 wait complete. 2023-10-16T11:00:55.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:00:55.915 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:55.914+0000 7fa52585c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:00:55.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.916+0000 7f701eae4700 1 Processor -- start 2023-10-16T11:00:55.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.916+0000 7f701eae4700 1 -- start start 2023-10-16T11:00:55.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.916+0000 7f701eae4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f701806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:55.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.916+0000 7f701eae4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701815dfb0 con 0x7f701815dbd0 2023-10-16T11:00:55.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.916+0000 7f701eae4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70180cd3a0 con 0x7f70180cdd80 2023-10-16T11:00:55.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.916+0000 7f701eae4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180cd830 con 0x7f70180ce200 2023-10-16T11:00:55.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f701c880700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f701815dbd0 0x7f70180d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33820/0 (socket says 172.21.15.181:33820) 2023-10-16T11:00:55.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f701c880700 1 -- 172.21.15.181:0/2366518512 learned_addr learned my addr 172.21.15.181:0/2366518512 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:55.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f701d081700 1 --2- 172.21.15.181:0/2366518512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f701806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:55.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f70177fe700 1 -- 172.21.15.181:0/2366518512 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856034365 0 0) 0x7f700c001110 con 0x7f701815dbd0 2023-10-16T11:00:55.920 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:55.917+0000 7fe8be8e2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:00:55.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f70177fe700 1 -- 172.21.15.181:0/2366518512 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70080035b0 con 0x7f701815dbd0 2023-10-16T11:00:55.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f70177fe700 1 -- 172.21.15.181:0/2366518512 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213418393 0 0) 0x7f70180cd3a0 con 0x7f70180cdd80 2023-10-16T11:00:55.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f70177fe700 1 -- 172.21.15.181:0/2366518512 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f700c001110 con 0x7f70180cdd80 2023-10-16T11:00:55.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f701d081700 1 -- 172.21.15.181:0/2366518512 >> v1:172.21.15.190:6789/0 conn(0x7f70180cdd80 legacy=0x7f701806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:55.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f701d081700 1 -- 172.21.15.181:0/2366518512 >> v1:172.21.15.181:6789/0 conn(0x7f701815dbd0 legacy=0x7f70180d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:55.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f701d081700 1 -- 172.21.15.181:0/2366518512 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180d1fc0 con 0x7f70180ce200 2023-10-16T11:00:55.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f701d081700 1 --2- 172.21.15.181:0/2366518512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f701806be00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7010009dc0 tx=0x7f701000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547ec70a4c8145ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:55.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f70177fe700 1 -- 172.21.15.181:0/2366518512 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701000e070 con 0x7f70180ce200 2023-10-16T11:00:55.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f70177fe700 1 -- 172.21.15.181:0/2366518512 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701000bb20 con 0x7f70180ce200 2023-10-16T11:00:55.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.917+0000 7f70177fe700 1 -- 172.21.15.181:0/2366518512 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010012430 con 0x7f70180ce200 2023-10-16T11:00:55.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.918+0000 7f701eae4700 1 -- 172.21.15.181:0/2366518512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 msgr2=0x7f701806be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:55.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.918+0000 7f701eae4700 1 --2- 172.21.15.181:0/2366518512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f701806be00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7010009dc0 tx=0x7f701000b330 comp rx=0 tx=0).stop 2023-10-16T11:00:55.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.918+0000 7f701eae4700 1 -- 172.21.15.181:0/2366518512 shutdown_connections 2023-10-16T11:00:55.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.918+0000 7f701eae4700 1 --2- 172.21.15.181:0/2366518512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f701806be00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:55.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.918+0000 7f701eae4700 1 -- 172.21.15.181:0/2366518512 >> 172.21.15.181:0/2366518512 conn(0x7f701809a930 msgr2=0x7f70180d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:55.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.918+0000 7f701eae4700 1 -- 172.21.15.181:0/2366518512 shutdown_connections 2023-10-16T11:00:55.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.918+0000 7f701eae4700 1 -- 172.21.15.181:0/2366518512 wait complete. 2023-10-16T11:00:55.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.919+0000 7f701eae4700 1 Processor -- start 2023-10-16T11:00:55.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.919+0000 7f701eae4700 1 -- start start 2023-10-16T11:00:55.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701eae4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70180cdd80 0x7f701815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:55.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701eae4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f70180b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:55.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701eae4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f701815dbd0 0x7f701806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:55.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701eae4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70181f3080 con 0x7f701815dbd0 2023-10-16T11:00:55.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701eae4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70180b5cf0 con 0x7f70180cdd80 2023-10-16T11:00:55.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701eae4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701815e150 con 0x7f70180ce200 2023-10-16T11:00:55.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701d081700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f701815dbd0 0x7f701806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:55.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701c880700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70180cdd80 0x7f701815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f7017fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f70180b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701d081700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f701815dbd0 0x7f701806ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50982/0 (socket says 172.21.15.181:50982) 2023-10-16T11:00:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701c880700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70180cdd80 0x7f701815a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:56028/0 (socket says 172.21.15.181:56028) 2023-10-16T11:00:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f701d081700 1 -- 172.21.15.181:0/395603822 learned_addr learned my addr 172.21.15.181:0/395603822 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:55.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.920+0000 7f7017fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f70180b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:48006/0 (socket says 172.21.15.181:48006) 2023-10-16T11:00:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701d081700 1 -- 172.21.15.181:0/395603822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 msgr2=0x7f70180b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701d081700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f70180b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701d081700 1 -- 172.21.15.181:0/395603822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70180cdd80 msgr2=0x7f701815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701d081700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70180cdd80 0x7f701815a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:55.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701d081700 1 -- 172.21.15.181:0/395603822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7010009a70 con 0x7f701815dbd0 2023-10-16T11:00:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701d081700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f701815dbd0 0x7f701806ed60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f701000b370 tx=0x7f7010004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f70157fa700 1 -- 172.21.15.181:0/395603822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701001b810 con 0x7f701815dbd0 2023-10-16T11:00:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f70157fa700 1 -- 172.21.15.181:0/395603822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7010020070 con 0x7f701815dbd0 2023-10-16T11:00:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70180b5f70 con 0x7f701815dbd0 2023-10-16T11:00:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f70157fa700 1 -- 172.21.15.181:0/395603822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701001bbb0 con 0x7f701815dbd0 2023-10-16T11:00:55.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.921+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70180b6530 con 0x7f701815dbd0 2023-10-16T11:00:55.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.922+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7018067d20 con 0x7f701815dbd0 2023-10-16T11:00:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.923+0000 7f70157fa700 1 -- 172.21.15.181:0/395603822 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 100) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f701001bd10 con 0x7f701815dbd0 2023-10-16T11:00:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.923+0000 7f70157fa700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 0x7f6ffc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.923+0000 7f701c880700 1 -- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 msgr2=0x7f6ffc06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.923+0000 7f70157fa700 1 -- 172.21.15.181:0/395603822 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f7010022070 con 0x7f701815dbd0 2023-10-16T11:00:55.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.923+0000 7f701c880700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 0x7f6ffc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:55.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:55.932+0000 7f70157fa700 1 -- 172.21.15.181:0/395603822 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7010056610 con 0x7f701815dbd0 2023-10-16T11:00:55.981 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:55.980+0000 7fa52585c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:00:55.991 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:55.990+0000 7fe8be8e2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:00:56.053 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:56.052+0000 7fa52585c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:00:56.065 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:56.064+0000 7fe8be8e2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:00:56.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.124+0000 7f701c880700 1 -- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 msgr2=0x7f6ffc06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:56.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.124+0000 7f701c880700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 0x7f6ffc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:56.131 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:56.130+0000 7fa52585c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:00:56.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.132+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f70180cebb0 con 0x7f701815dbd0 2023-10-16T11:00:56.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.135+0000 7f70157fa700 1 -- 172.21.15.181:0/395603822 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7010059c30 con 0x7f701815dbd0 2023-10-16T11:00:56.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.137+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 msgr2=0x7f6ffc06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.137+0000 7f701eae4700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 0x7f6ffc06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.137+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f701815dbd0 msgr2=0x7f701806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f701815dbd0 0x7f701806ed60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f701000b370 tx=0x7f7010004ad0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 shutdown_connections 2023-10-16T11:00:56.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f6ffc06b070 0x7f6ffc06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f701815dbd0 0x7f701806ed60 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70180cdd80 0x7f701815a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 --2- 172.21.15.181:0/395603822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70180ce200 0x7f70180b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 >> 172.21.15.181:0/395603822 conn(0x7f701809a930 msgr2=0x7f701815bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:56.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 shutdown_connections 2023-10-16T11:00:56.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.138+0000 7f701eae4700 1 -- 172.21.15.181:0/395603822 wait complete. 2023-10-16T11:00:56.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:56.146+0000 7fe8be8e2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:00:56.152 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:00:56.272 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:56.271+0000 7fa52585c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:00:56.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.279+0000 7f72ce0c3700 1 Processor -- start 2023-10-16T11:00:56.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.279+0000 7f72ce0c3700 1 -- start start 2023-10-16T11:00:56.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72ce0c3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f72c815da70 0x7f72c80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72ce0c3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72c80ce6a0 0x7f72c806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72ce0c3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c80cef00 con 0x7f72c8072760 2023-10-16T11:00:56.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72ce0c3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c8072b40 con 0x7f72c80ce6a0 2023-10-16T11:00:56.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72ce0c3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c80cec20 con 0x7f72c815da70 2023-10-16T11:00:56.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72c80ce6a0 0x7f72c806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72c80ce6a0 0x7f72c806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51014/0 (socket says 172.21.15.181:51014) 2023-10-16T11:00:56.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 -- 172.21.15.181:0/1009799395 learned_addr learned my addr 172.21.15.181:0/1009799395 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:56.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c77fe700 1 --2- 172.21.15.181:0/1009799395 >> v2:172.21.15.190:3300/0 conn(0x7f72c815da70 0x7f72c80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 -- 172.21.15.181:0/1009799395 >> v2:172.21.15.190:3300/0 conn(0x7f72c815da70 msgr2=0x7f72c80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:56.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 --2- 172.21.15.181:0/1009799395 >> v2:172.21.15.190:3300/0 conn(0x7f72c815da70 0x7f72c80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 -- 172.21.15.181:0/1009799395 >> v1:172.21.15.190:6789/0 conn(0x7f72c8072760 legacy=0x7f72c81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 -- 172.21.15.181:0/1009799395 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c8071c80 con 0x7f72c80ce6a0 2023-10-16T11:00:56.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.280+0000 7f72c6ffd700 1 --2- 172.21.15.181:0/1009799395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c80ce6a0 0x7f72c806a8a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f72c80675e0 tx=0x7f72bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c295af5cd81923e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:56.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72c67fc700 1 -- 172.21.15.181:0/1009799395 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc011830 con 0x7f72c80ce6a0 2023-10-16T11:00:56.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72c67fc700 1 -- 172.21.15.181:0/1009799395 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc011990 con 0x7f72c80ce6a0 2023-10-16T11:00:56.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72c67fc700 1 -- 172.21.15.181:0/1009799395 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc011d20 con 0x7f72c80ce6a0 2023-10-16T11:00:56.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 -- 172.21.15.181:0/1009799395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c80ce6a0 msgr2=0x7f72c806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/1009799395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c80ce6a0 0x7f72c806a8a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f72c80675e0 tx=0x7f72bc009bd0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 -- 172.21.15.181:0/1009799395 shutdown_connections 2023-10-16T11:00:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/1009799395 >> v2:172.21.15.190:3300/0 conn(0x7f72c815da70 0x7f72c80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/1009799395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c80ce6a0 0x7f72c806a8a0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 -- 172.21.15.181:0/1009799395 >> 172.21.15.181:0/1009799395 conn(0x7f72c80cd010 msgr2=0x7f72c80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 -- 172.21.15.181:0/1009799395 shutdown_connections 2023-10-16T11:00:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.281+0000 7f72ce0c3700 1 -- 172.21.15.181:0/1009799395 wait complete. 2023-10-16T11:00:56.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.283+0000 7f72ce0c3700 1 Processor -- start 2023-10-16T11:00:56.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.283+0000 7f72ce0c3700 1 -- start start 2023-10-16T11:00:56.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72ce0c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c8072760 0x7f72c80c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72ce0c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72c80ce6a0 0x7f72c80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72ce0c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c815da70 0x7f72c80acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72ce0c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80b57a0 con 0x7f72c8072760 2023-10-16T11:00:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72ce0c3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80b3350 con 0x7f72c815da70 2023-10-16T11:00:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72ce0c3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80b35f0 con 0x7f72c80ce6a0 2023-10-16T11:00:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c8072760 0x7f72c80c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c8072760 0x7f72c80c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51018/0 (socket says 172.21.15.181:51018) 2023-10-16T11:00:56.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72c77fe700 1 -- 172.21.15.181:0/3506518920 learned_addr learned my addr 172.21.15.181:0/3506518920 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:56.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72c6ffd700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72c80ce6a0 0x7f72c80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72c7fff700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c815da70 0x7f72c80acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.284+0000 7f72c77fe700 1 -- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72c80ce6a0 msgr2=0x7f72c80b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72c77fe700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72c80ce6a0 0x7f72c80b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72c77fe700 1 -- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c815da70 msgr2=0x7f72c80acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72c77fe700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c815da70 0x7f72c80acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72c77fe700 1 -- 172.21.15.181:0/3506518920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc009550 con 0x7f72c8072760 2023-10-16T11:00:56.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72c77fe700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c8072760 0x7f72c80c4e00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e9b0 tx=0x7f72b800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:56.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72affff700 1 -- 172.21.15.181:0/3506518920 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8009710 con 0x7f72c8072760 2023-10-16T11:00:56.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72affff700 1 -- 172.21.15.181:0/3506518920 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800ee50 con 0x7f72c8072760 2023-10-16T11:00:56.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c80b38d0 con 0x7f72c8072760 2023-10-16T11:00:56.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72affff700 1 -- 172.21.15.181:0/3506518920 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800ca50 con 0x7f72c8072760 2023-10-16T11:00:56.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.285+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c80b5490 con 0x7f72c8072760 2023-10-16T11:00:56.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.286+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c806f6c0 con 0x7f72c8072760 2023-10-16T11:00:56.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.287+0000 7f72affff700 1 -- 172.21.15.181:0/3506518920 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 100) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f72b800cbb0 con 0x7f72c8072760 2023-10-16T11:00:56.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.288+0000 7f72affff700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 0x7f72b406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.288+0000 7f72c6ffd700 1 -- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 msgr2=0x7f72b406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:56.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.288+0000 7f72affff700 1 -- 172.21.15.181:0/3506518920 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f72b8089aa0 con 0x7f72c8072760 2023-10-16T11:00:56.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.288+0000 7f72c6ffd700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 0x7f72b406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.296+0000 7f72affff700 1 -- 172.21.15.181:0/3506518920 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f72b8054730 con 0x7f72c8072760 2023-10-16T11:00:56.306 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:56.304+0000 7fe8be8e2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:00:56.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.489+0000 7f72c6ffd700 1 -- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 msgr2=0x7f72b406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:56.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.489+0000 7f72c6ffd700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 0x7f72b406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:56.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.502+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f72c80b2e50 con 0x7f72c8072760 2023-10-16T11:00:56.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.505+0000 7f72affff700 1 -- 172.21.15.181:0/3506518920 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f72b800ce60 con 0x7f72c8072760 2023-10-16T11:00:56.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 msgr2=0x7f72b406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:00:56.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 0x7f72b406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c8072760 msgr2=0x7f72c80c4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c8072760 0x7f72c80c4e00 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f72b800e9b0 tx=0x7f72b800ecc0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 shutdown_connections 2023-10-16T11:00:56.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7f72b406b070 0x7f72b406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c815da70 0x7f72c80acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c8072760 0x7f72c80c4e00 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 --2- 172.21.15.181:0/3506518920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72c80ce6a0 0x7f72c80b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 >> 172.21.15.181:0/3506518920 conn(0x7f72c80cd010 msgr2=0x7f72c80cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:56.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 shutdown_connections 2023-10-16T11:00:56.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.508+0000 7f72ce0c3700 1 -- 172.21.15.181:0/3506518920 wait complete. 2023-10-16T11:00:56.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:00:56.522 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:00:56.542 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-10-16T11:00:56.595 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:56.594+0000 7fa52585c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:00:56.650 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:56.649+0000 7fe8be8e2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:00:56.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.710+0000 7ff9611f9700 1 Processor -- start 2023-10-16T11:00:56.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff9611f9700 1 -- start start 2023-10-16T11:00:56.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff9611f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff95c0b2010 0x7ff95c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff9611f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff9611f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95c0ce1f0 con 0x7ff95c071be0 2023-10-16T11:00:56.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff9611f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c072090 con 0x7ff95c0b2010 2023-10-16T11:00:56.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff9611f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0ccda0 con 0x7ff95c0ce680 2023-10-16T11:00:56.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff95a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff95c071be0 0x7ff95c0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:33892/0 (socket says 172.21.15.181:33892) 2023-10-16T11:00:56.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.711+0000 7ff95a59c700 1 -- 172.21.15.181:0/3277820019 learned_addr learned my addr 172.21.15.181:0/3277820019 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:00:56.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/3277820019 >> v2:172.21.15.190:3300/0 conn(0x7ff95c0b2010 0x7ff95c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95b59e700 1 --2- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff959d9b700 1 -- 172.21.15.181:0/3277820019 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840152295 0 0) 0x7ff95c0ce1f0 con 0x7ff95c071be0 2023-10-16T11:00:56.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff959d9b700 1 -- 172.21.15.181:0/3277820019 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9440035b0 con 0x7ff95c071be0 2023-10-16T11:00:56.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95ad9d700 1 -- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 msgr2=0x7ff95c0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:00:56.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c0ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95ad9d700 1 -- 172.21.15.181:0/3277820019 >> v1:172.21.15.181:6789/0 conn(0x7ff95c071be0 legacy=0x7ff95c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95ad9d700 1 -- 172.21.15.181:0/3277820019 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0727e0 con 0x7ff95c0b2010 2023-10-16T11:00:56.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95b59e700 1 --2- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:56.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0b2010 0x7ff95c0cc6e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009840 tx=0x7ff94c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc72c353335fc5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:56.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff959d9b700 1 -- 172.21.15.181:0/3277820019 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c011830 con 0x7ff95c0b2010 2023-10-16T11:00:56.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff959d9b700 1 -- 172.21.15.181:0/3277820019 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c011990 con 0x7ff95c0b2010 2023-10-16T11:00:56.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.712+0000 7ff959d9b700 1 -- 172.21.15.181:0/3277820019 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c011d20 con 0x7ff95c0b2010 2023-10-16T11:00:56.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 -- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0b2010 msgr2=0x7ff95c0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 --2- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0b2010 0x7ff95c0cc6e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009840 tx=0x7ff94c009b50 comp rx=0 tx=0).stop 2023-10-16T11:00:56.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 -- 172.21.15.181:0/3277820019 shutdown_connections 2023-10-16T11:00:56.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 --2- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 --2- 172.21.15.181:0/3277820019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0b2010 0x7ff95c0cc6e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 -- 172.21.15.181:0/3277820019 >> 172.21.15.181:0/3277820019 conn(0x7ff95c0cd010 msgr2=0x7ff95c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:00:56.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 -- 172.21.15.181:0/3277820019 shutdown_connections 2023-10-16T11:00:56.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.713+0000 7ff9611f9700 1 -- 172.21.15.181:0/3277820019 wait complete. 2023-10-16T11:00:56.719 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:56.714+0000 7fa52585c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:00:56.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.714+0000 7ff9611f9700 1 Processor -- start 2023-10-16T11:00:56.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff9611f9700 1 -- start start 2023-10-16T11:00:56.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff9611f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff95c071be0 0x7ff95c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff9611f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff9611f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c15ccc0 0x7ff95c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff9611f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c15d2d0 con 0x7ff95c071be0 2023-10-16T11:00:56.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff9611f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c15bbe0 con 0x7ff95c15ccc0 2023-10-16T11:00:56.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff9611f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0b3230 con 0x7ff95c0ce680 2023-10-16T11:00:56.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff95ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff95c071be0 0x7ff95c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff95a59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff95b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c15ccc0 0x7ff95c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:00:56.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff95ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff95c071be0 0x7ff95c158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51050/0 (socket says 172.21.15.181:51050) 2023-10-16T11:00:56.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff95ad9d700 1 -- 172.21.15.181:0/1141180649 learned_addr learned my addr 172.21.15.181:0/1141180649 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:00:56.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.715+0000 7ff95b59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c15ccc0 0x7ff95c156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:56104/0 (socket says 172.21.15.181:56104) 2023-10-16T11:00:56.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95a59c700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c15ccc0 msgr2=0x7ff95c156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95a59c700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c15ccc0 0x7ff95c156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95a59c700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff95c071be0 msgr2=0x7ff95c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:00:56.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95a59c700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff95c071be0 0x7ff95c158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:00:56.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95a59c700 1 -- 172.21.15.181:0/1141180649 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff954009a70 con 0x7ff95c0ce680 2023-10-16T11:00:56.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95b59e700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c15ccc0 0x7ff95c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:56.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff95c071be0 0x7ff95c158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:00:56.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff95a59c700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c15c780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff95000c4c0 tx=0x7ff95000c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:00:56.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950005d70 con 0x7ff95c0ce680 2023-10-16T11:00:56.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9500070d0 con 0x7ff95c0ce680 2023-10-16T11:00:56.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c0094a0 con 0x7ff95c0ce680 2023-10-16T11:00:56.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950007440 con 0x7ff95c0ce680 2023-10-16T11:00:56.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.716+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c0b37f0 con 0x7ff95c0ce680 2023-10-16T11:00:56.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.717+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c0b6510 con 0x7ff95c0ce680 2023-10-16T11:00:56.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.718+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 100) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff9500075a0 con 0x7ff95c0ce680 2023-10-16T11:00:56.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.718+0000 7ff9437fe700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 0x7ff94806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:00:56.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.718+0000 7ff95ad9d700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 msgr2=0x7ff94806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:56.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.719+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6205+0+0 (secure 0 0 0) 0x7ff950088db0 con 0x7ff95c0ce680 2023-10-16T11:00:56.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.719+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 0x7ff94806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:00:56.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.723+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff950053ac0 con 0x7ff95c0ce680 2023-10-16T11:00:56.772 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:56.771+0000 7fe8be8e2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:00:56.789 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:56.788+0000 7fa52585c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:00:56.848 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:56.848+0000 7fe8be8e2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:00:56.868 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:56.867+0000 7fa52585c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:00:56.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.920+0000 7ff95ad9d700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 msgr2=0x7ff94806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:56.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.920+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 0x7ff94806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:00:56.930 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:56.930+0000 7fe8be8e2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:00:56.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:56.951+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 --> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7ff95c072da0 con 0x7ff94806b070 2023-10-16T11:00:57.236 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:57.235+0000 7fa52585c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:00:57.301 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:57.301+0000 7fe8be8e2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:00:57.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:57.321+0000 7ff95ad9d700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 msgr2=0x7ff94806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:57.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:57.321+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 0x7ff94806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:00:57.801 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:57.800+0000 7fa52585c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:00:57.870 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:57.869+0000 7fa52585c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:00:57.873 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:57.872+0000 7fe8be8e2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:00:57.943 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:57.943+0000 7fe8be8e2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:00:58.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:58.122+0000 7ff95ad9d700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 msgr2=0x7ff94806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:58.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:58.122+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 0x7ff94806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:00:58.497 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:58.496+0000 7fa52585c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:58.574 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:58.572+0000 7fa52585c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:00:58.576 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:58.575+0000 7fe8be8e2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:58.653 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:58.653+0000 7fe8be8e2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:00:58.717 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:58.715+0000 7fa52585c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:58.800 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:58.799+0000 7fe8be8e2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:00:58.943 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:58.941+0000 7fa52585c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:00:59.015 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:59.014+0000 7fa52585c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:00:59.029 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:59.029+0000 7fe8be8e2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:00:59.104 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:59.103+0000 7fe8be8e2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:00:59.278 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:59.277+0000 7fa52585c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:00:59.370 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:59.370+0000 7fe8be8e2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:00:59.497 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:59.496+0000 7fa52585c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:00:59.594 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:59.593+0000 7fe8be8e2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:00:59.647 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:59.646+0000 7fa52585c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:00:59.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:59.723+0000 7ff95ad9d700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 msgr2=0x7ff94806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/551790976 2023-10-16T11:00:59.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:00:59.723+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 0x7ff94806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:00:59.745 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:59.745+0000 7fe8be8e2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:00:59.761 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:59.760+0000 7fa52585c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:00:59.862 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:00:59.862+0000 7fe8be8e2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:00:59.934 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:00:59.933+0000 7fa52585c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:01:00.022 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:00.021+0000 7fe8be8e2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:01:00.195 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:00.194+0000 7fa52585c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:00.285 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:00.285+0000 7fe8be8e2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:00.354 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:00.353+0000 7fa52585c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:01:00.449 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:00.448+0000 7fe8be8e2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:01:00.875 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:00.874+0000 7fa52585c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:01:00.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:00.944+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mgrmap(e 101) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff950087040 con 0x7ff95c0ce680 2023-10-16T11:01:00.980 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:00.979+0000 7fe8be8e2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:01:01.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:01.000+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 8 ==== mgrmap(e 102) v1 ==== 88957+0+0 (secure 0 0 0) 0x7ff950053e90 con 0x7ff95c0ce680 2023-10-16T11:01:01.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:01.001+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 msgr2=0x7ff94806d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:01.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:01.001+0000 7ff9437fe700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/551790976,v1:172.21.15.181:6833/551790976] conn(0x7ff94806b070 0x7ff94806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:01.147 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8348c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.147 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8348c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.147 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8348c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.147 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8348c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8348c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8318bd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8318bd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8318bd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8318bd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:01.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:01.146+0000 7fe8318bd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:02.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.003+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mon.2 v2:172.21.15.190:3301/0 9 ==== mgrmap(e 103) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff950054760 con 0x7ff95c0ce680 2023-10-16T11:01:02.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.004+0000 7ff9437fe700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7ff9480279d0 0x7ff948027db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.004+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 --> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7ff948071070 con 0x7ff9480279d0 2023-10-16T11:01:02.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.004+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7ff9480279d0 0x7ff948027db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.007+0000 7ff95ad9d700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7ff9480279d0 0x7ff948027db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009b90 tx=0x7ff94c008030 comp rx=0 tx=0).ready entity=mgr.9849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:02.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.067+0000 7ff9437fe700 1 -- 172.21.15.181:0/1141180649 <== mgr.9849 v2:172.21.15.181:6832/4215027768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff948071070 con 0x7ff9480279d0 2023-10-16T11:01:02.068 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1 2023-10-16T11:01:02.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7ff9480279d0 msgr2=0x7ff948027db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7ff9480279d0 0x7ff948027db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009b90 tx=0x7ff94c008030 comp rx=0 tx=0).stop 2023-10-16T11:01:02.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 msgr2=0x7ff95c15c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 --2- 172.21.15.181:0/1141180649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff95c0ce680 0x7ff95c15c780 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff95000c4c0 tx=0x7ff95000c7d0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff95b59e700 1 -- 172.21.15.181:0/1141180649 reap_dead start 2023-10-16T11:01:02.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 shutdown_connections 2023-10-16T11:01:02.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 >> 172.21.15.181:0/1141180649 conn(0x7ff95c0cd010 msgr2=0x7ff95c0cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:02.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 shutdown_connections 2023-10-16T11:01:02.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.069+0000 7ff9611f9700 1 -- 172.21.15.181:0/1141180649 wait complete. 2023-10-16T11:01:02.083 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T11:01:02.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T11:01:02.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.208+0000 7f2cba5a2700 1 Processor -- start 2023-10-16T11:01:02.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.208+0000 7f2cba5a2700 1 -- start start 2023-10-16T11:01:02.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cba5a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cba5a2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2cb415db30 0x7f2cb40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cba5a2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb415ea60 con 0x7f2cb40cbed0 2023-10-16T11:01:02.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cba5a2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb40cb390 con 0x7f2cb415db30 2023-10-16T11:01:02.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cba5a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415dfb0 con 0x7f2cb4159e50 2023-10-16T11:01:02.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2cb415db30 0x7f2cb40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2cb415db30 0x7f2cb40cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51222/0 (socket says 172.21.15.181:51222) 2023-10-16T11:01:02.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 -- 172.21.15.181:0/755298723 learned_addr learned my addr 172.21.15.181:0/755298723 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:02.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb3fff700 1 --2- 172.21.15.181:0/755298723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 -- 172.21.15.181:0/755298723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 msgr2=0x7f2cb4071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:02.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 --2- 172.21.15.181:0/755298723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 -- 172.21.15.181:0/755298723 >> v1:172.21.15.190:6789/0 conn(0x7f2cb40cbed0 legacy=0x7f2cb40b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 -- 172.21.15.181:0/755298723 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb415e1c0 con 0x7f2cb415db30 2023-10-16T11:01:02.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb37fe700 1 --2- 172.21.15.181:0/755298723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb415db30 0x7f2cb40cae50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400bb90 tx=0x7f2ca400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854b8a1eaf85dc96 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:02.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.209+0000 7f2cb2ffd700 1 -- 172.21.15.181:0/755298723 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400dd30 con 0x7f2cb415db30 2023-10-16T11:01:02.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cb2ffd700 1 -- 172.21.15.181:0/755298723 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca400de90 con 0x7f2cb415db30 2023-10-16T11:01:02.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cb2ffd700 1 -- 172.21.15.181:0/755298723 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4010690 con 0x7f2cb415db30 2023-10-16T11:01:02.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 -- 172.21.15.181:0/755298723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb415db30 msgr2=0x7f2cb40cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/755298723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb415db30 0x7f2cb40cae50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400bb90 tx=0x7f2ca400bea0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 -- 172.21.15.181:0/755298723 shutdown_connections 2023-10-16T11:01:02.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/755298723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/755298723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb415db30 0x7f2cb40cae50 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 -- 172.21.15.181:0/755298723 >> 172.21.15.181:0/755298723 conn(0x7f2cb406aed0 msgr2=0x7f2cb406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:02.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 -- 172.21.15.181:0/755298723 shutdown_connections 2023-10-16T11:01:02.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.210+0000 7f2cba5a2700 1 -- 172.21.15.181:0/755298723 wait complete. 2023-10-16T11:01:02.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.212+0000 7f2cba5a2700 1 Processor -- start 2023-10-16T11:01:02.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.212+0000 7f2cba5a2700 1 -- start start 2023-10-16T11:01:02.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.212+0000 7f2cba5a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb40cbed0 0x7f2cb40c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cba5a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cba5a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb415db30 0x7f2cb4155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cba5a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb41477c0 con 0x7f2cb40cbed0 2023-10-16T11:01:02.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cba5a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40c4f80 con 0x7f2cb415db30 2023-10-16T11:01:02.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cba5a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40c5260 con 0x7f2cb4159e50 2023-10-16T11:01:02.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb40cbed0 0x7f2cb40c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb40cbed0 0x7f2cb40c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51228/0 (socket says 172.21.15.181:51228) 2023-10-16T11:01:02.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 -- 172.21.15.181:0/2508015115 learned_addr learned my addr 172.21.15.181:0/2508015115 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:02.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb37fe700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb8b3f700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb415db30 0x7f2cb4155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 -- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 msgr2=0x7f2cb4155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 -- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb415db30 msgr2=0x7f2cb4155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb415db30 0x7f2cb4155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 -- 172.21.15.181:0/2508015115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca400b7f0 con 0x7f2cb40cbed0 2023-10-16T11:01:02.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb8b3f700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb415db30 0x7f2cb4155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:02.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb3fff700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb40cbed0 0x7f2cb40c5d40 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9b0 tx=0x7f2ca800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:02.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb0ff9700 1 -- 172.21.15.181:0/2508015115 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8009710 con 0x7f2cb40cbed0 2023-10-16T11:01:02.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb0ff9700 1 -- 172.21.15.181:0/2508015115 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca800c850 con 0x7f2cb40cbed0 2023-10-16T11:01:02.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb40c5540 con 0x7f2cb40cbed0 2023-10-16T11:01:02.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cb0ff9700 1 -- 172.21.15.181:0/2508015115 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800cbc0 con 0x7f2cb40cbed0 2023-10-16T11:01:02.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.213+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb41f2540 con 0x7f2cb40cbed0 2023-10-16T11:01:02.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.215+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb404ebb0 con 0x7f2cb40cbed0 2023-10-16T11:01:02.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.215+0000 7f2cb0ff9700 1 -- 172.21.15.181:0/2508015115 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 103) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2ca800cd20 con 0x7f2cb40cbed0 2023-10-16T11:01:02.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.215+0000 7f2cb0ff9700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f2ca006ada0 0x7f2ca006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.215+0000 7f2cb0ff9700 1 -- 172.21.15.181:0/2508015115 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f2ca80895f0 con 0x7f2cb40cbed0 2023-10-16T11:01:02.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.215+0000 7f2cb37fe700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f2ca006ada0 0x7f2ca006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.216+0000 7f2cb37fe700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f2ca006ada0 0x7f2ca006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400bee0 tx=0x7f2ca4013000 comp rx=0 tx=0).ready entity=mgr.9849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:02.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.220+0000 7f2cb0ff9700 1 -- 172.21.15.181:0/2508015115 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2ca8054690 con 0x7f2cb40cbed0 2023-10-16T11:01:02.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.437+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 --> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cb40c50f0 con 0x7f2ca006ada0 2023-10-16T11:01:02.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.502+0000 7f2cb0ff9700 1 -- 172.21.15.181:0/2508015115 <== mgr.9849 v2:172.21.15.181:6832/4215027768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2cb40c50f0 con 0x7f2ca006ada0 2023-10-16T11:01:02.503 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path snap_test_dir1 2023-10-16T11:01:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.504+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f2ca006ada0 msgr2=0x7f2ca006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.504+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f2ca006ada0 0x7f2ca006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400bee0 tx=0x7f2ca4013000 comp rx=0 tx=0).stop 2023-10-16T11:01:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.504+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb40cbed0 msgr2=0x7f2cb40c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.504+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb40cbed0 0x7f2cb40c5d40 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9b0 tx=0x7f2ca800ecc0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 shutdown_connections 2023-10-16T11:01:02.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f2ca006ada0 0x7f2ca006d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb415db30 0x7f2cb4155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb40cbed0 0x7f2cb40c5d40 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 --2- 172.21.15.181:0/2508015115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb4159e50 0x7f2cb4155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 >> 172.21.15.181:0/2508015115 conn(0x7f2cb406aed0 msgr2=0x7f2cb40afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:02.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 shutdown_connections 2023-10-16T11:01:02.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.505+0000 7f2cba5a2700 1 -- 172.21.15.181:0/2508015115 wait complete. 2023-10-16T11:01:02.517 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T11:01:02.518 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-16T11:01:02.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.640+0000 7f5645ae7700 1 Processor -- start 2023-10-16T11:01:02.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.640+0000 7f5645ae7700 1 -- start start 2023-10-16T11:01:02.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f5645ae7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400cc6e0 0x7f5640071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f5645ae7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56400ccb60 con 0x7f56400ce680 2023-10-16T11:01:02.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f5645ae7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56400ce1f0 con 0x7f56400b2010 2023-10-16T11:01:02.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f5645ae7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5640072120 con 0x7f56400cc6e0 2023-10-16T11:01:02.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f56400ce680 0x7f56400ceb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34086/0 (socket says 172.21.15.181:34086) 2023-10-16T11:01:02.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563ffff700 1 -- 172.21.15.181:0/2669520398 learned_addr learned my addr 172.21.15.181:0/2669520398 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563effd700 1 --2- 172.21.15.181:0/2669520398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400cc6e0 0x7f5640071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592984862 0 0) 0x7f5638001490 con 0x7f56400ce680 2023-10-16T11:01:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f562c0035b0 con 0x7f56400ce680 2023-10-16T11:01:02.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839085028 0 0) 0x7f56400ce1f0 con 0x7f56400b2010 2023-10-16T11:01:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5638001490 con 0x7f56400b2010 2023-10-16T11:01:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563effd700 1 -- 172.21.15.181:0/2669520398 >> v1:172.21.15.190:6789/0 conn(0x7f56400b2010 legacy=0x7f56400b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205842445 0 0) 0x7f562c0035b0 con 0x7f56400ce680 2023-10-16T11:01:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563effd700 1 -- 172.21.15.181:0/2669520398 >> v1:172.21.15.181:6789/0 conn(0x7f56400ce680 legacy=0x7f56400ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.641+0000 7f563effd700 1 -- 172.21.15.181:0/2669520398 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400ccda0 con 0x7f56400cc6e0 2023-10-16T11:01:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f563effd700 1 --2- 172.21.15.181:0/2669520398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400cc6e0 0x7f5640071be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5630009870 tx=0x7f5630009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2053722d0b8cac77 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638001b80 con 0x7f56400cc6e0 2023-10-16T11:01:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630004030 con 0x7f56400cc6e0 2023-10-16T11:01:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f563e7fc700 1 -- 172.21.15.181:0/2669520398 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300043c0 con 0x7f56400cc6e0 2023-10-16T11:01:02.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f5645ae7700 1 -- 172.21.15.181:0/2669520398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400cc6e0 msgr2=0x7f5640071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f5645ae7700 1 --2- 172.21.15.181:0/2669520398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400cc6e0 0x7f5640071be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5630009870 tx=0x7f5630009b80 comp rx=0 tx=0).stop 2023-10-16T11:01:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f5645ae7700 1 -- 172.21.15.181:0/2669520398 shutdown_connections 2023-10-16T11:01:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f5645ae7700 1 --2- 172.21.15.181:0/2669520398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400cc6e0 0x7f5640071be0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f5645ae7700 1 -- 172.21.15.181:0/2669520398 >> 172.21.15.181:0/2669520398 conn(0x7f56400cd010 msgr2=0x7f56400cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:02.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f5645ae7700 1 -- 172.21.15.181:0/2669520398 shutdown_connections 2023-10-16T11:01:02.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.642+0000 7f5645ae7700 1 -- 172.21.15.181:0/2669520398 wait complete. 2023-10-16T11:01:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.644+0000 7f5645ae7700 1 Processor -- start 2023-10-16T11:01:02.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.644+0000 7f5645ae7700 1 -- start start 2023-10-16T11:01:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.644+0000 7f5645ae7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400b2010 0x7f56400c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.644+0000 7f5645ae7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56400cc6e0 0x7f56401ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5645ae7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400ce680 0x7f56400c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5645ae7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564015dbe0 con 0x7f56400b2010 2023-10-16T11:01:02.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5645ae7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564015d340 con 0x7f56400cc6e0 2023-10-16T11:01:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5645ae7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564015d620 con 0x7f56400ce680 2023-10-16T11:01:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400b2010 0x7f56400c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400b2010 0x7f56400c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51242/0 (socket says 172.21.15.181:51242) 2023-10-16T11:01:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 -- 172.21.15.181:0/3134291929 learned_addr learned my addr 172.21.15.181:0/3134291929 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:02.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563ffff700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400ce680 0x7f56400c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563effd700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56400cc6e0 0x7f56401ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 -- 172.21.15.181:0/3134291929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400ce680 msgr2=0x7f56400c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56400ce680 0x7f56400c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 -- 172.21.15.181:0/3134291929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56400cc6e0 msgr2=0x7f56401ef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:02.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56400cc6e0 0x7f56401ef5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 -- 172.21.15.181:0/3134291929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56300094a0 con 0x7f56400b2010 2023-10-16T11:01:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f563f7fe700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56400b2010 0x7f56400c8050 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f562800e710 tx=0x7f562800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5637fff700 1 -- 172.21.15.181:0/3134291929 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280052d0 con 0x7f56400b2010 2023-10-16T11:01:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5637fff700 1 -- 172.21.15.181:0/3134291929 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5628005430 con 0x7f56400b2010 2023-10-16T11:01:02.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5645ae7700 1 -- 172.21.15.181:0/3134291929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564015d900 con 0x7f56400b2010 2023-10-16T11:01:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5637fff700 1 -- 172.21.15.181:0/3134291929 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562800c940 con 0x7f56400b2010 2023-10-16T11:01:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.645+0000 7f5645ae7700 1 -- 172.21.15.181:0/3134291929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56400c8590 con 0x7f56400b2010 2023-10-16T11:01:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.646+0000 7f5645ae7700 1 -- 172.21.15.181:0/3134291929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56401f3760 con 0x7f56400b2010 2023-10-16T11:01:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.647+0000 7f5637fff700 1 -- 172.21.15.181:0/3134291929 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 103) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f562800caa0 con 0x7f56400b2010 2023-10-16T11:01:02.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.647+0000 7f5637fff700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f562006ada0 0x7f562006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.647+0000 7f5637fff700 1 -- 172.21.15.181:0/3134291929 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f562808b6c0 con 0x7f56400b2010 2023-10-16T11:01:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.647+0000 7f563effd700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f562006ada0 0x7f562006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.647+0000 7f563effd700 1 --2- 172.21.15.181:0/3134291929 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f562006ada0 0x7f562006d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56401597f0 tx=0x7f5630013040 comp rx=0 tx=0).ready entity=mgr.9849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:02.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.652+0000 7f5637fff700 1 -- 172.21.15.181:0/3134291929 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5628056760 con 0x7f56400b2010 2023-10-16T11:01:02.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.868+0000 7f5645ae7700 1 -- 172.21.15.181:0/3134291929 --> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5640072ac0 con 0x7f562006ada0 2023-10-16T11:01:02.874 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:02.874+0000 7fe83b911700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-16T11:01:02.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:02.874+0000 7f5637fff700 1 -- 172.21.15.181:0/3134291929 <== mgr.9849 v2:172.21.15.181:6832/4215027768 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f5630004570 con 0x7f562006ada0 2023-10-16T11:01:02.876 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-16T11:01:02.880 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T11:01:02.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:02.881 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:01:02.900 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:01:03.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.069+0000 7f3c1c1e5700 1 Processor -- start 2023-10-16T11:01:03.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.069+0000 7f3c1c1e5700 1 -- start start 2023-10-16T11:01:03.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c1c1e5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c140d01d0 0x7f3c140ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c1c1e5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c140d1500 0x7f3c140d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c1c1e5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c140cf580 con 0x7f3c1415ea40 2023-10-16T11:01:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c1c1e5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c140d4290 con 0x7f3c140d01d0 2023-10-16T11:01:03.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c1c1e5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c140ce970 con 0x7f3c140d1500 2023-10-16T11:01:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c19f81700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c140d01d0 0x7f3c140ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c19780700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3c1415ea40 0x7f3c140d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34116/0 (socket says 172.21.15.181:34116) 2023-10-16T11:01:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c19780700 1 -- 172.21.15.181:0/1797081580 learned_addr learned my addr 172.21.15.181:0/1797081580 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c1a782700 1 --2- 172.21.15.181:0/1797081580 >> v2:172.21.15.190:3300/0 conn(0x7f3c140d1500 0x7f3c140d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c19f81700 1 -- 172.21.15.181:0/1797081580 >> v2:172.21.15.190:3300/0 conn(0x7f3c140d1500 msgr2=0x7f3c140d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:03.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c19f81700 1 --2- 172.21.15.181:0/1797081580 >> v2:172.21.15.190:3300/0 conn(0x7f3c140d1500 0x7f3c140d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c19f81700 1 -- 172.21.15.181:0/1797081580 >> v1:172.21.15.181:6789/0 conn(0x7f3c1415ea40 legacy=0x7f3c140d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c19f81700 1 -- 172.21.15.181:0/1797081580 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140cec50 con 0x7f3c140d01d0 2023-10-16T11:01:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.070+0000 7f3c18f7f700 1 -- 172.21.15.181:0/1797081580 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919704553 0 0) 0x7f3c140cf580 con 0x7f3c1415ea40 2023-10-16T11:01:03.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.071+0000 7f3c19f81700 1 --2- 172.21.15.181:0/1797081580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d01d0 0x7f3c140ce2d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f3c141625d0 tx=0x7f3c10009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712c007e3afa5ac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.071+0000 7f3c18f7f700 1 -- 172.21.15.181:0/1797081580 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10011830 con 0x7f3c140d01d0 2023-10-16T11:01:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.071+0000 7f3c18f7f700 1 -- 172.21.15.181:0/1797081580 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10011990 con 0x7f3c140d01d0 2023-10-16T11:01:03.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.071+0000 7f3c18f7f700 1 -- 172.21.15.181:0/1797081580 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10011d20 con 0x7f3c140d01d0 2023-10-16T11:01:03.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/1797081580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d01d0 msgr2=0x7f3c140ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:03.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/1797081580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d01d0 0x7f3c140ce2d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f3c141625d0 tx=0x7f3c10009b20 comp rx=0 tx=0).stop 2023-10-16T11:01:03.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/1797081580 shutdown_connections 2023-10-16T11:01:03.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/1797081580 >> v2:172.21.15.190:3300/0 conn(0x7f3c140d1500 0x7f3c140d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:03.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/1797081580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d01d0 0x7f3c140ce2d0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:03.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/1797081580 >> 172.21.15.181:0/1797081580 conn(0x7f3c1409cb60 msgr2=0x7f3c140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:03.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/1797081580 shutdown_connections 2023-10-16T11:01:03.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.072+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/1797081580 wait complete. 2023-10-16T11:01:03.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.073+0000 7f3c1c1e5700 1 Processor -- start 2023-10-16T11:01:03.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.073+0000 7f3c1c1e5700 1 -- start start 2023-10-16T11:01:03.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c1c1e5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c140d01d0 0x7f3c1415f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:03.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c1c1e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d1500 0x7f3c14161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c1c1e5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1415ea40 0x7f3c1415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c1c1e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141e7330 con 0x7f3c140d1500 2023-10-16T11:01:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c1c1e5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141e78f0 con 0x7f3c1415ea40 2023-10-16T11:01:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c1c1e5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140b63e0 con 0x7f3c140d01d0 2023-10-16T11:01:03.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d1500 0x7f3c14161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d1500 0x7f3c14161540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51276/0 (socket says 172.21.15.181:51276) 2023-10-16T11:01:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 -- 172.21.15.181:0/2147147394 learned_addr learned my addr 172.21.15.181:0/2147147394 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c1a782700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1415ea40 0x7f3c1415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19f81700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c140d01d0 0x7f3c1415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:03.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 -- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c140d01d0 msgr2=0x7f3c1415f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c140d01d0 0x7f3c1415f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 -- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1415ea40 msgr2=0x7f3c1415c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1415ea40 0x7f3c1415c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 -- 172.21.15.181:0/2147147394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100094a0 con 0x7f3c140d1500 2023-10-16T11:01:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c19780700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d1500 0x7f3c14161540 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c750 tx=0x7f3c0400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:03.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.074+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04004c80 con 0x7f3c140d1500 2023-10-16T11:01:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.075+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c04004e00 con 0x7f3c140d1500 2023-10-16T11:01:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.075+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c14161a80 con 0x7f3c140d1500 2023-10-16T11:01:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.075+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04007310 con 0x7f3c140d1500 2023-10-16T11:01:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.075+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c14161fa0 con 0x7f3c140d1500 2023-10-16T11:01:03.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.076+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1404ebb0 con 0x7f3c140d1500 2023-10-16T11:01:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.076+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 103) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3c04007470 con 0x7f3c140d1500 2023-10-16T11:01:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.076+0000 7f3c0a7fc700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f3bf806ada0 0x7f3bf806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.077+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f3c040891f0 con 0x7f3c140d1500 2023-10-16T11:01:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.077+0000 7f3c19f81700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f3bf806ada0 0x7f3bf806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:03.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.077+0000 7f3c19f81700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f3bf806ada0 0x7f3bf806d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3c100090b0 tx=0x7f3c1001c040 comp rx=0 tx=0).ready entity=mgr.9849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:03.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.081+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3c04054290 con 0x7f3c140d1500 2023-10-16T11:01:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:03.329+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f3c14161a80 con 0x7f3c140d1500 2023-10-16T11:01:04.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.011+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v104) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3c14161a80 con 0x7f3c140d1500 2023-10-16T11:01:04.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.018+0000 7f3c0a7fc700 1 -- 172.21.15.181:0/2147147394 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3c0405e080 con 0x7f3c140d1500 2023-10-16T11:01:04.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f3bf806ada0 msgr2=0x7f3bf806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f3bf806ada0 0x7f3bf806d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3c100090b0 tx=0x7f3c1001c040 comp rx=0 tx=0).stop 2023-10-16T11:01:04.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d1500 msgr2=0x7f3c14161540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d1500 0x7f3c14161540 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c750 tx=0x7f3c0400cb10 comp rx=0 tx=0).stop 2023-10-16T11:01:04.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 shutdown_connections 2023-10-16T11:01:04.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c140d01d0 0x7f3c1415f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f3bf806ada0 0x7f3bf806d260 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c1415ea40 0x7f3c1415c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 --2- 172.21.15.181:0/2147147394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c140d1500 0x7f3c14161540 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 >> 172.21.15.181:0/2147147394 conn(0x7f3c1409cb60 msgr2=0x7f3c1406ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:04.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 shutdown_connections 2023-10-16T11:01:04.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.019+0000 7f3c1c1e5700 1 -- 172.21.15.181:0/2147147394 wait complete. 2023-10-16T11:01:04.036 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:01:04.036 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:01:04.079 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:01:04.127 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:01:04.127 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:01:04.128 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:04.146 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:01:04.146 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:04.165 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:04.164+0000 7f5bb86bb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:01:04.170 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:04.170+0000 7f42d8c9c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:01:04.187 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:04.231 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:04.229+0000 7f5bb86bb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:04.239 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:04.238+0000 7f42d8c9c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:04.303 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:04.302+0000 7f5bb86bb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:01:04.314 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:04.314+0000 7f42d8c9c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:01:04.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.352+0000 7fe2bf73b700 1 Processor -- start 2023-10-16T11:01:04.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.352+0000 7fe2bf73b700 1 -- start start 2023-10-16T11:01:04.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bf73b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe2b815dd00 0x7fe2b8071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bf73b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b8071870 0x7fe2b81574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bf73b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b8071df0 con 0x7fe2b815e0e0 2023-10-16T11:01:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bf73b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b80720d0 con 0x7fe2b815dd00 2023-10-16T11:01:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bf73b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80723b0 con 0x7fe2b8071870 2023-10-16T11:01:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe2b815dd00 0x7fe2b8071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe2b815dd00 0x7fe2b8071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51324/0 (socket says 172.21.15.181:51324) 2023-10-16T11:01:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/3680145674 learned_addr learned my addr 172.21.15.181:0/3680145674 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bdcd8700 1 --1- 172.21.15.181:0/3680145674 >> v1:172.21.15.181:6789/0 conn(0x7fe2b815e0e0 0x7fe2b815a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34174/0 (socket says 172.21.15.181:34174) 2023-10-16T11:01:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bccd6700 1 --2- 172.21.15.181:0/3680145674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b8071870 0x7fe2b81574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/3680145674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b8071870 msgr2=0x7fe2b81574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 --2- 172.21.15.181:0/3680145674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b8071870 0x7fe2b81574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/3680145674 >> v1:172.21.15.181:6789/0 conn(0x7fe2b815e0e0 legacy=0x7fe2b815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/3680145674 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8072690 con 0x7fe2b815dd00 2023-10-16T11:01:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bccd6700 1 --2- 172.21.15.181:0/3680145674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b8071870 0x7fe2b81574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2bd4d7700 1 --2- 172.21.15.181:0/3680145674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b815dd00 0x7fe2b8071330 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40098f0 tx=0x7fe2b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bcfaeb3f0a83a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.353+0000 7fe2affff700 1 -- 172.21.15.181:0/3680145674 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400b8a0 con 0x7fe2b815dd00 2023-10-16T11:01:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2affff700 1 -- 172.21.15.181:0/3680145674 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b400ba00 con 0x7fe2b815dd00 2023-10-16T11:01:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2affff700 1 -- 172.21.15.181:0/3680145674 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400bd90 con 0x7fe2b815dd00 2023-10-16T11:01:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 -- 172.21.15.181:0/3680145674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b815dd00 msgr2=0x7fe2b8071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/3680145674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b815dd00 0x7fe2b8071330 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40098f0 tx=0x7fe2b4009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 -- 172.21.15.181:0/3680145674 shutdown_connections 2023-10-16T11:01:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/3680145674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b815dd00 0x7fe2b8071330 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/3680145674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b8071870 0x7fe2b81574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 -- 172.21.15.181:0/3680145674 >> 172.21.15.181:0/3680145674 conn(0x7fe2b806aed0 msgr2=0x7fe2b80b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 -- 172.21.15.181:0/3680145674 shutdown_connections 2023-10-16T11:01:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.354+0000 7fe2bf73b700 1 -- 172.21.15.181:0/3680145674 wait complete. 2023-10-16T11:01:04.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.355+0000 7fe2bf73b700 1 Processor -- start 2023-10-16T11:01:04.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.355+0000 7fe2bf73b700 1 -- start start 2023-10-16T11:01:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bf73b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b8071870 0x7fe2b80af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bf73b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2b815dd00 0x7fe2b81551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bf73b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b815e0e0 0x7fe2b8155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bf73b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b814b820 con 0x7fe2b8071870 2023-10-16T11:01:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bf73b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80c2b60 con 0x7fe2b815dd00 2023-10-16T11:01:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bf73b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80c2e40 con 0x7fe2b815e0e0 2023-10-16T11:01:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bd4d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b8071870 0x7fe2b80af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bd4d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b8071870 0x7fe2b80af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51330/0 (socket says 172.21.15.181:51330) 2023-10-16T11:01:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/2563656923 learned_addr learned my addr 172.21.15.181:0/2563656923 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bccd6700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2b815dd00 0x7fe2b81551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.356+0000 7fe2bdcd8700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b815e0e0 0x7fe2b8155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b815e0e0 msgr2=0x7fe2b8155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bd4d7700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b815e0e0 0x7fe2b8155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2b815dd00 msgr2=0x7fe2b81551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bd4d7700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2b815dd00 0x7fe2b81551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bd4d7700 1 -- 172.21.15.181:0/2563656923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4009550 con 0x7fe2b8071870 2023-10-16T11:01:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bd4d7700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b8071870 0x7fe2b80af710 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009c40 tx=0x7fe2b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2adffb700 1 -- 172.21.15.181:0/2563656923 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40044d0 con 0x7fe2b8071870 2023-10-16T11:01:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2adffb700 1 -- 172.21.15.181:0/2563656923 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b401f070 con 0x7fe2b8071870 2023-10-16T11:01:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b80c3120 con 0x7fe2b8071870 2023-10-16T11:01:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2adffb700 1 -- 172.21.15.181:0/2563656923 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40038e0 con 0x7fe2b8071870 2023-10-16T11:01:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.357+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b80c36e0 con 0x7fe2b8071870 2023-10-16T11:01:04.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.358+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b8153920 con 0x7fe2b8071870 2023-10-16T11:01:04.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.359+0000 7fe2adffb700 1 -- 172.21.15.181:0/2563656923 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fe2b4003a40 con 0x7fe2b8071870 2023-10-16T11:01:04.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.359+0000 7fe2adffb700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 0x7fe29c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.360+0000 7fe2bccd6700 1 -- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 msgr2=0x7fe29c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:04.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.360+0000 7fe2adffb700 1 -- 172.21.15.181:0/2563656923 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fe2b4021070 con 0x7fe2b8071870 2023-10-16T11:01:04.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.360+0000 7fe2bccd6700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 0x7fe29c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:04.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.364+0000 7fe2adffb700 1 -- 172.21.15.181:0/2563656923 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2b4055ec0 con 0x7fe2b8071870 2023-10-16T11:01:04.380 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:04.379+0000 7f5bb86bb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:01:04.398 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:04.398+0000 7f42d8c9c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:01:04.523 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:04.522+0000 7f5bb86bb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:01:04.546 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:04.546+0000 7f42d8c9c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:01:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.561+0000 7fe2bccd6700 1 -- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 msgr2=0x7fe29c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.561+0000 7fe2bccd6700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 0x7fe29c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:04.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.566+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2b804ebb0 con 0x7fe2b8071870 2023-10-16T11:01:04.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.567+0000 7fe2adffb700 1 -- 172.21.15.181:0/2563656923 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17880 (secure 0 0 0) 0x7fe2b40594e0 con 0x7fe2b8071870 2023-10-16T11:01:04.568 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:04.568 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":169,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:00.993337+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:00:44.693955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:00:45.236607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/642481654":"2023-10-17T11:01:00.993294+0000","172.21.15.181:6832/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/120971111":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/3800812080":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/2485734134":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/4030419984":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/1244048650":"2023-10-17T11:01:00.993294+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:04.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.569+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 msgr2=0x7fe29c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:04.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.569+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 0x7fe29c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.569+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b8071870 msgr2=0x7fe2b80af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.569+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b8071870 0x7fe2b80af710 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009c40 tx=0x7fe2b4004750 comp rx=0 tx=0).stop 2023-10-16T11:01:04.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 shutdown_connections 2023-10-16T11:01:04.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2b815e0e0 0x7fe2b8155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fe29c06b020 0x7fe29c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2b8071870 0x7fe2b80af710 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 --2- 172.21.15.181:0/2563656923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2b815dd00 0x7fe2b81551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 >> 172.21.15.181:0/2563656923 conn(0x7fe2b806aed0 msgr2=0x7fe2b80cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:04.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 shutdown_connections 2023-10-16T11:01:04.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.570+0000 7fe2bf73b700 1 -- 172.21.15.181:0/2563656923 wait complete. 2023-10-16T11:01:04.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:00:44.693955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:00:45.236607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:01:04.584 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:01:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.749+0000 7f4bc8950700 1 Processor -- start 2023-10-16T11:01:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.749+0000 7f4bc8950700 1 -- start start 2023-10-16T11:01:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc8950700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bc40ce6c0 0x7f4bc40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc8950700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc40b1f70 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc8950700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc40b2ae0 con 0x7f4bc415f150 2023-10-16T11:01:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc8950700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc40b2dc0 con 0x7f4bc40ce6c0 2023-10-16T11:01:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc1d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bc40ce6c0 0x7f4bc40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc1d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bc40ce6c0 0x7f4bc40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51364/0 (socket says 172.21.15.181:51364) 2023-10-16T11:01:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc1d9b700 1 -- 172.21.15.181:0/2184599650 learned_addr learned my addr 172.21.15.181:0/2184599650 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc1d9b700 1 -- 172.21.15.181:0/2184599650 >> v1:172.21.15.190:6789/0 conn(0x7f4bc415f150 legacy=0x7f4bc40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc1d9b700 1 -- 172.21.15.181:0/2184599650 >> v1:172.21.15.181:6789/0 conn(0x7f4bc40cc6a0 legacy=0x7f4bc40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc1d9b700 1 -- 172.21.15.181:0/2184599650 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40cf9d0 con 0x7f4bc40ce6c0 2023-10-16T11:01:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.750+0000 7f4bc1d9b700 1 --2- 172.21.15.181:0/2184599650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40ce6c0 0x7f4bc40cc160 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80098f0 tx=0x7f4bb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171b49b9249d957a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc159a700 1 -- 172.21.15.181:0/2184599650 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800b8a0 con 0x7f4bc40ce6c0 2023-10-16T11:01:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc159a700 1 -- 172.21.15.181:0/2184599650 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb800ba00 con 0x7f4bc40ce6c0 2023-10-16T11:01:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc159a700 1 -- 172.21.15.181:0/2184599650 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800bd90 con 0x7f4bc40ce6c0 2023-10-16T11:01:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc8950700 1 -- 172.21.15.181:0/2184599650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40ce6c0 msgr2=0x7f4bc40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2184599650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40ce6c0 0x7f4bc40cc160 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80098f0 tx=0x7f4bb8009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc8950700 1 -- 172.21.15.181:0/2184599650 shutdown_connections 2023-10-16T11:01:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2184599650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40ce6c0 0x7f4bc40cc160 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc8950700 1 -- 172.21.15.181:0/2184599650 >> 172.21.15.181:0/2184599650 conn(0x7f4bc409a930 msgr2=0x7f4bc40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc8950700 1 -- 172.21.15.181:0/2184599650 shutdown_connections 2023-10-16T11:01:04.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.751+0000 7f4bc8950700 1 -- 172.21.15.181:0/2184599650 wait complete. 2023-10-16T11:01:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.753+0000 7f4bc8950700 1 Processor -- start 2023-10-16T11:01:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.753+0000 7f4bc8950700 1 -- start start 2023-10-16T11:01:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc8950700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40cc6a0 0x7f4bc415d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc8950700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc40ce6c0 0x7f4bc41f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc8950700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc415f150 0x7f4bc41f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc8950700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c7830 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc8950700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40cb520 con 0x7f4bc40ce6c0 2023-10-16T11:01:04.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc8950700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c8800 con 0x7f4bc415f150 2023-10-16T11:01:04.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40cc6a0 0x7f4bc415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40cc6a0 0x7f4bc415d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51368/0 (socket says 172.21.15.181:51368) 2023-10-16T11:01:04.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 -- 172.21.15.181:0/2827285308 learned_addr learned my addr 172.21.15.181:0/2827285308 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:04.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc1d9b700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc40ce6c0 0x7f4bc41f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc2d9d700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc415f150 0x7f4bc41f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:04.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 -- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc415f150 msgr2=0x7f4bc41f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc415f150 0x7f4bc41f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 -- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc40ce6c0 msgr2=0x7f4bc41f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc40ce6c0 0x7f4bc41f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.754+0000 7f4bc259c700 1 -- 172.21.15.181:0/2827285308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb8009550 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.755+0000 7f4bc259c700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40cc6a0 0x7f4bc415d180 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c430 tx=0x7f4bb400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:04.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.755+0000 7f4bb2ffd700 1 -- 172.21.15.181:0/2827285308 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40052d0 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.755+0000 7f4bb2ffd700 1 -- 172.21.15.181:0/2827285308 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb4005450 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.755+0000 7f4bb2ffd700 1 -- 172.21.15.181:0/2827285308 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4006aa0 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.755+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc40c8ae0 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.755+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc40c90a0 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.756+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc40b0700 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.756+0000 7f4bb2ffd700 1 -- 172.21.15.181:0/2827285308 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4bb4003680 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.757+0000 7f4bb2ffd700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.757+0000 7f4bc1d9b700 1 -- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 msgr2=0x7f4ba406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:04.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.757+0000 7f4bb2ffd700 1 -- 172.21.15.181:0/2827285308 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f4bb4088c20 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.757+0000 7f4bc1d9b700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:04.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.765+0000 7f4bb2ffd700 1 -- 172.21.15.181:0/2827285308 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4bb4053880 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.850 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:04.849+0000 7f5bb86bb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:01:04.889 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:04.889+0000 7f42d8c9c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:01:04.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.958+0000 7f4bc1d9b700 1 -- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 msgr2=0x7f4ba406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:04.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.958+0000 7f4bc1d9b700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:04.965 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:04.964+0000 7f5bb86bb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:01:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.968+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bc4067d20 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.968+0000 7f4bb2ffd700 1 -- 172.21.15.181:0/2827285308 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bb4056ea0 con 0x7f4bc40cc6a0 2023-10-16T11:01:04.969 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:01:04.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 msgr2=0x7f4ba406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:04.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40cc6a0 msgr2=0x7f4bc415d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:04.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40cc6a0 0x7f4bc415d180 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c430 tx=0x7f4bb400c740 comp rx=0 tx=0).stop 2023-10-16T11:01:04.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 shutdown_connections 2023-10-16T11:01:04.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc415f150 0x7f4bc41f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f4ba406b020 0x7f4ba406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc40cc6a0 0x7f4bc415d180 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 --2- 172.21.15.181:0/2827285308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4bc40ce6c0 0x7f4bc41f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 >> 172.21.15.181:0/2827285308 conn(0x7f4bc409a930 msgr2=0x7f4bc40af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 shutdown_connections 2023-10-16T11:01:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:04.971+0000 7f4bc8950700 1 -- 172.21.15.181:0/2827285308 wait complete. 2023-10-16T11:01:04.985 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:01:05.016 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:05.015+0000 7f42d8c9c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:01:05.038 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:05.037+0000 7f5bb86bb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:01:05.093 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:05.093+0000 7f42d8c9c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:01:05.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.112+0000 7f6e7be06700 1 Processor -- start 2023-10-16T11:01:05.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.112+0000 7f6e7be06700 1 -- start start 2023-10-16T11:01:05.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7be06700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e7415fb30 0x7f6e7406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7be06700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e74070590 con 0x7f6e740cb690 2023-10-16T11:01:05.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7be06700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e74156330 con 0x7f6e740cb2b0 2023-10-16T11:01:05.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7be06700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7406fcf0 con 0x7f6e7415fb30 2023-10-16T11:01:05.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e793a1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6e740cb690 0x7f6e7415f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34226/0 (socket says 172.21.15.181:34226) 2023-10-16T11:01:05.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e793a1700 1 -- 172.21.15.181:0/1877959537 learned_addr learned my addr 172.21.15.181:0/1877959537 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:05.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7a3a3700 1 --2- 172.21.15.181:0/1877959537 >> v2:172.21.15.190:3300/0 conn(0x7f6e7415fb30 0x7f6e7406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e78ba0700 1 -- 172.21.15.181:0/1877959537 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278647735 0 0) 0x7f6e74070590 con 0x7f6e740cb690 2023-10-16T11:01:05.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e78ba0700 1 -- 172.21.15.181:0/1877959537 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e600035b0 con 0x7f6e740cb690 2023-10-16T11:01:05.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7a3a3700 1 -- 172.21.15.181:0/1877959537 >> v1:172.21.15.190:6789/0 conn(0x7f6e740cb2b0 legacy=0x7f6e740b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7a3a3700 1 -- 172.21.15.181:0/1877959537 >> v1:172.21.15.181:6789/0 conn(0x7f6e740cb690 legacy=0x7f6e7415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7a3a3700 1 -- 172.21.15.181:0/1877959537 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7406ffd0 con 0x7f6e7415fb30 2023-10-16T11:01:05.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.113+0000 7f6e7a3a3700 1 --2- 172.21.15.181:0/1877959537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e7415fb30 0x7f6e7406f710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009d90 tx=0x7f6e6c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327c573138903fe5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:05.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e78ba0700 1 -- 172.21.15.181:0/1877959537 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c00e070 con 0x7f6e7415fb30 2023-10-16T11:01:05.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e78ba0700 1 -- 172.21.15.181:0/1877959537 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c00bb20 con 0x7f6e7415fb30 2023-10-16T11:01:05.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e78ba0700 1 -- 172.21.15.181:0/1877959537 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c012430 con 0x7f6e7415fb30 2023-10-16T11:01:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e7be06700 1 -- 172.21.15.181:0/1877959537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e7415fb30 msgr2=0x7f6e7406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e7be06700 1 --2- 172.21.15.181:0/1877959537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e7415fb30 0x7f6e7406f710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009d90 tx=0x7f6e6c00b330 comp rx=0 tx=0).stop 2023-10-16T11:01:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e7be06700 1 -- 172.21.15.181:0/1877959537 shutdown_connections 2023-10-16T11:01:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e7be06700 1 --2- 172.21.15.181:0/1877959537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e7415fb30 0x7f6e7406f710 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e7be06700 1 -- 172.21.15.181:0/1877959537 >> 172.21.15.181:0/1877959537 conn(0x7f6e740cd010 msgr2=0x7f6e740cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e7be06700 1 -- 172.21.15.181:0/1877959537 shutdown_connections 2023-10-16T11:01:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.114+0000 7f6e7be06700 1 -- 172.21.15.181:0/1877959537 wait complete. 2023-10-16T11:01:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 Processor -- start 2023-10-16T11:01:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 -- start start 2023-10-16T11:01:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740cb2b0 0x7f6e740c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e740cb690 0x7f6e740c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e740c6730 0x7f6e740c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e740c6fd0 con 0x7f6e740cb690 2023-10-16T11:01:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e740b61c0 con 0x7f6e740c6730 2023-10-16T11:01:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7be06700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7414e390 con 0x7f6e740cb2b0 2023-10-16T11:01:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e793a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e740cb690 0x7f6e740c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e793a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e740cb690 0x7f6e740c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51386/0 (socket says 172.21.15.181:51386) 2023-10-16T11:01:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e793a1700 1 -- 172.21.15.181:0/3248949023 learned_addr learned my addr 172.21.15.181:0/3248949023 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e7a3a3700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e740c6730 0x7f6e740c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.116+0000 7f6e79ba2700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740cb2b0 0x7f6e740c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e793a1700 1 -- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740cb2b0 msgr2=0x7f6e740c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e793a1700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740cb2b0 0x7f6e740c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e793a1700 1 -- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e740c6730 msgr2=0x7f6e740c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e793a1700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e740c6730 0x7f6e740c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e793a1700 1 -- 172.21.15.181:0/3248949023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c009a70 con 0x7f6e740cb690 2023-10-16T11:01:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e793a1700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e740cb690 0x7f6e740c4980 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c700 tx=0x7f6e6400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e6a7fc700 1 -- 172.21.15.181:0/3248949023 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64005d70 con 0x7f6e740cb690 2023-10-16T11:01:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e740c4650 con 0x7f6e740cb690 2023-10-16T11:01:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e6a7fc700 1 -- 172.21.15.181:0/3248949023 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e640070d0 con 0x7f6e740cb690 2023-10-16T11:01:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e6a7fc700 1 -- 172.21.15.181:0/3248949023 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64007460 con 0x7f6e740cb690 2023-10-16T11:01:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.117+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e740b56e0 con 0x7f6e740cb690 2023-10-16T11:01:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.118+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c0052f0 con 0x7f6e740cb690 2023-10-16T11:01:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.118+0000 7f6e6a7fc700 1 -- 172.21.15.181:0/3248949023 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f6e640075c0 con 0x7f6e740cb690 2023-10-16T11:01:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.119+0000 7f6e6a7fc700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 0x7f6e5806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.119+0000 7f6e79ba2700 1 -- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 msgr2=0x7f6e5806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:05.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.119+0000 7f6e6a7fc700 1 -- 172.21.15.181:0/3248949023 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f6e64088f40 con 0x7f6e740cb690 2023-10-16T11:01:05.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.119+0000 7f6e79ba2700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 0x7f6e5806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:05.129 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:05.118+0000 7f5bb86bb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:01:05.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.127+0000 7f6e6a7fc700 1 -- 172.21.15.181:0/3248949023 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6e64053b10 con 0x7f6e740cb690 2023-10-16T11:01:05.184 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:05.183+0000 7f42d8c9c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:01:05.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.320+0000 7f6e79ba2700 1 -- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 msgr2=0x7f6e5806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:05.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.320+0000 7f6e79ba2700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 0x7f6e5806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.326+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e5c005b20 con 0x7f6e740cb690 2023-10-16T11:01:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.327+0000 7f6e6a7fc700 1 -- 172.21.15.181:0/3248949023 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v169) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e64057130 con 0x7f6e740cb690 2023-10-16T11:01:05.328 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:01:05.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 msgr2=0x7f6e5806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:05.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 0x7f6e5806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e740cb690 msgr2=0x7f6e740c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e740cb690 0x7f6e740c4980 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c700 tx=0x7f6e6400ca10 comp rx=0 tx=0).stop 2023-10-16T11:01:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 shutdown_connections 2023-10-16T11:01:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e740cb2b0 0x7f6e740c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6e5806b020 0x7f6e5806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e740c6730 0x7f6e740c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 --2- 172.21.15.181:0/3248949023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e740cb690 0x7f6e740c4980 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 >> 172.21.15.181:0/3248949023 conn(0x7f6e740cd010 msgr2=0x7f6e740b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 shutdown_connections 2023-10-16T11:01:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.329+0000 7f6e7be06700 1 -- 172.21.15.181:0/3248949023 wait complete. 2023-10-16T11:01:05.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:01:05.495 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:05.494+0000 7f5bb86bb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:01:05.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.509+0000 7f6b47a66700 1 Processor -- start 2023-10-16T11:01:05.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.509+0000 7f6b47a66700 1 -- start start 2023-10-16T11:01:05.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b47a66700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6b400ce200 0x7f6b4006be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b47a66700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4015dfb0 con 0x7f6b4015dbd0 2023-10-16T11:01:05.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b47a66700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b400cd3a0 con 0x7f6b400cdd80 2023-10-16T11:01:05.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b47a66700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b400cd830 con 0x7f6b400ce200 2023-10-16T11:01:05.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b45802700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6b4015dbd0 0x7f6b400d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34254/0 (socket says 172.21.15.181:34254) 2023-10-16T11:01:05.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b45802700 1 -- 172.21.15.181:0/81381294 learned_addr learned my addr 172.21.15.181:0/81381294 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:05.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b46003700 1 --2- 172.21.15.181:0/81381294 >> v2:172.21.15.181:3300/0 conn(0x7f6b400ce200 0x7f6b4006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b44800700 1 -- 172.21.15.181:0/81381294 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706161854 0 0) 0x7f6b400cd3a0 con 0x7f6b400cdd80 2023-10-16T11:01:05.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.510+0000 7f6b46003700 1 -- 172.21.15.181:0/81381294 >> v1:172.21.15.190:6789/0 conn(0x7f6b400cdd80 legacy=0x7f6b4006b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.511+0000 7f6b46003700 1 -- 172.21.15.181:0/81381294 >> v1:172.21.15.181:6789/0 conn(0x7f6b4015dbd0 legacy=0x7f6b400d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.511+0000 7f6b46003700 1 -- 172.21.15.181:0/81381294 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400ce6b0 con 0x7f6b400ce200 2023-10-16T11:01:05.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.511+0000 7f6b46003700 1 --2- 172.21.15.181:0/81381294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400ce200 0x7f6b4006be00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009d90 tx=0x7f6b3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bee26ecd0c4e296 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:05.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.511+0000 7f6b44800700 1 -- 172.21.15.181:0/81381294 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800e070 con 0x7f6b400ce200 2023-10-16T11:01:05.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.511+0000 7f6b44800700 1 -- 172.21.15.181:0/81381294 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3800bad0 con 0x7f6b400ce200 2023-10-16T11:01:05.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.511+0000 7f6b44800700 1 -- 172.21.15.181:0/81381294 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3800be80 con 0x7f6b400ce200 2023-10-16T11:01:05.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.512+0000 7f6b47a66700 1 -- 172.21.15.181:0/81381294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400ce200 msgr2=0x7f6b4006be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.512+0000 7f6b47a66700 1 --2- 172.21.15.181:0/81381294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400ce200 0x7f6b4006be00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009d90 tx=0x7f6b3800b330 comp rx=0 tx=0).stop 2023-10-16T11:01:05.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.512+0000 7f6b47a66700 1 -- 172.21.15.181:0/81381294 shutdown_connections 2023-10-16T11:01:05.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.512+0000 7f6b47a66700 1 --2- 172.21.15.181:0/81381294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400ce200 0x7f6b4006be00 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.512+0000 7f6b47a66700 1 -- 172.21.15.181:0/81381294 >> 172.21.15.181:0/81381294 conn(0x7f6b4009a930 msgr2=0x7f6b400d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:05.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.512+0000 7f6b47a66700 1 -- 172.21.15.181:0/81381294 shutdown_connections 2023-10-16T11:01:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.512+0000 7f6b47a66700 1 -- 172.21.15.181:0/81381294 wait complete. 2023-10-16T11:01:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.513+0000 7f6b47a66700 1 Processor -- start 2023-10-16T11:01:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.513+0000 7f6b47a66700 1 -- start start 2023-10-16T11:01:05.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b47a66700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400cdd80 0x7f6b4014ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b47a66700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b400ce200 0x7f6b4009d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b47a66700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b4015dbd0 0x7f6b401f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b47a66700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401f3f30 con 0x7f6b400cdd80 2023-10-16T11:01:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b47a66700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4014f4b0 con 0x7f6b400ce200 2023-10-16T11:01:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b47a66700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401f33b0 con 0x7f6b4015dbd0 2023-10-16T11:01:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400cdd80 0x7f6b4014ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400cdd80 0x7f6b4014ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51414/0 (socket says 172.21.15.181:51414) 2023-10-16T11:01:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 -- 172.21.15.181:0/2352551405 learned_addr learned my addr 172.21.15.181:0/2352551405 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b46003700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b4015dbd0 0x7f6b401f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45001700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b400ce200 0x7f6b4009d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 -- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b4015dbd0 msgr2=0x7f6b401f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b4015dbd0 0x7f6b401f2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 -- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b400ce200 msgr2=0x7f6b4009d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b400ce200 0x7f6b4009d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 -- 172.21.15.181:0/2352551405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38009a70 con 0x7f6b400cdd80 2023-10-16T11:01:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b45802700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400cdd80 0x7f6b4014ef70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c850 tx=0x7f6b3c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b367fc700 1 -- 172.21.15.181:0/2352551405 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004c80 con 0x7f6b400cdd80 2023-10-16T11:01:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b367fc700 1 -- 172.21.15.181:0/2352551405 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c004de0 con 0x7f6b400cdd80 2023-10-16T11:01:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b367fc700 1 -- 172.21.15.181:0/2352551405 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c007310 con 0x7f6b400cdd80 2023-10-16T11:01:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.514+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b401f3690 con 0x7f6b400cdd80 2023-10-16T11:01:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.515+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b401f3c50 con 0x7f6b400cdd80 2023-10-16T11:01:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.516+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b401f65c0 con 0x7f6b400cdd80 2023-10-16T11:01:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.516+0000 7f6b367fc700 1 -- 172.21.15.181:0/2352551405 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f6b3c007470 con 0x7f6b400cdd80 2023-10-16T11:01:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.517+0000 7f6b367fc700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 0x7f6b2406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.517+0000 7f6b45001700 1 -- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 msgr2=0x7f6b2406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:05.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.517+0000 7f6b367fc700 1 -- 172.21.15.181:0/2352551405 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f6b3c088b90 con 0x7f6b400cdd80 2023-10-16T11:01:05.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.517+0000 7f6b45001700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 0x7f6b2406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:05.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.521+0000 7f6b367fc700 1 -- 172.21.15.181:0/2352551405 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6b3c053950 con 0x7f6b400cdd80 2023-10-16T11:01:05.575 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:05.574+0000 7f42d8c9c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:01:05.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.718+0000 7f6b45001700 1 -- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 msgr2=0x7f6b2406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:05.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.718+0000 7f6b45001700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 0x7f6b2406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:05.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.722+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b401f65c0 con 0x7f6b400cdd80 2023-10-16T11:01:05.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.723+0000 7f6b367fc700 1 -- 172.21.15.181:0/2352551405 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b401f65c0 con 0x7f6b400cdd80 2023-10-16T11:01:05.724 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:01:05.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 msgr2=0x7f6b2406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:05.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 0x7f6b2406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400cdd80 msgr2=0x7f6b4014ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400cdd80 0x7f6b4014ef70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c850 tx=0x7f6b3c00cb60 comp rx=0 tx=0).stop 2023-10-16T11:01:05.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 shutdown_connections 2023-10-16T11:01:05.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b4015dbd0 0x7f6b401f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f6b2406b020 0x7f6b2406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b400cdd80 0x7f6b4014ef70 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 --2- 172.21.15.181:0/2352551405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b400ce200 0x7f6b4009d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 >> 172.21.15.181:0/2352551405 conn(0x7f6b4009a930 msgr2=0x7f6b4006cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 shutdown_connections 2023-10-16T11:01:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.725+0000 7f6b47a66700 1 -- 172.21.15.181:0/2352551405 wait complete. 2023-10-16T11:01:05.738 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:01:05.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.862+0000 7f14d1643700 1 Processor -- start 2023-10-16T11:01:05.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.862+0000 7f14d1643700 1 -- start start 2023-10-16T11:01:05.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14d1643700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f14cc15e460 0x7f14cc0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14d1643700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14cc0cb110 0x7f14cc15e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14d1643700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc06ed00 con 0x7f14cc15a0f0 2023-10-16T11:01:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14d1643700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14cc15ed80 con 0x7f14cc15e460 2023-10-16T11:01:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14d1643700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14cc06e0f0 con 0x7f14cc0cb110 2023-10-16T11:01:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f14cc15e460 0x7f14cc0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f14cc15e460 0x7f14cc0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51424/0 (socket says 172.21.15.181:51424) 2023-10-16T11:01:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 -- 172.21.15.181:0/2101624878 learned_addr learned my addr 172.21.15.181:0/2101624878 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14ca7fc700 1 --2- 172.21.15.181:0/2101624878 >> v2:172.21.15.190:3300/0 conn(0x7f14cc0cb110 0x7f14cc15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 -- 172.21.15.181:0/2101624878 >> v2:172.21.15.190:3300/0 conn(0x7f14cc0cb110 msgr2=0x7f14cc15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 --2- 172.21.15.181:0/2101624878 >> v2:172.21.15.190:3300/0 conn(0x7f14cc0cb110 0x7f14cc15e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 -- 172.21.15.181:0/2101624878 >> v1:172.21.15.181:6789/0 conn(0x7f14cc15a0f0 legacy=0x7f14cc15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 -- 172.21.15.181:0/2101624878 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc06e3d0 con 0x7f14cc15e460 2023-10-16T11:01:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14ca7fc700 1 --2- 172.21.15.181:0/2101624878 >> v2:172.21.15.190:3300/0 conn(0x7f14cc0cb110 0x7f14cc15e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.863+0000 7f14caffd700 1 --2- 172.21.15.181:0/2101624878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15e460 0x7f14cc0d0fc0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f14cc15e3c0 tx=0x7f14bc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea7a7858731145e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14c9ffb700 1 -- 172.21.15.181:0/2101624878 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc005e30 con 0x7f14cc15e460 2023-10-16T11:01:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14c9ffb700 1 -- 172.21.15.181:0/2101624878 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc004030 con 0x7f14cc15e460 2023-10-16T11:01:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14c9ffb700 1 -- 172.21.15.181:0/2101624878 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0043e0 con 0x7f14cc15e460 2023-10-16T11:01:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 -- 172.21.15.181:0/2101624878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15e460 msgr2=0x7f14cc0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 --2- 172.21.15.181:0/2101624878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15e460 0x7f14cc0d0fc0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f14cc15e3c0 tx=0x7f14bc00be70 comp rx=0 tx=0).stop 2023-10-16T11:01:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 -- 172.21.15.181:0/2101624878 shutdown_connections 2023-10-16T11:01:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 --2- 172.21.15.181:0/2101624878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15e460 0x7f14cc0d0fc0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 --2- 172.21.15.181:0/2101624878 >> v2:172.21.15.190:3300/0 conn(0x7f14cc0cb110 0x7f14cc15e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 -- 172.21.15.181:0/2101624878 >> 172.21.15.181:0/2101624878 conn(0x7f14cc0d04a0 msgr2=0x7f14cc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 -- 172.21.15.181:0/2101624878 shutdown_connections 2023-10-16T11:01:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.864+0000 7f14d1643700 1 -- 172.21.15.181:0/2101624878 wait complete. 2023-10-16T11:01:05.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 Processor -- start 2023-10-16T11:01:05.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 -- start start 2023-10-16T11:01:05.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14cc0cb110 0x7f14cc0c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15a0f0 0x7f14cc0c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14cc15e460 0x7f14cc0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc154d30 con 0x7f14cc15a0f0 2023-10-16T11:01:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc154f40 con 0x7f14cc0cb110 2023-10-16T11:01:05.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.866+0000 7f14d1643700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc155220 con 0x7f14cc15e460 2023-10-16T11:01:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15a0f0 0x7f14cc0c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15a0f0 0x7f14cc0c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51430/0 (socket says 172.21.15.181:51430) 2023-10-16T11:01:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 -- 172.21.15.181:0/2831848198 learned_addr learned my addr 172.21.15.181:0/2831848198 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14caffd700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14cc0cb110 0x7f14cc0c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14cb7fe700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14cc15e460 0x7f14cc0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 -- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14cc15e460 msgr2=0x7f14cc0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14cc15e460 0x7f14cc0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 -- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14cc0cb110 msgr2=0x7f14cc0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14cc0cb110 0x7f14cc0c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 -- 172.21.15.181:0/2831848198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc00b7a0 con 0x7f14cc15a0f0 2023-10-16T11:01:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14ca7fc700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15a0f0 0x7f14cc0c94b0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e810 tx=0x7f14c000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14b37fe700 1 -- 172.21.15.181:0/2831848198 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c000c020 con 0x7f14cc15a0f0 2023-10-16T11:01:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14b37fe700 1 -- 172.21.15.181:0/2831848198 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0004250 con 0x7f14cc15a0f0 2023-10-16T11:01:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc155500 con 0x7f14cc15a0f0 2023-10-16T11:01:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14b37fe700 1 -- 172.21.15.181:0/2831848198 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c00036a0 con 0x7f14cc15a0f0 2023-10-16T11:01:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.867+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc159430 con 0x7f14cc15a0f0 2023-10-16T11:01:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.868+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc155b50 con 0x7f14cc15a0f0 2023-10-16T11:01:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.869+0000 7f14b37fe700 1 -- 172.21.15.181:0/2831848198 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f14c0003800 con 0x7f14cc15a0f0 2023-10-16T11:01:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.869+0000 7f14b37fe700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 0x7f14b806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.869+0000 7f14caffd700 1 -- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 msgr2=0x7f14b806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.869+0000 7f14b37fe700 1 -- 172.21.15.181:0/2831848198 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f14c00894f0 con 0x7f14cc15a0f0 2023-10-16T11:01:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.869+0000 7f14caffd700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 0x7f14b806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:05.874+0000 7f14b37fe700 1 -- 172.21.15.181:0/2831848198 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f14c0054030 con 0x7f14cc15a0f0 2023-10-16T11:01:06.064 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:06.062+0000 7f5bb86bb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:01:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.070+0000 7f14caffd700 1 -- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 msgr2=0x7f14b806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:06.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.070+0000 7f14caffd700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 0x7f14b806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:06.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.123+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f14cc06f170 con 0x7f14cc15a0f0 2023-10-16T11:01:06.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.123+0000 7f14b37fe700 1 -- 172.21.15.181:0/2831848198 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v169) v1 ==== 80+0+474 (secure 0 0 0) 0x7f14cc06f170 con 0x7f14cc15a0f0 2023-10-16T11:01:06.124 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/551790976 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.125 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/642481654 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.125 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/551790976 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.125 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/120971111 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.125 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3800812080 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.125 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2485734134 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.126 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/4030419984 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.126 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1244048650 2023-10-17T11:01:00.993294+0000 2023-10-16T11:01:06.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 msgr2=0x7f14b806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:06.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 0x7f14b806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15a0f0 msgr2=0x7f14cc0c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15a0f0 0x7f14cc0c94b0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e810 tx=0x7f14c000eb20 comp rx=0 tx=0).stop 2023-10-16T11:01:06.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 shutdown_connections 2023-10-16T11:01:06.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14cc15e460 0x7f14cc0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f14b806b020 0x7f14b806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14cc0cb110 0x7f14cc0c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 --2- 172.21.15.181:0/2831848198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14cc15a0f0 0x7f14cc0c94b0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 >> 172.21.15.181:0/2831848198 conn(0x7f14cc0d04a0 msgr2=0x7f14cc0aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:06.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 shutdown_connections 2023-10-16T11:01:06.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.126+0000 7f14d1643700 1 -- 172.21.15.181:0/2831848198 wait complete. 2023-10-16T11:01:06.129 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T11:01:06.133 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:06.132+0000 7f5bb86bb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:01:06.140 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:01:06.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:06.141 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:01:06.174 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:06.174+0000 7f42d8c9c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:01:06.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:01:06.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:06.187 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:01:06.244 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:01:06.245 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:06.248 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:06.248+0000 7f42d8c9c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:01:06.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:01:06.263 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:01:06.263 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:06.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:01:06.280 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:01:06.281 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:06.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:01:06.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:01:06.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.443+0000 7f51e5ca8700 1 Processor -- start 2023-10-16T11:01:06.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51e5ca8700 1 -- start start 2023-10-16T11:01:06.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51e5ca8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51e5ca8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51e0160d20 0x7f51e01611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51e5ca8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e006ce20 con 0x7f51e0161f40 2023-10-16T11:01:06.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51e5ca8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e0162800 con 0x7f51e0160d20 2023-10-16T11:01:06.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51e5ca8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00d43e0 con 0x7f51e0161b60 2023-10-16T11:01:06.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51dffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51e0160d20 0x7f51e01611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51deffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f51e0161f40 0x7f51e00d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34284/0 (socket says 172.21.15.181:34284) 2023-10-16T11:01:06.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51deffd700 1 -- 172.21.15.181:0/1615056243 learned_addr learned my addr 172.21.15.181:0/1615056243 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:06.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.444+0000 7f51dffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51e0160d20 0x7f51e01611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51442/0 (socket says 172.21.15.181:51442) 2023-10-16T11:01:06.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51df7fe700 1 --2- 172.21.15.181:0/1615056243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51dffff700 1 -- 172.21.15.181:0/1615056243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 msgr2=0x7f51e006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:06.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51dffff700 1 --2- 172.21.15.181:0/1615056243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51dffff700 1 -- 172.21.15.181:0/1615056243 >> v1:172.21.15.181:6789/0 conn(0x7f51e0161f40 legacy=0x7f51e00d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51dffff700 1 -- 172.21.15.181:0/1615056243 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00d45f0 con 0x7f51e0160d20 2023-10-16T11:01:06.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51dffff700 1 --2- 172.21.15.181:0/1615056243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e01611d0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009840 tx=0x7f51d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b81e21eca424c5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:06.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51de7fc700 1 -- 172.21.15.181:0/1615056243 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8011830 con 0x7f51e0160d20 2023-10-16T11:01:06.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51de7fc700 1 -- 172.21.15.181:0/1615056243 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8011990 con 0x7f51e0160d20 2023-10-16T11:01:06.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.445+0000 7f51de7fc700 1 -- 172.21.15.181:0/1615056243 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8011d20 con 0x7f51e0160d20 2023-10-16T11:01:06.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 -- 172.21.15.181:0/1615056243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 msgr2=0x7f51e01611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/1615056243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e01611d0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009840 tx=0x7f51d8009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:06.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 -- 172.21.15.181:0/1615056243 shutdown_connections 2023-10-16T11:01:06.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/1615056243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e01611d0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/1615056243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 -- 172.21.15.181:0/1615056243 >> 172.21.15.181:0/1615056243 conn(0x7f51e009cba0 msgr2=0x7f51e00d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:06.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 -- 172.21.15.181:0/1615056243 shutdown_connections 2023-10-16T11:01:06.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.446+0000 7f51e5ca8700 1 -- 172.21.15.181:0/1615056243 wait complete. 2023-10-16T11:01:06.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.447+0000 7f51e5ca8700 1 Processor -- start 2023-10-16T11:01:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.447+0000 7f51e5ca8700 1 -- start start 2023-10-16T11:01:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51e5ca8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51e5ca8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e00b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51e5ca8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e0161f40 0x7f51e00b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51e5ca8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b5710 con 0x7f51e0160d20 2023-10-16T11:01:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51e5ca8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b60e0 con 0x7f51e0161f40 2023-10-16T11:01:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51e5ca8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b4b00 con 0x7f51e0161b60 2023-10-16T11:01:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e014ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51444/0 (socket says 172.21.15.181:51444) 2023-10-16T11:01:06.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 -- 172.21.15.181:0/2741564324 learned_addr learned my addr 172.21.15.181:0/2741564324 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:06.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51deffd700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e00b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51dffff700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e0161f40 0x7f51e00b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 -- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 msgr2=0x7f51e00b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e00b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 -- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e0161f40 msgr2=0x7f51e00b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e0161f40 0x7f51e00b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 -- 172.21.15.181:0/2741564324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d80094a0 con 0x7f51e0160d20 2023-10-16T11:01:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.448+0000 7f51df7fe700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e014ed50 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f51d000efc0 tx=0x7f51d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.449+0000 7f51c7fff700 1 -- 172.21.15.181:0/2741564324 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000cd10 con 0x7f51e0160d20 2023-10-16T11:01:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.449+0000 7f51c7fff700 1 -- 172.21.15.181:0/2741564324 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d000f040 con 0x7f51e0160d20 2023-10-16T11:01:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.449+0000 7f51c7fff700 1 -- 172.21.15.181:0/2741564324 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00146a0 con 0x7f51e0160d20 2023-10-16T11:01:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.449+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e00b4de0 con 0x7f51e0160d20 2023-10-16T11:01:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.449+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e00b53a0 con 0x7f51e0160d20 2023-10-16T11:01:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.450+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e00b59f0 con 0x7f51e0160d20 2023-10-16T11:01:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.450+0000 7f51c7fff700 1 -- 172.21.15.181:0/2741564324 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f51d0014800 con 0x7f51e0160d20 2023-10-16T11:01:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.450+0000 7f51c7fff700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 0x7f51cc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.450+0000 7f51deffd700 1 -- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 msgr2=0x7f51cc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.451+0000 7f51c7fff700 1 -- 172.21.15.181:0/2741564324 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f51d008a520 con 0x7f51e0160d20 2023-10-16T11:01:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.451+0000 7f51deffd700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 0x7f51cc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.455+0000 7f51c7fff700 1 -- 172.21.15.181:0/2741564324 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f51d00551a0 con 0x7f51e0160d20 2023-10-16T11:01:06.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.652+0000 7f51deffd700 1 -- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 msgr2=0x7f51cc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:06.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.652+0000 7f51deffd700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 0x7f51cc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:06.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.695+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51e00b5530 con 0x7f51e0160d20 2023-10-16T11:01:06.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.695+0000 7f51c7fff700 1 -- 172.21.15.181:0/2741564324 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f51d0009c70 con 0x7f51e0160d20 2023-10-16T11:01:06.697 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:06.697 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1460537182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1460537182},{"type":"v1","addr":"172.21.15.190:6833","nonce":1460537182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3626960628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3626960628},{"type":"v1","addr":"172.21.15.190:6835","nonce":3626960628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1772227629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1772227629},{"type":"v1","addr":"172.21.15.181:6836","nonce":1772227629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:00:46.276243+0000","modified":"2023-10-16T11:00:49.298812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9651},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9651":{"gid":9651,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6837/2931073086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2931073086},{"type":"v1","addr":"172.21.15.181:6837","nonce":2931073086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:01:06.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.697+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 msgr2=0x7f51cc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:06.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.697+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 0x7f51cc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.697+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 msgr2=0x7f51e014ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e014ed50 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f51d000efc0 tx=0x7f51d000c3a0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 shutdown_connections 2023-10-16T11:01:06.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f51cc06b020 0x7f51cc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e0161f40 0x7f51e00b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51e0160d20 0x7f51e014ed50 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 --2- 172.21.15.181:0/2741564324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0161b60 0x7f51e00b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 >> 172.21.15.181:0/2741564324 conn(0x7f51e009cba0 msgr2=0x7f51e015ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:06.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 shutdown_connections 2023-10-16T11:01:06.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.698+0000 7f51e5ca8700 1 -- 172.21.15.181:0/2741564324 wait complete. 2023-10-16T11:01:06.701 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2023-10-16T11:01:06.711 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:06.770 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:06.769+0000 7f5bb86bb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.837+0000 7f00045d5700 1 Processor -- start 2023-10-16T11:01:06.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.837+0000 7f00045d5700 1 -- start start 2023-10-16T11:01:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f00045d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efffc06d5f0 0x7efffc0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f00045d5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc0d2560 con 0x7efffc160d20 2023-10-16T11:01:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f00045d5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc06ce20 con 0x7efffc161b60 2023-10-16T11:01:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f00045d5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc162800 con 0x7efffc06d5f0 2023-10-16T11:01:06.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0001b70700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efffc06d5f0 0x7efffc0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0001b70700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efffc06d5f0 0x7efffc0d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51456/0 (socket says 172.21.15.181:51456) 2023-10-16T11:01:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0001b70700 1 -- 172.21.15.181:0/4159133698 learned_addr learned my addr 172.21.15.181:0/4159133698 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0002b72700 1 --1- 172.21.15.181:0/4159133698 >> v1:172.21.15.181:6789/0 conn(0x7efffc160d20 0x7efffc0b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34304/0 (socket says 172.21.15.181:34304) 2023-10-16T11:01:06.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0001b70700 1 -- 172.21.15.181:0/4159133698 >> v1:172.21.15.190:6789/0 conn(0x7efffc161b60 legacy=0x7efffc161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0001b70700 1 -- 172.21.15.181:0/4159133698 >> v1:172.21.15.181:6789/0 conn(0x7efffc160d20 legacy=0x7efffc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0001b70700 1 -- 172.21.15.181:0/4159133698 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0d45f0 con 0x7efffc06d5f0 2023-10-16T11:01:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f0001b70700 1 --2- 172.21.15.181:0/4159133698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc06d5f0 0x7efffc0d2020 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7effec0098f0 tx=0x7effec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacb27a094053eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f000136f700 1 -- 172.21.15.181:0/4159133698 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec00b8a0 con 0x7efffc06d5f0 2023-10-16T11:01:06.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f000136f700 1 -- 172.21.15.181:0/4159133698 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec00ba00 con 0x7efffc06d5f0 2023-10-16T11:01:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.839+0000 7f000136f700 1 -- 172.21.15.181:0/4159133698 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec00bd90 con 0x7efffc06d5f0 2023-10-16T11:01:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.840+0000 7f00045d5700 1 -- 172.21.15.181:0/4159133698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc06d5f0 msgr2=0x7efffc0d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.840+0000 7f00045d5700 1 --2- 172.21.15.181:0/4159133698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc06d5f0 0x7efffc0d2020 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7effec0098f0 tx=0x7effec009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.840+0000 7f00045d5700 1 -- 172.21.15.181:0/4159133698 shutdown_connections 2023-10-16T11:01:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.840+0000 7f00045d5700 1 --2- 172.21.15.181:0/4159133698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc06d5f0 0x7efffc0d2020 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.840+0000 7f00045d5700 1 -- 172.21.15.181:0/4159133698 >> 172.21.15.181:0/4159133698 conn(0x7efffc09cba0 msgr2=0x7efffc0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.840+0000 7f00045d5700 1 -- 172.21.15.181:0/4159133698 shutdown_connections 2023-10-16T11:01:06.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.840+0000 7f00045d5700 1 -- 172.21.15.181:0/4159133698 wait complete. 2023-10-16T11:01:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.842+0000 7f00045d5700 1 Processor -- start 2023-10-16T11:01:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.842+0000 7f00045d5700 1 -- start start 2023-10-16T11:01:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efffc06d5f0 0x7efffc14dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc160d20 0x7efffc0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efffc161b60 0x7efffc0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0b5b30 con 0x7efffc160d20 2023-10-16T11:01:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc14e180 con 0x7efffc06d5f0 2023-10-16T11:01:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc0b4f20 con 0x7efffc161b60 2023-10-16T11:01:06.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc160d20 0x7efffc0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc160d20 0x7efffc0b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51462/0 (socket says 172.21.15.181:51462) 2023-10-16T11:01:06.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 -- 172.21.15.181:0/4095441875 learned_addr learned my addr 172.21.15.181:0/4095441875 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:06.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0002371700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efffc06d5f0 0x7efffc14dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0002b72700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efffc161b60 0x7efffc0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:06.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 -- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efffc161b60 msgr2=0x7efffc0b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efffc161b60 0x7efffc0b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 -- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efffc06d5f0 msgr2=0x7efffc14dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:06.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efffc06d5f0 0x7efffc14dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:06.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 -- 172.21.15.181:0/4095441875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec009550 con 0x7efffc160d20 2023-10-16T11:01:06.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f0001b70700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc160d20 0x7efffc0b6420 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7effec009130 tx=0x7effec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7efff2ffd700 1 -- 172.21.15.181:0/4095441875 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0036a0 con 0x7efffc160d20 2023-10-16T11:01:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7efff2ffd700 1 -- 172.21.15.181:0/4095441875 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec01e070 con 0x7efffc160d20 2023-10-16T11:01:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc0b5200 con 0x7efffc160d20 2023-10-16T11:01:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7efff2ffd700 1 -- 172.21.15.181:0/4095441875 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec003a40 con 0x7efffc160d20 2023-10-16T11:01:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.843+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0b57c0 con 0x7efffc160d20 2023-10-16T11:01:06.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.844+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc0caff0 con 0x7efffc160d20 2023-10-16T11:01:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.845+0000 7efff2ffd700 1 -- 172.21.15.181:0/4095441875 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7effec003ba0 con 0x7efffc160d20 2023-10-16T11:01:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.846+0000 7efff2ffd700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 0x7effe006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.846+0000 7f0002371700 1 -- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 msgr2=0x7effe006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.846+0000 7efff2ffd700 1 -- 172.21.15.181:0/4095441875 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7effec020070 con 0x7efffc160d20 2023-10-16T11:01:06.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.846+0000 7f0002371700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 0x7effe006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:06.854 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:06.846+0000 7f5bb86bb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:01:06.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:06.850+0000 7efff2ffd700 1 -- 172.21.15.181:0/4095441875 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efffc0caff0 con 0x7efffc160d20 2023-10-16T11:01:06.912 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:06.912+0000 7f42d8c9c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:06.993 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:06.992+0000 7f5bb86bb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:06.994 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:06.993+0000 7f42d8c9c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:01:07.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.047+0000 7f0002371700 1 -- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 msgr2=0x7effe006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:07.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.047+0000 7f0002371700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 0x7effe006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:07.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.054+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efffc0caff0 con 0x7efffc160d20 2023-10-16T11:01:07.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.055+0000 7efff2ffd700 1 -- 172.21.15.181:0/4095441875 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17880 (secure 0 0 0) 0x7efffc0caff0 con 0x7efffc160d20 2023-10-16T11:01:07.056 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:07.057 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":169,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:00.993337+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:00:44.693955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:00:45.236607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/642481654":"2023-10-17T11:01:00.993294+0000","172.21.15.181:6832/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/120971111":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/3800812080":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/2485734134":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/4030419984":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/1244048650":"2023-10-17T11:01:00.993294+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:07.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 msgr2=0x7effe006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:07.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 0x7effe006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc160d20 msgr2=0x7efffc0b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc160d20 0x7efffc0b6420 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7effec009130 tx=0x7effec004750 comp rx=0 tx=0).stop 2023-10-16T11:01:07.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 shutdown_connections 2023-10-16T11:01:07.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efffc161b60 0x7efffc0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7effe006b020 0x7effe006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efffc06d5f0 0x7efffc14dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 --2- 172.21.15.181:0/4095441875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efffc160d20 0x7efffc0b6420 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 >> 172.21.15.181:0/4095441875 conn(0x7efffc09cba0 msgr2=0x7efffc073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:07.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 shutdown_connections 2023-10-16T11:01:07.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.058+0000 7f00045d5700 1 -- 172.21.15.181:0/4095441875 wait complete. 2023-10-16T11:01:07.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:00:44.693955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:00:45.236607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:01:07.072 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:01:07.151 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:07.151+0000 7f42d8c9c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.197+0000 7f60690bf700 1 Processor -- start 2023-10-16T11:01:07.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.197+0000 7f60690bf700 1 -- start start 2023-10-16T11:01:07.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f60690bf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f606415da70 0x7f60640ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f60690bf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f606406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f60690bf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60640cef00 con 0x7f6064072760 2023-10-16T11:01:07.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f60690bf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60640cec20 con 0x7f606415da70 2023-10-16T11:01:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f60690bf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6064071c80 con 0x7f60640ce6a0 2023-10-16T11:01:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f606415da70 0x7f60640ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f606415da70 0x7f60640ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51492/0 (socket says 172.21.15.181:51492) 2023-10-16T11:01:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 -- 172.21.15.181:0/1667559120 learned_addr learned my addr 172.21.15.181:0/1667559120 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:07.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f606259c700 1 --2- 172.21.15.181:0/1667559120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f606406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 -- 172.21.15.181:0/1667559120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 msgr2=0x7f606406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 --2- 172.21.15.181:0/1667559120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f606406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 -- 172.21.15.181:0/1667559120 >> v1:172.21.15.190:6789/0 conn(0x7f6064072760 legacy=0x7f60641615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 -- 172.21.15.181:0/1667559120 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6064071f60 con 0x7f606415da70 2023-10-16T11:01:07.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.198+0000 7f6062d9d700 1 --2- 172.21.15.181:0/1667559120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640ce160 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f60580098f0 tx=0x7f6058009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6ebe17234ac12d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f6061d9b700 1 -- 172.21.15.181:0/1667559120 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058011830 con 0x7f606415da70 2023-10-16T11:01:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f6061d9b700 1 -- 172.21.15.181:0/1667559120 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6058011990 con 0x7f606415da70 2023-10-16T11:01:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f6061d9b700 1 -- 172.21.15.181:0/1667559120 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058011d20 con 0x7f606415da70 2023-10-16T11:01:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 -- 172.21.15.181:0/1667559120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 msgr2=0x7f60640ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 --2- 172.21.15.181:0/1667559120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640ce160 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f60580098f0 tx=0x7f6058009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 -- 172.21.15.181:0/1667559120 shutdown_connections 2023-10-16T11:01:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 --2- 172.21.15.181:0/1667559120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640ce160 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 --2- 172.21.15.181:0/1667559120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f606406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 -- 172.21.15.181:0/1667559120 >> 172.21.15.181:0/1667559120 conn(0x7f60640cd010 msgr2=0x7f60640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:07.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 -- 172.21.15.181:0/1667559120 shutdown_connections 2023-10-16T11:01:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.199+0000 7f60690bf700 1 -- 172.21.15.181:0/1667559120 wait complete. 2023-10-16T11:01:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 Processor -- start 2023-10-16T11:01:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 -- start start 2023-10-16T11:01:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6064072760 0x7f60640b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f6064154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6064154ac0 con 0x7f606415da70 2023-10-16T11:01:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60641555d0 con 0x7f6064072760 2023-10-16T11:01:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.201+0000 7f60690bf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60641547e0 con 0x7f60640ce6a0 2023-10-16T11:01:07.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f6062d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6064072760 0x7f60640b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51502/0 (socket says 172.21.15.181:51502) 2023-10-16T11:01:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 -- 172.21.15.181:0/1048257142 learned_addr learned my addr 172.21.15.181:0/1048257142 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606259c700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f6064154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 -- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 msgr2=0x7f6064154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f6064154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 -- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6064072760 msgr2=0x7f60640b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6064072760 0x7f60640b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 -- 172.21.15.181:0/1048257142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058009550 con 0x7f606415da70 2023-10-16T11:01:07.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f6062d9d700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6064072760 0x7f60640b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:07.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f606359e700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640b5980 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f605c00cb50 tx=0x7f605c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:07.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f604b7fe700 1 -- 172.21.15.181:0/1048257142 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c003c00 con 0x7f606415da70 2023-10-16T11:01:07.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f604b7fe700 1 -- 172.21.15.181:0/1048257142 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c003d60 con 0x7f606415da70 2023-10-16T11:01:07.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606415e170 con 0x7f606415da70 2023-10-16T11:01:07.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f604b7fe700 1 -- 172.21.15.181:0/1048257142 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c00a9c0 con 0x7f606415da70 2023-10-16T11:01:07.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.202+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606415e730 con 0x7f606415da70 2023-10-16T11:01:07.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.203+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606406f6c0 con 0x7f606415da70 2023-10-16T11:01:07.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.204+0000 7f604b7fe700 1 -- 172.21.15.181:0/1048257142 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f605c00ab20 con 0x7f606415da70 2023-10-16T11:01:07.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.204+0000 7f604b7fe700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 0x7f605006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.204+0000 7f6062d9d700 1 -- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 msgr2=0x7f605006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:07.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.204+0000 7f604b7fe700 1 -- 172.21.15.181:0/1048257142 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f605c0899b0 con 0x7f606415da70 2023-10-16T11:01:07.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.204+0000 7f6062d9d700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 0x7f605006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:07.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.209+0000 7f604b7fe700 1 -- 172.21.15.181:0/1048257142 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f605c054600 con 0x7f606415da70 2023-10-16T11:01:07.222 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:07.221+0000 7f5bb86bb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:01:07.297 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:07.295+0000 7f5bb86bb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:01:07.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.405+0000 7f6062d9d700 1 -- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 msgr2=0x7f605006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:07.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.405+0000 7f6062d9d700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 0x7f605006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:07.409 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:07.408+0000 7f42d8c9c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:01:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.430+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60640cca60 con 0x7f606415da70 2023-10-16T11:01:07.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.430+0000 7f604b7fe700 1 -- 172.21.15.181:0/1048257142 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f605c057c20 con 0x7f606415da70 2023-10-16T11:01:07.432 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:01:07.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 msgr2=0x7f605006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:07.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 0x7f605006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 msgr2=0x7f60640b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640b5980 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f605c00cb50 tx=0x7f605c00ce60 comp rx=0 tx=0).stop 2023-10-16T11:01:07.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 shutdown_connections 2023-10-16T11:01:07.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f605006b020 0x7f605006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f606415da70 0x7f60640b5980 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6064072760 0x7f60640b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 --2- 172.21.15.181:0/1048257142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60640ce6a0 0x7f6064154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 >> 172.21.15.181:0/1048257142 conn(0x7f60640cd010 msgr2=0x7f606406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:07.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 shutdown_connections 2023-10-16T11:01:07.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.433+0000 7f60690bf700 1 -- 172.21.15.181:0/1048257142 wait complete. 2023-10-16T11:01:07.446 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:01:07.491 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:07.490+0000 7f42d8c9c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:01:07.563 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:07.561+0000 7f5bb86bb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:01:07.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.575+0000 7f2967a8e700 1 Processor -- start 2023-10-16T11:01:07.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.575+0000 7f2967a8e700 1 -- start start 2023-10-16T11:01:07.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f2967a8e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2960071be0 0x7f29600ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f2967a8e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600ce1f0 con 0x7f29600b2010 2023-10-16T11:01:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f2967a8e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600ccda0 con 0x7f29600cc6e0 2023-10-16T11:01:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f2967a8e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29600ccb90 con 0x7f2960071be0 2023-10-16T11:01:07.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f296582a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f29600b2010 0x7f29600b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34354/0 (socket says 172.21.15.181:34354) 2023-10-16T11:01:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f296582a700 1 -- 172.21.15.181:0/1458836295 learned_addr learned my addr 172.21.15.181:0/1458836295 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f296602b700 1 --2- 172.21.15.181:0/1458836295 >> v2:172.21.15.190:3300/0 conn(0x7f2960071be0 0x7f29600ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f2964828700 1 -- 172.21.15.181:0/1458836295 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457962763 0 0) 0x7f29600ce1f0 con 0x7f29600b2010 2023-10-16T11:01:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.576+0000 7f2964828700 1 -- 172.21.15.181:0/1458836295 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c0035b0 con 0x7f29600b2010 2023-10-16T11:01:07.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.577+0000 7f296602b700 1 -- 172.21.15.181:0/1458836295 >> v1:172.21.15.190:6789/0 conn(0x7f29600cc6e0 legacy=0x7f29600ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.577+0000 7f296602b700 1 -- 172.21.15.181:0/1458836295 >> v1:172.21.15.181:6789/0 conn(0x7f29600b2010 legacy=0x7f29600b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.577+0000 7f296602b700 1 -- 172.21.15.181:0/1458836295 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29600727e0 con 0x7f2960071be0 2023-10-16T11:01:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.577+0000 7f296602b700 1 --2- 172.21.15.181:0/1458836295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2960071be0 0x7f29600ced90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2958009d90 tx=0x7f295800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c545ae3d2e50ef79 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.577+0000 7f2964828700 1 -- 172.21.15.181:0/1458836295 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295800e070 con 0x7f2960071be0 2023-10-16T11:01:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.577+0000 7f2964828700 1 -- 172.21.15.181:0/1458836295 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295800bb20 con 0x7f2960071be0 2023-10-16T11:01:07.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.577+0000 7f2964828700 1 -- 172.21.15.181:0/1458836295 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958012430 con 0x7f2960071be0 2023-10-16T11:01:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.578+0000 7f2967a8e700 1 -- 172.21.15.181:0/1458836295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2960071be0 msgr2=0x7f29600ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.578+0000 7f2967a8e700 1 --2- 172.21.15.181:0/1458836295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2960071be0 0x7f29600ced90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2958009d90 tx=0x7f295800b330 comp rx=0 tx=0).stop 2023-10-16T11:01:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.578+0000 7f2967a8e700 1 -- 172.21.15.181:0/1458836295 shutdown_connections 2023-10-16T11:01:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.578+0000 7f2967a8e700 1 --2- 172.21.15.181:0/1458836295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2960071be0 0x7f29600ced90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.578+0000 7f2967a8e700 1 -- 172.21.15.181:0/1458836295 >> 172.21.15.181:0/1458836295 conn(0x7f29600cd010 msgr2=0x7f29600cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.578+0000 7f2967a8e700 1 -- 172.21.15.181:0/1458836295 shutdown_connections 2023-10-16T11:01:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.578+0000 7f2967a8e700 1 -- 172.21.15.181:0/1458836295 wait complete. 2023-10-16T11:01:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.579+0000 7f2967a8e700 1 Processor -- start 2023-10-16T11:01:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.579+0000 7f2967a8e700 1 -- start start 2023-10-16T11:01:07.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f2967a8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2960071be0 0x7f2960155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f2967a8e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b2010 0x7f29601f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f2967a8e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cc6e0 0x7f29600c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f2967a8e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29601563e0 con 0x7f2960071be0 2023-10-16T11:01:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f2967a8e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29601f0cd0 con 0x7f29600cc6e0 2023-10-16T11:01:07.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f2967a8e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29600ce9f0 con 0x7f29600b2010 2023-10-16T11:01:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2960071be0 0x7f2960155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2960071be0 0x7f2960155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51512/0 (socket says 172.21.15.181:51512) 2023-10-16T11:01:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 -- 172.21.15.181:0/958781828 learned_addr learned my addr 172.21.15.181:0/958781828 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296602b700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cc6e0 0x7f29600c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f2965029700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b2010 0x7f29601f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 -- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b2010 msgr2=0x7f29601f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b2010 0x7f29601f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 -- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cc6e0 msgr2=0x7f29600c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cc6e0 0x7f29600c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 -- 172.21.15.181:0/958781828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958009a70 con 0x7f2960071be0 2023-10-16T11:01:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296602b700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cc6e0 0x7f29600c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.580+0000 7f296582a700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2960071be0 0x7f2960155ea0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f295c001eb0 tx=0x7f295c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.581+0000 7f29567fc700 1 -- 172.21.15.181:0/958781828 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c005d70 con 0x7f2960071be0 2023-10-16T11:01:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.581+0000 7f29567fc700 1 -- 172.21.15.181:0/958781828 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c0070d0 con 0x7f2960071be0 2023-10-16T11:01:07.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.581+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29600cecd0 con 0x7f2960071be0 2023-10-16T11:01:07.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.581+0000 7f29567fc700 1 -- 172.21.15.181:0/958781828 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c007460 con 0x7f2960071be0 2023-10-16T11:01:07.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.581+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29600cf290 con 0x7f2960071be0 2023-10-16T11:01:07.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.582+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29600d1490 con 0x7f2960071be0 2023-10-16T11:01:07.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.582+0000 7f29567fc700 1 -- 172.21.15.181:0/958781828 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f295c0075c0 con 0x7f2960071be0 2023-10-16T11:01:07.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.583+0000 7f29567fc700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 0x7f294406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.583+0000 7f2965029700 1 -- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 msgr2=0x7f294406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:07.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.583+0000 7f29567fc700 1 -- 172.21.15.181:0/958781828 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f295c088ea0 con 0x7f2960071be0 2023-10-16T11:01:07.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.583+0000 7f2965029700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 0x7f294406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:07.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.587+0000 7f29567fc700 1 -- 172.21.15.181:0/958781828 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f295c053af0 con 0x7f2960071be0 2023-10-16T11:01:07.767 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:07.767+0000 7f42d8c9c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:01:07.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.784+0000 7f2965029700 1 -- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 msgr2=0x7f294406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:07.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.784+0000 7f2965029700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 0x7f294406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:07.785 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:07.784+0000 7f5bb86bb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:01:07.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.787+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29600b6510 con 0x7f2960071be0 2023-10-16T11:01:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.787+0000 7f29567fc700 1 -- 172.21.15.181:0/958781828 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v169) v1 ==== 104+0+11 (secure 0 0 0) 0x7f295c017020 con 0x7f2960071be0 2023-10-16T11:01:07.788 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:01:07.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 msgr2=0x7f294406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:07.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 0x7f294406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2960071be0 msgr2=0x7f2960155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2960071be0 0x7f2960155ea0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f295c001eb0 tx=0x7f295c00c7b0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 shutdown_connections 2023-10-16T11:01:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f294406b020 0x7f294406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cc6e0 0x7f29600c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2960071be0 0x7f2960155ea0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 --2- 172.21.15.181:0/958781828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b2010 0x7f29601f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 >> 172.21.15.181:0/958781828 conn(0x7f29600cd010 msgr2=0x7f29600b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 shutdown_connections 2023-10-16T11:01:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.790+0000 7f2967a8e700 1 -- 172.21.15.181:0/958781828 wait complete. 2023-10-16T11:01:07.804 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:01:07.934 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:07.933+0000 7f5bb86bb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:01:07.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.941+0000 7fa884a6f700 1 Processor -- start 2023-10-16T11:01:07.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.942+0000 7fa884a6f700 1 -- start start 2023-10-16T11:01:07.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.942+0000 7fa884a6f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa88015f150 0x7fa8800cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.942+0000 7fa884a6f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa8800ce4f0 0x7fa8800b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.942+0000 7fa884a6f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8800b14a0 0x7fa8800b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa884a6f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800ce9a0 con 0x7fa88015f150 2023-10-16T11:01:07.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa884a6f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800b2ae0 con 0x7fa8800ce4f0 2023-10-16T11:01:07.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa884a6f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800b2dc0 con 0x7fa8800b14a0 2023-10-16T11:01:07.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa88015f150 0x7fa8800cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa88015f150 0x7fa8800cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51522/0 (socket says 172.21.15.181:51522) 2023-10-16T11:01:07.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 -- 172.21.15.181:0/1730587710 learned_addr learned my addr 172.21.15.181:0/1730587710 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:07.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87ed9d700 1 --2- 172.21.15.181:0/1730587710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8800b14a0 0x7fa8800b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87dd9b700 1 --2- 172.21.15.181:0/1730587710 >> v2:172.21.15.190:3300/0 conn(0x7fa8800ce4f0 0x7fa8800b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 -- 172.21.15.181:0/1730587710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8800b14a0 msgr2=0x7fa8800b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:07.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 --2- 172.21.15.181:0/1730587710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8800b14a0 0x7fa8800b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 -- 172.21.15.181:0/1730587710 >> v2:172.21.15.190:3300/0 conn(0x7fa8800ce4f0 msgr2=0x7fa8800b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:07.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 --2- 172.21.15.181:0/1730587710 >> v2:172.21.15.190:3300/0 conn(0x7fa8800ce4f0 0x7fa8800b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 -- 172.21.15.181:0/1730587710 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800cf9d0 con 0x7fa88015f150 2023-10-16T11:01:07.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87e59c700 1 --2- 172.21.15.181:0/1730587710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa88015f150 0x7fa8800cdfb0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fa8700098f0 tx=0x7fa870009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9c2d477a5213c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:07.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87d59a700 1 -- 172.21.15.181:0/1730587710 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870011830 con 0x7fa88015f150 2023-10-16T11:01:07.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87d59a700 1 -- 172.21.15.181:0/1730587710 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa870011990 con 0x7fa88015f150 2023-10-16T11:01:07.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.943+0000 7fa87d59a700 1 -- 172.21.15.181:0/1730587710 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870011d20 con 0x7fa88015f150 2023-10-16T11:01:07.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 -- 172.21.15.181:0/1730587710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa88015f150 msgr2=0x7fa8800cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1730587710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa88015f150 0x7fa8800cdfb0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fa8700098f0 tx=0x7fa870009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:07.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 -- 172.21.15.181:0/1730587710 shutdown_connections 2023-10-16T11:01:07.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1730587710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa88015f150 0x7fa8800cdfb0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1730587710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8800b14a0 0x7fa8800b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1730587710 >> v2:172.21.15.190:3300/0 conn(0x7fa8800ce4f0 0x7fa8800b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 -- 172.21.15.181:0/1730587710 >> 172.21.15.181:0/1730587710 conn(0x7fa88009a930 msgr2=0x7fa8800d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:07.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 -- 172.21.15.181:0/1730587710 shutdown_connections 2023-10-16T11:01:07.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.944+0000 7fa884a6f700 1 -- 172.21.15.181:0/1730587710 wait complete. 2023-10-16T11:01:07.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.946+0000 7fa884a6f700 1 Processor -- start 2023-10-16T11:01:07.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.946+0000 7fa884a6f700 1 -- start start 2023-10-16T11:01:07.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa884a6f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8800b14a0 0x7fa880157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa884a6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8800ce4f0 0x7fa880159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa884a6f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa88015f150 0x7fa8800c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa884a6f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880156cc0 con 0x7fa8800ce4f0 2023-10-16T11:01:07.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa884a6f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880155c20 con 0x7fa8800b14a0 2023-10-16T11:01:07.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa884a6f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880155f00 con 0x7fa88015f150 2023-10-16T11:01:07.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8800ce4f0 0x7fa880159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8800ce4f0 0x7fa880159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51530/0 (socket says 172.21.15.181:51530) 2023-10-16T11:01:07.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 -- 172.21.15.181:0/1862372762 learned_addr learned my addr 172.21.15.181:0/1862372762 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:07.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87e59c700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8800b14a0 0x7fa880157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87ed9d700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa88015f150 0x7fa8800c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 -- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa88015f150 msgr2=0x7fa8800c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa88015f150 0x7fa8800c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 -- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8800b14a0 msgr2=0x7fa880157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8800b14a0 0x7fa880157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 -- 172.21.15.181:0/1862372762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009550 con 0x7fa8800ce4f0 2023-10-16T11:01:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.947+0000 7fa87dd9b700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8800ce4f0 0x7fa880159ea0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e9b0 tx=0x7fa87400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.948+0000 7fa86effd700 1 -- 172.21.15.181:0/1862372762 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874009710 con 0x7fa8800ce4f0 2023-10-16T11:01:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.948+0000 7fa86effd700 1 -- 172.21.15.181:0/1862372762 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87400ee50 con 0x7fa8800ce4f0 2023-10-16T11:01:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.948+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8801561e0 con 0x7fa8800ce4f0 2023-10-16T11:01:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.948+0000 7fa86effd700 1 -- 172.21.15.181:0/1862372762 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400ca50 con 0x7fa8800ce4f0 2023-10-16T11:01:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.948+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8801567a0 con 0x7fa8800ce4f0 2023-10-16T11:01:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.949+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa880067d20 con 0x7fa8800ce4f0 2023-10-16T11:01:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.949+0000 7fa86effd700 1 -- 172.21.15.181:0/1862372762 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa87400cbb0 con 0x7fa8800ce4f0 2023-10-16T11:01:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.950+0000 7fa86effd700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 0x7fa86006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.950+0000 7fa87e59c700 1 -- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 msgr2=0x7fa86006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:07.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.950+0000 7fa86effd700 1 -- 172.21.15.181:0/1862372762 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fa8740899f0 con 0x7fa8800ce4f0 2023-10-16T11:01:07.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.950+0000 7fa87e59c700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 0x7fa86006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:07.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:07.954+0000 7fa86effd700 1 -- 172.21.15.181:0/1862372762 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa8740546d0 con 0x7fa8800ce4f0 2023-10-16T11:01:08.019 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:08.018+0000 7f42d8c9c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:01:08.051 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:08.050+0000 7f5bb86bb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:01:08.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.151+0000 7fa87e59c700 1 -- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 msgr2=0x7fa86006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:08.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.151+0000 7fa87e59c700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 0x7fa86006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:08.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.153+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8800cda20 con 0x7fa8800ce4f0 2023-10-16T11:01:08.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.153+0000 7fa86effd700 1 -- 172.21.15.181:0/1862372762 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8740210a0 con 0x7fa8800ce4f0 2023-10-16T11:01:08.154 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:01:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 msgr2=0x7fa86006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 0x7fa86006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8800ce4f0 msgr2=0x7fa880159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8800ce4f0 0x7fa880159ea0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e9b0 tx=0x7fa87400ecc0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 shutdown_connections 2023-10-16T11:01:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa88015f150 0x7fa8800c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fa86006b020 0x7fa86006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8800b14a0 0x7fa880157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 --2- 172.21.15.181:0/1862372762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8800ce4f0 0x7fa880159ea0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 >> 172.21.15.181:0/1862372762 conn(0x7fa88009a930 msgr2=0x7fa8800ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 shutdown_connections 2023-10-16T11:01:08.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.156+0000 7fa884a6f700 1 -- 172.21.15.181:0/1862372762 wait complete. 2023-10-16T11:01:08.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:01:08.177 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:08.177+0000 7f42d8c9c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:01:08.207 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:08.206+0000 7f5bb86bb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:01:08.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.292+0000 7f86207c9700 1 Processor -- start 2023-10-16T11:01:08.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.292+0000 7f86207c9700 1 -- start start 2023-10-16T11:01:08.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f86207c9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86180d0370 0x7f86180ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f86207c9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86180cd2d0 0x7f86180d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f86207c9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86180cc510 con 0x7f861815b990 2023-10-16T11:01:08.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f86207c9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86180d24f0 con 0x7f86180d0370 2023-10-16T11:01:08.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f86207c9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86180cc7f0 con 0x7f86180cd2d0 2023-10-16T11:01:08.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861dd64700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86180d0370 0x7f86180ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861dd64700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86180d0370 0x7f86180ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51540/0 (socket says 172.21.15.181:51540) 2023-10-16T11:01:08.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861dd64700 1 -- 172.21.15.181:0/3564316480 learned_addr learned my addr 172.21.15.181:0/3564316480 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:08.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861ed66700 1 --2- 172.21.15.181:0/3564316480 >> v2:172.21.15.190:3300/0 conn(0x7f86180cd2d0 0x7f86180d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861dd64700 1 -- 172.21.15.181:0/3564316480 >> v2:172.21.15.190:3300/0 conn(0x7f86180cd2d0 msgr2=0x7f86180d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:08.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861dd64700 1 --2- 172.21.15.181:0/3564316480 >> v2:172.21.15.190:3300/0 conn(0x7f86180cd2d0 0x7f86180d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861dd64700 1 -- 172.21.15.181:0/3564316480 >> v1:172.21.15.181:6789/0 conn(0x7f861815b990 legacy=0x7f86180cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.293+0000 7f861dd64700 1 -- 172.21.15.181:0/3564316480 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180ccad0 con 0x7f86180d0370 2023-10-16T11:01:08.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.294+0000 7f861dd64700 1 --2- 172.21.15.181:0/3564316480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180d0370 0x7f86180ce8f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f86180cbc60 tx=0x7f8608009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9088069f62e79c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:08.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.294+0000 7f861d563700 1 -- 172.21.15.181:0/3564316480 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608011830 con 0x7f86180d0370 2023-10-16T11:01:08.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.294+0000 7f861d563700 1 -- 172.21.15.181:0/3564316480 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8608011990 con 0x7f86180d0370 2023-10-16T11:01:08.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.294+0000 7f861d563700 1 -- 172.21.15.181:0/3564316480 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608011d20 con 0x7f86180d0370 2023-10-16T11:01:08.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.294+0000 7f86207c9700 1 -- 172.21.15.181:0/3564316480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180d0370 msgr2=0x7f86180ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.294+0000 7f86207c9700 1 --2- 172.21.15.181:0/3564316480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180d0370 0x7f86180ce8f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f86180cbc60 tx=0x7f8608009bd0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.295+0000 7f86207c9700 1 -- 172.21.15.181:0/3564316480 shutdown_connections 2023-10-16T11:01:08.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.295+0000 7f86207c9700 1 --2- 172.21.15.181:0/3564316480 >> v2:172.21.15.190:3300/0 conn(0x7f86180cd2d0 0x7f86180d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.295+0000 7f86207c9700 1 --2- 172.21.15.181:0/3564316480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180d0370 0x7f86180ce8f0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.295+0000 7f86207c9700 1 -- 172.21.15.181:0/3564316480 >> 172.21.15.181:0/3564316480 conn(0x7f861809a9a0 msgr2=0x7f86180d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:08.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.295+0000 7f86207c9700 1 -- 172.21.15.181:0/3564316480 shutdown_connections 2023-10-16T11:01:08.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.295+0000 7f86207c9700 1 -- 172.21.15.181:0/3564316480 wait complete. 2023-10-16T11:01:08.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.296+0000 7f86207c9700 1 Processor -- start 2023-10-16T11:01:08.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.296+0000 7f86207c9700 1 -- start start 2023-10-16T11:01:08.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.296+0000 7f86207c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180cd2d0 0x7f86180c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.296+0000 7f86207c9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86180d0370 0x7f8618154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f86207c9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861815b990 0x7f861815e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f86207c9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180c7e30 con 0x7f86180cd2d0 2023-10-16T11:01:08.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f86207c9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180c7870 con 0x7f861815b990 2023-10-16T11:01:08.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f86207c9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86180c7b50 con 0x7f86180d0370 2023-10-16T11:01:08.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180cd2d0 0x7f86180c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180cd2d0 0x7f86180c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51544/0 (socket says 172.21.15.181:51544) 2023-10-16T11:01:08.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 -- 172.21.15.181:0/2836307819 learned_addr learned my addr 172.21.15.181:0/2836307819 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:08.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861dd64700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86180d0370 0x7f8618154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861ed66700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861815b990 0x7f861815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 -- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86180d0370 msgr2=0x7f8618154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86180d0370 0x7f8618154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 -- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861815b990 msgr2=0x7f861815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861815b990 0x7f861815e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 -- 172.21.15.181:0/2836307819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608009550 con 0x7f86180cd2d0 2023-10-16T11:01:08.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f861e565700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180cd2d0 0x7f86180c7040 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f861400c4d0 tx=0x7f861400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:08.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f860effd700 1 -- 172.21.15.181:0/2836307819 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86140054e0 con 0x7f86180cd2d0 2023-10-16T11:01:08.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f860effd700 1 -- 172.21.15.181:0/2836307819 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8614003680 con 0x7f86180cd2d0 2023-10-16T11:01:08.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86180b5420 con 0x7f86180cd2d0 2023-10-16T11:01:08.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f860effd700 1 -- 172.21.15.181:0/2836307819 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614003a30 con 0x7f86180cd2d0 2023-10-16T11:01:08.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.297+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86180c7580 con 0x7f86180cd2d0 2023-10-16T11:01:08.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.298+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86180b5940 con 0x7f86180cd2d0 2023-10-16T11:01:08.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.300+0000 7f860effd700 1 -- 172.21.15.181:0/2836307819 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8614003b90 con 0x7f86180cd2d0 2023-10-16T11:01:08.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.300+0000 7f860effd700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 0x7f85fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.300+0000 7f861dd64700 1 -- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 msgr2=0x7f85fc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:08.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.300+0000 7f860effd700 1 -- 172.21.15.181:0/2836307819 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f8614089000 con 0x7f86180cd2d0 2023-10-16T11:01:08.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.300+0000 7f861dd64700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 0x7f85fc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:08.311 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:08.301+0000 7f42d8c9c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:01:08.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.304+0000 7f860effd700 1 -- 172.21.15.181:0/2836307819 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8614053dc0 con 0x7f86180cd2d0 2023-10-16T11:01:08.466 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:08.466+0000 7f42d8c9c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:01:08.471 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:08.470+0000 7f5bb86bb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.501+0000 7f861dd64700 1 -- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 msgr2=0x7f85fc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.501+0000 7f861dd64700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 0x7f85fc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:08.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.549+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86180b5940 con 0x7f86180cd2d0 2023-10-16T11:01:08.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.550+0000 7f860effd700 1 -- 172.21.15.181:0/2836307819 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f86180b5940 con 0x7f86180cd2d0 2023-10-16T11:01:08.551 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:08.551 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1460537182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1460537182},{"type":"v1","addr":"172.21.15.190:6833","nonce":1460537182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3626960628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3626960628},{"type":"v1","addr":"172.21.15.190:6835","nonce":3626960628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1772227629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1772227629},{"type":"v1","addr":"172.21.15.181:6836","nonce":1772227629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:00:46.276243+0000","modified":"2023-10-16T11:00:49.298812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9651},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9651":{"gid":9651,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6837/2931073086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2931073086},{"type":"v1","addr":"172.21.15.181:6837","nonce":2931073086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:01:08.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 msgr2=0x7f85fc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:08.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 0x7f85fc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180cd2d0 msgr2=0x7f86180c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180cd2d0 0x7f86180c7040 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f861400c4d0 tx=0x7f861400c7e0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 shutdown_connections 2023-10-16T11:01:08.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f85fc06b020 0x7f85fc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f861815b990 0x7f861815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180cd2d0 0x7f86180c7040 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 --2- 172.21.15.181:0/2836307819 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86180d0370 0x7f8618154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.552+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 >> 172.21.15.181:0/2836307819 conn(0x7f861809a9a0 msgr2=0x7f86180d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:08.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.553+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 shutdown_connections 2023-10-16T11:01:08.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.553+0000 7f86207c9700 1 -- 172.21.15.181:0/2836307819 wait complete. 2023-10-16T11:01:08.556 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2023-10-16T11:01:08.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:08.631 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:08.630+0000 7f5bb86bb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:01:08.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.730+0000 7fb8f1baf700 1 Processor -- start 2023-10-16T11:01:08.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.730+0000 7fb8f1baf700 1 -- start start 2023-10-16T11:01:08.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.730+0000 7fb8f1baf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8ec15d9b0 0x7fb8ec0b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.730+0000 7fb8f1baf700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb8ec0cf860 0x7fb8ec0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.730+0000 7fb8f1baf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0d1640 con 0x7fb8ec1621c0 2023-10-16T11:01:08.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.730+0000 7fb8f1baf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec0d1920 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.730+0000 7fb8f1baf700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec0d1c00 con 0x7fb8ec0cf860 2023-10-16T11:01:08.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8ec15d9b0 0x7fb8ec0b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8ec15d9b0 0x7fb8ec0b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51554/0 (socket says 172.21.15.181:51554) 2023-10-16T11:01:08.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 -- 172.21.15.181:0/2389392404 learned_addr learned my addr 172.21.15.181:0/2389392404 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:08.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8ebfff700 1 --2- 172.21.15.181:0/2389392404 >> v2:172.21.15.190:3300/0 conn(0x7fb8ec0cf860 0x7fb8ec0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 -- 172.21.15.181:0/2389392404 >> v2:172.21.15.190:3300/0 conn(0x7fb8ec0cf860 msgr2=0x7fb8ec0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:08.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 --2- 172.21.15.181:0/2389392404 >> v2:172.21.15.190:3300/0 conn(0x7fb8ec0cf860 0x7fb8ec0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 -- 172.21.15.181:0/2389392404 >> v1:172.21.15.190:6789/0 conn(0x7fb8ec1621c0 legacy=0x7fb8ec0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 -- 172.21.15.181:0/2389392404 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0d1ee0 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8eb7fe700 1 --2- 172.21.15.181:0/2389392404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b39d0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0d0320 tx=0x7fb8dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52eb174744dfacd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:08.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8ea7fc700 1 -- 172.21.15.181:0/2389392404 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00e070 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8ea7fc700 1 -- 172.21.15.181:0/2389392404 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00b910 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.731+0000 7fb8ea7fc700 1 -- 172.21.15.181:0/2389392404 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00bcc0 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 -- 172.21.15.181:0/2389392404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 msgr2=0x7fb8ec0b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/2389392404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b39d0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0d0320 tx=0x7fb8dc00b330 comp rx=0 tx=0).stop 2023-10-16T11:01:08.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 -- 172.21.15.181:0/2389392404 shutdown_connections 2023-10-16T11:01:08.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/2389392404 >> v2:172.21.15.190:3300/0 conn(0x7fb8ec0cf860 0x7fb8ec0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/2389392404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b39d0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 -- 172.21.15.181:0/2389392404 >> 172.21.15.181:0/2389392404 conn(0x7fb8ec0665a0 msgr2=0x7fb8ec0669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:08.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 -- 172.21.15.181:0/2389392404 shutdown_connections 2023-10-16T11:01:08.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.732+0000 7fb8f1baf700 1 -- 172.21.15.181:0/2389392404 wait complete. 2023-10-16T11:01:08.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.733+0000 7fb8f1baf700 1 Processor -- start 2023-10-16T11:01:08.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.734+0000 7fb8f1baf700 1 -- start start 2023-10-16T11:01:08.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8f1baf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8ec0cf860 0x7fb8ec0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8f1baf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8f1baf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8ec1621c0 0x7fb8ec0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8f1baf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec15e000 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8f1baf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec14c6a0 con 0x7fb8ec0cf860 2023-10-16T11:01:08.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8f1baf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec14c980 con 0x7fb8ec1621c0 2023-10-16T11:01:08.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51562/0 (socket says 172.21.15.181:51562) 2023-10-16T11:01:08.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eb7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8ec0cf860 0x7fb8ec0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 -- 172.21.15.181:0/3690948400 learned_addr learned my addr 172.21.15.181:0/3690948400 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:08.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8ebfff700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8ec1621c0 0x7fb8ec0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:08.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 -- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8ec1621c0 msgr2=0x7fb8ec0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8ec1621c0 0x7fb8ec0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 -- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8ec0cf860 msgr2=0x7fb8ec0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8ec0cf860 0x7fb8ec0c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 -- 172.21.15.181:0/3690948400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc009b30 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eb7fe700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8ec0cf860 0x7fb8ec0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:08.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8ebfff700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8ec1621c0 0x7fb8ec0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:08.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8eaffd700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b5970 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c430 tx=0x7fb8e000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:08.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8d3fff700 1 -- 172.21.15.181:0/3690948400 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00050d0 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8d3fff700 1 -- 172.21.15.181:0/3690948400 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e0005250 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec14cc60 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.735+0000 7fb8d3fff700 1 -- 172.21.15.181:0/3690948400 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0006860 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.736+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec0c56b0 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.736+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec04ebb0 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.737+0000 7fb8d3fff700 1 -- 172.21.15.181:0/3690948400 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb8e0003680 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.737+0000 7fb8d3fff700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 0x7fb8d806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:08.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.737+0000 7fb8eb7fe700 1 -- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 msgr2=0x7fb8d806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:08.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.737+0000 7fb8d3fff700 1 -- 172.21.15.181:0/3690948400 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb8e0088b80 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.737+0000 7fb8eb7fe700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 0x7fb8d806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:08.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.742+0000 7fb8d3fff700 1 -- 172.21.15.181:0/3690948400 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb8e00537d0 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.748 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:08.745+0000 7f42d8c9c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:08.917 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:08.916+0000 7f42d8c9c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:01:08.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.938+0000 7fb8eb7fe700 1 -- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 msgr2=0x7fb8d806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:08.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.938+0000 7fb8eb7fe700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 0x7fb8d806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:08.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.944+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8ec0b7500 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.945+0000 7fb8d3fff700 1 -- 172.21.15.181:0/3690948400 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17880 (secure 0 0 0) 0x7fb8e0056df0 con 0x7fb8ec15d9b0 2023-10-16T11:01:08.947 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:08.947 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":169,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:00.993337+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:00:44.693955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:00:45.236607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/642481654":"2023-10-17T11:01:00.993294+0000","172.21.15.181:6832/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/120971111":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/3800812080":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/2485734134":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/4030419984":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/1244048650":"2023-10-17T11:01:00.993294+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:08.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 msgr2=0x7fb8d806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:08.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 0x7fb8d806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 msgr2=0x7fb8ec0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:08.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b5970 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c430 tx=0x7fb8e000c740 comp rx=0 tx=0).stop 2023-10-16T11:01:08.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 shutdown_connections 2023-10-16T11:01:08.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8ec1621c0 0x7fb8ec0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb8d806b020 0x7fb8d806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8ec0cf860 0x7fb8ec0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 --2- 172.21.15.181:0/3690948400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8ec15d9b0 0x7fb8ec0b5970 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:08.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 >> 172.21.15.181:0/3690948400 conn(0x7fb8ec0665a0 msgr2=0x7fb8ec15b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:08.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 shutdown_connections 2023-10-16T11:01:08.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:08.949+0000 7fb8f1baf700 1 -- 172.21.15.181:0/3690948400 wait complete. 2023-10-16T11:01:08.967 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:01:08.968 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:01:09.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.097+0000 7fb50849b700 1 Processor -- start 2023-10-16T11:01:09.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.097+0000 7fb50849b700 1 -- start start 2023-10-16T11:01:09.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.097+0000 7fb50849b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb50015db70 0x7fb5000665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.097+0000 7fb50849b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000ca2e0 0x7fb5000ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.097+0000 7fb50849b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb500066c60 con 0x7fb5000cedd0 2023-10-16T11:01:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.097+0000 7fb50849b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5000cda10 con 0x7fb50015db70 2023-10-16T11:01:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb50849b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50006c1a0 con 0x7fb5000ca2e0 2023-10-16T11:01:09.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb50015db70 0x7fb5000665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb50015db70 0x7fb5000665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51572/0 (socket says 172.21.15.181:51572) 2023-10-16T11:01:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 -- 172.21.15.181:0/524046275 learned_addr learned my addr 172.21.15.181:0/524046275 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506a38700 1 --2- 172.21.15.181:0/524046275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000ca2e0 0x7fb5000ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 -- 172.21.15.181:0/524046275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000ca2e0 msgr2=0x7fb5000ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:09.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 --2- 172.21.15.181:0/524046275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000ca2e0 0x7fb5000ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 -- 172.21.15.181:0/524046275 >> v1:172.21.15.190:6789/0 conn(0x7fb5000cedd0 legacy=0x7fb5000cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 -- 172.21.15.181:0/524046275 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50006b3e0 con 0x7fb50015db70 2023-10-16T11:01:09.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb506237700 1 --2- 172.21.15.181:0/524046275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb50015db70 0x7fb5000665a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0098f0 tx=0x7fb4fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4dc1de0cb25a56 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb505235700 1 -- 172.21.15.181:0/524046275 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc011830 con 0x7fb50015db70 2023-10-16T11:01:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb505235700 1 -- 172.21.15.181:0/524046275 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc011990 con 0x7fb50015db70 2023-10-16T11:01:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.098+0000 7fb505235700 1 -- 172.21.15.181:0/524046275 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc011d20 con 0x7fb50015db70 2023-10-16T11:01:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 -- 172.21.15.181:0/524046275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb50015db70 msgr2=0x7fb5000665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 --2- 172.21.15.181:0/524046275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb50015db70 0x7fb5000665a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0098f0 tx=0x7fb4fc009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:09.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 -- 172.21.15.181:0/524046275 shutdown_connections 2023-10-16T11:01:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 --2- 172.21.15.181:0/524046275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5000ca2e0 0x7fb5000ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 --2- 172.21.15.181:0/524046275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb50015db70 0x7fb5000665a0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 -- 172.21.15.181:0/524046275 >> 172.21.15.181:0/524046275 conn(0x7fb5000b1780 msgr2=0x7fb5000b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 -- 172.21.15.181:0/524046275 shutdown_connections 2023-10-16T11:01:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.099+0000 7fb50849b700 1 -- 172.21.15.181:0/524046275 wait complete. 2023-10-16T11:01:09.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.100+0000 7fb50849b700 1 Processor -- start 2023-10-16T11:01:09.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.100+0000 7fb50849b700 1 -- start start 2023-10-16T11:01:09.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb50849b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5000ca2e0 0x7fb500160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb50849b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5000cedd0 0x7fb50015df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb50849b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb50015db70 0x7fb50015e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb50849b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50015a330 con 0x7fb5000cedd0 2023-10-16T11:01:09.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb50849b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000c7930 con 0x7fb5000ca2e0 2023-10-16T11:01:09.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb50849b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000c6d60 con 0x7fb50015db70 2023-10-16T11:01:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5000cedd0 0x7fb50015df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5000cedd0 0x7fb50015df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51578/0 (socket says 172.21.15.181:51578) 2023-10-16T11:01:09.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 -- 172.21.15.181:0/4198913239 learned_addr learned my addr 172.21.15.181:0/4198913239 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:09.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb506a38700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb50015db70 0x7fb50015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb506237700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5000ca2e0 0x7fb500160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 -- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb50015db70 msgr2=0x7fb50015e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb50015db70 0x7fb50015e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 -- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5000ca2e0 msgr2=0x7fb500160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5000ca2e0 0x7fb500160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 -- 172.21.15.181:0/4198913239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc009550 con 0x7fb5000cedd0 2023-10-16T11:01:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.101+0000 7fb505a36700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5000cedd0 0x7fb50015df50 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400c4e0 tx=0x7fb4f400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.102+0000 7fb4f2ffd700 1 -- 172.21.15.181:0/4198913239 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40052d0 con 0x7fb5000cedd0 2023-10-16T11:01:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.102+0000 7fb4f2ffd700 1 -- 172.21.15.181:0/4198913239 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4005430 con 0x7fb5000cedd0 2023-10-16T11:01:09.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.102+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5000c7000 con 0x7fb5000cedd0 2023-10-16T11:01:09.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.102+0000 7fb4f2ffd700 1 -- 172.21.15.181:0/4198913239 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40038c0 con 0x7fb5000cedd0 2023-10-16T11:01:09.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.102+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5000c75c0 con 0x7fb5000cedd0 2023-10-16T11:01:09.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.103+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb500067d20 con 0x7fb5000cedd0 2023-10-16T11:01:09.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.103+0000 7fb4f2ffd700 1 -- 172.21.15.181:0/4198913239 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb4f4003a20 con 0x7fb5000cedd0 2023-10-16T11:01:09.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.103+0000 7fb4f2ffd700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 0x7fb4e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.103+0000 7fb506237700 1 -- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 msgr2=0x7fb4e406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:09.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.104+0000 7fb4f2ffd700 1 -- 172.21.15.181:0/4198913239 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fb4f4088d50 con 0x7fb5000cedd0 2023-10-16T11:01:09.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.104+0000 7fb506237700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 0x7fb4e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:09.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.108+0000 7fb4f2ffd700 1 -- 172.21.15.181:0/4198913239 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb4f408c030 con 0x7fb5000cedd0 2023-10-16T11:01:09.162 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:09.161+0000 7f5bb86bb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:01:09.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.305+0000 7fb506237700 1 -- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 msgr2=0x7fb4e406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:09.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.305+0000 7fb506237700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 0x7fb4e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.368+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb500072510 con 0x7fb5000cedd0 2023-10-16T11:01:09.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.369+0000 7fb4f2ffd700 1 -- 172.21.15.181:0/4198913239 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4929 (secure 0 0 0) 0x7fb4f4053a30 con 0x7fb5000cedd0 2023-10-16T11:01:09.370 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:09.370 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1460537182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1460537182},{"type":"v1","addr":"172.21.15.190:6833","nonce":1460537182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3626960628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3626960628},{"type":"v1","addr":"172.21.15.190:6835","nonce":3626960628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1772227629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1772227629},{"type":"v1","addr":"172.21.15.181:6836","nonce":1772227629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:00:46.276243+0000","modified":"2023-10-16T11:00:49.298812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9651},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9651":{"gid":9651,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6837/2931073086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2931073086},{"type":"v1","addr":"172.21.15.181:6837","nonce":2931073086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:01:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 msgr2=0x7fb4e406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 0x7fb4e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5000cedd0 msgr2=0x7fb50015df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5000cedd0 0x7fb50015df50 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400c4e0 tx=0x7fb4f400c7f0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 shutdown_connections 2023-10-16T11:01:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb50015db70 0x7fb50015e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7fb4e406b020 0x7fb4e406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5000ca2e0 0x7fb500160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 --2- 172.21.15.181:0/4198913239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5000cedd0 0x7fb50015df50 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 >> 172.21.15.181:0/4198913239 conn(0x7fb5000b1780 msgr2=0x7fb5000b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:09.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 shutdown_connections 2023-10-16T11:01:09.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.371+0000 7fb50849b700 1 -- 172.21.15.181:0/4198913239 wait complete. 2023-10-16T11:01:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2023-10-16T11:01:09.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:01:09.504 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.503+0000 7f42d8c9c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:01:09.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.508+0000 7f633fab6700 1 Processor -- start 2023-10-16T11:01:09.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.508+0000 7f633fab6700 1 -- start start 2023-10-16T11:01:09.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633fab6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f633815dc20 0x7f63380d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633fab6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63380d1e60 0x7f63380d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633fab6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63380ce4b0 0x7f63380ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633fab6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380b3b80 con 0x7f633815dc20 2023-10-16T11:01:09.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633fab6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380d2820 con 0x7f63380d1e60 2023-10-16T11:01:09.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633fab6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b2f00 con 0x7f63380ce4b0 2023-10-16T11:01:09.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f633815dc20 0x7f63380d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f633815dc20 0x7f63380d41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51586/0 (socket says 172.21.15.181:51586) 2023-10-16T11:01:09.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 -- 172.21.15.181:0/969312490 learned_addr learned my addr 172.21.15.181:0/969312490 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:09.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633e053700 1 --2- 172.21.15.181:0/969312490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63380ce4b0 0x7f63380ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d051700 1 --2- 172.21.15.181:0/969312490 >> v2:172.21.15.190:3300/0 conn(0x7f63380d1e60 0x7f63380d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 -- 172.21.15.181:0/969312490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63380ce4b0 msgr2=0x7f63380ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 --2- 172.21.15.181:0/969312490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63380ce4b0 0x7f63380ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 -- 172.21.15.181:0/969312490 >> v2:172.21.15.190:3300/0 conn(0x7f63380d1e60 msgr2=0x7f63380d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 --2- 172.21.15.181:0/969312490 >> v2:172.21.15.190:3300/0 conn(0x7f63380d1e60 0x7f63380d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 -- 172.21.15.181:0/969312490 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380b31e0 con 0x7f633815dc20 2023-10-16T11:01:09.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633d852700 1 --2- 172.21.15.181:0/969312490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f633815dc20 0x7f63380d41c0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6338161750 tx=0x7f6334009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65b0cac2dd16c24 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:09.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.509+0000 7f633c850700 1 -- 172.21.15.181:0/969312490 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334005e30 con 0x7f633815dc20 2023-10-16T11:01:09.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633c850700 1 -- 172.21.15.181:0/969312490 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334004030 con 0x7f633815dc20 2023-10-16T11:01:09.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633c850700 1 -- 172.21.15.181:0/969312490 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340043e0 con 0x7f633815dc20 2023-10-16T11:01:09.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 -- 172.21.15.181:0/969312490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f633815dc20 msgr2=0x7f63380d41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 --2- 172.21.15.181:0/969312490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f633815dc20 0x7f63380d41c0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f6338161750 tx=0x7f6334009bd0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 -- 172.21.15.181:0/969312490 shutdown_connections 2023-10-16T11:01:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 --2- 172.21.15.181:0/969312490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f633815dc20 0x7f63380d41c0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 --2- 172.21.15.181:0/969312490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63380ce4b0 0x7f63380ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 --2- 172.21.15.181:0/969312490 >> v2:172.21.15.190:3300/0 conn(0x7f63380d1e60 0x7f63380d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 -- 172.21.15.181:0/969312490 >> 172.21.15.181:0/969312490 conn(0x7f633808af90 msgr2=0x7f63380d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:09.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 -- 172.21.15.181:0/969312490 shutdown_connections 2023-10-16T11:01:09.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.510+0000 7f633fab6700 1 -- 172.21.15.181:0/969312490 wait complete. 2023-10-16T11:01:09.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.511+0000 7f633fab6700 1 Processor -- start 2023-10-16T11:01:09.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.511+0000 7f633fab6700 1 -- start start 2023-10-16T11:01:09.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633fab6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63380ce4b0 0x7f6338160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633fab6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63380d1e60 0x7f63380cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633fab6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f633815dc20 0x7f63381518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633fab6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380cc210 con 0x7f63380ce4b0 2023-10-16T11:01:09.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633fab6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633815fb90 con 0x7f63380d1e60 2023-10-16T11:01:09.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633fab6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63380cbf30 con 0x7f633815dc20 2023-10-16T11:01:09.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63380ce4b0 0x7f6338160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d051700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63380d1e60 0x7f63380cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63380ce4b0 0x7f6338160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51592/0 (socket says 172.21.15.181:51592) 2023-10-16T11:01:09.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 -- 172.21.15.181:0/1579786507 learned_addr learned my addr 172.21.15.181:0/1579786507 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:09.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633e053700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f633815dc20 0x7f63381518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 -- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f633815dc20 msgr2=0x7f63381518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f633815dc20 0x7f63381518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 -- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63380d1e60 msgr2=0x7f63380cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63380d1e60 0x7f63380cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d852700 1 -- 172.21.15.181:0/1579786507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334009500 con 0x7f63380ce4b0 2023-10-16T11:01:09.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633e053700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f633815dc20 0x7f63381518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:09.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.512+0000 7f633d051700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63380d1e60 0x7f63380cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:09.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.513+0000 7f633d852700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63380ce4b0 0x7f6338160630 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6334009c10 tx=0x7f6334004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:09.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.513+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334003bb0 con 0x7f63380ce4b0 2023-10-16T11:01:09.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.513+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334003d10 con 0x7f63380ce4b0 2023-10-16T11:01:09.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.513+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633815fe30 con 0x7f63380ce4b0 2023-10-16T11:01:09.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.513+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633401c660 con 0x7f63380ce4b0 2023-10-16T11:01:09.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.513+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63380cdb20 con 0x7f63380ce4b0 2023-10-16T11:01:09.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.514+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63380cdcb0 con 0x7f63380ce4b0 2023-10-16T11:01:09.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.514+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 104) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f633401c7c0 con 0x7f63380ce4b0 2023-10-16T11:01:09.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.514+0000 7f632e7fc700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f631c06b020 0x7f631c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.515+0000 7f633d051700 1 -- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f631c06b020 msgr2=0x7f631c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4215027768 2023-10-16T11:01:09.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.515+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f633408a8a0 con 0x7f63380ce4b0 2023-10-16T11:01:09.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.515+0000 7f633d051700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f631c06b020 0x7f631c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:09.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.519+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63380cdcb0 con 0x7f63380ce4b0 2023-10-16T11:01:09.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.525+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 105) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f6334055950 con 0x7f63380ce4b0 2023-10-16T11:01:09.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.526+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f631c06b020 msgr2=0x7f631c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:09.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.526+0000 7f632e7fc700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f631c06b020 0x7f631c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.592 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4251482700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.592 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4251482700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.592 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4251482700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.592 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4251482700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.593 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4251482700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.593 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4254488700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.593 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4254488700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.593 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4254488700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.593 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4254488700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.594 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:09.591+0000 7f4254488700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:09.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.777+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63380cdcb0 con 0x7f63380ce4b0 2023-10-16T11:01:09.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.778+0000 7f632e7fc700 1 -- 172.21.15.181:0/1579786507 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f63380cdcb0 con 0x7f63380ce4b0 2023-10-16T11:01:09.779 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:09.779 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1460537182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1460537182},{"type":"v1","addr":"172.21.15.190:6833","nonce":1460537182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3626960628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3626960628},{"type":"v1","addr":"172.21.15.190:6835","nonce":3626960628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1772227629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1772227629},{"type":"v1","addr":"172.21.15.181:6836","nonce":1772227629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:00:46.276243+0000","modified":"2023-10-16T11:00:49.298812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9651},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9651":{"gid":9651,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":4,"addr":"172.21.15.181:6837/2931073086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2931073086},{"type":"v1","addr":"172.21.15.181:6837","nonce":2931073086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-16T11:01:09.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63380ce4b0 msgr2=0x7f6338160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63380ce4b0 0x7f6338160630 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f6334009c10 tx=0x7f6334004750 comp rx=0 tx=0).stop 2023-10-16T11:01:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 shutdown_connections 2023-10-16T11:01:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f633815dc20 0x7f63381518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:6832/4215027768,v1:172.21.15.181:6833/4215027768] conn(0x7f631c06b020 0x7f631c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63380ce4b0 0x7f6338160630 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 --2- 172.21.15.181:0/1579786507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63380d1e60 0x7f63380cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 >> 172.21.15.181:0/1579786507 conn(0x7f633808af90 msgr2=0x7f6338152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 shutdown_connections 2023-10-16T11:01:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.780+0000 7f633fab6700 1 -- 172.21.15.181:0/1579786507 wait complete. 2023-10-16T11:01:09.783 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 85 2023-10-16T11:01:09.793 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:09.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.915+0000 7fe27b2fe700 1 Processor -- start 2023-10-16T11:01:09.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.915+0000 7fe27b2fe700 1 -- start start 2023-10-16T11:01:09.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27b2fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2740cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27b2fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe27406e010 0x7fe27406e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27b2fe700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe274066c90 con 0x7fe27415b8b0 2023-10-16T11:01:09.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27b2fe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27406e9c0 con 0x7fe27406e010 2023-10-16T11:01:09.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27b2fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe274066740 con 0x7fe2740b22a0 2023-10-16T11:01:09.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe27406e010 0x7fe27406e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe27406e010 0x7fe27406e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51672/0 (socket says 172.21.15.181:51672) 2023-10-16T11:01:09.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 -- 172.21.15.181:0/640023411 learned_addr learned my addr 172.21.15.181:0/640023411 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:09.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe278899700 1 --2- 172.21.15.181:0/640023411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2740cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 -- 172.21.15.181:0/640023411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 msgr2=0x7fe2740cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:09.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 --2- 172.21.15.181:0/640023411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2740cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 -- 172.21.15.181:0/640023411 >> v1:172.21.15.190:6789/0 conn(0x7fe27415b8b0 legacy=0x7fe27415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:09.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 -- 172.21.15.181:0/640023411 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2740669b0 con 0x7fe27406e010 2023-10-16T11:01:09.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe278899700 1 --2- 172.21.15.181:0/640023411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2740cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:09.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe27989b700 1 --2- 172.21.15.181:0/640023411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27406e010 0x7fe27406e480 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009e50 tx=0x7fe26c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167d075ffaea9a78 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:09.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe26bfff700 1 -- 172.21.15.181:0/640023411 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00e070 con 0x7fe27406e010 2023-10-16T11:01:09.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe26bfff700 1 -- 172.21.15.181:0/640023411 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c003a70 con 0x7fe27406e010 2023-10-16T11:01:09.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.916+0000 7fe26bfff700 1 -- 172.21.15.181:0/640023411 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c003e20 con 0x7fe27406e010 2023-10-16T11:01:09.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 -- 172.21.15.181:0/640023411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27406e010 msgr2=0x7fe27406e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 --2- 172.21.15.181:0/640023411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27406e010 0x7fe27406e480 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009e50 tx=0x7fe26c00b330 comp rx=0 tx=0).stop 2023-10-16T11:01:09.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 -- 172.21.15.181:0/640023411 shutdown_connections 2023-10-16T11:01:09.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 --2- 172.21.15.181:0/640023411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27406e010 0x7fe27406e480 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 --2- 172.21.15.181:0/640023411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2740cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 -- 172.21.15.181:0/640023411 >> 172.21.15.181:0/640023411 conn(0x7fe2740b1780 msgr2=0x7fe2740b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:09.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 -- 172.21.15.181:0/640023411 shutdown_connections 2023-10-16T11:01:09.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.917+0000 7fe27b2fe700 1 -- 172.21.15.181:0/640023411 wait complete. 2023-10-16T11:01:09.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.918+0000 7fe27b2fe700 1 Processor -- start 2023-10-16T11:01:09.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.918+0000 7fe27b2fe700 1 -- start start 2023-10-16T11:01:09.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27b2fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27406e010 0x7fe274159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27b2fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2741e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27b2fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27415b8b0 0x7fe27415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:09.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27b2fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27415bee0 con 0x7fe27415b8b0 2023-10-16T11:01:09.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27b2fe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2741e7ff0 con 0x7fe27406e010 2023-10-16T11:01:09.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27b2fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2741e7750 con 0x7fe2740b22a0 2023-10-16T11:01:09.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27415b8b0 0x7fe27415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27415b8b0 0x7fe27415c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51678/0 (socket says 172.21.15.181:51678) 2023-10-16T11:01:09.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 -- 172.21.15.181:0/1939161768 learned_addr learned my addr 172.21.15.181:0/1939161768 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:09.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe278899700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2741e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27909a700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27406e010 0x7fe274159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:09.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 -- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 msgr2=0x7fe2741e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2741e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 -- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27406e010 msgr2=0x7fe274159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:09.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27406e010 0x7fe274159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:09.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.919+0000 7fe27989b700 1 -- 172.21.15.181:0/1939161768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c009b30 con 0x7fe27415b8b0 2023-10-16T11:01:09.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.920+0000 7fe27989b700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27415b8b0 0x7fe27415c370 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fe2740747e0 tx=0x7fe26c004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:09.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.920+0000 7fe269ffb700 1 -- 172.21.15.181:0/1939161768 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c004440 con 0x7fe27415b8b0 2023-10-16T11:01:09.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.920+0000 7fe269ffb700 1 -- 172.21.15.181:0/1939161768 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c0045e0 con 0x7fe27415b8b0 2023-10-16T11:01:09.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.920+0000 7fe269ffb700 1 -- 172.21.15.181:0/1939161768 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c004990 con 0x7fe27415b8b0 2023-10-16T11:01:09.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.920+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2741e7a30 con 0x7fe27415b8b0 2023-10-16T11:01:09.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.920+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe274159d60 con 0x7fe27415b8b0 2023-10-16T11:01:09.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.921+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27415f690 con 0x7fe27415b8b0 2023-10-16T11:01:09.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.921+0000 7fe269ffb700 1 -- 172.21.15.181:0/1939161768 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 105) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe26c012430 con 0x7fe27415b8b0 2023-10-16T11:01:09.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.922+0000 7fe269ffb700 1 -- 172.21.15.181:0/1939161768 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6893+0+0 (secure 0 0 0) 0x7fe26c08a730 con 0x7fe27415b8b0 2023-10-16T11:01:09.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:09.927+0000 7fe269ffb700 1 -- 172.21.15.181:0/1939161768 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe27415f690 con 0x7fe27415b8b0 2023-10-16T11:01:10.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.123+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe274066c90 con 0x7fe27415b8b0 2023-10-16T11:01:10.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.124+0000 7fe269ffb700 1 -- 172.21.15.181:0/1939161768 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+18388 (secure 0 0 0) 0x7fe274066c90 con 0x7fe27415b8b0 2023-10-16T11:01:10.125 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:10.125 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":170,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:09.519721+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:00:44.693955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-16T11:00:45.236607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1993379789":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/683890550":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/1832107878":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/314629984":"2023-10-17T11:01:09.519680+0000","172.21.15.181:6833/4215027768":"2023-10-17T11:01:09.519680+0000","172.21.15.181:6832/4215027768":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/3226890773":"2023-10-17T11:01:09.519680+0000","172.21.15.181:6833/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/1134301943":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/642481654":"2023-10-17T11:01:00.993294+0000","172.21.15.181:6832/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/120971111":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/3800812080":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/2485734134":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/4030419984":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/1244048650":"2023-10-17T11:01:00.993294+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:10.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27415b8b0 msgr2=0x7fe27415c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27415b8b0 0x7fe27415c370 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fe2740747e0 tx=0x7fe26c004b20 comp rx=0 tx=0).stop 2023-10-16T11:01:10.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 shutdown_connections 2023-10-16T11:01:10.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27415b8b0 0x7fe27415c370 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27406e010 0x7fe274159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 --2- 172.21.15.181:0/1939161768 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2740b22a0 0x7fe2741e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 >> 172.21.15.181:0/1939161768 conn(0x7fe2740b1780 msgr2=0x7fe27415a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:10.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 shutdown_connections 2023-10-16T11:01:10.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.126+0000 7fe27b2fe700 1 -- 172.21.15.181:0/1939161768 wait complete. 2023-10-16T11:01:10.139 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:01:10.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.259+0000 7f94a231d700 1 Processor -- start 2023-10-16T11:01:10.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.259+0000 7f94a231d700 1 -- start start 2023-10-16T11:01:10.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f94a231d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f949c0cb110 0x7f949c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f94a231d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15a0f0 0x7f949c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f94a231d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c06ed00 con 0x7f949c15e460 2023-10-16T11:01:10.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f94a231d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c06e0f0 con 0x7f949c0cb110 2023-10-16T11:01:10.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f94a231d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c06e3d0 con 0x7f949c15a0f0 2023-10-16T11:01:10.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f949b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f949c0cb110 0x7f949c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f949bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f949c15e460 0x7f949c15e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34526/0 (socket says 172.21.15.181:34526) 2023-10-16T11:01:10.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f949b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f949c0cb110 0x7f949c0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51682/0 (socket says 172.21.15.181:51682) 2023-10-16T11:01:10.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.260+0000 7f949bfff700 1 -- 172.21.15.181:0/3091643445 learned_addr learned my addr 172.21.15.181:0/3091643445 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:10.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f94a08ba700 1 --2- 172.21.15.181:0/3091643445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15a0f0 0x7f949c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949b7fe700 1 -- 172.21.15.181:0/3091643445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15a0f0 msgr2=0x7f949c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:10.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949b7fe700 1 --2- 172.21.15.181:0/3091643445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15a0f0 0x7f949c15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949b7fe700 1 -- 172.21.15.181:0/3091643445 >> v1:172.21.15.181:6789/0 conn(0x7f949c15e460 legacy=0x7f949c15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949affd700 1 -- 172.21.15.181:0/3091643445 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683409049 0 0) 0x7f949c06ed00 con 0x7f949c15e460 2023-10-16T11:01:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949b7fe700 1 -- 172.21.15.181:0/3091643445 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c06e6b0 con 0x7f949c0cb110 2023-10-16T11:01:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949b7fe700 1 --2- 172.21.15.181:0/3091643445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c0cb110 0x7f949c0d0fc0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f948c009840 tx=0x7f948c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cf097433b274f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949affd700 1 -- 172.21.15.181:0/3091643445 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c011830 con 0x7f949c0cb110 2023-10-16T11:01:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949affd700 1 -- 172.21.15.181:0/3091643445 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c011990 con 0x7f949c0cb110 2023-10-16T11:01:10.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f949affd700 1 -- 172.21.15.181:0/3091643445 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c011d20 con 0x7f949c0cb110 2023-10-16T11:01:10.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f94a231d700 1 -- 172.21.15.181:0/3091643445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c0cb110 msgr2=0x7f949c0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f94a231d700 1 --2- 172.21.15.181:0/3091643445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c0cb110 0x7f949c0d0fc0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f948c009840 tx=0x7f948c009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:10.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f94a231d700 1 -- 172.21.15.181:0/3091643445 shutdown_connections 2023-10-16T11:01:10.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f94a231d700 1 --2- 172.21.15.181:0/3091643445 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15a0f0 0x7f949c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f94a231d700 1 --2- 172.21.15.181:0/3091643445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c0cb110 0x7f949c0d0fc0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.261+0000 7f94a231d700 1 -- 172.21.15.181:0/3091643445 >> 172.21.15.181:0/3091643445 conn(0x7f949c0d04a0 msgr2=0x7f949c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:10.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.262+0000 7f94a231d700 1 -- 172.21.15.181:0/3091643445 shutdown_connections 2023-10-16T11:01:10.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.262+0000 7f94a231d700 1 -- 172.21.15.181:0/3091643445 wait complete. 2023-10-16T11:01:10.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.263+0000 7f94a231d700 1 Processor -- start 2023-10-16T11:01:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.263+0000 7f94a231d700 1 -- start start 2023-10-16T11:01:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f949c0cb110 0x7f949c0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c15a0f0 0x7f949c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15e460 0x7f949c15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15b7a0 con 0x7f949c15a0f0 2023-10-16T11:01:10.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15ba40 con 0x7f949c0cb110 2023-10-16T11:01:10.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15bd20 con 0x7f949c15e460 2023-10-16T11:01:10.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c15a0f0 0x7f949c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c15a0f0 0x7f949c0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51688/0 (socket says 172.21.15.181:51688) 2023-10-16T11:01:10.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 -- 172.21.15.181:0/2227601241 learned_addr learned my addr 172.21.15.181:0/2227601241 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:10.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a08ba700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15e460 0x7f949c15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949bfff700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f949c0cb110 0x7f949c0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 -- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15e460 msgr2=0x7f949c15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15e460 0x7f949c15b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 -- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f949c0cb110 msgr2=0x7f949c0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f949c0cb110 0x7f949c0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 -- 172.21.15.181:0/2227601241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0094a0 con 0x7f949c15a0f0 2023-10-16T11:01:10.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949bfff700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f949c0cb110 0x7f949c0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:10.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f949b7fe700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c15a0f0 0x7f949c0b5970 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f948c000c00 tx=0x7f948c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:10.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f9498ff9700 1 -- 172.21.15.181:0/2227601241 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0045d0 con 0x7f949c15a0f0 2023-10-16T11:01:10.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f9498ff9700 1 -- 172.21.15.181:0/2227601241 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c01e070 con 0x7f949c15a0f0 2023-10-16T11:01:10.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c15c000 con 0x7f949c15a0f0 2023-10-16T11:01:10.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f9498ff9700 1 -- 172.21.15.181:0/2227601241 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0038e0 con 0x7f949c15a0f0 2023-10-16T11:01:10.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.264+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c15c5c0 con 0x7f949c15a0f0 2023-10-16T11:01:10.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.265+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c06f170 con 0x7f949c15a0f0 2023-10-16T11:01:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.266+0000 7f9498ff9700 1 -- 172.21.15.181:0/2227601241 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 105) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f948c003a40 con 0x7f949c15a0f0 2023-10-16T11:01:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.266+0000 7f9498ff9700 1 -- 172.21.15.181:0/2227601241 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6893+0+0 (secure 0 0 0) 0x7f948c08a550 con 0x7f949c15a0f0 2023-10-16T11:01:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.270+0000 7f9498ff9700 1 -- 172.21.15.181:0/2227601241 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f949c06f170 con 0x7f949c15a0f0 2023-10-16T11:01:10.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.510+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f949c06f170 con 0x7f949c15a0f0 2023-10-16T11:01:10.512 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:01:10.511+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:01:10.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.512+0000 7f9498ff9700 1 -- 172.21.15.181:0/2227601241 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7f949c06f170 con 0x7f949c15a0f0 2023-10-16T11:01:10.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.515+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c15a0f0 msgr2=0x7f949c0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.515+0000 7f94a231d700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c15a0f0 0x7f949c0b5970 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f948c000c00 tx=0x7f948c004750 comp rx=0 tx=0).stop 2023-10-16T11:01:10.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.516+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 shutdown_connections 2023-10-16T11:01:10.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.516+0000 7f94a231d700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f949c15e460 0x7f949c15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.516+0000 7f94a231d700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f949c0cb110 0x7f949c0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.516+0000 7f94a231d700 1 --2- 172.21.15.181:0/2227601241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949c15a0f0 0x7f949c0b5970 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.516+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 >> 172.21.15.181:0/2227601241 conn(0x7f949c0d04a0 msgr2=0x7f949c15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.516+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 shutdown_connections 2023-10-16T11:01:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.516+0000 7f94a231d700 1 -- 172.21.15.181:0/2227601241 wait complete. 2023-10-16T11:01:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:01:10.519 INFO:tasks.ceph.mds.c.smithi181.stderr: -4841> 2023-10-16T11:00:50.836+0000 7f9aa3b42700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:01:10.537 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:01:10.556 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:01:10.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.660+0000 7f6c7d90f700 1 Processor -- start 2023-10-16T11:01:10.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.660+0000 7f6c7d90f700 1 -- start start 2023-10-16T11:01:10.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.660+0000 7f6c7d90f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6c78161f70 0x7f6c780cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.660+0000 7f6c7d90f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780d0cb0 con 0x7f6c7815d740 2023-10-16T11:01:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.660+0000 7f6c7d90f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780ce820 con 0x7f6c780ce410 2023-10-16T11:01:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.660+0000 7f6c7d90f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c78162400 con 0x7f6c78161f70 2023-10-16T11:01:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c767fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6c78161f70 0x7f6c780cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c76ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6c7815d740 0x7f6c780cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34570/0 (socket says 172.21.15.181:34570) 2023-10-16T11:01:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c76ffd700 1 -- 172.21.15.181:0/1366038013 learned_addr learned my addr 172.21.15.181:0/1366038013 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c75ffb700 1 -- 172.21.15.181:0/1366038013 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749836708 0 0) 0x7f6c780d0cb0 con 0x7f6c7815d740 2023-10-16T11:01:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c75ffb700 1 -- 172.21.15.181:0/1366038013 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c600035b0 con 0x7f6c7815d740 2023-10-16T11:01:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c767fc700 1 -- 172.21.15.181:0/1366038013 >> v1:172.21.15.190:6789/0 conn(0x7f6c780ce410 legacy=0x7f6c780ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:10.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c767fc700 1 -- 172.21.15.181:0/1366038013 >> v1:172.21.15.181:6789/0 conn(0x7f6c7815d740 legacy=0x7f6c780cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c767fc700 1 -- 172.21.15.181:0/1366038013 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780d00a0 con 0x7f6c78161f70 2023-10-16T11:01:10.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c767fc700 1 --2- 172.21.15.181:0/1366038013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c78161f70 0x7f6c780cded0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009840 tx=0x7f6c68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91140c333233bfa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:10.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c75ffb700 1 -- 172.21.15.181:0/1366038013 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68004230 con 0x7f6c78161f70 2023-10-16T11:01:10.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c75ffb700 1 -- 172.21.15.181:0/1366038013 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68004390 con 0x7f6c78161f70 2023-10-16T11:01:10.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.661+0000 7f6c75ffb700 1 -- 172.21.15.181:0/1366038013 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680036c0 con 0x7f6c78161f70 2023-10-16T11:01:10.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.662+0000 7f6c7d90f700 1 -- 172.21.15.181:0/1366038013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c78161f70 msgr2=0x7f6c780cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.662+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/1366038013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c78161f70 0x7f6c780cded0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009840 tx=0x7f6c68009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:10.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.662+0000 7f6c7d90f700 1 -- 172.21.15.181:0/1366038013 shutdown_connections 2023-10-16T11:01:10.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.662+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/1366038013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c78161f70 0x7f6c780cded0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.662+0000 7f6c7d90f700 1 -- 172.21.15.181:0/1366038013 >> 172.21.15.181:0/1366038013 conn(0x7f6c7809a930 msgr2=0x7f6c780d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:10.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.662+0000 7f6c7d90f700 1 -- 172.21.15.181:0/1366038013 shutdown_connections 2023-10-16T11:01:10.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.662+0000 7f6c7d90f700 1 -- 172.21.15.181:0/1366038013 wait complete. 2023-10-16T11:01:10.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.663+0000 7f6c7d90f700 1 Processor -- start 2023-10-16T11:01:10.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.663+0000 7f6c7d90f700 1 -- start start 2023-10-16T11:01:10.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c7d90f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c780ce410 0x7f6c7814fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c7d90f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7815d740 0x7f6c780c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c7d90f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c78161f70 0x7f6c780cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c7d90f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7814ffa0 con 0x7f6c7815d740 2023-10-16T11:01:10.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c7d90f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780c5320 con 0x7f6c780ce410 2023-10-16T11:01:10.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c7d90f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780c5600 con 0x7f6c78161f70 2023-10-16T11:01:10.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7815d740 0x7f6c780c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7815d740 0x7f6c780c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51730/0 (socket says 172.21.15.181:51730) 2023-10-16T11:01:10.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 -- 172.21.15.181:0/350229859 learned_addr learned my addr 172.21.15.181:0/350229859 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:10.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c76ffd700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c780ce410 0x7f6c7814fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c777fe700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c78161f70 0x7f6c780cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 -- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c78161f70 msgr2=0x7f6c780cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c78161f70 0x7f6c780cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 -- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c780ce410 msgr2=0x7f6c7814fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:10.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c780ce410 0x7f6c7814fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:10.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 -- 172.21.15.181:0/350229859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680094a0 con 0x7f6c7815d740 2023-10-16T11:01:10.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c767fc700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7815d740 0x7f6c780c8900 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f6c78071620 tx=0x7f6c68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:10.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.664+0000 7f6c5ffff700 1 -- 172.21.15.181:0/350229859 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68003e00 con 0x7f6c7815d740 2023-10-16T11:01:10.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.665+0000 7f6c5ffff700 1 -- 172.21.15.181:0/350229859 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68011630 con 0x7f6c7815d740 2023-10-16T11:01:10.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.665+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780c58e0 con 0x7f6c7815d740 2023-10-16T11:01:10.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.665+0000 7f6c5ffff700 1 -- 172.21.15.181:0/350229859 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680119a0 con 0x7f6c7815d740 2023-10-16T11:01:10.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.665+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c780c5ea0 con 0x7f6c7815d740 2023-10-16T11:01:10.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.666+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7804ebb0 con 0x7f6c7815d740 2023-10-16T11:01:10.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.666+0000 7f6c5ffff700 1 -- 172.21.15.181:0/350229859 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 106) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6c68018050 con 0x7f6c7815d740 2023-10-16T11:01:10.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.667+0000 7f6c5ffff700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6c6406ad50 0x7f6c6406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:10.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.667+0000 7f6c76ffd700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6c6406ad50 0x7f6c6406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:10.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.667+0000 7f6c5ffff700 1 -- 172.21.15.181:0/350229859 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6979+0+0 (secure 0 0 0) 0x7f6c6808a9b0 con 0x7f6c7815d740 2023-10-16T11:01:10.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.667+0000 7f6c76ffd700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6c6406ad50 0x7f6c6406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005910 tx=0x7f6c6c009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:10.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.671+0000 7f6c5ffff700 1 -- 172.21.15.181:0/350229859 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6c68055a80 con 0x7f6c7815d740 2023-10-16T11:01:10.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:10.911+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c7806d590 con 0x7f6c7815d740 2023-10-16T11:01:11.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.533+0000 7f6c5ffff700 1 -- 172.21.15.181:0/350229859 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c680590a0 con 0x7f6c7815d740 2023-10-16T11:01:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.536+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6c6406ad50 msgr2=0x7f6c6406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.536+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6c6406ad50 0x7f6c6406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005910 tx=0x7f6c6c009040 comp rx=0 tx=0).stop 2023-10-16T11:01:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.536+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7815d740 msgr2=0x7f6c780c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:11.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.536+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7815d740 0x7f6c780c8900 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f6c78071620 tx=0x7f6c68004750 comp rx=0 tx=0).stop 2023-10-16T11:01:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 shutdown_connections 2023-10-16T11:01:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6c6406ad50 0x7f6c6406d210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c78161f70 0x7f6c780cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c780ce410 0x7f6c7814fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 --2- 172.21.15.181:0/350229859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7815d740 0x7f6c780c8900 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 >> 172.21.15.181:0/350229859 conn(0x7f6c7809a930 msgr2=0x7f6c780cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 shutdown_connections 2023-10-16T11:01:11.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.537+0000 7f6c7d90f700 1 -- 172.21.15.181:0/350229859 wait complete. 2023-10-16T11:01:11.553 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:01:11.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.720+0000 7f8c013a2700 1 Processor -- start 2023-10-16T11:01:11.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.720+0000 7f8c013a2700 1 -- start start 2023-10-16T11:01:11.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8c013a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15dc60 0x7f8bfc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:11.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8c013a2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8bfc0d06a0 0x7f8bfc0b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:11.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8c013a2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc0d0af0 con 0x7f8bfc15d880 2023-10-16T11:01:11.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8c013a2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc0d22a0 con 0x7f8bfc0d06a0 2023-10-16T11:01:11.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8c013a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0d1690 con 0x7f8bfc15dc60 2023-10-16T11:01:11.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfb7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8bfc0d06a0 0x7f8bfc0b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:11.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15dc60 0x7f8bfc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:11.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfb7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8bfc0d06a0 0x7f8bfc0b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:56810/0 (socket says 172.21.15.181:56810) 2023-10-16T11:01:11.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1306601931 learned_addr learned my addr 172.21.15.181:0/1306601931 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:11.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfaffd700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8bfc15d880 0x7f8bfc0cefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:46776/0 (socket says 172.21.15.181:46776) 2023-10-16T11:01:11.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1306601931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15dc60 msgr2=0x7f8bfc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:11.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfb7fe700 1 --2- 172.21.15.181:0/1306601931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15dc60 0x7f8bfc0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1306601931 >> v1:172.21.15.190:6789/0 conn(0x7f8bfc15d880 legacy=0x7f8bfc0cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:11.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.721+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1306601931 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0d1970 con 0x7f8bfc0d06a0 2023-10-16T11:01:11.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8bfb7fe700 1 --2- 172.21.15.181:0/1306601931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 0x7f8bfc0b8890 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009e50 tx=0x7f8bf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbdf27ae7fd291d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:11.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8bf9ffb700 1 -- 172.21.15.181:0/1306601931 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400e070 con 0x7f8bfc0d06a0 2023-10-16T11:01:11.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8bf9ffb700 1 -- 172.21.15.181:0/1306601931 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf400b910 con 0x7f8bfc0d06a0 2023-10-16T11:01:11.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8bf9ffb700 1 -- 172.21.15.181:0/1306601931 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400bcc0 con 0x7f8bfc0d06a0 2023-10-16T11:01:11.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 -- 172.21.15.181:0/1306601931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 msgr2=0x7f8bfc0b8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:11.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1306601931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 0x7f8bfc0b8890 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009e50 tx=0x7f8bf400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:11.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 -- 172.21.15.181:0/1306601931 shutdown_connections 2023-10-16T11:01:11.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1306601931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 0x7f8bfc0b8890 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1306601931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15dc60 0x7f8bfc0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 -- 172.21.15.181:0/1306601931 >> 172.21.15.181:0/1306601931 conn(0x7f8bfc09cb10 msgr2=0x7f8bfc0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:11.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 -- 172.21.15.181:0/1306601931 shutdown_connections 2023-10-16T11:01:11.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.722+0000 7f8c013a2700 1 -- 172.21.15.181:0/1306601931 wait complete. 2023-10-16T11:01:11.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 Processor -- start 2023-10-16T11:01:11.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 -- start start 2023-10-16T11:01:11.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 0x7f8bfc1f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:11.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15d880 0x7f8bfc15f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:11.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc15dc60 0x7f8bfc15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:11.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1f2e40 con 0x7f8bfc15dc60 2023-10-16T11:01:11.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1f3100 con 0x7f8bfc0d06a0 2023-10-16T11:01:11.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.724+0000 7f8c013a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1f33e0 con 0x7f8bfc15d880 2023-10-16T11:01:11.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc15dc60 0x7f8bfc15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:11.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc15dc60 0x7f8bfc15f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51766/0 (socket says 172.21.15.181:51766) 2023-10-16T11:01:11.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1020984728 learned_addr learned my addr 172.21.15.181:0/1020984728 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:11.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfaffd700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 0x7f8bfc1f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:11.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfa7fc700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15d880 0x7f8bfc15f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:11.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15d880 msgr2=0x7f8bfc15f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:11.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15d880 0x7f8bfc15f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 msgr2=0x7f8bfc1f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:11.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 0x7f8bfc1f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:11.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 -- 172.21.15.181:0/1020984728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009b30 con 0x7f8bfc15dc60 2023-10-16T11:01:11.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8bfb7fe700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc15dc60 0x7f8bfc15f5c0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06b3d0 tx=0x7f8bf4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:11.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004960 con 0x7f8bfc15dc60 2023-10-16T11:01:11.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf40092e0 con 0x7f8bfc15dc60 2023-10-16T11:01:11.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf401c660 con 0x7f8bfc15dc60 2023-10-16T11:01:11.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc1f36c0 con 0x7f8bfc15dc60 2023-10-16T11:01:11.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.725+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc1f6050 con 0x7f8bfc15dc60 2023-10-16T11:01:11.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.726+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc1f6210 con 0x7f8bfc15dc60 2023-10-16T11:01:11.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.727+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 106) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f8bf401c7c0 con 0x7f8bfc15dc60 2023-10-16T11:01:11.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.727+0000 7f8be37fe700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f8be806ad50 0x7f8be806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:11.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.727+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6979+0+0 (secure 0 0 0) 0x7f8bf408b500 con 0x7f8bfc15dc60 2023-10-16T11:01:11.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.727+0000 7f8bfaffd700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f8be806ad50 0x7f8be806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:11.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.727+0000 7f8bfaffd700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f8be806ad50 0x7f8be806d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000fb0 tx=0x7f8bf0007ce0 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:11.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.732+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8bf40563a0 con 0x7f8bfc15dc60 2023-10-16T11:01:11.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:11.924+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8bfc1f3b80 con 0x7f8bfc15dc60 2023-10-16T11:01:12.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.029+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 107) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8bf408b6c0 con 0x7f8bfc15dc60 2023-10-16T11:01:12.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.541+0000 7f8be37fe700 1 -- 172.21.15.181:0/1020984728 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v172) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8bf4083c90 con 0x7f8bfc15dc60 2023-10-16T11:01:12.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f8be806ad50 msgr2=0x7f8be806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:12.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f8be806ad50 0x7f8be806d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000fb0 tx=0x7f8bf0007ce0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc15dc60 msgr2=0x7f8bfc15f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:12.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc15dc60 0x7f8bfc15f5c0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc06b3d0 tx=0x7f8bf4004ad0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 shutdown_connections 2023-10-16T11:01:12.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f8be806ad50 0x7f8be806d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bfc15dc60 0x7f8bfc15f5c0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8bfc0d06a0 0x7f8bfc1f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 --2- 172.21.15.181:0/1020984728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8bfc15d880 0x7f8bfc15f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 >> 172.21.15.181:0/1020984728 conn(0x7f8bfc09cb10 msgr2=0x7f8bfc06d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:12.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 shutdown_connections 2023-10-16T11:01:12.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.544+0000 7f8c013a2700 1 -- 172.21.15.181:0/1020984728 wait complete. 2023-10-16T11:01:12.549 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:01:12.561 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:01:12.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.687+0000 7f44b1cbe700 1 Processor -- start 2023-10-16T11:01:12.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.687+0000 7f44b1cbe700 1 -- start start 2023-10-16T11:01:12.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44b1cbe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac15e610 0x7f44ac15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:12.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44b1cbe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44ac0cf040 0x7f44ac0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44b1cbe700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ac06ed90 con 0x7f44ac0665a0 2023-10-16T11:01:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44b1cbe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ac15f260 con 0x7f44ac0cf040 2023-10-16T11:01:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44b1cbe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac0ceb40 con 0x7f44ac15e610 2023-10-16T11:01:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44abfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44ac0cf040 0x7f44ac0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44abfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44ac0cf040 0x7f44ac0cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51774/0 (socket says 172.21.15.181:51774) 2023-10-16T11:01:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44abfff700 1 -- 172.21.15.181:0/1851212036 learned_addr learned my addr 172.21.15.181:0/1851212036 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44abfff700 1 -- 172.21.15.181:0/1851212036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac15e610 msgr2=0x7f44ac15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44abfff700 1 --2- 172.21.15.181:0/1851212036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac15e610 0x7f44ac15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44abfff700 1 -- 172.21.15.181:0/1851212036 >> v1:172.21.15.190:6789/0 conn(0x7f44ac0665a0 legacy=0x7f44ac06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.688+0000 7f44abfff700 1 -- 172.21.15.181:0/1851212036 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac06e7d0 con 0x7f44ac0cf040 2023-10-16T11:01:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44abfff700 1 --2- 172.21.15.181:0/1851212036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac0cf040 0x7f44ac0cf450 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009e80 tx=0x7f44a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939dde7cf47eaa18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44aa7fc700 1 -- 172.21.15.181:0/1851212036 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400e070 con 0x7f44ac0cf040 2023-10-16T11:01:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44aa7fc700 1 -- 172.21.15.181:0/1851212036 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a4003a20 con 0x7f44ac0cf040 2023-10-16T11:01:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44aa7fc700 1 -- 172.21.15.181:0/1851212036 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4003dd0 con 0x7f44ac0cf040 2023-10-16T11:01:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1851212036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac0cf040 msgr2=0x7f44ac0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1851212036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac0cf040 0x7f44ac0cf450 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009e80 tx=0x7f44a400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1851212036 shutdown_connections 2023-10-16T11:01:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1851212036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac0cf040 0x7f44ac0cf450 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1851212036 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac15e610 0x7f44ac15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1851212036 >> 172.21.15.181:0/1851212036 conn(0x7f44ac0b1780 msgr2=0x7f44ac0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1851212036 shutdown_connections 2023-10-16T11:01:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.689+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1851212036 wait complete. 2023-10-16T11:01:12.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.690+0000 7f44b1cbe700 1 Processor -- start 2023-10-16T11:01:12.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44b1cbe700 1 -- start start 2023-10-16T11:01:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44b1cbe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac0665a0 0x7f44ac154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44b1cbe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac0cf040 0x7f44ac0c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44b1cbe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac15e610 0x7f44ac156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44b1cbe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac0c9ad0 con 0x7f44ac15e610 2023-10-16T11:01:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44b1cbe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac0c4030 con 0x7f44ac0cf040 2023-10-16T11:01:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44b1cbe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac155520 con 0x7f44ac0665a0 2023-10-16T11:01:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac15e610 0x7f44ac156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44abfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac15e610 0x7f44ac156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51780/0 (socket says 172.21.15.181:51780) 2023-10-16T11:01:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.691+0000 7f44abfff700 1 -- 172.21.15.181:0/1243443761 learned_addr learned my addr 172.21.15.181:0/1243443761 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44ab7fe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac0665a0 0x7f44ac154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44aaffd700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac0cf040 0x7f44ac0c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44abfff700 1 -- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac0665a0 msgr2=0x7f44ac154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44abfff700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac0665a0 0x7f44ac154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44abfff700 1 -- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac0cf040 msgr2=0x7f44ac0c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44abfff700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac0cf040 0x7f44ac0c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:12.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44abfff700 1 -- 172.21.15.181:0/1243443761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009b30 con 0x7f44ac15e610 2023-10-16T11:01:12.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44ab7fe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac0665a0 0x7f44ac154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:12.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44aaffd700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac0cf040 0x7f44ac0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:12.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44abfff700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac15e610 0x7f44ac156e20 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f44a4015040 tx=0x7f44a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:12.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f4493fff700 1 -- 172.21.15.181:0/1243443761 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a40042c0 con 0x7f44ac15e610 2023-10-16T11:01:12.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f4493fff700 1 -- 172.21.15.181:0/1243443761 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a4004460 con 0x7f44ac15e610 2023-10-16T11:01:12.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ac0c45e0 con 0x7f44ac15e610 2023-10-16T11:01:12.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f4493fff700 1 -- 172.21.15.181:0/1243443761 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4004810 con 0x7f44ac15e610 2023-10-16T11:01:12.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.692+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ac0c93b0 con 0x7f44ac15e610 2023-10-16T11:01:12.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.693+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ac0c4770 con 0x7f44ac15e610 2023-10-16T11:01:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.693+0000 7f4493fff700 1 -- 172.21.15.181:0/1243443761 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 107) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f44a4019070 con 0x7f44ac15e610 2023-10-16T11:01:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.694+0000 7f4493fff700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f449806af90 0x7f449806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.694+0000 7f4493fff700 1 -- 172.21.15.181:0/1243443761 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6541+0+0 (secure 0 0 0) 0x7f44a408aa60 con 0x7f44ac15e610 2023-10-16T11:01:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.694+0000 7f44ab7fe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f449806af90 0x7f449806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.694+0000 7f44ab7fe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f449806af90 0x7f449806d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f44a0006fd0 tx=0x7f44a0008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:12.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.698+0000 7f4493fff700 1 -- 172.21.15.181:0/1243443761 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f44a4055760 con 0x7f44ac15e610 2023-10-16T11:01:12.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:12.897+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f44ac06fd40 con 0x7f44ac15e610 2023-10-16T11:01:13.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.543+0000 7f4493fff700 1 -- 172.21.15.181:0/1243443761 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) v1 ==== 169+0+0 (secure 0 0 0) 0x7f44a40200d0 con 0x7f44ac15e610 2023-10-16T11:01:13.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f449806af90 msgr2=0x7f449806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f449806af90 0x7f449806d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f44a0006fd0 tx=0x7f44a0008040 comp rx=0 tx=0).stop 2023-10-16T11:01:13.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac15e610 msgr2=0x7f44ac156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac15e610 0x7f44ac156e20 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f44a4015040 tx=0x7f44a4004ad0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 shutdown_connections 2023-10-16T11:01:13.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f449806af90 0x7f449806d450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44ac0665a0 0x7f44ac154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44ac15e610 0x7f44ac156e20 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 --2- 172.21.15.181:0/1243443761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44ac0cf040 0x7f44ac0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.546+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 >> 172.21.15.181:0/1243443761 conn(0x7f44ac0b1780 msgr2=0x7f44ac0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:13.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.547+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 shutdown_connections 2023-10-16T11:01:13.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.547+0000 7f44b1cbe700 1 -- 172.21.15.181:0/1243443761 wait complete. 2023-10-16T11:01:13.551 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:01:13.562 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:01:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.684+0000 7f86c9944700 1 Processor -- start 2023-10-16T11:01:13.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.684+0000 7f86c9944700 1 -- start start 2023-10-16T11:01:13.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c9944700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86c4072760 0x7f86c406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c9944700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c406ade0 0x7f86c41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c9944700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c406b210 con 0x7f86c415da70 2023-10-16T11:01:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c9944700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c40cef00 con 0x7f86c4072760 2023-10-16T11:01:13.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c9944700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40cec20 con 0x7f86c406ade0 2023-10-16T11:01:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f86c415da70 0x7f86c40ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34630/0 (socket says 172.21.15.181:34630) 2023-10-16T11:01:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c2ffd700 1 -- 172.21.15.181:0/2206027140 learned_addr learned my addr 172.21.15.181:0/2206027140 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c27fc700 1 --2- 172.21.15.181:0/2206027140 >> v2:172.21.15.181:3300/0 conn(0x7f86c4072760 0x7f86c406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c1ffb700 1 -- 172.21.15.181:0/2206027140 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106415112 0 0) 0x7f86c406b210 con 0x7f86c415da70 2023-10-16T11:01:13.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c37fe700 1 --2- 172.21.15.181:0/2206027140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c406ade0 0x7f86c41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:13.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c1ffb700 1 -- 172.21.15.181:0/2206027140 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86ac0035b0 con 0x7f86c415da70 2023-10-16T11:01:13.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c27fc700 1 -- 172.21.15.181:0/2206027140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c406ade0 msgr2=0x7f86c41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:13.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c27fc700 1 --2- 172.21.15.181:0/2206027140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c406ade0 0x7f86c41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c27fc700 1 -- 172.21.15.181:0/2206027140 >> v1:172.21.15.181:6789/0 conn(0x7f86c415da70 legacy=0x7f86c40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c27fc700 1 -- 172.21.15.181:0/2206027140 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4071c80 con 0x7f86c4072760 2023-10-16T11:01:13.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.685+0000 7f86c27fc700 1 --2- 172.21.15.181:0/2206027140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4072760 0x7f86c406a8a0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009840 tx=0x7f86b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bd380d04619948 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:13.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c1ffb700 1 -- 172.21.15.181:0/2206027140 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4011830 con 0x7f86c4072760 2023-10-16T11:01:13.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c1ffb700 1 -- 172.21.15.181:0/2206027140 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b4011990 con 0x7f86c4072760 2023-10-16T11:01:13.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c1ffb700 1 -- 172.21.15.181:0/2206027140 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4011d20 con 0x7f86c4072760 2023-10-16T11:01:13.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 -- 172.21.15.181:0/2206027140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4072760 msgr2=0x7f86c406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 --2- 172.21.15.181:0/2206027140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4072760 0x7f86c406a8a0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009840 tx=0x7f86b4009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:13.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 -- 172.21.15.181:0/2206027140 shutdown_connections 2023-10-16T11:01:13.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 --2- 172.21.15.181:0/2206027140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c406ade0 0x7f86c41615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 --2- 172.21.15.181:0/2206027140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c4072760 0x7f86c406a8a0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 -- 172.21.15.181:0/2206027140 >> 172.21.15.181:0/2206027140 conn(0x7f86c40cd010 msgr2=0x7f86c40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:13.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 -- 172.21.15.181:0/2206027140 shutdown_connections 2023-10-16T11:01:13.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.686+0000 7f86c9944700 1 -- 172.21.15.181:0/2206027140 wait complete. 2023-10-16T11:01:13.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.687+0000 7f86c9944700 1 Processor -- start 2023-10-16T11:01:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c9944700 1 -- start start 2023-10-16T11:01:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c9944700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c406ade0 0x7f86c41f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c9944700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c4072760 0x7f86c40b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c9944700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c415da70 0x7f86c40adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:13.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c9944700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b4cb0 con 0x7f86c406ade0 2023-10-16T11:01:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c9944700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b4410 con 0x7f86c4072760 2023-10-16T11:01:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c9944700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b46f0 con 0x7f86c415da70 2023-10-16T11:01:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c406ade0 0x7f86c41f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c406ade0 0x7f86c41f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51790/0 (socket says 172.21.15.181:51790) 2023-10-16T11:01:13.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.688+0000 7f86c2ffd700 1 -- 172.21.15.181:0/670472670 learned_addr learned my addr 172.21.15.181:0/670472670 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:13.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c27fc700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c4072760 0x7f86c40b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:13.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c37fe700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c415da70 0x7f86c40adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:13.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c2ffd700 1 -- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c415da70 msgr2=0x7f86c40adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c2ffd700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c415da70 0x7f86c40adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c2ffd700 1 -- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c4072760 msgr2=0x7f86c40b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c2ffd700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c4072760 0x7f86c40b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c2ffd700 1 -- 172.21.15.181:0/670472670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b40094a0 con 0x7f86c406ade0 2023-10-16T11:01:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c2ffd700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c406ade0 0x7f86c41f3b80 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f86b8001eb0 tx=0x7f86b800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86abfff700 1 -- 172.21.15.181:0/670472670 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8005d70 con 0x7f86c406ade0 2023-10-16T11:01:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86abfff700 1 -- 172.21.15.181:0/670472670 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80070d0 con 0x7f86c406ade0 2023-10-16T11:01:13.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c40b49d0 con 0x7f86c406ade0 2023-10-16T11:01:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86abfff700 1 -- 172.21.15.181:0/670472670 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8007460 con 0x7f86c406ade0 2023-10-16T11:01:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.689+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c40b40d0 con 0x7f86c406ade0 2023-10-16T11:01:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.690+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c41f4150 con 0x7f86c406ade0 2023-10-16T11:01:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.690+0000 7f86abfff700 1 -- 172.21.15.181:0/670472670 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 107) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f86b80075c0 con 0x7f86c406ade0 2023-10-16T11:01:13.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.691+0000 7f86abfff700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f86b006af90 0x7f86b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:13.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.691+0000 7f86abfff700 1 -- 172.21.15.181:0/670472670 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f86b8088ca0 con 0x7f86c406ade0 2023-10-16T11:01:13.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.691+0000 7f86c27fc700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f86b006af90 0x7f86b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:13.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.691+0000 7f86c27fc700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f86b006af90 0x7f86b006d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f86b4000c00 tx=0x7f86b4008030 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:13.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.695+0000 7f86abfff700 1 -- 172.21.15.181:0/670472670 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f86c41f4150 con 0x7f86c406ade0 2023-10-16T11:01:13.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.888+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f86c41f4530 con 0x7f86c406ade0 2023-10-16T11:01:13.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.892+0000 7f86abfff700 1 -- 172.21.15.181:0/670472670 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v61) v1 ==== 115+0+0 (secure 0 0 0) 0x7f86c41f4530 con 0x7f86c406ade0 2023-10-16T11:01:13.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f86b006af90 msgr2=0x7f86b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f86b006af90 0x7f86b006d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f86b4000c00 tx=0x7f86b4008030 comp rx=0 tx=0).stop 2023-10-16T11:01:13.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c406ade0 msgr2=0x7f86c41f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:13.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c406ade0 0x7f86c41f3b80 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f86b8001eb0 tx=0x7f86b800c860 comp rx=0 tx=0).stop 2023-10-16T11:01:13.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 shutdown_connections 2023-10-16T11:01:13.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f86b006af90 0x7f86b006d450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c415da70 0x7f86c40adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86c406ade0 0x7f86c41f3b80 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 --2- 172.21.15.181:0/670472670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c4072760 0x7f86c40b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:13.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 >> 172.21.15.181:0/670472670 conn(0x7f86c40cd010 msgr2=0x7f86c40cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:13.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 shutdown_connections 2023-10-16T11:01:13.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:13.894+0000 7f86c9944700 1 -- 172.21.15.181:0/670472670 wait complete. 2023-10-16T11:01:13.909 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:01:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.030+0000 7ff0d9711700 1 Processor -- start 2023-10-16T11:01:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.030+0000 7ff0d9711700 1 -- start start 2023-10-16T11:01:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d9711700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff0d40b2010 0x7ff0d40cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d9711700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d40ce1f0 con 0x7ff0d40ce680 2023-10-16T11:01:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d9711700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d4072090 con 0x7ff0d4071be0 2023-10-16T11:01:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d9711700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d40ccda0 con 0x7ff0d40b2010 2023-10-16T11:01:14.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff0d40b2010 0x7ff0d40cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff0d40ce680 0x7ff0d40ceaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34648/0 (socket says 172.21.15.181:34648) 2023-10-16T11:01:14.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d37fe700 1 -- 172.21.15.181:0/1801601076 learned_addr learned my addr 172.21.15.181:0/1801601076 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:14.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d2ffd700 1 -- 172.21.15.181:0/1801601076 >> v1:172.21.15.190:6789/0 conn(0x7ff0d4071be0 legacy=0x7ff0d40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d2ffd700 1 -- 172.21.15.181:0/1801601076 >> v1:172.21.15.181:6789/0 conn(0x7ff0d40ce680 legacy=0x7ff0d40ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d2ffd700 1 -- 172.21.15.181:0/1801601076 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40727e0 con 0x7ff0d40b2010 2023-10-16T11:01:14.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d1ffb700 1 -- 172.21.15.181:0/1801601076 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472462183 0 0) 0x7ff0d40ce1f0 con 0x7ff0d40ce680 2023-10-16T11:01:14.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d2ffd700 1 --2- 172.21.15.181:0/1801601076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40b2010 0x7ff0d40cc6e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009840 tx=0x7ff0c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9929d1aeaf5c60 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d1ffb700 1 -- 172.21.15.181:0/1801601076 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011830 con 0x7ff0d40b2010 2023-10-16T11:01:14.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d1ffb700 1 -- 172.21.15.181:0/1801601076 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4011990 con 0x7ff0d40b2010 2023-10-16T11:01:14.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.031+0000 7ff0d1ffb700 1 -- 172.21.15.181:0/1801601076 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011d20 con 0x7ff0d40b2010 2023-10-16T11:01:14.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.032+0000 7ff0d9711700 1 -- 172.21.15.181:0/1801601076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40b2010 msgr2=0x7ff0d40cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.032+0000 7ff0d9711700 1 --2- 172.21.15.181:0/1801601076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40b2010 0x7ff0d40cc6e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009840 tx=0x7ff0c4009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:14.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.032+0000 7ff0d9711700 1 -- 172.21.15.181:0/1801601076 shutdown_connections 2023-10-16T11:01:14.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.032+0000 7ff0d9711700 1 --2- 172.21.15.181:0/1801601076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40b2010 0x7ff0d40cc6e0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.032+0000 7ff0d9711700 1 -- 172.21.15.181:0/1801601076 >> 172.21.15.181:0/1801601076 conn(0x7ff0d40cd010 msgr2=0x7ff0d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:14.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.032+0000 7ff0d9711700 1 -- 172.21.15.181:0/1801601076 shutdown_connections 2023-10-16T11:01:14.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.032+0000 7ff0d9711700 1 -- 172.21.15.181:0/1801601076 wait complete. 2023-10-16T11:01:14.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.033+0000 7ff0d9711700 1 Processor -- start 2023-10-16T11:01:14.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.033+0000 7ff0d9711700 1 -- start start 2023-10-16T11:01:14.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d9711700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d4071be0 0x7ff0d4150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d9711700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40b2010 0x7ff0d40ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d9711700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40ce680 0x7ff0d40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d9711700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d415d8d0 con 0x7ff0d4071be0 2023-10-16T11:01:14.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d9711700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40b1930 con 0x7ff0d40b2010 2023-10-16T11:01:14.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d9711700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40ac880 con 0x7ff0d40ce680 2023-10-16T11:01:14.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d4071be0 0x7ff0d4150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d4071be0 0x7ff0d4150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51804/0 (socket says 172.21.15.181:51804) 2023-10-16T11:01:14.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d2ffd700 1 -- 172.21.15.181:0/2595580359 learned_addr learned my addr 172.21.15.181:0/2595580359 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:14.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d37fe700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40ce680 0x7ff0d40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.034+0000 7ff0d27fc700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40b2010 0x7ff0d40ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.035+0000 7ff0d2ffd700 1 -- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40ce680 msgr2=0x7ff0d40cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.035+0000 7ff0d2ffd700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40ce680 0x7ff0d40cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.035+0000 7ff0d2ffd700 1 -- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40b2010 msgr2=0x7ff0d40ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.035+0000 7ff0d2ffd700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40b2010 0x7ff0d40ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.035+0000 7ff0d2ffd700 1 -- 172.21.15.181:0/2595580359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40094a0 con 0x7ff0d4071be0 2023-10-16T11:01:14.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.035+0000 7ff0d2ffd700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d4071be0 0x7ff0d4150bc0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4014040 tx=0x7ff0c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.039+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2595580359 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c401a7e0 con 0x7ff0d4071be0 2023-10-16T11:01:14.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.039+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d40acb60 con 0x7ff0d4071be0 2023-10-16T11:01:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.039+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2595580359 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c401a940 con 0x7ff0d4071be0 2023-10-16T11:01:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.039+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2595580359 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c401acb0 con 0x7ff0d4071be0 2023-10-16T11:01:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.039+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d40ad120 con 0x7ff0d4071be0 2023-10-16T11:01:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.041+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2595580359 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff0c4022070 con 0x7ff0d4071be0 2023-10-16T11:01:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.041+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d40b6510 con 0x7ff0d4071be0 2023-10-16T11:01:14.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.041+0000 7ff0bb7fe700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff0c006b020 0x7ff0c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.041+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2595580359 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7ff0c408a5f0 con 0x7ff0d4071be0 2023-10-16T11:01:14.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.041+0000 7ff0d27fc700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff0c006b020 0x7ff0c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.042+0000 7ff0d27fc700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff0c006b020 0x7ff0c006d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40b56e0 tx=0x7ff0c8009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.046+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2595580359 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0c40554b0 con 0x7ff0d4071be0 2023-10-16T11:01:14.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.239+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ff0d4074db0 con 0x7ff0d4071be0 2023-10-16T11:01:14.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.242+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2595580359 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v62) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff0c4058ad0 con 0x7ff0d4071be0 2023-10-16T11:01:14.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff0c006b020 msgr2=0x7ff0c006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff0c006b020 0x7ff0c006d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40b56e0 tx=0x7ff0c8009040 comp rx=0 tx=0).stop 2023-10-16T11:01:14.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d4071be0 msgr2=0x7ff0d4150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d4071be0 0x7ff0d4150bc0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4014040 tx=0x7ff0c4004750 comp rx=0 tx=0).stop 2023-10-16T11:01:14.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 shutdown_connections 2023-10-16T11:01:14.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff0c006b020 0x7ff0c006d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40ce680 0x7ff0d40cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d4071be0 0x7ff0d4150bc0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 --2- 172.21.15.181:0/2595580359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40b2010 0x7ff0d40ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 >> 172.21.15.181:0/2595580359 conn(0x7ff0d40cd010 msgr2=0x7ff0d40cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:14.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 shutdown_connections 2023-10-16T11:01:14.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.245+0000 7ff0d9711700 1 -- 172.21.15.181:0/2595580359 wait complete. 2023-10-16T11:01:14.259 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:01:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.380+0000 7f039bef5700 1 Processor -- start 2023-10-16T11:01:14.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.380+0000 7f039bef5700 1 -- start start 2023-10-16T11:01:14.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f039bef5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 0x7f03940cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f039bef5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0394071be0 0x7f03940b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f039bef5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03940ce1f0 con 0x7f039406f3b0 2023-10-16T11:01:14.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f039bef5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03940b2930 con 0x7f0394071be0 2023-10-16T11:01:14.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f039bef5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03940ccda0 con 0x7f03940b2010 2023-10-16T11:01:14.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f039a492700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f039406f3b0 0x7f039406f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34660/0 (socket says 172.21.15.181:34660) 2023-10-16T11:01:14.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f039a492700 1 -- 172.21.15.181:0/4268805176 learned_addr learned my addr 172.21.15.181:0/4268805176 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:14.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365921175 0 0) 0x7f03940ce1f0 con 0x7f039406f3b0 2023-10-16T11:01:14.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f0399490700 1 --2- 172.21.15.181:0/4268805176 >> v2:172.21.15.190:3300/0 conn(0x7f0394071be0 0x7f03940b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03800035b0 con 0x7f039406f3b0 2023-10-16T11:01:14.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.381+0000 7f0399c91700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 0x7f03940cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2848951140 0 0) 0x7f03800035b0 con 0x7f039406f3b0 2023-10-16T11:01:14.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 msgr2=0x7f03940cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:14.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 0x7f03940cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 >> v2:172.21.15.190:3300/0 conn(0x7f0394071be0 msgr2=0x7f03940b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:14.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 --2- 172.21.15.181:0/4268805176 >> v2:172.21.15.190:3300/0 conn(0x7f0394071be0 0x7f03940b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940727e0 con 0x7f039406f3b0 2023-10-16T11:01:14.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f038c0019f0 con 0x7f039406f3b0 2023-10-16T11:01:14.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 >> v1:172.21.15.181:6789/0 conn(0x7f039406f3b0 legacy=0x7f039406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 0x7f0380003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0380003f80 0x7f0380004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380004d90 0x7f0380005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940b2930 con 0x7f0380003f80 2023-10-16T11:01:14.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940ccda0 con 0x7f0380004d90 2023-10-16T11:01:14.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038000ba60 con 0x7f03940b2010 2023-10-16T11:01:14.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399490700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0380003f80 0x7f0380004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f039a492700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380004d90 0x7f0380005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399c91700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 0x7f0380003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399490700 1 -- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 msgr2=0x7f0380003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:14.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399490700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03940b2010 0x7f0380003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399490700 1 -- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380004d90 msgr2=0x7f0380005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:14.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399490700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380004d90 0x7f0380005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399490700 1 -- 172.21.15.181:0/4268805176 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03800035b0 con 0x7f0380003f80 2023-10-16T11:01:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f039a492700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380004d90 0x7f0380005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0399490700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0380003f80 0x7f0380004850 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f0390009b60 tx=0x7f0380005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381663d22b8e4f2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000e6c0 con 0x7f0380003f80 2023-10-16T11:01:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000e820 con 0x7f0380003f80 2023-10-16T11:01:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.382+0000 7f0398c8f700 1 -- 172.21.15.181:0/4268805176 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000eb90 con 0x7f0380003f80 2023-10-16T11:01:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.383+0000 7f039bef5700 1 -- 172.21.15.181:0/4268805176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0380003f80 msgr2=0x7f0380004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.383+0000 7f039bef5700 1 --2- 172.21.15.181:0/4268805176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0380003f80 0x7f0380004850 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f0390009b60 tx=0x7f0380005880 comp rx=0 tx=0).stop 2023-10-16T11:01:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.383+0000 7f039a492700 1 -- 172.21.15.181:0/4268805176 reap_dead start 2023-10-16T11:01:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.383+0000 7f039bef5700 1 -- 172.21.15.181:0/4268805176 shutdown_connections 2023-10-16T11:01:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.383+0000 7f039bef5700 1 -- 172.21.15.181:0/4268805176 >> 172.21.15.181:0/4268805176 conn(0x7f03940cd010 msgr2=0x7f03940cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.383+0000 7f039bef5700 1 -- 172.21.15.181:0/4268805176 shutdown_connections 2023-10-16T11:01:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.383+0000 7f039bef5700 1 -- 172.21.15.181:0/4268805176 wait complete. 2023-10-16T11:01:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.384+0000 7f039bef5700 1 Processor -- start 2023-10-16T11:01:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.384+0000 7f039bef5700 1 -- start start 2023-10-16T11:01:14.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f039bef5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380003f80 0x7f039414be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f039bef5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039414b690 0x7f039414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f039bef5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f039414e440 0x7f03940c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f039bef5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039415cb10 con 0x7f039414b690 2023-10-16T11:01:14.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f039bef5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039415ad20 con 0x7f0380003f80 2023-10-16T11:01:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f039bef5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039414ea70 con 0x7f039414e440 2023-10-16T11:01:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039414b690 0x7f039414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039414b690 0x7f039414df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51826/0 (socket says 172.21.15.181:51826) 2023-10-16T11:01:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 -- 172.21.15.181:0/2287554753 learned_addr learned my addr 172.21.15.181:0/2287554753 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399c91700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380003f80 0x7f039414be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f039a492700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f039414e440 0x7f03940c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 -- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f039414e440 msgr2=0x7f03940c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f039414e440 0x7f03940c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 -- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380003f80 msgr2=0x7f039414be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380003f80 0x7f039414be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 -- 172.21.15.181:0/2287554753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039414ed50 con 0x7f039414b690 2023-10-16T11:01:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.385+0000 7f0399490700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039414b690 0x7f039414df00 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f0390000c00 tx=0x7f03900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.386+0000 7f038a7fc700 1 -- 172.21.15.181:0/2287554753 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390021070 con 0x7f039414b690 2023-10-16T11:01:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.386+0000 7f038a7fc700 1 -- 172.21.15.181:0/2287554753 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03900043f0 con 0x7f039414b690 2023-10-16T11:01:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.386+0000 7f038a7fc700 1 -- 172.21.15.181:0/2287554753 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000e6c0 con 0x7f039414b690 2023-10-16T11:01:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.386+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039415c730 con 0x7f039414b690 2023-10-16T11:01:14.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.386+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039415c350 con 0x7f039414b690 2023-10-16T11:01:14.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.387+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039414e820 con 0x7f039414b690 2023-10-16T11:01:14.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.387+0000 7f038a7fc700 1 -- 172.21.15.181:0/2287554753 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0390025070 con 0x7f039414b690 2023-10-16T11:01:14.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.388+0000 7f038a7fc700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f037806b020 0x7f037806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.388+0000 7f0399c91700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f037806b020 0x7f037806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.388+0000 7f038a7fc700 1 -- 172.21.15.181:0/2287554753 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f039008aa50 con 0x7f039414b690 2023-10-16T11:01:14.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.388+0000 7f0399c91700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f037806b020 0x7f037806d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f03840043b0 tx=0x7f0384008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.392+0000 7f038a7fc700 1 -- 172.21.15.181:0/2287554753 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0390059370 con 0x7f039414b690 2023-10-16T11:01:14.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.591+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f039415c540 con 0x7f039414b690 2023-10-16T11:01:14.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.594+0000 7f038a7fc700 1 -- 172.21.15.181:0/2287554753 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0390058f00 con 0x7f039414b690 2023-10-16T11:01:14.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f037806b020 msgr2=0x7f037806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f037806b020 0x7f037806d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f03840043b0 tx=0x7f0384008040 comp rx=0 tx=0).stop 2023-10-16T11:01:14.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039414b690 msgr2=0x7f039414df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039414b690 0x7f039414df00 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f0390000c00 tx=0x7f03900047c0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 shutdown_connections 2023-10-16T11:01:14.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f037806b020 0x7f037806d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f039414e440 0x7f03940c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0380003f80 0x7f039414be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 --2- 172.21.15.181:0/2287554753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039414b690 0x7f039414df00 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.596+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 >> 172.21.15.181:0/2287554753 conn(0x7f03940cd010 msgr2=0x7f03940afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:14.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.597+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 shutdown_connections 2023-10-16T11:01:14.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.597+0000 7f039bef5700 1 -- 172.21.15.181:0/2287554753 wait complete. 2023-10-16T11:01:14.611 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-16T11:01:14.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.732+0000 7f7f61b35700 1 Processor -- start 2023-10-16T11:01:14.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.732+0000 7f7f61b35700 1 -- start start 2023-10-16T11:01:14.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.733+0000 7f7f61b35700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0af450 0x7f7f5c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.733+0000 7f7f61b35700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f5c0cd470 0x7f7f5c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.733+0000 7f7f61b35700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c06f870 con 0x7f7f5c15dfd0 2023-10-16T11:01:14.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.733+0000 7f7f61b35700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5c0d1160 con 0x7f7f5c0cd470 2023-10-16T11:01:14.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.733+0000 7f7f61b35700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0cc8e0 con 0x7f7f5c0af450 2023-10-16T11:01:14.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0af450 0x7f7f5c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5bfff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f5c0cd470 0x7f7f5c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0af450 0x7f7f5c0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:48860/0 (socket says 172.21.15.181:48860) 2023-10-16T11:01:14.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5affd700 1 -- 172.21.15.181:0/566074405 learned_addr learned my addr 172.21.15.181:0/566074405 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:14.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5b7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f7f5c15dfd0 0x7f7f5c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:46852/0 (socket says 172.21.15.181:46852) 2023-10-16T11:01:14.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5bfff700 1 -- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0af450 msgr2=0x7f7f5c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:14.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5bfff700 1 --2- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0af450 0x7f7f5c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5bfff700 1 -- 172.21.15.181:0/566074405 >> v1:172.21.15.190:6789/0 conn(0x7f7f5c15dfd0 legacy=0x7f7f5c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5bfff700 1 -- 172.21.15.181:0/566074405 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0ccbc0 con 0x7f7f5c0cd470 2023-10-16T11:01:14.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5affd700 1 --2- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0af450 0x7f7f5c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:14.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5bfff700 1 --2- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c0cd470 0x7f7f5c0cd920 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009d90 tx=0x7f7f5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f021c812b33fa66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5a7fc700 1 -- 172.21.15.181:0/566074405 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400e070 con 0x7f7f5c0cd470 2023-10-16T11:01:14.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5a7fc700 1 -- 172.21.15.181:0/566074405 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5400bb20 con 0x7f7f5c0cd470 2023-10-16T11:01:14.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.734+0000 7f7f5a7fc700 1 -- 172.21.15.181:0/566074405 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54012430 con 0x7f7f5c0cd470 2023-10-16T11:01:14.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 -- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c0cd470 msgr2=0x7f7f5c0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 --2- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c0cd470 0x7f7f5c0cd920 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009d90 tx=0x7f7f5400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:14.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 -- 172.21.15.181:0/566074405 shutdown_connections 2023-10-16T11:01:14.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 --2- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c0cd470 0x7f7f5c0cd920 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 --2- 172.21.15.181:0/566074405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0af450 0x7f7f5c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 -- 172.21.15.181:0/566074405 >> 172.21.15.181:0/566074405 conn(0x7f7f5c0d04a0 msgr2=0x7f7f5c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:14.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 -- 172.21.15.181:0/566074405 shutdown_connections 2023-10-16T11:01:14.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.735+0000 7f7f61b35700 1 -- 172.21.15.181:0/566074405 wait complete. 2023-10-16T11:01:14.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.736+0000 7f7f61b35700 1 Processor -- start 2023-10-16T11:01:14.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.736+0000 7f7f61b35700 1 -- start start 2023-10-16T11:01:14.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f61b35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f5c0af450 0x7f7f5c0b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f61b35700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0cd470 0x7f7f5c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f61b35700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c15dfd0 0x7f7f5c0c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f61b35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0c5b40 con 0x7f7f5c0af450 2023-10-16T11:01:14.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f61b35700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0c5e20 con 0x7f7f5c15dfd0 2023-10-16T11:01:14.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f61b35700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0c6100 con 0x7f7f5c0cd470 2023-10-16T11:01:14.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f5c0af450 0x7f7f5c0b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f5c0af450 0x7f7f5c0b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51838/0 (socket says 172.21.15.181:51838) 2023-10-16T11:01:14.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f5b7fe700 1 -- 172.21.15.181:0/3575938424 learned_addr learned my addr 172.21.15.181:0/3575938424 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:14.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f5affd700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0cd470 0x7f7f5c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.737+0000 7f7f5bfff700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c15dfd0 0x7f7f5c0c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f5b7fe700 1 -- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0cd470 msgr2=0x7f7f5c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f5b7fe700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0cd470 0x7f7f5c0c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f5b7fe700 1 -- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c15dfd0 msgr2=0x7f7f5c0c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:14.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f5b7fe700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c15dfd0 0x7f7f5c0c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:14.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f5b7fe700 1 -- 172.21.15.181:0/3575938424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c009550 con 0x7f7f5c0af450 2023-10-16T11:01:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f5b7fe700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f5c0af450 0x7f7f5c0b2910 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000c460 tx=0x7f7f5000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f43fff700 1 -- 172.21.15.181:0/3575938424 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50005d00 con 0x7f7f5c0af450 2023-10-16T11:01:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f43fff700 1 -- 172.21.15.181:0/3575938424 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f50005e60 con 0x7f7f5c0af450 2023-10-16T11:01:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f54009a70 con 0x7f7f5c0af450 2023-10-16T11:01:14.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f43fff700 1 -- 172.21.15.181:0/3575938424 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50007310 con 0x7f7f5c0af450 2023-10-16T11:01:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.738+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c0c66c0 con 0x7f7f5c0af450 2023-10-16T11:01:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.739+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c0cf880 con 0x7f7f5c0af450 2023-10-16T11:01:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.739+0000 7f7f43fff700 1 -- 172.21.15.181:0/3575938424 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7f50007470 con 0x7f7f5c0af450 2023-10-16T11:01:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.740+0000 7f7f43fff700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7f4806b070 0x7f7f4806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:14.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.740+0000 7f7f43fff700 1 -- 172.21.15.181:0/3575938424 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f7f50088b90 con 0x7f7f5c0af450 2023-10-16T11:01:14.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.740+0000 7f7f5affd700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7f4806b070 0x7f7f4806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:14.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.740+0000 7f7f5affd700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7f4806b070 0x7f7f4806d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0b5d70 tx=0x7f7f4c0092c0 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:14.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:14.744+0000 7f7f43fff700 1 -- 172.21.15.181:0/3575938424 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7f50053a70 con 0x7f7f5c0af450 2023-10-16T11:01:15.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.018+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f7f5c067d20 con 0x7f7f5c0af450 2023-10-16T11:01:15.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.549+0000 7f7f43fff700 1 -- 172.21.15.181:0/3575938424 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2522) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7f50017020 con 0x7f7f5c0af450 2023-10-16T11:01:15.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7f4806b070 msgr2=0x7f7f4806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7f4806b070 0x7f7f4806d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0b5d70 tx=0x7f7f4c0092c0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f5c0af450 msgr2=0x7f7f5c0b2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f5c0af450 0x7f7f5c0b2910 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000c460 tx=0x7f7f5000c770 comp rx=0 tx=0).stop 2023-10-16T11:01:15.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 shutdown_connections 2023-10-16T11:01:15.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7f4806b070 0x7f7f4806d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f5c15dfd0 0x7f7f5c0c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f5c0af450 0x7f7f5c0b2910 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 --2- 172.21.15.181:0/3575938424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f5c0cd470 0x7f7f5c0c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 >> 172.21.15.181:0/3575938424 conn(0x7f7f5c0d04a0 msgr2=0x7f7f5c06d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:15.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 shutdown_connections 2023-10-16T11:01:15.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.552+0000 7f7f61b35700 1 -- 172.21.15.181:0/3575938424 wait complete. 2023-10-16T11:01:15.568 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-16T11:01:15.569 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-16T11:01:15.569 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T11:01:15.570 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:15.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.694+0000 7fef6d7e7700 1 Processor -- start 2023-10-16T11:01:15.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.694+0000 7fef6d7e7700 1 -- start start 2023-10-16T11:01:15.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef6d7e7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68161b60 0x7fef6806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:15.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef6d7e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef68161f40 0x7fef680d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef6d7e7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680d2560 con 0x7fef68160d20 2023-10-16T11:01:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef6d7e7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef68162800 con 0x7fef68161f40 2023-10-16T11:01:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef6d7e7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef680d43e0 con 0x7fef68161b60 2023-10-16T11:01:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef667fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef68161f40 0x7fef680d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef667fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef68161f40 0x7fef680d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51846/0 (socket says 172.21.15.181:51846) 2023-10-16T11:01:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef667fc700 1 -- 172.21.15.181:0/1420171646 learned_addr learned my addr 172.21.15.181:0/1420171646 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef66ffd700 1 --2- 172.21.15.181:0/1420171646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68161b60 0x7fef6806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef667fc700 1 -- 172.21.15.181:0/1420171646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68161b60 msgr2=0x7fef6806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:15.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef667fc700 1 --2- 172.21.15.181:0/1420171646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68161b60 0x7fef6806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef667fc700 1 -- 172.21.15.181:0/1420171646 >> v1:172.21.15.190:6789/0 conn(0x7fef68160d20 legacy=0x7fef680b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.695+0000 7fef667fc700 1 -- 172.21.15.181:0/1420171646 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6806ce20 con 0x7fef68161f40 2023-10-16T11:01:15.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef667fc700 1 --2- 172.21.15.181:0/1420171646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680d2020 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fef580098f0 tx=0x7fef58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229bf5754fb0799e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:15.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef65ffb700 1 -- 172.21.15.181:0/1420171646 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800b8a0 con 0x7fef68161f40 2023-10-16T11:01:15.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef65ffb700 1 -- 172.21.15.181:0/1420171646 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5800ba00 con 0x7fef68161f40 2023-10-16T11:01:15.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef65ffb700 1 -- 172.21.15.181:0/1420171646 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800bd90 con 0x7fef68161f40 2023-10-16T11:01:15.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1420171646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 msgr2=0x7fef680d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1420171646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680d2020 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fef580098f0 tx=0x7fef58009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:15.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1420171646 shutdown_connections 2023-10-16T11:01:15.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1420171646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68161b60 0x7fef6806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1420171646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680d2020 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1420171646 >> 172.21.15.181:0/1420171646 conn(0x7fef6809cba0 msgr2=0x7fef680d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:15.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1420171646 shutdown_connections 2023-10-16T11:01:15.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.696+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1420171646 wait complete. 2023-10-16T11:01:15.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 Processor -- start 2023-10-16T11:01:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 -- start start 2023-10-16T11:01:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68160d20 0x7fef681f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef68161b60 0x7fef680c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:15.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680cb920 con 0x7fef68161f40 2023-10-16T11:01:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680cbb70 con 0x7fef68161b60 2023-10-16T11:01:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef6d7e7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef680cbe50 con 0x7fef68160d20 2023-10-16T11:01:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:15.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef677fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51854/0 (socket says 172.21.15.181:51854) 2023-10-16T11:01:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef677fe700 1 -- 172.21.15.181:0/1367667982 learned_addr learned my addr 172.21.15.181:0/1367667982 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:15.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.698+0000 7fef667fc700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef68161b60 0x7fef680c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef66ffd700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68160d20 0x7fef681f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef677fe700 1 -- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68160d20 msgr2=0x7fef681f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef677fe700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68160d20 0x7fef681f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef677fe700 1 -- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef68161b60 msgr2=0x7fef680c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef677fe700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef68161b60 0x7fef680c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef677fe700 1 -- 172.21.15.181:0/1367667982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58009550 con 0x7fef68161f40 2023-10-16T11:01:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef677fe700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680cc930 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fef6000caa0 tx=0x7fef6000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef4f7fe700 1 -- 172.21.15.181:0/1367667982 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60003c00 con 0x7fef68161f40 2023-10-16T11:01:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef4f7fe700 1 -- 172.21.15.181:0/1367667982 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60003d60 con 0x7fef68161f40 2023-10-16T11:01:15.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef680cc130 con 0x7fef68161f40 2023-10-16T11:01:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef4f7fe700 1 -- 172.21.15.181:0/1367667982 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000aa20 con 0x7fef68161f40 2023-10-16T11:01:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.699+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6815d510 con 0x7fef68161f40 2023-10-16T11:01:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.700+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef680cc650 con 0x7fef68161f40 2023-10-16T11:01:15.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.701+0000 7fef4f7fe700 1 -- 172.21.15.181:0/1367667982 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef6000ab80 con 0x7fef68161f40 2023-10-16T11:01:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.701+0000 7fef4f7fe700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fef5406b020 0x7fef5406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.701+0000 7fef4f7fe700 1 -- 172.21.15.181:0/1367667982 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7fef60089770 con 0x7fef68161f40 2023-10-16T11:01:15.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.701+0000 7fef66ffd700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fef5406b020 0x7fef5406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:15.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.701+0000 7fef66ffd700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fef5406b020 0x7fef5406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:15.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.706+0000 7fef4f7fe700 1 -- 172.21.15.181:0/1367667982 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fef680cc650 con 0x7fef68161f40 2023-10-16T11:01:15.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.900+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef680cc650 con 0x7fef68161f40 2023-10-16T11:01:15.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.901+0000 7fef4f7fe700 1 -- 172.21.15.181:0/1367667982 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+14808 (secure 0 0 0) 0x7fef680cc650 con 0x7fef68161f40 2023-10-16T11:01:15.902 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:15.903 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":173,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:13.543182+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/2931073086":"2023-10-17T11:01:10.511518+0000","172.21.15.181:6835/2931073086":"2023-10-17T11:01:10.511518+0000","172.21.15.181:0/1993379789":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/683890550":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/1832107878":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/314629984":"2023-10-17T11:01:09.519680+0000","172.21.15.181:6833/4215027768":"2023-10-17T11:01:09.519680+0000","172.21.15.181:6832/4215027768":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/3226890773":"2023-10-17T11:01:09.519680+0000","172.21.15.181:6833/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/1134301943":"2023-10-17T11:01:09.519680+0000","172.21.15.181:0/642481654":"2023-10-17T11:01:00.993294+0000","172.21.15.181:6832/551790976":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/120971111":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/3800812080":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/2485734134":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/4030419984":"2023-10-17T11:01:00.993294+0000","172.21.15.181:0/1244048650":"2023-10-17T11:01:00.993294+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fef5406b020 msgr2=0x7fef5406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fef5406b020 0x7fef5406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c008040 comp rx=0 tx=0).stop 2023-10-16T11:01:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 msgr2=0x7fef680cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680cc930 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fef6000caa0 tx=0x7fef6000cdb0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 shutdown_connections 2023-10-16T11:01:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fef5406b020 0x7fef5406d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef68160d20 0x7fef681f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef68161f40 0x7fef680cc930 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 --2- 172.21.15.181:0/1367667982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef68161b60 0x7fef680c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:15.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 >> 172.21.15.181:0/1367667982 conn(0x7fef6809cba0 msgr2=0x7fef680d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:15.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 shutdown_connections 2023-10-16T11:01:15.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:15.903+0000 7fef6d7e7700 1 -- 172.21.15.181:0/1367667982 wait complete. 2023-10-16T11:01:15.917 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:01:15.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:01:16.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.039+0000 7f731de6a700 1 Processor -- start 2023-10-16T11:01:16.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.039+0000 7f731de6a700 1 -- start start 2023-10-16T11:01:16.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.040+0000 7f731de6a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f731806afb0 0x7f73181615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.040+0000 7f731de6a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73180cef00 con 0x7f731815da70 2023-10-16T11:01:16.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.040+0000 7f731de6a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73180cec20 con 0x7f7318072760 2023-10-16T11:01:16.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.040+0000 7f731de6a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7318071c80 con 0x7f731806afb0 2023-10-16T11:01:16.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.040+0000 7f73177fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f731815da70 0x7f73180ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34704/0 (socket says 172.21.15.181:34704) 2023-10-16T11:01:16.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.040+0000 7f73177fe700 1 -- 172.21.15.181:0/311956821 learned_addr learned my addr 172.21.15.181:0/311956821 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:16.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f7317fff700 1 --2- 172.21.15.181:0/311956821 >> v2:172.21.15.190:3300/0 conn(0x7f731806afb0 0x7f73181615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700043106 0 0) 0x7f73180cef00 con 0x7f731815da70 2023-10-16T11:01:16.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73040035b0 con 0x7f731815da70 2023-10-16T11:01:16.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892380038 0 0) 0x7f73040035b0 con 0x7f731815da70 2023-10-16T11:01:16.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 >> v2:172.21.15.190:3300/0 conn(0x7f731806afb0 msgr2=0x7f73181615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:16.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 --2- 172.21.15.181:0/311956821 >> v2:172.21.15.190:3300/0 conn(0x7f731806afb0 0x7f73181615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 >> v1:172.21.15.190:6789/0 conn(0x7f7318072760 legacy=0x7f731806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7318071f60 con 0x7f731815da70 2023-10-16T11:01:16.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7300001670 con 0x7f731815da70 2023-10-16T11:01:16.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f7317fff700 1 --2- 172.21.15.181:0/311956821 >> v2:172.21.15.190:3300/0 conn(0x7f731806afb0 0x7f73181615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:16.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 >> v1:172.21.15.181:6789/0 conn(0x7f731815da70 legacy=0x7f73180ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304003a00 0x7f7304003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304004850 0x7f7304004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7304005330 0x7f730400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7318071c80 con 0x7f7304003a00 2023-10-16T11:01:16.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730400c200 con 0x7f7304004850 2023-10-16T11:01:16.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730400c3c0 con 0x7f7304005330 2023-10-16T11:01:16.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73177fe700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304003a00 0x7f7304003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f7316ffd700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304004850 0x7f7304004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f7317fff700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7304005330 0x7f730400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73177fe700 1 -- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7304005330 msgr2=0x7f730400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:16.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73177fe700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7304005330 0x7f730400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73177fe700 1 -- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304004850 msgr2=0x7f7304004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:16.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73177fe700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304004850 0x7f7304004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f73177fe700 1 -- 172.21.15.181:0/311956821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040035b0 con 0x7f7304003a00 2023-10-16T11:01:16.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f7317fff700 1 -- 172.21.15.181:0/311956821 reap_dead start 2023-10-16T11:01:16.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.041+0000 7f7316ffd700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304004850 0x7f7304004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:16.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f73177fe700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304003a00 0x7f7304003e20 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f730000c210 tx=0x7f7304005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d831ee7e8b7d1a51 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:16.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300007c10 con 0x7f7304003a00 2023-10-16T11:01:16.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730000cd50 con 0x7f7304003a00 2023-10-16T11:01:16.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f73167fc700 1 -- 172.21.15.181:0/311956821 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300005ba0 con 0x7f7304003a00 2023-10-16T11:01:16.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f731de6a700 1 -- 172.21.15.181:0/311956821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304003a00 msgr2=0x7f7304003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f731de6a700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304003a00 0x7f7304003e20 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f730000c210 tx=0x7f7304005880 comp rx=0 tx=0).stop 2023-10-16T11:01:16.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f731de6a700 1 -- 172.21.15.181:0/311956821 shutdown_connections 2023-10-16T11:01:16.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f731de6a700 1 --2- 172.21.15.181:0/311956821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7304003a00 0x7f7304003e20 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f731de6a700 1 -- 172.21.15.181:0/311956821 >> 172.21.15.181:0/311956821 conn(0x7f73180cd010 msgr2=0x7f73180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f731de6a700 1 -- 172.21.15.181:0/311956821 shutdown_connections 2023-10-16T11:01:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.042+0000 7f731de6a700 1 -- 172.21.15.181:0/311956821 wait complete. 2023-10-16T11:01:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.043+0000 7f731de6a700 1 Processor -- start 2023-10-16T11:01:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f731de6a700 1 -- start start 2023-10-16T11:01:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f731de6a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304003a00 0x7f73180c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f731de6a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73180b32d0 0x7f73180b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f731de6a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73180b5850 0x7f73180b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f731de6a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180b3840 con 0x7f73180b32d0 2023-10-16T11:01:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f731de6a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180b3b20 con 0x7f7304003a00 2023-10-16T11:01:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f731de6a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73180b3e00 con 0x7f73180b5850 2023-10-16T11:01:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f7316ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73180b32d0 0x7f73180b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f7316ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73180b32d0 0x7f73180b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51872/0 (socket says 172.21.15.181:51872) 2023-10-16T11:01:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.044+0000 7f7316ffd700 1 -- 172.21.15.181:0/1570739690 learned_addr learned my addr 172.21.15.181:0/1570739690 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:16.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f73177fe700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304003a00 0x7f73180c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7317fff700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73180b5850 0x7f73180b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7316ffd700 1 -- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73180b5850 msgr2=0x7f73180b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7316ffd700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73180b5850 0x7f73180b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7316ffd700 1 -- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304003a00 msgr2=0x7f73180c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7316ffd700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304003a00 0x7f73180c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7316ffd700 1 -- 172.21.15.181:0/1570739690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180b40e0 con 0x7f73180b32d0 2023-10-16T11:01:16.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7317fff700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73180b5850 0x7f73180b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:16.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f73177fe700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304003a00 0x7f73180c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:16.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f7316ffd700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73180b32d0 0x7f73180b5310 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f730800e460 tx=0x7f730800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:16.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f730ffff700 1 -- 172.21.15.181:0/1570739690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080052e0 con 0x7f73180b32d0 2023-10-16T11:01:16.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f730ffff700 1 -- 172.21.15.181:0/1570739690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7308005440 con 0x7f73180b32d0 2023-10-16T11:01:16.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73180b43c0 con 0x7f73180b32d0 2023-10-16T11:01:16.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f730ffff700 1 -- 172.21.15.181:0/1570739690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080038c0 con 0x7f73180b32d0 2023-10-16T11:01:16.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.045+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73180b48e0 con 0x7f73180b32d0 2023-10-16T11:01:16.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.046+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73180b4730 con 0x7f73180b32d0 2023-10-16T11:01:16.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.046+0000 7f730ffff700 1 -- 172.21.15.181:0/1570739690 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7308003a20 con 0x7f73180b32d0 2023-10-16T11:01:16.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.047+0000 7f730ffff700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f72f806b020 0x7f72f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.047+0000 7f730ffff700 1 -- 172.21.15.181:0/1570739690 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f7308088ff0 con 0x7f73180b32d0 2023-10-16T11:01:16.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.047+0000 7f73177fe700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f72f806b020 0x7f72f806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.047+0000 7f73177fe700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f72f806b020 0x7f72f806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7300000c00 tx=0x7f7300007000 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:16.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.051+0000 7f730ffff700 1 -- 172.21.15.181:0/1570739690 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f73080bbfe0 con 0x7f73180b32d0 2023-10-16T11:01:16.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.243+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73180b4b00 con 0x7f73180b32d0 2023-10-16T11:01:16.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.244+0000 7f730ffff700 1 -- 172.21.15.181:0/1570739690 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7308019020 con 0x7f73180b32d0 2023-10-16T11:01:16.246 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:01:16.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f72f806b020 msgr2=0x7f72f806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f72f806b020 0x7f72f806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7300000c00 tx=0x7f7300007000 comp rx=0 tx=0).stop 2023-10-16T11:01:16.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73180b32d0 msgr2=0x7f73180b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73180b32d0 0x7f73180b5310 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f730800e460 tx=0x7f730800e820 comp rx=0 tx=0).stop 2023-10-16T11:01:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 shutdown_connections 2023-10-16T11:01:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f72f806b020 0x7f72f806d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73180b5850 0x7f73180b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7304003a00 0x7f73180c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 --2- 172.21.15.181:0/1570739690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73180b32d0 0x7f73180b5310 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 >> 172.21.15.181:0/1570739690 conn(0x7f73180cd010 msgr2=0x7f731815caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 shutdown_connections 2023-10-16T11:01:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.246+0000 7f731de6a700 1 -- 172.21.15.181:0/1570739690 wait complete. 2023-10-16T11:01:16.260 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-16T11:01:16.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.380+0000 7f21142c8700 1 Processor -- start 2023-10-16T11:01:16.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.380+0000 7f21142c8700 1 -- start start 2023-10-16T11:01:16.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f21142c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15e6d0 0x7f210c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f21142c8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f210c0cb0c0 0x7f210c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f21142c8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c06ed00 con 0x7f210c15eab0 2023-10-16T11:01:16.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f21142c8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c0d02d0 con 0x7f210c0cb0c0 2023-10-16T11:01:16.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f21142c8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c15ad40 con 0x7f210c15e6d0 2023-10-16T11:01:16.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f2111863700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f210c15eab0 0x7f210c15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34728/0 (socket says 172.21.15.181:34728) 2023-10-16T11:01:16.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f2111863700 1 -- 172.21.15.181:0/1607966571 learned_addr learned my addr 172.21.15.181:0/1607966571 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:16.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f2112865700 1 --2- 172.21.15.181:0/1607966571 >> v2:172.21.15.190:3300/0 conn(0x7f210c0cb0c0 0x7f210c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f2112064700 1 --2- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15e6d0 0x7f210c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f2111062700 1 -- 172.21.15.181:0/1607966571 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289519627 0 0) 0x7f210c06ed00 con 0x7f210c15eab0 2023-10-16T11:01:16.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.381+0000 7f2111062700 1 -- 172.21.15.181:0/1607966571 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20f80035b0 con 0x7f210c15eab0 2023-10-16T11:01:16.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2112865700 1 -- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15e6d0 msgr2=0x7f210c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:16.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2112865700 1 --2- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15e6d0 0x7f210c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2112865700 1 -- 172.21.15.181:0/1607966571 >> v1:172.21.15.181:6789/0 conn(0x7f210c15eab0 legacy=0x7f210c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2111062700 1 -- 172.21.15.181:0/1607966571 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132946938 0 0) 0x7f20f80035b0 con 0x7f210c15eab0 2023-10-16T11:01:16.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2112865700 1 -- 172.21.15.181:0/1607966571 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c06e0f0 con 0x7f210c0cb0c0 2023-10-16T11:01:16.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2112865700 1 --2- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c0cb0c0 0x7f210c15a800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2104009d90 tx=0x7f210400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c08ca1099f4cfd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:16.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2111062700 1 -- 172.21.15.181:0/1607966571 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210400e070 con 0x7f210c0cb0c0 2023-10-16T11:01:16.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2111062700 1 -- 172.21.15.181:0/1607966571 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2104003a20 con 0x7f210c0cb0c0 2023-10-16T11:01:16.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f2111062700 1 -- 172.21.15.181:0/1607966571 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108001670 con 0x7f210c0cb0c0 2023-10-16T11:01:16.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f21142c8700 1 -- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c0cb0c0 msgr2=0x7f210c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.382+0000 7f21142c8700 1 --2- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c0cb0c0 0x7f210c15a800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2104009d90 tx=0x7f210400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:16.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.383+0000 7f21142c8700 1 -- 172.21.15.181:0/1607966571 shutdown_connections 2023-10-16T11:01:16.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.383+0000 7f21142c8700 1 --2- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c0cb0c0 0x7f210c15a800 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.383+0000 7f21142c8700 1 --2- 172.21.15.181:0/1607966571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15e6d0 0x7f210c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.383+0000 7f21142c8700 1 -- 172.21.15.181:0/1607966571 >> 172.21.15.181:0/1607966571 conn(0x7f210c0d04a0 msgr2=0x7f210c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:16.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.383+0000 7f21142c8700 1 -- 172.21.15.181:0/1607966571 shutdown_connections 2023-10-16T11:01:16.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.383+0000 7f21142c8700 1 -- 172.21.15.181:0/1607966571 wait complete. 2023-10-16T11:01:16.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 Processor -- start 2023-10-16T11:01:16.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 -- start start 2023-10-16T11:01:16.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f210c0cb0c0 0x7f210c14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c15e6d0 0x7f210c0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15eab0 0x7f210c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0c5b70 con 0x7f210c0cb0c0 2023-10-16T11:01:16.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0c6130 con 0x7f210c15e6d0 2023-10-16T11:01:16.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.384+0000 7f21142c8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0b5a00 con 0x7f210c15eab0 2023-10-16T11:01:16.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f210c0cb0c0 0x7f210c14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f210c0cb0c0 0x7f210c14dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51886/0 (socket says 172.21.15.181:51886) 2023-10-16T11:01:16.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 -- 172.21.15.181:0/2476946286 learned_addr learned my addr 172.21.15.181:0/2476946286 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:16.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2111863700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c15e6d0 0x7f210c0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112865700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15eab0 0x7f210c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 -- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15eab0 msgr2=0x7f210c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15eab0 0x7f210c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 -- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c15e6d0 msgr2=0x7f210c0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:16.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c15e6d0 0x7f210c0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:16.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 -- 172.21.15.181:0/2476946286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2104009a70 con 0x7f210c0cb0c0 2023-10-16T11:01:16.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2111863700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c15e6d0 0x7f210c0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:16.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112865700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15eab0 0x7f210c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:16.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2112064700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f210c0cb0c0 0x7f210c14dcc0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00e900 tx=0x7f20fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:16.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2102ffd700 1 -- 172.21.15.181:0/2476946286 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc009650 con 0x7f210c0cb0c0 2023-10-16T11:01:16.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2102ffd700 1 -- 172.21.15.181:0/2476946286 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00c850 con 0x7f210c0cb0c0 2023-10-16T11:01:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c0b5ce0 con 0x7f210c0cb0c0 2023-10-16T11:01:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f2102ffd700 1 -- 172.21.15.181:0/2476946286 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00cbc0 con 0x7f210c0cb0c0 2023-10-16T11:01:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.385+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c0c5dc0 con 0x7f210c0cb0c0 2023-10-16T11:01:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.386+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c067d20 con 0x7f210c0cb0c0 2023-10-16T11:01:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.387+0000 7f2102ffd700 1 -- 172.21.15.181:0/2476946286 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f20fc00cd20 con 0x7f210c0cb0c0 2023-10-16T11:01:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.387+0000 7f2102ffd700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f20f006b020 0x7f20f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.387+0000 7f2111863700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f20f006b020 0x7f20f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.387+0000 7f2102ffd700 1 -- 172.21.15.181:0/2476946286 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f20fc089de0 con 0x7f210c0cb0c0 2023-10-16T11:01:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.388+0000 7f2111863700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f20f006b020 0x7f20f006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f210c155ca0 tx=0x7f2108009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:16.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.392+0000 7f2102ffd700 1 -- 172.21.15.181:0/2476946286 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f20fc054b60 con 0x7f210c0cb0c0 2023-10-16T11:01:16.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:16.663+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f210c0d1260 con 0x7f210c0cb0c0 2023-10-16T11:01:17.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.555+0000 7f2102ffd700 1 -- 172.21.15.181:0/2476946286 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2524) v1 ==== 168+0+0 (secure 0 0 0) 0x7f20fc058180 con 0x7f210c0cb0c0 2023-10-16T11:01:17.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.558+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f20f006b020 msgr2=0x7f20f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.558+0000 7f21142c8700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f20f006b020 0x7f20f006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f210c155ca0 tx=0x7f2108009040 comp rx=0 tx=0).stop 2023-10-16T11:01:17.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.558+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f210c0cb0c0 msgr2=0x7f210c14dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.558+0000 7f21142c8700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f210c0cb0c0 0x7f210c14dcc0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00e900 tx=0x7f20fc00ecc0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 shutdown_connections 2023-10-16T11:01:17.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f20f006b020 0x7f20f006d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f210c15eab0 0x7f210c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f210c0cb0c0 0x7f210c14dcc0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 --2- 172.21.15.181:0/2476946286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f210c15e6d0 0x7f210c0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 >> 172.21.15.181:0/2476946286 conn(0x7f210c0d04a0 msgr2=0x7f210c0ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:17.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 shutdown_connections 2023-10-16T11:01:17.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.559+0000 7f21142c8700 1 -- 172.21.15.181:0/2476946286 wait complete. 2023-10-16T11:01:17.575 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:01:17.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.703+0000 7f6fd3679700 1 Processor -- start 2023-10-16T11:01:17.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.703+0000 7f6fd3679700 1 -- start start 2023-10-16T11:01:17.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd3679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 0x7f6fcc06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd3679700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6fcc0ceb60 0x7f6fcc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd3679700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0cb340 con 0x7f6fcc15ac20 2023-10-16T11:01:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd3679700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc0cf530 con 0x7f6fcc0ceb60 2023-10-16T11:01:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd3679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc15b0d0 con 0x7f6fcc158360 2023-10-16T11:01:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd0c14700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6fcc15ac20 0x7f6fcc15ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34738/0 (socket says 172.21.15.181:34738) 2023-10-16T11:01:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd0c14700 1 -- 172.21.15.181:0/721079205 learned_addr learned my addr 172.21.15.181:0/721079205 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990562988 0 0) 0x7f6fcc0cb340 con 0x7f6fcc15ac20 2023-10-16T11:01:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb80035b0 con 0x7f6fcc15ac20 2023-10-16T11:01:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/721079205 >> v2:172.21.15.190:3300/0 conn(0x7f6fcc0ceb60 0x7f6fcc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd1415700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 0x7f6fcc06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2261519424 0 0) 0x7f6fb80035b0 con 0x7f6fcc15ac20 2023-10-16T11:01:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 msgr2=0x7f6fcc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 0x7f6fcc06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 >> v2:172.21.15.190:3300/0 conn(0x7f6fcc0ceb60 msgr2=0x7f6fcc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 --2- 172.21.15.181:0/721079205 >> v2:172.21.15.190:3300/0 conn(0x7f6fcc0ceb60 0x7f6fcc0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc15b3b0 con 0x7f6fcc15ac20 2023-10-16T11:01:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f6fc8001670 con 0x7f6fcc15ac20 2023-10-16T11:01:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 >> v1:172.21.15.181:6789/0 conn(0x7f6fcc15ac20 legacy=0x7f6fcc15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 0x7f6fb8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fb8003f80 0x7f6fb8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb8004d90 0x7f6fb8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0cf530 con 0x7f6fb8004d90 2023-10-16T11:01:17.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc15b0d0 con 0x7f6fb8003f80 2023-10-16T11:01:17.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb800ba10 con 0x7f6fcc158360 2023-10-16T11:01:17.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb8004d90 0x7f6fb8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd1415700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 0x7f6fb8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.704+0000 7f6fd0c14700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fb8003f80 0x7f6fb8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd1c16700 1 -- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 msgr2=0x7f6fb8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:17.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc158360 0x7f6fb8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd1c16700 1 -- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fb8003f80 msgr2=0x7f6fb8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:17.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fb8003f80 0x7f6fb8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd1c16700 1 -- 172.21.15.181:0/721079205 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80035b0 con 0x7f6fb8004d90 2023-10-16T11:01:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb8004d90 0x7f6fb8005210 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009fc0 tx=0x7f6fb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf9e66337224415 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400ebd0 con 0x7f6fb8004d90 2023-10-16T11:01:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc400f040 con 0x7f6fb8004d90 2023-10-16T11:01:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fc3fff700 1 -- 172.21.15.181:0/721079205 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4014480 con 0x7f6fb8004d90 2023-10-16T11:01:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd3679700 1 -- 172.21.15.181:0/721079205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb8004d90 msgr2=0x7f6fb8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd3679700 1 --2- 172.21.15.181:0/721079205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fb8004d90 0x7f6fb8005210 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009fc0 tx=0x7f6fb8005880 comp rx=0 tx=0).stop 2023-10-16T11:01:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.705+0000 7f6fd1c16700 1 -- 172.21.15.181:0/721079205 reap_dead start 2023-10-16T11:01:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.706+0000 7f6fd3679700 1 -- 172.21.15.181:0/721079205 shutdown_connections 2023-10-16T11:01:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.706+0000 7f6fd3679700 1 -- 172.21.15.181:0/721079205 >> 172.21.15.181:0/721079205 conn(0x7f6fcc0cc730 msgr2=0x7f6fcc0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.706+0000 7f6fd3679700 1 -- 172.21.15.181:0/721079205 shutdown_connections 2023-10-16T11:01:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.706+0000 7f6fd3679700 1 -- 172.21.15.181:0/721079205 wait complete. 2023-10-16T11:01:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.707+0000 7f6fd3679700 1 Processor -- start 2023-10-16T11:01:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.707+0000 7f6fd3679700 1 -- start start 2023-10-16T11:01:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.707+0000 7f6fd3679700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc1529e0 0x7f6fcc0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd3679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc152dc0 0x7f6fcc156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd3679700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fcc1eeaf0 0x7f6fcc155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd3679700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc156ff0 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd3679700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0c7360 con 0x7f6fcc1529e0 2023-10-16T11:01:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd3679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0c7640 con 0x7f6fcc152dc0 2023-10-16T11:01:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fcc1eeaf0 0x7f6fcc155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fcc1eeaf0 0x7f6fcc155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51908/0 (socket says 172.21.15.181:51908) 2023-10-16T11:01:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 -- 172.21.15.181:0/476130444 learned_addr learned my addr 172.21.15.181:0/476130444 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1415700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc1529e0 0x7f6fcc0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd0c14700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc152dc0 0x7f6fcc156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 -- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc152dc0 msgr2=0x7f6fcc156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc152dc0 0x7f6fcc156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 -- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc1529e0 msgr2=0x7f6fcc0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc1529e0 0x7f6fcc0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 -- 172.21.15.181:0/476130444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc0c7920 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1415700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc1529e0 0x7f6fcc0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd1c16700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fcc1eeaf0 0x7f6fcc155f10 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4016000 tx=0x7f6fc4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fc1ffb700 1 -- 172.21.15.181:0/476130444 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4007000 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fc1ffb700 1 -- 172.21.15.181:0/476130444 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc4004470 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc0c7c00 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fc1ffb700 1 -- 172.21.15.181:0/476130444 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4004820 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.708+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc1eefa0 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.709+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc0c8120 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.710+0000 7f6fc1ffb700 1 -- 172.21.15.181:0/476130444 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6fc4014480 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.710+0000 7f6fc1ffb700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6fb006b050 0x7f6fb006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:17.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.710+0000 7f6fc1ffb700 1 -- 172.21.15.181:0/476130444 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f6fc408b210 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.710+0000 7f6fd1415700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6fb006b050 0x7f6fb006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:17.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.711+0000 7f6fd1415700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6fb006b050 0x7f6fb006d510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009c60 tx=0x7f6fbc008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:17.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.715+0000 7f6fc1ffb700 1 -- 172.21.15.181:0/476130444 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6fcc0c8120 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.908+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6fcc0c8120 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.913+0000 7f6fc1ffb700 1 -- 172.21.15.181:0/476130444 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6fcc0c8120 con 0x7f6fcc1eeaf0 2023-10-16T11:01:17.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6fb006b050 msgr2=0x7f6fb006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6fb006b050 0x7f6fb006d510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009c60 tx=0x7f6fbc008040 comp rx=0 tx=0).stop 2023-10-16T11:01:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fcc1eeaf0 msgr2=0x7f6fcc155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fcc1eeaf0 0x7f6fcc155f10 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4016000 tx=0x7f6fc4009750 comp rx=0 tx=0).stop 2023-10-16T11:01:17.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 shutdown_connections 2023-10-16T11:01:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f6fb006b050 0x7f6fb006d510 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fcc1eeaf0 0x7f6fcc155f10 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc1529e0 0x7f6fcc0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 --2- 172.21.15.181:0/476130444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc152dc0 0x7f6fcc156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:17.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 >> 172.21.15.181:0/476130444 conn(0x7f6fcc0cc730 msgr2=0x7f6fcc0b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:17.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.915+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 shutdown_connections 2023-10-16T11:01:17.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:17.916+0000 7f6fd3679700 1 -- 172.21.15.181:0/476130444 wait complete. 2023-10-16T11:01:17.929 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:01:17.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:01:17.971 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:01:18.016 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:01:18.016 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:01:18.017 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:18.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:01:18.035 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:01:18.035 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:18.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:01:18.076 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:01:18.076 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:18.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:01:18.094 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:01:18.094 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:18.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:01:18.135 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:01:18.136 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:01:18.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.254+0000 7f9d4b33e700 1 Processor -- start 2023-10-16T11:01:18.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.254+0000 7f9d4b33e700 1 -- start start 2023-10-16T11:01:18.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d4b33e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d4415b990 0x7f9d440ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d4b33e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d440cd2d0 0x7f9d440d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d4b33e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d440cfc60 0x7f9d440d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d4b33e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d440d0630 con 0x7f9d440cfc60 2023-10-16T11:01:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d4b33e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d440d24f0 con 0x7f9d4415b990 2023-10-16T11:01:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d4b33e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440cc510 con 0x7f9d440cd2d0 2023-10-16T11:01:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d440cfc60 0x7f9d440d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d440cfc60 0x7f9d440d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51916/0 (socket says 172.21.15.181:51916) 2023-10-16T11:01:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 -- 172.21.15.181:0/3089474407 learned_addr learned my addr 172.21.15.181:0/3089474407 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d490da700 1 --2- 172.21.15.181:0/3089474407 >> v2:172.21.15.190:3300/0 conn(0x7f9d4415b990 0x7f9d440ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 -- 172.21.15.181:0/3089474407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d440cd2d0 msgr2=0x7f9d440d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d488d9700 1 --2- 172.21.15.181:0/3089474407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d440cd2d0 0x7f9d440d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 --2- 172.21.15.181:0/3089474407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d440cd2d0 0x7f9d440d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 -- 172.21.15.181:0/3089474407 >> v2:172.21.15.190:3300/0 conn(0x7f9d4415b990 msgr2=0x7f9d440ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 --2- 172.21.15.181:0/3089474407 >> v2:172.21.15.190:3300/0 conn(0x7f9d4415b990 0x7f9d440ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 -- 172.21.15.181:0/3089474407 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d440cc7f0 con 0x7f9d440cfc60 2023-10-16T11:01:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d490da700 1 --2- 172.21.15.181:0/3089474407 >> v2:172.21.15.190:3300/0 conn(0x7f9d4415b990 0x7f9d440ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d498db700 1 --2- 172.21.15.181:0/3089474407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cfc60 0x7f9d440d00f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f9d44073000 tx=0x7f9d3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f499effd7a5a9ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d3bfff700 1 -- 172.21.15.181:0/3089474407 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00e070 con 0x7f9d440cfc60 2023-10-16T11:01:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d3bfff700 1 -- 172.21.15.181:0/3089474407 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00b9c0 con 0x7f9d440cfc60 2023-10-16T11:01:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.255+0000 7f9d3bfff700 1 -- 172.21.15.181:0/3089474407 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00bd70 con 0x7f9d440cfc60 2023-10-16T11:01:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 -- 172.21.15.181:0/3089474407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cfc60 msgr2=0x7f9d440d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/3089474407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cfc60 0x7f9d440d00f0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f9d44073000 tx=0x7f9d3c00b330 comp rx=0 tx=0).stop 2023-10-16T11:01:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 -- 172.21.15.181:0/3089474407 shutdown_connections 2023-10-16T11:01:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/3089474407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cfc60 0x7f9d440d00f0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/3089474407 >> v2:172.21.15.190:3300/0 conn(0x7f9d4415b990 0x7f9d440ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/3089474407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d440cd2d0 0x7f9d440d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 -- 172.21.15.181:0/3089474407 >> 172.21.15.181:0/3089474407 conn(0x7f9d4409a9a0 msgr2=0x7f9d440d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 -- 172.21.15.181:0/3089474407 shutdown_connections 2023-10-16T11:01:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.256+0000 7f9d4b33e700 1 -- 172.21.15.181:0/3089474407 wait complete. 2023-10-16T11:01:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.257+0000 7f9d4b33e700 1 Processor -- start 2023-10-16T11:01:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.257+0000 7f9d4b33e700 1 -- start start 2023-10-16T11:01:18.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d4b33e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cd2d0 0x7f9d441ee940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d4b33e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d440cfc60 0x7f9d440b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d4b33e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d4415b990 0x7f9d440b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d4b33e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440ae730 con 0x7f9d440cd2d0 2023-10-16T11:01:18.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d4b33e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440b7560 con 0x7f9d440cfc60 2023-10-16T11:01:18.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d4b33e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440b6fa0 con 0x7f9d4415b990 2023-10-16T11:01:18.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cd2d0 0x7f9d441ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cd2d0 0x7f9d441ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51918/0 (socket says 172.21.15.181:51918) 2023-10-16T11:01:18.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 -- 172.21.15.181:0/971692962 learned_addr learned my addr 172.21.15.181:0/971692962 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:18.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 -- 172.21.15.181:0/971692962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d4415b990 msgr2=0x7f9d440b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d4415b990 0x7f9d440b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 -- 172.21.15.181:0/971692962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d440cfc60 msgr2=0x7f9d440b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d440cfc60 0x7f9d440b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 -- 172.21.15.181:0/971692962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c009b30 con 0x7f9d440cd2d0 2023-10-16T11:01:18.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.258+0000 7f9d490da700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cd2d0 0x7f9d441ee940 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c7f0 tx=0x7f9d3400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:18.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.259+0000 7f9d39ffb700 1 -- 172.21.15.181:0/971692962 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34005c20 con 0x7f9d440cd2d0 2023-10-16T11:01:18.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.259+0000 7f9d39ffb700 1 -- 172.21.15.181:0/971692962 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d34005d80 con 0x7f9d440cd2d0 2023-10-16T11:01:18.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.259+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d440b7280 con 0x7f9d440cd2d0 2023-10-16T11:01:18.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.259+0000 7f9d39ffb700 1 -- 172.21.15.181:0/971692962 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34011820 con 0x7f9d440cd2d0 2023-10-16T11:01:18.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.259+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d441eee80 con 0x7f9d440cd2d0 2023-10-16T11:01:18.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.260+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d44067d20 con 0x7f9d440cd2d0 2023-10-16T11:01:18.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.260+0000 7f9d39ffb700 1 -- 172.21.15.181:0/971692962 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9d34011980 con 0x7f9d440cd2d0 2023-10-16T11:01:18.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.260+0000 7f9d39ffb700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9d2806b020 0x7f9d2806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.261+0000 7f9d39ffb700 1 -- 172.21.15.181:0/971692962 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f9d34089760 con 0x7f9d440cd2d0 2023-10-16T11:01:18.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.261+0000 7f9d488d9700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9d2806b020 0x7f9d2806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.261+0000 7f9d488d9700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9d2806b020 0x7f9d2806d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d440afee0 tx=0x7f9d40008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:18.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.265+0000 7f9d39ffb700 1 -- 172.21.15.181:0/971692962 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9d340544e0 con 0x7f9d440cd2d0 2023-10-16T11:01:18.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.500+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d440cfa40 con 0x7f9d440cd2d0 2023-10-16T11:01:18.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.501+0000 7f9d39ffb700 1 -- 172.21.15.181:0/971692962 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3712 (secure 0 0 0) 0x7f9d3401ee90 con 0x7f9d440cd2d0 2023-10-16T11:01:18.503 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:18.503 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1460537182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1460537182},{"type":"v1","addr":"172.21.15.190:6833","nonce":1460537182}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8623,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3626960628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3626960628},{"type":"v1","addr":"172.21.15.190:6835","nonce":3626960628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1772227629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1772227629},{"type":"v1","addr":"172.21.15.181:6836","nonce":1772227629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":10092,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3557055323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3557055323},{"type":"v1","addr":"172.21.15.181:6837","nonce":3557055323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[]} 2023-10-16T11:01:18.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9d2806b020 msgr2=0x7f9d2806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9d2806b020 0x7f9d2806d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d440afee0 tx=0x7f9d40008040 comp rx=0 tx=0).stop 2023-10-16T11:01:18.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cd2d0 msgr2=0x7f9d441ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cd2d0 0x7f9d441ee940 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c7f0 tx=0x7f9d3400cbb0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 shutdown_connections 2023-10-16T11:01:18.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9d2806b020 0x7f9d2806d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d4415b990 0x7f9d440b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d440cd2d0 0x7f9d441ee940 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 --2- 172.21.15.181:0/971692962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d440cfc60 0x7f9d440b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 >> 172.21.15.181:0/971692962 conn(0x7f9d4409a9a0 msgr2=0x7f9d440b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:18.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 shutdown_connections 2023-10-16T11:01:18.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.503+0000 7f9d4b33e700 1 -- 172.21.15.181:0/971692962 wait complete. 2023-10-16T11:01:18.508 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 87 2023-10-16T11:01:18.517 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:01:18.517 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:01:18.518 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:01:18.518 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:01:18.518 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:01:18.518 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:01:18.519 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:01:18.519 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:01:18.519 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:01:18.520 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:01:18.520 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:01:18.520 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:01:18.520 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:01:18.521 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:01:18.521 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:01:18.521 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:01:18.526 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:01:18.527 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:01:18.528 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:01:18.528 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:01:18.531 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:01:18.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:01:18.533 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:01:18.533 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:01:18.571 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:01:18.572 DEBUG:teuthology.parallel:result is None 2023-10-16T11:01:18.572 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:01:18.573 DEBUG:teuthology.parallel:result is None 2023-10-16T11:01:18.575 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:01:18.575 DEBUG:teuthology.parallel:result is None 2023-10-16T11:01:18.576 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:01:18.577 DEBUG:teuthology.parallel:result is None 2023-10-16T11:01:18.577 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:01:18.650 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:01:18.651 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:01:18.661 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:01:18.662 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:01:18.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.704+0000 7f296797b700 1 Processor -- start 2023-10-16T11:01:18.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.704+0000 7f296797b700 1 -- start start 2023-10-16T11:01:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f296797b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f29600cfc60 0x7f29600d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f296797b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f296797b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600d24f0 con 0x7f296015bd50 2023-10-16T11:01:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f296797b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296015ccc0 con 0x7f29600cfc60 2023-10-16T11:01:18.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f296797b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296015c320 con 0x7f29600b05d0 2023-10-16T11:01:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f2964f16700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f296015bd50 0x7f29600d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34798/0 (socket says 172.21.15.181:34798) 2023-10-16T11:01:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f2964f16700 1 -- 172.21.15.181:0/3509968754 learned_addr learned my addr 172.21.15.181:0/3509968754 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f2965717700 1 --2- 172.21.15.181:0/3509968754 >> v2:172.21.15.190:3300/0 conn(0x7f29600cfc60 0x7f29600d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.705+0000 7f2965f18700 1 --2- 172.21.15.181:0/3509968754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f295ffff700 1 -- 172.21.15.181:0/3509968754 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568278103 0 0) 0x7f29600d24f0 con 0x7f296015bd50 2023-10-16T11:01:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f295ffff700 1 -- 172.21.15.181:0/3509968754 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29480035b0 con 0x7f296015bd50 2023-10-16T11:01:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f2965f18700 1 -- 172.21.15.181:0/3509968754 >> v2:172.21.15.190:3300/0 conn(0x7f29600cfc60 msgr2=0x7f29600d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f2965f18700 1 --2- 172.21.15.181:0/3509968754 >> v2:172.21.15.190:3300/0 conn(0x7f29600cfc60 0x7f29600d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f2965f18700 1 -- 172.21.15.181:0/3509968754 >> v1:172.21.15.181:6789/0 conn(0x7f296015bd50 legacy=0x7f29600d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f2965f18700 1 -- 172.21.15.181:0/3509968754 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296015c600 con 0x7f29600b05d0 2023-10-16T11:01:18.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f2965717700 1 --2- 172.21.15.181:0/3509968754 >> v2:172.21.15.190:3300/0 conn(0x7f29600cfc60 0x7f29600d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:18.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f2965f18700 1 --2- 172.21.15.181:0/3509968754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600cc200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2950009d90 tx=0x7f295000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f5740dfb590f1a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:18.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f295ffff700 1 -- 172.21.15.181:0/3509968754 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000e070 con 0x7f29600b05d0 2023-10-16T11:01:18.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f295ffff700 1 -- 172.21.15.181:0/3509968754 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295000bb20 con 0x7f29600b05d0 2023-10-16T11:01:18.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.706+0000 7f295ffff700 1 -- 172.21.15.181:0/3509968754 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000be90 con 0x7f29600b05d0 2023-10-16T11:01:18.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 -- 172.21.15.181:0/3509968754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 msgr2=0x7f29600cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 --2- 172.21.15.181:0/3509968754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600cc200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2950009d90 tx=0x7f295000b330 comp rx=0 tx=0).stop 2023-10-16T11:01:18.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 -- 172.21.15.181:0/3509968754 shutdown_connections 2023-10-16T11:01:18.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 --2- 172.21.15.181:0/3509968754 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600cc200 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 --2- 172.21.15.181:0/3509968754 >> v2:172.21.15.190:3300/0 conn(0x7f29600cfc60 0x7f29600d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 -- 172.21.15.181:0/3509968754 >> 172.21.15.181:0/3509968754 conn(0x7f296009a9a0 msgr2=0x7f29600d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:18.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 -- 172.21.15.181:0/3509968754 shutdown_connections 2023-10-16T11:01:18.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.707+0000 7f296797b700 1 -- 172.21.15.181:0/3509968754 wait complete. 2023-10-16T11:01:18.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.708+0000 7f296797b700 1 Processor -- start 2023-10-16T11:01:18.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.708+0000 7f296797b700 1 -- start start 2023-10-16T11:01:18.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f296797b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f296797b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cfc60 0x7f2960151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f296797b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f296015bd50 0x7f29600b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f296797b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600b4cb0 con 0x7f296015bd50 2023-10-16T11:01:18.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f296797b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2960159ee0 con 0x7f29600cfc60 2023-10-16T11:01:18.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f296797b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29600b6840 con 0x7f29600b05d0 2023-10-16T11:01:18.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f296015bd50 0x7f29600b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f296015bd50 0x7f29600b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51960/0 (socket says 172.21.15.181:51960) 2023-10-16T11:01:18.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 -- 172.21.15.181:0/27643407 learned_addr learned my addr 172.21.15.181:0/27643407 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:18.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965717700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2964f16700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cfc60 0x7f2960151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 -- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 msgr2=0x7f29600b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 -- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cfc60 msgr2=0x7f2960151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:18.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cfc60 0x7f2960151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:18.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 -- 172.21.15.181:0/27643407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580094a0 con 0x7f296015bd50 2023-10-16T11:01:18.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2964f16700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cfc60 0x7f2960151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:18.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f2965f18700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f296015bd50 0x7f29600b5cd0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f2950014040 tx=0x7f2950004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:18.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f295dffb700 1 -- 172.21.15.181:0/27643407 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950004960 con 0x7f296015bd50 2023-10-16T11:01:18.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f295dffb700 1 -- 172.21.15.181:0/27643407 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2950020070 con 0x7f296015bd50 2023-10-16T11:01:18.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2950009a70 con 0x7f296015bd50 2023-10-16T11:01:18.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.709+0000 7f295dffb700 1 -- 172.21.15.181:0/27643407 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29500276c0 con 0x7f296015bd50 2023-10-16T11:01:18.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.710+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29600b45f0 con 0x7f296015bd50 2023-10-16T11:01:18.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.710+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29600b2f90 con 0x7f296015bd50 2023-10-16T11:01:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.711+0000 7f295dffb700 1 -- 172.21.15.181:0/27643407 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2950012430 con 0x7f296015bd50 2023-10-16T11:01:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.711+0000 7f295dffb700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f294c06b020 0x7f294c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.712+0000 7f2965717700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f294c06b020 0x7f294c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.712+0000 7f295dffb700 1 -- 172.21.15.181:0/27643407 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6127+0+0 (secure 0 0 0) 0x7f2950022070 con 0x7f296015bd50 2023-10-16T11:01:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.712+0000 7f2965717700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f294c06b020 0x7f294c06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2958009d00 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:18.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.716+0000 7f295dffb700 1 -- 172.21.15.181:0/27643407 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f29500563f0 con 0x7f296015bd50 2023-10-16T11:01:18.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-16T11:01:18.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-10-16T11:01:18.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-16T11:01:18.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-16T11:01:18.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:18.926+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f29600b4df0 con 0x7f296015bd50 2023-10-16T11:01:19.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.571+0000 7f295dffb700 1 -- 172.21.15.181:0/27643407 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v175) v1 ==== 97+0+0 (secure 0 0 0) 0x7f29600b4df0 con 0x7f296015bd50 2023-10-16T11:01:19.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f294c06b020 msgr2=0x7f294c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:19.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f294c06b020 0x7f294c06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2958009d00 tx=0x7f2958008040 comp rx=0 tx=0).stop 2023-10-16T11:01:19.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f296015bd50 msgr2=0x7f29600b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:19.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f296015bd50 0x7f29600b5cd0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f2950014040 tx=0x7f2950004ad0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 shutdown_connections 2023-10-16T11:01:19.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f294c06b020 0x7f294c06d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29600b05d0 0x7f29600b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f296015bd50 0x7f29600b5cd0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 --2- 172.21.15.181:0/27643407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29600cfc60 0x7f2960151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 >> 172.21.15.181:0/27643407 conn(0x7f296009a9a0 msgr2=0x7f296015f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:19.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 shutdown_connections 2023-10-16T11:01:19.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.575+0000 7f296797b700 1 -- 172.21.15.181:0/27643407 wait complete. 2023-10-16T11:01:19.580 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T11:01:19.593 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:01:19.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.714+0000 7eff4293b700 1 Processor -- start 2023-10-16T11:01:19.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.714+0000 7eff4293b700 1 -- start start 2023-10-16T11:01:19.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.714+0000 7eff4293b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c066ac0 0x7eff3c15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:19.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.714+0000 7eff4293b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff3c0cdd90 0x7eff3c0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:19.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.714+0000 7eff4293b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c06b600 con 0x7eff3c15b9a0 2023-10-16T11:01:19.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.714+0000 7eff4293b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c06b8e0 con 0x7eff3c0cdd90 2023-10-16T11:01:19.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.714+0000 7eff4293b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c06bbc0 con 0x7eff3c066ac0 2023-10-16T11:01:19.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff40ed8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff3c0cdd90 0x7eff3c0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:19.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff3bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7eff3c15b9a0 0x7eff3c15f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34812/0 (socket says 172.21.15.181:34812) 2023-10-16T11:01:19.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff3bfff700 1 -- 172.21.15.181:0/1550817181 learned_addr learned my addr 172.21.15.181:0/1550817181 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:19.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff3b7fe700 1 --2- 172.21.15.181:0/1550817181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c066ac0 0x7eff3c15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:19.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff40ed8700 1 -- 172.21.15.181:0/1550817181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c066ac0 msgr2=0x7eff3c15fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:19.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff40ed8700 1 --2- 172.21.15.181:0/1550817181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c066ac0 0x7eff3c15fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff40ed8700 1 -- 172.21.15.181:0/1550817181 >> v1:172.21.15.181:6789/0 conn(0x7eff3c15b9a0 legacy=0x7eff3c15f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:19.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff40ed8700 1 -- 172.21.15.181:0/1550817181 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c06bea0 con 0x7eff3c0cdd90 2023-10-16T11:01:19.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff40ed8700 1 --2- 172.21.15.181:0/1550817181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c0ce220 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7eff34009d90 tx=0x7eff3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d895e0d6cd6471cd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:19.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff3affd700 1 -- 172.21.15.181:0/1550817181 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400e070 con 0x7eff3c0cdd90 2023-10-16T11:01:19.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff3affd700 1 -- 172.21.15.181:0/1550817181 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3400bbd0 con 0x7eff3c0cdd90 2023-10-16T11:01:19.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.715+0000 7eff3affd700 1 -- 172.21.15.181:0/1550817181 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34012430 con 0x7eff3c0cdd90 2023-10-16T11:01:19.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 -- 172.21.15.181:0/1550817181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 msgr2=0x7eff3c0ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:19.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 --2- 172.21.15.181:0/1550817181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c0ce220 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7eff34009d90 tx=0x7eff3400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:19.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 -- 172.21.15.181:0/1550817181 shutdown_connections 2023-10-16T11:01:19.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 --2- 172.21.15.181:0/1550817181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c0ce220 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 --2- 172.21.15.181:0/1550817181 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c066ac0 0x7eff3c15fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 -- 172.21.15.181:0/1550817181 >> 172.21.15.181:0/1550817181 conn(0x7eff3c0b16e0 msgr2=0x7eff3c0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:19.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 -- 172.21.15.181:0/1550817181 shutdown_connections 2023-10-16T11:01:19.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.716+0000 7eff4293b700 1 -- 172.21.15.181:0/1550817181 wait complete. 2023-10-16T11:01:19.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.717+0000 7eff4293b700 1 Processor -- start 2023-10-16T11:01:19.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.717+0000 7eff4293b700 1 -- start start 2023-10-16T11:01:19.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff3c066ac0 0x7eff3c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:19.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:19.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c158d70 0x7eff3c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:19.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c156150 con 0x7eff3c0cdd90 2023-10-16T11:01:19.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c15adb0 con 0x7eff3c066ac0 2023-10-16T11:01:19.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c15dd10 con 0x7eff3c158d70 2023-10-16T11:01:19.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:19.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c1556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51974/0 (socket says 172.21.15.181:51974) 2023-10-16T11:01:19.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 -- 172.21.15.181:0/437845114 learned_addr learned my addr 172.21.15.181:0/437845114 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:19.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff40ed8700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c158d70 0x7eff3c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:19.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 -- 172.21.15.181:0/437845114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c158d70 msgr2=0x7eff3c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:19.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c158d70 0x7eff3c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 -- 172.21.15.181:0/437845114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff3c066ac0 msgr2=0x7eff3c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:19.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff3c066ac0 0x7eff3c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:19.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 -- 172.21.15.181:0/437845114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34009a70 con 0x7eff3c0cdd90 2023-10-16T11:01:19.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff3b7fe700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c1556d0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c7f0 tx=0x7eff2400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:19.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff38ff9700 1 -- 172.21.15.181:0/437845114 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24011800 con 0x7eff3c0cdd90 2023-10-16T11:01:19.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff38ff9700 1 -- 172.21.15.181:0/437845114 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24011960 con 0x7eff3c0cdd90 2023-10-16T11:01:19.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c15dff0 con 0x7eff3c0cdd90 2023-10-16T11:01:19.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff38ff9700 1 -- 172.21.15.181:0/437845114 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24011cf0 con 0x7eff3c0cdd90 2023-10-16T11:01:19.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.718+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c15e980 con 0x7eff3c0cdd90 2023-10-16T11:01:19.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.719+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c0c3720 con 0x7eff3c0cdd90 2023-10-16T11:01:19.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.720+0000 7eff38ff9700 1 -- 172.21.15.181:0/437845114 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7eff24018360 con 0x7eff3c0cdd90 2023-10-16T11:01:19.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.720+0000 7eff38ff9700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7eff1c06b020 0x7eff1c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:19.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.720+0000 7eff38ff9700 1 -- 172.21.15.181:0/437845114 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5353+0+0 (secure 0 0 0) 0x7eff24089690 con 0x7eff3c0cdd90 2023-10-16T11:01:19.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.720+0000 7eff3bfff700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7eff1c06b020 0x7eff1c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:19.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.721+0000 7eff3bfff700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7eff1c06b020 0x7eff1c06d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0015d0 tx=0x7eff2c007ad0 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:19.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:19.725+0000 7eff38ff9700 1 -- 172.21.15.181:0/437845114 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7eff3c0c3720 con 0x7eff3c0cdd90 2023-10-16T11:01:20.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.001+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7eff3c0c3720 con 0x7eff3c0cdd90 2023-10-16T11:01:20.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.001+0000 7eff38ff9700 1 -- 172.21.15.181:0/437845114 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+5304 (secure 0 0 0) 0x7eff3c0c3720 con 0x7eff3c0cdd90 2023-10-16T11:01:20.003 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:20.003 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:01:20.004 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T11:01:20.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.004 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T11:01:20.004 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T11:01:20.004 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.005 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.005 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:01:20.005 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:01:20.005 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.006 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T11:01:20.006 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T11:01:20.006 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:01:20.007 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:01:20.007 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.007 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.008 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T11:01:20.008 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T11:01:20.008 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.008 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.009 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:01:20.009 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:01:20.009 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.010 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.010 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T11:01:20.010 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T11:01:20.011 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.011 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.011 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:01:20.011 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:01:20.011 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.012 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T11:01:20.012 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T11:01:20.012 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.013 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.013 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.013 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.013 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.014 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T11:01:20.014 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T11:01:20.014 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.015 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.015 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.015 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.015 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.016 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T11:01:20.016 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T11:01:20.016 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.017 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.017 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.017 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.017 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.018 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T11:01:20.018 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T11:01:20.019 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.019 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.019 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.019 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.019 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.020 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T11:01:20.020 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T11:01:20.020 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.020 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.021 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.021 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.021 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.022 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T11:01:20.022 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T11:01:20.022 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.022 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.024 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T11:01:20.024 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T11:01:20.024 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.024 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.024 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.025 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.025 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.025 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T11:01:20.026 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T11:01:20.026 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.026 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:01:20.026 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:01:20.026 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.027 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.027 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T11:01:20.027 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T11:01:20.028 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.028 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.028 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T11:01:20.028 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:01:20.028 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.029 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T11:01:20.029 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T11:01:20.029 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.030 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.030 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:01:20.030 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:01:20.030 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:01:20.030 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.031 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T11:01:20.031 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T11:01:20.031 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:01:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:01:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:01:20.032 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.033 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T11:01:20.033 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T11:01:20.034 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.034 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:01:20.034 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:01:20.034 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:01:20.034 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:01:20.035 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.035 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T11:01:20.035 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T11:01:20.035 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.036 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:01:20.036 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:01:20.036 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:01:20.036 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.037 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T11:01:20.037 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T11:01:20.037 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.038 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:01:20.038 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:01:20.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.038 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:01:20.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:20.039 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T11:01:20.039 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T11:01:20.039 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:01:20.039 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:01:20.040 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:01:20.040 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:01:20.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:20.040 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T11:01:20.040 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:01:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7eff1c06b020 msgr2=0x7eff1c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7eff1c06b020 0x7eff1c06d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0015d0 tx=0x7eff2c007ad0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 msgr2=0x7eff3c1556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c1556d0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c7f0 tx=0x7eff2400cbb0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 shutdown_connections 2023-10-16T11:01:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7eff1c06b020 0x7eff1c06d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff3c158d70 0x7eff3c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff3c066ac0 0x7eff3c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 --2- 172.21.15.181:0/437845114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff3c0cdd90 0x7eff3c1556d0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 >> 172.21.15.181:0/437845114 conn(0x7eff3c0b16e0 msgr2=0x7eff3c159750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 shutdown_connections 2023-10-16T11:01:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.004+0000 7eff4293b700 1 -- 172.21.15.181:0/437845114 wait complete. 2023-10-16T11:01:20.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:20.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.164+0000 7f985c659700 1 Processor -- start 2023-10-16T11:01:20.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.164+0000 7f985c659700 1 -- start start 2023-10-16T11:01:20.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985c659700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f985415dc20 0x7f98540d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985c659700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98540d1e60 0x7f98540d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985c659700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98540ce4b0 0x7f98540ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985c659700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540b3b80 con 0x7f98540ce4b0 2023-10-16T11:01:20.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985c659700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540d2820 con 0x7f985415dc20 2023-10-16T11:01:20.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985c659700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98540b2f00 con 0x7f98540d1e60 2023-10-16T11:01:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985abf6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98540ce4b0 0x7f98540ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985abf6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98540ce4b0 0x7f98540ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51984/0 (socket says 172.21.15.181:51984) 2023-10-16T11:01:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.165+0000 7f985abf6700 1 -- 172.21.15.181:0/2560569506 learned_addr learned my addr 172.21.15.181:0/2560569506 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:20.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985abf6700 1 -- 172.21.15.181:0/2560569506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98540d1e60 msgr2=0x7f98540d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985abf6700 1 --2- 172.21.15.181:0/2560569506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98540d1e60 0x7f98540d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985abf6700 1 -- 172.21.15.181:0/2560569506 >> v2:172.21.15.190:3300/0 conn(0x7f985415dc20 msgr2=0x7f98540d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985abf6700 1 --2- 172.21.15.181:0/2560569506 >> v2:172.21.15.190:3300/0 conn(0x7f985415dc20 0x7f98540d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985abf6700 1 -- 172.21.15.181:0/2560569506 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98540b31e0 con 0x7f98540ce4b0 2023-10-16T11:01:20.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985abf6700 1 --2- 172.21.15.181:0/2560569506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540ce4b0 0x7f98540ce940 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f985406b6e0 tx=0x7f984c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e797726371c514 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f98593f3700 1 -- 172.21.15.181:0/2560569506 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00e070 con 0x7f98540ce4b0 2023-10-16T11:01:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f98593f3700 1 -- 172.21.15.181:0/2560569506 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c00b9c0 con 0x7f98540ce4b0 2023-10-16T11:01:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f98593f3700 1 -- 172.21.15.181:0/2560569506 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00bd70 con 0x7f98540ce4b0 2023-10-16T11:01:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985c659700 1 -- 172.21.15.181:0/2560569506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540ce4b0 msgr2=0x7f98540ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.166+0000 7f985c659700 1 --2- 172.21.15.181:0/2560569506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540ce4b0 0x7f98540ce940 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f985406b6e0 tx=0x7f984c00b330 comp rx=0 tx=0).stop 2023-10-16T11:01:20.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.167+0000 7f985c659700 1 -- 172.21.15.181:0/2560569506 shutdown_connections 2023-10-16T11:01:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.167+0000 7f985c659700 1 --2- 172.21.15.181:0/2560569506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540ce4b0 0x7f98540ce940 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.167+0000 7f985c659700 1 --2- 172.21.15.181:0/2560569506 >> v2:172.21.15.190:3300/0 conn(0x7f985415dc20 0x7f98540d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.167+0000 7f985c659700 1 --2- 172.21.15.181:0/2560569506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98540d1e60 0x7f98540d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.167+0000 7f985c659700 1 -- 172.21.15.181:0/2560569506 >> 172.21.15.181:0/2560569506 conn(0x7f985408af90 msgr2=0x7f98540d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.167+0000 7f985c659700 1 -- 172.21.15.181:0/2560569506 shutdown_connections 2023-10-16T11:01:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.167+0000 7f985c659700 1 -- 172.21.15.181:0/2560569506 wait complete. 2023-10-16T11:01:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.168+0000 7f985c659700 1 Processor -- start 2023-10-16T11:01:20.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.168+0000 7f985c659700 1 -- start start 2023-10-16T11:01:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98540ce4b0 0x7f98541f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540d1e60 0x7f985415e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f985415dc20 0x7f98540b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985415e540 con 0x7f98540d1e60 2023-10-16T11:01:20.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985415f380 con 0x7f98540ce4b0 2023-10-16T11:01:20.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985415e770 con 0x7f985415dc20 2023-10-16T11:01:20.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540d1e60 0x7f985415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540d1e60 0x7f985415e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51988/0 (socket says 172.21.15.181:51988) 2023-10-16T11:01:20.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 -- 172.21.15.181:0/156114378 learned_addr learned my addr 172.21.15.181:0/156114378 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:20.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985a3f5700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98540ce4b0 0x7f98541f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985abf6700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f985415dc20 0x7f98540b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 -- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f985415dc20 msgr2=0x7f98540b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f985415dc20 0x7f98540b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 -- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98540ce4b0 msgr2=0x7f98541f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98540ce4b0 0x7f98541f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 -- 172.21.15.181:0/156114378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c009b30 con 0x7f98540d1e60 2023-10-16T11:01:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985a3f5700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98540ce4b0 0x7f98541f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:20.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f9859bf4700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540d1e60 0x7f985415e000 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f985000b530 tx=0x7f985000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f984affd700 1 -- 172.21.15.181:0/156114378 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850011830 con 0x7f98540d1e60 2023-10-16T11:01:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f984affd700 1 -- 172.21.15.181:0/156114378 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500119b0 con 0x7f98540d1e60 2023-10-16T11:01:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985415ea50 con 0x7f98540d1e60 2023-10-16T11:01:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f984affd700 1 -- 172.21.15.181:0/156114378 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850011d60 con 0x7f98540d1e60 2023-10-16T11:01:20.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.169+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985415f010 con 0x7f98540d1e60 2023-10-16T11:01:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.170+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985415f4c0 con 0x7f98540d1e60 2023-10-16T11:01:20.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.171+0000 7f984affd700 1 -- 172.21.15.181:0/156114378 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f985001a400 con 0x7f98540d1e60 2023-10-16T11:01:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.171+0000 7f984affd700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f983806b020 0x7f983806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.171+0000 7f985a3f5700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f983806b020 0x7f983806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.171+0000 7f984affd700 1 -- 172.21.15.181:0/156114378 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f98500896e0 con 0x7f98540d1e60 2023-10-16T11:01:20.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.172+0000 7f985a3f5700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f983806b020 0x7f983806d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9844006fd0 tx=0x7f9844008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.176+0000 7f984affd700 1 -- 172.21.15.181:0/156114378 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f98500547e0 con 0x7f98540d1e60 2023-10-16T11:01:20.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.367+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98540cdc60 con 0x7f98540d1e60 2023-10-16T11:01:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.368+0000 7f984affd700 1 -- 172.21.15.181:0/156114378 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+13663 (secure 0 0 0) 0x7f9850015090 con 0x7f98540d1e60 2023-10-16T11:01:20.369 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:20.370 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":177,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:19.585064+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:20.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f983806b020 msgr2=0x7f983806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f983806b020 0x7f983806d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9844006fd0 tx=0x7f9844008040 comp rx=0 tx=0).stop 2023-10-16T11:01:20.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540d1e60 msgr2=0x7f985415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540d1e60 0x7f985415e000 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f985000b530 tx=0x7f985000b8f0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 shutdown_connections 2023-10-16T11:01:20.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f983806b020 0x7f983806d4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f985415dc20 0x7f98540b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98540ce4b0 0x7f98541f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 --2- 172.21.15.181:0/156114378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98540d1e60 0x7f985415e000 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 >> 172.21.15.181:0/156114378 conn(0x7f985408af90 msgr2=0x7f985406cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:20.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 shutdown_connections 2023-10-16T11:01:20.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.370+0000 7f985c659700 1 -- 172.21.15.181:0/156114378 wait complete. 2023-10-16T11:01:20.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:01:20.384 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:01:20.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.506+0000 7f9cec6e0700 1 Processor -- start 2023-10-16T11:01:20.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.506+0000 7f9cec6e0700 1 -- start start 2023-10-16T11:01:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cec6e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce415f150 0x7f9ce40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cec6e0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ce40ce4f0 0x7f9ce40b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cec6e0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ce40b14a0 0x7f9ce40b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cec6e0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce40ce970 con 0x7f9ce40b14a0 2023-10-16T11:01:20.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cec6e0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce40b2ae0 con 0x7f9ce40ce4f0 2023-10-16T11:01:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cec6e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40b2dc0 con 0x7f9ce415f150 2023-10-16T11:01:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ce40b14a0 0x7f9ce40b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ce40b14a0 0x7f9ce40b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51998/0 (socket says 172.21.15.181:51998) 2023-10-16T11:01:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 -- 172.21.15.181:0/83905987 learned_addr learned my addr 172.21.15.181:0/83905987 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:20.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ce9c7b700 1 --2- 172.21.15.181:0/83905987 >> v2:172.21.15.190:3300/0 conn(0x7f9ce40ce4f0 0x7f9ce40b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cea47c700 1 --2- 172.21.15.181:0/83905987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce415f150 0x7f9ce40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 -- 172.21.15.181:0/83905987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce415f150 msgr2=0x7f9ce40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:20.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 --2- 172.21.15.181:0/83905987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce415f150 0x7f9ce40cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 -- 172.21.15.181:0/83905987 >> v2:172.21.15.190:3300/0 conn(0x7f9ce40ce4f0 msgr2=0x7f9ce40b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 --2- 172.21.15.181:0/83905987 >> v2:172.21.15.190:3300/0 conn(0x7f9ce40ce4f0 0x7f9ce40b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 -- 172.21.15.181:0/83905987 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce40cf9d0 con 0x7f9ce40b14a0 2023-10-16T11:01:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9cea47c700 1 --2- 172.21.15.181:0/83905987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce415f150 0x7f9ce40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ceac7d700 1 --2- 172.21.15.181:0/83905987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce40b14a0 0x7f9ce40b1930 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4071730 tx=0x7f9cdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7aa91bb3ffae04 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ce947a700 1 -- 172.21.15.181:0/83905987 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00e070 con 0x7f9ce40b14a0 2023-10-16T11:01:20.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ce947a700 1 -- 172.21.15.181:0/83905987 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc00b910 con 0x7f9ce40b14a0 2023-10-16T11:01:20.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.507+0000 7f9ce947a700 1 -- 172.21.15.181:0/83905987 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00bcc0 con 0x7f9ce40b14a0 2023-10-16T11:01:20.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 -- 172.21.15.181:0/83905987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce40b14a0 msgr2=0x7f9ce40b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/83905987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce40b14a0 0x7f9ce40b1930 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4071730 tx=0x7f9cdc00b330 comp rx=0 tx=0).stop 2023-10-16T11:01:20.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 -- 172.21.15.181:0/83905987 shutdown_connections 2023-10-16T11:01:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/83905987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce40b14a0 0x7f9ce40b1930 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/83905987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce415f150 0x7f9ce40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/83905987 >> v2:172.21.15.190:3300/0 conn(0x7f9ce40ce4f0 0x7f9ce40b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 -- 172.21.15.181:0/83905987 >> 172.21.15.181:0/83905987 conn(0x7f9ce409a930 msgr2=0x7f9ce40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 -- 172.21.15.181:0/83905987 shutdown_connections 2023-10-16T11:01:20.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.508+0000 7f9cec6e0700 1 -- 172.21.15.181:0/83905987 wait complete. 2023-10-16T11:01:20.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.509+0000 7f9cec6e0700 1 Processor -- start 2023-10-16T11:01:20.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.509+0000 7f9cec6e0700 1 -- start start 2023-10-16T11:01:20.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9cec6e0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce40b14a0 0x7f9ce4158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9cec6e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce40ce4f0 0x7f9ce40b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9cec6e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce415f150 0x7f9ce40700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9cec6e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40c79f0 con 0x7f9ce415f150 2023-10-16T11:01:20.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9cec6e0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40c6b90 con 0x7f9ce40b14a0 2023-10-16T11:01:20.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9cec6e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40c6e70 con 0x7f9ce40ce4f0 2023-10-16T11:01:20.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce415f150 0x7f9ce40700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce415f150 0x7f9ce40700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52004/0 (socket says 172.21.15.181:52004) 2023-10-16T11:01:20.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 -- 172.21.15.181:0/3405097214 learned_addr learned my addr 172.21.15.181:0/3405097214 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:20.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9cea47c700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce40b14a0 0x7f9ce4158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ce9c7b700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce40ce4f0 0x7f9ce40b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 -- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce40ce4f0 msgr2=0x7f9ce40b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce40ce4f0 0x7f9ce40b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 -- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce40b14a0 msgr2=0x7f9ce4158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce40b14a0 0x7f9ce4158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 -- 172.21.15.181:0/3405097214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009b30 con 0x7f9ce415f150 2023-10-16T11:01:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.510+0000 7f9ceac7d700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce415f150 0x7f9ce40700b0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b370 tx=0x7f9cdc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.511+0000 7f9cdaffd700 1 -- 172.21.15.181:0/3405097214 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0099b0 con 0x7f9ce415f150 2023-10-16T11:01:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.511+0000 7f9cdaffd700 1 -- 172.21.15.181:0/3405097214 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc0092e0 con 0x7f9ce415f150 2023-10-16T11:01:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.511+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce40c7150 con 0x7f9ce415f150 2023-10-16T11:01:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.511+0000 7f9cdaffd700 1 -- 172.21.15.181:0/3405097214 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004450 con 0x7f9ce415f150 2023-10-16T11:01:20.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.511+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce40c7710 con 0x7f9ce415f150 2023-10-16T11:01:20.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.512+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce40c8760 con 0x7f9ce415f150 2023-10-16T11:01:20.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.512+0000 7f9cdaffd700 1 -- 172.21.15.181:0/3405097214 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9cdc0045b0 con 0x7f9ce415f150 2023-10-16T11:01:20.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.512+0000 7f9cdaffd700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9cc806b020 0x7f9cc806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.513+0000 7f9cea47c700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9cc806b020 0x7f9cc806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.513+0000 7f9cdaffd700 1 -- 172.21.15.181:0/3405097214 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f9cdc08a990 con 0x7f9ce415f150 2023-10-16T11:01:20.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.513+0000 7f9cea47c700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9cc806b020 0x7f9cc806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400c940 tx=0x7f9cd4008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.517+0000 7f9cdaffd700 1 -- 172.21.15.181:0/3405097214 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9ce40c8760 con 0x7f9ce415f150 2023-10-16T11:01:20.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.711+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ce40c8760 con 0x7f9ce415f150 2023-10-16T11:01:20.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.711+0000 7f9cdaffd700 1 -- 172.21.15.181:0/3405097214 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ce40c8760 con 0x7f9ce415f150 2023-10-16T11:01:20.714 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:01:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9cc806b020 msgr2=0x7f9cc806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9cc806b020 0x7f9cc806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400c940 tx=0x7f9cd4008040 comp rx=0 tx=0).stop 2023-10-16T11:01:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce415f150 msgr2=0x7f9ce40700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce415f150 0x7f9ce40700b0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b370 tx=0x7f9cdc004ad0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 shutdown_connections 2023-10-16T11:01:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f9cc806b020 0x7f9cc806d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ce415f150 0x7f9ce40700b0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ce40b14a0 0x7f9ce4158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 --2- 172.21.15.181:0/3405097214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ce40ce4f0 0x7f9ce40b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 >> 172.21.15.181:0/3405097214 conn(0x7f9ce409a930 msgr2=0x7f9ce40d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:20.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 shutdown_connections 2023-10-16T11:01:20.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.714+0000 7f9cec6e0700 1 -- 172.21.15.181:0/3405097214 wait complete. 2023-10-16T11:01:20.728 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:01:20.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.847+0000 7f629d2ed700 1 Processor -- start 2023-10-16T11:01:20.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.847+0000 7f629d2ed700 1 -- start start 2023-10-16T11:01:20.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f629d2ed700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6298160d20 0x7f62981611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f629d2ed700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629806ce20 con 0x7f629806d5f0 2023-10-16T11:01:20.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f629d2ed700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62980d45f0 con 0x7f6298161b60 2023-10-16T11:01:20.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f629d2ed700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6298162800 con 0x7f6298160d20 2023-10-16T11:01:20.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f62967fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f629806d5f0 0x7f62980d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34858/0 (socket says 172.21.15.181:34858) 2023-10-16T11:01:20.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f62967fc700 1 -- 172.21.15.181:0/1755747403 learned_addr learned my addr 172.21.15.181:0/1755747403 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:20.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f62977fe700 1 --2- 172.21.15.181:0/1755747403 >> v2:172.21.15.181:3300/0 conn(0x7f6298160d20 0x7f62981611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f6295ffb700 1 -- 172.21.15.181:0/1755747403 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606152074 0 0) 0x7f629806ce20 con 0x7f629806d5f0 2023-10-16T11:01:20.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f6295ffb700 1 -- 172.21.15.181:0/1755747403 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62800035b0 con 0x7f629806d5f0 2023-10-16T11:01:20.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f62977fe700 1 -- 172.21.15.181:0/1755747403 >> v1:172.21.15.190:6789/0 conn(0x7f6298161b60 legacy=0x7f6298161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f62977fe700 1 -- 172.21.15.181:0/1755747403 >> v1:172.21.15.181:6789/0 conn(0x7f629806d5f0 legacy=0x7f62980d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f62977fe700 1 -- 172.21.15.181:0/1755747403 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629806d2d0 con 0x7f6298160d20 2023-10-16T11:01:20.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f62977fe700 1 --2- 172.21.15.181:0/1755747403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62981611d0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f6290009d90 tx=0x7f629000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe846020cf7883bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.848+0000 7f6295ffb700 1 -- 172.21.15.181:0/1755747403 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000e070 con 0x7f6298160d20 2023-10-16T11:01:20.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f6295ffb700 1 -- 172.21.15.181:0/1755747403 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629000bb20 con 0x7f6298160d20 2023-10-16T11:01:20.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f6295ffb700 1 -- 172.21.15.181:0/1755747403 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290012430 con 0x7f6298160d20 2023-10-16T11:01:20.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f629d2ed700 1 -- 172.21.15.181:0/1755747403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 msgr2=0x7f62981611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f629d2ed700 1 --2- 172.21.15.181:0/1755747403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62981611d0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f6290009d90 tx=0x7f629000b330 comp rx=0 tx=0).stop 2023-10-16T11:01:20.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f629d2ed700 1 -- 172.21.15.181:0/1755747403 shutdown_connections 2023-10-16T11:01:20.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f629d2ed700 1 --2- 172.21.15.181:0/1755747403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62981611d0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f629d2ed700 1 -- 172.21.15.181:0/1755747403 >> 172.21.15.181:0/1755747403 conn(0x7f629809cba0 msgr2=0x7f62980d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:20.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f629d2ed700 1 -- 172.21.15.181:0/1755747403 shutdown_connections 2023-10-16T11:01:20.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.849+0000 7f629d2ed700 1 -- 172.21.15.181:0/1755747403 wait complete. 2023-10-16T11:01:20.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.850+0000 7f629d2ed700 1 Processor -- start 2023-10-16T11:01:20.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.850+0000 7f629d2ed700 1 -- start start 2023-10-16T11:01:20.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f629d2ed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f629806d5f0 0x7f629815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f629d2ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62980cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f629d2ed700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6298161b60 0x7f62980cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f629d2ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980cc020 con 0x7f6298160d20 2023-10-16T11:01:20.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f629d2ed700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980cc300 con 0x7f6298161b60 2023-10-16T11:01:20.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f629d2ed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980cc5e0 con 0x7f629806d5f0 2023-10-16T11:01:20.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f62967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62980cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f62967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62980cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52018/0 (socket says 172.21.15.181:52018) 2023-10-16T11:01:20.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f62967fc700 1 -- 172.21.15.181:0/3639974943 learned_addr learned my addr 172.21.15.181:0/3639974943 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:20.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f6296ffd700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f629806d5f0 0x7f629815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.851+0000 7f62977fe700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6298161b60 0x7f62980cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f62967fc700 1 -- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f629806d5f0 msgr2=0x7f629815c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f62967fc700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f629806d5f0 0x7f629815c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f62967fc700 1 -- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6298161b60 msgr2=0x7f62980cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:20.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f62967fc700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6298161b60 0x7f62980cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:20.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f62967fc700 1 -- 172.21.15.181:0/3639974943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6290009a70 con 0x7f6298160d20 2023-10-16T11:01:20.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f62967fc700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62980cb2e0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f628800c700 tx=0x7f628800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f627f7fe700 1 -- 172.21.15.181:0/3639974943 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288005d70 con 0x7f6298160d20 2023-10-16T11:01:20.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f627f7fe700 1 -- 172.21.15.181:0/3639974943 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62880070d0 con 0x7f6298160d20 2023-10-16T11:01:20.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62980b65c0 con 0x7f6298160d20 2023-10-16T11:01:20.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f627f7fe700 1 -- 172.21.15.181:0/3639974943 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288007460 con 0x7f6298160d20 2023-10-16T11:01:20.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.852+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62980cd3a0 con 0x7f6298160d20 2023-10-16T11:01:20.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.853+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62980b6420 con 0x7f6298160d20 2023-10-16T11:01:20.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.853+0000 7f627f7fe700 1 -- 172.21.15.181:0/3639974943 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f62880075c0 con 0x7f6298160d20 2023-10-16T11:01:20.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.854+0000 7f627f7fe700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f628406b020 0x7f628406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:20.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.854+0000 7f627f7fe700 1 -- 172.21.15.181:0/3639974943 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f6288088a30 con 0x7f6298160d20 2023-10-16T11:01:20.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.854+0000 7f6296ffd700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f628406b020 0x7f628406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:20.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.854+0000 7f6296ffd700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f628406b020 0x7f628406d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f628c000fb0 tx=0x7f628c009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:20.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:20.858+0000 7f627f7fe700 1 -- 172.21.15.181:0/3639974943 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f62980b6420 con 0x7f6298160d20 2023-10-16T11:01:21.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.054+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f62980b6420 con 0x7f6298160d20 2023-10-16T11:01:21.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.055+0000 7f627f7fe700 1 -- 172.21.15.181:0/3639974943 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v177) v1 ==== 84+0+65 (secure 0 0 0) 0x7f62980b6420 con 0x7f6298160d20 2023-10-16T11:01:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:21.057 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:01:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:01:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:01:21.058 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:21.058 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:01:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.056+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f628406b020 msgr2=0x7f628406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.056+0000 7f629d2ed700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f628406b020 0x7f628406d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f628c000fb0 tx=0x7f628c009040 comp rx=0 tx=0).stop 2023-10-16T11:01:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 msgr2=0x7f62980cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62980cb2e0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f628800c700 tx=0x7f628800ca10 comp rx=0 tx=0).stop 2023-10-16T11:01:21.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 shutdown_connections 2023-10-16T11:01:21.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f628406b020 0x7f628406d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f629806d5f0 0x7f629815c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6298161b60 0x7f62980cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 --2- 172.21.15.181:0/3639974943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6298160d20 0x7f62980cb2e0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 >> 172.21.15.181:0/3639974943 conn(0x7f629809cba0 msgr2=0x7f629815de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:21.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 shutdown_connections 2023-10-16T11:01:21.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.057+0000 7f629d2ed700 1 -- 172.21.15.181:0/3639974943 wait complete. 2023-10-16T11:01:21.071 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:01:21.071 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:01:21.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.194+0000 7f7beafea700 1 Processor -- start 2023-10-16T11:01:21.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.194+0000 7f7beafea700 1 -- start start 2023-10-16T11:01:21.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.194+0000 7f7beafea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7be40b0f70 0x7f7be40b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.194+0000 7f7beafea700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40d23c0 con 0x7f7be40ce810 2023-10-16T11:01:21.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.194+0000 7f7beafea700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40cd990 con 0x7f7be415eea0 2023-10-16T11:01:21.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7beafea700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be40cdc70 con 0x7f7be40b0f70 2023-10-16T11:01:21.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7be40b0f70 0x7f7be40b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be9587700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7be40ce810 0x7f7be415f620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34874/0 (socket says 172.21.15.181:34874) 2023-10-16T11:01:21.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be9587700 1 -- 172.21.15.181:0/57447626 learned_addr learned my addr 172.21.15.181:0/57447626 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:21.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be3fff700 1 -- 172.21.15.181:0/57447626 >> v1:172.21.15.190:6789/0 conn(0x7f7be415eea0 legacy=0x7f7be40ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:21.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be3fff700 1 -- 172.21.15.181:0/57447626 >> v1:172.21.15.181:6789/0 conn(0x7f7be40ce810 legacy=0x7f7be415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be3fff700 1 -- 172.21.15.181:0/57447626 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40cce60 con 0x7f7be40b0f70 2023-10-16T11:01:21.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be3fff700 1 --2- 172.21.15.181:0/57447626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40b0f70 0x7f7be40b03e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009840 tx=0x7f7bd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c50be1c60a0de9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be37fe700 1 -- 172.21.15.181:0/57447626 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0004230 con 0x7f7be40b0f70 2023-10-16T11:01:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be37fe700 1 -- 172.21.15.181:0/57447626 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd0004390 con 0x7f7be40b0f70 2023-10-16T11:01:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.195+0000 7f7be37fe700 1 -- 172.21.15.181:0/57447626 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00036a0 con 0x7f7be40b0f70 2023-10-16T11:01:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.196+0000 7f7beafea700 1 -- 172.21.15.181:0/57447626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40b0f70 msgr2=0x7f7be40b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.196+0000 7f7beafea700 1 --2- 172.21.15.181:0/57447626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40b0f70 0x7f7be40b03e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009840 tx=0x7f7bd0009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.196+0000 7f7beafea700 1 -- 172.21.15.181:0/57447626 shutdown_connections 2023-10-16T11:01:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.196+0000 7f7beafea700 1 --2- 172.21.15.181:0/57447626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40b0f70 0x7f7be40b03e0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.196+0000 7f7beafea700 1 -- 172.21.15.181:0/57447626 >> 172.21.15.181:0/57447626 conn(0x7f7be40d1610 msgr2=0x7f7be40d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.196+0000 7f7beafea700 1 -- 172.21.15.181:0/57447626 shutdown_connections 2023-10-16T11:01:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.196+0000 7f7beafea700 1 -- 172.21.15.181:0/57447626 wait complete. 2023-10-16T11:01:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.197+0000 7f7beafea700 1 Processor -- start 2023-10-16T11:01:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.197+0000 7f7beafea700 1 -- start start 2023-10-16T11:01:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7beafea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7be40b0f70 0x7f7be4154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7beafea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40ce810 0x7f7be40c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7beafea700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7be415eea0 0x7f7be40b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7beafea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be4155120 con 0x7f7be40ce810 2023-10-16T11:01:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7beafea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40b5670 con 0x7f7be40b0f70 2023-10-16T11:01:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7beafea700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be415b370 con 0x7f7be415eea0 2023-10-16T11:01:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40ce810 0x7f7be40c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40ce810 0x7f7be40c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52032/0 (socket says 172.21.15.181:52032) 2023-10-16T11:01:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 -- 172.21.15.181:0/900735896 learned_addr learned my addr 172.21.15.181:0/900735896 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be8d86700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7be40b0f70 0x7f7be4154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be9587700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7be415eea0 0x7f7be40b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 -- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7be415eea0 msgr2=0x7f7be40b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7be415eea0 0x7f7be40b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 -- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7be40b0f70 msgr2=0x7f7be4154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7be40b0f70 0x7f7be4154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 -- 172.21.15.181:0/900735896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd00094a0 con 0x7f7be40ce810 2023-10-16T11:01:21.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.198+0000 7f7be3fff700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40ce810 0x7f7be40c6610 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00090b0 tx=0x7f7bd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:21.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.199+0000 7f7be17fa700 1 -- 172.21.15.181:0/900735896 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0011830 con 0x7f7be40ce810 2023-10-16T11:01:21.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.199+0000 7f7be17fa700 1 -- 172.21.15.181:0/900735896 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd001e070 con 0x7f7be40ce810 2023-10-16T11:01:21.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.199+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be415b650 con 0x7f7be40ce810 2023-10-16T11:01:21.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.199+0000 7f7be17fa700 1 -- 172.21.15.181:0/900735896 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0011bf0 con 0x7f7be40ce810 2023-10-16T11:01:21.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.199+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be415bc10 con 0x7f7be40ce810 2023-10-16T11:01:21.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.200+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be415be00 con 0x7f7be40ce810 2023-10-16T11:01:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.200+0000 7f7be17fa700 1 -- 172.21.15.181:0/900735896 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7bd0011d50 con 0x7f7be40ce810 2023-10-16T11:01:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.200+0000 7f7be17fa700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7bc806b020 0x7f7bc806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.201+0000 7f7be17fa700 1 -- 172.21.15.181:0/900735896 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f7bd008a940 con 0x7f7be40ce810 2023-10-16T11:01:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.201+0000 7f7be8d86700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7bc806b020 0x7f7bc806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.201+0000 7f7be8d86700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7bc806b020 0x7f7bc806d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009870 tx=0x7f7bd8008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.205+0000 7f7be17fa700 1 -- 172.21.15.181:0/900735896 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7bd0055ad0 con 0x7f7be40ce810 2023-10-16T11:01:21.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.398+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7be4067d20 con 0x7f7be40ce810 2023-10-16T11:01:21.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.580+0000 7f7be17fa700 1 -- 172.21.15.181:0/900735896 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v178) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7bd0020020 con 0x7f7be40ce810 2023-10-16T11:01:21.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7bc806b020 msgr2=0x7f7bc806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7bc806b020 0x7f7bc806d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009870 tx=0x7f7bd8008040 comp rx=0 tx=0).stop 2023-10-16T11:01:21.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40ce810 msgr2=0x7f7be40c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40ce810 0x7f7be40c6610 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00090b0 tx=0x7f7bd0004750 comp rx=0 tx=0).stop 2023-10-16T11:01:21.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 shutdown_connections 2023-10-16T11:01:21.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7bc806b020 0x7f7bc806d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7be415eea0 0x7f7be40b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7be40b0f70 0x7f7be4154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 --2- 172.21.15.181:0/900735896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be40ce810 0x7f7be40c6610 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 >> 172.21.15.181:0/900735896 conn(0x7f7be40d1610 msgr2=0x7f7be40d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:21.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 shutdown_connections 2023-10-16T11:01:21.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.583+0000 7f7beafea700 1 -- 172.21.15.181:0/900735896 wait complete. 2023-10-16T11:01:21.588 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T11:01:21.600 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:01:21.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.727+0000 7fde137a3700 1 Processor -- start 2023-10-16T11:01:21.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.727+0000 7fde137a3700 1 -- start start 2023-10-16T11:01:21.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde137a3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c072760 0x7fde0c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde137a3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fde0c06ade0 0x7fde0c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde137a3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0c06b230 con 0x7fde0c15da70 2023-10-16T11:01:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde137a3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c0cef00 con 0x7fde0c06ade0 2023-10-16T11:01:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde137a3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0cec20 con 0x7fde0c072760 2023-10-16T11:01:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde11d40700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fde0c06ade0 0x7fde0c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde1153f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fde0c15da70 0x7fde0c0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34884/0 (socket says 172.21.15.181:34884) 2023-10-16T11:01:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde1153f700 1 -- 172.21.15.181:0/1447803816 learned_addr learned my addr 172.21.15.181:0/1447803816 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde10d3e700 1 --2- 172.21.15.181:0/1447803816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c072760 0x7fde0c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde11d40700 1 -- 172.21.15.181:0/1447803816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c072760 msgr2=0x7fde0c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde11d40700 1 --2- 172.21.15.181:0/1447803816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c072760 0x7fde0c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde11d40700 1 -- 172.21.15.181:0/1447803816 >> v1:172.21.15.181:6789/0 conn(0x7fde0c15da70 legacy=0x7fde0c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde03fff700 1 -- 172.21.15.181:0/1447803816 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204133285 0 0) 0x7fde08001110 con 0x7fde0c15da70 2023-10-16T11:01:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde11d40700 1 -- 172.21.15.181:0/1447803816 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c071c80 con 0x7fde0c06ade0 2023-10-16T11:01:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde10d3e700 1 --2- 172.21.15.181:0/1447803816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c072760 0x7fde0c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde11d40700 1 --2- 172.21.15.181:0/1447803816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c06ade0 0x7fde0c1615f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fde04009d90 tx=0x7fde0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c61e49a0c4a75f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:21.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde03fff700 1 -- 172.21.15.181:0/1447803816 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0400e070 con 0x7fde0c06ade0 2023-10-16T11:01:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde03fff700 1 -- 172.21.15.181:0/1447803816 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0400bbd0 con 0x7fde0c06ade0 2023-10-16T11:01:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.728+0000 7fde03fff700 1 -- 172.21.15.181:0/1447803816 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04012430 con 0x7fde0c06ade0 2023-10-16T11:01:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 -- 172.21.15.181:0/1447803816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c06ade0 msgr2=0x7fde0c1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 --2- 172.21.15.181:0/1447803816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c06ade0 0x7fde0c1615f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fde04009d90 tx=0x7fde0400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:21.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 -- 172.21.15.181:0/1447803816 shutdown_connections 2023-10-16T11:01:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 --2- 172.21.15.181:0/1447803816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c06ade0 0x7fde0c1615f0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 --2- 172.21.15.181:0/1447803816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c072760 0x7fde0c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 -- 172.21.15.181:0/1447803816 >> 172.21.15.181:0/1447803816 conn(0x7fde0c0cd010 msgr2=0x7fde0c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:21.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 -- 172.21.15.181:0/1447803816 shutdown_connections 2023-10-16T11:01:21.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.729+0000 7fde137a3700 1 -- 172.21.15.181:0/1447803816 wait complete. 2023-10-16T11:01:21.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.730+0000 7fde137a3700 1 Processor -- start 2023-10-16T11:01:21.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.730+0000 7fde137a3700 1 -- start start 2023-10-16T11:01:21.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde137a3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c06ade0 0x7fde0c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde137a3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c072760 0x7fde0c14a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde137a3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c15da70 0x7fde0c14cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde137a3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0c7660 con 0x7fde0c072760 2023-10-16T11:01:21.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde137a3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0ca010 con 0x7fde0c06ade0 2023-10-16T11:01:21.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde137a3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0c7ec0 con 0x7fde0c15da70 2023-10-16T11:01:21.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c072760 0x7fde0c14a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c072760 0x7fde0c14a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52046/0 (socket says 172.21.15.181:52046) 2023-10-16T11:01:21.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 -- 172.21.15.181:0/661049980 learned_addr learned my addr 172.21.15.181:0/661049980 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:21.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde11d40700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c15da70 0x7fde0c14cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde1153f700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c06ade0 0x7fde0c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 -- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c15da70 msgr2=0x7fde0c14cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c15da70 0x7fde0c14cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 -- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c06ade0 msgr2=0x7fde0c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:21.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c06ade0 0x7fde0c0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:21.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.731+0000 7fde10d3e700 1 -- 172.21.15.181:0/661049980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde04009a70 con 0x7fde0c072760 2023-10-16T11:01:21.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.732+0000 7fde10d3e700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c072760 0x7fde0c14a350 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c750 tx=0x7fddfc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:21.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.732+0000 7fde01ffb700 1 -- 172.21.15.181:0/661049980 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc011800 con 0x7fde0c072760 2023-10-16T11:01:21.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.732+0000 7fde01ffb700 1 -- 172.21.15.181:0/661049980 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc011960 con 0x7fde0c072760 2023-10-16T11:01:21.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.732+0000 7fde01ffb700 1 -- 172.21.15.181:0/661049980 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc011cd0 con 0x7fde0c072760 2023-10-16T11:01:21.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.732+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c0c7900 con 0x7fde0c072760 2023-10-16T11:01:21.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.732+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c0ca2b0 con 0x7fde0c072760 2023-10-16T11:01:21.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.733+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c0c9e70 con 0x7fde0c072760 2023-10-16T11:01:21.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.733+0000 7fde01ffb700 1 -- 172.21.15.181:0/661049980 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fddfc021350 con 0x7fde0c072760 2023-10-16T11:01:21.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.734+0000 7fde01ffb700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fddf006b050 0x7fddf006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:21.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.734+0000 7fde01ffb700 1 -- 172.21.15.181:0/661049980 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5723+0+0 (secure 0 0 0) 0x7fddfc089770 con 0x7fde0c072760 2023-10-16T11:01:21.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.734+0000 7fde1153f700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fddf006b050 0x7fddf006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:21.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.734+0000 7fde1153f700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fddf006b050 0x7fddf006d510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fde08000f60 tx=0x7fde08007a40 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:21.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.739+0000 7fde01ffb700 1 -- 172.21.15.181:0/661049980 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde0c0c9e70 con 0x7fde0c072760 2023-10-16T11:01:21.778 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:01:21.781 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:01:21.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:21.936+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fde0c0c9e70 con 0x7fde0c072760 2023-10-16T11:01:22.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.583+0000 7fde01ffb700 1 -- 172.21.15.181:0/661049980 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v179) v1 ==== 170+0+0 (secure 0 0 0) 0x7fde0c0c9e70 con 0x7fde0c072760 2023-10-16T11:01:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.585+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fddf006b050 msgr2=0x7fddf006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.585+0000 7fde137a3700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fddf006b050 0x7fddf006d510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fde08000f60 tx=0x7fde08007a40 comp rx=0 tx=0).stop 2023-10-16T11:01:22.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.585+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c072760 msgr2=0x7fde0c14a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:22.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.585+0000 7fde137a3700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c072760 0x7fde0c14a350 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00c750 tx=0x7fddfc00cb10 comp rx=0 tx=0).stop 2023-10-16T11:01:22.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 shutdown_connections 2023-10-16T11:01:22.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fddf006b050 0x7fddf006d510 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c15da70 0x7fde0c14cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c06ade0 0x7fde0c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 --2- 172.21.15.181:0/661049980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde0c072760 0x7fde0c14a350 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 >> 172.21.15.181:0/661049980 conn(0x7fde0c0cd010 msgr2=0x7fde0c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:22.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 shutdown_connections 2023-10-16T11:01:22.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.589+0000 7fde137a3700 1 -- 172.21.15.181:0/661049980 wait complete. 2023-10-16T11:01:22.602 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T11:01:22.614 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:01:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.738+0000 7feaf2a51700 1 Processor -- start 2023-10-16T11:01:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.738+0000 7feaf2a51700 1 -- start start 2023-10-16T11:01:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaf2a51700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feaec15ea40 0x7feaec0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaf2a51700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec0cf580 con 0x7feaec0ce2d0 2023-10-16T11:01:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaf2a51700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec0ce970 con 0x7feaec0d01d0 2023-10-16T11:01:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaf2a51700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaec0d4290 con 0x7feaec15ea40 2023-10-16T11:01:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaeb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7feaec0ce2d0 0x7feaec0d1500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34900/0 (socket says 172.21.15.181:34900) 2023-10-16T11:01:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaeb7fe700 1 -- 172.21.15.181:0/4056790126 learned_addr learned my addr 172.21.15.181:0/4056790126 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:22.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaf0fee700 1 --2- 172.21.15.181:0/4056790126 >> v2:172.21.15.190:3300/0 conn(0x7feaec15ea40 0x7feaec0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.739+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362720284 0 0) 0x7feaec0cf580 con 0x7feaec0ce2d0 2023-10-16T11:01:22.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead40035b0 con 0x7feaec0ce2d0 2023-10-16T11:01:22.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543511803 0 0) 0x7fead40035b0 con 0x7feaec0ce2d0 2023-10-16T11:01:22.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 >> v2:172.21.15.190:3300/0 conn(0x7feaec15ea40 msgr2=0x7feaec0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:22.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 --2- 172.21.15.181:0/4056790126 >> v2:172.21.15.190:3300/0 conn(0x7feaec15ea40 0x7feaec0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 >> v1:172.21.15.190:6789/0 conn(0x7feaec0d01d0 legacy=0x7feaec0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:22.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec0cec50 con 0x7feaec0ce2d0 2023-10-16T11:01:22.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7feadc001670 con 0x7feaec0ce2d0 2023-10-16T11:01:22.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 >> v1:172.21.15.181:6789/0 conn(0x7feaec0ce2d0 legacy=0x7feaec0d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:22.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fead4003a00 0x7fead4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead4004400 0x7fead40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead40050d0 0x7fead400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead400c000 con 0x7fead4004400 2023-10-16T11:01:22.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead400c170 con 0x7fead4003a00 2023-10-16T11:01:22.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead400c330 con 0x7fead40050d0 2023-10-16T11:01:22.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeb7fe700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead4004400 0x7fead40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaebfff700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fead4003a00 0x7fead4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaf0fee700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead40050d0 0x7fead400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeb7fe700 1 -- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead40050d0 msgr2=0x7fead400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:22.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeb7fe700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead40050d0 0x7fead400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeb7fe700 1 -- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fead4003a00 msgr2=0x7fead4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:22.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeb7fe700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fead4003a00 0x7fead4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeb7fe700 1 -- 172.21.15.181:0/4056790126 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead40035b0 con 0x7fead4004400 2023-10-16T11:01:22.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaf0fee700 1 -- 172.21.15.181:0/4056790126 reap_dead start 2023-10-16T11:01:22.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeb7fe700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead4004400 0x7fead40048d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c2f0 tx=0x7fead4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f944e83451ef62 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:22.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.740+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00cdb0 con 0x7fead4004400 2023-10-16T11:01:22.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc005960 con 0x7fead4004400 2023-10-16T11:01:22.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaeaffd700 1 -- 172.21.15.181:0/4056790126 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc005d10 con 0x7fead4004400 2023-10-16T11:01:22.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaf2a51700 1 -- 172.21.15.181:0/4056790126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead4004400 msgr2=0x7fead40048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:22.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaf2a51700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead4004400 0x7fead40048d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c2f0 tx=0x7fead4005880 comp rx=0 tx=0).stop 2023-10-16T11:01:22.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaf2a51700 1 -- 172.21.15.181:0/4056790126 shutdown_connections 2023-10-16T11:01:22.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaf2a51700 1 --2- 172.21.15.181:0/4056790126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead4004400 0x7fead40048d0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaf2a51700 1 -- 172.21.15.181:0/4056790126 >> 172.21.15.181:0/4056790126 conn(0x7feaec09cb60 msgr2=0x7feaec0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:22.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaf2a51700 1 -- 172.21.15.181:0/4056790126 shutdown_connections 2023-10-16T11:01:22.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.741+0000 7feaf2a51700 1 -- 172.21.15.181:0/4056790126 wait complete. 2023-10-16T11:01:22.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.742+0000 7feaf2a51700 1 Processor -- start 2023-10-16T11:01:22.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf2a51700 1 -- start start 2023-10-16T11:01:22.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf2a51700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead4004400 0x7feaec1f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf2a51700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaec0b8970 0x7feaec151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf2a51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec160670 0x7feaec160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf2a51700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec0b3710 con 0x7feaec160670 2023-10-16T11:01:22.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf2a51700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec161060 con 0x7feaec0b8970 2023-10-16T11:01:22.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf2a51700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec0b3120 con 0x7fead4004400 2023-10-16T11:01:22.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf0fee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec160670 0x7feaec160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf0fee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec160670 0x7feaec160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52068/0 (socket says 172.21.15.181:52068) 2023-10-16T11:01:22.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaf0fee700 1 -- 172.21.15.181:0/405910637 learned_addr learned my addr 172.21.15.181:0/405910637 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:22.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaeb7fe700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaec0b8970 0x7feaec151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.743+0000 7feaebfff700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead4004400 0x7feaec1f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf0fee700 1 -- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead4004400 msgr2=0x7feaec1f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:22.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf0fee700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead4004400 0x7feaec1f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf0fee700 1 -- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaec0b8970 msgr2=0x7feaec151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:22.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf0fee700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaec0b8970 0x7feaec151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:22.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf0fee700 1 -- 172.21.15.181:0/405910637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec0b9210 con 0x7feaec160670 2023-10-16T11:01:22.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf0fee700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec160670 0x7feaec160b20 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7feaec0d0db0 tx=0x7feae4011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:22.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feae8ff9700 1 -- 172.21.15.181:0/405910637 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400e040 con 0x7feaec160670 2023-10-16T11:01:22.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feae8ff9700 1 -- 172.21.15.181:0/405910637 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae400f040 con 0x7feaec160670 2023-10-16T11:01:22.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec0b94f0 con 0x7feaec160670 2023-10-16T11:01:22.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feae8ff9700 1 -- 172.21.15.181:0/405910637 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400fbe0 con 0x7feaec160670 2023-10-16T11:01:22.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.744+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec152110 con 0x7feaec160670 2023-10-16T11:01:22.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.745+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec0b3850 con 0x7feaec160670 2023-10-16T11:01:22.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.746+0000 7feae8ff9700 1 -- 172.21.15.181:0/405910637 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feae400fd40 con 0x7feaec160670 2023-10-16T11:01:22.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.746+0000 7feae8ff9700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fead806b020 0x7fead806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:22.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.746+0000 7feae8ff9700 1 -- 172.21.15.181:0/405910637 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6105+0+0 (secure 0 0 0) 0x7feae4089e20 con 0x7feaec160670 2023-10-16T11:01:22.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.746+0000 7feaebfff700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fead806b020 0x7fead806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:22.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.746+0000 7feaebfff700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fead806b020 0x7fead806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feaec1613a0 tx=0x7feae0009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:22.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.751+0000 7feae8ff9700 1 -- 172.21.15.181:0/405910637 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feaec0b3850 con 0x7feaec160670 2023-10-16T11:01:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:22.985+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7feaec0b3850 con 0x7feaec160670 2023-10-16T11:01:23.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.613+0000 7feae8ff9700 1 -- 172.21.15.181:0/405910637 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v90) v1 ==== 174+0+0 (secure 0 0 0) 0x7feaec0b3850 con 0x7feaec160670 2023-10-16T11:01:23.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.619+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fead806b020 msgr2=0x7fead806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.619+0000 7feaf2a51700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fead806b020 0x7fead806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feaec1613a0 tx=0x7feae0009040 comp rx=0 tx=0).stop 2023-10-16T11:01:23.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.619+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec160670 msgr2=0x7feaec160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.619+0000 7feaf2a51700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec160670 0x7feaec160b20 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7feaec0d0db0 tx=0x7feae4011ed0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.620+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 shutdown_connections 2023-10-16T11:01:23.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.620+0000 7feaf2a51700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fead806b020 0x7fead806d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.620+0000 7feaf2a51700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fead4004400 0x7feaec1f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.620+0000 7feaf2a51700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec160670 0x7feaec160b20 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.620+0000 7feaf2a51700 1 --2- 172.21.15.181:0/405910637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feaec0b8970 0x7feaec151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.620+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 >> 172.21.15.181:0/405910637 conn(0x7feaec09cb60 msgr2=0x7feaec15f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:23.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.621+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 shutdown_connections 2023-10-16T11:01:23.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.621+0000 7feaf2a51700 1 -- 172.21.15.181:0/405910637 wait complete. 2023-10-16T11:01:23.624 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 22 and data pool 23 2023-10-16T11:01:23.640 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:23.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.760+0000 7f4113bd8700 1 Processor -- start 2023-10-16T11:01:23.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.760+0000 7f4113bd8700 1 -- start start 2023-10-16T11:01:23.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4113bd8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:23.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4113bd8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f410c0cd2d0 0x7f410c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:23.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4113bd8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410c0cc510 con 0x7f410c15b990 2023-10-16T11:01:23.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4113bd8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c0d24f0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4113bd8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0cc7f0 con 0x7f410c0d0370 2023-10-16T11:01:23.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4112175700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f410c0cd2d0 0x7f410c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:23.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4112175700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f410c0cd2d0 0x7f410c0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52076/0 (socket says 172.21.15.181:52076) 2023-10-16T11:01:23.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4112175700 1 -- 172.21.15.181:0/1783299990 learned_addr learned my addr 172.21.15.181:0/1783299990 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:23.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4111173700 1 --2- 172.21.15.181:0/1783299990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:23.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4112175700 1 -- 172.21.15.181:0/1783299990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 msgr2=0x7f410c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:23.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4112175700 1 --2- 172.21.15.181:0/1783299990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4112175700 1 -- 172.21.15.181:0/1783299990 >> v1:172.21.15.190:6789/0 conn(0x7f410c15b990 legacy=0x7f410c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4112175700 1 -- 172.21.15.181:0/1783299990 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0ccad0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.761+0000 7f4111173700 1 --2- 172.21.15.181:0/1783299990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:23.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4112175700 1 --2- 172.21.15.181:0/1783299990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c0d1fb0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f4104009e50 tx=0x7f410400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c85471431f8be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:23.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4110972700 1 -- 172.21.15.181:0/1783299990 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410400e070 con 0x7f410c0cd2d0 2023-10-16T11:01:23.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4110972700 1 -- 172.21.15.181:0/1783299990 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410400b9c0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4110972700 1 -- 172.21.15.181:0/1783299990 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410400bd70 con 0x7f410c0cd2d0 2023-10-16T11:01:23.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 -- 172.21.15.181:0/1783299990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 msgr2=0x7f410c0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 --2- 172.21.15.181:0/1783299990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c0d1fb0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f4104009e50 tx=0x7f410400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 -- 172.21.15.181:0/1783299990 shutdown_connections 2023-10-16T11:01:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 --2- 172.21.15.181:0/1783299990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c0d1fb0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 --2- 172.21.15.181:0/1783299990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 -- 172.21.15.181:0/1783299990 >> 172.21.15.181:0/1783299990 conn(0x7f410c09a9a0 msgr2=0x7f410c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:23.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 -- 172.21.15.181:0/1783299990 shutdown_connections 2023-10-16T11:01:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.762+0000 7f4113bd8700 1 -- 172.21.15.181:0/1783299990 wait complete. 2023-10-16T11:01:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 Processor -- start 2023-10-16T11:01:23.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 -- start start 2023-10-16T11:01:23.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f410c15b990 0x7f410c0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c06c6a0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c06c980 con 0x7f410c15b990 2023-10-16T11:01:23.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4113bd8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c06cc60 con 0x7f410c0d0370 2023-10-16T11:01:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4111974700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4111974700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52078/0 (socket says 172.21.15.181:52078) 2023-10-16T11:01:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4111974700 1 -- 172.21.15.181:0/2679057941 learned_addr learned my addr 172.21.15.181:0/2679057941 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.764+0000 7f4111173700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4112175700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f410c15b990 0x7f410c0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:23.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4111974700 1 -- 172.21.15.181:0/2679057941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 msgr2=0x7f410c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4111974700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4111974700 1 -- 172.21.15.181:0/2679057941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f410c15b990 msgr2=0x7f410c0c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4111974700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f410c15b990 0x7f410c0c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4111974700 1 -- 172.21.15.181:0/2679057941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009b30 con 0x7f410c0cd2d0 2023-10-16T11:01:23.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4111974700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c159c30 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00c430 tx=0x7f40fc00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:23.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f41027fc700 1 -- 172.21.15.181:0/2679057941 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0052d0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f41027fc700 1 -- 172.21.15.181:0/2679057941 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc005450 con 0x7f410c0cd2d0 2023-10-16T11:01:23.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f41027fc700 1 -- 172.21.15.181:0/2679057941 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc006af0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c06cf40 con 0x7f410c0cd2d0 2023-10-16T11:01:23.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.765+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c15a170 con 0x7f410c0cd2d0 2023-10-16T11:01:23.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.766+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c06c7e0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.766+0000 7f41027fc700 1 -- 172.21.15.181:0/2679057941 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f40fc0036d0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.767+0000 7f41027fc700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f40f006b020 0x7f40f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:23.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.767+0000 7f41027fc700 1 -- 172.21.15.181:0/2679057941 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f40fc088ae0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.767+0000 7f4111173700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f40f006b020 0x7f40f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:23.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.767+0000 7f4111173700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f40f006b020 0x7f40f006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f410c15afb0 tx=0x7f4108008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:23.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.771+0000 7f41027fc700 1 -- 172.21.15.181:0/2679057941 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f410c06c7e0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.964+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f410c06c7e0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.965+0000 7f41027fc700 1 -- 172.21.15.181:0/2679057941 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+17380 (secure 0 0 0) 0x7f410c06c7e0 con 0x7f410c0cd2d0 2023-10-16T11:01:23.967 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:23.967 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":180,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:22.987504+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:01:21.420382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:01:21.958383+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:23.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f40f006b020 msgr2=0x7f40f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f40f006b020 0x7f40f006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f410c15afb0 tx=0x7f4108008040 comp rx=0 tx=0).stop 2023-10-16T11:01:23.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 msgr2=0x7f410c159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:23.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c159c30 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00c430 tx=0x7f40fc00c740 comp rx=0 tx=0).stop 2023-10-16T11:01:23.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 shutdown_connections 2023-10-16T11:01:23.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f40f006b020 0x7f40f006d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f410c15b990 0x7f410c0c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f410c0cd2d0 0x7f410c159c30 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 --2- 172.21.15.181:0/2679057941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f410c0d0370 0x7f410c15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:23.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 >> 172.21.15.181:0/2679057941 conn(0x7f410c09a9a0 msgr2=0x7f410c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:23.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 shutdown_connections 2023-10-16T11:01:23.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:23.968+0000 7f4113bd8700 1 -- 172.21.15.181:0/2679057941 wait complete. 2023-10-16T11:01:23.982 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:24.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.101+0000 7fdf787de700 1 Processor -- start 2023-10-16T11:01:24.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.101+0000 7fdf787de700 1 -- start start 2023-10-16T11:01:24.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf787de700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdf700cdb80 0x7fdf700af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf787de700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7006f870 con 0x7fdf700af450 2023-10-16T11:01:24.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf787de700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf700d1160 con 0x7fdf7015dfd0 2023-10-16T11:01:24.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf787de700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf700cc8e0 con 0x7fdf700cdb80 2023-10-16T11:01:24.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75d79700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdf700af450 0x7fdf700cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34934/0 (socket says 172.21.15.181:34934) 2023-10-16T11:01:24.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75d79700 1 -- 172.21.15.181:0/2022601056 learned_addr learned my addr 172.21.15.181:0/2022601056 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:24.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697226531 0 0) 0x7fdf7006f870 con 0x7fdf700af450 2023-10-16T11:01:24.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf76d7b700 1 --2- 172.21.15.181:0/2022601056 >> v2:172.21.15.190:3300/0 conn(0x7fdf700cdb80 0x7fdf700af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf5c0035b0 con 0x7fdf700af450 2023-10-16T11:01:24.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2681759348 0 0) 0x7fdf5c0035b0 con 0x7fdf700af450 2023-10-16T11:01:24.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 >> v2:172.21.15.190:3300/0 conn(0x7fdf700cdb80 msgr2=0x7fdf700af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:24.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 --2- 172.21.15.181:0/2022601056 >> v2:172.21.15.190:3300/0 conn(0x7fdf700cdb80 0x7fdf700af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 >> v1:172.21.15.190:6789/0 conn(0x7fdf7015dfd0 legacy=0x7fdf7006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700ccbc0 con 0x7fdf700af450 2023-10-16T11:01:24.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fdf60001670 con 0x7fdf700af450 2023-10-16T11:01:24.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 >> v1:172.21.15.181:6789/0 conn(0x7fdf700af450 legacy=0x7fdf700cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c003a00 0x7fdf5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c004850 0x7fdf5c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf5c005210 0x7fdf5c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700cc8e0 con 0x7fdf5c004850 2023-10-16T11:01:24.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c00c1b0 con 0x7fdf5c003a00 2023-10-16T11:01:24.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.102+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c00c370 con 0x7fdf5c005210 2023-10-16T11:01:24.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75d79700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c004850 0x7fdf5c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf7657a700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c003a00 0x7fdf5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf76d7b700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf5c005210 0x7fdf5c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75d79700 1 -- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf5c005210 msgr2=0x7fdf5c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:24.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75d79700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf5c005210 0x7fdf5c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75d79700 1 -- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c003a00 msgr2=0x7fdf5c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:24.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75d79700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c003a00 0x7fdf5c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75d79700 1 -- 172.21.15.181:0/2022601056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0035b0 con 0x7fdf5c004850 2023-10-16T11:01:24.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf76d7b700 1 -- 172.21.15.181:0/2022601056 reap_dead start 2023-10-16T11:01:24.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75d79700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c004850 0x7fdf5c004cd0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c240 tx=0x7fdf5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4d6b5c4c51bcec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:24.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60007c10 con 0x7fdf5c004850 2023-10-16T11:01:24.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6000cd60 con 0x7fdf5c004850 2023-10-16T11:01:24.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.103+0000 7fdf75578700 1 -- 172.21.15.181:0/2022601056 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60005ba0 con 0x7fdf5c004850 2023-10-16T11:01:24.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.104+0000 7fdf787de700 1 -- 172.21.15.181:0/2022601056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c004850 msgr2=0x7fdf5c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.104+0000 7fdf787de700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c004850 0x7fdf5c004cd0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c240 tx=0x7fdf5c005880 comp rx=0 tx=0).stop 2023-10-16T11:01:24.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.104+0000 7fdf787de700 1 -- 172.21.15.181:0/2022601056 shutdown_connections 2023-10-16T11:01:24.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.104+0000 7fdf787de700 1 --2- 172.21.15.181:0/2022601056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c004850 0x7fdf5c004cd0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.104+0000 7fdf787de700 1 -- 172.21.15.181:0/2022601056 >> 172.21.15.181:0/2022601056 conn(0x7fdf700d04a0 msgr2=0x7fdf700d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:24.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.104+0000 7fdf787de700 1 -- 172.21.15.181:0/2022601056 shutdown_connections 2023-10-16T11:01:24.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.104+0000 7fdf787de700 1 -- 172.21.15.181:0/2022601056 wait complete. 2023-10-16T11:01:24.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.105+0000 7fdf787de700 1 Processor -- start 2023-10-16T11:01:24.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.105+0000 7fdf787de700 1 -- start start 2023-10-16T11:01:24.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c004850 0x7fdf701502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7015c290 0x7fdf7015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf700c85c0 0x7fdf700c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7015cc50 con 0x7fdf700c85c0 2023-10-16T11:01:24.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b4510 con 0x7fdf5c004850 2023-10-16T11:01:24.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b4db0 con 0x7fdf7015c290 2023-10-16T11:01:24.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf700c85c0 0x7fdf700c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf700c85c0 0x7fdf700c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52102/0 (socket says 172.21.15.181:52102) 2023-10-16T11:01:24.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 -- 172.21.15.181:0/26671612 learned_addr learned my addr 172.21.15.181:0/26671612 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:24.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf75d79700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7015c290 0x7fdf7015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf7657a700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c004850 0x7fdf701502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 -- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7015c290 msgr2=0x7fdf7015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7015c290 0x7fdf7015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 -- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c004850 msgr2=0x7fdf701502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c004850 0x7fdf701502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 -- 172.21.15.181:0/26671612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700c4520 con 0x7fdf700c85c0 2023-10-16T11:01:24.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf76d7b700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf700c85c0 0x7fdf700c8a40 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fdf700cdfd0 tx=0x7fdf68011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:24.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf66ffd700 1 -- 172.21.15.181:0/26671612 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800e040 con 0x7fdf700c85c0 2023-10-16T11:01:24.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf66ffd700 1 -- 172.21.15.181:0/26671612 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6800f040 con 0x7fdf700c85c0 2023-10-16T11:01:24.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf70150800 con 0x7fdf700c85c0 2023-10-16T11:01:24.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf66ffd700 1 -- 172.21.15.181:0/26671612 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800fd80 con 0x7fdf700c85c0 2023-10-16T11:01:24.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.106+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf700b4760 con 0x7fdf700c85c0 2023-10-16T11:01:24.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.107+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf70067d20 con 0x7fdf700c85c0 2023-10-16T11:01:24.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.108+0000 7fdf66ffd700 1 -- 172.21.15.181:0/26671612 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdf68015430 con 0x7fdf700c85c0 2023-10-16T11:01:24.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.108+0000 7fdf66ffd700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fdf5406b020 0x7fdf5406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.108+0000 7fdf66ffd700 1 -- 172.21.15.181:0/26671612 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fdf6808a0f0 con 0x7fdf700c85c0 2023-10-16T11:01:24.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.108+0000 7fdf7657a700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fdf5406b020 0x7fdf5406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.109+0000 7fdf7657a700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fdf5406b020 0x7fdf5406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c000c00 tx=0x7fdf6c009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:24.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.113+0000 7fdf66ffd700 1 -- 172.21.15.181:0/26671612 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdf68054e20 con 0x7fdf700c85c0 2023-10-16T11:01:24.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.306+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf700b4ef0 con 0x7fdf700c85c0 2023-10-16T11:01:24.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.307+0000 7fdf66ffd700 1 -- 172.21.15.181:0/26671612 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+17380 (secure 0 0 0) 0x7fdf68058440 con 0x7fdf700c85c0 2023-10-16T11:01:24.310 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:24.311 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":180,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:22.987504+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:01:21.420382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:01:21.958383+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:24.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fdf5406b020 msgr2=0x7fdf5406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fdf5406b020 0x7fdf5406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c000c00 tx=0x7fdf6c009040 comp rx=0 tx=0).stop 2023-10-16T11:01:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf700c85c0 msgr2=0x7fdf700c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf700c85c0 0x7fdf700c8a40 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fdf700cdfd0 tx=0x7fdf68011f80 comp rx=0 tx=0).stop 2023-10-16T11:01:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 shutdown_connections 2023-10-16T11:01:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fdf5406b020 0x7fdf5406d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf700c85c0 0x7fdf700c8a40 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf5c004850 0x7fdf701502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 --2- 172.21.15.181:0/26671612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf7015c290 0x7fdf7015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 >> 172.21.15.181:0/26671612 conn(0x7fdf700d04a0 msgr2=0x7fdf700adba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 shutdown_connections 2023-10-16T11:01:24.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.309+0000 7fdf787de700 1 -- 172.21.15.181:0/26671612 wait complete. 2023-10-16T11:01:24.325 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:01:24.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.448+0000 7efed52ca700 1 Processor -- start 2023-10-16T11:01:24.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.448+0000 7efed52ca700 1 -- start start 2023-10-16T11:01:24.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.449+0000 7efed52ca700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed015da70 0x7efed00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.449+0000 7efed52ca700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efed006afb0 0x7efed01615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.449+0000 7efed52ca700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed00cef00 con 0x7efed00ce6a0 2023-10-16T11:01:24.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.449+0000 7efed52ca700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed00cec20 con 0x7efed006afb0 2023-10-16T11:01:24.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.449+0000 7efed52ca700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed0071c80 con 0x7efed015da70 2023-10-16T11:01:24.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.449+0000 7efeceffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed015da70 0x7efed00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efecf7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efed006afb0 0x7efed01615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efeceffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed015da70 0x7efed00ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:49132/0 (socket says 172.21.15.181:49132) 2023-10-16T11:01:24.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efeceffd700 1 -- 172.21.15.181:0/2621732408 learned_addr learned my addr 172.21.15.181:0/2621732408 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:24.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efeceffd700 1 -- 172.21.15.181:0/2621732408 >> v2:172.21.15.190:3300/0 conn(0x7efed006afb0 msgr2=0x7efed01615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:24.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efeceffd700 1 --2- 172.21.15.181:0/2621732408 >> v2:172.21.15.190:3300/0 conn(0x7efed006afb0 0x7efed01615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efeceffd700 1 -- 172.21.15.181:0/2621732408 >> v1:172.21.15.190:6789/0 conn(0x7efed00ce6a0 legacy=0x7efed006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efeceffd700 1 -- 172.21.15.181:0/2621732408 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0071f60 con 0x7efed015da70 2023-10-16T11:01:24.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efeceffd700 1 --2- 172.21.15.181:0/2621732408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed015da70 0x7efed00ce160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efec40098f0 tx=0x7efec4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=712fa6d61a277834 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:24.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efecdffb700 1 -- 172.21.15.181:0/2621732408 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4011830 con 0x7efed015da70 2023-10-16T11:01:24.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efecdffb700 1 -- 172.21.15.181:0/2621732408 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec4011990 con 0x7efed015da70 2023-10-16T11:01:24.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.450+0000 7efecdffb700 1 -- 172.21.15.181:0/2621732408 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4011d20 con 0x7efed015da70 2023-10-16T11:01:24.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 -- 172.21.15.181:0/2621732408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed015da70 msgr2=0x7efed00ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 --2- 172.21.15.181:0/2621732408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed015da70 0x7efed00ce160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efec40098f0 tx=0x7efec4009c00 comp rx=0 tx=0).stop 2023-10-16T11:01:24.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 -- 172.21.15.181:0/2621732408 shutdown_connections 2023-10-16T11:01:24.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 --2- 172.21.15.181:0/2621732408 >> v2:172.21.15.190:3300/0 conn(0x7efed006afb0 0x7efed01615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 --2- 172.21.15.181:0/2621732408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed015da70 0x7efed00ce160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 -- 172.21.15.181:0/2621732408 >> 172.21.15.181:0/2621732408 conn(0x7efed00cd010 msgr2=0x7efed00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:24.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 -- 172.21.15.181:0/2621732408 shutdown_connections 2023-10-16T11:01:24.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.451+0000 7efed52ca700 1 -- 172.21.15.181:0/2621732408 wait complete. 2023-10-16T11:01:24.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.452+0000 7efed52ca700 1 Processor -- start 2023-10-16T11:01:24.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.452+0000 7efed52ca700 1 -- start start 2023-10-16T11:01:24.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efed52ca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed006afb0 0x7efed014b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efed52ca700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed00ce6a0 0x7efed0161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efed52ca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed01f4330 0x7efed0161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efed52ca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed015ec90 con 0x7efed006afb0 2023-10-16T11:01:24.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efed52ca700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed01f3640 con 0x7efed01f4330 2023-10-16T11:01:24.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efed52ca700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed01f3ea0 con 0x7efed00ce6a0 2023-10-16T11:01:24.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed006afb0 0x7efed014b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed006afb0 0x7efed014b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52110/0 (socket says 172.21.15.181:52110) 2023-10-16T11:01:24.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 -- 172.21.15.181:0/2887913818 learned_addr learned my addr 172.21.15.181:0/2887913818 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:24.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efece7fc700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed00ce6a0 0x7efed0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efecf7fe700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed01f4330 0x7efed0161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 -- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed00ce6a0 msgr2=0x7efed0161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed00ce6a0 0x7efed0161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 -- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed01f4330 msgr2=0x7efed0161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:24.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed01f4330 0x7efed0161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:24.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.453+0000 7efeceffd700 1 -- 172.21.15.181:0/2887913818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4009550 con 0x7efed006afb0 2023-10-16T11:01:24.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.454+0000 7efeceffd700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed006afb0 0x7efed014b590 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7efec4000c00 tx=0x7efec4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:24.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.454+0000 7efeb77fe700 1 -- 172.21.15.181:0/2887913818 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4004340 con 0x7efed006afb0 2023-10-16T11:01:24.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.454+0000 7efeb77fe700 1 -- 172.21.15.181:0/2887913818 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec401f070 con 0x7efed006afb0 2023-10-16T11:01:24.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.454+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed014ea50 con 0x7efed006afb0 2023-10-16T11:01:24.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.454+0000 7efeb77fe700 1 -- 172.21.15.181:0/2887913818 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec40036a0 con 0x7efed006afb0 2023-10-16T11:01:24.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.454+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed01f3b30 con 0x7efed006afb0 2023-10-16T11:01:24.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.455+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed006f6c0 con 0x7efed006afb0 2023-10-16T11:01:24.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.455+0000 7efeb77fe700 1 -- 172.21.15.181:0/2887913818 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efec4021030 con 0x7efed006afb0 2023-10-16T11:01:24.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.456+0000 7efeb77fe700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7efebc06b020 0x7efebc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:24.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.456+0000 7efeb77fe700 1 -- 172.21.15.181:0/2887913818 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6205+0+0 (secure 0 0 0) 0x7efec408aac0 con 0x7efed006afb0 2023-10-16T11:01:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.456+0000 7efece7fc700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7efebc06b020 0x7efebc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.456+0000 7efece7fc700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7efebc06b020 0x7efebc06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efed01621f0 tx=0x7efec0009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:24.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.460+0000 7efeb77fe700 1 -- 172.21.15.181:0/2887913818 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efec4055880 con 0x7efed006afb0 2023-10-16T11:01:24.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:24.704+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efed00cc100 con 0x7efed006afb0 2023-10-16T11:01:25.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-16T11:01:25.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-10-16T11:01:25.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-16T11:01:25.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-16T11:01:25.580 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-10-16T11:01:25.579+0000 7f6ada3a5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:01:25.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.628+0000 7efeb77fe700 1 -- 172.21.15.181:0/2887913818 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) v1 ==== 118+0+0 (secure 0 0 0) 0x7efec4058ea0 con 0x7efed006afb0 2023-10-16T11:01:25.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7efebc06b020 msgr2=0x7efebc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:25.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7efebc06b020 0x7efebc06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efed01621f0 tx=0x7efec0009040 comp rx=0 tx=0).stop 2023-10-16T11:01:25.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed006afb0 msgr2=0x7efed014b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:25.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed006afb0 0x7efed014b590 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7efec4000c00 tx=0x7efec4004750 comp rx=0 tx=0).stop 2023-10-16T11:01:25.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 shutdown_connections 2023-10-16T11:01:25.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7efebc06b020 0x7efebc06d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed01f4330 0x7efed0161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed006afb0 0x7efed014b590 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 --2- 172.21.15.181:0/2887913818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed00ce6a0 0x7efed0161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 >> 172.21.15.181:0/2887913818 conn(0x7efed00cd010 msgr2=0x7efed006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:25.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 shutdown_connections 2023-10-16T11:01:25.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.631+0000 7efed52ca700 1 -- 172.21.15.181:0/2887913818 wait complete. 2023-10-16T11:01:25.647 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:01:25.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:01:25.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.776+0000 7fece50eb700 1 Processor -- start 2023-10-16T11:01:25.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.776+0000 7fece50eb700 1 -- start start 2023-10-16T11:01:25.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fece50eb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fece00cdd00 0x7fece006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:25.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fece50eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00666a0 0x7fece006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:25.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fece50eb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece00ca300 con 0x7fece00ce0e0 2023-10-16T11:01:25.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fece50eb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece006e5e0 con 0x7fece00cdd00 2023-10-16T11:01:25.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fece50eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece00ca580 con 0x7fece00666a0 2023-10-16T11:01:25.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecded9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fece00cdd00 0x7fece006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:25.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecde59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00666a0 0x7fece006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:25.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecdf59e700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fece00ce0e0 0x7fece006ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:47142/0 (socket says 172.21.15.181:47142) 2023-10-16T11:01:25.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecdf59e700 1 -- 172.21.15.181:0/2921364791 learned_addr learned my addr 172.21.15.181:0/2921364791 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:25.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecddd9b700 1 -- 172.21.15.181:0/2921364791 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035977373 0 0) 0x7fece00ca300 con 0x7fece00ce0e0 2023-10-16T11:01:25.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecddd9b700 1 -- 172.21.15.181:0/2921364791 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc80035b0 con 0x7fece00ce0e0 2023-10-16T11:01:25.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecded9d700 1 -- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00666a0 msgr2=0x7fece006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:25.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecded9d700 1 --2- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00666a0 0x7fece006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.777+0000 7fecded9d700 1 -- 172.21.15.181:0/2921364791 >> v1:172.21.15.190:6789/0 conn(0x7fece00ce0e0 legacy=0x7fece006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:25.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fecded9d700 1 -- 172.21.15.181:0/2921364791 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00ca860 con 0x7fece00cdd00 2023-10-16T11:01:25.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fecde59c700 1 --2- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00666a0 0x7fece006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:25.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fecded9d700 1 --2- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00cdd00 0x7fece006f5f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009840 tx=0x7fecd0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea40720cb10539f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:25.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fecddd9b700 1 -- 172.21.15.181:0/2921364791 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0011830 con 0x7fece00cdd00 2023-10-16T11:01:25.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fecddd9b700 1 -- 172.21.15.181:0/2921364791 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd0011990 con 0x7fece00cdd00 2023-10-16T11:01:25.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fecddd9b700 1 -- 172.21.15.181:0/2921364791 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0011d20 con 0x7fece00cdd00 2023-10-16T11:01:25.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fece50eb700 1 -- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00cdd00 msgr2=0x7fece006f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:25.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fece50eb700 1 --2- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00cdd00 0x7fece006f5f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009840 tx=0x7fecd0009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:25.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.778+0000 7fece50eb700 1 -- 172.21.15.181:0/2921364791 shutdown_connections 2023-10-16T11:01:25.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.779+0000 7fece50eb700 1 --2- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00cdd00 0x7fece006f5f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.779+0000 7fece50eb700 1 --2- 172.21.15.181:0/2921364791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00666a0 0x7fece006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.779+0000 7fece50eb700 1 -- 172.21.15.181:0/2921364791 >> 172.21.15.181:0/2921364791 conn(0x7fece00cc730 msgr2=0x7fece00ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:25.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.779+0000 7fece50eb700 1 -- 172.21.15.181:0/2921364791 shutdown_connections 2023-10-16T11:01:25.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.779+0000 7fece50eb700 1 -- 172.21.15.181:0/2921364791 wait complete. 2023-10-16T11:01:25.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.780+0000 7fece50eb700 1 Processor -- start 2023-10-16T11:01:25.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.780+0000 7fece50eb700 1 -- start start 2023-10-16T11:01:25.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fece50eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece00666a0 0x7fece006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:25.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fece50eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00cdd00 0x7fece015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:25.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fece50eb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00ce0e0 0x7fece0143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:25.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fece50eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece015cf70 con 0x7fece00666a0 2023-10-16T11:01:25.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fece50eb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece015d250 con 0x7fece00ce0e0 2023-10-16T11:01:25.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fece50eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece015d530 con 0x7fece00cdd00 2023-10-16T11:01:25.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece00666a0 0x7fece006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:25.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece00666a0 0x7fece006ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52124/0 (socket says 172.21.15.181:52124) 2023-10-16T11:01:25.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 -- 172.21.15.181:0/4180592498 learned_addr learned my addr 172.21.15.181:0/4180592498 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:25.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecde59c700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00cdd00 0x7fece015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:25.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecdf59e700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00ce0e0 0x7fece0143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:25.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 -- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00cdd00 msgr2=0x7fece015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:25.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00cdd00 0x7fece015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 -- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00ce0e0 msgr2=0x7fece0143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00ce0e0 0x7fece0143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 -- 172.21.15.181:0/4180592498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40094a0 con 0x7fece00666a0 2023-10-16T11:01:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecdf59e700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00ce0e0 0x7fece0143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.781+0000 7fecded9d700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece00666a0 0x7fece006ab10 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fecd00090b0 tx=0x7fecd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.782+0000 7fecc77fe700 1 -- 172.21.15.181:0/4180592498 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00036a0 con 0x7fece00666a0 2023-10-16T11:01:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.782+0000 7fecc77fe700 1 -- 172.21.15.181:0/4180592498 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd001e070 con 0x7fece00666a0 2023-10-16T11:01:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.782+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd00094a0 con 0x7fece00666a0 2023-10-16T11:01:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.782+0000 7fecc77fe700 1 -- 172.21.15.181:0/4180592498 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0003a40 con 0x7fece00666a0 2023-10-16T11:01:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.782+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece015daf0 con 0x7fece00666a0 2023-10-16T11:01:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.783+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece015dce0 con 0x7fece00666a0 2023-10-16T11:01:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.783+0000 7fecc77fe700 1 -- 172.21.15.181:0/4180592498 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fecd0003ba0 con 0x7fece00666a0 2023-10-16T11:01:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.783+0000 7fecc77fe700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7feccc06b020 0x7feccc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.784+0000 7fecc77fe700 1 -- 172.21.15.181:0/4180592498 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fecd008a7f0 con 0x7fece00666a0 2023-10-16T11:01:25.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.784+0000 7fecde59c700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7feccc06b020 0x7feccc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.784+0000 7fecde59c700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7feccc06b020 0x7feccc06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fece00ad170 tx=0x7fecd4009210 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:25.788+0000 7fecc77fe700 1 -- 172.21.15.181:0/4180592498 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fecd0055630 con 0x7fece00666a0 2023-10-16T11:01:26.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.026+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fece01612f0 con 0x7fece00666a0 2023-10-16T11:01:26.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.776+0000 7fecc77fe700 1 -- 172.21.15.181:0/4180592498 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7fecd0058c50 con 0x7fece00666a0 2023-10-16T11:01:26.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7feccc06b020 msgr2=0x7feccc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:26.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7feccc06b020 0x7feccc06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fece00ad170 tx=0x7fecd4009210 comp rx=0 tx=0).stop 2023-10-16T11:01:26.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece00666a0 msgr2=0x7fece006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:26.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece00666a0 0x7fece006ab10 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fecd00090b0 tx=0x7fecd0004750 comp rx=0 tx=0).stop 2023-10-16T11:01:26.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 shutdown_connections 2023-10-16T11:01:26.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7feccc06b020 0x7feccc06d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece00ce0e0 0x7fece0143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece00666a0 0x7fece006ab10 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 --2- 172.21.15.181:0/4180592498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fece00cdd00 0x7fece015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 >> 172.21.15.181:0/4180592498 conn(0x7fece00cc730 msgr2=0x7fece0160550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:26.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 shutdown_connections 2023-10-16T11:01:26.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.780+0000 7fece50eb700 1 -- 172.21.15.181:0/4180592498 wait complete. 2023-10-16T11:01:26.798 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:01:26.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:01:26.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.927+0000 7ff77fe4a700 1 Processor -- start 2023-10-16T11:01:26.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.928+0000 7ff77fe4a700 1 -- start start 2023-10-16T11:01:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77fe4a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780af450 0x7ff7780af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77fe4a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7780cd470 0x7ff7780cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77fe4a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77806f870 con 0x7ff77815dfd0 2023-10-16T11:01:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77fe4a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7780d1160 con 0x7ff7780cd470 2023-10-16T11:01:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77fe4a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7780cc8e0 con 0x7ff7780af450 2023-10-16T11:01:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77d3e5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780af450 0x7ff7780af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77e3e7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7780cd470 0x7ff7780cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77dbe6700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff77815dfd0 0x7ff77806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:47152/0 (socket says 172.21.15.181:47152) 2023-10-16T11:01:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77dbe6700 1 -- 172.21.15.181:0/2122811807 learned_addr learned my addr 172.21.15.181:0/2122811807 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77e3e7700 1 -- 172.21.15.181:0/2122811807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780af450 msgr2=0x7ff7780af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77e3e7700 1 --2- 172.21.15.181:0/2122811807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780af450 0x7ff7780af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77e3e7700 1 -- 172.21.15.181:0/2122811807 >> v1:172.21.15.190:6789/0 conn(0x7ff77815dfd0 legacy=0x7ff77806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77e3e7700 1 -- 172.21.15.181:0/2122811807 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7780ccbc0 con 0x7ff7780cd470 2023-10-16T11:01:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.929+0000 7ff77cbe4700 1 -- 172.21.15.181:0/2122811807 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947270013 0 0) 0x7ff77806f870 con 0x7ff77815dfd0 2023-10-16T11:01:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77e3e7700 1 --2- 172.21.15.181:0/2122811807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780cd470 0x7ff7780cd920 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff770009d90 tx=0x7ff77000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d774405d448de105 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77cbe4700 1 -- 172.21.15.181:0/2122811807 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77000e070 con 0x7ff7780cd470 2023-10-16T11:01:26.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77cbe4700 1 -- 172.21.15.181:0/2122811807 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77000bb20 con 0x7ff7780cd470 2023-10-16T11:01:26.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77cbe4700 1 -- 172.21.15.181:0/2122811807 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770012430 con 0x7ff7780cd470 2023-10-16T11:01:26.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 -- 172.21.15.181:0/2122811807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780cd470 msgr2=0x7ff7780cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:26.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/2122811807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780cd470 0x7ff7780cd920 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff770009d90 tx=0x7ff77000b330 comp rx=0 tx=0).stop 2023-10-16T11:01:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 -- 172.21.15.181:0/2122811807 shutdown_connections 2023-10-16T11:01:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/2122811807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780cd470 0x7ff7780cd920 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/2122811807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780af450 0x7ff7780af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 -- 172.21.15.181:0/2122811807 >> 172.21.15.181:0/2122811807 conn(0x7ff7780d04a0 msgr2=0x7ff7780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 -- 172.21.15.181:0/2122811807 shutdown_connections 2023-10-16T11:01:26.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.930+0000 7ff77fe4a700 1 -- 172.21.15.181:0/2122811807 wait complete. 2023-10-16T11:01:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 Processor -- start 2023-10-16T11:01:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 -- start start 2023-10-16T11:01:26.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780af450 0x7ff7781f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780cd470 0x7ff7780b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff77815dfd0 0x7ff7780c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7780b1eb0 con 0x7ff77815dfd0 2023-10-16T11:01:26.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7781f2e40 con 0x7ff7780af450 2023-10-16T11:01:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.932+0000 7ff77fe4a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7780b12a0 con 0x7ff7780cd470 2023-10-16T11:01:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff77815dfd0 0x7ff7780c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff77815dfd0 0x7ff7780c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52142/0 (socket says 172.21.15.181:52142) 2023-10-16T11:01:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 -- 172.21.15.181:0/163202633 learned_addr learned my addr 172.21.15.181:0/163202633 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:26.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77dbe6700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780af450 0x7ff7781f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77d3e5700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780cd470 0x7ff7780b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 -- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780cd470 msgr2=0x7ff7780b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780cd470 0x7ff7780b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 -- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780af450 msgr2=0x7ff7781f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:26.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780af450 0x7ff7781f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 -- 172.21.15.181:0/163202633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770009a70 con 0x7ff77815dfd0 2023-10-16T11:01:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77e3e7700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff77815dfd0 0x7ff7780c5440 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff7780716a0 tx=0x7ff770004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:26.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff76e7fc700 1 -- 172.21.15.181:0/163202633 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7700098f0 con 0x7ff77815dfd0 2023-10-16T11:01:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff76e7fc700 1 -- 172.21.15.181:0/163202633 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770020070 con 0x7ff77815dfd0 2023-10-16T11:01:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7780b1580 con 0x7ff77815dfd0 2023-10-16T11:01:26.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff76e7fc700 1 -- 172.21.15.181:0/163202633 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7700276a0 con 0x7ff77815dfd0 2023-10-16T11:01:26.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.933+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7780b1b40 con 0x7ff77815dfd0 2023-10-16T11:01:26.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.934+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7780b2020 con 0x7ff77815dfd0 2023-10-16T11:01:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.935+0000 7ff76e7fc700 1 -- 172.21.15.181:0/163202633 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff770012430 con 0x7ff77815dfd0 2023-10-16T11:01:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.935+0000 7ff76e7fc700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff75c06b020 0x7ff75c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.935+0000 7ff76e7fc700 1 -- 172.21.15.181:0/163202633 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7ff770022070 con 0x7ff77815dfd0 2023-10-16T11:01:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.935+0000 7ff77dbe6700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff75c06b020 0x7ff75c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:26.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.935+0000 7ff77dbe6700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff75c06b020 0x7ff75c06d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff768000fb0 tx=0x7ff768007ce0 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:26.939+0000 7ff76e7fc700 1 -- 172.21.15.181:0/163202633 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff770056660 con 0x7ff77815dfd0 2023-10-16T11:01:27.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.178+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff778067d20 con 0x7ff77815dfd0 2023-10-16T11:01:27.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.179+0000 7ff76e7fc700 1 -- 172.21.15.181:0/163202633 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4930 (secure 0 0 0) 0x7ff770059c80 con 0x7ff77815dfd0 2023-10-16T11:01:27.181 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:27.181 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1903420066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1903420066},{"type":"v1","addr":"172.21.15.181:6836","nonce":1903420066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/586483327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":586483327},{"type":"v1","addr":"172.21.15.190:6835","nonce":586483327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2078148321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2078148321},{"type":"v1","addr":"172.21.15.181:6837","nonce":2078148321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:01:22.987835+0000","modified":"2023-10-16T11:01:26.027862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10197},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10197":{"gid":10197,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2135938520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2135938520},{"type":"v1","addr":"172.21.15.190:6834","nonce":2135938520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:01:27.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff75c06b020 msgr2=0x7ff75c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff75c06b020 0x7ff75c06d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff768000fb0 tx=0x7ff768007ce0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff77815dfd0 msgr2=0x7ff7780c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff77815dfd0 0x7ff7780c5440 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff7780716a0 tx=0x7ff770004ad0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 shutdown_connections 2023-10-16T11:01:27.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7ff75c06b020 0x7ff75c06d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff77815dfd0 0x7ff7780c5440 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7780af450 0x7ff7781f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 --2- 172.21.15.181:0/163202633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7780cd470 0x7ff7780b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 >> 172.21.15.181:0/163202633 conn(0x7ff7780d04a0 msgr2=0x7ff77806d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:27.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 shutdown_connections 2023-10-16T11:01:27.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.181+0000 7ff77fe4a700 1 -- 172.21.15.181:0/163202633 wait complete. 2023-10-16T11:01:27.186 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 94 2023-10-16T11:01:27.195 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:01:27.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.315+0000 7f5040480700 1 Processor -- start 2023-10-16T11:01:27.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.315+0000 7f5040480700 1 -- start start 2023-10-16T11:01:27.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f5040480700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50380af450 0x7f50380af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f5040480700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50380cd470 0x7f50380cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f5040480700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503806f870 con 0x7f503815dfd0 2023-10-16T11:01:27.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f5040480700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50380d1160 con 0x7f50380cd470 2023-10-16T11:01:27.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f5040480700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50380cc8e0 con 0x7f50380af450 2023-10-16T11:01:27.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f503ea1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50380cd470 0x7f50380cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f503e21c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f503815dfd0 0x7f503806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34992/0 (socket says 172.21.15.181:34992) 2023-10-16T11:01:27.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.316+0000 7f503e21c700 1 -- 172.21.15.181:0/2223166882 learned_addr learned my addr 172.21.15.181:0/2223166882 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:27.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503da1b700 1 --2- 172.21.15.181:0/2223166882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50380af450 0x7f50380af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503ea1d700 1 -- 172.21.15.181:0/2223166882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50380af450 msgr2=0x7f50380af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503ea1d700 1 --2- 172.21.15.181:0/2223166882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50380af450 0x7f50380af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503ea1d700 1 -- 172.21.15.181:0/2223166882 >> v1:172.21.15.181:6789/0 conn(0x7f503815dfd0 legacy=0x7f503806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503ea1d700 1 -- 172.21.15.181:0/2223166882 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380ccbc0 con 0x7f50380cd470 2023-10-16T11:01:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503d21a700 1 -- 172.21.15.181:0/2223166882 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495865035 0 0) 0x7f503806f870 con 0x7f503815dfd0 2023-10-16T11:01:27.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503da1b700 1 --2- 172.21.15.181:0/2223166882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50380af450 0x7f50380af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:27.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503ea1d700 1 --2- 172.21.15.181:0/2223166882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f50380cd920 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f5030009d90 tx=0x7f503000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34670fc462dcd55a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:27.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503d21a700 1 -- 172.21.15.181:0/2223166882 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503000e070 con 0x7f50380cd470 2023-10-16T11:01:27.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503d21a700 1 -- 172.21.15.181:0/2223166882 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503000bbd0 con 0x7f50380cd470 2023-10-16T11:01:27.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.317+0000 7f503d21a700 1 -- 172.21.15.181:0/2223166882 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030012430 con 0x7f50380cd470 2023-10-16T11:01:27.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 -- 172.21.15.181:0/2223166882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 msgr2=0x7f50380cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 --2- 172.21.15.181:0/2223166882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f50380cd920 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f5030009d90 tx=0x7f503000b330 comp rx=0 tx=0).stop 2023-10-16T11:01:27.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 -- 172.21.15.181:0/2223166882 shutdown_connections 2023-10-16T11:01:27.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 --2- 172.21.15.181:0/2223166882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f50380cd920 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 --2- 172.21.15.181:0/2223166882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50380af450 0x7f50380af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 -- 172.21.15.181:0/2223166882 >> 172.21.15.181:0/2223166882 conn(0x7f50380d04a0 msgr2=0x7f50380d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:27.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 -- 172.21.15.181:0/2223166882 shutdown_connections 2023-10-16T11:01:27.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.318+0000 7f5040480700 1 -- 172.21.15.181:0/2223166882 wait complete. 2023-10-16T11:01:27.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.319+0000 7f5040480700 1 Processor -- start 2023-10-16T11:01:27.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.319+0000 7f5040480700 1 -- start start 2023-10-16T11:01:27.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f5040480700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50380af450 0x7f5038159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f5040480700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f503814b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f5040480700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f503815dfd0 0x7f50381592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f5040480700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503815a570 con 0x7f50380cd470 2023-10-16T11:01:27.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f5040480700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50380c8120 con 0x7f50380af450 2023-10-16T11:01:27.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f5040480700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50380c8400 con 0x7f503815dfd0 2023-10-16T11:01:27.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f503da1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f503814b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f503da1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f503814b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52152/0 (socket says 172.21.15.181:52152) 2023-10-16T11:01:27.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f503da1b700 1 -- 172.21.15.181:0/3936031692 learned_addr learned my addr 172.21.15.181:0/3936031692 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:27.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f503ea1d700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f503815dfd0 0x7f50381592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.320+0000 7f503e21c700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50380af450 0x7f5038159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f503da1b700 1 -- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f503815dfd0 msgr2=0x7f50381592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f503da1b700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f503815dfd0 0x7f50381592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f503da1b700 1 -- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50380af450 msgr2=0x7f5038159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f503da1b700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50380af450 0x7f5038159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f503da1b700 1 -- 172.21.15.181:0/3936031692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5030009a70 con 0x7f50380cd470 2023-10-16T11:01:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f503ea1d700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f503815dfd0 0x7f50381592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f503da1b700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f503814b880 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f503400e810 tx=0x7f503400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f502effd700 1 -- 172.21.15.181:0/3936031692 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034004230 con 0x7f50380cd470 2023-10-16T11:01:27.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f502effd700 1 -- 172.21.15.181:0/3936031692 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034004390 con 0x7f50380cd470 2023-10-16T11:01:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50380c86e0 con 0x7f50380cd470 2023-10-16T11:01:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f502effd700 1 -- 172.21.15.181:0/3936031692 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50340036a0 con 0x7f50380cd470 2023-10-16T11:01:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.321+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503814bdc0 con 0x7f50380cd470 2023-10-16T11:01:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.322+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5038067d20 con 0x7f50380cd470 2023-10-16T11:01:27.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.322+0000 7f502effd700 1 -- 172.21.15.181:0/3936031692 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5034003800 con 0x7f50380cd470 2023-10-16T11:01:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.323+0000 7f502effd700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f501c06b020 0x7f501c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.323+0000 7f503e21c700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f501c06b020 0x7f501c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.323+0000 7f502effd700 1 -- 172.21.15.181:0/3936031692 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f50340894a0 con 0x7f50380cd470 2023-10-16T11:01:27.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.323+0000 7f503e21c700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f501c06b020 0x7f501c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50380ca570 tx=0x7f5028009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.327+0000 7f502effd700 1 -- 172.21.15.181:0/3936031692 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50340541d0 con 0x7f50380cd470 2023-10-16T11:01:27.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.521+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50380cec90 con 0x7f50380cd470 2023-10-16T11:01:27.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.522+0000 7f502effd700 1 -- 172.21.15.181:0/3936031692 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+17367 (secure 0 0 0) 0x7f50340577f0 con 0x7f50380cd470 2023-10-16T11:01:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:27.524 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":181,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:01:24.599019+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:01:21.420382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:01:21.958383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:01:27.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f501c06b020 msgr2=0x7f501c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f501c06b020 0x7f501c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50380ca570 tx=0x7f5028009040 comp rx=0 tx=0).stop 2023-10-16T11:01:27.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 msgr2=0x7f503814b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f503814b880 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f503400e810 tx=0x7f503400ebd0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 shutdown_connections 2023-10-16T11:01:27.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f501c06b020 0x7f501c06d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f503815dfd0 0x7f50381592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50380af450 0x7f5038159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 --2- 172.21.15.181:0/3936031692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50380cd470 0x7f503814b880 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 >> 172.21.15.181:0/3936031692 conn(0x7f50380d04a0 msgr2=0x7f503806d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:27.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 shutdown_connections 2023-10-16T11:01:27.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.524+0000 7f5040480700 1 -- 172.21.15.181:0/3936031692 wait complete. 2023-10-16T11:01:27.539 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:01:27.539 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:01:27.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.658+0000 7f7a4d12d700 1 Processor -- start 2023-10-16T11:01:27.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.658+0000 7f7a4d12d700 1 -- start start 2023-10-16T11:01:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.659+0000 7f7a4d12d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.659+0000 7f7a4d12d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4806ed00 con 0x7f7a480cb0c0 2023-10-16T11:01:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.659+0000 7f7a4d12d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a480d02d0 con 0x7f7a4815e6d0 2023-10-16T11:01:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.659+0000 7f7a4d12d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4806e0f0 con 0x7f7a4815eab0 2023-10-16T11:01:27.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.659+0000 7f7a4759e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7a480cb0c0 0x7f7a4815a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35010/0 (socket says 172.21.15.181:35010) 2023-10-16T11:01:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.659+0000 7f7a4759e700 1 -- 172.21.15.181:0/2450188630 learned_addr learned my addr 172.21.15.181:0/2450188630 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a4659c700 1 --2- 172.21.15.181:0/2450188630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064156187 0 0) 0x7f7a4806ed00 con 0x7f7a480cb0c0 2023-10-16T11:01:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a300035b0 con 0x7f7a480cb0c0 2023-10-16T11:01:27.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149791547 0 0) 0x7f7a480d02d0 con 0x7f7a4815e6d0 2023-10-16T11:01:27.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a4806ed00 con 0x7f7a4815e6d0 2023-10-16T11:01:27.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 818676246 0 0) 0x7f7a300035b0 con 0x7f7a480cb0c0 2023-10-16T11:01:27.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a4659c700 1 -- 172.21.15.181:0/2450188630 >> v1:172.21.15.190:6789/0 conn(0x7f7a4815e6d0 legacy=0x7f7a4815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a4659c700 1 -- 172.21.15.181:0/2450188630 >> v1:172.21.15.181:6789/0 conn(0x7f7a480cb0c0 legacy=0x7f7a4815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a4659c700 1 -- 172.21.15.181:0/2450188630 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4806e3d0 con 0x7f7a4815eab0 2023-10-16T11:01:27.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a4659c700 1 --2- 172.21.15.181:0/2450188630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480d0fc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009870 tx=0x7f7a38009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3bf914bfcbca989 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:27.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400019f0 con 0x7f7a4815eab0 2023-10-16T11:01:27.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a38004030 con 0x7f7a4815eab0 2023-10-16T11:01:27.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.660+0000 7f7a45d9b700 1 -- 172.21.15.181:0/2450188630 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380043c0 con 0x7f7a4815eab0 2023-10-16T11:01:27.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.661+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2450188630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 msgr2=0x7f7a480d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.661+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2450188630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480d0fc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009870 tx=0x7f7a38009b80 comp rx=0 tx=0).stop 2023-10-16T11:01:27.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.661+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2450188630 shutdown_connections 2023-10-16T11:01:27.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.661+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2450188630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480d0fc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.661+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2450188630 >> 172.21.15.181:0/2450188630 conn(0x7f7a480d04a0 msgr2=0x7f7a480d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:27.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.661+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2450188630 shutdown_connections 2023-10-16T11:01:27.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.661+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2450188630 wait complete. 2023-10-16T11:01:27.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.662+0000 7f7a4d12d700 1 Processor -- start 2023-10-16T11:01:27.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.662+0000 7f7a4d12d700 1 -- start start 2023-10-16T11:01:27.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4d12d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a480cb0c0 0x7f7a4815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4d12d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a4815e6d0 0x7f7a481585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4d12d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4d12d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4815d8e0 con 0x7f7a480cb0c0 2023-10-16T11:01:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4d12d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a481f1460 con 0x7f7a4815e6d0 2023-10-16T11:01:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4d12d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4815cfb0 con 0x7f7a4815eab0 2023-10-16T11:01:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a46d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a480cb0c0 0x7f7a4815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a46d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a480cb0c0 0x7f7a4815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52166/0 (socket says 172.21.15.181:52166) 2023-10-16T11:01:27.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a46d9d700 1 -- 172.21.15.181:0/2611630615 learned_addr learned my addr 172.21.15.181:0/2611630615 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4659c700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a4815e6d0 0x7f7a481585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.663+0000 7f7a4759e700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a46d9d700 1 -- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 msgr2=0x7f7a480c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a46d9d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a46d9d700 1 -- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a4815e6d0 msgr2=0x7f7a481585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a46d9d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a4815e6d0 0x7f7a481585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a46d9d700 1 -- 172.21.15.181:0/2611630615 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380094a0 con 0x7f7a480cb0c0 2023-10-16T11:01:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a46d9d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a480cb0c0 0x7f7a4815af60 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c001f00 tx=0x7f7a3c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a2f7fe700 1 -- 172.21.15.181:0/2611630615 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0052d0 con 0x7f7a480cb0c0 2023-10-16T11:01:27.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a2f7fe700 1 -- 172.21.15.181:0/2611630615 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c005430 con 0x7f7a480cb0c0 2023-10-16T11:01:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4815d290 con 0x7f7a480cb0c0 2023-10-16T11:01:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a2f7fe700 1 -- 172.21.15.181:0/2611630615 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0038f0 con 0x7f7a480cb0c0 2023-10-16T11:01:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.664+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4815dd70 con 0x7f7a480cb0c0 2023-10-16T11:01:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.665+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4815dbc0 con 0x7f7a480cb0c0 2023-10-16T11:01:27.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.665+0000 7f7a2f7fe700 1 -- 172.21.15.181:0/2611630615 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7a3c003a50 con 0x7f7a480cb0c0 2023-10-16T11:01:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.666+0000 7f7a2f7fe700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7a3406b020 0x7f7a3406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.666+0000 7f7a4659c700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7a3406b020 0x7f7a3406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.666+0000 7f7a2f7fe700 1 -- 172.21.15.181:0/2611630615 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f7a3c088e90 con 0x7f7a480cb0c0 2023-10-16T11:01:27.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.666+0000 7f7a4659c700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7a3406b020 0x7f7a3406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a380090b0 tx=0x7f7a38008030 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:27.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.670+0000 7f7a2f7fe700 1 -- 172.21.15.181:0/2611630615 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a3c053c40 con 0x7f7a480cb0c0 2023-10-16T11:01:27.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.858+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 --> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a4815df00 con 0x7f7a3406b020 2023-10-16T11:01:27.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.866+0000 7f7a2f7fe700 1 -- 172.21.15.181:0/2611630615 <== mgr.7889 v2:172.21.15.181:6832/303240986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290384 (secure 0 0 0) 0x7f7a38004570 con 0x7f7a3406b020 2023-10-16T11:01:27.872 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:27.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7a3406b020 msgr2=0x7f7a3406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7a3406b020 0x7f7a3406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a380090b0 tx=0x7f7a38008030 comp rx=0 tx=0).stop 2023-10-16T11:01:27.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a480cb0c0 msgr2=0x7f7a4815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:27.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a480cb0c0 0x7f7a4815af60 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c001f00 tx=0x7f7a3c00c860 comp rx=0 tx=0).stop 2023-10-16T11:01:27.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 shutdown_connections 2023-10-16T11:01:27.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f7a3406b020 0x7f7a3406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a4815eab0 0x7f7a480c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a480cb0c0 0x7f7a4815af60 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 --2- 172.21.15.181:0/2611630615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a4815e6d0 0x7f7a481585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:27.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 >> 172.21.15.181:0/2611630615 conn(0x7f7a480d04a0 msgr2=0x7f7a480cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:27.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 shutdown_connections 2023-10-16T11:01:27.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:27.870+0000 7f7a4d12d700 1 -- 172.21.15.181:0/2611630615 wait complete. 2023-10-16T11:01:27.877 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:01:27.884 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T11:01:27.541851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405732,"kb_used_data":18864,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754568988,"statfs":{"total":773094113280,"available":772678643712,"internally_reserved":0,"allocated":19316736,"data_stored":10317068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":396021304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.959520"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014069+0000","last_change":"2023-10-16T11:01:23.625128+0000","last_active":"2023-10-16T11:01:25.014069+0000","last_peered":"2023-10-16T11:01:25.014069+0000","last_clean":"2023-10-16T11:01:25.014069+0000","last_became_active":"2023-10-16T11:01:23.624860+0000","last_became_peered":"2023-10-16T11:01:23.624860+0000","last_unstale":"2023-10-16T11:01:25.014069+0000","last_undegraded":"2023-10-16T11:01:25.014069+0000","last_fullsized":"2023-10-16T11:01:25.014069+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:59.036068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013631+0000","last_change":"2023-10-16T11:01:23.613248+0000","last_active":"2023-10-16T11:01:25.013631+0000","last_peered":"2023-10-16T11:01:25.013631+0000","last_clean":"2023-10-16T11:01:25.013631+0000","last_became_active":"2023-10-16T11:01:23.612966+0000","last_became_peered":"2023-10-16T11:01:23.612966+0000","last_unstale":"2023-10-16T11:01:25.013631+0000","last_undegraded":"2023-10-16T11:01:25.013631+0000","last_fullsized":"2023-10-16T11:01:25.013631+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:06.144124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604633+0000","last_change":"2023-10-16T11:01:23.626806+0000","last_active":"2023-10-16T11:01:24.604633+0000","last_peered":"2023-10-16T11:01:24.604633+0000","last_clean":"2023-10-16T11:01:24.604633+0000","last_became_active":"2023-10-16T11:01:23.626523+0000","last_became_peered":"2023-10-16T11:01:23.626523+0000","last_unstale":"2023-10-16T11:01:24.604633+0000","last_undegraded":"2023-10-16T11:01:24.604633+0000","last_fullsized":"2023-10-16T11:01:24.604633+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:40.009403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013526+0000","last_change":"2023-10-16T11:01:23.611019+0000","last_active":"2023-10-16T11:01:25.013526+0000","last_peered":"2023-10-16T11:01:25.013526+0000","last_clean":"2023-10-16T11:01:25.013526+0000","last_became_active":"2023-10-16T11:01:23.610692+0000","last_became_peered":"2023-10-16T11:01:23.610692+0000","last_unstale":"2023-10-16T11:01:25.013526+0000","last_undegraded":"2023-10-16T11:01:25.013526+0000","last_fullsized":"2023-10-16T11:01:25.013526+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:28.380320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013832+0000","last_change":"2023-10-16T11:01:23.616848+0000","last_active":"2023-10-16T11:01:25.013832+0000","last_peered":"2023-10-16T11:01:25.013832+0000","last_clean":"2023-10-16T11:01:25.013832+0000","last_became_active":"2023-10-16T11:01:23.616586+0000","last_became_peered":"2023-10-16T11:01:23.616586+0000","last_unstale":"2023-10-16T11:01:25.013832+0000","last_undegraded":"2023-10-16T11:01:25.013832+0000","last_fullsized":"2023-10-16T11:01:25.013832+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:21.313008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603997+0000","last_change":"2023-10-16T11:01:23.617874+0000","last_active":"2023-10-16T11:01:24.603997+0000","last_peered":"2023-10-16T11:01:24.603997+0000","last_clean":"2023-10-16T11:01:24.603997+0000","last_became_active":"2023-10-16T11:01:23.617586+0000","last_became_peered":"2023-10-16T11:01:23.617586+0000","last_unstale":"2023-10-16T11:01:24.603997+0000","last_undegraded":"2023-10-16T11:01:24.603997+0000","last_fullsized":"2023-10-16T11:01:24.603997+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:21.313008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014111+0000","last_change":"2023-10-16T11:01:23.611029+0000","last_active":"2023-10-16T11:01:25.014111+0000","last_peered":"2023-10-16T11:01:25.014111+0000","last_clean":"2023-10-16T11:01:25.014111+0000","last_became_active":"2023-10-16T11:01:23.610655+0000","last_became_peered":"2023-10-16T11:01:23.610655+0000","last_unstale":"2023-10-16T11:01:25.014111+0000","last_undegraded":"2023-10-16T11:01:25.014111+0000","last_fullsized":"2023-10-16T11:01:25.014111+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:30.419217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014147+0000","last_change":"2023-10-16T11:01:23.613754+0000","last_active":"2023-10-16T11:01:25.014147+0000","last_peered":"2023-10-16T11:01:25.014147+0000","last_clean":"2023-10-16T11:01:25.014147+0000","last_became_active":"2023-10-16T11:01:23.612799+0000","last_became_peered":"2023-10-16T11:01:23.612799+0000","last_unstale":"2023-10-16T11:01:25.014147+0000","last_undegraded":"2023-10-16T11:01:25.014147+0000","last_fullsized":"2023-10-16T11:01:25.014147+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:00.693125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014086+0000","last_change":"2023-10-16T11:01:23.611460+0000","last_active":"2023-10-16T11:01:25.014086+0000","last_peered":"2023-10-16T11:01:25.014086+0000","last_clean":"2023-10-16T11:01:25.014086+0000","last_became_active":"2023-10-16T11:01:23.611184+0000","last_became_peered":"2023-10-16T11:01:23.611184+0000","last_unstale":"2023-10-16T11:01:25.014086+0000","last_undegraded":"2023-10-16T11:01:25.014086+0000","last_fullsized":"2023-10-16T11:01:25.014086+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:29.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013839+0000","last_change":"2023-10-16T11:01:23.614241+0000","last_active":"2023-10-16T11:01:25.013839+0000","last_peered":"2023-10-16T11:01:25.013839+0000","last_clean":"2023-10-16T11:01:25.013839+0000","last_became_active":"2023-10-16T11:01:23.614009+0000","last_became_peered":"2023-10-16T11:01:23.614009+0000","last_unstale":"2023-10-16T11:01:25.013839+0000","last_undegraded":"2023-10-16T11:01:25.013839+0000","last_fullsized":"2023-10-16T11:01:25.013839+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:07.785867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013987+0000","last_change":"2023-10-16T11:01:23.617503+0000","last_active":"2023-10-16T11:01:25.013987+0000","last_peered":"2023-10-16T11:01:25.013987+0000","last_clean":"2023-10-16T11:01:25.013987+0000","last_became_active":"2023-10-16T11:01:23.617240+0000","last_became_peered":"2023-10-16T11:01:23.617240+0000","last_unstale":"2023-10-16T11:01:25.013987+0000","last_undegraded":"2023-10-16T11:01:25.013987+0000","last_fullsized":"2023-10-16T11:01:25.013987+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:41.282912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013748+0000","last_change":"2023-10-16T11:01:23.612273+0000","last_active":"2023-10-16T11:01:25.013748+0000","last_peered":"2023-10-16T11:01:25.013748+0000","last_clean":"2023-10-16T11:01:25.013748+0000","last_became_active":"2023-10-16T11:01:23.611952+0000","last_became_peered":"2023-10-16T11:01:23.611952+0000","last_unstale":"2023-10-16T11:01:25.013748+0000","last_undegraded":"2023-10-16T11:01:25.013748+0000","last_fullsized":"2023-10-16T11:01:25.013748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:30.728808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604342+0000","last_change":"2023-10-16T11:01:23.622565+0000","last_active":"2023-10-16T11:01:24.604342+0000","last_peered":"2023-10-16T11:01:24.604342+0000","last_clean":"2023-10-16T11:01:24.604342+0000","last_became_active":"2023-10-16T11:01:23.622255+0000","last_became_peered":"2023-10-16T11:01:23.622255+0000","last_unstale":"2023-10-16T11:01:24.604342+0000","last_undegraded":"2023-10-16T11:01:24.604342+0000","last_fullsized":"2023-10-16T11:01:24.604342+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:02:46.611922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604139+0000","last_change":"2023-10-16T11:01:23.621490+0000","last_active":"2023-10-16T11:01:24.604139+0000","last_peered":"2023-10-16T11:01:24.604139+0000","last_clean":"2023-10-16T11:01:24.604139+0000","last_became_active":"2023-10-16T11:01:23.621081+0000","last_became_peered":"2023-10-16T11:01:23.621081+0000","last_unstale":"2023-10-16T11:01:24.604139+0000","last_undegraded":"2023-10-16T11:01:24.604139+0000","last_fullsized":"2023-10-16T11:01:24.604139+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:00.693125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013378+0000","last_change":"2023-10-16T11:01:23.611811+0000","last_active":"2023-10-16T11:01:25.013378+0000","last_peered":"2023-10-16T11:01:25.013378+0000","last_clean":"2023-10-16T11:01:25.013378+0000","last_became_active":"2023-10-16T11:01:23.611533+0000","last_became_peered":"2023-10-16T11:01:23.611533+0000","last_unstale":"2023-10-16T11:01:25.013378+0000","last_undegraded":"2023-10-16T11:01:25.013378+0000","last_fullsized":"2023-10-16T11:01:25.013378+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:15.289849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013051+0000","last_change":"2023-10-16T11:01:23.613126+0000","last_active":"2023-10-16T11:01:25.013051+0000","last_peered":"2023-10-16T11:01:25.013051+0000","last_clean":"2023-10-16T11:01:25.013051+0000","last_became_active":"2023-10-16T11:01:23.612874+0000","last_became_peered":"2023-10-16T11:01:23.612874+0000","last_unstale":"2023-10-16T11:01:25.013051+0000","last_undegraded":"2023-10-16T11:01:25.013051+0000","last_fullsized":"2023-10-16T11:01:25.013051+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:09.162169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604095+0000","last_change":"2023-10-16T11:01:23.624705+0000","last_active":"2023-10-16T11:01:24.604095+0000","last_peered":"2023-10-16T11:01:24.604095+0000","last_clean":"2023-10-16T11:01:24.604095+0000","last_became_active":"2023-10-16T11:01:23.624416+0000","last_became_peered":"2023-10-16T11:01:23.624416+0000","last_unstale":"2023-10-16T11:01:24.604095+0000","last_undegraded":"2023-10-16T11:01:24.604095+0000","last_fullsized":"2023-10-16T11:01:24.604095+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:29.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013272+0000","last_change":"2023-10-16T11:01:23.616538+0000","last_active":"2023-10-16T11:01:25.013272+0000","last_peered":"2023-10-16T11:01:25.013272+0000","last_clean":"2023-10-16T11:01:25.013272+0000","last_became_active":"2023-10-16T11:01:23.616028+0000","last_became_peered":"2023-10-16T11:01:23.616028+0000","last_unstale":"2023-10-16T11:01:25.013272+0000","last_undegraded":"2023-10-16T11:01:25.013272+0000","last_fullsized":"2023-10-16T11:01:25.013272+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:19.843632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604045+0000","last_change":"2023-10-16T11:01:23.619098+0000","last_active":"2023-10-16T11:01:24.604045+0000","last_peered":"2023-10-16T11:01:24.604045+0000","last_clean":"2023-10-16T11:01:24.604045+0000","last_became_active":"2023-10-16T11:01:23.618827+0000","last_became_peered":"2023-10-16T11:01:23.618827+0000","last_unstale":"2023-10-16T11:01:24.604045+0000","last_undegraded":"2023-10-16T11:01:24.604045+0000","last_fullsized":"2023-10-16T11:01:24.604045+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:07.785867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604461+0000","last_change":"2023-10-16T11:01:23.623468+0000","last_active":"2023-10-16T11:01:24.604461+0000","last_peered":"2023-10-16T11:01:24.604461+0000","last_clean":"2023-10-16T11:01:24.604461+0000","last_became_active":"2023-10-16T11:01:23.623193+0000","last_became_peered":"2023-10-16T11:01:23.623193+0000","last_unstale":"2023-10-16T11:01:24.604461+0000","last_undegraded":"2023-10-16T11:01:24.604461+0000","last_fullsized":"2023-10-16T11:01:24.604461+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:23.906440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604102+0000","last_change":"2023-10-16T11:01:23.624441+0000","last_active":"2023-10-16T11:01:24.604102+0000","last_peered":"2023-10-16T11:01:24.604102+0000","last_clean":"2023-10-16T11:01:24.604102+0000","last_became_active":"2023-10-16T11:01:23.624176+0000","last_became_peered":"2023-10-16T11:01:23.624176+0000","last_unstale":"2023-10-16T11:01:24.604102+0000","last_undegraded":"2023-10-16T11:01:24.604102+0000","last_fullsized":"2023-10-16T11:01:24.604102+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:00.693125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013566+0000","last_change":"2023-10-16T11:01:23.609492+0000","last_active":"2023-10-16T11:01:25.013566+0000","last_peered":"2023-10-16T11:01:25.013566+0000","last_clean":"2023-10-16T11:01:25.013566+0000","last_became_active":"2023-10-16T11:01:23.609115+0000","last_became_peered":"2023-10-16T11:01:23.609115+0000","last_unstale":"2023-10-16T11:01:25.013566+0000","last_undegraded":"2023-10-16T11:01:25.013566+0000","last_fullsized":"2023-10-16T11:01:25.013566+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:49.126756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603836+0000","last_change":"2023-10-16T11:01:23.619886+0000","last_active":"2023-10-16T11:01:24.603836+0000","last_peered":"2023-10-16T11:01:24.603836+0000","last_clean":"2023-10-16T11:01:24.603836+0000","last_became_active":"2023-10-16T11:01:23.619600+0000","last_became_peered":"2023-10-16T11:01:23.619600+0000","last_unstale":"2023-10-16T11:01:24.603836+0000","last_undegraded":"2023-10-16T11:01:24.603836+0000","last_fullsized":"2023-10-16T11:01:24.603836+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:48.469185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013432+0000","last_change":"2023-10-16T11:01:23.624584+0000","last_active":"2023-10-16T11:01:25.013432+0000","last_peered":"2023-10-16T11:01:25.013432+0000","last_clean":"2023-10-16T11:01:25.013432+0000","last_became_active":"2023-10-16T11:01:23.624285+0000","last_became_peered":"2023-10-16T11:01:23.624285+0000","last_unstale":"2023-10-16T11:01:25.013432+0000","last_undegraded":"2023-10-16T11:01:25.013432+0000","last_fullsized":"2023-10-16T11:01:25.013432+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:48.469185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604793+0000","last_change":"2023-10-16T11:01:23.621709+0000","last_active":"2023-10-16T11:01:24.604793+0000","last_peered":"2023-10-16T11:01:24.604793+0000","last_clean":"2023-10-16T11:01:24.604793+0000","last_became_active":"2023-10-16T11:01:23.621286+0000","last_became_peered":"2023-10-16T11:01:23.621286+0000","last_unstale":"2023-10-16T11:01:24.604793+0000","last_undegraded":"2023-10-16T11:01:24.604793+0000","last_fullsized":"2023-10-16T11:01:24.604793+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:01:02.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014504+0000","last_change":"2023-10-16T11:01:23.611732+0000","last_active":"2023-10-16T11:01:25.014504+0000","last_peered":"2023-10-16T11:01:25.014504+0000","last_clean":"2023-10-16T11:01:25.014504+0000","last_became_active":"2023-10-16T11:01:23.611462+0000","last_became_peered":"2023-10-16T11:01:23.611462+0000","last_unstale":"2023-10-16T11:01:25.014504+0000","last_undegraded":"2023-10-16T11:01:25.014504+0000","last_fullsized":"2023-10-16T11:01:25.014504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:32.890786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604726+0000","last_change":"2023-10-16T11:01:23.621946+0000","last_active":"2023-10-16T11:01:24.604726+0000","last_peered":"2023-10-16T11:01:24.604726+0000","last_clean":"2023-10-16T11:01:24.604726+0000","last_became_active":"2023-10-16T11:01:23.621675+0000","last_became_peered":"2023-10-16T11:01:23.621675+0000","last_unstale":"2023-10-16T11:01:24.604726+0000","last_undegraded":"2023-10-16T11:01:24.604726+0000","last_fullsized":"2023-10-16T11:01:24.604726+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:48.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013976+0000","last_change":"2023-10-16T11:01:23.616237+0000","last_active":"2023-10-16T11:01:25.013976+0000","last_peered":"2023-10-16T11:01:25.013976+0000","last_clean":"2023-10-16T11:01:25.013976+0000","last_became_active":"2023-10-16T11:01:23.615967+0000","last_became_peered":"2023-10-16T11:01:23.615967+0000","last_unstale":"2023-10-16T11:01:25.013976+0000","last_undegraded":"2023-10-16T11:01:25.013976+0000","last_fullsized":"2023-10-16T11:01:25.013976+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:14:25.171750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604516+0000","last_change":"2023-10-16T11:01:23.622442+0000","last_active":"2023-10-16T11:01:24.604516+0000","last_peered":"2023-10-16T11:01:24.604516+0000","last_clean":"2023-10-16T11:01:24.604516+0000","last_became_active":"2023-10-16T11:01:23.622103+0000","last_became_peered":"2023-10-16T11:01:23.622103+0000","last_unstale":"2023-10-16T11:01:24.604516+0000","last_undegraded":"2023-10-16T11:01:24.604516+0000","last_fullsized":"2023-10-16T11:01:24.604516+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:27.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605689+0000","last_change":"2023-10-16T11:01:23.615161+0000","last_active":"2023-10-16T11:01:24.605689+0000","last_peered":"2023-10-16T11:01:24.605689+0000","last_clean":"2023-10-16T11:01:24.605689+0000","last_became_active":"2023-10-16T11:01:23.614809+0000","last_became_peered":"2023-10-16T11:01:23.614809+0000","last_unstale":"2023-10-16T11:01:24.605689+0000","last_undegraded":"2023-10-16T11:01:24.605689+0000","last_fullsized":"2023-10-16T11:01:24.605689+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:26.688383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605488+0000","last_change":"2023-10-16T11:01:23.610446+0000","last_active":"2023-10-16T11:01:24.605488+0000","last_peered":"2023-10-16T11:01:24.605488+0000","last_clean":"2023-10-16T11:01:24.605488+0000","last_became_active":"2023-10-16T11:01:23.610175+0000","last_became_peered":"2023-10-16T11:01:23.610175+0000","last_unstale":"2023-10-16T11:01:24.605488+0000","last_undegraded":"2023-10-16T11:01:24.605488+0000","last_fullsized":"2023-10-16T11:01:24.605488+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:34.363924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013068+0000","last_change":"2023-10-16T11:01:23.618805+0000","last_active":"2023-10-16T11:01:25.013068+0000","last_peered":"2023-10-16T11:01:25.013068+0000","last_clean":"2023-10-16T11:01:25.013068+0000","last_became_active":"2023-10-16T11:01:23.615924+0000","last_became_peered":"2023-10-16T11:01:23.615924+0000","last_unstale":"2023-10-16T11:01:25.013068+0000","last_undegraded":"2023-10-16T11:01:25.013068+0000","last_fullsized":"2023-10-16T11:01:25.013068+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:22.370874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014629+0000","last_change":"2023-10-16T11:01:23.611656+0000","last_active":"2023-10-16T11:01:25.014629+0000","last_peered":"2023-10-16T11:01:25.014629+0000","last_clean":"2023-10-16T11:01:25.014629+0000","last_became_active":"2023-10-16T11:01:23.611399+0000","last_became_peered":"2023-10-16T11:01:23.611399+0000","last_unstale":"2023-10-16T11:01:25.014629+0000","last_undegraded":"2023-10-16T11:01:25.014629+0000","last_fullsized":"2023-10-16T11:01:25.014629+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:48.469185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"180'4","reported_seq":16,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013988+0000","last_change":"2023-10-16T11:01:22.601746+0000","last_active":"2023-10-16T11:01:25.013988+0000","last_peered":"2023-10-16T11:01:25.013988+0000","last_clean":"2023-10-16T11:01:25.013988+0000","last_became_active":"2023-10-16T11:01:22.601480+0000","last_became_peered":"2023-10-16T11:01:22.601480+0000","last_unstale":"2023-10-16T11:01:25.013988+0000","last_undegraded":"2023-10-16T11:01:25.013988+0000","last_fullsized":"2023-10-16T11:01:25.013988+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:39.616473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605990+0000","last_change":"2023-10-16T11:01:23.610653+0000","last_active":"2023-10-16T11:01:24.605990+0000","last_peered":"2023-10-16T11:01:24.605990+0000","last_clean":"2023-10-16T11:01:24.605990+0000","last_became_active":"2023-10-16T11:01:23.610381+0000","last_became_peered":"2023-10-16T11:01:23.610381+0000","last_unstale":"2023-10-16T11:01:24.605990+0000","last_undegraded":"2023-10-16T11:01:24.605990+0000","last_fullsized":"2023-10-16T11:01:24.605990+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:58.960773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605836+0000","last_change":"2023-10-16T11:01:22.602658+0000","last_active":"2023-10-16T11:01:24.605836+0000","last_peered":"2023-10-16T11:01:24.605836+0000","last_clean":"2023-10-16T11:01:24.605836+0000","last_became_active":"2023-10-16T11:01:22.601728+0000","last_became_peered":"2023-10-16T11:01:22.601728+0000","last_unstale":"2023-10-16T11:01:24.605836+0000","last_undegraded":"2023-10-16T11:01:24.605836+0000","last_fullsized":"2023-10-16T11:01:24.605836+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:06.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604089+0000","last_change":"2023-10-16T11:01:23.622047+0000","last_active":"2023-10-16T11:01:24.604089+0000","last_peered":"2023-10-16T11:01:24.604089+0000","last_clean":"2023-10-16T11:01:24.604089+0000","last_became_active":"2023-10-16T11:01:23.621766+0000","last_became_peered":"2023-10-16T11:01:23.621766+0000","last_unstale":"2023-10-16T11:01:24.604089+0000","last_undegraded":"2023-10-16T11:01:24.604089+0000","last_fullsized":"2023-10-16T11:01:24.604089+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:41.282912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013721+0000","last_change":"2023-10-16T11:01:22.604600+0000","last_active":"2023-10-16T11:01:25.013721+0000","last_peered":"2023-10-16T11:01:25.013721+0000","last_clean":"2023-10-16T11:01:25.013721+0000","last_became_active":"2023-10-16T11:01:22.604259+0000","last_became_peered":"2023-10-16T11:01:22.604259+0000","last_unstale":"2023-10-16T11:01:25.013721+0000","last_undegraded":"2023-10-16T11:01:25.013721+0000","last_fullsized":"2023-10-16T11:01:25.013721+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:31:48.077668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604819+0000","last_change":"2023-10-16T11:01:23.621641+0000","last_active":"2023-10-16T11:01:24.604819+0000","last_peered":"2023-10-16T11:01:24.604819+0000","last_clean":"2023-10-16T11:01:24.604819+0000","last_became_active":"2023-10-16T11:01:23.621326+0000","last_became_peered":"2023-10-16T11:01:23.621326+0000","last_unstale":"2023-10-16T11:01:24.604819+0000","last_undegraded":"2023-10-16T11:01:24.604819+0000","last_fullsized":"2023-10-16T11:01:24.604819+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:26.544254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014265+0000","last_change":"2023-10-16T11:01:22.603164+0000","last_active":"2023-10-16T11:01:25.014265+0000","last_peered":"2023-10-16T11:01:25.014265+0000","last_clean":"2023-10-16T11:01:25.014265+0000","last_became_active":"2023-10-16T11:01:22.602204+0000","last_became_peered":"2023-10-16T11:01:22.602204+0000","last_unstale":"2023-10-16T11:01:25.014265+0000","last_undegraded":"2023-10-16T11:01:25.014265+0000","last_fullsized":"2023-10-16T11:01:25.014265+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:24.172489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014242+0000","last_change":"2023-10-16T11:01:23.607207+0000","last_active":"2023-10-16T11:01:25.014242+0000","last_peered":"2023-10-16T11:01:25.014242+0000","last_clean":"2023-10-16T11:01:25.014242+0000","last_became_active":"2023-10-16T11:01:23.606174+0000","last_became_peered":"2023-10-16T11:01:23.606174+0000","last_unstale":"2023-10-16T11:01:25.014242+0000","last_undegraded":"2023-10-16T11:01:25.014242+0000","last_fullsized":"2023-10-16T11:01:25.014242+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:57.849844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604440+0000","last_change":"2023-10-16T11:01:22.602206+0000","last_active":"2023-10-16T11:01:24.604440+0000","last_peered":"2023-10-16T11:01:24.604440+0000","last_clean":"2023-10-16T11:01:24.604440+0000","last_became_active":"2023-10-16T11:01:22.601945+0000","last_became_peered":"2023-10-16T11:01:22.601945+0000","last_unstale":"2023-10-16T11:01:24.604440+0000","last_undegraded":"2023-10-16T11:01:24.604440+0000","last_fullsized":"2023-10-16T11:01:24.604440+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:20.305491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013706+0000","last_change":"2023-10-16T11:01:23.610061+0000","last_active":"2023-10-16T11:01:25.013706+0000","last_peered":"2023-10-16T11:01:25.013706+0000","last_clean":"2023-10-16T11:01:25.013706+0000","last_became_active":"2023-10-16T11:01:23.609706+0000","last_became_peered":"2023-10-16T11:01:23.609706+0000","last_unstale":"2023-10-16T11:01:25.013706+0000","last_undegraded":"2023-10-16T11:01:25.013706+0000","last_fullsized":"2023-10-16T11:01:25.013706+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:55:27.321104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013179+0000","last_change":"2023-10-16T11:01:22.606161+0000","last_active":"2023-10-16T11:01:25.013179+0000","last_peered":"2023-10-16T11:01:25.013179+0000","last_clean":"2023-10-16T11:01:25.013179+0000","last_became_active":"2023-10-16T11:01:22.605539+0000","last_became_peered":"2023-10-16T11:01:22.605539+0000","last_unstale":"2023-10-16T11:01:25.013179+0000","last_undegraded":"2023-10-16T11:01:25.013179+0000","last_fullsized":"2023-10-16T11:01:25.013179+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:45.097655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013464+0000","last_change":"2023-10-16T11:01:23.615935+0000","last_active":"2023-10-16T11:01:25.013464+0000","last_peered":"2023-10-16T11:01:25.013464+0000","last_clean":"2023-10-16T11:01:25.013464+0000","last_became_active":"2023-10-16T11:01:23.615412+0000","last_became_peered":"2023-10-16T11:01:23.615412+0000","last_unstale":"2023-10-16T11:01:25.013464+0000","last_undegraded":"2023-10-16T11:01:25.013464+0000","last_fullsized":"2023-10-16T11:01:25.013464+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:03.291177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013567+0000","last_change":"2023-10-16T11:01:22.605428+0000","last_active":"2023-10-16T11:01:25.013567+0000","last_peered":"2023-10-16T11:01:25.013567+0000","last_clean":"2023-10-16T11:01:25.013567+0000","last_became_active":"2023-10-16T11:01:22.605124+0000","last_became_peered":"2023-10-16T11:01:22.605124+0000","last_unstale":"2023-10-16T11:01:25.013567+0000","last_undegraded":"2023-10-16T11:01:25.013567+0000","last_fullsized":"2023-10-16T11:01:25.013567+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:56.842327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603834+0000","last_change":"2023-10-16T11:01:23.617853+0000","last_active":"2023-10-16T11:01:24.603834+0000","last_peered":"2023-10-16T11:01:24.603834+0000","last_clean":"2023-10-16T11:01:24.603834+0000","last_became_active":"2023-10-16T11:01:23.617569+0000","last_became_peered":"2023-10-16T11:01:23.617569+0000","last_unstale":"2023-10-16T11:01:24.603834+0000","last_undegraded":"2023-10-16T11:01:24.603834+0000","last_fullsized":"2023-10-16T11:01:24.603834+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:49.126756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013737+0000","last_change":"2023-10-16T11:01:22.604779+0000","last_active":"2023-10-16T11:01:25.013737+0000","last_peered":"2023-10-16T11:01:25.013737+0000","last_clean":"2023-10-16T11:01:25.013737+0000","last_became_active":"2023-10-16T11:01:22.604526+0000","last_became_peered":"2023-10-16T11:01:22.604526+0000","last_unstale":"2023-10-16T11:01:25.013737+0000","last_undegraded":"2023-10-16T11:01:25.013737+0000","last_fullsized":"2023-10-16T11:01:25.013737+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:14.971959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014178+0000","last_change":"2023-10-16T11:01:23.607120+0000","last_active":"2023-10-16T11:01:25.014178+0000","last_peered":"2023-10-16T11:01:25.014178+0000","last_clean":"2023-10-16T11:01:25.014178+0000","last_became_active":"2023-10-16T11:01:23.606839+0000","last_became_peered":"2023-10-16T11:01:23.606839+0000","last_unstale":"2023-10-16T11:01:25.014178+0000","last_undegraded":"2023-10-16T11:01:25.014178+0000","last_fullsized":"2023-10-16T11:01:25.014178+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:40.623989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604021+0000","last_change":"2023-10-16T11:01:22.599810+0000","last_active":"2023-10-16T11:01:24.604021+0000","last_peered":"2023-10-16T11:01:24.604021+0000","last_clean":"2023-10-16T11:01:24.604021+0000","last_became_active":"2023-10-16T11:01:22.599552+0000","last_became_peered":"2023-10-16T11:01:22.599552+0000","last_unstale":"2023-10-16T11:01:24.604021+0000","last_undegraded":"2023-10-16T11:01:24.604021+0000","last_fullsized":"2023-10-16T11:01:24.604021+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:23.085725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013347+0000","last_change":"2023-10-16T11:01:23.609616+0000","last_active":"2023-10-16T11:01:25.013347+0000","last_peered":"2023-10-16T11:01:25.013347+0000","last_clean":"2023-10-16T11:01:25.013347+0000","last_became_active":"2023-10-16T11:01:23.609357+0000","last_became_peered":"2023-10-16T11:01:23.609357+0000","last_unstale":"2023-10-16T11:01:25.013347+0000","last_undegraded":"2023-10-16T11:01:25.013347+0000","last_fullsized":"2023-10-16T11:01:25.013347+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:18.354218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"180'3","reported_seq":15,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604197+0000","last_change":"2023-10-16T11:01:22.603723+0000","last_active":"2023-10-16T11:01:24.604197+0000","last_peered":"2023-10-16T11:01:24.604197+0000","last_clean":"2023-10-16T11:01:24.604197+0000","last_became_active":"2023-10-16T11:01:22.599707+0000","last_became_peered":"2023-10-16T11:01:22.599707+0000","last_unstale":"2023-10-16T11:01:24.604197+0000","last_undegraded":"2023-10-16T11:01:24.604197+0000","last_fullsized":"2023-10-16T11:01:24.604197+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:31.883270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605975+0000","last_change":"2023-10-16T11:01:23.628153+0000","last_active":"2023-10-16T11:01:24.605975+0000","last_peered":"2023-10-16T11:01:24.605975+0000","last_clean":"2023-10-16T11:01:24.605975+0000","last_became_active":"2023-10-16T11:01:23.627737+0000","last_became_peered":"2023-10-16T11:01:23.627737+0000","last_unstale":"2023-10-16T11:01:24.605975+0000","last_undegraded":"2023-10-16T11:01:24.605975+0000","last_fullsized":"2023-10-16T11:01:24.605975+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:20.110705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"180'6","reported_seq":18,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605840+0000","last_change":"2023-10-16T11:01:22.599787+0000","last_active":"2023-10-16T11:01:24.605840+0000","last_peered":"2023-10-16T11:01:24.605840+0000","last_clean":"2023-10-16T11:01:24.605840+0000","last_became_active":"2023-10-16T11:01:22.599464+0000","last_became_peered":"2023-10-16T11:01:22.599464+0000","last_unstale":"2023-10-16T11:01:24.605840+0000","last_undegraded":"2023-10-16T11:01:24.605840+0000","last_fullsized":"2023-10-16T11:01:24.605840+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:54.551737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014915+0000","last_change":"2023-10-16T11:01:23.610843+0000","last_active":"2023-10-16T11:01:25.014915+0000","last_peered":"2023-10-16T11:01:25.014915+0000","last_clean":"2023-10-16T11:01:25.014915+0000","last_became_active":"2023-10-16T11:01:23.610402+0000","last_became_peered":"2023-10-16T11:01:23.610402+0000","last_unstale":"2023-10-16T11:01:25.014915+0000","last_undegraded":"2023-10-16T11:01:25.014915+0000","last_fullsized":"2023-10-16T11:01:25.014915+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:01.858571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013696+0000","last_change":"2023-10-16T11:01:22.604825+0000","last_active":"2023-10-16T11:01:25.013696+0000","last_peered":"2023-10-16T11:01:25.013696+0000","last_clean":"2023-10-16T11:01:25.013696+0000","last_became_active":"2023-10-16T11:01:22.604420+0000","last_became_peered":"2023-10-16T11:01:22.604420+0000","last_unstale":"2023-10-16T11:01:25.013696+0000","last_undegraded":"2023-10-16T11:01:25.013696+0000","last_fullsized":"2023-10-16T11:01:25.013696+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:13:55.406467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014347+0000","last_change":"2023-10-16T11:01:23.626921+0000","last_active":"2023-10-16T11:01:25.014347+0000","last_peered":"2023-10-16T11:01:25.014347+0000","last_clean":"2023-10-16T11:01:25.014347+0000","last_became_active":"2023-10-16T11:01:23.626600+0000","last_became_peered":"2023-10-16T11:01:23.626600+0000","last_unstale":"2023-10-16T11:01:25.014347+0000","last_undegraded":"2023-10-16T11:01:25.014347+0000","last_fullsized":"2023-10-16T11:01:25.014347+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:01.374062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013439+0000","last_change":"2023-10-16T11:01:22.592799+0000","last_active":"2023-10-16T11:01:25.013439+0000","last_peered":"2023-10-16T11:01:25.013439+0000","last_clean":"2023-10-16T11:01:25.013439+0000","last_became_active":"2023-10-16T11:01:22.592536+0000","last_became_peered":"2023-10-16T11:01:22.592536+0000","last_unstale":"2023-10-16T11:01:25.013439+0000","last_undegraded":"2023-10-16T11:01:25.013439+0000","last_fullsized":"2023-10-16T11:01:25.013439+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:22.757609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013628+0000","last_change":"2023-10-16T11:01:23.624279+0000","last_active":"2023-10-16T11:01:25.013628+0000","last_peered":"2023-10-16T11:01:25.013628+0000","last_clean":"2023-10-16T11:01:25.013628+0000","last_became_active":"2023-10-16T11:01:23.624016+0000","last_became_peered":"2023-10-16T11:01:23.624016+0000","last_unstale":"2023-10-16T11:01:25.013628+0000","last_undegraded":"2023-10-16T11:01:25.013628+0000","last_fullsized":"2023-10-16T11:01:25.013628+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:23.847285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013401+0000","last_change":"2023-10-16T11:01:22.605602+0000","last_active":"2023-10-16T11:01:25.013401+0000","last_peered":"2023-10-16T11:01:25.013401+0000","last_clean":"2023-10-16T11:01:25.013401+0000","last_became_active":"2023-10-16T11:01:22.605319+0000","last_became_peered":"2023-10-16T11:01:22.605319+0000","last_unstale":"2023-10-16T11:01:25.013401+0000","last_undegraded":"2023-10-16T11:01:25.013401+0000","last_fullsized":"2023-10-16T11:01:25.013401+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:11.968377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604138+0000","last_change":"2023-10-16T11:01:23.626390+0000","last_active":"2023-10-16T11:01:24.604138+0000","last_peered":"2023-10-16T11:01:24.604138+0000","last_clean":"2023-10-16T11:01:24.604138+0000","last_became_active":"2023-10-16T11:01:23.626105+0000","last_became_peered":"2023-10-16T11:01:23.626105+0000","last_unstale":"2023-10-16T11:01:24.604138+0000","last_undegraded":"2023-10-16T11:01:24.604138+0000","last_fullsized":"2023-10-16T11:01:24.604138+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:06.459351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"180'8","reported_seq":20,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605475+0000","last_change":"2023-10-16T11:01:22.600693+0000","last_active":"2023-10-16T11:01:24.605475+0000","last_peered":"2023-10-16T11:01:24.605475+0000","last_clean":"2023-10-16T11:01:24.605475+0000","last_became_active":"2023-10-16T11:01:22.600406+0000","last_became_peered":"2023-10-16T11:01:22.600406+0000","last_unstale":"2023-10-16T11:01:24.605475+0000","last_undegraded":"2023-10-16T11:01:24.605475+0000","last_fullsized":"2023-10-16T11:01:24.605475+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:26.525834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013948+0000","last_change":"2023-10-16T11:01:23.622416+0000","last_active":"2023-10-16T11:01:25.013948+0000","last_peered":"2023-10-16T11:01:25.013948+0000","last_clean":"2023-10-16T11:01:25.013948+0000","last_became_active":"2023-10-16T11:01:23.622062+0000","last_became_peered":"2023-10-16T11:01:23.622062+0000","last_unstale":"2023-10-16T11:01:25.013948+0000","last_undegraded":"2023-10-16T11:01:25.013948+0000","last_fullsized":"2023-10-16T11:01:25.013948+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:33.243686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604836+0000","last_change":"2023-10-16T11:01:22.606146+0000","last_active":"2023-10-16T11:01:24.604836+0000","last_peered":"2023-10-16T11:01:24.604836+0000","last_clean":"2023-10-16T11:01:24.604836+0000","last_became_active":"2023-10-16T11:01:22.605818+0000","last_became_peered":"2023-10-16T11:01:22.605818+0000","last_unstale":"2023-10-16T11:01:24.604836+0000","last_undegraded":"2023-10-16T11:01:24.604836+0000","last_fullsized":"2023-10-16T11:01:24.604836+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:40.275395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013577+0000","last_change":"2023-10-16T11:01:23.616809+0000","last_active":"2023-10-16T11:01:25.013577+0000","last_peered":"2023-10-16T11:01:25.013577+0000","last_clean":"2023-10-16T11:01:25.013577+0000","last_became_active":"2023-10-16T11:01:23.616541+0000","last_became_peered":"2023-10-16T11:01:23.616541+0000","last_unstale":"2023-10-16T11:01:25.013577+0000","last_undegraded":"2023-10-16T11:01:25.013577+0000","last_fullsized":"2023-10-16T11:01:25.013577+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:21:08.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604479+0000","last_change":"2023-10-16T11:01:22.595216+0000","last_active":"2023-10-16T11:01:24.604479+0000","last_peered":"2023-10-16T11:01:24.604479+0000","last_clean":"2023-10-16T11:01:24.604479+0000","last_became_active":"2023-10-16T11:01:22.594916+0000","last_became_peered":"2023-10-16T11:01:22.594916+0000","last_unstale":"2023-10-16T11:01:24.604479+0000","last_undegraded":"2023-10-16T11:01:24.604479+0000","last_fullsized":"2023-10-16T11:01:24.604479+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:05.136608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014107+0000","last_change":"2023-10-16T11:01:23.608430+0000","last_active":"2023-10-16T11:01:25.014107+0000","last_peered":"2023-10-16T11:01:25.014107+0000","last_clean":"2023-10-16T11:01:25.014107+0000","last_became_active":"2023-10-16T11:01:23.608128+0000","last_became_peered":"2023-10-16T11:01:23.608128+0000","last_unstale":"2023-10-16T11:01:25.014107+0000","last_undegraded":"2023-10-16T11:01:25.014107+0000","last_fullsized":"2023-10-16T11:01:25.014107+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:44.748533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013777+0000","last_change":"2023-10-16T11:01:22.600535+0000","last_active":"2023-10-16T11:01:25.013777+0000","last_peered":"2023-10-16T11:01:25.013777+0000","last_clean":"2023-10-16T11:01:25.013777+0000","last_became_active":"2023-10-16T11:01:22.600270+0000","last_became_peered":"2023-10-16T11:01:22.600270+0000","last_unstale":"2023-10-16T11:01:25.013777+0000","last_undegraded":"2023-10-16T11:01:25.013777+0000","last_fullsized":"2023-10-16T11:01:25.013777+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:21:07.243413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013936+0000","last_change":"2023-10-16T11:01:23.611319+0000","last_active":"2023-10-16T11:01:25.013936+0000","last_peered":"2023-10-16T11:01:25.013936+0000","last_clean":"2023-10-16T11:01:25.013936+0000","last_became_active":"2023-10-16T11:01:23.610879+0000","last_became_peered":"2023-10-16T11:01:23.610879+0000","last_unstale":"2023-10-16T11:01:25.013936+0000","last_undegraded":"2023-10-16T11:01:25.013936+0000","last_fullsized":"2023-10-16T11:01:25.013936+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:41.672049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014161+0000","last_change":"2023-10-16T11:01:22.607281+0000","last_active":"2023-10-16T11:01:25.014161+0000","last_peered":"2023-10-16T11:01:25.014161+0000","last_clean":"2023-10-16T11:01:25.014161+0000","last_became_active":"2023-10-16T11:01:22.606936+0000","last_became_peered":"2023-10-16T11:01:22.606936+0000","last_unstale":"2023-10-16T11:01:25.014161+0000","last_undegraded":"2023-10-16T11:01:25.014161+0000","last_fullsized":"2023-10-16T11:01:25.014161+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:06.782631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604214+0000","last_change":"2023-10-16T11:01:23.626330+0000","last_active":"2023-10-16T11:01:24.604214+0000","last_peered":"2023-10-16T11:01:24.604214+0000","last_clean":"2023-10-16T11:01:24.604214+0000","last_became_active":"2023-10-16T11:01:23.626057+0000","last_became_peered":"2023-10-16T11:01:23.626057+0000","last_unstale":"2023-10-16T11:01:24.604214+0000","last_undegraded":"2023-10-16T11:01:24.604214+0000","last_fullsized":"2023-10-16T11:01:24.604214+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:56:57.344075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013783+0000","last_change":"2023-10-16T11:01:22.603347+0000","last_active":"2023-10-16T11:01:25.013783+0000","last_peered":"2023-10-16T11:01:25.013783+0000","last_clean":"2023-10-16T11:01:25.013783+0000","last_became_active":"2023-10-16T11:01:22.603064+0000","last_became_peered":"2023-10-16T11:01:22.603064+0000","last_unstale":"2023-10-16T11:01:25.013783+0000","last_undegraded":"2023-10-16T11:01:25.013783+0000","last_fullsized":"2023-10-16T11:01:25.013783+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:50:41.596545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014441+0000","last_change":"2023-10-16T11:01:23.612337+0000","last_active":"2023-10-16T11:01:25.014441+0000","last_peered":"2023-10-16T11:01:25.014441+0000","last_clean":"2023-10-16T11:01:25.014441+0000","last_became_active":"2023-10-16T11:01:23.612071+0000","last_became_peered":"2023-10-16T11:01:23.612071+0000","last_unstale":"2023-10-16T11:01:25.014441+0000","last_undegraded":"2023-10-16T11:01:25.014441+0000","last_fullsized":"2023-10-16T11:01:25.014441+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:24.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014062+0000","last_change":"2023-10-16T11:01:22.600363+0000","last_active":"2023-10-16T11:01:25.014062+0000","last_peered":"2023-10-16T11:01:25.014062+0000","last_clean":"2023-10-16T11:01:25.014062+0000","last_became_active":"2023-10-16T11:01:22.600048+0000","last_became_peered":"2023-10-16T11:01:22.600048+0000","last_unstale":"2023-10-16T11:01:25.014062+0000","last_undegraded":"2023-10-16T11:01:25.014062+0000","last_fullsized":"2023-10-16T11:01:25.014062+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:17:33.657858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013504+0000","last_change":"2023-10-16T11:01:23.616293+0000","last_active":"2023-10-16T11:01:25.013504+0000","last_peered":"2023-10-16T11:01:25.013504+0000","last_clean":"2023-10-16T11:01:25.013504+0000","last_became_active":"2023-10-16T11:01:23.616040+0000","last_became_peered":"2023-10-16T11:01:23.616040+0000","last_unstale":"2023-10-16T11:01:25.013504+0000","last_undegraded":"2023-10-16T11:01:25.013504+0000","last_fullsized":"2023-10-16T11:01:25.013504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:21.969423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013551+0000","last_change":"2023-10-16T11:01:22.601051+0000","last_active":"2023-10-16T11:01:25.013551+0000","last_peered":"2023-10-16T11:01:25.013551+0000","last_clean":"2023-10-16T11:01:25.013551+0000","last_became_active":"2023-10-16T11:01:22.600514+0000","last_became_peered":"2023-10-16T11:01:22.600514+0000","last_unstale":"2023-10-16T11:01:25.013551+0000","last_undegraded":"2023-10-16T11:01:25.013551+0000","last_fullsized":"2023-10-16T11:01:25.013551+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:16.320730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603971+0000","last_change":"2023-10-16T11:01:23.619566+0000","last_active":"2023-10-16T11:01:24.603971+0000","last_peered":"2023-10-16T11:01:24.603971+0000","last_clean":"2023-10-16T11:01:24.603971+0000","last_became_active":"2023-10-16T11:01:23.619255+0000","last_became_peered":"2023-10-16T11:01:23.619255+0000","last_unstale":"2023-10-16T11:01:24.603971+0000","last_undegraded":"2023-10-16T11:01:24.603971+0000","last_fullsized":"2023-10-16T11:01:24.603971+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:30.728808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013604+0000","last_change":"2023-10-16T11:01:22.602453+0000","last_active":"2023-10-16T11:01:25.013604+0000","last_peered":"2023-10-16T11:01:25.013604+0000","last_clean":"2023-10-16T11:01:25.013604+0000","last_became_active":"2023-10-16T11:01:22.602148+0000","last_became_peered":"2023-10-16T11:01:22.602148+0000","last_unstale":"2023-10-16T11:01:25.013604+0000","last_undegraded":"2023-10-16T11:01:25.013604+0000","last_fullsized":"2023-10-16T11:01:25.013604+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:00.851055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013482+0000","last_change":"2023-10-16T11:01:23.617273+0000","last_active":"2023-10-16T11:01:25.013482+0000","last_peered":"2023-10-16T11:01:25.013482+0000","last_clean":"2023-10-16T11:01:25.013482+0000","last_became_active":"2023-10-16T11:01:23.616391+0000","last_became_peered":"2023-10-16T11:01:23.616391+0000","last_unstale":"2023-10-16T11:01:25.013482+0000","last_undegraded":"2023-10-16T11:01:25.013482+0000","last_fullsized":"2023-10-16T11:01:25.013482+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:27.879098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013670+0000","last_change":"2023-10-16T11:01:22.605983+0000","last_active":"2023-10-16T11:01:25.013670+0000","last_peered":"2023-10-16T11:01:25.013670+0000","last_clean":"2023-10-16T11:01:25.013670+0000","last_became_active":"2023-10-16T11:01:22.605712+0000","last_became_peered":"2023-10-16T11:01:22.605712+0000","last_unstale":"2023-10-16T11:01:25.013670+0000","last_undegraded":"2023-10-16T11:01:25.013670+0000","last_fullsized":"2023-10-16T11:01:25.013670+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:36:02.895039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014099+0000","last_change":"2023-10-16T11:01:23.616247+0000","last_active":"2023-10-16T11:01:25.014099+0000","last_peered":"2023-10-16T11:01:25.014099+0000","last_clean":"2023-10-16T11:01:25.014099+0000","last_became_active":"2023-10-16T11:01:23.615981+0000","last_became_peered":"2023-10-16T11:01:23.615981+0000","last_unstale":"2023-10-16T11:01:25.014099+0000","last_undegraded":"2023-10-16T11:01:25.014099+0000","last_fullsized":"2023-10-16T11:01:25.014099+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:03:13.273710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604511+0000","last_change":"2023-10-16T11:01:22.600488+0000","last_active":"2023-10-16T11:01:24.604511+0000","last_peered":"2023-10-16T11:01:24.604511+0000","last_clean":"2023-10-16T11:01:24.604511+0000","last_became_active":"2023-10-16T11:01:22.600217+0000","last_became_peered":"2023-10-16T11:01:22.600217+0000","last_unstale":"2023-10-16T11:01:24.604511+0000","last_undegraded":"2023-10-16T11:01:24.604511+0000","last_fullsized":"2023-10-16T11:01:24.604511+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:29.721291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603932+0000","last_change":"2023-10-16T11:01:23.621720+0000","last_active":"2023-10-16T11:01:24.603932+0000","last_peered":"2023-10-16T11:01:24.603932+0000","last_clean":"2023-10-16T11:01:24.603932+0000","last_became_active":"2023-10-16T11:01:23.621318+0000","last_became_peered":"2023-10-16T11:01:23.621318+0000","last_unstale":"2023-10-16T11:01:24.603932+0000","last_undegraded":"2023-10-16T11:01:24.603932+0000","last_fullsized":"2023-10-16T11:01:24.603932+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:06.144124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013984+0000","last_change":"2023-10-16T11:01:22.601660+0000","last_active":"2023-10-16T11:01:25.013984+0000","last_peered":"2023-10-16T11:01:25.013984+0000","last_clean":"2023-10-16T11:01:25.013984+0000","last_became_active":"2023-10-16T11:01:22.601392+0000","last_became_peered":"2023-10-16T11:01:22.601392+0000","last_unstale":"2023-10-16T11:01:25.013984+0000","last_undegraded":"2023-10-16T11:01:25.013984+0000","last_fullsized":"2023-10-16T11:01:25.013984+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:23.085725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013650+0000","last_change":"2023-10-16T11:01:23.615073+0000","last_active":"2023-10-16T11:01:25.013650+0000","last_peered":"2023-10-16T11:01:25.013650+0000","last_clean":"2023-10-16T11:01:25.013650+0000","last_became_active":"2023-10-16T11:01:23.614778+0000","last_became_peered":"2023-10-16T11:01:23.614778+0000","last_unstale":"2023-10-16T11:01:25.013650+0000","last_undegraded":"2023-10-16T11:01:25.013650+0000","last_fullsized":"2023-10-16T11:01:25.013650+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:07.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014752+0000","last_change":"2023-10-16T11:01:22.595898+0000","last_active":"2023-10-16T11:01:25.014752+0000","last_peered":"2023-10-16T11:01:25.014752+0000","last_clean":"2023-10-16T11:01:25.014752+0000","last_became_active":"2023-10-16T11:01:22.595644+0000","last_became_peered":"2023-10-16T11:01:22.595644+0000","last_unstale":"2023-10-16T11:01:25.014752+0000","last_undegraded":"2023-10-16T11:01:25.014752+0000","last_fullsized":"2023-10-16T11:01:25.014752+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:03.815129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013351+0000","last_change":"2023-10-16T11:01:23.616884+0000","last_active":"2023-10-16T11:01:25.013351+0000","last_peered":"2023-10-16T11:01:25.013351+0000","last_clean":"2023-10-16T11:01:25.013351+0000","last_became_active":"2023-10-16T11:01:23.616513+0000","last_became_peered":"2023-10-16T11:01:23.616513+0000","last_unstale":"2023-10-16T11:01:25.013351+0000","last_undegraded":"2023-10-16T11:01:25.013351+0000","last_fullsized":"2023-10-16T11:01:25.013351+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:17:34.665375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605708+0000","last_change":"2023-10-16T11:01:22.599754+0000","last_active":"2023-10-16T11:01:24.605708+0000","last_peered":"2023-10-16T11:01:24.605708+0000","last_clean":"2023-10-16T11:01:24.605708+0000","last_became_active":"2023-10-16T11:01:22.599412+0000","last_became_peered":"2023-10-16T11:01:22.599412+0000","last_unstale":"2023-10-16T11:01:24.605708+0000","last_undegraded":"2023-10-16T11:01:24.605708+0000","last_fullsized":"2023-10-16T11:01:24.605708+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:45.074422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605541+0000","last_change":"2023-10-16T11:01:23.610939+0000","last_active":"2023-10-16T11:01:24.605541+0000","last_peered":"2023-10-16T11:01:24.605541+0000","last_clean":"2023-10-16T11:01:24.605541+0000","last_became_active":"2023-10-16T11:01:23.610248+0000","last_became_peered":"2023-10-16T11:01:23.610248+0000","last_unstale":"2023-10-16T11:01:24.605541+0000","last_undegraded":"2023-10-16T11:01:24.605541+0000","last_fullsized":"2023-10-16T11:01:24.605541+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:44:29.210679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013273+0000","last_change":"2023-10-16T11:01:22.606342+0000","last_active":"2023-10-16T11:01:25.013273+0000","last_peered":"2023-10-16T11:01:25.013273+0000","last_clean":"2023-10-16T11:01:25.013273+0000","last_became_active":"2023-10-16T11:01:22.605885+0000","last_became_peered":"2023-10-16T11:01:22.605885+0000","last_unstale":"2023-10-16T11:01:25.013273+0000","last_undegraded":"2023-10-16T11:01:25.013273+0000","last_fullsized":"2023-10-16T11:01:25.013273+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:50.760465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013096+0000","last_change":"2023-10-16T11:01:23.613625+0000","last_active":"2023-10-16T11:01:25.013096+0000","last_peered":"2023-10-16T11:01:25.013096+0000","last_clean":"2023-10-16T11:01:25.013096+0000","last_became_active":"2023-10-16T11:01:23.612643+0000","last_became_peered":"2023-10-16T11:01:23.612643+0000","last_unstale":"2023-10-16T11:01:25.013096+0000","last_undegraded":"2023-10-16T11:01:25.013096+0000","last_fullsized":"2023-10-16T11:01:25.013096+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:55.193152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013765+0000","last_change":"2023-10-16T11:01:22.605395+0000","last_active":"2023-10-16T11:01:25.013765+0000","last_peered":"2023-10-16T11:01:25.013765+0000","last_clean":"2023-10-16T11:01:25.013765+0000","last_became_active":"2023-10-16T11:01:22.605107+0000","last_became_peered":"2023-10-16T11:01:22.605107+0000","last_unstale":"2023-10-16T11:01:25.013765+0000","last_undegraded":"2023-10-16T11:01:25.013765+0000","last_fullsized":"2023-10-16T11:01:25.013765+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:00.366546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604150+0000","last_change":"2023-10-16T11:01:23.623068+0000","last_active":"2023-10-16T11:01:24.604150+0000","last_peered":"2023-10-16T11:01:24.604150+0000","last_clean":"2023-10-16T11:01:24.604150+0000","last_became_active":"2023-10-16T11:01:23.622806+0000","last_became_peered":"2023-10-16T11:01:23.622806+0000","last_unstale":"2023-10-16T11:01:24.604150+0000","last_undegraded":"2023-10-16T11:01:24.604150+0000","last_fullsized":"2023-10-16T11:01:24.604150+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:29.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605614+0000","last_change":"2023-10-16T11:01:22.602845+0000","last_active":"2023-10-16T11:01:24.605614+0000","last_peered":"2023-10-16T11:01:24.605614+0000","last_clean":"2023-10-16T11:01:24.605614+0000","last_became_active":"2023-10-16T11:01:22.602558+0000","last_became_peered":"2023-10-16T11:01:22.602558+0000","last_unstale":"2023-10-16T11:01:24.605614+0000","last_undegraded":"2023-10-16T11:01:24.605614+0000","last_fullsized":"2023-10-16T11:01:24.605614+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:23.573151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013423+0000","last_change":"2023-10-16T11:01:23.617290+0000","last_active":"2023-10-16T11:01:25.013423+0000","last_peered":"2023-10-16T11:01:25.013423+0000","last_clean":"2023-10-16T11:01:25.013423+0000","last_became_active":"2023-10-16T11:01:23.616817+0000","last_became_peered":"2023-10-16T11:01:23.616817+0000","last_unstale":"2023-10-16T11:01:25.013423+0000","last_undegraded":"2023-10-16T11:01:25.013423+0000","last_fullsized":"2023-10-16T11:01:25.013423+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:05.119821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605998+0000","last_change":"2023-10-16T11:01:22.603190+0000","last_active":"2023-10-16T11:01:24.605998+0000","last_peered":"2023-10-16T11:01:24.605998+0000","last_clean":"2023-10-16T11:01:24.605998+0000","last_became_active":"2023-10-16T11:01:22.602894+0000","last_became_peered":"2023-10-16T11:01:22.602894+0000","last_unstale":"2023-10-16T11:01:24.605998+0000","last_undegraded":"2023-10-16T11:01:24.605998+0000","last_fullsized":"2023-10-16T11:01:24.605998+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:00.055026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"171'293","reported_seq":973,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-16T11:01:23.608804+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:01:23.608804+0000","last_peered":"2023-10-16T11:01:23.608804+0000","last_clean":"2023-10-16T11:01:23.608804+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:01:23.608804+0000","last_undegraded":"2023-10-16T11:01:23.608804+0000","last_fullsized":"2023-10-16T11:01:23.608804+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543067,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1729064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76700000000000002,"5min":0.78600000000000003,"15min":0.83499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.754,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.89200000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.81100000000000005,"15min":0.88100000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69799999999999995,"5min":0.81399999999999995,"15min":0.83899999999999997},"last":0.62}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.85199999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.86699999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.876},"last":0.38400000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.308},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.85999999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.89300000000000002},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.92400000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48199999999999998}]}]},{"osd":6,"up_from":14,"seq":60129543066,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37540,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334300,"statfs":{"total":96636764160,"available":96598323200,"internally_reserved":0,"allocated":2252800,"data_stored":1136364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.32600000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.495},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.60599999999999998,"15min":0.624},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.378},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.91100000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.432}]},{"osd":5,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.33700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.90200000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52700000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.36699999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38,"15min":0.34599999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.60499999999999998}]}]},{"osd":1,"up_from":14,"seq":60129543066,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2265088,"data_stored":1143172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.39400000000000002,"15min":0.39600000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.221},{"interface":"front","average":{"1min":0.374,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.44,"5min":0.57299999999999995,"15min":0.68000000000000005},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46000000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.308,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59499999999999997,"5min":0.68899999999999995,"15min":0.745},"last":0.34999999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.55400000000000005,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45400000000000001,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.73399999999999999},"last":0.32100000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.622,"15min":0.64500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33000000000000002},"max":{"1min":0.78900000000000003,"5min":1.004,"15min":1.044},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.94399999999999995,"15min":1.0409999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84999999999999998,"5min":0.97599999999999998,"15min":1.038},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":1.038,"15min":1.038},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68899999999999995},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78800000000000003,"5min":1.0169999999999999,"15min":1.105},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66700000000000004,"15min":0.68000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.41399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94699999999999995,"15min":1.091},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.81100000000000005,"5min":1.0389999999999999,"15min":1.1299999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83299999999999996,"5min":1.034,"15min":1.034},"last":0.63100000000000001}]}]},{"osd":0,"up_from":14,"seq":60129543067,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56180,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315660,"statfs":{"total":96636764160,"available":96579235840,"internally_reserved":0,"allocated":2854912,"data_stored":1733506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.22900000000000001},"max":{"1min":0.54500000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.218},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.69199999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.64200000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.33400000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.72599999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66000000000000003,"5min":0.71599999999999997,"15min":0.74099999999999999},"last":0.438}]},{"osd":4,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.35899999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.39700000000000002},"max":{"1min":0.79000000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.999,"15min":1.0820000000000001},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.67000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.48399999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.98699999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.496}]}]},{"osd":2,"up_from":14,"seq":60129543066,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59764,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312076,"statfs":{"total":96636764160,"available":96575565824,"internally_reserved":0,"allocated":2265088,"data_stored":1143228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.39700000000000002,"15min":0.38500000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.78900000000000003},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46100000000000002,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.26200000000000001},"max":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.67900000000000005},"last":0.441}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.624,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.502,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.751},"last":0.49099999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.97999999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.42599999999999999},"max":{"1min":0.88300000000000001,"5min":0.98699999999999999,"15min":1.022},"last":0.68500000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86199999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.749},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68400000000000005,"15min":0.66900000000000004},"min":{"1min":0.45700000000000002,"5min":0.40999999999999998,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.64400000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68799999999999994,"15min":0.67000000000000004},"min":{"1min":0.47299999999999998,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89800000000000002,"5min":1.159,"15min":1.159},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68999999999999995,"15min":0.67900000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.437},"max":{"1min":0.90200000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":1.121,"15min":1.121},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93200000000000005,"5min":1.095,"15min":1.095},"last":0.71399999999999997}]}]},{"osd":3,"up_from":14,"seq":60129543067,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59764,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312076,"statfs":{"total":96636764160,"available":96575565824,"internally_reserved":0,"allocated":2265088,"data_stored":1143172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.441},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59399999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.318}]},{"osd":1,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52300000000000002,"5min":0.621,"15min":0.76400000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.441},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.66600000000000004},"last":0.42399999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28499999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.64700000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.28499999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.96999999999999997},"last":0.33600000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.92400000000000004,"15min":0.98499999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81399999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.0029999999999999},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.0349999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.995},"last":0.499}]},{"osd":7,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95599999999999996,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94099999999999995,"5min":0.96599999999999997,"15min":0.98399999999999999},"last":0.437}]}]},{"osd":4,"up_from":14,"seq":60129543066,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51456,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320384,"statfs":{"total":96636764160,"available":96584073216,"internally_reserved":0,"allocated":2277376,"data_stored":1144820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72999999999999998,"5min":0.83799999999999997,"15min":0.90100000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.32300000000000001},"max":{"1min":0.77500000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.88400000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.68799999999999994,"5min":0.77400000000000002,"15min":0.80800000000000005},"last":0.39700000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.34200000000000003},"max":{"1min":0.72499999999999998,"5min":0.86199999999999999,"15min":1.077},"last":0.38},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41099999999999998},"max":{"1min":0.72099999999999997,"5min":0.81100000000000005,"15min":0.97899999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.84099999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.48299999999999998,"5min":0.432,"15min":0.41099999999999998},"max":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.61799999999999999,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44500000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.86899999999999999,"15min":1.03},"last":0.496},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71299999999999997,"5min":0.89100000000000001,"15min":0.96099999999999997},"last":0.58899999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.69099999999999995,"5min":0.88,"15min":0.88},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69299999999999995,"5min":0.83999999999999997,"15min":1.133},"last":0.57499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129543067,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37556,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334284,"statfs":{"total":96636764160,"available":96598306816,"internally_reserved":0,"allocated":2269184,"data_stored":1143742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78200000000000003,"5min":0.82699999999999996,"15min":0.876},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70699999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.32900000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80500000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.624,"15min":0.63800000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.86099999999999999},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.502,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.32200000000000001,"15min":0.315},"max":{"1min":0.66200000000000003,"5min":0.753,"15min":0.78700000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78300000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58299999999999996,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.373,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.84899999999999998},"last":0.47799999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36099999999999999},"max":{"1min":0.75900000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:01:27.890 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:01:28.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.043+0000 7fa7d77fe700 1 Processor -- start 2023-10-16T11:01:28.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.043+0000 7fa7d77fe700 1 -- start start 2023-10-16T11:01:28.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7d015dfd0 0x7fa7d00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d77fe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d006f870 con 0x7fa7d00af450 2023-10-16T11:01:28.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d77fe700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d00d1160 con 0x7fa7d00cd470 2023-10-16T11:01:28.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d77fe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00cc8e0 con 0x7fa7d015dfd0 2023-10-16T11:01:28.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d559a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7d015dfd0 0x7fa7d00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d4d99700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa7d00af450 0x7fa7d006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35020/0 (socket says 172.21.15.181:35020) 2023-10-16T11:01:28.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d4d99700 1 -- 172.21.15.181:0/3890059426 learned_addr learned my addr 172.21.15.181:0/3890059426 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d559a700 1 -- 172.21.15.181:0/3890059426 >> v1:172.21.15.190:6789/0 conn(0x7fa7d00cd470 legacy=0x7fa7d006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d559a700 1 -- 172.21.15.181:0/3890059426 >> v1:172.21.15.181:6789/0 conn(0x7fa7d00af450 legacy=0x7fa7d006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d559a700 1 -- 172.21.15.181:0/3890059426 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00ccbc0 con 0x7fa7d015dfd0 2023-10-16T11:01:28.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7c7fff700 1 -- 172.21.15.181:0/3890059426 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766556505 0 0) 0x7fa7d006f870 con 0x7fa7d00af450 2023-10-16T11:01:28.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7d559a700 1 --2- 172.21.15.181:0/3890059426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00af9a0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009820 tx=0x7fa7cc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f873d27f177da4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7c7fff700 1 -- 172.21.15.181:0/3890059426 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc005e30 con 0x7fa7d015dfd0 2023-10-16T11:01:28.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7c7fff700 1 -- 172.21.15.181:0/3890059426 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc004030 con 0x7fa7d015dfd0 2023-10-16T11:01:28.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.044+0000 7fa7c7fff700 1 -- 172.21.15.181:0/3890059426 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0043c0 con 0x7fa7d015dfd0 2023-10-16T11:01:28.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.045+0000 7fa7d77fe700 1 -- 172.21.15.181:0/3890059426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 msgr2=0x7fa7d00af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.045+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/3890059426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00af9a0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009820 tx=0x7fa7cc009b30 comp rx=0 tx=0).stop 2023-10-16T11:01:28.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.045+0000 7fa7d77fe700 1 -- 172.21.15.181:0/3890059426 shutdown_connections 2023-10-16T11:01:28.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.045+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/3890059426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00af9a0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.045+0000 7fa7d77fe700 1 -- 172.21.15.181:0/3890059426 >> 172.21.15.181:0/3890059426 conn(0x7fa7d00d04a0 msgr2=0x7fa7d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:28.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.045+0000 7fa7d77fe700 1 -- 172.21.15.181:0/3890059426 shutdown_connections 2023-10-16T11:01:28.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.045+0000 7fa7d77fe700 1 -- 172.21.15.181:0/3890059426 wait complete. 2023-10-16T11:01:28.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.046+0000 7fa7d77fe700 1 Processor -- start 2023-10-16T11:01:28.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.046+0000 7fa7d77fe700 1 -- start start 2023-10-16T11:01:28.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7d00af450 0x7fa7d0156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d00cd470 0x7fa7d00c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d77fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c4070 con 0x7fa7d015dfd0 2023-10-16T11:01:28.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d77fe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00b1460 con 0x7fa7d00cd470 2023-10-16T11:01:28.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d77fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c4310 con 0x7fa7d00af450 2023-10-16T11:01:28.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52180/0 (socket says 172.21.15.181:52180) 2023-10-16T11:01:28.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 -- 172.21.15.181:0/2492985879 learned_addr learned my addr 172.21.15.181:0/2492985879 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:28.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d4d99700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d00cd470 0x7fa7d00c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d559a700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7d00af450 0x7fa7d0156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 -- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7d00af450 msgr2=0x7fa7d0156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7d00af450 0x7fa7d0156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 -- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d00cd470 msgr2=0x7fa7d00c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d00cd470 0x7fa7d00c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d5d9b700 1 -- 172.21.15.181:0/2492985879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc009430 con 0x7fa7d015dfd0 2023-10-16T11:01:28.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.047+0000 7fa7d559a700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7d00af450 0x7fa7d0156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:28.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.048+0000 7fa7d5d9b700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00b0c30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c7a0 tx=0x7fa7c800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.048+0000 7fa7c5ffb700 1 -- 172.21.15.181:0/2492985879 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8003e10 con 0x7fa7d015dfd0 2023-10-16T11:01:28.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.048+0000 7fa7c5ffb700 1 -- 172.21.15.181:0/2492985879 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8005070 con 0x7fa7d015dfd0 2023-10-16T11:01:28.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.048+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d00c45f0 con 0x7fa7d015dfd0 2023-10-16T11:01:28.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.048+0000 7fa7c5ffb700 1 -- 172.21.15.181:0/2492985879 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8005420 con 0x7fa7d015dfd0 2023-10-16T11:01:28.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.048+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d00b1740 con 0x7fa7d015dfd0 2023-10-16T11:01:28.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.049+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d00c3ec0 con 0x7fa7d015dfd0 2023-10-16T11:01:28.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.049+0000 7fa7c5ffb700 1 -- 172.21.15.181:0/2492985879 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa7c8005580 con 0x7fa7d015dfd0 2023-10-16T11:01:28.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.050+0000 7fa7c5ffb700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fa7b406b020 0x7fa7b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.050+0000 7fa7c5ffb700 1 -- 172.21.15.181:0/2492985879 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fa7c8088f20 con 0x7fa7d015dfd0 2023-10-16T11:01:28.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.050+0000 7fa7d559a700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fa7b406b020 0x7fa7b406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.050+0000 7fa7d559a700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fa7b406b020 0x7fa7b406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc008070 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.054+0000 7fa7c5ffb700 1 -- 172.21.15.181:0/2492985879 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa7c8053d60 con 0x7fa7d015dfd0 2023-10-16T11:01:28.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.242+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 --> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7d00ccf50 con 0x7fa7b406b020 2023-10-16T11:01:28.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.250+0000 7fa7c5ffb700 1 -- 172.21.15.181:0/2492985879 <== mgr.7889 v2:172.21.15.181:6832/303240986 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290384 (secure 0 0 0) 0x7fa7cc003980 con 0x7fa7b406b020 2023-10-16T11:01:28.254 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:28.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fa7b406b020 msgr2=0x7fa7b406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fa7b406b020 0x7fa7b406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc008070 comp rx=0 tx=0).stop 2023-10-16T11:01:28.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 msgr2=0x7fa7d00b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00b0c30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c7a0 tx=0x7fa7c800cab0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 shutdown_connections 2023-10-16T11:01:28.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fa7b406b020 0x7fa7b406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7d00af450 0x7fa7d0156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7d015dfd0 0x7fa7d00b0c30 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 --2- 172.21.15.181:0/2492985879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7d00cd470 0x7fa7d00c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 >> 172.21.15.181:0/2492985879 conn(0x7fa7d00d04a0 msgr2=0x7fa7d006d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:28.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 shutdown_connections 2023-10-16T11:01:28.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.253+0000 7fa7d77fe700 1 -- 172.21.15.181:0/2492985879 wait complete. 2023-10-16T11:01:28.259 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:01:28.267 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T11:01:27.541851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405732,"kb_used_data":18864,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754568988,"statfs":{"total":773094113280,"available":772678643712,"internally_reserved":0,"allocated":19316736,"data_stored":10317068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":396021304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.959520"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014069+0000","last_change":"2023-10-16T11:01:23.625128+0000","last_active":"2023-10-16T11:01:25.014069+0000","last_peered":"2023-10-16T11:01:25.014069+0000","last_clean":"2023-10-16T11:01:25.014069+0000","last_became_active":"2023-10-16T11:01:23.624860+0000","last_became_peered":"2023-10-16T11:01:23.624860+0000","last_unstale":"2023-10-16T11:01:25.014069+0000","last_undegraded":"2023-10-16T11:01:25.014069+0000","last_fullsized":"2023-10-16T11:01:25.014069+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:59.036068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013631+0000","last_change":"2023-10-16T11:01:23.613248+0000","last_active":"2023-10-16T11:01:25.013631+0000","last_peered":"2023-10-16T11:01:25.013631+0000","last_clean":"2023-10-16T11:01:25.013631+0000","last_became_active":"2023-10-16T11:01:23.612966+0000","last_became_peered":"2023-10-16T11:01:23.612966+0000","last_unstale":"2023-10-16T11:01:25.013631+0000","last_undegraded":"2023-10-16T11:01:25.013631+0000","last_fullsized":"2023-10-16T11:01:25.013631+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:06.144124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604633+0000","last_change":"2023-10-16T11:01:23.626806+0000","last_active":"2023-10-16T11:01:24.604633+0000","last_peered":"2023-10-16T11:01:24.604633+0000","last_clean":"2023-10-16T11:01:24.604633+0000","last_became_active":"2023-10-16T11:01:23.626523+0000","last_became_peered":"2023-10-16T11:01:23.626523+0000","last_unstale":"2023-10-16T11:01:24.604633+0000","last_undegraded":"2023-10-16T11:01:24.604633+0000","last_fullsized":"2023-10-16T11:01:24.604633+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:42:40.009403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013526+0000","last_change":"2023-10-16T11:01:23.611019+0000","last_active":"2023-10-16T11:01:25.013526+0000","last_peered":"2023-10-16T11:01:25.013526+0000","last_clean":"2023-10-16T11:01:25.013526+0000","last_became_active":"2023-10-16T11:01:23.610692+0000","last_became_peered":"2023-10-16T11:01:23.610692+0000","last_unstale":"2023-10-16T11:01:25.013526+0000","last_undegraded":"2023-10-16T11:01:25.013526+0000","last_fullsized":"2023-10-16T11:01:25.013526+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:28.380320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013832+0000","last_change":"2023-10-16T11:01:23.616848+0000","last_active":"2023-10-16T11:01:25.013832+0000","last_peered":"2023-10-16T11:01:25.013832+0000","last_clean":"2023-10-16T11:01:25.013832+0000","last_became_active":"2023-10-16T11:01:23.616586+0000","last_became_peered":"2023-10-16T11:01:23.616586+0000","last_unstale":"2023-10-16T11:01:25.013832+0000","last_undegraded":"2023-10-16T11:01:25.013832+0000","last_fullsized":"2023-10-16T11:01:25.013832+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:21.313008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603997+0000","last_change":"2023-10-16T11:01:23.617874+0000","last_active":"2023-10-16T11:01:24.603997+0000","last_peered":"2023-10-16T11:01:24.603997+0000","last_clean":"2023-10-16T11:01:24.603997+0000","last_became_active":"2023-10-16T11:01:23.617586+0000","last_became_peered":"2023-10-16T11:01:23.617586+0000","last_unstale":"2023-10-16T11:01:24.603997+0000","last_undegraded":"2023-10-16T11:01:24.603997+0000","last_fullsized":"2023-10-16T11:01:24.603997+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:21.313008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014111+0000","last_change":"2023-10-16T11:01:23.611029+0000","last_active":"2023-10-16T11:01:25.014111+0000","last_peered":"2023-10-16T11:01:25.014111+0000","last_clean":"2023-10-16T11:01:25.014111+0000","last_became_active":"2023-10-16T11:01:23.610655+0000","last_became_peered":"2023-10-16T11:01:23.610655+0000","last_unstale":"2023-10-16T11:01:25.014111+0000","last_undegraded":"2023-10-16T11:01:25.014111+0000","last_fullsized":"2023-10-16T11:01:25.014111+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:30.419217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014147+0000","last_change":"2023-10-16T11:01:23.613754+0000","last_active":"2023-10-16T11:01:25.014147+0000","last_peered":"2023-10-16T11:01:25.014147+0000","last_clean":"2023-10-16T11:01:25.014147+0000","last_became_active":"2023-10-16T11:01:23.612799+0000","last_became_peered":"2023-10-16T11:01:23.612799+0000","last_unstale":"2023-10-16T11:01:25.014147+0000","last_undegraded":"2023-10-16T11:01:25.014147+0000","last_fullsized":"2023-10-16T11:01:25.014147+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:00.693125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014086+0000","last_change":"2023-10-16T11:01:23.611460+0000","last_active":"2023-10-16T11:01:25.014086+0000","last_peered":"2023-10-16T11:01:25.014086+0000","last_clean":"2023-10-16T11:01:25.014086+0000","last_became_active":"2023-10-16T11:01:23.611184+0000","last_became_peered":"2023-10-16T11:01:23.611184+0000","last_unstale":"2023-10-16T11:01:25.014086+0000","last_undegraded":"2023-10-16T11:01:25.014086+0000","last_fullsized":"2023-10-16T11:01:25.014086+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:29.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013839+0000","last_change":"2023-10-16T11:01:23.614241+0000","last_active":"2023-10-16T11:01:25.013839+0000","last_peered":"2023-10-16T11:01:25.013839+0000","last_clean":"2023-10-16T11:01:25.013839+0000","last_became_active":"2023-10-16T11:01:23.614009+0000","last_became_peered":"2023-10-16T11:01:23.614009+0000","last_unstale":"2023-10-16T11:01:25.013839+0000","last_undegraded":"2023-10-16T11:01:25.013839+0000","last_fullsized":"2023-10-16T11:01:25.013839+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:07.785867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013987+0000","last_change":"2023-10-16T11:01:23.617503+0000","last_active":"2023-10-16T11:01:25.013987+0000","last_peered":"2023-10-16T11:01:25.013987+0000","last_clean":"2023-10-16T11:01:25.013987+0000","last_became_active":"2023-10-16T11:01:23.617240+0000","last_became_peered":"2023-10-16T11:01:23.617240+0000","last_unstale":"2023-10-16T11:01:25.013987+0000","last_undegraded":"2023-10-16T11:01:25.013987+0000","last_fullsized":"2023-10-16T11:01:25.013987+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:41.282912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013748+0000","last_change":"2023-10-16T11:01:23.612273+0000","last_active":"2023-10-16T11:01:25.013748+0000","last_peered":"2023-10-16T11:01:25.013748+0000","last_clean":"2023-10-16T11:01:25.013748+0000","last_became_active":"2023-10-16T11:01:23.611952+0000","last_became_peered":"2023-10-16T11:01:23.611952+0000","last_unstale":"2023-10-16T11:01:25.013748+0000","last_undegraded":"2023-10-16T11:01:25.013748+0000","last_fullsized":"2023-10-16T11:01:25.013748+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:30.728808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604342+0000","last_change":"2023-10-16T11:01:23.622565+0000","last_active":"2023-10-16T11:01:24.604342+0000","last_peered":"2023-10-16T11:01:24.604342+0000","last_clean":"2023-10-16T11:01:24.604342+0000","last_became_active":"2023-10-16T11:01:23.622255+0000","last_became_peered":"2023-10-16T11:01:23.622255+0000","last_unstale":"2023-10-16T11:01:24.604342+0000","last_undegraded":"2023-10-16T11:01:24.604342+0000","last_fullsized":"2023-10-16T11:01:24.604342+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:02:46.611922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604139+0000","last_change":"2023-10-16T11:01:23.621490+0000","last_active":"2023-10-16T11:01:24.604139+0000","last_peered":"2023-10-16T11:01:24.604139+0000","last_clean":"2023-10-16T11:01:24.604139+0000","last_became_active":"2023-10-16T11:01:23.621081+0000","last_became_peered":"2023-10-16T11:01:23.621081+0000","last_unstale":"2023-10-16T11:01:24.604139+0000","last_undegraded":"2023-10-16T11:01:24.604139+0000","last_fullsized":"2023-10-16T11:01:24.604139+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:00.693125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013378+0000","last_change":"2023-10-16T11:01:23.611811+0000","last_active":"2023-10-16T11:01:25.013378+0000","last_peered":"2023-10-16T11:01:25.013378+0000","last_clean":"2023-10-16T11:01:25.013378+0000","last_became_active":"2023-10-16T11:01:23.611533+0000","last_became_peered":"2023-10-16T11:01:23.611533+0000","last_unstale":"2023-10-16T11:01:25.013378+0000","last_undegraded":"2023-10-16T11:01:25.013378+0000","last_fullsized":"2023-10-16T11:01:25.013378+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:15.289849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013051+0000","last_change":"2023-10-16T11:01:23.613126+0000","last_active":"2023-10-16T11:01:25.013051+0000","last_peered":"2023-10-16T11:01:25.013051+0000","last_clean":"2023-10-16T11:01:25.013051+0000","last_became_active":"2023-10-16T11:01:23.612874+0000","last_became_peered":"2023-10-16T11:01:23.612874+0000","last_unstale":"2023-10-16T11:01:25.013051+0000","last_undegraded":"2023-10-16T11:01:25.013051+0000","last_fullsized":"2023-10-16T11:01:25.013051+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:09.162169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604095+0000","last_change":"2023-10-16T11:01:23.624705+0000","last_active":"2023-10-16T11:01:24.604095+0000","last_peered":"2023-10-16T11:01:24.604095+0000","last_clean":"2023-10-16T11:01:24.604095+0000","last_became_active":"2023-10-16T11:01:23.624416+0000","last_became_peered":"2023-10-16T11:01:23.624416+0000","last_unstale":"2023-10-16T11:01:24.604095+0000","last_undegraded":"2023-10-16T11:01:24.604095+0000","last_fullsized":"2023-10-16T11:01:24.604095+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:29.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013272+0000","last_change":"2023-10-16T11:01:23.616538+0000","last_active":"2023-10-16T11:01:25.013272+0000","last_peered":"2023-10-16T11:01:25.013272+0000","last_clean":"2023-10-16T11:01:25.013272+0000","last_became_active":"2023-10-16T11:01:23.616028+0000","last_became_peered":"2023-10-16T11:01:23.616028+0000","last_unstale":"2023-10-16T11:01:25.013272+0000","last_undegraded":"2023-10-16T11:01:25.013272+0000","last_fullsized":"2023-10-16T11:01:25.013272+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:19.843632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604045+0000","last_change":"2023-10-16T11:01:23.619098+0000","last_active":"2023-10-16T11:01:24.604045+0000","last_peered":"2023-10-16T11:01:24.604045+0000","last_clean":"2023-10-16T11:01:24.604045+0000","last_became_active":"2023-10-16T11:01:23.618827+0000","last_became_peered":"2023-10-16T11:01:23.618827+0000","last_unstale":"2023-10-16T11:01:24.604045+0000","last_undegraded":"2023-10-16T11:01:24.604045+0000","last_fullsized":"2023-10-16T11:01:24.604045+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:07.785867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604461+0000","last_change":"2023-10-16T11:01:23.623468+0000","last_active":"2023-10-16T11:01:24.604461+0000","last_peered":"2023-10-16T11:01:24.604461+0000","last_clean":"2023-10-16T11:01:24.604461+0000","last_became_active":"2023-10-16T11:01:23.623193+0000","last_became_peered":"2023-10-16T11:01:23.623193+0000","last_unstale":"2023-10-16T11:01:24.604461+0000","last_undegraded":"2023-10-16T11:01:24.604461+0000","last_fullsized":"2023-10-16T11:01:24.604461+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:48:23.906440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604102+0000","last_change":"2023-10-16T11:01:23.624441+0000","last_active":"2023-10-16T11:01:24.604102+0000","last_peered":"2023-10-16T11:01:24.604102+0000","last_clean":"2023-10-16T11:01:24.604102+0000","last_became_active":"2023-10-16T11:01:23.624176+0000","last_became_peered":"2023-10-16T11:01:23.624176+0000","last_unstale":"2023-10-16T11:01:24.604102+0000","last_undegraded":"2023-10-16T11:01:24.604102+0000","last_fullsized":"2023-10-16T11:01:24.604102+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:00.693125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013566+0000","last_change":"2023-10-16T11:01:23.609492+0000","last_active":"2023-10-16T11:01:25.013566+0000","last_peered":"2023-10-16T11:01:25.013566+0000","last_clean":"2023-10-16T11:01:25.013566+0000","last_became_active":"2023-10-16T11:01:23.609115+0000","last_became_peered":"2023-10-16T11:01:23.609115+0000","last_unstale":"2023-10-16T11:01:25.013566+0000","last_undegraded":"2023-10-16T11:01:25.013566+0000","last_fullsized":"2023-10-16T11:01:25.013566+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:49.126756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603836+0000","last_change":"2023-10-16T11:01:23.619886+0000","last_active":"2023-10-16T11:01:24.603836+0000","last_peered":"2023-10-16T11:01:24.603836+0000","last_clean":"2023-10-16T11:01:24.603836+0000","last_became_active":"2023-10-16T11:01:23.619600+0000","last_became_peered":"2023-10-16T11:01:23.619600+0000","last_unstale":"2023-10-16T11:01:24.603836+0000","last_undegraded":"2023-10-16T11:01:24.603836+0000","last_fullsized":"2023-10-16T11:01:24.603836+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:48.469185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013432+0000","last_change":"2023-10-16T11:01:23.624584+0000","last_active":"2023-10-16T11:01:25.013432+0000","last_peered":"2023-10-16T11:01:25.013432+0000","last_clean":"2023-10-16T11:01:25.013432+0000","last_became_active":"2023-10-16T11:01:23.624285+0000","last_became_peered":"2023-10-16T11:01:23.624285+0000","last_unstale":"2023-10-16T11:01:25.013432+0000","last_undegraded":"2023-10-16T11:01:25.013432+0000","last_fullsized":"2023-10-16T11:01:25.013432+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:48.469185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604793+0000","last_change":"2023-10-16T11:01:23.621709+0000","last_active":"2023-10-16T11:01:24.604793+0000","last_peered":"2023-10-16T11:01:24.604793+0000","last_clean":"2023-10-16T11:01:24.604793+0000","last_became_active":"2023-10-16T11:01:23.621286+0000","last_became_peered":"2023-10-16T11:01:23.621286+0000","last_unstale":"2023-10-16T11:01:24.604793+0000","last_undegraded":"2023-10-16T11:01:24.604793+0000","last_fullsized":"2023-10-16T11:01:24.604793+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:01:02.039841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014504+0000","last_change":"2023-10-16T11:01:23.611732+0000","last_active":"2023-10-16T11:01:25.014504+0000","last_peered":"2023-10-16T11:01:25.014504+0000","last_clean":"2023-10-16T11:01:25.014504+0000","last_became_active":"2023-10-16T11:01:23.611462+0000","last_became_peered":"2023-10-16T11:01:23.611462+0000","last_unstale":"2023-10-16T11:01:25.014504+0000","last_undegraded":"2023-10-16T11:01:25.014504+0000","last_fullsized":"2023-10-16T11:01:25.014504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:32.890786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604726+0000","last_change":"2023-10-16T11:01:23.621946+0000","last_active":"2023-10-16T11:01:24.604726+0000","last_peered":"2023-10-16T11:01:24.604726+0000","last_clean":"2023-10-16T11:01:24.604726+0000","last_became_active":"2023-10-16T11:01:23.621675+0000","last_became_peered":"2023-10-16T11:01:23.621675+0000","last_unstale":"2023-10-16T11:01:24.604726+0000","last_undegraded":"2023-10-16T11:01:24.604726+0000","last_fullsized":"2023-10-16T11:01:24.604726+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:48.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013976+0000","last_change":"2023-10-16T11:01:23.616237+0000","last_active":"2023-10-16T11:01:25.013976+0000","last_peered":"2023-10-16T11:01:25.013976+0000","last_clean":"2023-10-16T11:01:25.013976+0000","last_became_active":"2023-10-16T11:01:23.615967+0000","last_became_peered":"2023-10-16T11:01:23.615967+0000","last_unstale":"2023-10-16T11:01:25.013976+0000","last_undegraded":"2023-10-16T11:01:25.013976+0000","last_fullsized":"2023-10-16T11:01:25.013976+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:14:25.171750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604516+0000","last_change":"2023-10-16T11:01:23.622442+0000","last_active":"2023-10-16T11:01:24.604516+0000","last_peered":"2023-10-16T11:01:24.604516+0000","last_clean":"2023-10-16T11:01:24.604516+0000","last_became_active":"2023-10-16T11:01:23.622103+0000","last_became_peered":"2023-10-16T11:01:23.622103+0000","last_unstale":"2023-10-16T11:01:24.604516+0000","last_undegraded":"2023-10-16T11:01:24.604516+0000","last_fullsized":"2023-10-16T11:01:24.604516+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:27.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605689+0000","last_change":"2023-10-16T11:01:23.615161+0000","last_active":"2023-10-16T11:01:24.605689+0000","last_peered":"2023-10-16T11:01:24.605689+0000","last_clean":"2023-10-16T11:01:24.605689+0000","last_became_active":"2023-10-16T11:01:23.614809+0000","last_became_peered":"2023-10-16T11:01:23.614809+0000","last_unstale":"2023-10-16T11:01:24.605689+0000","last_undegraded":"2023-10-16T11:01:24.605689+0000","last_fullsized":"2023-10-16T11:01:24.605689+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:26.688383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605488+0000","last_change":"2023-10-16T11:01:23.610446+0000","last_active":"2023-10-16T11:01:24.605488+0000","last_peered":"2023-10-16T11:01:24.605488+0000","last_clean":"2023-10-16T11:01:24.605488+0000","last_became_active":"2023-10-16T11:01:23.610175+0000","last_became_peered":"2023-10-16T11:01:23.610175+0000","last_unstale":"2023-10-16T11:01:24.605488+0000","last_undegraded":"2023-10-16T11:01:24.605488+0000","last_fullsized":"2023-10-16T11:01:24.605488+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:34.363924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013068+0000","last_change":"2023-10-16T11:01:23.618805+0000","last_active":"2023-10-16T11:01:25.013068+0000","last_peered":"2023-10-16T11:01:25.013068+0000","last_clean":"2023-10-16T11:01:25.013068+0000","last_became_active":"2023-10-16T11:01:23.615924+0000","last_became_peered":"2023-10-16T11:01:23.615924+0000","last_unstale":"2023-10-16T11:01:25.013068+0000","last_undegraded":"2023-10-16T11:01:25.013068+0000","last_fullsized":"2023-10-16T11:01:25.013068+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:28:22.370874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014629+0000","last_change":"2023-10-16T11:01:23.611656+0000","last_active":"2023-10-16T11:01:25.014629+0000","last_peered":"2023-10-16T11:01:25.014629+0000","last_clean":"2023-10-16T11:01:25.014629+0000","last_became_active":"2023-10-16T11:01:23.611399+0000","last_became_peered":"2023-10-16T11:01:23.611399+0000","last_unstale":"2023-10-16T11:01:25.014629+0000","last_undegraded":"2023-10-16T11:01:25.014629+0000","last_fullsized":"2023-10-16T11:01:25.014629+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:48.469185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"180'4","reported_seq":16,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013988+0000","last_change":"2023-10-16T11:01:22.601746+0000","last_active":"2023-10-16T11:01:25.013988+0000","last_peered":"2023-10-16T11:01:25.013988+0000","last_clean":"2023-10-16T11:01:25.013988+0000","last_became_active":"2023-10-16T11:01:22.601480+0000","last_became_peered":"2023-10-16T11:01:22.601480+0000","last_unstale":"2023-10-16T11:01:25.013988+0000","last_undegraded":"2023-10-16T11:01:25.013988+0000","last_fullsized":"2023-10-16T11:01:25.013988+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:39.616473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605990+0000","last_change":"2023-10-16T11:01:23.610653+0000","last_active":"2023-10-16T11:01:24.605990+0000","last_peered":"2023-10-16T11:01:24.605990+0000","last_clean":"2023-10-16T11:01:24.605990+0000","last_became_active":"2023-10-16T11:01:23.610381+0000","last_became_peered":"2023-10-16T11:01:23.610381+0000","last_unstale":"2023-10-16T11:01:24.605990+0000","last_undegraded":"2023-10-16T11:01:24.605990+0000","last_fullsized":"2023-10-16T11:01:24.605990+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:39:58.960773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605836+0000","last_change":"2023-10-16T11:01:22.602658+0000","last_active":"2023-10-16T11:01:24.605836+0000","last_peered":"2023-10-16T11:01:24.605836+0000","last_clean":"2023-10-16T11:01:24.605836+0000","last_became_active":"2023-10-16T11:01:22.601728+0000","last_became_peered":"2023-10-16T11:01:22.601728+0000","last_unstale":"2023-10-16T11:01:24.605836+0000","last_undegraded":"2023-10-16T11:01:24.605836+0000","last_fullsized":"2023-10-16T11:01:24.605836+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:01:06.184465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604089+0000","last_change":"2023-10-16T11:01:23.622047+0000","last_active":"2023-10-16T11:01:24.604089+0000","last_peered":"2023-10-16T11:01:24.604089+0000","last_clean":"2023-10-16T11:01:24.604089+0000","last_became_active":"2023-10-16T11:01:23.621766+0000","last_became_peered":"2023-10-16T11:01:23.621766+0000","last_unstale":"2023-10-16T11:01:24.604089+0000","last_undegraded":"2023-10-16T11:01:24.604089+0000","last_fullsized":"2023-10-16T11:01:24.604089+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:41.282912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013721+0000","last_change":"2023-10-16T11:01:22.604600+0000","last_active":"2023-10-16T11:01:25.013721+0000","last_peered":"2023-10-16T11:01:25.013721+0000","last_clean":"2023-10-16T11:01:25.013721+0000","last_became_active":"2023-10-16T11:01:22.604259+0000","last_became_peered":"2023-10-16T11:01:22.604259+0000","last_unstale":"2023-10-16T11:01:25.013721+0000","last_undegraded":"2023-10-16T11:01:25.013721+0000","last_fullsized":"2023-10-16T11:01:25.013721+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:31:48.077668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604819+0000","last_change":"2023-10-16T11:01:23.621641+0000","last_active":"2023-10-16T11:01:24.604819+0000","last_peered":"2023-10-16T11:01:24.604819+0000","last_clean":"2023-10-16T11:01:24.604819+0000","last_became_active":"2023-10-16T11:01:23.621326+0000","last_became_peered":"2023-10-16T11:01:23.621326+0000","last_unstale":"2023-10-16T11:01:24.604819+0000","last_undegraded":"2023-10-16T11:01:24.604819+0000","last_fullsized":"2023-10-16T11:01:24.604819+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:26.544254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014265+0000","last_change":"2023-10-16T11:01:22.603164+0000","last_active":"2023-10-16T11:01:25.014265+0000","last_peered":"2023-10-16T11:01:25.014265+0000","last_clean":"2023-10-16T11:01:25.014265+0000","last_became_active":"2023-10-16T11:01:22.602204+0000","last_became_peered":"2023-10-16T11:01:22.602204+0000","last_unstale":"2023-10-16T11:01:25.014265+0000","last_undegraded":"2023-10-16T11:01:25.014265+0000","last_fullsized":"2023-10-16T11:01:25.014265+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:24.172489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014242+0000","last_change":"2023-10-16T11:01:23.607207+0000","last_active":"2023-10-16T11:01:25.014242+0000","last_peered":"2023-10-16T11:01:25.014242+0000","last_clean":"2023-10-16T11:01:25.014242+0000","last_became_active":"2023-10-16T11:01:23.606174+0000","last_became_peered":"2023-10-16T11:01:23.606174+0000","last_unstale":"2023-10-16T11:01:25.014242+0000","last_undegraded":"2023-10-16T11:01:25.014242+0000","last_fullsized":"2023-10-16T11:01:25.014242+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:57.849844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604440+0000","last_change":"2023-10-16T11:01:22.602206+0000","last_active":"2023-10-16T11:01:24.604440+0000","last_peered":"2023-10-16T11:01:24.604440+0000","last_clean":"2023-10-16T11:01:24.604440+0000","last_became_active":"2023-10-16T11:01:22.601945+0000","last_became_peered":"2023-10-16T11:01:22.601945+0000","last_unstale":"2023-10-16T11:01:24.604440+0000","last_undegraded":"2023-10-16T11:01:24.604440+0000","last_fullsized":"2023-10-16T11:01:24.604440+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:36:20.305491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013706+0000","last_change":"2023-10-16T11:01:23.610061+0000","last_active":"2023-10-16T11:01:25.013706+0000","last_peered":"2023-10-16T11:01:25.013706+0000","last_clean":"2023-10-16T11:01:25.013706+0000","last_became_active":"2023-10-16T11:01:23.609706+0000","last_became_peered":"2023-10-16T11:01:23.609706+0000","last_unstale":"2023-10-16T11:01:25.013706+0000","last_undegraded":"2023-10-16T11:01:25.013706+0000","last_fullsized":"2023-10-16T11:01:25.013706+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:55:27.321104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013179+0000","last_change":"2023-10-16T11:01:22.606161+0000","last_active":"2023-10-16T11:01:25.013179+0000","last_peered":"2023-10-16T11:01:25.013179+0000","last_clean":"2023-10-16T11:01:25.013179+0000","last_became_active":"2023-10-16T11:01:22.605539+0000","last_became_peered":"2023-10-16T11:01:22.605539+0000","last_unstale":"2023-10-16T11:01:25.013179+0000","last_undegraded":"2023-10-16T11:01:25.013179+0000","last_fullsized":"2023-10-16T11:01:25.013179+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:45.097655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013464+0000","last_change":"2023-10-16T11:01:23.615935+0000","last_active":"2023-10-16T11:01:25.013464+0000","last_peered":"2023-10-16T11:01:25.013464+0000","last_clean":"2023-10-16T11:01:25.013464+0000","last_became_active":"2023-10-16T11:01:23.615412+0000","last_became_peered":"2023-10-16T11:01:23.615412+0000","last_unstale":"2023-10-16T11:01:25.013464+0000","last_undegraded":"2023-10-16T11:01:25.013464+0000","last_fullsized":"2023-10-16T11:01:25.013464+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:03.291177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013567+0000","last_change":"2023-10-16T11:01:22.605428+0000","last_active":"2023-10-16T11:01:25.013567+0000","last_peered":"2023-10-16T11:01:25.013567+0000","last_clean":"2023-10-16T11:01:25.013567+0000","last_became_active":"2023-10-16T11:01:22.605124+0000","last_became_peered":"2023-10-16T11:01:22.605124+0000","last_unstale":"2023-10-16T11:01:25.013567+0000","last_undegraded":"2023-10-16T11:01:25.013567+0000","last_fullsized":"2023-10-16T11:01:25.013567+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:56.842327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603834+0000","last_change":"2023-10-16T11:01:23.617853+0000","last_active":"2023-10-16T11:01:24.603834+0000","last_peered":"2023-10-16T11:01:24.603834+0000","last_clean":"2023-10-16T11:01:24.603834+0000","last_became_active":"2023-10-16T11:01:23.617569+0000","last_became_peered":"2023-10-16T11:01:23.617569+0000","last_unstale":"2023-10-16T11:01:24.603834+0000","last_undegraded":"2023-10-16T11:01:24.603834+0000","last_fullsized":"2023-10-16T11:01:24.603834+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:49.126756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013737+0000","last_change":"2023-10-16T11:01:22.604779+0000","last_active":"2023-10-16T11:01:25.013737+0000","last_peered":"2023-10-16T11:01:25.013737+0000","last_clean":"2023-10-16T11:01:25.013737+0000","last_became_active":"2023-10-16T11:01:22.604526+0000","last_became_peered":"2023-10-16T11:01:22.604526+0000","last_unstale":"2023-10-16T11:01:25.013737+0000","last_undegraded":"2023-10-16T11:01:25.013737+0000","last_fullsized":"2023-10-16T11:01:25.013737+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:41:14.971959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014178+0000","last_change":"2023-10-16T11:01:23.607120+0000","last_active":"2023-10-16T11:01:25.014178+0000","last_peered":"2023-10-16T11:01:25.014178+0000","last_clean":"2023-10-16T11:01:25.014178+0000","last_became_active":"2023-10-16T11:01:23.606839+0000","last_became_peered":"2023-10-16T11:01:23.606839+0000","last_unstale":"2023-10-16T11:01:25.014178+0000","last_undegraded":"2023-10-16T11:01:25.014178+0000","last_fullsized":"2023-10-16T11:01:25.014178+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:40.623989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604021+0000","last_change":"2023-10-16T11:01:22.599810+0000","last_active":"2023-10-16T11:01:24.604021+0000","last_peered":"2023-10-16T11:01:24.604021+0000","last_clean":"2023-10-16T11:01:24.604021+0000","last_became_active":"2023-10-16T11:01:22.599552+0000","last_became_peered":"2023-10-16T11:01:22.599552+0000","last_unstale":"2023-10-16T11:01:24.604021+0000","last_undegraded":"2023-10-16T11:01:24.604021+0000","last_fullsized":"2023-10-16T11:01:24.604021+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:23.085725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013347+0000","last_change":"2023-10-16T11:01:23.609616+0000","last_active":"2023-10-16T11:01:25.013347+0000","last_peered":"2023-10-16T11:01:25.013347+0000","last_clean":"2023-10-16T11:01:25.013347+0000","last_became_active":"2023-10-16T11:01:23.609357+0000","last_became_peered":"2023-10-16T11:01:23.609357+0000","last_unstale":"2023-10-16T11:01:25.013347+0000","last_undegraded":"2023-10-16T11:01:25.013347+0000","last_fullsized":"2023-10-16T11:01:25.013347+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:18.354218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"180'3","reported_seq":15,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604197+0000","last_change":"2023-10-16T11:01:22.603723+0000","last_active":"2023-10-16T11:01:24.604197+0000","last_peered":"2023-10-16T11:01:24.604197+0000","last_clean":"2023-10-16T11:01:24.604197+0000","last_became_active":"2023-10-16T11:01:22.599707+0000","last_became_peered":"2023-10-16T11:01:22.599707+0000","last_unstale":"2023-10-16T11:01:24.604197+0000","last_undegraded":"2023-10-16T11:01:24.604197+0000","last_fullsized":"2023-10-16T11:01:24.604197+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:31.883270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605975+0000","last_change":"2023-10-16T11:01:23.628153+0000","last_active":"2023-10-16T11:01:24.605975+0000","last_peered":"2023-10-16T11:01:24.605975+0000","last_clean":"2023-10-16T11:01:24.605975+0000","last_became_active":"2023-10-16T11:01:23.627737+0000","last_became_peered":"2023-10-16T11:01:23.627737+0000","last_unstale":"2023-10-16T11:01:24.605975+0000","last_undegraded":"2023-10-16T11:01:24.605975+0000","last_fullsized":"2023-10-16T11:01:24.605975+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:06:20.110705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"180'6","reported_seq":18,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605840+0000","last_change":"2023-10-16T11:01:22.599787+0000","last_active":"2023-10-16T11:01:24.605840+0000","last_peered":"2023-10-16T11:01:24.605840+0000","last_clean":"2023-10-16T11:01:24.605840+0000","last_became_active":"2023-10-16T11:01:22.599464+0000","last_became_peered":"2023-10-16T11:01:22.599464+0000","last_unstale":"2023-10-16T11:01:24.605840+0000","last_undegraded":"2023-10-16T11:01:24.605840+0000","last_fullsized":"2023-10-16T11:01:24.605840+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:24:54.551737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014915+0000","last_change":"2023-10-16T11:01:23.610843+0000","last_active":"2023-10-16T11:01:25.014915+0000","last_peered":"2023-10-16T11:01:25.014915+0000","last_clean":"2023-10-16T11:01:25.014915+0000","last_became_active":"2023-10-16T11:01:23.610402+0000","last_became_peered":"2023-10-16T11:01:23.610402+0000","last_unstale":"2023-10-16T11:01:25.014915+0000","last_undegraded":"2023-10-16T11:01:25.014915+0000","last_fullsized":"2023-10-16T11:01:25.014915+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:01.858571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013696+0000","last_change":"2023-10-16T11:01:22.604825+0000","last_active":"2023-10-16T11:01:25.013696+0000","last_peered":"2023-10-16T11:01:25.013696+0000","last_clean":"2023-10-16T11:01:25.013696+0000","last_became_active":"2023-10-16T11:01:22.604420+0000","last_became_peered":"2023-10-16T11:01:22.604420+0000","last_unstale":"2023-10-16T11:01:25.013696+0000","last_undegraded":"2023-10-16T11:01:25.013696+0000","last_fullsized":"2023-10-16T11:01:25.013696+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:13:55.406467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014347+0000","last_change":"2023-10-16T11:01:23.626921+0000","last_active":"2023-10-16T11:01:25.014347+0000","last_peered":"2023-10-16T11:01:25.014347+0000","last_clean":"2023-10-16T11:01:25.014347+0000","last_became_active":"2023-10-16T11:01:23.626600+0000","last_became_peered":"2023-10-16T11:01:23.626600+0000","last_unstale":"2023-10-16T11:01:25.014347+0000","last_undegraded":"2023-10-16T11:01:25.014347+0000","last_fullsized":"2023-10-16T11:01:25.014347+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:01.374062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013439+0000","last_change":"2023-10-16T11:01:22.592799+0000","last_active":"2023-10-16T11:01:25.013439+0000","last_peered":"2023-10-16T11:01:25.013439+0000","last_clean":"2023-10-16T11:01:25.013439+0000","last_became_active":"2023-10-16T11:01:22.592536+0000","last_became_peered":"2023-10-16T11:01:22.592536+0000","last_unstale":"2023-10-16T11:01:25.013439+0000","last_undegraded":"2023-10-16T11:01:25.013439+0000","last_fullsized":"2023-10-16T11:01:25.013439+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:22.757609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013628+0000","last_change":"2023-10-16T11:01:23.624279+0000","last_active":"2023-10-16T11:01:25.013628+0000","last_peered":"2023-10-16T11:01:25.013628+0000","last_clean":"2023-10-16T11:01:25.013628+0000","last_became_active":"2023-10-16T11:01:23.624016+0000","last_became_peered":"2023-10-16T11:01:23.624016+0000","last_unstale":"2023-10-16T11:01:25.013628+0000","last_undegraded":"2023-10-16T11:01:25.013628+0000","last_fullsized":"2023-10-16T11:01:25.013628+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:27:23.847285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013401+0000","last_change":"2023-10-16T11:01:22.605602+0000","last_active":"2023-10-16T11:01:25.013401+0000","last_peered":"2023-10-16T11:01:25.013401+0000","last_clean":"2023-10-16T11:01:25.013401+0000","last_became_active":"2023-10-16T11:01:22.605319+0000","last_became_peered":"2023-10-16T11:01:22.605319+0000","last_unstale":"2023-10-16T11:01:25.013401+0000","last_undegraded":"2023-10-16T11:01:25.013401+0000","last_fullsized":"2023-10-16T11:01:25.013401+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:11.968377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604138+0000","last_change":"2023-10-16T11:01:23.626390+0000","last_active":"2023-10-16T11:01:24.604138+0000","last_peered":"2023-10-16T11:01:24.604138+0000","last_clean":"2023-10-16T11:01:24.604138+0000","last_became_active":"2023-10-16T11:01:23.626105+0000","last_became_peered":"2023-10-16T11:01:23.626105+0000","last_unstale":"2023-10-16T11:01:24.604138+0000","last_undegraded":"2023-10-16T11:01:24.604138+0000","last_fullsized":"2023-10-16T11:01:24.604138+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:06.459351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"180'8","reported_seq":20,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605475+0000","last_change":"2023-10-16T11:01:22.600693+0000","last_active":"2023-10-16T11:01:24.605475+0000","last_peered":"2023-10-16T11:01:24.605475+0000","last_clean":"2023-10-16T11:01:24.605475+0000","last_became_active":"2023-10-16T11:01:22.600406+0000","last_became_peered":"2023-10-16T11:01:22.600406+0000","last_unstale":"2023-10-16T11:01:24.605475+0000","last_undegraded":"2023-10-16T11:01:24.605475+0000","last_fullsized":"2023-10-16T11:01:24.605475+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:26.525834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013948+0000","last_change":"2023-10-16T11:01:23.622416+0000","last_active":"2023-10-16T11:01:25.013948+0000","last_peered":"2023-10-16T11:01:25.013948+0000","last_clean":"2023-10-16T11:01:25.013948+0000","last_became_active":"2023-10-16T11:01:23.622062+0000","last_became_peered":"2023-10-16T11:01:23.622062+0000","last_unstale":"2023-10-16T11:01:25.013948+0000","last_undegraded":"2023-10-16T11:01:25.013948+0000","last_fullsized":"2023-10-16T11:01:25.013948+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:32:33.243686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604836+0000","last_change":"2023-10-16T11:01:22.606146+0000","last_active":"2023-10-16T11:01:24.604836+0000","last_peered":"2023-10-16T11:01:24.604836+0000","last_clean":"2023-10-16T11:01:24.604836+0000","last_became_active":"2023-10-16T11:01:22.605818+0000","last_became_peered":"2023-10-16T11:01:22.605818+0000","last_unstale":"2023-10-16T11:01:24.604836+0000","last_undegraded":"2023-10-16T11:01:24.604836+0000","last_fullsized":"2023-10-16T11:01:24.604836+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:11:40.275395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013577+0000","last_change":"2023-10-16T11:01:23.616809+0000","last_active":"2023-10-16T11:01:25.013577+0000","last_peered":"2023-10-16T11:01:25.013577+0000","last_clean":"2023-10-16T11:01:25.013577+0000","last_became_active":"2023-10-16T11:01:23.616541+0000","last_became_peered":"2023-10-16T11:01:23.616541+0000","last_unstale":"2023-10-16T11:01:25.013577+0000","last_undegraded":"2023-10-16T11:01:25.013577+0000","last_fullsized":"2023-10-16T11:01:25.013577+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:21:08.250929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604479+0000","last_change":"2023-10-16T11:01:22.595216+0000","last_active":"2023-10-16T11:01:24.604479+0000","last_peered":"2023-10-16T11:01:24.604479+0000","last_clean":"2023-10-16T11:01:24.604479+0000","last_became_active":"2023-10-16T11:01:22.594916+0000","last_became_peered":"2023-10-16T11:01:22.594916+0000","last_unstale":"2023-10-16T11:01:24.604479+0000","last_undegraded":"2023-10-16T11:01:24.604479+0000","last_fullsized":"2023-10-16T11:01:24.604479+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:05.136608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014107+0000","last_change":"2023-10-16T11:01:23.608430+0000","last_active":"2023-10-16T11:01:25.014107+0000","last_peered":"2023-10-16T11:01:25.014107+0000","last_clean":"2023-10-16T11:01:25.014107+0000","last_became_active":"2023-10-16T11:01:23.608128+0000","last_became_peered":"2023-10-16T11:01:23.608128+0000","last_unstale":"2023-10-16T11:01:25.014107+0000","last_undegraded":"2023-10-16T11:01:25.014107+0000","last_fullsized":"2023-10-16T11:01:25.014107+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:36:44.748533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"180'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013777+0000","last_change":"2023-10-16T11:01:22.600535+0000","last_active":"2023-10-16T11:01:25.013777+0000","last_peered":"2023-10-16T11:01:25.013777+0000","last_clean":"2023-10-16T11:01:25.013777+0000","last_became_active":"2023-10-16T11:01:22.600270+0000","last_became_peered":"2023-10-16T11:01:22.600270+0000","last_unstale":"2023-10-16T11:01:25.013777+0000","last_undegraded":"2023-10-16T11:01:25.013777+0000","last_fullsized":"2023-10-16T11:01:25.013777+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:21:07.243413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013936+0000","last_change":"2023-10-16T11:01:23.611319+0000","last_active":"2023-10-16T11:01:25.013936+0000","last_peered":"2023-10-16T11:01:25.013936+0000","last_clean":"2023-10-16T11:01:25.013936+0000","last_became_active":"2023-10-16T11:01:23.610879+0000","last_became_peered":"2023-10-16T11:01:23.610879+0000","last_unstale":"2023-10-16T11:01:25.013936+0000","last_undegraded":"2023-10-16T11:01:25.013936+0000","last_fullsized":"2023-10-16T11:01:25.013936+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:41.672049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014161+0000","last_change":"2023-10-16T11:01:22.607281+0000","last_active":"2023-10-16T11:01:25.014161+0000","last_peered":"2023-10-16T11:01:25.014161+0000","last_clean":"2023-10-16T11:01:25.014161+0000","last_became_active":"2023-10-16T11:01:22.606936+0000","last_became_peered":"2023-10-16T11:01:22.606936+0000","last_unstale":"2023-10-16T11:01:25.014161+0000","last_undegraded":"2023-10-16T11:01:25.014161+0000","last_fullsized":"2023-10-16T11:01:25.014161+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:06.782631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604214+0000","last_change":"2023-10-16T11:01:23.626330+0000","last_active":"2023-10-16T11:01:24.604214+0000","last_peered":"2023-10-16T11:01:24.604214+0000","last_clean":"2023-10-16T11:01:24.604214+0000","last_became_active":"2023-10-16T11:01:23.626057+0000","last_became_peered":"2023-10-16T11:01:23.626057+0000","last_unstale":"2023-10-16T11:01:24.604214+0000","last_undegraded":"2023-10-16T11:01:24.604214+0000","last_fullsized":"2023-10-16T11:01:24.604214+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:56:57.344075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013783+0000","last_change":"2023-10-16T11:01:22.603347+0000","last_active":"2023-10-16T11:01:25.013783+0000","last_peered":"2023-10-16T11:01:25.013783+0000","last_clean":"2023-10-16T11:01:25.013783+0000","last_became_active":"2023-10-16T11:01:22.603064+0000","last_became_peered":"2023-10-16T11:01:22.603064+0000","last_unstale":"2023-10-16T11:01:25.013783+0000","last_undegraded":"2023-10-16T11:01:25.013783+0000","last_fullsized":"2023-10-16T11:01:25.013783+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:50:41.596545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014441+0000","last_change":"2023-10-16T11:01:23.612337+0000","last_active":"2023-10-16T11:01:25.014441+0000","last_peered":"2023-10-16T11:01:25.014441+0000","last_clean":"2023-10-16T11:01:25.014441+0000","last_became_active":"2023-10-16T11:01:23.612071+0000","last_became_peered":"2023-10-16T11:01:23.612071+0000","last_unstale":"2023-10-16T11:01:25.014441+0000","last_undegraded":"2023-10-16T11:01:25.014441+0000","last_fullsized":"2023-10-16T11:01:25.014441+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:24.093242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014062+0000","last_change":"2023-10-16T11:01:22.600363+0000","last_active":"2023-10-16T11:01:25.014062+0000","last_peered":"2023-10-16T11:01:25.014062+0000","last_clean":"2023-10-16T11:01:25.014062+0000","last_became_active":"2023-10-16T11:01:22.600048+0000","last_became_peered":"2023-10-16T11:01:22.600048+0000","last_unstale":"2023-10-16T11:01:25.014062+0000","last_undegraded":"2023-10-16T11:01:25.014062+0000","last_fullsized":"2023-10-16T11:01:25.014062+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:17:33.657858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013504+0000","last_change":"2023-10-16T11:01:23.616293+0000","last_active":"2023-10-16T11:01:25.013504+0000","last_peered":"2023-10-16T11:01:25.013504+0000","last_clean":"2023-10-16T11:01:25.013504+0000","last_became_active":"2023-10-16T11:01:23.616040+0000","last_became_peered":"2023-10-16T11:01:23.616040+0000","last_unstale":"2023-10-16T11:01:25.013504+0000","last_undegraded":"2023-10-16T11:01:25.013504+0000","last_fullsized":"2023-10-16T11:01:25.013504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:21.969423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013551+0000","last_change":"2023-10-16T11:01:22.601051+0000","last_active":"2023-10-16T11:01:25.013551+0000","last_peered":"2023-10-16T11:01:25.013551+0000","last_clean":"2023-10-16T11:01:25.013551+0000","last_became_active":"2023-10-16T11:01:22.600514+0000","last_became_peered":"2023-10-16T11:01:22.600514+0000","last_unstale":"2023-10-16T11:01:25.013551+0000","last_undegraded":"2023-10-16T11:01:25.013551+0000","last_fullsized":"2023-10-16T11:01:25.013551+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:59:16.320730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603971+0000","last_change":"2023-10-16T11:01:23.619566+0000","last_active":"2023-10-16T11:01:24.603971+0000","last_peered":"2023-10-16T11:01:24.603971+0000","last_clean":"2023-10-16T11:01:24.603971+0000","last_became_active":"2023-10-16T11:01:23.619255+0000","last_became_peered":"2023-10-16T11:01:23.619255+0000","last_unstale":"2023-10-16T11:01:24.603971+0000","last_undegraded":"2023-10-16T11:01:24.603971+0000","last_fullsized":"2023-10-16T11:01:24.603971+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:30.728808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013604+0000","last_change":"2023-10-16T11:01:22.602453+0000","last_active":"2023-10-16T11:01:25.013604+0000","last_peered":"2023-10-16T11:01:25.013604+0000","last_clean":"2023-10-16T11:01:25.013604+0000","last_became_active":"2023-10-16T11:01:22.602148+0000","last_became_peered":"2023-10-16T11:01:22.602148+0000","last_unstale":"2023-10-16T11:01:25.013604+0000","last_undegraded":"2023-10-16T11:01:25.013604+0000","last_fullsized":"2023-10-16T11:01:25.013604+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:00.851055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013482+0000","last_change":"2023-10-16T11:01:23.617273+0000","last_active":"2023-10-16T11:01:25.013482+0000","last_peered":"2023-10-16T11:01:25.013482+0000","last_clean":"2023-10-16T11:01:25.013482+0000","last_became_active":"2023-10-16T11:01:23.616391+0000","last_became_peered":"2023-10-16T11:01:23.616391+0000","last_unstale":"2023-10-16T11:01:25.013482+0000","last_undegraded":"2023-10-16T11:01:25.013482+0000","last_fullsized":"2023-10-16T11:01:25.013482+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:27.879098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"180'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013670+0000","last_change":"2023-10-16T11:01:22.605983+0000","last_active":"2023-10-16T11:01:25.013670+0000","last_peered":"2023-10-16T11:01:25.013670+0000","last_clean":"2023-10-16T11:01:25.013670+0000","last_became_active":"2023-10-16T11:01:22.605712+0000","last_became_peered":"2023-10-16T11:01:22.605712+0000","last_unstale":"2023-10-16T11:01:25.013670+0000","last_undegraded":"2023-10-16T11:01:25.013670+0000","last_fullsized":"2023-10-16T11:01:25.013670+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:36:02.895039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014099+0000","last_change":"2023-10-16T11:01:23.616247+0000","last_active":"2023-10-16T11:01:25.014099+0000","last_peered":"2023-10-16T11:01:25.014099+0000","last_clean":"2023-10-16T11:01:25.014099+0000","last_became_active":"2023-10-16T11:01:23.615981+0000","last_became_peered":"2023-10-16T11:01:23.615981+0000","last_unstale":"2023-10-16T11:01:25.014099+0000","last_undegraded":"2023-10-16T11:01:25.014099+0000","last_fullsized":"2023-10-16T11:01:25.014099+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:03:13.273710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604511+0000","last_change":"2023-10-16T11:01:22.600488+0000","last_active":"2023-10-16T11:01:24.604511+0000","last_peered":"2023-10-16T11:01:24.604511+0000","last_clean":"2023-10-16T11:01:24.604511+0000","last_became_active":"2023-10-16T11:01:22.600217+0000","last_became_peered":"2023-10-16T11:01:22.600217+0000","last_unstale":"2023-10-16T11:01:24.604511+0000","last_undegraded":"2023-10-16T11:01:24.604511+0000","last_fullsized":"2023-10-16T11:01:24.604511+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:29.721291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.603932+0000","last_change":"2023-10-16T11:01:23.621720+0000","last_active":"2023-10-16T11:01:24.603932+0000","last_peered":"2023-10-16T11:01:24.603932+0000","last_clean":"2023-10-16T11:01:24.603932+0000","last_became_active":"2023-10-16T11:01:23.621318+0000","last_became_peered":"2023-10-16T11:01:23.621318+0000","last_unstale":"2023-10-16T11:01:24.603932+0000","last_undegraded":"2023-10-16T11:01:24.603932+0000","last_fullsized":"2023-10-16T11:01:24.603932+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:06.144124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013984+0000","last_change":"2023-10-16T11:01:22.601660+0000","last_active":"2023-10-16T11:01:25.013984+0000","last_peered":"2023-10-16T11:01:25.013984+0000","last_clean":"2023-10-16T11:01:25.013984+0000","last_became_active":"2023-10-16T11:01:22.601392+0000","last_became_peered":"2023-10-16T11:01:22.601392+0000","last_unstale":"2023-10-16T11:01:25.013984+0000","last_undegraded":"2023-10-16T11:01:25.013984+0000","last_fullsized":"2023-10-16T11:01:25.013984+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:23.085725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013650+0000","last_change":"2023-10-16T11:01:23.615073+0000","last_active":"2023-10-16T11:01:25.013650+0000","last_peered":"2023-10-16T11:01:25.013650+0000","last_clean":"2023-10-16T11:01:25.013650+0000","last_became_active":"2023-10-16T11:01:23.614778+0000","last_became_peered":"2023-10-16T11:01:23.614778+0000","last_unstale":"2023-10-16T11:01:25.013650+0000","last_undegraded":"2023-10-16T11:01:25.013650+0000","last_fullsized":"2023-10-16T11:01:25.013650+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:37:07.790148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.014752+0000","last_change":"2023-10-16T11:01:22.595898+0000","last_active":"2023-10-16T11:01:25.014752+0000","last_peered":"2023-10-16T11:01:25.014752+0000","last_clean":"2023-10-16T11:01:25.014752+0000","last_became_active":"2023-10-16T11:01:22.595644+0000","last_became_peered":"2023-10-16T11:01:22.595644+0000","last_unstale":"2023-10-16T11:01:25.014752+0000","last_undegraded":"2023-10-16T11:01:25.014752+0000","last_fullsized":"2023-10-16T11:01:25.014752+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:03.815129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013351+0000","last_change":"2023-10-16T11:01:23.616884+0000","last_active":"2023-10-16T11:01:25.013351+0000","last_peered":"2023-10-16T11:01:25.013351+0000","last_clean":"2023-10-16T11:01:25.013351+0000","last_became_active":"2023-10-16T11:01:23.616513+0000","last_became_peered":"2023-10-16T11:01:23.616513+0000","last_unstale":"2023-10-16T11:01:25.013351+0000","last_undegraded":"2023-10-16T11:01:25.013351+0000","last_fullsized":"2023-10-16T11:01:25.013351+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:17:34.665375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605708+0000","last_change":"2023-10-16T11:01:22.599754+0000","last_active":"2023-10-16T11:01:24.605708+0000","last_peered":"2023-10-16T11:01:24.605708+0000","last_clean":"2023-10-16T11:01:24.605708+0000","last_became_active":"2023-10-16T11:01:22.599412+0000","last_became_peered":"2023-10-16T11:01:22.599412+0000","last_unstale":"2023-10-16T11:01:24.605708+0000","last_undegraded":"2023-10-16T11:01:24.605708+0000","last_fullsized":"2023-10-16T11:01:24.605708+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:03:45.074422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605541+0000","last_change":"2023-10-16T11:01:23.610939+0000","last_active":"2023-10-16T11:01:24.605541+0000","last_peered":"2023-10-16T11:01:24.605541+0000","last_clean":"2023-10-16T11:01:24.605541+0000","last_became_active":"2023-10-16T11:01:23.610248+0000","last_became_peered":"2023-10-16T11:01:23.610248+0000","last_unstale":"2023-10-16T11:01:24.605541+0000","last_undegraded":"2023-10-16T11:01:24.605541+0000","last_fullsized":"2023-10-16T11:01:24.605541+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:44:29.210679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013273+0000","last_change":"2023-10-16T11:01:22.606342+0000","last_active":"2023-10-16T11:01:25.013273+0000","last_peered":"2023-10-16T11:01:25.013273+0000","last_clean":"2023-10-16T11:01:25.013273+0000","last_became_active":"2023-10-16T11:01:22.605885+0000","last_became_peered":"2023-10-16T11:01:22.605885+0000","last_unstale":"2023-10-16T11:01:25.013273+0000","last_undegraded":"2023-10-16T11:01:25.013273+0000","last_fullsized":"2023-10-16T11:01:25.013273+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:14:50.760465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013096+0000","last_change":"2023-10-16T11:01:23.613625+0000","last_active":"2023-10-16T11:01:25.013096+0000","last_peered":"2023-10-16T11:01:25.013096+0000","last_clean":"2023-10-16T11:01:25.013096+0000","last_became_active":"2023-10-16T11:01:23.612643+0000","last_became_peered":"2023-10-16T11:01:23.612643+0000","last_unstale":"2023-10-16T11:01:25.013096+0000","last_undegraded":"2023-10-16T11:01:25.013096+0000","last_fullsized":"2023-10-16T11:01:25.013096+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:55.193152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013765+0000","last_change":"2023-10-16T11:01:22.605395+0000","last_active":"2023-10-16T11:01:25.013765+0000","last_peered":"2023-10-16T11:01:25.013765+0000","last_clean":"2023-10-16T11:01:25.013765+0000","last_became_active":"2023-10-16T11:01:22.605107+0000","last_became_peered":"2023-10-16T11:01:22.605107+0000","last_unstale":"2023-10-16T11:01:25.013765+0000","last_undegraded":"2023-10-16T11:01:25.013765+0000","last_fullsized":"2023-10-16T11:01:25.013765+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:29:00.366546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.604150+0000","last_change":"2023-10-16T11:01:23.623068+0000","last_active":"2023-10-16T11:01:24.604150+0000","last_peered":"2023-10-16T11:01:24.604150+0000","last_clean":"2023-10-16T11:01:24.604150+0000","last_became_active":"2023-10-16T11:01:23.622806+0000","last_became_peered":"2023-10-16T11:01:23.622806+0000","last_unstale":"2023-10-16T11:01:24.604150+0000","last_undegraded":"2023-10-16T11:01:24.604150+0000","last_fullsized":"2023-10-16T11:01:24.604150+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:42:29.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605614+0000","last_change":"2023-10-16T11:01:22.602845+0000","last_active":"2023-10-16T11:01:24.605614+0000","last_peered":"2023-10-16T11:01:24.605614+0000","last_clean":"2023-10-16T11:01:24.605614+0000","last_became_active":"2023-10-16T11:01:22.602558+0000","last_became_peered":"2023-10-16T11:01:22.602558+0000","last_unstale":"2023-10-16T11:01:24.605614+0000","last_undegraded":"2023-10-16T11:01:24.605614+0000","last_fullsized":"2023-10-16T11:01:24.605614+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:23.573151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:25.013423+0000","last_change":"2023-10-16T11:01:23.617290+0000","last_active":"2023-10-16T11:01:25.013423+0000","last_peered":"2023-10-16T11:01:25.013423+0000","last_clean":"2023-10-16T11:01:25.013423+0000","last_became_active":"2023-10-16T11:01:23.616817+0000","last_became_peered":"2023-10-16T11:01:23.616817+0000","last_unstale":"2023-10-16T11:01:25.013423+0000","last_undegraded":"2023-10-16T11:01:25.013423+0000","last_fullsized":"2023-10-16T11:01:25.013423+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:22.581974+0000","last_clean_scrub_stamp":"2023-10-16T11:01:22.581974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:05.119821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-16T11:01:24.605998+0000","last_change":"2023-10-16T11:01:22.603190+0000","last_active":"2023-10-16T11:01:24.605998+0000","last_peered":"2023-10-16T11:01:24.605998+0000","last_clean":"2023-10-16T11:01:24.605998+0000","last_became_active":"2023-10-16T11:01:22.602894+0000","last_became_peered":"2023-10-16T11:01:22.602894+0000","last_unstale":"2023-10-16T11:01:24.605998+0000","last_undegraded":"2023-10-16T11:01:24.605998+0000","last_fullsized":"2023-10-16T11:01:24.605998+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:01:21.574458+0000","last_clean_scrub_stamp":"2023-10-16T11:01:21.574458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:18:00.055026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"171'293","reported_seq":973,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-16T11:01:23.608804+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:01:23.608804+0000","last_peered":"2023-10-16T11:01:23.608804+0000","last_clean":"2023-10-16T11:01:23.608804+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:01:23.608804+0000","last_undegraded":"2023-10-16T11:01:23.608804+0000","last_fullsized":"2023-10-16T11:01:23.608804+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543067,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1729064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76700000000000002,"5min":0.78600000000000003,"15min":0.83499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.754,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.32000000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.81399999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.46999999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.82499999999999996,"15min":0.89200000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.81100000000000005,"15min":0.88100000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.69799999999999995,"5min":0.81399999999999995,"15min":0.83899999999999997},"last":0.62}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.85199999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.40100000000000002,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.73599999999999999,"15min":0.86699999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.876},"last":0.38400000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.308},"max":{"1min":0.71899999999999997,"5min":0.79900000000000004,"15min":0.85999999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.89300000000000002},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.441,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.92400000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48199999999999998}]}]},{"osd":6,"up_from":14,"seq":60129543066,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37540,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334300,"statfs":{"total":96636764160,"available":96598323200,"internally_reserved":0,"allocated":2252800,"data_stored":1136364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60099999999999998,"15min":0.60999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32400000000000001,"15min":0.29299999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.32600000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.495},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.60599999999999998,"15min":0.624},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.378},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.627},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48199999999999998,"15min":0.49199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.91100000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.432}]},{"osd":5,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.33700000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.90200000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52700000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.36699999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38,"15min":0.34599999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.60499999999999998}]}]},{"osd":1,"up_from":14,"seq":60129543066,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2265088,"data_stored":1143172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.39400000000000002,"15min":0.39600000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.221},{"interface":"front","average":{"1min":0.374,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.44,"5min":0.57299999999999995,"15min":0.68000000000000005},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46000000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.308,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59499999999999997,"5min":0.68899999999999995,"15min":0.745},"last":0.34999999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32100000000000001},"max":{"1min":0.55400000000000005,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45400000000000001,"15min":0.46200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.29899999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.73399999999999999},"last":0.32100000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.622,"15min":0.64500000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33000000000000002},"max":{"1min":0.78900000000000003,"5min":1.004,"15min":1.044},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.94399999999999995,"15min":1.0409999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84999999999999998,"5min":0.97599999999999998,"15min":1.038},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":1.038,"15min":1.038},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67400000000000004,"15min":0.68899999999999995},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78800000000000003,"5min":1.0169999999999999,"15min":1.105},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66700000000000004,"15min":0.68000000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.41399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94699999999999995,"15min":1.091},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.81100000000000005,"5min":1.0389999999999999,"15min":1.1299999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.67300000000000004,"15min":0.69299999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83299999999999996,"5min":1.034,"15min":1.034},"last":0.63100000000000001}]}]},{"osd":0,"up_from":14,"seq":60129543067,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56180,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315660,"statfs":{"total":96636764160,"available":96579235840,"internally_reserved":0,"allocated":2854912,"data_stored":1733506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40000000000000002,"15min":0.39700000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.22900000000000001},"max":{"1min":0.54500000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.218},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.69199999999999995},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.27500000000000002,"5min":0.26400000000000001,"15min":0.23599999999999999},"max":{"1min":0.61399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44700000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29599999999999999},"max":{"1min":0.64200000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.33400000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.59299999999999997,"5min":0.66400000000000003,"15min":0.72599999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66000000000000003,"5min":0.71599999999999997,"15min":0.74099999999999999},"last":0.438}]},{"osd":4,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.35899999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.39700000000000002},"max":{"1min":0.79000000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69199999999999995,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.999,"15min":1.0820000000000001},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.67000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.48399999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.98699999999999999},"last":0.46600000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:00:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.995,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.496}]}]},{"osd":2,"up_from":14,"seq":60129543066,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59764,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312076,"statfs":{"total":96636764160,"available":96575565824,"internally_reserved":0,"allocated":2265088,"data_stored":1143228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.39700000000000002,"15min":0.38500000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.438,"15min":0.42599999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.78900000000000003},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46100000000000002,"15min":0.44900000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.26200000000000001},"max":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.67900000000000005},"last":0.441}]},{"osd":3,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.437,"15min":0.433},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.624,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.502,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.751},"last":0.49099999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.97999999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.42599999999999999},"max":{"1min":0.88300000000000001,"5min":0.98699999999999999,"15min":1.022},"last":0.68500000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67800000000000005,"15min":0.66500000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86199999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.749},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.68400000000000005,"15min":0.66900000000000004},"min":{"1min":0.45700000000000002,"5min":0.40999999999999998,"15min":0.376},"max":{"1min":0.93999999999999995,"5min":1.026,"15min":1.026},"last":0.64400000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68799999999999994,"15min":0.67000000000000004},"min":{"1min":0.47299999999999998,"5min":0.437,"15min":0.42099999999999999},"max":{"1min":0.89800000000000002,"5min":1.159,"15min":1.159},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68999999999999995,"15min":0.67900000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.437},"max":{"1min":0.90200000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:00:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":1.121,"15min":1.121},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93200000000000005,"5min":1.095,"15min":1.095},"last":0.71399999999999997}]}]},{"osd":3,"up_from":14,"seq":60129543067,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59764,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312076,"statfs":{"total":96636764160,"available":96575565824,"internally_reserved":0,"allocated":2265088,"data_stored":1143172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58915272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55500000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.441},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59399999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.318}]},{"osd":1,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.40600000000000003,"15min":0.41399999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52300000000000002,"5min":0.621,"15min":0.76400000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.441},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.66600000000000004},"last":0.42399999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28499999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.64700000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.28499999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.96999999999999997},"last":0.33600000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41299999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64400000000000002,"15min":0.66200000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86399999999999999,"5min":0.92400000000000004,"15min":0.98499999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81399999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.0029999999999999},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":1.0349999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.995},"last":0.499}]},{"osd":7,"last update":"Mon Oct 16 11:00:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95599999999999996,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68600000000000005,"15min":0.69299999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94099999999999995,"5min":0.96599999999999997,"15min":0.98399999999999999},"last":0.437}]}]},{"osd":4,"up_from":14,"seq":60129543066,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51456,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320384,"statfs":{"total":96636764160,"available":96584073216,"internally_reserved":0,"allocated":2277376,"data_stored":1144820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72999999999999998,"5min":0.83799999999999997,"15min":0.90100000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.32300000000000001},"max":{"1min":0.77500000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63200000000000001,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.88400000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":0.68799999999999994,"5min":0.77400000000000002,"15min":0.80800000000000005},"last":0.39700000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.621,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.40300000000000002,"15min":0.34200000000000003},"max":{"1min":0.72499999999999998,"5min":0.86199999999999999,"15min":1.077},"last":0.38},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41099999999999998},"max":{"1min":0.72099999999999997,"5min":0.81100000000000005,"15min":0.97899999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.84099999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.48299999999999998,"5min":0.432,"15min":0.41099999999999998},"max":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.61799999999999999,"5min":0.84099999999999997,"15min":0.90800000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65000000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44500000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.38400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.86899999999999999,"15min":1.03},"last":0.496},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71299999999999997,"5min":0.89100000000000001,"15min":0.96099999999999997},"last":0.58899999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.69099999999999995,"5min":0.88,"15min":0.88},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69299999999999995,"5min":0.83999999999999997,"15min":1.133},"last":0.57499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129543067,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37556,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334284,"statfs":{"total":96636764160,"available":96598306816,"internally_reserved":0,"allocated":2269184,"data_stored":1143742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.28799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.83599999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.78200000000000003,"5min":0.82699999999999996,"15min":0.876},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70699999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.32900000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80500000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61299999999999999,"15min":0.628},"min":{"1min":0.38700000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.624,"15min":0.63800000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.86099999999999999},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.502,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.32200000000000001,"15min":0.315},"max":{"1min":0.66200000000000003,"5min":0.753,"15min":0.78700000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78300000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58299999999999996,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.373,"15min":0.33000000000000002},"max":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.84899999999999998},"last":0.47799999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36099999999999999},"max":{"1min":0.75900000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:01:28.273 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:01:28.273 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:01:28.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.433+0000 7f1812132700 1 Processor -- start 2023-10-16T11:01:28.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.433+0000 7f1812132700 1 -- start start 2023-10-16T11:01:28.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f1812132700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f180c15da70 0x7f180c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f1812132700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f180c0ce6a0 0x7f180c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f1812132700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c072760 0x7f180c06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f1812132700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c0cef00 con 0x7f180c0ce6a0 2023-10-16T11:01:28.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f1812132700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c0cec20 con 0x7f180c15da70 2023-10-16T11:01:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f1812132700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c071c80 con 0x7f180c072760 2023-10-16T11:01:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f180c0ce6a0 0x7f180c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f180c0ce6a0 0x7f180c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52184/0 (socket says 172.21.15.181:52184) 2023-10-16T11:01:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 -- 172.21.15.181:0/778033604 learned_addr learned my addr 172.21.15.181:0/778033604 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 -- 172.21.15.181:0/778033604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c072760 msgr2=0x7f180c06ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180bfff700 1 --2- 172.21.15.181:0/778033604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c072760 0x7f180c06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 --2- 172.21.15.181:0/778033604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c072760 0x7f180c06ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 -- 172.21.15.181:0/778033604 >> v2:172.21.15.190:3300/0 conn(0x7f180c15da70 msgr2=0x7f180c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 --2- 172.21.15.181:0/778033604 >> v2:172.21.15.190:3300/0 conn(0x7f180c15da70 0x7f180c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.434+0000 7f180affd700 1 -- 172.21.15.181:0/778033604 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c071f60 con 0x7f180c0ce6a0 2023-10-16T11:01:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f180affd700 1 --2- 172.21.15.181:0/778033604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c0ce6a0 0x7f180c06a8a0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f180c0675e0 tx=0x7f17fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d650e647595a12 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f180a7fc700 1 -- 172.21.15.181:0/778033604 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00b870 con 0x7f180c0ce6a0 2023-10-16T11:01:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f180a7fc700 1 -- 172.21.15.181:0/778033604 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc00b9d0 con 0x7f180c0ce6a0 2023-10-16T11:01:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f180a7fc700 1 -- 172.21.15.181:0/778033604 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00bd60 con 0x7f180c0ce6a0 2023-10-16T11:01:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 -- 172.21.15.181:0/778033604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c0ce6a0 msgr2=0x7f180c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 --2- 172.21.15.181:0/778033604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c0ce6a0 0x7f180c06a8a0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f180c0675e0 tx=0x7f17fc009bd0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 -- 172.21.15.181:0/778033604 shutdown_connections 2023-10-16T11:01:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 --2- 172.21.15.181:0/778033604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c072760 0x7f180c06ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 --2- 172.21.15.181:0/778033604 >> v2:172.21.15.190:3300/0 conn(0x7f180c15da70 0x7f180c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 --2- 172.21.15.181:0/778033604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c0ce6a0 0x7f180c06a8a0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 -- 172.21.15.181:0/778033604 >> 172.21.15.181:0/778033604 conn(0x7f180c0cd010 msgr2=0x7f180c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 -- 172.21.15.181:0/778033604 shutdown_connections 2023-10-16T11:01:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.435+0000 7f1812132700 1 -- 172.21.15.181:0/778033604 wait complete. 2023-10-16T11:01:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.437+0000 7f1812132700 1 Processor -- start 2023-10-16T11:01:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.437+0000 7f1812132700 1 -- start start 2023-10-16T11:01:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.437+0000 7f1812132700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c072760 0x7f180c0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.437+0000 7f1812132700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0ce6a0 0x7f180c0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.437+0000 7f1812132700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c15da70 0x7f180c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.437+0000 7f1812132700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0ad2d0 con 0x7f180c072760 2023-10-16T11:01:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.437+0000 7f1812132700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0ad5b0 con 0x7f180c0ce6a0 2023-10-16T11:01:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f1812132700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0ad890 con 0x7f180c15da70 2023-10-16T11:01:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c072760 0x7f180c0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c072760 0x7f180c0b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52188/0 (socket says 172.21.15.181:52188) 2023-10-16T11:01:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 -- 172.21.15.181:0/587073871 learned_addr learned my addr 172.21.15.181:0/587073871 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180bfff700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c15da70 0x7f180c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180affd700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0ce6a0 0x7f180c0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 -- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c15da70 msgr2=0x7f180c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c15da70 0x7f180c159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 -- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0ce6a0 msgr2=0x7f180c0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0ce6a0 0x7f180c0ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 -- 172.21.15.181:0/587073871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc009550 con 0x7f180c072760 2023-10-16T11:01:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180affd700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0ce6a0 0x7f180c0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f180b7fe700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c072760 0x7f180c0b1c90 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f17f400b530 tx=0x7f17f400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f1803fff700 1 -- 172.21.15.181:0/587073871 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4011830 con 0x7f180c072760 2023-10-16T11:01:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f1803fff700 1 -- 172.21.15.181:0/587073871 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f40119b0 con 0x7f180c072760 2023-10-16T11:01:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c0adb70 con 0x7f180c072760 2023-10-16T11:01:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f1803fff700 1 -- 172.21.15.181:0/587073871 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4011d60 con 0x7f180c072760 2023-10-16T11:01:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.438+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c0b5a50 con 0x7f180c072760 2023-10-16T11:01:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.439+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c06f6c0 con 0x7f180c072760 2023-10-16T11:01:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.440+0000 7f1803fff700 1 -- 172.21.15.181:0/587073871 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f17f401a400 con 0x7f180c072760 2023-10-16T11:01:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.440+0000 7f1803fff700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f17ec06b020 0x7f17ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.440+0000 7f1803fff700 1 -- 172.21.15.181:0/587073871 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f17f4089a10 con 0x7f180c072760 2023-10-16T11:01:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.440+0000 7f180affd700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f17ec06b020 0x7f17ec06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.441+0000 7f180affd700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f17ec06b020 0x7f17ec06d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c7360 tx=0x7f17fc008030 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.445+0000 7f1803fff700 1 -- 172.21.15.181:0/587073871 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f17f40547c0 con 0x7f180c072760 2023-10-16T11:01:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.714+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f180c0afbd0 con 0x7f180c072760 2023-10-16T11:01:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.716+0000 7f1803fff700 1 -- 172.21.15.181:0/587073871 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v44) v1 ==== 190+0+147 (secure 0 0 0) 0x7f17f4015020 con 0x7f180c072760 2023-10-16T11:01:28.718 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T11:01:28.718 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T11:01:28.718 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T11:01:28.718 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T11:01:28.718 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:01:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f17ec06b020 msgr2=0x7f17ec06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f17ec06b020 0x7f17ec06d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c7360 tx=0x7f17fc008030 comp rx=0 tx=0).stop 2023-10-16T11:01:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c072760 msgr2=0x7f180c0b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c072760 0x7f180c0b1c90 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f17f400b530 tx=0x7f17f400b8f0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 shutdown_connections 2023-10-16T11:01:28.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f17ec06b020 0x7f17ec06d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f180c15da70 0x7f180c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f180c072760 0x7f180c0b1c90 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 --2- 172.21.15.181:0/587073871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f180c0ce6a0 0x7f180c0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 >> 172.21.15.181:0/587073871 conn(0x7f180c0cd010 msgr2=0x7f180c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 shutdown_connections 2023-10-16T11:01:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.718+0000 7f1812132700 1 -- 172.21.15.181:0/587073871 wait complete. 2023-10-16T11:01:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T11:01:28.732 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:01:28.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.851+0000 7f59941d1700 1 Processor -- start 2023-10-16T11:01:28.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f59941d1700 1 -- start start 2023-10-16T11:01:28.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f59941d1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c0cab40 0x7f598c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f59941d1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f598c15e7f0 0x7f598c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f59941d1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c0cc310 con 0x7f598c0b1ee0 2023-10-16T11:01:28.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f59941d1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c15e0f0 con 0x7f598c15e7f0 2023-10-16T11:01:28.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f59941d1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c0cf820 con 0x7f598c0cab40 2023-10-16T11:01:28.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f5991f6d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f598c0b1ee0 0x7f598c0b22c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35042/0 (socket says 172.21.15.181:35042) 2023-10-16T11:01:28.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.852+0000 7f5991f6d700 1 -- 172.21.15.181:0/2663455935 learned_addr learned my addr 172.21.15.181:0/2663455935 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:28.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599176c700 1 --2- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c0cab40 0x7f598c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599276e700 1 --2- 172.21.15.181:0/2663455935 >> v2:172.21.15.190:3300/0 conn(0x7f598c15e7f0 0x7f598c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f5990f6b700 1 -- 172.21.15.181:0/2663455935 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299887934 0 0) 0x7f598c0cc310 con 0x7f598c0b1ee0 2023-10-16T11:01:28.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f5990f6b700 1 -- 172.21.15.181:0/2663455935 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59780035b0 con 0x7f598c0b1ee0 2023-10-16T11:01:28.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599276e700 1 -- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c0cab40 msgr2=0x7f598c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:28.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599276e700 1 --2- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c0cab40 0x7f598c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599276e700 1 -- 172.21.15.181:0/2663455935 >> v1:172.21.15.181:6789/0 conn(0x7f598c0b1ee0 legacy=0x7f598c0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599276e700 1 -- 172.21.15.181:0/2663455935 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0cc590 con 0x7f598c15e7f0 2023-10-16T11:01:28.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599176c700 1 --2- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c0cab40 0x7f598c0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:28.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f599276e700 1 --2- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c15e7f0 0x7f598c06a8a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5984009d90 tx=0x7f598400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74025d8bbe8f6db6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f5990f6b700 1 -- 172.21.15.181:0/2663455935 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598400e070 con 0x7f598c15e7f0 2023-10-16T11:01:28.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f5990f6b700 1 -- 172.21.15.181:0/2663455935 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598400bb20 con 0x7f598c15e7f0 2023-10-16T11:01:28.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.853+0000 7f5990f6b700 1 -- 172.21.15.181:0/2663455935 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598400be90 con 0x7f598c15e7f0 2023-10-16T11:01:28.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 -- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c15e7f0 msgr2=0x7f598c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 --2- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c15e7f0 0x7f598c06a8a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5984009d90 tx=0x7f598400b330 comp rx=0 tx=0).stop 2023-10-16T11:01:28.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 -- 172.21.15.181:0/2663455935 shutdown_connections 2023-10-16T11:01:28.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 --2- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c15e7f0 0x7f598c06a8a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 --2- 172.21.15.181:0/2663455935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c0cab40 0x7f598c0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 -- 172.21.15.181:0/2663455935 >> 172.21.15.181:0/2663455935 conn(0x7f598c0d0460 msgr2=0x7f598c0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:28.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 -- 172.21.15.181:0/2663455935 shutdown_connections 2023-10-16T11:01:28.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.854+0000 7f59941d1700 1 -- 172.21.15.181:0/2663455935 wait complete. 2023-10-16T11:01:28.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.855+0000 7f59941d1700 1 Processor -- start 2023-10-16T11:01:28.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.855+0000 7f59941d1700 1 -- start start 2023-10-16T11:01:28.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f59941d1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c0b1ee0 0x7f598c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f59941d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f598c0cab40 0x7f598c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f59941d1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c15e7f0 0x7f598c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f59941d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c1f3230 con 0x7f598c0cab40 2023-10-16T11:01:28.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f59941d1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c0ca1b0 con 0x7f598c0b1ee0 2023-10-16T11:01:28.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f59941d1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c1f3440 con 0x7f598c15e7f0 2023-10-16T11:01:28.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f598c0cab40 0x7f598c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f598c0cab40 0x7f598c0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52204/0 (socket says 172.21.15.181:52204) 2023-10-16T11:01:28.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 -- 172.21.15.181:0/2876498440 learned_addr learned my addr 172.21.15.181:0/2876498440 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:28.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599276e700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c15e7f0 0x7f598c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f5991f6d700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c0b1ee0 0x7f598c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 -- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c15e7f0 msgr2=0x7f598c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c15e7f0 0x7f598c0b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 -- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c0b1ee0 msgr2=0x7f598c15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:28.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c0b1ee0 0x7f598c15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:28.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 -- 172.21.15.181:0/2876498440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59880094a0 con 0x7f598c0cab40 2023-10-16T11:01:28.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599276e700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c15e7f0 0x7f598c0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:28.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.856+0000 7f599176c700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f598c0cab40 0x7f598c0c9760 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f598800ebe0 tx=0x7f5988009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.857+0000 7f59827fc700 1 -- 172.21.15.181:0/2876498440 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800c910 con 0x7f598c0cab40 2023-10-16T11:01:28.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.857+0000 7f59827fc700 1 -- 172.21.15.181:0/2876498440 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598800ca70 con 0x7f598c0cab40 2023-10-16T11:01:28.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.857+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5984009a70 con 0x7f598c0cab40 2023-10-16T11:01:28.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.857+0000 7f59827fc700 1 -- 172.21.15.181:0/2876498440 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800cde0 con 0x7f598c0cab40 2023-10-16T11:01:28.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.857+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c1f3a00 con 0x7f598c0cab40 2023-10-16T11:01:28.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.858+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c067d20 con 0x7f598c0cab40 2023-10-16T11:01:28.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.858+0000 7f59827fc700 1 -- 172.21.15.181:0/2876498440 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5988013470 con 0x7f598c0cab40 2023-10-16T11:01:28.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.859+0000 7f59827fc700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f597006b070 0x7f597006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:28.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.859+0000 7f59827fc700 1 -- 172.21.15.181:0/2876498440 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f5988089960 con 0x7f598c0cab40 2023-10-16T11:01:28.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.859+0000 7f5991f6d700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f597006b070 0x7f597006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:28.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.859+0000 7f5991f6d700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f597006b070 0x7f597006d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f597c001360 tx=0x7f597c009040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:28.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:28.863+0000 7f59827fc700 1 -- 172.21.15.181:0/2876498440 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5988054880 con 0x7f598c0cab40 2023-10-16T11:01:29.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.096+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f598c0d0f80 con 0x7f598c0cab40 2023-10-16T11:01:29.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.097+0000 7f59827fc700 1 -- 172.21.15.181:0/2876498440 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f5988057ea0 con 0x7f598c0cab40 2023-10-16T11:01:29.099 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:01:29.100 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1903420066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1903420066},{"type":"v1","addr":"172.21.15.181:6836","nonce":1903420066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/586483327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":586483327},{"type":"v1","addr":"172.21.15.190:6835","nonce":586483327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2078148321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2078148321},{"type":"v1","addr":"172.21.15.181:6837","nonce":2078148321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:01:22.987835+0000","modified":"2023-10-16T11:01:26.027862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10197},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10197":{"gid":10197,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2135938520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2135938520},{"type":"v1","addr":"172.21.15.190:6834","nonce":2135938520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:01:29.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f597006b070 msgr2=0x7f597006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f597006b070 0x7f597006d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f597c001360 tx=0x7f597c009040 comp rx=0 tx=0).stop 2023-10-16T11:01:29.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f598c0cab40 msgr2=0x7f598c0c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f598c0cab40 0x7f598c0c9760 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f598800ebe0 tx=0x7f5988009ec0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 shutdown_connections 2023-10-16T11:01:29.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f597006b070 0x7f597006d530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598c15e7f0 0x7f598c0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f598c0b1ee0 0x7f598c15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 --2- 172.21.15.181:0/2876498440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f598c0cab40 0x7f598c0c9760 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 >> 172.21.15.181:0/2876498440 conn(0x7f598c0d0460 msgr2=0x7f598c0af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 shutdown_connections 2023-10-16T11:01:29.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.099+0000 7f59941d1700 1 -- 172.21.15.181:0/2876498440 wait complete. 2023-10-16T11:01:29.104 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 94 2023-10-16T11:01:29.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:01:22.987835+0000', 'modified': '2023-10-16T11:01:26.027862+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10197}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10197': {'gid': 10197, 'name': 'd', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6834/2135938520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 2135938520}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 2135938520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:01:29.516 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:01:29.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-16T11:01:29.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.641+0000 7faf48ad1700 1 Processor -- start 2023-10-16T11:01:29.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.642+0000 7faf48ad1700 1 -- start start 2023-10-16T11:01:29.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.642+0000 7faf48ad1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf440d0fc0 0x7faf440cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.642+0000 7faf48ad1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf4406a8a0 0x7faf4406ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.642+0000 7faf48ad1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf440cddf0 con 0x7faf4415df60 2023-10-16T11:01:29.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.642+0000 7faf48ad1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf440af9c0 con 0x7faf4406a8a0 2023-10-16T11:01:29.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.642+0000 7faf48ad1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4406b270 con 0x7faf440d0fc0 2023-10-16T11:01:29.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf41d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf440d0fc0 0x7faf440cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf42d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf4406a8a0 0x7faf4406ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf4259c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7faf4415df60 0x7faf440cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:47230/0 (socket says 172.21.15.181:47230) 2023-10-16T11:01:29.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf4259c700 1 -- 172.21.15.181:0/3928960048 learned_addr learned my addr 172.21.15.181:0/3928960048 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:29.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf42d9d700 1 -- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf440d0fc0 msgr2=0x7faf440cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf42d9d700 1 --2- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf440d0fc0 0x7faf440cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf42d9d700 1 -- 172.21.15.181:0/3928960048 >> v1:172.21.15.190:6789/0 conn(0x7faf4415df60 legacy=0x7faf440cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf42d9d700 1 -- 172.21.15.181:0/3928960048 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf440ccf50 con 0x7faf4406a8a0 2023-10-16T11:01:29.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf41d9b700 1 --2- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf440d0fc0 0x7faf440cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:29.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.643+0000 7faf42d9d700 1 --2- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf4406a8a0 0x7faf4406ad30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009d90 tx=0x7faf3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d598a1205895a07b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf4159a700 1 -- 172.21.15.181:0/3928960048 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c00e070 con 0x7faf4406a8a0 2023-10-16T11:01:29.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf4159a700 1 -- 172.21.15.181:0/3928960048 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c00bb20 con 0x7faf4406a8a0 2023-10-16T11:01:29.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf4159a700 1 -- 172.21.15.181:0/3928960048 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c012430 con 0x7faf4406a8a0 2023-10-16T11:01:29.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 -- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf4406a8a0 msgr2=0x7faf4406ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 --2- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf4406a8a0 0x7faf4406ad30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009d90 tx=0x7faf3c00b330 comp rx=0 tx=0).stop 2023-10-16T11:01:29.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 -- 172.21.15.181:0/3928960048 shutdown_connections 2023-10-16T11:01:29.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 --2- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf4406a8a0 0x7faf4406ad30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 --2- 172.21.15.181:0/3928960048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf440d0fc0 0x7faf440cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 -- 172.21.15.181:0/3928960048 >> 172.21.15.181:0/3928960048 conn(0x7faf440d04a0 msgr2=0x7faf440d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 -- 172.21.15.181:0/3928960048 shutdown_connections 2023-10-16T11:01:29.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.644+0000 7faf48ad1700 1 -- 172.21.15.181:0/3928960048 wait complete. 2023-10-16T11:01:29.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 Processor -- start 2023-10-16T11:01:29.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 -- start start 2023-10-16T11:01:29.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4406a8a0 0x7faf440b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf440d0fc0 0x7faf44158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4415df60 0x7faf440c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf44158f50 con 0x7faf4406a8a0 2023-10-16T11:01:29.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440b4db0 con 0x7faf440d0fc0 2023-10-16T11:01:29.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.646+0000 7faf48ad1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf440b5090 con 0x7faf4415df60 2023-10-16T11:01:29.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4406a8a0 0x7faf440b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4406a8a0 0x7faf440b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52216/0 (socket says 172.21.15.181:52216) 2023-10-16T11:01:29.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 -- 172.21.15.181:0/1393275949 learned_addr learned my addr 172.21.15.181:0/1393275949 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:29.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf42d9d700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4415df60 0x7faf440c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf41d9b700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf440d0fc0 0x7faf44158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 -- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4415df60 msgr2=0x7faf440c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4415df60 0x7faf440c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 -- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf440d0fc0 msgr2=0x7faf44158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf440d0fc0 0x7faf44158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 -- 172.21.15.181:0/1393275949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf340094a0 con 0x7faf4406a8a0 2023-10-16T11:01:29.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf4259c700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4406a8a0 0x7faf440b5e50 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c430 tx=0x7faf3800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf32ffd700 1 -- 172.21.15.181:0/1393275949 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38005dc0 con 0x7faf4406a8a0 2023-10-16T11:01:29.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf32ffd700 1 -- 172.21.15.181:0/1393275949 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf380070d0 con 0x7faf4406a8a0 2023-10-16T11:01:29.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c009a70 con 0x7faf4406a8a0 2023-10-16T11:01:29.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf32ffd700 1 -- 172.21.15.181:0/1393275949 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38007460 con 0x7faf4406a8a0 2023-10-16T11:01:29.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.647+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf440b5650 con 0x7faf4406a8a0 2023-10-16T11:01:29.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.649+0000 7faf32ffd700 1 -- 172.21.15.181:0/1393275949 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faf380075c0 con 0x7faf4406a8a0 2023-10-16T11:01:29.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.649+0000 7faf32ffd700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7faf2406b020 0x7faf2406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.649+0000 7faf41d9b700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7faf2406b020 0x7faf2406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.649+0000 7faf32ffd700 1 -- 172.21.15.181:0/1393275949 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7faf38088d90 con 0x7faf4406a8a0 2023-10-16T11:01:29.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.650+0000 7faf41d9b700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7faf2406b020 0x7faf2406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf44159350 tx=0x7faf34008040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.651+0000 7faf54cbbb80 1 Processor -- start 2023-10-16T11:01:29.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.651+0000 7faf54cbbb80 1 -- start start 2023-10-16T11:01:29.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83aafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83a83010 0x558d83a836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d839f9c20 con 0x558d83a83010 2023-10-16T11:01:29.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d839dec10 con 0x558d83aad630 2023-10-16T11:01:29.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d839dc530 con 0x558d83aa8220 2023-10-16T11:01:29.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83a83010 0x558d83a836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83a83010 0x558d83a836c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52228/0 (socket says 172.21.15.181:52228) 2023-10-16T11:01:29.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 -- 172.21.15.181:0/1257262779 learned_addr learned my addr 172.21.15.181:0/1257262779 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:29.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf4259c700 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83aafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf41d9b700 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 -- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 msgr2=0x558d83aafa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83aafa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 -- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 msgr2=0x558d83aaffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf42d9d700 1 -- 172.21.15.181:0/1257262779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d839953a0 con 0x558d83a83010 2023-10-16T11:01:29.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.652+0000 7faf41d9b700 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:29.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf42d9d700 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83a83010 0x558d83a836c0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faf3c013000 tx=0x7faf3c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681da7c6c56ec5eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf23fff700 1 -- 172.21.15.181:0/1257262779 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c003a20 con 0x558d83a83010 2023-10-16T11:01:29.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf23fff700 1 -- 172.21.15.181:0/1257262779 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c003b80 con 0x558d83a83010 2023-10-16T11:01:29.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf23fff700 1 -- 172.21.15.181:0/1257262779 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0125e0 con 0x558d83a83010 2023-10-16T11:01:29.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1257262779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83a83010 msgr2=0x558d83a836c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83a83010 0x558d83a836c0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faf3c013000 tx=0x7faf3c010000 comp rx=0 tx=0).stop 2023-10-16T11:01:29.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1257262779 shutdown_connections 2023-10-16T11:01:29.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83aafa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83a83010 0x558d83a836c0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1257262779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.653+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1257262779 >> 172.21.15.181:0/1257262779 conn(0x558d839b2d80 msgr2=0x558d83acdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.654+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1257262779 shutdown_connections 2023-10-16T11:01:29.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.654+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1257262779 wait complete. 2023-10-16T11:01:29.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.654+0000 7faf54cbbb80 1 -- start start 2023-10-16T11:01:29.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.654+0000 7faf54cbbb80 1 Processor -- start 2023-10-16T11:01:29.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83aa5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83ab2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83a79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d839d7e20 con 0x558d83aad630 2023-10-16T11:01:29.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d839746c0 con 0x558d83a83010 2023-10-16T11:01:29.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d83974360 con 0x558d83aa8220 2023-10-16T11:01:29.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf54cbbb80 10 client.0 _ll_register_callbacks cb 0x558d837d17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:01:29.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83a79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83a79f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52234/0 (socket says 172.21.15.181:52234) 2023-10-16T11:01:29.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 -- 172.21.15.181:0/1404593118 learned_addr learned my addr 172.21.15.181:0/1404593118 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:29.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf4259c700 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83aa5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf41d9b700 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83ab2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 -- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 msgr2=0x558d83ab2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83ab2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 -- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 msgr2=0x558d83aa5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83aa5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.655+0000 7faf42d9d700 1 -- 172.21.15.181:0/1404593118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf440ccf50 con 0x558d83aad630 2023-10-16T11:01:29.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf42d9d700 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83a79f60 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7faf440716a0 tx=0x7faf3c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198e4f14359ab54b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf217fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:01:29.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf217fa700 1 -- 172.21.15.181:0/1404593118 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c003a20 con 0x558d83aad630 2023-10-16T11:01:29.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d83a785b0 con 0x558d83aad630 2023-10-16T11:01:29.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf217fa700 1 -- 172.21.15.181:0/1404593118 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7faf3c004760 con 0x558d83aad630 2023-10-16T11:01:29.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf217fa700 1 -- 172.21.15.181:0/1404593118 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf3c004920 con 0x558d83aad630 2023-10-16T11:01:29.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 10 client.10326 fetch_fsmap learned FSMap version 94 2023-10-16T11:01:29.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x558d83802d90 con 0x558d83aad630 2023-10-16T11:01:29.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf217fa700 1 -- 172.21.15.181:0/1404593118 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d83a785b0 con 0x558d83aad630 2023-10-16T11:01:29.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf217fa700 1 -- 172.21.15.181:0/1404593118 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 94) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faf3c012530 con 0x558d83aad630 2023-10-16T11:01:29.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 10 client.10326 fetch_fsmap finished waiting for FSMap version 94 2023-10-16T11:01:29.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 10 client.10326 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T11:01:29.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 20 client.10326 populate_metadata read hostname 'smithi181' 2023-10-16T11:01:29.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83ab0c80 0x558d83ab1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 4 client.10326 mds_command: new command op to 10197 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:01:29.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 --> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d839d0e00 con 0x558d83ab0c80 2023-10-16T11:01:29.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.656+0000 7faf4259c700 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83ab0c80 0x558d83ab1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.657+0000 7faf217fa700 10 client.10326 ms_handle_connect on v2:172.21.15.190:6832/2135938520 2023-10-16T11:01:29.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.657+0000 7faf4259c700 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83ab0c80 0x558d83ab1060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a3f01c5b435b28 server_cookie=3d91f60e7d1b4e25 in_seq=0 out_seq=0 2023-10-16T11:01:29.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.657+0000 7faf217fa700 1 -- 172.21.15.181:0/1404593118 <== mds.0 v2:172.21.15.190:6832/2135938520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558d839d0e00 con 0x558d83ab0c80 2023-10-16T11:01:29.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.657+0000 7faf217fa700 10 client.10326 handle_command_reply: tid=0 2023-10-16T11:01:29.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.657+0000 7faf54cbbb80 1 client.10326 shutdown 2023-10-16T11:01:29.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 msgr2=0x558d83a79f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83a79f60 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7faf440716a0 tx=0x7faf3c004b40 comp rx=0 tx=0).stop 2023-10-16T11:01:29.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 shutdown_connections 2023-10-16T11:01:29.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83a79f60 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83aa5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83ab0c80 0x558d83ab1060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/1404593118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83ab2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf217fa700 0 client.10326 ms_handle_reset on v2:172.21.15.190:6832/2135938520 2023-10-16T11:01:29.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 >> 172.21.15.181:0/1404593118 conn(0x558d839b2d80 msgr2=0x558d83a849f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 shutdown_connections 2023-10-16T11:01:29.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 1 -- 172.21.15.181:0/1404593118 wait complete. 2023-10-16T11:01:29.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.658+0000 7faf54cbbb80 20 client.10326 trim_cache size 0 max 16384 2023-10-16T11:01:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.675+0000 7faf54cbbb80 1 Processor -- start 2023-10-16T11:01:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.675+0000 7faf54cbbb80 1 -- start start 2023-10-16T11:01:29.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83a83010 0x558d83acabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aa8220 0x558d83aafa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d838588d0 con 0x558d83aa8220 2023-10-16T11:01:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d83aa63a0 con 0x558d83aad630 2023-10-16T11:01:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d83ad9d00 con 0x558d83a83010 2023-10-16T11:01:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aa8220 0x558d83aafa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aa8220 0x558d83aafa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52240/0 (socket says 172.21.15.181:52240) 2023-10-16T11:01:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 -- 172.21.15.181:0/4082026722 learned_addr learned my addr 172.21.15.181:0/4082026722 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf4259c700 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83a83010 0x558d83acabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf42d9d700 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 -- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83a83010 msgr2=0x558d83acabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83a83010 0x558d83acabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 -- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 msgr2=0x558d83aaff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 -- 172.21.15.181:0/4082026722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d837ebc10 con 0x558d83aa8220 2023-10-16T11:01:29.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf42d9d700 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:01:29.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf41d9b700 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aa8220 0x558d83aafa20 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7faf4415c7c0 tx=0x7faf34013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359caf8c2d205e68 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf30ff9700 1 -- 172.21.15.181:0/4082026722 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34015ce0 con 0x558d83aa8220 2023-10-16T11:01:29.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf30ff9700 1 -- 172.21.15.181:0/4082026722 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf34015e40 con 0x558d83aa8220 2023-10-16T11:01:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.676+0000 7faf30ff9700 1 -- 172.21.15.181:0/4082026722 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34016640 con 0x558d83aa8220 2023-10-16T11:01:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 -- 172.21.15.181:0/4082026722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aa8220 msgr2=0x558d83aafa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aa8220 0x558d83aafa20 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7faf4415c7c0 tx=0x7faf34013000 comp rx=0 tx=0).stop 2023-10-16T11:01:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 -- 172.21.15.181:0/4082026722 shutdown_connections 2023-10-16T11:01:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83a83010 0x558d83acabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83aad630 0x558d83aaff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/4082026722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aa8220 0x558d83aafa20 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 -- 172.21.15.181:0/4082026722 >> 172.21.15.181:0/4082026722 conn(0x558d839b2d80 msgr2=0x558d83aec780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 -- 172.21.15.181:0/4082026722 shutdown_connections 2023-10-16T11:01:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.677+0000 7faf54cbbb80 1 -- 172.21.15.181:0/4082026722 wait complete. 2023-10-16T11:01:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.678+0000 7faf54cbbb80 1 -- start start 2023-10-16T11:01:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.678+0000 7faf54cbbb80 1 Processor -- start 2023-10-16T11:01:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.678+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83b34d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83b352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83ad9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d83b31a00 con 0x558d83aad630 2023-10-16T11:01:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d83ad8680 con 0x558d83a83010 2023-10-16T11:01:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d83b348d0 con 0x558d83aa8220 2023-10-16T11:01:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 10 client.0 _ll_register_callbacks cb 0x7faf3c00b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:01:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83ad9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83ad9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52248/0 (socket says 172.21.15.181:52248) 2023-10-16T11:01:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 -- 172.21.15.181:0/373419037 learned_addr learned my addr 172.21.15.181:0/373419037 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf4259c700 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83b34d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf41d9b700 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83b352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 -- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 msgr2=0x558d83b352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83b352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 -- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 msgr2=0x558d83b34d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83b34d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 -- 172.21.15.181:0/373419037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3400e050 con 0x558d83aad630 2023-10-16T11:01:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf42d9d700 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83ad9100 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00d010 tx=0x7faf3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391c8b8e9dab4238 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf217fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:01:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf217fa700 1 -- 172.21.15.181:0/373419037 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c003a20 con 0x558d83aad630 2023-10-16T11:01:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf217fa700 1 -- 172.21.15.181:0/373419037 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7faf3c012630 con 0x558d83aad630 2023-10-16T11:01:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf3c003b80 con 0x558d83aad630 2023-10-16T11:01:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf217fa700 1 -- 172.21.15.181:0/373419037 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf3c0043d0 con 0x558d83aad630 2023-10-16T11:01:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 10 client.10332 fetch_fsmap learned FSMap version 94 2023-10-16T11:01:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x558d83829200 con 0x558d83aad630 2023-10-16T11:01:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.679+0000 7faf217fa700 1 -- 172.21.15.181:0/373419037 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c003b80 con 0x558d83aad630 2023-10-16T11:01:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf217fa700 1 -- 172.21.15.181:0/373419037 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 94) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faf3c025460 con 0x558d83aad630 2023-10-16T11:01:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf54cbbb80 10 client.10332 fetch_fsmap finished waiting for FSMap version 94 2023-10-16T11:01:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf54cbbb80 10 client.10332 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T11:01:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf54cbbb80 20 client.10332 populate_metadata read hostname 'smithi181' 2023-10-16T11:01:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83b30db0 0x558d83b31190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf54cbbb80 4 client.10332 mds_command: new command op to 10197 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:01:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 --> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558d839d0b20 con 0x558d83b30db0 2023-10-16T11:01:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf4259c700 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83b30db0 0x558d83b31190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf4259c700 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83b30db0 0x558d83b31190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d769fec382229b server_cookie=32c8fd3b87af5d17 in_seq=0 out_seq=0 2023-10-16T11:01:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf217fa700 10 client.10332 ms_handle_connect on v2:172.21.15.190:6832/2135938520 2023-10-16T11:01:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf217fa700 1 -- 172.21.15.181:0/373419037 <== mds.0 v2:172.21.15.190:6832/2135938520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x558d839d0b20 con 0x558d83b30db0 2023-10-16T11:01:29.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf217fa700 10 client.10332 handle_command_reply: tid=0 2023-10-16T11:01:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.680+0000 7faf54cbbb80 1 client.10332 shutdown 2023-10-16T11:01:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 msgr2=0x558d83ad9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83ad9100 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00d010 tx=0x7faf3c011000 comp rx=0 tx=0).stop 2023-10-16T11:01:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 shutdown_connections 2023-10-16T11:01:29.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558d83aad630 0x558d83ad9100 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d83a83010 0x558d83b34d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:6832/2135938520,v1:172.21.15.190:6834/2135938520] conn(0x558d83b30db0 0x558d83b31190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 --2- 172.21.15.181:0/373419037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558d83aa8220 0x558d83b352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf217fa700 0 client.10332 ms_handle_reset on v2:172.21.15.190:6832/2135938520 2023-10-16T11:01:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 >> 172.21.15.181:0/373419037 conn(0x558d839b2d80 msgr2=0x558d83a86a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 shutdown_connections 2023-10-16T11:01:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 1 -- 172.21.15.181:0/373419037 wait complete. 2023-10-16T11:01:29.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf54cbbb80 20 client.10332 trim_cache size 0 max 16384 2023-10-16T11:01:29.705 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:01:29.706 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:01:29.706 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:01:29.706 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 10197, 2023-10-16T11:01:29.706 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:01:29.706 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:01:29.707 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T11:01:29.707 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 6.0664510839999997, 2023-10-16T11:01:29.707 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 94, 2023-10-16T11:01:29.707 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 181, 2023-10-16T11:01:29.707 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 181, 2023-10-16T11:01:29.708 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 11.019081895999999 2023-10-16T11:01:29.708 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:01:29.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7faf2406b020 msgr2=0x7faf2406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7faf2406b020 0x7faf2406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf44159350 tx=0x7faf34008040 comp rx=0 tx=0).stop 2023-10-16T11:01:29.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4406a8a0 msgr2=0x7faf440b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4406a8a0 0x7faf440b5e50 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c430 tx=0x7faf3800c7f0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 shutdown_connections 2023-10-16T11:01:29.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7faf2406b020 0x7faf2406d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4415df60 0x7faf440c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf4406a8a0 0x7faf440b5e50 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 --2- 172.21.15.181:0/1393275949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf440d0fc0 0x7faf44158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 >> 172.21.15.181:0/1393275949 conn(0x7faf440d04a0 msgr2=0x7faf440cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 shutdown_connections 2023-10-16T11:01:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.681+0000 7faf48ad1700 1 -- 172.21.15.181:0/1393275949 wait complete. 2023-10-16T11:01:29.712 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:01:29.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.833+0000 7f18d4db7700 1 Processor -- start 2023-10-16T11:01:29.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.833+0000 7f18d4db7700 1 -- start start 2023-10-16T11:01:29.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18d4db7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18d015dfd0 0x7f18d00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18d4db7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 0x7f18d006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18d4db7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d006f870 con 0x7f18d006db80 2023-10-16T11:01:29.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18d4db7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00d1160 con 0x7f18d015dfd0 2023-10-16T11:01:29.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18d4db7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cc8e0 con 0x7f18d00af450 2023-10-16T11:01:29.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18ced9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f18d006db80 0x7f18d00cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35102/0 (socket says 172.21.15.181:35102) 2023-10-16T11:01:29.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18ced9d700 1 -- 172.21.15.181:0/1979667331 learned_addr learned my addr 172.21.15.181:0/1979667331 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:29.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cdd9b700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 0x7f18d006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18ce59c700 1 --2- 172.21.15.181:0/1979667331 >> v2:172.21.15.190:3300/0 conn(0x7f18d015dfd0 0x7f18d00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529140049 0 0) 0x7f18d006f870 con 0x7f18d006db80 2023-10-16T11:01:29.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b80035b0 con 0x7f18d006db80 2023-10-16T11:01:29.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 910911406 0 0) 0x7f18b80035b0 con 0x7f18d006db80 2023-10-16T11:01:29.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 msgr2=0x7f18d006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 0x7f18d006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 >> v2:172.21.15.190:3300/0 conn(0x7f18d015dfd0 msgr2=0x7f18d00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 --2- 172.21.15.181:0/1979667331 >> v2:172.21.15.190:3300/0 conn(0x7f18d015dfd0 0x7f18d00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00ccbc0 con 0x7f18d006db80 2023-10-16T11:01:29.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.834+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f18c80019f0 con 0x7f18d006db80 2023-10-16T11:01:29.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 >> v1:172.21.15.181:6789/0 conn(0x7f18d006db80 legacy=0x7f18d00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8003a00 0x7f18b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 0x7f18b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18b8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00d1160 con 0x7f18b8004d90 2023-10-16T11:01:29.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cc8e0 con 0x7f18b8003a00 2023-10-16T11:01:29.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b800ba10 con 0x7f18d00af450 2023-10-16T11:01:29.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ced9d700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18b8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ce59c700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8003a00 0x7f18b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cdd9b700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 0x7f18b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ced9d700 1 -- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 msgr2=0x7f18b8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ced9d700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d00af450 0x7f18b8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ced9d700 1 -- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8003a00 msgr2=0x7f18b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:29.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ced9d700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8003a00 0x7f18b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ced9d700 1 -- 172.21.15.181:0/1979667331 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80035b0 con 0x7f18b8004d90 2023-10-16T11:01:29.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18ced9d700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18b8005210 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f18c800a860 tx=0x7f18b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db152955284bfd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8012600 con 0x7f18b8004d90 2023-10-16T11:01:29.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8012760 con 0x7f18b8004d90 2023-10-16T11:01:29.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.835+0000 7f18cd59a700 1 -- 172.21.15.181:0/1979667331 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8012b10 con 0x7f18b8004d90 2023-10-16T11:01:29.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.836+0000 7f18d4db7700 1 -- 172.21.15.181:0/1979667331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 msgr2=0x7f18b8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.836+0000 7f18d4db7700 1 --2- 172.21.15.181:0/1979667331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18b8005210 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f18c800a860 tx=0x7f18b8005880 comp rx=0 tx=0).stop 2023-10-16T11:01:29.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.836+0000 7f18ced9d700 1 -- 172.21.15.181:0/1979667331 reap_dead start 2023-10-16T11:01:29.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.836+0000 7f18d4db7700 1 -- 172.21.15.181:0/1979667331 shutdown_connections 2023-10-16T11:01:29.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.836+0000 7f18d4db7700 1 -- 172.21.15.181:0/1979667331 >> 172.21.15.181:0/1979667331 conn(0x7f18d00d04a0 msgr2=0x7f18d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:29.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.836+0000 7f18d4db7700 1 -- 172.21.15.181:0/1979667331 shutdown_connections 2023-10-16T11:01:29.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.836+0000 7f18d4db7700 1 -- 172.21.15.181:0/1979667331 wait complete. 2023-10-16T11:01:29.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.837+0000 7f18d4db7700 1 Processor -- start 2023-10-16T11:01:29.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.837+0000 7f18d4db7700 1 -- start start 2023-10-16T11:01:29.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18d4db7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18d01550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18d4db7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d0160b60 0x7f18d00c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18d4db7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d00c5b30 0x7f18d00c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18d4db7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0161100 con 0x7f18b8004d90 2023-10-16T11:01:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18d4db7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d015fa40 con 0x7f18d00c5b30 2023-10-16T11:01:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18d4db7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d015fd20 con 0x7f18d0160b60 2023-10-16T11:01:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18d01550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18d01550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52266/0 (socket says 172.21.15.181:52266) 2023-10-16T11:01:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 -- 172.21.15.181:0/1934371628 learned_addr learned my addr 172.21.15.181:0/1934371628 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 -- 172.21.15.181:0/1934371628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d0160b60 msgr2=0x7f18d00c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d0160b60 0x7f18d00c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 -- 172.21.15.181:0/1934371628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d00c5b30 msgr2=0x7f18d00c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d00c5b30 0x7f18d00c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 -- 172.21.15.181:0/1934371628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00c6520 con 0x7f18b8004d90 2023-10-16T11:01:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.838+0000 7f18ce59c700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18d01550a0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f18c0011a70 tx=0x7f18c0011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.839+0000 7f18beffd700 1 -- 172.21.15.181:0/1934371628 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000f980 con 0x7f18b8004d90 2023-10-16T11:01:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.839+0000 7f18beffd700 1 -- 172.21.15.181:0/1934371628 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000fb00 con 0x7f18b8004d90 2023-10-16T11:01:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.839+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d01555e0 con 0x7f18b8004d90 2023-10-16T11:01:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.839+0000 7f18beffd700 1 -- 172.21.15.181:0/1934371628 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000feb0 con 0x7f18b8004d90 2023-10-16T11:01:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.839+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d015e7e0 con 0x7f18b8004d90 2023-10-16T11:01:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.840+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d00b4500 con 0x7f18b8004d90 2023-10-16T11:01:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.840+0000 7f18beffd700 1 -- 172.21.15.181:0/1934371628 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f18c0014440 con 0x7f18b8004d90 2023-10-16T11:01:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.841+0000 7f18beffd700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f18b006b020 0x7f18b006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.841+0000 7f18beffd700 1 -- 172.21.15.181:0/1934371628 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f18c008a010 con 0x7f18b8004d90 2023-10-16T11:01:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.841+0000 7f18cdd9b700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f18b006b020 0x7f18b006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.841+0000 7f18cdd9b700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f18b006b020 0x7f18b006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18d009e460 tx=0x7f18c4009670 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:29.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:29.845+0000 7f18beffd700 1 -- 172.21.15.181:0/1934371628 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f18c0054f00 con 0x7f18b8004d90 2023-10-16T11:01:30.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.082+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18d015fe90 con 0x7f18b8004d90 2023-10-16T11:01:30.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.082+0000 7f18beffd700 1 -- 172.21.15.181:0/1934371628 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7f18c0058520 con 0x7f18b8004d90 2023-10-16T11:01:30.084 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:01:30.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.084+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f18b006b020 msgr2=0x7f18b006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:30.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.084+0000 7f18d4db7700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f18b006b020 0x7f18b006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18d009e460 tx=0x7f18c4009670 comp rx=0 tx=0).stop 2023-10-16T11:01:30.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.084+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 msgr2=0x7f18d01550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:30.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.084+0000 7f18d4db7700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18d01550a0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f18c0011a70 tx=0x7f18c0011e30 comp rx=0 tx=0).stop 2023-10-16T11:01:30.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 shutdown_connections 2023-10-16T11:01:30.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f18b006b020 0x7f18b006d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:30.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18d00c5b30 0x7f18d00c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:30.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18b8004d90 0x7f18d01550a0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:30.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 --2- 172.21.15.181:0/1934371628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18d0160b60 0x7f18d00c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:30.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 >> 172.21.15.181:0/1934371628 conn(0x7f18d00d04a0 msgr2=0x7f18d00ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:30.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 shutdown_connections 2023-10-16T11:01:30.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:30.085+0000 7f18d4db7700 1 -- 172.21.15.181:0/1934371628 wait complete. 2023-10-16T11:01:30.098 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:01:30.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:01:30.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T11:01:30.099 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:01:30.100 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:01:30.100 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:01:30.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:01:30.101 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:01:30.101 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:01:30.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.102 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T11:01:30.119 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:01:30.120 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:01:30.120 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:01:30.120 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:01:30.120 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T11:01:30.120 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:01:30.121 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:01:30.121 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:01:30.121 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:01:30.121 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:01:30.121 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:01:30.122 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T11:01:30.122 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:01:30.122 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T11:01:30.122 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:01:30.122 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 37779sec preferred_lft 37779sec 2023-10-16T11:01:30.123 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T11:01:30.123 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:01:30.123 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:01:30.123 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:01:30.123 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:01:30.124 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:01:30.124 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T11:01:30.124 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:01:30.124 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:01:30.124 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:01:30.125 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T11:01:30.125 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:01:30.125 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:01:30.125 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:01:30.125 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T11:01:30.125 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:01:30.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.126 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:01:30.184 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T11:01:30.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.186 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T11:01:30.245 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T11:01:30.246 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:01:30.246 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:01:30.246 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:01:30.246 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:01:30.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.247 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T11:01:30.248 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T11:01:30.248 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:01:30.248 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:01:30.249 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:01:30.249 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T11:01:30.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.351 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:01:30.367 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:01:30.368 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:01:30.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:01:30.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T11:01:30.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:01:30.435 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:01:30.491 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:01:30.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:01:30.493 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:01:30.548 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T11:01:30.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.549 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:01:30.633 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:01:30.634 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:01:30.634 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:01:30.642 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:01:30.642 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:01:30.643 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:01:30.643 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:01:30.643 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:01:30.717 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:01:30.718 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:01:30.718 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:01:30.718 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:01:30.718 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:01:30.718 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T11:01:30.719 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T11:01:30.719 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T11:01:30.720 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:01:30.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.721 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:01:30.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:30.749 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:01:30.836 INFO:teuthology.orchestra.run.smithi181.stdout:client9148 2023-10-16T11:01:30.836 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:01:30.837 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:01:30.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:01:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.013+0000 7f64cf8cd700 1 Processor -- start 2023-10-16T11:01:31.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.013+0000 7f64cf8cd700 1 -- start start 2023-10-16T11:01:31.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cf8cd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815e460 0x7f64c80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cf8cd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64c80cb110 0x7f64c815e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cf8cd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64c815a0f0 0x7f64c815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cf8cd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c806ed00 con 0x7f64c80cb110 2023-10-16T11:01:31.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cf8cd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c815ed80 con 0x7f64c815a0f0 2023-10-16T11:01:31.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cf8cd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c806e0f0 con 0x7f64c815e460 2023-10-16T11:01:31.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cce68700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64c80cb110 0x7f64c815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cce68700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64c80cb110 0x7f64c815e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52278/0 (socket says 172.21.15.181:52278) 2023-10-16T11:01:31.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.014+0000 7f64cce68700 1 -- 172.21.15.181:0/4153746480 learned_addr learned my addr 172.21.15.181:0/4153746480 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:31.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cce68700 1 -- 172.21.15.181:0/4153746480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815e460 msgr2=0x7f64c80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cde6a700 1 --2- 172.21.15.181:0/4153746480 >> v2:172.21.15.190:3300/0 conn(0x7f64c815a0f0 0x7f64c815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cd669700 1 --2- 172.21.15.181:0/4153746480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815e460 0x7f64c80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cce68700 1 --2- 172.21.15.181:0/4153746480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815e460 0x7f64c80d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cce68700 1 -- 172.21.15.181:0/4153746480 >> v2:172.21.15.190:3300/0 conn(0x7f64c815a0f0 msgr2=0x7f64c815a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cce68700 1 --2- 172.21.15.181:0/4153746480 >> v2:172.21.15.190:3300/0 conn(0x7f64c815a0f0 0x7f64c815a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cce68700 1 -- 172.21.15.181:0/4153746480 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c806e3d0 con 0x7f64c80cb110 2023-10-16T11:01:31.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cce68700 1 --2- 172.21.15.181:0/4153746480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c815e840 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f64c8071b00 tx=0x7f64c400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e382dc973ab8c427 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64bffff700 1 -- 172.21.15.181:0/4153746480 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c400dd30 con 0x7f64c80cb110 2023-10-16T11:01:31.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64bffff700 1 -- 172.21.15.181:0/4153746480 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c400de90 con 0x7f64c80cb110 2023-10-16T11:01:31.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64bffff700 1 -- 172.21.15.181:0/4153746480 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4010690 con 0x7f64c80cb110 2023-10-16T11:01:31.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cf8cd700 1 -- 172.21.15.181:0/4153746480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 msgr2=0x7f64c815e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/4153746480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c815e840 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f64c8071b00 tx=0x7f64c400be70 comp rx=0 tx=0).stop 2023-10-16T11:01:31.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cf8cd700 1 -- 172.21.15.181:0/4153746480 shutdown_connections 2023-10-16T11:01:31.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/4153746480 >> v2:172.21.15.190:3300/0 conn(0x7f64c815a0f0 0x7f64c815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/4153746480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815e460 0x7f64c80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/4153746480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c815e840 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.015+0000 7f64cf8cd700 1 -- 172.21.15.181:0/4153746480 >> 172.21.15.181:0/4153746480 conn(0x7f64c80d04a0 msgr2=0x7f64c80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:31.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.016+0000 7f64cf8cd700 1 -- 172.21.15.181:0/4153746480 shutdown_connections 2023-10-16T11:01:31.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.016+0000 7f64cf8cd700 1 -- 172.21.15.181:0/4153746480 wait complete. 2023-10-16T11:01:31.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 Processor -- start 2023-10-16T11:01:31.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 -- start start 2023-10-16T11:01:31.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c80b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815a0f0 0x7f64c80aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c815e460 0x7f64c80af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c81f7650 con 0x7f64c80cb110 2023-10-16T11:01:31.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80c88a0 con 0x7f64c815e460 2023-10-16T11:01:31.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.017+0000 7f64cf8cd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80b2b90 con 0x7f64c815a0f0 2023-10-16T11:01:31.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c80b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c80b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52282/0 (socket says 172.21.15.181:52282) 2023-10-16T11:01:31.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 -- 172.21.15.181:0/2726490935 learned_addr learned my addr 172.21.15.181:0/2726490935 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:31.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cce68700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815a0f0 0x7f64c80aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cde6a700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c815e460 0x7f64c80af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 -- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815a0f0 msgr2=0x7f64c80aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815a0f0 0x7f64c80aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 -- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c815e460 msgr2=0x7f64c80af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c815e460 0x7f64c80af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 -- 172.21.15.181:0/2726490935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c400b7f0 con 0x7f64c80cb110 2023-10-16T11:01:31.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cd669700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c80b2650 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c7f0 tx=0x7f64b800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64bdffb700 1 -- 172.21.15.181:0/2726490935 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8004220 con 0x7f64c80cb110 2023-10-16T11:01:31.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64bdffb700 1 -- 172.21.15.181:0/2726490935 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b8004380 con 0x7f64c80cb110 2023-10-16T11:01:31.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64bdffb700 1 -- 172.21.15.181:0/2726490935 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80036a0 con 0x7f64c80cb110 2023-10-16T11:01:31.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c81f7370 con 0x7f64c80cb110 2023-10-16T11:01:31.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.018+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c80c84c0 con 0x7f64c80cb110 2023-10-16T11:01:31.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.019+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c80c8040 con 0x7f64c80cb110 2023-10-16T11:01:31.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.020+0000 7f64bdffb700 1 -- 172.21.15.181:0/2726490935 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64b8003820 con 0x7f64c80cb110 2023-10-16T11:01:31.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.020+0000 7f64bdffb700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f64ac06b020 0x7f64ac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.020+0000 7f64cce68700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f64ac06b020 0x7f64ac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.020+0000 7f64bdffb700 1 -- 172.21.15.181:0/2726490935 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f64b8089510 con 0x7f64c80cb110 2023-10-16T11:01:31.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.021+0000 7f64cce68700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f64ac06b020 0x7f64ac06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64c400beb0 tx=0x7f64c401c040 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.025+0000 7f64bdffb700 1 -- 172.21.15.181:0/2726490935 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f64c80c8040 con 0x7f64c80cb110 2023-10-16T11:01:31.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.245+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 --> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f64c806e990 con 0x7f64ac06b020 2023-10-16T11:01:31.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.246+0000 7f64bdffb700 1 -- 172.21.15.181:0/2726490935 <== mgr.7889 v2:172.21.15.181:6832/303240986 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f64c4004270 con 0x7f64ac06b020 2023-10-16T11:01:31.249 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:01:31.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:01:31.249 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T11:01:31.249 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:01:31.250 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:01:31.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f64ac06b020 msgr2=0x7f64ac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f64ac06b020 0x7f64ac06d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64c400beb0 tx=0x7f64c401c040 comp rx=0 tx=0).stop 2023-10-16T11:01:31.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 msgr2=0x7f64c80b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c80b2650 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c7f0 tx=0x7f64b800cb00 comp rx=0 tx=0).stop 2023-10-16T11:01:31.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 shutdown_connections 2023-10-16T11:01:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f64ac06b020 0x7f64ac06d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64c815e460 0x7f64c80af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c80cb110 0x7f64c80b2650 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 --2- 172.21.15.181:0/2726490935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64c815a0f0 0x7f64c80aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.248+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 >> 172.21.15.181:0/2726490935 conn(0x7f64c80d04a0 msgr2=0x7f64c80adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.249+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 shutdown_connections 2023-10-16T11:01:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.249+0000 7f64cf8cd700 1 -- 172.21.15.181:0/2726490935 wait complete. 2023-10-16T11:01:31.262 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:01:31.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.383+0000 7fe2ef68b700 1 Processor -- start 2023-10-16T11:01:31.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.384+0000 7fe2ef68b700 1 -- start start 2023-10-16T11:01:31.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.384+0000 7fe2ef68b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2e80b2010 0x7fe2e80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.384+0000 7fe2ef68b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 0x7fe2e80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.384+0000 7fe2ef68b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80ce1f0 con 0x7fe2e806f3b0 2023-10-16T11:01:31.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.384+0000 7fe2ef68b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e80b2930 con 0x7fe2e80b2010 2023-10-16T11:01:31.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.384+0000 7fe2ef68b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80ccda0 con 0x7fe2e8071be0 2023-10-16T11:01:31.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2edc28700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe2e806f3b0 0x7fe2e806f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35140/0 (socket says 172.21.15.181:35140) 2023-10-16T11:01:31.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2edc28700 1 -- 172.21.15.181:0/1395183780 learned_addr learned my addr 172.21.15.181:0/1395183780 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:31.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ed427700 1 --2- 172.21.15.181:0/1395183780 >> v2:172.21.15.190:3300/0 conn(0x7fe2e80b2010 0x7fe2e80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ecc26700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 0x7fe2e80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220294546 0 0) 0x7fe2e80ce1f0 con 0x7fe2e806f3b0 2023-10-16T11:01:31.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d40035b0 con 0x7fe2e806f3b0 2023-10-16T11:01:31.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2411151622 0 0) 0x7fe2d40035b0 con 0x7fe2e806f3b0 2023-10-16T11:01:31.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 msgr2=0x7fe2e80b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 0x7fe2e80b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 >> v2:172.21.15.190:3300/0 conn(0x7fe2e80b2010 msgr2=0x7fe2e80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 --2- 172.21.15.181:0/1395183780 >> v2:172.21.15.190:3300/0 conn(0x7fe2e80b2010 0x7fe2e80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80727e0 con 0x7fe2e806f3b0 2023-10-16T11:01:31.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe2e00019f0 con 0x7fe2e806f3b0 2023-10-16T11:01:31.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 >> v1:172.21.15.181:6789/0 conn(0x7fe2e806f3b0 legacy=0x7fe2e806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 0x7fe2d4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2d4003f80 0x7fe2d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4004d90 0x7fe2d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80b2930 con 0x7fe2d4003f80 2023-10-16T11:01:31.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80ccda0 con 0x7fe2d4004d90 2023-10-16T11:01:31.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d400ba10 con 0x7fe2e8071be0 2023-10-16T11:01:31.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ecc26700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2d4003f80 0x7fe2d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ed427700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 0x7fe2d4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2edc28700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4004d90 0x7fe2d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ecc26700 1 -- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 msgr2=0x7fe2d4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ecc26700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2e8071be0 0x7fe2d4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ecc26700 1 -- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4004d90 msgr2=0x7fe2d4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.385+0000 7fe2ecc26700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4004d90 0x7fe2d4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ecc26700 1 -- 172.21.15.181:0/1395183780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d40035b0 con 0x7fe2d4003f80 2023-10-16T11:01:31.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2edc28700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4004d90 0x7fe2d4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:31.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ecc26700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2d4003f80 0x7fe2d4004850 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009b60 tx=0x7fe2d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7563f1ad139123aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400e660 con 0x7fe2d4003f80 2023-10-16T11:01:31.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e400e7c0 con 0x7fe2d4003f80 2023-10-16T11:01:31.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2dffff700 1 -- 172.21.15.181:0/1395183780 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400eb30 con 0x7fe2d4003f80 2023-10-16T11:01:31.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ef68b700 1 -- 172.21.15.181:0/1395183780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2d4003f80 msgr2=0x7fe2d4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ef68b700 1 --2- 172.21.15.181:0/1395183780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2d4003f80 0x7fe2d4004850 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009b60 tx=0x7fe2d4005880 comp rx=0 tx=0).stop 2023-10-16T11:01:31.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2edc28700 1 -- 172.21.15.181:0/1395183780 reap_dead start 2023-10-16T11:01:31.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ef68b700 1 -- 172.21.15.181:0/1395183780 shutdown_connections 2023-10-16T11:01:31.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ef68b700 1 -- 172.21.15.181:0/1395183780 >> 172.21.15.181:0/1395183780 conn(0x7fe2e80cd010 msgr2=0x7fe2e80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:31.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ef68b700 1 -- 172.21.15.181:0/1395183780 shutdown_connections 2023-10-16T11:01:31.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.386+0000 7fe2ef68b700 1 -- 172.21.15.181:0/1395183780 wait complete. 2023-10-16T11:01:31.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 Processor -- start 2023-10-16T11:01:31.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 -- start start 2023-10-16T11:01:31.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4003f80 0x7fe2e8154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e80c7660 0x7fe2e8151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2e8152160 0x7fe2e80c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80c8700 con 0x7fe2e8152160 2023-10-16T11:01:31.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80d0d50 con 0x7fe2e80c7660 2023-10-16T11:01:31.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2ef68b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80c7af0 con 0x7fe2d4003f80 2023-10-16T11:01:31.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2edc28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2e8152160 0x7fe2e80c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2edc28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2e8152160 0x7fe2e80c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52306/0 (socket says 172.21.15.181:52306) 2023-10-16T11:01:31.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.388+0000 7fe2edc28700 1 -- 172.21.15.181:0/2736118629 learned_addr learned my addr 172.21.15.181:0/2736118629 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:31.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2ed427700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4003f80 0x7fe2e8154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2ecc26700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e80c7660 0x7fe2e8151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2edc28700 1 -- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4003f80 msgr2=0x7fe2e8154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2edc28700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4003f80 0x7fe2e8154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2edc28700 1 -- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e80c7660 msgr2=0x7fe2e8151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2edc28700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e80c7660 0x7fe2e8151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2edc28700 1 -- 172.21.15.181:0/2736118629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80c7dd0 con 0x7fe2e8152160 2023-10-16T11:01:31.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2edc28700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2e8152160 0x7fe2e80c8b90 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0000f80 tx=0x7fe2e000e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2ddffb700 1 -- 172.21.15.181:0/2736118629 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0017070 con 0x7fe2e8152160 2023-10-16T11:01:31.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2ddffb700 1 -- 172.21.15.181:0/2736118629 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e00019d0 con 0x7fe2e8152160 2023-10-16T11:01:31.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2ddffb700 1 -- 172.21.15.181:0/2736118629 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000ede0 con 0x7fe2e8152160 2023-10-16T11:01:31.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e80c80b0 con 0x7fe2e8152160 2023-10-16T11:01:31.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.389+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e81553c0 con 0x7fe2e8152160 2023-10-16T11:01:31.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.390+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e80c88b0 con 0x7fe2e8152160 2023-10-16T11:01:31.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.390+0000 7fe2ddffb700 1 -- 172.21.15.181:0/2736118629 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 108) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe2e0013420 con 0x7fe2e8152160 2023-10-16T11:01:31.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.391+0000 7fe2ddffb700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fe2cc06b020 0x7fe2cc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.391+0000 7fe2ddffb700 1 -- 172.21.15.181:0/2736118629 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7fe2e00896a0 con 0x7fe2e8152160 2023-10-16T11:01:31.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.391+0000 7fe2ed427700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fe2cc06b020 0x7fe2cc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.391+0000 7fe2ed427700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fe2cc06b020 0x7fe2cc06d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009ce0 tx=0x7fe2d8009210 comp rx=0 tx=0).ready entity=mgr.7889 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.395+0000 7fe2ddffb700 1 -- 172.21.15.181:0/2736118629 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2e80c88b0 con 0x7fe2e8152160 2023-10-16T11:01:31.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.646+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe2e80c88b0 con 0x7fe2e8152160 2023-10-16T11:01:31.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.800+0000 7fe2ddffb700 1 -- 172.21.15.181:0/2736118629 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v109) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe2e80c88b0 con 0x7fe2e8152160 2023-10-16T11:01:31.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fe2cc06b020 msgr2=0x7fe2cc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fe2cc06b020 0x7fe2cc06d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009ce0 tx=0x7fe2d8009210 comp rx=0 tx=0).stop 2023-10-16T11:01:31.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2e8152160 msgr2=0x7fe2e80c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2e8152160 0x7fe2e80c8b90 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0000f80 tx=0x7fe2e000e670 comp rx=0 tx=0).stop 2023-10-16T11:01:31.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 shutdown_connections 2023-10-16T11:01:31.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7fe2cc06b020 0x7fe2cc06d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2d4003f80 0x7fe2e8154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2e8152160 0x7fe2e80c8b90 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 --2- 172.21.15.181:0/2736118629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2e80c7660 0x7fe2e8151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.803+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 >> 172.21.15.181:0/2736118629 conn(0x7fe2e80cd010 msgr2=0x7fe2e8152c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:31.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.804+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 shutdown_connections 2023-10-16T11:01:31.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.805+0000 7fe2ef68b700 1 -- 172.21.15.181:0/2736118629 wait complete. 2023-10-16T11:01:31.821 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:01:31.951 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:31.949+0000 7f1bc6788200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:01:31.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.948+0000 7f776df9b700 1 Processor -- start 2023-10-16T11:01:31.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.949+0000 7f776df9b700 1 -- start start 2023-10-16T11:01:31.953 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:31.949+0000 7fcf409c7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:01:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.949+0000 7f776df9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77680b2010 0x7f77680cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.949+0000 7f776df9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7768071be0 0x7f77680b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.949+0000 7f776df9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f776806f3b0 0x7f776806f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.949+0000 7f776df9b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77680ce1f0 con 0x7f77680b2010 2023-10-16T11:01:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.949+0000 7f776df9b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77680b2930 con 0x7f776806f3b0 2023-10-16T11:01:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.949+0000 7f776df9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77680ccda0 con 0x7f7768071be0 2023-10-16T11:01:31.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77680b2010 0x7f77680cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77680b2010 0x7f77680cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52336/0 (socket says 172.21.15.181:52336) 2023-10-16T11:01:31.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 -- 172.21.15.181:0/1964696743 learned_addr learned my addr 172.21.15.181:0/1964696743 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:31.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f7767fff700 1 --2- 172.21.15.181:0/1964696743 >> v2:172.21.15.190:3300/0 conn(0x7f776806f3b0 0x7f776806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f7766ffd700 1 --2- 172.21.15.181:0/1964696743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7768071be0 0x7f77680b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 -- 172.21.15.181:0/1964696743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7768071be0 msgr2=0x7f77680b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 --2- 172.21.15.181:0/1964696743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7768071be0 0x7f77680b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 -- 172.21.15.181:0/1964696743 >> v2:172.21.15.190:3300/0 conn(0x7f776806f3b0 msgr2=0x7f776806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 --2- 172.21.15.181:0/1964696743 >> v2:172.21.15.190:3300/0 conn(0x7f776806f3b0 0x7f776806f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 -- 172.21.15.181:0/1964696743 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77680727e0 con 0x7f77680b2010 2023-10-16T11:01:31.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77677fe700 1 --2- 172.21.15.181:0/1964696743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77680b2010 0x7f77680cc6e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f7768070750 tx=0x7f7758009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2163d813bea8455 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77667fc700 1 -- 172.21.15.181:0/1964696743 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758011830 con 0x7f77680b2010 2023-10-16T11:01:31.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77667fc700 1 -- 172.21.15.181:0/1964696743 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7758011990 con 0x7f77680b2010 2023-10-16T11:01:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.950+0000 7f77667fc700 1 -- 172.21.15.181:0/1964696743 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758011d20 con 0x7f77680b2010 2023-10-16T11:01:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 -- 172.21.15.181:0/1964696743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77680b2010 msgr2=0x7f77680cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 --2- 172.21.15.181:0/1964696743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77680b2010 0x7f77680cc6e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f7768070750 tx=0x7f7758009bd0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 -- 172.21.15.181:0/1964696743 shutdown_connections 2023-10-16T11:01:31.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 --2- 172.21.15.181:0/1964696743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77680b2010 0x7f77680cc6e0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 --2- 172.21.15.181:0/1964696743 >> v2:172.21.15.190:3300/0 conn(0x7f776806f3b0 0x7f776806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 --2- 172.21.15.181:0/1964696743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7768071be0 0x7f77680b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 -- 172.21.15.181:0/1964696743 >> 172.21.15.181:0/1964696743 conn(0x7f77680cd010 msgr2=0x7f77680cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 -- 172.21.15.181:0/1964696743 shutdown_connections 2023-10-16T11:01:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.951+0000 7f776df9b700 1 -- 172.21.15.181:0/1964696743 wait complete. 2023-10-16T11:01:31.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.952+0000 7f776df9b700 1 Processor -- start 2023-10-16T11:01:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.952+0000 7f776df9b700 1 -- start start 2023-10-16T11:01:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f776df9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f776806f3b0 0x7f77680b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f776df9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7768071be0 0x7f77680c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f776df9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77680b2010 0x7f77681f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f776df9b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680b5cb0 con 0x7f7768071be0 2023-10-16T11:01:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f776df9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680ad620 con 0x7f77680b2010 2023-10-16T11:01:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f776df9b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77680ad2f0 con 0x7f776806f3b0 2023-10-16T11:01:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7768071be0 0x7f77680c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7768071be0 0x7f77680c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52344/0 (socket says 172.21.15.181:52344) 2023-10-16T11:01:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 -- 172.21.15.181:0/2669819159 learned_addr learned my addr 172.21.15.181:0/2669819159 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7767fff700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77680b2010 0x7f77681f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f77677fe700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f776806f3b0 0x7f77680b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 -- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f776806f3b0 msgr2=0x7f77680b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f776806f3b0 0x7f77680b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 -- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77680b2010 msgr2=0x7f77681f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77680b2010 0x7f77681f3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7766ffd700 1 -- 172.21.15.181:0/2669819159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758009550 con 0x7f7768071be0 2023-10-16T11:01:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f7767fff700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77680b2010 0x7f77681f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.953+0000 7f77677fe700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f776806f3b0 0x7f77680b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:01:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.954+0000 7f7766ffd700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7768071be0 0x7f77680c9ce0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f775000e870 tx=0x7f775000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.954+0000 7f775ffff700 1 -- 172.21.15.181:0/2669819159 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750009590 con 0x7f7768071be0 2023-10-16T11:01:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.954+0000 7f775ffff700 1 -- 172.21.15.181:0/2669819159 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750013680 con 0x7f7768071be0 2023-10-16T11:01:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.954+0000 7f775ffff700 1 -- 172.21.15.181:0/2669819159 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77500139f0 con 0x7f7768071be0 2023-10-16T11:01:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.954+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77680c81a0 con 0x7f7768071be0 2023-10-16T11:01:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.954+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77680c8760 con 0x7f7768071be0 2023-10-16T11:01:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.955+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77680ad7d0 con 0x7f7768071be0 2023-10-16T11:01:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.955+0000 7f775ffff700 1 -- 172.21.15.181:0/2669819159 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 109) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7750013b50 con 0x7f7768071be0 2023-10-16T11:01:31.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.956+0000 7f775ffff700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 0x7f774806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:31.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.956+0000 7f77677fe700 1 -- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 msgr2=0x7f774806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:31.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.956+0000 7f775ffff700 1 -- 172.21.15.181:0/2669819159 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f7750089740 con 0x7f7768071be0 2023-10-16T11:01:31.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.956+0000 7f77677fe700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 0x7f774806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:31.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:31.960+0000 7f775ffff700 1 -- 172.21.15.181:0/2669819159 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f77680ad7d0 con 0x7f7768071be0 2023-10-16T11:01:32.016 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.015+0000 7f1bc6788200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:32.020 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.019+0000 7fcf409c7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:32.089 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.087+0000 7f1bc6788200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:01:32.094 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.093+0000 7fcf409c7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:01:32.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.157+0000 7f77677fe700 1 -- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 msgr2=0x7f774806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:32.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.157+0000 7f77677fe700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 0x7f774806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:32.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.164+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f77680ad790 con 0x7f7768071be0 2023-10-16T11:01:32.166 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.165+0000 7f1bc6788200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:01:32.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.168+0000 7f775ffff700 1 -- 172.21.15.181:0/2669819159 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7750057f80 con 0x7f7768071be0 2023-10-16T11:01:32.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 msgr2=0x7f774806d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:32.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 0x7f774806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7768071be0 msgr2=0x7f77680c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7768071be0 0x7f77680c9ce0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f775000e870 tx=0x7f775000eb80 comp rx=0 tx=0).stop 2023-10-16T11:01:32.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 shutdown_connections 2023-10-16T11:01:32.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f774806b070 0x7f774806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f776806f3b0 0x7f77680b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77680b2010 0x7f77681f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 --2- 172.21.15.181:0/2669819159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7768071be0 0x7f77680c9ce0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 >> 172.21.15.181:0/2669819159 conn(0x7f77680cd010 msgr2=0x7f77680b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:32.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 shutdown_connections 2023-10-16T11:01:32.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.170+0000 7f776df9b700 1 -- 172.21.15.181:0/2669819159 wait complete. 2023-10-16T11:01:32.175 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.173+0000 7fcf409c7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:01:32.184 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:01:32.308 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.306+0000 7f1bc6788200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:01:32.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.310+0000 7f7583acb700 1 Processor -- start 2023-10-16T11:01:32.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.311+0000 7f7583acb700 1 -- start start 2023-10-16T11:01:32.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.311+0000 7f7583acb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f757c072760 0x7f757c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.311+0000 7f7583acb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c06b210 con 0x7f757c15da70 2023-10-16T11:01:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.311+0000 7f7583acb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0cef00 con 0x7f757c06ade0 2023-10-16T11:01:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.311+0000 7f7583acb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c0cec20 con 0x7f757c072760 2023-10-16T11:01:32.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7581066700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f757c072760 0x7f757c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7581867700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f757c15da70 0x7f757c0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35242/0 (socket says 172.21.15.181:35242) 2023-10-16T11:01:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7581867700 1 -- 172.21.15.181:0/104746955 learned_addr learned my addr 172.21.15.181:0/104746955 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7581066700 1 -- 172.21.15.181:0/104746955 >> v1:172.21.15.190:6789/0 conn(0x7f757c06ade0 legacy=0x7f757c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7581066700 1 -- 172.21.15.181:0/104746955 >> v1:172.21.15.181:6789/0 conn(0x7f757c15da70 legacy=0x7f757c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7581066700 1 -- 172.21.15.181:0/104746955 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c071c80 con 0x7f757c072760 2023-10-16T11:01:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7580865700 1 -- 172.21.15.181:0/104746955 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133084926 0 0) 0x7f757c06b210 con 0x7f757c15da70 2023-10-16T11:01:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7581066700 1 --2- 172.21.15.181:0/104746955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c06a8a0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f756c009840 tx=0x7f756c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf615e984590c6cb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7580865700 1 -- 172.21.15.181:0/104746955 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c011830 con 0x7f757c072760 2023-10-16T11:01:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7580865700 1 -- 172.21.15.181:0/104746955 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c011990 con 0x7f757c072760 2023-10-16T11:01:32.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.312+0000 7f7580865700 1 -- 172.21.15.181:0/104746955 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c011d20 con 0x7f757c072760 2023-10-16T11:01:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.313+0000 7f7583acb700 1 -- 172.21.15.181:0/104746955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 msgr2=0x7f757c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.313+0000 7f7583acb700 1 --2- 172.21.15.181:0/104746955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c06a8a0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f756c009840 tx=0x7f756c009b50 comp rx=0 tx=0).stop 2023-10-16T11:01:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.313+0000 7f7583acb700 1 -- 172.21.15.181:0/104746955 shutdown_connections 2023-10-16T11:01:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.313+0000 7f7583acb700 1 --2- 172.21.15.181:0/104746955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c06a8a0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.313+0000 7f7583acb700 1 -- 172.21.15.181:0/104746955 >> 172.21.15.181:0/104746955 conn(0x7f757c0cd010 msgr2=0x7f757c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:32.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.313+0000 7f7583acb700 1 -- 172.21.15.181:0/104746955 shutdown_connections 2023-10-16T11:01:32.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.313+0000 7f7583acb700 1 -- 172.21.15.181:0/104746955 wait complete. 2023-10-16T11:01:32.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.314+0000 7f7583acb700 1 Processor -- start 2023-10-16T11:01:32.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.314+0000 7f7583acb700 1 -- start start 2023-10-16T11:01:32.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7583acb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f757c06ade0 0x7f757c0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7583acb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7583acb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f757c15da70 0x7f757c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7583acb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c15a900 con 0x7f757c072760 2023-10-16T11:01:32.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7583acb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b4c80 con 0x7f757c06ade0 2023-10-16T11:01:32.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7583acb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b4f60 con 0x7f757c15da70 2023-10-16T11:01:32.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c15a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52404/0 (socket says 172.21.15.181:52404) 2023-10-16T11:01:32.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 -- 172.21.15.181:0/221725841 learned_addr learned my addr 172.21.15.181:0/221725841 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:32.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581867700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f757c06ade0 0x7f757c0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7582068700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f757c15da70 0x7f757c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 -- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f757c15da70 msgr2=0x7f757c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f757c15da70 0x7f757c0b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 -- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f757c06ade0 msgr2=0x7f757c0b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f757c06ade0 0x7f757c0b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 -- 172.21.15.181:0/221725841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0094a0 con 0x7f757c072760 2023-10-16T11:01:32.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581867700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f757c06ade0 0x7f757c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:32.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f7581066700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c15a3c0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f756c0090b0 tx=0x7f756c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:32.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.315+0000 7f75727fc700 1 -- 172.21.15.181:0/221725841 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c01a850 con 0x7f757c072760 2023-10-16T11:01:32.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.316+0000 7f75727fc700 1 -- 172.21.15.181:0/221725841 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c01a9b0 con 0x7f757c072760 2023-10-16T11:01:32.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.316+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c0b5240 con 0x7f757c072760 2023-10-16T11:01:32.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.316+0000 7f75727fc700 1 -- 172.21.15.181:0/221725841 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c01ad40 con 0x7f757c072760 2023-10-16T11:01:32.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.316+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c0b5800 con 0x7f757c072760 2023-10-16T11:01:32.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.317+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c0b5990 con 0x7f757c072760 2023-10-16T11:01:32.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.317+0000 7f75727fc700 1 -- 172.21.15.181:0/221725841 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 109) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f756c022070 con 0x7f757c072760 2023-10-16T11:01:32.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.317+0000 7f75727fc700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 0x7f756006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.317+0000 7f7581867700 1 -- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 msgr2=0x7f756006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:32.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.318+0000 7f75727fc700 1 -- 172.21.15.181:0/221725841 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f756c08ac30 con 0x7f757c072760 2023-10-16T11:01:32.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.318+0000 7f7581867700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 0x7f756006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:32.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.322+0000 7f75727fc700 1 -- 172.21.15.181:0/221725841 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f756c055b90 con 0x7f757c072760 2023-10-16T11:01:32.329 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.327+0000 7fcf409c7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:01:32.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.519+0000 7f7581867700 1 -- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 msgr2=0x7f756006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:32.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.519+0000 7f7581867700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 0x7f756006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:32.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.535+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f757c067ca0 con 0x7f757c072760 2023-10-16T11:01:32.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.538+0000 7f75727fc700 1 -- 172.21.15.181:0/221725841 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7f756c0290c0 con 0x7f757c072760 2023-10-16T11:01:32.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 msgr2=0x7f756006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 0x7f756006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 msgr2=0x7f757c15a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c15a3c0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f756c0090b0 tx=0x7f756c004750 comp rx=0 tx=0).stop 2023-10-16T11:01:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 shutdown_connections 2023-10-16T11:01:32.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f756006b070 0x7f756006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f757c15da70 0x7f757c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f757c06ade0 0x7f757c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 --2- 172.21.15.181:0/221725841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c072760 0x7f757c15a3c0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.540+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 >> 172.21.15.181:0/221725841 conn(0x7f757c0cd010 msgr2=0x7f757c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:32.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.541+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 shutdown_connections 2023-10-16T11:01:32.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.541+0000 7f7583acb700 1 -- 172.21.15.181:0/221725841 wait complete. 2023-10-16T11:01:32.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:32.556 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:01:32.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:01:32.577 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-16T11:01:32.635 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1m 2023-10-16T11:01:32.636 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.634+0000 7f1bc6788200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:01:32.673 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.671+0000 7fcf409c7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:01:32.757 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.751+0000 7f1bc6788200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:01:32.795 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.794+0000 7fcf409c7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:01:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.798+0000 7f91a96ad700 1 Processor -- start 2023-10-16T11:01:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.798+0000 7f91a96ad700 1 -- start start 2023-10-16T11:01:32.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.799+0000 7f91a96ad700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f91a40cb690 0x7f91a406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.799+0000 7f91a96ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a4070590 con 0x7f91a40b5fd0 2023-10-16T11:01:32.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.799+0000 7f91a96ad700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a4156330 con 0x7f91a40cb2b0 2023-10-16T11:01:32.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.799+0000 7f91a96ad700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a406fcf0 con 0x7f91a40cb690 2023-10-16T11:01:32.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.799+0000 7f91a37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f91a40b5fd0 0x7f91a415f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35306/0 (socket says 172.21.15.181:35306) 2023-10-16T11:01:32.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.799+0000 7f91a37fe700 1 -- 172.21.15.181:0/3068873409 learned_addr learned my addr 172.21.15.181:0/3068873409 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:32.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3068873409 >> v2:172.21.15.190:3300/0 conn(0x7f91a40cb690 0x7f91a406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a27fc700 1 -- 172.21.15.181:0/3068873409 >> v1:172.21.15.190:6789/0 conn(0x7f91a40cb2b0 legacy=0x7f91a40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a1ffb700 1 -- 172.21.15.181:0/3068873409 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960030885 0 0) 0x7f91a4070590 con 0x7f91a40b5fd0 2023-10-16T11:01:32.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a27fc700 1 -- 172.21.15.181:0/3068873409 >> v1:172.21.15.181:6789/0 conn(0x7f91a40b5fd0 legacy=0x7f91a415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a27fc700 1 -- 172.21.15.181:0/3068873409 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a406ffd0 con 0x7f91a40cb690 2023-10-16T11:01:32.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3068873409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40cb690 0x7f91a406f710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f91980097c0 tx=0x7f9198009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2416b1040b5cc6e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:32.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a1ffb700 1 -- 172.21.15.181:0/3068873409 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198004230 con 0x7f91a40cb690 2023-10-16T11:01:32.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a1ffb700 1 -- 172.21.15.181:0/3068873409 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9198004390 con 0x7f91a40cb690 2023-10-16T11:01:32.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.800+0000 7f91a1ffb700 1 -- 172.21.15.181:0/3068873409 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91980036a0 con 0x7f91a40cb690 2023-10-16T11:01:32.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.801+0000 7f91a96ad700 1 -- 172.21.15.181:0/3068873409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40cb690 msgr2=0x7f91a406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.801+0000 7f91a96ad700 1 --2- 172.21.15.181:0/3068873409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40cb690 0x7f91a406f710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f91980097c0 tx=0x7f9198009ad0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.801+0000 7f91a96ad700 1 -- 172.21.15.181:0/3068873409 shutdown_connections 2023-10-16T11:01:32.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.801+0000 7f91a96ad700 1 --2- 172.21.15.181:0/3068873409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40cb690 0x7f91a406f710 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.801+0000 7f91a96ad700 1 -- 172.21.15.181:0/3068873409 >> 172.21.15.181:0/3068873409 conn(0x7f91a40cd010 msgr2=0x7f91a40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:32.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.801+0000 7f91a96ad700 1 -- 172.21.15.181:0/3068873409 shutdown_connections 2023-10-16T11:01:32.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.801+0000 7f91a96ad700 1 -- 172.21.15.181:0/3068873409 wait complete. 2023-10-16T11:01:32.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.802+0000 7f91a96ad700 1 Processor -- start 2023-10-16T11:01:32.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.802+0000 7f91a96ad700 1 -- start start 2023-10-16T11:01:32.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a96ad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40b5fd0 0x7f91a414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a96ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a40cb2b0 0x7f91a40cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a96ad700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a40cb690 0x7f91a40c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a96ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a40c4320 con 0x7f91a40cb2b0 2023-10-16T11:01:32.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a96ad700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a40ce910 con 0x7f91a40b5fd0 2023-10-16T11:01:32.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a96ad700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a40cebf0 con 0x7f91a40cb690 2023-10-16T11:01:32.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a40cb2b0 0x7f91a40cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a40cb2b0 0x7f91a40cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52464/0 (socket says 172.21.15.181:52464) 2023-10-16T11:01:32.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a27fc700 1 -- 172.21.15.181:0/3685938834 learned_addr learned my addr 172.21.15.181:0/3685938834 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:32.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40b5fd0 0x7f91a414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.803+0000 7f91a37fe700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a40cb690 0x7f91a40c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:32.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a2ffd700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a40cb690 msgr2=0x7f91a40c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a2ffd700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a40cb690 0x7f91a40c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a2ffd700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a40cb2b0 msgr2=0x7f91a40cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:32.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a2ffd700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91a40cb2b0 0x7f91a40cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:32.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a2ffd700 1 -- 172.21.15.181:0/3685938834 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91980094a0 con 0x7f91a40b5fd0 2023-10-16T11:01:32.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a2ffd700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40b5fd0 0x7f91a414b8c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f919400c750 tx=0x7f919400ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:32.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91940054e0 con 0x7f91a40b5fd0 2023-10-16T11:01:32.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9194003680 con 0x7f91a40b5fd0 2023-10-16T11:01:32.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91940039f0 con 0x7f91a40b5fd0 2023-10-16T11:01:32.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a40ceed0 con 0x7f91a40b5fd0 2023-10-16T11:01:32.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.804+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a40cf490 con 0x7f91a40b5fd0 2023-10-16T11:01:32.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.805+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a406f4f0 con 0x7f91a40b5fd0 2023-10-16T11:01:32.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.806+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 109) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9194003b50 con 0x7f91a40b5fd0 2023-10-16T11:01:32.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.806+0000 7f918b7fe700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 0x7f919006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:32.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.806+0000 7f91a27fc700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 msgr2=0x7f919006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:32.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.806+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6205+0+0 (secure 0 0 0) 0x7f9194088c60 con 0x7f91a40b5fd0 2023-10-16T11:01:32.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.806+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 0x7f919006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:01:32.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:32.811+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9194053b50 con 0x7f91a40b5fd0 2023-10-16T11:01:32.826 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.825+0000 7f1bc6788200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:01:32.874 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.872+0000 7fcf409c7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:01:32.907 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:32.906+0000 7f1bc6788200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:01:32.957 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:32.956+0000 7fcf409c7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:01:33.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:33.007+0000 7f91a27fc700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 msgr2=0x7f919006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:33.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:33.007+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 0x7f919006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:01:33.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:33.038+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 --> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f91a40b2140 con 0x7f919006b070 2023-10-16T11:01:33.275 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:33.274+0000 7f1bc6788200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:01:33.331 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:33.329+0000 7fcf409c7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:01:33.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:33.408+0000 7f91a27fc700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 msgr2=0x7f919006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:33.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:33.408+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 0x7f919006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:01:33.842 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:33.841+0000 7f1bc6788200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:01:33.899 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:33.897+0000 7fcf409c7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:01:33.912 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:33.911+0000 7f1bc6788200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:01:33.969 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:33.967+0000 7fcf409c7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:01:34.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:34.209+0000 7f91a27fc700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 msgr2=0x7f919006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:34.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:34.209+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 0x7f919006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:01:34.542 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:34.540+0000 7f1bc6788200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:34.601 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:34.599+0000 7fcf409c7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:34.620 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:34.619+0000 7f1bc6788200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:01:34.679 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:34.677+0000 7fcf409c7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:01:34.765 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:34.763+0000 7f1bc6788200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:34.823 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:34.822+0000 7fcf409c7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:01:34.994 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:34.993+0000 7f1bc6788200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:01:35.052 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:35.050+0000 7fcf409c7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:01:35.069 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:35.067+0000 7f1bc6788200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:01:35.126 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:35.124+0000 7fcf409c7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:01:35.336 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:35.335+0000 7f1bc6788200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:01:35.392 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:35.391+0000 7fcf409c7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:01:35.559 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:35.558+0000 7f1bc6788200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:01:35.614 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:35.613+0000 7fcf409c7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:01:35.709 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:35.708+0000 7f1bc6788200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:01:35.766 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:35.763+0000 7fcf409c7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:01:35.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:35.810+0000 7f91a27fc700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 msgr2=0x7f919006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/303240986 2023-10-16T11:01:35.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:35.810+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 0x7f919006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:01:35.822 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:35.820+0000 7f1bc6788200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:01:35.880 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:35.878+0000 7fcf409c7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:01:35.978 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:35.977+0000 7f1bc6788200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:01:36.038 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:36.036+0000 7fcf409c7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:01:36.240 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:36.239+0000 7f1bc6788200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:36.302 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:36.300+0000 7fcf409c7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:01:36.400 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:36.399+0000 7f1bc6788200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:01:36.466 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:36.464+0000 7fcf409c7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:01:36.924 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:01:36.923+0000 7f1bc6788200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:01:36.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:36.994+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 110) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f91940bbfe0 con 0x7f91a40b5fd0 2023-10-16T11:01:36.996 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:36.994+0000 7fcf409c7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:01:37.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:37.016+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mgrmap(e 111) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f9194053f20 con 0x7f91a40b5fd0 2023-10-16T11:01:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:37.016+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 msgr2=0x7f919006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:01:37.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:37.016+0000 7f918b7fe700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/303240986,v1:172.21.15.181:6833/303240986] conn(0x7f919006b070 0x7f919006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:37.169 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb69a8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.169 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb69a8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.169 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb69a8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.170 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb69a8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.170 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb69a8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.170 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb39a2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.170 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb39a2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.170 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb39a2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.171 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb39a2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:37.171 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:01:37.167+0000 7fceb39a2700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:01:38.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.020+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mon.1 v2:172.21.15.190:3300/0 9 ==== mgrmap(e 112) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f91940547f0 con 0x7f91a40b5fd0 2023-10-16T11:01:38.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.020+0000 7f918b7fe700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f919001d250 0x7f9190075620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:38.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.020+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 --> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9190071070 con 0x7f919001d250 2023-10-16T11:01:38.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.022+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f919001d250 0x7f9190075620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:38.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.024+0000 7f91a27fc700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f919001d250 0x7f9190075620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91a41f6940 tx=0x7f9198008030 comp rx=0 tx=0).ready entity=mgr.10362 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:38.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.086+0000 7f918b7fe700 1 -- 172.21.15.181:0/3685938834 <== mgr.10362 v2:172.21.15.181:6832/3601081788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9190071070 con 0x7f919001d250 2023-10-16T11:01:38.088 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-16T11:01:38.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f919001d250 msgr2=0x7f9190075620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f919001d250 0x7f9190075620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91a41f6940 tx=0x7f9198008030 comp rx=0 tx=0).stop 2023-10-16T11:01:38.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40b5fd0 msgr2=0x7f91a414b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 --2- 172.21.15.181:0/3685938834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a40b5fd0 0x7f91a414b8c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f919400c750 tx=0x7f919400ca60 comp rx=0 tx=0).stop 2023-10-16T11:01:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a37fe700 1 -- 172.21.15.181:0/3685938834 reap_dead start 2023-10-16T11:01:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 shutdown_connections 2023-10-16T11:01:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 >> 172.21.15.181:0/3685938834 conn(0x7f91a40cd010 msgr2=0x7f91a415e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:38.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 shutdown_connections 2023-10-16T11:01:38.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.088+0000 7f91a96ad700 1 -- 172.21.15.181:0/3685938834 wait complete. 2023-10-16T11:01:38.103 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-16T11:01:38.104 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1m 2023-10-16T11:01:38.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.226+0000 7f3c9c5af700 1 Processor -- start 2023-10-16T11:01:38.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.226+0000 7f3c9c5af700 1 -- start start 2023-10-16T11:01:38.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9c5af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c9415dbd0 0x7f3c940d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:38.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9c5af700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c940ce490 0x7f3c9406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:38.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9c5af700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9415dfb0 con 0x7f3c940d0610 2023-10-16T11:01:38.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9c5af700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c940cd3a0 con 0x7f3c9415dbd0 2023-10-16T11:01:38.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9c5af700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c940cd830 con 0x7f3c940ce490 2023-10-16T11:01:38.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9a34b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c9415dbd0 0x7f3c940d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:38.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c99b4a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3c940d0610 0x7f3c940cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35530/0 (socket says 172.21.15.181:35530) 2023-10-16T11:01:38.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c99b4a700 1 -- 172.21.15.181:0/1287652041 learned_addr learned my addr 172.21.15.181:0/1287652041 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:01:38.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9ab4c700 1 --2- 172.21.15.181:0/1287652041 >> v2:172.21.15.190:3300/0 conn(0x7f3c940ce490 0x7f3c9406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:38.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9a34b700 1 -- 172.21.15.181:0/1287652041 >> v2:172.21.15.190:3300/0 conn(0x7f3c940ce490 msgr2=0x7f3c9406b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:01:38.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9a34b700 1 --2- 172.21.15.181:0/1287652041 >> v2:172.21.15.190:3300/0 conn(0x7f3c940ce490 0x7f3c9406b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9a34b700 1 -- 172.21.15.181:0/1287652041 >> v1:172.21.15.181:6789/0 conn(0x7f3c940d0610 legacy=0x7f3c940cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9a34b700 1 -- 172.21.15.181:0/1287652041 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940d1fc0 con 0x7f3c9415dbd0 2023-10-16T11:01:38.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c99349700 1 -- 172.21.15.181:0/1287652041 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544077456 0 0) 0x7f3c84001110 con 0x7f3c940d0610 2023-10-16T11:01:38.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c9a34b700 1 --2- 172.21.15.181:0/1287652041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c9415dbd0 0x7f3c940d00d0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f3c94072600 tx=0x7f3c90009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba2a4cbd75bcacf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:38.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c99349700 1 -- 172.21.15.181:0/1287652041 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90011830 con 0x7f3c9415dbd0 2023-10-16T11:01:38.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c99349700 1 -- 172.21.15.181:0/1287652041 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c90011990 con 0x7f3c9415dbd0 2023-10-16T11:01:38.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.227+0000 7f3c99349700 1 -- 172.21.15.181:0/1287652041 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90011d20 con 0x7f3c9415dbd0 2023-10-16T11:01:38.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1287652041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c9415dbd0 msgr2=0x7f3c940d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1287652041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c9415dbd0 0x7f3c940d00d0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f3c94072600 tx=0x7f3c90009b20 comp rx=0 tx=0).stop 2023-10-16T11:01:38.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1287652041 shutdown_connections 2023-10-16T11:01:38.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1287652041 >> v2:172.21.15.190:3300/0 conn(0x7f3c940ce490 0x7f3c9406b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1287652041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c9415dbd0 0x7f3c940d00d0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1287652041 >> 172.21.15.181:0/1287652041 conn(0x7f3c9409a930 msgr2=0x7f3c940d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:38.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1287652041 shutdown_connections 2023-10-16T11:01:38.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.228+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1287652041 wait complete. 2023-10-16T11:01:38.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.229+0000 7f3c9c5af700 1 Processor -- start 2023-10-16T11:01:38.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.229+0000 7f3c9c5af700 1 -- start start 2023-10-16T11:01:38.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9c5af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c940ce490 0x7f3c94160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:38.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9c5af700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c940d0610 0x7f3c940ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:38.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9c5af700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c9415dbd0 0x7f3c940ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:38.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9c5af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b5d80 con 0x7f3c940ce490 2023-10-16T11:01:38.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9c5af700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940cb240 con 0x7f3c9415dbd0 2023-10-16T11:01:38.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9c5af700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b6020 con 0x7f3c940d0610 2023-10-16T11:01:38.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c940ce490 0x7f3c94160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:38.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c940ce490 0x7f3c94160800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52690/0 (socket says 172.21.15.181:52690) 2023-10-16T11:01:38.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 -- 172.21.15.181:0/1677754251 learned_addr learned my addr 172.21.15.181:0/1677754251 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:01:38.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9ab4c700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c9415dbd0 0x7f3c940ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:38.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c99b4a700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c940d0610 0x7f3c940ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:38.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 -- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c940d0610 msgr2=0x7f3c940ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c940d0610 0x7f3c940ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 -- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c9415dbd0 msgr2=0x7f3c940ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c9415dbd0 0x7f3c940ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c9a34b700 1 -- 172.21.15.181:0/1677754251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900094a0 con 0x7f3c940ce490 2023-10-16T11:01:38.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.230+0000 7f3c99b4a700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c940d0610 0x7f3c940ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:01:38.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.231+0000 7f3c9a34b700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c940ce490 0x7f3c94160800 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f3c900090b0 tx=0x7f3c90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:38.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.231+0000 7f3c8affd700 1 -- 172.21.15.181:0/1677754251 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c900036a0 con 0x7f3c940ce490 2023-10-16T11:01:38.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.231+0000 7f3c8affd700 1 -- 172.21.15.181:0/1677754251 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9001e070 con 0x7f3c940ce490 2023-10-16T11:01:38.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.231+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c940b6300 con 0x7f3c940ce490 2023-10-16T11:01:38.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.231+0000 7f3c8affd700 1 -- 172.21.15.181:0/1677754251 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90003a00 con 0x7f3c940ce490 2023-10-16T11:01:38.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.231+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c94160f00 con 0x7f3c940ce490 2023-10-16T11:01:38.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.232+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c94067d20 con 0x7f3c940ce490 2023-10-16T11:01:38.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.232+0000 7f3c8affd700 1 -- 172.21.15.181:0/1677754251 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 112) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3c90003b60 con 0x7f3c940ce490 2023-10-16T11:01:38.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.232+0000 7f3c8affd700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f3c7806ada0 0x7f3c7806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:01:38.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.232+0000 7f3c8affd700 1 -- 172.21.15.181:0/1677754251 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f3c9008a6b0 con 0x7f3c940ce490 2023-10-16T11:01:38.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.232+0000 7f3c99b4a700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f3c7806ada0 0x7f3c7806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:01:38.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.233+0000 7f3c99b4a700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f3c7806ada0 0x7f3c7806d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3c941584f0 tx=0x7f3c84007ae0 comp rx=0 tx=0).ready entity=mgr.10362 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:01:38.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.237+0000 7f3c8affd700 1 -- 172.21.15.181:0/1677754251 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3c90055750 con 0x7f3c940ce490 2023-10-16T11:01:38.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.457+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 --> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c940cca70 con 0x7f3c7806ada0 2023-10-16T11:01:38.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.510+0000 7f3c8affd700 1 -- 172.21.15.181:0/1677754251 <== mgr.10362 v2:172.21.15.181:6832/3601081788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f3c840059a0 con 0x7f3c7806ada0 2023-10-16T11:01:38.512 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-16T11:01:38.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f3c7806ada0 msgr2=0x7f3c7806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f3c7806ada0 0x7f3c7806d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3c941584f0 tx=0x7f3c84007ae0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c940ce490 msgr2=0x7f3c94160800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:01:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c940ce490 0x7f3c94160800 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f3c900090b0 tx=0x7f3c90004750 comp rx=0 tx=0).stop 2023-10-16T11:01:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 shutdown_connections 2023-10-16T11:01:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f3c7806ada0 0x7f3c7806d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c9415dbd0 0x7f3c940ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c940ce490 0x7f3c94160800 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 --2- 172.21.15.181:0/1677754251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c940d0610 0x7f3c940ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:01:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 >> 172.21.15.181:0/1677754251 conn(0x7f3c9409a930 msgr2=0x7f3c940cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:01:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 shutdown_connections 2023-10-16T11:01:38.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:01:38.512+0000 7f3c9c5af700 1 -- 172.21.15.181:0/1677754251 wait complete. 2023-10-16T11:01:38.526 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-16T11:01:52.149 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:01:52.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:00.523 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:02:00.522+0000 7fcea5946700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:02:00.524 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:02:00.522+0000 7fcea5946700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:02:00.524 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:02:00.522+0000 7fcea5946700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:02:00.524 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:02:00.522+0000 7fcea5946700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:02:00.524 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:02:00.522+0000 7fcea5946700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:02:22.474 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:22.478 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:52.613 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:02:52.617 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:22.722 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:22.726 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:52.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:03:52.864 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:23.013 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:23.016 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:53.187 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:04:53.191 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:23.317 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:23.320 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:53.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:05:53.451 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:06:23.617 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:06:23.620 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:06:53.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:06:53.770 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:23.910 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:23.914 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:54.049 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:07:54.053 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:24.189 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:24.193 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:54.286 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:08:54.290 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:24.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:24.451 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:54.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:09:54.608 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:24.760 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:24.764 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:54.921 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:10:54.924 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:25.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:25.046 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:55.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:11:55.201 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:12:15.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:12:15.529 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-16T11:12:15.552 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_02_00_UTC 2023-10-16T11:12:15.553 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_03_00_UTC 2023-10-16T11:12:15.553 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_04_00_UTC 2023-10-16T11:12:15.553 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_05_00_UTC 2023-10-16T11:12:15.553 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_06_00_UTC 2023-10-16T11:12:15.553 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:12:15.554 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:12:15.554 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:12:15.554 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:12:15.554 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:12:15.554 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:12:15.555 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-11_02_00_UTC', 'scheduled-2023-10-16-11_03_00_UTC', 'scheduled-2023-10-16-11_04_00_UTC', 'scheduled-2023-10-16-11_05_00_UTC', 'scheduled-2023-10-16-11_06_00_UTC', 'scheduled-2023-10-16-11_07_00_UTC', 'scheduled-2023-10-16-11_08_00_UTC', 'scheduled-2023-10-16-11_09_00_UTC', 'scheduled-2023-10-16-11_10_00_UTC', 'scheduled-2023-10-16-11_11_00_UTC', 'scheduled-2023-10-16-11_12_00_UTC'] 2023-10-16T11:12:15.555 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-16T11:12:15.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.726+0000 7f8ce0158700 1 Processor -- start 2023-10-16T11:12:15.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.726+0000 7f8ce0158700 1 -- start start 2023-10-16T11:12:15.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8ce0158700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 0x7f8cd80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8ce0158700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80cef00 con 0x7f8cd80ce6a0 2023-10-16T11:12:15.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8ce0158700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80cec20 con 0x7f8cd806afb0 2023-10-16T11:12:15.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8ce0158700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd8071c80 con 0x7f8cd815da70 2023-10-16T11:12:15.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8cdd6f3700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8cd80ce6a0 0x7f8cd806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35574/0 (socket says 172.21.15.181:35574) 2023-10-16T11:12:15.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8cdd6f3700 1 -- 172.21.15.181:0/1648340549 learned_addr learned my addr 172.21.15.181:0/1648340549 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:12:15.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8cddef4700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 0x7f8cd80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201988104 0 0) 0x7f8cd80cef00 con 0x7f8cd80ce6a0 2023-10-16T11:12:15.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.727+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc40035b0 con 0x7f8cd80ce6a0 2023-10-16T11:12:15.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3415193208 0 0) 0x7f8cc40035b0 con 0x7f8cd80ce6a0 2023-10-16T11:12:15.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 msgr2=0x7f8cd80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:12:15.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 0x7f8cd80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 >> v1:172.21.15.190:6789/0 conn(0x7f8cd806afb0 legacy=0x7f8cd81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:15.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd8071f60 con 0x7f8cd80ce6a0 2023-10-16T11:12:15.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f8cd4001670 con 0x7f8cd80ce6a0 2023-10-16T11:12:15.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 >> v1:172.21.15.181:6789/0 conn(0x7f8cd80ce6a0 legacy=0x7f8cd806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:15.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc4003a30 0x7f8cc4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 0x7f8cc4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4004d90 0x7f8cc4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd8071c80 con 0x7f8cc4003a30 2023-10-16T11:12:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400bbd0 con 0x7f8cc4004d90 2023-10-16T11:12:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400bd90 con 0x7f8cd815da70 2023-10-16T11:12:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cddef4700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc4003a30 0x7f8cc4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cdd6f3700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 0x7f8cc4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cde6f5700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4004d90 0x7f8cc4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cddef4700 1 -- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 msgr2=0x7f8cc4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:12:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cddef4700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd815da70 0x7f8cc4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cddef4700 1 -- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4004d90 msgr2=0x7f8cc4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:12:15.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cddef4700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4004d90 0x7f8cc4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cddef4700 1 -- 172.21.15.181:0/1648340549 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40035b0 con 0x7f8cc4003a30 2023-10-16T11:12:15.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cde6f5700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4004d90 0x7f8cc4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:12:15.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.728+0000 7f8cddef4700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc4003a30 0x7f8cc4003e50 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009b60 tx=0x7f8cc4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b697ba006c14c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:12:15.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800e6c0 con 0x7f8cc4003a30 2023-10-16T11:12:15.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc800e820 con 0x7f8cc4003a30 2023-10-16T11:12:15.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8cdcef2700 1 -- 172.21.15.181:0/1648340549 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800eb90 con 0x7f8cc4003a30 2023-10-16T11:12:15.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8ce0158700 1 -- 172.21.15.181:0/1648340549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc4003a30 msgr2=0x7f8cc4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:15.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8ce0158700 1 --2- 172.21.15.181:0/1648340549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc4003a30 0x7f8cc4003e50 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009b60 tx=0x7f8cc4005880 comp rx=0 tx=0).stop 2023-10-16T11:12:15.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8cde6f5700 1 -- 172.21.15.181:0/1648340549 reap_dead start 2023-10-16T11:12:15.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8ce0158700 1 -- 172.21.15.181:0/1648340549 shutdown_connections 2023-10-16T11:12:15.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8ce0158700 1 -- 172.21.15.181:0/1648340549 >> 172.21.15.181:0/1648340549 conn(0x7f8cd80cd010 msgr2=0x7f8cd80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:12:15.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8ce0158700 1 -- 172.21.15.181:0/1648340549 shutdown_connections 2023-10-16T11:12:15.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.729+0000 7f8ce0158700 1 -- 172.21.15.181:0/1648340549 wait complete. 2023-10-16T11:12:15.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.731+0000 7f8ce0158700 1 Processor -- start 2023-10-16T11:12:15.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.731+0000 7f8ce0158700 1 -- start start 2023-10-16T11:12:15.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8ce0158700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4003a30 0x7f8cd80b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8ce0158700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd81470b0 0x7f8cd80c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8ce0158700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd80c70c0 0x7f8cd80c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8ce0158700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80b5720 con 0x7f8cd81470b0 2023-10-16T11:12:15.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8ce0158700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80b60f0 con 0x7f8cc4003a30 2023-10-16T11:12:15.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8ce0158700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80b4b10 con 0x7f8cd80c70c0 2023-10-16T11:12:15.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd81470b0 0x7f8cd80c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd81470b0 0x7f8cd80c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52740/0 (socket says 172.21.15.181:52740) 2023-10-16T11:12:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 -- 172.21.15.181:0/698030335 learned_addr learned my addr 172.21.15.181:0/698030335 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:12:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 -- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd80c70c0 msgr2=0x7f8cd80c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cddef4700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4003a30 0x7f8cd80b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cde6f5700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd80c70c0 0x7f8cd80c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd80c70c0 0x7f8cd80c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 -- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4003a30 msgr2=0x7f8cd80b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4003a30 0x7f8cd80b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 -- 172.21.15.181:0/698030335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80b4df0 con 0x7f8cd81470b0 2023-10-16T11:12:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cddef4700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4003a30 0x7f8cd80b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:12:15.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cdd6f3700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd81470b0 0x7f8cd80c6b80 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40072c0 tx=0x7f8cd4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:12:15.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.732+0000 7f8cce7fc700 1 -- 172.21.15.181:0/698030335 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40077a0 con 0x7f8cd81470b0 2023-10-16T11:12:15.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.733+0000 7f8cce7fc700 1 -- 172.21.15.181:0/698030335 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd400f8d0 con 0x7f8cd81470b0 2023-10-16T11:12:15.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.733+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd80b50d0 con 0x7f8cd81470b0 2023-10-16T11:12:15.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.733+0000 7f8cce7fc700 1 -- 172.21.15.181:0/698030335 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400fc80 con 0x7f8cd81470b0 2023-10-16T11:12:15.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.733+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd80b47a0 con 0x7f8cd81470b0 2023-10-16T11:12:15.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.733+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd806a8a0 con 0x7f8cd81470b0 2023-10-16T11:12:15.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.734+0000 7f8cce7fc700 1 -- 172.21.15.181:0/698030335 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 116) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f8cd401d460 con 0x7f8cd81470b0 2023-10-16T11:12:15.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.734+0000 7f8cce7fc700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f8cbc06b070 0x7f8cbc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:15.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.734+0000 7f8cce7fc700 1 -- 172.21.15.181:0/698030335 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f8cd4089bd0 con 0x7f8cd81470b0 2023-10-16T11:12:15.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.734+0000 7f8cddef4700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f8cbc06b070 0x7f8cbc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:15.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.735+0000 7f8cddef4700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f8cbc06b070 0x7f8cbc06d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80090b0 tx=0x7f8cc8008030 comp rx=0 tx=0).ready entity=mgr.10362 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:12:15.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.739+0000 7f8cce7fc700 1 -- 172.21.15.181:0/698030335 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8cd40548d0 con 0x7f8cd81470b0 2023-10-16T11:12:15.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.956+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 --> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cd80c5f90 con 0x7f8cbc06b070 2023-10-16T11:12:15.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.963+0000 7f8cce7fc700 1 -- 172.21.15.181:0/698030335 <== mgr.10362 v2:172.21.15.181:6832/3601081788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+375 (secure 0 0 0) 0x7f8cd80c5f90 con 0x7f8cbc06b070 2023-10-16T11:12:15.966 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:12:15.966 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:01:38", "first": "2023-10-16T11:02:00", "last": "2023-10-16T11:12:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-16T11:12:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f8cbc06b070 msgr2=0x7f8cbc06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f8cbc06b070 0x7f8cbc06d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80090b0 tx=0x7f8cc8008030 comp rx=0 tx=0).stop 2023-10-16T11:12:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd81470b0 msgr2=0x7f8cd80c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:15.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd81470b0 0x7f8cd80c6b80 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40072c0 tx=0x7f8cd4007680 comp rx=0 tx=0).stop 2023-10-16T11:12:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 shutdown_connections 2023-10-16T11:12:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7f8cbc06b070 0x7f8cbc06d530 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cd80c70c0 0x7f8cd80c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8cc4003a30 0x7f8cd80b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 --2- 172.21.15.181:0/698030335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd81470b0 0x7f8cd80c6b80 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:15.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 >> 172.21.15.181:0/698030335 conn(0x7f8cd80cd010 msgr2=0x7f8cd806f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:12:15.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 shutdown_connections 2023-10-16T11:12:15.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:15.966+0000 7f8ce0158700 1 -- 172.21.15.181:0/698030335 wait complete. 2023-10-16T11:12:15.979 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:01:38", "first": "2023-10-16T11:02:00", "last": "2023-10-16T11:12:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-16T11:12:15.980 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T11:01:38', 'first': '2023-10-16T11:02:00', 'last': '2023-10-16T11:12:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-16T11:12:15.980 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-16T11:12:15.981 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-16T11:12:15.981 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-16T11:12:16.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.100+0000 7fbd7de7b700 1 Processor -- start 2023-10-16T11:12:16.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.100+0000 7fbd7de7b700 1 -- start start 2023-10-16T11:12:16.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd7de7b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd78159730 0x7fbd7815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:16.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd7de7b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd780cee70 con 0x7fbd780cd5c0 2023-10-16T11:12:16.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd7de7b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd78159b10 con 0x7fbd780cc400 2023-10-16T11:12:16.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd7de7b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd780ce540 con 0x7fbd78159730 2023-10-16T11:12:16.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd777fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd78159730 0x7fbd7815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd777fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd78159730 0x7fbd7815cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52746/0 (socket says 172.21.15.181:52746) 2023-10-16T11:12:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd777fe700 1 -- 172.21.15.181:0/2623937712 learned_addr learned my addr 172.21.15.181:0/2623937712 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:12:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd777fe700 1 -- 172.21.15.181:0/2623937712 >> v1:172.21.15.190:6789/0 conn(0x7fbd780cc400 legacy=0x7fbd780cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:12:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd777fe700 1 -- 172.21.15.181:0/2623937712 >> v1:172.21.15.181:6789/0 conn(0x7fbd780cd5c0 legacy=0x7fbd780cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd777fe700 1 -- 172.21.15.181:0/2623937712 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd780ce820 con 0x7fbd78159730 2023-10-16T11:12:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd767fc700 1 -- 172.21.15.181:0/2623937712 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884076643 0 0) 0x7fbd780cee70 con 0x7fbd780cd5c0 2023-10-16T11:12:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.101+0000 7fbd777fe700 1 --2- 172.21.15.181:0/2623937712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd78159730 0x7fbd7815cd20 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c006fd0 tx=0x7fbd6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd77e05dbcb5539 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:12:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd767fc700 1 -- 172.21.15.181:0/2623937712 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c005e30 con 0x7fbd78159730 2023-10-16T11:12:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd767fc700 1 -- 172.21.15.181:0/2623937712 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c006030 con 0x7fbd78159730 2023-10-16T11:12:16.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd767fc700 1 -- 172.21.15.181:0/2623937712 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0063e0 con 0x7fbd78159730 2023-10-16T11:12:16.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd7de7b700 1 -- 172.21.15.181:0/2623937712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd78159730 msgr2=0x7fbd7815cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:16.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/2623937712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd78159730 0x7fbd7815cd20 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c006fd0 tx=0x7fbd6c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd7de7b700 1 -- 172.21.15.181:0/2623937712 shutdown_connections 2023-10-16T11:12:16.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/2623937712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd78159730 0x7fbd7815cd20 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd7de7b700 1 -- 172.21.15.181:0/2623937712 >> 172.21.15.181:0/2623937712 conn(0x7fbd780d04a0 msgr2=0x7fbd780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:12:16.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd7de7b700 1 -- 172.21.15.181:0/2623937712 shutdown_connections 2023-10-16T11:12:16.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.102+0000 7fbd7de7b700 1 -- 172.21.15.181:0/2623937712 wait complete. 2023-10-16T11:12:16.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.103+0000 7fbd7de7b700 1 Processor -- start 2023-10-16T11:12:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.103+0000 7fbd7de7b700 1 -- start start 2023-10-16T11:12:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd7de7b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd780cc400 0x7fbd78156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd7de7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd780cd5c0 0x7fbd780c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd7de7b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78159730 0x7fbd780ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:16.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd7de7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd78157170 con 0x7fbd780cd5c0 2023-10-16T11:12:16.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd7de7b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780c9230 con 0x7fbd78159730 2023-10-16T11:12:16.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd7de7b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780c9510 con 0x7fbd780cc400 2023-10-16T11:12:16.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd76ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd780cd5c0 0x7fbd780c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:16.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd76ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd780cd5c0 0x7fbd780c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:52752/0 (socket says 172.21.15.181:52752) 2023-10-16T11:12:16.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd76ffd700 1 -- 172.21.15.181:0/503536481 learned_addr learned my addr 172.21.15.181:0/503536481 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:12:16.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd777fe700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd780cc400 0x7fbd78156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:16.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.104+0000 7fbd77fff700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78159730 0x7fbd780ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:16.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd76ffd700 1 -- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd780cc400 msgr2=0x7fbd78156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:16.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd76ffd700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd780cc400 0x7fbd78156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd76ffd700 1 -- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78159730 msgr2=0x7fbd780ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:16.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd76ffd700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78159730 0x7fbd780ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd76ffd700 1 -- 172.21.15.181:0/503536481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c009500 con 0x7fbd780cd5c0 2023-10-16T11:12:16.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd77fff700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78159730 0x7fbd780ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:12:16.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd76ffd700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd780cd5c0 0x7fbd780c9f50 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800c4a0 tx=0x7fbd6800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:12:16.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd5ffff700 1 -- 172.21.15.181:0/503536481 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd680054e0 con 0x7fbd780cd5c0 2023-10-16T11:12:16.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd5ffff700 1 -- 172.21.15.181:0/503536481 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd68003680 con 0x7fbd780cd5c0 2023-10-16T11:12:16.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd780c97f0 con 0x7fbd780cd5c0 2023-10-16T11:12:16.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd5ffff700 1 -- 172.21.15.181:0/503536481 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003a30 con 0x7fbd780cd5c0 2023-10-16T11:12:16.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.105+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7809b900 con 0x7fbd780cd5c0 2023-10-16T11:12:16.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.106+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7809b760 con 0x7fbd780cd5c0 2023-10-16T11:12:16.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.106+0000 7fbd5ffff700 1 -- 172.21.15.181:0/503536481 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 116) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fbd68003b90 con 0x7fbd780cd5c0 2023-10-16T11:12:16.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.107+0000 7fbd5ffff700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7fbd6406b0a0 0x7fbd6406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:12:16.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.107+0000 7fbd5ffff700 1 -- 172.21.15.181:0/503536481 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fbd68088ac0 con 0x7fbd780cd5c0 2023-10-16T11:12:16.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.107+0000 7fbd777fe700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7fbd6406b0a0 0x7fbd6406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:12:16.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.107+0000 7fbd777fe700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7fbd6406b0a0 0x7fbd6406d560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c013040 tx=0x7fbd6c008070 comp rx=0 tx=0).ready entity=mgr.10362 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:12:16.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.111+0000 7fbd5ffff700 1 -- 172.21.15.181:0/503536481 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbd7809b760 con 0x7fbd780cd5c0 2023-10-16T11:12:16.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.365+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbd7809b760 con 0x7fbd780cd5c0 2023-10-16T11:12:16.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.368+0000 7fbd5ffff700 1 -- 172.21.15.181:0/503536481 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v116) v1 ==== 74+0+171247 (secure 0 0 0) 0x7fbd7809b760 con 0x7fbd780cd5c0 2023-10-16T11:12:16.372 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:12:16.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7fbd6406b0a0 msgr2=0x7fbd6406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:16.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7fbd6406b0a0 0x7fbd6406d560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c013040 tx=0x7fbd6c008070 comp rx=0 tx=0).stop 2023-10-16T11:12:16.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd780cd5c0 msgr2=0x7fbd780c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:12:16.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd780cd5c0 0x7fbd780c9f50 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800c4a0 tx=0x7fbd6800c7b0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 shutdown_connections 2023-10-16T11:12:16.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:6832/3601081788,v1:172.21.15.181:6833/3601081788] conn(0x7fbd6406b0a0 0x7fbd6406d560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd780cc400 0x7fbd78156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78159730 0x7fbd780ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 --2- 172.21.15.181:0/503536481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd780cd5c0 0x7fbd780c9f50 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:12:16.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 >> 172.21.15.181:0/503536481 conn(0x7fbd780d04a0 msgr2=0x7fbd780cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:12:16.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 shutdown_connections 2023-10-16T11:12:16.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:12:16.371+0000 7fbd7de7b700 1 -- 172.21.15.181:0/503536481 wait complete. 2023-10-16T11:12:16.384 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":116,"active_gid":10362,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":3601081788},{"type":"v1","addr":"172.21.15.181:6833","nonce":3601081788}]},"active_addr":"172.21.15.181:6833/3601081788","active_change":"2023-10-16T11:01:37.009279+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":9169,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":182,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":4003028162}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1499024696}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3290548668}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2238590139}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1195985489}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":231495737}]}]} 2023-10-16T11:12:16.389 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-10-16T11:12:16.389 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-16T11:12:16.390 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-10-16T11:12:16.390 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-16T11:12:16.390 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:12:16.419 INFO:tasks.ceph.mgr.y:Stopped 2023-10-16T11:12:16.419 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-16T11:12:16.461 INFO:tasks.ceph.mgr.y:Started 2023-10-16T11:12:16.634 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:16.632+0000 7f6c4b2aa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:12:16.700 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:16.698+0000 7f6c4b2aa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:12:16.771 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:16.769+0000 7f6c4b2aa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:12:16.848 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:16.846+0000 7f6c4b2aa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:12:16.990 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:16.988+0000 7f6c4b2aa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:12:17.312 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:17.310+0000 7f6c4b2aa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:12:17.427 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:17.425+0000 7f6c4b2aa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:12:17.500 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:17.498+0000 7f6c4b2aa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:12:17.579 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:17.577+0000 7f6c4b2aa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:12:17.949 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:17.946+0000 7f6c4b2aa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:12:18.518 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:18.516+0000 7f6c4b2aa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:12:18.587 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:18.586+0000 7f6c4b2aa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:12:19.219 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:19.217+0000 7f6c4b2aa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:12:19.297 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:19.295+0000 7f6c4b2aa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:12:19.442 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:19.440+0000 7f6c4b2aa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:12:19.672 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:19.671+0000 7f6c4b2aa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:12:19.747 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:19.745+0000 7f6c4b2aa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:12:20.015 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:20.014+0000 7f6c4b2aa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:12:20.237 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:20.235+0000 7f6c4b2aa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:12:20.387 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:20.385+0000 7f6c4b2aa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:12:20.501 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:20.499+0000 7f6c4b2aa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:12:20.659 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:20.657+0000 7f6c4b2aa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:12:20.923 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:20.921+0000 7f6c4b2aa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:12:21.086 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.084+0000 7f6c4b2aa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:12:21.615 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.614+0000 7f6c4b2aa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:12:21.796 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.794+0000 7f6bbda84700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.794+0000 7f6bbda84700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.794+0000 7f6bbda84700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.794+0000 7f6bbda84700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.794+0000 7f6bbda84700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.798 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.795+0000 7f6bc0a8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.798 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.795+0000 7f6bc0a8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.798 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.795+0000 7f6bc0a8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.798 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.795+0000 7f6bc0a8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:21.799 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:12:21.795+0000 7f6bc0a8a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:12:25.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:12:25.316 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:12:55.460 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:12:55.464 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:00.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:13:00.795+0000 7f6bc8a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:13:00.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:13:00.795+0000 7f6bc8a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:13:00.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:13:00.795+0000 7f6bc8a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:13:00.797 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:13:00.795+0000 7f6bc8a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:13:00.798 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:13:00.795+0000 7f6bc8a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:13:25.607 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:25.610 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:55.744 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:13:55.747 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:25.867 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:25.871 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:55.997 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:14:56.000 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:26.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:26.154 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:56.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:15:56.301 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:26.403 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:26.406 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:56.549 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:16:56.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:16.462 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1m 2023-10-16T11:17:16.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.597+0000 7fde8ddcd700 1 Processor -- start 2023-10-16T11:17:16.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.597+0000 7fde8ddcd700 1 -- start start 2023-10-16T11:17:16.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde8ddcd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fde880b0350 0x7fde880d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde8ddcd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161470 0x7fde880d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde8ddcd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde880b31e0 con 0x7fde88161090 2023-10-16T11:17:16.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde8ddcd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde880b3860 con 0x7fde880b0350 2023-10-16T11:17:16.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde8ddcd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880d2590 con 0x7fde88161470 2023-10-16T11:17:16.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde86ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fde880b0350 0x7fde880d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde87fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161470 0x7fde880d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde86ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fde880b0350 0x7fde880d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:58068/0 (socket says 172.21.15.181:58068) 2023-10-16T11:17:16.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde87fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161470 0x7fde880d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:50046/0 (socket says 172.21.15.181:50046) 2023-10-16T11:17:16.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde86ffd700 1 -- 172.21.15.181:0/1704613939 learned_addr learned my addr 172.21.15.181:0/1704613939 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:16.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde87fff700 1 -- 172.21.15.181:0/1704613939 >> v2:172.21.15.190:3300/0 conn(0x7fde880b0350 msgr2=0x7fde880d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:16.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde87fff700 1 --2- 172.21.15.181:0/1704613939 >> v2:172.21.15.190:3300/0 conn(0x7fde880b0350 0x7fde880d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde87fff700 1 -- 172.21.15.181:0/1704613939 >> v1:172.21.15.190:6789/0 conn(0x7fde88161090 legacy=0x7fde880d40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde87fff700 1 -- 172.21.15.181:0/1704613939 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880b3480 con 0x7fde88161470 2023-10-16T11:17:16.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde867fc700 1 -- 172.21.15.181:0/1704613939 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502869733 0 0) 0x7fde880b31e0 con 0x7fde88161090 2023-10-16T11:17:16.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.598+0000 7fde86ffd700 1 --2- 172.21.15.181:0/1704613939 >> v2:172.21.15.190:3300/0 conn(0x7fde880b0350 0x7fde880d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:16.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde87fff700 1 --2- 172.21.15.181:0/1704613939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161470 0x7fde880d06b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d90 tx=0x7fde8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef170ba7e7a41df2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde867fc700 1 -- 172.21.15.181:0/1704613939 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000e070 con 0x7fde88161470 2023-10-16T11:17:16.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde867fc700 1 -- 172.21.15.181:0/1704613939 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8000bb20 con 0x7fde88161470 2023-10-16T11:17:16.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde867fc700 1 -- 172.21.15.181:0/1704613939 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80012430 con 0x7fde88161470 2023-10-16T11:17:16.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1704613939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161470 msgr2=0x7fde880d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1704613939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161470 0x7fde880d06b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d90 tx=0x7fde8000b330 comp rx=0 tx=0).stop 2023-10-16T11:17:16.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1704613939 shutdown_connections 2023-10-16T11:17:16.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1704613939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161470 0x7fde880d06b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1704613939 >> v2:172.21.15.190:3300/0 conn(0x7fde880b0350 0x7fde880d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1704613939 >> 172.21.15.181:0/1704613939 conn(0x7fde8809caf0 msgr2=0x7fde880d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:16.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1704613939 shutdown_connections 2023-10-16T11:17:16.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.599+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1704613939 wait complete. 2023-10-16T11:17:16.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.601+0000 7fde8ddcd700 1 Processor -- start 2023-10-16T11:17:16.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.601+0000 7fde8ddcd700 1 -- start start 2023-10-16T11:17:16.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde8ddcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde880b0350 0x7fde8815e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde8ddcd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161090 0x7fde880ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde8ddcd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde88161470 0x7fde880c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde8ddcd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880ca040 con 0x7fde880b0350 2023-10-16T11:17:16.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde8ddcd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880ca2a0 con 0x7fde88161470 2023-10-16T11:17:16.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde8ddcd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880b6490 con 0x7fde88161090 2023-10-16T11:17:16.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde880b0350 0x7fde8815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde880b0350 0x7fde8815e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53022/0 (socket says 172.21.15.181:53022) 2023-10-16T11:17:16.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde877fe700 1 -- 172.21.15.181:0/1293942024 learned_addr learned my addr 172.21.15.181:0/1293942024 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:16.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde86ffd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161090 0x7fde880ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.602+0000 7fde87fff700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde88161470 0x7fde880c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde877fe700 1 -- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161090 msgr2=0x7fde880ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde877fe700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161090 0x7fde880ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde877fe700 1 -- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde88161470 msgr2=0x7fde880c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde877fe700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde88161470 0x7fde880c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde877fe700 1 -- 172.21.15.181:0/1293942024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c009550 con 0x7fde880b0350 2023-10-16T11:17:16.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde877fe700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde880b0350 0x7fde8815e380 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fde7800c4e0 tx=0x7fde7800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde6ffff700 1 -- 172.21.15.181:0/1293942024 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde780054e0 con 0x7fde880b0350 2023-10-16T11:17:16.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde6ffff700 1 -- 172.21.15.181:0/1293942024 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde780036d0 con 0x7fde880b0350 2023-10-16T11:17:16.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde6ffff700 1 -- 172.21.15.181:0/1293942024 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78003a80 con 0x7fde880b0350 2023-10-16T11:17:16.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde80009a70 con 0x7fde880b0350 2023-10-16T11:17:16.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.603+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde880b6a50 con 0x7fde880b0350 2023-10-16T11:17:16.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.604+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde880b6c10 con 0x7fde880b0350 2023-10-16T11:17:16.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.604+0000 7fde6ffff700 1 -- 172.21.15.181:0/1293942024 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 122) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fde78003be0 con 0x7fde880b0350 2023-10-16T11:17:16.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.605+0000 7fde6ffff700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fde7406b070 0x7fde7406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:16.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.605+0000 7fde6ffff700 1 -- 172.21.15.181:0/1293942024 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6893+0+0 (secure 0 0 0) 0x7fde78089010 con 0x7fde880b0350 2023-10-16T11:17:16.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.605+0000 7fde86ffd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fde7406b070 0x7fde7406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:16.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.605+0000 7fde86ffd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fde7406b070 0x7fde7406d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde8806c320 tx=0x7fde7c0092c0 comp rx=0 tx=0).ready entity=mgr.10461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:16.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.610+0000 7fde6ffff700 1 -- 172.21.15.181:0/1293942024 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fde78053bd0 con 0x7fde880b0350 2023-10-16T11:17:16.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.833+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 --> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fde880d2030 con 0x7fde7406b070 2023-10-16T11:17:16.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.902+0000 7fde6ffff700 1 -- 172.21.15.181:0/1293942024 <== mgr.10461 v2:172.21.15.181:6832/2958900944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fde7c0099e0 con 0x7fde7406b070 2023-10-16T11:17:16.905 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-16T11:17:16.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fde7406b070 msgr2=0x7fde7406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fde7406b070 0x7fde7406d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde8806c320 tx=0x7fde7c0092c0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde880b0350 msgr2=0x7fde8815e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:16.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde880b0350 0x7fde8815e380 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fde7800c4e0 tx=0x7fde7800c7f0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 shutdown_connections 2023-10-16T11:17:16.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde88161470 0x7fde880c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde880b0350 0x7fde8815e380 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fde7406b070 0x7fde7406d530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 --2- 172.21.15.181:0/1293942024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde88161090 0x7fde880ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:16.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 >> 172.21.15.181:0/1293942024 conn(0x7fde8809caf0 msgr2=0x7fde8806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:16.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 shutdown_connections 2023-10-16T11:17:16.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:16.905+0000 7fde8ddcd700 1 -- 172.21.15.181:0/1293942024 wait complete. 2023-10-16T11:17:16.918 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-16T11:17:16.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:16.919 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-16T11:17:16.937 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_02_00_UTC 2023-10-16T11:17:16.937 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_03_00_UTC 2023-10-16T11:17:16.938 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_04_00_UTC 2023-10-16T11:17:16.938 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_05_00_UTC 2023-10-16T11:17:16.938 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_06_00_UTC 2023-10-16T11:17:16.938 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:17:16.939 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:17:16.939 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:17:16.939 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:17:16.939 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:17:16.939 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:17:16.940 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_13_00_UTC 2023-10-16T11:17:16.940 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_14_00_UTC 2023-10-16T11:17:16.940 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_15_00_UTC 2023-10-16T11:17:16.940 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_16_00_UTC 2023-10-16T11:17:16.940 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_17_00_UTC 2023-10-16T11:17:16.941 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-16-11_02_00_UTC', 'scheduled-2023-10-16-11_03_00_UTC', 'scheduled-2023-10-16-11_04_00_UTC', 'scheduled-2023-10-16-11_05_00_UTC', 'scheduled-2023-10-16-11_06_00_UTC', 'scheduled-2023-10-16-11_07_00_UTC', 'scheduled-2023-10-16-11_08_00_UTC', 'scheduled-2023-10-16-11_09_00_UTC', 'scheduled-2023-10-16-11_10_00_UTC', 'scheduled-2023-10-16-11_11_00_UTC', 'scheduled-2023-10-16-11_12_00_UTC', 'scheduled-2023-10-16-11_13_00_UTC', 'scheduled-2023-10-16-11_14_00_UTC', 'scheduled-2023-10-16-11_15_00_UTC', 'scheduled-2023-10-16-11_16_00_UTC', 'scheduled-2023-10-16-11_17_00_UTC'] 2023-10-16T11:17:16.941 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-16T11:17:17.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.097+0000 7f198f480700 1 Processor -- start 2023-10-16T11:17:17.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.097+0000 7f198f480700 1 -- start start 2023-10-16T11:17:17.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198f480700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f19880b2010 0x7f19880cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198f480700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19880ce680 0x7f19880ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198f480700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19880ce1f0 con 0x7f1988071be0 2023-10-16T11:17:17.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198f480700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1988072090 con 0x7f19880ce680 2023-10-16T11:17:17.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198f480700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880ccda0 con 0x7f19880b2010 2023-10-16T11:17:17.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19880ce680 0x7f19880ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19880ce680 0x7f19880ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53034/0 (socket says 172.21.15.181:53034) 2023-10-16T11:17:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 -- 172.21.15.181:0/371293955 learned_addr learned my addr 172.21.15.181:0/371293955 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198d21c700 1 --2- 172.21.15.181:0/371293955 >> v2:172.21.15.190:3300/0 conn(0x7f19880b2010 0x7f19880cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 -- 172.21.15.181:0/371293955 >> v2:172.21.15.190:3300/0 conn(0x7f19880b2010 msgr2=0x7f19880cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:17.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 --2- 172.21.15.181:0/371293955 >> v2:172.21.15.190:3300/0 conn(0x7f19880b2010 0x7f19880cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 -- 172.21.15.181:0/371293955 >> v1:172.21.15.190:6789/0 conn(0x7f1988071be0 legacy=0x7f19880b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:17.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 -- 172.21.15.181:0/371293955 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880727e0 con 0x7f19880ce680 2023-10-16T11:17:17.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.098+0000 7f198da1d700 1 --2- 172.21.15.181:0/371293955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ce680 0x7f19880ceaf0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f1988070430 tx=0x7f198000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33a280bf1502a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:17.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f197ffff700 1 -- 172.21.15.181:0/371293955 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000e070 con 0x7f19880ce680 2023-10-16T11:17:17.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f197ffff700 1 -- 172.21.15.181:0/371293955 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980003a20 con 0x7f19880ce680 2023-10-16T11:17:17.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f197ffff700 1 -- 172.21.15.181:0/371293955 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980003dd0 con 0x7f19880ce680 2023-10-16T11:17:17.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 -- 172.21.15.181:0/371293955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ce680 msgr2=0x7f19880ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 --2- 172.21.15.181:0/371293955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ce680 0x7f19880ceaf0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f1988070430 tx=0x7f198000b330 comp rx=0 tx=0).stop 2023-10-16T11:17:17.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 -- 172.21.15.181:0/371293955 shutdown_connections 2023-10-16T11:17:17.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 --2- 172.21.15.181:0/371293955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ce680 0x7f19880ceaf0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 --2- 172.21.15.181:0/371293955 >> v2:172.21.15.190:3300/0 conn(0x7f19880b2010 0x7f19880cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 -- 172.21.15.181:0/371293955 >> 172.21.15.181:0/371293955 conn(0x7f19880cd010 msgr2=0x7f19880cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:17.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 -- 172.21.15.181:0/371293955 shutdown_connections 2023-10-16T11:17:17.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.099+0000 7f198f480700 1 -- 172.21.15.181:0/371293955 wait complete. 2023-10-16T11:17:17.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.100+0000 7f198f480700 1 Processor -- start 2023-10-16T11:17:17.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.100+0000 7f198f480700 1 -- start start 2023-10-16T11:17:17.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198f480700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1988071be0 0x7f19880aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198f480700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880b2010 0x7f19881f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198f480700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19880ce680 0x7f19881507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198f480700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880af330 con 0x7f19880b2010 2023-10-16T11:17:17.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198f480700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880ad880 con 0x7f19880ce680 2023-10-16T11:17:17.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198f480700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19881f3590 con 0x7f1988071be0 2023-10-16T11:17:17.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198ca1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880b2010 0x7f19881f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198ca1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880b2010 0x7f19881f3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53038/0 (socket says 172.21.15.181:53038) 2023-10-16T11:17:17.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198ca1b700 1 -- 172.21.15.181:0/2116629742 learned_addr learned my addr 172.21.15.181:0/2116629742 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:17.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198da1d700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19880ce680 0x7f19881507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.101+0000 7f198d21c700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1988071be0 0x7f19880aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198ca1b700 1 -- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1988071be0 msgr2=0x7f19880aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198ca1b700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1988071be0 0x7f19880aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198ca1b700 1 -- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19880ce680 msgr2=0x7f19881507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198ca1b700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19880ce680 0x7f19881507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198ca1b700 1 -- 172.21.15.181:0/2116629742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009b30 con 0x7f19880b2010 2023-10-16T11:17:17.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198ca1b700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880b2010 0x7f19881f3050 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f197800b4a0 tx=0x7f197800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:17.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f197dffb700 1 -- 172.21.15.181:0/2116629742 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978011830 con 0x7f19880b2010 2023-10-16T11:17:17.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19880ad5a0 con 0x7f19880b2010 2023-10-16T11:17:17.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f197dffb700 1 -- 172.21.15.181:0/2116629742 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978011990 con 0x7f19880b2010 2023-10-16T11:17:17.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f197dffb700 1 -- 172.21.15.181:0/2116629742 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978011d00 con 0x7f19880b2010 2023-10-16T11:17:17.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.102+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19880c8650 con 0x7f19880b2010 2023-10-16T11:17:17.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.103+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19880b6510 con 0x7f19880b2010 2023-10-16T11:17:17.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.103+0000 7f197dffb700 1 -- 172.21.15.181:0/2116629742 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 122) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f197801a400 con 0x7f19880b2010 2023-10-16T11:17:17.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.104+0000 7f197dffb700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196c06b070 0x7f196c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.104+0000 7f197dffb700 1 -- 172.21.15.181:0/2116629742 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6893+0+0 (secure 0 0 0) 0x7f1978089ab0 con 0x7f19880b2010 2023-10-16T11:17:17.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.104+0000 7f198d21c700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196c06b070 0x7f196c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.104+0000 7f198d21c700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196c06b070 0x7f196c06d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f19881f6c20 tx=0x7f1984008040 comp rx=0 tx=0).ready entity=mgr.10461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:17.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.108+0000 7f197dffb700 1 -- 172.21.15.181:0/2116629742 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f19780545c0 con 0x7f19880b2010 2023-10-16T11:17:17.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.323+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 --> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1988072da0 con 0x7f196c06b070 2023-10-16T11:17:17.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.330+0000 7f197dffb700 1 -- 172.21.15.181:0/2116629742 <== mgr.10461 v2:172.21.15.181:6832/2958900944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+376 (secure 0 0 0) 0x7f198400c040 con 0x7f196c06b070 2023-10-16T11:17:17.332 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:17.332 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:01:38", "first": "2023-10-16T11:02:00", "last": "2023-10-16T11:17:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-16T11:17:17.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.332+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196c06b070 msgr2=0x7f196c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.332+0000 7f198f480700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196c06b070 0x7f196c06d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f19881f6c20 tx=0x7f1984008040 comp rx=0 tx=0).stop 2023-10-16T11:17:17.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.332+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880b2010 msgr2=0x7f19881f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.332+0000 7f198f480700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880b2010 0x7f19881f3050 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f197800b4a0 tx=0x7f197800b7b0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.332+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 shutdown_connections 2023-10-16T11:17:17.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.333+0000 7f198f480700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196c06b070 0x7f196c06d530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.333+0000 7f198f480700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1988071be0 0x7f19880aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.333+0000 7f198f480700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19880ce680 0x7f19881507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.333+0000 7f198f480700 1 --2- 172.21.15.181:0/2116629742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880b2010 0x7f19881f3050 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.333+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 >> 172.21.15.181:0/2116629742 conn(0x7f19880cd010 msgr2=0x7f1988159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:17.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.333+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 shutdown_connections 2023-10-16T11:17:17.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.333+0000 7f198f480700 1 -- 172.21.15.181:0/2116629742 wait complete. 2023-10-16T11:17:17.346 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "group": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:01:38", "first": "2023-10-16T11:02:00", "last": "2023-10-16T11:17:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-16T11:17:17.347 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'group': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-16T00:00:00', 'created': '2023-10-16T11:01:38', 'first': '2023-10-16T11:02:00', 'last': '2023-10-16T11:17:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-16T11:17:17.347 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-16T11:17:17.347 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-16T11:17:17.348 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1m 2023-10-16T11:17:17.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.470+0000 7fb0748cd700 1 Processor -- start 2023-10-16T11:17:17.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.471+0000 7fb0748cd700 1 -- start start 2023-10-16T11:17:17.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.471+0000 7fb0748cd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb07015f150 0x7fb0700cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.471+0000 7fb0748cd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 0x7fb0700b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.471+0000 7fb0748cd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0700b2ae0 con 0x7fb0700ce4f0 2023-10-16T11:17:17.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.471+0000 7fb0748cd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0700ce9a0 con 0x7fb07015f150 2023-10-16T11:17:17.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.471+0000 7fb0748cd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700b2dc0 con 0x7fb0700cd170 2023-10-16T11:17:17.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb0700ce4f0 0x7fb0700b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35890/0 (socket says 172.21.15.181:35890) 2023-10-16T11:17:17.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06dd9b700 1 -- 172.21.15.181:0/3848163415 learned_addr learned my addr 172.21.15.181:0/3848163415 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:17.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06ed9d700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 0x7fb0700b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06e59c700 1 --2- 172.21.15.181:0/3848163415 >> v2:172.21.15.190:3300/0 conn(0x7fb07015f150 0x7fb0700cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847807497 0 0) 0x7fb0700b2ae0 con 0x7fb0700ce4f0 2023-10-16T11:17:17.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0580035b0 con 0x7fb0700ce4f0 2023-10-16T11:17:17.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833503786 0 0) 0x7fb0580035b0 con 0x7fb0700ce4f0 2023-10-16T11:17:17.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 msgr2=0x7fb0700b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:17.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 0x7fb0700b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 >> v2:172.21.15.190:3300/0 conn(0x7fb07015f150 msgr2=0x7fb0700cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:17.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 --2- 172.21.15.181:0/3848163415 >> v2:172.21.15.190:3300/0 conn(0x7fb07015f150 0x7fb0700cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700cf9d0 con 0x7fb0700ce4f0 2023-10-16T11:17:17.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fb060001670 con 0x7fb0700ce4f0 2023-10-16T11:17:17.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 >> v1:172.21.15.181:6789/0 conn(0x7fb0700ce4f0 legacy=0x7fb0700b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb058003a00 0x7fb058003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 0x7fb058004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb058004d90 0x7fb058005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700ce9a0 con 0x7fb058003a00 2023-10-16T11:17:17.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700b2dc0 con 0x7fb058004d90 2023-10-16T11:17:17.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05800ba10 con 0x7fb0700cd170 2023-10-16T11:17:17.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06e59c700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb058003a00 0x7fb058003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06dd9b700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 0x7fb058004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.472+0000 7fb06ed9d700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb058004d90 0x7fb058005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06e59c700 1 -- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 msgr2=0x7fb058004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:17.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06e59c700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cd170 0x7fb058004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06e59c700 1 -- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb058004d90 msgr2=0x7fb058005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:17.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06e59c700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb058004d90 0x7fb058005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06e59c700 1 -- 172.21.15.181:0/3848163415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0580035b0 con 0x7fb058003a00 2023-10-16T11:17:17.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06e59c700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb058003a00 0x7fb058003e20 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fb064009b60 tx=0x7fb058005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986ed24645a543c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:17.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06400e6c0 con 0x7fb058003a00 2023-10-16T11:17:17.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06400e820 con 0x7fb058003a00 2023-10-16T11:17:17.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06d59a700 1 -- 172.21.15.181:0/3848163415 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06400eb90 con 0x7fb058003a00 2023-10-16T11:17:17.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb0748cd700 1 -- 172.21.15.181:0/3848163415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb058003a00 msgr2=0x7fb058003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb0748cd700 1 --2- 172.21.15.181:0/3848163415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb058003a00 0x7fb058003e20 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fb064009b60 tx=0x7fb058005880 comp rx=0 tx=0).stop 2023-10-16T11:17:17.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb06ed9d700 1 -- 172.21.15.181:0/3848163415 reap_dead start 2023-10-16T11:17:17.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb0748cd700 1 -- 172.21.15.181:0/3848163415 shutdown_connections 2023-10-16T11:17:17.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb0748cd700 1 -- 172.21.15.181:0/3848163415 >> 172.21.15.181:0/3848163415 conn(0x7fb07009a930 msgr2=0x7fb0700d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:17.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb0748cd700 1 -- 172.21.15.181:0/3848163415 shutdown_connections 2023-10-16T11:17:17.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.473+0000 7fb0748cd700 1 -- 172.21.15.181:0/3848163415 wait complete. 2023-10-16T11:17:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.475+0000 7fb0748cd700 1 Processor -- start 2023-10-16T11:17:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.475+0000 7fb0748cd700 1 -- start start 2023-10-16T11:17:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb058003a00 0x7fb0700c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb070158930 0x7fb0700c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700c7d60 0x7fb0700ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700c8210 con 0x7fb0700c7d60 2023-10-16T11:17:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700c7440 con 0x7fb070158930 2023-10-16T11:17:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700cb210 con 0x7fb058003a00 2023-10-16T11:17:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700c7d60 0x7fb0700ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700c7d60 0x7fb0700ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53060/0 (socket says 172.21.15.181:53060) 2023-10-16T11:17:17.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 -- 172.21.15.181:0/1634185898 learned_addr learned my addr 172.21.15.181:0/1634185898 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:17.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06e59c700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb058003a00 0x7fb0700c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06dd9b700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb070158930 0x7fb0700c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 -- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb058003a00 msgr2=0x7fb0700c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb058003a00 0x7fb0700c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 -- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb070158930 msgr2=0x7fb0700c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb070158930 0x7fb0700c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 -- 172.21.15.181:0/1634185898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700c6420 con 0x7fb0700c7d60 2023-10-16T11:17:17.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06dd9b700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb070158930 0x7fb0700c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:17.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb06ed9d700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700c7d60 0x7fb0700ca520 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fb07006b2c0 tx=0x7fb068011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:17.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb05effd700 1 -- 172.21.15.181:0/1634185898 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800e040 con 0x7fb0700c7d60 2023-10-16T11:17:17.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb05effd700 1 -- 172.21.15.181:0/1634185898 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06800f040 con 0x7fb0700c7d60 2023-10-16T11:17:17.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0700c6700 con 0x7fb0700c7d60 2023-10-16T11:17:17.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb05effd700 1 -- 172.21.15.181:0/1634185898 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800fdb0 con 0x7fb0700c7d60 2023-10-16T11:17:17.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.476+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0700c6040 con 0x7fb0700c7d60 2023-10-16T11:17:17.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.477+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0700c6c20 con 0x7fb0700c7d60 2023-10-16T11:17:17.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.478+0000 7fb05effd700 1 -- 172.21.15.181:0/1634185898 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 122) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fb068022460 con 0x7fb0700c7d60 2023-10-16T11:17:17.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.478+0000 7fb05effd700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fb05006b070 0x7fb05006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:17.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.478+0000 7fb05effd700 1 -- 172.21.15.181:0/1634185898 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6893+0+0 (secure 0 0 0) 0x7fb06808a300 con 0x7fb0700c7d60 2023-10-16T11:17:17.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.478+0000 7fb06e59c700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fb05006b070 0x7fb05006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:17.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.478+0000 7fb06e59c700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fb05006b070 0x7fb05006d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0640090b0 tx=0x7fb064008030 comp rx=0 tx=0).ready entity=mgr.10461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:17.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.483+0000 7fb05effd700 1 -- 172.21.15.181:0/1634185898 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb0700c6c20 con 0x7fb0700c7d60 2023-10-16T11:17:17.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.702+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 --> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0700cff90 con 0x7fb05006b070 2023-10-16T11:17:17.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.780+0000 7fb05effd700 1 -- 172.21.15.181:0/1634185898 <== mgr.10461 v2:172.21.15.181:6832/2958900944 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fb064006c60 con 0x7fb05006b070 2023-10-16T11:17:17.783 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-16T11:17:17.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fb05006b070 msgr2=0x7fb05006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fb05006b070 0x7fb05006d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0640090b0 tx=0x7fb064008030 comp rx=0 tx=0).stop 2023-10-16T11:17:17.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700c7d60 msgr2=0x7fb0700ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:17.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700c7d60 0x7fb0700ca520 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fb07006b2c0 tx=0x7fb068011f80 comp rx=0 tx=0).stop 2023-10-16T11:17:17.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 shutdown_connections 2023-10-16T11:17:17.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fb05006b070 0x7fb05006d530 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb058003a00 0x7fb0700c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700c7d60 0x7fb0700ca520 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 --2- 172.21.15.181:0/1634185898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb070158930 0x7fb0700c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:17.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 >> 172.21.15.181:0/1634185898 conn(0x7fb07009a930 msgr2=0x7fb07015a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:17.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 shutdown_connections 2023-10-16T11:17:17.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:17.783+0000 7fb0748cd700 1 -- 172.21.15.181:0/1634185898 wait complete. 2023-10-16T11:17:17.796 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-16T11:17:17.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:17.797 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-16T11:17:17.815 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_02_00_UTC 2023-10-16T11:17:17.815 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_03_00_UTC 2023-10-16T11:17:17.815 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_04_00_UTC 2023-10-16T11:17:17.816 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_05_00_UTC 2023-10-16T11:17:17.816 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_06_00_UTC 2023-10-16T11:17:17.816 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:17:17.816 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:17:17.816 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:17:17.817 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:17:17.817 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:17:17.817 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:17:17.817 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_13_00_UTC 2023-10-16T11:17:17.817 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_14_00_UTC 2023-10-16T11:17:17.817 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_15_00_UTC 2023-10-16T11:17:17.818 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_16_00_UTC 2023-10-16T11:17:17.818 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_17_00_UTC 2023-10-16T11:17:17.818 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_02_00_UTC 2023-10-16T11:17:17.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:17.819 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_02_00_UTC) 2023-10-16T11:17:17.878 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_03_00_UTC 2023-10-16T11:17:17.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:17.879 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_03_00_UTC) 2023-10-16T11:17:17.940 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_04_00_UTC 2023-10-16T11:17:17.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:17.941 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_04_00_UTC) 2023-10-16T11:17:18.002 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_05_00_UTC 2023-10-16T11:17:18.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.003 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_05_00_UTC) 2023-10-16T11:17:18.063 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_06_00_UTC 2023-10-16T11:17:18.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.064 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_06_00_UTC) 2023-10-16T11:17:18.125 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_07_00_UTC 2023-10-16T11:17:18.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.126 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_07_00_UTC) 2023-10-16T11:17:18.187 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_08_00_UTC 2023-10-16T11:17:18.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.188 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_08_00_UTC) 2023-10-16T11:17:18.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_09_00_UTC 2023-10-16T11:17:18.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.250 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_09_00_UTC) 2023-10-16T11:17:18.311 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_10_00_UTC 2023-10-16T11:17:18.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.312 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_10_00_UTC) 2023-10-16T11:17:18.374 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_11_00_UTC 2023-10-16T11:17:18.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.375 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_11_00_UTC) 2023-10-16T11:17:18.437 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_12_00_UTC 2023-10-16T11:17:18.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.438 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_12_00_UTC) 2023-10-16T11:17:18.499 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_13_00_UTC 2023-10-16T11:17:18.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.500 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_13_00_UTC) 2023-10-16T11:17:18.561 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_14_00_UTC 2023-10-16T11:17:18.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.562 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_14_00_UTC) 2023-10-16T11:17:18.622 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_15_00_UTC 2023-10-16T11:17:18.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.623 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_15_00_UTC) 2023-10-16T11:17:18.684 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_16_00_UTC 2023-10-16T11:17:18.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.685 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_16_00_UTC) 2023-10-16T11:17:18.747 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_17_00_UTC 2023-10-16T11:17:18.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.748 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-16-11_17_00_UTC) 2023-10-16T11:17:18.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:18.810 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-16T11:17:18.869 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:17:19.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.028+0000 7f62ca861700 1 Processor -- start 2023-10-16T11:17:19.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.028+0000 7f62ca861700 1 -- start start 2023-10-16T11:17:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62ca861700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62c41593b0 0x7f62c4070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62ca861700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62c415e3d0 0x7f62c4071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62ca861700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c4159790 con 0x7f62c4071760 2023-10-16T11:17:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62ca861700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c415cf70 con 0x7f62c415e3d0 2023-10-16T11:17:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62ca861700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c415c320 con 0x7f62c41593b0 2023-10-16T11:17:19.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62c415e3d0 0x7f62c4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62c415e3d0 0x7f62c4071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53068/0 (socket says 172.21.15.181:53068) 2023-10-16T11:17:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 -- 172.21.15.181:0/3641913701 learned_addr learned my addr 172.21.15.181:0/3641913701 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c3fff700 1 --2- 172.21.15.181:0/3641913701 >> v2:172.21.15.190:3300/0 conn(0x7f62c41593b0 0x7f62c4070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 -- 172.21.15.181:0/3641913701 >> v2:172.21.15.190:3300/0 conn(0x7f62c41593b0 msgr2=0x7f62c4070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:19.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 --2- 172.21.15.181:0/3641913701 >> v2:172.21.15.190:3300/0 conn(0x7f62c41593b0 0x7f62c4070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 -- 172.21.15.181:0/3641913701 >> v1:172.21.15.190:6789/0 conn(0x7f62c4071760 legacy=0x7f62c40b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 -- 172.21.15.181:0/3641913701 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c415c600 con 0x7f62c415e3d0 2023-10-16T11:17:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c8dfe700 1 --2- 172.21.15.181:0/3641913701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c4071050 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f62c40cf410 tx=0x7f62bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf3df027fa08263 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c2ffd700 1 -- 172.21.15.181:0/3641913701 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00e070 con 0x7f62c415e3d0 2023-10-16T11:17:19.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c2ffd700 1 -- 172.21.15.181:0/3641913701 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc00b9c0 con 0x7f62c415e3d0 2023-10-16T11:17:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.029+0000 7f62c2ffd700 1 -- 172.21.15.181:0/3641913701 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00bd70 con 0x7f62c415e3d0 2023-10-16T11:17:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 -- 172.21.15.181:0/3641913701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 msgr2=0x7f62c4071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 --2- 172.21.15.181:0/3641913701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c4071050 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f62c40cf410 tx=0x7f62bc00b330 comp rx=0 tx=0).stop 2023-10-16T11:17:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 -- 172.21.15.181:0/3641913701 shutdown_connections 2023-10-16T11:17:19.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 --2- 172.21.15.181:0/3641913701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c4071050 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 --2- 172.21.15.181:0/3641913701 >> v2:172.21.15.190:3300/0 conn(0x7f62c41593b0 0x7f62c4070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 -- 172.21.15.181:0/3641913701 >> 172.21.15.181:0/3641913701 conn(0x7f62c406aed0 msgr2=0x7f62c406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:19.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 -- 172.21.15.181:0/3641913701 shutdown_connections 2023-10-16T11:17:19.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.030+0000 7f62ca861700 1 -- 172.21.15.181:0/3641913701 wait complete. 2023-10-16T11:17:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.031+0000 7f62ca861700 1 Processor -- start 2023-10-16T11:17:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62ca861700 1 -- start start 2023-10-16T11:17:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62ca861700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62c4071760 0x7f62c4150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62ca861700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c41593b0 0x7f62c40c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62ca861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c40c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62ca861700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c41513c0 con 0x7f62c415e3d0 2023-10-16T11:17:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62ca861700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40b03c0 con 0x7f62c41593b0 2023-10-16T11:17:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62ca861700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40afad0 con 0x7f62c4071760 2023-10-16T11:17:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62c8dfe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c40c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62c8dfe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c40c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53074/0 (socket says 172.21.15.181:53074) 2023-10-16T11:17:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62c8dfe700 1 -- 172.21.15.181:0/394696638 learned_addr learned my addr 172.21.15.181:0/394696638 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62c3fff700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62c4071760 0x7f62c4150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.032+0000 7f62c37fe700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c41593b0 0x7f62c40c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c8dfe700 1 -- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62c4071760 msgr2=0x7f62c4150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c8dfe700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62c4071760 0x7f62c4150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c8dfe700 1 -- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c41593b0 msgr2=0x7f62c40c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c8dfe700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c41593b0 0x7f62c40c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c8dfe700 1 -- 172.21.15.181:0/394696638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc009b30 con 0x7f62c415e3d0 2023-10-16T11:17:19.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c8dfe700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c40c4f60 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f62c406ee80 tx=0x7f62bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c0ff9700 1 -- 172.21.15.181:0/394696638 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc004880 con 0x7f62c415e3d0 2023-10-16T11:17:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c0ff9700 1 -- 172.21.15.181:0/394696638 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc0099b0 con 0x7f62c415e3d0 2023-10-16T11:17:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c41e65a0 con 0x7f62c415e3d0 2023-10-16T11:17:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62c0ff9700 1 -- 172.21.15.181:0/394696638 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc01b670 con 0x7f62c415e3d0 2023-10-16T11:17:19.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.033+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c40b0050 con 0x7f62c415e3d0 2023-10-16T11:17:19.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.034+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c40b0550 con 0x7f62c415e3d0 2023-10-16T11:17:19.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.034+0000 7f62c0ff9700 1 -- 172.21.15.181:0/394696638 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 122) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f62bc01b7d0 con 0x7f62c415e3d0 2023-10-16T11:17:19.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.035+0000 7f62c0ff9700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f62a406b070 0x7f62a406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.035+0000 7f62c0ff9700 1 -- 172.21.15.181:0/394696638 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6893+0+0 (secure 0 0 0) 0x7f62bc08b4d0 con 0x7f62c415e3d0 2023-10-16T11:17:19.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.035+0000 7f62c3fff700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f62a406b070 0x7f62a406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.035+0000 7f62c3fff700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f62a406b070 0x7f62a406d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f62ac006b20 tx=0x7f62ac008040 comp rx=0 tx=0).ready entity=mgr.10461 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:19.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.039+0000 7f62c0ff9700 1 -- 172.21.15.181:0/394696638 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f62bc055f50 con 0x7f62c415e3d0 2023-10-16T11:17:19.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.290+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f62c404ebb0 con 0x7f62c415e3d0 2023-10-16T11:17:19.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.424+0000 7f62c0ff9700 1 -- 172.21.15.181:0/394696638 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v123) v1 ==== 93+0+0 (secure 0 0 0) 0x7f62bc059570 con 0x7f62c415e3d0 2023-10-16T11:17:19.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f62a406b070 msgr2=0x7f62a406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f62a406b070 0x7f62a406d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f62ac006b20 tx=0x7f62ac008040 comp rx=0 tx=0).stop 2023-10-16T11:17:19.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 msgr2=0x7f62c40c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c40c4f60 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f62c406ee80 tx=0x7f62bc004ad0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 shutdown_connections 2023-10-16T11:17:19.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f62a406b070 0x7f62a406d530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62c4071760 0x7f62c4150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62c415e3d0 0x7f62c40c4f60 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 --2- 172.21.15.181:0/394696638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62c41593b0 0x7f62c40c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.430+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 >> 172.21.15.181:0/394696638 conn(0x7f62c406aed0 msgr2=0x7f62c41574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:19.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.431+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 shutdown_connections 2023-10-16T11:17:19.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.431+0000 7f62ca861700 1 -- 172.21.15.181:0/394696638 wait complete. 2023-10-16T11:17:19.453 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:17:19.454 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:17:19.505 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:17:19.559 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:17:19.560 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:17:19.560 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:19.578 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:17:19.578 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:19.604 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:19.601+0000 7f28b07ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:17:19.609 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:19.608+0000 7f53c5306200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:17:19.621 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:19.682 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:19.681+0000 7f53c5306200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:19.683 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:19.681+0000 7f28b07ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:19.760 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:19.759+0000 7f53c5306200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:17:19.784 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:19.782+0000 7f28b07ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:17:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.824+0000 7fc1067aa700 1 Processor -- start 2023-10-16T11:17:19.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.824+0000 7fc1067aa700 1 -- start start 2023-10-16T11:17:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc1067aa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1000cb0c0 0x7fc1000d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc1067aa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10006ed00 con 0x7fc10015e6d0 2023-10-16T11:17:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc1067aa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1000d02d0 con 0x7fc10015eab0 2023-10-16T11:17:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc1067aa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10006e0f0 con 0x7fc1000cb0c0 2023-10-16T11:17:19.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc104d47700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1000cb0c0 0x7fc1000d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc0fffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc10015e6d0 0x7fc10015a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:35968/0 (socket says 172.21.15.181:35968) 2023-10-16T11:17:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc0fffff700 1 -- 172.21.15.181:0/3996007519 learned_addr learned my addr 172.21.15.181:0/3996007519 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc104d47700 1 -- 172.21.15.181:0/3996007519 >> v1:172.21.15.190:6789/0 conn(0x7fc10015eab0 legacy=0x7fc10015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc104d47700 1 -- 172.21.15.181:0/3996007519 >> v1:172.21.15.181:6789/0 conn(0x7fc10015e6d0 legacy=0x7fc10015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc0feffd700 1 -- 172.21.15.181:0/3996007519 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227507385 0 0) 0x7fc10006ed00 con 0x7fc10015e6d0 2023-10-16T11:17:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc104d47700 1 -- 172.21.15.181:0/3996007519 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10006e3d0 con 0x7fc1000cb0c0 2023-10-16T11:17:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.825+0000 7fc104d47700 1 --2- 172.21.15.181:0/3996007519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1000cb0c0 0x7fc1000d0fc0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009d90 tx=0x7fc0f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48df824d704aa440 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc0feffd700 1 -- 172.21.15.181:0/3996007519 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800e070 con 0x7fc1000cb0c0 2023-10-16T11:17:19.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc0feffd700 1 -- 172.21.15.181:0/3996007519 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f800bbd0 con 0x7fc1000cb0c0 2023-10-16T11:17:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc0feffd700 1 -- 172.21.15.181:0/3996007519 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8012430 con 0x7fc1000cb0c0 2023-10-16T11:17:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc1067aa700 1 -- 172.21.15.181:0/3996007519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1000cb0c0 msgr2=0x7fc1000d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc1067aa700 1 --2- 172.21.15.181:0/3996007519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1000cb0c0 0x7fc1000d0fc0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009d90 tx=0x7fc0f800b330 comp rx=0 tx=0).stop 2023-10-16T11:17:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc1067aa700 1 -- 172.21.15.181:0/3996007519 shutdown_connections 2023-10-16T11:17:19.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc1067aa700 1 --2- 172.21.15.181:0/3996007519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1000cb0c0 0x7fc1000d0fc0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc1067aa700 1 -- 172.21.15.181:0/3996007519 >> 172.21.15.181:0/3996007519 conn(0x7fc1000d04a0 msgr2=0x7fc1000d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:19.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc1067aa700 1 -- 172.21.15.181:0/3996007519 shutdown_connections 2023-10-16T11:17:19.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.826+0000 7fc1067aa700 1 -- 172.21.15.181:0/3996007519 wait complete. 2023-10-16T11:17:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.827+0000 7fc1067aa700 1 Processor -- start 2023-10-16T11:17:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc1067aa700 1 -- start start 2023-10-16T11:17:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc1067aa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1000cb0c0 0x7fc1000b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc1067aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc10015e6d0 0x7fc10014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc1067aa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10015eab0 0x7fc10015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc1067aa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10015b4a0 con 0x7fc10015e6d0 2023-10-16T11:17:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc1067aa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10015b700 con 0x7fc10015eab0 2023-10-16T11:17:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc1067aa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10015b9e0 con 0x7fc1000cb0c0 2023-10-16T11:17:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc10015e6d0 0x7fc10014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc104d47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10015eab0 0x7fc10015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1000cb0c0 0x7fc1000b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.828+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc10015e6d0 0x7fc10014fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53128/0 (socket says 172.21.15.181:53128) 2023-10-16T11:17:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0ff7fe700 1 -- 172.21.15.181:0/1095972229 learned_addr learned my addr 172.21.15.181:0/1095972229 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0ff7fe700 1 -- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1000cb0c0 msgr2=0x7fc1000b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0ff7fe700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1000cb0c0 0x7fc1000b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0ff7fe700 1 -- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10015eab0 msgr2=0x7fc10015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0ff7fe700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10015eab0 0x7fc10015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0ff7fe700 1 -- 172.21.15.181:0/1095972229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009a70 con 0x7fc10015e6d0 2023-10-16T11:17:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0fffff700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1000cb0c0 0x7fc1000b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc104d47700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10015eab0 0x7fc10015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0ff7fe700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc10015e6d0 0x7fc10014fac0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c4e0 tx=0x7fc0e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0fcff9700 1 -- 172.21.15.181:0/1095972229 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80052d0 con 0x7fc10015e6d0 2023-10-16T11:17:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10015bcc0 con 0x7fc10015e6d0 2023-10-16T11:17:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0fcff9700 1 -- 172.21.15.181:0/1095972229 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8005430 con 0x7fc10015e6d0 2023-10-16T11:17:19.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc0fcff9700 1 -- 172.21.15.181:0/1095972229 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80038c0 con 0x7fc10015e6d0 2023-10-16T11:17:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.829+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1000c88a0 con 0x7fc10015e6d0 2023-10-16T11:17:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.830+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1000c8ac0 con 0x7fc10015e6d0 2023-10-16T11:17:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.831+0000 7fc0fcff9700 1 -- 172.21.15.181:0/1095972229 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fc0e8003a20 con 0x7fc10015e6d0 2023-10-16T11:17:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.831+0000 7fc0fcff9700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 0x7fc0e006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:19.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.831+0000 7fc0fffff700 1 -- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 msgr2=0x7fc0e006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:19.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.831+0000 7fc0fcff9700 1 -- 172.21.15.181:0/1095972229 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fc0e80893e0 con 0x7fc10015e6d0 2023-10-16T11:17:19.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.831+0000 7fc0fffff700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 0x7fc0e006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:19.842 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:19.836+0000 7f53c5306200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:17:19.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:19.835+0000 7fc0fcff9700 1 -- 172.21.15.181:0/1095972229 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc0e8053f10 con 0x7fc10015e6d0 2023-10-16T11:17:19.869 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:19.867+0000 7f28b07ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:17:19.978 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:19.976+0000 7f53c5306200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:17:20.017 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:20.015+0000 7f28b07ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:17:20.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.032+0000 7fc0fffff700 1 -- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 msgr2=0x7fc0e006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:20.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.032+0000 7fc0fffff700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 0x7fc0e006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:20.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.037+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc10006e680 con 0x7fc10015e6d0 2023-10-16T11:17:20.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.039+0000 7fc0fcff9700 1 -- 172.21.15.181:0/1095972229 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+18546 (secure 0 0 0) 0x7fc0e8057530 con 0x7fc10015e6d0 2023-10-16T11:17:20.041 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:20.041 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":184,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:19.416835+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:01:21.420382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:01:21.958383+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":184,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/4003028162":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/3290548668":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6833/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6832/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/2238590139":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/231495737":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/1499024696":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/458160361":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1195985489":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/4000110033":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/990187329":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3304094772":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6833/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1162194396":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6832/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3436855298":"2023-10-17T11:01:37.009118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":13},{"begin":18,"length":13}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":13},{"begin":18,"length":13}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 msgr2=0x7fc0e006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 0x7fc0e006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc10015e6d0 msgr2=0x7fc10014fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc10015e6d0 0x7fc10014fac0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800c4e0 tx=0x7fc0e800c7f0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 shutdown_connections 2023-10-16T11:17:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fc0e006b020 0x7fc0e006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1000cb0c0 0x7fc1000b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc10015eab0 0x7fc10015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 --2- 172.21.15.181:0/1095972229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc10015e6d0 0x7fc10014fac0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 >> 172.21.15.181:0/1095972229 conn(0x7fc1000d04a0 msgr2=0x7fc1000c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 shutdown_connections 2023-10-16T11:17:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.041+0000 7fc1067aa700 1 -- 172.21.15.181:0/1095972229 wait complete. 2023-10-16T11:17:20.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:01:21.420382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '180', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:01:21.958383+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 184, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:17:20.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:17:20.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.219+0000 7f0c22afc700 1 Processor -- start 2023-10-16T11:17:20.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.219+0000 7f0c22afc700 1 -- start start 2023-10-16T11:17:20.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c22afc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c1c15f150 0x7f0c1c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c22afc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c22afc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0b2ae0 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c22afc700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c0ce970 con 0x7f0c1c15f150 2023-10-16T11:17:20.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c22afc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0b2dc0 con 0x7f0c1c0ce4f0 2023-10-16T11:17:20.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c21099700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0c1c0b14a0 0x7f0c1c0b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36012/0 (socket says 172.21.15.181:36012) 2023-10-16T11:17:20.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c21099700 1 -- 172.21.15.181:0/2848621874 learned_addr learned my addr 172.21.15.181:0/2848621874 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:20.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c20898700 1 --2- 172.21.15.181:0/2848621874 >> v2:172.21.15.190:3300/0 conn(0x7f0c1c15f150 0x7f0c1c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c1bfff700 1 --2- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c1b7fe700 1 -- 172.21.15.181:0/2848621874 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344487560 0 0) 0x7f0c1c0b2ae0 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.220+0000 7f0c1b7fe700 1 -- 172.21.15.181:0/2848621874 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c0c0035b0 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c20898700 1 -- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0ce4f0 msgr2=0x7f0c1c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:20.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c20898700 1 --2- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c20898700 1 -- 172.21.15.181:0/2848621874 >> v1:172.21.15.181:6789/0 conn(0x7f0c1c0b14a0 legacy=0x7f0c1c0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c20898700 1 -- 172.21.15.181:0/2848621874 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0cf9d0 con 0x7f0c1c15f150 2023-10-16T11:17:20.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c1bfff700 1 --2- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:20.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c20898700 1 --2- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c15f150 0x7f0c1c0cdfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009840 tx=0x7f0c10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82882b77699cc6d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:20.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c1b7fe700 1 -- 172.21.15.181:0/2848621874 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011830 con 0x7f0c1c15f150 2023-10-16T11:17:20.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c1b7fe700 1 -- 172.21.15.181:0/2848621874 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10011990 con 0x7f0c1c15f150 2023-10-16T11:17:20.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c1b7fe700 1 -- 172.21.15.181:0/2848621874 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011d20 con 0x7f0c1c15f150 2023-10-16T11:17:20.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c22afc700 1 -- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c15f150 msgr2=0x7f0c1c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.221+0000 7f0c22afc700 1 --2- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c15f150 0x7f0c1c0cdfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009840 tx=0x7f0c10009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:20.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.222+0000 7f0c22afc700 1 -- 172.21.15.181:0/2848621874 shutdown_connections 2023-10-16T11:17:20.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.222+0000 7f0c22afc700 1 --2- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c15f150 0x7f0c1c0cdfb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.222+0000 7f0c22afc700 1 --2- 172.21.15.181:0/2848621874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.222+0000 7f0c22afc700 1 -- 172.21.15.181:0/2848621874 >> 172.21.15.181:0/2848621874 conn(0x7f0c1c09a930 msgr2=0x7f0c1c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:20.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.222+0000 7f0c22afc700 1 -- 172.21.15.181:0/2848621874 shutdown_connections 2023-10-16T11:17:20.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.222+0000 7f0c22afc700 1 -- 172.21.15.181:0/2848621874 wait complete. 2023-10-16T11:17:20.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.223+0000 7f0c22afc700 1 Processor -- start 2023-10-16T11:17:20.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.223+0000 7f0c22afc700 1 -- start start 2023-10-16T11:17:20.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c22afc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0b14a0 0x7f0c1c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c22afc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c22afc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1c15f150 0x7f0c1c06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c22afc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c9b30 con 0x7f0c1c15f150 2023-10-16T11:17:20.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c22afc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c8f20 con 0x7f0c1c0ce4f0 2023-10-16T11:17:20.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c22afc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0c9200 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c21099700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1c15f150 0x7f0c1c06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c21099700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1c15f150 0x7f0c1c06fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53172/0 (socket says 172.21.15.181:53172) 2023-10-16T11:17:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c21099700 1 -- 172.21.15.181:0/820247309 learned_addr learned my addr 172.21.15.181:0/820247309 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c20898700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0b14a0 0x7f0c1c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c1bfff700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c20898700 1 -- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c0ce4f0 msgr2=0x7f0c1c0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c20898700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c20898700 1 -- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1c15f150 msgr2=0x7f0c1c06fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c20898700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1c15f150 0x7f0c1c06fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c20898700 1 -- 172.21.15.181:0/820247309 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080094a0 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.224+0000 7f0c20898700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0b14a0 0x7f0c1c06c800 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c10016000 tx=0x7f0c10004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:20.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.225+0000 7f0c197fa700 1 -- 172.21.15.181:0/820247309 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10004590 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.225+0000 7f0c197fa700 1 -- 172.21.15.181:0/820247309 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1001e070 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.225+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c100094a0 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.225+0000 7f0c197fa700 1 -- 172.21.15.181:0/820247309 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100038e0 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.225+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c0c97c0 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.226+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c067d20 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.226+0000 7f0c197fa700 1 -- 172.21.15.181:0/820247309 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f0c10003a40 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.226+0000 7f0c197fa700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 0x7f0c0006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.226+0000 7f0c1bfff700 1 -- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 msgr2=0x7f0c0006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:20.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.227+0000 7f0c197fa700 1 -- 172.21.15.181:0/820247309 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f0c1008a920 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.227+0000 7f0c1bfff700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 0x7f0c0006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:20.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.231+0000 7f0c197fa700 1 -- 172.21.15.181:0/820247309 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0c10055450 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.302 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:20.301+0000 7f53c5306200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:17:20.359 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:20.357+0000 7f28b07ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:17:20.416 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:20.415+0000 7f53c5306200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:17:20.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.428+0000 7f0c1bfff700 1 -- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 msgr2=0x7f0c0006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:20.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.428+0000 7f0c1bfff700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 0x7f0c0006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:20.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.436+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c1c0c9340 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.436+0000 7f0c197fa700 1 -- 172.21.15.181:0/820247309 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c10020020 con 0x7f0c1c0b14a0 2023-10-16T11:17:20.438 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:17:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 msgr2=0x7f0c0006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 0x7f0c0006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0b14a0 msgr2=0x7f0c1c06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0b14a0 0x7f0c1c06c800 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c10016000 tx=0x7f0c10004750 comp rx=0 tx=0).stop 2023-10-16T11:17:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 shutdown_connections 2023-10-16T11:17:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f0c0006b020 0x7f0c0006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c0b14a0 0x7f0c1c06c800 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1c15f150 0x7f0c1c06fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 --2- 172.21.15.181:0/820247309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c1c0ce4f0 0x7f0c1c0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 >> 172.21.15.181:0/820247309 conn(0x7f0c1c09a930 msgr2=0x7f0c1c0ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:20.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 shutdown_connections 2023-10-16T11:17:20.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.439+0000 7f0c22afc700 1 -- 172.21.15.181:0/820247309 wait complete. 2023-10-16T11:17:20.455 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:17:20.483 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:20.481+0000 7f28b07ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:17:20.494 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:20.493+0000 7f53c5306200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:17:20.564 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:20.563+0000 7f28b07ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:17:20.577 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:20.576+0000 7f53c5306200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:17:20.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.582+0000 7f72969f2700 1 Processor -- start 2023-10-16T11:17:20.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.583+0000 7f72969f2700 1 -- start start 2023-10-16T11:17:20.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.583+0000 7f72969f2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f729015d9e0 0x7f72900d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.583+0000 7f72969f2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72900cefa0 0x7f72900cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.583+0000 7f72969f2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72900d4180 con 0x7f72900d00c0 2023-10-16T11:17:20.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.583+0000 7f72969f2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72900d20c0 con 0x7f729015d9e0 2023-10-16T11:17:20.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.583+0000 7f72969f2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900d1440 con 0x7f72900cefa0 2023-10-16T11:17:20.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f729015d9e0 0x7f72900d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f729015d9e0 0x7f72900d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53178/0 (socket says 172.21.15.181:53178) 2023-10-16T11:17:20.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f7294f8f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f72900d00c0 0x7f72900b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36028/0 (socket says 172.21.15.181:36028) 2023-10-16T11:17:20.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 -- 172.21.15.181:0/804691594 learned_addr learned my addr 172.21.15.181:0/804691594 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:20.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728f7fe700 1 --2- 172.21.15.181:0/804691594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72900cefa0 0x7f72900cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 -- 172.21.15.181:0/804691594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72900cefa0 msgr2=0x7f72900cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:20.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 --2- 172.21.15.181:0/804691594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72900cefa0 0x7f72900cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 -- 172.21.15.181:0/804691594 >> v1:172.21.15.181:6789/0 conn(0x7f72900d00c0 legacy=0x7f72900b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 -- 172.21.15.181:0/804691594 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900d1720 con 0x7f729015d9e0 2023-10-16T11:17:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728f7fe700 1 --2- 172.21.15.181:0/804691594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72900cefa0 0x7f72900cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728effd700 1 -- 172.21.15.181:0/804691594 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812322161 0 0) 0x7f72900d4180 con 0x7f72900d00c0 2023-10-16T11:17:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728ffff700 1 --2- 172.21.15.181:0/804691594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f729015d9e0 0x7f72900d24e0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f72800098f0 tx=0x7f7280009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20104b039b191db3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:20.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728effd700 1 -- 172.21.15.181:0/804691594 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728000b8a0 con 0x7f729015d9e0 2023-10-16T11:17:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728effd700 1 -- 172.21.15.181:0/804691594 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728000ba00 con 0x7f729015d9e0 2023-10-16T11:17:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.584+0000 7f728effd700 1 -- 172.21.15.181:0/804691594 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728000bd90 con 0x7f729015d9e0 2023-10-16T11:17:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 -- 172.21.15.181:0/804691594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f729015d9e0 msgr2=0x7f72900d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 --2- 172.21.15.181:0/804691594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f729015d9e0 0x7f72900d24e0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f72800098f0 tx=0x7f7280009c00 comp rx=0 tx=0).stop 2023-10-16T11:17:20.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 -- 172.21.15.181:0/804691594 shutdown_connections 2023-10-16T11:17:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 --2- 172.21.15.181:0/804691594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f729015d9e0 0x7f72900d24e0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 --2- 172.21.15.181:0/804691594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72900cefa0 0x7f72900cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 -- 172.21.15.181:0/804691594 >> 172.21.15.181:0/804691594 conn(0x7f729009cb40 msgr2=0x7f72900d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 -- 172.21.15.181:0/804691594 shutdown_connections 2023-10-16T11:17:20.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.585+0000 7f72969f2700 1 -- 172.21.15.181:0/804691594 wait complete. 2023-10-16T11:17:20.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.586+0000 7f72969f2700 1 Processor -- start 2023-10-16T11:17:20.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.587+0000 7f72969f2700 1 -- start start 2023-10-16T11:17:20.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.587+0000 7f72969f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72900cefa0 0x7f72900b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.587+0000 7f72969f2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72900d00c0 0x7f72900ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.587+0000 7f72969f2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f729015d9e0 0x7f72900ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.587+0000 7f72969f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729014d470 con 0x7f72900cefa0 2023-10-16T11:17:20.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.587+0000 7f72969f2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b42a0 con 0x7f72900d00c0 2023-10-16T11:17:20.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.587+0000 7f72969f2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729014c860 con 0x7f729015d9e0 2023-10-16T11:17:20.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72900cefa0 0x7f72900b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72900cefa0 0x7f72900b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53184/0 (socket says 172.21.15.181:53184) 2023-10-16T11:17:20.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 -- 172.21.15.181:0/1231647075 learned_addr learned my addr 172.21.15.181:0/1231647075 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:20.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728f7fe700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72900d00c0 0x7f72900ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f7294f8f700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f729015d9e0 0x7f72900ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 -- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f729015d9e0 msgr2=0x7f72900ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f729015d9e0 0x7f72900ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 -- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72900d00c0 msgr2=0x7f72900ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72900d00c0 0x7f72900ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 -- 172.21.15.181:0/1231647075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280009550 con 0x7f72900cefa0 2023-10-16T11:17:20.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728ffff700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72900cefa0 0x7f72900b3d60 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f729006b590 tx=0x7f7280004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:20.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728cff9700 1 -- 172.21.15.181:0/1231647075 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280004300 con 0x7f72900cefa0 2023-10-16T11:17:20.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728cff9700 1 -- 172.21.15.181:0/1231647075 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728001f070 con 0x7f72900cefa0 2023-10-16T11:17:20.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729014cb40 con 0x7f72900cefa0 2023-10-16T11:17:20.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f728cff9700 1 -- 172.21.15.181:0/1231647075 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800036a0 con 0x7f72900cefa0 2023-10-16T11:17:20.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.588+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729014d100 con 0x7f72900cefa0 2023-10-16T11:17:20.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.589+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7290067d20 con 0x7f72900cefa0 2023-10-16T11:17:20.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.590+0000 7f728cff9700 1 -- 172.21.15.181:0/1231647075 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f7280021030 con 0x7f72900cefa0 2023-10-16T11:17:20.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.590+0000 7f728cff9700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 0x7f727c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.590+0000 7f728f7fe700 1 -- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 msgr2=0x7f727c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:20.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.590+0000 7f728cff9700 1 -- 172.21.15.181:0/1231647075 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f728008af20 con 0x7f72900cefa0 2023-10-16T11:17:20.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.590+0000 7f728f7fe700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 0x7f727c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:20.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.595+0000 7f728cff9700 1 -- 172.21.15.181:0/1231647075 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7280055b70 con 0x7f72900cefa0 2023-10-16T11:17:20.650 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:20.648+0000 7f28b07ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:17:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.791+0000 7f728f7fe700 1 -- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 msgr2=0x7f727c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:20.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.791+0000 7f728f7fe700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 0x7f727c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:20.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.800+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f729015b8d0 con 0x7f72900cefa0 2023-10-16T11:17:20.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.801+0000 7f728cff9700 1 -- 172.21.15.181:0/1231647075 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v185) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7280059190 con 0x7f72900cefa0 2023-10-16T11:17:20.803 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:17:20.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 msgr2=0x7f727c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:20.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 0x7f727c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72900cefa0 msgr2=0x7f72900b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72900cefa0 0x7f72900b3d60 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f729006b590 tx=0x7f7280004750 comp rx=0 tx=0).stop 2023-10-16T11:17:20.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 shutdown_connections 2023-10-16T11:17:20.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f727c06b020 0x7f727c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f729015d9e0 0x7f72900ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72900cefa0 0x7f72900b3d60 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 --2- 172.21.15.181:0/1231647075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72900d00c0 0x7f72900ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 >> 172.21.15.181:0/1231647075 conn(0x7f729009cb40 msgr2=0x7f729015c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:20.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 shutdown_connections 2023-10-16T11:17:20.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.803+0000 7f72969f2700 1 -- 172.21.15.181:0/1231647075 wait complete. 2023-10-16T11:17:20.817 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:17:20.946 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:20.945+0000 7f53c5306200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:17:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 Processor -- start 2023-10-16T11:17:20.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 -- start start 2023-10-16T11:17:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe67c0b2010 0x7fe67c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c071be0 0x7fe67c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe67c06f3b0 0x7fe67c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c0ce1f0 con 0x7fe67c06f3b0 2023-10-16T11:17:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c0b2930 con 0x7fe67c0b2010 2023-10-16T11:17:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.983+0000 7fe6828a5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0ccda0 con 0x7fe67c071be0 2023-10-16T11:17:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe67c06f3b0 0x7fe67c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe67c06f3b0 0x7fe67c06f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53224/0 (socket says 172.21.15.181:53224) 2023-10-16T11:17:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 -- 172.21.15.181:0/2389205744 learned_addr learned my addr 172.21.15.181:0/2389205744 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe67bfff700 1 --2- 172.21.15.181:0/2389205744 >> v2:172.21.15.190:3300/0 conn(0x7fe67c0b2010 0x7fe67c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe67b7fe700 1 --2- 172.21.15.181:0/2389205744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c071be0 0x7fe67c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 -- 172.21.15.181:0/2389205744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c071be0 msgr2=0x7fe67c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 --2- 172.21.15.181:0/2389205744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c071be0 0x7fe67c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 -- 172.21.15.181:0/2389205744 >> v2:172.21.15.190:3300/0 conn(0x7fe67c0b2010 msgr2=0x7fe67c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 --2- 172.21.15.181:0/2389205744 >> v2:172.21.15.190:3300/0 conn(0x7fe67c0b2010 0x7fe67c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 -- 172.21.15.181:0/2389205744 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0727e0 con 0x7fe67c06f3b0 2023-10-16T11:17:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe67bfff700 1 --2- 172.21.15.181:0/2389205744 >> v2:172.21.15.190:3300/0 conn(0x7fe67c0b2010 0x7fe67c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe680e42700 1 --2- 172.21.15.181:0/2389205744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c06f3b0 0x7fe67c06f840 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fe67c070430 tx=0x7fe67400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289257449d23580 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:20.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe67affd700 1 -- 172.21.15.181:0/2389205744 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400e070 con 0x7fe67c06f3b0 2023-10-16T11:17:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe67affd700 1 -- 172.21.15.181:0/2389205744 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67400b910 con 0x7fe67c06f3b0 2023-10-16T11:17:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.984+0000 7fe67affd700 1 -- 172.21.15.181:0/2389205744 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400bcc0 con 0x7fe67c06f3b0 2023-10-16T11:17:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 -- 172.21.15.181:0/2389205744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c06f3b0 msgr2=0x7fe67c06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 --2- 172.21.15.181:0/2389205744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c06f3b0 0x7fe67c06f840 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fe67c070430 tx=0x7fe67400b330 comp rx=0 tx=0).stop 2023-10-16T11:17:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 -- 172.21.15.181:0/2389205744 shutdown_connections 2023-10-16T11:17:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 --2- 172.21.15.181:0/2389205744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c06f3b0 0x7fe67c06f840 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 --2- 172.21.15.181:0/2389205744 >> v2:172.21.15.190:3300/0 conn(0x7fe67c0b2010 0x7fe67c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 --2- 172.21.15.181:0/2389205744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c071be0 0x7fe67c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 -- 172.21.15.181:0/2389205744 >> 172.21.15.181:0/2389205744 conn(0x7fe67c0cd010 msgr2=0x7fe67c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 -- 172.21.15.181:0/2389205744 shutdown_connections 2023-10-16T11:17:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.985+0000 7fe6828a5700 1 -- 172.21.15.181:0/2389205744 wait complete. 2023-10-16T11:17:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.986+0000 7fe6828a5700 1 Processor -- start 2023-10-16T11:17:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.986+0000 7fe6828a5700 1 -- start start 2023-10-16T11:17:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe6828a5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c06f3b0 0x7fe67c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe6828a5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c071be0 0x7fe67c0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe6828a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c0b2010 0x7fe67c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe6828a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c144610 con 0x7fe67c0b2010 2023-10-16T11:17:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe6828a5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c144090 con 0x7fe67c071be0 2023-10-16T11:17:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe6828a5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0ca4b0 con 0x7fe67c06f3b0 2023-10-16T11:17:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe680e42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c0b2010 0x7fe67c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe680e42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c0b2010 0x7fe67c0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53230/0 (socket says 172.21.15.181:53230) 2023-10-16T11:17:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe680e42700 1 -- 172.21.15.181:0/3969962479 learned_addr learned my addr 172.21.15.181:0/3969962479 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe67b7fe700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c071be0 0x7fe67c0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.987+0000 7fe67bfff700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c06f3b0 0x7fe67c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe680e42700 1 -- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c06f3b0 msgr2=0x7fe67c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe680e42700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c06f3b0 0x7fe67c15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe680e42700 1 -- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c071be0 msgr2=0x7fe67c0c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe680e42700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c071be0 0x7fe67c0c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe680e42700 1 -- 172.21.15.181:0/3969962479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c00ba70 con 0x7fe67c0b2010 2023-10-16T11:17:21.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe680e42700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c0b2010 0x7fe67c0d00e0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7fe67400b680 tx=0x7fe674004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:21.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe678ff9700 1 -- 172.21.15.181:0/3969962479 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6740099b0 con 0x7fe67c0b2010 2023-10-16T11:17:21.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe678ff9700 1 -- 172.21.15.181:0/3969962479 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67400ce80 con 0x7fe67c0b2010 2023-10-16T11:17:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe674009b30 con 0x7fe67c0b2010 2023-10-16T11:17:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe678ff9700 1 -- 172.21.15.181:0/3969962479 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674004490 con 0x7fe67c0b2010 2023-10-16T11:17:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.988+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c15c510 con 0x7fe67c0b2010 2023-10-16T11:17:21.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.989+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c0b6510 con 0x7fe67c0b2010 2023-10-16T11:17:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.990+0000 7fe678ff9700 1 -- 172.21.15.181:0/3969962479 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe6740045f0 con 0x7fe67c0b2010 2023-10-16T11:17:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.990+0000 7fe678ff9700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 0x7fe66806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.990+0000 7fe67bfff700 1 -- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 msgr2=0x7fe66806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.990+0000 7fe678ff9700 1 -- 172.21.15.181:0/3969962479 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fe67408afc0 con 0x7fe67c0b2010 2023-10-16T11:17:21.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.990+0000 7fe67bfff700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 0x7fe66806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:20.994+0000 7fe678ff9700 1 -- 172.21.15.181:0/3969962479 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe6740bdfe0 con 0x7fe67c0b2010 2023-10-16T11:17:21.046 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:21.044+0000 7f28b07ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:17:21.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.191+0000 7fe67bfff700 1 -- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 msgr2=0x7fe66806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:21.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.191+0000 7fe67bfff700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 0x7fe66806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:21.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.194+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe67c0b5b00 con 0x7fe67c0b2010 2023-10-16T11:17:21.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.195+0000 7fe678ff9700 1 -- 172.21.15.181:0/3969962479 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v185) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6740594a0 con 0x7fe67c0b2010 2023-10-16T11:17:21.197 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:17:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 msgr2=0x7fe66806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 0x7fe66806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c0b2010 msgr2=0x7fe67c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c0b2010 0x7fe67c0d00e0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7fe67400b680 tx=0x7fe674004ad0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 shutdown_connections 2023-10-16T11:17:21.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe66806b020 0x7fe66806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c06f3b0 0x7fe67c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe67c0b2010 0x7fe67c0d00e0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 --2- 172.21.15.181:0/3969962479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c071be0 0x7fe67c0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 >> 172.21.15.181:0/3969962479 conn(0x7fe67c0cd010 msgr2=0x7fe67c0b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 shutdown_connections 2023-10-16T11:17:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.197+0000 7fe6828a5700 1 -- 172.21.15.181:0/3969962479 wait complete. 2023-10-16T11:17:21.212 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:17:21.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.334+0000 7fef981d9700 1 Processor -- start 2023-10-16T11:17:21.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.334+0000 7fef981d9700 1 -- start start 2023-10-16T11:17:21.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef981d9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef9015b9f0 0x7fef900cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef981d9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fef900cfb50 0x7fef900ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef981d9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9006b370 con 0x7fef900cc400 2023-10-16T11:17:21.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef981d9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9006b800 con 0x7fef9015b9f0 2023-10-16T11:17:21.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef981d9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9006aa40 con 0x7fef900cfb50 2023-10-16T11:17:21.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef95f75700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef9015b9f0 0x7fef900cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef95774700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fef900cc400 0x7fef900cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36082/0 (socket says 172.21.15.181:36082) 2023-10-16T11:17:21.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef95774700 1 -- 172.21.15.181:0/4048861332 learned_addr learned my addr 172.21.15.181:0/4048861332 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:21.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef96776700 1 --2- 172.21.15.181:0/4048861332 >> v2:172.21.15.190:3300/0 conn(0x7fef900cfb50 0x7fef900ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef95f75700 1 -- 172.21.15.181:0/4048861332 >> v2:172.21.15.190:3300/0 conn(0x7fef900cfb50 msgr2=0x7fef900ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:21.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef95f75700 1 --2- 172.21.15.181:0/4048861332 >> v2:172.21.15.190:3300/0 conn(0x7fef900cfb50 0x7fef900ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef95f75700 1 -- 172.21.15.181:0/4048861332 >> v1:172.21.15.181:6789/0 conn(0x7fef900cc400 legacy=0x7fef900cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.335+0000 7fef95f75700 1 -- 172.21.15.181:0/4048861332 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9006ad20 con 0x7fef9015b9f0 2023-10-16T11:17:21.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef95f75700 1 --2- 172.21.15.181:0/4048861332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9015b9f0 0x7fef900cd950 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fef90070580 tx=0x7fef88009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba02db6b3b3a2f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:21.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef94f73700 1 -- 172.21.15.181:0/4048861332 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88005e30 con 0x7fef9015b9f0 2023-10-16T11:17:21.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef94f73700 1 -- 172.21.15.181:0/4048861332 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef88006030 con 0x7fef9015b9f0 2023-10-16T11:17:21.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef94f73700 1 -- 172.21.15.181:0/4048861332 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880063c0 con 0x7fef9015b9f0 2023-10-16T11:17:21.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef981d9700 1 -- 172.21.15.181:0/4048861332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9015b9f0 msgr2=0x7fef900cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef981d9700 1 --2- 172.21.15.181:0/4048861332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9015b9f0 0x7fef900cd950 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fef90070580 tx=0x7fef88009b00 comp rx=0 tx=0).stop 2023-10-16T11:17:21.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef981d9700 1 -- 172.21.15.181:0/4048861332 shutdown_connections 2023-10-16T11:17:21.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef981d9700 1 --2- 172.21.15.181:0/4048861332 >> v2:172.21.15.190:3300/0 conn(0x7fef900cfb50 0x7fef900ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef981d9700 1 --2- 172.21.15.181:0/4048861332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef9015b9f0 0x7fef900cd950 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.336+0000 7fef981d9700 1 -- 172.21.15.181:0/4048861332 >> 172.21.15.181:0/4048861332 conn(0x7fef900d04a0 msgr2=0x7fef900d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:21.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.337+0000 7fef981d9700 1 -- 172.21.15.181:0/4048861332 shutdown_connections 2023-10-16T11:17:21.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.337+0000 7fef981d9700 1 -- 172.21.15.181:0/4048861332 wait complete. 2023-10-16T11:17:21.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 Processor -- start 2023-10-16T11:17:21.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 -- start start 2023-10-16T11:17:21.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef900cc400 0x7fef90157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef900cfb50 0x7fef901f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9015b9f0 0x7fef900b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901f4590 con 0x7fef900cc400 2023-10-16T11:17:21.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900b5e40 con 0x7fef900cfb50 2023-10-16T11:17:21.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.338+0000 7fef981d9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900b6120 con 0x7fef9015b9f0 2023-10-16T11:17:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95f75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef900cc400 0x7fef90157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95f75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef900cc400 0x7fef90157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53242/0 (socket says 172.21.15.181:53242) 2023-10-16T11:17:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95f75700 1 -- 172.21.15.181:0/2623382204 learned_addr learned my addr 172.21.15.181:0/2623382204 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95774700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef900cfb50 0x7fef901f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef96776700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9015b9f0 0x7fef900b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95774700 1 -- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9015b9f0 msgr2=0x7fef900b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95774700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9015b9f0 0x7fef900b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95774700 1 -- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef900cc400 msgr2=0x7fef90157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95774700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef900cc400 0x7fef90157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95774700 1 -- 172.21.15.181:0/2623382204 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009430 con 0x7fef900cfb50 2023-10-16T11:17:21.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95f75700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef900cc400 0x7fef90157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:21.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef96776700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9015b9f0 0x7fef900b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:21.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef95774700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef900cfb50 0x7fef901f4a20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c500 tx=0x7fef8000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:21.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef867fc700 1 -- 172.21.15.181:0/2623382204 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80005d70 con 0x7fef900cfb50 2023-10-16T11:17:21.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef867fc700 1 -- 172.21.15.181:0/2623382204 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef800070d0 con 0x7fef900cfb50 2023-10-16T11:17:21.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef901f4220 con 0x7fef900cfb50 2023-10-16T11:17:21.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef867fc700 1 -- 172.21.15.181:0/2623382204 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80007440 con 0x7fef900cfb50 2023-10-16T11:17:21.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.339+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900b5b30 con 0x7fef900cfb50 2023-10-16T11:17:21.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.340+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef90154cc0 con 0x7fef900cfb50 2023-10-16T11:17:21.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.341+0000 7fef867fc700 1 -- 172.21.15.181:0/2623382204 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fef800075a0 con 0x7fef900cfb50 2023-10-16T11:17:21.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.341+0000 7fef867fc700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 0x7fef7406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.341+0000 7fef95f75700 1 -- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 msgr2=0x7fef7406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:21.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.341+0000 7fef95f75700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 0x7fef7406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:21.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.349+0000 7fef867fc700 1 -- 172.21.15.181:0/2623382204 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fef80057ae0 con 0x7fef900cfb50 2023-10-16T11:17:21.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.349+0000 7fef867fc700 1 -- 172.21.15.181:0/2623382204 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fef80053e20 con 0x7fef900cfb50 2023-10-16T11:17:21.515 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:21.513+0000 7f53c5306200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:17:21.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.542+0000 7fef95f75700 1 -- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 msgr2=0x7fef7406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:21.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.542+0000 7fef95f75700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 0x7fef7406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:21.584 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:21.582+0000 7f53c5306200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:17:21.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.597+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef901f4700 con 0x7fef900cfb50 2023-10-16T11:17:21.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.598+0000 7fef867fc700 1 -- 172.21.15.181:0/2623382204 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v185) v1 ==== 81+0+951 (secure 0 0 0) 0x7fef80017080 con 0x7fef900cfb50 2023-10-16T11:17:21.600 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/4003028162 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.600 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3290548668 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.600 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3601081788 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.601 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3601081788 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.601 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2238590139 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.601 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/231495737 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.601 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1499024696 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.601 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/458160361 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.602 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1195985489 2023-10-17T11:12:21.628206+0000 2023-10-16T11:17:21.602 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/4000110033 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.602 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/990187329 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.602 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3304094772 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.602 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/303240986 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.603 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1162194396 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.603 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/303240986 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.603 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3436855298 2023-10-17T11:01:37.009118+0000 2023-10-16T11:17:21.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 msgr2=0x7fef7406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 0x7fef7406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef900cfb50 msgr2=0x7fef901f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef900cfb50 0x7fef901f4a20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c500 tx=0x7fef8000c810 comp rx=0 tx=0).stop 2023-10-16T11:17:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 shutdown_connections 2023-10-16T11:17:21.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fef7406b020 0x7fef7406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef9015b9f0 0x7fef900b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef900cc400 0x7fef90157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 --2- 172.21.15.181:0/2623382204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef900cfb50 0x7fef901f4a20 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 >> 172.21.15.181:0/2623382204 conn(0x7fef900d04a0 msgr2=0x7fef900b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 shutdown_connections 2023-10-16T11:17:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.600+0000 7fef981d9700 1 -- 172.21.15.181:0/2623382204 wait complete. 2023-10-16T11:17:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:listed 16 entries 2023-10-16T11:17:21.615 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:17:21.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:21.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:17:21.646 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:21.645+0000 7f28b07ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:17:21.661 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:17:21.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:21.662 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:17:21.718 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:17:21.719 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:21.722 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:21.720+0000 7f28b07ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:17:21.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:17:21.736 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:17:21.737 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:21.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:17:21.755 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:17:21.755 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:21.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:17:21.794 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:17:21.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.916+0000 7f0513b76700 1 Processor -- start 2023-10-16T11:17:21.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.916+0000 7f0513b76700 1 -- start start 2023-10-16T11:17:21.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.917+0000 7f0513b76700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f050c0cac20 0x7f050c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0513b76700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cb000 0x7f050c15da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0513b76700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f050c0cc4d0 0x7f050c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0513b76700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050c0cc0b0 con 0x7f050c0cc4d0 2023-10-16T11:17:21.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0513b76700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050c15df40 con 0x7f050c0cac20 2023-10-16T11:17:21.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0513b76700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c15e210 con 0x7f050c0cb000 2023-10-16T11:17:21.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f050c0cc4d0 0x7f050c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f050c0cc4d0 0x7f050c0cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53256/0 (socket says 172.21.15.181:53256) 2023-10-16T11:17:21.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 -- 172.21.15.181:0/242469193 learned_addr learned my addr 172.21.15.181:0/242469193 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:21.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0511111700 1 --2- 172.21.15.181:0/242469193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cb000 0x7f050c15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0511912700 1 --2- 172.21.15.181:0/242469193 >> v2:172.21.15.190:3300/0 conn(0x7f050c0cac20 0x7f050c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 -- 172.21.15.181:0/242469193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cb000 msgr2=0x7f050c15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:21.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 --2- 172.21.15.181:0/242469193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cb000 0x7f050c15da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 -- 172.21.15.181:0/242469193 >> v2:172.21.15.190:3300/0 conn(0x7f050c0cac20 msgr2=0x7f050c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:21.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 --2- 172.21.15.181:0/242469193 >> v2:172.21.15.190:3300/0 conn(0x7f050c0cac20 0x7f050c15af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 -- 172.21.15.181:0/242469193 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c15e4f0 con 0x7f050c0cc4d0 2023-10-16T11:17:21.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0511912700 1 --2- 172.21.15.181:0/242469193 >> v2:172.21.15.190:3300/0 conn(0x7f050c0cac20 0x7f050c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:21.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0512113700 1 --2- 172.21.15.181:0/242469193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cc4d0 0x7f050c0cf090 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f050c0ccf80 tx=0x7f050400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af90cda3d9191eaf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:21.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0510910700 1 -- 172.21.15.181:0/242469193 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050400e070 con 0x7f050c0cc4d0 2023-10-16T11:17:21.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0510910700 1 -- 172.21.15.181:0/242469193 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050400b910 con 0x7f050c0cc4d0 2023-10-16T11:17:21.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.918+0000 7f0510910700 1 -- 172.21.15.181:0/242469193 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050400bcc0 con 0x7f050c0cc4d0 2023-10-16T11:17:21.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 -- 172.21.15.181:0/242469193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cc4d0 msgr2=0x7f050c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 --2- 172.21.15.181:0/242469193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cc4d0 0x7f050c0cf090 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f050c0ccf80 tx=0x7f050400b330 comp rx=0 tx=0).stop 2023-10-16T11:17:21.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 -- 172.21.15.181:0/242469193 shutdown_connections 2023-10-16T11:17:21.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 --2- 172.21.15.181:0/242469193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cc4d0 0x7f050c0cf090 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 --2- 172.21.15.181:0/242469193 >> v2:172.21.15.190:3300/0 conn(0x7f050c0cac20 0x7f050c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 --2- 172.21.15.181:0/242469193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cb000 0x7f050c15da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 -- 172.21.15.181:0/242469193 >> 172.21.15.181:0/242469193 conn(0x7f050c06aed0 msgr2=0x7f050c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:21.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 -- 172.21.15.181:0/242469193 shutdown_connections 2023-10-16T11:17:21.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.919+0000 7f0513b76700 1 -- 172.21.15.181:0/242469193 wait complete. 2023-10-16T11:17:21.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 Processor -- start 2023-10-16T11:17:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 -- start start 2023-10-16T11:17:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cac20 0x7f050c0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050c0cb000 0x7f050c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cc4d0 0x7f050c0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0c3580 con 0x7f050c0cac20 2023-10-16T11:17:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c154ea0 con 0x7f050c0cb000 2023-10-16T11:17:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0513b76700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0ae890 con 0x7f050c0cc4d0 2023-10-16T11:17:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.921+0000 7f0511912700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cac20 0x7f050c0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cac20 0x7f050c0c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53258/0 (socket says 172.21.15.181:53258) 2023-10-16T11:17:21.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 -- 172.21.15.181:0/4125548199 learned_addr learned my addr 172.21.15.181:0/4125548199 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:21.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511111700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050c0cb000 0x7f050c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0512113700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cc4d0 0x7f050c0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:21.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 -- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cc4d0 msgr2=0x7f050c0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cc4d0 0x7f050c0addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 -- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050c0cb000 msgr2=0x7f050c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:21.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050c0cb000 0x7f050c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:21.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 -- 172.21.15.181:0/4125548199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05080094a0 con 0x7f050c0cac20 2023-10-16T11:17:21.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0511912700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cac20 0x7f050c0c3040 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f050800ebe0 tx=0x7f050800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:21.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f05027fc700 1 -- 172.21.15.181:0/4125548199 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508009780 con 0x7f050c0cac20 2023-10-16T11:17:21.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f05027fc700 1 -- 172.21.15.181:0/4125548199 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050800c850 con 0x7f050c0cac20 2023-10-16T11:17:21.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0504009b30 con 0x7f050c0cac20 2023-10-16T11:17:21.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f05027fc700 1 -- 172.21.15.181:0/4125548199 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050800cbe0 con 0x7f050c0cac20 2023-10-16T11:17:21.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.922+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c0cbd80 con 0x7f050c0cac20 2023-10-16T11:17:21.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.923+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c04ebb0 con 0x7f050c0cac20 2023-10-16T11:17:21.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.924+0000 7f05027fc700 1 -- 172.21.15.181:0/4125548199 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f050800cd40 con 0x7f050c0cac20 2023-10-16T11:17:21.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.924+0000 7f05027fc700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 0x7f04f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:21.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.924+0000 7f0511111700 1 -- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 msgr2=0x7f04f006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:21.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.924+0000 7f05027fc700 1 -- 172.21.15.181:0/4125548199 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f0508089bf0 con 0x7f050c0cac20 2023-10-16T11:17:21.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.924+0000 7f0511111700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 0x7f04f006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:21.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:21.929+0000 7f05027fc700 1 -- 172.21.15.181:0/4125548199 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0508054720 con 0x7f050c0cac20 2023-10-16T11:17:22.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.125+0000 7f0511111700 1 -- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 msgr2=0x7f04f006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:22.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.125+0000 7f0511111700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 0x7f04f006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:22.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.173+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f050c15a990 con 0x7f050c0cac20 2023-10-16T11:17:22.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.174+0000 7f05027fc700 1 -- 172.21.15.181:0/4125548199 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f0508057d40 con 0x7f050c0cac20 2023-10-16T11:17:22.176 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:22.176 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1903420066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1903420066},{"type":"v1","addr":"172.21.15.181:6836","nonce":1903420066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/586483327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":586483327},{"type":"v1","addr":"172.21.15.190:6835","nonce":586483327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2078148321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2078148321},{"type":"v1","addr":"172.21.15.181:6837","nonce":2078148321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:01:22.987835+0000","modified":"2023-10-16T11:01:26.027862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10197},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10197":{"gid":10197,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2135938520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2135938520},{"type":"v1","addr":"172.21.15.190:6834","nonce":2135938520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:17:22.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 msgr2=0x7f04f006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 0x7f04f006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cac20 msgr2=0x7f050c0c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cac20 0x7f050c0c3040 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f050800ebe0 tx=0x7f050800eef0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 shutdown_connections 2023-10-16T11:17:22.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f04f006b020 0x7f04f006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050c0cc4d0 0x7f050c0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050c0cac20 0x7f050c0c3040 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 --2- 172.21.15.181:0/4125548199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050c0cb000 0x7f050c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 >> 172.21.15.181:0/4125548199 conn(0x7f050c06aed0 msgr2=0x7f050c0c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:22.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 shutdown_connections 2023-10-16T11:17:22.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.176+0000 7f0513b76700 1 -- 172.21.15.181:0/4125548199 wait complete. 2023-10-16T11:17:22.181 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 94 2023-10-16T11:17:22.191 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:22.221 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:22.219+0000 7f53c5306200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:22.298 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:22.296+0000 7f53c5306200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:17:22.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.316+0000 7ff3564d1700 1 Processor -- start 2023-10-16T11:17:22.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.316+0000 7ff3564d1700 1 -- start start 2023-10-16T11:17:22.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff3564d1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff35006d5f0 0x7ff3500d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff3564d1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff3501611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff3564d1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3501616f0 con 0x7ff350161b60 2023-10-16T11:17:22.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff3564d1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff350162800 con 0x7ff35006d5f0 2023-10-16T11:17:22.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff3564d1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500d43e0 con 0x7ff350160d20 2023-10-16T11:17:22.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff34ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff350161b60 0x7ff350161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36114/0 (socket says 172.21.15.181:36114) 2023-10-16T11:17:22.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff34ffff700 1 -- 172.21.15.181:0/828266016 learned_addr learned my addr 172.21.15.181:0/828266016 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:22.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff34f7fe700 1 --2- 172.21.15.181:0/828266016 >> v2:172.21.15.190:3300/0 conn(0x7ff35006d5f0 0x7ff3500d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff354a6e700 1 --2- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff3501611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff34effd700 1 -- 172.21.15.181:0/828266016 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498288543 0 0) 0x7ff3501616f0 con 0x7ff350161b60 2023-10-16T11:17:22.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.317+0000 7ff34effd700 1 -- 172.21.15.181:0/828266016 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3380035b0 con 0x7ff350161b60 2023-10-16T11:17:22.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34f7fe700 1 -- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 msgr2=0x7ff3501611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:22.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34f7fe700 1 --2- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff3501611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34f7fe700 1 -- 172.21.15.181:0/828266016 >> v1:172.21.15.181:6789/0 conn(0x7ff350161b60 legacy=0x7ff350161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34f7fe700 1 -- 172.21.15.181:0/828266016 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3500d2560 con 0x7ff35006d5f0 2023-10-16T11:17:22.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff354a6e700 1 --2- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff3501611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:22.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34f7fe700 1 --2- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff35006d5f0 0x7ff3500d2020 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff344009840 tx=0x7ff344009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9fd62194c74e860 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:22.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34effd700 1 -- 172.21.15.181:0/828266016 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344011830 con 0x7ff35006d5f0 2023-10-16T11:17:22.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34effd700 1 -- 172.21.15.181:0/828266016 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344011990 con 0x7ff35006d5f0 2023-10-16T11:17:22.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff34effd700 1 -- 172.21.15.181:0/828266016 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344011d20 con 0x7ff35006d5f0 2023-10-16T11:17:22.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff3564d1700 1 -- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff35006d5f0 msgr2=0x7ff3500d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.318+0000 7ff3564d1700 1 --2- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff35006d5f0 0x7ff3500d2020 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff344009840 tx=0x7ff344009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:22.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.319+0000 7ff3564d1700 1 -- 172.21.15.181:0/828266016 shutdown_connections 2023-10-16T11:17:22.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.319+0000 7ff3564d1700 1 --2- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff3501611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.319+0000 7ff3564d1700 1 --2- 172.21.15.181:0/828266016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff35006d5f0 0x7ff3500d2020 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.319+0000 7ff3564d1700 1 -- 172.21.15.181:0/828266016 >> 172.21.15.181:0/828266016 conn(0x7ff35009cba0 msgr2=0x7ff3500d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:22.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.319+0000 7ff3564d1700 1 -- 172.21.15.181:0/828266016 shutdown_connections 2023-10-16T11:17:22.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.319+0000 7ff3564d1700 1 -- 172.21.15.181:0/828266016 wait complete. 2023-10-16T11:17:22.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.320+0000 7ff3564d1700 1 Processor -- start 2023-10-16T11:17:22.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.320+0000 7ff3564d1700 1 -- start start 2023-10-16T11:17:22.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35006d5f0 0x7ff3500ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff35015d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff350161b60 0x7ff35015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cba30 con 0x7ff35006d5f0 2023-10-16T11:17:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500caeb0 con 0x7ff350161b60 2023-10-16T11:17:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cb190 con 0x7ff350160d20 2023-10-16T11:17:22.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35006d5f0 0x7ff3500ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35006d5f0 0x7ff3500ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53274/0 (socket says 172.21.15.181:53274) 2023-10-16T11:17:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 -- 172.21.15.181:0/2276348476 learned_addr learned my addr 172.21.15.181:0/2276348476 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34f7fe700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff35015d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff354a6e700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff350161b60 0x7ff35015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 -- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 msgr2=0x7ff35015d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff35015d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 -- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff350161b60 msgr2=0x7ff35015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff350161b60 0x7ff35015c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 -- 172.21.15.181:0/2276348476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348009a70 con 0x7ff35006d5f0 2023-10-16T11:17:22.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34ffff700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35006d5f0 0x7ff3500ca8d0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff340001260 tx=0x7ff34000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34cff9700 1 -- 172.21.15.181:0/2276348476 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3400074e0 con 0x7ff35006d5f0 2023-10-16T11:17:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34cff9700 1 -- 172.21.15.181:0/2276348476 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff340007640 con 0x7ff35006d5f0 2023-10-16T11:17:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3440094a0 con 0x7ff35006d5f0 2023-10-16T11:17:22.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff34cff9700 1 -- 172.21.15.181:0/2276348476 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340005a50 con 0x7ff35006d5f0 2023-10-16T11:17:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.321+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3500cb750 con 0x7ff35006d5f0 2023-10-16T11:17:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.322+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35004ebb0 con 0x7ff35006d5f0 2023-10-16T11:17:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.323+0000 7ff34cff9700 1 -- 172.21.15.181:0/2276348476 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff340005bb0 con 0x7ff35006d5f0 2023-10-16T11:17:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.323+0000 7ff34cff9700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 0x7ff33c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.323+0000 7ff34f7fe700 1 -- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 msgr2=0x7ff33c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.324+0000 7ff34cff9700 1 -- 172.21.15.181:0/2276348476 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7ff340089040 con 0x7ff35006d5f0 2023-10-16T11:17:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.324+0000 7ff34f7fe700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 0x7ff33c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:22.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.329+0000 7ff34cff9700 1 -- 172.21.15.181:0/2276348476 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff340053ae0 con 0x7ff35006d5f0 2023-10-16T11:17:22.389 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:22.387+0000 7f28b07ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:22.443 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:22.441+0000 7f53c5306200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:22.472 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:22.470+0000 7f28b07ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:17:22.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.525+0000 7ff34f7fe700 1 -- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 msgr2=0x7ff33c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:22.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.525+0000 7ff34f7fe700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 0x7ff33c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:22.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.530+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3500d0db0 con 0x7ff35006d5f0 2023-10-16T11:17:22.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.532+0000 7ff34cff9700 1 -- 172.21.15.181:0/2276348476 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+18545 (secure 0 0 0) 0x7ff340017090 con 0x7ff35006d5f0 2023-10-16T11:17:22.534 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:22.534 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":185,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:20.424880+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:01:21.420382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:01:21.958383+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":185,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/4003028162":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/3290548668":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6833/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6832/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/2238590139":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/231495737":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/1499024696":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/458160361":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1195985489":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/4000110033":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/990187329":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3304094772":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6833/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1162194396":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6832/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3436855298":"2023-10-17T11:01:37.009118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":15,"length":2},{"begin":31,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 msgr2=0x7ff33c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 0x7ff33c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35006d5f0 msgr2=0x7ff3500ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35006d5f0 0x7ff3500ca8d0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff340001260 tx=0x7ff34000c7b0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 shutdown_connections 2023-10-16T11:17:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff350161b60 0x7ff35015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff35006d5f0 0x7ff3500ca8d0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7ff33c06b020 0x7ff33c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 --2- 172.21.15.181:0/2276348476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff350160d20 0x7ff35015d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 >> 172.21.15.181:0/2276348476 conn(0x7ff35009cba0 msgr2=0x7ff3500cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 shutdown_connections 2023-10-16T11:17:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.534+0000 7ff3564d1700 1 -- 172.21.15.181:0/2276348476 wait complete. 2023-10-16T11:17:22.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:01:21.420382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '180', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:01:21.958383+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 185, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:17:22.550 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:17:22.626 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:22.624+0000 7f28b07ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:22.672 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:22.670+0000 7f53c5306200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:17:22.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8da60e700 1 Processor -- start 2023-10-16T11:17:22.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8da60e700 1 -- start start 2023-10-16T11:17:22.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8da60e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8d40cb2b0 0x7fd8d406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8da60e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8d40cb690 0x7fd8d40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8da60e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d4070590 con 0x7fd8d40b2ba0 2023-10-16T11:17:22.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8da60e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d4156330 con 0x7fd8d40cb690 2023-10-16T11:17:22.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8da60e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d406fcf0 con 0x7fd8d40cb2b0 2023-10-16T11:17:22.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.675+0000 7fd8d37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8d40cb690 0x7fd8d40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8d40cb690 0x7fd8d40b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53304/0 (socket says 172.21.15.181:53304) 2023-10-16T11:17:22.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d37fe700 1 -- 172.21.15.181:0/2791666822 learned_addr learned my addr 172.21.15.181:0/2791666822 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:22.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d8bab700 1 --1- 172.21.15.181:0/2791666822 >> v1:172.21.15.181:6789/0 conn(0x7fd8d40b2ba0 0x7fd8d415f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36154/0 (socket says 172.21.15.181:36154) 2023-10-16T11:17:22.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d3fff700 1 --2- 172.21.15.181:0/2791666822 >> v2:172.21.15.190:3300/0 conn(0x7fd8d40cb2b0 0x7fd8d406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d37fe700 1 -- 172.21.15.181:0/2791666822 >> v2:172.21.15.190:3300/0 conn(0x7fd8d40cb2b0 msgr2=0x7fd8d406f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:22.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d37fe700 1 --2- 172.21.15.181:0/2791666822 >> v2:172.21.15.190:3300/0 conn(0x7fd8d40cb2b0 0x7fd8d406f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d37fe700 1 -- 172.21.15.181:0/2791666822 >> v1:172.21.15.181:6789/0 conn(0x7fd8d40b2ba0 legacy=0x7fd8d415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d37fe700 1 -- 172.21.15.181:0/2791666822 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d406ffd0 con 0x7fd8d40cb690 2023-10-16T11:17:22.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d2ffd700 1 -- 172.21.15.181:0/2791666822 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694152805 0 0) 0x7fd8d4070590 con 0x7fd8d40b2ba0 2023-10-16T11:17:22.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d37fe700 1 --2- 172.21.15.181:0/2791666822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40b2660 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40d1600 tx=0x7fd8bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab2a476c1ceea09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d2ffd700 1 -- 172.21.15.181:0/2791666822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc011830 con 0x7fd8d40cb690 2023-10-16T11:17:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d2ffd700 1 -- 172.21.15.181:0/2791666822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc011990 con 0x7fd8d40cb690 2023-10-16T11:17:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.676+0000 7fd8d2ffd700 1 -- 172.21.15.181:0/2791666822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc011d20 con 0x7fd8d40cb690 2023-10-16T11:17:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 -- 172.21.15.181:0/2791666822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 msgr2=0x7fd8d40b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 --2- 172.21.15.181:0/2791666822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40b2660 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40d1600 tx=0x7fd8bc009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 -- 172.21.15.181:0/2791666822 shutdown_connections 2023-10-16T11:17:22.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 --2- 172.21.15.181:0/2791666822 >> v2:172.21.15.190:3300/0 conn(0x7fd8d40cb2b0 0x7fd8d406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 --2- 172.21.15.181:0/2791666822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40b2660 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 -- 172.21.15.181:0/2791666822 >> 172.21.15.181:0/2791666822 conn(0x7fd8d40cd010 msgr2=0x7fd8d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 -- 172.21.15.181:0/2791666822 shutdown_connections 2023-10-16T11:17:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.677+0000 7fd8da60e700 1 -- 172.21.15.181:0/2791666822 wait complete. 2023-10-16T11:17:22.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.678+0000 7fd8da60e700 1 Processor -- start 2023-10-16T11:17:22.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.679+0000 7fd8da60e700 1 -- start start 2023-10-16T11:17:22.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.679+0000 7fd8da60e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d40b2ba0 0x7fd8d40c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8da60e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d40cb2b0 0x7fd8d4154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8da60e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8da60e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d4154c60 con 0x7fd8d40cb690 2023-10-16T11:17:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8da60e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40c5c80 con 0x7fd8d40b2ba0 2023-10-16T11:17:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8da60e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40ca460 con 0x7fd8d40cb2b0 2023-10-16T11:17:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53314/0 (socket says 172.21.15.181:53314) 2023-10-16T11:17:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 -- 172.21.15.181:0/1161018556 learned_addr learned my addr 172.21.15.181:0/1161018556 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d37fe700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d40cb2b0 0x7fd8d4154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d3fff700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d40b2ba0 0x7fd8d40c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 -- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d40cb2b0 msgr2=0x7fd8d4154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d40cb2b0 0x7fd8d4154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 -- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d40b2ba0 msgr2=0x7fd8d40c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d40b2ba0 0x7fd8d40c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 -- 172.21.15.181:0/1161018556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc009550 con 0x7fd8d40cb690 2023-10-16T11:17:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d8bab700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40c9400 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc002230 tx=0x7fd8cc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d0ff9700 1 -- 172.21.15.181:0/1161018556 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00aac0 con 0x7fd8d40cb690 2023-10-16T11:17:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d0ff9700 1 -- 172.21.15.181:0/1161018556 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc00ac40 con 0x7fd8d40cb690 2023-10-16T11:17:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d40ca180 con 0x7fd8d40cb690 2023-10-16T11:17:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8d0ff9700 1 -- 172.21.15.181:0/1161018556 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc0070d0 con 0x7fd8d40cb690 2023-10-16T11:17:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.680+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d40c6200 con 0x7fd8d40cb690 2023-10-16T11:17:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.681+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d40c5ae0 con 0x7fd8d40cb690 2023-10-16T11:17:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.682+0000 7fd8d0ff9700 1 -- 172.21.15.181:0/1161018556 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd8cc007250 con 0x7fd8d40cb690 2023-10-16T11:17:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.682+0000 7fd8d0ff9700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.682+0000 7fd8d3fff700 1 -- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 msgr2=0x7fd8b406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.682+0000 7fd8d0ff9700 1 -- 172.21.15.181:0/1161018556 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fd8cc089aa0 con 0x7fd8d40cb690 2023-10-16T11:17:22.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.682+0000 7fd8d3fff700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:22.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.688+0000 7fd8d0ff9700 1 -- 172.21.15.181:0/1161018556 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd8d40c5ae0 con 0x7fd8d40cb690 2023-10-16T11:17:22.746 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:22.745+0000 7f53c5306200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:17:22.868 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:22.866+0000 7f28b07ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:17:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.883+0000 7fd8d3fff700 1 -- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 msgr2=0x7fd8b406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.883+0000 7fd8d3fff700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:22.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.885+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8d40c5ae0 con 0x7fd8d40cb690 2023-10-16T11:17:22.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.886+0000 7fd8d0ff9700 1 -- 172.21.15.181:0/1161018556 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8d40c5ae0 con 0x7fd8d40cb690 2023-10-16T11:17:22.888 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:17:22.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 msgr2=0x7fd8b406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:22.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 msgr2=0x7fd8d40c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:22.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40c9400 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc002230 tx=0x7fd8cc00cf50 comp rx=0 tx=0).stop 2023-10-16T11:17:22.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 shutdown_connections 2023-10-16T11:17:22.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8d40cb690 0x7fd8d40c9400 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8d40b2ba0 0x7fd8d40c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd8b406b020 0x7fd8b406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 --2- 172.21.15.181:0/1161018556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8d40cb2b0 0x7fd8d4154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:22.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 >> 172.21.15.181:0/1161018556 conn(0x7fd8d40cd010 msgr2=0x7fd8d4073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:22.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 shutdown_connections 2023-10-16T11:17:22.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:22.888+0000 7fd8da60e700 1 -- 172.21.15.181:0/1161018556 wait complete. 2023-10-16T11:17:22.902 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:17:22.949 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:22.947+0000 7f28b07ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:17:23.010 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:23.008+0000 7f53c5306200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:17:23.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.030+0000 7fe712d6f700 1 Processor -- start 2023-10-16T11:17:23.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.030+0000 7fe712d6f700 1 -- start start 2023-10-16T11:17:23.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.030+0000 7fe712d6f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe70c15d740 0x7fe70c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe712d6f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0d0cb0 con 0x7fe70c0cf890 2023-10-16T11:17:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe712d6f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0d00a0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe712d6f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c1624b0 con 0x7fe70c15d740 2023-10-16T11:17:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe70bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe70c0cf890 0x7fe70c0cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36168/0 (socket says 172.21.15.181:36168) 2023-10-16T11:17:23.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe70bfff700 1 -- 172.21.15.181:0/3549918853 learned_addr learned my addr 172.21.15.181:0/3549918853 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe710b0b700 1 --2- 172.21.15.181:0/3549918853 >> v2:172.21.15.190:3300/0 conn(0x7fe70c15d740 0x7fe70c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe70b7fe700 1 -- 172.21.15.181:0/3549918853 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033659805 0 0) 0x7fe70c0d0cb0 con 0x7fe70c0cf890 2023-10-16T11:17:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe70b7fe700 1 -- 172.21.15.181:0/3549918853 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6fc0035b0 con 0x7fe70c0cf890 2023-10-16T11:17:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe710b0b700 1 -- 172.21.15.181:0/3549918853 >> v1:172.21.15.190:6789/0 conn(0x7fe70c0ce5e0 legacy=0x7fe70c0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe710b0b700 1 -- 172.21.15.181:0/3549918853 >> v1:172.21.15.181:6789/0 conn(0x7fe70c0cf890 legacy=0x7fe70c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe70b7fe700 1 -- 172.21.15.181:0/3549918853 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302380004 0 0) 0x7fe70c0d00a0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe710b0b700 1 -- 172.21.15.181:0/3549918853 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c0d0380 con 0x7fe70c15d740 2023-10-16T11:17:23.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe710b0b700 1 --2- 172.21.15.181:0/3549918853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c15d740 0x7fe70c161f70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe700009840 tx=0x7fe700009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84260393a6d3d63b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:23.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe70b7fe700 1 -- 172.21.15.181:0/3549918853 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700011830 con 0x7fe70c15d740 2023-10-16T11:17:23.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.031+0000 7fe70b7fe700 1 -- 172.21.15.181:0/3549918853 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700011990 con 0x7fe70c15d740 2023-10-16T11:17:23.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe70b7fe700 1 -- 172.21.15.181:0/3549918853 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700011d20 con 0x7fe70c15d740 2023-10-16T11:17:23.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe712d6f700 1 -- 172.21.15.181:0/3549918853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c15d740 msgr2=0x7fe70c161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe712d6f700 1 --2- 172.21.15.181:0/3549918853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c15d740 0x7fe70c161f70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe700009840 tx=0x7fe700009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:23.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe712d6f700 1 -- 172.21.15.181:0/3549918853 shutdown_connections 2023-10-16T11:17:23.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe712d6f700 1 --2- 172.21.15.181:0/3549918853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c15d740 0x7fe70c161f70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe712d6f700 1 -- 172.21.15.181:0/3549918853 >> 172.21.15.181:0/3549918853 conn(0x7fe70c09a930 msgr2=0x7fe70c0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:23.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe712d6f700 1 -- 172.21.15.181:0/3549918853 shutdown_connections 2023-10-16T11:17:23.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.032+0000 7fe712d6f700 1 -- 172.21.15.181:0/3549918853 wait complete. 2023-10-16T11:17:23.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.033+0000 7fe712d6f700 1 Processor -- start 2023-10-16T11:17:23.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.034+0000 7fe712d6f700 1 -- start start 2023-10-16T11:17:23.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.034+0000 7fe712d6f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c0ce5e0 0x7fe70c0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.034+0000 7fe712d6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0cf890 0x7fe70c0c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.034+0000 7fe712d6f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe70c15d740 0x7fe70c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.034+0000 7fe712d6f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0c8160 con 0x7fe70c0cf890 2023-10-16T11:17:23.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.034+0000 7fe712d6f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0ccaf0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.034+0000 7fe712d6f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0cbf50 con 0x7fe70c15d740 2023-10-16T11:17:23.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe70bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0cf890 0x7fe70c0c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe710b0b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c0ce5e0 0x7fe70c0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe70bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0cf890 0x7fe70c0c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53326/0 (socket says 172.21.15.181:53326) 2023-10-16T11:17:23.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe70bfff700 1 -- 172.21.15.181:0/2525606465 learned_addr learned my addr 172.21.15.181:0/2525606465 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:23.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe71130c700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe70c15d740 0x7fe70c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe710b0b700 1 -- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe70c15d740 msgr2=0x7fe70c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe710b0b700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe70c15d740 0x7fe70c0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe710b0b700 1 -- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0cf890 msgr2=0x7fe70c0c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe710b0b700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0cf890 0x7fe70c0c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe710b0b700 1 -- 172.21.15.181:0/2525606465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000094a0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe70bfff700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0cf890 0x7fe70c0c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:23.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe710b0b700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c0ce5e0 0x7fe70c0c7c20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe700016000 tx=0x7fe700004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:23.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe7097fa700 1 -- 172.21.15.181:0/2525606465 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70001a9f0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe7097fa700 1 -- 172.21.15.181:0/2525606465 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70001ab70 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c0c8c60 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe7097fa700 1 -- 172.21.15.181:0/2525606465 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7000243f0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.035+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c0cc4a0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.036+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c0cbc60 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.037+0000 7fe7097fa700 1 -- 172.21.15.181:0/2525606465 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe70002d450 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.037+0000 7fe7097fa700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 0x7fe6f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.037+0000 7fe7097fa700 1 -- 172.21.15.181:0/2525606465 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fe70008aeb0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.037+0000 7fe70bfff700 1 -- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 msgr2=0x7fe6f006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.037+0000 7fe70bfff700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 0x7fe6f006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:23.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.042+0000 7fe7097fa700 1 -- 172.21.15.181:0/2525606465 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe700055b00 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.227 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:23.225+0000 7f28b07ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:17:23.231 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:23.230+0000 7f53c5306200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:17:23.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.237+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe70c1605f0 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.238+0000 7fe70bfff700 1 -- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 msgr2=0x7fe6f006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:23.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.238+0000 7fe7097fa700 1 -- 172.21.15.181:0/2525606465 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v185) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe700059120 con 0x7fe70c0ce5e0 2023-10-16T11:17:23.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.238+0000 7fe70bfff700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 0x7fe6f006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:23.240 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:17:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 msgr2=0x7fe6f006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 0x7fe6f006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c0ce5e0 msgr2=0x7fe70c0c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c0ce5e0 0x7fe70c0c7c20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe700016000 tx=0x7fe700004750 comp rx=0 tx=0).stop 2023-10-16T11:17:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 shutdown_connections 2023-10-16T11:17:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fe6f006b020 0x7fe6f006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe70c15d740 0x7fe70c0c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe70c0ce5e0 0x7fe70c0c7c20 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 --2- 172.21.15.181:0/2525606465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0cf890 0x7fe70c0c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 >> 172.21.15.181:0/2525606465 conn(0x7fe70c09a930 msgr2=0x7fe70c15fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 shutdown_connections 2023-10-16T11:17:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.240+0000 7fe712d6f700 1 -- 172.21.15.181:0/2525606465 wait complete. 2023-10-16T11:17:23.254 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:17:23.381 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:23.380+0000 7f53c5306200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:17:23.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.380+0000 7f3b4f59e700 1 Processor -- start 2023-10-16T11:17:23.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.380+0000 7f3b4f59e700 1 -- start start 2023-10-16T11:17:23.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.381+0000 7f3b4f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b500d3820 0x7f3b500d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.381+0000 7f3b4f59e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b500d1e70 0x7f3b5015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.381+0000 7f3b4f59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5006cea0 con 0x7f3b5015d980 2023-10-16T11:17:23.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.381+0000 7f3b4f59e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b500d4140 con 0x7f3b500d1e70 2023-10-16T11:17:23.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.381+0000 7f3b4f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5006c290 con 0x7f3b500d3820 2023-10-16T11:17:23.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4dd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3b5015d980 0x7f3b5006d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36182/0 (socket says 172.21.15.181:36182) 2023-10-16T11:17:23.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4dd9b700 1 -- 172.21.15.181:0/2363075784 learned_addr learned my addr 172.21.15.181:0/2363075784 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:23.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4ed9d700 1 --2- 172.21.15.181:0/2363075784 >> v2:172.21.15.190:3300/0 conn(0x7f3b500d1e70 0x7f3b5015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4e59c700 1 --2- 172.21.15.181:0/2363075784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b500d3820 0x7f3b500d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4d59a700 1 -- 172.21.15.181:0/2363075784 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731941390 0 0) 0x7f3b5006cea0 con 0x7f3b5015d980 2023-10-16T11:17:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4d59a700 1 -- 172.21.15.181:0/2363075784 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b380035b0 con 0x7f3b5015d980 2023-10-16T11:17:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4e59c700 1 -- 172.21.15.181:0/2363075784 >> v2:172.21.15.190:3300/0 conn(0x7f3b500d1e70 msgr2=0x7f3b5015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4e59c700 1 --2- 172.21.15.181:0/2363075784 >> v2:172.21.15.190:3300/0 conn(0x7f3b500d1e70 0x7f3b5015eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4e59c700 1 -- 172.21.15.181:0/2363075784 >> v1:172.21.15.181:6789/0 conn(0x7f3b5015d980 legacy=0x7f3b5006d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4e59c700 1 -- 172.21.15.181:0/2363075784 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5006c570 con 0x7f3b500d3820 2023-10-16T11:17:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4ed9d700 1 --2- 172.21.15.181:0/2363075784 >> v2:172.21.15.190:3300/0 conn(0x7f3b500d1e70 0x7f3b5015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4e59c700 1 --2- 172.21.15.181:0/2363075784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b500d3820 0x7f3b500d3c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009840 tx=0x7f3b40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c85ee0f27c615d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4d59a700 1 -- 172.21.15.181:0/2363075784 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40011830 con 0x7f3b500d3820 2023-10-16T11:17:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4d59a700 1 -- 172.21.15.181:0/2363075784 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b40011990 con 0x7f3b500d3820 2023-10-16T11:17:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.382+0000 7f3b4d59a700 1 -- 172.21.15.181:0/2363075784 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40011d20 con 0x7f3b500d3820 2023-10-16T11:17:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 -- 172.21.15.181:0/2363075784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b500d3820 msgr2=0x7f3b500d3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/2363075784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b500d3820 0x7f3b500d3c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009840 tx=0x7f3b40009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 -- 172.21.15.181:0/2363075784 shutdown_connections 2023-10-16T11:17:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/2363075784 >> v2:172.21.15.190:3300/0 conn(0x7f3b500d1e70 0x7f3b5015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/2363075784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b500d3820 0x7f3b500d3c00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 -- 172.21.15.181:0/2363075784 >> 172.21.15.181:0/2363075784 conn(0x7f3b5008af90 msgr2=0x7f3b5009cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 -- 172.21.15.181:0/2363075784 shutdown_connections 2023-10-16T11:17:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.383+0000 7f3b4f59e700 1 -- 172.21.15.181:0/2363075784 wait complete. 2023-10-16T11:17:23.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.384+0000 7f3b4f59e700 1 Processor -- start 2023-10-16T11:17:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.384+0000 7f3b4f59e700 1 -- start start 2023-10-16T11:17:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b500d1e70 0x7f3b5015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b500d3820 0x7f3b501f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4f59e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b5015d980 0x7f3b500b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500b68b0 con 0x7f3b500d1e70 2023-10-16T11:17:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500b8a20 con 0x7f3b500d3820 2023-10-16T11:17:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4f59e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500b8e80 con 0x7f3b5015d980 2023-10-16T11:17:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b500d1e70 0x7f3b5015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b500d1e70 0x7f3b5015ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53340/0 (socket says 172.21.15.181:53340) 2023-10-16T11:17:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4e59c700 1 -- 172.21.15.181:0/3519670299 learned_addr learned my addr 172.21.15.181:0/3519670299 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4ed9d700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b5015d980 0x7f3b500b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.385+0000 7f3b4dd9b700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b500d3820 0x7f3b501f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4e59c700 1 -- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b5015d980 msgr2=0x7f3b500b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4e59c700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b5015d980 0x7f3b500b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4e59c700 1 -- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b500d3820 msgr2=0x7f3b501f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4e59c700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b500d3820 0x7f3b501f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4e59c700 1 -- 172.21.15.181:0/3519670299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b48009a70 con 0x7f3b500d1e70 2023-10-16T11:17:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4e59c700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b500d1e70 0x7f3b5015ac70 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f3b400090b0 tx=0x7f3b40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b3effd700 1 -- 172.21.15.181:0/3519670299 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40004550 con 0x7f3b500d1e70 2023-10-16T11:17:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b3effd700 1 -- 172.21.15.181:0/3519670299 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4001f070 con 0x7f3b500d1e70 2023-10-16T11:17:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b400094a0 con 0x7f3b500d1e70 2023-10-16T11:17:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b3effd700 1 -- 172.21.15.181:0/3519670299 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b400038e0 con 0x7f3b500d1e70 2023-10-16T11:17:23.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.386+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b500b64d0 con 0x7f3b500d1e70 2023-10-16T11:17:23.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.387+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5006bee0 con 0x7f3b500d1e70 2023-10-16T11:17:23.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.390+0000 7f3b3effd700 1 -- 172.21.15.181:0/3519670299 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f3b40003a40 con 0x7f3b500d1e70 2023-10-16T11:17:23.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.390+0000 7f3b3effd700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 0x7f3b3006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.390+0000 7f3b4dd9b700 1 -- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 msgr2=0x7f3b3006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:23.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.390+0000 7f3b3effd700 1 -- 172.21.15.181:0/3519670299 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f3b40021070 con 0x7f3b500d1e70 2023-10-16T11:17:23.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.390+0000 7f3b4dd9b700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 0x7f3b3006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:23.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.393+0000 7f3b3effd700 1 -- 172.21.15.181:0/3519670299 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b40056090 con 0x7f3b500d1e70 2023-10-16T11:17:23.467 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:23.466+0000 7f28b07ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:17:23.496 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:23.495+0000 7f53c5306200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:17:23.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.591+0000 7f3b4dd9b700 1 -- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 msgr2=0x7f3b3006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:23.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.591+0000 7f3b4dd9b700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 0x7f3b3006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:23.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.591+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b5004f400 con 0x7f3b500d1e70 2023-10-16T11:17:23.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.592+0000 7f3b3effd700 1 -- 172.21.15.181:0/3519670299 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v185) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b400150a0 con 0x7f3b500d1e70 2023-10-16T11:17:23.594 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:17:23.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 msgr2=0x7f3b3006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:23.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 0x7f3b3006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b500d1e70 msgr2=0x7f3b5015ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b500d1e70 0x7f3b5015ac70 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f3b400090b0 tx=0x7f3b40004750 comp rx=0 tx=0).stop 2023-10-16T11:17:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 shutdown_connections 2023-10-16T11:17:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f3b3006b020 0x7f3b3006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b5015d980 0x7f3b500b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b500d1e70 0x7f3b5015ac70 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 --2- 172.21.15.181:0/3519670299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b500d3820 0x7f3b501f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 >> 172.21.15.181:0/3519670299 conn(0x7f3b5008af90 msgr2=0x7f3b500b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 shutdown_connections 2023-10-16T11:17:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.594+0000 7f3b4f59e700 1 -- 172.21.15.181:0/3519670299 wait complete. 2023-10-16T11:17:23.609 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:17:23.624 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:23.623+0000 7f28b07ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:17:23.652 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:23.651+0000 7f53c5306200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:17:23.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.734+0000 7f1988f85700 1 Processor -- start 2023-10-16T11:17:23.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.734+0000 7f1988f85700 1 -- start start 2023-10-16T11:17:23.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.735+0000 7f1988f85700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198415b990 0x7f19840ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.735+0000 7f1988f85700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f19840cd2d0 0x7f19840d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f1988f85700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19840cc510 con 0x7f19840cfc60 2023-10-16T11:17:23.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f1988f85700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19840d24f0 con 0x7f198415b990 2023-10-16T11:17:23.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f1988f85700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19840cc7f0 con 0x7f19840cd2d0 2023-10-16T11:17:23.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198415b990 0x7f19840ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198415b990 0x7f19840ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53350/0 (socket says 172.21.15.181:53350) 2023-10-16T11:17:23.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 -- 172.21.15.181:0/3763558014 learned_addr learned my addr 172.21.15.181:0/3763558014 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:23.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f1981d9b700 1 --2- 172.21.15.181:0/3763558014 >> v2:172.21.15.190:3300/0 conn(0x7f19840cd2d0 0x7f19840d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 -- 172.21.15.181:0/3763558014 >> v2:172.21.15.190:3300/0 conn(0x7f19840cd2d0 msgr2=0x7f19840d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:23.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 --2- 172.21.15.181:0/3763558014 >> v2:172.21.15.190:3300/0 conn(0x7f19840cd2d0 0x7f19840d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 -- 172.21.15.181:0/3763558014 >> v1:172.21.15.190:6789/0 conn(0x7f19840cfc60 legacy=0x7f19840ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 -- 172.21.15.181:0/3763558014 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19840ccad0 con 0x7f198415b990 2023-10-16T11:17:23.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198259c700 1 --2- 172.21.15.181:0/3763558014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f19840ce8f0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f19840b4230 tx=0x7f1974009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f398eddbabfaa53 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:23.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198159a700 1 -- 172.21.15.181:0/3763558014 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974005e30 con 0x7f198415b990 2023-10-16T11:17:23.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198159a700 1 -- 172.21.15.181:0/3763558014 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974004030 con 0x7f198415b990 2023-10-16T11:17:23.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.736+0000 7f198159a700 1 -- 172.21.15.181:0/3763558014 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740043e0 con 0x7f198415b990 2023-10-16T11:17:23.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 -- 172.21.15.181:0/3763558014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 msgr2=0x7f19840ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 --2- 172.21.15.181:0/3763558014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f19840ce8f0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f19840b4230 tx=0x7f1974009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 -- 172.21.15.181:0/3763558014 shutdown_connections 2023-10-16T11:17:23.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 --2- 172.21.15.181:0/3763558014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f19840ce8f0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 --2- 172.21.15.181:0/3763558014 >> v2:172.21.15.190:3300/0 conn(0x7f19840cd2d0 0x7f19840d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 -- 172.21.15.181:0/3763558014 >> 172.21.15.181:0/3763558014 conn(0x7f198409a9a0 msgr2=0x7f19840d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:23.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 -- 172.21.15.181:0/3763558014 shutdown_connections 2023-10-16T11:17:23.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.737+0000 7f1988f85700 1 -- 172.21.15.181:0/3763558014 wait complete. 2023-10-16T11:17:23.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.739+0000 7f1988f85700 1 Processor -- start 2023-10-16T11:17:23.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.739+0000 7f1988f85700 1 -- start start 2023-10-16T11:17:23.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1988f85700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19840cd2d0 0x7f19840c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1988f85700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19840cfc60 0x7f19840b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1988f85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f1984155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1988f85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840b5690 con 0x7f198415b990 2023-10-16T11:17:23.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1988f85700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840b47e0 con 0x7f19840cfc60 2023-10-16T11:17:23.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1988f85700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19840b4a80 con 0x7f19840cd2d0 2023-10-16T11:17:23.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f1984155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f1984155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53360/0 (socket says 172.21.15.181:53360) 2023-10-16T11:17:23.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 -- 172.21.15.181:0/775044857 learned_addr learned my addr 172.21.15.181:0/775044857 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:23.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f198259c700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19840cd2d0 0x7f19840c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1981d9b700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19840cfc60 0x7f19840b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:23.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 -- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19840cd2d0 msgr2=0x7f19840c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19840cd2d0 0x7f19840c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 -- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19840cfc60 msgr2=0x7f19840b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19840cfc60 0x7f19840b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 -- 172.21.15.181:0/775044857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974009500 con 0x7f198415b990 2023-10-16T11:17:23.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1982d9d700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f1984155af0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f197c00cb50 tx=0x7f197c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:23.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.741+0000 7f1972ffd700 1 -- 172.21.15.181:0/775044857 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c003c00 con 0x7f198415b990 2023-10-16T11:17:23.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.742+0000 7f1972ffd700 1 -- 172.21.15.181:0/775044857 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c003d60 con 0x7f198415b990 2023-10-16T11:17:23.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.742+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19840b4d60 con 0x7f198415b990 2023-10-16T11:17:23.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.742+0000 7f1972ffd700 1 -- 172.21.15.181:0/775044857 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c00aa20 con 0x7f198415b990 2023-10-16T11:17:23.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.742+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19840b5320 con 0x7f198415b990 2023-10-16T11:17:23.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.742+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19840b5800 con 0x7f198415b990 2023-10-16T11:17:23.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.743+0000 7f1972ffd700 1 -- 172.21.15.181:0/775044857 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f197c00ab80 con 0x7f198415b990 2023-10-16T11:17:23.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.743+0000 7f1972ffd700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 0x7f196406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:23.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.743+0000 7f198259c700 1 -- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 msgr2=0x7f196406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:23.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.743+0000 7f1972ffd700 1 -- 172.21.15.181:0/775044857 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f197c089c00 con 0x7f198415b990 2023-10-16T11:17:23.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.743+0000 7f198259c700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 0x7f196406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:23.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.748+0000 7f1972ffd700 1 -- 172.21.15.181:0/775044857 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f19840b5800 con 0x7f198415b990 2023-10-16T11:17:23.753 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:23.751+0000 7f28b07ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:17:23.915 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:23.914+0000 7f28b07ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:17:23.916 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:23.914+0000 7f53c5306200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:23.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.944+0000 7f198259c700 1 -- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 msgr2=0x7f196406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:23.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.944+0000 7f198259c700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 0x7f196406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:23.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.994+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198406b5e0 con 0x7f198415b990 2023-10-16T11:17:23.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.994+0000 7f1972ffd700 1 -- 172.21.15.181:0/775044857 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f197c090080 con 0x7f198415b990 2023-10-16T11:17:23.996 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:23.996 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1903420066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1903420066},{"type":"v1","addr":"172.21.15.181:6836","nonce":1903420066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/586483327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":586483327},{"type":"v1","addr":"172.21.15.190:6835","nonce":586483327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2078148321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2078148321},{"type":"v1","addr":"172.21.15.181:6837","nonce":2078148321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:01:22.987835+0000","modified":"2023-10-16T11:01:26.027862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10197},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10197":{"gid":10197,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2135938520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2135938520},{"type":"v1","addr":"172.21.15.190:6834","nonce":2135938520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:17:23.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 msgr2=0x7f196406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:23.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 0x7f196406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 msgr2=0x7f1984155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:23.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f1984155af0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f197c00cb50 tx=0x7f197c00ce60 comp rx=0 tx=0).stop 2023-10-16T11:17:23.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 shutdown_connections 2023-10-16T11:17:23.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f196406b020 0x7f196406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19840cd2d0 0x7f19840c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:23.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198415b990 0x7f1984155af0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 --2- 172.21.15.181:0/775044857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19840cfc60 0x7f19840b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 >> 172.21.15.181:0/775044857 conn(0x7f198409a9a0 msgr2=0x7f198406f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:24.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 shutdown_connections 2023-10-16T11:17:24.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:23.997+0000 7f1988f85700 1 -- 172.21.15.181:0/775044857 wait complete. 2023-10-16T11:17:24.001 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 94 2023-10-16T11:17:24.012 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:24.076 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:24.075+0000 7f53c5306200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:17:24.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.173+0000 7fbabc580700 1 Processor -- start 2023-10-16T11:17:24.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.173+0000 7fbabc580700 1 -- start start 2023-10-16T11:17:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.174+0000 7fbabc580700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbab40ce7f0 0x7fbab406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.174+0000 7fbabc580700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbab40d2390 0x7fbab40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.174+0000 7fbabc580700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab40cc470 con 0x7fbab40d0180 2023-10-16T11:17:24.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.174+0000 7fbabc580700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab40d05f0 con 0x7fbab40ce7f0 2023-10-16T11:17:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.174+0000 7fbabc580700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab406ca30 con 0x7fbab40d2390 2023-10-16T11:17:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbab40ce7f0 0x7fbab406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbab40ce7f0 0x7fbab406c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53366/0 (socket says 172.21.15.181:53366) 2023-10-16T11:17:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 -- 172.21.15.181:0/633284251 learned_addr learned my addr 172.21.15.181:0/633284251 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbab9b1b700 1 --2- 172.21.15.181:0/633284251 >> v2:172.21.15.190:3300/0 conn(0x7fbab40d2390 0x7fbab40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 -- 172.21.15.181:0/633284251 >> v2:172.21.15.190:3300/0 conn(0x7fbab40d2390 msgr2=0x7fbab40cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:24.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 --2- 172.21.15.181:0/633284251 >> v2:172.21.15.190:3300/0 conn(0x7fbab40d2390 0x7fbab40cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 -- 172.21.15.181:0/633284251 >> v1:172.21.15.190:6789/0 conn(0x7fbab40d0180 legacy=0x7fbab4161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 -- 172.21.15.181:0/633284251 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab406cd10 con 0x7fbab40ce7f0 2023-10-16T11:17:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbaba31c700 1 --2- 172.21.15.181:0/633284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab406c100 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fbab40cc1e0 tx=0x7fbab0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac67fb230bb5958 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbab931a700 1 -- 172.21.15.181:0/633284251 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0005e30 con 0x7fbab40ce7f0 2023-10-16T11:17:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbab931a700 1 -- 172.21.15.181:0/633284251 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab0004030 con 0x7fbab40ce7f0 2023-10-16T11:17:24.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.175+0000 7fbab931a700 1 -- 172.21.15.181:0/633284251 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab00043e0 con 0x7fbab40ce7f0 2023-10-16T11:17:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 -- 172.21.15.181:0/633284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 msgr2=0x7fbab406c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 --2- 172.21.15.181:0/633284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab406c100 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fbab40cc1e0 tx=0x7fbab0009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 -- 172.21.15.181:0/633284251 shutdown_connections 2023-10-16T11:17:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 --2- 172.21.15.181:0/633284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab406c100 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 --2- 172.21.15.181:0/633284251 >> v2:172.21.15.190:3300/0 conn(0x7fbab40d2390 0x7fbab40cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 -- 172.21.15.181:0/633284251 >> 172.21.15.181:0/633284251 conn(0x7fbab409a9a0 msgr2=0x7fbab40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 -- 172.21.15.181:0/633284251 shutdown_connections 2023-10-16T11:17:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.176+0000 7fbabc580700 1 -- 172.21.15.181:0/633284251 wait complete. 2023-10-16T11:17:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.177+0000 7fbabc580700 1 Processor -- start 2023-10-16T11:17:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.177+0000 7fbabc580700 1 -- start start 2023-10-16T11:17:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabc580700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab4158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabc580700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbab40d0180 0x7fbab40c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabc580700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbab40d2390 0x7fbab415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabc580700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40c70c0 con 0x7fbab40ce7f0 2023-10-16T11:17:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabc580700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40c73a0 con 0x7fbab40d0180 2023-10-16T11:17:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabc580700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40c7680 con 0x7fbab40d2390 2023-10-16T11:17:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab4158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab4158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53372/0 (socket says 172.21.15.181:53372) 2023-10-16T11:17:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 -- 172.21.15.181:0/4261173132 learned_addr learned my addr 172.21.15.181:0/4261173132 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:24.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbab9b1b700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbab40d0180 0x7fbab40c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabab1d700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbab40d2390 0x7fbab415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 -- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbab40d2390 msgr2=0x7fbab415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbab40d2390 0x7fbab415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 -- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbab40d0180 msgr2=0x7fbab40c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbab40d0180 0x7fbab40c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 -- 172.21.15.181:0/4261173132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0009500 con 0x7fbab40ce7f0 2023-10-16T11:17:24.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaba31c700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab4158b20 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009c10 tx=0x7fbab0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:24.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaaaffd700 1 -- 172.21.15.181:0/4261173132 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab0003c30 con 0x7fbab40ce7f0 2023-10-16T11:17:24.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaaaffd700 1 -- 172.21.15.181:0/4261173132 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab0003d90 con 0x7fbab40ce7f0 2023-10-16T11:17:24.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbaaaffd700 1 -- 172.21.15.181:0/4261173132 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab001c640 con 0x7fbab40ce7f0 2023-10-16T11:17:24.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.178+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab40c7960 con 0x7fbab40ce7f0 2023-10-16T11:17:24.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.179+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab40c6c90 con 0x7fbab40ce7f0 2023-10-16T11:17:24.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.180+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab40c7e80 con 0x7fbab40ce7f0 2023-10-16T11:17:24.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.180+0000 7fbaaaffd700 1 -- 172.21.15.181:0/4261173132 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fbab001c7a0 con 0x7fbab40ce7f0 2023-10-16T11:17:24.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.181+0000 7fbaaaffd700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 0x7fba9806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.181+0000 7fbab9b1b700 1 -- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 msgr2=0x7fba9806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:24.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.181+0000 7fbaaaffd700 1 -- 172.21.15.181:0/4261173132 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fbab008ab40 con 0x7fbab40ce7f0 2023-10-16T11:17:24.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.181+0000 7fbab9b1b700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 0x7fba9806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:24.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.185+0000 7fbaaaffd700 1 -- 172.21.15.181:0/4261173132 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbab40c7e80 con 0x7fbab40ce7f0 2023-10-16T11:17:24.193 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:24.192+0000 7f28b07ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:24.373 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:24.372+0000 7f28b07ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:17:24.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.382+0000 7fbab9b1b700 1 -- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 msgr2=0x7fba9806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.382+0000 7fbab9b1b700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 0x7fba9806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.401+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab40c7e80 con 0x7fbab40ce7f0 2023-10-16T11:17:24.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.402+0000 7fbaaaffd700 1 -- 172.21.15.181:0/4261173132 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+18545 (secure 0 0 0) 0x7fbab40c7e80 con 0x7fbab40ce7f0 2023-10-16T11:17:24.403 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:24.404 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":185,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:20.424880+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:01:21.420382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:01:21.958383+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":185,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/4003028162":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/3290548668":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6833/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6832/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/2238590139":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/231495737":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/1499024696":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/458160361":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1195985489":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/4000110033":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/990187329":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3304094772":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6833/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1162194396":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6832/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3436855298":"2023-10-17T11:01:37.009118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":15,"length":2},{"begin":31,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.404+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 msgr2=0x7fba9806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.404+0000 7fbabc580700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 0x7fba9806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.404+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 msgr2=0x7fbab4158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.404+0000 7fbabc580700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab4158b20 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009c10 tx=0x7fbab0004750 comp rx=0 tx=0).stop 2023-10-16T11:17:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 shutdown_connections 2023-10-16T11:17:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fba9806b020 0x7fba9806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbab40d2390 0x7fbab415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbab40ce7f0 0x7fbab4158b20 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 --2- 172.21.15.181:0/4261173132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbab40d0180 0x7fbab40c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 >> 172.21.15.181:0/4261173132 conn(0x7fbab409a9a0 msgr2=0x7fbab40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 shutdown_connections 2023-10-16T11:17:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.405+0000 7fbabc580700 1 -- 172.21.15.181:0/4261173132 wait complete. 2023-10-16T11:17:24.418 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:17:24.419 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:17:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.550+0000 7f9916ed8700 1 Processor -- start 2023-10-16T11:17:24.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.550+0000 7f9916ed8700 1 -- start start 2023-10-16T11:17:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9916ed8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f991015dc20 0x7f99100d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9916ed8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9916ed8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99100ce4b0 0x7f99100ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9916ed8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99100b3b80 con 0x7f99100ce4b0 2023-10-16T11:17:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9916ed8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99100d2820 con 0x7f991015dc20 2023-10-16T11:17:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9916ed8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100b2f00 con 0x7f99100d1e60 2023-10-16T11:17:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99100ce4b0 0x7f99100ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99100ce4b0 0x7f99100ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53386/0 (socket says 172.21.15.181:53386) 2023-10-16T11:17:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 -- 172.21.15.181:0/629314044 learned_addr learned my addr 172.21.15.181:0/629314044 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f990ffff700 1 --2- 172.21.15.181:0/629314044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9914c74700 1 --2- 172.21.15.181:0/629314044 >> v2:172.21.15.190:3300/0 conn(0x7f991015dc20 0x7f99100d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 -- 172.21.15.181:0/629314044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 msgr2=0x7f99100d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 --2- 172.21.15.181:0/629314044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 -- 172.21.15.181:0/629314044 >> v2:172.21.15.190:3300/0 conn(0x7f991015dc20 msgr2=0x7f99100d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:24.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 --2- 172.21.15.181:0/629314044 >> v2:172.21.15.190:3300/0 conn(0x7f991015dc20 0x7f99100d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 -- 172.21.15.181:0/629314044 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100b31e0 con 0x7f99100ce4b0 2023-10-16T11:17:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f9915475700 1 --2- 172.21.15.181:0/629314044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99100ce4b0 0x7f99100ce940 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f991006b6e0 tx=0x7f990800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10da1262769c9c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.551+0000 7f990f7fe700 1 -- 172.21.15.181:0/629314044 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800e070 con 0x7f99100ce4b0 2023-10-16T11:17:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f990f7fe700 1 -- 172.21.15.181:0/629314044 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800b910 con 0x7f99100ce4b0 2023-10-16T11:17:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f990f7fe700 1 -- 172.21.15.181:0/629314044 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800bcc0 con 0x7f99100ce4b0 2023-10-16T11:17:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 -- 172.21.15.181:0/629314044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99100ce4b0 msgr2=0x7f99100ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 --2- 172.21.15.181:0/629314044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99100ce4b0 0x7f99100ce940 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f991006b6e0 tx=0x7f990800b330 comp rx=0 tx=0).stop 2023-10-16T11:17:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 -- 172.21.15.181:0/629314044 shutdown_connections 2023-10-16T11:17:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 --2- 172.21.15.181:0/629314044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99100ce4b0 0x7f99100ce940 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 --2- 172.21.15.181:0/629314044 >> v2:172.21.15.190:3300/0 conn(0x7f991015dc20 0x7f99100d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 --2- 172.21.15.181:0/629314044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 -- 172.21.15.181:0/629314044 >> 172.21.15.181:0/629314044 conn(0x7f991008af90 msgr2=0x7f99100d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 -- 172.21.15.181:0/629314044 shutdown_connections 2023-10-16T11:17:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.552+0000 7f9916ed8700 1 -- 172.21.15.181:0/629314044 wait complete. 2023-10-16T11:17:24.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.553+0000 7f9916ed8700 1 Processor -- start 2023-10-16T11:17:24.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.553+0000 7f9916ed8700 1 -- start start 2023-10-16T11:17:24.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9916ed8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99100ce4b0 0x7f99101f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9916ed8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9916ed8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991015dc20 0x7f99101600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9916ed8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991015eab0 con 0x7f991015dc20 2023-10-16T11:17:24.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9916ed8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991015ed90 con 0x7f99100ce4b0 2023-10-16T11:17:24.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9916ed8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991015f070 con 0x7f99100d1e60 2023-10-16T11:17:24.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9915475700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991015dc20 0x7f99101600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f990ffff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9914c74700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99100ce4b0 0x7f99101f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9915475700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991015dc20 0x7f99101600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53392/0 (socket says 172.21.15.181:53392) 2023-10-16T11:17:24.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.554+0000 7f9915475700 1 -- 172.21.15.181:0/489237546 learned_addr learned my addr 172.21.15.181:0/489237546 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:24.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9915475700 1 -- 172.21.15.181:0/489237546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 msgr2=0x7f99100a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9915475700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9915475700 1 -- 172.21.15.181:0/489237546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99100ce4b0 msgr2=0x7f99101f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9915475700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99100ce4b0 0x7f99101f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9915475700 1 -- 172.21.15.181:0/489237546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908009b30 con 0x7f991015dc20 2023-10-16T11:17:24.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9915475700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991015dc20 0x7f99101600a0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f991008d0d0 tx=0x7f9908004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:24.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f990d7fa700 1 -- 172.21.15.181:0/489237546 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99080099b0 con 0x7f991015dc20 2023-10-16T11:17:24.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f990d7fa700 1 -- 172.21.15.181:0/489237546 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800ce80 con 0x7f991015dc20 2023-10-16T11:17:24.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991015f350 con 0x7f991015dc20 2023-10-16T11:17:24.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f990d7fa700 1 -- 172.21.15.181:0/489237546 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908004490 con 0x7f991015dc20 2023-10-16T11:17:24.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.555+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99101f2d20 con 0x7f991015dc20 2023-10-16T11:17:24.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.556+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991015f810 con 0x7f991015dc20 2023-10-16T11:17:24.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.556+0000 7f990d7fa700 1 -- 172.21.15.181:0/489237546 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f99080045f0 con 0x7f991015dc20 2023-10-16T11:17:24.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.557+0000 7f990d7fa700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 0x7f98f406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.557+0000 7f9914c74700 1 -- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 msgr2=0x7f98f406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:24.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.557+0000 7f990d7fa700 1 -- 172.21.15.181:0/489237546 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6965+0+0 (secure 0 0 0) 0x7f990808ad60 con 0x7f991015dc20 2023-10-16T11:17:24.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.557+0000 7f9914c74700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 0x7f98f406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:24.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.565+0000 7f990d7fa700 1 -- 172.21.15.181:0/489237546 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f99080557c0 con 0x7f991015dc20 2023-10-16T11:17:24.606 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:24.605+0000 7f53c5306200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:17:24.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.758+0000 7f9914c74700 1 -- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 msgr2=0x7f98f406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:24.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.758+0000 7f9914c74700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 0x7f98f406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:24.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.812+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99101f2eb0 con 0x7f991015dc20 2023-10-16T11:17:24.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.813+0000 7f990d7fa700 1 -- 172.21.15.181:0/489237546 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f990801f090 con 0x7f991015dc20 2023-10-16T11:17:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:24.814 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1903420066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1903420066},{"type":"v1","addr":"172.21.15.181:6836","nonce":1903420066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/586483327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":586483327},{"type":"v1","addr":"172.21.15.190:6835","nonce":586483327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2078148321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2078148321},{"type":"v1","addr":"172.21.15.181:6837","nonce":2078148321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:01:22.987835+0000","modified":"2023-10-16T11:01:26.027862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10197},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10197":{"gid":10197,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2135938520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2135938520},{"type":"v1","addr":"172.21.15.190:6834","nonce":2135938520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:17:24.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 msgr2=0x7f98f406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:24.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 0x7f98f406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991015dc20 msgr2=0x7f99101600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991015dc20 0x7f99101600a0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f991008d0d0 tx=0x7f9908004ad0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 shutdown_connections 2023-10-16T11:17:24.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991015dc20 0x7f99101600a0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99100ce4b0 0x7f99101f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7f98f406b020 0x7f98f406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 --2- 172.21.15.181:0/489237546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99100d1e60 0x7f99100a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 >> 172.21.15.181:0/489237546 conn(0x7f991008af90 msgr2=0x7f99100ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 shutdown_connections 2023-10-16T11:17:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.815+0000 7f9916ed8700 1 -- 172.21.15.181:0/489237546 wait complete. 2023-10-16T11:17:24.819 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 94 2023-10-16T11:17:24.833 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:17:24.951 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:24.950+0000 7f28b07ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:17:24.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.954+0000 7fd7f8b0c700 1 Processor -- start 2023-10-16T11:17:24.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.954+0000 7fd7f8b0c700 1 -- start start 2023-10-16T11:17:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.955+0000 7fd7f8b0c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40d2390 0x7fd7f40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.955+0000 7fd7f8b0c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7f40d0180 0x7fd7f40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.955+0000 7fd7f8b0c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f40cc470 con 0x7fd7f40ce7f0 2023-10-16T11:17:24.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.955+0000 7fd7f8b0c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f40d0610 con 0x7fd7f40d0180 2023-10-16T11:17:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.955+0000 7fd7f8b0c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f406c810 con 0x7fd7f40d2390 2023-10-16T11:17:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd7f40ce7f0 0x7fd7f406c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36244/0 (socket says 172.21.15.181:36244) 2023-10-16T11:17:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f259c700 1 -- 172.21.15.181:0/3693261093 learned_addr learned my addr 172.21.15.181:0/3693261093 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f2d9d700 1 --2- 172.21.15.181:0/3693261093 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40d0180 0x7fd7f40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f1d9b700 1 --2- 172.21.15.181:0/3693261093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40d2390 0x7fd7f40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f159a700 1 -- 172.21.15.181:0/3693261093 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717290613 0 0) 0x7fd7f40cc470 con 0x7fd7f40ce7f0 2023-10-16T11:17:24.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f159a700 1 -- 172.21.15.181:0/3693261093 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e00035b0 con 0x7fd7f40ce7f0 2023-10-16T11:17:24.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f1d9b700 1 -- 172.21.15.181:0/3693261093 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40d0180 msgr2=0x7fd7f40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:24.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f1d9b700 1 --2- 172.21.15.181:0/3693261093 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40d0180 0x7fd7f40cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f1d9b700 1 -- 172.21.15.181:0/3693261093 >> v1:172.21.15.181:6789/0 conn(0x7fd7f40ce7f0 legacy=0x7fd7f406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f1d9b700 1 -- 172.21.15.181:0/3693261093 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f406ca30 con 0x7fd7f40d2390 2023-10-16T11:17:24.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f2d9d700 1 --2- 172.21.15.181:0/3693261093 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40d0180 0x7fd7f40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:24.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f1d9b700 1 --2- 172.21.15.181:0/3693261093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40d2390 0x7fd7f40cfc40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009840 tx=0x7fd7dc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce834b7c6197ebcf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:24.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f159a700 1 -- 172.21.15.181:0/3693261093 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc011830 con 0x7fd7f40d2390 2023-10-16T11:17:24.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f159a700 1 -- 172.21.15.181:0/3693261093 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc011990 con 0x7fd7f40d2390 2023-10-16T11:17:24.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.956+0000 7fd7f159a700 1 -- 172.21.15.181:0/3693261093 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc011d20 con 0x7fd7f40d2390 2023-10-16T11:17:24.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3693261093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40d2390 msgr2=0x7fd7f40cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3693261093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40d2390 0x7fd7f40cfc40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009840 tx=0x7fd7dc009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:24.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3693261093 shutdown_connections 2023-10-16T11:17:24.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3693261093 >> v2:172.21.15.190:3300/0 conn(0x7fd7f40d0180 0x7fd7f40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3693261093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40d2390 0x7fd7f40cfc40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3693261093 >> 172.21.15.181:0/3693261093 conn(0x7fd7f409a9a0 msgr2=0x7fd7f40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:24.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3693261093 shutdown_connections 2023-10-16T11:17:24.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.957+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3693261093 wait complete. 2023-10-16T11:17:24.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.958+0000 7fd7f8b0c700 1 Processor -- start 2023-10-16T11:17:24.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.958+0000 7fd7f8b0c700 1 -- start start 2023-10-16T11:17:24.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f8b0c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40ce7f0 0x7fd7f4156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f8b0c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40d0180 0x7fd7f4159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f8b0c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f40d2390 0x7fd7f40b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f8b0c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f415b3d0 con 0x7fd7f40d0180 2023-10-16T11:17:24.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f8b0c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f415b6b0 con 0x7fd7f40d2390 2023-10-16T11:17:24.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f8b0c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f415b990 con 0x7fd7f40ce7f0 2023-10-16T11:17:24.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f1d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40d0180 0x7fd7f4159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f1d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40d0180 0x7fd7f4159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53406/0 (socket says 172.21.15.181:53406) 2023-10-16T11:17:24.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f1d9b700 1 -- 172.21.15.181:0/3420970753 learned_addr learned my addr 172.21.15.181:0/3420970753 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:24.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f259c700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40ce7f0 0x7fd7f4156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f2d9d700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f40d2390 0x7fd7f40b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.959+0000 7fd7f1d9b700 1 -- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40ce7f0 msgr2=0x7fd7f4156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7f1d9b700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40ce7f0 0x7fd7f4156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7f1d9b700 1 -- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f40d2390 msgr2=0x7fd7f40b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7f1d9b700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f40d2390 0x7fd7f40b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:24.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7f1d9b700 1 -- 172.21.15.181:0/3420970753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec009a70 con 0x7fd7f40d0180 2023-10-16T11:17:24.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7f1d9b700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40d0180 0x7fd7f4159ba0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0090b0 tx=0x7fd7dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:24.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0036a0 con 0x7fd7f40d0180 2023-10-16T11:17:24.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc01f070 con 0x7fd7f40d0180 2023-10-16T11:17:24.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc003a60 con 0x7fd7f40d0180 2023-10-16T11:17:24.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc0094a0 con 0x7fd7f40d0180 2023-10-16T11:17:24.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.960+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f415bf50 con 0x7fd7f40d0180 2023-10-16T11:17:24.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.961+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f4067d20 con 0x7fd7f40d0180 2023-10-16T11:17:24.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.961+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 123) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd7dc003bc0 con 0x7fd7f40d0180 2023-10-16T11:17:24.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.962+0000 7fd7eaffd700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd7d406b020 0x7fd7d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:24.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.962+0000 7fd7f259c700 1 -- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd7d406b020 msgr2=0x7fd7d406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2958900944 2023-10-16T11:17:24.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.962+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6965+0+0 (secure 0 0 0) 0x7fd7dc021070 con 0x7fd7f40d0180 2023-10-16T11:17:24.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.962+0000 7fd7f259c700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd7d406b020 0x7fd7d406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:24.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.966+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd7dc055f30 con 0x7fd7f40d0180 2023-10-16T11:17:24.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.974+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 124) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd7dc056300 con 0x7fd7f40d0180 2023-10-16T11:17:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.974+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd7d406b020 msgr2=0x7fd7d406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:24.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:24.974+0000 7fd7eaffd700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd7d406b020 0x7fd7d406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.038 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f2828fa0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.038 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f2828fa0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.038 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f2828fa0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.038 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f2828fa0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.039 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f2828fa0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.039 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f282bfe6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.040 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f282bfe6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.040 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f282bfe6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.040 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f282bfe6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.040 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:25.037+0000 7f282bfe6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:25.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.220+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7f40c8fe0 con 0x7fd7f40d0180 2023-10-16T11:17:25.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.221+0000 7fd7eaffd700 1 -- 172.21.15.181:0/3420970753 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4930 (secure 0 0 0) 0x7fd7dc05d110 con 0x7fd7f40d0180 2023-10-16T11:17:25.222 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:25.223 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1903420066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1903420066},{"type":"v1","addr":"172.21.15.181:6836","nonce":1903420066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/586483327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":586483327},{"type":"v1","addr":"172.21.15.190:6835","nonce":586483327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2078148321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2078148321},{"type":"v1","addr":"172.21.15.181:6837","nonce":2078148321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:01:22.987835+0000","modified":"2023-10-16T11:01:26.027862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10197},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10197":{"gid":10197,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/2135938520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2135938520},{"type":"v1","addr":"172.21.15.190:6834","nonce":2135938520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-16T11:17:25.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40d0180 msgr2=0x7fd7f4159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40d0180 0x7fd7f4159ba0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0090b0 tx=0x7fd7dc004750 comp rx=0 tx=0).stop 2023-10-16T11:17:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 shutdown_connections 2023-10-16T11:17:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:6832/2958900944,v1:172.21.15.181:6833/2958900944] conn(0x7fd7d406b020 0x7fd7d406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd7f40ce7f0 0x7fd7f4156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7f40d2390 0x7fd7f40b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 --2- 172.21.15.181:0/3420970753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40d0180 0x7fd7f4159ba0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 >> 172.21.15.181:0/3420970753 conn(0x7fd7f409a9a0 msgr2=0x7fd7f40703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 shutdown_connections 2023-10-16T11:17:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.223+0000 7fd7f8b0c700 1 -- 172.21.15.181:0/3420970753 wait complete. 2023-10-16T11:17:25.227 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 94 2023-10-16T11:17:25.236 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:25.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.355+0000 7f8c09a6f700 1 Processor -- start 2023-10-16T11:17:25.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.355+0000 7f8c09a6f700 1 -- start start 2023-10-16T11:17:25.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c09a6f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c04161b60 0x7f8c0406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c09a6f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c040d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c09a6f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040d2560 con 0x7f8c04160d20 2023-10-16T11:17:25.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c09a6f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c04162800 con 0x7f8c04161b60 2023-10-16T11:17:25.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c09a6f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040d43e0 con 0x7f8c04161f40 2023-10-16T11:17:25.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c04161b60 0x7f8c0406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c037fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c04161b60 0x7f8c0406d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53484/0 (socket says 172.21.15.181:53484) 2023-10-16T11:17:25.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c037fe700 1 -- 172.21.15.181:0/1228876868 learned_addr learned my addr 172.21.15.181:0/1228876868 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:25.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c02ffd700 1 --2- 172.21.15.181:0/1228876868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c040d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.356+0000 7f8c037fe700 1 -- 172.21.15.181:0/1228876868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 msgr2=0x7f8c040d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:25.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c037fe700 1 --2- 172.21.15.181:0/1228876868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c040d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c037fe700 1 -- 172.21.15.181:0/1228876868 >> v1:172.21.15.190:6789/0 conn(0x7f8c04160d20 legacy=0x7f8c040b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c037fe700 1 -- 172.21.15.181:0/1228876868 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0406ce20 con 0x7f8c04161b60 2023-10-16T11:17:25.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c037fe700 1 --2- 172.21.15.181:0/1228876868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c0406d5f0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006fd0 tx=0x7f8bf8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb254ab76e40207 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:25.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c027fc700 1 -- 172.21.15.181:0/1228876868 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8005e30 con 0x7f8c04161b60 2023-10-16T11:17:25.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c027fc700 1 -- 172.21.15.181:0/1228876868 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf8004030 con 0x7f8c04161b60 2023-10-16T11:17:25.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c027fc700 1 -- 172.21.15.181:0/1228876868 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80043e0 con 0x7f8c04161b60 2023-10-16T11:17:25.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 -- 172.21.15.181:0/1228876868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 msgr2=0x7f8c0406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 --2- 172.21.15.181:0/1228876868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c0406d5f0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006fd0 tx=0x7f8bf8009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 -- 172.21.15.181:0/1228876868 shutdown_connections 2023-10-16T11:17:25.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 --2- 172.21.15.181:0/1228876868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c0406d5f0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 --2- 172.21.15.181:0/1228876868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c040d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 -- 172.21.15.181:0/1228876868 >> 172.21.15.181:0/1228876868 conn(0x7f8c0409cba0 msgr2=0x7f8c040d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:25.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 -- 172.21.15.181:0/1228876868 shutdown_connections 2023-10-16T11:17:25.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.357+0000 7f8c09a6f700 1 -- 172.21.15.181:0/1228876868 wait complete. 2023-10-16T11:17:25.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.360+0000 7f8c09a6f700 1 Processor -- start 2023-10-16T11:17:25.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.360+0000 7f8c09a6f700 1 -- start start 2023-10-16T11:17:25.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.360+0000 7f8c09a6f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c04160d20 0x7f8c040cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.360+0000 7f8c09a6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c040cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c09a6f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c04152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c09a6f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040cce20 con 0x7f8c04161b60 2023-10-16T11:17:25.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c09a6f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040cd100 con 0x7f8c04160d20 2023-10-16T11:17:25.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c09a6f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040cd3e0 con 0x7f8c04161f40 2023-10-16T11:17:25.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c040cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c040cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53490/0 (socket says 172.21.15.181:53490) 2023-10-16T11:17:25.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 -- 172.21.15.181:0/2635198866 learned_addr learned my addr 172.21.15.181:0/2635198866 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:25.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c037fe700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c04160d20 0x7f8c040cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c03fff700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c04152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 -- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 msgr2=0x7f8c04152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c04152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 -- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c04160d20 msgr2=0x7f8c040cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c04160d20 0x7f8c040cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 -- 172.21.15.181:0/2635198866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf8009500 con 0x7f8c04161b60 2023-10-16T11:17:25.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c02ffd700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c040cc360 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d800 tx=0x7f8bf400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:25.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8bebfff700 1 -- 172.21.15.181:0/2635198866 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40125f0 con 0x7f8c04161b60 2023-10-16T11:17:25.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8bebfff700 1 -- 172.21.15.181:0/2635198866 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4012750 con 0x7f8c04161b60 2023-10-16T11:17:25.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c040cd6c0 con 0x7f8c04161b60 2023-10-16T11:17:25.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8bebfff700 1 -- 172.21.15.181:0/2635198866 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4012b00 con 0x7f8c04161b60 2023-10-16T11:17:25.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.361+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c040ca060 con 0x7f8c04161b60 2023-10-16T11:17:25.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.362+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0404ebb0 con 0x7f8c04161b60 2023-10-16T11:17:25.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.363+0000 7f8bebfff700 1 -- 172.21.15.181:0/2635198866 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 124) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8bf4012c60 con 0x7f8c04161b60 2023-10-16T11:17:25.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.363+0000 7f8bebfff700 1 -- 172.21.15.181:0/2635198866 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f8bf40896a0 con 0x7f8c04161b60 2023-10-16T11:17:25.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.367+0000 7f8bebfff700 1 -- 172.21.15.181:0/2635198866 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8bf4054580 con 0x7f8c04161b60 2023-10-16T11:17:25.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.563+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c0406d2a0 con 0x7f8c04161b60 2023-10-16T11:17:25.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.565+0000 7f8bebfff700 1 -- 172.21.15.181:0/2635198866 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+18959 (secure 0 0 0) 0x7f8bf4057ba0 con 0x7f8c04161b60 2023-10-16T11:17:25.566 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:25.566 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":187,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:24.967050+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:01:21.420382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-16T11:01:21.958383+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":186,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/2958900944":"2023-10-17T11:17:24.967015+0000","172.21.15.181:6832/2958900944":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/2426778736":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/2027727173":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/52227936":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/1720809791":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/2805455393":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/4003028162":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/3290548668":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/3559844799":"2023-10-17T11:17:24.967015+0000","172.21.15.181:6833/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6832/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/2238590139":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/231495737":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/1499024696":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/458160361":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1195985489":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/4000110033":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/990187329":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3304094772":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6833/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1162194396":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6832/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3436855298":"2023-10-17T11:01:37.009118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:25.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 msgr2=0x7f8c040cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c040cc360 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d800 tx=0x7f8bf400db10 comp rx=0 tx=0).stop 2023-10-16T11:17:25.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 shutdown_connections 2023-10-16T11:17:25.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c04161f40 0x7f8c04152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c04160d20 0x7f8c040cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 --2- 172.21.15.181:0/2635198866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c04161b60 0x7f8c040cc360 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 >> 172.21.15.181:0/2635198866 conn(0x7f8c0409cba0 msgr2=0x7f8c04073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 shutdown_connections 2023-10-16T11:17:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.567+0000 7f8c09a6f700 1 -- 172.21.15.181:0/2635198866 wait complete. 2023-10-16T11:17:25.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:17:25.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.701+0000 7f9112668700 1 Processor -- start 2023-10-16T11:17:25.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.701+0000 7f9112668700 1 -- start start 2023-10-16T11:17:25.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9112668700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9112668700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f910c0cb110 0x7f910c15e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9112668700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f910c15a0f0 0x7f910c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9112668700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c06ed00 con 0x7f910c15a0f0 2023-10-16T11:17:25.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9112668700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c15ed80 con 0x7f910c0cb110 2023-10-16T11:17:25.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9112668700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c06e0f0 con 0x7f910c15e460 2023-10-16T11:17:25.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f910c15a0f0 0x7f910c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f910c15a0f0 0x7f910c15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53498/0 (socket says 172.21.15.181:53498) 2023-10-16T11:17:25.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 -- 172.21.15.181:0/3953931073 learned_addr learned my addr 172.21.15.181:0/3953931073 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:25.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f910b7fe700 1 --2- 172.21.15.181:0/3953931073 >> v2:172.21.15.190:3300/0 conn(0x7f910c0cb110 0x7f910c15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f910bfff700 1 --2- 172.21.15.181:0/3953931073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 -- 172.21.15.181:0/3953931073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 msgr2=0x7f910c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:25.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 --2- 172.21.15.181:0/3953931073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 -- 172.21.15.181:0/3953931073 >> v2:172.21.15.190:3300/0 conn(0x7f910c0cb110 msgr2=0x7f910c15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:25.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 --2- 172.21.15.181:0/3953931073 >> v2:172.21.15.190:3300/0 conn(0x7f910c0cb110 0x7f910c15e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 -- 172.21.15.181:0/3953931073 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c06e3d0 con 0x7f910c15a0f0 2023-10-16T11:17:25.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.702+0000 7f9110c05700 1 --2- 172.21.15.181:0/3953931073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c15a5a0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f910c0713c0 tx=0x7f910400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a345145d8b667031 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:25.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f910affd700 1 -- 172.21.15.181:0/3953931073 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400e070 con 0x7f910c15a0f0 2023-10-16T11:17:25.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f910affd700 1 -- 172.21.15.181:0/3953931073 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910400b9c0 con 0x7f910c15a0f0 2023-10-16T11:17:25.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f910affd700 1 -- 172.21.15.181:0/3953931073 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400bd70 con 0x7f910c15a0f0 2023-10-16T11:17:25.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 -- 172.21.15.181:0/3953931073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 msgr2=0x7f910c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 --2- 172.21.15.181:0/3953931073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c15a5a0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f910c0713c0 tx=0x7f910400b330 comp rx=0 tx=0).stop 2023-10-16T11:17:25.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 -- 172.21.15.181:0/3953931073 shutdown_connections 2023-10-16T11:17:25.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 --2- 172.21.15.181:0/3953931073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c15a5a0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 --2- 172.21.15.181:0/3953931073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 --2- 172.21.15.181:0/3953931073 >> v2:172.21.15.190:3300/0 conn(0x7f910c0cb110 0x7f910c15e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 -- 172.21.15.181:0/3953931073 >> 172.21.15.181:0/3953931073 conn(0x7f910c0d04a0 msgr2=0x7f910c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:25.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 -- 172.21.15.181:0/3953931073 shutdown_connections 2023-10-16T11:17:25.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.703+0000 7f9112668700 1 -- 172.21.15.181:0/3953931073 wait complete. 2023-10-16T11:17:25.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.704+0000 7f9112668700 1 Processor -- start 2023-10-16T11:17:25.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9112668700 1 -- start start 2023-10-16T11:17:25.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9112668700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910c0cb110 0x7f910c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9112668700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c1f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9112668700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:25.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9112668700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c1e3da0 con 0x7f910c15a0f0 2023-10-16T11:17:25.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9112668700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0c7b50 con 0x7f910c0cb110 2023-10-16T11:17:25.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9112668700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0c7e30 con 0x7f910c15e460 2023-10-16T11:17:25.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f910b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c1f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f910b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c1f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53502/0 (socket says 172.21.15.181:53502) 2023-10-16T11:17:25.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f910b7fe700 1 -- 172.21.15.181:0/3908214894 learned_addr learned my addr 172.21.15.181:0/3908214894 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:25.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f910bfff700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910c0cb110 0x7f910c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.705+0000 7f9110c05700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:25.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f910b7fe700 1 -- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 msgr2=0x7f910c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f910b7fe700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f910b7fe700 1 -- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910c0cb110 msgr2=0x7f910c0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f910b7fe700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910c0cb110 0x7f910c0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f910b7fe700 1 -- 172.21.15.181:0/3908214894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104009b30 con 0x7f910c15a0f0 2023-10-16T11:17:25.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f910b7fe700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c1f2530 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c7f0 tx=0x7f90fc00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:25.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f9108ff9700 1 -- 172.21.15.181:0/3908214894 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc011800 con 0x7f910c15a0f0 2023-10-16T11:17:25.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f9108ff9700 1 -- 172.21.15.181:0/3908214894 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc011960 con 0x7f910c15a0f0 2023-10-16T11:17:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c0c8110 con 0x7f910c15a0f0 2023-10-16T11:17:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f9108ff9700 1 -- 172.21.15.181:0/3908214894 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc011cf0 con 0x7f910c15a0f0 2023-10-16T11:17:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.706+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c0c9c00 con 0x7f910c15a0f0 2023-10-16T11:17:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.707+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c1e3ee0 con 0x7f910c15a0f0 2023-10-16T11:17:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.707+0000 7f9108ff9700 1 -- 172.21.15.181:0/3908214894 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 124) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f90fc018360 con 0x7f910c15a0f0 2023-10-16T11:17:25.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.708+0000 7f9108ff9700 1 -- 172.21.15.181:0/3908214894 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f90fc089430 con 0x7f910c15a0f0 2023-10-16T11:17:25.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.712+0000 7f9108ff9700 1 -- 172.21.15.181:0/3908214894 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f910c1e3ee0 con 0x7f910c15a0f0 2023-10-16T11:17:25.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.948+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f910c1e3ee0 con 0x7f910c15a0f0 2023-10-16T11:17:25.949 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:17:25.948+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:17:25.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.950+0000 7f9108ff9700 1 -- 172.21.15.181:0/3908214894 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7f910c1e3ee0 con 0x7f910c15a0f0 2023-10-16T11:17:25.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 msgr2=0x7f910c1f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:25.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c1f2530 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c7f0 tx=0x7f90fc00cb00 comp rx=0 tx=0).stop 2023-10-16T11:17:25.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 shutdown_connections 2023-10-16T11:17:25.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f910c15e460 0x7f910c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910c0cb110 0x7f910c0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 --2- 172.21.15.181:0/3908214894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f910c15a0f0 0x7f910c1f2530 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:25.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 >> 172.21.15.181:0/3908214894 conn(0x7f910c0d04a0 msgr2=0x7f910c0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:25.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 shutdown_connections 2023-10-16T11:17:25.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:25.953+0000 7f9112668700 1 -- 172.21.15.181:0/3908214894 wait complete. 2023-10-16T11:17:25.956 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:17:25.966 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:17:25.997 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:17:26.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.093+0000 7f3b38579700 1 Processor -- start 2023-10-16T11:17:26.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.093+0000 7f3b38579700 1 -- start start 2023-10-16T11:17:26.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b38579700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b30159e10 0x7f3b300ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:26.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b38579700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b3015a1f0 0x7f3b3015db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:26.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b38579700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3006e4b0 con 0x7f3b3006bef0 2023-10-16T11:17:26.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b38579700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3006ed50 con 0x7f3b3015a1f0 2023-10-16T11:17:26.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b38579700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b300ceea0 con 0x7f3b30159e10 2023-10-16T11:17:26.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b35b14700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3b3006bef0 0x7f3b300caba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:48548/0 (socket says 172.21.15.181:48548) 2023-10-16T11:17:26.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b35b14700 1 -- 172.21.15.181:0/3446556908 learned_addr learned my addr 172.21.15.181:0/3446556908 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:26.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36315700 1 --2- 172.21.15.181:0/3446556908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b30159e10 0x7f3b300ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:26.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36b16700 1 --2- 172.21.15.181:0/3446556908 >> v2:172.21.15.190:3300/0 conn(0x7f3b3015a1f0 0x7f3b3015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:26.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b35313700 1 -- 172.21.15.181:0/3446556908 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421055009 0 0) 0x7f3b3006e4b0 con 0x7f3b3006bef0 2023-10-16T11:17:26.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b35313700 1 -- 172.21.15.181:0/3446556908 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b1c0035b0 con 0x7f3b3006bef0 2023-10-16T11:17:26.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36315700 1 -- 172.21.15.181:0/3446556908 >> v2:172.21.15.190:3300/0 conn(0x7f3b3015a1f0 msgr2=0x7f3b3015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:26.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36315700 1 --2- 172.21.15.181:0/3446556908 >> v2:172.21.15.190:3300/0 conn(0x7f3b3015a1f0 0x7f3b3015db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36315700 1 -- 172.21.15.181:0/3446556908 >> v1:172.21.15.190:6789/0 conn(0x7f3b3006bef0 legacy=0x7f3b300caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:26.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36315700 1 -- 172.21.15.181:0/3446556908 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3006e700 con 0x7f3b30159e10 2023-10-16T11:17:26.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36b16700 1 --2- 172.21.15.181:0/3446556908 >> v2:172.21.15.190:3300/0 conn(0x7f3b3015a1f0 0x7f3b3015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:26.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b36315700 1 --2- 172.21.15.181:0/3446556908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b30159e10 0x7f3b300ce5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009840 tx=0x7f3b2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d6eaebc856b549b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.094+0000 7f3b35313700 1 -- 172.21.15.181:0/3446556908 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c011830 con 0x7f3b30159e10 2023-10-16T11:17:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b35313700 1 -- 172.21.15.181:0/3446556908 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c011990 con 0x7f3b30159e10 2023-10-16T11:17:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b35313700 1 -- 172.21.15.181:0/3446556908 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c011d20 con 0x7f3b30159e10 2023-10-16T11:17:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 -- 172.21.15.181:0/3446556908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b30159e10 msgr2=0x7f3b300ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 --2- 172.21.15.181:0/3446556908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b30159e10 0x7f3b300ce5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009840 tx=0x7f3b2c009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 -- 172.21.15.181:0/3446556908 shutdown_connections 2023-10-16T11:17:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 --2- 172.21.15.181:0/3446556908 >> v2:172.21.15.190:3300/0 conn(0x7f3b3015a1f0 0x7f3b3015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 --2- 172.21.15.181:0/3446556908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b30159e10 0x7f3b300ce5a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 -- 172.21.15.181:0/3446556908 >> 172.21.15.181:0/3446556908 conn(0x7f3b300cc730 msgr2=0x7f3b300ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:26.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 -- 172.21.15.181:0/3446556908 shutdown_connections 2023-10-16T11:17:26.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.095+0000 7f3b38579700 1 -- 172.21.15.181:0/3446556908 wait complete. 2023-10-16T11:17:26.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.096+0000 7f3b38579700 1 Processor -- start 2023-10-16T11:17:26.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.096+0000 7f3b38579700 1 -- start start 2023-10-16T11:17:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b38579700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3006bef0 0x7f3b3014bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b38579700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b30159e10 0x7f3b30146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b38579700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3015a1f0 0x7f3b300c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b38579700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3014e330 con 0x7f3b30159e10 2023-10-16T11:17:26.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b38579700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3014e750 con 0x7f3b3015a1f0 2023-10-16T11:17:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b38579700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30147000 con 0x7f3b3006bef0 2023-10-16T11:17:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b35b14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b30159e10 0x7f3b30146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b35b14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b30159e10 0x7f3b30146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53534/0 (socket says 172.21.15.181:53534) 2023-10-16T11:17:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b35b14700 1 -- 172.21.15.181:0/1430313296 learned_addr learned my addr 172.21.15.181:0/1430313296 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:26.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b36b16700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3015a1f0 0x7f3b300c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.097+0000 7f3b36315700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3006bef0 0x7f3b3014bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b35b14700 1 -- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3006bef0 msgr2=0x7f3b3014bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b35b14700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3006bef0 0x7f3b3014bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b35b14700 1 -- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3015a1f0 msgr2=0x7f3b300c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:26.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b35b14700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3015a1f0 0x7f3b300c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b35b14700 1 -- 172.21.15.181:0/1430313296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28009a70 con 0x7f3b30159e10 2023-10-16T11:17:26.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b35b14700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b30159e10 0x7f3b30146690 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f3b20001950 tx=0x7f3b20001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:26.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b26ffd700 1 -- 172.21.15.181:0/1430313296 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20005b40 con 0x7f3b30159e10 2023-10-16T11:17:26.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b26ffd700 1 -- 172.21.15.181:0/1430313296 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20005ca0 con 0x7f3b30159e10 2023-10-16T11:17:26.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c0094a0 con 0x7f3b30159e10 2023-10-16T11:17:26.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b26ffd700 1 -- 172.21.15.181:0/1430313296 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200072b0 con 0x7f3b30159e10 2023-10-16T11:17:26.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.098+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3014dd20 con 0x7f3b30159e10 2023-10-16T11:17:26.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.099+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3014e470 con 0x7f3b30159e10 2023-10-16T11:17:26.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.099+0000 7f3b26ffd700 1 -- 172.21.15.181:0/1430313296 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 125) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3b20007410 con 0x7f3b30159e10 2023-10-16T11:17:26.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.099+0000 7f3b26ffd700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3b1406ad50 0x7f3b1406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:26.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.100+0000 7f3b26ffd700 1 -- 172.21.15.181:0/1430313296 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f3b2008d2b0 con 0x7f3b30159e10 2023-10-16T11:17:26.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.100+0000 7f3b36315700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3b1406ad50 0x7f3b1406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:26.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.100+0000 7f3b36315700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3b1406ad50 0x7f3b1406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000c00 tx=0x7f3b2c008030 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:26.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.104+0000 7f3b26ffd700 1 -- 172.21.15.181:0/1430313296 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b3014e470 con 0x7f3b30159e10 2023-10-16T11:17:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.339+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3b3014e470 con 0x7f3b30159e10 2023-10-16T11:17:26.704 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:26.706 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:26.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.991+0000 7f3b26ffd700 1 -- 172.21.15.181:0/1430313296 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3b3014e470 con 0x7f3b30159e10 2023-10-16T11:17:26.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3b1406ad50 msgr2=0x7f3b1406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:26.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3b1406ad50 0x7f3b1406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000c00 tx=0x7f3b2c008030 comp rx=0 tx=0).stop 2023-10-16T11:17:26.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b30159e10 msgr2=0x7f3b30146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:26.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b30159e10 0x7f3b30146690 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f3b20001950 tx=0x7f3b20001c60 comp rx=0 tx=0).stop 2023-10-16T11:17:26.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 shutdown_connections 2023-10-16T11:17:26.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3b1406ad50 0x7f3b1406d210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3006bef0 0x7f3b3014bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b3015a1f0 0x7f3b300c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 --2- 172.21.15.181:0/1430313296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b30159e10 0x7f3b30146690 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 >> 172.21.15.181:0/1430313296 conn(0x7f3b300cc730 msgr2=0x7f3b300cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 shutdown_connections 2023-10-16T11:17:26.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:26.994+0000 7f3b38579700 1 -- 172.21.15.181:0/1430313296 wait complete. 2023-10-16T11:17:27.008 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:17:27.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.134+0000 7f89c9c55700 1 Processor -- start 2023-10-16T11:17:27.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.134+0000 7f89c9c55700 1 -- start start 2023-10-16T11:17:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.134+0000 7f89c9c55700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89c40ce3a0 0x7f89c40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.134+0000 7f89c9c55700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89c40cd3f0 0x7f89c40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:27.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.134+0000 7f89c9c55700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c40ce7e0 con 0x7f89c40afa10 2023-10-16T11:17:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.134+0000 7f89c9c55700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c40cd880 con 0x7f89c40ce3a0 2023-10-16T11:17:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.134+0000 7f89c9c55700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c415b540 con 0x7f89c40cd3f0 2023-10-16T11:17:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89c40ce3a0 0x7f89c40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:27.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f89c40afa10 0x7f89c406d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36404/0 (socket says 172.21.15.181:36404) 2023-10-16T11:17:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89c40ce3a0 0x7f89c40cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53560/0 (socket says 172.21.15.181:53560) 2023-10-16T11:17:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c37fe700 1 -- 172.21.15.181:0/1864246872 learned_addr learned my addr 172.21.15.181:0/1864246872 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c3fff700 1 --2- 172.21.15.181:0/1864246872 >> v2:172.21.15.190:3300/0 conn(0x7f89c40cd3f0 0x7f89c40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c2ffd700 1 -- 172.21.15.181:0/1864246872 >> v2:172.21.15.190:3300/0 conn(0x7f89c40cd3f0 msgr2=0x7f89c40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c2ffd700 1 --2- 172.21.15.181:0/1864246872 >> v2:172.21.15.190:3300/0 conn(0x7f89c40cd3f0 0x7f89c40cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:27.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c2ffd700 1 -- 172.21.15.181:0/1864246872 >> v1:172.21.15.181:6789/0 conn(0x7f89c40afa10 legacy=0x7f89c406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c27fc700 1 -- 172.21.15.181:0/1864246872 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713965739 0 0) 0x7f89c40ce7e0 con 0x7f89c40afa10 2023-10-16T11:17:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c2ffd700 1 -- 172.21.15.181:0/1864246872 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c415b820 con 0x7f89c40ce3a0 2023-10-16T11:17:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c3fff700 1 --2- 172.21.15.181:0/1864246872 >> v2:172.21.15.190:3300/0 conn(0x7f89c40cd3f0 0x7f89c40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c2ffd700 1 --2- 172.21.15.181:0/1864246872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40ce3a0 0x7f89c40cceb0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f89c40703d0 tx=0x7f89ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b819b07f5cc10d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c27fc700 1 -- 172.21.15.181:0/1864246872 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac011830 con 0x7f89c40ce3a0 2023-10-16T11:17:27.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c27fc700 1 -- 172.21.15.181:0/1864246872 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac011990 con 0x7f89c40ce3a0 2023-10-16T11:17:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.135+0000 7f89c27fc700 1 -- 172.21.15.181:0/1864246872 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac011d20 con 0x7f89c40ce3a0 2023-10-16T11:17:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 -- 172.21.15.181:0/1864246872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40ce3a0 msgr2=0x7f89c40cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 --2- 172.21.15.181:0/1864246872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40ce3a0 0x7f89c40cceb0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f89c40703d0 tx=0x7f89ac009b20 comp rx=0 tx=0).stop 2023-10-16T11:17:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 -- 172.21.15.181:0/1864246872 shutdown_connections 2023-10-16T11:17:27.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 --2- 172.21.15.181:0/1864246872 >> v2:172.21.15.190:3300/0 conn(0x7f89c40cd3f0 0x7f89c40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 --2- 172.21.15.181:0/1864246872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40ce3a0 0x7f89c40cceb0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 -- 172.21.15.181:0/1864246872 >> 172.21.15.181:0/1864246872 conn(0x7f89c40d04a0 msgr2=0x7f89c40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 -- 172.21.15.181:0/1864246872 shutdown_connections 2023-10-16T11:17:27.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.136+0000 7f89c9c55700 1 -- 172.21.15.181:0/1864246872 wait complete. 2023-10-16T11:17:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.137+0000 7f89c9c55700 1 Processor -- start 2023-10-16T11:17:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.137+0000 7f89c9c55700 1 -- start start 2023-10-16T11:17:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c9c55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40afa10 0x7f89c4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c9c55700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89c40cd3f0 0x7f89c40c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c9c55700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89c40ce3a0 0x7f89c40ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c9c55700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c41f4390 con 0x7f89c40afa10 2023-10-16T11:17:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c9c55700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c40b4990 con 0x7f89c40ce3a0 2023-10-16T11:17:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c9c55700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c41f40b0 con 0x7f89c40cd3f0 2023-10-16T11:17:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40afa10 0x7f89c4156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40afa10 0x7f89c4156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53564/0 (socket says 172.21.15.181:53564) 2023-10-16T11:17:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 -- 172.21.15.181:0/122372916 learned_addr learned my addr 172.21.15.181:0/122372916 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c3fff700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89c40ce3a0 0x7f89c40ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 -- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89c40cd3f0 msgr2=0x7f89c40c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89c40cd3f0 0x7f89c40c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 -- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89c40ce3a0 msgr2=0x7f89c40ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89c40ce3a0 0x7f89c40ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c37fe700 1 -- 172.21.15.181:0/122372916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0094a0 con 0x7f89c40afa10 2023-10-16T11:17:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.138+0000 7f89c3fff700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89c40ce3a0 0x7f89c40ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.139+0000 7f89c37fe700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40afa10 0x7f89c4156770 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f89b40010e0 tx=0x7f89b400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.139+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40074e0 con 0x7f89c40afa10 2023-10-16T11:17:27.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.139+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b4007640 con 0x7f89c40afa10 2023-10-16T11:17:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.139+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c41f0080 con 0x7f89c40afa10 2023-10-16T11:17:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.139+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4005a70 con 0x7f89c40afa10 2023-10-16T11:17:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.139+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c41f0640 con 0x7f89c40afa10 2023-10-16T11:17:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.140+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c4067300 con 0x7f89c40afa10 2023-10-16T11:17:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.140+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 125) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f89b4005bd0 con 0x7f89c40afa10 2023-10-16T11:17:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.140+0000 7f89bbfff700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f89a406ad50 0x7f89a406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.140+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f89b408b7c0 con 0x7f89c40afa10 2023-10-16T11:17:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.140+0000 7f89c2ffd700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f89a406ad50 0x7f89a406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.141+0000 7f89c2ffd700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f89a406ad50 0x7f89a406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f89c409e330 tx=0x7f89ac010000 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.145+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f89b4056540 con 0x7f89c40afa10 2023-10-16T11:17:27.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.334+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89c40ccd20 con 0x7f89c40afa10 2023-10-16T11:17:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.459+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 126) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f89b4056910 con 0x7f89c40afa10 2023-10-16T11:17:27.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.993+0000 7f89bbfff700 1 -- 172.21.15.181:0/122372916 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v189) v1 ==== 181+0+0 (secure 0 0 0) 0x7f89b4075530 con 0x7f89c40afa10 2023-10-16T11:17:27.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f89a406ad50 msgr2=0x7f89a406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:27.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f89a406ad50 0x7f89a406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f89c409e330 tx=0x7f89ac010000 comp rx=0 tx=0).stop 2023-10-16T11:17:27.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40afa10 msgr2=0x7f89c4156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:28.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40afa10 0x7f89c4156770 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f89b40010e0 tx=0x7f89b400ca30 comp rx=0 tx=0).stop 2023-10-16T11:17:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 shutdown_connections 2023-10-16T11:17:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f89a406ad50 0x7f89a406d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89c40ce3a0 0x7f89c40ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89c40afa10 0x7f89c4156770 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 --2- 172.21.15.181:0/122372916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89c40cd3f0 0x7f89c40c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.997+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 >> 172.21.15.181:0/122372916 conn(0x7f89c40d04a0 msgr2=0x7f89c40ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:28.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.998+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 shutdown_connections 2023-10-16T11:17:28.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:27.998+0000 7f89c9c55700 1 -- 172.21.15.181:0/122372916 wait complete. 2023-10-16T11:17:28.004 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:17:28.013 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:17:28.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.141+0000 7f471bc64700 1 Processor -- start 2023-10-16T11:17:28.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.142+0000 7f471bc64700 1 -- start start 2023-10-16T11:17:28.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.142+0000 7f471bc64700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47140ce610 0x7f471406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:28.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.142+0000 7f471bc64700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47140ce9f0 0x7f471406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:28.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.142+0000 7f471bc64700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4714162650 con 0x7f47140d0fc0 2023-10-16T11:17:28.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.142+0000 7f471bc64700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4714162190 con 0x7f47140ce9f0 2023-10-16T11:17:28.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.142+0000 7f471bc64700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140d0230 con 0x7f47140ce610 2023-10-16T11:17:28.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f47191ff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f47140d0fc0 0x7f47140cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36418/0 (socket says 172.21.15.181:36418) 2023-10-16T11:17:28.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f47191ff700 1 -- 172.21.15.181:0/80070632 learned_addr learned my addr 172.21.15.181:0/80070632 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:28.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f471a201700 1 --2- 172.21.15.181:0/80070632 >> v2:172.21.15.181:3300/0 conn(0x7f47140ce9f0 0x7f471406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:28.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f47189fe700 1 -- 172.21.15.181:0/80070632 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800460015 0 0) 0x7f4714162650 con 0x7f47140d0fc0 2023-10-16T11:17:28.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f47189fe700 1 -- 172.21.15.181:0/80070632 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47000035b0 con 0x7f47140d0fc0 2023-10-16T11:17:28.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f471a201700 1 -- 172.21.15.181:0/80070632 >> v2:172.21.15.190:3300/0 conn(0x7f47140ce610 msgr2=0x7f471406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:28.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f471a201700 1 --2- 172.21.15.181:0/80070632 >> v2:172.21.15.190:3300/0 conn(0x7f47140ce610 0x7f471406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f471a201700 1 -- 172.21.15.181:0/80070632 >> v1:172.21.15.181:6789/0 conn(0x7f47140d0fc0 legacy=0x7f47140cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:28.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f471a201700 1 -- 172.21.15.181:0/80070632 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714162370 con 0x7f47140ce9f0 2023-10-16T11:17:28.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f471a201700 1 --2- 172.21.15.181:0/80070632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471406a8a0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24e045e16ccb4f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:28.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f47189fe700 1 -- 172.21.15.181:0/80070632 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00e070 con 0x7f47140ce9f0 2023-10-16T11:17:28.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f47189fe700 1 -- 172.21.15.181:0/80070632 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00bae0 con 0x7f47140ce9f0 2023-10-16T11:17:28.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.143+0000 7f47189fe700 1 -- 172.21.15.181:0/80070632 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00be90 con 0x7f47140ce9f0 2023-10-16T11:17:28.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 -- 172.21.15.181:0/80070632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 msgr2=0x7f471406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:28.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 --2- 172.21.15.181:0/80070632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471406a8a0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c00b330 comp rx=0 tx=0).stop 2023-10-16T11:17:28.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 -- 172.21.15.181:0/80070632 shutdown_connections 2023-10-16T11:17:28.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 --2- 172.21.15.181:0/80070632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471406a8a0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 --2- 172.21.15.181:0/80070632 >> v2:172.21.15.190:3300/0 conn(0x7f47140ce610 0x7f471406d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 -- 172.21.15.181:0/80070632 >> 172.21.15.181:0/80070632 conn(0x7f47140d04a0 msgr2=0x7f47140d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:28.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 -- 172.21.15.181:0/80070632 shutdown_connections 2023-10-16T11:17:28.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.144+0000 7f471bc64700 1 -- 172.21.15.181:0/80070632 wait complete. 2023-10-16T11:17:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.146+0000 7f471bc64700 1 Processor -- start 2023-10-16T11:17:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.146+0000 7f471bc64700 1 -- start start 2023-10-16T11:17:28.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47140ce610 0x7f471415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:28.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471409c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:28.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47140d0fc0 0x7f47140cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:28.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471409a180 con 0x7f47140ce9f0 2023-10-16T11:17:28.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471415a8a0 con 0x7f47140ce610 2023-10-16T11:17:28.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47141f4430 con 0x7f47140d0fc0 2023-10-16T11:17:28.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471409c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:28.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471409c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53580/0 (socket says 172.21.15.181:53580) 2023-10-16T11:17:28.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 -- 172.21.15.181:0/2272901245 learned_addr learned my addr 172.21.15.181:0/2272901245 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:28.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f4719a00700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47140ce610 0x7f471415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:28.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 -- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47140d0fc0 msgr2=0x7f47140cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:28.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47140d0fc0 0x7f47140cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 -- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47140ce610 msgr2=0x7f471415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:28.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47140ce610 0x7f471415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:28.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 -- 172.21.15.181:0/2272901245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c009a70 con 0x7f47140ce9f0 2023-10-16T11:17:28.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f4719a00700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47140ce610 0x7f471415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:28.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f47191ff700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471409c4b0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f4704001eb0 tx=0x7f470400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:28.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f470a7fc700 1 -- 172.21.15.181:0/2272901245 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704005d70 con 0x7f47140ce9f0 2023-10-16T11:17:28.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f470a7fc700 1 -- 172.21.15.181:0/2272901245 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47040070d0 con 0x7f47140ce9f0 2023-10-16T11:17:28.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471409a390 con 0x7f47140ce9f0 2023-10-16T11:17:28.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f470a7fc700 1 -- 172.21.15.181:0/2272901245 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704007460 con 0x7f47140ce9f0 2023-10-16T11:17:28.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.147+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471409a950 con 0x7f47140ce9f0 2023-10-16T11:17:28.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.148+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4714067d20 con 0x7f47140ce9f0 2023-10-16T11:17:28.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.149+0000 7f470a7fc700 1 -- 172.21.15.181:0/2272901245 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 126) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f47040075c0 con 0x7f47140ce9f0 2023-10-16T11:17:28.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.149+0000 7f470a7fc700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f46f806af90 0x7f46f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:28.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.149+0000 7f470a7fc700 1 -- 172.21.15.181:0/2272901245 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6913+0+0 (secure 0 0 0) 0x7f4704088f40 con 0x7f47140ce9f0 2023-10-16T11:17:28.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.149+0000 7f4719a00700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f46f806af90 0x7f46f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:28.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.149+0000 7f4719a00700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f46f806af90 0x7f46f806d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4710006fd0 tx=0x7f4710008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:28.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.153+0000 7f470a7fc700 1 -- 172.21.15.181:0/2272901245 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f47040bbfe0 con 0x7f47140ce9f0 2023-10-16T11:17:28.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.345+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4714066fb0 con 0x7f47140ce9f0 2023-10-16T11:17:28.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.995+0000 7f470a7fc700 1 -- 172.21.15.181:0/2272901245 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v190) v1 ==== 169+0+0 (secure 0 0 0) 0x7f47040170d0 con 0x7f47140ce9f0 2023-10-16T11:17:29.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.999+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f46f806af90 msgr2=0x7f46f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.999+0000 7f471bc64700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f46f806af90 0x7f46f806d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4710006fd0 tx=0x7f4710008040 comp rx=0 tx=0).stop 2023-10-16T11:17:29.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.999+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 msgr2=0x7f471409c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:28.999+0000 7f471bc64700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471409c4b0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f4704001eb0 tx=0x7f470400c860 comp rx=0 tx=0).stop 2023-10-16T11:17:29.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 shutdown_connections 2023-10-16T11:17:29.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f46f806af90 0x7f46f806d450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47140d0fc0 0x7f47140cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47140ce610 0x7f471415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 --2- 172.21.15.181:0/2272901245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47140ce9f0 0x7f471409c4b0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 >> 172.21.15.181:0/2272901245 conn(0x7f47140d04a0 msgr2=0x7f47140b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:29.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 shutdown_connections 2023-10-16T11:17:29.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.000+0000 7f471bc64700 1 -- 172.21.15.181:0/2272901245 wait complete. 2023-10-16T11:17:29.003 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:17:29.016 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:17:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.143+0000 7f9a9d6ed700 1 Processor -- start 2023-10-16T11:17:29.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.143+0000 7f9a9d6ed700 1 -- start start 2023-10-16T11:17:29.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.143+0000 7f9a9d6ed700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a9815d740 0x7f9a98161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.143+0000 7f9a9d6ed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980ce5e0 0x7f9a980ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.143+0000 7f9a9d6ed700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a980d0cb0 con 0x7f9a980cf890 2023-10-16T11:17:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.143+0000 7f9a9d6ed700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a981624b0 con 0x7f9a9815d740 2023-10-16T11:17:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.143+0000 7f9a9d6ed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980d00a0 con 0x7f9a980ce5e0 2023-10-16T11:17:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980ce5e0 0x7f9a980ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a96ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a9815d740 0x7f9a98161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980ce5e0 0x7f9a980ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:50616/0 (socket says 172.21.15.181:50616) 2023-10-16T11:17:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a96ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a9815d740 0x7f9a98161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:58636/0 (socket says 172.21.15.181:58636) 2023-10-16T11:17:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 -- 172.21.15.181:0/4134730410 learned_addr learned my addr 172.21.15.181:0/4134730410 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a967fc700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9a980cf890 0x7f9a980cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:48608/0 (socket says 172.21.15.181:48608) 2023-10-16T11:17:29.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 -- 172.21.15.181:0/4134730410 >> v2:172.21.15.190:3300/0 conn(0x7f9a9815d740 msgr2=0x7f9a98161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 --2- 172.21.15.181:0/4134730410 >> v2:172.21.15.190:3300/0 conn(0x7f9a9815d740 0x7f9a98161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 -- 172.21.15.181:0/4134730410 >> v1:172.21.15.190:6789/0 conn(0x7f9a980cf890 legacy=0x7f9a980cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 -- 172.21.15.181:0/4134730410 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a980d0380 con 0x7f9a980ce5e0 2023-10-16T11:17:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a96ffd700 1 --2- 172.21.15.181:0/4134730410 >> v2:172.21.15.190:3300/0 conn(0x7f9a9815d740 0x7f9a98161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a977fe700 1 --2- 172.21.15.181:0/4134730410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980ce5e0 0x7f9a980ccf00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009e50 tx=0x7f9a9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8fc59409aa36b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a95ffb700 1 -- 172.21.15.181:0/4134730410 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000e070 con 0x7f9a980ce5e0 2023-10-16T11:17:29.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.144+0000 7f9a95ffb700 1 -- 172.21.15.181:0/4134730410 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9000b910 con 0x7f9a980ce5e0 2023-10-16T11:17:29.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a95ffb700 1 -- 172.21.15.181:0/4134730410 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000bcc0 con 0x7f9a980ce5e0 2023-10-16T11:17:29.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/4134730410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980ce5e0 msgr2=0x7f9a980ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/4134730410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980ce5e0 0x7f9a980ccf00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009e50 tx=0x7f9a9000b330 comp rx=0 tx=0).stop 2023-10-16T11:17:29.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/4134730410 shutdown_connections 2023-10-16T11:17:29.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/4134730410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980ce5e0 0x7f9a980ccf00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/4134730410 >> v2:172.21.15.190:3300/0 conn(0x7f9a9815d740 0x7f9a98161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/4134730410 >> 172.21.15.181:0/4134730410 conn(0x7f9a9809a930 msgr2=0x7f9a980d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:29.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/4134730410 shutdown_connections 2023-10-16T11:17:29.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.145+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/4134730410 wait complete. 2023-10-16T11:17:29.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.146+0000 7f9a9d6ed700 1 Processor -- start 2023-10-16T11:17:29.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.146+0000 7f9a9d6ed700 1 -- start start 2023-10-16T11:17:29.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a9d6ed700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980ce5e0 0x7f9a980ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a9d6ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a980cf890 0x7f9a98151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a9d6ed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815d740 0x7f9a980ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a9d6ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980c6c10 con 0x7f9a980cf890 2023-10-16T11:17:29.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a9d6ed700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980ae760 con 0x7f9a980ce5e0 2023-10-16T11:17:29.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a9d6ed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980af5a0 con 0x7f9a9815d740 2023-10-16T11:17:29.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a980cf890 0x7f9a98151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a967fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a980cf890 0x7f9a98151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53592/0 (socket says 172.21.15.181:53592) 2023-10-16T11:17:29.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a967fc700 1 -- 172.21.15.181:0/2943847618 learned_addr learned my addr 172.21.15.181:0/2943847618 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:29.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a977fe700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815d740 0x7f9a980ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.147+0000 7f9a96ffd700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980ce5e0 0x7f9a980ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a967fc700 1 -- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815d740 msgr2=0x7f9a980ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a967fc700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815d740 0x7f9a980ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a967fc700 1 -- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980ce5e0 msgr2=0x7f9a980ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a967fc700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980ce5e0 0x7f9a980ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a967fc700 1 -- 172.21.15.181:0/2943847618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c0094a0 con 0x7f9a980cf890 2023-10-16T11:17:29.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a96ffd700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980ce5e0 0x7f9a980ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:29.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a967fc700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a980cf890 0x7f9a98151a60 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800c520 tx=0x7f9a8800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a7f7fe700 1 -- 172.21.15.181:0/2943847618 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88005cb0 con 0x7f9a980cf890 2023-10-16T11:17:29.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a7f7fe700 1 -- 172.21.15.181:0/2943847618 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88005e10 con 0x7f9a980cf890 2023-10-16T11:17:29.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a90009b30 con 0x7f9a980cf890 2023-10-16T11:17:29.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a7f7fe700 1 -- 172.21.15.181:0/2943847618 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a880072d0 con 0x7f9a980cf890 2023-10-16T11:17:29.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.148+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a980aec70 con 0x7f9a980cf890 2023-10-16T11:17:29.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.149+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a980c80e0 con 0x7f9a980cf890 2023-10-16T11:17:29.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.149+0000 7f9a7f7fe700 1 -- 172.21.15.181:0/2943847618 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 126) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9a88007430 con 0x7f9a980cf890 2023-10-16T11:17:29.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.150+0000 7f9a7f7fe700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9a8406af90 0x7f9a8406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.150+0000 7f9a7f7fe700 1 -- 172.21.15.181:0/2943847618 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f9a88088bb0 con 0x7f9a980cf890 2023-10-16T11:17:29.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.150+0000 7f9a96ffd700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9a8406af90 0x7f9a8406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.150+0000 7f9a96ffd700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9a8406af90 0x7f9a8406d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c011380 tx=0x7f9a8c009210 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.154+0000 7f9a7f7fe700 1 -- 172.21.15.181:0/2943847618 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9a8808f080 con 0x7f9a980cf890 2023-10-16T11:17:29.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.345+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f9a981622d0 con 0x7f9a980cf890 2023-10-16T11:17:29.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.348+0000 7f9a7f7fe700 1 -- 172.21.15.181:0/2943847618 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v67) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9a880170b0 con 0x7f9a980cf890 2023-10-16T11:17:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.350+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9a8406af90 msgr2=0x7f9a8406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.350+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9a8406af90 0x7f9a8406d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c011380 tx=0x7f9a8c009210 comp rx=0 tx=0).stop 2023-10-16T11:17:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a980cf890 msgr2=0x7f9a98151a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a980cf890 0x7f9a98151a60 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800c520 tx=0x7f9a8800c830 comp rx=0 tx=0).stop 2023-10-16T11:17:29.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 shutdown_connections 2023-10-16T11:17:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9a8406af90 0x7f9a8406d450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815d740 0x7f9a980ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980ce5e0 0x7f9a980ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 --2- 172.21.15.181:0/2943847618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a980cf890 0x7f9a98151a60 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 >> 172.21.15.181:0/2943847618 conn(0x7f9a9809a930 msgr2=0x7f9a9815e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 shutdown_connections 2023-10-16T11:17:29.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.351+0000 7f9a9d6ed700 1 -- 172.21.15.181:0/2943847618 wait complete. 2023-10-16T11:17:29.364 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:17:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.498+0000 7fdb2ef87700 1 Processor -- start 2023-10-16T11:17:29.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.498+0000 7fdb2ef87700 1 -- start start 2023-10-16T11:17:29.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2ef87700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb280ce6c0 0x7fdb280cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2ef87700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb280cc6a0 0x7fdb280ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2ef87700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb280b1f70 con 0x7fdb2815f150 2023-10-16T11:17:29.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2ef87700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb280b2ae0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2ef87700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280b2dc0 con 0x7fdb280ce6c0 2023-10-16T11:17:29.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb280cc6a0 0x7fdb280ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb280cc6a0 0x7fdb280ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53604/0 (socket says 172.21.15.181:53604) 2023-10-16T11:17:29.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 -- 172.21.15.181:0/1433761881 learned_addr learned my addr 172.21.15.181:0/1433761881 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:29.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb27fff700 1 --2- 172.21.15.181:0/1433761881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb280ce6c0 0x7fdb280cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 -- 172.21.15.181:0/1433761881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb280ce6c0 msgr2=0x7fdb280cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:29.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 --2- 172.21.15.181:0/1433761881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb280ce6c0 0x7fdb280cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 -- 172.21.15.181:0/1433761881 >> v1:172.21.15.190:6789/0 conn(0x7fdb2815f150 legacy=0x7fdb280cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 -- 172.21.15.181:0/1433761881 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280cf9d0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb27fff700 1 --2- 172.21.15.181:0/1433761881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb280ce6c0 0x7fdb280cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:29.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb2d524700 1 --2- 172.21.15.181:0/1433761881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280ccb50 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009e50 tx=0x7fdb2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cf96bec02658ab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb277fe700 1 -- 172.21.15.181:0/1433761881 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000e070 con 0x7fdb280cc6a0 2023-10-16T11:17:29.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.499+0000 7fdb277fe700 1 -- 172.21.15.181:0/1433761881 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2000b9c0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb277fe700 1 -- 172.21.15.181:0/1433761881 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000bd70 con 0x7fdb280cc6a0 2023-10-16T11:17:29.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 -- 172.21.15.181:0/1433761881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 msgr2=0x7fdb280ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/1433761881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280ccb50 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009e50 tx=0x7fdb2000b330 comp rx=0 tx=0).stop 2023-10-16T11:17:29.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 -- 172.21.15.181:0/1433761881 shutdown_connections 2023-10-16T11:17:29.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/1433761881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280ccb50 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/1433761881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb280ce6c0 0x7fdb280cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 -- 172.21.15.181:0/1433761881 >> 172.21.15.181:0/1433761881 conn(0x7fdb2809a930 msgr2=0x7fdb280d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:29.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 -- 172.21.15.181:0/1433761881 shutdown_connections 2023-10-16T11:17:29.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.500+0000 7fdb2ef87700 1 -- 172.21.15.181:0/1433761881 wait complete. 2023-10-16T11:17:29.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.501+0000 7fdb2ef87700 1 Processor -- start 2023-10-16T11:17:29.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.501+0000 7fdb2ef87700 1 -- start start 2023-10-16T11:17:29.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2ef87700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2ef87700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb280ce6c0 0x7fdb280ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2ef87700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb2815f150 0x7fdb2815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2ef87700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281f34b0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2ef87700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281f3790 con 0x7fdb280ce6c0 2023-10-16T11:17:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2ef87700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281f3a70 con 0x7fdb2815f150 2023-10-16T11:17:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2cd23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2cd23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53606/0 (socket says 172.21.15.181:53606) 2023-10-16T11:17:29.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2cd23700 1 -- 172.21.15.181:0/3720093111 learned_addr learned my addr 172.21.15.181:0/3720093111 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb27fff700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb280ce6c0 0x7fdb280ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2d524700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb2815f150 0x7fdb2815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2cd23700 1 -- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb2815f150 msgr2=0x7fdb2815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2cd23700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb2815f150 0x7fdb2815d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2cd23700 1 -- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb280ce6c0 msgr2=0x7fdb280ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.502+0000 7fdb2cd23700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb280ce6c0 0x7fdb280ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb2cd23700 1 -- 172.21.15.181:0/3720093111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb20009b30 con 0x7fdb280cc6a0 2023-10-16T11:17:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb27fff700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb280ce6c0 0x7fdb280ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:29.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb2cd23700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280c9c80 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00c450 tx=0x7fdb1c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb257fa700 1 -- 172.21.15.181:0/3720093111 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0052d0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb257fa700 1 -- 172.21.15.181:0/3720093111 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c005430 con 0x7fdb280cc6a0 2023-10-16T11:17:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb281f3d50 con 0x7fdb280cc6a0 2023-10-16T11:17:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb257fa700 1 -- 172.21.15.181:0/3720093111 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0038c0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.503+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb281f3160 con 0x7fdb280cc6a0 2023-10-16T11:17:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.504+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb28067d20 con 0x7fdb280cc6a0 2023-10-16T11:17:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.504+0000 7fdb257fa700 1 -- 172.21.15.181:0/3720093111 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdb1c003a20 con 0x7fdb280cc6a0 2023-10-16T11:17:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.505+0000 7fdb257fa700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fdb0c06b020 0x7fdb0c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.505+0000 7fdb257fa700 1 -- 172.21.15.181:0/3720093111 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7fdb1c088ea0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.505+0000 7fdb27fff700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fdb0c06b020 0x7fdb0c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.505+0000 7fdb27fff700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fdb0c06b020 0x7fdb0c06d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb281f7810 tx=0x7fdb14008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.509+0000 7fdb257fa700 1 -- 172.21.15.181:0/3720093111 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdb1c053aa0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.704+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fdb280cfc80 con 0x7fdb280cc6a0 2023-10-16T11:17:29.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.707+0000 7fdb257fa700 1 -- 172.21.15.181:0/3720093111 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v68) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdb1c0570c0 con 0x7fdb280cc6a0 2023-10-16T11:17:29.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fdb0c06b020 msgr2=0x7fdb0c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fdb0c06b020 0x7fdb0c06d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb281f7810 tx=0x7fdb14008040 comp rx=0 tx=0).stop 2023-10-16T11:17:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 msgr2=0x7fdb280c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280c9c80 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00c450 tx=0x7fdb1c00c760 comp rx=0 tx=0).stop 2023-10-16T11:17:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 shutdown_connections 2023-10-16T11:17:29.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fdb0c06b020 0x7fdb0c06d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb2815f150 0x7fdb2815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb280cc6a0 0x7fdb280c9c80 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 --2- 172.21.15.181:0/3720093111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb280ce6c0 0x7fdb280ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 >> 172.21.15.181:0/3720093111 conn(0x7fdb2809a930 msgr2=0x7fdb280d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:29.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 shutdown_connections 2023-10-16T11:17:29.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.709+0000 7fdb2ef87700 1 -- 172.21.15.181:0/3720093111 wait complete. 2023-10-16T11:17:29.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:17:29.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 Processor -- start 2023-10-16T11:17:29.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 -- start start 2023-10-16T11:17:29.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe5a00cb2b0 0x7fe5a006f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5a00b2ba0 0x7fe5a015f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a0070590 con 0x7fe5a00b2ba0 2023-10-16T11:17:29.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a0156330 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.845+0000 7fe5a5ee2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a006fcf0 con 0x7fe5a00cb690 2023-10-16T11:17:29.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5a00b2ba0 0x7fe5a015f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5a00b2ba0 0x7fe5a015f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53618/0 (socket says 172.21.15.181:53618) 2023-10-16T11:17:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 -- 172.21.15.181:0/2530244327 learned_addr learned my addr 172.21.15.181:0/2530244327 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59f7fe700 1 --2- 172.21.15.181:0/2530244327 >> v2:172.21.15.190:3300/0 conn(0x7fe5a00cb2b0 0x7fe5a006f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59effd700 1 --2- 172.21.15.181:0/2530244327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 -- 172.21.15.181:0/2530244327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 msgr2=0x7fe5a00b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 --2- 172.21.15.181:0/2530244327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 -- 172.21.15.181:0/2530244327 >> v2:172.21.15.190:3300/0 conn(0x7fe5a00cb2b0 msgr2=0x7fe5a006f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 --2- 172.21.15.181:0/2530244327 >> v2:172.21.15.190:3300/0 conn(0x7fe5a00cb2b0 0x7fe5a006f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 -- 172.21.15.181:0/2530244327 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a006ffd0 con 0x7fe5a00b2ba0 2023-10-16T11:17:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59f7fe700 1 --2- 172.21.15.181:0/2530244327 >> v2:172.21.15.190:3300/0 conn(0x7fe5a00cb2b0 0x7fe5a006f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59ffff700 1 --2- 172.21.15.181:0/2530244327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00b2ba0 0x7fe5a015f420 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d1600 tx=0x7fe590009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dd8be13ca14750 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59e7fc700 1 -- 172.21.15.181:0/2530244327 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000b870 con 0x7fe5a00b2ba0 2023-10-16T11:17:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59e7fc700 1 -- 172.21.15.181:0/2530244327 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59000b9d0 con 0x7fe5a00b2ba0 2023-10-16T11:17:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.846+0000 7fe59e7fc700 1 -- 172.21.15.181:0/2530244327 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000bd60 con 0x7fe5a00b2ba0 2023-10-16T11:17:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2530244327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00b2ba0 msgr2=0x7fe5a015f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2530244327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00b2ba0 0x7fe5a015f420 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00d1600 tx=0x7fe590009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2530244327 shutdown_connections 2023-10-16T11:17:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2530244327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00b2ba0 0x7fe5a015f420 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2530244327 >> v2:172.21.15.190:3300/0 conn(0x7fe5a00cb2b0 0x7fe5a006f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2530244327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2530244327 >> 172.21.15.181:0/2530244327 conn(0x7fe5a00cd010 msgr2=0x7fe5a00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:29.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2530244327 shutdown_connections 2023-10-16T11:17:29.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.847+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2530244327 wait complete. 2023-10-16T11:17:29.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.848+0000 7fe5a5ee2700 1 Processor -- start 2023-10-16T11:17:29.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.848+0000 7fe5a5ee2700 1 -- start start 2023-10-16T11:17:29.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe5a5ee2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a00b2ba0 0x7fe5a01f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe5a5ee2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cb2b0 0x7fe5a01f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe5a5ee2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe5a5ee2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a01f5430 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe5a5ee2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0153d50 con 0x7fe5a00b2ba0 2023-10-16T11:17:29.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe5a5ee2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00c9c70 con 0x7fe5a00cb690 2023-10-16T11:17:29.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cb2b0 0x7fe5a01f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cb2b0 0x7fe5a01f06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53622/0 (socket says 172.21.15.181:53622) 2023-10-16T11:17:29.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 -- 172.21.15.181:0/2961244206 learned_addr learned my addr 172.21.15.181:0/2961244206 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:29.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59f7fe700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a00b2ba0 0x7fe5a01f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59ffff700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 -- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 msgr2=0x7fe5a00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 -- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a00b2ba0 msgr2=0x7fe5a01f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a00b2ba0 0x7fe5a01f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 -- 172.21.15.181:0/2961244206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590009550 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.849+0000 7fe59effd700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cb2b0 0x7fe5a01f06c0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d800 tx=0x7fe58800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.850+0000 7fe597fff700 1 -- 172.21.15.181:0/2961244206 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880125f0 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.850+0000 7fe597fff700 1 -- 172.21.15.181:0/2961244206 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588012750 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.850+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a00c8bd0 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.850+0000 7fe597fff700 1 -- 172.21.15.181:0/2961244206 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588012b00 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.850+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a00c9190 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.851+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a006f4f0 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.851+0000 7fe597fff700 1 -- 172.21.15.181:0/2961244206 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe588012c60 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.852+0000 7fe597fff700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe58006b020 0x7fe58006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:29.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.852+0000 7fe59f7fe700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe58006b020 0x7fe58006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:29.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.852+0000 7fe597fff700 1 -- 172.21.15.181:0/2961244206 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7fe588089e20 con 0x7fe5a00cb2b0 2023-10-16T11:17:29.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.852+0000 7fe59f7fe700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe58006b020 0x7fe58006d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe598009a80 tx=0x7fe598003a20 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:29.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:29.856+0000 7fe597fff700 1 -- 172.21.15.181:0/2961244206 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe588054aa0 con 0x7fe5a00cb2b0 2023-10-16T11:17:30.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.052+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe5a0074120 con 0x7fe5a00cb2b0 2023-10-16T11:17:30.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.055+0000 7fe597fff700 1 -- 172.21.15.181:0/2961244206 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe5880580c0 con 0x7fe5a00cb2b0 2023-10-16T11:17:30.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe58006b020 msgr2=0x7fe58006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe58006b020 0x7fe58006d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe598009a80 tx=0x7fe598003a20 comp rx=0 tx=0).stop 2023-10-16T11:17:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cb2b0 msgr2=0x7fe5a01f06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cb2b0 0x7fe5a01f06c0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d800 tx=0x7fe58800dbc0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 shutdown_connections 2023-10-16T11:17:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe58006b020 0x7fe58006d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe5a00cb690 0x7fe5a00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5a00b2ba0 0x7fe5a01f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 --2- 172.21.15.181:0/2961244206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00cb2b0 0x7fe5a01f06c0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 >> 172.21.15.181:0/2961244206 conn(0x7fe5a00cd010 msgr2=0x7fe5a015e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:30.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 shutdown_connections 2023-10-16T11:17:30.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.057+0000 7fe5a5ee2700 1 -- 172.21.15.181:0/2961244206 wait complete. 2023-10-16T11:17:30.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-16T11:17:30.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.190+0000 7f32c97e0700 1 Processor -- start 2023-10-16T11:17:30.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.190+0000 7f32c97e0700 1 -- start start 2023-10-16T11:17:30.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c97e0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32c415eab0 0x7f32c40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:30.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c97e0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32c40cb0c0 0x7f32c415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c97e0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c406ed00 con 0x7f32c415e6d0 2023-10-16T11:17:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c97e0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c40d02d0 con 0x7f32c415eab0 2023-10-16T11:17:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c97e0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c415ad40 con 0x7f32c40cb0c0 2023-10-16T11:17:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32c415eab0 0x7f32c40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f32c415e6d0 0x7f32c415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36474/0 (socket says 172.21.15.181:36474) 2023-10-16T11:17:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.191+0000 7f32c2ffd700 1 -- 172.21.15.181:0/1039964345 learned_addr learned my addr 172.21.15.181:0/1039964345 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c37fe700 1 --2- 172.21.15.181:0/1039964345 >> v2:172.21.15.190:3300/0 conn(0x7f32c40cb0c0 0x7f32c415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c27fc700 1 -- 172.21.15.181:0/1039964345 >> v2:172.21.15.190:3300/0 conn(0x7f32c40cb0c0 msgr2=0x7f32c415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c27fc700 1 --2- 172.21.15.181:0/1039964345 >> v2:172.21.15.190:3300/0 conn(0x7f32c40cb0c0 0x7f32c415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c27fc700 1 -- 172.21.15.181:0/1039964345 >> v1:172.21.15.181:6789/0 conn(0x7f32c415e6d0 legacy=0x7f32c415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:30.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c1ffb700 1 -- 172.21.15.181:0/1039964345 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501758425 0 0) 0x7f32c406ed00 con 0x7f32c415e6d0 2023-10-16T11:17:30.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c27fc700 1 -- 172.21.15.181:0/1039964345 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c406e0f0 con 0x7f32c415eab0 2023-10-16T11:17:30.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c27fc700 1 --2- 172.21.15.181:0/1039964345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415eab0 0x7f32c40d0fc0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f32c4071650 tx=0x7f32b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f396a2b07ed652b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:30.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c1ffb700 1 -- 172.21.15.181:0/1039964345 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400f8a0 con 0x7f32c415eab0 2023-10-16T11:17:30.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c1ffb700 1 -- 172.21.15.181:0/1039964345 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b400fa00 con 0x7f32c415eab0 2023-10-16T11:17:30.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.192+0000 7f32c1ffb700 1 -- 172.21.15.181:0/1039964345 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400fd90 con 0x7f32c415eab0 2023-10-16T11:17:30.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 -- 172.21.15.181:0/1039964345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415eab0 msgr2=0x7f32c40d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:30.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 --2- 172.21.15.181:0/1039964345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415eab0 0x7f32c40d0fc0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f32c4071650 tx=0x7f32b4009b20 comp rx=0 tx=0).stop 2023-10-16T11:17:30.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 -- 172.21.15.181:0/1039964345 shutdown_connections 2023-10-16T11:17:30.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 --2- 172.21.15.181:0/1039964345 >> v2:172.21.15.190:3300/0 conn(0x7f32c40cb0c0 0x7f32c415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 --2- 172.21.15.181:0/1039964345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415eab0 0x7f32c40d0fc0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 -- 172.21.15.181:0/1039964345 >> 172.21.15.181:0/1039964345 conn(0x7f32c40d04a0 msgr2=0x7f32c40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:30.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 -- 172.21.15.181:0/1039964345 shutdown_connections 2023-10-16T11:17:30.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.193+0000 7f32c97e0700 1 -- 172.21.15.181:0/1039964345 wait complete. 2023-10-16T11:17:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.194+0000 7f32c97e0700 1 Processor -- start 2023-10-16T11:17:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.194+0000 7f32c97e0700 1 -- start start 2023-10-16T11:17:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c97e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c40cb0c0 0x7f32c4157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:30.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c97e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415e6d0 0x7f32c40b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:30.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c97e0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c415eab0 0x7f32c4156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:30.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c97e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40b41d0 con 0x7f32c415e6d0 2023-10-16T11:17:30.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c97e0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40b61f0 con 0x7f32c415eab0 2023-10-16T11:17:30.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c97e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40b5c30 con 0x7f32c40cb0c0 2023-10-16T11:17:30.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415e6d0 0x7f32c40b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:30.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415e6d0 0x7f32c40b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53636/0 (socket says 172.21.15.181:53636) 2023-10-16T11:17:30.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.195+0000 7f32c27fc700 1 -- 172.21.15.181:0/3668451252 learned_addr learned my addr 172.21.15.181:0/3668451252 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:30.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c2ffd700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c40cb0c0 0x7f32c4157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:30.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c37fe700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c415eab0 0x7f32c4156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:30.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c27fc700 1 -- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c40cb0c0 msgr2=0x7f32c4157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c27fc700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c40cb0c0 0x7f32c4157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c27fc700 1 -- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c415eab0 msgr2=0x7f32c4156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c27fc700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c415eab0 0x7f32c4156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c27fc700 1 -- 172.21.15.181:0/3668451252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40094a0 con 0x7f32c415e6d0 2023-10-16T11:17:30.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c37fe700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c415eab0 0x7f32c4156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c27fc700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415e6d0 0x7f32c40b3c90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f32b40090b0 tx=0x7f32b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32ab7fe700 1 -- 172.21.15.181:0/3668451252 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b401e070 con 0x7f32c415e6d0 2023-10-16T11:17:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32ab7fe700 1 -- 172.21.15.181:0/3668451252 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b4004520 con 0x7f32c415e6d0 2023-10-16T11:17:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c40b5f10 con 0x7f32c415e6d0 2023-10-16T11:17:30.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32ab7fe700 1 -- 172.21.15.181:0/3668451252 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40038c0 con 0x7f32c415e6d0 2023-10-16T11:17:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.196+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c40b3770 con 0x7f32c415e6d0 2023-10-16T11:17:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.197+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c40b32f0 con 0x7f32c415e6d0 2023-10-16T11:17:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.198+0000 7f32ab7fe700 1 -- 172.21.15.181:0/3668451252 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f32b4022070 con 0x7f32c415e6d0 2023-10-16T11:17:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.198+0000 7f32ab7fe700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f32b006b020 0x7f32b006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:30.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.198+0000 7f32c2ffd700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f32b006b020 0x7f32b006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.198+0000 7f32ab7fe700 1 -- 172.21.15.181:0/3668451252 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f32b408c390 con 0x7f32c415e6d0 2023-10-16T11:17:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.199+0000 7f32c2ffd700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f32b006b020 0x7f32b006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f32b8007fd0 tx=0x7f32b8009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:30.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.203+0000 7f32ab7fe700 1 -- 172.21.15.181:0/3668451252 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f32c40b32f0 con 0x7f32c415e6d0 2023-10-16T11:17:30.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.475+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f32c40b32f0 con 0x7f32c415e6d0 2023-10-16T11:17:30.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:30.997+0000 7f32ab7fe700 1 -- 172.21.15.181:0/3668451252 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v3025) v1 ==== 165+0+0 (secure 0 0 0) 0x7f32c40b32f0 con 0x7f32c415e6d0 2023-10-16T11:17:31.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f32b006b020 msgr2=0x7f32b006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f32b006b020 0x7f32b006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f32b8007fd0 tx=0x7f32b8009040 comp rx=0 tx=0).stop 2023-10-16T11:17:31.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415e6d0 msgr2=0x7f32c40b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415e6d0 0x7f32c40b3c90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f32b40090b0 tx=0x7f32b4004750 comp rx=0 tx=0).stop 2023-10-16T11:17:31.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 shutdown_connections 2023-10-16T11:17:31.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f32b006b020 0x7f32b006d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32c40cb0c0 0x7f32c4157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c415eab0 0x7f32c4156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 --2- 172.21.15.181:0/3668451252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32c415e6d0 0x7f32c40b3c90 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 >> 172.21.15.181:0/3668451252 conn(0x7f32c40d04a0 msgr2=0x7f32c40adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:31.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.000+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 shutdown_connections 2023-10-16T11:17:31.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.001+0000 7f32c97e0700 1 -- 172.21.15.181:0/3668451252 wait complete. 2023-10-16T11:17:31.016 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-16T11:17:31.017 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-16T11:17:31.017 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-16T11:17:31.018 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:31.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.144+0000 7f3482ae6700 1 Processor -- start 2023-10-16T11:17:31.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.144+0000 7f3482ae6700 1 -- start start 2023-10-16T11:17:31.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f3482ae6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f347c0af450 0x7f347c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f3482ae6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f3482ae6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c06f870 con 0x7f347c15dfd0 2023-10-16T11:17:31.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f3482ae6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c0d1160 con 0x7f347c0af450 2023-10-16T11:17:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f3482ae6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0cc8e0 con 0x7f347c0cd470 2023-10-16T11:17:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f347c0af450 0x7f347c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f347c0af450 0x7f347c0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53644/0 (socket says 172.21.15.181:53644) 2023-10-16T11:17:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 -- 172.21.15.181:0/1309682959 learned_addr learned my addr 172.21.15.181:0/1309682959 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:31.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f3481083700 1 --2- 172.21.15.181:0/1309682959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 -- 172.21.15.181:0/1309682959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 msgr2=0x7f347c0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 --2- 172.21.15.181:0/1309682959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 -- 172.21.15.181:0/1309682959 >> v1:172.21.15.190:6789/0 conn(0x7f347c15dfd0 legacy=0x7f347c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 -- 172.21.15.181:0/1309682959 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0ccbc0 con 0x7f347c0af450 2023-10-16T11:17:31.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347bfff700 1 --2- 172.21.15.181:0/1309682959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c0af450 0x7f347c0af9a0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f34680098f0 tx=0x7f3468009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f990086ed7f9e16 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347b7fe700 1 -- 172.21.15.181:0/1309682959 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800b8a0 con 0x7f347c0af450 2023-10-16T11:17:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347b7fe700 1 -- 172.21.15.181:0/1309682959 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346800ba00 con 0x7f347c0af450 2023-10-16T11:17:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.145+0000 7f347b7fe700 1 -- 172.21.15.181:0/1309682959 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800bd90 con 0x7f347c0af450 2023-10-16T11:17:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 -- 172.21.15.181:0/1309682959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c0af450 msgr2=0x7f347c0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1309682959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c0af450 0x7f347c0af9a0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f34680098f0 tx=0x7f3468009c00 comp rx=0 tx=0).stop 2023-10-16T11:17:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 -- 172.21.15.181:0/1309682959 shutdown_connections 2023-10-16T11:17:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1309682959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1309682959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c0af450 0x7f347c0af9a0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 -- 172.21.15.181:0/1309682959 >> 172.21.15.181:0/1309682959 conn(0x7f347c0d04a0 msgr2=0x7f347c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:31.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 -- 172.21.15.181:0/1309682959 shutdown_connections 2023-10-16T11:17:31.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.146+0000 7f3482ae6700 1 -- 172.21.15.181:0/1309682959 wait complete. 2023-10-16T11:17:31.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.147+0000 7f3482ae6700 1 Processor -- start 2023-10-16T11:17:31.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.147+0000 7f3482ae6700 1 -- start start 2023-10-16T11:17:31.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3482ae6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f347c0af450 0x7f347c1f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3482ae6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3482ae6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c15dfd0 0x7f347c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3482ae6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c1544e0 con 0x7f347c15dfd0 2023-10-16T11:17:31.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3482ae6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c159440 con 0x7f347c0af450 2023-10-16T11:17:31.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3482ae6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c159690 con 0x7f347c0cd470 2023-10-16T11:17:31.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3481083700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c15dfd0 0x7f347c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3481083700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c15dfd0 0x7f347c0b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53652/0 (socket says 172.21.15.181:53652) 2023-10-16T11:17:31.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3481083700 1 -- 172.21.15.181:0/1401203602 learned_addr learned my addr 172.21.15.181:0/1401203602 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:31.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f347bfff700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3480882700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f347c0af450 0x7f347c1f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.148+0000 7f3481083700 1 -- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 msgr2=0x7f347c0c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f3481083700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f3481083700 1 -- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f347c0af450 msgr2=0x7f347c1f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f3481083700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f347c0af450 0x7f347c1f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f3481083700 1 -- 172.21.15.181:0/1401203602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3468009550 con 0x7f347c15dfd0 2023-10-16T11:17:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f3481083700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c15dfd0 0x7f347c0b54c0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f3474007e40 tx=0x7f347400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f34797fa700 1 -- 172.21.15.181:0/1401203602 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400ed30 con 0x7f347c15dfd0 2023-10-16T11:17:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f34797fa700 1 -- 172.21.15.181:0/1401203602 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3474014420 con 0x7f347c15dfd0 2023-10-16T11:17:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c0b2920 con 0x7f347c15dfd0 2023-10-16T11:17:31.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f34797fa700 1 -- 172.21.15.181:0/1401203602 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34740147d0 con 0x7f347c15dfd0 2023-10-16T11:17:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.149+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c0c6ce0 con 0x7f347c15dfd0 2023-10-16T11:17:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.150+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c0b2e40 con 0x7f347c15dfd0 2023-10-16T11:17:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.150+0000 7f34797fa700 1 -- 172.21.15.181:0/1401203602 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3474014930 con 0x7f347c15dfd0 2023-10-16T11:17:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.151+0000 7f34797fa700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f346006b020 0x7f346006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.151+0000 7f34797fa700 1 -- 172.21.15.181:0/1401203602 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f347408a090 con 0x7f347c15dfd0 2023-10-16T11:17:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.151+0000 7f3480882700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f346006b020 0x7f346006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.151+0000 7f3480882700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f346006b020 0x7f346006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3470000fb0 tx=0x7f3470009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.155+0000 7f34797fa700 1 -- 172.21.15.181:0/1401203602 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f347c0b2e40 con 0x7f347c15dfd0 2023-10-16T11:17:31.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.347+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f347c0b2e40 con 0x7f347c15dfd0 2023-10-16T11:17:31.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.348+0000 7f34797fa700 1 -- 172.21.15.181:0/1401203602 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f347c0b2e40 con 0x7f347c15dfd0 2023-10-16T11:17:31.349 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:31.349 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":190,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:28.993705+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6834/2135938520":"2023-10-17T11:17:25.948740+0000","172.21.15.181:6833/2958900944":"2023-10-17T11:17:24.967015+0000","172.21.15.181:6832/2958900944":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/2426778736":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/2027727173":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/52227936":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/1720809791":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/2805455393":"2023-10-17T11:17:24.967015+0000","172.21.15.181:0/4003028162":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/3290548668":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/3559844799":"2023-10-17T11:17:24.967015+0000","172.21.15.181:6833/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:6832/3601081788":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/2238590139":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/231495737":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/1499024696":"2023-10-17T11:12:21.628206+0000","172.21.15.190:6832/2135938520":"2023-10-17T11:17:25.948740+0000","172.21.15.181:0/458160361":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1195985489":"2023-10-17T11:12:21.628206+0000","172.21.15.181:0/4000110033":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/990187329":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3304094772":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6833/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/1162194396":"2023-10-17T11:01:37.009118+0000","172.21.15.181:6832/303240986":"2023-10-17T11:01:37.009118+0000","172.21.15.181:0/3436855298":"2023-10-17T11:01:37.009118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:31.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f346006b020 msgr2=0x7f346006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f346006b020 0x7f346006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3470000fb0 tx=0x7f3470009040 comp rx=0 tx=0).stop 2023-10-16T11:17:31.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c15dfd0 msgr2=0x7f347c0b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c15dfd0 0x7f347c0b54c0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f3474007e40 tx=0x7f347400e3c0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 shutdown_connections 2023-10-16T11:17:31.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f346006b020 0x7f346006d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f347c15dfd0 0x7f347c0b54c0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f347c0af450 0x7f347c1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 --2- 172.21.15.181:0/1401203602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f347c0cd470 0x7f347c0c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 >> 172.21.15.181:0/1401203602 conn(0x7f347c0d04a0 msgr2=0x7f347c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:31.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 shutdown_connections 2023-10-16T11:17:31.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.350+0000 7f3482ae6700 1 -- 172.21.15.181:0/1401203602 wait complete. 2023-10-16T11:17:31.363 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:17:31.364 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:17:31.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.486+0000 7f8b38d5e700 1 Processor -- start 2023-10-16T11:17:31.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.486+0000 7f8b38d5e700 1 -- start start 2023-10-16T11:17:31.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.487+0000 7f8b38d5e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b340b2010 0x7f8b340cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.487+0000 7f8b38d5e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340ce1f0 con 0x7f8b34071be0 2023-10-16T11:17:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.487+0000 7f8b38d5e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b34072090 con 0x7f8b340ce680 2023-10-16T11:17:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.487+0000 7f8b38d5e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340ccda0 con 0x7f8b340b2010 2023-10-16T11:17:31.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.487+0000 7f8b31d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8b34071be0 0x7f8b340b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36504/0 (socket says 172.21.15.181:36504) 2023-10-16T11:17:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.487+0000 7f8b31d9b700 1 -- 172.21.15.181:0/2036504889 learned_addr learned my addr 172.21.15.181:0/2036504889 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.487+0000 7f8b3259c700 1 --2- 172.21.15.181:0/2036504889 >> v2:172.21.15.190:3300/0 conn(0x7f8b340b2010 0x7f8b340cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3159a700 1 -- 172.21.15.181:0/2036504889 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041179883 0 0) 0x7f8b340ce1f0 con 0x7f8b34071be0 2023-10-16T11:17:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3159a700 1 -- 172.21.15.181:0/2036504889 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1c0035b0 con 0x7f8b34071be0 2023-10-16T11:17:31.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3259c700 1 -- 172.21.15.181:0/2036504889 >> v1:172.21.15.190:6789/0 conn(0x7f8b340ce680 legacy=0x7f8b340ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3259c700 1 -- 172.21.15.181:0/2036504889 >> v1:172.21.15.181:6789/0 conn(0x7f8b34071be0 legacy=0x7f8b340b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3259c700 1 -- 172.21.15.181:0/2036504889 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340727e0 con 0x7f8b340b2010 2023-10-16T11:17:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3259c700 1 --2- 172.21.15.181:0/2036504889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340b2010 0x7f8b340cc6e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009840 tx=0x7f8b24009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c486f5bddf9c7c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3159a700 1 -- 172.21.15.181:0/2036504889 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24011830 con 0x7f8b340b2010 2023-10-16T11:17:31.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3159a700 1 -- 172.21.15.181:0/2036504889 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b24011990 con 0x7f8b340b2010 2023-10-16T11:17:31.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.488+0000 7f8b3159a700 1 -- 172.21.15.181:0/2036504889 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24011d20 con 0x7f8b340b2010 2023-10-16T11:17:31.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.489+0000 7f8b38d5e700 1 -- 172.21.15.181:0/2036504889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340b2010 msgr2=0x7f8b340cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.489+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/2036504889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340b2010 0x7f8b340cc6e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009840 tx=0x7f8b24009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:31.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.489+0000 7f8b38d5e700 1 -- 172.21.15.181:0/2036504889 shutdown_connections 2023-10-16T11:17:31.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.489+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/2036504889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340b2010 0x7f8b340cc6e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.489+0000 7f8b38d5e700 1 -- 172.21.15.181:0/2036504889 >> 172.21.15.181:0/2036504889 conn(0x7f8b340cd010 msgr2=0x7f8b340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:31.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.489+0000 7f8b38d5e700 1 -- 172.21.15.181:0/2036504889 shutdown_connections 2023-10-16T11:17:31.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.489+0000 7f8b38d5e700 1 -- 172.21.15.181:0/2036504889 wait complete. 2023-10-16T11:17:31.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.490+0000 7f8b38d5e700 1 Processor -- start 2023-10-16T11:17:31.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b38d5e700 1 -- start start 2023-10-16T11:17:31.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b38d5e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b34071be0 0x7f8b341f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b38d5e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b340b2010 0x7f8b341f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b38d5e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340ce680 0x7f8b34147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b38d5e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b341f1fd0 con 0x7f8b34071be0 2023-10-16T11:17:31.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b38d5e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b341f5510 con 0x7f8b340ce680 2023-10-16T11:17:31.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b38d5e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406ab90 con 0x7f8b340b2010 2023-10-16T11:17:31.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.491+0000 7f8b3259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b34071be0 0x7f8b341f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b34071be0 0x7f8b341f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53662/0 (socket says 172.21.15.181:53662) 2023-10-16T11:17:31.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 -- 172.21.15.181:0/3206078916 learned_addr learned my addr 172.21.15.181:0/3206078916 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:31.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b32d9d700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340ce680 0x7f8b34147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b31d9b700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b340b2010 0x7f8b341f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 -- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b340b2010 msgr2=0x7f8b341f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b340b2010 0x7f8b341f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 -- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340ce680 msgr2=0x7f8b34147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340ce680 0x7f8b34147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 -- 172.21.15.181:0/3206078916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b240094a0 con 0x7f8b34071be0 2023-10-16T11:17:31.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b31d9b700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b340b2010 0x7f8b341f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:31.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b3259c700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b34071be0 0x7f8b341f1a90 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009b90 tx=0x7f8b24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b22ffd700 1 -- 172.21.15.181:0/3206078916 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240038a0 con 0x7f8b34071be0 2023-10-16T11:17:31.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b22ffd700 1 -- 172.21.15.181:0/3206078916 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2401e070 con 0x7f8b34071be0 2023-10-16T11:17:31.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3406ae70 con 0x7f8b34071be0 2023-10-16T11:17:31.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b22ffd700 1 -- 172.21.15.181:0/3206078916 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24003c40 con 0x7f8b34071be0 2023-10-16T11:17:31.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.492+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b341f1570 con 0x7f8b34071be0 2023-10-16T11:17:31.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.493+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b340b6510 con 0x7f8b34071be0 2023-10-16T11:17:31.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.494+0000 7f8b22ffd700 1 -- 172.21.15.181:0/3206078916 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8b2402d450 con 0x7f8b34071be0 2023-10-16T11:17:31.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.494+0000 7f8b22ffd700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f8b1406b020 0x7f8b1406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.494+0000 7f8b22ffd700 1 -- 172.21.15.181:0/3206078916 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f8b2408aa10 con 0x7f8b34071be0 2023-10-16T11:17:31.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.494+0000 7f8b31d9b700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f8b1406b020 0x7f8b1406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.494+0000 7f8b31d9b700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f8b1406b020 0x7f8b1406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8b34158c10 tx=0x7f8b28009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.498+0000 7f8b22ffd700 1 -- 172.21.15.181:0/3206078916 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8b24055720 con 0x7f8b34071be0 2023-10-16T11:17:31.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.688+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b340d0780 con 0x7f8b34071be0 2023-10-16T11:17:31.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.688+0000 7f8b22ffd700 1 -- 172.21.15.181:0/3206078916 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b240150a0 con 0x7f8b34071be0 2023-10-16T11:17:31.690 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:17:31.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.690+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f8b1406b020 msgr2=0x7f8b1406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.690+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f8b1406b020 0x7f8b1406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8b34158c10 tx=0x7f8b28009040 comp rx=0 tx=0).stop 2023-10-16T11:17:31.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.690+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b34071be0 msgr2=0x7f8b341f1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.690+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b34071be0 0x7f8b341f1a90 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009b90 tx=0x7f8b24004750 comp rx=0 tx=0).stop 2023-10-16T11:17:31.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.690+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 shutdown_connections 2023-10-16T11:17:31.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.691+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f8b1406b020 0x7f8b1406d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.691+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b340ce680 0x7f8b34147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.691+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b34071be0 0x7f8b341f1a90 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.691+0000 7f8b38d5e700 1 --2- 172.21.15.181:0/3206078916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b340b2010 0x7f8b341f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.691+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 >> 172.21.15.181:0/3206078916 conn(0x7f8b340cd010 msgr2=0x7f8b340cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:31.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.691+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 shutdown_connections 2023-10-16T11:17:31.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.691+0000 7f8b38d5e700 1 -- 172.21.15.181:0/3206078916 wait complete. 2023-10-16T11:17:31.704 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-16T11:17:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.823+0000 7f3a08caa700 1 Processor -- start 2023-10-16T11:17:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.823+0000 7f3a08caa700 1 -- start start 2023-10-16T11:17:31.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.824+0000 7f3a08caa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a040ce7f0 0x7f3a0406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.824+0000 7f3a08caa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3a040d2390 0x7f3a040cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.824+0000 7f3a08caa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a040cc470 con 0x7f3a040d0180 2023-10-16T11:17:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.824+0000 7f3a08caa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a040d05f0 con 0x7f3a040ce7f0 2023-10-16T11:17:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.824+0000 7f3a08caa700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0406ca30 con 0x7f3a040d2390 2023-10-16T11:17:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.824+0000 7f3a0259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a040ce7f0 0x7f3a0406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a040ce7f0 0x7f3a0406c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53670/0 (socket says 172.21.15.181:53670) 2023-10-16T11:17:31.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0259c700 1 -- 172.21.15.181:0/3939362737 learned_addr learned my addr 172.21.15.181:0/3939362737 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:31.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a01d9b700 1 --2- 172.21.15.181:0/3939362737 >> v2:172.21.15.190:3300/0 conn(0x7f3a040d2390 0x7f3a040cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0259c700 1 -- 172.21.15.181:0/3939362737 >> v2:172.21.15.190:3300/0 conn(0x7f3a040d2390 msgr2=0x7f3a040cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:31.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0259c700 1 --2- 172.21.15.181:0/3939362737 >> v2:172.21.15.190:3300/0 conn(0x7f3a040d2390 0x7f3a040cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0259c700 1 -- 172.21.15.181:0/3939362737 >> v1:172.21.15.190:6789/0 conn(0x7f3a040d0180 legacy=0x7f3a04161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:31.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0259c700 1 -- 172.21.15.181:0/3939362737 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0406cd10 con 0x7f3a040ce7f0 2023-10-16T11:17:31.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a01d9b700 1 --2- 172.21.15.181:0/3939362737 >> v2:172.21.15.190:3300/0 conn(0x7f3a040d2390 0x7f3a040cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:31.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0259c700 1 --2- 172.21.15.181:0/3939362737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040ce7f0 0x7f3a0406c100 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cc1e0 tx=0x7f39ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3aa4e539552241e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0159a700 1 -- 172.21.15.181:0/3939362737 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec005e30 con 0x7f3a040ce7f0 2023-10-16T11:17:31.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0159a700 1 -- 172.21.15.181:0/3939362737 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec006030 con 0x7f3a040ce7f0 2023-10-16T11:17:31.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.825+0000 7f3a0159a700 1 -- 172.21.15.181:0/3939362737 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0063e0 con 0x7f3a040ce7f0 2023-10-16T11:17:31.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 -- 172.21.15.181:0/3939362737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040ce7f0 msgr2=0x7f3a0406c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3939362737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040ce7f0 0x7f3a0406c100 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cc1e0 tx=0x7f39ec009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 -- 172.21.15.181:0/3939362737 shutdown_connections 2023-10-16T11:17:31.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3939362737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040ce7f0 0x7f3a0406c100 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3939362737 >> v2:172.21.15.190:3300/0 conn(0x7f3a040d2390 0x7f3a040cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 -- 172.21.15.181:0/3939362737 >> 172.21.15.181:0/3939362737 conn(0x7f3a0409a9a0 msgr2=0x7f3a040d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:31.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 -- 172.21.15.181:0/3939362737 shutdown_connections 2023-10-16T11:17:31.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.826+0000 7f3a08caa700 1 -- 172.21.15.181:0/3939362737 wait complete. 2023-10-16T11:17:31.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.827+0000 7f3a08caa700 1 Processor -- start 2023-10-16T11:17:31.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.827+0000 7f3a08caa700 1 -- start start 2023-10-16T11:17:31.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a08caa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040ce7f0 0x7f3a0415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a08caa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040d0180 0x7f3a040c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a08caa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a040d2390 0x7f3a040b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a08caa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040b5920 con 0x7f3a040d0180 2023-10-16T11:17:31.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a08caa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0415b280 con 0x7f3a040ce7f0 2023-10-16T11:17:31.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a08caa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040b4d50 con 0x7f3a040d2390 2023-10-16T11:17:31.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040d0180 0x7f3a040c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040d0180 0x7f3a040c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53678/0 (socket says 172.21.15.181:53678) 2023-10-16T11:17:31.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 -- 172.21.15.181:0/3913433333 learned_addr learned my addr 172.21.15.181:0/3913433333 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:31.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a0259c700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040ce7f0 0x7f3a0415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a02d9d700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a040d2390 0x7f3a040b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 -- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a040d2390 msgr2=0x7f3a040b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a040d2390 0x7f3a040b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 -- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040ce7f0 msgr2=0x7f3a0415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:31.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040ce7f0 0x7f3a0415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:31.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 -- 172.21.15.181:0/3913433333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec009500 con 0x7f3a040d0180 2023-10-16T11:17:31.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a0259c700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040ce7f0 0x7f3a0415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:31.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.828+0000 7f3a01d9b700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040d0180 0x7f3a040c79d0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c750 tx=0x7f39f400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.829+0000 7f39faffd700 1 -- 172.21.15.181:0/3913433333 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4011800 con 0x7f3a040d0180 2023-10-16T11:17:31.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.829+0000 7f39faffd700 1 -- 172.21.15.181:0/3913433333 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f4011960 con 0x7f3a040d0180 2023-10-16T11:17:31.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.829+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a040b4ff0 con 0x7f3a040d0180 2023-10-16T11:17:31.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.829+0000 7f39faffd700 1 -- 172.21.15.181:0/3913433333 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4011cd0 con 0x7f3a040d0180 2023-10-16T11:17:31.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.829+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a040b55b0 con 0x7f3a040d0180 2023-10-16T11:17:31.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.829+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a04067d20 con 0x7f3a040d0180 2023-10-16T11:17:31.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.830+0000 7f39faffd700 1 -- 172.21.15.181:0/3913433333 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f39f4021350 con 0x7f3a040d0180 2023-10-16T11:17:31.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.830+0000 7f39faffd700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f39e406b020 0x7f39e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:31.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.831+0000 7f39faffd700 1 -- 172.21.15.181:0/3913433333 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f39f4089b20 con 0x7f3a040d0180 2023-10-16T11:17:31.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.831+0000 7f3a0259c700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f39e406b020 0x7f39e406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:31.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.831+0000 7f3a0259c700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f39e406b020 0x7f39e406d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cc1e0 tx=0x7f39ec008070 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:31.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:31.835+0000 7f39faffd700 1 -- 172.21.15.181:0/3913433333 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f39f4054690 con 0x7f3a040d0180 2023-10-16T11:17:32.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:32.108+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f3a040cef40 con 0x7f3a040d0180 2023-10-16T11:17:33.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.003+0000 7f39faffd700 1 -- 172.21.15.181:0/3913433333 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3027) v1 ==== 166+0+0 (secure 0 0 0) 0x7f39f401e0e0 con 0x7f3a040d0180 2023-10-16T11:17:33.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f39e406b020 msgr2=0x7f39e406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f39e406b020 0x7f39e406d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cc1e0 tx=0x7f39ec008070 comp rx=0 tx=0).stop 2023-10-16T11:17:33.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040d0180 msgr2=0x7f3a040c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040d0180 0x7f3a040c79d0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c750 tx=0x7f39f400cb10 comp rx=0 tx=0).stop 2023-10-16T11:17:33.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 shutdown_connections 2023-10-16T11:17:33.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f39e406b020 0x7f39e406d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a040d2390 0x7f3a040b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040ce7f0 0x7f3a0415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 --2- 172.21.15.181:0/3913433333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a040d0180 0x7f3a040c79d0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 >> 172.21.15.181:0/3913433333 conn(0x7f3a0409a9a0 msgr2=0x7f3a040c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:33.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 shutdown_connections 2023-10-16T11:17:33.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.007+0000 7f3a08caa700 1 -- 172.21.15.181:0/3913433333 wait complete. 2023-10-16T11:17:33.024 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:17:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.155+0000 7f3537547700 1 Processor -- start 2023-10-16T11:17:33.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.155+0000 7f3537547700 1 -- start start 2023-10-16T11:17:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3537547700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f353015da70 0x7f35300ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3537547700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35300ce6a0 0x7f353006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3537547700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35300cef00 con 0x7f3530072760 2023-10-16T11:17:33.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3537547700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3530072b40 con 0x7f35300ce6a0 2023-10-16T11:17:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3537547700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35300cec20 con 0x7f353015da70 2023-10-16T11:17:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3534ae2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35300ce6a0 0x7f353006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3535ae4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3530072760 0x7f35301615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36534/0 (socket says 172.21.15.181:36534) 2023-10-16T11:17:33.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3534ae2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35300ce6a0 0x7f353006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53686/0 (socket says 172.21.15.181:53686) 2023-10-16T11:17:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f3535ae4700 1 -- 172.21.15.181:0/3924721501 learned_addr learned my addr 172.21.15.181:0/3924721501 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.156+0000 7f35352e3700 1 --2- 172.21.15.181:0/3924721501 >> v2:172.21.15.190:3300/0 conn(0x7f353015da70 0x7f35300ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3534ae2700 1 -- 172.21.15.181:0/3924721501 >> v2:172.21.15.190:3300/0 conn(0x7f353015da70 msgr2=0x7f35300ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3534ae2700 1 --2- 172.21.15.181:0/3924721501 >> v2:172.21.15.190:3300/0 conn(0x7f353015da70 0x7f35300ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3534ae2700 1 -- 172.21.15.181:0/3924721501 >> v1:172.21.15.181:6789/0 conn(0x7f3530072760 legacy=0x7f35301615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3527fff700 1 -- 172.21.15.181:0/3924721501 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095825330 0 0) 0x7f35300cef00 con 0x7f3530072760 2023-10-16T11:17:33.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3534ae2700 1 -- 172.21.15.181:0/3924721501 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3530071c80 con 0x7f35300ce6a0 2023-10-16T11:17:33.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f35352e3700 1 --2- 172.21.15.181:0/3924721501 >> v2:172.21.15.190:3300/0 conn(0x7f353015da70 0x7f35300ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:33.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3534ae2700 1 --2- 172.21.15.181:0/3924721501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35300ce6a0 0x7f353006a8a0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f35300675e0 tx=0x7f352c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3125a28df7105cc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:33.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3527fff700 1 -- 172.21.15.181:0/3924721501 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c011830 con 0x7f35300ce6a0 2023-10-16T11:17:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3527fff700 1 -- 172.21.15.181:0/3924721501 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c011990 con 0x7f35300ce6a0 2023-10-16T11:17:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.157+0000 7f3527fff700 1 -- 172.21.15.181:0/3924721501 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c011d20 con 0x7f35300ce6a0 2023-10-16T11:17:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 -- 172.21.15.181:0/3924721501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35300ce6a0 msgr2=0x7f353006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 --2- 172.21.15.181:0/3924721501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35300ce6a0 0x7f353006a8a0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f35300675e0 tx=0x7f352c009b20 comp rx=0 tx=0).stop 2023-10-16T11:17:33.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 -- 172.21.15.181:0/3924721501 shutdown_connections 2023-10-16T11:17:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 --2- 172.21.15.181:0/3924721501 >> v2:172.21.15.190:3300/0 conn(0x7f353015da70 0x7f35300ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 --2- 172.21.15.181:0/3924721501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35300ce6a0 0x7f353006a8a0 secure :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f35300675e0 tx=0x7f352c009b20 comp rx=0 tx=0).stop 2023-10-16T11:17:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 -- 172.21.15.181:0/3924721501 >> 172.21.15.181:0/3924721501 conn(0x7f35300cd010 msgr2=0x7f35300cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 -- 172.21.15.181:0/3924721501 shutdown_connections 2023-10-16T11:17:33.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.158+0000 7f3537547700 1 -- 172.21.15.181:0/3924721501 wait complete. 2023-10-16T11:17:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.159+0000 7f3537547700 1 Processor -- start 2023-10-16T11:17:33.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.159+0000 7f3537547700 1 -- start start 2023-10-16T11:17:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3537547700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3530072760 0x7f35301f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3537547700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f353015da70 0x7f35301f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3537547700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530145300 0x7f35301f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3537547700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300c6b80 con 0x7f3530145300 2023-10-16T11:17:33.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3537547700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300ca540 con 0x7f3530072760 2023-10-16T11:17:33.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3537547700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3530160990 con 0x7f353015da70 2023-10-16T11:17:33.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530145300 0x7f35301f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530145300 0x7f35301f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53694/0 (socket says 172.21.15.181:53694) 2023-10-16T11:17:33.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 -- 172.21.15.181:0/1621881635 learned_addr learned my addr 172.21.15.181:0/1621881635 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:33.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f35352e3700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3530072760 0x7f35301f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3534ae2700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f353015da70 0x7f35301f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 -- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f353015da70 msgr2=0x7f35301f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f353015da70 0x7f35301f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 -- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3530072760 msgr2=0x7f35301f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3530072760 0x7f35301f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 -- 172.21.15.181:0/1621881635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0094a0 con 0x7f3530145300 2023-10-16T11:17:33.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.160+0000 7f3535ae4700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530145300 0x7f35301f7690 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f352800cca0 tx=0x7f352800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:33.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.161+0000 7f3525ffb700 1 -- 172.21.15.181:0/1621881635 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800cea0 con 0x7f3530145300 2023-10-16T11:17:33.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.161+0000 7f3525ffb700 1 -- 172.21.15.181:0/1621881635 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352800ade0 con 0x7f3530145300 2023-10-16T11:17:33.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.161+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3530160c40 con 0x7f3530145300 2023-10-16T11:17:33.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.161+0000 7f3525ffb700 1 -- 172.21.15.181:0/1621881635 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528007310 con 0x7f3530145300 2023-10-16T11:17:33.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.161+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35300c9a60 con 0x7f3530145300 2023-10-16T11:17:33.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.162+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35300af940 con 0x7f3530145300 2023-10-16T11:17:33.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.162+0000 7f3525ffb700 1 -- 172.21.15.181:0/1621881635 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3528007470 con 0x7f3530145300 2023-10-16T11:17:33.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.162+0000 7f3525ffb700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f351406b020 0x7f351406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.162+0000 7f3525ffb700 1 -- 172.21.15.181:0/1621881635 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f35280897d0 con 0x7f3530145300 2023-10-16T11:17:33.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.162+0000 7f35352e3700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f351406b020 0x7f351406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.163+0000 7f35352e3700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f351406b020 0x7f351406d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3520009930 tx=0x7f3520008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:33.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.167+0000 7f3525ffb700 1 -- 172.21.15.181:0/1621881635 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f35280543d0 con 0x7f3530145300 2023-10-16T11:17:33.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.361+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3530072210 con 0x7f3530145300 2023-10-16T11:17:33.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.364+0000 7f3525ffb700 1 -- 172.21.15.181:0/1621881635 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f35280579f0 con 0x7f3530145300 2023-10-16T11:17:33.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f351406b020 msgr2=0x7f351406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f351406b020 0x7f351406d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3520009930 tx=0x7f3520008040 comp rx=0 tx=0).stop 2023-10-16T11:17:33.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530145300 msgr2=0x7f35301f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530145300 0x7f35301f7690 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f352800cca0 tx=0x7f352800a340 comp rx=0 tx=0).stop 2023-10-16T11:17:33.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 shutdown_connections 2023-10-16T11:17:33.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f351406b020 0x7f351406d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530145300 0x7f35301f7690 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3530072760 0x7f35301f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 --2- 172.21.15.181:0/1621881635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f353015da70 0x7f35301f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 >> 172.21.15.181:0/1621881635 conn(0x7f35300cd010 msgr2=0x7f35300cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:33.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 shutdown_connections 2023-10-16T11:17:33.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.366+0000 7f3537547700 1 -- 172.21.15.181:0/1621881635 wait complete. 2023-10-16T11:17:33.379 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:17:33.380 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:17:33.428 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:17:33.479 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:17:33.479 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:17:33.479 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:33.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:17:33.497 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:17:33.497 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:33.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:17:33.517 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:17:33.518 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:33.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:17:33.554 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:17:33.554 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:33.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:17:33.576 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:17:33.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:17:33.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.713+0000 7f6850b1a700 1 Processor -- start 2023-10-16T11:17:33.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.713+0000 7f6850b1a700 1 -- start start 2023-10-16T11:17:33.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.714+0000 7f6850b1a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 0x7f684c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.714+0000 7f6850b1a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c06ed00 con 0x7f684c0cb0c0 2023-10-16T11:17:33.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.714+0000 7f6850b1a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0d02d0 con 0x7f684c15e6d0 2023-10-16T11:17:33.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.714+0000 7f6850b1a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c06e0f0 con 0x7f684c15eab0 2023-10-16T11:17:33.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.714+0000 7f684ad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f684c0cb0c0 0x7f684c15a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36548/0 (socket says 172.21.15.181:36548) 2023-10-16T11:17:33.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.714+0000 7f684ad9d700 1 -- 172.21.15.181:0/155438431 learned_addr learned my addr 172.21.15.181:0/155438431 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:33.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f6849d9b700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 0x7f684c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589179555 0 0) 0x7f684c06ed00 con 0x7f684c0cb0c0 2023-10-16T11:17:33.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68340035b0 con 0x7f684c0cb0c0 2023-10-16T11:17:33.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480117967 0 0) 0x7f684c0d02d0 con 0x7f684c15e6d0 2023-10-16T11:17:33.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f684c06ed00 con 0x7f684c15e6d0 2023-10-16T11:17:33.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3984355517 0 0) 0x7f68340035b0 con 0x7f684c0cb0c0 2023-10-16T11:17:33.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 msgr2=0x7f684c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:33.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 0x7f684c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 >> v1:172.21.15.190:6789/0 conn(0x7f684c15e6d0 legacy=0x7f684c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c06e3d0 con 0x7f684c0cb0c0 2023-10-16T11:17:33.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f68440019f0 con 0x7f684c0cb0c0 2023-10-16T11:17:33.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 >> v1:172.21.15.181:6789/0 conn(0x7f684c0cb0c0 legacy=0x7f684c15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6834003bc0 0x7f6834003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6834004850 0x7f6834004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 0x7f6834005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c06e0f0 con 0x7f6834003bc0 2023-10-16T11:17:33.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683400bac0 con 0x7f6834004850 2023-10-16T11:17:33.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683400bc50 con 0x7f684c15eab0 2023-10-16T11:17:33.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684a59c700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6834003bc0 0x7f6834003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f6849d9b700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6834004850 0x7f6834004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684ad9d700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 0x7f6834005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684a59c700 1 -- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 msgr2=0x7f6834005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:33.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684a59c700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f684c15eab0 0x7f6834005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684a59c700 1 -- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6834004850 msgr2=0x7f6834004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:33.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684a59c700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6834004850 0x7f6834004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.715+0000 7f684a59c700 1 -- 172.21.15.181:0/155438431 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68340035b0 con 0x7f6834003bc0 2023-10-16T11:17:33.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f684a59c700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6834003bc0 0x7f6834003fc0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f6840001580 tx=0x7f6834005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0d04c7b1361630 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:33.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400052d0 con 0x7f6834003bc0 2023-10-16T11:17:33.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840005430 con 0x7f6834003bc0 2023-10-16T11:17:33.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f684959a700 1 -- 172.21.15.181:0/155438431 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840003910 con 0x7f6834003bc0 2023-10-16T11:17:33.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f6850b1a700 1 -- 172.21.15.181:0/155438431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6834003bc0 msgr2=0x7f6834003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f6850b1a700 1 --2- 172.21.15.181:0/155438431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6834003bc0 0x7f6834003fc0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f6840001580 tx=0x7f6834005880 comp rx=0 tx=0).stop 2023-10-16T11:17:33.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f684ad9d700 1 -- 172.21.15.181:0/155438431 reap_dead start 2023-10-16T11:17:33.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f6850b1a700 1 -- 172.21.15.181:0/155438431 shutdown_connections 2023-10-16T11:17:33.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f6850b1a700 1 -- 172.21.15.181:0/155438431 >> 172.21.15.181:0/155438431 conn(0x7f684c0d04a0 msgr2=0x7f684c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:33.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f6850b1a700 1 -- 172.21.15.181:0/155438431 shutdown_connections 2023-10-16T11:17:33.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.716+0000 7f6850b1a700 1 -- 172.21.15.181:0/155438431 wait complete. 2023-10-16T11:17:33.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 Processor -- start 2023-10-16T11:17:33.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 -- start start 2023-10-16T11:17:33.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6834003bc0 0x7f684c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c14e940 0x7f684c15c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c14edc0 0x7f684c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0c47d0 con 0x7f684c14e940 2023-10-16T11:17:33.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c14e7e0 con 0x7f684c14edc0 2023-10-16T11:17:33.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6850b1a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0c4480 con 0x7f6834003bc0 2023-10-16T11:17:33.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.718+0000 7f6849d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c14e940 0x7f684c15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c14e940 0x7f684c15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53712/0 (socket says 172.21.15.181:53712) 2023-10-16T11:17:33.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 -- 172.21.15.181:0/2059906765 learned_addr learned my addr 172.21.15.181:0/2059906765 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:33.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f684a59c700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6834003bc0 0x7f684c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f684ad9d700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c14edc0 0x7f684c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 -- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6834003bc0 msgr2=0x7f684c14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6834003bc0 0x7f684c14e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 -- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c14edc0 msgr2=0x7f684c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c14edc0 0x7f684c15c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 -- 172.21.15.181:0/2059906765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c15cd00 con 0x7f684c14e940 2023-10-16T11:17:33.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f684ad9d700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c14edc0 0x7f684c15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:33.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f684a59c700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6834003bc0 0x7f684c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:33.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6849d9b700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c14e940 0x7f684c15c280 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f683c011aa0 tx=0x7f683c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:33.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f683affd700 1 -- 172.21.15.181:0/2059906765 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c011ee0 con 0x7f684c14e940 2023-10-16T11:17:33.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f683affd700 1 -- 172.21.15.181:0/2059906765 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c00f940 con 0x7f684c14e940 2023-10-16T11:17:33.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c153cc0 con 0x7f684c14e940 2023-10-16T11:17:33.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f683affd700 1 -- 172.21.15.181:0/2059906765 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c00fcf0 con 0x7f684c14e940 2023-10-16T11:17:33.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.719+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c154280 con 0x7f684c14e940 2023-10-16T11:17:33.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.720+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c067d20 con 0x7f684c14e940 2023-10-16T11:17:33.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.721+0000 7f683affd700 1 -- 172.21.15.181:0/2059906765 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f683c014440 con 0x7f684c14e940 2023-10-16T11:17:33.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.721+0000 7f683affd700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f682c06b020 0x7f682c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:33.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.721+0000 7f683affd700 1 -- 172.21.15.181:0/2059906765 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f683c08a060 con 0x7f684c14e940 2023-10-16T11:17:33.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.721+0000 7f684a59c700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f682c06b020 0x7f682c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:33.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.721+0000 7f684a59c700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f682c06b020 0x7f682c06d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6840005040 tx=0x7f6840020040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:33.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.725+0000 7f683affd700 1 -- 172.21.15.181:0/2059906765 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f683c054e30 con 0x7f684c14e940 2023-10-16T11:17:33.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.962+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f684c0b6320 con 0x7f684c14e940 2023-10-16T11:17:33.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.962+0000 7f683affd700 1 -- 172.21.15.181:0/2059906765 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3706 (secure 0 0 0) 0x7f683c058450 con 0x7f684c14e940 2023-10-16T11:17:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:33.964 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1903420066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1903420066},{"type":"v1","addr":"172.21.15.181:6836","nonce":1903420066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":8423,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/973512064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":973512064},{"type":"v1","addr":"172.21.15.190:6834","nonce":973512064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":9040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/586483327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":586483327},{"type":"v1","addr":"172.21.15.190:6835","nonce":586483327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10185,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2078148321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2078148321},{"type":"v1","addr":"172.21.15.181:6837","nonce":2078148321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[]} 2023-10-16T11:17:33.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f682c06b020 msgr2=0x7f682c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f682c06b020 0x7f682c06d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6840005040 tx=0x7f6840020040 comp rx=0 tx=0).stop 2023-10-16T11:17:33.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c14e940 msgr2=0x7f684c15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:33.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c14e940 0x7f684c15c280 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f683c011aa0 tx=0x7f683c011db0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 shutdown_connections 2023-10-16T11:17:33.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f682c06b020 0x7f682c06d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6834003bc0 0x7f684c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c14edc0 0x7f684c15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 --2- 172.21.15.181:0/2059906765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f684c14e940 0x7f684c15c280 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:33.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 >> 172.21.15.181:0/2059906765 conn(0x7f684c0d04a0 msgr2=0x7f684c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:33.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.964+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 shutdown_connections 2023-10-16T11:17:33.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:33.965+0000 7f6850b1a700 1 -- 172.21.15.181:0/2059906765 wait complete. 2023-10-16T11:17:33.968 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 96 2023-10-16T11:17:33.978 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:17:33.978 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:17:33.978 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:17:33.979 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:17:33.979 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:17:33.979 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:17:33.980 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:17:33.980 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:17:33.980 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:17:33.980 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:17:33.981 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:17:33.981 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:17:33.981 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:17:33.982 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:17:33.982 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:17:33.982 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:17:33.987 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:17:33.988 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:17:33.989 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:17:33.989 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:17:33.992 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:17:33.993 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:17:33.994 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:17:33.994 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:17:34.030 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:17:34.031 DEBUG:teuthology.parallel:result is None 2023-10-16T11:17:34.031 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:17:34.032 DEBUG:teuthology.parallel:result is None 2023-10-16T11:17:34.036 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:17:34.036 DEBUG:teuthology.parallel:result is None 2023-10-16T11:17:34.037 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:17:34.037 DEBUG:teuthology.parallel:result is None 2023-10-16T11:17:34.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:17:34.109 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:17:34.109 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:17:34.123 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:17:34.124 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:17:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.166+0000 7fbb03bda700 1 Processor -- start 2023-10-16T11:17:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.166+0000 7fbb03bda700 1 -- start start 2023-10-16T11:17:34.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb03bda700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbafc15b9f0 0x7fbafc0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:34.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb03bda700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbafc0cc400 0x7fbafc0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:34.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb03bda700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:34.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb03bda700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc06b800 con 0x7fbafc0cc400 2023-10-16T11:17:34.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb03bda700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc06b370 con 0x7fbafc15b9f0 2023-10-16T11:17:34.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb03bda700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0cfdc0 con 0x7fbafc0cf980 2023-10-16T11:17:34.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbafc0cc400 0x7fbafc0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:34.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbafc0cc400 0x7fbafc0cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53744/0 (socket says 172.21.15.181:53744) 2023-10-16T11:17:34.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 -- 172.21.15.181:0/1339645634 learned_addr learned my addr 172.21.15.181:0/1339645634 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:34.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb02177700 1 --2- 172.21.15.181:0/1339645634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:34.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01976700 1 --2- 172.21.15.181:0/1339645634 >> v2:172.21.15.190:3300/0 conn(0x7fbafc15b9f0 0x7fbafc0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:34.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 -- 172.21.15.181:0/1339645634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 msgr2=0x7fbafc0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:34.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 --2- 172.21.15.181:0/1339645634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc0ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 -- 172.21.15.181:0/1339645634 >> v2:172.21.15.190:3300/0 conn(0x7fbafc15b9f0 msgr2=0x7fbafc0cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:34.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 --2- 172.21.15.181:0/1339645634 >> v2:172.21.15.190:3300/0 conn(0x7fbafc15b9f0 0x7fbafc0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 -- 172.21.15.181:0/1339645634 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc06aa40 con 0x7fbafc0cc400 2023-10-16T11:17:34.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb01175700 1 --2- 172.21.15.181:0/1339645634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc0cc400 0x7fbafc0cf440 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fbafc06f870 tx=0x7fbaec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa3d5a9dbb5269e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:34.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb00974700 1 -- 172.21.15.181:0/1339645634 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec00b870 con 0x7fbafc0cc400 2023-10-16T11:17:34.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb00974700 1 -- 172.21.15.181:0/1339645634 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec00b9d0 con 0x7fbafc0cc400 2023-10-16T11:17:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.167+0000 7fbb00974700 1 -- 172.21.15.181:0/1339645634 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec00bd60 con 0x7fbafc0cc400 2023-10-16T11:17:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 -- 172.21.15.181:0/1339645634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc0cc400 msgr2=0x7fbafc0cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1339645634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc0cc400 0x7fbafc0cf440 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fbafc06f870 tx=0x7fbaec009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 -- 172.21.15.181:0/1339645634 shutdown_connections 2023-10-16T11:17:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1339645634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1339645634 >> v2:172.21.15.190:3300/0 conn(0x7fbafc15b9f0 0x7fbafc0cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1339645634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc0cc400 0x7fbafc0cf440 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 -- 172.21.15.181:0/1339645634 >> 172.21.15.181:0/1339645634 conn(0x7fbafc0d04a0 msgr2=0x7fbafc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 -- 172.21.15.181:0/1339645634 shutdown_connections 2023-10-16T11:17:34.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.168+0000 7fbb03bda700 1 -- 172.21.15.181:0/1339645634 wait complete. 2023-10-16T11:17:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.169+0000 7fbb03bda700 1 Processor -- start 2023-10-16T11:17:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.170+0000 7fbb03bda700 1 -- start start 2023-10-16T11:17:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.170+0000 7fbb03bda700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc0cc400 0x7fbafc09a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:34.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.170+0000 7fbb03bda700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc1f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.170+0000 7fbb03bda700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc15b9f0 0x7fbafc1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.170+0000 7fbb03bda700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0af5a0 con 0x7fbafc15b9f0 2023-10-16T11:17:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.170+0000 7fbb03bda700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09b6b0 con 0x7fbafc0cc400 2023-10-16T11:17:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.170+0000 7fbb03bda700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09a7d0 con 0x7fbafc0cf980 2023-10-16T11:17:34.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc15b9f0 0x7fbafc1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc15b9f0 0x7fbafc1f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53752/0 (socket says 172.21.15.181:53752) 2023-10-16T11:17:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 -- 172.21.15.181:0/1876079885 learned_addr learned my addr 172.21.15.181:0/1876079885 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb01175700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc1f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb01976700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc0cc400 0x7fbafc09a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:34.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 -- 172.21.15.181:0/1876079885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 msgr2=0x7fbafc1f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc1f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 -- 172.21.15.181:0/1876079885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc0cc400 msgr2=0x7fbafc09a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc0cc400 0x7fbafc09a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 -- 172.21.15.181:0/1876079885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec009550 con 0x7fbafc15b9f0 2023-10-16T11:17:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb02177700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc15b9f0 0x7fbafc1f4b20 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400ccf0 tx=0x7fbaf400a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:34.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbaf27fc700 1 -- 172.21.15.181:0/1876079885 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400ad60 con 0x7fbafc15b9f0 2023-10-16T11:17:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbaf27fc700 1 -- 172.21.15.181:0/1876079885 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf4003a20 con 0x7fbafc15b9f0 2023-10-16T11:17:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbaf27fc700 1 -- 172.21.15.181:0/1876079885 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4003db0 con 0x7fbafc15b9f0 2023-10-16T11:17:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc09aaa0 con 0x7fbafc15b9f0 2023-10-16T11:17:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.171+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc09b060 con 0x7fbafc15b9f0 2023-10-16T11:17:34.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.172+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc0d1410 con 0x7fbafc15b9f0 2023-10-16T11:17:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.173+0000 7fbaf27fc700 1 -- 172.21.15.181:0/1876079885 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbaf4003f50 con 0x7fbafc15b9f0 2023-10-16T11:17:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.173+0000 7fbaf27fc700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbae006b020 0x7fbae006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.173+0000 7fbaf27fc700 1 -- 172.21.15.181:0/1876079885 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6499+0+0 (secure 0 0 0) 0x7fbaf4012070 con 0x7fbafc15b9f0 2023-10-16T11:17:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.173+0000 7fbb01976700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbae006b020 0x7fbae006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:34.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.174+0000 7fbb01976700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbae006b020 0x7fbae006d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800aaa0 tx=0x7fbaf8008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:34.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.178+0000 7fbaf27fc700 1 -- 172.21.15.181:0/1876079885 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbaf4054dc0 con 0x7fbafc15b9f0 2023-10-16T11:17:34.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:34.384+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbafc0b30c0 con 0x7fbafc15b9f0 2023-10-16T11:17:36.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.001+0000 7fbaf27fc700 1 -- 172.21.15.181:0/1876079885 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v195) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbaf40583e0 con 0x7fbafc15b9f0 2023-10-16T11:17:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbae006b020 msgr2=0x7fbae006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbae006b020 0x7fbae006d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800aaa0 tx=0x7fbaf8008040 comp rx=0 tx=0).stop 2023-10-16T11:17:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc15b9f0 msgr2=0x7fbafc1f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc15b9f0 0x7fbafc1f4b20 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400ccf0 tx=0x7fbaf400a290 comp rx=0 tx=0).stop 2023-10-16T11:17:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 shutdown_connections 2023-10-16T11:17:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbae006b020 0x7fbae006d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbafc15b9f0 0x7fbafc1f4b20 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc0cc400 0x7fbafc09a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 --2- 172.21.15.181:0/1876079885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf980 0x7fbafc1f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 >> 172.21.15.181:0/1876079885 conn(0x7fbafc0d04a0 msgr2=0x7fbafc0b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 shutdown_connections 2023-10-16T11:17:36.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.004+0000 7fbb03bda700 1 -- 172.21.15.181:0/1876079885 wait complete. 2023-10-16T11:17:36.008 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T11:17:36.020 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:17:36.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.147+0000 7f3ce4794700 1 Processor -- start 2023-10-16T11:17:36.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.147+0000 7f3ce4794700 1 -- start start 2023-10-16T11:17:36.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce4794700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3cdc15dfd0 0x7f3cdc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce4794700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3cdc0cd470 0x7f3cdc0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce4794700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cdc06f870 con 0x7f3cdc0af450 2023-10-16T11:17:36.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce4794700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0cde40 con 0x7f3cdc0cd470 2023-10-16T11:17:36.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce4794700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0d1160 con 0x7f3cdc15dfd0 2023-10-16T11:17:36.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce2d31700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3cdc0cd470 0x7f3cdc0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce2d31700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3cdc0cd470 0x7f3cdc0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53764/0 (socket says 172.21.15.181:53764) 2023-10-16T11:17:36.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.148+0000 7f3ce2d31700 1 -- 172.21.15.181:0/2099137062 learned_addr learned my addr 172.21.15.181:0/2099137062 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:36.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce2530700 1 --2- 172.21.15.181:0/2099137062 >> v2:172.21.15.190:3300/0 conn(0x7f3cdc15dfd0 0x7f3cdc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce2d31700 1 -- 172.21.15.181:0/2099137062 >> v2:172.21.15.190:3300/0 conn(0x7f3cdc15dfd0 msgr2=0x7f3cdc0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:36.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce2d31700 1 --2- 172.21.15.181:0/2099137062 >> v2:172.21.15.190:3300/0 conn(0x7f3cdc15dfd0 0x7f3cdc0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce2d31700 1 -- 172.21.15.181:0/2099137062 >> v1:172.21.15.190:6789/0 conn(0x7f3cdc0af450 legacy=0x7f3cdc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:36.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce2d31700 1 -- 172.21.15.181:0/2099137062 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0cc8e0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce2d31700 1 --2- 172.21.15.181:0/2099137062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0cd900 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc070340 tx=0x7f3cd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460080befe08429d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce152e700 1 -- 172.21.15.181:0/2099137062 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400e070 con 0x7f3cdc0cd470 2023-10-16T11:17:36.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce152e700 1 -- 172.21.15.181:0/2099137062 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4003a20 con 0x7f3cdc0cd470 2023-10-16T11:17:36.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce152e700 1 -- 172.21.15.181:0/2099137062 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4003dd0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce4794700 1 -- 172.21.15.181:0/2099137062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 msgr2=0x7f3cdc0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.149+0000 7f3ce4794700 1 --2- 172.21.15.181:0/2099137062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0cd900 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc070340 tx=0x7f3cd400b330 comp rx=0 tx=0).stop 2023-10-16T11:17:36.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.150+0000 7f3ce4794700 1 -- 172.21.15.181:0/2099137062 shutdown_connections 2023-10-16T11:17:36.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.150+0000 7f3ce4794700 1 --2- 172.21.15.181:0/2099137062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0cd900 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.150+0000 7f3ce4794700 1 --2- 172.21.15.181:0/2099137062 >> v2:172.21.15.190:3300/0 conn(0x7f3cdc15dfd0 0x7f3cdc0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.150+0000 7f3ce4794700 1 -- 172.21.15.181:0/2099137062 >> 172.21.15.181:0/2099137062 conn(0x7f3cdc0d04a0 msgr2=0x7f3cdc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:36.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.150+0000 7f3ce4794700 1 -- 172.21.15.181:0/2099137062 shutdown_connections 2023-10-16T11:17:36.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.150+0000 7f3ce4794700 1 -- 172.21.15.181:0/2099137062 wait complete. 2023-10-16T11:17:36.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.151+0000 7f3ce4794700 1 Processor -- start 2023-10-16T11:17:36.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.151+0000 7f3ce4794700 1 -- start start 2023-10-16T11:17:36.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce4794700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0af450 0x7f3cdc1549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce4794700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce4794700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cdc15dfd0 0x7f3cdc153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce4794700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc0c7690 con 0x7f3cdc0cd470 2023-10-16T11:17:36.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce4794700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc0c78e0 con 0x7f3cdc15dfd0 2023-10-16T11:17:36.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce4794700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc0c7bc0 con 0x7f3cdc0af450 2023-10-16T11:17:36.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53770/0 (socket says 172.21.15.181:53770) 2023-10-16T11:17:36.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 -- 172.21.15.181:0/1567589210 learned_addr learned my addr 172.21.15.181:0/1567589210 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:36.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce2530700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0af450 0x7f3cdc1549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce2d31700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cdc15dfd0 0x7f3cdc153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 -- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0af450 msgr2=0x7f3cdc1549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0af450 0x7f3cdc1549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 -- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cdc15dfd0 msgr2=0x7f3cdc153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cdc15dfd0 0x7f3cdc153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce1d2f700 1 -- 172.21.15.181:0/1567589210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4009b30 con 0x7f3cdc0cd470 2023-10-16T11:17:36.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.152+0000 7f3ce2d31700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cdc15dfd0 0x7f3cdc153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:36.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.153+0000 7f3ce1d2f700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0c86a0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b4a0 tx=0x7f3cd800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.153+0000 7f3cd2ffd700 1 -- 172.21.15.181:0/1567589210 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8011830 con 0x7f3cdc0cd470 2023-10-16T11:17:36.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.153+0000 7f3cd2ffd700 1 -- 172.21.15.181:0/1567589210 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8011990 con 0x7f3cdc0cd470 2023-10-16T11:17:36.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.153+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cdc0c7ea0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.153+0000 7f3cd2ffd700 1 -- 172.21.15.181:0/1567589210 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8011d40 con 0x7f3cdc0cd470 2023-10-16T11:17:36.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.153+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cdc0c8be0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.154+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cdc0c83c0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.154+0000 7f3cd2ffd700 1 -- 172.21.15.181:0/1567589210 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3cd801a400 con 0x7f3cdc0cd470 2023-10-16T11:17:36.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.154+0000 7f3cd2ffd700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3cc006b020 0x7f3cc006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.155+0000 7f3cd2ffd700 1 -- 172.21.15.181:0/1567589210 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f3cd80892d0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.155+0000 7f3ce2530700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3cc006b020 0x7f3cc006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.155+0000 7f3ce2530700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3cc006b020 0x7f3cc006d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0cfed0 tx=0x7f3ccc008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.159+0000 7f3cd2ffd700 1 -- 172.21.15.181:0/1567589210 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3cdc0c83c0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.435+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3cdc0c83c0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.435+0000 7f3cd2ffd700 1 -- 172.21.15.181:0/1567589210 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v49) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3cdc0c83c0 con 0x7f3cdc0cd470 2023-10-16T11:17:36.436 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:36.437 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:17:36.437 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T11:17:36.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.437 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T11:17:36.437 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T11:17:36.438 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:17:36.438 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:17:36.438 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.439 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T11:17:36.439 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T11:17:36.440 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:17:36.440 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:17:36.440 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T11:17:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T11:17:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.442 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.442 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:17:36.442 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:17:36.442 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.442 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.443 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T11:17:36.443 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T11:17:36.443 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:17:36.444 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:17:36.444 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.445 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T11:17:36.445 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T11:17:36.445 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.445 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.445 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T11:17:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T11:17:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T11:17:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T11:17:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T11:17:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T11:17:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T11:17:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T11:17:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T11:17:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T11:17:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.454 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.454 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T11:17:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T11:17:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.456 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T11:17:36.456 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T11:17:36.456 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:17:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:17:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T11:17:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T11:17:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T11:17:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:17:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T11:17:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T11:17:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:17:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:17:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:17:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T11:17:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T11:17:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:17:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:17:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:17:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.463 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T11:17:36.463 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T11:17:36.463 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.463 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:17:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:17:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:17:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:17:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T11:17:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T11:17:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:17:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:17:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:17:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T11:17:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T11:17:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:17:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:17:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:17:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T11:17:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T11:17:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:17:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:17:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:17:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:17:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:36.470 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T11:17:36.470 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:17:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3cc006b020 msgr2=0x7f3cc006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3cc006b020 0x7f3cc006d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0cfed0 tx=0x7f3ccc008040 comp rx=0 tx=0).stop 2023-10-16T11:17:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 msgr2=0x7f3cdc0c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0c86a0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b4a0 tx=0x7f3cd800b860 comp rx=0 tx=0).stop 2023-10-16T11:17:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 shutdown_connections 2023-10-16T11:17:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f3cc006b020 0x7f3cc006d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0af450 0x7f3cdc1549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cdc15dfd0 0x7f3cdc153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 --2- 172.21.15.181:0/1567589210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cdc0cd470 0x7f3cdc0c86a0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 >> 172.21.15.181:0/1567589210 conn(0x7f3cdc0d04a0 msgr2=0x7f3cdc15d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 shutdown_connections 2023-10-16T11:17:36.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.437+0000 7f3ce4794700 1 -- 172.21.15.181:0/1567589210 wait complete. 2023-10-16T11:17:36.474 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:36.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.594+0000 7fec1c0b7700 1 Processor -- start 2023-10-16T11:17:36.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.594+0000 7fec1c0b7700 1 -- start start 2023-10-16T11:17:36.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1c0b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fec1415db70 0x7fec140665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1c0b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fec140ca2e0 0x7fec140ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1c0b7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec14066c60 con 0x7fec140cedd0 2023-10-16T11:17:36.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1c0b7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec140cda10 con 0x7fec140ca2e0 2023-10-16T11:17:36.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1c0b7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1406c1a0 con 0x7fec1415db70 2023-10-16T11:17:36.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fec140ca2e0 0x7fec140ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fec140ca2e0 0x7fec140ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53778/0 (socket says 172.21.15.181:53778) 2023-10-16T11:17:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec19652700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fec140cedd0 0x7fec140cf250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36620/0 (socket says 172.21.15.181:36620) 2023-10-16T11:17:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 -- 172.21.15.181:0/3739654385 learned_addr learned my addr 172.21.15.181:0/3739654385 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec19e53700 1 --2- 172.21.15.181:0/3739654385 >> v2:172.21.15.190:3300/0 conn(0x7fec1415db70 0x7fec140665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 -- 172.21.15.181:0/3739654385 >> v2:172.21.15.190:3300/0 conn(0x7fec1415db70 msgr2=0x7fec140665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:36.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 --2- 172.21.15.181:0/3739654385 >> v2:172.21.15.190:3300/0 conn(0x7fec1415db70 0x7fec140665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 -- 172.21.15.181:0/3739654385 >> v1:172.21.15.181:6789/0 conn(0x7fec140cedd0 legacy=0x7fec140cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 -- 172.21.15.181:0/3739654385 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1406b3e0 con 0x7fec140ca2e0 2023-10-16T11:17:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec18e51700 1 -- 172.21.15.181:0/3739654385 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663906398 0 0) 0x7fec14066c60 con 0x7fec140cedd0 2023-10-16T11:17:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec1a654700 1 --2- 172.21.15.181:0/3739654385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140ca2e0 0x7fec140ca770 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fec1406cc90 tx=0x7fec0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347f514453658cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.595+0000 7fec18e51700 1 -- 172.21.15.181:0/3739654385 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00e070 con 0x7fec140ca2e0 2023-10-16T11:17:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec18e51700 1 -- 172.21.15.181:0/3739654385 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00b910 con 0x7fec140ca2e0 2023-10-16T11:17:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec18e51700 1 -- 172.21.15.181:0/3739654385 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00bcc0 con 0x7fec140ca2e0 2023-10-16T11:17:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 -- 172.21.15.181:0/3739654385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140ca2e0 msgr2=0x7fec140ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/3739654385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140ca2e0 0x7fec140ca770 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fec1406cc90 tx=0x7fec0c00b330 comp rx=0 tx=0).stop 2023-10-16T11:17:36.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 -- 172.21.15.181:0/3739654385 shutdown_connections 2023-10-16T11:17:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/3739654385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140ca2e0 0x7fec140ca770 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/3739654385 >> v2:172.21.15.190:3300/0 conn(0x7fec1415db70 0x7fec140665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 -- 172.21.15.181:0/3739654385 >> 172.21.15.181:0/3739654385 conn(0x7fec140b1780 msgr2=0x7fec140b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 -- 172.21.15.181:0/3739654385 shutdown_connections 2023-10-16T11:17:36.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.596+0000 7fec1c0b7700 1 -- 172.21.15.181:0/3739654385 wait complete. 2023-10-16T11:17:36.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.597+0000 7fec1c0b7700 1 Processor -- start 2023-10-16T11:17:36.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.597+0000 7fec1c0b7700 1 -- start start 2023-10-16T11:17:36.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec1c0b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec140ca2e0 0x7fec140c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec1c0b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140cedd0 0x7fec141f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec1c0b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec1415db70 0x7fec141f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec1c0b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1409deb0 con 0x7fec140cedd0 2023-10-16T11:17:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec1c0b7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec141502d0 con 0x7fec140ca2e0 2023-10-16T11:17:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec1c0b7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec140a0a20 con 0x7fec1415db70 2023-10-16T11:17:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140cedd0 0x7fec141f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140cedd0 0x7fec141f6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53782/0 (socket says 172.21.15.181:53782) 2023-10-16T11:17:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 -- 172.21.15.181:0/2452241657 learned_addr learned my addr 172.21.15.181:0/2452241657 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19e53700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec140ca2e0 0x7fec140c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec1a654700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec1415db70 0x7fec141f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 -- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec1415db70 msgr2=0x7fec141f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec1415db70 0x7fec141f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 -- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec140ca2e0 msgr2=0x7fec140c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec140ca2e0 0x7fec140c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.598+0000 7fec19652700 1 -- 172.21.15.181:0/2452241657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c009b30 con 0x7fec140cedd0 2023-10-16T11:17:36.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.599+0000 7fec19652700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140cedd0 0x7fec141f6980 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fec040011e0 tx=0x7fec0400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.599+0000 7fec0a7fc700 1 -- 172.21.15.181:0/2452241657 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04004c80 con 0x7fec140cedd0 2023-10-16T11:17:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.599+0000 7fec0a7fc700 1 -- 172.21.15.181:0/2452241657 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec04004de0 con 0x7fec140cedd0 2023-10-16T11:17:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.599+0000 7fec0a7fc700 1 -- 172.21.15.181:0/2452241657 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04007310 con 0x7fec140cedd0 2023-10-16T11:17:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.599+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec140a0d00 con 0x7fec140cedd0 2023-10-16T11:17:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.599+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec140a0fe0 con 0x7fec140cedd0 2023-10-16T11:17:36.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.600+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1409e390 con 0x7fec140cedd0 2023-10-16T11:17:36.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.600+0000 7fec0a7fc700 1 -- 172.21.15.181:0/2452241657 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fec04007470 con 0x7fec140cedd0 2023-10-16T11:17:36.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.601+0000 7fec0a7fc700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7febf806b020 0x7febf806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.601+0000 7fec19e53700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7febf806b020 0x7febf806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.601+0000 7fec0a7fc700 1 -- 172.21.15.181:0/2452241657 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5381+0+0 (secure 0 0 0) 0x7fec04089390 con 0x7fec140cedd0 2023-10-16T11:17:36.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.601+0000 7fec19e53700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7febf806b020 0x7febf806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fec141f68e0 tx=0x7fec10009210 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.605+0000 7fec0a7fc700 1 -- 172.21.15.181:0/2452241657 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fec1409e390 con 0x7fec140cedd0 2023-10-16T11:17:36.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.798+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec1409e390 con 0x7fec140cedd0 2023-10-16T11:17:36.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.799+0000 7fec0a7fc700 1 -- 172.21.15.181:0/2452241657 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+13709 (secure 0 0 0) 0x7fec1409e390 con 0x7fec140cedd0 2023-10-16T11:17:36.801 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:36.801 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":195,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:35.999754+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:36.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7febf806b020 msgr2=0x7febf806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7febf806b020 0x7febf806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fec141f68e0 tx=0x7fec10009210 comp rx=0 tx=0).stop 2023-10-16T11:17:36.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140cedd0 msgr2=0x7fec141f6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140cedd0 0x7fec141f6980 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fec040011e0 tx=0x7fec0400ca30 comp rx=0 tx=0).stop 2023-10-16T11:17:36.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 shutdown_connections 2023-10-16T11:17:36.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7febf806b020 0x7febf806d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec1415db70 0x7fec141f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec140ca2e0 0x7fec140c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 --2- 172.21.15.181:0/2452241657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec140cedd0 0x7fec141f6980 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 >> 172.21.15.181:0/2452241657 conn(0x7fec140b1780 msgr2=0x7fec140c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:36.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 shutdown_connections 2023-10-16T11:17:36.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.802+0000 7fec1c0b7700 1 -- 172.21.15.181:0/2452241657 wait complete. 2023-10-16T11:17:36.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:17:36.816 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:17:36.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.938+0000 7f88f3d2e700 1 Processor -- start 2023-10-16T11:17:36.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.938+0000 7f88f3d2e700 1 -- start start 2023-10-16T11:17:36.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f3d2e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f88ec15d740 0x7f88ec161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f3d2e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0cf890 0x7f88ec0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f3d2e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec0d0cb0 con 0x7f88ec0ce410 2023-10-16T11:17:36.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f3d2e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec1624b0 con 0x7f88ec15d740 2023-10-16T11:17:36.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f3d2e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0ce880 con 0x7f88ec0cf890 2023-10-16T11:17:36.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f22cb700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f88ec0ce410 0x7f88ec0ccf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36638/0 (socket says 172.21.15.181:36638) 2023-10-16T11:17:36.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f22cb700 1 -- 172.21.15.181:0/4178336699 learned_addr learned my addr 172.21.15.181:0/4178336699 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:36.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f1aca700 1 --2- 172.21.15.181:0/4178336699 >> v2:172.21.15.190:3300/0 conn(0x7f88ec15d740 0x7f88ec161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f12c9700 1 --2- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0cf890 0x7f88ec0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.939+0000 7f88f0ac8700 1 -- 172.21.15.181:0/4178336699 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102389495 0 0) 0x7f88ec0d0cb0 con 0x7f88ec0ce410 2023-10-16T11:17:36.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f0ac8700 1 -- 172.21.15.181:0/4178336699 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d80035b0 con 0x7f88ec0ce410 2023-10-16T11:17:36.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f0ac8700 1 -- 172.21.15.181:0/4178336699 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514504996 0 0) 0x7f88d80035b0 con 0x7f88ec0ce410 2023-10-16T11:17:36.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f1aca700 1 -- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0cf890 msgr2=0x7f88ec0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:36.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f1aca700 1 --2- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0cf890 0x7f88ec0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f1aca700 1 -- 172.21.15.181:0/4178336699 >> v1:172.21.15.181:6789/0 conn(0x7f88ec0ce410 legacy=0x7f88ec0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f1aca700 1 -- 172.21.15.181:0/4178336699 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec0d00a0 con 0x7f88ec15d740 2023-10-16T11:17:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f1aca700 1 --2- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec161f70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009870 tx=0x7f88dc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a968afd70a40941c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f0ac8700 1 -- 172.21.15.181:0/4178336699 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e40019f0 con 0x7f88ec15d740 2023-10-16T11:17:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f0ac8700 1 -- 172.21.15.181:0/4178336699 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc004030 con 0x7f88ec15d740 2023-10-16T11:17:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.940+0000 7f88f0ac8700 1 -- 172.21.15.181:0/4178336699 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0043a0 con 0x7f88ec15d740 2023-10-16T11:17:36.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 -- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 msgr2=0x7f88ec161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec161f70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009870 tx=0x7f88dc009b80 comp rx=0 tx=0).stop 2023-10-16T11:17:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 -- 172.21.15.181:0/4178336699 shutdown_connections 2023-10-16T11:17:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec161f70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/4178336699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0cf890 0x7f88ec0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 -- 172.21.15.181:0/4178336699 >> 172.21.15.181:0/4178336699 conn(0x7f88ec09a930 msgr2=0x7f88ec0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:36.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 -- 172.21.15.181:0/4178336699 shutdown_connections 2023-10-16T11:17:36.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.941+0000 7f88f3d2e700 1 -- 172.21.15.181:0/4178336699 wait complete. 2023-10-16T11:17:36.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.942+0000 7f88f3d2e700 1 Processor -- start 2023-10-16T11:17:36.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.942+0000 7f88f3d2e700 1 -- start start 2023-10-16T11:17:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f3d2e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0ce410 0x7f88ec160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f3d2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88ec0cf890 0x7f88ec1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f3d2e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f3d2e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1606c0 con 0x7f88ec0cf890 2023-10-16T11:17:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f3d2e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1f3d90 con 0x7f88ec15d740 2023-10-16T11:17:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f3d2e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec15f3c0 con 0x7f88ec0ce410 2023-10-16T11:17:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88ec0cf890 0x7f88ec1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88ec0cf890 0x7f88ec1f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53796/0 (socket says 172.21.15.181:53796) 2023-10-16T11:17:36.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 -- 172.21.15.181:0/1427658473 learned_addr learned my addr 172.21.15.181:0/1427658473 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:36.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f22cb700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f1aca700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0ce410 0x7f88ec160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 -- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0ce410 msgr2=0x7f88ec160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0ce410 0x7f88ec160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 -- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 msgr2=0x7f88ec1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 -- 172.21.15.181:0/1427658473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc0094a0 con 0x7f88ec0cf890 2023-10-16T11:17:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f22cb700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:36.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f12c9700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88ec0cf890 0x7f88ec1f3850 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f88e800e9b0 tx=0x7f88e800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88e27fc700 1 -- 172.21.15.181:0/1427658473 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8009710 con 0x7f88ec0cf890 2023-10-16T11:17:36.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88e27fc700 1 -- 172.21.15.181:0/1427658473 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e800c850 con 0x7f88ec0cf890 2023-10-16T11:17:36.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec15f6a0 con 0x7f88ec0cf890 2023-10-16T11:17:36.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.943+0000 7f88e27fc700 1 -- 172.21.15.181:0/1427658473 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800cbc0 con 0x7f88ec0cf890 2023-10-16T11:17:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.944+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec15fc60 con 0x7f88ec0cf890 2023-10-16T11:17:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.944+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec15fe80 con 0x7f88ec0cf890 2023-10-16T11:17:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.945+0000 7f88e27fc700 1 -- 172.21.15.181:0/1427658473 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f88e800cd20 con 0x7f88ec0cf890 2023-10-16T11:17:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.945+0000 7f88e27fc700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f88d006b020 0x7f88d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:36.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.945+0000 7f88e27fc700 1 -- 172.21.15.181:0/1427658473 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f88e8089680 con 0x7f88ec0cf890 2023-10-16T11:17:36.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.945+0000 7f88f1aca700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f88d006b020 0x7f88d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:36.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.946+0000 7f88f1aca700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f88d006b020 0x7f88d006d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0097c0 tx=0x7f88dc013040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:36.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:36.950+0000 7f88e27fc700 1 -- 172.21.15.181:0/1427658473 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88e80547f0 con 0x7f88ec0cf890 2023-10-16T11:17:37.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.144+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88ec04ebb0 con 0x7f88ec0cf890 2023-10-16T11:17:37.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.144+0000 7f88e27fc700 1 -- 172.21.15.181:0/1427658473 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v195) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88e8057e10 con 0x7f88ec0cf890 2023-10-16T11:17:37.145 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:17:37.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f88d006b020 msgr2=0x7f88d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f88d006b020 0x7f88d006d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0097c0 tx=0x7f88dc013040 comp rx=0 tx=0).stop 2023-10-16T11:17:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88ec0cf890 msgr2=0x7f88ec1f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88ec0cf890 0x7f88ec1f3850 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f88e800e9b0 tx=0x7f88e800ed70 comp rx=0 tx=0).stop 2023-10-16T11:17:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 shutdown_connections 2023-10-16T11:17:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f88d006b020 0x7f88d006d4e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88ec0ce410 0x7f88ec160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88ec15d740 0x7f88ec1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 --2- 172.21.15.181:0/1427658473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88ec0cf890 0x7f88ec1f3850 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 >> 172.21.15.181:0/1427658473 conn(0x7f88ec09a930 msgr2=0x7f88ec06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 shutdown_connections 2023-10-16T11:17:37.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.146+0000 7f88f3d2e700 1 -- 172.21.15.181:0/1427658473 wait complete. 2023-10-16T11:17:37.159 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:17:37.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.279+0000 7f5080eaa700 1 Processor -- start 2023-10-16T11:17:37.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.279+0000 7f5080eaa700 1 -- start start 2023-10-16T11:17:37.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f5080eaa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 0x7f507c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f5080eaa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f507c0cd470 0x7f507c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f5080eaa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507c06f870 con 0x7f507c15dfd0 2023-10-16T11:17:37.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f5080eaa700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507c0d1160 con 0x7f507c0cd470 2023-10-16T11:17:37.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f5080eaa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0cc8e0 con 0x7f507c0af450 2023-10-16T11:17:37.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f507c15dfd0 0x7f507c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36648/0 (socket says 172.21.15.181:36648) 2023-10-16T11:17:37.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507a59c700 1 -- 172.21.15.181:0/1940339555 learned_addr learned my addr 172.21.15.181:0/1940339555 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:37.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557844844 0 0) 0x7f507c06f870 con 0x7f507c15dfd0 2023-10-16T11:17:37.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507ad9d700 1 --2- 172.21.15.181:0/1940339555 >> v2:172.21.15.190:3300/0 conn(0x7f507c0cd470 0x7f507c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f5079d9b700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 0x7f507c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50640035b0 con 0x7f507c15dfd0 2023-10-16T11:17:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028813053 0 0) 0x7f50640035b0 con 0x7f507c15dfd0 2023-10-16T11:17:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 msgr2=0x7f507c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 0x7f507c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 >> v2:172.21.15.190:3300/0 conn(0x7f507c0cd470 msgr2=0x7f507c0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 --2- 172.21.15.181:0/1940339555 >> v2:172.21.15.190:3300/0 conn(0x7f507c0cd470 0x7f507c0cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c0ccbc0 con 0x7f507c15dfd0 2023-10-16T11:17:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f506c0017f0 con 0x7f507c15dfd0 2023-10-16T11:17:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 >> v1:172.21.15.181:6789/0 conn(0x7f507c15dfd0 legacy=0x7f507c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 0x7f5064003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.280+0000 7f507959a700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064003f80 0x7f5064004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507959a700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f5064005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0d1160 con 0x7f5064004d90 2023-10-16T11:17:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0cc8e0 con 0x7f5064003f80 2023-10-16T11:17:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506400ba60 con 0x7f507c0af450 2023-10-16T11:17:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507ad9d700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f5064005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507a59c700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 0x7f5064003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f5079d9b700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064003f80 0x7f5064004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507ad9d700 1 -- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 msgr2=0x7f5064003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507ad9d700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0af450 0x7f5064003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507ad9d700 1 -- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064003f80 msgr2=0x7f5064004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507ad9d700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5064003f80 0x7f5064004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507ad9d700 1 -- 172.21.15.181:0/1940339555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640035b0 con 0x7f5064004d90 2023-10-16T11:17:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507ad9d700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f5064005210 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f507400e170 tx=0x7f5064005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d95a19cdef5d772 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507400ec90 con 0x7f5064004d90 2023-10-16T11:17:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507400f040 con 0x7f5064004d90 2023-10-16T11:17:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.281+0000 7f507959a700 1 -- 172.21.15.181:0/1940339555 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074014480 con 0x7f5064004d90 2023-10-16T11:17:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.282+0000 7f5080eaa700 1 -- 172.21.15.181:0/1940339555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 msgr2=0x7f5064005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.282+0000 7f5080eaa700 1 --2- 172.21.15.181:0/1940339555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f5064005210 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f507400e170 tx=0x7f5064005880 comp rx=0 tx=0).stop 2023-10-16T11:17:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.282+0000 7f507ad9d700 1 -- 172.21.15.181:0/1940339555 reap_dead start 2023-10-16T11:17:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.282+0000 7f5080eaa700 1 -- 172.21.15.181:0/1940339555 shutdown_connections 2023-10-16T11:17:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.282+0000 7f5080eaa700 1 -- 172.21.15.181:0/1940339555 >> 172.21.15.181:0/1940339555 conn(0x7f507c0d04a0 msgr2=0x7f507c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.282+0000 7f5080eaa700 1 -- 172.21.15.181:0/1940339555 shutdown_connections 2023-10-16T11:17:37.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.282+0000 7f5080eaa700 1 -- 172.21.15.181:0/1940339555 wait complete. 2023-10-16T11:17:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.283+0000 7f5080eaa700 1 Processor -- start 2023-10-16T11:17:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.283+0000 7f5080eaa700 1 -- start start 2023-10-16T11:17:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5080eaa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f507c0b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5080eaa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c0c82a0 0x7f507c0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5080eaa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0b1080 0x7f507c14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5080eaa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0c7570 con 0x7f5064004d90 2023-10-16T11:17:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5080eaa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0b1520 con 0x7f507c0c82a0 2023-10-16T11:17:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5080eaa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0c77c0 con 0x7f507c0b1080 2023-10-16T11:17:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f507c0b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f507c0b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53814/0 (socket says 172.21.15.181:53814) 2023-10-16T11:17:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 -- 172.21.15.181:0/717914827 learned_addr learned my addr 172.21.15.181:0/717914827 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 -- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0b1080 msgr2=0x7f507c14c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507ad9d700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0b1080 0x7f507c14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5079d9b700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c0c82a0 0x7f507c0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0b1080 0x7f507c14c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 -- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c0c82a0 msgr2=0x7f507c0b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c0c82a0 0x7f507c0b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 -- 172.21.15.181:0/717914827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c0c7aa0 con 0x7f5064004d90 2023-10-16T11:17:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f5079d9b700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c0c82a0 0x7f507c0b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.284+0000 7f507a59c700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f507c0b5730 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f506c00f930 tx=0x7f506c00fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.285+0000 7f506affd700 1 -- 172.21.15.181:0/717914827 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c0072e0 con 0x7f5064004d90 2023-10-16T11:17:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.285+0000 7f506affd700 1 -- 172.21.15.181:0/717914827 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c007440 con 0x7f5064004d90 2023-10-16T11:17:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.285+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c0c7d80 con 0x7f5064004d90 2023-10-16T11:17:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.285+0000 7f506affd700 1 -- 172.21.15.181:0/717914827 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c005830 con 0x7f5064004d90 2023-10-16T11:17:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.285+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c0b5e50 con 0x7f5064004d90 2023-10-16T11:17:37.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.285+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c0c7fc0 con 0x7f5064004d90 2023-10-16T11:17:37.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.286+0000 7f506affd700 1 -- 172.21.15.181:0/717914827 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f506c005990 con 0x7f5064004d90 2023-10-16T11:17:37.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.286+0000 7f506affd700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f505c06b020 0x7f505c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.286+0000 7f506affd700 1 -- 172.21.15.181:0/717914827 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f506c089590 con 0x7f5064004d90 2023-10-16T11:17:37.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.286+0000 7f5079d9b700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f505c06b020 0x7f505c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.287+0000 7f5079d9b700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f505c06b020 0x7f505c06d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f507c0c86e0 tx=0x7f5070008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:37.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.291+0000 7f506affd700 1 -- 172.21.15.181:0/717914827 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f507c0c7fc0 con 0x7f5064004d90 2023-10-16T11:17:37.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.485+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f507c0b5e50 con 0x7f5064004d90 2023-10-16T11:17:37.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.485+0000 7f506affd700 1 -- 172.21.15.181:0/717914827 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v195) v1 ==== 84+0+65 (secure 0 0 0) 0x7f507c0b5e50 con 0x7f5064004d90 2023-10-16T11:17:37.487 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:37.487 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:17:37.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:37.487 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:17:37.487 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:17:37.488 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:37.488 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:17:37.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f505c06b020 msgr2=0x7f505c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f505c06b020 0x7f505c06d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f507c0c86e0 tx=0x7f5070008040 comp rx=0 tx=0).stop 2023-10-16T11:17:37.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 msgr2=0x7f507c0b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f507c0b5730 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f506c00f930 tx=0x7f506c00fcf0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 shutdown_connections 2023-10-16T11:17:37.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f505c06b020 0x7f505c06d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f507c0b1080 0x7f507c14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5064004d90 0x7f507c0b5730 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 --2- 172.21.15.181:0/717914827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c0c82a0 0x7f507c0b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 >> 172.21.15.181:0/717914827 conn(0x7f507c0d04a0 msgr2=0x7f507c0cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:37.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 shutdown_connections 2023-10-16T11:17:37.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.487+0000 7f5080eaa700 1 -- 172.21.15.181:0/717914827 wait complete. 2023-10-16T11:17:37.500 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:17:37.501 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:17:37.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.624+0000 7f5f204bd700 1 Processor -- start 2023-10-16T11:17:37.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.625+0000 7f5f204bd700 1 -- start start 2023-10-16T11:17:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.625+0000 7f5f204bd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.625+0000 7f5f204bd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f180cc4a0 0x7f5f1815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.625+0000 7f5f204bd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f18160240 con 0x7f5f180cf5c0 2023-10-16T11:17:37.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.625+0000 7f5f204bd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1815bd80 con 0x7f5f180cc4a0 2023-10-16T11:17:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.625+0000 7f5f204bd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1806b6d0 con 0x7f5f1806a8a0 2023-10-16T11:17:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1ea5a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f180cc4a0 0x7f5f1815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:50850/0 (socket says 172.21.15.181:50850) 2023-10-16T11:17:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 -- 172.21.15.181:0/3721879970 learned_addr learned my addr 172.21.15.181:0/3721879970 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 -- 172.21.15.181:0/3721879970 >> v2:172.21.15.190:3300/0 conn(0x7f5f180cc4a0 msgr2=0x7f5f1815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 --2- 172.21.15.181:0/3721879970 >> v2:172.21.15.190:3300/0 conn(0x7f5f180cc4a0 0x7f5f1815c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 -- 172.21.15.181:0/3721879970 >> v1:172.21.15.190:6789/0 conn(0x7f5f180cf5c0 legacy=0x7f5f1806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 -- 172.21.15.181:0/3721879970 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1815ff60 con 0x7f5f1806a8a0 2023-10-16T11:17:37.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1ea5a700 1 --2- 172.21.15.181:0/3721879970 >> v2:172.21.15.190:3300/0 conn(0x7f5f180cc4a0 0x7f5f1815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:37.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.626+0000 7f5f1e259700 1 --2- 172.21.15.181:0/3721879970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180caaf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abafe59ca359f07 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:37.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f1d257700 1 -- 172.21.15.181:0/3721879970 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08005e30 con 0x7f5f1806a8a0 2023-10-16T11:17:37.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f1d257700 1 -- 172.21.15.181:0/3721879970 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08004030 con 0x7f5f1806a8a0 2023-10-16T11:17:37.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f1d257700 1 -- 172.21.15.181:0/3721879970 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080043e0 con 0x7f5f1806a8a0 2023-10-16T11:17:37.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 -- 172.21.15.181:0/3721879970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 msgr2=0x7f5f180caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 --2- 172.21.15.181:0/3721879970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180caaf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 -- 172.21.15.181:0/3721879970 shutdown_connections 2023-10-16T11:17:37.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 --2- 172.21.15.181:0/3721879970 >> v2:172.21.15.190:3300/0 conn(0x7f5f180cc4a0 0x7f5f1815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 --2- 172.21.15.181:0/3721879970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180caaf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 -- 172.21.15.181:0/3721879970 >> 172.21.15.181:0/3721879970 conn(0x7f5f180d04a0 msgr2=0x7f5f180d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:37.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 -- 172.21.15.181:0/3721879970 shutdown_connections 2023-10-16T11:17:37.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.627+0000 7f5f204bd700 1 -- 172.21.15.181:0/3721879970 wait complete. 2023-10-16T11:17:37.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.628+0000 7f5f204bd700 1 Processor -- start 2023-10-16T11:17:37.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.628+0000 7f5f204bd700 1 -- start start 2023-10-16T11:17:37.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f204bd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f204bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f180cc4a0 0x7f5f1815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f204bd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f180cf5c0 0x7f5f180c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f204bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180c4690 con 0x7f5f180cc4a0 2023-10-16T11:17:37.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f204bd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180c4ff0 con 0x7f5f180cf5c0 2023-10-16T11:17:37.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f204bd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180c65b0 con 0x7f5f1806a8a0 2023-10-16T11:17:37.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f1da58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f180cc4a0 0x7f5f1815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f1da58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f180cc4a0 0x7f5f1815e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53828/0 (socket says 172.21.15.181:53828) 2023-10-16T11:17:37.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f1da58700 1 -- 172.21.15.181:0/1366389387 learned_addr learned my addr 172.21.15.181:0/1366389387 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:37.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f1ea5a700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f180cf5c0 0x7f5f180c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.629+0000 7f5f1e259700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f1da58700 1 -- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 msgr2=0x7f5f180c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f1da58700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f1da58700 1 -- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f180cf5c0 msgr2=0x7f5f180c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:37.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f1da58700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f180cf5c0 0x7f5f180c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:37.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f1da58700 1 -- 172.21.15.181:0/1366389387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10009a70 con 0x7f5f180cc4a0 2023-10-16T11:17:37.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f1da58700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f180cc4a0 0x7f5f1815e5a0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c430 tx=0x7f5f1400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:37.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f0effd700 1 -- 172.21.15.181:0/1366389387 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f140052d0 con 0x7f5f180cc4a0 2023-10-16T11:17:37.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f0effd700 1 -- 172.21.15.181:0/1366389387 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14005450 con 0x7f5f180cc4a0 2023-10-16T11:17:37.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f0effd700 1 -- 172.21.15.181:0/1366389387 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14006aa0 con 0x7f5f180cc4a0 2023-10-16T11:17:37.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f08009500 con 0x7f5f180cc4a0 2023-10-16T11:17:37.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.630+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f180c42b0 con 0x7f5f180cc4a0 2023-10-16T11:17:37.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.631+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f180c4440 con 0x7f5f180cc4a0 2023-10-16T11:17:37.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.631+0000 7f5f0effd700 1 -- 172.21.15.181:0/1366389387 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5f14003680 con 0x7f5f180cc4a0 2023-10-16T11:17:37.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.632+0000 7f5f0effd700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f5efc06b020 0x7f5efc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:37.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.632+0000 7f5f1e259700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f5efc06b020 0x7f5efc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:37.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.632+0000 7f5f0effd700 1 -- 172.21.15.181:0/1366389387 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5381+0+0 (secure 0 0 0) 0x7f5f14088760 con 0x7f5f180cc4a0 2023-10-16T11:17:37.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.632+0000 7f5f1e259700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f5efc06b020 0x7f5efc06d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08008070 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:37.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.636+0000 7f5f0effd700 1 -- 172.21.15.181:0/1366389387 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5f140537a0 con 0x7f5f180cc4a0 2023-10-16T11:17:37.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:37.830+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5f180c47d0 con 0x7f5f180cc4a0 2023-10-16T11:17:38.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.024+0000 7f5f0effd700 1 -- 172.21.15.181:0/1366389387 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v196) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5f180c47d0 con 0x7f5f180cc4a0 2023-10-16T11:17:38.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f5efc06b020 msgr2=0x7f5efc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:38.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f5efc06b020 0x7f5efc06d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08008070 comp rx=0 tx=0).stop 2023-10-16T11:17:38.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f180cc4a0 msgr2=0x7f5f1815e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:38.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f180cc4a0 0x7f5f1815e5a0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c430 tx=0x7f5f1400c740 comp rx=0 tx=0).stop 2023-10-16T11:17:38.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 shutdown_connections 2023-10-16T11:17:38.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f5efc06b020 0x7f5efc06d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f1806a8a0 0x7f5f180c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f180cf5c0 0x7f5f180c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 --2- 172.21.15.181:0/1366389387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f180cc4a0 0x7f5f1815e5a0 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.027+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 >> 172.21.15.181:0/1366389387 conn(0x7f5f180d04a0 msgr2=0x7f5f1815b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:38.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.028+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 shutdown_connections 2023-10-16T11:17:38.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.029+0000 7f5f204bd700 1 -- 172.21.15.181:0/1366389387 wait complete. 2023-10-16T11:17:38.033 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T11:17:38.044 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:17:38.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.171+0000 7fe66470d700 1 Processor -- start 2023-10-16T11:17:38.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.171+0000 7fe66470d700 1 -- start start 2023-10-16T11:17:38.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.172+0000 7fe66470d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c15f150 0x7fe65c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:38.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.172+0000 7fe66470d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe65c0cd170 0x7fe65c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:38.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.172+0000 7fe66470d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65c0b2ae0 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.172+0000 7fe66470d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c0ce970 con 0x7fe65c0cd170 2023-10-16T11:17:38.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.172+0000 7fe66470d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0b2dc0 con 0x7fe65c15f150 2023-10-16T11:17:38.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe662caa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe65c0cd170 0x7fe65c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:38.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe661ca8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe65c0ce4f0 0x7fe65c0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36682/0 (socket says 172.21.15.181:36682) 2023-10-16T11:17:38.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe661ca8700 1 -- 172.21.15.181:0/1502204983 learned_addr learned my addr 172.21.15.181:0/1502204983 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:38.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe6624a9700 1 --2- 172.21.15.181:0/1502204983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c15f150 0x7fe65c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:38.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe662caa700 1 -- 172.21.15.181:0/1502204983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c15f150 msgr2=0x7fe65c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:38.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe662caa700 1 --2- 172.21.15.181:0/1502204983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c15f150 0x7fe65c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe662caa700 1 -- 172.21.15.181:0/1502204983 >> v1:172.21.15.181:6789/0 conn(0x7fe65c0ce4f0 legacy=0x7fe65c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:38.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe6614a7700 1 -- 172.21.15.181:0/1502204983 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384570626 0 0) 0x7fe65c0b2ae0 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe662caa700 1 -- 172.21.15.181:0/1502204983 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0cf9d0 con 0x7fe65c0cd170 2023-10-16T11:17:38.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe662caa700 1 --2- 172.21.15.181:0/1502204983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0cd170 0x7fe65c0b1670 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fe654009d90 tx=0x7fe65400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a531a448276d0b73 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:38.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe6614a7700 1 -- 172.21.15.181:0/1502204983 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65400e070 con 0x7fe65c0cd170 2023-10-16T11:17:38.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe6614a7700 1 -- 172.21.15.181:0/1502204983 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65400bbd0 con 0x7fe65c0cd170 2023-10-16T11:17:38.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.173+0000 7fe6614a7700 1 -- 172.21.15.181:0/1502204983 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654012430 con 0x7fe65c0cd170 2023-10-16T11:17:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 -- 172.21.15.181:0/1502204983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0cd170 msgr2=0x7fe65c0b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 --2- 172.21.15.181:0/1502204983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0cd170 0x7fe65c0b1670 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fe654009d90 tx=0x7fe65400b330 comp rx=0 tx=0).stop 2023-10-16T11:17:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 -- 172.21.15.181:0/1502204983 shutdown_connections 2023-10-16T11:17:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 --2- 172.21.15.181:0/1502204983 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0cd170 0x7fe65c0b1670 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 --2- 172.21.15.181:0/1502204983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c15f150 0x7fe65c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 -- 172.21.15.181:0/1502204983 >> 172.21.15.181:0/1502204983 conn(0x7fe65c09a930 msgr2=0x7fe65c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:38.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 -- 172.21.15.181:0/1502204983 shutdown_connections 2023-10-16T11:17:38.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.174+0000 7fe66470d700 1 -- 172.21.15.181:0/1502204983 wait complete. 2023-10-16T11:17:38.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.175+0000 7fe66470d700 1 Processor -- start 2023-10-16T11:17:38.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.175+0000 7fe66470d700 1 -- start start 2023-10-16T11:17:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe66470d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c0cd170 0x7fe65c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe66470d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0ce4f0 0x7fe65c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe66470d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe65c15f150 0x7fe65c15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe66470d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c1f7760 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe66470d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c15dcf0 con 0x7fe65c15f150 2023-10-16T11:17:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe66470d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0c55c0 con 0x7fe65c0cd170 2023-10-16T11:17:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0ce4f0 0x7fe65c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0ce4f0 0x7fe65c15d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53844/0 (socket says 172.21.15.181:53844) 2023-10-16T11:17:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 -- 172.21.15.181:0/2922646450 learned_addr learned my addr 172.21.15.181:0/2922646450 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe662caa700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe65c15f150 0x7fe65c15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:38.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe6624a9700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c0cd170 0x7fe65c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 -- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c0cd170 msgr2=0x7fe65c0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c0cd170 0x7fe65c0ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 -- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe65c15f150 msgr2=0x7fe65c15d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe65c15f150 0x7fe65c15d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:38.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 -- 172.21.15.181:0/2922646450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654009a70 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe661ca8700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0ce4f0 0x7fe65c15d270 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fe6580011e0 tx=0x7fe65800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.176+0000 7fe652ffd700 1 -- 172.21.15.181:0/2922646450 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658004c80 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.177+0000 7fe652ffd700 1 -- 172.21.15.181:0/2922646450 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe658004e00 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.177+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c0c7fe0 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.177+0000 7fe652ffd700 1 -- 172.21.15.181:0/2922646450 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658007310 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.177+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c1f6a50 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.177+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c067d20 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.178+0000 7fe652ffd700 1 -- 172.21.15.181:0/2922646450 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe658007470 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.178+0000 7fe652ffd700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe64006b020 0x7fe64006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:38.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.179+0000 7fe652ffd700 1 -- 172.21.15.181:0/2922646450 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5751+0+0 (secure 0 0 0) 0x7fe6580895d0 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.179+0000 7fe6624a9700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe64006b020 0x7fe64006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:38.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.179+0000 7fe6624a9700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe64006b020 0x7fe64006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe65c1f4380 tx=0x7fe64c008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:38.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.183+0000 7fe652ffd700 1 -- 172.21.15.181:0/2922646450 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe6580544a0 con 0x7fe65c0ce4f0 2023-10-16T11:17:38.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:38.373+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe65c0b2220 con 0x7fe65c0ce4f0 2023-10-16T11:17:39.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.028+0000 7fe652ffd700 1 -- 172.21.15.181:0/2922646450 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v197) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe658057ac0 con 0x7fe65c0ce4f0 2023-10-16T11:17:39.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe64006b020 msgr2=0x7fe64006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:39.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe64006b020 0x7fe64006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe65c1f4380 tx=0x7fe64c008040 comp rx=0 tx=0).stop 2023-10-16T11:17:39.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0ce4f0 msgr2=0x7fe65c15d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:39.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0ce4f0 0x7fe65c15d270 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fe6580011e0 tx=0x7fe65800cae0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 shutdown_connections 2023-10-16T11:17:39.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe64006b020 0x7fe64006d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe65c0cd170 0x7fe65c0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe65c15f150 0x7fe65c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 --2- 172.21.15.181:0/2922646450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c0ce4f0 0x7fe65c15d270 secure :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fe6580011e0 tx=0x7fe65800cae0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.033+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 >> 172.21.15.181:0/2922646450 conn(0x7fe65c09a930 msgr2=0x7fe65c06ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:39.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.034+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 shutdown_connections 2023-10-16T11:17:39.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.034+0000 7fe66470d700 1 -- 172.21.15.181:0/2922646450 wait complete. 2023-10-16T11:17:39.039 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T11:17:39.062 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:17:39.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.187+0000 7f9dc1a34700 1 Processor -- start 2023-10-16T11:17:39.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.187+0000 7f9dc1a34700 1 -- start start 2023-10-16T11:17:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dc1a34700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9dbc15f150 0x7f9dbc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dc1a34700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0cd170 0x7f9dbc0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dc1a34700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc0b2ae0 con 0x7f9dbc0ce4f0 2023-10-16T11:17:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dc1a34700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc0ce9a0 con 0x7f9dbc15f150 2023-10-16T11:17:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dc1a34700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0b2dc0 con 0x7f9dbc0cd170 2023-10-16T11:17:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dbaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9dbc15f150 0x7f9dbc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dbaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9dbc15f150 0x7f9dbc0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53850/0 (socket says 172.21.15.181:53850) 2023-10-16T11:17:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dbaffd700 1 -- 172.21.15.181:0/872729435 learned_addr learned my addr 172.21.15.181:0/872729435 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.188+0000 7f9dba7fc700 1 --1- 172.21.15.181:0/872729435 >> v1:172.21.15.181:6789/0 conn(0x7f9dbc0ce4f0 0x7f9dbc0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36694/0 (socket says 172.21.15.181:36694) 2023-10-16T11:17:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9dbb7fe700 1 --2- 172.21.15.181:0/872729435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0cd170 0x7f9dbc0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:39.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9dbaffd700 1 -- 172.21.15.181:0/872729435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0cd170 msgr2=0x7f9dbc0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:39.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9dbaffd700 1 --2- 172.21.15.181:0/872729435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0cd170 0x7f9dbc0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9dbaffd700 1 -- 172.21.15.181:0/872729435 >> v1:172.21.15.181:6789/0 conn(0x7f9dbc0ce4f0 legacy=0x7f9dbc0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:39.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9dbaffd700 1 -- 172.21.15.181:0/872729435 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0cf9d0 con 0x7f9dbc15f150 2023-10-16T11:17:39.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9db9ffb700 1 -- 172.21.15.181:0/872729435 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791950813 0 0) 0x7f9dbc0b2ae0 con 0x7f9dbc0ce4f0 2023-10-16T11:17:39.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9dbaffd700 1 --2- 172.21.15.181:0/872729435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc15f150 0x7f9dbc0cdfb0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009920 tx=0x7f9db0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46437838d609cab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:39.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9db9ffb700 1 -- 172.21.15.181:0/872729435 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011830 con 0x7f9dbc15f150 2023-10-16T11:17:39.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9db9ffb700 1 -- 172.21.15.181:0/872729435 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0011990 con 0x7f9dbc15f150 2023-10-16T11:17:39.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.189+0000 7f9db9ffb700 1 -- 172.21.15.181:0/872729435 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011d20 con 0x7f9dbc15f150 2023-10-16T11:17:39.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 -- 172.21.15.181:0/872729435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc15f150 msgr2=0x7f9dbc0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:39.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/872729435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc15f150 0x7f9dbc0cdfb0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009920 tx=0x7f9db0009c30 comp rx=0 tx=0).stop 2023-10-16T11:17:39.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 -- 172.21.15.181:0/872729435 shutdown_connections 2023-10-16T11:17:39.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/872729435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0cd170 0x7f9dbc0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/872729435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc15f150 0x7f9dbc0cdfb0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 -- 172.21.15.181:0/872729435 >> 172.21.15.181:0/872729435 conn(0x7f9dbc09a930 msgr2=0x7f9dbc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:39.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 -- 172.21.15.181:0/872729435 shutdown_connections 2023-10-16T11:17:39.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.190+0000 7f9dc1a34700 1 -- 172.21.15.181:0/872729435 wait complete. 2023-10-16T11:17:39.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.191+0000 7f9dc1a34700 1 Processor -- start 2023-10-16T11:17:39.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.191+0000 7f9dc1a34700 1 -- start start 2023-10-16T11:17:39.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc0cd170 0x7f9dbc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:39.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0ce4f0 0x7f9dbc151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:39.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dbc15f150 0x7f9dbc15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:39.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc1520c0 con 0x7f9dbc0cd170 2023-10-16T11:17:39.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0ca5a0 con 0x7f9dbc15f150 2023-10-16T11:17:39.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0c9990 con 0x7f9dbc0ce4f0 2023-10-16T11:17:39.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc0cd170 0x7f9dbc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:39.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc0cd170 0x7f9dbc0caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53854/0 (socket says 172.21.15.181:53854) 2023-10-16T11:17:39.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 -- 172.21.15.181:0/3506898898 learned_addr learned my addr 172.21.15.181:0/3506898898 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:39.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbb7fe700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dbc15f150 0x7f9dbc15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:39.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dba7fc700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0ce4f0 0x7f9dbc151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:39.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 -- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0ce4f0 msgr2=0x7f9dbc151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:39.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0ce4f0 0x7f9dbc151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 -- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dbc15f150 msgr2=0x7f9dbc15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:39.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dbc15f150 0x7f9dbc15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:39.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 -- 172.21.15.181:0/3506898898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0009550 con 0x7f9dbc0cd170 2023-10-16T11:17:39.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dbaffd700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc0cd170 0x7f9dbc0caa30 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f9db0000c00 tx=0x7f9db0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:39.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9da3fff700 1 -- 172.21.15.181:0/3506898898 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00038a0 con 0x7f9dbc0cd170 2023-10-16T11:17:39.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9da3fff700 1 -- 172.21.15.181:0/3506898898 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db001f070 con 0x7f9dbc0cd170 2023-10-16T11:17:39.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc0c9c70 con 0x7f9dbc0cd170 2023-10-16T11:17:39.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9da3fff700 1 -- 172.21.15.181:0/3506898898 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0003c40 con 0x7f9dbc0cd170 2023-10-16T11:17:39.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.192+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc0ca230 con 0x7f9dbc0cd170 2023-10-16T11:17:39.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.193+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc0ca3c0 con 0x7f9dbc0cd170 2023-10-16T11:17:39.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.194+0000 7f9da3fff700 1 -- 172.21.15.181:0/3506898898 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9db0026480 con 0x7f9dbc0cd170 2023-10-16T11:17:39.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.194+0000 7f9da3fff700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9da806b020 0x7f9da806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:39.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.194+0000 7f9da3fff700 1 -- 172.21.15.181:0/3506898898 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6133+0+0 (secure 0 0 0) 0x7f9db0021070 con 0x7f9dbc0cd170 2023-10-16T11:17:39.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.195+0000 7f9dba7fc700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9da806b020 0x7f9da806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:39.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.195+0000 7f9dba7fc700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9da806b020 0x7f9da806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0c8110 tx=0x7f9dac007be0 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:39.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.199+0000 7f9da3fff700 1 -- 172.21.15.181:0/3506898898 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9db0056160 con 0x7f9dbc0cd170 2023-10-16T11:17:39.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:39.434+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9dbc0c4f70 con 0x7f9dbc0cd170 2023-10-16T11:17:40.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.030+0000 7f9da3fff700 1 -- 172.21.15.181:0/3506898898 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v98) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9db00045b0 con 0x7f9dbc0cd170 2023-10-16T11:17:40.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9da806b020 msgr2=0x7f9da806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9da806b020 0x7f9da806d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0c8110 tx=0x7f9dac007be0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc0cd170 msgr2=0x7f9dbc0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc0cd170 0x7f9dbc0caa30 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f9db0000c00 tx=0x7f9db0004750 comp rx=0 tx=0).stop 2023-10-16T11:17:40.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 shutdown_connections 2023-10-16T11:17:40.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f9da806b020 0x7f9da806d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9dbc15f150 0x7f9dbc15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dbc0cd170 0x7f9dbc0caa30 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 --2- 172.21.15.181:0/3506898898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9dbc0ce4f0 0x7f9dbc151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 >> 172.21.15.181:0/3506898898 conn(0x7f9dbc09a930 msgr2=0x7f9dbc0ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:40.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 shutdown_connections 2023-10-16T11:17:40.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.033+0000 7f9dc1a34700 1 -- 172.21.15.181:0/3506898898 wait complete. 2023-10-16T11:17:40.040 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 24 and data pool 25 2023-10-16T11:17:40.053 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:40.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.172+0000 7fe7cdac7700 1 Processor -- start 2023-10-16T11:17:40.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.172+0000 7fe7cdac7700 1 -- start start 2023-10-16T11:17:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7cdac7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c815e6d0 0x7fe7c80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7cdac7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7c815eab0 0x7fe7c815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7cdac7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c806ed00 con 0x7fe7c80cb0c0 2023-10-16T11:17:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7cdac7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c80d02d0 con 0x7fe7c815eab0 2023-10-16T11:17:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7cdac7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c815ad40 con 0x7fe7c815e6d0 2023-10-16T11:17:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7c77fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c815e6d0 0x7fe7c80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7c6ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7c815eab0 0x7fe7c815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7c7fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe7c80cb0c0 0x7fe7c815a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:48898/0 (socket says 172.21.15.181:48898) 2023-10-16T11:17:40.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.173+0000 7fe7c7fff700 1 -- 172.21.15.181:0/3177346916 learned_addr learned my addr 172.21.15.181:0/3177346916 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:40.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c6ffd700 1 -- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c815e6d0 msgr2=0x7fe7c80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:40.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c6ffd700 1 --2- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c815e6d0 0x7fe7c80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c6ffd700 1 -- 172.21.15.181:0/3177346916 >> v1:172.21.15.190:6789/0 conn(0x7fe7c80cb0c0 legacy=0x7fe7c815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c67fc700 1 -- 172.21.15.181:0/3177346916 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795719449 0 0) 0x7fe7c806ed00 con 0x7fe7c80cb0c0 2023-10-16T11:17:40.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c6ffd700 1 -- 172.21.15.181:0/3177346916 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c806e0f0 con 0x7fe7c815eab0 2023-10-16T11:17:40.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c77fe700 1 --2- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c815e6d0 0x7fe7c80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:40.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c6ffd700 1 --2- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815eab0 0x7fe7c815a0f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009840 tx=0x7fe7b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba8979f6c52972ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c67fc700 1 -- 172.21.15.181:0/3177346916 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8011830 con 0x7fe7c815eab0 2023-10-16T11:17:40.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c67fc700 1 -- 172.21.15.181:0/3177346916 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8011990 con 0x7fe7c815eab0 2023-10-16T11:17:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.174+0000 7fe7c67fc700 1 -- 172.21.15.181:0/3177346916 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8011d20 con 0x7fe7c815eab0 2023-10-16T11:17:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 -- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815eab0 msgr2=0x7fe7c815a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815eab0 0x7fe7c815a0f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009840 tx=0x7fe7b8009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 -- 172.21.15.181:0/3177346916 shutdown_connections 2023-10-16T11:17:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c815e6d0 0x7fe7c80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/3177346916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815eab0 0x7fe7c815a0f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 -- 172.21.15.181:0/3177346916 >> 172.21.15.181:0/3177346916 conn(0x7fe7c80d04a0 msgr2=0x7fe7c80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 -- 172.21.15.181:0/3177346916 shutdown_connections 2023-10-16T11:17:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.175+0000 7fe7cdac7700 1 -- 172.21.15.181:0/3177346916 wait complete. 2023-10-16T11:17:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.176+0000 7fe7cdac7700 1 Processor -- start 2023-10-16T11:17:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.176+0000 7fe7cdac7700 1 -- start start 2023-10-16T11:17:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7cdac7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c80cb0c0 0x7fe7c80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7cdac7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815e6d0 0x7fe7c80c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7cdac7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7c815eab0 0x7fe7c80c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7cdac7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80b4d50 con 0x7fe7c815eab0 2023-10-16T11:17:40.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7cdac7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80b40d0 con 0x7fe7c815e6d0 2023-10-16T11:17:40.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7cdac7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80b43b0 con 0x7fe7c80cb0c0 2023-10-16T11:17:40.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7c815eab0 0x7fe7c80c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7c815eab0 0x7fe7c80c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53884/0 (socket says 172.21.15.181:53884) 2023-10-16T11:17:40.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 -- 172.21.15.181:0/4032795271 learned_addr learned my addr 172.21.15.181:0/4032795271 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:40.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c6ffd700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815e6d0 0x7fe7c80c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c77fe700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c80cb0c0 0x7fe7c80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 -- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c80cb0c0 msgr2=0x7fe7c80b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c80cb0c0 0x7fe7c80b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 -- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815e6d0 msgr2=0x7fe7c80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815e6d0 0x7fe7c80c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 -- 172.21.15.181:0/4032795271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b0009430 con 0x7fe7c815eab0 2023-10-16T11:17:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.177+0000 7fe7c7fff700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7c815eab0 0x7fe7c80c9c80 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000cba0 tx=0x7fe7c000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.178+0000 7fe7bffff700 1 -- 172.21.15.181:0/4032795271 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000aaa0 con 0x7fe7c815eab0 2023-10-16T11:17:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.178+0000 7fe7bffff700 1 -- 172.21.15.181:0/4032795271 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c000ac20 con 0x7fe7c815eab0 2023-10-16T11:17:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.178+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b80094a0 con 0x7fe7c815eab0 2023-10-16T11:17:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.178+0000 7fe7bffff700 1 -- 172.21.15.181:0/4032795271 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00070d0 con 0x7fe7c815eab0 2023-10-16T11:17:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.178+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c80b4970 con 0x7fe7c815eab0 2023-10-16T11:17:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.179+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c8067d20 con 0x7fe7c815eab0 2023-10-16T11:17:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.179+0000 7fe7bffff700 1 -- 172.21.15.181:0/4032795271 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe7c0007230 con 0x7fe7c815eab0 2023-10-16T11:17:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.179+0000 7fe7bffff700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe7a806b020 0x7fe7a806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.180+0000 7fe7c77fe700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe7a806b020 0x7fe7a806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.180+0000 7fe7bffff700 1 -- 172.21.15.181:0/4032795271 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6233+0+0 (secure 0 0 0) 0x7fe7c00896e0 con 0x7fe7c815eab0 2023-10-16T11:17:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.180+0000 7fe7c77fe700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe7a806b020 0x7fe7a806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009cb0 tx=0x7fe7b00091c0 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.184+0000 7fe7bffff700 1 -- 172.21.15.181:0/4032795271 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe7c0054470 con 0x7fe7c815eab0 2023-10-16T11:17:40.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.379+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7c80cb610 con 0x7fe7c815eab0 2023-10-16T11:17:40.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.380+0000 7fe7bffff700 1 -- 172.21.15.181:0/4032795271 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17434 (secure 0 0 0) 0x7fe7c0057a90 con 0x7fe7c815eab0 2023-10-16T11:17:40.383 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:40.383 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":198,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:39.436157+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:17:37.851564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:17:38.394398+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:40.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe7a806b020 msgr2=0x7fe7a806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe7a806b020 0x7fe7a806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009cb0 tx=0x7fe7b00091c0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7c815eab0 msgr2=0x7fe7c80c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7c815eab0 0x7fe7c80c9c80 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000cba0 tx=0x7fe7c000ceb0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 shutdown_connections 2023-10-16T11:17:40.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fe7a806b020 0x7fe7a806d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7c80cb0c0 0x7fe7c80b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7c815eab0 0x7fe7c80c9c80 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 --2- 172.21.15.181:0/4032795271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c815e6d0 0x7fe7c80c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.382+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 >> 172.21.15.181:0/4032795271 conn(0x7fe7c80d04a0 msgr2=0x7fe7c806f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:40.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.383+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 shutdown_connections 2023-10-16T11:17:40.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.383+0000 7fe7cdac7700 1 -- 172.21.15.181:0/4032795271 wait complete. 2023-10-16T11:17:40.397 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:40.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.515+0000 7f3413387700 1 Processor -- start 2023-10-16T11:17:40.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.515+0000 7f3413387700 1 -- start start 2023-10-16T11:17:40.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3413387700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f340c0d0370 0x7f340c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3413387700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0cc510 con 0x7f340c0cd2d0 2023-10-16T11:17:40.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3413387700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0cc7f0 con 0x7f340c15b990 2023-10-16T11:17:40.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3413387700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c0ccad0 con 0x7f340c0d0370 2023-10-16T11:17:40.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3410922700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f340c0d0370 0x7f340c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3411924700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f340c0cd2d0 0x7f340c0ae290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36736/0 (socket says 172.21.15.181:36736) 2023-10-16T11:17:40.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3411924700 1 -- 172.21.15.181:0/2678597487 learned_addr learned my addr 172.21.15.181:0/2678597487 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:40.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3410922700 1 -- 172.21.15.181:0/2678597487 >> v1:172.21.15.190:6789/0 conn(0x7f340c15b990 legacy=0x7f340c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:40.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3410922700 1 -- 172.21.15.181:0/2678597487 >> v1:172.21.15.181:6789/0 conn(0x7f340c0cd2d0 legacy=0x7f340c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3403fff700 1 -- 172.21.15.181:0/2678597487 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974980355 0 0) 0x7f340c0cc510 con 0x7f340c0cd2d0 2023-10-16T11:17:40.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3410922700 1 -- 172.21.15.181:0/2678597487 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0ccdb0 con 0x7f340c0d0370 2023-10-16T11:17:40.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.516+0000 7f3410922700 1 --2- 172.21.15.181:0/2678597487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c0d0370 0x7f340c0ce8f0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009840 tx=0x7f33fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e7760d28538b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3403fff700 1 -- 172.21.15.181:0/2678597487 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc004230 con 0x7f340c0d0370 2023-10-16T11:17:40.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3403fff700 1 -- 172.21.15.181:0/2678597487 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc004390 con 0x7f340c0d0370 2023-10-16T11:17:40.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3403fff700 1 -- 172.21.15.181:0/2678597487 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0036a0 con 0x7f340c0d0370 2023-10-16T11:17:40.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3413387700 1 -- 172.21.15.181:0/2678597487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c0d0370 msgr2=0x7f340c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3413387700 1 --2- 172.21.15.181:0/2678597487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c0d0370 0x7f340c0ce8f0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009840 tx=0x7f33fc009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:40.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3413387700 1 -- 172.21.15.181:0/2678597487 shutdown_connections 2023-10-16T11:17:40.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3413387700 1 --2- 172.21.15.181:0/2678597487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c0d0370 0x7f340c0ce8f0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3413387700 1 -- 172.21.15.181:0/2678597487 >> 172.21.15.181:0/2678597487 conn(0x7f340c09a9a0 msgr2=0x7f340c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:40.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3413387700 1 -- 172.21.15.181:0/2678597487 shutdown_connections 2023-10-16T11:17:40.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.517+0000 7f3413387700 1 -- 172.21.15.181:0/2678597487 wait complete. 2023-10-16T11:17:40.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.518+0000 7f3413387700 1 Processor -- start 2023-10-16T11:17:40.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.518+0000 7f3413387700 1 -- start start 2023-10-16T11:17:40.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3413387700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f340c0cd2d0 0x7f340c14b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3413387700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f340c0d0370 0x7f340c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3413387700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c15b990 0x7f340c1477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3413387700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c14b640 con 0x7f340c15b990 2023-10-16T11:17:40.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3413387700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0b7170 con 0x7f340c0cd2d0 2023-10-16T11:17:40.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3413387700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c14bb00 con 0x7f340c0d0370 2023-10-16T11:17:40.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3411924700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c15b990 0x7f340c1477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3411924700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c15b990 0x7f340c1477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53896/0 (socket says 172.21.15.181:53896) 2023-10-16T11:17:40.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3411924700 1 -- 172.21.15.181:0/4141691432 learned_addr learned my addr 172.21.15.181:0/4141691432 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:40.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3410922700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f340c0d0370 0x7f340c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.519+0000 7f3411123700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f340c0cd2d0 0x7f340c14b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3411924700 1 -- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f340c0d0370 msgr2=0x7f340c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3411924700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f340c0d0370 0x7f340c0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3411924700 1 -- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f340c0cd2d0 msgr2=0x7f340c14b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3411924700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f340c0cd2d0 0x7f340c14b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3411924700 1 -- 172.21.15.181:0/4141691432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0094a0 con 0x7f340c15b990 2023-10-16T11:17:40.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3411924700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c15b990 0x7f340c1477e0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f340400cb60 tx=0x7f340400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3401ffb700 1 -- 172.21.15.181:0/4141691432 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400a9e0 con 0x7f340c15b990 2023-10-16T11:17:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3401ffb700 1 -- 172.21.15.181:0/4141691432 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400ab40 con 0x7f340c15b990 2023-10-16T11:17:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3401ffb700 1 -- 172.21.15.181:0/4141691432 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040070d0 con 0x7f340c15b990 2023-10-16T11:17:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c0c90b0 con 0x7f340c15b990 2023-10-16T11:17:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.520+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c0b6840 con 0x7f340c15b990 2023-10-16T11:17:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.521+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c067d20 con 0x7f340c15b990 2023-10-16T11:17:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.521+0000 7f3401ffb700 1 -- 172.21.15.181:0/4141691432 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3404007230 con 0x7f340c15b990 2023-10-16T11:17:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.522+0000 7f3401ffb700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f33f006b020 0x7f33f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.522+0000 7f3401ffb700 1 -- 172.21.15.181:0/4141691432 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f34040892d0 con 0x7f340c15b990 2023-10-16T11:17:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.522+0000 7f3411123700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f33f006b020 0x7f33f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.522+0000 7f3411123700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f33f006b020 0x7f33f006d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34080050c0 tx=0x7f3408007f10 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.526+0000 7f3401ffb700 1 -- 172.21.15.181:0/4141691432 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f34040540f0 con 0x7f340c15b990 2023-10-16T11:17:40.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.720+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f340c0d1d90 con 0x7f340c15b990 2023-10-16T11:17:40.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.721+0000 7f3401ffb700 1 -- 172.21.15.181:0/4141691432 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17434 (secure 0 0 0) 0x7f3404057710 con 0x7f340c15b990 2023-10-16T11:17:40.723 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:40.724 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":198,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:39.436157+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:17:37.851564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:17:38.394398+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:40.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f33f006b020 msgr2=0x7f33f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f33f006b020 0x7f33f006d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34080050c0 tx=0x7f3408007f10 comp rx=0 tx=0).stop 2023-10-16T11:17:40.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c15b990 msgr2=0x7f340c1477e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c15b990 0x7f340c1477e0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f340400cb60 tx=0x7f340400ce70 comp rx=0 tx=0).stop 2023-10-16T11:17:40.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 shutdown_connections 2023-10-16T11:17:40.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f33f006b020 0x7f33f006d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f340c15b990 0x7f340c1477e0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f340c0cd2d0 0x7f340c14b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 --2- 172.21.15.181:0/4141691432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f340c0d0370 0x7f340c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 >> 172.21.15.181:0/4141691432 conn(0x7f340c09a9a0 msgr2=0x7f340c158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:40.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 shutdown_connections 2023-10-16T11:17:40.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.723+0000 7f3413387700 1 -- 172.21.15.181:0/4141691432 wait complete. 2023-10-16T11:17:40.738 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:17:40.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.859+0000 7f63197e5700 1 Processor -- start 2023-10-16T11:17:40.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.860+0000 7f63197e5700 1 -- start start 2023-10-16T11:17:40.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.860+0000 7f63197e5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f631415a5f0 0x7f63140cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.860+0000 7f63197e5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63140cd570 0x7f6314072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.860+0000 7f63197e5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631415b510 con 0x7f63140cd0c0 2023-10-16T11:17:40.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.860+0000 7f63197e5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63140720e0 con 0x7f63140cd570 2023-10-16T11:17:40.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.860+0000 7f63197e5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63140cbc90 con 0x7f631415a5f0 2023-10-16T11:17:40.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63137fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63140cd570 0x7f6314072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63127fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f63140cd0c0 0x7f631415a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36746/0 (socket says 172.21.15.181:36746) 2023-10-16T11:17:40.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63127fc700 1 -- 172.21.15.181:0/792915169 learned_addr learned my addr 172.21.15.181:0/792915169 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:40.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f6312ffd700 1 --2- 172.21.15.181:0/792915169 >> v2:172.21.15.190:3300/0 conn(0x7f631415a5f0 0x7f63140cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63137fe700 1 -- 172.21.15.181:0/792915169 >> v2:172.21.15.190:3300/0 conn(0x7f631415a5f0 msgr2=0x7f63140cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:40.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63137fe700 1 --2- 172.21.15.181:0/792915169 >> v2:172.21.15.190:3300/0 conn(0x7f631415a5f0 0x7f63140cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63137fe700 1 -- 172.21.15.181:0/792915169 >> v1:172.21.15.181:6789/0 conn(0x7f63140cd0c0 legacy=0x7f631415a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f6311ffb700 1 -- 172.21.15.181:0/792915169 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172712560 0 0) 0x7f631415b510 con 0x7f63140cd0c0 2023-10-16T11:17:40.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63137fe700 1 -- 172.21.15.181:0/792915169 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631415b7f0 con 0x7f63140cd570 2023-10-16T11:17:40.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f63137fe700 1 --2- 172.21.15.181:0/792915169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd570 0x7f6314072500 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f63140caf10 tx=0x7f630c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e580d7f345c3413 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f6311ffb700 1 -- 172.21.15.181:0/792915169 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c00e070 con 0x7f63140cd570 2023-10-16T11:17:40.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f6311ffb700 1 -- 172.21.15.181:0/792915169 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c00bae0 con 0x7f63140cd570 2023-10-16T11:17:40.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.861+0000 7f6311ffb700 1 -- 172.21.15.181:0/792915169 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c00be90 con 0x7f63140cd570 2023-10-16T11:17:40.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 -- 172.21.15.181:0/792915169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd570 msgr2=0x7f6314072500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 --2- 172.21.15.181:0/792915169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd570 0x7f6314072500 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f63140caf10 tx=0x7f630c00b330 comp rx=0 tx=0).stop 2023-10-16T11:17:40.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 -- 172.21.15.181:0/792915169 shutdown_connections 2023-10-16T11:17:40.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 --2- 172.21.15.181:0/792915169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd570 0x7f6314072500 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 --2- 172.21.15.181:0/792915169 >> v2:172.21.15.190:3300/0 conn(0x7f631415a5f0 0x7f63140cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 -- 172.21.15.181:0/792915169 >> 172.21.15.181:0/792915169 conn(0x7f631406aed0 msgr2=0x7f631406cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:40.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 -- 172.21.15.181:0/792915169 shutdown_connections 2023-10-16T11:17:40.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.862+0000 7f63197e5700 1 -- 172.21.15.181:0/792915169 wait complete. 2023-10-16T11:17:40.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.864+0000 7f63197e5700 1 Processor -- start 2023-10-16T11:17:40.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.864+0000 7f63197e5700 1 -- start start 2023-10-16T11:17:40.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.864+0000 7f63197e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd0c0 0x7f63140c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63197e5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63140cd570 0x7f631415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63197e5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f631415a5f0 0x7f63141570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63197e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140c3020 con 0x7f63140cd0c0 2023-10-16T11:17:40.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63197e5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631415daa0 con 0x7f631415a5f0 2023-10-16T11:17:40.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63197e5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63140c3270 con 0x7f63140cd570 2023-10-16T11:17:40.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd0c0 0x7f63140c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd0c0 0x7f63140c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53906/0 (socket says 172.21.15.181:53906) 2023-10-16T11:17:40.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 -- 172.21.15.181:0/2602626259 learned_addr learned my addr 172.21.15.181:0/2602626259 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:40.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63127fc700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63140cd570 0x7f631415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 -- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63140cd570 msgr2=0x7f631415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63140cd570 0x7f631415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63137fe700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f631415a5f0 0x7f63141570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 -- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f631415a5f0 msgr2=0x7f63141570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:40.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f631415a5f0 0x7f63141570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:40.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 -- 172.21.15.181:0/2602626259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009a70 con 0x7f63140cd0c0 2023-10-16T11:17:40.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f6312ffd700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd0c0 0x7f63140c4310 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f630800e8a0 tx=0x7f630800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f62fb7fe700 1 -- 172.21.15.181:0/2602626259 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308009590 con 0x7f63140cd0c0 2023-10-16T11:17:40.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f62fb7fe700 1 -- 172.21.15.181:0/2602626259 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630800eea0 con 0x7f63140cd0c0 2023-10-16T11:17:40.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63140c3550 con 0x7f63140cd0c0 2023-10-16T11:17:40.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f62fb7fe700 1 -- 172.21.15.181:0/2602626259 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800ca50 con 0x7f63140cd0c0 2023-10-16T11:17:40.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.865+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63140c3b10 con 0x7f63140cd0c0 2023-10-16T11:17:40.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.866+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631404ebb0 con 0x7f63140cd0c0 2023-10-16T11:17:40.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.867+0000 7f62fb7fe700 1 -- 172.21.15.181:0/2602626259 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f630800cbb0 con 0x7f63140cd0c0 2023-10-16T11:17:40.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.867+0000 7f62fb7fe700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f630006b020 0x7f630006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:40.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.867+0000 7f62fb7fe700 1 -- 172.21.15.181:0/2602626259 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f6308089910 con 0x7f63140cd0c0 2023-10-16T11:17:40.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.867+0000 7f63127fc700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f630006b020 0x7f630006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:40.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.868+0000 7f63127fc700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f630006b020 0x7f630006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63140cb610 tx=0x7f6304009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:40.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:40.872+0000 7f62fb7fe700 1 -- 172.21.15.181:0/2602626259 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63080546a0 con 0x7f63140cd0c0 2023-10-16T11:17:41.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:41.117+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f631415ad10 con 0x7f63140cd0c0 2023-10-16T11:17:42.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.050+0000 7f62fb7fe700 1 -- 172.21.15.181:0/2602626259 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) v1 ==== 118+0+0 (secure 0 0 0) 0x7f63080210b0 con 0x7f63140cd0c0 2023-10-16T11:17:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f630006b020 msgr2=0x7f630006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f630006b020 0x7f630006d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f63140cb610 tx=0x7f6304009040 comp rx=0 tx=0).stop 2023-10-16T11:17:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd0c0 msgr2=0x7f63140c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd0c0 0x7f63140c4310 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f630800e8a0 tx=0x7f630800ec60 comp rx=0 tx=0).stop 2023-10-16T11:17:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 shutdown_connections 2023-10-16T11:17:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f630006b020 0x7f630006d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f631415a5f0 0x7f63141570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63140cd0c0 0x7f63140c4310 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 --2- 172.21.15.181:0/2602626259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63140cd570 0x7f631415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 >> 172.21.15.181:0/2602626259 conn(0x7f631406aed0 msgr2=0x7f63140b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:42.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 shutdown_connections 2023-10-16T11:17:42.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.053+0000 7f63197e5700 1 -- 172.21.15.181:0/2602626259 wait complete. 2023-10-16T11:17:42.070 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:17:42.071 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:17:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.199+0000 7f2b83679700 1 Processor -- start 2023-10-16T11:17:42.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.199+0000 7f2b83679700 1 -- start start 2023-10-16T11:17:42.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b83679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0b2010 0x7f2b7c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b83679700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2b7c0ce680 0x7f2b7c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b83679700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c0ce1f0 con 0x7f2b7c071be0 2023-10-16T11:17:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b83679700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c0ccda0 con 0x7f2b7c0ce680 2023-10-16T11:17:42.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b83679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0727e0 con 0x7f2b7c0b2010 2023-10-16T11:17:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b81c16700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2b7c0ce680 0x7f2b7c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b80c14700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2b7c071be0 0x7f2b7c0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36762/0 (socket says 172.21.15.181:36762) 2023-10-16T11:17:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b81c16700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2b7c0ce680 0x7f2b7c0ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53918/0 (socket says 172.21.15.181:53918) 2023-10-16T11:17:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b80c14700 1 -- 172.21.15.181:0/1711537795 learned_addr learned my addr 172.21.15.181:0/1711537795 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b81c16700 1 -- 172.21.15.181:0/1711537795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0b2010 msgr2=0x7f2b7c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:42.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b81c16700 1 --2- 172.21.15.181:0/1711537795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0b2010 0x7f2b7c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b81c16700 1 -- 172.21.15.181:0/1711537795 >> v1:172.21.15.181:6789/0 conn(0x7f2b7c071be0 legacy=0x7f2b7c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:42.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b73fff700 1 -- 172.21.15.181:0/1711537795 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211273688 0 0) 0x7f2b7c0ce1f0 con 0x7f2b7c071be0 2023-10-16T11:17:42.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b81c16700 1 -- 172.21.15.181:0/1711537795 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c072ac0 con 0x7f2b7c0ce680 2023-10-16T11:17:42.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.200+0000 7f2b81c16700 1 --2- 172.21.15.181:0/1711537795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c0ce680 0x7f2b7c0ceb10 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009d90 tx=0x7f2b7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b681715f00f6e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:42.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b73fff700 1 -- 172.21.15.181:0/1711537795 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400e070 con 0x7f2b7c0ce680 2023-10-16T11:17:42.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b73fff700 1 -- 172.21.15.181:0/1711537795 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7400bbd0 con 0x7f2b7c0ce680 2023-10-16T11:17:42.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b73fff700 1 -- 172.21.15.181:0/1711537795 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74012430 con 0x7f2b7c0ce680 2023-10-16T11:17:42.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 -- 172.21.15.181:0/1711537795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c0ce680 msgr2=0x7f2b7c0ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:42.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 --2- 172.21.15.181:0/1711537795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c0ce680 0x7f2b7c0ceb10 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009d90 tx=0x7f2b7400b330 comp rx=0 tx=0).stop 2023-10-16T11:17:42.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 -- 172.21.15.181:0/1711537795 shutdown_connections 2023-10-16T11:17:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 --2- 172.21.15.181:0/1711537795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c0ce680 0x7f2b7c0ceb10 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 --2- 172.21.15.181:0/1711537795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0b2010 0x7f2b7c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 -- 172.21.15.181:0/1711537795 >> 172.21.15.181:0/1711537795 conn(0x7f2b7c0cd010 msgr2=0x7f2b7c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 -- 172.21.15.181:0/1711537795 shutdown_connections 2023-10-16T11:17:42.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.201+0000 7f2b83679700 1 -- 172.21.15.181:0/1711537795 wait complete. 2023-10-16T11:17:42.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.202+0000 7f2b83679700 1 Processor -- start 2023-10-16T11:17:42.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.202+0000 7f2b83679700 1 -- start start 2023-10-16T11:17:42.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b83679700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c071be0 0x7f2b7c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:42.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b83679700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b7c0b2010 0x7f2b7c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:42.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b83679700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0ce680 0x7f2b7c158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:42.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b83679700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1f0fd0 con 0x7f2b7c071be0 2023-10-16T11:17:42.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b83679700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1f1210 con 0x7f2b7c0b2010 2023-10-16T11:17:42.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b83679700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1f14f0 con 0x7f2b7c0ce680 2023-10-16T11:17:42.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c071be0 0x7f2b7c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:42.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c071be0 0x7f2b7c153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53922/0 (socket says 172.21.15.181:53922) 2023-10-16T11:17:42.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 -- 172.21.15.181:0/1612147068 learned_addr learned my addr 172.21.15.181:0/1612147068 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:42.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b80c14700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b7c0b2010 0x7f2b7c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:42.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81c16700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0ce680 0x7f2b7c158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:42.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 -- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0ce680 msgr2=0x7f2b7c158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:42.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0ce680 0x7f2b7c158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 -- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b7c0b2010 msgr2=0x7f2b7c1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:42.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b7c0b2010 0x7f2b7c1589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:42.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.203+0000 7f2b81415700 1 -- 172.21.15.181:0/1612147068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b74009a70 con 0x7f2b7c071be0 2023-10-16T11:17:42.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.204+0000 7f2b81415700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c071be0 0x7f2b7c153a40 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800b530 tx=0x7f2b7800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:42.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.204+0000 7f2b71ffb700 1 -- 172.21.15.181:0/1612147068 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78011830 con 0x7f2b7c071be0 2023-10-16T11:17:42.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.204+0000 7f2b71ffb700 1 -- 172.21.15.181:0/1612147068 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b780119b0 con 0x7f2b7c071be0 2023-10-16T11:17:42.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.204+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c1f17d0 con 0x7f2b7c071be0 2023-10-16T11:17:42.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.204+0000 7f2b71ffb700 1 -- 172.21.15.181:0/1612147068 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78011d60 con 0x7f2b7c071be0 2023-10-16T11:17:42.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.204+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c1f1d90 con 0x7f2b7c071be0 2023-10-16T11:17:42.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.205+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c1f1fb0 con 0x7f2b7c071be0 2023-10-16T11:17:42.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.205+0000 7f2b71ffb700 1 -- 172.21.15.181:0/1612147068 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2b7801a400 con 0x7f2b7c071be0 2023-10-16T11:17:42.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.206+0000 7f2b71ffb700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f2b6006b020 0x7f2b6006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:42.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.206+0000 7f2b71ffb700 1 -- 172.21.15.181:0/1612147068 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f2b780899a0 con 0x7f2b7c071be0 2023-10-16T11:17:42.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.206+0000 7f2b80c14700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f2b6006b020 0x7f2b6006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:42.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.206+0000 7f2b80c14700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f2b6006b020 0x7f2b6006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c159680 tx=0x7f2b6c007c90 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:42.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.210+0000 7f2b71ffb700 1 -- 172.21.15.181:0/1612147068 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2b780547c0 con 0x7f2b7c071be0 2023-10-16T11:17:42.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:42.453+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2b7c0b6510 con 0x7f2b7c071be0 2023-10-16T11:17:43.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.055+0000 7f2b71ffb700 1 -- 172.21.15.181:0/1612147068 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v102) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b78015020 con 0x7f2b7c071be0 2023-10-16T11:17:43.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f2b6006b020 msgr2=0x7f2b6006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f2b6006b020 0x7f2b6006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c159680 tx=0x7f2b6c007c90 comp rx=0 tx=0).stop 2023-10-16T11:17:43.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c071be0 msgr2=0x7f2b7c153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c071be0 0x7f2b7c153a40 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800b530 tx=0x7f2b7800b8f0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 shutdown_connections 2023-10-16T11:17:43.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f2b6006b020 0x7f2b6006d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7c0ce680 0x7f2b7c158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7c071be0 0x7f2b7c153a40 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 --2- 172.21.15.181:0/1612147068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b7c0b2010 0x7f2b7c1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 >> 172.21.15.181:0/1612147068 conn(0x7f2b7c0cd010 msgr2=0x7f2b7c0cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:43.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 shutdown_connections 2023-10-16T11:17:43.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.058+0000 7f2b83679700 1 -- 172.21.15.181:0/1612147068 wait complete. 2023-10-16T11:17:43.073 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:17:43.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:17:43.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.195+0000 7f1cdc546700 1 Processor -- start 2023-10-16T11:17:43.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.195+0000 7f1cdc546700 1 -- start start 2023-10-16T11:17:43.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cdc546700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1cd415db70 0x7f1cd40665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cdc546700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1cd40cedd0 0x7f1cd40cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cdc546700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd4066c60 con 0x7f1cd4161780 2023-10-16T11:17:43.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cdc546700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd40cda10 con 0x7f1cd415db70 2023-10-16T11:17:43.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cdc546700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd406c1a0 con 0x7f1cd40cedd0 2023-10-16T11:17:43.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cda2e2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1cd415db70 0x7f1cd40665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cdaae3700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1cd4161780 0x7f1cd4161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36778/0 (socket says 172.21.15.181:36778) 2023-10-16T11:17:43.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cda2e2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1cd415db70 0x7f1cd40665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53928/0 (socket says 172.21.15.181:53928) 2023-10-16T11:17:43.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cdaae3700 1 -- 172.21.15.181:0/1776269724 learned_addr learned my addr 172.21.15.181:0/1776269724 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:43.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cd9ae1700 1 --2- 172.21.15.181:0/1776269724 >> v2:172.21.15.190:3300/0 conn(0x7f1cd40cedd0 0x7f1cd40cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cda2e2700 1 -- 172.21.15.181:0/1776269724 >> v2:172.21.15.190:3300/0 conn(0x7f1cd40cedd0 msgr2=0x7f1cd40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:43.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cda2e2700 1 --2- 172.21.15.181:0/1776269724 >> v2:172.21.15.190:3300/0 conn(0x7f1cd40cedd0 0x7f1cd40cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cda2e2700 1 -- 172.21.15.181:0/1776269724 >> v1:172.21.15.181:6789/0 conn(0x7f1cd4161780 legacy=0x7f1cd4161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cda2e2700 1 -- 172.21.15.181:0/1776269724 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd406b3e0 con 0x7f1cd415db70 2023-10-16T11:17:43.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cd92e0700 1 -- 172.21.15.181:0/1776269724 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563496063 0 0) 0x7f1cd4066c60 con 0x7f1cd4161780 2023-10-16T11:17:43.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cda2e2700 1 --2- 172.21.15.181:0/1776269724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd415db70 0x7f1cd40665a0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4074710 tx=0x7f1cd0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26deb05930d286d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cd92e0700 1 -- 172.21.15.181:0/1776269724 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0011830 con 0x7f1cd415db70 2023-10-16T11:17:43.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cd92e0700 1 -- 172.21.15.181:0/1776269724 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd0011990 con 0x7f1cd415db70 2023-10-16T11:17:43.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.196+0000 7f1cd92e0700 1 -- 172.21.15.181:0/1776269724 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0011d20 con 0x7f1cd415db70 2023-10-16T11:17:43.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 -- 172.21.15.181:0/1776269724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd415db70 msgr2=0x7f1cd40665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 --2- 172.21.15.181:0/1776269724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd415db70 0x7f1cd40665a0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4074710 tx=0x7f1cd0009b20 comp rx=0 tx=0).stop 2023-10-16T11:17:43.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 -- 172.21.15.181:0/1776269724 shutdown_connections 2023-10-16T11:17:43.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 --2- 172.21.15.181:0/1776269724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd415db70 0x7f1cd40665a0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 --2- 172.21.15.181:0/1776269724 >> v2:172.21.15.190:3300/0 conn(0x7f1cd40cedd0 0x7f1cd40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 -- 172.21.15.181:0/1776269724 >> 172.21.15.181:0/1776269724 conn(0x7f1cd40b1780 msgr2=0x7f1cd40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:43.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 -- 172.21.15.181:0/1776269724 shutdown_connections 2023-10-16T11:17:43.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.197+0000 7f1cdc546700 1 -- 172.21.15.181:0/1776269724 wait complete. 2023-10-16T11:17:43.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.198+0000 7f1cdc546700 1 Processor -- start 2023-10-16T11:17:43.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.198+0000 7f1cdc546700 1 -- start start 2023-10-16T11:17:43.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdc546700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cd40cedd0 0x7f1cd40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdc546700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cd415db70 0x7f1cd4160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdc546700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd4161780 0x7f1cd40ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdc546700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd41f3520 con 0x7f1cd4161780 2023-10-16T11:17:43.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdc546700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd41f3800 con 0x7f1cd415db70 2023-10-16T11:17:43.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdc546700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd41f3ae0 con 0x7f1cd40cedd0 2023-10-16T11:17:43.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd4161780 0x7f1cd40ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd4161780 0x7f1cd40ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53938/0 (socket says 172.21.15.181:53938) 2023-10-16T11:17:43.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 -- 172.21.15.181:0/3954562745 learned_addr learned my addr 172.21.15.181:0/3954562745 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:43.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cd9ae1700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cd415db70 0x7f1cd4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cda2e2700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cd40cedd0 0x7f1cd40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 -- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cd40cedd0 msgr2=0x7f1cd40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cd40cedd0 0x7f1cd40c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 -- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cd415db70 msgr2=0x7f1cd4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cd415db70 0x7f1cd4160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 -- 172.21.15.181:0/3954562745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd00094a0 con 0x7f1cd4161780 2023-10-16T11:17:43.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.199+0000 7f1cdaae3700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd4161780 0x7f1cd40ca430 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00cb40 tx=0x7f1ccc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.200+0000 7f1ccaffd700 1 -- 172.21.15.181:0/3954562745 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc00ab50 con 0x7f1cd4161780 2023-10-16T11:17:43.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.200+0000 7f1ccaffd700 1 -- 172.21.15.181:0/3954562745 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc00acd0 con 0x7f1cd4161780 2023-10-16T11:17:43.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.200+0000 7f1ccaffd700 1 -- 172.21.15.181:0/3954562745 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc007310 con 0x7f1cd4161780 2023-10-16T11:17:43.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.200+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd41f3dc0 con 0x7f1cd4161780 2023-10-16T11:17:43.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.200+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd40c8e70 con 0x7f1cd4161780 2023-10-16T11:17:43.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.200+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd41f67c0 con 0x7f1cd4161780 2023-10-16T11:17:43.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.201+0000 7f1ccaffd700 1 -- 172.21.15.181:0/3954562745 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1ccc007470 con 0x7f1cd4161780 2023-10-16T11:17:43.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.201+0000 7f1ccaffd700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1cb806b020 0x7f1cb806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.201+0000 7f1ccaffd700 1 -- 172.21.15.181:0/3954562745 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f1ccc089490 con 0x7f1cd4161780 2023-10-16T11:17:43.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.201+0000 7f1cda2e2700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1cb806b020 0x7f1cb806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.202+0000 7f1cda2e2700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1cb806b020 0x7f1cb806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00090b0 tx=0x7f1cd0008030 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.206+0000 7f1ccaffd700 1 -- 172.21.15.181:0/3954562745 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1cd41f67c0 con 0x7f1cd4161780 2023-10-16T11:17:43.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.440+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cd41f67c0 con 0x7f1cd4161780 2023-10-16T11:17:43.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.441+0000 7f1ccaffd700 1 -- 172.21.15.181:0/3954562745 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f1cd41f67c0 con 0x7f1cd4161780 2023-10-16T11:17:43.443 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:43.443 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3757723265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3757723265},{"type":"v1","addr":"172.21.15.190:6834","nonce":3757723265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2012626771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2012626771},{"type":"v1","addr":"172.21.15.190:6835","nonce":2012626771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10782,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3218502962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3218502962},{"type":"v1","addr":"172.21.15.181:6836","nonce":3218502962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:17:39.436499+0000","modified":"2023-10-16T11:17:42.454721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10785":{"gid":10785,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3625484950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3625484950},{"type":"v1","addr":"172.21.15.181:6837","nonce":3625484950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:17:43.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1cb806b020 msgr2=0x7f1cb806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1cb806b020 0x7f1cb806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00090b0 tx=0x7f1cd0008030 comp rx=0 tx=0).stop 2023-10-16T11:17:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd4161780 msgr2=0x7f1cd40ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd4161780 0x7f1cd40ca430 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00cb40 tx=0x7f1ccc00cf00 comp rx=0 tx=0).stop 2023-10-16T11:17:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 shutdown_connections 2023-10-16T11:17:43.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1cb806b020 0x7f1cb806d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cd40cedd0 0x7f1cd40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd4161780 0x7f1cd40ca430 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 --2- 172.21.15.181:0/3954562745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cd415db70 0x7f1cd4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 >> 172.21.15.181:0/3954562745 conn(0x7f1cd40b1780 msgr2=0x7f1cd40c2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:43.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 shutdown_connections 2023-10-16T11:17:43.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.443+0000 7f1cdc546700 1 -- 172.21.15.181:0/3954562745 wait complete. 2023-10-16T11:17:43.448 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 102 2023-10-16T11:17:43.457 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:17:43.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.575+0000 7f70f852f700 1 Processor -- start 2023-10-16T11:17:43.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.576+0000 7f70f852f700 1 -- start start 2023-10-16T11:17:43.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.576+0000 7f70f852f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f015b990 0x7f70f00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.576+0000 7f70f852f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70f00d0370 0x7f70f00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f852f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f00cc510 con 0x7f70f00cd2d0 2023-10-16T11:17:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f852f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f00d24f0 con 0x7f70f00d0370 2023-10-16T11:17:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f852f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00cc7f0 con 0x7f70f015b990 2023-10-16T11:17:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f6acc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70f00d0370 0x7f70f00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f5aca700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f70f00cd2d0 0x7f70f00cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36790/0 (socket says 172.21.15.181:36790) 2023-10-16T11:17:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f5aca700 1 -- 172.21.15.181:0/491551484 learned_addr learned my addr 172.21.15.181:0/491551484 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f6acc700 1 -- 172.21.15.181:0/491551484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f015b990 msgr2=0x7f70f00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f62cb700 1 --2- 172.21.15.181:0/491551484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f015b990 0x7f70f00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f52c9700 1 -- 172.21.15.181:0/491551484 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965016851 0 0) 0x7f70f00cc510 con 0x7f70f00cd2d0 2023-10-16T11:17:43.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f6acc700 1 --2- 172.21.15.181:0/491551484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f015b990 0x7f70f00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f6acc700 1 -- 172.21.15.181:0/491551484 >> v1:172.21.15.181:6789/0 conn(0x7f70f00cd2d0 legacy=0x7f70f00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f6acc700 1 -- 172.21.15.181:0/491551484 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00ccad0 con 0x7f70f00d0370 2023-10-16T11:17:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f6acc700 1 --2- 172.21.15.181:0/491551484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00d0370 0x7f70f00d1fb0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009d90 tx=0x7f70e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1eec1389bf8bca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f52c9700 1 -- 172.21.15.181:0/491551484 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800e070 con 0x7f70f00d0370 2023-10-16T11:17:43.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f52c9700 1 -- 172.21.15.181:0/491551484 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e800bbd0 con 0x7f70f00d0370 2023-10-16T11:17:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.577+0000 7f70f52c9700 1 -- 172.21.15.181:0/491551484 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8012430 con 0x7f70f00d0370 2023-10-16T11:17:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 -- 172.21.15.181:0/491551484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00d0370 msgr2=0x7f70f00d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 --2- 172.21.15.181:0/491551484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00d0370 0x7f70f00d1fb0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009d90 tx=0x7f70e800b330 comp rx=0 tx=0).stop 2023-10-16T11:17:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 -- 172.21.15.181:0/491551484 shutdown_connections 2023-10-16T11:17:43.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 --2- 172.21.15.181:0/491551484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00d0370 0x7f70f00d1fb0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 --2- 172.21.15.181:0/491551484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f015b990 0x7f70f00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 -- 172.21.15.181:0/491551484 >> 172.21.15.181:0/491551484 conn(0x7f70f009a9a0 msgr2=0x7f70f00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 -- 172.21.15.181:0/491551484 shutdown_connections 2023-10-16T11:17:43.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.578+0000 7f70f852f700 1 -- 172.21.15.181:0/491551484 wait complete. 2023-10-16T11:17:43.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.579+0000 7f70f852f700 1 Processor -- start 2023-10-16T11:17:43.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.579+0000 7f70f852f700 1 -- start start 2023-10-16T11:17:43.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f852f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00cd2d0 0x7f70f0154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f852f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f00d0370 0x7f70f00c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f852f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70f015b990 0x7f70f00af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f852f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c87f0 con 0x7f70f00cd2d0 2023-10-16T11:17:43.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f852f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c8ad0 con 0x7f70f015b990 2023-10-16T11:17:43.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f852f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c8db0 con 0x7f70f00d0370 2023-10-16T11:17:43.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00cd2d0 0x7f70f0154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00cd2d0 0x7f70f0154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53948/0 (socket says 172.21.15.181:53948) 2023-10-16T11:17:43.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 -- 172.21.15.181:0/3527926566 learned_addr learned my addr 172.21.15.181:0/3527926566 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:43.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f5aca700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f00d0370 0x7f70f00c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f6acc700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70f015b990 0x7f70f00af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 -- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f00d0370 msgr2=0x7f70f00c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f00d0370 0x7f70f00c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 -- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70f015b990 msgr2=0x7f70f00af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70f015b990 0x7f70f00af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 -- 172.21.15.181:0/3527926566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e8009a70 con 0x7f70f00cd2d0 2023-10-16T11:17:43.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f5aca700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f00d0370 0x7f70f00c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f6acc700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70f015b990 0x7f70f00af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70f62cb700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00cd2d0 0x7f70f0154cc0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f70e000c7f0 tx=0x7f70e000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.580+0000 7f70e6ffd700 1 -- 172.21.15.181:0/3527926566 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0011800 con 0x7f70f00cd2d0 2023-10-16T11:17:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.581+0000 7f70e6ffd700 1 -- 172.21.15.181:0/3527926566 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e0011960 con 0x7f70f00cd2d0 2023-10-16T11:17:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.581+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f00c9090 con 0x7f70f00cd2d0 2023-10-16T11:17:43.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.581+0000 7f70e6ffd700 1 -- 172.21.15.181:0/3527926566 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0011cf0 con 0x7f70f00cd2d0 2023-10-16T11:17:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.581+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f0155200 con 0x7f70f00cd2d0 2023-10-16T11:17:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.581+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f0067980 con 0x7f70f00cd2d0 2023-10-16T11:17:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.582+0000 7f70e6ffd700 1 -- 172.21.15.181:0/3527926566 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f70e0018360 con 0x7f70f00cd2d0 2023-10-16T11:17:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.582+0000 7f70e6ffd700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f70d406b020 0x7f70d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.582+0000 7f70e6ffd700 1 -- 172.21.15.181:0/3527926566 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f70e0089980 con 0x7f70f00cd2d0 2023-10-16T11:17:43.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.582+0000 7f70f5aca700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f70d406b020 0x7f70d406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.583+0000 7f70f5aca700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f70d406b020 0x7f70d406d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70f00b5870 tx=0x7f70ec009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.587+0000 7f70e6ffd700 1 -- 172.21.15.181:0/3527926566 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f70e0054600 con 0x7f70f00cd2d0 2023-10-16T11:17:43.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.781+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70f0067d20 con 0x7f70f00cd2d0 2023-10-16T11:17:43.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.782+0000 7f70e6ffd700 1 -- 172.21.15.181:0/3527926566 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+17421 (secure 0 0 0) 0x7f70e001f090 con 0x7f70f00cd2d0 2023-10-16T11:17:43.785 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:43.785 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":199,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:17:40.985906+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:17:37.851564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:17:38.394398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:17:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f70d406b020 msgr2=0x7f70d406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f70d406b020 0x7f70d406d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70f00b5870 tx=0x7f70ec009040 comp rx=0 tx=0).stop 2023-10-16T11:17:43.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00cd2d0 msgr2=0x7f70f0154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00cd2d0 0x7f70f0154cc0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f70e000c7f0 tx=0x7f70e000cbb0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 shutdown_connections 2023-10-16T11:17:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f70d406b020 0x7f70d406d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70f015b990 0x7f70f00af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f00cd2d0 0x7f70f0154cc0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 --2- 172.21.15.181:0/3527926566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f00d0370 0x7f70f00c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 >> 172.21.15.181:0/3527926566 conn(0x7f70f009a9a0 msgr2=0x7f70f00ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 shutdown_connections 2023-10-16T11:17:43.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.785+0000 7f70f852f700 1 -- 172.21.15.181:0/3527926566 wait complete. 2023-10-16T11:17:43.799 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:17:43.800 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:17:43.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.917+0000 7f7da40a3700 1 Processor -- start 2023-10-16T11:17:43.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.918+0000 7f7da40a3700 1 -- start start 2023-10-16T11:17:43.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.918+0000 7f7da40a3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7d9c15dfd0 0x7f7d9c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.918+0000 7f7da40a3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7d9c0cd470 0x7f7d9c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.918+0000 7f7da40a3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c06f870 con 0x7f7d9c0af450 2023-10-16T11:17:43.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.918+0000 7f7da40a3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c0cde40 con 0x7f7d9c0cd470 2023-10-16T11:17:43.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da40a3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c0d1160 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7d9c0cd470 0x7f7d9c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7d9c0cd470 0x7f7d9c0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53960/0 (socket says 172.21.15.181:53960) 2023-10-16T11:17:43.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 -- 172.21.15.181:0/3997517963 learned_addr learned my addr 172.21.15.181:0/3997517963 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:43.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da1e3f700 1 --2- 172.21.15.181:0/3997517963 >> v2:172.21.15.190:3300/0 conn(0x7f7d9c15dfd0 0x7f7d9c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 -- 172.21.15.181:0/3997517963 >> v2:172.21.15.190:3300/0 conn(0x7f7d9c15dfd0 msgr2=0x7f7d9c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:43.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 --2- 172.21.15.181:0/3997517963 >> v2:172.21.15.190:3300/0 conn(0x7f7d9c15dfd0 0x7f7d9c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 -- 172.21.15.181:0/3997517963 >> v1:172.21.15.190:6789/0 conn(0x7f7d9c0af450 legacy=0x7f7d9c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 -- 172.21.15.181:0/3997517963 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0cc8e0 con 0x7f7d9c0cd470 2023-10-16T11:17:43.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da1e3f700 1 --2- 172.21.15.181:0/3997517963 >> v2:172.21.15.190:3300/0 conn(0x7f7d9c15dfd0 0x7f7d9c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:43.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da2640700 1 --2- 172.21.15.181:0/3997517963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c0cd470 0x7f7d9c0cd900 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c070340 tx=0x7f7d9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934e681bb3f2cebd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da0e3d700 1 -- 172.21.15.181:0/3997517963 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400e070 con 0x7f7d9c0cd470 2023-10-16T11:17:43.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da0e3d700 1 -- 172.21.15.181:0/3997517963 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9400b9c0 con 0x7f7d9c0cd470 2023-10-16T11:17:43.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.919+0000 7f7da0e3d700 1 -- 172.21.15.181:0/3997517963 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400bd70 con 0x7f7d9c0cd470 2023-10-16T11:17:43.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 -- 172.21.15.181:0/3997517963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c0cd470 msgr2=0x7f7d9c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 --2- 172.21.15.181:0/3997517963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c0cd470 0x7f7d9c0cd900 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c070340 tx=0x7f7d9400b330 comp rx=0 tx=0).stop 2023-10-16T11:17:43.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 -- 172.21.15.181:0/3997517963 shutdown_connections 2023-10-16T11:17:43.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 --2- 172.21.15.181:0/3997517963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c0cd470 0x7f7d9c0cd900 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 --2- 172.21.15.181:0/3997517963 >> v2:172.21.15.190:3300/0 conn(0x7f7d9c15dfd0 0x7f7d9c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 -- 172.21.15.181:0/3997517963 >> 172.21.15.181:0/3997517963 conn(0x7f7d9c0d04a0 msgr2=0x7f7d9c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:43.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 -- 172.21.15.181:0/3997517963 shutdown_connections 2023-10-16T11:17:43.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.920+0000 7f7da40a3700 1 -- 172.21.15.181:0/3997517963 wait complete. 2023-10-16T11:17:43.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.921+0000 7f7da40a3700 1 Processor -- start 2023-10-16T11:17:43.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.921+0000 7f7da40a3700 1 -- start start 2023-10-16T11:17:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da40a3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d9c0af450 0x7f7d9c0c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da40a3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d9c0cd470 0x7f7d9c14a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da40a3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c15dfd0 0x7f7d9c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da40a3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b3700 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da40a3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b5a50 con 0x7f7d9c0af450 2023-10-16T11:17:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da40a3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b5d30 con 0x7f7d9c0cd470 2023-10-16T11:17:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c15dfd0 0x7f7d9c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c15dfd0 0x7f7d9c0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53966/0 (socket says 172.21.15.181:53966) 2023-10-16T11:17:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 -- 172.21.15.181:0/2841899892 learned_addr learned my addr 172.21.15.181:0/2841899892 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da1e3f700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d9c0af450 0x7f7d9c0c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da163e700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d9c0cd470 0x7f7d9c14a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 -- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d9c0cd470 msgr2=0x7f7d9c14a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d9c0cd470 0x7f7d9c14a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 -- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d9c0af450 msgr2=0x7f7d9c0c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:43.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d9c0af450 0x7f7d9c0c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:43.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.922+0000 7f7da2640700 1 -- 172.21.15.181:0/2841899892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009b30 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.923+0000 7f7da2640700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c15dfd0 0x7f7d9c0b5000 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b370 tx=0x7f7d94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.923+0000 7f7d927fc700 1 -- 172.21.15.181:0/2841899892 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d940099b0 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.923+0000 7f7d927fc700 1 -- 172.21.15.181:0/2841899892 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d940092e0 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.923+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c0b3390 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.923+0000 7f7d927fc700 1 -- 172.21.15.181:0/2841899892 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9401c640 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.923+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c0ca040 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.924+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c067d20 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.924+0000 7f7d927fc700 1 -- 172.21.15.181:0/2841899892 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7d9401c7a0 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.924+0000 7f7d927fc700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f7d8006b020 0x7f7d8006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.925+0000 7f7d927fc700 1 -- 172.21.15.181:0/2841899892 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f7d9408b470 con 0x7f7d9c15dfd0 2023-10-16T11:17:43.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.925+0000 7f7da1e3f700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f7d8006b020 0x7f7d8006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.925+0000 7f7da1e3f700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f7d8006b020 0x7f7d8006d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c920 tx=0x7f7d8c008040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:43.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:43.929+0000 7f7d927fc700 1 -- 172.21.15.181:0/2841899892 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7d94056180 con 0x7f7d9c15dfd0 2023-10-16T11:17:44.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.119+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 --> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d9c0ccf50 con 0x7f7d8006b020 2023-10-16T11:17:44.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.127+0000 7f7d927fc700 1 -- 172.21.15.181:0/2841899892 <== mgr.10542 v2:172.21.15.181:6832/1532329351 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290276 (secure 0 0 0) 0x7f7d8c00c5b0 con 0x7f7d8006b020 2023-10-16T11:17:44.133 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.130+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f7d8006b020 msgr2=0x7f7d8006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.130+0000 7f7da40a3700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f7d8006b020 0x7f7d8006d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c920 tx=0x7f7d8c008040 comp rx=0 tx=0).stop 2023-10-16T11:17:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c15dfd0 msgr2=0x7f7d9c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c15dfd0 0x7f7d9c0b5000 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b370 tx=0x7f7d94004ad0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 shutdown_connections 2023-10-16T11:17:44.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f7d8006b020 0x7f7d8006d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d9c15dfd0 0x7f7d9c0b5000 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d9c0af450 0x7f7d9c0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 --2- 172.21.15.181:0/2841899892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d9c0cd470 0x7f7d9c14a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 >> 172.21.15.181:0/2841899892 conn(0x7f7d9c0d04a0 msgr2=0x7f7d9c15d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:44.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 shutdown_connections 2023-10-16T11:17:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.131+0000 7f7da40a3700 1 -- 172.21.15.181:0/2841899892 wait complete. 2023-10-16T11:17:44.138 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:17:44.145 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T11:17:42.985586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":424104,"kb_used_data":20584,"kb_used_omap":12,"kb_used_meta":403379,"kb_avail":754550616,"statfs":{"total":773094113280,"available":772659830784,"internally_reserved":0,"allocated":21078016,"data_stored":11300372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":413060664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.960728"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003815+0000","last_change":"2023-10-16T11:17:40.002067+0000","last_active":"2023-10-16T11:17:40.003815+0000","last_peered":"2023-10-16T11:17:40.003815+0000","last_clean":"2023-10-16T11:17:40.003815+0000","last_became_active":"2023-10-16T11:17:40.001815+0000","last_became_peered":"2023-10-16T11:17:40.001815+0000","last_unstale":"2023-10-16T11:17:40.003815+0000","last_undegraded":"2023-10-16T11:17:40.003815+0000","last_fullsized":"2023-10-16T11:17:40.003815+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:34.945432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007347+0000","last_change":"2023-10-16T11:17:40.002661+0000","last_active":"2023-10-16T11:17:40.007347+0000","last_peered":"2023-10-16T11:17:40.007347+0000","last_clean":"2023-10-16T11:17:40.007347+0000","last_became_active":"2023-10-16T11:17:40.002402+0000","last_became_peered":"2023-10-16T11:17:40.002402+0000","last_unstale":"2023-10-16T11:17:40.007347+0000","last_undegraded":"2023-10-16T11:17:40.007347+0000","last_fullsized":"2023-10-16T11:17:40.007347+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:21.387159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.999340+0000","last_change":"2023-10-16T11:17:39.998136+0000","last_active":"2023-10-16T11:17:39.999340+0000","last_peered":"2023-10-16T11:17:39.999340+0000","last_clean":"2023-10-16T11:17:39.999340+0000","last_became_active":"2023-10-16T11:17:39.997838+0000","last_became_peered":"2023-10-16T11:17:39.997838+0000","last_unstale":"2023-10-16T11:17:39.999340+0000","last_undegraded":"2023-10-16T11:17:39.999340+0000","last_fullsized":"2023-10-16T11:17:39.999340+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:20:12.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007391+0000","last_change":"2023-10-16T11:17:40.002961+0000","last_active":"2023-10-16T11:17:40.007391+0000","last_peered":"2023-10-16T11:17:40.007391+0000","last_clean":"2023-10-16T11:17:40.007391+0000","last_became_active":"2023-10-16T11:17:40.002479+0000","last_became_peered":"2023-10-16T11:17:40.002479+0000","last_unstale":"2023-10-16T11:17:40.007391+0000","last_undegraded":"2023-10-16T11:17:40.007391+0000","last_fullsized":"2023-10-16T11:17:40.007391+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:10:05.191201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989997+0000","last_change":"2023-10-16T11:17:40.006528+0000","last_active":"2023-10-16T11:17:40.989997+0000","last_peered":"2023-10-16T11:17:40.989997+0000","last_clean":"2023-10-16T11:17:40.989997+0000","last_became_active":"2023-10-16T11:17:40.006247+0000","last_became_peered":"2023-10-16T11:17:40.006247+0000","last_unstale":"2023-10-16T11:17:40.989997+0000","last_undegraded":"2023-10-16T11:17:40.989997+0000","last_fullsized":"2023-10-16T11:17:40.989997+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:53.566509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007385+0000","last_change":"2023-10-16T11:17:40.000087+0000","last_active":"2023-10-16T11:17:40.007385+0000","last_peered":"2023-10-16T11:17:40.007385+0000","last_clean":"2023-10-16T11:17:40.007385+0000","last_became_active":"2023-10-16T11:17:39.999845+0000","last_became_peered":"2023-10-16T11:17:39.999845+0000","last_unstale":"2023-10-16T11:17:40.007385+0000","last_undegraded":"2023-10-16T11:17:40.007385+0000","last_fullsized":"2023-10-16T11:17:40.007385+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:45.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990904+0000","last_change":"2023-10-16T11:17:39.999005+0000","last_active":"2023-10-16T11:17:40.990904+0000","last_peered":"2023-10-16T11:17:40.990904+0000","last_clean":"2023-10-16T11:17:40.990904+0000","last_became_active":"2023-10-16T11:17:39.998648+0000","last_became_peered":"2023-10-16T11:17:39.998648+0000","last_unstale":"2023-10-16T11:17:40.990904+0000","last_undegraded":"2023-10-16T11:17:40.990904+0000","last_fullsized":"2023-10-16T11:17:40.990904+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:40:24.078483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001429+0000","last_change":"2023-10-16T11:17:40.000258+0000","last_active":"2023-10-16T11:17:40.001429+0000","last_peered":"2023-10-16T11:17:40.001429+0000","last_clean":"2023-10-16T11:17:40.001429+0000","last_became_active":"2023-10-16T11:17:39.999988+0000","last_became_peered":"2023-10-16T11:17:39.999988+0000","last_unstale":"2023-10-16T11:17:40.001429+0000","last_undegraded":"2023-10-16T11:17:40.001429+0000","last_fullsized":"2023-10-16T11:17:40.001429+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:31.212627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990202+0000","last_change":"2023-10-16T11:17:40.003342+0000","last_active":"2023-10-16T11:17:40.990202+0000","last_peered":"2023-10-16T11:17:40.990202+0000","last_clean":"2023-10-16T11:17:40.990202+0000","last_became_active":"2023-10-16T11:17:40.003038+0000","last_became_peered":"2023-10-16T11:17:40.003038+0000","last_unstale":"2023-10-16T11:17:40.990202+0000","last_undegraded":"2023-10-16T11:17:40.990202+0000","last_fullsized":"2023-10-16T11:17:40.990202+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:22.537991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.000348+0000","last_change":"2023-10-16T11:17:39.999623+0000","last_active":"2023-10-16T11:17:40.000348+0000","last_peered":"2023-10-16T11:17:40.000348+0000","last_clean":"2023-10-16T11:17:40.000348+0000","last_became_active":"2023-10-16T11:17:39.999354+0000","last_became_peered":"2023-10-16T11:17:39.999354+0000","last_unstale":"2023-10-16T11:17:40.000348+0000","last_undegraded":"2023-10-16T11:17:40.000348+0000","last_fullsized":"2023-10-16T11:17:40.000348+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:43.659165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991381+0000","last_change":"2023-10-16T11:17:40.003172+0000","last_active":"2023-10-16T11:17:40.991381+0000","last_peered":"2023-10-16T11:17:40.991381+0000","last_clean":"2023-10-16T11:17:40.991381+0000","last_became_active":"2023-10-16T11:17:40.002845+0000","last_became_peered":"2023-10-16T11:17:40.002845+0000","last_unstale":"2023-10-16T11:17:40.991381+0000","last_undegraded":"2023-10-16T11:17:40.991381+0000","last_fullsized":"2023-10-16T11:17:40.991381+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:27.380826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007373+0000","last_change":"2023-10-16T11:17:40.002773+0000","last_active":"2023-10-16T11:17:40.007373+0000","last_peered":"2023-10-16T11:17:40.007373+0000","last_clean":"2023-10-16T11:17:40.007373+0000","last_became_active":"2023-10-16T11:17:40.002171+0000","last_became_peered":"2023-10-16T11:17:40.002171+0000","last_unstale":"2023-10-16T11:17:40.007373+0000","last_undegraded":"2023-10-16T11:17:40.007373+0000","last_fullsized":"2023-10-16T11:17:40.007373+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:25.015905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.004938+0000","last_change":"2023-10-16T11:17:40.000380+0000","last_active":"2023-10-16T11:17:40.004938+0000","last_peered":"2023-10-16T11:17:40.004938+0000","last_clean":"2023-10-16T11:17:40.004938+0000","last_became_active":"2023-10-16T11:17:39.999721+0000","last_became_peered":"2023-10-16T11:17:39.999721+0000","last_unstale":"2023-10-16T11:17:40.004938+0000","last_undegraded":"2023-10-16T11:17:40.004938+0000","last_fullsized":"2023-10-16T11:17:40.004938+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:43.659165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989972+0000","last_change":"2023-10-16T11:17:40.004408+0000","last_active":"2023-10-16T11:17:40.989972+0000","last_peered":"2023-10-16T11:17:40.989972+0000","last_clean":"2023-10-16T11:17:40.989972+0000","last_became_active":"2023-10-16T11:17:40.004104+0000","last_became_peered":"2023-10-16T11:17:40.004104+0000","last_unstale":"2023-10-16T11:17:40.989972+0000","last_undegraded":"2023-10-16T11:17:40.989972+0000","last_fullsized":"2023-10-16T11:17:40.989972+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:16.847184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.005904+0000","last_change":"2023-10-16T11:17:40.001429+0000","last_active":"2023-10-16T11:17:40.005904+0000","last_peered":"2023-10-16T11:17:40.005904+0000","last_clean":"2023-10-16T11:17:40.005904+0000","last_became_active":"2023-10-16T11:17:40.001157+0000","last_became_peered":"2023-10-16T11:17:40.001157+0000","last_unstale":"2023-10-16T11:17:40.005904+0000","last_undegraded":"2023-10-16T11:17:40.005904+0000","last_fullsized":"2023-10-16T11:17:40.005904+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:56.088053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001064+0000","last_change":"2023-10-16T11:17:39.997261+0000","last_active":"2023-10-16T11:17:40.001064+0000","last_peered":"2023-10-16T11:17:40.001064+0000","last_clean":"2023-10-16T11:17:40.001064+0000","last_became_active":"2023-10-16T11:17:39.996970+0000","last_became_peered":"2023-10-16T11:17:39.996970+0000","last_unstale":"2023-10-16T11:17:40.001064+0000","last_undegraded":"2023-10-16T11:17:40.001064+0000","last_fullsized":"2023-10-16T11:17:40.001064+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:30.025545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989702+0000","last_change":"2023-10-16T11:17:40.003406+0000","last_active":"2023-10-16T11:17:40.989702+0000","last_peered":"2023-10-16T11:17:40.989702+0000","last_clean":"2023-10-16T11:17:40.989702+0000","last_became_active":"2023-10-16T11:17:40.003118+0000","last_became_peered":"2023-10-16T11:17:40.003118+0000","last_unstale":"2023-10-16T11:17:40.989702+0000","last_undegraded":"2023-10-16T11:17:40.989702+0000","last_fullsized":"2023-10-16T11:17:40.989702+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:03.164357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001354+0000","last_change":"2023-10-16T11:17:39.999356+0000","last_active":"2023-10-16T11:17:40.001354+0000","last_peered":"2023-10-16T11:17:40.001354+0000","last_clean":"2023-10-16T11:17:40.001354+0000","last_became_active":"2023-10-16T11:17:39.998587+0000","last_became_peered":"2023-10-16T11:17:39.998587+0000","last_unstale":"2023-10-16T11:17:40.001354+0000","last_undegraded":"2023-10-16T11:17:40.001354+0000","last_fullsized":"2023-10-16T11:17:40.001354+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:18.549246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.002142+0000","last_change":"2023-10-16T11:17:40.000460+0000","last_active":"2023-10-16T11:17:40.002142+0000","last_peered":"2023-10-16T11:17:40.002142+0000","last_clean":"2023-10-16T11:17:40.002142+0000","last_became_active":"2023-10-16T11:17:40.000196+0000","last_became_peered":"2023-10-16T11:17:40.000196+0000","last_unstale":"2023-10-16T11:17:40.002142+0000","last_undegraded":"2023-10-16T11:17:40.002142+0000","last_fullsized":"2023-10-16T11:17:40.002142+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.173968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989668+0000","last_change":"2023-10-16T11:17:40.002691+0000","last_active":"2023-10-16T11:17:40.989668+0000","last_peered":"2023-10-16T11:17:40.989668+0000","last_clean":"2023-10-16T11:17:40.989668+0000","last_became_active":"2023-10-16T11:17:40.002416+0000","last_became_peered":"2023-10-16T11:17:40.002416+0000","last_unstale":"2023-10-16T11:17:40.989668+0000","last_undegraded":"2023-10-16T11:17:40.989668+0000","last_fullsized":"2023-10-16T11:17:40.989668+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:32.527025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990091+0000","last_change":"2023-10-16T11:17:40.001981+0000","last_active":"2023-10-16T11:17:40.990091+0000","last_peered":"2023-10-16T11:17:40.990091+0000","last_clean":"2023-10-16T11:17:40.990091+0000","last_became_active":"2023-10-16T11:17:40.001576+0000","last_became_peered":"2023-10-16T11:17:40.001576+0000","last_unstale":"2023-10-16T11:17:40.990091+0000","last_undegraded":"2023-10-16T11:17:40.990091+0000","last_fullsized":"2023-10-16T11:17:40.990091+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:27.874724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006604+0000","last_change":"2023-10-16T11:17:40.002928+0000","last_active":"2023-10-16T11:17:40.006604+0000","last_peered":"2023-10-16T11:17:40.006604+0000","last_clean":"2023-10-16T11:17:40.006604+0000","last_became_active":"2023-10-16T11:17:40.002673+0000","last_became_peered":"2023-10-16T11:17:40.002673+0000","last_unstale":"2023-10-16T11:17:40.006604+0000","last_undegraded":"2023-10-16T11:17:40.006604+0000","last_fullsized":"2023-10-16T11:17:40.006604+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:23.780765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.998513+0000","last_change":"2023-10-16T11:17:39.996631+0000","last_active":"2023-10-16T11:17:39.998513+0000","last_peered":"2023-10-16T11:17:39.998513+0000","last_clean":"2023-10-16T11:17:39.998513+0000","last_became_active":"2023-10-16T11:17:39.996366+0000","last_became_peered":"2023-10-16T11:17:39.996366+0000","last_unstale":"2023-10-16T11:17:39.998513+0000","last_undegraded":"2023-10-16T11:17:39.998513+0000","last_fullsized":"2023-10-16T11:17:39.998513+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:12:39.677767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001044+0000","last_change":"2023-10-16T11:17:39.994342+0000","last_active":"2023-10-16T11:17:40.001044+0000","last_peered":"2023-10-16T11:17:40.001044+0000","last_clean":"2023-10-16T11:17:40.001044+0000","last_became_active":"2023-10-16T11:17:39.993965+0000","last_became_peered":"2023-10-16T11:17:39.993965+0000","last_unstale":"2023-10-16T11:17:40.001044+0000","last_undegraded":"2023-10-16T11:17:40.001044+0000","last_fullsized":"2023-10-16T11:17:40.001044+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:45.752651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.004380+0000","last_change":"2023-10-16T11:17:40.000972+0000","last_active":"2023-10-16T11:17:40.004380+0000","last_peered":"2023-10-16T11:17:40.004380+0000","last_clean":"2023-10-16T11:17:40.004380+0000","last_became_active":"2023-10-16T11:17:40.000690+0000","last_became_peered":"2023-10-16T11:17:40.000690+0000","last_unstale":"2023-10-16T11:17:40.004380+0000","last_undegraded":"2023-10-16T11:17:40.004380+0000","last_fullsized":"2023-10-16T11:17:40.004380+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.173968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991528+0000","last_change":"2023-10-16T11:17:40.001789+0000","last_active":"2023-10-16T11:17:40.991528+0000","last_peered":"2023-10-16T11:17:40.991528+0000","last_clean":"2023-10-16T11:17:40.991528+0000","last_became_active":"2023-10-16T11:17:40.001503+0000","last_became_peered":"2023-10-16T11:17:40.001503+0000","last_unstale":"2023-10-16T11:17:40.991528+0000","last_undegraded":"2023-10-16T11:17:40.991528+0000","last_fullsized":"2023-10-16T11:17:40.991528+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:29.353229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007247+0000","last_change":"2023-10-16T11:17:40.001054+0000","last_active":"2023-10-16T11:17:40.007247+0000","last_peered":"2023-10-16T11:17:40.007247+0000","last_clean":"2023-10-16T11:17:40.007247+0000","last_became_active":"2023-10-16T11:17:39.999825+0000","last_became_peered":"2023-10-16T11:17:39.999825+0000","last_unstale":"2023-10-16T11:17:40.007247+0000","last_undegraded":"2023-10-16T11:17:40.007247+0000","last_fullsized":"2023-10-16T11:17:40.007247+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:53.686911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989893+0000","last_change":"2023-10-16T11:17:39.997423+0000","last_active":"2023-10-16T11:17:40.989893+0000","last_peered":"2023-10-16T11:17:40.989893+0000","last_clean":"2023-10-16T11:17:40.989893+0000","last_became_active":"2023-10-16T11:17:39.997145+0000","last_became_peered":"2023-10-16T11:17:39.997145+0000","last_unstale":"2023-10-16T11:17:40.989893+0000","last_undegraded":"2023-10-16T11:17:40.989893+0000","last_fullsized":"2023-10-16T11:17:40.989893+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:24.597213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.004963+0000","last_change":"2023-10-16T11:17:40.000564+0000","last_active":"2023-10-16T11:17:40.004963+0000","last_peered":"2023-10-16T11:17:40.004963+0000","last_clean":"2023-10-16T11:17:40.004963+0000","last_became_active":"2023-10-16T11:17:40.000118+0000","last_became_peered":"2023-10-16T11:17:40.000118+0000","last_unstale":"2023-10-16T11:17:40.004963+0000","last_undegraded":"2023-10-16T11:17:40.004963+0000","last_fullsized":"2023-10-16T11:17:40.004963+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:23.780765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989972+0000","last_change":"2023-10-16T11:17:40.004594+0000","last_active":"2023-10-16T11:17:40.989972+0000","last_peered":"2023-10-16T11:17:40.989972+0000","last_clean":"2023-10-16T11:17:40.989972+0000","last_became_active":"2023-10-16T11:17:40.000497+0000","last_became_peered":"2023-10-16T11:17:40.000497+0000","last_unstale":"2023-10-16T11:17:40.989972+0000","last_undegraded":"2023-10-16T11:17:40.989972+0000","last_fullsized":"2023-10-16T11:17:40.989972+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:01.138780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007430+0000","last_change":"2023-10-16T11:17:40.004245+0000","last_active":"2023-10-16T11:17:40.007430+0000","last_peered":"2023-10-16T11:17:40.007430+0000","last_clean":"2023-10-16T11:17:40.007430+0000","last_became_active":"2023-10-16T11:17:40.003994+0000","last_became_peered":"2023-10-16T11:17:40.003994+0000","last_unstale":"2023-10-16T11:17:40.007430+0000","last_undegraded":"2023-10-16T11:17:40.007430+0000","last_fullsized":"2023-10-16T11:17:40.007430+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:48.531341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989504+0000","last_change":"2023-10-16T11:17:40.004924+0000","last_active":"2023-10-16T11:17:40.989504+0000","last_peered":"2023-10-16T11:17:40.989504+0000","last_clean":"2023-10-16T11:17:40.989504+0000","last_became_active":"2023-10-16T11:17:40.004653+0000","last_became_peered":"2023-10-16T11:17:40.004653+0000","last_unstale":"2023-10-16T11:17:40.989504+0000","last_undegraded":"2023-10-16T11:17:40.989504+0000","last_fullsized":"2023-10-16T11:17:40.989504+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:43:42.264332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001330+0000","last_change":"2023-10-16T11:17:39.999441+0000","last_active":"2023-10-16T11:17:40.001330+0000","last_peered":"2023-10-16T11:17:40.001330+0000","last_clean":"2023-10-16T11:17:40.001330+0000","last_became_active":"2023-10-16T11:17:39.998692+0000","last_became_peered":"2023-10-16T11:17:39.998692+0000","last_unstale":"2023-10-16T11:17:40.001330+0000","last_undegraded":"2023-10-16T11:17:40.001330+0000","last_fullsized":"2023-10-16T11:17:40.001330+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:48.513765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"198'4","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989805+0000","last_change":"2023-10-16T11:17:39.047373+0000","last_active":"2023-10-16T11:17:40.989805+0000","last_peered":"2023-10-16T11:17:40.989805+0000","last_clean":"2023-10-16T11:17:40.989805+0000","last_became_active":"2023-10-16T11:17:39.046457+0000","last_became_peered":"2023-10-16T11:17:39.046457+0000","last_unstale":"2023-10-16T11:17:40.989805+0000","last_undegraded":"2023-10-16T11:17:40.989805+0000","last_fullsized":"2023-10-16T11:17:40.989805+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:17:52.491142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007526+0000","last_change":"2023-10-16T11:17:40.006114+0000","last_active":"2023-10-16T11:17:40.007526+0000","last_peered":"2023-10-16T11:17:40.007526+0000","last_clean":"2023-10-16T11:17:40.007526+0000","last_became_active":"2023-10-16T11:17:40.005833+0000","last_became_peered":"2023-10-16T11:17:40.005833+0000","last_unstale":"2023-10-16T11:17:40.007526+0000","last_undegraded":"2023-10-16T11:17:40.007526+0000","last_fullsized":"2023-10-16T11:17:40.007526+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:41.394184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"198'1","reported_seq":13,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990115+0000","last_change":"2023-10-16T11:17:39.046026+0000","last_active":"2023-10-16T11:17:40.990115+0000","last_peered":"2023-10-16T11:17:40.990115+0000","last_clean":"2023-10-16T11:17:40.990115+0000","last_became_active":"2023-10-16T11:17:39.045708+0000","last_became_peered":"2023-10-16T11:17:39.045708+0000","last_unstale":"2023-10-16T11:17:40.990115+0000","last_undegraded":"2023-10-16T11:17:40.990115+0000","last_fullsized":"2023-10-16T11:17:40.990115+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:52.558904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991308+0000","last_change":"2023-10-16T11:17:40.005500+0000","last_active":"2023-10-16T11:17:40.991308+0000","last_peered":"2023-10-16T11:17:40.991308+0000","last_clean":"2023-10-16T11:17:40.991308+0000","last_became_active":"2023-10-16T11:17:40.005206+0000","last_became_peered":"2023-10-16T11:17:40.005206+0000","last_unstale":"2023-10-16T11:17:40.991308+0000","last_undegraded":"2023-10-16T11:17:40.991308+0000","last_fullsized":"2023-10-16T11:17:40.991308+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:57.538992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041906+0000","last_change":"2023-10-16T11:17:39.052152+0000","last_active":"2023-10-16T11:17:40.041906+0000","last_peered":"2023-10-16T11:17:40.041906+0000","last_clean":"2023-10-16T11:17:40.041906+0000","last_became_active":"2023-10-16T11:17:39.051683+0000","last_became_peered":"2023-10-16T11:17:39.051683+0000","last_unstale":"2023-10-16T11:17:40.041906+0000","last_undegraded":"2023-10-16T11:17:40.041906+0000","last_fullsized":"2023-10-16T11:17:40.041906+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:20.262802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990112+0000","last_change":"2023-10-16T11:17:39.999259+0000","last_active":"2023-10-16T11:17:40.990112+0000","last_peered":"2023-10-16T11:17:40.990112+0000","last_clean":"2023-10-16T11:17:40.990112+0000","last_became_active":"2023-10-16T11:17:39.999002+0000","last_became_peered":"2023-10-16T11:17:39.999002+0000","last_unstale":"2023-10-16T11:17:40.990112+0000","last_undegraded":"2023-10-16T11:17:40.990112+0000","last_fullsized":"2023-10-16T11:17:40.990112+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:03.507916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041841+0000","last_change":"2023-10-16T11:17:39.046049+0000","last_active":"2023-10-16T11:17:40.041841+0000","last_peered":"2023-10-16T11:17:40.041841+0000","last_clean":"2023-10-16T11:17:40.041841+0000","last_became_active":"2023-10-16T11:17:39.045693+0000","last_became_peered":"2023-10-16T11:17:39.045693+0000","last_unstale":"2023-10-16T11:17:40.041841+0000","last_undegraded":"2023-10-16T11:17:40.041841+0000","last_fullsized":"2023-10-16T11:17:40.041841+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:02.161144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006299+0000","last_change":"2023-10-16T11:17:40.002337+0000","last_active":"2023-10-16T11:17:40.006299+0000","last_peered":"2023-10-16T11:17:40.006299+0000","last_clean":"2023-10-16T11:17:40.006299+0000","last_became_active":"2023-10-16T11:17:40.002046+0000","last_became_peered":"2023-10-16T11:17:40.002046+0000","last_unstale":"2023-10-16T11:17:40.006299+0000","last_undegraded":"2023-10-16T11:17:40.006299+0000","last_fullsized":"2023-10-16T11:17:40.006299+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:06.985430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"198'1","reported_seq":13,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990047+0000","last_change":"2023-10-16T11:17:39.051284+0000","last_active":"2023-10-16T11:17:40.990047+0000","last_peered":"2023-10-16T11:17:40.990047+0000","last_clean":"2023-10-16T11:17:40.990047+0000","last_became_active":"2023-10-16T11:17:39.049365+0000","last_became_peered":"2023-10-16T11:17:39.049365+0000","last_unstale":"2023-10-16T11:17:40.990047+0000","last_undegraded":"2023-10-16T11:17:40.990047+0000","last_fullsized":"2023-10-16T11:17:40.990047+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:00.671500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.002252+0000","last_change":"2023-10-16T11:17:39.995054+0000","last_active":"2023-10-16T11:17:40.002252+0000","last_peered":"2023-10-16T11:17:40.002252+0000","last_clean":"2023-10-16T11:17:40.002252+0000","last_became_active":"2023-10-16T11:17:39.994786+0000","last_became_peered":"2023-10-16T11:17:39.994786+0000","last_unstale":"2023-10-16T11:17:40.002252+0000","last_undegraded":"2023-10-16T11:17:40.002252+0000","last_fullsized":"2023-10-16T11:17:40.002252+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:45:39.207599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"198'1","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.042912+0000","last_change":"2023-10-16T11:17:39.055907+0000","last_active":"2023-10-16T11:17:40.042912+0000","last_peered":"2023-10-16T11:17:40.042912+0000","last_clean":"2023-10-16T11:17:40.042912+0000","last_became_active":"2023-10-16T11:17:39.055614+0000","last_became_peered":"2023-10-16T11:17:39.055614+0000","last_unstale":"2023-10-16T11:17:40.042912+0000","last_undegraded":"2023-10-16T11:17:40.042912+0000","last_fullsized":"2023-10-16T11:17:40.042912+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:22.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990947+0000","last_change":"2023-10-16T11:17:40.003698+0000","last_active":"2023-10-16T11:17:40.990947+0000","last_peered":"2023-10-16T11:17:40.990947+0000","last_clean":"2023-10-16T11:17:40.990947+0000","last_became_active":"2023-10-16T11:17:40.003410+0000","last_became_peered":"2023-10-16T11:17:40.003410+0000","last_unstale":"2023-10-16T11:17:40.990947+0000","last_undegraded":"2023-10-16T11:17:40.990947+0000","last_fullsized":"2023-10-16T11:17:40.990947+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:43.507470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.988634+0000","last_change":"2023-10-16T11:17:39.053231+0000","last_active":"2023-10-16T11:17:39.988634+0000","last_peered":"2023-10-16T11:17:39.988634+0000","last_clean":"2023-10-16T11:17:39.988634+0000","last_became_active":"2023-10-16T11:17:39.052948+0000","last_became_peered":"2023-10-16T11:17:39.052948+0000","last_unstale":"2023-10-16T11:17:39.988634+0000","last_undegraded":"2023-10-16T11:17:39.988634+0000","last_fullsized":"2023-10-16T11:17:39.988634+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:54:24.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007463+0000","last_change":"2023-10-16T11:17:40.003379+0000","last_active":"2023-10-16T11:17:40.007463+0000","last_peered":"2023-10-16T11:17:40.007463+0000","last_clean":"2023-10-16T11:17:40.007463+0000","last_became_active":"2023-10-16T11:17:40.003099+0000","last_became_peered":"2023-10-16T11:17:40.003099+0000","last_unstale":"2023-10-16T11:17:40.007463+0000","last_undegraded":"2023-10-16T11:17:40.007463+0000","last_fullsized":"2023-10-16T11:17:40.007463+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:44.483361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991446+0000","last_change":"2023-10-16T11:17:39.043680+0000","last_active":"2023-10-16T11:17:40.991446+0000","last_peered":"2023-10-16T11:17:40.991446+0000","last_clean":"2023-10-16T11:17:40.991446+0000","last_became_active":"2023-10-16T11:17:39.043381+0000","last_became_peered":"2023-10-16T11:17:39.043381+0000","last_unstale":"2023-10-16T11:17:40.991446+0000","last_undegraded":"2023-10-16T11:17:40.991446+0000","last_fullsized":"2023-10-16T11:17:40.991446+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:54.398904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001318+0000","last_change":"2023-10-16T11:17:39.999766+0000","last_active":"2023-10-16T11:17:40.001318+0000","last_peered":"2023-10-16T11:17:40.001318+0000","last_clean":"2023-10-16T11:17:40.001318+0000","last_became_active":"2023-10-16T11:17:39.999509+0000","last_became_peered":"2023-10-16T11:17:39.999509+0000","last_unstale":"2023-10-16T11:17:40.001318+0000","last_undegraded":"2023-10-16T11:17:40.001318+0000","last_fullsized":"2023-10-16T11:17:40.001318+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:47.749603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989662+0000","last_change":"2023-10-16T11:17:39.047754+0000","last_active":"2023-10-16T11:17:40.989662+0000","last_peered":"2023-10-16T11:17:40.989662+0000","last_clean":"2023-10-16T11:17:40.989662+0000","last_became_active":"2023-10-16T11:17:39.047468+0000","last_became_peered":"2023-10-16T11:17:39.047468+0000","last_unstale":"2023-10-16T11:17:40.989662+0000","last_undegraded":"2023-10-16T11:17:40.989662+0000","last_fullsized":"2023-10-16T11:17:40.989662+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:30.092326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991422+0000","last_change":"2023-10-16T11:17:40.006735+0000","last_active":"2023-10-16T11:17:40.991422+0000","last_peered":"2023-10-16T11:17:40.991422+0000","last_clean":"2023-10-16T11:17:40.991422+0000","last_became_active":"2023-10-16T11:17:40.006482+0000","last_became_peered":"2023-10-16T11:17:40.006482+0000","last_unstale":"2023-10-16T11:17:40.991422+0000","last_undegraded":"2023-10-16T11:17:40.991422+0000","last_fullsized":"2023-10-16T11:17:40.991422+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:39:20.230419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"198'3","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989560+0000","last_change":"2023-10-16T11:17:39.054415+0000","last_active":"2023-10-16T11:17:40.989560+0000","last_peered":"2023-10-16T11:17:40.989560+0000","last_clean":"2023-10-16T11:17:40.989560+0000","last_became_active":"2023-10-16T11:17:39.054149+0000","last_became_peered":"2023-10-16T11:17:39.054149+0000","last_unstale":"2023-10-16T11:17:40.989560+0000","last_undegraded":"2023-10-16T11:17:40.989560+0000","last_fullsized":"2023-10-16T11:17:40.989560+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:02.161144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006543+0000","last_change":"2023-10-16T11:17:40.004003+0000","last_active":"2023-10-16T11:17:40.006543+0000","last_peered":"2023-10-16T11:17:40.006543+0000","last_clean":"2023-10-16T11:17:40.006543+0000","last_became_active":"2023-10-16T11:17:40.003740+0000","last_became_peered":"2023-10-16T11:17:40.003740+0000","last_unstale":"2023-10-16T11:17:40.006543+0000","last_undegraded":"2023-10-16T11:17:40.006543+0000","last_fullsized":"2023-10-16T11:17:40.006543+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:16:11.386464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"198'6","reported_seq":17,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.043948+0000","last_change":"2023-10-16T11:17:39.050581+0000","last_active":"2023-10-16T11:17:40.043948+0000","last_peered":"2023-10-16T11:17:40.043948+0000","last_clean":"2023-10-16T11:17:40.043948+0000","last_became_active":"2023-10-16T11:17:39.049782+0000","last_became_peered":"2023-10-16T11:17:39.049782+0000","last_unstale":"2023-10-16T11:17:40.043948+0000","last_undegraded":"2023-10-16T11:17:40.043948+0000","last_fullsized":"2023-10-16T11:17:40.043948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:23:04.098588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.005503+0000","last_change":"2023-10-16T11:17:39.999592+0000","last_active":"2023-10-16T11:17:40.005503+0000","last_peered":"2023-10-16T11:17:40.005503+0000","last_clean":"2023-10-16T11:17:40.005503+0000","last_became_active":"2023-10-16T11:17:39.999313+0000","last_became_peered":"2023-10-16T11:17:39.999313+0000","last_unstale":"2023-10-16T11:17:40.005503+0000","last_undegraded":"2023-10-16T11:17:40.005503+0000","last_fullsized":"2023-10-16T11:17:40.005503+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:30.009623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.043044+0000","last_change":"2023-10-16T11:17:39.048970+0000","last_active":"2023-10-16T11:17:40.043044+0000","last_peered":"2023-10-16T11:17:40.043044+0000","last_clean":"2023-10-16T11:17:40.043044+0000","last_became_active":"2023-10-16T11:17:39.048647+0000","last_became_peered":"2023-10-16T11:17:39.048647+0000","last_unstale":"2023-10-16T11:17:40.043044+0000","last_undegraded":"2023-10-16T11:17:40.043044+0000","last_fullsized":"2023-10-16T11:17:40.043044+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:21.692024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991068+0000","last_change":"2023-10-16T11:17:40.002404+0000","last_active":"2023-10-16T11:17:40.991068+0000","last_peered":"2023-10-16T11:17:40.991068+0000","last_clean":"2023-10-16T11:17:40.991068+0000","last_became_active":"2023-10-16T11:17:40.001513+0000","last_became_peered":"2023-10-16T11:17:40.001513+0000","last_unstale":"2023-10-16T11:17:40.991068+0000","last_undegraded":"2023-10-16T11:17:40.991068+0000","last_fullsized":"2023-10-16T11:17:40.991068+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:43:07.370151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.997965+0000","last_change":"2023-10-16T11:17:39.050438+0000","last_active":"2023-10-16T11:17:39.997965+0000","last_peered":"2023-10-16T11:17:39.997965+0000","last_clean":"2023-10-16T11:17:39.997965+0000","last_became_active":"2023-10-16T11:17:39.050131+0000","last_became_peered":"2023-10-16T11:17:39.050131+0000","last_unstale":"2023-10-16T11:17:39.997965+0000","last_undegraded":"2023-10-16T11:17:39.997965+0000","last_fullsized":"2023-10-16T11:17:39.997965+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:24.008300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989810+0000","last_change":"2023-10-16T11:17:39.996329+0000","last_active":"2023-10-16T11:17:40.989810+0000","last_peered":"2023-10-16T11:17:40.989810+0000","last_clean":"2023-10-16T11:17:40.989810+0000","last_became_active":"2023-10-16T11:17:39.996054+0000","last_became_peered":"2023-10-16T11:17:39.996054+0000","last_unstale":"2023-10-16T11:17:40.989810+0000","last_undegraded":"2023-10-16T11:17:40.989810+0000","last_fullsized":"2023-10-16T11:17:40.989810+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:56.958648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.987705+0000","last_change":"2023-10-16T11:17:39.051060+0000","last_active":"2023-10-16T11:17:39.987705+0000","last_peered":"2023-10-16T11:17:39.987705+0000","last_clean":"2023-10-16T11:17:39.987705+0000","last_became_active":"2023-10-16T11:17:39.050786+0000","last_became_peered":"2023-10-16T11:17:39.050786+0000","last_unstale":"2023-10-16T11:17:39.987705+0000","last_undegraded":"2023-10-16T11:17:39.987705+0000","last_fullsized":"2023-10-16T11:17:39.987705+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:26.653770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006478+0000","last_change":"2023-10-16T11:17:40.002834+0000","last_active":"2023-10-16T11:17:40.006478+0000","last_peered":"2023-10-16T11:17:40.006478+0000","last_clean":"2023-10-16T11:17:40.006478+0000","last_became_active":"2023-10-16T11:17:40.002392+0000","last_became_peered":"2023-10-16T11:17:40.002392+0000","last_unstale":"2023-10-16T11:17:40.006478+0000","last_undegraded":"2023-10-16T11:17:40.006478+0000","last_fullsized":"2023-10-16T11:17:40.006478+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:22.376382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"198'8","reported_seq":19,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.044618+0000","last_change":"2023-10-16T11:17:39.053140+0000","last_active":"2023-10-16T11:17:40.044618+0000","last_peered":"2023-10-16T11:17:40.044618+0000","last_clean":"2023-10-16T11:17:40.044618+0000","last_became_active":"2023-10-16T11:17:39.052866+0000","last_became_peered":"2023-10-16T11:17:39.052866+0000","last_unstale":"2023-10-16T11:17:40.044618+0000","last_undegraded":"2023-10-16T11:17:40.044618+0000","last_fullsized":"2023-10-16T11:17:40.044618+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:51.213414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.005505+0000","last_change":"2023-10-16T11:17:40.002324+0000","last_active":"2023-10-16T11:17:40.005505+0000","last_peered":"2023-10-16T11:17:40.005505+0000","last_clean":"2023-10-16T11:17:40.005505+0000","last_became_active":"2023-10-16T11:17:40.001890+0000","last_became_peered":"2023-10-16T11:17:40.001890+0000","last_unstale":"2023-10-16T11:17:40.005505+0000","last_undegraded":"2023-10-16T11:17:40.005505+0000","last_fullsized":"2023-10-16T11:17:40.005505+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:46.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"198'1","reported_seq":13,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989801+0000","last_change":"2023-10-16T11:17:39.050248+0000","last_active":"2023-10-16T11:17:40.989801+0000","last_peered":"2023-10-16T11:17:40.989801+0000","last_clean":"2023-10-16T11:17:40.989801+0000","last_became_active":"2023-10-16T11:17:39.049907+0000","last_became_peered":"2023-10-16T11:17:39.049907+0000","last_unstale":"2023-10-16T11:17:40.989801+0000","last_undegraded":"2023-10-16T11:17:40.989801+0000","last_fullsized":"2023-10-16T11:17:40.989801+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:39.434497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990399+0000","last_change":"2023-10-16T11:17:40.000430+0000","last_active":"2023-10-16T11:17:40.990399+0000","last_peered":"2023-10-16T11:17:40.990399+0000","last_clean":"2023-10-16T11:17:40.990399+0000","last_became_active":"2023-10-16T11:17:40.000126+0000","last_became_peered":"2023-10-16T11:17:40.000126+0000","last_unstale":"2023-10-16T11:17:40.990399+0000","last_undegraded":"2023-10-16T11:17:40.990399+0000","last_fullsized":"2023-10-16T11:17:40.990399+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:21.481341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041455+0000","last_change":"2023-10-16T11:17:39.052796+0000","last_active":"2023-10-16T11:17:40.041455+0000","last_peered":"2023-10-16T11:17:40.041455+0000","last_clean":"2023-10-16T11:17:40.041455+0000","last_became_active":"2023-10-16T11:17:39.052487+0000","last_became_peered":"2023-10-16T11:17:39.052487+0000","last_unstale":"2023-10-16T11:17:40.041455+0000","last_undegraded":"2023-10-16T11:17:40.041455+0000","last_fullsized":"2023-10-16T11:17:40.041455+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:30.001594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989950+0000","last_change":"2023-10-16T11:17:40.002312+0000","last_active":"2023-10-16T11:17:40.989950+0000","last_peered":"2023-10-16T11:17:40.989950+0000","last_clean":"2023-10-16T11:17:40.989950+0000","last_became_active":"2023-10-16T11:17:40.002012+0000","last_became_peered":"2023-10-16T11:17:40.002012+0000","last_unstale":"2023-10-16T11:17:40.989950+0000","last_undegraded":"2023-10-16T11:17:40.989950+0000","last_fullsized":"2023-10-16T11:17:40.989950+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:37.782255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"198'1","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.042936+0000","last_change":"2023-10-16T11:17:39.051842+0000","last_active":"2023-10-16T11:17:40.042936+0000","last_peered":"2023-10-16T11:17:40.042936+0000","last_clean":"2023-10-16T11:17:40.042936+0000","last_became_active":"2023-10-16T11:17:39.051539+0000","last_became_peered":"2023-10-16T11:17:39.051539+0000","last_unstale":"2023-10-16T11:17:40.042936+0000","last_undegraded":"2023-10-16T11:17:40.042936+0000","last_fullsized":"2023-10-16T11:17:40.042936+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:26.397820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007702+0000","last_change":"2023-10-16T11:17:40.004053+0000","last_active":"2023-10-16T11:17:40.007702+0000","last_peered":"2023-10-16T11:17:40.007702+0000","last_clean":"2023-10-16T11:17:40.007702+0000","last_became_active":"2023-10-16T11:17:40.003752+0000","last_became_peered":"2023-10-16T11:17:40.003752+0000","last_unstale":"2023-10-16T11:17:40.007702+0000","last_undegraded":"2023-10-16T11:17:40.007702+0000","last_fullsized":"2023-10-16T11:17:40.007702+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:06.985430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991433+0000","last_change":"2023-10-16T11:17:39.056947+0000","last_active":"2023-10-16T11:17:40.991433+0000","last_peered":"2023-10-16T11:17:40.991433+0000","last_clean":"2023-10-16T11:17:40.991433+0000","last_became_active":"2023-10-16T11:17:39.056722+0000","last_became_peered":"2023-10-16T11:17:39.056722+0000","last_unstale":"2023-10-16T11:17:40.991433+0000","last_undegraded":"2023-10-16T11:17:40.991433+0000","last_fullsized":"2023-10-16T11:17:40.991433+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:02:36.679484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007476+0000","last_change":"2023-10-16T11:17:40.006066+0000","last_active":"2023-10-16T11:17:40.007476+0000","last_peered":"2023-10-16T11:17:40.007476+0000","last_clean":"2023-10-16T11:17:40.007476+0000","last_became_active":"2023-10-16T11:17:40.005787+0000","last_became_peered":"2023-10-16T11:17:40.005787+0000","last_unstale":"2023-10-16T11:17:40.007476+0000","last_undegraded":"2023-10-16T11:17:40.007476+0000","last_fullsized":"2023-10-16T11:17:40.007476+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:48.932559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041783+0000","last_change":"2023-10-16T11:17:39.049474+0000","last_active":"2023-10-16T11:17:40.041783+0000","last_peered":"2023-10-16T11:17:40.041783+0000","last_clean":"2023-10-16T11:17:40.041783+0000","last_became_active":"2023-10-16T11:17:39.049169+0000","last_became_peered":"2023-10-16T11:17:39.049169+0000","last_unstale":"2023-10-16T11:17:40.041783+0000","last_undegraded":"2023-10-16T11:17:40.041783+0000","last_fullsized":"2023-10-16T11:17:40.041783+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:02.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991116+0000","last_change":"2023-10-16T11:17:40.006132+0000","last_active":"2023-10-16T11:17:40.991116+0000","last_peered":"2023-10-16T11:17:40.991116+0000","last_clean":"2023-10-16T11:17:40.991116+0000","last_became_active":"2023-10-16T11:17:40.005863+0000","last_became_peered":"2023-10-16T11:17:40.005863+0000","last_unstale":"2023-10-16T11:17:40.991116+0000","last_undegraded":"2023-10-16T11:17:40.991116+0000","last_fullsized":"2023-10-16T11:17:40.991116+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:43.610346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989582+0000","last_change":"2023-10-16T11:17:39.049430+0000","last_active":"2023-10-16T11:17:40.989582+0000","last_peered":"2023-10-16T11:17:40.989582+0000","last_clean":"2023-10-16T11:17:40.989582+0000","last_became_active":"2023-10-16T11:17:39.049078+0000","last_became_peered":"2023-10-16T11:17:39.049078+0000","last_unstale":"2023-10-16T11:17:40.989582+0000","last_undegraded":"2023-10-16T11:17:40.989582+0000","last_fullsized":"2023-10-16T11:17:40.989582+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:29.002116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003758+0000","last_change":"2023-10-16T11:17:40.000592+0000","last_active":"2023-10-16T11:17:40.003758+0000","last_peered":"2023-10-16T11:17:40.003758+0000","last_clean":"2023-10-16T11:17:40.003758+0000","last_became_active":"2023-10-16T11:17:40.000294+0000","last_became_peered":"2023-10-16T11:17:40.000294+0000","last_unstale":"2023-10-16T11:17:40.003758+0000","last_undegraded":"2023-10-16T11:17:40.003758+0000","last_fullsized":"2023-10-16T11:17:40.003758+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:41.394184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.990984+0000","last_change":"2023-10-16T11:17:39.049330+0000","last_active":"2023-10-16T11:17:39.990984+0000","last_peered":"2023-10-16T11:17:39.990984+0000","last_clean":"2023-10-16T11:17:39.990984+0000","last_became_active":"2023-10-16T11:17:39.049026+0000","last_became_peered":"2023-10-16T11:17:39.049026+0000","last_unstale":"2023-10-16T11:17:39.990984+0000","last_undegraded":"2023-10-16T11:17:39.990984+0000","last_fullsized":"2023-10-16T11:17:39.990984+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:52:29.654874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989547+0000","last_change":"2023-10-16T11:17:40.005019+0000","last_active":"2023-10-16T11:17:40.989547+0000","last_peered":"2023-10-16T11:17:40.989547+0000","last_clean":"2023-10-16T11:17:40.989547+0000","last_became_active":"2023-10-16T11:17:40.004338+0000","last_became_peered":"2023-10-16T11:17:40.004338+0000","last_unstale":"2023-10-16T11:17:40.989547+0000","last_undegraded":"2023-10-16T11:17:40.989547+0000","last_fullsized":"2023-10-16T11:17:40.989547+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:14.535881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990031+0000","last_change":"2023-10-16T11:17:39.044038+0000","last_active":"2023-10-16T11:17:40.990031+0000","last_peered":"2023-10-16T11:17:40.990031+0000","last_clean":"2023-10-16T11:17:40.990031+0000","last_became_active":"2023-10-16T11:17:39.043756+0000","last_became_peered":"2023-10-16T11:17:39.043756+0000","last_unstale":"2023-10-16T11:17:40.990031+0000","last_undegraded":"2023-10-16T11:17:40.990031+0000","last_fullsized":"2023-10-16T11:17:40.990031+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:26.867119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989706+0000","last_change":"2023-10-16T11:17:40.000889+0000","last_active":"2023-10-16T11:17:40.989706+0000","last_peered":"2023-10-16T11:17:40.989706+0000","last_clean":"2023-10-16T11:17:40.989706+0000","last_became_active":"2023-10-16T11:17:40.000612+0000","last_became_peered":"2023-10-16T11:17:40.000612+0000","last_unstale":"2023-10-16T11:17:40.989706+0000","last_undegraded":"2023-10-16T11:17:40.989706+0000","last_fullsized":"2023-10-16T11:17:40.989706+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:44.523350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.040982+0000","last_change":"2023-10-16T11:17:39.048622+0000","last_active":"2023-10-16T11:17:40.040982+0000","last_peered":"2023-10-16T11:17:40.040982+0000","last_clean":"2023-10-16T11:17:40.040982+0000","last_became_active":"2023-10-16T11:17:39.048301+0000","last_became_peered":"2023-10-16T11:17:39.048301+0000","last_unstale":"2023-10-16T11:17:40.040982+0000","last_undegraded":"2023-10-16T11:17:40.040982+0000","last_fullsized":"2023-10-16T11:17:40.040982+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:12.460241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006459+0000","last_change":"2023-10-16T11:17:40.002875+0000","last_active":"2023-10-16T11:17:40.006459+0000","last_peered":"2023-10-16T11:17:40.006459+0000","last_clean":"2023-10-16T11:17:40.006459+0000","last_became_active":"2023-10-16T11:17:40.002193+0000","last_became_peered":"2023-10-16T11:17:40.002193+0000","last_unstale":"2023-10-16T11:17:40.006459+0000","last_undegraded":"2023-10-16T11:17:40.006459+0000","last_fullsized":"2023-10-16T11:17:40.006459+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:58.913565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.997188+0000","last_change":"2023-10-16T11:17:39.041489+0000","last_active":"2023-10-16T11:17:39.997188+0000","last_peered":"2023-10-16T11:17:39.997188+0000","last_clean":"2023-10-16T11:17:39.997188+0000","last_became_active":"2023-10-16T11:17:39.041213+0000","last_became_peered":"2023-10-16T11:17:39.041213+0000","last_unstale":"2023-10-16T11:17:39.997188+0000","last_undegraded":"2023-10-16T11:17:39.997188+0000","last_fullsized":"2023-10-16T11:17:39.997188+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:55.080448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.000740+0000","last_change":"2023-10-16T11:17:39.994258+0000","last_active":"2023-10-16T11:17:40.000740+0000","last_peered":"2023-10-16T11:17:40.000740+0000","last_clean":"2023-10-16T11:17:40.000740+0000","last_became_active":"2023-10-16T11:17:39.993977+0000","last_became_peered":"2023-10-16T11:17:39.993977+0000","last_unstale":"2023-10-16T11:17:40.000740+0000","last_undegraded":"2023-10-16T11:17:40.000740+0000","last_fullsized":"2023-10-16T11:17:40.000740+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:14.087424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.997617+0000","last_change":"2023-10-16T11:17:39.053279+0000","last_active":"2023-10-16T11:17:39.997617+0000","last_peered":"2023-10-16T11:17:39.997617+0000","last_clean":"2023-10-16T11:17:39.997617+0000","last_became_active":"2023-10-16T11:17:39.053022+0000","last_became_peered":"2023-10-16T11:17:39.053022+0000","last_unstale":"2023-10-16T11:17:39.997617+0000","last_undegraded":"2023-10-16T11:17:39.997617+0000","last_fullsized":"2023-10-16T11:17:39.997617+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:20.379554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.002518+0000","last_change":"2023-10-16T11:17:39.998421+0000","last_active":"2023-10-16T11:17:40.002518+0000","last_peered":"2023-10-16T11:17:40.002518+0000","last_clean":"2023-10-16T11:17:40.002518+0000","last_became_active":"2023-10-16T11:17:39.997664+0000","last_became_peered":"2023-10-16T11:17:39.997664+0000","last_unstale":"2023-10-16T11:17:40.002518+0000","last_undegraded":"2023-10-16T11:17:40.002518+0000","last_fullsized":"2023-10-16T11:17:40.002518+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:59.689629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.988120+0000","last_change":"2023-10-16T11:17:39.052554+0000","last_active":"2023-10-16T11:17:39.988120+0000","last_peered":"2023-10-16T11:17:39.988120+0000","last_clean":"2023-10-16T11:17:39.988120+0000","last_became_active":"2023-10-16T11:17:39.052258+0000","last_became_peered":"2023-10-16T11:17:39.052258+0000","last_unstale":"2023-10-16T11:17:39.988120+0000","last_undegraded":"2023-10-16T11:17:39.988120+0000","last_fullsized":"2023-10-16T11:17:39.988120+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:34.083507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.000229+0000","last_change":"2023-10-16T11:17:39.999182+0000","last_active":"2023-10-16T11:17:40.000229+0000","last_peered":"2023-10-16T11:17:40.000229+0000","last_clean":"2023-10-16T11:17:40.000229+0000","last_became_active":"2023-10-16T11:17:39.998883+0000","last_became_peered":"2023-10-16T11:17:39.998883+0000","last_unstale":"2023-10-16T11:17:40.000229+0000","last_undegraded":"2023-10-16T11:17:40.000229+0000","last_fullsized":"2023-10-16T11:17:40.000229+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:06.612719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991033+0000","last_change":"2023-10-16T11:17:39.036812+0000","last_active":"2023-10-16T11:17:40.991033+0000","last_peered":"2023-10-16T11:17:40.991033+0000","last_clean":"2023-10-16T11:17:40.991033+0000","last_became_active":"2023-10-16T11:17:39.036521+0000","last_became_peered":"2023-10-16T11:17:39.036521+0000","last_unstale":"2023-10-16T11:17:40.991033+0000","last_undegraded":"2023-10-16T11:17:40.991033+0000","last_fullsized":"2023-10-16T11:17:40.991033+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:43.515745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003755+0000","last_change":"2023-10-16T11:17:40.001553+0000","last_active":"2023-10-16T11:17:40.003755+0000","last_peered":"2023-10-16T11:17:40.003755+0000","last_clean":"2023-10-16T11:17:40.003755+0000","last_became_active":"2023-10-16T11:17:40.001286+0000","last_became_peered":"2023-10-16T11:17:40.001286+0000","last_unstale":"2023-10-16T11:17:40.003755+0000","last_undegraded":"2023-10-16T11:17:40.003755+0000","last_fullsized":"2023-10-16T11:17:40.003755+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:02.762892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990089+0000","last_change":"2023-10-16T11:17:39.040895+0000","last_active":"2023-10-16T11:17:40.990089+0000","last_peered":"2023-10-16T11:17:40.990089+0000","last_clean":"2023-10-16T11:17:40.990089+0000","last_became_active":"2023-10-16T11:17:39.040579+0000","last_became_peered":"2023-10-16T11:17:39.040579+0000","last_unstale":"2023-10-16T11:17:40.990089+0000","last_undegraded":"2023-10-16T11:17:40.990089+0000","last_fullsized":"2023-10-16T11:17:40.990089+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:23.589608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003717+0000","last_change":"2023-10-16T11:17:40.000211+0000","last_active":"2023-10-16T11:17:40.003717+0000","last_peered":"2023-10-16T11:17:40.003717+0000","last_clean":"2023-10-16T11:17:40.003717+0000","last_became_active":"2023-10-16T11:17:39.999762+0000","last_became_peered":"2023-10-16T11:17:39.999762+0000","last_unstale":"2023-10-16T11:17:40.003717+0000","last_undegraded":"2023-10-16T11:17:40.003717+0000","last_fullsized":"2023-10-16T11:17:40.003717+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:58.913565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.998133+0000","last_change":"2023-10-16T11:17:39.043356+0000","last_active":"2023-10-16T11:17:39.998133+0000","last_peered":"2023-10-16T11:17:39.998133+0000","last_clean":"2023-10-16T11:17:39.998133+0000","last_became_active":"2023-10-16T11:17:39.043013+0000","last_became_peered":"2023-10-16T11:17:39.043013+0000","last_unstale":"2023-10-16T11:17:39.998133+0000","last_undegraded":"2023-10-16T11:17:39.998133+0000","last_fullsized":"2023-10-16T11:17:39.998133+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:29.002116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989576+0000","last_change":"2023-10-16T11:17:40.005239+0000","last_active":"2023-10-16T11:17:40.989576+0000","last_peered":"2023-10-16T11:17:40.989576+0000","last_clean":"2023-10-16T11:17:40.989576+0000","last_became_active":"2023-10-16T11:17:40.004760+0000","last_became_peered":"2023-10-16T11:17:40.004760+0000","last_unstale":"2023-10-16T11:17:40.989576+0000","last_undegraded":"2023-10-16T11:17:40.989576+0000","last_fullsized":"2023-10-16T11:17:40.989576+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:08.800146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.991672+0000","last_change":"2023-10-16T11:17:39.050290+0000","last_active":"2023-10-16T11:17:39.991672+0000","last_peered":"2023-10-16T11:17:39.991672+0000","last_clean":"2023-10-16T11:17:39.991672+0000","last_became_active":"2023-10-16T11:17:39.050040+0000","last_became_peered":"2023-10-16T11:17:39.050040+0000","last_unstale":"2023-10-16T11:17:39.991672+0000","last_undegraded":"2023-10-16T11:17:39.991672+0000","last_fullsized":"2023-10-16T11:17:39.991672+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:03.742360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001213+0000","last_change":"2023-10-16T11:17:40.000070+0000","last_active":"2023-10-16T11:17:40.001213+0000","last_peered":"2023-10-16T11:17:40.001213+0000","last_clean":"2023-10-16T11:17:40.001213+0000","last_became_active":"2023-10-16T11:17:39.999790+0000","last_became_peered":"2023-10-16T11:17:39.999790+0000","last_unstale":"2023-10-16T11:17:40.001213+0000","last_undegraded":"2023-10-16T11:17:40.001213+0000","last_fullsized":"2023-10-16T11:17:40.001213+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:29.427180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.988919+0000","last_change":"2023-10-16T11:17:39.045021+0000","last_active":"2023-10-16T11:17:39.988919+0000","last_peered":"2023-10-16T11:17:39.988919+0000","last_clean":"2023-10-16T11:17:39.988919+0000","last_became_active":"2023-10-16T11:17:39.044715+0000","last_became_peered":"2023-10-16T11:17:39.044715+0000","last_unstale":"2023-10-16T11:17:39.988919+0000","last_undegraded":"2023-10-16T11:17:39.988919+0000","last_fullsized":"2023-10-16T11:17:39.988919+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:41:20.050417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"188'326","reported_seq":1113,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.035240+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:17:39.035240+0000","last_peered":"2023-10-16T11:17:39.035240+0000","last_clean":"2023-10-16T11:17:39.035240+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:17:39.035240+0000","last_undegraded":"2023-10-16T11:17:39.035240+0000","last_fullsized":"2023-10-16T11:17:39.035240+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543260,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52244,"kb_used_data":3008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319596,"statfs":{"total":96636764160,"available":96583266304,"internally_reserved":0,"allocated":3080192,"data_stored":1854824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.434,"5min":0.316,"15min":0.23799999999999999},"max":{"1min":0.75900000000000001,"5min":0.78200000000000003,"15min":0.84499999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.371,"15min":0.27800000000000002},"max":{"1min":0.748,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56899999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79300000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.46700000000000003,"5min":0.33600000000000002,"15min":0.33300000000000002},"max":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.40500000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.62},"min":{"1min":0.45500000000000002,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.625}]},{"osd":3,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.74199999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.436,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":0.91500000000000004,"15min":1.002},"last":0.59499999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.76600000000000001,"15min":0.83899999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51300000000000001,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.33100000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.78700000000000003,"15min":0.82599999999999996},"last":0.42299999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.52700000000000002,"5min":0.45900000000000002,"15min":0.32400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.39200000000000002,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.45300000000000001,"5min":0.39200000000000002,"15min":0.371},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.871},"last":0.5}]}]},{"osd":6,"up_from":14,"seq":60129543259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37768,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2478080,"data_stored":1264352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.81899999999999995,"15min":0.875},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.42299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75600000000000001,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.47999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63200000000000001,"15min":0.61599999999999999},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81100000000000005,"5min":0.82899999999999996,"15min":0.83299999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.39400000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.495},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.29699999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.39100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.28100000000000003},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.82699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66500000000000004}]},{"osd":7,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.47199999999999998,"5min":0.376,"15min":0.32100000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.622}]}]},{"osd":1,"up_from":14,"seq":60129543259,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55824,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316016,"statfs":{"total":96636764160,"available":96579600384,"internally_reserved":0,"allocated":2490368,"data_stored":1265964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.318,"5min":0.224,"15min":0.221},"max":{"1min":0.58499999999999996,"5min":0.629,"15min":0.68700000000000006},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.28199999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53700000000000003,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.312}]},{"osd":2,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.34399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51700000000000002,"5min":0.61599999999999999,"15min":0.98899999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.60899999999999999,"5min":0.69399999999999995,"15min":0.83099999999999996},"last":0.30099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.496,"15min":0.497},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.29399999999999998},"max":{"1min":0.57999999999999996,"5min":0.755,"15min":0.78300000000000003},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.30299999999999999},"max":{"1min":0.624,"5min":0.68300000000000005,"15min":0.70699999999999996},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.66600000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90000000000000002,"15min":0.98399999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.56200000000000006,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.097},"last":0.54900000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68600000000000005,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.79500000000000004,"5min":0.96699999999999997,"15min":1.147},"last":0.51400000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.0509999999999999,"15min":1.357},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.58699999999999997,"5min":0.49299999999999999,"15min":0.47899999999999998},"max":{"1min":0.84799999999999998,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.0600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.51000000000000001,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":1.044,"15min":1.1060000000000001},"last":0.46700000000000003}]}]},{"osd":0,"up_from":14,"seq":60129543260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56396,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315444,"statfs":{"total":96636764160,"available":96579014656,"internally_reserved":0,"allocated":3076096,"data_stored":1854742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.28199999999999997,"5min":0.214,"15min":0.214},"max":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.91600000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.432}]},{"osd":2,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.311,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58299999999999996,"5min":0.67300000000000004,"15min":0.70299999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44800000000000001,"15min":0.45600000000000002},"min":{"1min":0.32800000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.625,"5min":0.63300000000000001,"15min":0.86799999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.66200000000000003,"15min":0.68300000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.39800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64400000000000002,"5min":0.69799999999999995,"15min":0.84599999999999997},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.50700000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.99199999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.499,"5min":0.41199999999999998,"15min":0.379},"max":{"1min":0.91400000000000003,"5min":0.93700000000000006,"15min":1.008},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66700000000000004},"min":{"1min":0.54700000000000004,"5min":0.45300000000000001,"15min":0.433},"max":{"1min":0.91500000000000004,"5min":1.004,"15min":1.018},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.69799999999999995},"min":{"1min":0.55200000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88400000000000001,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.67600000000000005},"min":{"1min":0.53600000000000003,"5min":0.42199999999999999,"15min":0.41499999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0049999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.51500000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.84599999999999997,"5min":1.01,"15min":1.0449999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.95899999999999996,"15min":1.077},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.70399999999999996},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.87,"5min":0.95699999999999996,"15min":0.96599999999999997},"last":0.69999999999999996}]}]},{"osd":2,"up_from":14,"seq":60129543259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59984,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311856,"statfs":{"total":96636764160,"available":96575340544,"internally_reserved":0,"allocated":2490368,"data_stored":1265046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.52900000000000003,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.63},"last":0.31}]},{"osd":1,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.44,"15min":0.44400000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.26700000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":0.38}]},{"osd":3,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.436,"15min":0.439},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28399999999999997},"max":{"1min":0.623,"5min":0.623,"15min":0.65500000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.81200000000000006},"last":0.41499999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.47999999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67600000000000005},"min":{"1min":0.52800000000000002,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":0.97199999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":0.98399999999999999},"last":0.745},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.46500000000000002},"max":{"1min":1.0129999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.55800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96199999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.499},"max":{"1min":0.97299999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72199999999999998}]}]},{"osd":3,"up_from":14,"seq":60129543260,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64144,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307696,"statfs":{"total":96636764160,"available":96571080704,"internally_reserved":0,"allocated":2490368,"data_stored":1264546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":63175112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.52600000000000002,"5min":0.627,"15min":0.70399999999999996},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.38200000000000001,"5min":0.32800000000000001,"15min":0.24399999999999999},"max":{"1min":0.52500000000000002,"5min":0.62,"15min":0.70999999999999996},"last":0.28199999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.56699999999999995,"5min":0.64100000000000001,"15min":0.78100000000000003},"last":0.253},{"interface":"front","average":{"1min":0.443,"5min":0.45600000000000002,"15min":0.45700000000000002},"min":{"1min":0.314,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.58699999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.32200000000000001},"max":{"1min":0.57399999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.58699999999999997,"5min":0.70399999999999996,"15min":0.749},"last":0.316}]},{"osd":4,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.52100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86399999999999999,"5min":1.034,"15min":1.0740000000000001},"last":0.375},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.42999999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.65300000000000002},"min":{"1min":0.49099999999999999,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":0.88400000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.501,"5min":0.46700000000000003,"15min":0.46600000000000003},"max":{"1min":0.92400000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.52700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89300000000000002,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.44},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65500000000000003,"15min":0.66600000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.34599999999999997},"max":{"1min":0.76100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.46899999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.51000000000000001,"5min":0.47999999999999998,"15min":0.40000000000000002},"max":{"1min":0.877,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":1.016,"15min":1.0489999999999999},"last":0.38900000000000001}]}]},{"osd":4,"up_from":14,"seq":60129543259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55824,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316016,"statfs":{"total":96636764160,"available":96579600384,"internally_reserved":0,"allocated":2494464,"data_stored":1266546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.372},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.38600000000000001,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.68600000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.52400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.621},"min":{"1min":0.434,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.79800000000000004,"15min":0.82099999999999995},"last":0.66200000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.38600000000000001},"max":{"1min":0.77700000000000002,"5min":0.83399999999999996,"15min":0.85099999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.47999999999999998,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.503,"5min":0.433,"15min":0.34399999999999997},"max":{"1min":0.80200000000000005,"5min":0.81799999999999995,"15min":0.85999999999999999},"last":0.629},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65400000000000003},"min":{"1min":0.51800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80500000000000005,"5min":0.879,"15min":0.879},"last":0.67100000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.80100000000000005,"15min":0.84599999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.89500000000000002},"last":0.55300000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.443,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.91300000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.35899999999999999,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89000000000000001,"15min":0.89700000000000002},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.82299999999999995,"15min":0.89000000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.629},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.95299999999999996},"last":0.70299999999999996}]}]},{"osd":5,"up_from":14,"seq":60129543260,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41920,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329920,"statfs":{"total":96636764160,"available":96593838080,"internally_reserved":0,"allocated":2478080,"data_stored":1264352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.50600000000000001,"5min":0.40100000000000002,"15min":0.29599999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.747,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.84099999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.47499999999999998,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":0.84199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.80000000000000004,"15min":0.84899999999999998},"last":0.505}]},{"osd":3,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.86799999999999999,"15min":1.1379999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.51900000000000002,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.81399999999999995,"5min":0.871,"15min":0.92100000000000004},"last":0.40699999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.316},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.85499999999999998},"last":0.55100000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.69999999999999996,"5min":0.873,"15min":0.873},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76900000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.41599999999999998,"5min":0.371,"15min":0.35299999999999998},"max":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.88500000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:17:44.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:17:44.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.303+0000 7fa836585700 1 Processor -- start 2023-10-16T11:17:44.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.303+0000 7fa836585700 1 -- start start 2023-10-16T11:17:44.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa836585700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa83015dfd0 0x7fa8300af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa836585700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa8300af450 0x7fa83006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa836585700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83006f870 con 0x7fa83006db80 2023-10-16T11:17:44.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa836585700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300d1160 con 0x7fa83015dfd0 2023-10-16T11:17:44.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa836585700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300cc8e0 con 0x7fa8300af450 2023-10-16T11:17:44.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa82ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa83015dfd0 0x7fa8300af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa82ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa83015dfd0 0x7fa8300af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53972/0 (socket says 172.21.15.181:53972) 2023-10-16T11:17:44.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa82ffff700 1 -- 172.21.15.181:0/3277806713 learned_addr learned my addr 172.21.15.181:0/3277806713 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:44.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.304+0000 7fa82f7fe700 1 --2- 172.21.15.181:0/3277806713 >> v2:172.21.15.190:3300/0 conn(0x7fa8300af450 0x7fa83006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82ffff700 1 -- 172.21.15.181:0/3277806713 >> v2:172.21.15.190:3300/0 conn(0x7fa8300af450 msgr2=0x7fa83006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:44.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82ffff700 1 --2- 172.21.15.181:0/3277806713 >> v2:172.21.15.190:3300/0 conn(0x7fa8300af450 0x7fa83006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82ffff700 1 -- 172.21.15.181:0/3277806713 >> v1:172.21.15.190:6789/0 conn(0x7fa83006db80 legacy=0x7fa8300cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82ffff700 1 -- 172.21.15.181:0/3277806713 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300ccbc0 con 0x7fa83015dfd0 2023-10-16T11:17:44.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82ffff700 1 --2- 172.21.15.181:0/3277806713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300af9a0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b3070 tx=0x7fa824009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e865095f6104549f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:44.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82effd700 1 -- 172.21.15.181:0/3277806713 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824005e30 con 0x7fa83015dfd0 2023-10-16T11:17:44.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82effd700 1 -- 172.21.15.181:0/3277806713 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa824004030 con 0x7fa83015dfd0 2023-10-16T11:17:44.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa82effd700 1 -- 172.21.15.181:0/3277806713 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240043e0 con 0x7fa83015dfd0 2023-10-16T11:17:44.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa836585700 1 -- 172.21.15.181:0/3277806713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 msgr2=0x7fa8300af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa836585700 1 --2- 172.21.15.181:0/3277806713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300af9a0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b3070 tx=0x7fa824009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa836585700 1 -- 172.21.15.181:0/3277806713 shutdown_connections 2023-10-16T11:17:44.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa836585700 1 --2- 172.21.15.181:0/3277806713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300af9a0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa836585700 1 --2- 172.21.15.181:0/3277806713 >> v2:172.21.15.190:3300/0 conn(0x7fa8300af450 0x7fa83006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa836585700 1 -- 172.21.15.181:0/3277806713 >> 172.21.15.181:0/3277806713 conn(0x7fa8300d04a0 msgr2=0x7fa8300d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:44.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.305+0000 7fa836585700 1 -- 172.21.15.181:0/3277806713 shutdown_connections 2023-10-16T11:17:44.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.306+0000 7fa836585700 1 -- 172.21.15.181:0/3277806713 wait complete. 2023-10-16T11:17:44.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.307+0000 7fa836585700 1 Processor -- start 2023-10-16T11:17:44.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.307+0000 7fa836585700 1 -- start start 2023-10-16T11:17:44.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.307+0000 7fa836585700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83006db80 0x7fa8300af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa836585700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8300af450 0x7fa8300c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa836585700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa836585700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300c4c10 con 0x7fa83015dfd0 2023-10-16T11:17:44.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa836585700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300b2e80 con 0x7fa8300af450 2023-10-16T11:17:44.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa836585700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300c7ab0 con 0x7fa83006db80 2023-10-16T11:17:44.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53980/0 (socket says 172.21.15.181:53980) 2023-10-16T11:17:44.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 -- 172.21.15.181:0/959896253 learned_addr learned my addr 172.21.15.181:0/959896253 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:44.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa82ffff700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83006db80 0x7fa8300af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa82f7fe700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8300af450 0x7fa8300c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 -- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83006db80 msgr2=0x7fa8300af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83006db80 0x7fa8300af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 -- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8300af450 msgr2=0x7fa8300c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8300af450 0x7fa8300c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 -- 172.21.15.181:0/959896253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009500 con 0x7fa83015dfd0 2023-10-16T11:17:44.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa82f7fe700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8300af450 0x7fa8300c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:44.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa834b22700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300c7f80 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fa82800cc50 tx=0x7fa82800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:44.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa82cff9700 1 -- 172.21.15.181:0/959896253 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828003c00 con 0x7fa83015dfd0 2023-10-16T11:17:44.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa82cff9700 1 -- 172.21.15.181:0/959896253 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828003d60 con 0x7fa83015dfd0 2023-10-16T11:17:44.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8300ca0c0 con 0x7fa83015dfd0 2023-10-16T11:17:44.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa82cff9700 1 -- 172.21.15.181:0/959896253 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82800ab60 con 0x7fa83015dfd0 2023-10-16T11:17:44.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.308+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8300ca320 con 0x7fa83015dfd0 2023-10-16T11:17:44.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.309+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8300c4d50 con 0x7fa83015dfd0 2023-10-16T11:17:44.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.310+0000 7fa82cff9700 1 -- 172.21.15.181:0/959896253 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa82800acc0 con 0x7fa83015dfd0 2023-10-16T11:17:44.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.310+0000 7fa82cff9700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa81c06b020 0x7fa81c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.310+0000 7fa82cff9700 1 -- 172.21.15.181:0/959896253 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7fa828089940 con 0x7fa83015dfd0 2023-10-16T11:17:44.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.310+0000 7fa82ffff700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa81c06b020 0x7fa81c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.311+0000 7fa82ffff700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa81c06b020 0x7fa81c06d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b3070 tx=0x7fa824008070 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:44.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.315+0000 7fa82cff9700 1 -- 172.21.15.181:0/959896253 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa8300c4d50 con 0x7fa83015dfd0 2023-10-16T11:17:44.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.506+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 --> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8300cc590 con 0x7fa81c06b020 2023-10-16T11:17:44.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.514+0000 7fa82cff9700 1 -- 172.21.15.181:0/959896253 <== mgr.10542 v2:172.21.15.181:6832/1532329351 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290276 (secure 0 0 0) 0x7fa824022070 con 0x7fa81c06b020 2023-10-16T11:17:44.519 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:44.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa81c06b020 msgr2=0x7fa81c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa81c06b020 0x7fa81c06d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b3070 tx=0x7fa824008070 comp rx=0 tx=0).stop 2023-10-16T11:17:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 msgr2=0x7fa8300c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300c7f80 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fa82800cc50 tx=0x7fa82800cf60 comp rx=0 tx=0).stop 2023-10-16T11:17:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 shutdown_connections 2023-10-16T11:17:44.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa81c06b020 0x7fa81c06d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa83006db80 0x7fa8300af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83015dfd0 0x7fa8300c7f80 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 --2- 172.21.15.181:0/959896253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8300af450 0x7fa8300c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 >> 172.21.15.181:0/959896253 conn(0x7fa8300d04a0 msgr2=0x7fa8300cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:44.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.517+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 shutdown_connections 2023-10-16T11:17:44.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.518+0000 7fa836585700 1 -- 172.21.15.181:0/959896253 wait complete. 2023-10-16T11:17:44.524 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:17:44.531 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-16T11:17:42.985586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":424104,"kb_used_data":20584,"kb_used_omap":12,"kb_used_meta":403379,"kb_avail":754550616,"statfs":{"total":773094113280,"available":772659830784,"internally_reserved":0,"allocated":21078016,"data_stored":11300372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":413060664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.960728"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003815+0000","last_change":"2023-10-16T11:17:40.002067+0000","last_active":"2023-10-16T11:17:40.003815+0000","last_peered":"2023-10-16T11:17:40.003815+0000","last_clean":"2023-10-16T11:17:40.003815+0000","last_became_active":"2023-10-16T11:17:40.001815+0000","last_became_peered":"2023-10-16T11:17:40.001815+0000","last_unstale":"2023-10-16T11:17:40.003815+0000","last_undegraded":"2023-10-16T11:17:40.003815+0000","last_fullsized":"2023-10-16T11:17:40.003815+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:34:34.945432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007347+0000","last_change":"2023-10-16T11:17:40.002661+0000","last_active":"2023-10-16T11:17:40.007347+0000","last_peered":"2023-10-16T11:17:40.007347+0000","last_clean":"2023-10-16T11:17:40.007347+0000","last_became_active":"2023-10-16T11:17:40.002402+0000","last_became_peered":"2023-10-16T11:17:40.002402+0000","last_unstale":"2023-10-16T11:17:40.007347+0000","last_undegraded":"2023-10-16T11:17:40.007347+0000","last_fullsized":"2023-10-16T11:17:40.007347+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:21.387159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.999340+0000","last_change":"2023-10-16T11:17:39.998136+0000","last_active":"2023-10-16T11:17:39.999340+0000","last_peered":"2023-10-16T11:17:39.999340+0000","last_clean":"2023-10-16T11:17:39.999340+0000","last_became_active":"2023-10-16T11:17:39.997838+0000","last_became_peered":"2023-10-16T11:17:39.997838+0000","last_unstale":"2023-10-16T11:17:39.999340+0000","last_undegraded":"2023-10-16T11:17:39.999340+0000","last_fullsized":"2023-10-16T11:17:39.999340+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:20:12.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007391+0000","last_change":"2023-10-16T11:17:40.002961+0000","last_active":"2023-10-16T11:17:40.007391+0000","last_peered":"2023-10-16T11:17:40.007391+0000","last_clean":"2023-10-16T11:17:40.007391+0000","last_became_active":"2023-10-16T11:17:40.002479+0000","last_became_peered":"2023-10-16T11:17:40.002479+0000","last_unstale":"2023-10-16T11:17:40.007391+0000","last_undegraded":"2023-10-16T11:17:40.007391+0000","last_fullsized":"2023-10-16T11:17:40.007391+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:10:05.191201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989997+0000","last_change":"2023-10-16T11:17:40.006528+0000","last_active":"2023-10-16T11:17:40.989997+0000","last_peered":"2023-10-16T11:17:40.989997+0000","last_clean":"2023-10-16T11:17:40.989997+0000","last_became_active":"2023-10-16T11:17:40.006247+0000","last_became_peered":"2023-10-16T11:17:40.006247+0000","last_unstale":"2023-10-16T11:17:40.989997+0000","last_undegraded":"2023-10-16T11:17:40.989997+0000","last_fullsized":"2023-10-16T11:17:40.989997+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:53.566509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007385+0000","last_change":"2023-10-16T11:17:40.000087+0000","last_active":"2023-10-16T11:17:40.007385+0000","last_peered":"2023-10-16T11:17:40.007385+0000","last_clean":"2023-10-16T11:17:40.007385+0000","last_became_active":"2023-10-16T11:17:39.999845+0000","last_became_peered":"2023-10-16T11:17:39.999845+0000","last_unstale":"2023-10-16T11:17:40.007385+0000","last_undegraded":"2023-10-16T11:17:40.007385+0000","last_fullsized":"2023-10-16T11:17:40.007385+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:20:45.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990904+0000","last_change":"2023-10-16T11:17:39.999005+0000","last_active":"2023-10-16T11:17:40.990904+0000","last_peered":"2023-10-16T11:17:40.990904+0000","last_clean":"2023-10-16T11:17:40.990904+0000","last_became_active":"2023-10-16T11:17:39.998648+0000","last_became_peered":"2023-10-16T11:17:39.998648+0000","last_unstale":"2023-10-16T11:17:40.990904+0000","last_undegraded":"2023-10-16T11:17:40.990904+0000","last_fullsized":"2023-10-16T11:17:40.990904+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:40:24.078483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001429+0000","last_change":"2023-10-16T11:17:40.000258+0000","last_active":"2023-10-16T11:17:40.001429+0000","last_peered":"2023-10-16T11:17:40.001429+0000","last_clean":"2023-10-16T11:17:40.001429+0000","last_became_active":"2023-10-16T11:17:39.999988+0000","last_became_peered":"2023-10-16T11:17:39.999988+0000","last_unstale":"2023-10-16T11:17:40.001429+0000","last_undegraded":"2023-10-16T11:17:40.001429+0000","last_fullsized":"2023-10-16T11:17:40.001429+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:01:31.212627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990202+0000","last_change":"2023-10-16T11:17:40.003342+0000","last_active":"2023-10-16T11:17:40.990202+0000","last_peered":"2023-10-16T11:17:40.990202+0000","last_clean":"2023-10-16T11:17:40.990202+0000","last_became_active":"2023-10-16T11:17:40.003038+0000","last_became_peered":"2023-10-16T11:17:40.003038+0000","last_unstale":"2023-10-16T11:17:40.990202+0000","last_undegraded":"2023-10-16T11:17:40.990202+0000","last_fullsized":"2023-10-16T11:17:40.990202+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:22.537991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.000348+0000","last_change":"2023-10-16T11:17:39.999623+0000","last_active":"2023-10-16T11:17:40.000348+0000","last_peered":"2023-10-16T11:17:40.000348+0000","last_clean":"2023-10-16T11:17:40.000348+0000","last_became_active":"2023-10-16T11:17:39.999354+0000","last_became_peered":"2023-10-16T11:17:39.999354+0000","last_unstale":"2023-10-16T11:17:40.000348+0000","last_undegraded":"2023-10-16T11:17:40.000348+0000","last_fullsized":"2023-10-16T11:17:40.000348+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:43.659165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991381+0000","last_change":"2023-10-16T11:17:40.003172+0000","last_active":"2023-10-16T11:17:40.991381+0000","last_peered":"2023-10-16T11:17:40.991381+0000","last_clean":"2023-10-16T11:17:40.991381+0000","last_became_active":"2023-10-16T11:17:40.002845+0000","last_became_peered":"2023-10-16T11:17:40.002845+0000","last_unstale":"2023-10-16T11:17:40.991381+0000","last_undegraded":"2023-10-16T11:17:40.991381+0000","last_fullsized":"2023-10-16T11:17:40.991381+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:27.380826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007373+0000","last_change":"2023-10-16T11:17:40.002773+0000","last_active":"2023-10-16T11:17:40.007373+0000","last_peered":"2023-10-16T11:17:40.007373+0000","last_clean":"2023-10-16T11:17:40.007373+0000","last_became_active":"2023-10-16T11:17:40.002171+0000","last_became_peered":"2023-10-16T11:17:40.002171+0000","last_unstale":"2023-10-16T11:17:40.007373+0000","last_undegraded":"2023-10-16T11:17:40.007373+0000","last_fullsized":"2023-10-16T11:17:40.007373+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:25.015905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.004938+0000","last_change":"2023-10-16T11:17:40.000380+0000","last_active":"2023-10-16T11:17:40.004938+0000","last_peered":"2023-10-16T11:17:40.004938+0000","last_clean":"2023-10-16T11:17:40.004938+0000","last_became_active":"2023-10-16T11:17:39.999721+0000","last_became_peered":"2023-10-16T11:17:39.999721+0000","last_unstale":"2023-10-16T11:17:40.004938+0000","last_undegraded":"2023-10-16T11:17:40.004938+0000","last_fullsized":"2023-10-16T11:17:40.004938+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:32:43.659165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989972+0000","last_change":"2023-10-16T11:17:40.004408+0000","last_active":"2023-10-16T11:17:40.989972+0000","last_peered":"2023-10-16T11:17:40.989972+0000","last_clean":"2023-10-16T11:17:40.989972+0000","last_became_active":"2023-10-16T11:17:40.004104+0000","last_became_peered":"2023-10-16T11:17:40.004104+0000","last_unstale":"2023-10-16T11:17:40.989972+0000","last_undegraded":"2023-10-16T11:17:40.989972+0000","last_fullsized":"2023-10-16T11:17:40.989972+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:16.847184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.005904+0000","last_change":"2023-10-16T11:17:40.001429+0000","last_active":"2023-10-16T11:17:40.005904+0000","last_peered":"2023-10-16T11:17:40.005904+0000","last_clean":"2023-10-16T11:17:40.005904+0000","last_became_active":"2023-10-16T11:17:40.001157+0000","last_became_peered":"2023-10-16T11:17:40.001157+0000","last_unstale":"2023-10-16T11:17:40.005904+0000","last_undegraded":"2023-10-16T11:17:40.005904+0000","last_fullsized":"2023-10-16T11:17:40.005904+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:56.088053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001064+0000","last_change":"2023-10-16T11:17:39.997261+0000","last_active":"2023-10-16T11:17:40.001064+0000","last_peered":"2023-10-16T11:17:40.001064+0000","last_clean":"2023-10-16T11:17:40.001064+0000","last_became_active":"2023-10-16T11:17:39.996970+0000","last_became_peered":"2023-10-16T11:17:39.996970+0000","last_unstale":"2023-10-16T11:17:40.001064+0000","last_undegraded":"2023-10-16T11:17:40.001064+0000","last_fullsized":"2023-10-16T11:17:40.001064+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:42:30.025545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989702+0000","last_change":"2023-10-16T11:17:40.003406+0000","last_active":"2023-10-16T11:17:40.989702+0000","last_peered":"2023-10-16T11:17:40.989702+0000","last_clean":"2023-10-16T11:17:40.989702+0000","last_became_active":"2023-10-16T11:17:40.003118+0000","last_became_peered":"2023-10-16T11:17:40.003118+0000","last_unstale":"2023-10-16T11:17:40.989702+0000","last_undegraded":"2023-10-16T11:17:40.989702+0000","last_fullsized":"2023-10-16T11:17:40.989702+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:03.164357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001354+0000","last_change":"2023-10-16T11:17:39.999356+0000","last_active":"2023-10-16T11:17:40.001354+0000","last_peered":"2023-10-16T11:17:40.001354+0000","last_clean":"2023-10-16T11:17:40.001354+0000","last_became_active":"2023-10-16T11:17:39.998587+0000","last_became_peered":"2023-10-16T11:17:39.998587+0000","last_unstale":"2023-10-16T11:17:40.001354+0000","last_undegraded":"2023-10-16T11:17:40.001354+0000","last_fullsized":"2023-10-16T11:17:40.001354+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:18.549246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.002142+0000","last_change":"2023-10-16T11:17:40.000460+0000","last_active":"2023-10-16T11:17:40.002142+0000","last_peered":"2023-10-16T11:17:40.002142+0000","last_clean":"2023-10-16T11:17:40.002142+0000","last_became_active":"2023-10-16T11:17:40.000196+0000","last_became_peered":"2023-10-16T11:17:40.000196+0000","last_unstale":"2023-10-16T11:17:40.002142+0000","last_undegraded":"2023-10-16T11:17:40.002142+0000","last_fullsized":"2023-10-16T11:17:40.002142+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.173968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989668+0000","last_change":"2023-10-16T11:17:40.002691+0000","last_active":"2023-10-16T11:17:40.989668+0000","last_peered":"2023-10-16T11:17:40.989668+0000","last_clean":"2023-10-16T11:17:40.989668+0000","last_became_active":"2023-10-16T11:17:40.002416+0000","last_became_peered":"2023-10-16T11:17:40.002416+0000","last_unstale":"2023-10-16T11:17:40.989668+0000","last_undegraded":"2023-10-16T11:17:40.989668+0000","last_fullsized":"2023-10-16T11:17:40.989668+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:32.527025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990091+0000","last_change":"2023-10-16T11:17:40.001981+0000","last_active":"2023-10-16T11:17:40.990091+0000","last_peered":"2023-10-16T11:17:40.990091+0000","last_clean":"2023-10-16T11:17:40.990091+0000","last_became_active":"2023-10-16T11:17:40.001576+0000","last_became_peered":"2023-10-16T11:17:40.001576+0000","last_unstale":"2023-10-16T11:17:40.990091+0000","last_undegraded":"2023-10-16T11:17:40.990091+0000","last_fullsized":"2023-10-16T11:17:40.990091+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:27.874724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006604+0000","last_change":"2023-10-16T11:17:40.002928+0000","last_active":"2023-10-16T11:17:40.006604+0000","last_peered":"2023-10-16T11:17:40.006604+0000","last_clean":"2023-10-16T11:17:40.006604+0000","last_became_active":"2023-10-16T11:17:40.002673+0000","last_became_peered":"2023-10-16T11:17:40.002673+0000","last_unstale":"2023-10-16T11:17:40.006604+0000","last_undegraded":"2023-10-16T11:17:40.006604+0000","last_fullsized":"2023-10-16T11:17:40.006604+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:23.780765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.998513+0000","last_change":"2023-10-16T11:17:39.996631+0000","last_active":"2023-10-16T11:17:39.998513+0000","last_peered":"2023-10-16T11:17:39.998513+0000","last_clean":"2023-10-16T11:17:39.998513+0000","last_became_active":"2023-10-16T11:17:39.996366+0000","last_became_peered":"2023-10-16T11:17:39.996366+0000","last_unstale":"2023-10-16T11:17:39.998513+0000","last_undegraded":"2023-10-16T11:17:39.998513+0000","last_fullsized":"2023-10-16T11:17:39.998513+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:12:39.677767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001044+0000","last_change":"2023-10-16T11:17:39.994342+0000","last_active":"2023-10-16T11:17:40.001044+0000","last_peered":"2023-10-16T11:17:40.001044+0000","last_clean":"2023-10-16T11:17:40.001044+0000","last_became_active":"2023-10-16T11:17:39.993965+0000","last_became_peered":"2023-10-16T11:17:39.993965+0000","last_unstale":"2023-10-16T11:17:40.001044+0000","last_undegraded":"2023-10-16T11:17:40.001044+0000","last_fullsized":"2023-10-16T11:17:40.001044+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:45.752651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.004380+0000","last_change":"2023-10-16T11:17:40.000972+0000","last_active":"2023-10-16T11:17:40.004380+0000","last_peered":"2023-10-16T11:17:40.004380+0000","last_clean":"2023-10-16T11:17:40.004380+0000","last_became_active":"2023-10-16T11:17:40.000690+0000","last_became_peered":"2023-10-16T11:17:40.000690+0000","last_unstale":"2023-10-16T11:17:40.004380+0000","last_undegraded":"2023-10-16T11:17:40.004380+0000","last_fullsized":"2023-10-16T11:17:40.004380+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:10.173968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991528+0000","last_change":"2023-10-16T11:17:40.001789+0000","last_active":"2023-10-16T11:17:40.991528+0000","last_peered":"2023-10-16T11:17:40.991528+0000","last_clean":"2023-10-16T11:17:40.991528+0000","last_became_active":"2023-10-16T11:17:40.001503+0000","last_became_peered":"2023-10-16T11:17:40.001503+0000","last_unstale":"2023-10-16T11:17:40.991528+0000","last_undegraded":"2023-10-16T11:17:40.991528+0000","last_fullsized":"2023-10-16T11:17:40.991528+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:29.353229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007247+0000","last_change":"2023-10-16T11:17:40.001054+0000","last_active":"2023-10-16T11:17:40.007247+0000","last_peered":"2023-10-16T11:17:40.007247+0000","last_clean":"2023-10-16T11:17:40.007247+0000","last_became_active":"2023-10-16T11:17:39.999825+0000","last_became_peered":"2023-10-16T11:17:39.999825+0000","last_unstale":"2023-10-16T11:17:40.007247+0000","last_undegraded":"2023-10-16T11:17:40.007247+0000","last_fullsized":"2023-10-16T11:17:40.007247+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:53.686911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989893+0000","last_change":"2023-10-16T11:17:39.997423+0000","last_active":"2023-10-16T11:17:40.989893+0000","last_peered":"2023-10-16T11:17:40.989893+0000","last_clean":"2023-10-16T11:17:40.989893+0000","last_became_active":"2023-10-16T11:17:39.997145+0000","last_became_peered":"2023-10-16T11:17:39.997145+0000","last_unstale":"2023-10-16T11:17:40.989893+0000","last_undegraded":"2023-10-16T11:17:40.989893+0000","last_fullsized":"2023-10-16T11:17:40.989893+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:24.597213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.004963+0000","last_change":"2023-10-16T11:17:40.000564+0000","last_active":"2023-10-16T11:17:40.004963+0000","last_peered":"2023-10-16T11:17:40.004963+0000","last_clean":"2023-10-16T11:17:40.004963+0000","last_became_active":"2023-10-16T11:17:40.000118+0000","last_became_peered":"2023-10-16T11:17:40.000118+0000","last_unstale":"2023-10-16T11:17:40.004963+0000","last_undegraded":"2023-10-16T11:17:40.004963+0000","last_fullsized":"2023-10-16T11:17:40.004963+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:23.780765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989972+0000","last_change":"2023-10-16T11:17:40.004594+0000","last_active":"2023-10-16T11:17:40.989972+0000","last_peered":"2023-10-16T11:17:40.989972+0000","last_clean":"2023-10-16T11:17:40.989972+0000","last_became_active":"2023-10-16T11:17:40.000497+0000","last_became_peered":"2023-10-16T11:17:40.000497+0000","last_unstale":"2023-10-16T11:17:40.989972+0000","last_undegraded":"2023-10-16T11:17:40.989972+0000","last_fullsized":"2023-10-16T11:17:40.989972+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:01.138780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007430+0000","last_change":"2023-10-16T11:17:40.004245+0000","last_active":"2023-10-16T11:17:40.007430+0000","last_peered":"2023-10-16T11:17:40.007430+0000","last_clean":"2023-10-16T11:17:40.007430+0000","last_became_active":"2023-10-16T11:17:40.003994+0000","last_became_peered":"2023-10-16T11:17:40.003994+0000","last_unstale":"2023-10-16T11:17:40.007430+0000","last_undegraded":"2023-10-16T11:17:40.007430+0000","last_fullsized":"2023-10-16T11:17:40.007430+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:48.531341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989504+0000","last_change":"2023-10-16T11:17:40.004924+0000","last_active":"2023-10-16T11:17:40.989504+0000","last_peered":"2023-10-16T11:17:40.989504+0000","last_clean":"2023-10-16T11:17:40.989504+0000","last_became_active":"2023-10-16T11:17:40.004653+0000","last_became_peered":"2023-10-16T11:17:40.004653+0000","last_unstale":"2023-10-16T11:17:40.989504+0000","last_undegraded":"2023-10-16T11:17:40.989504+0000","last_fullsized":"2023-10-16T11:17:40.989504+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:43:42.264332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001330+0000","last_change":"2023-10-16T11:17:39.999441+0000","last_active":"2023-10-16T11:17:40.001330+0000","last_peered":"2023-10-16T11:17:40.001330+0000","last_clean":"2023-10-16T11:17:40.001330+0000","last_became_active":"2023-10-16T11:17:39.998692+0000","last_became_peered":"2023-10-16T11:17:39.998692+0000","last_unstale":"2023-10-16T11:17:40.001330+0000","last_undegraded":"2023-10-16T11:17:40.001330+0000","last_fullsized":"2023-10-16T11:17:40.001330+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:48.513765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"198'4","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989805+0000","last_change":"2023-10-16T11:17:39.047373+0000","last_active":"2023-10-16T11:17:40.989805+0000","last_peered":"2023-10-16T11:17:40.989805+0000","last_clean":"2023-10-16T11:17:40.989805+0000","last_became_active":"2023-10-16T11:17:39.046457+0000","last_became_peered":"2023-10-16T11:17:39.046457+0000","last_unstale":"2023-10-16T11:17:40.989805+0000","last_undegraded":"2023-10-16T11:17:40.989805+0000","last_fullsized":"2023-10-16T11:17:40.989805+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:17:52.491142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007526+0000","last_change":"2023-10-16T11:17:40.006114+0000","last_active":"2023-10-16T11:17:40.007526+0000","last_peered":"2023-10-16T11:17:40.007526+0000","last_clean":"2023-10-16T11:17:40.007526+0000","last_became_active":"2023-10-16T11:17:40.005833+0000","last_became_peered":"2023-10-16T11:17:40.005833+0000","last_unstale":"2023-10-16T11:17:40.007526+0000","last_undegraded":"2023-10-16T11:17:40.007526+0000","last_fullsized":"2023-10-16T11:17:40.007526+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:41.394184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"198'1","reported_seq":13,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990115+0000","last_change":"2023-10-16T11:17:39.046026+0000","last_active":"2023-10-16T11:17:40.990115+0000","last_peered":"2023-10-16T11:17:40.990115+0000","last_clean":"2023-10-16T11:17:40.990115+0000","last_became_active":"2023-10-16T11:17:39.045708+0000","last_became_peered":"2023-10-16T11:17:39.045708+0000","last_unstale":"2023-10-16T11:17:40.990115+0000","last_undegraded":"2023-10-16T11:17:40.990115+0000","last_fullsized":"2023-10-16T11:17:40.990115+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:18:52.558904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991308+0000","last_change":"2023-10-16T11:17:40.005500+0000","last_active":"2023-10-16T11:17:40.991308+0000","last_peered":"2023-10-16T11:17:40.991308+0000","last_clean":"2023-10-16T11:17:40.991308+0000","last_became_active":"2023-10-16T11:17:40.005206+0000","last_became_peered":"2023-10-16T11:17:40.005206+0000","last_unstale":"2023-10-16T11:17:40.991308+0000","last_undegraded":"2023-10-16T11:17:40.991308+0000","last_fullsized":"2023-10-16T11:17:40.991308+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:12:57.538992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041906+0000","last_change":"2023-10-16T11:17:39.052152+0000","last_active":"2023-10-16T11:17:40.041906+0000","last_peered":"2023-10-16T11:17:40.041906+0000","last_clean":"2023-10-16T11:17:40.041906+0000","last_became_active":"2023-10-16T11:17:39.051683+0000","last_became_peered":"2023-10-16T11:17:39.051683+0000","last_unstale":"2023-10-16T11:17:40.041906+0000","last_undegraded":"2023-10-16T11:17:40.041906+0000","last_fullsized":"2023-10-16T11:17:40.041906+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:20.262802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990112+0000","last_change":"2023-10-16T11:17:39.999259+0000","last_active":"2023-10-16T11:17:40.990112+0000","last_peered":"2023-10-16T11:17:40.990112+0000","last_clean":"2023-10-16T11:17:40.990112+0000","last_became_active":"2023-10-16T11:17:39.999002+0000","last_became_peered":"2023-10-16T11:17:39.999002+0000","last_unstale":"2023-10-16T11:17:40.990112+0000","last_undegraded":"2023-10-16T11:17:40.990112+0000","last_fullsized":"2023-10-16T11:17:40.990112+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:03.507916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041841+0000","last_change":"2023-10-16T11:17:39.046049+0000","last_active":"2023-10-16T11:17:40.041841+0000","last_peered":"2023-10-16T11:17:40.041841+0000","last_clean":"2023-10-16T11:17:40.041841+0000","last_became_active":"2023-10-16T11:17:39.045693+0000","last_became_peered":"2023-10-16T11:17:39.045693+0000","last_unstale":"2023-10-16T11:17:40.041841+0000","last_undegraded":"2023-10-16T11:17:40.041841+0000","last_fullsized":"2023-10-16T11:17:40.041841+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:02.161144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006299+0000","last_change":"2023-10-16T11:17:40.002337+0000","last_active":"2023-10-16T11:17:40.006299+0000","last_peered":"2023-10-16T11:17:40.006299+0000","last_clean":"2023-10-16T11:17:40.006299+0000","last_became_active":"2023-10-16T11:17:40.002046+0000","last_became_peered":"2023-10-16T11:17:40.002046+0000","last_unstale":"2023-10-16T11:17:40.006299+0000","last_undegraded":"2023-10-16T11:17:40.006299+0000","last_fullsized":"2023-10-16T11:17:40.006299+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:06.985430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"198'1","reported_seq":13,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990047+0000","last_change":"2023-10-16T11:17:39.051284+0000","last_active":"2023-10-16T11:17:40.990047+0000","last_peered":"2023-10-16T11:17:40.990047+0000","last_clean":"2023-10-16T11:17:40.990047+0000","last_became_active":"2023-10-16T11:17:39.049365+0000","last_became_peered":"2023-10-16T11:17:39.049365+0000","last_unstale":"2023-10-16T11:17:40.990047+0000","last_undegraded":"2023-10-16T11:17:40.990047+0000","last_fullsized":"2023-10-16T11:17:40.990047+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:00.671500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.002252+0000","last_change":"2023-10-16T11:17:39.995054+0000","last_active":"2023-10-16T11:17:40.002252+0000","last_peered":"2023-10-16T11:17:40.002252+0000","last_clean":"2023-10-16T11:17:40.002252+0000","last_became_active":"2023-10-16T11:17:39.994786+0000","last_became_peered":"2023-10-16T11:17:39.994786+0000","last_unstale":"2023-10-16T11:17:40.002252+0000","last_undegraded":"2023-10-16T11:17:40.002252+0000","last_fullsized":"2023-10-16T11:17:40.002252+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:45:39.207599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"198'1","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.042912+0000","last_change":"2023-10-16T11:17:39.055907+0000","last_active":"2023-10-16T11:17:40.042912+0000","last_peered":"2023-10-16T11:17:40.042912+0000","last_clean":"2023-10-16T11:17:40.042912+0000","last_became_active":"2023-10-16T11:17:39.055614+0000","last_became_peered":"2023-10-16T11:17:39.055614+0000","last_unstale":"2023-10-16T11:17:40.042912+0000","last_undegraded":"2023-10-16T11:17:40.042912+0000","last_fullsized":"2023-10-16T11:17:40.042912+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:22.133549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990947+0000","last_change":"2023-10-16T11:17:40.003698+0000","last_active":"2023-10-16T11:17:40.990947+0000","last_peered":"2023-10-16T11:17:40.990947+0000","last_clean":"2023-10-16T11:17:40.990947+0000","last_became_active":"2023-10-16T11:17:40.003410+0000","last_became_peered":"2023-10-16T11:17:40.003410+0000","last_unstale":"2023-10-16T11:17:40.990947+0000","last_undegraded":"2023-10-16T11:17:40.990947+0000","last_fullsized":"2023-10-16T11:17:40.990947+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:43.507470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.988634+0000","last_change":"2023-10-16T11:17:39.053231+0000","last_active":"2023-10-16T11:17:39.988634+0000","last_peered":"2023-10-16T11:17:39.988634+0000","last_clean":"2023-10-16T11:17:39.988634+0000","last_became_active":"2023-10-16T11:17:39.052948+0000","last_became_peered":"2023-10-16T11:17:39.052948+0000","last_unstale":"2023-10-16T11:17:39.988634+0000","last_undegraded":"2023-10-16T11:17:39.988634+0000","last_fullsized":"2023-10-16T11:17:39.988634+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:54:24.384273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007463+0000","last_change":"2023-10-16T11:17:40.003379+0000","last_active":"2023-10-16T11:17:40.007463+0000","last_peered":"2023-10-16T11:17:40.007463+0000","last_clean":"2023-10-16T11:17:40.007463+0000","last_became_active":"2023-10-16T11:17:40.003099+0000","last_became_peered":"2023-10-16T11:17:40.003099+0000","last_unstale":"2023-10-16T11:17:40.007463+0000","last_undegraded":"2023-10-16T11:17:40.007463+0000","last_fullsized":"2023-10-16T11:17:40.007463+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:44.483361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991446+0000","last_change":"2023-10-16T11:17:39.043680+0000","last_active":"2023-10-16T11:17:40.991446+0000","last_peered":"2023-10-16T11:17:40.991446+0000","last_clean":"2023-10-16T11:17:40.991446+0000","last_became_active":"2023-10-16T11:17:39.043381+0000","last_became_peered":"2023-10-16T11:17:39.043381+0000","last_unstale":"2023-10-16T11:17:40.991446+0000","last_undegraded":"2023-10-16T11:17:40.991446+0000","last_fullsized":"2023-10-16T11:17:40.991446+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:54.398904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001318+0000","last_change":"2023-10-16T11:17:39.999766+0000","last_active":"2023-10-16T11:17:40.001318+0000","last_peered":"2023-10-16T11:17:40.001318+0000","last_clean":"2023-10-16T11:17:40.001318+0000","last_became_active":"2023-10-16T11:17:39.999509+0000","last_became_peered":"2023-10-16T11:17:39.999509+0000","last_unstale":"2023-10-16T11:17:40.001318+0000","last_undegraded":"2023-10-16T11:17:40.001318+0000","last_fullsized":"2023-10-16T11:17:40.001318+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:47.749603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989662+0000","last_change":"2023-10-16T11:17:39.047754+0000","last_active":"2023-10-16T11:17:40.989662+0000","last_peered":"2023-10-16T11:17:40.989662+0000","last_clean":"2023-10-16T11:17:40.989662+0000","last_became_active":"2023-10-16T11:17:39.047468+0000","last_became_peered":"2023-10-16T11:17:39.047468+0000","last_unstale":"2023-10-16T11:17:40.989662+0000","last_undegraded":"2023-10-16T11:17:40.989662+0000","last_fullsized":"2023-10-16T11:17:40.989662+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:32:30.092326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991422+0000","last_change":"2023-10-16T11:17:40.006735+0000","last_active":"2023-10-16T11:17:40.991422+0000","last_peered":"2023-10-16T11:17:40.991422+0000","last_clean":"2023-10-16T11:17:40.991422+0000","last_became_active":"2023-10-16T11:17:40.006482+0000","last_became_peered":"2023-10-16T11:17:40.006482+0000","last_unstale":"2023-10-16T11:17:40.991422+0000","last_undegraded":"2023-10-16T11:17:40.991422+0000","last_fullsized":"2023-10-16T11:17:40.991422+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:39:20.230419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"198'3","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989560+0000","last_change":"2023-10-16T11:17:39.054415+0000","last_active":"2023-10-16T11:17:40.989560+0000","last_peered":"2023-10-16T11:17:40.989560+0000","last_clean":"2023-10-16T11:17:40.989560+0000","last_became_active":"2023-10-16T11:17:39.054149+0000","last_became_peered":"2023-10-16T11:17:39.054149+0000","last_unstale":"2023-10-16T11:17:40.989560+0000","last_undegraded":"2023-10-16T11:17:40.989560+0000","last_fullsized":"2023-10-16T11:17:40.989560+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:02.161144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006543+0000","last_change":"2023-10-16T11:17:40.004003+0000","last_active":"2023-10-16T11:17:40.006543+0000","last_peered":"2023-10-16T11:17:40.006543+0000","last_clean":"2023-10-16T11:17:40.006543+0000","last_became_active":"2023-10-16T11:17:40.003740+0000","last_became_peered":"2023-10-16T11:17:40.003740+0000","last_unstale":"2023-10-16T11:17:40.006543+0000","last_undegraded":"2023-10-16T11:17:40.006543+0000","last_fullsized":"2023-10-16T11:17:40.006543+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:16:11.386464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"198'6","reported_seq":17,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.043948+0000","last_change":"2023-10-16T11:17:39.050581+0000","last_active":"2023-10-16T11:17:40.043948+0000","last_peered":"2023-10-16T11:17:40.043948+0000","last_clean":"2023-10-16T11:17:40.043948+0000","last_became_active":"2023-10-16T11:17:39.049782+0000","last_became_peered":"2023-10-16T11:17:39.049782+0000","last_unstale":"2023-10-16T11:17:40.043948+0000","last_undegraded":"2023-10-16T11:17:40.043948+0000","last_fullsized":"2023-10-16T11:17:40.043948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:23:04.098588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.005503+0000","last_change":"2023-10-16T11:17:39.999592+0000","last_active":"2023-10-16T11:17:40.005503+0000","last_peered":"2023-10-16T11:17:40.005503+0000","last_clean":"2023-10-16T11:17:40.005503+0000","last_became_active":"2023-10-16T11:17:39.999313+0000","last_became_peered":"2023-10-16T11:17:39.999313+0000","last_unstale":"2023-10-16T11:17:40.005503+0000","last_undegraded":"2023-10-16T11:17:40.005503+0000","last_fullsized":"2023-10-16T11:17:40.005503+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:30.009623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.043044+0000","last_change":"2023-10-16T11:17:39.048970+0000","last_active":"2023-10-16T11:17:40.043044+0000","last_peered":"2023-10-16T11:17:40.043044+0000","last_clean":"2023-10-16T11:17:40.043044+0000","last_became_active":"2023-10-16T11:17:39.048647+0000","last_became_peered":"2023-10-16T11:17:39.048647+0000","last_unstale":"2023-10-16T11:17:40.043044+0000","last_undegraded":"2023-10-16T11:17:40.043044+0000","last_fullsized":"2023-10-16T11:17:40.043044+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:21.692024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991068+0000","last_change":"2023-10-16T11:17:40.002404+0000","last_active":"2023-10-16T11:17:40.991068+0000","last_peered":"2023-10-16T11:17:40.991068+0000","last_clean":"2023-10-16T11:17:40.991068+0000","last_became_active":"2023-10-16T11:17:40.001513+0000","last_became_peered":"2023-10-16T11:17:40.001513+0000","last_unstale":"2023-10-16T11:17:40.991068+0000","last_undegraded":"2023-10-16T11:17:40.991068+0000","last_fullsized":"2023-10-16T11:17:40.991068+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:43:07.370151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.997965+0000","last_change":"2023-10-16T11:17:39.050438+0000","last_active":"2023-10-16T11:17:39.997965+0000","last_peered":"2023-10-16T11:17:39.997965+0000","last_clean":"2023-10-16T11:17:39.997965+0000","last_became_active":"2023-10-16T11:17:39.050131+0000","last_became_peered":"2023-10-16T11:17:39.050131+0000","last_unstale":"2023-10-16T11:17:39.997965+0000","last_undegraded":"2023-10-16T11:17:39.997965+0000","last_fullsized":"2023-10-16T11:17:39.997965+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:24.008300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989810+0000","last_change":"2023-10-16T11:17:39.996329+0000","last_active":"2023-10-16T11:17:40.989810+0000","last_peered":"2023-10-16T11:17:40.989810+0000","last_clean":"2023-10-16T11:17:40.989810+0000","last_became_active":"2023-10-16T11:17:39.996054+0000","last_became_peered":"2023-10-16T11:17:39.996054+0000","last_unstale":"2023-10-16T11:17:40.989810+0000","last_undegraded":"2023-10-16T11:17:40.989810+0000","last_fullsized":"2023-10-16T11:17:40.989810+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:53:56.958648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.987705+0000","last_change":"2023-10-16T11:17:39.051060+0000","last_active":"2023-10-16T11:17:39.987705+0000","last_peered":"2023-10-16T11:17:39.987705+0000","last_clean":"2023-10-16T11:17:39.987705+0000","last_became_active":"2023-10-16T11:17:39.050786+0000","last_became_peered":"2023-10-16T11:17:39.050786+0000","last_unstale":"2023-10-16T11:17:39.987705+0000","last_undegraded":"2023-10-16T11:17:39.987705+0000","last_fullsized":"2023-10-16T11:17:39.987705+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:40:26.653770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006478+0000","last_change":"2023-10-16T11:17:40.002834+0000","last_active":"2023-10-16T11:17:40.006478+0000","last_peered":"2023-10-16T11:17:40.006478+0000","last_clean":"2023-10-16T11:17:40.006478+0000","last_became_active":"2023-10-16T11:17:40.002392+0000","last_became_peered":"2023-10-16T11:17:40.002392+0000","last_unstale":"2023-10-16T11:17:40.006478+0000","last_undegraded":"2023-10-16T11:17:40.006478+0000","last_fullsized":"2023-10-16T11:17:40.006478+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:46:22.376382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"198'8","reported_seq":19,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.044618+0000","last_change":"2023-10-16T11:17:39.053140+0000","last_active":"2023-10-16T11:17:40.044618+0000","last_peered":"2023-10-16T11:17:40.044618+0000","last_clean":"2023-10-16T11:17:40.044618+0000","last_became_active":"2023-10-16T11:17:39.052866+0000","last_became_peered":"2023-10-16T11:17:39.052866+0000","last_unstale":"2023-10-16T11:17:40.044618+0000","last_undegraded":"2023-10-16T11:17:40.044618+0000","last_fullsized":"2023-10-16T11:17:40.044618+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:51.213414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.005505+0000","last_change":"2023-10-16T11:17:40.002324+0000","last_active":"2023-10-16T11:17:40.005505+0000","last_peered":"2023-10-16T11:17:40.005505+0000","last_clean":"2023-10-16T11:17:40.005505+0000","last_became_active":"2023-10-16T11:17:40.001890+0000","last_became_peered":"2023-10-16T11:17:40.001890+0000","last_unstale":"2023-10-16T11:17:40.005505+0000","last_undegraded":"2023-10-16T11:17:40.005505+0000","last_fullsized":"2023-10-16T11:17:40.005505+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:35:46.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"198'1","reported_seq":13,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989801+0000","last_change":"2023-10-16T11:17:39.050248+0000","last_active":"2023-10-16T11:17:40.989801+0000","last_peered":"2023-10-16T11:17:40.989801+0000","last_clean":"2023-10-16T11:17:40.989801+0000","last_became_active":"2023-10-16T11:17:39.049907+0000","last_became_peered":"2023-10-16T11:17:39.049907+0000","last_unstale":"2023-10-16T11:17:40.989801+0000","last_undegraded":"2023-10-16T11:17:40.989801+0000","last_fullsized":"2023-10-16T11:17:40.989801+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:40:39.434497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990399+0000","last_change":"2023-10-16T11:17:40.000430+0000","last_active":"2023-10-16T11:17:40.990399+0000","last_peered":"2023-10-16T11:17:40.990399+0000","last_clean":"2023-10-16T11:17:40.990399+0000","last_became_active":"2023-10-16T11:17:40.000126+0000","last_became_peered":"2023-10-16T11:17:40.000126+0000","last_unstale":"2023-10-16T11:17:40.990399+0000","last_undegraded":"2023-10-16T11:17:40.990399+0000","last_fullsized":"2023-10-16T11:17:40.990399+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:21.481341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041455+0000","last_change":"2023-10-16T11:17:39.052796+0000","last_active":"2023-10-16T11:17:40.041455+0000","last_peered":"2023-10-16T11:17:40.041455+0000","last_clean":"2023-10-16T11:17:40.041455+0000","last_became_active":"2023-10-16T11:17:39.052487+0000","last_became_peered":"2023-10-16T11:17:39.052487+0000","last_unstale":"2023-10-16T11:17:40.041455+0000","last_undegraded":"2023-10-16T11:17:40.041455+0000","last_fullsized":"2023-10-16T11:17:40.041455+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:30.001594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989950+0000","last_change":"2023-10-16T11:17:40.002312+0000","last_active":"2023-10-16T11:17:40.989950+0000","last_peered":"2023-10-16T11:17:40.989950+0000","last_clean":"2023-10-16T11:17:40.989950+0000","last_became_active":"2023-10-16T11:17:40.002012+0000","last_became_peered":"2023-10-16T11:17:40.002012+0000","last_unstale":"2023-10-16T11:17:40.989950+0000","last_undegraded":"2023-10-16T11:17:40.989950+0000","last_fullsized":"2023-10-16T11:17:40.989950+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:37.782255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"198'1","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.042936+0000","last_change":"2023-10-16T11:17:39.051842+0000","last_active":"2023-10-16T11:17:40.042936+0000","last_peered":"2023-10-16T11:17:40.042936+0000","last_clean":"2023-10-16T11:17:40.042936+0000","last_became_active":"2023-10-16T11:17:39.051539+0000","last_became_peered":"2023-10-16T11:17:39.051539+0000","last_unstale":"2023-10-16T11:17:40.042936+0000","last_undegraded":"2023-10-16T11:17:40.042936+0000","last_fullsized":"2023-10-16T11:17:40.042936+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:26.397820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007702+0000","last_change":"2023-10-16T11:17:40.004053+0000","last_active":"2023-10-16T11:17:40.007702+0000","last_peered":"2023-10-16T11:17:40.007702+0000","last_clean":"2023-10-16T11:17:40.007702+0000","last_became_active":"2023-10-16T11:17:40.003752+0000","last_became_peered":"2023-10-16T11:17:40.003752+0000","last_unstale":"2023-10-16T11:17:40.007702+0000","last_undegraded":"2023-10-16T11:17:40.007702+0000","last_fullsized":"2023-10-16T11:17:40.007702+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:06.985430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991433+0000","last_change":"2023-10-16T11:17:39.056947+0000","last_active":"2023-10-16T11:17:40.991433+0000","last_peered":"2023-10-16T11:17:40.991433+0000","last_clean":"2023-10-16T11:17:40.991433+0000","last_became_active":"2023-10-16T11:17:39.056722+0000","last_became_peered":"2023-10-16T11:17:39.056722+0000","last_unstale":"2023-10-16T11:17:40.991433+0000","last_undegraded":"2023-10-16T11:17:40.991433+0000","last_fullsized":"2023-10-16T11:17:40.991433+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:02:36.679484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.007476+0000","last_change":"2023-10-16T11:17:40.006066+0000","last_active":"2023-10-16T11:17:40.007476+0000","last_peered":"2023-10-16T11:17:40.007476+0000","last_clean":"2023-10-16T11:17:40.007476+0000","last_became_active":"2023-10-16T11:17:40.005787+0000","last_became_peered":"2023-10-16T11:17:40.005787+0000","last_unstale":"2023-10-16T11:17:40.007476+0000","last_undegraded":"2023-10-16T11:17:40.007476+0000","last_fullsized":"2023-10-16T11:17:40.007476+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:48.932559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.041783+0000","last_change":"2023-10-16T11:17:39.049474+0000","last_active":"2023-10-16T11:17:40.041783+0000","last_peered":"2023-10-16T11:17:40.041783+0000","last_clean":"2023-10-16T11:17:40.041783+0000","last_became_active":"2023-10-16T11:17:39.049169+0000","last_became_peered":"2023-10-16T11:17:39.049169+0000","last_unstale":"2023-10-16T11:17:40.041783+0000","last_undegraded":"2023-10-16T11:17:40.041783+0000","last_fullsized":"2023-10-16T11:17:40.041783+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:02.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991116+0000","last_change":"2023-10-16T11:17:40.006132+0000","last_active":"2023-10-16T11:17:40.991116+0000","last_peered":"2023-10-16T11:17:40.991116+0000","last_clean":"2023-10-16T11:17:40.991116+0000","last_became_active":"2023-10-16T11:17:40.005863+0000","last_became_peered":"2023-10-16T11:17:40.005863+0000","last_unstale":"2023-10-16T11:17:40.991116+0000","last_undegraded":"2023-10-16T11:17:40.991116+0000","last_fullsized":"2023-10-16T11:17:40.991116+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:04:43.610346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989582+0000","last_change":"2023-10-16T11:17:39.049430+0000","last_active":"2023-10-16T11:17:40.989582+0000","last_peered":"2023-10-16T11:17:40.989582+0000","last_clean":"2023-10-16T11:17:40.989582+0000","last_became_active":"2023-10-16T11:17:39.049078+0000","last_became_peered":"2023-10-16T11:17:39.049078+0000","last_unstale":"2023-10-16T11:17:40.989582+0000","last_undegraded":"2023-10-16T11:17:40.989582+0000","last_fullsized":"2023-10-16T11:17:40.989582+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:29.002116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003758+0000","last_change":"2023-10-16T11:17:40.000592+0000","last_active":"2023-10-16T11:17:40.003758+0000","last_peered":"2023-10-16T11:17:40.003758+0000","last_clean":"2023-10-16T11:17:40.003758+0000","last_became_active":"2023-10-16T11:17:40.000294+0000","last_became_peered":"2023-10-16T11:17:40.000294+0000","last_unstale":"2023-10-16T11:17:40.003758+0000","last_undegraded":"2023-10-16T11:17:40.003758+0000","last_fullsized":"2023-10-16T11:17:40.003758+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:24:41.394184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.990984+0000","last_change":"2023-10-16T11:17:39.049330+0000","last_active":"2023-10-16T11:17:39.990984+0000","last_peered":"2023-10-16T11:17:39.990984+0000","last_clean":"2023-10-16T11:17:39.990984+0000","last_became_active":"2023-10-16T11:17:39.049026+0000","last_became_peered":"2023-10-16T11:17:39.049026+0000","last_unstale":"2023-10-16T11:17:39.990984+0000","last_undegraded":"2023-10-16T11:17:39.990984+0000","last_fullsized":"2023-10-16T11:17:39.990984+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:52:29.654874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989547+0000","last_change":"2023-10-16T11:17:40.005019+0000","last_active":"2023-10-16T11:17:40.989547+0000","last_peered":"2023-10-16T11:17:40.989547+0000","last_clean":"2023-10-16T11:17:40.989547+0000","last_became_active":"2023-10-16T11:17:40.004338+0000","last_became_peered":"2023-10-16T11:17:40.004338+0000","last_unstale":"2023-10-16T11:17:40.989547+0000","last_undegraded":"2023-10-16T11:17:40.989547+0000","last_fullsized":"2023-10-16T11:17:40.989547+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:24:14.535881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990031+0000","last_change":"2023-10-16T11:17:39.044038+0000","last_active":"2023-10-16T11:17:40.990031+0000","last_peered":"2023-10-16T11:17:40.990031+0000","last_clean":"2023-10-16T11:17:40.990031+0000","last_became_active":"2023-10-16T11:17:39.043756+0000","last_became_peered":"2023-10-16T11:17:39.043756+0000","last_unstale":"2023-10-16T11:17:40.990031+0000","last_undegraded":"2023-10-16T11:17:40.990031+0000","last_fullsized":"2023-10-16T11:17:40.990031+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:57:26.867119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989706+0000","last_change":"2023-10-16T11:17:40.000889+0000","last_active":"2023-10-16T11:17:40.989706+0000","last_peered":"2023-10-16T11:17:40.989706+0000","last_clean":"2023-10-16T11:17:40.989706+0000","last_became_active":"2023-10-16T11:17:40.000612+0000","last_became_peered":"2023-10-16T11:17:40.000612+0000","last_unstale":"2023-10-16T11:17:40.989706+0000","last_undegraded":"2023-10-16T11:17:40.989706+0000","last_fullsized":"2023-10-16T11:17:40.989706+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:44.523350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"198'2","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.040982+0000","last_change":"2023-10-16T11:17:39.048622+0000","last_active":"2023-10-16T11:17:40.040982+0000","last_peered":"2023-10-16T11:17:40.040982+0000","last_clean":"2023-10-16T11:17:40.040982+0000","last_became_active":"2023-10-16T11:17:39.048301+0000","last_became_peered":"2023-10-16T11:17:39.048301+0000","last_unstale":"2023-10-16T11:17:40.040982+0000","last_undegraded":"2023-10-16T11:17:40.040982+0000","last_fullsized":"2023-10-16T11:17:40.040982+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:37:12.460241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.006459+0000","last_change":"2023-10-16T11:17:40.002875+0000","last_active":"2023-10-16T11:17:40.006459+0000","last_peered":"2023-10-16T11:17:40.006459+0000","last_clean":"2023-10-16T11:17:40.006459+0000","last_became_active":"2023-10-16T11:17:40.002193+0000","last_became_peered":"2023-10-16T11:17:40.002193+0000","last_unstale":"2023-10-16T11:17:40.006459+0000","last_undegraded":"2023-10-16T11:17:40.006459+0000","last_fullsized":"2023-10-16T11:17:40.006459+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:58.913565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.997188+0000","last_change":"2023-10-16T11:17:39.041489+0000","last_active":"2023-10-16T11:17:39.997188+0000","last_peered":"2023-10-16T11:17:39.997188+0000","last_clean":"2023-10-16T11:17:39.997188+0000","last_became_active":"2023-10-16T11:17:39.041213+0000","last_became_peered":"2023-10-16T11:17:39.041213+0000","last_unstale":"2023-10-16T11:17:39.997188+0000","last_undegraded":"2023-10-16T11:17:39.997188+0000","last_fullsized":"2023-10-16T11:17:39.997188+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:55.080448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.000740+0000","last_change":"2023-10-16T11:17:39.994258+0000","last_active":"2023-10-16T11:17:40.000740+0000","last_peered":"2023-10-16T11:17:40.000740+0000","last_clean":"2023-10-16T11:17:40.000740+0000","last_became_active":"2023-10-16T11:17:39.993977+0000","last_became_peered":"2023-10-16T11:17:39.993977+0000","last_unstale":"2023-10-16T11:17:40.000740+0000","last_undegraded":"2023-10-16T11:17:40.000740+0000","last_fullsized":"2023-10-16T11:17:40.000740+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:14.087424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.997617+0000","last_change":"2023-10-16T11:17:39.053279+0000","last_active":"2023-10-16T11:17:39.997617+0000","last_peered":"2023-10-16T11:17:39.997617+0000","last_clean":"2023-10-16T11:17:39.997617+0000","last_became_active":"2023-10-16T11:17:39.053022+0000","last_became_peered":"2023-10-16T11:17:39.053022+0000","last_unstale":"2023-10-16T11:17:39.997617+0000","last_undegraded":"2023-10-16T11:17:39.997617+0000","last_fullsized":"2023-10-16T11:17:39.997617+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:20.379554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.002518+0000","last_change":"2023-10-16T11:17:39.998421+0000","last_active":"2023-10-16T11:17:40.002518+0000","last_peered":"2023-10-16T11:17:40.002518+0000","last_clean":"2023-10-16T11:17:40.002518+0000","last_became_active":"2023-10-16T11:17:39.997664+0000","last_became_peered":"2023-10-16T11:17:39.997664+0000","last_unstale":"2023-10-16T11:17:40.002518+0000","last_undegraded":"2023-10-16T11:17:40.002518+0000","last_fullsized":"2023-10-16T11:17:40.002518+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:00:59.689629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.988120+0000","last_change":"2023-10-16T11:17:39.052554+0000","last_active":"2023-10-16T11:17:39.988120+0000","last_peered":"2023-10-16T11:17:39.988120+0000","last_clean":"2023-10-16T11:17:39.988120+0000","last_became_active":"2023-10-16T11:17:39.052258+0000","last_became_peered":"2023-10-16T11:17:39.052258+0000","last_unstale":"2023-10-16T11:17:39.988120+0000","last_undegraded":"2023-10-16T11:17:39.988120+0000","last_fullsized":"2023-10-16T11:17:39.988120+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:34.083507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.000229+0000","last_change":"2023-10-16T11:17:39.999182+0000","last_active":"2023-10-16T11:17:40.000229+0000","last_peered":"2023-10-16T11:17:40.000229+0000","last_clean":"2023-10-16T11:17:40.000229+0000","last_became_active":"2023-10-16T11:17:39.998883+0000","last_became_peered":"2023-10-16T11:17:39.998883+0000","last_unstale":"2023-10-16T11:17:40.000229+0000","last_undegraded":"2023-10-16T11:17:40.000229+0000","last_fullsized":"2023-10-16T11:17:40.000229+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:06.612719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.991033+0000","last_change":"2023-10-16T11:17:39.036812+0000","last_active":"2023-10-16T11:17:40.991033+0000","last_peered":"2023-10-16T11:17:40.991033+0000","last_clean":"2023-10-16T11:17:40.991033+0000","last_became_active":"2023-10-16T11:17:39.036521+0000","last_became_peered":"2023-10-16T11:17:39.036521+0000","last_unstale":"2023-10-16T11:17:40.991033+0000","last_undegraded":"2023-10-16T11:17:40.991033+0000","last_fullsized":"2023-10-16T11:17:40.991033+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:43.515745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003755+0000","last_change":"2023-10-16T11:17:40.001553+0000","last_active":"2023-10-16T11:17:40.003755+0000","last_peered":"2023-10-16T11:17:40.003755+0000","last_clean":"2023-10-16T11:17:40.003755+0000","last_became_active":"2023-10-16T11:17:40.001286+0000","last_became_peered":"2023-10-16T11:17:40.001286+0000","last_unstale":"2023-10-16T11:17:40.003755+0000","last_undegraded":"2023-10-16T11:17:40.003755+0000","last_fullsized":"2023-10-16T11:17:40.003755+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:00:02.762892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.990089+0000","last_change":"2023-10-16T11:17:39.040895+0000","last_active":"2023-10-16T11:17:40.990089+0000","last_peered":"2023-10-16T11:17:40.990089+0000","last_clean":"2023-10-16T11:17:40.990089+0000","last_became_active":"2023-10-16T11:17:39.040579+0000","last_became_peered":"2023-10-16T11:17:39.040579+0000","last_unstale":"2023-10-16T11:17:40.990089+0000","last_undegraded":"2023-10-16T11:17:40.990089+0000","last_fullsized":"2023-10-16T11:17:40.990089+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:23.589608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.003717+0000","last_change":"2023-10-16T11:17:40.000211+0000","last_active":"2023-10-16T11:17:40.003717+0000","last_peered":"2023-10-16T11:17:40.003717+0000","last_clean":"2023-10-16T11:17:40.003717+0000","last_became_active":"2023-10-16T11:17:39.999762+0000","last_became_peered":"2023-10-16T11:17:39.999762+0000","last_unstale":"2023-10-16T11:17:40.003717+0000","last_undegraded":"2023-10-16T11:17:40.003717+0000","last_fullsized":"2023-10-16T11:17:40.003717+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:58.913565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.998133+0000","last_change":"2023-10-16T11:17:39.043356+0000","last_active":"2023-10-16T11:17:39.998133+0000","last_peered":"2023-10-16T11:17:39.998133+0000","last_clean":"2023-10-16T11:17:39.998133+0000","last_became_active":"2023-10-16T11:17:39.043013+0000","last_became_peered":"2023-10-16T11:17:39.043013+0000","last_unstale":"2023-10-16T11:17:39.998133+0000","last_undegraded":"2023-10-16T11:17:39.998133+0000","last_fullsized":"2023-10-16T11:17:39.998133+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:29.002116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":11,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.989576+0000","last_change":"2023-10-16T11:17:40.005239+0000","last_active":"2023-10-16T11:17:40.989576+0000","last_peered":"2023-10-16T11:17:40.989576+0000","last_clean":"2023-10-16T11:17:40.989576+0000","last_became_active":"2023-10-16T11:17:40.004760+0000","last_became_peered":"2023-10-16T11:17:40.004760+0000","last_unstale":"2023-10-16T11:17:40.989576+0000","last_undegraded":"2023-10-16T11:17:40.989576+0000","last_fullsized":"2023-10-16T11:17:40.989576+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:08.800146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.991672+0000","last_change":"2023-10-16T11:17:39.050290+0000","last_active":"2023-10-16T11:17:39.991672+0000","last_peered":"2023-10-16T11:17:39.991672+0000","last_clean":"2023-10-16T11:17:39.991672+0000","last_became_active":"2023-10-16T11:17:39.050040+0000","last_became_peered":"2023-10-16T11:17:39.050040+0000","last_unstale":"2023-10-16T11:17:39.991672+0000","last_undegraded":"2023-10-16T11:17:39.991672+0000","last_fullsized":"2023-10-16T11:17:39.991672+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:03.742360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:40.001213+0000","last_change":"2023-10-16T11:17:40.000070+0000","last_active":"2023-10-16T11:17:40.001213+0000","last_peered":"2023-10-16T11:17:40.001213+0000","last_clean":"2023-10-16T11:17:40.001213+0000","last_became_active":"2023-10-16T11:17:39.999790+0000","last_became_peered":"2023-10-16T11:17:39.999790+0000","last_unstale":"2023-10-16T11:17:40.001213+0000","last_undegraded":"2023-10-16T11:17:40.001213+0000","last_fullsized":"2023-10-16T11:17:40.001213+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:39.025029+0000","last_clean_scrub_stamp":"2023-10-16T11:17:39.025029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:22:29.427180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.988919+0000","last_change":"2023-10-16T11:17:39.045021+0000","last_active":"2023-10-16T11:17:39.988919+0000","last_peered":"2023-10-16T11:17:39.988919+0000","last_clean":"2023-10-16T11:17:39.988919+0000","last_became_active":"2023-10-16T11:17:39.044715+0000","last_became_peered":"2023-10-16T11:17:39.044715+0000","last_unstale":"2023-10-16T11:17:39.988919+0000","last_undegraded":"2023-10-16T11:17:39.988919+0000","last_fullsized":"2023-10-16T11:17:39.988919+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:17:38.017424+0000","last_clean_scrub_stamp":"2023-10-16T11:17:38.017424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:41:20.050417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"188'326","reported_seq":1113,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-16T11:17:39.035240+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:17:39.035240+0000","last_peered":"2023-10-16T11:17:39.035240+0000","last_clean":"2023-10-16T11:17:39.035240+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:17:39.035240+0000","last_undegraded":"2023-10-16T11:17:39.035240+0000","last_fullsized":"2023-10-16T11:17:39.035240+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543260,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52244,"kb_used_data":3008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319596,"statfs":{"total":96636764160,"available":96583266304,"internally_reserved":0,"allocated":3080192,"data_stored":1854824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.434,"5min":0.316,"15min":0.23799999999999999},"max":{"1min":0.75900000000000001,"5min":0.78200000000000003,"15min":0.84499999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.371,"15min":0.27800000000000002},"max":{"1min":0.748,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56899999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79300000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.46700000000000003,"5min":0.33600000000000002,"15min":0.33300000000000002},"max":{"1min":0.70099999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.40500000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73399999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.62},"min":{"1min":0.45500000000000002,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.625}]},{"osd":3,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.74199999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.63},"min":{"1min":0.436,"5min":0.307,"15min":0.307},"max":{"1min":0.76500000000000001,"5min":0.91500000000000004,"15min":1.002},"last":0.59499999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.76600000000000001,"15min":0.83899999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51300000000000001,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.33100000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.78700000000000003,"15min":0.82599999999999996},"last":0.42299999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.40200000000000002,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63400000000000001,"15min":0.63100000000000001},"min":{"1min":0.52700000000000002,"5min":0.45900000000000002,"15min":0.32400000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.39200000000000002,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.45300000000000001,"5min":0.39200000000000002,"15min":0.371},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.871},"last":0.5}]}]},{"osd":6,"up_from":14,"seq":60129543259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37768,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2478080,"data_stored":1264352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.81899999999999995,"15min":0.875},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.34699999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.315},"max":{"1min":0.76400000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.42299999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75600000000000001,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.47999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63200000000000001,"15min":0.61599999999999999},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81100000000000005,"5min":0.82899999999999996,"15min":0.83299999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.39400000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.495},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.29699999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.39100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41299999999999998,"5min":0.35599999999999998,"15min":0.28100000000000003},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.82699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66500000000000004}]},{"osd":7,"last update":"Mon Oct 16 11:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.47199999999999998,"5min":0.376,"15min":0.32100000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.45600000000000002,"5min":0.36599999999999999,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.622}]}]},{"osd":1,"up_from":14,"seq":60129543259,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55824,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316016,"statfs":{"total":96636764160,"available":96579600384,"internally_reserved":0,"allocated":2490368,"data_stored":1265964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.318,"5min":0.224,"15min":0.221},"max":{"1min":0.58499999999999996,"5min":0.629,"15min":0.68700000000000006},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.42999999999999999,"15min":0.437},"min":{"1min":0.28199999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53700000000000003,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.312}]},{"osd":2,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.34399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51700000000000002,"5min":0.61599999999999999,"15min":0.98899999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46300000000000002,"15min":0.46899999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.60899999999999999,"5min":0.69399999999999995,"15min":0.83099999999999996},"last":0.30099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.496,"15min":0.497},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.29399999999999998},"max":{"1min":0.57999999999999996,"5min":0.755,"15min":0.78300000000000003},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.47399999999999998,"15min":0.47699999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.30299999999999999},"max":{"1min":0.624,"5min":0.68300000000000005,"15min":0.70699999999999996},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.66600000000000004},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90000000000000002,"15min":0.98399999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.56200000000000006,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.097},"last":0.54900000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68600000000000005,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.79500000000000004,"5min":0.96699999999999997,"15min":1.147},"last":0.51400000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.98699999999999999,"5min":1.0509999999999999,"15min":1.357},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71399999999999997,"15min":0.70599999999999996},"min":{"1min":0.58699999999999997,"5min":0.49299999999999999,"15min":0.47899999999999998},"max":{"1min":0.84799999999999998,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.0600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.57599999999999996,"5min":0.51000000000000001,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":1.044,"15min":1.1060000000000001},"last":0.46700000000000003}]}]},{"osd":0,"up_from":14,"seq":60129543260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56396,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315444,"statfs":{"total":96636764160,"available":96579014656,"internally_reserved":0,"allocated":3076096,"data_stored":1854742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.28199999999999997,"5min":0.214,"15min":0.214},"max":{"1min":0.54500000000000004,"5min":0.63600000000000001,"15min":0.91600000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.41899999999999998,"15min":0.42699999999999999},"min":{"1min":0.30599999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.432}]},{"osd":2,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.311,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58299999999999996,"5min":0.67300000000000004,"15min":0.70299999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44800000000000001,"15min":0.45600000000000002},"min":{"1min":0.32800000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.625,"5min":0.63300000000000001,"15min":0.86799999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60599999999999998,"5min":0.66200000000000003,"15min":0.68300000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.39800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64400000000000002,"5min":0.69799999999999995,"15min":0.84599999999999997},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.50700000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.99199999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67600000000000005,"15min":0.67900000000000005},"min":{"1min":0.499,"5min":0.41199999999999998,"15min":0.379},"max":{"1min":0.91400000000000003,"5min":0.93700000000000006,"15min":1.008},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66700000000000004},"min":{"1min":0.54700000000000004,"5min":0.45300000000000001,"15min":0.433},"max":{"1min":0.91500000000000004,"5min":1.004,"15min":1.018},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.69799999999999995},"min":{"1min":0.55200000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88400000000000001,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.67600000000000005},"min":{"1min":0.53600000000000003,"5min":0.42199999999999999,"15min":0.41499999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0049999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.51500000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.84599999999999997,"5min":1.01,"15min":1.0449999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 16 11:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.95899999999999996,"15min":1.077},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.70399999999999996},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.87,"5min":0.95699999999999996,"15min":0.96599999999999997},"last":0.69999999999999996}]}]},{"osd":2,"up_from":14,"seq":60129543259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59984,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311856,"statfs":{"total":96636764160,"available":96575340544,"internally_reserved":0,"allocated":2490368,"data_stored":1265046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.52900000000000003,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.42799999999999999},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.63},"last":0.31}]},{"osd":1,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.44,"15min":0.44400000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.26700000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":0.38}]},{"osd":3,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.436,"15min":0.439},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28399999999999997},"max":{"1min":0.623,"5min":0.623,"15min":0.65500000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.502},"min":{"1min":0.41099999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.81200000000000006},"last":0.41499999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.47999999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67600000000000005},"min":{"1min":0.52800000000000002,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":0.97199999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67200000000000004},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93899999999999995,"5min":0.97599999999999998,"15min":0.98399999999999999},"last":0.745},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.46500000000000002},"max":{"1min":1.0129999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.55800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96199999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.70199999999999996},"min":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.499},"max":{"1min":0.97299999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72199999999999998}]}]},{"osd":3,"up_from":14,"seq":60129543260,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64144,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307696,"statfs":{"total":96636764160,"available":96571080704,"internally_reserved":0,"allocated":2490368,"data_stored":1264546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":63175112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.41399999999999998,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.52600000000000002,"5min":0.627,"15min":0.70399999999999996},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.45700000000000002,"15min":0.45600000000000002},"min":{"1min":0.38200000000000001,"5min":0.32800000000000001,"15min":0.24399999999999999},"max":{"1min":0.52500000000000002,"5min":0.62,"15min":0.70999999999999996},"last":0.28199999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.28599999999999998,"15min":0.26200000000000001},"max":{"1min":0.56699999999999995,"5min":0.64100000000000001,"15min":0.78100000000000003},"last":0.253},{"interface":"front","average":{"1min":0.443,"5min":0.45600000000000002,"15min":0.45700000000000002},"min":{"1min":0.314,"5min":0.29499999999999998,"15min":0.26200000000000001},"max":{"1min":0.58699999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.32200000000000001},"max":{"1min":0.57399999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.58699999999999997,"5min":0.70399999999999996,"15min":0.749},"last":0.316}]},{"osd":4,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.52100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86399999999999999,"5min":1.034,"15min":1.0740000000000001},"last":0.375},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.36499999999999999},"max":{"1min":0.85699999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.42999999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.65300000000000002},"min":{"1min":0.49099999999999999,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":0.88400000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.501,"5min":0.46700000000000003,"15min":0.46600000000000003},"max":{"1min":0.92400000000000004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.52700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89300000000000002,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.44},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65500000000000003,"15min":0.66600000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.34599999999999997},"max":{"1min":0.76100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.46899999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.66900000000000004},"min":{"1min":0.51000000000000001,"5min":0.47999999999999998,"15min":0.40000000000000002},"max":{"1min":0.877,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.54100000000000004,"5min":0.51300000000000001,"15min":0.441},"max":{"1min":0.91500000000000004,"5min":1.016,"15min":1.0489999999999999},"last":0.38900000000000001}]}]},{"osd":4,"up_from":14,"seq":60129543259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55824,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316016,"statfs":{"total":96636764160,"available":96579600384,"internally_reserved":0,"allocated":2494464,"data_stored":1266546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.45000000000000001,"5min":0.38600000000000001,"15min":0.372},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.38600000000000001,"15min":0.372},"max":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.68600000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.52400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.621,"15min":0.621},"min":{"1min":0.434,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.79800000000000004,"15min":0.82099999999999995},"last":0.66200000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.38600000000000001},"max":{"1min":0.77700000000000002,"5min":0.83399999999999996,"15min":0.85099999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.47999999999999998,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.75700000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.503,"5min":0.433,"15min":0.34399999999999997},"max":{"1min":0.80200000000000005,"5min":0.81799999999999995,"15min":0.85999999999999999},"last":0.629},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65400000000000003},"min":{"1min":0.51800000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80500000000000005,"5min":0.879,"15min":0.879},"last":0.67100000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.80100000000000005,"15min":0.84599999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.39200000000000002,"5min":0.35799999999999998,"15min":0.33900000000000002},"max":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.89500000000000002},"last":0.55300000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.443,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.91300000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.44400000000000001,"5min":0.35899999999999999,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":0.89000000000000001,"15min":0.89700000000000002},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.36199999999999999},"max":{"1min":0.80600000000000005,"5min":0.82299999999999995,"15min":0.89000000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.629},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.95299999999999996},"last":0.70299999999999996}]}]},{"osd":5,"up_from":14,"seq":60129543260,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41920,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329920,"statfs":{"total":96636764160,"available":96593838080,"internally_reserved":0,"allocated":2478080,"data_stored":1264352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.50600000000000001,"5min":0.40100000000000002,"15min":0.29599999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.46400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.747,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.84099999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.47499999999999998,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":0.84199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34699999999999998},"max":{"1min":0.76100000000000001,"5min":0.80000000000000004,"15min":0.84899999999999998},"last":0.505}]},{"osd":3,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.86799999999999999,"15min":1.1379999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.51900000000000002,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.81399999999999995,"5min":0.871,"15min":0.92100000000000004},"last":0.40699999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.76900000000000002,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.316},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.85499999999999998},"last":0.55100000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.69999999999999996,"5min":0.873,"15min":0.873},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76900000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.41599999999999998,"5min":0.371,"15min":0.35299999999999998},"max":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.88500000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63400000000000001,"15min":0.627},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:17:44.537 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:17:44.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:17:44.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.697+0000 7f1f4ed94700 1 Processor -- start 2023-10-16T11:17:44.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.697+0000 7f1f4ed94700 1 -- start start 2023-10-16T11:17:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f4ed94700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f4815da70 0x7f1f480ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f4ed94700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f480cef00 con 0x7f1f480ce6a0 2023-10-16T11:17:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f4ed94700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f480cec20 con 0x7f1f4806afb0 2023-10-16T11:17:44.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f4ed94700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f48071c80 con 0x7f1f4815da70 2023-10-16T11:17:44.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f47fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1f480ce6a0 0x7f1f4806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36830/0 (socket says 172.21.15.181:36830) 2023-10-16T11:17:44.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f47fff700 1 -- 172.21.15.181:0/3844264565 learned_addr learned my addr 172.21.15.181:0/3844264565 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:44.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f4cb30700 1 --2- 172.21.15.181:0/3844264565 >> v2:172.21.15.190:3300/0 conn(0x7f1f4815da70 0x7f1f480ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f477fe700 1 -- 172.21.15.181:0/3844264565 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036871412 0 0) 0x7f1f480cef00 con 0x7f1f480ce6a0 2023-10-16T11:17:44.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f477fe700 1 -- 172.21.15.181:0/3844264565 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f380035b0 con 0x7f1f480ce6a0 2023-10-16T11:17:44.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.698+0000 7f1f477fe700 1 -- 172.21.15.181:0/3844264565 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607143990 0 0) 0x7f1f480cec20 con 0x7f1f4806afb0 2023-10-16T11:17:44.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f477fe700 1 -- 172.21.15.181:0/3844264565 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f480cef00 con 0x7f1f4806afb0 2023-10-16T11:17:44.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4cb30700 1 -- 172.21.15.181:0/3844264565 >> v1:172.21.15.190:6789/0 conn(0x7f1f4806afb0 legacy=0x7f1f481615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4cb30700 1 -- 172.21.15.181:0/3844264565 >> v1:172.21.15.181:6789/0 conn(0x7f1f480ce6a0 legacy=0x7f1f4806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4cb30700 1 -- 172.21.15.181:0/3844264565 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48071f60 con 0x7f1f4815da70 2023-10-16T11:17:44.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4cb30700 1 --2- 172.21.15.181:0/3844264565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 0x7f1f480ce160 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009840 tx=0x7f1f3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7fd60093765171b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:44.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f477fe700 1 -- 172.21.15.181:0/3844264565 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011830 con 0x7f1f4815da70 2023-10-16T11:17:44.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f477fe700 1 -- 172.21.15.181:0/3844264565 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c011990 con 0x7f1f4815da70 2023-10-16T11:17:44.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f477fe700 1 -- 172.21.15.181:0/3844264565 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011d00 con 0x7f1f4815da70 2023-10-16T11:17:44.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3844264565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 msgr2=0x7f1f480ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3844264565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 0x7f1f480ce160 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009840 tx=0x7f1f3c009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:44.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3844264565 shutdown_connections 2023-10-16T11:17:44.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3844264565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 0x7f1f480ce160 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.699+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3844264565 >> 172.21.15.181:0/3844264565 conn(0x7f1f480cd010 msgr2=0x7f1f480cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:44.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.700+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3844264565 shutdown_connections 2023-10-16T11:17:44.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.700+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3844264565 wait complete. 2023-10-16T11:17:44.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.701+0000 7f1f4ed94700 1 Processor -- start 2023-10-16T11:17:44.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.701+0000 7f1f4ed94700 1 -- start start 2023-10-16T11:17:44.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.701+0000 7f1f4ed94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f4806afb0 0x7f1f480b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.701+0000 7f1f4ed94700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f480ce6a0 0x7f1f480c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4ed94700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 0x7f1f481519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4ed94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480b4730 con 0x7f1f4806afb0 2023-10-16T11:17:44.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4ed94700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480b4a10 con 0x7f1f4815da70 2023-10-16T11:17:44.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4ed94700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480b4cf0 con 0x7f1f480ce6a0 2023-10-16T11:17:44.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f4806afb0 0x7f1f480b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f4806afb0 0x7f1f480b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53990/0 (socket says 172.21.15.181:53990) 2023-10-16T11:17:44.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 -- 172.21.15.181:0/3389250451 learned_addr learned my addr 172.21.15.181:0/3389250451 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:44.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4d331700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 0x7f1f481519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f47fff700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f480ce6a0 0x7f1f480c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 -- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f480ce6a0 msgr2=0x7f1f480c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f480ce6a0 0x7f1f480c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 -- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 msgr2=0x7f1f481519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 0x7f1f481519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 -- 172.21.15.181:0/3389250451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c0094a0 con 0x7f1f4806afb0 2023-10-16T11:17:44.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4cb30700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f4806afb0 0x7f1f480b4050 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009fd0 tx=0x7f1f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:44.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f457fa700 1 -- 172.21.15.181:0/3389250451 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0044d0 con 0x7f1f4806afb0 2023-10-16T11:17:44.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f480b4fd0 con 0x7f1f4806afb0 2023-10-16T11:17:44.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f457fa700 1 -- 172.21.15.181:0/3389250451 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c01f070 con 0x7f1f4806afb0 2023-10-16T11:17:44.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f457fa700 1 -- 172.21.15.181:0/3389250451 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0038e0 con 0x7f1f4806afb0 2023-10-16T11:17:44.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.702+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f480ca260 con 0x7f1f4806afb0 2023-10-16T11:17:44.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.703+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4806fe90 con 0x7f1f4806afb0 2023-10-16T11:17:44.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.704+0000 7f1f457fa700 1 -- 172.21.15.181:0/3389250451 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1f3c003a40 con 0x7f1f4806afb0 2023-10-16T11:17:44.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.704+0000 7f1f457fa700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1f2c06b020 0x7f1f2c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:44.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.704+0000 7f1f457fa700 1 -- 172.21.15.181:0/3389250451 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f1f3c021070 con 0x7f1f4806afb0 2023-10-16T11:17:44.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.704+0000 7f1f47fff700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1f2c06b020 0x7f1f2c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:44.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.705+0000 7f1f47fff700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1f2c06b020 0x7f1f2c06d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f480cbe20 tx=0x7f1f34007de0 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:44.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.708+0000 7f1f457fa700 1 -- 172.21.15.181:0/3389250451 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1f3c055e70 con 0x7f1f4806afb0 2023-10-16T11:17:44.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.978+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1f480b0240 con 0x7f1f4806afb0 2023-10-16T11:17:44.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.980+0000 7f1f457fa700 1 -- 172.21.15.181:0/3389250451 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v50) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1f3c0150b0 con 0x7f1f4806afb0 2023-10-16T11:17:44.983 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T11:17:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T11:17:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T11:17:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T11:17:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:17:44.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1f2c06b020 msgr2=0x7f1f2c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1f2c06b020 0x7f1f2c06d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f480cbe20 tx=0x7f1f34007de0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f4806afb0 msgr2=0x7f1f480b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:44.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f4806afb0 0x7f1f480b4050 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009fd0 tx=0x7f1f3c004750 comp rx=0 tx=0).stop 2023-10-16T11:17:44.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 shutdown_connections 2023-10-16T11:17:44.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f1f2c06b020 0x7f1f2c06d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4815da70 0x7f1f481519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f4806afb0 0x7f1f480b4050 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 --2- 172.21.15.181:0/3389250451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f480ce6a0 0x7f1f480c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:44.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 >> 172.21.15.181:0/3389250451 conn(0x7f1f480cd010 msgr2=0x7f1f4815b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:44.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 shutdown_connections 2023-10-16T11:17:44.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:44.983+0000 7f1f4ed94700 1 -- 172.21.15.181:0/3389250451 wait complete. 2023-10-16T11:17:44.988 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T11:17:44.998 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:17:45.018 INFO:tasks.ceph.mds.a.smithi181.stderr:2023-10-16T11:17:45.017+0000 7f2f1173d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:17:45.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.119+0000 7f745feb8700 1 Processor -- start 2023-10-16T11:17:45.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.119+0000 7f745feb8700 1 -- start start 2023-10-16T11:17:45.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745feb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 0x7f74580af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745feb8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74580cd470 0x7f74580cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745feb8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745806f870 con 0x7f745815dfd0 2023-10-16T11:17:45.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745feb8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580d1160 con 0x7f74580cd470 2023-10-16T11:17:45.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745feb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580cc8e0 con 0x7f74580af450 2023-10-16T11:17:45.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745dc54700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f745815dfd0 0x7f745806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36844/0 (socket says 172.21.15.181:36844) 2023-10-16T11:17:45.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745dc54700 1 -- 172.21.15.181:0/445545832 learned_addr learned my addr 172.21.15.181:0/445545832 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:45.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335548705 0 0) 0x7f745806f870 con 0x7f745815dfd0 2023-10-16T11:17:45.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74440035b0 con 0x7f745815dfd0 2023-10-16T11:17:45.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745d453700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 0x7f74580af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745e455700 1 --2- 172.21.15.181:0/445545832 >> v2:172.21.15.190:3300/0 conn(0x7f74580cd470 0x7f74580cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1941012887 0 0) 0x7f74440035b0 con 0x7f745815dfd0 2023-10-16T11:17:45.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 msgr2=0x7f74580af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 0x7f74580af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 >> v2:172.21.15.190:3300/0 conn(0x7f74580cd470 msgr2=0x7f74580cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 --2- 172.21.15.181:0/445545832 >> v2:172.21.15.190:3300/0 conn(0x7f74580cd470 0x7f74580cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580ccbc0 con 0x7f745815dfd0 2023-10-16T11:17:45.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f74540017f0 con 0x7f745815dfd0 2023-10-16T11:17:45.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745e455700 1 --2- 172.21.15.181:0/445545832 >> v2:172.21.15.190:3300/0 conn(0x7f74580cd470 0x7f74580cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:45.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 >> v1:172.21.15.181:6789/0 conn(0x7f745815dfd0 legacy=0x7f745806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 0x7f7444003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7444003f80 0x7f7444004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7444004d90 0x7f7444005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580d1160 con 0x7f7444004d90 2023-10-16T11:17:45.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580cc8e0 con 0x7f7444003f80 2023-10-16T11:17:45.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.120+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744400bb10 con 0x7f74580af450 2023-10-16T11:17:45.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745e455700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7444004d90 0x7f7444005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745d453700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7444003f80 0x7f7444004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745dc54700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 0x7f7444003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745e455700 1 -- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 msgr2=0x7f7444003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745e455700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580af450 0x7f7444003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745e455700 1 -- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7444003f80 msgr2=0x7f7444004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745e455700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7444003f80 0x7f7444004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745e455700 1 -- 172.21.15.181:0/445545832 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440035b0 con 0x7f7444004d90 2023-10-16T11:17:45.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745e455700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7444004d90 0x7f7444005210 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f7450007410 tx=0x7f7444005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63886944134de7ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450014400 con 0x7f7444004d90 2023-10-16T11:17:45.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450014560 con 0x7f7444004d90 2023-10-16T11:17:45.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.121+0000 7f745cc52700 1 -- 172.21.15.181:0/445545832 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74500148d0 con 0x7f7444004d90 2023-10-16T11:17:45.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.122+0000 7f745feb8700 1 -- 172.21.15.181:0/445545832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7444004d90 msgr2=0x7f7444005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.122+0000 7f745feb8700 1 --2- 172.21.15.181:0/445545832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7444004d90 0x7f7444005210 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f7450007410 tx=0x7f7444005880 comp rx=0 tx=0).stop 2023-10-16T11:17:45.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.122+0000 7f745e455700 1 -- 172.21.15.181:0/445545832 reap_dead start 2023-10-16T11:17:45.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.122+0000 7f745feb8700 1 -- 172.21.15.181:0/445545832 shutdown_connections 2023-10-16T11:17:45.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.122+0000 7f745feb8700 1 -- 172.21.15.181:0/445545832 >> 172.21.15.181:0/445545832 conn(0x7f74580d04a0 msgr2=0x7f74580d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.122+0000 7f745feb8700 1 -- 172.21.15.181:0/445545832 shutdown_connections 2023-10-16T11:17:45.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.122+0000 7f745feb8700 1 -- 172.21.15.181:0/445545832 wait complete. 2023-10-16T11:17:45.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.123+0000 7f745feb8700 1 Processor -- start 2023-10-16T11:17:45.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.123+0000 7f745feb8700 1 -- start start 2023-10-16T11:17:45.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580b19d0 0x7f74580affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9e90 0x7f74580b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74580b22f0 0x7f74580b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c83f0 con 0x7f74580b22f0 2023-10-16T11:17:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c8690 con 0x7f74580c9e90 2023-10-16T11:17:45.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c8970 con 0x7f74580b19d0 2023-10-16T11:17:45.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74580b22f0 0x7f74580b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74580b22f0 0x7f74580b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54014/0 (socket says 172.21.15.181:54014) 2023-10-16T11:17:45.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 -- 172.21.15.181:0/3070479644 learned_addr learned my addr 172.21.15.181:0/3070479644 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:45.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745d453700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9e90 0x7f74580b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745dc54700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580b19d0 0x7f74580affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 -- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580b19d0 msgr2=0x7f74580affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580b19d0 0x7f74580affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 -- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9e90 msgr2=0x7f74580b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9e90 0x7f74580b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 -- 172.21.15.181:0/3070479644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580c8c50 con 0x7f74580b22f0 2023-10-16T11:17:45.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745d453700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9e90 0x7f74580b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:45.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745dc54700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580b19d0 0x7f74580affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:45.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745e455700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74580b22f0 0x7f74580b4340 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f7450002230 tx=0x7f7450004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f744e7fc700 1 -- 172.21.15.181:0/3070479644 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450026070 con 0x7f74580b22f0 2023-10-16T11:17:45.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f744e7fc700 1 -- 172.21.15.181:0/3070479644 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745001e410 con 0x7f74580b22f0 2023-10-16T11:17:45.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580c8f30 con 0x7f74580b22f0 2023-10-16T11:17:45.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f744e7fc700 1 -- 172.21.15.181:0/3070479644 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450014400 con 0x7f74580b22f0 2023-10-16T11:17:45.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.124+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580ca310 con 0x7f74580b22f0 2023-10-16T11:17:45.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.125+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74580c9120 con 0x7f74580b22f0 2023-10-16T11:17:45.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.126+0000 7f744e7fc700 1 -- 172.21.15.181:0/3070479644 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f745001b050 con 0x7f74580b22f0 2023-10-16T11:17:45.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.126+0000 7f744e7fc700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f743c06b020 0x7f743c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.126+0000 7f744e7fc700 1 -- 172.21.15.181:0/3070479644 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f745008a000 con 0x7f74580b22f0 2023-10-16T11:17:45.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.126+0000 7f745dc54700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f743c06b020 0x7f743c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.127+0000 7f745dc54700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f743c06b020 0x7f743c06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7454000ef0 tx=0x7f7454009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.131+0000 7f744e7fc700 1 -- 172.21.15.181:0/3070479644 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7450054e20 con 0x7f74580b22f0 2023-10-16T11:17:45.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.364+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7458067d20 con 0x7f74580b22f0 2023-10-16T11:17:45.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.365+0000 7f744e7fc700 1 -- 172.21.15.181:0/3070479644 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f7450019090 con 0x7f74580b22f0 2023-10-16T11:17:45.367 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:45.367 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3757723265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3757723265},{"type":"v1","addr":"172.21.15.190:6834","nonce":3757723265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2012626771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2012626771},{"type":"v1","addr":"172.21.15.190:6835","nonce":2012626771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10782,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3218502962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3218502962},{"type":"v1","addr":"172.21.15.181:6836","nonce":3218502962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:17:39.436499+0000","modified":"2023-10-16T11:17:42.454721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10785":{"gid":10785,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3625484950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3625484950},{"type":"v1","addr":"172.21.15.181:6837","nonce":3625484950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:17:45.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f743c06b020 msgr2=0x7f743c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f743c06b020 0x7f743c06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7454000ef0 tx=0x7f7454009040 comp rx=0 tx=0).stop 2023-10-16T11:17:45.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74580b22f0 msgr2=0x7f74580b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74580b22f0 0x7f74580b4340 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f7450002230 tx=0x7f7450004b40 comp rx=0 tx=0).stop 2023-10-16T11:17:45.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 shutdown_connections 2023-10-16T11:17:45.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f743c06b020 0x7f743c06d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580b19d0 0x7f74580affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74580b22f0 0x7f74580b4340 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 --2- 172.21.15.181:0/3070479644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9e90 0x7f74580b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 >> 172.21.15.181:0/3070479644 conn(0x7f74580d04a0 msgr2=0x7f74580cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 shutdown_connections 2023-10-16T11:17:45.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.367+0000 7f745feb8700 1 -- 172.21.15.181:0/3070479644 wait complete. 2023-10-16T11:17:45.371 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 102 2023-10-16T11:17:45.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:17:39.436499+0000', 'modified': '2023-10-16T11:17:42.454721+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10785}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10785': {'gid': 10785, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/3625484950', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 3625484950}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3625484950}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:17:45.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:17:45.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T11:17:45.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.905+0000 7f11a3ba9700 1 Processor -- start 2023-10-16T11:17:45.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.906+0000 7f11a3ba9700 1 -- start start 2023-10-16T11:17:45.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.906+0000 7f11a3ba9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f119c15df60 0x7f119c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.906+0000 7f11a3ba9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a3ba9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c0ccf50 con 0x7f119c06a8a0 2023-10-16T11:17:45.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a3ba9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c0af9c0 con 0x7f119c15df60 2023-10-16T11:17:45.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a3ba9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0cc600 con 0x7f119c0d0fc0 2023-10-16T11:17:45.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1945700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f119c15df60 0x7f119c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1144700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a2146700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f119c06a8a0 0x7f119c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:49040/0 (socket says 172.21.15.181:49040) 2023-10-16T11:17:45.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a2146700 1 -- 172.21.15.181:0/3642412450 learned_addr learned my addr 172.21.15.181:0/3642412450 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:45.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1144700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c0cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:51044/0 (socket says 172.21.15.181:51044) 2023-10-16T11:17:45.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1144700 1 -- 172.21.15.181:0/3642412450 >> v2:172.21.15.190:3300/0 conn(0x7f119c15df60 msgr2=0x7f119c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1144700 1 --2- 172.21.15.181:0/3642412450 >> v2:172.21.15.190:3300/0 conn(0x7f119c15df60 0x7f119c0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1144700 1 -- 172.21.15.181:0/3642412450 >> v1:172.21.15.190:6789/0 conn(0x7f119c06a8a0 legacy=0x7f119c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1144700 1 -- 172.21.15.181:0/3642412450 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c0cc8a0 con 0x7f119c0d0fc0 2023-10-16T11:17:45.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1945700 1 --2- 172.21.15.181:0/3642412450 >> v2:172.21.15.190:3300/0 conn(0x7f119c15df60 0x7f119c0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:45.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.907+0000 7f11a1144700 1 --2- 172.21.15.181:0/3642412450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c0cd6e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1198009840 tx=0x7f1198009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c806ac253c2bf63 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a0943700 1 -- 172.21.15.181:0/3642412450 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011830 con 0x7f119c0d0fc0 2023-10-16T11:17:45.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a0943700 1 -- 172.21.15.181:0/3642412450 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198011990 con 0x7f119c0d0fc0 2023-10-16T11:17:45.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a0943700 1 -- 172.21.15.181:0/3642412450 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011d20 con 0x7f119c0d0fc0 2023-10-16T11:17:45.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3642412450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 msgr2=0x7f119c0cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3642412450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c0cd6e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1198009840 tx=0x7f1198009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:45.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3642412450 shutdown_connections 2023-10-16T11:17:45.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3642412450 >> v2:172.21.15.190:3300/0 conn(0x7f119c15df60 0x7f119c0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3642412450 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c0cd6e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3642412450 >> 172.21.15.181:0/3642412450 conn(0x7f119c0d04a0 msgr2=0x7f119c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3642412450 shutdown_connections 2023-10-16T11:17:45.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.908+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3642412450 wait complete. 2023-10-16T11:17:45.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.909+0000 7f11a3ba9700 1 Processor -- start 2023-10-16T11:17:45.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.909+0000 7f11a3ba9700 1 -- start start 2023-10-16T11:17:45.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a3ba9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f119c06a8a0 0x7f119c0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a3ba9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a3ba9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15df60 0x7f119c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a3ba9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0b4be0 con 0x7f119c15df60 2023-10-16T11:17:45.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a3ba9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0b4ec0 con 0x7f119c06a8a0 2023-10-16T11:17:45.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a3ba9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0b51a0 con 0x7f119c0d0fc0 2023-10-16T11:17:45.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a2146700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15df60 0x7f119c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a2146700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15df60 0x7f119c0ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54026/0 (socket says 172.21.15.181:54026) 2023-10-16T11:17:45.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a2146700 1 -- 172.21.15.181:0/3085196300 learned_addr learned my addr 172.21.15.181:0/3085196300 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:45.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a1945700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f119c06a8a0 0x7f119c0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.910+0000 7f11a1144700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a2146700 1 -- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 msgr2=0x7f119c154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a2146700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a2146700 1 -- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f119c06a8a0 msgr2=0x7f119c0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a2146700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f119c06a8a0 0x7f119c0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a2146700 1 -- 172.21.15.181:0/3085196300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c009430 con 0x7f119c15df60 2023-10-16T11:17:45.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a2146700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15df60 0x7f119c0ce3a0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f119400ca80 tx=0x7f119400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11927fc700 1 -- 172.21.15.181:0/3085196300 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400a9e0 con 0x7f119c15df60 2023-10-16T11:17:45.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11927fc700 1 -- 172.21.15.181:0/3085196300 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119400ab40 con 0x7f119c15df60 2023-10-16T11:17:45.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11927fc700 1 -- 172.21.15.181:0/3085196300 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194012400 con 0x7f119c15df60 2023-10-16T11:17:45.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11980094a0 con 0x7f119c15df60 2023-10-16T11:17:45.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.911+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c0b5760 con 0x7f119c15df60 2023-10-16T11:17:45.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.912+0000 7f11927fc700 1 -- 172.21.15.181:0/3085196300 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f119401b3b0 con 0x7f119c15df60 2023-10-16T11:17:45.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.913+0000 7f11927fc700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f118006b020 0x7f118006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.913+0000 7f11927fc700 1 -- 172.21.15.181:0/3085196300 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f1194089340 con 0x7f119c15df60 2023-10-16T11:17:45.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.913+0000 7f11a1945700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f118006b020 0x7f118006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.913+0000 7f11a1945700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f118006b020 0x7f118006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f118c009d50 tx=0x7f118c0091c0 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.915+0000 7f11afd93b80 1 Processor -- start 2023-10-16T11:17:45.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.915+0000 7f11afd93b80 1 -- start start 2023-10-16T11:17:45.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145b91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b91fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b65010 0x55e145b656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11afd93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e145adbc20 con 0x55e145b8a220 2023-10-16T11:17:45.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e145ac0c10 con 0x55e145b65010 2023-10-16T11:17:45.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e145abe530 con 0x55e145b8f630 2023-10-16T11:17:45.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145b91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145b91a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54030/0 (socket says 172.21.15.181:54030) 2023-10-16T11:17:45.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 -- 172.21.15.181:0/340615474 learned_addr learned my addr 172.21.15.181:0/340615474 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:45.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 -- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 msgr2=0x55e145b91fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a2146700 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b65010 0x55e145b656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1144700 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b91fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b91fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 -- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b65010 msgr2=0x55e145b656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b65010 0x55e145b656c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 -- 172.21.15.181:0/340615474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e145a773a0 con 0x55e145b8a220 2023-10-16T11:17:45.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a2146700 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b65010 0x55e145b656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:45.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f11a1945700 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145b91a70 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f118c011e60 tx=0x7f118c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a26ae25da4c2576 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f117f7fe700 1 -- 172.21.15.181:0/340615474 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c014e70 con 0x55e145b8a220 2023-10-16T11:17:45.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f117f7fe700 1 -- 172.21.15.181:0/340615474 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c011e90 con 0x55e145b8a220 2023-10-16T11:17:45.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.916+0000 7f117f7fe700 1 -- 172.21.15.181:0/340615474 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0215f0 con 0x55e145b8a220 2023-10-16T11:17:45.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 -- 172.21.15.181:0/340615474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 msgr2=0x55e145b91a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145b91a70 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f118c011e60 tx=0x7f118c014000 comp rx=0 tx=0).stop 2023-10-16T11:17:45.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 -- 172.21.15.181:0/340615474 shutdown_connections 2023-10-16T11:17:45.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b65010 0x55e145b656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145b91a70 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 --2- 172.21.15.181:0/340615474 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b91fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 -- 172.21.15.181:0/340615474 >> 172.21.15.181:0/340615474 conn(0x55e145a94d80 msgr2=0x55e145bafff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 -- 172.21.15.181:0/340615474 shutdown_connections 2023-10-16T11:17:45.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.917+0000 7f11afd93b80 1 -- 172.21.15.181:0/340615474 wait complete. 2023-10-16T11:17:45.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.918+0000 7f11afd93b80 1 -- start start 2023-10-16T11:17:45.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.918+0000 7f11afd93b80 1 Processor -- start 2023-10-16T11:17:45.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145b95390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b8ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e145ab9e20 con 0x55e145b65010 2023-10-16T11:17:45.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e145a566c0 con 0x55e145b8a220 2023-10-16T11:17:45.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e145a56360 con 0x55e145b8f630 2023-10-16T11:17:45.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 10 client.0 _ll_register_callbacks cb 0x55e1458b37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:17:45.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145b95390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145b95390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54036/0 (socket says 172.21.15.181:54036) 2023-10-16T11:17:45.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 -- 172.21.15.181:0/377819825 learned_addr learned my addr 172.21.15.181:0/377819825 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:45.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1144700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b8ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a2146700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 -- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 msgr2=0x55e145b88e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b88e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 -- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 msgr2=0x55e145b8ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b8ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 -- 172.21.15.181:0/377819825 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c011b40 con 0x55e145b65010 2023-10-16T11:17:45.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a2146700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:45.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1144700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b8ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:45.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11a1945700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145b95390 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f118c014ab0 tx=0x7f118c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f25b3adebd8adf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f117cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:17:45.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f117cff9700 1 -- 172.21.15.181:0/377819825 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c003c30 con 0x55e145b65010 2023-10-16T11:17:45.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f117cff9700 1 -- 172.21.15.181:0/377819825 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f118c01c040 con 0x55e145b65010 2023-10-16T11:17:45.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.919+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e145b5a5b0 con 0x55e145b65010 2023-10-16T11:17:45.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f117cff9700 1 -- 172.21.15.181:0/377819825 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f118c003d90 con 0x55e145b65010 2023-10-16T11:17:45.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 10 client.10932 fetch_fsmap learned FSMap version 102 2023-10-16T11:17:45.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55e1458e4d90 con 0x55e145b65010 2023-10-16T11:17:45.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f117cff9700 1 -- 172.21.15.181:0/377819825 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e145b5a5b0 con 0x55e145b65010 2023-10-16T11:17:45.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f117cff9700 1 -- 172.21.15.181:0/377819825 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f118c005a20 con 0x55e145b65010 2023-10-16T11:17:45.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 10 client.10932 fetch_fsmap finished waiting for FSMap version 102 2023-10-16T11:17:45.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 10 client.10932 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:17:45.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 20 client.10932 populate_metadata read hostname 'smithi181' 2023-10-16T11:17:45.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b65450 0x55e145b65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 4 client.10932 mds_command: new command op to 10785 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:17:45.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 --> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e145ab2e00 con 0x55e145b65450 2023-10-16T11:17:45.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11a1144700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b65450 0x55e145b65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f117cff9700 10 client.10932 ms_handle_connect on v2:172.21.15.181:6835/3625484950 2023-10-16T11:17:45.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.920+0000 7f11a1144700 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b65450 0x55e145b65830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11aa96e7e688aad6 server_cookie=5abee24168f64f1d in_seq=0 out_seq=0 2023-10-16T11:17:45.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f117cff9700 1 -- 172.21.15.181:0/377819825 <== mds.0 v2:172.21.15.181:6835/3625484950 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e145ab2e00 con 0x55e145b65450 2023-10-16T11:17:45.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f117cff9700 10 client.10932 handle_command_reply: tid=0 2023-10-16T11:17:45.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 client.10932 shutdown 2023-10-16T11:17:45.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 msgr2=0x55e145b95390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145b95390 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f118c014ab0 tx=0x7f118c0047c0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 shutdown_connections 2023-10-16T11:17:45.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b65450 0x55e145b65830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f117cff9700 0 client.10932 ms_handle_reset on v2:172.21.15.181:6835/3625484950 2023-10-16T11:17:45.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145b95390 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 --2- 172.21.15.181:0/377819825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b8ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 >> 172.21.15.181:0/377819825 conn(0x55e145a94d80 msgr2=0x55e145bad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 shutdown_connections 2023-10-16T11:17:45.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 1 -- 172.21.15.181:0/377819825 wait complete. 2023-10-16T11:17:45.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.921+0000 7f11afd93b80 20 client.10932 trim_cache size 0 max 16384 2023-10-16T11:17:45.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 Processor -- start 2023-10-16T11:17:45.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 -- start start 2023-10-16T11:17:45.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145bad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b951a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b8f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14593a8d0 con 0x55e145b65010 2023-10-16T11:17:45.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e145b8bba0 con 0x55e145b8a220 2023-10-16T11:17:45.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e145b662e0 con 0x55e145b8f630 2023-10-16T11:17:45.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11a1945700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145bad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11a1945700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145bad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54044/0 (socket says 172.21.15.181:54044) 2023-10-16T11:17:45.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.939+0000 7f11a1945700 1 -- 172.21.15.181:0/144233891 learned_addr learned my addr 172.21.15.181:0/144233891 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:45.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11a1945700 1 -- 172.21.15.181:0/144233891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 msgr2=0x55e145b8f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11a1945700 1 --2- 172.21.15.181:0/144233891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b8f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11a1945700 1 -- 172.21.15.181:0/144233891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 msgr2=0x55e145b951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11a1945700 1 --2- 172.21.15.181:0/144233891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11a1945700 1 -- 172.21.15.181:0/144233891 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e145b63b20 con 0x55e145b65010 2023-10-16T11:17:45.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11a1945700 1 --2- 172.21.15.181:0/144233891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145bad890 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f118c00b010 tx=0x7f118c02d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e42c5bccaa4add server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f117ffff700 1 -- 172.21.15.181:0/144233891 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0036a0 con 0x55e145b65010 2023-10-16T11:17:45.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f117ffff700 1 -- 172.21.15.181:0/144233891 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c003a30 con 0x55e145b65010 2023-10-16T11:17:45.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f117ffff700 1 -- 172.21.15.181:0/144233891 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c003da0 con 0x55e145b65010 2023-10-16T11:17:45.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11afd93b80 1 -- 172.21.15.181:0/144233891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 msgr2=0x55e145bad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11afd93b80 1 --2- 172.21.15.181:0/144233891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145bad890 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f118c00b010 tx=0x7f118c02d040 comp rx=0 tx=0).stop 2023-10-16T11:17:45.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11afd93b80 1 -- 172.21.15.181:0/144233891 shutdown_connections 2023-10-16T11:17:45.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.940+0000 7f11afd93b80 1 --2- 172.21.15.181:0/144233891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b8f630 0x55e145b8f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.941+0000 7f11afd93b80 1 --2- 172.21.15.181:0/144233891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b65010 0x55e145bad890 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.941+0000 7f11afd93b80 1 --2- 172.21.15.181:0/144233891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8a220 0x55e145b951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.941+0000 7f11afd93b80 1 -- 172.21.15.181:0/144233891 >> 172.21.15.181:0/144233891 conn(0x55e145a94d80 msgr2=0x55e145bce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.941+0000 7f11afd93b80 1 -- 172.21.15.181:0/144233891 shutdown_connections 2023-10-16T11:17:45.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.941+0000 7f11afd93b80 1 -- 172.21.15.181:0/144233891 wait complete. 2023-10-16T11:17:45.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.941+0000 7f11afd93b80 1 -- start start 2023-10-16T11:17:45.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.941+0000 7f11afd93b80 1 Processor -- start 2023-10-16T11:17:45.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b65010 0x55e145b89ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145c13450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11afd93b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8f630 0x55e145ba9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11afd93b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e145bbe4c0 con 0x55e145b8a220 2023-10-16T11:17:45.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e145baaa50 con 0x55e145b8f630 2023-10-16T11:17:45.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11afd93b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e145babbb0 con 0x55e145b65010 2023-10-16T11:17:45.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11afd93b80 10 client.0 _ll_register_callbacks cb 0x7f118c011d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:17:45.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145c13450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145c13450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54052/0 (socket says 172.21.15.181:54052) 2023-10-16T11:17:45.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 -- 172.21.15.181:0/2462876785 learned_addr learned my addr 172.21.15.181:0/2462876785 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:45.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a2146700 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8f630 0x55e145ba9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1945700 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b65010 0x55e145b89ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 -- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b65010 msgr2=0x55e145b89ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b65010 0x55e145b89ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 -- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8f630 msgr2=0x55e145ba9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:45.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8f630 0x55e145ba9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 -- 172.21.15.181:0/2462876785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1458e4d90 con 0x55e145b8a220 2023-10-16T11:17:45.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f11a1144700 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145c13450 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f11980090b0 tx=0x7f1198004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb88a11efff533b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:45.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.942+0000 7f117cff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:17:45.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f117cff9700 1 -- 172.21.15.181:0/2462876785 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198020040 con 0x55e145b8a220 2023-10-16T11:17:45.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f117cff9700 1 -- 172.21.15.181:0/2462876785 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f1198029bf0 con 0x55e145b8a220 2023-10-16T11:17:45.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f118c011e90 con 0x55e145b8a220 2023-10-16T11:17:45.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f117cff9700 1 -- 172.21.15.181:0/2462876785 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1198029db0 con 0x55e145b8a220 2023-10-16T11:17:45.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 10 client.10938 fetch_fsmap learned FSMap version 102 2023-10-16T11:17:45.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55e1458cdc10 con 0x55e145b8a220 2023-10-16T11:17:45.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f117cff9700 1 -- 172.21.15.181:0/2462876785 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c011e90 con 0x55e145b8a220 2023-10-16T11:17:45.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f117cff9700 1 -- 172.21.15.181:0/2462876785 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 102) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f11980290d0 con 0x55e145b8a220 2023-10-16T11:17:45.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 10 client.10938 fetch_fsmap finished waiting for FSMap version 102 2023-10-16T11:17:45.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 10 client.10938 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:17:45.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 20 client.10938 populate_metadata read hostname 'smithi181' 2023-10-16T11:17:45.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b87820 0x55e145b87c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:45.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 4 client.10938 mds_command: new command op to 10785 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:17:45.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 --> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e145ab2b20 con 0x55e145b87820 2023-10-16T11:17:45.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11a1945700 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b87820 0x55e145b87c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:45.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f117cff9700 10 client.10938 ms_handle_connect on v2:172.21.15.181:6835/3625484950 2023-10-16T11:17:45.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.943+0000 7f11a1945700 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b87820 0x55e145b87c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bff0f80dcaa3f21 server_cookie=7ea506588df52ae1 in_seq=0 out_seq=0 2023-10-16T11:17:45.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f117cff9700 1 -- 172.21.15.181:0/2462876785 <== mds.0 v2:172.21.15.181:6835/3625484950 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55e145ab2b20 con 0x55e145b87820 2023-10-16T11:17:45.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f117cff9700 10 client.10938 handle_command_reply: tid=0 2023-10-16T11:17:45.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 client.10938 shutdown 2023-10-16T11:17:45.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 msgr2=0x55e145c13450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145c13450 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f11980090b0 tx=0x7f1198004750 comp rx=0 tx=0).stop 2023-10-16T11:17:45.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 shutdown_connections 2023-10-16T11:17:45.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:6835/3625484950,v1:172.21.15.181:6837/3625484950] conn(0x55e145b87820 0x55e145b87c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e145b65010 0x55e145b89ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f117cff9700 0 client.10938 ms_handle_reset on v2:172.21.15.181:6835/3625484950 2023-10-16T11:17:45.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e145b8f630 0x55e145ba9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 --2- 172.21.15.181:0/2462876785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e145b8a220 0x55e145c13450 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 >> 172.21.15.181:0/2462876785 conn(0x55e145a94d80 msgr2=0x55e145bce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 shutdown_connections 2023-10-16T11:17:45.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 1 -- 172.21.15.181:0/2462876785 wait complete. 2023-10-16T11:17:45.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11afd93b80 20 client.10938 trim_cache size 0 max 16384 2023-10-16T11:17:45.970 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:17:45.971 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:17:45.971 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:17:45.971 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 10785, 2023-10-16T11:17:45.971 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:17:45.971 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:17:45.972 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T11:17:45.972 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9132994669999999, 2023-10-16T11:17:45.972 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 102, 2023-10-16T11:17:45.972 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 199, 2023-10-16T11:17:45.972 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 199, 2023-10-16T11:17:45.973 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 11.83488301 2023-10-16T11:17:45.973 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:17:45.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f118006b020 msgr2=0x7f118006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f118006b020 0x7f118006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f118c009d50 tx=0x7f118c0091c0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15df60 msgr2=0x7f119c0ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15df60 0x7f119c0ce3a0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f119400ca80 tx=0x7f119400cd90 comp rx=0 tx=0).stop 2023-10-16T11:17:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 shutdown_connections 2023-10-16T11:17:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f118006b020 0x7f118006d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15df60 0x7f119c0ce3a0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f119c06a8a0 0x7f119c0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.944+0000 7f11a3ba9700 1 --2- 172.21.15.181:0/3085196300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f119c0d0fc0 0x7f119c154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:45.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.945+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 >> 172.21.15.181:0/3085196300 conn(0x7f119c0d04a0 msgr2=0x7f119c0ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:45.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.945+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 shutdown_connections 2023-10-16T11:17:45.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:45.945+0000 7f11a3ba9700 1 -- 172.21.15.181:0/3085196300 wait complete. 2023-10-16T11:17:45.977 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:17:46.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.097+0000 7f376fbe7700 1 Processor -- start 2023-10-16T11:17:46.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.097+0000 7f376fbe7700 1 -- start start 2023-10-16T11:17:46.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376fbe7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f376815b990 0x7f37680ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:46.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376fbe7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37680d0370 0x7f37680d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:46.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376fbe7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37680cc510 con 0x7f37680cd2d0 2023-10-16T11:17:46.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376fbe7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37680d24f0 con 0x7f376815b990 2023-10-16T11:17:46.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376fbe7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37680cc7f0 con 0x7f37680d0370 2023-10-16T11:17:46.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d983700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f376815b990 0x7f37680ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:46.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d182700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f37680cd2d0 0x7f37680cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36906/0 (socket says 172.21.15.181:36906) 2023-10-16T11:17:46.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d983700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f376815b990 0x7f37680ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54058/0 (socket says 172.21.15.181:54058) 2023-10-16T11:17:46.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d182700 1 -- 172.21.15.181:0/1888638419 learned_addr learned my addr 172.21.15.181:0/1888638419 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:46.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376e184700 1 --2- 172.21.15.181:0/1888638419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37680d0370 0x7f37680d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:46.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d983700 1 -- 172.21.15.181:0/1888638419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37680d0370 msgr2=0x7f37680d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:46.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d983700 1 --2- 172.21.15.181:0/1888638419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37680d0370 0x7f37680d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d983700 1 -- 172.21.15.181:0/1888638419 >> v1:172.21.15.181:6789/0 conn(0x7f37680cd2d0 legacy=0x7f37680cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:46.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376d983700 1 -- 172.21.15.181:0/1888638419 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680ccad0 con 0x7f376815b990 2023-10-16T11:17:46.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376c981700 1 -- 172.21.15.181:0/1888638419 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006903380 0 0) 0x7f37680cc510 con 0x7f37680cd2d0 2023-10-16T11:17:46.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.098+0000 7f376e184700 1 --2- 172.21.15.181:0/1888638419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37680d0370 0x7f37680d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:46.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376d983700 1 --2- 172.21.15.181:0/1888638419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376815b990 0x7f37680ce8f0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f3758009820 tx=0x7f3758009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f1c0afb97d820d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:46.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376c981700 1 -- 172.21.15.181:0/1888638419 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758005e30 con 0x7f376815b990 2023-10-16T11:17:46.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376c981700 1 -- 172.21.15.181:0/1888638419 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3758006030 con 0x7f376815b990 2023-10-16T11:17:46.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376c981700 1 -- 172.21.15.181:0/1888638419 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580063e0 con 0x7f376815b990 2023-10-16T11:17:46.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 -- 172.21.15.181:0/1888638419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376815b990 msgr2=0x7f37680ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:46.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1888638419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376815b990 0x7f37680ce8f0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f3758009820 tx=0x7f3758009b30 comp rx=0 tx=0).stop 2023-10-16T11:17:46.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 -- 172.21.15.181:0/1888638419 shutdown_connections 2023-10-16T11:17:46.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1888638419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37680d0370 0x7f37680d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1888638419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376815b990 0x7f37680ce8f0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 -- 172.21.15.181:0/1888638419 >> 172.21.15.181:0/1888638419 conn(0x7f376809a9a0 msgr2=0x7f37680d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:46.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 -- 172.21.15.181:0/1888638419 shutdown_connections 2023-10-16T11:17:46.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.099+0000 7f376fbe7700 1 -- 172.21.15.181:0/1888638419 wait complete. 2023-10-16T11:17:46.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.100+0000 7f376fbe7700 1 Processor -- start 2023-10-16T11:17:46.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376fbe7700 1 -- start start 2023-10-16T11:17:46.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376fbe7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37680cd2d0 0x7f376814b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:46.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376fbe7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37680d0370 0x7f376815e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:46.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376fbe7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f376815b990 0x7f376806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:46.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376fbe7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680b6a80 con 0x7f37680d0370 2023-10-16T11:17:46.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376fbe7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376814b920 con 0x7f37680cd2d0 2023-10-16T11:17:46.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376fbe7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37680b5e00 con 0x7f376815b990 2023-10-16T11:17:46.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376d182700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37680d0370 0x7f376815e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:46.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376d182700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37680d0370 0x7f376815e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54066/0 (socket says 172.21.15.181:54066) 2023-10-16T11:17:46.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376d182700 1 -- 172.21.15.181:0/1689542691 learned_addr learned my addr 172.21.15.181:0/1689542691 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:46.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376d983700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37680cd2d0 0x7f376814b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:46.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.101+0000 7f376e184700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f376815b990 0x7f376806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:46.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376d182700 1 -- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f376815b990 msgr2=0x7f376806fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:46.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376d182700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f376815b990 0x7f376806fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376d182700 1 -- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37680cd2d0 msgr2=0x7f376814b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:46.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376d182700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37680cd2d0 0x7f376814b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376d182700 1 -- 172.21.15.181:0/1689542691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3758009430 con 0x7f37680d0370 2023-10-16T11:17:46.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376d182700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37680d0370 0x7f376815e870 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f37640011e0 tx=0x7f376400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:46.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f375e7fc700 1 -- 172.21.15.181:0/1689542691 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764004c80 con 0x7f37680d0370 2023-10-16T11:17:46.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f375e7fc700 1 -- 172.21.15.181:0/1689542691 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3764004e00 con 0x7f37680d0370 2023-10-16T11:17:46.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37680b60e0 con 0x7f37680d0370 2023-10-16T11:17:46.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f375e7fc700 1 -- 172.21.15.181:0/1689542691 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764007310 con 0x7f37680d0370 2023-10-16T11:17:46.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.102+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37680b66a0 con 0x7f37680d0370 2023-10-16T11:17:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.103+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37680b6bc0 con 0x7f37680d0370 2023-10-16T11:17:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.103+0000 7f375e7fc700 1 -- 172.21.15.181:0/1689542691 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3764007470 con 0x7f37680d0370 2023-10-16T11:17:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.104+0000 7f375e7fc700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f374c06b020 0x7f374c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.104+0000 7f375e7fc700 1 -- 172.21.15.181:0/1689542691 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f37640895a0 con 0x7f37680d0370 2023-10-16T11:17:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.104+0000 7f376d983700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f374c06b020 0x7f374c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:46.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.104+0000 7f376d983700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f374c06b020 0x7f374c06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3758013040 tx=0x7f3758008070 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:46.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.108+0000 7f375e7fc700 1 -- 172.21.15.181:0/1689542691 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f37680b6bc0 con 0x7f37680d0370 2023-10-16T11:17:46.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.345+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37680b6bc0 con 0x7f37680d0370 2023-10-16T11:17:46.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.345+0000 7f375e7fc700 1 -- 172.21.15.181:0/1689542691 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) v1 ==== 53+0+73 (secure 0 0 0) 0x7f37680b6bc0 con 0x7f37680d0370 2023-10-16T11:17:46.347 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:17:46.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f374c06b020 msgr2=0x7f374c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:46.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f374c06b020 0x7f374c06d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3758013040 tx=0x7f3758008070 comp rx=0 tx=0).stop 2023-10-16T11:17:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37680d0370 msgr2=0x7f376815e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37680d0370 0x7f376815e870 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f37640011e0 tx=0x7f376400ca30 comp rx=0 tx=0).stop 2023-10-16T11:17:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 shutdown_connections 2023-10-16T11:17:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f374c06b020 0x7f374c06d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f376815b990 0x7f376806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37680cd2d0 0x7f376814b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 --2- 172.21.15.181:0/1689542691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37680d0370 0x7f376815e870 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 >> 172.21.15.181:0/1689542691 conn(0x7f376809a9a0 msgr2=0x7f37680ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 shutdown_connections 2023-10-16T11:17:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:46.347+0000 7f376fbe7700 1 -- 172.21.15.181:0/1689542691 wait complete. 2023-10-16T11:17:46.361 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:17:46.361 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:17:46.362 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T11:17:46.362 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:17:46.362 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:17:46.363 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:17:46.363 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:17:46.363 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:17:46.363 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:17:46.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.364 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T11:17:46.381 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:17:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:17:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:17:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:17:46.382 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T11:17:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:17:46.383 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:17:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:17:46.383 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:17:46.383 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:17:46.384 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:17:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T11:17:46.384 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:17:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T11:17:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:17:46.384 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 36803sec preferred_lft 36803sec 2023-10-16T11:17:46.385 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T11:17:46.385 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:17:46.385 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:17:46.385 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:17:46.385 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:17:46.386 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:17:46.386 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T11:17:46.386 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:17:46.386 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:17:46.386 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:17:46.387 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T11:17:46.387 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:17:46.387 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:17:46.387 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:17:46.387 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T11:17:46.387 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:17:46.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.388 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:17:46.447 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T11:17:46.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.448 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T11:17:46.507 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T11:17:46.508 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:17:46.508 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:17:46.508 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:17:46.508 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:17:46.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.509 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T11:17:46.509 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T11:17:46.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:17:46.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:17:46.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:17:46.511 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T11:17:46.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.613 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:17:46.629 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:17:46.629 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:17:46.629 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:17:46.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.630 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T11:17:46.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:17:46.697 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:17:46.753 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:17:46.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:17:46.754 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:17:46.810 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T11:17:46.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:17:46.894 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:17:46.895 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:17:46.895 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:17:46.903 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:17:46.903 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:17:46.904 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:17:46.904 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:17:46.904 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:17:46.931 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:17:46.931 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:17:46.932 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:17:46.932 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:17:46.932 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:17:46.932 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T11:17:46.932 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T11:17:46.932 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T11:17:46.935 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:17:46.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.936 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:17:46.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:46.968 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:17:47.051 INFO:teuthology.orchestra.run.smithi181.stdout:client9667 2023-10-16T11:17:47.051 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:17:47.052 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:17:47.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:17:47.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.267+0000 7f098a0b1700 1 Processor -- start 2023-10-16T11:17:47.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.267+0000 7f098a0b1700 1 -- start start 2023-10-16T11:17:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f098a0b1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f098415da60 0x7f09840665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f098a0b1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09840cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f098a0b1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09840cf340 0x7f098406cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f098a0b1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09840c3660 con 0x7f098415da60 2023-10-16T11:17:47.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f098a0b1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0984066ae0 con 0x7f09840cf340 2023-10-16T11:17:47.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f098a0b1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098415e240 con 0x7f09840b22a0 2023-10-16T11:17:47.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f09837fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f098415da60 0x7f09840665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f09837fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f098415da60 0x7f09840665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54076/0 (socket says 172.21.15.181:54076) 2023-10-16T11:17:47.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f09837fe700 1 -- 172.21.15.181:0/1004347714 learned_addr learned my addr 172.21.15.181:0/1004347714 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:47.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.268+0000 7f0982ffd700 1 --2- 172.21.15.181:0/1004347714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09840cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09837fe700 1 -- 172.21.15.181:0/1004347714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 msgr2=0x7f09840cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09837fe700 1 --2- 172.21.15.181:0/1004347714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09840cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09837fe700 1 -- 172.21.15.181:0/1004347714 >> v2:172.21.15.190:3300/0 conn(0x7f09840cf340 msgr2=0x7f098406cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f0983fff700 1 --2- 172.21.15.181:0/1004347714 >> v2:172.21.15.190:3300/0 conn(0x7f09840cf340 0x7f098406cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09837fe700 1 --2- 172.21.15.181:0/1004347714 >> v2:172.21.15.190:3300/0 conn(0x7f09840cf340 0x7f098406cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09837fe700 1 -- 172.21.15.181:0/1004347714 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098415e520 con 0x7f098415da60 2023-10-16T11:17:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09837fe700 1 --2- 172.21.15.181:0/1004347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098415da60 0x7f09840665a0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f0984070b20 tx=0x7f0978009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9b901a53b89b94 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09827fc700 1 -- 172.21.15.181:0/1004347714 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097800b870 con 0x7f098415da60 2023-10-16T11:17:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09827fc700 1 -- 172.21.15.181:0/1004347714 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097800b9d0 con 0x7f098415da60 2023-10-16T11:17:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f09827fc700 1 -- 172.21.15.181:0/1004347714 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097800bd60 con 0x7f098415da60 2023-10-16T11:17:47.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 -- 172.21.15.181:0/1004347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098415da60 msgr2=0x7f09840665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 --2- 172.21.15.181:0/1004347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098415da60 0x7f09840665a0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f0984070b20 tx=0x7f0978009bd0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 -- 172.21.15.181:0/1004347714 shutdown_connections 2023-10-16T11:17:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 --2- 172.21.15.181:0/1004347714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098415da60 0x7f09840665a0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 --2- 172.21.15.181:0/1004347714 >> v2:172.21.15.190:3300/0 conn(0x7f09840cf340 0x7f098406cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 --2- 172.21.15.181:0/1004347714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09840cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 -- 172.21.15.181:0/1004347714 >> 172.21.15.181:0/1004347714 conn(0x7f09840b1780 msgr2=0x7f09840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.269+0000 7f098a0b1700 1 -- 172.21.15.181:0/1004347714 shutdown_connections 2023-10-16T11:17:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.270+0000 7f098a0b1700 1 -- 172.21.15.181:0/1004347714 wait complete. 2023-10-16T11:17:47.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.271+0000 7f098a0b1700 1 Processor -- start 2023-10-16T11:17:47.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.271+0000 7f098a0b1700 1 -- start start 2023-10-16T11:17:47.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09841ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09840cf340 0x7f09841e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f098415da60 0x7f09841e6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840c4f10 con 0x7f09840cf340 2023-10-16T11:17:47.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840c5a90 con 0x7f098415da60 2023-10-16T11:17:47.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09840c5160 con 0x7f09840b22a0 2023-10-16T11:17:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09840cf340 0x7f09841e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09840cf340 0x7f09841e61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54084/0 (socket says 172.21.15.181:54084) 2023-10-16T11:17:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 -- 172.21.15.181:0/2694637736 learned_addr learned my addr 172.21.15.181:0/2694637736 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f09837fe700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09841ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0983fff700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f098415da60 0x7f09841e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 -- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 msgr2=0x7f09841ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09841ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 -- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f098415da60 msgr2=0x7f09841e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f098415da60 0x7f09841e6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 -- 172.21.15.181:0/2694637736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0978009550 con 0x7f09840cf340 2023-10-16T11:17:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f0982ffd700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09840cf340 0x7f09841e61c0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f097400c7f0 tx=0x7f097400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f096bfff700 1 -- 172.21.15.181:0/2694637736 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974011800 con 0x7f09840cf340 2023-10-16T11:17:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f096bfff700 1 -- 172.21.15.181:0/2694637736 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974011960 con 0x7f09840cf340 2023-10-16T11:17:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09840c5440 con 0x7f09840cf340 2023-10-16T11:17:47.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f096bfff700 1 -- 172.21.15.181:0/2694637736 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974011cf0 con 0x7f09840cf340 2023-10-16T11:17:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.272+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09840c5f20 con 0x7f09840cf340 2023-10-16T11:17:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.273+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0984067d20 con 0x7f09840cf340 2023-10-16T11:17:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.274+0000 7f096bfff700 1 -- 172.21.15.181:0/2694637736 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0974018360 con 0x7f09840cf340 2023-10-16T11:17:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.274+0000 7f096bfff700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f097006b020 0x7f097006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.274+0000 7f096bfff700 1 -- 172.21.15.181:0/2694637736 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f09740896f0 con 0x7f09840cf340 2023-10-16T11:17:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.274+0000 7f09837fe700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f097006b020 0x7f097006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.275+0000 7f09837fe700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f097006b020 0x7f097006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0978000c00 tx=0x7f0978008030 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:47.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.279+0000 7f096bfff700 1 -- 172.21.15.181:0/2694637736 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0974054400 con 0x7f09840cf340 2023-10-16T11:17:47.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.499+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 --> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f09840c52d0 con 0x7f097006b020 2023-10-16T11:17:47.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.500+0000 7f096bfff700 1 -- 172.21.15.181:0/2694637736 <== mgr.10542 v2:172.21.15.181:6832/1532329351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f09840c52d0 con 0x7f097006b020 2023-10-16T11:17:47.502 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:17:47.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:17:47.503 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T11:17:47.503 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:17:47.503 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:17:47.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f097006b020 msgr2=0x7f097006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f097006b020 0x7f097006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0978000c00 tx=0x7f0978008030 comp rx=0 tx=0).stop 2023-10-16T11:17:47.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09840cf340 msgr2=0x7f09841e61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09840cf340 0x7f09841e61c0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f097400c7f0 tx=0x7f097400cb00 comp rx=0 tx=0).stop 2023-10-16T11:17:47.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 shutdown_connections 2023-10-16T11:17:47.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f097006b020 0x7f097006d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09840b22a0 0x7f09841ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f098415da60 0x7f09841e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 --2- 172.21.15.181:0/2694637736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09840cf340 0x7f09841e61c0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 >> 172.21.15.181:0/2694637736 conn(0x7f09840b1780 msgr2=0x7f09840a06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:47.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 shutdown_connections 2023-10-16T11:17:47.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.502+0000 7f098a0b1700 1 -- 172.21.15.181:0/2694637736 wait complete. 2023-10-16T11:17:47.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:17:47.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.637+0000 7fbfaf1c2700 1 Processor -- start 2023-10-16T11:17:47.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.637+0000 7fbfaf1c2700 1 -- start start 2023-10-16T11:17:47.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfaf1c2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfa815da70 0x7fbfa80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfaf1c2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfa806afb0 0x7fbfa81615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfaf1c2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa80cef00 con 0x7fbfa80ce6a0 2023-10-16T11:17:47.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfaf1c2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa80cec20 con 0x7fbfa815da70 2023-10-16T11:17:47.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfaf1c2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa8071c80 con 0x7fbfa806afb0 2023-10-16T11:17:47.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfacf5e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfa815da70 0x7fbfa80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfa7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbfa80ce6a0 0x7fbfa806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36936/0 (socket says 172.21.15.181:36936) 2023-10-16T11:17:47.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfa7fff700 1 -- 172.21.15.181:0/3691809061 learned_addr learned my addr 172.21.15.181:0/3691809061 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:47.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfad75f700 1 --2- 172.21.15.181:0/3691809061 >> v2:172.21.15.190:3300/0 conn(0x7fbfa806afb0 0x7fbfa81615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfacf5e700 1 -- 172.21.15.181:0/3691809061 >> v2:172.21.15.190:3300/0 conn(0x7fbfa806afb0 msgr2=0x7fbfa81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:47.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfacf5e700 1 --2- 172.21.15.181:0/3691809061 >> v2:172.21.15.190:3300/0 conn(0x7fbfa806afb0 0x7fbfa81615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfacf5e700 1 -- 172.21.15.181:0/3691809061 >> v1:172.21.15.181:6789/0 conn(0x7fbfa80ce6a0 legacy=0x7fbfa806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfacf5e700 1 -- 172.21.15.181:0/3691809061 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8071f60 con 0x7fbfa815da70 2023-10-16T11:17:47.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfa77fe700 1 -- 172.21.15.181:0/3691809061 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936722987 0 0) 0x7fbfa80cef00 con 0x7fbfa80ce6a0 2023-10-16T11:17:47.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfad75f700 1 --2- 172.21.15.181:0/3691809061 >> v2:172.21.15.190:3300/0 conn(0x7fbfa806afb0 0x7fbfa81615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:17:47.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.638+0000 7fbfacf5e700 1 --2- 172.21.15.181:0/3691809061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa80ce160 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80b6680 tx=0x7fbfa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9663a83af7e737dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:47.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfa77fe700 1 -- 172.21.15.181:0/3691809061 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000e070 con 0x7fbfa815da70 2023-10-16T11:17:47.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfa77fe700 1 -- 172.21.15.181:0/3691809061 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa000bbd0 con 0x7fbfa815da70 2023-10-16T11:17:47.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfa77fe700 1 -- 172.21.15.181:0/3691809061 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0012430 con 0x7fbfa815da70 2023-10-16T11:17:47.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3691809061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 msgr2=0x7fbfa80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3691809061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa80ce160 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80b6680 tx=0x7fbfa000b330 comp rx=0 tx=0).stop 2023-10-16T11:17:47.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3691809061 shutdown_connections 2023-10-16T11:17:47.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3691809061 >> v2:172.21.15.190:3300/0 conn(0x7fbfa806afb0 0x7fbfa81615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3691809061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa80ce160 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3691809061 >> 172.21.15.181:0/3691809061 conn(0x7fbfa80cd010 msgr2=0x7fbfa80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:47.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3691809061 shutdown_connections 2023-10-16T11:17:47.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.639+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3691809061 wait complete. 2023-10-16T11:17:47.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.641+0000 7fbfaf1c2700 1 Processor -- start 2023-10-16T11:17:47.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.641+0000 7fbfaf1c2700 1 -- start start 2023-10-16T11:17:47.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfaf1c2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa806afb0 0x7fbfa81f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfaf1c2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80ce6a0 0x7fbfa815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfaf1c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfaf1c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa815ea30 con 0x7fbfa815da70 2023-10-16T11:17:47.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfaf1c2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa81602d0 con 0x7fbfa80ce6a0 2023-10-16T11:17:47.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfaf1c2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa815e100 con 0x7fbfa806afb0 2023-10-16T11:17:47.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa8157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54100/0 (socket says 172.21.15.181:54100) 2023-10-16T11:17:47.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 -- 172.21.15.181:0/3667450099 learned_addr learned my addr 172.21.15.181:0/3667450099 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:47.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfa7fff700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80ce6a0 0x7fbfa815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfacf5e700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa806afb0 0x7fbfa81f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 -- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa806afb0 msgr2=0x7fbfa81f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa806afb0 0x7fbfa81f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 -- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80ce6a0 msgr2=0x7fbfa815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:47.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80ce6a0 0x7fbfa815eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:47.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 -- 172.21.15.181:0/3667450099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0009a70 con 0x7fbfa815da70 2023-10-16T11:17:47.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfa7fff700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80ce6a0 0x7fbfa815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:17:47.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.642+0000 7fbfad75f700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa8157cc0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c7f0 tx=0x7fbf9800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:47.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.643+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98011800 con 0x7fbfa815da70 2023-10-16T11:17:47.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.643+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98011980 con 0x7fbfa815da70 2023-10-16T11:17:47.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.643+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa815e3e0 con 0x7fbfa815da70 2023-10-16T11:17:47.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.643+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98011d30 con 0x7fbfa815da70 2023-10-16T11:17:47.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.643+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa815fa50 con 0x7fbfa815da70 2023-10-16T11:17:47.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.644+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa815eb70 con 0x7fbfa815da70 2023-10-16T11:17:47.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.644+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 127) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbf98022350 con 0x7fbfa815da70 2023-10-16T11:17:47.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.645+0000 7fbfa57fa700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbf8806b020 0x7fbf8806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:47.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.645+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7fbf98089ad0 con 0x7fbfa815da70 2023-10-16T11:17:47.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.645+0000 7fbfacf5e700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbf8806b020 0x7fbf8806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:47.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.645+0000 7fbfacf5e700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbf8806b020 0x7fbf8806d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0014040 tx=0x7fbfa0011000 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:47.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.649+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbf980546b0 con 0x7fbfa815da70 2023-10-16T11:17:47.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:47.896+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fbfa80b0190 con 0x7fbfa815da70 2023-10-16T11:17:48.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.072+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v128) v1 ==== 92+0+0 (secure 0 0 0) 0x7fbf9801f0a0 con 0x7fbfa815da70 2023-10-16T11:17:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbf8806b020 msgr2=0x7fbf8806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbf8806b020 0x7fbf8806d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0014040 tx=0x7fbfa0011000 comp rx=0 tx=0).stop 2023-10-16T11:17:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 msgr2=0x7fbfa8157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa8157cc0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c7f0 tx=0x7fbf9800cbb0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfa57fa700 1 -- 172.21.15.181:0/3667450099 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 128) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fbf98054a80 con 0x7fbfa815da70 2023-10-16T11:17:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 shutdown_connections 2023-10-16T11:17:48.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fbf8806b020 0x7fbf8806d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfa806afb0 0x7fbfa81f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa815da70 0x7fbfa8157cc0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 --2- 172.21.15.181:0/3667450099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfa80ce6a0 0x7fbfa815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 >> 172.21.15.181:0/3667450099 conn(0x7fbfa80cd010 msgr2=0x7fbfa81616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:48.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 shutdown_connections 2023-10-16T11:17:48.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.078+0000 7fbfaf1c2700 1 -- 172.21.15.181:0/3667450099 wait complete. 2023-10-16T11:17:48.093 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:17:48.220 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:48.218+0000 7ff125eed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:17:48.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.225+0000 7fa46fd38700 1 Processor -- start 2023-10-16T11:17:48.228 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:48.225+0000 7f6e54f83200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:17:48.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.226+0000 7fa46fd38700 1 -- start start 2023-10-16T11:17:48.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46fd38700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa4680b2010 0x7fa4680cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46fd38700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46fd38700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4680ce1f0 con 0x7fa468071be0 2023-10-16T11:17:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46fd38700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa468072090 con 0x7fa4680b2010 2023-10-16T11:17:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46fd38700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680ccda0 con 0x7fa4680ce680 2023-10-16T11:17:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46e2d5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46dad4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa4680b2010 0x7fa4680cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46d2d3700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa468071be0 0x7fa4680b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:49152/0 (socket says 172.21.15.181:49152) 2023-10-16T11:17:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46d2d3700 1 -- 172.21.15.181:0/3140296816 learned_addr learned my addr 172.21.15.181:0/3140296816 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46dad4700 1 -- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 msgr2=0x7fa4680ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46dad4700 1 --2- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46dad4700 1 -- 172.21.15.181:0/3140296816 >> v1:172.21.15.190:6789/0 conn(0x7fa468071be0 legacy=0x7fa4680b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46cad2700 1 -- 172.21.15.181:0/3140296816 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921024009 0 0) 0x7fa4680ce1f0 con 0x7fa468071be0 2023-10-16T11:17:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46dad4700 1 -- 172.21.15.181:0/3140296816 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4680727e0 con 0x7fa4680b2010 2023-10-16T11:17:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46e2d5700 1 --2- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.227+0000 7fa46dad4700 1 --2- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680cc6e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa464009840 tx=0x7fa464009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a606bd9a70fb95a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46cad2700 1 -- 172.21.15.181:0/3140296816 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464011830 con 0x7fa4680b2010 2023-10-16T11:17:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46cad2700 1 -- 172.21.15.181:0/3140296816 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa464011990 con 0x7fa4680b2010 2023-10-16T11:17:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46cad2700 1 -- 172.21.15.181:0/3140296816 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464011d20 con 0x7fa4680b2010 2023-10-16T11:17:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 -- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 msgr2=0x7fa4680cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 --2- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680cc6e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa464009840 tx=0x7fa464009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 -- 172.21.15.181:0/3140296816 shutdown_connections 2023-10-16T11:17:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 --2- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 --2- 172.21.15.181:0/3140296816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680cc6e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 -- 172.21.15.181:0/3140296816 >> 172.21.15.181:0/3140296816 conn(0x7fa4680cd010 msgr2=0x7fa4680cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 -- 172.21.15.181:0/3140296816 shutdown_connections 2023-10-16T11:17:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.228+0000 7fa46fd38700 1 -- 172.21.15.181:0/3140296816 wait complete. 2023-10-16T11:17:48.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 Processor -- start 2023-10-16T11:17:48.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 -- start start 2023-10-16T11:17:48.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468071be0 0x7fa46814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680ae8f0 con 0x7fa468071be0 2023-10-16T11:17:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680aebd0 con 0x7fa4680b2010 2023-10-16T11:17:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46fd38700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680aeeb0 con 0x7fa4680ce680 2023-10-16T11:17:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46dad4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468071be0 0x7fa46814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46dad4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468071be0 0x7fa46814a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54136/0 (socket says 172.21.15.181:54136) 2023-10-16T11:17:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46dad4700 1 -- 172.21.15.181:0/852281286 learned_addr learned my addr 172.21.15.181:0/852281286 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46d2d3700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.230+0000 7fa46e2d5700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46dad4700 1 -- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 msgr2=0x7fa4680cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46dad4700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46dad4700 1 -- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 msgr2=0x7fa4680ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46dad4700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46dad4700 1 -- 172.21.15.181:0/852281286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460009a70 con 0x7fa468071be0 2023-10-16T11:17:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46d2d3700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46dad4700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468071be0 0x7fa46814a2c0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fa464009b90 tx=0x7fa464004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa45e7fc700 1 -- 172.21.15.181:0/852281286 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4640036a0 con 0x7fa468071be0 2023-10-16T11:17:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa45e7fc700 1 -- 172.21.15.181:0/852281286 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46401f070 con 0x7fa468071be0 2023-10-16T11:17:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4640094a0 con 0x7fa468071be0 2023-10-16T11:17:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa45e7fc700 1 -- 172.21.15.181:0/852281286 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa464003a20 con 0x7fa468071be0 2023-10-16T11:17:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.231+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46814a800 con 0x7fa468071be0 2023-10-16T11:17:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.232+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46814aba0 con 0x7fa468071be0 2023-10-16T11:17:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.232+0000 7fa45e7fc700 1 -- 172.21.15.181:0/852281286 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 128) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa464003b80 con 0x7fa468071be0 2023-10-16T11:17:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.233+0000 7fa45e7fc700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 0x7fa44c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.233+0000 7fa46d2d3700 1 -- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 msgr2=0x7fa44c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.233+0000 7fa45e7fc700 1 -- 172.21.15.181:0/852281286 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7fa464021070 con 0x7fa468071be0 2023-10-16T11:17:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.233+0000 7fa46d2d3700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 0x7fa44c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.237+0000 7fa45e7fc700 1 -- 172.21.15.181:0/852281286 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa464055fc0 con 0x7fa468071be0 2023-10-16T11:17:48.287 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:48.285+0000 7ff125eed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:48.303 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:48.302+0000 7f6e54f83200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:48.360 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:48.357+0000 7ff125eed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:17:48.377 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:48.376+0000 7f6e54f83200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:17:48.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.434+0000 7fa46d2d3700 1 -- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 msgr2=0x7fa44c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:48.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.434+0000 7fa46d2d3700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 0x7fa44c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:48.436 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:48.435+0000 7ff125eed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:17:48.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.449+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fa4680cf110 con 0x7fa468071be0 2023-10-16T11:17:48.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.452+0000 7fa45e7fc700 1 -- 172.21.15.181:0/852281286 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa4640595e0 con 0x7fa468071be0 2023-10-16T11:17:48.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 msgr2=0x7fa44c06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:48.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 0x7fa44c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468071be0 msgr2=0x7fa46814a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468071be0 0x7fa46814a2c0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fa464009b90 tx=0x7fa464004750 comp rx=0 tx=0).stop 2023-10-16T11:17:48.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 shutdown_connections 2023-10-16T11:17:48.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7fa44c06b070 0x7fa44c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4680ce680 0x7fa4680cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468071be0 0x7fa46814a2c0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 --2- 172.21.15.181:0/852281286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4680b2010 0x7fa4680ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 >> 172.21.15.181:0/852281286 conn(0x7fa4680cd010 msgr2=0x7fa46806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:48.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 shutdown_connections 2023-10-16T11:17:48.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.454+0000 7fa46fd38700 1 -- 172.21.15.181:0/852281286 wait complete. 2023-10-16T11:17:48.459 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:48.456+0000 7f6e54f83200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:17:48.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:17:48.578 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:48.576+0000 7ff125eed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:17:48.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.594+0000 7f319b75a700 1 Processor -- start 2023-10-16T11:17:48.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.594+0000 7f319b75a700 1 -- start start 2023-10-16T11:17:48.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f319b75a700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31940d01d0 0x7f31940ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f319b75a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31940cf580 con 0x7f319415ea40 2023-10-16T11:17:48.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f319b75a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31940ce970 con 0x7f31940d1500 2023-10-16T11:17:48.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f319b75a700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940d4290 con 0x7f31940d01d0 2023-10-16T11:17:48.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f3198cf5700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f319415ea40 0x7f31940d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37018/0 (socket says 172.21.15.181:37018) 2023-10-16T11:17:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f3198cf5700 1 -- 172.21.15.181:0/1916023003 learned_addr learned my addr 172.21.15.181:0/1916023003 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f31994f6700 1 --2- 172.21.15.181:0/1916023003 >> v2:172.21.15.190:3300/0 conn(0x7f31940d01d0 0x7f31940ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715998329 0 0) 0x7f31940cf580 con 0x7f319415ea40 2023-10-16T11:17:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31800035b0 con 0x7f319415ea40 2023-10-16T11:17:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283200431 0 0) 0x7f31940ce970 con 0x7f31940d1500 2023-10-16T11:17:48.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31940cf580 con 0x7f31940d1500 2023-10-16T11:17:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f31994f6700 1 -- 172.21.15.181:0/1916023003 >> v1:172.21.15.190:6789/0 conn(0x7f31940d1500 legacy=0x7f31940d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139269965 0 0) 0x7f31800035b0 con 0x7f319415ea40 2023-10-16T11:17:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f31994f6700 1 -- 172.21.15.181:0/1916023003 >> v1:172.21.15.181:6789/0 conn(0x7f319415ea40 legacy=0x7f31940d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.595+0000 7f31994f6700 1 -- 172.21.15.181:0/1916023003 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940cec50 con 0x7f31940d01d0 2023-10-16T11:17:48.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f31994f6700 1 --2- 172.21.15.181:0/1916023003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d01d0 0x7f31940ce2d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3190009840 tx=0x7f3190009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ccc813dd94271e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184001670 con 0x7f31940d01d0 2023-10-16T11:17:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190004030 con 0x7f31940d01d0 2023-10-16T11:17:48.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f318bfff700 1 -- 172.21.15.181:0/1916023003 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31900043c0 con 0x7f31940d01d0 2023-10-16T11:17:48.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f319b75a700 1 -- 172.21.15.181:0/1916023003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d01d0 msgr2=0x7f31940ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f319b75a700 1 --2- 172.21.15.181:0/1916023003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d01d0 0x7f31940ce2d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3190009840 tx=0x7f3190009b50 comp rx=0 tx=0).stop 2023-10-16T11:17:48.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f319b75a700 1 -- 172.21.15.181:0/1916023003 shutdown_connections 2023-10-16T11:17:48.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f319b75a700 1 --2- 172.21.15.181:0/1916023003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d01d0 0x7f31940ce2d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.596+0000 7f319b75a700 1 -- 172.21.15.181:0/1916023003 >> 172.21.15.181:0/1916023003 conn(0x7f319409cb60 msgr2=0x7f31940d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:48.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.597+0000 7f319b75a700 1 -- 172.21.15.181:0/1916023003 shutdown_connections 2023-10-16T11:17:48.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.597+0000 7f319b75a700 1 -- 172.21.15.181:0/1916023003 wait complete. 2023-10-16T11:17:48.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 Processor -- start 2023-10-16T11:17:48.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 -- start start 2023-10-16T11:17:48.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31940d01d0 0x7f31940c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d1500 0x7f31941f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f319415ea40 0x7f31940c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31941f4750 con 0x7f31940d01d0 2023-10-16T11:17:48.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c7bf0 con 0x7f31940d1500 2023-10-16T11:17:48.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.598+0000 7f319b75a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c7ed0 con 0x7f319415ea40 2023-10-16T11:17:48.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f31994f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31940d01d0 0x7f31940c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3198cf5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d1500 0x7f31941f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f31994f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31940d01d0 0x7f31940c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54178/0 (socket says 172.21.15.181:54178) 2023-10-16T11:17:48.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f31994f6700 1 -- 172.21.15.181:0/852782423 learned_addr learned my addr 172.21.15.181:0/852782423 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:48.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3199cf7700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f319415ea40 0x7f31940c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:48.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3198cf5700 1 -- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f319415ea40 msgr2=0x7f31940c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3198cf5700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f319415ea40 0x7f31940c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3198cf5700 1 -- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31940d01d0 msgr2=0x7f31940c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3198cf5700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31940d01d0 0x7f31940c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3198cf5700 1 -- 172.21.15.181:0/852782423 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31900094a0 con 0x7f31940d1500 2023-10-16T11:17:48.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3198cf5700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d1500 0x7f31941f4210 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f318400c470 tx=0x7f318400c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:48.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3189ffb700 1 -- 172.21.15.181:0/852782423 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184004c80 con 0x7f31940d1500 2023-10-16T11:17:48.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3189ffb700 1 -- 172.21.15.181:0/852782423 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3184004de0 con 0x7f31940d1500 2023-10-16T11:17:48.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31941f3a10 con 0x7f31940d1500 2023-10-16T11:17:48.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f3189ffb700 1 -- 172.21.15.181:0/852782423 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184007310 con 0x7f31940d1500 2023-10-16T11:17:48.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.599+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31941f3700 con 0x7f31940d1500 2023-10-16T11:17:48.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.600+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31941f3f30 con 0x7f31940d1500 2023-10-16T11:17:48.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.601+0000 7f3189ffb700 1 -- 172.21.15.181:0/852782423 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 128) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f3184007470 con 0x7f31940d1500 2023-10-16T11:17:48.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.601+0000 7f3189ffb700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 0x7f317806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:48.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.601+0000 7f31994f6700 1 -- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 msgr2=0x7f317806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:48.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.601+0000 7f3189ffb700 1 -- 172.21.15.181:0/852782423 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f3184089650 con 0x7f31940d1500 2023-10-16T11:17:48.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.601+0000 7f31994f6700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 0x7f317806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:48.611 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:48.605+0000 7f6e54f83200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:17:48.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.606+0000 7f3189ffb700 1 -- 172.21.15.181:0/852782423 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31941f3f30 con 0x7f31940d1500 2023-10-16T11:17:48.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.802+0000 7f31994f6700 1 -- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 msgr2=0x7f317806d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:48.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.802+0000 7f31994f6700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 0x7f317806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:48.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.806+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f31941f3f30 con 0x7f31940d1500 2023-10-16T11:17:48.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.811+0000 7f3189ffb700 1 -- 172.21.15.181:0/852782423 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f31941f3f30 con 0x7f31940d1500 2023-10-16T11:17:48.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 msgr2=0x7f317806d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:48.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 0x7f317806d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d1500 msgr2=0x7f31941f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:48.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d1500 0x7f31941f4210 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f318400c470 tx=0x7f318400c780 comp rx=0 tx=0).stop 2023-10-16T11:17:48.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 shutdown_connections 2023-10-16T11:17:48.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f317806b070 0x7f317806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f319415ea40 0x7f31940c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31940d01d0 0x7f31940c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 --2- 172.21.15.181:0/852782423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31940d1500 0x7f31941f4210 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:48.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 >> 172.21.15.181:0/852782423 conn(0x7f319409cb60 msgr2=0x7f319415f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:48.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 shutdown_connections 2023-10-16T11:17:48.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:48.813+0000 7f319b75a700 1 -- 172.21.15.181:0/852782423 wait complete. 2023-10-16T11:17:48.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:48.828 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:17:48.849 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T11:17:48.901 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:48.899+0000 7ff125eed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:17:48.946 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:48.944+0000 7f6e54f83200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:17:49.015 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:49.013+0000 7ff125eed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:17:49.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.014+0000 7f4f3e613700 1 Processor -- start 2023-10-16T11:17:49.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.014+0000 7f4f3e613700 1 -- start start 2023-10-16T11:17:49.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3e613700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3815f150 0x7f4f380cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:49.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3e613700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f380ce4f0 0x7f4f380b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:49.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3e613700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f380b14a0 0x7f4f380b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:49.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3e613700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f380ce970 con 0x7f4f380b14a0 2023-10-16T11:17:49.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3e613700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f380b2ae0 con 0x7f4f380ce4f0 2023-10-16T11:17:49.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3e613700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380b2dc0 con 0x7f4f3815f150 2023-10-16T11:17:49.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f380b14a0 0x7f4f380b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:49.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f380b14a0 0x7f4f380b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54226/0 (socket says 172.21.15.181:54226) 2023-10-16T11:17:49.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 -- 172.21.15.181:0/2192557006 learned_addr learned my addr 172.21.15.181:0/2192557006 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:49.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f377fe700 1 --2- 172.21.15.181:0/2192557006 >> v2:172.21.15.190:3300/0 conn(0x7f4f380ce4f0 0x7f4f380b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:49.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f37fff700 1 --2- 172.21.15.181:0/2192557006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3815f150 0x7f4f380cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:49.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 -- 172.21.15.181:0/2192557006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3815f150 msgr2=0x7f4f380cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:49.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 --2- 172.21.15.181:0/2192557006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3815f150 0x7f4f380cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:49.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 -- 172.21.15.181:0/2192557006 >> v2:172.21.15.190:3300/0 conn(0x7f4f380ce4f0 msgr2=0x7f4f380b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:49.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 --2- 172.21.15.181:0/2192557006 >> v2:172.21.15.190:3300/0 conn(0x7f4f380ce4f0 0x7f4f380b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:49.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.015+0000 7f4f3cbb0700 1 -- 172.21.15.181:0/2192557006 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f380cf9d0 con 0x7f4f380b14a0 2023-10-16T11:17:49.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3cbb0700 1 --2- 172.21.15.181:0/2192557006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380b1930 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f4f38071730 tx=0x7f4f3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca6e3d0eed5a2b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:49.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f36ffd700 1 -- 172.21.15.181:0/2192557006 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000e070 con 0x7f4f380b14a0 2023-10-16T11:17:49.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f36ffd700 1 -- 172.21.15.181:0/2192557006 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3000b910 con 0x7f4f380b14a0 2023-10-16T11:17:49.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f36ffd700 1 -- 172.21.15.181:0/2192557006 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000bcc0 con 0x7f4f380b14a0 2023-10-16T11:17:49.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3e613700 1 -- 172.21.15.181:0/2192557006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 msgr2=0x7f4f380b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:49.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3e613700 1 --2- 172.21.15.181:0/2192557006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380b1930 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f4f38071730 tx=0x7f4f3000b330 comp rx=0 tx=0).stop 2023-10-16T11:17:49.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3e613700 1 -- 172.21.15.181:0/2192557006 shutdown_connections 2023-10-16T11:17:49.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3e613700 1 --2- 172.21.15.181:0/2192557006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380b1930 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:49.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3e613700 1 --2- 172.21.15.181:0/2192557006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f3815f150 0x7f4f380cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:49.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3e613700 1 --2- 172.21.15.181:0/2192557006 >> v2:172.21.15.190:3300/0 conn(0x7f4f380ce4f0 0x7f4f380b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:49.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.016+0000 7f4f3e613700 1 -- 172.21.15.181:0/2192557006 >> 172.21.15.181:0/2192557006 conn(0x7f4f3809a930 msgr2=0x7f4f380d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:49.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.017+0000 7f4f3e613700 1 -- 172.21.15.181:0/2192557006 shutdown_connections 2023-10-16T11:17:49.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.017+0000 7f4f3e613700 1 -- 172.21.15.181:0/2192557006 wait complete. 2023-10-16T11:17:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.018+0000 7f4f3e613700 1 Processor -- start 2023-10-16T11:17:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.018+0000 7f4f3e613700 1 -- start start 2023-10-16T11:17:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3e613700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3e613700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f380ce4f0 0x7f4f381559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:49.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3e613700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3815f150 0x7f4f380cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3e613700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3806cbb0 con 0x7f4f380b14a0 2023-10-16T11:17:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3e613700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380c56f0 con 0x7f4f3815f150 2023-10-16T11:17:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3e613700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380caf40 con 0x7f4f380ce4f0 2023-10-16T11:17:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54228/0 (socket says 172.21.15.181:54228) 2023-10-16T11:17:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 -- 172.21.15.181:0/893300516 learned_addr learned my addr 172.21.15.181:0/893300516 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3cbb0700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3815f150 0x7f4f380cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f380ce4f0 0x7f4f381559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f380ce4f0 msgr2=0x7f4f381559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f380ce4f0 0x7f4f381559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:49.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3815f150 msgr2=0x7f4f380cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:49.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3815f150 0x7f4f380cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:49.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 -- 172.21.15.181:0/893300516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f30009b30 con 0x7f4f380b14a0 2023-10-16T11:17:49.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f3cbb0700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f3815f150 0x7f4f380cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:49.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f37fff700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380caa00 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e870 tx=0x7f4f2800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:49.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.019+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28009590 con 0x7f4f380b14a0 2023-10-16T11:17:49.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.020+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f28013680 con 0x7f4f380b14a0 2023-10-16T11:17:49.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.020+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f380c5130 con 0x7f4f380b14a0 2023-10-16T11:17:49.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.020+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f280139f0 con 0x7f4f380b14a0 2023-10-16T11:17:49.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.020+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3806c4f0 con 0x7f4f380b14a0 2023-10-16T11:17:49.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.020+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f38067d20 con 0x7f4f380b14a0 2023-10-16T11:17:49.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.021+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 128) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f4f28013b50 con 0x7f4f380b14a0 2023-10-16T11:17:49.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.021+0000 7f4f34ff9700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 0x7f4f2406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:49.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.022+0000 7f4f377fe700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 msgr2=0x7f4f2406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:49.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.022+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6233+0+0 (secure 0 0 0) 0x7f4f280897b0 con 0x7f4f380b14a0 2023-10-16T11:17:49.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.022+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 0x7f4f2406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:17:49.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.026+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f280545d0 con 0x7f4f380b14a0 2023-10-16T11:17:49.068 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:49.067+0000 7f6e54f83200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:17:49.089 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:49.088+0000 7ff125eed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:17:49.146 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:49.144+0000 7f6e54f83200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:17:49.169 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:49.167+0000 7ff125eed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:17:49.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.223+0000 7f4f377fe700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 msgr2=0x7f4f2406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:49.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.223+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 0x7f4f2406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:17:49.228 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:49.227+0000 7f6e54f83200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:17:49.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.256+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 --> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f380cff90 con 0x7f4f2406b070 2023-10-16T11:17:49.537 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:49.536+0000 7ff125eed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:17:49.598 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:49.596+0000 7f6e54f83200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:17:49.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.624+0000 7f4f377fe700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 msgr2=0x7f4f2406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:49.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:49.624+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 0x7f4f2406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:17:50.105 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:50.104+0000 7ff125eed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:17:50.165 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:50.163+0000 7f6e54f83200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:17:50.175 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:50.174+0000 7ff125eed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:17:50.234 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:50.233+0000 7f6e54f83200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:17:50.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:50.425+0000 7f4f377fe700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 msgr2=0x7f4f2406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:50.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:50.425+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 0x7f4f2406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:17:50.803 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:50.802+0000 7ff125eed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:50.862 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:50.861+0000 7f6e54f83200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:50.881 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:50.879+0000 7ff125eed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:17:50.941 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:50.939+0000 7f6e54f83200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:17:51.025 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:51.024+0000 7ff125eed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:51.085 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:51.083+0000 7f6e54f83200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:17:51.254 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:51.253+0000 7ff125eed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:17:51.314 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:51.312+0000 7f6e54f83200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:17:51.329 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:51.327+0000 7ff125eed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:17:51.388 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:51.386+0000 7f6e54f83200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:17:51.592 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:51.591+0000 7ff125eed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:17:51.654 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:51.652+0000 7f6e54f83200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:17:51.812 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:51.811+0000 7ff125eed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:17:51.875 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:51.873+0000 7f6e54f83200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:17:51.962 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:51.960+0000 7ff125eed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:17:52.025 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:52.023+0000 7f6e54f83200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:17:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:52.026+0000 7f4f377fe700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 msgr2=0x7f4f2406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1532329351 2023-10-16T11:17:52.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:52.026+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 0x7f4f2406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:17:52.074 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:52.073+0000 7ff125eed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:17:52.139 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:52.138+0000 7f6e54f83200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:17:52.229 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:52.227+0000 7ff125eed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:17:52.297 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:52.295+0000 7f6e54f83200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:17:52.491 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:52.489+0000 7ff125eed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:52.562 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:52.560+0000 7f6e54f83200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:17:52.653 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:52.652+0000 7ff125eed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:17:52.724 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:52.722+0000 7f6e54f83200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:17:53.181 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:17:53.180+0000 7ff125eed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:17:53.252 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.250+0000 7f6e54f83200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:17:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:53.250+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 129) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f4f280549a0 con 0x7f4f380b14a0 2023-10-16T11:17:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:53.276+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 130) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f4f280584f0 con 0x7f4f380b14a0 2023-10-16T11:17:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:53.276+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 msgr2=0x7f4f2406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:17:53.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:53.276+0000 7f4f34ff9700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1532329351,v1:172.21.15.181:6833/1532329351] conn(0x7f4f2406b070 0x7f4f2406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:53.423 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dcaf64700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.424 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dcaf64700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.424 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dcaf64700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.424 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dcaf64700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.424 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dcaf64700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.425 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dc7f5e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.425 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dc7f5e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.425 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dc7f5e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.425 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dc7f5e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:53.425 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:17:53.421+0000 7f6dc7f5e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:17:54.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.280+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 131) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f4f28055270 con 0x7f4f380b14a0 2023-10-16T11:17:54.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.280+0000 7f4f34ff9700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4f2401d250 0x7f4f24075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.280+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 --> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f24071070 con 0x7f4f2401d250 2023-10-16T11:17:54.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.282+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4f2401d250 0x7f4f24075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.285+0000 7f4f377fe700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4f2401d250 0x7f4f24075680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f380b0310 tx=0x7f4f2c009210 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.347+0000 7f4f34ff9700 1 -- 172.21.15.181:0/893300516 <== mgr.10977 v2:172.21.15.181:6832/1119809348 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4f24071070 con 0x7f4f2401d250 2023-10-16T11:17:54.349 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1 2023-10-16T11:17:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4f2401d250 msgr2=0x7f4f24075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4f2401d250 0x7f4f24075680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f380b0310 tx=0x7f4f2c009210 comp rx=0 tx=0).stop 2023-10-16T11:17:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 msgr2=0x7f4f380caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 --2- 172.21.15.181:0/893300516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f380b14a0 0x7f4f380caa00 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e870 tx=0x7f4f2800eb80 comp rx=0 tx=0).stop 2023-10-16T11:17:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3cbb0700 1 -- 172.21.15.181:0/893300516 reap_dead start 2023-10-16T11:17:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 shutdown_connections 2023-10-16T11:17:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 >> 172.21.15.181:0/893300516 conn(0x7f4f3809a930 msgr2=0x7f4f3815dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 shutdown_connections 2023-10-16T11:17:54.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.349+0000 7f4f3e613700 1 -- 172.21.15.181:0/893300516 wait complete. 2023-10-16T11:17:54.366 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T11:17:54.366 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-16T11:17:54.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.490+0000 7f7ca585c700 1 Processor -- start 2023-10-16T11:17:54.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.490+0000 7f7ca585c700 1 -- start start 2023-10-16T11:17:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.491+0000 7f7ca585c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ca006d530 0x7f7ca00d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.491+0000 7f7ca585c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca015dd80 con 0x7f7ca00d01e0 2023-10-16T11:17:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.491+0000 7f7ca585c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca006ba60 con 0x7f7ca00ce270 2023-10-16T11:17:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.491+0000 7f7ca585c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca00d42a0 con 0x7f7ca006d530 2023-10-16T11:17:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.491+0000 7f7c9effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7ca00d01e0 0x7f7ca00d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37266/0 (socket says 172.21.15.181:37266) 2023-10-16T11:17:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.491+0000 7f7c9effd700 1 -- 172.21.15.181:0/395515855 learned_addr learned my addr 172.21.15.181:0/395515855 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.491+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/395515855 >> v2:172.21.15.190:3300/0 conn(0x7f7ca006d530 0x7f7ca00d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965904597 0 0) 0x7f7ca015dd80 con 0x7f7ca00d01e0 2023-10-16T11:17:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c880035b0 con 0x7f7ca00d01e0 2023-10-16T11:17:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760127882 0 0) 0x7f7ca006ba60 con 0x7f7ca00ce270 2023-10-16T11:17:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca015dd80 con 0x7f7ca00ce270 2023-10-16T11:17:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901524278 0 0) 0x7f7c880035b0 con 0x7f7ca00d01e0 2023-10-16T11:17:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 >> v2:172.21.15.190:3300/0 conn(0x7f7ca006d530 msgr2=0x7f7ca00d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 --2- 172.21.15.181:0/395515855 >> v2:172.21.15.190:3300/0 conn(0x7f7ca006d530 0x7f7ca00d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 >> v1:172.21.15.190:6789/0 conn(0x7f7ca00ce270 legacy=0x7f7ca00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca00d4580 con 0x7f7ca00d01e0 2023-10-16T11:17:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7c94001670 con 0x7f7ca00d01e0 2023-10-16T11:17:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 >> v1:172.21.15.181:6789/0 conn(0x7f7ca00d01e0 legacy=0x7f7ca00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c88003b90 0x7f7c88003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7c88004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c88005210 0x7f7c8800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00d42a0 con 0x7f7c88004850 2023-10-16T11:17:54.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8800c090 con 0x7f7c88005210 2023-10-16T11:17:54.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8800c250 con 0x7f7c88003b90 2023-10-16T11:17:54.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/395515855 >> v2:172.21.15.190:3300/0 conn(0x7f7ca006d530 0x7f7ca00d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:17:54.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7c88004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9effd700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c88003b90 0x7f7c88003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9f7fe700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c88005210 0x7f7c8800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 -- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c88003b90 msgr2=0x7f7c88003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:54.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c88003b90 0x7f7c88003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 -- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c88005210 msgr2=0x7f7c8800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c88005210 0x7f7c8800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9f7fe700 1 -- 172.21.15.181:0/395515855 reap_dead start 2023-10-16T11:17:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 -- 172.21.15.181:0/395515855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880035b0 con 0x7f7c88004850 2023-10-16T11:17:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.492+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7c88004cd0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009d10 tx=0x7f7c88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6274e4ac491211 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90009e60 con 0x7f7c88004850 2023-10-16T11:17:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9000e9a0 con 0x7f7c88004850 2023-10-16T11:17:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7c9dffb700 1 -- 172.21.15.181:0/395515855 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000ed30 con 0x7f7c88004850 2023-10-16T11:17:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7ca585c700 1 -- 172.21.15.181:0/395515855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 msgr2=0x7f7c88004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7ca585c700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7c88004cd0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009d10 tx=0x7f7c88005880 comp rx=0 tx=0).stop 2023-10-16T11:17:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7ca585c700 1 -- 172.21.15.181:0/395515855 shutdown_connections 2023-10-16T11:17:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7ca585c700 1 --2- 172.21.15.181:0/395515855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7c88004cd0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7ca585c700 1 -- 172.21.15.181:0/395515855 >> 172.21.15.181:0/395515855 conn(0x7f7ca008af90 msgr2=0x7f7ca00d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7ca585c700 1 -- 172.21.15.181:0/395515855 shutdown_connections 2023-10-16T11:17:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.493+0000 7f7ca585c700 1 -- 172.21.15.181:0/395515855 wait complete. 2023-10-16T11:17:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.494+0000 7f7ca585c700 1 Processor -- start 2023-10-16T11:17:54.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7ca585c700 1 -- start start 2023-10-16T11:17:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7ca585c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7ca00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7ca585c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ca00c96d0 0x7f7ca00b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7ca585c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ca00c9b50 0x7f7ca00b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7ca585c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00c9240 con 0x7f7c88004850 2023-10-16T11:17:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7ca585c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00c8910 con 0x7f7ca00c96d0 2023-10-16T11:17:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7ca585c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00c8bf0 con 0x7f7ca00c9b50 2023-10-16T11:17:54.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7c9effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7ca00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7c9effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7ca00b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54432/0 (socket says 172.21.15.181:54432) 2023-10-16T11:17:54.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7c9effd700 1 -- 172.21.15.181:0/1158155981 learned_addr learned my addr 172.21.15.181:0/1158155981 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:54.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ca00c96d0 0x7f7ca00b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.495+0000 7f7c9f7fe700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ca00c9b50 0x7f7ca00b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c9effd700 1 -- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ca00c9b50 msgr2=0x7f7ca00b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c9effd700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ca00c9b50 0x7f7ca00b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c9effd700 1 -- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ca00c96d0 msgr2=0x7f7ca00b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c9effd700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ca00c96d0 0x7f7ca00b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c9effd700 1 -- 172.21.15.181:0/1158155981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca00c8ed0 con 0x7f7c88004850 2023-10-16T11:17:54.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c9effd700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7ca00b0cf0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f7c94007340 tx=0x7f7c94007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:54.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c87fff700 1 -- 172.21.15.181:0/1158155981 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400f660 con 0x7f7c88004850 2023-10-16T11:17:54.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c87fff700 1 -- 172.21.15.181:0/1158155981 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9400f7c0 con 0x7f7c88004850 2023-10-16T11:17:54.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca00c8570 con 0x7f7c88004850 2023-10-16T11:17:54.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7c87fff700 1 -- 172.21.15.181:0/1158155981 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400fb70 con 0x7f7c88004850 2023-10-16T11:17:54.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.496+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca00b6300 con 0x7f7c88004850 2023-10-16T11:17:54.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.497+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca00c9520 con 0x7f7c88004850 2023-10-16T11:17:54.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.497+0000 7f7c87fff700 1 -- 172.21.15.181:0/1158155981 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 131) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f7c9400fcd0 con 0x7f7c88004850 2023-10-16T11:17:54.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.498+0000 7f7c87fff700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f7c8c06add0 0x7f7c8c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.498+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f7c8c06add0 0x7f7c8c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.498+0000 7f7c87fff700 1 -- 172.21.15.181:0/1158155981 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f7c94089440 con 0x7f7c88004850 2023-10-16T11:17:54.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.498+0000 7f7c9e7fc700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f7c8c06add0 0x7f7c8c06d290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7c900090b0 tx=0x7f7c9001d040 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:54.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.503+0000 7f7c87fff700 1 -- 172.21.15.181:0/1158155981 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7c940544c0 con 0x7f7c88004850 2023-10-16T11:17:54.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.724+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 --> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ca00c8a80 con 0x7f7c8c06add0 2023-10-16T11:17:54.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.775+0000 7f7c87fff700 1 -- 172.21.15.181:0/1158155981 <== mgr.10977 v2:172.21.15.181:6832/1119809348 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f7ca00c8a80 con 0x7f7c8c06add0 2023-10-16T11:17:54.778 INFO:teuthology.orchestra.run.smithi181.stdout:Retention added to path snap_test_dir1 2023-10-16T11:17:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f7c8c06add0 msgr2=0x7f7c8c06d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f7c8c06add0 0x7f7c8c06d290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7c900090b0 tx=0x7f7c9001d040 comp rx=0 tx=0).stop 2023-10-16T11:17:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 msgr2=0x7f7ca00b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7ca00b0cf0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f7c94007340 tx=0x7f7c94007650 comp rx=0 tx=0).stop 2023-10-16T11:17:54.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 shutdown_connections 2023-10-16T11:17:54.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f7c8c06add0 0x7f7c8c06d290 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ca00c9b50 0x7f7ca00b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c88004850 0x7f7ca00b0cf0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 --2- 172.21.15.181:0/1158155981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ca00c96d0 0x7f7ca00b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 >> 172.21.15.181:0/1158155981 conn(0x7f7ca008af90 msgr2=0x7f7ca00ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:54.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 shutdown_connections 2023-10-16T11:17:54.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.778+0000 7f7ca585c700 1 -- 172.21.15.181:0/1158155981 wait complete. 2023-10-16T11:17:54.792 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-16T11:17:54.792 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-11_18_00 in ~6s... 2023-10-16T11:17:54.793 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-16T11:17:54.793 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T11:17:54.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.913+0000 7f6f18a81700 1 Processor -- start 2023-10-16T11:17:54.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.914+0000 7f6f18a81700 1 -- start start 2023-10-16T11:17:54.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.914+0000 7f6f18a81700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f14072760 0x7f6f1406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.914+0000 7f6f18a81700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1406b210 con 0x7f6f1415da70 2023-10-16T11:17:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.914+0000 7f6f18a81700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f140cef00 con 0x7f6f1406ade0 2023-10-16T11:17:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.914+0000 7f6f18a81700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f140cec20 con 0x7f6f14072760 2023-10-16T11:17:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6f1415da70 0x7f6f140ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37286/0 (socket says 172.21.15.181:37286) 2023-10-16T11:17:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1259c700 1 -- 172.21.15.181:0/2555326230 learned_addr learned my addr 172.21.15.181:0/2555326230 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:17:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f11d9b700 1 --2- 172.21.15.181:0/2555326230 >> v2:172.21.15.190:3300/0 conn(0x7f6f14072760 0x7f6f1406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079180937 0 0) 0x7f6f1406b210 con 0x7f6f1415da70 2023-10-16T11:17:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6efc0035b0 con 0x7f6f1415da70 2023-10-16T11:17:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 235293377 0 0) 0x7f6efc0035b0 con 0x7f6f1415da70 2023-10-16T11:17:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 >> v2:172.21.15.190:3300/0 conn(0x7f6f14072760 msgr2=0x7f6f1406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 --2- 172.21.15.181:0/2555326230 >> v2:172.21.15.190:3300/0 conn(0x7f6f14072760 0x7f6f1406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 >> v1:172.21.15.190:6789/0 conn(0x7f6f1406ade0 legacy=0x7f6f141615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f14071c80 con 0x7f6f1415da70 2023-10-16T11:17:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f6f04001670 con 0x7f6f1415da70 2023-10-16T11:17:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 >> v1:172.21.15.181:6789/0 conn(0x7f6f1415da70 legacy=0x7f6f140ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6efc003a00 0x7f6efc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6efc004850 0x7f6efc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6efc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140cec20 con 0x7f6efc005330 2023-10-16T11:17:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc00c200 con 0x7f6efc004850 2023-10-16T11:17:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc00c3c0 con 0x7f6efc003a00 2023-10-16T11:17:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f12d9d700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6efc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f11d9b700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6efc004850 0x7f6efc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.915+0000 7f6f1259c700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6efc003a00 0x7f6efc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f12d9d700 1 -- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6efc003a00 msgr2=0x7f6efc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f12d9d700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6efc003a00 0x7f6efc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f12d9d700 1 -- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6efc004850 msgr2=0x7f6efc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:17:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f12d9d700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6efc004850 0x7f6efc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f12d9d700 1 -- 172.21.15.181:0/2555326230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc0035b0 con 0x7f6efc005330 2023-10-16T11:17:54.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f12d9d700 1 -- 172.21.15.181:0/2555326230 reap_dead start 2023-10-16T11:17:54.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f12d9d700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6efc00ba10 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c160 tx=0x7f6efc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfff9fddeedaa02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:54.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c001db0 con 0x7f6efc005330 2023-10-16T11:17:54.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c00cca0 con 0x7f6efc005330 2023-10-16T11:17:54.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f1159a700 1 -- 172.21.15.181:0/2555326230 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c003c20 con 0x7f6efc005330 2023-10-16T11:17:54.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f18a81700 1 -- 172.21.15.181:0/2555326230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 msgr2=0x7f6efc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f18a81700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6efc00ba10 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c160 tx=0x7f6efc005880 comp rx=0 tx=0).stop 2023-10-16T11:17:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f18a81700 1 -- 172.21.15.181:0/2555326230 shutdown_connections 2023-10-16T11:17:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f18a81700 1 --2- 172.21.15.181:0/2555326230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6efc00ba10 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f18a81700 1 -- 172.21.15.181:0/2555326230 >> 172.21.15.181:0/2555326230 conn(0x7f6f140cd010 msgr2=0x7f6f140cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f18a81700 1 -- 172.21.15.181:0/2555326230 shutdown_connections 2023-10-16T11:17:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.916+0000 7f6f18a81700 1 -- 172.21.15.181:0/2555326230 wait complete. 2023-10-16T11:17:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.918+0000 7f6f18a81700 1 Processor -- start 2023-10-16T11:17:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.918+0000 7f6f18a81700 1 -- start start 2023-10-16T11:17:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6f140c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f14160370 0x7f6f1415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f141607f0 0x7f6f1415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140c5f60 con 0x7f6efc005330 2023-10-16T11:17:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140b58c0 con 0x7f6f14160370 2023-10-16T11:17:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140b31b0 con 0x7f6f141607f0 2023-10-16T11:17:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6f140c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6f140c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54452/0 (socket says 172.21.15.181:54452) 2023-10-16T11:17:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 -- 172.21.15.181:0/4291724069 learned_addr learned my addr 172.21.15.181:0/4291724069 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:17:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f12d9d700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f141607f0 0x7f6f1415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f11d9b700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f14160370 0x7f6f1415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 -- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f141607f0 msgr2=0x7f6f1415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f141607f0 0x7f6f1415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 -- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f14160370 msgr2=0x7f6f1415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f14160370 0x7f6f1415f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 -- 172.21.15.181:0/4291724069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140b3490 con 0x7f6efc005330 2023-10-16T11:17:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f1259c700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6f140c5a20 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f6f040072c0 tx=0x7f6f040075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f02ffd700 1 -- 172.21.15.181:0/4291724069 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400f6c0 con 0x7f6efc005330 2023-10-16T11:17:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f02ffd700 1 -- 172.21.15.181:0/4291724069 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0400f820 con 0x7f6efc005330 2023-10-16T11:17:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140b3770 con 0x7f6efc005330 2023-10-16T11:17:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f02ffd700 1 -- 172.21.15.181:0/4291724069 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400fbd0 con 0x7f6efc005330 2023-10-16T11:17:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.919+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f140b5e40 con 0x7f6efc005330 2023-10-16T11:17:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.920+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f140ce920 con 0x7f6efc005330 2023-10-16T11:17:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.921+0000 7f6f02ffd700 1 -- 172.21.15.181:0/4291724069 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 131) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f6f0400fd30 con 0x7f6efc005330 2023-10-16T11:17:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.921+0000 7f6f02ffd700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f6ef406ada0 0x7f6ef406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:17:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.921+0000 7f6f02ffd700 1 -- 172.21.15.181:0/4291724069 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f6f04089410 con 0x7f6efc005330 2023-10-16T11:17:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.921+0000 7f6f11d9b700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f6ef406ada0 0x7f6ef406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:17:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.921+0000 7f6f11d9b700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f6ef406ada0 0x7f6ef406d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6f140c71c0 tx=0x7f6f08009210 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:17:54.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:54.926+0000 7f6f02ffd700 1 -- 172.21.15.181:0/4291724069 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6f04054490 con 0x7f6efc005330 2023-10-16T11:17:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.148+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 --> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f140b39b0 con 0x7f6ef406ada0 2023-10-16T11:17:55.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.157+0000 7f6f02ffd700 1 -- 172.21.15.181:0/4291724069 <== mgr.10977 v2:172.21.15.181:6832/1119809348 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f6f08019070 con 0x7f6ef406ada0 2023-10-16T11:17:55.159 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:17:55.159 INFO:teuthology.orchestra.run.smithi181.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:17:55.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f6ef406ada0 msgr2=0x7f6ef406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:55.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f6ef406ada0 0x7f6ef406d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6f140c71c0 tx=0x7f6f08009210 comp rx=0 tx=0).stop 2023-10-16T11:17:55.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 msgr2=0x7f6f140c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:17:55.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6f140c5a20 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f6f040072c0 tx=0x7f6f040075d0 comp rx=0 tx=0).stop 2023-10-16T11:17:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 shutdown_connections 2023-10-16T11:17:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f6ef406ada0 0x7f6ef406d260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f141607f0 0x7f6f1415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc005330 0x7f6f140c5a20 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:55.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 --2- 172.21.15.181:0/4291724069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f14160370 0x7f6f1415f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:17:55.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 >> 172.21.15.181:0/4291724069 conn(0x7f6f140cd010 msgr2=0x7f6f140b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:17:55.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 shutdown_connections 2023-10-16T11:17:55.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:17:55.159+0000 7f6f18a81700 1 -- 172.21.15.181:0/4291724069 wait complete. 2023-10-16T11:17:55.173 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:17:55.173 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-16T11:17:55.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:55.174 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:17:55.192 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:17:55.192 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:17:55.192 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:17:56.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:56.194 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:17:56.216 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:17:56.217 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:17:56.217 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:17:56.860 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:56.863 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:17:57.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:57.219 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:17:57.241 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:17:57.242 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:17:57.242 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:17:58.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:58.244 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:17:58.266 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:17:58.267 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:17:58.267 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:17:59.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:17:59.269 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:17:59.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:17:59.291 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:17:59.291 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:00.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:00.293 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:00.315 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-16T11:18:00.315 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:00.315 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:00.788 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:18:00.786+0000 7f6db9f42700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.788 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:18:00.786+0000 7f6db9f42700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.789 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:18:00.786+0000 7f6db9f42700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.789 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:18:00.786+0000 7f6db9f42700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:00.789 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:18:00.786+0000 7f6db9f42700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:18:01.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:01.317 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:01.338 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:01.339 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:01.339 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:01.339 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:01.340 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-16-11_18_00_UTC'] 2023-10-16T11:18:01.340 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697455074.792786, now = 1697455081.3404198, timo = 6 2023-10-16T11:18:02.341 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-16-11_18_00 in ~66s... 2023-10-16T11:18:02.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:02.342 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:02.362 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:02.363 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:02.363 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:02.364 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:03.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:03.366 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:03.387 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:03.388 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:03.389 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:03.389 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:04.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:04.391 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:04.412 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:04.413 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:04.413 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:04.413 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:05.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:05.415 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:05.435 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:05.436 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:05.437 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:05.437 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:06.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:06.439 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:06.460 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:06.461 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:06.461 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:06.462 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:07.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:07.464 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:07.485 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:07.485 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:07.486 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:07.486 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:08.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:08.489 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:08.510 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:08.511 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:08.512 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:08.512 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:09.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:09.513 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:09.534 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:09.535 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:09.536 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:09.536 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:10.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:10.538 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:10.559 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:10.560 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:10.560 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:10.561 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:11.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:11.562 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:11.583 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:11.584 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:11.585 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:11.585 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:12.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:12.587 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:12.608 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:12.609 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:12.609 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:12.610 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:13.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:13.611 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:13.633 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:13.678 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:13.678 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:13.679 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:14.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:14.680 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:14.701 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:14.702 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:14.703 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:14.703 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:15.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:15.706 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:15.728 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:15.728 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:15.729 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:15.729 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:16.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:16.730 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:16.751 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:16.752 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:16.753 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:16.753 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:17.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:17.754 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:17.777 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:17.777 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:17.778 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:17.778 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:18.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:18.779 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:18.801 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:18.802 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:18.802 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:18.802 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:19.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:19.804 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:19.825 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:19.826 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:19.826 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:19.826 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:20.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:20.828 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:20.850 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:20.851 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:20.851 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:20.851 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:21.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:21.853 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:21.875 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:21.876 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:21.876 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:21.876 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:22.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:22.878 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:22.899 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:22.900 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:22.901 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:22.901 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:23.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:23.903 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:23.925 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:23.925 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:23.926 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:23.926 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:24.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:24.928 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:24.949 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:24.950 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:24.951 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:24.951 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:25.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:25.954 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:25.975 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:25.976 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:25.976 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:25.977 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:26.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:26.978 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:26.989 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:27.000 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:27.000 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:27.000 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:27.001 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:27.001 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:28.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:28.002 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:28.023 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:28.024 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:28.024 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:28.024 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:29.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:29.026 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:29.047 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:29.048 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:29.048 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:29.048 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:30.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:30.051 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:30.071 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:30.072 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:30.073 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:30.073 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:31.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:31.074 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:31.095 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:31.096 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:31.096 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:31.097 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:32.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:32.098 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:32.119 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:32.120 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:32.120 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:32.120 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:33.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:33.122 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:33.143 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:33.144 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:33.145 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:33.145 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:34.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:34.174 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:34.196 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:34.197 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:34.197 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:34.197 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:35.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:35.199 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:35.221 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:35.221 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:35.222 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:35.222 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:36.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:36.223 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:36.244 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:36.245 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:36.246 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:36.246 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:37.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:37.248 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:37.269 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:37.269 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:37.270 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:37.270 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:38.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:38.271 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:38.292 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:38.293 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:38.293 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:38.293 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:39.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:39.295 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:39.315 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:39.316 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:39.317 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:39.317 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:40.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:40.319 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:40.340 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:40.340 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:40.341 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:40.341 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:41.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:41.343 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:41.364 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:41.365 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:41.365 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:41.365 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:42.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:42.367 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:42.388 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:42.388 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:42.389 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:42.389 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:43.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:43.390 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:43.411 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:43.413 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:43.413 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:43.413 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:44.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:44.415 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:44.437 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:44.437 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:44.437 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:44.438 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:45.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:45.440 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:45.462 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:45.462 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:45.462 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:45.463 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:46.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:46.464 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:46.484 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:46.485 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:46.486 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:46.486 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:47.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:47.488 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:47.510 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:47.510 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:47.511 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:47.511 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:48.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:48.513 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:48.534 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:48.535 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:48.535 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:48.536 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:49.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:49.538 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:49.560 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:49.561 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:49.561 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:49.562 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:50.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:50.564 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:50.585 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:50.586 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:50.586 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:50.586 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:51.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:51.589 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:51.610 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:51.611 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:51.611 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:51.612 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:52.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:52.613 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:52.635 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:52.635 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:52.636 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:52.636 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:53.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:53.638 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:53.659 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:53.661 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:53.661 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:53.662 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:54.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:54.664 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:54.686 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:54.686 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:54.686 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:54.687 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:55.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:55.689 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:55.709 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:55.711 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:55.711 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:55.712 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:56.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:56.713 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:56.736 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:56.737 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:56.737 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:56.737 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:57.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:57.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:18:57.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:57.739 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:57.760 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:57.761 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:57.761 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:57.762 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:58.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:58.763 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:58.784 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:58.785 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:58.786 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:58.786 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:18:59.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:18:59.787 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:18:59.808 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:18:59.809 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:18:59.810 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:18:59.810 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:19:00.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:00.812 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:19:00.833 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_18_00_UTC 2023-10-16T11:19:00.834 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:19:00.834 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-16T11:19:00.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-16T11:19:01.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:01.836 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:19:01.857 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_19_00_UTC 2023-10-16T11:19:01.858 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-16-11_19_00_UTC'} 2023-10-16T11:19:01.859 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-16-11_19_00_UTC'} 2023-10-16T11:19:01.859 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-16-11_18_00_UTC'} 2023-10-16T11:19:01.859 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-16-11_18_00_UTC'] 2023-10-16T11:19:01.859 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697455074.792786, now = 1697455141.8599792, timo = 66 2023-10-16T11:19:02.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T11:19:02.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.995+0000 7fce2402a700 1 Processor -- start 2023-10-16T11:19:02.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.995+0000 7fce2402a700 1 -- start start 2023-10-16T11:19:02.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce2402a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:02.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce2402a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fce1c0ce6a0 0x7fce1c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:02.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce2402a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1c0cef00 con 0x7fce1c072760 2023-10-16T11:19:02.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce2402a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1c0cec20 con 0x7fce1c0ce6a0 2023-10-16T11:19:03.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce2402a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c071c80 con 0x7fce1c15da70 2023-10-16T11:19:03.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce215c5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fce1c0ce6a0 0x7fce1c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce225c7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fce1c072760 0x7fce1c1615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37340/0 (socket says 172.21.15.181:37340) 2023-10-16T11:19:03.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce225c7700 1 -- 172.21.15.181:0/1551796885 learned_addr learned my addr 172.21.15.181:0/1551796885 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:03.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce215c5700 1 -- 172.21.15.181:0/1551796885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 msgr2=0x7fce1c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:03.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce215c5700 1 --2- 172.21.15.181:0/1551796885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce215c5700 1 -- 172.21.15.181:0/1551796885 >> v1:172.21.15.181:6789/0 conn(0x7fce1c072760 legacy=0x7fce1c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce215c5700 1 -- 172.21.15.181:0/1551796885 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c071f60 con 0x7fce1c0ce6a0 2023-10-16T11:19:03.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce20dc4700 1 -- 172.21.15.181:0/1551796885 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265311887 0 0) 0x7fce1c0cef00 con 0x7fce1c072760 2023-10-16T11:19:03.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.996+0000 7fce215c5700 1 --2- 172.21.15.181:0/1551796885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c06a8a0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009840 tx=0x7fce0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c9b83e5198480d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:03.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce20dc4700 1 -- 172.21.15.181:0/1551796885 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c011830 con 0x7fce1c0ce6a0 2023-10-16T11:19:03.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce20dc4700 1 -- 172.21.15.181:0/1551796885 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c011990 con 0x7fce1c0ce6a0 2023-10-16T11:19:03.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce20dc4700 1 -- 172.21.15.181:0/1551796885 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c011d20 con 0x7fce1c0ce6a0 2023-10-16T11:19:03.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 -- 172.21.15.181:0/1551796885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0ce6a0 msgr2=0x7fce1c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 --2- 172.21.15.181:0/1551796885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c06a8a0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009840 tx=0x7fce0c009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:03.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 -- 172.21.15.181:0/1551796885 shutdown_connections 2023-10-16T11:19:03.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 --2- 172.21.15.181:0/1551796885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 --2- 172.21.15.181:0/1551796885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c06a8a0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 -- 172.21.15.181:0/1551796885 >> 172.21.15.181:0/1551796885 conn(0x7fce1c0cd010 msgr2=0x7fce1c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:03.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 -- 172.21.15.181:0/1551796885 shutdown_connections 2023-10-16T11:19:03.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.997+0000 7fce2402a700 1 -- 172.21.15.181:0/1551796885 wait complete. 2023-10-16T11:19:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.998+0000 7fce2402a700 1 Processor -- start 2023-10-16T11:19:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.998+0000 7fce2402a700 1 -- start start 2023-10-16T11:19:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce2402a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c072760 0x7fce1c0c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce2402a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce2402a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce2402a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0c7ba0 con 0x7fce1c072760 2023-10-16T11:19:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce2402a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0b5730 con 0x7fce1c0ce6a0 2023-10-16T11:19:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce2402a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0c40c0 con 0x7fce1c15da70 2023-10-16T11:19:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce21dc6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c072760 0x7fce1c0c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce21dc6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c072760 0x7fce1c0c5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54498/0 (socket says 172.21.15.181:54498) 2023-10-16T11:19:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce21dc6700 1 -- 172.21.15.181:0/3732416769 learned_addr learned my addr 172.21.15.181:0/3732416769 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce215c5700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce225c7700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:02.999+0000 7fce21dc6700 1 -- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 msgr2=0x7fce1c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce21dc6700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce21dc6700 1 -- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce1c0ce6a0 msgr2=0x7fce1c0c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce21dc6700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c0c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce21dc6700 1 -- 172.21.15.181:0/3732416769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c0094a0 con 0x7fce1c072760 2023-10-16T11:19:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce215c5700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c0c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce225c7700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce21dc6700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c072760 0x7fce1c0c5a50 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7fce1800b590 tx=0x7fce1800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce127fc700 1 -- 172.21.15.181:0/3732416769 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18011830 con 0x7fce1c072760 2023-10-16T11:19:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce127fc700 1 -- 172.21.15.181:0/3732416769 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce18011990 con 0x7fce1c072760 2023-10-16T11:19:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c0c7800 con 0x7fce1c072760 2023-10-16T11:19:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce127fc700 1 -- 172.21.15.181:0/3732416769 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18011d40 con 0x7fce1c072760 2023-10-16T11:19:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.000+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c0b5450 con 0x7fce1c072760 2023-10-16T11:19:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.001+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c0c7640 con 0x7fce1c072760 2023-10-16T11:19:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.001+0000 7fce127fc700 1 -- 172.21.15.181:0/3732416769 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 134) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fce1801a400 con 0x7fce1c072760 2023-10-16T11:19:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.002+0000 7fce127fc700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fce0006b190 0x7fce0006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.002+0000 7fce215c5700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fce0006b190 0x7fce0006d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.002+0000 7fce127fc700 1 -- 172.21.15.181:0/3732416769 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6635+0+0 (secure 0 0 0) 0x7fce18089e80 con 0x7fce1c072760 2023-10-16T11:19:03.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.002+0000 7fce215c5700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fce0006b190 0x7fce0006d650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0c7150 tx=0x7fce0c008030 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:03.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.006+0000 7fce127fc700 1 -- 172.21.15.181:0/3732416769 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fce18054820 con 0x7fce1c072760 2023-10-16T11:19:03.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.226+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 --> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fce1c15de50 con 0x7fce0006b190 2023-10-16T11:19:03.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.288+0000 7fce127fc700 1 -- 172.21.15.181:0/3732416769 <== mgr.10977 v2:172.21.15.181:6832/1119809348 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fce0c011c00 con 0x7fce0006b190 2023-10-16T11:19:03.289 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path snap_test_dir1 2023-10-16T11:19:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fce0006b190 msgr2=0x7fce0006d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fce0006b190 0x7fce0006d650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0c7150 tx=0x7fce0c008030 comp rx=0 tx=0).stop 2023-10-16T11:19:03.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c072760 msgr2=0x7fce1c0c5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c072760 0x7fce1c0c5a50 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7fce1800b590 tx=0x7fce1800b950 comp rx=0 tx=0).stop 2023-10-16T11:19:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 shutdown_connections 2023-10-16T11:19:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fce0006b190 0x7fce0006d650 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce1c15da70 0x7fce1c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce1c072760 0x7fce1c0c5a50 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 --2- 172.21.15.181:0/3732416769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce1c0ce6a0 0x7fce1c0c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 >> 172.21.15.181:0/3732416769 conn(0x7fce1c0cd010 msgr2=0x7fce1c15b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:03.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 shutdown_connections 2023-10-16T11:19:03.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.290+0000 7fce2402a700 1 -- 172.21.15.181:0/3732416769 wait complete. 2023-10-16T11:19:03.303 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T11:19:03.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:03.304 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:19:03.321 INFO:teuthology.orchestra.run.smithi181.stdout:scheduled-2023-10-16-11_19_00_UTC 2023-10-16T11:19:03.321 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-16-11_19_00_UTC 2023-10-16T11:19:03.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:03.322 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-16-11_19_00_UTC) 2023-10-16T11:19:03.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:03.382 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:19:03.441 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:19:03.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.601+0000 7fe273426700 1 Processor -- start 2023-10-16T11:19:03.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.601+0000 7fe273426700 1 -- start start 2023-10-16T11:19:03.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.602+0000 7fe273426700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe26c071be0 0x7fe26c0ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.602+0000 7fe273426700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26c0ce1f0 con 0x7fe26c0cc6e0 2023-10-16T11:19:03.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.602+0000 7fe273426700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26c0ccda0 con 0x7fe26c0b2010 2023-10-16T11:19:03.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.602+0000 7fe273426700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26c0ccb90 con 0x7fe26c071be0 2023-10-16T11:19:03.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.602+0000 7fe2709c1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe26c0cc6e0 0x7fe26c0ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37352/0 (socket says 172.21.15.181:37352) 2023-10-16T11:19:03.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.602+0000 7fe2709c1700 1 -- 172.21.15.181:0/2105143853 learned_addr learned my addr 172.21.15.181:0/2105143853 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:03.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839194994 0 0) 0x7fe26c0ce1f0 con 0x7fe26c0cc6e0 2023-10-16T11:19:03.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 --2- 172.21.15.181:0/2105143853 >> v2:172.21.15.190:3300/0 conn(0x7fe26c071be0 0x7fe26c0ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2580035b0 con 0x7fe26c0cc6e0 2023-10-16T11:19:03.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152082431 0 0) 0x7fe2580035b0 con 0x7fe26c0cc6e0 2023-10-16T11:19:03.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 >> v2:172.21.15.190:3300/0 conn(0x7fe26c071be0 msgr2=0x7fe26c0ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:03.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 --2- 172.21.15.181:0/2105143853 >> v2:172.21.15.190:3300/0 conn(0x7fe26c071be0 0x7fe26c0ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 >> v1:172.21.15.190:6789/0 conn(0x7fe26c0b2010 legacy=0x7fe26c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c0727e0 con 0x7fe26c0cc6e0 2023-10-16T11:19:03.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe268001670 con 0x7fe26c0cc6e0 2023-10-16T11:19:03.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 >> v1:172.21.15.181:6789/0 conn(0x7fe26c0cc6e0 legacy=0x7fe26c0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258003a00 0x7fe258003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe258004850 0x7fe258004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe258005210 0x7fe25800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c0ccb90 con 0x7fe258005210 2023-10-16T11:19:03.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25800c0b0 con 0x7fe258004850 2023-10-16T11:19:03.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25800c270 con 0x7fe258003a00 2023-10-16T11:19:03.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe258005210 0x7fe25800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2709c1700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe258004850 0x7fe258004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2711c2700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258003a00 0x7fe258003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 -- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258003a00 msgr2=0x7fe258003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:03.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258003a00 0x7fe258003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 -- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe258004850 msgr2=0x7fe258004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:03.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe258004850 0x7fe258004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 -- 172.21.15.181:0/2105143853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580035b0 con 0x7fe258005210 2023-10-16T11:19:03.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.603+0000 7fe2719c3700 1 -- 172.21.15.181:0/2105143853 reap_dead start 2023-10-16T11:19:03.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe2719c3700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe258005210 0x7fe25800ba10 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fe264009fc0 tx=0x7fe258005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b94bd6af5fbb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:03.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264003a20 con 0x7fe258005210 2023-10-16T11:19:03.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26400f040 con 0x7fe258005210 2023-10-16T11:19:03.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe263fff700 1 -- 172.21.15.181:0/2105143853 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264003de0 con 0x7fe258005210 2023-10-16T11:19:03.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe273426700 1 -- 172.21.15.181:0/2105143853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe258005210 msgr2=0x7fe25800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe273426700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe258005210 0x7fe25800ba10 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fe264009fc0 tx=0x7fe258005880 comp rx=0 tx=0).stop 2023-10-16T11:19:03.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe273426700 1 -- 172.21.15.181:0/2105143853 shutdown_connections 2023-10-16T11:19:03.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe273426700 1 --2- 172.21.15.181:0/2105143853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe258005210 0x7fe25800ba10 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe273426700 1 -- 172.21.15.181:0/2105143853 >> 172.21.15.181:0/2105143853 conn(0x7fe26c0cd010 msgr2=0x7fe26c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:03.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe273426700 1 -- 172.21.15.181:0/2105143853 shutdown_connections 2023-10-16T11:19:03.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.604+0000 7fe273426700 1 -- 172.21.15.181:0/2105143853 wait complete. 2023-10-16T11:19:03.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.606+0000 7fe273426700 1 Processor -- start 2023-10-16T11:19:03.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.606+0000 7fe273426700 1 -- start start 2023-10-16T11:19:03.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe273426700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258005210 0x7fe26c0c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe273426700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe26c153400 0x7fe26c0c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe273426700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe26c0c9db0 0x7fe26c143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe273426700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c0ca230 con 0x7fe26c153400 2023-10-16T11:19:03.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe273426700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c147320 con 0x7fe26c0c9db0 2023-10-16T11:19:03.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe273426700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c153880 con 0x7fe258005210 2023-10-16T11:19:03.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe26c153400 0x7fe26c0c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe26c153400 0x7fe26c0c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54516/0 (socket says 172.21.15.181:54516) 2023-10-16T11:19:03.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 -- 172.21.15.181:0/2791994 learned_addr learned my addr 172.21.15.181:0/2791994 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:03.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2711c2700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258005210 0x7fe26c0c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2719c3700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe26c0c9db0 0x7fe26c143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 -- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258005210 msgr2=0x7fe26c0c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258005210 0x7fe26c0c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 -- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe26c0c9db0 msgr2=0x7fe26c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:03.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe26c0c9db0 0x7fe26c143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:03.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 -- 172.21.15.181:0/2791994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c144440 con 0x7fe26c153400 2023-10-16T11:19:03.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.607+0000 7fe2709c1700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe26c153400 0x7fe26c0c9870 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fe2680072c0 tx=0x7fe2680075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:03.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.608+0000 7fe261ffb700 1 -- 172.21.15.181:0/2791994 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2680077b0 con 0x7fe26c153400 2023-10-16T11:19:03.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.608+0000 7fe261ffb700 1 -- 172.21.15.181:0/2791994 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26800f8d0 con 0x7fe26c153400 2023-10-16T11:19:03.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.608+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c144160 con 0x7fe26c153400 2023-10-16T11:19:03.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.608+0000 7fe261ffb700 1 -- 172.21.15.181:0/2791994 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26800fc80 con 0x7fe26c153400 2023-10-16T11:19:03.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.608+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c152c00 con 0x7fe26c153400 2023-10-16T11:19:03.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.609+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c152780 con 0x7fe26c153400 2023-10-16T11:19:03.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.609+0000 7fe261ffb700 1 -- 172.21.15.181:0/2791994 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 134) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe26801e3b0 con 0x7fe26c153400 2023-10-16T11:19:03.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.610+0000 7fe261ffb700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fe25006b190 0x7fe25006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:03.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.610+0000 7fe261ffb700 1 -- 172.21.15.181:0/2791994 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6635+0+0 (secure 0 0 0) 0x7fe2680899f0 con 0x7fe26c153400 2023-10-16T11:19:03.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.610+0000 7fe2711c2700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fe25006b190 0x7fe25006d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:03.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.610+0000 7fe2711c2700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fe25006b190 0x7fe25006d650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe25c007fd0 tx=0x7fe25c009040 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:03.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.614+0000 7fe261ffb700 1 -- 172.21.15.181:0/2791994 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe26c152780 con 0x7fe26c153400 2023-10-16T11:19:03.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:03.867+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe26c152780 con 0x7fe26c153400 2023-10-16T11:19:04.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.013+0000 7fe261ffb700 1 -- 172.21.15.181:0/2791994 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v135) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe26c152780 con 0x7fe26c153400 2023-10-16T11:19:04.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.016+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fe25006b190 msgr2=0x7fe25006d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.016+0000 7fe273426700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fe25006b190 0x7fe25006d650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe25c007fd0 tx=0x7fe25c009040 comp rx=0 tx=0).stop 2023-10-16T11:19:04.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.016+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe26c153400 msgr2=0x7fe26c0c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.016+0000 7fe273426700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe26c153400 0x7fe26c0c9870 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fe2680072c0 tx=0x7fe2680075d0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 shutdown_connections 2023-10-16T11:19:04.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fe25006b190 0x7fe25006d650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe258005210 0x7fe26c0c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe26c0c9db0 0x7fe26c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 --2- 172.21.15.181:0/2791994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe26c153400 0x7fe26c0c9870 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 >> 172.21.15.181:0/2791994 conn(0x7fe26c0cd010 msgr2=0x7fe26c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:04.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 shutdown_connections 2023-10-16T11:19:04.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.018+0000 7fe273426700 1 -- 172.21.15.181:0/2791994 wait complete. 2023-10-16T11:19:04.032 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:19:04.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:19:04.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:19:04.106 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:19:04.106 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:04.107 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:04.127 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:04.127 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:04.164 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:04.163+0000 7f4bd423a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:19:04.165 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:04.164+0000 7fbda211c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:19:04.181 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:04.230 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:04.229+0000 7fbda211c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:04.232 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:04.232+0000 7f4bd423a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:04.302 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:04.300+0000 7fbda211c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:19:04.308 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:04.307+0000 7f4bd423a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:19:04.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.344+0000 7f3a4a3f6700 1 Processor -- start 2023-10-16T11:19:04.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.344+0000 7f3a4a3f6700 1 -- start start 2023-10-16T11:19:04.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.345+0000 7f3a4a3f6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3a440d1e60 0x7f3a440d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.345+0000 7f3a4a3f6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a440b2f00 con 0x7f3a4415dc20 2023-10-16T11:19:04.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.345+0000 7f3a4a3f6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a440b31e0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.345+0000 7f3a4a3f6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440b3b80 con 0x7f3a440d1e60 2023-10-16T11:19:04.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.345+0000 7f3a43fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3a4415dc20 0x7f3a440d41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37428/0 (socket says 172.21.15.181:37428) 2023-10-16T11:19:04.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.345+0000 7f3a43fff700 1 -- 172.21.15.181:0/481271199 learned_addr learned my addr 172.21.15.181:0/481271199 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:04.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.345+0000 7f3a437fe700 1 --2- 172.21.15.181:0/481271199 >> v2:172.21.15.190:3300/0 conn(0x7f3a440d1e60 0x7f3a440d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090415428 0 0) 0x7f3a440b2f00 con 0x7f3a4415dc20 2023-10-16T11:19:04.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a2c0035b0 con 0x7f3a4415dc20 2023-10-16T11:19:04.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863792408 0 0) 0x7f3a440b31e0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a440b2f00 con 0x7f3a440b3fa0 2023-10-16T11:19:04.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621611213 0 0) 0x7f3a2c0035b0 con 0x7f3a4415dc20 2023-10-16T11:19:04.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a437fe700 1 -- 172.21.15.181:0/481271199 >> v1:172.21.15.190:6789/0 conn(0x7f3a440b3fa0 legacy=0x7f3a440b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a437fe700 1 -- 172.21.15.181:0/481271199 >> v1:172.21.15.181:6789/0 conn(0x7f3a4415dc20 legacy=0x7f3a440d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a437fe700 1 -- 172.21.15.181:0/481271199 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a440b34c0 con 0x7f3a440d1e60 2023-10-16T11:19:04.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a437fe700 1 --2- 172.21.15.181:0/481271199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a440d1e60 0x7f3a440d22e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009840 tx=0x7f3a38009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc03d754f069c43 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:04.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a340017f0 con 0x7f3a440d1e60 2023-10-16T11:19:04.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38004030 con 0x7f3a440d1e60 2023-10-16T11:19:04.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.346+0000 7f3a42ffd700 1 -- 172.21.15.181:0/481271199 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a380043a0 con 0x7f3a440d1e60 2023-10-16T11:19:04.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.347+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/481271199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a440d1e60 msgr2=0x7f3a440d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.347+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/481271199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a440d1e60 0x7f3a440d22e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009840 tx=0x7f3a38009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:04.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.347+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/481271199 shutdown_connections 2023-10-16T11:19:04.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.347+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/481271199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a440d1e60 0x7f3a440d22e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.347+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/481271199 >> 172.21.15.181:0/481271199 conn(0x7f3a4408af90 msgr2=0x7f3a440d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:04.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.347+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/481271199 shutdown_connections 2023-10-16T11:19:04.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.347+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/481271199 wait complete. 2023-10-16T11:19:04.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.348+0000 7f3a4a3f6700 1 Processor -- start 2023-10-16T11:19:04.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.348+0000 7f3a4a3f6700 1 -- start start 2023-10-16T11:19:04.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a4a3f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a440b3fa0 0x7f3a4415e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a4a3f6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a440d1e60 0x7f3a440cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a4a3f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a4415dc20 0x7f3a440cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a4a3f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a441545f0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a4a3f6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440b1ee0 con 0x7f3a4415dc20 2023-10-16T11:19:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a4a3f6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440cd750 con 0x7f3a440d1e60 2023-10-16T11:19:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a440b3fa0 0x7f3a4415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a440b3fa0 0x7f3a4415e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54588/0 (socket says 172.21.15.181:54588) 2023-10-16T11:19:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 -- 172.21.15.181:0/3351611585 learned_addr learned my addr 172.21.15.181:0/3351611585 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:04.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a437fe700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a440d1e60 0x7f3a440cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a48993700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a4415dc20 0x7f3a440cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 -- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a440d1e60 msgr2=0x7f3a440cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a440d1e60 0x7f3a440cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 -- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a4415dc20 msgr2=0x7f3a440cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a4415dc20 0x7f3a440cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.349+0000 7f3a43fff700 1 -- 172.21.15.181:0/3351611585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380094a0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.350+0000 7f3a43fff700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a440b3fa0 0x7f3a4415e640 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f3a34001150 tx=0x7f3a3400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.350+0000 7f3a40ff9700 1 -- 172.21.15.181:0/3351611585 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a340074e0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.350+0000 7f3a40ff9700 1 -- 172.21.15.181:0/3351611585 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a34007640 con 0x7f3a440b3fa0 2023-10-16T11:19:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.350+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a441539e0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.350+0000 7f3a40ff9700 1 -- 172.21.15.181:0/3351611585 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34005a70 con 0x7f3a440b3fa0 2023-10-16T11:19:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.350+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a44153fa0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.351+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a44154730 con 0x7f3a440b3fa0 2023-10-16T11:19:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.352+0000 7f3a40ff9700 1 -- 172.21.15.181:0/3351611585 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f3a34005bd0 con 0x7f3a440b3fa0 2023-10-16T11:19:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.352+0000 7f3a40ff9700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 0x7f3a3006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.352+0000 7f3a437fe700 1 -- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 msgr2=0x7f3a3006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:04.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.352+0000 7f3a40ff9700 1 -- 172.21.15.181:0/3351611585 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6635+0+0 (secure 0 0 0) 0x7f3a34089530 con 0x7f3a440b3fa0 2023-10-16T11:19:04.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.352+0000 7f3a437fe700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 0x7f3a3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:04.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.356+0000 7f3a40ff9700 1 -- 172.21.15.181:0/3351611585 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3a34054050 con 0x7f3a440b3fa0 2023-10-16T11:19:04.380 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:04.379+0000 7fbda211c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:19:04.393 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:04.392+0000 7f4bd423a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:19:04.520 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:04.519+0000 7fbda211c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:19:04.537 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:04.537+0000 7f4bd423a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:19:04.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.553+0000 7f3a437fe700 1 -- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 msgr2=0x7f3a3006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:04.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.553+0000 7f3a437fe700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 0x7f3a3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:04.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.556+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a44066960 con 0x7f3a440b3fa0 2023-10-16T11:19:04.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.557+0000 7f3a40ff9700 1 -- 172.21.15.181:0/3351611585 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+17970 (secure 0 0 0) 0x7f3a34017090 con 0x7f3a440b3fa0 2023-10-16T11:19:04.559 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:04.559 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":202,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:18:55.097682+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:17:37.851564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:17:38.394398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.7","mappings":[{"from":0,"to":2}]},{"pgid":"25.2d","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1746525923":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3361524667":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/1822518943":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6833/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6832/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3260777321":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3430010207":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/969172037":"2023-10-17T11:17:53.265498+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 msgr2=0x7f3a3006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 0x7f3a3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a440b3fa0 msgr2=0x7f3a4415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a440b3fa0 0x7f3a4415e640 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f3a34001150 tx=0x7f3a3400c820 comp rx=0 tx=0).stop 2023-10-16T11:19:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 shutdown_connections 2023-10-16T11:19:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f3a3006b140 0x7f3a3006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a4415dc20 0x7f3a440cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a440b3fa0 0x7f3a4415e640 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 --2- 172.21.15.181:0/3351611585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a440d1e60 0x7f3a440cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 >> 172.21.15.181:0/3351611585 conn(0x7f3a4408af90 msgr2=0x7f3a4406d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 shutdown_connections 2023-10-16T11:19:04.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.560+0000 7f3a4a3f6700 1 -- 172.21.15.181:0/3351611585 wait complete. 2023-10-16T11:19:04.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:17:37.851564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:17:38.394398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:19:04.574 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:19:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.738+0000 7efe6142b700 1 Processor -- start 2023-10-16T11:19:04.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.738+0000 7efe6142b700 1 -- start start 2023-10-16T11:19:04.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe6142b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efe5c15f150 0x7efe5c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe6142b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5c0b2ae0 con 0x7efe5c0cd170 2023-10-16T11:19:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe6142b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5c0b2dc0 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe6142b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5c0ce9a0 con 0x7efe5c15f150 2023-10-16T11:19:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe5b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efe5c0cd170 0x7efe5c0b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37482/0 (socket says 172.21.15.181:37482) 2023-10-16T11:19:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe5b7fe700 1 -- 172.21.15.181:0/318575417 learned_addr learned my addr 172.21.15.181:0/318575417 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe5affd700 1 --2- 172.21.15.181:0/318575417 >> v2:172.21.15.190:3300/0 conn(0x7efe5c15f150 0x7efe5c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe59ffb700 1 -- 172.21.15.181:0/318575417 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954011044 0 0) 0x7efe5c0b2ae0 con 0x7efe5c0cd170 2023-10-16T11:19:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe59ffb700 1 -- 172.21.15.181:0/318575417 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe440035b0 con 0x7efe5c0cd170 2023-10-16T11:19:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe5affd700 1 -- 172.21.15.181:0/318575417 >> v1:172.21.15.190:6789/0 conn(0x7efe5c0ce4f0 legacy=0x7efe5c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe5affd700 1 -- 172.21.15.181:0/318575417 >> v1:172.21.15.181:6789/0 conn(0x7efe5c0cd170 legacy=0x7efe5c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.740+0000 7efe5affd700 1 -- 172.21.15.181:0/318575417 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0cf9d0 con 0x7efe5c15f150 2023-10-16T11:19:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe5affd700 1 --2- 172.21.15.181:0/318575417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 0x7efe5c0cdfb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009870 tx=0x7efe4c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd3d7bcf2a8997 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe59ffb700 1 -- 172.21.15.181:0/318575417 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011830 con 0x7efe5c15f150 2023-10-16T11:19:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe59ffb700 1 -- 172.21.15.181:0/318575417 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c011990 con 0x7efe5c15f150 2023-10-16T11:19:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe59ffb700 1 -- 172.21.15.181:0/318575417 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011d00 con 0x7efe5c15f150 2023-10-16T11:19:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe6142b700 1 -- 172.21.15.181:0/318575417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 msgr2=0x7efe5c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe6142b700 1 --2- 172.21.15.181:0/318575417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 0x7efe5c0cdfb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009870 tx=0x7efe4c009b80 comp rx=0 tx=0).stop 2023-10-16T11:19:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe6142b700 1 -- 172.21.15.181:0/318575417 shutdown_connections 2023-10-16T11:19:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe6142b700 1 --2- 172.21.15.181:0/318575417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 0x7efe5c0cdfb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe6142b700 1 -- 172.21.15.181:0/318575417 >> 172.21.15.181:0/318575417 conn(0x7efe5c09a930 msgr2=0x7efe5c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe6142b700 1 -- 172.21.15.181:0/318575417 shutdown_connections 2023-10-16T11:19:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.741+0000 7efe6142b700 1 -- 172.21.15.181:0/318575417 wait complete. 2023-10-16T11:19:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.743+0000 7efe6142b700 1 Processor -- start 2023-10-16T11:19:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.743+0000 7efe6142b700 1 -- start start 2023-10-16T11:19:04.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe6142b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe5c0cd170 0x7efe5c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe6142b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c0ce4f0 0x7efe5c0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe6142b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 0x7efe5c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe6142b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c157930 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe6142b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c156d20 con 0x7efe5c15f150 2023-10-16T11:19:04.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe6142b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c157000 con 0x7efe5c0cd170 2023-10-16T11:19:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe5a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c0ce4f0 0x7efe5c0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe5a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c0ce4f0 0x7efe5c0c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54640/0 (socket says 172.21.15.181:54640) 2023-10-16T11:19:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe5a7fc700 1 -- 172.21.15.181:0/202918791 learned_addr learned my addr 172.21.15.181:0/202918791 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe5b7fe700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 0x7efe5c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.744+0000 7efe5affd700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe5c0cd170 0x7efe5c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe5a7fc700 1 -- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe5c0cd170 msgr2=0x7efe5c151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe5a7fc700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe5c0cd170 0x7efe5c151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe5a7fc700 1 -- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 msgr2=0x7efe5c1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe5a7fc700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 0x7efe5c1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe5a7fc700 1 -- 172.21.15.181:0/202918791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0094a0 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe5a7fc700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c0ce4f0 0x7efe5c0c86d0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7efe5000c4e0 tx=0x7efe5000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe437fe700 1 -- 172.21.15.181:0/202918791 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500052d0 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe437fe700 1 -- 172.21.15.181:0/202918791 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe50005430 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c1572e0 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe437fe700 1 -- 172.21.15.181:0/202918791 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500038c0 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.745+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c151b90 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.746+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c09ae40 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.746+0000 7efe437fe700 1 -- 172.21.15.181:0/202918791 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7efe50003a20 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.747+0000 7efe437fe700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 0x7efe4806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.747+0000 7efe5affd700 1 -- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 msgr2=0x7efe4806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:04.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.747+0000 7efe437fe700 1 -- 172.21.15.181:0/202918791 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6635+0+0 (secure 0 0 0) 0x7efe50089070 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.747+0000 7efe5affd700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 0x7efe4806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:04.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.751+0000 7efe437fe700 1 -- 172.21.15.181:0/202918791 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efe50053af0 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.842 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:04.840+0000 7fbda211c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:19:04.977 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:04.877+0000 7f4bd423a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:19:04.977 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:04.955+0000 7fbda211c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:19:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.948+0000 7efe5affd700 1 -- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 msgr2=0x7efe4806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.948+0000 7efe5affd700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 0x7efe4806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.950+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe5c067d20 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.950+0000 7efe437fe700 1 -- 172.21.15.181:0/202918791 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v202) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe50057110 con 0x7efe5c0ce4f0 2023-10-16T11:19:04.979 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:19:04.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 msgr2=0x7efe4806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:04.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 0x7efe4806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c0ce4f0 msgr2=0x7efe5c0c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:04.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c0ce4f0 0x7efe5c0c86d0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7efe5000c4e0 tx=0x7efe5000c7f0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 shutdown_connections 2023-10-16T11:19:04.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efe4806b140 0x7efe4806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe5c0cd170 0x7efe5c151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe5c15f150 0x7efe5c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 --2- 172.21.15.181:0/202918791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c0ce4f0 0x7efe5c0c86d0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:04.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 >> 172.21.15.181:0/202918791 conn(0x7efe5c09a930 msgr2=0x7efe5c0ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:04.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 shutdown_connections 2023-10-16T11:19:04.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:04.952+0000 7efe6142b700 1 -- 172.21.15.181:0/202918791 wait complete. 2023-10-16T11:19:04.983 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:19:05.002 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:05.001+0000 7f4bd423a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:19:05.033 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:05.032+0000 7fbda211c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:19:05.088 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:05.086+0000 7f4bd423a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:19:05.118 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:05.116+0000 7fbda211c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:19:05.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.117+0000 7f8a184ce700 1 Processor -- start 2023-10-16T11:19:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.117+0000 7f8a184ce700 1 -- start start 2023-10-16T11:19:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a184ce700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a100d01d0 0x7f8a100ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a184ce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1015ea40 0x7f8a100d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a184ce700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8a10161530 0x7f8a101619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a184ce700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a100d4290 con 0x7f8a100d01d0 2023-10-16T11:19:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a184ce700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a100cf580 con 0x7f8a10161530 2023-10-16T11:19:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a184ce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a100d0af0 con 0x7f8a1015ea40 2023-10-16T11:19:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a100d01d0 0x7f8a100ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a100d01d0 0x7f8a100ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54648/0 (socket says 172.21.15.181:54648) 2023-10-16T11:19:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 -- 172.21.15.181:0/4021651389 learned_addr learned my addr 172.21.15.181:0/4021651389 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a15a69700 1 --2- 172.21.15.181:0/4021651389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1015ea40 0x7f8a100d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a16a6b700 1 --2- 172.21.15.181:0/4021651389 >> v2:172.21.15.190:3300/0 conn(0x7f8a10161530 0x7f8a101619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 -- 172.21.15.181:0/4021651389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1015ea40 msgr2=0x7f8a100d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:05.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 --2- 172.21.15.181:0/4021651389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1015ea40 0x7f8a100d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 -- 172.21.15.181:0/4021651389 >> v2:172.21.15.190:3300/0 conn(0x7f8a10161530 msgr2=0x7f8a101619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 --2- 172.21.15.181:0/4021651389 >> v2:172.21.15.190:3300/0 conn(0x7f8a10161530 0x7f8a101619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 -- 172.21.15.181:0/4021651389 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a100ce970 con 0x7f8a100d01d0 2023-10-16T11:19:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a15a69700 1 --2- 172.21.15.181:0/4021651389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1015ea40 0x7f8a100d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:05.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a16a6b700 1 --2- 172.21.15.181:0/4021651389 >> v2:172.21.15.190:3300/0 conn(0x7f8a10161530 0x7f8a101619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a1626a700 1 --2- 172.21.15.181:0/4021651389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a100d01d0 0x7f8a100ce2d0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f8a1008daa0 tx=0x7f8a0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ca86160d28c1e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.118+0000 7f8a15268700 1 -- 172.21.15.181:0/4021651389 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00b870 con 0x7f8a100d01d0 2023-10-16T11:19:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a15268700 1 -- 172.21.15.181:0/4021651389 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c00b9d0 con 0x7f8a100d01d0 2023-10-16T11:19:05.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a15268700 1 -- 172.21.15.181:0/4021651389 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00bd60 con 0x7f8a100d01d0 2023-10-16T11:19:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 -- 172.21.15.181:0/4021651389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a100d01d0 msgr2=0x7f8a100ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 --2- 172.21.15.181:0/4021651389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a100d01d0 0x7f8a100ce2d0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f8a1008daa0 tx=0x7f8a0c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 -- 172.21.15.181:0/4021651389 shutdown_connections 2023-10-16T11:19:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 --2- 172.21.15.181:0/4021651389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a100d01d0 0x7f8a100ce2d0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 --2- 172.21.15.181:0/4021651389 >> v2:172.21.15.190:3300/0 conn(0x7f8a10161530 0x7f8a101619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 --2- 172.21.15.181:0/4021651389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1015ea40 0x7f8a100d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 -- 172.21.15.181:0/4021651389 >> 172.21.15.181:0/4021651389 conn(0x7f8a1009cb60 msgr2=0x7f8a100d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 -- 172.21.15.181:0/4021651389 shutdown_connections 2023-10-16T11:19:05.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.119+0000 7f8a184ce700 1 -- 172.21.15.181:0/4021651389 wait complete. 2023-10-16T11:19:05.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.120+0000 7f8a184ce700 1 Processor -- start 2023-10-16T11:19:05.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a184ce700 1 -- start start 2023-10-16T11:19:05.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a184ce700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a100d01d0 0x7f8a100b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a184ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1015ea40 0x7f8a101f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a184ce700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a10161530 0x7f8a100c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a184ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a101f3470 con 0x7f8a1015ea40 2023-10-16T11:19:05.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a184ce700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a100c7d70 con 0x7f8a10161530 2023-10-16T11:19:05.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a184ce700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a101f3710 con 0x7f8a100d01d0 2023-10-16T11:19:05.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a1626a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a100d01d0 0x7f8a100b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a15a69700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1015ea40 0x7f8a101f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a1626a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a100d01d0 0x7f8a100b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:51680/0 (socket says 172.21.15.181:51680) 2023-10-16T11:19:05.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a1626a700 1 -- 172.21.15.181:0/2228293124 learned_addr learned my addr 172.21.15.181:0/2228293124 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:05.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.121+0000 7f8a16a6b700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a10161530 0x7f8a100c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a1626a700 1 -- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a10161530 msgr2=0x7f8a100c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a1626a700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a10161530 0x7f8a100c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a1626a700 1 -- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1015ea40 msgr2=0x7f8a101f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a1626a700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1015ea40 0x7f8a101f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a1626a700 1 -- 172.21.15.181:0/2228293124 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c009550 con 0x7f8a100d01d0 2023-10-16T11:19:05.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a15a69700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1015ea40 0x7f8a101f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:05.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a1626a700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a100d01d0 0x7f8a100b5930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000c00 tx=0x7f8a0c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:05.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a06ffd700 1 -- 172.21.15.181:0/2228293124 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c004310 con 0x7f8a100d01d0 2023-10-16T11:19:05.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a06ffd700 1 -- 172.21.15.181:0/2228293124 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c01e070 con 0x7f8a100d01d0 2023-10-16T11:19:05.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a101f39f0 con 0x7f8a100d01d0 2023-10-16T11:19:05.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a06ffd700 1 -- 172.21.15.181:0/2228293124 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0036a0 con 0x7f8a100d01d0 2023-10-16T11:19:05.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.122+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a100b2990 con 0x7f8a100d01d0 2023-10-16T11:19:05.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.123+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1004ebb0 con 0x7f8a100d01d0 2023-10-16T11:19:05.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.124+0000 7f8a06ffd700 1 -- 172.21.15.181:0/2228293124 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8a0c020030 con 0x7f8a100d01d0 2023-10-16T11:19:05.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.124+0000 7f8a06ffd700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 0x7f89f406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.124+0000 7f8a15a69700 1 -- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 msgr2=0x7f89f406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:05.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.124+0000 7f8a06ffd700 1 -- 172.21.15.181:0/2228293124 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f8a0c08abb0 con 0x7f8a100d01d0 2023-10-16T11:19:05.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.124+0000 7f8a15a69700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 0x7f89f406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:05.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.129+0000 7f8a06ffd700 1 -- 172.21.15.181:0/2228293124 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a0c055660 con 0x7f8a100d01d0 2023-10-16T11:19:05.171 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:05.170+0000 7f4bd423a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:19:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.325+0000 7f8a15a69700 1 -- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 msgr2=0x7f89f406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:05.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.325+0000 7f8a15a69700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 0x7f89f406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.328+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a100d0e00 con 0x7f8a100d01d0 2023-10-16T11:19:05.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.328+0000 7f8a06ffd700 1 -- 172.21.15.181:0/2228293124 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v203) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a0c003800 con 0x7f8a100d01d0 2023-10-16T11:19:05.329 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:19:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 msgr2=0x7f89f406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 0x7f89f406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a100d01d0 msgr2=0x7f8a100b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a100d01d0 0x7f8a100b5930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000c00 tx=0x7f8a0c004750 comp rx=0 tx=0).stop 2023-10-16T11:19:05.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 shutdown_connections 2023-10-16T11:19:05.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f89f406b140 0x7f89f406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a100d01d0 0x7f8a100b5930 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a10161530 0x7f8a100c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 --2- 172.21.15.181:0/2228293124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a1015ea40 0x7f8a101f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 >> 172.21.15.181:0/2228293124 conn(0x7f8a1009cb60 msgr2=0x7f8a101619c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:05.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.330+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 shutdown_connections 2023-10-16T11:19:05.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.331+0000 7f8a184ce700 1 -- 172.21.15.181:0/2228293124 wait complete. 2023-10-16T11:19:05.344 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:19:05.486 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:05.484+0000 7fbda211c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:19:05.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.511+0000 7fbe5414e700 1 Processor -- start 2023-10-16T11:19:05.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.512+0000 7fbe5414e700 1 -- start start 2023-10-16T11:19:05.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.512+0000 7fbe5414e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbe4c0cc6e0 0x7fbe4c071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.512+0000 7fbe5414e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe4c0ce680 0x7fbe4c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.512+0000 7fbe5414e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4c0ce1f0 con 0x7fbe4c0b2010 2023-10-16T11:19:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.512+0000 7fbe5414e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c0ccb90 con 0x7fbe4c0ce680 2023-10-16T11:19:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.512+0000 7fbe5414e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c072120 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe526eb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe4c0ce680 0x7fbe4c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe51eea700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbe4c0b2010 0x7fbe4c0b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37554/0 (socket says 172.21.15.181:37554) 2023-10-16T11:19:05.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe51eea700 1 -- 172.21.15.181:0/72668232 learned_addr learned my addr 172.21.15.181:0/72668232 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:05.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe516e9700 1 --2- 172.21.15.181:0/72668232 >> v2:172.21.15.190:3300/0 conn(0x7fbe4c0cc6e0 0x7fbe4c071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe50ee8700 1 -- 172.21.15.181:0/72668232 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110188831 0 0) 0x7fbe4c0ce1f0 con 0x7fbe4c0b2010 2023-10-16T11:19:05.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe50ee8700 1 -- 172.21.15.181:0/72668232 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe380035b0 con 0x7fbe4c0b2010 2023-10-16T11:19:05.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe526eb700 1 -- 172.21.15.181:0/72668232 >> v2:172.21.15.190:3300/0 conn(0x7fbe4c0cc6e0 msgr2=0x7fbe4c071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:05.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe526eb700 1 --2- 172.21.15.181:0/72668232 >> v2:172.21.15.190:3300/0 conn(0x7fbe4c0cc6e0 0x7fbe4c071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe526eb700 1 -- 172.21.15.181:0/72668232 >> v1:172.21.15.181:6789/0 conn(0x7fbe4c0b2010 legacy=0x7fbe4c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe526eb700 1 -- 172.21.15.181:0/72668232 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c0ccda0 con 0x7fbe4c0ce680 2023-10-16T11:19:05.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe516e9700 1 --2- 172.21.15.181:0/72668232 >> v2:172.21.15.190:3300/0 conn(0x7fbe4c0cc6e0 0x7fbe4c071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe526eb700 1 --2- 172.21.15.181:0/72668232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0ceaf0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c15e1c0 tx=0x7fbe4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488ea960f9eaea10 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe50ee8700 1 -- 172.21.15.181:0/72668232 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400e070 con 0x7fbe4c0ce680 2023-10-16T11:19:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe50ee8700 1 -- 172.21.15.181:0/72668232 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4400bb20 con 0x7fbe4c0ce680 2023-10-16T11:19:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.513+0000 7fbe50ee8700 1 -- 172.21.15.181:0/72668232 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400be90 con 0x7fbe4c0ce680 2023-10-16T11:19:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 -- 172.21.15.181:0/72668232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0ce680 msgr2=0x7fbe4c0ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 --2- 172.21.15.181:0/72668232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0ceaf0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c15e1c0 tx=0x7fbe4400b330 comp rx=0 tx=0).stop 2023-10-16T11:19:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 -- 172.21.15.181:0/72668232 shutdown_connections 2023-10-16T11:19:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 --2- 172.21.15.181:0/72668232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0ceaf0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 --2- 172.21.15.181:0/72668232 >> v2:172.21.15.190:3300/0 conn(0x7fbe4c0cc6e0 0x7fbe4c071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 -- 172.21.15.181:0/72668232 >> 172.21.15.181:0/72668232 conn(0x7fbe4c0cd010 msgr2=0x7fbe4c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 -- 172.21.15.181:0/72668232 shutdown_connections 2023-10-16T11:19:05.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.514+0000 7fbe5414e700 1 -- 172.21.15.181:0/72668232 wait complete. 2023-10-16T11:19:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.515+0000 7fbe5414e700 1 Processor -- start 2023-10-16T11:19:05.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.515+0000 7fbe5414e700 1 -- start start 2023-10-16T11:19:05.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe5414e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe4c0b2010 0x7fbe4c151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe5414e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0cc6e0 0x7fbe4c14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe5414e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe5414e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c1f5410 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe5414e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c0ca540 con 0x7fbe4c0ce680 2023-10-16T11:19:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe5414e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c0c88e0 con 0x7fbe4c0b2010 2023-10-16T11:19:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0cc6e0 0x7fbe4c14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0cc6e0 0x7fbe4c14fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54716/0 (socket says 172.21.15.181:54716) 2023-10-16T11:19:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 -- 172.21.15.181:0/687348298 learned_addr learned my addr 172.21.15.181:0/687348298 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe51eea700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe4c0b2010 0x7fbe4c151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe526eb700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 -- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe4c0b2010 msgr2=0x7fbe4c151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe4c0b2010 0x7fbe4c151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 -- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe4c0ce680 msgr2=0x7fbe4c0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe516e9700 1 -- 172.21.15.181:0/687348298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe44009a70 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe51eea700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe4c0b2010 0x7fbe4c151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:05.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.516+0000 7fbe526eb700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:05.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.517+0000 7fbe516e9700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0cc6e0 0x7fbe4c14fa90 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00e810 tx=0x7fbe3c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:05.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.517+0000 7fbe427fc700 1 -- 172.21.15.181:0/687348298 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00c020 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.517+0000 7fbe427fc700 1 -- 172.21.15.181:0/687348298 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c004250 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.517+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c0c9f80 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.517+0000 7fbe427fc700 1 -- 172.21.15.181:0/687348298 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0036a0 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.517+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c14ffd0 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.518+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4c0c86b0 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.518+0000 7fbe427fc700 1 -- 172.21.15.181:0/687348298 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fbe3c003800 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.518+0000 7fbe427fc700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 0x7fbe3006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.518+0000 7fbe51eea700 1 -- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 msgr2=0x7fbe3006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:05.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.519+0000 7fbe427fc700 1 -- 172.21.15.181:0/687348298 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fbe3c089cd0 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.519+0000 7fbe51eea700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 0x7fbe3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:05.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.523+0000 7fbe427fc700 1 -- 172.21.15.181:0/687348298 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbe4c0c86b0 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.558 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:05.557+0000 7f4bd423a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:19:05.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.720+0000 7fbe51eea700 1 -- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 msgr2=0x7fbe3006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:05.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.720+0000 7fbe51eea700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 0x7fbe3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:05.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.724+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe4c0c86b0 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.724+0000 7fbe427fc700 1 -- 172.21.15.181:0/687348298 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v203) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe4c0c86b0 con 0x7fbe4c0cc6e0 2023-10-16T11:19:05.725 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:19:05.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 msgr2=0x7fbe3006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:05.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 0x7fbe3006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0cc6e0 msgr2=0x7fbe4c14fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0cc6e0 0x7fbe4c14fa90 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00e810 tx=0x7fbe3c00ebd0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 shutdown_connections 2023-10-16T11:19:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fbe3006b140 0x7fbe3006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe4c0b2010 0x7fbe4c151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe4c0ce680 0x7fbe4c0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 --2- 172.21.15.181:0/687348298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe4c0cc6e0 0x7fbe4c14fa90 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 >> 172.21.15.181:0/687348298 conn(0x7fbe4c0cd010 msgr2=0x7fbe4c0b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:05.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 shutdown_connections 2023-10-16T11:19:05.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.726+0000 7fbe5414e700 1 -- 172.21.15.181:0/687348298 wait complete. 2023-10-16T11:19:05.740 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:19:05.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.863+0000 7f4bad77d700 1 Processor -- start 2023-10-16T11:19:05.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.863+0000 7f4bad77d700 1 -- start start 2023-10-16T11:19:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4bad77d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba80cb110 0x7f4ba80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4bad77d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ba815a0f0 0x7f4ba815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4bad77d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba806ed00 con 0x7f4ba815e460 2023-10-16T11:19:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4bad77d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba806e0f0 con 0x7f4ba815a0f0 2023-10-16T11:19:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4bad77d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba806e3d0 con 0x7f4ba80cb110 2023-10-16T11:19:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4ba6ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4ba815e460 0x7f4ba815e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37570/0 (socket says 172.21.15.181:37570) 2023-10-16T11:19:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4ba6ffd700 1 -- 172.21.15.181:0/314478913 learned_addr learned my addr 172.21.15.181:0/314478913 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4ba77fe700 1 --2- 172.21.15.181:0/314478913 >> v2:172.21.15.181:3300/0 conn(0x7f4ba815a0f0 0x7f4ba815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.864+0000 7f4ba67fc700 1 --2- 172.21.15.181:0/314478913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba80cb110 0x7f4ba80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba5ffb700 1 -- 172.21.15.181:0/314478913 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023225036 0 0) 0x7f4ba806ed00 con 0x7f4ba815e460 2023-10-16T11:19:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba5ffb700 1 -- 172.21.15.181:0/314478913 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b900035b0 con 0x7f4ba815e460 2023-10-16T11:19:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba77fe700 1 -- 172.21.15.181:0/314478913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba80cb110 msgr2=0x7f4ba80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:05.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba77fe700 1 --2- 172.21.15.181:0/314478913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba80cb110 0x7f4ba80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba77fe700 1 -- 172.21.15.181:0/314478913 >> v1:172.21.15.181:6789/0 conn(0x7f4ba815e460 legacy=0x7f4ba815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba77fe700 1 -- 172.21.15.181:0/314478913 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba806e6b0 con 0x7f4ba815a0f0 2023-10-16T11:19:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba77fe700 1 --2- 172.21.15.181:0/314478913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba815a5a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009dc0 tx=0x7f4ba000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6190dcf854f47402 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:05.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba5ffb700 1 -- 172.21.15.181:0/314478913 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000e070 con 0x7f4ba815a0f0 2023-10-16T11:19:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba5ffb700 1 -- 172.21.15.181:0/314478913 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba000bae0 con 0x7f4ba815a0f0 2023-10-16T11:19:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.865+0000 7f4ba5ffb700 1 -- 172.21.15.181:0/314478913 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000be50 con 0x7f4ba815a0f0 2023-10-16T11:19:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 -- 172.21.15.181:0/314478913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 msgr2=0x7f4ba815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 --2- 172.21.15.181:0/314478913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba815a5a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009dc0 tx=0x7f4ba000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:05.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 -- 172.21.15.181:0/314478913 shutdown_connections 2023-10-16T11:19:05.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 --2- 172.21.15.181:0/314478913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba815a5a0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 --2- 172.21.15.181:0/314478913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba80cb110 0x7f4ba80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 -- 172.21.15.181:0/314478913 >> 172.21.15.181:0/314478913 conn(0x7f4ba80d04a0 msgr2=0x7f4ba80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:05.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 -- 172.21.15.181:0/314478913 shutdown_connections 2023-10-16T11:19:05.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.866+0000 7f4bad77d700 1 -- 172.21.15.181:0/314478913 wait complete. 2023-10-16T11:19:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.868+0000 7f4bad77d700 1 Processor -- start 2023-10-16T11:19:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.868+0000 7f4bad77d700 1 -- start start 2023-10-16T11:19:05.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.869+0000 7f4bad77d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba80cb110 0x7f4ba81593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.869+0000 7f4bad77d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba80b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.869+0000 7f4bad77d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba815e460 0x7f4ba80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.869+0000 7f4bad77d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b20f0 con 0x7f4ba815a0f0 2023-10-16T11:19:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.869+0000 7f4bad77d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b14e0 con 0x7f4ba80cb110 2023-10-16T11:19:05.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.869+0000 7f4bad77d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b17c0 con 0x7f4ba815e460 2023-10-16T11:19:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba80b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba80b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54732/0 (socket says 172.21.15.181:54732) 2023-10-16T11:19:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 -- 172.21.15.181:0/3870698223 learned_addr learned my addr 172.21.15.181:0/3870698223 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba6ffd700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba80cb110 0x7f4ba81593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba77fe700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba815e460 0x7f4ba80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 -- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba815e460 msgr2=0x7f4ba80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba815e460 0x7f4ba80b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 -- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba80cb110 msgr2=0x7f4ba81593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba80cb110 0x7f4ba81593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:05.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 -- 172.21.15.181:0/3870698223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0009a70 con 0x7f4ba815a0f0 2023-10-16T11:19:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4ba67fc700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba80b2580 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00e870 tx=0x7f4b9c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4b8f7fe700 1 -- 172.21.15.181:0/3870698223 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c009590 con 0x7f4ba815a0f0 2023-10-16T11:19:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80b1aa0 con 0x7f4ba815a0f0 2023-10-16T11:19:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4b8f7fe700 1 -- 172.21.15.181:0/3870698223 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c013680 con 0x7f4ba815a0f0 2023-10-16T11:19:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4b8f7fe700 1 -- 172.21.15.181:0/3870698223 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0139f0 con 0x7f4ba815a0f0 2023-10-16T11:19:05.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.870+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba8159930 con 0x7f4ba815a0f0 2023-10-16T11:19:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.872+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba8067d20 con 0x7f4ba815a0f0 2023-10-16T11:19:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.872+0000 7f4b8f7fe700 1 -- 172.21.15.181:0/3870698223 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f4b9c013b50 con 0x7f4ba815a0f0 2023-10-16T11:19:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.872+0000 7f4b8f7fe700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 0x7f4b9406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.872+0000 7f4ba6ffd700 1 -- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 msgr2=0x7f4b9406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:05.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.873+0000 7f4b8f7fe700 1 -- 172.21.15.181:0/3870698223 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f4b9c089da0 con 0x7f4ba815a0f0 2023-10-16T11:19:05.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.873+0000 7f4ba6ffd700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 0x7f4b9406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:05.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:05.877+0000 7f4b8f7fe700 1 -- 172.21.15.181:0/3870698223 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4b9c054660 con 0x7f4ba815a0f0 2023-10-16T11:19:06.053 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:06.052+0000 7fbda211c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:19:06.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.074+0000 7f4ba6ffd700 1 -- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 msgr2=0x7f4b9406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:06.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.074+0000 7f4ba6ffd700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 0x7f4b9406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:06.127 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:06.125+0000 7fbda211c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:19:06.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.135+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ba80b2230 con 0x7f4ba815a0f0 2023-10-16T11:19:06.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.135+0000 7f4b8f7fe700 1 -- 172.21.15.181:0/3870698223 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v204) v1 ==== 80+0+477 (secure 0 0 0) 0x7f4b9c0211f0 con 0x7f4ba815a0f0 2023-10-16T11:19:06.137 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1746525923 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.137 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3361524667 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.137 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1822518943 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.137 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/1532329351 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.137 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/1532329351 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.138 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3260777321 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.138 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3430010207 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.138 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/969172037 2023-10-17T11:17:53.265498+0000 2023-10-16T11:19:06.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 msgr2=0x7f4b9406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:06.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 0x7f4b9406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 msgr2=0x7f4ba80b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba80b2580 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00e870 tx=0x7f4b9c00eb80 comp rx=0 tx=0).stop 2023-10-16T11:19:06.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 shutdown_connections 2023-10-16T11:19:06.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f4b9406b140 0x7f4b9406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ba815e460 0x7f4ba80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ba80cb110 0x7f4ba81593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 --2- 172.21.15.181:0/3870698223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba815a0f0 0x7f4ba80b2580 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 >> 172.21.15.181:0/3870698223 conn(0x7f4ba80d04a0 msgr2=0x7f4ba815e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 shutdown_connections 2023-10-16T11:19:06.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.138+0000 7f4bad77d700 1 -- 172.21.15.181:0/3870698223 wait complete. 2023-10-16T11:19:06.142 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T11:19:06.152 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:19:06.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:06.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:19:06.160 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:06.159+0000 7f4bd423a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:19:06.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:19:06.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:06.194 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:19:06.234 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:06.233+0000 7f4bd423a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:19:06.250 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:19:06.251 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:06.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:19:06.268 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:06.268 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:06.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:19:06.308 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:19:06.309 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:06.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:19:06.326 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:06.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.470+0000 7fba8bc1b700 1 Processor -- start 2023-10-16T11:19:06.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.470+0000 7fba8bc1b700 1 -- start start 2023-10-16T11:19:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba8bc1b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba8415e6d0 0x7fba840d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba8bc1b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fba840cb0c0 0x7fba8415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba8bc1b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8406ed00 con 0x7fba8415eab0 2023-10-16T11:19:06.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba8bc1b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840d02d0 con 0x7fba8415e6d0 2023-10-16T11:19:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba8bc1b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8415ad40 con 0x7fba840cb0c0 2023-10-16T11:19:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba899b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba8415e6d0 0x7fba840d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba891b6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fba8415eab0 0x7fba8415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37586/0 (socket says 172.21.15.181:37586) 2023-10-16T11:19:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba891b6700 1 -- 172.21.15.181:0/582052899 learned_addr learned my addr 172.21.15.181:0/582052899 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:06.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba8a1b8700 1 --2- 172.21.15.181:0/582052899 >> v2:172.21.15.190:3300/0 conn(0x7fba840cb0c0 0x7fba8415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba899b7700 1 -- 172.21.15.181:0/582052899 >> v2:172.21.15.190:3300/0 conn(0x7fba840cb0c0 msgr2=0x7fba8415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba899b7700 1 --2- 172.21.15.181:0/582052899 >> v2:172.21.15.190:3300/0 conn(0x7fba840cb0c0 0x7fba8415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba899b7700 1 -- 172.21.15.181:0/582052899 >> v1:172.21.15.181:6789/0 conn(0x7fba8415eab0 legacy=0x7fba8415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba889b5700 1 -- 172.21.15.181:0/582052899 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185876404 0 0) 0x7fba8406ed00 con 0x7fba8415eab0 2023-10-16T11:19:06.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.471+0000 7fba899b7700 1 -- 172.21.15.181:0/582052899 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8406e0f0 con 0x7fba8415e6d0 2023-10-16T11:19:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba899b7700 1 --2- 172.21.15.181:0/582052899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba840d0fc0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fba84071650 tx=0x7fba74009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b213d1b99ff3ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba889b5700 1 -- 172.21.15.181:0/582052899 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74005e30 con 0x7fba8415e6d0 2023-10-16T11:19:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba889b5700 1 -- 172.21.15.181:0/582052899 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba74006030 con 0x7fba8415e6d0 2023-10-16T11:19:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba889b5700 1 -- 172.21.15.181:0/582052899 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740063c0 con 0x7fba8415e6d0 2023-10-16T11:19:06.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba8bc1b700 1 -- 172.21.15.181:0/582052899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 msgr2=0x7fba840d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/582052899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba840d0fc0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fba84071650 tx=0x7fba74009b00 comp rx=0 tx=0).stop 2023-10-16T11:19:06.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba8bc1b700 1 -- 172.21.15.181:0/582052899 shutdown_connections 2023-10-16T11:19:06.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/582052899 >> v2:172.21.15.190:3300/0 conn(0x7fba840cb0c0 0x7fba8415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/582052899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba840d0fc0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.472+0000 7fba8bc1b700 1 -- 172.21.15.181:0/582052899 >> 172.21.15.181:0/582052899 conn(0x7fba840d04a0 msgr2=0x7fba840d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:06.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.473+0000 7fba8bc1b700 1 -- 172.21.15.181:0/582052899 shutdown_connections 2023-10-16T11:19:06.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.473+0000 7fba8bc1b700 1 -- 172.21.15.181:0/582052899 wait complete. 2023-10-16T11:19:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 Processor -- start 2023-10-16T11:19:06.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 -- start start 2023-10-16T11:19:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba840cb0c0 0x7fba840b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba8415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8415eab0 0x7fba84151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840afee0 con 0x7fba8415e6d0 2023-10-16T11:19:06.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840b0a60 con 0x7fba8415eab0 2023-10-16T11:19:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.474+0000 7fba8bc1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840b0130 con 0x7fba840cb0c0 2023-10-16T11:19:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba8415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba8415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54746/0 (socket says 172.21.15.181:54746) 2023-10-16T11:19:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 -- 172.21.15.181:0/3281017466 learned_addr learned my addr 172.21.15.181:0/3281017466 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:06.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba899b7700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba840cb0c0 0x7fba840b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba8a1b8700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8415eab0 0x7fba84151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 -- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba840cb0c0 msgr2=0x7fba840b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba840cb0c0 0x7fba840b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 -- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8415eab0 msgr2=0x7fba84151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8415eab0 0x7fba84151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 -- 172.21.15.181:0/3281017466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba74009430 con 0x7fba8415e6d0 2023-10-16T11:19:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba891b6700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba8415b890 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fba8000c750 tx=0x7fba8000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba7a7fc700 1 -- 172.21.15.181:0/3281017466 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80004c80 con 0x7fba8415e6d0 2023-10-16T11:19:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba7a7fc700 1 -- 172.21.15.181:0/3281017466 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba80004e00 con 0x7fba8415e6d0 2023-10-16T11:19:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba840b0410 con 0x7fba8415e6d0 2023-10-16T11:19:06.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba7a7fc700 1 -- 172.21.15.181:0/3281017466 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80007310 con 0x7fba8415e6d0 2023-10-16T11:19:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.475+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8415bdd0 con 0x7fba8415e6d0 2023-10-16T11:19:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.476+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba84067d20 con 0x7fba8415e6d0 2023-10-16T11:19:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.477+0000 7fba7a7fc700 1 -- 172.21.15.181:0/3281017466 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fba80007470 con 0x7fba8415e6d0 2023-10-16T11:19:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.477+0000 7fba7a7fc700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 0x7fba6806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.477+0000 7fba899b7700 1 -- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 msgr2=0x7fba6806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.477+0000 7fba7a7fc700 1 -- 172.21.15.181:0/3281017466 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7fba80089fa0 con 0x7fba8415e6d0 2023-10-16T11:19:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.477+0000 7fba899b7700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 0x7fba6806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:06.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.481+0000 7fba7a7fc700 1 -- 172.21.15.181:0/3281017466 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fba80054a70 con 0x7fba8415e6d0 2023-10-16T11:19:06.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.678+0000 7fba899b7700 1 -- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 msgr2=0x7fba6806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:06.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.678+0000 7fba899b7700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 0x7fba6806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:06.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.726+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba840ca8d0 con 0x7fba8415e6d0 2023-10-16T11:19:06.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.727+0000 7fba7a7fc700 1 -- 172.21.15.181:0/3281017466 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fba80058090 con 0x7fba8415e6d0 2023-10-16T11:19:06.728 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:06.728 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3757723265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3757723265},{"type":"v1","addr":"172.21.15.190:6834","nonce":3757723265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2012626771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2012626771},{"type":"v1","addr":"172.21.15.190:6835","nonce":2012626771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10782,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3218502962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3218502962},{"type":"v1","addr":"172.21.15.181:6836","nonce":3218502962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:17:39.436499+0000","modified":"2023-10-16T11:17:42.454721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10785":{"gid":10785,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3625484950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3625484950},{"type":"v1","addr":"172.21.15.181:6837","nonce":3625484950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:19:06.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 msgr2=0x7fba6806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 0x7fba6806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 msgr2=0x7fba8415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba8415b890 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fba8000c750 tx=0x7fba8000ca60 comp rx=0 tx=0).stop 2023-10-16T11:19:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 shutdown_connections 2023-10-16T11:19:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba6806b140 0x7fba6806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba840cb0c0 0x7fba840b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8415eab0 0x7fba84151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 --2- 172.21.15.181:0/3281017466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8415e6d0 0x7fba8415b890 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 >> 172.21.15.181:0/3281017466 conn(0x7fba840d04a0 msgr2=0x7fba840cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:06.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 shutdown_connections 2023-10-16T11:19:06.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.729+0000 7fba8bc1b700 1 -- 172.21.15.181:0/3281017466 wait complete. 2023-10-16T11:19:06.733 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 102 2023-10-16T11:19:06.744 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:06.761 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:06.760+0000 7fbda211c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:06.839 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:06.838+0000 7fbda211c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:19:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.868+0000 7fba9b6a8700 1 Processor -- start 2023-10-16T11:19:06.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.869+0000 7fba9b6a8700 1 -- start start 2023-10-16T11:19:06.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba9b6a8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fba9415e6d0 0x7fba940d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba9b6a8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba9415eab0 0x7fba9415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba9b6a8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9406ed00 con 0x7fba940cb0c0 2023-10-16T11:19:06.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba9b6a8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba940d02d0 con 0x7fba9415eab0 2023-10-16T11:19:06.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba9b6a8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9415ad40 con 0x7fba9415e6d0 2023-10-16T11:19:06.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba9415eab0 0x7fba9415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba9415eab0 0x7fba9415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54756/0 (socket says 172.21.15.181:54756) 2023-10-16T11:19:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 -- 172.21.15.181:0/2584677271 learned_addr learned my addr 172.21.15.181:0/2584677271 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba99444700 1 --2- 172.21.15.181:0/2584677271 >> v2:172.21.15.190:3300/0 conn(0x7fba9415e6d0 0x7fba940d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 -- 172.21.15.181:0/2584677271 >> v2:172.21.15.190:3300/0 conn(0x7fba9415e6d0 msgr2=0x7fba940d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 --2- 172.21.15.181:0/2584677271 >> v2:172.21.15.190:3300/0 conn(0x7fba9415e6d0 0x7fba940d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 -- 172.21.15.181:0/2584677271 >> v1:172.21.15.190:6789/0 conn(0x7fba940cb0c0 legacy=0x7fba9415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba8bfff700 1 -- 172.21.15.181:0/2584677271 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679515496 0 0) 0x7fba9406ed00 con 0x7fba940cb0c0 2023-10-16T11:19:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 -- 172.21.15.181:0/2584677271 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9406e0f0 con 0x7fba9415eab0 2023-10-16T11:19:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba98c43700 1 --2- 172.21.15.181:0/2584677271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415a0f0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fba940717b0 tx=0x7fba84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2c55dba0c176a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba8bfff700 1 -- 172.21.15.181:0/2584677271 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84011830 con 0x7fba9415eab0 2023-10-16T11:19:06.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba8bfff700 1 -- 172.21.15.181:0/2584677271 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84011990 con 0x7fba9415eab0 2023-10-16T11:19:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.870+0000 7fba8bfff700 1 -- 172.21.15.181:0/2584677271 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84011d20 con 0x7fba9415eab0 2023-10-16T11:19:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 -- 172.21.15.181:0/2584677271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 msgr2=0x7fba9415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/2584677271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415a0f0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fba940717b0 tx=0x7fba84009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 -- 172.21.15.181:0/2584677271 shutdown_connections 2023-10-16T11:19:06.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/2584677271 >> v2:172.21.15.190:3300/0 conn(0x7fba9415e6d0 0x7fba940d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/2584677271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415a0f0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 -- 172.21.15.181:0/2584677271 >> 172.21.15.181:0/2584677271 conn(0x7fba940d04a0 msgr2=0x7fba940d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 -- 172.21.15.181:0/2584677271 shutdown_connections 2023-10-16T11:19:06.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.871+0000 7fba9b6a8700 1 -- 172.21.15.181:0/2584677271 wait complete. 2023-10-16T11:19:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.873+0000 7fba9b6a8700 1 Processor -- start 2023-10-16T11:19:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.873+0000 7fba9b6a8700 1 -- start start 2023-10-16T11:19:06.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba9b6a8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba940cb0c0 0x7fba9415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba9b6a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba9415e6d0 0x7fba94150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba9b6a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba9b6a8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9415cbe0 con 0x7fba9415eab0 2023-10-16T11:19:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba9b6a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9415cec0 con 0x7fba9415e6d0 2023-10-16T11:19:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba9b6a8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9415d1a0 con 0x7fba940cb0c0 2023-10-16T11:19:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54764/0 (socket says 172.21.15.181:54764) 2023-10-16T11:19:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 -- 172.21.15.181:0/1792927575 learned_addr learned my addr 172.21.15.181:0/1792927575 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba98c43700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba9415e6d0 0x7fba94150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99444700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba940cb0c0 0x7fba9415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 -- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba940cb0c0 msgr2=0x7fba9415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba940cb0c0 0x7fba9415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 -- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba9415e6d0 msgr2=0x7fba94150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba9415e6d0 0x7fba94150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 -- 172.21.15.181:0/1792927575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84009550 con 0x7fba9415eab0 2023-10-16T11:19:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.874+0000 7fba99c45700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415ae80 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c7a0 tx=0x7fba8c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.875+0000 7fba89ffb700 1 -- 172.21.15.181:0/1792927575 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c003e30 con 0x7fba9415eab0 2023-10-16T11:19:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.875+0000 7fba89ffb700 1 -- 172.21.15.181:0/1792927575 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c005070 con 0x7fba9415eab0 2023-10-16T11:19:06.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.875+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9415d480 con 0x7fba9415eab0 2023-10-16T11:19:06.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.875+0000 7fba89ffb700 1 -- 172.21.15.181:0/1792927575 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c005420 con 0x7fba9415eab0 2023-10-16T11:19:06.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.875+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba941509c0 con 0x7fba9415eab0 2023-10-16T11:19:06.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.876+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba94067d20 con 0x7fba9415eab0 2023-10-16T11:19:06.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.876+0000 7fba89ffb700 1 -- 172.21.15.181:0/1792927575 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fba8c005580 con 0x7fba9415eab0 2023-10-16T11:19:06.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.876+0000 7fba89ffb700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 0x7fba7806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:06.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.877+0000 7fba99444700 1 -- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 msgr2=0x7fba7806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:06.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.877+0000 7fba89ffb700 1 -- 172.21.15.181:0/1792927575 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7fba8c089450 con 0x7fba9415eab0 2023-10-16T11:19:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.877+0000 7fba99444700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 0x7fba7806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:06.881+0000 7fba89ffb700 1 -- 172.21.15.181:0/1792927575 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fba8c0577a0 con 0x7fba9415eab0 2023-10-16T11:19:06.905 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:06.904+0000 7f4bd423a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:06.984 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:06.982+0000 7fbda211c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:06.985 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:06.984+0000 7f4bd423a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:19:07.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.078+0000 7fba99444700 1 -- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 msgr2=0x7fba7806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:07.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.078+0000 7fba99444700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 0x7fba7806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:07.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.083+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba9406e680 con 0x7fba9415eab0 2023-10-16T11:19:07.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.085+0000 7fba89ffb700 1 -- 172.21.15.181:0/1792927575 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+18105 (secure 0 0 0) 0x7fba8c017090 con 0x7fba9415eab0 2023-10-16T11:19:07.086 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:07.086 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":204,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:06.020057+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:17:37.851564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:17:38.394398+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":204,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.7","mappings":[{"from":0,"to":2}]},{"pgid":"25.2d","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1746525923":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3361524667":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/1822518943":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6833/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6832/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3260777321":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3430010207":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/969172037":"2023-10-17T11:17:53.265498+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":25,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:07.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 msgr2=0x7fba7806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:07.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 0x7fba7806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 msgr2=0x7fba9415ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415ae80 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c7a0 tx=0x7fba8c00cab0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 shutdown_connections 2023-10-16T11:19:07.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fba7806b140 0x7fba7806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba940cb0c0 0x7fba9415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba9415eab0 0x7fba9415ae80 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 --2- 172.21.15.181:0/1792927575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba9415e6d0 0x7fba94150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 >> 172.21.15.181:0/1792927575 conn(0x7fba940d04a0 msgr2=0x7fba941514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:07.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 shutdown_connections 2023-10-16T11:19:07.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.087+0000 7fba9b6a8700 1 -- 172.21.15.181:0/1792927575 wait complete. 2023-10-16T11:19:07.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:17:37.851564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:17:38.394398+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '204', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 204, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:19:07.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:19:07.141 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:07.140+0000 7f4bd423a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:07.212 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:07.211+0000 7fbda211c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:19:07.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.227+0000 7f24307d1700 1 Processor -- start 2023-10-16T11:19:07.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.227+0000 7f24307d1700 1 -- start start 2023-10-16T11:19:07.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f24307d1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24280ce6c0 0x7f24280cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f24307d1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280b1f70 con 0x7f242815f150 2023-10-16T11:19:07.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f24307d1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280b2ae0 con 0x7f24280cc6a0 2023-10-16T11:19:07.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f24307d1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24280b2dc0 con 0x7f24280ce6c0 2023-10-16T11:19:07.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f242dd6c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24280ce6c0 0x7f24280cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f242e56d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f242815f150 0x7f24280cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37650/0 (socket says 172.21.15.181:37650) 2023-10-16T11:19:07.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f242e56d700 1 -- 172.21.15.181:0/2309270639 learned_addr learned my addr 172.21.15.181:0/2309270639 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:07.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f242dd6c700 1 -- 172.21.15.181:0/2309270639 >> v1:172.21.15.190:6789/0 conn(0x7f24280cc6a0 legacy=0x7f24280ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f242dd6c700 1 -- 172.21.15.181:0/2309270639 >> v1:172.21.15.181:6789/0 conn(0x7f242815f150 legacy=0x7f24280cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f242d56b700 1 -- 172.21.15.181:0/2309270639 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889651118 0 0) 0x7f24280b1f70 con 0x7f242815f150 2023-10-16T11:19:07.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.228+0000 7f242dd6c700 1 -- 172.21.15.181:0/2309270639 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280cf9d0 con 0x7f24280ce6c0 2023-10-16T11:19:07.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f242dd6c700 1 --2- 172.21.15.181:0/2309270639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280ce6c0 0x7f24280cc160 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2418009840 tx=0x7f2418009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a123fe8effad74d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:07.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f242d56b700 1 -- 172.21.15.181:0/2309270639 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418011830 con 0x7f24280ce6c0 2023-10-16T11:19:07.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f242d56b700 1 -- 172.21.15.181:0/2309270639 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418011990 con 0x7f24280ce6c0 2023-10-16T11:19:07.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f242d56b700 1 -- 172.21.15.181:0/2309270639 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418011d20 con 0x7f24280ce6c0 2023-10-16T11:19:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f24307d1700 1 -- 172.21.15.181:0/2309270639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280ce6c0 msgr2=0x7f24280cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f24307d1700 1 --2- 172.21.15.181:0/2309270639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280ce6c0 0x7f24280cc160 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2418009840 tx=0x7f2418009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:07.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f24307d1700 1 -- 172.21.15.181:0/2309270639 shutdown_connections 2023-10-16T11:19:07.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f24307d1700 1 --2- 172.21.15.181:0/2309270639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280ce6c0 0x7f24280cc160 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f24307d1700 1 -- 172.21.15.181:0/2309270639 >> 172.21.15.181:0/2309270639 conn(0x7f242809a930 msgr2=0x7f24280d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:07.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f24307d1700 1 -- 172.21.15.181:0/2309270639 shutdown_connections 2023-10-16T11:19:07.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.229+0000 7f24307d1700 1 -- 172.21.15.181:0/2309270639 wait complete. 2023-10-16T11:19:07.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 Processor -- start 2023-10-16T11:19:07.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 -- start start 2023-10-16T11:19:07.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280cc6a0 0x7f242806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24280ce6c0 0x7f2428151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f242815f150 0x7f242815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242815a3e0 con 0x7f24280cc6a0 2023-10-16T11:19:07.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242815a100 con 0x7f24280ce6c0 2023-10-16T11:19:07.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f24307d1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428153a00 con 0x7f242815f150 2023-10-16T11:19:07.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f242e56d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280cc6a0 0x7f242806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f242e56d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280cc6a0 0x7f242806b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54810/0 (socket says 172.21.15.181:54810) 2023-10-16T11:19:07.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f242e56d700 1 -- 172.21.15.181:0/2831077954 learned_addr learned my addr 172.21.15.181:0/2831077954 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:07.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.231+0000 7f242dd6c700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24280ce6c0 0x7f2428151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f242ed6e700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f242815f150 0x7f242815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f242e56d700 1 -- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f242815f150 msgr2=0x7f242815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f242e56d700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f242815f150 0x7f242815b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f242e56d700 1 -- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24280ce6c0 msgr2=0x7f2428151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f242e56d700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24280ce6c0 0x7f2428151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f242e56d700 1 -- 172.21.15.181:0/2831077954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24180094a0 con 0x7f24280cc6a0 2023-10-16T11:19:07.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f242e56d700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280cc6a0 0x7f242806b9b0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f242400c520 tx=0x7f242400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:07.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f241effd700 1 -- 172.21.15.181:0/2831077954 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424004c80 con 0x7f24280cc6a0 2023-10-16T11:19:07.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f241effd700 1 -- 172.21.15.181:0/2831077954 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2424004de0 con 0x7f24280cc6a0 2023-10-16T11:19:07.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2428153ce0 con 0x7f24280cc6a0 2023-10-16T11:19:07.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f241effd700 1 -- 172.21.15.181:0/2831077954 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424007310 con 0x7f24280cc6a0 2023-10-16T11:19:07.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.232+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242806bef0 con 0x7f24280cc6a0 2023-10-16T11:19:07.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.233+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428067d20 con 0x7f24280cc6a0 2023-10-16T11:19:07.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.233+0000 7f241effd700 1 -- 172.21.15.181:0/2831077954 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f2424007470 con 0x7f24280cc6a0 2023-10-16T11:19:07.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.234+0000 7f241effd700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 0x7f240c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.234+0000 7f242dd6c700 1 -- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 msgr2=0x7f240c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:07.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.234+0000 7f241effd700 1 -- 172.21.15.181:0/2831077954 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7f2424089970 con 0x7f24280cc6a0 2023-10-16T11:19:07.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.234+0000 7f242dd6c700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 0x7f240c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:07.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.238+0000 7f241effd700 1 -- 172.21.15.181:0/2831077954 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f24240542d0 con 0x7f24280cc6a0 2023-10-16T11:19:07.286 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:07.285+0000 7fbda211c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:19:07.390 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:07.390+0000 7f4bd423a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:19:07.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.435+0000 7f242dd6c700 1 -- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 msgr2=0x7f240c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:07.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.435+0000 7f242dd6c700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 0x7f240c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.446+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24280706b0 con 0x7f24280cc6a0 2023-10-16T11:19:07.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.447+0000 7f241effd700 1 -- 172.21.15.181:0/2831077954 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v204) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24240578f0 con 0x7f24280cc6a0 2023-10-16T11:19:07.448 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:19:07.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 msgr2=0x7f240c06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:07.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 0x7f240c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280cc6a0 msgr2=0x7f242806b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280cc6a0 0x7f242806b9b0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f242400c520 tx=0x7f242400c830 comp rx=0 tx=0).stop 2023-10-16T11:19:07.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 shutdown_connections 2023-10-16T11:19:07.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f240c06b140 0x7f240c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f242815f150 0x7f242815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24280cc6a0 0x7f242806b9b0 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 --2- 172.21.15.181:0/2831077954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24280ce6c0 0x7f2428151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 >> 172.21.15.181:0/2831077954 conn(0x7f242809a930 msgr2=0x7f242815d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:07.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 shutdown_connections 2023-10-16T11:19:07.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.449+0000 7f24307d1700 1 -- 172.21.15.181:0/2831077954 wait complete. 2023-10-16T11:19:07.463 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:19:07.468 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:07.467+0000 7f4bd423a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:19:07.549 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:07.547+0000 7fbda211c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:19:07.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.596+0000 7f433c082700 1 Processor -- start 2023-10-16T11:19:07.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.596+0000 7f433c082700 1 -- start start 2023-10-16T11:19:07.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433c082700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f433415da70 0x7f43340ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433c082700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43340ce6a0 0x7f433406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433c082700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43340cef00 con 0x7f4334072760 2023-10-16T11:19:07.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433c082700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43340cec20 con 0x7f43340ce6a0 2023-10-16T11:19:07.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433c082700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4334071c80 con 0x7f433415da70 2023-10-16T11:19:07.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433961d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43340ce6a0 0x7f433406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433a61f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4334072760 0x7f43341615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37670/0 (socket says 172.21.15.181:37670) 2023-10-16T11:19:07.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f433a61f700 1 -- 172.21.15.181:0/291408292 learned_addr learned my addr 172.21.15.181:0/291408292 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:07.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.597+0000 7f4339e1e700 1 --2- 172.21.15.181:0/291408292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f433415da70 0x7f43340ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f433961d700 1 -- 172.21.15.181:0/291408292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f433415da70 msgr2=0x7f43340ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:07.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f433961d700 1 --2- 172.21.15.181:0/291408292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f433415da70 0x7f43340ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f433961d700 1 -- 172.21.15.181:0/291408292 >> v1:172.21.15.181:6789/0 conn(0x7f4334072760 legacy=0x7f43341615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f4338e1c700 1 -- 172.21.15.181:0/291408292 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105343896 0 0) 0x7f43340cef00 con 0x7f4334072760 2023-10-16T11:19:07.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f433961d700 1 -- 172.21.15.181:0/291408292 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334071f60 con 0x7f43340ce6a0 2023-10-16T11:19:07.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f433961d700 1 --2- 172.21.15.181:0/291408292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43340ce6a0 0x7f433406a8a0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f4330009840 tx=0x7f4330009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa509fe47d7ef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:07.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f4338e1c700 1 -- 172.21.15.181:0/291408292 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330011830 con 0x7f43340ce6a0 2023-10-16T11:19:07.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f4338e1c700 1 -- 172.21.15.181:0/291408292 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330011990 con 0x7f43340ce6a0 2023-10-16T11:19:07.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.598+0000 7f4338e1c700 1 -- 172.21.15.181:0/291408292 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330011d20 con 0x7f43340ce6a0 2023-10-16T11:19:07.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 -- 172.21.15.181:0/291408292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43340ce6a0 msgr2=0x7f433406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 --2- 172.21.15.181:0/291408292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43340ce6a0 0x7f433406a8a0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f4330009840 tx=0x7f4330009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:07.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 -- 172.21.15.181:0/291408292 shutdown_connections 2023-10-16T11:19:07.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 --2- 172.21.15.181:0/291408292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f433415da70 0x7f43340ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 --2- 172.21.15.181:0/291408292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43340ce6a0 0x7f433406a8a0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 -- 172.21.15.181:0/291408292 >> 172.21.15.181:0/291408292 conn(0x7f43340cd010 msgr2=0x7f43340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:07.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 -- 172.21.15.181:0/291408292 shutdown_connections 2023-10-16T11:19:07.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.599+0000 7f433c082700 1 -- 172.21.15.181:0/291408292 wait complete. 2023-10-16T11:19:07.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.601+0000 7f433c082700 1 Processor -- start 2023-10-16T11:19:07.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.601+0000 7f433c082700 1 -- start start 2023-10-16T11:19:07.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433c082700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4334072760 0x7f4334151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433c082700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43340ce6a0 0x7f43340c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433c082700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433415da70 0x7f43340b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433c082700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340c5740 con 0x7f433415da70 2023-10-16T11:19:07.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433c082700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340c4b70 con 0x7f43340ce6a0 2023-10-16T11:19:07.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433c082700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43340c4e10 con 0x7f4334072760 2023-10-16T11:19:07.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433a61f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433415da70 0x7f43340b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433a61f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433415da70 0x7f43340b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54830/0 (socket says 172.21.15.181:54830) 2023-10-16T11:19:07.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433a61f700 1 -- 172.21.15.181:0/3840752637 learned_addr learned my addr 172.21.15.181:0/3840752637 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:07.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f4339e1e700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4334072760 0x7f4334151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.602+0000 7f433961d700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43340ce6a0 0x7f43340c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433a61f700 1 -- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4334072760 msgr2=0x7f4334151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433a61f700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4334072760 0x7f4334151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433a61f700 1 -- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43340ce6a0 msgr2=0x7f43340c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433a61f700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43340ce6a0 0x7f43340c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433a61f700 1 -- 172.21.15.181:0/3840752637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43300094a0 con 0x7f433415da70 2023-10-16T11:19:07.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433a61f700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433415da70 0x7f43340b31b0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f432c00cba0 tx=0x7f432c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:07.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f432a7fc700 1 -- 172.21.15.181:0/3840752637 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c00aaa0 con 0x7f433415da70 2023-10-16T11:19:07.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f432a7fc700 1 -- 172.21.15.181:0/3840752637 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c00ac20 con 0x7f433415da70 2023-10-16T11:19:07.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f432a7fc700 1 -- 172.21.15.181:0/3840752637 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c0070d0 con 0x7f433415da70 2023-10-16T11:19:07.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43340c50f0 con 0x7f433415da70 2023-10-16T11:19:07.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.603+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43340c5bd0 con 0x7f433415da70 2023-10-16T11:19:07.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.604+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43340c58b0 con 0x7f433415da70 2023-10-16T11:19:07.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.605+0000 7f432a7fc700 1 -- 172.21.15.181:0/3840752637 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f432c007230 con 0x7f433415da70 2023-10-16T11:19:07.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.605+0000 7f432a7fc700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 0x7f431806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.605+0000 7f4339e1e700 1 -- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 msgr2=0x7f431806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:07.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.605+0000 7f432a7fc700 1 -- 172.21.15.181:0/3840752637 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7f432c089880 con 0x7f433415da70 2023-10-16T11:19:07.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.605+0000 7f4339e1e700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 0x7f431806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:07.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.610+0000 7f432a7fc700 1 -- 172.21.15.181:0/3840752637 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f432c0541e0 con 0x7f433415da70 2023-10-16T11:19:07.750 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:07.750+0000 7f4bd423a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:19:07.770 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:07.769+0000 7fbda211c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:19:07.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.806+0000 7f4339e1e700 1 -- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 msgr2=0x7f431806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:07.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.806+0000 7f4339e1e700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 0x7f431806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:07.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.812+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f433406f6c0 con 0x7f433415da70 2023-10-16T11:19:07.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.812+0000 7f432a7fc700 1 -- 172.21.15.181:0/3840752637 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v204) v1 ==== 104+0+11 (secure 0 0 0) 0x7f432c018020 con 0x7f433415da70 2023-10-16T11:19:07.813 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:19:07.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 msgr2=0x7f431806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:07.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 0x7f431806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433415da70 msgr2=0x7f43340b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433415da70 0x7f43340b31b0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f432c00cba0 tx=0x7f432c00ceb0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 shutdown_connections 2023-10-16T11:19:07.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f431806b140 0x7f431806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4334072760 0x7f4334151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433415da70 0x7f43340b31b0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 --2- 172.21.15.181:0/3840752637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43340ce6a0 0x7f43340c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 >> 172.21.15.181:0/3840752637 conn(0x7f43340cd010 msgr2=0x7f43340cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:07.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 shutdown_connections 2023-10-16T11:19:07.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.814+0000 7f433c082700 1 -- 172.21.15.181:0/3840752637 wait complete. 2023-10-16T11:19:07.828 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:19:07.919 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:07.917+0000 7fbda211c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:19:07.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.953+0000 7fdb394f1700 1 Processor -- start 2023-10-16T11:19:07.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.953+0000 7fdb394f1700 1 -- start start 2023-10-16T11:19:07.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb394f1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb340ce2d0 0x7fdb3415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb394f1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3415ef80 con 0x7fdb340d1500 2023-10-16T11:19:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb394f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb340cf580 con 0x7fdb340d01d0 2023-10-16T11:19:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb394f1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb340d4290 con 0x7fdb340ce2d0 2023-10-16T11:19:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb327fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb340ce2d0 0x7fdb3415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb337fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdb340d1500 0x7fdb340d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37686/0 (socket says 172.21.15.181:37686) 2023-10-16T11:19:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb337fe700 1 -- 172.21.15.181:0/3383186057 learned_addr learned my addr 172.21.15.181:0/3383186057 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb327fc700 1 -- 172.21.15.181:0/3383186057 >> v1:172.21.15.190:6789/0 conn(0x7fdb340d01d0 legacy=0x7fdb340d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb327fc700 1 -- 172.21.15.181:0/3383186057 >> v1:172.21.15.181:6789/0 conn(0x7fdb340d1500 legacy=0x7fdb340d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb327fc700 1 -- 172.21.15.181:0/3383186057 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340ce970 con 0x7fdb340ce2d0 2023-10-16T11:19:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb31ffb700 1 -- 172.21.15.181:0/3383186057 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327722824 0 0) 0x7fdb2c001490 con 0x7fdb340d1500 2023-10-16T11:19:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.954+0000 7fdb327fc700 1 --2- 172.21.15.181:0/3383186057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340ce2d0 0x7fdb3415ea40 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009840 tx=0x7fdb28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdb40c529863941 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb31ffb700 1 -- 172.21.15.181:0/3383186057 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28011830 con 0x7fdb340ce2d0 2023-10-16T11:19:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb31ffb700 1 -- 172.21.15.181:0/3383186057 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28011990 con 0x7fdb340ce2d0 2023-10-16T11:19:07.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb31ffb700 1 -- 172.21.15.181:0/3383186057 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28011d20 con 0x7fdb340ce2d0 2023-10-16T11:19:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb394f1700 1 -- 172.21.15.181:0/3383186057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340ce2d0 msgr2=0x7fdb3415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb394f1700 1 --2- 172.21.15.181:0/3383186057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340ce2d0 0x7fdb3415ea40 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009840 tx=0x7fdb28009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb394f1700 1 -- 172.21.15.181:0/3383186057 shutdown_connections 2023-10-16T11:19:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb394f1700 1 --2- 172.21.15.181:0/3383186057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340ce2d0 0x7fdb3415ea40 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb394f1700 1 -- 172.21.15.181:0/3383186057 >> 172.21.15.181:0/3383186057 conn(0x7fdb3409cb60 msgr2=0x7fdb340d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb394f1700 1 -- 172.21.15.181:0/3383186057 shutdown_connections 2023-10-16T11:19:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.955+0000 7fdb394f1700 1 -- 172.21.15.181:0/3383186057 wait complete. 2023-10-16T11:19:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 Processor -- start 2023-10-16T11:19:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 -- start start 2023-10-16T11:19:07.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb340ce2d0 0x7fdb340cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb340d01d0 0x7fdb340b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340d1500 0x7fdb340c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340cb330 con 0x7fdb340d1500 2023-10-16T11:19:07.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340cb610 con 0x7fdb340d01d0 2023-10-16T11:19:07.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb394f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340cb8f0 con 0x7fdb340ce2d0 2023-10-16T11:19:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340d1500 0x7fdb340c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340d1500 0x7fdb340c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54846/0 (socket says 172.21.15.181:54846) 2023-10-16T11:19:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb337fe700 1 -- 172.21.15.181:0/287802345 learned_addr learned my addr 172.21.15.181:0/287802345 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb327fc700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb340d01d0 0x7fdb340b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.957+0000 7fdb32ffd700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb340ce2d0 0x7fdb340cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb337fe700 1 -- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb340ce2d0 msgr2=0x7fdb340cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb337fe700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb340ce2d0 0x7fdb340cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb337fe700 1 -- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb340d01d0 msgr2=0x7fdb340b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb337fe700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb340d01d0 0x7fdb340b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:07.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb337fe700 1 -- 172.21.15.181:0/287802345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280094a0 con 0x7fdb340d1500 2023-10-16T11:19:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb337fe700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340d1500 0x7fdb340c9310 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00cba0 tx=0x7fdb2c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb1b7fe700 1 -- 172.21.15.181:0/287802345 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00aaa0 con 0x7fdb340d1500 2023-10-16T11:19:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb1b7fe700 1 -- 172.21.15.181:0/287802345 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c00ac20 con 0x7fdb340d1500 2023-10-16T11:19:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb1b7fe700 1 -- 172.21.15.181:0/287802345 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0070d0 con 0x7fdb340d1500 2023-10-16T11:19:07.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb340cbbd0 con 0x7fdb340d1500 2023-10-16T11:19:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.958+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb340b4920 con 0x7fdb340d1500 2023-10-16T11:19:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.959+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb341f44c0 con 0x7fdb340d1500 2023-10-16T11:19:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.960+0000 7fdb1b7fe700 1 -- 172.21.15.181:0/287802345 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fdb2c007230 con 0x7fdb340d1500 2023-10-16T11:19:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.960+0000 7fdb1b7fe700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 0x7fdb2006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:07.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.960+0000 7fdb32ffd700 1 -- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 msgr2=0x7fdb2006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:07.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.960+0000 7fdb1b7fe700 1 -- 172.21.15.181:0/287802345 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7fdb2c0896d0 con 0x7fdb340d1500 2023-10-16T11:19:07.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.960+0000 7fdb32ffd700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 0x7fdb2006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:07.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:07.965+0000 7fdb1b7fe700 1 -- 172.21.15.181:0/287802345 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fdb2c054030 con 0x7fdb340d1500 2023-10-16T11:19:07.989 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:07.988+0000 7f4bd423a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:19:08.034 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:08.032+0000 7fbda211c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:19:08.148 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:08.147+0000 7f4bd423a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:19:08.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.161+0000 7fdb32ffd700 1 -- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 msgr2=0x7fdb2006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:08.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.161+0000 7fdb32ffd700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 0x7fdb2006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:08.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.169+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb3415ef80 con 0x7fdb340d1500 2023-10-16T11:19:08.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.170+0000 7fdb1b7fe700 1 -- 172.21.15.181:0/287802345 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v204) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb2c018020 con 0x7fdb340d1500 2023-10-16T11:19:08.171 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:19:08.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 msgr2=0x7fdb2006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 0x7fdb2006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340d1500 msgr2=0x7fdb340c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340d1500 0x7fdb340c9310 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00cba0 tx=0x7fdb2c00ceb0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 shutdown_connections 2023-10-16T11:19:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7fdb2006b140 0x7fdb2006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb340ce2d0 0x7fdb340cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb340d1500 0x7fdb340c9310 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 --2- 172.21.15.181:0/287802345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb340d01d0 0x7fdb340b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 >> 172.21.15.181:0/287802345 conn(0x7fdb3409cb60 msgr2=0x7fdb340b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:08.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 shutdown_connections 2023-10-16T11:19:08.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.172+0000 7fdb394f1700 1 -- 172.21.15.181:0/287802345 wait complete. 2023-10-16T11:19:08.186 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:08.189 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:08.188+0000 7fbda211c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:19:08.270 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:08.269+0000 7f4bd423a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:19:08.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.308+0000 7ff0b4c37700 1 Processor -- start 2023-10-16T11:19:08.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.308+0000 7ff0b4c37700 1 -- start start 2023-10-16T11:19:08.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.309+0000 7ff0b4c37700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0b0160d20 0x7ff0b01611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.309+0000 7ff0b4c37700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b006ce20 con 0x7ff0b0161b60 2023-10-16T11:19:08.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.309+0000 7ff0b4c37700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b00d45f0 con 0x7ff0b006d5f0 2023-10-16T11:19:08.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.309+0000 7ff0b4c37700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b0162800 con 0x7ff0b0160d20 2023-10-16T11:19:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.309+0000 7ff0ae59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff0b0161b60 0x7ff0b0161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37698/0 (socket says 172.21.15.181:37698) 2023-10-16T11:19:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.309+0000 7ff0ae59c700 1 -- 172.21.15.181:0/3329156789 learned_addr learned my addr 172.21.15.181:0/3329156789 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0aed9d700 1 --2- 172.21.15.181:0/3329156789 >> v2:172.21.15.190:3300/0 conn(0x7ff0b0160d20 0x7ff0b01611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650590876 0 0) 0x7ff0b006ce20 con 0x7ff0b0161b60 2023-10-16T11:19:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0980035b0 con 0x7ff0b0161b60 2023-10-16T11:19:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898277702 0 0) 0x7ff0b00d45f0 con 0x7ff0b006d5f0 2023-10-16T11:19:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b006ce20 con 0x7ff0b006d5f0 2023-10-16T11:19:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122731734 0 0) 0x7ff0980035b0 con 0x7ff0b0161b60 2023-10-16T11:19:08.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 >> v2:172.21.15.190:3300/0 conn(0x7ff0b0160d20 msgr2=0x7ff0b01611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:08.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 --2- 172.21.15.181:0/3329156789 >> v2:172.21.15.190:3300/0 conn(0x7ff0b0160d20 0x7ff0b01611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 >> v1:172.21.15.190:6789/0 conn(0x7ff0b006d5f0 legacy=0x7ff0b00d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b006d2d0 con 0x7ff0b0161b60 2023-10-16T11:19:08.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff0a00017f0 con 0x7ff0b0161b60 2023-10-16T11:19:08.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 >> v1:172.21.15.181:6789/0 conn(0x7ff0b0161b60 legacy=0x7ff0b0161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff098003b90 0x7ff098003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098004850 0x7ff098004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff098005210 0x7ff09800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0162800 con 0x7ff098003b90 2023-10-16T11:19:08.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09800c140 con 0x7ff098005210 2023-10-16T11:19:08.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09800c300 con 0x7ff098004850 2023-10-16T11:19:08.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ae59c700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff098003b90 0x7ff098003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0add9b700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098004850 0x7ff098004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0aed9d700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff098005210 0x7ff09800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ae59c700 1 -- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098004850 msgr2=0x7ff098004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:08.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ae59c700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098004850 0x7ff098004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ae59c700 1 -- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff098005210 msgr2=0x7ff09800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:08.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ae59c700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff098005210 0x7ff09800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0aed9d700 1 -- 172.21.15.181:0/3329156789 reap_dead start 2023-10-16T11:19:08.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.310+0000 7ff0ae59c700 1 -- 172.21.15.181:0/3329156789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980035b0 con 0x7ff098003b90 2023-10-16T11:19:08.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0ae59c700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff098003b90 0x7ff098003f90 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c240 tx=0x7ff098005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7aeb7320049c3e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:08.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000cd80 con 0x7ff098003b90 2023-10-16T11:19:08.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0001c70 con 0x7ff098003b90 2023-10-16T11:19:08.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0ad59a700 1 -- 172.21.15.181:0/3329156789 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0007310 con 0x7ff098003b90 2023-10-16T11:19:08.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0b4c37700 1 -- 172.21.15.181:0/3329156789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff098003b90 msgr2=0x7ff098003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff098003b90 0x7ff098003f90 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c240 tx=0x7ff098005880 comp rx=0 tx=0).stop 2023-10-16T11:19:08.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0b4c37700 1 -- 172.21.15.181:0/3329156789 shutdown_connections 2023-10-16T11:19:08.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/3329156789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff098003b90 0x7ff098003f90 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0b4c37700 1 -- 172.21.15.181:0/3329156789 >> 172.21.15.181:0/3329156789 conn(0x7ff0b009cba0 msgr2=0x7ff0b00d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:08.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0b4c37700 1 -- 172.21.15.181:0/3329156789 shutdown_connections 2023-10-16T11:19:08.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.311+0000 7ff0b4c37700 1 -- 172.21.15.181:0/3329156789 wait complete. 2023-10-16T11:19:08.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.313+0000 7ff0b4c37700 1 Processor -- start 2023-10-16T11:19:08.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.313+0000 7ff0b4c37700 1 -- start start 2023-10-16T11:19:08.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.313+0000 7ff0b4c37700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098003b90 0x7ff0b00b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.313+0000 7ff0b4c37700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00b82a0 0x7ff0b00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0b4c37700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b00b3690 0x7ff0b00b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0b4c37700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b00b3b10 con 0x7ff0b00b3690 2023-10-16T11:19:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0b4c37700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b00b7200 con 0x7ff0b00b82a0 2023-10-16T11:19:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0b4c37700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b00b74e0 con 0x7ff098003b90 2023-10-16T11:19:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b00b3690 0x7ff0b00b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b00b3690 0x7ff0b00b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54866/0 (socket says 172.21.15.181:54866) 2023-10-16T11:19:08.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 -- 172.21.15.181:0/4130279545 learned_addr learned my addr 172.21.15.181:0/4130279545 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:08.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0ae59c700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098003b90 0x7ff0b00b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0add9b700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00b82a0 0x7ff0b00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 -- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098003b90 msgr2=0x7ff0b00b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098003b90 0x7ff0b00b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 -- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00b82a0 msgr2=0x7ff0b00b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00b82a0 0x7ff0b00b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 -- 172.21.15.181:0/4130279545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b00b77c0 con 0x7ff0b00b3690 2023-10-16T11:19:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0aed9d700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b00b3690 0x7ff0b00b5600 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7ff0b006b640 tx=0x7ff0a8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff09effd700 1 -- 172.21.15.181:0/4130279545 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800e040 con 0x7ff0b00b3690 2023-10-16T11:19:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff09effd700 1 -- 172.21.15.181:0/4130279545 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a800f040 con 0x7ff0b00b3690 2023-10-16T11:19:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b00b7aa0 con 0x7ff0b00b3690 2023-10-16T11:19:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff09effd700 1 -- 172.21.15.181:0/4130279545 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800fc40 con 0x7ff0b00b3690 2023-10-16T11:19:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.314+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b00b5b40 con 0x7ff0b00b3690 2023-10-16T11:19:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.315+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b004ebb0 con 0x7ff0b00b3690 2023-10-16T11:19:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.316+0000 7ff09effd700 1 -- 172.21.15.181:0/4130279545 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7ff0a8022460 con 0x7ff0b00b3690 2023-10-16T11:19:08.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.316+0000 7ff09effd700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 0x7ff09006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.316+0000 7ff0ae59c700 1 -- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 msgr2=0x7ff09006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.316+0000 7ff09effd700 1 -- 172.21.15.181:0/4130279545 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7ff0a808a390 con 0x7ff0b00b3690 2023-10-16T11:19:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.316+0000 7ff0ae59c700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 0x7ff09006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:08.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.321+0000 7ff09effd700 1 -- 172.21.15.181:0/4130279545 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0a8054be0 con 0x7ff0b00b3690 2023-10-16T11:19:08.436 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:08.435+0000 7f4bd423a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:19:08.452 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:08.451+0000 7fbda211c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:08.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.517+0000 7ff0ae59c700 1 -- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 msgr2=0x7ff09006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:08.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.517+0000 7ff0ae59c700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 0x7ff09006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:08.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.562+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b00b8680 con 0x7ff0b00b3690 2023-10-16T11:19:08.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.563+0000 7ff09effd700 1 -- 172.21.15.181:0/4130279545 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4935 (secure 0 0 0) 0x7ff0a8058200 con 0x7ff0b00b3690 2023-10-16T11:19:08.564 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:08.564 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3757723265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3757723265},{"type":"v1","addr":"172.21.15.190:6834","nonce":3757723265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2012626771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2012626771},{"type":"v1","addr":"172.21.15.190:6835","nonce":2012626771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10782,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3218502962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3218502962},{"type":"v1","addr":"172.21.15.181:6836","nonce":3218502962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:17:39.436499+0000","modified":"2023-10-16T11:17:42.454721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10785":{"gid":10785,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3625484950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3625484950},{"type":"v1","addr":"172.21.15.181:6837","nonce":3625484950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:19:08.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 msgr2=0x7ff09006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:08.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 0x7ff09006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b00b3690 msgr2=0x7ff0b00b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b00b3690 0x7ff0b00b5600 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7ff0b006b640 tx=0x7ff0a8011e20 comp rx=0 tx=0).stop 2023-10-16T11:19:08.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 shutdown_connections 2023-10-16T11:19:08.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7ff09006b140 0x7ff09006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff098003b90 0x7ff0b00b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0b00b3690 0x7ff0b00b5600 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 --2- 172.21.15.181:0/4130279545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00b82a0 0x7ff0b00b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 >> 172.21.15.181:0/4130279545 conn(0x7ff0b009cba0 msgr2=0x7ff0b006c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:08.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.565+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 shutdown_connections 2023-10-16T11:19:08.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.566+0000 7ff0b4c37700 1 -- 172.21.15.181:0/4130279545 wait complete. 2023-10-16T11:19:08.569 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 102 2023-10-16T11:19:08.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:08.612 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:08.611+0000 7fbda211c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:19:08.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.702+0000 7f634694b700 1 Processor -- start 2023-10-16T11:19:08.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.702+0000 7f634694b700 1 -- start start 2023-10-16T11:19:08.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f634694b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f634015b990 0x7f63400ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f634694b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cd2d0 0x7f63400d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f634694b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63400cc510 con 0x7f63400cfc60 2023-10-16T11:19:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f634694b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63400d24f0 con 0x7f634015b990 2023-10-16T11:19:08.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f634694b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63400cc7f0 con 0x7f63400cd2d0 2023-10-16T11:19:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f634015b990 0x7f63400ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f634015b990 0x7f63400ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54872/0 (socket says 172.21.15.181:54872) 2023-10-16T11:19:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 -- 172.21.15.181:0/195401581 learned_addr learned my addr 172.21.15.181:0/195401581 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633f7fe700 1 --2- 172.21.15.181:0/195401581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cd2d0 0x7f63400d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 -- 172.21.15.181:0/195401581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cd2d0 msgr2=0x7f63400d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 --2- 172.21.15.181:0/195401581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cd2d0 0x7f63400d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 -- 172.21.15.181:0/195401581 >> v1:172.21.15.190:6789/0 conn(0x7f63400cfc60 legacy=0x7f63400ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633effd700 1 -- 172.21.15.181:0/195401581 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424017646 0 0) 0x7f63400cc510 con 0x7f63400cfc60 2023-10-16T11:19:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 -- 172.21.15.181:0/195401581 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63400ccad0 con 0x7f634015b990 2023-10-16T11:19:08.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.703+0000 7f633ffff700 1 --2- 172.21.15.181:0/195401581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634015b990 0x7f63400ce8f0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f6334006fd0 tx=0x7f6334009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5001828fe67516a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f633effd700 1 -- 172.21.15.181:0/195401581 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334005e30 con 0x7f634015b990 2023-10-16T11:19:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f633effd700 1 -- 172.21.15.181:0/195401581 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334004030 con 0x7f634015b990 2023-10-16T11:19:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f633effd700 1 -- 172.21.15.181:0/195401581 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340043e0 con 0x7f634015b990 2023-10-16T11:19:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 -- 172.21.15.181:0/195401581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634015b990 msgr2=0x7f63400ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 --2- 172.21.15.181:0/195401581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634015b990 0x7f63400ce8f0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f6334006fd0 tx=0x7f6334009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 -- 172.21.15.181:0/195401581 shutdown_connections 2023-10-16T11:19:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 --2- 172.21.15.181:0/195401581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634015b990 0x7f63400ce8f0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 --2- 172.21.15.181:0/195401581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cd2d0 0x7f63400d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 -- 172.21.15.181:0/195401581 >> 172.21.15.181:0/195401581 conn(0x7f634009a9a0 msgr2=0x7f63400d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:08.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 -- 172.21.15.181:0/195401581 shutdown_connections 2023-10-16T11:19:08.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.704+0000 7f634694b700 1 -- 172.21.15.181:0/195401581 wait complete. 2023-10-16T11:19:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.705+0000 7f634694b700 1 Processor -- start 2023-10-16T11:19:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f634694b700 1 -- start start 2023-10-16T11:19:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f634694b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63400cd2d0 0x7f63401f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f634694b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cfc60 0x7f6340158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f634694b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634015b990 0x7f63401e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f634694b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634015c6b0 con 0x7f63400cd2d0 2023-10-16T11:19:08.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f634694b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63400ca2c0 con 0x7f634015b990 2023-10-16T11:19:08.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f634694b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63400ca5a0 con 0x7f63400cfc60 2023-10-16T11:19:08.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f633ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63400cd2d0 0x7f63401f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f633ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63400cd2d0 0x7f63401f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54878/0 (socket says 172.21.15.181:54878) 2023-10-16T11:19:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f633ffff700 1 -- 172.21.15.181:0/3797773261 learned_addr learned my addr 172.21.15.181:0/3797773261 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f633f7fe700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cfc60 0x7f6340158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.706+0000 7f6344ee8700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634015b990 0x7f63401e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633ffff700 1 -- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cfc60 msgr2=0x7f6340158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633ffff700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cfc60 0x7f6340158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633ffff700 1 -- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634015b990 msgr2=0x7f63401e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633ffff700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634015b990 0x7f63401e52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633ffff700 1 -- 172.21.15.181:0/3797773261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334009500 con 0x7f63400cd2d0 2023-10-16T11:19:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633ffff700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63400cd2d0 0x7f63401f50b0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f6334009c10 tx=0x7f6334004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633cff9700 1 -- 172.21.15.181:0/3797773261 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334003ea0 con 0x7f63400cd2d0 2023-10-16T11:19:08.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633cff9700 1 -- 172.21.15.181:0/3797773261 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334011420 con 0x7f63400cd2d0 2023-10-16T11:19:08.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f633cff9700 1 -- 172.21.15.181:0/3797773261 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340117d0 con 0x7f63400cd2d0 2023-10-16T11:19:08.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634015c060 con 0x7f63400cd2d0 2023-10-16T11:19:08.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.707+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63400cb0d0 con 0x7f63400cd2d0 2023-10-16T11:19:08.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.708+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63400cb260 con 0x7f63400cd2d0 2023-10-16T11:19:08.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.709+0000 7f633cff9700 1 -- 172.21.15.181:0/3797773261 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f6334018030 con 0x7f63400cd2d0 2023-10-16T11:19:08.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.709+0000 7f633cff9700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 0x7f632c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:08.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.709+0000 7f633f7fe700 1 -- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 msgr2=0x7f632c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:08.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.710+0000 7f633cff9700 1 -- 172.21.15.181:0/3797773261 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7f633408b060 con 0x7f63400cd2d0 2023-10-16T11:19:08.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.710+0000 7f633f7fe700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 0x7f632c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:08.719 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:08.713+0000 7f4bd423a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:08.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.714+0000 7f633cff9700 1 -- 172.21.15.181:0/3797773261 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f63340559c0 con 0x7f63400cd2d0 2023-10-16T11:19:08.882 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:08.882+0000 7f4bd423a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:19:08.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.911+0000 7f633f7fe700 1 -- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 msgr2=0x7f632c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:08.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.911+0000 7f633f7fe700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 0x7f632c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:08.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.913+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6340067d20 con 0x7f63400cd2d0 2023-10-16T11:19:08.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.914+0000 7f633cff9700 1 -- 172.21.15.181:0/3797773261 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+18105 (secure 0 0 0) 0x7f6334058fe0 con 0x7f63400cd2d0 2023-10-16T11:19:08.915 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:08.916 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":204,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:06.020057+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:17:37.851564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:17:38.394398+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":204,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.7","mappings":[{"from":0,"to":2}]},{"pgid":"25.2d","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1746525923":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3361524667":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/1822518943":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6833/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6832/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3260777321":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3430010207":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/969172037":"2023-10-17T11:17:53.265498+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":25,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:08.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 msgr2=0x7f632c06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 0x7f632c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63400cd2d0 msgr2=0x7f63401f50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63400cd2d0 0x7f63401f50b0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f6334009c10 tx=0x7f6334004750 comp rx=0 tx=0).stop 2023-10-16T11:19:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 shutdown_connections 2023-10-16T11:19:08.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f632c06b140 0x7f632c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f634015b990 0x7f63401e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63400cd2d0 0x7f63401f50b0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 --2- 172.21.15.181:0/3797773261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63400cfc60 0x7f6340158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 >> 172.21.15.181:0/3797773261 conn(0x7f634009a9a0 msgr2=0x7f634015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:08.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 shutdown_connections 2023-10-16T11:19:08.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:08.916+0000 7f634694b700 1 -- 172.21.15.181:0/3797773261 wait complete. 2023-10-16T11:19:08.930 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:19:08.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:09.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.053+0000 7f8c8d2b3700 1 Processor -- start 2023-10-16T11:19:09.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.053+0000 7f8c8d2b3700 1 -- start start 2023-10-16T11:19:09.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c8d2b3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c8d2b3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c880af450 0x7f8c8806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c8d2b3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8806f870 con 0x7f8c8806db80 2023-10-16T11:19:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c8d2b3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c880d1160 con 0x7f8c880af450 2023-10-16T11:19:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c8d2b3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880cc8e0 con 0x7f8c8815dfd0 2023-10-16T11:19:09.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c86ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c867fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c880af450 0x7f8c8806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c877fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8c8806db80 0x7f8c880cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:49912/0 (socket says 172.21.15.181:49912) 2023-10-16T11:19:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c86ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:51914/0 (socket says 172.21.15.181:51914) 2023-10-16T11:19:09.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c867fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c880af450 0x7f8c8806d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:59940/0 (socket says 172.21.15.181:59940) 2023-10-16T11:19:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c877fe700 1 -- 172.21.15.181:0/2327499880 learned_addr learned my addr 172.21.15.181:0/2327499880 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c86ffd700 1 -- 172.21.15.181:0/2327499880 >> v2:172.21.15.190:3300/0 conn(0x7f8c880af450 msgr2=0x7f8c8806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c86ffd700 1 --2- 172.21.15.181:0/2327499880 >> v2:172.21.15.190:3300/0 conn(0x7f8c880af450 0x7f8c8806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c86ffd700 1 -- 172.21.15.181:0/2327499880 >> v1:172.21.15.190:6789/0 conn(0x7f8c8806db80 legacy=0x7f8c880cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c86ffd700 1 -- 172.21.15.181:0/2327499880 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880ccbc0 con 0x7f8c8815dfd0 2023-10-16T11:19:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c85ffb700 1 -- 172.21.15.181:0/2327499880 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422220243 0 0) 0x7f8c8806f870 con 0x7f8c8806db80 2023-10-16T11:19:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.054+0000 7f8c867fc700 1 --2- 172.21.15.181:0/2327499880 >> v2:172.21.15.190:3300/0 conn(0x7f8c880af450 0x7f8c8806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c86ffd700 1 --2- 172.21.15.181:0/2327499880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880af9a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009820 tx=0x7f8c78009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d049268b23e1ab51 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c85ffb700 1 -- 172.21.15.181:0/2327499880 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78005e30 con 0x7f8c8815dfd0 2023-10-16T11:19:09.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c85ffb700 1 -- 172.21.15.181:0/2327499880 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78006030 con 0x7f8c8815dfd0 2023-10-16T11:19:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c85ffb700 1 -- 172.21.15.181:0/2327499880 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780063c0 con 0x7f8c8815dfd0 2023-10-16T11:19:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/2327499880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 msgr2=0x7f8c880af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/2327499880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880af9a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009820 tx=0x7f8c78009b30 comp rx=0 tx=0).stop 2023-10-16T11:19:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/2327499880 shutdown_connections 2023-10-16T11:19:09.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/2327499880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880af9a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/2327499880 >> v2:172.21.15.190:3300/0 conn(0x7f8c880af450 0x7f8c8806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/2327499880 >> 172.21.15.181:0/2327499880 conn(0x7f8c880d04a0 msgr2=0x7f8c880d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/2327499880 shutdown_connections 2023-10-16T11:19:09.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.055+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/2327499880 wait complete. 2023-10-16T11:19:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.057+0000 7f8c8d2b3700 1 Processor -- start 2023-10-16T11:19:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.057+0000 7f8c8d2b3700 1 -- start start 2023-10-16T11:19:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c8d2b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c8806db80 0x7f8c8814a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c8d2b3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c880af450 0x7f8c880c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c8d2b3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c8d2b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880b4f60 con 0x7f8c8806db80 2023-10-16T11:19:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c8d2b3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c4510 con 0x7f8c880af450 2023-10-16T11:19:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c8d2b3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c6ac0 con 0x7f8c8815dfd0 2023-10-16T11:19:09.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c8806db80 0x7f8c8814a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c8806db80 0x7f8c8814a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54894/0 (socket says 172.21.15.181:54894) 2023-10-16T11:19:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 -- 172.21.15.181:0/3059396751 learned_addr learned my addr 172.21.15.181:0/3059396751 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c867fc700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c880af450 0x7f8c880c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c877fe700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 -- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 msgr2=0x7f8c880aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 -- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c880af450 msgr2=0x7f8c880c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c880af450 0x7f8c880c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.058+0000 7f8c86ffd700 1 -- 172.21.15.181:0/3059396751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c0094a0 con 0x7f8c8806db80 2023-10-16T11:19:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.059+0000 7f8c86ffd700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c8806db80 0x7f8c8814a7c0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f8c78000c00 tx=0x7f8c78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:09.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.059+0000 7f8c6f7fe700 1 -- 172.21.15.181:0/3059396751 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780063a0 con 0x7f8c8806db80 2023-10-16T11:19:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.059+0000 7f8c6f7fe700 1 -- 172.21.15.181:0/3059396751 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78006500 con 0x7f8c8806db80 2023-10-16T11:19:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.059+0000 7f8c6f7fe700 1 -- 172.21.15.181:0/3059396751 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78006890 con 0x7f8c8806db80 2023-10-16T11:19:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.059+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c78009430 con 0x7f8c8806db80 2023-10-16T11:19:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.059+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c880b2d40 con 0x7f8c8806db80 2023-10-16T11:19:09.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.060+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c880c6fe0 con 0x7f8c8806db80 2023-10-16T11:19:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.060+0000 7f8c6f7fe700 1 -- 172.21.15.181:0/3059396751 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8c78022070 con 0x7f8c8806db80 2023-10-16T11:19:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.061+0000 7f8c6f7fe700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 0x7f8c7406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.061+0000 7f8c867fc700 1 -- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 msgr2=0x7f8c7406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:09.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.061+0000 7f8c6f7fe700 1 -- 172.21.15.181:0/3059396751 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7f8c7808b3a0 con 0x7f8c8806db80 2023-10-16T11:19:09.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.061+0000 7f8c867fc700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 0x7f8c7406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:09.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.065+0000 7f8c6f7fe700 1 -- 172.21.15.181:0/3059396751 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8c78055d90 con 0x7f8c8806db80 2023-10-16T11:19:09.140 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:09.138+0000 7fbda211c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:19:09.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.262+0000 7f8c867fc700 1 -- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 msgr2=0x7f8c7406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:09.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.262+0000 7f8c867fc700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 0x7f8c7406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:09.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.309+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c880b4ce0 con 0x7f8c8806db80 2023-10-16T11:19:09.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.309+0000 7f8c6f7fe700 1 -- 172.21.15.181:0/3059396751 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f8c7802a090 con 0x7f8c8806db80 2023-10-16T11:19:09.310 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:09.311 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3757723265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3757723265},{"type":"v1","addr":"172.21.15.190:6834","nonce":3757723265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2012626771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2012626771},{"type":"v1","addr":"172.21.15.190:6835","nonce":2012626771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10782,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3218502962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3218502962},{"type":"v1","addr":"172.21.15.181:6836","nonce":3218502962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:17:39.436499+0000","modified":"2023-10-16T11:17:42.454721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10785":{"gid":10785,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3625484950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3625484950},{"type":"v1","addr":"172.21.15.181:6837","nonce":3625484950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:19:09.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.311+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 msgr2=0x7f8c7406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:09.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.311+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 0x7f8c7406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.311+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c8806db80 msgr2=0x7f8c8814a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.311+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c8806db80 0x7f8c8814a7c0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f8c78000c00 tx=0x7f8c78004750 comp rx=0 tx=0).stop 2023-10-16T11:19:09.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 shutdown_connections 2023-10-16T11:19:09.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7f8c7406b140 0x7f8c7406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8815dfd0 0x7f8c880aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c8806db80 0x7f8c8814a7c0 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 --2- 172.21.15.181:0/3059396751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c880af450 0x7f8c880c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 >> 172.21.15.181:0/3059396751 conn(0x7f8c880d04a0 msgr2=0x7f8c880ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:09.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 shutdown_connections 2023-10-16T11:19:09.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.312+0000 7f8c8d2b3700 1 -- 172.21.15.181:0/3059396751 wait complete. 2023-10-16T11:19:09.315 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 102 2023-10-16T11:19:09.325 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:09.445 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.445+0000 7f4bd423a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:19:09.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.447+0000 7efcaba92700 1 Processor -- start 2023-10-16T11:19:09.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.447+0000 7efcaba92700 1 -- start start 2023-10-16T11:19:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efcaba92700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efca415eab0 0x7efca40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efcaba92700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efca40cb0c0 0x7efca415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efcaba92700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca406ed00 con 0x7efca415e6d0 2023-10-16T11:19:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efcaba92700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca40d02d0 con 0x7efca415eab0 2023-10-16T11:19:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efcaba92700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca415ad40 con 0x7efca40cb0c0 2023-10-16T11:19:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efca902d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efca415eab0 0x7efca40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efca982e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efca415e6d0 0x7efca415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37750/0 (socket says 172.21.15.181:37750) 2023-10-16T11:19:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.448+0000 7efca982e700 1 -- 172.21.15.181:0/4084908208 learned_addr learned my addr 172.21.15.181:0/4084908208 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efcaa02f700 1 --2- 172.21.15.181:0/4084908208 >> v2:172.21.15.190:3300/0 conn(0x7efca40cb0c0 0x7efca415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca902d700 1 -- 172.21.15.181:0/4084908208 >> v2:172.21.15.190:3300/0 conn(0x7efca40cb0c0 msgr2=0x7efca415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca902d700 1 --2- 172.21.15.181:0/4084908208 >> v2:172.21.15.190:3300/0 conn(0x7efca40cb0c0 0x7efca415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca902d700 1 -- 172.21.15.181:0/4084908208 >> v1:172.21.15.181:6789/0 conn(0x7efca415e6d0 legacy=0x7efca415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca902d700 1 -- 172.21.15.181:0/4084908208 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca406e0f0 con 0x7efca415eab0 2023-10-16T11:19:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca882c700 1 -- 172.21.15.181:0/4084908208 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718076865 0 0) 0x7efca406ed00 con 0x7efca415e6d0 2023-10-16T11:19:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca902d700 1 --2- 172.21.15.181:0/4084908208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca40d0fc0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7efca4071650 tx=0x7efc94009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580c970264eb4f23 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca882c700 1 -- 172.21.15.181:0/4084908208 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94011830 con 0x7efca415eab0 2023-10-16T11:19:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca882c700 1 -- 172.21.15.181:0/4084908208 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc94011990 con 0x7efca415eab0 2023-10-16T11:19:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.449+0000 7efca882c700 1 -- 172.21.15.181:0/4084908208 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94011d20 con 0x7efca415eab0 2023-10-16T11:19:09.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 -- 172.21.15.181:0/4084908208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 msgr2=0x7efca40d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 --2- 172.21.15.181:0/4084908208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca40d0fc0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7efca4071650 tx=0x7efc94009b20 comp rx=0 tx=0).stop 2023-10-16T11:19:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 -- 172.21.15.181:0/4084908208 shutdown_connections 2023-10-16T11:19:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 --2- 172.21.15.181:0/4084908208 >> v2:172.21.15.190:3300/0 conn(0x7efca40cb0c0 0x7efca415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 --2- 172.21.15.181:0/4084908208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca40d0fc0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 -- 172.21.15.181:0/4084908208 >> 172.21.15.181:0/4084908208 conn(0x7efca40d04a0 msgr2=0x7efca40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 -- 172.21.15.181:0/4084908208 shutdown_connections 2023-10-16T11:19:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.450+0000 7efcaba92700 1 -- 172.21.15.181:0/4084908208 wait complete. 2023-10-16T11:19:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.451+0000 7efcaba92700 1 Processor -- start 2023-10-16T11:19:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.451+0000 7efcaba92700 1 -- start start 2023-10-16T11:19:09.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaba92700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efca40cb0c0 0x7efca40c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaba92700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca415e6d0 0x7efca415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaba92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca415c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaba92700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40b3c20 con 0x7efca415eab0 2023-10-16T11:19:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaba92700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41f3e10 con 0x7efca415e6d0 2023-10-16T11:19:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaba92700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca40b32f0 con 0x7efca40cb0c0 2023-10-16T11:19:09.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca415c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54914/0 (socket says 172.21.15.181:54914) 2023-10-16T11:19:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 -- 172.21.15.181:0/2955280762 learned_addr learned my addr 172.21.15.181:0/2955280762 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efca982e700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efca40cb0c0 0x7efca40c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efca902d700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca415e6d0 0x7efca415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 -- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efca40cb0c0 msgr2=0x7efca40c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efca40cb0c0 0x7efca40c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 -- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca415e6d0 msgr2=0x7efca415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca415e6d0 0x7efca415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 -- 172.21.15.181:0/2955280762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940094a0 con 0x7efca415eab0 2023-10-16T11:19:09.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaa02f700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca415c670 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00efc0 tx=0x7efc9c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:09.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00cad0 con 0x7efca415eab0 2023-10-16T11:19:09.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c00f040 con 0x7efca415eab0 2023-10-16T11:19:09.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca40b35d0 con 0x7efca415eab0 2023-10-16T11:19:09.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.452+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00ce70 con 0x7efca415eab0 2023-10-16T11:19:09.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.453+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca40c5fb0 con 0x7efca415eab0 2023-10-16T11:19:09.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.453+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40c61a0 con 0x7efca415eab0 2023-10-16T11:19:09.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.454+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 135) v1 ==== 89445+0+0 (secure 0 0 0) 0x7efc9c012430 con 0x7efca415eab0 2023-10-16T11:19:09.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.454+0000 7efc9a7fc700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efc8806b140 0x7efc8806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.454+0000 7efca982e700 1 -- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efc8806b140 msgr2=0x7efc8806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1119809348 2023-10-16T11:19:09.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.454+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6707+0+0 (secure 0 0 0) 0x7efc9c016070 con 0x7efca415eab0 2023-10-16T11:19:09.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.454+0000 7efca982e700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efc8806b140 0x7efc8806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:09.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.459+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efc9c056e70 con 0x7efca415eab0 2023-10-16T11:19:09.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.470+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 136) v1 ==== 88940+0+0 (secure 0 0 0) 0x7efc9c057240 con 0x7efca415eab0 2023-10-16T11:19:09.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.470+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efc8806b140 msgr2=0x7efc8806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:09.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.470+0000 7efc9a7fc700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efc8806b140 0x7efc8806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.532 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4fa26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.532 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4fa26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.532 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4fa26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.532 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4fa26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.533 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4fa26700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.533 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4ca20700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.533 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4ca20700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.533 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4ca20700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.533 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4ca20700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.534 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:09.531+0000 7f4b4ca20700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:09.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.711+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efca4067d20 con 0x7efca415eab0 2023-10-16T11:19:09.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.712+0000 7efc9a7fc700 1 -- 172.21.15.181:0/2955280762 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4935 (secure 0 0 0) 0x7efc9c0730b0 con 0x7efca415eab0 2023-10-16T11:19:09.713 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:09.713 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3757723265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3757723265},{"type":"v1","addr":"172.21.15.190:6834","nonce":3757723265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2012626771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2012626771},{"type":"v1","addr":"172.21.15.190:6835","nonce":2012626771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10782,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3218502962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3218502962},{"type":"v1","addr":"172.21.15.181:6836","nonce":3218502962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:17:39.436499+0000","modified":"2023-10-16T11:17:42.454721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10785":{"gid":10785,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3625484950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3625484950},{"type":"v1","addr":"172.21.15.181:6837","nonce":3625484950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-16T11:19:09.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 msgr2=0x7efca415c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca415c670 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00efc0 tx=0x7efc9c00c0c0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 shutdown_connections 2023-10-16T11:19:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:6832/1119809348,v1:172.21.15.181:6833/1119809348] conn(0x7efc8806b140 0x7efc8806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efca40cb0c0 0x7efca40c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efca415eab0 0x7efca415c670 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 --2- 172.21.15.181:0/2955280762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efca415e6d0 0x7efca415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 >> 172.21.15.181:0/2955280762 conn(0x7efca40d04a0 msgr2=0x7efca40ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:09.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 shutdown_connections 2023-10-16T11:19:09.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.714+0000 7efcaba92700 1 -- 172.21.15.181:0/2955280762 wait complete. 2023-10-16T11:19:09.717 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 102 2023-10-16T11:19:09.727 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.848+0000 7ff706286700 1 Processor -- start 2023-10-16T11:19:09.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.848+0000 7ff706286700 1 -- start start 2023-10-16T11:19:09.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff706286700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff7000cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff706286700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7000b22a0 0x7ff70015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff706286700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff700066c90 con 0x7ff70006e010 2023-10-16T11:19:09.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff706286700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff700066740 con 0x7ff7000b22a0 2023-10-16T11:19:09.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff706286700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7001603f0 con 0x7ff70015b8b0 2023-10-16T11:19:09.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6ff7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff7000b22a0 0x7ff70015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff704823700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff70006e010 0x7ff70006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:50012/0 (socket says 172.21.15.181:50012) 2023-10-16T11:19:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff704823700 1 -- 172.21.15.181:0/2429804631 learned_addr learned my addr 172.21.15.181:0/2429804631 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6fffff700 1 --2- 172.21.15.181:0/2429804631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff7000cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6feffd700 1 -- 172.21.15.181:0/2429804631 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280353951 0 0) 0x7ff6f80014e0 con 0x7ff70006e010 2023-10-16T11:19:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6feffd700 1 -- 172.21.15.181:0/2429804631 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6ec0035b0 con 0x7ff70006e010 2023-10-16T11:19:09.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6fffff700 1 -- 172.21.15.181:0/2429804631 >> v2:172.21.15.190:3300/0 conn(0x7ff7000b22a0 msgr2=0x7ff70015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6fffff700 1 --2- 172.21.15.181:0/2429804631 >> v2:172.21.15.190:3300/0 conn(0x7ff7000b22a0 0x7ff70015feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6fffff700 1 -- 172.21.15.181:0/2429804631 >> v1:172.21.15.190:6789/0 conn(0x7ff70006e010 legacy=0x7ff70006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6fffff700 1 -- 172.21.15.181:0/2429804631 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000669b0 con 0x7ff70015b8b0 2023-10-16T11:19:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.849+0000 7ff6ff7fe700 1 --2- 172.21.15.181:0/2429804631 >> v2:172.21.15.190:3300/0 conn(0x7ff7000b22a0 0x7ff70015feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:09.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff6fffff700 1 --2- 172.21.15.181:0/2429804631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff7000cc160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009840 tx=0x7ff6f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155e3c1d6601d74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff6feffd700 1 -- 172.21.15.181:0/2429804631 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011830 con 0x7ff70015b8b0 2023-10-16T11:19:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff6feffd700 1 -- 172.21.15.181:0/2429804631 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0011990 con 0x7ff70015b8b0 2023-10-16T11:19:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff6feffd700 1 -- 172.21.15.181:0/2429804631 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011d20 con 0x7ff70015b8b0 2023-10-16T11:19:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 -- 172.21.15.181:0/2429804631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 msgr2=0x7ff7000cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 --2- 172.21.15.181:0/2429804631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff7000cc160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009840 tx=0x7ff6f0009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 -- 172.21.15.181:0/2429804631 shutdown_connections 2023-10-16T11:19:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 --2- 172.21.15.181:0/2429804631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff7000cc160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 --2- 172.21.15.181:0/2429804631 >> v2:172.21.15.190:3300/0 conn(0x7ff7000b22a0 0x7ff70015feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 -- 172.21.15.181:0/2429804631 >> 172.21.15.181:0/2429804631 conn(0x7ff7000b1780 msgr2=0x7ff7000b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 -- 172.21.15.181:0/2429804631 shutdown_connections 2023-10-16T11:19:09.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.850+0000 7ff706286700 1 -- 172.21.15.181:0/2429804631 wait complete. 2023-10-16T11:19:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.851+0000 7ff706286700 1 Processor -- start 2023-10-16T11:19:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff706286700 1 -- start start 2023-10-16T11:19:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff706286700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff70006e010 0x7ff7001f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff706286700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7000b22a0 0x7ff7001f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff706286700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff70015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff706286700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7001f5470 con 0x7ff7000b22a0 2023-10-16T11:19:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff706286700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7001f1470 con 0x7ff70006e010 2023-10-16T11:19:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff706286700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70015cef0 con 0x7ff70015b8b0 2023-10-16T11:19:09.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff6ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7000b22a0 0x7ff7001f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff6ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7000b22a0 0x7ff7001f0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54994/0 (socket says 172.21.15.181:54994) 2023-10-16T11:19:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff6fffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff70006e010 0x7ff7001f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff6ff7fe700 1 -- 172.21.15.181:0/2685264888 learned_addr learned my addr 172.21.15.181:0/2685264888 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.852+0000 7ff704823700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff70015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:09.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6ff7fe700 1 -- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 msgr2=0x7ff70015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6ff7fe700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff70015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6ff7fe700 1 -- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff70006e010 msgr2=0x7ff7001f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6ff7fe700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff70006e010 0x7ff7001f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6ff7fe700 1 -- 172.21.15.181:0/2685264888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80094a0 con 0x7ff7000b22a0 2023-10-16T11:19:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6fffff700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff70006e010 0x7ff7001f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:09.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6ff7fe700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7000b22a0 0x7ff7001f0f30 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800ebe0 tx=0x7ff6e800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6fcff9700 1 -- 172.21.15.181:0/2685264888 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800c850 con 0x7ff7000b22a0 2023-10-16T11:19:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6fcff9700 1 -- 172.21.15.181:0/2685264888 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e800c9b0 con 0x7ff7000b22a0 2023-10-16T11:19:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f00094a0 con 0x7ff7000b22a0 2023-10-16T11:19:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff6fcff9700 1 -- 172.21.15.181:0/2685264888 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800cd40 con 0x7ff7000b22a0 2023-10-16T11:19:09.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.853+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7001f1980 con 0x7ff7000b22a0 2023-10-16T11:19:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.854+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70015d060 con 0x7ff7000b22a0 2023-10-16T11:19:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.854+0000 7ff6fcff9700 1 -- 172.21.15.181:0/2685264888 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 136) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff6e8013470 con 0x7ff7000b22a0 2023-10-16T11:19:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.855+0000 7ff6fcff9700 1 -- 172.21.15.181:0/2685264888 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 7093+0+0 (secure 0 0 0) 0x7ff6e8089670 con 0x7ff7000b22a0 2023-10-16T11:19:09.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:09.859+0000 7ff6fcff9700 1 -- 172.21.15.181:0/2685264888 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff6e8054600 con 0x7ff7000b22a0 2023-10-16T11:19:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.054+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff70015cce0 con 0x7ff7000b22a0 2023-10-16T11:19:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.055+0000 7ff6fcff9700 1 -- 172.21.15.181:0/2685264888 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) v1 ==== 74+0+18670 (secure 0 0 0) 0x7ff6e800cea0 con 0x7ff7000b22a0 2023-10-16T11:19:10.057 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:10.057 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":205,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:09.459313+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:17:37.851564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-16T11:17:38.394398+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":204,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.7","mappings":[{"from":0,"to":2}]},{"pgid":"25.2d","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/509806514":"2023-10-17T11:19:09.459266+0000","172.21.15.181:6832/1119809348":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/956037434":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/1940715811":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/665777945":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/2092158185":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/1746525923":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3179161179":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/3361524667":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/221011703":"2023-10-17T11:19:09.459266+0000","172.21.15.181:6833/1119809348":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/1822518943":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6833/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6832/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3260777321":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/2465214735":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/3430010207":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/969172037":"2023-10-17T11:17:53.265498+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.057+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7000b22a0 msgr2=0x7ff7001f0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.057+0000 7ff706286700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7000b22a0 0x7ff7001f0f30 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800ebe0 tx=0x7ff6e800eef0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.057+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 shutdown_connections 2023-10-16T11:19:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.057+0000 7ff706286700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff70015b8b0 0x7ff70015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.057+0000 7ff706286700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff70006e010 0x7ff7001f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.057+0000 7ff706286700 1 --2- 172.21.15.181:0/2685264888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7000b22a0 0x7ff7001f0f30 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.058+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 >> 172.21.15.181:0/2685264888 conn(0x7ff7000b1780 msgr2=0x7ff70015a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.058+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 shutdown_connections 2023-10-16T11:19:10.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.058+0000 7ff706286700 1 -- 172.21.15.181:0/2685264888 wait complete. 2023-10-16T11:19:10.071 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:19:10.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.192+0000 7f2d7b44b700 1 Processor -- start 2023-10-16T11:19:10.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.192+0000 7f2d7b44b700 1 -- start start 2023-10-16T11:19:10.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d7b44b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2d7415df60 0x7f2d740cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d7b44b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d740d0fc0 0x7f2d740cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d7b44b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d740ccf50 con 0x7f2d7406a8a0 2023-10-16T11:19:10.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d7b44b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740af9c0 con 0x7f2d7415df60 2023-10-16T11:19:10.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d7b44b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740cc600 con 0x7f2d740d0fc0 2023-10-16T11:19:10.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d789e6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d740d0fc0 0x7f2d740cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d791e7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2d7415df60 0x7f2d740cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d789e6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d740d0fc0 0x7f2d740cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:52026/0 (socket says 172.21.15.181:52026) 2023-10-16T11:19:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d791e7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2d7415df60 0x7f2d740cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:60050/0 (socket says 172.21.15.181:60050) 2023-10-16T11:19:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d799e8700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2d7406a8a0 0x7f2d7406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:50020/0 (socket says 172.21.15.181:50020) 2023-10-16T11:19:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.193+0000 7f2d789e6700 1 -- 172.21.15.181:0/2527908811 learned_addr learned my addr 172.21.15.181:0/2527908811 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d789e6700 1 -- 172.21.15.181:0/2527908811 >> v2:172.21.15.190:3300/0 conn(0x7f2d7415df60 msgr2=0x7f2d740cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:10.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d789e6700 1 --2- 172.21.15.181:0/2527908811 >> v2:172.21.15.190:3300/0 conn(0x7f2d7415df60 0x7f2d740cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d789e6700 1 -- 172.21.15.181:0/2527908811 >> v1:172.21.15.190:6789/0 conn(0x7f2d7406a8a0 legacy=0x7f2d7406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d6bfff700 1 -- 172.21.15.181:0/2527908811 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963063580 0 0) 0x7f2d740ccf50 con 0x7f2d7406a8a0 2023-10-16T11:19:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d789e6700 1 -- 172.21.15.181:0/2527908811 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740cc8a0 con 0x7f2d740d0fc0 2023-10-16T11:19:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d791e7700 1 --2- 172.21.15.181:0/2527908811 >> v2:172.21.15.190:3300/0 conn(0x7f2d7415df60 0x7f2d740cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:10.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d789e6700 1 --2- 172.21.15.181:0/2527908811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d740d0fc0 0x7f2d740cd6e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d640098f0 tx=0x7f2d64009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b355496a7c91d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:10.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d6bfff700 1 -- 172.21.15.181:0/2527908811 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64011830 con 0x7f2d740d0fc0 2023-10-16T11:19:10.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d6bfff700 1 -- 172.21.15.181:0/2527908811 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d64011990 con 0x7f2d740d0fc0 2023-10-16T11:19:10.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.194+0000 7f2d6bfff700 1 -- 172.21.15.181:0/2527908811 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64011d20 con 0x7f2d740d0fc0 2023-10-16T11:19:10.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2527908811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d740d0fc0 msgr2=0x7f2d740cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 --2- 172.21.15.181:0/2527908811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d740d0fc0 0x7f2d740cd6e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d640098f0 tx=0x7f2d64009c00 comp rx=0 tx=0).stop 2023-10-16T11:19:10.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2527908811 shutdown_connections 2023-10-16T11:19:10.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 --2- 172.21.15.181:0/2527908811 >> v2:172.21.15.190:3300/0 conn(0x7f2d7415df60 0x7f2d740cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 --2- 172.21.15.181:0/2527908811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d740d0fc0 0x7f2d740cd6e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2527908811 >> 172.21.15.181:0/2527908811 conn(0x7f2d740d04a0 msgr2=0x7f2d740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:10.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2527908811 shutdown_connections 2023-10-16T11:19:10.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.195+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2527908811 wait complete. 2023-10-16T11:19:10.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.196+0000 7f2d7b44b700 1 Processor -- start 2023-10-16T11:19:10.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.196+0000 7f2d7b44b700 1 -- start start 2023-10-16T11:19:10.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d7b44b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7406a8a0 0x7f2d74156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d7b44b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d740d0fc0 0x7f2d74154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d7b44b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d7415df60 0x7f2d740ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d7b44b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74159080 con 0x7f2d7406a8a0 2023-10-16T11:19:10.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d7b44b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740b53c0 con 0x7f2d740d0fc0 2023-10-16T11:19:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d7b44b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740b4020 con 0x7f2d7415df60 2023-10-16T11:19:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7406a8a0 0x7f2d74156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7406a8a0 0x7f2d74156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55004/0 (socket says 172.21.15.181:55004) 2023-10-16T11:19:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 -- 172.21.15.181:0/2293619278 learned_addr learned my addr 172.21.15.181:0/2293619278 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:10.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d789e6700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d740d0fc0 0x7f2d74154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d799e8700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d7415df60 0x7f2d740ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 -- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d7415df60 msgr2=0x7f2d740ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d7415df60 0x7f2d740ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 -- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d740d0fc0 msgr2=0x7f2d74154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d740d0fc0 0x7f2d74154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 -- 172.21.15.181:0/2293619278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009430 con 0x7f2d7406a8a0 2023-10-16T11:19:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d789e6700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d740d0fc0 0x7f2d74154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d791e7700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7406a8a0 0x7f2d74156a80 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000ebe0 tx=0x7f2d70009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:10.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d69ffb700 1 -- 172.21.15.181:0/2293619278 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70005a20 con 0x7f2d7406a8a0 2023-10-16T11:19:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d69ffb700 1 -- 172.21.15.181:0/2293619278 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d70005b80 con 0x7f2d7406a8a0 2023-10-16T11:19:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d69ffb700 1 -- 172.21.15.181:0/2293619278 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000c910 con 0x7f2d7406a8a0 2023-10-16T11:19:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.197+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d64009550 con 0x7f2d7406a8a0 2023-10-16T11:19:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.198+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d74158ca0 con 0x7f2d7406a8a0 2023-10-16T11:19:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.199+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d74158e30 con 0x7f2d7406a8a0 2023-10-16T11:19:10.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.199+0000 7f2d69ffb700 1 -- 172.21.15.181:0/2293619278 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 136) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f2d7000ca70 con 0x7f2d7406a8a0 2023-10-16T11:19:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.200+0000 7f2d69ffb700 1 -- 172.21.15.181:0/2293619278 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f2d70089950 con 0x7f2d7406a8a0 2023-10-16T11:19:10.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.204+0000 7f2d69ffb700 1 -- 172.21.15.181:0/2293619278 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2d700548e0 con 0x7f2d7406a8a0 2023-10-16T11:19:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.441+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2d741592f0 con 0x7f2d7406a8a0 2023-10-16T11:19:10.443 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:19:10.442+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:19:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.443+0000 7f2d69ffb700 1 -- 172.21.15.181:0/2293619278 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2d70020090 con 0x7f2d7406a8a0 2023-10-16T11:19:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.446+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7406a8a0 msgr2=0x7f2d74156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.446+0000 7f2d7b44b700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7406a8a0 0x7f2d74156a80 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000ebe0 tx=0x7f2d70009ea0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.447+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 shutdown_connections 2023-10-16T11:19:10.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.447+0000 7f2d7b44b700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d7415df60 0x7f2d740ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.447+0000 7f2d7b44b700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d7406a8a0 0x7f2d74156a80 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.447+0000 7f2d7b44b700 1 --2- 172.21.15.181:0/2293619278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d740d0fc0 0x7f2d74154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.447+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 >> 172.21.15.181:0/2293619278 conn(0x7f2d740d04a0 msgr2=0x7f2d740ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:10.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.447+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 shutdown_connections 2023-10-16T11:19:10.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.447+0000 7f2d7b44b700 1 -- 172.21.15.181:0/2293619278 wait complete. 2023-10-16T11:19:10.450 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:19:10.462 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:19:10.487 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:19:10.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.589+0000 7f35d8e46700 1 Processor -- start 2023-10-16T11:19:10.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.589+0000 7f35d8e46700 1 -- start start 2023-10-16T11:19:10.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.590+0000 7f35d8e46700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f35d40b2010 0x7f35d40cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.590+0000 7f35d8e46700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35d40ce680 0x7f35d40ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.590+0000 7f35d8e46700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40ce1f0 con 0x7f35d4071be0 2023-10-16T11:19:10.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.590+0000 7f35d8e46700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d4072090 con 0x7f35d40ce680 2023-10-16T11:19:10.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.590+0000 7f35d8e46700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40ccda0 con 0x7f35d40b2010 2023-10-16T11:19:10.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.590+0000 7f35d2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35d40ce680 0x7f35d40ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d2d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35d40ce680 0x7f35d40ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55046/0 (socket says 172.21.15.181:55046) 2023-10-16T11:19:10.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d2d9d700 1 -- 172.21.15.181:0/1668655750 learned_addr learned my addr 172.21.15.181:0/1668655750 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:10.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d1d9b700 1 --1- 172.21.15.181:0/1668655750 >> v1:172.21.15.181:6789/0 conn(0x7f35d4071be0 0x7f35d40b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37890/0 (socket says 172.21.15.181:37890) 2023-10-16T11:19:10.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d259c700 1 --2- 172.21.15.181:0/1668655750 >> v2:172.21.15.190:3300/0 conn(0x7f35d40b2010 0x7f35d40cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d2d9d700 1 -- 172.21.15.181:0/1668655750 >> v2:172.21.15.190:3300/0 conn(0x7f35d40b2010 msgr2=0x7f35d40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:10.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d2d9d700 1 --2- 172.21.15.181:0/1668655750 >> v2:172.21.15.190:3300/0 conn(0x7f35d40b2010 0x7f35d40cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d2d9d700 1 -- 172.21.15.181:0/1668655750 >> v1:172.21.15.181:6789/0 conn(0x7f35d4071be0 legacy=0x7f35d40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d2d9d700 1 -- 172.21.15.181:0/1668655750 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40727e0 con 0x7f35d40ce680 2023-10-16T11:19:10.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d159a700 1 -- 172.21.15.181:0/1668655750 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325150916 0 0) 0x7f35d40ce1f0 con 0x7f35d4071be0 2023-10-16T11:19:10.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d259c700 1 --2- 172.21.15.181:0/1668655750 >> v2:172.21.15.190:3300/0 conn(0x7f35d40b2010 0x7f35d40cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:10.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d2d9d700 1 --2- 172.21.15.181:0/1668655750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40ceaf0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f35d4070430 tx=0x7f35cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f47749d9b61823 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:10.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d159a700 1 -- 172.21.15.181:0/1668655750 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00e070 con 0x7f35d40ce680 2023-10-16T11:19:10.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d159a700 1 -- 172.21.15.181:0/1668655750 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc00b9c0 con 0x7f35d40ce680 2023-10-16T11:19:10.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.591+0000 7f35d159a700 1 -- 172.21.15.181:0/1668655750 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00bd70 con 0x7f35d40ce680 2023-10-16T11:19:10.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 -- 172.21.15.181:0/1668655750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 msgr2=0x7f35d40ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 --2- 172.21.15.181:0/1668655750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40ceaf0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f35d4070430 tx=0x7f35cc00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:10.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 -- 172.21.15.181:0/1668655750 shutdown_connections 2023-10-16T11:19:10.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 --2- 172.21.15.181:0/1668655750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40ceaf0 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 --2- 172.21.15.181:0/1668655750 >> v2:172.21.15.190:3300/0 conn(0x7f35d40b2010 0x7f35d40cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 -- 172.21.15.181:0/1668655750 >> 172.21.15.181:0/1668655750 conn(0x7f35d40cd010 msgr2=0x7f35d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:10.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 -- 172.21.15.181:0/1668655750 shutdown_connections 2023-10-16T11:19:10.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.592+0000 7f35d8e46700 1 -- 172.21.15.181:0/1668655750 wait complete. 2023-10-16T11:19:10.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.593+0000 7f35d8e46700 1 Processor -- start 2023-10-16T11:19:10.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.593+0000 7f35d8e46700 1 -- start start 2023-10-16T11:19:10.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d8e46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d4071be0 0x7f35d415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d8e46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d40b2010 0x7f35d40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d8e46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d8e46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40b1350 con 0x7f35d40ce680 2023-10-16T11:19:10.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d8e46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40acf80 con 0x7f35d40b2010 2023-10-16T11:19:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d8e46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40ad260 con 0x7f35d4071be0 2023-10-16T11:19:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55050/0 (socket says 172.21.15.181:55050) 2023-10-16T11:19:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 -- 172.21.15.181:0/3888871525 learned_addr learned my addr 172.21.15.181:0/3888871525 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:10.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d1d9b700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d40b2010 0x7f35d40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d259c700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d4071be0 0x7f35d415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 -- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d4071be0 msgr2=0x7f35d415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d4071be0 0x7f35d415d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 -- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d40b2010 msgr2=0x7f35d40ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:10.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d40b2010 0x7f35d40ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:10.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 -- 172.21.15.181:0/3888871525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc009b30 con 0x7f35d40ce680 2023-10-16T11:19:10.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35d2d9d700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40b0e10 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b370 tx=0x7f35cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:10.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.594+0000 7f35caffd700 1 -- 172.21.15.181:0/3888871525 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0092e0 con 0x7f35d40ce680 2023-10-16T11:19:10.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.595+0000 7f35caffd700 1 -- 172.21.15.181:0/3888871525 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc0099b0 con 0x7f35d40ce680 2023-10-16T11:19:10.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.595+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d40ad540 con 0x7f35d40ce680 2023-10-16T11:19:10.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.595+0000 7f35caffd700 1 -- 172.21.15.181:0/3888871525 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc01c680 con 0x7f35d40ce680 2023-10-16T11:19:10.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.595+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40adb00 con 0x7f35d40ce680 2023-10-16T11:19:10.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.596+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d40adcc0 con 0x7f35d40ce680 2023-10-16T11:19:10.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.596+0000 7f35caffd700 1 -- 172.21.15.181:0/3888871525 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f35cc01c7e0 con 0x7f35d40ce680 2023-10-16T11:19:10.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.596+0000 7f35caffd700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f35b406ad50 0x7f35b406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:10.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.596+0000 7f35caffd700 1 -- 172.21.15.181:0/3888871525 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f35cc08b6c0 con 0x7f35d40ce680 2023-10-16T11:19:10.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.597+0000 7f35d259c700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f35b406ad50 0x7f35b406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:10.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.597+0000 7f35d259c700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f35b406ad50 0x7f35b406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f35bc006fd0 tx=0x7f35bc008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:10.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.601+0000 7f35caffd700 1 -- 172.21.15.181:0/3888871525 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f35cc056470 con 0x7f35d40ce680 2023-10-16T11:19:10.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:10.835+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f35d40b6510 con 0x7f35d40ce680 2023-10-16T11:19:11.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.472+0000 7f35caffd700 1 -- 172.21.15.181:0/3888871525 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) v1 ==== 104+0+0 (secure 0 0 0) 0x7f35cc059a90 con 0x7f35d40ce680 2023-10-16T11:19:11.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f35b406ad50 msgr2=0x7f35b406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:11.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f35b406ad50 0x7f35b406d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f35bc006fd0 tx=0x7f35bc008040 comp rx=0 tx=0).stop 2023-10-16T11:19:11.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 msgr2=0x7f35d40b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:11.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40b0e10 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b370 tx=0x7f35cc004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 shutdown_connections 2023-10-16T11:19:11.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f35b406ad50 0x7f35b406d210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d4071be0 0x7f35d415d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35d40ce680 0x7f35d40b0e10 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 --2- 172.21.15.181:0/3888871525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d40b2010 0x7f35d40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.475+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 >> 172.21.15.181:0/3888871525 conn(0x7f35d40cd010 msgr2=0x7f35d40cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.476+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 shutdown_connections 2023-10-16T11:19:11.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.476+0000 7f35d8e46700 1 -- 172.21.15.181:0/3888871525 wait complete. 2023-10-16T11:19:11.491 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:19:11.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.656+0000 7f5dfae47700 1 Processor -- start 2023-10-16T11:19:11.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.656+0000 7f5dfae47700 1 -- start start 2023-10-16T11:19:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.656+0000 7f5dfae47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.656+0000 7f5dfae47700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df415e430 con 0x7f5df415dee0 2023-10-16T11:19:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.656+0000 7f5dfae47700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df415f290 con 0x7f5df40cfef0 2023-10-16T11:19:11.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.656+0000 7f5dfae47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df415e680 con 0x7f5df415db00 2023-10-16T11:19:11.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5df415dee0 0x7f5df40cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37926/0 (socket says 172.21.15.181:37926) 2023-10-16T11:19:11.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df3fff700 1 -- 172.21.15.181:0/1846069651 learned_addr learned my addr 172.21.15.181:0/1846069651 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:11.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df8be3700 1 --2- 172.21.15.181:0/1846069651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:11.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897873591 0 0) 0x7f5df415e430 con 0x7f5df415dee0 2023-10-16T11:19:11.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de40057c0 con 0x7f5df415dee0 2023-10-16T11:19:11.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176034485 0 0) 0x7f5df415f290 con 0x7f5df40cfef0 2023-10-16T11:19:11.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5df415e430 con 0x7f5df40cfef0 2023-10-16T11:19:11.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1809796562 0 0) 0x7f5de40057c0 con 0x7f5df415dee0 2023-10-16T11:19:11.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df8be3700 1 -- 172.21.15.181:0/1846069651 >> v1:172.21.15.190:6789/0 conn(0x7f5df40cfef0 legacy=0x7f5df40b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:11.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df8be3700 1 -- 172.21.15.181:0/1846069651 >> v1:172.21.15.181:6789/0 conn(0x7f5df415dee0 legacy=0x7f5df40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:11.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df8be3700 1 -- 172.21.15.181:0/1846069651 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df415e960 con 0x7f5df415db00 2023-10-16T11:19:11.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df8be3700 1 --2- 172.21.15.181:0/1846069651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40b4040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5de800f9e0 tx=0x7f5de800fcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bae3a42482d4d2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:11.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.657+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0001670 con 0x7f5df415db00 2023-10-16T11:19:11.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de800f320 con 0x7f5df415db00 2023-10-16T11:19:11.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5df37fe700 1 -- 172.21.15.181:0/1846069651 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8004230 con 0x7f5df415db00 2023-10-16T11:19:11.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5dfae47700 1 -- 172.21.15.181:0/1846069651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 msgr2=0x7f5df40b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:11.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5dfae47700 1 --2- 172.21.15.181:0/1846069651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40b4040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5de800f9e0 tx=0x7f5de800fcf0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5dfae47700 1 -- 172.21.15.181:0/1846069651 shutdown_connections 2023-10-16T11:19:11.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5dfae47700 1 --2- 172.21.15.181:0/1846069651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40b4040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5dfae47700 1 -- 172.21.15.181:0/1846069651 >> 172.21.15.181:0/1846069651 conn(0x7f5df406aa80 msgr2=0x7f5df406ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:11.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5dfae47700 1 -- 172.21.15.181:0/1846069651 shutdown_connections 2023-10-16T11:19:11.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.658+0000 7f5dfae47700 1 -- 172.21.15.181:0/1846069651 wait complete. 2023-10-16T11:19:11.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.659+0000 7f5dfae47700 1 Processor -- start 2023-10-16T11:19:11.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.659+0000 7f5dfae47700 1 -- start start 2023-10-16T11:19:11.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5dfae47700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df40cfef0 0x7f5df41ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:11.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5dfae47700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:11.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5dfae47700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5df415dee0 0x7f5df415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:11.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5dfae47700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40c7b40 con 0x7f5df415dee0 2023-10-16T11:19:11.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5dfae47700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40c7de0 con 0x7f5df40cfef0 2023-10-16T11:19:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5dfae47700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40c80c0 con 0x7f5df415db00 2023-10-16T11:19:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5df93e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5df415dee0 0x7f5df415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5df93e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5df415dee0 0x7f5df415b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55086/0 (socket says 172.21.15.181:55086) 2023-10-16T11:19:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5df93e4700 1 -- 172.21.15.181:0/4090302683 learned_addr learned my addr 172.21.15.181:0/4090302683 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:11.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5df8be3700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df40cfef0 0x7f5df41ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.660+0000 7f5df3fff700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df93e4700 1 -- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 msgr2=0x7f5df40c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df93e4700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df93e4700 1 -- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df40cfef0 msgr2=0x7f5df41ef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:11.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df93e4700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df40cfef0 0x7f5df41ef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:11.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df93e4700 1 -- 172.21.15.181:0/4090302683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de800f610 con 0x7f5df415dee0 2023-10-16T11:19:11.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df8be3700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df40cfef0 0x7f5df41ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:11.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df93e4700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5df415dee0 0x7f5df415b330 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00cb50 tx=0x7f5dec00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:11.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec003c00 con 0x7f5df415dee0 2023-10-16T11:19:11.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec003d60 con 0x7f5df415dee0 2023-10-16T11:19:11.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00ab70 con 0x7f5df415dee0 2023-10-16T11:19:11.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df40c83a0 con 0x7f5df415dee0 2023-10-16T11:19:11.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.661+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df40c8960 con 0x7f5df415dee0 2023-10-16T11:19:11.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.662+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df406ae60 con 0x7f5df415dee0 2023-10-16T11:19:11.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.662+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 137) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5dec00acd0 con 0x7f5df415dee0 2023-10-16T11:19:11.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.663+0000 7f5df17fa700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5dd806ad50 0x7f5dd806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:11.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.663+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f5dec089900 con 0x7f5df415dee0 2023-10-16T11:19:11.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.663+0000 7f5df8be3700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5dd806ad50 0x7f5dd806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:11.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.663+0000 7f5df8be3700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5dd806ad50 0x7f5dd806d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5de80090b0 tx=0x7f5de8010000 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:11.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.667+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5dec0546b0 con 0x7f5df415dee0 2023-10-16T11:19:11.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:11.861+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5df404ebb0 con 0x7f5df415dee0 2023-10-16T11:19:12.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.476+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v207) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5dec057cd0 con 0x7f5df415dee0 2023-10-16T11:19:12.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.485+0000 7f5df17fa700 1 -- 172.21.15.181:0/4090302683 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 138) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5dec054a80 con 0x7f5df415dee0 2023-10-16T11:19:12.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.486+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5dd806ad50 msgr2=0x7f5dd806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:12.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.486+0000 7f5dfae47700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5dd806ad50 0x7f5dd806d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5de80090b0 tx=0x7f5de8010000 comp rx=0 tx=0).stop 2023-10-16T11:19:12.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.486+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5df415dee0 msgr2=0x7f5df415b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:12.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.486+0000 7f5dfae47700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5df415dee0 0x7f5df415b330 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00cb50 tx=0x7f5dec00cf10 comp rx=0 tx=0).stop 2023-10-16T11:19:12.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 shutdown_connections 2023-10-16T11:19:12.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5dd806ad50 0x7f5dd806d210 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5de80090b0 tx=0x7f5de8010000 comp rx=0 tx=0).stop 2023-10-16T11:19:12.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5df415dee0 0x7f5df415b330 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5df40cfef0 0x7f5df41ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 --2- 172.21.15.181:0/4090302683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5df415db00 0x7f5df40c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 >> 172.21.15.181:0/4090302683 conn(0x7f5df406aa80 msgr2=0x7f5df4070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:12.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 shutdown_connections 2023-10-16T11:19:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.487+0000 7f5dfae47700 1 -- 172.21.15.181:0/4090302683 wait complete. 2023-10-16T11:19:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:19:12.504 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:19:12.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.630+0000 7f4be9792700 1 Processor -- start 2023-10-16T11:19:12.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.630+0000 7f4be9792700 1 -- start start 2023-10-16T11:19:12.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be9792700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be41615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be9792700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4be40b2560 0x7f4be40d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be9792700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be40d4380 con 0x7f4be40b2c00 2023-10-16T11:19:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be9792700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be40cee80 con 0x7f4be40b2560 2023-10-16T11:19:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be9792700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40cf440 con 0x7f4be415dbe0 2023-10-16T11:19:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4be40b2560 0x7f4be40d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:12.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4be40b2560 0x7f4be40d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55094/0 (socket says 172.21.15.181:55094) 2023-10-16T11:19:12.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 -- 172.21.15.181:0/3974631881 learned_addr learned my addr 172.21.15.181:0/3974631881 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:12.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 -- 172.21.15.181:0/3974631881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 msgr2=0x7f4be41615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:12.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be2ffd700 1 --2- 172.21.15.181:0/3974631881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be41615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:12.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 --2- 172.21.15.181:0/3974631881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be41615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 -- 172.21.15.181:0/3974631881 >> v1:172.21.15.190:6789/0 conn(0x7f4be40b2c00 legacy=0x7f4be4161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 -- 172.21.15.181:0/3974631881 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40d45d0 con 0x7f4be40b2560 2023-10-16T11:19:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be37fe700 1 --2- 172.21.15.181:0/3974631881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be40d1430 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009e50 tx=0x7f4bdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f99515a56fb49b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be1ffb700 1 -- 172.21.15.181:0/3974631881 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00e070 con 0x7f4be40b2560 2023-10-16T11:19:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be1ffb700 1 -- 172.21.15.181:0/3974631881 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc003a20 con 0x7f4be40b2560 2023-10-16T11:19:12.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.631+0000 7f4be1ffb700 1 -- 172.21.15.181:0/3974631881 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc003dd0 con 0x7f4be40b2560 2023-10-16T11:19:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 -- 172.21.15.181:0/3974631881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 msgr2=0x7f4be40d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 --2- 172.21.15.181:0/3974631881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be40d1430 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009e50 tx=0x7f4bdc00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 -- 172.21.15.181:0/3974631881 shutdown_connections 2023-10-16T11:19:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 --2- 172.21.15.181:0/3974631881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be40d1430 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 --2- 172.21.15.181:0/3974631881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be41615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 -- 172.21.15.181:0/3974631881 >> 172.21.15.181:0/3974631881 conn(0x7f4be40d3860 msgr2=0x7f4be40d3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 -- 172.21.15.181:0/3974631881 shutdown_connections 2023-10-16T11:19:12.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.632+0000 7f4be9792700 1 -- 172.21.15.181:0/3974631881 wait complete. 2023-10-16T11:19:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.633+0000 7f4be9792700 1 Processor -- start 2023-10-16T11:19:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.633+0000 7f4be9792700 1 -- start start 2023-10-16T11:19:12.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be9792700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be9792700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be40b2c00 0x7f4be4160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be9792700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be415efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be9792700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be415eb50 con 0x7f4be40b2560 2023-10-16T11:19:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be9792700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be415e260 con 0x7f4be40b2c00 2023-10-16T11:19:12.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be9792700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4160e40 con 0x7f4be415dbe0 2023-10-16T11:19:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be2ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be406cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55100/0 (socket says 172.21.15.181:55100) 2023-10-16T11:19:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be2ffd700 1 -- 172.21.15.181:0/1536688043 learned_addr learned my addr 172.21.15.181:0/1536688043 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be37fe700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be415efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:12.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.634+0000 7f4be27fc700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be40b2c00 0x7f4be4160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be2ffd700 1 -- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 msgr2=0x7f4be415efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be2ffd700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be415efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be2ffd700 1 -- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be40b2c00 msgr2=0x7f4be4160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be2ffd700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be40b2c00 0x7f4be4160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be2ffd700 1 -- 172.21.15.181:0/1536688043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009b30 con 0x7f4be40b2560 2023-10-16T11:19:12.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be2ffd700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be406cc30 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c7f0 tx=0x7f4bd800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4bcb7fe700 1 -- 172.21.15.181:0/1536688043 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8004220 con 0x7f4be40b2560 2023-10-16T11:19:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4bcb7fe700 1 -- 172.21.15.181:0/1536688043 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd8004380 con 0x7f4be40b2560 2023-10-16T11:19:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be415e500 con 0x7f4be40b2560 2023-10-16T11:19:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4bcb7fe700 1 -- 172.21.15.181:0/1536688043 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80036a0 con 0x7f4be40b2560 2023-10-16T11:19:12.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.635+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be406d280 con 0x7f4be40b2560 2023-10-16T11:19:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.636+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be415ee30 con 0x7f4be40b2560 2023-10-16T11:19:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.636+0000 7f4bcb7fe700 1 -- 172.21.15.181:0/1536688043 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 138) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4bd8003800 con 0x7f4be40b2560 2023-10-16T11:19:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.637+0000 7f4bcb7fe700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4bd006af90 0x7f4bd006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.637+0000 7f4bcb7fe700 1 -- 172.21.15.181:0/1536688043 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6741+0+0 (secure 0 0 0) 0x7f4bd8089560 con 0x7f4be40b2560 2023-10-16T11:19:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.637+0000 7f4be27fc700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4bd006af90 0x7f4bd006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:12.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.637+0000 7f4be27fc700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4bd006af90 0x7f4bd006d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b86d0 tx=0x7f4bd4007ec0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:12.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.641+0000 7f4bcb7fe700 1 -- 172.21.15.181:0/1536688043 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4bd8054190 con 0x7f4be40b2560 2023-10-16T11:19:12.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:12.840+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4be404ebb0 con 0x7f4be40b2560 2023-10-16T11:19:13.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.478+0000 7f4bcb7fe700 1 -- 172.21.15.181:0/1536688043 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v208) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4bd80577b0 con 0x7f4be40b2560 2023-10-16T11:19:13.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4bd006af90 msgr2=0x7f4bd006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4bd006af90 0x7f4bd006d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b86d0 tx=0x7f4bd4007ec0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 msgr2=0x7f4be406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be406cc30 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c7f0 tx=0x7f4bd800cb00 comp rx=0 tx=0).stop 2023-10-16T11:19:13.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 shutdown_connections 2023-10-16T11:19:13.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4bd006af90 0x7f4bd006d450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4be415dbe0 0x7f4be415efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be40b2560 0x7f4be406cc30 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 --2- 172.21.15.181:0/1536688043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4be40b2c00 0x7f4be4160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 >> 172.21.15.181:0/1536688043 conn(0x7f4be40d3860 msgr2=0x7f4be40b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:13.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 shutdown_connections 2023-10-16T11:19:13.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.481+0000 7f4be9792700 1 -- 172.21.15.181:0/1536688043 wait complete. 2023-10-16T11:19:13.485 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:19:13.501 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:19:13.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.628+0000 7fa6fb901700 1 Processor -- start 2023-10-16T11:19:13.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.628+0000 7fa6fb901700 1 -- start start 2023-10-16T11:19:13.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6fb901700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa6f4072760 0x7fa6f406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6fb901700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 0x7fa6f41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6fb901700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f406b210 con 0x7fa6f415da70 2023-10-16T11:19:13.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6fb901700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f40cef00 con 0x7fa6f4072760 2023-10-16T11:19:13.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6fb901700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40cec20 con 0x7fa6f406ade0 2023-10-16T11:19:13.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6f969d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa6f415da70 0x7fa6f40ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37950/0 (socket says 172.21.15.181:37950) 2023-10-16T11:19:13.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6f969d700 1 -- 172.21.15.181:0/3554488319 learned_addr learned my addr 172.21.15.181:0/3554488319 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:13.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3554488319 >> v2:172.21.15.190:3300/0 conn(0x7fa6f4072760 0x7fa6f406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6f9e9e700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 0x7fa6f41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119917151 0 0) 0x7fa6f406b210 con 0x7fa6f415da70 2023-10-16T11:19:13.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e00035b0 con 0x7fa6f415da70 2023-10-16T11:19:13.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090944481 0 0) 0x7fa6e00035b0 con 0x7fa6f415da70 2023-10-16T11:19:13.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 msgr2=0x7fa6f41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:13.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 0x7fa6f41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 >> v2:172.21.15.190:3300/0 conn(0x7fa6f4072760 msgr2=0x7fa6f406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:13.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 --2- 172.21.15.181:0/3554488319 >> v2:172.21.15.190:3300/0 conn(0x7fa6f4072760 0x7fa6f406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4071c80 con 0x7fa6f415da70 2023-10-16T11:19:13.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa6f0001670 con 0x7fa6f415da70 2023-10-16T11:19:13.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 >> v1:172.21.15.181:6789/0 conn(0x7fa6f415da70 legacy=0x7fa6f40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e0003a00 0x7fa6e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.629+0000 7fa6ebfff700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6ebfff700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 0x7fa6e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40cef00 con 0x7fa6e0004850 2023-10-16T11:19:13.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40cec20 con 0x7fa6e0003a00 2023-10-16T11:19:13.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e000bac0 con 0x7fa6f406ade0 2023-10-16T11:19:13.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f969d700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e0003a00 0x7fa6e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f9e9e700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 0x7fa6e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f8e9c700 1 -- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 msgr2=0x7fa6e0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f406ade0 0x7fa6e0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f8e9c700 1 -- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e0003a00 msgr2=0x7fa6e0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e0003a00 0x7fa6e0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f8e9c700 1 -- 172.21.15.181:0/3554488319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00035b0 con 0x7fa6e0004850 2023-10-16T11:19:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6e0004cd0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009b60 tx=0x7fa6e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ae395293b22557 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400e6c0 con 0x7fa6e0004850 2023-10-16T11:19:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e400e820 con 0x7fa6e0004850 2023-10-16T11:19:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.630+0000 7fa6ebfff700 1 -- 172.21.15.181:0/3554488319 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400eb90 con 0x7fa6e0004850 2023-10-16T11:19:13.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.631+0000 7fa6fb901700 1 -- 172.21.15.181:0/3554488319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 msgr2=0x7fa6e0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.631+0000 7fa6fb901700 1 --2- 172.21.15.181:0/3554488319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6e0004cd0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009b60 tx=0x7fa6e0005880 comp rx=0 tx=0).stop 2023-10-16T11:19:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.631+0000 7fa6f9e9e700 1 -- 172.21.15.181:0/3554488319 reap_dead start 2023-10-16T11:19:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.631+0000 7fa6fb901700 1 -- 172.21.15.181:0/3554488319 shutdown_connections 2023-10-16T11:19:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.631+0000 7fa6fb901700 1 -- 172.21.15.181:0/3554488319 >> 172.21.15.181:0/3554488319 conn(0x7fa6f40cd010 msgr2=0x7fa6f40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.631+0000 7fa6fb901700 1 -- 172.21.15.181:0/3554488319 shutdown_connections 2023-10-16T11:19:13.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.631+0000 7fa6fb901700 1 -- 172.21.15.181:0/3554488319 wait complete. 2023-10-16T11:19:13.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.632+0000 7fa6fb901700 1 Processor -- start 2023-10-16T11:19:13.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.632+0000 7fa6fb901700 1 -- start start 2023-10-16T11:19:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6fb901700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6f4159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6fb901700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f40c9f30 0x7fa6f40b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6fb901700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f40ca3b0 0x7fa6f40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6fb901700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40b53b0 con 0x7fa6e0004850 2023-10-16T11:19:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6fb901700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f415e4a0 con 0x7fa6f40c9f30 2023-10-16T11:19:13.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6fb901700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f415e780 con 0x7fa6f40ca3b0 2023-10-16T11:19:13.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6f4159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6f4159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55116/0 (socket says 172.21.15.181:55116) 2023-10-16T11:19:13.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 -- 172.21.15.181:0/3208460321 learned_addr learned my addr 172.21.15.181:0/3208460321 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f40c9f30 0x7fa6f40b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f9e9e700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f40ca3b0 0x7fa6f40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 -- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f40ca3b0 msgr2=0x7fa6f40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f40ca3b0 0x7fa6f40b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 -- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f40c9f30 msgr2=0x7fa6f40b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f40c9f30 0x7fa6f40b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 -- 172.21.15.181:0/3208460321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f415ea60 con 0x7fa6e0004850 2023-10-16T11:19:13.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6f969d700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6f4159ac0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0007310 tx=0x7fa6f0007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:13.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6e9ffb700 1 -- 172.21.15.181:0/3208460321 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0011580 con 0x7fa6e0004850 2023-10-16T11:19:13.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6e9ffb700 1 -- 172.21.15.181:0/3208460321 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f00116e0 con 0x7fa6e0004850 2023-10-16T11:19:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f415ed40 con 0x7fa6e0004850 2023-10-16T11:19:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.633+0000 7fa6e9ffb700 1 -- 172.21.15.181:0/3208460321 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0011a70 con 0x7fa6e0004850 2023-10-16T11:19:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.634+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f415e0c0 con 0x7fa6e0004850 2023-10-16T11:19:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.635+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f415e250 con 0x7fa6e0004850 2023-10-16T11:19:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.635+0000 7fa6e9ffb700 1 -- 172.21.15.181:0/3208460321 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 138) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa6f0011bd0 con 0x7fa6e0004850 2023-10-16T11:19:13.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.635+0000 7fa6e9ffb700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa6d806af90 0x7fa6d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.635+0000 7fa6e9ffb700 1 -- 172.21.15.181:0/3208460321 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7fa6f0089840 con 0x7fa6e0004850 2023-10-16T11:19:13.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.635+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa6d806af90 0x7fa6d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.636+0000 7fa6f8e9c700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa6d806af90 0x7fa6d806d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40090b0 tx=0x7fa6e4008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:13.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.640+0000 7fa6e9ffb700 1 -- 172.21.15.181:0/3208460321 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6f0054790 con 0x7fa6e0004850 2023-10-16T11:19:13.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.841+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fa6f406f6c0 con 0x7fa6e0004850 2023-10-16T11:19:13.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.844+0000 7fa6e9ffb700 1 -- 172.21.15.181:0/3208460321 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v73) v1 ==== 115+0+0 (secure 0 0 0) 0x7fa6f0057db0 con 0x7fa6e0004850 2023-10-16T11:19:13.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa6d806af90 msgr2=0x7fa6d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa6d806af90 0x7fa6d806d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40090b0 tx=0x7fa6e4008030 comp rx=0 tx=0).stop 2023-10-16T11:19:13.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 msgr2=0x7fa6f4159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6f4159ac0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0007310 tx=0x7fa6f0007620 comp rx=0 tx=0).stop 2023-10-16T11:19:13.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 shutdown_connections 2023-10-16T11:19:13.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa6d806af90 0x7fa6d806d450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f40ca3b0 0x7fa6f40b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e0004850 0x7fa6f4159ac0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 --2- 172.21.15.181:0/3208460321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f40c9f30 0x7fa6f40b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 >> 172.21.15.181:0/3208460321 conn(0x7fa6f40cd010 msgr2=0x7fa6f40b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 shutdown_connections 2023-10-16T11:19:13.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.847+0000 7fa6fb901700 1 -- 172.21.15.181:0/3208460321 wait complete. 2023-10-16T11:19:13.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:19:13.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.982+0000 7f4f72da0700 1 Processor -- start 2023-10-16T11:19:13.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.982+0000 7f4f72da0700 1 -- start start 2023-10-16T11:19:13.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f72da0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f6c06bef0 0x7f4f6c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f72da0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f72da0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c06e4b0 con 0x7f4f6c159e10 2023-10-16T11:19:13.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f72da0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0cecb0 con 0x7f4f6c06bef0 2023-10-16T11:19:13.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f72da0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c06ed50 con 0x7f4f6c0cb0e0 2023-10-16T11:19:13.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f6bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f6c06bef0 0x7f4f6c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f70b3c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4f6c159e10 0x7f4f6c0ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37970/0 (socket says 172.21.15.181:37970) 2023-10-16T11:19:13.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f70b3c700 1 -- 172.21.15.181:0/3553636183 learned_addr learned my addr 172.21.15.181:0/3553636183 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:13.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f7133d700 1 --2- 172.21.15.181:0/3553636183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f6bfff700 1 -- 172.21.15.181:0/3553636183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 msgr2=0x7f4f6c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:13.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f6bfff700 1 --2- 172.21.15.181:0/3553636183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f6bfff700 1 -- 172.21.15.181:0/3553636183 >> v1:172.21.15.181:6789/0 conn(0x7f4f6c159e10 legacy=0x7f4f6c0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f6b7fe700 1 -- 172.21.15.181:0/3553636183 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576208413 0 0) 0x7f4f6c06e4b0 con 0x7f4f6c159e10 2023-10-16T11:19:13.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f6bfff700 1 -- 172.21.15.181:0/3553636183 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0ceea0 con 0x7f4f6c06bef0 2023-10-16T11:19:13.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.983+0000 7f4f6bfff700 1 --2- 172.21.15.181:0/3553636183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c0caba0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800bae0 tx=0x7f4f5800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718a772a2d02acd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:13.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f6b7fe700 1 -- 172.21.15.181:0/3553636183 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5800dc20 con 0x7f4f6c06bef0 2023-10-16T11:19:13.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f6b7fe700 1 -- 172.21.15.181:0/3553636183 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5800dd80 con 0x7f4f6c06bef0 2023-10-16T11:19:13.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f6b7fe700 1 -- 172.21.15.181:0/3553636183 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58017630 con 0x7f4f6c06bef0 2023-10-16T11:19:13.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 -- 172.21.15.181:0/3553636183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 msgr2=0x7f4f6c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3553636183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c0caba0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800bae0 tx=0x7f4f5800bdf0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 -- 172.21.15.181:0/3553636183 shutdown_connections 2023-10-16T11:19:13.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3553636183 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3553636183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c0caba0 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 -- 172.21.15.181:0/3553636183 >> 172.21.15.181:0/3553636183 conn(0x7f4f6c0cc730 msgr2=0x7f4f6c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:13.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 -- 172.21.15.181:0/3553636183 shutdown_connections 2023-10-16T11:19:13.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.984+0000 7f4f72da0700 1 -- 172.21.15.181:0/3553636183 wait complete. 2023-10-16T11:19:13.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.985+0000 7f4f72da0700 1 Processor -- start 2023-10-16T11:19:13.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.985+0000 7f4f72da0700 1 -- start start 2023-10-16T11:19:13.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f72da0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c1e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f72da0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c14f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f72da0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f6c159e10 0x7f4f6c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f72da0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c1e3ad0 con 0x7f4f6c06bef0 2023-10-16T11:19:13.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f72da0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c1e3d20 con 0x7f4f6c159e10 2023-10-16T11:19:13.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f72da0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c1e4000 con 0x7f4f6c0cb0e0 2023-10-16T11:19:13.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f70b3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c1e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f70b3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c1e4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55130/0 (socket says 172.21.15.181:55130) 2023-10-16T11:19:13.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.986+0000 7f4f70b3c700 1 -- 172.21.15.181:0/3716018180 learned_addr learned my addr 172.21.15.181:0/3716018180 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:13.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f7133d700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f6c159e10 0x7f4f6c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f6bfff700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c14f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:13.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f70b3c700 1 -- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 msgr2=0x7f4f6c14f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f70b3c700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c14f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f70b3c700 1 -- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f6c159e10 msgr2=0x7f4f6c151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:13.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f70b3c700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f6c159e10 0x7f4f6c151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:13.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f70b3c700 1 -- 172.21.15.181:0/3716018180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5800b740 con 0x7f4f6c06bef0 2023-10-16T11:19:13.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f7133d700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f6c159e10 0x7f4f6c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:13.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f70b3c700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c1e4dc0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c550 tx=0x7f4f6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:13.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f697fa700 1 -- 172.21.15.181:0/3716018180 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60004c80 con 0x7f4f6c06bef0 2023-10-16T11:19:13.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f697fa700 1 -- 172.21.15.181:0/3716018180 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f60004e00 con 0x7f4f6c06bef0 2023-10-16T11:19:13.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c1e42e0 con 0x7f4f6c06bef0 2023-10-16T11:19:13.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f697fa700 1 -- 172.21.15.181:0/3716018180 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60007310 con 0x7f4f6c06bef0 2023-10-16T11:19:13.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.987+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c1e48a0 con 0x7f4f6c06bef0 2023-10-16T11:19:13.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.988+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c1e4ac0 con 0x7f4f6c06bef0 2023-10-16T11:19:13.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.988+0000 7f4f697fa700 1 -- 172.21.15.181:0/3716018180 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 138) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4f60007470 con 0x7f4f6c06bef0 2023-10-16T11:19:13.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.989+0000 7f4f697fa700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f5006af90 0x7f4f5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:13.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.989+0000 7f4f697fa700 1 -- 172.21.15.181:0/3716018180 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f4f60089520 con 0x7f4f6c06bef0 2023-10-16T11:19:13.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.989+0000 7f4f6bfff700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f5006af90 0x7f4f5006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.989+0000 7f4f6bfff700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f5006af90 0x7f4f5006d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c06c300 tx=0x7f4f58014000 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:14.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:13.993+0000 7f4f697fa700 1 -- 172.21.15.181:0/3716018180 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f600543b0 con 0x7f4f6c06bef0 2023-10-16T11:19:14.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.183+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f4f6c159630 con 0x7f4f6c06bef0 2023-10-16T11:19:14.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.188+0000 7f4f697fa700 1 -- 172.21.15.181:0/3716018180 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v74) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4f600579d0 con 0x7f4f6c06bef0 2023-10-16T11:19:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f5006af90 msgr2=0x7f4f5006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f5006af90 0x7f4f5006d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c06c300 tx=0x7f4f58014000 comp rx=0 tx=0).stop 2023-10-16T11:19:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 msgr2=0x7f4f6c1e4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c1e4dc0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c550 tx=0x7f4f6000c910 comp rx=0 tx=0).stop 2023-10-16T11:19:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 shutdown_connections 2023-10-16T11:19:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f5006af90 0x7f4f5006d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f6c159e10 0x7f4f6c151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f6c06bef0 0x7f4f6c1e4dc0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 --2- 172.21.15.181:0/3716018180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f6c0cb0e0 0x7f4f6c14f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 >> 172.21.15.181:0/3716018180 conn(0x7f4f6c0cc730 msgr2=0x7f4f6c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 shutdown_connections 2023-10-16T11:19:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.191+0000 7f4f72da0700 1 -- 172.21.15.181:0/3716018180 wait complete. 2023-10-16T11:19:14.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:19:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.328+0000 7f5be79d3700 1 Processor -- start 2023-10-16T11:19:14.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.328+0000 7f5be79d3700 1 -- start start 2023-10-16T11:19:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be79d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5be00ce6c0 0x7f5be00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be79d3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5be00cc6a0 0x7f5be00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be79d3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00b1f70 con 0x7f5be015f150 2023-10-16T11:19:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be79d3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be00b2ae0 con 0x7f5be00ce6c0 2023-10-16T11:19:14.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be79d3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be00b2dc0 con 0x7f5be00cc6a0 2023-10-16T11:19:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be4f6e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5be00ce6c0 0x7f5be00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be4f6e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5be00ce6c0 0x7f5be00cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55140/0 (socket says 172.21.15.181:55140) 2023-10-16T11:19:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be4f6e700 1 -- 172.21.15.181:0/649805391 learned_addr learned my addr 172.21.15.181:0/649805391 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:14.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be5f70700 1 --2- 172.21.15.181:0/649805391 >> v2:172.21.15.190:3300/0 conn(0x7f5be00cc6a0 0x7f5be00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.329+0000 7f5be4f6e700 1 -- 172.21.15.181:0/649805391 >> v2:172.21.15.190:3300/0 conn(0x7f5be00cc6a0 msgr2=0x7f5be00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be4f6e700 1 --2- 172.21.15.181:0/649805391 >> v2:172.21.15.190:3300/0 conn(0x7f5be00cc6a0 0x7f5be00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be4f6e700 1 -- 172.21.15.181:0/649805391 >> v1:172.21.15.190:6789/0 conn(0x7f5be015f150 legacy=0x7f5be00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be4f6e700 1 -- 172.21.15.181:0/649805391 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00cf9d0 con 0x7f5be00ce6c0 2023-10-16T11:19:14.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be4f6e700 1 --2- 172.21.15.181:0/649805391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be00ce6c0 0x7f5be00cc160 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f5be00ceb70 tx=0x7f5bdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3addc3107f2125f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5bd7fff700 1 -- 172.21.15.181:0/649805391 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00b870 con 0x7f5be00ce6c0 2023-10-16T11:19:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5bd7fff700 1 -- 172.21.15.181:0/649805391 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc00b9d0 con 0x7f5be00ce6c0 2023-10-16T11:19:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5bd7fff700 1 -- 172.21.15.181:0/649805391 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00bd60 con 0x7f5be00ce6c0 2023-10-16T11:19:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be79d3700 1 -- 172.21.15.181:0/649805391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be00ce6c0 msgr2=0x7f5be00cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be79d3700 1 --2- 172.21.15.181:0/649805391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be00ce6c0 0x7f5be00cc160 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f5be00ceb70 tx=0x7f5bdc009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be79d3700 1 -- 172.21.15.181:0/649805391 shutdown_connections 2023-10-16T11:19:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be79d3700 1 --2- 172.21.15.181:0/649805391 >> v2:172.21.15.190:3300/0 conn(0x7f5be00cc6a0 0x7f5be00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be79d3700 1 --2- 172.21.15.181:0/649805391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be00ce6c0 0x7f5be00cc160 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be79d3700 1 -- 172.21.15.181:0/649805391 >> 172.21.15.181:0/649805391 conn(0x7f5be009a930 msgr2=0x7f5be00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:14.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.330+0000 7f5be79d3700 1 -- 172.21.15.181:0/649805391 shutdown_connections 2023-10-16T11:19:14.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.331+0000 7f5be79d3700 1 -- 172.21.15.181:0/649805391 wait complete. 2023-10-16T11:19:14.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 Processor -- start 2023-10-16T11:19:14.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 -- start start 2023-10-16T11:19:14.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5be00cc6a0 0x7f5be0159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be00ce6c0 0x7f5be006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be015f150 0x7f5be006fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be015a480 con 0x7f5be015f150 2023-10-16T11:19:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00ca400 con 0x7f5be00ce6c0 2023-10-16T11:19:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be79d3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00ca6e0 con 0x7f5be00cc6a0 2023-10-16T11:19:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be5f70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be015f150 0x7f5be006fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be5f70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be015f150 0x7f5be006fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55148/0 (socket says 172.21.15.181:55148) 2023-10-16T11:19:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.332+0000 7f5be5f70700 1 -- 172.21.15.181:0/541488075 learned_addr learned my addr 172.21.15.181:0/541488075 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be4f6e700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be00ce6c0 0x7f5be006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be576f700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5be00cc6a0 0x7f5be0159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be5f70700 1 -- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5be00cc6a0 msgr2=0x7f5be0159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be5f70700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5be00cc6a0 0x7f5be0159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be5f70700 1 -- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be00ce6c0 msgr2=0x7f5be006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be5f70700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be00ce6c0 0x7f5be006f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be5f70700 1 -- 172.21.15.181:0/541488075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc009550 con 0x7f5be015f150 2023-10-16T11:19:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be5f70700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be015f150 0x7f5be006fb00 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8007e60 tx=0x7f5bd800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:14.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800ed30 con 0x7f5be015f150 2023-10-16T11:19:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8014420 con 0x7f5be015f150 2023-10-16T11:19:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be00b4780 con 0x7f5be015f150 2023-10-16T11:19:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80147d0 con 0x7f5be015f150 2023-10-16T11:19:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.333+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be00b4d40 con 0x7f5be015f150 2023-10-16T11:19:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.334+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0067d20 con 0x7f5be015f150 2023-10-16T11:19:14.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.335+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 138) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5bd8014930 con 0x7f5be015f150 2023-10-16T11:19:14.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.335+0000 7f5bd5ffb700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5bc406af90 0x7f5bc406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.335+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f5bd8089dd0 con 0x7f5be015f150 2023-10-16T11:19:14.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.335+0000 7f5be576f700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5bc406af90 0x7f5bc406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.335+0000 7f5be576f700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5bc406af90 0x7f5bc406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0000f60 tx=0x7f5bd0009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:14.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.339+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5bd8054d10 con 0x7f5be015f150 2023-10-16T11:19:14.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.490+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5bd80550e0 con 0x7f5be015f150 2023-10-16T11:19:14.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.535+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5be006f0a0 con 0x7f5be015f150 2023-10-16T11:19:14.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.539+0000 7f5bd5ffb700 1 -- 172.21.15.181:0/541488075 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5bd8087430 con 0x7f5be015f150 2023-10-16T11:19:14.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5bc406af90 msgr2=0x7f5bc406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5bc406af90 0x7f5bc406d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0000f60 tx=0x7f5bd0009040 comp rx=0 tx=0).stop 2023-10-16T11:19:14.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be015f150 msgr2=0x7f5be006fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be015f150 0x7f5be006fb00 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8007e60 tx=0x7f5bd800e3c0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 shutdown_connections 2023-10-16T11:19:14.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5bc406af90 0x7f5bc406d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5be00cc6a0 0x7f5be0159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be015f150 0x7f5be006fb00 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 --2- 172.21.15.181:0/541488075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be00ce6c0 0x7f5be006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 >> 172.21.15.181:0/541488075 conn(0x7f5be009a930 msgr2=0x7f5be015ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:14.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 shutdown_connections 2023-10-16T11:19:14.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.541+0000 7f5be79d3700 1 -- 172.21.15.181:0/541488075 wait complete. 2023-10-16T11:19:14.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-16T11:19:14.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.674+0000 7fc85cbd8700 1 Processor -- start 2023-10-16T11:19:14.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.674+0000 7fc85cbd8700 1 -- start start 2023-10-16T11:19:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc85cbd8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8580cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc85cbd8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8580d0180 0x7fc8580cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc85cbd8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8580cc470 con 0x7fc8580ce7f0 2023-10-16T11:19:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc85cbd8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8580d0610 con 0x7fc8580d0180 2023-10-16T11:19:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc85cbd8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85806c810 con 0x7fc8580d2390 2023-10-16T11:19:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc855d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8580cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc856d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8580d0180 0x7fc8580cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc855d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8580cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:52180/0 (socket says 172.21.15.181:52180) 2023-10-16T11:19:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc855d9b700 1 -- 172.21.15.181:0/630587055 learned_addr learned my addr 172.21.15.181:0/630587055 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.675+0000 7fc85659c700 1 --1- 172.21.15.181:0/630587055 >> v1:172.21.15.190:6789/0 conn(0x7fc8580ce7f0 0x7fc85806c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:50172/0 (socket says 172.21.15.181:50172) 2023-10-16T11:19:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc85559a700 1 -- 172.21.15.181:0/630587055 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946644047 0 0) 0x7fc8580cc470 con 0x7fc8580ce7f0 2023-10-16T11:19:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc85559a700 1 -- 172.21.15.181:0/630587055 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8440035b0 con 0x7fc8580ce7f0 2023-10-16T11:19:14.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc856d9d700 1 -- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 msgr2=0x7fc8580cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc856d9d700 1 --2- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8580cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc856d9d700 1 -- 172.21.15.181:0/630587055 >> v1:172.21.15.190:6789/0 conn(0x7fc8580ce7f0 legacy=0x7fc85806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc856d9d700 1 -- 172.21.15.181:0/630587055 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85806ca30 con 0x7fc8580d0180 2023-10-16T11:19:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc855d9b700 1 --2- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8580cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:14.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc856d9d700 1 --2- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580d0180 0x7fc8580cc930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc850009d90 tx=0x7fc85000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74dc27aeeff3c5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc85559a700 1 -- 172.21.15.181:0/630587055 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000e070 con 0x7fc8580d0180 2023-10-16T11:19:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc85559a700 1 -- 172.21.15.181:0/630587055 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85000bb20 con 0x7fc8580d0180 2023-10-16T11:19:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.676+0000 7fc85559a700 1 -- 172.21.15.181:0/630587055 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850012430 con 0x7fc8580d0180 2023-10-16T11:19:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 -- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580d0180 msgr2=0x7fc8580cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580d0180 0x7fc8580cc930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc850009d90 tx=0x7fc85000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 -- 172.21.15.181:0/630587055 shutdown_connections 2023-10-16T11:19:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580d0180 0x7fc8580cc930 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/630587055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8580cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 -- 172.21.15.181:0/630587055 >> 172.21.15.181:0/630587055 conn(0x7fc85809a9a0 msgr2=0x7fc8580d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:14.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 -- 172.21.15.181:0/630587055 shutdown_connections 2023-10-16T11:19:14.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.677+0000 7fc85cbd8700 1 -- 172.21.15.181:0/630587055 wait complete. 2023-10-16T11:19:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.678+0000 7fc85cbd8700 1 Processor -- start 2023-10-16T11:19:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.678+0000 7fc85cbd8700 1 -- start start 2023-10-16T11:19:14.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc85cbd8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580ce7f0 0x7fc8581f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc85cbd8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8580d0180 0x7fc858156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc85cbd8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8581f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc85cbd8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581f36c0 con 0x7fc8580d0180 2023-10-16T11:19:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc85cbd8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581f73a0 con 0x7fc8580ce7f0 2023-10-16T11:19:14.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc85cbd8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581f70c0 con 0x7fc8580d2390 2023-10-16T11:19:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8580d0180 0x7fc858156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8580d0180 0x7fc858156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55162/0 (socket says 172.21.15.181:55162) 2023-10-16T11:19:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 -- 172.21.15.181:0/187401303 learned_addr learned my addr 172.21.15.181:0/187401303 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc85659c700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580ce7f0 0x7fc8581f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc856d9d700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8581f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 -- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 msgr2=0x7fc8581f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8581f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 -- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580ce7f0 msgr2=0x7fc8581f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580ce7f0 0x7fc8581f3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 -- 172.21.15.181:0/187401303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840009550 con 0x7fc8580d0180 2023-10-16T11:19:14.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.679+0000 7fc855d9b700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8580d0180 0x7fc858156d00 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fc84000ebe0 tx=0x7fc84000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:14.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.680+0000 7fc84effd700 1 -- 172.21.15.181:0/187401303 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000c850 con 0x7fc8580d0180 2023-10-16T11:19:14.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.680+0000 7fc84effd700 1 -- 172.21.15.181:0/187401303 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000c9b0 con 0x7fc8580d0180 2023-10-16T11:19:14.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.680+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc850009a70 con 0x7fc8580d0180 2023-10-16T11:19:14.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.680+0000 7fc84effd700 1 -- 172.21.15.181:0/187401303 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000cd40 con 0x7fc8580d0180 2023-10-16T11:19:14.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.680+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8581f3350 con 0x7fc8580d0180 2023-10-16T11:19:14.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.680+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8581f3870 con 0x7fc8580d0180 2023-10-16T11:19:14.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.681+0000 7fc84effd700 1 -- 172.21.15.181:0/187401303 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc840013470 con 0x7fc8580d0180 2023-10-16T11:19:14.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.681+0000 7fc84effd700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc83806b020 0x7fc83806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:14.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.681+0000 7fc84effd700 1 -- 172.21.15.181:0/187401303 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7fc840089b00 con 0x7fc8580d0180 2023-10-16T11:19:14.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.681+0000 7fc85659c700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc83806b020 0x7fc83806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:14.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.682+0000 7fc85659c700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc83806b020 0x7fc83806d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc848001230 tx=0x7fc848007db0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:14.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.686+0000 7fc84effd700 1 -- 172.21.15.181:0/187401303 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc8581f3870 con 0x7fc8580d0180 2023-10-16T11:19:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:14.959+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fc858067d20 con 0x7fc8580d0180 2023-10-16T11:19:15.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.487+0000 7fc84effd700 1 -- 172.21.15.181:0/187401303 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3094) v1 ==== 163+0+0 (secure 0 0 0) 0x7fc840054870 con 0x7fc8580d0180 2023-10-16T11:19:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc83806b020 msgr2=0x7fc83806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc83806b020 0x7fc83806d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc848001230 tx=0x7fc848007db0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8580d0180 msgr2=0x7fc858156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8580d0180 0x7fc858156d00 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fc84000ebe0 tx=0x7fc84000eef0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 shutdown_connections 2023-10-16T11:19:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc83806b020 0x7fc83806d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8580d2390 0x7fc8581f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8580ce7f0 0x7fc8581f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 --2- 172.21.15.181:0/187401303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8580d0180 0x7fc858156d00 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 >> 172.21.15.181:0/187401303 conn(0x7fc85809a9a0 msgr2=0x7fc8580cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:15.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 shutdown_connections 2023-10-16T11:19:15.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.490+0000 7fc85cbd8700 1 -- 172.21.15.181:0/187401303 wait complete. 2023-10-16T11:19:15.504 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-16T11:19:15.505 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-10-16T11:19:15.505 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-16T11:19:15.505 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:15.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.628+0000 7f4c43599700 1 Processor -- start 2023-10-16T11:19:15.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.628+0000 7f4c43599700 1 -- start start 2023-10-16T11:19:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.629+0000 7f4c43599700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c3c15dfd0 0x7f4c3c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.629+0000 7f4c43599700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c3c0cd470 0x7f4c3c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.629+0000 7f4c43599700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c06f870 con 0x7f4c3c0af450 2023-10-16T11:19:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.629+0000 7f4c43599700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c0cde40 con 0x7f4c3c0cd470 2023-10-16T11:19:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.629+0000 7f4c43599700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3c0d1160 con 0x7f4c3c15dfd0 2023-10-16T11:19:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.629+0000 7f4c41b36700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c3c0cd470 0x7f4c3c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.629+0000 7f4c40b34700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4c3c0af450 0x7f4c3c06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38012/0 (socket says 172.21.15.181:38012) 2023-10-16T11:19:15.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c40b34700 1 -- 172.21.15.181:0/899678488 learned_addr learned my addr 172.21.15.181:0/899678488 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:15.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c41335700 1 --2- 172.21.15.181:0/899678488 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c15dfd0 0x7f4c3c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c41b36700 1 -- 172.21.15.181:0/899678488 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c15dfd0 msgr2=0x7f4c3c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:15.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c41b36700 1 --2- 172.21.15.181:0/899678488 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c15dfd0 0x7f4c3c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c41b36700 1 -- 172.21.15.181:0/899678488 >> v1:172.21.15.181:6789/0 conn(0x7f4c3c0af450 legacy=0x7f4c3c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c41b36700 1 -- 172.21.15.181:0/899678488 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0cc8e0 con 0x7f4c3c0cd470 2023-10-16T11:19:15.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c33fff700 1 -- 172.21.15.181:0/899678488 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907078681 0 0) 0x7f4c3c06f870 con 0x7f4c3c0af450 2023-10-16T11:19:15.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c41b36700 1 --2- 172.21.15.181:0/899678488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c0cd900 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c070340 tx=0x7f4c3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aad067980690d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:15.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c33fff700 1 -- 172.21.15.181:0/899678488 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400e070 con 0x7f4c3c0cd470 2023-10-16T11:19:15.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c33fff700 1 -- 172.21.15.181:0/899678488 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400bbd0 con 0x7f4c3c0cd470 2023-10-16T11:19:15.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.630+0000 7f4c33fff700 1 -- 172.21.15.181:0/899678488 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34012430 con 0x7f4c3c0cd470 2023-10-16T11:19:15.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 -- 172.21.15.181:0/899678488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 msgr2=0x7f4c3c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 --2- 172.21.15.181:0/899678488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c0cd900 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c070340 tx=0x7f4c3400b330 comp rx=0 tx=0).stop 2023-10-16T11:19:15.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 -- 172.21.15.181:0/899678488 shutdown_connections 2023-10-16T11:19:15.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 --2- 172.21.15.181:0/899678488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c0cd900 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 --2- 172.21.15.181:0/899678488 >> v2:172.21.15.190:3300/0 conn(0x7f4c3c15dfd0 0x7f4c3c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 -- 172.21.15.181:0/899678488 >> 172.21.15.181:0/899678488 conn(0x7f4c3c0d04a0 msgr2=0x7f4c3c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:15.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 -- 172.21.15.181:0/899678488 shutdown_connections 2023-10-16T11:19:15.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.631+0000 7f4c43599700 1 -- 172.21.15.181:0/899678488 wait complete. 2023-10-16T11:19:15.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.632+0000 7f4c43599700 1 Processor -- start 2023-10-16T11:19:15.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.632+0000 7f4c43599700 1 -- start start 2023-10-16T11:19:15.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c43599700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c0af450 0x7f4c3c0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c43599700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c14c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c43599700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c15dfd0 0x7f4c3c0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c43599700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c1e5560 con 0x7f4c3c0cd470 2023-10-16T11:19:15.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c43599700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0c57b0 con 0x7f4c3c0af450 2023-10-16T11:19:15.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c43599700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0b42a0 con 0x7f4c3c15dfd0 2023-10-16T11:19:15.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c14c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c14c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55172/0 (socket says 172.21.15.181:55172) 2023-10-16T11:19:15.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 -- 172.21.15.181:0/994500765 learned_addr learned my addr 172.21.15.181:0/994500765 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:15.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c41335700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c0af450 0x7f4c3c0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c41b36700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c15dfd0 0x7f4c3c0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 -- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c15dfd0 msgr2=0x7f4c3c0b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c15dfd0 0x7f4c3c0b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 -- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c0af450 msgr2=0x7f4c3c0ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c0af450 0x7f4c3c0ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 -- 172.21.15.181:0/994500765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009a70 con 0x7f4c3c0cd470 2023-10-16T11:19:15.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c41335700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c0af450 0x7f4c3c0ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:15.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c40b34700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c14c7d0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c580 tx=0x7f4c3800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:15.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c31ffb700 1 -- 172.21.15.181:0/994500765 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38004c80 con 0x7f4c3c0cd470 2023-10-16T11:19:15.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c31ffb700 1 -- 172.21.15.181:0/994500765 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38004e00 con 0x7f4c3c0cd470 2023-10-16T11:19:15.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c0b3fc0 con 0x7f4c3c0cd470 2023-10-16T11:19:15.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.633+0000 7f4c31ffb700 1 -- 172.21.15.181:0/994500765 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38007310 con 0x7f4c3c0cd470 2023-10-16T11:19:15.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.634+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c0c5440 con 0x7f4c3c0cd470 2023-10-16T11:19:15.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.634+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c067d20 con 0x7f4c3c0cd470 2023-10-16T11:19:15.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.635+0000 7f4c31ffb700 1 -- 172.21.15.181:0/994500765 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4c38007470 con 0x7f4c3c0cd470 2023-10-16T11:19:15.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.635+0000 7f4c31ffb700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c2006b020 0x7f4c2006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.635+0000 7f4c31ffb700 1 -- 172.21.15.181:0/994500765 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f4c38089660 con 0x7f4c3c0cd470 2023-10-16T11:19:15.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.635+0000 7f4c41335700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c2006b020 0x7f4c2006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.636+0000 7f4c41335700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c2006b020 0x7f4c2006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009a00 tx=0x7f4c2c008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:15.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.640+0000 7f4c31ffb700 1 -- 172.21.15.181:0/994500765 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c380543d0 con 0x7f4c3c0cd470 2023-10-16T11:19:15.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.831+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c3c06cc30 con 0x7f4c3c0cd470 2023-10-16T11:19:15.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.832+0000 7f4c31ffb700 1 -- 172.21.15.181:0/994500765 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+15026 (secure 0 0 0) 0x7f4c380579f0 con 0x7f4c3c0cd470 2023-10-16T11:19:15.833 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:15.833 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":208,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:13.477113+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/3625484950":"2023-10-17T11:19:10.442168+0000","172.21.15.181:6835/3625484950":"2023-10-17T11:19:10.442168+0000","172.21.15.181:0/509806514":"2023-10-17T11:19:09.459266+0000","172.21.15.181:6832/1119809348":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/956037434":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/1940715811":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/665777945":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/2092158185":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/1746525923":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3179161179":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/3361524667":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/221011703":"2023-10-17T11:19:09.459266+0000","172.21.15.181:6833/1119809348":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/1822518943":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6833/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:6832/1532329351":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/3260777321":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/2465214735":"2023-10-17T11:19:09.459266+0000","172.21.15.181:0/3430010207":"2023-10-17T11:17:53.265498+0000","172.21.15.181:0/969172037":"2023-10-17T11:17:53.265498+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:15.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c2006b020 msgr2=0x7f4c2006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c2006b020 0x7f4c2006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009a00 tx=0x7f4c2c008040 comp rx=0 tx=0).stop 2023-10-16T11:19:15.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 msgr2=0x7f4c3c14c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c14c7d0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c580 tx=0x7f4c3800c940 comp rx=0 tx=0).stop 2023-10-16T11:19:15.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 shutdown_connections 2023-10-16T11:19:15.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c2006b020 0x7f4c2006d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c3c15dfd0 0x7f4c3c0b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c3c0af450 0x7f4c3c0ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 --2- 172.21.15.181:0/994500765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c3c0cd470 0x7f4c3c14c7d0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 >> 172.21.15.181:0/994500765 conn(0x7f4c3c0d04a0 msgr2=0x7f4c3c0ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:15.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 shutdown_connections 2023-10-16T11:19:15.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.834+0000 7f4c43599700 1 -- 172.21.15.181:0/994500765 wait complete. 2023-10-16T11:19:15.847 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:19:15.848 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:19:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.970+0000 7f4b38297700 1 Processor -- start 2023-10-16T11:19:15.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.970+0000 7f4b38297700 1 -- start start 2023-10-16T11:19:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.971+0000 7f4b38297700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b300b2010 0x7f4b300cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.971+0000 7f4b38297700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b30071be0 0x7f4b300b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.971+0000 7f4b38297700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4b3006f3b0 0x7f4b3006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.971+0000 7f4b38297700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b300ce1f0 con 0x7f4b3006f3b0 2023-10-16T11:19:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.971+0000 7f4b38297700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b300b2930 con 0x7f4b30071be0 2023-10-16T11:19:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.971+0000 7f4b38297700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300ccda0 con 0x7f4b300b2010 2023-10-16T11:19:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.971+0000 7f4b36834700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4b3006f3b0 0x7f4b3006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4b3006f3b0 0x7f4b3006f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55182/0 (socket says 172.21.15.181:55182) 2023-10-16T11:19:15.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 -- 172.21.15.181:0/3436291197 learned_addr learned my addr 172.21.15.181:0/3436291197 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36033700 1 --2- 172.21.15.181:0/3436291197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b300b2010 0x7f4b300cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b35832700 1 --2- 172.21.15.181:0/3436291197 >> v2:172.21.15.190:3300/0 conn(0x7f4b30071be0 0x7f4b300b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 -- 172.21.15.181:0/3436291197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b300b2010 msgr2=0x7f4b300cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 --2- 172.21.15.181:0/3436291197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b300b2010 0x7f4b300cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 -- 172.21.15.181:0/3436291197 >> v2:172.21.15.190:3300/0 conn(0x7f4b30071be0 msgr2=0x7f4b300b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:15.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 --2- 172.21.15.181:0/3436291197 >> v2:172.21.15.190:3300/0 conn(0x7f4b30071be0 0x7f4b300b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 -- 172.21.15.181:0/3436291197 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300727e0 con 0x7f4b3006f3b0 2023-10-16T11:19:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b36834700 1 --2- 172.21.15.181:0/3436291197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3006f3b0 0x7f4b3006f840 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f4b30067600 tx=0x7f4b2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd76373ae2891c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b35031700 1 -- 172.21.15.181:0/3436291197 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800e070 con 0x7f4b3006f3b0 2023-10-16T11:19:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b35031700 1 -- 172.21.15.181:0/3436291197 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2800b910 con 0x7f4b3006f3b0 2023-10-16T11:19:15.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.972+0000 7f4b35031700 1 -- 172.21.15.181:0/3436291197 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800bcc0 con 0x7f4b3006f3b0 2023-10-16T11:19:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 -- 172.21.15.181:0/3436291197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3006f3b0 msgr2=0x7f4b3006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 --2- 172.21.15.181:0/3436291197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3006f3b0 0x7f4b3006f840 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f4b30067600 tx=0x7f4b2800b330 comp rx=0 tx=0).stop 2023-10-16T11:19:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 -- 172.21.15.181:0/3436291197 shutdown_connections 2023-10-16T11:19:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 --2- 172.21.15.181:0/3436291197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3006f3b0 0x7f4b3006f840 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 --2- 172.21.15.181:0/3436291197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b300b2010 0x7f4b300cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 --2- 172.21.15.181:0/3436291197 >> v2:172.21.15.190:3300/0 conn(0x7f4b30071be0 0x7f4b300b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 -- 172.21.15.181:0/3436291197 >> 172.21.15.181:0/3436291197 conn(0x7f4b300cd010 msgr2=0x7f4b300cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 -- 172.21.15.181:0/3436291197 shutdown_connections 2023-10-16T11:19:15.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.973+0000 7f4b38297700 1 -- 172.21.15.181:0/3436291197 wait complete. 2023-10-16T11:19:15.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.974+0000 7f4b38297700 1 Processor -- start 2023-10-16T11:19:15.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.974+0000 7f4b38297700 1 -- start start 2023-10-16T11:19:15.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b38297700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b3006f3b0 0x7f4b301572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b38297700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b30071be0 0x7f4b30151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b38297700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b300b2010 0x7f4b301f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b38297700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b301f45b0 con 0x7f4b300b2010 2023-10-16T11:19:15.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b38297700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b301527d0 con 0x7f4b3006f3b0 2023-10-16T11:19:15.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b38297700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b30152ab0 con 0x7f4b30071be0 2023-10-16T11:19:15.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b300b2010 0x7f4b301f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b300b2010 0x7f4b301f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55188/0 (socket says 172.21.15.181:55188) 2023-10-16T11:19:15.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 -- 172.21.15.181:0/797026632 learned_addr learned my addr 172.21.15.181:0/797026632 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36033700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b3006f3b0 0x7f4b301572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 -- 172.21.15.181:0/797026632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b30071be0 msgr2=0x7f4b30151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b30071be0 0x7f4b30151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 -- 172.21.15.181:0/797026632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b3006f3b0 msgr2=0x7f4b301572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b3006f3b0 0x7f4b301572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:15.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 -- 172.21.15.181:0/797026632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b28009b30 con 0x7f4b300b2010 2023-10-16T11:19:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b36834700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b300b2010 0x7f4b301f4d70 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f4b30070460 tx=0x7f4b28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b26ffd700 1 -- 172.21.15.181:0/797026632 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b280099b0 con 0x7f4b300b2010 2023-10-16T11:19:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b26ffd700 1 -- 172.21.15.181:0/797026632 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b280092e0 con 0x7f4b300b2010 2023-10-16T11:19:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b301f4850 con 0x7f4b300b2010 2023-10-16T11:19:15.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.975+0000 7f4b26ffd700 1 -- 172.21.15.181:0/797026632 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2801b630 con 0x7f4b300b2010 2023-10-16T11:19:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.976+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b30152d90 con 0x7f4b300b2010 2023-10-16T11:19:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.976+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b301f4a90 con 0x7f4b300b2010 2023-10-16T11:19:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.977+0000 7f4b26ffd700 1 -- 172.21.15.181:0/797026632 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4b2801b790 con 0x7f4b300b2010 2023-10-16T11:19:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.977+0000 7f4b26ffd700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4b1406b020 0x7f4b1406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:15.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.977+0000 7f4b26ffd700 1 -- 172.21.15.181:0/797026632 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f4b2808af90 con 0x7f4b300b2010 2023-10-16T11:19:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.977+0000 7f4b36033700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4b1406b020 0x7f4b1406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.978+0000 7f4b36033700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4b1406b020 0x7f4b1406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4b300ca660 tx=0x7f4b20008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:15.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:15.982+0000 7f4b26ffd700 1 -- 172.21.15.181:0/797026632 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4b280bdfe0 con 0x7f4b300b2010 2023-10-16T11:19:16.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.173+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b3015c4d0 con 0x7f4b300b2010 2023-10-16T11:19:16.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.174+0000 7f4b26ffd700 1 -- 172.21.15.181:0/797026632 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v208) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b2801f020 con 0x7f4b300b2010 2023-10-16T11:19:16.175 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:19:16.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4b1406b020 msgr2=0x7f4b1406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:16.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4b1406b020 0x7f4b1406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4b300ca660 tx=0x7f4b20008040 comp rx=0 tx=0).stop 2023-10-16T11:19:16.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b300b2010 msgr2=0x7f4b301f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b300b2010 0x7f4b301f4d70 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f4b30070460 tx=0x7f4b28004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 shutdown_connections 2023-10-16T11:19:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4b1406b020 0x7f4b1406d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b300b2010 0x7f4b301f4d70 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b3006f3b0 0x7f4b301572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 --2- 172.21.15.181:0/797026632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b30071be0 0x7f4b30151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 >> 172.21.15.181:0/797026632 conn(0x7f4b300cd010 msgr2=0x7f4b300b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 shutdown_connections 2023-10-16T11:19:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.176+0000 7f4b38297700 1 -- 172.21.15.181:0/797026632 wait complete. 2023-10-16T11:19:16.189 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-16T11:19:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.309+0000 7f54462ff700 1 Processor -- start 2023-10-16T11:19:16.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.309+0000 7f54462ff700 1 -- start start 2023-10-16T11:19:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f54462ff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f544015e6d0 0x7f54400d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:16.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f54462ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400cb0c0 0x7f544015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f54462ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544006ed00 con 0x7f544015eab0 2023-10-16T11:19:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f54462ff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54400d02d0 con 0x7f544015e6d0 2023-10-16T11:19:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f54462ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544015ad40 con 0x7f54400cb0c0 2023-10-16T11:19:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f543ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f544015e6d0 0x7f54400d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f543ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f544015e6d0 0x7f54400d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55192/0 (socket says 172.21.15.181:55192) 2023-10-16T11:19:16.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f543ffff700 1 -- 172.21.15.181:0/1279594871 learned_addr learned my addr 172.21.15.181:0/1279594871 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.310+0000 7f544489c700 1 --2- 172.21.15.181:0/1279594871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400cb0c0 0x7f544015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543ffff700 1 -- 172.21.15.181:0/1279594871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400cb0c0 msgr2=0x7f544015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543ffff700 1 --2- 172.21.15.181:0/1279594871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400cb0c0 0x7f544015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543ffff700 1 -- 172.21.15.181:0/1279594871 >> v1:172.21.15.190:6789/0 conn(0x7f544015eab0 legacy=0x7f544015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:16.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543ffff700 1 -- 172.21.15.181:0/1279594871 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544006e0f0 con 0x7f544015e6d0 2023-10-16T11:19:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543ffff700 1 --2- 172.21.15.181:0/1279594871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f544015e6d0 0x7f54400d0fc0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f5434006fd0 tx=0x7f5434009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a8c503759a666a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543effd700 1 -- 172.21.15.181:0/1279594871 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434005e30 con 0x7f544015e6d0 2023-10-16T11:19:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543effd700 1 -- 172.21.15.181:0/1279594871 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5434004030 con 0x7f544015e6d0 2023-10-16T11:19:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f543effd700 1 -- 172.21.15.181:0/1279594871 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54340043e0 con 0x7f544015e6d0 2023-10-16T11:19:16.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 -- 172.21.15.181:0/1279594871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f544015e6d0 msgr2=0x7f54400d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 --2- 172.21.15.181:0/1279594871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f544015e6d0 0x7f54400d0fc0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f5434006fd0 tx=0x7f5434009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 -- 172.21.15.181:0/1279594871 shutdown_connections 2023-10-16T11:19:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 --2- 172.21.15.181:0/1279594871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400cb0c0 0x7f544015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 --2- 172.21.15.181:0/1279594871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f544015e6d0 0x7f54400d0fc0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 -- 172.21.15.181:0/1279594871 >> 172.21.15.181:0/1279594871 conn(0x7f54400d04a0 msgr2=0x7f54400d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:16.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 -- 172.21.15.181:0/1279594871 shutdown_connections 2023-10-16T11:19:16.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.311+0000 7f54462ff700 1 -- 172.21.15.181:0/1279594871 wait complete. 2023-10-16T11:19:16.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.313+0000 7f54462ff700 1 Processor -- start 2023-10-16T11:19:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.313+0000 7f54462ff700 1 -- start start 2023-10-16T11:19:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f54462ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54400cb0c0 0x7f544015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f54462ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544015e6d0 0x7f54401e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f54462ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544015eab0 0x7f54401e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:16.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f54462ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54401e44f0 con 0x7f54400cb0c0 2023-10-16T11:19:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f54462ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54401e47d0 con 0x7f544015e6d0 2023-10-16T11:19:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f54462ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54401e4ab0 con 0x7f544015eab0 2023-10-16T11:19:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54400cb0c0 0x7f544015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54400cb0c0 0x7f544015bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55196/0 (socket says 172.21.15.181:55196) 2023-10-16T11:19:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 -- 172.21.15.181:0/2202162146 learned_addr learned my addr 172.21.15.181:0/2202162146 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:16.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f544489c700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544015eab0 0x7f54401e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543f7fe700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544015e6d0 0x7f54401e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 -- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544015eab0 msgr2=0x7f54401e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544015eab0 0x7f54401e3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 -- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544015e6d0 msgr2=0x7f54401e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:16.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544015e6d0 0x7f54401e5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543ffff700 1 -- 172.21.15.181:0/2202162146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5434009500 con 0x7f54400cb0c0 2023-10-16T11:19:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f544489c700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544015eab0 0x7f54401e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.314+0000 7f543f7fe700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544015e6d0 0x7f54401e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.315+0000 7f543ffff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54400cb0c0 0x7f544015bc10 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f5434009130 tx=0x7f5434004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.315+0000 7f543cff9700 1 -- 172.21.15.181:0/2202162146 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543401e070 con 0x7f54400cb0c0 2023-10-16T11:19:16.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.315+0000 7f543cff9700 1 -- 172.21.15.181:0/2202162146 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5434003b10 con 0x7f54400cb0c0 2023-10-16T11:19:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.315+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54401e4d90 con 0x7f54400cb0c0 2023-10-16T11:19:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.315+0000 7f543cff9700 1 -- 172.21.15.181:0/2202162146 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434003ea0 con 0x7f54400cb0c0 2023-10-16T11:19:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.315+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54401e41e0 con 0x7f54400cb0c0 2023-10-16T11:19:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.316+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54401e52b0 con 0x7f54400cb0c0 2023-10-16T11:19:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.316+0000 7f543cff9700 1 -- 172.21.15.181:0/2202162146 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5434018420 con 0x7f54400cb0c0 2023-10-16T11:19:16.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.317+0000 7f543cff9700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f542c06b050 0x7f542c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.317+0000 7f543cff9700 1 -- 172.21.15.181:0/2202162146 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f543408bb20 con 0x7f54400cb0c0 2023-10-16T11:19:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.317+0000 7f543f7fe700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f542c06b050 0x7f542c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.317+0000 7f543f7fe700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f542c06b050 0x7f542c06d510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54401f3af0 tx=0x7f5430009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:16.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.321+0000 7f543cff9700 1 -- 172.21.15.181:0/2202162146 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5434056920 con 0x7f54400cb0c0 2023-10-16T11:19:16.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:16.595+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f5440066fb0 con 0x7f54400cb0c0 2023-10-16T11:19:17.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.493+0000 7f543cff9700 1 -- 172.21.15.181:0/2202162146 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3096) v1 ==== 189+0+0 (secure 0 0 0) 0x7f5434016030 con 0x7f54400cb0c0 2023-10-16T11:19:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f542c06b050 msgr2=0x7f542c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f542c06b050 0x7f542c06d510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f54401f3af0 tx=0x7f5430009040 comp rx=0 tx=0).stop 2023-10-16T11:19:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54400cb0c0 msgr2=0x7f544015bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54400cb0c0 0x7f544015bc10 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f5434009130 tx=0x7f5434004750 comp rx=0 tx=0).stop 2023-10-16T11:19:17.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 shutdown_connections 2023-10-16T11:19:17.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f542c06b050 0x7f542c06d510 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f544015eab0 0x7f54401e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54400cb0c0 0x7f544015bc10 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 --2- 172.21.15.181:0/2202162146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f544015e6d0 0x7f54401e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 >> 172.21.15.181:0/2202162146 conn(0x7f54400d04a0 msgr2=0x7f544015a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:17.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 shutdown_connections 2023-10-16T11:19:17.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.496+0000 7f54462ff700 1 -- 172.21.15.181:0/2202162146 wait complete. 2023-10-16T11:19:17.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:19:17.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.639+0000 7f9cc0163700 1 Processor -- start 2023-10-16T11:19:17.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.639+0000 7f9cc0163700 1 -- start start 2023-10-16T11:19:17.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cc0163700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cb815dbd0 0x7f9cb80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:17.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cc0163700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb806b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:17.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cc0163700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb815dfb0 con 0x7f9cb80d0610 2023-10-16T11:19:17.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cc0163700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb80cd3a0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cc0163700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80cd830 con 0x7f9cb80ce490 2023-10-16T11:19:17.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cbdeff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cb815dbd0 0x7f9cb80d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:17.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cbe700700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb806b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:17.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cbe700700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb806b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:52234/0 (socket says 172.21.15.181:52234) 2023-10-16T11:19:17.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cbdeff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cb815dbd0 0x7f9cb80d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:60254/0 (socket says 172.21.15.181:60254) 2023-10-16T11:19:17.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cbe700700 1 -- 172.21.15.181:0/2371670244 learned_addr learned my addr 172.21.15.181:0/2371670244 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:17.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.640+0000 7f9cbd6fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9cb80d0610 0x7f9cb80cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:50224/0 (socket says 172.21.15.181:50224) 2023-10-16T11:19:17.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbdeff700 1 -- 172.21.15.181:0/2371670244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 msgr2=0x7f9cb806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:17.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbdeff700 1 --2- 172.21.15.181:0/2371670244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb806b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbdeff700 1 -- 172.21.15.181:0/2371670244 >> v1:172.21.15.190:6789/0 conn(0x7f9cb80d0610 legacy=0x7f9cb80cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbdeff700 1 -- 172.21.15.181:0/2371670244 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80d1fc0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbcefd700 1 -- 172.21.15.181:0/2371670244 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344429839 0 0) 0x7f9ca8001110 con 0x7f9cb80d0610 2023-10-16T11:19:17.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbdeff700 1 --2- 172.21.15.181:0/2371670244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb815dbd0 0x7f9cb80d00d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009840 tx=0x7f9cb4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f9c5dd9e73d45c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:17.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbcefd700 1 -- 172.21.15.181:0/2371670244 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4011830 con 0x7f9cb815dbd0 2023-10-16T11:19:17.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbcefd700 1 -- 172.21.15.181:0/2371670244 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb4011990 con 0x7f9cb815dbd0 2023-10-16T11:19:17.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cbcefd700 1 -- 172.21.15.181:0/2371670244 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4011d20 con 0x7f9cb815dbd0 2023-10-16T11:19:17.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cc0163700 1 -- 172.21.15.181:0/2371670244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb815dbd0 msgr2=0x7f9cb80d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.641+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2371670244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb815dbd0 0x7f9cb80d00d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009840 tx=0x7f9cb4009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:17.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.642+0000 7f9cc0163700 1 -- 172.21.15.181:0/2371670244 shutdown_connections 2023-10-16T11:19:17.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.642+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2371670244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb806b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.642+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2371670244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb815dbd0 0x7f9cb80d00d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.642+0000 7f9cc0163700 1 -- 172.21.15.181:0/2371670244 >> 172.21.15.181:0/2371670244 conn(0x7f9cb809a930 msgr2=0x7f9cb80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:17.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.642+0000 7f9cc0163700 1 -- 172.21.15.181:0/2371670244 shutdown_connections 2023-10-16T11:19:17.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.642+0000 7f9cc0163700 1 -- 172.21.15.181:0/2371670244 wait complete. 2023-10-16T11:19:17.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.643+0000 7f9cc0163700 1 Processor -- start 2023-10-16T11:19:17.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.643+0000 7f9cc0163700 1 -- start start 2023-10-16T11:19:17.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb81601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:17.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb80d0610 0x7f9cb81f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:17.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb815dbd0 0x7f9cb81f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:17.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80cb5d0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb81f3160 con 0x7f9cb80d0610 2023-10-16T11:19:17.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb815ff70 con 0x7f9cb80ce490 2023-10-16T11:19:17.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb815dbd0 0x7f9cb81f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:17.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb815dbd0 0x7f9cb81f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55214/0 (socket says 172.21.15.181:55214) 2023-10-16T11:19:17.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 -- 172.21.15.181:0/2795894536 learned_addr learned my addr 172.21.15.181:0/2795894536 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:17.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbd6fe700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb80d0610 0x7f9cb81f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:17.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbdeff700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb81601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 -- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 msgr2=0x7f9cb81601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb81601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 -- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb80d0610 msgr2=0x7f9cb81f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb80d0610 0x7f9cb81f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 -- 172.21.15.181:0/2795894536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb40094a0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cbe700700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb815dbd0 0x7f9cb81f35f0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000efc0 tx=0x7f9cb000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cae7fc700 1 -- 172.21.15.181:0/2795894536 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0003c20 con 0x7f9cb815dbd0 2023-10-16T11:19:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cae7fc700 1 -- 172.21.15.181:0/2795894536 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb000f040 con 0x7f9cb815dbd0 2023-10-16T11:19:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb80c8bd0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cae7fc700 1 -- 172.21.15.181:0/2795894536 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000c660 con 0x7f9cb815dbd0 2023-10-16T11:19:17.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.644+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb80c9190 con 0x7f9cb815dbd0 2023-10-16T11:19:17.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.645+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb8067d20 con 0x7f9cb815dbd0 2023-10-16T11:19:17.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.646+0000 7f9cae7fc700 1 -- 172.21.15.181:0/2795894536 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9cb000c7c0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.646+0000 7f9cae7fc700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9c9c06b020 0x7f9c9c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:17.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.646+0000 7f9cae7fc700 1 -- 172.21.15.181:0/2795894536 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f9cb0089c00 con 0x7f9cb815dbd0 2023-10-16T11:19:17.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.646+0000 7f9cbdeff700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9c9c06b020 0x7f9c9c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:17.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.647+0000 7f9cbdeff700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9c9c06b020 0x7f9c9c06d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4000c00 tx=0x7f9cb4008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:17.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.651+0000 7f9cae7fc700 1 -- 172.21.15.181:0/2795894536 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9cb0090080 con 0x7f9cb815dbd0 2023-10-16T11:19:17.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.843+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9cb81f32a0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.846+0000 7f9cae7fc700 1 -- 172.21.15.181:0/2795894536 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9cb00548f0 con 0x7f9cb815dbd0 2023-10-16T11:19:17.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9c9c06b020 msgr2=0x7f9c9c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9c9c06b020 0x7f9c9c06d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4000c00 tx=0x7f9cb4008030 comp rx=0 tx=0).stop 2023-10-16T11:19:17.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb815dbd0 msgr2=0x7f9cb81f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:17.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb815dbd0 0x7f9cb81f35f0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000efc0 tx=0x7f9cb000c0c0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 shutdown_connections 2023-10-16T11:19:17.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9c9c06b020 0x7f9c9c06d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cb80ce490 0x7f9cb81601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb815dbd0 0x7f9cb81f35f0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 --2- 172.21.15.181:0/2795894536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cb80d0610 0x7f9cb81f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:17.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 >> 172.21.15.181:0/2795894536 conn(0x7f9cb809a930 msgr2=0x7f9cb815ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:17.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 shutdown_connections 2023-10-16T11:19:17.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:17.848+0000 7f9cc0163700 1 -- 172.21.15.181:0/2795894536 wait complete. 2023-10-16T11:19:17.862 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:19:17.863 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:19:17.905 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:19:17.953 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:19:17.953 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:17.954 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:17.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:19:17.971 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:19:17.972 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:17.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:19:17.991 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:17.992 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:18.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:19:18.027 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:19:18.028 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:18.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:19:18.051 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:19:18.051 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:18.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.186+0000 7f5eaacf8700 1 Processor -- start 2023-10-16T11:19:18.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.186+0000 7f5eaacf8700 1 -- start start 2023-10-16T11:19:18.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5eaacf8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5ea415e6d0 0x7f5ea40d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5eaacf8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5eaacf8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea406ed00 con 0x7f5ea415eab0 2023-10-16T11:19:18.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5eaacf8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea40d02d0 con 0x7f5ea415e6d0 2023-10-16T11:19:18.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5eaacf8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea415ad40 con 0x7f5ea40cb0c0 2023-10-16T11:19:18.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea8a94700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5ea415e6d0 0x7f5ea40d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5ea415eab0 0x7f5ea415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38064/0 (socket says 172.21.15.181:38064) 2023-10-16T11:19:18.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea3fff700 1 -- 172.21.15.181:0/2848414313 learned_addr learned my addr 172.21.15.181:0/2848414313 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:18.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea8a94700 1 -- 172.21.15.181:0/2848414313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 msgr2=0x7f5ea415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:18.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea9295700 1 --2- 172.21.15.181:0/2848414313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea8a94700 1 --2- 172.21.15.181:0/2848414313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea415a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea37fe700 1 -- 172.21.15.181:0/2848414313 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570628361 0 0) 0x7f5ea406ed00 con 0x7f5ea415eab0 2023-10-16T11:19:18.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea8a94700 1 -- 172.21.15.181:0/2848414313 >> v1:172.21.15.181:6789/0 conn(0x7f5ea415eab0 legacy=0x7f5ea415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.187+0000 7f5ea8a94700 1 -- 172.21.15.181:0/2848414313 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea406e0f0 con 0x7f5ea415e6d0 2023-10-16T11:19:18.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5ea8a94700 1 --2- 172.21.15.181:0/2848414313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415e6d0 0x7f5ea40d0fc0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009820 tx=0x7f5e98009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc15fe153f6392e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:18.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5ea37fe700 1 -- 172.21.15.181:0/2848414313 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98005e30 con 0x7f5ea415e6d0 2023-10-16T11:19:18.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5ea37fe700 1 -- 172.21.15.181:0/2848414313 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98006030 con 0x7f5ea415e6d0 2023-10-16T11:19:18.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5ea37fe700 1 -- 172.21.15.181:0/2848414313 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980063e0 con 0x7f5ea415e6d0 2023-10-16T11:19:18.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 -- 172.21.15.181:0/2848414313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415e6d0 msgr2=0x7f5ea40d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/2848414313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415e6d0 0x7f5ea40d0fc0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009820 tx=0x7f5e98009b30 comp rx=0 tx=0).stop 2023-10-16T11:19:18.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 -- 172.21.15.181:0/2848414313 shutdown_connections 2023-10-16T11:19:18.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/2848414313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/2848414313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415e6d0 0x7f5ea40d0fc0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 -- 172.21.15.181:0/2848414313 >> 172.21.15.181:0/2848414313 conn(0x7f5ea40d04a0 msgr2=0x7f5ea40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:18.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 -- 172.21.15.181:0/2848414313 shutdown_connections 2023-10-16T11:19:18.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.188+0000 7f5eaacf8700 1 -- 172.21.15.181:0/2848414313 wait complete. 2023-10-16T11:19:18.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.189+0000 7f5eaacf8700 1 Processor -- start 2023-10-16T11:19:18.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.189+0000 7f5eaacf8700 1 -- start start 2023-10-16T11:19:18.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5eaacf8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea40ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5eaacf8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ea415e6d0 0x7f5ea40ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5eaacf8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415eab0 0x7f5ea415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5eaacf8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40b4930 con 0x7f5ea415eab0 2023-10-16T11:19:18.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5eaacf8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40b3890 con 0x7f5ea415e6d0 2023-10-16T11:19:18.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5eaacf8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40b3b70 con 0x7f5ea40cb0c0 2023-10-16T11:19:18.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5ea9295700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415eab0 0x7f5ea415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5ea9295700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415eab0 0x7f5ea415c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55226/0 (socket says 172.21.15.181:55226) 2023-10-16T11:19:18.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.190+0000 7f5ea9295700 1 -- 172.21.15.181:0/446888293 learned_addr learned my addr 172.21.15.181:0/446888293 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:18.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea8a94700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea40ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea3fff700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ea415e6d0 0x7f5ea40ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea9295700 1 -- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 msgr2=0x7f5ea40ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea9295700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea40ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea9295700 1 -- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ea415e6d0 msgr2=0x7f5ea40ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea9295700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ea415e6d0 0x7f5ea40ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea9295700 1 -- 172.21.15.181:0/446888293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009430 con 0x7f5ea415eab0 2023-10-16T11:19:18.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea9295700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415eab0 0x7f5ea415c6e0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00cc30 tx=0x7f5e9c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:18.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea17fa700 1 -- 172.21.15.181:0/446888293 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00aa50 con 0x7f5ea415eab0 2023-10-16T11:19:18.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea17fa700 1 -- 172.21.15.181:0/446888293 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c00abb0 con 0x7f5ea415eab0 2023-10-16T11:19:18.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea40b3e50 con 0x7f5ea415eab0 2023-10-16T11:19:18.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5ea17fa700 1 -- 172.21.15.181:0/446888293 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c003a20 con 0x7f5ea415eab0 2023-10-16T11:19:18.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.191+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea40b4410 con 0x7f5ea415eab0 2023-10-16T11:19:18.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.192+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea40b4630 con 0x7f5ea415eab0 2023-10-16T11:19:18.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.192+0000 7f5ea17fa700 1 -- 172.21.15.181:0/446888293 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5e9c00f3f0 con 0x7f5ea415eab0 2023-10-16T11:19:18.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.193+0000 7f5ea17fa700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5e8806b020 0x7f5e8806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.193+0000 7f5ea17fa700 1 -- 172.21.15.181:0/446888293 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f5e9c089ae0 con 0x7f5ea415eab0 2023-10-16T11:19:18.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.193+0000 7f5ea8a94700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5e8806b020 0x7f5e8806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.193+0000 7f5ea8a94700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5e8806b020 0x7f5e8806d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e98013040 tx=0x7f5e98008070 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:18.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.197+0000 7f5ea17fa700 1 -- 172.21.15.181:0/446888293 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5e9c0547d0 con 0x7f5ea415eab0 2023-10-16T11:19:18.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.434+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ea415a240 con 0x7f5ea415eab0 2023-10-16T11:19:18.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.435+0000 7f5ea17fa700 1 -- 172.21.15.181:0/446888293 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3715 (secure 0 0 0) 0x7f5e9c01d070 con 0x7f5ea415eab0 2023-10-16T11:19:18.438 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:18.438 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8489,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3757723265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3757723265},{"type":"v1","addr":"172.21.15.190:6834","nonce":3757723265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2012626771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2012626771},{"type":"v1","addr":"172.21.15.190:6835","nonce":2012626771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10782,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3218502962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3218502962},{"type":"v1","addr":"172.21.15.181:6836","nonce":3218502962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":11226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1099590361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1099590361},{"type":"v1","addr":"172.21.15.181:6837","nonce":1099590361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":104}],"filesystems":[]} 2023-10-16T11:19:18.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5e8806b020 msgr2=0x7f5e8806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5e8806b020 0x7f5e8806d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e98013040 tx=0x7f5e98008070 comp rx=0 tx=0).stop 2023-10-16T11:19:18.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415eab0 msgr2=0x7f5ea415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415eab0 0x7f5ea415c6e0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00cc30 tx=0x7f5e9c00cf40 comp rx=0 tx=0).stop 2023-10-16T11:19:18.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 shutdown_connections 2023-10-16T11:19:18.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f5e8806b020 0x7f5e8806d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ea40cb0c0 0x7f5ea40ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ea415eab0 0x7f5ea415c6e0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 --2- 172.21.15.181:0/446888293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ea415e6d0 0x7f5ea40ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 >> 172.21.15.181:0/446888293 conn(0x7f5ea40d04a0 msgr2=0x7f5ea40c4910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:18.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 shutdown_connections 2023-10-16T11:19:18.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.437+0000 7f5eaacf8700 1 -- 172.21.15.181:0/446888293 wait complete. 2023-10-16T11:19:18.442 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 104 2023-10-16T11:19:18.451 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:19:18.452 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:19:18.452 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:19:18.452 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:19:18.453 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:19:18.453 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:19:18.453 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:19:18.453 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:19:18.454 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:19:18.454 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:19:18.454 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:19:18.454 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:19:18.455 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:19:18.455 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:19:18.455 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:19:18.456 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:19:18.461 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:19:18.461 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:19:18.462 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:19:18.462 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:19:18.465 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:19:18.465 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:19:18.466 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:19:18.467 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:19:18.504 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:19:18.505 DEBUG:teuthology.parallel:result is None 2023-10-16T11:19:18.506 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:19:18.506 DEBUG:teuthology.parallel:result is None 2023-10-16T11:19:18.508 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:19:18.509 DEBUG:teuthology.parallel:result is None 2023-10-16T11:19:18.509 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:19:18.510 DEBUG:teuthology.parallel:result is None 2023-10-16T11:19:18.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:19:18.583 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:19:18.584 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:19:18.594 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:19:18.594 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:19:18.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.636+0000 7f415e9a3700 1 Processor -- start 2023-10-16T11:19:18.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.636+0000 7f415e9a3700 1 -- start start 2023-10-16T11:19:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f415e9a3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41580cb110 0x7f41580d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f415e9a3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415806ed00 con 0x7f415815e460 2023-10-16T11:19:18.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f415e9a3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415806e0f0 con 0x7f415815a0f0 2023-10-16T11:19:18.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f415e9a3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415806e3d0 con 0x7f41580cb110 2023-10-16T11:19:18.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4157fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f415815e460 0x7f415815e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38100/0 (socket says 172.21.15.181:38100) 2023-10-16T11:19:18.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4157fff700 1 -- 172.21.15.181:0/1687191627 learned_addr learned my addr 172.21.15.181:0/1687191627 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:18.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f41577fe700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41580cb110 0x7f41580d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146443463 0 0) 0x7f415806ed00 con 0x7f415815e460 2023-10-16T11:19:18.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41400035b0 con 0x7f415815e460 2023-10-16T11:19:18.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605260765 0 0) 0x7f41400035b0 con 0x7f415815e460 2023-10-16T11:19:18.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41580cb110 msgr2=0x7f41580d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:18.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41580cb110 0x7f41580d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 >> v1:172.21.15.190:6789/0 conn(0x7f415815a0f0 legacy=0x7f415815a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415806e6b0 con 0x7f415815e460 2023-10-16T11:19:18.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.637+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f41480017f0 con 0x7f415815e460 2023-10-16T11:19:18.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 >> v1:172.21.15.181:6789/0 conn(0x7f415815e460 legacy=0x7f415815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4140003a30 0x7f4140003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140004390 0x7f4140004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4140005070 0x7f414000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415806e3d0 con 0x7f4140005070 2023-10-16T11:19:18.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414000bf80 con 0x7f4140004390 2023-10-16T11:19:18.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414000c140 con 0x7f4140003a30 2023-10-16T11:19:18.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f41577fe700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41580cb110 0x7f41580d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:18.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f415cf40700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4140005070 0x7f414000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4157fff700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4140003a30 0x7f4140003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f41577fe700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140004390 0x7f4140004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f415cf40700 1 -- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4140003a30 msgr2=0x7f4140003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:18.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f415cf40700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4140003a30 0x7f4140003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f415cf40700 1 -- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140004390 msgr2=0x7f4140004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:18.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f415cf40700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4140004390 0x7f4140004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f415cf40700 1 -- 172.21.15.181:0/1687191627 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41400035b0 con 0x7f4140005070 2023-10-16T11:19:18.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f415cf40700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4140005070 0x7f414000ba10 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f415000c290 tx=0x7f4140005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d423b24596a2c4f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:18.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150001e70 con 0x7f4140005070 2023-10-16T11:19:18.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415000ce50 con 0x7f4140005070 2023-10-16T11:19:18.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.638+0000 7f4156ffd700 1 -- 172.21.15.181:0/1687191627 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150012660 con 0x7f4140005070 2023-10-16T11:19:18.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.639+0000 7f415e9a3700 1 -- 172.21.15.181:0/1687191627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4140005070 msgr2=0x7f414000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.639+0000 7f415e9a3700 1 --2- 172.21.15.181:0/1687191627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4140005070 0x7f414000ba10 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f415000c290 tx=0x7f4140005880 comp rx=0 tx=0).stop 2023-10-16T11:19:18.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.639+0000 7f415cf40700 1 -- 172.21.15.181:0/1687191627 reap_dead start 2023-10-16T11:19:18.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.639+0000 7f415e9a3700 1 -- 172.21.15.181:0/1687191627 shutdown_connections 2023-10-16T11:19:18.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.639+0000 7f415e9a3700 1 -- 172.21.15.181:0/1687191627 >> 172.21.15.181:0/1687191627 conn(0x7f41580d04a0 msgr2=0x7f41580d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:18.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.639+0000 7f415e9a3700 1 -- 172.21.15.181:0/1687191627 shutdown_connections 2023-10-16T11:19:18.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.639+0000 7f415e9a3700 1 -- 172.21.15.181:0/1687191627 wait complete. 2023-10-16T11:19:18.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.640+0000 7f415e9a3700 1 Processor -- start 2023-10-16T11:19:18.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.641+0000 7f415e9a3700 1 -- start start 2023-10-16T11:19:18.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.641+0000 7f415e9a3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41581532f0 0x7f41581e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.641+0000 7f415e9a3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41581e5fc0 0x7f41580b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.641+0000 7f415e9a3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41581536d0 0x7f41580b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.641+0000 7f415e9a3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41580c40a0 con 0x7f41581e5fc0 2023-10-16T11:19:18.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.641+0000 7f415e9a3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41580c4360 con 0x7f41581536d0 2023-10-16T11:19:18.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.641+0000 7f415e9a3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41580c4640 con 0x7f41581532f0 2023-10-16T11:19:18.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41581e5fc0 0x7f41580b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41581e5fc0 0x7f41580b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55268/0 (socket says 172.21.15.181:55268) 2023-10-16T11:19:18.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 -- 172.21.15.181:0/518173742 learned_addr learned my addr 172.21.15.181:0/518173742 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:18.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f415cf40700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41581536d0 0x7f41580b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f4157fff700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41581532f0 0x7f41581e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 -- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41581532f0 msgr2=0x7f41581e5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41581532f0 0x7f41581e5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 -- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41581536d0 msgr2=0x7f41580b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:18.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41581536d0 0x7f41580b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:18.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 -- 172.21.15.181:0/518173742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580c4920 con 0x7f41581e5fc0 2023-10-16T11:19:18.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f41577fe700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41581e5fc0 0x7f41580b18b0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f414c011c00 tx=0x7f414c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:18.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f4154ff9700 1 -- 172.21.15.181:0/518173742 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c00f980 con 0x7f41581e5fc0 2023-10-16T11:19:18.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f4154ff9700 1 -- 172.21.15.181:0/518173742 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c00fae0 con 0x7f41581e5fc0 2023-10-16T11:19:18.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415815d230 con 0x7f41581e5fc0 2023-10-16T11:19:18.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f4154ff9700 1 -- 172.21.15.181:0/518173742 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c00fe70 con 0x7f41581e5fc0 2023-10-16T11:19:18.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.642+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415815b770 con 0x7f41581e5fc0 2023-10-16T11:19:18.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.643+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4158067d20 con 0x7f41581e5fc0 2023-10-16T11:19:18.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.643+0000 7f4154ff9700 1 -- 172.21.15.181:0/518173742 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f414c004030 con 0x7f41581e5fc0 2023-10-16T11:19:18.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.644+0000 7f4154ff9700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f414406b020 0x7f414406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:18.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.644+0000 7f4154ff9700 1 -- 172.21.15.181:0/518173742 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f414c089e30 con 0x7f41581e5fc0 2023-10-16T11:19:18.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.644+0000 7f4157fff700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f414406b020 0x7f414406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:18.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.644+0000 7f4157fff700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f414406b020 0x7f414406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f41480011d0 tx=0x7f4148009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:18.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.648+0000 7f4154ff9700 1 -- 172.21.15.181:0/518173742 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f414c054ba0 con 0x7f41581e5fc0 2023-10-16T11:19:18.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:18.855+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f41580cb610 con 0x7f41581e5fc0 2023-10-16T11:19:19.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.504+0000 7f4154ff9700 1 -- 172.21.15.181:0/518173742 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v210) v1 ==== 97+0+0 (secure 0 0 0) 0x7f414c022020 con 0x7f41581e5fc0 2023-10-16T11:19:19.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f414406b020 msgr2=0x7f414406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f414406b020 0x7f414406d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f41480011d0 tx=0x7f4148009040 comp rx=0 tx=0).stop 2023-10-16T11:19:19.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41581e5fc0 msgr2=0x7f41580b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41581e5fc0 0x7f41580b18b0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f414c011c00 tx=0x7f414c011f10 comp rx=0 tx=0).stop 2023-10-16T11:19:19.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 shutdown_connections 2023-10-16T11:19:19.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f414406b020 0x7f414406d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41581532f0 0x7f41581e5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41581536d0 0x7f41580b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 --2- 172.21.15.181:0/518173742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41581e5fc0 0x7f41580b18b0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.507+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 >> 172.21.15.181:0/518173742 conn(0x7f41580d04a0 msgr2=0x7f415815e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:19.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.508+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 shutdown_connections 2023-10-16T11:19:19.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.508+0000 7f415e9a3700 1 -- 172.21.15.181:0/518173742 wait complete. 2023-10-16T11:19:19.512 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T11:19:19.527 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:19:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.649+0000 7f095b75d700 1 Processor -- start 2023-10-16T11:19:19.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.649+0000 7f095b75d700 1 -- start start 2023-10-16T11:19:19.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f095b75d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f095415e6d0 0x7f09540d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:19.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f095b75d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09540cb0c0 0x7f095415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:19.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f095b75d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095406ed00 con 0x7f095415eab0 2023-10-16T11:19:19.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f095b75d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09540d02d0 con 0x7f09540cb0c0 2023-10-16T11:19:19.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f095b75d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095415ad40 con 0x7f095415e6d0 2023-10-16T11:19:19.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f0959cfa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09540cb0c0 0x7f095415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:19.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f0959cfa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09540cb0c0 0x7f095415a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55280/0 (socket says 172.21.15.181:55280) 2023-10-16T11:19:19.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f0959cfa700 1 -- 172.21.15.181:0/3308971715 learned_addr learned my addr 172.21.15.181:0/3308971715 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:19.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f09594f9700 1 --2- 172.21.15.181:0/3308971715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f095415e6d0 0x7f09540d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:19.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f0959cfa700 1 -- 172.21.15.181:0/3308971715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f095415e6d0 msgr2=0x7f09540d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:19.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f0959cfa700 1 --2- 172.21.15.181:0/3308971715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f095415e6d0 0x7f09540d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f0959cfa700 1 -- 172.21.15.181:0/3308971715 >> v1:172.21.15.190:6789/0 conn(0x7f095415eab0 legacy=0x7f095415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.650+0000 7f0959cfa700 1 -- 172.21.15.181:0/3308971715 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095406e0f0 con 0x7f09540cb0c0 2023-10-16T11:19:19.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f0959cfa700 1 --2- 172.21.15.181:0/3308971715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09540cb0c0 0x7f095415a800 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f094c00d150 tx=0x7f094c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b568d3ef94c39900 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:19.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f094bfff700 1 -- 172.21.15.181:0/3308971715 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c010070 con 0x7f09540cb0c0 2023-10-16T11:19:19.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f094bfff700 1 -- 172.21.15.181:0/3308971715 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00ddc0 con 0x7f09540cb0c0 2023-10-16T11:19:19.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f094bfff700 1 -- 172.21.15.181:0/3308971715 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00b310 con 0x7f09540cb0c0 2023-10-16T11:19:19.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 -- 172.21.15.181:0/3308971715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09540cb0c0 msgr2=0x7f095415a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 --2- 172.21.15.181:0/3308971715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09540cb0c0 0x7f095415a800 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f094c00d150 tx=0x7f094c00d460 comp rx=0 tx=0).stop 2023-10-16T11:19:19.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 -- 172.21.15.181:0/3308971715 shutdown_connections 2023-10-16T11:19:19.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 --2- 172.21.15.181:0/3308971715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09540cb0c0 0x7f095415a800 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 --2- 172.21.15.181:0/3308971715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f095415e6d0 0x7f09540d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 -- 172.21.15.181:0/3308971715 >> 172.21.15.181:0/3308971715 conn(0x7f09540d04a0 msgr2=0x7f09540d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:19.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 -- 172.21.15.181:0/3308971715 shutdown_connections 2023-10-16T11:19:19.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.651+0000 7f095b75d700 1 -- 172.21.15.181:0/3308971715 wait complete. 2023-10-16T11:19:19.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 Processor -- start 2023-10-16T11:19:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 -- start start 2023-10-16T11:19:19.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cb0c0 0x7f09541e4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:19.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f095415e6d0 0x7f09540b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:19.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f095415eab0 0x7f095414e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:19.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540b4f20 con 0x7f095415eab0 2023-10-16T11:19:19.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540b5200 con 0x7f095415e6d0 2023-10-16T11:19:19.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f095b75d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540b54e0 con 0x7f09540cb0c0 2023-10-16T11:19:19.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f0959cfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f095415eab0 0x7f095414e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:19.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f0959cfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f095415eab0 0x7f095414e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55286/0 (socket says 172.21.15.181:55286) 2023-10-16T11:19:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f0959cfa700 1 -- 172.21.15.181:0/3917880708 learned_addr learned my addr 172.21.15.181:0/3917880708 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.653+0000 7f09594f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cb0c0 0x7f09541e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0958cf8700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f095415e6d0 0x7f09540b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0959cfa700 1 -- 172.21.15.181:0/3917880708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cb0c0 msgr2=0x7f09541e4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0959cfa700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cb0c0 0x7f09541e4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0959cfa700 1 -- 172.21.15.181:0/3917880708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f095415e6d0 msgr2=0x7f09540b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0959cfa700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f095415e6d0 0x7f09540b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0959cfa700 1 -- 172.21.15.181:0/3917880708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c00bd40 con 0x7f095415eab0 2023-10-16T11:19:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0959cfa700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f095415eab0 0x7f095414e380 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f0954071580 tx=0x7f094c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0949ffb700 1 -- 172.21.15.181:0/3917880708 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00bbc0 con 0x7f095415eab0 2023-10-16T11:19:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0949ffb700 1 -- 172.21.15.181:0/3917880708 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00b4c0 con 0x7f095415eab0 2023-10-16T11:19:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09540b57c0 con 0x7f095415eab0 2023-10-16T11:19:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f0949ffb700 1 -- 172.21.15.181:0/3917880708 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00ae70 con 0x7f095415eab0 2023-10-16T11:19:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.654+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095414e8c0 con 0x7f095415eab0 2023-10-16T11:19:19.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.655+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095414eab0 con 0x7f095415eab0 2023-10-16T11:19:19.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.656+0000 7f0949ffb700 1 -- 172.21.15.181:0/3917880708 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f094c004040 con 0x7f095415eab0 2023-10-16T11:19:19.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.656+0000 7f0949ffb700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f093806b020 0x7f093806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:19.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.656+0000 7f0949ffb700 1 -- 172.21.15.181:0/3917880708 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f094c08cac0 con 0x7f095415eab0 2023-10-16T11:19:19.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.656+0000 7f09594f9700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f093806b020 0x7f093806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:19.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.656+0000 7f09594f9700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f093806b020 0x7f093806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:19.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.660+0000 7f0949ffb700 1 -- 172.21.15.181:0/3917880708 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f094c057b00 con 0x7f095415eab0 2023-10-16T11:19:19.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.932+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0954067d20 con 0x7f095415eab0 2023-10-16T11:19:19.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.932+0000 7f0949ffb700 1 -- 172.21.15.181:0/3917880708 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v50) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f094c021090 con 0x7f095415eab0 2023-10-16T11:19:19.935 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:19.935 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:19:19.935 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T11:19:19.935 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.936 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T11:19:19.936 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T11:19:19.936 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.936 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.937 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:19:19.937 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:19:19.937 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.937 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.938 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T11:19:19.938 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T11:19:19.938 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.938 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.939 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:19:19.939 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:19:19.939 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.939 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.940 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.940 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T11:19:19.940 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T11:19:19.940 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.941 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.941 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:19:19.941 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:19:19.941 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.941 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.942 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T11:19:19.943 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T11:19:19.943 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.943 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.943 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:19:19.943 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:19:19.944 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.944 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.944 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T11:19:19.945 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T11:19:19.945 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.945 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.945 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.945 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.946 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.946 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.946 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T11:19:19.947 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T11:19:19.947 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.947 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.947 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.947 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.948 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.948 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T11:19:19.949 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T11:19:19.949 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.949 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.949 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.949 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.950 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.950 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T11:19:19.950 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T11:19:19.951 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.951 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.951 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.951 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.952 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.952 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T11:19:19.953 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T11:19:19.953 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.953 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.953 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.954 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.954 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.954 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T11:19:19.955 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T11:19:19.955 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.955 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.955 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.955 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.956 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.956 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T11:19:19.956 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T11:19:19.957 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.957 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.957 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.957 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.958 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.958 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T11:19:19.958 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T11:19:19.959 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.959 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:19:19.959 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:19:19.959 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.960 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T11:19:19.960 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T11:19:19.960 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.961 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.961 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T11:19:19.961 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:19:19.961 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.962 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T11:19:19.962 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T11:19:19.962 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.963 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.963 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:19:19.963 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:19:19.963 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:19:19.963 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.964 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T11:19:19.964 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T11:19:19.964 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.965 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.965 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:19:19.965 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:19:19.965 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:19:19.965 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.966 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.966 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T11:19:19.966 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T11:19:19.967 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.967 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:19:19.967 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:19:19.967 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:19:19.967 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:19:19.968 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.968 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T11:19:19.968 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T11:19:19.969 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.969 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:19:19.969 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:19:19.969 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:19:19.969 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.970 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.970 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T11:19:19.971 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T11:19:19.971 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.971 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:19:19.971 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:19:19.971 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.972 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:19.972 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:19.972 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T11:19:19.972 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T11:19:19.973 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:19:19.973 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:19:19.973 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:19:19.973 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:19:19.974 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.974 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:19.974 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T11:19:19.974 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:19:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.934+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f093806b020 msgr2=0x7f093806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.934+0000 7f095b75d700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f093806b020 0x7f093806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).stop 2023-10-16T11:19:19.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.934+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f095415eab0 msgr2=0x7f095414e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.934+0000 7f095b75d700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f095415eab0 0x7f095414e380 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f0954071580 tx=0x7f094c004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 shutdown_connections 2023-10-16T11:19:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f093806b020 0x7f093806d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cb0c0 0x7f09541e4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f095415eab0 0x7f095414e380 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 --2- 172.21.15.181:0/3917880708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f095415e6d0 0x7f09540b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 >> 172.21.15.181:0/3917880708 conn(0x7f09540d04a0 msgr2=0x7f09540ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 shutdown_connections 2023-10-16T11:19:19.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:19.935+0000 7f095b75d700 1 -- 172.21.15.181:0/3917880708 wait complete. 2023-10-16T11:19:19.979 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:20.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.098+0000 7f99ce0ba700 1 Processor -- start 2023-10-16T11:19:20.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.098+0000 7f99ce0ba700 1 -- start start 2023-10-16T11:19:20.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99ce0ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c815dfd0 0x7f99c80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99ce0ba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99c80cd470 0x7f99c80cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99ce0ba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c806f870 con 0x7f99c80af450 2023-10-16T11:19:20.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99ce0ba700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c80cde40 con 0x7f99c80cd470 2023-10-16T11:19:20.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99ce0ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80d1160 con 0x7f99c815dfd0 2023-10-16T11:19:20.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c6ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f99c80af450 0x7f99c806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38136/0 (socket says 172.21.15.181:38136) 2023-10-16T11:19:20.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99c80cd470 0x7f99c80cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c6ffd700 1 -- 172.21.15.181:0/1604006796 learned_addr learned my addr 172.21.15.181:0/1604006796 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:20.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c77fe700 1 --2- 172.21.15.181:0/1604006796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c815dfd0 0x7f99c80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c7fff700 1 -- 172.21.15.181:0/1604006796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c815dfd0 msgr2=0x7f99c80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:20.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c7fff700 1 --2- 172.21.15.181:0/1604006796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c815dfd0 0x7f99c80af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c7fff700 1 -- 172.21.15.181:0/1604006796 >> v1:172.21.15.181:6789/0 conn(0x7f99c80af450 legacy=0x7f99c806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c7fff700 1 -- 172.21.15.181:0/1604006796 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80cc8e0 con 0x7f99c80cd470 2023-10-16T11:19:20.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c67fc700 1 -- 172.21.15.181:0/1604006796 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286535652 0 0) 0x7f99c806f870 con 0x7f99c80af450 2023-10-16T11:19:20.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.099+0000 7f99c77fe700 1 --2- 172.21.15.181:0/1604006796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c815dfd0 0x7f99c80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:20.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99c7fff700 1 --2- 172.21.15.181:0/1604006796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c80cd470 0x7f99c80cd900 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009d90 tx=0x7f99c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd29cfda53924bb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99c67fc700 1 -- 172.21.15.181:0/1604006796 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c000e070 con 0x7f99c80cd470 2023-10-16T11:19:20.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99c67fc700 1 -- 172.21.15.181:0/1604006796 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c000bbd0 con 0x7f99c80cd470 2023-10-16T11:19:20.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99c67fc700 1 -- 172.21.15.181:0/1604006796 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0012430 con 0x7f99c80cd470 2023-10-16T11:19:20.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99ce0ba700 1 -- 172.21.15.181:0/1604006796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c80cd470 msgr2=0x7f99c80cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/1604006796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c80cd470 0x7f99c80cd900 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009d90 tx=0x7f99c000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99ce0ba700 1 -- 172.21.15.181:0/1604006796 shutdown_connections 2023-10-16T11:19:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/1604006796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c80cd470 0x7f99c80cd900 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/1604006796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c815dfd0 0x7f99c80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99ce0ba700 1 -- 172.21.15.181:0/1604006796 >> 172.21.15.181:0/1604006796 conn(0x7f99c80d04a0 msgr2=0x7f99c80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.100+0000 7f99ce0ba700 1 -- 172.21.15.181:0/1604006796 shutdown_connections 2023-10-16T11:19:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.101+0000 7f99ce0ba700 1 -- 172.21.15.181:0/1604006796 wait complete. 2023-10-16T11:19:20.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.102+0000 7f99ce0ba700 1 Processor -- start 2023-10-16T11:19:20.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.102+0000 7f99ce0ba700 1 -- start start 2023-10-16T11:19:20.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c80af450 0x7f99c814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c80cd470 0x7f99c80c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c815dfd0 0x7f99c80c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8157a40 con 0x7f99c815dfd0 2023-10-16T11:19:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c81e7a80 con 0x7f99c80af450 2023-10-16T11:19:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c81e7190 con 0x7f99c80cd470 2023-10-16T11:19:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c815dfd0 0x7f99c80c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c815dfd0 0x7f99c80c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55300/0 (socket says 172.21.15.181:55300) 2023-10-16T11:19:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 -- 172.21.15.181:0/2301530702 learned_addr learned my addr 172.21.15.181:0/2301530702 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c77fe700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c80af450 0x7f99c814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c6ffd700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c80cd470 0x7f99c80c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 -- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c80cd470 msgr2=0x7f99c80c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c80cd470 0x7f99c80c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 -- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c80af450 msgr2=0x7f99c814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c80af450 0x7f99c814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 -- 172.21.15.181:0/2301530702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0009a70 con 0x7f99c815dfd0 2023-10-16T11:19:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99c7fff700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c815dfd0 0x7f99c80c9b50 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f99c0014040 tx=0x7f99c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99affff700 1 -- 172.21.15.181:0/2301530702 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0012430 con 0x7f99c815dfd0 2023-10-16T11:19:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99affff700 1 -- 172.21.15.181:0/2301530702 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c0020070 con 0x7f99c815dfd0 2023-10-16T11:19:20.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c81e7430 con 0x7f99c815dfd0 2023-10-16T11:19:20.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99affff700 1 -- 172.21.15.181:0/2301530702 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0004280 con 0x7f99c815dfd0 2023-10-16T11:19:20.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.103+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81e7f10 con 0x7f99c815dfd0 2023-10-16T11:19:20.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.104+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c81e7bf0 con 0x7f99c815dfd0 2023-10-16T11:19:20.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.105+0000 7f99affff700 1 -- 172.21.15.181:0/2301530702 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f99c00043e0 con 0x7f99c815dfd0 2023-10-16T11:19:20.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.105+0000 7f99affff700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f99b406b020 0x7f99b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.105+0000 7f99affff700 1 -- 172.21.15.181:0/2301530702 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f99c0022070 con 0x7f99c815dfd0 2023-10-16T11:19:20.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.105+0000 7f99c77fe700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f99b406b020 0x7f99b406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.106+0000 7f99c77fe700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f99b406b020 0x7f99b406d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c920 tx=0x7f99bc008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.110+0000 7f99affff700 1 -- 172.21.15.181:0/2301530702 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f99c0056720 con 0x7f99c815dfd0 2023-10-16T11:19:20.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.301+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c8067d20 con 0x7f99c815dfd0 2023-10-16T11:19:20.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.302+0000 7f99affff700 1 -- 172.21.15.181:0/2301530702 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+13754 (secure 0 0 0) 0x7f99c0059d40 con 0x7f99c815dfd0 2023-10-16T11:19:20.304 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:20.304 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":212,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:19.515142+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":201,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:20.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f99b406b020 msgr2=0x7f99b406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f99b406b020 0x7f99b406d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c920 tx=0x7f99bc008040 comp rx=0 tx=0).stop 2023-10-16T11:19:20.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c815dfd0 msgr2=0x7f99c80c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c815dfd0 0x7f99c80c9b50 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f99c0014040 tx=0x7f99c0004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 shutdown_connections 2023-10-16T11:19:20.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f99b406b020 0x7f99b406d4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99c815dfd0 0x7f99c80c9b50 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c80af450 0x7f99c814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 --2- 172.21.15.181:0/2301530702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c80cd470 0x7f99c80c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 >> 172.21.15.181:0/2301530702 conn(0x7f99c80d04a0 msgr2=0x7f99c81598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 shutdown_connections 2023-10-16T11:19:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.304+0000 7f99ce0ba700 1 -- 172.21.15.181:0/2301530702 wait complete. 2023-10-16T11:19:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:19:20.319 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:19:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.440+0000 7f3eae39c700 1 Processor -- start 2023-10-16T11:19:20.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.440+0000 7f3eae39c700 1 -- start start 2023-10-16T11:19:20.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3eae39c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ea8072760 0x7f3ea806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3eae39c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea806b210 con 0x7f3ea815da70 2023-10-16T11:19:20.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3eae39c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea80cef00 con 0x7f3ea806ade0 2023-10-16T11:19:20.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3eae39c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea80cec20 con 0x7f3ea8072760 2023-10-16T11:19:20.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3ea77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ea8072760 0x7f3ea806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3ea77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ea8072760 0x7f3ea806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55306/0 (socket says 172.21.15.181:55306) 2023-10-16T11:19:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3ea77fe700 1 -- 172.21.15.181:0/1371419249 learned_addr learned my addr 172.21.15.181:0/1371419249 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.441+0000 7f3ea7fff700 1 --1- 172.21.15.181:0/1371419249 >> v1:172.21.15.181:6789/0 conn(0x7f3ea815da70 0x7f3ea80ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38150/0 (socket says 172.21.15.181:38150) 2023-10-16T11:19:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3ea77fe700 1 -- 172.21.15.181:0/1371419249 >> v1:172.21.15.190:6789/0 conn(0x7f3ea806ade0 legacy=0x7f3ea81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3ea77fe700 1 -- 172.21.15.181:0/1371419249 >> v1:172.21.15.181:6789/0 conn(0x7f3ea815da70 legacy=0x7f3ea80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3ea77fe700 1 -- 172.21.15.181:0/1371419249 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8071c80 con 0x7f3ea8072760 2023-10-16T11:19:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3ea77fe700 1 --2- 172.21.15.181:0/1371419249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea806a8a0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f3e980098f0 tx=0x7f3e98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d2d2f3cee4941e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3ea6ffd700 1 -- 172.21.15.181:0/1371419249 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98004230 con 0x7f3ea8072760 2023-10-16T11:19:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3ea6ffd700 1 -- 172.21.15.181:0/1371419249 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98004390 con 0x7f3ea8072760 2023-10-16T11:19:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3ea6ffd700 1 -- 172.21.15.181:0/1371419249 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980036a0 con 0x7f3ea8072760 2023-10-16T11:19:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3eae39c700 1 -- 172.21.15.181:0/1371419249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 msgr2=0x7f3ea806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3eae39c700 1 --2- 172.21.15.181:0/1371419249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea806a8a0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f3e980098f0 tx=0x7f3e98009c00 comp rx=0 tx=0).stop 2023-10-16T11:19:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3eae39c700 1 -- 172.21.15.181:0/1371419249 shutdown_connections 2023-10-16T11:19:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3eae39c700 1 --2- 172.21.15.181:0/1371419249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea806a8a0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3eae39c700 1 -- 172.21.15.181:0/1371419249 >> 172.21.15.181:0/1371419249 conn(0x7f3ea80cd010 msgr2=0x7f3ea80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:20.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.442+0000 7f3eae39c700 1 -- 172.21.15.181:0/1371419249 shutdown_connections 2023-10-16T11:19:20.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.443+0000 7f3eae39c700 1 -- 172.21.15.181:0/1371419249 wait complete. 2023-10-16T11:19:20.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.444+0000 7f3eae39c700 1 Processor -- start 2023-10-16T11:19:20.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.444+0000 7f3eae39c700 1 -- start start 2023-10-16T11:19:20.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.444+0000 7f3eae39c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ea806ade0 0x7f3ea81f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.444+0000 7f3eae39c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea80ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.444+0000 7f3eae39c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ea815da70 0x7f3ea8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3eae39c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea80ad020 con 0x7f3ea8072760 2023-10-16T11:19:20.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3eae39c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea80acdc0 con 0x7f3ea806ade0 2023-10-16T11:19:20.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3eae39c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea80ae440 con 0x7f3ea815da70 2023-10-16T11:19:20.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea80ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea80ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55312/0 (socket says 172.21.15.181:55312) 2023-10-16T11:19:20.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 -- 172.21.15.181:0/916972068 learned_addr learned my addr 172.21.15.181:0/916972068 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:20.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea7fff700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ea806ade0 0x7f3ea81f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3eac939700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ea815da70 0x7f3ea8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 -- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ea815da70 msgr2=0x7f3ea8155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ea815da70 0x7f3ea8155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 -- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ea806ade0 msgr2=0x7f3ea81f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ea806ade0 0x7f3ea81f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 -- 172.21.15.181:0/916972068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98009550 con 0x7f3ea8072760 2023-10-16T11:19:20.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea7fff700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ea806ade0 0x7f3ea81f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:20.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea77fe700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea80ac880 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f3ea806b3a0 tx=0x7f3e98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea4ff9700 1 -- 172.21.15.181:0/916972068 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800b660 con 0x7f3ea8072760 2023-10-16T11:19:20.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea4ff9700 1 -- 172.21.15.181:0/916972068 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9800b7e0 con 0x7f3ea8072760 2023-10-16T11:19:20.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea80ae700 con 0x7f3ea8072760 2023-10-16T11:19:20.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3ea4ff9700 1 -- 172.21.15.181:0/916972068 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800bb90 con 0x7f3ea8072760 2023-10-16T11:19:20.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.445+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea80aecc0 con 0x7f3ea8072760 2023-10-16T11:19:20.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.446+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea8159020 con 0x7f3ea8072760 2023-10-16T11:19:20.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.447+0000 7f3ea4ff9700 1 -- 172.21.15.181:0/916972068 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3e9800bcf0 con 0x7f3ea8072760 2023-10-16T11:19:20.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.447+0000 7f3ea4ff9700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3e9406b020 0x7f3e9406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.447+0000 7f3ea4ff9700 1 -- 172.21.15.181:0/916972068 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f3e9808b040 con 0x7f3ea8072760 2023-10-16T11:19:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.447+0000 7f3ea7fff700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3e9406b020 0x7f3e9406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.448+0000 7f3ea7fff700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3e9406b020 0x7f3e9406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c000f60 tx=0x7f3e9c007df0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.451+0000 7f3ea4ff9700 1 -- 172.21.15.181:0/916972068 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3e98056190 con 0x7f3ea8072760 2023-10-16T11:19:20.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.646+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ea806f6c0 con 0x7f3ea8072760 2023-10-16T11:19:20.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.647+0000 7f3ea4ff9700 1 -- 172.21.15.181:0/916972068 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v212) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e980597b0 con 0x7f3ea8072760 2023-10-16T11:19:20.649 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:19:20.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3e9406b020 msgr2=0x7f3e9406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3e9406b020 0x7f3e9406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c000f60 tx=0x7f3e9c007df0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 msgr2=0x7f3ea80ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea80ac880 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f3ea806b3a0 tx=0x7f3e98004750 comp rx=0 tx=0).stop 2023-10-16T11:19:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 shutdown_connections 2023-10-16T11:19:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3e9406b020 0x7f3e9406d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ea815da70 0x7f3ea8155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ea806ade0 0x7f3ea81f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 --2- 172.21.15.181:0/916972068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea8072760 0x7f3ea80ac880 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 >> 172.21.15.181:0/916972068 conn(0x7f3ea80cd010 msgr2=0x7f3ea80d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 shutdown_connections 2023-10-16T11:19:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.649+0000 7f3eae39c700 1 -- 172.21.15.181:0/916972068 wait complete. 2023-10-16T11:19:20.663 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:19:20.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.782+0000 7fd6ec85e700 1 Processor -- start 2023-10-16T11:19:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.782+0000 7fd6ec85e700 1 -- start start 2023-10-16T11:19:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.782+0000 7fd6ec85e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6e80af450 0x7fd6e80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6ec85e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6ec85e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e806f870 con 0x7fd6e815dfd0 2023-10-16T11:19:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6ec85e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e80d1160 con 0x7fd6e80af450 2023-10-16T11:19:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6ec85e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80cc8e0 con 0x7fd6e80cd470 2023-10-16T11:19:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6e80af450 0x7fd6e80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6e80af450 0x7fd6e80af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55322/0 (socket says 172.21.15.181:55322) 2023-10-16T11:19:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 -- 172.21.15.181:0/3431052788 learned_addr learned my addr 172.21.15.181:0/3431052788 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:20.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e6d9d700 1 --2- 172.21.15.181:0/3431052788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e80cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 -- 172.21.15.181:0/3431052788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 msgr2=0x7fd6e80cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:20.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 --2- 172.21.15.181:0/3431052788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e80cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 -- 172.21.15.181:0/3431052788 >> v1:172.21.15.190:6789/0 conn(0x7fd6e815dfd0 legacy=0x7fd6e806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:20.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 -- 172.21.15.181:0/3431052788 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80ccbc0 con 0x7fd6e80af450 2023-10-16T11:19:20.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e6d9d700 1 --2- 172.21.15.181:0/3431052788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:20.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e5d9b700 1 --2- 172.21.15.181:0/3431052788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e80af9a0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80098f0 tx=0x7fd6d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594aaa5604c0a10c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e559a700 1 -- 172.21.15.181:0/3431052788 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8004230 con 0x7fd6e80af450 2023-10-16T11:19:20.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e559a700 1 -- 172.21.15.181:0/3431052788 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d8004390 con 0x7fd6e80af450 2023-10-16T11:19:20.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.783+0000 7fd6e559a700 1 -- 172.21.15.181:0/3431052788 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80036a0 con 0x7fd6e80af450 2023-10-16T11:19:20.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 -- 172.21.15.181:0/3431052788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 msgr2=0x7fd6e80af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/3431052788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e80af9a0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80098f0 tx=0x7fd6d8009c00 comp rx=0 tx=0).stop 2023-10-16T11:19:20.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 -- 172.21.15.181:0/3431052788 shutdown_connections 2023-10-16T11:19:20.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/3431052788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/3431052788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e80af9a0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 -- 172.21.15.181:0/3431052788 >> 172.21.15.181:0/3431052788 conn(0x7fd6e80d04a0 msgr2=0x7fd6e80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 -- 172.21.15.181:0/3431052788 shutdown_connections 2023-10-16T11:19:20.792 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.784+0000 7fd6ec85e700 1 -- 172.21.15.181:0/3431052788 wait complete. 2023-10-16T11:19:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.785+0000 7fd6ec85e700 1 Processor -- start 2023-10-16T11:19:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.785+0000 7fd6ec85e700 1 -- start start 2023-10-16T11:19:20.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6ec85e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e8155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6ec85e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6ec85e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e815dfd0 0x7fd6e80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6ec85e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e81604b0 con 0x7fd6e80af450 2023-10-16T11:19:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6ec85e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8156420 con 0x7fd6e815dfd0 2023-10-16T11:19:20.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6ec85e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e815f830 con 0x7fd6e80cd470 2023-10-16T11:19:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6e659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e8155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6e659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e8155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55326/0 (socket says 172.21.15.181:55326) 2023-10-16T11:19:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6e659c700 1 -- 172.21.15.181:0/23251247 learned_addr learned my addr 172.21.15.181:0/23251247 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6e5d9b700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.786+0000 7fd6e6d9d700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e815dfd0 0x7fd6e80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6e659c700 1 -- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 msgr2=0x7fd6e815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6e659c700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6e659c700 1 -- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e815dfd0 msgr2=0x7fd6e80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6e659c700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e815dfd0 0x7fd6e80b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6e659c700 1 -- 172.21.15.181:0/23251247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8009550 con 0x7fd6e80af450 2023-10-16T11:19:20.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6e659c700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e8155ee0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00b450 tx=0x7fd6dc00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6d6ffd700 1 -- 172.21.15.181:0/23251247 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00be60 con 0x7fd6e80af450 2023-10-16T11:19:20.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6d6ffd700 1 -- 172.21.15.181:0/23251247 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc004020 con 0x7fd6e80af450 2023-10-16T11:19:20.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e815fb10 con 0x7fd6e80af450 2023-10-16T11:19:20.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6d6ffd700 1 -- 172.21.15.181:0/23251247 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0043d0 con 0x7fd6e80af450 2023-10-16T11:19:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.787+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e81600d0 con 0x7fd6e80af450 2023-10-16T11:19:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.788+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e8067ab0 con 0x7fd6e80af450 2023-10-16T11:19:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.788+0000 7fd6d6ffd700 1 -- 172.21.15.181:0/23251247 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd6dc0036d0 con 0x7fd6e80af450 2023-10-16T11:19:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.789+0000 7fd6d6ffd700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd6c806b020 0x7fd6c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:20.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.789+0000 7fd6d6ffd700 1 -- 172.21.15.181:0/23251247 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5409+0+0 (secure 0 0 0) 0x7fd6dc088d30 con 0x7fd6e80af450 2023-10-16T11:19:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.789+0000 7fd6e5d9b700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd6c806b020 0x7fd6c806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.789+0000 7fd6e5d9b700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd6c806b020 0x7fd6c806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009840 tx=0x7fd6d8008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:20.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.793+0000 7fd6d6ffd700 1 -- 172.21.15.181:0/23251247 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd6dc053df0 con 0x7fd6e80af450 2023-10-16T11:19:20.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.987+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd6e8067d20 con 0x7fd6e80af450 2023-10-16T11:19:20.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.988+0000 7fd6d6ffd700 1 -- 172.21.15.181:0/23251247 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v212) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd6dc015070 con 0x7fd6e80af450 2023-10-16T11:19:20.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:20.990 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:19:20.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:20.991 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:19:20.991 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:19:20.991 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:20.991 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:19:20.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.989+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd6c806b020 msgr2=0x7fd6c806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.989+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd6c806b020 0x7fd6c806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009840 tx=0x7fd6d8008030 comp rx=0 tx=0).stop 2023-10-16T11:19:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.989+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 msgr2=0x7fd6e8155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.989+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e8155ee0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00b450 tx=0x7fd6dc00b760 comp rx=0 tx=0).stop 2023-10-16T11:19:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 shutdown_connections 2023-10-16T11:19:20.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd6c806b020 0x7fd6c806d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e815dfd0 0x7fd6e80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6e80af450 0x7fd6e8155ee0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 --2- 172.21.15.181:0/23251247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80cd470 0x7fd6e815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 >> 172.21.15.181:0/23251247 conn(0x7fd6e80d04a0 msgr2=0x7fd6e80cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:20.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 shutdown_connections 2023-10-16T11:19:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:20.990+0000 7fd6ec85e700 1 -- 172.21.15.181:0/23251247 wait complete. 2023-10-16T11:19:21.004 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:19:21.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:19:21.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.127+0000 7f62ae8e0700 1 Processor -- start 2023-10-16T11:19:21.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.127+0000 7f62ae8e0700 1 -- start start 2023-10-16T11:19:21.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ae8e0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62a80ce610 0x7f62a806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ae8e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ae8e0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62a80d0060 0x7f62a806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ae8e0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a8162190 con 0x7f62a80ce610 2023-10-16T11:19:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ae8e0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a8162650 con 0x7f62a80d0060 2023-10-16T11:19:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ae8e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8162370 con 0x7f62a80d0fc0 2023-10-16T11:19:21.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62a80ce610 0x7f62a806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62a80ce610 0x7f62a806d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55334/0 (socket says 172.21.15.181:55334) 2023-10-16T11:19:21.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 -- 172.21.15.181:0/1239380052 learned_addr learned my addr 172.21.15.181:0/1239380052 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:21.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ace7d700 1 --2- 172.21.15.181:0/1239380052 >> v2:172.21.15.190:3300/0 conn(0x7f62a80d0060 0x7f62a806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a77fe700 1 --2- 172.21.15.181:0/1239380052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 -- 172.21.15.181:0/1239380052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 msgr2=0x7f62a80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 --2- 172.21.15.181:0/1239380052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a80cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 -- 172.21.15.181:0/1239380052 >> v2:172.21.15.190:3300/0 conn(0x7f62a80d0060 msgr2=0x7f62a806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 --2- 172.21.15.181:0/1239380052 >> v2:172.21.15.190:3300/0 conn(0x7f62a80d0060 0x7f62a806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62a7fff700 1 -- 172.21.15.181:0/1239380052 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a806f0f0 con 0x7f62a80ce610 2023-10-16T11:19:21.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.128+0000 7f62ace7d700 1 --2- 172.21.15.181:0/1239380052 >> v2:172.21.15.190:3300/0 conn(0x7f62a80d0060 0x7f62a806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62a7fff700 1 --2- 172.21.15.181:0/1239380052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a80ce610 0x7f62a806d660 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f62a806e220 tx=0x7f629c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dbd72b84aa2e72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62a6ffd700 1 -- 172.21.15.181:0/1239380052 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c005e30 con 0x7f62a80ce610 2023-10-16T11:19:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62a6ffd700 1 -- 172.21.15.181:0/1239380052 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c004030 con 0x7f62a80ce610 2023-10-16T11:19:21.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62a6ffd700 1 -- 172.21.15.181:0/1239380052 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0043e0 con 0x7f62a80ce610 2023-10-16T11:19:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 -- 172.21.15.181:0/1239380052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a80ce610 msgr2=0x7f62a806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/1239380052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a80ce610 0x7f62a806d660 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f62a806e220 tx=0x7f629c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 -- 172.21.15.181:0/1239380052 shutdown_connections 2023-10-16T11:19:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/1239380052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a80ce610 0x7f62a806d660 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/1239380052 >> v2:172.21.15.190:3300/0 conn(0x7f62a80d0060 0x7f62a806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/1239380052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a80cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 -- 172.21.15.181:0/1239380052 >> 172.21.15.181:0/1239380052 conn(0x7f62a80d04a0 msgr2=0x7f62a80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 -- 172.21.15.181:0/1239380052 shutdown_connections 2023-10-16T11:19:21.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.129+0000 7f62ae8e0700 1 -- 172.21.15.181:0/1239380052 wait complete. 2023-10-16T11:19:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.131+0000 7f62ae8e0700 1 Processor -- start 2023-10-16T11:19:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.131+0000 7f62ae8e0700 1 -- start start 2023-10-16T11:19:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ae8e0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a80ce610 0x7f62a81f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ae8e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a809c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ae8e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a809cea0 0x7f62a815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ae8e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a809b1e0 con 0x7f62a809cea0 2023-10-16T11:19:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ae8e0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a809a5d0 con 0x7f62a80ce610 2023-10-16T11:19:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ae8e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a809a8b0 con 0x7f62a80d0fc0 2023-10-16T11:19:21.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ace7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a809cea0 0x7f62a815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ace7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a809cea0 0x7f62a815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55344/0 (socket says 172.21.15.181:55344) 2023-10-16T11:19:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ace7d700 1 -- 172.21.15.181:0/4251856790 learned_addr learned my addr 172.21.15.181:0/4251856790 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62a7fff700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a80ce610 0x7f62a81f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62a77fe700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a809c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ace7d700 1 -- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 msgr2=0x7f62a809c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.132+0000 7f62ace7d700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a809c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62ace7d700 1 -- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a80ce610 msgr2=0x7f62a81f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62ace7d700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a80ce610 0x7f62a81f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62ace7d700 1 -- 172.21.15.181:0/4251856790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c009500 con 0x7f62a809cea0 2023-10-16T11:19:21.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62a7fff700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a80ce610 0x7f62a81f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62ace7d700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a809cea0 0x7f62a815b3c0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007de0 tx=0x7f62a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62a4ff9700 1 -- 172.21.15.181:0/4251856790 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000ddd0 con 0x7f62a809cea0 2023-10-16T11:19:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62a4ff9700 1 -- 172.21.15.181:0/4251856790 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a000f040 con 0x7f62a809cea0 2023-10-16T11:19:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62a4ff9700 1 -- 172.21.15.181:0/4251856790 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0003c60 con 0x7f62a809cea0 2023-10-16T11:19:21.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a809ab90 con 0x7f62a809cea0 2023-10-16T11:19:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.133+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a81f7120 con 0x7f62a809cea0 2023-10-16T11:19:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.134+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a809b390 con 0x7f62a809cea0 2023-10-16T11:19:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.134+0000 7f62a4ff9700 1 -- 172.21.15.181:0/4251856790 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f62a0004030 con 0x7f62a809cea0 2023-10-16T11:19:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.135+0000 7f62a4ff9700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f629406b020 0x7f629406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.135+0000 7f62a7fff700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f629406b020 0x7f629406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.135+0000 7f62a4ff9700 1 -- 172.21.15.181:0/4251856790 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f62a0089850 con 0x7f62a809cea0 2023-10-16T11:19:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.135+0000 7f62a7fff700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f629406b020 0x7f629406d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62a806e220 tx=0x7f629c008070 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.139+0000 7f62a4ff9700 1 -- 172.21.15.181:0/4251856790 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f62a809b390 con 0x7f62a809cea0 2023-10-16T11:19:21.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.333+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f62a809b570 con 0x7f62a809cea0 2023-10-16T11:19:21.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.517+0000 7f62a4ff9700 1 -- 172.21.15.181:0/4251856790 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v213) v1 ==== 138+0+0 (secure 0 0 0) 0x7f62a001a0b0 con 0x7f62a809cea0 2023-10-16T11:19:21.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.519+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f629406b020 msgr2=0x7f629406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.519+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f629406b020 0x7f629406d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62a806e220 tx=0x7f629c008070 comp rx=0 tx=0).stop 2023-10-16T11:19:21.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.519+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a809cea0 msgr2=0x7f62a815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.519+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a809cea0 0x7f62a815b3c0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007de0 tx=0x7f62a000d3b0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.520+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 shutdown_connections 2023-10-16T11:19:21.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.520+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f629406b020 0x7f629406d4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.520+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a809cea0 0x7f62a815b3c0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.520+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a80ce610 0x7f62a81f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.520+0000 7f62ae8e0700 1 --2- 172.21.15.181:0/4251856790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a80d0fc0 0x7f62a809c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.520+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 >> 172.21.15.181:0/4251856790 conn(0x7f62a80d04a0 msgr2=0x7f62a80cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:21.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.521+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 shutdown_connections 2023-10-16T11:19:21.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.521+0000 7f62ae8e0700 1 -- 172.21.15.181:0/4251856790 wait complete. 2023-10-16T11:19:21.525 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T11:19:21.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:19:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.664+0000 7fc5da10e700 1 Processor -- start 2023-10-16T11:19:21.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.664+0000 7fc5da10e700 1 -- start start 2023-10-16T11:19:21.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5da10e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc5d4072760 0x7fc5d406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5da10e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5d406ade0 0x7fc5d41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5da10e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d406b210 con 0x7fc5d415da70 2023-10-16T11:19:21.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5da10e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d40cef00 con 0x7fc5d406ade0 2023-10-16T11:19:21.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5da10e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d40cec20 con 0x7fc5d4072760 2023-10-16T11:19:21.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d3fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5d406ade0 0x7fc5d41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc5d415da70 0x7fc5d40ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38194/0 (socket says 172.21.15.181:38194) 2023-10-16T11:19:21.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d37fe700 1 -- 172.21.15.181:0/2844656999 learned_addr learned my addr 172.21.15.181:0/2844656999 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:21.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d2ffd700 1 --2- 172.21.15.181:0/2844656999 >> v2:172.21.15.190:3300/0 conn(0x7fc5d4072760 0x7fc5d406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d3fff700 1 -- 172.21.15.181:0/2844656999 >> v2:172.21.15.190:3300/0 conn(0x7fc5d4072760 msgr2=0x7fc5d406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:21.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d3fff700 1 --2- 172.21.15.181:0/2844656999 >> v2:172.21.15.190:3300/0 conn(0x7fc5d4072760 0x7fc5d406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d3fff700 1 -- 172.21.15.181:0/2844656999 >> v1:172.21.15.181:6789/0 conn(0x7fc5d415da70 legacy=0x7fc5d40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d3fff700 1 -- 172.21.15.181:0/2844656999 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4071c80 con 0x7fc5d406ade0 2023-10-16T11:19:21.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d27fc700 1 -- 172.21.15.181:0/2844656999 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248087412 0 0) 0x7fc5d406b210 con 0x7fc5d415da70 2023-10-16T11:19:21.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.665+0000 7fc5d3fff700 1 --2- 172.21.15.181:0/2844656999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d41615f0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fc5d406ef60 tx=0x7fc5cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a220b4c399f6fe81 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5d27fc700 1 -- 172.21.15.181:0/2844656999 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00e070 con 0x7fc5d406ade0 2023-10-16T11:19:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5d27fc700 1 -- 172.21.15.181:0/2844656999 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc00bbd0 con 0x7fc5d406ade0 2023-10-16T11:19:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5d27fc700 1 -- 172.21.15.181:0/2844656999 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc012430 con 0x7fc5d406ade0 2023-10-16T11:19:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 -- 172.21.15.181:0/2844656999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 msgr2=0x7fc5d41615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 --2- 172.21.15.181:0/2844656999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d41615f0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fc5d406ef60 tx=0x7fc5cc00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 -- 172.21.15.181:0/2844656999 shutdown_connections 2023-10-16T11:19:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 --2- 172.21.15.181:0/2844656999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d41615f0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 --2- 172.21.15.181:0/2844656999 >> v2:172.21.15.190:3300/0 conn(0x7fc5d4072760 0x7fc5d406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 -- 172.21.15.181:0/2844656999 >> 172.21.15.181:0/2844656999 conn(0x7fc5d40cd010 msgr2=0x7fc5d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 -- 172.21.15.181:0/2844656999 shutdown_connections 2023-10-16T11:19:21.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.666+0000 7fc5da10e700 1 -- 172.21.15.181:0/2844656999 wait complete. 2023-10-16T11:19:21.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.667+0000 7fc5da10e700 1 Processor -- start 2023-10-16T11:19:21.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5da10e700 1 -- start start 2023-10-16T11:19:21.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5da10e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d4159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5da10e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d4072760 0x7fc5d40add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5da10e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d415da70 0x7fc5d415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5da10e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d415a420 con 0x7fc5d406ade0 2023-10-16T11:19:21.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5da10e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40ae770 con 0x7fc5d415da70 2023-10-16T11:19:21.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5da10e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40aea50 con 0x7fc5d4072760 2023-10-16T11:19:21.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d4159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d4159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55350/0 (socket says 172.21.15.181:55350) 2023-10-16T11:19:21.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.668+0000 7fc5d37fe700 1 -- 172.21.15.181:0/1567919430 learned_addr learned my addr 172.21.15.181:0/1567919430 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:21.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d3fff700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d415da70 0x7fc5d415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d2ffd700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d4072760 0x7fc5d40add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d37fe700 1 -- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d4072760 msgr2=0x7fc5d40add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d37fe700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d4072760 0x7fc5d40add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d37fe700 1 -- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d415da70 msgr2=0x7fc5d415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d37fe700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d415da70 0x7fc5d415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:21.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d37fe700 1 -- 172.21.15.181:0/1567919430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc009a70 con 0x7fc5d406ade0 2023-10-16T11:19:21.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d3fff700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d415da70 0x7fc5d415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:21.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5d37fe700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d4159ee0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80011e0 tx=0x7fc5c800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:21.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5bbfff700 1 -- 172.21.15.181:0/1567919430 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004c80 con 0x7fc5d406ade0 2023-10-16T11:19:21.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5bbfff700 1 -- 172.21.15.181:0/1567919430 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8004e00 con 0x7fc5d406ade0 2023-10-16T11:19:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d40aed30 con 0x7fc5d406ade0 2023-10-16T11:19:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5bbfff700 1 -- 172.21.15.181:0/1567919430 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8007310 con 0x7fc5d406ade0 2023-10-16T11:19:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.669+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d40af2f0 con 0x7fc5d406ade0 2023-10-16T11:19:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.670+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d406f6c0 con 0x7fc5d406ade0 2023-10-16T11:19:21.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.671+0000 7fc5bbfff700 1 -- 172.21.15.181:0/1567919430 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc5c8007470 con 0x7fc5d406ade0 2023-10-16T11:19:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.671+0000 7fc5bbfff700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc5c006b020 0x7fc5c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.672+0000 7fc5d2ffd700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc5c006b020 0x7fc5c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.672+0000 7fc5bbfff700 1 -- 172.21.15.181:0/1567919430 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 5779+0+0 (secure 0 0 0) 0x7fc5c8089c10 con 0x7fc5d406ade0 2023-10-16T11:19:21.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.672+0000 7fc5d2ffd700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc5c006b020 0x7fc5c006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009930 tx=0x7fc5c4008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:21.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.676+0000 7fc5bbfff700 1 -- 172.21.15.181:0/1567919430 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc5c8054bc0 con 0x7fc5d406ade0 2023-10-16T11:19:21.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:21.866+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc5d4072210 con 0x7fc5d406ade0 2023-10-16T11:19:22.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.519+0000 7fc5bbfff700 1 -- 172.21.15.181:0/1567919430 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v214) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc5c80581e0 con 0x7fc5d406ade0 2023-10-16T11:19:22.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc5c006b020 msgr2=0x7fc5c006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:22.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc5c006b020 0x7fc5c006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009930 tx=0x7fc5c4008040 comp rx=0 tx=0).stop 2023-10-16T11:19:22.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 msgr2=0x7fc5d4159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:22.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d4159ee0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80011e0 tx=0x7fc5c800cae0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 shutdown_connections 2023-10-16T11:19:22.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc5c006b020 0x7fc5c006d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5d415da70 0x7fc5d415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d406ade0 0x7fc5d4159ee0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 --2- 172.21.15.181:0/1567919430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5d4072760 0x7fc5d40add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 >> 172.21.15.181:0/1567919430 conn(0x7fc5d40cd010 msgr2=0x7fc5d40b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:22.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 shutdown_connections 2023-10-16T11:19:22.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.522+0000 7fc5da10e700 1 -- 172.21.15.181:0/1567919430 wait complete. 2023-10-16T11:19:22.527 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T11:19:22.549 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:19:22.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.672+0000 7feb8378e700 1 Processor -- start 2023-10-16T11:19:22.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.673+0000 7feb8378e700 1 -- start start 2023-10-16T11:19:22.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.673+0000 7feb8378e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb7c072760 0x7feb7c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:22.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.673+0000 7feb8378e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c06ade0 0x7feb7c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.673+0000 7feb8378e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c06b210 con 0x7feb7c15da70 2023-10-16T11:19:22.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.673+0000 7feb8378e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7c0cef00 con 0x7feb7c072760 2023-10-16T11:19:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.673+0000 7feb8378e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c0cec20 con 0x7feb7c06ade0 2023-10-16T11:19:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb81d2b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c06ade0 0x7feb7c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:22.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb80d29700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb7c072760 0x7feb7c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:22.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb8152a700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7feb7c15da70 0x7feb7c0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:50378/0 (socket says 172.21.15.181:50378) 2023-10-16T11:19:22.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb80d29700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feb7c072760 0x7feb7c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:60410/0 (socket says 172.21.15.181:60410) 2023-10-16T11:19:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb81d2b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c06ade0 0x7feb7c1615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:52388/0 (socket says 172.21.15.181:52388) 2023-10-16T11:19:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb8152a700 1 -- 172.21.15.181:0/1278355481 learned_addr learned my addr 172.21.15.181:0/1278355481 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:22.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb81d2b700 1 -- 172.21.15.181:0/1278355481 >> v2:172.21.15.190:3300/0 conn(0x7feb7c072760 msgr2=0x7feb7c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:22.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb81d2b700 1 --2- 172.21.15.181:0/1278355481 >> v2:172.21.15.190:3300/0 conn(0x7feb7c072760 0x7feb7c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb81d2b700 1 -- 172.21.15.181:0/1278355481 >> v1:172.21.15.190:6789/0 conn(0x7feb7c15da70 legacy=0x7feb7c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:22.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb73fff700 1 -- 172.21.15.181:0/1278355481 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683183691 0 0) 0x7feb7c06b210 con 0x7feb7c15da70 2023-10-16T11:19:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb81d2b700 1 -- 172.21.15.181:0/1278355481 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c071c80 con 0x7feb7c06ade0 2023-10-16T11:19:22.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb80d29700 1 --2- 172.21.15.181:0/1278355481 >> v2:172.21.15.190:3300/0 conn(0x7feb7c072760 0x7feb7c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb81d2b700 1 --2- 172.21.15.181:0/1278355481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c06ade0 0x7feb7c1615f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feb74009d90 tx=0x7feb7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406ec69c4fa885c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb73fff700 1 -- 172.21.15.181:0/1278355481 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400e070 con 0x7feb7c06ade0 2023-10-16T11:19:22.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb73fff700 1 -- 172.21.15.181:0/1278355481 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7400bb20 con 0x7feb7c06ade0 2023-10-16T11:19:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.674+0000 7feb73fff700 1 -- 172.21.15.181:0/1278355481 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74012430 con 0x7feb7c06ade0 2023-10-16T11:19:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 -- 172.21.15.181:0/1278355481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c06ade0 msgr2=0x7feb7c1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:22.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 --2- 172.21.15.181:0/1278355481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c06ade0 0x7feb7c1615f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feb74009d90 tx=0x7feb7400b330 comp rx=0 tx=0).stop 2023-10-16T11:19:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 -- 172.21.15.181:0/1278355481 shutdown_connections 2023-10-16T11:19:22.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 --2- 172.21.15.181:0/1278355481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c06ade0 0x7feb7c1615f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 --2- 172.21.15.181:0/1278355481 >> v2:172.21.15.190:3300/0 conn(0x7feb7c072760 0x7feb7c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 -- 172.21.15.181:0/1278355481 >> 172.21.15.181:0/1278355481 conn(0x7feb7c0cd010 msgr2=0x7feb7c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:22.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 -- 172.21.15.181:0/1278355481 shutdown_connections 2023-10-16T11:19:22.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.675+0000 7feb8378e700 1 -- 172.21.15.181:0/1278355481 wait complete. 2023-10-16T11:19:22.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.676+0000 7feb8378e700 1 Processor -- start 2023-10-16T11:19:22.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.676+0000 7feb8378e700 1 -- start start 2023-10-16T11:19:22.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8378e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c06ade0 0x7feb7c1f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:22.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8378e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb7c072760 0x7feb7c150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:22.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8378e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c15da70 0x7feb7c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:22.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8378e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c1f4400 con 0x7feb7c06ade0 2023-10-16T11:19:22.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8378e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c1f3e40 con 0x7feb7c072760 2023-10-16T11:19:22.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8378e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c1f4120 con 0x7feb7c15da70 2023-10-16T11:19:22.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8152a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c06ade0 0x7feb7c1f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:22.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8152a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c06ade0 0x7feb7c1f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55364/0 (socket says 172.21.15.181:55364) 2023-10-16T11:19:22.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb8152a700 1 -- 172.21.15.181:0/1121667586 learned_addr learned my addr 172.21.15.181:0/1121667586 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:22.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb81d2b700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c15da70 0x7feb7c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:22.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.677+0000 7feb80d29700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb7c072760 0x7feb7c150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:22.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8152a700 1 -- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c15da70 msgr2=0x7feb7c0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:22.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8152a700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c15da70 0x7feb7c0c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8152a700 1 -- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb7c072760 msgr2=0x7feb7c150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:22.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8152a700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb7c072760 0x7feb7c150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:22.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8152a700 1 -- 172.21.15.181:0/1121667586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb780094a0 con 0x7feb7c06ade0 2023-10-16T11:19:22.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb80d29700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb7c072760 0x7feb7c150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:22.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8152a700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c06ade0 0x7feb7c1f3460 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00c550 tx=0x7feb6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:22.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb71ffb700 1 -- 172.21.15.181:0/1121667586 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004c80 con 0x7feb7c06ade0 2023-10-16T11:19:22.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb71ffb700 1 -- 172.21.15.181:0/1121667586 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c004e00 con 0x7feb7c06ade0 2023-10-16T11:19:22.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb74009a70 con 0x7feb7c06ade0 2023-10-16T11:19:22.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb71ffb700 1 -- 172.21.15.181:0/1121667586 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c007310 con 0x7feb7c06ade0 2023-10-16T11:19:22.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.678+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c0c44d0 con 0x7feb7c06ade0 2023-10-16T11:19:22.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.679+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c0d0070 con 0x7feb7c06ade0 2023-10-16T11:19:22.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.679+0000 7feb71ffb700 1 -- 172.21.15.181:0/1121667586 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb6c007470 con 0x7feb7c06ade0 2023-10-16T11:19:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.680+0000 7feb71ffb700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7feb6006b020 0x7feb6006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.680+0000 7feb71ffb700 1 -- 172.21.15.181:0/1121667586 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6161+0+0 (secure 0 0 0) 0x7feb6c089620 con 0x7feb7c06ade0 2023-10-16T11:19:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.680+0000 7feb80d29700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7feb6006b020 0x7feb6006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:22.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.680+0000 7feb80d29700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7feb6006b020 0x7feb6006d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb78009dc0 tx=0x7feb78008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:22.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.685+0000 7feb71ffb700 1 -- 172.21.15.181:0/1121667586 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7feb6c054400 con 0x7feb7c06ade0 2023-10-16T11:19:22.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:22.923+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7feb7c06f6c0 con 0x7feb7c06ade0 2023-10-16T11:19:23.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.547+0000 7feb71ffb700 1 -- 172.21.15.181:0/1121667586 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v107) v1 ==== 174+0+0 (secure 0 0 0) 0x7feb6c057a20 con 0x7feb7c06ade0 2023-10-16T11:19:23.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.555+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7feb6006b020 msgr2=0x7feb6006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:23.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.555+0000 7feb8378e700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7feb6006b020 0x7feb6006d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb78009dc0 tx=0x7feb78008040 comp rx=0 tx=0).stop 2023-10-16T11:19:23.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.555+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c06ade0 msgr2=0x7feb7c1f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:23.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.555+0000 7feb8378e700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c06ade0 0x7feb7c1f3460 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00c550 tx=0x7feb6c00c910 comp rx=0 tx=0).stop 2023-10-16T11:19:23.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.556+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 shutdown_connections 2023-10-16T11:19:23.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.556+0000 7feb8378e700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7feb6006b020 0x7feb6006d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.556+0000 7feb8378e700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feb7c15da70 0x7feb7c0c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.556+0000 7feb8378e700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c06ade0 0x7feb7c1f3460 secure :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00c550 tx=0x7feb6c00c910 comp rx=0 tx=0).stop 2023-10-16T11:19:23.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.556+0000 7feb8378e700 1 --2- 172.21.15.181:0/1121667586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb7c072760 0x7feb7c150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.556+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 >> 172.21.15.181:0/1121667586 conn(0x7feb7c0cd010 msgr2=0x7feb7c0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:23.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.561+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 shutdown_connections 2023-10-16T11:19:23.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.561+0000 7feb8378e700 1 -- 172.21.15.181:0/1121667586 wait complete. 2023-10-16T11:19:23.565 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 26 and data pool 27 2023-10-16T11:19:23.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.701+0000 7f9d5c4bd700 1 Processor -- start 2023-10-16T11:19:23.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.701+0000 7f9d5c4bd700 1 -- start start 2023-10-16T11:19:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5c4bd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d540665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:23.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5c4bd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d540ca2e0 0x7f9d540ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5c4bd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d54066c60 con 0x7f9d540cedd0 2023-10-16T11:19:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5c4bd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540cda10 con 0x7f9d540ca2e0 2023-10-16T11:19:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5c4bd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5406c1a0 con 0x7f9d5415db70 2023-10-16T11:19:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d540ca2e0 0x7f9d540ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:23.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d540ca2e0 0x7f9d540ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55374/0 (socket says 172.21.15.181:55374) 2023-10-16T11:19:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 -- 172.21.15.181:0/3465648309 learned_addr learned my addr 172.21.15.181:0/3465648309 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5a259700 1 --2- 172.21.15.181:0/3465648309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d540665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 -- 172.21.15.181:0/3465648309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 msgr2=0x7f9d540665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:23.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 --2- 172.21.15.181:0/3465648309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d540665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 -- 172.21.15.181:0/3465648309 >> v1:172.21.15.190:6789/0 conn(0x7f9d540cedd0 legacy=0x7f9d540cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 -- 172.21.15.181:0/3465648309 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5406b3e0 con 0x7f9d540ca2e0 2023-10-16T11:19:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5a259700 1 --2- 172.21.15.181:0/3465648309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d540665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d5aa5a700 1 --2- 172.21.15.181:0/3465648309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d540ca770 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009e50 tx=0x7f9d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946243a7ad08a8aa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:23.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d59257700 1 -- 172.21.15.181:0/3465648309 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00e070 con 0x7f9d540ca2e0 2023-10-16T11:19:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.702+0000 7f9d59257700 1 -- 172.21.15.181:0/3465648309 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c003a70 con 0x7f9d540ca2e0 2023-10-16T11:19:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d59257700 1 -- 172.21.15.181:0/3465648309 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c003e20 con 0x7f9d540ca2e0 2023-10-16T11:19:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/3465648309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 msgr2=0x7f9d540ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:23.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/3465648309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d540ca770 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009e50 tx=0x7f9d4c00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/3465648309 shutdown_connections 2023-10-16T11:19:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/3465648309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d540ca770 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/3465648309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d540665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/3465648309 >> 172.21.15.181:0/3465648309 conn(0x7f9d540b1780 msgr2=0x7f9d540b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:23.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/3465648309 shutdown_connections 2023-10-16T11:19:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.703+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/3465648309 wait complete. 2023-10-16T11:19:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.704+0000 7f9d5c4bd700 1 Processor -- start 2023-10-16T11:19:23.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.704+0000 7f9d5c4bd700 1 -- start start 2023-10-16T11:19:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5c4bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d5409f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5c4bd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540cedd0 0x7f9d540c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5c4bd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d5414cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:23.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5c4bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5414e350 con 0x7f9d540ca2e0 2023-10-16T11:19:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5c4bd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540c29f0 con 0x7f9d540cedd0 2023-10-16T11:19:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5c4bd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5414d4a0 con 0x7f9d5415db70 2023-10-16T11:19:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d5409f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d5409f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55378/0 (socket says 172.21.15.181:55378) 2023-10-16T11:19:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 -- 172.21.15.181:0/1723435607 learned_addr learned my addr 172.21.15.181:0/1723435607 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5aa5a700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d5414cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d59a58700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540cedd0 0x7f9d540c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 -- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 msgr2=0x7f9d5414cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d5414cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 -- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540cedd0 msgr2=0x7f9d540c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540cedd0 0x7f9d540c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d5a259700 1 -- 172.21.15.181:0/1723435607 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009b30 con 0x7f9d540ca2e0 2023-10-16T11:19:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.705+0000 7f9d59a58700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540cedd0 0x7f9d540c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.706+0000 7f9d5a259700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d5409f9c0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c7f0 tx=0x7f9d5000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.706+0000 7f9d4affd700 1 -- 172.21.15.181:0/1723435607 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50011800 con 0x7f9d540ca2e0 2023-10-16T11:19:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.706+0000 7f9d4affd700 1 -- 172.21.15.181:0/1723435607 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50011960 con 0x7f9d540ca2e0 2023-10-16T11:19:23.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.706+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5414d740 con 0x7f9d540ca2e0 2023-10-16T11:19:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.706+0000 7f9d4affd700 1 -- 172.21.15.181:0/1723435607 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50011cf0 con 0x7f9d540ca2e0 2023-10-16T11:19:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.706+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5414dd00 con 0x7f9d540ca2e0 2023-10-16T11:19:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.707+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d54067d20 con 0x7f9d540ca2e0 2023-10-16T11:19:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.707+0000 7f9d4affd700 1 -- 172.21.15.181:0/1723435607 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9d50018360 con 0x7f9d540ca2e0 2023-10-16T11:19:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.708+0000 7f9d4affd700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d3806b020 0x7f9d3806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.708+0000 7f9d59a58700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d3806b020 0x7f9d3806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.708+0000 7f9d4affd700 1 -- 172.21.15.181:0/1723435607 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9d50089ab0 con 0x7f9d540ca2e0 2023-10-16T11:19:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.708+0000 7f9d59a58700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d3806b020 0x7f9d3806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d5414f770 tx=0x7f9d44009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.712+0000 7f9d4affd700 1 -- 172.21.15.181:0/1723435607 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9d50054690 con 0x7f9d540ca2e0 2023-10-16T11:19:23.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.908+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d54072510 con 0x7f9d540ca2e0 2023-10-16T11:19:23.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.909+0000 7f9d4affd700 1 -- 172.21.15.181:0/1723435607 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+17483 (secure 0 0 0) 0x7f9d50057cb0 con 0x7f9d540ca2e0 2023-10-16T11:19:23.911 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:23.912 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":215,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:22.925646+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:23.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d3806b020 msgr2=0x7f9d3806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:23.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d3806b020 0x7f9d3806d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d5414f770 tx=0x7f9d44009040 comp rx=0 tx=0).stop 2023-10-16T11:19:23.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 msgr2=0x7f9d5409f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:23.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d5409f9c0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c7f0 tx=0x7f9d5000cb00 comp rx=0 tx=0).stop 2023-10-16T11:19:23.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 shutdown_connections 2023-10-16T11:19:23.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d3806b020 0x7f9d3806d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d5415db70 0x7f9d5414cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d540ca2e0 0x7f9d5409f9c0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 --2- 172.21.15.181:0/1723435607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d540cedd0 0x7f9d540c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:23.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 >> 172.21.15.181:0/1723435607 conn(0x7f9d540b1780 msgr2=0x7f9d540cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:23.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 shutdown_connections 2023-10-16T11:19:23.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:23.911+0000 7f9d5c4bd700 1 -- 172.21.15.181:0/1723435607 wait complete. 2023-10-16T11:19:23.926 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:24.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.043+0000 7f80ee3ee700 1 Processor -- start 2023-10-16T11:19:24.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.043+0000 7f80ee3ee700 1 -- start start 2023-10-16T11:19:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ee3ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e815dfd0 0x7f80e80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ee3ee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80e80cd470 0x7f80e80cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ee3ee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e806f870 con 0x7f80e80af450 2023-10-16T11:19:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ee3ee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e80cde40 con 0x7f80e80cd470 2023-10-16T11:19:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ee3ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e80d1160 con 0x7f80e815dfd0 2023-10-16T11:19:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ec98b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80e80cd470 0x7f80e80cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80e77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f80e80af450 0x7f80e806d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38230/0 (socket says 172.21.15.181:38230) 2023-10-16T11:19:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ec98b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80e80cd470 0x7f80e80cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55388/0 (socket says 172.21.15.181:55388) 2023-10-16T11:19:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80e77fe700 1 -- 172.21.15.181:0/2838666416 learned_addr learned my addr 172.21.15.181:0/2838666416 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80e7fff700 1 --2- 172.21.15.181:0/2838666416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e815dfd0 0x7f80e80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ec98b700 1 -- 172.21.15.181:0/2838666416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e815dfd0 msgr2=0x7f80e80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ec98b700 1 --2- 172.21.15.181:0/2838666416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e815dfd0 0x7f80e80af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ec98b700 1 -- 172.21.15.181:0/2838666416 >> v1:172.21.15.181:6789/0 conn(0x7f80e80af450 legacy=0x7f80e806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80ec98b700 1 -- 172.21.15.181:0/2838666416 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e80cc8e0 con 0x7f80e80cd470 2023-10-16T11:19:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.044+0000 7f80e6ffd700 1 -- 172.21.15.181:0/2838666416 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986671905 0 0) 0x7f80e806f870 con 0x7f80e80af450 2023-10-16T11:19:24.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ec98b700 1 --2- 172.21.15.181:0/2838666416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80cd900 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009d90 tx=0x7f80e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06802bb5779d0e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:24.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80e6ffd700 1 -- 172.21.15.181:0/2838666416 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e000e070 con 0x7f80e80cd470 2023-10-16T11:19:24.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80e6ffd700 1 -- 172.21.15.181:0/2838666416 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e000bb20 con 0x7f80e80cd470 2023-10-16T11:19:24.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80e6ffd700 1 -- 172.21.15.181:0/2838666416 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0012430 con 0x7f80e80cd470 2023-10-16T11:19:24.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 -- 172.21.15.181:0/2838666416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 msgr2=0x7f80e80cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/2838666416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80cd900 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009d90 tx=0x7f80e000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:24.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 -- 172.21.15.181:0/2838666416 shutdown_connections 2023-10-16T11:19:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/2838666416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80cd900 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/2838666416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e815dfd0 0x7f80e80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 -- 172.21.15.181:0/2838666416 >> 172.21.15.181:0/2838666416 conn(0x7f80e80d04a0 msgr2=0x7f80e80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 -- 172.21.15.181:0/2838666416 shutdown_connections 2023-10-16T11:19:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.045+0000 7f80ee3ee700 1 -- 172.21.15.181:0/2838666416 wait complete. 2023-10-16T11:19:24.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 Processor -- start 2023-10-16T11:19:24.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 -- start start 2023-10-16T11:19:24.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e80af450 0x7f80e8159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e815dfd0 0x7f80e80b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e8159ac0 con 0x7f80e80cd470 2023-10-16T11:19:24.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e81f6780 con 0x7f80e815dfd0 2023-10-16T11:19:24.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.047+0000 7f80ee3ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e81f6a60 con 0x7f80e80af450 2023-10-16T11:19:24.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55390/0 (socket says 172.21.15.181:55390) 2023-10-16T11:19:24.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 -- 172.21.15.181:0/3337708601 learned_addr learned my addr 172.21.15.181:0/3337708601 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:24.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e7fff700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e80af450 0x7f80e8159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80ec98b700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e815dfd0 0x7f80e80b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 -- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e80af450 msgr2=0x7f80e8159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e80af450 0x7f80e8159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 -- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e815dfd0 msgr2=0x7f80e80b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e815dfd0 0x7f80e80b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 -- 172.21.15.181:0/3337708601 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0009a70 con 0x7f80e80cd470 2023-10-16T11:19:24.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e77fe700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80b54c0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c550 tx=0x7f80d800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:24.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e4ff9700 1 -- 172.21.15.181:0/3337708601 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8004c80 con 0x7f80e80cd470 2023-10-16T11:19:24.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e4ff9700 1 -- 172.21.15.181:0/3337708601 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8004e00 con 0x7f80e80cd470 2023-10-16T11:19:24.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e81f6d40 con 0x7f80e80cd470 2023-10-16T11:19:24.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80e4ff9700 1 -- 172.21.15.181:0/3337708601 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8007310 con 0x7f80e80cd470 2023-10-16T11:19:24.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.048+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e81f7300 con 0x7f80e80cd470 2023-10-16T11:19:24.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.049+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e81f7520 con 0x7f80e80cd470 2023-10-16T11:19:24.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.050+0000 7f80e4ff9700 1 -- 172.21.15.181:0/3337708601 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f80d8007470 con 0x7f80e80cd470 2023-10-16T11:19:24.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.050+0000 7f80e4ff9700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d406b020 0x7f80d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.050+0000 7f80e4ff9700 1 -- 172.21.15.181:0/3337708601 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f80d8089540 con 0x7f80e80cd470 2023-10-16T11:19:24.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.050+0000 7f80e7fff700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d406b020 0x7f80d406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.051+0000 7f80e7fff700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d406b020 0x7f80d406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009bc0 tx=0x7f80dc0091c0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:24.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.055+0000 7f80e4ff9700 1 -- 172.21.15.181:0/3337708601 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f80d80542a0 con 0x7f80e80cd470 2023-10-16T11:19:24.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.249+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80e8067d20 con 0x7f80e80cd470 2023-10-16T11:19:24.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.250+0000 7f80e4ff9700 1 -- 172.21.15.181:0/3337708601 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+17483 (secure 0 0 0) 0x7f80d80578c0 con 0x7f80e80cd470 2023-10-16T11:19:24.253 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:24.253 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":215,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:22.925646+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:24.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d406b020 msgr2=0x7f80d406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d406b020 0x7f80d406d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009bc0 tx=0x7f80dc0091c0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 msgr2=0x7f80e80b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80b54c0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c550 tx=0x7f80d800c860 comp rx=0 tx=0).stop 2023-10-16T11:19:24.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 shutdown_connections 2023-10-16T11:19:24.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d406b020 0x7f80d406d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e80af450 0x7f80e8159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e815dfd0 0x7f80e80b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 --2- 172.21.15.181:0/3337708601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80e80cd470 0x7f80e80b54c0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 >> 172.21.15.181:0/3337708601 conn(0x7f80e80d04a0 msgr2=0x7f80e8073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:24.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 shutdown_connections 2023-10-16T11:19:24.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.253+0000 7f80ee3ee700 1 -- 172.21.15.181:0/3337708601 wait complete. 2023-10-16T11:19:24.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:19:24.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.388+0000 7f0e389e4700 1 Processor -- start 2023-10-16T11:19:24.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.388+0000 7f0e389e4700 1 -- start start 2023-10-16T11:19:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e389e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e34072760 0x7f0e3406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e389e4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e3406ade0 0x7f0e341615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e389e4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3406b210 con 0x7f0e3415da70 2023-10-16T11:19:24.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e389e4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e340cef00 con 0x7f0e34072760 2023-10-16T11:19:24.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e389e4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e340cec20 con 0x7f0e3406ade0 2023-10-16T11:19:24.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e3259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0e3415da70 0x7f0e340ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38244/0 (socket says 172.21.15.181:38244) 2023-10-16T11:19:24.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e31d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e34072760 0x7f0e3406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.389+0000 7f0e3259c700 1 -- 172.21.15.181:0/3443762476 learned_addr learned my addr 172.21.15.181:0/3443762476 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e32d9d700 1 --2- 172.21.15.181:0/3443762476 >> v2:172.21.15.190:3300/0 conn(0x7f0e3406ade0 0x7f0e341615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e31d9b700 1 -- 172.21.15.181:0/3443762476 >> v2:172.21.15.190:3300/0 conn(0x7f0e3406ade0 msgr2=0x7f0e341615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e31d9b700 1 --2- 172.21.15.181:0/3443762476 >> v2:172.21.15.190:3300/0 conn(0x7f0e3406ade0 0x7f0e341615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e31d9b700 1 -- 172.21.15.181:0/3443762476 >> v1:172.21.15.181:6789/0 conn(0x7f0e3415da70 legacy=0x7f0e340ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e31d9b700 1 -- 172.21.15.181:0/3443762476 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34071c80 con 0x7f0e34072760 2023-10-16T11:19:24.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e31d9b700 1 --2- 172.21.15.181:0/3443762476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e34072760 0x7f0e3406a8a0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f0e340b66b0 tx=0x7f0e28009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6559e60dccb7189b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:24.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e3159a700 1 -- 172.21.15.181:0/3443762476 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28011830 con 0x7f0e34072760 2023-10-16T11:19:24.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e3159a700 1 -- 172.21.15.181:0/3443762476 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e28011990 con 0x7f0e34072760 2023-10-16T11:19:24.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e3159a700 1 -- 172.21.15.181:0/3443762476 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28011d20 con 0x7f0e34072760 2023-10-16T11:19:24.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e389e4700 1 -- 172.21.15.181:0/3443762476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e34072760 msgr2=0x7f0e3406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e389e4700 1 --2- 172.21.15.181:0/3443762476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e34072760 0x7f0e3406a8a0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f0e340b66b0 tx=0x7f0e28009b20 comp rx=0 tx=0).stop 2023-10-16T11:19:24.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e389e4700 1 -- 172.21.15.181:0/3443762476 shutdown_connections 2023-10-16T11:19:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e389e4700 1 --2- 172.21.15.181:0/3443762476 >> v2:172.21.15.190:3300/0 conn(0x7f0e3406ade0 0x7f0e341615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e389e4700 1 --2- 172.21.15.181:0/3443762476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e34072760 0x7f0e3406a8a0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.390+0000 7f0e389e4700 1 -- 172.21.15.181:0/3443762476 >> 172.21.15.181:0/3443762476 conn(0x7f0e340cd010 msgr2=0x7f0e340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.391+0000 7f0e389e4700 1 -- 172.21.15.181:0/3443762476 shutdown_connections 2023-10-16T11:19:24.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.391+0000 7f0e389e4700 1 -- 172.21.15.181:0/3443762476 wait complete. 2023-10-16T11:19:24.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.392+0000 7f0e389e4700 1 Processor -- start 2023-10-16T11:19:24.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.392+0000 7f0e389e4700 1 -- start start 2023-10-16T11:19:24.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e3406ade0 0x7f0e341f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e34072760 0x7f0e340c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e3415da70 0x7f0e340c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340c4e60 con 0x7f0e3406ade0 2023-10-16T11:19:24.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3414b6a0 con 0x7f0e34072760 2023-10-16T11:19:24.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e341f6bd0 con 0x7f0e3415da70 2023-10-16T11:19:24.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e3406ade0 0x7f0e341f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e3406ade0 0x7f0e341f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55404/0 (socket says 172.21.15.181:55404) 2023-10-16T11:19:24.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 -- 172.21.15.181:0/1996057798 learned_addr learned my addr 172.21.15.181:0/1996057798 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:24.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 -- 172.21.15.181:0/1996057798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e3415da70 msgr2=0x7f0e340c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e3415da70 0x7f0e340c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 -- 172.21.15.181:0/1996057798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e34072760 msgr2=0x7f0e340c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e34072760 0x7f0e340c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 -- 172.21.15.181:0/1996057798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e280094a0 con 0x7f0e3406ade0 2023-10-16T11:19:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e3259c700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e3406ade0 0x7f0e341f64d0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c520 tx=0x7f0e2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e22ffd700 1 -- 172.21.15.181:0/1996057798 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24005d70 con 0x7f0e3406ade0 2023-10-16T11:19:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e22ffd700 1 -- 172.21.15.181:0/1996057798 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e240070d0 con 0x7f0e3406ade0 2023-10-16T11:19:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e341f6eb0 con 0x7f0e3406ade0 2023-10-16T11:19:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e22ffd700 1 -- 172.21.15.181:0/1996057798 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007440 con 0x7f0e3406ade0 2023-10-16T11:19:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.393+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e340c6d70 con 0x7f0e3406ade0 2023-10-16T11:19:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.394+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e341f73d0 con 0x7f0e3406ade0 2023-10-16T11:19:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.395+0000 7f0e22ffd700 1 -- 172.21.15.181:0/1996057798 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0e240075a0 con 0x7f0e3406ade0 2023-10-16T11:19:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.395+0000 7f0e22ffd700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f0e1406b020 0x7f0e1406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.395+0000 7f0e22ffd700 1 -- 172.21.15.181:0/1996057798 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f0e24088d90 con 0x7f0e3406ade0 2023-10-16T11:19:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.395+0000 7f0e31d9b700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f0e1406b020 0x7f0e1406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.396+0000 7f0e31d9b700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f0e1406b020 0x7f0e1406d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e340d1650 tx=0x7f0e28010000 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.399+0000 7f0e22ffd700 1 -- 172.21.15.181:0/1996057798 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0e24053bb0 con 0x7f0e3406ade0 2023-10-16T11:19:24.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:24.637+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0e340c6f00 con 0x7f0e3406ade0 2023-10-16T11:19:25.518 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-10-16T11:19:25.516+0000 7fc17649f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:19:25.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.560+0000 7f0e22ffd700 1 -- 172.21.15.181:0/1996057798 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v110) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0e240170b0 con 0x7f0e3406ade0 2023-10-16T11:19:25.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f0e1406b020 msgr2=0x7f0e1406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:25.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f0e1406b020 0x7f0e1406d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e340d1650 tx=0x7f0e28010000 comp rx=0 tx=0).stop 2023-10-16T11:19:25.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e3406ade0 msgr2=0x7f0e341f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:25.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e3406ade0 0x7f0e341f64d0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c520 tx=0x7f0e2400c8e0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 shutdown_connections 2023-10-16T11:19:25.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f0e1406b020 0x7f0e1406d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e3415da70 0x7f0e340c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e3406ade0 0x7f0e341f64d0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 --2- 172.21.15.181:0/1996057798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e34072760 0x7f0e340c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 >> 172.21.15.181:0/1996057798 conn(0x7f0e340cd010 msgr2=0x7f0e340ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 shutdown_connections 2023-10-16T11:19:25.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.563+0000 7f0e389e4700 1 -- 172.21.15.181:0/1996057798 wait complete. 2023-10-16T11:19:25.579 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:19:25.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:19:25.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.708+0000 7f04841f9700 1 Processor -- start 2023-10-16T11:19:25.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.708+0000 7f04841f9700 1 -- start start 2023-10-16T11:19:25.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f04841f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 0x7f047c071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f04841f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0ccb60 con 0x7f047c0ce680 2023-10-16T11:19:25.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f04841f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0ce1f0 con 0x7f047c0b2010 2023-10-16T11:19:25.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f04841f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c072120 con 0x7f047c0cc6e0 2023-10-16T11:19:25.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f0482796700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f047c0ce680 0x7f047c0ceb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38262/0 (socket says 172.21.15.181:38262) 2023-10-16T11:19:25.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f0482796700 1 -- 172.21.15.181:0/3788134294 learned_addr learned my addr 172.21.15.181:0/3788134294 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f0481794700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 0x7f047c071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189163463 0 0) 0x7f0474001490 con 0x7f047c0ce680 2023-10-16T11:19:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.709+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04680035b0 con 0x7f047c0ce680 2023-10-16T11:19:25.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3823439348 0 0) 0x7f04680035b0 con 0x7f047c0ce680 2023-10-16T11:19:25.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 msgr2=0x7f047c071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:25.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 0x7f047c071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 >> v1:172.21.15.190:6789/0 conn(0x7f047c0b2010 legacy=0x7f047c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:25.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0ccda0 con 0x7f047c0ce680 2023-10-16T11:19:25.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f0474001b80 con 0x7f047c0ce680 2023-10-16T11:19:25.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 >> v1:172.21.15.181:6789/0 conn(0x7f047c0ce680 legacy=0x7f047c0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:25.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0468003a30 0x7f0468003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04680043e0 0x7f04680048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 0x7f0468005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c072120 con 0x7f0468003a30 2023-10-16T11:19:25.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046800ba10 con 0x7f04680043e0 2023-10-16T11:19:25.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046800bbd0 con 0x7f047c0cc6e0 2023-10-16T11:19:25.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481f95700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0468003a30 0x7f0468003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481794700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04680043e0 0x7f04680048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0482796700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 0x7f0468005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481f95700 1 -- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 msgr2=0x7f0468005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:25.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481f95700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 0x7f0468005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481f95700 1 -- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04680043e0 msgr2=0x7f04680048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:25.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481f95700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04680043e0 0x7f04680048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481f95700 1 -- 172.21.15.181:0/3788134294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04680035b0 con 0x7f0468003a30 2023-10-16T11:19:25.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0482796700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f047c0cc6e0 0x7f0468005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:25.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0481f95700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0468003a30 0x7f0468003e10 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f046c0018d0 tx=0x7f0468005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf7867b29608f2a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:25.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c010690 con 0x7f0468003a30 2023-10-16T11:19:25.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c010810 con 0x7f0468003a30 2023-10-16T11:19:25.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.710+0000 7f0480f93700 1 -- 172.21.15.181:0/3788134294 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c010bc0 con 0x7f0468003a30 2023-10-16T11:19:25.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.711+0000 7f04841f9700 1 -- 172.21.15.181:0/3788134294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0468003a30 msgr2=0x7f0468003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:25.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.711+0000 7f04841f9700 1 --2- 172.21.15.181:0/3788134294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0468003a30 0x7f0468003e10 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f046c0018d0 tx=0x7f0468005880 comp rx=0 tx=0).stop 2023-10-16T11:19:25.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.711+0000 7f0482796700 1 -- 172.21.15.181:0/3788134294 reap_dead start 2023-10-16T11:19:25.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.711+0000 7f04841f9700 1 -- 172.21.15.181:0/3788134294 shutdown_connections 2023-10-16T11:19:25.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.711+0000 7f04841f9700 1 -- 172.21.15.181:0/3788134294 >> 172.21.15.181:0/3788134294 conn(0x7f047c0cd010 msgr2=0x7f047c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:25.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.711+0000 7f04841f9700 1 -- 172.21.15.181:0/3788134294 shutdown_connections 2023-10-16T11:19:25.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.711+0000 7f04841f9700 1 -- 172.21.15.181:0/3788134294 wait complete. 2023-10-16T11:19:25.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.712+0000 7f04841f9700 1 Processor -- start 2023-10-16T11:19:25.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.712+0000 7f04841f9700 1 -- start start 2023-10-16T11:19:25.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f04841f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468003a30 0x7f047c14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f04841f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1584d0 0x7f047c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f04841f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f047c15c4e0 0x7f047c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f04841f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c158e90 con 0x7f047c1584d0 2023-10-16T11:19:25.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f04841f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0c7890 con 0x7f047c15c4e0 2023-10-16T11:19:25.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f04841f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c15d260 con 0x7f0468003a30 2023-10-16T11:19:25.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1584d0 0x7f047c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1584d0 0x7f047c158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55426/0 (socket says 172.21.15.181:55426) 2023-10-16T11:19:25.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 -- 172.21.15.181:0/862955343 learned_addr learned my addr 172.21.15.181:0/862955343 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:25.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481f95700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468003a30 0x7f047c14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0482796700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f047c15c4e0 0x7f047c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 -- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468003a30 msgr2=0x7f047c14ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:25.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468003a30 0x7f047c14ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 -- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f047c15c4e0 msgr2=0x7f047c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:25.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f047c15c4e0 0x7f047c15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:25.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 -- 172.21.15.181:0/862955343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c15d540 con 0x7f047c1584d0 2023-10-16T11:19:25.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.713+0000 7f0481794700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1584d0 0x7f047c158950 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f0478011aa0 tx=0x7f0478011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:25.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.714+0000 7f04727fc700 1 -- 172.21.15.181:0/862955343 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478011ee0 con 0x7f047c1584d0 2023-10-16T11:19:25.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.714+0000 7f04727fc700 1 -- 172.21.15.181:0/862955343 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047800f8c0 con 0x7f047c1584d0 2023-10-16T11:19:25.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.714+0000 7f04727fc700 1 -- 172.21.15.181:0/862955343 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800fc70 con 0x7f047c1584d0 2023-10-16T11:19:25.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.714+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c15d820 con 0x7f047c1584d0 2023-10-16T11:19:25.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.714+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c1eed00 con 0x7f047c1584d0 2023-10-16T11:19:25.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.715+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c15da60 con 0x7f047c1584d0 2023-10-16T11:19:25.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.715+0000 7f04727fc700 1 -- 172.21.15.181:0/862955343 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0478014440 con 0x7f047c1584d0 2023-10-16T11:19:25.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.715+0000 7f04727fc700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f046006b020 0x7f046006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:25.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.715+0000 7f04727fc700 1 -- 172.21.15.181:0/862955343 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f0478089e90 con 0x7f047c1584d0 2023-10-16T11:19:25.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.716+0000 7f0481f95700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f046006b020 0x7f046006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:25.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.716+0000 7f0481f95700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f046006b020 0x7f046006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f046c0018d0 tx=0x7f046c01d040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:25.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.720+0000 7f04727fc700 1 -- 172.21.15.181:0/862955343 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f047c15da60 con 0x7f047c1584d0 2023-10-16T11:19:25.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:25.959+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f047c15da60 con 0x7f047c1584d0 2023-10-16T11:19:26.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.561+0000 7f04727fc700 1 -- 172.21.15.181:0/862955343 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v111) v1 ==== 122+0+0 (secure 0 0 0) 0x7f047c15da60 con 0x7f047c1584d0 2023-10-16T11:19:26.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f046006b020 msgr2=0x7f046006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:26.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f046006b020 0x7f046006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f046c0018d0 tx=0x7f046c01d040 comp rx=0 tx=0).stop 2023-10-16T11:19:26.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1584d0 msgr2=0x7f047c158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:26.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1584d0 0x7f047c158950 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f0478011aa0 tx=0x7f0478011db0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 shutdown_connections 2023-10-16T11:19:26.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f046006b020 0x7f046006d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0468003a30 0x7f047c14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f047c15c4e0 0x7f047c15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 --2- 172.21.15.181:0/862955343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f047c1584d0 0x7f047c158950 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 >> 172.21.15.181:0/862955343 conn(0x7f047c0cd010 msgr2=0x7f047c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:26.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 shutdown_connections 2023-10-16T11:19:26.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.564+0000 7f04841f9700 1 -- 172.21.15.181:0/862955343 wait complete. 2023-10-16T11:19:26.580 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:19:26.581 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:26.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.704+0000 7f813b4b4700 1 Processor -- start 2023-10-16T11:19:26.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.704+0000 7f813b4b4700 1 -- start start 2023-10-16T11:19:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f813b4b4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f813415dd00 0x7f8134071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f813b4b4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8134157c00 0x7f81340c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:26.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f813b4b4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8134071df0 con 0x7f8134071870 2023-10-16T11:19:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f813b4b4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81340720d0 con 0x7f8134157c00 2023-10-16T11:19:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f813b4b4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81340723b0 con 0x7f813415dd00 2023-10-16T11:19:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8134157c00 0x7f81340c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:26.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8134157c00 0x7f81340c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55436/0 (socket says 172.21.15.181:55436) 2023-10-16T11:19:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 -- 172.21.15.181:0/3578394787 learned_addr learned my addr 172.21.15.181:0/3578394787 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139250700 1 --2- 172.21.15.181:0/3578394787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f813415dd00 0x7f8134071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 -- 172.21.15.181:0/3578394787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f813415dd00 msgr2=0x7f8134071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:26.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 --2- 172.21.15.181:0/3578394787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f813415dd00 0x7f8134071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 -- 172.21.15.181:0/3578394787 >> v1:172.21.15.190:6789/0 conn(0x7f8134071870 legacy=0x7f81341574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 -- 172.21.15.181:0/3578394787 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134072690 con 0x7f8134157c00 2023-10-16T11:19:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139250700 1 --2- 172.21.15.181:0/3578394787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f813415dd00 0x7f8134071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f8139a51700 1 --2- 172.21.15.181:0/3578394787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f81340c9b30 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f812c009e50 tx=0x7f812c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d974f4dfb52368d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f812bfff700 1 -- 172.21.15.181:0/3578394787 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c00e070 con 0x7f8134157c00 2023-10-16T11:19:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f812bfff700 1 -- 172.21.15.181:0/3578394787 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c003a20 con 0x7f8134157c00 2023-10-16T11:19:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.705+0000 7f812bfff700 1 -- 172.21.15.181:0/3578394787 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c003dd0 con 0x7f8134157c00 2023-10-16T11:19:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 -- 172.21.15.181:0/3578394787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 msgr2=0x7f81340c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:26.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 --2- 172.21.15.181:0/3578394787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f81340c9b30 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f812c009e50 tx=0x7f812c00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 -- 172.21.15.181:0/3578394787 shutdown_connections 2023-10-16T11:19:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 --2- 172.21.15.181:0/3578394787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f81340c9b30 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 --2- 172.21.15.181:0/3578394787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f813415dd00 0x7f8134071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 -- 172.21.15.181:0/3578394787 >> 172.21.15.181:0/3578394787 conn(0x7f813406aed0 msgr2=0x7f81340b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:26.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 -- 172.21.15.181:0/3578394787 shutdown_connections 2023-10-16T11:19:26.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.706+0000 7f813b4b4700 1 -- 172.21.15.181:0/3578394787 wait complete. 2023-10-16T11:19:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.708+0000 7f813b4b4700 1 Processor -- start 2023-10-16T11:19:26.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.708+0000 7f813b4b4700 1 -- start start 2023-10-16T11:19:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f813b4b4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8134071870 0x7f813414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f813b4b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f813414f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f813b4b4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813415dd00 0x7f81340c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f813b4b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81341f10c0 con 0x7f8134157c00 2023-10-16T11:19:26.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f813b4b4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81341f0440 con 0x7f813415dd00 2023-10-16T11:19:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f813b4b4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81341f0720 con 0x7f8134071870 2023-10-16T11:19:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f813414f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f813414f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55440/0 (socket says 172.21.15.181:55440) 2023-10-16T11:19:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 -- 172.21.15.181:0/4187036940 learned_addr learned my addr 172.21.15.181:0/4187036940 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:26.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8139250700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8134071870 0x7f813414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:26.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8139a51700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813415dd00 0x7f81340c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:26.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 -- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8134071870 msgr2=0x7f813414ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:26.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8134071870 0x7f813414ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 -- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813415dd00 msgr2=0x7f81340c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:26.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813415dd00 0x7f81340c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 -- 172.21.15.181:0/4187036940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c009b30 con 0x7f8134157c00 2023-10-16T11:19:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8138a4f700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f813414f440 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f812400c4d0 tx=0x7f812400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8129ffb700 1 -- 172.21.15.181:0/4187036940 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011790 con 0x7f8134157c00 2023-10-16T11:19:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8129ffb700 1 -- 172.21.15.181:0/4187036940 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81240118f0 con 0x7f8134157c00 2023-10-16T11:19:26.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81341f0a00 con 0x7f8134157c00 2023-10-16T11:19:26.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.709+0000 7f8129ffb700 1 -- 172.21.15.181:0/4187036940 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011ca0 con 0x7f8134157c00 2023-10-16T11:19:26.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.710+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81341f00c0 con 0x7f8134157c00 2023-10-16T11:19:26.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.710+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81340cb210 con 0x7f8134157c00 2023-10-16T11:19:26.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.711+0000 7f8129ffb700 1 -- 172.21.15.181:0/4187036940 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f812401c460 con 0x7f8134157c00 2023-10-16T11:19:26.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.711+0000 7f8129ffb700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f811806b020 0x7f811806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:26.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.711+0000 7f8129ffb700 1 -- 172.21.15.181:0/4187036940 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f8124089760 con 0x7f8134157c00 2023-10-16T11:19:26.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.711+0000 7f8139250700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f811806b020 0x7f811806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:26.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.711+0000 7f8139250700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f811806b020 0x7f811806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8130006fd0 tx=0x7f8130008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:26.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.716+0000 7f8129ffb700 1 -- 172.21.15.181:0/4187036940 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8124054560 con 0x7f8134157c00 2023-10-16T11:19:26.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.949+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f813404ebb0 con 0x7f8134157c00 2023-10-16T11:19:26.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.950+0000 7f8129ffb700 1 -- 172.21.15.181:0/4187036940 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f8124057b80 con 0x7f8134157c00 2023-10-16T11:19:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:26.953 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":111,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":11325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-16T11:19:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f811806b020 msgr2=0x7f811806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:26.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f811806b020 0x7f811806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8130006fd0 tx=0x7f8130008040 comp rx=0 tx=0).stop 2023-10-16T11:19:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 msgr2=0x7f813414f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f813414f440 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f812400c4d0 tx=0x7f812400c890 comp rx=0 tx=0).stop 2023-10-16T11:19:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 shutdown_connections 2023-10-16T11:19:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f811806b020 0x7f811806d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8134071870 0x7f813414ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f813415dd00 0x7f81340c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 --2- 172.21.15.181:0/4187036940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8134157c00 0x7f813414f440 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:26.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 >> 172.21.15.181:0/4187036940 conn(0x7f813406aed0 msgr2=0x7f8134155430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:26.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 shutdown_connections 2023-10-16T11:19:26.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:26.952+0000 7f813b4b4700 1 -- 172.21.15.181:0/4187036940 wait complete. 2023-10-16T11:19:26.957 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 111 2023-10-16T11:19:26.966 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.084+0000 7ff1c7cc1700 1 Processor -- start 2023-10-16T11:19:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.084+0000 7ff1c7cc1700 1 -- start start 2023-10-16T11:19:27.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c7cc1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00ce7f0 0x7ff1c006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c7cc1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1c00d2390 0x7ff1c00cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c7cc1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1c00d0180 0x7ff1c0161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c7cc1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c00d0630 con 0x7ff1c00d0180 2023-10-16T11:19:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c7cc1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c00cc470 con 0x7ff1c00d2390 2023-10-16T11:19:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c7cc1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c006ca30 con 0x7ff1c00ce7f0 2023-10-16T11:19:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c625e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1c00d0180 0x7ff1c0161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c625e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1c00d0180 0x7ff1c0161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55450/0 (socket says 172.21.15.181:55450) 2023-10-16T11:19:27.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c625e700 1 -- 172.21.15.181:0/1056448651 learned_addr learned my addr 172.21.15.181:0/1056448651 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c525c700 1 --2- 172.21.15.181:0/1056448651 >> v2:172.21.15.190:3300/0 conn(0x7ff1c00d2390 0x7ff1c00cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.085+0000 7ff1c5a5d700 1 --2- 172.21.15.181:0/1056448651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00ce7f0 0x7ff1c006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c625e700 1 -- 172.21.15.181:0/1056448651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00ce7f0 msgr2=0x7ff1c006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c625e700 1 --2- 172.21.15.181:0/1056448651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00ce7f0 0x7ff1c006c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c625e700 1 -- 172.21.15.181:0/1056448651 >> v2:172.21.15.190:3300/0 conn(0x7ff1c00d2390 msgr2=0x7ff1c00cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c625e700 1 --2- 172.21.15.181:0/1056448651 >> v2:172.21.15.190:3300/0 conn(0x7ff1c00d2390 0x7ff1c00cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c625e700 1 -- 172.21.15.181:0/1056448651 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c006cd10 con 0x7ff1c00d0180 2023-10-16T11:19:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c525c700 1 --2- 172.21.15.181:0/1056448651 >> v2:172.21.15.190:3300/0 conn(0x7ff1c00d2390 0x7ff1c00cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c625e700 1 --2- 172.21.15.181:0/1056448651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d0180 0x7ff1c0161510 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7ff1c006b210 tx=0x7ff1b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73ba4bafc8c1832 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c4a5b700 1 -- 172.21.15.181:0/1056448651 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800e070 con 0x7ff1c00d0180 2023-10-16T11:19:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c4a5b700 1 -- 172.21.15.181:0/1056448651 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b800b9c0 con 0x7ff1c00d0180 2023-10-16T11:19:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c4a5b700 1 -- 172.21.15.181:0/1056448651 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800bd70 con 0x7ff1c00d0180 2023-10-16T11:19:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/1056448651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d0180 msgr2=0x7ff1c0161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/1056448651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d0180 0x7ff1c0161510 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7ff1c006b210 tx=0x7ff1b800b330 comp rx=0 tx=0).stop 2023-10-16T11:19:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/1056448651 shutdown_connections 2023-10-16T11:19:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/1056448651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d0180 0x7ff1c0161510 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/1056448651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00ce7f0 0x7ff1c006c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/1056448651 >> v2:172.21.15.190:3300/0 conn(0x7ff1c00d2390 0x7ff1c00cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.086+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/1056448651 >> 172.21.15.181:0/1056448651 conn(0x7ff1c009a9a0 msgr2=0x7ff1c00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:27.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.087+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/1056448651 shutdown_connections 2023-10-16T11:19:27.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.087+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/1056448651 wait complete. 2023-10-16T11:19:27.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 Processor -- start 2023-10-16T11:19:27.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 -- start start 2023-10-16T11:19:27.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c00ce7f0 0x7ff1c015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00d0180 0x7ff1c00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d2390 0x7ff1c00c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c00b42a0 con 0x7ff1c00d2390 2023-10-16T11:19:27.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c015c940 con 0x7ff1c00ce7f0 2023-10-16T11:19:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c7cc1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c00b44f0 con 0x7ff1c00d0180 2023-10-16T11:19:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c625e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d2390 0x7ff1c00c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c625e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d2390 0x7ff1c00c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55456/0 (socket says 172.21.15.181:55456) 2023-10-16T11:19:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.088+0000 7ff1c625e700 1 -- 172.21.15.181:0/419937063 learned_addr learned my addr 172.21.15.181:0/419937063 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:27.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c5a5d700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c00ce7f0 0x7ff1c015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c525c700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00d0180 0x7ff1c00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c625e700 1 -- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00d0180 msgr2=0x7ff1c00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c625e700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00d0180 0x7ff1c00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c625e700 1 -- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c00ce7f0 msgr2=0x7ff1c015c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c625e700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c00ce7f0 0x7ff1c015c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c625e700 1 -- 172.21.15.181:0/419937063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009b30 con 0x7ff1c00d2390 2023-10-16T11:19:27.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c525c700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00d0180 0x7ff1c00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:27.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c625e700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d2390 0x7ff1c00c9ac0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7ff1c008bdb0 tx=0x7ff1b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1b67fc700 1 -- 172.21.15.181:0/419937063 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80099b0 con 0x7ff1c00d2390 2023-10-16T11:19:27.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1b67fc700 1 -- 172.21.15.181:0/419937063 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8004250 con 0x7ff1c00d2390 2023-10-16T11:19:27.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c00b47d0 con 0x7ff1c00d2390 2023-10-16T11:19:27.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1b67fc700 1 -- 172.21.15.181:0/419937063 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8004600 con 0x7ff1c00d2390 2023-10-16T11:19:27.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.089+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c00b4d90 con 0x7ff1c00d2390 2023-10-16T11:19:27.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.090+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c0067d20 con 0x7ff1c00d2390 2023-10-16T11:19:27.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.090+0000 7ff1b67fc700 1 -- 172.21.15.181:0/419937063 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff1b8004760 con 0x7ff1c00d2390 2023-10-16T11:19:27.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.091+0000 7ff1b67fc700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff1a406b020 0x7ff1a406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.091+0000 7ff1b67fc700 1 -- 172.21.15.181:0/419937063 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ff1b808ab80 con 0x7ff1c00d2390 2023-10-16T11:19:27.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.091+0000 7ff1c5a5d700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff1a406b020 0x7ff1a406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.091+0000 7ff1c5a5d700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff1a406b020 0x7ff1a406d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006fd0 tx=0x7ff1b0008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.095+0000 7ff1b67fc700 1 -- 172.21.15.181:0/419937063 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff1b8055870 con 0x7ff1c00d2390 2023-10-16T11:19:27.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.289+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1c00cef40 con 0x7ff1c00d2390 2023-10-16T11:19:27.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.290+0000 7ff1b67fc700 1 -- 172.21.15.181:0/419937063 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+17470 (secure 0 0 0) 0x7ff1b801f020 con 0x7ff1c00d2390 2023-10-16T11:19:27.293 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:27.293 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":216,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:24.532965+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:27.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.292+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff1a406b020 msgr2=0x7ff1a406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.292+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff1a406b020 0x7ff1a406d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006fd0 tx=0x7ff1b0008040 comp rx=0 tx=0).stop 2023-10-16T11:19:27.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.292+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d2390 msgr2=0x7ff1c00c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.292+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d2390 0x7ff1c00c9ac0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7ff1c008bdb0 tx=0x7ff1b8004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 shutdown_connections 2023-10-16T11:19:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff1a406b020 0x7ff1a406d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c00d2390 0x7ff1c00c9ac0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1c00ce7f0 0x7ff1c015c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 --2- 172.21.15.181:0/419937063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1c00d0180 0x7ff1c00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 >> 172.21.15.181:0/419937063 conn(0x7ff1c009a9a0 msgr2=0x7ff1c00cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:27.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 shutdown_connections 2023-10-16T11:19:27.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.293+0000 7ff1c7cc1700 1 -- 172.21.15.181:0/419937063 wait complete. 2023-10-16T11:19:27.307 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:19:27.308 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:19:27.309 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:27.311 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:27.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.429+0000 7f8848ba2700 1 Processor -- start 2023-10-16T11:19:27.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.429+0000 7f8848ba2700 1 -- start start 2023-10-16T11:19:27.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8848ba2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 0x7f88440cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8848ba2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f884406e740 0x7f88440aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8848ba2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88440cf1f0 con 0x7f88440d0b70 2023-10-16T11:19:27.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8848ba2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884406e5b0 con 0x7f884406e740 2023-10-16T11:19:27.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8848ba2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88440afd10 con 0x7f884406d910 2023-10-16T11:19:27.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8842ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f88440d0b70 0x7f884406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38306/0 (socket says 172.21.15.181:38306) 2023-10-16T11:19:27.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8842ffd700 1 -- 172.21.15.181:0/2715205900 learned_addr learned my addr 172.21.15.181:0/2715205900 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:27.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572252169 0 0) 0x7f88440cf1f0 con 0x7f88440d0b70 2023-10-16T11:19:27.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88300035b0 con 0x7f88440d0b70 2023-10-16T11:19:27.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f8843fff700 1 --2- 172.21.15.181:0/2715205900 >> v2:172.21.15.190:3300/0 conn(0x7f884406e740 0x7f88440aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.430+0000 7f88437fe700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 0x7f88440cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2266590928 0 0) 0x7f88300035b0 con 0x7f88440d0b70 2023-10-16T11:19:27.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 msgr2=0x7f88440cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 0x7f88440cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 >> v2:172.21.15.190:3300/0 conn(0x7f884406e740 msgr2=0x7f88440aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 --2- 172.21.15.181:0/2715205900 >> v2:172.21.15.190:3300/0 conn(0x7f884406e740 0x7f88440aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440afa30 con 0x7f88440d0b70 2023-10-16T11:19:27.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f8834001670 con 0x7f88440d0b70 2023-10-16T11:19:27.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 >> v1:172.21.15.181:6789/0 conn(0x7f88440d0b70 legacy=0x7f884406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830003a00 0x7f8830003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 0x7f8830004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830004d90 0x7f8830005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884406e5b0 con 0x7f8830003a00 2023-10-16T11:19:27.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88440afd10 con 0x7f8830004d90 2023-10-16T11:19:27.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883000dc20 con 0x7f884406d910 2023-10-16T11:19:27.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88437fe700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830003a00 0x7f8830003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f8842ffd700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 0x7f8830004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f8843fff700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830004d90 0x7f8830005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88437fe700 1 -- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 msgr2=0x7f8830004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88437fe700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f884406d910 0x7f8830004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88437fe700 1 -- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830004d90 msgr2=0x7f8830005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88437fe700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830004d90 0x7f8830005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88437fe700 1 -- 172.21.15.181:0/2715205900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300035b0 con 0x7f8830003a00 2023-10-16T11:19:27.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88437fe700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830003a00 0x7f8830003e20 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f882c009b40 tx=0x7f8830005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8601f6cc3dde0ff8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c005a20 con 0x7f8830003a00 2023-10-16T11:19:27.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c005b80 con 0x7f8830003a00 2023-10-16T11:19:27.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.431+0000 7f88427fc700 1 -- 172.21.15.181:0/2715205900 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00e6c0 con 0x7f8830003a00 2023-10-16T11:19:27.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.432+0000 7f8848ba2700 1 -- 172.21.15.181:0/2715205900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830003a00 msgr2=0x7f8830003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.432+0000 7f8848ba2700 1 --2- 172.21.15.181:0/2715205900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8830003a00 0x7f8830003e20 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f882c009b40 tx=0x7f8830005880 comp rx=0 tx=0).stop 2023-10-16T11:19:27.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.432+0000 7f8843fff700 1 -- 172.21.15.181:0/2715205900 reap_dead start 2023-10-16T11:19:27.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.432+0000 7f8848ba2700 1 -- 172.21.15.181:0/2715205900 shutdown_connections 2023-10-16T11:19:27.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.432+0000 7f8848ba2700 1 -- 172.21.15.181:0/2715205900 >> 172.21.15.181:0/2715205900 conn(0x7f884406c430 msgr2=0x7f88440d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:27.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.432+0000 7f8848ba2700 1 -- 172.21.15.181:0/2715205900 shutdown_connections 2023-10-16T11:19:27.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.432+0000 7f8848ba2700 1 -- 172.21.15.181:0/2715205900 wait complete. 2023-10-16T11:19:27.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.434+0000 7f8848ba2700 1 Processor -- start 2023-10-16T11:19:27.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.434+0000 7f8848ba2700 1 -- start start 2023-10-16T11:19:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8848ba2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830003a00 0x7f8844149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8848ba2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8844156b10 0x7f88440c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8848ba2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88440b4d00 0x7f88440c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8848ba2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884414a000 con 0x7f8844156b10 2023-10-16T11:19:27.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8848ba2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88440cd8b0 con 0x7f8830003a00 2023-10-16T11:19:27.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8848ba2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88440b4220 con 0x7f88440b4d00 2023-10-16T11:19:27.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8844156b10 0x7f88440c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8844156b10 0x7f88440c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55474/0 (socket says 172.21.15.181:55474) 2023-10-16T11:19:27.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 -- 172.21.15.181:0/824944091 learned_addr learned my addr 172.21.15.181:0/824944091 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8843fff700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88440b4d00 0x7f88440c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f88437fe700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830003a00 0x7f8844149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 -- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88440b4d00 msgr2=0x7f88440c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88440b4d00 0x7f88440c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 -- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830003a00 msgr2=0x7f8844149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830003a00 0x7f8844149ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 -- 172.21.15.181:0/824944091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440b4500 con 0x7f8844156b10 2023-10-16T11:19:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f88437fe700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830003a00 0x7f8844149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8842ffd700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8844156b10 0x7f88440c75e0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f8834001aa0 tx=0x7f88340075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f883bfff700 1 -- 172.21.15.181:0/824944091 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340077d0 con 0x7f8844156b10 2023-10-16T11:19:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f883bfff700 1 -- 172.21.15.181:0/824944091 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883400f8a0 con 0x7f8844156b10 2023-10-16T11:19:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88440b47e0 con 0x7f8844156b10 2023-10-16T11:19:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.435+0000 7f883bfff700 1 -- 172.21.15.181:0/824944091 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883400fc50 con 0x7f8844156b10 2023-10-16T11:19:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.436+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88440c8060 con 0x7f8844156b10 2023-10-16T11:19:27.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.436+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88440b4a00 con 0x7f8844156b10 2023-10-16T11:19:27.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.437+0000 7f883bfff700 1 -- 172.21.15.181:0/824944091 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f883401d460 con 0x7f8844156b10 2023-10-16T11:19:27.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.437+0000 7f883bfff700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f882406b020 0x7f882406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.437+0000 7f883bfff700 1 -- 172.21.15.181:0/824944091 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f8834089980 con 0x7f8844156b10 2023-10-16T11:19:27.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.437+0000 7f88437fe700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f882406b020 0x7f882406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.438+0000 7f88437fe700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f882406b020 0x7f882406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f882c000c00 tx=0x7f882c008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.442+0000 7f883bfff700 1 -- 172.21.15.181:0/824944091 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f88340546f0 con 0x7f8844156b10 2023-10-16T11:19:27.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.632+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f884406f380 con 0x7f882406b020 2023-10-16T11:19:27.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.641+0000 7f883bfff700 1 -- 172.21.15.181:0/824944091 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290025 (secure 0 0 0) 0x7f882c005db0 con 0x7f882406b020 2023-10-16T11:19:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:27.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f882406b020 msgr2=0x7f882406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f882406b020 0x7f882406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f882c000c00 tx=0x7f882c008030 comp rx=0 tx=0).stop 2023-10-16T11:19:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8844156b10 msgr2=0x7f88440c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8844156b10 0x7f88440c75e0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f8834001aa0 tx=0x7f88340075f0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 shutdown_connections 2023-10-16T11:19:27.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f882406b020 0x7f882406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88440b4d00 0x7f88440c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8830003a00 0x7f8844149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 --2- 172.21.15.181:0/824944091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8844156b10 0x7f88440c75e0 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 >> 172.21.15.181:0/824944091 conn(0x7f884406c430 msgr2=0x7f88440ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:27.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 shutdown_connections 2023-10-16T11:19:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.644+0000 7f8848ba2700 1 -- 172.21.15.181:0/824944091 wait complete. 2023-10-16T11:19:27.652 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:19:27.658 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-16T11:19:27.481304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421712,"kb_used_data":22328,"kb_used_omap":12,"kb_used_meta":399219,"kb_avail":754553008,"statfs":{"total":773094113280,"available":772662280192,"internally_reserved":0,"allocated":22863872,"data_stored":12261828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":408800824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.951733"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553571+0000","last_change":"2023-10-16T11:19:23.547472+0000","last_active":"2023-10-16T11:19:23.553571+0000","last_peered":"2023-10-16T11:19:23.553571+0000","last_clean":"2023-10-16T11:19:23.553571+0000","last_became_active":"2023-10-16T11:19:23.547148+0000","last_became_peered":"2023-10-16T11:19:23.547148+0000","last_unstale":"2023-10-16T11:19:23.553571+0000","last_undegraded":"2023-10-16T11:19:23.553571+0000","last_fullsized":"2023-10-16T11:19:23.553571+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540677+0000","last_change":"2023-10-16T11:19:23.552427+0000","last_active":"2023-10-16T11:19:24.540677+0000","last_peered":"2023-10-16T11:19:24.540677+0000","last_clean":"2023-10-16T11:19:24.540677+0000","last_became_active":"2023-10-16T11:19:23.552156+0000","last_became_peered":"2023-10-16T11:19:23.552156+0000","last_unstale":"2023-10-16T11:19:24.540677+0000","last_undegraded":"2023-10-16T11:19:24.540677+0000","last_fullsized":"2023-10-16T11:19:24.540677+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555675+0000","last_change":"2023-10-16T11:19:23.552160+0000","last_active":"2023-10-16T11:19:23.555675+0000","last_peered":"2023-10-16T11:19:23.555675+0000","last_clean":"2023-10-16T11:19:23.555675+0000","last_became_active":"2023-10-16T11:19:23.551805+0000","last_became_peered":"2023-10-16T11:19:23.551805+0000","last_unstale":"2023-10-16T11:19:23.555675+0000","last_undegraded":"2023-10-16T11:19:23.555675+0000","last_fullsized":"2023-10-16T11:19:23.555675+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540224+0000","last_change":"2023-10-16T11:19:23.547136+0000","last_active":"2023-10-16T11:19:24.540224+0000","last_peered":"2023-10-16T11:19:24.540224+0000","last_clean":"2023-10-16T11:19:24.540224+0000","last_became_active":"2023-10-16T11:19:23.546834+0000","last_became_peered":"2023-10-16T11:19:23.546834+0000","last_unstale":"2023-10-16T11:19:24.540224+0000","last_undegraded":"2023-10-16T11:19:24.540224+0000","last_fullsized":"2023-10-16T11:19:24.540224+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:45.683847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.544475+0000","last_change":"2023-10-16T11:19:23.541856+0000","last_active":"2023-10-16T11:19:23.544475+0000","last_peered":"2023-10-16T11:19:23.544475+0000","last_clean":"2023-10-16T11:19:23.544475+0000","last_became_active":"2023-10-16T11:19:23.541549+0000","last_became_peered":"2023-10-16T11:19:23.541549+0000","last_unstale":"2023-10-16T11:19:23.544475+0000","last_undegraded":"2023-10-16T11:19:23.544475+0000","last_fullsized":"2023-10-16T11:19:23.544475+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:27.510199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.552795+0000","last_change":"2023-10-16T11:19:23.550179+0000","last_active":"2023-10-16T11:19:23.552795+0000","last_peered":"2023-10-16T11:19:23.552795+0000","last_clean":"2023-10-16T11:19:23.552795+0000","last_became_active":"2023-10-16T11:19:23.549811+0000","last_became_peered":"2023-10-16T11:19:23.549811+0000","last_unstale":"2023-10-16T11:19:23.552795+0000","last_undegraded":"2023-10-16T11:19:23.552795+0000","last_fullsized":"2023-10-16T11:19:23.552795+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:44.413095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553442+0000","last_change":"2023-10-16T11:19:23.547259+0000","last_active":"2023-10-16T11:19:23.553442+0000","last_peered":"2023-10-16T11:19:23.553442+0000","last_clean":"2023-10-16T11:19:23.553442+0000","last_became_active":"2023-10-16T11:19:23.546957+0000","last_became_peered":"2023-10-16T11:19:23.546957+0000","last_unstale":"2023-10-16T11:19:23.553442+0000","last_undegraded":"2023-10-16T11:19:23.553442+0000","last_fullsized":"2023-10-16T11:19:23.553442+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:35.177289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.552844+0000","last_change":"2023-10-16T11:19:23.550227+0000","last_active":"2023-10-16T11:19:23.552844+0000","last_peered":"2023-10-16T11:19:23.552844+0000","last_clean":"2023-10-16T11:19:23.552844+0000","last_became_active":"2023-10-16T11:19:23.549938+0000","last_became_peered":"2023-10-16T11:19:23.549938+0000","last_unstale":"2023-10-16T11:19:23.552844+0000","last_undegraded":"2023-10-16T11:19:23.552844+0000","last_fullsized":"2023-10-16T11:19:23.552844+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:31.219643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540007+0000","last_change":"2023-10-16T11:19:23.552080+0000","last_active":"2023-10-16T11:19:24.540007+0000","last_peered":"2023-10-16T11:19:24.540007+0000","last_clean":"2023-10-16T11:19:24.540007+0000","last_became_active":"2023-10-16T11:19:23.551779+0000","last_became_peered":"2023-10-16T11:19:23.551779+0000","last_unstale":"2023-10-16T11:19:24.540007+0000","last_undegraded":"2023-10-16T11:19:24.540007+0000","last_fullsized":"2023-10-16T11:19:24.540007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536060+0000","last_change":"2023-10-16T11:19:23.546735+0000","last_active":"2023-10-16T11:19:24.536060+0000","last_peered":"2023-10-16T11:19:24.536060+0000","last_clean":"2023-10-16T11:19:24.536060+0000","last_became_active":"2023-10-16T11:19:23.546404+0000","last_became_peered":"2023-10-16T11:19:23.546404+0000","last_unstale":"2023-10-16T11:19:24.536060+0000","last_undegraded":"2023-10-16T11:19:24.536060+0000","last_fullsized":"2023-10-16T11:19:24.536060+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:29.845437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564649+0000","last_change":"2023-10-16T11:19:23.564191+0000","last_active":"2023-10-16T11:19:23.564649+0000","last_peered":"2023-10-16T11:19:23.564649+0000","last_clean":"2023-10-16T11:19:23.564649+0000","last_became_active":"2023-10-16T11:19:23.563291+0000","last_became_peered":"2023-10-16T11:19:23.563291+0000","last_unstale":"2023-10-16T11:19:23.564649+0000","last_undegraded":"2023-10-16T11:19:23.564649+0000","last_fullsized":"2023-10-16T11:19:23.564649+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.562906+0000","last_change":"2023-10-16T11:19:23.560016+0000","last_active":"2023-10-16T11:19:23.562906+0000","last_peered":"2023-10-16T11:19:23.562906+0000","last_clean":"2023-10-16T11:19:23.562906+0000","last_became_active":"2023-10-16T11:19:23.559619+0000","last_became_peered":"2023-10-16T11:19:23.559619+0000","last_unstale":"2023-10-16T11:19:23.562906+0000","last_undegraded":"2023-10-16T11:19:23.562906+0000","last_fullsized":"2023-10-16T11:19:23.562906+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:06.791712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564625+0000","last_change":"2023-10-16T11:19:23.560050+0000","last_active":"2023-10-16T11:19:23.564625+0000","last_peered":"2023-10-16T11:19:23.564625+0000","last_clean":"2023-10-16T11:19:23.564625+0000","last_became_active":"2023-10-16T11:19:23.559558+0000","last_became_peered":"2023-10-16T11:19:23.559558+0000","last_unstale":"2023-10-16T11:19:23.564625+0000","last_undegraded":"2023-10-16T11:19:23.564625+0000","last_fullsized":"2023-10-16T11:19:23.564625+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:13.873593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.560877+0000","last_change":"2023-10-16T11:19:23.559938+0000","last_active":"2023-10-16T11:19:23.560877+0000","last_peered":"2023-10-16T11:19:23.560877+0000","last_clean":"2023-10-16T11:19:23.560877+0000","last_became_active":"2023-10-16T11:19:23.559569+0000","last_became_peered":"2023-10-16T11:19:23.559569+0000","last_unstale":"2023-10-16T11:19:23.560877+0000","last_undegraded":"2023-10-16T11:19:23.560877+0000","last_fullsized":"2023-10-16T11:19:23.560877+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:34:43.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555408+0000","last_change":"2023-10-16T11:19:23.551280+0000","last_active":"2023-10-16T11:19:23.555408+0000","last_peered":"2023-10-16T11:19:23.555408+0000","last_clean":"2023-10-16T11:19:23.555408+0000","last_became_active":"2023-10-16T11:19:23.551018+0000","last_became_peered":"2023-10-16T11:19:23.551018+0000","last_unstale":"2023-10-16T11:19:23.555408+0000","last_undegraded":"2023-10-16T11:19:23.555408+0000","last_fullsized":"2023-10-16T11:19:23.555408+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:04.497716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.535849+0000","last_change":"2023-10-16T11:19:23.563754+0000","last_active":"2023-10-16T11:19:24.535849+0000","last_peered":"2023-10-16T11:19:24.535849+0000","last_clean":"2023-10-16T11:19:24.535849+0000","last_became_active":"2023-10-16T11:19:23.563487+0000","last_became_peered":"2023-10-16T11:19:23.563487+0000","last_unstale":"2023-10-16T11:19:24.535849+0000","last_undegraded":"2023-10-16T11:19:24.535849+0000","last_fullsized":"2023-10-16T11:19:24.535849+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:13.814299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555196+0000","last_change":"2023-10-16T11:19:23.548276+0000","last_active":"2023-10-16T11:19:23.555196+0000","last_peered":"2023-10-16T11:19:23.555196+0000","last_clean":"2023-10-16T11:19:23.555196+0000","last_became_active":"2023-10-16T11:19:23.547177+0000","last_became_peered":"2023-10-16T11:19:23.547177+0000","last_unstale":"2023-10-16T11:19:23.555196+0000","last_undegraded":"2023-10-16T11:19:23.555196+0000","last_fullsized":"2023-10-16T11:19:23.555196+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:10.899194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536199+0000","last_change":"2023-10-16T11:19:23.549769+0000","last_active":"2023-10-16T11:19:24.536199+0000","last_peered":"2023-10-16T11:19:24.536199+0000","last_clean":"2023-10-16T11:19:24.536199+0000","last_became_active":"2023-10-16T11:19:23.549505+0000","last_became_peered":"2023-10-16T11:19:23.549505+0000","last_unstale":"2023-10-16T11:19:24.536199+0000","last_undegraded":"2023-10-16T11:19:24.536199+0000","last_fullsized":"2023-10-16T11:19:24.536199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:20.931218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536011+0000","last_change":"2023-10-16T11:19:23.547740+0000","last_active":"2023-10-16T11:19:24.536011+0000","last_peered":"2023-10-16T11:19:24.536011+0000","last_clean":"2023-10-16T11:19:24.536011+0000","last_became_active":"2023-10-16T11:19:23.546421+0000","last_became_peered":"2023-10-16T11:19:23.546421+0000","last_unstale":"2023-10-16T11:19:24.536011+0000","last_undegraded":"2023-10-16T11:19:24.536011+0000","last_fullsized":"2023-10-16T11:19:24.536011+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:27.685924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.544641+0000","last_change":"2023-10-16T11:19:23.542321+0000","last_active":"2023-10-16T11:19:23.544641+0000","last_peered":"2023-10-16T11:19:23.544641+0000","last_clean":"2023-10-16T11:19:23.544641+0000","last_became_active":"2023-10-16T11:19:23.541862+0000","last_became_peered":"2023-10-16T11:19:23.541862+0000","last_unstale":"2023-10-16T11:19:23.544641+0000","last_undegraded":"2023-10-16T11:19:23.544641+0000","last_fullsized":"2023-10-16T11:19:23.544641+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:20.905021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539849+0000","last_change":"2023-10-16T11:19:23.550209+0000","last_active":"2023-10-16T11:19:24.539849+0000","last_peered":"2023-10-16T11:19:24.539849+0000","last_clean":"2023-10-16T11:19:24.539849+0000","last_became_active":"2023-10-16T11:19:23.549849+0000","last_became_peered":"2023-10-16T11:19:23.549849+0000","last_unstale":"2023-10-16T11:19:24.539849+0000","last_undegraded":"2023-10-16T11:19:24.539849+0000","last_fullsized":"2023-10-16T11:19:24.539849+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:29.715382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773951+0000","last_change":"2023-10-16T11:19:23.546038+0000","last_active":"2023-10-16T11:19:24.773951+0000","last_peered":"2023-10-16T11:19:24.773951+0000","last_clean":"2023-10-16T11:19:24.773951+0000","last_became_active":"2023-10-16T11:19:23.545752+0000","last_became_peered":"2023-10-16T11:19:23.545752+0000","last_unstale":"2023-10-16T11:19:24.773951+0000","last_undegraded":"2023-10-16T11:19:24.773951+0000","last_fullsized":"2023-10-16T11:19:24.773951+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:13.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773725+0000","last_change":"2023-10-16T11:19:23.552048+0000","last_active":"2023-10-16T11:19:24.773725+0000","last_peered":"2023-10-16T11:19:24.773725+0000","last_clean":"2023-10-16T11:19:24.773725+0000","last_became_active":"2023-10-16T11:19:23.551676+0000","last_became_peered":"2023-10-16T11:19:23.551676+0000","last_unstale":"2023-10-16T11:19:24.773725+0000","last_undegraded":"2023-10-16T11:19:24.773725+0000","last_fullsized":"2023-10-16T11:19:24.773725+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:16.450373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539945+0000","last_change":"2023-10-16T11:19:23.548868+0000","last_active":"2023-10-16T11:19:24.539945+0000","last_peered":"2023-10-16T11:19:24.539945+0000","last_clean":"2023-10-16T11:19:24.539945+0000","last_became_active":"2023-10-16T11:19:23.548583+0000","last_became_peered":"2023-10-16T11:19:23.548583+0000","last_unstale":"2023-10-16T11:19:24.539945+0000","last_undegraded":"2023-10-16T11:19:24.539945+0000","last_fullsized":"2023-10-16T11:19:24.539945+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:02.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540107+0000","last_change":"2023-10-16T11:19:23.546857+0000","last_active":"2023-10-16T11:19:24.540107+0000","last_peered":"2023-10-16T11:19:24.540107+0000","last_clean":"2023-10-16T11:19:24.540107+0000","last_became_active":"2023-10-16T11:19:23.546592+0000","last_became_peered":"2023-10-16T11:19:23.546592+0000","last_unstale":"2023-10-16T11:19:24.540107+0000","last_undegraded":"2023-10-16T11:19:24.540107+0000","last_fullsized":"2023-10-16T11:19:24.540107+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553425+0000","last_change":"2023-10-16T11:19:23.546460+0000","last_active":"2023-10-16T11:19:23.553425+0000","last_peered":"2023-10-16T11:19:23.553425+0000","last_clean":"2023-10-16T11:19:23.553425+0000","last_became_active":"2023-10-16T11:19:23.546187+0000","last_became_peered":"2023-10-16T11:19:23.546187+0000","last_unstale":"2023-10-16T11:19:23.553425+0000","last_undegraded":"2023-10-16T11:19:23.553425+0000","last_fullsized":"2023-10-16T11:19:23.553425+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.562585+0000","last_change":"2023-10-16T11:19:23.559781+0000","last_active":"2023-10-16T11:19:23.562585+0000","last_peered":"2023-10-16T11:19:23.562585+0000","last_clean":"2023-10-16T11:19:23.562585+0000","last_became_active":"2023-10-16T11:19:23.559451+0000","last_became_peered":"2023-10-16T11:19:23.559451+0000","last_unstale":"2023-10-16T11:19:23.562585+0000","last_undegraded":"2023-10-16T11:19:23.562585+0000","last_fullsized":"2023-10-16T11:19:23.562585+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:34:43.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.556270+0000","last_change":"2023-10-16T11:19:23.555152+0000","last_active":"2023-10-16T11:19:23.556270+0000","last_peered":"2023-10-16T11:19:23.556270+0000","last_clean":"2023-10-16T11:19:23.556270+0000","last_became_active":"2023-10-16T11:19:23.554747+0000","last_became_peered":"2023-10-16T11:19:23.554747+0000","last_unstale":"2023-10-16T11:19:23.556270+0000","last_undegraded":"2023-10-16T11:19:23.556270+0000","last_fullsized":"2023-10-16T11:19:23.556270+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.566220+0000","last_change":"2023-10-16T11:19:23.565127+0000","last_active":"2023-10-16T11:19:23.566220+0000","last_peered":"2023-10-16T11:19:23.566220+0000","last_clean":"2023-10-16T11:19:23.566220+0000","last_became_active":"2023-10-16T11:19:23.564852+0000","last_became_peered":"2023-10-16T11:19:23.564852+0000","last_unstale":"2023-10-16T11:19:23.566220+0000","last_undegraded":"2023-10-16T11:19:23.566220+0000","last_fullsized":"2023-10-16T11:19:23.566220+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:04.497716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539767+0000","last_change":"2023-10-16T11:19:23.547762+0000","last_active":"2023-10-16T11:19:24.539767+0000","last_peered":"2023-10-16T11:19:24.539767+0000","last_clean":"2023-10-16T11:19:24.539767+0000","last_became_active":"2023-10-16T11:19:23.547353+0000","last_became_peered":"2023-10-16T11:19:23.547353+0000","last_unstale":"2023-10-16T11:19:24.539767+0000","last_undegraded":"2023-10-16T11:19:24.539767+0000","last_fullsized":"2023-10-16T11:19:24.539767+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536330+0000","last_change":"2023-10-16T11:19:23.542878+0000","last_active":"2023-10-16T11:19:24.536330+0000","last_peered":"2023-10-16T11:19:24.536330+0000","last_clean":"2023-10-16T11:19:24.536330+0000","last_became_active":"2023-10-16T11:19:23.542593+0000","last_became_peered":"2023-10-16T11:19:23.542593+0000","last_unstale":"2023-10-16T11:19:24.536330+0000","last_undegraded":"2023-10-16T11:19:24.536330+0000","last_fullsized":"2023-10-16T11:19:24.536330+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:39.898154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.545664+0000","last_change":"2023-10-16T11:19:23.544178+0000","last_active":"2023-10-16T11:19:23.545664+0000","last_peered":"2023-10-16T11:19:23.545664+0000","last_clean":"2023-10-16T11:19:23.545664+0000","last_became_active":"2023-10-16T11:19:23.543898+0000","last_became_peered":"2023-10-16T11:19:23.543898+0000","last_unstale":"2023-10-16T11:19:23.545664+0000","last_undegraded":"2023-10-16T11:19:23.545664+0000","last_fullsized":"2023-10-16T11:19:23.545664+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:58:47.638973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540529+0000","last_change":"2023-10-16T11:19:23.551775+0000","last_active":"2023-10-16T11:19:24.540529+0000","last_peered":"2023-10-16T11:19:24.540529+0000","last_clean":"2023-10-16T11:19:24.540529+0000","last_became_active":"2023-10-16T11:19:23.551490+0000","last_became_peered":"2023-10-16T11:19:23.551490+0000","last_unstale":"2023-10-16T11:19:24.540529+0000","last_undegraded":"2023-10-16T11:19:24.540529+0000","last_fullsized":"2023-10-16T11:19:24.540529+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.566447+0000","last_change":"2023-10-16T11:19:22.538829+0000","last_active":"2023-10-16T11:19:23.566447+0000","last_peered":"2023-10-16T11:19:23.566447+0000","last_clean":"2023-10-16T11:19:23.566447+0000","last_became_active":"2023-10-16T11:19:22.538537+0000","last_became_peered":"2023-10-16T11:19:22.538537+0000","last_unstale":"2023-10-16T11:19:23.566447+0000","last_undegraded":"2023-10-16T11:19:23.566447+0000","last_fullsized":"2023-10-16T11:19:23.566447+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:09.892801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773981+0000","last_change":"2023-10-16T11:19:23.559791+0000","last_active":"2023-10-16T11:19:24.773981+0000","last_peered":"2023-10-16T11:19:24.773981+0000","last_clean":"2023-10-16T11:19:24.773981+0000","last_became_active":"2023-10-16T11:19:23.559513+0000","last_became_peered":"2023-10-16T11:19:23.559513+0000","last_unstale":"2023-10-16T11:19:24.773981+0000","last_undegraded":"2023-10-16T11:19:24.773981+0000","last_fullsized":"2023-10-16T11:19:24.773981+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:39.986537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540469+0000","last_change":"2023-10-16T11:19:22.536812+0000","last_active":"2023-10-16T11:19:24.540469+0000","last_peered":"2023-10-16T11:19:24.540469+0000","last_clean":"2023-10-16T11:19:24.540469+0000","last_became_active":"2023-10-16T11:19:22.536545+0000","last_became_peered":"2023-10-16T11:19:22.536545+0000","last_unstale":"2023-10-16T11:19:24.540469+0000","last_undegraded":"2023-10-16T11:19:24.540469+0000","last_fullsized":"2023-10-16T11:19:24.540469+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553472+0000","last_change":"2023-10-16T11:19:23.546849+0000","last_active":"2023-10-16T11:19:23.553472+0000","last_peered":"2023-10-16T11:19:23.553472+0000","last_clean":"2023-10-16T11:19:23.553472+0000","last_became_active":"2023-10-16T11:19:23.546575+0000","last_became_peered":"2023-10-16T11:19:23.546575+0000","last_unstale":"2023-10-16T11:19:23.553472+0000","last_undegraded":"2023-10-16T11:19:23.553472+0000","last_fullsized":"2023-10-16T11:19:23.553472+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:41:37.262229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564933+0000","last_change":"2023-10-16T11:19:22.533919+0000","last_active":"2023-10-16T11:19:23.564933+0000","last_peered":"2023-10-16T11:19:23.564933+0000","last_clean":"2023-10-16T11:19:23.564933+0000","last_became_active":"2023-10-16T11:19:22.533695+0000","last_became_peered":"2023-10-16T11:19:22.533695+0000","last_unstale":"2023-10-16T11:19:23.564933+0000","last_undegraded":"2023-10-16T11:19:23.564933+0000","last_fullsized":"2023-10-16T11:19:23.564933+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536127+0000","last_change":"2023-10-16T11:19:23.552923+0000","last_active":"2023-10-16T11:19:24.536127+0000","last_peered":"2023-10-16T11:19:24.536127+0000","last_clean":"2023-10-16T11:19:24.536127+0000","last_became_active":"2023-10-16T11:19:23.552685+0000","last_became_peered":"2023-10-16T11:19:23.552685+0000","last_unstale":"2023-10-16T11:19:24.536127+0000","last_undegraded":"2023-10-16T11:19:24.536127+0000","last_fullsized":"2023-10-16T11:19:24.536127+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:13.492062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540349+0000","last_change":"2023-10-16T11:19:22.535919+0000","last_active":"2023-10-16T11:19:24.540349+0000","last_peered":"2023-10-16T11:19:24.540349+0000","last_clean":"2023-10-16T11:19:24.540349+0000","last_became_active":"2023-10-16T11:19:22.535612+0000","last_became_peered":"2023-10-16T11:19:22.535612+0000","last_unstale":"2023-10-16T11:19:24.540349+0000","last_undegraded":"2023-10-16T11:19:24.540349+0000","last_fullsized":"2023-10-16T11:19:24.540349+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:54.628429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539859+0000","last_change":"2023-10-16T11:19:23.546200+0000","last_active":"2023-10-16T11:19:24.539859+0000","last_peered":"2023-10-16T11:19:24.539859+0000","last_clean":"2023-10-16T11:19:24.539859+0000","last_became_active":"2023-10-16T11:19:23.545867+0000","last_became_peered":"2023-10-16T11:19:23.545867+0000","last_unstale":"2023-10-16T11:19:24.539859+0000","last_undegraded":"2023-10-16T11:19:24.539859+0000","last_fullsized":"2023-10-16T11:19:24.539859+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.670688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.566380+0000","last_change":"2023-10-16T11:19:22.543956+0000","last_active":"2023-10-16T11:19:23.566380+0000","last_peered":"2023-10-16T11:19:23.566380+0000","last_clean":"2023-10-16T11:19:23.566380+0000","last_became_active":"2023-10-16T11:19:22.543633+0000","last_became_peered":"2023-10-16T11:19:22.543633+0000","last_unstale":"2023-10-16T11:19:23.566380+0000","last_undegraded":"2023-10-16T11:19:23.566380+0000","last_fullsized":"2023-10-16T11:19:23.566380+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:12:53.412047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.544506+0000","last_change":"2023-10-16T11:19:23.543708+0000","last_active":"2023-10-16T11:19:23.544506+0000","last_peered":"2023-10-16T11:19:23.544506+0000","last_clean":"2023-10-16T11:19:23.544506+0000","last_became_active":"2023-10-16T11:19:23.543449+0000","last_became_peered":"2023-10-16T11:19:23.543449+0000","last_unstale":"2023-10-16T11:19:23.544506+0000","last_undegraded":"2023-10-16T11:19:23.544506+0000","last_fullsized":"2023-10-16T11:19:23.544506+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:45.477205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.538252+0000","last_change":"2023-10-16T11:19:22.547192+0000","last_active":"2023-10-16T11:19:23.538252+0000","last_peered":"2023-10-16T11:19:23.538252+0000","last_clean":"2023-10-16T11:19:23.538252+0000","last_became_active":"2023-10-16T11:19:22.546912+0000","last_became_peered":"2023-10-16T11:19:22.546912+0000","last_unstale":"2023-10-16T11:19:23.538252+0000","last_undegraded":"2023-10-16T11:19:23.538252+0000","last_fullsized":"2023-10-16T11:19:23.538252+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:17.017260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540051+0000","last_change":"2023-10-16T11:19:23.546966+0000","last_active":"2023-10-16T11:19:24.540051+0000","last_peered":"2023-10-16T11:19:24.540051+0000","last_clean":"2023-10-16T11:19:24.540051+0000","last_became_active":"2023-10-16T11:19:23.546685+0000","last_became_peered":"2023-10-16T11:19:23.546685+0000","last_unstale":"2023-10-16T11:19:24.540051+0000","last_undegraded":"2023-10-16T11:19:24.540051+0000","last_fullsized":"2023-10-16T11:19:24.540051+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:13.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773537+0000","last_change":"2023-10-16T11:19:22.534080+0000","last_active":"2023-10-16T11:19:24.773537+0000","last_peered":"2023-10-16T11:19:24.773537+0000","last_clean":"2023-10-16T11:19:24.773537+0000","last_became_active":"2023-10-16T11:19:22.533693+0000","last_became_peered":"2023-10-16T11:19:22.533693+0000","last_unstale":"2023-10-16T11:19:24.773537+0000","last_undegraded":"2023-10-16T11:19:24.773537+0000","last_fullsized":"2023-10-16T11:19:24.773537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:57.023698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540224+0000","last_change":"2023-10-16T11:19:23.553070+0000","last_active":"2023-10-16T11:19:24.540224+0000","last_peered":"2023-10-16T11:19:24.540224+0000","last_clean":"2023-10-16T11:19:24.540224+0000","last_became_active":"2023-10-16T11:19:23.552761+0000","last_became_peered":"2023-10-16T11:19:23.552761+0000","last_unstale":"2023-10-16T11:19:24.540224+0000","last_undegraded":"2023-10-16T11:19:24.540224+0000","last_fullsized":"2023-10-16T11:19:24.540224+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:36.373122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540399+0000","last_change":"2023-10-16T11:19:22.545557+0000","last_active":"2023-10-16T11:19:23.540399+0000","last_peered":"2023-10-16T11:19:23.540399+0000","last_clean":"2023-10-16T11:19:23.540399+0000","last_became_active":"2023-10-16T11:19:22.545253+0000","last_became_peered":"2023-10-16T11:19:22.545253+0000","last_unstale":"2023-10-16T11:19:23.540399+0000","last_undegraded":"2023-10-16T11:19:23.540399+0000","last_fullsized":"2023-10-16T11:19:23.540399+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:11.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.561454+0000","last_change":"2023-10-16T11:19:23.560046+0000","last_active":"2023-10-16T11:19:23.561454+0000","last_peered":"2023-10-16T11:19:23.561454+0000","last_clean":"2023-10-16T11:19:23.561454+0000","last_became_active":"2023-10-16T11:19:23.559718+0000","last_became_peered":"2023-10-16T11:19:23.559718+0000","last_unstale":"2023-10-16T11:19:23.561454+0000","last_undegraded":"2023-10-16T11:19:23.561454+0000","last_fullsized":"2023-10-16T11:19:23.561454+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:46.256661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773995+0000","last_change":"2023-10-16T11:19:22.535771+0000","last_active":"2023-10-16T11:19:24.773995+0000","last_peered":"2023-10-16T11:19:24.773995+0000","last_clean":"2023-10-16T11:19:24.773995+0000","last_became_active":"2023-10-16T11:19:22.535486+0000","last_became_peered":"2023-10-16T11:19:22.535486+0000","last_unstale":"2023-10-16T11:19:24.773995+0000","last_undegraded":"2023-10-16T11:19:24.773995+0000","last_fullsized":"2023-10-16T11:19:24.773995+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:25.214100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540419+0000","last_change":"2023-10-16T11:19:23.563660+0000","last_active":"2023-10-16T11:19:24.540419+0000","last_peered":"2023-10-16T11:19:24.540419+0000","last_clean":"2023-10-16T11:19:24.540419+0000","last_became_active":"2023-10-16T11:19:23.563402+0000","last_became_peered":"2023-10-16T11:19:23.563402+0000","last_unstale":"2023-10-16T11:19:24.540419+0000","last_undegraded":"2023-10-16T11:19:24.540419+0000","last_fullsized":"2023-10-16T11:19:24.540419+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:41.606189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536228+0000","last_change":"2023-10-16T11:19:22.544432+0000","last_active":"2023-10-16T11:19:24.536228+0000","last_peered":"2023-10-16T11:19:24.536228+0000","last_clean":"2023-10-16T11:19:24.536228+0000","last_became_active":"2023-10-16T11:19:22.543046+0000","last_became_peered":"2023-10-16T11:19:22.543046+0000","last_unstale":"2023-10-16T11:19:24.536228+0000","last_undegraded":"2023-10-16T11:19:24.536228+0000","last_fullsized":"2023-10-16T11:19:24.536228+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:08.672505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.554606+0000","last_change":"2023-10-16T11:19:23.548103+0000","last_active":"2023-10-16T11:19:23.554606+0000","last_peered":"2023-10-16T11:19:23.554606+0000","last_clean":"2023-10-16T11:19:23.554606+0000","last_became_active":"2023-10-16T11:19:23.547665+0000","last_became_peered":"2023-10-16T11:19:23.547665+0000","last_unstale":"2023-10-16T11:19:23.554606+0000","last_undegraded":"2023-10-16T11:19:23.554606+0000","last_fullsized":"2023-10-16T11:19:23.554606+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:08.885647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"215'6","reported_seq":18,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540455+0000","last_change":"2023-10-16T11:19:22.532339+0000","last_active":"2023-10-16T11:19:24.540455+0000","last_peered":"2023-10-16T11:19:24.540455+0000","last_clean":"2023-10-16T11:19:24.540455+0000","last_became_active":"2023-10-16T11:19:22.532053+0000","last_became_peered":"2023-10-16T11:19:22.532053+0000","last_unstale":"2023-10-16T11:19:24.540455+0000","last_undegraded":"2023-10-16T11:19:24.540455+0000","last_fullsized":"2023-10-16T11:19:24.540455+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:21.190168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.552817+0000","last_change":"2023-10-16T11:19:23.550398+0000","last_active":"2023-10-16T11:19:23.552817+0000","last_peered":"2023-10-16T11:19:23.552817+0000","last_clean":"2023-10-16T11:19:23.552817+0000","last_became_active":"2023-10-16T11:19:23.545907+0000","last_became_peered":"2023-10-16T11:19:23.545907+0000","last_unstale":"2023-10-16T11:19:23.552817+0000","last_undegraded":"2023-10-16T11:19:23.552817+0000","last_fullsized":"2023-10-16T11:19:23.552817+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:10.743840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773746+0000","last_change":"2023-10-16T11:19:22.544032+0000","last_active":"2023-10-16T11:19:24.773746+0000","last_peered":"2023-10-16T11:19:24.773746+0000","last_clean":"2023-10-16T11:19:24.773746+0000","last_became_active":"2023-10-16T11:19:22.543713+0000","last_became_peered":"2023-10-16T11:19:22.543713+0000","last_unstale":"2023-10-16T11:19:24.773746+0000","last_undegraded":"2023-10-16T11:19:24.773746+0000","last_fullsized":"2023-10-16T11:19:24.773746+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:01.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536202+0000","last_change":"2023-10-16T11:19:23.552779+0000","last_active":"2023-10-16T11:19:24.536202+0000","last_peered":"2023-10-16T11:19:24.536202+0000","last_clean":"2023-10-16T11:19:24.536202+0000","last_became_active":"2023-10-16T11:19:23.552491+0000","last_became_peered":"2023-10-16T11:19:23.552491+0000","last_unstale":"2023-10-16T11:19:24.536202+0000","last_undegraded":"2023-10-16T11:19:24.536202+0000","last_fullsized":"2023-10-16T11:19:24.536202+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:02.969587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773976+0000","last_change":"2023-10-16T11:19:22.538147+0000","last_active":"2023-10-16T11:19:24.773976+0000","last_peered":"2023-10-16T11:19:24.773976+0000","last_clean":"2023-10-16T11:19:24.773976+0000","last_became_active":"2023-10-16T11:19:22.537850+0000","last_became_peered":"2023-10-16T11:19:22.537850+0000","last_unstale":"2023-10-16T11:19:24.773976+0000","last_undegraded":"2023-10-16T11:19:24.773976+0000","last_fullsized":"2023-10-16T11:19:24.773976+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:44.794069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564185+0000","last_change":"2023-10-16T11:19:23.563603+0000","last_active":"2023-10-16T11:19:23.564185+0000","last_peered":"2023-10-16T11:19:23.564185+0000","last_clean":"2023-10-16T11:19:23.564185+0000","last_became_active":"2023-10-16T11:19:23.563305+0000","last_became_peered":"2023-10-16T11:19:23.563305+0000","last_unstale":"2023-10-16T11:19:23.564185+0000","last_undegraded":"2023-10-16T11:19:23.564185+0000","last_fullsized":"2023-10-16T11:19:23.564185+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:04.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.535827+0000","last_change":"2023-10-16T11:19:22.538008+0000","last_active":"2023-10-16T11:19:24.535827+0000","last_peered":"2023-10-16T11:19:24.535827+0000","last_clean":"2023-10-16T11:19:24.535827+0000","last_became_active":"2023-10-16T11:19:22.537752+0000","last_became_peered":"2023-10-16T11:19:22.537752+0000","last_unstale":"2023-10-16T11:19:24.535827+0000","last_undegraded":"2023-10-16T11:19:24.535827+0000","last_fullsized":"2023-10-16T11:19:24.535827+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:20.859898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555425+0000","last_change":"2023-10-16T11:19:23.552950+0000","last_active":"2023-10-16T11:19:23.555425+0000","last_peered":"2023-10-16T11:19:23.555425+0000","last_clean":"2023-10-16T11:19:23.555425+0000","last_became_active":"2023-10-16T11:19:23.552686+0000","last_became_peered":"2023-10-16T11:19:23.552686+0000","last_unstale":"2023-10-16T11:19:23.555425+0000","last_undegraded":"2023-10-16T11:19:23.555425+0000","last_fullsized":"2023-10-16T11:19:23.555425+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:38:30.386843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":19,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.565891+0000","last_change":"2023-10-16T11:19:22.534093+0000","last_active":"2023-10-16T11:19:23.565891+0000","last_peered":"2023-10-16T11:19:23.565891+0000","last_clean":"2023-10-16T11:19:23.565891+0000","last_became_active":"2023-10-16T11:19:22.533637+0000","last_became_peered":"2023-10-16T11:19:22.533637+0000","last_unstale":"2023-10-16T11:19:23.565891+0000","last_undegraded":"2023-10-16T11:19:23.565891+0000","last_fullsized":"2023-10-16T11:19:23.565891+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540565+0000","last_change":"2023-10-16T11:19:23.547530+0000","last_active":"2023-10-16T11:19:24.540565+0000","last_peered":"2023-10-16T11:19:24.540565+0000","last_clean":"2023-10-16T11:19:24.540565+0000","last_became_active":"2023-10-16T11:19:23.547255+0000","last_became_peered":"2023-10-16T11:19:23.547255+0000","last_unstale":"2023-10-16T11:19:24.540565+0000","last_undegraded":"2023-10-16T11:19:24.540565+0000","last_fullsized":"2023-10-16T11:19:24.540565+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.557798+0000","last_change":"2023-10-16T11:19:22.545027+0000","last_active":"2023-10-16T11:19:23.557798+0000","last_peered":"2023-10-16T11:19:23.557798+0000","last_clean":"2023-10-16T11:19:23.557798+0000","last_became_active":"2023-10-16T11:19:22.544756+0000","last_became_peered":"2023-10-16T11:19:22.544756+0000","last_unstale":"2023-10-16T11:19:23.557798+0000","last_undegraded":"2023-10-16T11:19:23.557798+0000","last_fullsized":"2023-10-16T11:19:23.557798+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:23.693322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540221+0000","last_change":"2023-10-16T11:19:23.546218+0000","last_active":"2023-10-16T11:19:24.540221+0000","last_peered":"2023-10-16T11:19:24.540221+0000","last_clean":"2023-10-16T11:19:24.540221+0000","last_became_active":"2023-10-16T11:19:23.545896+0000","last_became_peered":"2023-10-16T11:19:23.545896+0000","last_unstale":"2023-10-16T11:19:24.540221+0000","last_undegraded":"2023-10-16T11:19:24.540221+0000","last_fullsized":"2023-10-16T11:19:24.540221+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540640+0000","last_change":"2023-10-16T11:19:22.539558+0000","last_active":"2023-10-16T11:19:24.540640+0000","last_peered":"2023-10-16T11:19:24.540640+0000","last_clean":"2023-10-16T11:19:24.540640+0000","last_became_active":"2023-10-16T11:19:22.539284+0000","last_became_peered":"2023-10-16T11:19:22.539284+0000","last_unstale":"2023-10-16T11:19:24.540640+0000","last_undegraded":"2023-10-16T11:19:24.540640+0000","last_fullsized":"2023-10-16T11:19:24.540640+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.556031+0000","last_change":"2023-10-16T11:19:23.553745+0000","last_active":"2023-10-16T11:19:23.556031+0000","last_peered":"2023-10-16T11:19:23.556031+0000","last_clean":"2023-10-16T11:19:23.556031+0000","last_became_active":"2023-10-16T11:19:23.553462+0000","last_became_peered":"2023-10-16T11:19:23.553462+0000","last_unstale":"2023-10-16T11:19:23.556031+0000","last_undegraded":"2023-10-16T11:19:23.556031+0000","last_fullsized":"2023-10-16T11:19:23.556031+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536257+0000","last_change":"2023-10-16T11:19:22.533601+0000","last_active":"2023-10-16T11:19:24.536257+0000","last_peered":"2023-10-16T11:19:24.536257+0000","last_clean":"2023-10-16T11:19:24.536257+0000","last_became_active":"2023-10-16T11:19:22.533374+0000","last_became_peered":"2023-10-16T11:19:22.533374+0000","last_unstale":"2023-10-16T11:19:24.536257+0000","last_undegraded":"2023-10-16T11:19:24.536257+0000","last_fullsized":"2023-10-16T11:19:24.536257+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:52.564361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536152+0000","last_change":"2023-10-16T11:19:23.545760+0000","last_active":"2023-10-16T11:19:24.536152+0000","last_peered":"2023-10-16T11:19:24.536152+0000","last_clean":"2023-10-16T11:19:24.536152+0000","last_became_active":"2023-10-16T11:19:23.545499+0000","last_became_peered":"2023-10-16T11:19:23.545499+0000","last_unstale":"2023-10-16T11:19:24.536152+0000","last_undegraded":"2023-10-16T11:19:24.536152+0000","last_fullsized":"2023-10-16T11:19:24.536152+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:32.701590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.548366+0000","last_change":"2023-10-16T11:19:22.545030+0000","last_active":"2023-10-16T11:19:23.548366+0000","last_peered":"2023-10-16T11:19:23.548366+0000","last_clean":"2023-10-16T11:19:23.548366+0000","last_became_active":"2023-10-16T11:19:22.544773+0000","last_became_peered":"2023-10-16T11:19:22.544773+0000","last_unstale":"2023-10-16T11:19:23.548366+0000","last_undegraded":"2023-10-16T11:19:23.548366+0000","last_fullsized":"2023-10-16T11:19:23.548366+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:22.946353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536520+0000","last_change":"2023-10-16T11:19:23.543334+0000","last_active":"2023-10-16T11:19:24.536520+0000","last_peered":"2023-10-16T11:19:24.536520+0000","last_clean":"2023-10-16T11:19:24.536520+0000","last_became_active":"2023-10-16T11:19:23.542527+0000","last_became_peered":"2023-10-16T11:19:23.542527+0000","last_unstale":"2023-10-16T11:19:24.536520+0000","last_undegraded":"2023-10-16T11:19:24.536520+0000","last_fullsized":"2023-10-16T11:19:24.536520+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:59.083108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536104+0000","last_change":"2023-10-16T11:19:22.536403+0000","last_active":"2023-10-16T11:19:24.536104+0000","last_peered":"2023-10-16T11:19:24.536104+0000","last_clean":"2023-10-16T11:19:24.536104+0000","last_became_active":"2023-10-16T11:19:22.536132+0000","last_became_peered":"2023-10-16T11:19:22.536132+0000","last_unstale":"2023-10-16T11:19:24.536104+0000","last_undegraded":"2023-10-16T11:19:24.536104+0000","last_fullsized":"2023-10-16T11:19:24.536104+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:53.563487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.556247+0000","last_change":"2023-10-16T11:19:23.554949+0000","last_active":"2023-10-16T11:19:23.556247+0000","last_peered":"2023-10-16T11:19:23.556247+0000","last_clean":"2023-10-16T11:19:23.556247+0000","last_became_active":"2023-10-16T11:19:23.554300+0000","last_became_peered":"2023-10-16T11:19:23.554300+0000","last_unstale":"2023-10-16T11:19:23.556247+0000","last_undegraded":"2023-10-16T11:19:23.556247+0000","last_fullsized":"2023-10-16T11:19:23.556247+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:06.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540159+0000","last_change":"2023-10-16T11:19:22.542021+0000","last_active":"2023-10-16T11:19:23.540159+0000","last_peered":"2023-10-16T11:19:23.540159+0000","last_clean":"2023-10-16T11:19:23.540159+0000","last_became_active":"2023-10-16T11:19:22.541718+0000","last_became_peered":"2023-10-16T11:19:22.541718+0000","last_unstale":"2023-10-16T11:19:23.540159+0000","last_undegraded":"2023-10-16T11:19:23.540159+0000","last_fullsized":"2023-10-16T11:19:23.540159+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:12.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539246+0000","last_change":"2023-10-16T11:19:23.548561+0000","last_active":"2023-10-16T11:19:24.539246+0000","last_peered":"2023-10-16T11:19:24.539246+0000","last_clean":"2023-10-16T11:19:24.539246+0000","last_became_active":"2023-10-16T11:19:23.548310+0000","last_became_peered":"2023-10-16T11:19:23.548310+0000","last_unstale":"2023-10-16T11:19:24.539246+0000","last_undegraded":"2023-10-16T11:19:24.539246+0000","last_fullsized":"2023-10-16T11:19:24.539246+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540497+0000","last_change":"2023-10-16T11:19:22.532222+0000","last_active":"2023-10-16T11:19:23.540497+0000","last_peered":"2023-10-16T11:19:23.540497+0000","last_clean":"2023-10-16T11:19:23.540497+0000","last_became_active":"2023-10-16T11:19:22.531939+0000","last_became_peered":"2023-10-16T11:19:22.531939+0000","last_unstale":"2023-10-16T11:19:23.540497+0000","last_undegraded":"2023-10-16T11:19:23.540497+0000","last_fullsized":"2023-10-16T11:19:23.540497+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773737+0000","last_change":"2023-10-16T11:19:23.563941+0000","last_active":"2023-10-16T11:19:24.773737+0000","last_peered":"2023-10-16T11:19:24.773737+0000","last_clean":"2023-10-16T11:19:24.773737+0000","last_became_active":"2023-10-16T11:19:23.563666+0000","last_became_peered":"2023-10-16T11:19:23.563666+0000","last_unstale":"2023-10-16T11:19:24.773737+0000","last_undegraded":"2023-10-16T11:19:24.773737+0000","last_fullsized":"2023-10-16T11:19:24.773737+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:43.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.539184+0000","last_change":"2023-10-16T11:19:22.547829+0000","last_active":"2023-10-16T11:19:23.539184+0000","last_peered":"2023-10-16T11:19:23.539184+0000","last_clean":"2023-10-16T11:19:23.539184+0000","last_became_active":"2023-10-16T11:19:22.547563+0000","last_became_peered":"2023-10-16T11:19:22.547563+0000","last_unstale":"2023-10-16T11:19:23.539184+0000","last_undegraded":"2023-10-16T11:19:23.539184+0000","last_fullsized":"2023-10-16T11:19:23.539184+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:15.819032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.565522+0000","last_change":"2023-10-16T11:19:23.561790+0000","last_active":"2023-10-16T11:19:23.565522+0000","last_peered":"2023-10-16T11:19:23.565522+0000","last_clean":"2023-10-16T11:19:23.565522+0000","last_became_active":"2023-10-16T11:19:23.561531+0000","last_became_peered":"2023-10-16T11:19:23.561531+0000","last_unstale":"2023-10-16T11:19:23.565522+0000","last_undegraded":"2023-10-16T11:19:23.565522+0000","last_fullsized":"2023-10-16T11:19:23.565522+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:29:06.096292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.565979+0000","last_change":"2023-10-16T11:19:22.537256+0000","last_active":"2023-10-16T11:19:23.565979+0000","last_peered":"2023-10-16T11:19:23.565979+0000","last_clean":"2023-10-16T11:19:23.565979+0000","last_became_active":"2023-10-16T11:19:22.536996+0000","last_became_peered":"2023-10-16T11:19:22.536996+0000","last_unstale":"2023-10-16T11:19:23.565979+0000","last_undegraded":"2023-10-16T11:19:23.565979+0000","last_fullsized":"2023-10-16T11:19:23.565979+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:22.462616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564664+0000","last_change":"2023-10-16T11:19:23.564018+0000","last_active":"2023-10-16T11:19:23.564664+0000","last_peered":"2023-10-16T11:19:23.564664+0000","last_clean":"2023-10-16T11:19:23.564664+0000","last_became_active":"2023-10-16T11:19:23.563738+0000","last_became_peered":"2023-10-16T11:19:23.563738+0000","last_unstale":"2023-10-16T11:19:23.564664+0000","last_undegraded":"2023-10-16T11:19:23.564664+0000","last_fullsized":"2023-10-16T11:19:23.564664+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:29:06.096292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539759+0000","last_change":"2023-10-16T11:19:22.537758+0000","last_active":"2023-10-16T11:19:24.539759+0000","last_peered":"2023-10-16T11:19:24.539759+0000","last_clean":"2023-10-16T11:19:24.539759+0000","last_became_active":"2023-10-16T11:19:22.537201+0000","last_became_peered":"2023-10-16T11:19:22.537201+0000","last_unstale":"2023-10-16T11:19:24.539759+0000","last_undegraded":"2023-10-16T11:19:24.539759+0000","last_fullsized":"2023-10-16T11:19:24.539759+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:26.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539580+0000","last_change":"2023-10-16T11:19:23.556539+0000","last_active":"2023-10-16T11:19:24.539580+0000","last_peered":"2023-10-16T11:19:24.539580+0000","last_clean":"2023-10-16T11:19:24.539580+0000","last_became_active":"2023-10-16T11:19:23.556209+0000","last_became_peered":"2023-10-16T11:19:23.556209+0000","last_unstale":"2023-10-16T11:19:24.539580+0000","last_undegraded":"2023-10-16T11:19:24.539580+0000","last_fullsized":"2023-10-16T11:19:24.539580+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540930+0000","last_change":"2023-10-16T11:19:22.543844+0000","last_active":"2023-10-16T11:19:23.540930+0000","last_peered":"2023-10-16T11:19:23.540930+0000","last_clean":"2023-10-16T11:19:23.540930+0000","last_became_active":"2023-10-16T11:19:22.543538+0000","last_became_peered":"2023-10-16T11:19:22.543538+0000","last_unstale":"2023-10-16T11:19:23.540930+0000","last_undegraded":"2023-10-16T11:19:23.540930+0000","last_fullsized":"2023-10-16T11:19:23.540930+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:00.946603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.774005+0000","last_change":"2023-10-16T11:19:23.546472+0000","last_active":"2023-10-16T11:19:24.774005+0000","last_peered":"2023-10-16T11:19:24.774005+0000","last_clean":"2023-10-16T11:19:24.774005+0000","last_became_active":"2023-10-16T11:19:23.546160+0000","last_became_peered":"2023-10-16T11:19:23.546160+0000","last_unstale":"2023-10-16T11:19:24.774005+0000","last_undegraded":"2023-10-16T11:19:24.774005+0000","last_fullsized":"2023-10-16T11:19:24.774005+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:06.395224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.548518+0000","last_change":"2023-10-16T11:19:22.537543+0000","last_active":"2023-10-16T11:19:23.548518+0000","last_peered":"2023-10-16T11:19:23.548518+0000","last_clean":"2023-10-16T11:19:23.548518+0000","last_became_active":"2023-10-16T11:19:22.537050+0000","last_became_peered":"2023-10-16T11:19:22.537050+0000","last_unstale":"2023-10-16T11:19:23.548518+0000","last_undegraded":"2023-10-16T11:19:23.548518+0000","last_fullsized":"2023-10-16T11:19:23.548518+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:03.218818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553445+0000","last_change":"2023-10-16T11:19:23.546380+0000","last_active":"2023-10-16T11:19:23.553445+0000","last_peered":"2023-10-16T11:19:23.553445+0000","last_clean":"2023-10-16T11:19:23.553445+0000","last_became_active":"2023-10-16T11:19:23.546098+0000","last_became_peered":"2023-10-16T11:19:23.546098+0000","last_unstale":"2023-10-16T11:19:23.553445+0000","last_undegraded":"2023-10-16T11:19:23.553445+0000","last_fullsized":"2023-10-16T11:19:23.553445+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:06.791712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539990+0000","last_change":"2023-10-16T11:19:22.543881+0000","last_active":"2023-10-16T11:19:24.539990+0000","last_peered":"2023-10-16T11:19:24.539990+0000","last_clean":"2023-10-16T11:19:24.539990+0000","last_became_active":"2023-10-16T11:19:22.543647+0000","last_became_peered":"2023-10-16T11:19:22.543647+0000","last_unstale":"2023-10-16T11:19:24.539990+0000","last_undegraded":"2023-10-16T11:19:24.539990+0000","last_fullsized":"2023-10-16T11:19:24.539990+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:41.755970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564603+0000","last_change":"2023-10-16T11:19:23.558505+0000","last_active":"2023-10-16T11:19:23.564603+0000","last_peered":"2023-10-16T11:19:23.564603+0000","last_clean":"2023-10-16T11:19:23.564603+0000","last_became_active":"2023-10-16T11:19:23.558188+0000","last_became_peered":"2023-10-16T11:19:23.558188+0000","last_unstale":"2023-10-16T11:19:23.564603+0000","last_undegraded":"2023-10-16T11:19:23.564603+0000","last_fullsized":"2023-10-16T11:19:23.564603+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:35.177289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540372+0000","last_change":"2023-10-16T11:19:22.544852+0000","last_active":"2023-10-16T11:19:23.540372+0000","last_peered":"2023-10-16T11:19:23.540372+0000","last_clean":"2023-10-16T11:19:23.540372+0000","last_became_active":"2023-10-16T11:19:22.544565+0000","last_became_peered":"2023-10-16T11:19:22.544565+0000","last_unstale":"2023-10-16T11:19:23.540372+0000","last_undegraded":"2023-10-16T11:19:23.540372+0000","last_fullsized":"2023-10-16T11:19:23.540372+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:07.144566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564170+0000","last_change":"2023-10-16T11:19:23.563504+0000","last_active":"2023-10-16T11:19:23.564170+0000","last_peered":"2023-10-16T11:19:23.564170+0000","last_clean":"2023-10-16T11:19:23.564170+0000","last_became_active":"2023-10-16T11:19:23.563243+0000","last_became_peered":"2023-10-16T11:19:23.563243+0000","last_unstale":"2023-10-16T11:19:23.564170+0000","last_undegraded":"2023-10-16T11:19:23.564170+0000","last_fullsized":"2023-10-16T11:19:23.564170+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773857+0000","last_change":"2023-10-16T11:19:22.539000+0000","last_active":"2023-10-16T11:19:24.773857+0000","last_peered":"2023-10-16T11:19:24.773857+0000","last_clean":"2023-10-16T11:19:24.773857+0000","last_became_active":"2023-10-16T11:19:22.538728+0000","last_became_peered":"2023-10-16T11:19:22.538728+0000","last_unstale":"2023-10-16T11:19:24.773857+0000","last_undegraded":"2023-10-16T11:19:24.773857+0000","last_fullsized":"2023-10-16T11:19:24.773857+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:03.944394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539267+0000","last_change":"2023-10-16T11:19:23.563089+0000","last_active":"2023-10-16T11:19:24.539267+0000","last_peered":"2023-10-16T11:19:24.539267+0000","last_clean":"2023-10-16T11:19:24.539267+0000","last_became_active":"2023-10-16T11:19:23.562759+0000","last_became_peered":"2023-10-16T11:19:23.562759+0000","last_unstale":"2023-10-16T11:19:24.539267+0000","last_undegraded":"2023-10-16T11:19:24.539267+0000","last_fullsized":"2023-10-16T11:19:24.539267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.559527+0000","last_change":"2023-10-16T11:19:22.537241+0000","last_active":"2023-10-16T11:19:23.559527+0000","last_peered":"2023-10-16T11:19:23.559527+0000","last_clean":"2023-10-16T11:19:23.559527+0000","last_became_active":"2023-10-16T11:19:22.536947+0000","last_became_peered":"2023-10-16T11:19:22.536947+0000","last_unstale":"2023-10-16T11:19:23.559527+0000","last_undegraded":"2023-10-16T11:19:23.559527+0000","last_fullsized":"2023-10-16T11:19:23.559527+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555402+0000","last_change":"2023-10-16T11:19:23.551428+0000","last_active":"2023-10-16T11:19:23.555402+0000","last_peered":"2023-10-16T11:19:23.555402+0000","last_clean":"2023-10-16T11:19:23.555402+0000","last_became_active":"2023-10-16T11:19:23.551083+0000","last_became_peered":"2023-10-16T11:19:23.551083+0000","last_unstale":"2023-10-16T11:19:23.555402+0000","last_undegraded":"2023-10-16T11:19:23.555402+0000","last_fullsized":"2023-10-16T11:19:23.555402+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:32:09.096145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564650+0000","last_change":"2023-10-16T11:19:22.536415+0000","last_active":"2023-10-16T11:19:23.564650+0000","last_peered":"2023-10-16T11:19:23.564650+0000","last_clean":"2023-10-16T11:19:23.564650+0000","last_became_active":"2023-10-16T11:19:22.536117+0000","last_became_peered":"2023-10-16T11:19:22.536117+0000","last_unstale":"2023-10-16T11:19:23.564650+0000","last_undegraded":"2023-10-16T11:19:23.564650+0000","last_fullsized":"2023-10-16T11:19:23.564650+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:07.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"206'348","reported_seq":1209,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540937+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:19:23.540937+0000","last_peered":"2023-10-16T11:19:23.540937+0000","last_clean":"2023-10-16T11:19:23.540937+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:19:23.540937+0000","last_undegraded":"2023-10-16T11:19:23.540937+0000","last_fullsized":"2023-10-16T11:19:23.540937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543279,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52472,"kb_used_data":3236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319368,"statfs":{"total":96636764160,"available":96583032832,"internally_reserved":0,"allocated":3313664,"data_stored":1979940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.27800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67500000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42399999999999999,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.76300000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.79900000000000004,"15min":0.86299999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.307},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.002},"last":0.80300000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33100000000000002},"max":{"1min":8.1950000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.83699999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.871},"last":0.81899999999999995}]}]},{"osd":6,"up_from":14,"seq":60129543278,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37976,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333864,"statfs":{"total":96636764160,"available":96597876736,"internally_reserved":0,"allocated":2699264,"data_stored":1381690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.875},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.51200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.83099999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.78700000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.752,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.32100000000000001},"max":{"1min":0.79500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.91100000000000003},"last":0.59499999999999997}]}]},{"osd":1,"up_from":14,"seq":60129543278,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60196,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311644,"statfs":{"total":96636764160,"available":96575123456,"internally_reserved":0,"allocated":2703360,"data_stored":1381164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.224},"max":{"1min":0.60199999999999998,"5min":0.629,"15min":0.68700000000000006},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.434},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.62,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.55300000000000005,"5min":0.58699999999999997,"15min":0.98899999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.33600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.78300000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.0289999999999999,"15min":1.097},"last":0.57499999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":1.147},"last":0.72699999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.88,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":0.98899999999999999,"5min":1.004,"15min":1.0600000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46300000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.1060000000000001},"last":0.77700000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543279,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60772,"kb_used_data":3216,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311068,"statfs":{"total":96636764160,"available":96574533632,"internally_reserved":0,"allocated":3293184,"data_stored":1971566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39900000000000002,"15min":0.39800000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.214},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.54400000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38800000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.25800000000000001},"max":{"1min":0.59599999999999997,"5min":0.67300000000000004,"15min":0.70299999999999996},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.33700000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.58499999999999996,"5min":0.66200000000000003,"15min":0.68300000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32900000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.374}]},{"osd":4,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":0.95799999999999996,"15min":0.99199999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.85599999999999998,"5min":0.93700000000000006,"15min":1.008},"last":0.54000000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.872,"5min":1.004,"15min":1.008},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68400000000000005,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.48899999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91000000000000003,"5min":1.01,"15min":1.0449999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.81499999999999995,"5min":0.95899999999999996,"15min":1.077},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.875,"5min":0.95699999999999996,"15min":0.96599999999999997},"last":0.629}]}]},{"osd":2,"up_from":14,"seq":60129543278,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60188,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311652,"statfs":{"total":96636764160,"available":96575131648,"internally_reserved":0,"allocated":2695168,"data_stored":1381108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.63},"last":0.378}]},{"osd":1,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":0.435},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":0.48399999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.623,"15min":0.65500000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.81200000000000006},"last":0.502}]},{"osd":4,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.97599999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":1.052,"15min":1.052},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.66000000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68700000000000006,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64900000000000002}]}]},{"osd":3,"up_from":14,"seq":60129543279,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47732,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324108,"statfs":{"total":96636764160,"available":96587886592,"internally_reserved":0,"allocated":2719744,"data_stored":1388142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.442},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.36799999999999999,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.46300000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.378},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":0.60699999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.499}]},{"osd":4,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":1.034,"15min":1.0740000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67300000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.749},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.0489999999999999},"last":0.73099999999999998}]}]},{"osd":4,"up_from":14,"seq":60129543278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60220,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311620,"statfs":{"total":96636764160,"available":96575098880,"internally_reserved":0,"allocated":2727936,"data_stored":1389676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.628},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.48699999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.627},"min":{"1min":0.438,"5min":0.40400000000000003,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60399999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.879},"last":0.81100000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.69499999999999995,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.89500000000000002},"last":0.46100000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.40600000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":0.875,"15min":0.875},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.51200000000000001}]}]},{"osd":5,"up_from":14,"seq":60129543279,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42156,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329684,"statfs":{"total":96636764160,"available":96593596416,"internally_reserved":0,"allocated":2711552,"data_stored":1388542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":1.873,"15min":1.873},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.46200000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":1.821,"15min":1.821},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44700000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":1.919,"15min":1.919},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.873},"last":0.498},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.63},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":2.02,"15min":2.02},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:19:27.665 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:19:27.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.819+0000 7fcf3407b700 1 Processor -- start 2023-10-16T11:19:27.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.819+0000 7fcf3407b700 1 -- start start 2023-10-16T11:19:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf3407b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf2c15eab0 0x7fcf2c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf3407b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 0x7fcf2c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf3407b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c06ed00 con 0x7fcf2c15e6d0 2023-10-16T11:19:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf3407b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2c0d02d0 con 0x7fcf2c15eab0 2023-10-16T11:19:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf3407b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c15ad40 con 0x7fcf2c0cb0c0 2023-10-16T11:19:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf31e17700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcf2c15e6d0 0x7fcf2c15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38326/0 (socket says 172.21.15.181:38326) 2023-10-16T11:19:27.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf31e17700 1 -- 172.21.15.181:0/336667523 learned_addr learned my addr 172.21.15.181:0/336667523 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf31616700 1 --2- 172.21.15.181:0/336667523 >> v2:172.21.15.190:3300/0 conn(0x7fcf2c15eab0 0x7fcf2c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf32618700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 0x7fcf2c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335803619 0 0) 0x7fcf2c06ed00 con 0x7fcf2c15e6d0 2023-10-16T11:19:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.820+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf180035b0 con 0x7fcf2c15e6d0 2023-10-16T11:19:27.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2071732757 0 0) 0x7fcf180035b0 con 0x7fcf2c15e6d0 2023-10-16T11:19:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 msgr2=0x7fcf2c15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 0x7fcf2c15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 >> v2:172.21.15.190:3300/0 conn(0x7fcf2c15eab0 msgr2=0x7fcf2c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 --2- 172.21.15.181:0/336667523 >> v2:172.21.15.190:3300/0 conn(0x7fcf2c15eab0 0x7fcf2c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c06e0f0 con 0x7fcf2c15e6d0 2023-10-16T11:19:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fcf1c0017f0 con 0x7fcf2c15e6d0 2023-10-16T11:19:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 >> v1:172.21.15.181:6789/0 conn(0x7fcf2c15e6d0 legacy=0x7fcf2c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf18003a00 0x7fcf18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 0x7fcf18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18004d90 0x7fcf18005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c0d02d0 con 0x7fcf18003a00 2023-10-16T11:19:27.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c15ad40 con 0x7fcf18004d90 2023-10-16T11:19:27.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1800ba10 con 0x7fcf2c0cb0c0 2023-10-16T11:19:27.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31e17700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf18003a00 0x7fcf18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31616700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 0x7fcf18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf32618700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18004d90 0x7fcf18005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31e17700 1 -- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 msgr2=0x7fcf18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31e17700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf2c0cb0c0 0x7fcf18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31e17700 1 -- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18004d90 msgr2=0x7fcf18005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:27.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31e17700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf18004d90 0x7fcf18005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31e17700 1 -- 172.21.15.181:0/336667523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf180035b0 con 0x7fcf18003a00 2023-10-16T11:19:27.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf31e17700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf18003a00 0x7fcf18003e20 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c270 tx=0x7fcf18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1149e92bd1ed8eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.821+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00cdd0 con 0x7fcf18003a00 2023-10-16T11:19:27.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c007bd0 con 0x7fcf18003a00 2023-10-16T11:19:27.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf30e15700 1 -- 172.21.15.181:0/336667523 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c007310 con 0x7fcf18003a00 2023-10-16T11:19:27.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf3407b700 1 -- 172.21.15.181:0/336667523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf18003a00 msgr2=0x7fcf18003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf3407b700 1 --2- 172.21.15.181:0/336667523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf18003a00 0x7fcf18003e20 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c270 tx=0x7fcf18005880 comp rx=0 tx=0).stop 2023-10-16T11:19:27.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf32618700 1 -- 172.21.15.181:0/336667523 reap_dead start 2023-10-16T11:19:27.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf3407b700 1 -- 172.21.15.181:0/336667523 shutdown_connections 2023-10-16T11:19:27.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf3407b700 1 -- 172.21.15.181:0/336667523 >> 172.21.15.181:0/336667523 conn(0x7fcf2c0d04a0 msgr2=0x7fcf2c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:27.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf3407b700 1 -- 172.21.15.181:0/336667523 shutdown_connections 2023-10-16T11:19:27.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.822+0000 7fcf3407b700 1 -- 172.21.15.181:0/336667523 wait complete. 2023-10-16T11:19:27.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.823+0000 7fcf3407b700 1 Processor -- start 2023-10-16T11:19:27.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.823+0000 7fcf3407b700 1 -- start start 2023-10-16T11:19:27.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf3407b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf18003a00 0x7fcf2c158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf3407b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf2c0b4390 0x7fcf2c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf3407b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf2c1efd60 0x7fcf2c1f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf3407b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c0ca6a0 con 0x7fcf2c0b4390 2023-10-16T11:19:27.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf3407b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c158ab0 con 0x7fcf2c1efd60 2023-10-16T11:19:27.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf3407b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c1f0730 con 0x7fcf18003a00 2023-10-16T11:19:27.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf31616700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf2c0b4390 0x7fcf2c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf31616700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf2c0b4390 0x7fcf2c0b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55494/0 (socket says 172.21.15.181:55494) 2023-10-16T11:19:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf31616700 1 -- 172.21.15.181:0/1336466488 learned_addr learned my addr 172.21.15.181:0/1336466488 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf32618700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf2c1efd60 0x7fcf2c1f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.824+0000 7fcf31e17700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf18003a00 0x7fcf2c158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf31616700 1 -- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf18003a00 msgr2=0x7fcf2c158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf31616700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf18003a00 0x7fcf2c158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf31616700 1 -- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf2c1efd60 msgr2=0x7fcf2c1f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf31616700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf2c1efd60 0x7fcf2c1f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf31616700 1 -- 172.21.15.181:0/1336466488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c0c9e00 con 0x7fcf2c0b4390 2023-10-16T11:19:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf31616700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf2c0b4390 0x7fcf2c0b2740 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fcf28011a20 tx=0x7fcf28011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf227fc700 1 -- 172.21.15.181:0/1336466488 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28011ec0 con 0x7fcf2c0b4390 2023-10-16T11:19:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf227fc700 1 -- 172.21.15.181:0/1336466488 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2800f940 con 0x7fcf2c0b4390 2023-10-16T11:19:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c0ca0e0 con 0x7fcf2c0b4390 2023-10-16T11:19:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf227fc700 1 -- 172.21.15.181:0/1336466488 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2800fcf0 con 0x7fcf2c0b4390 2023-10-16T11:19:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.825+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c15c500 con 0x7fcf2c0b4390 2023-10-16T11:19:27.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.826+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c15c720 con 0x7fcf2c0b4390 2023-10-16T11:19:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.826+0000 7fcf227fc700 1 -- 172.21.15.181:0/1336466488 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf28014540 con 0x7fcf2c0b4390 2023-10-16T11:19:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.827+0000 7fcf227fc700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fcf1006b020 0x7fcf1006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.827+0000 7fcf227fc700 1 -- 172.21.15.181:0/1336466488 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fcf2808bad0 con 0x7fcf2c0b4390 2023-10-16T11:19:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.827+0000 7fcf31e17700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fcf1006b020 0x7fcf1006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.827+0000 7fcf31e17700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fcf1006b020 0x7fcf1006d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c000ef0 tx=0x7fcf1c00a000 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:27.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:27.831+0000 7fcf227fc700 1 -- 172.21.15.181:0/1336466488 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf280568d0 con 0x7fcf2c0b4390 2023-10-16T11:19:28.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.017+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf2c15c380 con 0x7fcf1006b020 2023-10-16T11:19:28.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.025+0000 7fcf227fc700 1 -- 172.21.15.181:0/1336466488 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290025 (secure 0 0 0) 0x7fcf1c023070 con 0x7fcf1006b020 2023-10-16T11:19:28.030 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:28.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.028+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fcf1006b020 msgr2=0x7fcf1006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.028+0000 7fcf3407b700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fcf1006b020 0x7fcf1006d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c000ef0 tx=0x7fcf1c00a000 comp rx=0 tx=0).stop 2023-10-16T11:19:28.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.028+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf2c0b4390 msgr2=0x7fcf2c0b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.028+0000 7fcf3407b700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf2c0b4390 0x7fcf2c0b2740 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fcf28011a20 tx=0x7fcf28011d30 comp rx=0 tx=0).stop 2023-10-16T11:19:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 shutdown_connections 2023-10-16T11:19:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fcf1006b020 0x7fcf1006d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf18003a00 0x7fcf2c158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf2c1efd60 0x7fcf2c1f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 --2- 172.21.15.181:0/1336466488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf2c0b4390 0x7fcf2c0b2740 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 >> 172.21.15.181:0/1336466488 conn(0x7fcf2c0d04a0 msgr2=0x7fcf2c0c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 shutdown_connections 2023-10-16T11:19:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.029+0000 7fcf3407b700 1 -- 172.21.15.181:0/1336466488 wait complete. 2023-10-16T11:19:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:19:28.043 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-16T11:19:27.481304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421712,"kb_used_data":22328,"kb_used_omap":12,"kb_used_meta":399219,"kb_avail":754553008,"statfs":{"total":773094113280,"available":772662280192,"internally_reserved":0,"allocated":22863872,"data_stored":12261828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":408800824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.951733"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553571+0000","last_change":"2023-10-16T11:19:23.547472+0000","last_active":"2023-10-16T11:19:23.553571+0000","last_peered":"2023-10-16T11:19:23.553571+0000","last_clean":"2023-10-16T11:19:23.553571+0000","last_became_active":"2023-10-16T11:19:23.547148+0000","last_became_peered":"2023-10-16T11:19:23.547148+0000","last_unstale":"2023-10-16T11:19:23.553571+0000","last_undegraded":"2023-10-16T11:19:23.553571+0000","last_fullsized":"2023-10-16T11:19:23.553571+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540677+0000","last_change":"2023-10-16T11:19:23.552427+0000","last_active":"2023-10-16T11:19:24.540677+0000","last_peered":"2023-10-16T11:19:24.540677+0000","last_clean":"2023-10-16T11:19:24.540677+0000","last_became_active":"2023-10-16T11:19:23.552156+0000","last_became_peered":"2023-10-16T11:19:23.552156+0000","last_unstale":"2023-10-16T11:19:24.540677+0000","last_undegraded":"2023-10-16T11:19:24.540677+0000","last_fullsized":"2023-10-16T11:19:24.540677+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555675+0000","last_change":"2023-10-16T11:19:23.552160+0000","last_active":"2023-10-16T11:19:23.555675+0000","last_peered":"2023-10-16T11:19:23.555675+0000","last_clean":"2023-10-16T11:19:23.555675+0000","last_became_active":"2023-10-16T11:19:23.551805+0000","last_became_peered":"2023-10-16T11:19:23.551805+0000","last_unstale":"2023-10-16T11:19:23.555675+0000","last_undegraded":"2023-10-16T11:19:23.555675+0000","last_fullsized":"2023-10-16T11:19:23.555675+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540224+0000","last_change":"2023-10-16T11:19:23.547136+0000","last_active":"2023-10-16T11:19:24.540224+0000","last_peered":"2023-10-16T11:19:24.540224+0000","last_clean":"2023-10-16T11:19:24.540224+0000","last_became_active":"2023-10-16T11:19:23.546834+0000","last_became_peered":"2023-10-16T11:19:23.546834+0000","last_unstale":"2023-10-16T11:19:24.540224+0000","last_undegraded":"2023-10-16T11:19:24.540224+0000","last_fullsized":"2023-10-16T11:19:24.540224+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:45.683847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.544475+0000","last_change":"2023-10-16T11:19:23.541856+0000","last_active":"2023-10-16T11:19:23.544475+0000","last_peered":"2023-10-16T11:19:23.544475+0000","last_clean":"2023-10-16T11:19:23.544475+0000","last_became_active":"2023-10-16T11:19:23.541549+0000","last_became_peered":"2023-10-16T11:19:23.541549+0000","last_unstale":"2023-10-16T11:19:23.544475+0000","last_undegraded":"2023-10-16T11:19:23.544475+0000","last_fullsized":"2023-10-16T11:19:23.544475+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:27.510199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.552795+0000","last_change":"2023-10-16T11:19:23.550179+0000","last_active":"2023-10-16T11:19:23.552795+0000","last_peered":"2023-10-16T11:19:23.552795+0000","last_clean":"2023-10-16T11:19:23.552795+0000","last_became_active":"2023-10-16T11:19:23.549811+0000","last_became_peered":"2023-10-16T11:19:23.549811+0000","last_unstale":"2023-10-16T11:19:23.552795+0000","last_undegraded":"2023-10-16T11:19:23.552795+0000","last_fullsized":"2023-10-16T11:19:23.552795+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:44.413095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553442+0000","last_change":"2023-10-16T11:19:23.547259+0000","last_active":"2023-10-16T11:19:23.553442+0000","last_peered":"2023-10-16T11:19:23.553442+0000","last_clean":"2023-10-16T11:19:23.553442+0000","last_became_active":"2023-10-16T11:19:23.546957+0000","last_became_peered":"2023-10-16T11:19:23.546957+0000","last_unstale":"2023-10-16T11:19:23.553442+0000","last_undegraded":"2023-10-16T11:19:23.553442+0000","last_fullsized":"2023-10-16T11:19:23.553442+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:35.177289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.552844+0000","last_change":"2023-10-16T11:19:23.550227+0000","last_active":"2023-10-16T11:19:23.552844+0000","last_peered":"2023-10-16T11:19:23.552844+0000","last_clean":"2023-10-16T11:19:23.552844+0000","last_became_active":"2023-10-16T11:19:23.549938+0000","last_became_peered":"2023-10-16T11:19:23.549938+0000","last_unstale":"2023-10-16T11:19:23.552844+0000","last_undegraded":"2023-10-16T11:19:23.552844+0000","last_fullsized":"2023-10-16T11:19:23.552844+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:31.219643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540007+0000","last_change":"2023-10-16T11:19:23.552080+0000","last_active":"2023-10-16T11:19:24.540007+0000","last_peered":"2023-10-16T11:19:24.540007+0000","last_clean":"2023-10-16T11:19:24.540007+0000","last_became_active":"2023-10-16T11:19:23.551779+0000","last_became_peered":"2023-10-16T11:19:23.551779+0000","last_unstale":"2023-10-16T11:19:24.540007+0000","last_undegraded":"2023-10-16T11:19:24.540007+0000","last_fullsized":"2023-10-16T11:19:24.540007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536060+0000","last_change":"2023-10-16T11:19:23.546735+0000","last_active":"2023-10-16T11:19:24.536060+0000","last_peered":"2023-10-16T11:19:24.536060+0000","last_clean":"2023-10-16T11:19:24.536060+0000","last_became_active":"2023-10-16T11:19:23.546404+0000","last_became_peered":"2023-10-16T11:19:23.546404+0000","last_unstale":"2023-10-16T11:19:24.536060+0000","last_undegraded":"2023-10-16T11:19:24.536060+0000","last_fullsized":"2023-10-16T11:19:24.536060+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:29.845437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564649+0000","last_change":"2023-10-16T11:19:23.564191+0000","last_active":"2023-10-16T11:19:23.564649+0000","last_peered":"2023-10-16T11:19:23.564649+0000","last_clean":"2023-10-16T11:19:23.564649+0000","last_became_active":"2023-10-16T11:19:23.563291+0000","last_became_peered":"2023-10-16T11:19:23.563291+0000","last_unstale":"2023-10-16T11:19:23.564649+0000","last_undegraded":"2023-10-16T11:19:23.564649+0000","last_fullsized":"2023-10-16T11:19:23.564649+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.562906+0000","last_change":"2023-10-16T11:19:23.560016+0000","last_active":"2023-10-16T11:19:23.562906+0000","last_peered":"2023-10-16T11:19:23.562906+0000","last_clean":"2023-10-16T11:19:23.562906+0000","last_became_active":"2023-10-16T11:19:23.559619+0000","last_became_peered":"2023-10-16T11:19:23.559619+0000","last_unstale":"2023-10-16T11:19:23.562906+0000","last_undegraded":"2023-10-16T11:19:23.562906+0000","last_fullsized":"2023-10-16T11:19:23.562906+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:06.791712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564625+0000","last_change":"2023-10-16T11:19:23.560050+0000","last_active":"2023-10-16T11:19:23.564625+0000","last_peered":"2023-10-16T11:19:23.564625+0000","last_clean":"2023-10-16T11:19:23.564625+0000","last_became_active":"2023-10-16T11:19:23.559558+0000","last_became_peered":"2023-10-16T11:19:23.559558+0000","last_unstale":"2023-10-16T11:19:23.564625+0000","last_undegraded":"2023-10-16T11:19:23.564625+0000","last_fullsized":"2023-10-16T11:19:23.564625+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:13.873593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.560877+0000","last_change":"2023-10-16T11:19:23.559938+0000","last_active":"2023-10-16T11:19:23.560877+0000","last_peered":"2023-10-16T11:19:23.560877+0000","last_clean":"2023-10-16T11:19:23.560877+0000","last_became_active":"2023-10-16T11:19:23.559569+0000","last_became_peered":"2023-10-16T11:19:23.559569+0000","last_unstale":"2023-10-16T11:19:23.560877+0000","last_undegraded":"2023-10-16T11:19:23.560877+0000","last_fullsized":"2023-10-16T11:19:23.560877+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:34:43.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555408+0000","last_change":"2023-10-16T11:19:23.551280+0000","last_active":"2023-10-16T11:19:23.555408+0000","last_peered":"2023-10-16T11:19:23.555408+0000","last_clean":"2023-10-16T11:19:23.555408+0000","last_became_active":"2023-10-16T11:19:23.551018+0000","last_became_peered":"2023-10-16T11:19:23.551018+0000","last_unstale":"2023-10-16T11:19:23.555408+0000","last_undegraded":"2023-10-16T11:19:23.555408+0000","last_fullsized":"2023-10-16T11:19:23.555408+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:04.497716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.535849+0000","last_change":"2023-10-16T11:19:23.563754+0000","last_active":"2023-10-16T11:19:24.535849+0000","last_peered":"2023-10-16T11:19:24.535849+0000","last_clean":"2023-10-16T11:19:24.535849+0000","last_became_active":"2023-10-16T11:19:23.563487+0000","last_became_peered":"2023-10-16T11:19:23.563487+0000","last_unstale":"2023-10-16T11:19:24.535849+0000","last_undegraded":"2023-10-16T11:19:24.535849+0000","last_fullsized":"2023-10-16T11:19:24.535849+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:13.814299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555196+0000","last_change":"2023-10-16T11:19:23.548276+0000","last_active":"2023-10-16T11:19:23.555196+0000","last_peered":"2023-10-16T11:19:23.555196+0000","last_clean":"2023-10-16T11:19:23.555196+0000","last_became_active":"2023-10-16T11:19:23.547177+0000","last_became_peered":"2023-10-16T11:19:23.547177+0000","last_unstale":"2023-10-16T11:19:23.555196+0000","last_undegraded":"2023-10-16T11:19:23.555196+0000","last_fullsized":"2023-10-16T11:19:23.555196+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:10.899194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536199+0000","last_change":"2023-10-16T11:19:23.549769+0000","last_active":"2023-10-16T11:19:24.536199+0000","last_peered":"2023-10-16T11:19:24.536199+0000","last_clean":"2023-10-16T11:19:24.536199+0000","last_became_active":"2023-10-16T11:19:23.549505+0000","last_became_peered":"2023-10-16T11:19:23.549505+0000","last_unstale":"2023-10-16T11:19:24.536199+0000","last_undegraded":"2023-10-16T11:19:24.536199+0000","last_fullsized":"2023-10-16T11:19:24.536199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:20.931218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536011+0000","last_change":"2023-10-16T11:19:23.547740+0000","last_active":"2023-10-16T11:19:24.536011+0000","last_peered":"2023-10-16T11:19:24.536011+0000","last_clean":"2023-10-16T11:19:24.536011+0000","last_became_active":"2023-10-16T11:19:23.546421+0000","last_became_peered":"2023-10-16T11:19:23.546421+0000","last_unstale":"2023-10-16T11:19:24.536011+0000","last_undegraded":"2023-10-16T11:19:24.536011+0000","last_fullsized":"2023-10-16T11:19:24.536011+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:27.685924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.544641+0000","last_change":"2023-10-16T11:19:23.542321+0000","last_active":"2023-10-16T11:19:23.544641+0000","last_peered":"2023-10-16T11:19:23.544641+0000","last_clean":"2023-10-16T11:19:23.544641+0000","last_became_active":"2023-10-16T11:19:23.541862+0000","last_became_peered":"2023-10-16T11:19:23.541862+0000","last_unstale":"2023-10-16T11:19:23.544641+0000","last_undegraded":"2023-10-16T11:19:23.544641+0000","last_fullsized":"2023-10-16T11:19:23.544641+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:20.905021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539849+0000","last_change":"2023-10-16T11:19:23.550209+0000","last_active":"2023-10-16T11:19:24.539849+0000","last_peered":"2023-10-16T11:19:24.539849+0000","last_clean":"2023-10-16T11:19:24.539849+0000","last_became_active":"2023-10-16T11:19:23.549849+0000","last_became_peered":"2023-10-16T11:19:23.549849+0000","last_unstale":"2023-10-16T11:19:24.539849+0000","last_undegraded":"2023-10-16T11:19:24.539849+0000","last_fullsized":"2023-10-16T11:19:24.539849+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:29.715382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773951+0000","last_change":"2023-10-16T11:19:23.546038+0000","last_active":"2023-10-16T11:19:24.773951+0000","last_peered":"2023-10-16T11:19:24.773951+0000","last_clean":"2023-10-16T11:19:24.773951+0000","last_became_active":"2023-10-16T11:19:23.545752+0000","last_became_peered":"2023-10-16T11:19:23.545752+0000","last_unstale":"2023-10-16T11:19:24.773951+0000","last_undegraded":"2023-10-16T11:19:24.773951+0000","last_fullsized":"2023-10-16T11:19:24.773951+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:13.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773725+0000","last_change":"2023-10-16T11:19:23.552048+0000","last_active":"2023-10-16T11:19:24.773725+0000","last_peered":"2023-10-16T11:19:24.773725+0000","last_clean":"2023-10-16T11:19:24.773725+0000","last_became_active":"2023-10-16T11:19:23.551676+0000","last_became_peered":"2023-10-16T11:19:23.551676+0000","last_unstale":"2023-10-16T11:19:24.773725+0000","last_undegraded":"2023-10-16T11:19:24.773725+0000","last_fullsized":"2023-10-16T11:19:24.773725+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:16.450373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539945+0000","last_change":"2023-10-16T11:19:23.548868+0000","last_active":"2023-10-16T11:19:24.539945+0000","last_peered":"2023-10-16T11:19:24.539945+0000","last_clean":"2023-10-16T11:19:24.539945+0000","last_became_active":"2023-10-16T11:19:23.548583+0000","last_became_peered":"2023-10-16T11:19:23.548583+0000","last_unstale":"2023-10-16T11:19:24.539945+0000","last_undegraded":"2023-10-16T11:19:24.539945+0000","last_fullsized":"2023-10-16T11:19:24.539945+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:02.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540107+0000","last_change":"2023-10-16T11:19:23.546857+0000","last_active":"2023-10-16T11:19:24.540107+0000","last_peered":"2023-10-16T11:19:24.540107+0000","last_clean":"2023-10-16T11:19:24.540107+0000","last_became_active":"2023-10-16T11:19:23.546592+0000","last_became_peered":"2023-10-16T11:19:23.546592+0000","last_unstale":"2023-10-16T11:19:24.540107+0000","last_undegraded":"2023-10-16T11:19:24.540107+0000","last_fullsized":"2023-10-16T11:19:24.540107+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553425+0000","last_change":"2023-10-16T11:19:23.546460+0000","last_active":"2023-10-16T11:19:23.553425+0000","last_peered":"2023-10-16T11:19:23.553425+0000","last_clean":"2023-10-16T11:19:23.553425+0000","last_became_active":"2023-10-16T11:19:23.546187+0000","last_became_peered":"2023-10-16T11:19:23.546187+0000","last_unstale":"2023-10-16T11:19:23.553425+0000","last_undegraded":"2023-10-16T11:19:23.553425+0000","last_fullsized":"2023-10-16T11:19:23.553425+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.562585+0000","last_change":"2023-10-16T11:19:23.559781+0000","last_active":"2023-10-16T11:19:23.562585+0000","last_peered":"2023-10-16T11:19:23.562585+0000","last_clean":"2023-10-16T11:19:23.562585+0000","last_became_active":"2023-10-16T11:19:23.559451+0000","last_became_peered":"2023-10-16T11:19:23.559451+0000","last_unstale":"2023-10-16T11:19:23.562585+0000","last_undegraded":"2023-10-16T11:19:23.562585+0000","last_fullsized":"2023-10-16T11:19:23.562585+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:34:43.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.556270+0000","last_change":"2023-10-16T11:19:23.555152+0000","last_active":"2023-10-16T11:19:23.556270+0000","last_peered":"2023-10-16T11:19:23.556270+0000","last_clean":"2023-10-16T11:19:23.556270+0000","last_became_active":"2023-10-16T11:19:23.554747+0000","last_became_peered":"2023-10-16T11:19:23.554747+0000","last_unstale":"2023-10-16T11:19:23.556270+0000","last_undegraded":"2023-10-16T11:19:23.556270+0000","last_fullsized":"2023-10-16T11:19:23.556270+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.566220+0000","last_change":"2023-10-16T11:19:23.565127+0000","last_active":"2023-10-16T11:19:23.566220+0000","last_peered":"2023-10-16T11:19:23.566220+0000","last_clean":"2023-10-16T11:19:23.566220+0000","last_became_active":"2023-10-16T11:19:23.564852+0000","last_became_peered":"2023-10-16T11:19:23.564852+0000","last_unstale":"2023-10-16T11:19:23.566220+0000","last_undegraded":"2023-10-16T11:19:23.566220+0000","last_fullsized":"2023-10-16T11:19:23.566220+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:04.497716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539767+0000","last_change":"2023-10-16T11:19:23.547762+0000","last_active":"2023-10-16T11:19:24.539767+0000","last_peered":"2023-10-16T11:19:24.539767+0000","last_clean":"2023-10-16T11:19:24.539767+0000","last_became_active":"2023-10-16T11:19:23.547353+0000","last_became_peered":"2023-10-16T11:19:23.547353+0000","last_unstale":"2023-10-16T11:19:24.539767+0000","last_undegraded":"2023-10-16T11:19:24.539767+0000","last_fullsized":"2023-10-16T11:19:24.539767+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536330+0000","last_change":"2023-10-16T11:19:23.542878+0000","last_active":"2023-10-16T11:19:24.536330+0000","last_peered":"2023-10-16T11:19:24.536330+0000","last_clean":"2023-10-16T11:19:24.536330+0000","last_became_active":"2023-10-16T11:19:23.542593+0000","last_became_peered":"2023-10-16T11:19:23.542593+0000","last_unstale":"2023-10-16T11:19:24.536330+0000","last_undegraded":"2023-10-16T11:19:24.536330+0000","last_fullsized":"2023-10-16T11:19:24.536330+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:39.898154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.545664+0000","last_change":"2023-10-16T11:19:23.544178+0000","last_active":"2023-10-16T11:19:23.545664+0000","last_peered":"2023-10-16T11:19:23.545664+0000","last_clean":"2023-10-16T11:19:23.545664+0000","last_became_active":"2023-10-16T11:19:23.543898+0000","last_became_peered":"2023-10-16T11:19:23.543898+0000","last_unstale":"2023-10-16T11:19:23.545664+0000","last_undegraded":"2023-10-16T11:19:23.545664+0000","last_fullsized":"2023-10-16T11:19:23.545664+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:58:47.638973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540529+0000","last_change":"2023-10-16T11:19:23.551775+0000","last_active":"2023-10-16T11:19:24.540529+0000","last_peered":"2023-10-16T11:19:24.540529+0000","last_clean":"2023-10-16T11:19:24.540529+0000","last_became_active":"2023-10-16T11:19:23.551490+0000","last_became_peered":"2023-10-16T11:19:23.551490+0000","last_unstale":"2023-10-16T11:19:24.540529+0000","last_undegraded":"2023-10-16T11:19:24.540529+0000","last_fullsized":"2023-10-16T11:19:24.540529+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.566447+0000","last_change":"2023-10-16T11:19:22.538829+0000","last_active":"2023-10-16T11:19:23.566447+0000","last_peered":"2023-10-16T11:19:23.566447+0000","last_clean":"2023-10-16T11:19:23.566447+0000","last_became_active":"2023-10-16T11:19:22.538537+0000","last_became_peered":"2023-10-16T11:19:22.538537+0000","last_unstale":"2023-10-16T11:19:23.566447+0000","last_undegraded":"2023-10-16T11:19:23.566447+0000","last_fullsized":"2023-10-16T11:19:23.566447+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:09.892801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773981+0000","last_change":"2023-10-16T11:19:23.559791+0000","last_active":"2023-10-16T11:19:24.773981+0000","last_peered":"2023-10-16T11:19:24.773981+0000","last_clean":"2023-10-16T11:19:24.773981+0000","last_became_active":"2023-10-16T11:19:23.559513+0000","last_became_peered":"2023-10-16T11:19:23.559513+0000","last_unstale":"2023-10-16T11:19:24.773981+0000","last_undegraded":"2023-10-16T11:19:24.773981+0000","last_fullsized":"2023-10-16T11:19:24.773981+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:39.986537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540469+0000","last_change":"2023-10-16T11:19:22.536812+0000","last_active":"2023-10-16T11:19:24.540469+0000","last_peered":"2023-10-16T11:19:24.540469+0000","last_clean":"2023-10-16T11:19:24.540469+0000","last_became_active":"2023-10-16T11:19:22.536545+0000","last_became_peered":"2023-10-16T11:19:22.536545+0000","last_unstale":"2023-10-16T11:19:24.540469+0000","last_undegraded":"2023-10-16T11:19:24.540469+0000","last_fullsized":"2023-10-16T11:19:24.540469+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553472+0000","last_change":"2023-10-16T11:19:23.546849+0000","last_active":"2023-10-16T11:19:23.553472+0000","last_peered":"2023-10-16T11:19:23.553472+0000","last_clean":"2023-10-16T11:19:23.553472+0000","last_became_active":"2023-10-16T11:19:23.546575+0000","last_became_peered":"2023-10-16T11:19:23.546575+0000","last_unstale":"2023-10-16T11:19:23.553472+0000","last_undegraded":"2023-10-16T11:19:23.553472+0000","last_fullsized":"2023-10-16T11:19:23.553472+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:41:37.262229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564933+0000","last_change":"2023-10-16T11:19:22.533919+0000","last_active":"2023-10-16T11:19:23.564933+0000","last_peered":"2023-10-16T11:19:23.564933+0000","last_clean":"2023-10-16T11:19:23.564933+0000","last_became_active":"2023-10-16T11:19:22.533695+0000","last_became_peered":"2023-10-16T11:19:22.533695+0000","last_unstale":"2023-10-16T11:19:23.564933+0000","last_undegraded":"2023-10-16T11:19:23.564933+0000","last_fullsized":"2023-10-16T11:19:23.564933+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536127+0000","last_change":"2023-10-16T11:19:23.552923+0000","last_active":"2023-10-16T11:19:24.536127+0000","last_peered":"2023-10-16T11:19:24.536127+0000","last_clean":"2023-10-16T11:19:24.536127+0000","last_became_active":"2023-10-16T11:19:23.552685+0000","last_became_peered":"2023-10-16T11:19:23.552685+0000","last_unstale":"2023-10-16T11:19:24.536127+0000","last_undegraded":"2023-10-16T11:19:24.536127+0000","last_fullsized":"2023-10-16T11:19:24.536127+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:13.492062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540349+0000","last_change":"2023-10-16T11:19:22.535919+0000","last_active":"2023-10-16T11:19:24.540349+0000","last_peered":"2023-10-16T11:19:24.540349+0000","last_clean":"2023-10-16T11:19:24.540349+0000","last_became_active":"2023-10-16T11:19:22.535612+0000","last_became_peered":"2023-10-16T11:19:22.535612+0000","last_unstale":"2023-10-16T11:19:24.540349+0000","last_undegraded":"2023-10-16T11:19:24.540349+0000","last_fullsized":"2023-10-16T11:19:24.540349+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:54.628429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539859+0000","last_change":"2023-10-16T11:19:23.546200+0000","last_active":"2023-10-16T11:19:24.539859+0000","last_peered":"2023-10-16T11:19:24.539859+0000","last_clean":"2023-10-16T11:19:24.539859+0000","last_became_active":"2023-10-16T11:19:23.545867+0000","last_became_peered":"2023-10-16T11:19:23.545867+0000","last_unstale":"2023-10-16T11:19:24.539859+0000","last_undegraded":"2023-10-16T11:19:24.539859+0000","last_fullsized":"2023-10-16T11:19:24.539859+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.670688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.566380+0000","last_change":"2023-10-16T11:19:22.543956+0000","last_active":"2023-10-16T11:19:23.566380+0000","last_peered":"2023-10-16T11:19:23.566380+0000","last_clean":"2023-10-16T11:19:23.566380+0000","last_became_active":"2023-10-16T11:19:22.543633+0000","last_became_peered":"2023-10-16T11:19:22.543633+0000","last_unstale":"2023-10-16T11:19:23.566380+0000","last_undegraded":"2023-10-16T11:19:23.566380+0000","last_fullsized":"2023-10-16T11:19:23.566380+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:12:53.412047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.544506+0000","last_change":"2023-10-16T11:19:23.543708+0000","last_active":"2023-10-16T11:19:23.544506+0000","last_peered":"2023-10-16T11:19:23.544506+0000","last_clean":"2023-10-16T11:19:23.544506+0000","last_became_active":"2023-10-16T11:19:23.543449+0000","last_became_peered":"2023-10-16T11:19:23.543449+0000","last_unstale":"2023-10-16T11:19:23.544506+0000","last_undegraded":"2023-10-16T11:19:23.544506+0000","last_fullsized":"2023-10-16T11:19:23.544506+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:45.477205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.538252+0000","last_change":"2023-10-16T11:19:22.547192+0000","last_active":"2023-10-16T11:19:23.538252+0000","last_peered":"2023-10-16T11:19:23.538252+0000","last_clean":"2023-10-16T11:19:23.538252+0000","last_became_active":"2023-10-16T11:19:22.546912+0000","last_became_peered":"2023-10-16T11:19:22.546912+0000","last_unstale":"2023-10-16T11:19:23.538252+0000","last_undegraded":"2023-10-16T11:19:23.538252+0000","last_fullsized":"2023-10-16T11:19:23.538252+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:17.017260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540051+0000","last_change":"2023-10-16T11:19:23.546966+0000","last_active":"2023-10-16T11:19:24.540051+0000","last_peered":"2023-10-16T11:19:24.540051+0000","last_clean":"2023-10-16T11:19:24.540051+0000","last_became_active":"2023-10-16T11:19:23.546685+0000","last_became_peered":"2023-10-16T11:19:23.546685+0000","last_unstale":"2023-10-16T11:19:24.540051+0000","last_undegraded":"2023-10-16T11:19:24.540051+0000","last_fullsized":"2023-10-16T11:19:24.540051+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:13.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773537+0000","last_change":"2023-10-16T11:19:22.534080+0000","last_active":"2023-10-16T11:19:24.773537+0000","last_peered":"2023-10-16T11:19:24.773537+0000","last_clean":"2023-10-16T11:19:24.773537+0000","last_became_active":"2023-10-16T11:19:22.533693+0000","last_became_peered":"2023-10-16T11:19:22.533693+0000","last_unstale":"2023-10-16T11:19:24.773537+0000","last_undegraded":"2023-10-16T11:19:24.773537+0000","last_fullsized":"2023-10-16T11:19:24.773537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:57.023698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540224+0000","last_change":"2023-10-16T11:19:23.553070+0000","last_active":"2023-10-16T11:19:24.540224+0000","last_peered":"2023-10-16T11:19:24.540224+0000","last_clean":"2023-10-16T11:19:24.540224+0000","last_became_active":"2023-10-16T11:19:23.552761+0000","last_became_peered":"2023-10-16T11:19:23.552761+0000","last_unstale":"2023-10-16T11:19:24.540224+0000","last_undegraded":"2023-10-16T11:19:24.540224+0000","last_fullsized":"2023-10-16T11:19:24.540224+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:36.373122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540399+0000","last_change":"2023-10-16T11:19:22.545557+0000","last_active":"2023-10-16T11:19:23.540399+0000","last_peered":"2023-10-16T11:19:23.540399+0000","last_clean":"2023-10-16T11:19:23.540399+0000","last_became_active":"2023-10-16T11:19:22.545253+0000","last_became_peered":"2023-10-16T11:19:22.545253+0000","last_unstale":"2023-10-16T11:19:23.540399+0000","last_undegraded":"2023-10-16T11:19:23.540399+0000","last_fullsized":"2023-10-16T11:19:23.540399+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:11.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.561454+0000","last_change":"2023-10-16T11:19:23.560046+0000","last_active":"2023-10-16T11:19:23.561454+0000","last_peered":"2023-10-16T11:19:23.561454+0000","last_clean":"2023-10-16T11:19:23.561454+0000","last_became_active":"2023-10-16T11:19:23.559718+0000","last_became_peered":"2023-10-16T11:19:23.559718+0000","last_unstale":"2023-10-16T11:19:23.561454+0000","last_undegraded":"2023-10-16T11:19:23.561454+0000","last_fullsized":"2023-10-16T11:19:23.561454+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:46.256661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773995+0000","last_change":"2023-10-16T11:19:22.535771+0000","last_active":"2023-10-16T11:19:24.773995+0000","last_peered":"2023-10-16T11:19:24.773995+0000","last_clean":"2023-10-16T11:19:24.773995+0000","last_became_active":"2023-10-16T11:19:22.535486+0000","last_became_peered":"2023-10-16T11:19:22.535486+0000","last_unstale":"2023-10-16T11:19:24.773995+0000","last_undegraded":"2023-10-16T11:19:24.773995+0000","last_fullsized":"2023-10-16T11:19:24.773995+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:25.214100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540419+0000","last_change":"2023-10-16T11:19:23.563660+0000","last_active":"2023-10-16T11:19:24.540419+0000","last_peered":"2023-10-16T11:19:24.540419+0000","last_clean":"2023-10-16T11:19:24.540419+0000","last_became_active":"2023-10-16T11:19:23.563402+0000","last_became_peered":"2023-10-16T11:19:23.563402+0000","last_unstale":"2023-10-16T11:19:24.540419+0000","last_undegraded":"2023-10-16T11:19:24.540419+0000","last_fullsized":"2023-10-16T11:19:24.540419+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:41.606189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536228+0000","last_change":"2023-10-16T11:19:22.544432+0000","last_active":"2023-10-16T11:19:24.536228+0000","last_peered":"2023-10-16T11:19:24.536228+0000","last_clean":"2023-10-16T11:19:24.536228+0000","last_became_active":"2023-10-16T11:19:22.543046+0000","last_became_peered":"2023-10-16T11:19:22.543046+0000","last_unstale":"2023-10-16T11:19:24.536228+0000","last_undegraded":"2023-10-16T11:19:24.536228+0000","last_fullsized":"2023-10-16T11:19:24.536228+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:08.672505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.554606+0000","last_change":"2023-10-16T11:19:23.548103+0000","last_active":"2023-10-16T11:19:23.554606+0000","last_peered":"2023-10-16T11:19:23.554606+0000","last_clean":"2023-10-16T11:19:23.554606+0000","last_became_active":"2023-10-16T11:19:23.547665+0000","last_became_peered":"2023-10-16T11:19:23.547665+0000","last_unstale":"2023-10-16T11:19:23.554606+0000","last_undegraded":"2023-10-16T11:19:23.554606+0000","last_fullsized":"2023-10-16T11:19:23.554606+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:08.885647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"215'6","reported_seq":18,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540455+0000","last_change":"2023-10-16T11:19:22.532339+0000","last_active":"2023-10-16T11:19:24.540455+0000","last_peered":"2023-10-16T11:19:24.540455+0000","last_clean":"2023-10-16T11:19:24.540455+0000","last_became_active":"2023-10-16T11:19:22.532053+0000","last_became_peered":"2023-10-16T11:19:22.532053+0000","last_unstale":"2023-10-16T11:19:24.540455+0000","last_undegraded":"2023-10-16T11:19:24.540455+0000","last_fullsized":"2023-10-16T11:19:24.540455+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:21.190168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.552817+0000","last_change":"2023-10-16T11:19:23.550398+0000","last_active":"2023-10-16T11:19:23.552817+0000","last_peered":"2023-10-16T11:19:23.552817+0000","last_clean":"2023-10-16T11:19:23.552817+0000","last_became_active":"2023-10-16T11:19:23.545907+0000","last_became_peered":"2023-10-16T11:19:23.545907+0000","last_unstale":"2023-10-16T11:19:23.552817+0000","last_undegraded":"2023-10-16T11:19:23.552817+0000","last_fullsized":"2023-10-16T11:19:23.552817+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:10.743840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773746+0000","last_change":"2023-10-16T11:19:22.544032+0000","last_active":"2023-10-16T11:19:24.773746+0000","last_peered":"2023-10-16T11:19:24.773746+0000","last_clean":"2023-10-16T11:19:24.773746+0000","last_became_active":"2023-10-16T11:19:22.543713+0000","last_became_peered":"2023-10-16T11:19:22.543713+0000","last_unstale":"2023-10-16T11:19:24.773746+0000","last_undegraded":"2023-10-16T11:19:24.773746+0000","last_fullsized":"2023-10-16T11:19:24.773746+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:01.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536202+0000","last_change":"2023-10-16T11:19:23.552779+0000","last_active":"2023-10-16T11:19:24.536202+0000","last_peered":"2023-10-16T11:19:24.536202+0000","last_clean":"2023-10-16T11:19:24.536202+0000","last_became_active":"2023-10-16T11:19:23.552491+0000","last_became_peered":"2023-10-16T11:19:23.552491+0000","last_unstale":"2023-10-16T11:19:24.536202+0000","last_undegraded":"2023-10-16T11:19:24.536202+0000","last_fullsized":"2023-10-16T11:19:24.536202+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:02.969587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773976+0000","last_change":"2023-10-16T11:19:22.538147+0000","last_active":"2023-10-16T11:19:24.773976+0000","last_peered":"2023-10-16T11:19:24.773976+0000","last_clean":"2023-10-16T11:19:24.773976+0000","last_became_active":"2023-10-16T11:19:22.537850+0000","last_became_peered":"2023-10-16T11:19:22.537850+0000","last_unstale":"2023-10-16T11:19:24.773976+0000","last_undegraded":"2023-10-16T11:19:24.773976+0000","last_fullsized":"2023-10-16T11:19:24.773976+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:44.794069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564185+0000","last_change":"2023-10-16T11:19:23.563603+0000","last_active":"2023-10-16T11:19:23.564185+0000","last_peered":"2023-10-16T11:19:23.564185+0000","last_clean":"2023-10-16T11:19:23.564185+0000","last_became_active":"2023-10-16T11:19:23.563305+0000","last_became_peered":"2023-10-16T11:19:23.563305+0000","last_unstale":"2023-10-16T11:19:23.564185+0000","last_undegraded":"2023-10-16T11:19:23.564185+0000","last_fullsized":"2023-10-16T11:19:23.564185+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:04.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.535827+0000","last_change":"2023-10-16T11:19:22.538008+0000","last_active":"2023-10-16T11:19:24.535827+0000","last_peered":"2023-10-16T11:19:24.535827+0000","last_clean":"2023-10-16T11:19:24.535827+0000","last_became_active":"2023-10-16T11:19:22.537752+0000","last_became_peered":"2023-10-16T11:19:22.537752+0000","last_unstale":"2023-10-16T11:19:24.535827+0000","last_undegraded":"2023-10-16T11:19:24.535827+0000","last_fullsized":"2023-10-16T11:19:24.535827+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:20.859898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555425+0000","last_change":"2023-10-16T11:19:23.552950+0000","last_active":"2023-10-16T11:19:23.555425+0000","last_peered":"2023-10-16T11:19:23.555425+0000","last_clean":"2023-10-16T11:19:23.555425+0000","last_became_active":"2023-10-16T11:19:23.552686+0000","last_became_peered":"2023-10-16T11:19:23.552686+0000","last_unstale":"2023-10-16T11:19:23.555425+0000","last_undegraded":"2023-10-16T11:19:23.555425+0000","last_fullsized":"2023-10-16T11:19:23.555425+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:38:30.386843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":19,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.565891+0000","last_change":"2023-10-16T11:19:22.534093+0000","last_active":"2023-10-16T11:19:23.565891+0000","last_peered":"2023-10-16T11:19:23.565891+0000","last_clean":"2023-10-16T11:19:23.565891+0000","last_became_active":"2023-10-16T11:19:22.533637+0000","last_became_peered":"2023-10-16T11:19:22.533637+0000","last_unstale":"2023-10-16T11:19:23.565891+0000","last_undegraded":"2023-10-16T11:19:23.565891+0000","last_fullsized":"2023-10-16T11:19:23.565891+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540565+0000","last_change":"2023-10-16T11:19:23.547530+0000","last_active":"2023-10-16T11:19:24.540565+0000","last_peered":"2023-10-16T11:19:24.540565+0000","last_clean":"2023-10-16T11:19:24.540565+0000","last_became_active":"2023-10-16T11:19:23.547255+0000","last_became_peered":"2023-10-16T11:19:23.547255+0000","last_unstale":"2023-10-16T11:19:24.540565+0000","last_undegraded":"2023-10-16T11:19:24.540565+0000","last_fullsized":"2023-10-16T11:19:24.540565+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.557798+0000","last_change":"2023-10-16T11:19:22.545027+0000","last_active":"2023-10-16T11:19:23.557798+0000","last_peered":"2023-10-16T11:19:23.557798+0000","last_clean":"2023-10-16T11:19:23.557798+0000","last_became_active":"2023-10-16T11:19:22.544756+0000","last_became_peered":"2023-10-16T11:19:22.544756+0000","last_unstale":"2023-10-16T11:19:23.557798+0000","last_undegraded":"2023-10-16T11:19:23.557798+0000","last_fullsized":"2023-10-16T11:19:23.557798+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:23.693322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540221+0000","last_change":"2023-10-16T11:19:23.546218+0000","last_active":"2023-10-16T11:19:24.540221+0000","last_peered":"2023-10-16T11:19:24.540221+0000","last_clean":"2023-10-16T11:19:24.540221+0000","last_became_active":"2023-10-16T11:19:23.545896+0000","last_became_peered":"2023-10-16T11:19:23.545896+0000","last_unstale":"2023-10-16T11:19:24.540221+0000","last_undegraded":"2023-10-16T11:19:24.540221+0000","last_fullsized":"2023-10-16T11:19:24.540221+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540640+0000","last_change":"2023-10-16T11:19:22.539558+0000","last_active":"2023-10-16T11:19:24.540640+0000","last_peered":"2023-10-16T11:19:24.540640+0000","last_clean":"2023-10-16T11:19:24.540640+0000","last_became_active":"2023-10-16T11:19:22.539284+0000","last_became_peered":"2023-10-16T11:19:22.539284+0000","last_unstale":"2023-10-16T11:19:24.540640+0000","last_undegraded":"2023-10-16T11:19:24.540640+0000","last_fullsized":"2023-10-16T11:19:24.540640+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.556031+0000","last_change":"2023-10-16T11:19:23.553745+0000","last_active":"2023-10-16T11:19:23.556031+0000","last_peered":"2023-10-16T11:19:23.556031+0000","last_clean":"2023-10-16T11:19:23.556031+0000","last_became_active":"2023-10-16T11:19:23.553462+0000","last_became_peered":"2023-10-16T11:19:23.553462+0000","last_unstale":"2023-10-16T11:19:23.556031+0000","last_undegraded":"2023-10-16T11:19:23.556031+0000","last_fullsized":"2023-10-16T11:19:23.556031+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536257+0000","last_change":"2023-10-16T11:19:22.533601+0000","last_active":"2023-10-16T11:19:24.536257+0000","last_peered":"2023-10-16T11:19:24.536257+0000","last_clean":"2023-10-16T11:19:24.536257+0000","last_became_active":"2023-10-16T11:19:22.533374+0000","last_became_peered":"2023-10-16T11:19:22.533374+0000","last_unstale":"2023-10-16T11:19:24.536257+0000","last_undegraded":"2023-10-16T11:19:24.536257+0000","last_fullsized":"2023-10-16T11:19:24.536257+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:52.564361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536152+0000","last_change":"2023-10-16T11:19:23.545760+0000","last_active":"2023-10-16T11:19:24.536152+0000","last_peered":"2023-10-16T11:19:24.536152+0000","last_clean":"2023-10-16T11:19:24.536152+0000","last_became_active":"2023-10-16T11:19:23.545499+0000","last_became_peered":"2023-10-16T11:19:23.545499+0000","last_unstale":"2023-10-16T11:19:24.536152+0000","last_undegraded":"2023-10-16T11:19:24.536152+0000","last_fullsized":"2023-10-16T11:19:24.536152+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:32.701590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.548366+0000","last_change":"2023-10-16T11:19:22.545030+0000","last_active":"2023-10-16T11:19:23.548366+0000","last_peered":"2023-10-16T11:19:23.548366+0000","last_clean":"2023-10-16T11:19:23.548366+0000","last_became_active":"2023-10-16T11:19:22.544773+0000","last_became_peered":"2023-10-16T11:19:22.544773+0000","last_unstale":"2023-10-16T11:19:23.548366+0000","last_undegraded":"2023-10-16T11:19:23.548366+0000","last_fullsized":"2023-10-16T11:19:23.548366+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:22.946353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536520+0000","last_change":"2023-10-16T11:19:23.543334+0000","last_active":"2023-10-16T11:19:24.536520+0000","last_peered":"2023-10-16T11:19:24.536520+0000","last_clean":"2023-10-16T11:19:24.536520+0000","last_became_active":"2023-10-16T11:19:23.542527+0000","last_became_peered":"2023-10-16T11:19:23.542527+0000","last_unstale":"2023-10-16T11:19:24.536520+0000","last_undegraded":"2023-10-16T11:19:24.536520+0000","last_fullsized":"2023-10-16T11:19:24.536520+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:59.083108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.536104+0000","last_change":"2023-10-16T11:19:22.536403+0000","last_active":"2023-10-16T11:19:24.536104+0000","last_peered":"2023-10-16T11:19:24.536104+0000","last_clean":"2023-10-16T11:19:24.536104+0000","last_became_active":"2023-10-16T11:19:22.536132+0000","last_became_peered":"2023-10-16T11:19:22.536132+0000","last_unstale":"2023-10-16T11:19:24.536104+0000","last_undegraded":"2023-10-16T11:19:24.536104+0000","last_fullsized":"2023-10-16T11:19:24.536104+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:53.563487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.556247+0000","last_change":"2023-10-16T11:19:23.554949+0000","last_active":"2023-10-16T11:19:23.556247+0000","last_peered":"2023-10-16T11:19:23.556247+0000","last_clean":"2023-10-16T11:19:23.556247+0000","last_became_active":"2023-10-16T11:19:23.554300+0000","last_became_peered":"2023-10-16T11:19:23.554300+0000","last_unstale":"2023-10-16T11:19:23.556247+0000","last_undegraded":"2023-10-16T11:19:23.556247+0000","last_fullsized":"2023-10-16T11:19:23.556247+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:06.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540159+0000","last_change":"2023-10-16T11:19:22.542021+0000","last_active":"2023-10-16T11:19:23.540159+0000","last_peered":"2023-10-16T11:19:23.540159+0000","last_clean":"2023-10-16T11:19:23.540159+0000","last_became_active":"2023-10-16T11:19:22.541718+0000","last_became_peered":"2023-10-16T11:19:22.541718+0000","last_unstale":"2023-10-16T11:19:23.540159+0000","last_undegraded":"2023-10-16T11:19:23.540159+0000","last_fullsized":"2023-10-16T11:19:23.540159+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:12.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539246+0000","last_change":"2023-10-16T11:19:23.548561+0000","last_active":"2023-10-16T11:19:24.539246+0000","last_peered":"2023-10-16T11:19:24.539246+0000","last_clean":"2023-10-16T11:19:24.539246+0000","last_became_active":"2023-10-16T11:19:23.548310+0000","last_became_peered":"2023-10-16T11:19:23.548310+0000","last_unstale":"2023-10-16T11:19:24.539246+0000","last_undegraded":"2023-10-16T11:19:24.539246+0000","last_fullsized":"2023-10-16T11:19:24.539246+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540497+0000","last_change":"2023-10-16T11:19:22.532222+0000","last_active":"2023-10-16T11:19:23.540497+0000","last_peered":"2023-10-16T11:19:23.540497+0000","last_clean":"2023-10-16T11:19:23.540497+0000","last_became_active":"2023-10-16T11:19:22.531939+0000","last_became_peered":"2023-10-16T11:19:22.531939+0000","last_unstale":"2023-10-16T11:19:23.540497+0000","last_undegraded":"2023-10-16T11:19:23.540497+0000","last_fullsized":"2023-10-16T11:19:23.540497+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773737+0000","last_change":"2023-10-16T11:19:23.563941+0000","last_active":"2023-10-16T11:19:24.773737+0000","last_peered":"2023-10-16T11:19:24.773737+0000","last_clean":"2023-10-16T11:19:24.773737+0000","last_became_active":"2023-10-16T11:19:23.563666+0000","last_became_peered":"2023-10-16T11:19:23.563666+0000","last_unstale":"2023-10-16T11:19:24.773737+0000","last_undegraded":"2023-10-16T11:19:24.773737+0000","last_fullsized":"2023-10-16T11:19:24.773737+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:43.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.539184+0000","last_change":"2023-10-16T11:19:22.547829+0000","last_active":"2023-10-16T11:19:23.539184+0000","last_peered":"2023-10-16T11:19:23.539184+0000","last_clean":"2023-10-16T11:19:23.539184+0000","last_became_active":"2023-10-16T11:19:22.547563+0000","last_became_peered":"2023-10-16T11:19:22.547563+0000","last_unstale":"2023-10-16T11:19:23.539184+0000","last_undegraded":"2023-10-16T11:19:23.539184+0000","last_fullsized":"2023-10-16T11:19:23.539184+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:15.819032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.565522+0000","last_change":"2023-10-16T11:19:23.561790+0000","last_active":"2023-10-16T11:19:23.565522+0000","last_peered":"2023-10-16T11:19:23.565522+0000","last_clean":"2023-10-16T11:19:23.565522+0000","last_became_active":"2023-10-16T11:19:23.561531+0000","last_became_peered":"2023-10-16T11:19:23.561531+0000","last_unstale":"2023-10-16T11:19:23.565522+0000","last_undegraded":"2023-10-16T11:19:23.565522+0000","last_fullsized":"2023-10-16T11:19:23.565522+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:29:06.096292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.565979+0000","last_change":"2023-10-16T11:19:22.537256+0000","last_active":"2023-10-16T11:19:23.565979+0000","last_peered":"2023-10-16T11:19:23.565979+0000","last_clean":"2023-10-16T11:19:23.565979+0000","last_became_active":"2023-10-16T11:19:22.536996+0000","last_became_peered":"2023-10-16T11:19:22.536996+0000","last_unstale":"2023-10-16T11:19:23.565979+0000","last_undegraded":"2023-10-16T11:19:23.565979+0000","last_fullsized":"2023-10-16T11:19:23.565979+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:22.462616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564664+0000","last_change":"2023-10-16T11:19:23.564018+0000","last_active":"2023-10-16T11:19:23.564664+0000","last_peered":"2023-10-16T11:19:23.564664+0000","last_clean":"2023-10-16T11:19:23.564664+0000","last_became_active":"2023-10-16T11:19:23.563738+0000","last_became_peered":"2023-10-16T11:19:23.563738+0000","last_unstale":"2023-10-16T11:19:23.564664+0000","last_undegraded":"2023-10-16T11:19:23.564664+0000","last_fullsized":"2023-10-16T11:19:23.564664+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:29:06.096292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539759+0000","last_change":"2023-10-16T11:19:22.537758+0000","last_active":"2023-10-16T11:19:24.539759+0000","last_peered":"2023-10-16T11:19:24.539759+0000","last_clean":"2023-10-16T11:19:24.539759+0000","last_became_active":"2023-10-16T11:19:22.537201+0000","last_became_peered":"2023-10-16T11:19:22.537201+0000","last_unstale":"2023-10-16T11:19:24.539759+0000","last_undegraded":"2023-10-16T11:19:24.539759+0000","last_fullsized":"2023-10-16T11:19:24.539759+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:26.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539580+0000","last_change":"2023-10-16T11:19:23.556539+0000","last_active":"2023-10-16T11:19:24.539580+0000","last_peered":"2023-10-16T11:19:24.539580+0000","last_clean":"2023-10-16T11:19:24.539580+0000","last_became_active":"2023-10-16T11:19:23.556209+0000","last_became_peered":"2023-10-16T11:19:23.556209+0000","last_unstale":"2023-10-16T11:19:24.539580+0000","last_undegraded":"2023-10-16T11:19:24.539580+0000","last_fullsized":"2023-10-16T11:19:24.539580+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540930+0000","last_change":"2023-10-16T11:19:22.543844+0000","last_active":"2023-10-16T11:19:23.540930+0000","last_peered":"2023-10-16T11:19:23.540930+0000","last_clean":"2023-10-16T11:19:23.540930+0000","last_became_active":"2023-10-16T11:19:22.543538+0000","last_became_peered":"2023-10-16T11:19:22.543538+0000","last_unstale":"2023-10-16T11:19:23.540930+0000","last_undegraded":"2023-10-16T11:19:23.540930+0000","last_fullsized":"2023-10-16T11:19:23.540930+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:00.946603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.774005+0000","last_change":"2023-10-16T11:19:23.546472+0000","last_active":"2023-10-16T11:19:24.774005+0000","last_peered":"2023-10-16T11:19:24.774005+0000","last_clean":"2023-10-16T11:19:24.774005+0000","last_became_active":"2023-10-16T11:19:23.546160+0000","last_became_peered":"2023-10-16T11:19:23.546160+0000","last_unstale":"2023-10-16T11:19:24.774005+0000","last_undegraded":"2023-10-16T11:19:24.774005+0000","last_fullsized":"2023-10-16T11:19:24.774005+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:06.395224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.548518+0000","last_change":"2023-10-16T11:19:22.537543+0000","last_active":"2023-10-16T11:19:23.548518+0000","last_peered":"2023-10-16T11:19:23.548518+0000","last_clean":"2023-10-16T11:19:23.548518+0000","last_became_active":"2023-10-16T11:19:22.537050+0000","last_became_peered":"2023-10-16T11:19:22.537050+0000","last_unstale":"2023-10-16T11:19:23.548518+0000","last_undegraded":"2023-10-16T11:19:23.548518+0000","last_fullsized":"2023-10-16T11:19:23.548518+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:03.218818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.553445+0000","last_change":"2023-10-16T11:19:23.546380+0000","last_active":"2023-10-16T11:19:23.553445+0000","last_peered":"2023-10-16T11:19:23.553445+0000","last_clean":"2023-10-16T11:19:23.553445+0000","last_became_active":"2023-10-16T11:19:23.546098+0000","last_became_peered":"2023-10-16T11:19:23.546098+0000","last_unstale":"2023-10-16T11:19:23.553445+0000","last_undegraded":"2023-10-16T11:19:23.553445+0000","last_fullsized":"2023-10-16T11:19:23.553445+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:06.791712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539990+0000","last_change":"2023-10-16T11:19:22.543881+0000","last_active":"2023-10-16T11:19:24.539990+0000","last_peered":"2023-10-16T11:19:24.539990+0000","last_clean":"2023-10-16T11:19:24.539990+0000","last_became_active":"2023-10-16T11:19:22.543647+0000","last_became_peered":"2023-10-16T11:19:22.543647+0000","last_unstale":"2023-10-16T11:19:24.539990+0000","last_undegraded":"2023-10-16T11:19:24.539990+0000","last_fullsized":"2023-10-16T11:19:24.539990+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:41.755970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564603+0000","last_change":"2023-10-16T11:19:23.558505+0000","last_active":"2023-10-16T11:19:23.564603+0000","last_peered":"2023-10-16T11:19:23.564603+0000","last_clean":"2023-10-16T11:19:23.564603+0000","last_became_active":"2023-10-16T11:19:23.558188+0000","last_became_peered":"2023-10-16T11:19:23.558188+0000","last_unstale":"2023-10-16T11:19:23.564603+0000","last_undegraded":"2023-10-16T11:19:23.564603+0000","last_fullsized":"2023-10-16T11:19:23.564603+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:35.177289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540372+0000","last_change":"2023-10-16T11:19:22.544852+0000","last_active":"2023-10-16T11:19:23.540372+0000","last_peered":"2023-10-16T11:19:23.540372+0000","last_clean":"2023-10-16T11:19:23.540372+0000","last_became_active":"2023-10-16T11:19:22.544565+0000","last_became_peered":"2023-10-16T11:19:22.544565+0000","last_unstale":"2023-10-16T11:19:23.540372+0000","last_undegraded":"2023-10-16T11:19:23.540372+0000","last_fullsized":"2023-10-16T11:19:23.540372+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:07.144566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564170+0000","last_change":"2023-10-16T11:19:23.563504+0000","last_active":"2023-10-16T11:19:23.564170+0000","last_peered":"2023-10-16T11:19:23.564170+0000","last_clean":"2023-10-16T11:19:23.564170+0000","last_became_active":"2023-10-16T11:19:23.563243+0000","last_became_peered":"2023-10-16T11:19:23.563243+0000","last_unstale":"2023-10-16T11:19:23.564170+0000","last_undegraded":"2023-10-16T11:19:23.564170+0000","last_fullsized":"2023-10-16T11:19:23.564170+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.773857+0000","last_change":"2023-10-16T11:19:22.539000+0000","last_active":"2023-10-16T11:19:24.773857+0000","last_peered":"2023-10-16T11:19:24.773857+0000","last_clean":"2023-10-16T11:19:24.773857+0000","last_became_active":"2023-10-16T11:19:22.538728+0000","last_became_peered":"2023-10-16T11:19:22.538728+0000","last_unstale":"2023-10-16T11:19:24.773857+0000","last_undegraded":"2023-10-16T11:19:24.773857+0000","last_fullsized":"2023-10-16T11:19:24.773857+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:03.944394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.539267+0000","last_change":"2023-10-16T11:19:23.563089+0000","last_active":"2023-10-16T11:19:24.539267+0000","last_peered":"2023-10-16T11:19:24.539267+0000","last_clean":"2023-10-16T11:19:24.539267+0000","last_became_active":"2023-10-16T11:19:23.562759+0000","last_became_peered":"2023-10-16T11:19:23.562759+0000","last_unstale":"2023-10-16T11:19:24.539267+0000","last_undegraded":"2023-10-16T11:19:24.539267+0000","last_fullsized":"2023-10-16T11:19:24.539267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.559527+0000","last_change":"2023-10-16T11:19:22.537241+0000","last_active":"2023-10-16T11:19:23.559527+0000","last_peered":"2023-10-16T11:19:23.559527+0000","last_clean":"2023-10-16T11:19:23.559527+0000","last_became_active":"2023-10-16T11:19:22.536947+0000","last_became_peered":"2023-10-16T11:19:22.536947+0000","last_unstale":"2023-10-16T11:19:23.559527+0000","last_undegraded":"2023-10-16T11:19:23.559527+0000","last_fullsized":"2023-10-16T11:19:23.559527+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.555402+0000","last_change":"2023-10-16T11:19:23.551428+0000","last_active":"2023-10-16T11:19:23.555402+0000","last_peered":"2023-10-16T11:19:23.555402+0000","last_clean":"2023-10-16T11:19:23.555402+0000","last_became_active":"2023-10-16T11:19:23.551083+0000","last_became_peered":"2023-10-16T11:19:23.551083+0000","last_unstale":"2023-10-16T11:19:23.555402+0000","last_undegraded":"2023-10-16T11:19:23.555402+0000","last_fullsized":"2023-10-16T11:19:23.555402+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:32:09.096145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.564650+0000","last_change":"2023-10-16T11:19:22.536415+0000","last_active":"2023-10-16T11:19:23.564650+0000","last_peered":"2023-10-16T11:19:23.564650+0000","last_clean":"2023-10-16T11:19:23.564650+0000","last_became_active":"2023-10-16T11:19:22.536117+0000","last_became_peered":"2023-10-16T11:19:22.536117+0000","last_unstale":"2023-10-16T11:19:23.564650+0000","last_undegraded":"2023-10-16T11:19:23.564650+0000","last_fullsized":"2023-10-16T11:19:23.564650+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:07.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"206'348","reported_seq":1209,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-16T11:19:23.540937+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:19:23.540937+0000","last_peered":"2023-10-16T11:19:23.540937+0000","last_clean":"2023-10-16T11:19:23.540937+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:19:23.540937+0000","last_undegraded":"2023-10-16T11:19:23.540937+0000","last_fullsized":"2023-10-16T11:19:23.540937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543279,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52472,"kb_used_data":3236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319368,"statfs":{"total":96636764160,"available":96583032832,"internally_reserved":0,"allocated":3313664,"data_stored":1979940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.27800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67500000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42399999999999999,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.76300000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.79900000000000004,"15min":0.86299999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.307},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.002},"last":0.80300000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33100000000000002},"max":{"1min":8.1950000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.83699999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.871},"last":0.81899999999999995}]}]},{"osd":6,"up_from":14,"seq":60129543278,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37976,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333864,"statfs":{"total":96636764160,"available":96597876736,"internally_reserved":0,"allocated":2699264,"data_stored":1381690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.875},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.51200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.83099999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.78700000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.752,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.32100000000000001},"max":{"1min":0.79500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.91100000000000003},"last":0.59499999999999997}]}]},{"osd":1,"up_from":14,"seq":60129543278,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60196,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311644,"statfs":{"total":96636764160,"available":96575123456,"internally_reserved":0,"allocated":2703360,"data_stored":1381164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.224},"max":{"1min":0.60199999999999998,"5min":0.629,"15min":0.68700000000000006},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.434},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.62,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.55300000000000005,"5min":0.58699999999999997,"15min":0.98899999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.33600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.78300000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.0289999999999999,"15min":1.097},"last":0.57499999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":1.147},"last":0.72699999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.88,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":0.98899999999999999,"5min":1.004,"15min":1.0600000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46300000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.1060000000000001},"last":0.77700000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543279,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60772,"kb_used_data":3216,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311068,"statfs":{"total":96636764160,"available":96574533632,"internally_reserved":0,"allocated":3293184,"data_stored":1971566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.39900000000000002,"15min":0.39800000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.214},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.54400000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38800000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.25800000000000001},"max":{"1min":0.59599999999999997,"5min":0.67300000000000004,"15min":0.70299999999999996},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.33700000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.58499999999999996,"5min":0.66200000000000003,"15min":0.68300000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32900000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.374}]},{"osd":4,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":0.95799999999999996,"15min":0.99199999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.67100000000000004,"15min":0.67200000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.85599999999999998,"5min":0.93700000000000006,"15min":1.008},"last":0.54000000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.872,"5min":1.004,"15min":1.008},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68400000000000005,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.94199999999999995,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.48899999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.78400000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91000000000000003,"5min":1.01,"15min":1.0449999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.44900000000000001},"max":{"1min":0.81499999999999995,"5min":0.95899999999999996,"15min":1.077},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.875,"5min":0.95699999999999996,"15min":0.96599999999999997},"last":0.629}]}]},{"osd":2,"up_from":14,"seq":60129543278,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60188,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311652,"statfs":{"total":96636764160,"available":96575131648,"internally_reserved":0,"allocated":2695168,"data_stored":1381108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.63},"last":0.378}]},{"osd":1,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":0.435},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":0.48399999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.623,"15min":0.65500000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.81200000000000006},"last":0.502}]},{"osd":4,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.97599999999999998},"last":0.71799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":1.052,"15min":1.052},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.66000000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68700000000000006,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64900000000000002}]}]},{"osd":3,"up_from":14,"seq":60129543279,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47732,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324108,"statfs":{"total":96636764160,"available":96587886592,"internally_reserved":0,"allocated":2719744,"data_stored":1388142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.442},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.36799999999999999,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.46300000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.378},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":0.60699999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.499}]},{"osd":4,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":1.034,"15min":1.0740000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67300000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.749},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.0489999999999999},"last":0.73099999999999998}]}]},{"osd":4,"up_from":14,"seq":60129543278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60220,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311620,"statfs":{"total":96636764160,"available":96575098880,"internally_reserved":0,"allocated":2727936,"data_stored":1389676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.628},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.48699999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.627},"min":{"1min":0.438,"5min":0.40400000000000003,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60399999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.879},"last":0.81100000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.69499999999999995,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.89500000000000002},"last":0.46100000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.40600000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":0.875,"15min":0.875},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.51200000000000001}]}]},{"osd":5,"up_from":14,"seq":60129543279,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42156,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329684,"statfs":{"total":96636764160,"available":96593596416,"internally_reserved":0,"allocated":2711552,"data_stored":1388542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":1.873,"15min":1.873},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.46200000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":1.821,"15min":1.821},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44700000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":1.919,"15min":1.919},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.873},"last":0.498},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.63},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":2.02,"15min":2.02},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:19:28.049 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:19:28.049 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:19:28.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.207+0000 7f34fc547700 1 Processor -- start 2023-10-16T11:19:28.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.207+0000 7f34fc547700 1 -- start start 2023-10-16T11:19:28.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.208+0000 7f34fc547700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34f40cc6e0 0x7f34f4071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.208+0000 7f34fc547700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40ce680 0x7f34f40ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.208+0000 7f34fc547700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f40ce1f0 con 0x7f34f40b2010 2023-10-16T11:19:28.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.208+0000 7f34fc547700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f40ccb90 con 0x7f34f40cc6e0 2023-10-16T11:19:28.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.208+0000 7f34fc547700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f4072120 con 0x7f34f40ce680 2023-10-16T11:19:28.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34fa2e3700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f34f40b2010 0x7f34f40b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38346/0 (socket says 172.21.15.181:38346) 2023-10-16T11:19:28.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34fa2e3700 1 -- 172.21.15.181:0/220979016 learned_addr learned my addr 172.21.15.181:0/220979016 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:28.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f9ae2700 1 --2- 172.21.15.181:0/220979016 >> v2:172.21.15.190:3300/0 conn(0x7f34f40cc6e0 0x7f34f4071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34faae4700 1 --2- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40ce680 0x7f34f40ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f92e1700 1 -- 172.21.15.181:0/220979016 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012432967 0 0) 0x7f34f40ce1f0 con 0x7f34f40b2010 2023-10-16T11:19:28.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f92e1700 1 -- 172.21.15.181:0/220979016 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e00035b0 con 0x7f34f40b2010 2023-10-16T11:19:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f92e1700 1 -- 172.21.15.181:0/220979016 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2498443344 0 0) 0x7f34e00035b0 con 0x7f34f40b2010 2023-10-16T11:19:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f9ae2700 1 -- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40ce680 msgr2=0x7f34f40ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f9ae2700 1 --2- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40ce680 0x7f34f40ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f9ae2700 1 -- 172.21.15.181:0/220979016 >> v1:172.21.15.181:6789/0 conn(0x7f34f40b2010 legacy=0x7f34f40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f9ae2700 1 -- 172.21.15.181:0/220979016 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f40ccda0 con 0x7f34f40cc6e0 2023-10-16T11:19:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34faae4700 1 --2- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40ce680 0x7f34f40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f9ae2700 1 --2- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40cc6e0 0x7f34f4071be0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009840 tx=0x7f34f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=645ff6071ea4623a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f92e1700 1 -- 172.21.15.181:0/220979016 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4001670 con 0x7f34f40cc6e0 2023-10-16T11:19:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f92e1700 1 -- 172.21.15.181:0/220979016 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f0004030 con 0x7f34f40cc6e0 2023-10-16T11:19:28.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.209+0000 7f34f92e1700 1 -- 172.21.15.181:0/220979016 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00043c0 con 0x7f34f40cc6e0 2023-10-16T11:19:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 -- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40cc6e0 msgr2=0x7f34f4071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 --2- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40cc6e0 0x7f34f4071be0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009840 tx=0x7f34f0009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 -- 172.21.15.181:0/220979016 shutdown_connections 2023-10-16T11:19:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 --2- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40ce680 0x7f34f40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 --2- 172.21.15.181:0/220979016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40cc6e0 0x7f34f4071be0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 -- 172.21.15.181:0/220979016 >> 172.21.15.181:0/220979016 conn(0x7f34f40cd010 msgr2=0x7f34f40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 -- 172.21.15.181:0/220979016 shutdown_connections 2023-10-16T11:19:28.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.210+0000 7f34fc547700 1 -- 172.21.15.181:0/220979016 wait complete. 2023-10-16T11:19:28.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.211+0000 7f34fc547700 1 Processor -- start 2023-10-16T11:19:28.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.211+0000 7f34fc547700 1 -- start start 2023-10-16T11:19:28.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fc547700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40b2010 0x7f34f40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fc547700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40cc6e0 0x7f34f414f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fc547700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34f40ce680 0x7f34f41f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fc547700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f40b3f50 con 0x7f34f40ce680 2023-10-16T11:19:28.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fc547700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f4152680 con 0x7f34f40b2010 2023-10-16T11:19:28.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fc547700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f41f1670 con 0x7f34f40cc6e0 2023-10-16T11:19:28.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34f40ce680 0x7f34f41f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34f40ce680 0x7f34f41f0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55510/0 (socket says 172.21.15.181:55510) 2023-10-16T11:19:28.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 -- 172.21.15.181:0/1632414454 learned_addr learned my addr 172.21.15.181:0/1632414454 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:28.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34f9ae2700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40cc6e0 0x7f34f414f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fa2e3700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40b2010 0x7f34f40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 -- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40cc6e0 msgr2=0x7f34f414f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40cc6e0 0x7f34f414f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 -- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40b2010 msgr2=0x7f34f40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40b2010 0x7f34f40b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34faae4700 1 -- 172.21.15.181:0/1632414454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec009a70 con 0x7f34f40ce680 2023-10-16T11:19:28.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34fa2e3700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40b2010 0x7f34f40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:28.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.212+0000 7f34f9ae2700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40cc6e0 0x7f34f414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:28.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.213+0000 7f34faae4700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34f40ce680 0x7f34f41f0bf0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00efc0 tx=0x7f34ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:28.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.213+0000 7f34eaffd700 1 -- 172.21.15.181:0/1632414454 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec009d70 con 0x7f34f40ce680 2023-10-16T11:19:28.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.213+0000 7f34eaffd700 1 -- 172.21.15.181:0/1632414454 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec00f040 con 0x7f34f40ce680 2023-10-16T11:19:28.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.213+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f00094a0 con 0x7f34f40ce680 2023-10-16T11:19:28.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.213+0000 7f34eaffd700 1 -- 172.21.15.181:0/1632414454 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00ce00 con 0x7f34f40ce680 2023-10-16T11:19:28.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.213+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f41f1c30 con 0x7f34f40ce680 2023-10-16T11:19:28.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.214+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f40b6510 con 0x7f34f40ce680 2023-10-16T11:19:28.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.214+0000 7f34eaffd700 1 -- 172.21.15.181:0/1632414454 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f34ec014480 con 0x7f34f40ce680 2023-10-16T11:19:28.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.215+0000 7f34eaffd700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f34d806b020 0x7f34d806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.215+0000 7f34eaffd700 1 -- 172.21.15.181:0/1632414454 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f34ec08a200 con 0x7f34f40ce680 2023-10-16T11:19:28.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.215+0000 7f34fa2e3700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f34d806b020 0x7f34d806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.215+0000 7f34fa2e3700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f34d806b020 0x7f34d806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34e4001eb0 tx=0x7f34e4001ce0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:28.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.219+0000 7f34eaffd700 1 -- 172.21.15.181:0/1632414454 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f34ec054ef0 con 0x7f34f40ce680 2023-10-16T11:19:28.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.487+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f34f40b2800 con 0x7f34f40ce680 2023-10-16T11:19:28.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.489+0000 7f34eaffd700 1 -- 172.21.15.181:0/1632414454 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v51) v1 ==== 190+0+147 (secure 0 0 0) 0x7f34ec058510 con 0x7f34f40ce680 2023-10-16T11:19:28.491 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T11:19:28.492 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T11:19:28.492 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T11:19:28.492 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T11:19:28.492 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:19:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f34d806b020 msgr2=0x7f34d806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f34d806b020 0x7f34d806d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34e4001eb0 tx=0x7f34e4001ce0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34f40ce680 msgr2=0x7f34f41f0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34f40ce680 0x7f34f41f0bf0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00efc0 tx=0x7f34ec00c3a0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 shutdown_connections 2023-10-16T11:19:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f34d806b020 0x7f34d806d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34f40ce680 0x7f34f41f0bf0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34f40b2010 0x7f34f40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 --2- 172.21.15.181:0/1632414454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34f40cc6e0 0x7f34f414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 >> 172.21.15.181:0/1632414454 conn(0x7f34f40cd010 msgr2=0x7f34f40b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 shutdown_connections 2023-10-16T11:19:28.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.492+0000 7f34fc547700 1 -- 172.21.15.181:0/1632414454 wait complete. 2023-10-16T11:19:28.497 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T11:19:28.506 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:28.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.625+0000 7f615b1a6700 1 Processor -- start 2023-10-16T11:19:28.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.625+0000 7f615b1a6700 1 -- start start 2023-10-16T11:19:28.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f615b1a6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f615415a680 0x7f61540cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f615b1a6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61540cd220 0x7f6154164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f615b1a6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61540cb3b0 con 0x7f61540d0c80 2023-10-16T11:19:28.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f615b1a6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61540cbd50 con 0x7f61540cd220 2023-10-16T11:19:28.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f615b1a6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61540cb690 con 0x7f615415a680 2023-10-16T11:19:28.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f6153fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f61540d0c80 0x7f61540ccb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38360/0 (socket says 172.21.15.181:38360) 2023-10-16T11:19:28.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f6153fff700 1 -- 172.21.15.181:0/2080208050 learned_addr learned my addr 172.21.15.181:0/2080208050 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:28.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f6159743700 1 --2- 172.21.15.181:0/2080208050 >> v2:172.21.15.190:3300/0 conn(0x7f61540cd220 0x7f6154164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f6158f42700 1 --2- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f615415a680 0x7f61540cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f61537fe700 1 -- 172.21.15.181:0/2080208050 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552929952 0 0) 0x7f61540cb3b0 con 0x7f61540d0c80 2023-10-16T11:19:28.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.626+0000 7f61537fe700 1 -- 172.21.15.181:0/2080208050 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61440035b0 con 0x7f61540d0c80 2023-10-16T11:19:28.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f61537fe700 1 -- 172.21.15.181:0/2080208050 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3139700872 0 0) 0x7f61440035b0 con 0x7f61540d0c80 2023-10-16T11:19:28.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f6159743700 1 -- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f615415a680 msgr2=0x7f61540cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:28.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f6159743700 1 --2- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f615415a680 0x7f61540cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f6159743700 1 -- 172.21.15.181:0/2080208050 >> v1:172.21.15.181:6789/0 conn(0x7f61540d0c80 legacy=0x7f61540ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f6159743700 1 -- 172.21.15.181:0/2080208050 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540cb970 con 0x7f61540cd220 2023-10-16T11:19:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f6158f42700 1 --2- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f615415a680 0x7f61540cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f6159743700 1 --2- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61540cd220 0x7f6154164710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f614c009d90 tx=0x7f614c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a3f09a2d85b9d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f61537fe700 1 -- 172.21.15.181:0/2080208050 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00e070 con 0x7f61540cd220 2023-10-16T11:19:28.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f61537fe700 1 -- 172.21.15.181:0/2080208050 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c003a20 con 0x7f61540cd220 2023-10-16T11:19:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.627+0000 7f61537fe700 1 -- 172.21.15.181:0/2080208050 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140001670 con 0x7f61540cd220 2023-10-16T11:19:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 -- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61540cd220 msgr2=0x7f6154164710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 --2- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61540cd220 0x7f6154164710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f614c009d90 tx=0x7f614c00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 -- 172.21.15.181:0/2080208050 shutdown_connections 2023-10-16T11:19:28.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 --2- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61540cd220 0x7f6154164710 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 --2- 172.21.15.181:0/2080208050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f615415a680 0x7f61540cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 -- 172.21.15.181:0/2080208050 >> 172.21.15.181:0/2080208050 conn(0x7f6154099a00 msgr2=0x7f61540d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 -- 172.21.15.181:0/2080208050 shutdown_connections 2023-10-16T11:19:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.628+0000 7f615b1a6700 1 -- 172.21.15.181:0/2080208050 wait complete. 2023-10-16T11:19:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.629+0000 7f615b1a6700 1 Processor -- start 2023-10-16T11:19:28.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.629+0000 7f615b1a6700 1 -- start start 2023-10-16T11:19:28.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cd220 0x7f61541556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61540d0c80 0x7f61540b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f615415a680 0x7f6154157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540b5f10 con 0x7f61540cd220 2023-10-16T11:19:28.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540c9680 con 0x7f615415a680 2023-10-16T11:19:28.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61540c9960 con 0x7f61540d0c80 2023-10-16T11:19:28.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cd220 0x7f61541556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cd220 0x7f61541556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55520/0 (socket says 172.21.15.181:55520) 2023-10-16T11:19:28.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 -- 172.21.15.181:0/3309902042 learned_addr learned my addr 172.21.15.181:0/3309902042 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:28.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 -- 172.21.15.181:0/3309902042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61540d0c80 msgr2=0x7f61540b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61540d0c80 0x7f61540b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 -- 172.21.15.181:0/3309902042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f615415a680 msgr2=0x7f6154157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f615415a680 0x7f6154157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 -- 172.21.15.181:0/3309902042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6148009430 con 0x7f61540cd220 2023-10-16T11:19:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f6158f42700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cd220 0x7f61541556f0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f6148010df0 tx=0x7f614800e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f61517fa700 1 -- 172.21.15.181:0/3309902042 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148005c20 con 0x7f61540cd220 2023-10-16T11:19:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f61517fa700 1 -- 172.21.15.181:0/3309902042 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6148005d80 con 0x7f61540cd220 2023-10-16T11:19:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f61517fa700 1 -- 172.21.15.181:0/3309902042 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148006250 con 0x7f61540cd220 2023-10-16T11:19:28.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c009a70 con 0x7f61540cd220 2023-10-16T11:19:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.630+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61540c9f20 con 0x7f61540cd220 2023-10-16T11:19:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.631+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61540cddc0 con 0x7f61540cd220 2023-10-16T11:19:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.632+0000 7f61517fa700 1 -- 172.21.15.181:0/3309902042 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f61480063b0 con 0x7f61540cd220 2023-10-16T11:19:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.632+0000 7f61517fa700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f613806b020 0x7f613806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.632+0000 7f61517fa700 1 -- 172.21.15.181:0/3309902042 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f614808be60 con 0x7f61540cd220 2023-10-16T11:19:28.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.632+0000 7f6153fff700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f613806b020 0x7f613806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:28.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.633+0000 7f6153fff700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f613806b020 0x7f613806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6140001eb0 tx=0x7f6140001a20 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:28.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.637+0000 7f61517fa700 1 -- 172.21.15.181:0/3309902042 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6148056b40 con 0x7f61540cd220 2023-10-16T11:19:28.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.871+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61540b1090 con 0x7f61540cd220 2023-10-16T11:19:28.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.872+0000 7f61517fa700 1 -- 172.21.15.181:0/3309902042 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f614805a160 con 0x7f61540cd220 2023-10-16T11:19:28.874 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:28.874 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":111,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":11325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-16T11:19:28.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f613806b020 msgr2=0x7f613806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f613806b020 0x7f613806d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6140001eb0 tx=0x7f6140001a20 comp rx=0 tx=0).stop 2023-10-16T11:19:28.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cd220 msgr2=0x7f61541556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:28.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cd220 0x7f61541556f0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f6148010df0 tx=0x7f614800e640 comp rx=0 tx=0).stop 2023-10-16T11:19:28.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 shutdown_connections 2023-10-16T11:19:28.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f613806b020 0x7f613806d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f615415a680 0x7f6154157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cd220 0x7f61541556f0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 --2- 172.21.15.181:0/3309902042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61540d0c80 0x7f61540b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 >> 172.21.15.181:0/3309902042 conn(0x7f6154099a00 msgr2=0x7f61540cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 shutdown_connections 2023-10-16T11:19:28.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:28.874+0000 7f615b1a6700 1 -- 172.21.15.181:0/3309902042 wait complete. 2023-10-16T11:19:28.879 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 111 2023-10-16T11:19:29.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:19:22.925980+0000', 'modified': '2023-10-16T11:19:25.961114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11340}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11340': {'gid': 11340, 'name': 'd', 'rank': 0, 'incarnation': 107, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/2252072910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 2252072910}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 2252072910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:19:29.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:19:29.292 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-16T11:19:29.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.420+0000 7f39d2550700 1 Processor -- start 2023-10-16T11:19:29.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.420+0000 7f39d2550700 1 -- start start 2023-10-16T11:19:29.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.421+0000 7f39d2550700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc15df60 0x7f39cc0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.421+0000 7f39d2550700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc0cddf0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.421+0000 7f39d2550700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc0ccf50 con 0x7f39cc0d0fc0 2023-10-16T11:19:29.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.421+0000 7f39d2550700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0af9c0 con 0x7f39cc15df60 2023-10-16T11:19:29.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.421+0000 7f39d0aed700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f39cc06a8a0 0x7f39cc06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38376/0 (socket says 172.21.15.181:38376) 2023-10-16T11:19:29.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.421+0000 7f39d0aed700 1 -- 172.21.15.181:0/3022373873 learned_addr learned my addr 172.21.15.181:0/3022373873 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:29.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39cbfff700 1 --2- 172.21.15.181:0/3022373873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc15df60 0x7f39cc0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546080685 0 0) 0x7f39cc0cddf0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b40035b0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687052629 0 0) 0x7f39cc0ccf50 con 0x7f39cc0d0fc0 2023-10-16T11:19:29.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39cc0cddf0 con 0x7f39cc0d0fc0 2023-10-16T11:19:29.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032583877 0 0) 0x7f39b40035b0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39cbfff700 1 -- 172.21.15.181:0/3022373873 >> v1:172.21.15.190:6789/0 conn(0x7f39cc0d0fc0 legacy=0x7f39cc0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39cbfff700 1 -- 172.21.15.181:0/3022373873 >> v1:172.21.15.181:6789/0 conn(0x7f39cc06a8a0 legacy=0x7f39cc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39cbfff700 1 -- 172.21.15.181:0/3022373873 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc0cc600 con 0x7f39cc15df60 2023-10-16T11:19:29.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39cbfff700 1 --2- 172.21.15.181:0/3022373873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc15df60 0x7f39cc0cae20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009850 tx=0x7f39c0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa4045713a57daf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40019f0 con 0x7f39cc15df60 2023-10-16T11:19:29.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c0005c30 con 0x7f39cc15df60 2023-10-16T11:19:29.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.422+0000 7f39caffd700 1 -- 172.21.15.181:0/3022373873 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0004030 con 0x7f39cc15df60 2023-10-16T11:19:29.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.423+0000 7f39d2550700 1 -- 172.21.15.181:0/3022373873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc15df60 msgr2=0x7f39cc0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.423+0000 7f39d2550700 1 --2- 172.21.15.181:0/3022373873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc15df60 0x7f39cc0cae20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009850 tx=0x7f39c0009b60 comp rx=0 tx=0).stop 2023-10-16T11:19:29.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.423+0000 7f39d2550700 1 -- 172.21.15.181:0/3022373873 shutdown_connections 2023-10-16T11:19:29.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.423+0000 7f39d2550700 1 --2- 172.21.15.181:0/3022373873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc15df60 0x7f39cc0cae20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.423+0000 7f39d2550700 1 -- 172.21.15.181:0/3022373873 >> 172.21.15.181:0/3022373873 conn(0x7f39cc0d04a0 msgr2=0x7f39cc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.423+0000 7f39d2550700 1 -- 172.21.15.181:0/3022373873 shutdown_connections 2023-10-16T11:19:29.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.423+0000 7f39d2550700 1 -- 172.21.15.181:0/3022373873 wait complete. 2023-10-16T11:19:29.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.424+0000 7f39d2550700 1 Processor -- start 2023-10-16T11:19:29.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.424+0000 7f39d2550700 1 -- start start 2023-10-16T11:19:29.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39d2550700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc06a8a0 0x7f39cc15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39d2550700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc0d0fc0 0x7f39cc14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39d2550700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc15df60 0x7f39cc15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39d2550700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc15b4e0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39d2550700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0c7710 con 0x7f39cc15df60 2023-10-16T11:19:29.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39d2550700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0c79f0 con 0x7f39cc0d0fc0 2023-10-16T11:19:29.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc06a8a0 0x7f39cc15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc06a8a0 0x7f39cc15cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55532/0 (socket says 172.21.15.181:55532) 2023-10-16T11:19:29.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 -- 172.21.15.181:0/1172444933 learned_addr learned my addr 172.21.15.181:0/1172444933 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:29.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39d0aed700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc15df60 0x7f39cc15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc0d0fc0 0x7f39cc14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 -- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc0d0fc0 msgr2=0x7f39cc14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc0d0fc0 0x7f39cc14e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 -- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc15df60 msgr2=0x7f39cc15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc15df60 0x7f39cc15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.425+0000 7f39cbfff700 1 -- 172.21.15.181:0/1172444933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c0009430 con 0x7f39cc06a8a0 2023-10-16T11:19:29.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.426+0000 7f39cbfff700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc06a8a0 0x7f39cc15cea0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f39c0004000 tx=0x7f39c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.426+0000 7f39c8ff9700 1 -- 172.21.15.181:0/1172444933 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c00038b0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.426+0000 7f39c8ff9700 1 -- 172.21.15.181:0/1172444933 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c0003a30 con 0x7f39cc06a8a0 2023-10-16T11:19:29.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.426+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc0c7cd0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.426+0000 7f39c8ff9700 1 -- 172.21.15.181:0/1172444933 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0003de0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.426+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc0b32f0 con 0x7f39cc06a8a0 2023-10-16T11:19:29.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.427+0000 7f39c8ff9700 1 -- 172.21.15.181:0/1172444933 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f39c0018030 con 0x7f39cc06a8a0 2023-10-16T11:19:29.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.428+0000 7f39c8ff9700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f39b806b020 0x7f39b806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.428+0000 7f39c8ff9700 1 -- 172.21.15.181:0/1172444933 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f39c008ab10 con 0x7f39cc06a8a0 2023-10-16T11:19:29.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.428+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f39b806b020 0x7f39b806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.428+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f39b806b020 0x7f39b806d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39cc09e460 tx=0x7f39bc009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 Processor -- start 2023-10-16T11:19:29.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 -- start start 2023-10-16T11:19:29.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733844220 0x55d73384ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d73381f010 0x55d73381f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d733795c20 con 0x55d73381f010 2023-10-16T11:19:29.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73377ac10 con 0x55d733849630 2023-10-16T11:19:29.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d733778530 con 0x55d733844220 2023-10-16T11:19:29.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39d0aed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d73381f010 0x55d73381f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39d0aed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d73381f010 0x55d73381f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55544/0 (socket says 172.21.15.181:55544) 2023-10-16T11:19:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.430+0000 7f39d0aed700 1 -- 172.21.15.181:0/384993547 learned_addr learned my addr 172.21.15.181:0/384993547 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39cbfff700 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733844220 0x55d73384ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39d0aed700 1 -- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733844220 msgr2=0x55d73384ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39d0aed700 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733844220 0x55d73384ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39d0aed700 1 -- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 msgr2=0x55d73384bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39d0aed700 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39d0aed700 1 -- 172.21.15.181:0/384993547 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7337313a0 con 0x55d73381f010 2023-10-16T11:19:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39cbfff700 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733844220 0x55d73384ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39d0aed700 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d73381f010 0x55d73381f6c0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f39c4007270 tx=0x7f39c4007580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473d8fef98747967 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39b1ffb700 1 -- 172.21.15.181:0/384993547 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4001840 con 0x55d73381f010 2023-10-16T11:19:29.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39b1ffb700 1 -- 172.21.15.181:0/384993547 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4012560 con 0x55d73381f010 2023-10-16T11:19:29.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.431+0000 7f39b1ffb700 1 -- 172.21.15.181:0/384993547 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40128f0 con 0x55d73381f010 2023-10-16T11:19:29.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 -- 172.21.15.181:0/384993547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d73381f010 msgr2=0x55d73381f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d73381f010 0x55d73381f6c0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f39c4007270 tx=0x7f39c4007580 comp rx=0 tx=0).stop 2023-10-16T11:19:29.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 -- 172.21.15.181:0/384993547 shutdown_connections 2023-10-16T11:19:29.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733844220 0x55d73384ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d73381f010 0x55d73381f6c0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 --2- 172.21.15.181:0/384993547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 -- 172.21.15.181:0/384993547 >> 172.21.15.181:0/384993547 conn(0x55d73374ed80 msgr2=0x55d733869ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 -- 172.21.15.181:0/384993547 shutdown_connections 2023-10-16T11:19:29.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 -- 172.21.15.181:0/384993547 wait complete. 2023-10-16T11:19:29.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 -- start start 2023-10-16T11:19:29.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.432+0000 7f39de73ab80 1 Processor -- start 2023-10-16T11:19:29.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d73381f010 0x55d7338606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733849630 0x55d733842b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39de73ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d733773e20 con 0x55d733844220 2023-10-16T11:19:29.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7337106c0 con 0x55d73381f010 2023-10-16T11:19:29.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d733710360 con 0x55d733849630 2023-10-16T11:19:29.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39de73ab80 10 client.0 _ll_register_callbacks cb 0x55d73356d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:19:29.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55548/0 (socket says 172.21.15.181:55548) 2023-10-16T11:19:29.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39cb7fe700 1 -- 172.21.15.181:0/791000587 learned_addr learned my addr 172.21.15.181:0/791000587 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:29.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39cbfff700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d73381f010 0x55d7338606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.433+0000 7f39d0aed700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733849630 0x55d733842b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39cb7fe700 1 -- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733849630 msgr2=0x55d733842b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733849630 0x55d733842b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39cb7fe700 1 -- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d73381f010 msgr2=0x55d7338606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d73381f010 0x55d7338606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39cb7fe700 1 -- 172.21.15.181:0/791000587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c40019a0 con 0x55d733844220 2023-10-16T11:19:29.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39cbfff700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d73381f010 0x55d7338606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:29.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39d0aed700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733849630 0x55d733842b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:29.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384b790 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0011e0 tx=0x7f39bc001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c49ec7be16324b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:19:29.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39aaffd700 1 -- 172.21.15.181:0/791000587 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc013800 con 0x55d733844220 2023-10-16T11:19:29.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39aaffd700 1 -- 172.21.15.181:0/791000587 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f39bc013960 con 0x55d733844220 2023-10-16T11:19:29.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7338145b0 con 0x55d733844220 2023-10-16T11:19:29.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39aaffd700 1 -- 172.21.15.181:0/791000587 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39bc013b20 con 0x55d733844220 2023-10-16T11:19:29.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 10 client.11472 fetch_fsmap learned FSMap version 111 2023-10-16T11:19:29.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55d73359ed90 con 0x55d733844220 2023-10-16T11:19:29.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39aaffd700 1 -- 172.21.15.181:0/791000587 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d7338145b0 con 0x55d733844220 2023-10-16T11:19:29.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39aaffd700 1 -- 172.21.15.181:0/791000587 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39bc013c80 con 0x55d733844220 2023-10-16T11:19:29.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 10 client.11472 fetch_fsmap finished waiting for FSMap version 111 2023-10-16T11:19:29.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 10 client.11472 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T11:19:29.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 20 client.11472 populate_metadata read hostname 'smithi181' 2023-10-16T11:19:29.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d733879a80 0x55d73384c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 4 client.11472 mds_command: new command op to 11340 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:19:29.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.434+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 --> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d73376ce00 con 0x55d733879a80 2023-10-16T11:19:29.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.435+0000 7f39cbfff700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d733879a80 0x55d73384c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.435+0000 7f39aaffd700 10 client.11472 ms_handle_connect on v2:172.21.15.190:6833/2252072910 2023-10-16T11:19:29.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.435+0000 7f39cbfff700 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d733879a80 0x55d73384c730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c72c341f3189c81 server_cookie=a027fd2a421e5d9c in_seq=0 out_seq=0 2023-10-16T11:19:29.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39aaffd700 1 -- 172.21.15.181:0/791000587 <== mds.0 v2:172.21.15.190:6833/2252072910 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d73376ce00 con 0x55d733879a80 2023-10-16T11:19:29.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39aaffd700 10 client.11472 handle_command_reply: tid=0 2023-10-16T11:19:29.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 client.11472 shutdown 2023-10-16T11:19:29.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 msgr2=0x55d73384b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384b790 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0011e0 tx=0x7f39bc001150 comp rx=0 tx=0).stop 2023-10-16T11:19:29.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 shutdown_connections 2023-10-16T11:19:29.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d733849630 0x55d733842b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d733879a80 0x55d73384c730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d73381f010 0x55d7338606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 --2- 172.21.15.181:0/791000587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384b790 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39aaffd700 0 client.11472 ms_handle_reset on v2:172.21.15.190:6833/2252072910 2023-10-16T11:19:29.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 >> 172.21.15.181:0/791000587 conn(0x55d73374ed80 msgr2=0x55d7338674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 shutdown_connections 2023-10-16T11:19:29.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 1 -- 172.21.15.181:0/791000587 wait complete. 2023-10-16T11:19:29.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.436+0000 7f39de73ab80 20 client.11472 trim_cache size 0 max 16384 2023-10-16T11:19:29.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 Processor -- start 2023-10-16T11:19:29.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 -- start start 2023-10-16T11:19:29.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d7338888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d733869f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7335f48d0 con 0x55d733844220 2023-10-16T11:19:29.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7337fe490 con 0x55d733849630 2023-10-16T11:19:29.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d73384e7e0 con 0x55d73381f010 2023-10-16T11:19:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.454+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d733869f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d733869f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55556/0 (socket says 172.21.15.181:55556) 2023-10-16T11:19:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 -- 172.21.15.181:0/1232067694 learned_addr learned my addr 172.21.15.181:0/1232067694 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39d0aed700 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cbfff700 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d7338888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 -- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 msgr2=0x55d7338888d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d7338888d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 -- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 msgr2=0x55d73384eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 -- 172.21.15.181:0/1232067694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d733847ff0 con 0x55d733844220 2023-10-16T11:19:29.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cbfff700 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d7338888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39d0aed700 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d733869f90 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0079d0 tx=0x7f39bc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f49507acf7a9da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39b27fc700 1 -- 172.21.15.181:0/1232067694 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc013700 con 0x55d733844220 2023-10-16T11:19:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39b27fc700 1 -- 172.21.15.181:0/1232067694 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc013860 con 0x55d733844220 2023-10-16T11:19:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.455+0000 7f39b27fc700 1 -- 172.21.15.181:0/1232067694 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0139d0 con 0x55d733844220 2023-10-16T11:19:29.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 -- 172.21.15.181:0/1232067694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 msgr2=0x55d733869f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d733869f90 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0079d0 tx=0x7f39bc01a000 comp rx=0 tx=0).stop 2023-10-16T11:19:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 -- 172.21.15.181:0/1232067694 shutdown_connections 2023-10-16T11:19:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d7338888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 --2- 172.21.15.181:0/1232067694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d733869f90 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 -- 172.21.15.181:0/1232067694 >> 172.21.15.181:0/1232067694 conn(0x55d73374ed80 msgr2=0x55d73381d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 -- 172.21.15.181:0/1232067694 shutdown_connections 2023-10-16T11:19:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 -- 172.21.15.181:0/1232067694 wait complete. 2023-10-16T11:19:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 -- start start 2023-10-16T11:19:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.456+0000 7f39de73ab80 1 Processor -- start 2023-10-16T11:19:29.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d733866be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39de73ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39de73ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73384e470 con 0x55d733844220 2023-10-16T11:19:29.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d733864640 con 0x55d733849630 2023-10-16T11:19:29.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39de73ab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7338671e0 con 0x55d73381f010 2023-10-16T11:19:29.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39de73ab80 10 client.0 _ll_register_callbacks cb 0x7f39bc0053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:19:29.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55562/0 (socket says 172.21.15.181:55562) 2023-10-16T11:19:29.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39cb7fe700 1 -- 172.21.15.181:0/146227359 learned_addr learned my addr 172.21.15.181:0/146227359 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:29.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39d0aed700 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.457+0000 7f39cbfff700 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d733866be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39cb7fe700 1 -- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 msgr2=0x55d733866be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d733866be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39cb7fe700 1 -- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 msgr2=0x55d73384eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39cb7fe700 1 -- 172.21.15.181:0/146227359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d73359ed90 con 0x55d733844220 2023-10-16T11:19:29.477 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39cb7fe700 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384e9b0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0011e0 tx=0x7f39bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9eaa0ea0e4f080 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:19:29.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39aaffd700 1 -- 172.21.15.181:0/146227359 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc013be0 con 0x55d733844220 2023-10-16T11:19:29.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39c4012560 con 0x55d733844220 2023-10-16T11:19:29.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39aaffd700 1 -- 172.21.15.181:0/146227359 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f39bc013d40 con 0x55d733844220 2023-10-16T11:19:29.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39aaffd700 1 -- 172.21.15.181:0/146227359 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39bc022710 con 0x55d733844220 2023-10-16T11:19:29.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 10 client.11478 fetch_fsmap learned FSMap version 111 2023-10-16T11:19:29.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55d733587c10 con 0x55d733844220 2023-10-16T11:19:29.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39aaffd700 1 -- 172.21.15.181:0/146227359 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4012560 con 0x55d733844220 2023-10-16T11:19:29.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39aaffd700 1 -- 172.21.15.181:0/146227359 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 111) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39bc02a070 con 0x55d733844220 2023-10-16T11:19:29.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 10 client.11478 fetch_fsmap finished waiting for FSMap version 111 2023-10-16T11:19:29.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 10 client.11478 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-16T11:19:29.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 20 client.11478 populate_metadata read hostname 'smithi181' 2023-10-16T11:19:29.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d7338417d0 0x55d733841bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 4 client.11478 mds_command: new command op to 11340 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:19:29.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.458+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 --> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d73376cb20 con 0x55d7338417d0 2023-10-16T11:19:29.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39cbfff700 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d7338417d0 0x55d733841bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39aaffd700 10 client.11478 ms_handle_connect on v2:172.21.15.190:6833/2252072910 2023-10-16T11:19:29.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39cbfff700 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d7338417d0 0x55d733841bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc683377c89af76 server_cookie=d4e6534be73762d7 in_seq=0 out_seq=0 2023-10-16T11:19:29.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39aaffd700 1 -- 172.21.15.181:0/146227359 <== mds.0 v2:172.21.15.190:6833/2252072910 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55d73376cb20 con 0x55d7338417d0 2023-10-16T11:19:29.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39aaffd700 10 client.11478 handle_command_reply: tid=0 2023-10-16T11:19:29.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 client.11478 shutdown 2023-10-16T11:19:29.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 msgr2=0x55d73384e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384e9b0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0011e0 tx=0x7f39bc013000 comp rx=0 tx=0).stop 2023-10-16T11:19:29.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 shutdown_connections 2023-10-16T11:19:29.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d73381f010 0x55d733866be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:6833/2252072910,v1:172.21.15.190:6835/2252072910] conn(0x55d7338417d0 0x55d733841bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d733849630 0x55d73384eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 --2- 172.21.15.181:0/146227359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d733844220 0x55d73384e9b0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39aaffd700 0 client.11478 ms_handle_reset on v2:172.21.15.190:6833/2252072910 2023-10-16T11:19:29.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 >> 172.21.15.181:0/146227359 conn(0x55d73374ed80 msgr2=0x55d73381d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 shutdown_connections 2023-10-16T11:19:29.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.459+0000 7f39de73ab80 1 -- 172.21.15.181:0/146227359 wait complete. 2023-10-16T11:19:29.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39de73ab80 20 client.11478 trim_cache size 0 max 16384 2023-10-16T11:19:29.484 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:19:29.484 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:19:29.485 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:19:29.485 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 11340, 2023-10-16T11:19:29.485 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:19:29.485 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:19:29.485 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T11:19:29.486 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9123779059999997, 2023-10-16T11:19:29.486 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 111, 2023-10-16T11:19:29.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 216, 2023-10-16T11:19:29.486 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 216, 2023-10-16T11:19:29.486 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.86521275 2023-10-16T11:19:29.487 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:19:29.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f39b806b020 msgr2=0x7f39b806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f39b806b020 0x7f39b806d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39cc09e460 tx=0x7f39bc009040 comp rx=0 tx=0).stop 2023-10-16T11:19:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc06a8a0 msgr2=0x7f39cc15cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc06a8a0 0x7f39cc15cea0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f39c0004000 tx=0x7f39c0004750 comp rx=0 tx=0).stop 2023-10-16T11:19:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 shutdown_connections 2023-10-16T11:19:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f39b806b020 0x7f39b806d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39cc15df60 0x7f39cc15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc06a8a0 0x7f39cc15cea0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 --2- 172.21.15.181:0/1172444933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39cc0d0fc0 0x7f39cc14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 >> 172.21.15.181:0/1172444933 conn(0x7f39cc0d04a0 msgr2=0x7f39cc06f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 shutdown_connections 2023-10-16T11:19:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.460+0000 7f39d2550700 1 -- 172.21.15.181:0/1172444933 wait complete. 2023-10-16T11:19:29.491 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:19:29.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.612+0000 7fb2be37b700 1 Processor -- start 2023-10-16T11:19:29.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.612+0000 7fb2be37b700 1 -- start start 2023-10-16T11:19:29.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2be37b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2b80665a0 0x7fb2b80ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2be37b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b8066a30 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2be37b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b8066c60 con 0x7fb2b815db70 2023-10-16T11:19:29.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2be37b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80cda10 con 0x7fb2b80665a0 2023-10-16T11:19:29.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2bc918700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2b80665a0 0x7fb2b80ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2b77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb2b80cf4e0 0x7fb2b80ca2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38416/0 (socket says 172.21.15.181:38416) 2023-10-16T11:19:29.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2b77fe700 1 -- 172.21.15.181:0/695103403 learned_addr learned my addr 172.21.15.181:0/695103403 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:29.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2bc918700 1 -- 172.21.15.181:0/695103403 >> v1:172.21.15.190:6789/0 conn(0x7fb2b815db70 legacy=0x7fb2b80cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:29.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2b6ffd700 1 -- 172.21.15.181:0/695103403 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696541162 0 0) 0x7fb2b8066a30 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2bc918700 1 -- 172.21.15.181:0/695103403 >> v1:172.21.15.181:6789/0 conn(0x7fb2b80cf4e0 legacy=0x7fb2b80ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.613+0000 7fb2bc918700 1 -- 172.21.15.181:0/695103403 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b806c1a0 con 0x7fb2b80665a0 2023-10-16T11:19:29.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2bc918700 1 --2- 172.21.15.181:0/695103403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80665a0 0x7fb2b80ca9f0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009d90 tx=0x7fb2b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e6b7f5fad662fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2b6ffd700 1 -- 172.21.15.181:0/695103403 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000e070 con 0x7fb2b80665a0 2023-10-16T11:19:29.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2b6ffd700 1 -- 172.21.15.181:0/695103403 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b000bb90 con 0x7fb2b80665a0 2023-10-16T11:19:29.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2b6ffd700 1 -- 172.21.15.181:0/695103403 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0003a70 con 0x7fb2b80665a0 2023-10-16T11:19:29.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2be37b700 1 -- 172.21.15.181:0/695103403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80665a0 msgr2=0x7fb2b80ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2be37b700 1 --2- 172.21.15.181:0/695103403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80665a0 0x7fb2b80ca9f0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009d90 tx=0x7fb2b000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:29.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2be37b700 1 -- 172.21.15.181:0/695103403 shutdown_connections 2023-10-16T11:19:29.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2be37b700 1 --2- 172.21.15.181:0/695103403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80665a0 0x7fb2b80ca9f0 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2be37b700 1 -- 172.21.15.181:0/695103403 >> 172.21.15.181:0/695103403 conn(0x7fb2b80b1780 msgr2=0x7fb2b80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2be37b700 1 -- 172.21.15.181:0/695103403 shutdown_connections 2023-10-16T11:19:29.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.614+0000 7fb2be37b700 1 -- 172.21.15.181:0/695103403 wait complete. 2023-10-16T11:19:29.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.615+0000 7fb2be37b700 1 Processor -- start 2023-10-16T11:19:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.616+0000 7fb2be37b700 1 -- start start 2023-10-16T11:19:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.616+0000 7fb2be37b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b80665a0 0x7fb2b80c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.616+0000 7fb2be37b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80cf4e0 0x7fb2b80c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.616+0000 7fb2be37b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b815db70 0x7fb2b80c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.616+0000 7fb2be37b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c7090 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.616+0000 7fb2be37b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c7370 con 0x7fb2b815db70 2023-10-16T11:19:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.616+0000 7fb2be37b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c7650 con 0x7fb2b80665a0 2023-10-16T11:19:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80cf4e0 0x7fb2b80c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80cf4e0 0x7fb2b80c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55576/0 (socket says 172.21.15.181:55576) 2023-10-16T11:19:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 -- 172.21.15.181:0/2586009514 learned_addr learned my addr 172.21.15.181:0/2586009514 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2bc918700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b815db70 0x7fb2b80c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b7fff700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b80665a0 0x7fb2b80c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 -- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b80665a0 msgr2=0x7fb2b80c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b80665a0 0x7fb2b80c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 -- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b815db70 msgr2=0x7fb2b80c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b815db70 0x7fb2b80c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 -- 172.21.15.181:0/2586009514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0009a70 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b7fff700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b80665a0 0x7fb2b80c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b77fe700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80cf4e0 0x7fb2b80c8130 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000c520 tx=0x7fb2a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b4ff9700 1 -- 172.21.15.181:0/2586009514 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0004c80 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b4ff9700 1 -- 172.21.15.181:0/2586009514 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a0004de0 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80c7930 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2b4ff9700 1 -- 172.21.15.181:0/2586009514 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0007310 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.617+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b80c6d60 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.618+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b80c7e50 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.619+0000 7fb2b4ff9700 1 -- 172.21.15.181:0/2586009514 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb2a0007470 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.619+0000 7fb2b4ff9700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fb29806b020 0x7fb29806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.619+0000 7fb2b4ff9700 1 -- 172.21.15.181:0/2586009514 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fb2a00899b0 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.619+0000 7fb2b7fff700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fb29806b020 0x7fb29806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.620+0000 7fb2b7fff700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fb29806b020 0x7fb29806d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8160670 tx=0x7fb2a8008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:29.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.623+0000 7fb2b4ff9700 1 -- 172.21.15.181:0/2586009514 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb2a0054890 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.857+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2b80c7e50 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.858+0000 7fb2b4ff9700 1 -- 172.21.15.181:0/2586009514 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb2b80c7e50 con 0x7fb2b80cf4e0 2023-10-16T11:19:29.860 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:19:29.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.859+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fb29806b020 msgr2=0x7fb29806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.859+0000 7fb2be37b700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fb29806b020 0x7fb29806d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8160670 tx=0x7fb2a8008040 comp rx=0 tx=0).stop 2023-10-16T11:19:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.859+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80cf4e0 msgr2=0x7fb2b80c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.859+0000 7fb2be37b700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80cf4e0 0x7fb2b80c8130 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000c520 tx=0x7fb2a000c830 comp rx=0 tx=0).stop 2023-10-16T11:19:29.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.859+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 shutdown_connections 2023-10-16T11:19:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.860+0000 7fb2be37b700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fb29806b020 0x7fb29806d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.860+0000 7fb2be37b700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2b80665a0 0x7fb2b80c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.860+0000 7fb2be37b700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2b815db70 0x7fb2b80c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.860+0000 7fb2be37b700 1 --2- 172.21.15.181:0/2586009514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2b80cf4e0 0x7fb2b80c8130 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:29.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.860+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 >> 172.21.15.181:0/2586009514 conn(0x7fb2b80b1780 msgr2=0x7fb2b80c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:29.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.860+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 shutdown_connections 2023-10-16T11:19:29.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:29.860+0000 7fb2be37b700 1 -- 172.21.15.181:0/2586009514 wait complete. 2023-10-16T11:19:29.874 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:19:29.874 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:19:29.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T11:19:29.875 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:19:29.875 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:19:29.876 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:19:29.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:19:29.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:19:29.877 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:19:29.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:29.878 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T11:19:29.895 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:19:29.896 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:19:29.896 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:19:29.896 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:19:29.896 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T11:19:29.896 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:19:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:19:29.897 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:19:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:19:29.897 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:19:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:19:29.898 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T11:19:29.898 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:19:29.898 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T11:19:29.898 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:19:29.898 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 36700sec preferred_lft 36700sec 2023-10-16T11:19:29.899 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T11:19:29.899 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:19:29.899 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:19:29.899 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:19:29.899 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:19:29.900 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:19:29.900 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T11:19:29.900 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:19:29.900 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:19:29.900 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:19:29.901 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T11:19:29.901 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:19:29.901 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:19:29.901 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:19:29.902 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T11:19:29.902 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:19:29.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:29.903 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:19:29.961 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T11:19:29.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:29.963 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T11:19:30.021 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T11:19:30.022 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:19:30.022 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:19:30.022 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:19:30.022 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:19:30.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:30.023 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T11:19:30.024 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T11:19:30.024 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:19:30.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:19:30.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:19:30.025 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T11:19:30.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:30.127 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:19:30.143 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:19:30.144 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:19:30.144 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:19:30.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:30.145 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T11:19:30.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:19:30.210 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:19:30.266 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:19:30.266 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:19:30.267 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:19:30.323 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T11:19:30.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:30.323 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:19:30.407 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:19:30.408 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:19:30.408 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:19:30.416 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:19:30.417 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:19:30.417 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:19:30.417 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:19:30.417 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:19:30.491 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:19:30.525 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:19:30.526 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:19:30.526 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:19:30.526 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:19:30.526 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T11:19:30.526 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T11:19:30.527 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T11:19:30.528 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:19:30.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:30.529 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:19:30.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:30.556 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:19:30.643 INFO:teuthology.orchestra.run.smithi181.stdout:client8909 2023-10-16T11:19:30.643 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:30.644 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:30.660 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-16T11:19:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.823+0000 7f9aa4ea8700 1 Processor -- start 2023-10-16T11:19:30.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.823+0000 7f9aa4ea8700 1 -- start start 2023-10-16T11:19:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9aa4ea8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9aa015eea0 0x7f9aa00b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9aa4ea8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9aa00b0f70 0x7f9aa00ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9aa4ea8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa00d23c0 con 0x7f9aa00ce640 2023-10-16T11:19:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9aa4ea8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa00cd990 con 0x7f9aa00b0f70 2023-10-16T11:19:30.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9aa4ea8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa00cdc70 con 0x7f9aa015eea0 2023-10-16T11:19:30.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9aa00b0f70 0x7f9aa00ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:30.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9aa00ce640 0x7f9aa00cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38432/0 (socket says 172.21.15.181:38432) 2023-10-16T11:19:30.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9ed9d700 1 -- 172.21.15.181:0/636108191 learned_addr learned my addr 172.21.15.181:0/636108191 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:30.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9e59c700 1 --2- 172.21.15.181:0/636108191 >> v2:172.21.15.190:3300/0 conn(0x7f9aa015eea0 0x7f9aa00b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:30.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9d59a700 1 -- 172.21.15.181:0/636108191 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852019044 0 0) 0x7f9aa00d23c0 con 0x7f9aa00ce640 2023-10-16T11:19:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9d59a700 1 -- 172.21.15.181:0/636108191 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a880035b0 con 0x7f9aa00ce640 2023-10-16T11:19:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9dd9b700 1 -- 172.21.15.181:0/636108191 >> v2:172.21.15.190:3300/0 conn(0x7f9aa015eea0 msgr2=0x7f9aa00b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9dd9b700 1 --2- 172.21.15.181:0/636108191 >> v2:172.21.15.190:3300/0 conn(0x7f9aa015eea0 0x7f9aa00b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9dd9b700 1 -- 172.21.15.181:0/636108191 >> v1:172.21.15.181:6789/0 conn(0x7f9aa00ce640 legacy=0x7f9aa00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:30.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.824+0000 7f9a9dd9b700 1 -- 172.21.15.181:0/636108191 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa00cce60 con 0x7f9aa00b0f70 2023-10-16T11:19:30.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9a9dd9b700 1 --2- 172.21.15.181:0/636108191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa00b0f70 0x7f9aa00ce100 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00b4180 tx=0x7f9a94009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab05bc675f48bb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:30.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9a9d59a700 1 -- 172.21.15.181:0/636108191 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94011830 con 0x7f9aa00b0f70 2023-10-16T11:19:30.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9a9d59a700 1 -- 172.21.15.181:0/636108191 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a94011990 con 0x7f9aa00b0f70 2023-10-16T11:19:30.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9a9d59a700 1 -- 172.21.15.181:0/636108191 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94011d20 con 0x7f9aa00b0f70 2023-10-16T11:19:30.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/636108191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa00b0f70 msgr2=0x7f9aa00ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:30.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/636108191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa00b0f70 0x7f9aa00ce100 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00b4180 tx=0x7f9a94009b20 comp rx=0 tx=0).stop 2023-10-16T11:19:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/636108191 shutdown_connections 2023-10-16T11:19:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/636108191 >> v2:172.21.15.190:3300/0 conn(0x7f9aa015eea0 0x7f9aa00b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/636108191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa00b0f70 0x7f9aa00ce100 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/636108191 >> 172.21.15.181:0/636108191 conn(0x7f9aa00d1610 msgr2=0x7f9aa00d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:30.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/636108191 shutdown_connections 2023-10-16T11:19:30.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.825+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/636108191 wait complete. 2023-10-16T11:19:30.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 Processor -- start 2023-10-16T11:19:30.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 -- start start 2023-10-16T11:19:30.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa00b0f70 0x7f9aa00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:30.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa00ce640 0x7f9aa0070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:30.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa015eea0 0x7f9aa00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:30.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa00caf60 con 0x7f9aa015eea0 2023-10-16T11:19:30.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa00ca470 con 0x7f9aa00ce640 2023-10-16T11:19:30.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9aa4ea8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa00b0b40 con 0x7f9aa00b0f70 2023-10-16T11:19:30.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9a9ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa015eea0 0x7f9aa00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:30.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9a9ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa015eea0 0x7f9aa00ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55594/0 (socket says 172.21.15.181:55594) 2023-10-16T11:19:30.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9a9ed9d700 1 -- 172.21.15.181:0/1478216941 learned_addr learned my addr 172.21.15.181:0/1478216941 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:30.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.827+0000 7f9a9e59c700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa00b0f70 0x7f9aa00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:30.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a9dd9b700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa00ce640 0x7f9aa0070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:30.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a9ed9d700 1 -- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa00b0f70 msgr2=0x7f9aa00b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:30.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a9ed9d700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa00b0f70 0x7f9aa00b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:30.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a9ed9d700 1 -- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa00ce640 msgr2=0x7f9aa0070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:30.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a9ed9d700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa00ce640 0x7f9aa0070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:30.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a9ed9d700 1 -- 172.21.15.181:0/1478216941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940094a0 con 0x7f9aa015eea0 2023-10-16T11:19:30.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a9ed9d700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa015eea0 0x7f9aa00ae7c0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800cb40 tx=0x7f9a9800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:30.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a8effd700 1 -- 172.21.15.181:0/1478216941 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9800a9e0 con 0x7f9aa015eea0 2023-10-16T11:19:30.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a8effd700 1 -- 172.21.15.181:0/1478216941 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9800ab40 con 0x7f9aa015eea0 2023-10-16T11:19:30.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa00ca6c0 con 0x7f9aa015eea0 2023-10-16T11:19:30.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9a8effd700 1 -- 172.21.15.181:0/1478216941 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98012400 con 0x7f9aa015eea0 2023-10-16T11:19:30.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.828+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa00cac80 con 0x7f9aa015eea0 2023-10-16T11:19:30.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.829+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa0067d20 con 0x7f9aa015eea0 2023-10-16T11:19:30.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.830+0000 7f9a8effd700 1 -- 172.21.15.181:0/1478216941 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9a9801b3b0 con 0x7f9aa015eea0 2023-10-16T11:19:30.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.830+0000 7f9a8effd700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9a8006b020 0x7f9a8006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:30.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.830+0000 7f9a8effd700 1 -- 172.21.15.181:0/1478216941 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9a98089400 con 0x7f9aa015eea0 2023-10-16T11:19:30.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.830+0000 7f9a9e59c700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9a8006b020 0x7f9a8006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:30.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.830+0000 7f9a9e59c700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9a8006b020 0x7f9a8006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a90006b20 tx=0x7f9a90008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:30.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:30.834+0000 7f9a8effd700 1 -- 172.21.15.181:0/1478216941 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9a980542b0 con 0x7f9aa015eea0 2023-10-16T11:19:31.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.069+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9aa00b7270 con 0x7f9aa015eea0 2023-10-16T11:19:31.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.587+0000 7f9a8effd700 1 -- 172.21.15.181:0/1478216941 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v112) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9a980578d0 con 0x7f9aa015eea0 2023-10-16T11:19:31.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9a8006b020 msgr2=0x7f9a8006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9a8006b020 0x7f9a8006d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a90006b20 tx=0x7f9a90008040 comp rx=0 tx=0).stop 2023-10-16T11:19:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa015eea0 msgr2=0x7f9aa00ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa015eea0 0x7f9aa00ae7c0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800cb40 tx=0x7f9a9800ce50 comp rx=0 tx=0).stop 2023-10-16T11:19:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 shutdown_connections 2023-10-16T11:19:31.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9a8006b020 0x7f9a8006d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa00b0f70 0x7f9aa00b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa015eea0 0x7f9aa00ae7c0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 --2- 172.21.15.181:0/1478216941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa00ce640 0x7f9aa0070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 >> 172.21.15.181:0/1478216941 conn(0x7f9aa00d1610 msgr2=0x7f9aa00d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 shutdown_connections 2023-10-16T11:19:31.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.589+0000 7f9aa4ea8700 1 -- 172.21.15.181:0/1478216941 wait complete. 2023-10-16T11:19:31.605 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:31.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.724+0000 7f06a27a0700 1 Processor -- start 2023-10-16T11:19:31.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.724+0000 7f06a27a0700 1 -- start start 2023-10-16T11:19:31.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a27a0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f069c15db70 0x7f069c161db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:31.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a27a0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c0af820 0x7f069c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:31.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a27a0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0cc610 con 0x7f069c0af2d0 2023-10-16T11:19:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a27a0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0cc8f0 con 0x7f069c15db70 2023-10-16T11:19:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a27a0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c1615b0 con 0x7f069c0af820 2023-10-16T11:19:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a179e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f069c15db70 0x7f069c161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a179e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f069c15db70 0x7f069c161db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55598/0 (socket says 172.21.15.181:55598) 2023-10-16T11:19:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.725+0000 7f06a179e700 1 -- 172.21.15.181:0/2949305751 learned_addr learned my addr 172.21.15.181:0/2949305751 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f06a0f9d700 1 --2- 172.21.15.181:0/2949305751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c0af820 0x7f069c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f06a179e700 1 -- 172.21.15.181:0/2949305751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c0af820 msgr2=0x7f069c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f06a179e700 1 --2- 172.21.15.181:0/2949305751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c0af820 0x7f069c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f06a179e700 1 -- 172.21.15.181:0/2949305751 >> v1:172.21.15.190:6789/0 conn(0x7f069c0af2d0 legacy=0x7f069c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f06a179e700 1 -- 172.21.15.181:0/2949305751 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c161890 con 0x7f069c15db70 2023-10-16T11:19:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f06a0f9d700 1 --2- 172.21.15.181:0/2949305751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c0af820 0x7f069c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f06a179e700 1 --2- 172.21.15.181:0/2949305751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c15db70 0x7f069c161db0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f0690006fd0 tx=0x7f0690009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2892925c3b440f82 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f068ffff700 1 -- 172.21.15.181:0/2949305751 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690005e30 con 0x7f069c15db70 2023-10-16T11:19:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f068ffff700 1 -- 172.21.15.181:0/2949305751 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690004030 con 0x7f069c15db70 2023-10-16T11:19:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.726+0000 7f068ffff700 1 -- 172.21.15.181:0/2949305751 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900043e0 con 0x7f069c15db70 2023-10-16T11:19:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 -- 172.21.15.181:0/2949305751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c15db70 msgr2=0x7f069c161db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 --2- 172.21.15.181:0/2949305751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c15db70 0x7f069c161db0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f0690006fd0 tx=0x7f0690009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 -- 172.21.15.181:0/2949305751 shutdown_connections 2023-10-16T11:19:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 --2- 172.21.15.181:0/2949305751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c15db70 0x7f069c161db0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 --2- 172.21.15.181:0/2949305751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c0af820 0x7f069c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 -- 172.21.15.181:0/2949305751 >> 172.21.15.181:0/2949305751 conn(0x7f069c06c430 msgr2=0x7f069c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:31.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 -- 172.21.15.181:0/2949305751 shutdown_connections 2023-10-16T11:19:31.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.727+0000 7f06a27a0700 1 -- 172.21.15.181:0/2949305751 wait complete. 2023-10-16T11:19:31.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.728+0000 7f06a27a0700 1 Processor -- start 2023-10-16T11:19:31.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.728+0000 7f06a27a0700 1 -- start start 2023-10-16T11:19:31.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a27a0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f069c0af2d0 0x7f069c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:31.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a27a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0af820 0x7f069c0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:31.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a27a0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c15db70 0x7f069c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:31.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a27a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c78a0 con 0x7f069c0af820 2023-10-16T11:19:31.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a27a0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c7b80 con 0x7f069c0af2d0 2023-10-16T11:19:31.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a27a0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c7e60 con 0x7f069c15db70 2023-10-16T11:19:31.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0af820 0x7f069c0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:31.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0af820 0x7f069c0c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55604/0 (socket says 172.21.15.181:55604) 2023-10-16T11:19:31.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 -- 172.21.15.181:0/4132533568 learned_addr learned my addr 172.21.15.181:0/4132533568 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:31.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a1f9f700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c15db70 0x7f069c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:31.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 -- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c15db70 msgr2=0x7f069c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c15db70 0x7f069c15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a179e700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f069c0af2d0 0x7f069c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:31.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 -- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f069c0af2d0 msgr2=0x7f069c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f069c0af2d0 0x7f069c151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 -- 172.21.15.181:0/4132533568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690009500 con 0x7f069c0af820 2023-10-16T11:19:31.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a0f9d700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0af820 0x7f069c0c8940 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f069800e7e0 tx=0x7f069800eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:31.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f068dffb700 1 -- 172.21.15.181:0/4132533568 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698004230 con 0x7f069c0af820 2023-10-16T11:19:31.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f068dffb700 1 -- 172.21.15.181:0/4132533568 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0698004390 con 0x7f069c0af820 2023-10-16T11:19:31.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c162470 con 0x7f069c0af820 2023-10-16T11:19:31.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.729+0000 7f068dffb700 1 -- 172.21.15.181:0/4132533568 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06980036a0 con 0x7f069c0af820 2023-10-16T11:19:31.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.730+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c0c8420 con 0x7f069c0af820 2023-10-16T11:19:31.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.731+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c0c85b0 con 0x7f069c0af820 2023-10-16T11:19:31.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.731+0000 7f068dffb700 1 -- 172.21.15.181:0/4132533568 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0698003800 con 0x7f069c0af820 2023-10-16T11:19:31.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.731+0000 7f068dffb700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:31.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.731+0000 7f068dffb700 1 -- 172.21.15.181:0/4132533568 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f0698089530 con 0x7f069c0af820 2023-10-16T11:19:31.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.731+0000 7f06a179e700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:31.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.732+0000 7f06a179e700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f069c0cbe90 tx=0x7f0690008070 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:31.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.736+0000 7f068dffb700 1 -- 172.21.15.181:0/4132533568 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0698054190 con 0x7f069c0af820 2023-10-16T11:19:31.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.928+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f069c0cfff0 con 0x7f069c0af820 2023-10-16T11:19:31.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.929+0000 7f068dffb700 1 -- 172.21.15.181:0/4132533568 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+17470 (secure 0 0 0) 0x7f069c0cfff0 con 0x7f069c0af820 2023-10-16T11:19:31.931 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:31.931 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":216,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:24.532965+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:31.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 msgr2=0x7f068006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f069c0cbe90 tx=0x7f0690008070 comp rx=0 tx=0).stop 2023-10-16T11:19:31.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0af820 msgr2=0x7f069c0c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0af820 0x7f069c0c8940 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f069800e7e0 tx=0x7f069800eba0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 shutdown_connections 2023-10-16T11:19:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f069c15db70 0x7f069c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f069c0af2d0 0x7f069c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 --2- 172.21.15.181:0/4132533568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f069c0af820 0x7f069c0c8940 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 >> 172.21.15.181:0/4132533568 conn(0x7f069c06c430 msgr2=0x7f069c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 shutdown_connections 2023-10-16T11:19:31.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:31.931+0000 7f06a27a0700 1 -- 172.21.15.181:0/4132533568 wait complete. 2023-10-16T11:19:31.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:19:21.355919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:19:21.889329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:19:31.946 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:19:32.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.067+0000 7f6864235700 1 Processor -- start 2023-10-16T11:19:32.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.067+0000 7f6864235700 1 -- start start 2023-10-16T11:19:32.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.068+0000 7f6864235700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f685c15e520 0x7f685c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.068+0000 7f6864235700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0b1e20 con 0x7f685c0cdde0 2023-10-16T11:19:32.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.068+0000 7f6864235700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0cd300 con 0x7f685c06a8a0 2023-10-16T11:19:32.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.068+0000 7f6864235700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0cd0f0 con 0x7f685c15e520 2023-10-16T11:19:32.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.068+0000 7f68617d0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f685c0cdde0 0x7f685c0b1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38458/0 (socket says 172.21.15.181:38458) 2023-10-16T11:19:32.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.068+0000 7f68617d0700 1 -- 172.21.15.181:0/1429445544 learned_addr learned my addr 172.21.15.181:0/1429445544 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:32.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.068+0000 7f6861fd1700 1 --2- 172.21.15.181:0/1429445544 >> v2:172.21.15.190:3300/0 conn(0x7f685c15e520 0x7f685c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873354235 0 0) 0x7f685c0b1e20 con 0x7f685c0cdde0 2023-10-16T11:19:32.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68480035b0 con 0x7f685c0cdde0 2023-10-16T11:19:32.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309121277 0 0) 0x7f685c0cd300 con 0x7f685c06a8a0 2023-10-16T11:19:32.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f685c0b1e20 con 0x7f685c06a8a0 2023-10-16T11:19:32.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 230766238 0 0) 0x7f68480035b0 con 0x7f685c0cdde0 2023-10-16T11:19:32.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 >> v2:172.21.15.190:3300/0 conn(0x7f685c15e520 msgr2=0x7f685c15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:32.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 --2- 172.21.15.181:0/1429445544 >> v2:172.21.15.190:3300/0 conn(0x7f685c15e520 0x7f685c15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 >> v1:172.21.15.190:6789/0 conn(0x7f685c06a8a0 legacy=0x7f685c0cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0cd5e0 con 0x7f685c0cdde0 2023-10-16T11:19:32.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f6858001670 con 0x7f685c0cdde0 2023-10-16T11:19:32.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 >> v1:172.21.15.181:6789/0 conn(0x7f685c0cdde0 legacy=0x7f685c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003b90 0x7f6848003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848004850 0x7f6848004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6848005210 0x7f684800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0cd0f0 con 0x7f6848005210 2023-10-16T11:19:32.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684800c090 con 0x7f6848003b90 2023-10-16T11:19:32.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684800c250 con 0x7f6848004850 2023-10-16T11:19:32.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68627d2700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6848005210 0x7f684800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f6861fd1700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003b90 0x7f6848003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68617d0700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848004850 0x7f6848004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68627d2700 1 -- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848004850 msgr2=0x7f6848004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:32.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68627d2700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848004850 0x7f6848004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68627d2700 1 -- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003b90 msgr2=0x7f6848003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:32.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68627d2700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6848003b90 0x7f6848003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68627d2700 1 -- 172.21.15.181:0/1429445544 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480035b0 con 0x7f6848005210 2023-10-16T11:19:32.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.069+0000 7f68627d2700 1 -- 172.21.15.181:0/1429445544 reap_dead start 2023-10-16T11:19:32.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f68627d2700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6848005210 0x7f684800ba10 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f685400c2f0 tx=0x7f6848005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f50ca5e64e93c97 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854001f30 con 0x7f6848005210 2023-10-16T11:19:32.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854003a00 con 0x7f6848005210 2023-10-16T11:19:32.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6860fcf700 1 -- 172.21.15.181:0/1429445544 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854003db0 con 0x7f6848005210 2023-10-16T11:19:32.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6864235700 1 -- 172.21.15.181:0/1429445544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6848005210 msgr2=0x7f684800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6864235700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6848005210 0x7f684800ba10 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f685400c2f0 tx=0x7f6848005880 comp rx=0 tx=0).stop 2023-10-16T11:19:32.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6864235700 1 -- 172.21.15.181:0/1429445544 shutdown_connections 2023-10-16T11:19:32.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6864235700 1 --2- 172.21.15.181:0/1429445544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6848005210 0x7f684800ba10 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6864235700 1 -- 172.21.15.181:0/1429445544 >> 172.21.15.181:0/1429445544 conn(0x7f685c06c430 msgr2=0x7f685c099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:32.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6864235700 1 -- 172.21.15.181:0/1429445544 shutdown_connections 2023-10-16T11:19:32.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.070+0000 7f6864235700 1 -- 172.21.15.181:0/1429445544 wait complete. 2023-10-16T11:19:32.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 Processor -- start 2023-10-16T11:19:32.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 -- start start 2023-10-16T11:19:32.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848005210 0x7f685c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f685c1599c0 0x7f685c159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c0ad340 0x7f685c0ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c15a380 con 0x7f685c1599c0 2023-10-16T11:19:32.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ae070 con 0x7f685c0ad340 2023-10-16T11:19:32.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6864235700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ae350 con 0x7f6848005210 2023-10-16T11:19:32.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f68617d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f685c1599c0 0x7f685c159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f68617d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f685c1599c0 0x7f685c159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55624/0 (socket says 172.21.15.181:55624) 2023-10-16T11:19:32.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f68617d0700 1 -- 172.21.15.181:0/24707571 learned_addr learned my addr 172.21.15.181:0/24707571 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:32.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.072+0000 7f6861fd1700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848005210 0x7f685c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68627d2700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c0ad340 0x7f685c0ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68617d0700 1 -- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848005210 msgr2=0x7f685c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68617d0700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848005210 0x7f685c155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68617d0700 1 -- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c0ad340 msgr2=0x7f685c0ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68617d0700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c0ad340 0x7f685c0ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68617d0700 1 -- 172.21.15.181:0/24707571 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0ae630 con 0x7f685c1599c0 2023-10-16T11:19:32.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68617d0700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f685c1599c0 0x7f685c159e40 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f6858007340 tx=0x7f6858007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68527fc700 1 -- 172.21.15.181:0/24707571 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800f6c0 con 0x7f685c1599c0 2023-10-16T11:19:32.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68527fc700 1 -- 172.21.15.181:0/24707571 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685800f840 con 0x7f685c1599c0 2023-10-16T11:19:32.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0ae910 con 0x7f685c1599c0 2023-10-16T11:19:32.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f68527fc700 1 -- 172.21.15.181:0/24707571 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800fbf0 con 0x7f685c1599c0 2023-10-16T11:19:32.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.073+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c15ff30 con 0x7f685c1599c0 2023-10-16T11:19:32.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.074+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c067d20 con 0x7f685c1599c0 2023-10-16T11:19:32.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.074+0000 7f68527fc700 1 -- 172.21.15.181:0/24707571 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f685800fd50 con 0x7f685c1599c0 2023-10-16T11:19:32.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.075+0000 7f68527fc700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f684006b020 0x7f684006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.075+0000 7f68527fc700 1 -- 172.21.15.181:0/24707571 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f68580898c0 con 0x7f685c1599c0 2023-10-16T11:19:32.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.075+0000 7f6861fd1700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f684006b020 0x7f684006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.075+0000 7f6861fd1700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f684006b020 0x7f684006d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f684c009be0 tx=0x7f684c009210 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.079+0000 7f68527fc700 1 -- 172.21.15.181:0/24707571 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f68580546c0 con 0x7f685c1599c0 2023-10-16T11:19:32.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.269+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f685c06b460 con 0x7f685c1599c0 2023-10-16T11:19:32.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.270+0000 7f68527fc700 1 -- 172.21.15.181:0/24707571 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v216) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6858057ce0 con 0x7f685c1599c0 2023-10-16T11:19:32.272 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:19:32.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f684006b020 msgr2=0x7f684006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f684006b020 0x7f684006d4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f684c009be0 tx=0x7f684c009210 comp rx=0 tx=0).stop 2023-10-16T11:19:32.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f685c1599c0 msgr2=0x7f685c159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f685c1599c0 0x7f685c159e40 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f6858007340 tx=0x7f6858007650 comp rx=0 tx=0).stop 2023-10-16T11:19:32.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 shutdown_connections 2023-10-16T11:19:32.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f684006b020 0x7f684006d4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6848005210 0x7f685c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f685c0ad340 0x7f685c0ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 --2- 172.21.15.181:0/24707571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f685c1599c0 0x7f685c159e40 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 >> 172.21.15.181:0/24707571 conn(0x7f685c06c430 msgr2=0x7f685c0cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:32.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 shutdown_connections 2023-10-16T11:19:32.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.272+0000 7f6864235700 1 -- 172.21.15.181:0/24707571 wait complete. 2023-10-16T11:19:32.286 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:19:32.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.409+0000 7f4e6132f700 1 Processor -- start 2023-10-16T11:19:32.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.409+0000 7f4e6132f700 1 -- start start 2023-10-16T11:19:32.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.409+0000 7f4e6132f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4e5c15f150 0x7f4e5c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.409+0000 7f4e6132f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5c0b2ae0 con 0x7f4e5c0ce4f0 2023-10-16T11:19:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.409+0000 7f4e6132f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5c0b2dc0 con 0x7f4e5c0cd170 2023-10-16T11:19:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.409+0000 7f4e6132f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5c0ce9a0 con 0x7f4e5c15f150 2023-10-16T11:19:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e5a7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4e5c0ce4f0 0x7f4e5c0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38478/0 (socket says 172.21.15.181:38478) 2023-10-16T11:19:32.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e5a7fc700 1 -- 172.21.15.181:0/2719514196 learned_addr learned my addr 172.21.15.181:0/2719514196 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:32.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e5affd700 1 --2- 172.21.15.181:0/2719514196 >> v2:172.21.15.190:3300/0 conn(0x7f4e5c15f150 0x7f4e5c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426465949 0 0) 0x7f4e5c0b2ae0 con 0x7f4e5c0ce4f0 2023-10-16T11:19:32.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e440035b0 con 0x7f4e5c0ce4f0 2023-10-16T11:19:32.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1631816043 0 0) 0x7f4e440035b0 con 0x7f4e5c0ce4f0 2023-10-16T11:19:32.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 >> v2:172.21.15.190:3300/0 conn(0x7f4e5c15f150 msgr2=0x7f4e5c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:32.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 --2- 172.21.15.181:0/2719514196 >> v2:172.21.15.190:3300/0 conn(0x7f4e5c15f150 0x7f4e5c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 >> v1:172.21.15.190:6789/0 conn(0x7f4e5c0cd170 legacy=0x7f4e5c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c0cf9d0 con 0x7f4e5c0ce4f0 2023-10-16T11:19:32.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4e50001670 con 0x7f4e5c0ce4f0 2023-10-16T11:19:32.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 >> v1:172.21.15.181:6789/0 conn(0x7f4e5c0ce4f0 legacy=0x7f4e5c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e44003a00 0x7f4e44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44004850 0x7f4e44004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44005210 0x7f4e4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c0ce9a0 con 0x7f4e44003a00 2023-10-16T11:19:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400c0b0 con 0x7f4e44004850 2023-10-16T11:19:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4400c270 con 0x7f4e44005210 2023-10-16T11:19:32.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e5affd700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e44003a00 0x7f4e44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e5a7fc700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44004850 0x7f4e44004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.410+0000 7f4e5b7fe700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44005210 0x7f4e4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5affd700 1 -- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44005210 msgr2=0x7f4e4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5affd700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44005210 0x7f4e4400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5affd700 1 -- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44004850 msgr2=0x7f4e44004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5affd700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44004850 0x7f4e44004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5affd700 1 -- 172.21.15.181:0/2719514196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440035b0 con 0x7f4e44003a00 2023-10-16T11:19:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5a7fc700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44004850 0x7f4e44004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5b7fe700 1 -- 172.21.15.181:0/2719514196 reap_dead start 2023-10-16T11:19:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e5affd700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e44003a00 0x7f4e44003e20 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009b60 tx=0x7f4e44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9ab4deef49d83c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00e6c0 con 0x7f4e44003a00 2023-10-16T11:19:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c00e820 con 0x7f4e44003a00 2023-10-16T11:19:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e59ffb700 1 -- 172.21.15.181:0/2719514196 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00ebb0 con 0x7f4e44003a00 2023-10-16T11:19:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e6132f700 1 -- 172.21.15.181:0/2719514196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e44003a00 msgr2=0x7f4e44003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e6132f700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e44003a00 0x7f4e44003e20 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009b60 tx=0x7f4e44005880 comp rx=0 tx=0).stop 2023-10-16T11:19:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e6132f700 1 -- 172.21.15.181:0/2719514196 shutdown_connections 2023-10-16T11:19:32.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e6132f700 1 --2- 172.21.15.181:0/2719514196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e44003a00 0x7f4e44003e20 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e6132f700 1 -- 172.21.15.181:0/2719514196 >> 172.21.15.181:0/2719514196 conn(0x7f4e5c09a930 msgr2=0x7f4e5c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e6132f700 1 -- 172.21.15.181:0/2719514196 shutdown_connections 2023-10-16T11:19:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.411+0000 7f4e6132f700 1 -- 172.21.15.181:0/2719514196 wait complete. 2023-10-16T11:19:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.413+0000 7f4e6132f700 1 Processor -- start 2023-10-16T11:19:32.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.413+0000 7f4e6132f700 1 -- start start 2023-10-16T11:19:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44003a00 0x7f4e5c1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0c5130 0x7f4e5c0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e5c0c8b90 0x7f4e5c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c0c56f0 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c0c59d0 con 0x7f4e44003a00 2023-10-16T11:19:32.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c0c5cb0 con 0x7f4e5c0c5130 2023-10-16T11:19:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e5c0c8b90 0x7f4e5c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e5c0c8b90 0x7f4e5c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55646/0 (socket says 172.21.15.181:55646) 2023-10-16T11:19:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 -- 172.21.15.181:0/975589777 learned_addr learned my addr 172.21.15.181:0/975589777 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5a7fc700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0c5130 0x7f4e5c0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 -- 172.21.15.181:0/975589777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0c5130 msgr2=0x7f4e5c0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0c5130 0x7f4e5c0b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 -- 172.21.15.181:0/975589777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44003a00 msgr2=0x7f4e5c1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44003a00 0x7f4e5c1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 -- 172.21.15.181:0/975589777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c0c5f90 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e5b7fe700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e5c0c8b90 0x7f4e5c06ed60 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c06b2c0 tx=0x7f4e5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e437fe700 1 -- 172.21.15.181:0/975589777 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54003c10 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e437fe700 1 -- 172.21.15.181:0/975589777 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e54003d70 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c0c6270 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e437fe700 1 -- 172.21.15.181:0/975589777 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54005310 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.414+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c0c9010 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.415+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c067d20 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.416+0000 7f4e437fe700 1 -- 172.21.15.181:0/975589777 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4e54005470 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.416+0000 7f4e437fe700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4e4806b020 0x7f4e4806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.416+0000 7f4e437fe700 1 -- 172.21.15.181:0/975589777 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f4e54089960 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.416+0000 7f4e5affd700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4e4806b020 0x7f4e4806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.417+0000 7f4e5affd700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4e4806b020 0x7f4e4806d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0090b0 tx=0x7f4e4c013000 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.421+0000 7f4e437fe700 1 -- 172.21.15.181:0/975589777 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4e540546d0 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.611+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e5c0b1b80 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.612+0000 7f4e437fe700 1 -- 172.21.15.181:0/975589777 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v216) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e54057cf0 con 0x7f4e5c0c8b90 2023-10-16T11:19:32.614 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:19:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4e4806b020 msgr2=0x7f4e4806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4e4806b020 0x7f4e4806d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0090b0 tx=0x7f4e4c013000 comp rx=0 tx=0).stop 2023-10-16T11:19:32.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e5c0c8b90 msgr2=0x7f4e5c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e5c0c8b90 0x7f4e5c06ed60 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c06b2c0 tx=0x7f4e5400ee80 comp rx=0 tx=0).stop 2023-10-16T11:19:32.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 shutdown_connections 2023-10-16T11:19:32.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4e4806b020 0x7f4e4806d4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e5c0c8b90 0x7f4e5c06ed60 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e44003a00 0x7f4e5c1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 --2- 172.21.15.181:0/975589777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0c5130 0x7f4e5c0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 >> 172.21.15.181:0/975589777 conn(0x7f4e5c09a930 msgr2=0x7f4e5c15aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:32.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 shutdown_connections 2023-10-16T11:19:32.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.614+0000 7f4e6132f700 1 -- 172.21.15.181:0/975589777 wait complete. 2023-10-16T11:19:32.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:19:32.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.750+0000 7f4facd39700 1 Processor -- start 2023-10-16T11:19:32.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.750+0000 7f4facd39700 1 -- start start 2023-10-16T11:19:32.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4facd39700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fa80ce6c0 0x7f4fa80cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4facd39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa80cc6a0 0x7f4fa80ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4facd39700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa80b1f70 con 0x7f4fa815f150 2023-10-16T11:19:32.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4facd39700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa80b2ae0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4facd39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa80b2dc0 con 0x7f4fa80cc6a0 2023-10-16T11:19:32.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa5d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fa80ce6c0 0x7f4fa80cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa5d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fa80ce6c0 0x7f4fa80cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55652/0 (socket says 172.21.15.181:55652) 2023-10-16T11:19:32.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4fa815f150 0x7f4fa80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38496/0 (socket says 172.21.15.181:38496) 2023-10-16T11:19:32.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/1429554880 learned_addr learned my addr 172.21.15.181:0/1429554880 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:32.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa6d9d700 1 --2- 172.21.15.181:0/1429554880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa80cc6a0 0x7f4fa80ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/1429554880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa80cc6a0 msgr2=0x7f4fa80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:32.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa5d9b700 1 --2- 172.21.15.181:0/1429554880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa80cc6a0 0x7f4fa80ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/1429554880 >> v1:172.21.15.181:6789/0 conn(0x7f4fa815f150 legacy=0x7f4fa80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/1429554880 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa80cf9d0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.751+0000 7f4fa559a700 1 -- 172.21.15.181:0/1429554880 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397477364 0 0) 0x7f4fa80b1f70 con 0x7f4fa815f150 2023-10-16T11:19:32.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4fa5d9b700 1 --2- 172.21.15.181:0/1429554880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80cc160 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f4f980098f0 tx=0x7f4f98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f524edeac26fd486 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4fa559a700 1 -- 172.21.15.181:0/1429554880 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800b8a0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4fa559a700 1 -- 172.21.15.181:0/1429554880 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9800ba00 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4fa559a700 1 -- 172.21.15.181:0/1429554880 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800bd90 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 -- 172.21.15.181:0/1429554880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 msgr2=0x7f4fa80cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 --2- 172.21.15.181:0/1429554880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80cc160 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f4f980098f0 tx=0x7f4f98009c00 comp rx=0 tx=0).stop 2023-10-16T11:19:32.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 -- 172.21.15.181:0/1429554880 shutdown_connections 2023-10-16T11:19:32.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 --2- 172.21.15.181:0/1429554880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa80cc6a0 0x7f4fa80ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 --2- 172.21.15.181:0/1429554880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80cc160 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 -- 172.21.15.181:0/1429554880 >> 172.21.15.181:0/1429554880 conn(0x7f4fa809a930 msgr2=0x7f4fa80d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:32.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 -- 172.21.15.181:0/1429554880 shutdown_connections 2023-10-16T11:19:32.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.752+0000 7f4facd39700 1 -- 172.21.15.181:0/1429554880 wait complete. 2023-10-16T11:19:32.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.754+0000 7f4facd39700 1 Processor -- start 2023-10-16T11:19:32.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.754+0000 7f4facd39700 1 -- start start 2023-10-16T11:19:32.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.754+0000 7f4facd39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa80cc6a0 0x7f4fa80c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.754+0000 7f4facd39700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.754+0000 7f4facd39700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa815f150 0x7f4fa80b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.754+0000 7f4facd39700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81f14e0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4facd39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81f5a00 con 0x7f4fa80cc6a0 2023-10-16T11:19:32.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4facd39700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa80cb2d0 con 0x7f4fa815f150 2023-10-16T11:19:32.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55658/0 (socket says 172.21.15.181:55658) 2023-10-16T11:19:32.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/2511814974 learned_addr learned my addr 172.21.15.181:0/2511814974 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:32.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa6d9d700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa815f150 0x7f4fa80b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa659c700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa80cc6a0 0x7f4fa80c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa815f150 msgr2=0x7f4fa80b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa815f150 0x7f4fa80b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa80cc6a0 msgr2=0x7f4fa80c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa80cc6a0 0x7f4fa80c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 -- 172.21.15.181:0/2511814974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f98009550 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa6d9d700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa815f150 0x7f4fa80b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:32.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa659c700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa80cc6a0 0x7f4fa80c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:32.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4fa5d9b700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80b1a90 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009c40 tx=0x7f4f98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4f96ffd700 1 -- 172.21.15.181:0/2511814974 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98004300 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4f96ffd700 1 -- 172.21.15.181:0/2511814974 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9801f070 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4f96ffd700 1 -- 172.21.15.181:0/2511814974 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f980036a0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa81f1d40 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.755+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa81f16f0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.756+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa8067d20 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.757+0000 7f4f96ffd700 1 -- 172.21.15.181:0/2511814974 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4f98021030 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.757+0000 7f4f96ffd700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f8806b020 0x7f4f8806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:32.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.757+0000 7f4fa659c700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f8806b020 0x7f4f8806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:32.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.757+0000 7f4f96ffd700 1 -- 172.21.15.181:0/2511814974 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f4f9808acb0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.758+0000 7f4fa659c700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f8806b020 0x7f4f8806d4e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c000f60 tx=0x7f4f9c007d30 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:32.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.762+0000 7f4f96ffd700 1 -- 172.21.15.181:0/2511814974 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f98055bd0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.952+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fa80cd9a0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.952+0000 7f4f96ffd700 1 -- 172.21.15.181:0/2511814974 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v216) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f980150a0 con 0x7f4fa80ce6c0 2023-10-16T11:19:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:19:32.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.954+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f8806b020 msgr2=0x7f4f8806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.954+0000 7f4facd39700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f8806b020 0x7f4f8806d4e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c000f60 tx=0x7f4f9c007d30 comp rx=0 tx=0).stop 2023-10-16T11:19:32.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 msgr2=0x7f4fa80b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:32.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80b1a90 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009c40 tx=0x7f4f98004750 comp rx=0 tx=0).stop 2023-10-16T11:19:32.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 shutdown_connections 2023-10-16T11:19:32.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4f8806b020 0x7f4f8806d4e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa815f150 0x7f4fa80b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa80cc6a0 0x7f4fa80c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 --2- 172.21.15.181:0/2511814974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80ce6c0 0x7f4fa80b1a90 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:32.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 >> 172.21.15.181:0/2511814974 conn(0x7f4fa809a930 msgr2=0x7f4fa80b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:32.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 shutdown_connections 2023-10-16T11:19:32.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:32.955+0000 7f4facd39700 1 -- 172.21.15.181:0/2511814974 wait complete. 2023-10-16T11:19:32.969 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:19:33.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.087+0000 7f3d31727700 1 Processor -- start 2023-10-16T11:19:33.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.087+0000 7f3d31727700 1 -- start start 2023-10-16T11:19:33.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d31727700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d2c15c210 0x7f3d2c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d31727700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c06a8a0 0x7f3d2c06ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d31727700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c06b240 con 0x7f3d2c15f3b0 2023-10-16T11:19:33.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d31727700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c1562a0 con 0x7f3d2c15c210 2023-10-16T11:19:33.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d31727700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0cf380 con 0x7f3d2c06a8a0 2023-10-16T11:19:33.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d2c15c210 0x7f3d2c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d2c15c210 0x7f3d2c0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55664/0 (socket says 172.21.15.181:55664) 2023-10-16T11:19:33.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 -- 172.21.15.181:0/2821629707 learned_addr learned my addr 172.21.15.181:0/2821629707 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:33.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 -- 172.21.15.181:0/2821629707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c06a8a0 msgr2=0x7f3d2c06ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:33.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 --2- 172.21.15.181:0/2821629707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c06a8a0 0x7f3d2c06ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 -- 172.21.15.181:0/2821629707 >> v1:172.21.15.190:6789/0 conn(0x7f3d2c15f3b0 legacy=0x7f3d2c15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:33.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 -- 172.21.15.181:0/2821629707 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c15cd00 con 0x7f3d2c15c210 2023-10-16T11:19:33.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.088+0000 7f3d2affd700 1 --2- 172.21.15.181:0/2821629707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0cc1a0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f3d20006fd0 tx=0x7f3d20009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb69ef94623c1273 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:33.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d29ffb700 1 -- 172.21.15.181:0/2821629707 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20005e30 con 0x7f3d2c15c210 2023-10-16T11:19:33.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d29ffb700 1 -- 172.21.15.181:0/2821629707 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d20006030 con 0x7f3d2c15c210 2023-10-16T11:19:33.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d29ffb700 1 -- 172.21.15.181:0/2821629707 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d200063c0 con 0x7f3d2c15c210 2023-10-16T11:19:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 -- 172.21.15.181:0/2821629707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 msgr2=0x7f3d2c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 --2- 172.21.15.181:0/2821629707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0cc1a0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f3d20006fd0 tx=0x7f3d20009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 -- 172.21.15.181:0/2821629707 shutdown_connections 2023-10-16T11:19:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 --2- 172.21.15.181:0/2821629707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c06a8a0 0x7f3d2c06ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 --2- 172.21.15.181:0/2821629707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0cc1a0 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 -- 172.21.15.181:0/2821629707 >> 172.21.15.181:0/2821629707 conn(0x7f3d2c099a00 msgr2=0x7f3d2c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 -- 172.21.15.181:0/2821629707 shutdown_connections 2023-10-16T11:19:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.089+0000 7f3d31727700 1 -- 172.21.15.181:0/2821629707 wait complete. 2023-10-16T11:19:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.090+0000 7f3d31727700 1 Processor -- start 2023-10-16T11:19:33.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d31727700 1 -- start start 2023-10-16T11:19:33.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d31727700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d2c06a8a0 0x7f3d2c154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d31727700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d31727700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c15f3b0 0x7f3d2c0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d31727700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0c84a0 con 0x7f3d2c15c210 2023-10-16T11:19:33.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d31727700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0c92c0 con 0x7f3d2c06a8a0 2023-10-16T11:19:33.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d31727700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0c86b0 con 0x7f3d2c15f3b0 2023-10-16T11:19:33.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d2a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d2a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55670/0 (socket says 172.21.15.181:55670) 2023-10-16T11:19:33.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.091+0000 7f3d2a7fc700 1 -- 172.21.15.181:0/3352551960 learned_addr learned my addr 172.21.15.181:0/3352551960 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2b7fe700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c15f3b0 0x7f3d2c0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2affd700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d2c06a8a0 0x7f3d2c154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2a7fc700 1 -- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c15f3b0 msgr2=0x7f3d2c0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2a7fc700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c15f3b0 0x7f3d2c0ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2a7fc700 1 -- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d2c06a8a0 msgr2=0x7f3d2c154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2a7fc700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d2c06a8a0 0x7f3d2c154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2a7fc700 1 -- 172.21.15.181:0/3352551960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d20009500 con 0x7f3d2c15c210 2023-10-16T11:19:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d2a7fc700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0c9c80 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d137fe700 1 -- 172.21.15.181:0/3352551960 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c011830 con 0x7f3d2c15c210 2023-10-16T11:19:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d137fe700 1 -- 172.21.15.181:0/3352551960 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c011990 con 0x7f3d2c15c210 2023-10-16T11:19:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c0c8990 con 0x7f3d2c15c210 2023-10-16T11:19:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d137fe700 1 -- 172.21.15.181:0/3352551960 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c011d00 con 0x7f3d2c15c210 2023-10-16T11:19:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.092+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c0c8f50 con 0x7f3d2c15c210 2023-10-16T11:19:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.093+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c0c9470 con 0x7f3d2c15c210 2023-10-16T11:19:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.093+0000 7f3d137fe700 1 -- 172.21.15.181:0/3352551960 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3d1c01a400 con 0x7f3d2c15c210 2023-10-16T11:19:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.094+0000 7f3d137fe700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3d1806b020 0x7f3d1806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.094+0000 7f3d2affd700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3d1806b020 0x7f3d1806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.094+0000 7f3d137fe700 1 -- 172.21.15.181:0/3352551960 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f3d1c089920 con 0x7f3d2c15c210 2023-10-16T11:19:33.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.094+0000 7f3d2affd700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3d1806b020 0x7f3d1806d4e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d20013040 tx=0x7f3d20008070 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:33.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.098+0000 7f3d137fe700 1 -- 172.21.15.181:0/3352551960 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3d2c0c9470 con 0x7f3d2c15c210 2023-10-16T11:19:33.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.292+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3d2c0c9430 con 0x7f3d2c15c210 2023-10-16T11:19:33.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.292+0000 7f3d137fe700 1 -- 172.21.15.181:0/3352551960 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v216) v1 ==== 84+0+209 (secure 0 0 0) 0x7f3d1c054690 con 0x7f3d2c15c210 2023-10-16T11:19:33.294 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:33.295 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:19:33.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:33.295 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:19:33.295 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:19:33.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:33.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:33.296 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 26, 2023-10-16T11:19:33.296 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "cephfs_metadata" 2023-10-16T11:19:33.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:33.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:33.296 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 27, 2023-10-16T11:19:33.297 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "cephfs_data" 2023-10-16T11:19:33.297 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:33.297 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:19:33.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3d1806b020 msgr2=0x7f3d1806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3d1806b020 0x7f3d1806d4e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d20013040 tx=0x7f3d20008070 comp rx=0 tx=0).stop 2023-10-16T11:19:33.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 msgr2=0x7f3d2c0c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0c9c80 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c00b7e0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 shutdown_connections 2023-10-16T11:19:33.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f3d1806b020 0x7f3d1806d4e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d2c15f3b0 0x7f3d2c0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d2c06a8a0 0x7f3d2c154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 --2- 172.21.15.181:0/3352551960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d2c15c210 0x7f3d2c0c9c80 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 >> 172.21.15.181:0/3352551960 conn(0x7f3d2c099a00 msgr2=0x7f3d2c06db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:33.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 shutdown_connections 2023-10-16T11:19:33.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.294+0000 7f3d31727700 1 -- 172.21.15.181:0/3352551960 wait complete. 2023-10-16T11:19:33.308 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-16T11:19:33.309 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-16T11:19:33.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.431+0000 7ffb8351d700 1 Processor -- start 2023-10-16T11:19:33.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.431+0000 7ffb8351d700 1 -- start start 2023-10-16T11:19:33.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb8351d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb7c0afa10 0x7ffb7c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb8351d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c06d870 0x7ffb7c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb8351d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb7c0cd3f0 0x7ffb7c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb8351d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c0cd850 con 0x7ffb7c0cd3f0 2023-10-16T11:19:33.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb8351d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c15b540 con 0x7ffb7c0afa10 2023-10-16T11:19:33.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb8351d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c15b820 con 0x7ffb7c06d870 2023-10-16T11:19:33.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb7c0cd3f0 0x7ffb7c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb7c0cd3f0 0x7ffb7c0cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55678/0 (socket says 172.21.15.181:55678) 2023-10-16T11:19:33.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 -- 172.21.15.181:0/2864768683 learned_addr learned my addr 172.21.15.181:0/2864768683 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:33.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb80ab8700 1 --2- 172.21.15.181:0/2864768683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c06d870 0x7ffb7c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb812b9700 1 --2- 172.21.15.181:0/2864768683 >> v2:172.21.15.190:3300/0 conn(0x7ffb7c0afa10 0x7ffb7c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 -- 172.21.15.181:0/2864768683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c06d870 msgr2=0x7ffb7c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 --2- 172.21.15.181:0/2864768683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c06d870 0x7ffb7c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 -- 172.21.15.181:0/2864768683 >> v2:172.21.15.190:3300/0 conn(0x7ffb7c0afa10 msgr2=0x7ffb7c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 --2- 172.21.15.181:0/2864768683 >> v2:172.21.15.190:3300/0 conn(0x7ffb7c0afa10 0x7ffb7c0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 -- 172.21.15.181:0/2864768683 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c15bb00 con 0x7ffb7c0cd3f0 2023-10-16T11:19:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb81aba700 1 --2- 172.21.15.181:0/2864768683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c0cd3f0 0x7ffb7c0cb8b0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ce020 tx=0x7ffb7400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f203f989fddb8a46 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb73fff700 1 -- 172.21.15.181:0/2864768683 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74010070 con 0x7ffb7c0cd3f0 2023-10-16T11:19:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb73fff700 1 -- 172.21.15.181:0/2864768683 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7400dd90 con 0x7ffb7c0cd3f0 2023-10-16T11:19:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.432+0000 7ffb73fff700 1 -- 172.21.15.181:0/2864768683 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400b310 con 0x7ffb7c0cd3f0 2023-10-16T11:19:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 -- 172.21.15.181:0/2864768683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c0cd3f0 msgr2=0x7ffb7c0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2864768683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c0cd3f0 0x7ffb7c0cb8b0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ce020 tx=0x7ffb7400d430 comp rx=0 tx=0).stop 2023-10-16T11:19:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 -- 172.21.15.181:0/2864768683 shutdown_connections 2023-10-16T11:19:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2864768683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c0cd3f0 0x7ffb7c0cb8b0 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2864768683 >> v2:172.21.15.190:3300/0 conn(0x7ffb7c0afa10 0x7ffb7c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2864768683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c06d870 0x7ffb7c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 -- 172.21.15.181:0/2864768683 >> 172.21.15.181:0/2864768683 conn(0x7ffb7c0d04a0 msgr2=0x7ffb7c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 -- 172.21.15.181:0/2864768683 shutdown_connections 2023-10-16T11:19:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.433+0000 7ffb8351d700 1 -- 172.21.15.181:0/2864768683 wait complete. 2023-10-16T11:19:33.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.434+0000 7ffb8351d700 1 Processor -- start 2023-10-16T11:19:33.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.434+0000 7ffb8351d700 1 -- start start 2023-10-16T11:19:33.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb8351d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c06d870 0x7ffb7c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb8351d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0afa10 0x7ffb7c0c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb8351d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cd3f0 0x7ffb7c0c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb8351d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0ca370 con 0x7ffb7c06d870 2023-10-16T11:19:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb8351d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0c6300 con 0x7ffb7c0cd3f0 2023-10-16T11:19:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb8351d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0c65e0 con 0x7ffb7c0afa10 2023-10-16T11:19:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c06d870 0x7ffb7c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c06d870 0x7ffb7c154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55682/0 (socket says 172.21.15.181:55682) 2023-10-16T11:19:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 -- 172.21.15.181:0/2478156400 learned_addr learned my addr 172.21.15.181:0/2478156400 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb80ab8700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0afa10 0x7ffb7c0c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb81aba700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cd3f0 0x7ffb7c0c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 -- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0afa10 msgr2=0x7ffb7c0c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0afa10 0x7ffb7c0c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 -- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cd3f0 msgr2=0x7ffb7c0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cd3f0 0x7ffb7c0c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.435+0000 7ffb812b9700 1 -- 172.21.15.181:0/2478156400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7400bd40 con 0x7ffb7c06d870 2023-10-16T11:19:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.436+0000 7ffb812b9700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c06d870 0x7ffb7c154fd0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e730 tx=0x7ffb7800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.436+0000 7ffb71ffb700 1 -- 172.21.15.181:0/2478156400 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78013610 con 0x7ffb7c06d870 2023-10-16T11:19:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.436+0000 7ffb71ffb700 1 -- 172.21.15.181:0/2478156400 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78013770 con 0x7ffb7c06d870 2023-10-16T11:19:33.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.436+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c0c68c0 con 0x7ffb7c06d870 2023-10-16T11:19:33.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.436+0000 7ffb71ffb700 1 -- 172.21.15.181:0/2478156400 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78013b20 con 0x7ffb7c06d870 2023-10-16T11:19:33.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.436+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c155e10 con 0x7ffb7c06d870 2023-10-16T11:19:33.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.437+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c0ccd20 con 0x7ffb7c06d870 2023-10-16T11:19:33.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.437+0000 7ffb71ffb700 1 -- 172.21.15.181:0/2478156400 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffb78013c80 con 0x7ffb7c06d870 2023-10-16T11:19:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.438+0000 7ffb71ffb700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ffb6006b020 0x7ffb6006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.438+0000 7ffb71ffb700 1 -- 172.21.15.181:0/2478156400 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ffb78089420 con 0x7ffb7c06d870 2023-10-16T11:19:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.438+0000 7ffb80ab8700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ffb6006b020 0x7ffb6006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.438+0000 7ffb80ab8700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ffb6006b020 0x7ffb6006d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c1561c0 tx=0x7ffb6c008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.442+0000 7ffb71ffb700 1 -- 172.21.15.181:0/2478156400 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb78053ff0 con 0x7ffb7c06d870 2023-10-16T11:19:33.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:33.635+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ffb7c0727a0 con 0x7ffb7c06d870 2023-10-16T11:19:34.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.598+0000 7ffb71ffb700 1 -- 172.21.15.181:0/2478156400 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v217) v1 ==== 144+0+0 (secure 0 0 0) 0x7ffb78057610 con 0x7ffb7c06d870 2023-10-16T11:19:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ffb6006b020 msgr2=0x7ffb6006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ffb6006b020 0x7ffb6006d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c1561c0 tx=0x7ffb6c008040 comp rx=0 tx=0).stop 2023-10-16T11:19:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c06d870 msgr2=0x7ffb7c154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c06d870 0x7ffb7c154fd0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e730 tx=0x7ffb7800ea40 comp rx=0 tx=0).stop 2023-10-16T11:19:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 shutdown_connections 2023-10-16T11:19:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ffb6006b020 0x7ffb6006d4e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cd3f0 0x7ffb7c0c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb7c06d870 0x7ffb7c154fd0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 --2- 172.21.15.181:0/2478156400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0afa10 0x7ffb7c0c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.603+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 >> 172.21.15.181:0/2478156400 conn(0x7ffb7c0d04a0 msgr2=0x7ffb7c0cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:34.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.604+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 shutdown_connections 2023-10-16T11:19:34.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.604+0000 7ffb8351d700 1 -- 172.21.15.181:0/2478156400 wait complete. 2023-10-16T11:19:34.608 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_metadata' created 2023-10-16T11:19:34.624 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:19:34.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.754+0000 7ff320f3d700 1 Processor -- start 2023-10-16T11:19:34.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.755+0000 7ff320f3d700 1 -- start start 2023-10-16T11:19:34.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.755+0000 7ff320f3d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff31c15f150 0x7ff31c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff320f3d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 0x7ff31c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff320f3d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c0b2ae0 con 0x7ff31c0ce4f0 2023-10-16T11:19:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff320f3d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c0ce9a0 con 0x7ff31c15f150 2023-10-16T11:19:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff320f3d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0b2dc0 con 0x7ff31c0cd170 2023-10-16T11:19:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff319d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff31c0ce4f0 0x7ff31c0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38538/0 (socket says 172.21.15.181:38538) 2023-10-16T11:19:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff319d9b700 1 -- 172.21.15.181:0/2490567263 learned_addr learned my addr 172.21.15.181:0/2490567263 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31ad9d700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 0x7ff31c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31a59c700 1 --2- 172.21.15.181:0/2490567263 >> v2:172.21.15.190:3300/0 conn(0x7ff31c15f150 0x7ff31c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177645297 0 0) 0x7ff31c0b2ae0 con 0x7ff31c0ce4f0 2023-10-16T11:19:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3040035b0 con 0x7ff31c0ce4f0 2023-10-16T11:19:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260810284 0 0) 0x7ff3040035b0 con 0x7ff31c0ce4f0 2023-10-16T11:19:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 msgr2=0x7ff31c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 0x7ff31c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 >> v2:172.21.15.190:3300/0 conn(0x7ff31c15f150 msgr2=0x7ff31c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 --2- 172.21.15.181:0/2490567263 >> v2:172.21.15.190:3300/0 conn(0x7ff31c15f150 0x7ff31c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0cf9d0 con 0x7ff31c0ce4f0 2023-10-16T11:19:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff30c001670 con 0x7ff31c0ce4f0 2023-10-16T11:19:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 >> v1:172.21.15.181:6789/0 conn(0x7ff31c0ce4f0 legacy=0x7ff31c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 0x7ff304003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff304003f80 0x7ff304004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff304005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0ce9a0 con 0x7ff304004d90 2023-10-16T11:19:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0b2dc0 con 0x7ff304003f80 2023-10-16T11:19:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30400ba10 con 0x7ff31c0cd170 2023-10-16T11:19:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.756+0000 7ff31ad9d700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff304005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31a59c700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 0x7ff304003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff319d9b700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff304003f80 0x7ff304004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31ad9d700 1 -- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 msgr2=0x7ff304003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31ad9d700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0cd170 0x7ff304003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31ad9d700 1 -- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff304003f80 msgr2=0x7ff304004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31ad9d700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff304003f80 0x7ff304004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31ad9d700 1 -- 172.21.15.181:0/2490567263 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040035b0 con 0x7ff304004d90 2023-10-16T11:19:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31ad9d700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff304005210 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7ff314009fc0 tx=0x7ff304005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42bf1df317c612c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:34.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400ebd0 con 0x7ff304004d90 2023-10-16T11:19:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31400f040 con 0x7ff304004d90 2023-10-16T11:19:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.757+0000 7ff31959a700 1 -- 172.21.15.181:0/2490567263 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314014480 con 0x7ff304004d90 2023-10-16T11:19:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.758+0000 7ff320f3d700 1 -- 172.21.15.181:0/2490567263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 msgr2=0x7ff304005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.758+0000 7ff320f3d700 1 --2- 172.21.15.181:0/2490567263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff304005210 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7ff314009fc0 tx=0x7ff304005880 comp rx=0 tx=0).stop 2023-10-16T11:19:34.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.758+0000 7ff31ad9d700 1 -- 172.21.15.181:0/2490567263 reap_dead start 2023-10-16T11:19:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.758+0000 7ff320f3d700 1 -- 172.21.15.181:0/2490567263 shutdown_connections 2023-10-16T11:19:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.758+0000 7ff320f3d700 1 -- 172.21.15.181:0/2490567263 >> 172.21.15.181:0/2490567263 conn(0x7ff31c09a930 msgr2=0x7ff31c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.758+0000 7ff320f3d700 1 -- 172.21.15.181:0/2490567263 shutdown_connections 2023-10-16T11:19:34.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.758+0000 7ff320f3d700 1 -- 172.21.15.181:0/2490567263 wait complete. 2023-10-16T11:19:34.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.759+0000 7ff320f3d700 1 Processor -- start 2023-10-16T11:19:34.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.759+0000 7ff320f3d700 1 -- start start 2023-10-16T11:19:34.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff320f3d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff31c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff320f3d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff31c1f6e90 0x7ff31c06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff320f3d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0b6930 0x7ff31c0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff320f3d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c06bef0 con 0x7ff304004d90 2023-10-16T11:19:34.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff320f3d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c06cd30 con 0x7ff31c1f6e90 2023-10-16T11:19:34.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff320f3d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c1f7310 con 0x7ff31c0b6930 2023-10-16T11:19:34.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff31c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff31c06d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55702/0 (socket says 172.21.15.181:55702) 2023-10-16T11:19:34.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 -- 172.21.15.181:0/3642790293 learned_addr learned my addr 172.21.15.181:0/3642790293 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:34.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31ad9d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0b6930 0x7ff31c0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff319d9b700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff31c1f6e90 0x7ff31c06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 -- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0b6930 msgr2=0x7ff31c0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:34.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0b6930 0x7ff31c0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 -- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff31c1f6e90 msgr2=0x7ff31c06b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:34.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff31c1f6e90 0x7ff31c06b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:34.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 -- 172.21.15.181:0/3642790293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0b6de0 con 0x7ff304004d90 2023-10-16T11:19:34.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31ad9d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0b6930 0x7ff31c0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:34.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff319d9b700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff31c1f6e90 0x7ff31c06b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:34.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff31a59c700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff31c06d1c0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7ff310011ad0 tx=0x7ff310011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:34.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff30affd700 1 -- 172.21.15.181:0/3642790293 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31000fa00 con 0x7ff304004d90 2023-10-16T11:19:34.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.760+0000 7ff30affd700 1 -- 172.21.15.181:0/3642790293 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31000fb60 con 0x7ff304004d90 2023-10-16T11:19:34.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.761+0000 7ff30affd700 1 -- 172.21.15.181:0/3642790293 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310014440 con 0x7ff304004d90 2023-10-16T11:19:34.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.761+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c06c120 con 0x7ff304004d90 2023-10-16T11:19:34.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.761+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c06c6e0 con 0x7ff304004d90 2023-10-16T11:19:34.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.762+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c067d20 con 0x7ff304004d90 2023-10-16T11:19:34.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.762+0000 7ff30affd700 1 -- 172.21.15.181:0/3642790293 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff3100145a0 con 0x7ff304004d90 2023-10-16T11:19:34.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.763+0000 7ff30affd700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:34.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.763+0000 7ff319d9b700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:34.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.763+0000 7ff30affd700 1 -- 172.21.15.181:0/3642790293 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6634+0+0 (secure 0 0 0) 0x7ff31008a1c0 con 0x7ff304004d90 2023-10-16T11:19:34.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.763+0000 7ff319d9b700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff31c15a660 tx=0x7ff30c009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:34.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.768+0000 7ff30affd700 1 -- 172.21.15.181:0/3642790293 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff310054e50 con 0x7ff304004d90 2023-10-16T11:19:34.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:34.960+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff31c0cd790 con 0x7ff304004d90 2023-10-16T11:19:35.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.609+0000 7ff30affd700 1 -- 172.21.15.181:0/3642790293 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v218) v1 ==== 176+0+0 (secure 0 0 0) 0x7ff310058470 con 0x7ff304004d90 2023-10-16T11:19:35.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.611+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff2fc06b020 msgr2=0x7ff2fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff31c15a660 tx=0x7ff30c009040 comp rx=0 tx=0).stop 2023-10-16T11:19:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 msgr2=0x7ff31c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff31c06d1c0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7ff310011ad0 tx=0x7ff310011e90 comp rx=0 tx=0).stop 2023-10-16T11:19:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 shutdown_connections 2023-10-16T11:19:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff2fc06b020 0x7ff2fc06d4e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff31c0b6930 0x7ff31c0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff304004d90 0x7ff31c06d1c0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 --2- 172.21.15.181:0/3642790293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff31c1f6e90 0x7ff31c06b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.612+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 >> 172.21.15.181:0/3642790293 conn(0x7ff31c09a930 msgr2=0x7ff31c15a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.613+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 shutdown_connections 2023-10-16T11:19:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.613+0000 7ff320f3d700 1 -- 172.21.15.181:0/3642790293 wait complete. 2023-10-16T11:19:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_data' created 2023-10-16T11:19:35.643 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-16T11:19:35.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.766+0000 7f4c9a3f6700 1 Processor -- start 2023-10-16T11:19:35.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.766+0000 7f4c9a3f6700 1 -- start start 2023-10-16T11:19:35.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c9a3f6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c940d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:35.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c9a3f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c940ce490 0x7f4c9406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:35.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c9a3f6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9415dfb0 con 0x7f4c940d0610 2023-10-16T11:19:35.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c9a3f6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940cd3a0 con 0x7f4c940ce490 2023-10-16T11:19:35.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c9a3f6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940cd830 con 0x7f4c9415dbd0 2023-10-16T11:19:35.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c98993700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c940ce490 0x7f4c9406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:35.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c98993700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c940ce490 0x7f4c9406b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55714/0 (socket says 172.21.15.181:55714) 2023-10-16T11:19:35.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.767+0000 7f4c98993700 1 -- 172.21.15.181:0/848338904 learned_addr learned my addr 172.21.15.181:0/848338904 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:35.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c93fff700 1 --2- 172.21.15.181:0/848338904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c940d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:35.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c98993700 1 -- 172.21.15.181:0/848338904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 msgr2=0x7f4c940d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:35.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c98993700 1 --2- 172.21.15.181:0/848338904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c940d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c98993700 1 -- 172.21.15.181:0/848338904 >> v1:172.21.15.190:6789/0 conn(0x7f4c940d0610 legacy=0x7f4c940cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:35.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c98993700 1 -- 172.21.15.181:0/848338904 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940d1fc0 con 0x7f4c940ce490 2023-10-16T11:19:35.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c93fff700 1 --2- 172.21.15.181:0/848338904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c940d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:35.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c98993700 1 --2- 172.21.15.181:0/848338904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c9406b6f0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009e50 tx=0x7f4c8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1101ad22c649da3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:35.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c92ffd700 1 -- 172.21.15.181:0/848338904 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00e070 con 0x7f4c940ce490 2023-10-16T11:19:35.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c92ffd700 1 -- 172.21.15.181:0/848338904 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c003a20 con 0x7f4c940ce490 2023-10-16T11:19:35.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c92ffd700 1 -- 172.21.15.181:0/848338904 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003dd0 con 0x7f4c940ce490 2023-10-16T11:19:35.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/848338904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 msgr2=0x7f4c9406b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:35.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.768+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/848338904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c9406b6f0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009e50 tx=0x7f4c8c00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:35.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.769+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/848338904 shutdown_connections 2023-10-16T11:19:35.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.769+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/848338904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c9406b6f0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.769+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/848338904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c940d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.769+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/848338904 >> 172.21.15.181:0/848338904 conn(0x7f4c9409a930 msgr2=0x7f4c940d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:35.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.769+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/848338904 shutdown_connections 2023-10-16T11:19:35.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.769+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/848338904 wait complete. 2023-10-16T11:19:35.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.770+0000 7f4c9a3f6700 1 Processor -- start 2023-10-16T11:19:35.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.770+0000 7f4c9a3f6700 1 -- start start 2023-10-16T11:19:35.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c94157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:35.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c940d0610 0x7f4c941f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:35.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c94160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:35.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94156a90 con 0x7f4c940ce490 2023-10-16T11:19:35.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94156d70 con 0x7f4c940d0610 2023-10-16T11:19:35.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94157050 con 0x7f4c9415dbd0 2023-10-16T11:19:35.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c94157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:35.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c94157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55716/0 (socket says 172.21.15.181:55716) 2023-10-16T11:19:35.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 -- 172.21.15.181:0/711779471 learned_addr learned my addr 172.21.15.181:0/711779471 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:35.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c98993700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c94160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:35.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c937fe700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c940d0610 0x7f4c941f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:35.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 -- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 msgr2=0x7f4c94160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:35.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c94160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 -- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c940d0610 msgr2=0x7f4c941f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:35.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c940d0610 0x7f4c941f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:35.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 -- 172.21.15.181:0/711779471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009b30 con 0x7f4c940ce490 2023-10-16T11:19:35.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c937fe700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c940d0610 0x7f4c941f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:35.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c93fff700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c94157b30 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c7f0 tx=0x7f4c8400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:35.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c90ff9700 1 -- 172.21.15.181:0/711779471 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84011800 con 0x7f4c940ce490 2023-10-16T11:19:35.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c90ff9700 1 -- 172.21.15.181:0/711779471 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84011960 con 0x7f4c940ce490 2023-10-16T11:19:35.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c94157330 con 0x7f4c940ce490 2023-10-16T11:19:35.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c90ff9700 1 -- 172.21.15.181:0/711779471 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84011cf0 con 0x7f4c940ce490 2023-10-16T11:19:35.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.771+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c941611b0 con 0x7f4c940ce490 2023-10-16T11:19:35.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.772+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94067d20 con 0x7f4c940ce490 2023-10-16T11:19:35.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.773+0000 7f4c90ff9700 1 -- 172.21.15.181:0/711779471 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4c84018360 con 0x7f4c940ce490 2023-10-16T11:19:35.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.773+0000 7f4c90ff9700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c8006b020 0x7f4c8006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:35.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.774+0000 7f4c90ff9700 1 -- 172.21.15.181:0/711779471 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 7019+0+0 (secure 0 0 0) 0x7f4c84089d10 con 0x7f4c940ce490 2023-10-16T11:19:35.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.774+0000 7f4c937fe700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c8006b020 0x7f4c8006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:35.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.774+0000 7f4c937fe700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c8006b020 0x7f4c8006d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c940cf700 tx=0x7f4c88009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:35.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:35.778+0000 7f4c90ff9700 1 -- 172.21.15.181:0/711779471 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c84054690 con 0x7f4c940ce490 2023-10-16T11:19:36.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:36.014+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f4c940cebb0 con 0x7f4c940ce490 2023-10-16T11:19:37.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.012+0000 7f4c90ff9700 1 -- 172.21.15.181:0/711779471 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 28 and data pool 29 v113) v1 ==== 183+0+0 (secure 0 0 0) 0x7f4c84057cb0 con 0x7f4c940ce490 2023-10-16T11:19:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c8006b020 msgr2=0x7f4c8006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c8006b020 0x7f4c8006d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c940cf700 tx=0x7f4c88009040 comp rx=0 tx=0).stop 2023-10-16T11:19:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 msgr2=0x7f4c94157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c94157b30 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c7f0 tx=0x7f4c8400cb00 comp rx=0 tx=0).stop 2023-10-16T11:19:37.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 shutdown_connections 2023-10-16T11:19:37.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4c8006b020 0x7f4c8006d4e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9415dbd0 0x7f4c94160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c940ce490 0x7f4c94157b30 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 --2- 172.21.15.181:0/711779471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c940d0610 0x7f4c941f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 >> 172.21.15.181:0/711779471 conn(0x7f4c9409a930 msgr2=0x7f4c940cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:37.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 shutdown_connections 2023-10-16T11:19:37.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.017+0000 7f4c9a3f6700 1 -- 172.21.15.181:0/711779471 wait complete. 2023-10-16T11:19:37.022 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 28 and data pool 29 2023-10-16T11:19:37.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:37.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.162+0000 7f9daaa71700 1 Processor -- start 2023-10-16T11:19:37.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.162+0000 7f9daaa71700 1 -- start start 2023-10-16T11:19:37.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.163+0000 7f9daaa71700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9da415dfd0 0x7f9da40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.163+0000 7f9daaa71700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da406f870 con 0x7f9da40af450 2023-10-16T11:19:37.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.163+0000 7f9daaa71700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da40d1160 con 0x7f9da40cd470 2023-10-16T11:19:37.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.163+0000 7f9daaa71700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da40cc8e0 con 0x7f9da415dfd0 2023-10-16T11:19:37.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.163+0000 7f9da3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9da40af450 0x7f9da406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38586/0 (socket says 172.21.15.181:38586) 2023-10-16T11:19:37.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.163+0000 7f9da3fff700 1 -- 172.21.15.181:0/1291161051 learned_addr learned my addr 172.21.15.181:0/1291161051 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:37.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289986078 0 0) 0x7f9da406f870 con 0x7f9da40af450 2023-10-16T11:19:37.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d940035b0 con 0x7f9da40af450 2023-10-16T11:19:37.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da880d700 1 --2- 172.21.15.181:0/1291161051 >> v2:172.21.15.190:3300/0 conn(0x7f9da415dfd0 0x7f9da40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872174395 0 0) 0x7f9d940035b0 con 0x7f9da40af450 2023-10-16T11:19:37.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 >> v2:172.21.15.190:3300/0 conn(0x7f9da415dfd0 msgr2=0x7f9da40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:37.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 --2- 172.21.15.181:0/1291161051 >> v2:172.21.15.190:3300/0 conn(0x7f9da415dfd0 0x7f9da40af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 >> v1:172.21.15.190:6789/0 conn(0x7f9da40cd470 legacy=0x7f9da406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40ccbc0 con 0x7f9da40af450 2023-10-16T11:19:37.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f9d90001670 con 0x7f9da40af450 2023-10-16T11:19:37.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 >> v1:172.21.15.181:6789/0 conn(0x7f9da40af450 legacy=0x7f9da406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d94003a00 0x7f9d94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94004850 0x7f9d94004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94005330 0x7f9d9400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40cc8e0 con 0x7f9d94004850 2023-10-16T11:19:37.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9400c200 con 0x7f9d94005330 2023-10-16T11:19:37.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9400c3c0 con 0x7f9d94003a00 2023-10-16T11:19:37.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da3fff700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94004850 0x7f9d94004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da880d700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d94003a00 0x7f9d94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da900e700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94005330 0x7f9d9400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da3fff700 1 -- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d94003a00 msgr2=0x7f9d94003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:37.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da3fff700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d94003a00 0x7f9d94003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da3fff700 1 -- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94005330 msgr2=0x7f9d9400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:37.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da3fff700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94005330 0x7f9d9400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da900e700 1 -- 172.21.15.181:0/1291161051 reap_dead start 2023-10-16T11:19:37.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.164+0000 7f9da3fff700 1 -- 172.21.15.181:0/1291161051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d940035b0 con 0x7f9d94004850 2023-10-16T11:19:37.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9da3fff700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94004850 0x7f9d94004cd0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c210 tx=0x7f9d94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ccfc1bc6438e29 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90007c10 con 0x7f9d94004850 2023-10-16T11:19:37.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000cdd0 con 0x7f9d94004850 2023-10-16T11:19:37.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9da37fe700 1 -- 172.21.15.181:0/1291161051 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90005ba0 con 0x7f9d94004850 2023-10-16T11:19:37.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9daaa71700 1 -- 172.21.15.181:0/1291161051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94004850 msgr2=0x7f9d94004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94004850 0x7f9d94004cd0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c210 tx=0x7f9d94005880 comp rx=0 tx=0).stop 2023-10-16T11:19:37.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9daaa71700 1 -- 172.21.15.181:0/1291161051 shutdown_connections 2023-10-16T11:19:37.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1291161051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d94004850 0x7f9d94004cd0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9daaa71700 1 -- 172.21.15.181:0/1291161051 >> 172.21.15.181:0/1291161051 conn(0x7f9da40d04a0 msgr2=0x7f9da40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:37.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9daaa71700 1 -- 172.21.15.181:0/1291161051 shutdown_connections 2023-10-16T11:19:37.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.165+0000 7f9daaa71700 1 -- 172.21.15.181:0/1291161051 wait complete. 2023-10-16T11:19:37.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.166+0000 7f9daaa71700 1 Processor -- start 2023-10-16T11:19:37.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.167+0000 7f9daaa71700 1 -- start start 2023-10-16T11:19:37.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.167+0000 7f9daaa71700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94004850 0x7f9da415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.167+0000 7f9daaa71700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9da415d070 0x7f9da40c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.167+0000 7f9daaa71700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9da40c5560 0x7f9da40c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.167+0000 7f9daaa71700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da415d4f0 con 0x7f9da40c5560 2023-10-16T11:19:37.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.167+0000 7f9daaa71700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40b3740 con 0x7f9d94004850 2023-10-16T11:19:37.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.167+0000 7f9daaa71700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40b3a20 con 0x7f9da415d070 2023-10-16T11:19:37.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9da40c5560 0x7f9da40c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9da40c5560 0x7f9da40c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55756/0 (socket says 172.21.15.181:55756) 2023-10-16T11:19:37.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 -- 172.21.15.181:0/1315232592 learned_addr learned my addr 172.21.15.181:0/1315232592 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:37.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da880d700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94004850 0x7f9da415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da3fff700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9da415d070 0x7f9da40c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 -- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9da415d070 msgr2=0x7f9da40c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9da415d070 0x7f9da40c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 -- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94004850 msgr2=0x7f9da415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94004850 0x7f9da415cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 -- 172.21.15.181:0/1315232592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40b3d00 con 0x7f9da40c5560 2023-10-16T11:19:37.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da900e700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9da40c5560 0x7f9da40c59c0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f9da406f3a0 tx=0x7f9d9c00ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da17fa700 1 -- 172.21.15.181:0/1315232592 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c003cd0 con 0x7f9da40c5560 2023-10-16T11:19:37.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da17fa700 1 -- 172.21.15.181:0/1315232592 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c003e70 con 0x7f9da40c5560 2023-10-16T11:19:37.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da40b3fe0 con 0x7f9da40c5560 2023-10-16T11:19:37.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9da17fa700 1 -- 172.21.15.181:0/1315232592 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0052f0 con 0x7f9da40c5560 2023-10-16T11:19:37.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.168+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da40b32f0 con 0x7f9da40c5560 2023-10-16T11:19:37.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.169+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da40b34b0 con 0x7f9da40c5560 2023-10-16T11:19:37.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.170+0000 7f9da17fa700 1 -- 172.21.15.181:0/1315232592 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9d9c005450 con 0x7f9da40c5560 2023-10-16T11:19:37.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.170+0000 7f9da17fa700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d8806b050 0x7f9d8806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.170+0000 7f9da17fa700 1 -- 172.21.15.181:0/1315232592 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f9d9c089db0 con 0x7f9da40c5560 2023-10-16T11:19:37.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.170+0000 7f9da880d700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d8806b050 0x7f9d8806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.170+0000 7f9da880d700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d8806b050 0x7f9d8806d510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006fd0 tx=0x7f9d98008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.175+0000 7f9da17fa700 1 -- 172.21.15.181:0/1315232592 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9d9c0547c0 con 0x7f9da40c5560 2023-10-16T11:19:37.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.367+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da4067d20 con 0x7f9da40c5560 2023-10-16T11:19:37.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.368+0000 7f9da17fa700 1 -- 172.21.15.181:0/1315232592 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v219) v1 ==== 74+0+21207 (secure 0 0 0) 0x7f9d9c089020 con 0x7f9da40c5560 2023-10-16T11:19:37.370 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:37.371 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":219,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:36.016347+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:37.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d8806b050 msgr2=0x7f9d8806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d8806b050 0x7f9d8806d510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006fd0 tx=0x7f9d98008040 comp rx=0 tx=0).stop 2023-10-16T11:19:37.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9da40c5560 msgr2=0x7f9da40c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9da40c5560 0x7f9da40c59c0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f9da406f3a0 tx=0x7f9d9c00ff00 comp rx=0 tx=0).stop 2023-10-16T11:19:37.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 shutdown_connections 2023-10-16T11:19:37.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f9d8806b050 0x7f9d8806d510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9da40c5560 0x7f9da40c59c0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d94004850 0x7f9da415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 --2- 172.21.15.181:0/1315232592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9da415d070 0x7f9da40c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 >> 172.21.15.181:0/1315232592 conn(0x7f9da40d04a0 msgr2=0x7f9da40cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:37.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 shutdown_connections 2023-10-16T11:19:37.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.370+0000 7f9daaa71700 1 -- 172.21.15.181:0/1315232592 wait complete. 2023-10-16T11:19:37.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:37.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.502+0000 7f06a5bd5700 1 Processor -- start 2023-10-16T11:19:37.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.502+0000 7f06a5bd5700 1 -- start start 2023-10-16T11:19:37.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f06a5bd5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06a00cf4e0 0x7f06a00665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f06a5bd5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f06a00ca2e0 0x7f06a00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f06a5bd5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a0066c60 con 0x7f06a015db70 2023-10-16T11:19:37.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f06a5bd5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a00cda10 con 0x7f06a00cf4e0 2023-10-16T11:19:37.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f06a5bd5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a006c1a0 con 0x7f06a00ca2e0 2023-10-16T11:19:37.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f069effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06a00cf4e0 0x7f06a00665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f069f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f06a015db70 0x7f06a00cedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38606/0 (socket says 172.21.15.181:38606) 2023-10-16T11:19:37.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f069f7fe700 1 -- 172.21.15.181:0/3723632097 learned_addr learned my addr 172.21.15.181:0/3723632097 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:37.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.503+0000 7f069effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06a00cf4e0 0x7f06a00665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55762/0 (socket says 172.21.15.181:55762) 2023-10-16T11:19:37.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069ffff700 1 --2- 172.21.15.181:0/3723632097 >> v2:172.21.15.190:3300/0 conn(0x7f06a00ca2e0 0x7f06a00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069effd700 1 -- 172.21.15.181:0/3723632097 >> v2:172.21.15.190:3300/0 conn(0x7f06a00ca2e0 msgr2=0x7f06a00ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:37.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069effd700 1 --2- 172.21.15.181:0/3723632097 >> v2:172.21.15.190:3300/0 conn(0x7f06a00ca2e0 0x7f06a00ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069effd700 1 -- 172.21.15.181:0/3723632097 >> v1:172.21.15.181:6789/0 conn(0x7f06a015db70 legacy=0x7f06a00cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069effd700 1 -- 172.21.15.181:0/3723632097 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a006b3e0 con 0x7f06a00cf4e0 2023-10-16T11:19:37.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069effd700 1 --2- 172.21.15.181:0/3723632097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a00cf4e0 0x7f06a00665a0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f06a00cfc30 tx=0x7f0688009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1d813f23ad57f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069e7fc700 1 -- 172.21.15.181:0/3723632097 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688011830 con 0x7f06a00cf4e0 2023-10-16T11:19:37.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069e7fc700 1 -- 172.21.15.181:0/3723632097 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688011990 con 0x7f06a00cf4e0 2023-10-16T11:19:37.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f069e7fc700 1 -- 172.21.15.181:0/3723632097 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688011d20 con 0x7f06a00cf4e0 2023-10-16T11:19:37.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f06a5bd5700 1 -- 172.21.15.181:0/3723632097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a00cf4e0 msgr2=0x7f06a00665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/3723632097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a00cf4e0 0x7f06a00665a0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f06a00cfc30 tx=0x7f0688009b20 comp rx=0 tx=0).stop 2023-10-16T11:19:37.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f06a5bd5700 1 -- 172.21.15.181:0/3723632097 shutdown_connections 2023-10-16T11:19:37.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/3723632097 >> v2:172.21.15.190:3300/0 conn(0x7f06a00ca2e0 0x7f06a00ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/3723632097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a00cf4e0 0x7f06a00665a0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.504+0000 7f06a5bd5700 1 -- 172.21.15.181:0/3723632097 >> 172.21.15.181:0/3723632097 conn(0x7f06a00b1780 msgr2=0x7f06a00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:37.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.505+0000 7f06a5bd5700 1 -- 172.21.15.181:0/3723632097 shutdown_connections 2023-10-16T11:19:37.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.505+0000 7f06a5bd5700 1 -- 172.21.15.181:0/3723632097 wait complete. 2023-10-16T11:19:37.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.506+0000 7f06a5bd5700 1 Processor -- start 2023-10-16T11:19:37.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.506+0000 7f06a5bd5700 1 -- start start 2023-10-16T11:19:37.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f06a5bd5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06a00ca2e0 0x7f06a00c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f06a5bd5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06a00cf4e0 0x7f06a00c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f06a5bd5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a015db70 0x7f06a009f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f06a5bd5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015e960 con 0x7f06a015db70 2023-10-16T11:19:37.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f06a5bd5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015eb70 con 0x7f06a00cf4e0 2023-10-16T11:19:37.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f06a5bd5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015ee50 con 0x7f06a00ca2e0 2023-10-16T11:19:37.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f069ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a015db70 0x7f06a009f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f069ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a015db70 0x7f06a009f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55770/0 (socket says 172.21.15.181:55770) 2023-10-16T11:19:37.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f069ffff700 1 -- 172.21.15.181:0/215210000 learned_addr learned my addr 172.21.15.181:0/215210000 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:37.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f069f7fe700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06a00ca2e0 0x7f06a00c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.507+0000 7f069effd700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06a00cf4e0 0x7f06a00c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f069ffff700 1 -- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06a00ca2e0 msgr2=0x7f06a00c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f069ffff700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06a00ca2e0 0x7f06a00c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f069ffff700 1 -- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06a00cf4e0 msgr2=0x7f06a00c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f069ffff700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06a00cf4e0 0x7f06a00c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f069ffff700 1 -- 172.21.15.181:0/215210000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880094a0 con 0x7f06a015db70 2023-10-16T11:19:37.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f069effd700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06a00cf4e0 0x7f06a00c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:37.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f069ffff700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a015db70 0x7f06a009f670 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f0698007e40 tx=0x7f069800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f0697fff700 1 -- 172.21.15.181:0/215210000 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800eea0 con 0x7f06a015db70 2023-10-16T11:19:37.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f0697fff700 1 -- 172.21.15.181:0/215210000 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0698014420 con 0x7f06a015db70 2023-10-16T11:19:37.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a015f130 con 0x7f06a015db70 2023-10-16T11:19:37.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f0697fff700 1 -- 172.21.15.181:0/215210000 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06980147b0 con 0x7f06a015db70 2023-10-16T11:19:37.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.508+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a015f6f0 con 0x7f06a015db70 2023-10-16T11:19:37.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.509+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a0067d20 con 0x7f06a015db70 2023-10-16T11:19:37.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.509+0000 7f0697fff700 1 -- 172.21.15.181:0/215210000 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0698014910 con 0x7f06a015db70 2023-10-16T11:19:37.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.509+0000 7f0697fff700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.510+0000 7f0697fff700 1 -- 172.21.15.181:0/215210000 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f069808a420 con 0x7f06a015db70 2023-10-16T11:19:37.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.510+0000 7f069f7fe700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.510+0000 7f069f7fe700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0690005910 tx=0x7f0690009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.514+0000 7f0697fff700 1 -- 172.21.15.181:0/215210000 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0698054ec0 con 0x7f06a015db70 2023-10-16T11:19:37.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.710+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06a0072510 con 0x7f06a015db70 2023-10-16T11:19:37.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.712+0000 7f0697fff700 1 -- 172.21.15.181:0/215210000 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+21194 (secure 0 0 0) 0x7f06980584e0 con 0x7f06a015db70 2023-10-16T11:19:37.714 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:37.715 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":220,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:37.612689+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:37.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 msgr2=0x7f068006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0690005910 tx=0x7f0690009040 comp rx=0 tx=0).stop 2023-10-16T11:19:37.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a015db70 msgr2=0x7f06a009f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a015db70 0x7f06a009f670 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f0698007e40 tx=0x7f069800e3c0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 shutdown_connections 2023-10-16T11:19:37.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f068006b020 0x7f068006d4e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06a00ca2e0 0x7f06a00c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a015db70 0x7f06a009f670 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 --2- 172.21.15.181:0/215210000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06a00cf4e0 0x7f06a00c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 >> 172.21.15.181:0/215210000 conn(0x7f06a00b1780 msgr2=0x7f06a00c3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:37.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 shutdown_connections 2023-10-16T11:19:37.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.714+0000 7f06a5bd5700 1 -- 172.21.15.181:0/215210000 wait complete. 2023-10-16T11:19:37.729 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-16T11:19:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.850+0000 7fc725ca5700 1 Processor -- start 2023-10-16T11:19:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.850+0000 7fc725ca5700 1 -- start start 2023-10-16T11:19:37.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc725ca5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc72015da70 0x7fc7200ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc725ca5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7200ce6a0 0x7fc72006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc725ca5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7200cef00 con 0x7fc720072760 2023-10-16T11:19:37.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc725ca5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc720072b40 con 0x7fc7200ce6a0 2023-10-16T11:19:37.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc725ca5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7200cec20 con 0x7fc72015da70 2023-10-16T11:19:37.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc71effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7200ce6a0 0x7fc72006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc71ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc720072760 0x7fc7201615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38624/0 (socket says 172.21.15.181:38624) 2023-10-16T11:19:37.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.851+0000 7fc71ffff700 1 -- 172.21.15.181:0/1775443459 learned_addr learned my addr 172.21.15.181:0/1775443459 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:37.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71effd700 1 -- 172.21.15.181:0/1775443459 >> v2:172.21.15.190:3300/0 conn(0x7fc72015da70 msgr2=0x7fc7200ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:37.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71f7fe700 1 --2- 172.21.15.181:0/1775443459 >> v2:172.21.15.190:3300/0 conn(0x7fc72015da70 0x7fc7200ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71e7fc700 1 -- 172.21.15.181:0/1775443459 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790496479 0 0) 0x7fc7200cef00 con 0x7fc720072760 2023-10-16T11:19:37.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71effd700 1 --2- 172.21.15.181:0/1775443459 >> v2:172.21.15.190:3300/0 conn(0x7fc72015da70 0x7fc7200ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71effd700 1 -- 172.21.15.181:0/1775443459 >> v1:172.21.15.181:6789/0 conn(0x7fc720072760 legacy=0x7fc7201615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71effd700 1 -- 172.21.15.181:0/1775443459 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720071c80 con 0x7fc7200ce6a0 2023-10-16T11:19:37.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71f7fe700 1 --2- 172.21.15.181:0/1775443459 >> v2:172.21.15.190:3300/0 conn(0x7fc72015da70 0x7fc7200ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:37.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71effd700 1 --2- 172.21.15.181:0/1775443459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7200ce6a0 0x7fc72006a8a0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fc7200675e0 tx=0x7fc710009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73049a581a5163a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71e7fc700 1 -- 172.21.15.181:0/1775443459 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710011830 con 0x7fc7200ce6a0 2023-10-16T11:19:37.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71e7fc700 1 -- 172.21.15.181:0/1775443459 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc710011990 con 0x7fc7200ce6a0 2023-10-16T11:19:37.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.852+0000 7fc71e7fc700 1 -- 172.21.15.181:0/1775443459 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710011d20 con 0x7fc7200ce6a0 2023-10-16T11:19:37.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 -- 172.21.15.181:0/1775443459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7200ce6a0 msgr2=0x7fc72006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 --2- 172.21.15.181:0/1775443459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7200ce6a0 0x7fc72006a8a0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fc7200675e0 tx=0x7fc710009b20 comp rx=0 tx=0).stop 2023-10-16T11:19:37.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 -- 172.21.15.181:0/1775443459 shutdown_connections 2023-10-16T11:19:37.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 --2- 172.21.15.181:0/1775443459 >> v2:172.21.15.190:3300/0 conn(0x7fc72015da70 0x7fc7200ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 --2- 172.21.15.181:0/1775443459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7200ce6a0 0x7fc72006a8a0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 -- 172.21.15.181:0/1775443459 >> 172.21.15.181:0/1775443459 conn(0x7fc7200cd010 msgr2=0x7fc7200cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:37.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 -- 172.21.15.181:0/1775443459 shutdown_connections 2023-10-16T11:19:37.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.853+0000 7fc725ca5700 1 -- 172.21.15.181:0/1775443459 wait complete. 2023-10-16T11:19:37.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.854+0000 7fc725ca5700 1 Processor -- start 2023-10-16T11:19:37.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.854+0000 7fc725ca5700 1 -- start start 2023-10-16T11:19:37.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc725ca5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc720072760 0x7fc7200b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc725ca5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7200ce6a0 0x7fc72015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc725ca5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc72015da70 0x7fc720159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc725ca5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720160ef0 con 0x7fc720072760 2023-10-16T11:19:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc725ca5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7201611d0 con 0x7fc7200ce6a0 2023-10-16T11:19:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc725ca5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200b5460 con 0x7fc72015da70 2023-10-16T11:19:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc720072760 0x7fc7200b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc720072760 0x7fc7200b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55780/0 (socket says 172.21.15.181:55780) 2023-10-16T11:19:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 -- 172.21.15.181:0/640561638 learned_addr learned my addr 172.21.15.181:0/640561638 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71ffff700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc72015da70 0x7fc720159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71effd700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7200ce6a0 0x7fc72015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 -- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc72015da70 msgr2=0x7fc720159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc72015da70 0x7fc720159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 -- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7200ce6a0 msgr2=0x7fc72015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:37.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7200ce6a0 0x7fc72015bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:37.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 -- 172.21.15.181:0/640561638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100094a0 con 0x7fc720072760 2023-10-16T11:19:37.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71effd700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7200ce6a0 0x7fc72015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:37.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc71f7fe700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc720072760 0x7fc7200b5c40 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c750 tx=0x7fc71400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc707fff700 1 -- 172.21.15.181:0/640561638 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714011800 con 0x7fc720072760 2023-10-16T11:19:37.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.855+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7200b5740 con 0x7fc720072760 2023-10-16T11:19:37.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.856+0000 7fc707fff700 1 -- 172.21.15.181:0/640561638 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc714011960 con 0x7fc720072760 2023-10-16T11:19:37.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.856+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc720159c40 con 0x7fc720072760 2023-10-16T11:19:37.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.856+0000 7fc707fff700 1 -- 172.21.15.181:0/640561638 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714011cd0 con 0x7fc720072760 2023-10-16T11:19:37.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.857+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7000052f0 con 0x7fc720072760 2023-10-16T11:19:37.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.857+0000 7fc707fff700 1 -- 172.21.15.181:0/640561638 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc7140193f0 con 0x7fc720072760 2023-10-16T11:19:37.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.857+0000 7fc707fff700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc70c06b020 0x7fc70c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:37.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.857+0000 7fc707fff700 1 -- 172.21.15.181:0/640561638 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fc71408a0a0 con 0x7fc720072760 2023-10-16T11:19:37.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.857+0000 7fc71effd700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc70c06b020 0x7fc70c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:37.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.858+0000 7fc71effd700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc70c06b020 0x7fc70c06d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc720160890 tx=0x7fc710008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:37.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:37.862+0000 7fc707fff700 1 -- 172.21.15.181:0/640561638 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7140549b0 con 0x7fc720072760 2023-10-16T11:19:38.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:38.102+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc700005640 con 0x7fc720072760 2023-10-16T11:19:39.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.022+0000 7fc707fff700 1 -- 172.21.15.181:0/640561638 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v116) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc714057fd0 con 0x7fc720072760 2023-10-16T11:19:39.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc70c06b020 msgr2=0x7fc70c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:39.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc70c06b020 0x7fc70c06d4e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc720160890 tx=0x7fc710008030 comp rx=0 tx=0).stop 2023-10-16T11:19:39.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc720072760 msgr2=0x7fc7200b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:39.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc720072760 0x7fc7200b5c40 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c750 tx=0x7fc71400cb10 comp rx=0 tx=0).stop 2023-10-16T11:19:39.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 shutdown_connections 2023-10-16T11:19:39.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fc70c06b020 0x7fc70c06d4e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc72015da70 0x7fc720159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc720072760 0x7fc7200b5c40 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 --2- 172.21.15.181:0/640561638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7200ce6a0 0x7fc72015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.025+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 >> 172.21.15.181:0/640561638 conn(0x7fc7200cd010 msgr2=0x7fc7200cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:39.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.026+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 shutdown_connections 2023-10-16T11:19:39.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.026+0000 7fc725ca5700 1 -- 172.21.15.181:0/640561638 wait complete. 2023-10-16T11:19:39.042 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:19:39.043 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-16T11:19:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.170+0000 7f80f4cc4700 1 Processor -- start 2023-10-16T11:19:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.170+0000 7f80f4cc4700 1 -- start start 2023-10-16T11:19:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.171+0000 7f80f4cc4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80f00b2010 0x7f80f00cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.171+0000 7f80f4cc4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80f0071be0 0x7f80f00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.171+0000 7f80f4cc4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f00ce1f0 con 0x7f80f006f3b0 2023-10-16T11:19:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.171+0000 7f80f4cc4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f0072090 con 0x7f80f0071be0 2023-10-16T11:19:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.171+0000 7f80f4cc4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f00b2930 con 0x7f80f00b2010 2023-10-16T11:19:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.171+0000 7f80edd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80f0071be0 0x7f80f00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80edd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80f0071be0 0x7f80f00b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55788/0 (socket says 172.21.15.181:55788) 2023-10-16T11:19:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80edd9b700 1 -- 172.21.15.181:0/235464593 learned_addr learned my addr 172.21.15.181:0/235464593 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80ee59c700 1 --2- 172.21.15.181:0/235464593 >> v2:172.21.15.190:3300/0 conn(0x7f80f00b2010 0x7f80f00cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80edd9b700 1 -- 172.21.15.181:0/235464593 >> v2:172.21.15.190:3300/0 conn(0x7f80f00b2010 msgr2=0x7f80f00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80edd9b700 1 --2- 172.21.15.181:0/235464593 >> v2:172.21.15.190:3300/0 conn(0x7f80f00b2010 0x7f80f00cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80edd9b700 1 -- 172.21.15.181:0/235464593 >> v1:172.21.15.190:6789/0 conn(0x7f80f006f3b0 legacy=0x7f80f006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:39.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80edd9b700 1 -- 172.21.15.181:0/235464593 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00ccda0 con 0x7f80f0071be0 2023-10-16T11:19:39.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80edd9b700 1 --2- 172.21.15.181:0/235464593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f0071be0 0x7f80f00b23f0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f80f0070430 tx=0x7f80e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a066698cc6e0e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:39.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80ed59a700 1 -- 172.21.15.181:0/235464593 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e000b870 con 0x7f80f0071be0 2023-10-16T11:19:39.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80ed59a700 1 -- 172.21.15.181:0/235464593 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e000b9d0 con 0x7f80f0071be0 2023-10-16T11:19:39.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.172+0000 7f80ed59a700 1 -- 172.21.15.181:0/235464593 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e000bd60 con 0x7f80f0071be0 2023-10-16T11:19:39.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 -- 172.21.15.181:0/235464593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f0071be0 msgr2=0x7f80f00b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:39.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/235464593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f0071be0 0x7f80f00b23f0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f80f0070430 tx=0x7f80e0009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 -- 172.21.15.181:0/235464593 shutdown_connections 2023-10-16T11:19:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/235464593 >> v2:172.21.15.190:3300/0 conn(0x7f80f00b2010 0x7f80f00cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/235464593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f0071be0 0x7f80f00b23f0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 -- 172.21.15.181:0/235464593 >> 172.21.15.181:0/235464593 conn(0x7f80f00cd010 msgr2=0x7f80f00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 -- 172.21.15.181:0/235464593 shutdown_connections 2023-10-16T11:19:39.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.173+0000 7f80f4cc4700 1 -- 172.21.15.181:0/235464593 wait complete. 2023-10-16T11:19:39.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.174+0000 7f80f4cc4700 1 Processor -- start 2023-10-16T11:19:39.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.174+0000 7f80f4cc4700 1 -- start start 2023-10-16T11:19:39.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80f4cc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f006f3b0 0x7f80f01507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:39.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80f4cc4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80f0071be0 0x7f80f01e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:39.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80f4cc4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80f00b2010 0x7f80f01e7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:39.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80f4cc4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0150d20 con 0x7f80f006f3b0 2023-10-16T11:19:39.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80f4cc4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00afbe0 con 0x7f80f00b2010 2023-10-16T11:19:39.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80f4cc4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00afec0 con 0x7f80f0071be0 2023-10-16T11:19:39.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f006f3b0 0x7f80f01507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:39.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f006f3b0 0x7f80f01507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55794/0 (socket says 172.21.15.181:55794) 2023-10-16T11:19:39.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 -- 172.21.15.181:0/263472808 learned_addr learned my addr 172.21.15.181:0/263472808 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:39.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80edd9b700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80f0071be0 0x7f80f01e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:39.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80eed9d700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80f00b2010 0x7f80f01e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:39.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 -- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80f0071be0 msgr2=0x7f80f01e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:39.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80f0071be0 0x7f80f01e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 -- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80f00b2010 msgr2=0x7f80f01e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:39.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80f00b2010 0x7f80f01e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:39.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 -- 172.21.15.181:0/263472808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0009550 con 0x7f80f006f3b0 2023-10-16T11:19:39.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80edd9b700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80f0071be0 0x7f80f01e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:39.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80eed9d700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80f00b2010 0x7f80f01e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:39.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80ee59c700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f006f3b0 0x7f80f01507e0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f80e400e810 tx=0x7f80e400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:39.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.175+0000 7f80deffd700 1 -- 172.21.15.181:0/263472808 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4013610 con 0x7f80f006f3b0 2023-10-16T11:19:39.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.176+0000 7f80deffd700 1 -- 172.21.15.181:0/263472808 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e4013770 con 0x7f80f006f3b0 2023-10-16T11:19:39.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.176+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f00b01a0 con 0x7f80f006f3b0 2023-10-16T11:19:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.176+0000 7f80deffd700 1 -- 172.21.15.181:0/263472808 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4013b20 con 0x7f80f006f3b0 2023-10-16T11:19:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.176+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f00b0760 con 0x7f80f006f3b0 2023-10-16T11:19:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.176+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f00b6510 con 0x7f80f006f3b0 2023-10-16T11:19:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.177+0000 7f80deffd700 1 -- 172.21.15.181:0/263472808 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f80e4013c80 con 0x7f80f006f3b0 2023-10-16T11:19:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.177+0000 7f80deffd700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d006b020 0x7f80d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:39.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.177+0000 7f80deffd700 1 -- 172.21.15.181:0/263472808 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f80e4089ac0 con 0x7f80f006f3b0 2023-10-16T11:19:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.177+0000 7f80edd9b700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d006b020 0x7f80d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.178+0000 7f80edd9b700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d006b020 0x7f80d006d4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80f00cf5f0 tx=0x7f80e0008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:39.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.182+0000 7f80deffd700 1 -- 172.21.15.181:0/263472808 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f80e40543d0 con 0x7f80f006f3b0 2023-10-16T11:19:39.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:39.417+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f80f0074db0 con 0x7f80f006f3b0 2023-10-16T11:19:39.509 INFO:tasks.ceph.mds.c.smithi181.stderr:2023-10-16T11:19:39.507+0000 7f7fd22fc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:19:40.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.026+0000 7f80deffd700 1 -- 172.21.15.181:0/263472808 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v117) v1 ==== 125+0+0 (secure 0 0 0) 0x7f80e40579f0 con 0x7f80f006f3b0 2023-10-16T11:19:40.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.028+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d006b020 msgr2=0x7f80d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d006b020 0x7f80d006d4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80f00cf5f0 tx=0x7f80e0008030 comp rx=0 tx=0).stop 2023-10-16T11:19:40.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f006f3b0 msgr2=0x7f80f01507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f006f3b0 0x7f80f01507e0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f80e400e810 tx=0x7f80e400ebd0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 shutdown_connections 2023-10-16T11:19:40.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f80d006b020 0x7f80d006d4e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80f00b2010 0x7f80f01e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f006f3b0 0x7f80f01507e0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 --2- 172.21.15.181:0/263472808 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80f0071be0 0x7f80f01e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 >> 172.21.15.181:0/263472808 conn(0x7f80f00cd010 msgr2=0x7f80f00b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:40.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 shutdown_connections 2023-10-16T11:19:40.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.029+0000 7f80f4cc4700 1 -- 172.21.15.181:0/263472808 wait complete. 2023-10-16T11:19:40.043 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-16T11:19:40.044 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:40.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.162+0000 7f8e67ef3700 1 Processor -- start 2023-10-16T11:19:40.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.162+0000 7f8e67ef3700 1 -- start start 2023-10-16T11:19:40.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.163+0000 7f8e67ef3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e600c26b0 0x7f8e600c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.163+0000 7f8e67ef3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e600b2710 con 0x7f8e6015dc40 2023-10-16T11:19:40.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.163+0000 7f8e67ef3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e600c18f0 con 0x7f8e6015e020 2023-10-16T11:19:40.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.163+0000 7f8e67ef3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6015ebc0 con 0x7f8e600c26b0 2023-10-16T11:19:40.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.163+0000 7f8e6548e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e600c26b0 0x7f8e600c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.163+0000 7f8e6548e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e600c26b0 0x7f8e600c2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55804/0 (socket says 172.21.15.181:55804) 2023-10-16T11:19:40.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e6548e700 1 -- 172.21.15.181:0/1153735607 learned_addr learned my addr 172.21.15.181:0/1153735607 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:40.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.163+0000 7f8e65c8f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8e6015dc40 0x7f8e6006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38648/0 (socket says 172.21.15.181:38648) 2023-10-16T11:19:40.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e6548e700 1 -- 172.21.15.181:0/1153735607 >> v1:172.21.15.190:6789/0 conn(0x7f8e6015e020 legacy=0x7f8e6015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:40.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e6548e700 1 -- 172.21.15.181:0/1153735607 >> v1:172.21.15.181:6789/0 conn(0x7f8e6015dc40 legacy=0x7f8e6006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e6548e700 1 -- 172.21.15.181:0/1153735607 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600c1bd0 con 0x7f8e600c26b0 2023-10-16T11:19:40.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e64c8d700 1 -- 172.21.15.181:0/1153735607 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507218244 0 0) 0x7f8e50001110 con 0x7f8e6015dc40 2023-10-16T11:19:40.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e6548e700 1 --2- 172.21.15.181:0/1153735607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c2b10 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0098f0 tx=0x7f8e5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd98862c5ac364a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e64c8d700 1 -- 172.21.15.181:0/1153735607 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c004230 con 0x7f8e600c26b0 2023-10-16T11:19:40.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e64c8d700 1 -- 172.21.15.181:0/1153735607 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c004390 con 0x7f8e600c26b0 2023-10-16T11:19:40.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.164+0000 7f8e64c8d700 1 -- 172.21.15.181:0/1153735607 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0036a0 con 0x7f8e600c26b0 2023-10-16T11:19:40.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.165+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1153735607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 msgr2=0x7f8e600c2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.165+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1153735607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c2b10 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0098f0 tx=0x7f8e5c009c00 comp rx=0 tx=0).stop 2023-10-16T11:19:40.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.165+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1153735607 shutdown_connections 2023-10-16T11:19:40.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.165+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1153735607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c2b10 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.165+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1153735607 >> 172.21.15.181:0/1153735607 conn(0x7f8e600b1780 msgr2=0x7f8e600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:40.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.165+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1153735607 shutdown_connections 2023-10-16T11:19:40.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.165+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1153735607 wait complete. 2023-10-16T11:19:40.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.166+0000 7f8e67ef3700 1 Processor -- start 2023-10-16T11:19:40.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.166+0000 7f8e67ef3700 1 -- start start 2023-10-16T11:19:40.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6015dc40 0x7f8e600cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6015e020 0x7f8e60153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e601e3a40 con 0x7f8e600c26b0 2023-10-16T11:19:40.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6006dce0 con 0x7f8e6015e020 2023-10-16T11:19:40.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e60155c70 con 0x7f8e6015dc40 2023-10-16T11:19:40.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55808/0 (socket says 172.21.15.181:55808) 2023-10-16T11:19:40.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 -- 172.21.15.181:0/1644714741 learned_addr learned my addr 172.21.15.181:0/1644714741 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:40.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 -- 172.21.15.181:0/1644714741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6015dc40 msgr2=0x7f8e600cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6015dc40 0x7f8e600cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 -- 172.21.15.181:0/1644714741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6015e020 msgr2=0x7f8e60153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6015e020 0x7f8e60153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 -- 172.21.15.181:0/1644714741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c009550 con 0x7f8e600c26b0 2023-10-16T11:19:40.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e65c8f700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c83b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f8e50001d40 tx=0x7f8e5000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e567fc700 1 -- 172.21.15.181:0/1644714741 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50004c80 con 0x7f8e600c26b0 2023-10-16T11:19:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e567fc700 1 -- 172.21.15.181:0/1644714741 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e50004e00 con 0x7f8e600c26b0 2023-10-16T11:19:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e567fc700 1 -- 172.21.15.181:0/1644714741 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50007310 con 0x7f8e600c26b0 2023-10-16T11:19:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6006cdf0 con 0x7f8e600c26b0 2023-10-16T11:19:40.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.167+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6006d3b0 con 0x7f8e600c26b0 2023-10-16T11:19:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.168+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e60067d20 con 0x7f8e600c26b0 2023-10-16T11:19:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.169+0000 7f8e567fc700 1 -- 172.21.15.181:0/1644714741 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8e50007470 con 0x7f8e600c26b0 2023-10-16T11:19:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.169+0000 7f8e567fc700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8e4406b020 0x7f8e4406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.169+0000 7f8e567fc700 1 -- 172.21.15.181:0/1644714741 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f8e5008b7e0 con 0x7f8e600c26b0 2023-10-16T11:19:40.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.169+0000 7f8e6548e700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8e4406b020 0x7f8e4406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.169+0000 7f8e6548e700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8e4406b020 0x7f8e4406d4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e600c7de0 tx=0x7f8e5c010000 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.174+0000 7f8e567fc700 1 -- 172.21.15.181:0/1644714741 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8e50056160 con 0x7f8e600c26b0 2023-10-16T11:19:40.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.410+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e600b2710 con 0x7f8e600c26b0 2023-10-16T11:19:40.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.411+0000 7f8e567fc700 1 -- 172.21.15.181:0/1644714741 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+6165 (secure 0 0 0) 0x7f8e600b2710 con 0x7f8e600c26b0 2023-10-16T11:19:40.413 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:40.413 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:19:40.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8e4406b020 msgr2=0x7f8e4406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8e4406b020 0x7f8e4406d4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e600c7de0 tx=0x7f8e5c010000 comp rx=0 tx=0).stop 2023-10-16T11:19:40.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 msgr2=0x7f8e600c83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c83b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f8e50001d40 tx=0x7f8e5000c9e0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 shutdown_connections 2023-10-16T11:19:40.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8e4406b020 0x7f8e4406d4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e6015e020 0x7f8e60153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e600c26b0 0x7f8e600c83b0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 --2- 172.21.15.181:0/1644714741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e6015dc40 0x7f8e600cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 >> 172.21.15.181:0/1644714741 conn(0x7f8e600b1780 msgr2=0x7f8e6015a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:40.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 shutdown_connections 2023-10-16T11:19:40.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.413+0000 7f8e67ef3700 1 -- 172.21.15.181:0/1644714741 wait complete. 2023-10-16T11:19:40.418 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 117 2023-10-16T11:19:40.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:40.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.546+0000 7f24b65fb700 1 Processor -- start 2023-10-16T11:19:40.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.547+0000 7f24b65fb700 1 -- start start 2023-10-16T11:19:40.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.547+0000 7f24b65fb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 0x7f24b00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.547+0000 7f24b65fb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b006ed00 con 0x7f24b015eab0 2023-10-16T11:19:40.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.547+0000 7f24b65fb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b00d02d0 con 0x7f24b015e6d0 2023-10-16T11:19:40.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.547+0000 7f24b65fb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b006e0f0 con 0x7f24b00cb0c0 2023-10-16T11:19:40.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.547+0000 7f24af7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f24b015eab0 0x7f24b015a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38664/0 (socket says 172.21.15.181:38664) 2023-10-16T11:19:40.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.547+0000 7f24af7fe700 1 -- 172.21.15.181:0/3245383700 learned_addr learned my addr 172.21.15.181:0/3245383700 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:40.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24b4b98700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 0x7f24b00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521394189 0 0) 0x7f24b006ed00 con 0x7f24b015eab0 2023-10-16T11:19:40.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24980035b0 con 0x7f24b015eab0 2023-10-16T11:19:40.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689515421 0 0) 0x7f24b00d02d0 con 0x7f24b015e6d0 2023-10-16T11:19:40.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b006ed00 con 0x7f24b015e6d0 2023-10-16T11:19:40.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251317603 0 0) 0x7f24980035b0 con 0x7f24b015eab0 2023-10-16T11:19:40.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 msgr2=0x7f24b00d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:40.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 0x7f24b00d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 >> v1:172.21.15.190:6789/0 conn(0x7f24b015e6d0 legacy=0x7f24b015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b006e3d0 con 0x7f24b015eab0 2023-10-16T11:19:40.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f24a4001670 con 0x7f24b015eab0 2023-10-16T11:19:40.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 >> v1:172.21.15.181:6789/0 conn(0x7f24b015eab0 legacy=0x7f24b015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f2498003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2498004850 0x7f2498004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 0x7f2498005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b006e0f0 con 0x7f2498003bc0 2023-10-16T11:19:40.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249800baf0 con 0x7f2498004850 2023-10-16T11:19:40.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249800bcb0 con 0x7f24b00cb0c0 2023-10-16T11:19:40.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24affff700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f2498003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24b4b98700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 0x7f2498005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24af7fe700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2498004850 0x7f2498004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24affff700 1 -- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 msgr2=0x7f2498005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:40.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24affff700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b00cb0c0 0x7f2498005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24affff700 1 -- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2498004850 msgr2=0x7f2498004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:40.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24affff700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2498004850 0x7f2498004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24affff700 1 -- 172.21.15.181:0/3245383700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980035b0 con 0x7f2498003bc0 2023-10-16T11:19:40.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.548+0000 7f24af7fe700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2498004850 0x7f2498004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:40.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24affff700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f2498003fc0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f24a00019a0 tx=0x7f2498005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d3e74f28a10226 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0001dd0 con 0x7f2498003bc0 2023-10-16T11:19:40.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a00052d0 con 0x7f2498003bc0 2023-10-16T11:19:40.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24aeffd700 1 -- 172.21.15.181:0/3245383700 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0001460 con 0x7f2498003bc0 2023-10-16T11:19:40.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24b65fb700 1 -- 172.21.15.181:0/3245383700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 msgr2=0x7f2498003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24b65fb700 1 --2- 172.21.15.181:0/3245383700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f2498003fc0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f24a00019a0 tx=0x7f2498005880 comp rx=0 tx=0).stop 2023-10-16T11:19:40.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24b4b98700 1 -- 172.21.15.181:0/3245383700 reap_dead start 2023-10-16T11:19:40.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24b65fb700 1 -- 172.21.15.181:0/3245383700 shutdown_connections 2023-10-16T11:19:40.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24b65fb700 1 -- 172.21.15.181:0/3245383700 >> 172.21.15.181:0/3245383700 conn(0x7f24b00d04a0 msgr2=0x7f24b00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:40.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24b65fb700 1 -- 172.21.15.181:0/3245383700 shutdown_connections 2023-10-16T11:19:40.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.549+0000 7f24b65fb700 1 -- 172.21.15.181:0/3245383700 wait complete. 2023-10-16T11:19:40.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.550+0000 7f24b65fb700 1 Processor -- start 2023-10-16T11:19:40.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.551+0000 7f24b65fb700 1 -- start start 2023-10-16T11:19:40.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.551+0000 7f24b65fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f24b0156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.551+0000 7f24b65fb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b00af960 0x7f24b00c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.551+0000 7f24b65fb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b015d170 0x7f24b015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.551+0000 7f24b65fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b015db40 con 0x7f2498003bc0 2023-10-16T11:19:40.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.551+0000 7f24b65fb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00ca7a0 con 0x7f24b00af960 2023-10-16T11:19:40.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.551+0000 7f24b65fb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00afe60 con 0x7f24b015d170 2023-10-16T11:19:40.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f24b0156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f24b0156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55828/0 (socket says 172.21.15.181:55828) 2023-10-16T11:19:40.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 -- 172.21.15.181:0/1517271389 learned_addr learned my addr 172.21.15.181:0/1517271389 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:40.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24b4b98700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b015d170 0x7f24b015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24af7fe700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b00af960 0x7f24b00c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 -- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b015d170 msgr2=0x7f24b015d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b015d170 0x7f24b015d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 -- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b00af960 msgr2=0x7f24b00c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b00af960 0x7f24b00c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 -- 172.21.15.181:0/1517271389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00b0140 con 0x7f2498003bc0 2023-10-16T11:19:40.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24affff700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f24b0156550 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f24a0004000 tx=0x7f24a0001320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24acff9700 1 -- 172.21.15.181:0/1517271389 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00050d0 con 0x7f2498003bc0 2023-10-16T11:19:40.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24acff9700 1 -- 172.21.15.181:0/1517271389 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a00013c0 con 0x7f2498003bc0 2023-10-16T11:19:40.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b00b0420 con 0x7f2498003bc0 2023-10-16T11:19:40.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24acff9700 1 -- 172.21.15.181:0/1517271389 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0007280 con 0x7f2498003bc0 2023-10-16T11:19:40.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.552+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b00b09e0 con 0x7f2498003bc0 2023-10-16T11:19:40.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.553+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b0067d20 con 0x7f2498003bc0 2023-10-16T11:19:40.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.554+0000 7f24acff9700 1 -- 172.21.15.181:0/1517271389 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f24a00073e0 con 0x7f2498003bc0 2023-10-16T11:19:40.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.554+0000 7f24acff9700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f249c06b020 0x7f249c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.554+0000 7f24acff9700 1 -- 172.21.15.181:0/1517271389 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f24a008a690 con 0x7f2498003bc0 2023-10-16T11:19:40.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.554+0000 7f24af7fe700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f249c06b020 0x7f249c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.554+0000 7f24af7fe700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f249c06b020 0x7f249c06d4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24b00c3ce0 tx=0x7f24a4007040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.558+0000 7f24acff9700 1 -- 172.21.15.181:0/1517271389 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f24a00551e0 con 0x7f2498003bc0 2023-10-16T11:19:40.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.751+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24b00cb610 con 0x7f2498003bc0 2023-10-16T11:19:40.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.753+0000 7f24acff9700 1 -- 172.21.15.181:0/1517271389 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+21194 (secure 0 0 0) 0x7f24a0058800 con 0x7f2498003bc0 2023-10-16T11:19:40.755 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:40.755 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":220,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:37.612689+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:40.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f249c06b020 msgr2=0x7f249c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f249c06b020 0x7f249c06d4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24b00c3ce0 tx=0x7f24a4007040 comp rx=0 tx=0).stop 2023-10-16T11:19:40.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 msgr2=0x7f24b0156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f24b0156550 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f24a0004000 tx=0x7f24a0001320 comp rx=0 tx=0).stop 2023-10-16T11:19:40.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 shutdown_connections 2023-10-16T11:19:40.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f249c06b020 0x7f249c06d4e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b015d170 0x7f24b015d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2498003bc0 0x7f24b0156550 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 --2- 172.21.15.181:0/1517271389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b00af960 0x7f24b00c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 >> 172.21.15.181:0/1517271389 conn(0x7f24b00d04a0 msgr2=0x7f24b00aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:40.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 shutdown_connections 2023-10-16T11:19:40.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.755+0000 7f24b65fb700 1 -- 172.21.15.181:0/1517271389 wait complete. 2023-10-16T11:19:40.769 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:19:40.770 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:19:40.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.888+0000 7f8a7b0ad700 1 Processor -- start 2023-10-16T11:19:40.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.888+0000 7f8a7b0ad700 1 -- start start 2023-10-16T11:19:40.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a7b0ad700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8a740cb0c0 0x7f8a740d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a7b0ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7406ed00 con 0x7f8a7415eab0 2023-10-16T11:19:40.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a7b0ad700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740d02d0 con 0x7f8a7415e6d0 2023-10-16T11:19:40.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a7b0ad700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7406e0f0 con 0x7f8a740cb0c0 2023-10-16T11:19:40.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a73fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8a7415eab0 0x7f8a7415a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38684/0 (socket says 172.21.15.181:38684) 2023-10-16T11:19:40.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a73fff700 1 -- 172.21.15.181:0/2455954901 learned_addr learned my addr 172.21.15.181:0/2455954901 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:40.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a7964a700 1 --2- 172.21.15.181:0/2455954901 >> v2:172.21.15.190:3300/0 conn(0x7f8a740cb0c0 0x7f8a740d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855360996 0 0) 0x7f8a7406ed00 con 0x7f8a7415eab0 2023-10-16T11:19:40.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.889+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a640035b0 con 0x7f8a7415eab0 2023-10-16T11:19:40.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965445182 0 0) 0x7f8a640035b0 con 0x7f8a7415eab0 2023-10-16T11:19:40.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 >> v2:172.21.15.190:3300/0 conn(0x7f8a740cb0c0 msgr2=0x7f8a740d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:40.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 --2- 172.21.15.181:0/2455954901 >> v2:172.21.15.190:3300/0 conn(0x7f8a740cb0c0 0x7f8a740d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 >> v1:172.21.15.190:6789/0 conn(0x7f8a7415e6d0 legacy=0x7f8a7415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7406e3d0 con 0x7f8a7415eab0 2023-10-16T11:19:40.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f8a60001670 con 0x7f8a7415eab0 2023-10-16T11:19:40.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 >> v1:172.21.15.181:6789/0 conn(0x7f8a7415eab0 legacy=0x7f8a7415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a64003a00 0x7f8a64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a64004400 0x7f8a640048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a640050d0 0x7f8a6400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7406e0f0 con 0x7f8a64004400 2023-10-16T11:19:40.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6400c030 con 0x7f8a640050d0 2023-10-16T11:19:40.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6400c1f0 con 0x7f8a64003a00 2023-10-16T11:19:40.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a73fff700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a64004400 0x7f8a640048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a78e49700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a64003a00 0x7f8a64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a7964a700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a640050d0 0x7f8a6400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a73fff700 1 -- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a64003a00 msgr2=0x7f8a64003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:40.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a73fff700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a64003a00 0x7f8a64003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a73fff700 1 -- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a640050d0 msgr2=0x7f8a6400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a73fff700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a640050d0 0x7f8a6400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a7964a700 1 -- 172.21.15.181:0/2455954901 reap_dead start 2023-10-16T11:19:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a73fff700 1 -- 172.21.15.181:0/2455954901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640035b0 con 0x7f8a64004400 2023-10-16T11:19:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.890+0000 7f8a73fff700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a64004400 0x7f8a640048d0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c2f0 tx=0x7f8a64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62680ae3afcf3ea server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000cdb0 con 0x7f8a64004400 2023-10-16T11:19:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60005960 con 0x7f8a64004400 2023-10-16T11:19:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a737fe700 1 -- 172.21.15.181:0/2455954901 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60005d10 con 0x7f8a64004400 2023-10-16T11:19:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/2455954901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a64004400 msgr2=0x7f8a640048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a64004400 0x7f8a640048d0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c2f0 tx=0x7f8a64005880 comp rx=0 tx=0).stop 2023-10-16T11:19:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/2455954901 shutdown_connections 2023-10-16T11:19:40.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/2455954901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a64004400 0x7f8a640048d0 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/2455954901 >> 172.21.15.181:0/2455954901 conn(0x7f8a740d04a0 msgr2=0x7f8a740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/2455954901 shutdown_connections 2023-10-16T11:19:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.891+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/2455954901 wait complete. 2023-10-16T11:19:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.892+0000 7f8a7b0ad700 1 Processor -- start 2023-10-16T11:19:40.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.892+0000 7f8a7b0ad700 1 -- start start 2023-10-16T11:19:40.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a7b0ad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a64004400 0x7f8a74149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a7b0ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740c9b00 0x7f8a740c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a7b0ad700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7406ae40 0x7f8a7406b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a7b0ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7406b800 con 0x7f8a740c9b00 2023-10-16T11:19:40.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a7b0ad700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740b58c0 con 0x7f8a64004400 2023-10-16T11:19:40.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a7b0ad700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740b49d0 con 0x7f8a7406ae40 2023-10-16T11:19:40.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740c9b00 0x7f8a740c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740c9b00 0x7f8a740c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55850/0 (socket says 172.21.15.181:55850) 2023-10-16T11:19:40.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.893+0000 7f8a73fff700 1 -- 172.21.15.181:0/3625170211 learned_addr learned my addr 172.21.15.181:0/3625170211 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:40.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a78e49700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a64004400 0x7f8a74149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a7964a700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7406ae40 0x7f8a7406b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a73fff700 1 -- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7406ae40 msgr2=0x7f8a7406b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a73fff700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7406ae40 0x7f8a7406b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a73fff700 1 -- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a64004400 msgr2=0x7f8a74149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:40.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a73fff700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a64004400 0x7f8a74149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:40.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a73fff700 1 -- 172.21.15.181:0/3625170211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740b4cb0 con 0x7f8a740c9b00 2023-10-16T11:19:40.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a7964a700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7406ae40 0x7f8a7406b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:40.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a78e49700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a64004400 0x7f8a74149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:40.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a73fff700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740c9b00 0x7f8a740c9f80 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c000 tx=0x7f8a60001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a717fa700 1 -- 172.21.15.181:0/3625170211 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60007c20 con 0x7f8a740c9b00 2023-10-16T11:19:40.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a717fa700 1 -- 172.21.15.181:0/3625170211 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60004870 con 0x7f8a740c9b00 2023-10-16T11:19:40.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a740b4f90 con 0x7f8a740c9b00 2023-10-16T11:19:40.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a717fa700 1 -- 172.21.15.181:0/3625170211 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60004be0 con 0x7f8a740c9b00 2023-10-16T11:19:40.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.894+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a740b5550 con 0x7f8a740c9b00 2023-10-16T11:19:40.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.895+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a740b5710 con 0x7f8a740c9b00 2023-10-16T11:19:40.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.895+0000 7f8a717fa700 1 -- 172.21.15.181:0/3625170211 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8a60004d40 con 0x7f8a740c9b00 2023-10-16T11:19:40.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.896+0000 7f8a717fa700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8a5806b010 0x7f8a5806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:40.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.896+0000 7f8a78e49700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8a5806b010 0x7f8a5806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:40.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.896+0000 7f8a717fa700 1 -- 172.21.15.181:0/3625170211 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f8a6008ad50 con 0x7f8a740c9b00 2023-10-16T11:19:40.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.896+0000 7f8a78e49700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8a5806b010 0x7f8a5806d4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000c00 tx=0x7f8a68009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:40.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:40.900+0000 7f8a717fa700 1 -- 172.21.15.181:0/3625170211 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a6008e030 con 0x7f8a740c9b00 2023-10-16T11:19:41.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.090+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a7406e990 con 0x7f8a5806b010 2023-10-16T11:19:41.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.103+0000 7f8a717fa700 1 -- 172.21.15.181:0/3625170211 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+516853 (secure 0 0 0) 0x7f8a680013a0 con 0x7f8a5806b010 2023-10-16T11:19:41.111 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:41.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8a5806b010 msgr2=0x7f8a5806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8a5806b010 0x7f8a5806d4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000c00 tx=0x7f8a68009040 comp rx=0 tx=0).stop 2023-10-16T11:19:41.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740c9b00 msgr2=0x7f8a740c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740c9b00 0x7f8a740c9f80 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c000 tx=0x7f8a60001a00 comp rx=0 tx=0).stop 2023-10-16T11:19:41.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 shutdown_connections 2023-10-16T11:19:41.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8a5806b010 0x7f8a5806d4d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a7406ae40 0x7f8a7406b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a64004400 0x7f8a74149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 --2- 172.21.15.181:0/3625170211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a740c9b00 0x7f8a740c9f80 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 >> 172.21.15.181:0/3625170211 conn(0x7f8a740d04a0 msgr2=0x7f8a740cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:41.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 shutdown_connections 2023-10-16T11:19:41.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.107+0000 7f8a7b0ad700 1 -- 172.21.15.181:0/3625170211 wait complete. 2023-10-16T11:19:41.118 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:19:41.121 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-10-16T11:19:39.484465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594025,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":626,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":389,"ondisk_log_size":389,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421800,"kb_used_data":22472,"kb_used_omap":12,"kb_used_meta":399219,"kb_avail":754552920,"statfs":{"total":773094113280,"available":772662190080,"internally_reserved":0,"allocated":23011328,"data_stored":12349546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":408800824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.883080"},"pg_stats":[{"pgid":"1.0","version":"206'348","reported_seq":1210,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540056+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:19:24.540056+0000","last_peered":"2023-10-16T11:19:24.540056+0000","last_clean":"2023-10-16T11:19:24.540056+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:19:24.540056+0000","last_undegraded":"2023-10-16T11:19:24.540056+0000","last_fullsized":"2023-10-16T11:19:24.540056+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646510+0000","last_change":"2023-10-16T11:19:22.536415+0000","last_active":"2023-10-16T11:19:26.646510+0000","last_peered":"2023-10-16T11:19:26.646510+0000","last_clean":"2023-10-16T11:19:26.646510+0000","last_became_active":"2023-10-16T11:19:22.536117+0000","last_became_peered":"2023-10-16T11:19:22.536117+0000","last_unstale":"2023-10-16T11:19:26.646510+0000","last_undegraded":"2023-10-16T11:19:26.646510+0000","last_fullsized":"2023-10-16T11:19:26.646510+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:07.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645725+0000","last_change":"2023-10-16T11:19:23.551428+0000","last_active":"2023-10-16T11:19:26.645725+0000","last_peered":"2023-10-16T11:19:26.645725+0000","last_clean":"2023-10-16T11:19:26.645725+0000","last_became_active":"2023-10-16T11:19:23.551083+0000","last_became_peered":"2023-10-16T11:19:23.551083+0000","last_unstale":"2023-10-16T11:19:26.645725+0000","last_undegraded":"2023-10-16T11:19:26.645725+0000","last_fullsized":"2023-10-16T11:19:26.645725+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:52.800093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607616+0000","last_change":"2023-10-16T11:19:34.607616+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607616+0000","last_undegraded":"2023-10-16T11:19:34.607616+0000","last_fullsized":"2023-10-16T11:19:34.607616+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646184+0000","last_change":"2023-10-16T11:19:22.537241+0000","last_active":"2023-10-16T11:19:26.646184+0000","last_peered":"2023-10-16T11:19:26.646184+0000","last_clean":"2023-10-16T11:19:26.646184+0000","last_became_active":"2023-10-16T11:19:22.536947+0000","last_became_peered":"2023-10-16T11:19:22.536947+0000","last_unstale":"2023-10-16T11:19:26.646184+0000","last_undegraded":"2023-10-16T11:19:26.646184+0000","last_fullsized":"2023-10-16T11:19:26.646184+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478816+0000","last_change":"2023-10-16T11:19:23.563089+0000","last_active":"2023-10-16T11:19:25.478816+0000","last_peered":"2023-10-16T11:19:25.478816+0000","last_clean":"2023-10-16T11:19:25.478816+0000","last_became_active":"2023-10-16T11:19:23.562759+0000","last_became_peered":"2023-10-16T11:19:23.562759+0000","last_unstale":"2023-10-16T11:19:25.478816+0000","last_undegraded":"2023-10-16T11:19:25.478816+0000","last_fullsized":"2023-10-16T11:19:25.478816+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.602978+0000","last_change":"2023-10-16T11:19:34.602978+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.602978+0000","last_undegraded":"2023-10-16T11:19:34.602978+0000","last_fullsized":"2023-10-16T11:19:34.602978+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479067+0000","last_change":"2023-10-16T11:19:22.539000+0000","last_active":"2023-10-16T11:19:25.479067+0000","last_peered":"2023-10-16T11:19:25.479067+0000","last_clean":"2023-10-16T11:19:25.479067+0000","last_became_active":"2023-10-16T11:19:22.538728+0000","last_became_peered":"2023-10-16T11:19:22.538728+0000","last_unstale":"2023-10-16T11:19:25.479067+0000","last_undegraded":"2023-10-16T11:19:25.479067+0000","last_fullsized":"2023-10-16T11:19:25.479067+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:03.944394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645984+0000","last_change":"2023-10-16T11:19:23.563504+0000","last_active":"2023-10-16T11:19:26.645984+0000","last_peered":"2023-10-16T11:19:26.645984+0000","last_clean":"2023-10-16T11:19:26.645984+0000","last_became_active":"2023-10-16T11:19:23.563243+0000","last_became_peered":"2023-10-16T11:19:23.563243+0000","last_unstale":"2023-10-16T11:19:26.645984+0000","last_undegraded":"2023-10-16T11:19:26.645984+0000","last_fullsized":"2023-10-16T11:19:26.645984+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607783+0000","last_change":"2023-10-16T11:19:34.607783+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607783+0000","last_undegraded":"2023-10-16T11:19:34.607783+0000","last_fullsized":"2023-10-16T11:19:34.607783+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545888+0000","last_change":"2023-10-16T11:19:22.544852+0000","last_active":"2023-10-16T11:19:25.545888+0000","last_peered":"2023-10-16T11:19:25.545888+0000","last_clean":"2023-10-16T11:19:25.545888+0000","last_became_active":"2023-10-16T11:19:22.544565+0000","last_became_peered":"2023-10-16T11:19:22.544565+0000","last_unstale":"2023-10-16T11:19:25.545888+0000","last_undegraded":"2023-10-16T11:19:25.545888+0000","last_fullsized":"2023-10-16T11:19:25.545888+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:07.144566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646564+0000","last_change":"2023-10-16T11:19:23.558505+0000","last_active":"2023-10-16T11:19:26.646564+0000","last_peered":"2023-10-16T11:19:26.646564+0000","last_clean":"2023-10-16T11:19:26.646564+0000","last_became_active":"2023-10-16T11:19:23.558188+0000","last_became_peered":"2023-10-16T11:19:23.558188+0000","last_unstale":"2023-10-16T11:19:26.646564+0000","last_undegraded":"2023-10-16T11:19:26.646564+0000","last_fullsized":"2023-10-16T11:19:26.646564+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604846+0000","last_change":"2023-10-16T11:19:34.604846+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604846+0000","last_undegraded":"2023-10-16T11:19:34.604846+0000","last_fullsized":"2023-10-16T11:19:34.604846+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479062+0000","last_change":"2023-10-16T11:19:22.543881+0000","last_active":"2023-10-16T11:19:25.479062+0000","last_peered":"2023-10-16T11:19:25.479062+0000","last_clean":"2023-10-16T11:19:25.479062+0000","last_became_active":"2023-10-16T11:19:22.543647+0000","last_became_peered":"2023-10-16T11:19:22.543647+0000","last_unstale":"2023-10-16T11:19:25.479062+0000","last_undegraded":"2023-10-16T11:19:25.479062+0000","last_fullsized":"2023-10-16T11:19:25.479062+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:41.755970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645587+0000","last_change":"2023-10-16T11:19:23.546380+0000","last_active":"2023-10-16T11:19:26.645587+0000","last_peered":"2023-10-16T11:19:26.645587+0000","last_clean":"2023-10-16T11:19:26.645587+0000","last_became_active":"2023-10-16T11:19:23.546098+0000","last_became_peered":"2023-10-16T11:19:23.546098+0000","last_unstale":"2023-10-16T11:19:26.645587+0000","last_undegraded":"2023-10-16T11:19:26.645587+0000","last_fullsized":"2023-10-16T11:19:26.645587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:06.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.603595+0000","last_change":"2023-10-16T11:19:34.603595+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.603595+0000","last_undegraded":"2023-10-16T11:19:34.603595+0000","last_fullsized":"2023-10-16T11:19:34.603595+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646437+0000","last_change":"2023-10-16T11:19:22.537543+0000","last_active":"2023-10-16T11:19:26.646437+0000","last_peered":"2023-10-16T11:19:26.646437+0000","last_clean":"2023-10-16T11:19:26.646437+0000","last_became_active":"2023-10-16T11:19:22.537050+0000","last_became_peered":"2023-10-16T11:19:22.537050+0000","last_unstale":"2023-10-16T11:19:26.646437+0000","last_undegraded":"2023-10-16T11:19:26.646437+0000","last_fullsized":"2023-10-16T11:19:26.646437+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:03.218818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479172+0000","last_change":"2023-10-16T11:19:23.546472+0000","last_active":"2023-10-16T11:19:25.479172+0000","last_peered":"2023-10-16T11:19:25.479172+0000","last_clean":"2023-10-16T11:19:25.479172+0000","last_became_active":"2023-10-16T11:19:23.546160+0000","last_became_peered":"2023-10-16T11:19:23.546160+0000","last_unstale":"2023-10-16T11:19:25.479172+0000","last_undegraded":"2023-10-16T11:19:25.479172+0000","last_fullsized":"2023-10-16T11:19:25.479172+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:06.395224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610080+0000","last_change":"2023-10-16T11:19:34.610080+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610080+0000","last_undegraded":"2023-10-16T11:19:34.610080+0000","last_fullsized":"2023-10-16T11:19:34.610080+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545804+0000","last_change":"2023-10-16T11:19:22.543844+0000","last_active":"2023-10-16T11:19:25.545804+0000","last_peered":"2023-10-16T11:19:25.545804+0000","last_clean":"2023-10-16T11:19:25.545804+0000","last_became_active":"2023-10-16T11:19:22.543538+0000","last_became_peered":"2023-10-16T11:19:22.543538+0000","last_unstale":"2023-10-16T11:19:25.545804+0000","last_undegraded":"2023-10-16T11:19:25.545804+0000","last_fullsized":"2023-10-16T11:19:25.545804+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:00.946603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478841+0000","last_change":"2023-10-16T11:19:23.556539+0000","last_active":"2023-10-16T11:19:25.478841+0000","last_peered":"2023-10-16T11:19:25.478841+0000","last_clean":"2023-10-16T11:19:25.478841+0000","last_became_active":"2023-10-16T11:19:23.556209+0000","last_became_peered":"2023-10-16T11:19:23.556209+0000","last_unstale":"2023-10-16T11:19:25.478841+0000","last_undegraded":"2023-10-16T11:19:25.478841+0000","last_fullsized":"2023-10-16T11:19:25.478841+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607610+0000","last_change":"2023-10-16T11:19:34.607610+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607610+0000","last_undegraded":"2023-10-16T11:19:34.607610+0000","last_fullsized":"2023-10-16T11:19:34.607610+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:28.599812+0000","last_change":"2023-10-16T11:19:22.537758+0000","last_active":"2023-10-16T11:19:28.599812+0000","last_peered":"2023-10-16T11:19:28.599812+0000","last_clean":"2023-10-16T11:19:28.599812+0000","last_became_active":"2023-10-16T11:19:22.537201+0000","last_became_peered":"2023-10-16T11:19:22.537201+0000","last_unstale":"2023-10-16T11:19:28.599812+0000","last_undegraded":"2023-10-16T11:19:28.599812+0000","last_fullsized":"2023-10-16T11:19:28.599812+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:26.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646005+0000","last_change":"2023-10-16T11:19:23.564018+0000","last_active":"2023-10-16T11:19:26.646005+0000","last_peered":"2023-10-16T11:19:26.646005+0000","last_clean":"2023-10-16T11:19:26.646005+0000","last_became_active":"2023-10-16T11:19:23.563738+0000","last_became_peered":"2023-10-16T11:19:23.563738+0000","last_unstale":"2023-10-16T11:19:26.646005+0000","last_undegraded":"2023-10-16T11:19:26.646005+0000","last_fullsized":"2023-10-16T11:19:26.646005+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604846+0000","last_change":"2023-10-16T11:19:34.604846+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604846+0000","last_undegraded":"2023-10-16T11:19:34.604846+0000","last_fullsized":"2023-10-16T11:19:34.604846+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646652+0000","last_change":"2023-10-16T11:19:22.537256+0000","last_active":"2023-10-16T11:19:26.646652+0000","last_peered":"2023-10-16T11:19:26.646652+0000","last_clean":"2023-10-16T11:19:26.646652+0000","last_became_active":"2023-10-16T11:19:22.536996+0000","last_became_peered":"2023-10-16T11:19:22.536996+0000","last_unstale":"2023-10-16T11:19:26.646652+0000","last_undegraded":"2023-10-16T11:19:26.646652+0000","last_fullsized":"2023-10-16T11:19:26.646652+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:22.462616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646503+0000","last_change":"2023-10-16T11:19:23.561790+0000","last_active":"2023-10-16T11:19:26.646503+0000","last_peered":"2023-10-16T11:19:26.646503+0000","last_clean":"2023-10-16T11:19:26.646503+0000","last_became_active":"2023-10-16T11:19:23.561531+0000","last_became_peered":"2023-10-16T11:19:23.561531+0000","last_unstale":"2023-10-16T11:19:26.646503+0000","last_undegraded":"2023-10-16T11:19:26.646503+0000","last_fullsized":"2023-10-16T11:19:26.646503+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607676+0000","last_change":"2023-10-16T11:19:34.607676+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607676+0000","last_undegraded":"2023-10-16T11:19:34.607676+0000","last_fullsized":"2023-10-16T11:19:34.607676+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545608+0000","last_change":"2023-10-16T11:19:22.547829+0000","last_active":"2023-10-16T11:19:25.545608+0000","last_peered":"2023-10-16T11:19:25.545608+0000","last_clean":"2023-10-16T11:19:25.545608+0000","last_became_active":"2023-10-16T11:19:22.547563+0000","last_became_peered":"2023-10-16T11:19:22.547563+0000","last_unstale":"2023-10-16T11:19:25.545608+0000","last_undegraded":"2023-10-16T11:19:25.545608+0000","last_fullsized":"2023-10-16T11:19:25.545608+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:15.819032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479044+0000","last_change":"2023-10-16T11:19:23.563941+0000","last_active":"2023-10-16T11:19:25.479044+0000","last_peered":"2023-10-16T11:19:25.479044+0000","last_clean":"2023-10-16T11:19:25.479044+0000","last_became_active":"2023-10-16T11:19:23.563666+0000","last_became_peered":"2023-10-16T11:19:23.563666+0000","last_unstale":"2023-10-16T11:19:25.479044+0000","last_undegraded":"2023-10-16T11:19:25.479044+0000","last_fullsized":"2023-10-16T11:19:25.479044+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:43.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610615+0000","last_change":"2023-10-16T11:19:34.610615+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610615+0000","last_undegraded":"2023-10-16T11:19:34.610615+0000","last_fullsized":"2023-10-16T11:19:34.610615+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645872+0000","last_change":"2023-10-16T11:19:22.532222+0000","last_active":"2023-10-16T11:19:26.645872+0000","last_peered":"2023-10-16T11:19:26.645872+0000","last_clean":"2023-10-16T11:19:26.645872+0000","last_became_active":"2023-10-16T11:19:22.531939+0000","last_became_peered":"2023-10-16T11:19:22.531939+0000","last_unstale":"2023-10-16T11:19:26.645872+0000","last_undegraded":"2023-10-16T11:19:26.645872+0000","last_fullsized":"2023-10-16T11:19:26.645872+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478778+0000","last_change":"2023-10-16T11:19:23.548561+0000","last_active":"2023-10-16T11:19:25.478778+0000","last_peered":"2023-10-16T11:19:25.478778+0000","last_clean":"2023-10-16T11:19:25.478778+0000","last_became_active":"2023-10-16T11:19:23.548310+0000","last_became_peered":"2023-10-16T11:19:23.548310+0000","last_unstale":"2023-10-16T11:19:25.478778+0000","last_undegraded":"2023-10-16T11:19:25.478778+0000","last_fullsized":"2023-10-16T11:19:25.478778+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604887+0000","last_change":"2023-10-16T11:19:34.604887+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604887+0000","last_undegraded":"2023-10-16T11:19:34.604887+0000","last_fullsized":"2023-10-16T11:19:34.604887+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645841+0000","last_change":"2023-10-16T11:19:22.542021+0000","last_active":"2023-10-16T11:19:26.645841+0000","last_peered":"2023-10-16T11:19:26.645841+0000","last_clean":"2023-10-16T11:19:26.645841+0000","last_became_active":"2023-10-16T11:19:22.541718+0000","last_became_peered":"2023-10-16T11:19:22.541718+0000","last_unstale":"2023-10-16T11:19:26.645841+0000","last_undegraded":"2023-10-16T11:19:26.645841+0000","last_fullsized":"2023-10-16T11:19:26.645841+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:12.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646346+0000","last_change":"2023-10-16T11:19:23.554949+0000","last_active":"2023-10-16T11:19:26.646346+0000","last_peered":"2023-10-16T11:19:26.646346+0000","last_clean":"2023-10-16T11:19:26.646346+0000","last_became_active":"2023-10-16T11:19:23.554300+0000","last_became_peered":"2023-10-16T11:19:23.554300+0000","last_unstale":"2023-10-16T11:19:26.646346+0000","last_undegraded":"2023-10-16T11:19:26.646346+0000","last_fullsized":"2023-10-16T11:19:26.646346+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604641+0000","last_change":"2023-10-16T11:19:34.604641+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604641+0000","last_undegraded":"2023-10-16T11:19:34.604641+0000","last_fullsized":"2023-10-16T11:19:34.604641+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475451+0000","last_change":"2023-10-16T11:19:22.536403+0000","last_active":"2023-10-16T11:19:25.475451+0000","last_peered":"2023-10-16T11:19:25.475451+0000","last_clean":"2023-10-16T11:19:25.475451+0000","last_became_active":"2023-10-16T11:19:22.536132+0000","last_became_peered":"2023-10-16T11:19:22.536132+0000","last_unstale":"2023-10-16T11:19:25.475451+0000","last_undegraded":"2023-10-16T11:19:25.475451+0000","last_fullsized":"2023-10-16T11:19:25.475451+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:53.563487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475436+0000","last_change":"2023-10-16T11:19:23.543334+0000","last_active":"2023-10-16T11:19:25.475436+0000","last_peered":"2023-10-16T11:19:25.475436+0000","last_clean":"2023-10-16T11:19:25.475436+0000","last_became_active":"2023-10-16T11:19:23.542527+0000","last_became_peered":"2023-10-16T11:19:23.542527+0000","last_unstale":"2023-10-16T11:19:25.475436+0000","last_undegraded":"2023-10-16T11:19:25.475436+0000","last_fullsized":"2023-10-16T11:19:25.475436+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:59.083108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.602964+0000","last_change":"2023-10-16T11:19:34.602964+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.602964+0000","last_undegraded":"2023-10-16T11:19:34.602964+0000","last_fullsized":"2023-10-16T11:19:34.602964+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646351+0000","last_change":"2023-10-16T11:19:22.545030+0000","last_active":"2023-10-16T11:19:26.646351+0000","last_peered":"2023-10-16T11:19:26.646351+0000","last_clean":"2023-10-16T11:19:26.646351+0000","last_became_active":"2023-10-16T11:19:22.544773+0000","last_became_peered":"2023-10-16T11:19:22.544773+0000","last_unstale":"2023-10-16T11:19:26.646351+0000","last_undegraded":"2023-10-16T11:19:26.646351+0000","last_fullsized":"2023-10-16T11:19:26.646351+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:22.946353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475507+0000","last_change":"2023-10-16T11:19:23.545760+0000","last_active":"2023-10-16T11:19:25.475507+0000","last_peered":"2023-10-16T11:19:25.475507+0000","last_clean":"2023-10-16T11:19:25.475507+0000","last_became_active":"2023-10-16T11:19:23.545499+0000","last_became_peered":"2023-10-16T11:19:23.545499+0000","last_unstale":"2023-10-16T11:19:25.475507+0000","last_undegraded":"2023-10-16T11:19:25.475507+0000","last_fullsized":"2023-10-16T11:19:25.475507+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:32.701590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607606+0000","last_change":"2023-10-16T11:19:34.607606+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607606+0000","last_undegraded":"2023-10-16T11:19:34.607606+0000","last_fullsized":"2023-10-16T11:19:34.607606+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475536+0000","last_change":"2023-10-16T11:19:22.533601+0000","last_active":"2023-10-16T11:19:25.475536+0000","last_peered":"2023-10-16T11:19:25.475536+0000","last_clean":"2023-10-16T11:19:25.475536+0000","last_became_active":"2023-10-16T11:19:22.533374+0000","last_became_peered":"2023-10-16T11:19:22.533374+0000","last_unstale":"2023-10-16T11:19:25.475536+0000","last_undegraded":"2023-10-16T11:19:25.475536+0000","last_fullsized":"2023-10-16T11:19:25.475536+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:52.564361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646427+0000","last_change":"2023-10-16T11:19:23.553745+0000","last_active":"2023-10-16T11:19:26.646427+0000","last_peered":"2023-10-16T11:19:26.646427+0000","last_clean":"2023-10-16T11:19:26.646427+0000","last_became_active":"2023-10-16T11:19:23.553462+0000","last_became_peered":"2023-10-16T11:19:23.553462+0000","last_unstale":"2023-10-16T11:19:26.646427+0000","last_undegraded":"2023-10-16T11:19:26.646427+0000","last_fullsized":"2023-10-16T11:19:26.646427+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604947+0000","last_change":"2023-10-16T11:19:34.604947+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604947+0000","last_undegraded":"2023-10-16T11:19:34.604947+0000","last_fullsized":"2023-10-16T11:19:34.604947+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478908+0000","last_change":"2023-10-16T11:19:22.539558+0000","last_active":"2023-10-16T11:19:25.478908+0000","last_peered":"2023-10-16T11:19:25.478908+0000","last_clean":"2023-10-16T11:19:25.478908+0000","last_became_active":"2023-10-16T11:19:22.539284+0000","last_became_peered":"2023-10-16T11:19:22.539284+0000","last_unstale":"2023-10-16T11:19:25.478908+0000","last_undegraded":"2023-10-16T11:19:25.478908+0000","last_fullsized":"2023-10-16T11:19:25.478908+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478999+0000","last_change":"2023-10-16T11:19:23.546218+0000","last_active":"2023-10-16T11:19:25.478999+0000","last_peered":"2023-10-16T11:19:25.478999+0000","last_clean":"2023-10-16T11:19:25.478999+0000","last_became_active":"2023-10-16T11:19:23.545896+0000","last_became_peered":"2023-10-16T11:19:23.545896+0000","last_unstale":"2023-10-16T11:19:25.478999+0000","last_undegraded":"2023-10-16T11:19:25.478999+0000","last_fullsized":"2023-10-16T11:19:25.478999+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.605391+0000","last_change":"2023-10-16T11:19:34.605391+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.605391+0000","last_undegraded":"2023-10-16T11:19:34.605391+0000","last_fullsized":"2023-10-16T11:19:34.605391+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545784+0000","last_change":"2023-10-16T11:19:22.545027+0000","last_active":"2023-10-16T11:19:25.545784+0000","last_peered":"2023-10-16T11:19:25.545784+0000","last_clean":"2023-10-16T11:19:25.545784+0000","last_became_active":"2023-10-16T11:19:22.544756+0000","last_became_peered":"2023-10-16T11:19:22.544756+0000","last_unstale":"2023-10-16T11:19:25.545784+0000","last_undegraded":"2023-10-16T11:19:25.545784+0000","last_fullsized":"2023-10-16T11:19:25.545784+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:23.693322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478882+0000","last_change":"2023-10-16T11:19:23.547530+0000","last_active":"2023-10-16T11:19:25.478882+0000","last_peered":"2023-10-16T11:19:25.478882+0000","last_clean":"2023-10-16T11:19:25.478882+0000","last_became_active":"2023-10-16T11:19:23.547255+0000","last_became_peered":"2023-10-16T11:19:23.547255+0000","last_unstale":"2023-10-16T11:19:25.478882+0000","last_undegraded":"2023-10-16T11:19:25.478882+0000","last_fullsized":"2023-10-16T11:19:25.478882+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.608216+0000","last_change":"2023-10-16T11:19:34.608216+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.608216+0000","last_undegraded":"2023-10-16T11:19:34.608216+0000","last_fullsized":"2023-10-16T11:19:34.608216+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":21,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646100+0000","last_change":"2023-10-16T11:19:22.534093+0000","last_active":"2023-10-16T11:19:26.646100+0000","last_peered":"2023-10-16T11:19:26.646100+0000","last_clean":"2023-10-16T11:19:26.646100+0000","last_became_active":"2023-10-16T11:19:22.533637+0000","last_became_peered":"2023-10-16T11:19:22.533637+0000","last_unstale":"2023-10-16T11:19:26.646100+0000","last_undegraded":"2023-10-16T11:19:26.646100+0000","last_fullsized":"2023-10-16T11:19:26.646100+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645780+0000","last_change":"2023-10-16T11:19:23.552950+0000","last_active":"2023-10-16T11:19:26.645780+0000","last_peered":"2023-10-16T11:19:26.645780+0000","last_clean":"2023-10-16T11:19:26.645780+0000","last_became_active":"2023-10-16T11:19:23.552686+0000","last_became_peered":"2023-10-16T11:19:23.552686+0000","last_unstale":"2023-10-16T11:19:26.645780+0000","last_undegraded":"2023-10-16T11:19:26.645780+0000","last_fullsized":"2023-10-16T11:19:26.645780+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610145+0000","last_change":"2023-10-16T11:19:34.610145+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610145+0000","last_undegraded":"2023-10-16T11:19:34.610145+0000","last_fullsized":"2023-10-16T11:19:34.610145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475421+0000","last_change":"2023-10-16T11:19:22.538008+0000","last_active":"2023-10-16T11:19:25.475421+0000","last_peered":"2023-10-16T11:19:25.475421+0000","last_clean":"2023-10-16T11:19:25.475421+0000","last_became_active":"2023-10-16T11:19:22.537752+0000","last_became_peered":"2023-10-16T11:19:22.537752+0000","last_unstale":"2023-10-16T11:19:25.475421+0000","last_undegraded":"2023-10-16T11:19:25.475421+0000","last_fullsized":"2023-10-16T11:19:25.475421+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:20.859898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645905+0000","last_change":"2023-10-16T11:19:23.563603+0000","last_active":"2023-10-16T11:19:26.645905+0000","last_peered":"2023-10-16T11:19:26.645905+0000","last_clean":"2023-10-16T11:19:26.645905+0000","last_became_active":"2023-10-16T11:19:23.563305+0000","last_became_peered":"2023-10-16T11:19:23.563305+0000","last_unstale":"2023-10-16T11:19:26.645905+0000","last_undegraded":"2023-10-16T11:19:26.645905+0000","last_fullsized":"2023-10-16T11:19:26.645905+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.602965+0000","last_change":"2023-10-16T11:19:34.602965+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.602965+0000","last_undegraded":"2023-10-16T11:19:34.602965+0000","last_fullsized":"2023-10-16T11:19:34.602965+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479114+0000","last_change":"2023-10-16T11:19:22.538147+0000","last_active":"2023-10-16T11:19:25.479114+0000","last_peered":"2023-10-16T11:19:25.479114+0000","last_clean":"2023-10-16T11:19:25.479114+0000","last_became_active":"2023-10-16T11:19:22.537850+0000","last_became_peered":"2023-10-16T11:19:22.537850+0000","last_unstale":"2023-10-16T11:19:25.479114+0000","last_undegraded":"2023-10-16T11:19:25.479114+0000","last_fullsized":"2023-10-16T11:19:25.479114+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:44.794069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475465+0000","last_change":"2023-10-16T11:19:23.552779+0000","last_active":"2023-10-16T11:19:25.475465+0000","last_peered":"2023-10-16T11:19:25.475465+0000","last_clean":"2023-10-16T11:19:25.475465+0000","last_became_active":"2023-10-16T11:19:23.552491+0000","last_became_peered":"2023-10-16T11:19:23.552491+0000","last_unstale":"2023-10-16T11:19:25.475465+0000","last_undegraded":"2023-10-16T11:19:25.475465+0000","last_fullsized":"2023-10-16T11:19:25.475465+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:02.969587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.605301+0000","last_change":"2023-10-16T11:19:34.605301+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.605301+0000","last_undegraded":"2023-10-16T11:19:34.605301+0000","last_fullsized":"2023-10-16T11:19:34.605301+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479149+0000","last_change":"2023-10-16T11:19:22.544032+0000","last_active":"2023-10-16T11:19:25.479149+0000","last_peered":"2023-10-16T11:19:25.479149+0000","last_clean":"2023-10-16T11:19:25.479149+0000","last_became_active":"2023-10-16T11:19:22.543713+0000","last_became_peered":"2023-10-16T11:19:22.543713+0000","last_unstale":"2023-10-16T11:19:25.479149+0000","last_undegraded":"2023-10-16T11:19:25.479149+0000","last_fullsized":"2023-10-16T11:19:25.479149+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:01.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545729+0000","last_change":"2023-10-16T11:19:23.550398+0000","last_active":"2023-10-16T11:19:25.545729+0000","last_peered":"2023-10-16T11:19:25.545729+0000","last_clean":"2023-10-16T11:19:25.545729+0000","last_became_active":"2023-10-16T11:19:23.545907+0000","last_became_peered":"2023-10-16T11:19:23.545907+0000","last_unstale":"2023-10-16T11:19:25.545729+0000","last_undegraded":"2023-10-16T11:19:25.545729+0000","last_fullsized":"2023-10-16T11:19:25.545729+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:05.456349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607838+0000","last_change":"2023-10-16T11:19:34.607838+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607838+0000","last_undegraded":"2023-10-16T11:19:34.607838+0000","last_fullsized":"2023-10-16T11:19:34.607838+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"216'8","reported_seq":21,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:33.599726+0000","last_change":"2023-10-16T11:19:22.532339+0000","last_active":"2023-10-16T11:19:33.599726+0000","last_peered":"2023-10-16T11:19:33.599726+0000","last_clean":"2023-10-16T11:19:33.599726+0000","last_became_active":"2023-10-16T11:19:22.532053+0000","last_became_peered":"2023-10-16T11:19:22.532053+0000","last_unstale":"2023-10-16T11:19:33.599726+0000","last_undegraded":"2023-10-16T11:19:33.599726+0000","last_fullsized":"2023-10-16T11:19:33.599726+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:21.190168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2859,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645752+0000","last_change":"2023-10-16T11:19:23.548103+0000","last_active":"2023-10-16T11:19:26.645752+0000","last_peered":"2023-10-16T11:19:26.645752+0000","last_clean":"2023-10-16T11:19:26.645752+0000","last_became_active":"2023-10-16T11:19:23.547665+0000","last_became_peered":"2023-10-16T11:19:23.547665+0000","last_unstale":"2023-10-16T11:19:26.645752+0000","last_undegraded":"2023-10-16T11:19:26.645752+0000","last_fullsized":"2023-10-16T11:19:26.645752+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610953+0000","last_change":"2023-10-16T11:19:34.610953+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610953+0000","last_undegraded":"2023-10-16T11:19:34.610953+0000","last_fullsized":"2023-10-16T11:19:34.610953+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":16,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475523+0000","last_change":"2023-10-16T11:19:22.544432+0000","last_active":"2023-10-16T11:19:25.475523+0000","last_peered":"2023-10-16T11:19:25.475523+0000","last_clean":"2023-10-16T11:19:25.475523+0000","last_became_active":"2023-10-16T11:19:22.543046+0000","last_became_peered":"2023-10-16T11:19:22.543046+0000","last_unstale":"2023-10-16T11:19:25.475523+0000","last_undegraded":"2023-10-16T11:19:25.475523+0000","last_fullsized":"2023-10-16T11:19:25.475523+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:08.672505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479100+0000","last_change":"2023-10-16T11:19:23.563660+0000","last_active":"2023-10-16T11:19:25.479100+0000","last_peered":"2023-10-16T11:19:25.479100+0000","last_clean":"2023-10-16T11:19:25.479100+0000","last_became_active":"2023-10-16T11:19:23.563402+0000","last_became_peered":"2023-10-16T11:19:23.563402+0000","last_unstale":"2023-10-16T11:19:25.479100+0000","last_undegraded":"2023-10-16T11:19:25.479100+0000","last_fullsized":"2023-10-16T11:19:25.479100+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:41.606189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.605645+0000","last_change":"2023-10-16T11:19:34.605645+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.605645+0000","last_undegraded":"2023-10-16T11:19:34.605645+0000","last_fullsized":"2023-10-16T11:19:34.605645+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479196+0000","last_change":"2023-10-16T11:19:22.535771+0000","last_active":"2023-10-16T11:19:25.479196+0000","last_peered":"2023-10-16T11:19:25.479196+0000","last_clean":"2023-10-16T11:19:25.479196+0000","last_became_active":"2023-10-16T11:19:22.535486+0000","last_became_peered":"2023-10-16T11:19:22.535486+0000","last_unstale":"2023-10-16T11:19:25.479196+0000","last_undegraded":"2023-10-16T11:19:25.479196+0000","last_fullsized":"2023-10-16T11:19:25.479196+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:25.214100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646695+0000","last_change":"2023-10-16T11:19:23.560046+0000","last_active":"2023-10-16T11:19:26.646695+0000","last_peered":"2023-10-16T11:19:26.646695+0000","last_clean":"2023-10-16T11:19:26.646695+0000","last_became_active":"2023-10-16T11:19:23.559718+0000","last_became_peered":"2023-10-16T11:19:23.559718+0000","last_unstale":"2023-10-16T11:19:26.646695+0000","last_undegraded":"2023-10-16T11:19:26.646695+0000","last_fullsized":"2023-10-16T11:19:26.646695+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:22.701369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.609835+0000","last_change":"2023-10-16T11:19:34.609835+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.609835+0000","last_undegraded":"2023-10-16T11:19:34.609835+0000","last_fullsized":"2023-10-16T11:19:34.609835+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545940+0000","last_change":"2023-10-16T11:19:22.545557+0000","last_active":"2023-10-16T11:19:25.545940+0000","last_peered":"2023-10-16T11:19:25.545940+0000","last_clean":"2023-10-16T11:19:25.545940+0000","last_became_active":"2023-10-16T11:19:22.545253+0000","last_became_peered":"2023-10-16T11:19:22.545253+0000","last_unstale":"2023-10-16T11:19:25.545940+0000","last_undegraded":"2023-10-16T11:19:25.545940+0000","last_fullsized":"2023-10-16T11:19:25.545940+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:11.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479179+0000","last_change":"2023-10-16T11:19:23.553070+0000","last_active":"2023-10-16T11:19:25.479179+0000","last_peered":"2023-10-16T11:19:25.479179+0000","last_clean":"2023-10-16T11:19:25.479179+0000","last_became_active":"2023-10-16T11:19:23.552761+0000","last_became_peered":"2023-10-16T11:19:23.552761+0000","last_unstale":"2023-10-16T11:19:25.479179+0000","last_undegraded":"2023-10-16T11:19:25.479179+0000","last_fullsized":"2023-10-16T11:19:25.479179+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:36.373122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.608115+0000","last_change":"2023-10-16T11:19:34.608115+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.608115+0000","last_undegraded":"2023-10-16T11:19:34.608115+0000","last_fullsized":"2023-10-16T11:19:34.608115+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478941+0000","last_change":"2023-10-16T11:19:22.534080+0000","last_active":"2023-10-16T11:19:25.478941+0000","last_peered":"2023-10-16T11:19:25.478941+0000","last_clean":"2023-10-16T11:19:25.478941+0000","last_became_active":"2023-10-16T11:19:22.533693+0000","last_became_peered":"2023-10-16T11:19:22.533693+0000","last_unstale":"2023-10-16T11:19:25.478941+0000","last_undegraded":"2023-10-16T11:19:25.478941+0000","last_fullsized":"2023-10-16T11:19:25.478941+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:57.023698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479023+0000","last_change":"2023-10-16T11:19:23.546966+0000","last_active":"2023-10-16T11:19:25.479023+0000","last_peered":"2023-10-16T11:19:25.479023+0000","last_clean":"2023-10-16T11:19:25.479023+0000","last_became_active":"2023-10-16T11:19:23.546685+0000","last_became_peered":"2023-10-16T11:19:23.546685+0000","last_unstale":"2023-10-16T11:19:25.479023+0000","last_undegraded":"2023-10-16T11:19:25.479023+0000","last_fullsized":"2023-10-16T11:19:25.479023+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:13.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.609902+0000","last_change":"2023-10-16T11:19:34.609902+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.609902+0000","last_undegraded":"2023-10-16T11:19:34.609902+0000","last_fullsized":"2023-10-16T11:19:34.609902+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545426+0000","last_change":"2023-10-16T11:19:22.547192+0000","last_active":"2023-10-16T11:19:25.545426+0000","last_peered":"2023-10-16T11:19:25.545426+0000","last_clean":"2023-10-16T11:19:25.545426+0000","last_became_active":"2023-10-16T11:19:22.546912+0000","last_became_peered":"2023-10-16T11:19:22.546912+0000","last_unstale":"2023-10-16T11:19:25.545426+0000","last_undegraded":"2023-10-16T11:19:25.545426+0000","last_fullsized":"2023-10-16T11:19:25.545426+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:17.017260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545575+0000","last_change":"2023-10-16T11:19:23.543708+0000","last_active":"2023-10-16T11:19:25.545575+0000","last_peered":"2023-10-16T11:19:25.545575+0000","last_clean":"2023-10-16T11:19:25.545575+0000","last_became_active":"2023-10-16T11:19:23.543449+0000","last_became_peered":"2023-10-16T11:19:23.543449+0000","last_unstale":"2023-10-16T11:19:25.545575+0000","last_undegraded":"2023-10-16T11:19:25.545575+0000","last_fullsized":"2023-10-16T11:19:25.545575+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:06.406149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610145+0000","last_change":"2023-10-16T11:19:34.610145+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610145+0000","last_undegraded":"2023-10-16T11:19:34.610145+0000","last_fullsized":"2023-10-16T11:19:34.610145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545752+0000","last_change":"2023-10-16T11:19:22.543956+0000","last_active":"2023-10-16T11:19:25.545752+0000","last_peered":"2023-10-16T11:19:25.545752+0000","last_clean":"2023-10-16T11:19:25.545752+0000","last_became_active":"2023-10-16T11:19:22.543633+0000","last_became_peered":"2023-10-16T11:19:22.543633+0000","last_unstale":"2023-10-16T11:19:25.545752+0000","last_undegraded":"2023-10-16T11:19:25.545752+0000","last_fullsized":"2023-10-16T11:19:25.545752+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:12:53.412047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479047+0000","last_change":"2023-10-16T11:19:23.546200+0000","last_active":"2023-10-16T11:19:25.479047+0000","last_peered":"2023-10-16T11:19:25.479047+0000","last_clean":"2023-10-16T11:19:25.479047+0000","last_became_active":"2023-10-16T11:19:23.545867+0000","last_became_peered":"2023-10-16T11:19:23.545867+0000","last_unstale":"2023-10-16T11:19:25.479047+0000","last_undegraded":"2023-10-16T11:19:25.479047+0000","last_fullsized":"2023-10-16T11:19:25.479047+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.670688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.609868+0000","last_change":"2023-10-16T11:19:34.609868+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.609868+0000","last_undegraded":"2023-10-16T11:19:34.609868+0000","last_fullsized":"2023-10-16T11:19:34.609868+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479132+0000","last_change":"2023-10-16T11:19:22.535919+0000","last_active":"2023-10-16T11:19:25.479132+0000","last_peered":"2023-10-16T11:19:25.479132+0000","last_clean":"2023-10-16T11:19:25.479132+0000","last_became_active":"2023-10-16T11:19:22.535612+0000","last_became_peered":"2023-10-16T11:19:22.535612+0000","last_unstale":"2023-10-16T11:19:25.479132+0000","last_undegraded":"2023-10-16T11:19:25.479132+0000","last_fullsized":"2023-10-16T11:19:25.479132+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:54.628429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475493+0000","last_change":"2023-10-16T11:19:23.552923+0000","last_active":"2023-10-16T11:19:25.475493+0000","last_peered":"2023-10-16T11:19:25.475493+0000","last_clean":"2023-10-16T11:19:25.475493+0000","last_became_active":"2023-10-16T11:19:23.552685+0000","last_became_peered":"2023-10-16T11:19:23.552685+0000","last_unstale":"2023-10-16T11:19:25.475493+0000","last_undegraded":"2023-10-16T11:19:25.475493+0000","last_fullsized":"2023-10-16T11:19:25.475493+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:13.492062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604821+0000","last_change":"2023-10-16T11:19:34.604821+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604821+0000","last_undegraded":"2023-10-16T11:19:34.604821+0000","last_fullsized":"2023-10-16T11:19:34.604821+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646598+0000","last_change":"2023-10-16T11:19:22.533919+0000","last_active":"2023-10-16T11:19:26.646598+0000","last_peered":"2023-10-16T11:19:26.646598+0000","last_clean":"2023-10-16T11:19:26.646598+0000","last_became_active":"2023-10-16T11:19:22.533695+0000","last_became_peered":"2023-10-16T11:19:22.533695+0000","last_unstale":"2023-10-16T11:19:26.646598+0000","last_undegraded":"2023-10-16T11:19:26.646598+0000","last_fullsized":"2023-10-16T11:19:26.646598+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645700+0000","last_change":"2023-10-16T11:19:23.546849+0000","last_active":"2023-10-16T11:19:26.645700+0000","last_peered":"2023-10-16T11:19:26.645700+0000","last_clean":"2023-10-16T11:19:26.645700+0000","last_became_active":"2023-10-16T11:19:23.546575+0000","last_became_peered":"2023-10-16T11:19:23.546575+0000","last_unstale":"2023-10-16T11:19:26.645700+0000","last_undegraded":"2023-10-16T11:19:26.645700+0000","last_fullsized":"2023-10-16T11:19:26.645700+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.884361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.606124+0000","last_change":"2023-10-16T11:19:34.606124+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.606124+0000","last_undegraded":"2023-10-16T11:19:34.606124+0000","last_fullsized":"2023-10-16T11:19:34.606124+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478867+0000","last_change":"2023-10-16T11:19:22.536812+0000","last_active":"2023-10-16T11:19:25.478867+0000","last_peered":"2023-10-16T11:19:25.478867+0000","last_clean":"2023-10-16T11:19:25.478867+0000","last_became_active":"2023-10-16T11:19:22.536545+0000","last_became_peered":"2023-10-16T11:19:22.536545+0000","last_unstale":"2023-10-16T11:19:25.478867+0000","last_undegraded":"2023-10-16T11:19:25.478867+0000","last_fullsized":"2023-10-16T11:19:25.478867+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479216+0000","last_change":"2023-10-16T11:19:23.559791+0000","last_active":"2023-10-16T11:19:25.479216+0000","last_peered":"2023-10-16T11:19:25.479216+0000","last_clean":"2023-10-16T11:19:25.479216+0000","last_became_active":"2023-10-16T11:19:23.559513+0000","last_became_peered":"2023-10-16T11:19:23.559513+0000","last_unstale":"2023-10-16T11:19:25.479216+0000","last_undegraded":"2023-10-16T11:19:25.479216+0000","last_fullsized":"2023-10-16T11:19:25.479216+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:39.986537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604598+0000","last_change":"2023-10-16T11:19:34.604598+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604598+0000","last_undegraded":"2023-10-16T11:19:34.604598+0000","last_fullsized":"2023-10-16T11:19:34.604598+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":17,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646632+0000","last_change":"2023-10-16T11:19:22.538829+0000","last_active":"2023-10-16T11:19:26.646632+0000","last_peered":"2023-10-16T11:19:26.646632+0000","last_clean":"2023-10-16T11:19:26.646632+0000","last_became_active":"2023-10-16T11:19:22.538537+0000","last_became_peered":"2023-10-16T11:19:22.538537+0000","last_unstale":"2023-10-16T11:19:26.646632+0000","last_undegraded":"2023-10-16T11:19:26.646632+0000","last_fullsized":"2023-10-16T11:19:26.646632+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:09.892801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478853+0000","last_change":"2023-10-16T11:19:23.551775+0000","last_active":"2023-10-16T11:19:25.478853+0000","last_peered":"2023-10-16T11:19:25.478853+0000","last_clean":"2023-10-16T11:19:25.478853+0000","last_became_active":"2023-10-16T11:19:23.551490+0000","last_became_peered":"2023-10-16T11:19:23.551490+0000","last_unstale":"2023-10-16T11:19:25.478853+0000","last_undegraded":"2023-10-16T11:19:25.478853+0000","last_fullsized":"2023-10-16T11:19:25.478853+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607691+0000","last_change":"2023-10-16T11:19:34.607691+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607691+0000","last_undegraded":"2023-10-16T11:19:34.607691+0000","last_fullsized":"2023-10-16T11:19:34.607691+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645427+0000","last_change":"2023-10-16T11:19:23.544178+0000","last_active":"2023-10-16T11:19:26.645427+0000","last_peered":"2023-10-16T11:19:26.645427+0000","last_clean":"2023-10-16T11:19:26.645427+0000","last_became_active":"2023-10-16T11:19:23.543898+0000","last_became_peered":"2023-10-16T11:19:23.543898+0000","last_unstale":"2023-10-16T11:19:26.645427+0000","last_undegraded":"2023-10-16T11:19:26.645427+0000","last_fullsized":"2023-10-16T11:19:26.645427+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475405+0000","last_change":"2023-10-16T11:19:23.542878+0000","last_active":"2023-10-16T11:19:25.475405+0000","last_peered":"2023-10-16T11:19:25.475405+0000","last_clean":"2023-10-16T11:19:25.475405+0000","last_became_active":"2023-10-16T11:19:23.542593+0000","last_became_peered":"2023-10-16T11:19:23.542593+0000","last_unstale":"2023-10-16T11:19:25.475405+0000","last_undegraded":"2023-10-16T11:19:25.475405+0000","last_fullsized":"2023-10-16T11:19:25.475405+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:39.898154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478829+0000","last_change":"2023-10-16T11:19:23.547762+0000","last_active":"2023-10-16T11:19:25.478829+0000","last_peered":"2023-10-16T11:19:25.478829+0000","last_clean":"2023-10-16T11:19:25.478829+0000","last_became_active":"2023-10-16T11:19:23.547353+0000","last_became_peered":"2023-10-16T11:19:23.547353+0000","last_unstale":"2023-10-16T11:19:25.478829+0000","last_undegraded":"2023-10-16T11:19:25.478829+0000","last_fullsized":"2023-10-16T11:19:25.478829+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646339+0000","last_change":"2023-10-16T11:19:23.565127+0000","last_active":"2023-10-16T11:19:26.646339+0000","last_peered":"2023-10-16T11:19:26.646339+0000","last_clean":"2023-10-16T11:19:26.646339+0000","last_became_active":"2023-10-16T11:19:23.564852+0000","last_became_peered":"2023-10-16T11:19:23.564852+0000","last_unstale":"2023-10-16T11:19:26.646339+0000","last_undegraded":"2023-10-16T11:19:26.646339+0000","last_fullsized":"2023-10-16T11:19:26.646339+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646255+0000","last_change":"2023-10-16T11:19:23.555152+0000","last_active":"2023-10-16T11:19:26.646255+0000","last_peered":"2023-10-16T11:19:26.646255+0000","last_clean":"2023-10-16T11:19:26.646255+0000","last_became_active":"2023-10-16T11:19:23.554747+0000","last_became_peered":"2023-10-16T11:19:23.554747+0000","last_unstale":"2023-10-16T11:19:26.646255+0000","last_undegraded":"2023-10-16T11:19:26.646255+0000","last_fullsized":"2023-10-16T11:19:26.646255+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646479+0000","last_change":"2023-10-16T11:19:23.559781+0000","last_active":"2023-10-16T11:19:26.646479+0000","last_peered":"2023-10-16T11:19:26.646479+0000","last_clean":"2023-10-16T11:19:26.646479+0000","last_became_active":"2023-10-16T11:19:23.559451+0000","last_became_peered":"2023-10-16T11:19:23.559451+0000","last_unstale":"2023-10-16T11:19:26.646479+0000","last_undegraded":"2023-10-16T11:19:26.646479+0000","last_fullsized":"2023-10-16T11:19:26.646479+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645942+0000","last_change":"2023-10-16T11:19:23.546460+0000","last_active":"2023-10-16T11:19:26.645942+0000","last_peered":"2023-10-16T11:19:26.645942+0000","last_clean":"2023-10-16T11:19:26.645942+0000","last_became_active":"2023-10-16T11:19:23.546187+0000","last_became_peered":"2023-10-16T11:19:23.546187+0000","last_unstale":"2023-10-16T11:19:26.645942+0000","last_undegraded":"2023-10-16T11:19:26.645942+0000","last_fullsized":"2023-10-16T11:19:26.645942+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478979+0000","last_change":"2023-10-16T11:19:23.546857+0000","last_active":"2023-10-16T11:19:25.478979+0000","last_peered":"2023-10-16T11:19:25.478979+0000","last_clean":"2023-10-16T11:19:25.478979+0000","last_became_active":"2023-10-16T11:19:23.546592+0000","last_became_peered":"2023-10-16T11:19:23.546592+0000","last_unstale":"2023-10-16T11:19:25.478979+0000","last_undegraded":"2023-10-16T11:19:25.478979+0000","last_fullsized":"2023-10-16T11:19:25.478979+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478981+0000","last_change":"2023-10-16T11:19:23.548868+0000","last_active":"2023-10-16T11:19:25.478981+0000","last_peered":"2023-10-16T11:19:25.478981+0000","last_clean":"2023-10-16T11:19:25.478981+0000","last_became_active":"2023-10-16T11:19:23.548583+0000","last_became_peered":"2023-10-16T11:19:23.548583+0000","last_unstale":"2023-10-16T11:19:25.478981+0000","last_undegraded":"2023-10-16T11:19:25.478981+0000","last_fullsized":"2023-10-16T11:19:25.478981+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:02.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479015+0000","last_change":"2023-10-16T11:19:23.552048+0000","last_active":"2023-10-16T11:19:25.479015+0000","last_peered":"2023-10-16T11:19:25.479015+0000","last_clean":"2023-10-16T11:19:25.479015+0000","last_became_active":"2023-10-16T11:19:23.551676+0000","last_became_peered":"2023-10-16T11:19:23.551676+0000","last_unstale":"2023-10-16T11:19:25.479015+0000","last_undegraded":"2023-10-16T11:19:25.479015+0000","last_fullsized":"2023-10-16T11:19:25.479015+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:16.450373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479089+0000","last_change":"2023-10-16T11:19:23.546038+0000","last_active":"2023-10-16T11:19:25.479089+0000","last_peered":"2023-10-16T11:19:25.479089+0000","last_clean":"2023-10-16T11:19:25.479089+0000","last_became_active":"2023-10-16T11:19:23.545752+0000","last_became_peered":"2023-10-16T11:19:23.545752+0000","last_unstale":"2023-10-16T11:19:25.479089+0000","last_undegraded":"2023-10-16T11:19:25.479089+0000","last_fullsized":"2023-10-16T11:19:25.479089+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:13.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479037+0000","last_change":"2023-10-16T11:19:23.550209+0000","last_active":"2023-10-16T11:19:25.479037+0000","last_peered":"2023-10-16T11:19:25.479037+0000","last_clean":"2023-10-16T11:19:25.479037+0000","last_became_active":"2023-10-16T11:19:23.549849+0000","last_became_peered":"2023-10-16T11:19:23.549849+0000","last_unstale":"2023-10-16T11:19:25.479037+0000","last_undegraded":"2023-10-16T11:19:25.479037+0000","last_fullsized":"2023-10-16T11:19:25.479037+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:29.715382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545908+0000","last_change":"2023-10-16T11:19:23.542321+0000","last_active":"2023-10-16T11:19:25.545908+0000","last_peered":"2023-10-16T11:19:25.545908+0000","last_clean":"2023-10-16T11:19:25.545908+0000","last_became_active":"2023-10-16T11:19:23.541862+0000","last_became_peered":"2023-10-16T11:19:23.541862+0000","last_unstale":"2023-10-16T11:19:25.545908+0000","last_undegraded":"2023-10-16T11:19:25.545908+0000","last_fullsized":"2023-10-16T11:19:25.545908+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:24:50.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475479+0000","last_change":"2023-10-16T11:19:23.547740+0000","last_active":"2023-10-16T11:19:25.475479+0000","last_peered":"2023-10-16T11:19:25.475479+0000","last_clean":"2023-10-16T11:19:25.475479+0000","last_became_active":"2023-10-16T11:19:23.546421+0000","last_became_peered":"2023-10-16T11:19:23.546421+0000","last_unstale":"2023-10-16T11:19:25.475479+0000","last_undegraded":"2023-10-16T11:19:25.475479+0000","last_fullsized":"2023-10-16T11:19:25.475479+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:27.685924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475560+0000","last_change":"2023-10-16T11:19:23.549769+0000","last_active":"2023-10-16T11:19:25.475560+0000","last_peered":"2023-10-16T11:19:25.475560+0000","last_clean":"2023-10-16T11:19:25.475560+0000","last_became_active":"2023-10-16T11:19:23.549505+0000","last_became_peered":"2023-10-16T11:19:23.549505+0000","last_unstale":"2023-10-16T11:19:25.475560+0000","last_undegraded":"2023-10-16T11:19:25.475560+0000","last_fullsized":"2023-10-16T11:19:25.475560+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:20.931218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645936+0000","last_change":"2023-10-16T11:19:23.548276+0000","last_active":"2023-10-16T11:19:26.645936+0000","last_peered":"2023-10-16T11:19:26.645936+0000","last_clean":"2023-10-16T11:19:26.645936+0000","last_became_active":"2023-10-16T11:19:23.547177+0000","last_became_peered":"2023-10-16T11:19:23.547177+0000","last_unstale":"2023-10-16T11:19:26.645936+0000","last_undegraded":"2023-10-16T11:19:26.645936+0000","last_fullsized":"2023-10-16T11:19:26.645936+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475381+0000","last_change":"2023-10-16T11:19:23.563754+0000","last_active":"2023-10-16T11:19:25.475381+0000","last_peered":"2023-10-16T11:19:25.475381+0000","last_clean":"2023-10-16T11:19:25.475381+0000","last_became_active":"2023-10-16T11:19:23.563487+0000","last_became_peered":"2023-10-16T11:19:23.563487+0000","last_unstale":"2023-10-16T11:19:25.475381+0000","last_undegraded":"2023-10-16T11:19:25.475381+0000","last_fullsized":"2023-10-16T11:19:25.475381+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:13.814299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645632+0000","last_change":"2023-10-16T11:19:23.551280+0000","last_active":"2023-10-16T11:19:26.645632+0000","last_peered":"2023-10-16T11:19:26.645632+0000","last_clean":"2023-10-16T11:19:26.645632+0000","last_became_active":"2023-10-16T11:19:23.551018+0000","last_became_peered":"2023-10-16T11:19:23.551018+0000","last_unstale":"2023-10-16T11:19:26.645632+0000","last_undegraded":"2023-10-16T11:19:26.645632+0000","last_fullsized":"2023-10-16T11:19:26.645632+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:46.256661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645457+0000","last_change":"2023-10-16T11:19:23.559938+0000","last_active":"2023-10-16T11:19:26.645457+0000","last_peered":"2023-10-16T11:19:26.645457+0000","last_clean":"2023-10-16T11:19:26.645457+0000","last_became_active":"2023-10-16T11:19:23.559569+0000","last_became_peered":"2023-10-16T11:19:23.559569+0000","last_unstale":"2023-10-16T11:19:26.645457+0000","last_undegraded":"2023-10-16T11:19:26.645457+0000","last_fullsized":"2023-10-16T11:19:26.645457+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646311+0000","last_change":"2023-10-16T11:19:23.560050+0000","last_active":"2023-10-16T11:19:26.646311+0000","last_peered":"2023-10-16T11:19:26.646311+0000","last_clean":"2023-10-16T11:19:26.646311+0000","last_became_active":"2023-10-16T11:19:23.559558+0000","last_became_peered":"2023-10-16T11:19:23.559558+0000","last_unstale":"2023-10-16T11:19:26.646311+0000","last_undegraded":"2023-10-16T11:19:26.646311+0000","last_fullsized":"2023-10-16T11:19:26.646311+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646446+0000","last_change":"2023-10-16T11:19:23.560016+0000","last_active":"2023-10-16T11:19:26.646446+0000","last_peered":"2023-10-16T11:19:26.646446+0000","last_clean":"2023-10-16T11:19:26.646446+0000","last_became_active":"2023-10-16T11:19:23.559619+0000","last_became_peered":"2023-10-16T11:19:23.559619+0000","last_unstale":"2023-10-16T11:19:26.646446+0000","last_undegraded":"2023-10-16T11:19:26.646446+0000","last_fullsized":"2023-10-16T11:19:26.646446+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:04.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646475+0000","last_change":"2023-10-16T11:19:23.564191+0000","last_active":"2023-10-16T11:19:26.646475+0000","last_peered":"2023-10-16T11:19:26.646475+0000","last_clean":"2023-10-16T11:19:26.646475+0000","last_became_active":"2023-10-16T11:19:23.563291+0000","last_became_peered":"2023-10-16T11:19:23.563291+0000","last_unstale":"2023-10-16T11:19:26.646475+0000","last_undegraded":"2023-10-16T11:19:26.646475+0000","last_fullsized":"2023-10-16T11:19:26.646475+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475548+0000","last_change":"2023-10-16T11:19:23.546735+0000","last_active":"2023-10-16T11:19:25.475548+0000","last_peered":"2023-10-16T11:19:25.475548+0000","last_clean":"2023-10-16T11:19:25.475548+0000","last_became_active":"2023-10-16T11:19:23.546404+0000","last_became_peered":"2023-10-16T11:19:23.546404+0000","last_unstale":"2023-10-16T11:19:25.475548+0000","last_undegraded":"2023-10-16T11:19:25.475548+0000","last_fullsized":"2023-10-16T11:19:25.475548+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:29.845437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478954+0000","last_change":"2023-10-16T11:19:23.552080+0000","last_active":"2023-10-16T11:19:25.478954+0000","last_peered":"2023-10-16T11:19:25.478954+0000","last_clean":"2023-10-16T11:19:25.478954+0000","last_became_active":"2023-10-16T11:19:23.551779+0000","last_became_peered":"2023-10-16T11:19:23.551779+0000","last_unstale":"2023-10-16T11:19:25.478954+0000","last_undegraded":"2023-10-16T11:19:25.478954+0000","last_fullsized":"2023-10-16T11:19:25.478954+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545485+0000","last_change":"2023-10-16T11:19:23.550227+0000","last_active":"2023-10-16T11:19:25.545485+0000","last_peered":"2023-10-16T11:19:25.545485+0000","last_clean":"2023-10-16T11:19:25.545485+0000","last_became_active":"2023-10-16T11:19:23.549938+0000","last_became_peered":"2023-10-16T11:19:23.549938+0000","last_unstale":"2023-10-16T11:19:25.545485+0000","last_undegraded":"2023-10-16T11:19:25.545485+0000","last_fullsized":"2023-10-16T11:19:25.545485+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:13.268771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645456+0000","last_change":"2023-10-16T11:19:23.547259+0000","last_active":"2023-10-16T11:19:26.645456+0000","last_peered":"2023-10-16T11:19:26.645456+0000","last_clean":"2023-10-16T11:19:26.645456+0000","last_became_active":"2023-10-16T11:19:23.546957+0000","last_became_peered":"2023-10-16T11:19:23.546957+0000","last_unstale":"2023-10-16T11:19:26.645456+0000","last_undegraded":"2023-10-16T11:19:26.645456+0000","last_fullsized":"2023-10-16T11:19:26.645456+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545598+0000","last_change":"2023-10-16T11:19:23.550179+0000","last_active":"2023-10-16T11:19:25.545598+0000","last_peered":"2023-10-16T11:19:25.545598+0000","last_clean":"2023-10-16T11:19:25.545598+0000","last_became_active":"2023-10-16T11:19:23.549811+0000","last_became_peered":"2023-10-16T11:19:23.549811+0000","last_unstale":"2023-10-16T11:19:25.545598+0000","last_undegraded":"2023-10-16T11:19:25.545598+0000","last_fullsized":"2023-10-16T11:19:25.545598+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:58.221233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545931+0000","last_change":"2023-10-16T11:19:23.541856+0000","last_active":"2023-10-16T11:19:25.545931+0000","last_peered":"2023-10-16T11:19:25.545931+0000","last_clean":"2023-10-16T11:19:25.545931+0000","last_became_active":"2023-10-16T11:19:23.541549+0000","last_became_peered":"2023-10-16T11:19:23.541549+0000","last_unstale":"2023-10-16T11:19:25.545931+0000","last_undegraded":"2023-10-16T11:19:25.545931+0000","last_fullsized":"2023-10-16T11:19:25.545931+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:07.709989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479082+0000","last_change":"2023-10-16T11:19:23.547136+0000","last_active":"2023-10-16T11:19:25.479082+0000","last_peered":"2023-10-16T11:19:25.479082+0000","last_clean":"2023-10-16T11:19:25.479082+0000","last_became_active":"2023-10-16T11:19:23.546834+0000","last_became_peered":"2023-10-16T11:19:23.546834+0000","last_unstale":"2023-10-16T11:19:25.479082+0000","last_undegraded":"2023-10-16T11:19:25.479082+0000","last_fullsized":"2023-10-16T11:19:25.479082+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:45.683847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646548+0000","last_change":"2023-10-16T11:19:23.552160+0000","last_active":"2023-10-16T11:19:26.646548+0000","last_peered":"2023-10-16T11:19:26.646548+0000","last_clean":"2023-10-16T11:19:26.646548+0000","last_became_active":"2023-10-16T11:19:23.551805+0000","last_became_peered":"2023-10-16T11:19:23.551805+0000","last_unstale":"2023-10-16T11:19:26.646548+0000","last_undegraded":"2023-10-16T11:19:26.646548+0000","last_fullsized":"2023-10-16T11:19:26.646548+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478933+0000","last_change":"2023-10-16T11:19:23.552427+0000","last_active":"2023-10-16T11:19:25.478933+0000","last_peered":"2023-10-16T11:19:25.478933+0000","last_clean":"2023-10-16T11:19:25.478933+0000","last_became_active":"2023-10-16T11:19:23.552156+0000","last_became_peered":"2023-10-16T11:19:23.552156+0000","last_unstale":"2023-10-16T11:19:25.478933+0000","last_undegraded":"2023-10-16T11:19:25.478933+0000","last_fullsized":"2023-10-16T11:19:25.478933+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645961+0000","last_change":"2023-10-16T11:19:23.547472+0000","last_active":"2023-10-16T11:19:26.645961+0000","last_peered":"2023-10-16T11:19:26.645961+0000","last_clean":"2023-10-16T11:19:26.645961+0000","last_became_active":"2023-10-16T11:19:23.547148+0000","last_became_peered":"2023-10-16T11:19:23.547148+0000","last_unstale":"2023-10-16T11:19:26.645961+0000","last_undegraded":"2023-10-16T11:19:26.645961+0000","last_fullsized":"2023-10-16T11:19:26.645961+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3657,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543281,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3325952,"data_stored":1988414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.27800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42399999999999999,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69599999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.57799999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.79900000000000004,"15min":0.86299999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.307},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.002},"last":0.71199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33100000000000002},"max":{"1min":8.1950000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54400000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.504},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.871},"last":0.52800000000000002}]}]},{"osd":6,"up_from":14,"seq":60129543280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2723840,"data_stored":1395769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.875},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.51200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.497}]},{"osd":1,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55900000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.752,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.32100000000000001},"max":{"1min":0.79500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.91100000000000003},"last":0.63600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129543280,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60212,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311628,"statfs":{"total":96636764160,"available":96575107072,"internally_reserved":0,"allocated":2727936,"data_stored":1395243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.224},"max":{"1min":0.60199999999999998,"5min":0.629,"15min":0.68700000000000006},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.434},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.62,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.26300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.55300000000000005,"5min":0.58699999999999997,"15min":0.98899999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.34899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.78300000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.33400000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.0289999999999999,"15min":1.097},"last":0.55700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":1.147},"last":0.71699999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.88,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":0.98899999999999999,"5min":1.004,"15min":1.0600000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46300000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.1060000000000001},"last":0.621}]}]},{"osd":0,"up_from":14,"seq":60129543281,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60788,"kb_used_data":3240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311052,"statfs":{"total":96636764160,"available":96574517248,"internally_reserved":0,"allocated":3317760,"data_stored":1985645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.46400000000000002,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.27200000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58799999999999997,"5min":0.63700000000000001,"15min":0.70299999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.32000000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.68300000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.42599999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.99199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":1.008},"last":0.46200000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":1.004,"15min":1.008},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59599999999999997}]}]},{"osd":2,"up_from":14,"seq":60129543280,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60204,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311636,"statfs":{"total":96636764160,"available":96575115264,"internally_reserved":0,"allocated":2719744,"data_stored":1395187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":3.2229999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.63},"last":3.2509999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":3.5499999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":3.7440000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.623,"15min":0.65500000000000003},"last":3.2959999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.81200000000000006},"last":3.4929999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":3.6880000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.97599999999999998},"last":3.593}]},{"osd":5,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":3.6779999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":3.7050000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":1.052,"15min":1.052},"last":3.6549999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":3.633}]},{"osd":7,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":3.7200000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68700000000000006,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":3.7320000000000002}]}]},{"osd":3,"up_from":14,"seq":60129543281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2732032,"data_stored":1395369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.19},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.36799999999999999,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":0.60699999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.253}]},{"osd":2,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.28899999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":1.034,"15min":1.0740000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.5}]},{"osd":5,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.0489999999999999},"last":0.74199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60224,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311616,"statfs":{"total":96636764160,"available":96575094784,"internally_reserved":0,"allocated":2740224,"data_stored":1398150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.628},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.66900000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.627},"min":{"1min":0.438,"5min":0.40400000000000003,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.879},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.69499999999999995,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.89500000000000002},"last":0.51400000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.60799999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":0.875,"15min":0.875},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.55000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129543281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42160,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329680,"statfs":{"total":96636764160,"available":96593592320,"internally_reserved":0,"allocated":2723840,"data_stored":1395769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.501},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":1.873,"15min":1.873},"last":0.55300000000000005}]},{"osd":2,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.53700000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":1.821,"15min":1.821},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52400000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":1.919,"15min":1.919},"last":0.34799999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.873},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":2.02,"15min":2.02},"last":0.58399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:19:41.132 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:19:41.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.280+0000 7ff097569700 1 Processor -- start 2023-10-16T11:19:41.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.280+0000 7ff097569700 1 -- start start 2023-10-16T11:19:41.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff097569700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff09015dd00 0x7ff090071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff097569700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff090071870 0x7ff0901574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff097569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff09015e0e0 0x7ff090157a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff097569700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff090157f70 con 0x7ff09015dd00 2023-10-16T11:19:41.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff097569700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff090071df0 con 0x7ff090071870 2023-10-16T11:19:41.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff097569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900720d0 con 0x7ff09015e0e0 2023-10-16T11:19:41.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff095305700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff09015dd00 0x7ff090071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff095305700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff09015dd00 0x7ff090071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55856/0 (socket says 172.21.15.181:55856) 2023-10-16T11:19:41.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.281+0000 7ff095305700 1 -- 172.21.15.181:0/2141103076 learned_addr learned my addr 172.21.15.181:0/2141103076 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:41.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff095b06700 1 --2- 172.21.15.181:0/2141103076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff09015e0e0 0x7ff090157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff095305700 1 -- 172.21.15.181:0/2141103076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff09015e0e0 msgr2=0x7ff090157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:41.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff095305700 1 --2- 172.21.15.181:0/2141103076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff09015e0e0 0x7ff090157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff095305700 1 -- 172.21.15.181:0/2141103076 >> v2:172.21.15.190:3300/0 conn(0x7ff090071870 msgr2=0x7ff0901574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:41.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff094b04700 1 --2- 172.21.15.181:0/2141103076 >> v2:172.21.15.190:3300/0 conn(0x7ff090071870 0x7ff0901574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff095305700 1 --2- 172.21.15.181:0/2141103076 >> v2:172.21.15.190:3300/0 conn(0x7ff090071870 0x7ff0901574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff095305700 1 -- 172.21.15.181:0/2141103076 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900723b0 con 0x7ff09015dd00 2023-10-16T11:19:41.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff094b04700 1 --2- 172.21.15.181:0/2141103076 >> v2:172.21.15.190:3300/0 conn(0x7ff090071870 0x7ff0901574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:41.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff095305700 1 --2- 172.21.15.181:0/2141103076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff090071330 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7ff09006ebd0 tx=0x7ff080009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6cb5408e2ed11d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:41.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff087fff700 1 -- 172.21.15.181:0/2141103076 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000b870 con 0x7ff09015dd00 2023-10-16T11:19:41.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff087fff700 1 -- 172.21.15.181:0/2141103076 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08000b9d0 con 0x7ff09015dd00 2023-10-16T11:19:41.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.282+0000 7ff087fff700 1 -- 172.21.15.181:0/2141103076 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000bd60 con 0x7ff09015dd00 2023-10-16T11:19:41.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 -- 172.21.15.181:0/2141103076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 msgr2=0x7ff090071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 --2- 172.21.15.181:0/2141103076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff090071330 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7ff09006ebd0 tx=0x7ff080009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 -- 172.21.15.181:0/2141103076 shutdown_connections 2023-10-16T11:19:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 --2- 172.21.15.181:0/2141103076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff090071330 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 --2- 172.21.15.181:0/2141103076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff09015e0e0 0x7ff090157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 --2- 172.21.15.181:0/2141103076 >> v2:172.21.15.190:3300/0 conn(0x7ff090071870 0x7ff0901574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 -- 172.21.15.181:0/2141103076 >> 172.21.15.181:0/2141103076 conn(0x7ff09006aed0 msgr2=0x7ff0900b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:41.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 -- 172.21.15.181:0/2141103076 shutdown_connections 2023-10-16T11:19:41.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.283+0000 7ff097569700 1 -- 172.21.15.181:0/2141103076 wait complete. 2023-10-16T11:19:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.284+0000 7ff097569700 1 Processor -- start 2023-10-16T11:19:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.284+0000 7ff097569700 1 -- start start 2023-10-16T11:19:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff097569700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff090071870 0x7ff090152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff097569700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff0900c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff097569700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff09015e0e0 0x7ff090140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff097569700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090153060 con 0x7ff09015dd00 2023-10-16T11:19:41.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff097569700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900c50f0 con 0x7ff09015e0e0 2023-10-16T11:19:41.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff097569700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900c53d0 con 0x7ff090071870 2023-10-16T11:19:41.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff0900c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff0900c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55864/0 (socket says 172.21.15.181:55864) 2023-10-16T11:19:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 -- 172.21.15.181:0/2613307405 learned_addr learned my addr 172.21.15.181:0/2613307405 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff095305700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff090071870 0x7ff090152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff095b06700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff09015e0e0 0x7ff090140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 -- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff090071870 msgr2=0x7ff090152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff090071870 0x7ff090152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 -- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff09015e0e0 msgr2=0x7ff090140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff09015e0e0 0x7ff090140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 -- 172.21.15.181:0/2613307405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080009550 con 0x7ff09015dd00 2023-10-16T11:19:41.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff095305700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff090071870 0x7ff090152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:41.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff094b04700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff0900c3110 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c740 tx=0x7ff08c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:41.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff085ffb700 1 -- 172.21.15.181:0/2613307405 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011800 con 0x7ff09015dd00 2023-10-16T11:19:41.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff085ffb700 1 -- 172.21.15.181:0/2613307405 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c011960 con 0x7ff09015dd00 2023-10-16T11:19:41.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0900c56b0 con 0x7ff09015dd00 2023-10-16T11:19:41.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.285+0000 7ff085ffb700 1 -- 172.21.15.181:0/2613307405 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011cd0 con 0x7ff09015dd00 2023-10-16T11:19:41.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.286+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0900c5c70 con 0x7ff09015dd00 2023-10-16T11:19:41.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.286+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0900c5e90 con 0x7ff09015dd00 2023-10-16T11:19:41.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.287+0000 7ff085ffb700 1 -- 172.21.15.181:0/2613307405 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff08c021350 con 0x7ff09015dd00 2023-10-16T11:19:41.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.287+0000 7ff085ffb700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff07406b020 0x7ff07406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.287+0000 7ff085ffb700 1 -- 172.21.15.181:0/2613307405 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7ff08c089cd0 con 0x7ff09015dd00 2023-10-16T11:19:41.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.287+0000 7ff095305700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff07406b020 0x7ff07406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.288+0000 7ff095305700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff07406b020 0x7ff07406d4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff080009130 tx=0x7ff080008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:41.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.292+0000 7ff085ffb700 1 -- 172.21.15.181:0/2613307405 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff08c0545e0 con 0x7ff09015dd00 2023-10-16T11:19:41.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.479+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff090072970 con 0x7ff07406b020 2023-10-16T11:19:41.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.493+0000 7ff085ffb700 1 -- 172.21.15.181:0/2613307405 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+516853 (secure 0 0 0) 0x7ff08000bbc0 con 0x7ff07406b020 2023-10-16T11:19:41.500 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:41.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff07406b020 msgr2=0x7ff07406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff07406b020 0x7ff07406d4e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff080009130 tx=0x7ff080008030 comp rx=0 tx=0).stop 2023-10-16T11:19:41.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 msgr2=0x7ff0900c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff0900c3110 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c740 tx=0x7ff08c00cb00 comp rx=0 tx=0).stop 2023-10-16T11:19:41.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 shutdown_connections 2023-10-16T11:19:41.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7ff07406b020 0x7ff07406d4e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff090071870 0x7ff090152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff09015e0e0 0x7ff090140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 --2- 172.21.15.181:0/2613307405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff09015dd00 0x7ff0900c3110 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 >> 172.21.15.181:0/2613307405 conn(0x7ff09006aed0 msgr2=0x7ff0901555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:41.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 shutdown_connections 2023-10-16T11:19:41.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.497+0000 7ff097569700 1 -- 172.21.15.181:0/2613307405 wait complete. 2023-10-16T11:19:41.507 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:19:41.511 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-10-16T11:19:39.484465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594025,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":626,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":389,"ondisk_log_size":389,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421800,"kb_used_data":22472,"kb_used_omap":12,"kb_used_meta":399219,"kb_avail":754552920,"statfs":{"total":773094113280,"available":772662190080,"internally_reserved":0,"allocated":23011328,"data_stored":12349546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":408800824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.883080"},"pg_stats":[{"pgid":"1.0","version":"206'348","reported_seq":1210,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:24.540056+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:19:24.540056+0000","last_peered":"2023-10-16T11:19:24.540056+0000","last_clean":"2023-10-16T11:19:24.540056+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:19:24.540056+0000","last_undegraded":"2023-10-16T11:19:24.540056+0000","last_fullsized":"2023-10-16T11:19:24.540056+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646510+0000","last_change":"2023-10-16T11:19:22.536415+0000","last_active":"2023-10-16T11:19:26.646510+0000","last_peered":"2023-10-16T11:19:26.646510+0000","last_clean":"2023-10-16T11:19:26.646510+0000","last_became_active":"2023-10-16T11:19:22.536117+0000","last_became_peered":"2023-10-16T11:19:22.536117+0000","last_unstale":"2023-10-16T11:19:26.646510+0000","last_undegraded":"2023-10-16T11:19:26.646510+0000","last_fullsized":"2023-10-16T11:19:26.646510+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:07.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645725+0000","last_change":"2023-10-16T11:19:23.551428+0000","last_active":"2023-10-16T11:19:26.645725+0000","last_peered":"2023-10-16T11:19:26.645725+0000","last_clean":"2023-10-16T11:19:26.645725+0000","last_became_active":"2023-10-16T11:19:23.551083+0000","last_became_peered":"2023-10-16T11:19:23.551083+0000","last_unstale":"2023-10-16T11:19:26.645725+0000","last_undegraded":"2023-10-16T11:19:26.645725+0000","last_fullsized":"2023-10-16T11:19:26.645725+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:52.800093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607616+0000","last_change":"2023-10-16T11:19:34.607616+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607616+0000","last_undegraded":"2023-10-16T11:19:34.607616+0000","last_fullsized":"2023-10-16T11:19:34.607616+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646184+0000","last_change":"2023-10-16T11:19:22.537241+0000","last_active":"2023-10-16T11:19:26.646184+0000","last_peered":"2023-10-16T11:19:26.646184+0000","last_clean":"2023-10-16T11:19:26.646184+0000","last_became_active":"2023-10-16T11:19:22.536947+0000","last_became_peered":"2023-10-16T11:19:22.536947+0000","last_unstale":"2023-10-16T11:19:26.646184+0000","last_undegraded":"2023-10-16T11:19:26.646184+0000","last_fullsized":"2023-10-16T11:19:26.646184+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478816+0000","last_change":"2023-10-16T11:19:23.563089+0000","last_active":"2023-10-16T11:19:25.478816+0000","last_peered":"2023-10-16T11:19:25.478816+0000","last_clean":"2023-10-16T11:19:25.478816+0000","last_became_active":"2023-10-16T11:19:23.562759+0000","last_became_peered":"2023-10-16T11:19:23.562759+0000","last_unstale":"2023-10-16T11:19:25.478816+0000","last_undegraded":"2023-10-16T11:19:25.478816+0000","last_fullsized":"2023-10-16T11:19:25.478816+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.602978+0000","last_change":"2023-10-16T11:19:34.602978+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.602978+0000","last_undegraded":"2023-10-16T11:19:34.602978+0000","last_fullsized":"2023-10-16T11:19:34.602978+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479067+0000","last_change":"2023-10-16T11:19:22.539000+0000","last_active":"2023-10-16T11:19:25.479067+0000","last_peered":"2023-10-16T11:19:25.479067+0000","last_clean":"2023-10-16T11:19:25.479067+0000","last_became_active":"2023-10-16T11:19:22.538728+0000","last_became_peered":"2023-10-16T11:19:22.538728+0000","last_unstale":"2023-10-16T11:19:25.479067+0000","last_undegraded":"2023-10-16T11:19:25.479067+0000","last_fullsized":"2023-10-16T11:19:25.479067+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:03.944394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645984+0000","last_change":"2023-10-16T11:19:23.563504+0000","last_active":"2023-10-16T11:19:26.645984+0000","last_peered":"2023-10-16T11:19:26.645984+0000","last_clean":"2023-10-16T11:19:26.645984+0000","last_became_active":"2023-10-16T11:19:23.563243+0000","last_became_peered":"2023-10-16T11:19:23.563243+0000","last_unstale":"2023-10-16T11:19:26.645984+0000","last_undegraded":"2023-10-16T11:19:26.645984+0000","last_fullsized":"2023-10-16T11:19:26.645984+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607783+0000","last_change":"2023-10-16T11:19:34.607783+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607783+0000","last_undegraded":"2023-10-16T11:19:34.607783+0000","last_fullsized":"2023-10-16T11:19:34.607783+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545888+0000","last_change":"2023-10-16T11:19:22.544852+0000","last_active":"2023-10-16T11:19:25.545888+0000","last_peered":"2023-10-16T11:19:25.545888+0000","last_clean":"2023-10-16T11:19:25.545888+0000","last_became_active":"2023-10-16T11:19:22.544565+0000","last_became_peered":"2023-10-16T11:19:22.544565+0000","last_unstale":"2023-10-16T11:19:25.545888+0000","last_undegraded":"2023-10-16T11:19:25.545888+0000","last_fullsized":"2023-10-16T11:19:25.545888+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:07.144566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646564+0000","last_change":"2023-10-16T11:19:23.558505+0000","last_active":"2023-10-16T11:19:26.646564+0000","last_peered":"2023-10-16T11:19:26.646564+0000","last_clean":"2023-10-16T11:19:26.646564+0000","last_became_active":"2023-10-16T11:19:23.558188+0000","last_became_peered":"2023-10-16T11:19:23.558188+0000","last_unstale":"2023-10-16T11:19:26.646564+0000","last_undegraded":"2023-10-16T11:19:26.646564+0000","last_fullsized":"2023-10-16T11:19:26.646564+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604846+0000","last_change":"2023-10-16T11:19:34.604846+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604846+0000","last_undegraded":"2023-10-16T11:19:34.604846+0000","last_fullsized":"2023-10-16T11:19:34.604846+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479062+0000","last_change":"2023-10-16T11:19:22.543881+0000","last_active":"2023-10-16T11:19:25.479062+0000","last_peered":"2023-10-16T11:19:25.479062+0000","last_clean":"2023-10-16T11:19:25.479062+0000","last_became_active":"2023-10-16T11:19:22.543647+0000","last_became_peered":"2023-10-16T11:19:22.543647+0000","last_unstale":"2023-10-16T11:19:25.479062+0000","last_undegraded":"2023-10-16T11:19:25.479062+0000","last_fullsized":"2023-10-16T11:19:25.479062+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:41.755970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645587+0000","last_change":"2023-10-16T11:19:23.546380+0000","last_active":"2023-10-16T11:19:26.645587+0000","last_peered":"2023-10-16T11:19:26.645587+0000","last_clean":"2023-10-16T11:19:26.645587+0000","last_became_active":"2023-10-16T11:19:23.546098+0000","last_became_peered":"2023-10-16T11:19:23.546098+0000","last_unstale":"2023-10-16T11:19:26.645587+0000","last_undegraded":"2023-10-16T11:19:26.645587+0000","last_fullsized":"2023-10-16T11:19:26.645587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:06.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.603595+0000","last_change":"2023-10-16T11:19:34.603595+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.603595+0000","last_undegraded":"2023-10-16T11:19:34.603595+0000","last_fullsized":"2023-10-16T11:19:34.603595+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646437+0000","last_change":"2023-10-16T11:19:22.537543+0000","last_active":"2023-10-16T11:19:26.646437+0000","last_peered":"2023-10-16T11:19:26.646437+0000","last_clean":"2023-10-16T11:19:26.646437+0000","last_became_active":"2023-10-16T11:19:22.537050+0000","last_became_peered":"2023-10-16T11:19:22.537050+0000","last_unstale":"2023-10-16T11:19:26.646437+0000","last_undegraded":"2023-10-16T11:19:26.646437+0000","last_fullsized":"2023-10-16T11:19:26.646437+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:03.218818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479172+0000","last_change":"2023-10-16T11:19:23.546472+0000","last_active":"2023-10-16T11:19:25.479172+0000","last_peered":"2023-10-16T11:19:25.479172+0000","last_clean":"2023-10-16T11:19:25.479172+0000","last_became_active":"2023-10-16T11:19:23.546160+0000","last_became_peered":"2023-10-16T11:19:23.546160+0000","last_unstale":"2023-10-16T11:19:25.479172+0000","last_undegraded":"2023-10-16T11:19:25.479172+0000","last_fullsized":"2023-10-16T11:19:25.479172+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:06.395224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610080+0000","last_change":"2023-10-16T11:19:34.610080+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610080+0000","last_undegraded":"2023-10-16T11:19:34.610080+0000","last_fullsized":"2023-10-16T11:19:34.610080+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545804+0000","last_change":"2023-10-16T11:19:22.543844+0000","last_active":"2023-10-16T11:19:25.545804+0000","last_peered":"2023-10-16T11:19:25.545804+0000","last_clean":"2023-10-16T11:19:25.545804+0000","last_became_active":"2023-10-16T11:19:22.543538+0000","last_became_peered":"2023-10-16T11:19:22.543538+0000","last_unstale":"2023-10-16T11:19:25.545804+0000","last_undegraded":"2023-10-16T11:19:25.545804+0000","last_fullsized":"2023-10-16T11:19:25.545804+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:00.946603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478841+0000","last_change":"2023-10-16T11:19:23.556539+0000","last_active":"2023-10-16T11:19:25.478841+0000","last_peered":"2023-10-16T11:19:25.478841+0000","last_clean":"2023-10-16T11:19:25.478841+0000","last_became_active":"2023-10-16T11:19:23.556209+0000","last_became_peered":"2023-10-16T11:19:23.556209+0000","last_unstale":"2023-10-16T11:19:25.478841+0000","last_undegraded":"2023-10-16T11:19:25.478841+0000","last_fullsized":"2023-10-16T11:19:25.478841+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607610+0000","last_change":"2023-10-16T11:19:34.607610+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607610+0000","last_undegraded":"2023-10-16T11:19:34.607610+0000","last_fullsized":"2023-10-16T11:19:34.607610+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:28.599812+0000","last_change":"2023-10-16T11:19:22.537758+0000","last_active":"2023-10-16T11:19:28.599812+0000","last_peered":"2023-10-16T11:19:28.599812+0000","last_clean":"2023-10-16T11:19:28.599812+0000","last_became_active":"2023-10-16T11:19:22.537201+0000","last_became_peered":"2023-10-16T11:19:22.537201+0000","last_unstale":"2023-10-16T11:19:28.599812+0000","last_undegraded":"2023-10-16T11:19:28.599812+0000","last_fullsized":"2023-10-16T11:19:28.599812+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:26.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646005+0000","last_change":"2023-10-16T11:19:23.564018+0000","last_active":"2023-10-16T11:19:26.646005+0000","last_peered":"2023-10-16T11:19:26.646005+0000","last_clean":"2023-10-16T11:19:26.646005+0000","last_became_active":"2023-10-16T11:19:23.563738+0000","last_became_peered":"2023-10-16T11:19:23.563738+0000","last_unstale":"2023-10-16T11:19:26.646005+0000","last_undegraded":"2023-10-16T11:19:26.646005+0000","last_fullsized":"2023-10-16T11:19:26.646005+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604846+0000","last_change":"2023-10-16T11:19:34.604846+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604846+0000","last_undegraded":"2023-10-16T11:19:34.604846+0000","last_fullsized":"2023-10-16T11:19:34.604846+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646652+0000","last_change":"2023-10-16T11:19:22.537256+0000","last_active":"2023-10-16T11:19:26.646652+0000","last_peered":"2023-10-16T11:19:26.646652+0000","last_clean":"2023-10-16T11:19:26.646652+0000","last_became_active":"2023-10-16T11:19:22.536996+0000","last_became_peered":"2023-10-16T11:19:22.536996+0000","last_unstale":"2023-10-16T11:19:26.646652+0000","last_undegraded":"2023-10-16T11:19:26.646652+0000","last_fullsized":"2023-10-16T11:19:26.646652+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:22.462616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646503+0000","last_change":"2023-10-16T11:19:23.561790+0000","last_active":"2023-10-16T11:19:26.646503+0000","last_peered":"2023-10-16T11:19:26.646503+0000","last_clean":"2023-10-16T11:19:26.646503+0000","last_became_active":"2023-10-16T11:19:23.561531+0000","last_became_peered":"2023-10-16T11:19:23.561531+0000","last_unstale":"2023-10-16T11:19:26.646503+0000","last_undegraded":"2023-10-16T11:19:26.646503+0000","last_fullsized":"2023-10-16T11:19:26.646503+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607676+0000","last_change":"2023-10-16T11:19:34.607676+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607676+0000","last_undegraded":"2023-10-16T11:19:34.607676+0000","last_fullsized":"2023-10-16T11:19:34.607676+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545608+0000","last_change":"2023-10-16T11:19:22.547829+0000","last_active":"2023-10-16T11:19:25.545608+0000","last_peered":"2023-10-16T11:19:25.545608+0000","last_clean":"2023-10-16T11:19:25.545608+0000","last_became_active":"2023-10-16T11:19:22.547563+0000","last_became_peered":"2023-10-16T11:19:22.547563+0000","last_unstale":"2023-10-16T11:19:25.545608+0000","last_undegraded":"2023-10-16T11:19:25.545608+0000","last_fullsized":"2023-10-16T11:19:25.545608+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:15.819032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479044+0000","last_change":"2023-10-16T11:19:23.563941+0000","last_active":"2023-10-16T11:19:25.479044+0000","last_peered":"2023-10-16T11:19:25.479044+0000","last_clean":"2023-10-16T11:19:25.479044+0000","last_became_active":"2023-10-16T11:19:23.563666+0000","last_became_peered":"2023-10-16T11:19:23.563666+0000","last_unstale":"2023-10-16T11:19:25.479044+0000","last_undegraded":"2023-10-16T11:19:25.479044+0000","last_fullsized":"2023-10-16T11:19:25.479044+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:43.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610615+0000","last_change":"2023-10-16T11:19:34.610615+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610615+0000","last_undegraded":"2023-10-16T11:19:34.610615+0000","last_fullsized":"2023-10-16T11:19:34.610615+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645872+0000","last_change":"2023-10-16T11:19:22.532222+0000","last_active":"2023-10-16T11:19:26.645872+0000","last_peered":"2023-10-16T11:19:26.645872+0000","last_clean":"2023-10-16T11:19:26.645872+0000","last_became_active":"2023-10-16T11:19:22.531939+0000","last_became_peered":"2023-10-16T11:19:22.531939+0000","last_unstale":"2023-10-16T11:19:26.645872+0000","last_undegraded":"2023-10-16T11:19:26.645872+0000","last_fullsized":"2023-10-16T11:19:26.645872+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478778+0000","last_change":"2023-10-16T11:19:23.548561+0000","last_active":"2023-10-16T11:19:25.478778+0000","last_peered":"2023-10-16T11:19:25.478778+0000","last_clean":"2023-10-16T11:19:25.478778+0000","last_became_active":"2023-10-16T11:19:23.548310+0000","last_became_peered":"2023-10-16T11:19:23.548310+0000","last_unstale":"2023-10-16T11:19:25.478778+0000","last_undegraded":"2023-10-16T11:19:25.478778+0000","last_fullsized":"2023-10-16T11:19:25.478778+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604887+0000","last_change":"2023-10-16T11:19:34.604887+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604887+0000","last_undegraded":"2023-10-16T11:19:34.604887+0000","last_fullsized":"2023-10-16T11:19:34.604887+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645841+0000","last_change":"2023-10-16T11:19:22.542021+0000","last_active":"2023-10-16T11:19:26.645841+0000","last_peered":"2023-10-16T11:19:26.645841+0000","last_clean":"2023-10-16T11:19:26.645841+0000","last_became_active":"2023-10-16T11:19:22.541718+0000","last_became_peered":"2023-10-16T11:19:22.541718+0000","last_unstale":"2023-10-16T11:19:26.645841+0000","last_undegraded":"2023-10-16T11:19:26.645841+0000","last_fullsized":"2023-10-16T11:19:26.645841+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:12.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646346+0000","last_change":"2023-10-16T11:19:23.554949+0000","last_active":"2023-10-16T11:19:26.646346+0000","last_peered":"2023-10-16T11:19:26.646346+0000","last_clean":"2023-10-16T11:19:26.646346+0000","last_became_active":"2023-10-16T11:19:23.554300+0000","last_became_peered":"2023-10-16T11:19:23.554300+0000","last_unstale":"2023-10-16T11:19:26.646346+0000","last_undegraded":"2023-10-16T11:19:26.646346+0000","last_fullsized":"2023-10-16T11:19:26.646346+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604641+0000","last_change":"2023-10-16T11:19:34.604641+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604641+0000","last_undegraded":"2023-10-16T11:19:34.604641+0000","last_fullsized":"2023-10-16T11:19:34.604641+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475451+0000","last_change":"2023-10-16T11:19:22.536403+0000","last_active":"2023-10-16T11:19:25.475451+0000","last_peered":"2023-10-16T11:19:25.475451+0000","last_clean":"2023-10-16T11:19:25.475451+0000","last_became_active":"2023-10-16T11:19:22.536132+0000","last_became_peered":"2023-10-16T11:19:22.536132+0000","last_unstale":"2023-10-16T11:19:25.475451+0000","last_undegraded":"2023-10-16T11:19:25.475451+0000","last_fullsized":"2023-10-16T11:19:25.475451+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:53.563487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475436+0000","last_change":"2023-10-16T11:19:23.543334+0000","last_active":"2023-10-16T11:19:25.475436+0000","last_peered":"2023-10-16T11:19:25.475436+0000","last_clean":"2023-10-16T11:19:25.475436+0000","last_became_active":"2023-10-16T11:19:23.542527+0000","last_became_peered":"2023-10-16T11:19:23.542527+0000","last_unstale":"2023-10-16T11:19:25.475436+0000","last_undegraded":"2023-10-16T11:19:25.475436+0000","last_fullsized":"2023-10-16T11:19:25.475436+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:59.083108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.602964+0000","last_change":"2023-10-16T11:19:34.602964+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.602964+0000","last_undegraded":"2023-10-16T11:19:34.602964+0000","last_fullsized":"2023-10-16T11:19:34.602964+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646351+0000","last_change":"2023-10-16T11:19:22.545030+0000","last_active":"2023-10-16T11:19:26.646351+0000","last_peered":"2023-10-16T11:19:26.646351+0000","last_clean":"2023-10-16T11:19:26.646351+0000","last_became_active":"2023-10-16T11:19:22.544773+0000","last_became_peered":"2023-10-16T11:19:22.544773+0000","last_unstale":"2023-10-16T11:19:26.646351+0000","last_undegraded":"2023-10-16T11:19:26.646351+0000","last_fullsized":"2023-10-16T11:19:26.646351+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:22.946353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475507+0000","last_change":"2023-10-16T11:19:23.545760+0000","last_active":"2023-10-16T11:19:25.475507+0000","last_peered":"2023-10-16T11:19:25.475507+0000","last_clean":"2023-10-16T11:19:25.475507+0000","last_became_active":"2023-10-16T11:19:23.545499+0000","last_became_peered":"2023-10-16T11:19:23.545499+0000","last_unstale":"2023-10-16T11:19:25.475507+0000","last_undegraded":"2023-10-16T11:19:25.475507+0000","last_fullsized":"2023-10-16T11:19:25.475507+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:32.701590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607606+0000","last_change":"2023-10-16T11:19:34.607606+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607606+0000","last_undegraded":"2023-10-16T11:19:34.607606+0000","last_fullsized":"2023-10-16T11:19:34.607606+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475536+0000","last_change":"2023-10-16T11:19:22.533601+0000","last_active":"2023-10-16T11:19:25.475536+0000","last_peered":"2023-10-16T11:19:25.475536+0000","last_clean":"2023-10-16T11:19:25.475536+0000","last_became_active":"2023-10-16T11:19:22.533374+0000","last_became_peered":"2023-10-16T11:19:22.533374+0000","last_unstale":"2023-10-16T11:19:25.475536+0000","last_undegraded":"2023-10-16T11:19:25.475536+0000","last_fullsized":"2023-10-16T11:19:25.475536+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:52.564361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646427+0000","last_change":"2023-10-16T11:19:23.553745+0000","last_active":"2023-10-16T11:19:26.646427+0000","last_peered":"2023-10-16T11:19:26.646427+0000","last_clean":"2023-10-16T11:19:26.646427+0000","last_became_active":"2023-10-16T11:19:23.553462+0000","last_became_peered":"2023-10-16T11:19:23.553462+0000","last_unstale":"2023-10-16T11:19:26.646427+0000","last_undegraded":"2023-10-16T11:19:26.646427+0000","last_fullsized":"2023-10-16T11:19:26.646427+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604947+0000","last_change":"2023-10-16T11:19:34.604947+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604947+0000","last_undegraded":"2023-10-16T11:19:34.604947+0000","last_fullsized":"2023-10-16T11:19:34.604947+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478908+0000","last_change":"2023-10-16T11:19:22.539558+0000","last_active":"2023-10-16T11:19:25.478908+0000","last_peered":"2023-10-16T11:19:25.478908+0000","last_clean":"2023-10-16T11:19:25.478908+0000","last_became_active":"2023-10-16T11:19:22.539284+0000","last_became_peered":"2023-10-16T11:19:22.539284+0000","last_unstale":"2023-10-16T11:19:25.478908+0000","last_undegraded":"2023-10-16T11:19:25.478908+0000","last_fullsized":"2023-10-16T11:19:25.478908+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478999+0000","last_change":"2023-10-16T11:19:23.546218+0000","last_active":"2023-10-16T11:19:25.478999+0000","last_peered":"2023-10-16T11:19:25.478999+0000","last_clean":"2023-10-16T11:19:25.478999+0000","last_became_active":"2023-10-16T11:19:23.545896+0000","last_became_peered":"2023-10-16T11:19:23.545896+0000","last_unstale":"2023-10-16T11:19:25.478999+0000","last_undegraded":"2023-10-16T11:19:25.478999+0000","last_fullsized":"2023-10-16T11:19:25.478999+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.605391+0000","last_change":"2023-10-16T11:19:34.605391+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.605391+0000","last_undegraded":"2023-10-16T11:19:34.605391+0000","last_fullsized":"2023-10-16T11:19:34.605391+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545784+0000","last_change":"2023-10-16T11:19:22.545027+0000","last_active":"2023-10-16T11:19:25.545784+0000","last_peered":"2023-10-16T11:19:25.545784+0000","last_clean":"2023-10-16T11:19:25.545784+0000","last_became_active":"2023-10-16T11:19:22.544756+0000","last_became_peered":"2023-10-16T11:19:22.544756+0000","last_unstale":"2023-10-16T11:19:25.545784+0000","last_undegraded":"2023-10-16T11:19:25.545784+0000","last_fullsized":"2023-10-16T11:19:25.545784+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:23.693322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478882+0000","last_change":"2023-10-16T11:19:23.547530+0000","last_active":"2023-10-16T11:19:25.478882+0000","last_peered":"2023-10-16T11:19:25.478882+0000","last_clean":"2023-10-16T11:19:25.478882+0000","last_became_active":"2023-10-16T11:19:23.547255+0000","last_became_peered":"2023-10-16T11:19:23.547255+0000","last_unstale":"2023-10-16T11:19:25.478882+0000","last_undegraded":"2023-10-16T11:19:25.478882+0000","last_fullsized":"2023-10-16T11:19:25.478882+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.608216+0000","last_change":"2023-10-16T11:19:34.608216+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.608216+0000","last_undegraded":"2023-10-16T11:19:34.608216+0000","last_fullsized":"2023-10-16T11:19:34.608216+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":21,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646100+0000","last_change":"2023-10-16T11:19:22.534093+0000","last_active":"2023-10-16T11:19:26.646100+0000","last_peered":"2023-10-16T11:19:26.646100+0000","last_clean":"2023-10-16T11:19:26.646100+0000","last_became_active":"2023-10-16T11:19:22.533637+0000","last_became_peered":"2023-10-16T11:19:22.533637+0000","last_unstale":"2023-10-16T11:19:26.646100+0000","last_undegraded":"2023-10-16T11:19:26.646100+0000","last_fullsized":"2023-10-16T11:19:26.646100+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645780+0000","last_change":"2023-10-16T11:19:23.552950+0000","last_active":"2023-10-16T11:19:26.645780+0000","last_peered":"2023-10-16T11:19:26.645780+0000","last_clean":"2023-10-16T11:19:26.645780+0000","last_became_active":"2023-10-16T11:19:23.552686+0000","last_became_peered":"2023-10-16T11:19:23.552686+0000","last_unstale":"2023-10-16T11:19:26.645780+0000","last_undegraded":"2023-10-16T11:19:26.645780+0000","last_fullsized":"2023-10-16T11:19:26.645780+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610145+0000","last_change":"2023-10-16T11:19:34.610145+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610145+0000","last_undegraded":"2023-10-16T11:19:34.610145+0000","last_fullsized":"2023-10-16T11:19:34.610145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475421+0000","last_change":"2023-10-16T11:19:22.538008+0000","last_active":"2023-10-16T11:19:25.475421+0000","last_peered":"2023-10-16T11:19:25.475421+0000","last_clean":"2023-10-16T11:19:25.475421+0000","last_became_active":"2023-10-16T11:19:22.537752+0000","last_became_peered":"2023-10-16T11:19:22.537752+0000","last_unstale":"2023-10-16T11:19:25.475421+0000","last_undegraded":"2023-10-16T11:19:25.475421+0000","last_fullsized":"2023-10-16T11:19:25.475421+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:20.859898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645905+0000","last_change":"2023-10-16T11:19:23.563603+0000","last_active":"2023-10-16T11:19:26.645905+0000","last_peered":"2023-10-16T11:19:26.645905+0000","last_clean":"2023-10-16T11:19:26.645905+0000","last_became_active":"2023-10-16T11:19:23.563305+0000","last_became_peered":"2023-10-16T11:19:23.563305+0000","last_unstale":"2023-10-16T11:19:26.645905+0000","last_undegraded":"2023-10-16T11:19:26.645905+0000","last_fullsized":"2023-10-16T11:19:26.645905+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.602965+0000","last_change":"2023-10-16T11:19:34.602965+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.602965+0000","last_undegraded":"2023-10-16T11:19:34.602965+0000","last_fullsized":"2023-10-16T11:19:34.602965+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479114+0000","last_change":"2023-10-16T11:19:22.538147+0000","last_active":"2023-10-16T11:19:25.479114+0000","last_peered":"2023-10-16T11:19:25.479114+0000","last_clean":"2023-10-16T11:19:25.479114+0000","last_became_active":"2023-10-16T11:19:22.537850+0000","last_became_peered":"2023-10-16T11:19:22.537850+0000","last_unstale":"2023-10-16T11:19:25.479114+0000","last_undegraded":"2023-10-16T11:19:25.479114+0000","last_fullsized":"2023-10-16T11:19:25.479114+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:44.794069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475465+0000","last_change":"2023-10-16T11:19:23.552779+0000","last_active":"2023-10-16T11:19:25.475465+0000","last_peered":"2023-10-16T11:19:25.475465+0000","last_clean":"2023-10-16T11:19:25.475465+0000","last_became_active":"2023-10-16T11:19:23.552491+0000","last_became_peered":"2023-10-16T11:19:23.552491+0000","last_unstale":"2023-10-16T11:19:25.475465+0000","last_undegraded":"2023-10-16T11:19:25.475465+0000","last_fullsized":"2023-10-16T11:19:25.475465+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:02.969587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.605301+0000","last_change":"2023-10-16T11:19:34.605301+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.605301+0000","last_undegraded":"2023-10-16T11:19:34.605301+0000","last_fullsized":"2023-10-16T11:19:34.605301+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479149+0000","last_change":"2023-10-16T11:19:22.544032+0000","last_active":"2023-10-16T11:19:25.479149+0000","last_peered":"2023-10-16T11:19:25.479149+0000","last_clean":"2023-10-16T11:19:25.479149+0000","last_became_active":"2023-10-16T11:19:22.543713+0000","last_became_peered":"2023-10-16T11:19:22.543713+0000","last_unstale":"2023-10-16T11:19:25.479149+0000","last_undegraded":"2023-10-16T11:19:25.479149+0000","last_fullsized":"2023-10-16T11:19:25.479149+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:01.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545729+0000","last_change":"2023-10-16T11:19:23.550398+0000","last_active":"2023-10-16T11:19:25.545729+0000","last_peered":"2023-10-16T11:19:25.545729+0000","last_clean":"2023-10-16T11:19:25.545729+0000","last_became_active":"2023-10-16T11:19:23.545907+0000","last_became_peered":"2023-10-16T11:19:23.545907+0000","last_unstale":"2023-10-16T11:19:25.545729+0000","last_undegraded":"2023-10-16T11:19:25.545729+0000","last_fullsized":"2023-10-16T11:19:25.545729+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:05.456349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607838+0000","last_change":"2023-10-16T11:19:34.607838+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607838+0000","last_undegraded":"2023-10-16T11:19:34.607838+0000","last_fullsized":"2023-10-16T11:19:34.607838+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"216'8","reported_seq":21,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:33.599726+0000","last_change":"2023-10-16T11:19:22.532339+0000","last_active":"2023-10-16T11:19:33.599726+0000","last_peered":"2023-10-16T11:19:33.599726+0000","last_clean":"2023-10-16T11:19:33.599726+0000","last_became_active":"2023-10-16T11:19:22.532053+0000","last_became_peered":"2023-10-16T11:19:22.532053+0000","last_unstale":"2023-10-16T11:19:33.599726+0000","last_undegraded":"2023-10-16T11:19:33.599726+0000","last_fullsized":"2023-10-16T11:19:33.599726+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:21.190168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2859,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645752+0000","last_change":"2023-10-16T11:19:23.548103+0000","last_active":"2023-10-16T11:19:26.645752+0000","last_peered":"2023-10-16T11:19:26.645752+0000","last_clean":"2023-10-16T11:19:26.645752+0000","last_became_active":"2023-10-16T11:19:23.547665+0000","last_became_peered":"2023-10-16T11:19:23.547665+0000","last_unstale":"2023-10-16T11:19:26.645752+0000","last_undegraded":"2023-10-16T11:19:26.645752+0000","last_fullsized":"2023-10-16T11:19:26.645752+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610953+0000","last_change":"2023-10-16T11:19:34.610953+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610953+0000","last_undegraded":"2023-10-16T11:19:34.610953+0000","last_fullsized":"2023-10-16T11:19:34.610953+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":16,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475523+0000","last_change":"2023-10-16T11:19:22.544432+0000","last_active":"2023-10-16T11:19:25.475523+0000","last_peered":"2023-10-16T11:19:25.475523+0000","last_clean":"2023-10-16T11:19:25.475523+0000","last_became_active":"2023-10-16T11:19:22.543046+0000","last_became_peered":"2023-10-16T11:19:22.543046+0000","last_unstale":"2023-10-16T11:19:25.475523+0000","last_undegraded":"2023-10-16T11:19:25.475523+0000","last_fullsized":"2023-10-16T11:19:25.475523+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:08.672505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479100+0000","last_change":"2023-10-16T11:19:23.563660+0000","last_active":"2023-10-16T11:19:25.479100+0000","last_peered":"2023-10-16T11:19:25.479100+0000","last_clean":"2023-10-16T11:19:25.479100+0000","last_became_active":"2023-10-16T11:19:23.563402+0000","last_became_peered":"2023-10-16T11:19:23.563402+0000","last_unstale":"2023-10-16T11:19:25.479100+0000","last_undegraded":"2023-10-16T11:19:25.479100+0000","last_fullsized":"2023-10-16T11:19:25.479100+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:41.606189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.605645+0000","last_change":"2023-10-16T11:19:34.605645+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.605645+0000","last_undegraded":"2023-10-16T11:19:34.605645+0000","last_fullsized":"2023-10-16T11:19:34.605645+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479196+0000","last_change":"2023-10-16T11:19:22.535771+0000","last_active":"2023-10-16T11:19:25.479196+0000","last_peered":"2023-10-16T11:19:25.479196+0000","last_clean":"2023-10-16T11:19:25.479196+0000","last_became_active":"2023-10-16T11:19:22.535486+0000","last_became_peered":"2023-10-16T11:19:22.535486+0000","last_unstale":"2023-10-16T11:19:25.479196+0000","last_undegraded":"2023-10-16T11:19:25.479196+0000","last_fullsized":"2023-10-16T11:19:25.479196+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:25.214100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646695+0000","last_change":"2023-10-16T11:19:23.560046+0000","last_active":"2023-10-16T11:19:26.646695+0000","last_peered":"2023-10-16T11:19:26.646695+0000","last_clean":"2023-10-16T11:19:26.646695+0000","last_became_active":"2023-10-16T11:19:23.559718+0000","last_became_peered":"2023-10-16T11:19:23.559718+0000","last_unstale":"2023-10-16T11:19:26.646695+0000","last_undegraded":"2023-10-16T11:19:26.646695+0000","last_fullsized":"2023-10-16T11:19:26.646695+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:22.701369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.609835+0000","last_change":"2023-10-16T11:19:34.609835+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.609835+0000","last_undegraded":"2023-10-16T11:19:34.609835+0000","last_fullsized":"2023-10-16T11:19:34.609835+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545940+0000","last_change":"2023-10-16T11:19:22.545557+0000","last_active":"2023-10-16T11:19:25.545940+0000","last_peered":"2023-10-16T11:19:25.545940+0000","last_clean":"2023-10-16T11:19:25.545940+0000","last_became_active":"2023-10-16T11:19:22.545253+0000","last_became_peered":"2023-10-16T11:19:22.545253+0000","last_unstale":"2023-10-16T11:19:25.545940+0000","last_undegraded":"2023-10-16T11:19:25.545940+0000","last_fullsized":"2023-10-16T11:19:25.545940+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:11.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479179+0000","last_change":"2023-10-16T11:19:23.553070+0000","last_active":"2023-10-16T11:19:25.479179+0000","last_peered":"2023-10-16T11:19:25.479179+0000","last_clean":"2023-10-16T11:19:25.479179+0000","last_became_active":"2023-10-16T11:19:23.552761+0000","last_became_peered":"2023-10-16T11:19:23.552761+0000","last_unstale":"2023-10-16T11:19:25.479179+0000","last_undegraded":"2023-10-16T11:19:25.479179+0000","last_fullsized":"2023-10-16T11:19:25.479179+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:36.373122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.608115+0000","last_change":"2023-10-16T11:19:34.608115+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.608115+0000","last_undegraded":"2023-10-16T11:19:34.608115+0000","last_fullsized":"2023-10-16T11:19:34.608115+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478941+0000","last_change":"2023-10-16T11:19:22.534080+0000","last_active":"2023-10-16T11:19:25.478941+0000","last_peered":"2023-10-16T11:19:25.478941+0000","last_clean":"2023-10-16T11:19:25.478941+0000","last_became_active":"2023-10-16T11:19:22.533693+0000","last_became_peered":"2023-10-16T11:19:22.533693+0000","last_unstale":"2023-10-16T11:19:25.478941+0000","last_undegraded":"2023-10-16T11:19:25.478941+0000","last_fullsized":"2023-10-16T11:19:25.478941+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:57.023698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479023+0000","last_change":"2023-10-16T11:19:23.546966+0000","last_active":"2023-10-16T11:19:25.479023+0000","last_peered":"2023-10-16T11:19:25.479023+0000","last_clean":"2023-10-16T11:19:25.479023+0000","last_became_active":"2023-10-16T11:19:23.546685+0000","last_became_peered":"2023-10-16T11:19:23.546685+0000","last_unstale":"2023-10-16T11:19:25.479023+0000","last_undegraded":"2023-10-16T11:19:25.479023+0000","last_fullsized":"2023-10-16T11:19:25.479023+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:13.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.609902+0000","last_change":"2023-10-16T11:19:34.609902+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.609902+0000","last_undegraded":"2023-10-16T11:19:34.609902+0000","last_fullsized":"2023-10-16T11:19:34.609902+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545426+0000","last_change":"2023-10-16T11:19:22.547192+0000","last_active":"2023-10-16T11:19:25.545426+0000","last_peered":"2023-10-16T11:19:25.545426+0000","last_clean":"2023-10-16T11:19:25.545426+0000","last_became_active":"2023-10-16T11:19:22.546912+0000","last_became_peered":"2023-10-16T11:19:22.546912+0000","last_unstale":"2023-10-16T11:19:25.545426+0000","last_undegraded":"2023-10-16T11:19:25.545426+0000","last_fullsized":"2023-10-16T11:19:25.545426+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:17.017260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545575+0000","last_change":"2023-10-16T11:19:23.543708+0000","last_active":"2023-10-16T11:19:25.545575+0000","last_peered":"2023-10-16T11:19:25.545575+0000","last_clean":"2023-10-16T11:19:25.545575+0000","last_became_active":"2023-10-16T11:19:23.543449+0000","last_became_peered":"2023-10-16T11:19:23.543449+0000","last_unstale":"2023-10-16T11:19:25.545575+0000","last_undegraded":"2023-10-16T11:19:25.545575+0000","last_fullsized":"2023-10-16T11:19:25.545575+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:06.406149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.610145+0000","last_change":"2023-10-16T11:19:34.610145+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.610145+0000","last_undegraded":"2023-10-16T11:19:34.610145+0000","last_fullsized":"2023-10-16T11:19:34.610145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545752+0000","last_change":"2023-10-16T11:19:22.543956+0000","last_active":"2023-10-16T11:19:25.545752+0000","last_peered":"2023-10-16T11:19:25.545752+0000","last_clean":"2023-10-16T11:19:25.545752+0000","last_became_active":"2023-10-16T11:19:22.543633+0000","last_became_peered":"2023-10-16T11:19:22.543633+0000","last_unstale":"2023-10-16T11:19:25.545752+0000","last_undegraded":"2023-10-16T11:19:25.545752+0000","last_fullsized":"2023-10-16T11:19:25.545752+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:12:53.412047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479047+0000","last_change":"2023-10-16T11:19:23.546200+0000","last_active":"2023-10-16T11:19:25.479047+0000","last_peered":"2023-10-16T11:19:25.479047+0000","last_clean":"2023-10-16T11:19:25.479047+0000","last_became_active":"2023-10-16T11:19:23.545867+0000","last_became_peered":"2023-10-16T11:19:23.545867+0000","last_unstale":"2023-10-16T11:19:25.479047+0000","last_undegraded":"2023-10-16T11:19:25.479047+0000","last_fullsized":"2023-10-16T11:19:25.479047+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.670688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.609868+0000","last_change":"2023-10-16T11:19:34.609868+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.609868+0000","last_undegraded":"2023-10-16T11:19:34.609868+0000","last_fullsized":"2023-10-16T11:19:34.609868+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479132+0000","last_change":"2023-10-16T11:19:22.535919+0000","last_active":"2023-10-16T11:19:25.479132+0000","last_peered":"2023-10-16T11:19:25.479132+0000","last_clean":"2023-10-16T11:19:25.479132+0000","last_became_active":"2023-10-16T11:19:22.535612+0000","last_became_peered":"2023-10-16T11:19:22.535612+0000","last_unstale":"2023-10-16T11:19:25.479132+0000","last_undegraded":"2023-10-16T11:19:25.479132+0000","last_fullsized":"2023-10-16T11:19:25.479132+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:54.628429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475493+0000","last_change":"2023-10-16T11:19:23.552923+0000","last_active":"2023-10-16T11:19:25.475493+0000","last_peered":"2023-10-16T11:19:25.475493+0000","last_clean":"2023-10-16T11:19:25.475493+0000","last_became_active":"2023-10-16T11:19:23.552685+0000","last_became_peered":"2023-10-16T11:19:23.552685+0000","last_unstale":"2023-10-16T11:19:25.475493+0000","last_undegraded":"2023-10-16T11:19:25.475493+0000","last_fullsized":"2023-10-16T11:19:25.475493+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:13.492062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604821+0000","last_change":"2023-10-16T11:19:34.604821+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604821+0000","last_undegraded":"2023-10-16T11:19:34.604821+0000","last_fullsized":"2023-10-16T11:19:34.604821+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646598+0000","last_change":"2023-10-16T11:19:22.533919+0000","last_active":"2023-10-16T11:19:26.646598+0000","last_peered":"2023-10-16T11:19:26.646598+0000","last_clean":"2023-10-16T11:19:26.646598+0000","last_became_active":"2023-10-16T11:19:22.533695+0000","last_became_peered":"2023-10-16T11:19:22.533695+0000","last_unstale":"2023-10-16T11:19:26.646598+0000","last_undegraded":"2023-10-16T11:19:26.646598+0000","last_fullsized":"2023-10-16T11:19:26.646598+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645700+0000","last_change":"2023-10-16T11:19:23.546849+0000","last_active":"2023-10-16T11:19:26.645700+0000","last_peered":"2023-10-16T11:19:26.645700+0000","last_clean":"2023-10-16T11:19:26.645700+0000","last_became_active":"2023-10-16T11:19:23.546575+0000","last_became_peered":"2023-10-16T11:19:23.546575+0000","last_unstale":"2023-10-16T11:19:26.645700+0000","last_undegraded":"2023-10-16T11:19:26.645700+0000","last_fullsized":"2023-10-16T11:19:26.645700+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.884361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.606124+0000","last_change":"2023-10-16T11:19:34.606124+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.606124+0000","last_undegraded":"2023-10-16T11:19:34.606124+0000","last_fullsized":"2023-10-16T11:19:34.606124+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478867+0000","last_change":"2023-10-16T11:19:22.536812+0000","last_active":"2023-10-16T11:19:25.478867+0000","last_peered":"2023-10-16T11:19:25.478867+0000","last_clean":"2023-10-16T11:19:25.478867+0000","last_became_active":"2023-10-16T11:19:22.536545+0000","last_became_peered":"2023-10-16T11:19:22.536545+0000","last_unstale":"2023-10-16T11:19:25.478867+0000","last_undegraded":"2023-10-16T11:19:25.478867+0000","last_fullsized":"2023-10-16T11:19:25.478867+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479216+0000","last_change":"2023-10-16T11:19:23.559791+0000","last_active":"2023-10-16T11:19:25.479216+0000","last_peered":"2023-10-16T11:19:25.479216+0000","last_clean":"2023-10-16T11:19:25.479216+0000","last_became_active":"2023-10-16T11:19:23.559513+0000","last_became_peered":"2023-10-16T11:19:23.559513+0000","last_unstale":"2023-10-16T11:19:25.479216+0000","last_undegraded":"2023-10-16T11:19:25.479216+0000","last_fullsized":"2023-10-16T11:19:25.479216+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:39.986537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.604598+0000","last_change":"2023-10-16T11:19:34.604598+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.604598+0000","last_undegraded":"2023-10-16T11:19:34.604598+0000","last_fullsized":"2023-10-16T11:19:34.604598+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":17,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646632+0000","last_change":"2023-10-16T11:19:22.538829+0000","last_active":"2023-10-16T11:19:26.646632+0000","last_peered":"2023-10-16T11:19:26.646632+0000","last_clean":"2023-10-16T11:19:26.646632+0000","last_became_active":"2023-10-16T11:19:22.538537+0000","last_became_peered":"2023-10-16T11:19:22.538537+0000","last_unstale":"2023-10-16T11:19:26.646632+0000","last_undegraded":"2023-10-16T11:19:26.646632+0000","last_fullsized":"2023-10-16T11:19:26.646632+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:09.892801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478853+0000","last_change":"2023-10-16T11:19:23.551775+0000","last_active":"2023-10-16T11:19:25.478853+0000","last_peered":"2023-10-16T11:19:25.478853+0000","last_clean":"2023-10-16T11:19:25.478853+0000","last_became_active":"2023-10-16T11:19:23.551490+0000","last_became_peered":"2023-10-16T11:19:23.551490+0000","last_unstale":"2023-10-16T11:19:25.478853+0000","last_undegraded":"2023-10-16T11:19:25.478853+0000","last_fullsized":"2023-10-16T11:19:25.478853+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2023-10-16T11:19:34.607691+0000","last_change":"2023-10-16T11:19:34.607691+0000","last_active":"2023-10-16T11:19:34.594658+0000","last_peered":"2023-10-16T11:19:34.594658+0000","last_clean":"2023-10-16T11:19:34.594658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:34.607691+0000","last_undegraded":"2023-10-16T11:19:34.607691+0000","last_fullsized":"2023-10-16T11:19:34.607691+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645427+0000","last_change":"2023-10-16T11:19:23.544178+0000","last_active":"2023-10-16T11:19:26.645427+0000","last_peered":"2023-10-16T11:19:26.645427+0000","last_clean":"2023-10-16T11:19:26.645427+0000","last_became_active":"2023-10-16T11:19:23.543898+0000","last_became_peered":"2023-10-16T11:19:23.543898+0000","last_unstale":"2023-10-16T11:19:26.645427+0000","last_undegraded":"2023-10-16T11:19:26.645427+0000","last_fullsized":"2023-10-16T11:19:26.645427+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475405+0000","last_change":"2023-10-16T11:19:23.542878+0000","last_active":"2023-10-16T11:19:25.475405+0000","last_peered":"2023-10-16T11:19:25.475405+0000","last_clean":"2023-10-16T11:19:25.475405+0000","last_became_active":"2023-10-16T11:19:23.542593+0000","last_became_peered":"2023-10-16T11:19:23.542593+0000","last_unstale":"2023-10-16T11:19:25.475405+0000","last_undegraded":"2023-10-16T11:19:25.475405+0000","last_fullsized":"2023-10-16T11:19:25.475405+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:39.898154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478829+0000","last_change":"2023-10-16T11:19:23.547762+0000","last_active":"2023-10-16T11:19:25.478829+0000","last_peered":"2023-10-16T11:19:25.478829+0000","last_clean":"2023-10-16T11:19:25.478829+0000","last_became_active":"2023-10-16T11:19:23.547353+0000","last_became_peered":"2023-10-16T11:19:23.547353+0000","last_unstale":"2023-10-16T11:19:25.478829+0000","last_undegraded":"2023-10-16T11:19:25.478829+0000","last_fullsized":"2023-10-16T11:19:25.478829+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646339+0000","last_change":"2023-10-16T11:19:23.565127+0000","last_active":"2023-10-16T11:19:26.646339+0000","last_peered":"2023-10-16T11:19:26.646339+0000","last_clean":"2023-10-16T11:19:26.646339+0000","last_became_active":"2023-10-16T11:19:23.564852+0000","last_became_peered":"2023-10-16T11:19:23.564852+0000","last_unstale":"2023-10-16T11:19:26.646339+0000","last_undegraded":"2023-10-16T11:19:26.646339+0000","last_fullsized":"2023-10-16T11:19:26.646339+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646255+0000","last_change":"2023-10-16T11:19:23.555152+0000","last_active":"2023-10-16T11:19:26.646255+0000","last_peered":"2023-10-16T11:19:26.646255+0000","last_clean":"2023-10-16T11:19:26.646255+0000","last_became_active":"2023-10-16T11:19:23.554747+0000","last_became_peered":"2023-10-16T11:19:23.554747+0000","last_unstale":"2023-10-16T11:19:26.646255+0000","last_undegraded":"2023-10-16T11:19:26.646255+0000","last_fullsized":"2023-10-16T11:19:26.646255+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646479+0000","last_change":"2023-10-16T11:19:23.559781+0000","last_active":"2023-10-16T11:19:26.646479+0000","last_peered":"2023-10-16T11:19:26.646479+0000","last_clean":"2023-10-16T11:19:26.646479+0000","last_became_active":"2023-10-16T11:19:23.559451+0000","last_became_peered":"2023-10-16T11:19:23.559451+0000","last_unstale":"2023-10-16T11:19:26.646479+0000","last_undegraded":"2023-10-16T11:19:26.646479+0000","last_fullsized":"2023-10-16T11:19:26.646479+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645942+0000","last_change":"2023-10-16T11:19:23.546460+0000","last_active":"2023-10-16T11:19:26.645942+0000","last_peered":"2023-10-16T11:19:26.645942+0000","last_clean":"2023-10-16T11:19:26.645942+0000","last_became_active":"2023-10-16T11:19:23.546187+0000","last_became_peered":"2023-10-16T11:19:23.546187+0000","last_unstale":"2023-10-16T11:19:26.645942+0000","last_undegraded":"2023-10-16T11:19:26.645942+0000","last_fullsized":"2023-10-16T11:19:26.645942+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478979+0000","last_change":"2023-10-16T11:19:23.546857+0000","last_active":"2023-10-16T11:19:25.478979+0000","last_peered":"2023-10-16T11:19:25.478979+0000","last_clean":"2023-10-16T11:19:25.478979+0000","last_became_active":"2023-10-16T11:19:23.546592+0000","last_became_peered":"2023-10-16T11:19:23.546592+0000","last_unstale":"2023-10-16T11:19:25.478979+0000","last_undegraded":"2023-10-16T11:19:25.478979+0000","last_fullsized":"2023-10-16T11:19:25.478979+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478981+0000","last_change":"2023-10-16T11:19:23.548868+0000","last_active":"2023-10-16T11:19:25.478981+0000","last_peered":"2023-10-16T11:19:25.478981+0000","last_clean":"2023-10-16T11:19:25.478981+0000","last_became_active":"2023-10-16T11:19:23.548583+0000","last_became_peered":"2023-10-16T11:19:23.548583+0000","last_unstale":"2023-10-16T11:19:25.478981+0000","last_undegraded":"2023-10-16T11:19:25.478981+0000","last_fullsized":"2023-10-16T11:19:25.478981+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:02.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479015+0000","last_change":"2023-10-16T11:19:23.552048+0000","last_active":"2023-10-16T11:19:25.479015+0000","last_peered":"2023-10-16T11:19:25.479015+0000","last_clean":"2023-10-16T11:19:25.479015+0000","last_became_active":"2023-10-16T11:19:23.551676+0000","last_became_peered":"2023-10-16T11:19:23.551676+0000","last_unstale":"2023-10-16T11:19:25.479015+0000","last_undegraded":"2023-10-16T11:19:25.479015+0000","last_fullsized":"2023-10-16T11:19:25.479015+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:16.450373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479089+0000","last_change":"2023-10-16T11:19:23.546038+0000","last_active":"2023-10-16T11:19:25.479089+0000","last_peered":"2023-10-16T11:19:25.479089+0000","last_clean":"2023-10-16T11:19:25.479089+0000","last_became_active":"2023-10-16T11:19:23.545752+0000","last_became_peered":"2023-10-16T11:19:23.545752+0000","last_unstale":"2023-10-16T11:19:25.479089+0000","last_undegraded":"2023-10-16T11:19:25.479089+0000","last_fullsized":"2023-10-16T11:19:25.479089+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:13.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479037+0000","last_change":"2023-10-16T11:19:23.550209+0000","last_active":"2023-10-16T11:19:25.479037+0000","last_peered":"2023-10-16T11:19:25.479037+0000","last_clean":"2023-10-16T11:19:25.479037+0000","last_became_active":"2023-10-16T11:19:23.549849+0000","last_became_peered":"2023-10-16T11:19:23.549849+0000","last_unstale":"2023-10-16T11:19:25.479037+0000","last_undegraded":"2023-10-16T11:19:25.479037+0000","last_fullsized":"2023-10-16T11:19:25.479037+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:29.715382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545908+0000","last_change":"2023-10-16T11:19:23.542321+0000","last_active":"2023-10-16T11:19:25.545908+0000","last_peered":"2023-10-16T11:19:25.545908+0000","last_clean":"2023-10-16T11:19:25.545908+0000","last_became_active":"2023-10-16T11:19:23.541862+0000","last_became_peered":"2023-10-16T11:19:23.541862+0000","last_unstale":"2023-10-16T11:19:25.545908+0000","last_undegraded":"2023-10-16T11:19:25.545908+0000","last_fullsized":"2023-10-16T11:19:25.545908+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:24:50.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475479+0000","last_change":"2023-10-16T11:19:23.547740+0000","last_active":"2023-10-16T11:19:25.475479+0000","last_peered":"2023-10-16T11:19:25.475479+0000","last_clean":"2023-10-16T11:19:25.475479+0000","last_became_active":"2023-10-16T11:19:23.546421+0000","last_became_peered":"2023-10-16T11:19:23.546421+0000","last_unstale":"2023-10-16T11:19:25.475479+0000","last_undegraded":"2023-10-16T11:19:25.475479+0000","last_fullsized":"2023-10-16T11:19:25.475479+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:27.685924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475560+0000","last_change":"2023-10-16T11:19:23.549769+0000","last_active":"2023-10-16T11:19:25.475560+0000","last_peered":"2023-10-16T11:19:25.475560+0000","last_clean":"2023-10-16T11:19:25.475560+0000","last_became_active":"2023-10-16T11:19:23.549505+0000","last_became_peered":"2023-10-16T11:19:23.549505+0000","last_unstale":"2023-10-16T11:19:25.475560+0000","last_undegraded":"2023-10-16T11:19:25.475560+0000","last_fullsized":"2023-10-16T11:19:25.475560+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:20.931218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645936+0000","last_change":"2023-10-16T11:19:23.548276+0000","last_active":"2023-10-16T11:19:26.645936+0000","last_peered":"2023-10-16T11:19:26.645936+0000","last_clean":"2023-10-16T11:19:26.645936+0000","last_became_active":"2023-10-16T11:19:23.547177+0000","last_became_peered":"2023-10-16T11:19:23.547177+0000","last_unstale":"2023-10-16T11:19:26.645936+0000","last_undegraded":"2023-10-16T11:19:26.645936+0000","last_fullsized":"2023-10-16T11:19:26.645936+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475381+0000","last_change":"2023-10-16T11:19:23.563754+0000","last_active":"2023-10-16T11:19:25.475381+0000","last_peered":"2023-10-16T11:19:25.475381+0000","last_clean":"2023-10-16T11:19:25.475381+0000","last_became_active":"2023-10-16T11:19:23.563487+0000","last_became_peered":"2023-10-16T11:19:23.563487+0000","last_unstale":"2023-10-16T11:19:25.475381+0000","last_undegraded":"2023-10-16T11:19:25.475381+0000","last_fullsized":"2023-10-16T11:19:25.475381+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:13.814299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645632+0000","last_change":"2023-10-16T11:19:23.551280+0000","last_active":"2023-10-16T11:19:26.645632+0000","last_peered":"2023-10-16T11:19:26.645632+0000","last_clean":"2023-10-16T11:19:26.645632+0000","last_became_active":"2023-10-16T11:19:23.551018+0000","last_became_peered":"2023-10-16T11:19:23.551018+0000","last_unstale":"2023-10-16T11:19:26.645632+0000","last_undegraded":"2023-10-16T11:19:26.645632+0000","last_fullsized":"2023-10-16T11:19:26.645632+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:46.256661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645457+0000","last_change":"2023-10-16T11:19:23.559938+0000","last_active":"2023-10-16T11:19:26.645457+0000","last_peered":"2023-10-16T11:19:26.645457+0000","last_clean":"2023-10-16T11:19:26.645457+0000","last_became_active":"2023-10-16T11:19:23.559569+0000","last_became_peered":"2023-10-16T11:19:23.559569+0000","last_unstale":"2023-10-16T11:19:26.645457+0000","last_undegraded":"2023-10-16T11:19:26.645457+0000","last_fullsized":"2023-10-16T11:19:26.645457+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646311+0000","last_change":"2023-10-16T11:19:23.560050+0000","last_active":"2023-10-16T11:19:26.646311+0000","last_peered":"2023-10-16T11:19:26.646311+0000","last_clean":"2023-10-16T11:19:26.646311+0000","last_became_active":"2023-10-16T11:19:23.559558+0000","last_became_peered":"2023-10-16T11:19:23.559558+0000","last_unstale":"2023-10-16T11:19:26.646311+0000","last_undegraded":"2023-10-16T11:19:26.646311+0000","last_fullsized":"2023-10-16T11:19:26.646311+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646446+0000","last_change":"2023-10-16T11:19:23.560016+0000","last_active":"2023-10-16T11:19:26.646446+0000","last_peered":"2023-10-16T11:19:26.646446+0000","last_clean":"2023-10-16T11:19:26.646446+0000","last_became_active":"2023-10-16T11:19:23.559619+0000","last_became_peered":"2023-10-16T11:19:23.559619+0000","last_unstale":"2023-10-16T11:19:26.646446+0000","last_undegraded":"2023-10-16T11:19:26.646446+0000","last_fullsized":"2023-10-16T11:19:26.646446+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:04.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646475+0000","last_change":"2023-10-16T11:19:23.564191+0000","last_active":"2023-10-16T11:19:26.646475+0000","last_peered":"2023-10-16T11:19:26.646475+0000","last_clean":"2023-10-16T11:19:26.646475+0000","last_became_active":"2023-10-16T11:19:23.563291+0000","last_became_peered":"2023-10-16T11:19:23.563291+0000","last_unstale":"2023-10-16T11:19:26.646475+0000","last_undegraded":"2023-10-16T11:19:26.646475+0000","last_fullsized":"2023-10-16T11:19:26.646475+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.475548+0000","last_change":"2023-10-16T11:19:23.546735+0000","last_active":"2023-10-16T11:19:25.475548+0000","last_peered":"2023-10-16T11:19:25.475548+0000","last_clean":"2023-10-16T11:19:25.475548+0000","last_became_active":"2023-10-16T11:19:23.546404+0000","last_became_peered":"2023-10-16T11:19:23.546404+0000","last_unstale":"2023-10-16T11:19:25.475548+0000","last_undegraded":"2023-10-16T11:19:25.475548+0000","last_fullsized":"2023-10-16T11:19:25.475548+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:29.845437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478954+0000","last_change":"2023-10-16T11:19:23.552080+0000","last_active":"2023-10-16T11:19:25.478954+0000","last_peered":"2023-10-16T11:19:25.478954+0000","last_clean":"2023-10-16T11:19:25.478954+0000","last_became_active":"2023-10-16T11:19:23.551779+0000","last_became_peered":"2023-10-16T11:19:23.551779+0000","last_unstale":"2023-10-16T11:19:25.478954+0000","last_undegraded":"2023-10-16T11:19:25.478954+0000","last_fullsized":"2023-10-16T11:19:25.478954+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545485+0000","last_change":"2023-10-16T11:19:23.550227+0000","last_active":"2023-10-16T11:19:25.545485+0000","last_peered":"2023-10-16T11:19:25.545485+0000","last_clean":"2023-10-16T11:19:25.545485+0000","last_became_active":"2023-10-16T11:19:23.549938+0000","last_became_peered":"2023-10-16T11:19:23.549938+0000","last_unstale":"2023-10-16T11:19:25.545485+0000","last_undegraded":"2023-10-16T11:19:25.545485+0000","last_fullsized":"2023-10-16T11:19:25.545485+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:13.268771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645456+0000","last_change":"2023-10-16T11:19:23.547259+0000","last_active":"2023-10-16T11:19:26.645456+0000","last_peered":"2023-10-16T11:19:26.645456+0000","last_clean":"2023-10-16T11:19:26.645456+0000","last_became_active":"2023-10-16T11:19:23.546957+0000","last_became_peered":"2023-10-16T11:19:23.546957+0000","last_unstale":"2023-10-16T11:19:26.645456+0000","last_undegraded":"2023-10-16T11:19:26.645456+0000","last_fullsized":"2023-10-16T11:19:26.645456+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545598+0000","last_change":"2023-10-16T11:19:23.550179+0000","last_active":"2023-10-16T11:19:25.545598+0000","last_peered":"2023-10-16T11:19:25.545598+0000","last_clean":"2023-10-16T11:19:25.545598+0000","last_became_active":"2023-10-16T11:19:23.549811+0000","last_became_peered":"2023-10-16T11:19:23.549811+0000","last_unstale":"2023-10-16T11:19:25.545598+0000","last_undegraded":"2023-10-16T11:19:25.545598+0000","last_fullsized":"2023-10-16T11:19:25.545598+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:58.221233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.545931+0000","last_change":"2023-10-16T11:19:23.541856+0000","last_active":"2023-10-16T11:19:25.545931+0000","last_peered":"2023-10-16T11:19:25.545931+0000","last_clean":"2023-10-16T11:19:25.545931+0000","last_became_active":"2023-10-16T11:19:23.541549+0000","last_became_peered":"2023-10-16T11:19:23.541549+0000","last_unstale":"2023-10-16T11:19:25.545931+0000","last_undegraded":"2023-10-16T11:19:25.545931+0000","last_fullsized":"2023-10-16T11:19:25.545931+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:07.709989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.479082+0000","last_change":"2023-10-16T11:19:23.547136+0000","last_active":"2023-10-16T11:19:25.479082+0000","last_peered":"2023-10-16T11:19:25.479082+0000","last_clean":"2023-10-16T11:19:25.479082+0000","last_became_active":"2023-10-16T11:19:23.546834+0000","last_became_peered":"2023-10-16T11:19:23.546834+0000","last_unstale":"2023-10-16T11:19:25.479082+0000","last_undegraded":"2023-10-16T11:19:25.479082+0000","last_fullsized":"2023-10-16T11:19:25.479082+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:45.683847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.646548+0000","last_change":"2023-10-16T11:19:23.552160+0000","last_active":"2023-10-16T11:19:26.646548+0000","last_peered":"2023-10-16T11:19:26.646548+0000","last_clean":"2023-10-16T11:19:26.646548+0000","last_became_active":"2023-10-16T11:19:23.551805+0000","last_became_peered":"2023-10-16T11:19:23.551805+0000","last_unstale":"2023-10-16T11:19:26.646548+0000","last_undegraded":"2023-10-16T11:19:26.646548+0000","last_fullsized":"2023-10-16T11:19:26.646548+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:25.478933+0000","last_change":"2023-10-16T11:19:23.552427+0000","last_active":"2023-10-16T11:19:25.478933+0000","last_peered":"2023-10-16T11:19:25.478933+0000","last_clean":"2023-10-16T11:19:25.478933+0000","last_became_active":"2023-10-16T11:19:23.552156+0000","last_became_peered":"2023-10-16T11:19:23.552156+0000","last_unstale":"2023-10-16T11:19:25.478933+0000","last_undegraded":"2023-10-16T11:19:25.478933+0000","last_fullsized":"2023-10-16T11:19:25.478933+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-16T11:19:35.600376+0000","last_change":"2023-10-16T11:19:35.600376+0000","last_active":"2023-10-16T11:19:35.600376+0000","last_peered":"2023-10-16T11:19:35.600376+0000","last_clean":"2023-10-16T11:19:35.600376+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T11:19:35.600376+0000","last_undegraded":"2023-10-16T11:19:35.600376+0000","last_fullsized":"2023-10-16T11:19:35.600376+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-16T11:19:26.645961+0000","last_change":"2023-10-16T11:19:23.547472+0000","last_active":"2023-10-16T11:19:26.645961+0000","last_peered":"2023-10-16T11:19:26.645961+0000","last_clean":"2023-10-16T11:19:26.645961+0000","last_became_active":"2023-10-16T11:19:23.547148+0000","last_became_peered":"2023-10-16T11:19:23.547148+0000","last_unstale":"2023-10-16T11:19:26.645961+0000","last_undegraded":"2023-10-16T11:19:26.645961+0000","last_fullsized":"2023-10-16T11:19:26.645961+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3657,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543281,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3325952,"data_stored":1988414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.27800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42399999999999999,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69599999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.57799999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.79900000000000004,"15min":0.86299999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.307},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.002},"last":0.71199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33100000000000002},"max":{"1min":8.1950000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54400000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.504},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.871},"last":0.52800000000000002}]}]},{"osd":6,"up_from":14,"seq":60129543280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2723840,"data_stored":1395769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.875},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.51200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.497}]},{"osd":1,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55900000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.752,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.32100000000000001},"max":{"1min":0.79500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.91100000000000003},"last":0.63600000000000001}]}]},{"osd":1,"up_from":14,"seq":60129543280,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60212,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311628,"statfs":{"total":96636764160,"available":96575107072,"internally_reserved":0,"allocated":2727936,"data_stored":1395243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.224},"max":{"1min":0.60199999999999998,"5min":0.629,"15min":0.68700000000000006},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.434},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.62,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.26300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.55300000000000005,"5min":0.58699999999999997,"15min":0.98899999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.34899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.78300000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.33400000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.0289999999999999,"15min":1.097},"last":0.55700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":1.147},"last":0.71699999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.88,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":0.98899999999999999,"5min":1.004,"15min":1.0600000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46300000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.1060000000000001},"last":0.621}]}]},{"osd":0,"up_from":14,"seq":60129543281,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60788,"kb_used_data":3240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311052,"statfs":{"total":96636764160,"available":96574517248,"internally_reserved":0,"allocated":3317760,"data_stored":1985645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.46400000000000002,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.27200000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58799999999999997,"5min":0.63700000000000001,"15min":0.70299999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.32000000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.68300000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.42599999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.99199999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":1.008},"last":0.46200000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":1.004,"15min":1.008},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59599999999999997}]}]},{"osd":2,"up_from":14,"seq":60129543280,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60204,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311636,"statfs":{"total":96636764160,"available":96575115264,"internally_reserved":0,"allocated":2719744,"data_stored":1395187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":3.2229999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.63},"last":3.2509999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":3.5499999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":3.7440000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.623,"15min":0.65500000000000003},"last":3.2959999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.81200000000000006},"last":3.4929999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":3.6880000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.97599999999999998},"last":3.593}]},{"osd":5,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":3.6779999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":3.7050000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":1.052,"15min":1.052},"last":3.6549999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":3.633}]},{"osd":7,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":3.7200000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68700000000000006,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":3.7320000000000002}]}]},{"osd":3,"up_from":14,"seq":60129543281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2732032,"data_stored":1395369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.19},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.36799999999999999,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":0.60699999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.253}]},{"osd":2,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.28899999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":1.034,"15min":1.0740000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.5}]},{"osd":5,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.0489999999999999},"last":0.74199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60224,"kb_used_data":2676,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311616,"statfs":{"total":96636764160,"available":96575094784,"internally_reserved":0,"allocated":2740224,"data_stored":1398150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.628},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.66900000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.627},"min":{"1min":0.438,"5min":0.40400000000000003,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.53400000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.879},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.69499999999999995,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.89500000000000002},"last":0.51400000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.60799999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":0.875,"15min":0.875},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.55000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129543281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42160,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329680,"statfs":{"total":96636764160,"available":96593592320,"internally_reserved":0,"allocated":2723840,"data_stored":1395769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.501},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":1.873,"15min":1.873},"last":0.55300000000000005}]},{"osd":2,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.53700000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":1.821,"15min":1.821},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52400000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":1.919,"15min":1.919},"last":0.34799999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.873},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":2.02,"15min":2.02},"last":0.58399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:19:41.521 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:19:41.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.669+0000 7fa60b607700 1 Processor -- start 2023-10-16T11:19:41.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.670+0000 7fa60b607700 1 -- start start 2023-10-16T11:19:41.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.670+0000 7fa60b607700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa60415dfd0 0x7fa6040af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.670+0000 7fa60b607700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040af450 0x7fa60406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.670+0000 7fa60b607700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa60406db80 0x7fa6040cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.670+0000 7fa60b607700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60406f870 con 0x7fa60406db80 2023-10-16T11:19:41.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.670+0000 7fa60b607700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6040d1160 con 0x7fa60415dfd0 2023-10-16T11:19:41.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.670+0000 7fa60b607700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040cc8e0 con 0x7fa6040af450 2023-10-16T11:19:41.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa60406db80 0x7fa6040cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa60406db80 0x7fa6040cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55874/0 (socket says 172.21.15.181:55874) 2023-10-16T11:19:41.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 -- 172.21.15.181:0/1464579145 learned_addr learned my addr 172.21.15.181:0/1464579145 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:41.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa6093a3700 1 --2- 172.21.15.181:0/1464579145 >> v2:172.21.15.190:3300/0 conn(0x7fa60415dfd0 0x7fa6040af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa608ba2700 1 --2- 172.21.15.181:0/1464579145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040af450 0x7fa60406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 -- 172.21.15.181:0/1464579145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040af450 msgr2=0x7fa60406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:41.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 --2- 172.21.15.181:0/1464579145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040af450 0x7fa60406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 -- 172.21.15.181:0/1464579145 >> v2:172.21.15.190:3300/0 conn(0x7fa60415dfd0 msgr2=0x7fa6040af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:41.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 --2- 172.21.15.181:0/1464579145 >> v2:172.21.15.190:3300/0 conn(0x7fa60415dfd0 0x7fa6040af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 -- 172.21.15.181:0/1464579145 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040ccbc0 con 0x7fa60406db80 2023-10-16T11:19:41.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa6093a3700 1 --2- 172.21.15.181:0/1464579145 >> v2:172.21.15.190:3300/0 conn(0x7fa60415dfd0 0x7fa6040af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:41.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa608ba2700 1 --2- 172.21.15.181:0/1464579145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040af450 0x7fa60406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:41.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa609ba4700 1 --2- 172.21.15.181:0/1464579145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60406db80 0x7fa6040cd470 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa604070340 tx=0x7fa5fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e675ce98e6c9c1c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:41.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa5fbfff700 1 -- 172.21.15.181:0/1464579145 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00e070 con 0x7fa60406db80 2023-10-16T11:19:41.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa5fbfff700 1 -- 172.21.15.181:0/1464579145 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00b9c0 con 0x7fa60406db80 2023-10-16T11:19:41.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.671+0000 7fa5fbfff700 1 -- 172.21.15.181:0/1464579145 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00bd70 con 0x7fa60406db80 2023-10-16T11:19:41.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 -- 172.21.15.181:0/1464579145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60406db80 msgr2=0x7fa6040cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 --2- 172.21.15.181:0/1464579145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60406db80 0x7fa6040cd470 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa604070340 tx=0x7fa5fc00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:41.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 -- 172.21.15.181:0/1464579145 shutdown_connections 2023-10-16T11:19:41.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 --2- 172.21.15.181:0/1464579145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60406db80 0x7fa6040cd470 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 --2- 172.21.15.181:0/1464579145 >> v2:172.21.15.190:3300/0 conn(0x7fa60415dfd0 0x7fa6040af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 --2- 172.21.15.181:0/1464579145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6040af450 0x7fa60406d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 -- 172.21.15.181:0/1464579145 >> 172.21.15.181:0/1464579145 conn(0x7fa6040d04a0 msgr2=0x7fa6040d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:41.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 -- 172.21.15.181:0/1464579145 shutdown_connections 2023-10-16T11:19:41.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.672+0000 7fa60b607700 1 -- 172.21.15.181:0/1464579145 wait complete. 2023-10-16T11:19:41.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.673+0000 7fa60b607700 1 Processor -- start 2023-10-16T11:19:41.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.673+0000 7fa60b607700 1 -- start start 2023-10-16T11:19:41.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa60b607700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60406db80 0x7fa6040c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa60b607700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6040af450 0x7fa6041f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa60b607700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60415dfd0 0x7fa6040c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa60b607700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040c51c0 con 0x7fa60415dfd0 2023-10-16T11:19:41.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa60b607700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040c45b0 con 0x7fa6040af450 2023-10-16T11:19:41.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa60b607700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040c4890 con 0x7fa60406db80 2023-10-16T11:19:41.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60415dfd0 0x7fa6040c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60415dfd0 0x7fa6040c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55880/0 (socket says 172.21.15.181:55880) 2023-10-16T11:19:41.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 -- 172.21.15.181:0/963391299 learned_addr learned my addr 172.21.15.181:0/963391299 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:41.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa6093a3700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60406db80 0x7fa6040c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 -- 172.21.15.181:0/963391299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60406db80 msgr2=0x7fa6040c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60406db80 0x7fa6040c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 -- 172.21.15.181:0/963391299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6040af450 msgr2=0x7fa6041f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6040af450 0x7fa6041f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 -- 172.21.15.181:0/963391299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc009b30 con 0x7fa60415dfd0 2023-10-16T11:19:41.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa609ba4700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60415dfd0 0x7fa6040c5780 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00b370 tx=0x7fa5fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:41.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa5f9ffb700 1 -- 172.21.15.181:0/963391299 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc004250 con 0x7fa60415dfd0 2023-10-16T11:19:41.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa5f9ffb700 1 -- 172.21.15.181:0/963391299 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc0043b0 con 0x7fa60415dfd0 2023-10-16T11:19:41.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa5f9ffb700 1 -- 172.21.15.181:0/963391299 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc004760 con 0x7fa60415dfd0 2023-10-16T11:19:41.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.674+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040c4b70 con 0x7fa60415dfd0 2023-10-16T11:19:41.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.675+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa604159910 con 0x7fa60415dfd0 2023-10-16T11:19:41.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.675+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa604067d20 con 0x7fa60415dfd0 2023-10-16T11:19:41.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.676+0000 7fa5f9ffb700 1 -- 172.21.15.181:0/963391299 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa5fc01c440 con 0x7fa60415dfd0 2023-10-16T11:19:41.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.676+0000 7fa5f9ffb700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa5e806b020 0x7fa5e806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:41.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.676+0000 7fa5f9ffb700 1 -- 172.21.15.181:0/963391299 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fa5fc08b300 con 0x7fa60415dfd0 2023-10-16T11:19:41.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.676+0000 7fa6093a3700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa5e806b020 0x7fa5e806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:41.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.677+0000 7fa6093a3700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa5e806b020 0x7fa5e806d4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d8b0 tx=0x7fa5f4008040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:41.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.681+0000 7fa5f9ffb700 1 -- 172.21.15.181:0/963391299 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa5fc055c90 con 0x7fa60415dfd0 2023-10-16T11:19:41.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.868+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6040cc590 con 0x7fa5e806b020 2023-10-16T11:19:41.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.882+0000 7fa5f9ffb700 1 -- 172.21.15.181:0/963391299 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529056 (secure 0 0 0) 0x7fa5f4014460 con 0x7fa5e806b020 2023-10-16T11:19:41.890 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:41.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa5e806b020 msgr2=0x7fa5e806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa5e806b020 0x7fa5e806d4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d8b0 tx=0x7fa5f4008040 comp rx=0 tx=0).stop 2023-10-16T11:19:41.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60415dfd0 msgr2=0x7fa6040c5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:41.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60415dfd0 0x7fa6040c5780 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00b370 tx=0x7fa5fc004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 shutdown_connections 2023-10-16T11:19:41.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa5e806b020 0x7fa5e806d4e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60406db80 0x7fa6040c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa60415dfd0 0x7fa6040c5780 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 --2- 172.21.15.181:0/963391299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6040af450 0x7fa6041f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:41.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 >> 172.21.15.181:0/963391299 conn(0x7fa6040d04a0 msgr2=0x7fa60415c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:41.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 shutdown_connections 2023-10-16T11:19:41.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:41.887+0000 7fa60b607700 1 -- 172.21.15.181:0/963391299 wait complete. 2023-10-16T11:19:41.898 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:19:41.901 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-16T11:19:41.493813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596435,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":671,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":341,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422172,"kb_used_data":22824,"kb_used_omap":12,"kb_used_meta":399219,"kb_avail":754552548,"statfs":{"total":773094113280,"available":772661809152,"internally_reserved":0,"allocated":23371776,"data_stored":12543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":408800824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.010471"},"pg_stats":[{"pgid":"1.0","version":"206'348","reported_seq":1213,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619646+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:19:36.619646+0000","last_peered":"2023-10-16T11:19:36.619646+0000","last_clean":"2023-10-16T11:19:36.619646+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:19:36.619646+0000","last_undegraded":"2023-10-16T11:19:36.619646+0000","last_fullsized":"2023-10-16T11:19:36.619646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629864+0000","last_change":"2023-10-16T11:19:22.536415+0000","last_active":"2023-10-16T11:19:36.629864+0000","last_peered":"2023-10-16T11:19:36.629864+0000","last_clean":"2023-10-16T11:19:36.629864+0000","last_became_active":"2023-10-16T11:19:22.536117+0000","last_became_peered":"2023-10-16T11:19:22.536117+0000","last_unstale":"2023-10-16T11:19:36.629864+0000","last_undegraded":"2023-10-16T11:19:36.629864+0000","last_fullsized":"2023-10-16T11:19:36.629864+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:07.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620141+0000","last_change":"2023-10-16T11:19:23.551428+0000","last_active":"2023-10-16T11:19:36.620141+0000","last_peered":"2023-10-16T11:19:36.620141+0000","last_clean":"2023-10-16T11:19:36.620141+0000","last_became_active":"2023-10-16T11:19:23.551083+0000","last_became_peered":"2023-10-16T11:19:23.551083+0000","last_unstale":"2023-10-16T11:19:36.620141+0000","last_undegraded":"2023-10-16T11:19:36.620141+0000","last_fullsized":"2023-10-16T11:19:36.620141+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:52.800093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023169+0000","last_change":"2023-10-16T11:19:35.631939+0000","last_active":"2023-10-16T11:19:37.023169+0000","last_peered":"2023-10-16T11:19:37.023169+0000","last_clean":"2023-10-16T11:19:37.023169+0000","last_became_active":"2023-10-16T11:19:35.631645+0000","last_became_peered":"2023-10-16T11:19:35.631645+0000","last_unstale":"2023-10-16T11:19:37.023169+0000","last_undegraded":"2023-10-16T11:19:37.023169+0000","last_fullsized":"2023-10-16T11:19:37.023169+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:56.488955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893598+0000","last_change":"2023-10-16T11:19:36.631354+0000","last_active":"2023-10-16T11:19:37.893598+0000","last_peered":"2023-10-16T11:19:37.893598+0000","last_clean":"2023-10-16T11:19:37.893598+0000","last_became_active":"2023-10-16T11:19:36.631071+0000","last_became_peered":"2023-10-16T11:19:36.631071+0000","last_unstale":"2023-10-16T11:19:37.893598+0000","last_undegraded":"2023-10-16T11:19:37.893598+0000","last_fullsized":"2023-10-16T11:19:37.893598+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:44.063667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630084+0000","last_change":"2023-10-16T11:19:22.537241+0000","last_active":"2023-10-16T11:19:36.630084+0000","last_peered":"2023-10-16T11:19:36.630084+0000","last_clean":"2023-10-16T11:19:36.630084+0000","last_became_active":"2023-10-16T11:19:22.536947+0000","last_became_peered":"2023-10-16T11:19:22.536947+0000","last_unstale":"2023-10-16T11:19:36.630084+0000","last_undegraded":"2023-10-16T11:19:36.630084+0000","last_fullsized":"2023-10-16T11:19:36.630084+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618113+0000","last_change":"2023-10-16T11:19:23.563089+0000","last_active":"2023-10-16T11:19:36.618113+0000","last_peered":"2023-10-16T11:19:36.618113+0000","last_clean":"2023-10-16T11:19:36.618113+0000","last_became_active":"2023-10-16T11:19:23.562759+0000","last_became_peered":"2023-10-16T11:19:23.562759+0000","last_unstale":"2023-10-16T11:19:36.618113+0000","last_undegraded":"2023-10-16T11:19:36.618113+0000","last_fullsized":"2023-10-16T11:19:36.618113+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893516+0000","last_change":"2023-10-16T11:19:35.628235+0000","last_active":"2023-10-16T11:19:37.893516+0000","last_peered":"2023-10-16T11:19:37.893516+0000","last_clean":"2023-10-16T11:19:37.893516+0000","last_became_active":"2023-10-16T11:19:35.627756+0000","last_became_peered":"2023-10-16T11:19:35.627756+0000","last_unstale":"2023-10-16T11:19:37.893516+0000","last_undegraded":"2023-10-16T11:19:37.893516+0000","last_fullsized":"2023-10-16T11:19:37.893516+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:06.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894143+0000","last_change":"2023-10-16T11:19:36.632258+0000","last_active":"2023-10-16T11:19:37.894143+0000","last_peered":"2023-10-16T11:19:37.894143+0000","last_clean":"2023-10-16T11:19:37.894143+0000","last_became_active":"2023-10-16T11:19:36.631963+0000","last_became_peered":"2023-10-16T11:19:36.631963+0000","last_unstale":"2023-10-16T11:19:37.894143+0000","last_undegraded":"2023-10-16T11:19:37.894143+0000","last_fullsized":"2023-10-16T11:19:37.894143+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624408+0000","last_change":"2023-10-16T11:19:22.539000+0000","last_active":"2023-10-16T11:19:36.624408+0000","last_peered":"2023-10-16T11:19:36.624408+0000","last_clean":"2023-10-16T11:19:36.624408+0000","last_became_active":"2023-10-16T11:19:22.538728+0000","last_became_peered":"2023-10-16T11:19:22.538728+0000","last_unstale":"2023-10-16T11:19:36.624408+0000","last_undegraded":"2023-10-16T11:19:36.624408+0000","last_fullsized":"2023-10-16T11:19:36.624408+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:03.944394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629081+0000","last_change":"2023-10-16T11:19:23.563504+0000","last_active":"2023-10-16T11:19:36.629081+0000","last_peered":"2023-10-16T11:19:36.629081+0000","last_clean":"2023-10-16T11:19:36.629081+0000","last_became_active":"2023-10-16T11:19:23.563243+0000","last_became_peered":"2023-10-16T11:19:23.563243+0000","last_unstale":"2023-10-16T11:19:36.629081+0000","last_undegraded":"2023-10-16T11:19:36.629081+0000","last_fullsized":"2023-10-16T11:19:36.629081+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026018+0000","last_change":"2023-10-16T11:19:35.632100+0000","last_active":"2023-10-16T11:19:37.026018+0000","last_peered":"2023-10-16T11:19:37.026018+0000","last_clean":"2023-10-16T11:19:37.026018+0000","last_became_active":"2023-10-16T11:19:35.631782+0000","last_became_peered":"2023-10-16T11:19:35.631782+0000","last_unstale":"2023-10-16T11:19:37.026018+0000","last_undegraded":"2023-10-16T11:19:37.026018+0000","last_fullsized":"2023-10-16T11:19:37.026018+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:19.485660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892620+0000","last_change":"2023-10-16T11:19:36.630004+0000","last_active":"2023-10-16T11:19:37.892620+0000","last_peered":"2023-10-16T11:19:37.892620+0000","last_clean":"2023-10-16T11:19:37.892620+0000","last_became_active":"2023-10-16T11:19:36.629689+0000","last_became_peered":"2023-10-16T11:19:36.629689+0000","last_unstale":"2023-10-16T11:19:37.892620+0000","last_undegraded":"2023-10-16T11:19:37.892620+0000","last_fullsized":"2023-10-16T11:19:37.892620+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:13.752266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615026+0000","last_change":"2023-10-16T11:19:22.544852+0000","last_active":"2023-10-16T11:19:36.615026+0000","last_peered":"2023-10-16T11:19:36.615026+0000","last_clean":"2023-10-16T11:19:36.615026+0000","last_became_active":"2023-10-16T11:19:22.544565+0000","last_became_peered":"2023-10-16T11:19:22.544565+0000","last_unstale":"2023-10-16T11:19:36.615026+0000","last_undegraded":"2023-10-16T11:19:36.615026+0000","last_fullsized":"2023-10-16T11:19:36.615026+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:07.144566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630082+0000","last_change":"2023-10-16T11:19:23.558505+0000","last_active":"2023-10-16T11:19:36.630082+0000","last_peered":"2023-10-16T11:19:36.630082+0000","last_clean":"2023-10-16T11:19:36.630082+0000","last_became_active":"2023-10-16T11:19:23.558188+0000","last_became_peered":"2023-10-16T11:19:23.558188+0000","last_unstale":"2023-10-16T11:19:36.630082+0000","last_undegraded":"2023-10-16T11:19:36.630082+0000","last_fullsized":"2023-10-16T11:19:36.630082+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630289+0000","last_change":"2023-10-16T11:19:35.630984+0000","last_active":"2023-10-16T11:19:36.630289+0000","last_peered":"2023-10-16T11:19:36.630289+0000","last_clean":"2023-10-16T11:19:36.630289+0000","last_became_active":"2023-10-16T11:19:35.630524+0000","last_became_peered":"2023-10-16T11:19:35.630524+0000","last_unstale":"2023-10-16T11:19:36.630289+0000","last_undegraded":"2023-10-16T11:19:36.630289+0000","last_fullsized":"2023-10-16T11:19:36.630289+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893641+0000","last_change":"2023-10-16T11:19:36.624656+0000","last_active":"2023-10-16T11:19:37.893641+0000","last_peered":"2023-10-16T11:19:37.893641+0000","last_clean":"2023-10-16T11:19:37.893641+0000","last_became_active":"2023-10-16T11:19:36.623815+0000","last_became_peered":"2023-10-16T11:19:36.623815+0000","last_unstale":"2023-10-16T11:19:37.893641+0000","last_undegraded":"2023-10-16T11:19:37.893641+0000","last_fullsized":"2023-10-16T11:19:37.893641+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:59:16.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616791+0000","last_change":"2023-10-16T11:19:22.543881+0000","last_active":"2023-10-16T11:19:36.616791+0000","last_peered":"2023-10-16T11:19:36.616791+0000","last_clean":"2023-10-16T11:19:36.616791+0000","last_became_active":"2023-10-16T11:19:22.543647+0000","last_became_peered":"2023-10-16T11:19:22.543647+0000","last_unstale":"2023-10-16T11:19:36.616791+0000","last_undegraded":"2023-10-16T11:19:36.616791+0000","last_fullsized":"2023-10-16T11:19:36.616791+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:41.755970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620300+0000","last_change":"2023-10-16T11:19:23.546380+0000","last_active":"2023-10-16T11:19:36.620300+0000","last_peered":"2023-10-16T11:19:36.620300+0000","last_clean":"2023-10-16T11:19:36.620300+0000","last_became_active":"2023-10-16T11:19:23.546098+0000","last_became_peered":"2023-10-16T11:19:23.546098+0000","last_unstale":"2023-10-16T11:19:36.620300+0000","last_undegraded":"2023-10-16T11:19:36.620300+0000","last_fullsized":"2023-10-16T11:19:36.620300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:06.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619997+0000","last_change":"2023-10-16T11:19:35.628388+0000","last_active":"2023-10-16T11:19:36.619997+0000","last_peered":"2023-10-16T11:19:36.619997+0000","last_clean":"2023-10-16T11:19:36.619997+0000","last_became_active":"2023-10-16T11:19:35.627946+0000","last_became_peered":"2023-10-16T11:19:35.627946+0000","last_unstale":"2023-10-16T11:19:36.619997+0000","last_undegraded":"2023-10-16T11:19:36.619997+0000","last_fullsized":"2023-10-16T11:19:36.619997+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:25.890159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894002+0000","last_change":"2023-10-16T11:19:36.639241+0000","last_active":"2023-10-16T11:19:37.894002+0000","last_peered":"2023-10-16T11:19:37.894002+0000","last_clean":"2023-10-16T11:19:37.894002+0000","last_became_active":"2023-10-16T11:19:36.637259+0000","last_became_peered":"2023-10-16T11:19:36.637259+0000","last_unstale":"2023-10-16T11:19:37.894002+0000","last_undegraded":"2023-10-16T11:19:37.894002+0000","last_fullsized":"2023-10-16T11:19:37.894002+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:25.400158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629125+0000","last_change":"2023-10-16T11:19:22.537543+0000","last_active":"2023-10-16T11:19:36.629125+0000","last_peered":"2023-10-16T11:19:36.629125+0000","last_clean":"2023-10-16T11:19:36.629125+0000","last_became_active":"2023-10-16T11:19:22.537050+0000","last_became_peered":"2023-10-16T11:19:22.537050+0000","last_unstale":"2023-10-16T11:19:36.629125+0000","last_undegraded":"2023-10-16T11:19:36.629125+0000","last_fullsized":"2023-10-16T11:19:36.629125+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:03.218818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624306+0000","last_change":"2023-10-16T11:19:23.546472+0000","last_active":"2023-10-16T11:19:36.624306+0000","last_peered":"2023-10-16T11:19:36.624306+0000","last_clean":"2023-10-16T11:19:36.624306+0000","last_became_active":"2023-10-16T11:19:23.546160+0000","last_became_peered":"2023-10-16T11:19:23.546160+0000","last_unstale":"2023-10-16T11:19:36.624306+0000","last_undegraded":"2023-10-16T11:19:36.624306+0000","last_fullsized":"2023-10-16T11:19:36.624306+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:06.395224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620047+0000","last_change":"2023-10-16T11:19:35.624590+0000","last_active":"2023-10-16T11:19:36.620047+0000","last_peered":"2023-10-16T11:19:36.620047+0000","last_clean":"2023-10-16T11:19:36.620047+0000","last_became_active":"2023-10-16T11:19:35.623979+0000","last_became_peered":"2023-10-16T11:19:35.623979+0000","last_unstale":"2023-10-16T11:19:36.620047+0000","last_undegraded":"2023-10-16T11:19:36.620047+0000","last_fullsized":"2023-10-16T11:19:36.620047+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:57.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894860+0000","last_change":"2023-10-16T11:19:36.631096+0000","last_active":"2023-10-16T11:19:37.894860+0000","last_peered":"2023-10-16T11:19:37.894860+0000","last_clean":"2023-10-16T11:19:37.894860+0000","last_became_active":"2023-10-16T11:19:36.630386+0000","last_became_peered":"2023-10-16T11:19:36.630386+0000","last_unstale":"2023-10-16T11:19:37.894860+0000","last_undegraded":"2023-10-16T11:19:37.894860+0000","last_fullsized":"2023-10-16T11:19:37.894860+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:03.603833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616893+0000","last_change":"2023-10-16T11:19:22.543844+0000","last_active":"2023-10-16T11:19:36.616893+0000","last_peered":"2023-10-16T11:19:36.616893+0000","last_clean":"2023-10-16T11:19:36.616893+0000","last_became_active":"2023-10-16T11:19:22.543538+0000","last_became_peered":"2023-10-16T11:19:22.543538+0000","last_unstale":"2023-10-16T11:19:36.616893+0000","last_undegraded":"2023-10-16T11:19:36.616893+0000","last_fullsized":"2023-10-16T11:19:36.616893+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:00.946603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617382+0000","last_change":"2023-10-16T11:19:23.556539+0000","last_active":"2023-10-16T11:19:36.617382+0000","last_peered":"2023-10-16T11:19:36.617382+0000","last_clean":"2023-10-16T11:19:36.617382+0000","last_became_active":"2023-10-16T11:19:23.556209+0000","last_became_peered":"2023-10-16T11:19:23.556209+0000","last_unstale":"2023-10-16T11:19:36.617382+0000","last_undegraded":"2023-10-16T11:19:36.617382+0000","last_fullsized":"2023-10-16T11:19:36.617382+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618436+0000","last_change":"2023-10-16T11:19:35.620983+0000","last_active":"2023-10-16T11:19:36.618436+0000","last_peered":"2023-10-16T11:19:36.618436+0000","last_clean":"2023-10-16T11:19:36.618436+0000","last_became_active":"2023-10-16T11:19:35.620681+0000","last_became_peered":"2023-10-16T11:19:35.620681+0000","last_unstale":"2023-10-16T11:19:36.618436+0000","last_undegraded":"2023-10-16T11:19:36.618436+0000","last_fullsized":"2023-10-16T11:19:36.618436+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:24.394439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893282+0000","last_change":"2023-10-16T11:19:36.628083+0000","last_active":"2023-10-16T11:19:37.893282+0000","last_peered":"2023-10-16T11:19:37.893282+0000","last_clean":"2023-10-16T11:19:37.893282+0000","last_became_active":"2023-10-16T11:19:36.627768+0000","last_became_peered":"2023-10-16T11:19:36.627768+0000","last_unstale":"2023-10-16T11:19:37.893282+0000","last_undegraded":"2023-10-16T11:19:37.893282+0000","last_fullsized":"2023-10-16T11:19:37.893282+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:59.846608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619763+0000","last_change":"2023-10-16T11:19:22.537758+0000","last_active":"2023-10-16T11:19:36.619763+0000","last_peered":"2023-10-16T11:19:36.619763+0000","last_clean":"2023-10-16T11:19:36.619763+0000","last_became_active":"2023-10-16T11:19:22.537201+0000","last_became_peered":"2023-10-16T11:19:22.537201+0000","last_unstale":"2023-10-16T11:19:36.619763+0000","last_undegraded":"2023-10-16T11:19:36.619763+0000","last_fullsized":"2023-10-16T11:19:36.619763+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:26.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622052+0000","last_change":"2023-10-16T11:19:23.564018+0000","last_active":"2023-10-16T11:19:36.622052+0000","last_peered":"2023-10-16T11:19:36.622052+0000","last_clean":"2023-10-16T11:19:36.622052+0000","last_became_active":"2023-10-16T11:19:23.563738+0000","last_became_peered":"2023-10-16T11:19:23.563738+0000","last_unstale":"2023-10-16T11:19:36.622052+0000","last_undegraded":"2023-10-16T11:19:36.622052+0000","last_fullsized":"2023-10-16T11:19:36.622052+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630020+0000","last_change":"2023-10-16T11:19:35.630702+0000","last_active":"2023-10-16T11:19:36.630020+0000","last_peered":"2023-10-16T11:19:36.630020+0000","last_clean":"2023-10-16T11:19:36.630020+0000","last_became_active":"2023-10-16T11:19:35.630133+0000","last_became_peered":"2023-10-16T11:19:35.630133+0000","last_unstale":"2023-10-16T11:19:36.630020+0000","last_undegraded":"2023-10-16T11:19:36.630020+0000","last_fullsized":"2023-10-16T11:19:36.630020+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:57.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892972+0000","last_change":"2023-10-16T11:19:36.630883+0000","last_active":"2023-10-16T11:19:37.892972+0000","last_peered":"2023-10-16T11:19:37.892972+0000","last_clean":"2023-10-16T11:19:37.892972+0000","last_became_active":"2023-10-16T11:19:36.627590+0000","last_became_peered":"2023-10-16T11:19:36.627590+0000","last_unstale":"2023-10-16T11:19:37.892972+0000","last_undegraded":"2023-10-16T11:19:37.892972+0000","last_fullsized":"2023-10-16T11:19:37.892972+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:16.051165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631253+0000","last_change":"2023-10-16T11:19:22.537256+0000","last_active":"2023-10-16T11:19:36.631253+0000","last_peered":"2023-10-16T11:19:36.631253+0000","last_clean":"2023-10-16T11:19:36.631253+0000","last_became_active":"2023-10-16T11:19:22.536996+0000","last_became_peered":"2023-10-16T11:19:22.536996+0000","last_unstale":"2023-10-16T11:19:36.631253+0000","last_undegraded":"2023-10-16T11:19:36.631253+0000","last_fullsized":"2023-10-16T11:19:36.631253+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:22.462616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631014+0000","last_change":"2023-10-16T11:19:23.561790+0000","last_active":"2023-10-16T11:19:36.631014+0000","last_peered":"2023-10-16T11:19:36.631014+0000","last_clean":"2023-10-16T11:19:36.631014+0000","last_became_active":"2023-10-16T11:19:23.561531+0000","last_became_peered":"2023-10-16T11:19:23.561531+0000","last_unstale":"2023-10-16T11:19:36.631014+0000","last_undegraded":"2023-10-16T11:19:36.631014+0000","last_fullsized":"2023-10-16T11:19:36.631014+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616296+0000","last_change":"2023-10-16T11:19:35.632427+0000","last_active":"2023-10-16T11:19:36.616296+0000","last_peered":"2023-10-16T11:19:36.616296+0000","last_clean":"2023-10-16T11:19:36.616296+0000","last_became_active":"2023-10-16T11:19:35.632164+0000","last_became_peered":"2023-10-16T11:19:35.632164+0000","last_unstale":"2023-10-16T11:19:36.616296+0000","last_undegraded":"2023-10-16T11:19:36.616296+0000","last_fullsized":"2023-10-16T11:19:36.616296+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:40.048278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893018+0000","last_change":"2023-10-16T11:19:36.633276+0000","last_active":"2023-10-16T11:19:37.893018+0000","last_peered":"2023-10-16T11:19:37.893018+0000","last_clean":"2023-10-16T11:19:37.893018+0000","last_became_active":"2023-10-16T11:19:36.631425+0000","last_became_peered":"2023-10-16T11:19:36.631425+0000","last_unstale":"2023-10-16T11:19:37.893018+0000","last_undegraded":"2023-10-16T11:19:37.893018+0000","last_fullsized":"2023-10-16T11:19:37.893018+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615074+0000","last_change":"2023-10-16T11:19:22.547829+0000","last_active":"2023-10-16T11:19:36.615074+0000","last_peered":"2023-10-16T11:19:36.615074+0000","last_clean":"2023-10-16T11:19:36.615074+0000","last_became_active":"2023-10-16T11:19:22.547563+0000","last_became_peered":"2023-10-16T11:19:22.547563+0000","last_unstale":"2023-10-16T11:19:36.615074+0000","last_undegraded":"2023-10-16T11:19:36.615074+0000","last_fullsized":"2023-10-16T11:19:36.615074+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:15.819032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624614+0000","last_change":"2023-10-16T11:19:23.563941+0000","last_active":"2023-10-16T11:19:36.624614+0000","last_peered":"2023-10-16T11:19:36.624614+0000","last_clean":"2023-10-16T11:19:36.624614+0000","last_became_active":"2023-10-16T11:19:23.563666+0000","last_became_peered":"2023-10-16T11:19:23.563666+0000","last_unstale":"2023-10-16T11:19:36.624614+0000","last_undegraded":"2023-10-16T11:19:36.624614+0000","last_fullsized":"2023-10-16T11:19:36.624614+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:43.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.021295+0000","last_change":"2023-10-16T11:19:35.628428+0000","last_active":"2023-10-16T11:19:37.021295+0000","last_peered":"2023-10-16T11:19:37.021295+0000","last_clean":"2023-10-16T11:19:37.021295+0000","last_became_active":"2023-10-16T11:19:35.628146+0000","last_became_peered":"2023-10-16T11:19:35.628146+0000","last_unstale":"2023-10-16T11:19:37.021295+0000","last_undegraded":"2023-10-16T11:19:37.021295+0000","last_fullsized":"2023-10-16T11:19:37.021295+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894166+0000","last_change":"2023-10-16T11:19:36.634088+0000","last_active":"2023-10-16T11:19:37.894166+0000","last_peered":"2023-10-16T11:19:37.894166+0000","last_clean":"2023-10-16T11:19:37.894166+0000","last_became_active":"2023-10-16T11:19:36.633725+0000","last_became_peered":"2023-10-16T11:19:36.633725+0000","last_unstale":"2023-10-16T11:19:37.894166+0000","last_undegraded":"2023-10-16T11:19:37.894166+0000","last_fullsized":"2023-10-16T11:19:37.894166+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:49:49.669608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619400+0000","last_change":"2023-10-16T11:19:22.532222+0000","last_active":"2023-10-16T11:19:36.619400+0000","last_peered":"2023-10-16T11:19:36.619400+0000","last_clean":"2023-10-16T11:19:36.619400+0000","last_became_active":"2023-10-16T11:19:22.531939+0000","last_became_peered":"2023-10-16T11:19:22.531939+0000","last_unstale":"2023-10-16T11:19:36.619400+0000","last_undegraded":"2023-10-16T11:19:36.619400+0000","last_fullsized":"2023-10-16T11:19:36.619400+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615636+0000","last_change":"2023-10-16T11:19:23.548561+0000","last_active":"2023-10-16T11:19:36.615636+0000","last_peered":"2023-10-16T11:19:36.615636+0000","last_clean":"2023-10-16T11:19:36.615636+0000","last_became_active":"2023-10-16T11:19:23.548310+0000","last_became_peered":"2023-10-16T11:19:23.548310+0000","last_unstale":"2023-10-16T11:19:36.615636+0000","last_undegraded":"2023-10-16T11:19:36.615636+0000","last_fullsized":"2023-10-16T11:19:36.615636+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.024902+0000","last_change":"2023-10-16T11:19:35.630294+0000","last_active":"2023-10-16T11:19:37.024902+0000","last_peered":"2023-10-16T11:19:37.024902+0000","last_clean":"2023-10-16T11:19:37.024902+0000","last_became_active":"2023-10-16T11:19:35.629895+0000","last_became_peered":"2023-10-16T11:19:35.629895+0000","last_unstale":"2023-10-16T11:19:37.024902+0000","last_undegraded":"2023-10-16T11:19:37.024902+0000","last_fullsized":"2023-10-16T11:19:37.024902+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622628+0000","last_change":"2023-10-16T11:19:36.632583+0000","last_active":"2023-10-16T11:19:37.622628+0000","last_peered":"2023-10-16T11:19:37.622628+0000","last_clean":"2023-10-16T11:19:37.622628+0000","last_became_active":"2023-10-16T11:19:36.632316+0000","last_became_peered":"2023-10-16T11:19:36.632316+0000","last_unstale":"2023-10-16T11:19:37.622628+0000","last_undegraded":"2023-10-16T11:19:37.622628+0000","last_fullsized":"2023-10-16T11:19:37.622628+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619856+0000","last_change":"2023-10-16T11:19:22.542021+0000","last_active":"2023-10-16T11:19:36.619856+0000","last_peered":"2023-10-16T11:19:36.619856+0000","last_clean":"2023-10-16T11:19:36.619856+0000","last_became_active":"2023-10-16T11:19:22.541718+0000","last_became_peered":"2023-10-16T11:19:22.541718+0000","last_unstale":"2023-10-16T11:19:36.619856+0000","last_undegraded":"2023-10-16T11:19:36.619856+0000","last_fullsized":"2023-10-16T11:19:36.619856+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:12.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628983+0000","last_change":"2023-10-16T11:19:23.554949+0000","last_active":"2023-10-16T11:19:36.628983+0000","last_peered":"2023-10-16T11:19:36.628983+0000","last_clean":"2023-10-16T11:19:36.628983+0000","last_became_active":"2023-10-16T11:19:23.554300+0000","last_became_peered":"2023-10-16T11:19:23.554300+0000","last_unstale":"2023-10-16T11:19:36.628983+0000","last_undegraded":"2023-10-16T11:19:36.628983+0000","last_fullsized":"2023-10-16T11:19:36.628983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623336+0000","last_change":"2023-10-16T11:19:35.627888+0000","last_active":"2023-10-16T11:19:36.623336+0000","last_peered":"2023-10-16T11:19:36.623336+0000","last_clean":"2023-10-16T11:19:36.623336+0000","last_became_active":"2023-10-16T11:19:35.627453+0000","last_became_peered":"2023-10-16T11:19:35.627453+0000","last_unstale":"2023-10-16T11:19:36.623336+0000","last_undegraded":"2023-10-16T11:19:36.623336+0000","last_fullsized":"2023-10-16T11:19:36.623336+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:45.616063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894273+0000","last_change":"2023-10-16T11:19:36.638745+0000","last_active":"2023-10-16T11:19:37.894273+0000","last_peered":"2023-10-16T11:19:37.894273+0000","last_clean":"2023-10-16T11:19:37.894273+0000","last_became_active":"2023-10-16T11:19:36.634962+0000","last_became_peered":"2023-10-16T11:19:36.634962+0000","last_unstale":"2023-10-16T11:19:37.894273+0000","last_undegraded":"2023-10-16T11:19:37.894273+0000","last_fullsized":"2023-10-16T11:19:37.894273+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:19.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621691+0000","last_change":"2023-10-16T11:19:22.536403+0000","last_active":"2023-10-16T11:19:36.621691+0000","last_peered":"2023-10-16T11:19:36.621691+0000","last_clean":"2023-10-16T11:19:36.621691+0000","last_became_active":"2023-10-16T11:19:22.536132+0000","last_became_peered":"2023-10-16T11:19:22.536132+0000","last_unstale":"2023-10-16T11:19:36.621691+0000","last_undegraded":"2023-10-16T11:19:36.621691+0000","last_fullsized":"2023-10-16T11:19:36.621691+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:53.563487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621852+0000","last_change":"2023-10-16T11:19:23.543334+0000","last_active":"2023-10-16T11:19:36.621852+0000","last_peered":"2023-10-16T11:19:36.621852+0000","last_clean":"2023-10-16T11:19:36.621852+0000","last_became_active":"2023-10-16T11:19:23.542527+0000","last_became_peered":"2023-10-16T11:19:23.542527+0000","last_unstale":"2023-10-16T11:19:36.621852+0000","last_undegraded":"2023-10-16T11:19:36.621852+0000","last_fullsized":"2023-10-16T11:19:36.621852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:59.083108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.022227+0000","last_change":"2023-10-16T11:19:35.631031+0000","last_active":"2023-10-16T11:19:37.022227+0000","last_peered":"2023-10-16T11:19:37.022227+0000","last_clean":"2023-10-16T11:19:37.022227+0000","last_became_active":"2023-10-16T11:19:35.630599+0000","last_became_peered":"2023-10-16T11:19:35.630599+0000","last_unstale":"2023-10-16T11:19:37.022227+0000","last_undegraded":"2023-10-16T11:19:37.022227+0000","last_fullsized":"2023-10-16T11:19:37.022227+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:13:50.394449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894077+0000","last_change":"2023-10-16T11:19:36.635702+0000","last_active":"2023-10-16T11:19:37.894077+0000","last_peered":"2023-10-16T11:19:37.894077+0000","last_clean":"2023-10-16T11:19:37.894077+0000","last_became_active":"2023-10-16T11:19:36.635443+0000","last_became_peered":"2023-10-16T11:19:36.635443+0000","last_unstale":"2023-10-16T11:19:37.894077+0000","last_undegraded":"2023-10-16T11:19:37.894077+0000","last_fullsized":"2023-10-16T11:19:37.894077+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:37.781294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628927+0000","last_change":"2023-10-16T11:19:22.545030+0000","last_active":"2023-10-16T11:19:36.628927+0000","last_peered":"2023-10-16T11:19:36.628927+0000","last_clean":"2023-10-16T11:19:36.628927+0000","last_became_active":"2023-10-16T11:19:22.544773+0000","last_became_peered":"2023-10-16T11:19:22.544773+0000","last_unstale":"2023-10-16T11:19:36.628927+0000","last_undegraded":"2023-10-16T11:19:36.628927+0000","last_fullsized":"2023-10-16T11:19:36.628927+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:22.946353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623654+0000","last_change":"2023-10-16T11:19:23.545760+0000","last_active":"2023-10-16T11:19:36.623654+0000","last_peered":"2023-10-16T11:19:36.623654+0000","last_clean":"2023-10-16T11:19:36.623654+0000","last_became_active":"2023-10-16T11:19:23.545499+0000","last_became_peered":"2023-10-16T11:19:23.545499+0000","last_unstale":"2023-10-16T11:19:36.623654+0000","last_undegraded":"2023-10-16T11:19:36.623654+0000","last_fullsized":"2023-10-16T11:19:36.623654+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:32.701590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616714+0000","last_change":"2023-10-16T11:19:35.621290+0000","last_active":"2023-10-16T11:19:36.616714+0000","last_peered":"2023-10-16T11:19:36.616714+0000","last_clean":"2023-10-16T11:19:36.616714+0000","last_became_active":"2023-10-16T11:19:35.620989+0000","last_became_peered":"2023-10-16T11:19:35.620989+0000","last_unstale":"2023-10-16T11:19:36.616714+0000","last_undegraded":"2023-10-16T11:19:36.616714+0000","last_fullsized":"2023-10-16T11:19:36.616714+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:22.819700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.895061+0000","last_change":"2023-10-16T11:19:36.639230+0000","last_active":"2023-10-16T11:19:37.895061+0000","last_peered":"2023-10-16T11:19:37.895061+0000","last_clean":"2023-10-16T11:19:37.895061+0000","last_became_active":"2023-10-16T11:19:36.638912+0000","last_became_peered":"2023-10-16T11:19:36.638912+0000","last_unstale":"2023-10-16T11:19:37.895061+0000","last_undegraded":"2023-10-16T11:19:37.895061+0000","last_fullsized":"2023-10-16T11:19:37.895061+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:20.791568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623856+0000","last_change":"2023-10-16T11:19:22.533601+0000","last_active":"2023-10-16T11:19:36.623856+0000","last_peered":"2023-10-16T11:19:36.623856+0000","last_clean":"2023-10-16T11:19:36.623856+0000","last_became_active":"2023-10-16T11:19:22.533374+0000","last_became_peered":"2023-10-16T11:19:22.533374+0000","last_unstale":"2023-10-16T11:19:36.623856+0000","last_undegraded":"2023-10-16T11:19:36.623856+0000","last_fullsized":"2023-10-16T11:19:36.623856+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:52.564361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629027+0000","last_change":"2023-10-16T11:19:23.553745+0000","last_active":"2023-10-16T11:19:36.629027+0000","last_peered":"2023-10-16T11:19:36.629027+0000","last_clean":"2023-10-16T11:19:36.629027+0000","last_became_active":"2023-10-16T11:19:23.553462+0000","last_became_peered":"2023-10-16T11:19:23.553462+0000","last_unstale":"2023-10-16T11:19:36.629027+0000","last_undegraded":"2023-10-16T11:19:36.629027+0000","last_fullsized":"2023-10-16T11:19:36.629027+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023539+0000","last_change":"2023-10-16T11:19:35.632251+0000","last_active":"2023-10-16T11:19:37.023539+0000","last_peered":"2023-10-16T11:19:37.023539+0000","last_clean":"2023-10-16T11:19:37.023539+0000","last_became_active":"2023-10-16T11:19:35.631302+0000","last_became_peered":"2023-10-16T11:19:35.631302+0000","last_unstale":"2023-10-16T11:19:37.023539+0000","last_undegraded":"2023-10-16T11:19:37.023539+0000","last_fullsized":"2023-10-16T11:19:37.023539+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893096+0000","last_change":"2023-10-16T11:19:36.627034+0000","last_active":"2023-10-16T11:19:37.893096+0000","last_peered":"2023-10-16T11:19:37.893096+0000","last_clean":"2023-10-16T11:19:37.893096+0000","last_became_active":"2023-10-16T11:19:36.626768+0000","last_became_peered":"2023-10-16T11:19:36.626768+0000","last_unstale":"2023-10-16T11:19:37.893096+0000","last_undegraded":"2023-10-16T11:19:37.893096+0000","last_fullsized":"2023-10-16T11:19:37.893096+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:49.454701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616295+0000","last_change":"2023-10-16T11:19:22.539558+0000","last_active":"2023-10-16T11:19:36.616295+0000","last_peered":"2023-10-16T11:19:36.616295+0000","last_clean":"2023-10-16T11:19:36.616295+0000","last_became_active":"2023-10-16T11:19:22.539284+0000","last_became_peered":"2023-10-16T11:19:22.539284+0000","last_unstale":"2023-10-16T11:19:36.616295+0000","last_undegraded":"2023-10-16T11:19:36.616295+0000","last_fullsized":"2023-10-16T11:19:36.616295+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616463+0000","last_change":"2023-10-16T11:19:23.546218+0000","last_active":"2023-10-16T11:19:36.616463+0000","last_peered":"2023-10-16T11:19:36.616463+0000","last_clean":"2023-10-16T11:19:36.616463+0000","last_became_active":"2023-10-16T11:19:23.545896+0000","last_became_peered":"2023-10-16T11:19:23.545896+0000","last_unstale":"2023-10-16T11:19:36.616463+0000","last_undegraded":"2023-10-16T11:19:36.616463+0000","last_fullsized":"2023-10-16T11:19:36.616463+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629694+0000","last_change":"2023-10-16T11:19:35.631734+0000","last_active":"2023-10-16T11:19:36.629694+0000","last_peered":"2023-10-16T11:19:36.629694+0000","last_clean":"2023-10-16T11:19:36.629694+0000","last_became_active":"2023-10-16T11:19:35.631398+0000","last_became_peered":"2023-10-16T11:19:35.631398+0000","last_unstale":"2023-10-16T11:19:36.629694+0000","last_undegraded":"2023-10-16T11:19:36.629694+0000","last_fullsized":"2023-10-16T11:19:36.629694+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:32.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893267+0000","last_change":"2023-10-16T11:19:36.627229+0000","last_active":"2023-10-16T11:19:37.893267+0000","last_peered":"2023-10-16T11:19:37.893267+0000","last_clean":"2023-10-16T11:19:37.893267+0000","last_became_active":"2023-10-16T11:19:36.626587+0000","last_became_peered":"2023-10-16T11:19:36.626587+0000","last_unstale":"2023-10-16T11:19:37.893267+0000","last_undegraded":"2023-10-16T11:19:37.893267+0000","last_fullsized":"2023-10-16T11:19:37.893267+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:34.645872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615964+0000","last_change":"2023-10-16T11:19:22.545027+0000","last_active":"2023-10-16T11:19:36.615964+0000","last_peered":"2023-10-16T11:19:36.615964+0000","last_clean":"2023-10-16T11:19:36.615964+0000","last_became_active":"2023-10-16T11:19:22.544756+0000","last_became_peered":"2023-10-16T11:19:22.544756+0000","last_unstale":"2023-10-16T11:19:36.615964+0000","last_undegraded":"2023-10-16T11:19:36.615964+0000","last_fullsized":"2023-10-16T11:19:36.615964+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:23.693322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616002+0000","last_change":"2023-10-16T11:19:23.547530+0000","last_active":"2023-10-16T11:19:36.616002+0000","last_peered":"2023-10-16T11:19:36.616002+0000","last_clean":"2023-10-16T11:19:36.616002+0000","last_became_active":"2023-10-16T11:19:23.547255+0000","last_became_peered":"2023-10-16T11:19:23.547255+0000","last_unstale":"2023-10-16T11:19:36.616002+0000","last_undegraded":"2023-10-16T11:19:36.616002+0000","last_fullsized":"2023-10-16T11:19:36.616002+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616101+0000","last_change":"2023-10-16T11:19:35.632530+0000","last_active":"2023-10-16T11:19:36.616101+0000","last_peered":"2023-10-16T11:19:36.616101+0000","last_clean":"2023-10-16T11:19:36.616101+0000","last_became_active":"2023-10-16T11:19:35.632226+0000","last_became_peered":"2023-10-16T11:19:35.632226+0000","last_unstale":"2023-10-16T11:19:36.616101+0000","last_undegraded":"2023-10-16T11:19:36.616101+0000","last_fullsized":"2023-10-16T11:19:36.616101+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.620864+0000","last_change":"2023-10-16T11:19:36.632635+0000","last_active":"2023-10-16T11:19:37.620864+0000","last_peered":"2023-10-16T11:19:37.620864+0000","last_clean":"2023-10-16T11:19:37.620864+0000","last_became_active":"2023-10-16T11:19:36.632351+0000","last_became_peered":"2023-10-16T11:19:36.632351+0000","last_unstale":"2023-10-16T11:19:37.620864+0000","last_undegraded":"2023-10-16T11:19:37.620864+0000","last_fullsized":"2023-10-16T11:19:37.620864+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:23.825418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":24,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629784+0000","last_change":"2023-10-16T11:19:22.534093+0000","last_active":"2023-10-16T11:19:36.629784+0000","last_peered":"2023-10-16T11:19:36.629784+0000","last_clean":"2023-10-16T11:19:36.629784+0000","last_became_active":"2023-10-16T11:19:22.533637+0000","last_became_peered":"2023-10-16T11:19:22.533637+0000","last_unstale":"2023-10-16T11:19:36.629784+0000","last_undegraded":"2023-10-16T11:19:36.629784+0000","last_fullsized":"2023-10-16T11:19:36.629784+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619314+0000","last_change":"2023-10-16T11:19:23.552950+0000","last_active":"2023-10-16T11:19:36.619314+0000","last_peered":"2023-10-16T11:19:36.619314+0000","last_clean":"2023-10-16T11:19:36.619314+0000","last_became_active":"2023-10-16T11:19:23.552686+0000","last_became_peered":"2023-10-16T11:19:23.552686+0000","last_unstale":"2023-10-16T11:19:36.619314+0000","last_undegraded":"2023-10-16T11:19:36.619314+0000","last_fullsized":"2023-10-16T11:19:36.619314+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622198+0000","last_change":"2023-10-16T11:19:35.623459+0000","last_active":"2023-10-16T11:19:36.622198+0000","last_peered":"2023-10-16T11:19:36.622198+0000","last_clean":"2023-10-16T11:19:36.622198+0000","last_became_active":"2023-10-16T11:19:35.623199+0000","last_became_peered":"2023-10-16T11:19:35.623199+0000","last_unstale":"2023-10-16T11:19:36.622198+0000","last_undegraded":"2023-10-16T11:19:36.622198+0000","last_fullsized":"2023-10-16T11:19:36.622198+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894201+0000","last_change":"2023-10-16T11:19:36.637364+0000","last_active":"2023-10-16T11:19:37.894201+0000","last_peered":"2023-10-16T11:19:37.894201+0000","last_clean":"2023-10-16T11:19:37.894201+0000","last_became_active":"2023-10-16T11:19:36.637057+0000","last_became_peered":"2023-10-16T11:19:36.637057+0000","last_unstale":"2023-10-16T11:19:37.894201+0000","last_undegraded":"2023-10-16T11:19:37.894201+0000","last_fullsized":"2023-10-16T11:19:37.894201+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:03.151059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621449+0000","last_change":"2023-10-16T11:19:22.538008+0000","last_active":"2023-10-16T11:19:36.621449+0000","last_peered":"2023-10-16T11:19:36.621449+0000","last_clean":"2023-10-16T11:19:36.621449+0000","last_became_active":"2023-10-16T11:19:22.537752+0000","last_became_peered":"2023-10-16T11:19:22.537752+0000","last_unstale":"2023-10-16T11:19:36.621449+0000","last_undegraded":"2023-10-16T11:19:36.621449+0000","last_fullsized":"2023-10-16T11:19:36.621449+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:20.859898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628978+0000","last_change":"2023-10-16T11:19:23.563603+0000","last_active":"2023-10-16T11:19:36.628978+0000","last_peered":"2023-10-16T11:19:36.628978+0000","last_clean":"2023-10-16T11:19:36.628978+0000","last_became_active":"2023-10-16T11:19:23.563305+0000","last_became_peered":"2023-10-16T11:19:23.563305+0000","last_unstale":"2023-10-16T11:19:36.628978+0000","last_undegraded":"2023-10-16T11:19:36.628978+0000","last_fullsized":"2023-10-16T11:19:36.628978+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620238+0000","last_change":"2023-10-16T11:19:35.631312+0000","last_active":"2023-10-16T11:19:36.620238+0000","last_peered":"2023-10-16T11:19:36.620238+0000","last_clean":"2023-10-16T11:19:36.620238+0000","last_became_active":"2023-10-16T11:19:35.631045+0000","last_became_peered":"2023-10-16T11:19:35.631045+0000","last_unstale":"2023-10-16T11:19:36.620238+0000","last_undegraded":"2023-10-16T11:19:36.620238+0000","last_fullsized":"2023-10-16T11:19:36.620238+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:41.921297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894234+0000","last_change":"2023-10-16T11:19:36.636938+0000","last_active":"2023-10-16T11:19:37.894234+0000","last_peered":"2023-10-16T11:19:37.894234+0000","last_clean":"2023-10-16T11:19:37.894234+0000","last_became_active":"2023-10-16T11:19:36.636625+0000","last_became_peered":"2023-10-16T11:19:36.636625+0000","last_unstale":"2023-10-16T11:19:37.894234+0000","last_undegraded":"2023-10-16T11:19:37.894234+0000","last_fullsized":"2023-10-16T11:19:37.894234+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:23.128272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624395+0000","last_change":"2023-10-16T11:19:22.538147+0000","last_active":"2023-10-16T11:19:36.624395+0000","last_peered":"2023-10-16T11:19:36.624395+0000","last_clean":"2023-10-16T11:19:36.624395+0000","last_became_active":"2023-10-16T11:19:22.537850+0000","last_became_peered":"2023-10-16T11:19:22.537850+0000","last_unstale":"2023-10-16T11:19:36.624395+0000","last_undegraded":"2023-10-16T11:19:36.624395+0000","last_fullsized":"2023-10-16T11:19:36.624395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:44.794069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621587+0000","last_change":"2023-10-16T11:19:23.552779+0000","last_active":"2023-10-16T11:19:36.621587+0000","last_peered":"2023-10-16T11:19:36.621587+0000","last_clean":"2023-10-16T11:19:36.621587+0000","last_became_active":"2023-10-16T11:19:23.552491+0000","last_became_peered":"2023-10-16T11:19:23.552491+0000","last_unstale":"2023-10-16T11:19:36.621587+0000","last_undegraded":"2023-10-16T11:19:36.621587+0000","last_fullsized":"2023-10-16T11:19:36.621587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:02.969587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"219'6","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.027458+0000","last_change":"2023-10-16T11:19:35.629807+0000","last_active":"2023-10-16T11:19:37.027458+0000","last_peered":"2023-10-16T11:19:37.027458+0000","last_clean":"2023-10-16T11:19:37.027458+0000","last_became_active":"2023-10-16T11:19:35.629137+0000","last_became_peered":"2023-10-16T11:19:35.629137+0000","last_unstale":"2023-10-16T11:19:37.027458+0000","last_undegraded":"2023-10-16T11:19:37.027458+0000","last_fullsized":"2023-10-16T11:19:37.027458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894306+0000","last_change":"2023-10-16T11:19:36.632014+0000","last_active":"2023-10-16T11:19:37.894306+0000","last_peered":"2023-10-16T11:19:37.894306+0000","last_clean":"2023-10-16T11:19:37.894306+0000","last_became_active":"2023-10-16T11:19:36.631735+0000","last_became_peered":"2023-10-16T11:19:36.631735+0000","last_unstale":"2023-10-16T11:19:37.894306+0000","last_undegraded":"2023-10-16T11:19:37.894306+0000","last_fullsized":"2023-10-16T11:19:37.894306+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:25.064507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624578+0000","last_change":"2023-10-16T11:19:22.544032+0000","last_active":"2023-10-16T11:19:36.624578+0000","last_peered":"2023-10-16T11:19:36.624578+0000","last_clean":"2023-10-16T11:19:36.624578+0000","last_became_active":"2023-10-16T11:19:22.543713+0000","last_became_peered":"2023-10-16T11:19:22.543713+0000","last_unstale":"2023-10-16T11:19:36.624578+0000","last_undegraded":"2023-10-16T11:19:36.624578+0000","last_fullsized":"2023-10-16T11:19:36.624578+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:01.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615557+0000","last_change":"2023-10-16T11:19:23.550398+0000","last_active":"2023-10-16T11:19:36.615557+0000","last_peered":"2023-10-16T11:19:36.615557+0000","last_clean":"2023-10-16T11:19:36.615557+0000","last_became_active":"2023-10-16T11:19:23.545907+0000","last_became_peered":"2023-10-16T11:19:23.545907+0000","last_unstale":"2023-10-16T11:19:36.615557+0000","last_undegraded":"2023-10-16T11:19:36.615557+0000","last_fullsized":"2023-10-16T11:19:36.615557+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:05.456349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"219'3","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026764+0000","last_change":"2023-10-16T11:19:35.627232+0000","last_active":"2023-10-16T11:19:37.026764+0000","last_peered":"2023-10-16T11:19:37.026764+0000","last_clean":"2023-10-16T11:19:37.026764+0000","last_became_active":"2023-10-16T11:19:35.626952+0000","last_became_peered":"2023-10-16T11:19:35.626952+0000","last_unstale":"2023-10-16T11:19:37.026764+0000","last_undegraded":"2023-10-16T11:19:37.026764+0000","last_fullsized":"2023-10-16T11:19:37.026764+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:39.610324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621525+0000","last_change":"2023-10-16T11:19:36.636057+0000","last_active":"2023-10-16T11:19:37.621525+0000","last_peered":"2023-10-16T11:19:37.621525+0000","last_clean":"2023-10-16T11:19:37.621525+0000","last_became_active":"2023-10-16T11:19:36.635687+0000","last_became_peered":"2023-10-16T11:19:36.635687+0000","last_unstale":"2023-10-16T11:19:37.621525+0000","last_undegraded":"2023-10-16T11:19:37.621525+0000","last_fullsized":"2023-10-16T11:19:37.621525+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.14","version":"216'8","reported_seq":24,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617300+0000","last_change":"2023-10-16T11:19:22.532339+0000","last_active":"2023-10-16T11:19:36.617300+0000","last_peered":"2023-10-16T11:19:36.617300+0000","last_clean":"2023-10-16T11:19:36.617300+0000","last_became_active":"2023-10-16T11:19:22.532053+0000","last_became_peered":"2023-10-16T11:19:22.532053+0000","last_unstale":"2023-10-16T11:19:36.617300+0000","last_undegraded":"2023-10-16T11:19:36.617300+0000","last_fullsized":"2023-10-16T11:19:36.617300+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:21.190168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2859,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621913+0000","last_change":"2023-10-16T11:19:23.548103+0000","last_active":"2023-10-16T11:19:36.621913+0000","last_peered":"2023-10-16T11:19:36.621913+0000","last_clean":"2023-10-16T11:19:36.621913+0000","last_became_active":"2023-10-16T11:19:23.547665+0000","last_became_peered":"2023-10-16T11:19:23.547665+0000","last_unstale":"2023-10-16T11:19:36.621913+0000","last_undegraded":"2023-10-16T11:19:36.621913+0000","last_fullsized":"2023-10-16T11:19:36.621913+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619766+0000","last_change":"2023-10-16T11:19:35.624749+0000","last_active":"2023-10-16T11:19:36.619766+0000","last_peered":"2023-10-16T11:19:36.619766+0000","last_clean":"2023-10-16T11:19:36.619766+0000","last_became_active":"2023-10-16T11:19:35.624338+0000","last_became_peered":"2023-10-16T11:19:35.624338+0000","last_unstale":"2023-10-16T11:19:36.619766+0000","last_undegraded":"2023-10-16T11:19:36.619766+0000","last_fullsized":"2023-10-16T11:19:36.619766+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:32.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894693+0000","last_change":"2023-10-16T11:19:36.631186+0000","last_active":"2023-10-16T11:19:37.894693+0000","last_peered":"2023-10-16T11:19:37.894693+0000","last_clean":"2023-10-16T11:19:37.894693+0000","last_became_active":"2023-10-16T11:19:36.630761+0000","last_became_peered":"2023-10-16T11:19:36.630761+0000","last_unstale":"2023-10-16T11:19:37.894693+0000","last_undegraded":"2023-10-16T11:19:37.894693+0000","last_fullsized":"2023-10-16T11:19:37.894693+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:18.537927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623757+0000","last_change":"2023-10-16T11:19:22.544432+0000","last_active":"2023-10-16T11:19:36.623757+0000","last_peered":"2023-10-16T11:19:36.623757+0000","last_clean":"2023-10-16T11:19:36.623757+0000","last_became_active":"2023-10-16T11:19:22.543046+0000","last_became_peered":"2023-10-16T11:19:22.543046+0000","last_unstale":"2023-10-16T11:19:36.623757+0000","last_undegraded":"2023-10-16T11:19:36.623757+0000","last_fullsized":"2023-10-16T11:19:36.623757+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:08.672505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617875+0000","last_change":"2023-10-16T11:19:23.563660+0000","last_active":"2023-10-16T11:19:36.617875+0000","last_peered":"2023-10-16T11:19:36.617875+0000","last_clean":"2023-10-16T11:19:36.617875+0000","last_became_active":"2023-10-16T11:19:23.563402+0000","last_became_peered":"2023-10-16T11:19:23.563402+0000","last_unstale":"2023-10-16T11:19:36.617875+0000","last_undegraded":"2023-10-16T11:19:36.617875+0000","last_fullsized":"2023-10-16T11:19:36.617875+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:41.606189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026157+0000","last_change":"2023-10-16T11:19:35.633267+0000","last_active":"2023-10-16T11:19:37.026157+0000","last_peered":"2023-10-16T11:19:37.026157+0000","last_clean":"2023-10-16T11:19:37.026157+0000","last_became_active":"2023-10-16T11:19:35.632961+0000","last_became_peered":"2023-10-16T11:19:35.632961+0000","last_unstale":"2023-10-16T11:19:37.026157+0000","last_undegraded":"2023-10-16T11:19:37.026157+0000","last_fullsized":"2023-10-16T11:19:37.026157+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621385+0000","last_change":"2023-10-16T11:19:36.628797+0000","last_active":"2023-10-16T11:19:37.621385+0000","last_peered":"2023-10-16T11:19:37.621385+0000","last_clean":"2023-10-16T11:19:37.621385+0000","last_became_active":"2023-10-16T11:19:36.628522+0000","last_became_peered":"2023-10-16T11:19:36.628522+0000","last_unstale":"2023-10-16T11:19:37.621385+0000","last_undegraded":"2023-10-16T11:19:37.621385+0000","last_fullsized":"2023-10-16T11:19:37.621385+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.626248+0000","last_change":"2023-10-16T11:19:22.535771+0000","last_active":"2023-10-16T11:19:36.626248+0000","last_peered":"2023-10-16T11:19:36.626248+0000","last_clean":"2023-10-16T11:19:36.626248+0000","last_became_active":"2023-10-16T11:19:22.535486+0000","last_became_peered":"2023-10-16T11:19:22.535486+0000","last_unstale":"2023-10-16T11:19:36.626248+0000","last_undegraded":"2023-10-16T11:19:36.626248+0000","last_fullsized":"2023-10-16T11:19:36.626248+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:25.214100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631131+0000","last_change":"2023-10-16T11:19:23.560046+0000","last_active":"2023-10-16T11:19:36.631131+0000","last_peered":"2023-10-16T11:19:36.631131+0000","last_clean":"2023-10-16T11:19:36.631131+0000","last_became_active":"2023-10-16T11:19:23.559718+0000","last_became_peered":"2023-10-16T11:19:23.559718+0000","last_unstale":"2023-10-16T11:19:36.631131+0000","last_undegraded":"2023-10-16T11:19:36.631131+0000","last_fullsized":"2023-10-16T11:19:36.631131+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:22.701369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"219'8","reported_seq":20,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026701+0000","last_change":"2023-10-16T11:19:35.625566+0000","last_active":"2023-10-16T11:19:37.026701+0000","last_peered":"2023-10-16T11:19:37.026701+0000","last_clean":"2023-10-16T11:19:37.026701+0000","last_became_active":"2023-10-16T11:19:35.624392+0000","last_became_peered":"2023-10-16T11:19:35.624392+0000","last_unstale":"2023-10-16T11:19:37.026701+0000","last_undegraded":"2023-10-16T11:19:37.026701+0000","last_fullsized":"2023-10-16T11:19:37.026701+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:33:14.543389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621461+0000","last_change":"2023-10-16T11:19:36.626287+0000","last_active":"2023-10-16T11:19:37.621461+0000","last_peered":"2023-10-16T11:19:37.621461+0000","last_clean":"2023-10-16T11:19:37.621461+0000","last_became_active":"2023-10-16T11:19:36.625588+0000","last_became_peered":"2023-10-16T11:19:36.625588+0000","last_unstale":"2023-10-16T11:19:37.621461+0000","last_undegraded":"2023-10-16T11:19:37.621461+0000","last_fullsized":"2023-10-16T11:19:37.621461+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:37.781294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615845+0000","last_change":"2023-10-16T11:19:22.545557+0000","last_active":"2023-10-16T11:19:36.615845+0000","last_peered":"2023-10-16T11:19:36.615845+0000","last_clean":"2023-10-16T11:19:36.615845+0000","last_became_active":"2023-10-16T11:19:22.545253+0000","last_became_peered":"2023-10-16T11:19:22.545253+0000","last_unstale":"2023-10-16T11:19:36.615845+0000","last_undegraded":"2023-10-16T11:19:36.615845+0000","last_fullsized":"2023-10-16T11:19:36.615845+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:11.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619711+0000","last_change":"2023-10-16T11:19:23.553070+0000","last_active":"2023-10-16T11:19:36.619711+0000","last_peered":"2023-10-16T11:19:36.619711+0000","last_clean":"2023-10-16T11:19:36.619711+0000","last_became_active":"2023-10-16T11:19:23.552761+0000","last_became_peered":"2023-10-16T11:19:23.552761+0000","last_unstale":"2023-10-16T11:19:36.619711+0000","last_undegraded":"2023-10-16T11:19:36.619711+0000","last_fullsized":"2023-10-16T11:19:36.619711+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:36.373122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616224+0000","last_change":"2023-10-16T11:19:35.623394+0000","last_active":"2023-10-16T11:19:36.616224+0000","last_peered":"2023-10-16T11:19:36.616224+0000","last_clean":"2023-10-16T11:19:36.616224+0000","last_became_active":"2023-10-16T11:19:35.623119+0000","last_became_peered":"2023-10-16T11:19:35.623119+0000","last_unstale":"2023-10-16T11:19:36.616224+0000","last_undegraded":"2023-10-16T11:19:36.616224+0000","last_fullsized":"2023-10-16T11:19:36.616224+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:07.710682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892867+0000","last_change":"2023-10-16T11:19:36.633676+0000","last_active":"2023-10-16T11:19:37.892867+0000","last_peered":"2023-10-16T11:19:37.892867+0000","last_clean":"2023-10-16T11:19:37.892867+0000","last_became_active":"2023-10-16T11:19:36.633464+0000","last_became_peered":"2023-10-16T11:19:36.633464+0000","last_unstale":"2023-10-16T11:19:37.892867+0000","last_undegraded":"2023-10-16T11:19:37.892867+0000","last_fullsized":"2023-10-16T11:19:37.892867+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624527+0000","last_change":"2023-10-16T11:19:22.534080+0000","last_active":"2023-10-16T11:19:36.624527+0000","last_peered":"2023-10-16T11:19:36.624527+0000","last_clean":"2023-10-16T11:19:36.624527+0000","last_became_active":"2023-10-16T11:19:22.533693+0000","last_became_peered":"2023-10-16T11:19:22.533693+0000","last_unstale":"2023-10-16T11:19:36.624527+0000","last_undegraded":"2023-10-16T11:19:36.624527+0000","last_fullsized":"2023-10-16T11:19:36.624527+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:57.023698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617545+0000","last_change":"2023-10-16T11:19:23.546966+0000","last_active":"2023-10-16T11:19:36.617545+0000","last_peered":"2023-10-16T11:19:36.617545+0000","last_clean":"2023-10-16T11:19:36.617545+0000","last_became_active":"2023-10-16T11:19:23.546685+0000","last_became_peered":"2023-10-16T11:19:23.546685+0000","last_unstale":"2023-10-16T11:19:36.617545+0000","last_undegraded":"2023-10-16T11:19:36.617545+0000","last_fullsized":"2023-10-16T11:19:36.617545+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:13.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.025002+0000","last_change":"2023-10-16T11:19:35.624998+0000","last_active":"2023-10-16T11:19:37.025002+0000","last_peered":"2023-10-16T11:19:37.025002+0000","last_clean":"2023-10-16T11:19:37.025002+0000","last_became_active":"2023-10-16T11:19:35.624685+0000","last_became_peered":"2023-10-16T11:19:35.624685+0000","last_unstale":"2023-10-16T11:19:37.025002+0000","last_undegraded":"2023-10-16T11:19:37.025002+0000","last_fullsized":"2023-10-16T11:19:37.025002+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:52:30.461865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622678+0000","last_change":"2023-10-16T11:19:36.638230+0000","last_active":"2023-10-16T11:19:37.622678+0000","last_peered":"2023-10-16T11:19:37.622678+0000","last_clean":"2023-10-16T11:19:37.622678+0000","last_became_active":"2023-10-16T11:19:36.637937+0000","last_became_peered":"2023-10-16T11:19:36.637937+0000","last_unstale":"2023-10-16T11:19:37.622678+0000","last_undegraded":"2023-10-16T11:19:37.622678+0000","last_fullsized":"2023-10-16T11:19:37.622678+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:21.232537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616327+0000","last_change":"2023-10-16T11:19:22.547192+0000","last_active":"2023-10-16T11:19:36.616327+0000","last_peered":"2023-10-16T11:19:36.616327+0000","last_clean":"2023-10-16T11:19:36.616327+0000","last_became_active":"2023-10-16T11:19:22.546912+0000","last_became_peered":"2023-10-16T11:19:22.546912+0000","last_unstale":"2023-10-16T11:19:36.616327+0000","last_undegraded":"2023-10-16T11:19:36.616327+0000","last_fullsized":"2023-10-16T11:19:36.616327+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:17.017260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.614895+0000","last_change":"2023-10-16T11:19:23.543708+0000","last_active":"2023-10-16T11:19:36.614895+0000","last_peered":"2023-10-16T11:19:36.614895+0000","last_clean":"2023-10-16T11:19:36.614895+0000","last_became_active":"2023-10-16T11:19:23.543449+0000","last_became_peered":"2023-10-16T11:19:23.543449+0000","last_unstale":"2023-10-16T11:19:36.614895+0000","last_undegraded":"2023-10-16T11:19:36.614895+0000","last_fullsized":"2023-10-16T11:19:36.614895+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:06.406149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"219'4","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023703+0000","last_change":"2023-10-16T11:19:35.626621+0000","last_active":"2023-10-16T11:19:37.023703+0000","last_peered":"2023-10-16T11:19:37.023703+0000","last_clean":"2023-10-16T11:19:37.023703+0000","last_became_active":"2023-10-16T11:19:35.626338+0000","last_became_peered":"2023-10-16T11:19:35.626338+0000","last_unstale":"2023-10-16T11:19:37.023703+0000","last_undegraded":"2023-10-16T11:19:37.023703+0000","last_fullsized":"2023-10-16T11:19:37.023703+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894242+0000","last_change":"2023-10-16T11:19:36.638939+0000","last_active":"2023-10-16T11:19:37.894242+0000","last_peered":"2023-10-16T11:19:37.894242+0000","last_clean":"2023-10-16T11:19:37.894242+0000","last_became_active":"2023-10-16T11:19:36.637185+0000","last_became_peered":"2023-10-16T11:19:36.637185+0000","last_unstale":"2023-10-16T11:19:37.894242+0000","last_undegraded":"2023-10-16T11:19:37.894242+0000","last_fullsized":"2023-10-16T11:19:37.894242+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:26.350349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615222+0000","last_change":"2023-10-16T11:19:22.543956+0000","last_active":"2023-10-16T11:19:36.615222+0000","last_peered":"2023-10-16T11:19:36.615222+0000","last_clean":"2023-10-16T11:19:36.615222+0000","last_became_active":"2023-10-16T11:19:22.543633+0000","last_became_peered":"2023-10-16T11:19:22.543633+0000","last_unstale":"2023-10-16T11:19:36.615222+0000","last_undegraded":"2023-10-16T11:19:36.615222+0000","last_fullsized":"2023-10-16T11:19:36.615222+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:12:53.412047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617060+0000","last_change":"2023-10-16T11:19:23.546200+0000","last_active":"2023-10-16T11:19:36.617060+0000","last_peered":"2023-10-16T11:19:36.617060+0000","last_clean":"2023-10-16T11:19:36.617060+0000","last_became_active":"2023-10-16T11:19:23.545867+0000","last_became_peered":"2023-10-16T11:19:23.545867+0000","last_unstale":"2023-10-16T11:19:36.617060+0000","last_undegraded":"2023-10-16T11:19:36.617060+0000","last_fullsized":"2023-10-16T11:19:36.617060+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.670688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023629+0000","last_change":"2023-10-16T11:19:35.628631+0000","last_active":"2023-10-16T11:19:37.023629+0000","last_peered":"2023-10-16T11:19:37.023629+0000","last_clean":"2023-10-16T11:19:37.023629+0000","last_became_active":"2023-10-16T11:19:35.628341+0000","last_became_peered":"2023-10-16T11:19:35.628341+0000","last_unstale":"2023-10-16T11:19:37.023629+0000","last_undegraded":"2023-10-16T11:19:37.023629+0000","last_fullsized":"2023-10-16T11:19:37.023629+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:48:39.437826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621875+0000","last_change":"2023-10-16T11:19:36.635937+0000","last_active":"2023-10-16T11:19:37.621875+0000","last_peered":"2023-10-16T11:19:37.621875+0000","last_clean":"2023-10-16T11:19:37.621875+0000","last_became_active":"2023-10-16T11:19:36.635570+0000","last_became_peered":"2023-10-16T11:19:36.635570+0000","last_unstale":"2023-10-16T11:19:37.621875+0000","last_undegraded":"2023-10-16T11:19:37.621875+0000","last_fullsized":"2023-10-16T11:19:37.621875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:19.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617428+0000","last_change":"2023-10-16T11:19:22.535919+0000","last_active":"2023-10-16T11:19:36.617428+0000","last_peered":"2023-10-16T11:19:36.617428+0000","last_clean":"2023-10-16T11:19:36.617428+0000","last_became_active":"2023-10-16T11:19:22.535612+0000","last_became_peered":"2023-10-16T11:19:22.535612+0000","last_unstale":"2023-10-16T11:19:36.617428+0000","last_undegraded":"2023-10-16T11:19:36.617428+0000","last_fullsized":"2023-10-16T11:19:36.617428+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:54.628429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623571+0000","last_change":"2023-10-16T11:19:23.552923+0000","last_active":"2023-10-16T11:19:36.623571+0000","last_peered":"2023-10-16T11:19:36.623571+0000","last_clean":"2023-10-16T11:19:36.623571+0000","last_became_active":"2023-10-16T11:19:23.552685+0000","last_became_peered":"2023-10-16T11:19:23.552685+0000","last_unstale":"2023-10-16T11:19:36.623571+0000","last_undegraded":"2023-10-16T11:19:36.623571+0000","last_fullsized":"2023-10-16T11:19:36.623571+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:13.492062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023727+0000","last_change":"2023-10-16T11:19:35.630995+0000","last_active":"2023-10-16T11:19:37.023727+0000","last_peered":"2023-10-16T11:19:37.023727+0000","last_clean":"2023-10-16T11:19:37.023727+0000","last_became_active":"2023-10-16T11:19:35.630561+0000","last_became_peered":"2023-10-16T11:19:35.630561+0000","last_unstale":"2023-10-16T11:19:37.023727+0000","last_undegraded":"2023-10-16T11:19:37.023727+0000","last_fullsized":"2023-10-16T11:19:37.023727+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:30.099513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893198+0000","last_change":"2023-10-16T11:19:36.631163+0000","last_active":"2023-10-16T11:19:37.893198+0000","last_peered":"2023-10-16T11:19:37.893198+0000","last_clean":"2023-10-16T11:19:37.893198+0000","last_became_active":"2023-10-16T11:19:36.630820+0000","last_became_peered":"2023-10-16T11:19:36.630820+0000","last_unstale":"2023-10-16T11:19:37.893198+0000","last_undegraded":"2023-10-16T11:19:37.893198+0000","last_fullsized":"2023-10-16T11:19:37.893198+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:54.687768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628981+0000","last_change":"2023-10-16T11:19:22.533919+0000","last_active":"2023-10-16T11:19:36.628981+0000","last_peered":"2023-10-16T11:19:36.628981+0000","last_clean":"2023-10-16T11:19:36.628981+0000","last_became_active":"2023-10-16T11:19:22.533695+0000","last_became_peered":"2023-10-16T11:19:22.533695+0000","last_unstale":"2023-10-16T11:19:36.628981+0000","last_undegraded":"2023-10-16T11:19:36.628981+0000","last_fullsized":"2023-10-16T11:19:36.628981+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618694+0000","last_change":"2023-10-16T11:19:23.546849+0000","last_active":"2023-10-16T11:19:36.618694+0000","last_peered":"2023-10-16T11:19:36.618694+0000","last_clean":"2023-10-16T11:19:36.618694+0000","last_became_active":"2023-10-16T11:19:23.546575+0000","last_became_peered":"2023-10-16T11:19:23.546575+0000","last_unstale":"2023-10-16T11:19:36.618694+0000","last_undegraded":"2023-10-16T11:19:36.618694+0000","last_fullsized":"2023-10-16T11:19:36.618694+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.884361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.024943+0000","last_change":"2023-10-16T11:19:35.624948+0000","last_active":"2023-10-16T11:19:37.024943+0000","last_peered":"2023-10-16T11:19:37.024943+0000","last_clean":"2023-10-16T11:19:37.024943+0000","last_became_active":"2023-10-16T11:19:35.624339+0000","last_became_peered":"2023-10-16T11:19:35.624339+0000","last_unstale":"2023-10-16T11:19:37.024943+0000","last_undegraded":"2023-10-16T11:19:37.024943+0000","last_fullsized":"2023-10-16T11:19:37.024943+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893006+0000","last_change":"2023-10-16T11:19:36.635048+0000","last_active":"2023-10-16T11:19:37.893006+0000","last_peered":"2023-10-16T11:19:37.893006+0000","last_clean":"2023-10-16T11:19:37.893006+0000","last_became_active":"2023-10-16T11:19:36.634362+0000","last_became_peered":"2023-10-16T11:19:36.634362+0000","last_unstale":"2023-10-16T11:19:37.893006+0000","last_undegraded":"2023-10-16T11:19:37.893006+0000","last_fullsized":"2023-10-16T11:19:37.893006+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:52.979732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615890+0000","last_change":"2023-10-16T11:19:22.536812+0000","last_active":"2023-10-16T11:19:36.615890+0000","last_peered":"2023-10-16T11:19:36.615890+0000","last_clean":"2023-10-16T11:19:36.615890+0000","last_became_active":"2023-10-16T11:19:22.536545+0000","last_became_peered":"2023-10-16T11:19:22.536545+0000","last_unstale":"2023-10-16T11:19:36.615890+0000","last_undegraded":"2023-10-16T11:19:36.615890+0000","last_fullsized":"2023-10-16T11:19:36.615890+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624796+0000","last_change":"2023-10-16T11:19:23.559791+0000","last_active":"2023-10-16T11:19:36.624796+0000","last_peered":"2023-10-16T11:19:36.624796+0000","last_clean":"2023-10-16T11:19:36.624796+0000","last_became_active":"2023-10-16T11:19:23.559513+0000","last_became_peered":"2023-10-16T11:19:23.559513+0000","last_unstale":"2023-10-16T11:19:36.624796+0000","last_undegraded":"2023-10-16T11:19:36.624796+0000","last_fullsized":"2023-10-16T11:19:36.624796+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:39.986537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621822+0000","last_change":"2023-10-16T11:19:35.622388+0000","last_active":"2023-10-16T11:19:36.621822+0000","last_peered":"2023-10-16T11:19:36.621822+0000","last_clean":"2023-10-16T11:19:36.621822+0000","last_became_active":"2023-10-16T11:19:35.622128+0000","last_became_peered":"2023-10-16T11:19:35.622128+0000","last_unstale":"2023-10-16T11:19:36.621822+0000","last_undegraded":"2023-10-16T11:19:36.621822+0000","last_fullsized":"2023-10-16T11:19:36.621822+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:10.516621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892728+0000","last_change":"2023-10-16T11:19:36.636570+0000","last_active":"2023-10-16T11:19:37.892728+0000","last_peered":"2023-10-16T11:19:37.892728+0000","last_clean":"2023-10-16T11:19:37.892728+0000","last_became_active":"2023-10-16T11:19:36.636314+0000","last_became_peered":"2023-10-16T11:19:36.636314+0000","last_unstale":"2023-10-16T11:19:37.892728+0000","last_undegraded":"2023-10-16T11:19:37.892728+0000","last_fullsized":"2023-10-16T11:19:37.892728+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:19.250783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":20,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631343+0000","last_change":"2023-10-16T11:19:22.538829+0000","last_active":"2023-10-16T11:19:36.631343+0000","last_peered":"2023-10-16T11:19:36.631343+0000","last_clean":"2023-10-16T11:19:36.631343+0000","last_became_active":"2023-10-16T11:19:22.538537+0000","last_became_peered":"2023-10-16T11:19:22.538537+0000","last_unstale":"2023-10-16T11:19:36.631343+0000","last_undegraded":"2023-10-16T11:19:36.631343+0000","last_fullsized":"2023-10-16T11:19:36.631343+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:09.892801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615802+0000","last_change":"2023-10-16T11:19:23.551775+0000","last_active":"2023-10-16T11:19:36.615802+0000","last_peered":"2023-10-16T11:19:36.615802+0000","last_clean":"2023-10-16T11:19:36.615802+0000","last_became_active":"2023-10-16T11:19:23.551490+0000","last_became_peered":"2023-10-16T11:19:23.551490+0000","last_unstale":"2023-10-16T11:19:36.615802+0000","last_undegraded":"2023-10-16T11:19:36.615802+0000","last_fullsized":"2023-10-16T11:19:36.615802+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618085+0000","last_change":"2023-10-16T11:19:35.625342+0000","last_active":"2023-10-16T11:19:36.618085+0000","last_peered":"2023-10-16T11:19:36.618085+0000","last_clean":"2023-10-16T11:19:36.618085+0000","last_became_active":"2023-10-16T11:19:35.624933+0000","last_became_peered":"2023-10-16T11:19:35.624933+0000","last_unstale":"2023-10-16T11:19:36.618085+0000","last_undegraded":"2023-10-16T11:19:36.618085+0000","last_fullsized":"2023-10-16T11:19:36.618085+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:20.226818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892577+0000","last_change":"2023-10-16T11:19:36.630442+0000","last_active":"2023-10-16T11:19:37.892577+0000","last_peered":"2023-10-16T11:19:37.892577+0000","last_clean":"2023-10-16T11:19:37.892577+0000","last_became_active":"2023-10-16T11:19:36.630151+0000","last_became_peered":"2023-10-16T11:19:36.630151+0000","last_unstale":"2023-10-16T11:19:37.892577+0000","last_undegraded":"2023-10-16T11:19:37.892577+0000","last_fullsized":"2023-10-16T11:19:37.892577+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:33.781861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622379+0000","last_change":"2023-10-16T11:19:36.633999+0000","last_active":"2023-10-16T11:19:37.622379+0000","last_peered":"2023-10-16T11:19:37.622379+0000","last_clean":"2023-10-16T11:19:37.622379+0000","last_became_active":"2023-10-16T11:19:36.633722+0000","last_became_peered":"2023-10-16T11:19:36.633722+0000","last_unstale":"2023-10-16T11:19:37.622379+0000","last_undegraded":"2023-10-16T11:19:37.622379+0000","last_fullsized":"2023-10-16T11:19:37.622379+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618948+0000","last_change":"2023-10-16T11:19:23.544178+0000","last_active":"2023-10-16T11:19:36.618948+0000","last_peered":"2023-10-16T11:19:36.618948+0000","last_clean":"2023-10-16T11:19:36.618948+0000","last_became_active":"2023-10-16T11:19:23.543898+0000","last_became_peered":"2023-10-16T11:19:23.543898+0000","last_unstale":"2023-10-16T11:19:36.618948+0000","last_undegraded":"2023-10-16T11:19:36.618948+0000","last_fullsized":"2023-10-16T11:19:36.618948+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893954+0000","last_change":"2023-10-16T11:19:36.638088+0000","last_active":"2023-10-16T11:19:37.893954+0000","last_peered":"2023-10-16T11:19:37.893954+0000","last_clean":"2023-10-16T11:19:37.893954+0000","last_became_active":"2023-10-16T11:19:36.637717+0000","last_became_peered":"2023-10-16T11:19:36.637717+0000","last_unstale":"2023-10-16T11:19:37.893954+0000","last_undegraded":"2023-10-16T11:19:37.893954+0000","last_fullsized":"2023-10-16T11:19:37.893954+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:19.443732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621366+0000","last_change":"2023-10-16T11:19:23.542878+0000","last_active":"2023-10-16T11:19:36.621366+0000","last_peered":"2023-10-16T11:19:36.621366+0000","last_clean":"2023-10-16T11:19:36.621366+0000","last_became_active":"2023-10-16T11:19:23.542593+0000","last_became_peered":"2023-10-16T11:19:23.542593+0000","last_unstale":"2023-10-16T11:19:36.621366+0000","last_undegraded":"2023-10-16T11:19:36.621366+0000","last_fullsized":"2023-10-16T11:19:36.621366+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:39.898154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616551+0000","last_change":"2023-10-16T11:19:23.547762+0000","last_active":"2023-10-16T11:19:36.616551+0000","last_peered":"2023-10-16T11:19:36.616551+0000","last_clean":"2023-10-16T11:19:36.616551+0000","last_became_active":"2023-10-16T11:19:23.547353+0000","last_became_peered":"2023-10-16T11:19:23.547353+0000","last_unstale":"2023-10-16T11:19:36.616551+0000","last_undegraded":"2023-10-16T11:19:36.616551+0000","last_fullsized":"2023-10-16T11:19:36.616551+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893121+0000","last_change":"2023-10-16T11:19:36.630687+0000","last_active":"2023-10-16T11:19:37.893121+0000","last_peered":"2023-10-16T11:19:37.893121+0000","last_clean":"2023-10-16T11:19:37.893121+0000","last_became_active":"2023-10-16T11:19:36.629515+0000","last_became_peered":"2023-10-16T11:19:36.629515+0000","last_unstale":"2023-10-16T11:19:37.893121+0000","last_undegraded":"2023-10-16T11:19:37.893121+0000","last_fullsized":"2023-10-16T11:19:37.893121+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:25.525530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893412+0000","last_change":"2023-10-16T11:19:36.634680+0000","last_active":"2023-10-16T11:19:37.893412+0000","last_peered":"2023-10-16T11:19:37.893412+0000","last_clean":"2023-10-16T11:19:37.893412+0000","last_became_active":"2023-10-16T11:19:36.632581+0000","last_became_peered":"2023-10-16T11:19:36.632581+0000","last_unstale":"2023-10-16T11:19:37.893412+0000","last_undegraded":"2023-10-16T11:19:37.893412+0000","last_fullsized":"2023-10-16T11:19:37.893412+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.278663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630199+0000","last_change":"2023-10-16T11:19:23.565127+0000","last_active":"2023-10-16T11:19:36.630199+0000","last_peered":"2023-10-16T11:19:36.630199+0000","last_clean":"2023-10-16T11:19:36.630199+0000","last_became_active":"2023-10-16T11:19:23.564852+0000","last_became_peered":"2023-10-16T11:19:23.564852+0000","last_unstale":"2023-10-16T11:19:36.630199+0000","last_undegraded":"2023-10-16T11:19:36.630199+0000","last_fullsized":"2023-10-16T11:19:36.630199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629214+0000","last_change":"2023-10-16T11:19:23.555152+0000","last_active":"2023-10-16T11:19:36.629214+0000","last_peered":"2023-10-16T11:19:36.629214+0000","last_clean":"2023-10-16T11:19:36.629214+0000","last_became_active":"2023-10-16T11:19:23.554747+0000","last_became_peered":"2023-10-16T11:19:23.554747+0000","last_unstale":"2023-10-16T11:19:36.629214+0000","last_undegraded":"2023-10-16T11:19:36.629214+0000","last_fullsized":"2023-10-16T11:19:36.629214+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892848+0000","last_change":"2023-10-16T11:19:36.635095+0000","last_active":"2023-10-16T11:19:37.892848+0000","last_peered":"2023-10-16T11:19:37.892848+0000","last_clean":"2023-10-16T11:19:37.892848+0000","last_became_active":"2023-10-16T11:19:36.634778+0000","last_became_peered":"2023-10-16T11:19:36.634778+0000","last_unstale":"2023-10-16T11:19:37.892848+0000","last_undegraded":"2023-10-16T11:19:37.892848+0000","last_fullsized":"2023-10-16T11:19:37.892848+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:34.012796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630759+0000","last_change":"2023-10-16T11:19:23.559781+0000","last_active":"2023-10-16T11:19:36.630759+0000","last_peered":"2023-10-16T11:19:36.630759+0000","last_clean":"2023-10-16T11:19:36.630759+0000","last_became_active":"2023-10-16T11:19:23.559451+0000","last_became_peered":"2023-10-16T11:19:23.559451+0000","last_unstale":"2023-10-16T11:19:36.630759+0000","last_undegraded":"2023-10-16T11:19:36.630759+0000","last_fullsized":"2023-10-16T11:19:36.630759+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621656+0000","last_change":"2023-10-16T11:19:36.641041+0000","last_active":"2023-10-16T11:19:37.621656+0000","last_peered":"2023-10-16T11:19:37.621656+0000","last_clean":"2023-10-16T11:19:37.621656+0000","last_became_active":"2023-10-16T11:19:36.640761+0000","last_became_peered":"2023-10-16T11:19:36.640761+0000","last_unstale":"2023-10-16T11:19:37.621656+0000","last_undegraded":"2023-10-16T11:19:37.621656+0000","last_fullsized":"2023-10-16T11:19:37.621656+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:26.350349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622347+0000","last_change":"2023-10-16T11:19:23.546460+0000","last_active":"2023-10-16T11:19:36.622347+0000","last_peered":"2023-10-16T11:19:36.622347+0000","last_clean":"2023-10-16T11:19:36.622347+0000","last_became_active":"2023-10-16T11:19:23.546187+0000","last_became_peered":"2023-10-16T11:19:23.546187+0000","last_unstale":"2023-10-16T11:19:36.622347+0000","last_undegraded":"2023-10-16T11:19:36.622347+0000","last_fullsized":"2023-10-16T11:19:36.622347+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621306+0000","last_change":"2023-10-16T11:19:36.630530+0000","last_active":"2023-10-16T11:19:37.621306+0000","last_peered":"2023-10-16T11:19:37.621306+0000","last_clean":"2023-10-16T11:19:37.621306+0000","last_became_active":"2023-10-16T11:19:36.630239+0000","last_became_peered":"2023-10-16T11:19:36.630239+0000","last_unstale":"2023-10-16T11:19:37.621306+0000","last_undegraded":"2023-10-16T11:19:37.621306+0000","last_fullsized":"2023-10-16T11:19:37.621306+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616639+0000","last_change":"2023-10-16T11:19:23.546857+0000","last_active":"2023-10-16T11:19:36.616639+0000","last_peered":"2023-10-16T11:19:36.616639+0000","last_clean":"2023-10-16T11:19:36.616639+0000","last_became_active":"2023-10-16T11:19:23.546592+0000","last_became_peered":"2023-10-16T11:19:23.546592+0000","last_unstale":"2023-10-16T11:19:36.616639+0000","last_undegraded":"2023-10-16T11:19:36.616639+0000","last_fullsized":"2023-10-16T11:19:36.616639+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893169+0000","last_change":"2023-10-16T11:19:36.622716+0000","last_active":"2023-10-16T11:19:37.893169+0000","last_peered":"2023-10-16T11:19:37.893169+0000","last_clean":"2023-10-16T11:19:37.893169+0000","last_became_active":"2023-10-16T11:19:36.622420+0000","last_became_peered":"2023-10-16T11:19:36.622420+0000","last_unstale":"2023-10-16T11:19:37.893169+0000","last_undegraded":"2023-10-16T11:19:37.893169+0000","last_fullsized":"2023-10-16T11:19:37.893169+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:58.765425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893941+0000","last_change":"2023-10-16T11:19:36.633980+0000","last_active":"2023-10-16T11:19:37.893941+0000","last_peered":"2023-10-16T11:19:37.893941+0000","last_clean":"2023-10-16T11:19:37.893941+0000","last_became_active":"2023-10-16T11:19:36.633643+0000","last_became_peered":"2023-10-16T11:19:36.633643+0000","last_unstale":"2023-10-16T11:19:37.893941+0000","last_undegraded":"2023-10-16T11:19:37.893941+0000","last_fullsized":"2023-10-16T11:19:37.893941+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:27.398549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617918+0000","last_change":"2023-10-16T11:19:23.548868+0000","last_active":"2023-10-16T11:19:36.617918+0000","last_peered":"2023-10-16T11:19:36.617918+0000","last_clean":"2023-10-16T11:19:36.617918+0000","last_became_active":"2023-10-16T11:19:23.548583+0000","last_became_peered":"2023-10-16T11:19:23.548583+0000","last_unstale":"2023-10-16T11:19:36.617918+0000","last_undegraded":"2023-10-16T11:19:36.617918+0000","last_fullsized":"2023-10-16T11:19:36.617918+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:02.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622147+0000","last_change":"2023-10-16T11:19:36.628944+0000","last_active":"2023-10-16T11:19:37.622147+0000","last_peered":"2023-10-16T11:19:37.622147+0000","last_clean":"2023-10-16T11:19:37.622147+0000","last_became_active":"2023-10-16T11:19:36.628652+0000","last_became_peered":"2023-10-16T11:19:36.628652+0000","last_unstale":"2023-10-16T11:19:37.622147+0000","last_undegraded":"2023-10-16T11:19:37.622147+0000","last_fullsized":"2023-10-16T11:19:37.622147+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:18.537927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624713+0000","last_change":"2023-10-16T11:19:23.552048+0000","last_active":"2023-10-16T11:19:36.624713+0000","last_peered":"2023-10-16T11:19:36.624713+0000","last_clean":"2023-10-16T11:19:36.624713+0000","last_became_active":"2023-10-16T11:19:23.551676+0000","last_became_peered":"2023-10-16T11:19:23.551676+0000","last_unstale":"2023-10-16T11:19:36.624713+0000","last_undegraded":"2023-10-16T11:19:36.624713+0000","last_fullsized":"2023-10-16T11:19:36.624713+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:16.450373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893328+0000","last_change":"2023-10-16T11:19:36.626030+0000","last_active":"2023-10-16T11:19:37.893328+0000","last_peered":"2023-10-16T11:19:37.893328+0000","last_clean":"2023-10-16T11:19:37.893328+0000","last_became_active":"2023-10-16T11:19:36.625727+0000","last_became_peered":"2023-10-16T11:19:36.625727+0000","last_unstale":"2023-10-16T11:19:37.893328+0000","last_undegraded":"2023-10-16T11:19:37.893328+0000","last_fullsized":"2023-10-16T11:19:37.893328+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:07:58.374107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624300+0000","last_change":"2023-10-16T11:19:23.546038+0000","last_active":"2023-10-16T11:19:36.624300+0000","last_peered":"2023-10-16T11:19:36.624300+0000","last_clean":"2023-10-16T11:19:36.624300+0000","last_became_active":"2023-10-16T11:19:23.545752+0000","last_became_peered":"2023-10-16T11:19:23.545752+0000","last_unstale":"2023-10-16T11:19:36.624300+0000","last_undegraded":"2023-10-16T11:19:36.624300+0000","last_fullsized":"2023-10-16T11:19:36.624300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:13.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893888+0000","last_change":"2023-10-16T11:19:36.629691+0000","last_active":"2023-10-16T11:19:37.893888+0000","last_peered":"2023-10-16T11:19:37.893888+0000","last_clean":"2023-10-16T11:19:37.893888+0000","last_became_active":"2023-10-16T11:19:36.629384+0000","last_became_peered":"2023-10-16T11:19:36.629384+0000","last_unstale":"2023-10-16T11:19:37.893888+0000","last_undegraded":"2023-10-16T11:19:37.893888+0000","last_fullsized":"2023-10-16T11:19:37.893888+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:15.903147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619019+0000","last_change":"2023-10-16T11:19:23.550209+0000","last_active":"2023-10-16T11:19:36.619019+0000","last_peered":"2023-10-16T11:19:36.619019+0000","last_clean":"2023-10-16T11:19:36.619019+0000","last_became_active":"2023-10-16T11:19:23.549849+0000","last_became_peered":"2023-10-16T11:19:23.549849+0000","last_unstale":"2023-10-16T11:19:36.619019+0000","last_undegraded":"2023-10-16T11:19:36.619019+0000","last_fullsized":"2023-10-16T11:19:36.619019+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:29.715382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621124+0000","last_change":"2023-10-16T11:19:36.631361+0000","last_active":"2023-10-16T11:19:37.621124+0000","last_peered":"2023-10-16T11:19:37.621124+0000","last_clean":"2023-10-16T11:19:37.621124+0000","last_became_active":"2023-10-16T11:19:36.631013+0000","last_became_peered":"2023-10-16T11:19:36.631013+0000","last_unstale":"2023-10-16T11:19:37.621124+0000","last_undegraded":"2023-10-16T11:19:37.621124+0000","last_fullsized":"2023-10-16T11:19:37.621124+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615695+0000","last_change":"2023-10-16T11:19:23.542321+0000","last_active":"2023-10-16T11:19:36.615695+0000","last_peered":"2023-10-16T11:19:36.615695+0000","last_clean":"2023-10-16T11:19:36.615695+0000","last_became_active":"2023-10-16T11:19:23.541862+0000","last_became_peered":"2023-10-16T11:19:23.541862+0000","last_unstale":"2023-10-16T11:19:36.615695+0000","last_undegraded":"2023-10-16T11:19:36.615695+0000","last_fullsized":"2023-10-16T11:19:36.615695+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:24:50.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892711+0000","last_change":"2023-10-16T11:19:36.620937+0000","last_active":"2023-10-16T11:19:37.892711+0000","last_peered":"2023-10-16T11:19:37.892711+0000","last_clean":"2023-10-16T11:19:37.892711+0000","last_became_active":"2023-10-16T11:19:36.620641+0000","last_became_peered":"2023-10-16T11:19:36.620641+0000","last_unstale":"2023-10-16T11:19:37.892711+0000","last_undegraded":"2023-10-16T11:19:37.892711+0000","last_fullsized":"2023-10-16T11:19:37.892711+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:15.827874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623487+0000","last_change":"2023-10-16T11:19:23.547740+0000","last_active":"2023-10-16T11:19:36.623487+0000","last_peered":"2023-10-16T11:19:36.623487+0000","last_clean":"2023-10-16T11:19:36.623487+0000","last_became_active":"2023-10-16T11:19:23.546421+0000","last_became_peered":"2023-10-16T11:19:23.546421+0000","last_unstale":"2023-10-16T11:19:36.623487+0000","last_undegraded":"2023-10-16T11:19:36.623487+0000","last_fullsized":"2023-10-16T11:19:36.623487+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:27.685924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892763+0000","last_change":"2023-10-16T11:19:36.631333+0000","last_active":"2023-10-16T11:19:37.892763+0000","last_peered":"2023-10-16T11:19:37.892763+0000","last_clean":"2023-10-16T11:19:37.892763+0000","last_became_active":"2023-10-16T11:19:36.630735+0000","last_became_peered":"2023-10-16T11:19:36.630735+0000","last_unstale":"2023-10-16T11:19:37.892763+0000","last_undegraded":"2023-10-16T11:19:37.892763+0000","last_fullsized":"2023-10-16T11:19:37.892763+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:26.684291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623942+0000","last_change":"2023-10-16T11:19:23.549769+0000","last_active":"2023-10-16T11:19:36.623942+0000","last_peered":"2023-10-16T11:19:36.623942+0000","last_clean":"2023-10-16T11:19:36.623942+0000","last_became_active":"2023-10-16T11:19:23.549505+0000","last_became_peered":"2023-10-16T11:19:23.549505+0000","last_unstale":"2023-10-16T11:19:36.623942+0000","last_undegraded":"2023-10-16T11:19:36.623942+0000","last_fullsized":"2023-10-16T11:19:36.623942+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:20.931218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892826+0000","last_change":"2023-10-16T11:19:36.631224+0000","last_active":"2023-10-16T11:19:37.892826+0000","last_peered":"2023-10-16T11:19:37.892826+0000","last_clean":"2023-10-16T11:19:37.892826+0000","last_became_active":"2023-10-16T11:19:36.630966+0000","last_became_peered":"2023-10-16T11:19:36.630966+0000","last_unstale":"2023-10-16T11:19:37.892826+0000","last_undegraded":"2023-10-16T11:19:37.892826+0000","last_fullsized":"2023-10-16T11:19:37.892826+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:45.783168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622027+0000","last_change":"2023-10-16T11:19:23.548276+0000","last_active":"2023-10-16T11:19:36.622027+0000","last_peered":"2023-10-16T11:19:36.622027+0000","last_clean":"2023-10-16T11:19:36.622027+0000","last_became_active":"2023-10-16T11:19:23.547177+0000","last_became_peered":"2023-10-16T11:19:23.547177+0000","last_unstale":"2023-10-16T11:19:36.622027+0000","last_undegraded":"2023-10-16T11:19:36.622027+0000","last_fullsized":"2023-10-16T11:19:36.622027+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893905+0000","last_change":"2023-10-16T11:19:36.635809+0000","last_active":"2023-10-16T11:19:37.893905+0000","last_peered":"2023-10-16T11:19:37.893905+0000","last_clean":"2023-10-16T11:19:37.893905+0000","last_became_active":"2023-10-16T11:19:36.635543+0000","last_became_peered":"2023-10-16T11:19:36.635543+0000","last_unstale":"2023-10-16T11:19:37.893905+0000","last_undegraded":"2023-10-16T11:19:37.893905+0000","last_fullsized":"2023-10-16T11:19:37.893905+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:06:05.832943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621596+0000","last_change":"2023-10-16T11:19:23.563754+0000","last_active":"2023-10-16T11:19:36.621596+0000","last_peered":"2023-10-16T11:19:36.621596+0000","last_clean":"2023-10-16T11:19:36.621596+0000","last_became_active":"2023-10-16T11:19:23.563487+0000","last_became_peered":"2023-10-16T11:19:23.563487+0000","last_unstale":"2023-10-16T11:19:36.621596+0000","last_undegraded":"2023-10-16T11:19:36.621596+0000","last_fullsized":"2023-10-16T11:19:36.621596+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:13.814299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893984+0000","last_change":"2023-10-16T11:19:36.639804+0000","last_active":"2023-10-16T11:19:37.893984+0000","last_peered":"2023-10-16T11:19:37.893984+0000","last_clean":"2023-10-16T11:19:37.893984+0000","last_became_active":"2023-10-16T11:19:36.639421+0000","last_became_peered":"2023-10-16T11:19:36.639421+0000","last_unstale":"2023-10-16T11:19:37.893984+0000","last_undegraded":"2023-10-16T11:19:37.893984+0000","last_fullsized":"2023-10-16T11:19:37.893984+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:21.232537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619182+0000","last_change":"2023-10-16T11:19:23.551280+0000","last_active":"2023-10-16T11:19:36.619182+0000","last_peered":"2023-10-16T11:19:36.619182+0000","last_clean":"2023-10-16T11:19:36.619182+0000","last_became_active":"2023-10-16T11:19:23.551018+0000","last_became_peered":"2023-10-16T11:19:23.551018+0000","last_unstale":"2023-10-16T11:19:36.619182+0000","last_undegraded":"2023-10-16T11:19:36.619182+0000","last_fullsized":"2023-10-16T11:19:36.619182+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:46.256661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893878+0000","last_change":"2023-10-16T11:19:36.627799+0000","last_active":"2023-10-16T11:19:37.893878+0000","last_peered":"2023-10-16T11:19:37.893878+0000","last_clean":"2023-10-16T11:19:37.893878+0000","last_became_active":"2023-10-16T11:19:36.627495+0000","last_became_peered":"2023-10-16T11:19:36.627495+0000","last_unstale":"2023-10-16T11:19:37.893878+0000","last_undegraded":"2023-10-16T11:19:37.893878+0000","last_fullsized":"2023-10-16T11:19:37.893878+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:32.348270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620399+0000","last_change":"2023-10-16T11:19:23.559938+0000","last_active":"2023-10-16T11:19:36.620399+0000","last_peered":"2023-10-16T11:19:36.620399+0000","last_clean":"2023-10-16T11:19:36.620399+0000","last_became_active":"2023-10-16T11:19:23.559569+0000","last_became_peered":"2023-10-16T11:19:23.559569+0000","last_unstale":"2023-10-16T11:19:36.620399+0000","last_undegraded":"2023-10-16T11:19:36.620399+0000","last_fullsized":"2023-10-16T11:19:36.620399+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893192+0000","last_change":"2023-10-16T11:19:36.625387+0000","last_active":"2023-10-16T11:19:37.893192+0000","last_peered":"2023-10-16T11:19:37.893192+0000","last_clean":"2023-10-16T11:19:37.893192+0000","last_became_active":"2023-10-16T11:19:36.625097+0000","last_became_peered":"2023-10-16T11:19:36.625097+0000","last_unstale":"2023-10-16T11:19:37.893192+0000","last_undegraded":"2023-10-16T11:19:37.893192+0000","last_fullsized":"2023-10-16T11:19:37.893192+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:12.164686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630199+0000","last_change":"2023-10-16T11:19:23.560050+0000","last_active":"2023-10-16T11:19:36.630199+0000","last_peered":"2023-10-16T11:19:36.630199+0000","last_clean":"2023-10-16T11:19:36.630199+0000","last_became_active":"2023-10-16T11:19:23.559558+0000","last_became_peered":"2023-10-16T11:19:23.559558+0000","last_unstale":"2023-10-16T11:19:36.630199+0000","last_undegraded":"2023-10-16T11:19:36.630199+0000","last_fullsized":"2023-10-16T11:19:36.630199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621911+0000","last_change":"2023-10-16T11:19:36.629616+0000","last_active":"2023-10-16T11:19:37.621911+0000","last_peered":"2023-10-16T11:19:37.621911+0000","last_clean":"2023-10-16T11:19:37.621911+0000","last_became_active":"2023-10-16T11:19:36.629056+0000","last_became_peered":"2023-10-16T11:19:36.629056+0000","last_unstale":"2023-10-16T11:19:37.621911+0000","last_undegraded":"2023-10-16T11:19:37.621911+0000","last_fullsized":"2023-10-16T11:19:37.621911+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:11.302811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.632358+0000","last_change":"2023-10-16T11:19:23.560016+0000","last_active":"2023-10-16T11:19:36.632358+0000","last_peered":"2023-10-16T11:19:36.632358+0000","last_clean":"2023-10-16T11:19:36.632358+0000","last_became_active":"2023-10-16T11:19:23.559619+0000","last_became_peered":"2023-10-16T11:19:23.559619+0000","last_unstale":"2023-10-16T11:19:36.632358+0000","last_undegraded":"2023-10-16T11:19:36.632358+0000","last_fullsized":"2023-10-16T11:19:36.632358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:04.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893587+0000","last_change":"2023-10-16T11:19:36.628684+0000","last_active":"2023-10-16T11:19:37.893587+0000","last_peered":"2023-10-16T11:19:37.893587+0000","last_clean":"2023-10-16T11:19:37.893587+0000","last_became_active":"2023-10-16T11:19:36.628264+0000","last_became_peered":"2023-10-16T11:19:36.628264+0000","last_unstale":"2023-10-16T11:19:37.893587+0000","last_undegraded":"2023-10-16T11:19:37.893587+0000","last_fullsized":"2023-10-16T11:19:37.893587+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:41:54.029905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629289+0000","last_change":"2023-10-16T11:19:23.564191+0000","last_active":"2023-10-16T11:19:36.629289+0000","last_peered":"2023-10-16T11:19:36.629289+0000","last_clean":"2023-10-16T11:19:36.629289+0000","last_became_active":"2023-10-16T11:19:23.563291+0000","last_became_peered":"2023-10-16T11:19:23.563291+0000","last_unstale":"2023-10-16T11:19:36.629289+0000","last_undegraded":"2023-10-16T11:19:36.629289+0000","last_fullsized":"2023-10-16T11:19:36.629289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894719+0000","last_change":"2023-10-16T11:19:36.627078+0000","last_active":"2023-10-16T11:19:37.894719+0000","last_peered":"2023-10-16T11:19:37.894719+0000","last_clean":"2023-10-16T11:19:37.894719+0000","last_became_active":"2023-10-16T11:19:36.626791+0000","last_became_peered":"2023-10-16T11:19:36.626791+0000","last_unstale":"2023-10-16T11:19:37.894719+0000","last_undegraded":"2023-10-16T11:19:37.894719+0000","last_fullsized":"2023-10-16T11:19:37.894719+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:14.780239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623858+0000","last_change":"2023-10-16T11:19:23.546735+0000","last_active":"2023-10-16T11:19:36.623858+0000","last_peered":"2023-10-16T11:19:36.623858+0000","last_clean":"2023-10-16T11:19:36.623858+0000","last_became_active":"2023-10-16T11:19:23.546404+0000","last_became_peered":"2023-10-16T11:19:23.546404+0000","last_unstale":"2023-10-16T11:19:36.623858+0000","last_undegraded":"2023-10-16T11:19:36.623858+0000","last_fullsized":"2023-10-16T11:19:36.623858+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:29.845437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.620865+0000","last_change":"2023-10-16T11:19:36.630872+0000","last_active":"2023-10-16T11:19:37.620865+0000","last_peered":"2023-10-16T11:19:37.620865+0000","last_clean":"2023-10-16T11:19:37.620865+0000","last_became_active":"2023-10-16T11:19:36.630391+0000","last_became_peered":"2023-10-16T11:19:36.630391+0000","last_unstale":"2023-10-16T11:19:37.620865+0000","last_undegraded":"2023-10-16T11:19:37.620865+0000","last_fullsized":"2023-10-16T11:19:37.620865+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:23.825418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616656+0000","last_change":"2023-10-16T11:19:23.552080+0000","last_active":"2023-10-16T11:19:36.616656+0000","last_peered":"2023-10-16T11:19:36.616656+0000","last_clean":"2023-10-16T11:19:36.616656+0000","last_became_active":"2023-10-16T11:19:23.551779+0000","last_became_peered":"2023-10-16T11:19:23.551779+0000","last_unstale":"2023-10-16T11:19:36.616656+0000","last_undegraded":"2023-10-16T11:19:36.616656+0000","last_fullsized":"2023-10-16T11:19:36.616656+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894374+0000","last_change":"2023-10-16T11:19:36.636621+0000","last_active":"2023-10-16T11:19:37.894374+0000","last_peered":"2023-10-16T11:19:37.894374+0000","last_clean":"2023-10-16T11:19:37.894374+0000","last_became_active":"2023-10-16T11:19:36.636222+0000","last_became_peered":"2023-10-16T11:19:36.636222+0000","last_unstale":"2023-10-16T11:19:37.894374+0000","last_undegraded":"2023-10-16T11:19:37.894374+0000","last_fullsized":"2023-10-16T11:19:37.894374+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:51.038856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615178+0000","last_change":"2023-10-16T11:19:23.550227+0000","last_active":"2023-10-16T11:19:36.615178+0000","last_peered":"2023-10-16T11:19:36.615178+0000","last_clean":"2023-10-16T11:19:36.615178+0000","last_became_active":"2023-10-16T11:19:23.549938+0000","last_became_peered":"2023-10-16T11:19:23.549938+0000","last_unstale":"2023-10-16T11:19:36.615178+0000","last_undegraded":"2023-10-16T11:19:36.615178+0000","last_fullsized":"2023-10-16T11:19:36.615178+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:13.268771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893818+0000","last_change":"2023-10-16T11:19:36.636806+0000","last_active":"2023-10-16T11:19:37.893818+0000","last_peered":"2023-10-16T11:19:37.893818+0000","last_clean":"2023-10-16T11:19:37.893818+0000","last_became_active":"2023-10-16T11:19:36.636526+0000","last_became_peered":"2023-10-16T11:19:36.636526+0000","last_unstale":"2023-10-16T11:19:37.893818+0000","last_undegraded":"2023-10-16T11:19:37.893818+0000","last_fullsized":"2023-10-16T11:19:37.893818+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:26.018990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619096+0000","last_change":"2023-10-16T11:19:23.547259+0000","last_active":"2023-10-16T11:19:36.619096+0000","last_peered":"2023-10-16T11:19:36.619096+0000","last_clean":"2023-10-16T11:19:36.619096+0000","last_became_active":"2023-10-16T11:19:23.546957+0000","last_became_peered":"2023-10-16T11:19:23.546957+0000","last_unstale":"2023-10-16T11:19:36.619096+0000","last_undegraded":"2023-10-16T11:19:36.619096+0000","last_fullsized":"2023-10-16T11:19:36.619096+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893676+0000","last_change":"2023-10-16T11:19:36.626949+0000","last_active":"2023-10-16T11:19:37.893676+0000","last_peered":"2023-10-16T11:19:37.893676+0000","last_clean":"2023-10-16T11:19:37.893676+0000","last_became_active":"2023-10-16T11:19:36.626670+0000","last_became_peered":"2023-10-16T11:19:36.626670+0000","last_unstale":"2023-10-16T11:19:37.893676+0000","last_undegraded":"2023-10-16T11:19:37.893676+0000","last_fullsized":"2023-10-16T11:19:37.893676+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:14.675395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615351+0000","last_change":"2023-10-16T11:19:23.550179+0000","last_active":"2023-10-16T11:19:36.615351+0000","last_peered":"2023-10-16T11:19:36.615351+0000","last_clean":"2023-10-16T11:19:36.615351+0000","last_became_active":"2023-10-16T11:19:23.549811+0000","last_became_peered":"2023-10-16T11:19:23.549811+0000","last_unstale":"2023-10-16T11:19:36.615351+0000","last_undegraded":"2023-10-16T11:19:36.615351+0000","last_fullsized":"2023-10-16T11:19:36.615351+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:58.221233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894684+0000","last_change":"2023-10-16T11:19:36.632889+0000","last_active":"2023-10-16T11:19:37.894684+0000","last_peered":"2023-10-16T11:19:37.894684+0000","last_clean":"2023-10-16T11:19:37.894684+0000","last_became_active":"2023-10-16T11:19:36.632268+0000","last_became_peered":"2023-10-16T11:19:36.632268+0000","last_unstale":"2023-10-16T11:19:37.894684+0000","last_undegraded":"2023-10-16T11:19:37.894684+0000","last_fullsized":"2023-10-16T11:19:37.894684+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:11.302811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616511+0000","last_change":"2023-10-16T11:19:23.541856+0000","last_active":"2023-10-16T11:19:36.616511+0000","last_peered":"2023-10-16T11:19:36.616511+0000","last_clean":"2023-10-16T11:19:36.616511+0000","last_became_active":"2023-10-16T11:19:23.541549+0000","last_became_peered":"2023-10-16T11:19:23.541549+0000","last_unstale":"2023-10-16T11:19:36.616511+0000","last_undegraded":"2023-10-16T11:19:36.616511+0000","last_fullsized":"2023-10-16T11:19:36.616511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:07.709989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892542+0000","last_change":"2023-10-16T11:19:36.625337+0000","last_active":"2023-10-16T11:19:37.892542+0000","last_peered":"2023-10-16T11:19:37.892542+0000","last_clean":"2023-10-16T11:19:37.892542+0000","last_became_active":"2023-10-16T11:19:36.624796+0000","last_became_peered":"2023-10-16T11:19:36.624796+0000","last_unstale":"2023-10-16T11:19:37.892542+0000","last_undegraded":"2023-10-16T11:19:37.892542+0000","last_fullsized":"2023-10-16T11:19:37.892542+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:27.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619098+0000","last_change":"2023-10-16T11:19:23.547136+0000","last_active":"2023-10-16T11:19:36.619098+0000","last_peered":"2023-10-16T11:19:36.619098+0000","last_clean":"2023-10-16T11:19:36.619098+0000","last_became_active":"2023-10-16T11:19:23.546834+0000","last_became_peered":"2023-10-16T11:19:23.546834+0000","last_unstale":"2023-10-16T11:19:36.619098+0000","last_undegraded":"2023-10-16T11:19:36.619098+0000","last_fullsized":"2023-10-16T11:19:36.619098+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:45.683847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892567+0000","last_change":"2023-10-16T11:19:36.628989+0000","last_active":"2023-10-16T11:19:37.892567+0000","last_peered":"2023-10-16T11:19:37.892567+0000","last_clean":"2023-10-16T11:19:37.892567+0000","last_became_active":"2023-10-16T11:19:36.628665+0000","last_became_peered":"2023-10-16T11:19:36.628665+0000","last_unstale":"2023-10-16T11:19:37.892567+0000","last_undegraded":"2023-10-16T11:19:37.892567+0000","last_fullsized":"2023-10-16T11:19:37.892567+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:42.796960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629084+0000","last_change":"2023-10-16T11:19:23.552160+0000","last_active":"2023-10-16T11:19:36.629084+0000","last_peered":"2023-10-16T11:19:36.629084+0000","last_clean":"2023-10-16T11:19:36.629084+0000","last_became_active":"2023-10-16T11:19:23.551805+0000","last_became_peered":"2023-10-16T11:19:23.551805+0000","last_unstale":"2023-10-16T11:19:36.629084+0000","last_undegraded":"2023-10-16T11:19:36.629084+0000","last_fullsized":"2023-10-16T11:19:36.629084+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621385+0000","last_change":"2023-10-16T11:19:36.634749+0000","last_active":"2023-10-16T11:19:37.621385+0000","last_peered":"2023-10-16T11:19:37.621385+0000","last_clean":"2023-10-16T11:19:37.621385+0000","last_became_active":"2023-10-16T11:19:36.634467+0000","last_became_peered":"2023-10-16T11:19:36.634467+0000","last_unstale":"2023-10-16T11:19:37.621385+0000","last_undegraded":"2023-10-16T11:19:37.621385+0000","last_fullsized":"2023-10-16T11:19:37.621385+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616111+0000","last_change":"2023-10-16T11:19:23.552427+0000","last_active":"2023-10-16T11:19:36.616111+0000","last_peered":"2023-10-16T11:19:36.616111+0000","last_clean":"2023-10-16T11:19:36.616111+0000","last_became_active":"2023-10-16T11:19:23.552156+0000","last_became_peered":"2023-10-16T11:19:23.552156+0000","last_unstale":"2023-10-16T11:19:36.616111+0000","last_undegraded":"2023-10-16T11:19:36.616111+0000","last_fullsized":"2023-10-16T11:19:36.616111+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893142+0000","last_change":"2023-10-16T11:19:36.633460+0000","last_active":"2023-10-16T11:19:37.893142+0000","last_peered":"2023-10-16T11:19:37.893142+0000","last_clean":"2023-10-16T11:19:37.893142+0000","last_became_active":"2023-10-16T11:19:36.631460+0000","last_became_peered":"2023-10-16T11:19:36.631460+0000","last_unstale":"2023-10-16T11:19:37.893142+0000","last_undegraded":"2023-10-16T11:19:37.893142+0000","last_fullsized":"2023-10-16T11:19:37.893142+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621912+0000","last_change":"2023-10-16T11:19:23.547472+0000","last_active":"2023-10-16T11:19:36.621912+0000","last_peered":"2023-10-16T11:19:36.621912+0000","last_clean":"2023-10-16T11:19:36.621912+0000","last_became_active":"2023-10-16T11:19:23.547148+0000","last_became_peered":"2023-10-16T11:19:23.547148+0000","last_unstale":"2023-10-16T11:19:36.621912+0000","last_undegraded":"2023-10-16T11:19:36.621912+0000","last_fullsized":"2023-10-16T11:19:36.621912+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3657,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543282,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52516,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319324,"statfs":{"total":96636764160,"available":96582987776,"internally_reserved":0,"allocated":3366912,"data_stored":2012088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.27800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42399999999999999,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69599999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.57799999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.79900000000000004,"15min":0.86299999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.307},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.002},"last":0.71199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33100000000000002},"max":{"1min":8.1950000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54400000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.504},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.871},"last":0.52800000000000002}]}]},{"osd":6,"up_from":14,"seq":60129543281,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2768896,"data_stored":1419429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.875},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.51200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.56999999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46899999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.44800000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.752,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.48899999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.32100000000000001},"max":{"1min":0.79500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.91100000000000003},"last":0.41799999999999998}]}]},{"osd":1,"up_from":14,"seq":60129543281,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60256,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311584,"statfs":{"total":96636764160,"available":96575062016,"internally_reserved":0,"allocated":2768896,"data_stored":1418937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.224},"max":{"1min":0.60199999999999998,"5min":0.629,"15min":0.68700000000000006},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.434},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.62,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.42799999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.55300000000000005,"5min":0.58699999999999997,"15min":0.98899999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.78300000000000003},"last":0.504},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.0289999999999999,"15min":1.097},"last":0.74199999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":1.147},"last":0.77900000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.88,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.83899999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":0.98899999999999999,"5min":1.004,"15min":1.0600000000000001},"last":0.876},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46300000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.1060000000000001},"last":0.73099999999999998}]}]},{"osd":0,"up_from":14,"seq":60129543282,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60836,"kb_used_data":3284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311004,"statfs":{"total":96636764160,"available":96574468096,"internally_reserved":0,"allocated":3362816,"data_stored":2009921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.46400000000000002,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":0.432},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58799999999999997,"5min":0.63700000000000001,"15min":0.70299999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.314}]},{"osd":3,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.68300000000000005},"last":0.377},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.45000000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.99199999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":1.008},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":1.004,"15min":1.008},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.623}]},{"osd":6,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.504},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53400000000000003}]}]},{"osd":2,"up_from":14,"seq":60129543281,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":2777088,"data_stored":1420459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":0.249},{"interface":"front","average":{"1min":0.436,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.63},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":0.311}]},{"osd":3,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.623,"15min":0.65500000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.81200000000000006},"last":0.35599999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":1.052,"15min":1.052},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68700000000000006,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61099999999999999}]}]},{"osd":3,"up_from":14,"seq":60129543282,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47788,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324052,"statfs":{"total":96636764160,"available":96587829248,"internally_reserved":0,"allocated":2785280,"data_stored":1420655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.36799999999999999,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.307}]},{"osd":1,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":0.60699999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36099999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":1.034,"15min":1.0740000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.80200000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.0489999999999999},"last":0.61699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543281,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":2781184,"data_stored":1422336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.628},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.35499999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.627},"min":{"1min":0.438,"5min":0.40400000000000003,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.501},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.879},"last":0.41299999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.69499999999999995,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.89500000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.629}]},{"osd":7,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":0.875,"15min":0.875},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.67000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129543282,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42196,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329644,"statfs":{"total":96636764160,"available":96593555456,"internally_reserved":0,"allocated":2760704,"data_stored":1419373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":1.873,"15min":1.873},"last":0.44400000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.39900000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":1.821,"15min":1.821},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55500000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":1.919,"15min":1.919},"last":0.36599999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.873},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.45400000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":2.02,"15min":2.02},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:19:41.950 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-16T11:19:42.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 Processor -- start 2023-10-16T11:19:42.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 -- start start 2023-10-16T11:19:42.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15e6d0 0x7fa91c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa91c15eab0 0x7fa91c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa91c0cb0c0 0x7fa91c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c06ed00 con 0x7fa91c15eab0 2023-10-16T11:19:42.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c0d02d0 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.069+0000 7fa9213d5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c06e0f0 con 0x7fa91c15e6d0 2023-10-16T11:19:42.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa91c15eab0 0x7fa91c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa91c0cb0c0 0x7fa91c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa91c15eab0 0x7fa91c15a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55886/0 (socket says 172.21.15.181:55886) 2023-10-16T11:19:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91b7fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa91c0cb0c0 0x7fa91c15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:60938/0 (socket says 172.21.15.181:60938) 2023-10-16T11:19:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 -- 172.21.15.181:0/271871576 learned_addr learned my addr 172.21.15.181:0/271871576 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91affd700 1 --2- 172.21.15.181:0/271871576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15e6d0 0x7fa91c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 -- 172.21.15.181:0/271871576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15e6d0 msgr2=0x7fa91c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 --2- 172.21.15.181:0/271871576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15e6d0 0x7fa91c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 -- 172.21.15.181:0/271871576 >> v2:172.21.15.190:3300/0 conn(0x7fa91c0cb0c0 msgr2=0x7fa91c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 --2- 172.21.15.181:0/271871576 >> v2:172.21.15.190:3300/0 conn(0x7fa91c0cb0c0 0x7fa91c15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 -- 172.21.15.181:0/271871576 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c06e3d0 con 0x7fa91c15eab0 2023-10-16T11:19:42.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa91a7fc700 1 --2- 172.21.15.181:0/271871576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c15eab0 0x7fa91c15a0f0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0705c0 tx=0x7fa910009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e876b113f041e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:42.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa919ffb700 1 -- 172.21.15.181:0/271871576 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011830 con 0x7fa91c15eab0 2023-10-16T11:19:42.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa919ffb700 1 -- 172.21.15.181:0/271871576 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910011990 con 0x7fa91c15eab0 2023-10-16T11:19:42.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.070+0000 7fa919ffb700 1 -- 172.21.15.181:0/271871576 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011d20 con 0x7fa91c15eab0 2023-10-16T11:19:42.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 -- 172.21.15.181:0/271871576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c15eab0 msgr2=0x7fa91c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 --2- 172.21.15.181:0/271871576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c15eab0 0x7fa91c15a0f0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0705c0 tx=0x7fa910009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 -- 172.21.15.181:0/271871576 shutdown_connections 2023-10-16T11:19:42.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 --2- 172.21.15.181:0/271871576 >> v2:172.21.15.190:3300/0 conn(0x7fa91c0cb0c0 0x7fa91c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 --2- 172.21.15.181:0/271871576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15e6d0 0x7fa91c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 --2- 172.21.15.181:0/271871576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c15eab0 0x7fa91c15a0f0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 -- 172.21.15.181:0/271871576 >> 172.21.15.181:0/271871576 conn(0x7fa91c0d04a0 msgr2=0x7fa91c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:42.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 -- 172.21.15.181:0/271871576 shutdown_connections 2023-10-16T11:19:42.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.071+0000 7fa9213d5700 1 -- 172.21.15.181:0/271871576 wait complete. 2023-10-16T11:19:42.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.072+0000 7fa9213d5700 1 Processor -- start 2023-10-16T11:19:42.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.072+0000 7fa9213d5700 1 -- start start 2023-10-16T11:19:42.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa9213d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c0cb0c0 0x7fa91c0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa9213d5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c15e6d0 0x7fa91c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa9213d5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15eab0 0x7fa91c1f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa9213d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1f1bc0 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa9213d5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0c6dc0 con 0x7fa91c15e6d0 2023-10-16T11:19:42.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa9213d5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1f1320 con 0x7fa91c15eab0 2023-10-16T11:19:42.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c0cb0c0 0x7fa91c0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c0cb0c0 0x7fa91c0c6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55890/0 (socket says 172.21.15.181:55890) 2023-10-16T11:19:42.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 -- 172.21.15.181:0/4249233556 learned_addr learned my addr 172.21.15.181:0/4249233556 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:42.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91b7fe700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15eab0 0x7fa91c1f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91a7fc700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c15e6d0 0x7fa91c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 -- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15eab0 msgr2=0x7fa91c1f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15eab0 0x7fa91c1f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 -- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c15e6d0 msgr2=0x7fa91c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c15e6d0 0x7fa91c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 -- 172.21.15.181:0/4249233556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910009550 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91b7fe700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15eab0 0x7fa91c1f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:42.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.073+0000 7fa91affd700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c0cb0c0 0x7fa91c0c6880 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00e990 tx=0x7fa90c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:42.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.074+0000 7fa9037fe700 1 -- 172.21.15.181:0/4249233556 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0096f0 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.074+0000 7fa9037fe700 1 -- 172.21.15.181:0/4249233556 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00c850 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.074+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c1f1600 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.074+0000 7fa9037fe700 1 -- 172.21.15.181:0/4249233556 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00cbc0 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.074+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c0b38b0 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.075+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c067d20 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.075+0000 7fa9037fe700 1 -- 172.21.15.181:0/4249233556 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa90c00cd20 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.076+0000 7fa9037fe700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa90806b020 0x7fa90806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.076+0000 7fa9037fe700 1 -- 172.21.15.181:0/4249233556 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fa90c08a080 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.076+0000 7fa91a7fc700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa90806b020 0x7fa90806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.076+0000 7fa91a7fc700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa90806b020 0x7fa90806d4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa910009c10 tx=0x7fa910008030 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:42.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.080+0000 7fa9037fe700 1 -- 172.21.15.181:0/4249233556 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa90c054b20 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.352+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa91c0cb610 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.352+0000 7fa9037fe700 1 -- 172.21.15.181:0/4249233556 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1125 (secure 0 0 0) 0x7fa90c017480 con 0x7fa91c0cb0c0 2023-10-16T11:19:42.355 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:42.355 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5758,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":220,"num_osds":8,"num_up_osds":8,"osd_up_since":1697449434,"num_in_osds":8,"osd_in_since":1697449426,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":193}],"num_pgs":193,"num_pools":5,"num_objects":46,"data_bytes":596435,"bytes_used":432304128,"bytes_avail":772661809152,"bytes_total":773094113280,"write_bytes_sec":2385,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":117,"id":13,"up":1,"in":1,"max":1,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"c","status":"up:active","gid":11325},{"filesystem_id":13,"rank":0,"name":"d","status":"up:active","gid":11340}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2023-10-16T11:18:55.300985+0000","services":{}},"progress_events":{}} 2023-10-16T11:19:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.354+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa90806b020 msgr2=0x7fa90806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.354+0000 7fa9213d5700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa90806b020 0x7fa90806d4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa910009c10 tx=0x7fa910008030 comp rx=0 tx=0).stop 2023-10-16T11:19:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.354+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c0cb0c0 msgr2=0x7fa91c0c6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.354+0000 7fa9213d5700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c0cb0c0 0x7fa91c0c6880 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00e990 tx=0x7fa90c00ed50 comp rx=0 tx=0).stop 2023-10-16T11:19:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 shutdown_connections 2023-10-16T11:19:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fa90806b020 0x7fa90806d4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa91c15eab0 0x7fa91c1f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91c0cb0c0 0x7fa91c0c6880 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 --2- 172.21.15.181:0/4249233556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa91c15e6d0 0x7fa91c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 >> 172.21.15.181:0/4249233556 conn(0x7fa91c0d04a0 msgr2=0x7fa91c0c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:42.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 shutdown_connections 2023-10-16T11:19:42.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.355+0000 7fa9213d5700 1 -- 172.21.15.181:0/4249233556 wait complete. 2023-10-16T11:19:42.369 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-16T11:19:42.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:19:42.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.488+0000 7f4dff57d700 1 Processor -- start 2023-10-16T11:19:42.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.488+0000 7f4dff57d700 1 -- start start 2023-10-16T11:19:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dff57d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4df815dfd0 0x7f4df80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dff57d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4df80cd470 0x7f4df80cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dff57d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df806f870 con 0x7f4df80af450 2023-10-16T11:19:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dff57d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df80cde40 con 0x7f4df815dfd0 2023-10-16T11:19:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dff57d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df80d1160 con 0x7f4df80cd470 2023-10-16T11:19:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4df815dfd0 0x7f4df80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4df815dfd0 0x7f4df80af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55898/0 (socket says 172.21.15.181:55898) 2023-10-16T11:19:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 -- 172.21.15.181:0/1584146337 learned_addr learned my addr 172.21.15.181:0/1584146337 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfdb1a700 1 --2- 172.21.15.181:0/1584146337 >> v2:172.21.15.190:3300/0 conn(0x7f4df80cd470 0x7f4df80cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 -- 172.21.15.181:0/1584146337 >> v2:172.21.15.190:3300/0 conn(0x7f4df80cd470 msgr2=0x7f4df80cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 --2- 172.21.15.181:0/1584146337 >> v2:172.21.15.190:3300/0 conn(0x7f4df80cd470 0x7f4df80cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 -- 172.21.15.181:0/1584146337 >> v1:172.21.15.190:6789/0 conn(0x7f4df80af450 legacy=0x7f4df806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 -- 172.21.15.181:0/1584146337 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80cc8e0 con 0x7f4df815dfd0 2023-10-16T11:19:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfdb1a700 1 --2- 172.21.15.181:0/1584146337 >> v2:172.21.15.190:3300/0 conn(0x7f4df80cd470 0x7f4df80cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4dfd319700 1 --2- 172.21.15.181:0/1584146337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df80af9a0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f4df80af1c0 tx=0x7f4df4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9751d8a19f85a9f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.489+0000 7f4deffff700 1 -- 172.21.15.181:0/1584146337 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4005e30 con 0x7f4df815dfd0 2023-10-16T11:19:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4deffff700 1 -- 172.21.15.181:0/1584146337 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df4004030 con 0x7f4df815dfd0 2023-10-16T11:19:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4deffff700 1 -- 172.21.15.181:0/1584146337 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df40043e0 con 0x7f4df815dfd0 2023-10-16T11:19:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 -- 172.21.15.181:0/1584146337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 msgr2=0x7f4df80af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 --2- 172.21.15.181:0/1584146337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df80af9a0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f4df80af1c0 tx=0x7f4df4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 -- 172.21.15.181:0/1584146337 shutdown_connections 2023-10-16T11:19:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 --2- 172.21.15.181:0/1584146337 >> v2:172.21.15.190:3300/0 conn(0x7f4df80cd470 0x7f4df80cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 --2- 172.21.15.181:0/1584146337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df80af9a0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 -- 172.21.15.181:0/1584146337 >> 172.21.15.181:0/1584146337 conn(0x7f4df80d04a0 msgr2=0x7f4df80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 -- 172.21.15.181:0/1584146337 shutdown_connections 2023-10-16T11:19:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.490+0000 7f4dff57d700 1 -- 172.21.15.181:0/1584146337 wait complete. 2023-10-16T11:19:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.491+0000 7f4dff57d700 1 Processor -- start 2023-10-16T11:19:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.491+0000 7f4dff57d700 1 -- start start 2023-10-16T11:19:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dff57d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df80af450 0x7f4df81f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dff57d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df80cd470 0x7f4df814d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dff57d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df814dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dff57d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df814b220 con 0x7f4df815dfd0 2023-10-16T11:19:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dff57d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80b5030 con 0x7f4df80cd470 2023-10-16T11:19:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dff57d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80b5310 con 0x7f4df80af450 2023-10-16T11:19:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dfdb1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df814dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dfdb1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df814dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55908/0 (socket says 172.21.15.181:55908) 2023-10-16T11:19:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dfdb1a700 1 -- 172.21.15.181:0/2053554967 learned_addr learned my addr 172.21.15.181:0/2053554967 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dfcb18700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df80cd470 0x7f4df814d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.492+0000 7f4dfd319700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df80af450 0x7f4df81f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dfdb1a700 1 -- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df80af450 msgr2=0x7f4df81f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dfdb1a700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df80af450 0x7f4df81f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dfdb1a700 1 -- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df80cd470 msgr2=0x7f4df814d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dfdb1a700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df80cd470 0x7f4df814d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dfdb1a700 1 -- 172.21.15.181:0/2053554967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df4009500 con 0x7f4df815dfd0 2023-10-16T11:19:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dfdb1a700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df814dab0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f4df0007ef0 tx=0x7f4df000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:42.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dedffb700 1 -- 172.21.15.181:0/2053554967 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000dd80 con 0x7f4df815dfd0 2023-10-16T11:19:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dedffb700 1 -- 172.21.15.181:0/2053554967 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df000f040 con 0x7f4df815dfd0 2023-10-16T11:19:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df80b55f0 con 0x7f4df815dfd0 2023-10-16T11:19:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dedffb700 1 -- 172.21.15.181:0/2053554967 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0003c60 con 0x7f4df815dfd0 2023-10-16T11:19:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.493+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df80b5bb0 con 0x7f4df815dfd0 2023-10-16T11:19:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.494+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df80b5dd0 con 0x7f4df815dfd0 2023-10-16T11:19:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.494+0000 7f4dedffb700 1 -- 172.21.15.181:0/2053554967 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4df0004030 con 0x7f4df815dfd0 2023-10-16T11:19:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.495+0000 7f4dedffb700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4ddc06b020 0x7f4ddc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.495+0000 7f4dedffb700 1 -- 172.21.15.181:0/2053554967 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f4df0089eb0 con 0x7f4df815dfd0 2023-10-16T11:19:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.495+0000 7f4dfd319700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4ddc06b020 0x7f4ddc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:42.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.495+0000 7f4dfd319700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4ddc06b020 0x7f4ddc06d4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4df80af1c0 tx=0x7f4df4008070 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:42.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.499+0000 7f4dedffb700 1 -- 172.21.15.181:0/2053554967 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4df00547b0 con 0x7f4df815dfd0 2023-10-16T11:19:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.687+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4df80ccf50 con 0x7f4ddc06b020 2023-10-16T11:19:42.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.701+0000 7f4dedffb700 1 -- 172.21.15.181:0/2053554967 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529056 (secure 0 0 0) 0x7f4df40036a0 con 0x7f4ddc06b020 2023-10-16T11:19:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:42.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4ddc06b020 msgr2=0x7f4ddc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4ddc06b020 0x7f4ddc06d4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4df80af1c0 tx=0x7f4df4008070 comp rx=0 tx=0).stop 2023-10-16T11:19:42.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 msgr2=0x7f4df814dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:42.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df814dab0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f4df0007ef0 tx=0x7f4df000d3b0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 shutdown_connections 2023-10-16T11:19:42.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f4ddc06b020 0x7f4ddc06d4e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df80af450 0x7f4df81f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4df815dfd0 0x7f4df814dab0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 --2- 172.21.15.181:0/2053554967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4df80cd470 0x7f4df814d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:42.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 >> 172.21.15.181:0/2053554967 conn(0x7f4df80d04a0 msgr2=0x7f4df80ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:42.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 shutdown_connections 2023-10-16T11:19:42.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:42.706+0000 7f4dff57d700 1 -- 172.21.15.181:0/2053554967 wait complete. 2023-10-16T11:19:42.716 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:19:42.720 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-16T11:19:41.493813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596435,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":671,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":341,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422172,"kb_used_data":22824,"kb_used_omap":12,"kb_used_meta":399219,"kb_avail":754552548,"statfs":{"total":773094113280,"available":772661809152,"internally_reserved":0,"allocated":23371776,"data_stored":12543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":408800824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.010471"},"pg_stats":[{"pgid":"1.0","version":"206'348","reported_seq":1213,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619646+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:19:36.619646+0000","last_peered":"2023-10-16T11:19:36.619646+0000","last_clean":"2023-10-16T11:19:36.619646+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:19:36.619646+0000","last_undegraded":"2023-10-16T11:19:36.619646+0000","last_fullsized":"2023-10-16T11:19:36.619646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629864+0000","last_change":"2023-10-16T11:19:22.536415+0000","last_active":"2023-10-16T11:19:36.629864+0000","last_peered":"2023-10-16T11:19:36.629864+0000","last_clean":"2023-10-16T11:19:36.629864+0000","last_became_active":"2023-10-16T11:19:22.536117+0000","last_became_peered":"2023-10-16T11:19:22.536117+0000","last_unstale":"2023-10-16T11:19:36.629864+0000","last_undegraded":"2023-10-16T11:19:36.629864+0000","last_fullsized":"2023-10-16T11:19:36.629864+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:07.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620141+0000","last_change":"2023-10-16T11:19:23.551428+0000","last_active":"2023-10-16T11:19:36.620141+0000","last_peered":"2023-10-16T11:19:36.620141+0000","last_clean":"2023-10-16T11:19:36.620141+0000","last_became_active":"2023-10-16T11:19:23.551083+0000","last_became_peered":"2023-10-16T11:19:23.551083+0000","last_unstale":"2023-10-16T11:19:36.620141+0000","last_undegraded":"2023-10-16T11:19:36.620141+0000","last_fullsized":"2023-10-16T11:19:36.620141+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:52.800093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023169+0000","last_change":"2023-10-16T11:19:35.631939+0000","last_active":"2023-10-16T11:19:37.023169+0000","last_peered":"2023-10-16T11:19:37.023169+0000","last_clean":"2023-10-16T11:19:37.023169+0000","last_became_active":"2023-10-16T11:19:35.631645+0000","last_became_peered":"2023-10-16T11:19:35.631645+0000","last_unstale":"2023-10-16T11:19:37.023169+0000","last_undegraded":"2023-10-16T11:19:37.023169+0000","last_fullsized":"2023-10-16T11:19:37.023169+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:56.488955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893598+0000","last_change":"2023-10-16T11:19:36.631354+0000","last_active":"2023-10-16T11:19:37.893598+0000","last_peered":"2023-10-16T11:19:37.893598+0000","last_clean":"2023-10-16T11:19:37.893598+0000","last_became_active":"2023-10-16T11:19:36.631071+0000","last_became_peered":"2023-10-16T11:19:36.631071+0000","last_unstale":"2023-10-16T11:19:37.893598+0000","last_undegraded":"2023-10-16T11:19:37.893598+0000","last_fullsized":"2023-10-16T11:19:37.893598+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:44.063667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630084+0000","last_change":"2023-10-16T11:19:22.537241+0000","last_active":"2023-10-16T11:19:36.630084+0000","last_peered":"2023-10-16T11:19:36.630084+0000","last_clean":"2023-10-16T11:19:36.630084+0000","last_became_active":"2023-10-16T11:19:22.536947+0000","last_became_peered":"2023-10-16T11:19:22.536947+0000","last_unstale":"2023-10-16T11:19:36.630084+0000","last_undegraded":"2023-10-16T11:19:36.630084+0000","last_fullsized":"2023-10-16T11:19:36.630084+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618113+0000","last_change":"2023-10-16T11:19:23.563089+0000","last_active":"2023-10-16T11:19:36.618113+0000","last_peered":"2023-10-16T11:19:36.618113+0000","last_clean":"2023-10-16T11:19:36.618113+0000","last_became_active":"2023-10-16T11:19:23.562759+0000","last_became_peered":"2023-10-16T11:19:23.562759+0000","last_unstale":"2023-10-16T11:19:36.618113+0000","last_undegraded":"2023-10-16T11:19:36.618113+0000","last_fullsized":"2023-10-16T11:19:36.618113+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893516+0000","last_change":"2023-10-16T11:19:35.628235+0000","last_active":"2023-10-16T11:19:37.893516+0000","last_peered":"2023-10-16T11:19:37.893516+0000","last_clean":"2023-10-16T11:19:37.893516+0000","last_became_active":"2023-10-16T11:19:35.627756+0000","last_became_peered":"2023-10-16T11:19:35.627756+0000","last_unstale":"2023-10-16T11:19:37.893516+0000","last_undegraded":"2023-10-16T11:19:37.893516+0000","last_fullsized":"2023-10-16T11:19:37.893516+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:06.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894143+0000","last_change":"2023-10-16T11:19:36.632258+0000","last_active":"2023-10-16T11:19:37.894143+0000","last_peered":"2023-10-16T11:19:37.894143+0000","last_clean":"2023-10-16T11:19:37.894143+0000","last_became_active":"2023-10-16T11:19:36.631963+0000","last_became_peered":"2023-10-16T11:19:36.631963+0000","last_unstale":"2023-10-16T11:19:37.894143+0000","last_undegraded":"2023-10-16T11:19:37.894143+0000","last_fullsized":"2023-10-16T11:19:37.894143+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624408+0000","last_change":"2023-10-16T11:19:22.539000+0000","last_active":"2023-10-16T11:19:36.624408+0000","last_peered":"2023-10-16T11:19:36.624408+0000","last_clean":"2023-10-16T11:19:36.624408+0000","last_became_active":"2023-10-16T11:19:22.538728+0000","last_became_peered":"2023-10-16T11:19:22.538728+0000","last_unstale":"2023-10-16T11:19:36.624408+0000","last_undegraded":"2023-10-16T11:19:36.624408+0000","last_fullsized":"2023-10-16T11:19:36.624408+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:03.944394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629081+0000","last_change":"2023-10-16T11:19:23.563504+0000","last_active":"2023-10-16T11:19:36.629081+0000","last_peered":"2023-10-16T11:19:36.629081+0000","last_clean":"2023-10-16T11:19:36.629081+0000","last_became_active":"2023-10-16T11:19:23.563243+0000","last_became_peered":"2023-10-16T11:19:23.563243+0000","last_unstale":"2023-10-16T11:19:36.629081+0000","last_undegraded":"2023-10-16T11:19:36.629081+0000","last_fullsized":"2023-10-16T11:19:36.629081+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026018+0000","last_change":"2023-10-16T11:19:35.632100+0000","last_active":"2023-10-16T11:19:37.026018+0000","last_peered":"2023-10-16T11:19:37.026018+0000","last_clean":"2023-10-16T11:19:37.026018+0000","last_became_active":"2023-10-16T11:19:35.631782+0000","last_became_peered":"2023-10-16T11:19:35.631782+0000","last_unstale":"2023-10-16T11:19:37.026018+0000","last_undegraded":"2023-10-16T11:19:37.026018+0000","last_fullsized":"2023-10-16T11:19:37.026018+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:19.485660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892620+0000","last_change":"2023-10-16T11:19:36.630004+0000","last_active":"2023-10-16T11:19:37.892620+0000","last_peered":"2023-10-16T11:19:37.892620+0000","last_clean":"2023-10-16T11:19:37.892620+0000","last_became_active":"2023-10-16T11:19:36.629689+0000","last_became_peered":"2023-10-16T11:19:36.629689+0000","last_unstale":"2023-10-16T11:19:37.892620+0000","last_undegraded":"2023-10-16T11:19:37.892620+0000","last_fullsized":"2023-10-16T11:19:37.892620+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:13.752266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615026+0000","last_change":"2023-10-16T11:19:22.544852+0000","last_active":"2023-10-16T11:19:36.615026+0000","last_peered":"2023-10-16T11:19:36.615026+0000","last_clean":"2023-10-16T11:19:36.615026+0000","last_became_active":"2023-10-16T11:19:22.544565+0000","last_became_peered":"2023-10-16T11:19:22.544565+0000","last_unstale":"2023-10-16T11:19:36.615026+0000","last_undegraded":"2023-10-16T11:19:36.615026+0000","last_fullsized":"2023-10-16T11:19:36.615026+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:07.144566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630082+0000","last_change":"2023-10-16T11:19:23.558505+0000","last_active":"2023-10-16T11:19:36.630082+0000","last_peered":"2023-10-16T11:19:36.630082+0000","last_clean":"2023-10-16T11:19:36.630082+0000","last_became_active":"2023-10-16T11:19:23.558188+0000","last_became_peered":"2023-10-16T11:19:23.558188+0000","last_unstale":"2023-10-16T11:19:36.630082+0000","last_undegraded":"2023-10-16T11:19:36.630082+0000","last_fullsized":"2023-10-16T11:19:36.630082+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630289+0000","last_change":"2023-10-16T11:19:35.630984+0000","last_active":"2023-10-16T11:19:36.630289+0000","last_peered":"2023-10-16T11:19:36.630289+0000","last_clean":"2023-10-16T11:19:36.630289+0000","last_became_active":"2023-10-16T11:19:35.630524+0000","last_became_peered":"2023-10-16T11:19:35.630524+0000","last_unstale":"2023-10-16T11:19:36.630289+0000","last_undegraded":"2023-10-16T11:19:36.630289+0000","last_fullsized":"2023-10-16T11:19:36.630289+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893641+0000","last_change":"2023-10-16T11:19:36.624656+0000","last_active":"2023-10-16T11:19:37.893641+0000","last_peered":"2023-10-16T11:19:37.893641+0000","last_clean":"2023-10-16T11:19:37.893641+0000","last_became_active":"2023-10-16T11:19:36.623815+0000","last_became_peered":"2023-10-16T11:19:36.623815+0000","last_unstale":"2023-10-16T11:19:37.893641+0000","last_undegraded":"2023-10-16T11:19:37.893641+0000","last_fullsized":"2023-10-16T11:19:37.893641+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:59:16.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616791+0000","last_change":"2023-10-16T11:19:22.543881+0000","last_active":"2023-10-16T11:19:36.616791+0000","last_peered":"2023-10-16T11:19:36.616791+0000","last_clean":"2023-10-16T11:19:36.616791+0000","last_became_active":"2023-10-16T11:19:22.543647+0000","last_became_peered":"2023-10-16T11:19:22.543647+0000","last_unstale":"2023-10-16T11:19:36.616791+0000","last_undegraded":"2023-10-16T11:19:36.616791+0000","last_fullsized":"2023-10-16T11:19:36.616791+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:41.755970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620300+0000","last_change":"2023-10-16T11:19:23.546380+0000","last_active":"2023-10-16T11:19:36.620300+0000","last_peered":"2023-10-16T11:19:36.620300+0000","last_clean":"2023-10-16T11:19:36.620300+0000","last_became_active":"2023-10-16T11:19:23.546098+0000","last_became_peered":"2023-10-16T11:19:23.546098+0000","last_unstale":"2023-10-16T11:19:36.620300+0000","last_undegraded":"2023-10-16T11:19:36.620300+0000","last_fullsized":"2023-10-16T11:19:36.620300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:06.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619997+0000","last_change":"2023-10-16T11:19:35.628388+0000","last_active":"2023-10-16T11:19:36.619997+0000","last_peered":"2023-10-16T11:19:36.619997+0000","last_clean":"2023-10-16T11:19:36.619997+0000","last_became_active":"2023-10-16T11:19:35.627946+0000","last_became_peered":"2023-10-16T11:19:35.627946+0000","last_unstale":"2023-10-16T11:19:36.619997+0000","last_undegraded":"2023-10-16T11:19:36.619997+0000","last_fullsized":"2023-10-16T11:19:36.619997+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:25.890159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894002+0000","last_change":"2023-10-16T11:19:36.639241+0000","last_active":"2023-10-16T11:19:37.894002+0000","last_peered":"2023-10-16T11:19:37.894002+0000","last_clean":"2023-10-16T11:19:37.894002+0000","last_became_active":"2023-10-16T11:19:36.637259+0000","last_became_peered":"2023-10-16T11:19:36.637259+0000","last_unstale":"2023-10-16T11:19:37.894002+0000","last_undegraded":"2023-10-16T11:19:37.894002+0000","last_fullsized":"2023-10-16T11:19:37.894002+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:25.400158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629125+0000","last_change":"2023-10-16T11:19:22.537543+0000","last_active":"2023-10-16T11:19:36.629125+0000","last_peered":"2023-10-16T11:19:36.629125+0000","last_clean":"2023-10-16T11:19:36.629125+0000","last_became_active":"2023-10-16T11:19:22.537050+0000","last_became_peered":"2023-10-16T11:19:22.537050+0000","last_unstale":"2023-10-16T11:19:36.629125+0000","last_undegraded":"2023-10-16T11:19:36.629125+0000","last_fullsized":"2023-10-16T11:19:36.629125+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:03.218818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624306+0000","last_change":"2023-10-16T11:19:23.546472+0000","last_active":"2023-10-16T11:19:36.624306+0000","last_peered":"2023-10-16T11:19:36.624306+0000","last_clean":"2023-10-16T11:19:36.624306+0000","last_became_active":"2023-10-16T11:19:23.546160+0000","last_became_peered":"2023-10-16T11:19:23.546160+0000","last_unstale":"2023-10-16T11:19:36.624306+0000","last_undegraded":"2023-10-16T11:19:36.624306+0000","last_fullsized":"2023-10-16T11:19:36.624306+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:06.395224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620047+0000","last_change":"2023-10-16T11:19:35.624590+0000","last_active":"2023-10-16T11:19:36.620047+0000","last_peered":"2023-10-16T11:19:36.620047+0000","last_clean":"2023-10-16T11:19:36.620047+0000","last_became_active":"2023-10-16T11:19:35.623979+0000","last_became_peered":"2023-10-16T11:19:35.623979+0000","last_unstale":"2023-10-16T11:19:36.620047+0000","last_undegraded":"2023-10-16T11:19:36.620047+0000","last_fullsized":"2023-10-16T11:19:36.620047+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:57.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894860+0000","last_change":"2023-10-16T11:19:36.631096+0000","last_active":"2023-10-16T11:19:37.894860+0000","last_peered":"2023-10-16T11:19:37.894860+0000","last_clean":"2023-10-16T11:19:37.894860+0000","last_became_active":"2023-10-16T11:19:36.630386+0000","last_became_peered":"2023-10-16T11:19:36.630386+0000","last_unstale":"2023-10-16T11:19:37.894860+0000","last_undegraded":"2023-10-16T11:19:37.894860+0000","last_fullsized":"2023-10-16T11:19:37.894860+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:03.603833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616893+0000","last_change":"2023-10-16T11:19:22.543844+0000","last_active":"2023-10-16T11:19:36.616893+0000","last_peered":"2023-10-16T11:19:36.616893+0000","last_clean":"2023-10-16T11:19:36.616893+0000","last_became_active":"2023-10-16T11:19:22.543538+0000","last_became_peered":"2023-10-16T11:19:22.543538+0000","last_unstale":"2023-10-16T11:19:36.616893+0000","last_undegraded":"2023-10-16T11:19:36.616893+0000","last_fullsized":"2023-10-16T11:19:36.616893+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:00.946603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617382+0000","last_change":"2023-10-16T11:19:23.556539+0000","last_active":"2023-10-16T11:19:36.617382+0000","last_peered":"2023-10-16T11:19:36.617382+0000","last_clean":"2023-10-16T11:19:36.617382+0000","last_became_active":"2023-10-16T11:19:23.556209+0000","last_became_peered":"2023-10-16T11:19:23.556209+0000","last_unstale":"2023-10-16T11:19:36.617382+0000","last_undegraded":"2023-10-16T11:19:36.617382+0000","last_fullsized":"2023-10-16T11:19:36.617382+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618436+0000","last_change":"2023-10-16T11:19:35.620983+0000","last_active":"2023-10-16T11:19:36.618436+0000","last_peered":"2023-10-16T11:19:36.618436+0000","last_clean":"2023-10-16T11:19:36.618436+0000","last_became_active":"2023-10-16T11:19:35.620681+0000","last_became_peered":"2023-10-16T11:19:35.620681+0000","last_unstale":"2023-10-16T11:19:36.618436+0000","last_undegraded":"2023-10-16T11:19:36.618436+0000","last_fullsized":"2023-10-16T11:19:36.618436+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:24.394439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893282+0000","last_change":"2023-10-16T11:19:36.628083+0000","last_active":"2023-10-16T11:19:37.893282+0000","last_peered":"2023-10-16T11:19:37.893282+0000","last_clean":"2023-10-16T11:19:37.893282+0000","last_became_active":"2023-10-16T11:19:36.627768+0000","last_became_peered":"2023-10-16T11:19:36.627768+0000","last_unstale":"2023-10-16T11:19:37.893282+0000","last_undegraded":"2023-10-16T11:19:37.893282+0000","last_fullsized":"2023-10-16T11:19:37.893282+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:59.846608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619763+0000","last_change":"2023-10-16T11:19:22.537758+0000","last_active":"2023-10-16T11:19:36.619763+0000","last_peered":"2023-10-16T11:19:36.619763+0000","last_clean":"2023-10-16T11:19:36.619763+0000","last_became_active":"2023-10-16T11:19:22.537201+0000","last_became_peered":"2023-10-16T11:19:22.537201+0000","last_unstale":"2023-10-16T11:19:36.619763+0000","last_undegraded":"2023-10-16T11:19:36.619763+0000","last_fullsized":"2023-10-16T11:19:36.619763+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:26.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622052+0000","last_change":"2023-10-16T11:19:23.564018+0000","last_active":"2023-10-16T11:19:36.622052+0000","last_peered":"2023-10-16T11:19:36.622052+0000","last_clean":"2023-10-16T11:19:36.622052+0000","last_became_active":"2023-10-16T11:19:23.563738+0000","last_became_peered":"2023-10-16T11:19:23.563738+0000","last_unstale":"2023-10-16T11:19:36.622052+0000","last_undegraded":"2023-10-16T11:19:36.622052+0000","last_fullsized":"2023-10-16T11:19:36.622052+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630020+0000","last_change":"2023-10-16T11:19:35.630702+0000","last_active":"2023-10-16T11:19:36.630020+0000","last_peered":"2023-10-16T11:19:36.630020+0000","last_clean":"2023-10-16T11:19:36.630020+0000","last_became_active":"2023-10-16T11:19:35.630133+0000","last_became_peered":"2023-10-16T11:19:35.630133+0000","last_unstale":"2023-10-16T11:19:36.630020+0000","last_undegraded":"2023-10-16T11:19:36.630020+0000","last_fullsized":"2023-10-16T11:19:36.630020+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:57.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892972+0000","last_change":"2023-10-16T11:19:36.630883+0000","last_active":"2023-10-16T11:19:37.892972+0000","last_peered":"2023-10-16T11:19:37.892972+0000","last_clean":"2023-10-16T11:19:37.892972+0000","last_became_active":"2023-10-16T11:19:36.627590+0000","last_became_peered":"2023-10-16T11:19:36.627590+0000","last_unstale":"2023-10-16T11:19:37.892972+0000","last_undegraded":"2023-10-16T11:19:37.892972+0000","last_fullsized":"2023-10-16T11:19:37.892972+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:16.051165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631253+0000","last_change":"2023-10-16T11:19:22.537256+0000","last_active":"2023-10-16T11:19:36.631253+0000","last_peered":"2023-10-16T11:19:36.631253+0000","last_clean":"2023-10-16T11:19:36.631253+0000","last_became_active":"2023-10-16T11:19:22.536996+0000","last_became_peered":"2023-10-16T11:19:22.536996+0000","last_unstale":"2023-10-16T11:19:36.631253+0000","last_undegraded":"2023-10-16T11:19:36.631253+0000","last_fullsized":"2023-10-16T11:19:36.631253+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:22.462616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631014+0000","last_change":"2023-10-16T11:19:23.561790+0000","last_active":"2023-10-16T11:19:36.631014+0000","last_peered":"2023-10-16T11:19:36.631014+0000","last_clean":"2023-10-16T11:19:36.631014+0000","last_became_active":"2023-10-16T11:19:23.561531+0000","last_became_peered":"2023-10-16T11:19:23.561531+0000","last_unstale":"2023-10-16T11:19:36.631014+0000","last_undegraded":"2023-10-16T11:19:36.631014+0000","last_fullsized":"2023-10-16T11:19:36.631014+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616296+0000","last_change":"2023-10-16T11:19:35.632427+0000","last_active":"2023-10-16T11:19:36.616296+0000","last_peered":"2023-10-16T11:19:36.616296+0000","last_clean":"2023-10-16T11:19:36.616296+0000","last_became_active":"2023-10-16T11:19:35.632164+0000","last_became_peered":"2023-10-16T11:19:35.632164+0000","last_unstale":"2023-10-16T11:19:36.616296+0000","last_undegraded":"2023-10-16T11:19:36.616296+0000","last_fullsized":"2023-10-16T11:19:36.616296+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:40.048278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893018+0000","last_change":"2023-10-16T11:19:36.633276+0000","last_active":"2023-10-16T11:19:37.893018+0000","last_peered":"2023-10-16T11:19:37.893018+0000","last_clean":"2023-10-16T11:19:37.893018+0000","last_became_active":"2023-10-16T11:19:36.631425+0000","last_became_peered":"2023-10-16T11:19:36.631425+0000","last_unstale":"2023-10-16T11:19:37.893018+0000","last_undegraded":"2023-10-16T11:19:37.893018+0000","last_fullsized":"2023-10-16T11:19:37.893018+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615074+0000","last_change":"2023-10-16T11:19:22.547829+0000","last_active":"2023-10-16T11:19:36.615074+0000","last_peered":"2023-10-16T11:19:36.615074+0000","last_clean":"2023-10-16T11:19:36.615074+0000","last_became_active":"2023-10-16T11:19:22.547563+0000","last_became_peered":"2023-10-16T11:19:22.547563+0000","last_unstale":"2023-10-16T11:19:36.615074+0000","last_undegraded":"2023-10-16T11:19:36.615074+0000","last_fullsized":"2023-10-16T11:19:36.615074+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:15.819032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624614+0000","last_change":"2023-10-16T11:19:23.563941+0000","last_active":"2023-10-16T11:19:36.624614+0000","last_peered":"2023-10-16T11:19:36.624614+0000","last_clean":"2023-10-16T11:19:36.624614+0000","last_became_active":"2023-10-16T11:19:23.563666+0000","last_became_peered":"2023-10-16T11:19:23.563666+0000","last_unstale":"2023-10-16T11:19:36.624614+0000","last_undegraded":"2023-10-16T11:19:36.624614+0000","last_fullsized":"2023-10-16T11:19:36.624614+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:43.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.021295+0000","last_change":"2023-10-16T11:19:35.628428+0000","last_active":"2023-10-16T11:19:37.021295+0000","last_peered":"2023-10-16T11:19:37.021295+0000","last_clean":"2023-10-16T11:19:37.021295+0000","last_became_active":"2023-10-16T11:19:35.628146+0000","last_became_peered":"2023-10-16T11:19:35.628146+0000","last_unstale":"2023-10-16T11:19:37.021295+0000","last_undegraded":"2023-10-16T11:19:37.021295+0000","last_fullsized":"2023-10-16T11:19:37.021295+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894166+0000","last_change":"2023-10-16T11:19:36.634088+0000","last_active":"2023-10-16T11:19:37.894166+0000","last_peered":"2023-10-16T11:19:37.894166+0000","last_clean":"2023-10-16T11:19:37.894166+0000","last_became_active":"2023-10-16T11:19:36.633725+0000","last_became_peered":"2023-10-16T11:19:36.633725+0000","last_unstale":"2023-10-16T11:19:37.894166+0000","last_undegraded":"2023-10-16T11:19:37.894166+0000","last_fullsized":"2023-10-16T11:19:37.894166+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:49:49.669608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619400+0000","last_change":"2023-10-16T11:19:22.532222+0000","last_active":"2023-10-16T11:19:36.619400+0000","last_peered":"2023-10-16T11:19:36.619400+0000","last_clean":"2023-10-16T11:19:36.619400+0000","last_became_active":"2023-10-16T11:19:22.531939+0000","last_became_peered":"2023-10-16T11:19:22.531939+0000","last_unstale":"2023-10-16T11:19:36.619400+0000","last_undegraded":"2023-10-16T11:19:36.619400+0000","last_fullsized":"2023-10-16T11:19:36.619400+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615636+0000","last_change":"2023-10-16T11:19:23.548561+0000","last_active":"2023-10-16T11:19:36.615636+0000","last_peered":"2023-10-16T11:19:36.615636+0000","last_clean":"2023-10-16T11:19:36.615636+0000","last_became_active":"2023-10-16T11:19:23.548310+0000","last_became_peered":"2023-10-16T11:19:23.548310+0000","last_unstale":"2023-10-16T11:19:36.615636+0000","last_undegraded":"2023-10-16T11:19:36.615636+0000","last_fullsized":"2023-10-16T11:19:36.615636+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.024902+0000","last_change":"2023-10-16T11:19:35.630294+0000","last_active":"2023-10-16T11:19:37.024902+0000","last_peered":"2023-10-16T11:19:37.024902+0000","last_clean":"2023-10-16T11:19:37.024902+0000","last_became_active":"2023-10-16T11:19:35.629895+0000","last_became_peered":"2023-10-16T11:19:35.629895+0000","last_unstale":"2023-10-16T11:19:37.024902+0000","last_undegraded":"2023-10-16T11:19:37.024902+0000","last_fullsized":"2023-10-16T11:19:37.024902+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622628+0000","last_change":"2023-10-16T11:19:36.632583+0000","last_active":"2023-10-16T11:19:37.622628+0000","last_peered":"2023-10-16T11:19:37.622628+0000","last_clean":"2023-10-16T11:19:37.622628+0000","last_became_active":"2023-10-16T11:19:36.632316+0000","last_became_peered":"2023-10-16T11:19:36.632316+0000","last_unstale":"2023-10-16T11:19:37.622628+0000","last_undegraded":"2023-10-16T11:19:37.622628+0000","last_fullsized":"2023-10-16T11:19:37.622628+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619856+0000","last_change":"2023-10-16T11:19:22.542021+0000","last_active":"2023-10-16T11:19:36.619856+0000","last_peered":"2023-10-16T11:19:36.619856+0000","last_clean":"2023-10-16T11:19:36.619856+0000","last_became_active":"2023-10-16T11:19:22.541718+0000","last_became_peered":"2023-10-16T11:19:22.541718+0000","last_unstale":"2023-10-16T11:19:36.619856+0000","last_undegraded":"2023-10-16T11:19:36.619856+0000","last_fullsized":"2023-10-16T11:19:36.619856+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:12.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628983+0000","last_change":"2023-10-16T11:19:23.554949+0000","last_active":"2023-10-16T11:19:36.628983+0000","last_peered":"2023-10-16T11:19:36.628983+0000","last_clean":"2023-10-16T11:19:36.628983+0000","last_became_active":"2023-10-16T11:19:23.554300+0000","last_became_peered":"2023-10-16T11:19:23.554300+0000","last_unstale":"2023-10-16T11:19:36.628983+0000","last_undegraded":"2023-10-16T11:19:36.628983+0000","last_fullsized":"2023-10-16T11:19:36.628983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623336+0000","last_change":"2023-10-16T11:19:35.627888+0000","last_active":"2023-10-16T11:19:36.623336+0000","last_peered":"2023-10-16T11:19:36.623336+0000","last_clean":"2023-10-16T11:19:36.623336+0000","last_became_active":"2023-10-16T11:19:35.627453+0000","last_became_peered":"2023-10-16T11:19:35.627453+0000","last_unstale":"2023-10-16T11:19:36.623336+0000","last_undegraded":"2023-10-16T11:19:36.623336+0000","last_fullsized":"2023-10-16T11:19:36.623336+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:45.616063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894273+0000","last_change":"2023-10-16T11:19:36.638745+0000","last_active":"2023-10-16T11:19:37.894273+0000","last_peered":"2023-10-16T11:19:37.894273+0000","last_clean":"2023-10-16T11:19:37.894273+0000","last_became_active":"2023-10-16T11:19:36.634962+0000","last_became_peered":"2023-10-16T11:19:36.634962+0000","last_unstale":"2023-10-16T11:19:37.894273+0000","last_undegraded":"2023-10-16T11:19:37.894273+0000","last_fullsized":"2023-10-16T11:19:37.894273+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:19.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621691+0000","last_change":"2023-10-16T11:19:22.536403+0000","last_active":"2023-10-16T11:19:36.621691+0000","last_peered":"2023-10-16T11:19:36.621691+0000","last_clean":"2023-10-16T11:19:36.621691+0000","last_became_active":"2023-10-16T11:19:22.536132+0000","last_became_peered":"2023-10-16T11:19:22.536132+0000","last_unstale":"2023-10-16T11:19:36.621691+0000","last_undegraded":"2023-10-16T11:19:36.621691+0000","last_fullsized":"2023-10-16T11:19:36.621691+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:53.563487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621852+0000","last_change":"2023-10-16T11:19:23.543334+0000","last_active":"2023-10-16T11:19:36.621852+0000","last_peered":"2023-10-16T11:19:36.621852+0000","last_clean":"2023-10-16T11:19:36.621852+0000","last_became_active":"2023-10-16T11:19:23.542527+0000","last_became_peered":"2023-10-16T11:19:23.542527+0000","last_unstale":"2023-10-16T11:19:36.621852+0000","last_undegraded":"2023-10-16T11:19:36.621852+0000","last_fullsized":"2023-10-16T11:19:36.621852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:59.083108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.022227+0000","last_change":"2023-10-16T11:19:35.631031+0000","last_active":"2023-10-16T11:19:37.022227+0000","last_peered":"2023-10-16T11:19:37.022227+0000","last_clean":"2023-10-16T11:19:37.022227+0000","last_became_active":"2023-10-16T11:19:35.630599+0000","last_became_peered":"2023-10-16T11:19:35.630599+0000","last_unstale":"2023-10-16T11:19:37.022227+0000","last_undegraded":"2023-10-16T11:19:37.022227+0000","last_fullsized":"2023-10-16T11:19:37.022227+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:13:50.394449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894077+0000","last_change":"2023-10-16T11:19:36.635702+0000","last_active":"2023-10-16T11:19:37.894077+0000","last_peered":"2023-10-16T11:19:37.894077+0000","last_clean":"2023-10-16T11:19:37.894077+0000","last_became_active":"2023-10-16T11:19:36.635443+0000","last_became_peered":"2023-10-16T11:19:36.635443+0000","last_unstale":"2023-10-16T11:19:37.894077+0000","last_undegraded":"2023-10-16T11:19:37.894077+0000","last_fullsized":"2023-10-16T11:19:37.894077+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:37.781294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628927+0000","last_change":"2023-10-16T11:19:22.545030+0000","last_active":"2023-10-16T11:19:36.628927+0000","last_peered":"2023-10-16T11:19:36.628927+0000","last_clean":"2023-10-16T11:19:36.628927+0000","last_became_active":"2023-10-16T11:19:22.544773+0000","last_became_peered":"2023-10-16T11:19:22.544773+0000","last_unstale":"2023-10-16T11:19:36.628927+0000","last_undegraded":"2023-10-16T11:19:36.628927+0000","last_fullsized":"2023-10-16T11:19:36.628927+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:22.946353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623654+0000","last_change":"2023-10-16T11:19:23.545760+0000","last_active":"2023-10-16T11:19:36.623654+0000","last_peered":"2023-10-16T11:19:36.623654+0000","last_clean":"2023-10-16T11:19:36.623654+0000","last_became_active":"2023-10-16T11:19:23.545499+0000","last_became_peered":"2023-10-16T11:19:23.545499+0000","last_unstale":"2023-10-16T11:19:36.623654+0000","last_undegraded":"2023-10-16T11:19:36.623654+0000","last_fullsized":"2023-10-16T11:19:36.623654+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:32.701590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616714+0000","last_change":"2023-10-16T11:19:35.621290+0000","last_active":"2023-10-16T11:19:36.616714+0000","last_peered":"2023-10-16T11:19:36.616714+0000","last_clean":"2023-10-16T11:19:36.616714+0000","last_became_active":"2023-10-16T11:19:35.620989+0000","last_became_peered":"2023-10-16T11:19:35.620989+0000","last_unstale":"2023-10-16T11:19:36.616714+0000","last_undegraded":"2023-10-16T11:19:36.616714+0000","last_fullsized":"2023-10-16T11:19:36.616714+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:22.819700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.895061+0000","last_change":"2023-10-16T11:19:36.639230+0000","last_active":"2023-10-16T11:19:37.895061+0000","last_peered":"2023-10-16T11:19:37.895061+0000","last_clean":"2023-10-16T11:19:37.895061+0000","last_became_active":"2023-10-16T11:19:36.638912+0000","last_became_peered":"2023-10-16T11:19:36.638912+0000","last_unstale":"2023-10-16T11:19:37.895061+0000","last_undegraded":"2023-10-16T11:19:37.895061+0000","last_fullsized":"2023-10-16T11:19:37.895061+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:20.791568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623856+0000","last_change":"2023-10-16T11:19:22.533601+0000","last_active":"2023-10-16T11:19:36.623856+0000","last_peered":"2023-10-16T11:19:36.623856+0000","last_clean":"2023-10-16T11:19:36.623856+0000","last_became_active":"2023-10-16T11:19:22.533374+0000","last_became_peered":"2023-10-16T11:19:22.533374+0000","last_unstale":"2023-10-16T11:19:36.623856+0000","last_undegraded":"2023-10-16T11:19:36.623856+0000","last_fullsized":"2023-10-16T11:19:36.623856+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:52.564361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629027+0000","last_change":"2023-10-16T11:19:23.553745+0000","last_active":"2023-10-16T11:19:36.629027+0000","last_peered":"2023-10-16T11:19:36.629027+0000","last_clean":"2023-10-16T11:19:36.629027+0000","last_became_active":"2023-10-16T11:19:23.553462+0000","last_became_peered":"2023-10-16T11:19:23.553462+0000","last_unstale":"2023-10-16T11:19:36.629027+0000","last_undegraded":"2023-10-16T11:19:36.629027+0000","last_fullsized":"2023-10-16T11:19:36.629027+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023539+0000","last_change":"2023-10-16T11:19:35.632251+0000","last_active":"2023-10-16T11:19:37.023539+0000","last_peered":"2023-10-16T11:19:37.023539+0000","last_clean":"2023-10-16T11:19:37.023539+0000","last_became_active":"2023-10-16T11:19:35.631302+0000","last_became_peered":"2023-10-16T11:19:35.631302+0000","last_unstale":"2023-10-16T11:19:37.023539+0000","last_undegraded":"2023-10-16T11:19:37.023539+0000","last_fullsized":"2023-10-16T11:19:37.023539+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893096+0000","last_change":"2023-10-16T11:19:36.627034+0000","last_active":"2023-10-16T11:19:37.893096+0000","last_peered":"2023-10-16T11:19:37.893096+0000","last_clean":"2023-10-16T11:19:37.893096+0000","last_became_active":"2023-10-16T11:19:36.626768+0000","last_became_peered":"2023-10-16T11:19:36.626768+0000","last_unstale":"2023-10-16T11:19:37.893096+0000","last_undegraded":"2023-10-16T11:19:37.893096+0000","last_fullsized":"2023-10-16T11:19:37.893096+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:49.454701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616295+0000","last_change":"2023-10-16T11:19:22.539558+0000","last_active":"2023-10-16T11:19:36.616295+0000","last_peered":"2023-10-16T11:19:36.616295+0000","last_clean":"2023-10-16T11:19:36.616295+0000","last_became_active":"2023-10-16T11:19:22.539284+0000","last_became_peered":"2023-10-16T11:19:22.539284+0000","last_unstale":"2023-10-16T11:19:36.616295+0000","last_undegraded":"2023-10-16T11:19:36.616295+0000","last_fullsized":"2023-10-16T11:19:36.616295+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616463+0000","last_change":"2023-10-16T11:19:23.546218+0000","last_active":"2023-10-16T11:19:36.616463+0000","last_peered":"2023-10-16T11:19:36.616463+0000","last_clean":"2023-10-16T11:19:36.616463+0000","last_became_active":"2023-10-16T11:19:23.545896+0000","last_became_peered":"2023-10-16T11:19:23.545896+0000","last_unstale":"2023-10-16T11:19:36.616463+0000","last_undegraded":"2023-10-16T11:19:36.616463+0000","last_fullsized":"2023-10-16T11:19:36.616463+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629694+0000","last_change":"2023-10-16T11:19:35.631734+0000","last_active":"2023-10-16T11:19:36.629694+0000","last_peered":"2023-10-16T11:19:36.629694+0000","last_clean":"2023-10-16T11:19:36.629694+0000","last_became_active":"2023-10-16T11:19:35.631398+0000","last_became_peered":"2023-10-16T11:19:35.631398+0000","last_unstale":"2023-10-16T11:19:36.629694+0000","last_undegraded":"2023-10-16T11:19:36.629694+0000","last_fullsized":"2023-10-16T11:19:36.629694+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:32.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893267+0000","last_change":"2023-10-16T11:19:36.627229+0000","last_active":"2023-10-16T11:19:37.893267+0000","last_peered":"2023-10-16T11:19:37.893267+0000","last_clean":"2023-10-16T11:19:37.893267+0000","last_became_active":"2023-10-16T11:19:36.626587+0000","last_became_peered":"2023-10-16T11:19:36.626587+0000","last_unstale":"2023-10-16T11:19:37.893267+0000","last_undegraded":"2023-10-16T11:19:37.893267+0000","last_fullsized":"2023-10-16T11:19:37.893267+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:34.645872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615964+0000","last_change":"2023-10-16T11:19:22.545027+0000","last_active":"2023-10-16T11:19:36.615964+0000","last_peered":"2023-10-16T11:19:36.615964+0000","last_clean":"2023-10-16T11:19:36.615964+0000","last_became_active":"2023-10-16T11:19:22.544756+0000","last_became_peered":"2023-10-16T11:19:22.544756+0000","last_unstale":"2023-10-16T11:19:36.615964+0000","last_undegraded":"2023-10-16T11:19:36.615964+0000","last_fullsized":"2023-10-16T11:19:36.615964+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:23.693322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616002+0000","last_change":"2023-10-16T11:19:23.547530+0000","last_active":"2023-10-16T11:19:36.616002+0000","last_peered":"2023-10-16T11:19:36.616002+0000","last_clean":"2023-10-16T11:19:36.616002+0000","last_became_active":"2023-10-16T11:19:23.547255+0000","last_became_peered":"2023-10-16T11:19:23.547255+0000","last_unstale":"2023-10-16T11:19:36.616002+0000","last_undegraded":"2023-10-16T11:19:36.616002+0000","last_fullsized":"2023-10-16T11:19:36.616002+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616101+0000","last_change":"2023-10-16T11:19:35.632530+0000","last_active":"2023-10-16T11:19:36.616101+0000","last_peered":"2023-10-16T11:19:36.616101+0000","last_clean":"2023-10-16T11:19:36.616101+0000","last_became_active":"2023-10-16T11:19:35.632226+0000","last_became_peered":"2023-10-16T11:19:35.632226+0000","last_unstale":"2023-10-16T11:19:36.616101+0000","last_undegraded":"2023-10-16T11:19:36.616101+0000","last_fullsized":"2023-10-16T11:19:36.616101+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.620864+0000","last_change":"2023-10-16T11:19:36.632635+0000","last_active":"2023-10-16T11:19:37.620864+0000","last_peered":"2023-10-16T11:19:37.620864+0000","last_clean":"2023-10-16T11:19:37.620864+0000","last_became_active":"2023-10-16T11:19:36.632351+0000","last_became_peered":"2023-10-16T11:19:36.632351+0000","last_unstale":"2023-10-16T11:19:37.620864+0000","last_undegraded":"2023-10-16T11:19:37.620864+0000","last_fullsized":"2023-10-16T11:19:37.620864+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:23.825418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":24,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629784+0000","last_change":"2023-10-16T11:19:22.534093+0000","last_active":"2023-10-16T11:19:36.629784+0000","last_peered":"2023-10-16T11:19:36.629784+0000","last_clean":"2023-10-16T11:19:36.629784+0000","last_became_active":"2023-10-16T11:19:22.533637+0000","last_became_peered":"2023-10-16T11:19:22.533637+0000","last_unstale":"2023-10-16T11:19:36.629784+0000","last_undegraded":"2023-10-16T11:19:36.629784+0000","last_fullsized":"2023-10-16T11:19:36.629784+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619314+0000","last_change":"2023-10-16T11:19:23.552950+0000","last_active":"2023-10-16T11:19:36.619314+0000","last_peered":"2023-10-16T11:19:36.619314+0000","last_clean":"2023-10-16T11:19:36.619314+0000","last_became_active":"2023-10-16T11:19:23.552686+0000","last_became_peered":"2023-10-16T11:19:23.552686+0000","last_unstale":"2023-10-16T11:19:36.619314+0000","last_undegraded":"2023-10-16T11:19:36.619314+0000","last_fullsized":"2023-10-16T11:19:36.619314+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622198+0000","last_change":"2023-10-16T11:19:35.623459+0000","last_active":"2023-10-16T11:19:36.622198+0000","last_peered":"2023-10-16T11:19:36.622198+0000","last_clean":"2023-10-16T11:19:36.622198+0000","last_became_active":"2023-10-16T11:19:35.623199+0000","last_became_peered":"2023-10-16T11:19:35.623199+0000","last_unstale":"2023-10-16T11:19:36.622198+0000","last_undegraded":"2023-10-16T11:19:36.622198+0000","last_fullsized":"2023-10-16T11:19:36.622198+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894201+0000","last_change":"2023-10-16T11:19:36.637364+0000","last_active":"2023-10-16T11:19:37.894201+0000","last_peered":"2023-10-16T11:19:37.894201+0000","last_clean":"2023-10-16T11:19:37.894201+0000","last_became_active":"2023-10-16T11:19:36.637057+0000","last_became_peered":"2023-10-16T11:19:36.637057+0000","last_unstale":"2023-10-16T11:19:37.894201+0000","last_undegraded":"2023-10-16T11:19:37.894201+0000","last_fullsized":"2023-10-16T11:19:37.894201+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:03.151059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621449+0000","last_change":"2023-10-16T11:19:22.538008+0000","last_active":"2023-10-16T11:19:36.621449+0000","last_peered":"2023-10-16T11:19:36.621449+0000","last_clean":"2023-10-16T11:19:36.621449+0000","last_became_active":"2023-10-16T11:19:22.537752+0000","last_became_peered":"2023-10-16T11:19:22.537752+0000","last_unstale":"2023-10-16T11:19:36.621449+0000","last_undegraded":"2023-10-16T11:19:36.621449+0000","last_fullsized":"2023-10-16T11:19:36.621449+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:20.859898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628978+0000","last_change":"2023-10-16T11:19:23.563603+0000","last_active":"2023-10-16T11:19:36.628978+0000","last_peered":"2023-10-16T11:19:36.628978+0000","last_clean":"2023-10-16T11:19:36.628978+0000","last_became_active":"2023-10-16T11:19:23.563305+0000","last_became_peered":"2023-10-16T11:19:23.563305+0000","last_unstale":"2023-10-16T11:19:36.628978+0000","last_undegraded":"2023-10-16T11:19:36.628978+0000","last_fullsized":"2023-10-16T11:19:36.628978+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620238+0000","last_change":"2023-10-16T11:19:35.631312+0000","last_active":"2023-10-16T11:19:36.620238+0000","last_peered":"2023-10-16T11:19:36.620238+0000","last_clean":"2023-10-16T11:19:36.620238+0000","last_became_active":"2023-10-16T11:19:35.631045+0000","last_became_peered":"2023-10-16T11:19:35.631045+0000","last_unstale":"2023-10-16T11:19:36.620238+0000","last_undegraded":"2023-10-16T11:19:36.620238+0000","last_fullsized":"2023-10-16T11:19:36.620238+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:41.921297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894234+0000","last_change":"2023-10-16T11:19:36.636938+0000","last_active":"2023-10-16T11:19:37.894234+0000","last_peered":"2023-10-16T11:19:37.894234+0000","last_clean":"2023-10-16T11:19:37.894234+0000","last_became_active":"2023-10-16T11:19:36.636625+0000","last_became_peered":"2023-10-16T11:19:36.636625+0000","last_unstale":"2023-10-16T11:19:37.894234+0000","last_undegraded":"2023-10-16T11:19:37.894234+0000","last_fullsized":"2023-10-16T11:19:37.894234+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:23.128272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624395+0000","last_change":"2023-10-16T11:19:22.538147+0000","last_active":"2023-10-16T11:19:36.624395+0000","last_peered":"2023-10-16T11:19:36.624395+0000","last_clean":"2023-10-16T11:19:36.624395+0000","last_became_active":"2023-10-16T11:19:22.537850+0000","last_became_peered":"2023-10-16T11:19:22.537850+0000","last_unstale":"2023-10-16T11:19:36.624395+0000","last_undegraded":"2023-10-16T11:19:36.624395+0000","last_fullsized":"2023-10-16T11:19:36.624395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:44.794069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621587+0000","last_change":"2023-10-16T11:19:23.552779+0000","last_active":"2023-10-16T11:19:36.621587+0000","last_peered":"2023-10-16T11:19:36.621587+0000","last_clean":"2023-10-16T11:19:36.621587+0000","last_became_active":"2023-10-16T11:19:23.552491+0000","last_became_peered":"2023-10-16T11:19:23.552491+0000","last_unstale":"2023-10-16T11:19:36.621587+0000","last_undegraded":"2023-10-16T11:19:36.621587+0000","last_fullsized":"2023-10-16T11:19:36.621587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:02.969587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"219'6","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.027458+0000","last_change":"2023-10-16T11:19:35.629807+0000","last_active":"2023-10-16T11:19:37.027458+0000","last_peered":"2023-10-16T11:19:37.027458+0000","last_clean":"2023-10-16T11:19:37.027458+0000","last_became_active":"2023-10-16T11:19:35.629137+0000","last_became_peered":"2023-10-16T11:19:35.629137+0000","last_unstale":"2023-10-16T11:19:37.027458+0000","last_undegraded":"2023-10-16T11:19:37.027458+0000","last_fullsized":"2023-10-16T11:19:37.027458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894306+0000","last_change":"2023-10-16T11:19:36.632014+0000","last_active":"2023-10-16T11:19:37.894306+0000","last_peered":"2023-10-16T11:19:37.894306+0000","last_clean":"2023-10-16T11:19:37.894306+0000","last_became_active":"2023-10-16T11:19:36.631735+0000","last_became_peered":"2023-10-16T11:19:36.631735+0000","last_unstale":"2023-10-16T11:19:37.894306+0000","last_undegraded":"2023-10-16T11:19:37.894306+0000","last_fullsized":"2023-10-16T11:19:37.894306+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:25.064507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624578+0000","last_change":"2023-10-16T11:19:22.544032+0000","last_active":"2023-10-16T11:19:36.624578+0000","last_peered":"2023-10-16T11:19:36.624578+0000","last_clean":"2023-10-16T11:19:36.624578+0000","last_became_active":"2023-10-16T11:19:22.543713+0000","last_became_peered":"2023-10-16T11:19:22.543713+0000","last_unstale":"2023-10-16T11:19:36.624578+0000","last_undegraded":"2023-10-16T11:19:36.624578+0000","last_fullsized":"2023-10-16T11:19:36.624578+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:01.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615557+0000","last_change":"2023-10-16T11:19:23.550398+0000","last_active":"2023-10-16T11:19:36.615557+0000","last_peered":"2023-10-16T11:19:36.615557+0000","last_clean":"2023-10-16T11:19:36.615557+0000","last_became_active":"2023-10-16T11:19:23.545907+0000","last_became_peered":"2023-10-16T11:19:23.545907+0000","last_unstale":"2023-10-16T11:19:36.615557+0000","last_undegraded":"2023-10-16T11:19:36.615557+0000","last_fullsized":"2023-10-16T11:19:36.615557+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:05.456349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"219'3","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026764+0000","last_change":"2023-10-16T11:19:35.627232+0000","last_active":"2023-10-16T11:19:37.026764+0000","last_peered":"2023-10-16T11:19:37.026764+0000","last_clean":"2023-10-16T11:19:37.026764+0000","last_became_active":"2023-10-16T11:19:35.626952+0000","last_became_peered":"2023-10-16T11:19:35.626952+0000","last_unstale":"2023-10-16T11:19:37.026764+0000","last_undegraded":"2023-10-16T11:19:37.026764+0000","last_fullsized":"2023-10-16T11:19:37.026764+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:39.610324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621525+0000","last_change":"2023-10-16T11:19:36.636057+0000","last_active":"2023-10-16T11:19:37.621525+0000","last_peered":"2023-10-16T11:19:37.621525+0000","last_clean":"2023-10-16T11:19:37.621525+0000","last_became_active":"2023-10-16T11:19:36.635687+0000","last_became_peered":"2023-10-16T11:19:36.635687+0000","last_unstale":"2023-10-16T11:19:37.621525+0000","last_undegraded":"2023-10-16T11:19:37.621525+0000","last_fullsized":"2023-10-16T11:19:37.621525+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.14","version":"216'8","reported_seq":24,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617300+0000","last_change":"2023-10-16T11:19:22.532339+0000","last_active":"2023-10-16T11:19:36.617300+0000","last_peered":"2023-10-16T11:19:36.617300+0000","last_clean":"2023-10-16T11:19:36.617300+0000","last_became_active":"2023-10-16T11:19:22.532053+0000","last_became_peered":"2023-10-16T11:19:22.532053+0000","last_unstale":"2023-10-16T11:19:36.617300+0000","last_undegraded":"2023-10-16T11:19:36.617300+0000","last_fullsized":"2023-10-16T11:19:36.617300+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:21.190168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2859,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621913+0000","last_change":"2023-10-16T11:19:23.548103+0000","last_active":"2023-10-16T11:19:36.621913+0000","last_peered":"2023-10-16T11:19:36.621913+0000","last_clean":"2023-10-16T11:19:36.621913+0000","last_became_active":"2023-10-16T11:19:23.547665+0000","last_became_peered":"2023-10-16T11:19:23.547665+0000","last_unstale":"2023-10-16T11:19:36.621913+0000","last_undegraded":"2023-10-16T11:19:36.621913+0000","last_fullsized":"2023-10-16T11:19:36.621913+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619766+0000","last_change":"2023-10-16T11:19:35.624749+0000","last_active":"2023-10-16T11:19:36.619766+0000","last_peered":"2023-10-16T11:19:36.619766+0000","last_clean":"2023-10-16T11:19:36.619766+0000","last_became_active":"2023-10-16T11:19:35.624338+0000","last_became_peered":"2023-10-16T11:19:35.624338+0000","last_unstale":"2023-10-16T11:19:36.619766+0000","last_undegraded":"2023-10-16T11:19:36.619766+0000","last_fullsized":"2023-10-16T11:19:36.619766+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:32.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894693+0000","last_change":"2023-10-16T11:19:36.631186+0000","last_active":"2023-10-16T11:19:37.894693+0000","last_peered":"2023-10-16T11:19:37.894693+0000","last_clean":"2023-10-16T11:19:37.894693+0000","last_became_active":"2023-10-16T11:19:36.630761+0000","last_became_peered":"2023-10-16T11:19:36.630761+0000","last_unstale":"2023-10-16T11:19:37.894693+0000","last_undegraded":"2023-10-16T11:19:37.894693+0000","last_fullsized":"2023-10-16T11:19:37.894693+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:18.537927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623757+0000","last_change":"2023-10-16T11:19:22.544432+0000","last_active":"2023-10-16T11:19:36.623757+0000","last_peered":"2023-10-16T11:19:36.623757+0000","last_clean":"2023-10-16T11:19:36.623757+0000","last_became_active":"2023-10-16T11:19:22.543046+0000","last_became_peered":"2023-10-16T11:19:22.543046+0000","last_unstale":"2023-10-16T11:19:36.623757+0000","last_undegraded":"2023-10-16T11:19:36.623757+0000","last_fullsized":"2023-10-16T11:19:36.623757+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:08.672505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617875+0000","last_change":"2023-10-16T11:19:23.563660+0000","last_active":"2023-10-16T11:19:36.617875+0000","last_peered":"2023-10-16T11:19:36.617875+0000","last_clean":"2023-10-16T11:19:36.617875+0000","last_became_active":"2023-10-16T11:19:23.563402+0000","last_became_peered":"2023-10-16T11:19:23.563402+0000","last_unstale":"2023-10-16T11:19:36.617875+0000","last_undegraded":"2023-10-16T11:19:36.617875+0000","last_fullsized":"2023-10-16T11:19:36.617875+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:41.606189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026157+0000","last_change":"2023-10-16T11:19:35.633267+0000","last_active":"2023-10-16T11:19:37.026157+0000","last_peered":"2023-10-16T11:19:37.026157+0000","last_clean":"2023-10-16T11:19:37.026157+0000","last_became_active":"2023-10-16T11:19:35.632961+0000","last_became_peered":"2023-10-16T11:19:35.632961+0000","last_unstale":"2023-10-16T11:19:37.026157+0000","last_undegraded":"2023-10-16T11:19:37.026157+0000","last_fullsized":"2023-10-16T11:19:37.026157+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621385+0000","last_change":"2023-10-16T11:19:36.628797+0000","last_active":"2023-10-16T11:19:37.621385+0000","last_peered":"2023-10-16T11:19:37.621385+0000","last_clean":"2023-10-16T11:19:37.621385+0000","last_became_active":"2023-10-16T11:19:36.628522+0000","last_became_peered":"2023-10-16T11:19:36.628522+0000","last_unstale":"2023-10-16T11:19:37.621385+0000","last_undegraded":"2023-10-16T11:19:37.621385+0000","last_fullsized":"2023-10-16T11:19:37.621385+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.626248+0000","last_change":"2023-10-16T11:19:22.535771+0000","last_active":"2023-10-16T11:19:36.626248+0000","last_peered":"2023-10-16T11:19:36.626248+0000","last_clean":"2023-10-16T11:19:36.626248+0000","last_became_active":"2023-10-16T11:19:22.535486+0000","last_became_peered":"2023-10-16T11:19:22.535486+0000","last_unstale":"2023-10-16T11:19:36.626248+0000","last_undegraded":"2023-10-16T11:19:36.626248+0000","last_fullsized":"2023-10-16T11:19:36.626248+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:25.214100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631131+0000","last_change":"2023-10-16T11:19:23.560046+0000","last_active":"2023-10-16T11:19:36.631131+0000","last_peered":"2023-10-16T11:19:36.631131+0000","last_clean":"2023-10-16T11:19:36.631131+0000","last_became_active":"2023-10-16T11:19:23.559718+0000","last_became_peered":"2023-10-16T11:19:23.559718+0000","last_unstale":"2023-10-16T11:19:36.631131+0000","last_undegraded":"2023-10-16T11:19:36.631131+0000","last_fullsized":"2023-10-16T11:19:36.631131+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:22.701369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"219'8","reported_seq":20,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026701+0000","last_change":"2023-10-16T11:19:35.625566+0000","last_active":"2023-10-16T11:19:37.026701+0000","last_peered":"2023-10-16T11:19:37.026701+0000","last_clean":"2023-10-16T11:19:37.026701+0000","last_became_active":"2023-10-16T11:19:35.624392+0000","last_became_peered":"2023-10-16T11:19:35.624392+0000","last_unstale":"2023-10-16T11:19:37.026701+0000","last_undegraded":"2023-10-16T11:19:37.026701+0000","last_fullsized":"2023-10-16T11:19:37.026701+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:33:14.543389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621461+0000","last_change":"2023-10-16T11:19:36.626287+0000","last_active":"2023-10-16T11:19:37.621461+0000","last_peered":"2023-10-16T11:19:37.621461+0000","last_clean":"2023-10-16T11:19:37.621461+0000","last_became_active":"2023-10-16T11:19:36.625588+0000","last_became_peered":"2023-10-16T11:19:36.625588+0000","last_unstale":"2023-10-16T11:19:37.621461+0000","last_undegraded":"2023-10-16T11:19:37.621461+0000","last_fullsized":"2023-10-16T11:19:37.621461+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:37.781294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615845+0000","last_change":"2023-10-16T11:19:22.545557+0000","last_active":"2023-10-16T11:19:36.615845+0000","last_peered":"2023-10-16T11:19:36.615845+0000","last_clean":"2023-10-16T11:19:36.615845+0000","last_became_active":"2023-10-16T11:19:22.545253+0000","last_became_peered":"2023-10-16T11:19:22.545253+0000","last_unstale":"2023-10-16T11:19:36.615845+0000","last_undegraded":"2023-10-16T11:19:36.615845+0000","last_fullsized":"2023-10-16T11:19:36.615845+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:11.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619711+0000","last_change":"2023-10-16T11:19:23.553070+0000","last_active":"2023-10-16T11:19:36.619711+0000","last_peered":"2023-10-16T11:19:36.619711+0000","last_clean":"2023-10-16T11:19:36.619711+0000","last_became_active":"2023-10-16T11:19:23.552761+0000","last_became_peered":"2023-10-16T11:19:23.552761+0000","last_unstale":"2023-10-16T11:19:36.619711+0000","last_undegraded":"2023-10-16T11:19:36.619711+0000","last_fullsized":"2023-10-16T11:19:36.619711+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:36.373122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616224+0000","last_change":"2023-10-16T11:19:35.623394+0000","last_active":"2023-10-16T11:19:36.616224+0000","last_peered":"2023-10-16T11:19:36.616224+0000","last_clean":"2023-10-16T11:19:36.616224+0000","last_became_active":"2023-10-16T11:19:35.623119+0000","last_became_peered":"2023-10-16T11:19:35.623119+0000","last_unstale":"2023-10-16T11:19:36.616224+0000","last_undegraded":"2023-10-16T11:19:36.616224+0000","last_fullsized":"2023-10-16T11:19:36.616224+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:07.710682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892867+0000","last_change":"2023-10-16T11:19:36.633676+0000","last_active":"2023-10-16T11:19:37.892867+0000","last_peered":"2023-10-16T11:19:37.892867+0000","last_clean":"2023-10-16T11:19:37.892867+0000","last_became_active":"2023-10-16T11:19:36.633464+0000","last_became_peered":"2023-10-16T11:19:36.633464+0000","last_unstale":"2023-10-16T11:19:37.892867+0000","last_undegraded":"2023-10-16T11:19:37.892867+0000","last_fullsized":"2023-10-16T11:19:37.892867+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624527+0000","last_change":"2023-10-16T11:19:22.534080+0000","last_active":"2023-10-16T11:19:36.624527+0000","last_peered":"2023-10-16T11:19:36.624527+0000","last_clean":"2023-10-16T11:19:36.624527+0000","last_became_active":"2023-10-16T11:19:22.533693+0000","last_became_peered":"2023-10-16T11:19:22.533693+0000","last_unstale":"2023-10-16T11:19:36.624527+0000","last_undegraded":"2023-10-16T11:19:36.624527+0000","last_fullsized":"2023-10-16T11:19:36.624527+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:57.023698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617545+0000","last_change":"2023-10-16T11:19:23.546966+0000","last_active":"2023-10-16T11:19:36.617545+0000","last_peered":"2023-10-16T11:19:36.617545+0000","last_clean":"2023-10-16T11:19:36.617545+0000","last_became_active":"2023-10-16T11:19:23.546685+0000","last_became_peered":"2023-10-16T11:19:23.546685+0000","last_unstale":"2023-10-16T11:19:36.617545+0000","last_undegraded":"2023-10-16T11:19:36.617545+0000","last_fullsized":"2023-10-16T11:19:36.617545+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:13.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.025002+0000","last_change":"2023-10-16T11:19:35.624998+0000","last_active":"2023-10-16T11:19:37.025002+0000","last_peered":"2023-10-16T11:19:37.025002+0000","last_clean":"2023-10-16T11:19:37.025002+0000","last_became_active":"2023-10-16T11:19:35.624685+0000","last_became_peered":"2023-10-16T11:19:35.624685+0000","last_unstale":"2023-10-16T11:19:37.025002+0000","last_undegraded":"2023-10-16T11:19:37.025002+0000","last_fullsized":"2023-10-16T11:19:37.025002+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:52:30.461865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622678+0000","last_change":"2023-10-16T11:19:36.638230+0000","last_active":"2023-10-16T11:19:37.622678+0000","last_peered":"2023-10-16T11:19:37.622678+0000","last_clean":"2023-10-16T11:19:37.622678+0000","last_became_active":"2023-10-16T11:19:36.637937+0000","last_became_peered":"2023-10-16T11:19:36.637937+0000","last_unstale":"2023-10-16T11:19:37.622678+0000","last_undegraded":"2023-10-16T11:19:37.622678+0000","last_fullsized":"2023-10-16T11:19:37.622678+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:21.232537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616327+0000","last_change":"2023-10-16T11:19:22.547192+0000","last_active":"2023-10-16T11:19:36.616327+0000","last_peered":"2023-10-16T11:19:36.616327+0000","last_clean":"2023-10-16T11:19:36.616327+0000","last_became_active":"2023-10-16T11:19:22.546912+0000","last_became_peered":"2023-10-16T11:19:22.546912+0000","last_unstale":"2023-10-16T11:19:36.616327+0000","last_undegraded":"2023-10-16T11:19:36.616327+0000","last_fullsized":"2023-10-16T11:19:36.616327+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:17.017260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.614895+0000","last_change":"2023-10-16T11:19:23.543708+0000","last_active":"2023-10-16T11:19:36.614895+0000","last_peered":"2023-10-16T11:19:36.614895+0000","last_clean":"2023-10-16T11:19:36.614895+0000","last_became_active":"2023-10-16T11:19:23.543449+0000","last_became_peered":"2023-10-16T11:19:23.543449+0000","last_unstale":"2023-10-16T11:19:36.614895+0000","last_undegraded":"2023-10-16T11:19:36.614895+0000","last_fullsized":"2023-10-16T11:19:36.614895+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:06.406149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"219'4","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023703+0000","last_change":"2023-10-16T11:19:35.626621+0000","last_active":"2023-10-16T11:19:37.023703+0000","last_peered":"2023-10-16T11:19:37.023703+0000","last_clean":"2023-10-16T11:19:37.023703+0000","last_became_active":"2023-10-16T11:19:35.626338+0000","last_became_peered":"2023-10-16T11:19:35.626338+0000","last_unstale":"2023-10-16T11:19:37.023703+0000","last_undegraded":"2023-10-16T11:19:37.023703+0000","last_fullsized":"2023-10-16T11:19:37.023703+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894242+0000","last_change":"2023-10-16T11:19:36.638939+0000","last_active":"2023-10-16T11:19:37.894242+0000","last_peered":"2023-10-16T11:19:37.894242+0000","last_clean":"2023-10-16T11:19:37.894242+0000","last_became_active":"2023-10-16T11:19:36.637185+0000","last_became_peered":"2023-10-16T11:19:36.637185+0000","last_unstale":"2023-10-16T11:19:37.894242+0000","last_undegraded":"2023-10-16T11:19:37.894242+0000","last_fullsized":"2023-10-16T11:19:37.894242+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:26.350349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615222+0000","last_change":"2023-10-16T11:19:22.543956+0000","last_active":"2023-10-16T11:19:36.615222+0000","last_peered":"2023-10-16T11:19:36.615222+0000","last_clean":"2023-10-16T11:19:36.615222+0000","last_became_active":"2023-10-16T11:19:22.543633+0000","last_became_peered":"2023-10-16T11:19:22.543633+0000","last_unstale":"2023-10-16T11:19:36.615222+0000","last_undegraded":"2023-10-16T11:19:36.615222+0000","last_fullsized":"2023-10-16T11:19:36.615222+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:12:53.412047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617060+0000","last_change":"2023-10-16T11:19:23.546200+0000","last_active":"2023-10-16T11:19:36.617060+0000","last_peered":"2023-10-16T11:19:36.617060+0000","last_clean":"2023-10-16T11:19:36.617060+0000","last_became_active":"2023-10-16T11:19:23.545867+0000","last_became_peered":"2023-10-16T11:19:23.545867+0000","last_unstale":"2023-10-16T11:19:36.617060+0000","last_undegraded":"2023-10-16T11:19:36.617060+0000","last_fullsized":"2023-10-16T11:19:36.617060+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.670688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023629+0000","last_change":"2023-10-16T11:19:35.628631+0000","last_active":"2023-10-16T11:19:37.023629+0000","last_peered":"2023-10-16T11:19:37.023629+0000","last_clean":"2023-10-16T11:19:37.023629+0000","last_became_active":"2023-10-16T11:19:35.628341+0000","last_became_peered":"2023-10-16T11:19:35.628341+0000","last_unstale":"2023-10-16T11:19:37.023629+0000","last_undegraded":"2023-10-16T11:19:37.023629+0000","last_fullsized":"2023-10-16T11:19:37.023629+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:48:39.437826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621875+0000","last_change":"2023-10-16T11:19:36.635937+0000","last_active":"2023-10-16T11:19:37.621875+0000","last_peered":"2023-10-16T11:19:37.621875+0000","last_clean":"2023-10-16T11:19:37.621875+0000","last_became_active":"2023-10-16T11:19:36.635570+0000","last_became_peered":"2023-10-16T11:19:36.635570+0000","last_unstale":"2023-10-16T11:19:37.621875+0000","last_undegraded":"2023-10-16T11:19:37.621875+0000","last_fullsized":"2023-10-16T11:19:37.621875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:19.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617428+0000","last_change":"2023-10-16T11:19:22.535919+0000","last_active":"2023-10-16T11:19:36.617428+0000","last_peered":"2023-10-16T11:19:36.617428+0000","last_clean":"2023-10-16T11:19:36.617428+0000","last_became_active":"2023-10-16T11:19:22.535612+0000","last_became_peered":"2023-10-16T11:19:22.535612+0000","last_unstale":"2023-10-16T11:19:36.617428+0000","last_undegraded":"2023-10-16T11:19:36.617428+0000","last_fullsized":"2023-10-16T11:19:36.617428+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:54.628429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623571+0000","last_change":"2023-10-16T11:19:23.552923+0000","last_active":"2023-10-16T11:19:36.623571+0000","last_peered":"2023-10-16T11:19:36.623571+0000","last_clean":"2023-10-16T11:19:36.623571+0000","last_became_active":"2023-10-16T11:19:23.552685+0000","last_became_peered":"2023-10-16T11:19:23.552685+0000","last_unstale":"2023-10-16T11:19:36.623571+0000","last_undegraded":"2023-10-16T11:19:36.623571+0000","last_fullsized":"2023-10-16T11:19:36.623571+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:13.492062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023727+0000","last_change":"2023-10-16T11:19:35.630995+0000","last_active":"2023-10-16T11:19:37.023727+0000","last_peered":"2023-10-16T11:19:37.023727+0000","last_clean":"2023-10-16T11:19:37.023727+0000","last_became_active":"2023-10-16T11:19:35.630561+0000","last_became_peered":"2023-10-16T11:19:35.630561+0000","last_unstale":"2023-10-16T11:19:37.023727+0000","last_undegraded":"2023-10-16T11:19:37.023727+0000","last_fullsized":"2023-10-16T11:19:37.023727+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:30.099513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893198+0000","last_change":"2023-10-16T11:19:36.631163+0000","last_active":"2023-10-16T11:19:37.893198+0000","last_peered":"2023-10-16T11:19:37.893198+0000","last_clean":"2023-10-16T11:19:37.893198+0000","last_became_active":"2023-10-16T11:19:36.630820+0000","last_became_peered":"2023-10-16T11:19:36.630820+0000","last_unstale":"2023-10-16T11:19:37.893198+0000","last_undegraded":"2023-10-16T11:19:37.893198+0000","last_fullsized":"2023-10-16T11:19:37.893198+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:54.687768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628981+0000","last_change":"2023-10-16T11:19:22.533919+0000","last_active":"2023-10-16T11:19:36.628981+0000","last_peered":"2023-10-16T11:19:36.628981+0000","last_clean":"2023-10-16T11:19:36.628981+0000","last_became_active":"2023-10-16T11:19:22.533695+0000","last_became_peered":"2023-10-16T11:19:22.533695+0000","last_unstale":"2023-10-16T11:19:36.628981+0000","last_undegraded":"2023-10-16T11:19:36.628981+0000","last_fullsized":"2023-10-16T11:19:36.628981+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618694+0000","last_change":"2023-10-16T11:19:23.546849+0000","last_active":"2023-10-16T11:19:36.618694+0000","last_peered":"2023-10-16T11:19:36.618694+0000","last_clean":"2023-10-16T11:19:36.618694+0000","last_became_active":"2023-10-16T11:19:23.546575+0000","last_became_peered":"2023-10-16T11:19:23.546575+0000","last_unstale":"2023-10-16T11:19:36.618694+0000","last_undegraded":"2023-10-16T11:19:36.618694+0000","last_fullsized":"2023-10-16T11:19:36.618694+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.884361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.024943+0000","last_change":"2023-10-16T11:19:35.624948+0000","last_active":"2023-10-16T11:19:37.024943+0000","last_peered":"2023-10-16T11:19:37.024943+0000","last_clean":"2023-10-16T11:19:37.024943+0000","last_became_active":"2023-10-16T11:19:35.624339+0000","last_became_peered":"2023-10-16T11:19:35.624339+0000","last_unstale":"2023-10-16T11:19:37.024943+0000","last_undegraded":"2023-10-16T11:19:37.024943+0000","last_fullsized":"2023-10-16T11:19:37.024943+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893006+0000","last_change":"2023-10-16T11:19:36.635048+0000","last_active":"2023-10-16T11:19:37.893006+0000","last_peered":"2023-10-16T11:19:37.893006+0000","last_clean":"2023-10-16T11:19:37.893006+0000","last_became_active":"2023-10-16T11:19:36.634362+0000","last_became_peered":"2023-10-16T11:19:36.634362+0000","last_unstale":"2023-10-16T11:19:37.893006+0000","last_undegraded":"2023-10-16T11:19:37.893006+0000","last_fullsized":"2023-10-16T11:19:37.893006+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:52.979732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615890+0000","last_change":"2023-10-16T11:19:22.536812+0000","last_active":"2023-10-16T11:19:36.615890+0000","last_peered":"2023-10-16T11:19:36.615890+0000","last_clean":"2023-10-16T11:19:36.615890+0000","last_became_active":"2023-10-16T11:19:22.536545+0000","last_became_peered":"2023-10-16T11:19:22.536545+0000","last_unstale":"2023-10-16T11:19:36.615890+0000","last_undegraded":"2023-10-16T11:19:36.615890+0000","last_fullsized":"2023-10-16T11:19:36.615890+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624796+0000","last_change":"2023-10-16T11:19:23.559791+0000","last_active":"2023-10-16T11:19:36.624796+0000","last_peered":"2023-10-16T11:19:36.624796+0000","last_clean":"2023-10-16T11:19:36.624796+0000","last_became_active":"2023-10-16T11:19:23.559513+0000","last_became_peered":"2023-10-16T11:19:23.559513+0000","last_unstale":"2023-10-16T11:19:36.624796+0000","last_undegraded":"2023-10-16T11:19:36.624796+0000","last_fullsized":"2023-10-16T11:19:36.624796+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:39.986537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621822+0000","last_change":"2023-10-16T11:19:35.622388+0000","last_active":"2023-10-16T11:19:36.621822+0000","last_peered":"2023-10-16T11:19:36.621822+0000","last_clean":"2023-10-16T11:19:36.621822+0000","last_became_active":"2023-10-16T11:19:35.622128+0000","last_became_peered":"2023-10-16T11:19:35.622128+0000","last_unstale":"2023-10-16T11:19:36.621822+0000","last_undegraded":"2023-10-16T11:19:36.621822+0000","last_fullsized":"2023-10-16T11:19:36.621822+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:10.516621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892728+0000","last_change":"2023-10-16T11:19:36.636570+0000","last_active":"2023-10-16T11:19:37.892728+0000","last_peered":"2023-10-16T11:19:37.892728+0000","last_clean":"2023-10-16T11:19:37.892728+0000","last_became_active":"2023-10-16T11:19:36.636314+0000","last_became_peered":"2023-10-16T11:19:36.636314+0000","last_unstale":"2023-10-16T11:19:37.892728+0000","last_undegraded":"2023-10-16T11:19:37.892728+0000","last_fullsized":"2023-10-16T11:19:37.892728+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:19.250783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":20,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631343+0000","last_change":"2023-10-16T11:19:22.538829+0000","last_active":"2023-10-16T11:19:36.631343+0000","last_peered":"2023-10-16T11:19:36.631343+0000","last_clean":"2023-10-16T11:19:36.631343+0000","last_became_active":"2023-10-16T11:19:22.538537+0000","last_became_peered":"2023-10-16T11:19:22.538537+0000","last_unstale":"2023-10-16T11:19:36.631343+0000","last_undegraded":"2023-10-16T11:19:36.631343+0000","last_fullsized":"2023-10-16T11:19:36.631343+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:09.892801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615802+0000","last_change":"2023-10-16T11:19:23.551775+0000","last_active":"2023-10-16T11:19:36.615802+0000","last_peered":"2023-10-16T11:19:36.615802+0000","last_clean":"2023-10-16T11:19:36.615802+0000","last_became_active":"2023-10-16T11:19:23.551490+0000","last_became_peered":"2023-10-16T11:19:23.551490+0000","last_unstale":"2023-10-16T11:19:36.615802+0000","last_undegraded":"2023-10-16T11:19:36.615802+0000","last_fullsized":"2023-10-16T11:19:36.615802+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618085+0000","last_change":"2023-10-16T11:19:35.625342+0000","last_active":"2023-10-16T11:19:36.618085+0000","last_peered":"2023-10-16T11:19:36.618085+0000","last_clean":"2023-10-16T11:19:36.618085+0000","last_became_active":"2023-10-16T11:19:35.624933+0000","last_became_peered":"2023-10-16T11:19:35.624933+0000","last_unstale":"2023-10-16T11:19:36.618085+0000","last_undegraded":"2023-10-16T11:19:36.618085+0000","last_fullsized":"2023-10-16T11:19:36.618085+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:20.226818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892577+0000","last_change":"2023-10-16T11:19:36.630442+0000","last_active":"2023-10-16T11:19:37.892577+0000","last_peered":"2023-10-16T11:19:37.892577+0000","last_clean":"2023-10-16T11:19:37.892577+0000","last_became_active":"2023-10-16T11:19:36.630151+0000","last_became_peered":"2023-10-16T11:19:36.630151+0000","last_unstale":"2023-10-16T11:19:37.892577+0000","last_undegraded":"2023-10-16T11:19:37.892577+0000","last_fullsized":"2023-10-16T11:19:37.892577+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:33.781861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622379+0000","last_change":"2023-10-16T11:19:36.633999+0000","last_active":"2023-10-16T11:19:37.622379+0000","last_peered":"2023-10-16T11:19:37.622379+0000","last_clean":"2023-10-16T11:19:37.622379+0000","last_became_active":"2023-10-16T11:19:36.633722+0000","last_became_peered":"2023-10-16T11:19:36.633722+0000","last_unstale":"2023-10-16T11:19:37.622379+0000","last_undegraded":"2023-10-16T11:19:37.622379+0000","last_fullsized":"2023-10-16T11:19:37.622379+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618948+0000","last_change":"2023-10-16T11:19:23.544178+0000","last_active":"2023-10-16T11:19:36.618948+0000","last_peered":"2023-10-16T11:19:36.618948+0000","last_clean":"2023-10-16T11:19:36.618948+0000","last_became_active":"2023-10-16T11:19:23.543898+0000","last_became_peered":"2023-10-16T11:19:23.543898+0000","last_unstale":"2023-10-16T11:19:36.618948+0000","last_undegraded":"2023-10-16T11:19:36.618948+0000","last_fullsized":"2023-10-16T11:19:36.618948+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893954+0000","last_change":"2023-10-16T11:19:36.638088+0000","last_active":"2023-10-16T11:19:37.893954+0000","last_peered":"2023-10-16T11:19:37.893954+0000","last_clean":"2023-10-16T11:19:37.893954+0000","last_became_active":"2023-10-16T11:19:36.637717+0000","last_became_peered":"2023-10-16T11:19:36.637717+0000","last_unstale":"2023-10-16T11:19:37.893954+0000","last_undegraded":"2023-10-16T11:19:37.893954+0000","last_fullsized":"2023-10-16T11:19:37.893954+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:19.443732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621366+0000","last_change":"2023-10-16T11:19:23.542878+0000","last_active":"2023-10-16T11:19:36.621366+0000","last_peered":"2023-10-16T11:19:36.621366+0000","last_clean":"2023-10-16T11:19:36.621366+0000","last_became_active":"2023-10-16T11:19:23.542593+0000","last_became_peered":"2023-10-16T11:19:23.542593+0000","last_unstale":"2023-10-16T11:19:36.621366+0000","last_undegraded":"2023-10-16T11:19:36.621366+0000","last_fullsized":"2023-10-16T11:19:36.621366+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:39.898154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616551+0000","last_change":"2023-10-16T11:19:23.547762+0000","last_active":"2023-10-16T11:19:36.616551+0000","last_peered":"2023-10-16T11:19:36.616551+0000","last_clean":"2023-10-16T11:19:36.616551+0000","last_became_active":"2023-10-16T11:19:23.547353+0000","last_became_peered":"2023-10-16T11:19:23.547353+0000","last_unstale":"2023-10-16T11:19:36.616551+0000","last_undegraded":"2023-10-16T11:19:36.616551+0000","last_fullsized":"2023-10-16T11:19:36.616551+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893121+0000","last_change":"2023-10-16T11:19:36.630687+0000","last_active":"2023-10-16T11:19:37.893121+0000","last_peered":"2023-10-16T11:19:37.893121+0000","last_clean":"2023-10-16T11:19:37.893121+0000","last_became_active":"2023-10-16T11:19:36.629515+0000","last_became_peered":"2023-10-16T11:19:36.629515+0000","last_unstale":"2023-10-16T11:19:37.893121+0000","last_undegraded":"2023-10-16T11:19:37.893121+0000","last_fullsized":"2023-10-16T11:19:37.893121+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:25.525530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893412+0000","last_change":"2023-10-16T11:19:36.634680+0000","last_active":"2023-10-16T11:19:37.893412+0000","last_peered":"2023-10-16T11:19:37.893412+0000","last_clean":"2023-10-16T11:19:37.893412+0000","last_became_active":"2023-10-16T11:19:36.632581+0000","last_became_peered":"2023-10-16T11:19:36.632581+0000","last_unstale":"2023-10-16T11:19:37.893412+0000","last_undegraded":"2023-10-16T11:19:37.893412+0000","last_fullsized":"2023-10-16T11:19:37.893412+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.278663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630199+0000","last_change":"2023-10-16T11:19:23.565127+0000","last_active":"2023-10-16T11:19:36.630199+0000","last_peered":"2023-10-16T11:19:36.630199+0000","last_clean":"2023-10-16T11:19:36.630199+0000","last_became_active":"2023-10-16T11:19:23.564852+0000","last_became_peered":"2023-10-16T11:19:23.564852+0000","last_unstale":"2023-10-16T11:19:36.630199+0000","last_undegraded":"2023-10-16T11:19:36.630199+0000","last_fullsized":"2023-10-16T11:19:36.630199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629214+0000","last_change":"2023-10-16T11:19:23.555152+0000","last_active":"2023-10-16T11:19:36.629214+0000","last_peered":"2023-10-16T11:19:36.629214+0000","last_clean":"2023-10-16T11:19:36.629214+0000","last_became_active":"2023-10-16T11:19:23.554747+0000","last_became_peered":"2023-10-16T11:19:23.554747+0000","last_unstale":"2023-10-16T11:19:36.629214+0000","last_undegraded":"2023-10-16T11:19:36.629214+0000","last_fullsized":"2023-10-16T11:19:36.629214+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892848+0000","last_change":"2023-10-16T11:19:36.635095+0000","last_active":"2023-10-16T11:19:37.892848+0000","last_peered":"2023-10-16T11:19:37.892848+0000","last_clean":"2023-10-16T11:19:37.892848+0000","last_became_active":"2023-10-16T11:19:36.634778+0000","last_became_peered":"2023-10-16T11:19:36.634778+0000","last_unstale":"2023-10-16T11:19:37.892848+0000","last_undegraded":"2023-10-16T11:19:37.892848+0000","last_fullsized":"2023-10-16T11:19:37.892848+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:34.012796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630759+0000","last_change":"2023-10-16T11:19:23.559781+0000","last_active":"2023-10-16T11:19:36.630759+0000","last_peered":"2023-10-16T11:19:36.630759+0000","last_clean":"2023-10-16T11:19:36.630759+0000","last_became_active":"2023-10-16T11:19:23.559451+0000","last_became_peered":"2023-10-16T11:19:23.559451+0000","last_unstale":"2023-10-16T11:19:36.630759+0000","last_undegraded":"2023-10-16T11:19:36.630759+0000","last_fullsized":"2023-10-16T11:19:36.630759+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621656+0000","last_change":"2023-10-16T11:19:36.641041+0000","last_active":"2023-10-16T11:19:37.621656+0000","last_peered":"2023-10-16T11:19:37.621656+0000","last_clean":"2023-10-16T11:19:37.621656+0000","last_became_active":"2023-10-16T11:19:36.640761+0000","last_became_peered":"2023-10-16T11:19:36.640761+0000","last_unstale":"2023-10-16T11:19:37.621656+0000","last_undegraded":"2023-10-16T11:19:37.621656+0000","last_fullsized":"2023-10-16T11:19:37.621656+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:26.350349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622347+0000","last_change":"2023-10-16T11:19:23.546460+0000","last_active":"2023-10-16T11:19:36.622347+0000","last_peered":"2023-10-16T11:19:36.622347+0000","last_clean":"2023-10-16T11:19:36.622347+0000","last_became_active":"2023-10-16T11:19:23.546187+0000","last_became_peered":"2023-10-16T11:19:23.546187+0000","last_unstale":"2023-10-16T11:19:36.622347+0000","last_undegraded":"2023-10-16T11:19:36.622347+0000","last_fullsized":"2023-10-16T11:19:36.622347+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621306+0000","last_change":"2023-10-16T11:19:36.630530+0000","last_active":"2023-10-16T11:19:37.621306+0000","last_peered":"2023-10-16T11:19:37.621306+0000","last_clean":"2023-10-16T11:19:37.621306+0000","last_became_active":"2023-10-16T11:19:36.630239+0000","last_became_peered":"2023-10-16T11:19:36.630239+0000","last_unstale":"2023-10-16T11:19:37.621306+0000","last_undegraded":"2023-10-16T11:19:37.621306+0000","last_fullsized":"2023-10-16T11:19:37.621306+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616639+0000","last_change":"2023-10-16T11:19:23.546857+0000","last_active":"2023-10-16T11:19:36.616639+0000","last_peered":"2023-10-16T11:19:36.616639+0000","last_clean":"2023-10-16T11:19:36.616639+0000","last_became_active":"2023-10-16T11:19:23.546592+0000","last_became_peered":"2023-10-16T11:19:23.546592+0000","last_unstale":"2023-10-16T11:19:36.616639+0000","last_undegraded":"2023-10-16T11:19:36.616639+0000","last_fullsized":"2023-10-16T11:19:36.616639+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893169+0000","last_change":"2023-10-16T11:19:36.622716+0000","last_active":"2023-10-16T11:19:37.893169+0000","last_peered":"2023-10-16T11:19:37.893169+0000","last_clean":"2023-10-16T11:19:37.893169+0000","last_became_active":"2023-10-16T11:19:36.622420+0000","last_became_peered":"2023-10-16T11:19:36.622420+0000","last_unstale":"2023-10-16T11:19:37.893169+0000","last_undegraded":"2023-10-16T11:19:37.893169+0000","last_fullsized":"2023-10-16T11:19:37.893169+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:58.765425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893941+0000","last_change":"2023-10-16T11:19:36.633980+0000","last_active":"2023-10-16T11:19:37.893941+0000","last_peered":"2023-10-16T11:19:37.893941+0000","last_clean":"2023-10-16T11:19:37.893941+0000","last_became_active":"2023-10-16T11:19:36.633643+0000","last_became_peered":"2023-10-16T11:19:36.633643+0000","last_unstale":"2023-10-16T11:19:37.893941+0000","last_undegraded":"2023-10-16T11:19:37.893941+0000","last_fullsized":"2023-10-16T11:19:37.893941+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:27.398549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617918+0000","last_change":"2023-10-16T11:19:23.548868+0000","last_active":"2023-10-16T11:19:36.617918+0000","last_peered":"2023-10-16T11:19:36.617918+0000","last_clean":"2023-10-16T11:19:36.617918+0000","last_became_active":"2023-10-16T11:19:23.548583+0000","last_became_peered":"2023-10-16T11:19:23.548583+0000","last_unstale":"2023-10-16T11:19:36.617918+0000","last_undegraded":"2023-10-16T11:19:36.617918+0000","last_fullsized":"2023-10-16T11:19:36.617918+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:02.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622147+0000","last_change":"2023-10-16T11:19:36.628944+0000","last_active":"2023-10-16T11:19:37.622147+0000","last_peered":"2023-10-16T11:19:37.622147+0000","last_clean":"2023-10-16T11:19:37.622147+0000","last_became_active":"2023-10-16T11:19:36.628652+0000","last_became_peered":"2023-10-16T11:19:36.628652+0000","last_unstale":"2023-10-16T11:19:37.622147+0000","last_undegraded":"2023-10-16T11:19:37.622147+0000","last_fullsized":"2023-10-16T11:19:37.622147+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:18.537927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624713+0000","last_change":"2023-10-16T11:19:23.552048+0000","last_active":"2023-10-16T11:19:36.624713+0000","last_peered":"2023-10-16T11:19:36.624713+0000","last_clean":"2023-10-16T11:19:36.624713+0000","last_became_active":"2023-10-16T11:19:23.551676+0000","last_became_peered":"2023-10-16T11:19:23.551676+0000","last_unstale":"2023-10-16T11:19:36.624713+0000","last_undegraded":"2023-10-16T11:19:36.624713+0000","last_fullsized":"2023-10-16T11:19:36.624713+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:16.450373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893328+0000","last_change":"2023-10-16T11:19:36.626030+0000","last_active":"2023-10-16T11:19:37.893328+0000","last_peered":"2023-10-16T11:19:37.893328+0000","last_clean":"2023-10-16T11:19:37.893328+0000","last_became_active":"2023-10-16T11:19:36.625727+0000","last_became_peered":"2023-10-16T11:19:36.625727+0000","last_unstale":"2023-10-16T11:19:37.893328+0000","last_undegraded":"2023-10-16T11:19:37.893328+0000","last_fullsized":"2023-10-16T11:19:37.893328+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:07:58.374107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624300+0000","last_change":"2023-10-16T11:19:23.546038+0000","last_active":"2023-10-16T11:19:36.624300+0000","last_peered":"2023-10-16T11:19:36.624300+0000","last_clean":"2023-10-16T11:19:36.624300+0000","last_became_active":"2023-10-16T11:19:23.545752+0000","last_became_peered":"2023-10-16T11:19:23.545752+0000","last_unstale":"2023-10-16T11:19:36.624300+0000","last_undegraded":"2023-10-16T11:19:36.624300+0000","last_fullsized":"2023-10-16T11:19:36.624300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:13.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893888+0000","last_change":"2023-10-16T11:19:36.629691+0000","last_active":"2023-10-16T11:19:37.893888+0000","last_peered":"2023-10-16T11:19:37.893888+0000","last_clean":"2023-10-16T11:19:37.893888+0000","last_became_active":"2023-10-16T11:19:36.629384+0000","last_became_peered":"2023-10-16T11:19:36.629384+0000","last_unstale":"2023-10-16T11:19:37.893888+0000","last_undegraded":"2023-10-16T11:19:37.893888+0000","last_fullsized":"2023-10-16T11:19:37.893888+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:15.903147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619019+0000","last_change":"2023-10-16T11:19:23.550209+0000","last_active":"2023-10-16T11:19:36.619019+0000","last_peered":"2023-10-16T11:19:36.619019+0000","last_clean":"2023-10-16T11:19:36.619019+0000","last_became_active":"2023-10-16T11:19:23.549849+0000","last_became_peered":"2023-10-16T11:19:23.549849+0000","last_unstale":"2023-10-16T11:19:36.619019+0000","last_undegraded":"2023-10-16T11:19:36.619019+0000","last_fullsized":"2023-10-16T11:19:36.619019+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:29.715382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621124+0000","last_change":"2023-10-16T11:19:36.631361+0000","last_active":"2023-10-16T11:19:37.621124+0000","last_peered":"2023-10-16T11:19:37.621124+0000","last_clean":"2023-10-16T11:19:37.621124+0000","last_became_active":"2023-10-16T11:19:36.631013+0000","last_became_peered":"2023-10-16T11:19:36.631013+0000","last_unstale":"2023-10-16T11:19:37.621124+0000","last_undegraded":"2023-10-16T11:19:37.621124+0000","last_fullsized":"2023-10-16T11:19:37.621124+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615695+0000","last_change":"2023-10-16T11:19:23.542321+0000","last_active":"2023-10-16T11:19:36.615695+0000","last_peered":"2023-10-16T11:19:36.615695+0000","last_clean":"2023-10-16T11:19:36.615695+0000","last_became_active":"2023-10-16T11:19:23.541862+0000","last_became_peered":"2023-10-16T11:19:23.541862+0000","last_unstale":"2023-10-16T11:19:36.615695+0000","last_undegraded":"2023-10-16T11:19:36.615695+0000","last_fullsized":"2023-10-16T11:19:36.615695+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:24:50.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892711+0000","last_change":"2023-10-16T11:19:36.620937+0000","last_active":"2023-10-16T11:19:37.892711+0000","last_peered":"2023-10-16T11:19:37.892711+0000","last_clean":"2023-10-16T11:19:37.892711+0000","last_became_active":"2023-10-16T11:19:36.620641+0000","last_became_peered":"2023-10-16T11:19:36.620641+0000","last_unstale":"2023-10-16T11:19:37.892711+0000","last_undegraded":"2023-10-16T11:19:37.892711+0000","last_fullsized":"2023-10-16T11:19:37.892711+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:15.827874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623487+0000","last_change":"2023-10-16T11:19:23.547740+0000","last_active":"2023-10-16T11:19:36.623487+0000","last_peered":"2023-10-16T11:19:36.623487+0000","last_clean":"2023-10-16T11:19:36.623487+0000","last_became_active":"2023-10-16T11:19:23.546421+0000","last_became_peered":"2023-10-16T11:19:23.546421+0000","last_unstale":"2023-10-16T11:19:36.623487+0000","last_undegraded":"2023-10-16T11:19:36.623487+0000","last_fullsized":"2023-10-16T11:19:36.623487+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:27.685924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892763+0000","last_change":"2023-10-16T11:19:36.631333+0000","last_active":"2023-10-16T11:19:37.892763+0000","last_peered":"2023-10-16T11:19:37.892763+0000","last_clean":"2023-10-16T11:19:37.892763+0000","last_became_active":"2023-10-16T11:19:36.630735+0000","last_became_peered":"2023-10-16T11:19:36.630735+0000","last_unstale":"2023-10-16T11:19:37.892763+0000","last_undegraded":"2023-10-16T11:19:37.892763+0000","last_fullsized":"2023-10-16T11:19:37.892763+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:26.684291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623942+0000","last_change":"2023-10-16T11:19:23.549769+0000","last_active":"2023-10-16T11:19:36.623942+0000","last_peered":"2023-10-16T11:19:36.623942+0000","last_clean":"2023-10-16T11:19:36.623942+0000","last_became_active":"2023-10-16T11:19:23.549505+0000","last_became_peered":"2023-10-16T11:19:23.549505+0000","last_unstale":"2023-10-16T11:19:36.623942+0000","last_undegraded":"2023-10-16T11:19:36.623942+0000","last_fullsized":"2023-10-16T11:19:36.623942+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:20.931218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892826+0000","last_change":"2023-10-16T11:19:36.631224+0000","last_active":"2023-10-16T11:19:37.892826+0000","last_peered":"2023-10-16T11:19:37.892826+0000","last_clean":"2023-10-16T11:19:37.892826+0000","last_became_active":"2023-10-16T11:19:36.630966+0000","last_became_peered":"2023-10-16T11:19:36.630966+0000","last_unstale":"2023-10-16T11:19:37.892826+0000","last_undegraded":"2023-10-16T11:19:37.892826+0000","last_fullsized":"2023-10-16T11:19:37.892826+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:45.783168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622027+0000","last_change":"2023-10-16T11:19:23.548276+0000","last_active":"2023-10-16T11:19:36.622027+0000","last_peered":"2023-10-16T11:19:36.622027+0000","last_clean":"2023-10-16T11:19:36.622027+0000","last_became_active":"2023-10-16T11:19:23.547177+0000","last_became_peered":"2023-10-16T11:19:23.547177+0000","last_unstale":"2023-10-16T11:19:36.622027+0000","last_undegraded":"2023-10-16T11:19:36.622027+0000","last_fullsized":"2023-10-16T11:19:36.622027+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893905+0000","last_change":"2023-10-16T11:19:36.635809+0000","last_active":"2023-10-16T11:19:37.893905+0000","last_peered":"2023-10-16T11:19:37.893905+0000","last_clean":"2023-10-16T11:19:37.893905+0000","last_became_active":"2023-10-16T11:19:36.635543+0000","last_became_peered":"2023-10-16T11:19:36.635543+0000","last_unstale":"2023-10-16T11:19:37.893905+0000","last_undegraded":"2023-10-16T11:19:37.893905+0000","last_fullsized":"2023-10-16T11:19:37.893905+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:06:05.832943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621596+0000","last_change":"2023-10-16T11:19:23.563754+0000","last_active":"2023-10-16T11:19:36.621596+0000","last_peered":"2023-10-16T11:19:36.621596+0000","last_clean":"2023-10-16T11:19:36.621596+0000","last_became_active":"2023-10-16T11:19:23.563487+0000","last_became_peered":"2023-10-16T11:19:23.563487+0000","last_unstale":"2023-10-16T11:19:36.621596+0000","last_undegraded":"2023-10-16T11:19:36.621596+0000","last_fullsized":"2023-10-16T11:19:36.621596+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:13.814299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893984+0000","last_change":"2023-10-16T11:19:36.639804+0000","last_active":"2023-10-16T11:19:37.893984+0000","last_peered":"2023-10-16T11:19:37.893984+0000","last_clean":"2023-10-16T11:19:37.893984+0000","last_became_active":"2023-10-16T11:19:36.639421+0000","last_became_peered":"2023-10-16T11:19:36.639421+0000","last_unstale":"2023-10-16T11:19:37.893984+0000","last_undegraded":"2023-10-16T11:19:37.893984+0000","last_fullsized":"2023-10-16T11:19:37.893984+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:21.232537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619182+0000","last_change":"2023-10-16T11:19:23.551280+0000","last_active":"2023-10-16T11:19:36.619182+0000","last_peered":"2023-10-16T11:19:36.619182+0000","last_clean":"2023-10-16T11:19:36.619182+0000","last_became_active":"2023-10-16T11:19:23.551018+0000","last_became_peered":"2023-10-16T11:19:23.551018+0000","last_unstale":"2023-10-16T11:19:36.619182+0000","last_undegraded":"2023-10-16T11:19:36.619182+0000","last_fullsized":"2023-10-16T11:19:36.619182+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:46.256661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893878+0000","last_change":"2023-10-16T11:19:36.627799+0000","last_active":"2023-10-16T11:19:37.893878+0000","last_peered":"2023-10-16T11:19:37.893878+0000","last_clean":"2023-10-16T11:19:37.893878+0000","last_became_active":"2023-10-16T11:19:36.627495+0000","last_became_peered":"2023-10-16T11:19:36.627495+0000","last_unstale":"2023-10-16T11:19:37.893878+0000","last_undegraded":"2023-10-16T11:19:37.893878+0000","last_fullsized":"2023-10-16T11:19:37.893878+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:32.348270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620399+0000","last_change":"2023-10-16T11:19:23.559938+0000","last_active":"2023-10-16T11:19:36.620399+0000","last_peered":"2023-10-16T11:19:36.620399+0000","last_clean":"2023-10-16T11:19:36.620399+0000","last_became_active":"2023-10-16T11:19:23.559569+0000","last_became_peered":"2023-10-16T11:19:23.559569+0000","last_unstale":"2023-10-16T11:19:36.620399+0000","last_undegraded":"2023-10-16T11:19:36.620399+0000","last_fullsized":"2023-10-16T11:19:36.620399+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893192+0000","last_change":"2023-10-16T11:19:36.625387+0000","last_active":"2023-10-16T11:19:37.893192+0000","last_peered":"2023-10-16T11:19:37.893192+0000","last_clean":"2023-10-16T11:19:37.893192+0000","last_became_active":"2023-10-16T11:19:36.625097+0000","last_became_peered":"2023-10-16T11:19:36.625097+0000","last_unstale":"2023-10-16T11:19:37.893192+0000","last_undegraded":"2023-10-16T11:19:37.893192+0000","last_fullsized":"2023-10-16T11:19:37.893192+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:12.164686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630199+0000","last_change":"2023-10-16T11:19:23.560050+0000","last_active":"2023-10-16T11:19:36.630199+0000","last_peered":"2023-10-16T11:19:36.630199+0000","last_clean":"2023-10-16T11:19:36.630199+0000","last_became_active":"2023-10-16T11:19:23.559558+0000","last_became_peered":"2023-10-16T11:19:23.559558+0000","last_unstale":"2023-10-16T11:19:36.630199+0000","last_undegraded":"2023-10-16T11:19:36.630199+0000","last_fullsized":"2023-10-16T11:19:36.630199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621911+0000","last_change":"2023-10-16T11:19:36.629616+0000","last_active":"2023-10-16T11:19:37.621911+0000","last_peered":"2023-10-16T11:19:37.621911+0000","last_clean":"2023-10-16T11:19:37.621911+0000","last_became_active":"2023-10-16T11:19:36.629056+0000","last_became_peered":"2023-10-16T11:19:36.629056+0000","last_unstale":"2023-10-16T11:19:37.621911+0000","last_undegraded":"2023-10-16T11:19:37.621911+0000","last_fullsized":"2023-10-16T11:19:37.621911+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:11.302811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.632358+0000","last_change":"2023-10-16T11:19:23.560016+0000","last_active":"2023-10-16T11:19:36.632358+0000","last_peered":"2023-10-16T11:19:36.632358+0000","last_clean":"2023-10-16T11:19:36.632358+0000","last_became_active":"2023-10-16T11:19:23.559619+0000","last_became_peered":"2023-10-16T11:19:23.559619+0000","last_unstale":"2023-10-16T11:19:36.632358+0000","last_undegraded":"2023-10-16T11:19:36.632358+0000","last_fullsized":"2023-10-16T11:19:36.632358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:04.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893587+0000","last_change":"2023-10-16T11:19:36.628684+0000","last_active":"2023-10-16T11:19:37.893587+0000","last_peered":"2023-10-16T11:19:37.893587+0000","last_clean":"2023-10-16T11:19:37.893587+0000","last_became_active":"2023-10-16T11:19:36.628264+0000","last_became_peered":"2023-10-16T11:19:36.628264+0000","last_unstale":"2023-10-16T11:19:37.893587+0000","last_undegraded":"2023-10-16T11:19:37.893587+0000","last_fullsized":"2023-10-16T11:19:37.893587+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:41:54.029905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629289+0000","last_change":"2023-10-16T11:19:23.564191+0000","last_active":"2023-10-16T11:19:36.629289+0000","last_peered":"2023-10-16T11:19:36.629289+0000","last_clean":"2023-10-16T11:19:36.629289+0000","last_became_active":"2023-10-16T11:19:23.563291+0000","last_became_peered":"2023-10-16T11:19:23.563291+0000","last_unstale":"2023-10-16T11:19:36.629289+0000","last_undegraded":"2023-10-16T11:19:36.629289+0000","last_fullsized":"2023-10-16T11:19:36.629289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894719+0000","last_change":"2023-10-16T11:19:36.627078+0000","last_active":"2023-10-16T11:19:37.894719+0000","last_peered":"2023-10-16T11:19:37.894719+0000","last_clean":"2023-10-16T11:19:37.894719+0000","last_became_active":"2023-10-16T11:19:36.626791+0000","last_became_peered":"2023-10-16T11:19:36.626791+0000","last_unstale":"2023-10-16T11:19:37.894719+0000","last_undegraded":"2023-10-16T11:19:37.894719+0000","last_fullsized":"2023-10-16T11:19:37.894719+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:14.780239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623858+0000","last_change":"2023-10-16T11:19:23.546735+0000","last_active":"2023-10-16T11:19:36.623858+0000","last_peered":"2023-10-16T11:19:36.623858+0000","last_clean":"2023-10-16T11:19:36.623858+0000","last_became_active":"2023-10-16T11:19:23.546404+0000","last_became_peered":"2023-10-16T11:19:23.546404+0000","last_unstale":"2023-10-16T11:19:36.623858+0000","last_undegraded":"2023-10-16T11:19:36.623858+0000","last_fullsized":"2023-10-16T11:19:36.623858+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:29.845437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.620865+0000","last_change":"2023-10-16T11:19:36.630872+0000","last_active":"2023-10-16T11:19:37.620865+0000","last_peered":"2023-10-16T11:19:37.620865+0000","last_clean":"2023-10-16T11:19:37.620865+0000","last_became_active":"2023-10-16T11:19:36.630391+0000","last_became_peered":"2023-10-16T11:19:36.630391+0000","last_unstale":"2023-10-16T11:19:37.620865+0000","last_undegraded":"2023-10-16T11:19:37.620865+0000","last_fullsized":"2023-10-16T11:19:37.620865+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:23.825418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616656+0000","last_change":"2023-10-16T11:19:23.552080+0000","last_active":"2023-10-16T11:19:36.616656+0000","last_peered":"2023-10-16T11:19:36.616656+0000","last_clean":"2023-10-16T11:19:36.616656+0000","last_became_active":"2023-10-16T11:19:23.551779+0000","last_became_peered":"2023-10-16T11:19:23.551779+0000","last_unstale":"2023-10-16T11:19:36.616656+0000","last_undegraded":"2023-10-16T11:19:36.616656+0000","last_fullsized":"2023-10-16T11:19:36.616656+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894374+0000","last_change":"2023-10-16T11:19:36.636621+0000","last_active":"2023-10-16T11:19:37.894374+0000","last_peered":"2023-10-16T11:19:37.894374+0000","last_clean":"2023-10-16T11:19:37.894374+0000","last_became_active":"2023-10-16T11:19:36.636222+0000","last_became_peered":"2023-10-16T11:19:36.636222+0000","last_unstale":"2023-10-16T11:19:37.894374+0000","last_undegraded":"2023-10-16T11:19:37.894374+0000","last_fullsized":"2023-10-16T11:19:37.894374+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:51.038856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615178+0000","last_change":"2023-10-16T11:19:23.550227+0000","last_active":"2023-10-16T11:19:36.615178+0000","last_peered":"2023-10-16T11:19:36.615178+0000","last_clean":"2023-10-16T11:19:36.615178+0000","last_became_active":"2023-10-16T11:19:23.549938+0000","last_became_peered":"2023-10-16T11:19:23.549938+0000","last_unstale":"2023-10-16T11:19:36.615178+0000","last_undegraded":"2023-10-16T11:19:36.615178+0000","last_fullsized":"2023-10-16T11:19:36.615178+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:13.268771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893818+0000","last_change":"2023-10-16T11:19:36.636806+0000","last_active":"2023-10-16T11:19:37.893818+0000","last_peered":"2023-10-16T11:19:37.893818+0000","last_clean":"2023-10-16T11:19:37.893818+0000","last_became_active":"2023-10-16T11:19:36.636526+0000","last_became_peered":"2023-10-16T11:19:36.636526+0000","last_unstale":"2023-10-16T11:19:37.893818+0000","last_undegraded":"2023-10-16T11:19:37.893818+0000","last_fullsized":"2023-10-16T11:19:37.893818+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:26.018990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619096+0000","last_change":"2023-10-16T11:19:23.547259+0000","last_active":"2023-10-16T11:19:36.619096+0000","last_peered":"2023-10-16T11:19:36.619096+0000","last_clean":"2023-10-16T11:19:36.619096+0000","last_became_active":"2023-10-16T11:19:23.546957+0000","last_became_peered":"2023-10-16T11:19:23.546957+0000","last_unstale":"2023-10-16T11:19:36.619096+0000","last_undegraded":"2023-10-16T11:19:36.619096+0000","last_fullsized":"2023-10-16T11:19:36.619096+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893676+0000","last_change":"2023-10-16T11:19:36.626949+0000","last_active":"2023-10-16T11:19:37.893676+0000","last_peered":"2023-10-16T11:19:37.893676+0000","last_clean":"2023-10-16T11:19:37.893676+0000","last_became_active":"2023-10-16T11:19:36.626670+0000","last_became_peered":"2023-10-16T11:19:36.626670+0000","last_unstale":"2023-10-16T11:19:37.893676+0000","last_undegraded":"2023-10-16T11:19:37.893676+0000","last_fullsized":"2023-10-16T11:19:37.893676+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:14.675395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615351+0000","last_change":"2023-10-16T11:19:23.550179+0000","last_active":"2023-10-16T11:19:36.615351+0000","last_peered":"2023-10-16T11:19:36.615351+0000","last_clean":"2023-10-16T11:19:36.615351+0000","last_became_active":"2023-10-16T11:19:23.549811+0000","last_became_peered":"2023-10-16T11:19:23.549811+0000","last_unstale":"2023-10-16T11:19:36.615351+0000","last_undegraded":"2023-10-16T11:19:36.615351+0000","last_fullsized":"2023-10-16T11:19:36.615351+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:58.221233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894684+0000","last_change":"2023-10-16T11:19:36.632889+0000","last_active":"2023-10-16T11:19:37.894684+0000","last_peered":"2023-10-16T11:19:37.894684+0000","last_clean":"2023-10-16T11:19:37.894684+0000","last_became_active":"2023-10-16T11:19:36.632268+0000","last_became_peered":"2023-10-16T11:19:36.632268+0000","last_unstale":"2023-10-16T11:19:37.894684+0000","last_undegraded":"2023-10-16T11:19:37.894684+0000","last_fullsized":"2023-10-16T11:19:37.894684+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:11.302811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616511+0000","last_change":"2023-10-16T11:19:23.541856+0000","last_active":"2023-10-16T11:19:36.616511+0000","last_peered":"2023-10-16T11:19:36.616511+0000","last_clean":"2023-10-16T11:19:36.616511+0000","last_became_active":"2023-10-16T11:19:23.541549+0000","last_became_peered":"2023-10-16T11:19:23.541549+0000","last_unstale":"2023-10-16T11:19:36.616511+0000","last_undegraded":"2023-10-16T11:19:36.616511+0000","last_fullsized":"2023-10-16T11:19:36.616511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:07.709989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892542+0000","last_change":"2023-10-16T11:19:36.625337+0000","last_active":"2023-10-16T11:19:37.892542+0000","last_peered":"2023-10-16T11:19:37.892542+0000","last_clean":"2023-10-16T11:19:37.892542+0000","last_became_active":"2023-10-16T11:19:36.624796+0000","last_became_peered":"2023-10-16T11:19:36.624796+0000","last_unstale":"2023-10-16T11:19:37.892542+0000","last_undegraded":"2023-10-16T11:19:37.892542+0000","last_fullsized":"2023-10-16T11:19:37.892542+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:27.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619098+0000","last_change":"2023-10-16T11:19:23.547136+0000","last_active":"2023-10-16T11:19:36.619098+0000","last_peered":"2023-10-16T11:19:36.619098+0000","last_clean":"2023-10-16T11:19:36.619098+0000","last_became_active":"2023-10-16T11:19:23.546834+0000","last_became_peered":"2023-10-16T11:19:23.546834+0000","last_unstale":"2023-10-16T11:19:36.619098+0000","last_undegraded":"2023-10-16T11:19:36.619098+0000","last_fullsized":"2023-10-16T11:19:36.619098+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:45.683847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892567+0000","last_change":"2023-10-16T11:19:36.628989+0000","last_active":"2023-10-16T11:19:37.892567+0000","last_peered":"2023-10-16T11:19:37.892567+0000","last_clean":"2023-10-16T11:19:37.892567+0000","last_became_active":"2023-10-16T11:19:36.628665+0000","last_became_peered":"2023-10-16T11:19:36.628665+0000","last_unstale":"2023-10-16T11:19:37.892567+0000","last_undegraded":"2023-10-16T11:19:37.892567+0000","last_fullsized":"2023-10-16T11:19:37.892567+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:42.796960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629084+0000","last_change":"2023-10-16T11:19:23.552160+0000","last_active":"2023-10-16T11:19:36.629084+0000","last_peered":"2023-10-16T11:19:36.629084+0000","last_clean":"2023-10-16T11:19:36.629084+0000","last_became_active":"2023-10-16T11:19:23.551805+0000","last_became_peered":"2023-10-16T11:19:23.551805+0000","last_unstale":"2023-10-16T11:19:36.629084+0000","last_undegraded":"2023-10-16T11:19:36.629084+0000","last_fullsized":"2023-10-16T11:19:36.629084+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621385+0000","last_change":"2023-10-16T11:19:36.634749+0000","last_active":"2023-10-16T11:19:37.621385+0000","last_peered":"2023-10-16T11:19:37.621385+0000","last_clean":"2023-10-16T11:19:37.621385+0000","last_became_active":"2023-10-16T11:19:36.634467+0000","last_became_peered":"2023-10-16T11:19:36.634467+0000","last_unstale":"2023-10-16T11:19:37.621385+0000","last_undegraded":"2023-10-16T11:19:37.621385+0000","last_fullsized":"2023-10-16T11:19:37.621385+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616111+0000","last_change":"2023-10-16T11:19:23.552427+0000","last_active":"2023-10-16T11:19:36.616111+0000","last_peered":"2023-10-16T11:19:36.616111+0000","last_clean":"2023-10-16T11:19:36.616111+0000","last_became_active":"2023-10-16T11:19:23.552156+0000","last_became_peered":"2023-10-16T11:19:23.552156+0000","last_unstale":"2023-10-16T11:19:36.616111+0000","last_undegraded":"2023-10-16T11:19:36.616111+0000","last_fullsized":"2023-10-16T11:19:36.616111+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893142+0000","last_change":"2023-10-16T11:19:36.633460+0000","last_active":"2023-10-16T11:19:37.893142+0000","last_peered":"2023-10-16T11:19:37.893142+0000","last_clean":"2023-10-16T11:19:37.893142+0000","last_became_active":"2023-10-16T11:19:36.631460+0000","last_became_peered":"2023-10-16T11:19:36.631460+0000","last_unstale":"2023-10-16T11:19:37.893142+0000","last_undegraded":"2023-10-16T11:19:37.893142+0000","last_fullsized":"2023-10-16T11:19:37.893142+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621912+0000","last_change":"2023-10-16T11:19:23.547472+0000","last_active":"2023-10-16T11:19:36.621912+0000","last_peered":"2023-10-16T11:19:36.621912+0000","last_clean":"2023-10-16T11:19:36.621912+0000","last_became_active":"2023-10-16T11:19:23.547148+0000","last_became_peered":"2023-10-16T11:19:23.547148+0000","last_unstale":"2023-10-16T11:19:36.621912+0000","last_undegraded":"2023-10-16T11:19:36.621912+0000","last_fullsized":"2023-10-16T11:19:36.621912+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3657,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543282,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52516,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319324,"statfs":{"total":96636764160,"available":96582987776,"internally_reserved":0,"allocated":3366912,"data_stored":2012088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.27800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42399999999999999,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69599999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.57799999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.79900000000000004,"15min":0.86299999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.307},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.002},"last":0.71199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.443},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33100000000000002},"max":{"1min":8.1950000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.54400000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.504},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.871},"last":0.52800000000000002}]}]},{"osd":6,"up_from":14,"seq":60129543281,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2768896,"data_stored":1419429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.875},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.51200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.56999999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46899999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.44800000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.752,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.48899999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.32100000000000001},"max":{"1min":0.79500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.91100000000000003},"last":0.41799999999999998}]}]},{"osd":1,"up_from":14,"seq":60129543281,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60256,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311584,"statfs":{"total":96636764160,"available":96575062016,"internally_reserved":0,"allocated":2768896,"data_stored":1418937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.224},"max":{"1min":0.60199999999999998,"5min":0.629,"15min":0.68700000000000006},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.434},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.62,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.42799999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.55300000000000005,"5min":0.58699999999999997,"15min":0.98899999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.78300000000000003},"last":0.504},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.0289999999999999,"15min":1.097},"last":0.74199999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":1.147},"last":0.77900000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.88,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.83899999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":0.98899999999999999,"5min":1.004,"15min":1.0600000000000001},"last":0.876},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46300000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.1060000000000001},"last":0.73099999999999998}]}]},{"osd":0,"up_from":14,"seq":60129543282,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60836,"kb_used_data":3284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311004,"statfs":{"total":96636764160,"available":96574468096,"internally_reserved":0,"allocated":3362816,"data_stored":2009921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.46400000000000002,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":0.432},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58799999999999997,"5min":0.63700000000000001,"15min":0.70299999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.314}]},{"osd":3,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.68300000000000005},"last":0.377},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.45000000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.99199999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":1.008},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":1.004,"15min":1.008},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.623}]},{"osd":6,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.504},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53400000000000003}]}]},{"osd":2,"up_from":14,"seq":60129543281,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":2777088,"data_stored":1420459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":0.249},{"interface":"front","average":{"1min":0.436,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.63},"last":0.34100000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":0.311}]},{"osd":3,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.623,"15min":0.65500000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.81200000000000006},"last":0.35599999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":1.052,"15min":1.052},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68700000000000006,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61099999999999999}]}]},{"osd":3,"up_from":14,"seq":60129543282,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47788,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324052,"statfs":{"total":96636764160,"available":96587829248,"internally_reserved":0,"allocated":2785280,"data_stored":1420655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.36799999999999999,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.307}]},{"osd":1,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":0.60699999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36099999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":1.034,"15min":1.0740000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.80200000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.0489999999999999},"last":0.61699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543281,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":2781184,"data_stored":1422336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.628},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.35499999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.627},"min":{"1min":0.438,"5min":0.40400000000000003,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.501},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.879},"last":0.41299999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.69499999999999995,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.89500000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.629}]},{"osd":7,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":0.875,"15min":0.875},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.67000000000000004}]}]},{"osd":5,"up_from":14,"seq":60129543282,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42196,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329644,"statfs":{"total":96636764160,"available":96593555456,"internally_reserved":0,"allocated":2760704,"data_stored":1419373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":1.873,"15min":1.873},"last":0.44400000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.39900000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":1.821,"15min":1.821},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55500000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":1.919,"15min":1.919},"last":0.36599999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.873},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.45400000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":2.02,"15min":2.02},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:19:45.731 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:19:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.863+0000 7fd2a8728700 1 Processor -- start 2023-10-16T11:19:45.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.863+0000 7fd2a8728700 1 -- start start 2023-10-16T11:19:45.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.864+0000 7fd2a8728700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2a015dfd0 0x7fd2a00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.864+0000 7fd2a8728700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a006f870 con 0x7fd2a00cd470 2023-10-16T11:19:45.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.864+0000 7fd2a8728700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a00d1160 con 0x7fd2a00af450 2023-10-16T11:19:45.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.864+0000 7fd2a8728700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a00cc8e0 con 0x7fd2a015dfd0 2023-10-16T11:19:45.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.864+0000 7fd2a6cc5700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd2a00cd470 0x7fd2a006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38760/0 (socket says 172.21.15.181:38760) 2023-10-16T11:19:45.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.864+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3495227048 learned_addr learned my addr 172.21.15.181:0/3495227048 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:45.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a64c4700 1 --2- 172.21.15.181:0/3495227048 >> v2:172.21.15.190:3300/0 conn(0x7fd2a015dfd0 0x7fd2a00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097605686 0 0) 0x7fd2a006f870 con 0x7fd2a00cd470 2023-10-16T11:19:45.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd28c0035b0 con 0x7fd2a00cd470 2023-10-16T11:19:45.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1722850467 0 0) 0x7fd28c0035b0 con 0x7fd2a00cd470 2023-10-16T11:19:45.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 >> v2:172.21.15.190:3300/0 conn(0x7fd2a015dfd0 msgr2=0x7fd2a00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:45.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 --2- 172.21.15.181:0/3495227048 >> v2:172.21.15.190:3300/0 conn(0x7fd2a015dfd0 0x7fd2a00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 >> v1:172.21.15.190:6789/0 conn(0x7fd2a00af450 legacy=0x7fd2a006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a00ccbc0 con 0x7fd2a00cd470 2023-10-16T11:19:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd2980019f0 con 0x7fd2a00cd470 2023-10-16T11:19:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 >> v1:172.21.15.181:6789/0 conn(0x7fd2a00cd470 legacy=0x7fd2a006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28c003a00 0x7fd28c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c004850 0x7fd28c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c005210 0x7fd28c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00cc8e0 con 0x7fd28c005210 2023-10-16T11:19:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00c1b0 con 0x7fd28c004850 2023-10-16T11:19:45.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00c370 con 0x7fd28c003a00 2023-10-16T11:19:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a6cc5700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c005210 0x7fd28c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a5cc3700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c004850 0x7fd28c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a64c4700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28c003a00 0x7fd28c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28c003a00 msgr2=0x7fd28c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a6cc5700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28c003a00 0x7fd28c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c004850 msgr2=0x7fd28c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a6cc5700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c004850 0x7fd28c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3495227048 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0035b0 con 0x7fd28c005210 2023-10-16T11:19:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.865+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3495227048 reap_dead start 2023-10-16T11:19:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a6cc5700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c005210 0x7fd28c00ba10 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fd2980013d0 tx=0x7fd28c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4c4a8f62385f0e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:45.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298012600 con 0x7fd28c005210 2023-10-16T11:19:45.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2980127a0 con 0x7fd28c005210 2023-10-16T11:19:45.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a54c2700 1 -- 172.21.15.181:0/3495227048 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298012b50 con 0x7fd28c005210 2023-10-16T11:19:45.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a8728700 1 -- 172.21.15.181:0/3495227048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c005210 msgr2=0x7fd28c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:45.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c005210 0x7fd28c00ba10 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fd2980013d0 tx=0x7fd28c005880 comp rx=0 tx=0).stop 2023-10-16T11:19:45.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a8728700 1 -- 172.21.15.181:0/3495227048 shutdown_connections 2023-10-16T11:19:45.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3495227048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd28c005210 0x7fd28c00ba10 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:45.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a8728700 1 -- 172.21.15.181:0/3495227048 >> 172.21.15.181:0/3495227048 conn(0x7fd2a00d04a0 msgr2=0x7fd2a00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:45.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a8728700 1 -- 172.21.15.181:0/3495227048 shutdown_connections 2023-10-16T11:19:45.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.866+0000 7fd2a8728700 1 -- 172.21.15.181:0/3495227048 wait complete. 2023-10-16T11:19:45.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.867+0000 7fd2a8728700 1 Processor -- start 2023-10-16T11:19:45.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a8728700 1 -- start start 2023-10-16T11:19:45.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a8728700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c005210 0x7fd2a00c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a8728700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a0160d60 0x7fd2a00c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a8728700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a015e3b0 0x7fd2a015e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a8728700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a01516b0 con 0x7fd2a015e3b0 2023-10-16T11:19:45.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a8728700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00c6340 con 0x7fd28c005210 2023-10-16T11:19:45.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a8728700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00c7570 con 0x7fd2a0160d60 2023-10-16T11:19:45.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a6cc5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a015e3b0 0x7fd2a015e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a6cc5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a015e3b0 0x7fd2a015e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55926/0 (socket says 172.21.15.181:55926) 2023-10-16T11:19:45.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3502141616 learned_addr learned my addr 172.21.15.181:0/3502141616 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:45.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a64c4700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c005210 0x7fd2a00c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.868+0000 7fd2a5cc3700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a0160d60 0x7fd2a00c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a0160d60 msgr2=0x7fd2a00c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:45.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a6cc5700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a0160d60 0x7fd2a00c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:45.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c005210 msgr2=0x7fd2a00c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:45.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a6cc5700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c005210 0x7fd2a00c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:45.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a6cc5700 1 -- 172.21.15.181:0/3502141616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a00c6570 con 0x7fd2a015e3b0 2023-10-16T11:19:45.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a6cc5700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a015e3b0 0x7fd2a015e830 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fd2a006f3a0 tx=0x7fd298001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:45.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd296ffd700 1 -- 172.21.15.181:0/3502141616 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29801c610 con 0x7fd2a015e3b0 2023-10-16T11:19:45.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd296ffd700 1 -- 172.21.15.181:0/3502141616 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29801c770 con 0x7fd2a015e3b0 2023-10-16T11:19:45.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a00c77c0 con 0x7fd2a015e3b0 2023-10-16T11:19:45.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd296ffd700 1 -- 172.21.15.181:0/3502141616 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29801cb20 con 0x7fd2a015e3b0 2023-10-16T11:19:45.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.869+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a00c7d80 con 0x7fd2a015e3b0 2023-10-16T11:19:45.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.870+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a00c3ec0 con 0x7fd2a015e3b0 2023-10-16T11:19:45.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.871+0000 7fd296ffd700 1 -- 172.21.15.181:0/3502141616 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd298019070 con 0x7fd2a015e3b0 2023-10-16T11:19:45.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.871+0000 7fd296ffd700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd28406b020 0x7fd28406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:45.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.871+0000 7fd296ffd700 1 -- 172.21.15.181:0/3502141616 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fd29808b2b0 con 0x7fd2a015e3b0 2023-10-16T11:19:45.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.871+0000 7fd2a64c4700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd28406b020 0x7fd28406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:45.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.871+0000 7fd2a64c4700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd28406b020 0x7fd28406d4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd290009bc0 tx=0x7fd2900091c0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:45.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:45.876+0000 7fd296ffd700 1 -- 172.21.15.181:0/3502141616 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd298055e70 con 0x7fd2a015e3b0 2023-10-16T11:19:46.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.064+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2a00cc590 con 0x7fd28406b020 2023-10-16T11:19:46.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.078+0000 7fd296ffd700 1 -- 172.21.15.181:0/3502141616 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529029 (secure 0 0 0) 0x7fd290009db0 con 0x7fd28406b020 2023-10-16T11:19:46.086 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:46.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd28406b020 msgr2=0x7fd28406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd28406b020 0x7fd28406d4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd290009bc0 tx=0x7fd2900091c0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a015e3b0 msgr2=0x7fd2a015e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a015e3b0 0x7fd2a015e830 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fd2a006f3a0 tx=0x7fd298001360 comp rx=0 tx=0).stop 2023-10-16T11:19:46.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 shutdown_connections 2023-10-16T11:19:46.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fd28406b020 0x7fd28406d4e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2a015e3b0 0x7fd2a015e830 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd28c005210 0x7fd2a00c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 --2- 172.21.15.181:0/3502141616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2a0160d60 0x7fd2a00c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 >> 172.21.15.181:0/3502141616 conn(0x7fd2a00d04a0 msgr2=0x7fd2a00adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:46.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.082+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 shutdown_connections 2023-10-16T11:19:46.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.083+0000 7fd2a8728700 1 -- 172.21.15.181:0/3502141616 wait complete. 2023-10-16T11:19:46.093 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:19:46.097 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-16T11:19:45.495031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596435,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":671,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422172,"kb_used_data":22824,"kb_used_omap":12,"kb_used_meta":399219,"kb_avail":754552548,"statfs":{"total":773094113280,"available":772661809152,"internally_reserved":0,"allocated":23371776,"data_stored":12543198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":408800824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.881514"},"pg_stats":[{"pgid":"1.0","version":"206'348","reported_seq":1213,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619646+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:19:36.619646+0000","last_peered":"2023-10-16T11:19:36.619646+0000","last_clean":"2023-10-16T11:19:36.619646+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:19:36.619646+0000","last_undegraded":"2023-10-16T11:19:36.619646+0000","last_fullsized":"2023-10-16T11:19:36.619646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629864+0000","last_change":"2023-10-16T11:19:22.536415+0000","last_active":"2023-10-16T11:19:36.629864+0000","last_peered":"2023-10-16T11:19:36.629864+0000","last_clean":"2023-10-16T11:19:36.629864+0000","last_became_active":"2023-10-16T11:19:22.536117+0000","last_became_peered":"2023-10-16T11:19:22.536117+0000","last_unstale":"2023-10-16T11:19:36.629864+0000","last_undegraded":"2023-10-16T11:19:36.629864+0000","last_fullsized":"2023-10-16T11:19:36.629864+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:07.879254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620141+0000","last_change":"2023-10-16T11:19:23.551428+0000","last_active":"2023-10-16T11:19:36.620141+0000","last_peered":"2023-10-16T11:19:36.620141+0000","last_clean":"2023-10-16T11:19:36.620141+0000","last_became_active":"2023-10-16T11:19:23.551083+0000","last_became_peered":"2023-10-16T11:19:23.551083+0000","last_unstale":"2023-10-16T11:19:36.620141+0000","last_undegraded":"2023-10-16T11:19:36.620141+0000","last_fullsized":"2023-10-16T11:19:36.620141+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:52.800093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023169+0000","last_change":"2023-10-16T11:19:35.631939+0000","last_active":"2023-10-16T11:19:37.023169+0000","last_peered":"2023-10-16T11:19:37.023169+0000","last_clean":"2023-10-16T11:19:37.023169+0000","last_became_active":"2023-10-16T11:19:35.631645+0000","last_became_peered":"2023-10-16T11:19:35.631645+0000","last_unstale":"2023-10-16T11:19:37.023169+0000","last_undegraded":"2023-10-16T11:19:37.023169+0000","last_fullsized":"2023-10-16T11:19:37.023169+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:56.488955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893598+0000","last_change":"2023-10-16T11:19:36.631354+0000","last_active":"2023-10-16T11:19:37.893598+0000","last_peered":"2023-10-16T11:19:37.893598+0000","last_clean":"2023-10-16T11:19:37.893598+0000","last_became_active":"2023-10-16T11:19:36.631071+0000","last_became_peered":"2023-10-16T11:19:36.631071+0000","last_unstale":"2023-10-16T11:19:37.893598+0000","last_undegraded":"2023-10-16T11:19:37.893598+0000","last_fullsized":"2023-10-16T11:19:37.893598+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:21:44.063667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630084+0000","last_change":"2023-10-16T11:19:22.537241+0000","last_active":"2023-10-16T11:19:36.630084+0000","last_peered":"2023-10-16T11:19:36.630084+0000","last_clean":"2023-10-16T11:19:36.630084+0000","last_became_active":"2023-10-16T11:19:22.536947+0000","last_became_peered":"2023-10-16T11:19:22.536947+0000","last_unstale":"2023-10-16T11:19:36.630084+0000","last_undegraded":"2023-10-16T11:19:36.630084+0000","last_fullsized":"2023-10-16T11:19:36.630084+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618113+0000","last_change":"2023-10-16T11:19:23.563089+0000","last_active":"2023-10-16T11:19:36.618113+0000","last_peered":"2023-10-16T11:19:36.618113+0000","last_clean":"2023-10-16T11:19:36.618113+0000","last_became_active":"2023-10-16T11:19:23.562759+0000","last_became_peered":"2023-10-16T11:19:23.562759+0000","last_unstale":"2023-10-16T11:19:36.618113+0000","last_undegraded":"2023-10-16T11:19:36.618113+0000","last_fullsized":"2023-10-16T11:19:36.618113+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"220'1","reported_seq":13,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893516+0000","last_change":"2023-10-16T11:19:35.628235+0000","last_active":"2023-10-16T11:19:37.893516+0000","last_peered":"2023-10-16T11:19:37.893516+0000","last_clean":"2023-10-16T11:19:37.893516+0000","last_became_active":"2023-10-16T11:19:35.627756+0000","last_became_peered":"2023-10-16T11:19:35.627756+0000","last_unstale":"2023-10-16T11:19:37.893516+0000","last_undegraded":"2023-10-16T11:19:37.893516+0000","last_fullsized":"2023-10-16T11:19:37.893516+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:06.645740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894143+0000","last_change":"2023-10-16T11:19:36.632258+0000","last_active":"2023-10-16T11:19:37.894143+0000","last_peered":"2023-10-16T11:19:37.894143+0000","last_clean":"2023-10-16T11:19:37.894143+0000","last_became_active":"2023-10-16T11:19:36.631963+0000","last_became_peered":"2023-10-16T11:19:36.631963+0000","last_unstale":"2023-10-16T11:19:37.894143+0000","last_undegraded":"2023-10-16T11:19:37.894143+0000","last_fullsized":"2023-10-16T11:19:37.894143+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624408+0000","last_change":"2023-10-16T11:19:22.539000+0000","last_active":"2023-10-16T11:19:36.624408+0000","last_peered":"2023-10-16T11:19:36.624408+0000","last_clean":"2023-10-16T11:19:36.624408+0000","last_became_active":"2023-10-16T11:19:22.538728+0000","last_became_peered":"2023-10-16T11:19:22.538728+0000","last_unstale":"2023-10-16T11:19:36.624408+0000","last_undegraded":"2023-10-16T11:19:36.624408+0000","last_fullsized":"2023-10-16T11:19:36.624408+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:59:03.944394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629081+0000","last_change":"2023-10-16T11:19:23.563504+0000","last_active":"2023-10-16T11:19:36.629081+0000","last_peered":"2023-10-16T11:19:36.629081+0000","last_clean":"2023-10-16T11:19:36.629081+0000","last_became_active":"2023-10-16T11:19:23.563243+0000","last_became_peered":"2023-10-16T11:19:23.563243+0000","last_unstale":"2023-10-16T11:19:36.629081+0000","last_undegraded":"2023-10-16T11:19:36.629081+0000","last_fullsized":"2023-10-16T11:19:36.629081+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026018+0000","last_change":"2023-10-16T11:19:35.632100+0000","last_active":"2023-10-16T11:19:37.026018+0000","last_peered":"2023-10-16T11:19:37.026018+0000","last_clean":"2023-10-16T11:19:37.026018+0000","last_became_active":"2023-10-16T11:19:35.631782+0000","last_became_peered":"2023-10-16T11:19:35.631782+0000","last_unstale":"2023-10-16T11:19:37.026018+0000","last_undegraded":"2023-10-16T11:19:37.026018+0000","last_fullsized":"2023-10-16T11:19:37.026018+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:19.485660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892620+0000","last_change":"2023-10-16T11:19:36.630004+0000","last_active":"2023-10-16T11:19:37.892620+0000","last_peered":"2023-10-16T11:19:37.892620+0000","last_clean":"2023-10-16T11:19:37.892620+0000","last_became_active":"2023-10-16T11:19:36.629689+0000","last_became_peered":"2023-10-16T11:19:36.629689+0000","last_unstale":"2023-10-16T11:19:37.892620+0000","last_undegraded":"2023-10-16T11:19:37.892620+0000","last_fullsized":"2023-10-16T11:19:37.892620+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:13.752266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615026+0000","last_change":"2023-10-16T11:19:22.544852+0000","last_active":"2023-10-16T11:19:36.615026+0000","last_peered":"2023-10-16T11:19:36.615026+0000","last_clean":"2023-10-16T11:19:36.615026+0000","last_became_active":"2023-10-16T11:19:22.544565+0000","last_became_peered":"2023-10-16T11:19:22.544565+0000","last_unstale":"2023-10-16T11:19:36.615026+0000","last_undegraded":"2023-10-16T11:19:36.615026+0000","last_fullsized":"2023-10-16T11:19:36.615026+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:07.144566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630082+0000","last_change":"2023-10-16T11:19:23.558505+0000","last_active":"2023-10-16T11:19:36.630082+0000","last_peered":"2023-10-16T11:19:36.630082+0000","last_clean":"2023-10-16T11:19:36.630082+0000","last_became_active":"2023-10-16T11:19:23.558188+0000","last_became_peered":"2023-10-16T11:19:23.558188+0000","last_unstale":"2023-10-16T11:19:36.630082+0000","last_undegraded":"2023-10-16T11:19:36.630082+0000","last_fullsized":"2023-10-16T11:19:36.630082+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630289+0000","last_change":"2023-10-16T11:19:35.630984+0000","last_active":"2023-10-16T11:19:36.630289+0000","last_peered":"2023-10-16T11:19:36.630289+0000","last_clean":"2023-10-16T11:19:36.630289+0000","last_became_active":"2023-10-16T11:19:35.630524+0000","last_became_peered":"2023-10-16T11:19:35.630524+0000","last_unstale":"2023-10-16T11:19:36.630289+0000","last_undegraded":"2023-10-16T11:19:36.630289+0000","last_fullsized":"2023-10-16T11:19:36.630289+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893641+0000","last_change":"2023-10-16T11:19:36.624656+0000","last_active":"2023-10-16T11:19:37.893641+0000","last_peered":"2023-10-16T11:19:37.893641+0000","last_clean":"2023-10-16T11:19:37.893641+0000","last_became_active":"2023-10-16T11:19:36.623815+0000","last_became_peered":"2023-10-16T11:19:36.623815+0000","last_unstale":"2023-10-16T11:19:37.893641+0000","last_undegraded":"2023-10-16T11:19:37.893641+0000","last_fullsized":"2023-10-16T11:19:37.893641+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:59:16.765372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616791+0000","last_change":"2023-10-16T11:19:22.543881+0000","last_active":"2023-10-16T11:19:36.616791+0000","last_peered":"2023-10-16T11:19:36.616791+0000","last_clean":"2023-10-16T11:19:36.616791+0000","last_became_active":"2023-10-16T11:19:22.543647+0000","last_became_peered":"2023-10-16T11:19:22.543647+0000","last_unstale":"2023-10-16T11:19:36.616791+0000","last_undegraded":"2023-10-16T11:19:36.616791+0000","last_fullsized":"2023-10-16T11:19:36.616791+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:10:41.755970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620300+0000","last_change":"2023-10-16T11:19:23.546380+0000","last_active":"2023-10-16T11:19:36.620300+0000","last_peered":"2023-10-16T11:19:36.620300+0000","last_clean":"2023-10-16T11:19:36.620300+0000","last_became_active":"2023-10-16T11:19:23.546098+0000","last_became_peered":"2023-10-16T11:19:23.546098+0000","last_unstale":"2023-10-16T11:19:36.620300+0000","last_undegraded":"2023-10-16T11:19:36.620300+0000","last_fullsized":"2023-10-16T11:19:36.620300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:06.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619997+0000","last_change":"2023-10-16T11:19:35.628388+0000","last_active":"2023-10-16T11:19:36.619997+0000","last_peered":"2023-10-16T11:19:36.619997+0000","last_clean":"2023-10-16T11:19:36.619997+0000","last_became_active":"2023-10-16T11:19:35.627946+0000","last_became_peered":"2023-10-16T11:19:35.627946+0000","last_unstale":"2023-10-16T11:19:36.619997+0000","last_undegraded":"2023-10-16T11:19:36.619997+0000","last_fullsized":"2023-10-16T11:19:36.619997+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:25.890159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894002+0000","last_change":"2023-10-16T11:19:36.639241+0000","last_active":"2023-10-16T11:19:37.894002+0000","last_peered":"2023-10-16T11:19:37.894002+0000","last_clean":"2023-10-16T11:19:37.894002+0000","last_became_active":"2023-10-16T11:19:36.637259+0000","last_became_peered":"2023-10-16T11:19:36.637259+0000","last_unstale":"2023-10-16T11:19:37.894002+0000","last_undegraded":"2023-10-16T11:19:37.894002+0000","last_fullsized":"2023-10-16T11:19:37.894002+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:25.400158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629125+0000","last_change":"2023-10-16T11:19:22.537543+0000","last_active":"2023-10-16T11:19:36.629125+0000","last_peered":"2023-10-16T11:19:36.629125+0000","last_clean":"2023-10-16T11:19:36.629125+0000","last_became_active":"2023-10-16T11:19:22.537050+0000","last_became_peered":"2023-10-16T11:19:22.537050+0000","last_unstale":"2023-10-16T11:19:36.629125+0000","last_undegraded":"2023-10-16T11:19:36.629125+0000","last_fullsized":"2023-10-16T11:19:36.629125+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:03.218818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624306+0000","last_change":"2023-10-16T11:19:23.546472+0000","last_active":"2023-10-16T11:19:36.624306+0000","last_peered":"2023-10-16T11:19:36.624306+0000","last_clean":"2023-10-16T11:19:36.624306+0000","last_became_active":"2023-10-16T11:19:23.546160+0000","last_became_peered":"2023-10-16T11:19:23.546160+0000","last_unstale":"2023-10-16T11:19:36.624306+0000","last_undegraded":"2023-10-16T11:19:36.624306+0000","last_fullsized":"2023-10-16T11:19:36.624306+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:47:06.395224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620047+0000","last_change":"2023-10-16T11:19:35.624590+0000","last_active":"2023-10-16T11:19:36.620047+0000","last_peered":"2023-10-16T11:19:36.620047+0000","last_clean":"2023-10-16T11:19:36.620047+0000","last_became_active":"2023-10-16T11:19:35.623979+0000","last_became_peered":"2023-10-16T11:19:35.623979+0000","last_unstale":"2023-10-16T11:19:36.620047+0000","last_undegraded":"2023-10-16T11:19:36.620047+0000","last_fullsized":"2023-10-16T11:19:36.620047+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:57.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894860+0000","last_change":"2023-10-16T11:19:36.631096+0000","last_active":"2023-10-16T11:19:37.894860+0000","last_peered":"2023-10-16T11:19:37.894860+0000","last_clean":"2023-10-16T11:19:37.894860+0000","last_became_active":"2023-10-16T11:19:36.630386+0000","last_became_peered":"2023-10-16T11:19:36.630386+0000","last_unstale":"2023-10-16T11:19:37.894860+0000","last_undegraded":"2023-10-16T11:19:37.894860+0000","last_fullsized":"2023-10-16T11:19:37.894860+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:25:03.603833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616893+0000","last_change":"2023-10-16T11:19:22.543844+0000","last_active":"2023-10-16T11:19:36.616893+0000","last_peered":"2023-10-16T11:19:36.616893+0000","last_clean":"2023-10-16T11:19:36.616893+0000","last_became_active":"2023-10-16T11:19:22.543538+0000","last_became_peered":"2023-10-16T11:19:22.543538+0000","last_unstale":"2023-10-16T11:19:36.616893+0000","last_undegraded":"2023-10-16T11:19:36.616893+0000","last_fullsized":"2023-10-16T11:19:36.616893+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:00.946603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617382+0000","last_change":"2023-10-16T11:19:23.556539+0000","last_active":"2023-10-16T11:19:36.617382+0000","last_peered":"2023-10-16T11:19:36.617382+0000","last_clean":"2023-10-16T11:19:36.617382+0000","last_became_active":"2023-10-16T11:19:23.556209+0000","last_became_peered":"2023-10-16T11:19:23.556209+0000","last_unstale":"2023-10-16T11:19:36.617382+0000","last_undegraded":"2023-10-16T11:19:36.617382+0000","last_fullsized":"2023-10-16T11:19:36.617382+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618436+0000","last_change":"2023-10-16T11:19:35.620983+0000","last_active":"2023-10-16T11:19:36.618436+0000","last_peered":"2023-10-16T11:19:36.618436+0000","last_clean":"2023-10-16T11:19:36.618436+0000","last_became_active":"2023-10-16T11:19:35.620681+0000","last_became_peered":"2023-10-16T11:19:35.620681+0000","last_unstale":"2023-10-16T11:19:36.618436+0000","last_undegraded":"2023-10-16T11:19:36.618436+0000","last_fullsized":"2023-10-16T11:19:36.618436+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:24.394439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893282+0000","last_change":"2023-10-16T11:19:36.628083+0000","last_active":"2023-10-16T11:19:37.893282+0000","last_peered":"2023-10-16T11:19:37.893282+0000","last_clean":"2023-10-16T11:19:37.893282+0000","last_became_active":"2023-10-16T11:19:36.627768+0000","last_became_peered":"2023-10-16T11:19:36.627768+0000","last_unstale":"2023-10-16T11:19:37.893282+0000","last_undegraded":"2023-10-16T11:19:37.893282+0000","last_fullsized":"2023-10-16T11:19:37.893282+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:59.846608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619763+0000","last_change":"2023-10-16T11:19:22.537758+0000","last_active":"2023-10-16T11:19:36.619763+0000","last_peered":"2023-10-16T11:19:36.619763+0000","last_clean":"2023-10-16T11:19:36.619763+0000","last_became_active":"2023-10-16T11:19:22.537201+0000","last_became_peered":"2023-10-16T11:19:22.537201+0000","last_unstale":"2023-10-16T11:19:36.619763+0000","last_undegraded":"2023-10-16T11:19:36.619763+0000","last_fullsized":"2023-10-16T11:19:36.619763+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:26.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622052+0000","last_change":"2023-10-16T11:19:23.564018+0000","last_active":"2023-10-16T11:19:36.622052+0000","last_peered":"2023-10-16T11:19:36.622052+0000","last_clean":"2023-10-16T11:19:36.622052+0000","last_became_active":"2023-10-16T11:19:23.563738+0000","last_became_peered":"2023-10-16T11:19:23.563738+0000","last_unstale":"2023-10-16T11:19:36.622052+0000","last_undegraded":"2023-10-16T11:19:36.622052+0000","last_fullsized":"2023-10-16T11:19:36.622052+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630020+0000","last_change":"2023-10-16T11:19:35.630702+0000","last_active":"2023-10-16T11:19:36.630020+0000","last_peered":"2023-10-16T11:19:36.630020+0000","last_clean":"2023-10-16T11:19:36.630020+0000","last_became_active":"2023-10-16T11:19:35.630133+0000","last_became_peered":"2023-10-16T11:19:35.630133+0000","last_unstale":"2023-10-16T11:19:36.630020+0000","last_undegraded":"2023-10-16T11:19:36.630020+0000","last_fullsized":"2023-10-16T11:19:36.630020+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:50:57.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892972+0000","last_change":"2023-10-16T11:19:36.630883+0000","last_active":"2023-10-16T11:19:37.892972+0000","last_peered":"2023-10-16T11:19:37.892972+0000","last_clean":"2023-10-16T11:19:37.892972+0000","last_became_active":"2023-10-16T11:19:36.627590+0000","last_became_peered":"2023-10-16T11:19:36.627590+0000","last_unstale":"2023-10-16T11:19:37.892972+0000","last_undegraded":"2023-10-16T11:19:37.892972+0000","last_fullsized":"2023-10-16T11:19:37.892972+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:16.051165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631253+0000","last_change":"2023-10-16T11:19:22.537256+0000","last_active":"2023-10-16T11:19:36.631253+0000","last_peered":"2023-10-16T11:19:36.631253+0000","last_clean":"2023-10-16T11:19:36.631253+0000","last_became_active":"2023-10-16T11:19:22.536996+0000","last_became_peered":"2023-10-16T11:19:22.536996+0000","last_unstale":"2023-10-16T11:19:36.631253+0000","last_undegraded":"2023-10-16T11:19:36.631253+0000","last_fullsized":"2023-10-16T11:19:36.631253+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:22.462616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631014+0000","last_change":"2023-10-16T11:19:23.561790+0000","last_active":"2023-10-16T11:19:36.631014+0000","last_peered":"2023-10-16T11:19:36.631014+0000","last_clean":"2023-10-16T11:19:36.631014+0000","last_became_active":"2023-10-16T11:19:23.561531+0000","last_became_peered":"2023-10-16T11:19:23.561531+0000","last_unstale":"2023-10-16T11:19:36.631014+0000","last_undegraded":"2023-10-16T11:19:36.631014+0000","last_fullsized":"2023-10-16T11:19:36.631014+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616296+0000","last_change":"2023-10-16T11:19:35.632427+0000","last_active":"2023-10-16T11:19:36.616296+0000","last_peered":"2023-10-16T11:19:36.616296+0000","last_clean":"2023-10-16T11:19:36.616296+0000","last_became_active":"2023-10-16T11:19:35.632164+0000","last_became_peered":"2023-10-16T11:19:35.632164+0000","last_unstale":"2023-10-16T11:19:36.616296+0000","last_undegraded":"2023-10-16T11:19:36.616296+0000","last_fullsized":"2023-10-16T11:19:36.616296+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:40.048278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893018+0000","last_change":"2023-10-16T11:19:36.633276+0000","last_active":"2023-10-16T11:19:37.893018+0000","last_peered":"2023-10-16T11:19:37.893018+0000","last_clean":"2023-10-16T11:19:37.893018+0000","last_became_active":"2023-10-16T11:19:36.631425+0000","last_became_peered":"2023-10-16T11:19:36.631425+0000","last_unstale":"2023-10-16T11:19:37.893018+0000","last_undegraded":"2023-10-16T11:19:37.893018+0000","last_fullsized":"2023-10-16T11:19:37.893018+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615074+0000","last_change":"2023-10-16T11:19:22.547829+0000","last_active":"2023-10-16T11:19:36.615074+0000","last_peered":"2023-10-16T11:19:36.615074+0000","last_clean":"2023-10-16T11:19:36.615074+0000","last_became_active":"2023-10-16T11:19:22.547563+0000","last_became_peered":"2023-10-16T11:19:22.547563+0000","last_unstale":"2023-10-16T11:19:36.615074+0000","last_undegraded":"2023-10-16T11:19:36.615074+0000","last_fullsized":"2023-10-16T11:19:36.615074+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:15.819032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624614+0000","last_change":"2023-10-16T11:19:23.563941+0000","last_active":"2023-10-16T11:19:36.624614+0000","last_peered":"2023-10-16T11:19:36.624614+0000","last_clean":"2023-10-16T11:19:36.624614+0000","last_became_active":"2023-10-16T11:19:23.563666+0000","last_became_peered":"2023-10-16T11:19:23.563666+0000","last_unstale":"2023-10-16T11:19:36.624614+0000","last_undegraded":"2023-10-16T11:19:36.624614+0000","last_fullsized":"2023-10-16T11:19:36.624614+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:55:43.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.021295+0000","last_change":"2023-10-16T11:19:35.628428+0000","last_active":"2023-10-16T11:19:37.021295+0000","last_peered":"2023-10-16T11:19:37.021295+0000","last_clean":"2023-10-16T11:19:37.021295+0000","last_became_active":"2023-10-16T11:19:35.628146+0000","last_became_peered":"2023-10-16T11:19:35.628146+0000","last_unstale":"2023-10-16T11:19:37.021295+0000","last_undegraded":"2023-10-16T11:19:37.021295+0000","last_fullsized":"2023-10-16T11:19:37.021295+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894166+0000","last_change":"2023-10-16T11:19:36.634088+0000","last_active":"2023-10-16T11:19:37.894166+0000","last_peered":"2023-10-16T11:19:37.894166+0000","last_clean":"2023-10-16T11:19:37.894166+0000","last_became_active":"2023-10-16T11:19:36.633725+0000","last_became_peered":"2023-10-16T11:19:36.633725+0000","last_unstale":"2023-10-16T11:19:37.894166+0000","last_undegraded":"2023-10-16T11:19:37.894166+0000","last_fullsized":"2023-10-16T11:19:37.894166+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:49:49.669608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619400+0000","last_change":"2023-10-16T11:19:22.532222+0000","last_active":"2023-10-16T11:19:36.619400+0000","last_peered":"2023-10-16T11:19:36.619400+0000","last_clean":"2023-10-16T11:19:36.619400+0000","last_became_active":"2023-10-16T11:19:22.531939+0000","last_became_peered":"2023-10-16T11:19:22.531939+0000","last_unstale":"2023-10-16T11:19:36.619400+0000","last_undegraded":"2023-10-16T11:19:36.619400+0000","last_fullsized":"2023-10-16T11:19:36.619400+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:26.313767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615636+0000","last_change":"2023-10-16T11:19:23.548561+0000","last_active":"2023-10-16T11:19:36.615636+0000","last_peered":"2023-10-16T11:19:36.615636+0000","last_clean":"2023-10-16T11:19:36.615636+0000","last_became_active":"2023-10-16T11:19:23.548310+0000","last_became_peered":"2023-10-16T11:19:23.548310+0000","last_unstale":"2023-10-16T11:19:36.615636+0000","last_undegraded":"2023-10-16T11:19:36.615636+0000","last_fullsized":"2023-10-16T11:19:36.615636+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.024902+0000","last_change":"2023-10-16T11:19:35.630294+0000","last_active":"2023-10-16T11:19:37.024902+0000","last_peered":"2023-10-16T11:19:37.024902+0000","last_clean":"2023-10-16T11:19:37.024902+0000","last_became_active":"2023-10-16T11:19:35.629895+0000","last_became_peered":"2023-10-16T11:19:35.629895+0000","last_unstale":"2023-10-16T11:19:37.024902+0000","last_undegraded":"2023-10-16T11:19:37.024902+0000","last_fullsized":"2023-10-16T11:19:37.024902+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622628+0000","last_change":"2023-10-16T11:19:36.632583+0000","last_active":"2023-10-16T11:19:37.622628+0000","last_peered":"2023-10-16T11:19:37.622628+0000","last_clean":"2023-10-16T11:19:37.622628+0000","last_became_active":"2023-10-16T11:19:36.632316+0000","last_became_peered":"2023-10-16T11:19:36.632316+0000","last_unstale":"2023-10-16T11:19:37.622628+0000","last_undegraded":"2023-10-16T11:19:37.622628+0000","last_fullsized":"2023-10-16T11:19:37.622628+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619856+0000","last_change":"2023-10-16T11:19:22.542021+0000","last_active":"2023-10-16T11:19:36.619856+0000","last_peered":"2023-10-16T11:19:36.619856+0000","last_clean":"2023-10-16T11:19:36.619856+0000","last_became_active":"2023-10-16T11:19:22.541718+0000","last_became_peered":"2023-10-16T11:19:22.541718+0000","last_unstale":"2023-10-16T11:19:36.619856+0000","last_undegraded":"2023-10-16T11:19:36.619856+0000","last_fullsized":"2023-10-16T11:19:36.619856+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:17:12.867200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628983+0000","last_change":"2023-10-16T11:19:23.554949+0000","last_active":"2023-10-16T11:19:36.628983+0000","last_peered":"2023-10-16T11:19:36.628983+0000","last_clean":"2023-10-16T11:19:36.628983+0000","last_became_active":"2023-10-16T11:19:23.554300+0000","last_became_peered":"2023-10-16T11:19:23.554300+0000","last_unstale":"2023-10-16T11:19:36.628983+0000","last_undegraded":"2023-10-16T11:19:36.628983+0000","last_fullsized":"2023-10-16T11:19:36.628983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623336+0000","last_change":"2023-10-16T11:19:35.627888+0000","last_active":"2023-10-16T11:19:36.623336+0000","last_peered":"2023-10-16T11:19:36.623336+0000","last_clean":"2023-10-16T11:19:36.623336+0000","last_became_active":"2023-10-16T11:19:35.627453+0000","last_became_peered":"2023-10-16T11:19:35.627453+0000","last_unstale":"2023-10-16T11:19:36.623336+0000","last_undegraded":"2023-10-16T11:19:36.623336+0000","last_fullsized":"2023-10-16T11:19:36.623336+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:16:45.616063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894273+0000","last_change":"2023-10-16T11:19:36.638745+0000","last_active":"2023-10-16T11:19:37.894273+0000","last_peered":"2023-10-16T11:19:37.894273+0000","last_clean":"2023-10-16T11:19:37.894273+0000","last_became_active":"2023-10-16T11:19:36.634962+0000","last_became_peered":"2023-10-16T11:19:36.634962+0000","last_unstale":"2023-10-16T11:19:37.894273+0000","last_undegraded":"2023-10-16T11:19:37.894273+0000","last_fullsized":"2023-10-16T11:19:37.894273+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:19.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621691+0000","last_change":"2023-10-16T11:19:22.536403+0000","last_active":"2023-10-16T11:19:36.621691+0000","last_peered":"2023-10-16T11:19:36.621691+0000","last_clean":"2023-10-16T11:19:36.621691+0000","last_became_active":"2023-10-16T11:19:22.536132+0000","last_became_peered":"2023-10-16T11:19:22.536132+0000","last_unstale":"2023-10-16T11:19:36.621691+0000","last_undegraded":"2023-10-16T11:19:36.621691+0000","last_fullsized":"2023-10-16T11:19:36.621691+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:53.563487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621852+0000","last_change":"2023-10-16T11:19:23.543334+0000","last_active":"2023-10-16T11:19:36.621852+0000","last_peered":"2023-10-16T11:19:36.621852+0000","last_clean":"2023-10-16T11:19:36.621852+0000","last_became_active":"2023-10-16T11:19:23.542527+0000","last_became_peered":"2023-10-16T11:19:23.542527+0000","last_unstale":"2023-10-16T11:19:36.621852+0000","last_undegraded":"2023-10-16T11:19:36.621852+0000","last_fullsized":"2023-10-16T11:19:36.621852+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:59.083108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.022227+0000","last_change":"2023-10-16T11:19:35.631031+0000","last_active":"2023-10-16T11:19:37.022227+0000","last_peered":"2023-10-16T11:19:37.022227+0000","last_clean":"2023-10-16T11:19:37.022227+0000","last_became_active":"2023-10-16T11:19:35.630599+0000","last_became_peered":"2023-10-16T11:19:35.630599+0000","last_unstale":"2023-10-16T11:19:37.022227+0000","last_undegraded":"2023-10-16T11:19:37.022227+0000","last_fullsized":"2023-10-16T11:19:37.022227+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:13:50.394449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894077+0000","last_change":"2023-10-16T11:19:36.635702+0000","last_active":"2023-10-16T11:19:37.894077+0000","last_peered":"2023-10-16T11:19:37.894077+0000","last_clean":"2023-10-16T11:19:37.894077+0000","last_became_active":"2023-10-16T11:19:36.635443+0000","last_became_peered":"2023-10-16T11:19:36.635443+0000","last_unstale":"2023-10-16T11:19:37.894077+0000","last_undegraded":"2023-10-16T11:19:37.894077+0000","last_fullsized":"2023-10-16T11:19:37.894077+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:37.781294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628927+0000","last_change":"2023-10-16T11:19:22.545030+0000","last_active":"2023-10-16T11:19:36.628927+0000","last_peered":"2023-10-16T11:19:36.628927+0000","last_clean":"2023-10-16T11:19:36.628927+0000","last_became_active":"2023-10-16T11:19:22.544773+0000","last_became_peered":"2023-10-16T11:19:22.544773+0000","last_unstale":"2023-10-16T11:19:36.628927+0000","last_undegraded":"2023-10-16T11:19:36.628927+0000","last_fullsized":"2023-10-16T11:19:36.628927+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:22.946353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623654+0000","last_change":"2023-10-16T11:19:23.545760+0000","last_active":"2023-10-16T11:19:36.623654+0000","last_peered":"2023-10-16T11:19:36.623654+0000","last_clean":"2023-10-16T11:19:36.623654+0000","last_became_active":"2023-10-16T11:19:23.545499+0000","last_became_peered":"2023-10-16T11:19:23.545499+0000","last_unstale":"2023-10-16T11:19:36.623654+0000","last_undegraded":"2023-10-16T11:19:36.623654+0000","last_fullsized":"2023-10-16T11:19:36.623654+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:32.701590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616714+0000","last_change":"2023-10-16T11:19:35.621290+0000","last_active":"2023-10-16T11:19:36.616714+0000","last_peered":"2023-10-16T11:19:36.616714+0000","last_clean":"2023-10-16T11:19:36.616714+0000","last_became_active":"2023-10-16T11:19:35.620989+0000","last_became_peered":"2023-10-16T11:19:35.620989+0000","last_unstale":"2023-10-16T11:19:36.616714+0000","last_undegraded":"2023-10-16T11:19:36.616714+0000","last_fullsized":"2023-10-16T11:19:36.616714+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:22.819700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.895061+0000","last_change":"2023-10-16T11:19:36.639230+0000","last_active":"2023-10-16T11:19:37.895061+0000","last_peered":"2023-10-16T11:19:37.895061+0000","last_clean":"2023-10-16T11:19:37.895061+0000","last_became_active":"2023-10-16T11:19:36.638912+0000","last_became_peered":"2023-10-16T11:19:36.638912+0000","last_unstale":"2023-10-16T11:19:37.895061+0000","last_undegraded":"2023-10-16T11:19:37.895061+0000","last_fullsized":"2023-10-16T11:19:37.895061+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:20.791568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623856+0000","last_change":"2023-10-16T11:19:22.533601+0000","last_active":"2023-10-16T11:19:36.623856+0000","last_peered":"2023-10-16T11:19:36.623856+0000","last_clean":"2023-10-16T11:19:36.623856+0000","last_became_active":"2023-10-16T11:19:22.533374+0000","last_became_peered":"2023-10-16T11:19:22.533374+0000","last_unstale":"2023-10-16T11:19:36.623856+0000","last_undegraded":"2023-10-16T11:19:36.623856+0000","last_fullsized":"2023-10-16T11:19:36.623856+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:09:52.564361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629027+0000","last_change":"2023-10-16T11:19:23.553745+0000","last_active":"2023-10-16T11:19:36.629027+0000","last_peered":"2023-10-16T11:19:36.629027+0000","last_clean":"2023-10-16T11:19:36.629027+0000","last_became_active":"2023-10-16T11:19:23.553462+0000","last_became_peered":"2023-10-16T11:19:23.553462+0000","last_unstale":"2023-10-16T11:19:36.629027+0000","last_undegraded":"2023-10-16T11:19:36.629027+0000","last_fullsized":"2023-10-16T11:19:36.629027+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023539+0000","last_change":"2023-10-16T11:19:35.632251+0000","last_active":"2023-10-16T11:19:37.023539+0000","last_peered":"2023-10-16T11:19:37.023539+0000","last_clean":"2023-10-16T11:19:37.023539+0000","last_became_active":"2023-10-16T11:19:35.631302+0000","last_became_peered":"2023-10-16T11:19:35.631302+0000","last_unstale":"2023-10-16T11:19:37.023539+0000","last_undegraded":"2023-10-16T11:19:37.023539+0000","last_fullsized":"2023-10-16T11:19:37.023539+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893096+0000","last_change":"2023-10-16T11:19:36.627034+0000","last_active":"2023-10-16T11:19:37.893096+0000","last_peered":"2023-10-16T11:19:37.893096+0000","last_clean":"2023-10-16T11:19:37.893096+0000","last_became_active":"2023-10-16T11:19:36.626768+0000","last_became_peered":"2023-10-16T11:19:36.626768+0000","last_unstale":"2023-10-16T11:19:37.893096+0000","last_undegraded":"2023-10-16T11:19:37.893096+0000","last_fullsized":"2023-10-16T11:19:37.893096+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:49.454701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616295+0000","last_change":"2023-10-16T11:19:22.539558+0000","last_active":"2023-10-16T11:19:36.616295+0000","last_peered":"2023-10-16T11:19:36.616295+0000","last_clean":"2023-10-16T11:19:36.616295+0000","last_became_active":"2023-10-16T11:19:22.539284+0000","last_became_peered":"2023-10-16T11:19:22.539284+0000","last_unstale":"2023-10-16T11:19:36.616295+0000","last_undegraded":"2023-10-16T11:19:36.616295+0000","last_fullsized":"2023-10-16T11:19:36.616295+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616463+0000","last_change":"2023-10-16T11:19:23.546218+0000","last_active":"2023-10-16T11:19:36.616463+0000","last_peered":"2023-10-16T11:19:36.616463+0000","last_clean":"2023-10-16T11:19:36.616463+0000","last_became_active":"2023-10-16T11:19:23.545896+0000","last_became_peered":"2023-10-16T11:19:23.545896+0000","last_unstale":"2023-10-16T11:19:36.616463+0000","last_undegraded":"2023-10-16T11:19:36.616463+0000","last_fullsized":"2023-10-16T11:19:36.616463+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:32.007534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629694+0000","last_change":"2023-10-16T11:19:35.631734+0000","last_active":"2023-10-16T11:19:36.629694+0000","last_peered":"2023-10-16T11:19:36.629694+0000","last_clean":"2023-10-16T11:19:36.629694+0000","last_became_active":"2023-10-16T11:19:35.631398+0000","last_became_peered":"2023-10-16T11:19:35.631398+0000","last_unstale":"2023-10-16T11:19:36.629694+0000","last_undegraded":"2023-10-16T11:19:36.629694+0000","last_fullsized":"2023-10-16T11:19:36.629694+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:32.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893267+0000","last_change":"2023-10-16T11:19:36.627229+0000","last_active":"2023-10-16T11:19:37.893267+0000","last_peered":"2023-10-16T11:19:37.893267+0000","last_clean":"2023-10-16T11:19:37.893267+0000","last_became_active":"2023-10-16T11:19:36.626587+0000","last_became_peered":"2023-10-16T11:19:36.626587+0000","last_unstale":"2023-10-16T11:19:37.893267+0000","last_undegraded":"2023-10-16T11:19:37.893267+0000","last_fullsized":"2023-10-16T11:19:37.893267+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:25:34.645872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615964+0000","last_change":"2023-10-16T11:19:22.545027+0000","last_active":"2023-10-16T11:19:36.615964+0000","last_peered":"2023-10-16T11:19:36.615964+0000","last_clean":"2023-10-16T11:19:36.615964+0000","last_became_active":"2023-10-16T11:19:22.544756+0000","last_became_peered":"2023-10-16T11:19:22.544756+0000","last_unstale":"2023-10-16T11:19:36.615964+0000","last_undegraded":"2023-10-16T11:19:36.615964+0000","last_fullsized":"2023-10-16T11:19:36.615964+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:23.693322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616002+0000","last_change":"2023-10-16T11:19:23.547530+0000","last_active":"2023-10-16T11:19:36.616002+0000","last_peered":"2023-10-16T11:19:36.616002+0000","last_clean":"2023-10-16T11:19:36.616002+0000","last_became_active":"2023-10-16T11:19:23.547255+0000","last_became_peered":"2023-10-16T11:19:23.547255+0000","last_unstale":"2023-10-16T11:19:36.616002+0000","last_undegraded":"2023-10-16T11:19:36.616002+0000","last_fullsized":"2023-10-16T11:19:36.616002+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616101+0000","last_change":"2023-10-16T11:19:35.632530+0000","last_active":"2023-10-16T11:19:36.616101+0000","last_peered":"2023-10-16T11:19:36.616101+0000","last_clean":"2023-10-16T11:19:36.616101+0000","last_became_active":"2023-10-16T11:19:35.632226+0000","last_became_peered":"2023-10-16T11:19:35.632226+0000","last_unstale":"2023-10-16T11:19:36.616101+0000","last_undegraded":"2023-10-16T11:19:36.616101+0000","last_fullsized":"2023-10-16T11:19:36.616101+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.620864+0000","last_change":"2023-10-16T11:19:36.632635+0000","last_active":"2023-10-16T11:19:37.620864+0000","last_peered":"2023-10-16T11:19:37.620864+0000","last_clean":"2023-10-16T11:19:37.620864+0000","last_became_active":"2023-10-16T11:19:36.632351+0000","last_became_peered":"2023-10-16T11:19:36.632351+0000","last_unstale":"2023-10-16T11:19:37.620864+0000","last_undegraded":"2023-10-16T11:19:37.620864+0000","last_fullsized":"2023-10-16T11:19:37.620864+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:23.825418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":24,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629784+0000","last_change":"2023-10-16T11:19:22.534093+0000","last_active":"2023-10-16T11:19:36.629784+0000","last_peered":"2023-10-16T11:19:36.629784+0000","last_clean":"2023-10-16T11:19:36.629784+0000","last_became_active":"2023-10-16T11:19:22.533637+0000","last_became_peered":"2023-10-16T11:19:22.533637+0000","last_unstale":"2023-10-16T11:19:36.629784+0000","last_undegraded":"2023-10-16T11:19:36.629784+0000","last_fullsized":"2023-10-16T11:19:36.629784+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619314+0000","last_change":"2023-10-16T11:19:23.552950+0000","last_active":"2023-10-16T11:19:36.619314+0000","last_peered":"2023-10-16T11:19:36.619314+0000","last_clean":"2023-10-16T11:19:36.619314+0000","last_became_active":"2023-10-16T11:19:23.552686+0000","last_became_peered":"2023-10-16T11:19:23.552686+0000","last_unstale":"2023-10-16T11:19:36.619314+0000","last_undegraded":"2023-10-16T11:19:36.619314+0000","last_fullsized":"2023-10-16T11:19:36.619314+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:33:23.469010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622198+0000","last_change":"2023-10-16T11:19:35.623459+0000","last_active":"2023-10-16T11:19:36.622198+0000","last_peered":"2023-10-16T11:19:36.622198+0000","last_clean":"2023-10-16T11:19:36.622198+0000","last_became_active":"2023-10-16T11:19:35.623199+0000","last_became_peered":"2023-10-16T11:19:35.623199+0000","last_unstale":"2023-10-16T11:19:36.622198+0000","last_undegraded":"2023-10-16T11:19:36.622198+0000","last_fullsized":"2023-10-16T11:19:36.622198+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894201+0000","last_change":"2023-10-16T11:19:36.637364+0000","last_active":"2023-10-16T11:19:37.894201+0000","last_peered":"2023-10-16T11:19:37.894201+0000","last_clean":"2023-10-16T11:19:37.894201+0000","last_became_active":"2023-10-16T11:19:36.637057+0000","last_became_peered":"2023-10-16T11:19:36.637057+0000","last_unstale":"2023-10-16T11:19:37.894201+0000","last_undegraded":"2023-10-16T11:19:37.894201+0000","last_fullsized":"2023-10-16T11:19:37.894201+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:03.151059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621449+0000","last_change":"2023-10-16T11:19:22.538008+0000","last_active":"2023-10-16T11:19:36.621449+0000","last_peered":"2023-10-16T11:19:36.621449+0000","last_clean":"2023-10-16T11:19:36.621449+0000","last_became_active":"2023-10-16T11:19:22.537752+0000","last_became_peered":"2023-10-16T11:19:22.537752+0000","last_unstale":"2023-10-16T11:19:36.621449+0000","last_undegraded":"2023-10-16T11:19:36.621449+0000","last_fullsized":"2023-10-16T11:19:36.621449+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:20.859898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628978+0000","last_change":"2023-10-16T11:19:23.563603+0000","last_active":"2023-10-16T11:19:36.628978+0000","last_peered":"2023-10-16T11:19:36.628978+0000","last_clean":"2023-10-16T11:19:36.628978+0000","last_became_active":"2023-10-16T11:19:23.563305+0000","last_became_peered":"2023-10-16T11:19:23.563305+0000","last_unstale":"2023-10-16T11:19:36.628978+0000","last_undegraded":"2023-10-16T11:19:36.628978+0000","last_fullsized":"2023-10-16T11:19:36.628978+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620238+0000","last_change":"2023-10-16T11:19:35.631312+0000","last_active":"2023-10-16T11:19:36.620238+0000","last_peered":"2023-10-16T11:19:36.620238+0000","last_clean":"2023-10-16T11:19:36.620238+0000","last_became_active":"2023-10-16T11:19:35.631045+0000","last_became_peered":"2023-10-16T11:19:35.631045+0000","last_unstale":"2023-10-16T11:19:36.620238+0000","last_undegraded":"2023-10-16T11:19:36.620238+0000","last_fullsized":"2023-10-16T11:19:36.620238+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:41.921297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894234+0000","last_change":"2023-10-16T11:19:36.636938+0000","last_active":"2023-10-16T11:19:37.894234+0000","last_peered":"2023-10-16T11:19:37.894234+0000","last_clean":"2023-10-16T11:19:37.894234+0000","last_became_active":"2023-10-16T11:19:36.636625+0000","last_became_peered":"2023-10-16T11:19:36.636625+0000","last_unstale":"2023-10-16T11:19:37.894234+0000","last_undegraded":"2023-10-16T11:19:37.894234+0000","last_fullsized":"2023-10-16T11:19:37.894234+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:23.128272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624395+0000","last_change":"2023-10-16T11:19:22.538147+0000","last_active":"2023-10-16T11:19:36.624395+0000","last_peered":"2023-10-16T11:19:36.624395+0000","last_clean":"2023-10-16T11:19:36.624395+0000","last_became_active":"2023-10-16T11:19:22.537850+0000","last_became_peered":"2023-10-16T11:19:22.537850+0000","last_unstale":"2023-10-16T11:19:36.624395+0000","last_undegraded":"2023-10-16T11:19:36.624395+0000","last_fullsized":"2023-10-16T11:19:36.624395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:41:44.794069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621587+0000","last_change":"2023-10-16T11:19:23.552779+0000","last_active":"2023-10-16T11:19:36.621587+0000","last_peered":"2023-10-16T11:19:36.621587+0000","last_clean":"2023-10-16T11:19:36.621587+0000","last_became_active":"2023-10-16T11:19:23.552491+0000","last_became_peered":"2023-10-16T11:19:23.552491+0000","last_unstale":"2023-10-16T11:19:36.621587+0000","last_undegraded":"2023-10-16T11:19:36.621587+0000","last_fullsized":"2023-10-16T11:19:36.621587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:02.969587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"219'6","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.027458+0000","last_change":"2023-10-16T11:19:35.629807+0000","last_active":"2023-10-16T11:19:37.027458+0000","last_peered":"2023-10-16T11:19:37.027458+0000","last_clean":"2023-10-16T11:19:37.027458+0000","last_became_active":"2023-10-16T11:19:35.629137+0000","last_became_peered":"2023-10-16T11:19:35.629137+0000","last_unstale":"2023-10-16T11:19:37.027458+0000","last_undegraded":"2023-10-16T11:19:37.027458+0000","last_fullsized":"2023-10-16T11:19:37.027458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:06.206094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894306+0000","last_change":"2023-10-16T11:19:36.632014+0000","last_active":"2023-10-16T11:19:37.894306+0000","last_peered":"2023-10-16T11:19:37.894306+0000","last_clean":"2023-10-16T11:19:37.894306+0000","last_became_active":"2023-10-16T11:19:36.631735+0000","last_became_peered":"2023-10-16T11:19:36.631735+0000","last_unstale":"2023-10-16T11:19:37.894306+0000","last_undegraded":"2023-10-16T11:19:37.894306+0000","last_fullsized":"2023-10-16T11:19:37.894306+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:25.064507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624578+0000","last_change":"2023-10-16T11:19:22.544032+0000","last_active":"2023-10-16T11:19:36.624578+0000","last_peered":"2023-10-16T11:19:36.624578+0000","last_clean":"2023-10-16T11:19:36.624578+0000","last_became_active":"2023-10-16T11:19:22.543713+0000","last_became_peered":"2023-10-16T11:19:22.543713+0000","last_unstale":"2023-10-16T11:19:36.624578+0000","last_undegraded":"2023-10-16T11:19:36.624578+0000","last_fullsized":"2023-10-16T11:19:36.624578+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:14:01.363714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615557+0000","last_change":"2023-10-16T11:19:23.550398+0000","last_active":"2023-10-16T11:19:36.615557+0000","last_peered":"2023-10-16T11:19:36.615557+0000","last_clean":"2023-10-16T11:19:36.615557+0000","last_became_active":"2023-10-16T11:19:23.545907+0000","last_became_peered":"2023-10-16T11:19:23.545907+0000","last_unstale":"2023-10-16T11:19:36.615557+0000","last_undegraded":"2023-10-16T11:19:36.615557+0000","last_fullsized":"2023-10-16T11:19:36.615557+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:05.456349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"219'3","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026764+0000","last_change":"2023-10-16T11:19:35.627232+0000","last_active":"2023-10-16T11:19:37.026764+0000","last_peered":"2023-10-16T11:19:37.026764+0000","last_clean":"2023-10-16T11:19:37.026764+0000","last_became_active":"2023-10-16T11:19:35.626952+0000","last_became_peered":"2023-10-16T11:19:35.626952+0000","last_unstale":"2023-10-16T11:19:37.026764+0000","last_undegraded":"2023-10-16T11:19:37.026764+0000","last_fullsized":"2023-10-16T11:19:37.026764+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:32:39.610324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621525+0000","last_change":"2023-10-16T11:19:36.636057+0000","last_active":"2023-10-16T11:19:37.621525+0000","last_peered":"2023-10-16T11:19:37.621525+0000","last_clean":"2023-10-16T11:19:37.621525+0000","last_became_active":"2023-10-16T11:19:36.635687+0000","last_became_peered":"2023-10-16T11:19:36.635687+0000","last_unstale":"2023-10-16T11:19:37.621525+0000","last_undegraded":"2023-10-16T11:19:37.621525+0000","last_fullsized":"2023-10-16T11:19:37.621525+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:23:22.666485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.14","version":"216'8","reported_seq":24,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617300+0000","last_change":"2023-10-16T11:19:22.532339+0000","last_active":"2023-10-16T11:19:36.617300+0000","last_peered":"2023-10-16T11:19:36.617300+0000","last_clean":"2023-10-16T11:19:36.617300+0000","last_became_active":"2023-10-16T11:19:22.532053+0000","last_became_peered":"2023-10-16T11:19:22.532053+0000","last_unstale":"2023-10-16T11:19:36.617300+0000","last_undegraded":"2023-10-16T11:19:36.617300+0000","last_fullsized":"2023-10-16T11:19:36.617300+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:25:21.190168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2859,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621913+0000","last_change":"2023-10-16T11:19:23.548103+0000","last_active":"2023-10-16T11:19:36.621913+0000","last_peered":"2023-10-16T11:19:36.621913+0000","last_clean":"2023-10-16T11:19:36.621913+0000","last_became_active":"2023-10-16T11:19:23.547665+0000","last_became_peered":"2023-10-16T11:19:23.547665+0000","last_unstale":"2023-10-16T11:19:36.621913+0000","last_undegraded":"2023-10-16T11:19:36.621913+0000","last_fullsized":"2023-10-16T11:19:36.621913+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619766+0000","last_change":"2023-10-16T11:19:35.624749+0000","last_active":"2023-10-16T11:19:36.619766+0000","last_peered":"2023-10-16T11:19:36.619766+0000","last_clean":"2023-10-16T11:19:36.619766+0000","last_became_active":"2023-10-16T11:19:35.624338+0000","last_became_peered":"2023-10-16T11:19:35.624338+0000","last_unstale":"2023-10-16T11:19:36.619766+0000","last_undegraded":"2023-10-16T11:19:36.619766+0000","last_fullsized":"2023-10-16T11:19:36.619766+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:32.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894693+0000","last_change":"2023-10-16T11:19:36.631186+0000","last_active":"2023-10-16T11:19:37.894693+0000","last_peered":"2023-10-16T11:19:37.894693+0000","last_clean":"2023-10-16T11:19:37.894693+0000","last_became_active":"2023-10-16T11:19:36.630761+0000","last_became_peered":"2023-10-16T11:19:36.630761+0000","last_unstale":"2023-10-16T11:19:37.894693+0000","last_undegraded":"2023-10-16T11:19:37.894693+0000","last_fullsized":"2023-10-16T11:19:37.894693+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:18.537927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623757+0000","last_change":"2023-10-16T11:19:22.544432+0000","last_active":"2023-10-16T11:19:36.623757+0000","last_peered":"2023-10-16T11:19:36.623757+0000","last_clean":"2023-10-16T11:19:36.623757+0000","last_became_active":"2023-10-16T11:19:22.543046+0000","last_became_peered":"2023-10-16T11:19:22.543046+0000","last_unstale":"2023-10-16T11:19:36.623757+0000","last_undegraded":"2023-10-16T11:19:36.623757+0000","last_fullsized":"2023-10-16T11:19:36.623757+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:51:08.672505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617875+0000","last_change":"2023-10-16T11:19:23.563660+0000","last_active":"2023-10-16T11:19:36.617875+0000","last_peered":"2023-10-16T11:19:36.617875+0000","last_clean":"2023-10-16T11:19:36.617875+0000","last_became_active":"2023-10-16T11:19:23.563402+0000","last_became_peered":"2023-10-16T11:19:23.563402+0000","last_unstale":"2023-10-16T11:19:36.617875+0000","last_undegraded":"2023-10-16T11:19:36.617875+0000","last_fullsized":"2023-10-16T11:19:36.617875+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:41.606189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026157+0000","last_change":"2023-10-16T11:19:35.633267+0000","last_active":"2023-10-16T11:19:37.026157+0000","last_peered":"2023-10-16T11:19:37.026157+0000","last_clean":"2023-10-16T11:19:37.026157+0000","last_became_active":"2023-10-16T11:19:35.632961+0000","last_became_peered":"2023-10-16T11:19:35.632961+0000","last_unstale":"2023-10-16T11:19:37.026157+0000","last_undegraded":"2023-10-16T11:19:37.026157+0000","last_fullsized":"2023-10-16T11:19:37.026157+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:39.586059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621385+0000","last_change":"2023-10-16T11:19:36.628797+0000","last_active":"2023-10-16T11:19:37.621385+0000","last_peered":"2023-10-16T11:19:37.621385+0000","last_clean":"2023-10-16T11:19:37.621385+0000","last_became_active":"2023-10-16T11:19:36.628522+0000","last_became_peered":"2023-10-16T11:19:36.628522+0000","last_unstale":"2023-10-16T11:19:37.621385+0000","last_undegraded":"2023-10-16T11:19:37.621385+0000","last_fullsized":"2023-10-16T11:19:37.621385+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.626248+0000","last_change":"2023-10-16T11:19:22.535771+0000","last_active":"2023-10-16T11:19:36.626248+0000","last_peered":"2023-10-16T11:19:36.626248+0000","last_clean":"2023-10-16T11:19:36.626248+0000","last_became_active":"2023-10-16T11:19:22.535486+0000","last_became_peered":"2023-10-16T11:19:22.535486+0000","last_unstale":"2023-10-16T11:19:36.626248+0000","last_undegraded":"2023-10-16T11:19:36.626248+0000","last_fullsized":"2023-10-16T11:19:36.626248+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:25.214100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631131+0000","last_change":"2023-10-16T11:19:23.560046+0000","last_active":"2023-10-16T11:19:36.631131+0000","last_peered":"2023-10-16T11:19:36.631131+0000","last_clean":"2023-10-16T11:19:36.631131+0000","last_became_active":"2023-10-16T11:19:23.559718+0000","last_became_peered":"2023-10-16T11:19:23.559718+0000","last_unstale":"2023-10-16T11:19:36.631131+0000","last_undegraded":"2023-10-16T11:19:36.631131+0000","last_fullsized":"2023-10-16T11:19:36.631131+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:22.701369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"219'8","reported_seq":20,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.026701+0000","last_change":"2023-10-16T11:19:35.625566+0000","last_active":"2023-10-16T11:19:37.026701+0000","last_peered":"2023-10-16T11:19:37.026701+0000","last_clean":"2023-10-16T11:19:37.026701+0000","last_became_active":"2023-10-16T11:19:35.624392+0000","last_became_peered":"2023-10-16T11:19:35.624392+0000","last_unstale":"2023-10-16T11:19:37.026701+0000","last_undegraded":"2023-10-16T11:19:37.026701+0000","last_fullsized":"2023-10-16T11:19:37.026701+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:33:14.543389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621461+0000","last_change":"2023-10-16T11:19:36.626287+0000","last_active":"2023-10-16T11:19:37.621461+0000","last_peered":"2023-10-16T11:19:37.621461+0000","last_clean":"2023-10-16T11:19:37.621461+0000","last_became_active":"2023-10-16T11:19:36.625588+0000","last_became_peered":"2023-10-16T11:19:36.625588+0000","last_unstale":"2023-10-16T11:19:37.621461+0000","last_undegraded":"2023-10-16T11:19:37.621461+0000","last_fullsized":"2023-10-16T11:19:37.621461+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:44:37.781294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615845+0000","last_change":"2023-10-16T11:19:22.545557+0000","last_active":"2023-10-16T11:19:36.615845+0000","last_peered":"2023-10-16T11:19:36.615845+0000","last_clean":"2023-10-16T11:19:36.615845+0000","last_became_active":"2023-10-16T11:19:22.545253+0000","last_became_peered":"2023-10-16T11:19:22.545253+0000","last_unstale":"2023-10-16T11:19:36.615845+0000","last_undegraded":"2023-10-16T11:19:36.615845+0000","last_fullsized":"2023-10-16T11:19:36.615845+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:13:11.312186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619711+0000","last_change":"2023-10-16T11:19:23.553070+0000","last_active":"2023-10-16T11:19:36.619711+0000","last_peered":"2023-10-16T11:19:36.619711+0000","last_clean":"2023-10-16T11:19:36.619711+0000","last_became_active":"2023-10-16T11:19:23.552761+0000","last_became_peered":"2023-10-16T11:19:23.552761+0000","last_unstale":"2023-10-16T11:19:36.619711+0000","last_undegraded":"2023-10-16T11:19:36.619711+0000","last_fullsized":"2023-10-16T11:19:36.619711+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:36.373122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616224+0000","last_change":"2023-10-16T11:19:35.623394+0000","last_active":"2023-10-16T11:19:36.616224+0000","last_peered":"2023-10-16T11:19:36.616224+0000","last_clean":"2023-10-16T11:19:36.616224+0000","last_became_active":"2023-10-16T11:19:35.623119+0000","last_became_peered":"2023-10-16T11:19:35.623119+0000","last_unstale":"2023-10-16T11:19:36.616224+0000","last_undegraded":"2023-10-16T11:19:36.616224+0000","last_fullsized":"2023-10-16T11:19:36.616224+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:07.710682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892867+0000","last_change":"2023-10-16T11:19:36.633676+0000","last_active":"2023-10-16T11:19:37.892867+0000","last_peered":"2023-10-16T11:19:37.892867+0000","last_clean":"2023-10-16T11:19:37.892867+0000","last_became_active":"2023-10-16T11:19:36.633464+0000","last_became_peered":"2023-10-16T11:19:36.633464+0000","last_unstale":"2023-10-16T11:19:37.892867+0000","last_undegraded":"2023-10-16T11:19:37.892867+0000","last_fullsized":"2023-10-16T11:19:37.892867+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624527+0000","last_change":"2023-10-16T11:19:22.534080+0000","last_active":"2023-10-16T11:19:36.624527+0000","last_peered":"2023-10-16T11:19:36.624527+0000","last_clean":"2023-10-16T11:19:36.624527+0000","last_became_active":"2023-10-16T11:19:22.533693+0000","last_became_peered":"2023-10-16T11:19:22.533693+0000","last_unstale":"2023-10-16T11:19:36.624527+0000","last_undegraded":"2023-10-16T11:19:36.624527+0000","last_fullsized":"2023-10-16T11:19:36.624527+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:57.023698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617545+0000","last_change":"2023-10-16T11:19:23.546966+0000","last_active":"2023-10-16T11:19:36.617545+0000","last_peered":"2023-10-16T11:19:36.617545+0000","last_clean":"2023-10-16T11:19:36.617545+0000","last_became_active":"2023-10-16T11:19:23.546685+0000","last_became_peered":"2023-10-16T11:19:23.546685+0000","last_unstale":"2023-10-16T11:19:36.617545+0000","last_undegraded":"2023-10-16T11:19:36.617545+0000","last_fullsized":"2023-10-16T11:19:36.617545+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:13.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.025002+0000","last_change":"2023-10-16T11:19:35.624998+0000","last_active":"2023-10-16T11:19:37.025002+0000","last_peered":"2023-10-16T11:19:37.025002+0000","last_clean":"2023-10-16T11:19:37.025002+0000","last_became_active":"2023-10-16T11:19:35.624685+0000","last_became_peered":"2023-10-16T11:19:35.624685+0000","last_unstale":"2023-10-16T11:19:37.025002+0000","last_undegraded":"2023-10-16T11:19:37.025002+0000","last_fullsized":"2023-10-16T11:19:37.025002+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:52:30.461865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622678+0000","last_change":"2023-10-16T11:19:36.638230+0000","last_active":"2023-10-16T11:19:37.622678+0000","last_peered":"2023-10-16T11:19:37.622678+0000","last_clean":"2023-10-16T11:19:37.622678+0000","last_became_active":"2023-10-16T11:19:36.637937+0000","last_became_peered":"2023-10-16T11:19:36.637937+0000","last_unstale":"2023-10-16T11:19:37.622678+0000","last_undegraded":"2023-10-16T11:19:37.622678+0000","last_fullsized":"2023-10-16T11:19:37.622678+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:21.232537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616327+0000","last_change":"2023-10-16T11:19:22.547192+0000","last_active":"2023-10-16T11:19:36.616327+0000","last_peered":"2023-10-16T11:19:36.616327+0000","last_clean":"2023-10-16T11:19:36.616327+0000","last_became_active":"2023-10-16T11:19:22.546912+0000","last_became_peered":"2023-10-16T11:19:22.546912+0000","last_unstale":"2023-10-16T11:19:36.616327+0000","last_undegraded":"2023-10-16T11:19:36.616327+0000","last_fullsized":"2023-10-16T11:19:36.616327+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:17.017260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.614895+0000","last_change":"2023-10-16T11:19:23.543708+0000","last_active":"2023-10-16T11:19:36.614895+0000","last_peered":"2023-10-16T11:19:36.614895+0000","last_clean":"2023-10-16T11:19:36.614895+0000","last_became_active":"2023-10-16T11:19:23.543449+0000","last_became_peered":"2023-10-16T11:19:23.543449+0000","last_unstale":"2023-10-16T11:19:36.614895+0000","last_undegraded":"2023-10-16T11:19:36.614895+0000","last_fullsized":"2023-10-16T11:19:36.614895+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:06.406149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"219'4","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023703+0000","last_change":"2023-10-16T11:19:35.626621+0000","last_active":"2023-10-16T11:19:37.023703+0000","last_peered":"2023-10-16T11:19:37.023703+0000","last_clean":"2023-10-16T11:19:37.023703+0000","last_became_active":"2023-10-16T11:19:35.626338+0000","last_became_peered":"2023-10-16T11:19:35.626338+0000","last_unstale":"2023-10-16T11:19:37.023703+0000","last_undegraded":"2023-10-16T11:19:37.023703+0000","last_fullsized":"2023-10-16T11:19:37.023703+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:43:43.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894242+0000","last_change":"2023-10-16T11:19:36.638939+0000","last_active":"2023-10-16T11:19:37.894242+0000","last_peered":"2023-10-16T11:19:37.894242+0000","last_clean":"2023-10-16T11:19:37.894242+0000","last_became_active":"2023-10-16T11:19:36.637185+0000","last_became_peered":"2023-10-16T11:19:36.637185+0000","last_unstale":"2023-10-16T11:19:37.894242+0000","last_undegraded":"2023-10-16T11:19:37.894242+0000","last_fullsized":"2023-10-16T11:19:37.894242+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:26.350349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615222+0000","last_change":"2023-10-16T11:19:22.543956+0000","last_active":"2023-10-16T11:19:36.615222+0000","last_peered":"2023-10-16T11:19:36.615222+0000","last_clean":"2023-10-16T11:19:36.615222+0000","last_became_active":"2023-10-16T11:19:22.543633+0000","last_became_peered":"2023-10-16T11:19:22.543633+0000","last_unstale":"2023-10-16T11:19:36.615222+0000","last_undegraded":"2023-10-16T11:19:36.615222+0000","last_fullsized":"2023-10-16T11:19:36.615222+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:12:53.412047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617060+0000","last_change":"2023-10-16T11:19:23.546200+0000","last_active":"2023-10-16T11:19:36.617060+0000","last_peered":"2023-10-16T11:19:36.617060+0000","last_clean":"2023-10-16T11:19:36.617060+0000","last_became_active":"2023-10-16T11:19:23.545867+0000","last_became_peered":"2023-10-16T11:19:23.545867+0000","last_unstale":"2023-10-16T11:19:36.617060+0000","last_undegraded":"2023-10-16T11:19:36.617060+0000","last_fullsized":"2023-10-16T11:19:36.617060+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:14:00.670688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"219'2","reported_seq":14,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023629+0000","last_change":"2023-10-16T11:19:35.628631+0000","last_active":"2023-10-16T11:19:37.023629+0000","last_peered":"2023-10-16T11:19:37.023629+0000","last_clean":"2023-10-16T11:19:37.023629+0000","last_became_active":"2023-10-16T11:19:35.628341+0000","last_became_peered":"2023-10-16T11:19:35.628341+0000","last_unstale":"2023-10-16T11:19:37.023629+0000","last_undegraded":"2023-10-16T11:19:37.023629+0000","last_fullsized":"2023-10-16T11:19:37.023629+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:48:39.437826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621875+0000","last_change":"2023-10-16T11:19:36.635937+0000","last_active":"2023-10-16T11:19:37.621875+0000","last_peered":"2023-10-16T11:19:37.621875+0000","last_clean":"2023-10-16T11:19:37.621875+0000","last_became_active":"2023-10-16T11:19:36.635570+0000","last_became_peered":"2023-10-16T11:19:36.635570+0000","last_unstale":"2023-10-16T11:19:37.621875+0000","last_undegraded":"2023-10-16T11:19:37.621875+0000","last_fullsized":"2023-10-16T11:19:37.621875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:19.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617428+0000","last_change":"2023-10-16T11:19:22.535919+0000","last_active":"2023-10-16T11:19:36.617428+0000","last_peered":"2023-10-16T11:19:36.617428+0000","last_clean":"2023-10-16T11:19:36.617428+0000","last_became_active":"2023-10-16T11:19:22.535612+0000","last_became_peered":"2023-10-16T11:19:22.535612+0000","last_unstale":"2023-10-16T11:19:36.617428+0000","last_undegraded":"2023-10-16T11:19:36.617428+0000","last_fullsized":"2023-10-16T11:19:36.617428+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:54.628429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623571+0000","last_change":"2023-10-16T11:19:23.552923+0000","last_active":"2023-10-16T11:19:36.623571+0000","last_peered":"2023-10-16T11:19:36.623571+0000","last_clean":"2023-10-16T11:19:36.623571+0000","last_became_active":"2023-10-16T11:19:23.552685+0000","last_became_peered":"2023-10-16T11:19:23.552685+0000","last_unstale":"2023-10-16T11:19:36.623571+0000","last_undegraded":"2023-10-16T11:19:36.623571+0000","last_fullsized":"2023-10-16T11:19:36.623571+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:07:13.492062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.023727+0000","last_change":"2023-10-16T11:19:35.630995+0000","last_active":"2023-10-16T11:19:37.023727+0000","last_peered":"2023-10-16T11:19:37.023727+0000","last_clean":"2023-10-16T11:19:37.023727+0000","last_became_active":"2023-10-16T11:19:35.630561+0000","last_became_peered":"2023-10-16T11:19:35.630561+0000","last_unstale":"2023-10-16T11:19:37.023727+0000","last_undegraded":"2023-10-16T11:19:37.023727+0000","last_fullsized":"2023-10-16T11:19:37.023727+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:30.099513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893198+0000","last_change":"2023-10-16T11:19:36.631163+0000","last_active":"2023-10-16T11:19:37.893198+0000","last_peered":"2023-10-16T11:19:37.893198+0000","last_clean":"2023-10-16T11:19:37.893198+0000","last_became_active":"2023-10-16T11:19:36.630820+0000","last_became_peered":"2023-10-16T11:19:36.630820+0000","last_unstale":"2023-10-16T11:19:37.893198+0000","last_undegraded":"2023-10-16T11:19:37.893198+0000","last_fullsized":"2023-10-16T11:19:37.893198+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:58:54.687768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.628981+0000","last_change":"2023-10-16T11:19:22.533919+0000","last_active":"2023-10-16T11:19:36.628981+0000","last_peered":"2023-10-16T11:19:36.628981+0000","last_clean":"2023-10-16T11:19:36.628981+0000","last_became_active":"2023-10-16T11:19:22.533695+0000","last_became_peered":"2023-10-16T11:19:22.533695+0000","last_unstale":"2023-10-16T11:19:36.628981+0000","last_undegraded":"2023-10-16T11:19:36.628981+0000","last_fullsized":"2023-10-16T11:19:36.628981+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:50:51.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618694+0000","last_change":"2023-10-16T11:19:23.546849+0000","last_active":"2023-10-16T11:19:36.618694+0000","last_peered":"2023-10-16T11:19:36.618694+0000","last_clean":"2023-10-16T11:19:36.618694+0000","last_became_active":"2023-10-16T11:19:23.546575+0000","last_became_peered":"2023-10-16T11:19:23.546575+0000","last_unstale":"2023-10-16T11:19:36.618694+0000","last_undegraded":"2023-10-16T11:19:36.618694+0000","last_fullsized":"2023-10-16T11:19:36.618694+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.884361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"219'1","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.024943+0000","last_change":"2023-10-16T11:19:35.624948+0000","last_active":"2023-10-16T11:19:37.024943+0000","last_peered":"2023-10-16T11:19:37.024943+0000","last_clean":"2023-10-16T11:19:37.024943+0000","last_became_active":"2023-10-16T11:19:35.624339+0000","last_became_peered":"2023-10-16T11:19:35.624339+0000","last_unstale":"2023-10-16T11:19:37.024943+0000","last_undegraded":"2023-10-16T11:19:37.024943+0000","last_fullsized":"2023-10-16T11:19:37.024943+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:28.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893006+0000","last_change":"2023-10-16T11:19:36.635048+0000","last_active":"2023-10-16T11:19:37.893006+0000","last_peered":"2023-10-16T11:19:37.893006+0000","last_clean":"2023-10-16T11:19:37.893006+0000","last_became_active":"2023-10-16T11:19:36.634362+0000","last_became_peered":"2023-10-16T11:19:36.634362+0000","last_unstale":"2023-10-16T11:19:37.893006+0000","last_undegraded":"2023-10-16T11:19:37.893006+0000","last_fullsized":"2023-10-16T11:19:37.893006+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:52.979732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615890+0000","last_change":"2023-10-16T11:19:22.536812+0000","last_active":"2023-10-16T11:19:36.615890+0000","last_peered":"2023-10-16T11:19:36.615890+0000","last_clean":"2023-10-16T11:19:36.615890+0000","last_became_active":"2023-10-16T11:19:22.536545+0000","last_became_peered":"2023-10-16T11:19:22.536545+0000","last_unstale":"2023-10-16T11:19:36.615890+0000","last_undegraded":"2023-10-16T11:19:36.615890+0000","last_fullsized":"2023-10-16T11:19:36.615890+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624796+0000","last_change":"2023-10-16T11:19:23.559791+0000","last_active":"2023-10-16T11:19:36.624796+0000","last_peered":"2023-10-16T11:19:36.624796+0000","last_clean":"2023-10-16T11:19:36.624796+0000","last_became_active":"2023-10-16T11:19:23.559513+0000","last_became_peered":"2023-10-16T11:19:23.559513+0000","last_unstale":"2023-10-16T11:19:36.624796+0000","last_undegraded":"2023-10-16T11:19:36.624796+0000","last_fullsized":"2023-10-16T11:19:36.624796+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:52:39.986537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621822+0000","last_change":"2023-10-16T11:19:35.622388+0000","last_active":"2023-10-16T11:19:36.621822+0000","last_peered":"2023-10-16T11:19:36.621822+0000","last_clean":"2023-10-16T11:19:36.621822+0000","last_became_active":"2023-10-16T11:19:35.622128+0000","last_became_peered":"2023-10-16T11:19:35.622128+0000","last_unstale":"2023-10-16T11:19:36.621822+0000","last_undegraded":"2023-10-16T11:19:36.621822+0000","last_fullsized":"2023-10-16T11:19:36.621822+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:10.516621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892728+0000","last_change":"2023-10-16T11:19:36.636570+0000","last_active":"2023-10-16T11:19:37.892728+0000","last_peered":"2023-10-16T11:19:37.892728+0000","last_clean":"2023-10-16T11:19:37.892728+0000","last_became_active":"2023-10-16T11:19:36.636314+0000","last_became_peered":"2023-10-16T11:19:36.636314+0000","last_unstale":"2023-10-16T11:19:37.892728+0000","last_undegraded":"2023-10-16T11:19:37.892728+0000","last_fullsized":"2023-10-16T11:19:37.892728+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:19.250783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":20,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.631343+0000","last_change":"2023-10-16T11:19:22.538829+0000","last_active":"2023-10-16T11:19:36.631343+0000","last_peered":"2023-10-16T11:19:36.631343+0000","last_clean":"2023-10-16T11:19:36.631343+0000","last_became_active":"2023-10-16T11:19:22.538537+0000","last_became_peered":"2023-10-16T11:19:22.538537+0000","last_unstale":"2023-10-16T11:19:36.631343+0000","last_undegraded":"2023-10-16T11:19:36.631343+0000","last_fullsized":"2023-10-16T11:19:36.631343+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:21.512405+0000","last_clean_scrub_stamp":"2023-10-16T11:19:21.512405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:16:09.892801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615802+0000","last_change":"2023-10-16T11:19:23.551775+0000","last_active":"2023-10-16T11:19:36.615802+0000","last_peered":"2023-10-16T11:19:36.615802+0000","last_clean":"2023-10-16T11:19:36.615802+0000","last_became_active":"2023-10-16T11:19:23.551490+0000","last_became_peered":"2023-10-16T11:19:23.551490+0000","last_unstale":"2023-10-16T11:19:36.615802+0000","last_undegraded":"2023-10-16T11:19:36.615802+0000","last_fullsized":"2023-10-16T11:19:36.615802+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:57.581193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618085+0000","last_change":"2023-10-16T11:19:35.625342+0000","last_active":"2023-10-16T11:19:36.618085+0000","last_peered":"2023-10-16T11:19:36.618085+0000","last_clean":"2023-10-16T11:19:36.618085+0000","last_became_active":"2023-10-16T11:19:35.624933+0000","last_became_peered":"2023-10-16T11:19:35.624933+0000","last_unstale":"2023-10-16T11:19:36.618085+0000","last_undegraded":"2023-10-16T11:19:36.618085+0000","last_fullsized":"2023-10-16T11:19:36.618085+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:34.594658+0000","last_clean_scrub_stamp":"2023-10-16T11:19:34.594658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:20.226818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892577+0000","last_change":"2023-10-16T11:19:36.630442+0000","last_active":"2023-10-16T11:19:37.892577+0000","last_peered":"2023-10-16T11:19:37.892577+0000","last_clean":"2023-10-16T11:19:37.892577+0000","last_became_active":"2023-10-16T11:19:36.630151+0000","last_became_peered":"2023-10-16T11:19:36.630151+0000","last_unstale":"2023-10-16T11:19:37.892577+0000","last_undegraded":"2023-10-16T11:19:37.892577+0000","last_fullsized":"2023-10-16T11:19:37.892577+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:33.781861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622379+0000","last_change":"2023-10-16T11:19:36.633999+0000","last_active":"2023-10-16T11:19:37.622379+0000","last_peered":"2023-10-16T11:19:37.622379+0000","last_clean":"2023-10-16T11:19:37.622379+0000","last_became_active":"2023-10-16T11:19:36.633722+0000","last_became_peered":"2023-10-16T11:19:36.633722+0000","last_unstale":"2023-10-16T11:19:37.622379+0000","last_undegraded":"2023-10-16T11:19:37.622379+0000","last_fullsized":"2023-10-16T11:19:37.622379+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.618948+0000","last_change":"2023-10-16T11:19:23.544178+0000","last_active":"2023-10-16T11:19:36.618948+0000","last_peered":"2023-10-16T11:19:36.618948+0000","last_clean":"2023-10-16T11:19:36.618948+0000","last_became_active":"2023-10-16T11:19:23.543898+0000","last_became_peered":"2023-10-16T11:19:23.543898+0000","last_unstale":"2023-10-16T11:19:36.618948+0000","last_undegraded":"2023-10-16T11:19:36.618948+0000","last_fullsized":"2023-10-16T11:19:36.618948+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893954+0000","last_change":"2023-10-16T11:19:36.638088+0000","last_active":"2023-10-16T11:19:37.893954+0000","last_peered":"2023-10-16T11:19:37.893954+0000","last_clean":"2023-10-16T11:19:37.893954+0000","last_became_active":"2023-10-16T11:19:36.637717+0000","last_became_peered":"2023-10-16T11:19:36.637717+0000","last_unstale":"2023-10-16T11:19:37.893954+0000","last_undegraded":"2023-10-16T11:19:37.893954+0000","last_fullsized":"2023-10-16T11:19:37.893954+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:14:19.443732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621366+0000","last_change":"2023-10-16T11:19:23.542878+0000","last_active":"2023-10-16T11:19:36.621366+0000","last_peered":"2023-10-16T11:19:36.621366+0000","last_clean":"2023-10-16T11:19:36.621366+0000","last_became_active":"2023-10-16T11:19:23.542593+0000","last_became_peered":"2023-10-16T11:19:23.542593+0000","last_unstale":"2023-10-16T11:19:36.621366+0000","last_undegraded":"2023-10-16T11:19:36.621366+0000","last_fullsized":"2023-10-16T11:19:36.621366+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:47:39.898154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616551+0000","last_change":"2023-10-16T11:19:23.547762+0000","last_active":"2023-10-16T11:19:36.616551+0000","last_peered":"2023-10-16T11:19:36.616551+0000","last_clean":"2023-10-16T11:19:36.616551+0000","last_became_active":"2023-10-16T11:19:23.547353+0000","last_became_peered":"2023-10-16T11:19:23.547353+0000","last_unstale":"2023-10-16T11:19:36.616551+0000","last_undegraded":"2023-10-16T11:19:36.616551+0000","last_fullsized":"2023-10-16T11:19:36.616551+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893121+0000","last_change":"2023-10-16T11:19:36.630687+0000","last_active":"2023-10-16T11:19:37.893121+0000","last_peered":"2023-10-16T11:19:37.893121+0000","last_clean":"2023-10-16T11:19:37.893121+0000","last_became_active":"2023-10-16T11:19:36.629515+0000","last_became_peered":"2023-10-16T11:19:36.629515+0000","last_unstale":"2023-10-16T11:19:37.893121+0000","last_undegraded":"2023-10-16T11:19:37.893121+0000","last_fullsized":"2023-10-16T11:19:37.893121+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:37:25.525530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893412+0000","last_change":"2023-10-16T11:19:36.634680+0000","last_active":"2023-10-16T11:19:37.893412+0000","last_peered":"2023-10-16T11:19:37.893412+0000","last_clean":"2023-10-16T11:19:37.893412+0000","last_became_active":"2023-10-16T11:19:36.632581+0000","last_became_peered":"2023-10-16T11:19:36.632581+0000","last_unstale":"2023-10-16T11:19:37.893412+0000","last_undegraded":"2023-10-16T11:19:37.893412+0000","last_fullsized":"2023-10-16T11:19:37.893412+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:14:56.278663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630199+0000","last_change":"2023-10-16T11:19:23.565127+0000","last_active":"2023-10-16T11:19:36.630199+0000","last_peered":"2023-10-16T11:19:36.630199+0000","last_clean":"2023-10-16T11:19:36.630199+0000","last_became_active":"2023-10-16T11:19:23.564852+0000","last_became_peered":"2023-10-16T11:19:23.564852+0000","last_unstale":"2023-10-16T11:19:36.630199+0000","last_undegraded":"2023-10-16T11:19:36.630199+0000","last_fullsized":"2023-10-16T11:19:36.630199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:18.439202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629214+0000","last_change":"2023-10-16T11:19:23.555152+0000","last_active":"2023-10-16T11:19:36.629214+0000","last_peered":"2023-10-16T11:19:36.629214+0000","last_clean":"2023-10-16T11:19:36.629214+0000","last_became_active":"2023-10-16T11:19:23.554747+0000","last_became_peered":"2023-10-16T11:19:23.554747+0000","last_unstale":"2023-10-16T11:19:36.629214+0000","last_undegraded":"2023-10-16T11:19:36.629214+0000","last_fullsized":"2023-10-16T11:19:36.629214+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892848+0000","last_change":"2023-10-16T11:19:36.635095+0000","last_active":"2023-10-16T11:19:37.892848+0000","last_peered":"2023-10-16T11:19:37.892848+0000","last_clean":"2023-10-16T11:19:37.892848+0000","last_became_active":"2023-10-16T11:19:36.634778+0000","last_became_peered":"2023-10-16T11:19:36.634778+0000","last_unstale":"2023-10-16T11:19:37.892848+0000","last_undegraded":"2023-10-16T11:19:37.892848+0000","last_fullsized":"2023-10-16T11:19:37.892848+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:34.012796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630759+0000","last_change":"2023-10-16T11:19:23.559781+0000","last_active":"2023-10-16T11:19:36.630759+0000","last_peered":"2023-10-16T11:19:36.630759+0000","last_clean":"2023-10-16T11:19:36.630759+0000","last_became_active":"2023-10-16T11:19:23.559451+0000","last_became_peered":"2023-10-16T11:19:23.559451+0000","last_unstale":"2023-10-16T11:19:36.630759+0000","last_undegraded":"2023-10-16T11:19:36.630759+0000","last_fullsized":"2023-10-16T11:19:36.630759+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:02.043015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621656+0000","last_change":"2023-10-16T11:19:36.641041+0000","last_active":"2023-10-16T11:19:37.621656+0000","last_peered":"2023-10-16T11:19:37.621656+0000","last_clean":"2023-10-16T11:19:37.621656+0000","last_became_active":"2023-10-16T11:19:36.640761+0000","last_became_peered":"2023-10-16T11:19:36.640761+0000","last_unstale":"2023-10-16T11:19:37.621656+0000","last_undegraded":"2023-10-16T11:19:37.621656+0000","last_fullsized":"2023-10-16T11:19:37.621656+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:26.350349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622347+0000","last_change":"2023-10-16T11:19:23.546460+0000","last_active":"2023-10-16T11:19:36.622347+0000","last_peered":"2023-10-16T11:19:36.622347+0000","last_clean":"2023-10-16T11:19:36.622347+0000","last_became_active":"2023-10-16T11:19:23.546187+0000","last_became_peered":"2023-10-16T11:19:23.546187+0000","last_unstale":"2023-10-16T11:19:36.622347+0000","last_undegraded":"2023-10-16T11:19:36.622347+0000","last_fullsized":"2023-10-16T11:19:36.622347+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:42:11.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621306+0000","last_change":"2023-10-16T11:19:36.630530+0000","last_active":"2023-10-16T11:19:37.621306+0000","last_peered":"2023-10-16T11:19:37.621306+0000","last_clean":"2023-10-16T11:19:37.621306+0000","last_became_active":"2023-10-16T11:19:36.630239+0000","last_became_peered":"2023-10-16T11:19:36.630239+0000","last_unstale":"2023-10-16T11:19:37.621306+0000","last_undegraded":"2023-10-16T11:19:37.621306+0000","last_fullsized":"2023-10-16T11:19:37.621306+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:15.034574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616639+0000","last_change":"2023-10-16T11:19:23.546857+0000","last_active":"2023-10-16T11:19:36.616639+0000","last_peered":"2023-10-16T11:19:36.616639+0000","last_clean":"2023-10-16T11:19:36.616639+0000","last_became_active":"2023-10-16T11:19:23.546592+0000","last_became_peered":"2023-10-16T11:19:23.546592+0000","last_unstale":"2023-10-16T11:19:36.616639+0000","last_undegraded":"2023-10-16T11:19:36.616639+0000","last_fullsized":"2023-10-16T11:19:36.616639+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:29:06.634923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893169+0000","last_change":"2023-10-16T11:19:36.622716+0000","last_active":"2023-10-16T11:19:37.893169+0000","last_peered":"2023-10-16T11:19:37.893169+0000","last_clean":"2023-10-16T11:19:37.893169+0000","last_became_active":"2023-10-16T11:19:36.622420+0000","last_became_peered":"2023-10-16T11:19:36.622420+0000","last_unstale":"2023-10-16T11:19:37.893169+0000","last_undegraded":"2023-10-16T11:19:37.893169+0000","last_fullsized":"2023-10-16T11:19:37.893169+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:58.765425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893941+0000","last_change":"2023-10-16T11:19:36.633980+0000","last_active":"2023-10-16T11:19:37.893941+0000","last_peered":"2023-10-16T11:19:37.893941+0000","last_clean":"2023-10-16T11:19:37.893941+0000","last_became_active":"2023-10-16T11:19:36.633643+0000","last_became_peered":"2023-10-16T11:19:36.633643+0000","last_unstale":"2023-10-16T11:19:37.893941+0000","last_undegraded":"2023-10-16T11:19:37.893941+0000","last_fullsized":"2023-10-16T11:19:37.893941+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:06:27.398549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.617918+0000","last_change":"2023-10-16T11:19:23.548868+0000","last_active":"2023-10-16T11:19:36.617918+0000","last_peered":"2023-10-16T11:19:36.617918+0000","last_clean":"2023-10-16T11:19:36.617918+0000","last_became_active":"2023-10-16T11:19:23.548583+0000","last_became_peered":"2023-10-16T11:19:23.548583+0000","last_unstale":"2023-10-16T11:19:36.617918+0000","last_undegraded":"2023-10-16T11:19:36.617918+0000","last_fullsized":"2023-10-16T11:19:36.617918+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:02.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.622147+0000","last_change":"2023-10-16T11:19:36.628944+0000","last_active":"2023-10-16T11:19:37.622147+0000","last_peered":"2023-10-16T11:19:37.622147+0000","last_clean":"2023-10-16T11:19:37.622147+0000","last_became_active":"2023-10-16T11:19:36.628652+0000","last_became_peered":"2023-10-16T11:19:36.628652+0000","last_unstale":"2023-10-16T11:19:37.622147+0000","last_undegraded":"2023-10-16T11:19:37.622147+0000","last_fullsized":"2023-10-16T11:19:37.622147+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:18.537927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624713+0000","last_change":"2023-10-16T11:19:23.552048+0000","last_active":"2023-10-16T11:19:36.624713+0000","last_peered":"2023-10-16T11:19:36.624713+0000","last_clean":"2023-10-16T11:19:36.624713+0000","last_became_active":"2023-10-16T11:19:23.551676+0000","last_became_peered":"2023-10-16T11:19:23.551676+0000","last_unstale":"2023-10-16T11:19:36.624713+0000","last_undegraded":"2023-10-16T11:19:36.624713+0000","last_fullsized":"2023-10-16T11:19:36.624713+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:22:16.450373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893328+0000","last_change":"2023-10-16T11:19:36.626030+0000","last_active":"2023-10-16T11:19:37.893328+0000","last_peered":"2023-10-16T11:19:37.893328+0000","last_clean":"2023-10-16T11:19:37.893328+0000","last_became_active":"2023-10-16T11:19:36.625727+0000","last_became_peered":"2023-10-16T11:19:36.625727+0000","last_unstale":"2023-10-16T11:19:37.893328+0000","last_undegraded":"2023-10-16T11:19:37.893328+0000","last_fullsized":"2023-10-16T11:19:37.893328+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:07:58.374107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.624300+0000","last_change":"2023-10-16T11:19:23.546038+0000","last_active":"2023-10-16T11:19:36.624300+0000","last_peered":"2023-10-16T11:19:36.624300+0000","last_clean":"2023-10-16T11:19:36.624300+0000","last_became_active":"2023-10-16T11:19:23.545752+0000","last_became_peered":"2023-10-16T11:19:23.545752+0000","last_unstale":"2023-10-16T11:19:36.624300+0000","last_undegraded":"2023-10-16T11:19:36.624300+0000","last_fullsized":"2023-10-16T11:19:36.624300+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:51:13.311779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893888+0000","last_change":"2023-10-16T11:19:36.629691+0000","last_active":"2023-10-16T11:19:37.893888+0000","last_peered":"2023-10-16T11:19:37.893888+0000","last_clean":"2023-10-16T11:19:37.893888+0000","last_became_active":"2023-10-16T11:19:36.629384+0000","last_became_peered":"2023-10-16T11:19:36.629384+0000","last_unstale":"2023-10-16T11:19:37.893888+0000","last_undegraded":"2023-10-16T11:19:37.893888+0000","last_fullsized":"2023-10-16T11:19:37.893888+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:15.903147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619019+0000","last_change":"2023-10-16T11:19:23.550209+0000","last_active":"2023-10-16T11:19:36.619019+0000","last_peered":"2023-10-16T11:19:36.619019+0000","last_clean":"2023-10-16T11:19:36.619019+0000","last_became_active":"2023-10-16T11:19:23.549849+0000","last_became_peered":"2023-10-16T11:19:23.549849+0000","last_unstale":"2023-10-16T11:19:36.619019+0000","last_undegraded":"2023-10-16T11:19:36.619019+0000","last_fullsized":"2023-10-16T11:19:36.619019+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:29.715382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621124+0000","last_change":"2023-10-16T11:19:36.631361+0000","last_active":"2023-10-16T11:19:37.621124+0000","last_peered":"2023-10-16T11:19:37.621124+0000","last_clean":"2023-10-16T11:19:37.621124+0000","last_became_active":"2023-10-16T11:19:36.631013+0000","last_became_peered":"2023-10-16T11:19:36.631013+0000","last_unstale":"2023-10-16T11:19:37.621124+0000","last_undegraded":"2023-10-16T11:19:37.621124+0000","last_fullsized":"2023-10-16T11:19:37.621124+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615695+0000","last_change":"2023-10-16T11:19:23.542321+0000","last_active":"2023-10-16T11:19:36.615695+0000","last_peered":"2023-10-16T11:19:36.615695+0000","last_clean":"2023-10-16T11:19:36.615695+0000","last_became_active":"2023-10-16T11:19:23.541862+0000","last_became_peered":"2023-10-16T11:19:23.541862+0000","last_unstale":"2023-10-16T11:19:36.615695+0000","last_undegraded":"2023-10-16T11:19:36.615695+0000","last_fullsized":"2023-10-16T11:19:36.615695+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:24:50.522254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892711+0000","last_change":"2023-10-16T11:19:36.620937+0000","last_active":"2023-10-16T11:19:37.892711+0000","last_peered":"2023-10-16T11:19:37.892711+0000","last_clean":"2023-10-16T11:19:37.892711+0000","last_became_active":"2023-10-16T11:19:36.620641+0000","last_became_peered":"2023-10-16T11:19:36.620641+0000","last_unstale":"2023-10-16T11:19:37.892711+0000","last_undegraded":"2023-10-16T11:19:37.892711+0000","last_fullsized":"2023-10-16T11:19:37.892711+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:15.827874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623487+0000","last_change":"2023-10-16T11:19:23.547740+0000","last_active":"2023-10-16T11:19:36.623487+0000","last_peered":"2023-10-16T11:19:36.623487+0000","last_clean":"2023-10-16T11:19:36.623487+0000","last_became_active":"2023-10-16T11:19:23.546421+0000","last_became_peered":"2023-10-16T11:19:23.546421+0000","last_unstale":"2023-10-16T11:19:36.623487+0000","last_undegraded":"2023-10-16T11:19:36.623487+0000","last_fullsized":"2023-10-16T11:19:36.623487+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:56:27.685924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892763+0000","last_change":"2023-10-16T11:19:36.631333+0000","last_active":"2023-10-16T11:19:37.892763+0000","last_peered":"2023-10-16T11:19:37.892763+0000","last_clean":"2023-10-16T11:19:37.892763+0000","last_became_active":"2023-10-16T11:19:36.630735+0000","last_became_peered":"2023-10-16T11:19:36.630735+0000","last_unstale":"2023-10-16T11:19:37.892763+0000","last_undegraded":"2023-10-16T11:19:37.892763+0000","last_fullsized":"2023-10-16T11:19:37.892763+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:26.684291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623942+0000","last_change":"2023-10-16T11:19:23.549769+0000","last_active":"2023-10-16T11:19:36.623942+0000","last_peered":"2023-10-16T11:19:36.623942+0000","last_clean":"2023-10-16T11:19:36.623942+0000","last_became_active":"2023-10-16T11:19:23.549505+0000","last_became_peered":"2023-10-16T11:19:23.549505+0000","last_unstale":"2023-10-16T11:19:36.623942+0000","last_undegraded":"2023-10-16T11:19:36.623942+0000","last_fullsized":"2023-10-16T11:19:36.623942+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:56:20.931218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892826+0000","last_change":"2023-10-16T11:19:36.631224+0000","last_active":"2023-10-16T11:19:37.892826+0000","last_peered":"2023-10-16T11:19:37.892826+0000","last_clean":"2023-10-16T11:19:37.892826+0000","last_became_active":"2023-10-16T11:19:36.630966+0000","last_became_peered":"2023-10-16T11:19:36.630966+0000","last_unstale":"2023-10-16T11:19:37.892826+0000","last_undegraded":"2023-10-16T11:19:37.892826+0000","last_fullsized":"2023-10-16T11:19:37.892826+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:45.783168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.622027+0000","last_change":"2023-10-16T11:19:23.548276+0000","last_active":"2023-10-16T11:19:36.622027+0000","last_peered":"2023-10-16T11:19:36.622027+0000","last_clean":"2023-10-16T11:19:36.622027+0000","last_became_active":"2023-10-16T11:19:23.547177+0000","last_became_peered":"2023-10-16T11:19:23.547177+0000","last_unstale":"2023-10-16T11:19:36.622027+0000","last_undegraded":"2023-10-16T11:19:36.622027+0000","last_fullsized":"2023-10-16T11:19:36.622027+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:04.225211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893905+0000","last_change":"2023-10-16T11:19:36.635809+0000","last_active":"2023-10-16T11:19:37.893905+0000","last_peered":"2023-10-16T11:19:37.893905+0000","last_clean":"2023-10-16T11:19:37.893905+0000","last_became_active":"2023-10-16T11:19:36.635543+0000","last_became_peered":"2023-10-16T11:19:36.635543+0000","last_unstale":"2023-10-16T11:19:37.893905+0000","last_undegraded":"2023-10-16T11:19:37.893905+0000","last_fullsized":"2023-10-16T11:19:37.893905+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:06:05.832943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621596+0000","last_change":"2023-10-16T11:19:23.563754+0000","last_active":"2023-10-16T11:19:36.621596+0000","last_peered":"2023-10-16T11:19:36.621596+0000","last_clean":"2023-10-16T11:19:36.621596+0000","last_became_active":"2023-10-16T11:19:23.563487+0000","last_became_peered":"2023-10-16T11:19:23.563487+0000","last_unstale":"2023-10-16T11:19:36.621596+0000","last_undegraded":"2023-10-16T11:19:36.621596+0000","last_fullsized":"2023-10-16T11:19:36.621596+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:13.814299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893984+0000","last_change":"2023-10-16T11:19:36.639804+0000","last_active":"2023-10-16T11:19:37.893984+0000","last_peered":"2023-10-16T11:19:37.893984+0000","last_clean":"2023-10-16T11:19:37.893984+0000","last_became_active":"2023-10-16T11:19:36.639421+0000","last_became_peered":"2023-10-16T11:19:36.639421+0000","last_unstale":"2023-10-16T11:19:37.893984+0000","last_undegraded":"2023-10-16T11:19:37.893984+0000","last_fullsized":"2023-10-16T11:19:37.893984+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:53:21.232537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619182+0000","last_change":"2023-10-16T11:19:23.551280+0000","last_active":"2023-10-16T11:19:36.619182+0000","last_peered":"2023-10-16T11:19:36.619182+0000","last_clean":"2023-10-16T11:19:36.619182+0000","last_became_active":"2023-10-16T11:19:23.551018+0000","last_became_peered":"2023-10-16T11:19:23.551018+0000","last_unstale":"2023-10-16T11:19:36.619182+0000","last_undegraded":"2023-10-16T11:19:36.619182+0000","last_fullsized":"2023-10-16T11:19:36.619182+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:01:46.256661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893878+0000","last_change":"2023-10-16T11:19:36.627799+0000","last_active":"2023-10-16T11:19:37.893878+0000","last_peered":"2023-10-16T11:19:37.893878+0000","last_clean":"2023-10-16T11:19:37.893878+0000","last_became_active":"2023-10-16T11:19:36.627495+0000","last_became_peered":"2023-10-16T11:19:36.627495+0000","last_unstale":"2023-10-16T11:19:37.893878+0000","last_undegraded":"2023-10-16T11:19:37.893878+0000","last_fullsized":"2023-10-16T11:19:37.893878+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:43:32.348270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.620399+0000","last_change":"2023-10-16T11:19:23.559938+0000","last_active":"2023-10-16T11:19:36.620399+0000","last_peered":"2023-10-16T11:19:36.620399+0000","last_clean":"2023-10-16T11:19:36.620399+0000","last_became_active":"2023-10-16T11:19:23.559569+0000","last_became_peered":"2023-10-16T11:19:23.559569+0000","last_unstale":"2023-10-16T11:19:36.620399+0000","last_undegraded":"2023-10-16T11:19:36.620399+0000","last_fullsized":"2023-10-16T11:19:36.620399+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:56.961615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893192+0000","last_change":"2023-10-16T11:19:36.625387+0000","last_active":"2023-10-16T11:19:37.893192+0000","last_peered":"2023-10-16T11:19:37.893192+0000","last_clean":"2023-10-16T11:19:37.893192+0000","last_became_active":"2023-10-16T11:19:36.625097+0000","last_became_peered":"2023-10-16T11:19:36.625097+0000","last_unstale":"2023-10-16T11:19:37.893192+0000","last_undegraded":"2023-10-16T11:19:37.893192+0000","last_fullsized":"2023-10-16T11:19:37.893192+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:12.164686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.630199+0000","last_change":"2023-10-16T11:19:23.560050+0000","last_active":"2023-10-16T11:19:36.630199+0000","last_peered":"2023-10-16T11:19:36.630199+0000","last_clean":"2023-10-16T11:19:36.630199+0000","last_became_active":"2023-10-16T11:19:23.559558+0000","last_became_peered":"2023-10-16T11:19:23.559558+0000","last_unstale":"2023-10-16T11:19:36.630199+0000","last_undegraded":"2023-10-16T11:19:36.630199+0000","last_fullsized":"2023-10-16T11:19:36.630199+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:35.714788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621911+0000","last_change":"2023-10-16T11:19:36.629616+0000","last_active":"2023-10-16T11:19:37.621911+0000","last_peered":"2023-10-16T11:19:37.621911+0000","last_clean":"2023-10-16T11:19:37.621911+0000","last_became_active":"2023-10-16T11:19:36.629056+0000","last_became_peered":"2023-10-16T11:19:36.629056+0000","last_unstale":"2023-10-16T11:19:37.621911+0000","last_undegraded":"2023-10-16T11:19:37.621911+0000","last_fullsized":"2023-10-16T11:19:37.621911+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:11.302811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.632358+0000","last_change":"2023-10-16T11:19:23.560016+0000","last_active":"2023-10-16T11:19:36.632358+0000","last_peered":"2023-10-16T11:19:36.632358+0000","last_clean":"2023-10-16T11:19:36.632358+0000","last_became_active":"2023-10-16T11:19:23.559619+0000","last_became_peered":"2023-10-16T11:19:23.559619+0000","last_unstale":"2023-10-16T11:19:36.632358+0000","last_undegraded":"2023-10-16T11:19:36.632358+0000","last_fullsized":"2023-10-16T11:19:36.632358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:43:04.551791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893587+0000","last_change":"2023-10-16T11:19:36.628684+0000","last_active":"2023-10-16T11:19:37.893587+0000","last_peered":"2023-10-16T11:19:37.893587+0000","last_clean":"2023-10-16T11:19:37.893587+0000","last_became_active":"2023-10-16T11:19:36.628264+0000","last_became_peered":"2023-10-16T11:19:36.628264+0000","last_unstale":"2023-10-16T11:19:37.893587+0000","last_undegraded":"2023-10-16T11:19:37.893587+0000","last_fullsized":"2023-10-16T11:19:37.893587+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:41:54.029905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629289+0000","last_change":"2023-10-16T11:19:23.564191+0000","last_active":"2023-10-16T11:19:36.629289+0000","last_peered":"2023-10-16T11:19:36.629289+0000","last_clean":"2023-10-16T11:19:36.629289+0000","last_became_active":"2023-10-16T11:19:23.563291+0000","last_became_peered":"2023-10-16T11:19:23.563291+0000","last_unstale":"2023-10-16T11:19:36.629289+0000","last_undegraded":"2023-10-16T11:19:36.629289+0000","last_fullsized":"2023-10-16T11:19:36.629289+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:14.502969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894719+0000","last_change":"2023-10-16T11:19:36.627078+0000","last_active":"2023-10-16T11:19:37.894719+0000","last_peered":"2023-10-16T11:19:37.894719+0000","last_clean":"2023-10-16T11:19:37.894719+0000","last_became_active":"2023-10-16T11:19:36.626791+0000","last_became_peered":"2023-10-16T11:19:36.626791+0000","last_unstale":"2023-10-16T11:19:37.894719+0000","last_undegraded":"2023-10-16T11:19:37.894719+0000","last_fullsized":"2023-10-16T11:19:37.894719+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:14.780239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.623858+0000","last_change":"2023-10-16T11:19:23.546735+0000","last_active":"2023-10-16T11:19:36.623858+0000","last_peered":"2023-10-16T11:19:36.623858+0000","last_clean":"2023-10-16T11:19:36.623858+0000","last_became_active":"2023-10-16T11:19:23.546404+0000","last_became_peered":"2023-10-16T11:19:23.546404+0000","last_unstale":"2023-10-16T11:19:36.623858+0000","last_undegraded":"2023-10-16T11:19:36.623858+0000","last_fullsized":"2023-10-16T11:19:36.623858+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:29.845437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.620865+0000","last_change":"2023-10-16T11:19:36.630872+0000","last_active":"2023-10-16T11:19:37.620865+0000","last_peered":"2023-10-16T11:19:37.620865+0000","last_clean":"2023-10-16T11:19:37.620865+0000","last_became_active":"2023-10-16T11:19:36.630391+0000","last_became_peered":"2023-10-16T11:19:36.630391+0000","last_unstale":"2023-10-16T11:19:37.620865+0000","last_undegraded":"2023-10-16T11:19:37.620865+0000","last_fullsized":"2023-10-16T11:19:37.620865+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:23.825418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616656+0000","last_change":"2023-10-16T11:19:23.552080+0000","last_active":"2023-10-16T11:19:36.616656+0000","last_peered":"2023-10-16T11:19:36.616656+0000","last_clean":"2023-10-16T11:19:36.616656+0000","last_became_active":"2023-10-16T11:19:23.551779+0000","last_became_peered":"2023-10-16T11:19:23.551779+0000","last_unstale":"2023-10-16T11:19:36.616656+0000","last_undegraded":"2023-10-16T11:19:36.616656+0000","last_fullsized":"2023-10-16T11:19:36.616656+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894374+0000","last_change":"2023-10-16T11:19:36.636621+0000","last_active":"2023-10-16T11:19:37.894374+0000","last_peered":"2023-10-16T11:19:37.894374+0000","last_clean":"2023-10-16T11:19:37.894374+0000","last_became_active":"2023-10-16T11:19:36.636222+0000","last_became_peered":"2023-10-16T11:19:36.636222+0000","last_unstale":"2023-10-16T11:19:37.894374+0000","last_undegraded":"2023-10-16T11:19:37.894374+0000","last_fullsized":"2023-10-16T11:19:37.894374+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:03:51.038856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615178+0000","last_change":"2023-10-16T11:19:23.550227+0000","last_active":"2023-10-16T11:19:36.615178+0000","last_peered":"2023-10-16T11:19:36.615178+0000","last_clean":"2023-10-16T11:19:36.615178+0000","last_became_active":"2023-10-16T11:19:23.549938+0000","last_became_peered":"2023-10-16T11:19:23.549938+0000","last_unstale":"2023-10-16T11:19:36.615178+0000","last_undegraded":"2023-10-16T11:19:36.615178+0000","last_fullsized":"2023-10-16T11:19:36.615178+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:27:13.268771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893818+0000","last_change":"2023-10-16T11:19:36.636806+0000","last_active":"2023-10-16T11:19:37.893818+0000","last_peered":"2023-10-16T11:19:37.893818+0000","last_clean":"2023-10-16T11:19:37.893818+0000","last_became_active":"2023-10-16T11:19:36.636526+0000","last_became_peered":"2023-10-16T11:19:36.636526+0000","last_unstale":"2023-10-16T11:19:37.893818+0000","last_undegraded":"2023-10-16T11:19:37.893818+0000","last_fullsized":"2023-10-16T11:19:37.893818+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:26.018990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619096+0000","last_change":"2023-10-16T11:19:23.547259+0000","last_active":"2023-10-16T11:19:36.619096+0000","last_peered":"2023-10-16T11:19:36.619096+0000","last_clean":"2023-10-16T11:19:36.619096+0000","last_became_active":"2023-10-16T11:19:23.546957+0000","last_became_peered":"2023-10-16T11:19:23.546957+0000","last_unstale":"2023-10-16T11:19:36.619096+0000","last_undegraded":"2023-10-16T11:19:36.619096+0000","last_fullsized":"2023-10-16T11:19:36.619096+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:35:50.106488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893676+0000","last_change":"2023-10-16T11:19:36.626949+0000","last_active":"2023-10-16T11:19:37.893676+0000","last_peered":"2023-10-16T11:19:37.893676+0000","last_clean":"2023-10-16T11:19:37.893676+0000","last_became_active":"2023-10-16T11:19:36.626670+0000","last_became_peered":"2023-10-16T11:19:36.626670+0000","last_unstale":"2023-10-16T11:19:37.893676+0000","last_undegraded":"2023-10-16T11:19:37.893676+0000","last_fullsized":"2023-10-16T11:19:37.893676+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:33:14.675395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.615351+0000","last_change":"2023-10-16T11:19:23.550179+0000","last_active":"2023-10-16T11:19:36.615351+0000","last_peered":"2023-10-16T11:19:36.615351+0000","last_clean":"2023-10-16T11:19:36.615351+0000","last_became_active":"2023-10-16T11:19:23.549811+0000","last_became_peered":"2023-10-16T11:19:23.549811+0000","last_unstale":"2023-10-16T11:19:36.615351+0000","last_undegraded":"2023-10-16T11:19:36.615351+0000","last_fullsized":"2023-10-16T11:19:36.615351+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:45:58.221233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.894684+0000","last_change":"2023-10-16T11:19:36.632889+0000","last_active":"2023-10-16T11:19:37.894684+0000","last_peered":"2023-10-16T11:19:37.894684+0000","last_clean":"2023-10-16T11:19:37.894684+0000","last_became_active":"2023-10-16T11:19:36.632268+0000","last_became_peered":"2023-10-16T11:19:36.632268+0000","last_unstale":"2023-10-16T11:19:37.894684+0000","last_undegraded":"2023-10-16T11:19:37.894684+0000","last_fullsized":"2023-10-16T11:19:37.894684+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:11.302811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616511+0000","last_change":"2023-10-16T11:19:23.541856+0000","last_active":"2023-10-16T11:19:36.616511+0000","last_peered":"2023-10-16T11:19:36.616511+0000","last_clean":"2023-10-16T11:19:36.616511+0000","last_became_active":"2023-10-16T11:19:23.541549+0000","last_became_peered":"2023-10-16T11:19:23.541549+0000","last_unstale":"2023-10-16T11:19:36.616511+0000","last_undegraded":"2023-10-16T11:19:36.616511+0000","last_fullsized":"2023-10-16T11:19:36.616511+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:07.709989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892542+0000","last_change":"2023-10-16T11:19:36.625337+0000","last_active":"2023-10-16T11:19:37.892542+0000","last_peered":"2023-10-16T11:19:37.892542+0000","last_clean":"2023-10-16T11:19:37.892542+0000","last_became_active":"2023-10-16T11:19:36.624796+0000","last_became_peered":"2023-10-16T11:19:36.624796+0000","last_unstale":"2023-10-16T11:19:37.892542+0000","last_undegraded":"2023-10-16T11:19:37.892542+0000","last_fullsized":"2023-10-16T11:19:37.892542+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:27.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.619098+0000","last_change":"2023-10-16T11:19:23.547136+0000","last_active":"2023-10-16T11:19:36.619098+0000","last_peered":"2023-10-16T11:19:36.619098+0000","last_clean":"2023-10-16T11:19:36.619098+0000","last_became_active":"2023-10-16T11:19:23.546834+0000","last_became_peered":"2023-10-16T11:19:23.546834+0000","last_unstale":"2023-10-16T11:19:36.619098+0000","last_undegraded":"2023-10-16T11:19:36.619098+0000","last_fullsized":"2023-10-16T11:19:36.619098+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:45.683847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.892567+0000","last_change":"2023-10-16T11:19:36.628989+0000","last_active":"2023-10-16T11:19:37.892567+0000","last_peered":"2023-10-16T11:19:37.892567+0000","last_clean":"2023-10-16T11:19:37.892567+0000","last_became_active":"2023-10-16T11:19:36.628665+0000","last_became_peered":"2023-10-16T11:19:36.628665+0000","last_unstale":"2023-10-16T11:19:37.892567+0000","last_undegraded":"2023-10-16T11:19:37.892567+0000","last_fullsized":"2023-10-16T11:19:37.892567+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:57:42.796960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.629084+0000","last_change":"2023-10-16T11:19:23.552160+0000","last_active":"2023-10-16T11:19:36.629084+0000","last_peered":"2023-10-16T11:19:36.629084+0000","last_clean":"2023-10-16T11:19:36.629084+0000","last_became_active":"2023-10-16T11:19:23.551805+0000","last_became_peered":"2023-10-16T11:19:23.551805+0000","last_unstale":"2023-10-16T11:19:36.629084+0000","last_undegraded":"2023-10-16T11:19:36.629084+0000","last_fullsized":"2023-10-16T11:19:36.629084+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.621385+0000","last_change":"2023-10-16T11:19:36.634749+0000","last_active":"2023-10-16T11:19:37.621385+0000","last_peered":"2023-10-16T11:19:37.621385+0000","last_clean":"2023-10-16T11:19:37.621385+0000","last_became_active":"2023-10-16T11:19:36.634467+0000","last_became_peered":"2023-10-16T11:19:36.634467+0000","last_unstale":"2023-10-16T11:19:37.621385+0000","last_undegraded":"2023-10-16T11:19:37.621385+0000","last_fullsized":"2023-10-16T11:19:37.621385+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:10:57.494673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.616111+0000","last_change":"2023-10-16T11:19:23.552427+0000","last_active":"2023-10-16T11:19:36.616111+0000","last_peered":"2023-10-16T11:19:36.616111+0000","last_clean":"2023-10-16T11:19:36.616111+0000","last_became_active":"2023-10-16T11:19:23.552156+0000","last_became_peered":"2023-10-16T11:19:23.552156+0000","last_unstale":"2023-10-16T11:19:36.616111+0000","last_undegraded":"2023-10-16T11:19:36.616111+0000","last_fullsized":"2023-10-16T11:19:36.616111+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:18.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":11,"reported_epoch":220,"state":"active+clean","last_fresh":"2023-10-16T11:19:37.893142+0000","last_change":"2023-10-16T11:19:36.633460+0000","last_active":"2023-10-16T11:19:37.893142+0000","last_peered":"2023-10-16T11:19:37.893142+0000","last_clean":"2023-10-16T11:19:37.893142+0000","last_became_active":"2023-10-16T11:19:36.631460+0000","last_became_peered":"2023-10-16T11:19:36.631460+0000","last_unstale":"2023-10-16T11:19:37.893142+0000","last_undegraded":"2023-10-16T11:19:37.893142+0000","last_fullsized":"2023-10-16T11:19:37.893142+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":218,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:35.600376+0000","last_clean_scrub_stamp":"2023-10-16T11:19:35.600376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:13:07.500018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2023-10-16T11:19:36.621912+0000","last_change":"2023-10-16T11:19:23.547472+0000","last_active":"2023-10-16T11:19:36.621912+0000","last_peered":"2023-10-16T11:19:36.621912+0000","last_clean":"2023-10-16T11:19:36.621912+0000","last_became_active":"2023-10-16T11:19:23.547148+0000","last_became_peered":"2023-10-16T11:19:23.547148+0000","last_unstale":"2023-10-16T11:19:36.621912+0000","last_undegraded":"2023-10-16T11:19:36.621912+0000","last_fullsized":"2023-10-16T11:19:36.621912+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:19:22.518798+0000","last_clean_scrub_stamp":"2023-10-16T11:19:22.518798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:03:14.706396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":3657,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543283,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52516,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319324,"statfs":{"total":96636764160,"available":96582987776,"internally_reserved":0,"allocated":3366912,"data_stored":2012088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.27800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.496},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42399999999999999,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.83399999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.94199999999999995},"last":0.626}]},{"osd":3,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69899999999999995,"5min":0.79900000000000004,"15min":0.86299999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.307},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.002},"last":0.54900000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.35099999999999998,"15min":0.33100000000000002},"max":{"1min":8.1950000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.39200000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.92900000000000005,"15min":0.97699999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.83799999999999997,"15min":1.0169999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.871},"last":0.44500000000000001}]}]},{"osd":6,"up_from":14,"seq":60129543282,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2768896,"data_stored":1419429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.29899999999999999},"max":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.875},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.51200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.51100000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.82599999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77200000000000002,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.495}]},{"osd":2,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.69699999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45800000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40200000000000002},"max":{"1min":0.88300000000000001,"5min":0.88400000000000001,"15min":0.89100000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.74399999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38900000000000001,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.752,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.32100000000000001},"max":{"1min":0.79500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.91100000000000003},"last":0.57599999999999996}]}]},{"osd":1,"up_from":14,"seq":60129543282,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60256,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311584,"statfs":{"total":96636764160,"available":96575062016,"internally_reserved":0,"allocated":2768896,"data_stored":1418937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.224},"max":{"1min":0.60199999999999998,"5min":0.629,"15min":0.68700000000000006},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.41599999999999998,"15min":0.434},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.62,"5min":0.65900000000000003,"15min":0.91700000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41899999999999998},"min":{"1min":0.314,"5min":0.28000000000000003,"15min":0.24199999999999999},"max":{"1min":0.55300000000000005,"5min":0.58699999999999997,"15min":0.98899999999999999},"last":0.37},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.433}]},{"osd":3,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.78300000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.46899999999999997,"15min":0.47599999999999998},"min":{"1min":0.34000000000000002,"5min":0.30599999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.45100000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64700000000000002,"15min":0.66400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.42899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.50600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.0289999999999999,"15min":1.097},"last":0.76000000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.69199999999999995},"min":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":1.147},"last":0.81399999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51300000000000001,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.88,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0900000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.52600000000000002,"5min":0.47199999999999998,"15min":0.46000000000000002},"max":{"1min":0.98899999999999999,"5min":1.004,"15min":1.0600000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.46300000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.1060000000000001},"last":0.84399999999999997}]}]},{"osd":0,"up_from":14,"seq":60129543283,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60836,"kb_used_data":3284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311004,"statfs":{"total":96636764160,"available":96574468096,"internally_reserved":0,"allocated":3362816,"data_stored":2009921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.46400000000000002,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28799999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58799999999999997,"5min":0.63700000000000001,"15min":0.70299999999999996},"last":0.378},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.35399999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.68300000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.438}]},{"osd":4,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.99199999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":1.008},"last":0.67800000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":1.004,"15min":1.008},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.68899999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.622},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72599999999999998}]}]},{"osd":2,"up_from":14,"seq":60129543282,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":2777088,"data_stored":1420459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.67700000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.29199999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.63},"last":0.30499999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.438},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":0.65200000000000002,"15min":0.68799999999999994},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44500000000000001,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67400000000000004,"5min":0.67700000000000005,"15min":0.71599999999999997},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.435},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.56599999999999995,"5min":0.623,"15min":0.65500000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.81200000000000006},"last":0.42399999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.89900000000000002,"5min":0.97199999999999998,"15min":0.97599999999999998},"last":0.68899999999999995}]},{"osd":5,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.49099999999999999,"5min":0.45400000000000001,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.41999999999999998},"max":{"1min":0.86299999999999999,"5min":1.052,"15min":1.052},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.65200000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68700000000000006,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71399999999999997}]}]},{"osd":3,"up_from":14,"seq":60129543283,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47788,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324052,"statfs":{"total":96636764160,"available":96587829248,"internally_reserved":0,"allocated":2785280,"data_stored":1420655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.34999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.45400000000000001,"15min":0.45700000000000002},"min":{"1min":0.36799999999999999,"5min":0.28199999999999997,"15min":0.24399999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.28399999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.45500000000000002},"min":{"1min":0.316,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":0.60699999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45400000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":1.034,"15min":1.0740000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.46200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.76800000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0129999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":1.0489999999999999,"15min":1.0649999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.84699999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.48299999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.498,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97599999999999998,"5min":1.016,"15min":1.0489999999999999},"last":0.96399999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543282,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":2781184,"data_stored":1422336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.628},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.84399999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.627},"min":{"1min":0.438,"5min":0.40400000000000003,"15min":0.34899999999999998},"max":{"1min":0.72499999999999998,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.62,"15min":0.623},"min":{"1min":0.45800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50800000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.879},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.69499999999999995,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.70999999999999996,"5min":0.77300000000000002,"15min":0.89500000000000002},"last":0.52300000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.71999999999999997,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:18:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36199999999999999},"max":{"1min":0.81999999999999995,"5min":0.875,"15min":0.875},"last":0.441},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.40799999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80600000000000005,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.54900000000000004}]}]},{"osd":5,"up_from":14,"seq":60129543283,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42196,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329644,"statfs":{"total":96636764160,"available":96593555456,"internally_reserved":0,"allocated":2760704,"data_stored":1419373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.86099999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":1.873,"15min":1.873},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.39200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.66600000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":1.821,"15min":1.821},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.55900000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.69499999999999995,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":1.919,"15min":1.919},"last":0.68200000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.53800000000000003},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.873},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.63400000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35299999999999998},"max":{"1min":0.81299999999999994,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46999999999999997,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":2.02,"15min":2.02},"last":0.72999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:19:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:19:46.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:19:46.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.255+0000 7f7a9b4d0700 1 Processor -- start 2023-10-16T11:19:46.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.255+0000 7f7a9b4d0700 1 -- start start 2023-10-16T11:19:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9b4d0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a9415db70 0x7f7a940665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:46.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9b4d0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94066c60 con 0x7f7a940cedd0 2023-10-16T11:19:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9b4d0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9406c1a0 con 0x7f7a940ca2e0 2023-10-16T11:19:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9b4d0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940cda10 con 0x7f7a9415db70 2023-10-16T11:19:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a98a6b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7a940cedd0 0x7f7a940cf250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38780/0 (socket says 172.21.15.181:38780) 2023-10-16T11:19:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a98a6b700 1 -- 172.21.15.181:0/3994505141 learned_addr learned my addr 172.21.15.181:0/3994505141 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:46.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9926c700 1 --2- 172.21.15.181:0/3994505141 >> v2:172.21.15.190:3300/0 conn(0x7f7a9415db70 0x7f7a940665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:46.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a8bfff700 1 -- 172.21.15.181:0/3994505141 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073794723 0 0) 0x7f7a94066c60 con 0x7f7a940cedd0 2023-10-16T11:19:46.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a8bfff700 1 -- 172.21.15.181:0/3994505141 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a800035b0 con 0x7f7a940cedd0 2023-10-16T11:19:46.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9926c700 1 -- 172.21.15.181:0/3994505141 >> v1:172.21.15.190:6789/0 conn(0x7f7a940ca2e0 legacy=0x7f7a940ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9926c700 1 -- 172.21.15.181:0/3994505141 >> v1:172.21.15.181:6789/0 conn(0x7f7a940cedd0 legacy=0x7f7a940cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9926c700 1 -- 172.21.15.181:0/3994505141 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9406b3e0 con 0x7f7a9415db70 2023-10-16T11:19:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.256+0000 7f7a9926c700 1 --2- 172.21.15.181:0/3994505141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9415db70 0x7f7a940665a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009840 tx=0x7f7a90009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1432ab89c4985d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a8bfff700 1 -- 172.21.15.181:0/3994505141 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90011830 con 0x7f7a9415db70 2023-10-16T11:19:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a8bfff700 1 -- 172.21.15.181:0/3994505141 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90011990 con 0x7f7a9415db70 2023-10-16T11:19:46.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a8bfff700 1 -- 172.21.15.181:0/3994505141 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90011d00 con 0x7f7a9415db70 2023-10-16T11:19:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/3994505141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9415db70 msgr2=0x7f7a940665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/3994505141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9415db70 0x7f7a940665a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009840 tx=0x7f7a90009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/3994505141 shutdown_connections 2023-10-16T11:19:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/3994505141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a9415db70 0x7f7a940665a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/3994505141 >> 172.21.15.181:0/3994505141 conn(0x7f7a940b1780 msgr2=0x7f7a940b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:46.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/3994505141 shutdown_connections 2023-10-16T11:19:46.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.257+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/3994505141 wait complete. 2023-10-16T11:19:46.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.258+0000 7f7a9b4d0700 1 Processor -- start 2023-10-16T11:19:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.258+0000 7f7a9b4d0700 1 -- start start 2023-10-16T11:19:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9b4d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a940ca2e0 0x7f7a940c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9b4d0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a940cedd0 0x7f7a9415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:46.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9b4d0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9415db70 0x7f7a9415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9b4d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940c2bf0 con 0x7f7a940ca2e0 2023-10-16T11:19:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9b4d0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9409deb0 con 0x7f7a940cedd0 2023-10-16T11:19:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9b4d0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94160a40 con 0x7f7a9415db70 2023-10-16T11:19:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9926c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a940ca2e0 0x7f7a940c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:46.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9926c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a940ca2e0 0x7f7a940c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55936/0 (socket says 172.21.15.181:55936) 2023-10-16T11:19:46.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a9926c700 1 -- 172.21.15.181:0/1153498491 learned_addr learned my addr 172.21.15.181:0/1153498491 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:46.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a98a6b700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a940cedd0 0x7f7a9415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:46.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.259+0000 7f7a99a6d700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9415db70 0x7f7a9415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:46.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9926c700 1 -- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9415db70 msgr2=0x7f7a9415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9926c700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9415db70 0x7f7a9415b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9926c700 1 -- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a940cedd0 msgr2=0x7f7a9415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9926c700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a940cedd0 0x7f7a9415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9926c700 1 -- 172.21.15.181:0/1153498491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a900094a0 con 0x7f7a940ca2e0 2023-10-16T11:19:46.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a99a6d700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9415db70 0x7f7a9415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:46.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9926c700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a940ca2e0 0x7f7a940c26b0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009fd0 tx=0x7f7a90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:46.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a89ffb700 1 -- 172.21.15.181:0/1153498491 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900045d0 con 0x7f7a940ca2e0 2023-10-16T11:19:46.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a89ffb700 1 -- 172.21.15.181:0/1153498491 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9001e070 con 0x7f7a940ca2e0 2023-10-16T11:19:46.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9409e150 con 0x7f7a940ca2e0 2023-10-16T11:19:46.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a89ffb700 1 -- 172.21.15.181:0/1153498491 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900038c0 con 0x7f7a940ca2e0 2023-10-16T11:19:46.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.260+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a94160d20 con 0x7f7a940ca2e0 2023-10-16T11:19:46.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.261+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a94067d20 con 0x7f7a940ca2e0 2023-10-16T11:19:46.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.262+0000 7f7a89ffb700 1 -- 172.21.15.181:0/1153498491 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7a90003a20 con 0x7f7a940ca2e0 2023-10-16T11:19:46.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.262+0000 7f7a89ffb700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f7a7806b020 0x7f7a7806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:46.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.262+0000 7f7a89ffb700 1 -- 172.21.15.181:0/1153498491 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f7a9008ac00 con 0x7f7a940ca2e0 2023-10-16T11:19:46.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.262+0000 7f7a98a6b700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f7a7806b020 0x7f7a7806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:46.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.262+0000 7f7a98a6b700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f7a7806b020 0x7f7a7806d4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a9415a970 tx=0x7f7a84007dc0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:46.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.266+0000 7f7a89ffb700 1 -- 172.21.15.181:0/1153498491 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a90091080 con 0x7f7a940ca2e0 2023-10-16T11:19:46.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.501+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a94072510 con 0x7f7a940ca2e0 2023-10-16T11:19:46.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.501+0000 7f7a89ffb700 1 -- 172.21.15.181:0/1153498491 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+6165 (secure 0 0 0) 0x7f7a90020090 con 0x7f7a940ca2e0 2023-10-16T11:19:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:46.504 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:19:46.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f7a7806b020 msgr2=0x7f7a7806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f7a7806b020 0x7f7a7806d4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a9415a970 tx=0x7f7a84007dc0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a940ca2e0 msgr2=0x7f7a940c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:46.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a940ca2e0 0x7f7a940c26b0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009fd0 tx=0x7f7a90004750 comp rx=0 tx=0).stop 2023-10-16T11:19:46.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 shutdown_connections 2023-10-16T11:19:46.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f7a7806b020 0x7f7a7806d4e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9415db70 0x7f7a9415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a940ca2e0 0x7f7a940c26b0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 --2- 172.21.15.181:0/1153498491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a940cedd0 0x7f7a9415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:46.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.503+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 >> 172.21.15.181:0/1153498491 conn(0x7f7a940b1780 msgr2=0x7f7a940b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:46.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.504+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 shutdown_connections 2023-10-16T11:19:46.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:46.504+0000 7f7a9b4d0700 1 -- 172.21.15.181:0/1153498491 wait complete. 2023-10-16T11:19:46.508 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 117 2023-10-16T11:19:46.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 117, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:19:36.016729+0000', 'modified': '2023-10-16T11:19:39.419198+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11325}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11325': {'gid': 11325, 'name': 'c', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6837/4167832945', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 4167832945}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 4167832945}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:19:46.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:19:46.921 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T11:19:47.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.041+0000 7fae7fdd1700 1 Processor -- start 2023-10-16T11:19:47.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.041+0000 7fae7fdd1700 1 -- start start 2023-10-16T11:19:47.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7fdd1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fae780ce4d0 0x7fae780cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7fdd1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae780cc4c0 0x7fae7815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7fdd1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae780cf460 con 0x7fae7806d210 2023-10-16T11:19:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7fdd1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780d0380 con 0x7fae780cc4c0 2023-10-16T11:19:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7fdd1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780cc950 con 0x7fae780ce4d0 2023-10-16T11:19:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7e36e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae780cc4c0 0x7fae7815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7db6d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fae7806d210 0x7fae780cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38790/0 (socket says 172.21.15.181:38790) 2023-10-16T11:19:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7e36e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae780cc4c0 0x7fae7815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55948/0 (socket says 172.21.15.181:55948) 2023-10-16T11:19:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7db6d700 1 -- 172.21.15.181:0/3984418898 learned_addr learned my addr 172.21.15.181:0/3984418898 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7d36c700 1 --2- 172.21.15.181:0/3984418898 >> v2:172.21.15.190:3300/0 conn(0x7fae780ce4d0 0x7fae780cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7e36e700 1 -- 172.21.15.181:0/3984418898 >> v2:172.21.15.190:3300/0 conn(0x7fae780ce4d0 msgr2=0x7fae780cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7e36e700 1 --2- 172.21.15.181:0/3984418898 >> v2:172.21.15.190:3300/0 conn(0x7fae780ce4d0 0x7fae780cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7e36e700 1 -- 172.21.15.181:0/3984418898 >> v1:172.21.15.181:6789/0 conn(0x7fae7806d210 legacy=0x7fae780cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7e36e700 1 -- 172.21.15.181:0/3984418898 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7806c700 con 0x7fae780cc4c0 2023-10-16T11:19:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.042+0000 7fae7cb6b700 1 -- 172.21.15.181:0/3984418898 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961800756 0 0) 0x7fae780cf460 con 0x7fae7806d210 2023-10-16T11:19:47.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7e36e700 1 --2- 172.21.15.181:0/3984418898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae7815d2f0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fae7806e850 tx=0x7fae7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbc2726366f87f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7cb6b700 1 -- 172.21.15.181:0/3984418898 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7000e070 con 0x7fae780cc4c0 2023-10-16T11:19:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7cb6b700 1 -- 172.21.15.181:0/3984418898 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7000bbd0 con 0x7fae780cc4c0 2023-10-16T11:19:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7cb6b700 1 -- 172.21.15.181:0/3984418898 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70012430 con 0x7fae780cc4c0 2023-10-16T11:19:47.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 -- 172.21.15.181:0/3984418898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 msgr2=0x7fae7815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/3984418898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae7815d2f0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fae7806e850 tx=0x7fae7000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 -- 172.21.15.181:0/3984418898 shutdown_connections 2023-10-16T11:19:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/3984418898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae7815d2f0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/3984418898 >> v2:172.21.15.190:3300/0 conn(0x7fae780ce4d0 0x7fae780cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 -- 172.21.15.181:0/3984418898 >> 172.21.15.181:0/3984418898 conn(0x7fae7809a9a0 msgr2=0x7fae780d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 -- 172.21.15.181:0/3984418898 shutdown_connections 2023-10-16T11:19:47.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.043+0000 7fae7fdd1700 1 -- 172.21.15.181:0/3984418898 wait complete. 2023-10-16T11:19:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 Processor -- start 2023-10-16T11:19:47.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 -- start start 2023-10-16T11:19:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae7806d210 0x7fae780ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae780aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae780ce4d0 0x7fae780ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780b6ed0 con 0x7fae780cc4c0 2023-10-16T11:19:47.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae780b6350 con 0x7fae780ce4d0 2023-10-16T11:19:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7fdd1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae780b6630 con 0x7fae7806d210 2023-10-16T11:19:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7d36c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae780aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7d36c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae780aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55952/0 (socket says 172.21.15.181:55952) 2023-10-16T11:19:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.045+0000 7fae7d36c700 1 -- 172.21.15.181:0/200970276 learned_addr learned my addr 172.21.15.181:0/200970276 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7db6d700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae7806d210 0x7fae780ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7e36e700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae780ce4d0 0x7fae780ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7d36c700 1 -- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae7806d210 msgr2=0x7fae780ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7d36c700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae7806d210 0x7fae780ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7d36c700 1 -- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae780ce4d0 msgr2=0x7fae780ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7d36c700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae780ce4d0 0x7fae780ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7d36c700 1 -- 172.21.15.181:0/200970276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae70009a70 con 0x7fae780cc4c0 2023-10-16T11:19:47.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7d36c700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae780aefb0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e810 tx=0x7fae6800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae6e7fc700 1 -- 172.21.15.181:0/200970276 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800c070 con 0x7fae780cc4c0 2023-10-16T11:19:47.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae6e7fc700 1 -- 172.21.15.181:0/200970276 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae68013610 con 0x7fae780cc4c0 2023-10-16T11:19:47.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae780b6910 con 0x7fae780cc4c0 2023-10-16T11:19:47.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae6e7fc700 1 -- 172.21.15.181:0/200970276 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680139c0 con 0x7fae780cc4c0 2023-10-16T11:19:47.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.046+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae780af4f0 con 0x7fae780cc4c0 2023-10-16T11:19:47.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.048+0000 7fae6e7fc700 1 -- 172.21.15.181:0/200970276 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fae68013b20 con 0x7fae780cc4c0 2023-10-16T11:19:47.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.048+0000 7fae6e7fc700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fae5c06b020 0x7fae5c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.048+0000 7fae6e7fc700 1 -- 172.21.15.181:0/200970276 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fae68089b10 con 0x7fae780cc4c0 2023-10-16T11:19:47.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.048+0000 7fae7db6d700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fae5c06b020 0x7fae5c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.048+0000 7fae7db6d700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fae5c06b020 0x7fae5c06d4e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae74000f60 tx=0x7fae74007e70 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.050+0000 7fae8bfbbb80 1 Processor -- start 2023-10-16T11:19:47.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.050+0000 7fae8bfbbb80 1 -- start start 2023-10-16T11:19:47.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c85334a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85334fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c853086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8527ec20 con 0x556c85332630 2023-10-16T11:19:47.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c85263c10 con 0x556c85308010 2023-10-16T11:19:47.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c85261530 con 0x556c8532d220 2023-10-16T11:19:47.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85334fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85334fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55958/0 (socket says 172.21.15.181:55958) 2023-10-16T11:19:47.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 -- 172.21.15.181:0/2176555155 learned_addr learned my addr 172.21.15.181:0/2176555155 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7db6d700 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c85334a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7e36e700 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c853086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 -- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 msgr2=0x556c85334a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c85334a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 -- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 msgr2=0x556c853086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c853086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7d36c700 1 -- 172.21.15.181:0/2176555155 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c8521a3a0 con 0x556c85332630 2023-10-16T11:19:47.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.051+0000 7fae7e36e700 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c853086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:47.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae7d36c700 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85334fb0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e7e0 tx=0x7fae68057b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca390bf9a5a86f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae5b7fe700 1 -- 172.21.15.181:0/2176555155 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68058340 con 0x556c85332630 2023-10-16T11:19:47.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae5b7fe700 1 -- 172.21.15.181:0/2176555155 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae680584a0 con 0x556c85332630 2023-10-16T11:19:47.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae5b7fe700 1 -- 172.21.15.181:0/2176555155 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68058610 con 0x556c85332630 2023-10-16T11:19:47.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2176555155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 msgr2=0x556c85334fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85334fb0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e7e0 tx=0x7fae68057b70 comp rx=0 tx=0).stop 2023-10-16T11:19:47.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2176555155 shutdown_connections 2023-10-16T11:19:47.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c85334a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c853086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2176555155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85334fb0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2176555155 >> 172.21.15.181:0/2176555155 conn(0x556c85237d80 msgr2=0x556c85352ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2176555155 shutdown_connections 2023-10-16T11:19:47.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.052+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2176555155 wait complete. 2023-10-16T11:19:47.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.053+0000 7fae8bfbbb80 1 -- start start 2023-10-16T11:19:47.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.053+0000 7fae8bfbbb80 1 Processor -- start 2023-10-16T11:19:47.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c8532fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c8532d220 0x556c8532adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c85332630 0x556c852fef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8525ce20 con 0x556c85308010 2023-10-16T11:19:47.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c851f96c0 con 0x556c8532d220 2023-10-16T11:19:47.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c851f9360 con 0x556c85332630 2023-10-16T11:19:47.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae8bfbbb80 10 client.0 _ll_register_callbacks cb 0x556c850567b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:19:47.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c8532fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c8532fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55964/0 (socket says 172.21.15.181:55964) 2023-10-16T11:19:47.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 -- 172.21.15.181:0/4084985318 learned_addr learned my addr 172.21.15.181:0/4084985318 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7d36c700 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c8532d220 0x556c8532adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7e36e700 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c85332630 0x556c852fef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 -- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c85332630 msgr2=0x556c852fef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c85332630 0x556c852fef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 -- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c8532d220 msgr2=0x556c8532adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c8532d220 0x556c8532adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.054+0000 7fae7db6d700 1 -- 172.21.15.181:0/4084985318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae780af4f0 con 0x556c85308010 2023-10-16T11:19:47.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae7db6d700 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c8532fb90 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fae74001340 tx=0x7fae74000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6103f20e1002cdce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:19:47.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae58ff9700 1 -- 172.21.15.181:0/4084985318 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740134e0 con 0x556c85308010 2023-10-16T11:19:47.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae58ff9700 1 -- 172.21.15.181:0/4084985318 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fae7401b070 con 0x556c85308010 2023-10-16T11:19:47.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c852fd5b0 con 0x556c85308010 2023-10-16T11:19:47.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae58ff9700 1 -- 172.21.15.181:0/4084985318 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae74001990 con 0x556c85308010 2023-10-16T11:19:47.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 10 client.11649 fetch_fsmap learned FSMap version 117 2023-10-16T11:19:47.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x556c85087d90 con 0x556c85308010 2023-10-16T11:19:47.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae58ff9700 1 -- 172.21.15.181:0/4084985318 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c852fd5b0 con 0x556c85308010 2023-10-16T11:19:47.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae58ff9700 1 -- 172.21.15.181:0/4084985318 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 117) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fae74001af0 con 0x556c85308010 2023-10-16T11:19:47.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 10 client.11649 fetch_fsmap finished waiting for FSMap version 117 2023-10-16T11:19:47.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 10 client.11649 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:19:47.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 20 client.11649 populate_metadata read hostname 'smithi181' 2023-10-16T11:19:47.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c853360f0 0x556c8536c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 4 client.11649 mds_command: new command op to 11325 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:19:47.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 --> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c85255e00 con 0x556c853360f0 2023-10-16T11:19:47.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.055+0000 7fae7d36c700 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c853360f0 0x556c8536c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae58ff9700 10 client.11649 ms_handle_connect on v2:172.21.15.181:6835/4167832945 2023-10-16T11:19:47.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae7d36c700 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c853360f0 0x556c8536c5d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25562f0464f94cc0 server_cookie=c064dd09ba471715 in_seq=0 out_seq=0 2023-10-16T11:19:47.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae58ff9700 1 -- 172.21.15.181:0/4084985318 <== mds.0 v2:172.21.15.181:6835/4167832945 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c85255e00 con 0x556c853360f0 2023-10-16T11:19:47.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae58ff9700 10 client.11649 handle_command_reply: tid=0 2023-10-16T11:19:47.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 client.11649 shutdown 2023-10-16T11:19:47.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 msgr2=0x556c8532fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c8532fb90 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fae74001340 tx=0x7fae74000ed0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 shutdown_connections 2023-10-16T11:19:47.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c853360f0 0x556c8536c5d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c85332630 0x556c852fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c8532fb90 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae58ff9700 0 client.11649 ms_handle_reset on v2:172.21.15.181:6835/4167832945 2023-10-16T11:19:47.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/4084985318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c8532d220 0x556c8532adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.056+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 >> 172.21.15.181:0/4084985318 conn(0x556c85237d80 msgr2=0x556c85308c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.057+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 shutdown_connections 2023-10-16T11:19:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.057+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/4084985318 wait complete. 2023-10-16T11:19:47.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.057+0000 7fae8bfbbb80 20 client.11649 trim_cache size 0 max 16384 2023-10-16T11:19:47.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.074+0000 7fae8bfbbb80 1 Processor -- start 2023-10-16T11:19:47.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.074+0000 7fae8bfbbb80 1 -- start start 2023-10-16T11:19:47.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c8532ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c850dd8d0 con 0x556c85332630 2023-10-16T11:19:47.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8534f440 con 0x556c85308010 2023-10-16T11:19:47.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c852fd3e0 con 0x556c8532d220 2023-10-16T11:19:47.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55974/0 (socket says 172.21.15.181:55974) 2023-10-16T11:19:47.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 -- 172.21.15.181:0/23108910 learned_addr learned my addr 172.21.15.181:0/23108910 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7db6d700 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c8532ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7d36c700 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 -- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 msgr2=0x556c853705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 -- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 msgr2=0x556c8532ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c8532ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 -- 172.21.15.181:0/23108910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c85070c10 con 0x556c85332630 2023-10-16T11:19:47.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7d36c700 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:47.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7db6d700 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c8532ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:47.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.075+0000 7fae7e36e700 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85370b30 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b730 tx=0x7fae70011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaffd7d6d97e680 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae5bfff700 1 -- 172.21.15.181:0/23108910 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae700125e0 con 0x556c85332630 2023-10-16T11:19:47.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae5bfff700 1 -- 172.21.15.181:0/23108910 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70022040 con 0x556c85332630 2023-10-16T11:19:47.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae5bfff700 1 -- 172.21.15.181:0/23108910 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70012940 con 0x556c85332630 2023-10-16T11:19:47.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/23108910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 msgr2=0x556c85370b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85370b30 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b730 tx=0x7fae70011000 comp rx=0 tx=0).stop 2023-10-16T11:19:47.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/23108910 shutdown_connections 2023-10-16T11:19:47.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85308010 0x556c8532ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/23108910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/23108910 >> 172.21.15.181:0/23108910 conn(0x556c85237d80 msgr2=0x556c8530ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/23108910 shutdown_connections 2023-10-16T11:19:47.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.076+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/23108910 wait complete. 2023-10-16T11:19:47.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.077+0000 7fae8bfbbb80 1 -- start start 2023-10-16T11:19:47.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.077+0000 7fae8bfbbb80 1 Processor -- start 2023-10-16T11:19:47.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c853b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae8bfbbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c852fead0 con 0x556c85308010 2023-10-16T11:19:47.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c8534a250 con 0x556c85332630 2023-10-16T11:19:47.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae8bfbbb80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c85304c30 con 0x556c8532d220 2023-10-16T11:19:47.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae8bfbbb80 10 client.0 _ll_register_callbacks cb 0x7fae74013cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:19:47.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae7db6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c853b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae7db6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c853b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55978/0 (socket says 172.21.15.181:55978) 2023-10-16T11:19:47.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae7db6d700 1 -- 172.21.15.181:0/2607758910 learned_addr learned my addr 172.21.15.181:0/2607758910 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae7d36c700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.078+0000 7fae7e36e700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7db6d700 1 -- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 msgr2=0x556c853705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7db6d700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7db6d700 1 -- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85332630 msgr2=0x556c85370b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7db6d700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7db6d700 1 -- 172.21.15.181:0/2607758910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7806c700 con 0x556c85308010 2023-10-16T11:19:47.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7e36e700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:47.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7db6d700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c853b0110 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fae74004fd0 tx=0x7fae7400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d838d7ff97b1fe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:19:47.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae58ff9700 1 -- 172.21.15.181:0/2607758910 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740134e0 con 0x556c85308010 2023-10-16T11:19:47.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae58ff9700 1 -- 172.21.15.181:0/2607758910 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fae74028e40 con 0x556c85308010 2023-10-16T11:19:47.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae680584a0 con 0x556c85308010 2023-10-16T11:19:47.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae58ff9700 1 -- 172.21.15.181:0/2607758910 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae7403f070 con 0x556c85308010 2023-10-16T11:19:47.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 10 client.11655 fetch_fsmap learned FSMap version 117 2023-10-16T11:19:47.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x556c850ae200 con 0x556c85308010 2023-10-16T11:19:47.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae58ff9700 1 -- 172.21.15.181:0/2607758910 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae680584a0 con 0x556c85308010 2023-10-16T11:19:47.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae58ff9700 1 -- 172.21.15.181:0/2607758910 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 117) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fae74028660 con 0x556c85308010 2023-10-16T11:19:47.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 10 client.11655 fetch_fsmap finished waiting for FSMap version 117 2023-10-16T11:19:47.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 10 client.11655 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T11:19:47.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 20 client.11655 populate_metadata read hostname 'smithi181' 2023-10-16T11:19:47.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c8534db30 0x556c8534df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 4 client.11655 mds_command: new command op to 11325 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:19:47.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 --> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556c85255b20 con 0x556c8534db30 2023-10-16T11:19:47.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.079+0000 7fae7d36c700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c8534db30 0x556c8534df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae58ff9700 10 client.11655 ms_handle_connect on v2:172.21.15.181:6835/4167832945 2023-10-16T11:19:47.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae7d36c700 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c8534db30 0x556c8534df10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fb8aa25aa9f606 server_cookie=20c259176bc52a8c in_seq=0 out_seq=0 2023-10-16T11:19:47.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae58ff9700 1 -- 172.21.15.181:0/2607758910 <== mds.0 v2:172.21.15.181:6835/4167832945 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+338 (crc 0 0 0) 0x556c85255b20 con 0x556c8534db30 2023-10-16T11:19:47.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae58ff9700 10 client.11655 handle_command_reply: tid=0 2023-10-16T11:19:47.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 client.11655 shutdown 2023-10-16T11:19:47.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 msgr2=0x556c853b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c853b0110 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fae74004fd0 tx=0x7fae7400a000 comp rx=0 tx=0).stop 2023-10-16T11:19:47.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 shutdown_connections 2023-10-16T11:19:47.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:6835/4167832945,v1:172.21.15.181:6837/4167832945] conn(0x556c8534db30 0x556c8534df10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556c85332630 0x556c85370b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae58ff9700 0 client.11655 ms_handle_reset on v2:172.21.15.181:6835/4167832945 2023-10-16T11:19:47.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c85308010 0x556c853b0110 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 --2- 172.21.15.181:0/2607758910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556c8532d220 0x556c853705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 >> 172.21.15.181:0/2607758910 conn(0x556c85237d80 msgr2=0x556c85331bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 shutdown_connections 2023-10-16T11:19:47.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 1 -- 172.21.15.181:0/2607758910 wait complete. 2023-10-16T11:19:47.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.080+0000 7fae8bfbbb80 20 client.11655 trim_cache size 0 max 16384 2023-10-16T11:19:47.105 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:19:47.106 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:19:47.106 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:19:47.106 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 11325, 2023-10-16T11:19:47.106 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:19:47.107 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:19:47.107 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "backup_fs", 2023-10-16T11:19:47.107 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 10.067432736000001, 2023-10-16T11:19:47.107 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 117, 2023-10-16T11:19:47.107 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 220, 2023-10-16T11:19:47.108 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 220, 2023-10-16T11:19:47.108 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 28.496950997999999 2023-10-16T11:19:47.108 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:19:47.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fae5c06b020 msgr2=0x7fae5c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fae5c06b020 0x7fae5c06d4e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae74000f60 tx=0x7fae74007e70 comp rx=0 tx=0).stop 2023-10-16T11:19:47.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 msgr2=0x7fae780aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae780aefb0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fae6800e810 tx=0x7fae6800eb20 comp rx=0 tx=0).stop 2023-10-16T11:19:47.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 shutdown_connections 2023-10-16T11:19:47.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7fae5c06b020 0x7fae5c06d4e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae7806d210 0x7fae780ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae780ce4d0 0x7fae780ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 --2- 172.21.15.181:0/200970276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cc4c0 0x7fae780aefb0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 >> 172.21.15.181:0/200970276 conn(0x7fae7809a9a0 msgr2=0x7fae78070640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 shutdown_connections 2023-10-16T11:19:47.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.081+0000 7fae7fdd1700 1 -- 172.21.15.181:0/200970276 wait complete. 2023-10-16T11:19:47.112 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:19:47.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.231+0000 7f6557edb700 1 Processor -- start 2023-10-16T11:19:47.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.231+0000 7f6557edb700 1 -- start start 2023-10-16T11:19:47.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6557edb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65500d0fc0 0x7f65500cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6557edb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f655006a8a0 0x7f655006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6557edb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65500cddf0 con 0x7f655015df60 2023-10-16T11:19:47.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6557edb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655006b290 con 0x7f65500d0fc0 2023-10-16T11:19:47.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6557edb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65500af9c0 con 0x7f655006a8a0 2023-10-16T11:19:47.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6555476700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65500d0fc0 0x7f65500cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6555476700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65500d0fc0 0x7f65500cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55990/0 (socket says 172.21.15.181:55990) 2023-10-16T11:19:47.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6555476700 1 -- 172.21.15.181:0/1793346860 learned_addr learned my addr 172.21.15.181:0/1793346860 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6556478700 1 --2- 172.21.15.181:0/1793346860 >> v2:172.21.15.190:3300/0 conn(0x7f655006a8a0 0x7f655006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6555476700 1 -- 172.21.15.181:0/1793346860 >> v2:172.21.15.190:3300/0 conn(0x7f655006a8a0 msgr2=0x7f655006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6555476700 1 --2- 172.21.15.181:0/1793346860 >> v2:172.21.15.190:3300/0 conn(0x7f655006a8a0 0x7f655006ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6555476700 1 -- 172.21.15.181:0/1793346860 >> v1:172.21.15.190:6789/0 conn(0x7f655015df60 legacy=0x7f65500cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.232+0000 7f6555476700 1 -- 172.21.15.181:0/1793346860 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500ccf50 con 0x7f65500d0fc0 2023-10-16T11:19:47.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6555476700 1 --2- 172.21.15.181:0/1793346860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f65500cae20 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f655006f9e0 tx=0x7f654c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bccf602eb230409 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6554c75700 1 -- 172.21.15.181:0/1793346860 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c004230 con 0x7f65500d0fc0 2023-10-16T11:19:47.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6554c75700 1 -- 172.21.15.181:0/1793346860 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c004390 con 0x7f65500d0fc0 2023-10-16T11:19:47.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6554c75700 1 -- 172.21.15.181:0/1793346860 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0036a0 con 0x7f65500d0fc0 2023-10-16T11:19:47.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 -- 172.21.15.181:0/1793346860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 msgr2=0x7f65500cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 --2- 172.21.15.181:0/1793346860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f65500cae20 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f655006f9e0 tx=0x7f654c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 -- 172.21.15.181:0/1793346860 shutdown_connections 2023-10-16T11:19:47.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 --2- 172.21.15.181:0/1793346860 >> v2:172.21.15.190:3300/0 conn(0x7f655006a8a0 0x7f655006ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 --2- 172.21.15.181:0/1793346860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f65500cae20 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 -- 172.21.15.181:0/1793346860 >> 172.21.15.181:0/1793346860 conn(0x7f65500d04a0 msgr2=0x7f65500d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 -- 172.21.15.181:0/1793346860 shutdown_connections 2023-10-16T11:19:47.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.233+0000 7f6557edb700 1 -- 172.21.15.181:0/1793346860 wait complete. 2023-10-16T11:19:47.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.234+0000 7f6557edb700 1 Processor -- start 2023-10-16T11:19:47.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6557edb700 1 -- start start 2023-10-16T11:19:47.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6557edb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f655006a8a0 0x7f65500b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6557edb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f6550159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6557edb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f655015df60 0x7f65500b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6557edb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500b52f0 con 0x7f65500d0fc0 2023-10-16T11:19:47.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6557edb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500aff00 con 0x7f655015df60 2023-10-16T11:19:47.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6557edb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65501f3b20 con 0x7f655006a8a0 2023-10-16T11:19:47.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6555476700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f6550159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6555476700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f6550159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55992/0 (socket says 172.21.15.181:55992) 2023-10-16T11:19:47.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6555476700 1 -- 172.21.15.181:0/2533439152 learned_addr learned my addr 172.21.15.181:0/2533439152 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6555c77700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f655006a8a0 0x7f65500b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.235+0000 7f6556478700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f655015df60 0x7f65500b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6555476700 1 -- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f655006a8a0 msgr2=0x7f65500b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6555476700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f655006a8a0 0x7f65500b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6555476700 1 -- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f655015df60 msgr2=0x7f65500b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6555476700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f655015df60 0x7f65500b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6555476700 1 -- 172.21.15.181:0/2533439152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c009550 con 0x7f65500d0fc0 2023-10-16T11:19:47.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6555476700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f6550159bd0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f654c009130 tx=0x7f654c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f65467fc700 1 -- 172.21.15.181:0/2533439152 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c003d60 con 0x7f65500d0fc0 2023-10-16T11:19:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f65467fc700 1 -- 172.21.15.181:0/2533439152 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c011630 con 0x7f65500d0fc0 2023-10-16T11:19:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65501f3fb0 con 0x7f65500d0fc0 2023-10-16T11:19:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f65467fc700 1 -- 172.21.15.181:0/2533439152 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0119a0 con 0x7f65500d0fc0 2023-10-16T11:19:47.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.236+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65501f34d0 con 0x7f65500d0fc0 2023-10-16T11:19:47.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.237+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6550067d20 con 0x7f65500d0fc0 2023-10-16T11:19:47.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.237+0000 7f65467fc700 1 -- 172.21.15.181:0/2533439152 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f654c021070 con 0x7f65500d0fc0 2023-10-16T11:19:47.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.238+0000 7f65467fc700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f653406b020 0x7f653406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.238+0000 7f65467fc700 1 -- 172.21.15.181:0/2533439152 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f654c08b0e0 con 0x7f65500d0fc0 2023-10-16T11:19:47.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.238+0000 7f6555c77700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f653406b020 0x7f653406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.238+0000 7f6555c77700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f653406b020 0x7f653406d4e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f65400090e0 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.242+0000 7f65467fc700 1 -- 172.21.15.181:0/2533439152 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f654c055b80 con 0x7f65500d0fc0 2023-10-16T11:19:47.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.461+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f65500cc8a0 con 0x7f653406b020 2023-10-16T11:19:47.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.463+0000 7f65467fc700 1 -- 172.21.15.181:0/2533439152 <== mgr.11079 v2:172.21.15.181:6832/1781343309 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f6540009e20 con 0x7f653406b020 2023-10-16T11:19:47.465 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:19:47.465 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:47.466 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T11:19:47.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:19:47.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:19:47.466 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "backup_fs" 2023-10-16T11:19:47.467 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:19:47.467 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:19:47.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f653406b020 msgr2=0x7f653406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f653406b020 0x7f653406d4e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f65400090e0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 msgr2=0x7f6550159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f6550159bd0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f654c009130 tx=0x7f654c004750 comp rx=0 tx=0).stop 2023-10-16T11:19:47.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 shutdown_connections 2023-10-16T11:19:47.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f653406b020 0x7f653406d4e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f655006a8a0 0x7f65500b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f655015df60 0x7f65500b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 --2- 172.21.15.181:0/2533439152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65500d0fc0 0x7f6550159bd0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 >> 172.21.15.181:0/2533439152 conn(0x7f65500d04a0 msgr2=0x7f655015bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.465+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 shutdown_connections 2023-10-16T11:19:47.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.466+0000 7f6557edb700 1 -- 172.21.15.181:0/2533439152 wait complete. 2023-10-16T11:19:47.480 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:19:47.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.603+0000 7f00904e2700 1 Processor -- start 2023-10-16T11:19:47.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.603+0000 7f00904e2700 1 -- start start 2023-10-16T11:19:47.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.604+0000 7f00904e2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f00880ce2d0 0x7f008815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.604+0000 7f00904e2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008815ef80 con 0x7f00880d1500 2023-10-16T11:19:47.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.604+0000 7f00904e2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00880cf580 con 0x7f00880d01d0 2023-10-16T11:19:47.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.604+0000 7f00904e2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00880d4290 con 0x7f00880ce2d0 2023-10-16T11:19:47.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.604+0000 7f008ea7f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f00880d1500 0x7f00880d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38848/0 (socket says 172.21.15.181:38848) 2023-10-16T11:19:47.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008ea7f700 1 -- 172.21.15.181:0/3855739435 learned_addr learned my addr 172.21.15.181:0/3855739435 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:47.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008da7d700 1 --2- 172.21.15.181:0/3855739435 >> v2:172.21.15.190:3300/0 conn(0x7f00880ce2d0 0x7f008815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373105572 0 0) 0x7f0080001490 con 0x7f00880d1500 2023-10-16T11:19:47.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00740035b0 con 0x7f00880d1500 2023-10-16T11:19:47.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480038484 0 0) 0x7f00740035b0 con 0x7f00880d1500 2023-10-16T11:19:47.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 >> v2:172.21.15.190:3300/0 conn(0x7f00880ce2d0 msgr2=0x7f008815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 --2- 172.21.15.181:0/3855739435 >> v2:172.21.15.190:3300/0 conn(0x7f00880ce2d0 0x7f008815ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 >> v1:172.21.15.190:6789/0 conn(0x7f00880d01d0 legacy=0x7f00880d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880ce970 con 0x7f00880d1500 2023-10-16T11:19:47.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f0080001b80 con 0x7f00880d1500 2023-10-16T11:19:47.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 >> v1:172.21.15.181:6789/0 conn(0x7f00880d1500 legacy=0x7f00880d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0074003a00 0x7f0074003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0074004400 0x7f00740048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00740050d0 0x7f007400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007400c000 con 0x7f0074004400 2023-10-16T11:19:47.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007400c170 con 0x7f00740050d0 2023-10-16T11:19:47.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007400c330 con 0x7f0074003a00 2023-10-16T11:19:47.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008da7d700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0074004400 0x7f00740048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008ea7f700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00740050d0 0x7f007400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.605+0000 7f008e27e700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0074003a00 0x7f0074003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008da7d700 1 -- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0074003a00 msgr2=0x7f0074003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008da7d700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0074003a00 0x7f0074003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008da7d700 1 -- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00740050d0 msgr2=0x7f007400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:47.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008da7d700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00740050d0 0x7f007400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008ea7f700 1 -- 172.21.15.181:0/3855739435 reap_dead start 2023-10-16T11:19:47.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008da7d700 1 -- 172.21.15.181:0/3855739435 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740035b0 con 0x7f0074004400 2023-10-16T11:19:47.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008da7d700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0074004400 0x7f00740048d0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f0078009b60 tx=0x7f0074005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5231ede74f30ebd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800e6c0 con 0x7f0074004400 2023-10-16T11:19:47.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007800e820 con 0x7f0074004400 2023-10-16T11:19:47.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f008d27c700 1 -- 172.21.15.181:0/3855739435 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800eb90 con 0x7f0074004400 2023-10-16T11:19:47.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f00904e2700 1 -- 172.21.15.181:0/3855739435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0074004400 msgr2=0x7f00740048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f00904e2700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0074004400 0x7f00740048d0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f0078009b60 tx=0x7f0074005880 comp rx=0 tx=0).stop 2023-10-16T11:19:47.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f00904e2700 1 -- 172.21.15.181:0/3855739435 shutdown_connections 2023-10-16T11:19:47.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f00904e2700 1 --2- 172.21.15.181:0/3855739435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0074004400 0x7f00740048d0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f00904e2700 1 -- 172.21.15.181:0/3855739435 >> 172.21.15.181:0/3855739435 conn(0x7f008809cb60 msgr2=0x7f00880d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:47.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f00904e2700 1 -- 172.21.15.181:0/3855739435 shutdown_connections 2023-10-16T11:19:47.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.606+0000 7f00904e2700 1 -- 172.21.15.181:0/3855739435 wait complete. 2023-10-16T11:19:47.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 Processor -- start 2023-10-16T11:19:47.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 -- start start 2023-10-16T11:19:47.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0074004400 0x7f00880b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00880b2390 0x7f00880c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00880c7de0 0x7f00880b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880b6100 con 0x7f00880b2390 2023-10-16T11:19:47.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880b5880 con 0x7f0074004400 2023-10-16T11:19:47.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f00904e2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00880c9d80 con 0x7f00880c7de0 2023-10-16T11:19:47.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f008da7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00880b2390 0x7f00880c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f008da7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00880b2390 0x7f00880c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56012/0 (socket says 172.21.15.181:56012) 2023-10-16T11:19:47.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f008da7d700 1 -- 172.21.15.181:0/4281191671 learned_addr learned my addr 172.21.15.181:0/4281191671 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:47.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.608+0000 7f008e27e700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0074004400 0x7f00880b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f008ea7f700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00880c7de0 0x7f00880b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f008da7d700 1 -- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00880c7de0 msgr2=0x7f00880b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f008da7d700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00880c7de0 0x7f00880b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f008da7d700 1 -- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0074004400 msgr2=0x7f00880b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:47.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f008da7d700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0074004400 0x7f00880b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:47.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f008da7d700 1 -- 172.21.15.181:0/4281191671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880b5b40 con 0x7f00880b2390 2023-10-16T11:19:47.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f008da7d700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00880b2390 0x7f00880c78a0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f007800a010 tx=0x7f00780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f007effd700 1 -- 172.21.15.181:0/4281191671 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800e6c0 con 0x7f00880b2390 2023-10-16T11:19:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f007effd700 1 -- 172.21.15.181:0/4281191671 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0078003ad0 con 0x7f00880b2390 2023-10-16T11:19:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00880b6100 con 0x7f00880b2390 2023-10-16T11:19:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f007effd700 1 -- 172.21.15.181:0/4281191671 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078003e80 con 0x7f00880b2390 2023-10-16T11:19:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.609+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00880c70a0 con 0x7f00880b2390 2023-10-16T11:19:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.610+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008804ebb0 con 0x7f00880b2390 2023-10-16T11:19:47.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.610+0000 7f007effd700 1 -- 172.21.15.181:0/4281191671 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 139) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0078013470 con 0x7f00880b2390 2023-10-16T11:19:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.611+0000 7f007effd700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f006c06b020 0x7f006c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.611+0000 7f007effd700 1 -- 172.21.15.181:0/4281191671 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f007808b5b0 con 0x7f00880b2390 2023-10-16T11:19:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.611+0000 7f008e27e700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f006c06b020 0x7f006c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.611+0000 7f008e27e700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f006c06b020 0x7f006c06d4e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0084000c00 tx=0x7f0084009040 comp rx=0 tx=0).ready entity=mgr.11079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:47.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.615+0000 7f007effd700 1 -- 172.21.15.181:0/4281191671 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0078059c00 con 0x7f00880b2390 2023-10-16T11:19:47.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:47.861+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f00880cf200 con 0x7f00880b2390 2023-10-16T11:19:48.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.045+0000 7f007effd700 1 -- 172.21.15.181:0/4281191671 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v140) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0078059790 con 0x7f00880b2390 2023-10-16T11:19:48.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.048+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f006c06b020 msgr2=0x7f006c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.048+0000 7f00904e2700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f006c06b020 0x7f006c06d4e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0084000c00 tx=0x7f0084009040 comp rx=0 tx=0).stop 2023-10-16T11:19:48.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.048+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00880b2390 msgr2=0x7f00880c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00880b2390 0x7f00880c78a0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f007800a010 tx=0x7f00780047c0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 shutdown_connections 2023-10-16T11:19:48.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f006c06b020 0x7f006c06d4e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00880c7de0 0x7f00880b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0074004400 0x7f00880b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 --2- 172.21.15.181:0/4281191671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00880b2390 0x7f00880c78a0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 >> 172.21.15.181:0/4281191671 conn(0x7f008809cb60 msgr2=0x7f008815ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:48.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 shutdown_connections 2023-10-16T11:19:48.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.049+0000 7f00904e2700 1 -- 172.21.15.181:0/4281191671 wait complete. 2023-10-16T11:19:48.065 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:19:48.188 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:48.187+0000 7f3cceea9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:19:48.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.191+0000 7f6f17e38700 1 Processor -- start 2023-10-16T11:19:48.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.192+0000 7f6f17e38700 1 -- start start 2023-10-16T11:19:48.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.192+0000 7f6f17e38700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6f1015f150 0x7f6f100cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.192+0000 7f6f17e38700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f100cd170 0x7f6f100b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f17e38700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f100b2ae0 con 0x7f6f100ce4f0 2023-10-16T11:19:48.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f17e38700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f100ce9a0 con 0x7f6f100cd170 2023-10-16T11:19:48.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f17e38700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f100b2dc0 con 0x7f6f1015f150 2023-10-16T11:19:48.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f100cd170 0x7f6f100b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f100cd170 0x7f6f100b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56050/0 (socket says 172.21.15.181:56050) 2023-10-16T11:19:48.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 -- 172.21.15.181:0/2285369419 learned_addr learned my addr 172.21.15.181:0/2285369419 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:48.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f15bd4700 1 --2- 172.21.15.181:0/2285369419 >> v2:172.21.15.190:3300/0 conn(0x7f6f1015f150 0x7f6f100cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 -- 172.21.15.181:0/2285369419 >> v2:172.21.15.190:3300/0 conn(0x7f6f1015f150 msgr2=0x7f6f100cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:48.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 --2- 172.21.15.181:0/2285369419 >> v2:172.21.15.190:3300/0 conn(0x7f6f1015f150 0x7f6f100cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 -- 172.21.15.181:0/2285369419 >> v1:172.21.15.190:6789/0 conn(0x7f6f100ce4f0 legacy=0x7f6f100b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 -- 172.21.15.181:0/2285369419 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f100cf9d0 con 0x7f6f100cd170 2023-10-16T11:19:48.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f163d5700 1 --2- 172.21.15.181:0/2285369419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f100cd170 0x7f6f100b1670 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009e50 tx=0x7f6f0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cb1fc667fc18b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:48.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f14bd2700 1 -- 172.21.15.181:0/2285369419 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800e070 con 0x7f6f100cd170 2023-10-16T11:19:48.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f14bd2700 1 -- 172.21.15.181:0/2285369419 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0800b8d0 con 0x7f6f100cd170 2023-10-16T11:19:48.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.193+0000 7f6f14bd2700 1 -- 172.21.15.181:0/2285369419 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800bc80 con 0x7f6f100cd170 2023-10-16T11:19:48.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 -- 172.21.15.181:0/2285369419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f100cd170 msgr2=0x7f6f100b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 --2- 172.21.15.181:0/2285369419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f100cd170 0x7f6f100b1670 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009e50 tx=0x7f6f0800b330 comp rx=0 tx=0).stop 2023-10-16T11:19:48.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 -- 172.21.15.181:0/2285369419 shutdown_connections 2023-10-16T11:19:48.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 --2- 172.21.15.181:0/2285369419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f100cd170 0x7f6f100b1670 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 --2- 172.21.15.181:0/2285369419 >> v2:172.21.15.190:3300/0 conn(0x7f6f1015f150 0x7f6f100cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 -- 172.21.15.181:0/2285369419 >> 172.21.15.181:0/2285369419 conn(0x7f6f1009a930 msgr2=0x7f6f100d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:48.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 -- 172.21.15.181:0/2285369419 shutdown_connections 2023-10-16T11:19:48.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.194+0000 7f6f17e38700 1 -- 172.21.15.181:0/2285369419 wait complete. 2023-10-16T11:19:48.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.196+0000 7f6f17e38700 1 Processor -- start 2023-10-16T11:19:48.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.196+0000 7f6f17e38700 1 -- start start 2023-10-16T11:19:48.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f17e38700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f100cd170 0x7f6f1015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f17e38700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f100ce4f0 0x7f6f10158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f17e38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f1015f150 0x7f6f101e5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f17e38700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1015ce20 con 0x7f6f1015f150 2023-10-16T11:19:48.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f17e38700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100c74f0 con 0x7f6f100ce4f0 2023-10-16T11:19:48.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f17e38700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1015c440 con 0x7f6f100cd170 2023-10-16T11:19:48.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f163d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f1015f150 0x7f6f101e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f163d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f1015f150 0x7f6f101e5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56056/0 (socket says 172.21.15.181:56056) 2023-10-16T11:19:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f163d5700 1 -- 172.21.15.181:0/3960155405 learned_addr learned my addr 172.21.15.181:0/3960155405 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.197+0000 7f6f153d3700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f100ce4f0 0x7f6f10158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f15bd4700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f100cd170 0x7f6f1015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f163d5700 1 -- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f100cd170 msgr2=0x7f6f1015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f163d5700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f100cd170 0x7f6f1015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f163d5700 1 -- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f100ce4f0 msgr2=0x7f6f10158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f163d5700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f100ce4f0 0x7f6f10158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f163d5700 1 -- 172.21.15.181:0/3960155405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08009b30 con 0x7f6f1015f150 2023-10-16T11:19:48.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f163d5700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f1015f150 0x7f6f101e5290 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f6f1006b3a0 tx=0x7f6f08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f067fc700 1 -- 172.21.15.181:0/3960155405 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08004800 con 0x7f6f1015f150 2023-10-16T11:19:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f067fc700 1 -- 172.21.15.181:0/3960155405 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f080092e0 con 0x7f6f1015f150 2023-10-16T11:19:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f17e38700 1 -- 172.21.15.181:0/3960155405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f100c7980 con 0x7f6f1015f150 2023-10-16T11:19:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f067fc700 1 -- 172.21.15.181:0/3960155405 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0801b670 con 0x7f6f1015f150 2023-10-16T11:19:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.198+0000 7f6f17e38700 1 -- 172.21.15.181:0/3960155405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f100c6bc0 con 0x7f6f1015f150 2023-10-16T11:19:48.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.200+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec0052f0 con 0x7f6f1015f150 2023-10-16T11:19:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.200+0000 7f6f067fc700 1 -- 172.21.15.181:0/3960155405 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 140) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f6f0801b7d0 con 0x7f6f1015f150 2023-10-16T11:19:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.200+0000 7f6f067fc700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 0x7f6ef406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.200+0000 7f6f15bd4700 1 -- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 msgr2=0x7f6ef406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.200+0000 7f6f067fc700 1 -- 172.21.15.181:0/3960155405 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f6f0808b630 con 0x7f6f1015f150 2023-10-16T11:19:48.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.200+0000 7f6f15bd4700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 0x7f6ef406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:48.210 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:48.204+0000 7f79406f3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:19:48.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.205+0000 7f6f067fc700 1 -- 172.21.15.181:0/3960155405 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6f08056090 con 0x7f6f1015f150 2023-10-16T11:19:48.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:48.253+0000 7f3cceea9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:48.274 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:48.273+0000 7f79406f3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:48.327 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:48.326+0000 7f3cceea9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:19:48.348 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:48.346+0000 7f79406f3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:19:48.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.401+0000 7f6f15bd4700 1 -- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 msgr2=0x7f6ef406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:48.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.401+0000 7f6f15bd4700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 0x7f6ef406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:48.404 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:48.402+0000 7f3cceea9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:19:48.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.405+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f6eec0055a0 con 0x7f6f1015f150 2023-10-16T11:19:48.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.410+0000 7f6f067fc700 1 -- 172.21.15.181:0/3960155405 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6f080596b0 con 0x7f6f1015f150 2023-10-16T11:19:48.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 msgr2=0x7f6ef406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:48.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 0x7f6ef406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f1015f150 msgr2=0x7f6f101e5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f1015f150 0x7f6f101e5290 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f6f1006b3a0 tx=0x7f6f08004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 shutdown_connections 2023-10-16T11:19:48.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f6ef406b070 0x7f6ef406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f100cd170 0x7f6f1015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f1015f150 0x7f6f101e5290 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 --2- 172.21.15.181:0/3960155405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f100ce4f0 0x7f6f10158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 >> 172.21.15.181:0/3960155405 conn(0x7f6f1009a930 msgr2=0x7f6f100b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:48.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 shutdown_connections 2023-10-16T11:19:48.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.413+0000 7f6ef3fff700 1 -- 172.21.15.181:0/3960155405 wait complete. 2023-10-16T11:19:48.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:19:48.428 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:48.427+0000 7f79406f3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:19:48.544 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:48.542+0000 7f3cceea9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:19:48.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.552+0000 7f8f074bc700 1 Processor -- start 2023-10-16T11:19:48.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.552+0000 7f8f074bc700 1 -- start start 2023-10-16T11:19:48.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f074bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0015da70 0x7f8f000ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f074bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f000ce6a0 0x7f8f0006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f074bc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f000cef00 con 0x7f8f00072760 2023-10-16T11:19:48.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f074bc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f000cec20 con 0x7f8f000ce6a0 2023-10-16T11:19:48.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f074bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f00071c80 con 0x7f8f0015da70 2023-10-16T11:19:48.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f04a57700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f000ce6a0 0x7f8f0006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f05a59700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8f00072760 0x7f8f001615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38946/0 (socket says 172.21.15.181:38946) 2023-10-16T11:19:48.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f05a59700 1 -- 172.21.15.181:0/466896205 learned_addr learned my addr 172.21.15.181:0/466896205 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:48.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.553+0000 7f8f05258700 1 --2- 172.21.15.181:0/466896205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0015da70 0x7f8f000ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8f04a57700 1 -- 172.21.15.181:0/466896205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0015da70 msgr2=0x7f8f000ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:48.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8f04a57700 1 --2- 172.21.15.181:0/466896205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0015da70 0x7f8f000ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8f04a57700 1 -- 172.21.15.181:0/466896205 >> v1:172.21.15.181:6789/0 conn(0x7f8f00072760 legacy=0x7f8f001615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8f04a57700 1 -- 172.21.15.181:0/466896205 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f00071f60 con 0x7f8f000ce6a0 2023-10-16T11:19:48.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8ef7fff700 1 -- 172.21.15.181:0/466896205 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183673600 0 0) 0x7f8f000cef00 con 0x7f8f00072760 2023-10-16T11:19:48.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8f04a57700 1 --2- 172.21.15.181:0/466896205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f0006a8a0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009840 tx=0x7f8ef0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c277b82379d87b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:48.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8ef7fff700 1 -- 172.21.15.181:0/466896205 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0011830 con 0x7f8f000ce6a0 2023-10-16T11:19:48.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8ef7fff700 1 -- 172.21.15.181:0/466896205 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0011990 con 0x7f8f000ce6a0 2023-10-16T11:19:48.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.554+0000 7f8ef7fff700 1 -- 172.21.15.181:0/466896205 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0011d20 con 0x7f8f000ce6a0 2023-10-16T11:19:48.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 -- 172.21.15.181:0/466896205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 msgr2=0x7f8f0006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 --2- 172.21.15.181:0/466896205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f0006a8a0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009840 tx=0x7f8ef0009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:48.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 -- 172.21.15.181:0/466896205 shutdown_connections 2023-10-16T11:19:48.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 --2- 172.21.15.181:0/466896205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0015da70 0x7f8f000ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 --2- 172.21.15.181:0/466896205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f0006a8a0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 -- 172.21.15.181:0/466896205 >> 172.21.15.181:0/466896205 conn(0x7f8f000cd010 msgr2=0x7f8f000cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:48.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 -- 172.21.15.181:0/466896205 shutdown_connections 2023-10-16T11:19:48.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.555+0000 7f8f074bc700 1 -- 172.21.15.181:0/466896205 wait complete. 2023-10-16T11:19:48.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.556+0000 7f8f074bc700 1 Processor -- start 2023-10-16T11:19:48.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.556+0000 7f8f074bc700 1 -- start start 2023-10-16T11:19:48.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f074bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f00072760 0x7f8f000c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f074bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f000c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f074bc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0015da70 0x7f8f00151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f074bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000aea50 con 0x7f8f000ce6a0 2023-10-16T11:19:48.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f074bc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000aed30 con 0x7f8f0015da70 2023-10-16T11:19:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f074bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000af010 con 0x7f8f00072760 2023-10-16T11:19:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f000c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f000c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56104/0 (socket says 172.21.15.181:56104) 2023-10-16T11:19:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 -- 172.21.15.181:0/1310427559 learned_addr learned my addr 172.21.15.181:0/1310427559 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:48.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f05258700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f00072760 0x7f8f000c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f05a59700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0015da70 0x7f8f00151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 -- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f00072760 msgr2=0x7f8f000c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f00072760 0x7f8f000c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 -- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0015da70 msgr2=0x7f8f00151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0015da70 0x7f8f00151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 -- 172.21.15.181:0/1310427559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef00094a0 con 0x7f8f000ce6a0 2023-10-16T11:19:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f04a57700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f000c78d0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009fd0 tx=0x7f8ef0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8ef5ffb700 1 -- 172.21.15.181:0/1310427559 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0004480 con 0x7f8f000ce6a0 2023-10-16T11:19:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8ef5ffb700 1 -- 172.21.15.181:0/1310427559 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef001f070 con 0x7f8f000ce6a0 2023-10-16T11:19:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f00160370 con 0x7f8f000ce6a0 2023-10-16T11:19:48.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.557+0000 7f8ef5ffb700 1 -- 172.21.15.181:0/1310427559 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00038e0 con 0x7f8f000ce6a0 2023-10-16T11:19:48.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.558+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f000c9b20 con 0x7f8f000ce6a0 2023-10-16T11:19:48.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.559+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0015de50 con 0x7f8f000ce6a0 2023-10-16T11:19:48.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.559+0000 7f8ef5ffb700 1 -- 172.21.15.181:0/1310427559 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 140) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8ef0021030 con 0x7f8f000ce6a0 2023-10-16T11:19:48.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.559+0000 7f8ef5ffb700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 0x7f8ee406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.560+0000 7f8f05258700 1 -- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 msgr2=0x7f8ee406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:48.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.560+0000 7f8ef5ffb700 1 -- 172.21.15.181:0/1310427559 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f8ef008af50 con 0x7f8f000ce6a0 2023-10-16T11:19:48.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.560+0000 7f8f05258700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 0x7f8ee406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:48.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.564+0000 7f8ef5ffb700 1 -- 172.21.15.181:0/1310427559 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8ef0055b10 con 0x7f8f000ce6a0 2023-10-16T11:19:48.578 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:48.577+0000 7f79406f3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:19:48.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.761+0000 7f8f05258700 1 -- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 msgr2=0x7f8ee406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:48.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.761+0000 7f8f05258700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 0x7f8ee406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:48.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.767+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f8f000aeb90 con 0x7f8f000ce6a0 2023-10-16T11:19:48.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.772+0000 7f8ef5ffb700 1 -- 172.21.15.181:0/1310427559 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8f000aeb90 con 0x7f8f000ce6a0 2023-10-16T11:19:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 msgr2=0x7f8ee406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:48.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 0x7f8ee406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 msgr2=0x7f8f000c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f000c78d0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009fd0 tx=0x7f8ef0004750 comp rx=0 tx=0).stop 2023-10-16T11:19:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 shutdown_connections 2023-10-16T11:19:48.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f8ee406b070 0x7f8ee406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f00072760 0x7f8f000c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0015da70 0x7f8f00151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 --2- 172.21.15.181:0/1310427559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f000ce6a0 0x7f8f000c78d0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 >> 172.21.15.181:0/1310427559 conn(0x7f8f000cd010 msgr2=0x7f8f00073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:48.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 shutdown_connections 2023-10-16T11:19:48.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.774+0000 7f8f074bc700 1 -- 172.21.15.181:0/1310427559 wait complete. 2023-10-16T11:19:48.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:48.789 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-16T11:19:48.810 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:19:48.869 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:48.867+0000 7f3cceea9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:19:48.917 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:48.916+0000 7f79406f3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:19:48.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.977+0000 7f2078f1f700 1 Processor -- start 2023-10-16T11:19:48.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.977+0000 7f2078f1f700 1 -- start start 2023-10-16T11:19:48.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.978+0000 7f2078f1f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740cd5c0 0x7f20740cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.978+0000 7f2078f1f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20740ce7e0 con 0x7f20740ce3a0 2023-10-16T11:19:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.978+0000 7f2078f1f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207415b540 con 0x7f20740afa10 2023-10-16T11:19:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.978+0000 7f2078f1f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207415b820 con 0x7f20740cd5c0 2023-10-16T11:19:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.978+0000 7f2071d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f20740ce3a0 0x7f20740cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38994/0 (socket says 172.21.15.181:38994) 2023-10-16T11:19:48.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f2071d9b700 1 -- 172.21.15.181:0/18240854 learned_addr learned my addr 172.21.15.181:0/18240854 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f2072d9d700 1 --2- 172.21.15.181:0/18240854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740cd5c0 0x7f20740cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f207159a700 1 -- 172.21.15.181:0/18240854 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593021791 0 0) 0x7f20740ce7e0 con 0x7f20740ce3a0 2023-10-16T11:19:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f207159a700 1 -- 172.21.15.181:0/18240854 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f205c0035b0 con 0x7f20740ce3a0 2023-10-16T11:19:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f207159a700 1 -- 172.21.15.181:0/18240854 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057301951 0 0) 0x7f207415b540 con 0x7f20740afa10 2023-10-16T11:19:48.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f207159a700 1 -- 172.21.15.181:0/18240854 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20740ce7e0 con 0x7f20740afa10 2023-10-16T11:19:48.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f2072d9d700 1 -- 172.21.15.181:0/18240854 >> v1:172.21.15.190:6789/0 conn(0x7f20740afa10 legacy=0x7f207406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f2072d9d700 1 -- 172.21.15.181:0/18240854 >> v1:172.21.15.181:6789/0 conn(0x7f20740ce3a0 legacy=0x7f20740cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f2072d9d700 1 -- 172.21.15.181:0/18240854 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207415bb00 con 0x7f20740cd5c0 2023-10-16T11:19:48.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f2072d9d700 1 --2- 172.21.15.181:0/18240854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740cd5c0 0x7f20740cb8b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f206c009dc0 tx=0x7f206c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13425c48866bb30 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:48.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f207159a700 1 -- 172.21.15.181:0/18240854 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00e070 con 0x7f20740cd5c0 2023-10-16T11:19:48.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f207159a700 1 -- 172.21.15.181:0/18240854 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c00bb20 con 0x7f20740cd5c0 2023-10-16T11:19:48.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.979+0000 7f207159a700 1 -- 172.21.15.181:0/18240854 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c012430 con 0x7f20740cd5c0 2023-10-16T11:19:48.986 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:48.981+0000 7f3cceea9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:19:48.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.980+0000 7f2078f1f700 1 -- 172.21.15.181:0/18240854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740cd5c0 msgr2=0x7f20740cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.980+0000 7f2078f1f700 1 --2- 172.21.15.181:0/18240854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740cd5c0 0x7f20740cb8b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f206c009dc0 tx=0x7f206c00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:48.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.980+0000 7f2078f1f700 1 -- 172.21.15.181:0/18240854 shutdown_connections 2023-10-16T11:19:48.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.980+0000 7f2078f1f700 1 --2- 172.21.15.181:0/18240854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740cd5c0 0x7f20740cb8b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.980+0000 7f2078f1f700 1 -- 172.21.15.181:0/18240854 >> 172.21.15.181:0/18240854 conn(0x7f20740d04a0 msgr2=0x7f20740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:48.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.980+0000 7f2078f1f700 1 -- 172.21.15.181:0/18240854 shutdown_connections 2023-10-16T11:19:48.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.980+0000 7f2078f1f700 1 -- 172.21.15.181:0/18240854 wait complete. 2023-10-16T11:19:48.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.982+0000 7f2078f1f700 1 Processor -- start 2023-10-16T11:19:48.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.982+0000 7f2078f1f700 1 -- start start 2023-10-16T11:19:48.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.982+0000 7f2078f1f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740afa10 0x7f20740ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.982+0000 7f2078f1f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20740cd5c0 0x7f20741f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.982+0000 7f2078f1f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20740ce3a0 0x7f20741f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.982+0000 7f2078f1f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740ae830 con 0x7f20740cd5c0 2023-10-16T11:19:48.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2078f1f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20741f6bd0 con 0x7f20740ce3a0 2023-10-16T11:19:48.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2078f1f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740ca0c0 con 0x7f20740afa10 2023-10-16T11:19:48.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20740cd5c0 0x7f20741f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20740cd5c0 0x7f20741f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56154/0 (socket says 172.21.15.181:56154) 2023-10-16T11:19:48.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 -- 172.21.15.181:0/2007633916 learned_addr learned my addr 172.21.15.181:0/2007633916 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:48.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2072d9d700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20740ce3a0 0x7f20741f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740afa10 0x7f20740ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:48.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740afa10 msgr2=0x7f20740ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20740afa10 0x7f20740ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20740ce3a0 msgr2=0x7f20741f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:48.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20740ce3a0 0x7f20741f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:48.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 -- 172.21.15.181:0/2007633916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c009a70 con 0x7f20740cd5c0 2023-10-16T11:19:48.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2071d9b700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20740cd5c0 0x7f20741f6ff0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f206800c700 tx=0x7f206800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:48.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068005d70 con 0x7f20740cd5c0 2023-10-16T11:19:48.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20680070d0 con 0x7f20740cd5c0 2023-10-16T11:19:48.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068007440 con 0x7f20740cd5c0 2023-10-16T11:19:48.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20740adcb0 con 0x7f20740cd5c0 2023-10-16T11:19:48.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.983+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20740ae270 con 0x7f20740cd5c0 2023-10-16T11:19:48.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.985+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20740ccd20 con 0x7f20740cd5c0 2023-10-16T11:19:48.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.985+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 140) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f20680075a0 con 0x7f20740cd5c0 2023-10-16T11:19:48.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.985+0000 7f2062ffd700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 0x7f205406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:48.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.985+0000 7f207259c700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 msgr2=0x7f205406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:48.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.985+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f2068088f30 con 0x7f20740cd5c0 2023-10-16T11:19:48.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.985+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 0x7f205406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:48.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:48.990+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2068053a80 con 0x7f20740cd5c0 2023-10-16T11:19:49.039 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:49.037+0000 7f79406f3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:19:49.057 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:49.056+0000 7f3cceea9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:19:49.115 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:49.114+0000 7f79406f3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:19:49.137 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:49.136+0000 7f3cceea9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:19:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:49.186+0000 7f207259c700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 msgr2=0x7f205406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:49.186+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 0x7f205406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:49.198 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:49.197+0000 7f79406f3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:19:49.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:49.217+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 --> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f20740cb390 con 0x7f205406b070 2023-10-16T11:19:49.505 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:49.503+0000 7f3cceea9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:19:49.568 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:49.566+0000 7f79406f3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:19:49.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:49.587+0000 7f207259c700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 msgr2=0x7f205406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:49.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:49.587+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 0x7f205406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:19:50.073 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:50.071+0000 7f3cceea9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:19:50.139 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:50.137+0000 7f79406f3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:19:50.142 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:50.141+0000 7f3cceea9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:19:50.208 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:50.207+0000 7f79406f3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:19:50.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:50.388+0000 7f207259c700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 msgr2=0x7f205406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:50.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:50.388+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 0x7f205406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:19:50.772 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:50.770+0000 7f3cceea9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:50.843 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:50.841+0000 7f79406f3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:50.849 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:50.847+0000 7f3cceea9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:19:50.921 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:50.920+0000 7f79406f3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:19:51.016 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:50.991+0000 7f3cceea9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:51.069 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:51.067+0000 7f79406f3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:19:51.218 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:51.216+0000 7f3cceea9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:19:51.291 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:51.289+0000 7f3cceea9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:19:51.298 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:51.296+0000 7f79406f3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:19:51.372 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:51.370+0000 7f79406f3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:19:51.555 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:51.553+0000 7f3cceea9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:19:51.641 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:51.639+0000 7f79406f3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:19:51.777 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:51.776+0000 7f3cceea9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:19:51.865 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:51.864+0000 7f79406f3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:19:51.925 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:51.923+0000 7f3cceea9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:19:51.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:51.990+0000 7f207259c700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 msgr2=0x7f205406d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1781343309 2023-10-16T11:19:51.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:51.990+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 0x7f205406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:19:52.016 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:52.015+0000 7f79406f3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:19:52.036 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:52.035+0000 7f3cceea9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:19:52.131 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:52.130+0000 7f79406f3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:19:52.191 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:52.190+0000 7f3cceea9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:19:52.289 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:52.288+0000 7f79406f3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:19:52.454 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:52.452+0000 7f3cceea9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:52.554 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:52.553+0000 7f79406f3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:52.615 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:52.613+0000 7f3cceea9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:19:52.719 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:52.718+0000 7f79406f3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:19:53.141 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:53.140+0000 7f3cceea9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:19:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:53.211+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 141) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2068053e50 con 0x7f20740cd5c0 2023-10-16T11:19:53.254 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.253+0000 7f79406f3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:19:53.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:53.275+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 142) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f2068086e90 con 0x7f20740cd5c0 2023-10-16T11:19:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:53.275+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 msgr2=0x7f205406d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:53.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:53.275+0000 7f2062ffd700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/1781343309,v1:172.21.15.181:6833/1781343309] conn(0x7f205406b070 0x7f205406d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:53.503 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.503 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.504 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.504 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.504 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.504 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.505 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.505 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.505 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.505 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.502+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.512 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.513 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.513 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.513 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.513 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b2ecd700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.513 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.514 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.514 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.514 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:53.514 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:53.511+0000 7f78b5ed3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:19:54.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.280+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 143) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2068053160 con 0x7f20740cd5c0 2023-10-16T11:19:54.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.280+0000 7f2062ffd700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f205401cf00 0x7f2054077d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.280+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2054071a40 con 0x7f205401cf00 2023-10-16T11:19:54.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.281+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f205401cf00 0x7f2054077d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.284+0000 7f207259c700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f205401cf00 0x7f2054077d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2064001270 tx=0x7f2064009040 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:54.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.343+0000 7f2062ffd700 1 -- 172.21.15.181:0/2007633916 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7f2054071a40 con 0x7f205401cf00 2023-10-16T11:19:54.344 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-10-16T11:19:54.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f205401cf00 msgr2=0x7f2054077d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f205401cf00 0x7f2054077d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2064001270 tx=0x7f2064009040 comp rx=0 tx=0).stop 2023-10-16T11:19:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20740cd5c0 msgr2=0x7f20741f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 --2- 172.21.15.181:0/2007633916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20740cd5c0 0x7f20741f6ff0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f206800c700 tx=0x7f206800ca10 comp rx=0 tx=0).stop 2023-10-16T11:19:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2072d9d700 1 -- 172.21.15.181:0/2007633916 reap_dead start 2023-10-16T11:19:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 shutdown_connections 2023-10-16T11:19:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 >> 172.21.15.181:0/2007633916 conn(0x7f20740d04a0 msgr2=0x7f20740cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 shutdown_connections 2023-10-16T11:19:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.345+0000 7f2078f1f700 1 -- 172.21.15.181:0/2007633916 wait complete. 2023-10-16T11:19:54.360 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-10-16T11:19:54.360 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:19:54.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.486+0000 7f9423e5e700 1 Processor -- start 2023-10-16T11:19:54.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.487+0000 7f9423e5e700 1 -- start start 2023-10-16T11:19:54.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.487+0000 7f9423e5e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 0x7f941c0d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.487+0000 7f9423e5e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c15dd80 con 0x7f941c0ce270 2023-10-16T11:19:54.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.487+0000 7f9423e5e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c06ba60 con 0x7f941c0d01e0 2023-10-16T11:19:54.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.487+0000 7f9423e5e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0d42a0 con 0x7f941c06d530 2023-10-16T11:19:54.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.487+0000 7f94223fb700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f941c0ce270 0x7f941c0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39254/0 (socket says 172.21.15.181:39254) 2023-10-16T11:19:54.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.487+0000 7f94223fb700 1 -- 172.21.15.181:0/1761612598 learned_addr learned my addr 172.21.15.181:0/1761612598 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:54.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 0x7f941c0d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835753088 0 0) 0x7f941c15dd80 con 0x7f941c0ce270 2023-10-16T11:19:54.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94080035b0 con 0x7f941c0ce270 2023-10-16T11:19:54.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684325341 0 0) 0x7f941c06ba60 con 0x7f941c0d01e0 2023-10-16T11:19:54.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f941c15dd80 con 0x7f941c0d01e0 2023-10-16T11:19:54.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2024873379 0 0) 0x7f94080035b0 con 0x7f941c0ce270 2023-10-16T11:19:54.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 msgr2=0x7f941c0d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:54.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 0x7f941c0d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 >> v1:172.21.15.190:6789/0 conn(0x7f941c0d01e0 legacy=0x7f941c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0d4580 con 0x7f941c0ce270 2023-10-16T11:19:54.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f94140019f0 con 0x7f941c0ce270 2023-10-16T11:19:54.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 0x7f941c0d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:54.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 >> v1:172.21.15.181:6789/0 conn(0x7f941c0ce270 legacy=0x7f941c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 0x7f9408003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f9408004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9408004d90 0x7f9408005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0d42a0 con 0x7f9408004100 2023-10-16T11:19:54.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940800bac0 con 0x7f9408004d90 2023-10-16T11:19:54.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940800bc80 con 0x7f941c06d530 2023-10-16T11:19:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f9408004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f9421bfa700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 0x7f9408003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94223fb700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9408004d90 0x7f9408005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 -- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 msgr2=0x7f9408003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:54.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c06d530 0x7f9408003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 -- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9408004d90 msgr2=0x7f9408005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9408004d90 0x7f9408005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.488+0000 7f94213f9700 1 -- 172.21.15.181:0/1761612598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180094a0 con 0x7f9408004100 2023-10-16T11:19:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f94213f9700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f9408004850 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f9418009d10 tx=0x7f9408005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee149ff5dfd4e575 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:54.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418009e60 con 0x7f9408004100 2023-10-16T11:19:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941800e9a0 con 0x7f9408004100 2023-10-16T11:19:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9420bf8700 1 -- 172.21.15.181:0/1761612598 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941800ed50 con 0x7f9408004100 2023-10-16T11:19:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9423e5e700 1 -- 172.21.15.181:0/1761612598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 msgr2=0x7f9408004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9423e5e700 1 --2- 172.21.15.181:0/1761612598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f9408004850 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f9418009d10 tx=0x7f9408005880 comp rx=0 tx=0).stop 2023-10-16T11:19:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f94223fb700 1 -- 172.21.15.181:0/1761612598 reap_dead start 2023-10-16T11:19:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9423e5e700 1 -- 172.21.15.181:0/1761612598 shutdown_connections 2023-10-16T11:19:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9423e5e700 1 -- 172.21.15.181:0/1761612598 >> 172.21.15.181:0/1761612598 conn(0x7f941c08af90 msgr2=0x7f941c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9423e5e700 1 -- 172.21.15.181:0/1761612598 shutdown_connections 2023-10-16T11:19:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.489+0000 7f9423e5e700 1 -- 172.21.15.181:0/1761612598 wait complete. 2023-10-16T11:19:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.491+0000 7f9423e5e700 1 Processor -- start 2023-10-16T11:19:54.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.491+0000 7f9423e5e700 1 -- start start 2023-10-16T11:19:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9423e5e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f941c1f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9423e5e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c0b7240 0x7f941c0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9423e5e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c0b51c0 0x7f941c0b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9423e5e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c1585f0 con 0x7f9408004100 2023-10-16T11:19:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9423e5e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0b5b80 con 0x7f941c0b7240 2023-10-16T11:19:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9423e5e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0cd6f0 con 0x7f941c0b51c0 2023-10-16T11:19:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f941c1f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f941c1f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56416/0 (socket says 172.21.15.181:56416) 2023-10-16T11:19:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 -- 172.21.15.181:0/918701198 learned_addr learned my addr 172.21.15.181:0/918701198 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f94213f9700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c0b7240 0x7f941c0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f94223fb700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c0b51c0 0x7f941c0b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 -- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c0b51c0 msgr2=0x7f941c0b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c0b51c0 0x7f941c0b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 -- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c0b7240 msgr2=0x7f941c0b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c0b7240 0x7f941c0b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.492+0000 7f9421bfa700 1 -- 172.21.15.181:0/918701198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0ccae0 con 0x7f9408004100 2023-10-16T11:19:54.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.493+0000 7f9421bfa700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f941c1f2b80 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f940c00f810 tx=0x7f940c00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:54.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.493+0000 7f94127fc700 1 -- 172.21.15.181:0/918701198 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0052e0 con 0x7f9408004100 2023-10-16T11:19:54.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.493+0000 7f94127fc700 1 -- 172.21.15.181:0/918701198 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c005440 con 0x7f9408004100 2023-10-16T11:19:54.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.493+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c0ccdc0 con 0x7f9408004100 2023-10-16T11:19:54.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.493+0000 7f94127fc700 1 -- 172.21.15.181:0/918701198 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0038c0 con 0x7f9408004100 2023-10-16T11:19:54.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.493+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c0cd380 con 0x7f9408004100 2023-10-16T11:19:54.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.494+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c067d20 con 0x7f9408004100 2023-10-16T11:19:54.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.494+0000 7f94127fc700 1 -- 172.21.15.181:0/918701198 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 143) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f940c003a20 con 0x7f9408004100 2023-10-16T11:19:54.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.494+0000 7f94127fc700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f940006ada0 0x7f940006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.495+0000 7f94127fc700 1 -- 172.21.15.181:0/918701198 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f940c089890 con 0x7f9408004100 2023-10-16T11:19:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.495+0000 7f94213f9700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f940006ada0 0x7f940006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.495+0000 7f94213f9700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f940006ada0 0x7f940006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9418000c00 tx=0x7f9418008030 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:54.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.499+0000 7f94127fc700 1 -- 172.21.15.181:0/918701198 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f940c0545a0 con 0x7f9408004100 2023-10-16T11:19:54.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.724+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f941c06c000 con 0x7f940006ada0 2023-10-16T11:19:54.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.774+0000 7f94127fc700 1 -- 172.21.15.181:0/918701198 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f9418022040 con 0x7f940006ada0 2023-10-16T11:19:54.776 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-10-16T11:19:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f940006ada0 msgr2=0x7f940006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f940006ada0 0x7f940006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9418000c00 tx=0x7f9418008030 comp rx=0 tx=0).stop 2023-10-16T11:19:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 msgr2=0x7f941c1f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f941c1f2b80 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f940c00f810 tx=0x7f940c00fbd0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 shutdown_connections 2023-10-16T11:19:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f940006ada0 0x7f940006d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f941c0b51c0 0x7f941c0b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9408004100 0x7f941c1f2b80 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 --2- 172.21.15.181:0/918701198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c0b7240 0x7f941c0b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 >> 172.21.15.181:0/918701198 conn(0x7f941c08af90 msgr2=0x7f941c0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 shutdown_connections 2023-10-16T11:19:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.777+0000 7f9423e5e700 1 -- 172.21.15.181:0/918701198 wait complete. 2023-10-16T11:19:54.790 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-10-16T11:19:54.791 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:19:54.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.916+0000 7f9684eed700 1 Processor -- start 2023-10-16T11:19:54.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.916+0000 7f9684eed700 1 -- start start 2023-10-16T11:19:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f9684eed700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f968015f150 0x7f96800cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f9684eed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96800cd170 0x7f96800b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f9684eed700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96800b2ae0 con 0x7f96800ce4f0 2023-10-16T11:19:54.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f9684eed700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96800ce9a0 con 0x7f968015f150 2023-10-16T11:19:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f9684eed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b2dc0 con 0x7f96800cd170 2023-10-16T11:19:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f967e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f968015f150 0x7f96800cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f967e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f968015f150 0x7f96800cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56424/0 (socket says 172.21.15.181:56424) 2023-10-16T11:19:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f967e59c700 1 -- 172.21.15.181:0/659389503 learned_addr learned my addr 172.21.15.181:0/659389503 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:54.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f967ed9d700 1 --2- 172.21.15.181:0/659389503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96800cd170 0x7f96800b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.917+0000 7f967e59c700 1 -- 172.21.15.181:0/659389503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96800cd170 msgr2=0x7f96800b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f967e59c700 1 --2- 172.21.15.181:0/659389503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96800cd170 0x7f96800b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f967e59c700 1 -- 172.21.15.181:0/659389503 >> v1:172.21.15.181:6789/0 conn(0x7f96800ce4f0 legacy=0x7f96800b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f967e59c700 1 -- 172.21.15.181:0/659389503 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800cf9d0 con 0x7f968015f150 2023-10-16T11:19:54.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f967e59c700 1 --2- 172.21.15.181:0/659389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968015f150 0x7f96800cdfb0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f9674009920 tx=0x7f9674009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f964c68da8bfe100 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f967d59a700 1 -- 172.21.15.181:0/659389503 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967400ba20 con 0x7f968015f150 2023-10-16T11:19:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f967d59a700 1 -- 172.21.15.181:0/659389503 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967400bb80 con 0x7f968015f150 2023-10-16T11:19:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f967d59a700 1 -- 172.21.15.181:0/659389503 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674009e70 con 0x7f968015f150 2023-10-16T11:19:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f9684eed700 1 -- 172.21.15.181:0/659389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968015f150 msgr2=0x7f96800cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f9684eed700 1 --2- 172.21.15.181:0/659389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968015f150 0x7f96800cdfb0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f9674009920 tx=0x7f9674009c30 comp rx=0 tx=0).stop 2023-10-16T11:19:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f9684eed700 1 -- 172.21.15.181:0/659389503 shutdown_connections 2023-10-16T11:19:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f9684eed700 1 --2- 172.21.15.181:0/659389503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96800cd170 0x7f96800b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.918+0000 7f9684eed700 1 --2- 172.21.15.181:0/659389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968015f150 0x7f96800cdfb0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.919+0000 7f9684eed700 1 -- 172.21.15.181:0/659389503 >> 172.21.15.181:0/659389503 conn(0x7f968009a930 msgr2=0x7f96800d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:54.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.919+0000 7f9684eed700 1 -- 172.21.15.181:0/659389503 shutdown_connections 2023-10-16T11:19:54.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.919+0000 7f9684eed700 1 -- 172.21.15.181:0/659389503 wait complete. 2023-10-16T11:19:54.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.920+0000 7f9684eed700 1 Processor -- start 2023-10-16T11:19:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.920+0000 7f9684eed700 1 -- start start 2023-10-16T11:19:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96800cd170 0x7f968015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800ce4f0 0x7f968009b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015f150 0x7f96801f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800ca2d0 con 0x7f96800ce4f0 2023-10-16T11:19:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800ca5b0 con 0x7f96800cd170 2023-10-16T11:19:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800ca890 con 0x7f968015f150 2023-10-16T11:19:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800ce4f0 0x7f968009b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800ce4f0 0x7f968009b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56432/0 (socket says 172.21.15.181:56432) 2023-10-16T11:19:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 -- 172.21.15.181:0/3290498993 learned_addr learned my addr 172.21.15.181:0/3290498993 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967ed9d700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015f150 0x7f96801f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967e59c700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96800cd170 0x7f968015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 -- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015f150 msgr2=0x7f96801f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015f150 0x7f96801f0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 -- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96800cd170 msgr2=0x7f968015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96800cd170 0x7f968015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 -- 172.21.15.181:0/3290498993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674009550 con 0x7f96800ce4f0 2023-10-16T11:19:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967e59c700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96800cd170 0x7f968015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f967dd9b700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800ce4f0 0x7f968009b3a0 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f967000c4e0 tx=0x7f967000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f966effd700 1 -- 172.21.15.181:0/3290498993 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96700052d0 con 0x7f96800ce4f0 2023-10-16T11:19:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f966effd700 1 -- 172.21.15.181:0/3290498993 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9670005430 con 0x7f96800ce4f0 2023-10-16T11:19:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f966effd700 1 -- 172.21.15.181:0/3290498993 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96700038c0 con 0x7f96800ce4f0 2023-10-16T11:19:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96800cab70 con 0x7f96800ce4f0 2023-10-16T11:19:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.921+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968015ac20 con 0x7f96800ce4f0 2023-10-16T11:19:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.922+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96800cb090 con 0x7f96800ce4f0 2023-10-16T11:19:54.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.923+0000 7f966effd700 1 -- 172.21.15.181:0/3290498993 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 143) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f9670003a20 con 0x7f96800ce4f0 2023-10-16T11:19:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.923+0000 7f966effd700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f966006ada0 0x7f966006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.923+0000 7f966effd700 1 -- 172.21.15.181:0/3290498993 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f9670088d90 con 0x7f96800ce4f0 2023-10-16T11:19:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.923+0000 7f967e59c700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f966006ada0 0x7f966006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.924+0000 7f967e59c700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f966006ada0 0x7f966006d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9674009c70 tx=0x7f9674008030 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:54.928+0000 7f966effd700 1 -- 172.21.15.181:0/3290498993 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f96800cb090 con 0x7f96800ce4f0 2023-10-16T11:19:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.149+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f96800cff90 con 0x7f966006ada0 2023-10-16T11:19:55.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.205+0000 7f966effd700 1 -- 172.21.15.181:0/3290498993 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f9674028050 con 0x7f966006ada0 2023-10-16T11:19:55.207 INFO:teuthology.orchestra.run.smithi181.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-10-16T11:19:55.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f966006ada0 msgr2=0x7f966006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f966006ada0 0x7f966006d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9674009c70 tx=0x7f9674008030 comp rx=0 tx=0).stop 2023-10-16T11:19:55.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800ce4f0 msgr2=0x7f968009b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800ce4f0 0x7f968009b3a0 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f967000c4e0 tx=0x7f967000c7f0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 shutdown_connections 2023-10-16T11:19:55.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f966006ada0 0x7f966006d260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015f150 0x7f96801f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96800cd170 0x7f968015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 --2- 172.21.15.181:0/3290498993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96800ce4f0 0x7f968009b3a0 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 >> 172.21.15.181:0/3290498993 conn(0x7f968009a930 msgr2=0x7f96800ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:55.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 shutdown_connections 2023-10-16T11:19:55.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.208+0000 7f9684eed700 1 -- 172.21.15.181:0/3290498993 wait complete. 2023-10-16T11:19:55.221 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-10-16T11:19:55.221 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-16T11:19:55.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.352+0000 7fc24eb02700 1 Processor -- start 2023-10-16T11:19:55.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.352+0000 7fc24eb02700 1 -- start start 2023-10-16T11:19:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24eb02700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc2480ce610 0x7fc24806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24eb02700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2480ce9f0 0x7fc24806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24eb02700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc248162650 con 0x7fc2480d0fc0 2023-10-16T11:19:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24eb02700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc248162190 con 0x7fc2480ce9f0 2023-10-16T11:19:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24eb02700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2480d0230 con 0x7fc2480ce610 2023-10-16T11:19:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24d09f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2480ce9f0 0x7fc24806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc247fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc2480d0fc0 0x7fc2480cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39302/0 (socket says 172.21.15.181:39302) 2023-10-16T11:19:55.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc247fff700 1 -- 172.21.15.181:0/528564312 learned_addr learned my addr 172.21.15.181:0/528564312 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24c89e700 1 --2- 172.21.15.181:0/528564312 >> v2:172.21.15.190:3300/0 conn(0x7fc2480ce610 0x7fc24806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24d09f700 1 -- 172.21.15.181:0/528564312 >> v2:172.21.15.190:3300/0 conn(0x7fc2480ce610 msgr2=0x7fc24806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24d09f700 1 --2- 172.21.15.181:0/528564312 >> v2:172.21.15.190:3300/0 conn(0x7fc2480ce610 0x7fc24806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24d09f700 1 -- 172.21.15.181:0/528564312 >> v1:172.21.15.181:6789/0 conn(0x7fc2480d0fc0 legacy=0x7fc2480cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc24d09f700 1 -- 172.21.15.181:0/528564312 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc248162370 con 0x7fc2480ce9f0 2023-10-16T11:19:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.353+0000 7fc2477fe700 1 -- 172.21.15.181:0/528564312 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719579171 0 0) 0x7fc248162650 con 0x7fc2480d0fc0 2023-10-16T11:19:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24d09f700 1 --2- 172.21.15.181:0/528564312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480ce9f0 0x7fc24806a8a0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fc240009d90 tx=0x7fc24000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf1d021bb69062d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc2477fe700 1 -- 172.21.15.181:0/528564312 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24000e070 con 0x7fc2480ce9f0 2023-10-16T11:19:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc2477fe700 1 -- 172.21.15.181:0/528564312 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24000bae0 con 0x7fc2480ce9f0 2023-10-16T11:19:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc2477fe700 1 -- 172.21.15.181:0/528564312 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24000be90 con 0x7fc2480ce9f0 2023-10-16T11:19:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 -- 172.21.15.181:0/528564312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480ce9f0 msgr2=0x7fc24806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 --2- 172.21.15.181:0/528564312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480ce9f0 0x7fc24806a8a0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fc240009d90 tx=0x7fc24000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 -- 172.21.15.181:0/528564312 shutdown_connections 2023-10-16T11:19:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 --2- 172.21.15.181:0/528564312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480ce9f0 0x7fc24806a8a0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 --2- 172.21.15.181:0/528564312 >> v2:172.21.15.190:3300/0 conn(0x7fc2480ce610 0x7fc24806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 -- 172.21.15.181:0/528564312 >> 172.21.15.181:0/528564312 conn(0x7fc2480d04a0 msgr2=0x7fc2480d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 -- 172.21.15.181:0/528564312 shutdown_connections 2023-10-16T11:19:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.354+0000 7fc24eb02700 1 -- 172.21.15.181:0/528564312 wait complete. 2023-10-16T11:19:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.356+0000 7fc24eb02700 1 Processor -- start 2023-10-16T11:19:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.356+0000 7fc24eb02700 1 -- start start 2023-10-16T11:19:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.356+0000 7fc24eb02700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2480ce610 0x7fc24815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24eb02700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2480ce9f0 0x7fc24815fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24eb02700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480d0fc0 0x7fc2481f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24eb02700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480cb460 con 0x7fc2480d0fc0 2023-10-16T11:19:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24eb02700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480c7b10 con 0x7fc2480ce610 2023-10-16T11:19:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24eb02700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480c7df0 con 0x7fc2480ce9f0 2023-10-16T11:19:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480d0fc0 0x7fc2481f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480d0fc0 0x7fc2481f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56464/0 (socket says 172.21.15.181:56464) 2023-10-16T11:19:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 -- 172.21.15.181:0/3679719343 learned_addr learned my addr 172.21.15.181:0/3679719343 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24c89e700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2480ce610 0x7fc24815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc247fff700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2480ce9f0 0x7fc24815fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 -- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2480ce9f0 msgr2=0x7fc24815fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2480ce9f0 0x7fc24815fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 -- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2480ce610 msgr2=0x7fc24815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2480ce610 0x7fc24815f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 -- 172.21.15.181:0/3679719343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc240009a70 con 0x7fc2480d0fc0 2023-10-16T11:19:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24d09f700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480d0fc0 0x7fc2481f6a20 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fc240014040 tx=0x7fc240004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc2457fa700 1 -- 172.21.15.181:0/3679719343 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240004550 con 0x7fc2480d0fc0 2023-10-16T11:19:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc2457fa700 1 -- 172.21.15.181:0/3679719343 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc240020070 con 0x7fc2480d0fc0 2023-10-16T11:19:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc2457fa700 1 -- 172.21.15.181:0/3679719343 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2400048d0 con 0x7fc2480d0fc0 2023-10-16T11:19:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2480c80d0 con 0x7fc2480d0fc0 2023-10-16T11:19:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.357+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2480c7570 con 0x7fc2480d0fc0 2023-10-16T11:19:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.358+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2481f7730 con 0x7fc2480d0fc0 2023-10-16T11:19:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.359+0000 7fc2457fa700 1 -- 172.21.15.181:0/3679719343 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 143) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc240012430 con 0x7fc2480d0fc0 2023-10-16T11:19:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.359+0000 7fc2457fa700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc22c06ada0 0x7fc22c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.359+0000 7fc24c89e700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc22c06ada0 0x7fc22c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.359+0000 7fc2457fa700 1 -- 172.21.15.181:0/3679719343 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fc240022070 con 0x7fc2480d0fc0 2023-10-16T11:19:55.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.360+0000 7fc24c89e700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc22c06ada0 0x7fc22c06d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009bc0 tx=0x7fc23c0091c0 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:55.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.364+0000 7fc2457fa700 1 -- 172.21.15.181:0/3679719343 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc24005a100 con 0x7fc2480d0fc0 2023-10-16T11:19:55.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.588+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc24806f3d0 con 0x7fc22c06ada0 2023-10-16T11:19:55.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.594+0000 7fc2457fa700 1 -- 172.21.15.181:0/3679719343 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7fc23c005a20 con 0x7fc22c06ada0 2023-10-16T11:19:55.595 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:55.595 INFO:teuthology.orchestra.run.smithi181.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-16T11:19:55.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.596+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc22c06ada0 msgr2=0x7fc22c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.596+0000 7fc24eb02700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc22c06ada0 0x7fc22c06d260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009bc0 tx=0x7fc23c0091c0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.596+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480d0fc0 msgr2=0x7fc2481f6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.596+0000 7fc24eb02700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480d0fc0 0x7fc2481f6a20 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fc240014040 tx=0x7fc240004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 shutdown_connections 2023-10-16T11:19:55.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc22c06ada0 0x7fc22c06d260 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480d0fc0 0x7fc2481f6a20 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2480ce610 0x7fc24815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 --2- 172.21.15.181:0/3679719343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2480ce9f0 0x7fc24815fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 >> 172.21.15.181:0/3679719343 conn(0x7fc2480d04a0 msgr2=0x7fc2480b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:55.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 shutdown_connections 2023-10-16T11:19:55.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.597+0000 7fc24eb02700 1 -- 172.21.15.181:0/3679719343 wait complete. 2023-10-16T11:19:55.609 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-16T11:19:55.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-16T11:19:55.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.733+0000 7fc4d74ef700 1 Processor -- start 2023-10-16T11:19:55.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.733+0000 7fc4d74ef700 1 -- start start 2023-10-16T11:19:55.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d74ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4d00b22a0 0x7fc4d0066610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d74ef700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00cbfd0 0x7fc4d006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d74ef700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d0066bf0 con 0x7fc4d015bb70 2023-10-16T11:19:55.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d74ef700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d006e240 con 0x7fc4d00b22a0 2023-10-16T11:19:55.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d74ef700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d006cf40 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d4a8a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4d00b22a0 0x7fc4d0066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d4a8a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4d00b22a0 0x7fc4d0066610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56470/0 (socket says 172.21.15.181:56470) 2023-10-16T11:19:55.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.734+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/4252881707 learned_addr learned my addr 172.21.15.181:0/4252881707 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:55.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d5a8c700 1 --2- 172.21.15.181:0/4252881707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00cbfd0 0x7fc4d006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/4252881707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00cbfd0 msgr2=0x7fc4d006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:55.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d4a8a700 1 --2- 172.21.15.181:0/4252881707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00cbfd0 0x7fc4d006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/4252881707 >> v1:172.21.15.190:6789/0 conn(0x7fc4d015bb70 legacy=0x7fc4d00cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/4252881707 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d006d220 con 0x7fc4d00b22a0 2023-10-16T11:19:55.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d4a8a700 1 --2- 172.21.15.181:0/4252881707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00b22a0 0x7fc4d0066610 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0098f0 tx=0x7fc4cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4028c7636a0b820 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:55.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4c7fff700 1 -- 172.21.15.181:0/4252881707 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc011830 con 0x7fc4d00b22a0 2023-10-16T11:19:55.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4c7fff700 1 -- 172.21.15.181:0/4252881707 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc011990 con 0x7fc4d00b22a0 2023-10-16T11:19:55.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4c7fff700 1 -- 172.21.15.181:0/4252881707 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc011d20 con 0x7fc4d00b22a0 2023-10-16T11:19:55.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d74ef700 1 -- 172.21.15.181:0/4252881707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00b22a0 msgr2=0x7fc4d0066610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.735+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/4252881707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00b22a0 0x7fc4d0066610 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0098f0 tx=0x7fc4cc009c00 comp rx=0 tx=0).stop 2023-10-16T11:19:55.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.736+0000 7fc4d74ef700 1 -- 172.21.15.181:0/4252881707 shutdown_connections 2023-10-16T11:19:55.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.736+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/4252881707 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00cbfd0 0x7fc4d006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.736+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/4252881707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00b22a0 0x7fc4d0066610 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.736+0000 7fc4d74ef700 1 -- 172.21.15.181:0/4252881707 >> 172.21.15.181:0/4252881707 conn(0x7fc4d00b1780 msgr2=0x7fc4d00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:55.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.736+0000 7fc4d74ef700 1 -- 172.21.15.181:0/4252881707 shutdown_connections 2023-10-16T11:19:55.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.736+0000 7fc4d74ef700 1 -- 172.21.15.181:0/4252881707 wait complete. 2023-10-16T11:19:55.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.737+0000 7fc4d74ef700 1 Processor -- start 2023-10-16T11:19:55.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.737+0000 7fc4d74ef700 1 -- start start 2023-10-16T11:19:55.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.737+0000 7fc4d74ef700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00b22a0 0x7fc4d0157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.737+0000 7fc4d74ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00cbfd0 0x7fc4d015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d74ef700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d015bb70 0x7fc4d015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d74ef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d009a0f0 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d74ef700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d00c4fc0 con 0x7fc4d015bb70 2023-10-16T11:19:55.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d74ef700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01f11a0 con 0x7fc4d00b22a0 2023-10-16T11:19:55.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00cbfd0 0x7fc4d015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00cbfd0 0x7fc4d015cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56476/0 (socket says 172.21.15.181:56476) 2023-10-16T11:19:55.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/3271144093 learned_addr learned my addr 172.21.15.181:0/3271144093 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:55.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d528b700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00b22a0 0x7fc4d0157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d5a8c700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d015bb70 0x7fc4d015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00b22a0 msgr2=0x7fc4d0157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00b22a0 0x7fc4d0157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d015bb70 msgr2=0x7fc4d015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d015bb70 0x7fc4d015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 -- 172.21.15.181:0/3271144093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009550 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d4a8a700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00cbfd0 0x7fc4d015cfa0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc000c00 tx=0x7fc4cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:55.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4c5ffb700 1 -- 172.21.15.181:0/3271144093 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc004570 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4c5ffb700 1 -- 172.21.15.181:0/3271144093 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc01f070 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d0158070 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4c5ffb700 1 -- 172.21.15.181:0/3271144093 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0038c0 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.738+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d00c4690 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.739+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d00c34e0 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.740+0000 7fc4c5ffb700 1 -- 172.21.15.181:0/3271144093 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 143) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc4cc021030 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.740+0000 7fc4c5ffb700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc4b406ada0 0x7fc4b406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:55.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.740+0000 7fc4d528b700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc4b406ada0 0x7fc4b406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:55.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.740+0000 7fc4c5ffb700 1 -- 172.21.15.181:0/3271144093 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fc4cc08aeb0 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.741+0000 7fc4d528b700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc4b406ada0 0x7fc4b406d260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000f60 tx=0x7fc4c0009040 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:55.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.745+0000 7fc4c5ffb700 1 -- 172.21.15.181:0/3271144093 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc4cc055c10 con 0x7fc4d00cbfd0 2023-10-16T11:19:55.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.962+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4d006d7e0 con 0x7fc4b406ada0 2023-10-16T11:19:55.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.970+0000 7fc4c5ffb700 1 -- 172.21.15.181:0/3271144093 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+353 (secure 0 0 0) 0x7fc4c00012c0 con 0x7fc4b406ada0 2023-10-16T11:19:55.971 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:55.971 INFO:teuthology.orchestra.run.smithi181.stdout:[{"fs": "backup_fs", "subvol": null, "group": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:19:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T11:19:55.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc4b406ada0 msgr2=0x7fc4b406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc4b406ada0 0x7fc4b406d260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000f60 tx=0x7fc4c0009040 comp rx=0 tx=0).stop 2023-10-16T11:19:55.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00cbfd0 msgr2=0x7fc4d015cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:55.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00cbfd0 0x7fc4d015cfa0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc000c00 tx=0x7fc4cc004750 comp rx=0 tx=0).stop 2023-10-16T11:19:55.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 shutdown_connections 2023-10-16T11:19:55.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc4b406ada0 0x7fc4b406d260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d00b22a0 0x7fc4d0157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d015bb70 0x7fc4d015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 --2- 172.21.15.181:0/3271144093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4d00cbfd0 0x7fc4d015cfa0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:55.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 >> 172.21.15.181:0/3271144093 conn(0x7fc4d00b1780 msgr2=0x7fc4d00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:55.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 shutdown_connections 2023-10-16T11:19:55.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:55.972+0000 7fc4d74ef700 1 -- 172.21.15.181:0/3271144093 wait complete. 2023-10-16T11:19:55.985 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "group": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-16T00:00:00", "created": "2023-10-16T11:19:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-16T11:19:55.986 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:19:56.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.112+0000 7fd10b3e5700 1 Processor -- start 2023-10-16T11:19:56.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.112+0000 7fd10b3e5700 1 -- start start 2023-10-16T11:19:56.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd10b3e5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd10415f150 0x7fd1040cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd10b3e5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1040ce4f0 0x7fd1040b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd10b3e5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1040b2ae0 con 0x7fd1040b14a0 2023-10-16T11:19:56.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd10b3e5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1040ce970 con 0x7fd10415f150 2023-10-16T11:19:56.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd10b3e5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040b2dc0 con 0x7fd1040ce4f0 2023-10-16T11:19:56.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd109181700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd10415f150 0x7fd1040cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd108980700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1040ce4f0 0x7fd1040b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd109181700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd10415f150 0x7fd1040cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:33300/0 (socket says 172.21.15.181:33300) 2023-10-16T11:19:56.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd108980700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1040ce4f0 0x7fd1040b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:53508/0 (socket says 172.21.15.181:53508) 2023-10-16T11:19:56.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd109181700 1 -- 172.21.15.181:0/3441332077 learned_addr learned my addr 172.21.15.181:0/3441332077 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:56.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd109982700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd1040b14a0 0x7fd1040b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:51506/0 (socket says 172.21.15.181:51506) 2023-10-16T11:19:56.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd108980700 1 -- 172.21.15.181:0/3441332077 >> v2:172.21.15.190:3300/0 conn(0x7fd10415f150 msgr2=0x7fd1040cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:56.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd108980700 1 --2- 172.21.15.181:0/3441332077 >> v2:172.21.15.190:3300/0 conn(0x7fd10415f150 0x7fd1040cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd108980700 1 -- 172.21.15.181:0/3441332077 >> v1:172.21.15.190:6789/0 conn(0x7fd1040b14a0 legacy=0x7fd1040b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.113+0000 7fd108980700 1 -- 172.21.15.181:0/3441332077 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040cf9d0 con 0x7fd1040ce4f0 2023-10-16T11:19:56.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd108980700 1 --2- 172.21.15.181:0/3441332077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1040ce4f0 0x7fd1040b0f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd100009840 tx=0x7fd100009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b55f72d8455ec351 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd0fbfff700 1 -- 172.21.15.181:0/3441332077 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011830 con 0x7fd1040ce4f0 2023-10-16T11:19:56.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd0fbfff700 1 -- 172.21.15.181:0/3441332077 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100011990 con 0x7fd1040ce4f0 2023-10-16T11:19:56.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd0fbfff700 1 -- 172.21.15.181:0/3441332077 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011d20 con 0x7fd1040ce4f0 2023-10-16T11:19:56.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3441332077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1040ce4f0 msgr2=0x7fd1040b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3441332077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1040ce4f0 0x7fd1040b0f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd100009840 tx=0x7fd100009b50 comp rx=0 tx=0).stop 2023-10-16T11:19:56.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3441332077 shutdown_connections 2023-10-16T11:19:56.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3441332077 >> v2:172.21.15.190:3300/0 conn(0x7fd10415f150 0x7fd1040cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3441332077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1040ce4f0 0x7fd1040b0f60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3441332077 >> 172.21.15.181:0/3441332077 conn(0x7fd10409a930 msgr2=0x7fd1040d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:56.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.114+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3441332077 shutdown_connections 2023-10-16T11:19:56.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.115+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3441332077 wait complete. 2023-10-16T11:19:56.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 Processor -- start 2023-10-16T11:19:56.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 -- start start 2023-10-16T11:19:56.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1040b14a0 0x7fd1040c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1040ce4f0 0x7fd10415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd10415f150 0x7fd10415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040ca650 con 0x7fd1040b14a0 2023-10-16T11:19:56.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041f1b80 con 0x7fd1040ce4f0 2023-10-16T11:19:56.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd10b3e5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041f1010 con 0x7fd10415f150 2023-10-16T11:19:56.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.116+0000 7fd109181700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1040b14a0 0x7fd1040c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1040b14a0 0x7fd1040c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56488/0 (socket says 172.21.15.181:56488) 2023-10-16T11:19:56.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 -- 172.21.15.181:0/3339062042 learned_addr learned my addr 172.21.15.181:0/3339062042 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:56.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd108980700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1040ce4f0 0x7fd10415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109982700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd10415f150 0x7fd10415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 -- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd10415f150 msgr2=0x7fd10415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd10415f150 0x7fd10415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 -- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1040ce4f0 msgr2=0x7fd10415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1040ce4f0 0x7fd10415d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 -- 172.21.15.181:0/3339062042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000094a0 con 0x7fd1040b14a0 2023-10-16T11:19:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd109181700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1040b14a0 0x7fd1040c9e40 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e910 tx=0x7fd0f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40096a0 con 0x7fd1040b14a0 2023-10-16T11:19:56.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400c850 con 0x7fd1040b14a0 2023-10-16T11:19:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1041f1fa0 con 0x7fd1040b14a0 2023-10-16T11:19:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400cc00 con 0x7fd1040b14a0 2023-10-16T11:19:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.117+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1040ca380 con 0x7fd1040b14a0 2023-10-16T11:19:56.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.118+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd104067d20 con 0x7fd1040b14a0 2023-10-16T11:19:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.119+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 143) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd0f400cd60 con 0x7fd1040b14a0 2023-10-16T11:19:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.119+0000 7fd0f9ffb700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fd0e806ada0 0x7fd0e806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.119+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fd0f40897d0 con 0x7fd1040b14a0 2023-10-16T11:19:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.119+0000 7fd108980700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fd0e806ada0 0x7fd0e806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.120+0000 7fd108980700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fd0e806ada0 0x7fd0e806d260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd100009fd0 tx=0x7fd100008030 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.124+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd0f40544e0 con 0x7fd1040b14a0 2023-10-16T11:19:56.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.284+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 144) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fd0f40548b0 con 0x7fd1040b14a0 2023-10-16T11:19:56.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.346+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1040cff90 con 0x7fd0e806ada0 2023-10-16T11:19:56.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.408+0000 7fd0f9ffb700 1 -- 172.21.15.181:0/3339062042 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fd100011b00 con 0x7fd0e806ada0 2023-10-16T11:19:56.409 INFO:teuthology.orchestra.run.smithi181.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-10-16T11:19:56.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fd0e806ada0 msgr2=0x7fd0e806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fd0e806ada0 0x7fd0e806d260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd100009fd0 tx=0x7fd100008030 comp rx=0 tx=0).stop 2023-10-16T11:19:56.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1040b14a0 msgr2=0x7fd1040c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1040b14a0 0x7fd1040c9e40 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e910 tx=0x7fd0f400ecd0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 shutdown_connections 2023-10-16T11:19:56.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fd0e806ada0 0x7fd0e806d260 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd10415f150 0x7fd10415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1040b14a0 0x7fd1040c9e40 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 --2- 172.21.15.181:0/3339062042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1040ce4f0 0x7fd10415d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 >> 172.21.15.181:0/3339062042 conn(0x7fd10409a930 msgr2=0x7fd1040af460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:56.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 shutdown_connections 2023-10-16T11:19:56.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.410+0000 7fd10b3e5700 1 -- 172.21.15.181:0/3339062042 wait complete. 2023-10-16T11:19:56.424 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-10-16T11:19:56.424 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:19:56.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.550+0000 7fb546dfb700 1 Processor -- start 2023-10-16T11:19:56.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.550+0000 7fb546dfb700 1 -- start start 2023-10-16T11:19:56.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb546dfb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 0x7fb5400b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb546dfb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5400d23c0 con 0x7fb5400ce810 2023-10-16T11:19:56.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb546dfb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5400cd990 con 0x7fb54015eea0 2023-10-16T11:19:56.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb546dfb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400cdc70 con 0x7fb5400b0f70 2023-10-16T11:19:56.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb545398700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb5400ce810 0x7fb54015f620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39342/0 (socket says 172.21.15.181:39342) 2023-10-16T11:19:56.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb545398700 1 -- 172.21.15.181:0/484560821 learned_addr learned my addr 172.21.15.181:0/484560821 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:56.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851278006 0 0) 0x7fb5400d23c0 con 0x7fb5400ce810 2023-10-16T11:19:56.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5300035b0 con 0x7fb5400ce810 2023-10-16T11:19:56.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53ffff700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 0x7fb5400b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369149353 0 0) 0x7fb5300035b0 con 0x7fb5400ce810 2023-10-16T11:19:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 msgr2=0x7fb5400b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 0x7fb5400b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 >> v1:172.21.15.190:6789/0 conn(0x7fb54015eea0 legacy=0x7fb5400ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400cce60 con 0x7fb5400ce810 2023-10-16T11:19:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fb5380019f0 con 0x7fb5400ce810 2023-10-16T11:19:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 >> v1:172.21.15.181:6789/0 conn(0x7fb5400ce810 legacy=0x7fb54015f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb530003a30 0x7fb530003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 0x7fb530004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb530004d90 0x7fb530005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400cdc70 con 0x7fb530004d90 2023-10-16T11:19:56.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53000bac0 con 0x7fb530003a30 2023-10-16T11:19:56.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.551+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53000bc80 con 0x7fb5400b0f70 2023-10-16T11:19:56.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb545398700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb530004d90 0x7fb530005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb53ffff700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 0x7fb530004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb544b97700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb530003a30 0x7fb530003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb545398700 1 -- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 msgr2=0x7fb530004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb545398700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 0x7fb530004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb545398700 1 -- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb530003a30 msgr2=0x7fb530003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb545398700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb530003a30 0x7fb530003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb545398700 1 -- 172.21.15.181:0/484560821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5300035b0 con 0x7fb530004d90 2023-10-16T11:19:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb53ffff700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5400b0f70 0x7fb530004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb545398700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb530004d90 0x7fb530005210 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fb53800a800 tx=0x7fb530005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68434fd3154fbf2f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53800a2f0 con 0x7fb530004d90 2023-10-16T11:19:56.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538012600 con 0x7fb530004d90 2023-10-16T11:19:56.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.552+0000 7fb53f7fe700 1 -- 172.21.15.181:0/484560821 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380129b0 con 0x7fb530004d90 2023-10-16T11:19:56.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.553+0000 7fb546dfb700 1 -- 172.21.15.181:0/484560821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb530004d90 msgr2=0x7fb530005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.553+0000 7fb546dfb700 1 --2- 172.21.15.181:0/484560821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb530004d90 0x7fb530005210 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fb53800a800 tx=0x7fb530005880 comp rx=0 tx=0).stop 2023-10-16T11:19:56.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.553+0000 7fb545398700 1 -- 172.21.15.181:0/484560821 reap_dead start 2023-10-16T11:19:56.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.553+0000 7fb546dfb700 1 -- 172.21.15.181:0/484560821 shutdown_connections 2023-10-16T11:19:56.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.553+0000 7fb546dfb700 1 -- 172.21.15.181:0/484560821 >> 172.21.15.181:0/484560821 conn(0x7fb5400d1610 msgr2=0x7fb5400d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:56.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.553+0000 7fb546dfb700 1 -- 172.21.15.181:0/484560821 shutdown_connections 2023-10-16T11:19:56.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.553+0000 7fb546dfb700 1 -- 172.21.15.181:0/484560821 wait complete. 2023-10-16T11:19:56.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.554+0000 7fb546dfb700 1 Processor -- start 2023-10-16T11:19:56.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.554+0000 7fb546dfb700 1 -- start start 2023-10-16T11:19:56.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb546dfb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb530004d90 0x7fb5400ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb546dfb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5400c94a0 0x7fb5400c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb546dfb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5400b1430 0x7fb5400b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb546dfb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5401f41b0 con 0x7fb5400b1430 2023-10-16T11:19:56.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb546dfb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54006d8d0 con 0x7fb5400c94a0 2023-10-16T11:19:56.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb546dfb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400c86e0 con 0x7fb530004d90 2023-10-16T11:19:56.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5400b1430 0x7fb5400b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5400b1430 0x7fb5400b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56508/0 (socket says 172.21.15.181:56508) 2023-10-16T11:19:56.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 -- 172.21.15.181:0/3739118842 learned_addr learned my addr 172.21.15.181:0/3739118842 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:56.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb53ffff700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5400c94a0 0x7fb5400c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb544b97700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb530004d90 0x7fb5400ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 -- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb530004d90 msgr2=0x7fb5400ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb530004d90 0x7fb5400ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 -- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5400c94a0 msgr2=0x7fb5400c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5400c94a0 0x7fb5400c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 -- 172.21.15.181:0/3739118842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400c89c0 con 0x7fb5400b1430 2023-10-16T11:19:56.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb544b97700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb530004d90 0x7fb5400ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:56.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb53ffff700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5400c94a0 0x7fb5400c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:19:56.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb545398700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5400b1430 0x7fb5400b18b0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fb538012000 tx=0x7fb538001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb53d7fa700 1 -- 172.21.15.181:0/3739118842 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538025070 con 0x7fb5400b1430 2023-10-16T11:19:56.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb53d7fa700 1 -- 172.21.15.181:0/3739118842 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5380011c0 con 0x7fb5400b1430 2023-10-16T11:19:56.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5400c8ca0 con 0x7fb5400b1430 2023-10-16T11:19:56.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.555+0000 7fb53d7fa700 1 -- 172.21.15.181:0/3739118842 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380019f0 con 0x7fb5400b1430 2023-10-16T11:19:56.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.556+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5400ca7b0 con 0x7fb5400b1430 2023-10-16T11:19:56.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.556+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5400c91c0 con 0x7fb5400b1430 2023-10-16T11:19:56.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.557+0000 7fb53d7fa700 1 -- 172.21.15.181:0/3739118842 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 144) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fb53801e430 con 0x7fb5400b1430 2023-10-16T11:19:56.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.557+0000 7fb53d7fa700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fb52406b190 0x7fb52406d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.557+0000 7fb53d7fa700 1 -- 172.21.15.181:0/3739118842 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fb53808b7c0 con 0x7fb5400b1430 2023-10-16T11:19:56.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.557+0000 7fb544b97700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fb52406b190 0x7fb52406d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.558+0000 7fb544b97700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fb52406b190 0x7fb52406d650 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb5400b0430 tx=0x7fb534009040 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.562+0000 7fb53d7fa700 1 -- 172.21.15.181:0/3739118842 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb5400c91c0 con 0x7fb5400b1430 2023-10-16T11:19:56.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.780+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb54006da40 con 0x7fb52406b190 2023-10-16T11:19:56.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.839+0000 7fb53d7fa700 1 -- 172.21.15.181:0/3739118842 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7fb534005380 con 0x7fb52406b190 2023-10-16T11:19:56.840 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-16T11:19:56.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fb52406b190 msgr2=0x7fb52406d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fb52406b190 0x7fb52406d650 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb5400b0430 tx=0x7fb534009040 comp rx=0 tx=0).stop 2023-10-16T11:19:56.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5400b1430 msgr2=0x7fb5400b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5400b1430 0x7fb5400b18b0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fb538012000 tx=0x7fb538001d80 comp rx=0 tx=0).stop 2023-10-16T11:19:56.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 shutdown_connections 2023-10-16T11:19:56.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fb52406b190 0x7fb52406d650 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb530004d90 0x7fb5400ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5400b1430 0x7fb5400b18b0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 --2- 172.21.15.181:0/3739118842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5400c94a0 0x7fb5400c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 >> 172.21.15.181:0/3739118842 conn(0x7fb5400d1610 msgr2=0x7fb540070410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:56.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 shutdown_connections 2023-10-16T11:19:56.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.841+0000 7fb546dfb700 1 -- 172.21.15.181:0/3739118842 wait complete. 2023-10-16T11:19:56.854 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-16T11:19:56.855 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-16T11:19:56.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.980+0000 7f9b73436700 1 Processor -- start 2023-10-16T11:19:56.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.980+0000 7f9b73436700 1 -- start start 2023-10-16T11:19:56.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b73436700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b6c0d0fc0 0x7f9b6c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b73436700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b6c0ce9f0 0x7f9b6c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b73436700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c162650 con 0x7f9b6c0ce610 2023-10-16T11:19:56.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b73436700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6c162190 con 0x7f9b6c0ce9f0 2023-10-16T11:19:56.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b73436700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6c0d0230 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b6c0ce9f0 0x7f9b6c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b6c0ce9f0 0x7f9b6c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56514/0 (socket says 172.21.15.181:56514) 2023-10-16T11:19:56.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 -- 172.21.15.181:0/928510149 learned_addr learned my addr 172.21.15.181:0/928510149 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:56.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b709d1700 1 --2- 172.21.15.181:0/928510149 >> v2:172.21.15.190:3300/0 conn(0x7f9b6c0d0fc0 0x7f9b6c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 -- 172.21.15.181:0/928510149 >> v2:172.21.15.190:3300/0 conn(0x7f9b6c0d0fc0 msgr2=0x7f9b6c06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:56.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 --2- 172.21.15.181:0/928510149 >> v2:172.21.15.190:3300/0 conn(0x7f9b6c0d0fc0 0x7f9b6c06d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 -- 172.21.15.181:0/928510149 >> v1:172.21.15.190:6789/0 conn(0x7f9b6c0ce610 legacy=0x7f9b6c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:56.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 -- 172.21.15.181:0/928510149 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c162370 con 0x7f9b6c0ce9f0 2023-10-16T11:19:56.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b719d3700 1 --2- 172.21.15.181:0/928510149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0ce9f0 0x7f9b6c06a8a0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0c7520 tx=0x7f9b6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62835725af910eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b63fff700 1 -- 172.21.15.181:0/928510149 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6400e070 con 0x7f9b6c0ce9f0 2023-10-16T11:19:56.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b63fff700 1 -- 172.21.15.181:0/928510149 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b64003a20 con 0x7f9b6c0ce9f0 2023-10-16T11:19:56.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.981+0000 7f9b63fff700 1 -- 172.21.15.181:0/928510149 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64003dd0 con 0x7f9b6c0ce9f0 2023-10-16T11:19:56.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 -- 172.21.15.181:0/928510149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0ce9f0 msgr2=0x7f9b6c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 --2- 172.21.15.181:0/928510149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0ce9f0 0x7f9b6c06a8a0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0c7520 tx=0x7f9b6400b330 comp rx=0 tx=0).stop 2023-10-16T11:19:56.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 -- 172.21.15.181:0/928510149 shutdown_connections 2023-10-16T11:19:56.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 --2- 172.21.15.181:0/928510149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0ce9f0 0x7f9b6c06a8a0 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 --2- 172.21.15.181:0/928510149 >> v2:172.21.15.190:3300/0 conn(0x7f9b6c0d0fc0 0x7f9b6c06d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 -- 172.21.15.181:0/928510149 >> 172.21.15.181:0/928510149 conn(0x7f9b6c0d04a0 msgr2=0x7f9b6c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:56.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 -- 172.21.15.181:0/928510149 shutdown_connections 2023-10-16T11:19:56.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.982+0000 7f9b73436700 1 -- 172.21.15.181:0/928510149 wait complete. 2023-10-16T11:19:56.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.983+0000 7f9b73436700 1 Processor -- start 2023-10-16T11:19:56.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.983+0000 7f9b73436700 1 -- start start 2023-10-16T11:19:56.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b73436700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b6c0ce610 0x7f9b6c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b73436700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b6c0ce9f0 0x7f9b6c15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b73436700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0d0fc0 0x7f9b6c0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b73436700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c15b320 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b73436700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0c8e00 con 0x7f9b6c0ce610 2023-10-16T11:19:56.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b73436700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0c90a0 con 0x7f9b6c0ce9f0 2023-10-16T11:19:56.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0d0fc0 0x7f9b6c0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0d0fc0 0x7f9b6c0c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56522/0 (socket says 172.21.15.181:56522) 2023-10-16T11:19:56.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 -- 172.21.15.181:0/1590104467 learned_addr learned my addr 172.21.15.181:0/1590104467 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:56.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b711d2700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b6c0ce610 0x7f9b6c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b709d1700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b6c0ce9f0 0x7f9b6c15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 -- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b6c0ce9f0 msgr2=0x7f9b6c15ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b6c0ce9f0 0x7f9b6c15ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 -- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b6c0ce610 msgr2=0x7f9b6c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:56.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b6c0ce610 0x7f9b6c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:56.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 -- 172.21.15.181:0/1590104467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b64009b30 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b711d2700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b6c0ce610 0x7f9b6c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:19:56.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.984+0000 7f9b719d3700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0d0fc0 0x7f9b6c0c88c0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f9b64015040 tx=0x7f9b64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.985+0000 7f9b61ffb700 1 -- 172.21.15.181:0/1590104467 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64004440 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.985+0000 7f9b61ffb700 1 -- 172.21.15.181:0/1590104467 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b640045c0 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.985+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c0c76a0 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.985+0000 7f9b61ffb700 1 -- 172.21.15.181:0/1590104467 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64004970 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.985+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c0c7c60 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.986+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c067ab0 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.986+0000 7f9b61ffb700 1 -- 172.21.15.181:0/1590104467 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 144) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f9b64012430 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.987+0000 7f9b61ffb700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f9b5006b190 0x7f9b5006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:56.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.987+0000 7f9b61ffb700 1 -- 172.21.15.181:0/1590104467 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f9b6408b770 con 0x7f9b6c0d0fc0 2023-10-16T11:19:56.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.987+0000 7f9b711d2700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f9b5006b190 0x7f9b5006d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:56.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.987+0000 7f9b711d2700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f9b5006b190 0x7f9b5006d650 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001820 tx=0x7f9b68009040 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:56.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:56.991+0000 7f9b61ffb700 1 -- 172.21.15.181:0/1590104467 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9b64055db0 con 0x7f9b6c0d0fc0 2023-10-16T11:19:57.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.211+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 --> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b6c06f3d0 con 0x7f9b5006b190 2023-10-16T11:19:57.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.275+0000 7f9b61ffb700 1 -- 172.21.15.181:0/1590104467 <== mgr.11679 v2:172.21.15.181:6832/893959583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f9b68001410 con 0x7f9b5006b190 2023-10-16T11:19:57.276 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-10-16T11:19:57.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f9b5006b190 msgr2=0x7f9b5006d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:57.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f9b5006b190 0x7f9b5006d650 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001820 tx=0x7f9b68009040 comp rx=0 tx=0).stop 2023-10-16T11:19:57.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0d0fc0 msgr2=0x7f9b6c0c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:57.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0d0fc0 0x7f9b6c0c88c0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f9b64015040 tx=0x7f9b64004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 shutdown_connections 2023-10-16T11:19:57.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f9b5006b190 0x7f9b5006d650 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b6c0d0fc0 0x7f9b6c0c88c0 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b6c0ce610 0x7f9b6c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 --2- 172.21.15.181:0/1590104467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b6c0ce9f0 0x7f9b6c15ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 >> 172.21.15.181:0/1590104467 conn(0x7f9b6c0d04a0 msgr2=0x7f9b6c0cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:57.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.277+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 shutdown_connections 2023-10-16T11:19:57.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.278+0000 7f9b73436700 1 -- 172.21.15.181:0/1590104467 wait complete. 2023-10-16T11:19:57.291 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-10-16T11:19:57.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:19:57.293 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-16T11:19:57.312 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:19:57.456 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:57.458 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:19:57.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.475+0000 7f46a697c700 1 Processor -- start 2023-10-16T11:19:57.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.475+0000 7f46a697c700 1 -- start start 2023-10-16T11:19:57.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.475+0000 7f46a697c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46a00b2010 0x7f46a00cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:57.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.475+0000 7f46a697c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f46a00ce680 0x7f46a00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:57.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a697c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00ce1f0 con 0x7f46a0071be0 2023-10-16T11:19:57.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a697c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a0072090 con 0x7f46a00ce680 2023-10-16T11:19:57.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a697c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a00ccda0 con 0x7f46a00b2010 2023-10-16T11:19:57.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f46a00ce680 0x7f46a00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:57.479 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f46a00ce680 0x7f46a00ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56530/0 (socket says 172.21.15.181:56530) 2023-10-16T11:19:57.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 -- 172.21.15.181:0/2143913753 learned_addr learned my addr 172.21.15.181:0/2143913753 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:57.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f469ffff700 1 --2- 172.21.15.181:0/2143913753 >> v2:172.21.15.190:3300/0 conn(0x7f46a00b2010 0x7f46a00cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:57.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 -- 172.21.15.181:0/2143913753 >> v2:172.21.15.190:3300/0 conn(0x7f46a00b2010 msgr2=0x7f46a00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:57.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 --2- 172.21.15.181:0/2143913753 >> v2:172.21.15.190:3300/0 conn(0x7f46a00b2010 0x7f46a00cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.480 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 -- 172.21.15.181:0/2143913753 >> v1:172.21.15.190:6789/0 conn(0x7f46a0071be0 legacy=0x7f46a00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:57.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 -- 172.21.15.181:0/2143913753 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00727e0 con 0x7f46a00ce680 2023-10-16T11:19:57.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f46a4f19700 1 --2- 172.21.15.181:0/2143913753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00ce680 0x7f46a00ceaf0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f46a0070430 tx=0x7f469800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ff72d03d9afcbc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:57.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f469effd700 1 -- 172.21.15.181:0/2143913753 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800e070 con 0x7f46a00ce680 2023-10-16T11:19:57.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f469effd700 1 -- 172.21.15.181:0/2143913753 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800b910 con 0x7f46a00ce680 2023-10-16T11:19:57.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.476+0000 7f469effd700 1 -- 172.21.15.181:0/2143913753 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800bcc0 con 0x7f46a00ce680 2023-10-16T11:19:57.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 -- 172.21.15.181:0/2143913753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00ce680 msgr2=0x7f46a00ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:57.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 --2- 172.21.15.181:0/2143913753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00ce680 0x7f46a00ceaf0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f46a0070430 tx=0x7f469800b330 comp rx=0 tx=0).stop 2023-10-16T11:19:57.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 -- 172.21.15.181:0/2143913753 shutdown_connections 2023-10-16T11:19:57.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 --2- 172.21.15.181:0/2143913753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00ce680 0x7f46a00ceaf0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 --2- 172.21.15.181:0/2143913753 >> v2:172.21.15.190:3300/0 conn(0x7f46a00b2010 0x7f46a00cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 -- 172.21.15.181:0/2143913753 >> 172.21.15.181:0/2143913753 conn(0x7f46a00cd010 msgr2=0x7f46a00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:57.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 -- 172.21.15.181:0/2143913753 shutdown_connections 2023-10-16T11:19:57.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.477+0000 7f46a697c700 1 -- 172.21.15.181:0/2143913753 wait complete. 2023-10-16T11:19:57.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.479+0000 7f46a697c700 1 Processor -- start 2023-10-16T11:19:57.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.479+0000 7f46a697c700 1 -- start start 2023-10-16T11:19:57.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f46a697c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a0071be0 0x7f46a00ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:57.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f46a697c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00b2010 0x7f46a014e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:57.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f46a697c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a00ce680 0x7f46a00c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:57.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f46a697c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00af3c0 con 0x7f46a00b2010 2023-10-16T11:19:57.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f46a697c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00aca40 con 0x7f46a0071be0 2023-10-16T11:19:57.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f46a697c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a014e8b0 con 0x7f46a00ce680 2023-10-16T11:19:57.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a0071be0 0x7f46a00ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:57.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00b2010 0x7f46a014e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:57.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a0071be0 0x7f46a00ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:33350/0 (socket says 172.21.15.181:33350) 2023-10-16T11:19:57.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469ffff700 1 -- 172.21.15.181:0/3254411348 learned_addr learned my addr 172.21.15.181:0/3254411348 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:57.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f46a4f19700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a00ce680 0x7f46a00c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:57.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469f7fe700 1 -- 172.21.15.181:0/3254411348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a00ce680 msgr2=0x7f46a00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:57.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469f7fe700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a00ce680 0x7f46a00c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469f7fe700 1 -- 172.21.15.181:0/3254411348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a0071be0 msgr2=0x7f46a00ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:57.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469f7fe700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a0071be0 0x7f46a00ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:57.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469f7fe700 1 -- 172.21.15.181:0/3254411348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698009b30 con 0x7f46a00b2010 2023-10-16T11:19:57.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.480+0000 7f469f7fe700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00b2010 0x7f46a014e370 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f469400c400 tx=0x7f469400c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:57.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.481+0000 7f469cff9700 1 -- 172.21.15.181:0/3254411348 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46940052d0 con 0x7f46a00b2010 2023-10-16T11:19:57.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.481+0000 7f469cff9700 1 -- 172.21.15.181:0/3254411348 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4694005430 con 0x7f46a00b2010 2023-10-16T11:19:57.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.481+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a00c8f80 con 0x7f46a00b2010 2023-10-16T11:19:57.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.481+0000 7f469cff9700 1 -- 172.21.15.181:0/3254411348 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46940038a0 con 0x7f46a00b2010 2023-10-16T11:19:57.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.481+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a00aeb20 con 0x7f46a00b2010 2023-10-16T11:19:57.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.482+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a00acbb0 con 0x7f46a00b2010 2023-10-16T11:19:57.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.482+0000 7f469cff9700 1 -- 172.21.15.181:0/3254411348 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 144) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f4694003a00 con 0x7f46a00b2010 2023-10-16T11:19:57.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.483+0000 7f469cff9700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f468c06b190 0x7f468c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:57.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.483+0000 7f469cff9700 1 -- 172.21.15.181:0/3254411348 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f46940891e0 con 0x7f46a00b2010 2023-10-16T11:19:57.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.483+0000 7f469ffff700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f468c06b190 0x7f468c06d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:57.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.483+0000 7f469ffff700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f468c06b190 0x7f468c06d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4690009c00 tx=0x7f4690009210 comp rx=0 tx=0).ready entity=mgr.11679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:57.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.487+0000 7f469cff9700 1 -- 172.21.15.181:0/3254411348 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f46940539b0 con 0x7f46a00b2010 2023-10-16T11:19:57.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:57.737+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f46a00b52e0 con 0x7f46a00b2010 2023-10-16T11:19:58.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.292+0000 7f469cff9700 1 -- 172.21.15.181:0/3254411348 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v145) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4694003cb0 con 0x7f46a00b2010 2023-10-16T11:19:58.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f468c06b190 msgr2=0x7f468c06d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:58.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f468c06b190 0x7f468c06d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4690009c00 tx=0x7f4690009210 comp rx=0 tx=0).stop 2023-10-16T11:19:58.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00b2010 msgr2=0x7f46a014e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:58.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00b2010 0x7f46a014e370 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f469400c400 tx=0x7f469400c710 comp rx=0 tx=0).stop 2023-10-16T11:19:58.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 shutdown_connections 2023-10-16T11:19:58.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f468c06b190 0x7f468c06d650 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4690009c00 tx=0x7f4690009210 comp rx=0 tx=0).stop 2023-10-16T11:19:58.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46a00ce680 0x7f46a00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46a0071be0 0x7f46a00ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 --2- 172.21.15.181:0/3254411348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a00b2010 0x7f46a014e370 secure :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f469400c400 tx=0x7f469400c710 comp rx=0 tx=0).stop 2023-10-16T11:19:58.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.296+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 >> 172.21.15.181:0/3254411348 conn(0x7f46a00cd010 msgr2=0x7f46a00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:58.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.297+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 shutdown_connections 2023-10-16T11:19:58.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.297+0000 7f46a697c700 1 -- 172.21.15.181:0/3254411348 wait complete. 2023-10-16T11:19:58.313 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:19:58.314 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:19:58.352 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:19:58.399 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:19:58.400 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:58.400 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:58.419 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:19:58.419 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:19:58.446 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:58.444+0000 7f40a9ff5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:19:58.447 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:58.447+0000 7f45b410a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:19:58.464 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:19:58.512 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:58.509+0000 7f40a9ff5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:58.517 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:58.516+0000 7f45b410a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:19:58.584 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:58.582+0000 7f40a9ff5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:19:58.594 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:58.593+0000 7f45b410a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:19:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.630+0000 7f231f711700 1 Processor -- start 2023-10-16T11:19:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.630+0000 7f231f711700 1 -- start start 2023-10-16T11:19:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231f711700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815eab0 0x7f23180d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231f711700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23180cb0c0 0x7f231815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231f711700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231806ed00 con 0x7f231815e6d0 2023-10-16T11:19:58.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231f711700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23180d02d0 con 0x7f23180cb0c0 2023-10-16T11:19:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231f711700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231815ad40 con 0x7f231815eab0 2023-10-16T11:19:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231ccac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815eab0 0x7f23180d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231dcae700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23180cb0c0 0x7f231815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:58.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231ccac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815eab0 0x7f23180d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:53628/0 (socket says 172.21.15.181:53628) 2023-10-16T11:19:58.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231dcae700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23180cb0c0 0x7f231815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:33422/0 (socket says 172.21.15.181:33422) 2023-10-16T11:19:58.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231d4ad700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f231815e6d0 0x7f231815a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:51620/0 (socket says 172.21.15.181:51620) 2023-10-16T11:19:58.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.631+0000 7f231ccac700 1 -- 172.21.15.181:0/350835167 learned_addr learned my addr 172.21.15.181:0/350835167 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:58.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f231dcae700 1 -- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815eab0 msgr2=0x7f23180d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:58.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f231dcae700 1 --2- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815eab0 0x7f23180d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f231dcae700 1 -- 172.21.15.181:0/350835167 >> v1:172.21.15.190:6789/0 conn(0x7f231815e6d0 legacy=0x7f231815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:58.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f231dcae700 1 -- 172.21.15.181:0/350835167 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231806e0f0 con 0x7f23180cb0c0 2023-10-16T11:19:58.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f230ffff700 1 -- 172.21.15.181:0/350835167 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945549779 0 0) 0x7f231806ed00 con 0x7f231815e6d0 2023-10-16T11:19:58.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f231ccac700 1 --2- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815eab0 0x7f23180d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:58.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f231dcae700 1 --2- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231815a800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2310009d90 tx=0x7f231000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce63ece0cb1d003c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:58.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f230ffff700 1 -- 172.21.15.181:0/350835167 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231000e070 con 0x7f23180cb0c0 2023-10-16T11:19:58.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f230ffff700 1 -- 172.21.15.181:0/350835167 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231000bb20 con 0x7f23180cb0c0 2023-10-16T11:19:58.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.632+0000 7f230ffff700 1 -- 172.21.15.181:0/350835167 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310012430 con 0x7f23180cb0c0 2023-10-16T11:19:58.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 -- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 msgr2=0x7f231815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:58.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 --2- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231815a800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2310009d90 tx=0x7f231000b330 comp rx=0 tx=0).stop 2023-10-16T11:19:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 -- 172.21.15.181:0/350835167 shutdown_connections 2023-10-16T11:19:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 --2- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231815a800 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 --2- 172.21.15.181:0/350835167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815eab0 0x7f23180d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 -- 172.21.15.181:0/350835167 >> 172.21.15.181:0/350835167 conn(0x7f23180d04a0 msgr2=0x7f23180d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 -- 172.21.15.181:0/350835167 shutdown_connections 2023-10-16T11:19:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.633+0000 7f231f711700 1 -- 172.21.15.181:0/350835167 wait complete. 2023-10-16T11:19:58.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.634+0000 7f231f711700 1 Processor -- start 2023-10-16T11:19:58.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.634+0000 7f231f711700 1 -- start start 2023-10-16T11:19:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231f711700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231f711700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815e6d0 0x7f23180b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231f711700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231815eab0 0x7f2318158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231f711700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180b0660 con 0x7f231815eab0 2023-10-16T11:19:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231f711700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180afa20 con 0x7f23180cb0c0 2023-10-16T11:19:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231f711700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23180afcc0 con 0x7f231815e6d0 2023-10-16T11:19:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231815eab0 0x7f2318158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:58.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231d4ad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:58.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231ccac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815e6d0 0x7f23180b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:58.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231815eab0 0x7f2318158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56610/0 (socket says 172.21.15.181:56610) 2023-10-16T11:19:58.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 -- 172.21.15.181:0/4004523054 learned_addr learned my addr 172.21.15.181:0/4004523054 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:58.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231d4ad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231814f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:33424/0 (socket says 172.21.15.181:33424) 2023-10-16T11:19:58.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231ccac700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815e6d0 0x7f23180b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:53634/0 (socket says 172.21.15.181:53634) 2023-10-16T11:19:58.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 -- 172.21.15.181:0/4004523054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815e6d0 msgr2=0x7f23180b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:58.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815e6d0 0x7f23180b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 -- 172.21.15.181:0/4004523054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 msgr2=0x7f231814f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:58.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231814f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 -- 172.21.15.181:0/4004523054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314009550 con 0x7f231815eab0 2023-10-16T11:19:58.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231d4ad700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231814f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:58.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231dcae700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231815eab0 0x7f2318158340 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f2318071580 tx=0x7f2310004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:58.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f230dffb700 1 -- 172.21.15.181:0/4004523054 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310012430 con 0x7f231815eab0 2023-10-16T11:19:58.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f230dffb700 1 -- 172.21.15.181:0/4004523054 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2310020070 con 0x7f231815eab0 2023-10-16T11:19:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f230dffb700 1 -- 172.21.15.181:0/4004523054 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310004280 con 0x7f231815eab0 2023-10-16T11:19:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.635+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2310009a70 con 0x7f231815eab0 2023-10-16T11:19:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.636+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23180b0280 con 0x7f231815eab0 2023-10-16T11:19:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.637+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2318067d20 con 0x7f231815eab0 2023-10-16T11:19:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.637+0000 7f230dffb700 1 -- 172.21.15.181:0/4004523054 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f23100043e0 con 0x7f231815eab0 2023-10-16T11:19:58.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.637+0000 7f230dffb700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 0x7f22fc06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.637+0000 7f231d4ad700 1 -- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 msgr2=0x7f22fc06d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:19:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.637+0000 7f230dffb700 1 -- 172.21.15.181:0/4004523054 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f2310022070 con 0x7f231815eab0 2023-10-16T11:19:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.637+0000 7f231d4ad700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 0x7f22fc06d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:58.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.642+0000 7f230dffb700 1 -- 172.21.15.181:0/4004523054 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f23100563d0 con 0x7f231815eab0 2023-10-16T11:19:58.663 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:58.661+0000 7f40a9ff5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:19:58.678 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:58.678+0000 7f45b410a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:19:58.804 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:58.802+0000 7f40a9ff5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:19:58.822 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:58.822+0000 7f45b410a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:19:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.838+0000 7f231d4ad700 1 -- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 msgr2=0x7f22fc06d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:19:58.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.838+0000 7f231d4ad700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 0x7f22fc06d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.843+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23180b30b0 con 0x7f231815eab0 2023-10-16T11:19:58.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.844+0000 7f230dffb700 1 -- 172.21.15.181:0/4004523054 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+21701 (secure 0 0 0) 0x7f23100599f0 con 0x7f231815eab0 2023-10-16T11:19:58.846 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:19:58.846 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":221,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:53.266319+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:19:58.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.846+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 msgr2=0x7f22fc06d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:58.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.846+0000 7f231f711700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 0x7f22fc06d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.846+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231815eab0 msgr2=0x7f2318158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:58.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.846+0000 7f231f711700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231815eab0 0x7f2318158340 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f2318071580 tx=0x7f2310004ad0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 shutdown_connections 2023-10-16T11:19:58.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f22fc06b1d0 0x7f22fc06d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231815eab0 0x7f2318158340 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23180cb0c0 0x7f231814f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 --2- 172.21.15.181:0/4004523054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f231815e6d0 0x7f23180b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:58.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 >> 172.21.15.181:0/4004523054 conn(0x7f23180d04a0 msgr2=0x7f23180c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:58.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 shutdown_connections 2023-10-16T11:19:58.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:58.847+0000 7f231f711700 1 -- 172.21.15.181:0/4004523054 wait complete. 2023-10-16T11:19:58.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:19:21.355919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:19:21.889329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:19:33.658268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '219', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:19:34.982934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:19:58.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:19:59.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.028+0000 7f4cb1813700 1 Processor -- start 2023-10-16T11:19:59.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.028+0000 7f4cb1813700 1 -- start start 2023-10-16T11:19:59.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4cb1813700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4cb1813700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cac06b230 con 0x7f4cac15da70 2023-10-16T11:19:59.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4cb1813700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cac0cef00 con 0x7f4cac06ade0 2023-10-16T11:19:59.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4cb1813700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0cec20 con 0x7f4cac072760 2023-10-16T11:19:59.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4caaffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4cac15da70 0x7f4cac0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39506/0 (socket says 172.21.15.181:39506) 2023-10-16T11:19:59.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4caaffd700 1 -- 172.21.15.181:0/411286137 learned_addr learned my addr 172.21.15.181:0/411286137 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:59.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4caa7fc700 1 --2- 172.21.15.181:0/411286137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4ca9ffb700 1 -- 172.21.15.181:0/411286137 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001450563 0 0) 0x7f4c9c001110 con 0x7f4cac15da70 2023-10-16T11:19:59.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4ca9ffb700 1 -- 172.21.15.181:0/411286137 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c940035b0 con 0x7f4cac15da70 2023-10-16T11:19:59.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4ca9ffb700 1 -- 172.21.15.181:0/411286137 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652976023 0 0) 0x7f4cac0cef00 con 0x7f4cac06ade0 2023-10-16T11:19:59.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4ca9ffb700 1 -- 172.21.15.181:0/411286137 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c9c001110 con 0x7f4cac06ade0 2023-10-16T11:19:59.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4caa7fc700 1 -- 172.21.15.181:0/411286137 >> v1:172.21.15.190:6789/0 conn(0x7f4cac06ade0 legacy=0x7f4cac1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4caa7fc700 1 -- 172.21.15.181:0/411286137 >> v1:172.21.15.181:6789/0 conn(0x7f4cac15da70 legacy=0x7f4cac0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.029+0000 7f4caa7fc700 1 -- 172.21.15.181:0/411286137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac071c80 con 0x7f4cac072760 2023-10-16T11:19:59.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4caa7fc700 1 --2- 172.21.15.181:0/411286137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac06a8a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009870 tx=0x7f4ca0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269c3fd51dd51d79 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:59.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4ca9ffb700 1 -- 172.21.15.181:0/411286137 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0011830 con 0x7f4cac072760 2023-10-16T11:19:59.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4ca9ffb700 1 -- 172.21.15.181:0/411286137 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca0011990 con 0x7f4cac072760 2023-10-16T11:19:59.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4ca9ffb700 1 -- 172.21.15.181:0/411286137 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0011d20 con 0x7f4cac072760 2023-10-16T11:19:59.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4cb1813700 1 -- 172.21.15.181:0/411286137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 msgr2=0x7f4cac06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4cb1813700 1 --2- 172.21.15.181:0/411286137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac06a8a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009870 tx=0x7f4ca0009b80 comp rx=0 tx=0).stop 2023-10-16T11:19:59.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4cb1813700 1 -- 172.21.15.181:0/411286137 shutdown_connections 2023-10-16T11:19:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4cb1813700 1 --2- 172.21.15.181:0/411286137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac06a8a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4cb1813700 1 -- 172.21.15.181:0/411286137 >> 172.21.15.181:0/411286137 conn(0x7f4cac0cd010 msgr2=0x7f4cac0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4cb1813700 1 -- 172.21.15.181:0/411286137 shutdown_connections 2023-10-16T11:19:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.030+0000 7f4cb1813700 1 -- 172.21.15.181:0/411286137 wait complete. 2023-10-16T11:19:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 Processor -- start 2023-10-16T11:19:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 -- start start 2023-10-16T11:19:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cac06ade0 0x7f4cac1f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac15da70 0x7f4cac0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0c5a20 con 0x7f4cac15da70 2023-10-16T11:19:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac1f30a0 con 0x7f4cac06ade0 2023-10-16T11:19:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.032+0000 7f4cb1813700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0c4600 con 0x7f4cac072760 2023-10-16T11:19:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caaffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cac06ade0 0x7f4cac1f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac15da70 0x7f4cac0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac0c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:53694/0 (socket says 172.21.15.181:53694) 2023-10-16T11:19:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caaffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cac06ade0 0x7f4cac1f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:33484/0 (socket says 172.21.15.181:33484) 2023-10-16T11:19:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 -- 172.21.15.181:0/1135167681 learned_addr learned my addr 172.21.15.181:0/1135167681 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 -- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cac06ade0 msgr2=0x7f4cac1f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cac06ade0 0x7f4cac1f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 -- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac15da70 msgr2=0x7f4cac0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac15da70 0x7f4cac0ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 -- 172.21.15.181:0/1135167681 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca00094a0 con 0x7f4cac072760 2023-10-16T11:19:59.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caaffd700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cac06ade0 0x7f4cac1f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:59.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4cab7fe700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac15da70 0x7f4cac0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:59.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4caa7fc700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac0c40c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00090b0 tx=0x7f4ca0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:59.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4c93fff700 1 -- 172.21.15.181:0/1135167681 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca001a6e0 con 0x7f4cac072760 2023-10-16T11:19:59.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4c93fff700 1 -- 172.21.15.181:0/1135167681 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca001a840 con 0x7f4cac072760 2023-10-16T11:19:59.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cac0af830 con 0x7f4cac072760 2023-10-16T11:19:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4c93fff700 1 -- 172.21.15.181:0/1135167681 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca001abb0 con 0x7f4cac072760 2023-10-16T11:19:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.033+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac0acdc0 con 0x7f4cac072760 2023-10-16T11:19:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.034+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cac06f6c0 con 0x7f4cac072760 2023-10-16T11:19:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.035+0000 7f4c93fff700 1 -- 172.21.15.181:0/1135167681 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f4ca0022070 con 0x7f4cac072760 2023-10-16T11:19:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.035+0000 7f4c93fff700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 0x7f4c9806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.035+0000 7f4caaffd700 1 -- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 msgr2=0x7f4c9806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:19:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.035+0000 7f4c93fff700 1 -- 172.21.15.181:0/1135167681 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f4ca008b1d0 con 0x7f4cac072760 2023-10-16T11:19:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.035+0000 7f4caaffd700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 0x7f4c9806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.040+0000 7f4c93fff700 1 -- 172.21.15.181:0/1135167681 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4ca0055a00 con 0x7f4cac072760 2023-10-16T11:19:59.127 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:59.125+0000 7f40a9ff5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:19:59.175 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:59.174+0000 7f45b410a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:19:59.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.236+0000 7f4caaffd700 1 -- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 msgr2=0x7f4c9806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:19:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.236+0000 7f4caaffd700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 0x7f4c9806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:59.241 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:59.239+0000 7f40a9ff5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:19:59.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.253+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cac0af550 con 0x7f4cac072760 2023-10-16T11:19:59.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.254+0000 7f4c93fff700 1 -- 172.21.15.181:0/1135167681 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v221) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ca00150a0 con 0x7f4cac072760 2023-10-16T11:19:59.255 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:19:59.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 msgr2=0x7f4c9806d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 0x7f4c9806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 msgr2=0x7f4cac0c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac0c40c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00090b0 tx=0x7f4ca0004750 comp rx=0 tx=0).stop 2023-10-16T11:19:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 shutdown_connections 2023-10-16T11:19:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f4c9806b1d0 0x7f4c9806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac15da70 0x7f4cac0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cac06ade0 0x7f4cac1f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 --2- 172.21.15.181:0/1135167681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cac072760 0x7f4cac0c40c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 >> 172.21.15.181:0/1135167681 conn(0x7f4cac0cd010 msgr2=0x7f4cac0ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:59.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 shutdown_connections 2023-10-16T11:19:59.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.256+0000 7f4cb1813700 1 -- 172.21.15.181:0/1135167681 wait complete. 2023-10-16T11:19:59.271 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:19:59.296 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:59.295+0000 7f45b410a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:19:59.314 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:59.313+0000 7f40a9ff5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:19:59.373 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:59.372+0000 7f45b410a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:19:59.394 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:59.393+0000 7f40a9ff5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:19:59.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.397+0000 7f1498076700 1 Processor -- start 2023-10-16T11:19:59.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.398+0000 7f1498076700 1 -- start start 2023-10-16T11:19:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.398+0000 7f1498076700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f149015d9e0 0x7f14900d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1498076700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14900d07d0 0x7f14900cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1498076700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900d4180 con 0x7f14900cefa0 2023-10-16T11:19:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1498076700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14900d20c0 con 0x7f149015d9e0 2023-10-16T11:19:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1498076700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14900d1440 con 0x7f14900d07d0 2023-10-16T11:19:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f149015d9e0 0x7f14900d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f149015d9e0 0x7f14900d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56676/0 (socket says 172.21.15.181:56676) 2023-10-16T11:19:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 -- 172.21.15.181:0/1460147174 learned_addr learned my addr 172.21.15.181:0/1460147174 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1496613700 1 --2- 172.21.15.181:0/1460147174 >> v2:172.21.15.190:3300/0 conn(0x7f14900d07d0 0x7f14900cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 -- 172.21.15.181:0/1460147174 >> v2:172.21.15.190:3300/0 conn(0x7f14900d07d0 msgr2=0x7f14900cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 --2- 172.21.15.181:0/1460147174 >> v2:172.21.15.190:3300/0 conn(0x7f14900d07d0 0x7f14900cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 -- 172.21.15.181:0/1460147174 >> v1:172.21.15.190:6789/0 conn(0x7f14900cefa0 legacy=0x7f14900d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 -- 172.21.15.181:0/1460147174 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900d1720 con 0x7f149015d9e0 2023-10-16T11:19:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1495e12700 1 --2- 172.21.15.181:0/1460147174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149015d9e0 0x7f14900d24e0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f14900d1330 tx=0x7f148c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecafa78a4da6e58 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1494e10700 1 -- 172.21.15.181:0/1460147174 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c011830 con 0x7f149015d9e0 2023-10-16T11:19:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1494e10700 1 -- 172.21.15.181:0/1460147174 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c011990 con 0x7f149015d9e0 2023-10-16T11:19:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.399+0000 7f1494e10700 1 -- 172.21.15.181:0/1460147174 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c011d20 con 0x7f149015d9e0 2023-10-16T11:19:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 -- 172.21.15.181:0/1460147174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149015d9e0 msgr2=0x7f14900d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 --2- 172.21.15.181:0/1460147174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149015d9e0 0x7f14900d24e0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f14900d1330 tx=0x7f148c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 -- 172.21.15.181:0/1460147174 shutdown_connections 2023-10-16T11:19:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 --2- 172.21.15.181:0/1460147174 >> v2:172.21.15.190:3300/0 conn(0x7f14900d07d0 0x7f14900cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 --2- 172.21.15.181:0/1460147174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149015d9e0 0x7f14900d24e0 secure :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f14900d1330 tx=0x7f148c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 -- 172.21.15.181:0/1460147174 >> 172.21.15.181:0/1460147174 conn(0x7f149009cb40 msgr2=0x7f14900d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 -- 172.21.15.181:0/1460147174 shutdown_connections 2023-10-16T11:19:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.400+0000 7f1498076700 1 -- 172.21.15.181:0/1460147174 wait complete. 2023-10-16T11:19:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.401+0000 7f1498076700 1 Processor -- start 2023-10-16T11:19:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.401+0000 7f1498076700 1 -- start start 2023-10-16T11:19:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1498076700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14900cefa0 0x7f14900b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1498076700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14900d07d0 0x7f14900b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1498076700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f149015b070 0x7f14900b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1498076700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900cb750 con 0x7f14900cefa0 2023-10-16T11:19:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1498076700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b4ce0 con 0x7f14900d07d0 2023-10-16T11:19:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1498076700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b4720 con 0x7f149015b070 2023-10-16T11:19:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495e12700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14900cefa0 0x7f14900b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495e12700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14900cefa0 0x7f14900b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56682/0 (socket says 172.21.15.181:56682) 2023-10-16T11:19:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495e12700 1 -- 172.21.15.181:0/1955418166 learned_addr learned my addr 172.21.15.181:0/1955418166 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495611700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14900d07d0 0x7f14900b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1496613700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f149015b070 0x7f14900b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495e12700 1 -- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f149015b070 msgr2=0x7f14900b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495e12700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f149015b070 0x7f14900b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495e12700 1 -- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14900d07d0 msgr2=0x7f14900b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.402+0000 7f1495e12700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14900d07d0 0x7f14900b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.403+0000 7f1495e12700 1 -- 172.21.15.181:0/1955418166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009550 con 0x7f14900cefa0 2023-10-16T11:19:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.403+0000 7f1495e12700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14900cefa0 0x7f14900b8c30 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f148c000c00 tx=0x7f148c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.403+0000 7f14867fc700 1 -- 172.21.15.181:0/1955418166 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0044d0 con 0x7f14900cefa0 2023-10-16T11:19:59.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.403+0000 7f14867fc700 1 -- 172.21.15.181:0/1955418166 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c0036a0 con 0x7f14900cefa0 2023-10-16T11:19:59.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.403+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900b4a00 con 0x7f14900cefa0 2023-10-16T11:19:59.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.403+0000 7f14867fc700 1 -- 172.21.15.181:0/1955418166 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c003a50 con 0x7f14900cefa0 2023-10-16T11:19:59.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.403+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14900b9170 con 0x7f14900cefa0 2023-10-16T11:19:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.404+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1490067d20 con 0x7f14900cefa0 2023-10-16T11:19:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.404+0000 7f14867fc700 1 -- 172.21.15.181:0/1955418166 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f148c021070 con 0x7f14900cefa0 2023-10-16T11:19:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.405+0000 7f14867fc700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 0x7f147406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.405+0000 7f1495611700 1 -- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 msgr2=0x7f147406d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:19:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.405+0000 7f14867fc700 1 -- 172.21.15.181:0/1955418166 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f148c08b3f0 con 0x7f14900cefa0 2023-10-16T11:19:59.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.405+0000 7f1495611700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 0x7f147406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:59.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.413+0000 7f14867fc700 1 -- 172.21.15.181:0/1955418166 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f148c055a80 con 0x7f14900cefa0 2023-10-16T11:19:59.461 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:59.460+0000 7f45b410a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:19:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.606+0000 7f1495611700 1 -- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 msgr2=0x7f147406d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:19:59.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.606+0000 7f1495611700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 0x7f147406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:19:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.611+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14901602f0 con 0x7f14900cefa0 2023-10-16T11:19:59.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.612+0000 7f14867fc700 1 -- 172.21.15.181:0/1955418166 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 104+0+11 (secure 0 0 0) 0x7f148c0590a0 con 0x7f14900cefa0 2023-10-16T11:19:59.613 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:19:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 msgr2=0x7f147406d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:19:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 0x7f147406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14900cefa0 msgr2=0x7f14900b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14900cefa0 0x7f14900b8c30 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f148c000c00 tx=0x7f148c004750 comp rx=0 tx=0).stop 2023-10-16T11:19:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 shutdown_connections 2023-10-16T11:19:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f147406b1d0 0x7f147406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f149015b070 0x7f14900b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14900cefa0 0x7f14900b8c30 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 --2- 172.21.15.181:0/1955418166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14900d07d0 0x7f14900b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 >> 172.21.15.181:0/1955418166 conn(0x7f149009cb40 msgr2=0x7f149015b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 shutdown_connections 2023-10-16T11:19:59.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.614+0000 7f1498076700 1 -- 172.21.15.181:0/1955418166 wait complete. 2023-10-16T11:19:59.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:19:59.766 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:19:59.764+0000 7f40a9ff5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:19:59.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.794+0000 7fe01a505700 1 Processor -- start 2023-10-16T11:19:59.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.794+0000 7fe01a505700 1 -- start start 2023-10-16T11:19:59.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe01a505700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe014159e10 0x7fe0140ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe01a505700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe01a505700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01406e4b0 con 0x7fe01406bef0 2023-10-16T11:19:59.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe01a505700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01406ed50 con 0x7fe014159e10 2023-10-16T11:19:59.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe01a505700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140ceea0 con 0x7fe01415a1f0 2023-10-16T11:19:59.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe0137fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe01406bef0 0x7fe0140caba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39572/0 (socket says 172.21.15.181:39572) 2023-10-16T11:19:59.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe0137fe700 1 -- 172.21.15.181:0/4137446881 learned_addr learned my addr 172.21.15.181:0/4137446881 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:19:59.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe013fff700 1 --2- 172.21.15.181:0/4137446881 >> v2:172.21.15.190:3300/0 conn(0x7fe014159e10 0x7fe0140ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe018aa2700 1 --2- 172.21.15.181:0/4137446881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.795+0000 7fe012ffd700 1 -- 172.21.15.181:0/4137446881 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922996133 0 0) 0x7fe01406e4b0 con 0x7fe01406bef0 2023-10-16T11:19:59.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe012ffd700 1 -- 172.21.15.181:0/4137446881 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdffc0035b0 con 0x7fe01406bef0 2023-10-16T11:19:59.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe018aa2700 1 -- 172.21.15.181:0/4137446881 >> v2:172.21.15.190:3300/0 conn(0x7fe014159e10 msgr2=0x7fe0140ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:19:59.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe018aa2700 1 --2- 172.21.15.181:0/4137446881 >> v2:172.21.15.190:3300/0 conn(0x7fe014159e10 0x7fe0140ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe018aa2700 1 -- 172.21.15.181:0/4137446881 >> v1:172.21.15.181:6789/0 conn(0x7fe01406bef0 legacy=0x7fe0140caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe018aa2700 1 -- 172.21.15.181:0/4137446881 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01406e700 con 0x7fe01415a1f0 2023-10-16T11:19:59.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe013fff700 1 --2- 172.21.15.181:0/4137446881 >> v2:172.21.15.190:3300/0 conn(0x7fe014159e10 0x7fe0140ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:19:59.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe018aa2700 1 --2- 172.21.15.181:0/4137446881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415db90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009d90 tx=0x7fe00c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1679c560cf69a25 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:59.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe012ffd700 1 -- 172.21.15.181:0/4137446881 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00e070 con 0x7fe01415a1f0 2023-10-16T11:19:59.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe012ffd700 1 -- 172.21.15.181:0/4137446881 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c00bb20 con 0x7fe01415a1f0 2023-10-16T11:19:59.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.796+0000 7fe012ffd700 1 -- 172.21.15.181:0/4137446881 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c012430 con 0x7fe01415a1f0 2023-10-16T11:19:59.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 -- 172.21.15.181:0/4137446881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 msgr2=0x7fe01415db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 --2- 172.21.15.181:0/4137446881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415db90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009d90 tx=0x7fe00c00b330 comp rx=0 tx=0).stop 2023-10-16T11:19:59.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 -- 172.21.15.181:0/4137446881 shutdown_connections 2023-10-16T11:19:59.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 --2- 172.21.15.181:0/4137446881 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415db90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 --2- 172.21.15.181:0/4137446881 >> v2:172.21.15.190:3300/0 conn(0x7fe014159e10 0x7fe0140ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 -- 172.21.15.181:0/4137446881 >> 172.21.15.181:0/4137446881 conn(0x7fe0140cc730 msgr2=0x7fe0140ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:19:59.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 -- 172.21.15.181:0/4137446881 shutdown_connections 2023-10-16T11:19:59.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.797+0000 7fe01a505700 1 -- 172.21.15.181:0/4137446881 wait complete. 2023-10-16T11:19:59.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.798+0000 7fe01a505700 1 Processor -- start 2023-10-16T11:19:59.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.798+0000 7fe01a505700 1 -- start start 2023-10-16T11:19:59.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe01a505700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe01406bef0 0x7fe01414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe01a505700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe014159e10 0x7fe0140c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe01a505700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe01a505700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01414dde0 con 0x7fe014159e10 2023-10-16T11:19:59.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe01a505700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140c5c90 con 0x7fe01406bef0 2023-10-16T11:19:59.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe01a505700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01414e030 con 0x7fe01415a1f0 2023-10-16T11:19:59.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe014159e10 0x7fe0140c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe014159e10 0x7fe0140c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56734/0 (socket says 172.21.15.181:56734) 2023-10-16T11:19:59.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 -- 172.21.15.181:0/3302824960 learned_addr learned my addr 172.21.15.181:0/3302824960 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:19:59.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe018aa2700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe013fff700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe01406bef0 0x7fe01414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:19:59.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 -- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 msgr2=0x7fe01415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 -- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe01406bef0 msgr2=0x7fe01414edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:19:59.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe01406bef0 0x7fe01414edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:19:59.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 -- 172.21.15.181:0/3302824960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0040094a0 con 0x7fe014159e10 2023-10-16T11:19:59.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.799+0000 7fe0137fe700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe014159e10 0x7fe0140c5750 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fe008001eb0 tx=0x7fe00800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:19:59.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.800+0000 7fe010ff9700 1 -- 172.21.15.181:0/3302824960 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008005d70 con 0x7fe014159e10 2023-10-16T11:19:59.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.800+0000 7fe010ff9700 1 -- 172.21.15.181:0/3302824960 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00800ce70 con 0x7fe014159e10 2023-10-16T11:19:59.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.800+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c009a70 con 0x7fe014159e10 2023-10-16T11:19:59.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.800+0000 7fe010ff9700 1 -- 172.21.15.181:0/3302824960 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080072d0 con 0x7fe014159e10 2023-10-16T11:19:59.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.800+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01414e5f0 con 0x7fe014159e10 2023-10-16T11:19:59.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.801+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01415b240 con 0x7fe014159e10 2023-10-16T11:19:59.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.801+0000 7fe010ff9700 1 -- 172.21.15.181:0/3302824960 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fe008007430 con 0x7fe014159e10 2023-10-16T11:19:59.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.802+0000 7fe010ff9700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 0x7fe00006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:19:59.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.802+0000 7fe013fff700 1 -- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 msgr2=0x7fe00006d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:19:59.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.802+0000 7fe010ff9700 1 -- 172.21.15.181:0/3302824960 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fe008089420 con 0x7fe014159e10 2023-10-16T11:19:59.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.802+0000 7fe013fff700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 0x7fe00006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:19:59.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:19:59.806+0000 7fe010ff9700 1 -- 172.21.15.181:0/3302824960 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe008053b30 con 0x7fe014159e10 2023-10-16T11:19:59.847 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:19:59.846+0000 7f45b410a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:20:00.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.003+0000 7fe013fff700 1 -- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 msgr2=0x7fe00006d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:00.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.003+0000 7fe013fff700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 0x7fe00006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:00.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.005+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe01415d470 con 0x7fe014159e10 2023-10-16T11:20:00.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.006+0000 7fe010ff9700 1 -- 172.21.15.181:0/3302824960 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v221) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe008057150 con 0x7fe014159e10 2023-10-16T11:20:00.007 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:00.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 msgr2=0x7fe00006d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:00.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 0x7fe00006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe014159e10 msgr2=0x7fe0140c5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe014159e10 0x7fe0140c5750 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fe008001eb0 tx=0x7fe00800c7b0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 shutdown_connections 2023-10-16T11:20:00.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe00006b1d0 0x7fe00006d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe01415a1f0 0x7fe01415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe01406bef0 0x7fe01414edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 --2- 172.21.15.181:0/3302824960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe014159e10 0x7fe0140c5750 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.008+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 >> 172.21.15.181:0/3302824960 conn(0x7fe0140cc730 msgr2=0x7fe01415d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:00.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.009+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 shutdown_connections 2023-10-16T11:20:00.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.009+0000 7fe01a505700 1 -- 172.21.15.181:0/3302824960 wait complete. 2023-10-16T11:20:00.023 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:20:00.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.150+0000 7fccea9cd700 1 Processor -- start 2023-10-16T11:20:00.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.150+0000 7fccea9cd700 1 -- start start 2023-10-16T11:20:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.151+0000 7fccea9cd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcce40ce200 0x7fcce406be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.151+0000 7fccea9cd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce415dfb0 con 0x7fcce415dbd0 2023-10-16T11:20:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.151+0000 7fccea9cd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce40cd3a0 con 0x7fcce40cdd80 2023-10-16T11:20:00.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.151+0000 7fccea9cd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce40cd830 con 0x7fcce40ce200 2023-10-16T11:20:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.151+0000 7fcce3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcce415dbd0 0x7fcce40d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39588/0 (socket says 172.21.15.181:39588) 2023-10-16T11:20:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.151+0000 7fcce3fff700 1 -- 172.21.15.181:0/3907659997 learned_addr learned my addr 172.21.15.181:0/3907659997 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.151+0000 7fcce8f6a700 1 --2- 172.21.15.181:0/3907659997 >> v2:172.21.15.190:3300/0 conn(0x7fcce40ce200 0x7fcce406be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce2ffd700 1 -- 172.21.15.181:0/3907659997 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677197818 0 0) 0x7fccd4001110 con 0x7fcce415dbd0 2023-10-16T11:20:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce2ffd700 1 -- 172.21.15.181:0/3907659997 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcccc0035b0 con 0x7fcce415dbd0 2023-10-16T11:20:00.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce8f6a700 1 -- 172.21.15.181:0/3907659997 >> v1:172.21.15.190:6789/0 conn(0x7fcce40cdd80 legacy=0x7fcce406b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce8f6a700 1 -- 172.21.15.181:0/3907659997 >> v1:172.21.15.181:6789/0 conn(0x7fcce415dbd0 legacy=0x7fcce40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce2ffd700 1 -- 172.21.15.181:0/3907659997 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2411654739 0 0) 0x7fcccc0035b0 con 0x7fcce415dbd0 2023-10-16T11:20:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce8f6a700 1 -- 172.21.15.181:0/3907659997 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40ce6b0 con 0x7fcce40ce200 2023-10-16T11:20:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce8f6a700 1 --2- 172.21.15.181:0/3907659997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40ce200 0x7fcce406be00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009d90 tx=0x7fccdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68efc6bc88039bdf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:00.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce2ffd700 1 -- 172.21.15.181:0/3907659997 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00e070 con 0x7fcce40ce200 2023-10-16T11:20:00.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce2ffd700 1 -- 172.21.15.181:0/3907659997 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc003a20 con 0x7fcce40ce200 2023-10-16T11:20:00.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.152+0000 7fcce2ffd700 1 -- 172.21.15.181:0/3907659997 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4001800 con 0x7fcce40ce200 2023-10-16T11:20:00.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.153+0000 7fccea9cd700 1 -- 172.21.15.181:0/3907659997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40ce200 msgr2=0x7fcce406be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.153+0000 7fccea9cd700 1 --2- 172.21.15.181:0/3907659997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40ce200 0x7fcce406be00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009d90 tx=0x7fccdc00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:00.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.153+0000 7fccea9cd700 1 -- 172.21.15.181:0/3907659997 shutdown_connections 2023-10-16T11:20:00.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.153+0000 7fccea9cd700 1 --2- 172.21.15.181:0/3907659997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce40ce200 0x7fcce406be00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.153+0000 7fccea9cd700 1 -- 172.21.15.181:0/3907659997 >> 172.21.15.181:0/3907659997 conn(0x7fcce409a930 msgr2=0x7fcce40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:00.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.153+0000 7fccea9cd700 1 -- 172.21.15.181:0/3907659997 shutdown_connections 2023-10-16T11:20:00.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.153+0000 7fccea9cd700 1 -- 172.21.15.181:0/3907659997 wait complete. 2023-10-16T11:20:00.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.154+0000 7fccea9cd700 1 Processor -- start 2023-10-16T11:20:00.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.154+0000 7fccea9cd700 1 -- start start 2023-10-16T11:20:00.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fccea9cd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40cdd80 0x7fcce4148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fccea9cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce40ce200 0x7fcce41f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fccea9cd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce415dbd0 0x7fcce40ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fccea9cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4159ca0 con 0x7fcce40ce200 2023-10-16T11:20:00.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fccea9cd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce41f3050 con 0x7fcce415dbd0 2023-10-16T11:20:00.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fccea9cd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4159f10 con 0x7fcce40cdd80 2023-10-16T11:20:00.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce40ce200 0x7fcce41f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce40ce200 0x7fcce41f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56750/0 (socket says 172.21.15.181:56750) 2023-10-16T11:20:00.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce3fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40cdd80 0x7fcce4148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce37fe700 1 -- 172.21.15.181:0/1562826380 learned_addr learned my addr 172.21.15.181:0/1562826380 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:00.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce8f6a700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce415dbd0 0x7fcce40ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce37fe700 1 -- 172.21.15.181:0/1562826380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40cdd80 msgr2=0x7fcce4148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce37fe700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40cdd80 0x7fcce4148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.155+0000 7fcce37fe700 1 -- 172.21.15.181:0/1562826380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce415dbd0 msgr2=0x7fcce40ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fcce37fe700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce415dbd0 0x7fcce40ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fcce37fe700 1 -- 172.21.15.181:0/1562826380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc009a70 con 0x7fcce40ce200 2023-10-16T11:20:00.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fcce37fe700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce40ce200 0x7fcce41f2b10 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c3c0 tx=0x7fccd800c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:00.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fcce0ff9700 1 -- 172.21.15.181:0/1562826380 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80052d0 con 0x7fcce40ce200 2023-10-16T11:20:00.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fcce0ff9700 1 -- 172.21.15.181:0/1562826380 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8005430 con 0x7fcce40ce200 2023-10-16T11:20:00.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fcce0ff9700 1 -- 172.21.15.181:0/1562826380 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8006aa0 con 0x7fcce40ce200 2023-10-16T11:20:00.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce415a1f0 con 0x7fcce40ce200 2023-10-16T11:20:00.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.156+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce414b360 con 0x7fcce40ce200 2023-10-16T11:20:00.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.157+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce415a710 con 0x7fcce40ce200 2023-10-16T11:20:00.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.157+0000 7fcce0ff9700 1 -- 172.21.15.181:0/1562826380 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fccd8003680 con 0x7fcce40ce200 2023-10-16T11:20:00.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.158+0000 7fcce0ff9700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 0x7fccd006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.158+0000 7fcce3fff700 1 -- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 msgr2=0x7fccd006d6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:00.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.158+0000 7fcce0ff9700 1 -- 172.21.15.181:0/1562826380 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fccd80890c0 con 0x7fcce40ce200 2023-10-16T11:20:00.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.158+0000 7fcce3fff700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 0x7fccd006d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:00.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.162+0000 7fcce0ff9700 1 -- 172.21.15.181:0/1562826380 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcce415a710 con 0x7fcce40ce200 2023-10-16T11:20:00.332 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:00.330+0000 7f40a9ff5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:20:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.359+0000 7fcce3fff700 1 -- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 msgr2=0x7fccd006d6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:00.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.359+0000 7fcce3fff700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 0x7fccd006d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:00.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.397+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcce415a710 con 0x7fcce40ce200 2023-10-16T11:20:00.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.397+0000 7fcce0ff9700 1 -- 172.21.15.181:0/1562826380 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcce415a710 con 0x7fcce40ce200 2023-10-16T11:20:00.398 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.399+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 msgr2=0x7fccd006d6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.399+0000 7fccea9cd700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 0x7fccd006d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce40ce200 msgr2=0x7fcce41f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce40ce200 0x7fcce41f2b10 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c3c0 tx=0x7fccd800c6d0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 shutdown_connections 2023-10-16T11:20:00.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fccd006b220 0x7fccd006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcce40cdd80 0x7fcce4148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcce415dbd0 0x7fcce40ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 --2- 172.21.15.181:0/1562826380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce40ce200 0x7fcce41f2b10 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 >> 172.21.15.181:0/1562826380 conn(0x7fcce409a930 msgr2=0x7fcce40702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:00.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 shutdown_connections 2023-10-16T11:20:00.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.400+0000 7fccea9cd700 1 -- 172.21.15.181:0/1562826380 wait complete. 2023-10-16T11:20:00.403 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:00.399+0000 7f40a9ff5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:20:00.415 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:20:00.444 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:00.443+0000 7f45b410a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:20:00.518 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:00.517+0000 7f45b410a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:20:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.542+0000 7f8a1125f700 1 Processor -- start 2023-10-16T11:20:00.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.542+0000 7f8a1125f700 1 -- start start 2023-10-16T11:20:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a1125f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8a0c15f150 0x7f8a0c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a1125f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0b2ae0 con 0x7f8a0c0ce4f0 2023-10-16T11:20:00.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a1125f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0b2dc0 con 0x7f8a0c0cd170 2023-10-16T11:20:00.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a1125f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0ce9a0 con 0x7f8a0c15f150 2023-10-16T11:20:00.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a0a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8a0c0ce4f0 0x7f8a0c0b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39604/0 (socket says 172.21.15.181:39604) 2023-10-16T11:20:00.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a0a59c700 1 -- 172.21.15.181:0/23212595 learned_addr learned my addr 172.21.15.181:0/23212595 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:00.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a0ad9d700 1 --2- 172.21.15.181:0/23212595 >> v2:172.21.15.190:3300/0 conn(0x7f8a0c15f150 0x7f8a0c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740889503 0 0) 0x7f8a0c0b2ae0 con 0x7f8a0c0ce4f0 2023-10-16T11:20:00.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f40035b0 con 0x7f8a0c0ce4f0 2023-10-16T11:20:00.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967655504 0 0) 0x7f8a0c0b2dc0 con 0x7f8a0c0cd170 2023-10-16T11:20:00.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a0c0b2ae0 con 0x7f8a0c0cd170 2023-10-16T11:20:00.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322854767 0 0) 0x7f89f40035b0 con 0x7f8a0c0ce4f0 2023-10-16T11:20:00.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a0ad9d700 1 -- 172.21.15.181:0/23212595 >> v1:172.21.15.190:6789/0 conn(0x7f8a0c0cd170 legacy=0x7f8a0c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a0ad9d700 1 -- 172.21.15.181:0/23212595 >> v1:172.21.15.181:6789/0 conn(0x7f8a0c0ce4f0 legacy=0x7f8a0c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.543+0000 7f8a0ad9d700 1 -- 172.21.15.181:0/23212595 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0cf9d0 con 0x7f8a0c15f150 2023-10-16T11:20:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a0ad9d700 1 --2- 172.21.15.181:0/23212595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c15f150 0x7f8a0c0cdfb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009870 tx=0x7f89fc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=271ed5fe966bc27e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00001670 con 0x7f8a0c15f150 2023-10-16T11:20:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc004030 con 0x7f8a0c15f150 2023-10-16T11:20:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a09d9b700 1 -- 172.21.15.181:0/23212595 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0043c0 con 0x7f8a0c15f150 2023-10-16T11:20:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a1125f700 1 -- 172.21.15.181:0/23212595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c15f150 msgr2=0x7f8a0c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a1125f700 1 --2- 172.21.15.181:0/23212595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c15f150 0x7f8a0c0cdfb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009870 tx=0x7f89fc009b80 comp rx=0 tx=0).stop 2023-10-16T11:20:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a1125f700 1 -- 172.21.15.181:0/23212595 shutdown_connections 2023-10-16T11:20:00.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a1125f700 1 --2- 172.21.15.181:0/23212595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c15f150 0x7f8a0c0cdfb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a1125f700 1 -- 172.21.15.181:0/23212595 >> 172.21.15.181:0/23212595 conn(0x7f8a0c09a930 msgr2=0x7f8a0c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:00.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a1125f700 1 -- 172.21.15.181:0/23212595 shutdown_connections 2023-10-16T11:20:00.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.544+0000 7f8a1125f700 1 -- 172.21.15.181:0/23212595 wait complete. 2023-10-16T11:20:00.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.546+0000 7f8a1125f700 1 Processor -- start 2023-10-16T11:20:00.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.546+0000 7f8a1125f700 1 -- start start 2023-10-16T11:20:00.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c0cd170 0x7f8a0c0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a0c0ce4f0 0x7f8a0c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a0c15f150 0x7f8a0c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c15c600 con 0x7f8a0c15f150 2023-10-16T11:20:00.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cb5d0 con 0x7f8a0c0cd170 2023-10-16T11:20:00.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c06c4f0 con 0x7f8a0c0ce4f0 2023-10-16T11:20:00.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a0c15f150 0x7f8a0c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a0c15f150 0x7f8a0c06d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56768/0 (socket says 172.21.15.181:56768) 2023-10-16T11:20:00.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 -- 172.21.15.181:0/3108442534 learned_addr learned my addr 172.21.15.181:0/3108442534 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:00.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0a59c700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a0c0ce4f0 0x7f8a0c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0ad9d700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c0cd170 0x7f8a0c0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 -- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a0c0ce4f0 msgr2=0x7f8a0c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a0c0ce4f0 0x7f8a0c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 -- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c0cd170 msgr2=0x7f8a0c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c0cd170 0x7f8a0c0caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 -- 172.21.15.181:0/3108442534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0094a0 con 0x7f8a0c15f150 2023-10-16T11:20:00.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a0b59e700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a0c15f150 0x7f8a0c06d2b0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400cbc0 tx=0x7f8a0400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:00.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f89f37fe700 1 -- 172.21.15.181:0/3108442534 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003c00 con 0x7f8a0c15f150 2023-10-16T11:20:00.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c06c7d0 con 0x7f8a0c15f150 2023-10-16T11:20:00.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f89f37fe700 1 -- 172.21.15.181:0/3108442534 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a04003d60 con 0x7f8a0c15f150 2023-10-16T11:20:00.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f89f37fe700 1 -- 172.21.15.181:0/3108442534 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400aaa0 con 0x7f8a0c15f150 2023-10-16T11:20:00.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.547+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c06cd90 con 0x7f8a0c15f150 2023-10-16T11:20:00.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.548+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c06cfb0 con 0x7f8a0c15f150 2023-10-16T11:20:00.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.550+0000 7f89f37fe700 1 -- 172.21.15.181:0/3108442534 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f8a0400ac00 con 0x7f8a0c15f150 2023-10-16T11:20:00.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.550+0000 7f89f37fe700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 0x7f89f806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.550+0000 7f8a0ad9d700 1 -- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 msgr2=0x7f89f806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:00.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.550+0000 7f89f37fe700 1 -- 172.21.15.181:0/3108442534 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f8a0408aaf0 con 0x7f8a0c15f150 2023-10-16T11:20:00.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.550+0000 7f8a0ad9d700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 0x7f89f806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:00.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.554+0000 7f89f37fe700 1 -- 172.21.15.181:0/3108442534 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8a04055200 con 0x7f8a0c15f150 2023-10-16T11:20:00.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.751+0000 7f8a0ad9d700 1 -- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 msgr2=0x7f89f806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:00.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.751+0000 7f8a0ad9d700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 0x7f89f806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:00.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.756+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a0c067d20 con 0x7f8a0c15f150 2023-10-16T11:20:00.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.756+0000 7f89f37fe700 1 -- 172.21.15.181:0/3108442534 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v221) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8a04058820 con 0x7f8a0c15f150 2023-10-16T11:20:00.757 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:00.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 msgr2=0x7f89f806d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:00.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 0x7f89f806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a0c15f150 msgr2=0x7f8a0c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a0c15f150 0x7f8a0c06d2b0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400cbc0 tx=0x7f8a0400ced0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 shutdown_connections 2023-10-16T11:20:00.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f89f806b1d0 0x7f89f806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a0c15f150 0x7f8a0c06d2b0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a0c0cd170 0x7f8a0c0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 --2- 172.21.15.181:0/3108442534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a0c0ce4f0 0x7f8a0c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 >> 172.21.15.181:0/3108442534 conn(0x7f8a0c09a930 msgr2=0x7f8a0c1584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:00.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 shutdown_connections 2023-10-16T11:20:00.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.758+0000 7f8a1125f700 1 -- 172.21.15.181:0/3108442534 wait complete. 2023-10-16T11:20:00.772 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:20:00.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.934+0000 7fe040897700 1 Processor -- start 2023-10-16T11:20:00.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.934+0000 7fe040897700 1 -- start start 2023-10-16T11:20:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe040897700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe03c0cfc60 0x7fe03c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe040897700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe03c15bd50 0x7fe03c0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe040897700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c0d24f0 con 0x7fe03c0b05d0 2023-10-16T11:20:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe040897700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c15ccc0 con 0x7fe03c15bd50 2023-10-16T11:20:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe040897700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c0d0580 con 0x7fe03c0cfc60 2023-10-16T11:20:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe039d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe03c15bd50 0x7fe03c0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe039d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe03c15bd50 0x7fe03c0d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56776/0 (socket says 172.21.15.181:56776) 2023-10-16T11:20:00.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe039d9b700 1 -- 172.21.15.181:0/4277623952 learned_addr learned my addr 172.21.15.181:0/4277623952 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.935+0000 7fe03a59c700 1 --2- 172.21.15.181:0/4277623952 >> v2:172.21.15.190:3300/0 conn(0x7fe03c0cfc60 0x7fe03c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe039d9b700 1 -- 172.21.15.181:0/4277623952 >> v2:172.21.15.190:3300/0 conn(0x7fe03c0cfc60 msgr2=0x7fe03c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe039d9b700 1 --2- 172.21.15.181:0/4277623952 >> v2:172.21.15.190:3300/0 conn(0x7fe03c0cfc60 0x7fe03c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe039d9b700 1 -- 172.21.15.181:0/4277623952 >> v1:172.21.15.190:6789/0 conn(0x7fe03c0b05d0 legacy=0x7fe03c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe039d9b700 1 -- 172.21.15.181:0/4277623952 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c15c320 con 0x7fe03c15bd50 2023-10-16T11:20:00.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe039d9b700 1 --2- 172.21.15.181:0/4277623952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0d0040 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fe03c08bc90 tx=0x7fe030009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34b3bebf36c6f08 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:00.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe03959a700 1 -- 172.21.15.181:0/4277623952 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030011830 con 0x7fe03c15bd50 2023-10-16T11:20:00.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe03959a700 1 -- 172.21.15.181:0/4277623952 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe030011990 con 0x7fe03c15bd50 2023-10-16T11:20:00.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.936+0000 7fe03959a700 1 -- 172.21.15.181:0/4277623952 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030011d20 con 0x7fe03c15bd50 2023-10-16T11:20:00.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 -- 172.21.15.181:0/4277623952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 msgr2=0x7fe03c0d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 --2- 172.21.15.181:0/4277623952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0d0040 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fe03c08bc90 tx=0x7fe030009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 -- 172.21.15.181:0/4277623952 shutdown_connections 2023-10-16T11:20:00.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 --2- 172.21.15.181:0/4277623952 >> v2:172.21.15.190:3300/0 conn(0x7fe03c0cfc60 0x7fe03c0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 --2- 172.21.15.181:0/4277623952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0d0040 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 -- 172.21.15.181:0/4277623952 >> 172.21.15.181:0/4277623952 conn(0x7fe03c09a9a0 msgr2=0x7fe03c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:00.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 -- 172.21.15.181:0/4277623952 shutdown_connections 2023-10-16T11:20:00.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.937+0000 7fe040897700 1 -- 172.21.15.181:0/4277623952 wait complete. 2023-10-16T11:20:00.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.938+0000 7fe040897700 1 Processor -- start 2023-10-16T11:20:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.938+0000 7fe040897700 1 -- start start 2023-10-16T11:20:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe040897700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe03c0b05d0 0x7fe03c0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe040897700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe03c0cfc60 0x7fe03c14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe040897700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe040897700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0c8c90 con 0x7fe03c15bd50 2023-10-16T11:20:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe040897700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c1f4b60 con 0x7fe03c0cfc60 2023-10-16T11:20:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe040897700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c1f4e40 con 0x7fe03c0b05d0 2023-10-16T11:20:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56782/0 (socket says 172.21.15.181:56782) 2023-10-16T11:20:00.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 -- 172.21.15.181:0/2461169852 learned_addr learned my addr 172.21.15.181:0/2461169852 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:00.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03a59c700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe03c0b05d0 0x7fe03c0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe039d9b700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe03c0cfc60 0x7fe03c14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:00.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 -- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe03c0b05d0 msgr2=0x7fe03c0c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe03c0b05d0 0x7fe03c0c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 -- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe03c0cfc60 msgr2=0x7fe03c14fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:00.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe03c0cfc60 0x7fe03c14fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:00.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.939+0000 7fe03ad9d700 1 -- 172.21.15.181:0/2461169852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe030009550 con 0x7fe03c15bd50 2023-10-16T11:20:00.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.940+0000 7fe03ad9d700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0b1a60 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fe03400cc50 tx=0x7fe03400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:00.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.940+0000 7fe02affd700 1 -- 172.21.15.181:0/2461169852 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034003c00 con 0x7fe03c15bd50 2023-10-16T11:20:00.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.940+0000 7fe02affd700 1 -- 172.21.15.181:0/2461169852 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034003d60 con 0x7fe03c15bd50 2023-10-16T11:20:00.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.940+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c1f5120 con 0x7fe03c15bd50 2023-10-16T11:20:00.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.940+0000 7fe02affd700 1 -- 172.21.15.181:0/2461169852 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400aab0 con 0x7fe03c15bd50 2023-10-16T11:20:00.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.940+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c1f56e0 con 0x7fe03c15bd50 2023-10-16T11:20:00.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.941+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c158620 con 0x7fe03c15bd50 2023-10-16T11:20:00.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.941+0000 7fe02affd700 1 -- 172.21.15.181:0/2461169852 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fe03400ac10 con 0x7fe03c15bd50 2023-10-16T11:20:00.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.942+0000 7fe02affd700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 0x7fe01c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:00.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.942+0000 7fe03a59c700 1 -- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 msgr2=0x7fe01c06d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:00.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.942+0000 7fe02affd700 1 -- 172.21.15.181:0/2461169852 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fe03408a060 con 0x7fe03c15bd50 2023-10-16T11:20:00.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.942+0000 7fe03a59c700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 0x7fe01c06d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:00.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:00.946+0000 7fe02affd700 1 -- 172.21.15.181:0/2461169852 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe03c158620 con 0x7fe03c15bd50 2023-10-16T11:20:01.038 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:01.036+0000 7f40a9ff5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:01.115 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:01.114+0000 7f40a9ff5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:20:01.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.143+0000 7fe03a59c700 1 -- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 msgr2=0x7fe01c06d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:01.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.143+0000 7fe03a59c700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 0x7fe01c06d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:01.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.162+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe03c158620 con 0x7fe03c15bd50 2023-10-16T11:20:01.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.162+0000 7fe02affd700 1 -- 172.21.15.181:0/2461169852 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v221) v1 ==== 80+0+476 (secure 0 0 0) 0x7fe03c158620 con 0x7fe03c15bd50 2023-10-16T11:20:01.163 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1000076443 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.164 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3401982361 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.164 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/1781343309 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.164 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/1781343309 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.164 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/866682955 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.164 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/833955386 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.165 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1312491057 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.165 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2328723488 2023-10-17T11:19:53.266253+0000 2023-10-16T11:20:01.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 msgr2=0x7fe01c06d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:01.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 0x7fe01c06d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 msgr2=0x7fe03c0b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0b1a60 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fe03400cc50 tx=0x7fe03400cf60 comp rx=0 tx=0).stop 2023-10-16T11:20:01.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 shutdown_connections 2023-10-16T11:20:01.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fe01c06b1d0 0x7fe01c06d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe03c0b05d0 0x7fe03c0c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe03c15bd50 0x7fe03c0b1a60 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 --2- 172.21.15.181:0/2461169852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe03c0cfc60 0x7fe03c14fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.164+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 >> 172.21.15.181:0/2461169852 conn(0x7fe03c09a9a0 msgr2=0x7fe03c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:01.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.165+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 shutdown_connections 2023-10-16T11:20:01.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.165+0000 7fe040897700 1 -- 172.21.15.181:0/2461169852 wait complete. 2023-10-16T11:20:01.169 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T11:20:01.178 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:20:01.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:01.179 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:20:01.188 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:01.188+0000 7f45b410a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:01.218 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:20:01.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:01.219 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:20:01.261 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:01.260+0000 7f40a9ff5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:01.271 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:01.271+0000 7f45b410a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:20:01.276 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:20:01.276 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:01.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:20:01.294 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:20:01.295 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:01.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:20:01.333 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:20:01.334 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:01.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:20:01.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:01.424 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:01.424+0000 7f45b410a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:01.494 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:01.493+0000 7f40a9ff5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:20:01.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.495+0000 7f7f843b5700 1 Processor -- start 2023-10-16T11:20:01.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.496+0000 7f7f843b5700 1 -- start start 2023-10-16T11:20:01.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.496+0000 7f7f843b5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c15a7b0 0x7f7f7c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.496+0000 7f7f843b5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c15e390 con 0x7f7f7c0ca5e0 2023-10-16T11:20:01.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f843b5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c15e670 con 0x7f7f7c0c9a50 2023-10-16T11:20:01.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f843b5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0cf2b0 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f82952700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7f7c0ca5e0 0x7f7f7c0cb680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39670/0 (socket says 172.21.15.181:39670) 2023-10-16T11:20:01.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f82952700 1 -- 172.21.15.181:0/3555198158 learned_addr learned my addr 172.21.15.181:0/3555198158 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:01.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f82151700 1 --2- 172.21.15.181:0/3555198158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c15a7b0 0x7f7f7c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f8114f700 1 -- 172.21.15.181:0/3555198158 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918273580 0 0) 0x7f7f7c15e390 con 0x7f7f7c0ca5e0 2023-10-16T11:20:01.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f8114f700 1 -- 172.21.15.181:0/3555198158 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f680035b0 con 0x7f7f7c0ca5e0 2023-10-16T11:20:01.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f8114f700 1 -- 172.21.15.181:0/3555198158 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422946905 0 0) 0x7f7f7c15e670 con 0x7f7f7c0c9a50 2023-10-16T11:20:01.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f8114f700 1 -- 172.21.15.181:0/3555198158 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f7c15e390 con 0x7f7f7c0c9a50 2023-10-16T11:20:01.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f82151700 1 -- 172.21.15.181:0/3555198158 >> v1:172.21.15.190:6789/0 conn(0x7f7f7c0c9a50 legacy=0x7f7f7c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f82151700 1 -- 172.21.15.181:0/3555198158 >> v1:172.21.15.181:6789/0 conn(0x7f7f7c0ca5e0 legacy=0x7f7f7c0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f82151700 1 -- 172.21.15.181:0/3555198158 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c15e950 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f82151700 1 --2- 172.21.15.181:0/3555198158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c15a7b0 0x7f7f7c0cebb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009870 tx=0x7f7f6c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5b257ac08943eb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f8114f700 1 -- 172.21.15.181:0/3555198158 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011830 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.497+0000 7f7f8114f700 1 -- 172.21.15.181:0/3555198158 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c011990 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f8114f700 1 -- 172.21.15.181:0/3555198158 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c011d20 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f843b5700 1 -- 172.21.15.181:0/3555198158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c15a7b0 msgr2=0x7f7f7c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f843b5700 1 --2- 172.21.15.181:0/3555198158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c15a7b0 0x7f7f7c0cebb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009870 tx=0x7f7f6c009b80 comp rx=0 tx=0).stop 2023-10-16T11:20:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f843b5700 1 -- 172.21.15.181:0/3555198158 shutdown_connections 2023-10-16T11:20:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f843b5700 1 --2- 172.21.15.181:0/3555198158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c15a7b0 0x7f7f7c0cebb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f843b5700 1 -- 172.21.15.181:0/3555198158 >> 172.21.15.181:0/3555198158 conn(0x7f7f7c06aed0 msgr2=0x7f7f7c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f843b5700 1 -- 172.21.15.181:0/3555198158 shutdown_connections 2023-10-16T11:20:01.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.498+0000 7f7f843b5700 1 -- 172.21.15.181:0/3555198158 wait complete. 2023-10-16T11:20:01.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.499+0000 7f7f843b5700 1 Processor -- start 2023-10-16T11:20:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.499+0000 7f7f843b5700 1 -- start start 2023-10-16T11:20:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f843b5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0c9a50 0x7f7f7c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f843b5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c0ca5e0 0x7f7f7c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f843b5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7c15a7b0 0x7f7f7c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f843b5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c15ad20 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f843b5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c157ed0 con 0x7f7f7c0c9a50 2023-10-16T11:20:01.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f843b5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0c5950 con 0x7f7f7c0ca5e0 2023-10-16T11:20:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f82952700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7c15a7b0 0x7f7f7c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f82952700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7c15a7b0 0x7f7f7c159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56830/0 (socket says 172.21.15.181:56830) 2023-10-16T11:20:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f82151700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0c9a50 0x7f7f7c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f82952700 1 -- 172.21.15.181:0/2430023785 learned_addr learned my addr 172.21.15.181:0/2430023785 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:01.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.500+0000 7f7f81950700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c0ca5e0 0x7f7f7c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f82952700 1 -- 172.21.15.181:0/2430023785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c0ca5e0 msgr2=0x7f7f7c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f82952700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c0ca5e0 0x7f7f7c159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f82952700 1 -- 172.21.15.181:0/2430023785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0c9a50 msgr2=0x7f7f7c157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f82952700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0c9a50 0x7f7f7c157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f82952700 1 -- 172.21.15.181:0/2430023785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0094a0 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f82952700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7c15a7b0 0x7f7f7c159b70 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400cb10 tx=0x7f7f7400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:01.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f72ffd700 1 -- 172.21.15.181:0/2430023785 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400a9e0 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f72ffd700 1 -- 172.21.15.181:0/2430023785 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400ab60 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c0c5c30 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f72ffd700 1 -- 172.21.15.181:0/2430023785 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74012400 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.501+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c15af70 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.502+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c04ebb0 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.502+0000 7f7f72ffd700 1 -- 172.21.15.181:0/2430023785 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f7f740113a0 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.503+0000 7f7f72ffd700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 0x7f7f6006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.503+0000 7f7f82151700 1 -- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 msgr2=0x7f7f6006d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:01.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.503+0000 7f7f72ffd700 1 -- 172.21.15.181:0/2430023785 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f7f74089d90 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.503+0000 7f7f82151700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 0x7f7f6006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:01.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.507+0000 7f7f72ffd700 1 -- 172.21.15.181:0/2430023785 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7f7408d030 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.569 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:01.568+0000 7f40a9ff5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:20:01.667 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:01.666+0000 7f45b410a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:20:01.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.704+0000 7f7f82151700 1 -- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 msgr2=0x7f7f6006d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:01.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.704+0000 7f7f82151700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 0x7f7f6006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:01.744 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:01.743+0000 7f45b410a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:20:01.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.757+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f7c0c6290 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.757+0000 7f7f72ffd700 1 -- 172.21.15.181:0/2430023785 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+6165 (secure 0 0 0) 0x7f7f74054410 con 0x7f7f7c15a7b0 2023-10-16T11:20:01.759 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:01.760 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:20:01.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 msgr2=0x7f7f6006d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 0x7f7f6006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7c15a7b0 msgr2=0x7f7f7c159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7c15a7b0 0x7f7f7c159b70 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400cb10 tx=0x7f7f7400ce20 comp rx=0 tx=0).stop 2023-10-16T11:20:01.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 shutdown_connections 2023-10-16T11:20:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f7f6006b1d0 0x7f7f6006d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7c15a7b0 0x7f7f7c159b70 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0c9a50 0x7f7f7c157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 --2- 172.21.15.181:0/2430023785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f7c0ca5e0 0x7f7f7c159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.759+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 >> 172.21.15.181:0/2430023785 conn(0x7f7f7c06aed0 msgr2=0x7f7f7c0c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:01.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.760+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 shutdown_connections 2023-10-16T11:20:01.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.760+0000 7f7f843b5700 1 -- 172.21.15.181:0/2430023785 wait complete. 2023-10-16T11:20:01.765 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 117 2023-10-16T11:20:01.774 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:01.835 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:01.834+0000 7f40a9ff5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:20:01.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.900+0000 7f96f2893700 1 Processor -- start 2023-10-16T11:20:01.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.900+0000 7f96f2893700 1 -- start start 2023-10-16T11:20:01.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96f2893700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96ec161b60 0x7f96ec06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96f2893700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec161f40 0x7f96ec0d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96f2893700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec0d2560 con 0x7f96ec160d20 2023-10-16T11:20:01.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96f2893700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec162800 con 0x7f96ec161b60 2023-10-16T11:20:01.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96f2893700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0d43e0 con 0x7f96ec161f40 2023-10-16T11:20:01.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96f0e30700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f96ec160d20 0x7f96ec0b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39686/0 (socket says 172.21.15.181:39686) 2023-10-16T11:20:01.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96f0e30700 1 -- 172.21.15.181:0/3830516782 learned_addr learned my addr 172.21.15.181:0/3830516782 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:01.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96ebfff700 1 --2- 172.21.15.181:0/3830516782 >> v2:172.21.15.190:3300/0 conn(0x7f96ec161b60 0x7f96ec06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96eb7fe700 1 --2- 172.21.15.181:0/3830516782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec161f40 0x7f96ec0d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96eaffd700 1 -- 172.21.15.181:0/3830516782 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387562864 0 0) 0x7f96ec0d2560 con 0x7f96ec160d20 2023-10-16T11:20:01.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.901+0000 7f96eaffd700 1 -- 172.21.15.181:0/3830516782 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d40035b0 con 0x7f96ec160d20 2023-10-16T11:20:01.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eb7fe700 1 -- 172.21.15.181:0/3830516782 >> v2:172.21.15.190:3300/0 conn(0x7f96ec161b60 msgr2=0x7f96ec06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:01.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eb7fe700 1 --2- 172.21.15.181:0/3830516782 >> v2:172.21.15.190:3300/0 conn(0x7f96ec161b60 0x7f96ec06d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eb7fe700 1 -- 172.21.15.181:0/3830516782 >> v1:172.21.15.181:6789/0 conn(0x7f96ec160d20 legacy=0x7f96ec0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eb7fe700 1 -- 172.21.15.181:0/3830516782 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec06ce20 con 0x7f96ec161f40 2023-10-16T11:20:01.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96ebfff700 1 --2- 172.21.15.181:0/3830516782 >> v2:172.21.15.190:3300/0 conn(0x7f96ec161b60 0x7f96ec06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:01.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eb7fe700 1 --2- 172.21.15.181:0/3830516782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec161f40 0x7f96ec0d2020 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009840 tx=0x7f96e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfaff657f3ef0f40 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:01.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eaffd700 1 -- 172.21.15.181:0/3830516782 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011830 con 0x7f96ec161f40 2023-10-16T11:20:01.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eaffd700 1 -- 172.21.15.181:0/3830516782 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0011990 con 0x7f96ec161f40 2023-10-16T11:20:01.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.902+0000 7f96eaffd700 1 -- 172.21.15.181:0/3830516782 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011d00 con 0x7f96ec161f40 2023-10-16T11:20:01.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 -- 172.21.15.181:0/3830516782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec161f40 msgr2=0x7f96ec0d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 --2- 172.21.15.181:0/3830516782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec161f40 0x7f96ec0d2020 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009840 tx=0x7f96e0009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:01.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 -- 172.21.15.181:0/3830516782 shutdown_connections 2023-10-16T11:20:01.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 --2- 172.21.15.181:0/3830516782 >> v2:172.21.15.190:3300/0 conn(0x7f96ec161b60 0x7f96ec06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 --2- 172.21.15.181:0/3830516782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec161f40 0x7f96ec0d2020 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 -- 172.21.15.181:0/3830516782 >> 172.21.15.181:0/3830516782 conn(0x7f96ec09cba0 msgr2=0x7f96ec0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:01.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 -- 172.21.15.181:0/3830516782 shutdown_connections 2023-10-16T11:20:01.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.903+0000 7f96f2893700 1 -- 172.21.15.181:0/3830516782 wait complete. 2023-10-16T11:20:01.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.904+0000 7f96f2893700 1 Processor -- start 2023-10-16T11:20:01.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.904+0000 7f96f2893700 1 -- start start 2023-10-16T11:20:01.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f2893700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec160d20 0x7f96ec151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f2893700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96ec161b60 0x7f96ec0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f2893700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec161f40 0x7f96ec0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f2893700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0b0ea0 con 0x7f96ec161f40 2023-10-16T11:20:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f2893700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0b1740 con 0x7f96ec161b60 2023-10-16T11:20:01.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f2893700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0cbee0 con 0x7f96ec160d20 2023-10-16T11:20:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec161f40 0x7f96ec0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec161f40 0x7f96ec0c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56846/0 (socket says 172.21.15.181:56846) 2023-10-16T11:20:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 -- 172.21.15.181:0/888812567 learned_addr learned my addr 172.21.15.181:0/888812567 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96eb7fe700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96ec161b60 0x7f96ec0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96ebfff700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec160d20 0x7f96ec151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 -- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec160d20 msgr2=0x7f96ec151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec160d20 0x7f96ec151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 -- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96ec161b60 msgr2=0x7f96ec0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96ec161b60 0x7f96ec0c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:01.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 -- 172.21.15.181:0/888812567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc009430 con 0x7f96ec161f40 2023-10-16T11:20:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.905+0000 7f96f0e30700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec161f40 0x7f96ec0c8180 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c9f0 tx=0x7f96e400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.906+0000 7f96e8ff9700 1 -- 172.21.15.181:0/888812567 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400a9e0 con 0x7f96ec161f40 2023-10-16T11:20:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.906+0000 7f96e8ff9700 1 -- 172.21.15.181:0/888812567 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e400ce90 con 0x7f96ec161f40 2023-10-16T11:20:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.906+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e00094a0 con 0x7f96ec161f40 2023-10-16T11:20:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.906+0000 7f96e8ff9700 1 -- 172.21.15.181:0/888812567 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400ad60 con 0x7f96ec161f40 2023-10-16T11:20:01.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.906+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec0b13d0 con 0x7f96ec161f40 2023-10-16T11:20:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.907+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec0b18b0 con 0x7f96ec161f40 2023-10-16T11:20:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.908+0000 7f96e8ff9700 1 -- 172.21.15.181:0/888812567 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f96e401b3b0 con 0x7f96ec161f40 2023-10-16T11:20:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.908+0000 7f96e8ff9700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 0x7f96d806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.908+0000 7f96ebfff700 1 -- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 msgr2=0x7f96d806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:01.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.908+0000 7f96e8ff9700 1 -- 172.21.15.181:0/888812567 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f96e4089c00 con 0x7f96ec161f40 2023-10-16T11:20:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.908+0000 7f96ebfff700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 0x7f96d806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:01.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:01.913+0000 7f96e8ff9700 1 -- 172.21.15.181:0/888812567 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f96e4054310 con 0x7f96ec161f40 2023-10-16T11:20:02.025 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:02.024+0000 7f45b410a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:20:02.057 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:02.056+0000 7f40a9ff5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:20:02.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.109+0000 7f96ebfff700 1 -- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 msgr2=0x7f96d806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:02.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.109+0000 7f96ebfff700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 0x7f96d806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:02.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.110+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96ec04ebb0 con 0x7f96ec161f40 2023-10-16T11:20:02.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.112+0000 7f96e8ff9700 1 -- 172.21.15.181:0/888812567 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+21701 (secure 0 0 0) 0x7f96e405ce60 con 0x7f96ec161f40 2023-10-16T11:20:02.113 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:02.113 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":221,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:19:53.266319+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:02.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.114+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 msgr2=0x7f96d806d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:02.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.114+0000 7f96f2893700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 0x7f96d806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.114+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec161f40 msgr2=0x7f96ec0c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.114+0000 7f96f2893700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec161f40 0x7f96ec0c8180 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c9f0 tx=0x7f96e400cd00 comp rx=0 tx=0).stop 2023-10-16T11:20:02.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 shutdown_connections 2023-10-16T11:20:02.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f96d806b1d0 0x7f96d806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96ec160d20 0x7f96ec151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec161f40 0x7f96ec0c8180 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 --2- 172.21.15.181:0/888812567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96ec161b60 0x7f96ec0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 >> 172.21.15.181:0/888812567 conn(0x7f96ec09cba0 msgr2=0x7f96ec160280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:02.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 shutdown_connections 2023-10-16T11:20:02.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.115+0000 7f96f2893700 1 -- 172.21.15.181:0/888812567 wait complete. 2023-10-16T11:20:02.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:19:21.355919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:19:21.889329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:19:33.658268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '219', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:19:34.982934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:20:02.130 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:20:02.207 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:02.205+0000 7f40a9ff5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:20:02.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.255+0000 7f30ac002700 1 Processor -- start 2023-10-16T11:20:02.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.255+0000 7f30ac002700 1 -- start start 2023-10-16T11:20:02.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30ac002700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a40d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30ac002700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30a40d0600 0x7f30a40b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30ac002700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a40d2960 con 0x7f30a415d9e0 2023-10-16T11:20:02.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30ac002700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a40d20c0 con 0x7f30a40d0600 2023-10-16T11:20:02.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30ac002700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40d0a90 con 0x7f30a40d24e0 2023-10-16T11:20:02.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30a959d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a40d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30a9d9e700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f30a415d9e0 0x7f30a40cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:51872/0 (socket says 172.21.15.181:51872) 2023-10-16T11:20:02.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.256+0000 7f30a9d9e700 1 -- 172.21.15.181:0/2966668451 learned_addr learned my addr 172.21.15.181:0/2966668451 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:02.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30aa59f700 1 --2- 172.21.15.181:0/2966668451 >> v2:172.21.15.190:3300/0 conn(0x7f30a40d0600 0x7f30a40b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a8d9c700 1 -- 172.21.15.181:0/2966668451 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431256027 0 0) 0x7f30a40d2960 con 0x7f30a415d9e0 2023-10-16T11:20:02.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a8d9c700 1 -- 172.21.15.181:0/2966668451 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30900035b0 con 0x7f30a415d9e0 2023-10-16T11:20:02.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a959d700 1 -- 172.21.15.181:0/2966668451 >> v2:172.21.15.190:3300/0 conn(0x7f30a40d0600 msgr2=0x7f30a40b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:02.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a959d700 1 --2- 172.21.15.181:0/2966668451 >> v2:172.21.15.190:3300/0 conn(0x7f30a40d0600 0x7f30a40b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a959d700 1 -- 172.21.15.181:0/2966668451 >> v1:172.21.15.190:6789/0 conn(0x7f30a415d9e0 legacy=0x7f30a40cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a959d700 1 -- 172.21.15.181:0/2966668451 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40d4180 con 0x7f30a40d24e0 2023-10-16T11:20:02.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a959d700 1 --2- 172.21.15.181:0/2966668451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a40d00c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3094009840 tx=0x7f3094009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11214346f93777f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:02.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a8d9c700 1 -- 172.21.15.181:0/2966668451 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094011830 con 0x7f30a40d24e0 2023-10-16T11:20:02.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a8d9c700 1 -- 172.21.15.181:0/2966668451 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3094011990 con 0x7f30a40d24e0 2023-10-16T11:20:02.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.257+0000 7f30a8d9c700 1 -- 172.21.15.181:0/2966668451 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094011d20 con 0x7f30a40d24e0 2023-10-16T11:20:02.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 -- 172.21.15.181:0/2966668451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 msgr2=0x7f30a40d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 --2- 172.21.15.181:0/2966668451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a40d00c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3094009840 tx=0x7f3094009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:02.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 -- 172.21.15.181:0/2966668451 shutdown_connections 2023-10-16T11:20:02.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 --2- 172.21.15.181:0/2966668451 >> v2:172.21.15.190:3300/0 conn(0x7f30a40d0600 0x7f30a40b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 --2- 172.21.15.181:0/2966668451 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a40d00c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 -- 172.21.15.181:0/2966668451 >> 172.21.15.181:0/2966668451 conn(0x7f30a409cb40 msgr2=0x7f30a40d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 -- 172.21.15.181:0/2966668451 shutdown_connections 2023-10-16T11:20:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.258+0000 7f30ac002700 1 -- 172.21.15.181:0/2966668451 wait complete. 2023-10-16T11:20:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.259+0000 7f30ac002700 1 Processor -- start 2023-10-16T11:20:02.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.259+0000 7f30ac002700 1 -- start start 2023-10-16T11:20:02.266 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f45b410a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:20:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30ac002700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40d0600 0x7f30a41f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30ac002700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a41f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30ac002700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415d9e0 0x7f30a40c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30ac002700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40c8b80 con 0x7f30a415d9e0 2023-10-16T11:20:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30ac002700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a4156970 con 0x7f30a40d0600 2023-10-16T11:20:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30ac002700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40c6cb0 con 0x7f30a40d24e0 2023-10-16T11:20:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30aa59f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415d9e0 0x7f30a40c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30a9d9e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40d0600 0x7f30a41f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30aa59f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415d9e0 0x7f30a40c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56862/0 (socket says 172.21.15.181:56862) 2023-10-16T11:20:02.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30aa59f700 1 -- 172.21.15.181:0/2971272821 learned_addr learned my addr 172.21.15.181:0/2971272821 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:02.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.260+0000 7f30a959d700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a41f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30aa59f700 1 -- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 msgr2=0x7f30a41f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30aa59f700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a41f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30aa59f700 1 -- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40d0600 msgr2=0x7f30a41f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30aa59f700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40d0600 0x7f30a41f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30aa59f700 1 -- 172.21.15.181:0/2971272821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30940094a0 con 0x7f30a415d9e0 2023-10-16T11:20:02.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30a959d700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a41f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30aa59f700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415d9e0 0x7f30a40c7e00 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f309c00efc0 tx=0x7f309c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f309a7fc700 1 -- 172.21.15.181:0/2971272821 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00cad0 con 0x7f30a415d9e0 2023-10-16T11:20:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f309a7fc700 1 -- 172.21.15.181:0/2971272821 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c00f040 con 0x7f30a415d9e0 2023-10-16T11:20:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a41f4130 con 0x7f30a415d9e0 2023-10-16T11:20:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f309a7fc700 1 -- 172.21.15.181:0/2971272821 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00ce70 con 0x7f30a415d9e0 2023-10-16T11:20:02.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.261+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a41f46f0 con 0x7f30a415d9e0 2023-10-16T11:20:02.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.262+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a4067d20 con 0x7f30a415d9e0 2023-10-16T11:20:02.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.262+0000 7f309a7fc700 1 -- 172.21.15.181:0/2971272821 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f309c014480 con 0x7f30a415d9e0 2023-10-16T11:20:02.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.263+0000 7f309a7fc700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 0x7f308806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.263+0000 7f30a9d9e700 1 -- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 msgr2=0x7f308806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:02.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.263+0000 7f309a7fc700 1 -- 172.21.15.181:0/2971272821 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f309c08ab90 con 0x7f30a415d9e0 2023-10-16T11:20:02.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.263+0000 7f30a9d9e700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 0x7f308806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:02.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.267+0000 7f309a7fc700 1 -- 172.21.15.181:0/2971272821 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f309c055190 con 0x7f30a415d9e0 2023-10-16T11:20:02.321 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:02.319+0000 7f40a9ff5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:20:02.418 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:02.417+0000 7f45b410a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:20:02.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.464+0000 7f30a9d9e700 1 -- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 msgr2=0x7f308806d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:02.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.464+0000 7f30a9d9e700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 0x7f308806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:02.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.464+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30a40d0e50 con 0x7f30a415d9e0 2023-10-16T11:20:02.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.465+0000 7f309a7fc700 1 -- 172.21.15.181:0/2971272821 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v221) v1 ==== 93+0+10 (secure 0 0 0) 0x7f309c0587b0 con 0x7f30a415d9e0 2023-10-16T11:20:02.466 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:20:02.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 msgr2=0x7f308806d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:02.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 0x7f308806d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415d9e0 msgr2=0x7f30a40c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415d9e0 0x7f30a40c7e00 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f309c00efc0 tx=0x7f309c00c0c0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 shutdown_connections 2023-10-16T11:20:02.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f308806b1d0 0x7f308806d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415d9e0 0x7f30a40c7e00 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40d0600 0x7f30a41f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 --2- 172.21.15.181:0/2971272821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40d24e0 0x7f30a41f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 >> 172.21.15.181:0/2971272821 conn(0x7f30a409cb40 msgr2=0x7f30a415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:02.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 shutdown_connections 2023-10-16T11:20:02.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.468+0000 7f30ac002700 1 -- 172.21.15.181:0/2971272821 wait complete. 2023-10-16T11:20:02.476 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:02.475+0000 7f40a9ff5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:20:02.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:20:02.541 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:02.540+0000 7f45b410a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:20:02.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.612+0000 7f0d077bc700 1 Processor -- start 2023-10-16T11:20:02.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.612+0000 7f0d077bc700 1 -- start start 2023-10-16T11:20:02.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.613+0000 7f0d077bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d0015fee0 0x7f0d000ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.613+0000 7f0d077bc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0006e4b0 con 0x7f0d0015b850 2023-10-16T11:20:02.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.613+0000 7f0d077bc700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0006ed50 con 0x7f0d000665a0 2023-10-16T11:20:02.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.613+0000 7f0d077bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000cb0f0 con 0x7f0d0015fee0 2023-10-16T11:20:02.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.613+0000 7f0d05558700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0d0015b850 0x7f0d000c3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39714/0 (socket says 172.21.15.181:39714) 2023-10-16T11:20:02.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.613+0000 7f0d05558700 1 -- 172.21.15.181:0/830519344 learned_addr learned my addr 172.21.15.181:0/830519344 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:02.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0d05d59700 1 --2- 172.21.15.181:0/830519344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d0015fee0 0x7f0d000ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0cf7fff700 1 -- 172.21.15.181:0/830519344 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048376715 0 0) 0x7f0d0006e4b0 con 0x7f0d0015b850 2023-10-16T11:20:02.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0cf7fff700 1 -- 172.21.15.181:0/830519344 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cec0035b0 con 0x7f0d0015b850 2023-10-16T11:20:02.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0cf7fff700 1 -- 172.21.15.181:0/830519344 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904105592 0 0) 0x7f0d0006ed50 con 0x7f0d000665a0 2023-10-16T11:20:02.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0cf7fff700 1 -- 172.21.15.181:0/830519344 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d0006e4b0 con 0x7f0d000665a0 2023-10-16T11:20:02.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0d05d59700 1 -- 172.21.15.181:0/830519344 >> v1:172.21.15.190:6789/0 conn(0x7f0d000665a0 legacy=0x7f0d0006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0d05d59700 1 -- 172.21.15.181:0/830519344 >> v1:172.21.15.181:6789/0 conn(0x7f0d0015b850 legacy=0x7f0d000c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0d05d59700 1 -- 172.21.15.181:0/830519344 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d00066b80 con 0x7f0d0015fee0 2023-10-16T11:20:02.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0d05d59700 1 --2- 172.21.15.181:0/830519344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d0015fee0 0x7f0d000ca300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009dc0 tx=0x7f0cf800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e17eea6320a4230e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:02.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0cf7fff700 1 -- 172.21.15.181:0/830519344 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800e070 con 0x7f0d0015fee0 2023-10-16T11:20:02.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0cf7fff700 1 -- 172.21.15.181:0/830519344 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf800bb20 con 0x7f0d0015fee0 2023-10-16T11:20:02.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.614+0000 7f0cf7fff700 1 -- 172.21.15.181:0/830519344 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8012430 con 0x7f0d0015fee0 2023-10-16T11:20:02.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.615+0000 7f0d077bc700 1 -- 172.21.15.181:0/830519344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d0015fee0 msgr2=0x7f0d000ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.615+0000 7f0d077bc700 1 --2- 172.21.15.181:0/830519344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d0015fee0 0x7f0d000ca300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009dc0 tx=0x7f0cf800b330 comp rx=0 tx=0).stop 2023-10-16T11:20:02.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.615+0000 7f0d077bc700 1 -- 172.21.15.181:0/830519344 shutdown_connections 2023-10-16T11:20:02.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.615+0000 7f0d077bc700 1 --2- 172.21.15.181:0/830519344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d0015fee0 0x7f0d000ca300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.615+0000 7f0d077bc700 1 -- 172.21.15.181:0/830519344 >> 172.21.15.181:0/830519344 conn(0x7f0d000cc730 msgr2=0x7f0d000ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:02.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.615+0000 7f0d077bc700 1 -- 172.21.15.181:0/830519344 shutdown_connections 2023-10-16T11:20:02.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.615+0000 7f0d077bc700 1 -- 172.21.15.181:0/830519344 wait complete. 2023-10-16T11:20:02.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.617+0000 7f0d077bc700 1 Processor -- start 2023-10-16T11:20:02.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.617+0000 7f0d077bc700 1 -- start start 2023-10-16T11:20:02.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d077bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d000665a0 0x7f0d000c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d077bc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d0015b850 0x7f0d000b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d077bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d0015fee0 0x7f0d0014ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d077bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001418c0 con 0x7f0d0015fee0 2023-10-16T11:20:02.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d077bc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b3370 con 0x7f0d0015b850 2023-10-16T11:20:02.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d077bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b3640 con 0x7f0d000665a0 2023-10-16T11:20:02.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d0015fee0 0x7f0d0014ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d0015fee0 0x7f0d0014ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56876/0 (socket says 172.21.15.181:56876) 2023-10-16T11:20:02.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 -- 172.21.15.181:0/4152917974 learned_addr learned my addr 172.21.15.181:0/4152917974 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:02.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d04d57700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d0015b850 0x7f0d000b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05558700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d000665a0 0x7f0d000c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 -- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d000665a0 msgr2=0x7f0d000c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d000665a0 0x7f0d000c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 -- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d0015b850 msgr2=0x7f0d000b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d0015b850 0x7f0d000b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 -- 172.21.15.181:0/4152917974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009a70 con 0x7f0d0015fee0 2023-10-16T11:20:02.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d05d59700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d0015fee0 0x7f0d0014ebb0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b370 tx=0x7f0cf8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:02.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0cf5ffb700 1 -- 172.21.15.181:0/4152917974 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf801b810 con 0x7f0d0015fee0 2023-10-16T11:20:02.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0cf5ffb700 1 -- 172.21.15.181:0/4152917974 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8020070 con 0x7f0d0015fee0 2023-10-16T11:20:02.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d000b3920 con 0x7f0d0015fee0 2023-10-16T11:20:02.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.618+0000 7f0cf5ffb700 1 -- 172.21.15.181:0/4152917974 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf801bbb0 con 0x7f0d0015fee0 2023-10-16T11:20:02.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.619+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d000b3ee0 con 0x7f0d0015fee0 2023-10-16T11:20:02.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.620+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0006e6e0 con 0x7f0d0015fee0 2023-10-16T11:20:02.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.620+0000 7f0cf5ffb700 1 -- 172.21.15.181:0/4152917974 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f0cf801bd10 con 0x7f0d0015fee0 2023-10-16T11:20:02.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.620+0000 7f0cf5ffb700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 0x7f0ce406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.620+0000 7f0d05558700 1 -- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 msgr2=0x7f0ce406d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:02.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.620+0000 7f0cf5ffb700 1 -- 172.21.15.181:0/4152917974 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f0cf8022070 con 0x7f0d0015fee0 2023-10-16T11:20:02.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.620+0000 7f0d05558700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 0x7f0ce406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:02.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.625+0000 7f0cf5ffb700 1 -- 172.21.15.181:0/4152917974 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0d0006e6e0 con 0x7f0d0015fee0 2023-10-16T11:20:02.709 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:02.708+0000 7f45b410a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:20:02.740 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:02.739+0000 7f40a9ff5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:02.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.821+0000 7f0d05558700 1 -- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 msgr2=0x7f0ce406d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.821+0000 7f0d05558700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 0x7f0ce406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.821+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d0006e6b0 con 0x7f0d0015fee0 2023-10-16T11:20:02.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.822+0000 7f0cf5ffb700 1 -- 172.21.15.181:0/4152917974 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0cf802c0a0 con 0x7f0d0015fee0 2023-10-16T11:20:02.823 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:02.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 msgr2=0x7f0ce406d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 0x7f0ce406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d0015fee0 msgr2=0x7f0d0014ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d0015fee0 0x7f0d0014ebb0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b370 tx=0x7f0cf8004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 shutdown_connections 2023-10-16T11:20:02.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f0ce406b1d0 0x7f0ce406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d000665a0 0x7f0d000c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d0015fee0 0x7f0d0014ebb0 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 --2- 172.21.15.181:0/4152917974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d0015b850 0x7f0d000b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 >> 172.21.15.181:0/4152917974 conn(0x7f0d000cc730 msgr2=0x7f0d0015c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:02.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 shutdown_connections 2023-10-16T11:20:02.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.824+0000 7f0d077bc700 1 -- 172.21.15.181:0/4152917974 wait complete. 2023-10-16T11:20:02.839 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:20:02.902 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:02.901+0000 7f40a9ff5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:20:02.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.966+0000 7f522862d700 1 Processor -- start 2023-10-16T11:20:02.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.966+0000 7f522862d700 1 -- start start 2023-10-16T11:20:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f522862d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52200cc5e0 0x7f5220071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f522862d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f522015db30 0x7f52200cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f522862d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522015df80 con 0x7f5220159e50 2023-10-16T11:20:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f522862d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52200cb390 con 0x7f522015db30 2023-10-16T11:20:02.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f522862d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522015ea60 con 0x7f52200cc5e0 2023-10-16T11:20:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5226bca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f522015db30 0x7f52200cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5226bca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f522015db30 0x7f52200cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56886/0 (socket says 172.21.15.181:56886) 2023-10-16T11:20:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5226bca700 1 -- 172.21.15.181:0/1364454014 learned_addr learned my addr 172.21.15.181:0/1364454014 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5225bc8700 1 --2- 172.21.15.181:0/1364454014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52200cc5e0 0x7f5220071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5226bca700 1 -- 172.21.15.181:0/1364454014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52200cc5e0 msgr2=0x7f5220071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5226bca700 1 --2- 172.21.15.181:0/1364454014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52200cc5e0 0x7f5220071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5226bca700 1 -- 172.21.15.181:0/1364454014 >> v1:172.21.15.181:6789/0 conn(0x7f5220159e50 legacy=0x7f52200cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f5226bca700 1 -- 172.21.15.181:0/1364454014 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522015e1c0 con 0x7f522015db30 2023-10-16T11:20:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.967+0000 7f52253c7700 1 -- 172.21.15.181:0/1364454014 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712942001 0 0) 0x7f521c001110 con 0x7f5220159e50 2023-10-16T11:20:02.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f5226bca700 1 --2- 172.21.15.181:0/1364454014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f522015db30 0x7f52200cae50 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f521800d150 tx=0x7f521800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe3d7e8875bc456 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f52253c7700 1 -- 172.21.15.181:0/1364454014 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218010070 con 0x7f522015db30 2023-10-16T11:20:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f52253c7700 1 -- 172.21.15.181:0/1364454014 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521800dd10 con 0x7f522015db30 2023-10-16T11:20:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f52253c7700 1 -- 172.21.15.181:0/1364454014 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521800b310 con 0x7f522015db30 2023-10-16T11:20:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 -- 172.21.15.181:0/1364454014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f522015db30 msgr2=0x7f52200cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 --2- 172.21.15.181:0/1364454014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f522015db30 0x7f52200cae50 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f521800d150 tx=0x7f521800d460 comp rx=0 tx=0).stop 2023-10-16T11:20:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 -- 172.21.15.181:0/1364454014 shutdown_connections 2023-10-16T11:20:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 --2- 172.21.15.181:0/1364454014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f522015db30 0x7f52200cae50 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 --2- 172.21.15.181:0/1364454014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52200cc5e0 0x7f5220071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 -- 172.21.15.181:0/1364454014 >> 172.21.15.181:0/1364454014 conn(0x7f522006aed0 msgr2=0x7f522006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 -- 172.21.15.181:0/1364454014 shutdown_connections 2023-10-16T11:20:02.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.968+0000 7f522862d700 1 -- 172.21.15.181:0/1364454014 wait complete. 2023-10-16T11:20:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 Processor -- start 2023-10-16T11:20:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 -- start start 2023-10-16T11:20:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52200cc5e0 0x7f52201559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5220159e50 0x7f52200b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522015db30 0x7f52200b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52200ad890 con 0x7f52200cc5e0 2023-10-16T11:20:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52200ac880 con 0x7f522015db30 2023-10-16T11:20:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f522862d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52200acad0 con 0x7f5220159e50 2023-10-16T11:20:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.970+0000 7f52263c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52200cc5e0 0x7f52201559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5220159e50 0x7f52200b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f52263c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52200cc5e0 0x7f52201559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56888/0 (socket says 172.21.15.181:56888) 2023-10-16T11:20:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f52263c9700 1 -- 172.21.15.181:0/1901284535 learned_addr learned my addr 172.21.15.181:0/1901284535 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5226bca700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522015db30 0x7f52200b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5220159e50 0x7f52200b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:53916/0 (socket says 172.21.15.181:53916) 2023-10-16T11:20:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 -- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522015db30 msgr2=0x7f52200b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522015db30 0x7f52200b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 -- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52200cc5e0 msgr2=0x7f52201559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52200cc5e0 0x7f52201559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 -- 172.21.15.181:0/1901284535 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521800bd40 con 0x7f5220159e50 2023-10-16T11:20:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5225bc8700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5220159e50 0x7f52200b03b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f521000e8d0 tx=0x7f521000ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:02.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5216ffd700 1 -- 172.21.15.181:0/1901284535 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210013680 con 0x7f5220159e50 2023-10-16T11:20:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5216ffd700 1 -- 172.21.15.181:0/1901284535 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52100137e0 con 0x7f5220159e50 2023-10-16T11:20:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52200acdb0 con 0x7f5220159e50 2023-10-16T11:20:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f5216ffd700 1 -- 172.21.15.181:0/1901284535 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210013b90 con 0x7f5220159e50 2023-10-16T11:20:02.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.971+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52200ad370 con 0x7f5220159e50 2023-10-16T11:20:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.972+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522004ebb0 con 0x7f5220159e50 2023-10-16T11:20:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.973+0000 7f5216ffd700 1 -- 172.21.15.181:0/1901284535 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f5210013cf0 con 0x7f5220159e50 2023-10-16T11:20:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.973+0000 7f5216ffd700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 0x7f520406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.973+0000 7f52263c9700 1 -- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 msgr2=0x7f520406d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.973+0000 7f5216ffd700 1 -- 172.21.15.181:0/1901284535 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7f521008a010 con 0x7f5220159e50 2023-10-16T11:20:02.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.973+0000 7f52263c9700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 0x7f520406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:02.978+0000 7f5216ffd700 1 -- 172.21.15.181:0/1901284535 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5210054690 con 0x7f5220159e50 2023-10-16T11:20:02.987 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:02.986+0000 7f45b410a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:03.155 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.154+0000 7f45b410a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:20:03.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.174+0000 7f52263c9700 1 -- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 msgr2=0x7f520406d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:03.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.174+0000 7f52263c9700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 0x7f520406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:03.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.178+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52200acc10 con 0x7f5220159e50 2023-10-16T11:20:03.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.179+0000 7f5216ffd700 1 -- 172.21.15.181:0/1901284535 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v221) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5210017020 con 0x7f5220159e50 2023-10-16T11:20:03.180 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:03.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 msgr2=0x7f520406d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 0x7f520406d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5220159e50 msgr2=0x7f52200b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5220159e50 0x7f52200b03b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f521000e8d0 tx=0x7f521000ebe0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 shutdown_connections 2023-10-16T11:20:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7f520406b1d0 0x7f520406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f522015db30 0x7f52200b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52200cc5e0 0x7f52201559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 --2- 172.21.15.181:0/1901284535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5220159e50 0x7f52200b03b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 >> 172.21.15.181:0/1901284535 conn(0x7f522006aed0 msgr2=0x7f52200cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 shutdown_connections 2023-10-16T11:20:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.181+0000 7f522862d700 1 -- 172.21.15.181:0/1901284535 wait complete. 2023-10-16T11:20:03.195 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:20:03.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.320+0000 7fc762d2f700 1 Processor -- start 2023-10-16T11:20:03.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.320+0000 7fc762d2f700 1 -- start start 2023-10-16T11:20:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc762d2f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc75c158360 0x7fc75c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc762d2f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc762d2f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75c0cb340 con 0x7fc75c15ac20 2023-10-16T11:20:03.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc762d2f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75c0cf530 con 0x7fc75c158360 2023-10-16T11:20:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc762d2f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c15b0d0 con 0x7fc75c0ceb60 2023-10-16T11:20:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc760acb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc75c158360 0x7fc75c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc760acb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc75c158360 0x7fc75c06a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56898/0 (socket says 172.21.15.181:56898) 2023-10-16T11:20:03.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc7612cc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc760acb700 1 -- 172.21.15.181:0/3726929778 learned_addr learned my addr 172.21.15.181:0/3726929778 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.321+0000 7fc75bfff700 1 --1- 172.21.15.181:0/3726929778 >> v1:172.21.15.190:6789/0 conn(0x7fc75c15ac20 0x7fc75c15ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:51920/0 (socket says 172.21.15.181:51920) 2023-10-16T11:20:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc760acb700 1 -- 172.21.15.181:0/3726929778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 msgr2=0x7fc75c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc760acb700 1 --2- 172.21.15.181:0/3726929778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc760acb700 1 -- 172.21.15.181:0/3726929778 >> v1:172.21.15.190:6789/0 conn(0x7fc75c15ac20 legacy=0x7fc75c15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc75b7fe700 1 -- 172.21.15.181:0/3726929778 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526119767 0 0) 0x7fc75c0cb340 con 0x7fc75c15ac20 2023-10-16T11:20:03.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc760acb700 1 -- 172.21.15.181:0/3726929778 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c15b3b0 con 0x7fc75c158360 2023-10-16T11:20:03.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc760acb700 1 --2- 172.21.15.181:0/3726929778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c06a8a0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fc7500098f0 tx=0x7fc750009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef727a4ecbb461c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:03.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc75b7fe700 1 -- 172.21.15.181:0/3726929778 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750011830 con 0x7fc75c158360 2023-10-16T11:20:03.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc75b7fe700 1 -- 172.21.15.181:0/3726929778 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750011990 con 0x7fc75c158360 2023-10-16T11:20:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.322+0000 7fc75b7fe700 1 -- 172.21.15.181:0/3726929778 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750011d20 con 0x7fc75c158360 2023-10-16T11:20:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 -- 172.21.15.181:0/3726929778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 msgr2=0x7fc75c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3726929778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c06a8a0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fc7500098f0 tx=0x7fc750009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 -- 172.21.15.181:0/3726929778 shutdown_connections 2023-10-16T11:20:03.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3726929778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3726929778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c06a8a0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 -- 172.21.15.181:0/3726929778 >> 172.21.15.181:0/3726929778 conn(0x7fc75c0cc730 msgr2=0x7fc75c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 -- 172.21.15.181:0/3726929778 shutdown_connections 2023-10-16T11:20:03.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.323+0000 7fc762d2f700 1 -- 172.21.15.181:0/3726929778 wait complete. 2023-10-16T11:20:03.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.324+0000 7fc762d2f700 1 Processor -- start 2023-10-16T11:20:03.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.324+0000 7fc762d2f700 1 -- start start 2023-10-16T11:20:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc762d2f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc762d2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc762d2f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c15ac20 0x7fc75c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc762d2f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c0ae280 con 0x7fc75c158360 2023-10-16T11:20:03.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc762d2f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c0aee00 con 0x7fc75c15ac20 2023-10-16T11:20:03.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc762d2f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c0ae4d0 con 0x7fc75c0ceb60 2023-10-16T11:20:03.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56906/0 (socket says 172.21.15.181:56906) 2023-10-16T11:20:03.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 -- 172.21.15.181:0/3038080346 learned_addr learned my addr 172.21.15.181:0/3038080346 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:03.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc760acb700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc7612cc700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c15ac20 0x7fc75c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 -- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 msgr2=0x7fc75c156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 -- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c15ac20 msgr2=0x7fc75c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c15ac20 0x7fc75c159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc75bfff700 1 -- 172.21.15.181:0/3038080346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc750009550 con 0x7fc75c158360 2023-10-16T11:20:03.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.325+0000 7fc760acb700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:03.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.326+0000 7fc75bfff700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c159790 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c570 tx=0x7fc74800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:03.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.326+0000 7fc7597fa700 1 -- 172.21.15.181:0/3038080346 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748004c80 con 0x7fc75c158360 2023-10-16T11:20:03.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.326+0000 7fc7597fa700 1 -- 172.21.15.181:0/3038080346 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748004de0 con 0x7fc75c158360 2023-10-16T11:20:03.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.326+0000 7fc7597fa700 1 -- 172.21.15.181:0/3038080346 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748007310 con 0x7fc75c158360 2023-10-16T11:20:03.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.326+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c0ae7b0 con 0x7fc75c158360 2023-10-16T11:20:03.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.326+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c157460 con 0x7fc75c158360 2023-10-16T11:20:03.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.327+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75c0aef90 con 0x7fc75c158360 2023-10-16T11:20:03.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.327+0000 7fc7597fa700 1 -- 172.21.15.181:0/3038080346 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fc748007470 con 0x7fc75c158360 2023-10-16T11:20:03.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.328+0000 7fc7597fa700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 0x7fc74006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.328+0000 7fc760acb700 1 -- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 msgr2=0x7fc74006d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:03.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.328+0000 7fc7597fa700 1 -- 172.21.15.181:0/3038080346 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7fc748089c90 con 0x7fc75c158360 2023-10-16T11:20:03.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.328+0000 7fc760acb700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 0x7fc74006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:03.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.332+0000 7fc7597fa700 1 -- 172.21.15.181:0/3038080346 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc74808f020 con 0x7fc75c158360 2023-10-16T11:20:03.431 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:03.429+0000 7f40a9ff5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:20:03.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.529+0000 7fc760acb700 1 -- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 msgr2=0x7fc74006d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:03.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.529+0000 7fc760acb700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 0x7fc74006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:03.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.531+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc75c0cac40 con 0x7fc75c158360 2023-10-16T11:20:03.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.532+0000 7fc7597fa700 1 -- 172.21.15.181:0/3038080346 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc748054460 con 0x7fc75c158360 2023-10-16T11:20:03.533 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:03.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 msgr2=0x7fc74006d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:03.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 0x7fc74006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 msgr2=0x7fc75c159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c159790 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c570 tx=0x7fc74800c880 comp rx=0 tx=0).stop 2023-10-16T11:20:03.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 shutdown_connections 2023-10-16T11:20:03.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7fc74006b1d0 0x7fc74006d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc75c0ceb60 0x7fc75c156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc75c15ac20 0x7fc75c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 --2- 172.21.15.181:0/3038080346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc75c158360 0x7fc75c159790 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 >> 172.21.15.181:0/3038080346 conn(0x7fc75c0cc730 msgr2=0x7fc75c0afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:03.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 shutdown_connections 2023-10-16T11:20:03.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.534+0000 7fc762d2f700 1 -- 172.21.15.181:0/3038080346 wait complete. 2023-10-16T11:20:03.549 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:20:03.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.677+0000 7ff4533b8700 1 Processor -- start 2023-10-16T11:20:03.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.677+0000 7ff4533b8700 1 -- start start 2023-10-16T11:20:03.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff4533b8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff44c0d01d0 0x7ff44c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff4533b8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff44c0d1500 0x7ff44c0d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff4533b8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0cf580 con 0x7ff44c15ea40 2023-10-16T11:20:03.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff4533b8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0d4290 con 0x7ff44c0d1500 2023-10-16T11:20:03.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff4533b8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0ce970 con 0x7ff44c0d01d0 2023-10-16T11:20:03.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff451955700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff44c0d1500 0x7ff44c0d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff451955700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff44c0d1500 0x7ff44c0d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56918/0 (socket says 172.21.15.181:56918) 2023-10-16T11:20:03.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff450953700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff44c15ea40 0x7ff44c0d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39762/0 (socket says 172.21.15.181:39762) 2023-10-16T11:20:03.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff451955700 1 -- 172.21.15.181:0/3822792522 learned_addr learned my addr 172.21.15.181:0/3822792522 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:03.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.678+0000 7ff451154700 1 --2- 172.21.15.181:0/3822792522 >> v2:172.21.15.190:3300/0 conn(0x7ff44c0d01d0 0x7ff44c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff451955700 1 -- 172.21.15.181:0/3822792522 >> v2:172.21.15.190:3300/0 conn(0x7ff44c0d01d0 msgr2=0x7ff44c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:03.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff451955700 1 --2- 172.21.15.181:0/3822792522 >> v2:172.21.15.190:3300/0 conn(0x7ff44c0d01d0 0x7ff44c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff451955700 1 -- 172.21.15.181:0/3822792522 >> v1:172.21.15.181:6789/0 conn(0x7ff44c15ea40 legacy=0x7ff44c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff451955700 1 -- 172.21.15.181:0/3822792522 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0cec50 con 0x7ff44c0d1500 2023-10-16T11:20:03.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff443fff700 1 -- 172.21.15.181:0/3822792522 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200385134 0 0) 0x7ff44c0cf580 con 0x7ff44c15ea40 2023-10-16T11:20:03.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff451955700 1 --2- 172.21.15.181:0/3822792522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c0d1500 0x7ff44c0d1990 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ff44c08e050 tx=0x7ff44400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bc4977ac0766ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:03.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff443fff700 1 -- 172.21.15.181:0/3822792522 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400e070 con 0x7ff44c0d1500 2023-10-16T11:20:03.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff443fff700 1 -- 172.21.15.181:0/3822792522 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44400b9c0 con 0x7ff44c0d1500 2023-10-16T11:20:03.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.679+0000 7ff443fff700 1 -- 172.21.15.181:0/3822792522 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44400bd70 con 0x7ff44c0d1500 2023-10-16T11:20:03.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 -- 172.21.15.181:0/3822792522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c0d1500 msgr2=0x7ff44c0d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 --2- 172.21.15.181:0/3822792522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c0d1500 0x7ff44c0d1990 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ff44c08e050 tx=0x7ff44400b330 comp rx=0 tx=0).stop 2023-10-16T11:20:03.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 -- 172.21.15.181:0/3822792522 shutdown_connections 2023-10-16T11:20:03.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 --2- 172.21.15.181:0/3822792522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c0d1500 0x7ff44c0d1990 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 --2- 172.21.15.181:0/3822792522 >> v2:172.21.15.190:3300/0 conn(0x7ff44c0d01d0 0x7ff44c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 -- 172.21.15.181:0/3822792522 >> 172.21.15.181:0/3822792522 conn(0x7ff44c09cb60 msgr2=0x7ff44c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:03.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 -- 172.21.15.181:0/3822792522 shutdown_connections 2023-10-16T11:20:03.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.680+0000 7ff4533b8700 1 -- 172.21.15.181:0/3822792522 wait complete. 2023-10-16T11:20:03.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.681+0000 7ff4533b8700 1 Processor -- start 2023-10-16T11:20:03.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.681+0000 7ff4533b8700 1 -- start start 2023-10-16T11:20:03.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff4533b8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff44c0d01d0 0x7ff44c0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff4533b8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff44c0d1500 0x7ff44c0b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff4533b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c15ea40 0x7ff44c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff4533b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0b53e0 con 0x7ff44c15ea40 2023-10-16T11:20:03.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff4533b8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0b1030 con 0x7ff44c0d1500 2023-10-16T11:20:03.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff4533b8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0b1310 con 0x7ff44c0d01d0 2023-10-16T11:20:03.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c15ea40 0x7ff44c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c15ea40 0x7ff44c0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56924/0 (socket says 172.21.15.181:56924) 2023-10-16T11:20:03.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 -- 172.21.15.181:0/388455082 learned_addr learned my addr 172.21.15.181:0/388455082 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:03.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff450953700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff44c0d1500 0x7ff44c0b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451154700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff44c0d01d0 0x7ff44c0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:03.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 -- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff44c0d01d0 msgr2=0x7ff44c0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff44c0d01d0 0x7ff44c0b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 -- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff44c0d1500 msgr2=0x7ff44c0b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff44c0d1500 0x7ff44c0b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 -- 172.21.15.181:0/388455082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444009b30 con 0x7ff44c15ea40 2023-10-16T11:20:03.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff450953700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff44c0d1500 0x7ff44c0b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:03.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff451955700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c15ea40 0x7ff44c0b0990 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7ff44c08cf30 tx=0x7ff444004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:03.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440099b0 con 0x7ff44c15ea40 2023-10-16T11:20:03.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4440092e0 con 0x7ff44c15ea40 2023-10-16T11:20:03.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c0b15f0 con 0x7ff44c15ea40 2023-10-16T11:20:03.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.682+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44401c660 con 0x7ff44c15ea40 2023-10-16T11:20:03.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.683+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0b1bb0 con 0x7ff44c15ea40 2023-10-16T11:20:03.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.684+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c0b1dd0 con 0x7ff44c15ea40 2023-10-16T11:20:03.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.684+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 145) v1 ==== 89490+0+0 (secure 0 0 0) 0x7ff44401c7c0 con 0x7ff44c15ea40 2023-10-16T11:20:03.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.684+0000 7ff441ffb700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7ff43006b1d0 0x7ff43006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:03.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.685+0000 7ff451154700 1 -- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7ff43006b1d0 msgr2=0x7ff43006d690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/893959583 2023-10-16T11:20:03.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.685+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7469+0+0 (secure 0 0 0) 0x7ff44408bf10 con 0x7ff44c15ea40 2023-10-16T11:20:03.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.685+0000 7ff451154700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7ff43006b1d0 0x7ff43006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:03.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.689+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff444056510 con 0x7ff44c15ea40 2023-10-16T11:20:03.721 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.720+0000 7f45b410a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:20:03.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.743+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 146) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff4440568e0 con 0x7ff44c15ea40 2023-10-16T11:20:03.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.744+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7ff43006b1d0 msgr2=0x7ff43006d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:03.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.744+0000 7ff441ffb700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7ff43006b1d0 0x7ff43006d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.803 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.803 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.803 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.804 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.804 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.804 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.804 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.804 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.805 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.805 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.802+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.812 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.810+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.812 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.812 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.813 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.813 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452f8f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.814 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.814 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.815 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.815 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.815 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:03.811+0000 7f452c8f0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:03.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.907+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7ff44c04ebb0 con 0x7ff44c15ea40 2023-10-16T11:20:03.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.908+0000 7ff441ffb700 1 -- 172.21.15.181:0/388455082 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v222) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff44406fba0 con 0x7ff44c15ea40 2023-10-16T11:20:03.909 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:03.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c15ea40 msgr2=0x7ff44c0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:03.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c15ea40 0x7ff44c0b0990 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7ff44c08cf30 tx=0x7ff444004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 shutdown_connections 2023-10-16T11:20:03.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:6832/893959583,v1:172.21.15.181:6833/893959583] conn(0x7ff43006b1d0 0x7ff43006d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff44c0d01d0 0x7ff44c0b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff44c15ea40 0x7ff44c0b0990 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 --2- 172.21.15.181:0/388455082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff44c0d1500 0x7ff44c0b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:03.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 >> 172.21.15.181:0/388455082 conn(0x7ff44c09cb60 msgr2=0x7ff44c0d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:03.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 shutdown_connections 2023-10-16T11:20:03.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:03.910+0000 7ff4533b8700 1 -- 172.21.15.181:0/388455082 wait complete. 2023-10-16T11:20:03.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:04.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.044+0000 7f469c601700 1 Processor -- start 2023-10-16T11:20:04.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.044+0000 7f469c601700 1 -- start start 2023-10-16T11:20:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469c601700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 0x7f46940af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469c601700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46940af450 0x7f469406d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469c601700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469406f870 con 0x7f469406db80 2023-10-16T11:20:04.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469c601700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46940d1160 con 0x7f46940af450 2023-10-16T11:20:04.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469c601700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940cc8e0 con 0x7f469415dfd0 2023-10-16T11:20:04.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469ab9e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f469406db80 0x7f46940cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39876/0 (socket says 172.21.15.181:39876) 2023-10-16T11:20:04.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469ab9e700 1 -- 172.21.15.181:0/895972560 learned_addr learned my addr 172.21.15.181:0/895972560 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:04.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469a39d700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 0x7f46940af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914315609 0 0) 0x7f469406f870 con 0x7f469406db80 2023-10-16T11:20:04.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f4699b9c700 1 --2- 172.21.15.181:0/895972560 >> v2:172.21.15.190:3300/0 conn(0x7f46940af450 0x7f469406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46800035b0 con 0x7f469406db80 2023-10-16T11:20:04.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3245274215 0 0) 0x7f46800035b0 con 0x7f469406db80 2023-10-16T11:20:04.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 msgr2=0x7f46940af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:04.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 0x7f46940af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 >> v2:172.21.15.190:3300/0 conn(0x7f46940af450 msgr2=0x7f469406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:04.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 --2- 172.21.15.181:0/895972560 >> v2:172.21.15.190:3300/0 conn(0x7f46940af450 0x7f469406d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46940ccbc0 con 0x7f469406db80 2023-10-16T11:20:04.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f468c0019f0 con 0x7f469406db80 2023-10-16T11:20:04.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 >> v1:172.21.15.181:6789/0 conn(0x7f469406db80 legacy=0x7f46940cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 0x7f4680003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.045+0000 7f469939b700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4680003f80 0x7f4680004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469939b700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4680004d90 0x7f4680005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940d1160 con 0x7f4680003f80 2023-10-16T11:20:04.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940cc8e0 con 0x7f4680004d90 2023-10-16T11:20:04.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468000ba60 con 0x7f469415dfd0 2023-10-16T11:20:04.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469a39d700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 0x7f4680003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f4699b9c700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4680003f80 0x7f4680004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469ab9e700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4680004d90 0x7f4680005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f4699b9c700 1 -- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 msgr2=0x7f4680003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:04.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f4699b9c700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 0x7f4680003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f4699b9c700 1 -- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4680004d90 msgr2=0x7f4680005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:04.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f4699b9c700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4680004d90 0x7f4680005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f4699b9c700 1 -- 172.21.15.181:0/895972560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800035b0 con 0x7f4680003f80 2023-10-16T11:20:04.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469a39d700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f469415dfd0 0x7f4680003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:04.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f4699b9c700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4680003f80 0x7f4680004850 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f4684009b90 tx=0x7f4680005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1a421cc36c6453 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:04.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400e6c0 con 0x7f4680003f80 2023-10-16T11:20:04.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468400e820 con 0x7f4680003f80 2023-10-16T11:20:04.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.046+0000 7f469939b700 1 -- 172.21.15.181:0/895972560 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400eb90 con 0x7f4680003f80 2023-10-16T11:20:04.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.047+0000 7f469c601700 1 -- 172.21.15.181:0/895972560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4680003f80 msgr2=0x7f4680004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.047+0000 7f469c601700 1 --2- 172.21.15.181:0/895972560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4680003f80 0x7f4680004850 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f4684009b90 tx=0x7f4680005880 comp rx=0 tx=0).stop 2023-10-16T11:20:04.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.047+0000 7f469ab9e700 1 -- 172.21.15.181:0/895972560 reap_dead start 2023-10-16T11:20:04.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.047+0000 7f469c601700 1 -- 172.21.15.181:0/895972560 shutdown_connections 2023-10-16T11:20:04.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.047+0000 7f469c601700 1 -- 172.21.15.181:0/895972560 >> 172.21.15.181:0/895972560 conn(0x7f46940d04a0 msgr2=0x7f46940d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:04.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.047+0000 7f469c601700 1 -- 172.21.15.181:0/895972560 shutdown_connections 2023-10-16T11:20:04.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.047+0000 7f469c601700 1 -- 172.21.15.181:0/895972560 wait complete. 2023-10-16T11:20:04.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.048+0000 7f469c601700 1 Processor -- start 2023-10-16T11:20:04.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.048+0000 7f469c601700 1 -- start start 2023-10-16T11:20:04.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469c601700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4680003f80 0x7f46940c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469c601700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46940b5e20 0x7f46940c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469c601700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4694155860 0x7f469415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469c601700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940c87f0 con 0x7f46940b5e20 2023-10-16T11:20:04.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469c601700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940c8ad0 con 0x7f4694155860 2023-10-16T11:20:04.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469c601700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940c8db0 con 0x7f4680003f80 2023-10-16T11:20:04.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46940b5e20 0x7f46940c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46940b5e20 0x7f46940c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57038/0 (socket says 172.21.15.181:57038) 2023-10-16T11:20:04.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 -- 172.21.15.181:0/211918259 learned_addr learned my addr 172.21.15.181:0/211918259 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:04.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469a39d700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4680003f80 0x7f46940c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469ab9e700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4694155860 0x7f469415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 -- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4680003f80 msgr2=0x7f46940c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4680003f80 0x7f46940c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 -- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4694155860 msgr2=0x7f469415b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4694155860 0x7f469415b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 -- 172.21.15.181:0/211918259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46940c9090 con 0x7f46940b5e20 2023-10-16T11:20:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f469ab9e700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4694155860 0x7f469415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.049+0000 7f4699b9c700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46940b5e20 0x7f46940c7f50 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f4684009a80 tx=0x7f46840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.050+0000 7f468affd700 1 -- 172.21.15.181:0/211918259 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400e6c0 con 0x7f46940b5e20 2023-10-16T11:20:04.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.050+0000 7f468affd700 1 -- 172.21.15.181:0/211918259 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684003ab0 con 0x7f46940b5e20 2023-10-16T11:20:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.050+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46940c9370 con 0x7f46940b5e20 2023-10-16T11:20:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.050+0000 7f468affd700 1 -- 172.21.15.181:0/211918259 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684003e60 con 0x7f46940b5e20 2023-10-16T11:20:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.050+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46940c4510 con 0x7f46940b5e20 2023-10-16T11:20:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.051+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46940c96e0 con 0x7f46940b5e20 2023-10-16T11:20:04.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.051+0000 7f468affd700 1 -- 172.21.15.181:0/211918259 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 146) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4684013470 con 0x7f46940b5e20 2023-10-16T11:20:04.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.052+0000 7f468affd700 1 -- 172.21.15.181:0/211918259 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7942+0+0 (secure 0 0 0) 0x7f468408b3f0 con 0x7f46940b5e20 2023-10-16T11:20:04.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.056+0000 7f468affd700 1 -- 172.21.15.181:0/211918259 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4684056150 con 0x7f46940b5e20 2023-10-16T11:20:04.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.289+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46940c8c10 con 0x7f46940b5e20 2023-10-16T11:20:04.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.289+0000 7f468affd700 1 -- 172.21.15.181:0/211918259 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+6165 (secure 0 0 0) 0x7f46940c8c10 con 0x7f46940b5e20 2023-10-16T11:20:04.292 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:04.292 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:20:04.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.291+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46940b5e20 msgr2=0x7f46940c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.291+0000 7f469c601700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46940b5e20 0x7f46940c7f50 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f4684009a80 tx=0x7f46840047c0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.291+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 shutdown_connections 2023-10-16T11:20:04.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.291+0000 7f469c601700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4680003f80 0x7f46940c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.291+0000 7f469c601700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4694155860 0x7f469415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.291+0000 7f469c601700 1 --2- 172.21.15.181:0/211918259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46940b5e20 0x7f46940c7f50 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.291+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 >> 172.21.15.181:0/211918259 conn(0x7f46940d04a0 msgr2=0x7f46940ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:04.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.292+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 shutdown_connections 2023-10-16T11:20:04.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.292+0000 7f469c601700 1 -- 172.21.15.181:0/211918259 wait complete. 2023-10-16T11:20:04.297 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 117 2023-10-16T11:20:04.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:04.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb05e8fe700 1 Processor -- start 2023-10-16T11:20:04.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb05e8fe700 1 -- start start 2023-10-16T11:20:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb05e8fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0580c26b0 0x7fb0580c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb05e8fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb05815e020 0x7fb05815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb05e8fe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0580b2710 con 0x7fb05815dc40 2023-10-16T11:20:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb05e8fe700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0580c18f0 con 0x7fb0580c26b0 2023-10-16T11:20:04.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb05e8fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0580c1bd0 con 0x7fb05815e020 2023-10-16T11:20:04.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb057fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb05815dc40 0x7fb05806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39890/0 (socket says 172.21.15.181:39890) 2023-10-16T11:20:04.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.426+0000 7fb057fff700 1 -- 172.21.15.181:0/3289095093 learned_addr learned my addr 172.21.15.181:0/3289095093 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:04.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb0577fe700 1 --2- 172.21.15.181:0/3289095093 >> v2:172.21.15.190:3300/0 conn(0x7fb0580c26b0 0x7fb0580c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb05ce9b700 1 --2- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb05815e020 0x7fb05815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb056ffd700 1 -- 172.21.15.181:0/3289095093 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184921502 0 0) 0x7fb048001110 con 0x7fb05815dc40 2023-10-16T11:20:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb056ffd700 1 -- 172.21.15.181:0/3289095093 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0440035b0 con 0x7fb05815dc40 2023-10-16T11:20:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb056ffd700 1 -- 172.21.15.181:0/3289095093 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530087525 0 0) 0x7fb0440035b0 con 0x7fb05815dc40 2023-10-16T11:20:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb0577fe700 1 -- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb05815e020 msgr2=0x7fb05815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb0577fe700 1 --2- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb05815e020 0x7fb05815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb0577fe700 1 -- 172.21.15.181:0/3289095093 >> v1:172.21.15.181:6789/0 conn(0x7fb05815dc40 legacy=0x7fb05806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb0577fe700 1 -- 172.21.15.181:0/3289095093 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0580c1eb0 con 0x7fb0580c26b0 2023-10-16T11:20:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb05ce9b700 1 --2- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb05815e020 0x7fb05815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb0577fe700 1 --2- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0580c26b0 0x7fb0580c2b10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb040009840 tx=0x7fb040009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7847c43f7e2cfc7a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:04.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb056ffd700 1 -- 172.21.15.181:0/3289095093 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048001800 con 0x7fb0580c26b0 2023-10-16T11:20:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb056ffd700 1 -- 172.21.15.181:0/3289095093 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040004030 con 0x7fb0580c26b0 2023-10-16T11:20:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.427+0000 7fb056ffd700 1 -- 172.21.15.181:0/3289095093 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0400043c0 con 0x7fb0580c26b0 2023-10-16T11:20:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 -- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0580c26b0 msgr2=0x7fb0580c2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 --2- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0580c26b0 0x7fb0580c2b10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb040009840 tx=0x7fb040009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:04.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 -- 172.21.15.181:0/3289095093 shutdown_connections 2023-10-16T11:20:04.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 --2- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb05815e020 0x7fb05815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 --2- 172.21.15.181:0/3289095093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0580c26b0 0x7fb0580c2b10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 -- 172.21.15.181:0/3289095093 >> 172.21.15.181:0/3289095093 conn(0x7fb0580b1780 msgr2=0x7fb0580b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:04.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 -- 172.21.15.181:0/3289095093 shutdown_connections 2023-10-16T11:20:04.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.428+0000 7fb05e8fe700 1 -- 172.21.15.181:0/3289095093 wait complete. 2023-10-16T11:20:04.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.429+0000 7fb05e8fe700 1 Processor -- start 2023-10-16T11:20:04.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.429+0000 7fb05e8fe700 1 -- start start 2023-10-16T11:20:04.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb05e8fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0580c26b0 0x7fb0580c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb05e8fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05815dc40 0x7fb0581eea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb05e8fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05815e020 0x7fb05815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb05e8fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0581eefa0 con 0x7fb05815dc40 2023-10-16T11:20:04.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb05e8fe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0581592b0 con 0x7fb05815e020 2023-10-16T11:20:04.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb05e8fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb058158cf0 con 0x7fb0580c26b0 2023-10-16T11:20:04.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05815dc40 0x7fb0581eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05815dc40 0x7fb0581eea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57052/0 (socket says 172.21.15.181:57052) 2023-10-16T11:20:04.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 -- 172.21.15.181:0/2453044689 learned_addr learned my addr 172.21.15.181:0/2453044689 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:04.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb05ce9b700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05815e020 0x7fb05815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb057fff700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0580c26b0 0x7fb0580c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 -- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0580c26b0 msgr2=0x7fb0580c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0580c26b0 0x7fb0580c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 -- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05815e020 msgr2=0x7fb05815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05815e020 0x7fb05815c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.430+0000 7fb0577fe700 1 -- 172.21.15.181:0/2453044689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05000ab80 con 0x7fb05815dc40 2023-10-16T11:20:04.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.431+0000 7fb0577fe700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05815dc40 0x7fb0581eea60 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fb040009b90 tx=0x7fb040004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:04.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.431+0000 7fb054ff9700 1 -- 172.21.15.181:0/2453044689 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040003e20 con 0x7fb05815dc40 2023-10-16T11:20:04.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.431+0000 7fb054ff9700 1 -- 172.21.15.181:0/2453044689 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04001f070 con 0x7fb05815dc40 2023-10-16T11:20:04.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.431+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0400094a0 con 0x7fb05815dc40 2023-10-16T11:20:04.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.431+0000 7fb054ff9700 1 -- 172.21.15.181:0/2453044689 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011870 con 0x7fb05815dc40 2023-10-16T11:20:04.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.431+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05806ce90 con 0x7fb05815dc40 2023-10-16T11:20:04.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.432+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb058067d20 con 0x7fb05815dc40 2023-10-16T11:20:04.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.432+0000 7fb054ff9700 1 -- 172.21.15.181:0/2453044689 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 146) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fb0400119d0 con 0x7fb05815dc40 2023-10-16T11:20:04.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.433+0000 7fb054ff9700 1 -- 172.21.15.181:0/2453044689 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7942+0+0 (secure 0 0 0) 0x7fb040021070 con 0x7fb05815dc40 2023-10-16T11:20:04.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.437+0000 7fb054ff9700 1 -- 172.21.15.181:0/2453044689 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb040055ee0 con 0x7fb05815dc40 2023-10-16T11:20:04.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.631+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0580ceca0 con 0x7fb05815dc40 2023-10-16T11:20:04.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.633+0000 7fb054ff9700 1 -- 172.21.15.181:0/2453044689 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+22396 (secure 0 0 0) 0x7fb04002b070 con 0x7fb05815dc40 2023-10-16T11:20:04.635 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:04.636 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":222,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:03.733549+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2033295595":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/4244377789":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2576397499":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6833/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/754905711":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3499373399":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2963042451":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3469785622":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2131678440":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/134902336":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:04.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05815dc40 msgr2=0x7fb0581eea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05815dc40 0x7fb0581eea60 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fb040009b90 tx=0x7fb040004750 comp rx=0 tx=0).stop 2023-10-16T11:20:04.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 shutdown_connections 2023-10-16T11:20:04.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0580c26b0 0x7fb0580c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05815e020 0x7fb05815c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 --2- 172.21.15.181:0/2453044689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05815dc40 0x7fb0581eea60 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 >> 172.21.15.181:0/2453044689 conn(0x7fb0580b1780 msgr2=0x7fb0580b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:04.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 shutdown_connections 2023-10-16T11:20:04.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.635+0000 7fb05e8fe700 1 -- 172.21.15.181:0/2453044689 wait complete. 2023-10-16T11:20:04.649 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:20:04.650 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:04.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.776+0000 7fcf9f139700 1 Processor -- start 2023-10-16T11:20:04.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.776+0000 7fcf9f139700 1 -- start start 2023-10-16T11:20:04.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf9f139700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980ce7f0 0x7fcf9806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf9f139700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf980d2390 0x7fcf980cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf9f139700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf980cc470 con 0x7fcf980d0180 2023-10-16T11:20:04.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf9f139700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf980d0630 con 0x7fcf980d2390 2023-10-16T11:20:04.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf9f139700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9806ca30 con 0x7fcf980ce7f0 2023-10-16T11:20:04.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf980d2390 0x7fcf980cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf980d2390 0x7fcf980cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57080/0 (socket says 172.21.15.181:57080) 2023-10-16T11:20:04.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 -- 172.21.15.181:0/4043902350 learned_addr learned my addr 172.21.15.181:0/4043902350 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:04.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf9ced5700 1 --2- 172.21.15.181:0/4043902350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980ce7f0 0x7fcf9806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 -- 172.21.15.181:0/4043902350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980ce7f0 msgr2=0x7fcf9806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:04.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 --2- 172.21.15.181:0/4043902350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980ce7f0 0x7fcf9806c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 -- 172.21.15.181:0/4043902350 >> v1:172.21.15.190:6789/0 conn(0x7fcf980d0180 legacy=0x7fcf98161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 -- 172.21.15.181:0/4043902350 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9806cd10 con 0x7fcf980d2390 2023-10-16T11:20:04.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf97fff700 1 --2- 172.21.15.181:0/4043902350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980cfc40 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fcf840098f0 tx=0x7fcf84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41871a379379445b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:04.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf977fe700 1 -- 172.21.15.181:0/4043902350 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400b8a0 con 0x7fcf980d2390 2023-10-16T11:20:04.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf977fe700 1 -- 172.21.15.181:0/4043902350 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8400ba00 con 0x7fcf980d2390 2023-10-16T11:20:04.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.777+0000 7fcf977fe700 1 -- 172.21.15.181:0/4043902350 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400bd90 con 0x7fcf980d2390 2023-10-16T11:20:04.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 -- 172.21.15.181:0/4043902350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 msgr2=0x7fcf980cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 --2- 172.21.15.181:0/4043902350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980cfc40 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fcf840098f0 tx=0x7fcf84009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:04.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 -- 172.21.15.181:0/4043902350 shutdown_connections 2023-10-16T11:20:04.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 --2- 172.21.15.181:0/4043902350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980ce7f0 0x7fcf9806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 --2- 172.21.15.181:0/4043902350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980cfc40 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 -- 172.21.15.181:0/4043902350 >> 172.21.15.181:0/4043902350 conn(0x7fcf9809a9a0 msgr2=0x7fcf980d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:04.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 -- 172.21.15.181:0/4043902350 shutdown_connections 2023-10-16T11:20:04.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.778+0000 7fcf9f139700 1 -- 172.21.15.181:0/4043902350 wait complete. 2023-10-16T11:20:04.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.779+0000 7fcf9f139700 1 Processor -- start 2023-10-16T11:20:04.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.779+0000 7fcf9f139700 1 -- start start 2023-10-16T11:20:04.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9f139700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf980ce7f0 0x7fcf980c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9f139700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980d0180 0x7fcf980c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9f139700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9f139700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf98149820 con 0x7fcf980d2390 2023-10-16T11:20:04.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9f139700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980b6de0 con 0x7fcf980ce7f0 2023-10-16T11:20:04.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9f139700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980b4810 con 0x7fcf980d0180 2023-10-16T11:20:04.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9d6d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9d6d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57088/0 (socket says 172.21.15.181:57088) 2023-10-16T11:20:04.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9d6d6700 1 -- 172.21.15.181:0/2307644562 learned_addr learned my addr 172.21.15.181:0/2307644562 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:04.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf9ced5700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf980ce7f0 0x7fcf980c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.780+0000 7fcf97fff700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980d0180 0x7fcf980c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9d6d6700 1 -- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980d0180 msgr2=0x7fcf980c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9d6d6700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980d0180 0x7fcf980c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9d6d6700 1 -- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf980ce7f0 msgr2=0x7fcf980c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:04.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9d6d6700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf980ce7f0 0x7fcf980c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:04.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9d6d6700 1 -- 172.21.15.181:0/2307644562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84009550 con 0x7fcf980d2390 2023-10-16T11:20:04.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9d6d6700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980c7220 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000cb50 tx=0x7fcf9000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:04.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf957fa700 1 -- 172.21.15.181:0/2307644562 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90003c00 con 0x7fcf980d2390 2023-10-16T11:20:04.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf957fa700 1 -- 172.21.15.181:0/2307644562 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf90003d60 con 0x7fcf980d2390 2023-10-16T11:20:04.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf980b7080 con 0x7fcf980d2390 2023-10-16T11:20:04.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf957fa700 1 -- 172.21.15.181:0/2307644562 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9000aa20 con 0x7fcf980d2390 2023-10-16T11:20:04.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.781+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf980c7fe0 con 0x7fcf980d2390 2023-10-16T11:20:04.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.782+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf981499d0 con 0x7fcf980d2390 2023-10-16T11:20:04.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.782+0000 7fcf957fa700 1 -- 172.21.15.181:0/2307644562 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 147) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcf9000ab80 con 0x7fcf980d2390 2023-10-16T11:20:04.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.783+0000 7fcf957fa700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcf7c06ad50 0x7fcf7c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:04.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.783+0000 7fcf9ced5700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcf7c06ad50 0x7fcf7c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:04.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.783+0000 7fcf957fa700 1 -- 172.21.15.181:0/2307644562 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7942+0+0 (secure 0 0 0) 0x7fcf90089c10 con 0x7fcf980d2390 2023-10-16T11:20:04.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.783+0000 7fcf9ced5700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcf7c06ad50 0x7fcf7c06d210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c006fd0 tx=0x7fcf8c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:04.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:04.787+0000 7fcf957fa700 1 -- 172.21.15.181:0/2307644562 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcf981499d0 con 0x7fcf980d2390 2023-10-16T11:20:05.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.024+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf981499d0 con 0x7fcf980d2390 2023-10-16T11:20:05.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.024+0000 7fcf957fa700 1 -- 172.21.15.181:0/2307644562 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+6165 (secure 0 0 0) 0x7fcf981499d0 con 0x7fcf980d2390 2023-10-16T11:20:05.025 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:05.026 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:20:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcf7c06ad50 msgr2=0x7fcf7c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcf7c06ad50 0x7fcf7c06d210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c006fd0 tx=0x7fcf8c008040 comp rx=0 tx=0).stop 2023-10-16T11:20:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 msgr2=0x7fcf980c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980c7220 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000cb50 tx=0x7fcf9000ce60 comp rx=0 tx=0).stop 2023-10-16T11:20:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 shutdown_connections 2023-10-16T11:20:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcf7c06ad50 0x7fcf7c06d210 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf980d2390 0x7fcf980c7220 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf980ce7f0 0x7fcf980c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 --2- 172.21.15.181:0/2307644562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf980d0180 0x7fcf980c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 >> 172.21.15.181:0/2307644562 conn(0x7fcf9809a9a0 msgr2=0x7fcf980c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:05.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 shutdown_connections 2023-10-16T11:20:05.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.027+0000 7fcf9f139700 1 -- 172.21.15.181:0/2307644562 wait complete. 2023-10-16T11:20:05.030 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 117 2023-10-16T11:20:05.040 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:05.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.160+0000 7f802e367700 1 Processor -- start 2023-10-16T11:20:05.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.160+0000 7f802e367700 1 -- start start 2023-10-16T11:20:05.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802e367700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80280d2390 0x7f80280cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802e367700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80280d0180 0x7f80280cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802e367700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80280cc470 con 0x7f80280ce7f0 2023-10-16T11:20:05.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802e367700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802806c810 con 0x7f80280d0180 2023-10-16T11:20:05.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802e367700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80280d05f0 con 0x7f80280d2390 2023-10-16T11:20:05.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80280d0180 0x7f80280cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80280d0180 0x7f80280cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57096/0 (socket says 172.21.15.181:57096) 2023-10-16T11:20:05.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 -- 172.21.15.181:0/4190490374 learned_addr learned my addr 172.21.15.181:0/4190490374 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:05.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f80277fe700 1 --2- 172.21.15.181:0/4190490374 >> v2:172.21.15.190:3300/0 conn(0x7f80280d2390 0x7f80280cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 -- 172.21.15.181:0/4190490374 >> v2:172.21.15.190:3300/0 conn(0x7f80280d2390 msgr2=0x7f80280cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:05.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 --2- 172.21.15.181:0/4190490374 >> v2:172.21.15.190:3300/0 conn(0x7f80280d2390 0x7f80280cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 -- 172.21.15.181:0/4190490374 >> v1:172.21.15.190:6789/0 conn(0x7f80280ce7f0 legacy=0x7f802806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 -- 172.21.15.181:0/4190490374 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802806ca30 con 0x7f80280d0180 2023-10-16T11:20:05.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f802c904700 1 --2- 172.21.15.181:0/4190490374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280cc930 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f80280cc1e0 tx=0x7f802000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1669ee4ab26a34 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f8026ffd700 1 -- 172.21.15.181:0/4190490374 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802000e070 con 0x7f80280d0180 2023-10-16T11:20:05.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f8026ffd700 1 -- 172.21.15.181:0/4190490374 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802000b9c0 con 0x7f80280d0180 2023-10-16T11:20:05.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.161+0000 7f8026ffd700 1 -- 172.21.15.181:0/4190490374 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802000bd70 con 0x7f80280d0180 2023-10-16T11:20:05.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 -- 172.21.15.181:0/4190490374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 msgr2=0x7f80280cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 --2- 172.21.15.181:0/4190490374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280cc930 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f80280cc1e0 tx=0x7f802000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:05.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 -- 172.21.15.181:0/4190490374 shutdown_connections 2023-10-16T11:20:05.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 --2- 172.21.15.181:0/4190490374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280cc930 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 --2- 172.21.15.181:0/4190490374 >> v2:172.21.15.190:3300/0 conn(0x7f80280d2390 0x7f80280cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 -- 172.21.15.181:0/4190490374 >> 172.21.15.181:0/4190490374 conn(0x7f802809a9a0 msgr2=0x7f80280d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:05.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 -- 172.21.15.181:0/4190490374 shutdown_connections 2023-10-16T11:20:05.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.162+0000 7f802e367700 1 -- 172.21.15.181:0/4190490374 wait complete. 2023-10-16T11:20:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.163+0000 7f802e367700 1 Processor -- start 2023-10-16T11:20:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.163+0000 7f802e367700 1 -- start start 2023-10-16T11:20:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f802e367700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80280ce7f0 0x7f8028149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f802e367700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f802e367700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80280d2390 0x7f80280ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f802e367700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281497b0 con 0x7f80280d0180 2023-10-16T11:20:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f802e367700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280c7400 con 0x7f80280d2390 2023-10-16T11:20:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f802e367700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80280c76e0 con 0x7f80280ce7f0 2023-10-16T11:20:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57100/0 (socket says 172.21.15.181:57100) 2023-10-16T11:20:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 -- 172.21.15.181:0/40740989 learned_addr learned my addr 172.21.15.181:0/40740989 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f8027fff700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80280ce7f0 0x7f8028149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f802c904700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80280d2390 0x7f80280ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 -- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80280ce7f0 msgr2=0x7f8028149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80280ce7f0 0x7f8028149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 -- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80280d2390 msgr2=0x7f80280ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80280d2390 0x7f80280ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.164+0000 7f80277fe700 1 -- 172.21.15.181:0/40740989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8020009b30 con 0x7f80280d0180 2023-10-16T11:20:05.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.165+0000 7f80277fe700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280c69b0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f801c00e810 tx=0x7f801c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.165+0000 7f8024ff9700 1 -- 172.21.15.181:0/40740989 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00c070 con 0x7f80280d0180 2023-10-16T11:20:05.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.165+0000 7f8024ff9700 1 -- 172.21.15.181:0/40740989 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c013610 con 0x7f80280d0180 2023-10-16T11:20:05.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.165+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80280c79c0 con 0x7f80280d0180 2023-10-16T11:20:05.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.165+0000 7f8024ff9700 1 -- 172.21.15.181:0/40740989 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c0139c0 con 0x7f80280d0180 2023-10-16T11:20:05.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.165+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80280c7f80 con 0x7f80280d0180 2023-10-16T11:20:05.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.166+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80280cffa0 con 0x7f80280d0180 2023-10-16T11:20:05.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.166+0000 7f8024ff9700 1 -- 172.21.15.181:0/40740989 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 147) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f801c013b20 con 0x7f80280d0180 2023-10-16T11:20:05.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.166+0000 7f8024ff9700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f801406ad50 0x7f801406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.167+0000 7f8024ff9700 1 -- 172.21.15.181:0/40740989 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7942+0+0 (secure 0 0 0) 0x7f801c089580 con 0x7f80280d0180 2023-10-16T11:20:05.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.167+0000 7f8027fff700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f801406ad50 0x7f801406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.167+0000 7f8027fff700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f801406ad50 0x7f801406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8018000fb0 tx=0x7f8018009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.171+0000 7f8024ff9700 1 -- 172.21.15.181:0/40740989 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f801c0540b0 con 0x7f80280d0180 2023-10-16T11:20:05.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.407+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f802806d2a0 con 0x7f80280d0180 2023-10-16T11:20:05.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.407+0000 7f8024ff9700 1 -- 172.21.15.181:0/40740989 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+6165 (secure 0 0 0) 0x7f801c017020 con 0x7f80280d0180 2023-10-16T11:20:05.408 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:05.409 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":117,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:22.925980+0000","modified":"2023-10-16T11:19:25.961114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11340":{"gid":11340,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2252072910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2252072910},{"type":"v1","addr":"172.21.15.190:6835","nonce":2252072910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:20:05.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f801406ad50 msgr2=0x7f801406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f801406ad50 0x7f801406d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8018000fb0 tx=0x7f8018009040 comp rx=0 tx=0).stop 2023-10-16T11:20:05.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 msgr2=0x7f80280c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280c69b0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f801c00e810 tx=0x7f801c00eb20 comp rx=0 tx=0).stop 2023-10-16T11:20:05.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 shutdown_connections 2023-10-16T11:20:05.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f801406ad50 0x7f801406d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80280ce7f0 0x7f8028149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80280d2390 0x7f80280ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 --2- 172.21.15.181:0/40740989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80280d0180 0x7f80280c69b0 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 >> 172.21.15.181:0/40740989 conn(0x7f802809a9a0 msgr2=0x7f8028070680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:05.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.409+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 shutdown_connections 2023-10-16T11:20:05.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.410+0000 7f802e367700 1 -- 172.21.15.181:0/40740989 wait complete. 2023-10-16T11:20:05.414 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 117 2023-10-16T11:20:05.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:05.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.542+0000 7f395eeb8700 1 Processor -- start 2023-10-16T11:20:05.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.542+0000 7f395eeb8700 1 -- start start 2023-10-16T11:20:05.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395eeb8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f395815e6d0 0x7f39580d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395eeb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f395815eab0 0x7f395815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395eeb8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39580cb0c0 0x7f395815a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395eeb8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395806ed00 con 0x7f39580cb0c0 2023-10-16T11:20:05.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395eeb8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39580d02d0 con 0x7f395815e6d0 2023-10-16T11:20:05.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395eeb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395806e0f0 con 0x7f395815eab0 2023-10-16T11:20:05.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39580cb0c0 0x7f395815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39580cb0c0 0x7f395815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57110/0 (socket says 172.21.15.181:57110) 2023-10-16T11:20:05.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 -- 172.21.15.181:0/1156085848 learned_addr learned my addr 172.21.15.181:0/1156085848 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:05.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f3957fff700 1 --2- 172.21.15.181:0/1156085848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f395815eab0 0x7f395815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 -- 172.21.15.181:0/1156085848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f395815eab0 msgr2=0x7f395815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:05.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395cc54700 1 --2- 172.21.15.181:0/1156085848 >> v2:172.21.15.190:3300/0 conn(0x7f395815e6d0 0x7f39580d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 --2- 172.21.15.181:0/1156085848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f395815eab0 0x7f395815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 -- 172.21.15.181:0/1156085848 >> v2:172.21.15.190:3300/0 conn(0x7f395815e6d0 msgr2=0x7f39580d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:05.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 --2- 172.21.15.181:0/1156085848 >> v2:172.21.15.190:3300/0 conn(0x7f395815e6d0 0x7f39580d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 -- 172.21.15.181:0/1156085848 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395806e3d0 con 0x7f39580cb0c0 2023-10-16T11:20:05.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f3957fff700 1 --2- 172.21.15.181:0/1156085848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f395815eab0 0x7f395815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:05.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f395d455700 1 --2- 172.21.15.181:0/1156085848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39580cb0c0 0x7f395815a630 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f39580717b0 tx=0x7f395000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c959ef033d04cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.543+0000 7f39577fe700 1 -- 172.21.15.181:0/1156085848 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950010070 con 0x7f39580cb0c0 2023-10-16T11:20:05.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f39577fe700 1 -- 172.21.15.181:0/1156085848 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395000dd90 con 0x7f39580cb0c0 2023-10-16T11:20:05.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f39577fe700 1 -- 172.21.15.181:0/1156085848 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395000b310 con 0x7f39580cb0c0 2023-10-16T11:20:05.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 -- 172.21.15.181:0/1156085848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39580cb0c0 msgr2=0x7f395815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 --2- 172.21.15.181:0/1156085848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39580cb0c0 0x7f395815a630 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f39580717b0 tx=0x7f395000d430 comp rx=0 tx=0).stop 2023-10-16T11:20:05.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 -- 172.21.15.181:0/1156085848 shutdown_connections 2023-10-16T11:20:05.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 --2- 172.21.15.181:0/1156085848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39580cb0c0 0x7f395815a630 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 --2- 172.21.15.181:0/1156085848 >> v2:172.21.15.190:3300/0 conn(0x7f395815e6d0 0x7f39580d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 --2- 172.21.15.181:0/1156085848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f395815eab0 0x7f395815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 -- 172.21.15.181:0/1156085848 >> 172.21.15.181:0/1156085848 conn(0x7f39580d04a0 msgr2=0x7f39580d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:05.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 -- 172.21.15.181:0/1156085848 shutdown_connections 2023-10-16T11:20:05.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.544+0000 7f395eeb8700 1 -- 172.21.15.181:0/1156085848 wait complete. 2023-10-16T11:20:05.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.545+0000 7f395eeb8700 1 Processor -- start 2023-10-16T11:20:05.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.545+0000 7f395eeb8700 1 -- start start 2023-10-16T11:20:05.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395eeb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39580cb0c0 0x7f39581519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395eeb8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f395815e6d0 0x7f39580c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395eeb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f395815eab0 0x7f39580b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395eeb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b4da0 con 0x7f395815eab0 2023-10-16T11:20:05.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395eeb8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b4120 con 0x7f395815e6d0 2023-10-16T11:20:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395eeb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b4400 con 0x7f39580cb0c0 2023-10-16T11:20:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395d455700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f395815eab0 0x7f39580b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395d455700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f395815eab0 0x7f39580b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57116/0 (socket says 172.21.15.181:57116) 2023-10-16T11:20:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395d455700 1 -- 172.21.15.181:0/230006223 learned_addr learned my addr 172.21.15.181:0/230006223 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:05.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f395cc54700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39580cb0c0 0x7f39581519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.546+0000 7f3957fff700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f395815e6d0 0x7f39580c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395d455700 1 -- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39580cb0c0 msgr2=0x7f39581519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395d455700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39580cb0c0 0x7f39581519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395d455700 1 -- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f395815e6d0 msgr2=0x7f39580c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395d455700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f395815e6d0 0x7f39580c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395d455700 1 -- 172.21.15.181:0/230006223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395000bd40 con 0x7f395815eab0 2023-10-16T11:20:05.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395d455700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f395815eab0 0x7f39580b52f0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f3958071580 tx=0x7f3950004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f39557fa700 1 -- 172.21.15.181:0/230006223 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950009a10 con 0x7f395815eab0 2023-10-16T11:20:05.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f39557fa700 1 -- 172.21.15.181:0/230006223 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3950009b90 con 0x7f395815eab0 2023-10-16T11:20:05.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f39557fa700 1 -- 172.21.15.181:0/230006223 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395000bbc0 con 0x7f395815eab0 2023-10-16T11:20:05.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39580b46e0 con 0x7f395815eab0 2023-10-16T11:20:05.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.547+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39580c7ab0 con 0x7f395815eab0 2023-10-16T11:20:05.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.548+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39580c7c40 con 0x7f395815eab0 2023-10-16T11:20:05.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.549+0000 7f39557fa700 1 -- 172.21.15.181:0/230006223 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 147) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3950004040 con 0x7f395815eab0 2023-10-16T11:20:05.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.549+0000 7f39557fa700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f393c06ad50 0x7f393c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.549+0000 7f395cc54700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f393c06ad50 0x7f393c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.549+0000 7f39557fa700 1 -- 172.21.15.181:0/230006223 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7942+0+0 (secure 0 0 0) 0x7f395008b1a0 con 0x7f395815eab0 2023-10-16T11:20:05.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.549+0000 7f395cc54700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f393c06ad50 0x7f393c06d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f394c00ef50 tx=0x7f394c010040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.554+0000 7f39557fa700 1 -- 172.21.15.181:0/230006223 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f39580c7c40 con 0x7f395815eab0 2023-10-16T11:20:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.747+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39580b4ee0 con 0x7f395815eab0 2023-10-16T11:20:05.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.749+0000 7f39557fa700 1 -- 172.21.15.181:0/230006223 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+22396 (secure 0 0 0) 0x7f39580b4ee0 con 0x7f395815eab0 2023-10-16T11:20:05.750 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:05.750 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":222,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:03.733549+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:19:21.355919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-16T11:19:21.889329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2033295595":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/4244377789":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2576397499":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6833/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/754905711":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3499373399":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2963042451":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3469785622":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2131678440":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/134902336":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:05.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f393c06ad50 msgr2=0x7f393c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f393c06ad50 0x7f393c06d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f394c00ef50 tx=0x7f394c010040 comp rx=0 tx=0).stop 2023-10-16T11:20:05.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f395815eab0 msgr2=0x7f39580b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f395815eab0 0x7f39580b52f0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f3958071580 tx=0x7f3950004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 shutdown_connections 2023-10-16T11:20:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f393c06ad50 0x7f393c06d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39580cb0c0 0x7f39581519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f395815eab0 0x7f39580b52f0 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 --2- 172.21.15.181:0/230006223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f395815e6d0 0x7f39580c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.751+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 >> 172.21.15.181:0/230006223 conn(0x7f39580d04a0 msgr2=0x7f395806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.752+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 shutdown_connections 2023-10-16T11:20:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.752+0000 7f395eeb8700 1 -- 172.21.15.181:0/230006223 wait complete. 2023-10-16T11:20:05.765 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:20:05.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.891+0000 7f7db5e4c700 1 Processor -- start 2023-10-16T11:20:05.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.891+0000 7f7db5e4c700 1 -- start start 2023-10-16T11:20:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7db5e4c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db00cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7db5e4c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7db00cc4c0 0x7f7db015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7db5e4c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db00cf460 con 0x7f7db006d210 2023-10-16T11:20:05.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7db5e4c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00d0380 con 0x7f7db00cc4c0 2023-10-16T11:20:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7db5e4c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db006c700 con 0x7f7db00ce4d0 2023-10-16T11:20:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7daffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7db00cc4c0 0x7f7db015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7daf7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7db006d210 0x7f7db00cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39984/0 (socket says 172.21.15.181:39984) 2023-10-16T11:20:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7daf7fe700 1 -- 172.21.15.181:0/1616362826 learned_addr learned my addr 172.21.15.181:0/1616362826 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:05.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7daeffd700 1 --2- 172.21.15.181:0/1616362826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db00cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7daffff700 1 -- 172.21.15.181:0/1616362826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 msgr2=0x7f7db00cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7daffff700 1 --2- 172.21.15.181:0/1616362826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db00cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7daffff700 1 -- 172.21.15.181:0/1616362826 >> v1:172.21.15.181:6789/0 conn(0x7f7db006d210 legacy=0x7f7db00cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.892+0000 7f7dae7fc700 1 -- 172.21.15.181:0/1616362826 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217084612 0 0) 0x7f7db00cf460 con 0x7f7db006d210 2023-10-16T11:20:05.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7daffff700 1 -- 172.21.15.181:0/1616362826 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db006d5f0 con 0x7f7db00cc4c0 2023-10-16T11:20:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7daeffd700 1 --2- 172.21.15.181:0/1616362826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db00cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7daffff700 1 --2- 172.21.15.181:0/1616362826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db00cc4c0 0x7f7db015d2f0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009d90 tx=0x7f7da800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8817e8c79309b74f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7dae7fc700 1 -- 172.21.15.181:0/1616362826 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da800e070 con 0x7f7db00cc4c0 2023-10-16T11:20:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7dae7fc700 1 -- 172.21.15.181:0/1616362826 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da800bbd0 con 0x7f7db00cc4c0 2023-10-16T11:20:05.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7dae7fc700 1 -- 172.21.15.181:0/1616362826 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8012430 con 0x7f7db00cc4c0 2023-10-16T11:20:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 -- 172.21.15.181:0/1616362826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db00cc4c0 msgr2=0x7f7db015d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/1616362826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db00cc4c0 0x7f7db015d2f0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009d90 tx=0x7f7da800b330 comp rx=0 tx=0).stop 2023-10-16T11:20:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 -- 172.21.15.181:0/1616362826 shutdown_connections 2023-10-16T11:20:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/1616362826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db00cc4c0 0x7f7db015d2f0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/1616362826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db00cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 -- 172.21.15.181:0/1616362826 >> 172.21.15.181:0/1616362826 conn(0x7f7db009a9a0 msgr2=0x7f7db00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 -- 172.21.15.181:0/1616362826 shutdown_connections 2023-10-16T11:20:05.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.893+0000 7f7db5e4c700 1 -- 172.21.15.181:0/1616362826 wait complete. 2023-10-16T11:20:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 Processor -- start 2023-10-16T11:20:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 -- start start 2023-10-16T11:20:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006d210 0x7f7db0152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db00cc4c0 0x7f7db014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db01f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0152590 con 0x7f7db006d210 2023-10-16T11:20:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db014f2c0 con 0x7f7db00cc4c0 2023-10-16T11:20:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.895+0000 7f7db5e4c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db01f3570 con 0x7f7db00ce4d0 2023-10-16T11:20:05.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006d210 0x7f7db0152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006d210 0x7f7db0152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57144/0 (socket says 172.21.15.181:57144) 2023-10-16T11:20:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 -- 172.21.15.181:0/2158232764 learned_addr learned my addr 172.21.15.181:0/2158232764 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daeffd700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db00cc4c0 0x7f7db014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daffff700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db01f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 -- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 msgr2=0x7f7db01f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db01f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 -- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db00cc4c0 msgr2=0x7f7db014ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db00cc4c0 0x7f7db014ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 -- 172.21.15.181:0/2158232764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8009a70 con 0x7f7db006d210 2023-10-16T11:20:05.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daeffd700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db00cc4c0 0x7f7db014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7daf7fe700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006d210 0x7f7db0152050 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c550 tx=0x7f7da400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7d97fff700 1 -- 172.21.15.181:0/2158232764 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4004c80 con 0x7f7db006d210 2023-10-16T11:20:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7d97fff700 1 -- 172.21.15.181:0/2158232764 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da4004e00 con 0x7f7db006d210 2023-10-16T11:20:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db01f4060 con 0x7f7db006d210 2023-10-16T11:20:05.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7d97fff700 1 -- 172.21.15.181:0/2158232764 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4007310 con 0x7f7db006d210 2023-10-16T11:20:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.896+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db01f3aa0 con 0x7f7db006d210 2023-10-16T11:20:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.897+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db0067d20 con 0x7f7db006d210 2023-10-16T11:20:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.898+0000 7f7d97fff700 1 -- 172.21.15.181:0/2158232764 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 147) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7da4007470 con 0x7f7db006d210 2023-10-16T11:20:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.898+0000 7f7d97fff700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7d9c06ad50 0x7f7d9c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:05.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.898+0000 7f7d97fff700 1 -- 172.21.15.181:0/2158232764 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7942+0+0 (secure 0 0 0) 0x7f7da4089760 con 0x7f7db006d210 2023-10-16T11:20:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.898+0000 7f7daeffd700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7d9c06ad50 0x7f7d9c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.898+0000 7f7daeffd700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7d9c06ad50 0x7f7d9c06d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7db00b2410 tx=0x7f7da0008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:05.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:05.903+0000 7f7d97fff700 1 -- 172.21.15.181:0/2158232764 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7da4054290 con 0x7f7db006d210 2023-10-16T11:20:06.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.137+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7db00b2f00 con 0x7f7db006d210 2023-10-16T11:20:06.139 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:20:06.138+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:20:06.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.140+0000 7f7d97fff700 1 -- 172.21.15.181:0/2158232764 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v118) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7da40578b0 con 0x7f7db006d210 2023-10-16T11:20:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.142+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7d9c06ad50 msgr2=0x7f7d9c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.142+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7d9c06ad50 0x7f7d9c06d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7db00b2410 tx=0x7f7da0008040 comp rx=0 tx=0).stop 2023-10-16T11:20:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.142+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006d210 msgr2=0x7f7db0152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:06.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.142+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006d210 0x7f7db0152050 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c550 tx=0x7f7da400c860 comp rx=0 tx=0).stop 2023-10-16T11:20:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 shutdown_connections 2023-10-16T11:20:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7d9c06ad50 0x7f7d9c06d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7db00ce4d0 0x7f7db01f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7db006d210 0x7f7db0152050 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 --2- 172.21.15.181:0/2158232764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db00cc4c0 0x7f7db014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 >> 172.21.15.181:0/2158232764 conn(0x7f7db009a9a0 msgr2=0x7f7db00c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:06.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 shutdown_connections 2023-10-16T11:20:06.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.143+0000 7f7db5e4c700 1 -- 172.21.15.181:0/2158232764 wait complete. 2023-10-16T11:20:06.147 INFO:tasks.ceph.mds.d.smithi190.stderr: -5796> 2023-10-16T11:19:25.516+0000 7fc17649f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:20:06.147 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:20:06.157 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:20:06.178 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:20:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.278+0000 7fef81eb8700 1 Processor -- start 2023-10-16T11:20:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.278+0000 7fef81eb8700 1 -- start start 2023-10-16T11:20:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.278+0000 7fef81eb8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fef7c0cdd80 0x7fef7c0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.278+0000 7fef81eb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 0x7fef7c06bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.278+0000 7fef81eb8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c15dfb0 con 0x7fef7c15dbd0 2023-10-16T11:20:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.278+0000 7fef81eb8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7c0ce740 con 0x7fef7c0cdd80 2023-10-16T11:20:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.278+0000 7fef81eb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0cd3a0 con 0x7fef7c06b6f0 2023-10-16T11:20:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fef7c15dbd0 0x7fef7c0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39998/0 (socket says 172.21.15.181:39998) 2023-10-16T11:20:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7b7fe700 1 -- 172.21.15.181:0/4237883277 learned_addr learned my addr 172.21.15.181:0/4237883277 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:06.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7bfff700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 0x7fef7c06bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496250116 0 0) 0x7fef6c001110 con 0x7fef7c15dbd0 2023-10-16T11:20:06.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef640035b0 con 0x7fef7c15dbd0 2023-10-16T11:20:06.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7affd700 1 --2- 172.21.15.181:0/4237883277 >> v2:172.21.15.190:3300/0 conn(0x7fef7c0cdd80 0x7fef7c0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474209548 0 0) 0x7fef640035b0 con 0x7fef7c15dbd0 2023-10-16T11:20:06.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 msgr2=0x7fef7c06bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 0x7fef7c06bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 >> v2:172.21.15.190:3300/0 conn(0x7fef7c0cdd80 msgr2=0x7fef7c0ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 --2- 172.21.15.181:0/4237883277 >> v2:172.21.15.190:3300/0 conn(0x7fef7c0cdd80 0x7fef7c0ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0cd830 con 0x7fef7c15dbd0 2023-10-16T11:20:06.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fef6c001800 con 0x7fef7c15dbd0 2023-10-16T11:20:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 >> v1:172.21.15.181:6789/0 conn(0x7fef7c15dbd0 legacy=0x7fef7c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 0x7fef64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef64004d90 0x7fef64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0ce740 con 0x7fef64003a00 2023-10-16T11:20:06.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0cd3a0 con 0x7fef64004d90 2023-10-16T11:20:06.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6400ba40 con 0x7fef7c06b6f0 2023-10-16T11:20:06.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7affd700 1 --2- 172.21.15.181:0/4237883277 >> v2:172.21.15.190:3300/0 conn(0x7fef7c0cdd80 0x7fef7c0ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:06.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7b7fe700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7bfff700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef64004d90 0x7fef64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.279+0000 7fef7affd700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 0x7fef64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7b7fe700 1 -- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 msgr2=0x7fef64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:06.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7b7fe700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c06b6f0 0x7fef64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7b7fe700 1 -- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef64004d90 msgr2=0x7fef64005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:06.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7b7fe700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef64004d90 0x7fef64005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7b7fe700 1 -- 172.21.15.181:0/4237883277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640035b0 con 0x7fef64003a00 2023-10-16T11:20:06.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7b7fe700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef64003e20 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c320 tx=0x7fef64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387ec65234224899 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:06.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00ce60 con 0x7fef64003a00 2023-10-16T11:20:06.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c005960 con 0x7fef64003a00 2023-10-16T11:20:06.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7a7fc700 1 -- 172.21.15.181:0/4237883277 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c005d10 con 0x7fef64003a00 2023-10-16T11:20:06.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef81eb8700 1 -- 172.21.15.181:0/4237883277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 msgr2=0x7fef64003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:06.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef81eb8700 1 --2- 172.21.15.181:0/4237883277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef64003e20 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c320 tx=0x7fef64005880 comp rx=0 tx=0).stop 2023-10-16T11:20:06.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef7bfff700 1 -- 172.21.15.181:0/4237883277 reap_dead start 2023-10-16T11:20:06.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef81eb8700 1 -- 172.21.15.181:0/4237883277 shutdown_connections 2023-10-16T11:20:06.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef81eb8700 1 -- 172.21.15.181:0/4237883277 >> 172.21.15.181:0/4237883277 conn(0x7fef7c09a930 msgr2=0x7fef7c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:06.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef81eb8700 1 -- 172.21.15.181:0/4237883277 shutdown_connections 2023-10-16T11:20:06.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.280+0000 7fef81eb8700 1 -- 172.21.15.181:0/4237883277 wait complete. 2023-10-16T11:20:06.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.282+0000 7fef81eb8700 1 Processor -- start 2023-10-16T11:20:06.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.282+0000 7fef81eb8700 1 -- start start 2023-10-16T11:20:06.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef7c1601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0b5bb0 0x7fef7c1f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c160da0 0x7fef7c0b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1f41a0 con 0x7fef64003a00 2023-10-16T11:20:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c15fdd0 con 0x7fef7c160da0 2023-10-16T11:20:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c15f190 con 0x7fef7c0b5bb0 2023-10-16T11:20:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef7c1601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef7c1601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57164/0 (socket says 172.21.15.181:57164) 2023-10-16T11:20:06.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 -- 172.21.15.181:0/1879615951 learned_addr learned my addr 172.21.15.181:0/1879615951 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7bfff700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c160da0 0x7fef7c0b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7affd700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0b5bb0 0x7fef7c1f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 -- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0b5bb0 msgr2=0x7fef7c1f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0b5bb0 0x7fef7c1f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 -- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c160da0 msgr2=0x7fef7c0b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:06.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c160da0 0x7fef7c0b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:06.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 -- 172.21.15.181:0/1879615951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c15f430 con 0x7fef64003a00 2023-10-16T11:20:06.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef7b7fe700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef7c1601f0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fef6c001ac0 tx=0x7fef6c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:06.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c004870 con 0x7fef64003a00 2023-10-16T11:20:06.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c0049f0 con 0x7fef64003a00 2023-10-16T11:20:06.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c15f710 con 0x7fef64003a00 2023-10-16T11:20:06.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c004da0 con 0x7fef64003a00 2023-10-16T11:20:06.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.283+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c161220 con 0x7fef64003a00 2023-10-16T11:20:06.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.284+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c1613b0 con 0x7fef64003a00 2023-10-16T11:20:06.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.285+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 147) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fef6c005040 con 0x7fef64003a00 2023-10-16T11:20:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.285+0000 7fef63fff700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fef6806ad50 0x7fef6806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.285+0000 7fef7affd700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fef6806ad50 0x7fef6806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.285+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 8028+0+0 (secure 0 0 0) 0x7fef6c08b140 con 0x7fef64003a00 2023-10-16T11:20:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.286+0000 7fef7affd700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fef6806ad50 0x7fef6806d210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0b74c0 tx=0x7fef70008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:06.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.290+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fef6c055d40 con 0x7fef64003a00 2023-10-16T11:20:06.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.531+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fef7c067d20 con 0x7fef64003a00 2023-10-16T11:20:06.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:06.752+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 148) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fef6c056110 con 0x7fef64003a00 2023-10-16T11:20:07.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.140+0000 7fef63fff700 1 -- 172.21.15.181:0/1879615951 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v119) v1 ==== 104+0+0 (secure 0 0 0) 0x7fef6c05d5c0 con 0x7fef64003a00 2023-10-16T11:20:07.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fef6806ad50 msgr2=0x7fef6806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:07.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fef6806ad50 0x7fef6806d210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0b74c0 tx=0x7fef70008040 comp rx=0 tx=0).stop 2023-10-16T11:20:07.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 msgr2=0x7fef7c1601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:07.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef7c1601f0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fef6c001ac0 tx=0x7fef6c001b90 comp rx=0 tx=0).stop 2023-10-16T11:20:07.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 shutdown_connections 2023-10-16T11:20:07.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fef6806ad50 0x7fef6806d210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef7c160da0 0x7fef7c0b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef64003a00 0x7fef7c1601f0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 --2- 172.21.15.181:0/1879615951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef7c0b5bb0 0x7fef7c1f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 >> 172.21.15.181:0/1879615951 conn(0x7fef7c09a930 msgr2=0x7fef7c1615e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:07.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 shutdown_connections 2023-10-16T11:20:07.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.143+0000 7fef81eb8700 1 -- 172.21.15.181:0/1879615951 wait complete. 2023-10-16T11:20:07.159 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:20:07.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.284+0000 7fb7c3079700 1 Processor -- start 2023-10-16T11:20:07.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.284+0000 7fb7c3079700 1 -- start start 2023-10-16T11:20:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.284+0000 7fb7c3079700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7bc0b0f70 0x7fb7bc0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.284+0000 7fb7c3079700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7bc0ce810 0x7fb7bc15f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:07.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.284+0000 7fb7c3079700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7bc0d23c0 con 0x7fb7bc15eea0 2023-10-16T11:20:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.284+0000 7fb7c3079700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc0cd990 con 0x7fb7bc0b0f70 2023-10-16T11:20:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.284+0000 7fb7c3079700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7bc0cdc70 con 0x7fb7bc0ce810 2023-10-16T11:20:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7bc0b0f70 0x7fb7bc0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7bc0b0f70 0x7fb7bc0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57172/0 (socket says 172.21.15.181:57172) 2023-10-16T11:20:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 -- 172.21.15.181:0/2952373156 learned_addr learned my addr 172.21.15.181:0/2952373156 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:07.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7c1616700 1 --2- 172.21.15.181:0/2952373156 >> v2:172.21.15.190:3300/0 conn(0x7fb7bc0ce810 0x7fb7bc15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 -- 172.21.15.181:0/2952373156 >> v2:172.21.15.190:3300/0 conn(0x7fb7bc0ce810 msgr2=0x7fb7bc15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 --2- 172.21.15.181:0/2952373156 >> v2:172.21.15.190:3300/0 conn(0x7fb7bc0ce810 0x7fb7bc15f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 -- 172.21.15.181:0/2952373156 >> v1:172.21.15.190:6789/0 conn(0x7fb7bc15eea0 legacy=0x7fb7bc0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:07.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 -- 172.21.15.181:0/2952373156 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc0cce60 con 0x7fb7bc0b0f70 2023-10-16T11:20:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7c1616700 1 --2- 172.21.15.181:0/2952373156 >> v2:172.21.15.190:3300/0 conn(0x7fb7bc0ce810 0x7fb7bc15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bbfff700 1 --2- 172.21.15.181:0/2952373156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc0b0f70 0x7fb7bc0b03e0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0b0e80 tx=0x7fb7a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca90906b87b421d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bb7fe700 1 -- 172.21.15.181:0/2952373156 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800b870 con 0x7fb7bc0b0f70 2023-10-16T11:20:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bb7fe700 1 -- 172.21.15.181:0/2952373156 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a800b9d0 con 0x7fb7bc0b0f70 2023-10-16T11:20:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.285+0000 7fb7bb7fe700 1 -- 172.21.15.181:0/2952373156 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800bd60 con 0x7fb7bc0b0f70 2023-10-16T11:20:07.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 -- 172.21.15.181:0/2952373156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc0b0f70 msgr2=0x7fb7bc0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 --2- 172.21.15.181:0/2952373156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc0b0f70 0x7fb7bc0b03e0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0b0e80 tx=0x7fb7a8009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 -- 172.21.15.181:0/2952373156 shutdown_connections 2023-10-16T11:20:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 --2- 172.21.15.181:0/2952373156 >> v2:172.21.15.190:3300/0 conn(0x7fb7bc0ce810 0x7fb7bc15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 --2- 172.21.15.181:0/2952373156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc0b0f70 0x7fb7bc0b03e0 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 -- 172.21.15.181:0/2952373156 >> 172.21.15.181:0/2952373156 conn(0x7fb7bc0d1610 msgr2=0x7fb7bc0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:07.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 -- 172.21.15.181:0/2952373156 shutdown_connections 2023-10-16T11:20:07.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.286+0000 7fb7c3079700 1 -- 172.21.15.181:0/2952373156 wait complete. 2023-10-16T11:20:07.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.287+0000 7fb7c3079700 1 Processor -- start 2023-10-16T11:20:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.287+0000 7fb7c3079700 1 -- start start 2023-10-16T11:20:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c3079700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7bc0b0f70 0x7fb7bc15d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c3079700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7bc0ce810 0x7fb7bc0b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c3079700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc15eea0 0x7fb7bc1f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c3079700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0b6ad0 con 0x7fb7bc15eea0 2023-10-16T11:20:07.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c3079700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0c9730 con 0x7fb7bc0ce810 2023-10-16T11:20:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c3079700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc0c9a10 con 0x7fb7bc0b0f70 2023-10-16T11:20:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc15eea0 0x7fb7bc1f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc15eea0 0x7fb7bc1f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57182/0 (socket says 172.21.15.181:57182) 2023-10-16T11:20:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 -- 172.21.15.181:0/3676670749 learned_addr learned my addr 172.21.15.181:0/3676670749 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:07.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7bbfff700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7bc0ce810 0x7fb7bc0b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c0e15700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7bc0b0f70 0x7fb7bc15d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 -- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7bc0b0f70 msgr2=0x7fb7bc15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7bc0b0f70 0x7fb7bc15d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 -- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7bc0ce810 msgr2=0x7fb7bc0b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:07.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7bc0ce810 0x7fb7bc0b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 -- 172.21.15.181:0/3676670749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8009550 con 0x7fb7bc15eea0 2023-10-16T11:20:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7c1616700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc15eea0 0x7fb7bc1f4870 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4007ef0 tx=0x7fb7b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.288+0000 7fb7b97fa700 1 -- 172.21.15.181:0/3676670749 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400de60 con 0x7fb7bc15eea0 2023-10-16T11:20:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.289+0000 7fb7b97fa700 1 -- 172.21.15.181:0/3676670749 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b400f040 con 0x7fb7bc15eea0 2023-10-16T11:20:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.289+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc0c9cf0 con 0x7fb7bc15eea0 2023-10-16T11:20:07.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.289+0000 7fb7b97fa700 1 -- 172.21.15.181:0/3676670749 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4003c60 con 0x7fb7bc15eea0 2023-10-16T11:20:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.289+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc15d9a0 con 0x7fb7bc15eea0 2023-10-16T11:20:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.289+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7bc0ca210 con 0x7fb7bc15eea0 2023-10-16T11:20:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.290+0000 7fb7b97fa700 1 -- 172.21.15.181:0/3676670749 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 148) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb7b4004030 con 0x7fb7bc15eea0 2023-10-16T11:20:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.290+0000 7fb7b97fa700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb7a006b0b0 0x7fb7a006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:07.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.291+0000 7fb7b97fa700 1 -- 172.21.15.181:0/3676670749 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 8028+0+0 (secure 0 0 0) 0x7fb7b408a6b0 con 0x7fb7bc15eea0 2023-10-16T11:20:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.291+0000 7fb7c0e15700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb7a006b0b0 0x7fb7a006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.291+0000 7fb7c0e15700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb7a006b0b0 0x7fb7a006d570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0000f60 tx=0x7fb7b0009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:07.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.294+0000 7fb7b97fa700 1 -- 172.21.15.181:0/3676670749 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb7b4054c30 con 0x7fb7bc15eea0 2023-10-16T11:20:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:07.487+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7bc0ca210 con 0x7fb7bc15eea0 2023-10-16T11:20:08.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.326+0000 7fb7b97fa700 1 -- 172.21.15.181:0/3676670749 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v224) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb7bc0ca210 con 0x7fb7bc15eea0 2023-10-16T11:20:08.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb7a006b0b0 msgr2=0x7fb7a006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:08.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb7a006b0b0 0x7fb7a006d570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0000f60 tx=0x7fb7b0009040 comp rx=0 tx=0).stop 2023-10-16T11:20:08.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc15eea0 msgr2=0x7fb7bc1f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:08.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc15eea0 0x7fb7bc1f4870 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4007ef0 tx=0x7fb7b400d3b0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 shutdown_connections 2023-10-16T11:20:08.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb7a006b0b0 0x7fb7a006d570 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0000f60 tx=0x7fb7b0009040 comp rx=0 tx=0).stop 2023-10-16T11:20:08.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7bc0b0f70 0x7fb7bc15d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7bc15eea0 0x7fb7bc1f4870 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 --2- 172.21.15.181:0/3676670749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7bc0ce810 0x7fb7bc0b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.329+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 >> 172.21.15.181:0/3676670749 conn(0x7fb7bc0d1610 msgr2=0x7fb7bc0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:08.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.333+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 shutdown_connections 2023-10-16T11:20:08.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.334+0000 7fb7c3079700 1 -- 172.21.15.181:0/3676670749 wait complete. 2023-10-16T11:20:08.343 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:20:08.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:20:08.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.480+0000 7f7a8dd15700 1 Processor -- start 2023-10-16T11:20:08.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.480+0000 7f7a8dd15700 1 -- start start 2023-10-16T11:20:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.481+0000 7f7a8dd15700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a880afa10 0x7f7a880ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:08.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.481+0000 7f7a8dd15700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a8806d870 0x7f7a880cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.481+0000 7f7a8dd15700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a880cd850 con 0x7f7a880cd3f0 2023-10-16T11:20:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.481+0000 7f7a8dd15700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8815b540 con 0x7f7a880afa10 2023-10-16T11:20:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.481+0000 7f7a8dd15700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8815b820 con 0x7f7a8806d870 2023-10-16T11:20:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.481+0000 7f7a86ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a8806d870 0x7f7a880cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:08.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.481+0000 7f7a877fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a880afa10 0x7f7a880ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a877fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a880afa10 0x7f7a880ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:34004/0 (socket says 172.21.15.181:34004) 2023-10-16T11:20:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a86ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a8806d870 0x7f7a880cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:54214/0 (socket says 172.21.15.181:54214) 2023-10-16T11:20:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a877fe700 1 -- 172.21.15.181:0/3009019925 learned_addr learned my addr 172.21.15.181:0/3009019925 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a86ffd700 1 -- 172.21.15.181:0/3009019925 >> v2:172.21.15.190:3300/0 conn(0x7f7a880afa10 msgr2=0x7f7a880ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:08.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a86ffd700 1 --2- 172.21.15.181:0/3009019925 >> v2:172.21.15.190:3300/0 conn(0x7f7a880afa10 0x7f7a880ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a86ffd700 1 -- 172.21.15.181:0/3009019925 >> v1:172.21.15.190:6789/0 conn(0x7f7a880cd3f0 legacy=0x7f7a880cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a86ffd700 1 -- 172.21.15.181:0/3009019925 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8815bb00 con 0x7f7a8806d870 2023-10-16T11:20:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a877fe700 1 --2- 172.21.15.181:0/3009019925 >> v2:172.21.15.190:3300/0 conn(0x7f7a880afa10 0x7f7a880ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a86ffd700 1 --2- 172.21.15.181:0/3009019925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a8806d870 0x7f7a880cceb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009840 tx=0x7f7a78009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f945cf26c972e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:08.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a867fc700 1 -- 172.21.15.181:0/3009019925 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78011830 con 0x7f7a8806d870 2023-10-16T11:20:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a867fc700 1 -- 172.21.15.181:0/3009019925 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78011990 con 0x7f7a8806d870 2023-10-16T11:20:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.482+0000 7f7a867fc700 1 -- 172.21.15.181:0/3009019925 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78011d20 con 0x7f7a8806d870 2023-10-16T11:20:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 -- 172.21.15.181:0/3009019925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a8806d870 msgr2=0x7f7a880cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/3009019925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a8806d870 0x7f7a880cceb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009840 tx=0x7f7a78009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:08.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 -- 172.21.15.181:0/3009019925 shutdown_connections 2023-10-16T11:20:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/3009019925 >> v2:172.21.15.190:3300/0 conn(0x7f7a880afa10 0x7f7a880ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/3009019925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a8806d870 0x7f7a880cceb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 -- 172.21.15.181:0/3009019925 >> 172.21.15.181:0/3009019925 conn(0x7f7a880d04a0 msgr2=0x7f7a880d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 -- 172.21.15.181:0/3009019925 shutdown_connections 2023-10-16T11:20:08.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.483+0000 7f7a8dd15700 1 -- 172.21.15.181:0/3009019925 wait complete. 2023-10-16T11:20:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.484+0000 7f7a8dd15700 1 Processor -- start 2023-10-16T11:20:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.484+0000 7f7a8dd15700 1 -- start start 2023-10-16T11:20:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a8dd15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8806d870 0x7f7a880b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a8dd15700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a880afa10 0x7f7a880b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:08.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a8dd15700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a880cd3f0 0x7f7a8814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a8dd15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8814f3e0 con 0x7f7a8806d870 2023-10-16T11:20:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a8dd15700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a881510d0 con 0x7f7a880cd3f0 2023-10-16T11:20:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a8dd15700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8814f980 con 0x7f7a880afa10 2023-10-16T11:20:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8806d870 0x7f7a880b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:08.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8806d870 0x7f7a880b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57192/0 (socket says 172.21.15.181:57192) 2023-10-16T11:20:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 -- 172.21.15.181:0/1833169030 learned_addr learned my addr 172.21.15.181:0/1833169030 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a86ffd700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a880afa10 0x7f7a880b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a87fff700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a880cd3f0 0x7f7a8814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 -- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a880afa10 msgr2=0x7f7a880b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:08.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a880afa10 0x7f7a880b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 -- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a880cd3f0 msgr2=0x7f7a8814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a880cd3f0 0x7f7a8814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a877fe700 1 -- 172.21.15.181:0/1833169030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c009500 con 0x7f7a8806d870 2023-10-16T11:20:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a87fff700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a880cd3f0 0x7f7a8814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:08.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.485+0000 7f7a86ffd700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a880afa10 0x7f7a880b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.486+0000 7f7a877fe700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8806d870 0x7f7a880b5ad0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00ebb0 tx=0x7f7a7c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.486+0000 7f7a6ffff700 1 -- 172.21.15.181:0/1833169030 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c005a20 con 0x7f7a8806d870 2023-10-16T11:20:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.486+0000 7f7a6ffff700 1 -- 172.21.15.181:0/1833169030 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c005b80 con 0x7f7a8806d870 2023-10-16T11:20:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.486+0000 7f7a6ffff700 1 -- 172.21.15.181:0/1833169030 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00c8b0 con 0x7f7a8806d870 2023-10-16T11:20:08.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.486+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a780094a0 con 0x7f7a8806d870 2023-10-16T11:20:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.486+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8815e450 con 0x7f7a8806d870 2023-10-16T11:20:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.487+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8814fc60 con 0x7f7a8806d870 2023-10-16T11:20:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.487+0000 7f7a6ffff700 1 -- 172.21.15.181:0/1833169030 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7a7c00ca10 con 0x7f7a8806d870 2023-10-16T11:20:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.488+0000 7f7a6ffff700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a7406b140 0x7f7a7406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:08.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.488+0000 7f7a6ffff700 1 -- 172.21.15.181:0/1833169030 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 7590+0+0 (secure 0 0 0) 0x7f7a7c08a490 con 0x7f7a8806d870 2023-10-16T11:20:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.488+0000 7f7a86ffd700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a7406b140 0x7f7a7406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.488+0000 7f7a86ffd700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a7406b140 0x7f7a7406d600 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a78008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:08.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.492+0000 7f7a6ffff700 1 -- 172.21.15.181:0/1833169030 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a7c054a80 con 0x7f7a8806d870 2023-10-16T11:20:08.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:08.691+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a880ccd20 con 0x7f7a8806d870 2023-10-16T11:20:09.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.328+0000 7f7a6ffff700 1 -- 172.21.15.181:0/1833169030 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v225) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7a7c0580a0 con 0x7f7a8806d870 2023-10-16T11:20:09.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a7406b140 msgr2=0x7f7a7406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a7406b140 0x7f7a7406d600 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a78008030 comp rx=0 tx=0).stop 2023-10-16T11:20:09.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8806d870 msgr2=0x7f7a880b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8806d870 0x7f7a880b5ad0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00ebb0 tx=0x7f7a7c00ef70 comp rx=0 tx=0).stop 2023-10-16T11:20:09.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 shutdown_connections 2023-10-16T11:20:09.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a7406b140 0x7f7a7406d600 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a880cd3f0 0x7f7a8814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a8806d870 0x7f7a880b5ad0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 --2- 172.21.15.181:0/1833169030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a880afa10 0x7f7a880b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 >> 172.21.15.181:0/1833169030 conn(0x7f7a880d04a0 msgr2=0x7f7a88151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:09.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 shutdown_connections 2023-10-16T11:20:09.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.331+0000 7f7a8dd15700 1 -- 172.21.15.181:0/1833169030 wait complete. 2023-10-16T11:20:09.336 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:20:09.351 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.476+0000 7f40b5108700 1 Processor -- start 2023-10-16T11:20:09.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.476+0000 7f40b5108700 1 -- start start 2023-10-16T11:20:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40b5108700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 0x7f40b006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40b5108700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00cc470 con 0x7f40b00d2390 2023-10-16T11:20:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40b5108700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b006ca30 con 0x7f40b00d0350 2023-10-16T11:20:09.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40b5108700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b006cd10 con 0x7f40b00ce7f0 2023-10-16T11:20:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40ae59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f40b00d2390 0x7f40b00cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40046/0 (socket says 172.21.15.181:40046) 2023-10-16T11:20:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40ae59c700 1 -- 172.21.15.181:0/1289747206 learned_addr learned my addr 172.21.15.181:0/1289747206 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584986400 0 0) 0x7f40b00cc470 con 0x7f40b00d2390 2023-10-16T11:20:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40980035b0 con 0x7f40b00d2390 2023-10-16T11:20:09.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40aed9d700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 0x7f40b006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690496949 0 0) 0x7f40980035b0 con 0x7f40b00d2390 2023-10-16T11:20:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 msgr2=0x7f40b006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 0x7f40b006c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 >> v1:172.21.15.190:6789/0 conn(0x7f40b00d0350 legacy=0x7f40b00cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b006cff0 con 0x7f40b00d2390 2023-10-16T11:20:09.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f40a0001670 con 0x7f40b00d2390 2023-10-16T11:20:09.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.477+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 >> v1:172.21.15.181:6789/0 conn(0x7f40b00d2390 legacy=0x7f40b00cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4098003a30 0x7f4098003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 0x7f4098004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4098004d90 0x7f4098005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b006cd10 con 0x7f4098004d90 2023-10-16T11:20:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409800bac0 con 0x7f4098003a30 2023-10-16T11:20:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409800bc80 con 0x7f40b00ce7f0 2023-10-16T11:20:09.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40af59e700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4098004d90 0x7f4098005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40ae59c700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 0x7f4098004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40af59e700 1 -- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 msgr2=0x7f4098004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40af59e700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b00ce7f0 0x7f4098004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40af59e700 1 -- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4098003a30 msgr2=0x7f4098003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:09.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40af59e700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4098003a30 0x7f4098003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40af59e700 1 -- 172.21.15.181:0/1289747206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980035b0 con 0x7f4098004d90 2023-10-16T11:20:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40af59e700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4098004d90 0x7f4098005210 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f40b0072ba0 tx=0x7f4098005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a57e14735af2241 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8001db0 con 0x7f4098004d90 2023-10-16T11:20:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a800cc70 con 0x7f4098004d90 2023-10-16T11:20:09.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.478+0000 7f40add9b700 1 -- 172.21.15.181:0/1289747206 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003a00 con 0x7f4098004d90 2023-10-16T11:20:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.479+0000 7f40b5108700 1 -- 172.21.15.181:0/1289747206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4098004d90 msgr2=0x7f4098005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.479+0000 7f40b5108700 1 --2- 172.21.15.181:0/1289747206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4098004d90 0x7f4098005210 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f40b0072ba0 tx=0x7f4098005880 comp rx=0 tx=0).stop 2023-10-16T11:20:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.479+0000 7f40af59e700 1 -- 172.21.15.181:0/1289747206 reap_dead start 2023-10-16T11:20:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.479+0000 7f40b5108700 1 -- 172.21.15.181:0/1289747206 shutdown_connections 2023-10-16T11:20:09.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.479+0000 7f40b5108700 1 -- 172.21.15.181:0/1289747206 >> 172.21.15.181:0/1289747206 conn(0x7f40b009a9a0 msgr2=0x7f40b00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:09.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.479+0000 7f40b5108700 1 -- 172.21.15.181:0/1289747206 shutdown_connections 2023-10-16T11:20:09.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.479+0000 7f40b5108700 1 -- 172.21.15.181:0/1289747206 wait complete. 2023-10-16T11:20:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.480+0000 7f40b5108700 1 Processor -- start 2023-10-16T11:20:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.480+0000 7f40b5108700 1 -- start start 2023-10-16T11:20:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40b5108700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b015dc50 0x7f40b01f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40b5108700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b009df40 0x7f40b009e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40b5108700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b009c940 0x7f40b009cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40b5108700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b009e900 con 0x7f40b009c940 2023-10-16T11:20:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40b5108700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b009b8a0 con 0x7f40b009df40 2023-10-16T11:20:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40b5108700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b009bb80 con 0x7f40b015dc50 2023-10-16T11:20:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40af59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b009c940 0x7f40b009cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40af59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b009c940 0x7f40b009cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57212/0 (socket says 172.21.15.181:57212) 2023-10-16T11:20:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40af59e700 1 -- 172.21.15.181:0/2876180677 learned_addr learned my addr 172.21.15.181:0/2876180677 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40ae59c700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b009df40 0x7f40b009e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40aed9d700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b015dc50 0x7f40b01f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40af59e700 1 -- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b015dc50 msgr2=0x7f40b01f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40af59e700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b015dc50 0x7f40b01f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40af59e700 1 -- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b009df40 msgr2=0x7f40b009e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.481+0000 7f40af59e700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b009df40 0x7f40b009e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40af59e700 1 -- 172.21.15.181:0/2876180677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b009be60 con 0x7f40b009c940 2023-10-16T11:20:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40aed9d700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b015dc50 0x7f40b01f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40af59e700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b009c940 0x7f40b009cdc0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005040 tx=0x7f40a80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40977fe700 1 -- 172.21.15.181:0/2876180677 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003e10 con 0x7f40b009c940 2023-10-16T11:20:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40977fe700 1 -- 172.21.15.181:0/2876180677 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a80050d0 con 0x7f40b009c940 2023-10-16T11:20:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b009c140 con 0x7f40b009c940 2023-10-16T11:20:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40977fe700 1 -- 172.21.15.181:0/2876180677 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80054a0 con 0x7f40b009c940 2023-10-16T11:20:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.482+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b009d300 con 0x7f40b009c940 2023-10-16T11:20:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.483+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b0067d20 con 0x7f40b009c940 2023-10-16T11:20:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.483+0000 7f40977fe700 1 -- 172.21.15.181:0/2876180677 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f40a8019040 con 0x7f40b009c940 2023-10-16T11:20:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.484+0000 7f40977fe700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f409c06b140 0x7f409c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.484+0000 7f40aed9d700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f409c06b140 0x7f409c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.484+0000 7f40977fe700 1 -- 172.21.15.181:0/2876180677 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7f40a808ac30 con 0x7f40b009c940 2023-10-16T11:20:09.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.484+0000 7f40aed9d700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f409c06b140 0x7f409c06d600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f40a4004840 tx=0x7f40a4008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:09.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.488+0000 7f40977fe700 1 -- 172.21.15.181:0/2876180677 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f40a8055700 con 0x7f40b009c940 2023-10-16T11:20:09.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.683+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40b00cef40 con 0x7f40b009c940 2023-10-16T11:20:09.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.684+0000 7f40977fe700 1 -- 172.21.15.181:0/2876180677 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+18812 (secure 0 0 0) 0x7f40a8017090 con 0x7f40b009c940 2023-10-16T11:20:09.687 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:09.687 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":225,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:09.327947+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/2033295595":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/4244377789":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2576397499":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6833/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.190:6833/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/754905711":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3499373399":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2963042451":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3469785622":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2131678440":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/134902336":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:09.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f409c06b140 msgr2=0x7f409c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f409c06b140 0x7f409c06d600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f40a4004840 tx=0x7f40a4008040 comp rx=0 tx=0).stop 2023-10-16T11:20:09.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b009c940 msgr2=0x7f40b009cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b009c940 0x7f40b009cdc0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005040 tx=0x7f40a80014a0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 shutdown_connections 2023-10-16T11:20:09.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f409c06b140 0x7f409c06d600 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40b015dc50 0x7f40b01f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40b009c940 0x7f40b009cdc0 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 --2- 172.21.15.181:0/2876180677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40b009df40 0x7f40b009e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 >> 172.21.15.181:0/2876180677 conn(0x7f40b009a9a0 msgr2=0x7f40b00c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:09.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 shutdown_connections 2023-10-16T11:20:09.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.686+0000 7f40b5108700 1 -- 172.21.15.181:0/2876180677 wait complete. 2023-10-16T11:20:09.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:19:33.658268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '219', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:19:34.982934+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:20:09.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:20:09.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.824+0000 7f3a5d894700 1 Processor -- start 2023-10-16T11:20:09.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.824+0000 7f3a5d894700 1 -- start start 2023-10-16T11:20:09.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.825+0000 7f3a5d894700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 0x7f3a580d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.825+0000 7f3a5d894700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3a580d0610 0x7f3a580cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.825+0000 7f3a5d894700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5815dfb0 con 0x7f3a580ce2c0 2023-10-16T11:20:09.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.825+0000 7f3a5d894700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a580ce740 con 0x7f3a580d0610 2023-10-16T11:20:09.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.825+0000 7f3a5d894700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580cd3a0 con 0x7f3a5815dbd0 2023-10-16T11:20:09.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.825+0000 7f3a577fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3a580ce2c0 0x7f3a58161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40068/0 (socket says 172.21.15.181:40068) 2023-10-16T11:20:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.825+0000 7f3a577fe700 1 -- 172.21.15.181:0/2922589160 learned_addr learned my addr 172.21.15.181:0/2922589160 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a56ffd700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 0x7f3a580d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 --2- 172.21.15.181:0/2922589160 >> v2:172.21.15.190:3300/0 conn(0x7f3a580d0610 0x7f3a580cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207530721 0 0) 0x7f3a50001490 con 0x7f3a580ce2c0 2023-10-16T11:20:09.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a400035b0 con 0x7f3a580ce2c0 2023-10-16T11:20:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2102783258 0 0) 0x7f3a400035b0 con 0x7f3a580ce2c0 2023-10-16T11:20:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 msgr2=0x7f3a580d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 0x7f3a580d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 >> v2:172.21.15.190:3300/0 conn(0x7f3a580d0610 msgr2=0x7f3a580cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:09.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 --2- 172.21.15.181:0/2922589160 >> v2:172.21.15.190:3300/0 conn(0x7f3a580d0610 0x7f3a580cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a580cd830 con 0x7f3a580ce2c0 2023-10-16T11:20:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f3a50001b80 con 0x7f3a580ce2c0 2023-10-16T11:20:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 >> v1:172.21.15.181:6789/0 conn(0x7f3a580ce2c0 legacy=0x7f3a58161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 0x7f3a40003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a40004d90 0x7f3a40005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580ce740 con 0x7f3a40003f80 2023-10-16T11:20:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580cd3a0 con 0x7f3a40004d90 2023-10-16T11:20:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4000ba10 con 0x7f3a5815dbd0 2023-10-16T11:20:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a56ffd700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 0x7f3a40003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a577fe700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a40004d90 0x7f3a40005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 -- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 msgr2=0x7f3a40003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:09.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a5815dbd0 0x7f3a40003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 -- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a40004d90 msgr2=0x7f3a40005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:09.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a40004d90 0x7f3a40005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 -- 172.21.15.181:0/2922589160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a400035b0 con 0x7f3a40003f80 2023-10-16T11:20:09.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.826+0000 7f3a567fc700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a40004850 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009b30 tx=0x7f3a40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1ec394e8fb90a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:09.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800e660 con 0x7f3a40003f80 2023-10-16T11:20:09.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4800e7c0 con 0x7f3a40003f80 2023-10-16T11:20:09.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a55ffb700 1 -- 172.21.15.181:0/2922589160 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800eb30 con 0x7f3a40003f80 2023-10-16T11:20:09.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a5d894700 1 -- 172.21.15.181:0/2922589160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 msgr2=0x7f3a40004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a5d894700 1 --2- 172.21.15.181:0/2922589160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a40004850 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009b30 tx=0x7f3a40005880 comp rx=0 tx=0).stop 2023-10-16T11:20:09.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a577fe700 1 -- 172.21.15.181:0/2922589160 reap_dead start 2023-10-16T11:20:09.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a5d894700 1 -- 172.21.15.181:0/2922589160 shutdown_connections 2023-10-16T11:20:09.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a5d894700 1 -- 172.21.15.181:0/2922589160 >> 172.21.15.181:0/2922589160 conn(0x7f3a5809a930 msgr2=0x7f3a580d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a5d894700 1 -- 172.21.15.181:0/2922589160 shutdown_connections 2023-10-16T11:20:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.827+0000 7f3a5d894700 1 -- 172.21.15.181:0/2922589160 wait complete. 2023-10-16T11:20:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.828+0000 7f3a5d894700 1 Processor -- start 2023-10-16T11:20:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a5d894700 1 -- start start 2023-10-16T11:20:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a5d894700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a580b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a5d894700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a580b4a90 0x7f3a580b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a5d894700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a5815a910 0x7f3a58150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a5d894700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580b4600 con 0x7f3a40003f80 2023-10-16T11:20:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a5d894700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580b39f0 con 0x7f3a5815a910 2023-10-16T11:20:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a5d894700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580b3cd0 con 0x7f3a580b4a90 2023-10-16T11:20:09.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a56ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a580b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a56ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a580b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57230/0 (socket says 172.21.15.181:57230) 2023-10-16T11:20:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a56ffd700 1 -- 172.21.15.181:0/870183775 learned_addr learned my addr 172.21.15.181:0/870183775 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.829+0000 7f3a567fc700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a580b4a90 0x7f3a580b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a577fe700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a5815a910 0x7f3a58150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a56ffd700 1 -- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a580b4a90 msgr2=0x7f3a580b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a56ffd700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a580b4a90 0x7f3a580b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a56ffd700 1 -- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a5815a910 msgr2=0x7f3a58150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a56ffd700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a5815a910 0x7f3a58150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a56ffd700 1 -- 172.21.15.181:0/870183775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a580b3fb0 con 0x7f3a40003f80 2023-10-16T11:20:09.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a567fc700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a580b4a90 0x7f3a580b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a56ffd700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a580b30f0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c011aa0 tx=0x7f3a4c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a3ffff700 1 -- 172.21.15.181:0/870183775 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00f980 con 0x7f3a40003f80 2023-10-16T11:20:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a3ffff700 1 -- 172.21.15.181:0/870183775 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c00fae0 con 0x7f3a40003f80 2023-10-16T11:20:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a580b4290 con 0x7f3a40003f80 2023-10-16T11:20:09.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a3ffff700 1 -- 172.21.15.181:0/870183775 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00fe90 con 0x7f3a40003f80 2023-10-16T11:20:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.830+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a580b6080 con 0x7f3a40003f80 2023-10-16T11:20:09.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.831+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a580b48e0 con 0x7f3a40003f80 2023-10-16T11:20:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.831+0000 7f3a3ffff700 1 -- 172.21.15.181:0/870183775 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3a4c014440 con 0x7f3a40003f80 2023-10-16T11:20:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.832+0000 7f3a3ffff700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a4406b140 0x7f3a4406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.832+0000 7f3a3ffff700 1 -- 172.21.15.181:0/870183775 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7f3a4c08a870 con 0x7f3a40003f80 2023-10-16T11:20:09.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.832+0000 7f3a567fc700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a4406b140 0x7f3a4406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.832+0000 7f3a567fc700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a4406b140 0x7f3a4406d600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000c00 tx=0x7f3a4801e040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:09.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:09.836+0000 7f3a3ffff700 1 -- 172.21.15.181:0/870183775 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3a4c054f90 con 0x7f3a40003f80 2023-10-16T11:20:10.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.029+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a58067d20 con 0x7f3a40003f80 2023-10-16T11:20:10.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.030+0000 7f3a3ffff700 1 -- 172.21.15.181:0/870183775 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v225) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a4c0585b0 con 0x7f3a40003f80 2023-10-16T11:20:10.032 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:20:10.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a4406b140 msgr2=0x7f3a4406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a4406b140 0x7f3a4406d600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000c00 tx=0x7f3a4801e040 comp rx=0 tx=0).stop 2023-10-16T11:20:10.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 msgr2=0x7f3a580b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a580b30f0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c011aa0 tx=0x7f3a4c011e60 comp rx=0 tx=0).stop 2023-10-16T11:20:10.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 shutdown_connections 2023-10-16T11:20:10.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a4406b140 0x7f3a4406d600 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a5815a910 0x7f3a58150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a40003f80 0x7f3a580b30f0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 --2- 172.21.15.181:0/870183775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a580b4a90 0x7f3a580b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 >> 172.21.15.181:0/870183775 conn(0x7f3a5809a930 msgr2=0x7f3a5806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:10.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 shutdown_connections 2023-10-16T11:20:10.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.032+0000 7f3a5d894700 1 -- 172.21.15.181:0/870183775 wait complete. 2023-10-16T11:20:10.046 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:20:10.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.168+0000 7ff2385a0700 1 Processor -- start 2023-10-16T11:20:10.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.168+0000 7ff2385a0700 1 -- start start 2023-10-16T11:20:10.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.169+0000 7ff2385a0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2300ce6c0 0x7ff2300cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.169+0000 7ff2385a0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 0x7ff2300ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.169+0000 7ff2385a0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2300b1f70 con 0x7ff23015f150 2023-10-16T11:20:10.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.169+0000 7ff2385a0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2300b2ae0 con 0x7ff2300ce6c0 2023-10-16T11:20:10.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.169+0000 7ff2385a0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300b2dc0 con 0x7ff2300cc6a0 2023-10-16T11:20:10.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.169+0000 7ff23633c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff23015f150 0x7ff2300cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40082/0 (socket says 172.21.15.181:40082) 2023-10-16T11:20:10.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.169+0000 7ff23633c700 1 -- 172.21.15.181:0/4063178757 learned_addr learned my addr 172.21.15.181:0/4063178757 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:10.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff236b3d700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 0x7ff2300ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff235b3b700 1 --2- 172.21.15.181:0/4063178757 >> v2:172.21.15.190:3300/0 conn(0x7ff2300ce6c0 0x7ff2300cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441726831 0 0) 0x7ff2300b1f70 con 0x7ff23015f150 2023-10-16T11:20:10.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff21c0035b0 con 0x7ff23015f150 2023-10-16T11:20:10.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017458313 0 0) 0x7ff21c0035b0 con 0x7ff23015f150 2023-10-16T11:20:10.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 msgr2=0x7ff2300ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:10.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 0x7ff2300ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 >> v2:172.21.15.190:3300/0 conn(0x7ff2300ce6c0 msgr2=0x7ff2300cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:10.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 --2- 172.21.15.181:0/4063178757 >> v2:172.21.15.190:3300/0 conn(0x7ff2300ce6c0 0x7ff2300cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300cf9d0 con 0x7ff23015f150 2023-10-16T11:20:10.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff220001670 con 0x7ff23015f150 2023-10-16T11:20:10.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 >> v1:172.21.15.181:6789/0 conn(0x7ff23015f150 legacy=0x7ff2300cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 0x7ff21c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c003f80 0x7ff21c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff21c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300b2ae0 con 0x7ff21c004d90 2023-10-16T11:20:10.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300b2dc0 con 0x7ff21c003f80 2023-10-16T11:20:10.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c00ba10 con 0x7ff2300cc6a0 2023-10-16T11:20:10.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff236b3d700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff21c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff23633c700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 0x7ff21c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff235b3b700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c003f80 0x7ff21c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff236b3d700 1 -- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 msgr2=0x7ff21c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:10.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff236b3d700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2300cc6a0 0x7ff21c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff236b3d700 1 -- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c003f80 msgr2=0x7ff21c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:10.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff236b3d700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c003f80 0x7ff21c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.170+0000 7ff236b3d700 1 -- 172.21.15.181:0/4063178757 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0035b0 con 0x7ff21c004d90 2023-10-16T11:20:10.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff236b3d700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff21c005210 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7ff228009fc0 tx=0x7ff21c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a5fd359c904a18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22800ebd0 con 0x7ff21c004d90 2023-10-16T11:20:10.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22800f040 con 0x7ff21c004d90 2023-10-16T11:20:10.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff23533a700 1 -- 172.21.15.181:0/4063178757 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228014480 con 0x7ff21c004d90 2023-10-16T11:20:10.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff2385a0700 1 -- 172.21.15.181:0/4063178757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 msgr2=0x7ff21c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff2385a0700 1 --2- 172.21.15.181:0/4063178757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff21c005210 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7ff228009fc0 tx=0x7ff21c005880 comp rx=0 tx=0).stop 2023-10-16T11:20:10.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff236b3d700 1 -- 172.21.15.181:0/4063178757 reap_dead start 2023-10-16T11:20:10.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff2385a0700 1 -- 172.21.15.181:0/4063178757 shutdown_connections 2023-10-16T11:20:10.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff2385a0700 1 -- 172.21.15.181:0/4063178757 >> 172.21.15.181:0/4063178757 conn(0x7ff23009a930 msgr2=0x7ff2300d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:10.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff2385a0700 1 -- 172.21.15.181:0/4063178757 shutdown_connections 2023-10-16T11:20:10.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.171+0000 7ff2385a0700 1 -- 172.21.15.181:0/4063178757 wait complete. 2023-10-16T11:20:10.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 Processor -- start 2023-10-16T11:20:10.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 -- start start 2023-10-16T11:20:10.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff23006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300aff50 0x7ff2300aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2301e52c0 0x7ff2301e5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301e5f60 con 0x7ff21c004d90 2023-10-16T11:20:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301f7580 con 0x7ff2300aff50 2023-10-16T11:20:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff2385a0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301e5c90 con 0x7ff2301e52c0 2023-10-16T11:20:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff23633c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff23006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.173+0000 7ff23633c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff23006cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57248/0 (socket says 172.21.15.181:57248) 2023-10-16T11:20:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff23633c700 1 -- 172.21.15.181:0/3285139677 learned_addr learned my addr 172.21.15.181:0/3285139677 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff236b3d700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2301e52c0 0x7ff2301e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff235b3b700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300aff50 0x7ff2300aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff23633c700 1 -- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2301e52c0 msgr2=0x7ff2301e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff23633c700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2301e52c0 0x7ff2301e5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff23633c700 1 -- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300aff50 msgr2=0x7ff2300aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff23633c700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300aff50 0x7ff2300aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff23633c700 1 -- 172.21.15.181:0/3285139677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2301f70f0 con 0x7ff21c004d90 2023-10-16T11:20:10.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff23633c700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff23006cef0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7ff220007340 tx=0x7ff220001760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff226ffd700 1 -- 172.21.15.181:0/3285139677 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000f870 con 0x7ff21c004d90 2023-10-16T11:20:10.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff226ffd700 1 -- 172.21.15.181:0/3285139677 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22000f9d0 con 0x7ff21c004d90 2023-10-16T11:20:10.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2301f67c0 con 0x7ff21c004d90 2023-10-16T11:20:10.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff226ffd700 1 -- 172.21.15.181:0/3285139677 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000fd80 con 0x7ff21c004d90 2023-10-16T11:20:10.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.174+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2301f6d80 con 0x7ff21c004d90 2023-10-16T11:20:10.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.175+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff230067d20 con 0x7ff21c004d90 2023-10-16T11:20:10.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.175+0000 7ff226ffd700 1 -- 172.21.15.181:0/3285139677 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff22001e3b0 con 0x7ff21c004d90 2023-10-16T11:20:10.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.176+0000 7ff226ffd700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff21406b140 0x7ff21406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.176+0000 7ff235b3b700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff21406b140 0x7ff21406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.176+0000 7ff226ffd700 1 -- 172.21.15.181:0/3285139677 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7ff220089b90 con 0x7ff21c004d90 2023-10-16T11:20:10.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.176+0000 7ff235b3b700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff21406b140 0x7ff21406d600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009ba0 tx=0x7ff22c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.181+0000 7ff226ffd700 1 -- 172.21.15.181:0/3285139677 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff220054320 con 0x7ff21c004d90 2023-10-16T11:20:10.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.375+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2300cd4f0 con 0x7ff21c004d90 2023-10-16T11:20:10.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.376+0000 7ff226ffd700 1 -- 172.21.15.181:0/3285139677 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v225) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff22001a020 con 0x7ff21c004d90 2023-10-16T11:20:10.378 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff21406b140 msgr2=0x7ff21406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff21406b140 0x7ff21406d600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009ba0 tx=0x7ff22c008040 comp rx=0 tx=0).stop 2023-10-16T11:20:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 msgr2=0x7ff23006cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff23006cef0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7ff220007340 tx=0x7ff220001760 comp rx=0 tx=0).stop 2023-10-16T11:20:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 shutdown_connections 2023-10-16T11:20:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff21406b140 0x7ff21406d600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2301e52c0 0x7ff2301e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff21c004d90 0x7ff23006cef0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 --2- 172.21.15.181:0/3285139677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2300aff50 0x7ff2300aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 >> 172.21.15.181:0/3285139677 conn(0x7ff23009a930 msgr2=0x7ff23015cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:10.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 shutdown_connections 2023-10-16T11:20:10.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.378+0000 7ff2385a0700 1 -- 172.21.15.181:0/3285139677 wait complete. 2023-10-16T11:20:10.392 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:20:10.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.513+0000 7f564b403700 1 Processor -- start 2023-10-16T11:20:10.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.513+0000 7f564b403700 1 -- start start 2023-10-16T11:20:10.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564b403700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56440b2010 0x7f56440cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564b403700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5644071be0 0x7f56440b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564b403700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56440ce1f0 con 0x7f564406f3b0 2023-10-16T11:20:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564b403700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5644072090 con 0x7f5644071be0 2023-10-16T11:20:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564b403700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56440b2930 con 0x7f56440b2010 2023-10-16T11:20:10.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5644071be0 0x7f56440b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5644071be0 0x7f56440b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57258/0 (socket says 172.21.15.181:57258) 2023-10-16T11:20:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 -- 172.21.15.181:0/1325357298 learned_addr learned my addr 172.21.15.181:0/1325357298 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564919f700 1 --2- 172.21.15.181:0/1325357298 >> v2:172.21.15.190:3300/0 conn(0x7f56440b2010 0x7f56440cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 -- 172.21.15.181:0/1325357298 >> v2:172.21.15.190:3300/0 conn(0x7f56440b2010 msgr2=0x7f56440cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:10.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 --2- 172.21.15.181:0/1325357298 >> v2:172.21.15.190:3300/0 conn(0x7f56440b2010 0x7f56440cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 -- 172.21.15.181:0/1325357298 >> v1:172.21.15.190:6789/0 conn(0x7f564406f3b0 legacy=0x7f564406f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 -- 172.21.15.181:0/1325357298 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56440ccda0 con 0x7f5644071be0 2023-10-16T11:20:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f564899e700 1 --2- 172.21.15.181:0/1325357298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5644071be0 0x7f56440b23f0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f5644070430 tx=0x7f5634009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a4a72e513ca6e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.514+0000 7f563bfff700 1 -- 172.21.15.181:0/1325357298 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634004230 con 0x7f5644071be0 2023-10-16T11:20:10.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f563bfff700 1 -- 172.21.15.181:0/1325357298 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634004390 con 0x7f5644071be0 2023-10-16T11:20:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f563bfff700 1 -- 172.21.15.181:0/1325357298 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340036a0 con 0x7f5644071be0 2023-10-16T11:20:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 -- 172.21.15.181:0/1325357298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5644071be0 msgr2=0x7f56440b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 --2- 172.21.15.181:0/1325357298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5644071be0 0x7f56440b23f0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f5644070430 tx=0x7f5634009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 -- 172.21.15.181:0/1325357298 shutdown_connections 2023-10-16T11:20:10.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 --2- 172.21.15.181:0/1325357298 >> v2:172.21.15.190:3300/0 conn(0x7f56440b2010 0x7f56440cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 --2- 172.21.15.181:0/1325357298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5644071be0 0x7f56440b23f0 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 -- 172.21.15.181:0/1325357298 >> 172.21.15.181:0/1325357298 conn(0x7f56440cd010 msgr2=0x7f56440cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:10.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 -- 172.21.15.181:0/1325357298 shutdown_connections 2023-10-16T11:20:10.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.515+0000 7f564b403700 1 -- 172.21.15.181:0/1325357298 wait complete. 2023-10-16T11:20:10.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.516+0000 7f564b403700 1 Processor -- start 2023-10-16T11:20:10.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.516+0000 7f564b403700 1 -- start start 2023-10-16T11:20:10.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564b403700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564406f3b0 0x7f56441530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564b403700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5644071be0 0x7f56441f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564b403700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b2010 0x7f56440c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564b403700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f2030 con 0x7f564406f3b0 2023-10-16T11:20:10.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564b403700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f1a70 con 0x7f56440b2010 2023-10-16T11:20:10.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564b403700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f1d50 con 0x7f5644071be0 2023-10-16T11:20:10.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564406f3b0 0x7f56441530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564406f3b0 0x7f56441530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57262/0 (socket says 172.21.15.181:57262) 2023-10-16T11:20:10.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 -- 172.21.15.181:0/3581182443 learned_addr learned my addr 172.21.15.181:0/3581182443 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:10.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f56499a0700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b2010 0x7f56440c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564899e700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5644071be0 0x7f56441f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 -- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5644071be0 msgr2=0x7f56441f10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5644071be0 0x7f56441f10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 -- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b2010 msgr2=0x7f56440c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b2010 0x7f56440c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.517+0000 7f564919f700 1 -- 172.21.15.181:0/3581182443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5634009550 con 0x7f564406f3b0 2023-10-16T11:20:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.518+0000 7f564919f700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564406f3b0 0x7f56441530b0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f564000c7f0 tx=0x7f564000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.518+0000 7f5639ffb700 1 -- 172.21.15.181:0/3581182443 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011800 con 0x7f564406f3b0 2023-10-16T11:20:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.518+0000 7f5639ffb700 1 -- 172.21.15.181:0/3581182443 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5640011960 con 0x7f564406f3b0 2023-10-16T11:20:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.518+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56440c8750 con 0x7f564406f3b0 2023-10-16T11:20:10.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.518+0000 7f5639ffb700 1 -- 172.21.15.181:0/3581182443 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011cf0 con 0x7f564406f3b0 2023-10-16T11:20:10.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.518+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56441535f0 con 0x7f564406f3b0 2023-10-16T11:20:10.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.519+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56441537e0 con 0x7f564406f3b0 2023-10-16T11:20:10.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.519+0000 7f5639ffb700 1 -- 172.21.15.181:0/3581182443 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5640018360 con 0x7f564406f3b0 2023-10-16T11:20:10.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.519+0000 7f5639ffb700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f562806b140 0x7f562806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.520+0000 7f5639ffb700 1 -- 172.21.15.181:0/3581182443 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7f564008a0c0 con 0x7f564406f3b0 2023-10-16T11:20:10.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.520+0000 7f564899e700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f562806b140 0x7f562806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.520+0000 7f564899e700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f562806b140 0x7f562806d600 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5634009130 tx=0x7f5634008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.524+0000 7f5639ffb700 1 -- 172.21.15.181:0/3581182443 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5640054750 con 0x7f564406f3b0 2023-10-16T11:20:10.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.717+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f56440c8c70 con 0x7f564406f3b0 2023-10-16T11:20:10.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.718+0000 7f5639ffb700 1 -- 172.21.15.181:0/3581182443 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v225) v1 ==== 103+0+11 (secure 0 0 0) 0x7f56440c8c70 con 0x7f564406f3b0 2023-10-16T11:20:10.720 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f562806b140 msgr2=0x7f562806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f562806b140 0x7f562806d600 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5634009130 tx=0x7f5634008030 comp rx=0 tx=0).stop 2023-10-16T11:20:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564406f3b0 msgr2=0x7f56441530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564406f3b0 0x7f56441530b0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f564000c7f0 tx=0x7f564000cbb0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 shutdown_connections 2023-10-16T11:20:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f562806b140 0x7f562806d600 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b2010 0x7f56440c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564406f3b0 0x7f56441530b0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 --2- 172.21.15.181:0/3581182443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5644071be0 0x7f56441f10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 >> 172.21.15.181:0/3581182443 conn(0x7f56440cd010 msgr2=0x7f56440b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 shutdown_connections 2023-10-16T11:20:10.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.720+0000 7f564b403700 1 -- 172.21.15.181:0/3581182443 wait complete. 2023-10-16T11:20:10.734 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:10.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.853+0000 7fc215f6e700 1 Processor -- start 2023-10-16T11:20:10.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.853+0000 7fc215f6e700 1 -- start start 2023-10-16T11:20:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc215f6e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc21015dfd0 0x7fc2100af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc215f6e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2100cd470 0x7fc2100cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc215f6e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21006f870 con 0x7fc2100af450 2023-10-16T11:20:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc215f6e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2100cde40 con 0x7fc21015dfd0 2023-10-16T11:20:10.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc215f6e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100d1160 con 0x7fc2100cd470 2023-10-16T11:20:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc21015dfd0 0x7fc2100af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc21015dfd0 0x7fc2100af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57270/0 (socket says 172.21.15.181:57270) 2023-10-16T11:20:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 -- 172.21.15.181:0/2807408602 learned_addr learned my addr 172.21.15.181:0/2807408602 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc2100af450 0x7fc21006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40118/0 (socket says 172.21.15.181:40118) 2023-10-16T11:20:10.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 -- 172.21.15.181:0/2807408602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2100cd470 msgr2=0x7fc2100cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20ffff700 1 --2- 172.21.15.181:0/2807408602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2100cd470 0x7fc2100cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 --2- 172.21.15.181:0/2807408602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2100cd470 0x7fc2100cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 -- 172.21.15.181:0/2807408602 >> v1:172.21.15.181:6789/0 conn(0x7fc2100af450 legacy=0x7fc21006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 -- 172.21.15.181:0/2807408602 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100cc8e0 con 0x7fc21015dfd0 2023-10-16T11:20:10.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20e7fc700 1 -- 172.21.15.181:0/2807408602 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522988445 0 0) 0x7fc21006f870 con 0x7fc2100af450 2023-10-16T11:20:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20ffff700 1 --2- 172.21.15.181:0/2807408602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2100cd470 0x7fc2100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20f7fe700 1 --2- 172.21.15.181:0/2807408602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21015dfd0 0x7fc2100af9a0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fc200006fd0 tx=0x7fc200009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595d198278f8897c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20e7fc700 1 -- 172.21.15.181:0/2807408602 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200005e30 con 0x7fc21015dfd0 2023-10-16T11:20:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20e7fc700 1 -- 172.21.15.181:0/2807408602 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200004030 con 0x7fc21015dfd0 2023-10-16T11:20:10.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.854+0000 7fc20e7fc700 1 -- 172.21.15.181:0/2807408602 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000043e0 con 0x7fc21015dfd0 2023-10-16T11:20:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 -- 172.21.15.181:0/2807408602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21015dfd0 msgr2=0x7fc2100af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 --2- 172.21.15.181:0/2807408602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21015dfd0 0x7fc2100af9a0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fc200006fd0 tx=0x7fc200009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 -- 172.21.15.181:0/2807408602 shutdown_connections 2023-10-16T11:20:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 --2- 172.21.15.181:0/2807408602 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2100cd470 0x7fc2100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 --2- 172.21.15.181:0/2807408602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21015dfd0 0x7fc2100af9a0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 -- 172.21.15.181:0/2807408602 >> 172.21.15.181:0/2807408602 conn(0x7fc2100d04a0 msgr2=0x7fc2100d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 -- 172.21.15.181:0/2807408602 shutdown_connections 2023-10-16T11:20:10.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.855+0000 7fc215f6e700 1 -- 172.21.15.181:0/2807408602 wait complete. 2023-10-16T11:20:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.857+0000 7fc215f6e700 1 Processor -- start 2023-10-16T11:20:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.857+0000 7fc215f6e700 1 -- start start 2023-10-16T11:20:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2100af450 0x7fc2101591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2100cd470 0x7fc2100b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc21015dfd0 0x7fc2100b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100aff00 con 0x7fc2100af450 2023-10-16T11:20:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc210159d50 con 0x7fc2100cd470 2023-10-16T11:20:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100b0150 con 0x7fc21015dfd0 2023-10-16T11:20:10.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2100af450 0x7fc2101591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2100af450 0x7fc2101591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57276/0 (socket says 172.21.15.181:57276) 2023-10-16T11:20:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 -- 172.21.15.181:0/4204945969 learned_addr learned my addr 172.21.15.181:0/4204945969 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20ffff700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc21015dfd0 0x7fc2100b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 -- 172.21.15.181:0/4204945969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc21015dfd0 msgr2=0x7fc2100b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc21015dfd0 0x7fc2100b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 -- 172.21.15.181:0/4204945969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2100cd470 msgr2=0x7fc2100b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2100cd470 0x7fc2100b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 -- 172.21.15.181:0/4204945969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200009500 con 0x7fc2100af450 2023-10-16T11:20:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc20f7fe700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2100af450 0x7fc2101591c0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7fc200000c00 tx=0x7fc200004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc1f7fff700 1 -- 172.21.15.181:0/4204945969 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200003e30 con 0x7fc2100af450 2023-10-16T11:20:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc1f7fff700 1 -- 172.21.15.181:0/4204945969 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200011420 con 0x7fc2100af450 2023-10-16T11:20:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2100b0430 con 0x7fc2100af450 2023-10-16T11:20:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc1f7fff700 1 -- 172.21.15.181:0/4204945969 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000117d0 con 0x7fc2100af450 2023-10-16T11:20:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.858+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2100b09f0 con 0x7fc2100af450 2023-10-16T11:20:10.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.859+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2100cd900 con 0x7fc2100af450 2023-10-16T11:20:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.860+0000 7fc1f7fff700 1 -- 172.21.15.181:0/4204945969 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc200011930 con 0x7fc2100af450 2023-10-16T11:20:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.860+0000 7fc1f7fff700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fc1fc06b140 0x7fc1fc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.860+0000 7fc1f7fff700 1 -- 172.21.15.181:0/4204945969 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7fc20008b320 con 0x7fc2100af450 2023-10-16T11:20:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.860+0000 7fc20effd700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fc1fc06b140 0x7fc1fc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:10.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.860+0000 7fc20effd700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fc1fc06b140 0x7fc1fc06d600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc204001610 tx=0x7fc2040079f0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:10.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:10.865+0000 7fc1f7fff700 1 -- 172.21.15.181:0/4204945969 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc200055b40 con 0x7fc2100af450 2023-10-16T11:20:11.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.106+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc21015d830 con 0x7fc2100af450 2023-10-16T11:20:11.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.106+0000 7fc1f7fff700 1 -- 172.21.15.181:0/4204945969 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4939 (secure 0 0 0) 0x7fc200059160 con 0x7fc2100af450 2023-10-16T11:20:11.109 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:11.109 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":119,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9245,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/103948090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":103948090},{"type":"v1","addr":"172.21.15.190:6835","nonce":103948090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:20:11.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fc1fc06b140 msgr2=0x7fc1fc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fc1fc06b140 0x7fc1fc06d600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc204001610 tx=0x7fc2040079f0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2100af450 msgr2=0x7fc2101591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2100af450 0x7fc2101591c0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7fc200000c00 tx=0x7fc200004750 comp rx=0 tx=0).stop 2023-10-16T11:20:11.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 shutdown_connections 2023-10-16T11:20:11.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fc1fc06b140 0x7fc1fc06d600 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc21015dfd0 0x7fc2100b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2100af450 0x7fc2101591c0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 --2- 172.21.15.181:0/4204945969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2100cd470 0x7fc2100b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 >> 172.21.15.181:0/4204945969 conn(0x7fc2100d04a0 msgr2=0x7fc2100cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:11.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 shutdown_connections 2023-10-16T11:20:11.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.109+0000 7fc215f6e700 1 -- 172.21.15.181:0/4204945969 wait complete. 2023-10-16T11:20:11.113 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 119 2023-10-16T11:20:11.123 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:11.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.242+0000 7fd2fd4b3700 1 Processor -- start 2023-10-16T11:20:11.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.242+0000 7fd2fd4b3700 1 -- start start 2023-10-16T11:20:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2fd4b3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 0x7fd2f81611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2fd4b3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f806ce20 con 0x7fd2f8161b60 2023-10-16T11:20:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2fd4b3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f80d45f0 con 0x7fd2f806d5f0 2023-10-16T11:20:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2fd4b3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8162800 con 0x7fd2f8160d20 2023-10-16T11:20:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2f6ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd2f8161b60 0x7fd2f8161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40130/0 (socket says 172.21.15.181:40130) 2023-10-16T11:20:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2f6ffd700 1 -- 172.21.15.181:0/2784224480 learned_addr learned my addr 172.21.15.181:0/2784224480 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311940826 0 0) 0x7fd2f806ce20 con 0x7fd2f8161b60 2023-10-16T11:20:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2f77fe700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 0x7fd2f81611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.243+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2e00035b0 con 0x7fd2f8161b60 2023-10-16T11:20:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3008957677 0 0) 0x7fd2e00035b0 con 0x7fd2f8161b60 2023-10-16T11:20:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 msgr2=0x7fd2f81611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 0x7fd2f81611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 >> v1:172.21.15.190:6789/0 conn(0x7fd2f806d5f0 legacy=0x7fd2f80d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f806d2d0 con 0x7fd2f8161b60 2023-10-16T11:20:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fd2ec0017f0 con 0x7fd2f8161b60 2023-10-16T11:20:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 >> v1:172.21.15.181:6789/0 conn(0x7fd2f8161b60 legacy=0x7fd2f8161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0003a30 0x7fd2e0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 0x7fd2e0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8162800 con 0x7fd2e0004850 2023-10-16T11:20:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e000bb70 con 0x7fd2e0003a30 2023-10-16T11:20:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e000bd30 con 0x7fd2f8160d20 2023-10-16T11:20:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f67fc700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f6ffd700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0003a30 0x7fd2e0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f77fe700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 0x7fd2e0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f67fc700 1 -- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 msgr2=0x7fd2e0005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f67fc700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f8160d20 0x7fd2e0005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f67fc700 1 -- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0003a30 msgr2=0x7fd2e0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f67fc700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2e0003a30 0x7fd2e0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f67fc700 1 -- 172.21.15.181:0/2784224480 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00035b0 con 0x7fd2e0004850 2023-10-16T11:20:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f67fc700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2e0004cd0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001800 tx=0x7fd2e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c6fb36ea14da8f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80012f0 con 0x7fd2e0004850 2023-10-16T11:20:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e80052f0 con 0x7fd2e0004850 2023-10-16T11:20:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.244+0000 7fd2f5ffb700 1 -- 172.21.15.181:0/2784224480 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8006860 con 0x7fd2e0004850 2023-10-16T11:20:11.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.245+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/2784224480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 msgr2=0x7fd2e0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.245+0000 7fd2fd4b3700 1 --2- 172.21.15.181:0/2784224480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2e0004cd0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001800 tx=0x7fd2e0005880 comp rx=0 tx=0).stop 2023-10-16T11:20:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.245+0000 7fd2f77fe700 1 -- 172.21.15.181:0/2784224480 reap_dead start 2023-10-16T11:20:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.245+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/2784224480 shutdown_connections 2023-10-16T11:20:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.245+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/2784224480 >> 172.21.15.181:0/2784224480 conn(0x7fd2f809cba0 msgr2=0x7fd2f80d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:11.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.245+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/2784224480 shutdown_connections 2023-10-16T11:20:11.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.245+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/2784224480 wait complete. 2023-10-16T11:20:11.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.246+0000 7fd2fd4b3700 1 Processor -- start 2023-10-16T11:20:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.246+0000 7fd2fd4b3700 1 -- start start 2023-10-16T11:20:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2fd4b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2f80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2fd4b3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2f80b3710 0x7fd2f80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2fd4b3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f80b5e70 0x7fd2f8147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2fd4b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80ca300 con 0x7fd2e0004850 2023-10-16T11:20:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2fd4b3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81f6970 con 0x7fd2f80b3710 2023-10-16T11:20:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2fd4b3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80b7fe0 con 0x7fd2f80b5e70 2023-10-16T11:20:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2f6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2f80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2f6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2f80c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57296/0 (socket says 172.21.15.181:57296) 2023-10-16T11:20:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2f6ffd700 1 -- 172.21.15.181:0/1541132243 learned_addr learned my addr 172.21.15.181:0/1541132243 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2f77fe700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f80b5e70 0x7fd2f8147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.247+0000 7fd2f67fc700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2f80b3710 0x7fd2f80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2f6ffd700 1 -- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f80b5e70 msgr2=0x7fd2f8147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2f6ffd700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f80b5e70 0x7fd2f8147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2f6ffd700 1 -- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2f80b3710 msgr2=0x7fd2f80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2f6ffd700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2f80b3710 0x7fd2f80b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2f6ffd700 1 -- 172.21.15.181:0/1541132243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f80b73f0 con 0x7fd2e0004850 2023-10-16T11:20:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2f6ffd700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2f80c9dc0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00fae0 tx=0x7fd2ec00fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2df7fe700 1 -- 172.21.15.181:0/1541132243 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec001760 con 0x7fd2e0004850 2023-10-16T11:20:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2df7fe700 1 -- 172.21.15.181:0/1541132243 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec0072e0 con 0x7fd2e0004850 2023-10-16T11:20:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f80b76b0 con 0x7fd2e0004850 2023-10-16T11:20:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2df7fe700 1 -- 172.21.15.181:0/1541132243 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec007650 con 0x7fd2e0004850 2023-10-16T11:20:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.248+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f80b7c70 con 0x7fd2e0004850 2023-10-16T11:20:11.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.249+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f804ebb0 con 0x7fd2e0004850 2023-10-16T11:20:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.249+0000 7fd2df7fe700 1 -- 172.21.15.181:0/1541132243 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd2ec005830 con 0x7fd2e0004850 2023-10-16T11:20:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.250+0000 7fd2df7fe700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd2e406b170 0x7fd2e406d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.250+0000 7fd2df7fe700 1 -- 172.21.15.181:0/1541132243 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7fd2ec089eb0 con 0x7fd2e0004850 2023-10-16T11:20:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.250+0000 7fd2f67fc700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd2e406b170 0x7fd2e406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.250+0000 7fd2f67fc700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd2e406b170 0x7fd2e406d630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8014040 tx=0x7fd2e8009030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.254+0000 7fd2df7fe700 1 -- 172.21.15.181:0/1541132243 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd2ec054640 con 0x7fd2e0004850 2023-10-16T11:20:11.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.448+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2f80b7e00 con 0x7fd2e0004850 2023-10-16T11:20:11.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.450+0000 7fd2df7fe700 1 -- 172.21.15.181:0/1541132243 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+18812 (secure 0 0 0) 0x7fd2ec057c60 con 0x7fd2e0004850 2023-10-16T11:20:11.455 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:11.455 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":225,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:09.327947+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/2033295595":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/4244377789":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2576397499":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6833/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.190:6833/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/754905711":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3499373399":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2963042451":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3469785622":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2131678440":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/134902336":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:11.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd2e406b170 msgr2=0x7fd2e406d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd2e406b170 0x7fd2e406d630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8014040 tx=0x7fd2e8009030 comp rx=0 tx=0).stop 2023-10-16T11:20:11.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 msgr2=0x7fd2f80c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2f80c9dc0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00fae0 tx=0x7fd2ec00fdf0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 shutdown_connections 2023-10-16T11:20:11.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd2e406b170 0x7fd2e406d630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2f80b5e70 0x7fd2f8147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2e0004850 0x7fd2f80c9dc0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 --2- 172.21.15.181:0/1541132243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2f80b3710 0x7fd2f80b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 >> 172.21.15.181:0/1541132243 conn(0x7fd2f809cba0 msgr2=0x7fd2f806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:11.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 shutdown_connections 2023-10-16T11:20:11.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.452+0000 7fd2fd4b3700 1 -- 172.21.15.181:0/1541132243 wait complete. 2023-10-16T11:20:11.466 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-16T11:20:11.467 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:11.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.586+0000 7fb9372de700 1 Processor -- start 2023-10-16T11:20:11.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.586+0000 7fb9372de700 1 -- start start 2023-10-16T11:20:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb9372de700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb93015dfd0 0x7fb9300af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb9372de700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb9300af450 0x7fb93006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb9372de700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93006f870 con 0x7fb93006db80 2023-10-16T11:20:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb9372de700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9300d1160 con 0x7fb93015dfd0 2023-10-16T11:20:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb9372de700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9300cc8e0 con 0x7fb9300af450 2023-10-16T11:20:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb93507a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb93015dfd0 0x7fb9300af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb93507a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb93015dfd0 0x7fb9300af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57304/0 (socket says 172.21.15.181:57304) 2023-10-16T11:20:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb93587b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb93006db80 0x7fb9300cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40152/0 (socket says 172.21.15.181:40152) 2023-10-16T11:20:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.587+0000 7fb93507a700 1 -- 172.21.15.181:0/1939353987 learned_addr learned my addr 172.21.15.181:0/1939353987 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb93507a700 1 -- 172.21.15.181:0/1939353987 >> v2:172.21.15.190:3300/0 conn(0x7fb9300af450 msgr2=0x7fb93006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb934879700 1 --2- 172.21.15.181:0/1939353987 >> v2:172.21.15.190:3300/0 conn(0x7fb9300af450 0x7fb93006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb927fff700 1 -- 172.21.15.181:0/1939353987 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228695239 0 0) 0x7fb93006f870 con 0x7fb93006db80 2023-10-16T11:20:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb93507a700 1 --2- 172.21.15.181:0/1939353987 >> v2:172.21.15.190:3300/0 conn(0x7fb9300af450 0x7fb93006d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb93507a700 1 -- 172.21.15.181:0/1939353987 >> v1:172.21.15.181:6789/0 conn(0x7fb93006db80 legacy=0x7fb9300cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb93507a700 1 -- 172.21.15.181:0/1939353987 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300ccbc0 con 0x7fb93015dfd0 2023-10-16T11:20:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb93507a700 1 --2- 172.21.15.181:0/1939353987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93015dfd0 0x7fb9300af9a0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fb9300b3070 tx=0x7fb920009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c9b1f70a6028af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb927fff700 1 -- 172.21.15.181:0/1939353987 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920005e30 con 0x7fb93015dfd0 2023-10-16T11:20:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb927fff700 1 -- 172.21.15.181:0/1939353987 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920004030 con 0x7fb93015dfd0 2023-10-16T11:20:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb927fff700 1 -- 172.21.15.181:0/1939353987 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9200043e0 con 0x7fb93015dfd0 2023-10-16T11:20:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb9372de700 1 -- 172.21.15.181:0/1939353987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93015dfd0 msgr2=0x7fb9300af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb9372de700 1 --2- 172.21.15.181:0/1939353987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93015dfd0 0x7fb9300af9a0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fb9300b3070 tx=0x7fb920009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb9372de700 1 -- 172.21.15.181:0/1939353987 shutdown_connections 2023-10-16T11:20:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb9372de700 1 --2- 172.21.15.181:0/1939353987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93015dfd0 0x7fb9300af9a0 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb9372de700 1 --2- 172.21.15.181:0/1939353987 >> v2:172.21.15.190:3300/0 conn(0x7fb9300af450 0x7fb93006d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.588+0000 7fb9372de700 1 -- 172.21.15.181:0/1939353987 >> 172.21.15.181:0/1939353987 conn(0x7fb9300d04a0 msgr2=0x7fb9300d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:11.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.589+0000 7fb9372de700 1 -- 172.21.15.181:0/1939353987 shutdown_connections 2023-10-16T11:20:11.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.589+0000 7fb9372de700 1 -- 172.21.15.181:0/1939353987 wait complete. 2023-10-16T11:20:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 Processor -- start 2023-10-16T11:20:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 -- start start 2023-10-16T11:20:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93006db80 0x7fb930158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9300af450 0x7fb9300c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb93015dfd0 0x7fb9300ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9300c6050 con 0x7fb93006db80 2023-10-16T11:20:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301f2e40 con 0x7fb93015dfd0 2023-10-16T11:20:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.590+0000 7fb9372de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9300b2d00 con 0x7fb9300af450 2023-10-16T11:20:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93006db80 0x7fb930158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93006db80 0x7fb930158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57310/0 (socket says 172.21.15.181:57310) 2023-10-16T11:20:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 -- 172.21.15.181:0/1425339332 learned_addr learned my addr 172.21.15.181:0/1425339332 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 -- 172.21.15.181:0/1425339332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9300af450 msgr2=0x7fb9300c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9300af450 0x7fb9300c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 -- 172.21.15.181:0/1425339332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb93015dfd0 msgr2=0x7fb9300ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb93015dfd0 0x7fb9300ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 -- 172.21.15.181:0/1425339332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb920009500 con 0x7fb93006db80 2023-10-16T11:20:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb93507a700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93006db80 0x7fb930158b50 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fb920009130 tx=0x7fb920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb925ffb700 1 -- 172.21.15.181:0/1425339332 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92001d630 con 0x7fb93006db80 2023-10-16T11:20:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb925ffb700 1 -- 172.21.15.181:0/1425339332 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92001d7b0 con 0x7fb93006db80 2023-10-16T11:20:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9300c4510 con 0x7fb93006db80 2023-10-16T11:20:11.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb925ffb700 1 -- 172.21.15.181:0/1425339332 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92001db60 con 0x7fb93006db80 2023-10-16T11:20:11.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.591+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9301f3650 con 0x7fb93006db80 2023-10-16T11:20:11.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.592+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb930067d20 con 0x7fb93006db80 2023-10-16T11:20:11.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.593+0000 7fb925ffb700 1 -- 172.21.15.181:0/1425339332 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb92001dcc0 con 0x7fb93006db80 2023-10-16T11:20:11.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.593+0000 7fb925ffb700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb91406b140 0x7fb91406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.593+0000 7fb925ffb700 1 -- 172.21.15.181:0/1425339332 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7fb92008b460 con 0x7fb93006db80 2023-10-16T11:20:11.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.593+0000 7fb934879700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb91406b140 0x7fb91406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.594+0000 7fb934879700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb91406b140 0x7fb91406d600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb9301f74e0 tx=0x7fb92c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.598+0000 7fb925ffb700 1 -- 172.21.15.181:0/1425339332 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb920055c80 con 0x7fb93006db80 2023-10-16T11:20:11.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.833+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9300c4510 con 0x7fb93006db80 2023-10-16T11:20:11.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.834+0000 7fb925ffb700 1 -- 172.21.15.181:0/1425339332 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4939 (secure 0 0 0) 0x7fb9300c4510 con 0x7fb93006db80 2023-10-16T11:20:11.836 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:11.837 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":119,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9245,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/103948090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":103948090},{"type":"v1","addr":"172.21.15.190:6835","nonce":103948090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:20:11.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb91406b140 msgr2=0x7fb91406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb91406b140 0x7fb91406d600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb9301f74e0 tx=0x7fb92c008040 comp rx=0 tx=0).stop 2023-10-16T11:20:11.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93006db80 msgr2=0x7fb930158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93006db80 0x7fb930158b50 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fb920009130 tx=0x7fb920004750 comp rx=0 tx=0).stop 2023-10-16T11:20:11.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 shutdown_connections 2023-10-16T11:20:11.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb91406b140 0x7fb91406d600 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb93015dfd0 0x7fb9300ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb93006db80 0x7fb930158b50 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 --2- 172.21.15.181:0/1425339332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9300af450 0x7fb9300c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 >> 172.21.15.181:0/1425339332 conn(0x7fb9300d04a0 msgr2=0x7fb9300cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:11.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 shutdown_connections 2023-10-16T11:20:11.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.836+0000 7fb9372de700 1 -- 172.21.15.181:0/1425339332 wait complete. 2023-10-16T11:20:11.841 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 119 2023-10-16T11:20:11.850 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:11.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.969+0000 7f17196f5700 1 Processor -- start 2023-10-16T11:20:11.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.969+0000 7f17196f5700 1 -- start start 2023-10-16T11:20:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17196f5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17140ce540 0x7f17140b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17196f5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f171406bdb0 0x7f17140cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17196f5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1714161fc0 con 0x7f171415de50 2023-10-16T11:20:11.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17196f5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17140b4010 con 0x7f171406bdb0 2023-10-16T11:20:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17196f5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17141622a0 con 0x7f17140ce540 2023-10-16T11:20:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17137fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f171406bdb0 0x7f17140cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17137fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f171406bdb0 0x7f17140cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57322/0 (socket says 172.21.15.181:57322) 2023-10-16T11:20:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f1712ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f171415de50 0x7f17140cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40164/0 (socket says 172.21.15.181:40164) 2023-10-16T11:20:11.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17137fe700 1 -- 172.21.15.181:0/2018240971 learned_addr learned my addr 172.21.15.181:0/2018240971 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17127fc700 1 --2- 172.21.15.181:0/2018240971 >> v2:172.21.15.190:3300/0 conn(0x7f17140ce540 0x7f17140b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17137fe700 1 -- 172.21.15.181:0/2018240971 >> v2:172.21.15.190:3300/0 conn(0x7f17140ce540 msgr2=0x7f17140b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17137fe700 1 --2- 172.21.15.181:0/2018240971 >> v2:172.21.15.190:3300/0 conn(0x7f17140ce540 0x7f17140b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.970+0000 7f17137fe700 1 -- 172.21.15.181:0/2018240971 >> v1:172.21.15.181:6789/0 conn(0x7f171415de50 legacy=0x7f17140cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17137fe700 1 -- 172.21.15.181:0/2018240971 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714162580 con 0x7f171406bdb0 2023-10-16T11:20:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17137fe700 1 --2- 172.21.15.181:0/2018240971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f17140cf4f0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f17140b1040 tx=0x7f170c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d88968aae856799 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f1711ffb700 1 -- 172.21.15.181:0/2018240971 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00e070 con 0x7f171406bdb0 2023-10-16T11:20:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f1711ffb700 1 -- 172.21.15.181:0/2018240971 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00b9c0 con 0x7f171406bdb0 2023-10-16T11:20:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f1711ffb700 1 -- 172.21.15.181:0/2018240971 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00bd70 con 0x7f171406bdb0 2023-10-16T11:20:11.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 -- 172.21.15.181:0/2018240971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 msgr2=0x7f17140cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 --2- 172.21.15.181:0/2018240971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f17140cf4f0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f17140b1040 tx=0x7f170c00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:11.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 -- 172.21.15.181:0/2018240971 shutdown_connections 2023-10-16T11:20:11.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 --2- 172.21.15.181:0/2018240971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f17140cf4f0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 --2- 172.21.15.181:0/2018240971 >> v2:172.21.15.190:3300/0 conn(0x7f17140ce540 0x7f17140b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 -- 172.21.15.181:0/2018240971 >> 172.21.15.181:0/2018240971 conn(0x7f17140665a0 msgr2=0x7f1714066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 -- 172.21.15.181:0/2018240971 shutdown_connections 2023-10-16T11:20:11.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.971+0000 7f17196f5700 1 -- 172.21.15.181:0/2018240971 wait complete. 2023-10-16T11:20:11.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 Processor -- start 2023-10-16T11:20:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 -- start start 2023-10-16T11:20:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f171414afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17140ce540 0x7f171415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171415de50 0x7f17141569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141f7bd0 con 0x7f171406bdb0 2023-10-16T11:20:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17141f6d00 con 0x7f17140ce540 2023-10-16T11:20:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.973+0000 7f17196f5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171414b500 con 0x7f171415de50 2023-10-16T11:20:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f171414afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f171414afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57324/0 (socket says 172.21.15.181:57324) 2023-10-16T11:20:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 -- 172.21.15.181:0/3443744493 learned_addr learned my addr 172.21.15.181:0/3443744493 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f17137fe700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171415de50 0x7f17141569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f17127fc700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17140ce540 0x7f171415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 -- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171415de50 msgr2=0x7f17141569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171415de50 0x7f17141569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 -- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17140ce540 msgr2=0x7f171415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17140ce540 0x7f171415ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 -- 172.21.15.181:0/3443744493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c009b30 con 0x7f171406bdb0 2023-10-16T11:20:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f1712ffd700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f171414afc0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f170800c530 tx=0x7f170800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f16fb7fe700 1 -- 172.21.15.181:0/3443744493 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708005cb0 con 0x7f171406bdb0 2023-10-16T11:20:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f16fb7fe700 1 -- 172.21.15.181:0/3443744493 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708005e10 con 0x7f171406bdb0 2023-10-16T11:20:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17141f6f50 con 0x7f171406bdb0 2023-10-16T11:20:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f16fb7fe700 1 -- 172.21.15.181:0/3443744493 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080072d0 con 0x7f171406bdb0 2023-10-16T11:20:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.974+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17141f7510 con 0x7f171406bdb0 2023-10-16T11:20:11.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.975+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171406bb90 con 0x7f171406bdb0 2023-10-16T11:20:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.976+0000 7f16fb7fe700 1 -- 172.21.15.181:0/3443744493 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1708007430 con 0x7f171406bdb0 2023-10-16T11:20:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.976+0000 7f16fb7fe700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f170006b140 0x7f170006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.976+0000 7f16fb7fe700 1 -- 172.21.15.181:0/3443744493 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7f17080892d0 con 0x7f171406bdb0 2023-10-16T11:20:11.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.976+0000 7f17127fc700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f170006b140 0x7f170006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:11.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.976+0000 7f17127fc700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f170006b140 0x7f170006d600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f171415eb90 tx=0x7f1704008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:11.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:11.980+0000 7f16fb7fe700 1 -- 172.21.15.181:0/3443744493 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1708053b20 con 0x7f171406bdb0 2023-10-16T11:20:12.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.214+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1714158af0 con 0x7f171406bdb0 2023-10-16T11:20:12.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.214+0000 7f16fb7fe700 1 -- 172.21.15.181:0/3443744493 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4939 (secure 0 0 0) 0x7f1708057140 con 0x7f171406bdb0 2023-10-16T11:20:12.217 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:12.217 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":119,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9245,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/103948090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":103948090},{"type":"v1","addr":"172.21.15.190:6835","nonce":103948090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:19:36.016729+0000","modified":"2023-10-16T11:19:39.419198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"c","rank":0,"incarnation":113,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/4167832945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":4167832945},{"type":"v1","addr":"172.21.15.181:6837","nonce":4167832945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-16T11:20:12.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f170006b140 msgr2=0x7f170006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f170006b140 0x7f170006d600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f171415eb90 tx=0x7f1704008040 comp rx=0 tx=0).stop 2023-10-16T11:20:12.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 msgr2=0x7f171414afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f171414afc0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f170800c530 tx=0x7f170800c8f0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 shutdown_connections 2023-10-16T11:20:12.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f170006b140 0x7f170006d600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f171415de50 0x7f17141569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171406bdb0 0x7f171414afc0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 --2- 172.21.15.181:0/3443744493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17140ce540 0x7f171415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 >> 172.21.15.181:0/3443744493 conn(0x7f17140665a0 msgr2=0x7f171406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:12.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 shutdown_connections 2023-10-16T11:20:12.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.216+0000 7f17196f5700 1 -- 172.21.15.181:0/3443744493 wait complete. 2023-10-16T11:20:12.221 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 119 2023-10-16T11:20:12.231 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:12.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.348+0000 7fbc34ea9700 1 Processor -- start 2023-10-16T11:20:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.348+0000 7fbc34ea9700 1 -- start start 2023-10-16T11:20:12.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc34ea9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161b60 0x7fbc3006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc34ea9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbc30160d20 0x7fbc301611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc34ea9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3006ce20 con 0x7fbc30161f40 2023-10-16T11:20:12.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc34ea9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc30162800 con 0x7fbc30160d20 2023-10-16T11:20:12.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc34ea9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300d43e0 con 0x7fbc30161b60 2023-10-16T11:20:12.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc2e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161b60 0x7fbc3006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc2ed9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbc30160d20 0x7fbc301611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc2e59c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161b60 0x7fbc3006d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:54358/0 (socket says 172.21.15.181:54358) 2023-10-16T11:20:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc2dd9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbc30161f40 0x7fbc300d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:52354/0 (socket says 172.21.15.181:52354) 2023-10-16T11:20:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.349+0000 7fbc2e59c700 1 -- 172.21.15.181:0/3578456727 learned_addr learned my addr 172.21.15.181:0/3578456727 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:12.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.350+0000 7fbc2e59c700 1 -- 172.21.15.181:0/3578456727 >> v2:172.21.15.190:3300/0 conn(0x7fbc30160d20 msgr2=0x7fbc301611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.350+0000 7fbc2e59c700 1 --2- 172.21.15.181:0/3578456727 >> v2:172.21.15.190:3300/0 conn(0x7fbc30160d20 0x7fbc301611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.350+0000 7fbc2e59c700 1 -- 172.21.15.181:0/3578456727 >> v1:172.21.15.190:6789/0 conn(0x7fbc30161f40 legacy=0x7fbc300d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.350+0000 7fbc2e59c700 1 -- 172.21.15.181:0/3578456727 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300d45f0 con 0x7fbc30161b60 2023-10-16T11:20:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.350+0000 7fbc2ed9d700 1 --2- 172.21.15.181:0/3578456727 >> v2:172.21.15.190:3300/0 conn(0x7fbc30160d20 0x7fbc301611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:12.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.350+0000 7fbc2e59c700 1 --2- 172.21.15.181:0/3578456727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161b60 0x7fbc3006d5f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6152f1ef93c929ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:12.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.353+0000 7fbc2d59a700 1 -- 172.21.15.181:0/3578456727 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20005e30 con 0x7fbc30161b60 2023-10-16T11:20:12.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.353+0000 7fbc2d59a700 1 -- 172.21.15.181:0/3578456727 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc20004030 con 0x7fbc30161b60 2023-10-16T11:20:12.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.353+0000 7fbc2d59a700 1 -- 172.21.15.181:0/3578456727 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200043e0 con 0x7fbc30161b60 2023-10-16T11:20:12.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 -- 172.21.15.181:0/3578456727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161b60 msgr2=0x7fbc3006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/3578456727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161b60 0x7fbc3006d5f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 -- 172.21.15.181:0/3578456727 shutdown_connections 2023-10-16T11:20:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/3578456727 >> v2:172.21.15.190:3300/0 conn(0x7fbc30160d20 0x7fbc301611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/3578456727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161b60 0x7fbc3006d5f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 -- 172.21.15.181:0/3578456727 >> 172.21.15.181:0/3578456727 conn(0x7fbc3009cba0 msgr2=0x7fbc300d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:12.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 -- 172.21.15.181:0/3578456727 shutdown_connections 2023-10-16T11:20:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.354+0000 7fbc34ea9700 1 -- 172.21.15.181:0/3578456727 wait complete. 2023-10-16T11:20:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 Processor -- start 2023-10-16T11:20:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 -- start start 2023-10-16T11:20:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc30160d20 0x7fbc3015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc30161b60 0x7fbc300cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161f40 0x7fbc300b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300cbe40 con 0x7fbc30161b60 2023-10-16T11:20:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300b8c00 con 0x7fbc30160d20 2023-10-16T11:20:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc34ea9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3015ac50 con 0x7fbc30161f40 2023-10-16T11:20:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.356+0000 7fbc2dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc30161b60 0x7fbc300cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc30161b60 0x7fbc300cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57338/0 (socket says 172.21.15.181:57338) 2023-10-16T11:20:12.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 -- 172.21.15.181:0/1405760876 learned_addr learned my addr 172.21.15.181:0/1405760876 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2ed9d700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161f40 0x7fbc300b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2e59c700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc30160d20 0x7fbc3015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 -- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161f40 msgr2=0x7fbc300b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161f40 0x7fbc300b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 -- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc30160d20 msgr2=0x7fbc3015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc30160d20 0x7fbc3015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 -- 172.21.15.181:0/1405760876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc28009a70 con 0x7fbc30161b60 2023-10-16T11:20:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2ed9d700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161f40 0x7fbc300b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2e59c700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc30160d20 0x7fbc3015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc2dd9b700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc30161b60 0x7fbc300cc260 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c520 tx=0x7fbc2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:12.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc1effd700 1 -- 172.21.15.181:0/1405760876 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24005cb0 con 0x7fbc30161b60 2023-10-16T11:20:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc1effd700 1 -- 172.21.15.181:0/1405760876 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc24005e10 con 0x7fbc30161b60 2023-10-16T11:20:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc20009500 con 0x7fbc30161b60 2023-10-16T11:20:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc1effd700 1 -- 172.21.15.181:0/1405760876 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240072d0 con 0x7fbc30161b60 2023-10-16T11:20:12.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.357+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc300b8920 con 0x7fbc30161b60 2023-10-16T11:20:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.358+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3006d290 con 0x7fbc30161b60 2023-10-16T11:20:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.359+0000 7fbc1effd700 1 -- 172.21.15.181:0/1405760876 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbc24007430 con 0x7fbc30161b60 2023-10-16T11:20:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.359+0000 7fbc1effd700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbc1006b140 0x7fbc1006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.359+0000 7fbc1effd700 1 -- 172.21.15.181:0/1405760876 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7fbc240893d0 con 0x7fbc30161b60 2023-10-16T11:20:12.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.359+0000 7fbc2e59c700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbc1006b140 0x7fbc1006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.359+0000 7fbc2e59c700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbc1006b140 0x7fbc1006d600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20008070 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:12.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.364+0000 7fbc1effd700 1 -- 172.21.15.181:0/1405760876 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc24053c10 con 0x7fbc30161b60 2023-10-16T11:20:12.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.555+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc3004ebb0 con 0x7fbc30161b60 2023-10-16T11:20:12.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.556+0000 7fbc1effd700 1 -- 172.21.15.181:0/1405760876 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+18812 (secure 0 0 0) 0x7fbc24057230 con 0x7fbc30161b60 2023-10-16T11:20:12.558 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:12.559 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":225,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:09.327947+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:19:33.658268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:19:34.982934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/2033295595":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/4244377789":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2576397499":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6833/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.190:6833/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/754905711":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3499373399":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2963042451":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3469785622":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2131678440":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/134902336":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:12.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbc1006b140 msgr2=0x7fbc1006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbc1006b140 0x7fbc1006d600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20008070 comp rx=0 tx=0).stop 2023-10-16T11:20:12.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc30161b60 msgr2=0x7fbc300cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc30161b60 0x7fbc300cc260 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c520 tx=0x7fbc2400c8e0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 shutdown_connections 2023-10-16T11:20:12.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbc1006b140 0x7fbc1006d600 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc30161f40 0x7fbc300b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc30160d20 0x7fbc3015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 --2- 172.21.15.181:0/1405760876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc30161b60 0x7fbc300cc260 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 >> 172.21.15.181:0/1405760876 conn(0x7fbc3009cba0 msgr2=0x7fbc3015df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:12.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 shutdown_connections 2023-10-16T11:20:12.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.558+0000 7fbc34ea9700 1 -- 172.21.15.181:0/1405760876 wait complete. 2023-10-16T11:20:12.573 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-16T11:20:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.693+0000 7ffab91ee700 1 Processor -- start 2023-10-16T11:20:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.693+0000 7ffab91ee700 1 -- start start 2023-10-16T11:20:12.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.693+0000 7ffab91ee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffab406afb0 0x7ffab40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.693+0000 7ffab91ee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab40cddf0 con 0x7ffab40d0fc0 2023-10-16T11:20:12.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.693+0000 7ffab91ee700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab40ccf50 con 0x7ffab415df60 2023-10-16T11:20:12.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.693+0000 7ffab91ee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab40af9c0 con 0x7ffab406afb0 2023-10-16T11:20:12.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.693+0000 7ffab359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffab406afb0 0x7ffab40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffab406afb0 0x7ffab40cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57350/0 (socket says 172.21.15.181:57350) 2023-10-16T11:20:12.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ffab40d0fc0 0x7ffab406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40192/0 (socket says 172.21.15.181:40192) 2023-10-16T11:20:12.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab359e700 1 -- 172.21.15.181:0/1553066509 learned_addr learned my addr 172.21.15.181:0/1553066509 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab359e700 1 -- 172.21.15.181:0/1553066509 >> v1:172.21.15.190:6789/0 conn(0x7ffab415df60 legacy=0x7ffab40cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab359e700 1 -- 172.21.15.181:0/1553066509 >> v1:172.21.15.181:6789/0 conn(0x7ffab40d0fc0 legacy=0x7ffab406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab359e700 1 -- 172.21.15.181:0/1553066509 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40cc600 con 0x7ffab406afb0 2023-10-16T11:20:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab1d9b700 1 -- 172.21.15.181:0/1553066509 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086937108 0 0) 0x7ffab40cddf0 con 0x7ffab40d0fc0 2023-10-16T11:20:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab359e700 1 --2- 172.21.15.181:0/1553066509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab406afb0 0x7ffab40cae20 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009e50 tx=0x7ffaac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9544cc41da6b35bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab1d9b700 1 -- 172.21.15.181:0/1553066509 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00e070 con 0x7ffab406afb0 2023-10-16T11:20:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab1d9b700 1 -- 172.21.15.181:0/1553066509 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac00b9c0 con 0x7ffab406afb0 2023-10-16T11:20:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.694+0000 7ffab1d9b700 1 -- 172.21.15.181:0/1553066509 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00bd70 con 0x7ffab406afb0 2023-10-16T11:20:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.695+0000 7ffab91ee700 1 -- 172.21.15.181:0/1553066509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab406afb0 msgr2=0x7ffab40cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.695+0000 7ffab91ee700 1 --2- 172.21.15.181:0/1553066509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab406afb0 0x7ffab40cae20 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009e50 tx=0x7ffaac00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.695+0000 7ffab91ee700 1 -- 172.21.15.181:0/1553066509 shutdown_connections 2023-10-16T11:20:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.695+0000 7ffab91ee700 1 --2- 172.21.15.181:0/1553066509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab406afb0 0x7ffab40cae20 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.695+0000 7ffab91ee700 1 -- 172.21.15.181:0/1553066509 >> 172.21.15.181:0/1553066509 conn(0x7ffab40d04a0 msgr2=0x7ffab40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.695+0000 7ffab91ee700 1 -- 172.21.15.181:0/1553066509 shutdown_connections 2023-10-16T11:20:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.695+0000 7ffab91ee700 1 -- 172.21.15.181:0/1553066509 wait complete. 2023-10-16T11:20:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.696+0000 7ffab91ee700 1 Processor -- start 2023-10-16T11:20:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.697+0000 7ffab91ee700 1 -- start start 2023-10-16T11:20:12.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.697+0000 7ffab91ee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab406afb0 0x7ffab41479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.697+0000 7ffab91ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40d0fc0 0x7ffab40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.697+0000 7ffab91ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab415df60 0x7ffab40b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.697+0000 7ffab91ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40b09f0 con 0x7ffab415df60 2023-10-16T11:20:12.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.697+0000 7ffab91ee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40ca830 con 0x7ffab406afb0 2023-10-16T11:20:12.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.697+0000 7ffab91ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40ca270 con 0x7ffab40d0fc0 2023-10-16T11:20:12.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab415df60 0x7ffab40b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab415df60 0x7ffab40b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57356/0 (socket says 172.21.15.181:57356) 2023-10-16T11:20:12.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 -- 172.21.15.181:0/3032353914 learned_addr learned my addr 172.21.15.181:0/3032353914 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:12.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab2d9d700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab406afb0 0x7ffab41479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab259c700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40d0fc0 0x7ffab40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 -- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40d0fc0 msgr2=0x7ffab40c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40d0fc0 0x7ffab40c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 -- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab406afb0 msgr2=0x7ffab41479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab406afb0 0x7ffab41479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 -- 172.21.15.181:0/3032353914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac009b30 con 0x7ffab415df60 2023-10-16T11:20:12.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab359e700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab415df60 0x7ffab40b4340 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7ffab40716a0 tx=0x7ffaac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:12.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffa9b7fe700 1 -- 172.21.15.181:0/3032353914 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01b630 con 0x7ffab415df60 2023-10-16T11:20:12.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffa9b7fe700 1 -- 172.21.15.181:0/3032353914 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac01b7b0 con 0x7ffab415df60 2023-10-16T11:20:12.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffa9b7fe700 1 -- 172.21.15.181:0/3032353914 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01bb60 con 0x7ffab415df60 2023-10-16T11:20:12.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab40ca550 con 0x7ffab415df60 2023-10-16T11:20:12.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.698+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab4147f10 con 0x7ffab415df60 2023-10-16T11:20:12.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.699+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab40b0b30 con 0x7ffab415df60 2023-10-16T11:20:12.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.700+0000 7ffa9b7fe700 1 -- 172.21.15.181:0/3032353914 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ffaac01bcc0 con 0x7ffab415df60 2023-10-16T11:20:12.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.700+0000 7ffa9b7fe700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffaa006b140 0x7ffaa006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:12.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.700+0000 7ffa9b7fe700 1 -- 172.21.15.181:0/3032353914 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7176+0+0 (secure 0 0 0) 0x7ffaac08bb60 con 0x7ffab415df60 2023-10-16T11:20:12.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.700+0000 7ffab2d9d700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffaa006b140 0x7ffaa006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:12.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.701+0000 7ffab2d9d700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffaa006b140 0x7ffaa006d600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4000fb0 tx=0x7ffaa4009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:12.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.705+0000 7ffa9b7fe700 1 -- 172.21.15.181:0/3032353914 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffab40b0b30 con 0x7ffab415df60 2023-10-16T11:20:12.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.946+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7ffab40b0b30 con 0x7ffab415df60 2023-10-16T11:20:12.949 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:20:12.947+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:20:12.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.948+0000 7ffa9b7fe700 1 -- 172.21.15.181:0/3032353914 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v120) v1 ==== 167+0+0 (secure 0 0 0) 0x7ffab40b0b30 con 0x7ffab415df60 2023-10-16T11:20:12.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffaa006b140 msgr2=0x7ffaa006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffaa006b140 0x7ffaa006d600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4000fb0 tx=0x7ffaa4009040 comp rx=0 tx=0).stop 2023-10-16T11:20:12.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab415df60 msgr2=0x7ffab40b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:12.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab415df60 0x7ffab40b4340 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7ffab40716a0 tx=0x7ffaac004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 shutdown_connections 2023-10-16T11:20:12.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffaa006b140 0x7ffaa006d600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffab415df60 0x7ffab40b4340 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab406afb0 0x7ffab41479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 --2- 172.21.15.181:0/3032353914 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab40d0fc0 0x7ffab40c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:12.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 >> 172.21.15.181:0/3032353914 conn(0x7ffab40d04a0 msgr2=0x7ffab41e4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:12.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 shutdown_connections 2023-10-16T11:20:12.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:12.952+0000 7ffab91ee700 1 -- 172.21.15.181:0/3032353914 wait complete. 2023-10-16T11:20:12.957 INFO:tasks.ceph.mds.c.smithi181.stderr: -1826> 2023-10-16T11:19:39.507+0000 7f7fd22fc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:20:12.957 INFO:teuthology.orchestra.run.smithi181.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:20:12.967 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-16T11:20:12.980 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:20:13.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.088+0000 7fd5f27d4700 1 Processor -- start 2023-10-16T11:20:13.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.088+0000 7fd5f27d4700 1 -- start start 2023-10-16T11:20:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5f27d4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5ec15da70 0x7fd5ec0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5f27d4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec0ce6a0 0x7fd5ec06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:13.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5f27d4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec0cef00 con 0x7fd5ec072760 2023-10-16T11:20:13.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5f27d4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec0cec20 con 0x7fd5ec15da70 2023-10-16T11:20:13.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5f27d4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec071c80 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5f0d71700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd5ec072760 0x7fd5ec1615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40220/0 (socket says 172.21.15.181:40220) 2023-10-16T11:20:13.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5f0d71700 1 -- 172.21.15.181:0/1664758875 learned_addr learned my addr 172.21.15.181:0/1664758875 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5ebfff700 1 --2- 172.21.15.181:0/1664758875 >> v2:172.21.15.190:3300/0 conn(0x7fd5ec15da70 0x7fd5ec0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.089+0000 7fd5eb7fe700 1 --2- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec0ce6a0 0x7fd5ec06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5eaffd700 1 -- 172.21.15.181:0/1664758875 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593466873 0 0) 0x7fd5ec0cef00 con 0x7fd5ec072760 2023-10-16T11:20:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5eaffd700 1 -- 172.21.15.181:0/1664758875 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d80035b0 con 0x7fd5ec072760 2023-10-16T11:20:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5eaffd700 1 -- 172.21.15.181:0/1664758875 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3135112957 0 0) 0x7fd5d80035b0 con 0x7fd5ec072760 2023-10-16T11:20:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5ebfff700 1 -- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec0ce6a0 msgr2=0x7fd5ec06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5ebfff700 1 --2- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec0ce6a0 0x7fd5ec06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5ebfff700 1 -- 172.21.15.181:0/1664758875 >> v1:172.21.15.181:6789/0 conn(0x7fd5ec072760 legacy=0x7fd5ec1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5ebfff700 1 -- 172.21.15.181:0/1664758875 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec071f60 con 0x7fd5ec15da70 2023-10-16T11:20:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5eb7fe700 1 --2- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec0ce6a0 0x7fd5ec06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5ebfff700 1 --2- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 0x7fd5ec0ce160 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009840 tx=0x7fd5d4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567e4ed2a2d9dfaa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5eaffd700 1 -- 172.21.15.181:0/1664758875 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40019f0 con 0x7fd5ec15da70 2023-10-16T11:20:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5eaffd700 1 -- 172.21.15.181:0/1664758875 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4004030 con 0x7fd5ec15da70 2023-10-16T11:20:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.090+0000 7fd5eaffd700 1 -- 172.21.15.181:0/1664758875 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40043a0 con 0x7fd5ec15da70 2023-10-16T11:20:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 -- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 msgr2=0x7fd5ec0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 0x7fd5ec0ce160 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009840 tx=0x7fd5d4009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 -- 172.21.15.181:0/1664758875 shutdown_connections 2023-10-16T11:20:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 0x7fd5ec0ce160 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/1664758875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec0ce6a0 0x7fd5ec06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 -- 172.21.15.181:0/1664758875 >> 172.21.15.181:0/1664758875 conn(0x7fd5ec0cd010 msgr2=0x7fd5ec0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:13.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 -- 172.21.15.181:0/1664758875 shutdown_connections 2023-10-16T11:20:13.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.091+0000 7fd5f27d4700 1 -- 172.21.15.181:0/1664758875 wait complete. 2023-10-16T11:20:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.092+0000 7fd5f27d4700 1 Processor -- start 2023-10-16T11:20:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.092+0000 7fd5f27d4700 1 -- start start 2023-10-16T11:20:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5f27d4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec072760 0x7fd5ec0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5f27d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5ec0ce6a0 0x7fd5ec151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5f27d4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 0x7fd5ec15ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5f27d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0ada00 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5f27d4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec15f310 con 0x7fd5ec15da70 2023-10-16T11:20:13.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5f27d4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec160440 con 0x7fd5ec072760 2023-10-16T11:20:13.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5ec0ce6a0 0x7fd5ec151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:13.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5ec0ce6a0 0x7fd5ec151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57378/0 (socket says 172.21.15.181:57378) 2023-10-16T11:20:13.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 -- 172.21.15.181:0/2933056209 learned_addr learned my addr 172.21.15.181:0/2933056209 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:13.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5ebfff700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec072760 0x7fd5ec0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:13.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5f0d71700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 0x7fd5ec15ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:13.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 -- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec072760 msgr2=0x7fd5ec0ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:13.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec072760 0x7fd5ec0ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 -- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 msgr2=0x7fd5ec15ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:13.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 0x7fd5ec15ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 -- 172.21.15.181:0/2933056209 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc0094a0 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5ebfff700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec072760 0x7fd5ec0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:13.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.093+0000 7fd5eb7fe700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5ec0ce6a0 0x7fd5ec151b60 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00ec10 tx=0x7fd5dc009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:13.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.094+0000 7fd5e8ff9700 1 -- 172.21.15.181:0/2933056209 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00c8e0 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.094+0000 7fd5e8ff9700 1 -- 172.21.15.181:0/2933056209 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc00ca40 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.094+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d40094a0 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.094+0000 7fd5e8ff9700 1 -- 172.21.15.181:0/2933056209 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00cdd0 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.094+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec0ad0d0 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.095+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ec06f6c0 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.095+0000 7fd5e8ff9700 1 -- 172.21.15.181:0/2933056209 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd5dc013470 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.095+0000 7fd5e8ff9700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5cc06b140 0x7fd5cc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:13.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.096+0000 7fd5e8ff9700 1 -- 172.21.15.181:0/2933056209 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7262+0+0 (secure 0 0 0) 0x7fd5dc08a150 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.096+0000 7fd5ebfff700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5cc06b140 0x7fd5cc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:13.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.096+0000 7fd5ebfff700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5cc06b140 0x7fd5cc06d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40090b0 tx=0x7fd5d4008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:13.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.100+0000 7fd5e8ff9700 1 -- 172.21.15.181:0/2933056209 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5dc054890 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.339+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5ec0b0240 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.950+0000 7fd5e8ff9700 1 -- 172.21.15.181:0/2933056209 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v121) v1 ==== 107+0+0 (secure 0 0 0) 0x7fd5dc057eb0 con 0x7fd5ec0ce6a0 2023-10-16T11:20:13.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5cc06b140 msgr2=0x7fd5cc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:13.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5cc06b140 0x7fd5cc06d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40090b0 tx=0x7fd5d4008030 comp rx=0 tx=0).stop 2023-10-16T11:20:13.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5ec0ce6a0 msgr2=0x7fd5ec151b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5ec0ce6a0 0x7fd5ec151b60 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00ec10 tx=0x7fd5dc009e20 comp rx=0 tx=0).stop 2023-10-16T11:20:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 shutdown_connections 2023-10-16T11:20:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5cc06b140 0x7fd5cc06d600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5ec072760 0x7fd5ec0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5ec15da70 0x7fd5ec15ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 --2- 172.21.15.181:0/2933056209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5ec0ce6a0 0x7fd5ec151b60 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 >> 172.21.15.181:0/2933056209 conn(0x7fd5ec0cd010 msgr2=0x7fd5ec15b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 shutdown_connections 2023-10-16T11:20:13.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:13.953+0000 7fd5f27d4700 1 -- 172.21.15.181:0/2933056209 wait complete. 2023-10-16T11:20:13.969 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-16T11:20:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.138+0000 7f22b6c60700 1 Processor -- start 2023-10-16T11:20:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.138+0000 7f22b6c60700 1 -- start start 2023-10-16T11:20:14.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22b6c60700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d01e0 0x7f22b00d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22b6c60700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22b00d05c0 0x7f22b006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22b6c60700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b015dd80 con 0x7f22b006ca60 2023-10-16T11:20:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22b6c60700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b006ba60 con 0x7f22b00d05c0 2023-10-16T11:20:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22b6c60700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00d42a0 con 0x7f22b00d01e0 2023-10-16T11:20:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22b00d05c0 0x7f22b006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:14.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22b00d05c0 0x7f22b006c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57388/0 (socket says 172.21.15.181:57388) 2023-10-16T11:20:14.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 -- 172.21.15.181:0/3363282527 learned_addr learned my addr 172.21.15.181:0/3363282527 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:14.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22b49fc700 1 --2- 172.21.15.181:0/3363282527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d01e0 0x7f22b00d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:14.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 -- 172.21.15.181:0/3363282527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d01e0 msgr2=0x7f22b00d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:14.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 --2- 172.21.15.181:0/3363282527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d01e0 0x7f22b00d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 -- 172.21.15.181:0/3363282527 >> v1:172.21.15.190:6789/0 conn(0x7f22b006ca60 legacy=0x7f22b006cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:14.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 -- 172.21.15.181:0/3363282527 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00d4580 con 0x7f22b00d05c0 2023-10-16T11:20:14.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22b49fc700 1 --2- 172.21.15.181:0/3363282527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d01e0 0x7f22b00d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:14.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22affff700 1 --2- 172.21.15.181:0/3363282527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d05c0 0x7f22b006c520 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f229c0098f0 tx=0x7f229c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bfc6d84c315094 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:14.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22af7fe700 1 -- 172.21.15.181:0/3363282527 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00b8a0 con 0x7f22b00d05c0 2023-10-16T11:20:14.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22af7fe700 1 -- 172.21.15.181:0/3363282527 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c00ba00 con 0x7f22b00d05c0 2023-10-16T11:20:14.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.139+0000 7f22af7fe700 1 -- 172.21.15.181:0/3363282527 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00bd90 con 0x7f22b00d05c0 2023-10-16T11:20:14.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 -- 172.21.15.181:0/3363282527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d05c0 msgr2=0x7f22b006c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:14.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 --2- 172.21.15.181:0/3363282527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d05c0 0x7f22b006c520 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f229c0098f0 tx=0x7f229c009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:14.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 -- 172.21.15.181:0/3363282527 shutdown_connections 2023-10-16T11:20:14.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 --2- 172.21.15.181:0/3363282527 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d01e0 0x7f22b00d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 --2- 172.21.15.181:0/3363282527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d05c0 0x7f22b006c520 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 -- 172.21.15.181:0/3363282527 >> 172.21.15.181:0/3363282527 conn(0x7f22b008af90 msgr2=0x7f22b00d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:14.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 -- 172.21.15.181:0/3363282527 shutdown_connections 2023-10-16T11:20:14.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.140+0000 7f22b6c60700 1 -- 172.21.15.181:0/3363282527 wait complete. 2023-10-16T11:20:14.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.141+0000 7f22b6c60700 1 Processor -- start 2023-10-16T11:20:14.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.141+0000 7f22b6c60700 1 -- start start 2023-10-16T11:20:14.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b6c60700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22b006ca60 0x7f22b0157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:14.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b6c60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d01e0 0x7f22b00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:14.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b6c60700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d05c0 0x7f22b00b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:14.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b6c60700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00ca1f0 con 0x7f22b00d01e0 2023-10-16T11:20:14.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b6c60700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0157da0 con 0x7f22b006ca60 2023-10-16T11:20:14.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b6c60700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00cabd0 con 0x7f22b00d05c0 2023-10-16T11:20:14.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d01e0 0x7f22b00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:14.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d01e0 0x7f22b00b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57394/0 (socket says 172.21.15.181:57394) 2023-10-16T11:20:14.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22affff700 1 -- 172.21.15.181:0/863112255 learned_addr learned my addr 172.21.15.181:0/863112255 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:14.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b51fd700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d05c0 0x7f22b00b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:14.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22b49fc700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22b006ca60 0x7f22b0157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:14.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.142+0000 7f22affff700 1 -- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d05c0 msgr2=0x7f22b00b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:14.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22affff700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d05c0 0x7f22b00b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22affff700 1 -- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22b006ca60 msgr2=0x7f22b0157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:14.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22affff700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22b006ca60 0x7f22b0157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22affff700 1 -- 172.21.15.181:0/863112255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009550 con 0x7f22b00d01e0 2023-10-16T11:20:14.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22b49fc700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22b006ca60 0x7f22b0157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:14.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22b51fd700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d05c0 0x7f22b00b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:14.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22affff700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d01e0 0x7f22b00b5530 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f229c014040 tx=0x7f229c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:14.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22ad7fa700 1 -- 172.21.15.181:0/863112255 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0036a0 con 0x7f22b00d01e0 2023-10-16T11:20:14.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b015dfa0 con 0x7f22b00d01e0 2023-10-16T11:20:14.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22ad7fa700 1 -- 172.21.15.181:0/863112255 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c01e070 con 0x7f22b00d01e0 2023-10-16T11:20:14.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22ad7fa700 1 -- 172.21.15.181:0/863112255 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c003a60 con 0x7f22b00d01e0 2023-10-16T11:20:14.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.143+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b00cb3e0 con 0x7f22b00d01e0 2023-10-16T11:20:14.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.144+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b00cb240 con 0x7f22b00d01e0 2023-10-16T11:20:14.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.144+0000 7f22ad7fa700 1 -- 172.21.15.181:0/863112255 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f229c003bc0 con 0x7f22b00d01e0 2023-10-16T11:20:14.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.145+0000 7f22ad7fa700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f229406b140 0x7f229406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:14.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.145+0000 7f22ad7fa700 1 -- 172.21.15.181:0/863112255 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7262+0+0 (secure 0 0 0) 0x7f229c020070 con 0x7f22b00d01e0 2023-10-16T11:20:14.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.145+0000 7f22b49fc700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f229406b140 0x7f229406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:14.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.145+0000 7f22b49fc700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f229406b140 0x7f229406d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009a50 tx=0x7f22a4008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:14.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.149+0000 7f22ad7fa700 1 -- 172.21.15.181:0/863112255 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f22b00cb240 con 0x7f22b00d01e0 2023-10-16T11:20:14.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.342+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f22b00cb240 con 0x7f22b00d01e0 2023-10-16T11:20:14.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.952+0000 7f22ad7fa700 1 -- 172.21.15.181:0/863112255 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v227) v1 ==== 190+0+0 (secure 0 0 0) 0x7f22b00cb240 con 0x7f22b00d01e0 2023-10-16T11:20:14.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f229406b140 msgr2=0x7f229406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f229406b140 0x7f229406d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009a50 tx=0x7f22a4008040 comp rx=0 tx=0).stop 2023-10-16T11:20:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d01e0 msgr2=0x7f22b00b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d01e0 0x7f22b00b5530 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f229c014040 tx=0x7f229c004750 comp rx=0 tx=0).stop 2023-10-16T11:20:14.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 shutdown_connections 2023-10-16T11:20:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f229406b140 0x7f229406d600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22b00d05c0 0x7f22b00b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22b006ca60 0x7f22b0157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 --2- 172.21.15.181:0/863112255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22b00d01e0 0x7f22b00b5530 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:14.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.955+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 >> 172.21.15.181:0/863112255 conn(0x7f22b008af90 msgr2=0x7f22b006cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.956+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 shutdown_connections 2023-10-16T11:20:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:14.956+0000 7f22b6c60700 1 -- 172.21.15.181:0/863112255 wait complete. 2023-10-16T11:20:14.960 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_metadata' removed 2023-10-16T11:20:14.973 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-16T11:20:15.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.100+0000 7fe181256700 1 Processor -- start 2023-10-16T11:20:15.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.100+0000 7fe181256700 1 -- start start 2023-10-16T11:20:15.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.100+0000 7fe181256700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 0x7fe17c15e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.100+0000 7fe181256700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0cbf70 con 0x7fe17c15deb0 2023-10-16T11:20:15.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.100+0000 7fe181256700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0cc1c0 con 0x7fe17c15e290 2023-10-16T11:20:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.100+0000 7fe181256700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0cc4a0 con 0x7fe17c066830 2023-10-16T11:20:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe17ad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe17c15deb0 0x7fe17c06ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40246/0 (socket says 172.21.15.181:40246) 2023-10-16T11:20:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe17ad9d700 1 -- 172.21.15.181:0/504406980 learned_addr learned my addr 172.21.15.181:0/504406980 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514339939 0 0) 0x7fe17c0cbf70 con 0x7fe17c15deb0 2023-10-16T11:20:15.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1640035b0 con 0x7fe17c15deb0 2023-10-16T11:20:15.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe17b59e700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 0x7fe17c15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902996858 0 0) 0x7fe1640035b0 con 0x7fe17c15deb0 2023-10-16T11:20:15.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 msgr2=0x7fe17c15e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:15.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 0x7fe17c15e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:15.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 >> v1:172.21.15.190:6789/0 conn(0x7fe17c15e290 legacy=0x7fe17c0cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:15.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c06ba50 con 0x7fe17c15deb0 2023-10-16T11:20:15.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe16c001670 con 0x7fe17c15deb0 2023-10-16T11:20:15.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe17b59e700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 0x7fe17c15e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:15.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 >> v1:172.21.15.181:6789/0 conn(0x7fe17c15deb0 legacy=0x7fe17c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:15.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe164003a30 0x7fe164003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 0x7fe164004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe164004d90 0x7fe164005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0cc4a0 con 0x7fe164004d90 2023-10-16T11:20:15.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16400bc40 con 0x7fe164003a30 2023-10-16T11:20:15.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16400be00 con 0x7fe17c066830 2023-10-16T11:20:15.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe17b59e700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe164004d90 0x7fe164005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe17ad9d700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe164003a30 0x7fe164003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.101+0000 7fe17a59c700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 0x7fe164004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe17b59e700 1 -- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 msgr2=0x7fe164004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:15.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe17b59e700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c066830 0x7fe164004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:15.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe17b59e700 1 -- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe164003a30 msgr2=0x7fe164003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:15.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe17b59e700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe164003a30 0x7fe164003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:15.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe17b59e700 1 -- 172.21.15.181:0/504406980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640035b0 con 0x7fe164004d90 2023-10-16T11:20:15.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe17b59e700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe164004d90 0x7fe164005210 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fe1740073e0 tx=0x7fe164005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcfebd017c744a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:15.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174007ea0 con 0x7fe164004d90 2023-10-16T11:20:15.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe174014400 con 0x7fe164004d90 2023-10-16T11:20:15.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe179d9b700 1 -- 172.21.15.181:0/504406980 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174014790 con 0x7fe164004d90 2023-10-16T11:20:15.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe181256700 1 -- 172.21.15.181:0/504406980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe164004d90 msgr2=0x7fe164005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:15.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe181256700 1 --2- 172.21.15.181:0/504406980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe164004d90 0x7fe164005210 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fe1740073e0 tx=0x7fe164005880 comp rx=0 tx=0).stop 2023-10-16T11:20:15.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe17b59e700 1 -- 172.21.15.181:0/504406980 reap_dead start 2023-10-16T11:20:15.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe181256700 1 -- 172.21.15.181:0/504406980 shutdown_connections 2023-10-16T11:20:15.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe181256700 1 -- 172.21.15.181:0/504406980 >> 172.21.15.181:0/504406980 conn(0x7fe17c0b1780 msgr2=0x7fe17c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:15.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe181256700 1 -- 172.21.15.181:0/504406980 shutdown_connections 2023-10-16T11:20:15.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.102+0000 7fe181256700 1 -- 172.21.15.181:0/504406980 wait complete. 2023-10-16T11:20:15.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.104+0000 7fe181256700 1 Processor -- start 2023-10-16T11:20:15.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.104+0000 7fe181256700 1 -- start start 2023-10-16T11:20:15.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c155ca0 0x7fe17c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c156080 0x7fe17c0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c15ce30 0x7fe17c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0afd60 con 0x7fe17c156080 2023-10-16T11:20:15.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c15c690 con 0x7fe17c15ce30 2023-10-16T11:20:15.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c15c970 con 0x7fe17c155ca0 2023-10-16T11:20:15.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c156080 0x7fe17c0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c156080 0x7fe17c0afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57412/0 (socket says 172.21.15.181:57412) 2023-10-16T11:20:15.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 -- 172.21.15.181:0/1755470158 learned_addr learned my addr 172.21.15.181:0/1755470158 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:15.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17ad9d700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c155ca0 0x7fe17c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17b59e700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c15ce30 0x7fe17c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 -- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c155ca0 msgr2=0x7fe17c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:15.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c155ca0 0x7fe17c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:15.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 -- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c15ce30 msgr2=0x7fe17c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:15.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c15ce30 0x7fe17c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:15.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 -- 172.21.15.181:0/1755470158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0b0d90 con 0x7fe17c156080 2023-10-16T11:20:15.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17b59e700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c15ce30 0x7fe17c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:15.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe17a59c700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c156080 0x7fe17c0afff0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fe17000f7f0 tx=0x7fe17000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:15.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe1637fe700 1 -- 172.21.15.181:0/1755470158 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700052e0 con 0x7fe17c156080 2023-10-16T11:20:15.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe1637fe700 1 -- 172.21.15.181:0/1755470158 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe170005440 con 0x7fe17c156080 2023-10-16T11:20:15.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c0b0a70 con 0x7fe17c156080 2023-10-16T11:20:15.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe1637fe700 1 -- 172.21.15.181:0/1755470158 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700038c0 con 0x7fe17c156080 2023-10-16T11:20:15.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.105+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c0b3900 con 0x7fe17c156080 2023-10-16T11:20:15.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.106+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c15cb30 con 0x7fe17c156080 2023-10-16T11:20:15.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.107+0000 7fe1637fe700 1 -- 172.21.15.181:0/1755470158 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe170003a20 con 0x7fe17c156080 2023-10-16T11:20:15.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.107+0000 7fe1637fe700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe16806b140 0x7fe16806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:15.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.107+0000 7fe1637fe700 1 -- 172.21.15.181:0/1755470158 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6818+0+0 (secure 0 0 0) 0x7fe170089ce0 con 0x7fe17c156080 2023-10-16T11:20:15.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.107+0000 7fe17ad9d700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe16806b140 0x7fe16806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:15.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.108+0000 7fe17ad9d700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe16806b140 0x7fe16806d600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0048d0 tx=0x7fe16c009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:15.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.112+0000 7fe1637fe700 1 -- 172.21.15.181:0/1755470158 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe1700545d0 con 0x7fe17c156080 2023-10-16T11:20:15.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:15.303+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe17c06fd40 con 0x7fe17c156080 2023-10-16T11:20:16.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.022+0000 7fe1637fe700 1 -- 172.21.15.181:0/1755470158 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v228) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe170057bf0 con 0x7fe17c156080 2023-10-16T11:20:16.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.026+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe16806b140 msgr2=0x7fe16806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.026+0000 7fe181256700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe16806b140 0x7fe16806d600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0048d0 tx=0x7fe16c009040 comp rx=0 tx=0).stop 2023-10-16T11:20:16.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.026+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c156080 msgr2=0x7fe17c0afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.026+0000 7fe181256700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c156080 0x7fe17c0afff0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fe17000f7f0 tx=0x7fe17000fbb0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.027+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 shutdown_connections 2023-10-16T11:20:16.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.027+0000 7fe181256700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe16806b140 0x7fe16806d600 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.027+0000 7fe181256700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c155ca0 0x7fe17c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.027+0000 7fe181256700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c15ce30 0x7fe17c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.027+0000 7fe181256700 1 --2- 172.21.15.181:0/1755470158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c156080 0x7fe17c0afff0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.027+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 >> 172.21.15.181:0/1755470158 conn(0x7fe17c0b1780 msgr2=0x7fe17c0cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:16.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.027+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 shutdown_connections 2023-10-16T11:20:16.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.028+0000 7fe181256700 1 -- 172.21.15.181:0/1755470158 wait complete. 2023-10-16T11:20:16.032 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_data' removed 2023-10-16T11:20:16.047 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:20:16.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.173+0000 7fb325ad6700 1 Processor -- start 2023-10-16T11:20:16.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.173+0000 7fb325ad6700 1 -- start start 2023-10-16T11:20:16.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.173+0000 7fb325ad6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3200ce2d0 0x7fb32015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.173+0000 7fb325ad6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32015ef80 con 0x7fb3200d01d0 2023-10-16T11:20:16.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.173+0000 7fb325ad6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3200cf580 con 0x7fb3200d1500 2023-10-16T11:20:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.173+0000 7fb325ad6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3200d4290 con 0x7fb3200ce2d0 2023-10-16T11:20:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3200ce2d0 0x7fb32015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb3200d01d0 0x7fb3200d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40264/0 (socket says 172.21.15.181:40264) 2023-10-16T11:20:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31f7fe700 1 -- 172.21.15.181:0/2199041516 learned_addr learned my addr 172.21.15.181:0/2199041516 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:16.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31effd700 1 -- 172.21.15.181:0/2199041516 >> v1:172.21.15.190:6789/0 conn(0x7fb3200d1500 legacy=0x7fb3200d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31effd700 1 -- 172.21.15.181:0/2199041516 >> v1:172.21.15.181:6789/0 conn(0x7fb3200d01d0 legacy=0x7fb3200d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31e7fc700 1 -- 172.21.15.181:0/2199041516 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931704368 0 0) 0x7fb310001110 con 0x7fb3200d01d0 2023-10-16T11:20:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31effd700 1 -- 172.21.15.181:0/2199041516 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3200ce970 con 0x7fb3200ce2d0 2023-10-16T11:20:16.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31effd700 1 --2- 172.21.15.181:0/2199041516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200ce2d0 0x7fb32015ea40 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7fb308009840 tx=0x7fb308009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77689cb40c6f2ac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31e7fc700 1 -- 172.21.15.181:0/2199041516 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308004230 con 0x7fb3200ce2d0 2023-10-16T11:20:16.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31e7fc700 1 -- 172.21.15.181:0/2199041516 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308004390 con 0x7fb3200ce2d0 2023-10-16T11:20:16.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.174+0000 7fb31e7fc700 1 -- 172.21.15.181:0/2199041516 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080036a0 con 0x7fb3200ce2d0 2023-10-16T11:20:16.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.175+0000 7fb325ad6700 1 -- 172.21.15.181:0/2199041516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200ce2d0 msgr2=0x7fb32015ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.175+0000 7fb325ad6700 1 --2- 172.21.15.181:0/2199041516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200ce2d0 0x7fb32015ea40 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7fb308009840 tx=0x7fb308009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:16.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.175+0000 7fb325ad6700 1 -- 172.21.15.181:0/2199041516 shutdown_connections 2023-10-16T11:20:16.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.175+0000 7fb325ad6700 1 --2- 172.21.15.181:0/2199041516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200ce2d0 0x7fb32015ea40 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.175+0000 7fb325ad6700 1 -- 172.21.15.181:0/2199041516 >> 172.21.15.181:0/2199041516 conn(0x7fb32009cb60 msgr2=0x7fb3200d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:16.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.175+0000 7fb325ad6700 1 -- 172.21.15.181:0/2199041516 shutdown_connections 2023-10-16T11:20:16.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.175+0000 7fb325ad6700 1 -- 172.21.15.181:0/2199041516 wait complete. 2023-10-16T11:20:16.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.176+0000 7fb325ad6700 1 Processor -- start 2023-10-16T11:20:16.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.177+0000 7fb325ad6700 1 -- start start 2023-10-16T11:20:16.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.177+0000 7fb325ad6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3200ce2d0 0x7fb32014bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.177+0000 7fb325ad6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3200d01d0 0x7fb32014ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.177+0000 7fb325ad6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200d1500 0x7fb3200b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.177+0000 7fb325ad6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200cb6b0 con 0x7fb3200d1500 2023-10-16T11:20:16.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.177+0000 7fb325ad6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200caa30 con 0x7fb3200d01d0 2023-10-16T11:20:16.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.177+0000 7fb325ad6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200cad10 con 0x7fb3200ce2d0 2023-10-16T11:20:16.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200d1500 0x7fb3200b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200d1500 0x7fb3200b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57428/0 (socket says 172.21.15.181:57428) 2023-10-16T11:20:16.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 -- 172.21.15.181:0/4044472961 learned_addr learned my addr 172.21.15.181:0/4044472961 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:16.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31f7fe700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3200ce2d0 0x7fb32014bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 -- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3200ce2d0 msgr2=0x7fb32014bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3200ce2d0 0x7fb32014bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 -- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3200d01d0 msgr2=0x7fb32014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31effd700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3200d01d0 0x7fb32014ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3200d01d0 0x7fb32014ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 -- 172.21.15.181:0/4044472961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3080094a0 con 0x7fb3200d1500 2023-10-16T11:20:16.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb31ffff700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200d1500 0x7fb3200b14a0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cc10 tx=0x7fb31800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb317fff700 1 -- 172.21.15.181:0/4044472961 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31800abb0 con 0x7fb3200d1500 2023-10-16T11:20:16.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb317fff700 1 -- 172.21.15.181:0/4044472961 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31800ad10 con 0x7fb3200d1500 2023-10-16T11:20:16.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3200caff0 con 0x7fb3200d1500 2023-10-16T11:20:16.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb317fff700 1 -- 172.21.15.181:0/4044472961 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318012660 con 0x7fb3200d1500 2023-10-16T11:20:16.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.178+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32014b2a0 con 0x7fb3200d1500 2023-10-16T11:20:16.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.179+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32004ebb0 con 0x7fb3200d1500 2023-10-16T11:20:16.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.179+0000 7fb317fff700 1 -- 172.21.15.181:0/4044472961 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb3180127c0 con 0x7fb3200d1500 2023-10-16T11:20:16.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.180+0000 7fb317fff700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb30006b140 0x7fb30006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.180+0000 7fb317fff700 1 -- 172.21.15.181:0/4044472961 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7fb31808a410 con 0x7fb3200d1500 2023-10-16T11:20:16.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.180+0000 7fb31f7fe700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb30006b140 0x7fb30006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.180+0000 7fb31f7fe700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb30006b140 0x7fb30006d600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb310000f60 tx=0x7fb310007a40 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.184+0000 7fb317fff700 1 -- 172.21.15.181:0/4044472961 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb318054f40 con 0x7fb3200d1500 2023-10-16T11:20:16.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.388+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb3200d0e00 con 0x7fb3200d1500 2023-10-16T11:20:16.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.391+0000 7fb317fff700 1 -- 172.21.15.181:0/4044472961 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v79) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb3180160b0 con 0x7fb3200d1500 2023-10-16T11:20:16.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb30006b140 msgr2=0x7fb30006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb30006b140 0x7fb30006d600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb310000f60 tx=0x7fb310007a40 comp rx=0 tx=0).stop 2023-10-16T11:20:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200d1500 msgr2=0x7fb3200b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200d1500 0x7fb3200b14a0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cc10 tx=0x7fb31800a060 comp rx=0 tx=0).stop 2023-10-16T11:20:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 shutdown_connections 2023-10-16T11:20:16.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb30006b140 0x7fb30006d600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3200ce2d0 0x7fb32014bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3200d1500 0x7fb3200b14a0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 --2- 172.21.15.181:0/4044472961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3200d01d0 0x7fb32014ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.394+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 >> 172.21.15.181:0/4044472961 conn(0x7fb32009cb60 msgr2=0x7fb32015cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:16.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.395+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 shutdown_connections 2023-10-16T11:20:16.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.395+0000 7fb325ad6700 1 -- 172.21.15.181:0/4044472961 wait complete. 2023-10-16T11:20:16.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:20:16.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.530+0000 7f646c2e8700 1 Processor -- start 2023-10-16T11:20:16.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.530+0000 7f646c2e8700 1 -- start start 2023-10-16T11:20:16.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646c2e8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f646415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646c2e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64640d1500 0x7f64640d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646c2e8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646415ef80 con 0x7f64640d01d0 2023-10-16T11:20:16.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646c2e8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64640d4290 con 0x7f64640d1500 2023-10-16T11:20:16.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646c2e8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64640ce750 con 0x7f64640ce2d0 2023-10-16T11:20:16.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646a084700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f64640d01d0 0x7f64640d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40278/0 (socket says 172.21.15.181:40278) 2023-10-16T11:20:16.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646a084700 1 -- 172.21.15.181:0/645248834 learned_addr learned my addr 172.21.15.181:0/645248834 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:16.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.531+0000 7f646a885700 1 --2- 172.21.15.181:0/645248834 >> v2:172.21.15.181:3300/0 conn(0x7f64640d1500 0x7f64640d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f6469883700 1 --2- 172.21.15.181:0/645248834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f646415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f6469082700 1 -- 172.21.15.181:0/645248834 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508813256 0 0) 0x7f6454001110 con 0x7f64640d01d0 2023-10-16T11:20:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f6469082700 1 -- 172.21.15.181:0/645248834 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64500035b0 con 0x7f64640d01d0 2023-10-16T11:20:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f646a885700 1 -- 172.21.15.181:0/645248834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 msgr2=0x7f646415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:16.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f646a885700 1 --2- 172.21.15.181:0/645248834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f646415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f646a885700 1 -- 172.21.15.181:0/645248834 >> v1:172.21.15.181:6789/0 conn(0x7f64640d01d0 legacy=0x7f64640d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f646a885700 1 -- 172.21.15.181:0/645248834 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64640cf580 con 0x7f64640d1500 2023-10-16T11:20:16.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f646a885700 1 --2- 172.21.15.181:0/645248834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1990 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f645c009df0 tx=0x7f645c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f1e19a3291b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f6469082700 1 -- 172.21.15.181:0/645248834 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00e070 con 0x7f64640d1500 2023-10-16T11:20:16.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f6469082700 1 -- 172.21.15.181:0/645248834 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c00baa0 con 0x7f64640d1500 2023-10-16T11:20:16.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.532+0000 7f6469082700 1 -- 172.21.15.181:0/645248834 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00be10 con 0x7f64640d1500 2023-10-16T11:20:16.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 -- 172.21.15.181:0/645248834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 msgr2=0x7f64640d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 --2- 172.21.15.181:0/645248834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1990 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f645c009df0 tx=0x7f645c00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:16.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 -- 172.21.15.181:0/645248834 shutdown_connections 2023-10-16T11:20:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 --2- 172.21.15.181:0/645248834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1990 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 --2- 172.21.15.181:0/645248834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f646415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 -- 172.21.15.181:0/645248834 >> 172.21.15.181:0/645248834 conn(0x7f646409cb60 msgr2=0x7f64640d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 -- 172.21.15.181:0/645248834 shutdown_connections 2023-10-16T11:20:16.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.533+0000 7f646c2e8700 1 -- 172.21.15.181:0/645248834 wait complete. 2023-10-16T11:20:16.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.534+0000 7f646c2e8700 1 Processor -- start 2023-10-16T11:20:16.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.534+0000 7f646c2e8700 1 -- start start 2023-10-16T11:20:16.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f6464151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64640d01d0 0x7f64640b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64640c7f70 con 0x7f64640d1500 2023-10-16T11:20:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64640c8250 con 0x7f64640d01d0 2023-10-16T11:20:16.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64640c8530 con 0x7f64640ce2d0 2023-10-16T11:20:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57442/0 (socket says 172.21.15.181:57442) 2023-10-16T11:20:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 -- 172.21.15.181:0/2015839114 learned_addr learned my addr 172.21.15.181:0/2015839114 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:16.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f6469883700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64640d01d0 0x7f64640b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a084700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f6464151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 -- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 msgr2=0x7f6464151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f6464151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 -- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64640d01d0 msgr2=0x7f64640b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64640d01d0 0x7f64640b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 -- 172.21.15.181:0/2015839114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c009a70 con 0x7f64640d1500 2023-10-16T11:20:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646a885700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1910 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f645c00b370 tx=0x7f645c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f645affd700 1 -- 172.21.15.181:0/2015839114 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0098f0 con 0x7f64640d1500 2023-10-16T11:20:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f645affd700 1 -- 172.21.15.181:0/2015839114 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c003e10 con 0x7f64640d1500 2023-10-16T11:20:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64640c8810 con 0x7f64640d1500 2023-10-16T11:20:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f645affd700 1 -- 172.21.15.181:0/2015839114 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c012670 con 0x7f64640d1500 2023-10-16T11:20:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.535+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64640b3660 con 0x7f64640d1500 2023-10-16T11:20:16.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.536+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64640c8d30 con 0x7f64640d1500 2023-10-16T11:20:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.537+0000 7f645affd700 1 -- 172.21.15.181:0/2015839114 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f645c0127d0 con 0x7f64640d1500 2023-10-16T11:20:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.537+0000 7f645affd700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f644806b140 0x7f644806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.537+0000 7f645affd700 1 -- 172.21.15.181:0/2015839114 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f645c08b9c0 con 0x7f64640d1500 2023-10-16T11:20:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.537+0000 7f646a084700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f644806b140 0x7f644806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.538+0000 7f646a084700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f644806b140 0x7f644806d600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6454005910 tx=0x7f6454009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.542+0000 7f645affd700 1 -- 172.21.15.181:0/2015839114 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f64640c8d30 con 0x7f64640d1500 2023-10-16T11:20:16.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.735+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f64640c8d30 con 0x7f64640d1500 2023-10-16T11:20:16.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.738+0000 7f645affd700 1 -- 172.21.15.181:0/2015839114 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v80) v1 ==== 120+0+0 (secure 0 0 0) 0x7f64640c8d30 con 0x7f64640d1500 2023-10-16T11:20:16.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f644806b140 msgr2=0x7f644806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f644806b140 0x7f644806d600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6454005910 tx=0x7f6454009040 comp rx=0 tx=0).stop 2023-10-16T11:20:16.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 msgr2=0x7f64640d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1910 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f645c00b370 tx=0x7f645c004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 shutdown_connections 2023-10-16T11:20:16.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f644806b140 0x7f644806d600 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64640ce2d0 0x7f6464151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64640d1500 0x7f64640d1910 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 --2- 172.21.15.181:0/2015839114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64640d01d0 0x7f64640b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 >> 172.21.15.181:0/2015839114 conn(0x7f646409cb60 msgr2=0x7f646415d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:16.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 shutdown_connections 2023-10-16T11:20:16.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.741+0000 7f646c2e8700 1 -- 172.21.15.181:0/2015839114 wait complete. 2023-10-16T11:20:16.756 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:20:16.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.877+0000 7f7e50f97700 1 Processor -- start 2023-10-16T11:20:16.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.877+0000 7f7e50f97700 1 -- start start 2023-10-16T11:20:16.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e50f97700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e4c06afb0 0x7f7e4c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e50f97700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4c0cef00 con 0x7f7e4c15da70 2023-10-16T11:20:16.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e50f97700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4c0cec20 con 0x7f7e4c072760 2023-10-16T11:20:16.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e50f97700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4c071c80 con 0x7f7e4c06afb0 2023-10-16T11:20:16.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e4c06afb0 0x7f7e4c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e4c15da70 0x7f7e4c0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40292/0 (socket says 172.21.15.181:40292) 2023-10-16T11:20:16.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4a59c700 1 -- 172.21.15.181:0/586279092 learned_addr learned my addr 172.21.15.181:0/586279092 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:16.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4ad9d700 1 -- 172.21.15.181:0/586279092 >> v1:172.21.15.190:6789/0 conn(0x7f7e4c072760 legacy=0x7f7e4c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4ad9d700 1 -- 172.21.15.181:0/586279092 >> v1:172.21.15.181:6789/0 conn(0x7f7e4c15da70 legacy=0x7f7e4c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4ad9d700 1 -- 172.21.15.181:0/586279092 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c071f60 con 0x7f7e4c06afb0 2023-10-16T11:20:16.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4959a700 1 -- 172.21.15.181:0/586279092 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539454993 0 0) 0x7f7e4c0cef00 con 0x7f7e4c15da70 2023-10-16T11:20:16.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4ad9d700 1 --2- 172.21.15.181:0/586279092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c06afb0 0x7f7e4c1615f0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009d90 tx=0x7f7e4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce66de424d7beca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4959a700 1 -- 172.21.15.181:0/586279092 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400e070 con 0x7f7e4c06afb0 2023-10-16T11:20:16.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4959a700 1 -- 172.21.15.181:0/586279092 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4400bbd0 con 0x7f7e4c06afb0 2023-10-16T11:20:16.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.878+0000 7f7e4959a700 1 -- 172.21.15.181:0/586279092 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e44012430 con 0x7f7e4c06afb0 2023-10-16T11:20:16.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.879+0000 7f7e50f97700 1 -- 172.21.15.181:0/586279092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c06afb0 msgr2=0x7f7e4c1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.879+0000 7f7e50f97700 1 --2- 172.21.15.181:0/586279092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c06afb0 0x7f7e4c1615f0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009d90 tx=0x7f7e4400b330 comp rx=0 tx=0).stop 2023-10-16T11:20:16.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.879+0000 7f7e50f97700 1 -- 172.21.15.181:0/586279092 shutdown_connections 2023-10-16T11:20:16.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.879+0000 7f7e50f97700 1 --2- 172.21.15.181:0/586279092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c06afb0 0x7f7e4c1615f0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.879+0000 7f7e50f97700 1 -- 172.21.15.181:0/586279092 >> 172.21.15.181:0/586279092 conn(0x7f7e4c0cd010 msgr2=0x7f7e4c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:16.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.879+0000 7f7e50f97700 1 -- 172.21.15.181:0/586279092 shutdown_connections 2023-10-16T11:20:16.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.879+0000 7f7e50f97700 1 -- 172.21.15.181:0/586279092 wait complete. 2023-10-16T11:20:16.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.881+0000 7f7e50f97700 1 Processor -- start 2023-10-16T11:20:16.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.881+0000 7f7e50f97700 1 -- start start 2023-10-16T11:20:16.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e50f97700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e4c06afb0 0x7f7e4c0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e50f97700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e4c072760 0x7f7e4c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e50f97700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c15da70 0x7f7e4c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e50f97700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c0b4510 con 0x7f7e4c15da70 2023-10-16T11:20:16.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e50f97700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c0b47f0 con 0x7f7e4c06afb0 2023-10-16T11:20:16.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e50f97700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c0b4ad0 con 0x7f7e4c072760 2023-10-16T11:20:16.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c15da70 0x7f7e4c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c15da70 0x7f7e4c0b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57456/0 (socket says 172.21.15.181:57456) 2023-10-16T11:20:16.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 -- 172.21.15.181:0/969086802 learned_addr learned my addr 172.21.15.181:0/969086802 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:16.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4a59c700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e4c06afb0 0x7f7e4c0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e49d9b700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e4c072760 0x7f7e4c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 -- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e4c072760 msgr2=0x7f7e4c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e4c072760 0x7f7e4c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 -- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e4c06afb0 msgr2=0x7f7e4c0b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:16.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e4c06afb0 0x7f7e4c0b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:16.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 -- 172.21.15.181:0/969086802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44009a70 con 0x7f7e4c15da70 2023-10-16T11:20:16.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e4ad9d700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c15da70 0x7f7e4c0b3bc0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f7e44014040 tx=0x7f7e44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.882+0000 7f7e3affd700 1 -- 172.21.15.181:0/969086802 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4401b810 con 0x7f7e4c15da70 2023-10-16T11:20:16.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.883+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c0b4db0 con 0x7f7e4c15da70 2023-10-16T11:20:16.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.883+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c0b5af0 con 0x7f7e4c15da70 2023-10-16T11:20:16.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.883+0000 7f7e3affd700 1 -- 172.21.15.181:0/969086802 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44020070 con 0x7f7e4c15da70 2023-10-16T11:20:16.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.883+0000 7f7e3affd700 1 -- 172.21.15.181:0/969086802 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4401bb70 con 0x7f7e4c15da70 2023-10-16T11:20:16.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.884+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e300052f0 con 0x7f7e4c15da70 2023-10-16T11:20:16.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.884+0000 7f7e3affd700 1 -- 172.21.15.181:0/969086802 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7e4401bcd0 con 0x7f7e4c15da70 2023-10-16T11:20:16.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.884+0000 7f7e3affd700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7e2c06b140 0x7f7e2c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:16.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.884+0000 7f7e3affd700 1 -- 172.21.15.181:0/969086802 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f7e44022070 con 0x7f7e4c15da70 2023-10-16T11:20:16.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.885+0000 7f7e4a59c700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7e2c06b140 0x7f7e2c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:16.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.885+0000 7f7e4a59c700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7e2c06b140 0x7f7e2c06d600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c007b10 tx=0x7f7e3c007a60 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:16.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:16.889+0000 7f7e3affd700 1 -- 172.21.15.181:0/969086802 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7e44056fd0 con 0x7f7e4c15da70 2023-10-16T11:20:17.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.084+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7e30005b20 con 0x7f7e4c15da70 2023-10-16T11:20:17.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.087+0000 7f7e3affd700 1 -- 172.21.15.181:0/969086802 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v81) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7e4405a5f0 con 0x7f7e4c15da70 2023-10-16T11:20:17.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7e2c06b140 msgr2=0x7f7e2c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:17.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7e2c06b140 0x7f7e2c06d600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c007b10 tx=0x7f7e3c007a60 comp rx=0 tx=0).stop 2023-10-16T11:20:17.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c15da70 msgr2=0x7f7e4c0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:17.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c15da70 0x7f7e4c0b3bc0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f7e44014040 tx=0x7f7e44004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 shutdown_connections 2023-10-16T11:20:17.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7e2c06b140 0x7f7e2c06d600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4c15da70 0x7f7e4c0b3bc0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e4c06afb0 0x7f7e4c0b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 --2- 172.21.15.181:0/969086802 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e4c072760 0x7f7e4c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.089+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 >> 172.21.15.181:0/969086802 conn(0x7f7e4c0cd010 msgr2=0x7f7e4c0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.090+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 shutdown_connections 2023-10-16T11:20:17.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.090+0000 7f7e50f97700 1 -- 172.21.15.181:0/969086802 wait complete. 2023-10-16T11:20:17.104 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-16T11:20:17.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.223+0000 7f4f7a3f9700 1 Processor -- start 2023-10-16T11:20:17.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.223+0000 7f4f7a3f9700 1 -- start start 2023-10-16T11:20:17.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f7a3f9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f7415b990 0x7f4f740ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:17.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f7a3f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f740cd2d0 0x7f4f740d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f7a3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f740cfc60 0x7f4f740d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f7a3f9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f740d0630 con 0x7f4f7415b990 2023-10-16T11:20:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f7a3f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f740d24f0 con 0x7f4f740cd2d0 2023-10-16T11:20:17.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f7a3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740cc510 con 0x7f4f740cfc60 2023-10-16T11:20:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f7415b990 0x7f4f740ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f7415b990 0x7f4f740ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57460/0 (socket says 172.21.15.181:57460) 2023-10-16T11:20:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 -- 172.21.15.181:0/2874094675 learned_addr learned my addr 172.21.15.181:0/2874094675 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 -- 172.21.15.181:0/2874094675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f740cfc60 msgr2=0x7f4f740d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:17.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f78996700 1 --2- 172.21.15.181:0/2874094675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f740cfc60 0x7f4f740d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f737fe700 1 --2- 172.21.15.181:0/2874094675 >> v2:172.21.15.190:3300/0 conn(0x7f4f740cd2d0 0x7f4f740d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 --2- 172.21.15.181:0/2874094675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f740cfc60 0x7f4f740d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 -- 172.21.15.181:0/2874094675 >> v2:172.21.15.190:3300/0 conn(0x7f4f740cd2d0 msgr2=0x7f4f740d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 --2- 172.21.15.181:0/2874094675 >> v2:172.21.15.190:3300/0 conn(0x7f4f740cd2d0 0x7f4f740d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.224+0000 7f4f73fff700 1 -- 172.21.15.181:0/2874094675 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f740cc7f0 con 0x7f4f7415b990 2023-10-16T11:20:17.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f73fff700 1 --2- 172.21.15.181:0/2874094675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7415b990 0x7f4f740ce8f0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b4230 tx=0x7f4f68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bbe6b1a55ee152 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:17.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f72ffd700 1 -- 172.21.15.181:0/2874094675 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68005e30 con 0x7f4f7415b990 2023-10-16T11:20:17.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f72ffd700 1 -- 172.21.15.181:0/2874094675 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f68004030 con 0x7f4f7415b990 2023-10-16T11:20:17.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f72ffd700 1 -- 172.21.15.181:0/2874094675 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680043e0 con 0x7f4f7415b990 2023-10-16T11:20:17.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/2874094675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7415b990 msgr2=0x7f4f740ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:17.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/2874094675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7415b990 0x7f4f740ce8f0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f4f740b4230 tx=0x7f4f68009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/2874094675 shutdown_connections 2023-10-16T11:20:17.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/2874094675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f7415b990 0x7f4f740ce8f0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/2874094675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f740cfc60 0x7f4f740d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/2874094675 >> v2:172.21.15.190:3300/0 conn(0x7f4f740cd2d0 0x7f4f740d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/2874094675 >> 172.21.15.181:0/2874094675 conn(0x7f4f7409a9a0 msgr2=0x7f4f740d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:17.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/2874094675 shutdown_connections 2023-10-16T11:20:17.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.225+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/2874094675 wait complete. 2023-10-16T11:20:17.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.226+0000 7f4f7a3f9700 1 Processor -- start 2023-10-16T11:20:17.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f7a3f9700 1 -- start start 2023-10-16T11:20:17.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f7a3f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f740cd2d0 0x7f4f741f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:17.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f7a3f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740cfc60 0x7f4f7414c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:17.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f7a3f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7415b990 0x7f4f740ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:17.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f7a3f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7414c990 con 0x7f4f740cd2d0 2023-10-16T11:20:17.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f7a3f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740cb9d0 con 0x7f4f740cfc60 2023-10-16T11:20:17.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f7a3f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740cadc0 con 0x7f4f7415b990 2023-10-16T11:20:17.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.227+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f740cd2d0 0x7f4f741f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:17.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f740cd2d0 0x7f4f741f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57468/0 (socket says 172.21.15.181:57468) 2023-10-16T11:20:17.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 -- 172.21.15.181:0/3410900844 learned_addr learned my addr 172.21.15.181:0/3410900844 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:17.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f78996700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7415b990 0x7f4f740ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:17.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f737fe700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740cfc60 0x7f4f7414c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:17.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 -- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7415b990 msgr2=0x7f4f740ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:17.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7415b990 0x7f4f740ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 -- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740cfc60 msgr2=0x7f4f7414c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:17.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740cfc60 0x7f4f7414c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 -- 172.21.15.181:0/3410900844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68009500 con 0x7f4f740cd2d0 2023-10-16T11:20:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f73fff700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f740cd2d0 0x7f4f741f3de0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f4f68000c00 tx=0x7f4f68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f70ff9700 1 -- 172.21.15.181:0/3410900844 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68011620 con 0x7f4f740cd2d0 2023-10-16T11:20:17.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f70ff9700 1 -- 172.21.15.181:0/3410900844 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f680117a0 con 0x7f4f740cd2d0 2023-10-16T11:20:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f740cb0a0 con 0x7f4f740cd2d0 2023-10-16T11:20:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f70ff9700 1 -- 172.21.15.181:0/3410900844 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68011b50 con 0x7f4f740cd2d0 2023-10-16T11:20:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.228+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f740cb660 con 0x7f4f740cd2d0 2023-10-16T11:20:17.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.229+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f74067d20 con 0x7f4f740cd2d0 2023-10-16T11:20:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.230+0000 7f4f70ff9700 1 -- 172.21.15.181:0/3410900844 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4f68011cb0 con 0x7f4f740cd2d0 2023-10-16T11:20:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.230+0000 7f4f70ff9700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4f6006b140 0x7f4f6006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.230+0000 7f4f737fe700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4f6006b140 0x7f4f6006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.230+0000 7f4f70ff9700 1 -- 172.21.15.181:0/3410900844 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f4f6808b080 con 0x7f4f740cd2d0 2023-10-16T11:20:17.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.230+0000 7f4f737fe700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4f6006b140 0x7f4f6006d600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4f7406be90 tx=0x7f4f64008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:17.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.235+0000 7f4f70ff9700 1 -- 172.21.15.181:0/3410900844 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4f68055aa0 con 0x7f4f740cd2d0 2023-10-16T11:20:17.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:17.509+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f4f740cfa40 con 0x7f4f740cd2d0 2023-10-16T11:20:18.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.025+0000 7f4f70ff9700 1 -- 172.21.15.181:0/3410900844 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3150) v1 ==== 186+0+0 (secure 0 0 0) 0x7f4f680590c0 con 0x7f4f740cd2d0 2023-10-16T11:20:18.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4f6006b140 msgr2=0x7f4f6006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4f6006b140 0x7f4f6006d600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4f7406be90 tx=0x7f4f64008040 comp rx=0 tx=0).stop 2023-10-16T11:20:18.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f740cd2d0 msgr2=0x7f4f741f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f740cd2d0 0x7f4f741f3de0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f4f68000c00 tx=0x7f4f68004750 comp rx=0 tx=0).stop 2023-10-16T11:20:18.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 shutdown_connections 2023-10-16T11:20:18.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4f6006b140 0x7f4f6006d600 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f7415b990 0x7f4f740ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f740cd2d0 0x7f4f741f3de0 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 --2- 172.21.15.181:0/3410900844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f740cfc60 0x7f4f7414c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 >> 172.21.15.181:0/3410900844 conn(0x7f4f7409a9a0 msgr2=0x7f4f740b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:18.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 shutdown_connections 2023-10-16T11:20:18.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.028+0000 7f4f7a3f9700 1 -- 172.21.15.181:0/3410900844 wait complete. 2023-10-16T11:20:18.044 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-10-16T11:20:18.044 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-10-16T11:20:18.044 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-16T11:20:18.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:18.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.165+0000 7efecf7d2700 1 Processor -- start 2023-10-16T11:20:18.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.165+0000 7efecf7d2700 1 -- start start 2023-10-16T11:20:18.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efecf7d2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efec815dfd0 0x7efec80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efecf7d2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efec80af450 0x7efec806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efecf7d2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec806f870 con 0x7efec806db80 2023-10-16T11:20:18.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efecf7d2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec80d1160 con 0x7efec80af450 2023-10-16T11:20:18.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efecf7d2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec80cc8e0 con 0x7efec815dfd0 2023-10-16T11:20:18.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efeccd6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efec80af450 0x7efec806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efeccd6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efec80af450 0x7efec806d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57474/0 (socket says 172.21.15.181:57474) 2023-10-16T11:20:18.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efeccd6d700 1 -- 172.21.15.181:0/3456474265 learned_addr learned my addr 172.21.15.181:0/3456474265 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:18.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.166+0000 7efecd56e700 1 --2- 172.21.15.181:0/3456474265 >> v2:172.21.15.190:3300/0 conn(0x7efec815dfd0 0x7efec80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efeccd6d700 1 -- 172.21.15.181:0/3456474265 >> v2:172.21.15.190:3300/0 conn(0x7efec815dfd0 msgr2=0x7efec80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:18.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efeccd6d700 1 --2- 172.21.15.181:0/3456474265 >> v2:172.21.15.190:3300/0 conn(0x7efec815dfd0 0x7efec80af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efeccd6d700 1 -- 172.21.15.181:0/3456474265 >> v1:172.21.15.190:6789/0 conn(0x7efec806db80 legacy=0x7efec80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:18.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efeccd6d700 1 -- 172.21.15.181:0/3456474265 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80ccbc0 con 0x7efec80af450 2023-10-16T11:20:18.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efeccd6d700 1 --2- 172.21.15.181:0/3456474265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec80af450 0x7efec806d640 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7efec8070340 tx=0x7efec4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baebfb7e49eb3afe server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efebffff700 1 -- 172.21.15.181:0/3456474265 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4004230 con 0x7efec80af450 2023-10-16T11:20:18.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efebffff700 1 -- 172.21.15.181:0/3456474265 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec4004390 con 0x7efec80af450 2023-10-16T11:20:18.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efebffff700 1 -- 172.21.15.181:0/3456474265 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec40036a0 con 0x7efec80af450 2023-10-16T11:20:18.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efecf7d2700 1 -- 172.21.15.181:0/3456474265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec80af450 msgr2=0x7efec806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efecf7d2700 1 --2- 172.21.15.181:0/3456474265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec80af450 0x7efec806d640 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7efec8070340 tx=0x7efec4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efecf7d2700 1 -- 172.21.15.181:0/3456474265 shutdown_connections 2023-10-16T11:20:18.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efecf7d2700 1 --2- 172.21.15.181:0/3456474265 >> v2:172.21.15.190:3300/0 conn(0x7efec815dfd0 0x7efec80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efecf7d2700 1 --2- 172.21.15.181:0/3456474265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec80af450 0x7efec806d640 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.167+0000 7efecf7d2700 1 -- 172.21.15.181:0/3456474265 >> 172.21.15.181:0/3456474265 conn(0x7efec80d04a0 msgr2=0x7efec80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:18.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.168+0000 7efecf7d2700 1 -- 172.21.15.181:0/3456474265 shutdown_connections 2023-10-16T11:20:18.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.168+0000 7efecf7d2700 1 -- 172.21.15.181:0/3456474265 wait complete. 2023-10-16T11:20:18.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 Processor -- start 2023-10-16T11:20:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 -- start start 2023-10-16T11:20:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec806db80 0x7efec8159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efec80af450 0x7efec80b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec815dfd0 0x7efec80b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80b00a0 con 0x7efec815dfd0 2023-10-16T11:20:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80b0380 con 0x7efec806db80 2023-10-16T11:20:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.169+0000 7efecf7d2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec80b0660 con 0x7efec80af450 2023-10-16T11:20:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec815dfd0 0x7efec80b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec815dfd0 0x7efec80b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57482/0 (socket says 172.21.15.181:57482) 2023-10-16T11:20:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 -- 172.21.15.181:0/2055359306 learned_addr learned my addr 172.21.15.181:0/2055359306 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:18.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efeccd6d700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efec80af450 0x7efec80b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecd56e700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec806db80 0x7efec8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 -- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efec80af450 msgr2=0x7efec80b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efec80af450 0x7efec80b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 -- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec806db80 msgr2=0x7efec8159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec806db80 0x7efec8159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 -- 172.21.15.181:0/2055359306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4009550 con 0x7efec815dfd0 2023-10-16T11:20:18.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecdd6f700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec815dfd0 0x7efec80b5a00 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7efec000cc10 tx=0x7efec000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efebdffb700 1 -- 172.21.15.181:0/2055359306 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000abb0 con 0x7efec815dfd0 2023-10-16T11:20:18.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efebdffb700 1 -- 172.21.15.181:0/2055359306 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000ad10 con 0x7efec815dfd0 2023-10-16T11:20:18.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec80b0940 con 0x7efec815dfd0 2023-10-16T11:20:18.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efebdffb700 1 -- 172.21.15.181:0/2055359306 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0012660 con 0x7efec815dfd0 2023-10-16T11:20:18.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.170+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec8159cc0 con 0x7efec815dfd0 2023-10-16T11:20:18.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.171+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec8067d20 con 0x7efec815dfd0 2023-10-16T11:20:18.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.172+0000 7efebdffb700 1 -- 172.21.15.181:0/2055359306 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7efec00127c0 con 0x7efec815dfd0 2023-10-16T11:20:18.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.172+0000 7efebdffb700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7efeac06b140 0x7efeac06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.172+0000 7efebdffb700 1 -- 172.21.15.181:0/2055359306 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7efec008a430 con 0x7efec815dfd0 2023-10-16T11:20:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.172+0000 7efecd56e700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7efeac06b140 0x7efeac06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.173+0000 7efecd56e700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7efeac06b140 0x7efeac06d600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006fd0 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.177+0000 7efebdffb700 1 -- 172.21.15.181:0/2055359306 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efec0054ec0 con 0x7efec815dfd0 2023-10-16T11:20:18.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.370+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efec8066fb0 con 0x7efec815dfd0 2023-10-16T11:20:18.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.371+0000 7efebdffb700 1 -- 172.21.15.181:0/2055359306 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v228) v1 ==== 74+0+15220 (secure 0 0 0) 0x7efec0016090 con 0x7efec815dfd0 2023-10-16T11:20:18.373 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:18.373 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":228,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:16.019526+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/4167832945":"2023-10-17T11:20:12.948635+0000","172.21.15.181:6835/4167832945":"2023-10-17T11:20:12.948635+0000","172.21.15.190:6835/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/2033295595":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/4244377789":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2576397499":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6833/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/893959583":"2023-10-17T11:20:03.733490+0000","172.21.15.190:6833/2252072910":"2023-10-17T11:20:06.138513+0000","172.21.15.181:0/754905711":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3499373399":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/2963042451":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/3469785622":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/1000076443":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/3401982361":"2023-10-17T11:19:53.266253+0000","172.21.15.181:6833/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2131678440":"2023-10-17T11:20:03.733490+0000","172.21.15.181:0/134902336":"2023-10-17T11:20:03.733490+0000","172.21.15.181:6832/1781343309":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/866682955":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/833955386":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/1312491057":"2023-10-17T11:19:53.266253+0000","172.21.15.181:0/2328723488":"2023-10-17T11:19:53.266253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:18.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7efeac06b140 msgr2=0x7efeac06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7efeac06b140 0x7efeac06d600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006fd0 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2023-10-16T11:20:18.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec815dfd0 msgr2=0x7efec80b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec815dfd0 0x7efec80b5a00 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7efec000cc10 tx=0x7efec000a060 comp rx=0 tx=0).stop 2023-10-16T11:20:18.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 shutdown_connections 2023-10-16T11:20:18.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7efeac06b140 0x7efeac06d600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efec815dfd0 0x7efec80b5a00 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efec806db80 0x7efec8159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 --2- 172.21.15.181:0/2055359306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efec80af450 0x7efec80b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.373+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 >> 172.21.15.181:0/2055359306 conn(0x7efec80d04a0 msgr2=0x7efec815d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:18.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.374+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 shutdown_connections 2023-10-16T11:20:18.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.374+0000 7efecf7d2700 1 -- 172.21.15.181:0/2055359306 wait complete. 2023-10-16T11:20:18.388 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:20:18.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:20:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.511+0000 7f431cc46700 1 Processor -- start 2023-10-16T11:20:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.511+0000 7f431cc46700 1 -- start start 2023-10-16T11:20:18.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431cc46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f431815da70 0x7f43180ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431cc46700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43180cef00 con 0x7f431806afb0 2023-10-16T11:20:18.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431cc46700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43180cec20 con 0x7f43180ce6a0 2023-10-16T11:20:18.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431cc46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4318071c80 con 0x7f431815da70 2023-10-16T11:20:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f4316d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f431806afb0 0x7f43181615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40336/0 (socket says 172.21.15.181:40336) 2023-10-16T11:20:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f4316d9d700 1 -- 172.21.15.181:0/327991252 learned_addr learned my addr 172.21.15.181:0/327991252 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431659c700 1 --2- 172.21.15.181:0/327991252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f431815da70 0x7f43180ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515032430 0 0) 0x7f43180cef00 con 0x7f431806afb0 2023-10-16T11:20:18.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43000035b0 con 0x7f431806afb0 2023-10-16T11:20:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.512+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322648163 0 0) 0x7f43180cec20 con 0x7f43180ce6a0 2023-10-16T11:20:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43180cef00 con 0x7f43180ce6a0 2023-10-16T11:20:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2138072936 0 0) 0x7f43000035b0 con 0x7f431806afb0 2023-10-16T11:20:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431659c700 1 -- 172.21.15.181:0/327991252 >> v1:172.21.15.190:6789/0 conn(0x7f43180ce6a0 legacy=0x7f431806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431659c700 1 -- 172.21.15.181:0/327991252 >> v1:172.21.15.181:6789/0 conn(0x7f431806afb0 legacy=0x7f43181615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431659c700 1 -- 172.21.15.181:0/327991252 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4318071f60 con 0x7f431815da70 2023-10-16T11:20:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431659c700 1 --2- 172.21.15.181:0/327991252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f431815da70 0x7f43180ce160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4308009870 tx=0x7f4308009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ba92b32303cbdb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43100019f0 con 0x7f431815da70 2023-10-16T11:20:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4308004030 con 0x7f431815da70 2023-10-16T11:20:18.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431559a700 1 -- 172.21.15.181:0/327991252 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080043c0 con 0x7f431815da70 2023-10-16T11:20:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431cc46700 1 -- 172.21.15.181:0/327991252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f431815da70 msgr2=0x7f43180ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.513+0000 7f431cc46700 1 --2- 172.21.15.181:0/327991252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f431815da70 0x7f43180ce160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4308009870 tx=0x7f4308009b80 comp rx=0 tx=0).stop 2023-10-16T11:20:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.514+0000 7f431cc46700 1 -- 172.21.15.181:0/327991252 shutdown_connections 2023-10-16T11:20:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.514+0000 7f431cc46700 1 --2- 172.21.15.181:0/327991252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f431815da70 0x7f43180ce160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.514+0000 7f431cc46700 1 -- 172.21.15.181:0/327991252 >> 172.21.15.181:0/327991252 conn(0x7f43180cd010 msgr2=0x7f43180cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.514+0000 7f431cc46700 1 -- 172.21.15.181:0/327991252 shutdown_connections 2023-10-16T11:20:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.514+0000 7f431cc46700 1 -- 172.21.15.181:0/327991252 wait complete. 2023-10-16T11:20:18.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.515+0000 7f431cc46700 1 Processor -- start 2023-10-16T11:20:18.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.515+0000 7f431cc46700 1 -- start start 2023-10-16T11:20:18.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431cc46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431806afb0 0x7f4318157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431cc46700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43180ce6a0 0x7f43180c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431cc46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431815da70 0x7f43180b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431cc46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4318158070 con 0x7f431806afb0 2023-10-16T11:20:18.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431cc46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43181f64b0 con 0x7f431815da70 2023-10-16T11:20:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431cc46700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43181f6790 con 0x7f43180ce6a0 2023-10-16T11:20:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431806afb0 0x7f4318157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431806afb0 0x7f4318157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57492/0 (socket says 172.21.15.181:57492) 2023-10-16T11:20:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f431659c700 1 -- 172.21.15.181:0/1764679979 learned_addr learned my addr 172.21.15.181:0/1764679979 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:18.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.516+0000 7f4315d9b700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43180ce6a0 0x7f43180c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f4316d9d700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431815da70 0x7f43180b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431659c700 1 -- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43180ce6a0 msgr2=0x7f43180c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431659c700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43180ce6a0 0x7f43180c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431659c700 1 -- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431815da70 msgr2=0x7f43180b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431659c700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431815da70 0x7f43180b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431659c700 1 -- 172.21.15.181:0/1764679979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43080094a0 con 0x7f431806afb0 2023-10-16T11:20:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431659c700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431806afb0 0x7f4318157b30 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f4308000c00 tx=0x7f4308004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f4306ffd700 1 -- 172.21.15.181:0/1764679979 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308003c50 con 0x7f431806afb0 2023-10-16T11:20:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f4306ffd700 1 -- 172.21.15.181:0/1764679979 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430801f070 con 0x7f431806afb0 2023-10-16T11:20:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43181f6a70 con 0x7f431806afb0 2023-10-16T11:20:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f4306ffd700 1 -- 172.21.15.181:0/1764679979 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011630 con 0x7f431806afb0 2023-10-16T11:20:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.517+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43181f7030 con 0x7f431806afb0 2023-10-16T11:20:18.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.518+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43180ac880 con 0x7f431806afb0 2023-10-16T11:20:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.519+0000 7f4306ffd700 1 -- 172.21.15.181:0/1764679979 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4308021030 con 0x7f431806afb0 2023-10-16T11:20:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.519+0000 7f4306ffd700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f42f806b140 0x7f42f806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.519+0000 7f4306ffd700 1 -- 172.21.15.181:0/1764679979 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f430808af00 con 0x7f431806afb0 2023-10-16T11:20:18.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.519+0000 7f4315d9b700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f42f806b140 0x7f42f806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.519+0000 7f4315d9b700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f42f806b140 0x7f42f806d600 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f430c0011e0 tx=0x7f430c009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.523+0000 7f4306ffd700 1 -- 172.21.15.181:0/1764679979 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4308055b50 con 0x7f431806afb0 2023-10-16T11:20:18.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.717+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f431806f6c0 con 0x7f431806afb0 2023-10-16T11:20:18.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.717+0000 7f4306ffd700 1 -- 172.21.15.181:0/1764679979 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v228) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4308059170 con 0x7f431806afb0 2023-10-16T11:20:18.719 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:20:18.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f42f806b140 msgr2=0x7f42f806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f42f806b140 0x7f42f806d600 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f430c0011e0 tx=0x7f430c009040 comp rx=0 tx=0).stop 2023-10-16T11:20:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431806afb0 msgr2=0x7f4318157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431806afb0 0x7f4318157b30 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f4308000c00 tx=0x7f4308004750 comp rx=0 tx=0).stop 2023-10-16T11:20:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 shutdown_connections 2023-10-16T11:20:18.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f42f806b140 0x7f42f806d600 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f431815da70 0x7f43180b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f431806afb0 0x7f4318157b30 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 --2- 172.21.15.181:0/1764679979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43180ce6a0 0x7f43180c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 >> 172.21.15.181:0/1764679979 conn(0x7f43180cd010 msgr2=0x7f43180b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:18.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 shutdown_connections 2023-10-16T11:20:18.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.719+0000 7f431cc46700 1 -- 172.21.15.181:0/1764679979 wait complete. 2023-10-16T11:20:18.734 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-16T11:20:18.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.855+0000 7f637e567700 1 Processor -- start 2023-10-16T11:20:18.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.855+0000 7f637e567700 1 -- start start 2023-10-16T11:20:18.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.855+0000 7f637e567700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f637815e6d0 0x7f63780d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.855+0000 7f637e567700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63780cb0c0 0x7f637815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.855+0000 7f637e567700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637806ed00 con 0x7f637815eab0 2023-10-16T11:20:18.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.855+0000 7f637e567700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63780d02d0 con 0x7f637815e6d0 2023-10-16T11:20:18.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.855+0000 7f637e567700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637815ad40 con 0x7f63780cb0c0 2023-10-16T11:20:18.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6377fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f637815e6d0 0x7f63780d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f63777fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f637815eab0 0x7f637815a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40346/0 (socket says 172.21.15.181:40346) 2023-10-16T11:20:18.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f63777fe700 1 -- 172.21.15.181:0/517923404 learned_addr learned my addr 172.21.15.181:0/517923404 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:18.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f637cb04700 1 --2- 172.21.15.181:0/517923404 >> v2:172.21.15.190:3300/0 conn(0x7f63780cb0c0 0x7f637815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6377fff700 1 -- 172.21.15.181:0/517923404 >> v2:172.21.15.190:3300/0 conn(0x7f63780cb0c0 msgr2=0x7f637815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:18.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6377fff700 1 --2- 172.21.15.181:0/517923404 >> v2:172.21.15.190:3300/0 conn(0x7f63780cb0c0 0x7f637815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6377fff700 1 -- 172.21.15.181:0/517923404 >> v1:172.21.15.181:6789/0 conn(0x7f637815eab0 legacy=0x7f637815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6377fff700 1 -- 172.21.15.181:0/517923404 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637806e0f0 con 0x7f637815e6d0 2023-10-16T11:20:18.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6376ffd700 1 -- 172.21.15.181:0/517923404 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168682728 0 0) 0x7f637806ed00 con 0x7f637815eab0 2023-10-16T11:20:18.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6377fff700 1 --2- 172.21.15.181:0/517923404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f637815e6d0 0x7f63780d0fc0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f6378071650 tx=0x7f6368009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c91ae14ae332b28 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6376ffd700 1 -- 172.21.15.181:0/517923404 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368005e30 con 0x7f637815e6d0 2023-10-16T11:20:18.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6376ffd700 1 -- 172.21.15.181:0/517923404 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6368006030 con 0x7f637815e6d0 2023-10-16T11:20:18.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.856+0000 7f6376ffd700 1 -- 172.21.15.181:0/517923404 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63680063c0 con 0x7f637815e6d0 2023-10-16T11:20:18.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 -- 172.21.15.181:0/517923404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f637815e6d0 msgr2=0x7f63780d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 --2- 172.21.15.181:0/517923404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f637815e6d0 0x7f63780d0fc0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f6378071650 tx=0x7f6368009b00 comp rx=0 tx=0).stop 2023-10-16T11:20:18.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 -- 172.21.15.181:0/517923404 shutdown_connections 2023-10-16T11:20:18.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 --2- 172.21.15.181:0/517923404 >> v2:172.21.15.190:3300/0 conn(0x7f63780cb0c0 0x7f637815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 --2- 172.21.15.181:0/517923404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f637815e6d0 0x7f63780d0fc0 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 -- 172.21.15.181:0/517923404 >> 172.21.15.181:0/517923404 conn(0x7f63780d04a0 msgr2=0x7f63780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:18.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 -- 172.21.15.181:0/517923404 shutdown_connections 2023-10-16T11:20:18.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.857+0000 7f637e567700 1 -- 172.21.15.181:0/517923404 wait complete. 2023-10-16T11:20:18.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.858+0000 7f637e567700 1 Processor -- start 2023-10-16T11:20:18.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.858+0000 7f637e567700 1 -- start start 2023-10-16T11:20:18.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637e567700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63780cb0c0 0x7f637815d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637e567700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f637815e6d0 0x7f63780b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637e567700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f637815eab0 0x7f63780c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637e567700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780b2810 con 0x7f63780cb0c0 2023-10-16T11:20:18.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637e567700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637815b950 con 0x7f637815eab0 2023-10-16T11:20:18.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637e567700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637815dc90 con 0x7f637815e6d0 2023-10-16T11:20:18.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63780cb0c0 0x7f637815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63780cb0c0 0x7f637815d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57504/0 (socket says 172.21.15.181:57504) 2023-10-16T11:20:18.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 -- 172.21.15.181:0/2234580691 learned_addr learned my addr 172.21.15.181:0/2234580691 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:18.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 -- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f637815e6d0 msgr2=0x7f63780b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f63777fe700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f637815e6d0 0x7f63780b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637cb04700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f637815eab0 0x7f63780c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f637815e6d0 0x7f63780b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 -- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f637815eab0 msgr2=0x7f63780c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:18.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f637815eab0 0x7f63780c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:18.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 -- 172.21.15.181:0/2234580691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368009430 con 0x7f63780cb0c0 2023-10-16T11:20:18.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f637cb04700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f637815eab0 0x7f63780c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:18.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f63777fe700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f637815e6d0 0x7f63780b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:18.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6377fff700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63780cb0c0 0x7f637815d750 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f6368000c00 tx=0x7f6368004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6374ff9700 1 -- 172.21.15.181:0/2234580691 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368004040 con 0x7f63780cb0c0 2023-10-16T11:20:18.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6374ff9700 1 -- 172.21.15.181:0/2234580691 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6368020070 con 0x7f63780cb0c0 2023-10-16T11:20:18.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.859+0000 7f6374ff9700 1 -- 172.21.15.181:0/2234580691 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368004400 con 0x7f63780cb0c0 2023-10-16T11:20:18.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.860+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637815b020 con 0x7f63780cb0c0 2023-10-16T11:20:18.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.860+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637815b5e0 con 0x7f63780cb0c0 2023-10-16T11:20:18.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.860+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637815b7a0 con 0x7f63780cb0c0 2023-10-16T11:20:18.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.861+0000 7f6374ff9700 1 -- 172.21.15.181:0/2234580691 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6368022030 con 0x7f63780cb0c0 2023-10-16T11:20:18.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.861+0000 7f6374ff9700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f636406b140 0x7f636406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:18.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.861+0000 7f6374ff9700 1 -- 172.21.15.181:0/2234580691 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f636808b400 con 0x7f63780cb0c0 2023-10-16T11:20:18.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.861+0000 7f63777fe700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f636406b140 0x7f636406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:18.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.862+0000 7f63777fe700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f636406b140 0x7f636406d600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63780c6510 tx=0x7f636c009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:18.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:18.866+0000 7f6374ff9700 1 -- 172.21.15.181:0/2234580691 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6368055f30 con 0x7f63780cb0c0 2023-10-16T11:20:19.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:19.135+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f6378067d20 con 0x7f63780cb0c0 2023-10-16T11:20:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.035+0000 7f6374ff9700 1 -- 172.21.15.181:0/2234580691 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3152) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6368059550 con 0x7f63780cb0c0 2023-10-16T11:20:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f636406b140 msgr2=0x7f636406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f636406b140 0x7f636406d600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63780c6510 tx=0x7f636c009040 comp rx=0 tx=0).stop 2023-10-16T11:20:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63780cb0c0 msgr2=0x7f637815d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63780cb0c0 0x7f637815d750 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f6368000c00 tx=0x7f6368004750 comp rx=0 tx=0).stop 2023-10-16T11:20:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 shutdown_connections 2023-10-16T11:20:20.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f636406b140 0x7f636406d600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f637815eab0 0x7f63780c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63780cb0c0 0x7f637815d750 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 --2- 172.21.15.181:0/2234580691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f637815e6d0 0x7f63780b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 >> 172.21.15.181:0/2234580691 conn(0x7f63780d04a0 msgr2=0x7f63780cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:20.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 shutdown_connections 2023-10-16T11:20:20.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.039+0000 7f637e567700 1 -- 172.21.15.181:0/2234580691 wait complete. 2023-10-16T11:20:20.056 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:20:20.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.184+0000 7f7a2e781700 1 Processor -- start 2023-10-16T11:20:20.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a2e781700 1 -- start start 2023-10-16T11:20:20.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a2e781700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a280d01d0 0x7f7a280ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a2e781700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a280cf580 con 0x7f7a2815ea40 2023-10-16T11:20:20.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a2e781700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a280ce970 con 0x7f7a280d1500 2023-10-16T11:20:20.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a2e781700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a280d4290 con 0x7f7a280d01d0 2023-10-16T11:20:20.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a27fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a280d01d0 0x7f7a280ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a277fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7a2815ea40 0x7f7a280d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40360/0 (socket says 172.21.15.181:40360) 2023-10-16T11:20:20.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.185+0000 7f7a277fe700 1 -- 172.21.15.181:0/1296447675 learned_addr learned my addr 172.21.15.181:0/1296447675 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:20.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a27fff700 1 -- 172.21.15.181:0/1296447675 >> v1:172.21.15.190:6789/0 conn(0x7f7a280d1500 legacy=0x7f7a280d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:20.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a27fff700 1 -- 172.21.15.181:0/1296447675 >> v1:172.21.15.181:6789/0 conn(0x7f7a2815ea40 legacy=0x7f7a280d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a26ffd700 1 -- 172.21.15.181:0/1296447675 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622735791 0 0) 0x7f7a280cf580 con 0x7f7a2815ea40 2023-10-16T11:20:20.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a27fff700 1 -- 172.21.15.181:0/1296447675 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a280cec50 con 0x7f7a280d01d0 2023-10-16T11:20:20.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a27fff700 1 --2- 172.21.15.181:0/1296447675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d01d0 0x7f7a280ce2d0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009840 tx=0x7f7a1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa987202f26c510d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:20.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a26ffd700 1 -- 172.21.15.181:0/1296447675 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c011830 con 0x7f7a280d01d0 2023-10-16T11:20:20.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a26ffd700 1 -- 172.21.15.181:0/1296447675 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c011990 con 0x7f7a280d01d0 2023-10-16T11:20:20.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.186+0000 7f7a26ffd700 1 -- 172.21.15.181:0/1296447675 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c011d20 con 0x7f7a280d01d0 2023-10-16T11:20:20.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.187+0000 7f7a2e781700 1 -- 172.21.15.181:0/1296447675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d01d0 msgr2=0x7f7a280ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.187+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1296447675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d01d0 0x7f7a280ce2d0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009840 tx=0x7f7a1c009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:20.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.187+0000 7f7a2e781700 1 -- 172.21.15.181:0/1296447675 shutdown_connections 2023-10-16T11:20:20.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.187+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1296447675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d01d0 0x7f7a280ce2d0 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.187+0000 7f7a2e781700 1 -- 172.21.15.181:0/1296447675 >> 172.21.15.181:0/1296447675 conn(0x7f7a2809cb60 msgr2=0x7f7a280d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:20.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.187+0000 7f7a2e781700 1 -- 172.21.15.181:0/1296447675 shutdown_connections 2023-10-16T11:20:20.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.187+0000 7f7a2e781700 1 -- 172.21.15.181:0/1296447675 wait complete. 2023-10-16T11:20:20.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.188+0000 7f7a2e781700 1 Processor -- start 2023-10-16T11:20:20.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.188+0000 7f7a2e781700 1 -- start start 2023-10-16T11:20:20.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2e781700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a280d01d0 0x7f7a2806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2e781700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d1500 0x7f7a280c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2e781700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a2815ea40 0x7f7a2815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2e781700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2806c820 con 0x7f7a280d1500 2023-10-16T11:20:20.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2e781700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a280c85d0 con 0x7f7a280d01d0 2023-10-16T11:20:20.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2e781700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2806cce0 con 0x7f7a2815ea40 2023-10-16T11:20:20.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d1500 0x7f7a280c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d1500 0x7f7a280c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57520/0 (socket says 172.21.15.181:57520) 2023-10-16T11:20:20.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 -- 172.21.15.181:0/1334282067 learned_addr learned my addr 172.21.15.181:0/1334282067 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:20.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2cd1e700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a2815ea40 0x7f7a2815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a27fff700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a280d01d0 0x7f7a2806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 -- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a2815ea40 msgr2=0x7f7a2815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a2815ea40 0x7f7a2815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 -- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a280d01d0 msgr2=0x7f7a2806d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a280d01d0 0x7f7a2806d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 -- 172.21.15.181:0/1334282067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0094a0 con 0x7f7a280d1500 2023-10-16T11:20:20.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a277fe700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d1500 0x7f7a280c92a0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c550 tx=0x7f7a1800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a24ff9700 1 -- 172.21.15.181:0/1334282067 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18004c80 con 0x7f7a280d1500 2023-10-16T11:20:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a24ff9700 1 -- 172.21.15.181:0/1334282067 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18004de0 con 0x7f7a280d1500 2023-10-16T11:20:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2806ca00 con 0x7f7a280d1500 2023-10-16T11:20:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.189+0000 7f7a24ff9700 1 -- 172.21.15.181:0/1334282067 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18007310 con 0x7f7a280d1500 2023-10-16T11:20:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.190+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a280c8d80 con 0x7f7a280d1500 2023-10-16T11:20:20.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.190+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a280c8fa0 con 0x7f7a280d1500 2023-10-16T11:20:20.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.191+0000 7f7a24ff9700 1 -- 172.21.15.181:0/1334282067 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7a18007470 con 0x7f7a280d1500 2023-10-16T11:20:20.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.191+0000 7f7a24ff9700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a1406b140 0x7f7a1406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.192+0000 7f7a24ff9700 1 -- 172.21.15.181:0/1334282067 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f7a18089990 con 0x7f7a280d1500 2023-10-16T11:20:20.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.192+0000 7f7a27fff700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a1406b140 0x7f7a1406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.192+0000 7f7a27fff700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a1406b140 0x7f7a1406d600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0090b0 tx=0x7f7a1c008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:20.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.196+0000 7f7a24ff9700 1 -- 172.21.15.181:0/1334282067 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a18054430 con 0x7f7a280d1500 2023-10-16T11:20:20.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.389+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7a280c88d0 con 0x7f7a280d1500 2023-10-16T11:20:20.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.393+0000 7f7a24ff9700 1 -- 172.21.15.181:0/1334282067 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v82)=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a18057a50 con 0x7f7a280d1500 2023-10-16T11:20:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a1406b140 msgr2=0x7f7a1406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.397 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a1406b140 0x7f7a1406d600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0090b0 tx=0x7f7a1c008030 comp rx=0 tx=0).stop 2023-10-16T11:20:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d1500 msgr2=0x7f7a280c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d1500 0x7f7a280c92a0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800c550 tx=0x7f7a1800c910 comp rx=0 tx=0).stop 2023-10-16T11:20:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 shutdown_connections 2023-10-16T11:20:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f7a1406b140 0x7f7a1406d600 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.398 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a2815ea40 0x7f7a2815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a280d01d0 0x7f7a2806d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 --2- 172.21.15.181:0/1334282067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a280d1500 0x7f7a280c92a0 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 >> 172.21.15.181:0/1334282067 conn(0x7f7a2809cb60 msgr2=0x7f7a280b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:20.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 shutdown_connections 2023-10-16T11:20:20.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.395+0000 7f7a2e781700 1 -- 172.21.15.181:0/1334282067 wait complete. 2023-10-16T11:20:20.409 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:20:20.410 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:20:20.455 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:20:20.508 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:20:20.509 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:20:20.510 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:20.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:20:20.530 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:20:20.531 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:20.551 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:20:20.552 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:20:20.552 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:20.589 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:20:20.590 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:20:20.590 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:20.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:20:20.611 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:20:20.611 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.749+0000 7f50e9555700 1 Processor -- start 2023-10-16T11:20:20.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.749+0000 7f50e9555700 1 -- start start 2023-10-16T11:20:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.749+0000 7f50e9555700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e40af450 0x7f50e40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.749+0000 7f50e9555700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50e40cd470 0x7f50e40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.749+0000 7f50e9555700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e406f870 con 0x7f50e415dfd0 2023-10-16T11:20:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.749+0000 7f50e9555700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e40d1160 con 0x7f50e40cd470 2023-10-16T11:20:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.749+0000 7f50e9555700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40cc8e0 con 0x7f50e40af450 2023-10-16T11:20:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50e40cd470 0x7f50e40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50e40cd470 0x7f50e40cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57528/0 (socket says 172.21.15.181:57528) 2023-10-16T11:20:20.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 -- 172.21.15.181:0/1943092348 learned_addr learned my addr 172.21.15.181:0/1943092348 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:20.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e27fc700 1 --2- 172.21.15.181:0/1943092348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e40af450 0x7f50e40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 -- 172.21.15.181:0/1943092348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e40af450 msgr2=0x7f50e40af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:20.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 --2- 172.21.15.181:0/1943092348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e40af450 0x7f50e40af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 -- 172.21.15.181:0/1943092348 >> v1:172.21.15.190:6789/0 conn(0x7f50e415dfd0 legacy=0x7f50e406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:20.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 -- 172.21.15.181:0/1943092348 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40ccbc0 con 0x7f50e40cd470 2023-10-16T11:20:20.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e37fe700 1 --2- 172.21.15.181:0/1943092348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e40cd920 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009e50 tx=0x7f50dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0ca7779f5d9fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:20.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e1ffb700 1 -- 172.21.15.181:0/1943092348 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc00e070 con 0x7f50e40cd470 2023-10-16T11:20:20.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e1ffb700 1 -- 172.21.15.181:0/1943092348 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc003a20 con 0x7f50e40cd470 2023-10-16T11:20:20.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.750+0000 7f50e1ffb700 1 -- 172.21.15.181:0/1943092348 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc003dd0 con 0x7f50e40cd470 2023-10-16T11:20:20.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 -- 172.21.15.181:0/1943092348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 msgr2=0x7f50e40cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 --2- 172.21.15.181:0/1943092348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e40cd920 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009e50 tx=0x7f50dc00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:20.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 -- 172.21.15.181:0/1943092348 shutdown_connections 2023-10-16T11:20:20.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 --2- 172.21.15.181:0/1943092348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e40cd920 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 --2- 172.21.15.181:0/1943092348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e40af450 0x7f50e40af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 -- 172.21.15.181:0/1943092348 >> 172.21.15.181:0/1943092348 conn(0x7f50e40d04a0 msgr2=0x7f50e40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:20.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 -- 172.21.15.181:0/1943092348 shutdown_connections 2023-10-16T11:20:20.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.751+0000 7f50e9555700 1 -- 172.21.15.181:0/1943092348 wait complete. 2023-10-16T11:20:20.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.752+0000 7f50e9555700 1 Processor -- start 2023-10-16T11:20:20.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.752+0000 7f50e9555700 1 -- start start 2023-10-16T11:20:20.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e9555700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50e40af450 0x7f50e40c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e9555700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e4154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e9555700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e415dfd0 0x7f50e406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e9555700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40b2db0 con 0x7f50e40cd470 2023-10-16T11:20:20.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e9555700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40b43a0 con 0x7f50e40af450 2023-10-16T11:20:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e9555700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40b4680 con 0x7f50e415dfd0 2023-10-16T11:20:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e4154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e4154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57534/0 (socket says 172.21.15.181:57534) 2023-10-16T11:20:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 -- 172.21.15.181:0/2478266962 learned_addr learned my addr 172.21.15.181:0/2478266962 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:20.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e37fe700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e415dfd0 0x7f50e406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e2ffd700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50e40af450 0x7f50e40c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 -- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e415dfd0 msgr2=0x7f50e406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e415dfd0 0x7f50e406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 -- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50e40af450 msgr2=0x7f50e40c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50e40af450 0x7f50e40c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 -- 172.21.15.181:0/2478266962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc009b30 con 0x7f50e40cd470 2023-10-16T11:20:20.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e27fc700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e4154c30 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c7f0 tx=0x7f50d800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:20.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50cb7fe700 1 -- 172.21.15.181:0/2478266962 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8011800 con 0x7f50e40cd470 2023-10-16T11:20:20.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50cb7fe700 1 -- 172.21.15.181:0/2478266962 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8011960 con 0x7f50e40cd470 2023-10-16T11:20:20.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50cb7fe700 1 -- 172.21.15.181:0/2478266962 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8011cf0 con 0x7f50e40cd470 2023-10-16T11:20:20.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.753+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e40b4960 con 0x7f50e40cd470 2023-10-16T11:20:20.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.754+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e4155170 con 0x7f50e40cd470 2023-10-16T11:20:20.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.754+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e40b4e80 con 0x7f50e40cd470 2023-10-16T11:20:20.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.755+0000 7f50cb7fe700 1 -- 172.21.15.181:0/2478266962 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f50d8018360 con 0x7f50e40cd470 2023-10-16T11:20:20.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.755+0000 7f50cb7fe700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f50d006b140 0x7f50d006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:20.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.755+0000 7f50cb7fe700 1 -- 172.21.15.181:0/2478266962 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7f50d8089db0 con 0x7f50e40cd470 2023-10-16T11:20:20.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.755+0000 7f50e2ffd700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f50d006b140 0x7f50d006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:20.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.756+0000 7f50e2ffd700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f50d006b140 0x7f50d006d600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f50d4006fd0 tx=0x7f50d4008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:20.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.760+0000 7f50cb7fe700 1 -- 172.21.15.181:0/2478266962 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50e40b4e80 con 0x7f50e40cd470 2023-10-16T11:20:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.993+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50e40b4e80 con 0x7f50e40cd470 2023-10-16T11:20:20.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.993+0000 7f50cb7fe700 1 -- 172.21.15.181:0/2478266962 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+3715 (secure 0 0 0) 0x7f50e40b4e80 con 0x7f50e40cd470 2023-10-16T11:20:20.996 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:20.996 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":121,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8810,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2781841684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2781841684},{"type":"v1","addr":"172.21.15.190:6834","nonce":2781841684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":9245,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/103948090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":103948090},{"type":"v1","addr":"172.21.15.190:6835","nonce":103948090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":10000,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3087406129","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3087406129},{"type":"v1","addr":"172.21.15.181:6836","nonce":3087406129}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":12078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1745580686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1745580686},{"type":"v1","addr":"172.21.15.181:6837","nonce":1745580686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":121}],"filesystems":[]} 2023-10-16T11:20:20.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f50d006b140 msgr2=0x7f50d006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f50d006b140 0x7f50d006d600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f50d4006fd0 tx=0x7f50d4008040 comp rx=0 tx=0).stop 2023-10-16T11:20:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 msgr2=0x7f50e4154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e4154c30 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c7f0 tx=0x7f50d800cbb0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 shutdown_connections 2023-10-16T11:20:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f50d006b140 0x7f50d006d600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50e415dfd0 0x7f50e406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50e40af450 0x7f50e40c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 --2- 172.21.15.181:0/2478266962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50e40cd470 0x7f50e4154c30 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:20.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.995+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 >> 172.21.15.181:0/2478266962 conn(0x7f50e40d04a0 msgr2=0x7f50e4073b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.996+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 shutdown_connections 2023-10-16T11:20:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:20.996+0000 7f50e9555700 1 -- 172.21.15.181:0/2478266962 wait complete. 2023-10-16T11:20:21.000 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 121 2023-10-16T11:20:21.010 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:20:21.011 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:20:21.011 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:20:21.011 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:20:21.012 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:20:21.012 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:20:21.012 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:20:21.012 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:20:21.013 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:20:21.013 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:20:21.013 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:20:21.014 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:20:21.014 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:20:21.014 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:20:21.015 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:20:21.015 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:20:21.021 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:20:21.021 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:20:21.022 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:20:21.023 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:20:21.025 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:20:21.025 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:20:21.026 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:20:21.026 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:20:21.064 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:20:21.065 DEBUG:teuthology.parallel:result is None 2023-10-16T11:20:21.066 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:20:21.066 DEBUG:teuthology.parallel:result is None 2023-10-16T11:20:21.069 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:20:21.069 DEBUG:teuthology.parallel:result is None 2023-10-16T11:20:21.070 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:20:21.070 DEBUG:teuthology.parallel:result is None 2023-10-16T11:20:21.071 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:20:21.143 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:20:21.144 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:20:21.158 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:20:21.159 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:20:21.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.197+0000 7ffb56131700 1 Processor -- start 2023-10-16T11:20:21.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.197+0000 7ffb56131700 1 -- start start 2023-10-16T11:20:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb56131700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb5015bd50 0x7ffb500d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb56131700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb500b05d0 0x7ffb500cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb56131700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb500d24f0 con 0x7ffb500cfc60 2023-10-16T11:20:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb56131700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5015ccc0 con 0x7ffb500b05d0 2023-10-16T11:20:21.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb56131700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5015c320 con 0x7ffb5015bd50 2023-10-16T11:20:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb5015bd50 0x7ffb500d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb500b05d0 0x7ffb500cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb5015bd50 0x7ffb500d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:54592/0 (socket says 172.21.15.181:54592) 2023-10-16T11:20:21.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4effd700 1 -- 172.21.15.181:0/3203531398 learned_addr learned my addr 172.21.15.181:0/3203531398 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4ffff700 1 -- 172.21.15.181:0/3203531398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb5015bd50 msgr2=0x7ffb500d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4ffff700 1 --2- 172.21.15.181:0/3203531398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb5015bd50 0x7ffb500d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4ffff700 1 -- 172.21.15.181:0/3203531398 >> v1:172.21.15.190:6789/0 conn(0x7ffb500cfc60 legacy=0x7ffb500d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4ffff700 1 -- 172.21.15.181:0/3203531398 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5015c600 con 0x7ffb500b05d0 2023-10-16T11:20:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.198+0000 7ffb4ffff700 1 --2- 172.21.15.181:0/3203531398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb500b05d0 0x7ffb500cc200 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009d90 tx=0x7ffb4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83effa23f097ac7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:21.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb4e7fc700 1 -- 172.21.15.181:0/3203531398 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800e070 con 0x7ffb500b05d0 2023-10-16T11:20:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb4e7fc700 1 -- 172.21.15.181:0/3203531398 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4800bb20 con 0x7ffb500b05d0 2023-10-16T11:20:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb4e7fc700 1 -- 172.21.15.181:0/3203531398 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48012430 con 0x7ffb500b05d0 2023-10-16T11:20:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 -- 172.21.15.181:0/3203531398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb500b05d0 msgr2=0x7ffb500cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 --2- 172.21.15.181:0/3203531398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb500b05d0 0x7ffb500cc200 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009d90 tx=0x7ffb4800b330 comp rx=0 tx=0).stop 2023-10-16T11:20:21.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 -- 172.21.15.181:0/3203531398 shutdown_connections 2023-10-16T11:20:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 --2- 172.21.15.181:0/3203531398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb500b05d0 0x7ffb500cc200 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 --2- 172.21.15.181:0/3203531398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb5015bd50 0x7ffb500d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 -- 172.21.15.181:0/3203531398 >> 172.21.15.181:0/3203531398 conn(0x7ffb5009a9a0 msgr2=0x7ffb500d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 -- 172.21.15.181:0/3203531398 shutdown_connections 2023-10-16T11:20:21.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.199+0000 7ffb56131700 1 -- 172.21.15.181:0/3203531398 wait complete. 2023-10-16T11:20:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.200+0000 7ffb56131700 1 Processor -- start 2023-10-16T11:20:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb56131700 1 -- start start 2023-10-16T11:20:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb56131700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb500b05d0 0x7ffb50157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb56131700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb500cfc60 0x7ffb5015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb56131700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb5015bd50 0x7ffb5015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb56131700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501f4500 con 0x7ffb500b05d0 2023-10-16T11:20:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb56131700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500b6580 con 0x7ffb5015bd50 2023-10-16T11:20:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb56131700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501f4750 con 0x7ffb500cfc60 2023-10-16T11:20:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb500b05d0 0x7ffb50157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb4effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb500cfc60 0x7ffb5015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb500b05d0 0x7ffb50157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57570/0 (socket says 172.21.15.181:57570) 2023-10-16T11:20:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb4f7fe700 1 -- 172.21.15.181:0/1739611850 learned_addr learned my addr 172.21.15.181:0/1739611850 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb4effd700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb500cfc60 0x7ffb5015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:54598/0 (socket says 172.21.15.181:54598) 2023-10-16T11:20:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.201+0000 7ffb4ffff700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb5015bd50 0x7ffb5015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb4effd700 1 -- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb5015bd50 msgr2=0x7ffb5015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb4effd700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb5015bd50 0x7ffb5015a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb4effd700 1 -- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb500b05d0 msgr2=0x7ffb50157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb4effd700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb500b05d0 0x7ffb50157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb4effd700 1 -- 172.21.15.181:0/1739611850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb48009a70 con 0x7ffb500cfc60 2023-10-16T11:20:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb4f7fe700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb500b05d0 0x7ffb50157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb4effd700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb500cfc60 0x7ffb5015a090 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400e870 tx=0x7ffb4400eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb37fff700 1 -- 172.21.15.181:0/1739611850 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44013680 con 0x7ffb500cfc60 2023-10-16T11:20:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb37fff700 1 -- 172.21.15.181:0/1739611850 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb440137e0 con 0x7ffb500cfc60 2023-10-16T11:20:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb501f4a30 con 0x7ffb500cfc60 2023-10-16T11:20:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb37fff700 1 -- 172.21.15.181:0/1739611850 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44013b70 con 0x7ffb500cfc60 2023-10-16T11:20:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.202+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb501584e0 con 0x7ffb500cfc60 2023-10-16T11:20:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.203+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb50067d20 con 0x7ffb500cfc60 2023-10-16T11:20:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.204+0000 7ffb37fff700 1 -- 172.21.15.181:0/1739611850 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ffb44013cd0 con 0x7ffb500cfc60 2023-10-16T11:20:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.204+0000 7ffb37fff700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffb3c06b140 0x7ffb3c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.204+0000 7ffb37fff700 1 -- 172.21.15.181:0/1739611850 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6398+0+0 (secure 0 0 0) 0x7ffb44089960 con 0x7ffb500cfc60 2023-10-16T11:20:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.204+0000 7ffb4f7fe700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffb3c06b140 0x7ffb3c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:21.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.204+0000 7ffb4f7fe700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffb3c06b140 0x7ffb3c06d600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffb40001d10 tx=0x7ffb40009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:21.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.209+0000 7ffb37fff700 1 -- 172.21.15.181:0/1739611850 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffb44054490 con 0x7ffb500cfc60 2023-10-16T11:20:21.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:21.415+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffb500cfa40 con 0x7ffb500cfc60 2023-10-16T11:20:22.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.050+0000 7ffb37fff700 1 -- 172.21.15.181:0/1739611850 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v230) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb44057ab0 con 0x7ffb500cfc60 2023-10-16T11:20:22.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffb3c06b140 msgr2=0x7ffb3c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffb3c06b140 0x7ffb3c06d600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffb40001d10 tx=0x7ffb40009040 comp rx=0 tx=0).stop 2023-10-16T11:20:22.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb500cfc60 msgr2=0x7ffb5015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb500cfc60 0x7ffb5015a090 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400e870 tx=0x7ffb4400eb80 comp rx=0 tx=0).stop 2023-10-16T11:20:22.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 shutdown_connections 2023-10-16T11:20:22.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffb3c06b140 0x7ffb3c06d600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb5015bd50 0x7ffb5015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb500b05d0 0x7ffb50157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 --2- 172.21.15.181:0/1739611850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb500cfc60 0x7ffb5015a090 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 >> 172.21.15.181:0/1739611850 conn(0x7ffb5009a9a0 msgr2=0x7ffb500ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:22.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 shutdown_connections 2023-10-16T11:20:22.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.053+0000 7ffb56131700 1 -- 172.21.15.181:0/1739611850 wait complete. 2023-10-16T11:20:22.058 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T11:20:22.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:20:22.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.193+0000 7febd3005700 1 Processor -- start 2023-10-16T11:20:22.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.193+0000 7febd3005700 1 -- start start 2023-10-16T11:20:22.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd3005700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febcc0ce7f0 0x7febcc06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd3005700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd3005700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febcc0cc470 con 0x7febcc0d0180 2023-10-16T11:20:22.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd3005700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febcc0d0610 con 0x7febcc0ce7f0 2023-10-16T11:20:22.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd3005700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc06ca30 con 0x7febcc0d2390 2023-10-16T11:20:22.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febcc0ce7f0 0x7febcc06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febcc0ce7f0 0x7febcc06c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57580/0 (socket says 172.21.15.181:57580) 2023-10-16T11:20:22.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 -- 172.21.15.181:0/1619801067 learned_addr learned my addr 172.21.15.181:0/1619801067 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:22.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febcbfff700 1 --2- 172.21.15.181:0/1619801067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 -- 172.21.15.181:0/1619801067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 msgr2=0x7febcc0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:22.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 --2- 172.21.15.181:0/1619801067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 -- 172.21.15.181:0/1619801067 >> v1:172.21.15.190:6789/0 conn(0x7febcc0d0180 legacy=0x7febcc161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:22.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 -- 172.21.15.181:0/1619801067 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc06cd10 con 0x7febcc0ce7f0 2023-10-16T11:20:22.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febd0da1700 1 --2- 172.21.15.181:0/1619801067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc06c100 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7febc0006fd0 tx=0x7febc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbb612a10e5f6b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:22.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.194+0000 7febcb7fe700 1 -- 172.21.15.181:0/1619801067 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0005e30 con 0x7febcc0ce7f0 2023-10-16T11:20:22.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febcb7fe700 1 -- 172.21.15.181:0/1619801067 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc0006030 con 0x7febcc0ce7f0 2023-10-16T11:20:22.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febcb7fe700 1 -- 172.21.15.181:0/1619801067 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00063c0 con 0x7febcc0ce7f0 2023-10-16T11:20:22.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 -- 172.21.15.181:0/1619801067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 msgr2=0x7febcc06c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 --2- 172.21.15.181:0/1619801067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc06c100 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7febc0006fd0 tx=0x7febc0009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 -- 172.21.15.181:0/1619801067 shutdown_connections 2023-10-16T11:20:22.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 --2- 172.21.15.181:0/1619801067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc06c100 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 --2- 172.21.15.181:0/1619801067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 -- 172.21.15.181:0/1619801067 >> 172.21.15.181:0/1619801067 conn(0x7febcc09a9a0 msgr2=0x7febcc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:22.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 -- 172.21.15.181:0/1619801067 shutdown_connections 2023-10-16T11:20:22.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.195+0000 7febd3005700 1 -- 172.21.15.181:0/1619801067 wait complete. 2023-10-16T11:20:22.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.196+0000 7febd3005700 1 Processor -- start 2023-10-16T11:20:22.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.196+0000 7febd3005700 1 -- start start 2023-10-16T11:20:22.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd3005700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc0b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd3005700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febcc0d0180 0x7febcc09ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd3005700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc09cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd3005700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0b6280 con 0x7febcc0ce7f0 2023-10-16T11:20:22.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd3005700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc156fd0 con 0x7febcc0d0180 2023-10-16T11:20:22.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd3005700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc09f390 con 0x7febcc0d2390 2023-10-16T11:20:22.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc0b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc0b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57586/0 (socket says 172.21.15.181:57586) 2023-10-16T11:20:22.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 -- 172.21.15.181:0/1216206718 learned_addr learned my addr 172.21.15.181:0/1216206718 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:22.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd15a2700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc09cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febcbfff700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febcc0d0180 0x7febcc09ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 -- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 msgr2=0x7febcc09cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc09cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 -- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febcc0d0180 msgr2=0x7febcc09ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febcc0d0180 0x7febcc09ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.197+0000 7febd0da1700 1 -- 172.21.15.181:0/1216206718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009500 con 0x7febcc0ce7f0 2023-10-16T11:20:22.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.198+0000 7febcbfff700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febcc0d0180 0x7febcc09ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:22.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.198+0000 7febd0da1700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc0b5d40 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7febcc08bdb0 tx=0x7febc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:22.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.198+0000 7febc97fa700 1 -- 172.21.15.181:0/1216206718 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0006880 con 0x7febcc0ce7f0 2023-10-16T11:20:22.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.198+0000 7febc97fa700 1 -- 172.21.15.181:0/1216206718 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc0020070 con 0x7febcc0ce7f0 2023-10-16T11:20:22.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.198+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc09b3b0 con 0x7febcc0ce7f0 2023-10-16T11:20:22.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.198+0000 7febc97fa700 1 -- 172.21.15.181:0/1216206718 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0025670 con 0x7febcc0ce7f0 2023-10-16T11:20:22.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.198+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc09b970 con 0x7febcc0ce7f0 2023-10-16T11:20:22.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.199+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc067d20 con 0x7febcc0ce7f0 2023-10-16T11:20:22.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.199+0000 7febc97fa700 1 -- 172.21.15.181:0/1216206718 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7febc0022030 con 0x7febcc0ce7f0 2023-10-16T11:20:22.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.200+0000 7febc97fa700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7febb006b140 0x7febb006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.200+0000 7febc97fa700 1 -- 172.21.15.181:0/1216206718 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 5409+0+0 (secure 0 0 0) 0x7febc008adb0 con 0x7febcc0ce7f0 2023-10-16T11:20:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.200+0000 7febcbfff700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7febb006b140 0x7febb006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.200+0000 7febcbfff700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7febb006b140 0x7febb006d600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febcc1572f0 tx=0x7febb8008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:22.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.204+0000 7febc97fa700 1 -- 172.21.15.181:0/1216206718 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7febc0055cb0 con 0x7febcc0ce7f0 2023-10-16T11:20:22.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.473+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7febcc0cef40 con 0x7febcc0ce7f0 2023-10-16T11:20:22.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.473+0000 7febc97fa700 1 -- 172.21.15.181:0/1216206718 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v52) v1 ==== 80+0+5304 (secure 0 0 0) 0x7febc00592d0 con 0x7febcc0ce7f0 2023-10-16T11:20:22.476 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:22.476 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:20:22.476 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T11:20:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T11:20:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T11:20:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:20:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:20:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.479 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T11:20:22.479 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T11:20:22.479 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.479 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.480 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:20:22.480 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:20:22.480 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.480 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.481 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T11:20:22.481 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T11:20:22.481 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.481 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.482 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:20:22.482 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:20:22.482 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.482 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.483 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T11:20:22.483 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T11:20:22.483 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.484 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.484 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:20:22.485 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:20:22.485 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.485 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.486 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T11:20:22.486 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T11:20:22.486 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.486 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.486 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.487 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.487 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.487 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T11:20:22.488 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T11:20:22.488 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.488 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.488 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.488 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.489 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.489 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T11:20:22.489 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T11:20:22.489 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.490 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.490 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.490 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.490 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.491 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T11:20:22.491 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T11:20:22.491 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.491 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.491 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.492 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.492 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T11:20:22.492 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T11:20:22.493 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.493 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.493 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.493 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.493 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.494 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T11:20:22.494 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T11:20:22.494 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.495 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.495 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.495 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.495 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.496 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T11:20:22.496 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T11:20:22.496 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.496 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.496 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.497 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T11:20:22.498 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T11:20:22.498 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.498 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:20:22.498 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:20:22.498 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.499 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.499 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.499 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T11:20:22.499 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T11:20:22.499 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.500 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.500 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T11:20:22.500 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:20:22.500 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.501 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T11:20:22.501 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T11:20:22.501 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.501 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.501 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:20:22.502 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:20:22.502 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:20:22.502 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.503 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T11:20:22.503 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T11:20:22.503 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.503 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.503 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:20:22.504 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:20:22.504 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:20:22.504 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.505 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T11:20:22.505 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T11:20:22.505 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.505 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:20:22.505 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:20:22.506 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:20:22.506 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:20:22.506 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.507 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T11:20:22.507 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T11:20:22.507 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.507 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:20:22.508 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:20:22.508 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:20:22.508 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.508 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.509 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T11:20:22.509 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T11:20:22.509 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.509 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:20:22.509 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:20:22.510 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:22.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:22.510 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T11:20:22.511 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T11:20:22.511 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:20:22.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:20:22.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:20:22.511 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:20:22.512 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.512 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:22.512 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T11:20:22.512 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:20:22.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.475+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7febb006b140 msgr2=0x7febb006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.475+0000 7febd3005700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7febb006b140 0x7febb006d600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febcc1572f0 tx=0x7febb8008040 comp rx=0 tx=0).stop 2023-10-16T11:20:22.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.475+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 msgr2=0x7febcc0b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.475+0000 7febd3005700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc0b5d40 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7febcc08bdb0 tx=0x7febc0004750 comp rx=0 tx=0).stop 2023-10-16T11:20:22.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 shutdown_connections 2023-10-16T11:20:22.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7febb006b140 0x7febb006d600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febcc0d2390 0x7febcc09cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febcc0ce7f0 0x7febcc0b5d40 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 --2- 172.21.15.181:0/1216206718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febcc0d0180 0x7febcc09ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 >> 172.21.15.181:0/1216206718 conn(0x7febcc09a9a0 msgr2=0x7febcc06f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:22.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 shutdown_connections 2023-10-16T11:20:22.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.476+0000 7febd3005700 1 -- 172.21.15.181:0/1216206718 wait complete. 2023-10-16T11:20:22.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.635+0000 7ffa06888700 1 Processor -- start 2023-10-16T11:20:22.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.635+0000 7ffa06888700 1 -- start start 2023-10-16T11:20:22.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ffa06888700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa0015eab0 0x7ffa000d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ffa06888700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa000cb0c0 0x7ffa0015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ffa06888700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0006ed00 con 0x7ffa0015e6d0 2023-10-16T11:20:22.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ffa06888700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000d02d0 con 0x7ffa0015eab0 2023-10-16T11:20:22.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ffa06888700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0015ad40 con 0x7ffa000cb0c0 2023-10-16T11:20:22.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ff9ff7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa0015eab0 0x7ffa000d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ff9ff7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa0015eab0 0x7ffa000d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57596/0 (socket says 172.21.15.181:57596) 2023-10-16T11:20:22.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/2507102557 learned_addr learned my addr 172.21.15.181:0/2507102557 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:22.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ffa04e25700 1 --2- 172.21.15.181:0/2507102557 >> v2:172.21.15.190:3300/0 conn(0x7ffa000cb0c0 0x7ffa0015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/2507102557 >> v2:172.21.15.190:3300/0 conn(0x7ffa000cb0c0 msgr2=0x7ffa0015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:22.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ff9ff7fe700 1 --2- 172.21.15.181:0/2507102557 >> v2:172.21.15.190:3300/0 conn(0x7ffa000cb0c0 0x7ffa0015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/2507102557 >> v1:172.21.15.190:6789/0 conn(0x7ffa0015e6d0 legacy=0x7ffa0015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:22.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/2507102557 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0006e0f0 con 0x7ffa0015eab0 2023-10-16T11:20:22.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.636+0000 7ffa04e25700 1 --2- 172.21.15.181:0/2507102557 >> v2:172.21.15.190:3300/0 conn(0x7ffa000cb0c0 0x7ffa0015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:22.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ff9ff7fe700 1 --2- 172.21.15.181:0/2507102557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015eab0 0x7ffa000d0fc0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7ffa00071650 tx=0x7ff9f000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7406972ee9467fd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:22.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ff9feffd700 1 -- 172.21.15.181:0/2507102557 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000dcf0 con 0x7ffa0015eab0 2023-10-16T11:20:22.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ff9feffd700 1 -- 172.21.15.181:0/2507102557 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000de50 con 0x7ffa0015eab0 2023-10-16T11:20:22.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ff9feffd700 1 -- 172.21.15.181:0/2507102557 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004250 con 0x7ffa0015eab0 2023-10-16T11:20:22.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 -- 172.21.15.181:0/2507102557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015eab0 msgr2=0x7ffa000d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 --2- 172.21.15.181:0/2507102557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015eab0 0x7ffa000d0fc0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7ffa00071650 tx=0x7ff9f000be70 comp rx=0 tx=0).stop 2023-10-16T11:20:22.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 -- 172.21.15.181:0/2507102557 shutdown_connections 2023-10-16T11:20:22.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 --2- 172.21.15.181:0/2507102557 >> v2:172.21.15.190:3300/0 conn(0x7ffa000cb0c0 0x7ffa0015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 --2- 172.21.15.181:0/2507102557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015eab0 0x7ffa000d0fc0 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 -- 172.21.15.181:0/2507102557 >> 172.21.15.181:0/2507102557 conn(0x7ffa000d04a0 msgr2=0x7ffa000d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:22.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 -- 172.21.15.181:0/2507102557 shutdown_connections 2023-10-16T11:20:22.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.637+0000 7ffa06888700 1 -- 172.21.15.181:0/2507102557 wait complete. 2023-10-16T11:20:22.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.639+0000 7ffa06888700 1 Processor -- start 2023-10-16T11:20:22.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.639+0000 7ffa06888700 1 -- start start 2023-10-16T11:20:22.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ffa06888700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa000cb0c0 0x7ffa0015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ffa06888700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015e6d0 0x7ffa000b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ffa06888700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0015eab0 0x7ffa000b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ffa06888700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015c8e0 con 0x7ffa0015e6d0 2023-10-16T11:20:22.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ffa06888700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015bcd0 con 0x7ffa000cb0c0 2023-10-16T11:20:22.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ffa06888700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015bfb0 con 0x7ffa0015eab0 2023-10-16T11:20:22.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015e6d0 0x7ffa000b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015e6d0 0x7ffa000b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57602/0 (socket says 172.21.15.181:57602) 2023-10-16T11:20:22.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/1722117477 learned_addr learned my addr 172.21.15.181:0/1722117477 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:22.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9fffff700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa000cb0c0 0x7ffa0015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ffa04e25700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0015eab0 0x7ffa000b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0015eab0 msgr2=0x7ffa000b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0015eab0 0x7ffa000b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa000cb0c0 msgr2=0x7ffa0015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa000cb0c0 0x7ffa0015d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.640+0000 7ff9ff7fe700 1 -- 172.21.15.181:0/1722117477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f000b7f0 con 0x7ffa0015e6d0 2023-10-16T11:20:22.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.641+0000 7ff9ff7fe700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015e6d0 0x7ffa000b1e70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0000c00 tx=0x7ff9f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:22.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.641+0000 7ff9fcff9700 1 -- 172.21.15.181:0/1722117477 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0020070 con 0x7ffa0015e6d0 2023-10-16T11:20:22.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.641+0000 7ff9fcff9700 1 -- 172.21.15.181:0/1722117477 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0004030 con 0x7ffa0015e6d0 2023-10-16T11:20:22.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.641+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0015c290 con 0x7ffa0015e6d0 2023-10-16T11:20:22.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.641+0000 7ff9fcff9700 1 -- 172.21.15.181:0/1722117477 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0010470 con 0x7ffa0015e6d0 2023-10-16T11:20:22.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.641+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0015dab0 con 0x7ffa0015e6d0 2023-10-16T11:20:22.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.642+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0015dc40 con 0x7ffa0015e6d0 2023-10-16T11:20:22.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.642+0000 7ff9fcff9700 1 -- 172.21.15.181:0/1722117477 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff9f00193f0 con 0x7ffa0015e6d0 2023-10-16T11:20:22.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.643+0000 7ff9fcff9700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff9ec06b140 0x7ff9ec06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.643+0000 7ff9fcff9700 1 -- 172.21.15.181:0/1722117477 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 5409+0+0 (secure 0 0 0) 0x7ff9f008cff0 con 0x7ffa0015e6d0 2023-10-16T11:20:22.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.643+0000 7ff9fffff700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff9ec06b140 0x7ff9ec06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.643+0000 7ff9fffff700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff9ec06b140 0x7ff9ec06d600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4006fd0 tx=0x7ff9f40090e0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:22.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.647+0000 7ff9fcff9700 1 -- 172.21.15.181:0/1722117477 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff9f00bffe0 con 0x7ffa0015e6d0 2023-10-16T11:20:22.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.841+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa00067d20 con 0x7ffa0015e6d0 2023-10-16T11:20:22.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.842+0000 7ff9fcff9700 1 -- 172.21.15.181:0/1722117477 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) v1 ==== 74+0+13754 (secure 0 0 0) 0x7ff9f0057d80 con 0x7ffa0015e6d0 2023-10-16T11:20:22.846 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:22.846 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":232,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:22.054928+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:22.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff9ec06b140 msgr2=0x7ff9ec06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff9ec06b140 0x7ff9ec06d600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4006fd0 tx=0x7ff9f40090e0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015e6d0 msgr2=0x7ffa000b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015e6d0 0x7ffa000b1e70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0000c00 tx=0x7ff9f0004750 comp rx=0 tx=0).stop 2023-10-16T11:20:22.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 shutdown_connections 2023-10-16T11:20:22.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff9ec06b140 0x7ff9ec06d600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa0015eab0 0x7ffa000b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa000cb0c0 0x7ffa0015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 --2- 172.21.15.181:0/1722117477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0015e6d0 0x7ffa000b1e70 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 >> 172.21.15.181:0/1722117477 conn(0x7ffa000d04a0 msgr2=0x7ffa000cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:22.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 shutdown_connections 2023-10-16T11:20:22.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.844+0000 7ffa06888700 1 -- 172.21.15.181:0/1722117477 wait complete. 2023-10-16T11:20:22.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:20:22.859 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:20:22.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.981+0000 7f49f1adb700 1 Processor -- start 2023-10-16T11:20:22.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.981+0000 7f49f1adb700 1 -- start start 2023-10-16T11:20:22.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.982+0000 7f49f1adb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f49ec06afb0 0x7f49ec1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.982+0000 7f49f1adb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec0cef00 con 0x7f49ec072760 2023-10-16T11:20:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.982+0000 7f49f1adb700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec0cec20 con 0x7f49ec15da70 2023-10-16T11:20:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.982+0000 7f49f1adb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ec071c80 con 0x7f49ec06afb0 2023-10-16T11:20:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.982+0000 7f49eaffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f49ec072760 0x7f49ec06a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40456/0 (socket says 172.21.15.181:40456) 2023-10-16T11:20:22.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.982+0000 7f49eaffd700 1 -- 172.21.15.181:0/4281003467 learned_addr learned my addr 172.21.15.181:0/4281003467 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:22.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ebfff700 1 --2- 172.21.15.181:0/4281003467 >> v2:172.21.15.190:3300/0 conn(0x7f49ec06afb0 0x7f49ec1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448486694 0 0) 0x7f49ec0cef00 con 0x7f49ec072760 2023-10-16T11:20:22.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d40035b0 con 0x7f49ec072760 2023-10-16T11:20:22.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3109107272 0 0) 0x7f49d40035b0 con 0x7f49ec072760 2023-10-16T11:20:22.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 >> v2:172.21.15.190:3300/0 conn(0x7f49ec06afb0 msgr2=0x7f49ec1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:22.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 --2- 172.21.15.181:0/4281003467 >> v2:172.21.15.190:3300/0 conn(0x7f49ec06afb0 0x7f49ec1615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 >> v1:172.21.15.190:6789/0 conn(0x7f49ec15da70 legacy=0x7f49ec0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec071f60 con 0x7f49ec072760 2023-10-16T11:20:22.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f49dc001670 con 0x7f49ec072760 2023-10-16T11:20:22.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 >> v1:172.21.15.181:6789/0 conn(0x7f49ec072760 legacy=0x7f49ec06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d4004850 0x7f49d4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d4005330 0x7f49d400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec071c80 con 0x7f49d4003a00 2023-10-16T11:20:22.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d400c200 con 0x7f49d4004850 2023-10-16T11:20:22.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d400c3c0 con 0x7f49d4005330 2023-10-16T11:20:22.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eb7fe700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eaffd700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d4004850 0x7f49d4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ebfff700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d4005330 0x7f49d400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:22.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eb7fe700 1 -- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d4005330 msgr2=0x7f49d400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:22.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eb7fe700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d4005330 0x7f49d400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eb7fe700 1 -- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d4004850 msgr2=0x7f49d4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:22.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eb7fe700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d4004850 0x7f49d4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eb7fe700 1 -- 172.21.15.181:0/4281003467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40035b0 con 0x7f49d4003a00 2023-10-16T11:20:22.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49eaffd700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d4004850 0x7f49d4004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:22.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.983+0000 7f49ebfff700 1 -- 172.21.15.181:0/4281003467 reap_dead start 2023-10-16T11:20:22.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49eb7fe700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49d4003e20 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f49e0007fd0 tx=0x7f49d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b816354694d661 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:22.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00052d0 con 0x7f49d4003a00 2023-10-16T11:20:22.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e0005470 con 0x7f49d4003a00 2023-10-16T11:20:22.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49ea7fc700 1 -- 172.21.15.181:0/4281003467 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00038c0 con 0x7f49d4003a00 2023-10-16T11:20:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49f1adb700 1 -- 172.21.15.181:0/4281003467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 msgr2=0x7f49d4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49f1adb700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49d4003e20 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f49e0007fd0 tx=0x7f49d4005880 comp rx=0 tx=0).stop 2023-10-16T11:20:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49f1adb700 1 -- 172.21.15.181:0/4281003467 shutdown_connections 2023-10-16T11:20:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49f1adb700 1 --2- 172.21.15.181:0/4281003467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49d4003e20 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:22.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49f1adb700 1 -- 172.21.15.181:0/4281003467 >> 172.21.15.181:0/4281003467 conn(0x7f49ec0cd010 msgr2=0x7f49ec0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:22.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49f1adb700 1 -- 172.21.15.181:0/4281003467 shutdown_connections 2023-10-16T11:20:22.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.984+0000 7f49f1adb700 1 -- 172.21.15.181:0/4281003467 wait complete. 2023-10-16T11:20:22.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.985+0000 7f49f1adb700 1 Processor -- start 2023-10-16T11:20:22.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49f1adb700 1 -- start start 2023-10-16T11:20:22.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49f1adb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49ec0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49f1adb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49ec160850 0x7f49ec0c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49f1adb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49ec0c7e00 0x7f49ec0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49f1adb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0ca540 con 0x7f49d4003a00 2023-10-16T11:20:22.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49f1adb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c50f0 con 0x7f49ec160850 2023-10-16T11:20:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49f1adb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c4850 con 0x7f49ec0c7e00 2023-10-16T11:20:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49eb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49ec0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49eb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49ec0ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57620/0 (socket says 172.21.15.181:57620) 2023-10-16T11:20:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.986+0000 7f49eb7fe700 1 -- 172.21.15.181:0/3578801822 learned_addr learned my addr 172.21.15.181:0/3578801822 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49eaffd700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49ec160850 0x7f49ec0c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49ebfff700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49ec0c7e00 0x7f49ec0afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49eb7fe700 1 -- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49ec0c7e00 msgr2=0x7f49ec0afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49eb7fe700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49ec0c7e00 0x7f49ec0afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49eb7fe700 1 -- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49ec160850 msgr2=0x7f49ec0c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49eb7fe700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49ec160850 0x7f49ec0c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49eb7fe700 1 -- 172.21.15.181:0/3578801822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec0c4b30 con 0x7f49d4003a00 2023-10-16T11:20:23.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49eb7fe700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49ec0ca000 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f49e0000c00 tx=0x7f49e0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49d3fff700 1 -- 172.21.15.181:0/3578801822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0007240 con 0x7f49d4003a00 2023-10-16T11:20:23.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49d3fff700 1 -- 172.21.15.181:0/3578801822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e00073c0 con 0x7f49d4003a00 2023-10-16T11:20:23.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec0c4e10 con 0x7f49d4003a00 2023-10-16T11:20:23.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49d3fff700 1 -- 172.21.15.181:0/3578801822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0007770 con 0x7f49d4003a00 2023-10-16T11:20:23.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.987+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec0c4560 con 0x7f49d4003a00 2023-10-16T11:20:23.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.988+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec06fe90 con 0x7f49d4003a00 2023-10-16T11:20:23.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.988+0000 7f49d3fff700 1 -- 172.21.15.181:0/3578801822 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f49e0019070 con 0x7f49d4003a00 2023-10-16T11:20:23.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.989+0000 7f49d3fff700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f49d806b140 0x7f49d806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.989+0000 7f49eaffd700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f49d806b140 0x7f49d806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.989+0000 7f49d3fff700 1 -- 172.21.15.181:0/3578801822 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f49e0089e80 con 0x7f49d4003a00 2023-10-16T11:20:23.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.989+0000 7f49eaffd700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f49d806b140 0x7f49d806d600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49dc001c90 tx=0x7f49dc000ed0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:22.994+0000 7f49d3fff700 1 -- 172.21.15.181:0/3578801822 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f49e0054f70 con 0x7f49d4003a00 2023-10-16T11:20:23.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.187+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49ec15b960 con 0x7f49d4003a00 2023-10-16T11:20:23.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.188+0000 7f49d3fff700 1 -- 172.21.15.181:0/3578801822 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v232) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49e0058590 con 0x7f49d4003a00 2023-10-16T11:20:23.190 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:20:23.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f49d806b140 msgr2=0x7f49d806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f49d806b140 0x7f49d806d600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49dc001c90 tx=0x7f49dc000ed0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 msgr2=0x7f49ec0ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49ec0ca000 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f49e0000c00 tx=0x7f49e0001120 comp rx=0 tx=0).stop 2023-10-16T11:20:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 shutdown_connections 2023-10-16T11:20:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f49d806b140 0x7f49d806d600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49ec0c7e00 0x7f49ec0afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49d4003a00 0x7f49ec0ca000 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 --2- 172.21.15.181:0/3578801822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49ec160850 0x7f49ec0c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 >> 172.21.15.181:0/3578801822 conn(0x7f49ec0cd010 msgr2=0x7f49ec15be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 shutdown_connections 2023-10-16T11:20:23.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.190+0000 7f49f1adb700 1 -- 172.21.15.181:0/3578801822 wait complete. 2023-10-16T11:20:23.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:20:23.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.322+0000 7ffbecaf7700 1 Processor -- start 2023-10-16T11:20:23.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.322+0000 7ffbecaf7700 1 -- start start 2023-10-16T11:20:23.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbecaf7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 0x7ffbe80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbecaf7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe806f870 con 0x7ffbe80cd470 2023-10-16T11:20:23.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbecaf7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe80d1160 con 0x7ffbe815dfd0 2023-10-16T11:20:23.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbecaf7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80cc8e0 con 0x7ffbe80af450 2023-10-16T11:20:23.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ffbe80cd470 0x7ffbe806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40480/0 (socket says 172.21.15.181:40480) 2023-10-16T11:20:23.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe6d9d700 1 -- 172.21.15.181:0/1164175108 learned_addr learned my addr 172.21.15.181:0/1164175108 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:23.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe5d9b700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 0x7ffbe80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505035417 0 0) 0x7ffbe806f870 con 0x7ffbe80cd470 2023-10-16T11:20:23.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd00035b0 con 0x7ffbe80cd470 2023-10-16T11:20:23.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1684726441 0 0) 0x7ffbd00035b0 con 0x7ffbe80cd470 2023-10-16T11:20:23.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 msgr2=0x7ffbe80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:23.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 0x7ffbe80af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 >> v1:172.21.15.190:6789/0 conn(0x7ffbe815dfd0 legacy=0x7ffbe806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80ccbc0 con 0x7ffbe80cd470 2023-10-16T11:20:23.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ffbe00019f0 con 0x7ffbe80cd470 2023-10-16T11:20:23.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 >> v1:172.21.15.181:6789/0 conn(0x7ffbe80cd470 legacy=0x7ffbe806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 0x7ffbd0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.323+0000 7ffbe559a700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f70 0x7ffbd0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe559a700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbd0004d90 0x7ffbd0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80cc8e0 con 0x7ffbd0004d90 2023-10-16T11:20:23.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd000bb50 con 0x7ffbd0003f70 2023-10-16T11:20:23.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd000bd10 con 0x7ffbe80af450 2023-10-16T11:20:23.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe6d9d700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbd0004d90 0x7ffbd0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe659c700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 0x7ffbd0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe5d9b700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f70 0x7ffbd0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe6d9d700 1 -- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 msgr2=0x7ffbd0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:23.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe6d9d700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80af450 0x7ffbd0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe6d9d700 1 -- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f70 msgr2=0x7ffbd0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:23.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe6d9d700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f70 0x7ffbd0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe6d9d700 1 -- 172.21.15.181:0/1164175108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd00035b0 con 0x7ffbd0004d90 2023-10-16T11:20:23.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe6d9d700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbd0004d90 0x7ffbd0005210 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000a810 tx=0x7ffbd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f704ef11edc720b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000a250 con 0x7ffbd0004d90 2023-10-16T11:20:23.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe0012600 con 0x7ffbd0004d90 2023-10-16T11:20:23.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.324+0000 7ffbe559a700 1 -- 172.21.15.181:0/1164175108 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00129b0 con 0x7ffbd0004d90 2023-10-16T11:20:23.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.325+0000 7ffbecaf7700 1 -- 172.21.15.181:0/1164175108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbd0004d90 msgr2=0x7ffbd0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.325+0000 7ffbecaf7700 1 --2- 172.21.15.181:0/1164175108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbd0004d90 0x7ffbd0005210 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000a810 tx=0x7ffbd0005880 comp rx=0 tx=0).stop 2023-10-16T11:20:23.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.325+0000 7ffbe6d9d700 1 -- 172.21.15.181:0/1164175108 reap_dead start 2023-10-16T11:20:23.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.325+0000 7ffbecaf7700 1 -- 172.21.15.181:0/1164175108 shutdown_connections 2023-10-16T11:20:23.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.325+0000 7ffbecaf7700 1 -- 172.21.15.181:0/1164175108 >> 172.21.15.181:0/1164175108 conn(0x7ffbe80d04a0 msgr2=0x7ffbe80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:23.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.325+0000 7ffbecaf7700 1 -- 172.21.15.181:0/1164175108 shutdown_connections 2023-10-16T11:20:23.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.325+0000 7ffbecaf7700 1 -- 172.21.15.181:0/1164175108 wait complete. 2023-10-16T11:20:23.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.326+0000 7ffbecaf7700 1 Processor -- start 2023-10-16T11:20:23.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.326+0000 7ffbecaf7700 1 -- start start 2023-10-16T11:20:23.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbecaf7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d90 0x7ffbe80b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbecaf7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe81545c0 0x7ffbe80c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbecaf7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c9fb0 0x7ffbe80c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbecaf7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80b5330 con 0x7ffbe81545c0 2023-10-16T11:20:23.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbecaf7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80b61c0 con 0x7ffbd0004d90 2023-10-16T11:20:23.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbecaf7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80b46b0 con 0x7ffbe80c9fb0 2023-10-16T11:20:23.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe81545c0 0x7ffbe80c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe81545c0 0x7ffbe80c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57644/0 (socket says 172.21.15.181:57644) 2023-10-16T11:20:23.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 -- 172.21.15.181:0/606750784 learned_addr learned my addr 172.21.15.181:0/606750784 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:23.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe6d9d700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c9fb0 0x7ffbe80c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe659c700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d90 0x7ffbe80b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 -- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c9fb0 msgr2=0x7ffbe80c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c9fb0 0x7ffbe80c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 -- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d90 msgr2=0x7ffbe80b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d90 0x7ffbe80b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.327+0000 7ffbe5d9b700 1 -- 172.21.15.181:0/606750784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80b4990 con 0x7ffbe81545c0 2023-10-16T11:20:23.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.328+0000 7ffbe5d9b700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe81545c0 0x7ffbe80c9a70 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011aa0 tx=0x7ffbdc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.328+0000 7ffbd6ffd700 1 -- 172.21.15.181:0/606750784 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00f980 con 0x7ffbe81545c0 2023-10-16T11:20:23.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.328+0000 7ffbd6ffd700 1 -- 172.21.15.181:0/606750784 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc00fae0 con 0x7ffbe81545c0 2023-10-16T11:20:23.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.328+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe80b4c70 con 0x7ffbe81545c0 2023-10-16T11:20:23.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.328+0000 7ffbd6ffd700 1 -- 172.21.15.181:0/606750784 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00fe90 con 0x7ffbe81545c0 2023-10-16T11:20:23.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.328+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe80ca460 con 0x7ffbe81545c0 2023-10-16T11:20:23.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.329+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe8067120 con 0x7ffbe81545c0 2023-10-16T11:20:23.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.329+0000 7ffbd6ffd700 1 -- 172.21.15.181:0/606750784 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ffbdc014440 con 0x7ffbe81545c0 2023-10-16T11:20:23.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.329+0000 7ffbd6ffd700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffbc806b140 0x7ffbc806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.330+0000 7ffbd6ffd700 1 -- 172.21.15.181:0/606750784 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 5409+0+0 (secure 0 0 0) 0x7ffbdc08a030 con 0x7ffbe81545c0 2023-10-16T11:20:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.330+0000 7ffbe659c700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffbc806b140 0x7ffbc806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.330+0000 7ffbe659c700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffbc806b140 0x7ffbc806d600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.334+0000 7ffbd6ffd700 1 -- 172.21.15.181:0/606750784 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ffbdc054ff0 con 0x7ffbe81545c0 2023-10-16T11:20:23.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.528+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffbe80cafe0 con 0x7ffbe81545c0 2023-10-16T11:20:23.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.528+0000 7ffbd6ffd700 1 -- 172.21.15.181:0/606750784 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v232) v1 ==== 84+0+65 (secure 0 0 0) 0x7ffbdc058610 con 0x7ffbe81545c0 2023-10-16T11:20:23.531 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:23.531 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:20:23.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:23.531 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:20:23.532 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:20:23.532 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:23.532 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:20:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffbc806b140 msgr2=0x7ffbc806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffbc806b140 0x7ffbc806d600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8000c00 tx=0x7ffbd8009040 comp rx=0 tx=0).stop 2023-10-16T11:20:23.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe81545c0 msgr2=0x7ffbe80c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe81545c0 0x7ffbe80c9a70 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011aa0 tx=0x7ffbdc011e60 comp rx=0 tx=0).stop 2023-10-16T11:20:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 shutdown_connections 2023-10-16T11:20:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ffbc806b140 0x7ffbc806d600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c9fb0 0x7ffbe80c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d90 0x7ffbe80b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 --2- 172.21.15.181:0/606750784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe81545c0 0x7ffbe80c9a70 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 >> 172.21.15.181:0/606750784 conn(0x7ffbe80d04a0 msgr2=0x7ffbe806d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 shutdown_connections 2023-10-16T11:20:23.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.530+0000 7ffbecaf7700 1 -- 172.21.15.181:0/606750784 wait complete. 2023-10-16T11:20:23.544 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:20:23.545 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:20:23.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.669+0000 7fa6ff126700 1 Processor -- start 2023-10-16T11:20:23.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.669+0000 7fa6ff126700 1 -- start start 2023-10-16T11:20:23.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.669+0000 7fa6ff126700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa6f80ce3a0 0x7fa6f80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.669+0000 7fa6ff126700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 0x7fa6f80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.669+0000 7fa6ff126700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80ce7e0 con 0x7fa6f80afa10 2023-10-16T11:20:23.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.669+0000 7fa6ff126700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80cd880 con 0x7fa6f80ce3a0 2023-10-16T11:20:23.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.669+0000 7fa6ff126700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815b540 con 0x7fa6f80cd3f0 2023-10-16T11:20:23.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6fcec2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa6f80afa10 0x7fa6f806d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40496/0 (socket says 172.21.15.181:40496) 2023-10-16T11:20:23.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6fcec2700 1 -- 172.21.15.181:0/3725253478 learned_addr learned my addr 172.21.15.181:0/3725253478 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:23.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6fd6c3700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 0x7fa6f80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f7fff700 1 --2- 172.21.15.181:0/3725253478 >> v2:172.21.15.190:3300/0 conn(0x7fa6f80ce3a0 0x7fa6f80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342070679 0 0) 0x7fa6f80ce7e0 con 0x7fa6f80afa10 2023-10-16T11:20:23.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e80035b0 con 0x7fa6f80afa10 2023-10-16T11:20:23.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1020139428 0 0) 0x7fa6e80035b0 con 0x7fa6f80afa10 2023-10-16T11:20:23.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 msgr2=0x7fa6f80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:23.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 0x7fa6f80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.678 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 >> v2:172.21.15.190:3300/0 conn(0x7fa6f80ce3a0 msgr2=0x7fa6f80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:23.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 --2- 172.21.15.181:0/3725253478 >> v2:172.21.15.190:3300/0 conn(0x7fa6f80ce3a0 0x7fa6f80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f815b820 con 0x7fa6f80afa10 2023-10-16T11:20:23.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa6ec0017f0 con 0x7fa6f80afa10 2023-10-16T11:20:23.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 >> v1:172.21.15.181:6789/0 conn(0x7fa6f80afa10 legacy=0x7fa6f806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e8003a00 0x7fa6e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.679 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8004850 0x7fa6e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 0x7fa6e8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80cd880 con 0x7fa6e8003a00 2023-10-16T11:20:23.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815b540 con 0x7fa6e8004850 2023-10-16T11:20:23.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e800ba40 con 0x7fa6f80cd3f0 2023-10-16T11:20:23.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6fcec2700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e8003a00 0x7fa6e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6f7fff700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8004850 0x7fa6e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.670+0000 7fa6fd6c3700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 0x7fa6e8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6fcec2700 1 -- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 msgr2=0x7fa6e8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:23.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6fcec2700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80cd3f0 0x7fa6e8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6fcec2700 1 -- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8004850 msgr2=0x7fa6e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:23.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6fcec2700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8004850 0x7fa6e8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6fcec2700 1 -- 172.21.15.181:0/3725253478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80035b0 con 0x7fa6e8003a00 2023-10-16T11:20:23.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6fcec2700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e8003a00 0x7fa6e8003e20 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00c240 tx=0x7fa6e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ec7e4c5b011389 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00cde0 con 0x7fa6e8003a00 2023-10-16T11:20:23.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec007bd0 con 0x7fa6e8003a00 2023-10-16T11:20:23.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6f77fe700 1 -- 172.21.15.181:0/3725253478 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec007310 con 0x7fa6e8003a00 2023-10-16T11:20:23.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6ff126700 1 -- 172.21.15.181:0/3725253478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e8003a00 msgr2=0x7fa6e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6ff126700 1 --2- 172.21.15.181:0/3725253478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6e8003a00 0x7fa6e8003e20 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00c240 tx=0x7fa6e8005880 comp rx=0 tx=0).stop 2023-10-16T11:20:23.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6fd6c3700 1 -- 172.21.15.181:0/3725253478 reap_dead start 2023-10-16T11:20:23.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6ff126700 1 -- 172.21.15.181:0/3725253478 shutdown_connections 2023-10-16T11:20:23.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6ff126700 1 -- 172.21.15.181:0/3725253478 >> 172.21.15.181:0/3725253478 conn(0x7fa6f80d04a0 msgr2=0x7fa6f80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:23.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6ff126700 1 -- 172.21.15.181:0/3725253478 shutdown_connections 2023-10-16T11:20:23.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.671+0000 7fa6ff126700 1 -- 172.21.15.181:0/3725253478 wait complete. 2023-10-16T11:20:23.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.673+0000 7fa6ff126700 1 Processor -- start 2023-10-16T11:20:23.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.673+0000 7fa6ff126700 1 -- start start 2023-10-16T11:20:23.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8003a00 0x7fa6f80c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c68f0 0x7fa6f81f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f81f2d70 0x7fa6f81550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c9e50 con 0x7fa6f81f2d70 2023-10-16T11:20:23.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81f2120 con 0x7fa6e8003a00 2023-10-16T11:20:23.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c95b0 con 0x7fa6f80c68f0 2023-10-16T11:20:23.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f81f2d70 0x7fa6f81550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f81f2d70 0x7fa6f81550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57666/0 (socket says 172.21.15.181:57666) 2023-10-16T11:20:23.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 -- 172.21.15.181:0/361374888 learned_addr learned my addr 172.21.15.181:0/361374888 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:23.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6f7fff700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c68f0 0x7fa6f81f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 -- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c68f0 msgr2=0x7fa6f81f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c68f0 0x7fa6f81f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fcec2700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8003a00 0x7fa6f80c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 -- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8003a00 msgr2=0x7fa6f80c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:23.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8003a00 0x7fa6f80c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:23.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 -- 172.21.15.181:0/361374888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80c9890 con 0x7fa6f81f2d70 2023-10-16T11:20:23.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6fd6c3700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f81f2d70 0x7fa6f81550a0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80b6550 tx=0x7fa6f0011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6f57fa700 1 -- 172.21.15.181:0/361374888 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000e040 con 0x7fa6f81f2d70 2023-10-16T11:20:23.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6f57fa700 1 -- 172.21.15.181:0/361374888 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000f040 con 0x7fa6f81f2d70 2023-10-16T11:20:23.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f80c9b70 con 0x7fa6f81f2d70 2023-10-16T11:20:23.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6f57fa700 1 -- 172.21.15.181:0/361374888 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000fdb0 con 0x7fa6f81f2d70 2023-10-16T11:20:23.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.674+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f81555e0 con 0x7fa6f81f2d70 2023-10-16T11:20:23.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.675+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f80ccd20 con 0x7fa6f81f2d70 2023-10-16T11:20:23.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.676+0000 7fa6f57fa700 1 -- 172.21.15.181:0/361374888 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa6f0015430 con 0x7fa6f81f2d70 2023-10-16T11:20:23.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.676+0000 7fa6f57fa700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6dc06b130 0x7fa6dc06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:23.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.676+0000 7fa6f57fa700 1 -- 172.21.15.181:0/361374888 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 5409+0+0 (secure 0 0 0) 0x7fa6f008a2a0 con 0x7fa6f81f2d70 2023-10-16T11:20:23.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.676+0000 7fa6fcec2700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6dc06b130 0x7fa6dc06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:23.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.677+0000 7fa6fcec2700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6dc06b130 0x7fa6dc06d5f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec015040 tx=0x7fa6ec00a000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:23.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.680+0000 7fa6f57fa700 1 -- 172.21.15.181:0/361374888 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6f0055090 con 0x7fa6f81f2d70 2023-10-16T11:20:23.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:23.877+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa6f80727a0 con 0x7fa6f81f2d70 2023-10-16T11:20:24.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.051+0000 7fa6f57fa700 1 -- 172.21.15.181:0/361374888 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v233) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6f00586b0 con 0x7fa6f81f2d70 2023-10-16T11:20:24.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6dc06b130 msgr2=0x7fa6dc06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6dc06b130 0x7fa6dc06d5f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec015040 tx=0x7fa6ec00a000 comp rx=0 tx=0).stop 2023-10-16T11:20:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f81f2d70 msgr2=0x7fa6f81550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f81f2d70 0x7fa6f81550a0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80b6550 tx=0x7fa6f0011f80 comp rx=0 tx=0).stop 2023-10-16T11:20:24.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 shutdown_connections 2023-10-16T11:20:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6dc06b130 0x7fa6dc06d5f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6f81f2d70 0x7fa6f81550a0 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6e8003a00 0x7fa6f80c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 --2- 172.21.15.181:0/361374888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6f80c68f0 0x7fa6f81f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 >> 172.21.15.181:0/361374888 conn(0x7fa6f80d04a0 msgr2=0x7fa6f8158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 shutdown_connections 2023-10-16T11:20:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.053+0000 7fa6ff126700 1 -- 172.21.15.181:0/361374888 wait complete. 2023-10-16T11:20:24.058 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T11:20:24.073 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:20:24.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.199+0000 7f5d5814d700 1 Processor -- start 2023-10-16T11:20:24.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.200+0000 7f5d5814d700 1 -- start start 2023-10-16T11:20:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.200+0000 7f5d5814d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d500cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.200+0000 7f5d5814d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d500ce4f0 0x7f5d500b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.200+0000 7f5d5814d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500b2ae0 con 0x7f5d500b14a0 2023-10-16T11:20:24.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.200+0000 7f5d5814d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d500ce970 con 0x7f5d500ce4f0 2023-10-16T11:20:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.200+0000 7f5d5814d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b2dc0 con 0x7f5d5015f150 2023-10-16T11:20:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d500ce4f0 0x7f5d500b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d500ce4f0 0x7f5d500b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57670/0 (socket says 172.21.15.181:57670) 2023-10-16T11:20:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 -- 172.21.15.181:0/2570104069 learned_addr learned my addr 172.21.15.181:0/2570104069 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:24.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d55ee9700 1 --2- 172.21.15.181:0/2570104069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d500cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:24.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 -- 172.21.15.181:0/2570104069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 msgr2=0x7f5d500cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:24.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 --2- 172.21.15.181:0/2570104069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d500cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 -- 172.21.15.181:0/2570104069 >> v1:172.21.15.190:6789/0 conn(0x7f5d500b14a0 legacy=0x7f5d500b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:24.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 -- 172.21.15.181:0/2570104069 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500cf9d0 con 0x7f5d500ce4f0 2023-10-16T11:20:24.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d556e8700 1 --2- 172.21.15.181:0/2570104069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d500b0f60 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0098f0 tx=0x7f5d4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8799a72ad329ab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:24.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d54ee7700 1 -- 172.21.15.181:0/2570104069 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c004230 con 0x7f5d500ce4f0 2023-10-16T11:20:24.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d54ee7700 1 -- 172.21.15.181:0/2570104069 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c004390 con 0x7f5d500ce4f0 2023-10-16T11:20:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.201+0000 7f5d54ee7700 1 -- 172.21.15.181:0/2570104069 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0036a0 con 0x7f5d500ce4f0 2023-10-16T11:20:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 -- 172.21.15.181:0/2570104069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 msgr2=0x7f5d500b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 --2- 172.21.15.181:0/2570104069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d500b0f60 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0098f0 tx=0x7f5d4c009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:24.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 -- 172.21.15.181:0/2570104069 shutdown_connections 2023-10-16T11:20:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 --2- 172.21.15.181:0/2570104069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d500cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 --2- 172.21.15.181:0/2570104069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d500b0f60 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 -- 172.21.15.181:0/2570104069 >> 172.21.15.181:0/2570104069 conn(0x7f5d5009a930 msgr2=0x7f5d500d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:24.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 -- 172.21.15.181:0/2570104069 shutdown_connections 2023-10-16T11:20:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.202+0000 7f5d5814d700 1 -- 172.21.15.181:0/2570104069 wait complete. 2023-10-16T11:20:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.203+0000 7f5d5814d700 1 Processor -- start 2023-10-16T11:20:24.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.203+0000 7f5d5814d700 1 -- start start 2023-10-16T11:20:24.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d5814d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d500b14a0 0x7f5d5015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d5814d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d5009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d5814d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d501f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d5814d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5015ae40 con 0x7f5d500ce4f0 2023-10-16T11:20:24.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d5814d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500c9e90 con 0x7f5d500b14a0 2023-10-16T11:20:24.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d5814d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500c9280 con 0x7f5d5015f150 2023-10-16T11:20:24.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d5009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:24.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d5009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57678/0 (socket says 172.21.15.181:57678) 2023-10-16T11:20:24.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 -- 172.21.15.181:0/3675196191 learned_addr learned my addr 172.21.15.181:0/3675196191 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d55ee9700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d500b14a0 0x7f5d5015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 -- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 msgr2=0x7f5d501f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d566ea700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d501f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:24.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d501f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 -- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d500b14a0 msgr2=0x7f5d5015a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d500b14a0 0x7f5d5015a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 -- 172.21.15.181:0/3675196191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009550 con 0x7f5d500ce4f0 2023-10-16T11:20:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d55ee9700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d500b14a0 0x7f5d5015a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d556e8700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d5009ed00 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009840 tx=0x7f5d4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d467fc700 1 -- 172.21.15.181:0/3675196191 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c003e70 con 0x7f5d500ce4f0 2023-10-16T11:20:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d467fc700 1 -- 172.21.15.181:0/3675196191 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c01f070 con 0x7f5d500ce4f0 2023-10-16T11:20:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.204+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d500c9560 con 0x7f5d500ce4f0 2023-10-16T11:20:24.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.205+0000 7f5d467fc700 1 -- 172.21.15.181:0/3675196191 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00b8c0 con 0x7f5d500ce4f0 2023-10-16T11:20:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.205+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d500c9b20 con 0x7f5d500ce4f0 2023-10-16T11:20:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.206+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d500c9ce0 con 0x7f5d500ce4f0 2023-10-16T11:20:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.206+0000 7f5d467fc700 1 -- 172.21.15.181:0/3675196191 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5d4c021030 con 0x7f5d500ce4f0 2023-10-16T11:20:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.206+0000 7f5d467fc700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5d3406b140 0x7f5d3406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.207+0000 7f5d467fc700 1 -- 172.21.15.181:0/3675196191 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 5779+0+0 (secure 0 0 0) 0x7f5d4c08b1c0 con 0x7f5d500ce4f0 2023-10-16T11:20:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.207+0000 7f5d55ee9700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5d3406b140 0x7f5d3406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.207+0000 7f5d55ee9700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5d3406b140 0x7f5d3406d600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d40006fd0 tx=0x7f5d40008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:24.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.211+0000 7f5d467fc700 1 -- 172.21.15.181:0/3675196191 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5d4c056020 con 0x7f5d500ce4f0 2023-10-16T11:20:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:24.406+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5d500c9ce0 con 0x7f5d500ce4f0 2023-10-16T11:20:25.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.062+0000 7f5d467fc700 1 -- 172.21.15.181:0/3675196191 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v234) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5d4c059640 con 0x7f5d500ce4f0 2023-10-16T11:20:25.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.064+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5d3406b140 msgr2=0x7f5d3406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:25.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.064+0000 7f5d5814d700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5d3406b140 0x7f5d3406d600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d40006fd0 tx=0x7f5d40008040 comp rx=0 tx=0).stop 2023-10-16T11:20:25.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.064+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 msgr2=0x7f5d5009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:25.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.064+0000 7f5d5814d700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d5009ed00 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009840 tx=0x7f5d4c004750 comp rx=0 tx=0).stop 2023-10-16T11:20:25.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.064+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 shutdown_connections 2023-10-16T11:20:25.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.064+0000 7f5d5814d700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5d3406b140 0x7f5d3406d600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.064+0000 7f5d5814d700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d5015f150 0x7f5d501f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.065+0000 7f5d5814d700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d500b14a0 0x7f5d5015a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.065+0000 7f5d5814d700 1 --2- 172.21.15.181:0/3675196191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500ce4f0 0x7f5d5009ed00 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.065+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 >> 172.21.15.181:0/3675196191 conn(0x7f5d5009a930 msgr2=0x7f5d50070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:25.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.065+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 shutdown_connections 2023-10-16T11:20:25.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.065+0000 7f5d5814d700 1 -- 172.21.15.181:0/3675196191 wait complete. 2023-10-16T11:20:25.069 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T11:20:25.092 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:20:25.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.218+0000 7f1298f12700 1 Processor -- start 2023-10-16T11:20:25.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.218+0000 7f1298f12700 1 -- start start 2023-10-16T11:20:25.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1298f12700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12940cb690 0x7f129406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:25.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1298f12700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f12940b5fd0 0x7f129415f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:25.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1298f12700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1294070590 con 0x7f12940cb2b0 2023-10-16T11:20:25.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1298f12700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1294156330 con 0x7f12940cb690 2023-10-16T11:20:25.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1298f12700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129406fcf0 con 0x7f12940b5fd0 2023-10-16T11:20:25.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12940cb690 0x7f129406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:25.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12940cb690 0x7f129406f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57686/0 (socket says 172.21.15.181:57686) 2023-10-16T11:20:25.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 -- 172.21.15.181:0/1739557329 learned_addr learned my addr 172.21.15.181:0/1739557329 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:25.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1292d9d700 1 --2- 172.21.15.181:0/1739557329 >> v2:172.21.15.190:3300/0 conn(0x7f12940b5fd0 0x7f129415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:25.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 -- 172.21.15.181:0/1739557329 >> v2:172.21.15.190:3300/0 conn(0x7f12940b5fd0 msgr2=0x7f129415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:25.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 --2- 172.21.15.181:0/1739557329 >> v2:172.21.15.190:3300/0 conn(0x7f12940b5fd0 0x7f129415f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 -- 172.21.15.181:0/1739557329 >> v1:172.21.15.190:6789/0 conn(0x7f12940cb2b0 legacy=0x7f12940b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 -- 172.21.15.181:0/1739557329 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129406ffd0 con 0x7f12940cb690 2023-10-16T11:20:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f1291d9b700 1 --2- 172.21.15.181:0/1739557329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb690 0x7f129406f710 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f12940c7610 tx=0x7f1288009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c7a0f9bb17f68e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:25.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f129159a700 1 -- 172.21.15.181:0/1739557329 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800b870 con 0x7f12940cb690 2023-10-16T11:20:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f129159a700 1 -- 172.21.15.181:0/1739557329 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128800b9d0 con 0x7f12940cb690 2023-10-16T11:20:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.219+0000 7f129159a700 1 -- 172.21.15.181:0/1739557329 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800bd60 con 0x7f12940cb690 2023-10-16T11:20:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 -- 172.21.15.181:0/1739557329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb690 msgr2=0x7f129406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 --2- 172.21.15.181:0/1739557329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb690 0x7f129406f710 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f12940c7610 tx=0x7f1288009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 -- 172.21.15.181:0/1739557329 shutdown_connections 2023-10-16T11:20:25.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 --2- 172.21.15.181:0/1739557329 >> v2:172.21.15.190:3300/0 conn(0x7f12940b5fd0 0x7f129415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 --2- 172.21.15.181:0/1739557329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb690 0x7f129406f710 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 -- 172.21.15.181:0/1739557329 >> 172.21.15.181:0/1739557329 conn(0x7f12940cd010 msgr2=0x7f12940cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:25.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 -- 172.21.15.181:0/1739557329 shutdown_connections 2023-10-16T11:20:25.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.220+0000 7f1298f12700 1 -- 172.21.15.181:0/1739557329 wait complete. 2023-10-16T11:20:25.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.221+0000 7f1298f12700 1 Processor -- start 2023-10-16T11:20:25.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.221+0000 7f1298f12700 1 -- start start 2023-10-16T11:20:25.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1298f12700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12940b5fd0 0x7f12941f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:25.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1298f12700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb2b0 0x7f129415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:25.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1298f12700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12940cb690 0x7f12940b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:25.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1298f12700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129414b310 con 0x7f12940cb2b0 2023-10-16T11:20:25.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1298f12700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129415dcc0 con 0x7f12940b5fd0 2023-10-16T11:20:25.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1298f12700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12941f0b80 con 0x7f12940cb690 2023-10-16T11:20:25.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1291d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb2b0 0x7f129415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:25.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1291d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb2b0 0x7f129415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57692/0 (socket says 172.21.15.181:57692) 2023-10-16T11:20:25.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1291d9b700 1 -- 172.21.15.181:0/1169446099 learned_addr learned my addr 172.21.15.181:0/1169446099 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:25.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1292d9d700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12940cb690 0x7f12940b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:25.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f129259c700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12940b5fd0 0x7f12941f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:25.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1291d9b700 1 -- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12940cb690 msgr2=0x7f12940b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:25.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.222+0000 7f1291d9b700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12940cb690 0x7f12940b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1291d9b700 1 -- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12940b5fd0 msgr2=0x7f12941f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:25.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1291d9b700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12940b5fd0 0x7f12941f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:25.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1291d9b700 1 -- 172.21.15.181:0/1169446099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1288009550 con 0x7f12940cb2b0 2023-10-16T11:20:25.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1291d9b700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb2b0 0x7f129415e750 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f1288004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:25.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1282ffd700 1 -- 172.21.15.181:0/1169446099 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288004480 con 0x7f12940cb2b0 2023-10-16T11:20:25.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1282ffd700 1 -- 172.21.15.181:0/1169446099 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12880045e0 con 0x7f12940cb2b0 2023-10-16T11:20:25.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129415d0b0 con 0x7f12940cb2b0 2023-10-16T11:20:25.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1282ffd700 1 -- 172.21.15.181:0/1169446099 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880038e0 con 0x7f12940cb2b0 2023-10-16T11:20:25.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.223+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129415d670 con 0x7f12940cb2b0 2023-10-16T11:20:25.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.224+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129406f4f0 con 0x7f12940cb2b0 2023-10-16T11:20:25.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.225+0000 7f1282ffd700 1 -- 172.21.15.181:0/1169446099 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1288021070 con 0x7f12940cb2b0 2023-10-16T11:20:25.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.225+0000 7f1282ffd700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f127406b140 0x7f127406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:25.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.226+0000 7f129259c700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f127406b140 0x7f127406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:25.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.226+0000 7f1282ffd700 1 -- 172.21.15.181:0/1169446099 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f128808b750 con 0x7f12940cb2b0 2023-10-16T11:20:25.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.226+0000 7f129259c700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f127406b140 0x7f127406d600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1284007fd0 tx=0x7f1284009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:25.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.229+0000 7f1282ffd700 1 -- 172.21.15.181:0/1169446099 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1288056480 con 0x7f12940cb2b0 2023-10-16T11:20:25.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:25.466+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f12940cda80 con 0x7f12940cb2b0 2023-10-16T11:20:26.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.074+0000 7f1282ffd700 1 -- 172.21.15.181:0/1169446099 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 30 and data pool 31 v124) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1288059aa0 con 0x7f12940cb2b0 2023-10-16T11:20:26.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.082+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f127406b140 msgr2=0x7f127406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.082+0000 7f1298f12700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f127406b140 0x7f127406d600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1284007fd0 tx=0x7f1284009040 comp rx=0 tx=0).stop 2023-10-16T11:20:26.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.082+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb2b0 msgr2=0x7f129415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.082+0000 7f1298f12700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb2b0 0x7f129415e750 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f1288004750 comp rx=0 tx=0).stop 2023-10-16T11:20:26.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.083+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 shutdown_connections 2023-10-16T11:20:26.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.083+0000 7f1298f12700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f127406b140 0x7f127406d600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.083+0000 7f1298f12700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12940cb690 0x7f12940b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.083+0000 7f1298f12700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12940b5fd0 0x7f12941f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.083+0000 7f1298f12700 1 --2- 172.21.15.181:0/1169446099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12940cb2b0 0x7f129415e750 secure :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f1288004750 comp rx=0 tx=0).stop 2023-10-16T11:20:26.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.083+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 >> 172.21.15.181:0/1169446099 conn(0x7f12940cd010 msgr2=0x7f129415f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:26.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.086+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 shutdown_connections 2023-10-16T11:20:26.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.086+0000 7f1298f12700 1 -- 172.21.15.181:0/1169446099 wait complete. 2023-10-16T11:20:26.092 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 30 and data pool 31 2023-10-16T11:20:26.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:26.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.234+0000 7f1a56349700 1 Processor -- start 2023-10-16T11:20:26.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a56349700 1 -- start start 2023-10-16T11:20:26.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a56349700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500ce760 0x7f1a500cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a56349700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a500cfe70 0x7f1a500d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a56349700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a500cc3e0 con 0x7f1a5015f1e0 2023-10-16T11:20:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a56349700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5015fc90 con 0x7f1a500cfe70 2023-10-16T11:20:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a56349700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5015ff70 con 0x7f1a500ce760 2023-10-16T11:20:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a4ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1a5015f1e0 0x7f1a500ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40556/0 (socket says 172.21.15.181:40556) 2023-10-16T11:20:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.235+0000 7f1a4ffff700 1 -- 172.21.15.181:0/2852945845 learned_addr learned my addr 172.21.15.181:0/2852945845 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a548e6700 1 --2- 172.21.15.181:0/2852945845 >> v2:172.21.15.181:3300/0 conn(0x7f1a500cfe70 0x7f1a500d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a4f7fe700 1 --2- 172.21.15.181:0/2852945845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500ce760 0x7f1a500cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a4effd700 1 -- 172.21.15.181:0/2852945845 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804602256 0 0) 0x7f1a38001110 con 0x7f1a5015f1e0 2023-10-16T11:20:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a4effd700 1 -- 172.21.15.181:0/2852945845 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a3c0035b0 con 0x7f1a5015f1e0 2023-10-16T11:20:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a548e6700 1 -- 172.21.15.181:0/2852945845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500ce760 msgr2=0x7f1a500cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a548e6700 1 --2- 172.21.15.181:0/2852945845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500ce760 0x7f1a500cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a548e6700 1 -- 172.21.15.181:0/2852945845 >> v1:172.21.15.181:6789/0 conn(0x7f1a5015f1e0 legacy=0x7f1a500ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a548e6700 1 -- 172.21.15.181:0/2852945845 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50160250 con 0x7f1a500cfe70 2023-10-16T11:20:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a548e6700 1 --2- 172.21.15.181:0/2852945845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500cfe70 0x7f1a500d02a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009d90 tx=0x7f1a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b62211c216823c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a4effd700 1 -- 172.21.15.181:0/2852945845 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800e070 con 0x7f1a500cfe70 2023-10-16T11:20:26.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a4effd700 1 -- 172.21.15.181:0/2852945845 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4800bb20 con 0x7f1a500cfe70 2023-10-16T11:20:26.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.236+0000 7f1a4effd700 1 -- 172.21.15.181:0/2852945845 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48012430 con 0x7f1a500cfe70 2023-10-16T11:20:26.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 -- 172.21.15.181:0/2852945845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500cfe70 msgr2=0x7f1a500d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 --2- 172.21.15.181:0/2852945845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500cfe70 0x7f1a500d02a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009d90 tx=0x7f1a4800b330 comp rx=0 tx=0).stop 2023-10-16T11:20:26.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 -- 172.21.15.181:0/2852945845 shutdown_connections 2023-10-16T11:20:26.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 --2- 172.21.15.181:0/2852945845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500cfe70 0x7f1a500d02a0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 --2- 172.21.15.181:0/2852945845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500ce760 0x7f1a500cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 -- 172.21.15.181:0/2852945845 >> 172.21.15.181:0/2852945845 conn(0x7f1a500665a0 msgr2=0x7f1a50066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:26.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 -- 172.21.15.181:0/2852945845 shutdown_connections 2023-10-16T11:20:26.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.237+0000 7f1a56349700 1 -- 172.21.15.181:0/2852945845 wait complete. 2023-10-16T11:20:26.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.238+0000 7f1a56349700 1 Processor -- start 2023-10-16T11:20:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.238+0000 7f1a56349700 1 -- start start 2023-10-16T11:20:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a56349700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500ce760 0x7f1a500aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a56349700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500cfe70 0x7f1a500ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a56349700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a5015f1e0 0x7f1a500b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a56349700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500a93d0 con 0x7f1a500ce760 2023-10-16T11:20:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a56349700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500a96b0 con 0x7f1a500cfe70 2023-10-16T11:20:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a56349700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500a9990 con 0x7f1a5015f1e0 2023-10-16T11:20:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500ce760 0x7f1a500aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500ce760 0x7f1a500aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57716/0 (socket says 172.21.15.181:57716) 2023-10-16T11:20:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 -- 172.21.15.181:0/1895314089 learned_addr learned my addr 172.21.15.181:0/1895314089 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 -- 172.21.15.181:0/1895314089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a5015f1e0 msgr2=0x7f1a500b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a5015f1e0 0x7f1a500b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 -- 172.21.15.181:0/1895314089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500cfe70 msgr2=0x7f1a500ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500cfe70 0x7f1a500ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 -- 172.21.15.181:0/1895314089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a48009a70 con 0x7f1a500ce760 2023-10-16T11:20:26.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4ffff700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500ce760 0x7f1a500aa470 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f1a38007fd0 tx=0x7f1a3800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4cff9700 1 -- 172.21.15.181:0/1895314089 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38005d70 con 0x7f1a500ce760 2023-10-16T11:20:26.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4cff9700 1 -- 172.21.15.181:0/1895314089 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a380070d0 con 0x7f1a500ce760 2023-10-16T11:20:26.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a500a9c70 con 0x7f1a500ce760 2023-10-16T11:20:26.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.239+0000 7f1a4cff9700 1 -- 172.21.15.181:0/1895314089 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38007460 con 0x7f1a500ce760 2023-10-16T11:20:26.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.240+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a500ab820 con 0x7f1a500ce760 2023-10-16T11:20:26.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.240+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a500ab9e0 con 0x7f1a500ce760 2023-10-16T11:20:26.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.241+0000 7f1a4cff9700 1 -- 172.21.15.181:0/1895314089 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1a380075c0 con 0x7f1a500ce760 2023-10-16T11:20:26.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.241+0000 7f1a4cff9700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f1a3006b140 0x7f1a3006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.241+0000 7f1a4cff9700 1 -- 172.21.15.181:0/1895314089 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1a38088f50 con 0x7f1a500ce760 2023-10-16T11:20:26.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.241+0000 7f1a4f7fe700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f1a3006b140 0x7f1a3006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.242+0000 7f1a4f7fe700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f1a3006b140 0x7f1a3006d600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a500b3070 tx=0x7f1a40008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.246+0000 7f1a4cff9700 1 -- 172.21.15.181:0/1895314089 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a38053b00 con 0x7f1a500ce760 2023-10-16T11:20:26.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.437+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a500aa190 con 0x7f1a500ce760 2023-10-16T11:20:26.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.438+0000 7f1a4cff9700 1 -- 172.21.15.181:0/1895314089 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) v1 ==== 74+0+17419 (secure 0 0 0) 0x7f1a500aa190 con 0x7f1a500ce760 2023-10-16T11:20:26.441 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:26.441 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":235,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:25.467963+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:26.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f1a3006b140 msgr2=0x7f1a3006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f1a3006b140 0x7f1a3006d600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a500b3070 tx=0x7f1a40008040 comp rx=0 tx=0).stop 2023-10-16T11:20:26.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500ce760 msgr2=0x7f1a500aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500ce760 0x7f1a500aa470 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f1a38007fd0 tx=0x7f1a3800c860 comp rx=0 tx=0).stop 2023-10-16T11:20:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 shutdown_connections 2023-10-16T11:20:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f1a3006b140 0x7f1a3006d600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a5015f1e0 0x7f1a500b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a500ce760 0x7f1a500aa470 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 --2- 172.21.15.181:0/1895314089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500cfe70 0x7f1a500ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 >> 172.21.15.181:0/1895314089 conn(0x7f1a500665a0 msgr2=0x7f1a50066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 shutdown_connections 2023-10-16T11:20:26.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.441+0000 7f1a56349700 1 -- 172.21.15.181:0/1895314089 wait complete. 2023-10-16T11:20:26.455 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:26.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.573+0000 7f9cfe3a2700 1 Processor -- start 2023-10-16T11:20:26.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.573+0000 7f9cfe3a2700 1 -- start start 2023-10-16T11:20:26.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfe3a2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cf815e6d0 0x7f9cf80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfe3a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815eab0 0x7f9cf815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfe3a2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9cf80cb0c0 0x7f9cf815a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfe3a2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf806ed00 con 0x7f9cf80cb0c0 2023-10-16T11:20:26.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfe3a2700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80d02d0 con 0x7f9cf815e6d0 2023-10-16T11:20:26.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfe3a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf806e0f0 con 0x7f9cf815eab0 2023-10-16T11:20:26.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfc93f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9cf80cb0c0 0x7f9cf815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfc93f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9cf80cb0c0 0x7f9cf815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57728/0 (socket says 172.21.15.181:57728) 2023-10-16T11:20:26.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cfc93f700 1 -- 172.21.15.181:0/4123534310 learned_addr learned my addr 172.21.15.181:0/4123534310 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:26.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cf77fe700 1 --2- 172.21.15.181:0/4123534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815eab0 0x7f9cf815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.574+0000 7f9cf7fff700 1 --2- 172.21.15.181:0/4123534310 >> v2:172.21.15.190:3300/0 conn(0x7f9cf815e6d0 0x7f9cf80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfc93f700 1 -- 172.21.15.181:0/4123534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815eab0 msgr2=0x7f9cf815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:26.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfc93f700 1 --2- 172.21.15.181:0/4123534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815eab0 0x7f9cf815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfc93f700 1 -- 172.21.15.181:0/4123534310 >> v2:172.21.15.190:3300/0 conn(0x7f9cf815e6d0 msgr2=0x7f9cf80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:26.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfc93f700 1 --2- 172.21.15.181:0/4123534310 >> v2:172.21.15.190:3300/0 conn(0x7f9cf815e6d0 0x7f9cf80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfc93f700 1 -- 172.21.15.181:0/4123534310 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf806e3d0 con 0x7f9cf80cb0c0 2023-10-16T11:20:26.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfc93f700 1 --2- 172.21.15.181:0/4123534310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf80cb0c0 0x7f9cf815a630 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80717b0 tx=0x7f9cf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978cf507f024534f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cf6ffd700 1 -- 172.21.15.181:0/4123534310 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000e070 con 0x7f9cf80cb0c0 2023-10-16T11:20:26.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cf6ffd700 1 -- 172.21.15.181:0/4123534310 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000b910 con 0x7f9cf80cb0c0 2023-10-16T11:20:26.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cf6ffd700 1 -- 172.21.15.181:0/4123534310 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000bcc0 con 0x7f9cf80cb0c0 2023-10-16T11:20:26.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/4123534310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf80cb0c0 msgr2=0x7f9cf815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/4123534310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf80cb0c0 0x7f9cf815a630 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80717b0 tx=0x7f9cf000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:26.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/4123534310 shutdown_connections 2023-10-16T11:20:26.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/4123534310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf80cb0c0 0x7f9cf815a630 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/4123534310 >> v2:172.21.15.190:3300/0 conn(0x7f9cf815e6d0 0x7f9cf80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/4123534310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815eab0 0x7f9cf815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/4123534310 >> 172.21.15.181:0/4123534310 conn(0x7f9cf80d04a0 msgr2=0x7f9cf80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:26.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.575+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/4123534310 shutdown_connections 2023-10-16T11:20:26.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.576+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/4123534310 wait complete. 2023-10-16T11:20:26.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.577+0000 7f9cfe3a2700 1 Processor -- start 2023-10-16T11:20:26.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.577+0000 7f9cfe3a2700 1 -- start start 2023-10-16T11:20:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.577+0000 7f9cfe3a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cf80cb0c0 0x7f9cf8152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.577+0000 7f9cfe3a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815e6d0 0x7f9cf80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.577+0000 7f9cfe3a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf815eab0 0x7f9cf815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfe3a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80b4490 con 0x7f9cf815eab0 2023-10-16T11:20:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfe3a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8153030 con 0x7f9cf80cb0c0 2023-10-16T11:20:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfe3a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80c7710 con 0x7f9cf815e6d0 2023-10-16T11:20:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf815eab0 0x7f9cf815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf815eab0 0x7f9cf815b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57734/0 (socket says 172.21.15.181:57734) 2023-10-16T11:20:26.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 -- 172.21.15.181:0/1193451572 learned_addr learned my addr 172.21.15.181:0/1193451572 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:26.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cf77fe700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815e6d0 0x7f9cf80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 -- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815e6d0 msgr2=0x7f9cf80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815e6d0 0x7f9cf80b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 -- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cf80cb0c0 msgr2=0x7f9cf8152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cf7fff700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cf80cb0c0 0x7f9cf8152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cf80cb0c0 0x7f9cf8152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 -- 172.21.15.181:0/1193451572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009b30 con 0x7f9cf815eab0 2023-10-16T11:20:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfc93f700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf815eab0 0x7f9cf815b8e0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8071580 tx=0x7f9cf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cf4ff9700 1 -- 172.21.15.181:0/1193451572 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0014a10 con 0x7f9cf815eab0 2023-10-16T11:20:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cf4ff9700 1 -- 172.21.15.181:0/1193451572 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf0014b90 con 0x7f9cf815eab0 2023-10-16T11:20:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf80b3bf0 con 0x7f9cf815eab0 2023-10-16T11:20:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cf4ff9700 1 -- 172.21.15.181:0/1193451572 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000ce80 con 0x7f9cf815eab0 2023-10-16T11:20:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.578+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf80b41b0 con 0x7f9cf815eab0 2023-10-16T11:20:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.579+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf80c7570 con 0x7f9cf815eab0 2023-10-16T11:20:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.580+0000 7f9cf4ff9700 1 -- 172.21.15.181:0/1193451572 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9cf0004040 con 0x7f9cf815eab0 2023-10-16T11:20:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.580+0000 7f9cf4ff9700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ce406b140 0x7f9ce406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.580+0000 7f9cf7fff700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ce406b140 0x7f9ce406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.580+0000 7f9cf4ff9700 1 -- 172.21.15.181:0/1193451572 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9cf008d820 con 0x7f9cf815eab0 2023-10-16T11:20:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.581+0000 7f9cf7fff700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ce406b140 0x7f9ce406d600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009a90 tx=0x7f9cec008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.585+0000 7f9cf4ff9700 1 -- 172.21.15.181:0/1193451572 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9cf80c7570 con 0x7f9cf815eab0 2023-10-16T11:20:26.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.779+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cf80c7570 con 0x7f9cf815eab0 2023-10-16T11:20:26.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.780+0000 7f9cf4ff9700 1 -- 172.21.15.181:0/1193451572 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) v1 ==== 74+0+17419 (secure 0 0 0) 0x7f9cf80c7570 con 0x7f9cf815eab0 2023-10-16T11:20:26.783 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:26.783 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":235,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:25.467963+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:26.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ce406b140 msgr2=0x7f9ce406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ce406b140 0x7f9ce406d600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009a90 tx=0x7f9cec008040 comp rx=0 tx=0).stop 2023-10-16T11:20:26.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf815eab0 msgr2=0x7f9cf815b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf815eab0 0x7f9cf815b8e0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8071580 tx=0x7f9cf0004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 shutdown_connections 2023-10-16T11:20:26.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ce406b140 0x7f9ce406d600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf815eab0 0x7f9cf815b8e0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cf80cb0c0 0x7f9cf8152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 --2- 172.21.15.181:0/1193451572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cf815e6d0 0x7f9cf80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 >> 172.21.15.181:0/1193451572 conn(0x7f9cf80d04a0 msgr2=0x7f9cf815c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:26.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 shutdown_connections 2023-10-16T11:20:26.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.783+0000 7f9cfe3a2700 1 -- 172.21.15.181:0/1193451572 wait complete. 2023-10-16T11:20:26.798 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:20:26.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.920+0000 7fca684f9700 1 Processor -- start 2023-10-16T11:20:26.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.920+0000 7fca684f9700 1 -- start start 2023-10-16T11:20:26.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca684f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6015da70 0x7fca600ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca684f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca600cef00 con 0x7fca600ce6a0 2023-10-16T11:20:26.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca684f9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca600cec20 con 0x7fca6006afb0 2023-10-16T11:20:26.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca684f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca60071c80 con 0x7fca6015da70 2023-10-16T11:20:26.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca65a94700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fca600ce6a0 0x7fca6006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40584/0 (socket says 172.21.15.181:40584) 2023-10-16T11:20:26.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca65a94700 1 -- 172.21.15.181:0/607570695 learned_addr learned my addr 172.21.15.181:0/607570695 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:26.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca66295700 1 --2- 172.21.15.181:0/607570695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6015da70 0x7fca600ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871275423 0 0) 0x7fca600cef00 con 0x7fca600ce6a0 2023-10-16T11:20:26.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca4c0035b0 con 0x7fca600ce6a0 2023-10-16T11:20:26.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499345705 0 0) 0x7fca600cec20 con 0x7fca6006afb0 2023-10-16T11:20:26.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca600cef00 con 0x7fca6006afb0 2023-10-16T11:20:26.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2214257965 0 0) 0x7fca4c0035b0 con 0x7fca600ce6a0 2023-10-16T11:20:26.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca66295700 1 -- 172.21.15.181:0/607570695 >> v1:172.21.15.190:6789/0 conn(0x7fca6006afb0 legacy=0x7fca601615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca66295700 1 -- 172.21.15.181:0/607570695 >> v1:172.21.15.181:6789/0 conn(0x7fca600ce6a0 legacy=0x7fca6006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca66295700 1 -- 172.21.15.181:0/607570695 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60071f60 con 0x7fca6015da70 2023-10-16T11:20:26.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.921+0000 7fca66295700 1 --2- 172.21.15.181:0/607570695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6015da70 0x7fca600ce160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009870 tx=0x7fca5c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bbd3769d80ea757 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50001670 con 0x7fca6015da70 2023-10-16T11:20:26.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c004030 con 0x7fca6015da70 2023-10-16T11:20:26.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca65293700 1 -- 172.21.15.181:0/607570695 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0043a0 con 0x7fca6015da70 2023-10-16T11:20:26.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca684f9700 1 -- 172.21.15.181:0/607570695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6015da70 msgr2=0x7fca600ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca684f9700 1 --2- 172.21.15.181:0/607570695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6015da70 0x7fca600ce160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009870 tx=0x7fca5c009b80 comp rx=0 tx=0).stop 2023-10-16T11:20:26.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca684f9700 1 -- 172.21.15.181:0/607570695 shutdown_connections 2023-10-16T11:20:26.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca684f9700 1 --2- 172.21.15.181:0/607570695 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6015da70 0x7fca600ce160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca684f9700 1 -- 172.21.15.181:0/607570695 >> 172.21.15.181:0/607570695 conn(0x7fca600cd010 msgr2=0x7fca600cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:26.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca684f9700 1 -- 172.21.15.181:0/607570695 shutdown_connections 2023-10-16T11:20:26.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.922+0000 7fca684f9700 1 -- 172.21.15.181:0/607570695 wait complete. 2023-10-16T11:20:26.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.923+0000 7fca684f9700 1 Processor -- start 2023-10-16T11:20:26.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.923+0000 7fca684f9700 1 -- start start 2023-10-16T11:20:26.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca684f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6006afb0 0x7fca600ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca684f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca600ce6a0 0x7fca600b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca684f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6015da70 0x7fca600c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca684f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca600c40c0 con 0x7fca6015da70 2023-10-16T11:20:26.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca684f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca600c6a30 con 0x7fca600ce6a0 2023-10-16T11:20:26.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca684f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca600c6cb0 con 0x7fca6006afb0 2023-10-16T11:20:26.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca66a96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6015da70 0x7fca600c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca66a96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6015da70 0x7fca600c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57748/0 (socket says 172.21.15.181:57748) 2023-10-16T11:20:26.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca66a96700 1 -- 172.21.15.181:0/1678833072 learned_addr learned my addr 172.21.15.181:0/1678833072 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca66295700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6006afb0 0x7fca600ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.924+0000 7fca65a94700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca600ce6a0 0x7fca600b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca66a96700 1 -- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6006afb0 msgr2=0x7fca600ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca66a96700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6006afb0 0x7fca600ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca66a96700 1 -- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca600ce6a0 msgr2=0x7fca600b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca66a96700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca600ce6a0 0x7fca600b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca66a96700 1 -- 172.21.15.181:0/1678833072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0094a0 con 0x7fca6015da70 2023-10-16T11:20:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca66a96700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6015da70 0x7fca600c64f0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fca5800cb80 tx=0x7fca5800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca56ffd700 1 -- 172.21.15.181:0/1678833072 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca58003c00 con 0x7fca6015da70 2023-10-16T11:20:26.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca56ffd700 1 -- 172.21.15.181:0/1678833072 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca58003d60 con 0x7fca6015da70 2023-10-16T11:20:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca600c6f90 con 0x7fca6015da70 2023-10-16T11:20:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca56ffd700 1 -- 172.21.15.181:0/1678833072 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800aaa0 con 0x7fca6015da70 2023-10-16T11:20:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.925+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca600c45f0 con 0x7fca6015da70 2023-10-16T11:20:26.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.926+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca600c3880 con 0x7fca6015da70 2023-10-16T11:20:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.926+0000 7fca56ffd700 1 -- 172.21.15.181:0/1678833072 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fca5800ac00 con 0x7fca6015da70 2023-10-16T11:20:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.927+0000 7fca56ffd700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca4406b140 0x7fca4406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.927+0000 7fca56ffd700 1 -- 172.21.15.181:0/1678833072 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fca58089bb0 con 0x7fca6015da70 2023-10-16T11:20:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.927+0000 7fca66295700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca4406b140 0x7fca4406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:26.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.927+0000 7fca66295700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca4406b140 0x7fca4406d600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0097c0 tx=0x7fca5c008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:26.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:26.931+0000 7fca56ffd700 1 -- 172.21.15.181:0/1678833072 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fca580546e0 con 0x7fca6015da70 2023-10-16T11:20:27.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:27.173+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fca6006f6c0 con 0x7fca6015da70 2023-10-16T11:20:27.595 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:27.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:28.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.090+0000 7fca56ffd700 1 -- 172.21.15.181:0/1678833072 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v127) v1 ==== 118+0+0 (secure 0 0 0) 0x7fca58057d00 con 0x7fca6015da70 2023-10-16T11:20:28.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca4406b140 msgr2=0x7fca4406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:28.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca4406b140 0x7fca4406d600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0097c0 tx=0x7fca5c008030 comp rx=0 tx=0).stop 2023-10-16T11:20:28.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6015da70 msgr2=0x7fca600c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:28.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6015da70 0x7fca600c64f0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fca5800cb80 tx=0x7fca5800ce90 comp rx=0 tx=0).stop 2023-10-16T11:20:28.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 shutdown_connections 2023-10-16T11:20:28.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca4406b140 0x7fca4406d600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca6006afb0 0x7fca600ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca6015da70 0x7fca600c64f0 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 --2- 172.21.15.181:0/1678833072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca600ce6a0 0x7fca600b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 >> 172.21.15.181:0/1678833072 conn(0x7fca600cd010 msgr2=0x7fca6006f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:28.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 shutdown_connections 2023-10-16T11:20:28.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.092+0000 7fca684f9700 1 -- 172.21.15.181:0/1678833072 wait complete. 2023-10-16T11:20:28.108 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:20:28.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:20:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.233+0000 7f0400c94700 1 Processor -- start 2023-10-16T11:20:28.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.233+0000 7f0400c94700 1 -- start start 2023-10-16T11:20:28.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f0400c94700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 0x7f03fc0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f0400c94700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f03fc071be0 0x7f03fc0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f0400c94700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc0ce1f0 con 0x7f03fc06f3b0 2023-10-16T11:20:28.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f0400c94700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03fc0b2930 con 0x7f03fc071be0 2023-10-16T11:20:28.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f0400c94700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc0ccda0 con 0x7f03fc0b2010 2023-10-16T11:20:28.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f03fad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f03fc06f3b0 0x7f03fc06f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40602/0 (socket says 172.21.15.181:40602) 2023-10-16T11:20:28.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f03fad9d700 1 -- 172.21.15.181:0/201441584 learned_addr learned my addr 172.21.15.181:0/201441584 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:28.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109974031 0 0) 0x7f03fc0ce1f0 con 0x7f03fc06f3b0 2023-10-16T11:20:28.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f03fa59c700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 0x7f03fc0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e40035b0 con 0x7f03fc06f3b0 2023-10-16T11:20:28.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.234+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/201441584 >> v2:172.21.15.190:3300/0 conn(0x7f03fc071be0 0x7f03fc0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365069701 0 0) 0x7f03e40035b0 con 0x7f03fc06f3b0 2023-10-16T11:20:28.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 msgr2=0x7f03fc0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:28.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 0x7f03fc0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 >> v2:172.21.15.190:3300/0 conn(0x7f03fc071be0 msgr2=0x7f03fc0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:28.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 --2- 172.21.15.181:0/201441584 >> v2:172.21.15.190:3300/0 conn(0x7f03fc071be0 0x7f03fc0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0727e0 con 0x7f03fc06f3b0 2023-10-16T11:20:28.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f03f40019f0 con 0x7f03fc06f3b0 2023-10-16T11:20:28.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 >> v1:172.21.15.181:6789/0 conn(0x7f03fc06f3b0 legacy=0x7f03fc06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:28.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e4003a00 0x7f03e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 0x7f03e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc0b2930 con 0x7f03e4004850 2023-10-16T11:20:28.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc0ccda0 con 0x7f03e4003a00 2023-10-16T11:20:28.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e400ba10 con 0x7f03fc0b2010 2023-10-16T11:20:28.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03fa59c700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e4003a00 0x7f03e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03fad9d700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 0x7f03e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f9d9b700 1 -- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 msgr2=0x7f03e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:28.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b2010 0x7f03e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f9d9b700 1 -- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e4003a00 msgr2=0x7f03e4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:28.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e4003a00 0x7f03e4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f9d9b700 1 -- 172.21.15.181:0/201441584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40035b0 con 0x7f03e4004850 2023-10-16T11:20:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03fa59c700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03e4003a00 0x7f03e4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03e4004cd0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009b60 tx=0x7f03e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6c651a2d154f3d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00e6c0 con 0x7f03e4004850 2023-10-16T11:20:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.235+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec00e820 con 0x7f03e4004850 2023-10-16T11:20:28.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f03f959a700 1 -- 172.21.15.181:0/201441584 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00eb90 con 0x7f03e4004850 2023-10-16T11:20:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f0400c94700 1 -- 172.21.15.181:0/201441584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 msgr2=0x7f03e4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f0400c94700 1 --2- 172.21.15.181:0/201441584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03e4004cd0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009b60 tx=0x7f03e4005880 comp rx=0 tx=0).stop 2023-10-16T11:20:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f03fad9d700 1 -- 172.21.15.181:0/201441584 reap_dead start 2023-10-16T11:20:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f0400c94700 1 -- 172.21.15.181:0/201441584 shutdown_connections 2023-10-16T11:20:28.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f0400c94700 1 -- 172.21.15.181:0/201441584 >> 172.21.15.181:0/201441584 conn(0x7f03fc0cd010 msgr2=0x7f03fc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f0400c94700 1 -- 172.21.15.181:0/201441584 shutdown_connections 2023-10-16T11:20:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.236+0000 7f0400c94700 1 -- 172.21.15.181:0/201441584 wait complete. 2023-10-16T11:20:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.237+0000 7f0400c94700 1 Processor -- start 2023-10-16T11:20:28.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f0400c94700 1 -- start start 2023-10-16T11:20:28.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f0400c94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03fc14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f0400c94700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0c98a0 0x7f03fc0c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f0400c94700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b40e0 0x7f03fc0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f0400c94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc0b4ab0 con 0x7f03e4004850 2023-10-16T11:20:28.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f0400c94700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc0c89e0 con 0x7f03fc0c98a0 2023-10-16T11:20:28.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f0400c94700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1507a0 con 0x7f03fc0b40e0 2023-10-16T11:20:28.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f03fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03fc14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f03fa59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03fc14f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57764/0 (socket says 172.21.15.181:57764) 2023-10-16T11:20:28.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f03fa59c700 1 -- 172.21.15.181:0/341296712 learned_addr learned my addr 172.21.15.181:0/341296712 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:28.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f03fad9d700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b40e0 0x7f03fc0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.238+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0c98a0 0x7f03fc0c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03fa59c700 1 -- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b40e0 msgr2=0x7f03fc0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:28.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03fa59c700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b40e0 0x7f03fc0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03fa59c700 1 -- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0c98a0 msgr2=0x7f03fc0c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:28.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03fa59c700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0c98a0 0x7f03fc0c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:28.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03fa59c700 1 -- 172.21.15.181:0/341296712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc150a80 con 0x7f03e4004850 2023-10-16T11:20:28.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03fa59c700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03fc14f540 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f03f0011d00 tx=0x7f03f000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:28.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03eaffd700 1 -- 172.21.15.181:0/341296712 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f000fa80 con 0x7f03e4004850 2023-10-16T11:20:28.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03eaffd700 1 -- 172.21.15.181:0/341296712 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f000fbe0 con 0x7f03e4004850 2023-10-16T11:20:28.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f03eaffd700 1 -- 172.21.15.181:0/341296712 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0004020 con 0x7f03e4004850 2023-10-16T11:20:28.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc150d60 con 0x7f03e4004850 2023-10-16T11:20:28.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.239+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc14fa80 con 0x7f03e4004850 2023-10-16T11:20:28.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.240+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc0b6510 con 0x7f03e4004850 2023-10-16T11:20:28.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.241+0000 7f03eaffd700 1 -- 172.21.15.181:0/341296712 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f03f0004180 con 0x7f03e4004850 2023-10-16T11:20:28.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.241+0000 7f03eaffd700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f03dc06b140 0x7f03dc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:28.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.241+0000 7f03eaffd700 1 -- 172.21.15.181:0/341296712 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f03f0089fb0 con 0x7f03e4004850 2023-10-16T11:20:28.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.241+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f03dc06b140 0x7f03dc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:28.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.241+0000 7f03f9d9b700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f03dc06b140 0x7f03dc06d600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0090b0 tx=0x7f03ec008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:28.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.246+0000 7f03eaffd700 1 -- 172.21.15.181:0/341296712 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f03f00bcfe0 con 0x7f03e4004850 2023-10-16T11:20:28.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:28.482+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f03fc0b1cb0 con 0x7f03e4004850 2023-10-16T11:20:29.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.091+0000 7f03eaffd700 1 -- 172.21.15.181:0/341296712 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v128) v1 ==== 122+0+0 (secure 0 0 0) 0x7f03f0022070 con 0x7f03e4004850 2023-10-16T11:20:29.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f03dc06b140 msgr2=0x7f03dc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f03dc06b140 0x7f03dc06d600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0090b0 tx=0x7f03ec008030 comp rx=0 tx=0).stop 2023-10-16T11:20:29.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 msgr2=0x7f03fc14f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03fc14f540 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f03f0011d00 tx=0x7f03f000f520 comp rx=0 tx=0).stop 2023-10-16T11:20:29.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 shutdown_connections 2023-10-16T11:20:29.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f03dc06b140 0x7f03dc06d600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03fc0b40e0 0x7f03fc0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03e4004850 0x7f03fc14f540 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 --2- 172.21.15.181:0/341296712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0c98a0 0x7f03fc0c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 >> 172.21.15.181:0/341296712 conn(0x7f03fc0cd010 msgr2=0x7f03fc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:29.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 shutdown_connections 2023-10-16T11:20:29.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.094+0000 7f0400c94700 1 -- 172.21.15.181:0/341296712 wait complete. 2023-10-16T11:20:29.111 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:20:29.111 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:29.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.235+0000 7fb67bf2e700 1 Processor -- start 2023-10-16T11:20:29.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.235+0000 7fb67bf2e700 1 -- start start 2023-10-16T11:20:29.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb67bf2e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6741620a0 0x7fb6740af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb67bf2e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6741634e0 0x7fb6740c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb67bf2e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6740c39b0 0x7fb67406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb67bf2e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6740cb9a0 con 0x7fb6741620a0 2023-10-16T11:20:29.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb67bf2e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6740cebc0 con 0x7fb6740c39b0 2023-10-16T11:20:29.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb67bf2e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740c3db0 con 0x7fb6741634e0 2023-10-16T11:20:29.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6741620a0 0x7fb6740af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6741620a0 0x7fb6740af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57772/0 (socket says 172.21.15.181:57772) 2023-10-16T11:20:29.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 -- 172.21.15.181:0/399156939 learned_addr learned my addr 172.21.15.181:0/399156939 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:29.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb6794c9700 1 --2- 172.21.15.181:0/399156939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6741634e0 0x7fb6740c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb67a4cb700 1 --2- 172.21.15.181:0/399156939 >> v2:172.21.15.190:3300/0 conn(0x7fb6740c39b0 0x7fb67406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 -- 172.21.15.181:0/399156939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6741634e0 msgr2=0x7fb6740c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:29.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 --2- 172.21.15.181:0/399156939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6741634e0 0x7fb6740c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 -- 172.21.15.181:0/399156939 >> v2:172.21.15.190:3300/0 conn(0x7fb6740c39b0 msgr2=0x7fb67406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:29.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 --2- 172.21.15.181:0/399156939 >> v2:172.21.15.190:3300/0 conn(0x7fb6740c39b0 0x7fb67406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 -- 172.21.15.181:0/399156939 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740cee30 con 0x7fb6741620a0 2023-10-16T11:20:29.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.236+0000 7fb679cca700 1 --2- 172.21.15.181:0/399156939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740af340 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fb6740705b0 tx=0x7fb670009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55058ca6ea957b15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb678cc8700 1 -- 172.21.15.181:0/399156939 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670005e30 con 0x7fb6741620a0 2023-10-16T11:20:29.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb678cc8700 1 -- 172.21.15.181:0/399156939 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb670004030 con 0x7fb6741620a0 2023-10-16T11:20:29.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb678cc8700 1 -- 172.21.15.181:0/399156939 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700043e0 con 0x7fb6741620a0 2023-10-16T11:20:29.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 -- 172.21.15.181:0/399156939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 msgr2=0x7fb6740af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/399156939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740af340 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fb6740705b0 tx=0x7fb670009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 -- 172.21.15.181:0/399156939 shutdown_connections 2023-10-16T11:20:29.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/399156939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740af340 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/399156939 >> v2:172.21.15.190:3300/0 conn(0x7fb6740c39b0 0x7fb67406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/399156939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6741634e0 0x7fb6740c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 -- 172.21.15.181:0/399156939 >> 172.21.15.181:0/399156939 conn(0x7fb6740d04a0 msgr2=0x7fb6740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:29.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 -- 172.21.15.181:0/399156939 shutdown_connections 2023-10-16T11:20:29.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.237+0000 7fb67bf2e700 1 -- 172.21.15.181:0/399156939 wait complete. 2023-10-16T11:20:29.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.238+0000 7fb67bf2e700 1 Processor -- start 2023-10-16T11:20:29.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.238+0000 7fb67bf2e700 1 -- start start 2023-10-16T11:20:29.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67bf2e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6740c39b0 0x7fb674155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67bf2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67bf2e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6741634e0 0x7fb6740c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67bf2e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb674155fd0 con 0x7fb6741620a0 2023-10-16T11:20:29.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67bf2e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740b3420 con 0x7fb6741634e0 2023-10-16T11:20:29.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67bf2e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740b3700 con 0x7fb6740c39b0 2023-10-16T11:20:29.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57778/0 (socket says 172.21.15.181:57778) 2023-10-16T11:20:29.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 -- 172.21.15.181:0/3600490502 learned_addr learned my addr 172.21.15.181:0/3600490502 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:29.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb679cca700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6740c39b0 0x7fb674155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67a4cb700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6741634e0 0x7fb6740c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 -- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6740c39b0 msgr2=0x7fb674155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6740c39b0 0x7fb674155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 -- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6741634e0 msgr2=0x7fb6740c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6741634e0 0x7fb6740c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb6794c9700 1 -- 172.21.15.181:0/3600490502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb670009500 con 0x7fb6741620a0 2023-10-16T11:20:29.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.239+0000 7fb67a4cb700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6741634e0 0x7fb6740c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:29.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.240+0000 7fb6794c9700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740b0920 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fb66400e810 tx=0x7fb66400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.240+0000 7fb66a7fc700 1 -- 172.21.15.181:0/3600490502 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664013610 con 0x7fb6741620a0 2023-10-16T11:20:29.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.240+0000 7fb66a7fc700 1 -- 172.21.15.181:0/3600490502 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb664013770 con 0x7fb6741620a0 2023-10-16T11:20:29.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.240+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6740b39e0 con 0x7fb6741620a0 2023-10-16T11:20:29.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.240+0000 7fb66a7fc700 1 -- 172.21.15.181:0/3600490502 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664013b20 con 0x7fb6741620a0 2023-10-16T11:20:29.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.240+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6740b3fa0 con 0x7fb6741620a0 2023-10-16T11:20:29.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.241+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6740b4310 con 0x7fb6741620a0 2023-10-16T11:20:29.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.241+0000 7fb66a7fc700 1 -- 172.21.15.181:0/3600490502 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb664013c80 con 0x7fb6741620a0 2023-10-16T11:20:29.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.242+0000 7fb66a7fc700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb65806b140 0x7fb65806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.242+0000 7fb66a7fc700 1 -- 172.21.15.181:0/3600490502 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fb664089950 con 0x7fb6741620a0 2023-10-16T11:20:29.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.242+0000 7fb679cca700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb65806b140 0x7fb65806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.242+0000 7fb679cca700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb65806b140 0x7fb65806d600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6740705b0 tx=0x7fb670008070 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.246+0000 7fb66a7fc700 1 -- 172.21.15.181:0/3600490502 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb664054370 con 0x7fb6741620a0 2023-10-16T11:20:29.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.483+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb674067d20 con 0x7fb6741620a0 2023-10-16T11:20:29.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.484+0000 7fb66a7fc700 1 -- 172.21.15.181:0/3600490502 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4939 (secure 0 0 0) 0x7fb664057990 con 0x7fb6741620a0 2023-10-16T11:20:29.486 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:29.487 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":10690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-16T11:20:29.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb65806b140 msgr2=0x7fb65806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb65806b140 0x7fb65806d600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6740705b0 tx=0x7fb670008070 comp rx=0 tx=0).stop 2023-10-16T11:20:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 msgr2=0x7fb6740b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740b0920 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fb66400e810 tx=0x7fb66400eb20 comp rx=0 tx=0).stop 2023-10-16T11:20:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 shutdown_connections 2023-10-16T11:20:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fb65806b140 0x7fb65806d600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6740c39b0 0x7fb674155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6741634e0 0x7fb6740c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 --2- 172.21.15.181:0/3600490502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6741620a0 0x7fb6740b0920 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 >> 172.21.15.181:0/3600490502 conn(0x7fb6740d04a0 msgr2=0x7fb6740af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 shutdown_connections 2023-10-16T11:20:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.486+0000 7fb67bf2e700 1 -- 172.21.15.181:0/3600490502 wait complete. 2023-10-16T11:20:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 128 2023-10-16T11:20:29.500 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:29.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.619+0000 7f21e9b5d700 1 Processor -- start 2023-10-16T11:20:29.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.619+0000 7f21e9b5d700 1 -- start start 2023-10-16T11:20:29.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.619+0000 7f21e9b5d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21e415c210 0x7f21e40cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.619+0000 7f21e9b5d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f21e406a8a0 0x7f21e406ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.619+0000 7f21e9b5d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e40cf380 con 0x7f21e415f3b0 2023-10-16T11:20:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.619+0000 7f21e9b5d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e41562a0 con 0x7f21e415c210 2023-10-16T11:20:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.619+0000 7f21e9b5d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e406b270 con 0x7f21e406a8a0 2023-10-16T11:20:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21e415c210 0x7f21e40cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f21e415f3b0 0x7f21e415c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40630/0 (socket says 172.21.15.181:40630) 2023-10-16T11:20:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e2ffd700 1 -- 172.21.15.181:0/2909322180 learned_addr learned my addr 172.21.15.181:0/2909322180 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e3fff700 1 --2- 172.21.15.181:0/2909322180 >> v2:172.21.15.190:3300/0 conn(0x7f21e406a8a0 0x7f21e406ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e37fe700 1 -- 172.21.15.181:0/2909322180 >> v2:172.21.15.190:3300/0 conn(0x7f21e406a8a0 msgr2=0x7f21e406ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e37fe700 1 --2- 172.21.15.181:0/2909322180 >> v2:172.21.15.190:3300/0 conn(0x7f21e406a8a0 0x7f21e406ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e37fe700 1 -- 172.21.15.181:0/2909322180 >> v1:172.21.15.181:6789/0 conn(0x7f21e415f3b0 legacy=0x7f21e415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e37fe700 1 -- 172.21.15.181:0/2909322180 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e415cd00 con 0x7f21e415c210 2023-10-16T11:20:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e27fc700 1 -- 172.21.15.181:0/2909322180 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966174152 0 0) 0x7f21e40cf380 con 0x7f21e415f3b0 2023-10-16T11:20:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e37fe700 1 --2- 172.21.15.181:0/2909322180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e415c210 0x7f21e40cc1a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f21e40afce0 tx=0x7f21d4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adfad283a44246d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e27fc700 1 -- 172.21.15.181:0/2909322180 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4005e30 con 0x7f21e415c210 2023-10-16T11:20:29.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e27fc700 1 -- 172.21.15.181:0/2909322180 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d4006030 con 0x7f21e415c210 2023-10-16T11:20:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.620+0000 7f21e27fc700 1 -- 172.21.15.181:0/2909322180 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d40063c0 con 0x7f21e415c210 2023-10-16T11:20:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 -- 172.21.15.181:0/2909322180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e415c210 msgr2=0x7f21e40cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/2909322180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e415c210 0x7f21e40cc1a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f21e40afce0 tx=0x7f21d4009b30 comp rx=0 tx=0).stop 2023-10-16T11:20:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 -- 172.21.15.181:0/2909322180 shutdown_connections 2023-10-16T11:20:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/2909322180 >> v2:172.21.15.190:3300/0 conn(0x7f21e406a8a0 0x7f21e406ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/2909322180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e415c210 0x7f21e40cc1a0 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 -- 172.21.15.181:0/2909322180 >> 172.21.15.181:0/2909322180 conn(0x7f21e4099a00 msgr2=0x7f21e40d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 -- 172.21.15.181:0/2909322180 shutdown_connections 2023-10-16T11:20:29.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.621+0000 7f21e9b5d700 1 -- 172.21.15.181:0/2909322180 wait complete. 2023-10-16T11:20:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.622+0000 7f21e9b5d700 1 Processor -- start 2023-10-16T11:20:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e9b5d700 1 -- start start 2023-10-16T11:20:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e9b5d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e406a8a0 0x7f21e40c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e9b5d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21e415c210 0x7f21e40b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e9b5d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e415f3b0 0x7f21e40ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e9b5d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41f2040 con 0x7f21e406a8a0 2023-10-16T11:20:29.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e9b5d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41f13c0 con 0x7f21e415f3b0 2023-10-16T11:20:29.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e9b5d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41f16a0 con 0x7f21e415c210 2023-10-16T11:20:29.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e406a8a0 0x7f21e40c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e406a8a0 0x7f21e40c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57790/0 (socket says 172.21.15.181:57790) 2023-10-16T11:20:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.623+0000 7f21e37fe700 1 -- 172.21.15.181:0/3456550912 learned_addr learned my addr 172.21.15.181:0/3456550912 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e2ffd700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21e415c210 0x7f21e40b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e3fff700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e415f3b0 0x7f21e40ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e37fe700 1 -- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21e415c210 msgr2=0x7f21e40b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e37fe700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21e415c210 0x7f21e40b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e37fe700 1 -- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e415f3b0 msgr2=0x7f21e40ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e37fe700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e415f3b0 0x7f21e40ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e37fe700 1 -- 172.21.15.181:0/3456550912 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4009430 con 0x7f21e406a8a0 2023-10-16T11:20:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e2ffd700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21e415c210 0x7f21e40b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:29.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e37fe700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e406a8a0 0x7f21e40c75d0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21dbfff700 1 -- 172.21.15.181:0/3456550912 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4004240 con 0x7f21e406a8a0 2023-10-16T11:20:29.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21dbfff700 1 -- 172.21.15.181:0/3456550912 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d40043a0 con 0x7f21e406a8a0 2023-10-16T11:20:29.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e41f1980 con 0x7f21e406a8a0 2023-10-16T11:20:29.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21dbfff700 1 -- 172.21.15.181:0/3456550912 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d40036a0 con 0x7f21e406a8a0 2023-10-16T11:20:29.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.624+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e40b3090 con 0x7f21e406a8a0 2023-10-16T11:20:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.625+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e40679e0 con 0x7f21e406a8a0 2023-10-16T11:20:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.626+0000 7f21dbfff700 1 -- 172.21.15.181:0/3456550912 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f21d4022070 con 0x7f21e406a8a0 2023-10-16T11:20:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.626+0000 7f21dbfff700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f21c406b140 0x7f21c406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.626+0000 7f21dbfff700 1 -- 172.21.15.181:0/3456550912 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f21d408b490 con 0x7f21e406a8a0 2023-10-16T11:20:29.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.626+0000 7f21e2ffd700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f21c406b140 0x7f21c406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.626+0000 7f21e2ffd700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f21c406b140 0x7f21c406d600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000f60 tx=0x7f21cc007ca0 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.631+0000 7f21dbfff700 1 -- 172.21.15.181:0/3456550912 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f21d4056040 con 0x7f21e406a8a0 2023-10-16T11:20:29.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.824+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21e40cf0d0 con 0x7f21e406a8a0 2023-10-16T11:20:29.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.826+0000 7f21dbfff700 1 -- 172.21.15.181:0/3456550912 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+17406 (secure 0 0 0) 0x7f21d4059660 con 0x7f21e406a8a0 2023-10-16T11:20:29.828 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:29.828 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":236,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:27.065025+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:29.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f21c406b140 msgr2=0x7f21c406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f21c406b140 0x7f21c406d600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000f60 tx=0x7f21cc007ca0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e406a8a0 msgr2=0x7f21e40c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e406a8a0 0x7f21e40c75d0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d4004750 comp rx=0 tx=0).stop 2023-10-16T11:20:29.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 shutdown_connections 2023-10-16T11:20:29.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f21c406b140 0x7f21c406d600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e415f3b0 0x7f21e40ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21e406a8a0 0x7f21e40c75d0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 --2- 172.21.15.181:0/3456550912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21e415c210 0x7f21e40b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 >> 172.21.15.181:0/3456550912 conn(0x7f21e4099a00 msgr2=0x7f21e41545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:29.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 shutdown_connections 2023-10-16T11:20:29.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.828+0000 7f21e9b5d700 1 -- 172.21.15.181:0/3456550912 wait complete. 2023-10-16T11:20:29.843 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:20:29.843 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:20:29.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.961+0000 7f29bcce3700 1 Processor -- start 2023-10-16T11:20:29.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.961+0000 7f29bcce3700 1 -- start start 2023-10-16T11:20:29.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.962+0000 7f29bcce3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b8162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.962+0000 7f29bcce3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f29b815e700 0x7f29b806ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.962+0000 7f29bcce3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b806b240 con 0x7f29b806a8a0 2023-10-16T11:20:29.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.962+0000 7f29bcce3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b81584d0 con 0x7f29b815e700 2023-10-16T11:20:29.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.962+0000 7f29bcce3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80703b0 con 0x7f29b80ce9a0 2023-10-16T11:20:29.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f29b815e700 0x7f29b806ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b5d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f29b806a8a0 0x7f29b815dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40646/0 (socket says 172.21.15.181:40646) 2023-10-16T11:20:29.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b5d9b700 1 -- 172.21.15.181:0/500397531 learned_addr learned my addr 172.21.15.181:0/500397531 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:29.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b659c700 1 --2- 172.21.15.181:0/500397531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b8162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b6d9d700 1 -- 172.21.15.181:0/500397531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 msgr2=0x7f29b8162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:29.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b6d9d700 1 --2- 172.21.15.181:0/500397531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b8162140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b6d9d700 1 -- 172.21.15.181:0/500397531 >> v1:172.21.15.181:6789/0 conn(0x7f29b806a8a0 legacy=0x7f29b815dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b6d9d700 1 -- 172.21.15.181:0/500397531 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b815eeb0 con 0x7f29b815e700 2023-10-16T11:20:29.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b559a700 1 -- 172.21.15.181:0/500397531 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240498516 0 0) 0x7f29b806b240 con 0x7f29b806a8a0 2023-10-16T11:20:29.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b6d9d700 1 --2- 172.21.15.181:0/500397531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b806ad00 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009d90 tx=0x7f29b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfd9f7833ed9268 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b559a700 1 -- 172.21.15.181:0/500397531 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000e070 con 0x7f29b815e700 2023-10-16T11:20:29.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b559a700 1 -- 172.21.15.181:0/500397531 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b000bbd0 con 0x7f29b815e700 2023-10-16T11:20:29.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.963+0000 7f29b559a700 1 -- 172.21.15.181:0/500397531 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0012430 con 0x7f29b815e700 2023-10-16T11:20:29.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 -- 172.21.15.181:0/500397531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 msgr2=0x7f29b806ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 --2- 172.21.15.181:0/500397531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b806ad00 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009d90 tx=0x7f29b000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:29.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 -- 172.21.15.181:0/500397531 shutdown_connections 2023-10-16T11:20:29.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 --2- 172.21.15.181:0/500397531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b806ad00 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 --2- 172.21.15.181:0/500397531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b8162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 -- 172.21.15.181:0/500397531 >> 172.21.15.181:0/500397531 conn(0x7f29b80ccf80 msgr2=0x7f29b80cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:29.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 -- 172.21.15.181:0/500397531 shutdown_connections 2023-10-16T11:20:29.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.964+0000 7f29bcce3700 1 -- 172.21.15.181:0/500397531 wait complete. 2023-10-16T11:20:29.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.965+0000 7f29bcce3700 1 Processor -- start 2023-10-16T11:20:29.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.965+0000 7f29bcce3700 1 -- start start 2023-10-16T11:20:29.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29bcce3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b806a8a0 0x7f29b80b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29bcce3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b80b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29bcce3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b8154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29bcce3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80b4b80 con 0x7f29b815e700 2023-10-16T11:20:29.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29bcce3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80b4e20 con 0x7f29b806a8a0 2023-10-16T11:20:29.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29bcce3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b80b5100 con 0x7f29b80ce9a0 2023-10-16T11:20:29.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b8154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b8154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57806/0 (socket says 172.21.15.181:57806) 2023-10-16T11:20:29.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 -- 172.21.15.181:0/221667979 learned_addr learned my addr 172.21.15.181:0/221667979 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:29.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b5d9b700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b80b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b659c700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b806a8a0 0x7f29b80b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 -- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 msgr2=0x7f29b80b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b80b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 -- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b806a8a0 msgr2=0x7f29b80b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:29.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b806a8a0 0x7f29b80b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:29.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 -- 172.21.15.181:0/221667979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0009a70 con 0x7f29b815e700 2023-10-16T11:20:29.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29b6d9d700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b8154ac0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f29b0013000 tx=0x7f29b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.966+0000 7f29a6ffd700 1 -- 172.21.15.181:0/221667979 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0003ea0 con 0x7f29b815e700 2023-10-16T11:20:29.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.967+0000 7f29a6ffd700 1 -- 172.21.15.181:0/221667979 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0020070 con 0x7f29b815e700 2023-10-16T11:20:29.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.967+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b80b53e0 con 0x7f29b815e700 2023-10-16T11:20:29.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.967+0000 7f29a6ffd700 1 -- 172.21.15.181:0/221667979 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0004280 con 0x7f29b815e700 2023-10-16T11:20:29.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.967+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b80b59a0 con 0x7f29b815e700 2023-10-16T11:20:29.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.968+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b80cff60 con 0x7f29b815e700 2023-10-16T11:20:29.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.968+0000 7f29a6ffd700 1 -- 172.21.15.181:0/221667979 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f29b00043e0 con 0x7f29b815e700 2023-10-16T11:20:29.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.968+0000 7f29a6ffd700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f299806b170 0x7f299806d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:29.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.968+0000 7f29b659c700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f299806b170 0x7f299806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:29.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.968+0000 7f29a6ffd700 1 -- 172.21.15.181:0/221667979 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f29b0022070 con 0x7f29b815e700 2023-10-16T11:20:29.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.969+0000 7f29b659c700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f299806b170 0x7f299806d630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00a990 tx=0x7f29ac008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:29.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:29.973+0000 7f29a6ffd700 1 -- 172.21.15.181:0/221667979 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f29b0056a30 con 0x7f29b815e700 2023-10-16T11:20:30.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.167+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 --> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29b8161680 con 0x7f299806b170 2023-10-16T11:20:30.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.175+0000 7f29a6ffd700 1 -- 172.21.15.181:0/221667979 <== mgr.11814 v2:172.21.15.181:6832/3273985848 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290178 (secure 0 0 0) 0x7f29ac00a660 con 0x7f299806b170 2023-10-16T11:20:30.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:30.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f299806b170 msgr2=0x7f299806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f299806b170 0x7f299806d630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00a990 tx=0x7f29ac008040 comp rx=0 tx=0).stop 2023-10-16T11:20:30.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 msgr2=0x7f29b8154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b8154ac0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f29b0013000 tx=0x7f29b0004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 shutdown_connections 2023-10-16T11:20:30.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f299806b170 0x7f299806d630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b815e700 0x7f29b8154ac0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b806a8a0 0x7f29b80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 --2- 172.21.15.181:0/221667979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b80ce9a0 0x7f29b80b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 >> 172.21.15.181:0/221667979 conn(0x7f29b80ccf80 msgr2=0x7f29b81608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:30.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 shutdown_connections 2023-10-16T11:20:30.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.179+0000 7f29bcce3700 1 -- 172.21.15.181:0/221667979 wait complete. 2023-10-16T11:20:30.186 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:20:30.193 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-16T11:20:29.754825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":663,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":408,"ondisk_log_size":408,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465376,"kb_used_data":24424,"kb_used_omap":12,"kb_used_meta":440819,"kb_avail":754509344,"statfs":{"total":773094113280,"available":772617568256,"internally_reserved":0,"allocated":25010176,"data_stored":13452884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":451399224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.701151"},"pg_stats":[{"pgid":"31.3f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699092+0000","last_change":"2023-10-16T11:20:26.092187+0000","last_active":"2023-10-16T11:20:27.699092+0000","last_peered":"2023-10-16T11:20:27.699092+0000","last_clean":"2023-10-16T11:20:27.699092+0000","last_became_active":"2023-10-16T11:20:26.091884+0000","last_became_peered":"2023-10-16T11:20:26.091884+0000","last_unstale":"2023-10-16T11:20:27.699092+0000","last_undegraded":"2023-10-16T11:20:27.699092+0000","last_fullsized":"2023-10-16T11:20:27.699092+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:16.851198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698496+0000","last_change":"2023-10-16T11:20:26.085301+0000","last_active":"2023-10-16T11:20:27.698496+0000","last_peered":"2023-10-16T11:20:27.698496+0000","last_clean":"2023-10-16T11:20:27.698496+0000","last_became_active":"2023-10-16T11:20:26.084922+0000","last_became_peered":"2023-10-16T11:20:26.084922+0000","last_unstale":"2023-10-16T11:20:27.698496+0000","last_undegraded":"2023-10-16T11:20:27.698496+0000","last_fullsized":"2023-10-16T11:20:27.698496+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:36.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698594+0000","last_change":"2023-10-16T11:20:26.083255+0000","last_active":"2023-10-16T11:20:27.698594+0000","last_peered":"2023-10-16T11:20:27.698594+0000","last_clean":"2023-10-16T11:20:27.698594+0000","last_became_active":"2023-10-16T11:20:26.082915+0000","last_became_peered":"2023-10-16T11:20:26.082915+0000","last_unstale":"2023-10-16T11:20:27.698594+0000","last_undegraded":"2023-10-16T11:20:27.698594+0000","last_fullsized":"2023-10-16T11:20:27.698594+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:02.635859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699351+0000","last_change":"2023-10-16T11:20:26.084494+0000","last_active":"2023-10-16T11:20:27.699351+0000","last_peered":"2023-10-16T11:20:27.699351+0000","last_clean":"2023-10-16T11:20:27.699351+0000","last_became_active":"2023-10-16T11:20:26.084169+0000","last_became_peered":"2023-10-16T11:20:26.084169+0000","last_unstale":"2023-10-16T11:20:27.699351+0000","last_undegraded":"2023-10-16T11:20:27.699351+0000","last_fullsized":"2023-10-16T11:20:27.699351+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070698+0000","last_change":"2023-10-16T11:20:26.080175+0000","last_active":"2023-10-16T11:20:27.070698+0000","last_peered":"2023-10-16T11:20:27.070698+0000","last_clean":"2023-10-16T11:20:27.070698+0000","last_became_active":"2023-10-16T11:20:26.079884+0000","last_became_peered":"2023-10-16T11:20:26.079884+0000","last_unstale":"2023-10-16T11:20:27.070698+0000","last_undegraded":"2023-10-16T11:20:27.070698+0000","last_fullsized":"2023-10-16T11:20:27.070698+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:58.169049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699041+0000","last_change":"2023-10-16T11:20:26.099739+0000","last_active":"2023-10-16T11:20:27.699041+0000","last_peered":"2023-10-16T11:20:27.699041+0000","last_clean":"2023-10-16T11:20:27.699041+0000","last_became_active":"2023-10-16T11:20:26.099459+0000","last_became_peered":"2023-10-16T11:20:26.099459+0000","last_unstale":"2023-10-16T11:20:27.699041+0000","last_undegraded":"2023-10-16T11:20:27.699041+0000","last_fullsized":"2023-10-16T11:20:27.699041+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:14.530550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698719+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:27.698719+0000","last_peered":"2023-10-16T11:20:27.698719+0000","last_clean":"2023-10-16T11:20:27.698719+0000","last_became_active":"2023-10-16T11:20:26.098864+0000","last_became_peered":"2023-10-16T11:20:26.098864+0000","last_unstale":"2023-10-16T11:20:27.698719+0000","last_undegraded":"2023-10-16T11:20:27.698719+0000","last_fullsized":"2023-10-16T11:20:27.698719+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:04.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698487+0000","last_change":"2023-10-16T11:20:26.084562+0000","last_active":"2023-10-16T11:20:27.698487+0000","last_peered":"2023-10-16T11:20:27.698487+0000","last_clean":"2023-10-16T11:20:27.698487+0000","last_became_active":"2023-10-16T11:20:26.083055+0000","last_became_peered":"2023-10-16T11:20:26.083055+0000","last_unstale":"2023-10-16T11:20:27.698487+0000","last_undegraded":"2023-10-16T11:20:27.698487+0000","last_fullsized":"2023-10-16T11:20:27.698487+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.947551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070479+0000","last_change":"2023-10-16T11:20:26.077403+0000","last_active":"2023-10-16T11:20:27.070479+0000","last_peered":"2023-10-16T11:20:27.070479+0000","last_clean":"2023-10-16T11:20:27.070479+0000","last_became_active":"2023-10-16T11:20:26.077133+0000","last_became_peered":"2023-10-16T11:20:26.077133+0000","last_unstale":"2023-10-16T11:20:27.070479+0000","last_undegraded":"2023-10-16T11:20:27.070479+0000","last_fullsized":"2023-10-16T11:20:27.070479+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:15.067913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699656+0000","last_change":"2023-10-16T11:20:26.092685+0000","last_active":"2023-10-16T11:20:27.699656+0000","last_peered":"2023-10-16T11:20:27.699656+0000","last_clean":"2023-10-16T11:20:27.699656+0000","last_became_active":"2023-10-16T11:20:26.092413+0000","last_became_peered":"2023-10-16T11:20:26.092413+0000","last_unstale":"2023-10-16T11:20:27.699656+0000","last_undegraded":"2023-10-16T11:20:27.699656+0000","last_fullsized":"2023-10-16T11:20:27.699656+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:40.852817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068614+0000","last_change":"2023-10-16T11:20:26.094837+0000","last_active":"2023-10-16T11:20:27.068614+0000","last_peered":"2023-10-16T11:20:27.068614+0000","last_clean":"2023-10-16T11:20:27.068614+0000","last_became_active":"2023-10-16T11:20:26.094247+0000","last_became_peered":"2023-10-16T11:20:26.094247+0000","last_unstale":"2023-10-16T11:20:27.068614+0000","last_undegraded":"2023-10-16T11:20:27.068614+0000","last_fullsized":"2023-10-16T11:20:27.068614+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:16.136940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699661+0000","last_change":"2023-10-16T11:20:26.091607+0000","last_active":"2023-10-16T11:20:27.699661+0000","last_peered":"2023-10-16T11:20:27.699661+0000","last_clean":"2023-10-16T11:20:27.699661+0000","last_became_active":"2023-10-16T11:20:26.091336+0000","last_became_peered":"2023-10-16T11:20:26.091336+0000","last_unstale":"2023-10-16T11:20:27.699661+0000","last_undegraded":"2023-10-16T11:20:27.699661+0000","last_fullsized":"2023-10-16T11:20:27.699661+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:47.664336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699023+0000","last_change":"2023-10-16T11:20:26.101354+0000","last_active":"2023-10-16T11:20:27.699023+0000","last_peered":"2023-10-16T11:20:27.699023+0000","last_clean":"2023-10-16T11:20:27.699023+0000","last_became_active":"2023-10-16T11:20:26.101082+0000","last_became_peered":"2023-10-16T11:20:26.101082+0000","last_unstale":"2023-10-16T11:20:27.699023+0000","last_undegraded":"2023-10-16T11:20:27.699023+0000","last_fullsized":"2023-10-16T11:20:27.699023+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:48.559039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698824+0000","last_change":"2023-10-16T11:20:26.087929+0000","last_active":"2023-10-16T11:20:27.698824+0000","last_peered":"2023-10-16T11:20:27.698824+0000","last_clean":"2023-10-16T11:20:27.698824+0000","last_became_active":"2023-10-16T11:20:26.087611+0000","last_became_peered":"2023-10-16T11:20:26.087611+0000","last_unstale":"2023-10-16T11:20:27.698824+0000","last_undegraded":"2023-10-16T11:20:27.698824+0000","last_fullsized":"2023-10-16T11:20:27.698824+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:47.826756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070154+0000","last_change":"2023-10-16T11:20:26.076097+0000","last_active":"2023-10-16T11:20:27.070154+0000","last_peered":"2023-10-16T11:20:27.070154+0000","last_clean":"2023-10-16T11:20:27.070154+0000","last_became_active":"2023-10-16T11:20:26.075864+0000","last_became_peered":"2023-10-16T11:20:26.075864+0000","last_unstale":"2023-10-16T11:20:27.070154+0000","last_undegraded":"2023-10-16T11:20:27.070154+0000","last_fullsized":"2023-10-16T11:20:27.070154+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:36.478137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698402+0000","last_change":"2023-10-16T11:20:26.084690+0000","last_active":"2023-10-16T11:20:27.698402+0000","last_peered":"2023-10-16T11:20:27.698402+0000","last_clean":"2023-10-16T11:20:27.698402+0000","last_became_active":"2023-10-16T11:20:26.084390+0000","last_became_peered":"2023-10-16T11:20:26.084390+0000","last_unstale":"2023-10-16T11:20:27.698402+0000","last_undegraded":"2023-10-16T11:20:27.698402+0000","last_fullsized":"2023-10-16T11:20:27.698402+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:14.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699590+0000","last_change":"2023-10-16T11:20:26.089392+0000","last_active":"2023-10-16T11:20:27.699590+0000","last_peered":"2023-10-16T11:20:27.699590+0000","last_clean":"2023-10-16T11:20:27.699590+0000","last_became_active":"2023-10-16T11:20:26.089083+0000","last_became_peered":"2023-10-16T11:20:26.089083+0000","last_unstale":"2023-10-16T11:20:27.699590+0000","last_undegraded":"2023-10-16T11:20:27.699590+0000","last_fullsized":"2023-10-16T11:20:27.699590+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:16.986811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698635+0000","last_change":"2023-10-16T11:20:26.084395+0000","last_active":"2023-10-16T11:20:27.698635+0000","last_peered":"2023-10-16T11:20:27.698635+0000","last_clean":"2023-10-16T11:20:27.698635+0000","last_became_active":"2023-10-16T11:20:26.084119+0000","last_became_peered":"2023-10-16T11:20:26.084119+0000","last_unstale":"2023-10-16T11:20:27.698635+0000","last_undegraded":"2023-10-16T11:20:27.698635+0000","last_fullsized":"2023-10-16T11:20:27.698635+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.234510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699495+0000","last_change":"2023-10-16T11:20:26.095722+0000","last_active":"2023-10-16T11:20:27.699495+0000","last_peered":"2023-10-16T11:20:27.699495+0000","last_clean":"2023-10-16T11:20:27.699495+0000","last_became_active":"2023-10-16T11:20:26.095489+0000","last_became_peered":"2023-10-16T11:20:26.095489+0000","last_unstale":"2023-10-16T11:20:27.699495+0000","last_undegraded":"2023-10-16T11:20:27.699495+0000","last_fullsized":"2023-10-16T11:20:27.699495+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:32.623008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698700+0000","last_change":"2023-10-16T11:20:26.072060+0000","last_active":"2023-10-16T11:20:27.698700+0000","last_peered":"2023-10-16T11:20:27.698700+0000","last_clean":"2023-10-16T11:20:27.698700+0000","last_became_active":"2023-10-16T11:20:26.071739+0000","last_became_peered":"2023-10-16T11:20:26.071739+0000","last_unstale":"2023-10-16T11:20:27.698700+0000","last_undegraded":"2023-10-16T11:20:27.698700+0000","last_fullsized":"2023-10-16T11:20:27.698700+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:29.585136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699338+0000","last_change":"2023-10-16T11:20:26.083156+0000","last_active":"2023-10-16T11:20:27.699338+0000","last_peered":"2023-10-16T11:20:27.699338+0000","last_clean":"2023-10-16T11:20:27.699338+0000","last_became_active":"2023-10-16T11:20:26.082813+0000","last_became_peered":"2023-10-16T11:20:26.082813+0000","last_unstale":"2023-10-16T11:20:27.699338+0000","last_undegraded":"2023-10-16T11:20:27.699338+0000","last_fullsized":"2023-10-16T11:20:27.699338+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:26.407654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698680+0000","last_change":"2023-10-16T11:20:26.097008+0000","last_active":"2023-10-16T11:20:27.698680+0000","last_peered":"2023-10-16T11:20:27.698680+0000","last_clean":"2023-10-16T11:20:27.698680+0000","last_became_active":"2023-10-16T11:20:26.096754+0000","last_became_peered":"2023-10-16T11:20:26.096754+0000","last_unstale":"2023-10-16T11:20:27.698680+0000","last_undegraded":"2023-10-16T11:20:27.698680+0000","last_fullsized":"2023-10-16T11:20:27.698680+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:24.400518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698569+0000","last_change":"2023-10-16T11:20:26.093039+0000","last_active":"2023-10-16T11:20:27.698569+0000","last_peered":"2023-10-16T11:20:27.698569+0000","last_clean":"2023-10-16T11:20:27.698569+0000","last_became_active":"2023-10-16T11:20:26.092725+0000","last_became_peered":"2023-10-16T11:20:26.092725+0000","last_unstale":"2023-10-16T11:20:27.698569+0000","last_undegraded":"2023-10-16T11:20:27.698569+0000","last_fullsized":"2023-10-16T11:20:27.698569+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:37.794560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698256+0000","last_change":"2023-10-16T11:20:26.091655+0000","last_active":"2023-10-16T11:20:27.698256+0000","last_peered":"2023-10-16T11:20:27.698256+0000","last_clean":"2023-10-16T11:20:27.698256+0000","last_became_active":"2023-10-16T11:20:26.091402+0000","last_became_peered":"2023-10-16T11:20:26.091402+0000","last_unstale":"2023-10-16T11:20:27.698256+0000","last_undegraded":"2023-10-16T11:20:27.698256+0000","last_fullsized":"2023-10-16T11:20:27.698256+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:25.104508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698667+0000","last_change":"2023-10-16T11:20:26.088880+0000","last_active":"2023-10-16T11:20:27.698667+0000","last_peered":"2023-10-16T11:20:27.698667+0000","last_clean":"2023-10-16T11:20:27.698667+0000","last_became_active":"2023-10-16T11:20:26.088427+0000","last_became_peered":"2023-10-16T11:20:26.088427+0000","last_unstale":"2023-10-16T11:20:27.698667+0000","last_undegraded":"2023-10-16T11:20:27.698667+0000","last_fullsized":"2023-10-16T11:20:27.698667+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:30.068691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698978+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:27.698978+0000","last_peered":"2023-10-16T11:20:27.698978+0000","last_clean":"2023-10-16T11:20:27.698978+0000","last_became_active":"2023-10-16T11:20:26.098861+0000","last_became_peered":"2023-10-16T11:20:26.098861+0000","last_unstale":"2023-10-16T11:20:27.698978+0000","last_undegraded":"2023-10-16T11:20:27.698978+0000","last_fullsized":"2023-10-16T11:20:27.698978+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:03.247595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068750+0000","last_change":"2023-10-16T11:20:26.098406+0000","last_active":"2023-10-16T11:20:27.068750+0000","last_peered":"2023-10-16T11:20:27.068750+0000","last_clean":"2023-10-16T11:20:27.068750+0000","last_became_active":"2023-10-16T11:20:26.098082+0000","last_became_peered":"2023-10-16T11:20:26.098082+0000","last_unstale":"2023-10-16T11:20:27.068750+0000","last_undegraded":"2023-10-16T11:20:27.068750+0000","last_fullsized":"2023-10-16T11:20:27.068750+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:44.140416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698615+0000","last_change":"2023-10-16T11:20:26.082815+0000","last_active":"2023-10-16T11:20:27.698615+0000","last_peered":"2023-10-16T11:20:27.698615+0000","last_clean":"2023-10-16T11:20:27.698615+0000","last_became_active":"2023-10-16T11:20:26.082524+0000","last_became_peered":"2023-10-16T11:20:26.082524+0000","last_unstale":"2023-10-16T11:20:27.698615+0000","last_undegraded":"2023-10-16T11:20:27.698615+0000","last_fullsized":"2023-10-16T11:20:27.698615+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:16.774780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068714+0000","last_change":"2023-10-16T11:20:26.100460+0000","last_active":"2023-10-16T11:20:27.068714+0000","last_peered":"2023-10-16T11:20:27.068714+0000","last_clean":"2023-10-16T11:20:27.068714+0000","last_became_active":"2023-10-16T11:20:26.100088+0000","last_became_peered":"2023-10-16T11:20:26.100088+0000","last_unstale":"2023-10-16T11:20:27.068714+0000","last_undegraded":"2023-10-16T11:20:27.068714+0000","last_fullsized":"2023-10-16T11:20:27.068714+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:38.907350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070022+0000","last_change":"2023-10-16T11:20:26.070717+0000","last_active":"2023-10-16T11:20:27.070022+0000","last_peered":"2023-10-16T11:20:27.070022+0000","last_clean":"2023-10-16T11:20:27.070022+0000","last_became_active":"2023-10-16T11:20:26.070394+0000","last_became_peered":"2023-10-16T11:20:26.070394+0000","last_unstale":"2023-10-16T11:20:27.070022+0000","last_undegraded":"2023-10-16T11:20:27.070022+0000","last_fullsized":"2023-10-16T11:20:27.070022+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698387+0000","last_change":"2023-10-16T11:20:26.086782+0000","last_active":"2023-10-16T11:20:27.698387+0000","last_peered":"2023-10-16T11:20:27.698387+0000","last_clean":"2023-10-16T11:20:27.698387+0000","last_became_active":"2023-10-16T11:20:26.086510+0000","last_became_peered":"2023-10-16T11:20:26.086510+0000","last_unstale":"2023-10-16T11:20:27.698387+0000","last_undegraded":"2023-10-16T11:20:27.698387+0000","last_fullsized":"2023-10-16T11:20:27.698387+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:26:24.730788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698945+0000","last_change":"2023-10-16T11:20:26.083094+0000","last_active":"2023-10-16T11:20:27.698945+0000","last_peered":"2023-10-16T11:20:27.698945+0000","last_clean":"2023-10-16T11:20:27.698945+0000","last_became_active":"2023-10-16T11:20:26.080031+0000","last_became_peered":"2023-10-16T11:20:26.080031+0000","last_unstale":"2023-10-16T11:20:27.698945+0000","last_undegraded":"2023-10-16T11:20:27.698945+0000","last_fullsized":"2023-10-16T11:20:27.698945+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:06:55.285592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699365+0000","last_change":"2023-10-16T11:20:26.096214+0000","last_active":"2023-10-16T11:20:27.699365+0000","last_peered":"2023-10-16T11:20:27.699365+0000","last_clean":"2023-10-16T11:20:27.699365+0000","last_became_active":"2023-10-16T11:20:26.093850+0000","last_became_peered":"2023-10-16T11:20:26.093850+0000","last_unstale":"2023-10-16T11:20:27.699365+0000","last_undegraded":"2023-10-16T11:20:27.699365+0000","last_fullsized":"2023-10-16T11:20:27.699365+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:48.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.089287+0000","last_change":"2023-10-16T11:20:25.084166+0000","last_active":"2023-10-16T11:20:26.089287+0000","last_peered":"2023-10-16T11:20:26.089287+0000","last_clean":"2023-10-16T11:20:26.089287+0000","last_became_active":"2023-10-16T11:20:25.083857+0000","last_became_peered":"2023-10-16T11:20:25.083857+0000","last_unstale":"2023-10-16T11:20:26.089287+0000","last_undegraded":"2023-10-16T11:20:26.089287+0000","last_fullsized":"2023-10-16T11:20:26.089287+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:47.210086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068575+0000","last_change":"2023-10-16T11:20:26.094539+0000","last_active":"2023-10-16T11:20:27.068575+0000","last_peered":"2023-10-16T11:20:27.068575+0000","last_clean":"2023-10-16T11:20:27.068575+0000","last_became_active":"2023-10-16T11:20:26.094220+0000","last_became_peered":"2023-10-16T11:20:26.094220+0000","last_unstale":"2023-10-16T11:20:27.068575+0000","last_undegraded":"2023-10-16T11:20:27.068575+0000","last_fullsized":"2023-10-16T11:20:27.068575+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:05.280523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.097950+0000","last_change":"2023-10-16T11:20:25.080334+0000","last_active":"2023-10-16T11:20:26.097950+0000","last_peered":"2023-10-16T11:20:26.097950+0000","last_clean":"2023-10-16T11:20:26.097950+0000","last_became_active":"2023-10-16T11:20:25.080084+0000","last_became_peered":"2023-10-16T11:20:25.080084+0000","last_unstale":"2023-10-16T11:20:26.097950+0000","last_undegraded":"2023-10-16T11:20:26.097950+0000","last_fullsized":"2023-10-16T11:20:26.097950+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:19.309219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699594+0000","last_change":"2023-10-16T11:20:26.086141+0000","last_active":"2023-10-16T11:20:27.699594+0000","last_peered":"2023-10-16T11:20:27.699594+0000","last_clean":"2023-10-16T11:20:27.699594+0000","last_became_active":"2023-10-16T11:20:26.085865+0000","last_became_peered":"2023-10-16T11:20:26.085865+0000","last_unstale":"2023-10-16T11:20:27.699594+0000","last_undegraded":"2023-10-16T11:20:27.699594+0000","last_fullsized":"2023-10-16T11:20:27.699594+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:31.177316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.099164+0000","last_change":"2023-10-16T11:20:25.087325+0000","last_active":"2023-10-16T11:20:26.099164+0000","last_peered":"2023-10-16T11:20:26.099164+0000","last_clean":"2023-10-16T11:20:26.099164+0000","last_became_active":"2023-10-16T11:20:25.087033+0000","last_became_peered":"2023-10-16T11:20:25.087033+0000","last_unstale":"2023-10-16T11:20:26.099164+0000","last_undegraded":"2023-10-16T11:20:26.099164+0000","last_fullsized":"2023-10-16T11:20:26.099164+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:17:22.652116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068468+0000","last_change":"2023-10-16T11:20:26.090198+0000","last_active":"2023-10-16T11:20:27.068468+0000","last_peered":"2023-10-16T11:20:27.068468+0000","last_clean":"2023-10-16T11:20:27.068468+0000","last_became_active":"2023-10-16T11:20:26.089545+0000","last_became_peered":"2023-10-16T11:20:26.089545+0000","last_unstale":"2023-10-16T11:20:27.068468+0000","last_undegraded":"2023-10-16T11:20:27.068468+0000","last_fullsized":"2023-10-16T11:20:27.068468+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:03.204915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.094694+0000","last_change":"2023-10-16T11:20:25.084321+0000","last_active":"2023-10-16T11:20:26.094694+0000","last_peered":"2023-10-16T11:20:26.094694+0000","last_clean":"2023-10-16T11:20:26.094694+0000","last_became_active":"2023-10-16T11:20:25.083847+0000","last_became_peered":"2023-10-16T11:20:25.083847+0000","last_unstale":"2023-10-16T11:20:26.094694+0000","last_undegraded":"2023-10-16T11:20:26.094694+0000","last_fullsized":"2023-10-16T11:20:26.094694+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:16.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.069796+0000","last_change":"2023-10-16T11:20:26.089495+0000","last_active":"2023-10-16T11:20:27.069796+0000","last_peered":"2023-10-16T11:20:27.069796+0000","last_clean":"2023-10-16T11:20:27.069796+0000","last_became_active":"2023-10-16T11:20:26.089203+0000","last_became_peered":"2023-10-16T11:20:26.089203+0000","last_unstale":"2023-10-16T11:20:27.069796+0000","last_undegraded":"2023-10-16T11:20:27.069796+0000","last_fullsized":"2023-10-16T11:20:27.069796+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:59.154547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.096012+0000","last_change":"2023-10-16T11:20:25.083810+0000","last_active":"2023-10-16T11:20:26.096012+0000","last_peered":"2023-10-16T11:20:26.096012+0000","last_clean":"2023-10-16T11:20:26.096012+0000","last_became_active":"2023-10-16T11:20:25.083251+0000","last_became_peered":"2023-10-16T11:20:25.083251+0000","last_unstale":"2023-10-16T11:20:26.096012+0000","last_undegraded":"2023-10-16T11:20:26.096012+0000","last_fullsized":"2023-10-16T11:20:26.096012+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:44.723324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698187+0000","last_change":"2023-10-16T11:20:26.071683+0000","last_active":"2023-10-16T11:20:27.698187+0000","last_peered":"2023-10-16T11:20:27.698187+0000","last_clean":"2023-10-16T11:20:27.698187+0000","last_became_active":"2023-10-16T11:20:26.071396+0000","last_became_peered":"2023-10-16T11:20:26.071396+0000","last_unstale":"2023-10-16T11:20:27.698187+0000","last_undegraded":"2023-10-16T11:20:27.698187+0000","last_fullsized":"2023-10-16T11:20:27.698187+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:15.600869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.076828+0000","last_change":"2023-10-16T11:20:25.085339+0000","last_active":"2023-10-16T11:20:26.076828+0000","last_peered":"2023-10-16T11:20:26.076828+0000","last_clean":"2023-10-16T11:20:26.076828+0000","last_became_active":"2023-10-16T11:20:25.085073+0000","last_became_peered":"2023-10-16T11:20:25.085073+0000","last_unstale":"2023-10-16T11:20:26.076828+0000","last_undegraded":"2023-10-16T11:20:26.076828+0000","last_fullsized":"2023-10-16T11:20:26.076828+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:23.090533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698420+0000","last_change":"2023-10-16T11:20:26.085542+0000","last_active":"2023-10-16T11:20:27.698420+0000","last_peered":"2023-10-16T11:20:27.698420+0000","last_clean":"2023-10-16T11:20:27.698420+0000","last_became_active":"2023-10-16T11:20:26.083529+0000","last_became_peered":"2023-10-16T11:20:26.083529+0000","last_unstale":"2023-10-16T11:20:27.698420+0000","last_undegraded":"2023-10-16T11:20:27.698420+0000","last_fullsized":"2023-10-16T11:20:27.698420+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069395+0000","last_change":"2023-10-16T11:20:25.088288+0000","last_active":"2023-10-16T11:20:26.069395+0000","last_peered":"2023-10-16T11:20:26.069395+0000","last_clean":"2023-10-16T11:20:26.069395+0000","last_became_active":"2023-10-16T11:20:25.087144+0000","last_became_peered":"2023-10-16T11:20:25.087144+0000","last_unstale":"2023-10-16T11:20:26.069395+0000","last_undegraded":"2023-10-16T11:20:26.069395+0000","last_fullsized":"2023-10-16T11:20:26.069395+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:16:49.706199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698789+0000","last_change":"2023-10-16T11:20:26.085425+0000","last_active":"2023-10-16T11:20:27.698789+0000","last_peered":"2023-10-16T11:20:27.698789+0000","last_clean":"2023-10-16T11:20:27.698789+0000","last_became_active":"2023-10-16T11:20:26.085150+0000","last_became_peered":"2023-10-16T11:20:26.085150+0000","last_unstale":"2023-10-16T11:20:27.698789+0000","last_undegraded":"2023-10-16T11:20:27.698789+0000","last_fullsized":"2023-10-16T11:20:27.698789+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:14.978178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.076553+0000","last_change":"2023-10-16T11:20:25.078062+0000","last_active":"2023-10-16T11:20:26.076553+0000","last_peered":"2023-10-16T11:20:26.076553+0000","last_clean":"2023-10-16T11:20:26.076553+0000","last_became_active":"2023-10-16T11:20:25.077777+0000","last_became_peered":"2023-10-16T11:20:25.077777+0000","last_unstale":"2023-10-16T11:20:26.076553+0000","last_undegraded":"2023-10-16T11:20:26.076553+0000","last_fullsized":"2023-10-16T11:20:26.076553+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:13.108385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068962+0000","last_change":"2023-10-16T11:20:26.098504+0000","last_active":"2023-10-16T11:20:27.068962+0000","last_peered":"2023-10-16T11:20:27.068962+0000","last_clean":"2023-10-16T11:20:27.068962+0000","last_became_active":"2023-10-16T11:20:26.098021+0000","last_became_peered":"2023-10-16T11:20:26.098021+0000","last_unstale":"2023-10-16T11:20:27.068962+0000","last_undegraded":"2023-10-16T11:20:27.068962+0000","last_fullsized":"2023-10-16T11:20:27.068962+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:06.218021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069690+0000","last_change":"2023-10-16T11:20:25.088083+0000","last_active":"2023-10-16T11:20:26.069690+0000","last_peered":"2023-10-16T11:20:26.069690+0000","last_clean":"2023-10-16T11:20:26.069690+0000","last_became_active":"2023-10-16T11:20:25.087792+0000","last_became_peered":"2023-10-16T11:20:25.087792+0000","last_unstale":"2023-10-16T11:20:26.069690+0000","last_undegraded":"2023-10-16T11:20:26.069690+0000","last_fullsized":"2023-10-16T11:20:26.069690+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:22.682758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698815+0000","last_change":"2023-10-16T11:20:26.090580+0000","last_active":"2023-10-16T11:20:27.698815+0000","last_peered":"2023-10-16T11:20:27.698815+0000","last_clean":"2023-10-16T11:20:27.698815+0000","last_became_active":"2023-10-16T11:20:26.090296+0000","last_became_peered":"2023-10-16T11:20:26.090296+0000","last_unstale":"2023-10-16T11:20:27.698815+0000","last_undegraded":"2023-10-16T11:20:27.698815+0000","last_fullsized":"2023-10-16T11:20:27.698815+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:29.302528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"235'3","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.103092+0000","last_change":"2023-10-16T11:20:25.083562+0000","last_active":"2023-10-16T11:20:26.103092+0000","last_peered":"2023-10-16T11:20:26.103092+0000","last_clean":"2023-10-16T11:20:26.103092+0000","last_became_active":"2023-10-16T11:20:25.083270+0000","last_became_peered":"2023-10-16T11:20:25.083270+0000","last_unstale":"2023-10-16T11:20:26.103092+0000","last_undegraded":"2023-10-16T11:20:26.103092+0000","last_fullsized":"2023-10-16T11:20:26.103092+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:34.227829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698773+0000","last_change":"2023-10-16T11:20:26.079886+0000","last_active":"2023-10-16T11:20:27.698773+0000","last_peered":"2023-10-16T11:20:27.698773+0000","last_clean":"2023-10-16T11:20:27.698773+0000","last_became_active":"2023-10-16T11:20:26.079622+0000","last_became_peered":"2023-10-16T11:20:26.079622+0000","last_unstale":"2023-10-16T11:20:27.698773+0000","last_undegraded":"2023-10-16T11:20:27.698773+0000","last_fullsized":"2023-10-16T11:20:27.698773+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:30.843293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"235'6","reported_seq":18,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.105055+0000","last_change":"2023-10-16T11:20:25.083500+0000","last_active":"2023-10-16T11:20:26.105055+0000","last_peered":"2023-10-16T11:20:26.105055+0000","last_clean":"2023-10-16T11:20:26.105055+0000","last_became_active":"2023-10-16T11:20:25.083213+0000","last_became_peered":"2023-10-16T11:20:25.083213+0000","last_unstale":"2023-10-16T11:20:26.105055+0000","last_undegraded":"2023-10-16T11:20:26.105055+0000","last_fullsized":"2023-10-16T11:20:26.105055+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:04.495826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698906+0000","last_change":"2023-10-16T11:20:26.083135+0000","last_active":"2023-10-16T11:20:27.698906+0000","last_peered":"2023-10-16T11:20:27.698906+0000","last_clean":"2023-10-16T11:20:27.698906+0000","last_became_active":"2023-10-16T11:20:26.082824+0000","last_became_peered":"2023-10-16T11:20:26.082824+0000","last_unstale":"2023-10-16T11:20:27.698906+0000","last_undegraded":"2023-10-16T11:20:27.698906+0000","last_fullsized":"2023-10-16T11:20:27.698906+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:55.337626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.103444+0000","last_change":"2023-10-16T11:20:25.083057+0000","last_active":"2023-10-16T11:20:26.103444+0000","last_peered":"2023-10-16T11:20:26.103444+0000","last_clean":"2023-10-16T11:20:26.103444+0000","last_became_active":"2023-10-16T11:20:25.082601+0000","last_became_peered":"2023-10-16T11:20:25.082601+0000","last_unstale":"2023-10-16T11:20:26.103444+0000","last_undegraded":"2023-10-16T11:20:26.103444+0000","last_fullsized":"2023-10-16T11:20:26.103444+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.609347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.069797+0000","last_change":"2023-10-16T11:20:26.073157+0000","last_active":"2023-10-16T11:20:27.069797+0000","last_peered":"2023-10-16T11:20:27.069797+0000","last_clean":"2023-10-16T11:20:27.069797+0000","last_became_active":"2023-10-16T11:20:26.072892+0000","last_became_peered":"2023-10-16T11:20:26.072892+0000","last_unstale":"2023-10-16T11:20:27.069797+0000","last_undegraded":"2023-10-16T11:20:27.069797+0000","last_fullsized":"2023-10-16T11:20:27.069797+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:21:31.224577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069498+0000","last_change":"2023-10-16T11:20:25.079093+0000","last_active":"2023-10-16T11:20:26.069498+0000","last_peered":"2023-10-16T11:20:26.069498+0000","last_clean":"2023-10-16T11:20:26.069498+0000","last_became_active":"2023-10-16T11:20:25.078740+0000","last_became_peered":"2023-10-16T11:20:25.078740+0000","last_unstale":"2023-10-16T11:20:26.069498+0000","last_undegraded":"2023-10-16T11:20:26.069498+0000","last_fullsized":"2023-10-16T11:20:26.069498+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:27.511026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698358+0000","last_change":"2023-10-16T11:20:26.084841+0000","last_active":"2023-10-16T11:20:27.698358+0000","last_peered":"2023-10-16T11:20:27.698358+0000","last_clean":"2023-10-16T11:20:27.698358+0000","last_became_active":"2023-10-16T11:20:26.084153+0000","last_became_peered":"2023-10-16T11:20:26.084153+0000","last_unstale":"2023-10-16T11:20:27.698358+0000","last_undegraded":"2023-10-16T11:20:27.698358+0000","last_fullsized":"2023-10-16T11:20:27.698358+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:30.506645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.076788+0000","last_change":"2023-10-16T11:20:25.084386+0000","last_active":"2023-10-16T11:20:26.076788+0000","last_peered":"2023-10-16T11:20:26.076788+0000","last_clean":"2023-10-16T11:20:26.076788+0000","last_became_active":"2023-10-16T11:20:25.083974+0000","last_became_peered":"2023-10-16T11:20:25.083974+0000","last_unstale":"2023-10-16T11:20:26.076788+0000","last_undegraded":"2023-10-16T11:20:26.076788+0000","last_fullsized":"2023-10-16T11:20:26.076788+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:05.210033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698448+0000","last_change":"2023-10-16T11:20:26.096969+0000","last_active":"2023-10-16T11:20:27.698448+0000","last_peered":"2023-10-16T11:20:27.698448+0000","last_clean":"2023-10-16T11:20:27.698448+0000","last_became_active":"2023-10-16T11:20:26.096691+0000","last_became_peered":"2023-10-16T11:20:26.096691+0000","last_unstale":"2023-10-16T11:20:27.698448+0000","last_undegraded":"2023-10-16T11:20:27.698448+0000","last_fullsized":"2023-10-16T11:20:27.698448+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:05.355796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"235'8","reported_seq":20,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.104404+0000","last_change":"2023-10-16T11:20:25.082367+0000","last_active":"2023-10-16T11:20:26.104404+0000","last_peered":"2023-10-16T11:20:26.104404+0000","last_clean":"2023-10-16T11:20:26.104404+0000","last_became_active":"2023-10-16T11:20:25.082053+0000","last_became_peered":"2023-10-16T11:20:25.082053+0000","last_unstale":"2023-10-16T11:20:26.104404+0000","last_undegraded":"2023-10-16T11:20:26.104404+0000","last_fullsized":"2023-10-16T11:20:26.104404+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:29.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698992+0000","last_change":"2023-10-16T11:20:26.085864+0000","last_active":"2023-10-16T11:20:27.698992+0000","last_peered":"2023-10-16T11:20:27.698992+0000","last_clean":"2023-10-16T11:20:27.698992+0000","last_became_active":"2023-10-16T11:20:26.085644+0000","last_became_peered":"2023-10-16T11:20:26.085644+0000","last_unstale":"2023-10-16T11:20:27.698992+0000","last_undegraded":"2023-10-16T11:20:27.698992+0000","last_fullsized":"2023-10-16T11:20:27.698992+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:16.138806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.089803+0000","last_change":"2023-10-16T11:20:25.084626+0000","last_active":"2023-10-16T11:20:26.089803+0000","last_peered":"2023-10-16T11:20:26.089803+0000","last_clean":"2023-10-16T11:20:26.089803+0000","last_became_active":"2023-10-16T11:20:25.084276+0000","last_became_peered":"2023-10-16T11:20:25.084276+0000","last_unstale":"2023-10-16T11:20:26.089803+0000","last_undegraded":"2023-10-16T11:20:26.089803+0000","last_fullsized":"2023-10-16T11:20:26.089803+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.241621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699449+0000","last_change":"2023-10-16T11:20:26.084908+0000","last_active":"2023-10-16T11:20:27.699449+0000","last_peered":"2023-10-16T11:20:27.699449+0000","last_clean":"2023-10-16T11:20:27.699449+0000","last_became_active":"2023-10-16T11:20:26.084599+0000","last_became_peered":"2023-10-16T11:20:26.084599+0000","last_unstale":"2023-10-16T11:20:27.699449+0000","last_undegraded":"2023-10-16T11:20:27.699449+0000","last_fullsized":"2023-10-16T11:20:27.699449+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:13.336265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.097751+0000","last_change":"2023-10-16T11:20:25.087228+0000","last_active":"2023-10-16T11:20:26.097751+0000","last_peered":"2023-10-16T11:20:26.097751+0000","last_clean":"2023-10-16T11:20:26.097751+0000","last_became_active":"2023-10-16T11:20:25.086965+0000","last_became_peered":"2023-10-16T11:20:25.086965+0000","last_unstale":"2023-10-16T11:20:26.097751+0000","last_undegraded":"2023-10-16T11:20:26.097751+0000","last_fullsized":"2023-10-16T11:20:26.097751+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:08.265203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070276+0000","last_change":"2023-10-16T11:20:26.077503+0000","last_active":"2023-10-16T11:20:27.070276+0000","last_peered":"2023-10-16T11:20:27.070276+0000","last_clean":"2023-10-16T11:20:27.070276+0000","last_became_active":"2023-10-16T11:20:26.077247+0000","last_became_peered":"2023-10-16T11:20:26.077247+0000","last_unstale":"2023-10-16T11:20:27.070276+0000","last_undegraded":"2023-10-16T11:20:27.070276+0000","last_fullsized":"2023-10-16T11:20:27.070276+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:49.573161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.091830+0000","last_change":"2023-10-16T11:20:25.076671+0000","last_active":"2023-10-16T11:20:26.091830+0000","last_peered":"2023-10-16T11:20:26.091830+0000","last_clean":"2023-10-16T11:20:26.091830+0000","last_became_active":"2023-10-16T11:20:25.076356+0000","last_became_peered":"2023-10-16T11:20:25.076356+0000","last_unstale":"2023-10-16T11:20:26.091830+0000","last_undegraded":"2023-10-16T11:20:26.091830+0000","last_fullsized":"2023-10-16T11:20:26.091830+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:21.582269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699655+0000","last_change":"2023-10-16T11:20:26.086294+0000","last_active":"2023-10-16T11:20:27.699655+0000","last_peered":"2023-10-16T11:20:27.699655+0000","last_clean":"2023-10-16T11:20:27.699655+0000","last_became_active":"2023-10-16T11:20:26.085985+0000","last_became_peered":"2023-10-16T11:20:26.085985+0000","last_unstale":"2023-10-16T11:20:27.699655+0000","last_undegraded":"2023-10-16T11:20:27.699655+0000","last_fullsized":"2023-10-16T11:20:27.699655+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:58.159985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.088550+0000","last_change":"2023-10-16T11:20:25.085222+0000","last_active":"2023-10-16T11:20:26.088550+0000","last_peered":"2023-10-16T11:20:26.088550+0000","last_clean":"2023-10-16T11:20:26.088550+0000","last_became_active":"2023-10-16T11:20:25.084958+0000","last_became_peered":"2023-10-16T11:20:25.084958+0000","last_unstale":"2023-10-16T11:20:26.088550+0000","last_undegraded":"2023-10-16T11:20:26.088550+0000","last_fullsized":"2023-10-16T11:20:26.088550+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:15.130817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699314+0000","last_change":"2023-10-16T11:20:26.091707+0000","last_active":"2023-10-16T11:20:27.699314+0000","last_peered":"2023-10-16T11:20:27.699314+0000","last_clean":"2023-10-16T11:20:27.699314+0000","last_became_active":"2023-10-16T11:20:26.091470+0000","last_became_peered":"2023-10-16T11:20:26.091470+0000","last_unstale":"2023-10-16T11:20:27.699314+0000","last_undegraded":"2023-10-16T11:20:27.699314+0000","last_fullsized":"2023-10-16T11:20:27.699314+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.100597+0000","last_change":"2023-10-16T11:20:25.086865+0000","last_active":"2023-10-16T11:20:26.100597+0000","last_peered":"2023-10-16T11:20:26.100597+0000","last_clean":"2023-10-16T11:20:26.100597+0000","last_became_active":"2023-10-16T11:20:25.086528+0000","last_became_peered":"2023-10-16T11:20:25.086528+0000","last_unstale":"2023-10-16T11:20:26.100597+0000","last_undegraded":"2023-10-16T11:20:26.100597+0000","last_fullsized":"2023-10-16T11:20:26.100597+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:17.050518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698303+0000","last_change":"2023-10-16T11:20:26.080221+0000","last_active":"2023-10-16T11:20:27.698303+0000","last_peered":"2023-10-16T11:20:27.698303+0000","last_clean":"2023-10-16T11:20:27.698303+0000","last_became_active":"2023-10-16T11:20:26.079927+0000","last_became_peered":"2023-10-16T11:20:26.079927+0000","last_unstale":"2023-10-16T11:20:27.698303+0000","last_undegraded":"2023-10-16T11:20:27.698303+0000","last_fullsized":"2023-10-16T11:20:27.698303+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:27.603420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.075149+0000","last_change":"2023-10-16T11:20:25.084862+0000","last_active":"2023-10-16T11:20:26.075149+0000","last_peered":"2023-10-16T11:20:26.075149+0000","last_clean":"2023-10-16T11:20:26.075149+0000","last_became_active":"2023-10-16T11:20:25.084538+0000","last_became_peered":"2023-10-16T11:20:25.084538+0000","last_unstale":"2023-10-16T11:20:26.075149+0000","last_undegraded":"2023-10-16T11:20:26.075149+0000","last_fullsized":"2023-10-16T11:20:26.075149+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:11:13.297732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068322+0000","last_change":"2023-10-16T11:20:26.101781+0000","last_active":"2023-10-16T11:20:27.068322+0000","last_peered":"2023-10-16T11:20:27.068322+0000","last_clean":"2023-10-16T11:20:27.068322+0000","last_became_active":"2023-10-16T11:20:26.101470+0000","last_became_peered":"2023-10-16T11:20:26.101470+0000","last_unstale":"2023-10-16T11:20:27.068322+0000","last_undegraded":"2023-10-16T11:20:27.068322+0000","last_fullsized":"2023-10-16T11:20:27.068322+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069206+0000","last_change":"2023-10-16T11:20:25.081336+0000","last_active":"2023-10-16T11:20:26.069206+0000","last_peered":"2023-10-16T11:20:26.069206+0000","last_clean":"2023-10-16T11:20:26.069206+0000","last_became_active":"2023-10-16T11:20:25.081017+0000","last_became_peered":"2023-10-16T11:20:25.081017+0000","last_unstale":"2023-10-16T11:20:26.069206+0000","last_undegraded":"2023-10-16T11:20:26.069206+0000","last_fullsized":"2023-10-16T11:20:26.069206+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:38.114268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698984+0000","last_change":"2023-10-16T11:20:26.085568+0000","last_active":"2023-10-16T11:20:27.698984+0000","last_peered":"2023-10-16T11:20:27.698984+0000","last_clean":"2023-10-16T11:20:27.698984+0000","last_became_active":"2023-10-16T11:20:26.085298+0000","last_became_peered":"2023-10-16T11:20:26.085298+0000","last_unstale":"2023-10-16T11:20:27.698984+0000","last_undegraded":"2023-10-16T11:20:27.698984+0000","last_fullsized":"2023-10-16T11:20:27.698984+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:23:15.106414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.075564+0000","last_change":"2023-10-16T11:20:25.084776+0000","last_active":"2023-10-16T11:20:26.075564+0000","last_peered":"2023-10-16T11:20:26.075564+0000","last_clean":"2023-10-16T11:20:26.075564+0000","last_became_active":"2023-10-16T11:20:25.084415+0000","last_became_peered":"2023-10-16T11:20:25.084415+0000","last_unstale":"2023-10-16T11:20:26.075564+0000","last_undegraded":"2023-10-16T11:20:26.075564+0000","last_fullsized":"2023-10-16T11:20:26.075564+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:43.132428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699314+0000","last_change":"2023-10-16T11:20:26.080094+0000","last_active":"2023-10-16T11:20:27.699314+0000","last_peered":"2023-10-16T11:20:27.699314+0000","last_clean":"2023-10-16T11:20:27.699314+0000","last_became_active":"2023-10-16T11:20:26.079844+0000","last_became_peered":"2023-10-16T11:20:26.079844+0000","last_unstale":"2023-10-16T11:20:27.699314+0000","last_undegraded":"2023-10-16T11:20:27.699314+0000","last_fullsized":"2023-10-16T11:20:27.699314+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:54.114067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.102912+0000","last_change":"2023-10-16T11:20:25.084135+0000","last_active":"2023-10-16T11:20:26.102912+0000","last_peered":"2023-10-16T11:20:26.102912+0000","last_clean":"2023-10-16T11:20:26.102912+0000","last_became_active":"2023-10-16T11:20:25.083794+0000","last_became_peered":"2023-10-16T11:20:25.083794+0000","last_unstale":"2023-10-16T11:20:26.102912+0000","last_undegraded":"2023-10-16T11:20:26.102912+0000","last_fullsized":"2023-10-16T11:20:26.102912+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:57.161061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699220+0000","last_change":"2023-10-16T11:20:26.083428+0000","last_active":"2023-10-16T11:20:27.699220+0000","last_peered":"2023-10-16T11:20:27.699220+0000","last_clean":"2023-10-16T11:20:27.699220+0000","last_became_active":"2023-10-16T11:20:26.082473+0000","last_became_peered":"2023-10-16T11:20:26.082473+0000","last_unstale":"2023-10-16T11:20:27.699220+0000","last_undegraded":"2023-10-16T11:20:27.699220+0000","last_fullsized":"2023-10-16T11:20:27.699220+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:39.122257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.068350+0000","last_change":"2023-10-16T11:20:25.085721+0000","last_active":"2023-10-16T11:20:26.068350+0000","last_peered":"2023-10-16T11:20:26.068350+0000","last_clean":"2023-10-16T11:20:26.068350+0000","last_became_active":"2023-10-16T11:20:25.085421+0000","last_became_peered":"2023-10-16T11:20:25.085421+0000","last_unstale":"2023-10-16T11:20:26.068350+0000","last_undegraded":"2023-10-16T11:20:26.068350+0000","last_fullsized":"2023-10-16T11:20:26.068350+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:08.936039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068680+0000","last_change":"2023-10-16T11:20:26.098738+0000","last_active":"2023-10-16T11:20:27.068680+0000","last_peered":"2023-10-16T11:20:27.068680+0000","last_clean":"2023-10-16T11:20:27.068680+0000","last_became_active":"2023-10-16T11:20:26.098226+0000","last_became_peered":"2023-10-16T11:20:26.098226+0000","last_unstale":"2023-10-16T11:20:27.068680+0000","last_undegraded":"2023-10-16T11:20:27.068680+0000","last_fullsized":"2023-10-16T11:20:27.068680+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:48.218074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069541+0000","last_change":"2023-10-16T11:20:25.081249+0000","last_active":"2023-10-16T11:20:26.069541+0000","last_peered":"2023-10-16T11:20:26.069541+0000","last_clean":"2023-10-16T11:20:26.069541+0000","last_became_active":"2023-10-16T11:20:25.080938+0000","last_became_peered":"2023-10-16T11:20:25.080938+0000","last_unstale":"2023-10-16T11:20:26.069541+0000","last_undegraded":"2023-10-16T11:20:26.069541+0000","last_fullsized":"2023-10-16T11:20:26.069541+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:09.538322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068796+0000","last_change":"2023-10-16T11:20:26.094947+0000","last_active":"2023-10-16T11:20:27.068796+0000","last_peered":"2023-10-16T11:20:27.068796+0000","last_clean":"2023-10-16T11:20:27.068796+0000","last_became_active":"2023-10-16T11:20:26.094613+0000","last_became_peered":"2023-10-16T11:20:26.094613+0000","last_unstale":"2023-10-16T11:20:27.068796+0000","last_undegraded":"2023-10-16T11:20:27.068796+0000","last_fullsized":"2023-10-16T11:20:27.068796+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:24.098521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.088706+0000","last_change":"2023-10-16T11:20:25.082461+0000","last_active":"2023-10-16T11:20:26.088706+0000","last_peered":"2023-10-16T11:20:26.088706+0000","last_clean":"2023-10-16T11:20:26.088706+0000","last_became_active":"2023-10-16T11:20:25.082161+0000","last_became_peered":"2023-10-16T11:20:25.082161+0000","last_unstale":"2023-10-16T11:20:26.088706+0000","last_undegraded":"2023-10-16T11:20:26.088706+0000","last_fullsized":"2023-10-16T11:20:26.088706+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:13.522561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699176+0000","last_change":"2023-10-16T11:20:26.083434+0000","last_active":"2023-10-16T11:20:27.699176+0000","last_peered":"2023-10-16T11:20:27.699176+0000","last_clean":"2023-10-16T11:20:27.699176+0000","last_became_active":"2023-10-16T11:20:26.083146+0000","last_became_peered":"2023-10-16T11:20:26.083146+0000","last_unstale":"2023-10-16T11:20:27.699176+0000","last_undegraded":"2023-10-16T11:20:27.699176+0000","last_fullsized":"2023-10-16T11:20:27.699176+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:36.022948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.075295+0000","last_change":"2023-10-16T11:20:25.084296+0000","last_active":"2023-10-16T11:20:26.075295+0000","last_peered":"2023-10-16T11:20:26.075295+0000","last_clean":"2023-10-16T11:20:26.075295+0000","last_became_active":"2023-10-16T11:20:25.083831+0000","last_became_peered":"2023-10-16T11:20:25.083831+0000","last_unstale":"2023-10-16T11:20:26.075295+0000","last_undegraded":"2023-10-16T11:20:26.075295+0000","last_fullsized":"2023-10-16T11:20:26.075295+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:15.128952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699408+0000","last_change":"2023-10-16T11:20:26.097299+0000","last_active":"2023-10-16T11:20:27.699408+0000","last_peered":"2023-10-16T11:20:27.699408+0000","last_clean":"2023-10-16T11:20:27.699408+0000","last_became_active":"2023-10-16T11:20:26.096992+0000","last_became_peered":"2023-10-16T11:20:26.096992+0000","last_unstale":"2023-10-16T11:20:27.699408+0000","last_undegraded":"2023-10-16T11:20:27.699408+0000","last_fullsized":"2023-10-16T11:20:27.699408+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:01.203662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.079512+0000","last_change":"2023-10-16T11:20:25.085431+0000","last_active":"2023-10-16T11:20:26.079512+0000","last_peered":"2023-10-16T11:20:26.079512+0000","last_clean":"2023-10-16T11:20:26.079512+0000","last_became_active":"2023-10-16T11:20:25.084034+0000","last_became_peered":"2023-10-16T11:20:25.084034+0000","last_unstale":"2023-10-16T11:20:26.079512+0000","last_undegraded":"2023-10-16T11:20:26.079512+0000","last_fullsized":"2023-10-16T11:20:26.079512+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:44.288602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698642+0000","last_change":"2023-10-16T11:20:26.082969+0000","last_active":"2023-10-16T11:20:27.698642+0000","last_peered":"2023-10-16T11:20:27.698642+0000","last_clean":"2023-10-16T11:20:27.698642+0000","last_became_active":"2023-10-16T11:20:26.082661+0000","last_became_peered":"2023-10-16T11:20:26.082661+0000","last_unstale":"2023-10-16T11:20:27.698642+0000","last_undegraded":"2023-10-16T11:20:27.698642+0000","last_fullsized":"2023-10-16T11:20:27.698642+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:53.296236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069530+0000","last_change":"2023-10-16T11:20:25.080407+0000","last_active":"2023-10-16T11:20:26.069530+0000","last_peered":"2023-10-16T11:20:26.069530+0000","last_clean":"2023-10-16T11:20:26.069530+0000","last_became_active":"2023-10-16T11:20:25.079543+0000","last_became_peered":"2023-10-16T11:20:25.079543+0000","last_unstale":"2023-10-16T11:20:26.069530+0000","last_undegraded":"2023-10-16T11:20:26.069530+0000","last_fullsized":"2023-10-16T11:20:26.069530+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:12.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698544+0000","last_change":"2023-10-16T11:20:26.086262+0000","last_active":"2023-10-16T11:20:27.698544+0000","last_peered":"2023-10-16T11:20:27.698544+0000","last_clean":"2023-10-16T11:20:27.698544+0000","last_became_active":"2023-10-16T11:20:26.085901+0000","last_became_peered":"2023-10-16T11:20:26.085901+0000","last_unstale":"2023-10-16T11:20:27.698544+0000","last_undegraded":"2023-10-16T11:20:27.698544+0000","last_fullsized":"2023-10-16T11:20:27.698544+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:56.664462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.070135+0000","last_change":"2023-10-16T11:20:25.086844+0000","last_active":"2023-10-16T11:20:26.070135+0000","last_peered":"2023-10-16T11:20:26.070135+0000","last_clean":"2023-10-16T11:20:26.070135+0000","last_became_active":"2023-10-16T11:20:25.086570+0000","last_became_peered":"2023-10-16T11:20:25.086570+0000","last_unstale":"2023-10-16T11:20:26.070135+0000","last_undegraded":"2023-10-16T11:20:26.070135+0000","last_fullsized":"2023-10-16T11:20:26.070135+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:17.084881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070094+0000","last_change":"2023-10-16T11:20:26.078189+0000","last_active":"2023-10-16T11:20:27.070094+0000","last_peered":"2023-10-16T11:20:27.070094+0000","last_clean":"2023-10-16T11:20:27.070094+0000","last_became_active":"2023-10-16T11:20:26.077754+0000","last_became_peered":"2023-10-16T11:20:26.077754+0000","last_unstale":"2023-10-16T11:20:27.070094+0000","last_undegraded":"2023-10-16T11:20:27.070094+0000","last_fullsized":"2023-10-16T11:20:27.070094+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:45.296590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"235'4","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.099065+0000","last_change":"2023-10-16T11:20:25.087446+0000","last_active":"2023-10-16T11:20:26.099065+0000","last_peered":"2023-10-16T11:20:26.099065+0000","last_clean":"2023-10-16T11:20:26.099065+0000","last_became_active":"2023-10-16T11:20:25.086617+0000","last_became_peered":"2023-10-16T11:20:25.086617+0000","last_unstale":"2023-10-16T11:20:26.099065+0000","last_undegraded":"2023-10-16T11:20:26.099065+0000","last_fullsized":"2023-10-16T11:20:26.099065+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:50.072765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'370","reported_seq":1308,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069113+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:20:26.069113+0000","last_peered":"2023-10-16T11:20:26.069113+0000","last_clean":"2023-10-16T11:20:26.069113+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:20:26.069113+0000","last_undegraded":"2023-10-16T11:20:26.069113+0000","last_fullsized":"2023-10-16T11:20:26.069113+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543290,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52720,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319120,"statfs":{"total":96636764160,"available":96582778880,"internally_reserved":0,"allocated":3567616,"data_stored":2123854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.85699999999999998,"5min":0.878,"15min":0.878},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.27800000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.33300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.83399999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":1,"5min":1,"15min":1.002},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.33100000000000002},"max":{"1min":0.93500000000000005,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.628}]},{"osd":6,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70899999999999996}]}]},{"osd":6,"up_from":14,"seq":60129543289,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56296,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315544,"statfs":{"total":96636764160,"available":96579117056,"internally_reserved":0,"allocated":2977792,"data_stored":1534020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29899999999999999},"max":{"1min":0.66700000000000004,"5min":0.81699999999999995,"15min":0.84499999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83099999999999996,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.42599999999999999,"5min":0.40799999999999997,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68799999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76300000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54500000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.373,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129543289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60468,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311372,"statfs":{"total":96636764160,"available":96574844928,"internally_reserved":0,"allocated":2990080,"data_stored":1535152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39700000000000002,"15min":0.40600000000000003},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51800000000000002,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.40600000000000003,"15min":0.43099999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.49199999999999999,"5min":0.62,"15min":0.91700000000000004},"last":0.33200000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40200000000000002,"15min":0.41699999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45000000000000001,"5min":0.58599999999999997,"15min":0.98899999999999999},"last":0.314},{"interface":"front","average":{"1min":0.434,"5min":0.441,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63200000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.34899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46400000000000002,"15min":0.49199999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.78300000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.57999999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.36399999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77200000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.621},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85199999999999998,"5min":1.0289999999999999,"15min":1.097},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85499999999999998,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.495},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.90100000000000002,"15min":1.147},"last":0.53400000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.98699999999999999,"15min":1.0509999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.70299999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":1.0900000000000001},"last":0.52200000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":1.004,"15min":1.0600000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.50600000000000001}]}]},{"osd":0,"up_from":14,"seq":60129543290,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61032,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310808,"statfs":{"total":96636764160,"available":96574267392,"internally_reserved":0,"allocated":3567616,"data_stored":2123818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.46400000000000002,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58799999999999997,"5min":0.63700000000000001,"15min":0.70299999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.68300000000000005},"last":0.314},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.377}]},{"osd":4,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.99199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":1.008},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":1.004,"15min":1.008},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56499999999999995}]}]},{"osd":2,"up_from":14,"seq":60129543289,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47972,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323868,"statfs":{"total":96636764160,"available":96587640832,"internally_reserved":0,"allocated":2973696,"data_stored":1533506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.40500000000000003},"min":{"1min":0.24399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.28999999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.28499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.35099999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.436},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.4929999999999999,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.307}]},{"osd":4,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.6779999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.6549999999999998,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.7320000000000002,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.58099999999999996}]}]},{"osd":3,"up_from":14,"seq":60129543290,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47980,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323860,"statfs":{"total":96636764160,"available":96587632640,"internally_reserved":0,"allocated":2973696,"data_stored":1533998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.59799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.52600000000000002,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.32800000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.40600000000000003,"15min":0.42499999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58099999999999996,"5min":0.87,"15min":0.87},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54800000000000004,"5min":0.65100000000000002,"15min":0.76300000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.44400000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.49099999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.46400000000000002,"15min":0.48299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56100000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.496},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.65300000000000002},"min":{"1min":0.39900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.621}]},{"osd":5,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.53900000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65100000000000002,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.52800000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.66400000000000003},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96399999999999997,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.59199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543289,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78452,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94293388,"statfs":{"total":96636764160,"available":96556429312,"internally_reserved":0,"allocated":2990080,"data_stored":1535120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.74399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60399999999999998,"15min":0.61899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71899999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54000000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.83399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.628,"15min":0.63400000000000001},"min":{"1min":0.5,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":0.745,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.64200000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.879},"last":0.65300000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.52000000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.875,"15min":0.875},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.73499999999999999,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.55500000000000005}]}]},{"osd":5,"up_from":14,"seq":60129543290,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60456,"kb_used_data":2900,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311384,"statfs":{"total":96636764160,"available":96574857216,"internally_reserved":0,"allocated":2969600,"data_stored":1533416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":1.873,"15min":1.873},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.41999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.373,"15min":0.373},"max":{"1min":0.745,"5min":1.024,"15min":1.024},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.626,"15min":0.628},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73999999999999999,"5min":1.821,"15min":1.821},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79900000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.40200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":1.919,"15min":1.919},"last":0.76000000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.432},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94299999999999995,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.96899999999999997,"5min":2.02,"15min":2.02},"last":0.60299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:20:30.199 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:20:30.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.352+0000 7f58ae191700 1 Processor -- start 2023-10-16T11:20:30.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.352+0000 7f58ae191700 1 -- start start 2023-10-16T11:20:30.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58ae191700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58a8159730 0x7f58a815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58ae191700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80cd5c0 0x7f58a80cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58ae191700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a80cee70 con 0x7f58a80cc400 2023-10-16T11:20:30.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58ae191700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a8159b10 con 0x7f58a8159730 2023-10-16T11:20:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58ae191700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80ce540 con 0x7f58a80cd5c0 2023-10-16T11:20:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58a8159730 0x7f58a815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58a8159730 0x7f58a815cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57812/0 (socket says 172.21.15.181:57812) 2023-10-16T11:20:30.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 -- 172.21.15.181:0/102807685 learned_addr learned my addr 172.21.15.181:0/102807685 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a7fff700 1 --2- 172.21.15.181:0/102807685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80cd5c0 0x7f58a80cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 -- 172.21.15.181:0/102807685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80cd5c0 msgr2=0x7f58a80cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 --2- 172.21.15.181:0/102807685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80cd5c0 0x7f58a80cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 -- 172.21.15.181:0/102807685 >> v1:172.21.15.190:6789/0 conn(0x7f58a80cc400 legacy=0x7f58a80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 -- 172.21.15.181:0/102807685 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80ce820 con 0x7f58a8159730 2023-10-16T11:20:30.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.353+0000 7f58a77fe700 1 --2- 172.21.15.181:0/102807685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8159730 0x7f58a815cd20 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f589c006fd0 tx=0x7f589c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af89c3aebcf79d30 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:30.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58a67fc700 1 -- 172.21.15.181:0/102807685 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c005e30 con 0x7f58a8159730 2023-10-16T11:20:30.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58a67fc700 1 -- 172.21.15.181:0/102807685 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c006030 con 0x7f58a8159730 2023-10-16T11:20:30.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58a67fc700 1 -- 172.21.15.181:0/102807685 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0063c0 con 0x7f58a8159730 2023-10-16T11:20:30.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 -- 172.21.15.181:0/102807685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8159730 msgr2=0x7f58a815cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 --2- 172.21.15.181:0/102807685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8159730 0x7f58a815cd20 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f589c006fd0 tx=0x7f589c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 -- 172.21.15.181:0/102807685 shutdown_connections 2023-10-16T11:20:30.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 --2- 172.21.15.181:0/102807685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80cd5c0 0x7f58a80cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 --2- 172.21.15.181:0/102807685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8159730 0x7f58a815cd20 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 -- 172.21.15.181:0/102807685 >> 172.21.15.181:0/102807685 conn(0x7f58a80d04a0 msgr2=0x7f58a80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:30.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 -- 172.21.15.181:0/102807685 shutdown_connections 2023-10-16T11:20:30.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.354+0000 7f58ae191700 1 -- 172.21.15.181:0/102807685 wait complete. 2023-10-16T11:20:30.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.356+0000 7f58ae191700 1 Processor -- start 2023-10-16T11:20:30.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.356+0000 7f58ae191700 1 -- start start 2023-10-16T11:20:30.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58ae191700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80cc400 0x7f58a8157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58ae191700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a80cd5c0 0x7f58a81f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58ae191700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a8159730 0x7f58a81f0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58ae191700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80c76a0 con 0x7f58a80cd5c0 2023-10-16T11:20:30.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58ae191700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8156e00 con 0x7f58a80cc400 2023-10-16T11:20:30.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58ae191700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a81f56b0 con 0x7f58a8159730 2023-10-16T11:20:30.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a80cd5c0 0x7f58a81f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a80cd5c0 0x7f58a81f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57820/0 (socket says 172.21.15.181:57820) 2023-10-16T11:20:30.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 -- 172.21.15.181:0/1413805141 learned_addr learned my addr 172.21.15.181:0/1413805141 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:30.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a7fff700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a8159730 0x7f58a81f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a77fe700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80cc400 0x7f58a8157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 -- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a8159730 msgr2=0x7f58a81f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a8159730 0x7f58a81f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 -- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80cc400 msgr2=0x7f58a8157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80cc400 0x7f58a8157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.357+0000 7f58a6ffd700 1 -- 172.21.15.181:0/1413805141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c009500 con 0x7f58a80cd5c0 2023-10-16T11:20:30.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.358+0000 7f58a6ffd700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a80cd5c0 0x7f58a81f47a0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f589800c4d0 tx=0x7f589800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:30.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.358+0000 7f588ffff700 1 -- 172.21.15.181:0/1413805141 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898011790 con 0x7f58a80cd5c0 2023-10-16T11:20:30.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.358+0000 7f588ffff700 1 -- 172.21.15.181:0/1413805141 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58980118f0 con 0x7f58a80cd5c0 2023-10-16T11:20:30.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.358+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a81f4e10 con 0x7f58a80cd5c0 2023-10-16T11:20:30.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.358+0000 7f588ffff700 1 -- 172.21.15.181:0/1413805141 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898011ca0 con 0x7f58a80cd5c0 2023-10-16T11:20:30.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.358+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a81f53d0 con 0x7f58a80cd5c0 2023-10-16T11:20:30.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.359+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a80c7850 con 0x7f58a80cd5c0 2023-10-16T11:20:30.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.359+0000 7f588ffff700 1 -- 172.21.15.181:0/1413805141 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f589801c460 con 0x7f58a80cd5c0 2023-10-16T11:20:30.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.360+0000 7f588ffff700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f589406b140 0x7f589406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.360+0000 7f588ffff700 1 -- 172.21.15.181:0/1413805141 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f58980899b0 con 0x7f58a80cd5c0 2023-10-16T11:20:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.360+0000 7f58a77fe700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f589406b140 0x7f589406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.360+0000 7f58a77fe700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f589406b140 0x7f589406d600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f589c013040 tx=0x7f589c008070 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.364+0000 7f588ffff700 1 -- 172.21.15.181:0/1413805141 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f58a80c7850 con 0x7f58a80cd5c0 2023-10-16T11:20:30.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.550+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 --> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58a80ceb00 con 0x7f589406b140 2023-10-16T11:20:30.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.558+0000 7f588ffff700 1 -- 172.21.15.181:0/1413805141 <== mgr.11814 v2:172.21.15.181:6832/3273985848 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290178 (secure 0 0 0) 0x7f589c006cb0 con 0x7f589406b140 2023-10-16T11:20:30.564 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f589406b140 msgr2=0x7f589406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f589406b140 0x7f589406d600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f589c013040 tx=0x7f589c008070 comp rx=0 tx=0).stop 2023-10-16T11:20:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a80cd5c0 msgr2=0x7f58a81f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a80cd5c0 0x7f58a81f47a0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f589800c4d0 tx=0x7f589800c890 comp rx=0 tx=0).stop 2023-10-16T11:20:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 shutdown_connections 2023-10-16T11:20:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f589406b140 0x7f589406d600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a8159730 0x7f58a81f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80cc400 0x7f58a8157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 --2- 172.21.15.181:0/1413805141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a80cd5c0 0x7f58a81f47a0 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.561+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 >> 172.21.15.181:0/1413805141 conn(0x7f58a80d04a0 msgr2=0x7f58a8158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.562+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 shutdown_connections 2023-10-16T11:20:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.562+0000 7f58ae191700 1 -- 172.21.15.181:0/1413805141 wait complete. 2023-10-16T11:20:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:20:30.575 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-16T11:20:29.754825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":663,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":408,"ondisk_log_size":408,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465376,"kb_used_data":24424,"kb_used_omap":12,"kb_used_meta":440819,"kb_avail":754509344,"statfs":{"total":773094113280,"available":772617568256,"internally_reserved":0,"allocated":25010176,"data_stored":13452884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":451399224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.701151"},"pg_stats":[{"pgid":"31.3f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699092+0000","last_change":"2023-10-16T11:20:26.092187+0000","last_active":"2023-10-16T11:20:27.699092+0000","last_peered":"2023-10-16T11:20:27.699092+0000","last_clean":"2023-10-16T11:20:27.699092+0000","last_became_active":"2023-10-16T11:20:26.091884+0000","last_became_peered":"2023-10-16T11:20:26.091884+0000","last_unstale":"2023-10-16T11:20:27.699092+0000","last_undegraded":"2023-10-16T11:20:27.699092+0000","last_fullsized":"2023-10-16T11:20:27.699092+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:16.851198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698496+0000","last_change":"2023-10-16T11:20:26.085301+0000","last_active":"2023-10-16T11:20:27.698496+0000","last_peered":"2023-10-16T11:20:27.698496+0000","last_clean":"2023-10-16T11:20:27.698496+0000","last_became_active":"2023-10-16T11:20:26.084922+0000","last_became_peered":"2023-10-16T11:20:26.084922+0000","last_unstale":"2023-10-16T11:20:27.698496+0000","last_undegraded":"2023-10-16T11:20:27.698496+0000","last_fullsized":"2023-10-16T11:20:27.698496+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:36.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698594+0000","last_change":"2023-10-16T11:20:26.083255+0000","last_active":"2023-10-16T11:20:27.698594+0000","last_peered":"2023-10-16T11:20:27.698594+0000","last_clean":"2023-10-16T11:20:27.698594+0000","last_became_active":"2023-10-16T11:20:26.082915+0000","last_became_peered":"2023-10-16T11:20:26.082915+0000","last_unstale":"2023-10-16T11:20:27.698594+0000","last_undegraded":"2023-10-16T11:20:27.698594+0000","last_fullsized":"2023-10-16T11:20:27.698594+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:02.635859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699351+0000","last_change":"2023-10-16T11:20:26.084494+0000","last_active":"2023-10-16T11:20:27.699351+0000","last_peered":"2023-10-16T11:20:27.699351+0000","last_clean":"2023-10-16T11:20:27.699351+0000","last_became_active":"2023-10-16T11:20:26.084169+0000","last_became_peered":"2023-10-16T11:20:26.084169+0000","last_unstale":"2023-10-16T11:20:27.699351+0000","last_undegraded":"2023-10-16T11:20:27.699351+0000","last_fullsized":"2023-10-16T11:20:27.699351+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070698+0000","last_change":"2023-10-16T11:20:26.080175+0000","last_active":"2023-10-16T11:20:27.070698+0000","last_peered":"2023-10-16T11:20:27.070698+0000","last_clean":"2023-10-16T11:20:27.070698+0000","last_became_active":"2023-10-16T11:20:26.079884+0000","last_became_peered":"2023-10-16T11:20:26.079884+0000","last_unstale":"2023-10-16T11:20:27.070698+0000","last_undegraded":"2023-10-16T11:20:27.070698+0000","last_fullsized":"2023-10-16T11:20:27.070698+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:58.169049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699041+0000","last_change":"2023-10-16T11:20:26.099739+0000","last_active":"2023-10-16T11:20:27.699041+0000","last_peered":"2023-10-16T11:20:27.699041+0000","last_clean":"2023-10-16T11:20:27.699041+0000","last_became_active":"2023-10-16T11:20:26.099459+0000","last_became_peered":"2023-10-16T11:20:26.099459+0000","last_unstale":"2023-10-16T11:20:27.699041+0000","last_undegraded":"2023-10-16T11:20:27.699041+0000","last_fullsized":"2023-10-16T11:20:27.699041+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:14.530550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698719+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:27.698719+0000","last_peered":"2023-10-16T11:20:27.698719+0000","last_clean":"2023-10-16T11:20:27.698719+0000","last_became_active":"2023-10-16T11:20:26.098864+0000","last_became_peered":"2023-10-16T11:20:26.098864+0000","last_unstale":"2023-10-16T11:20:27.698719+0000","last_undegraded":"2023-10-16T11:20:27.698719+0000","last_fullsized":"2023-10-16T11:20:27.698719+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:04.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698487+0000","last_change":"2023-10-16T11:20:26.084562+0000","last_active":"2023-10-16T11:20:27.698487+0000","last_peered":"2023-10-16T11:20:27.698487+0000","last_clean":"2023-10-16T11:20:27.698487+0000","last_became_active":"2023-10-16T11:20:26.083055+0000","last_became_peered":"2023-10-16T11:20:26.083055+0000","last_unstale":"2023-10-16T11:20:27.698487+0000","last_undegraded":"2023-10-16T11:20:27.698487+0000","last_fullsized":"2023-10-16T11:20:27.698487+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.947551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070479+0000","last_change":"2023-10-16T11:20:26.077403+0000","last_active":"2023-10-16T11:20:27.070479+0000","last_peered":"2023-10-16T11:20:27.070479+0000","last_clean":"2023-10-16T11:20:27.070479+0000","last_became_active":"2023-10-16T11:20:26.077133+0000","last_became_peered":"2023-10-16T11:20:26.077133+0000","last_unstale":"2023-10-16T11:20:27.070479+0000","last_undegraded":"2023-10-16T11:20:27.070479+0000","last_fullsized":"2023-10-16T11:20:27.070479+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:15.067913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699656+0000","last_change":"2023-10-16T11:20:26.092685+0000","last_active":"2023-10-16T11:20:27.699656+0000","last_peered":"2023-10-16T11:20:27.699656+0000","last_clean":"2023-10-16T11:20:27.699656+0000","last_became_active":"2023-10-16T11:20:26.092413+0000","last_became_peered":"2023-10-16T11:20:26.092413+0000","last_unstale":"2023-10-16T11:20:27.699656+0000","last_undegraded":"2023-10-16T11:20:27.699656+0000","last_fullsized":"2023-10-16T11:20:27.699656+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:40.852817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068614+0000","last_change":"2023-10-16T11:20:26.094837+0000","last_active":"2023-10-16T11:20:27.068614+0000","last_peered":"2023-10-16T11:20:27.068614+0000","last_clean":"2023-10-16T11:20:27.068614+0000","last_became_active":"2023-10-16T11:20:26.094247+0000","last_became_peered":"2023-10-16T11:20:26.094247+0000","last_unstale":"2023-10-16T11:20:27.068614+0000","last_undegraded":"2023-10-16T11:20:27.068614+0000","last_fullsized":"2023-10-16T11:20:27.068614+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:16.136940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699661+0000","last_change":"2023-10-16T11:20:26.091607+0000","last_active":"2023-10-16T11:20:27.699661+0000","last_peered":"2023-10-16T11:20:27.699661+0000","last_clean":"2023-10-16T11:20:27.699661+0000","last_became_active":"2023-10-16T11:20:26.091336+0000","last_became_peered":"2023-10-16T11:20:26.091336+0000","last_unstale":"2023-10-16T11:20:27.699661+0000","last_undegraded":"2023-10-16T11:20:27.699661+0000","last_fullsized":"2023-10-16T11:20:27.699661+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:47.664336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699023+0000","last_change":"2023-10-16T11:20:26.101354+0000","last_active":"2023-10-16T11:20:27.699023+0000","last_peered":"2023-10-16T11:20:27.699023+0000","last_clean":"2023-10-16T11:20:27.699023+0000","last_became_active":"2023-10-16T11:20:26.101082+0000","last_became_peered":"2023-10-16T11:20:26.101082+0000","last_unstale":"2023-10-16T11:20:27.699023+0000","last_undegraded":"2023-10-16T11:20:27.699023+0000","last_fullsized":"2023-10-16T11:20:27.699023+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:48.559039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698824+0000","last_change":"2023-10-16T11:20:26.087929+0000","last_active":"2023-10-16T11:20:27.698824+0000","last_peered":"2023-10-16T11:20:27.698824+0000","last_clean":"2023-10-16T11:20:27.698824+0000","last_became_active":"2023-10-16T11:20:26.087611+0000","last_became_peered":"2023-10-16T11:20:26.087611+0000","last_unstale":"2023-10-16T11:20:27.698824+0000","last_undegraded":"2023-10-16T11:20:27.698824+0000","last_fullsized":"2023-10-16T11:20:27.698824+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:47.826756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070154+0000","last_change":"2023-10-16T11:20:26.076097+0000","last_active":"2023-10-16T11:20:27.070154+0000","last_peered":"2023-10-16T11:20:27.070154+0000","last_clean":"2023-10-16T11:20:27.070154+0000","last_became_active":"2023-10-16T11:20:26.075864+0000","last_became_peered":"2023-10-16T11:20:26.075864+0000","last_unstale":"2023-10-16T11:20:27.070154+0000","last_undegraded":"2023-10-16T11:20:27.070154+0000","last_fullsized":"2023-10-16T11:20:27.070154+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:36.478137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698402+0000","last_change":"2023-10-16T11:20:26.084690+0000","last_active":"2023-10-16T11:20:27.698402+0000","last_peered":"2023-10-16T11:20:27.698402+0000","last_clean":"2023-10-16T11:20:27.698402+0000","last_became_active":"2023-10-16T11:20:26.084390+0000","last_became_peered":"2023-10-16T11:20:26.084390+0000","last_unstale":"2023-10-16T11:20:27.698402+0000","last_undegraded":"2023-10-16T11:20:27.698402+0000","last_fullsized":"2023-10-16T11:20:27.698402+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:14.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699590+0000","last_change":"2023-10-16T11:20:26.089392+0000","last_active":"2023-10-16T11:20:27.699590+0000","last_peered":"2023-10-16T11:20:27.699590+0000","last_clean":"2023-10-16T11:20:27.699590+0000","last_became_active":"2023-10-16T11:20:26.089083+0000","last_became_peered":"2023-10-16T11:20:26.089083+0000","last_unstale":"2023-10-16T11:20:27.699590+0000","last_undegraded":"2023-10-16T11:20:27.699590+0000","last_fullsized":"2023-10-16T11:20:27.699590+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:16.986811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698635+0000","last_change":"2023-10-16T11:20:26.084395+0000","last_active":"2023-10-16T11:20:27.698635+0000","last_peered":"2023-10-16T11:20:27.698635+0000","last_clean":"2023-10-16T11:20:27.698635+0000","last_became_active":"2023-10-16T11:20:26.084119+0000","last_became_peered":"2023-10-16T11:20:26.084119+0000","last_unstale":"2023-10-16T11:20:27.698635+0000","last_undegraded":"2023-10-16T11:20:27.698635+0000","last_fullsized":"2023-10-16T11:20:27.698635+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.234510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699495+0000","last_change":"2023-10-16T11:20:26.095722+0000","last_active":"2023-10-16T11:20:27.699495+0000","last_peered":"2023-10-16T11:20:27.699495+0000","last_clean":"2023-10-16T11:20:27.699495+0000","last_became_active":"2023-10-16T11:20:26.095489+0000","last_became_peered":"2023-10-16T11:20:26.095489+0000","last_unstale":"2023-10-16T11:20:27.699495+0000","last_undegraded":"2023-10-16T11:20:27.699495+0000","last_fullsized":"2023-10-16T11:20:27.699495+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:32.623008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698700+0000","last_change":"2023-10-16T11:20:26.072060+0000","last_active":"2023-10-16T11:20:27.698700+0000","last_peered":"2023-10-16T11:20:27.698700+0000","last_clean":"2023-10-16T11:20:27.698700+0000","last_became_active":"2023-10-16T11:20:26.071739+0000","last_became_peered":"2023-10-16T11:20:26.071739+0000","last_unstale":"2023-10-16T11:20:27.698700+0000","last_undegraded":"2023-10-16T11:20:27.698700+0000","last_fullsized":"2023-10-16T11:20:27.698700+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:29.585136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699338+0000","last_change":"2023-10-16T11:20:26.083156+0000","last_active":"2023-10-16T11:20:27.699338+0000","last_peered":"2023-10-16T11:20:27.699338+0000","last_clean":"2023-10-16T11:20:27.699338+0000","last_became_active":"2023-10-16T11:20:26.082813+0000","last_became_peered":"2023-10-16T11:20:26.082813+0000","last_unstale":"2023-10-16T11:20:27.699338+0000","last_undegraded":"2023-10-16T11:20:27.699338+0000","last_fullsized":"2023-10-16T11:20:27.699338+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:26.407654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698680+0000","last_change":"2023-10-16T11:20:26.097008+0000","last_active":"2023-10-16T11:20:27.698680+0000","last_peered":"2023-10-16T11:20:27.698680+0000","last_clean":"2023-10-16T11:20:27.698680+0000","last_became_active":"2023-10-16T11:20:26.096754+0000","last_became_peered":"2023-10-16T11:20:26.096754+0000","last_unstale":"2023-10-16T11:20:27.698680+0000","last_undegraded":"2023-10-16T11:20:27.698680+0000","last_fullsized":"2023-10-16T11:20:27.698680+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:24.400518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698569+0000","last_change":"2023-10-16T11:20:26.093039+0000","last_active":"2023-10-16T11:20:27.698569+0000","last_peered":"2023-10-16T11:20:27.698569+0000","last_clean":"2023-10-16T11:20:27.698569+0000","last_became_active":"2023-10-16T11:20:26.092725+0000","last_became_peered":"2023-10-16T11:20:26.092725+0000","last_unstale":"2023-10-16T11:20:27.698569+0000","last_undegraded":"2023-10-16T11:20:27.698569+0000","last_fullsized":"2023-10-16T11:20:27.698569+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:37.794560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698256+0000","last_change":"2023-10-16T11:20:26.091655+0000","last_active":"2023-10-16T11:20:27.698256+0000","last_peered":"2023-10-16T11:20:27.698256+0000","last_clean":"2023-10-16T11:20:27.698256+0000","last_became_active":"2023-10-16T11:20:26.091402+0000","last_became_peered":"2023-10-16T11:20:26.091402+0000","last_unstale":"2023-10-16T11:20:27.698256+0000","last_undegraded":"2023-10-16T11:20:27.698256+0000","last_fullsized":"2023-10-16T11:20:27.698256+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:25.104508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698667+0000","last_change":"2023-10-16T11:20:26.088880+0000","last_active":"2023-10-16T11:20:27.698667+0000","last_peered":"2023-10-16T11:20:27.698667+0000","last_clean":"2023-10-16T11:20:27.698667+0000","last_became_active":"2023-10-16T11:20:26.088427+0000","last_became_peered":"2023-10-16T11:20:26.088427+0000","last_unstale":"2023-10-16T11:20:27.698667+0000","last_undegraded":"2023-10-16T11:20:27.698667+0000","last_fullsized":"2023-10-16T11:20:27.698667+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:30.068691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698978+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:27.698978+0000","last_peered":"2023-10-16T11:20:27.698978+0000","last_clean":"2023-10-16T11:20:27.698978+0000","last_became_active":"2023-10-16T11:20:26.098861+0000","last_became_peered":"2023-10-16T11:20:26.098861+0000","last_unstale":"2023-10-16T11:20:27.698978+0000","last_undegraded":"2023-10-16T11:20:27.698978+0000","last_fullsized":"2023-10-16T11:20:27.698978+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:03.247595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068750+0000","last_change":"2023-10-16T11:20:26.098406+0000","last_active":"2023-10-16T11:20:27.068750+0000","last_peered":"2023-10-16T11:20:27.068750+0000","last_clean":"2023-10-16T11:20:27.068750+0000","last_became_active":"2023-10-16T11:20:26.098082+0000","last_became_peered":"2023-10-16T11:20:26.098082+0000","last_unstale":"2023-10-16T11:20:27.068750+0000","last_undegraded":"2023-10-16T11:20:27.068750+0000","last_fullsized":"2023-10-16T11:20:27.068750+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:44.140416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698615+0000","last_change":"2023-10-16T11:20:26.082815+0000","last_active":"2023-10-16T11:20:27.698615+0000","last_peered":"2023-10-16T11:20:27.698615+0000","last_clean":"2023-10-16T11:20:27.698615+0000","last_became_active":"2023-10-16T11:20:26.082524+0000","last_became_peered":"2023-10-16T11:20:26.082524+0000","last_unstale":"2023-10-16T11:20:27.698615+0000","last_undegraded":"2023-10-16T11:20:27.698615+0000","last_fullsized":"2023-10-16T11:20:27.698615+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:16.774780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068714+0000","last_change":"2023-10-16T11:20:26.100460+0000","last_active":"2023-10-16T11:20:27.068714+0000","last_peered":"2023-10-16T11:20:27.068714+0000","last_clean":"2023-10-16T11:20:27.068714+0000","last_became_active":"2023-10-16T11:20:26.100088+0000","last_became_peered":"2023-10-16T11:20:26.100088+0000","last_unstale":"2023-10-16T11:20:27.068714+0000","last_undegraded":"2023-10-16T11:20:27.068714+0000","last_fullsized":"2023-10-16T11:20:27.068714+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:38.907350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070022+0000","last_change":"2023-10-16T11:20:26.070717+0000","last_active":"2023-10-16T11:20:27.070022+0000","last_peered":"2023-10-16T11:20:27.070022+0000","last_clean":"2023-10-16T11:20:27.070022+0000","last_became_active":"2023-10-16T11:20:26.070394+0000","last_became_peered":"2023-10-16T11:20:26.070394+0000","last_unstale":"2023-10-16T11:20:27.070022+0000","last_undegraded":"2023-10-16T11:20:27.070022+0000","last_fullsized":"2023-10-16T11:20:27.070022+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698387+0000","last_change":"2023-10-16T11:20:26.086782+0000","last_active":"2023-10-16T11:20:27.698387+0000","last_peered":"2023-10-16T11:20:27.698387+0000","last_clean":"2023-10-16T11:20:27.698387+0000","last_became_active":"2023-10-16T11:20:26.086510+0000","last_became_peered":"2023-10-16T11:20:26.086510+0000","last_unstale":"2023-10-16T11:20:27.698387+0000","last_undegraded":"2023-10-16T11:20:27.698387+0000","last_fullsized":"2023-10-16T11:20:27.698387+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:26:24.730788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698945+0000","last_change":"2023-10-16T11:20:26.083094+0000","last_active":"2023-10-16T11:20:27.698945+0000","last_peered":"2023-10-16T11:20:27.698945+0000","last_clean":"2023-10-16T11:20:27.698945+0000","last_became_active":"2023-10-16T11:20:26.080031+0000","last_became_peered":"2023-10-16T11:20:26.080031+0000","last_unstale":"2023-10-16T11:20:27.698945+0000","last_undegraded":"2023-10-16T11:20:27.698945+0000","last_fullsized":"2023-10-16T11:20:27.698945+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:06:55.285592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699365+0000","last_change":"2023-10-16T11:20:26.096214+0000","last_active":"2023-10-16T11:20:27.699365+0000","last_peered":"2023-10-16T11:20:27.699365+0000","last_clean":"2023-10-16T11:20:27.699365+0000","last_became_active":"2023-10-16T11:20:26.093850+0000","last_became_peered":"2023-10-16T11:20:26.093850+0000","last_unstale":"2023-10-16T11:20:27.699365+0000","last_undegraded":"2023-10-16T11:20:27.699365+0000","last_fullsized":"2023-10-16T11:20:27.699365+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:48.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.089287+0000","last_change":"2023-10-16T11:20:25.084166+0000","last_active":"2023-10-16T11:20:26.089287+0000","last_peered":"2023-10-16T11:20:26.089287+0000","last_clean":"2023-10-16T11:20:26.089287+0000","last_became_active":"2023-10-16T11:20:25.083857+0000","last_became_peered":"2023-10-16T11:20:25.083857+0000","last_unstale":"2023-10-16T11:20:26.089287+0000","last_undegraded":"2023-10-16T11:20:26.089287+0000","last_fullsized":"2023-10-16T11:20:26.089287+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:47.210086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068575+0000","last_change":"2023-10-16T11:20:26.094539+0000","last_active":"2023-10-16T11:20:27.068575+0000","last_peered":"2023-10-16T11:20:27.068575+0000","last_clean":"2023-10-16T11:20:27.068575+0000","last_became_active":"2023-10-16T11:20:26.094220+0000","last_became_peered":"2023-10-16T11:20:26.094220+0000","last_unstale":"2023-10-16T11:20:27.068575+0000","last_undegraded":"2023-10-16T11:20:27.068575+0000","last_fullsized":"2023-10-16T11:20:27.068575+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:05.280523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.097950+0000","last_change":"2023-10-16T11:20:25.080334+0000","last_active":"2023-10-16T11:20:26.097950+0000","last_peered":"2023-10-16T11:20:26.097950+0000","last_clean":"2023-10-16T11:20:26.097950+0000","last_became_active":"2023-10-16T11:20:25.080084+0000","last_became_peered":"2023-10-16T11:20:25.080084+0000","last_unstale":"2023-10-16T11:20:26.097950+0000","last_undegraded":"2023-10-16T11:20:26.097950+0000","last_fullsized":"2023-10-16T11:20:26.097950+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:19.309219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699594+0000","last_change":"2023-10-16T11:20:26.086141+0000","last_active":"2023-10-16T11:20:27.699594+0000","last_peered":"2023-10-16T11:20:27.699594+0000","last_clean":"2023-10-16T11:20:27.699594+0000","last_became_active":"2023-10-16T11:20:26.085865+0000","last_became_peered":"2023-10-16T11:20:26.085865+0000","last_unstale":"2023-10-16T11:20:27.699594+0000","last_undegraded":"2023-10-16T11:20:27.699594+0000","last_fullsized":"2023-10-16T11:20:27.699594+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:31.177316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.099164+0000","last_change":"2023-10-16T11:20:25.087325+0000","last_active":"2023-10-16T11:20:26.099164+0000","last_peered":"2023-10-16T11:20:26.099164+0000","last_clean":"2023-10-16T11:20:26.099164+0000","last_became_active":"2023-10-16T11:20:25.087033+0000","last_became_peered":"2023-10-16T11:20:25.087033+0000","last_unstale":"2023-10-16T11:20:26.099164+0000","last_undegraded":"2023-10-16T11:20:26.099164+0000","last_fullsized":"2023-10-16T11:20:26.099164+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:17:22.652116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068468+0000","last_change":"2023-10-16T11:20:26.090198+0000","last_active":"2023-10-16T11:20:27.068468+0000","last_peered":"2023-10-16T11:20:27.068468+0000","last_clean":"2023-10-16T11:20:27.068468+0000","last_became_active":"2023-10-16T11:20:26.089545+0000","last_became_peered":"2023-10-16T11:20:26.089545+0000","last_unstale":"2023-10-16T11:20:27.068468+0000","last_undegraded":"2023-10-16T11:20:27.068468+0000","last_fullsized":"2023-10-16T11:20:27.068468+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:03.204915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.094694+0000","last_change":"2023-10-16T11:20:25.084321+0000","last_active":"2023-10-16T11:20:26.094694+0000","last_peered":"2023-10-16T11:20:26.094694+0000","last_clean":"2023-10-16T11:20:26.094694+0000","last_became_active":"2023-10-16T11:20:25.083847+0000","last_became_peered":"2023-10-16T11:20:25.083847+0000","last_unstale":"2023-10-16T11:20:26.094694+0000","last_undegraded":"2023-10-16T11:20:26.094694+0000","last_fullsized":"2023-10-16T11:20:26.094694+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:16.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.069796+0000","last_change":"2023-10-16T11:20:26.089495+0000","last_active":"2023-10-16T11:20:27.069796+0000","last_peered":"2023-10-16T11:20:27.069796+0000","last_clean":"2023-10-16T11:20:27.069796+0000","last_became_active":"2023-10-16T11:20:26.089203+0000","last_became_peered":"2023-10-16T11:20:26.089203+0000","last_unstale":"2023-10-16T11:20:27.069796+0000","last_undegraded":"2023-10-16T11:20:27.069796+0000","last_fullsized":"2023-10-16T11:20:27.069796+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:59.154547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.096012+0000","last_change":"2023-10-16T11:20:25.083810+0000","last_active":"2023-10-16T11:20:26.096012+0000","last_peered":"2023-10-16T11:20:26.096012+0000","last_clean":"2023-10-16T11:20:26.096012+0000","last_became_active":"2023-10-16T11:20:25.083251+0000","last_became_peered":"2023-10-16T11:20:25.083251+0000","last_unstale":"2023-10-16T11:20:26.096012+0000","last_undegraded":"2023-10-16T11:20:26.096012+0000","last_fullsized":"2023-10-16T11:20:26.096012+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:44.723324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698187+0000","last_change":"2023-10-16T11:20:26.071683+0000","last_active":"2023-10-16T11:20:27.698187+0000","last_peered":"2023-10-16T11:20:27.698187+0000","last_clean":"2023-10-16T11:20:27.698187+0000","last_became_active":"2023-10-16T11:20:26.071396+0000","last_became_peered":"2023-10-16T11:20:26.071396+0000","last_unstale":"2023-10-16T11:20:27.698187+0000","last_undegraded":"2023-10-16T11:20:27.698187+0000","last_fullsized":"2023-10-16T11:20:27.698187+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:15.600869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.076828+0000","last_change":"2023-10-16T11:20:25.085339+0000","last_active":"2023-10-16T11:20:26.076828+0000","last_peered":"2023-10-16T11:20:26.076828+0000","last_clean":"2023-10-16T11:20:26.076828+0000","last_became_active":"2023-10-16T11:20:25.085073+0000","last_became_peered":"2023-10-16T11:20:25.085073+0000","last_unstale":"2023-10-16T11:20:26.076828+0000","last_undegraded":"2023-10-16T11:20:26.076828+0000","last_fullsized":"2023-10-16T11:20:26.076828+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:23.090533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698420+0000","last_change":"2023-10-16T11:20:26.085542+0000","last_active":"2023-10-16T11:20:27.698420+0000","last_peered":"2023-10-16T11:20:27.698420+0000","last_clean":"2023-10-16T11:20:27.698420+0000","last_became_active":"2023-10-16T11:20:26.083529+0000","last_became_peered":"2023-10-16T11:20:26.083529+0000","last_unstale":"2023-10-16T11:20:27.698420+0000","last_undegraded":"2023-10-16T11:20:27.698420+0000","last_fullsized":"2023-10-16T11:20:27.698420+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069395+0000","last_change":"2023-10-16T11:20:25.088288+0000","last_active":"2023-10-16T11:20:26.069395+0000","last_peered":"2023-10-16T11:20:26.069395+0000","last_clean":"2023-10-16T11:20:26.069395+0000","last_became_active":"2023-10-16T11:20:25.087144+0000","last_became_peered":"2023-10-16T11:20:25.087144+0000","last_unstale":"2023-10-16T11:20:26.069395+0000","last_undegraded":"2023-10-16T11:20:26.069395+0000","last_fullsized":"2023-10-16T11:20:26.069395+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:16:49.706199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698789+0000","last_change":"2023-10-16T11:20:26.085425+0000","last_active":"2023-10-16T11:20:27.698789+0000","last_peered":"2023-10-16T11:20:27.698789+0000","last_clean":"2023-10-16T11:20:27.698789+0000","last_became_active":"2023-10-16T11:20:26.085150+0000","last_became_peered":"2023-10-16T11:20:26.085150+0000","last_unstale":"2023-10-16T11:20:27.698789+0000","last_undegraded":"2023-10-16T11:20:27.698789+0000","last_fullsized":"2023-10-16T11:20:27.698789+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:14.978178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.076553+0000","last_change":"2023-10-16T11:20:25.078062+0000","last_active":"2023-10-16T11:20:26.076553+0000","last_peered":"2023-10-16T11:20:26.076553+0000","last_clean":"2023-10-16T11:20:26.076553+0000","last_became_active":"2023-10-16T11:20:25.077777+0000","last_became_peered":"2023-10-16T11:20:25.077777+0000","last_unstale":"2023-10-16T11:20:26.076553+0000","last_undegraded":"2023-10-16T11:20:26.076553+0000","last_fullsized":"2023-10-16T11:20:26.076553+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:13.108385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068962+0000","last_change":"2023-10-16T11:20:26.098504+0000","last_active":"2023-10-16T11:20:27.068962+0000","last_peered":"2023-10-16T11:20:27.068962+0000","last_clean":"2023-10-16T11:20:27.068962+0000","last_became_active":"2023-10-16T11:20:26.098021+0000","last_became_peered":"2023-10-16T11:20:26.098021+0000","last_unstale":"2023-10-16T11:20:27.068962+0000","last_undegraded":"2023-10-16T11:20:27.068962+0000","last_fullsized":"2023-10-16T11:20:27.068962+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:06.218021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069690+0000","last_change":"2023-10-16T11:20:25.088083+0000","last_active":"2023-10-16T11:20:26.069690+0000","last_peered":"2023-10-16T11:20:26.069690+0000","last_clean":"2023-10-16T11:20:26.069690+0000","last_became_active":"2023-10-16T11:20:25.087792+0000","last_became_peered":"2023-10-16T11:20:25.087792+0000","last_unstale":"2023-10-16T11:20:26.069690+0000","last_undegraded":"2023-10-16T11:20:26.069690+0000","last_fullsized":"2023-10-16T11:20:26.069690+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:22.682758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698815+0000","last_change":"2023-10-16T11:20:26.090580+0000","last_active":"2023-10-16T11:20:27.698815+0000","last_peered":"2023-10-16T11:20:27.698815+0000","last_clean":"2023-10-16T11:20:27.698815+0000","last_became_active":"2023-10-16T11:20:26.090296+0000","last_became_peered":"2023-10-16T11:20:26.090296+0000","last_unstale":"2023-10-16T11:20:27.698815+0000","last_undegraded":"2023-10-16T11:20:27.698815+0000","last_fullsized":"2023-10-16T11:20:27.698815+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:29.302528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"235'3","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.103092+0000","last_change":"2023-10-16T11:20:25.083562+0000","last_active":"2023-10-16T11:20:26.103092+0000","last_peered":"2023-10-16T11:20:26.103092+0000","last_clean":"2023-10-16T11:20:26.103092+0000","last_became_active":"2023-10-16T11:20:25.083270+0000","last_became_peered":"2023-10-16T11:20:25.083270+0000","last_unstale":"2023-10-16T11:20:26.103092+0000","last_undegraded":"2023-10-16T11:20:26.103092+0000","last_fullsized":"2023-10-16T11:20:26.103092+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:34.227829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698773+0000","last_change":"2023-10-16T11:20:26.079886+0000","last_active":"2023-10-16T11:20:27.698773+0000","last_peered":"2023-10-16T11:20:27.698773+0000","last_clean":"2023-10-16T11:20:27.698773+0000","last_became_active":"2023-10-16T11:20:26.079622+0000","last_became_peered":"2023-10-16T11:20:26.079622+0000","last_unstale":"2023-10-16T11:20:27.698773+0000","last_undegraded":"2023-10-16T11:20:27.698773+0000","last_fullsized":"2023-10-16T11:20:27.698773+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:30.843293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"235'6","reported_seq":18,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.105055+0000","last_change":"2023-10-16T11:20:25.083500+0000","last_active":"2023-10-16T11:20:26.105055+0000","last_peered":"2023-10-16T11:20:26.105055+0000","last_clean":"2023-10-16T11:20:26.105055+0000","last_became_active":"2023-10-16T11:20:25.083213+0000","last_became_peered":"2023-10-16T11:20:25.083213+0000","last_unstale":"2023-10-16T11:20:26.105055+0000","last_undegraded":"2023-10-16T11:20:26.105055+0000","last_fullsized":"2023-10-16T11:20:26.105055+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:04.495826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698906+0000","last_change":"2023-10-16T11:20:26.083135+0000","last_active":"2023-10-16T11:20:27.698906+0000","last_peered":"2023-10-16T11:20:27.698906+0000","last_clean":"2023-10-16T11:20:27.698906+0000","last_became_active":"2023-10-16T11:20:26.082824+0000","last_became_peered":"2023-10-16T11:20:26.082824+0000","last_unstale":"2023-10-16T11:20:27.698906+0000","last_undegraded":"2023-10-16T11:20:27.698906+0000","last_fullsized":"2023-10-16T11:20:27.698906+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:55.337626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.103444+0000","last_change":"2023-10-16T11:20:25.083057+0000","last_active":"2023-10-16T11:20:26.103444+0000","last_peered":"2023-10-16T11:20:26.103444+0000","last_clean":"2023-10-16T11:20:26.103444+0000","last_became_active":"2023-10-16T11:20:25.082601+0000","last_became_peered":"2023-10-16T11:20:25.082601+0000","last_unstale":"2023-10-16T11:20:26.103444+0000","last_undegraded":"2023-10-16T11:20:26.103444+0000","last_fullsized":"2023-10-16T11:20:26.103444+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.609347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.069797+0000","last_change":"2023-10-16T11:20:26.073157+0000","last_active":"2023-10-16T11:20:27.069797+0000","last_peered":"2023-10-16T11:20:27.069797+0000","last_clean":"2023-10-16T11:20:27.069797+0000","last_became_active":"2023-10-16T11:20:26.072892+0000","last_became_peered":"2023-10-16T11:20:26.072892+0000","last_unstale":"2023-10-16T11:20:27.069797+0000","last_undegraded":"2023-10-16T11:20:27.069797+0000","last_fullsized":"2023-10-16T11:20:27.069797+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:21:31.224577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069498+0000","last_change":"2023-10-16T11:20:25.079093+0000","last_active":"2023-10-16T11:20:26.069498+0000","last_peered":"2023-10-16T11:20:26.069498+0000","last_clean":"2023-10-16T11:20:26.069498+0000","last_became_active":"2023-10-16T11:20:25.078740+0000","last_became_peered":"2023-10-16T11:20:25.078740+0000","last_unstale":"2023-10-16T11:20:26.069498+0000","last_undegraded":"2023-10-16T11:20:26.069498+0000","last_fullsized":"2023-10-16T11:20:26.069498+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:27.511026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698358+0000","last_change":"2023-10-16T11:20:26.084841+0000","last_active":"2023-10-16T11:20:27.698358+0000","last_peered":"2023-10-16T11:20:27.698358+0000","last_clean":"2023-10-16T11:20:27.698358+0000","last_became_active":"2023-10-16T11:20:26.084153+0000","last_became_peered":"2023-10-16T11:20:26.084153+0000","last_unstale":"2023-10-16T11:20:27.698358+0000","last_undegraded":"2023-10-16T11:20:27.698358+0000","last_fullsized":"2023-10-16T11:20:27.698358+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:30.506645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.076788+0000","last_change":"2023-10-16T11:20:25.084386+0000","last_active":"2023-10-16T11:20:26.076788+0000","last_peered":"2023-10-16T11:20:26.076788+0000","last_clean":"2023-10-16T11:20:26.076788+0000","last_became_active":"2023-10-16T11:20:25.083974+0000","last_became_peered":"2023-10-16T11:20:25.083974+0000","last_unstale":"2023-10-16T11:20:26.076788+0000","last_undegraded":"2023-10-16T11:20:26.076788+0000","last_fullsized":"2023-10-16T11:20:26.076788+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:05.210033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698448+0000","last_change":"2023-10-16T11:20:26.096969+0000","last_active":"2023-10-16T11:20:27.698448+0000","last_peered":"2023-10-16T11:20:27.698448+0000","last_clean":"2023-10-16T11:20:27.698448+0000","last_became_active":"2023-10-16T11:20:26.096691+0000","last_became_peered":"2023-10-16T11:20:26.096691+0000","last_unstale":"2023-10-16T11:20:27.698448+0000","last_undegraded":"2023-10-16T11:20:27.698448+0000","last_fullsized":"2023-10-16T11:20:27.698448+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:05.355796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"235'8","reported_seq":20,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.104404+0000","last_change":"2023-10-16T11:20:25.082367+0000","last_active":"2023-10-16T11:20:26.104404+0000","last_peered":"2023-10-16T11:20:26.104404+0000","last_clean":"2023-10-16T11:20:26.104404+0000","last_became_active":"2023-10-16T11:20:25.082053+0000","last_became_peered":"2023-10-16T11:20:25.082053+0000","last_unstale":"2023-10-16T11:20:26.104404+0000","last_undegraded":"2023-10-16T11:20:26.104404+0000","last_fullsized":"2023-10-16T11:20:26.104404+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:29.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698992+0000","last_change":"2023-10-16T11:20:26.085864+0000","last_active":"2023-10-16T11:20:27.698992+0000","last_peered":"2023-10-16T11:20:27.698992+0000","last_clean":"2023-10-16T11:20:27.698992+0000","last_became_active":"2023-10-16T11:20:26.085644+0000","last_became_peered":"2023-10-16T11:20:26.085644+0000","last_unstale":"2023-10-16T11:20:27.698992+0000","last_undegraded":"2023-10-16T11:20:27.698992+0000","last_fullsized":"2023-10-16T11:20:27.698992+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:16.138806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.089803+0000","last_change":"2023-10-16T11:20:25.084626+0000","last_active":"2023-10-16T11:20:26.089803+0000","last_peered":"2023-10-16T11:20:26.089803+0000","last_clean":"2023-10-16T11:20:26.089803+0000","last_became_active":"2023-10-16T11:20:25.084276+0000","last_became_peered":"2023-10-16T11:20:25.084276+0000","last_unstale":"2023-10-16T11:20:26.089803+0000","last_undegraded":"2023-10-16T11:20:26.089803+0000","last_fullsized":"2023-10-16T11:20:26.089803+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.241621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699449+0000","last_change":"2023-10-16T11:20:26.084908+0000","last_active":"2023-10-16T11:20:27.699449+0000","last_peered":"2023-10-16T11:20:27.699449+0000","last_clean":"2023-10-16T11:20:27.699449+0000","last_became_active":"2023-10-16T11:20:26.084599+0000","last_became_peered":"2023-10-16T11:20:26.084599+0000","last_unstale":"2023-10-16T11:20:27.699449+0000","last_undegraded":"2023-10-16T11:20:27.699449+0000","last_fullsized":"2023-10-16T11:20:27.699449+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:13.336265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.097751+0000","last_change":"2023-10-16T11:20:25.087228+0000","last_active":"2023-10-16T11:20:26.097751+0000","last_peered":"2023-10-16T11:20:26.097751+0000","last_clean":"2023-10-16T11:20:26.097751+0000","last_became_active":"2023-10-16T11:20:25.086965+0000","last_became_peered":"2023-10-16T11:20:25.086965+0000","last_unstale":"2023-10-16T11:20:26.097751+0000","last_undegraded":"2023-10-16T11:20:26.097751+0000","last_fullsized":"2023-10-16T11:20:26.097751+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:08.265203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070276+0000","last_change":"2023-10-16T11:20:26.077503+0000","last_active":"2023-10-16T11:20:27.070276+0000","last_peered":"2023-10-16T11:20:27.070276+0000","last_clean":"2023-10-16T11:20:27.070276+0000","last_became_active":"2023-10-16T11:20:26.077247+0000","last_became_peered":"2023-10-16T11:20:26.077247+0000","last_unstale":"2023-10-16T11:20:27.070276+0000","last_undegraded":"2023-10-16T11:20:27.070276+0000","last_fullsized":"2023-10-16T11:20:27.070276+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:49.573161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"235'1","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.091830+0000","last_change":"2023-10-16T11:20:25.076671+0000","last_active":"2023-10-16T11:20:26.091830+0000","last_peered":"2023-10-16T11:20:26.091830+0000","last_clean":"2023-10-16T11:20:26.091830+0000","last_became_active":"2023-10-16T11:20:25.076356+0000","last_became_peered":"2023-10-16T11:20:25.076356+0000","last_unstale":"2023-10-16T11:20:26.091830+0000","last_undegraded":"2023-10-16T11:20:26.091830+0000","last_fullsized":"2023-10-16T11:20:26.091830+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:21.582269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699655+0000","last_change":"2023-10-16T11:20:26.086294+0000","last_active":"2023-10-16T11:20:27.699655+0000","last_peered":"2023-10-16T11:20:27.699655+0000","last_clean":"2023-10-16T11:20:27.699655+0000","last_became_active":"2023-10-16T11:20:26.085985+0000","last_became_peered":"2023-10-16T11:20:26.085985+0000","last_unstale":"2023-10-16T11:20:27.699655+0000","last_undegraded":"2023-10-16T11:20:27.699655+0000","last_fullsized":"2023-10-16T11:20:27.699655+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:58.159985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.088550+0000","last_change":"2023-10-16T11:20:25.085222+0000","last_active":"2023-10-16T11:20:26.088550+0000","last_peered":"2023-10-16T11:20:26.088550+0000","last_clean":"2023-10-16T11:20:26.088550+0000","last_became_active":"2023-10-16T11:20:25.084958+0000","last_became_peered":"2023-10-16T11:20:25.084958+0000","last_unstale":"2023-10-16T11:20:26.088550+0000","last_undegraded":"2023-10-16T11:20:26.088550+0000","last_fullsized":"2023-10-16T11:20:26.088550+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:15.130817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699314+0000","last_change":"2023-10-16T11:20:26.091707+0000","last_active":"2023-10-16T11:20:27.699314+0000","last_peered":"2023-10-16T11:20:27.699314+0000","last_clean":"2023-10-16T11:20:27.699314+0000","last_became_active":"2023-10-16T11:20:26.091470+0000","last_became_peered":"2023-10-16T11:20:26.091470+0000","last_unstale":"2023-10-16T11:20:27.699314+0000","last_undegraded":"2023-10-16T11:20:27.699314+0000","last_fullsized":"2023-10-16T11:20:27.699314+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.100597+0000","last_change":"2023-10-16T11:20:25.086865+0000","last_active":"2023-10-16T11:20:26.100597+0000","last_peered":"2023-10-16T11:20:26.100597+0000","last_clean":"2023-10-16T11:20:26.100597+0000","last_became_active":"2023-10-16T11:20:25.086528+0000","last_became_peered":"2023-10-16T11:20:25.086528+0000","last_unstale":"2023-10-16T11:20:26.100597+0000","last_undegraded":"2023-10-16T11:20:26.100597+0000","last_fullsized":"2023-10-16T11:20:26.100597+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:17.050518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698303+0000","last_change":"2023-10-16T11:20:26.080221+0000","last_active":"2023-10-16T11:20:27.698303+0000","last_peered":"2023-10-16T11:20:27.698303+0000","last_clean":"2023-10-16T11:20:27.698303+0000","last_became_active":"2023-10-16T11:20:26.079927+0000","last_became_peered":"2023-10-16T11:20:26.079927+0000","last_unstale":"2023-10-16T11:20:27.698303+0000","last_undegraded":"2023-10-16T11:20:27.698303+0000","last_fullsized":"2023-10-16T11:20:27.698303+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:27.603420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.075149+0000","last_change":"2023-10-16T11:20:25.084862+0000","last_active":"2023-10-16T11:20:26.075149+0000","last_peered":"2023-10-16T11:20:26.075149+0000","last_clean":"2023-10-16T11:20:26.075149+0000","last_became_active":"2023-10-16T11:20:25.084538+0000","last_became_peered":"2023-10-16T11:20:25.084538+0000","last_unstale":"2023-10-16T11:20:26.075149+0000","last_undegraded":"2023-10-16T11:20:26.075149+0000","last_fullsized":"2023-10-16T11:20:26.075149+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:11:13.297732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068322+0000","last_change":"2023-10-16T11:20:26.101781+0000","last_active":"2023-10-16T11:20:27.068322+0000","last_peered":"2023-10-16T11:20:27.068322+0000","last_clean":"2023-10-16T11:20:27.068322+0000","last_became_active":"2023-10-16T11:20:26.101470+0000","last_became_peered":"2023-10-16T11:20:26.101470+0000","last_unstale":"2023-10-16T11:20:27.068322+0000","last_undegraded":"2023-10-16T11:20:27.068322+0000","last_fullsized":"2023-10-16T11:20:27.068322+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069206+0000","last_change":"2023-10-16T11:20:25.081336+0000","last_active":"2023-10-16T11:20:26.069206+0000","last_peered":"2023-10-16T11:20:26.069206+0000","last_clean":"2023-10-16T11:20:26.069206+0000","last_became_active":"2023-10-16T11:20:25.081017+0000","last_became_peered":"2023-10-16T11:20:25.081017+0000","last_unstale":"2023-10-16T11:20:26.069206+0000","last_undegraded":"2023-10-16T11:20:26.069206+0000","last_fullsized":"2023-10-16T11:20:26.069206+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:38.114268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698984+0000","last_change":"2023-10-16T11:20:26.085568+0000","last_active":"2023-10-16T11:20:27.698984+0000","last_peered":"2023-10-16T11:20:27.698984+0000","last_clean":"2023-10-16T11:20:27.698984+0000","last_became_active":"2023-10-16T11:20:26.085298+0000","last_became_peered":"2023-10-16T11:20:26.085298+0000","last_unstale":"2023-10-16T11:20:27.698984+0000","last_undegraded":"2023-10-16T11:20:27.698984+0000","last_fullsized":"2023-10-16T11:20:27.698984+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:23:15.106414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.075564+0000","last_change":"2023-10-16T11:20:25.084776+0000","last_active":"2023-10-16T11:20:26.075564+0000","last_peered":"2023-10-16T11:20:26.075564+0000","last_clean":"2023-10-16T11:20:26.075564+0000","last_became_active":"2023-10-16T11:20:25.084415+0000","last_became_peered":"2023-10-16T11:20:25.084415+0000","last_unstale":"2023-10-16T11:20:26.075564+0000","last_undegraded":"2023-10-16T11:20:26.075564+0000","last_fullsized":"2023-10-16T11:20:26.075564+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:43.132428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699314+0000","last_change":"2023-10-16T11:20:26.080094+0000","last_active":"2023-10-16T11:20:27.699314+0000","last_peered":"2023-10-16T11:20:27.699314+0000","last_clean":"2023-10-16T11:20:27.699314+0000","last_became_active":"2023-10-16T11:20:26.079844+0000","last_became_peered":"2023-10-16T11:20:26.079844+0000","last_unstale":"2023-10-16T11:20:27.699314+0000","last_undegraded":"2023-10-16T11:20:27.699314+0000","last_fullsized":"2023-10-16T11:20:27.699314+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:54.114067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"235'2","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.102912+0000","last_change":"2023-10-16T11:20:25.084135+0000","last_active":"2023-10-16T11:20:26.102912+0000","last_peered":"2023-10-16T11:20:26.102912+0000","last_clean":"2023-10-16T11:20:26.102912+0000","last_became_active":"2023-10-16T11:20:25.083794+0000","last_became_peered":"2023-10-16T11:20:25.083794+0000","last_unstale":"2023-10-16T11:20:26.102912+0000","last_undegraded":"2023-10-16T11:20:26.102912+0000","last_fullsized":"2023-10-16T11:20:26.102912+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:57.161061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699220+0000","last_change":"2023-10-16T11:20:26.083428+0000","last_active":"2023-10-16T11:20:27.699220+0000","last_peered":"2023-10-16T11:20:27.699220+0000","last_clean":"2023-10-16T11:20:27.699220+0000","last_became_active":"2023-10-16T11:20:26.082473+0000","last_became_peered":"2023-10-16T11:20:26.082473+0000","last_unstale":"2023-10-16T11:20:27.699220+0000","last_undegraded":"2023-10-16T11:20:27.699220+0000","last_fullsized":"2023-10-16T11:20:27.699220+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:39.122257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.068350+0000","last_change":"2023-10-16T11:20:25.085721+0000","last_active":"2023-10-16T11:20:26.068350+0000","last_peered":"2023-10-16T11:20:26.068350+0000","last_clean":"2023-10-16T11:20:26.068350+0000","last_became_active":"2023-10-16T11:20:25.085421+0000","last_became_peered":"2023-10-16T11:20:25.085421+0000","last_unstale":"2023-10-16T11:20:26.068350+0000","last_undegraded":"2023-10-16T11:20:26.068350+0000","last_fullsized":"2023-10-16T11:20:26.068350+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:08.936039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068680+0000","last_change":"2023-10-16T11:20:26.098738+0000","last_active":"2023-10-16T11:20:27.068680+0000","last_peered":"2023-10-16T11:20:27.068680+0000","last_clean":"2023-10-16T11:20:27.068680+0000","last_became_active":"2023-10-16T11:20:26.098226+0000","last_became_peered":"2023-10-16T11:20:26.098226+0000","last_unstale":"2023-10-16T11:20:27.068680+0000","last_undegraded":"2023-10-16T11:20:27.068680+0000","last_fullsized":"2023-10-16T11:20:27.068680+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:48.218074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069541+0000","last_change":"2023-10-16T11:20:25.081249+0000","last_active":"2023-10-16T11:20:26.069541+0000","last_peered":"2023-10-16T11:20:26.069541+0000","last_clean":"2023-10-16T11:20:26.069541+0000","last_became_active":"2023-10-16T11:20:25.080938+0000","last_became_peered":"2023-10-16T11:20:25.080938+0000","last_unstale":"2023-10-16T11:20:26.069541+0000","last_undegraded":"2023-10-16T11:20:26.069541+0000","last_fullsized":"2023-10-16T11:20:26.069541+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:09.538322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.068796+0000","last_change":"2023-10-16T11:20:26.094947+0000","last_active":"2023-10-16T11:20:27.068796+0000","last_peered":"2023-10-16T11:20:27.068796+0000","last_clean":"2023-10-16T11:20:27.068796+0000","last_became_active":"2023-10-16T11:20:26.094613+0000","last_became_peered":"2023-10-16T11:20:26.094613+0000","last_unstale":"2023-10-16T11:20:27.068796+0000","last_undegraded":"2023-10-16T11:20:27.068796+0000","last_fullsized":"2023-10-16T11:20:27.068796+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:24.098521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.088706+0000","last_change":"2023-10-16T11:20:25.082461+0000","last_active":"2023-10-16T11:20:26.088706+0000","last_peered":"2023-10-16T11:20:26.088706+0000","last_clean":"2023-10-16T11:20:26.088706+0000","last_became_active":"2023-10-16T11:20:25.082161+0000","last_became_peered":"2023-10-16T11:20:25.082161+0000","last_unstale":"2023-10-16T11:20:26.088706+0000","last_undegraded":"2023-10-16T11:20:26.088706+0000","last_fullsized":"2023-10-16T11:20:26.088706+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:13.522561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699176+0000","last_change":"2023-10-16T11:20:26.083434+0000","last_active":"2023-10-16T11:20:27.699176+0000","last_peered":"2023-10-16T11:20:27.699176+0000","last_clean":"2023-10-16T11:20:27.699176+0000","last_became_active":"2023-10-16T11:20:26.083146+0000","last_became_peered":"2023-10-16T11:20:26.083146+0000","last_unstale":"2023-10-16T11:20:27.699176+0000","last_undegraded":"2023-10-16T11:20:27.699176+0000","last_fullsized":"2023-10-16T11:20:27.699176+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:36.022948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.075295+0000","last_change":"2023-10-16T11:20:25.084296+0000","last_active":"2023-10-16T11:20:26.075295+0000","last_peered":"2023-10-16T11:20:26.075295+0000","last_clean":"2023-10-16T11:20:26.075295+0000","last_became_active":"2023-10-16T11:20:25.083831+0000","last_became_peered":"2023-10-16T11:20:25.083831+0000","last_unstale":"2023-10-16T11:20:26.075295+0000","last_undegraded":"2023-10-16T11:20:26.075295+0000","last_fullsized":"2023-10-16T11:20:26.075295+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:15.128952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.699408+0000","last_change":"2023-10-16T11:20:26.097299+0000","last_active":"2023-10-16T11:20:27.699408+0000","last_peered":"2023-10-16T11:20:27.699408+0000","last_clean":"2023-10-16T11:20:27.699408+0000","last_became_active":"2023-10-16T11:20:26.096992+0000","last_became_peered":"2023-10-16T11:20:26.096992+0000","last_unstale":"2023-10-16T11:20:27.699408+0000","last_undegraded":"2023-10-16T11:20:27.699408+0000","last_fullsized":"2023-10-16T11:20:27.699408+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:01.203662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.079512+0000","last_change":"2023-10-16T11:20:25.085431+0000","last_active":"2023-10-16T11:20:26.079512+0000","last_peered":"2023-10-16T11:20:26.079512+0000","last_clean":"2023-10-16T11:20:26.079512+0000","last_became_active":"2023-10-16T11:20:25.084034+0000","last_became_peered":"2023-10-16T11:20:25.084034+0000","last_unstale":"2023-10-16T11:20:26.079512+0000","last_undegraded":"2023-10-16T11:20:26.079512+0000","last_fullsized":"2023-10-16T11:20:26.079512+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:44.288602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698642+0000","last_change":"2023-10-16T11:20:26.082969+0000","last_active":"2023-10-16T11:20:27.698642+0000","last_peered":"2023-10-16T11:20:27.698642+0000","last_clean":"2023-10-16T11:20:27.698642+0000","last_became_active":"2023-10-16T11:20:26.082661+0000","last_became_peered":"2023-10-16T11:20:26.082661+0000","last_unstale":"2023-10-16T11:20:27.698642+0000","last_undegraded":"2023-10-16T11:20:27.698642+0000","last_fullsized":"2023-10-16T11:20:27.698642+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:53.296236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069530+0000","last_change":"2023-10-16T11:20:25.080407+0000","last_active":"2023-10-16T11:20:26.069530+0000","last_peered":"2023-10-16T11:20:26.069530+0000","last_clean":"2023-10-16T11:20:26.069530+0000","last_became_active":"2023-10-16T11:20:25.079543+0000","last_became_peered":"2023-10-16T11:20:25.079543+0000","last_unstale":"2023-10-16T11:20:26.069530+0000","last_undegraded":"2023-10-16T11:20:26.069530+0000","last_fullsized":"2023-10-16T11:20:26.069530+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:12.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.698544+0000","last_change":"2023-10-16T11:20:26.086262+0000","last_active":"2023-10-16T11:20:27.698544+0000","last_peered":"2023-10-16T11:20:27.698544+0000","last_clean":"2023-10-16T11:20:27.698544+0000","last_became_active":"2023-10-16T11:20:26.085901+0000","last_became_peered":"2023-10-16T11:20:26.085901+0000","last_unstale":"2023-10-16T11:20:27.698544+0000","last_undegraded":"2023-10-16T11:20:27.698544+0000","last_fullsized":"2023-10-16T11:20:27.698544+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:56.664462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.070135+0000","last_change":"2023-10-16T11:20:25.086844+0000","last_active":"2023-10-16T11:20:26.070135+0000","last_peered":"2023-10-16T11:20:26.070135+0000","last_clean":"2023-10-16T11:20:26.070135+0000","last_became_active":"2023-10-16T11:20:25.086570+0000","last_became_peered":"2023-10-16T11:20:25.086570+0000","last_unstale":"2023-10-16T11:20:26.070135+0000","last_undegraded":"2023-10-16T11:20:26.070135+0000","last_fullsized":"2023-10-16T11:20:26.070135+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:17.084881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-16T11:20:27.070094+0000","last_change":"2023-10-16T11:20:26.078189+0000","last_active":"2023-10-16T11:20:27.070094+0000","last_peered":"2023-10-16T11:20:27.070094+0000","last_clean":"2023-10-16T11:20:27.070094+0000","last_became_active":"2023-10-16T11:20:26.077754+0000","last_became_peered":"2023-10-16T11:20:26.077754+0000","last_unstale":"2023-10-16T11:20:27.070094+0000","last_undegraded":"2023-10-16T11:20:27.070094+0000","last_fullsized":"2023-10-16T11:20:27.070094+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:45.296590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"235'4","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.099065+0000","last_change":"2023-10-16T11:20:25.087446+0000","last_active":"2023-10-16T11:20:26.099065+0000","last_peered":"2023-10-16T11:20:26.099065+0000","last_clean":"2023-10-16T11:20:26.099065+0000","last_became_active":"2023-10-16T11:20:25.086617+0000","last_became_peered":"2023-10-16T11:20:25.086617+0000","last_unstale":"2023-10-16T11:20:26.099065+0000","last_undegraded":"2023-10-16T11:20:26.099065+0000","last_fullsized":"2023-10-16T11:20:26.099065+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:50.072765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'370","reported_seq":1308,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-16T11:20:26.069113+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:20:26.069113+0000","last_peered":"2023-10-16T11:20:26.069113+0000","last_clean":"2023-10-16T11:20:26.069113+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:20:26.069113+0000","last_undegraded":"2023-10-16T11:20:26.069113+0000","last_fullsized":"2023-10-16T11:20:26.069113+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543290,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52720,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319120,"statfs":{"total":96636764160,"available":96582778880,"internally_reserved":0,"allocated":3567616,"data_stored":2123854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.85699999999999998,"5min":0.878,"15min":0.878},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.27800000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.33300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.83399999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":1,"5min":1,"15min":1.002},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.33100000000000002},"max":{"1min":0.93500000000000005,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.628}]},{"osd":6,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70899999999999996}]}]},{"osd":6,"up_from":14,"seq":60129543289,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56296,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315544,"statfs":{"total":96636764160,"available":96579117056,"internally_reserved":0,"allocated":2977792,"data_stored":1534020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29899999999999999},"max":{"1min":0.66700000000000004,"5min":0.81699999999999995,"15min":0.84499999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83099999999999996,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.68300000000000005}]},{"osd":1,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.42599999999999999,"5min":0.40799999999999997,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68799999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76300000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54500000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.373,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129543289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60468,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311372,"statfs":{"total":96636764160,"available":96574844928,"internally_reserved":0,"allocated":2990080,"data_stored":1535152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39700000000000002,"15min":0.40600000000000003},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51800000000000002,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.40600000000000003,"15min":0.43099999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.49199999999999999,"5min":0.62,"15min":0.91700000000000004},"last":0.33200000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40200000000000002,"15min":0.41699999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45000000000000001,"5min":0.58599999999999997,"15min":0.98899999999999999},"last":0.314},{"interface":"front","average":{"1min":0.434,"5min":0.441,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63200000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.34899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46400000000000002,"15min":0.49199999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.78300000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.57999999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.36399999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77200000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.621},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85199999999999998,"5min":1.0289999999999999,"15min":1.097},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85499999999999998,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.495},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.90100000000000002,"15min":1.147},"last":0.53400000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.98699999999999999,"15min":1.0509999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.70299999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":1.0900000000000001},"last":0.52200000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":1.004,"15min":1.0600000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.50600000000000001}]}]},{"osd":0,"up_from":14,"seq":60129543290,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61032,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310808,"statfs":{"total":96636764160,"available":96574267392,"internally_reserved":0,"allocated":3567616,"data_stored":2123818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.214},"max":{"1min":0.46400000000000002,"5min":0.67400000000000004,"15min":0.91600000000000004},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.58799999999999997,"5min":0.63700000000000001,"15min":0.70299999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.68300000000000005},"last":0.314},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.49199999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.71799999999999997,"15min":0.84599999999999997},"last":0.377}]},{"osd":4,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.65900000000000003,"15min":0.66400000000000003},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.99199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.91900000000000004,"15min":1.008},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91600000000000004,"5min":1.004,"15min":1.008},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.39400000000000002},"max":{"1min":0.80900000000000005,"5min":0.97099999999999997,"15min":1.1419999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66400000000000003,"15min":0.66900000000000004},"min":{"1min":0.48099999999999998,"5min":0.47799999999999998,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67600000000000005,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68300000000000005,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56499999999999995}]}]},{"osd":2,"up_from":14,"seq":60129543289,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47972,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323868,"statfs":{"total":96636764160,"available":96587640832,"internally_reserved":0,"allocated":2973696,"data_stored":1533506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.40500000000000003},"min":{"1min":0.24399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.28999999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.28499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.35099999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.436},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.4929999999999999,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.307}]},{"osd":4,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.6779999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.6549999999999998,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.7320000000000002,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.58099999999999996}]}]},{"osd":3,"up_from":14,"seq":60129543290,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47980,"kb_used_data":2904,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323860,"statfs":{"total":96636764160,"available":96587632640,"internally_reserved":0,"allocated":2973696,"data_stored":1533998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.59799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.52600000000000002,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.32800000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.40600000000000003,"15min":0.42499999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58099999999999996,"5min":0.87,"15min":0.87},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54800000000000004,"5min":0.65100000000000002,"15min":0.76300000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.44400000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.49099999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.46400000000000002,"15min":0.48299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56100000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.496},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.65300000000000002},"min":{"1min":0.39900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.621}]},{"osd":5,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.53900000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65100000000000002,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.52800000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.66400000000000003},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96399999999999997,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.59199999999999997}]}]},{"osd":4,"up_from":14,"seq":60129543289,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78452,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94293388,"statfs":{"total":96636764160,"available":96556429312,"internally_reserved":0,"allocated":2990080,"data_stored":1535120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.74399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60399999999999998,"15min":0.61899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71899999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.54000000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.83399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.628,"15min":0.63400000000000001},"min":{"1min":0.5,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":0.745,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.64200000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.879},"last":0.65300000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.52000000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.875,"15min":0.875},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.73499999999999999,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.55500000000000005}]}]},{"osd":5,"up_from":14,"seq":60129543290,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60456,"kb_used_data":2900,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311384,"statfs":{"total":96636764160,"available":96574857216,"internally_reserved":0,"allocated":2969600,"data_stored":1533416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":1.873,"15min":1.873},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.41999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.373,"15min":0.373},"max":{"1min":0.745,"5min":1.024,"15min":1.024},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.626,"15min":0.628},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73999999999999999,"5min":1.821,"15min":1.821},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79900000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.40200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":1.919,"15min":1.919},"last":0.76000000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.432},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94299999999999995,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.96899999999999997,"5min":2.02,"15min":2.02},"last":0.60299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:20:30.581 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:20:30.582 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:20:30.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.741+0000 7f6ff5102700 1 Processor -- start 2023-10-16T11:20:30.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.741+0000 7f6ff5102700 1 -- start start 2023-10-16T11:20:30.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6ff5102700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ff015da70 0x7f6ff00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6ff5102700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6ff00ce6a0 0x7f6ff006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6ff5102700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6ff5102700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff00cef00 con 0x7f6ff015da70 2023-10-16T11:20:30.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6ff5102700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff00cec20 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6ff5102700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0071c80 con 0x7f6ff0072760 2023-10-16T11:20:30.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ff015da70 0x7f6ff00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ff015da70 0x7f6ff00ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57826/0 (socket says 172.21.15.181:57826) 2023-10-16T11:20:30.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 -- 172.21.15.181:0/3292160313 learned_addr learned my addr 172.21.15.181:0/3292160313 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:30.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6fee59c700 1 --2- 172.21.15.181:0/3292160313 >> v2:172.21.15.190:3300/0 conn(0x7f6ff00ce6a0 0x7f6ff006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6fef59e700 1 --2- 172.21.15.181:0/3292160313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 -- 172.21.15.181:0/3292160313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 msgr2=0x7f6ff006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:30.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 --2- 172.21.15.181:0/3292160313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 -- 172.21.15.181:0/3292160313 >> v2:172.21.15.190:3300/0 conn(0x7f6ff00ce6a0 msgr2=0x7f6ff006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:30.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 --2- 172.21.15.181:0/3292160313 >> v2:172.21.15.190:3300/0 conn(0x7f6ff00ce6a0 0x7f6ff006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 -- 172.21.15.181:0/3292160313 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0071f60 con 0x7f6ff015da70 2023-10-16T11:20:30.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6fee59c700 1 --2- 172.21.15.181:0/3292160313 >> v2:172.21.15.190:3300/0 conn(0x7f6ff00ce6a0 0x7f6ff006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:30.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.742+0000 7f6feed9d700 1 --2- 172.21.15.181:0/3292160313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff015da70 0x7f6ff00ce160 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f6ff006ef60 tx=0x7f6fe4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4de96fd4f9e768f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:30.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6fedd9b700 1 -- 172.21.15.181:0/3292160313 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400b870 con 0x7f6ff015da70 2023-10-16T11:20:30.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6fedd9b700 1 -- 172.21.15.181:0/3292160313 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe400b9d0 con 0x7f6ff015da70 2023-10-16T11:20:30.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6fedd9b700 1 -- 172.21.15.181:0/3292160313 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400bd60 con 0x7f6ff015da70 2023-10-16T11:20:30.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 -- 172.21.15.181:0/3292160313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff015da70 msgr2=0x7f6ff00ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 --2- 172.21.15.181:0/3292160313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff015da70 0x7f6ff00ce160 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f6ff006ef60 tx=0x7f6fe4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 -- 172.21.15.181:0/3292160313 shutdown_connections 2023-10-16T11:20:30.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 --2- 172.21.15.181:0/3292160313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff015da70 0x7f6ff00ce160 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 --2- 172.21.15.181:0/3292160313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 --2- 172.21.15.181:0/3292160313 >> v2:172.21.15.190:3300/0 conn(0x7f6ff00ce6a0 0x7f6ff006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 -- 172.21.15.181:0/3292160313 >> 172.21.15.181:0/3292160313 conn(0x7f6ff00cd010 msgr2=0x7f6ff00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:30.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 -- 172.21.15.181:0/3292160313 shutdown_connections 2023-10-16T11:20:30.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.743+0000 7f6ff5102700 1 -- 172.21.15.181:0/3292160313 wait complete. 2023-10-16T11:20:30.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 Processor -- start 2023-10-16T11:20:30.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 -- start start 2023-10-16T11:20:30.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff00c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff00ce6a0 0x7f6ff0159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ff015da70 0x7f6ff00ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00c4600 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff015a090 con 0x7f6ff015da70 2023-10-16T11:20:30.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.745+0000 7f6ff5102700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00adc00 con 0x7f6ff0072760 2023-10-16T11:20:30.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff00ce6a0 0x7f6ff0159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff00ce6a0 0x7f6ff0159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57834/0 (socket says 172.21.15.181:57834) 2023-10-16T11:20:30.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 -- 172.21.15.181:0/1525589346 learned_addr learned my addr 172.21.15.181:0/1525589346 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:30.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fef59e700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ff015da70 0x7f6ff00ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6feed9d700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff00c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 -- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 msgr2=0x7f6ff00c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff00c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 -- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ff015da70 msgr2=0x7f6ff00ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:30.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ff015da70 0x7f6ff00ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:30.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 -- 172.21.15.181:0/1525589346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4009550 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fee59c700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff00ce6a0 0x7f6ff0159b50 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e810 tx=0x7f6fe000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fd77fe700 1 -- 172.21.15.181:0/1525589346 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000c070 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fd77fe700 1 -- 172.21.15.181:0/1525589346 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0004230 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff00af560 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6fd77fe700 1 -- 172.21.15.181:0/1525589346 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00036a0 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.746+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff00b4d30 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.747+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff00b0d80 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.748+0000 7f6fd77fe700 1 -- 172.21.15.181:0/1525589346 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6fe0003800 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.748+0000 7f6fd77fe700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f6fdc06b140 0x7f6fdc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:30.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.748+0000 7f6fd77fe700 1 -- 172.21.15.181:0/1525589346 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f6fe0089a10 con 0x7f6ff00ce6a0 2023-10-16T11:20:30.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.748+0000 7f6feed9d700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f6fdc06b140 0x7f6fdc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:30.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.748+0000 7f6feed9d700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f6fdc06b140 0x7f6fdc06d600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000c00 tx=0x7f6fe4008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:30.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:30.752+0000 7f6fd77fe700 1 -- 172.21.15.181:0/1525589346 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6fe0054430 con 0x7f6ff00ce6a0 2023-10-16T11:20:31.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.026+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6ff006f6c0 con 0x7f6ff00ce6a0 2023-10-16T11:20:31.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.028+0000 7f6fd77fe700 1 -- 172.21.15.181:0/1525589346 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v53) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6fe0057a50 con 0x7f6ff00ce6a0 2023-10-16T11:20:31.030 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T11:20:31.030 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T11:20:31.031 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T11:20:31.031 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T11:20:31.031 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:20:31.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f6fdc06b140 msgr2=0x7f6fdc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f6fdc06b140 0x7f6fdc06d600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000c00 tx=0x7f6fe4008030 comp rx=0 tx=0).stop 2023-10-16T11:20:31.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff00ce6a0 msgr2=0x7f6ff0159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff00ce6a0 0x7f6ff0159b50 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e810 tx=0x7f6fe000eb20 comp rx=0 tx=0).stop 2023-10-16T11:20:31.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 shutdown_connections 2023-10-16T11:20:31.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f6fdc06b140 0x7f6fdc06d600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ff0072760 0x7f6ff00c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ff015da70 0x7f6ff00ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 --2- 172.21.15.181:0/1525589346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ff00ce6a0 0x7f6ff0159b50 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 >> 172.21.15.181:0/1525589346 conn(0x7f6ff00cd010 msgr2=0x7f6ff00b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:31.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 shutdown_connections 2023-10-16T11:20:31.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.030+0000 7f6ff5102700 1 -- 172.21.15.181:0/1525589346 wait complete. 2023-10-16T11:20:31.035 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T11:20:31.044 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:31.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d6573700 1 Processor -- start 2023-10-16T11:20:31.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d6573700 1 -- start start 2023-10-16T11:20:31.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d6573700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6d015e6d0 0x7fa6d00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d6573700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015eab0 0x7fa6d015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d6573700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d006ed00 con 0x7fa6d00cb0c0 2023-10-16T11:20:31.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d6573700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d00d02d0 con 0x7fa6d015e6d0 2023-10-16T11:20:31.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d6573700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015ad40 con 0x7fa6d015eab0 2023-10-16T11:20:31.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6cffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6d015e6d0 0x7fa6d00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.165+0000 7fa6d4b10700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa6d00cb0c0 0x7fa6d015a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40692/0 (socket says 172.21.15.181:40692) 2023-10-16T11:20:31.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6d4b10700 1 -- 172.21.15.181:0/1766114299 learned_addr learned my addr 172.21.15.181:0/1766114299 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:31.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6cf7fe700 1 --2- 172.21.15.181:0/1766114299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015eab0 0x7fa6d015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6cffff700 1 -- 172.21.15.181:0/1766114299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015eab0 msgr2=0x7fa6d015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:31.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6cffff700 1 --2- 172.21.15.181:0/1766114299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015eab0 0x7fa6d015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6cffff700 1 -- 172.21.15.181:0/1766114299 >> v1:172.21.15.181:6789/0 conn(0x7fa6d00cb0c0 legacy=0x7fa6d015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6cffff700 1 -- 172.21.15.181:0/1766114299 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d006e0f0 con 0x7fa6d015e6d0 2023-10-16T11:20:31.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6ceffd700 1 -- 172.21.15.181:0/1766114299 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532330363 0 0) 0x7fa6d006ed00 con 0x7fa6d00cb0c0 2023-10-16T11:20:31.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6cffff700 1 --2- 172.21.15.181:0/1766114299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015e6d0 0x7fa6d00d0fc0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009820 tx=0x7fa6c0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7db056cb91a28e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6ceffd700 1 -- 172.21.15.181:0/1766114299 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0005e30 con 0x7fa6d015e6d0 2023-10-16T11:20:31.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6ceffd700 1 -- 172.21.15.181:0/1766114299 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c0006030 con 0x7fa6d015e6d0 2023-10-16T11:20:31.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.166+0000 7fa6ceffd700 1 -- 172.21.15.181:0/1766114299 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00063e0 con 0x7fa6d015e6d0 2023-10-16T11:20:31.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 -- 172.21.15.181:0/1766114299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015e6d0 msgr2=0x7fa6d00d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 --2- 172.21.15.181:0/1766114299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015e6d0 0x7fa6d00d0fc0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009820 tx=0x7fa6c0009b30 comp rx=0 tx=0).stop 2023-10-16T11:20:31.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 -- 172.21.15.181:0/1766114299 shutdown_connections 2023-10-16T11:20:31.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 --2- 172.21.15.181:0/1766114299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015e6d0 0x7fa6d00d0fc0 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 --2- 172.21.15.181:0/1766114299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015eab0 0x7fa6d015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 -- 172.21.15.181:0/1766114299 >> 172.21.15.181:0/1766114299 conn(0x7fa6d00d04a0 msgr2=0x7fa6d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:31.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 -- 172.21.15.181:0/1766114299 shutdown_connections 2023-10-16T11:20:31.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.167+0000 7fa6d6573700 1 -- 172.21.15.181:0/1766114299 wait complete. 2023-10-16T11:20:31.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.168+0000 7fa6d6573700 1 Processor -- start 2023-10-16T11:20:31.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.168+0000 7fa6d6573700 1 -- start start 2023-10-16T11:20:31.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d6573700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00cb0c0 0x7fa6d00ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d6573700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015e6d0 0x7fa6d00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d6573700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015eab0 0x7fa6d01f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d6573700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015d020 con 0x7fa6d015eab0 2023-10-16T11:20:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d6573700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015c3a0 con 0x7fa6d00cb0c0 2023-10-16T11:20:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d6573700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015c680 con 0x7fa6d015e6d0 2023-10-16T11:20:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015eab0 0x7fa6d01f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015eab0 0x7fa6d01f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57852/0 (socket says 172.21.15.181:57852) 2023-10-16T11:20:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 -- 172.21.15.181:0/3521686343 learned_addr learned my addr 172.21.15.181:0/3521686343 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6cf7fe700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015e6d0 0x7fa6d00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6cffff700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00cb0c0 0x7fa6d00ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 -- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015e6d0 msgr2=0x7fa6d00c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015e6d0 0x7fa6d00c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 -- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00cb0c0 msgr2=0x7fa6d00ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00cb0c0 0x7fa6d00ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 -- 172.21.15.181:0/3521686343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0009430 con 0x7fa6d015eab0 2023-10-16T11:20:31.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.169+0000 7fa6d4b10700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015eab0 0x7fa6d01f3070 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800cba0 tx=0x7fa6c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.170+0000 7fa6ccff9700 1 -- 172.21.15.181:0/3521686343 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800ab80 con 0x7fa6d015eab0 2023-10-16T11:20:31.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.170+0000 7fa6ccff9700 1 -- 172.21.15.181:0/3521686343 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c800ad00 con 0x7fa6d015eab0 2023-10-16T11:20:31.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.170+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d015c960 con 0x7fa6d015eab0 2023-10-16T11:20:31.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.170+0000 7fa6ccff9700 1 -- 172.21.15.181:0/3521686343 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8007310 con 0x7fa6d015eab0 2023-10-16T11:20:31.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.170+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d00ca560 con 0x7fa6d015eab0 2023-10-16T11:20:31.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.171+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d015d160 con 0x7fa6d015eab0 2023-10-16T11:20:31.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.171+0000 7fa6ccff9700 1 -- 172.21.15.181:0/3521686343 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa6c8007470 con 0x7fa6d015eab0 2023-10-16T11:20:31.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.171+0000 7fa6ccff9700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6b006b140 0x7fa6b006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.171+0000 7fa6ccff9700 1 -- 172.21.15.181:0/3521686343 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fa6c8089a00 con 0x7fa6d015eab0 2023-10-16T11:20:31.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.171+0000 7fa6cffff700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6b006b140 0x7fa6b006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.172+0000 7fa6cffff700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6b006b140 0x7fa6b006d600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cd670 tx=0x7fa6c0008070 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.176+0000 7fa6ccff9700 1 -- 172.21.15.181:0/3521686343 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa6d015d160 con 0x7fa6d015eab0 2023-10-16T11:20:31.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.414+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6d015d160 con 0x7fa6d015eab0 2023-10-16T11:20:31.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.415+0000 7fa6ccff9700 1 -- 172.21.15.181:0/3521686343 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4939 (secure 0 0 0) 0x7fa6d015d160 con 0x7fa6d015eab0 2023-10-16T11:20:31.417 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:31.418 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":10690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-16T11:20:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6b006b140 msgr2=0x7fa6b006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6b006b140 0x7fa6b006d600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cd670 tx=0x7fa6c0008070 comp rx=0 tx=0).stop 2023-10-16T11:20:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015eab0 msgr2=0x7fa6d01f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015eab0 0x7fa6d01f3070 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800cba0 tx=0x7fa6c800cf60 comp rx=0 tx=0).stop 2023-10-16T11:20:31.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 shutdown_connections 2023-10-16T11:20:31.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fa6b006b140 0x7fa6b006d600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d015eab0 0x7fa6d01f3070 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d00cb0c0 0x7fa6d00ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 --2- 172.21.15.181:0/3521686343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d015e6d0 0x7fa6d00c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 >> 172.21.15.181:0/3521686343 conn(0x7fa6d00d04a0 msgr2=0x7fa6d00b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:31.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 shutdown_connections 2023-10-16T11:20:31.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.417+0000 7fa6d6573700 1 -- 172.21.15.181:0/3521686343 wait complete. 2023-10-16T11:20:31.422 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 128 2023-10-16T11:20:31.834 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:20:25.468288+0000', 'modified': '2023-10-16T11:20:28.484587+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12171': {'gid': 12171, 'name': 'a', 'rank': 0, 'incarnation': 124, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6836/3483656577', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3483656577}, {'type': 'v1', 'addr': '172.21.15.181:6836', 'nonce': 3483656577}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:20:31.834 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:20:31.835 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-16T11:20:31.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.959+0000 7ff77c45f700 1 Processor -- start 2023-10-16T11:20:31.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.959+0000 7ff77c45f700 1 -- start start 2023-10-16T11:20:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.960+0000 7ff77c45f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7740d2300 0x7ff7740b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.960+0000 7ff77c45f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7740b0410 con 0x7ff7740d1e80 2023-10-16T11:20:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.960+0000 7ff77c45f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7740d2790 con 0x7ff77415dd50 2023-10-16T11:20:31.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.960+0000 7ff77c45f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7740b3c00 con 0x7ff7740d2300 2023-10-16T11:20:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.960+0000 7ff77a9fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7740d2300 0x7ff7740b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.960+0000 7ff7799fa700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff7740d1e80 0x7ff7740b4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40702/0 (socket says 172.21.15.181:40702) 2023-10-16T11:20:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.960+0000 7ff7799fa700 1 -- 172.21.15.181:0/306780757 learned_addr learned my addr 172.21.15.181:0/306780757 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff77a9fc700 1 -- 172.21.15.181:0/306780757 >> v1:172.21.15.190:6789/0 conn(0x7ff77415dd50 legacy=0x7ff7740d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:31.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff77a9fc700 1 -- 172.21.15.181:0/306780757 >> v1:172.21.15.181:6789/0 conn(0x7ff7740d1e80 legacy=0x7ff7740b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff7791f9700 1 -- 172.21.15.181:0/306780757 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691535364 0 0) 0x7ff7740b0410 con 0x7ff7740d1e80 2023-10-16T11:20:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff77a9fc700 1 -- 172.21.15.181:0/306780757 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740b2ca0 con 0x7ff7740d2300 2023-10-16T11:20:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/306780757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff7740b4730 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009d90 tx=0x7ff76c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0432922b17a43a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff7791f9700 1 -- 172.21.15.181:0/306780757 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c00e070 con 0x7ff7740d2300 2023-10-16T11:20:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff7791f9700 1 -- 172.21.15.181:0/306780757 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00bb90 con 0x7ff7740d2300 2023-10-16T11:20:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff7791f9700 1 -- 172.21.15.181:0/306780757 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c003a20 con 0x7ff7740d2300 2023-10-16T11:20:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff77c45f700 1 -- 172.21.15.181:0/306780757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 msgr2=0x7ff7740b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.961+0000 7ff77c45f700 1 --2- 172.21.15.181:0/306780757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff7740b4730 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009d90 tx=0x7ff76c00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.962+0000 7ff77c45f700 1 -- 172.21.15.181:0/306780757 shutdown_connections 2023-10-16T11:20:31.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.962+0000 7ff77c45f700 1 --2- 172.21.15.181:0/306780757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff7740b4730 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.962+0000 7ff77c45f700 1 -- 172.21.15.181:0/306780757 >> 172.21.15.181:0/306780757 conn(0x7ff77408af90 msgr2=0x7ff7740d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:31.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.962+0000 7ff77c45f700 1 -- 172.21.15.181:0/306780757 shutdown_connections 2023-10-16T11:20:31.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.962+0000 7ff77c45f700 1 -- 172.21.15.181:0/306780757 wait complete. 2023-10-16T11:20:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.963+0000 7ff77c45f700 1 Processor -- start 2023-10-16T11:20:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.963+0000 7ff77c45f700 1 -- start start 2023-10-16T11:20:31.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7740d1e80 0x7ff7741f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff774150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77415dd50 0x7ff7740ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff774150bd0 con 0x7ff7740d2300 2023-10-16T11:20:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740caeb0 con 0x7ff77415dd50 2023-10-16T11:20:31.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740cc650 con 0x7ff7740d1e80 2023-10-16T11:20:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff774150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff774150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57864/0 (socket says 172.21.15.181:57864) 2023-10-16T11:20:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 -- 172.21.15.181:0/3442843341 learned_addr learned my addr 172.21.15.181:0/3442843341 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77415dd50 0x7ff7740ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7740d1e80 0x7ff7741f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 -- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7740d1e80 msgr2=0x7ff7741f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7740d1e80 0x7ff7741f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 -- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77415dd50 msgr2=0x7ff7740ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77415dd50 0x7ff7740ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 -- 172.21.15.181:0/3442843341 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c009a70 con 0x7ff7740d2300 2023-10-16T11:20:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77415dd50 0x7ff7740ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7740d1e80 0x7ff7741f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff7799fa700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff774150690 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7ff7700011e0 tx=0x7ff77000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff76affd700 1 -- 172.21.15.181:0/3442843341 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770004c80 con 0x7ff7740d2300 2023-10-16T11:20:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff76affd700 1 -- 172.21.15.181:0/3442843341 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770004e00 con 0x7ff7740d2300 2023-10-16T11:20:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7740cc930 con 0x7ff7740d2300 2023-10-16T11:20:31.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff76affd700 1 -- 172.21.15.181:0/3442843341 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770007310 con 0x7ff7740d2300 2023-10-16T11:20:31.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.964+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7740ccef0 con 0x7ff7740d2300 2023-10-16T11:20:31.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.966+0000 7ff76affd700 1 -- 172.21.15.181:0/3442843341 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff770007470 con 0x7ff7740d2300 2023-10-16T11:20:31.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.966+0000 7ff76affd700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff75806b140 0x7ff75806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.966+0000 7ff76affd700 1 -- 172.21.15.181:0/3442843341 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ff770089a80 con 0x7ff7740d2300 2023-10-16T11:20:31.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.966+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff75806b140 0x7ff75806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.967+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff75806b140 0x7ff75806d600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff76400a8b0 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.968+0000 7ff788649b80 1 Processor -- start 2023-10-16T11:20:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.968+0000 7ff788649b80 1 -- start start 2023-10-16T11:20:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff788649b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf2bbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf28f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff788649b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf205c20 con 0x5596cf2b9630 2023-10-16T11:20:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf1eac10 con 0x5596cf28f010 2023-10-16T11:20:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596cf1e8530 con 0x5596cf2b4220 2023-10-16T11:20:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf2bbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf2bbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57872/0 (socket says 172.21.15.181:57872) 2023-10-16T11:20:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 -- 172.21.15.181:0/1477271932 learned_addr learned my addr 172.21.15.181:0/1477271932 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf28f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 -- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 msgr2=0x5596cf2bba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 -- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 msgr2=0x5596cf28f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf28f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 -- 172.21.15.181:0/1477271932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596cf1a13a0 con 0x5596cf2b9630 2023-10-16T11:20:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.969+0000 7ff7799fa700 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf2bbfb0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7ff770007fd0 tx=0x7ff770057b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f2c790b250b469 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff757fff700 1 -- 172.21.15.181:0/1477271932 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7700581c0 con 0x5596cf2b9630 2023-10-16T11:20:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff757fff700 1 -- 172.21.15.181:0/1477271932 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770089040 con 0x5596cf2b9630 2023-10-16T11:20:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff757fff700 1 -- 172.21.15.181:0/1477271932 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77005c310 con 0x5596cf2b9630 2023-10-16T11:20:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 -- 172.21.15.181:0/1477271932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 msgr2=0x5596cf2bbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf2bbfb0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7ff770007fd0 tx=0x7ff770057b20 comp rx=0 tx=0).stop 2023-10-16T11:20:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 -- 172.21.15.181:0/1477271932 shutdown_connections 2023-10-16T11:20:31.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf28f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 --2- 172.21.15.181:0/1477271932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf2bbfb0 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 -- 172.21.15.181:0/1477271932 >> 172.21.15.181:0/1477271932 conn(0x5596cf1bed80 msgr2=0x5596cf2d9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:31.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 -- 172.21.15.181:0/1477271932 shutdown_connections 2023-10-16T11:20:31.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.970+0000 7ff788649b80 1 -- 172.21.15.181:0/1477271932 wait complete. 2023-10-16T11:20:31.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.971+0000 7ff788649b80 1 -- start start 2023-10-16T11:20:31.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.971+0000 7ff788649b80 1 Processor -- start 2023-10-16T11:20:31.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff788649b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf28f010 0x5596cf2bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b9630 0x5596cf2b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff788649b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf1e3e20 con 0x5596cf28f010 2023-10-16T11:20:31.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf1806c0 con 0x5596cf2b9630 2023-10-16T11:20:31.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596cf180360 con 0x5596cf2b4220 2023-10-16T11:20:31.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff788649b80 10 client.0 _ll_register_callbacks cb 0x5596cefdd7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:20:31.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf28f010 0x5596cf2bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf28f010 0x5596cf2bb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57876/0 (socket says 172.21.15.181:57876) 2023-10-16T11:20:31.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 -- 172.21.15.181:0/1416129276 learned_addr learned my addr 172.21.15.181:0/1416129276 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:31.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff7799fa700 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b9630 0x5596cf2b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 -- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 msgr2=0x5596cf2bba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:31.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 -- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b9630 msgr2=0x5596cf2b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:31.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b9630 0x5596cf2b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.972+0000 7ff77a1fb700 1 -- 172.21.15.181:0/1416129276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7740ccef0 con 0x5596cf28f010 2023-10-16T11:20:31.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf28f010 0x5596cf2bb550 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ff764013e40 tx=0x7ff764004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cea65698dc0720 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:31.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7557fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:20:31.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7557fa700 1 -- 172.21.15.181:0/1416129276 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764022070 con 0x5596cf28f010 2023-10-16T11:20:31.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7557fa700 1 -- 172.21.15.181:0/1416129276 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ff764011e00 con 0x5596cf28f010 2023-10-16T11:20:31.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596cf2845b0 con 0x5596cf28f010 2023-10-16T11:20:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7557fa700 1 -- 172.21.15.181:0/1416129276 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff764004020 con 0x5596cf28f010 2023-10-16T11:20:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 10 client.12318 fetch_fsmap learned FSMap version 128 2023-10-16T11:20:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x5596cf00ed90 con 0x5596cf28f010 2023-10-16T11:20:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7557fa700 1 -- 172.21.15.181:0/1416129276 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5596cf2845b0 con 0x5596cf28f010 2023-10-16T11:20:31.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7557fa700 1 -- 172.21.15.181:0/1416129276 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff764025020 con 0x5596cf28f010 2023-10-16T11:20:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 10 client.12318 fetch_fsmap finished waiting for FSMap version 128 2023-10-16T11:20:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 10 client.12318 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:20:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 20 client.12318 populate_metadata read hostname 'smithi181' 2023-10-16T11:20:31.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf2b62e0 0x5596cf2bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:31.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 4 client.12318 mds_command: new command op to 12171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:20:31.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 --> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596cf1dce00 con 0x5596cf2b62e0 2023-10-16T11:20:31.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7799fa700 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf2b62e0 0x5596cf2bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:31.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.973+0000 7ff7557fa700 10 client.12318 ms_handle_connect on v2:172.21.15.181:6834/3483656577 2023-10-16T11:20:31.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff7799fa700 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf2b62e0 0x5596cf2bc730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa661bc901be9f14 server_cookie=1f37e57aa098f72 in_seq=0 out_seq=0 2023-10-16T11:20:31.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff7557fa700 1 -- 172.21.15.181:0/1416129276 <== mds.0 v2:172.21.15.181:6834/3483656577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596cf1dce00 con 0x5596cf2b62e0 2023-10-16T11:20:31.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff7557fa700 10 client.12318 handle_command_reply: tid=0 2023-10-16T11:20:31.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 client.12318 shutdown 2023-10-16T11:20:31.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf28f010 msgr2=0x5596cf2bb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:31.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf28f010 0x5596cf2bb550 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ff764013e40 tx=0x7ff764004760 comp rx=0 tx=0).stop 2023-10-16T11:20:31.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 shutdown_connections 2023-10-16T11:20:31.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf2b62e0 0x5596cf2bc730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b9630 0x5596cf2b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf28f010 0x5596cf2bb550 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff7557fa700 0 client.12318 ms_handle_reset on v2:172.21.15.181:6834/3483656577 2023-10-16T11:20:31.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 --2- 172.21.15.181:0/1416129276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b4220 0x5596cf2bba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:31.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 >> 172.21.15.181:0/1416129276 conn(0x5596cf1bed80 msgr2=0x5596cf2d7550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:31.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 shutdown_connections 2023-10-16T11:20:31.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.974+0000 7ff788649b80 1 -- 172.21.15.181:0/1416129276 wait complete. 2023-10-16T11:20:31.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.975+0000 7ff788649b80 20 client.12318 trim_cache size 0 max 16384 2023-10-16T11:20:32.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.992+0000 7ff788649b80 1 Processor -- start 2023-10-16T11:20:32.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.992+0000 7ff788649b80 1 -- start start 2023-10-16T11:20:32.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf2e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff788649b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b9630 0x5596cf337750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff788649b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf0648d0 con 0x5596cf2b4220 2023-10-16T11:20:32.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf2b2ce0 con 0x5596cf28f010 2023-10-16T11:20:32.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596cf28dcc0 con 0x5596cf2b9630 2023-10-16T11:20:32.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57886/0 (socket says 172.21.15.181:57886) 2023-10-16T11:20:32.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 -- 172.21.15.181:0/273763015 learned_addr learned my addr 172.21.15.181:0/273763015 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:32.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 -- 172.21.15.181:0/273763015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b9630 msgr2=0x5596cf337750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:32.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf2e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b9630 0x5596cf337750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 -- 172.21.15.181:0/273763015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 msgr2=0x5596cf2e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:32.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf2e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 -- 172.21.15.181:0/273763015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740b4fe0 con 0x5596cf2b4220 2023-10-16T11:20:32.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff7799fa700 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7ff770057b60 tx=0x7ff770054630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8fc03fe6f1b3cb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:32.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff768d82700 1 -- 172.21.15.181:0/273763015 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770087040 con 0x5596cf2b4220 2023-10-16T11:20:32.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff768d82700 1 -- 172.21.15.181:0/273763015 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770060540 con 0x5596cf2b4220 2023-10-16T11:20:32.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.993+0000 7ff768d82700 1 -- 172.21.15.181:0/273763015 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7700581c0 con 0x5596cf2b4220 2023-10-16T11:20:32.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 -- 172.21.15.181:0/273763015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b4220 msgr2=0x5596cf28ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7ff770057b60 tx=0x7ff770054630 comp rx=0 tx=0).stop 2023-10-16T11:20:32.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 -- 172.21.15.181:0/273763015 shutdown_connections 2023-10-16T11:20:32.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf2b9630 0x5596cf337750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf28f010 0x5596cf2e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 --2- 172.21.15.181:0/273763015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 -- 172.21.15.181:0/273763015 >> 172.21.15.181:0/273763015 conn(0x5596cf1bed80 msgr2=0x5596cf2d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:32.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 -- 172.21.15.181:0/273763015 shutdown_connections 2023-10-16T11:20:32.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 -- 172.21.15.181:0/273763015 wait complete. 2023-10-16T11:20:32.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 -- start start 2023-10-16T11:20:32.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.994+0000 7ff788649b80 1 Processor -- start 2023-10-16T11:20:32.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf28f010 0x5596cf2b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff788649b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff788649b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf28bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff788649b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf3200f0 con 0x5596cf2b9630 2023-10-16T11:20:32.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cf2b8e70 con 0x5596cf2b4220 2023-10-16T11:20:32.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff788649b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596cf2da8d0 con 0x5596cf28f010 2023-10-16T11:20:32.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff788649b80 10 client.0 _ll_register_callbacks cb 0x7ff7640137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:20:32.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff77a9fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf28bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff77a9fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf28bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57894/0 (socket says 172.21.15.181:57894) 2023-10-16T11:20:32.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.995+0000 7ff77a9fc700 1 -- 172.21.15.181:0/304173388 learned_addr learned my addr 172.21.15.181:0/304173388 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:32.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf28f010 0x5596cf2b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7799fa700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff77a9fc700 1 -- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf28f010 msgr2=0x5596cf2b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:32.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf28f010 0x5596cf2b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff77a9fc700 1 -- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b4220 msgr2=0x5596cf28ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:32.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff77a9fc700 1 -- 172.21.15.181:0/304173388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596cf1a13a0 con 0x5596cf2b9630 2023-10-16T11:20:32.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7799fa700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:32.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff77a9fc700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf28bfd0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7ff76c004000 tx=0x7ff76c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8eeab24d6bee51 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:32.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7557fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:20:32.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7557fa700 1 -- 172.21.15.181:0/304173388 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004820 con 0x5596cf2b9630 2023-10-16T11:20:32.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7557fa700 1 -- 172.21.15.181:0/304173388 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ff76c009410 con 0x5596cf2b9630 2023-10-16T11:20:32.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff770089040 con 0x5596cf2b9630 2023-10-16T11:20:32.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7557fa700 1 -- 172.21.15.181:0/304173388 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76c026470 con 0x5596cf2b9630 2023-10-16T11:20:32.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 10 client.12324 fetch_fsmap learned FSMap version 128 2023-10-16T11:20:32.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x5596ceff7c10 con 0x5596cf2b9630 2023-10-16T11:20:32.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7557fa700 1 -- 172.21.15.181:0/304173388 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770089040 con 0x5596cf2b9630 2023-10-16T11:20:32.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff7557fa700 1 -- 172.21.15.181:0/304173388 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff76c00b050 con 0x5596cf2b9630 2023-10-16T11:20:32.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 10 client.12324 fetch_fsmap finished waiting for FSMap version 128 2023-10-16T11:20:32.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 10 client.12324 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-16T11:20:32.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 20 client.12324 populate_metadata read hostname 'smithi181' 2023-10-16T11:20:32.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf3352f0 0x5596cf2e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 4 client.12324 mds_command: new command op to 12171 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:20:32.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.996+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 --> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5596cf1dcb20 con 0x5596cf3352f0 2023-10-16T11:20:32.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf3352f0 0x5596cf2e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff7557fa700 10 client.12324 ms_handle_connect on v2:172.21.15.181:6834/3483656577 2023-10-16T11:20:32.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff77a1fb700 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf3352f0 0x5596cf2e8320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b0a4c3035f84bea server_cookie=92aaa416374d53fe in_seq=0 out_seq=0 2023-10-16T11:20:32.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff7557fa700 1 -- 172.21.15.181:0/304173388 <== mds.0 v2:172.21.15.181:6834/3483656577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x5596cf1dcb20 con 0x5596cf3352f0 2023-10-16T11:20:32.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff7557fa700 10 client.12324 handle_command_reply: tid=0 2023-10-16T11:20:32.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 client.12324 shutdown 2023-10-16T11:20:32.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 msgr2=0x5596cf28bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf28bfd0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7ff76c004000 tx=0x7ff76c004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 shutdown_connections 2023-10-16T11:20:32.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5596cf28f010 0x5596cf2b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:6834/3483656577,v1:172.21.15.181:6836/3483656577] conn(0x5596cf3352f0 0x5596cf2e8320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596cf2b9630 0x5596cf28bfd0 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 --2- 172.21.15.181:0/304173388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5596cf2b4220 0x5596cf28ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff7557fa700 0 client.12324 ms_handle_reset on v2:172.21.15.181:6834/3483656577 2023-10-16T11:20:32.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 >> 172.21.15.181:0/304173388 conn(0x5596cf1bed80 msgr2=0x5596cf2b58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:32.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 shutdown_connections 2023-10-16T11:20:32.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 1 -- 172.21.15.181:0/304173388 wait complete. 2023-10-16T11:20:32.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.997+0000 7ff788649b80 20 client.12324 trim_cache size 0 max 16384 2023-10-16T11:20:32.021 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:20:32.022 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:20:32.022 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:20:32.022 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 12171, 2023-10-16T11:20:32.022 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:20:32.022 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:20:32.023 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T11:20:32.023 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9233746739999997, 2023-10-16T11:20:32.023 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 128, 2023-10-16T11:20:32.023 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 236, 2023-10-16T11:20:32.023 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 236, 2023-10-16T11:20:32.024 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.854047333 2023-10-16T11:20:32.024 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:20:32.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff75806b140 msgr2=0x7ff75806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff75806b140 0x7ff75806d600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff76400a8b0 tx=0x7ff764008040 comp rx=0 tx=0).stop 2023-10-16T11:20:32.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 msgr2=0x7ff774150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff774150690 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7ff7700011e0 tx=0x7ff77000cae0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 shutdown_connections 2023-10-16T11:20:32.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff75806b140 0x7ff75806d600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7740d1e80 0x7ff7741f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff77415dd50 0x7ff7740ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 --2- 172.21.15.181:0/3442843341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7740d2300 0x7ff774150690 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 >> 172.21.15.181:0/3442843341 conn(0x7ff77408af90 msgr2=0x7ff77406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:32.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 shutdown_connections 2023-10-16T11:20:32.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:31.998+0000 7ff77c45f700 1 -- 172.21.15.181:0/3442843341 wait complete. 2023-10-16T11:20:32.028 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:20:32.049 INFO:tasks.ceph.mds.a.smithi181.stderr:2023-10-16T11:20:32.048+0000 7f3a4cec7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:20:32.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.148+0000 7f68161ba700 1 Processor -- start 2023-10-16T11:20:32.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.148+0000 7f68161ba700 1 -- start start 2023-10-16T11:20:32.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.149+0000 7f68161ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68100d0370 0x7f68100ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.149+0000 7f68161ba700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68100cd2d0 0x7f68100d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.149+0000 7f68161ba700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68100cc510 con 0x7f681015b990 2023-10-16T11:20:32.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.149+0000 7f68161ba700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100d24f0 con 0x7f68100cd2d0 2023-10-16T11:20:32.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.149+0000 7f68161ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68100cc7f0 con 0x7f68100d0370 2023-10-16T11:20:32.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68100d0370 0x7f68100ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68100cd2d0 0x7f68100d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68100cd2d0 0x7f68100d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:34720/0 (socket says 172.21.15.181:34720) 2023-10-16T11:20:32.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680effd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68100d0370 0x7f68100ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:54926/0 (socket says 172.21.15.181:54926) 2023-10-16T11:20:32.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 -- 172.21.15.181:0/1184588475 learned_addr learned my addr 172.21.15.181:0/1184588475 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:32.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680f7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f681015b990 0x7f68100cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:52918/0 (socket says 172.21.15.181:52918) 2023-10-16T11:20:32.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 -- 172.21.15.181:0/1184588475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68100d0370 msgr2=0x7f68100ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:32.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 --2- 172.21.15.181:0/1184588475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68100d0370 0x7f68100ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 -- 172.21.15.181:0/1184588475 >> v1:172.21.15.190:6789/0 conn(0x7f681015b990 legacy=0x7f68100cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 -- 172.21.15.181:0/1184588475 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100ccad0 con 0x7f68100cd2d0 2023-10-16T11:20:32.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680e7fc700 1 -- 172.21.15.181:0/1184588475 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837660581 0 0) 0x7f68100cc510 con 0x7f681015b990 2023-10-16T11:20:32.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680ffff700 1 --2- 172.21.15.181:0/1184588475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100cd2d0 0x7f68100d1fb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6808009e50 tx=0x7f680800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73de04421a96f66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:32.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680e7fc700 1 -- 172.21.15.181:0/1184588475 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800e070 con 0x7f68100cd2d0 2023-10-16T11:20:32.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680e7fc700 1 -- 172.21.15.181:0/1184588475 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680800b910 con 0x7f68100cd2d0 2023-10-16T11:20:32.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.150+0000 7f680e7fc700 1 -- 172.21.15.181:0/1184588475 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800bcc0 con 0x7f68100cd2d0 2023-10-16T11:20:32.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 -- 172.21.15.181:0/1184588475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100cd2d0 msgr2=0x7f68100d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 --2- 172.21.15.181:0/1184588475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100cd2d0 0x7f68100d1fb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6808009e50 tx=0x7f680800b330 comp rx=0 tx=0).stop 2023-10-16T11:20:32.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 -- 172.21.15.181:0/1184588475 shutdown_connections 2023-10-16T11:20:32.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 --2- 172.21.15.181:0/1184588475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100cd2d0 0x7f68100d1fb0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 --2- 172.21.15.181:0/1184588475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68100d0370 0x7f68100ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 -- 172.21.15.181:0/1184588475 >> 172.21.15.181:0/1184588475 conn(0x7f681009a9a0 msgr2=0x7f68100d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:32.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 -- 172.21.15.181:0/1184588475 shutdown_connections 2023-10-16T11:20:32.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.151+0000 7f68161ba700 1 -- 172.21.15.181:0/1184588475 wait complete. 2023-10-16T11:20:32.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.152+0000 7f68161ba700 1 Processor -- start 2023-10-16T11:20:32.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.152+0000 7f68161ba700 1 -- start start 2023-10-16T11:20:32.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f68161ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68100cd2d0 0x7f681015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f68161ba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100d0370 0x7f6810154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f68161ba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f681015b990 0x7f68101f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f68161ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100b4900 con 0x7f68100cd2d0 2023-10-16T11:20:32.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f68161ba700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681015d090 con 0x7f68100d0370 2023-10-16T11:20:32.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f68161ba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68101f5990 con 0x7f681015b990 2023-10-16T11:20:32.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f680f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68100cd2d0 0x7f681015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f680f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68100cd2d0 0x7f681015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57904/0 (socket says 172.21.15.181:57904) 2023-10-16T11:20:32.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f680f7fe700 1 -- 172.21.15.181:0/3313291223 learned_addr learned my addr 172.21.15.181:0/3313291223 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:32.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f680effd700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100d0370 0x7f6810154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.153+0000 7f680ffff700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f681015b990 0x7f68101f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f680f7fe700 1 -- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f681015b990 msgr2=0x7f68101f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f680f7fe700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f681015b990 0x7f68101f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f680f7fe700 1 -- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100d0370 msgr2=0x7f6810154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f680f7fe700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100d0370 0x7f6810154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f680f7fe700 1 -- 172.21.15.181:0/3313291223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808009b30 con 0x7f68100cd2d0 2023-10-16T11:20:32.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f680f7fe700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68100cd2d0 0x7f681015c4e0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f680400c4b0 tx=0x7f680400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:32.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f67f7fff700 1 -- 172.21.15.181:0/3313291223 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804004cd0 con 0x7f68100cd2d0 2023-10-16T11:20:32.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f67f7fff700 1 -- 172.21.15.181:0/3313291223 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804004e50 con 0x7f68100cd2d0 2023-10-16T11:20:32.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68100b4340 con 0x7f68100cd2d0 2023-10-16T11:20:32.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f67f7fff700 1 -- 172.21.15.181:0/3313291223 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804007310 con 0x7f68100cd2d0 2023-10-16T11:20:32.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.154+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681015cd20 con 0x7f68100cd2d0 2023-10-16T11:20:32.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.155+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810067d20 con 0x7f68100cd2d0 2023-10-16T11:20:32.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.155+0000 7f67f7fff700 1 -- 172.21.15.181:0/3313291223 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6804007470 con 0x7f68100cd2d0 2023-10-16T11:20:32.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.156+0000 7f67f7fff700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f67fc06b140 0x7f67fc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:32.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.156+0000 7f67f7fff700 1 -- 172.21.15.181:0/3313291223 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f6804089850 con 0x7f68100cd2d0 2023-10-16T11:20:32.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.156+0000 7f680effd700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f67fc06b140 0x7f67fc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:32.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.156+0000 7f680effd700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f67fc06b140 0x7f67fc06d600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6810159aa0 tx=0x7f6800009210 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:32.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.160+0000 7f67f7fff700 1 -- 172.21.15.181:0/3313291223 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6804054370 con 0x7f68100cd2d0 2023-10-16T11:20:32.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.398+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f68100d08f0 con 0x7f68100cd2d0 2023-10-16T11:20:32.400 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.398+0000 7f67f7fff700 1 -- 172.21.15.181:0/3313291223 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v128) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6804057990 con 0x7f68100cd2d0 2023-10-16T11:20:32.400 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:20:32.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f67fc06b140 msgr2=0x7f67fc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f67fc06b140 0x7f67fc06d600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6810159aa0 tx=0x7f6800009210 comp rx=0 tx=0).stop 2023-10-16T11:20:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68100cd2d0 msgr2=0x7f681015c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68100cd2d0 0x7f681015c4e0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f680400c4b0 tx=0x7f680400c7c0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 shutdown_connections 2023-10-16T11:20:32.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f67fc06b140 0x7f67fc06d600 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f681015b990 0x7f68101f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68100cd2d0 0x7f681015c4e0 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 --2- 172.21.15.181:0/3313291223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68100d0370 0x7f6810154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 >> 172.21.15.181:0/3313291223 conn(0x7f681009a9a0 msgr2=0x7f68100cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:32.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 shutdown_connections 2023-10-16T11:20:32.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:32.400+0000 7f68161ba700 1 -- 172.21.15.181:0/3313291223 wait complete. 2023-10-16T11:20:32.414 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:20:32.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:20:32.415 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T11:20:32.415 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:20:32.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:20:32.416 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:20:32.416 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:20:32.417 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:20:32.417 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:20:32.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:32.418 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T11:20:32.472 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:20:32.472 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:20:32.473 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:20:32.473 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:20:32.473 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T11:20:32.473 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:20:32.473 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:20:32.474 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:20:32.474 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:20:32.474 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:20:32.474 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:20:32.474 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T11:20:32.475 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:20:32.475 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T11:20:32.475 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:20:32.476 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 36637sec preferred_lft 36637sec 2023-10-16T11:20:32.476 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T11:20:32.476 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:20:32.477 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:20:32.477 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:20:32.477 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:20:32.478 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:20:32.478 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T11:20:32.478 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:20:32.478 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:20:32.479 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:20:32.479 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T11:20:32.479 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:20:32.481 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:20:32.481 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:20:32.482 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T11:20:32.482 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:20:32.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:32.483 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:20:32.538 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T11:20:32.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:32.540 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T11:20:32.598 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T11:20:32.598 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:20:32.598 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:20:32.598 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:20:32.599 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:20:32.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:32.599 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T11:20:32.600 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T11:20:32.600 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:20:32.600 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:20:32.601 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:20:32.601 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T11:20:32.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:32.703 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:20:32.720 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:20:32.720 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:20:32.720 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:20:32.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:32.721 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T11:20:32.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:20:32.787 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:20:32.843 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:20:32.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:20:32.844 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:20:32.900 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T11:20:32.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:32.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:20:32.986 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:20:32.986 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:20:32.987 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:20:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:20:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:20:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:20:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:20:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:20:33.070 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:20:33.070 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:20:33.070 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:20:33.071 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:20:33.071 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:20:33.071 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T11:20:33.071 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T11:20:33.071 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T11:20:33.073 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:20:33.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:33.073 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:20:33.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:33.101 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:20:33.182 INFO:teuthology.orchestra.run.smithi181.stdout:client10783 2023-10-16T11:20:33.182 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:20:33.183 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:33.240 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-16T11:20:33.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.402+0000 7f30aaba8700 1 Processor -- start 2023-10-16T11:20:33.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.402+0000 7f30aaba8700 1 -- start start 2023-10-16T11:20:33.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30aaba8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30a415eea0 0x7f30a40b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:33.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30aaba8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a415f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:33.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30aaba8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a40d23c0 con 0x7f30a40b0f70 2023-10-16T11:20:33.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30aaba8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a40cd990 con 0x7f30a415eea0 2023-10-16T11:20:33.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30aaba8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40cdc70 con 0x7f30a40ce810 2023-10-16T11:20:33.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30a8944700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30a415eea0 0x7f30a40b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:33.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30a9145700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:33.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30a3fff700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f30a40b0f70 0x7f30a40ce100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:52938/0 (socket says 172.21.15.181:52938) 2023-10-16T11:20:33.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30a8944700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30a415eea0 0x7f30a40b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:34734/0 (socket says 172.21.15.181:34734) 2023-10-16T11:20:33.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30a9145700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a415f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:54946/0 (socket says 172.21.15.181:54946) 2023-10-16T11:20:33.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.403+0000 7f30a3fff700 1 -- 172.21.15.181:0/4041243296 learned_addr learned my addr 172.21.15.181:0/4041243296 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:33.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a8944700 1 -- 172.21.15.181:0/4041243296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 msgr2=0x7f30a415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:33.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a8944700 1 --2- 172.21.15.181:0/4041243296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a415f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:33.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a8944700 1 -- 172.21.15.181:0/4041243296 >> v1:172.21.15.190:6789/0 conn(0x7f30a40b0f70 legacy=0x7f30a40ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:33.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a8944700 1 -- 172.21.15.181:0/4041243296 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40cce60 con 0x7f30a415eea0 2023-10-16T11:20:33.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a37fe700 1 -- 172.21.15.181:0/4041243296 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472237949 0 0) 0x7f30a40d23c0 con 0x7f30a40b0f70 2023-10-16T11:20:33.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a8944700 1 --2- 172.21.15.181:0/4041243296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a415eea0 0x7f30a40b03e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3098009840 tx=0x7f3098009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb66d13468c9b7ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:33.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a37fe700 1 -- 172.21.15.181:0/4041243296 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098011830 con 0x7f30a415eea0 2023-10-16T11:20:33.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a37fe700 1 -- 172.21.15.181:0/4041243296 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3098011990 con 0x7f30a415eea0 2023-10-16T11:20:33.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.404+0000 7f30a37fe700 1 -- 172.21.15.181:0/4041243296 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098011d20 con 0x7f30a415eea0 2023-10-16T11:20:33.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 -- 172.21.15.181:0/4041243296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a415eea0 msgr2=0x7f30a40b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:33.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 --2- 172.21.15.181:0/4041243296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a415eea0 0x7f30a40b03e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3098009840 tx=0x7f3098009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:33.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 -- 172.21.15.181:0/4041243296 shutdown_connections 2023-10-16T11:20:33.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 --2- 172.21.15.181:0/4041243296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:33.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 --2- 172.21.15.181:0/4041243296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a415eea0 0x7f30a40b03e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:33.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 -- 172.21.15.181:0/4041243296 >> 172.21.15.181:0/4041243296 conn(0x7f30a40d1610 msgr2=0x7f30a40d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:33.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 -- 172.21.15.181:0/4041243296 shutdown_connections 2023-10-16T11:20:33.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.405+0000 7f30aaba8700 1 -- 172.21.15.181:0/4041243296 wait complete. 2023-10-16T11:20:33.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.406+0000 7f30aaba8700 1 Processor -- start 2023-10-16T11:20:33.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.406+0000 7f30aaba8700 1 -- start start 2023-10-16T11:20:33.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30aaba8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40b0f70 0x7f30a415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:33.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30aaba8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a4158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:33.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30aaba8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415eea0 0x7f30a40b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:33.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30aaba8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a415d380 con 0x7f30a415eea0 2023-10-16T11:20:33.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30aaba8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40b53f0 con 0x7f30a40b0f70 2023-10-16T11:20:33.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30aaba8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a415c700 con 0x7f30a40ce810 2023-10-16T11:20:33.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415eea0 0x7f30a40b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:33.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415eea0 0x7f30a40b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57926/0 (socket says 172.21.15.181:57926) 2023-10-16T11:20:33.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 -- 172.21.15.181:0/1450269082 learned_addr learned my addr 172.21.15.181:0/1450269082 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:33.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a3fff700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a4158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:33.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a8944700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40b0f70 0x7f30a415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:33.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 -- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 msgr2=0x7f30a4158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:33.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a4158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:33.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 -- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40b0f70 msgr2=0x7f30a415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:33.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40b0f70 0x7f30a415d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:33.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 -- 172.21.15.181:0/1450269082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980094a0 con 0x7f30a415eea0 2023-10-16T11:20:33.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a8944700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40b0f70 0x7f30a415d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:33.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a9145700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415eea0 0x7f30a40b67c0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f309c00efc0 tx=0x7f309c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:33.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a17fa700 1 -- 172.21.15.181:0/1450269082 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c009ea0 con 0x7f30a415eea0 2023-10-16T11:20:33.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.407+0000 7f30a17fa700 1 -- 172.21.15.181:0/1450269082 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c00f040 con 0x7f30a415eea0 2023-10-16T11:20:33.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.408+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a415c9e0 con 0x7f30a415eea0 2023-10-16T11:20:33.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.408+0000 7f30a17fa700 1 -- 172.21.15.181:0/1450269082 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c014480 con 0x7f30a415eea0 2023-10-16T11:20:33.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.408+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a415cfa0 con 0x7f30a415eea0 2023-10-16T11:20:33.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.408+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a415d4c0 con 0x7f30a415eea0 2023-10-16T11:20:33.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.409+0000 7f30a17fa700 1 -- 172.21.15.181:0/1450269082 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f309c0145e0 con 0x7f30a415eea0 2023-10-16T11:20:33.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.409+0000 7f30a17fa700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f308806b140 0x7f308806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:33.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.409+0000 7f30a17fa700 1 -- 172.21.15.181:0/1450269082 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f309c08a8c0 con 0x7f30a415eea0 2023-10-16T11:20:33.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.409+0000 7f30a8944700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f308806b140 0x7f308806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:33.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.410+0000 7f30a8944700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f308806b140 0x7f308806d600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30980090b0 tx=0x7f309801c040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:33.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.414+0000 7f30a17fa700 1 -- 172.21.15.181:0/1450269082 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f30a415d4c0 con 0x7f30a415eea0 2023-10-16T11:20:33.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:33.649+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f30a415d4c0 con 0x7f30a415eea0 2023-10-16T11:20:34.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.112+0000 7f30a17fa700 1 -- 172.21.15.181:0/1450269082 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v129) v1 ==== 136+0+0 (secure 0 0 0) 0x7f30a415d4c0 con 0x7f30a415eea0 2023-10-16T11:20:34.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f308806b140 msgr2=0x7f308806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f308806b140 0x7f308806d600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30980090b0 tx=0x7f309801c040 comp rx=0 tx=0).stop 2023-10-16T11:20:34.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415eea0 msgr2=0x7f30a40b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415eea0 0x7f30a40b67c0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f309c00efc0 tx=0x7f309c00c3a0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 shutdown_connections 2023-10-16T11:20:34.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f308806b140 0x7f308806d600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a415eea0 0x7f30a40b67c0 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40b0f70 0x7f30a415d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 --2- 172.21.15.181:0/1450269082 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40ce810 0x7f30a4158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 >> 172.21.15.181:0/1450269082 conn(0x7f30a40d1610 msgr2=0x7f30a40ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:34.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 shutdown_connections 2023-10-16T11:20:34.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.115+0000 7f30aaba8700 1 -- 172.21.15.181:0/1450269082 wait complete. 2023-10-16T11:20:34.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:34.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.251+0000 7fbd2a66f700 1 Processor -- start 2023-10-16T11:20:34.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.251+0000 7fbd2a66f700 1 -- start start 2023-10-16T11:20:34.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd2a66f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 0x7fbd240b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd2a66f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240b43d0 con 0x7fbd2415b920 2023-10-16T11:20:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd2a66f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240cc4e0 con 0x7fbd240cbbd0 2023-10-16T11:20:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd2a66f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240d1970 con 0x7fbd240cc050 2023-10-16T11:20:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd23fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbd2415b920 0x7fbd240ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40776/0 (socket says 172.21.15.181:40776) 2023-10-16T11:20:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd23fff700 1 -- 172.21.15.181:0/4266975029 learned_addr learned my addr 172.21.15.181:0/4266975029 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:34.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd28c0c700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 0x7fbd240b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983606215 0 0) 0x7fbd240b43d0 con 0x7fbd2415b920 2023-10-16T11:20:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd0c0035b0 con 0x7fbd2415b920 2023-10-16T11:20:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4285586679 0 0) 0x7fbd0c0035b0 con 0x7fbd2415b920 2023-10-16T11:20:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 msgr2=0x7fbd240b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 0x7fbd240b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 >> v1:172.21.15.190:6789/0 conn(0x7fbd240cbbd0 legacy=0x7fbd240b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240d1c50 con 0x7fbd2415b920 2023-10-16T11:20:34.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fbd14001670 con 0x7fbd2415b920 2023-10-16T11:20:34.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 >> v1:172.21.15.181:6789/0 conn(0x7fbd2415b920 legacy=0x7fbd240ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd0c003a30 0x7fbd0c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c004850 0x7fbd0c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 0x7fbd0c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240d1970 con 0x7fbd0c004850 2023-10-16T11:20:34.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c00bac0 con 0x7fbd0c003a30 2023-10-16T11:20:34.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c00bc80 con 0x7fbd240cc050 2023-10-16T11:20:34.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd237fe700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c004850 0x7fbd0c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.252+0000 7fbd23fff700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd0c003a30 0x7fbd0c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd28c0c700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 0x7fbd0c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd237fe700 1 -- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 msgr2=0x7fbd0c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd237fe700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd240cc050 0x7fbd0c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd237fe700 1 -- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd0c003a30 msgr2=0x7fbd0c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd237fe700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd0c003a30 0x7fbd0c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd237fe700 1 -- 172.21.15.181:0/4266975029 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0035b0 con 0x7fbd0c004850 2023-10-16T11:20:34.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd237fe700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c004850 0x7fbd0c004cd0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fbd18007fd0 tx=0x7fbd0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7d2c62fde73732 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd180012f0 con 0x7fbd0c004850 2023-10-16T11:20:34.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd180052f0 con 0x7fbd0c004850 2023-10-16T11:20:34.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd22ffd700 1 -- 172.21.15.181:0/4266975029 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18006860 con 0x7fbd0c004850 2023-10-16T11:20:34.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd2a66f700 1 -- 172.21.15.181:0/4266975029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c004850 msgr2=0x7fbd0c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd2a66f700 1 --2- 172.21.15.181:0/4266975029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd0c004850 0x7fbd0c004cd0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fbd18007fd0 tx=0x7fbd0c005880 comp rx=0 tx=0).stop 2023-10-16T11:20:34.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd28c0c700 1 -- 172.21.15.181:0/4266975029 reap_dead start 2023-10-16T11:20:34.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd2a66f700 1 -- 172.21.15.181:0/4266975029 shutdown_connections 2023-10-16T11:20:34.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.253+0000 7fbd2a66f700 1 -- 172.21.15.181:0/4266975029 >> 172.21.15.181:0/4266975029 conn(0x7fbd240665a0 msgr2=0x7fbd240669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:34.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.254+0000 7fbd2a66f700 1 -- 172.21.15.181:0/4266975029 shutdown_connections 2023-10-16T11:20:34.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.254+0000 7fbd2a66f700 1 -- 172.21.15.181:0/4266975029 wait complete. 2023-10-16T11:20:34.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.255+0000 7fbd2a66f700 1 Processor -- start 2023-10-16T11:20:34.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.255+0000 7fbd2a66f700 1 -- start start 2023-10-16T11:20:34.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd0c004850 0x7fbd2406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd240afac0 0x7fbd240aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd240c66a0 0x7fbd240c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240b0480 con 0x7fbd240c66a0 2023-10-16T11:20:34.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241f37e0 con 0x7fbd240afac0 2023-10-16T11:20:34.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241f3ac0 con 0x7fbd0c004850 2023-10-16T11:20:34.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd240c66a0 0x7fbd240c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd240c66a0 0x7fbd240c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57946/0 (socket says 172.21.15.181:57946) 2023-10-16T11:20:34.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 -- 172.21.15.181:0/1209645245 learned_addr learned my addr 172.21.15.181:0/1209645245 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:34.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd237fe700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd240afac0 0x7fbd240aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd23fff700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd0c004850 0x7fbd2406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 -- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd0c004850 msgr2=0x7fbd2406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd0c004850 0x7fbd2406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 -- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd240afac0 msgr2=0x7fbd240aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd240afac0 0x7fbd240aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 -- 172.21.15.181:0/1209645245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd241f3da0 con 0x7fbd240c66a0 2023-10-16T11:20:34.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd23fff700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd0c004850 0x7fbd2406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:34.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd237fe700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd240afac0 0x7fbd240aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:34.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd28c0c700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd240c66a0 0x7fbd240c6b50 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fbd2406fd20 tx=0x7fbd1c00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd20ff9700 1 -- 172.21.15.181:0/1209645245 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c00e040 con 0x7fbd240c66a0 2023-10-16T11:20:34.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd20ff9700 1 -- 172.21.15.181:0/1209645245 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c00f040 con 0x7fbd240c66a0 2023-10-16T11:20:34.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd241f4080 con 0x7fbd240c66a0 2023-10-16T11:20:34.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd20ff9700 1 -- 172.21.15.181:0/1209645245 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c015430 con 0x7fbd240c66a0 2023-10-16T11:20:34.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.256+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd241f3490 con 0x7fbd240c66a0 2023-10-16T11:20:34.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.257+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2404ebb0 con 0x7fbd240c66a0 2023-10-16T11:20:34.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.258+0000 7fbd20ff9700 1 -- 172.21.15.181:0/1209645245 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbd1c015590 con 0x7fbd240c66a0 2023-10-16T11:20:34.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.258+0000 7fbd20ff9700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbd1006b140 0x7fbd1006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.258+0000 7fbd23fff700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbd1006b140 0x7fbd1006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.258+0000 7fbd20ff9700 1 -- 172.21.15.181:0/1209645245 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fbd1c08a380 con 0x7fbd240c66a0 2023-10-16T11:20:34.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.259+0000 7fbd23fff700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbd1006b140 0x7fbd1006d600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007cf0 tx=0x7fbd14009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.263+0000 7fbd20ff9700 1 -- 172.21.15.181:0/1209645245 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbd1c054d90 con 0x7fbd240c66a0 2023-10-16T11:20:34.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.458+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd240b0a40 con 0x7fbd240c66a0 2023-10-16T11:20:34.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.459+0000 7fbd20ff9700 1 -- 172.21.15.181:0/1209645245 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+17406 (secure 0 0 0) 0x7fbd1c0583b0 con 0x7fbd240c66a0 2023-10-16T11:20:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:34.462 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":236,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:27.065025+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:34.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbd1006b140 msgr2=0x7fbd1006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbd1006b140 0x7fbd1006d600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007cf0 tx=0x7fbd14009040 comp rx=0 tx=0).stop 2023-10-16T11:20:34.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd240c66a0 msgr2=0x7fbd240c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd240c66a0 0x7fbd240c6b50 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fbd2406fd20 tx=0x7fbd1c00f490 comp rx=0 tx=0).stop 2023-10-16T11:20:34.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 shutdown_connections 2023-10-16T11:20:34.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fbd1006b140 0x7fbd1006d600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd0c004850 0x7fbd2406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd240c66a0 0x7fbd240c6b50 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 --2- 172.21.15.181:0/1209645245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd240afac0 0x7fbd240aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 >> 172.21.15.181:0/1209645245 conn(0x7fbd240665a0 msgr2=0x7fbd240b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:34.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 shutdown_connections 2023-10-16T11:20:34.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.462+0000 7fbd2a66f700 1 -- 172.21.15.181:0/1209645245 wait complete. 2023-10-16T11:20:34.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:20:23.899617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '235', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:20:24.428675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:20:34.477 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:20:34.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.598+0000 7f7274dbe700 1 Processor -- start 2023-10-16T11:20:34.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.598+0000 7f7274dbe700 1 -- start start 2023-10-16T11:20:34.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f7274dbe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f727015da70 0x7f72700ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f7274dbe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72700ce6a0 0x7f727006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f7274dbe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72700cef00 con 0x7f7270072760 2023-10-16T11:20:34.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f7274dbe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7270072b40 con 0x7f72700ce6a0 2023-10-16T11:20:34.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f7274dbe700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72700cec20 con 0x7f727015da70 2023-10-16T11:20:34.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72700ce6a0 0x7f727006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7270072760 0x7f72701615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40800/0 (socket says 172.21.15.181:40800) 2023-10-16T11:20:34.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726ed9d700 1 -- 172.21.15.181:0/2824788843 learned_addr learned my addr 172.21.15.181:0/2824788843 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:34.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726e59c700 1 --2- 172.21.15.181:0/2824788843 >> v2:172.21.15.190:3300/0 conn(0x7f727015da70 0x7f72700ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726dd9b700 1 -- 172.21.15.181:0/2824788843 >> v2:172.21.15.190:3300/0 conn(0x7f727015da70 msgr2=0x7f72700ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726dd9b700 1 --2- 172.21.15.181:0/2824788843 >> v2:172.21.15.190:3300/0 conn(0x7f727015da70 0x7f72700ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726dd9b700 1 -- 172.21.15.181:0/2824788843 >> v1:172.21.15.181:6789/0 conn(0x7f7270072760 legacy=0x7f72701615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726d59a700 1 -- 172.21.15.181:0/2824788843 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058936249 0 0) 0x7f72700cef00 con 0x7f7270072760 2023-10-16T11:20:34.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.599+0000 7f726dd9b700 1 -- 172.21.15.181:0/2824788843 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270071c80 con 0x7f72700ce6a0 2023-10-16T11:20:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f726dd9b700 1 --2- 172.21.15.181:0/2824788843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72700ce6a0 0x7f727006a8a0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f72700675e0 tx=0x7f7260009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ee8ebb0c025be8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f726d59a700 1 -- 172.21.15.181:0/2824788843 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260011830 con 0x7f72700ce6a0 2023-10-16T11:20:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f726d59a700 1 -- 172.21.15.181:0/2824788843 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260011990 con 0x7f72700ce6a0 2023-10-16T11:20:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f726d59a700 1 -- 172.21.15.181:0/2824788843 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260011d20 con 0x7f72700ce6a0 2023-10-16T11:20:34.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 -- 172.21.15.181:0/2824788843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72700ce6a0 msgr2=0x7f727006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 --2- 172.21.15.181:0/2824788843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72700ce6a0 0x7f727006a8a0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f72700675e0 tx=0x7f7260009b20 comp rx=0 tx=0).stop 2023-10-16T11:20:34.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 -- 172.21.15.181:0/2824788843 shutdown_connections 2023-10-16T11:20:34.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 --2- 172.21.15.181:0/2824788843 >> v2:172.21.15.190:3300/0 conn(0x7f727015da70 0x7f72700ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 --2- 172.21.15.181:0/2824788843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72700ce6a0 0x7f727006a8a0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 -- 172.21.15.181:0/2824788843 >> 172.21.15.181:0/2824788843 conn(0x7f72700cd010 msgr2=0x7f72700cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:34.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 -- 172.21.15.181:0/2824788843 shutdown_connections 2023-10-16T11:20:34.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.600+0000 7f7274dbe700 1 -- 172.21.15.181:0/2824788843 wait complete. 2023-10-16T11:20:34.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 Processor -- start 2023-10-16T11:20:34.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 -- start start 2023-10-16T11:20:34.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7270072760 0x7f7270157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72700ce6a0 0x7f72700c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f727015da70 0x7f72700c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700c9860 con 0x7f7270072760 2023-10-16T11:20:34.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7270158200 con 0x7f72700ce6a0 2023-10-16T11:20:34.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.602+0000 7f7274dbe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72700c8be0 con 0x7f727015da70 2023-10-16T11:20:34.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7270072760 0x7f7270157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7270072760 0x7f7270157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57956/0 (socket says 172.21.15.181:57956) 2023-10-16T11:20:34.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 -- 172.21.15.181:0/3920293337 learned_addr learned my addr 172.21.15.181:0/3920293337 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:34.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726ed9d700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f727015da70 0x7f72700c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726dd9b700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72700ce6a0 0x7f72700c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 -- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f727015da70 msgr2=0x7f72700c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f727015da70 0x7f72700c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 -- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72700ce6a0 msgr2=0x7f72700c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72700ce6a0 0x7f72700c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 -- 172.21.15.181:0/3920293337 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600094a0 con 0x7f7270072760 2023-10-16T11:20:34.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726dd9b700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72700ce6a0 0x7f72700c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:34.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726ed9d700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f727015da70 0x7f72700c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:34.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f726e59c700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7270072760 0x7f7270157cc0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f726400c820 tx=0x7f726400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f725effd700 1 -- 172.21.15.181:0/3920293337 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264011800 con 0x7f7270072760 2023-10-16T11:20:34.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f725effd700 1 -- 172.21.15.181:0/3920293337 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7264011960 con 0x7f7270072760 2023-10-16T11:20:34.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72700c8ec0 con 0x7f7270072760 2023-10-16T11:20:34.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f725effd700 1 -- 172.21.15.181:0/3920293337 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264011d10 con 0x7f7270072760 2023-10-16T11:20:34.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.603+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72700c9480 con 0x7f7270072760 2023-10-16T11:20:34.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.604+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72700cc870 con 0x7f7270072760 2023-10-16T11:20:34.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.605+0000 7f725effd700 1 -- 172.21.15.181:0/3920293337 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7264018360 con 0x7f7270072760 2023-10-16T11:20:34.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.605+0000 7f725effd700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f725006b140 0x7f725006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.605+0000 7f726dd9b700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f725006b140 0x7f725006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.605+0000 7f725effd700 1 -- 172.21.15.181:0/3920293337 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f7264089d60 con 0x7f7270072760 2023-10-16T11:20:34.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.606+0000 7f726dd9b700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f725006b140 0x7f725006d600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f72600090b0 tx=0x7f7260008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.610+0000 7f725effd700 1 -- 172.21.15.181:0/3920293337 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7264054770 con 0x7f7270072760 2023-10-16T11:20:34.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.802+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f727006f6c0 con 0x7f7270072760 2023-10-16T11:20:34.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.803+0000 7f725effd700 1 -- 172.21.15.181:0/3920293337 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v236) v1 ==== 93+0+10 (secure 0 0 0) 0x7f726401f0a0 con 0x7f7270072760 2023-10-16T11:20:34.805 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:20:34.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f725006b140 msgr2=0x7f725006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f725006b140 0x7f725006d600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f72600090b0 tx=0x7f7260008030 comp rx=0 tx=0).stop 2023-10-16T11:20:34.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7270072760 msgr2=0x7f7270157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7270072760 0x7f7270157cc0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f726400c820 tx=0x7f726400cbe0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 shutdown_connections 2023-10-16T11:20:34.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f725006b140 0x7f725006d600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f727015da70 0x7f72700c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7270072760 0x7f7270157cc0 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 --2- 172.21.15.181:0/3920293337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72700ce6a0 0x7f72700c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 >> 172.21.15.181:0/3920293337 conn(0x7f72700cd010 msgr2=0x7f72700cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:34.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 shutdown_connections 2023-10-16T11:20:34.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.805+0000 7f7274dbe700 1 -- 172.21.15.181:0/3920293337 wait complete. 2023-10-16T11:20:34.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:20:34.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.939+0000 7f573a4a1700 1 Processor -- start 2023-10-16T11:20:34.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.939+0000 7f573a4a1700 1 -- start start 2023-10-16T11:20:34.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.940+0000 7f573a4a1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 0x7f57340ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.940+0000 7f573a4a1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f573406afb0 0x7f57341615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.940+0000 7f573a4a1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57340cef00 con 0x7f57340ce6a0 2023-10-16T11:20:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.940+0000 7f573a4a1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57340cec20 con 0x7f573406afb0 2023-10-16T11:20:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.940+0000 7f573a4a1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5734071c80 con 0x7f573415da70 2023-10-16T11:20:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.940+0000 7f57337fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f57340ce6a0 0x7f573406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40810/0 (socket says 172.21.15.181:40810) 2023-10-16T11:20:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.940+0000 7f57337fe700 1 -- 172.21.15.181:0/3831459271 learned_addr learned my addr 172.21.15.181:0/3831459271 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101931316 0 0) 0x7f57340cef00 con 0x7f57340ce6a0 2023-10-16T11:20:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f571c0035b0 con 0x7f57340ce6a0 2023-10-16T11:20:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 0x7f57340ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5738a3e700 1 --2- 172.21.15.181:0/3831459271 >> v2:172.21.15.190:3300/0 conn(0x7f573406afb0 0x7f57341615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2099663507 0 0) 0x7f571c0035b0 con 0x7f57340ce6a0 2023-10-16T11:20:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 msgr2=0x7f57340ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 0x7f57340ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 >> v2:172.21.15.190:3300/0 conn(0x7f573406afb0 msgr2=0x7f57341615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 --2- 172.21.15.181:0/3831459271 >> v2:172.21.15.190:3300/0 conn(0x7f573406afb0 0x7f57341615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5734071f60 con 0x7f57340ce6a0 2023-10-16T11:20:34.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f5724001670 con 0x7f57340ce6a0 2023-10-16T11:20:34.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 >> v1:172.21.15.181:6789/0 conn(0x7f57340ce6a0 legacy=0x7f573406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f571c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f571c004850 0x7f571c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 0x7f571c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57340cec20 con 0x7f571c003a00 2023-10-16T11:20:34.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5734071c80 con 0x7f571c004850 2023-10-16T11:20:34.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c00bb40 con 0x7f573415da70 2023-10-16T11:20:34.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f571c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f57337fe700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f571c004850 0x7f571c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5738a3e700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 0x7f571c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 -- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 msgr2=0x7f571c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f573415da70 0x7f571c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 -- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f571c004850 msgr2=0x7f571c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:34.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f571c004850 0x7f571c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 -- 172.21.15.181:0/3831459271 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0035b0 con 0x7f571c003a00 2023-10-16T11:20:34.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.941+0000 7f5733fff700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f571c003e20 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f5728009b60 tx=0x7f571c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2077c66aad51c85 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572800e6c0 con 0x7f571c003a00 2023-10-16T11:20:34.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572800e820 con 0x7f571c003a00 2023-10-16T11:20:34.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f5732ffd700 1 -- 172.21.15.181:0/3831459271 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572800eb90 con 0x7f571c003a00 2023-10-16T11:20:34.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f573a4a1700 1 -- 172.21.15.181:0/3831459271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 msgr2=0x7f571c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f573a4a1700 1 --2- 172.21.15.181:0/3831459271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f571c003e20 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f5728009b60 tx=0x7f571c005880 comp rx=0 tx=0).stop 2023-10-16T11:20:34.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f5738a3e700 1 -- 172.21.15.181:0/3831459271 reap_dead start 2023-10-16T11:20:34.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f573a4a1700 1 -- 172.21.15.181:0/3831459271 shutdown_connections 2023-10-16T11:20:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f573a4a1700 1 -- 172.21.15.181:0/3831459271 >> 172.21.15.181:0/3831459271 conn(0x7f57340cd010 msgr2=0x7f57340cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f573a4a1700 1 -- 172.21.15.181:0/3831459271 shutdown_connections 2023-10-16T11:20:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.942+0000 7f573a4a1700 1 -- 172.21.15.181:0/3831459271 wait complete. 2023-10-16T11:20:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.943+0000 7f573a4a1700 1 Processor -- start 2023-10-16T11:20:34.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f573a4a1700 1 -- start start 2023-10-16T11:20:34.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f573a4a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f57340b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f573a4a1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57340c4340 0x7f57340b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f573a4a1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57340c9460 0x7f57340c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f573a4a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57340b6310 con 0x7f571c003a00 2023-10-16T11:20:34.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f573a4a1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57340c8fd0 con 0x7f57340c4340 2023-10-16T11:20:34.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f573a4a1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57340c47c0 con 0x7f57340c9460 2023-10-16T11:20:34.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f5733fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f57340b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f5733fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f57340b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57976/0 (socket says 172.21.15.181:57976) 2023-10-16T11:20:34.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f5733fff700 1 -- 172.21.15.181:0/2926220533 learned_addr learned my addr 172.21.15.181:0/2926220533 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:34.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f57337fe700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57340c4340 0x7f57340b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.944+0000 7f5738a3e700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57340c9460 0x7f57340c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5733fff700 1 -- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57340c9460 msgr2=0x7f57340c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5733fff700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57340c9460 0x7f57340c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5733fff700 1 -- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57340c4340 msgr2=0x7f57340b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:34.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5733fff700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57340c4340 0x7f57340b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:34.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5733fff700 1 -- 172.21.15.181:0/2926220533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57340b6030 con 0x7f571c003a00 2023-10-16T11:20:34.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5738a3e700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57340c9460 0x7f57340c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:34.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5733fff700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f57340b34f0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f57280042f0 tx=0x7f57280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5730ff9700 1 -- 172.21.15.181:0/2926220533 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572800e6c0 con 0x7f571c003a00 2023-10-16T11:20:34.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5730ff9700 1 -- 172.21.15.181:0/2926220533 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728003ad0 con 0x7f571c003a00 2023-10-16T11:20:34.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57340c8980 con 0x7f571c003a00 2023-10-16T11:20:34.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f5730ff9700 1 -- 172.21.15.181:0/2926220533 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728003e80 con 0x7f571c003a00 2023-10-16T11:20:34.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.945+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57340b5d30 con 0x7f571c003a00 2023-10-16T11:20:34.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.946+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5734162550 con 0x7f571c003a00 2023-10-16T11:20:34.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.946+0000 7f5730ff9700 1 -- 172.21.15.181:0/2926220533 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5728013470 con 0x7f571c003a00 2023-10-16T11:20:34.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.947+0000 7f5730ff9700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f572006b140 0x7f572006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:34.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.947+0000 7f5730ff9700 1 -- 172.21.15.181:0/2926220533 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f572808b5a0 con 0x7f571c003a00 2023-10-16T11:20:34.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.947+0000 7f57337fe700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f572006b140 0x7f572006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:34.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.947+0000 7f57337fe700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f572006b140 0x7f572006d600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f57340cd440 tx=0x7f5724009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:34.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:34.951+0000 7f5730ff9700 1 -- 172.21.15.181:0/2926220533 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5728056270 con 0x7f571c003a00 2023-10-16T11:20:35.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.141+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57340c87c0 con 0x7f571c003a00 2023-10-16T11:20:35.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.141+0000 7f5730ff9700 1 -- 172.21.15.181:0/2926220533 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v236) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5728059890 con 0x7f571c003a00 2023-10-16T11:20:35.144 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:35.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f572006b140 msgr2=0x7f572006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f572006b140 0x7f572006d600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f57340cd440 tx=0x7f5724009040 comp rx=0 tx=0).stop 2023-10-16T11:20:35.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 msgr2=0x7f57340b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f57340b34f0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f57280042f0 tx=0x7f57280047c0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 shutdown_connections 2023-10-16T11:20:35.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f572006b140 0x7f572006d600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57340c9460 0x7f57340c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f571c003a00 0x7f57340b34f0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 --2- 172.21.15.181:0/2926220533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57340c4340 0x7f57340b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 >> 172.21.15.181:0/2926220533 conn(0x7f57340cd010 msgr2=0x7f573406f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:35.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 shutdown_connections 2023-10-16T11:20:35.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.143+0000 7f573a4a1700 1 -- 172.21.15.181:0/2926220533 wait complete. 2023-10-16T11:20:35.157 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:20:35.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.279+0000 7f9aed89d700 1 Processor -- start 2023-10-16T11:20:35.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.279+0000 7f9aed89d700 1 -- start start 2023-10-16T11:20:35.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.279+0000 7f9aed89d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae8159e50 0x7f9ae8071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.279+0000 7f9aed89d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9ae815db30 0x7f9ae80cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9aed89d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ae80cbed0 0x7f9ae80cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9aed89d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae80cb390 con 0x7f9ae80cbed0 2023-10-16T11:20:35.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9aed89d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae815dfb0 con 0x7f9ae815db30 2023-10-16T11:20:35.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9aed89d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80cc880 con 0x7f9ae8159e50 2023-10-16T11:20:35.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ae80cbed0 0x7f9ae80cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ae80cbed0 0x7f9ae80cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57988/0 (socket says 172.21.15.181:57988) 2023-10-16T11:20:35.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2729323320 learned_addr learned my addr 172.21.15.181:0/2729323320 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:35.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae6ffd700 1 --2- 172.21.15.181:0/2729323320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae8159e50 0x7f9ae8071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae67fc700 1 --2- 172.21.15.181:0/2729323320 >> v2:172.21.15.190:3300/0 conn(0x7f9ae815db30 0x7f9ae80cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2729323320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae8159e50 msgr2=0x7f9ae8071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:35.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 --2- 172.21.15.181:0/2729323320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae8159e50 0x7f9ae8071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2729323320 >> v2:172.21.15.190:3300/0 conn(0x7f9ae815db30 msgr2=0x7f9ae80cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:35.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 --2- 172.21.15.181:0/2729323320 >> v2:172.21.15.190:3300/0 conn(0x7f9ae815db30 0x7f9ae80cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2729323320 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae815ea60 con 0x7f9ae80cbed0 2023-10-16T11:20:35.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae77fe700 1 --2- 172.21.15.181:0/2729323320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae80cbed0 0x7f9ae80cc340 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80b20b0 tx=0x7f9ae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb7ae205535c6b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae5ffb700 1 -- 172.21.15.181:0/2729323320 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000e070 con 0x7f9ae80cbed0 2023-10-16T11:20:35.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae5ffb700 1 -- 172.21.15.181:0/2729323320 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae000b9c0 con 0x7f9ae80cbed0 2023-10-16T11:20:35.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.280+0000 7f9ae5ffb700 1 -- 172.21.15.181:0/2729323320 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000bd70 con 0x7f9ae80cbed0 2023-10-16T11:20:35.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 -- 172.21.15.181:0/2729323320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae80cbed0 msgr2=0x7f9ae80cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2729323320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae80cbed0 0x7f9ae80cc340 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80b20b0 tx=0x7f9ae000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:35.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 -- 172.21.15.181:0/2729323320 shutdown_connections 2023-10-16T11:20:35.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2729323320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae80cbed0 0x7f9ae80cc340 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2729323320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae8159e50 0x7f9ae8071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2729323320 >> v2:172.21.15.190:3300/0 conn(0x7f9ae815db30 0x7f9ae80cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 -- 172.21.15.181:0/2729323320 >> 172.21.15.181:0/2729323320 conn(0x7f9ae806aed0 msgr2=0x7f9ae806cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:35.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 -- 172.21.15.181:0/2729323320 shutdown_connections 2023-10-16T11:20:35.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.281+0000 7f9aed89d700 1 -- 172.21.15.181:0/2729323320 wait complete. 2023-10-16T11:20:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.282+0000 7f9aed89d700 1 Processor -- start 2023-10-16T11:20:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.282+0000 7f9aed89d700 1 -- start start 2023-10-16T11:20:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9aed89d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae80cbed0 0x7f9ae80c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9aed89d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ae8159e50 0x7f9ae81504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9aed89d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae815db30 0x7f9ae815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9aed89d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80c2830 con 0x7f9ae815db30 2023-10-16T11:20:35.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9aed89d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8151a30 con 0x7f9ae8159e50 2023-10-16T11:20:35.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9aed89d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8151ec0 con 0x7f9ae80cbed0 2023-10-16T11:20:35.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae815db30 0x7f9ae815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae815db30 0x7f9ae815a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57994/0 (socket says 172.21.15.181:57994) 2023-10-16T11:20:35.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2253538941 learned_addr learned my addr 172.21.15.181:0/2253538941 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:35.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2253538941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae80cbed0 msgr2=0x7f9ae80c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae80cbed0 0x7f9ae80c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2253538941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ae8159e50 msgr2=0x7f9ae81504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ae8159e50 0x7f9ae81504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 -- 172.21.15.181:0/2253538941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0009b30 con 0x7f9ae815db30 2023-10-16T11:20:35.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.283+0000 7f9ae77fe700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae815db30 0x7f9ae815a230 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80b20e0 tx=0x7f9ae0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.284+0000 7f9acffff700 1 -- 172.21.15.181:0/2253538941 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0004250 con 0x7f9ae815db30 2023-10-16T11:20:35.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.284+0000 7f9acffff700 1 -- 172.21.15.181:0/2253538941 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae00043d0 con 0x7f9ae815db30 2023-10-16T11:20:35.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.284+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae8150e20 con 0x7f9ae815db30 2023-10-16T11:20:35.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.284+0000 7f9acffff700 1 -- 172.21.15.181:0/2253538941 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0004780 con 0x7f9ae815db30 2023-10-16T11:20:35.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.284+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae81513e0 con 0x7f9ae815db30 2023-10-16T11:20:35.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.284+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae804ebb0 con 0x7f9ae815db30 2023-10-16T11:20:35.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.285+0000 7f9acffff700 1 -- 172.21.15.181:0/2253538941 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9ae001c440 con 0x7f9ae815db30 2023-10-16T11:20:35.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.285+0000 7f9acffff700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ad406b140 0x7f9ad406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.285+0000 7f9acffff700 1 -- 172.21.15.181:0/2253538941 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9ae008b290 con 0x7f9ae815db30 2023-10-16T11:20:35.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.285+0000 7f9ae6ffd700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ad406b140 0x7f9ad406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.286+0000 7f9ae6ffd700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ad406b140 0x7f9ad406d600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800a990 tx=0x7f9ad8008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.290+0000 7f9acffff700 1 -- 172.21.15.181:0/2253538941 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9ae0055d30 con 0x7f9ae815db30 2023-10-16T11:20:35.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.480+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ae8151b70 con 0x7f9ae815db30 2023-10-16T11:20:35.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.481+0000 7f9acffff700 1 -- 172.21.15.181:0/2253538941 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v236) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ae0020020 con 0x7f9ae815db30 2023-10-16T11:20:35.483 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:35.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ad406b140 msgr2=0x7f9ad406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ad406b140 0x7f9ad406d600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800a990 tx=0x7f9ad8008040 comp rx=0 tx=0).stop 2023-10-16T11:20:35.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae815db30 msgr2=0x7f9ae815a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae815db30 0x7f9ae815a230 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80b20e0 tx=0x7f9ae0004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 shutdown_connections 2023-10-16T11:20:35.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9ad406b140 0x7f9ad406d600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ae80cbed0 0x7f9ae80c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ae815db30 0x7f9ae815a230 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 --2- 172.21.15.181:0/2253538941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ae8159e50 0x7f9ae81504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 >> 172.21.15.181:0/2253538941 conn(0x7f9ae806aed0 msgr2=0x7f9ae8157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:35.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 shutdown_connections 2023-10-16T11:20:35.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.483+0000 7f9aed89d700 1 -- 172.21.15.181:0/2253538941 wait complete. 2023-10-16T11:20:35.497 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:20:35.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.616+0000 7f05064eb700 1 Processor -- start 2023-10-16T11:20:35.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.616+0000 7f05064eb700 1 -- start start 2023-10-16T11:20:35.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f05064eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050015dfd0 0x7f05000af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f05064eb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f05000af450 0x7f050006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f05064eb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f050006f870 con 0x7f050006db80 2023-10-16T11:20:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f05064eb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05000d1160 con 0x7f05000af450 2023-10-16T11:20:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f05064eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05000cc8e0 con 0x7f050015dfd0 2023-10-16T11:20:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04ff7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f05000af450 0x7f050006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f0504a88700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f050006db80 0x7f05000cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40848/0 (socket says 172.21.15.181:40848) 2023-10-16T11:20:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04ff7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f05000af450 0x7f050006d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57998/0 (socket says 172.21.15.181:57998) 2023-10-16T11:20:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f0504a88700 1 -- 172.21.15.181:0/1823230688 learned_addr learned my addr 172.21.15.181:0/1823230688 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04fffff700 1 --2- 172.21.15.181:0/1823230688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050015dfd0 0x7f05000af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04ff7fe700 1 -- 172.21.15.181:0/1823230688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050015dfd0 msgr2=0x7f05000af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04ff7fe700 1 --2- 172.21.15.181:0/1823230688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050015dfd0 0x7f05000af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04ff7fe700 1 -- 172.21.15.181:0/1823230688 >> v1:172.21.15.181:6789/0 conn(0x7f050006db80 legacy=0x7f05000cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04feffd700 1 -- 172.21.15.181:0/1823230688 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789614865 0 0) 0x7f050006f870 con 0x7f050006db80 2023-10-16T11:20:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04ff7fe700 1 -- 172.21.15.181:0/1823230688 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05000ccbc0 con 0x7f05000af450 2023-10-16T11:20:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.617+0000 7f04fffff700 1 --2- 172.21.15.181:0/1823230688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050015dfd0 0x7f05000af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f04ff7fe700 1 --2- 172.21.15.181:0/1823230688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05000af450 0x7f050006d640 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009840 tx=0x7f04f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a9b72d8fce98f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f04feffd700 1 -- 172.21.15.181:0/1823230688 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4011830 con 0x7f05000af450 2023-10-16T11:20:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f04feffd700 1 -- 172.21.15.181:0/1823230688 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f4011990 con 0x7f05000af450 2023-10-16T11:20:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f04feffd700 1 -- 172.21.15.181:0/1823230688 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4011d20 con 0x7f05000af450 2023-10-16T11:20:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 -- 172.21.15.181:0/1823230688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05000af450 msgr2=0x7f050006d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 --2- 172.21.15.181:0/1823230688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05000af450 0x7f050006d640 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009840 tx=0x7f04f4009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 -- 172.21.15.181:0/1823230688 shutdown_connections 2023-10-16T11:20:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 --2- 172.21.15.181:0/1823230688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f050015dfd0 0x7f05000af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 --2- 172.21.15.181:0/1823230688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05000af450 0x7f050006d640 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 -- 172.21.15.181:0/1823230688 >> 172.21.15.181:0/1823230688 conn(0x7f05000d04a0 msgr2=0x7f05000d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 -- 172.21.15.181:0/1823230688 shutdown_connections 2023-10-16T11:20:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.618+0000 7f05064eb700 1 -- 172.21.15.181:0/1823230688 wait complete. 2023-10-16T11:20:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.619+0000 7f05064eb700 1 Processor -- start 2023-10-16T11:20:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.620+0000 7f05064eb700 1 -- start start 2023-10-16T11:20:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.620+0000 7f05064eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050006db80 0x7f0500160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.620+0000 7f05064eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05000af450 0x7f05000b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.620+0000 7f05064eb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050015dfd0 0x7f050015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.620+0000 7f05064eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05001552c0 con 0x7f050006db80 2023-10-16T11:20:35.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.620+0000 7f05064eb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05000c4460 con 0x7f050015dfd0 2023-10-16T11:20:35.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.620+0000 7f05064eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05000c4740 con 0x7f05000af450 2023-10-16T11:20:35.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050006db80 0x7f0500160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050006db80 0x7f0500160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58004/0 (socket says 172.21.15.181:58004) 2023-10-16T11:20:35.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 -- 172.21.15.181:0/4146579314 learned_addr learned my addr 172.21.15.181:0/4146579314 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:35.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04ff7fe700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05000af450 0x7f05000b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f0504a88700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050015dfd0 0x7f050015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 -- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05000af450 msgr2=0x7f05000b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05000af450 0x7f05000b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 -- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050015dfd0 msgr2=0x7f050015e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050015dfd0 0x7f050015e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 -- 172.21.15.181:0/4146579314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40094a0 con 0x7f050006db80 2023-10-16T11:20:35.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fffff700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050006db80 0x7f0500160f40 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f04f000c7f0 tx=0x7f04f000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fcff9700 1 -- 172.21.15.181:0/4146579314 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0005c20 con 0x7f050006db80 2023-10-16T11:20:35.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fcff9700 1 -- 172.21.15.181:0/4146579314 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f0005d80 con 0x7f050006db80 2023-10-16T11:20:35.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05000c4a20 con 0x7f050006db80 2023-10-16T11:20:35.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f04fcff9700 1 -- 172.21.15.181:0/4146579314 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0011820 con 0x7f050006db80 2023-10-16T11:20:35.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.621+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05000c4fe0 con 0x7f050006db80 2023-10-16T11:20:35.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.622+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0500067d20 con 0x7f050006db80 2023-10-16T11:20:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.623+0000 7f04fcff9700 1 -- 172.21.15.181:0/4146579314 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f04f0011980 con 0x7f050006db80 2023-10-16T11:20:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.623+0000 7f04fcff9700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f04ec06b140 0x7f04ec06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.623+0000 7f04ff7fe700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f04ec06b140 0x7f04ec06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.623+0000 7f04fcff9700 1 -- 172.21.15.181:0/4146579314 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f04f00899e0 con 0x7f050006db80 2023-10-16T11:20:35.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.623+0000 7f04ff7fe700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f04ec06b140 0x7f04ec06d600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f04f40090b0 tx=0x7f04f4008030 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.627+0000 7f04fcff9700 1 -- 172.21.15.181:0/4146579314 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f04f0054480 con 0x7f050006db80 2023-10-16T11:20:35.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.821+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f050006b790 con 0x7f050006db80 2023-10-16T11:20:35.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.822+0000 7f04fcff9700 1 -- 172.21.15.181:0/4146579314 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v236) v1 ==== 84+0+209 (secure 0 0 0) 0x7f04f001e0f0 con 0x7f050006db80 2023-10-16T11:20:35.824 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:35.824 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:20:35.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:35.825 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:20:35.825 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:20:35.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:35.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:35.826 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 30, 2023-10-16T11:20:35.826 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "cephfs_metadata" 2023-10-16T11:20:35.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:35.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:35.826 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 31, 2023-10-16T11:20:35.827 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": "cephfs_data" 2023-10-16T11:20:35.827 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:35.827 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:20:35.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f04ec06b140 msgr2=0x7f04ec06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f04ec06b140 0x7f04ec06d600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f04f40090b0 tx=0x7f04f4008030 comp rx=0 tx=0).stop 2023-10-16T11:20:35.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050006db80 msgr2=0x7f0500160f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050006db80 0x7f0500160f40 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f04f000c7f0 tx=0x7f04f000cb00 comp rx=0 tx=0).stop 2023-10-16T11:20:35.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 shutdown_connections 2023-10-16T11:20:35.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f04ec06b140 0x7f04ec06d600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f050015dfd0 0x7f050015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f050006db80 0x7f0500160f40 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 --2- 172.21.15.181:0/4146579314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05000af450 0x7f05000b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 >> 172.21.15.181:0/4146579314 conn(0x7f05000d04a0 msgr2=0x7f0500073b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:35.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 shutdown_connections 2023-10-16T11:20:35.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.824+0000 7f05064eb700 1 -- 172.21.15.181:0/4146579314 wait complete. 2023-10-16T11:20:35.838 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-16T11:20:35.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-16T11:20:35.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.962+0000 7f554cdc6700 1 Processor -- start 2023-10-16T11:20:35.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.962+0000 7f554cdc6700 1 -- start start 2023-10-16T11:20:35.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f554cdc6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55480b22a0 0x7f5548066610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f554cdc6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55480cbfd0 0x7f554806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f554cdc6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5548066bf0 con 0x7f554815bb70 2023-10-16T11:20:35.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f554cdc6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554806e240 con 0x7f55480cbfd0 2023-10-16T11:20:35.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f554cdc6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554806cf40 con 0x7f55480b22a0 2023-10-16T11:20:35.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5546d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55480cbfd0 0x7f554806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5546d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55480cbfd0 0x7f554806dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58016/0 (socket says 172.21.15.181:58016) 2023-10-16T11:20:35.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5546d9d700 1 -- 172.21.15.181:0/2450156668 learned_addr learned my addr 172.21.15.181:0/2450156668 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:35.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5545d9b700 1 --2- 172.21.15.181:0/2450156668 >> v2:172.21.15.190:3300/0 conn(0x7f55480b22a0 0x7f5548066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5546d9d700 1 -- 172.21.15.181:0/2450156668 >> v2:172.21.15.190:3300/0 conn(0x7f55480b22a0 msgr2=0x7f5548066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:35.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5546d9d700 1 --2- 172.21.15.181:0/2450156668 >> v2:172.21.15.190:3300/0 conn(0x7f55480b22a0 0x7f5548066610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5546d9d700 1 -- 172.21.15.181:0/2450156668 >> v1:172.21.15.190:6789/0 conn(0x7f554815bb70 legacy=0x7f55480cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.963+0000 7f5546d9d700 1 -- 172.21.15.181:0/2450156668 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554806d220 con 0x7f55480cbfd0 2023-10-16T11:20:35.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f5546d9d700 1 --2- 172.21.15.181:0/2450156668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55480cbfd0 0x7f554806dd00 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f5548072500 tx=0x7f554000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810194f619b140fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554559a700 1 -- 172.21.15.181:0/2450156668 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554000e070 con 0x7f55480cbfd0 2023-10-16T11:20:35.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554559a700 1 -- 172.21.15.181:0/2450156668 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554000b9c0 con 0x7f55480cbfd0 2023-10-16T11:20:35.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554559a700 1 -- 172.21.15.181:0/2450156668 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554000bd70 con 0x7f55480cbfd0 2023-10-16T11:20:35.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 -- 172.21.15.181:0/2450156668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55480cbfd0 msgr2=0x7f554806dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2450156668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55480cbfd0 0x7f554806dd00 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f5548072500 tx=0x7f554000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:35.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 -- 172.21.15.181:0/2450156668 shutdown_connections 2023-10-16T11:20:35.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2450156668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55480cbfd0 0x7f554806dd00 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2450156668 >> v2:172.21.15.190:3300/0 conn(0x7f55480b22a0 0x7f5548066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 -- 172.21.15.181:0/2450156668 >> 172.21.15.181:0/2450156668 conn(0x7f55480b1780 msgr2=0x7f55480b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:35.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 -- 172.21.15.181:0/2450156668 shutdown_connections 2023-10-16T11:20:35.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.964+0000 7f554cdc6700 1 -- 172.21.15.181:0/2450156668 wait complete. 2023-10-16T11:20:35.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 Processor -- start 2023-10-16T11:20:35.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 -- start start 2023-10-16T11:20:35.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55480b22a0 0x7f5548099b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55480cbfd0 0x7f554815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f554815bb70 0x7f554809e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554809ea50 con 0x7f554815bb70 2023-10-16T11:20:35.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554809ed30 con 0x7f55480cbfd0 2023-10-16T11:20:35.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.966+0000 7f554cdc6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554809f010 con 0x7f55480b22a0 2023-10-16T11:20:35.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f554815bb70 0x7f554809e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f554815bb70 0x7f554809e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58022/0 (socket says 172.21.15.181:58022) 2023-10-16T11:20:35.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 -- 172.21.15.181:0/2616427993 learned_addr learned my addr 172.21.15.181:0/2616427993 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:35.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f554659c700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55480b22a0 0x7f5548099b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5545d9b700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55480cbfd0 0x7f554815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 -- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55480b22a0 msgr2=0x7f5548099b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55480b22a0 0x7f5548099b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 -- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55480cbfd0 msgr2=0x7f554815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:35.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55480cbfd0 0x7f554815daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 -- 172.21.15.181:0/2616427993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5540009b30 con 0x7f554815bb70 2023-10-16T11:20:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5545d9b700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55480cbfd0 0x7f554815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5546d9d700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f554815bb70 0x7f554809e170 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f55480cfa70 tx=0x7f5540004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5536ffd700 1 -- 172.21.15.181:0/2616427993 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540004930 con 0x7f554815bb70 2023-10-16T11:20:35.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5536ffd700 1 -- 172.21.15.181:0/2616427993 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55400099b0 con 0x7f554815bb70 2023-10-16T11:20:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554809f2f0 con 0x7f554815bb70 2023-10-16T11:20:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f5536ffd700 1 -- 172.21.15.181:0/2616427993 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554001c680 con 0x7f554815bb70 2023-10-16T11:20:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.967+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554809e6b0 con 0x7f554815bb70 2023-10-16T11:20:35.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.968+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554806fd40 con 0x7f554815bb70 2023-10-16T11:20:35.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.969+0000 7f5536ffd700 1 -- 172.21.15.181:0/2616427993 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f554001c7e0 con 0x7f554815bb70 2023-10-16T11:20:35.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.969+0000 7f5536ffd700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f552806b140 0x7f552806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.969+0000 7f5536ffd700 1 -- 172.21.15.181:0/2616427993 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f554008b620 con 0x7f554815bb70 2023-10-16T11:20:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.969+0000 7f554659c700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f552806b140 0x7f552806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.969+0000 7f554659c700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f552806b140 0x7f552806d600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f55481596f0 tx=0x7f553c009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:35.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:35.973+0000 7f5536ffd700 1 -- 172.21.15.181:0/2616427993 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f55400560c0 con 0x7f554815bb70 2023-10-16T11:20:36.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:36.168+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f554809f7b0 con 0x7f554815bb70 2023-10-16T11:20:37.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.124+0000 7f5536ffd700 1 -- 172.21.15.181:0/2616427993 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v237) v1 ==== 144+0+0 (secure 0 0 0) 0x7f55400596e0 con 0x7f554815bb70 2023-10-16T11:20:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.127+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f552806b140 msgr2=0x7f552806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:37.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.127+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f552806b140 0x7f552806d600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f55481596f0 tx=0x7f553c009040 comp rx=0 tx=0).stop 2023-10-16T11:20:37.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.127+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f554815bb70 msgr2=0x7f554809e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:37.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.127+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f554815bb70 0x7f554809e170 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f55480cfa70 tx=0x7f5540004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 shutdown_connections 2023-10-16T11:20:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f552806b140 0x7f552806d600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55480b22a0 0x7f5548099b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f554815bb70 0x7f554809e170 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 --2- 172.21.15.181:0/2616427993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55480cbfd0 0x7f554815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 >> 172.21.15.181:0/2616427993 conn(0x7f55480b1780 msgr2=0x7f55480c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:37.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 shutdown_connections 2023-10-16T11:20:37.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.128+0000 7f554cdc6700 1 -- 172.21.15.181:0/2616427993 wait complete. 2023-10-16T11:20:37.135 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_metadata' created 2023-10-16T11:20:37.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:20:37.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.280+0000 7f3aaf387700 1 Processor -- start 2023-10-16T11:20:37.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.281+0000 7f3aaf387700 1 -- start start 2023-10-16T11:20:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.281+0000 7f3aaf387700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aa80cc6e0 0x7f3aa8071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.281+0000 7f3aaf387700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3aa80ce680 0x7f3aa80ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.281+0000 7f3aaf387700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80ce1f0 con 0x7f3aa80b2010 2023-10-16T11:20:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.281+0000 7f3aaf387700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa80ccb90 con 0x7f3aa80cc6e0 2023-10-16T11:20:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.281+0000 7f3aaf387700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa8072120 con 0x7f3aa80ce680 2023-10-16T11:20:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aa80cc6e0 0x7f3aa8071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:37.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aa80cc6e0 0x7f3aa8071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58028/0 (socket says 172.21.15.181:58028) 2023-10-16T11:20:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 -- 172.21.15.181:0/3595690058 learned_addr learned my addr 172.21.15.181:0/3595690058 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 -- 172.21.15.181:0/3595690058 >> v2:172.21.15.190:3300/0 conn(0x7f3aa80ce680 msgr2=0x7f3aa80ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 --2- 172.21.15.181:0/3595690058 >> v2:172.21.15.190:3300/0 conn(0x7f3aa80ce680 0x7f3aa80ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 -- 172.21.15.181:0/3595690058 >> v1:172.21.15.190:6789/0 conn(0x7f3aa80b2010 legacy=0x7f3aa80b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 -- 172.21.15.181:0/3595690058 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80ccda0 con 0x7f3aa80cc6e0 2023-10-16T11:20:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3aac922700 1 --2- 172.21.15.181:0/3595690058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80cc6e0 0x7f3aa8071be0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8071a30 tx=0x7f3aa4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8448afa96f1ca3c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3a9ffff700 1 -- 172.21.15.181:0/3595690058 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4004230 con 0x7f3aa80cc6e0 2023-10-16T11:20:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3a9ffff700 1 -- 172.21.15.181:0/3595690058 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4004390 con 0x7f3aa80cc6e0 2023-10-16T11:20:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.282+0000 7f3a9ffff700 1 -- 172.21.15.181:0/3595690058 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40036a0 con 0x7f3aa80cc6e0 2023-10-16T11:20:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 -- 172.21.15.181:0/3595690058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80cc6e0 msgr2=0x7f3aa8071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3595690058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80cc6e0 0x7f3aa8071be0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8071a30 tx=0x7f3aa4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 -- 172.21.15.181:0/3595690058 shutdown_connections 2023-10-16T11:20:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3595690058 >> v2:172.21.15.190:3300/0 conn(0x7f3aa80ce680 0x7f3aa80ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3595690058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80cc6e0 0x7f3aa8071be0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 -- 172.21.15.181:0/3595690058 >> 172.21.15.181:0/3595690058 conn(0x7f3aa80cd010 msgr2=0x7f3aa80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 -- 172.21.15.181:0/3595690058 shutdown_connections 2023-10-16T11:20:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.283+0000 7f3aaf387700 1 -- 172.21.15.181:0/3595690058 wait complete. 2023-10-16T11:20:37.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.284+0000 7f3aaf387700 1 Processor -- start 2023-10-16T11:20:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.284+0000 7f3aaf387700 1 -- start start 2023-10-16T11:20:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aaf387700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aa80b2010 0x7f3aa81efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aaf387700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa80cc6e0 0x7f3aa80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aaf387700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80ce680 0x7f3aa80c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aaf387700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80cff60 con 0x7f3aa80ce680 2023-10-16T11:20:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aaf387700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80d0380 con 0x7f3aa80cc6e0 2023-10-16T11:20:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aaf387700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80cf2e0 con 0x7f3aa80b2010 2023-10-16T11:20:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80ce680 0x7f3aa80c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:37.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80ce680 0x7f3aa80c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58036/0 (socket says 172.21.15.181:58036) 2023-10-16T11:20:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 -- 172.21.15.181:0/3171638297 learned_addr learned my addr 172.21.15.181:0/3171638297 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 -- 172.21.15.181:0/3171638297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aa80b2010 msgr2=0x7f3aa81efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aa80b2010 0x7f3aa81efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 -- 172.21.15.181:0/3171638297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa80cc6e0 msgr2=0x7f3aa80cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:37.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa80cc6e0 0x7f3aa80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 -- 172.21.15.181:0/3171638297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4009550 con 0x7f3aa80ce680 2023-10-16T11:20:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3aad924700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80ce680 0x7f3aa80c81c0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000b970 tx=0x7f3aa000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3a9dffb700 1 -- 172.21.15.181:0/3171638297 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000c9b0 con 0x7f3aa80ce680 2023-10-16T11:20:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.285+0000 7f3a9dffb700 1 -- 172.21.15.181:0/3171638297 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa000cb30 con 0x7f3aa80ce680 2023-10-16T11:20:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.286+0000 7f3a9dffb700 1 -- 172.21.15.181:0/3171638297 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0014440 con 0x7f3aa80ce680 2023-10-16T11:20:37.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.286+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa80cf5c0 con 0x7f3aa80ce680 2023-10-16T11:20:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.286+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa80cfb80 con 0x7f3aa80ce680 2023-10-16T11:20:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.287+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa80d00a0 con 0x7f3aa80ce680 2023-10-16T11:20:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.287+0000 7f3a9dffb700 1 -- 172.21.15.181:0/3171638297 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3aa001d3b0 con 0x7f3aa80ce680 2023-10-16T11:20:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.288+0000 7f3a9dffb700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a8c06b140 0x7f3a8c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.288+0000 7f3aad123700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a8c06b140 0x7f3a8c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:37.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.288+0000 7f3a9dffb700 1 -- 172.21.15.181:0/3171638297 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6634+0+0 (secure 0 0 0) 0x7f3aa0006e40 con 0x7f3aa80ce680 2023-10-16T11:20:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.288+0000 7f3aad123700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a8c06b140 0x7f3a8c06d600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a980096d0 tx=0x7f3a98006c30 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.292+0000 7f3a9dffb700 1 -- 172.21.15.181:0/3171638297 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3aa0054540 con 0x7f3aa80ce680 2023-10-16T11:20:37.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:37.486+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3aa80c6f90 con 0x7f3aa80ce680 2023-10-16T11:20:38.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.129+0000 7f3a9dffb700 1 -- 172.21.15.181:0/3171638297 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v238) v1 ==== 176+0+0 (secure 0 0 0) 0x7f3aa0019020 con 0x7f3aa80ce680 2023-10-16T11:20:38.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a8c06b140 msgr2=0x7f3a8c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:38.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a8c06b140 0x7f3a8c06d600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a980096d0 tx=0x7f3a98006c30 comp rx=0 tx=0).stop 2023-10-16T11:20:38.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80ce680 msgr2=0x7f3aa80c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:38.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80ce680 0x7f3aa80c81c0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000b970 tx=0x7f3aa000bd30 comp rx=0 tx=0).stop 2023-10-16T11:20:38.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 shutdown_connections 2023-10-16T11:20:38.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f3a8c06b140 0x7f3a8c06d600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aa80b2010 0x7f3aa81efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa80ce680 0x7f3aa80c81c0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 --2- 172.21.15.181:0/3171638297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aa80cc6e0 0x7f3aa80cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 >> 172.21.15.181:0/3171638297 conn(0x7f3aa80cd010 msgr2=0x7f3aa80b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 shutdown_connections 2023-10-16T11:20:38.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.132+0000 7f3aaf387700 1 -- 172.21.15.181:0/3171638297 wait complete. 2023-10-16T11:20:38.138 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_data' created 2023-10-16T11:20:38.157 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-16T11:20:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.280+0000 7fe24f36d700 1 Processor -- start 2023-10-16T11:20:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.280+0000 7fe24f36d700 1 -- start start 2023-10-16T11:20:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe24f36d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 0x7fe248166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe24f36d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480b1f70 con 0x7fe24815f150 2023-10-16T11:20:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe24f36d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2480b2ae0 con 0x7fe2480ce6c0 2023-10-16T11:20:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe24f36d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480cd880 con 0x7fe2481646b0 2023-10-16T11:20:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe24d109700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe24815f150 0x7fe2480cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40886/0 (socket says 172.21.15.181:40886) 2023-10-16T11:20:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe24d109700 1 -- 172.21.15.181:0/3547091528 learned_addr learned my addr 172.21.15.181:0/3547091528 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:38.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe24d90a700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 0x7fe248166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765387355 0 0) 0x7fe2480b1f70 con 0x7fe24815f150 2023-10-16T11:20:38.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2340035b0 con 0x7fe24815f150 2023-10-16T11:20:38.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266789260 0 0) 0x7fe2480b2ae0 con 0x7fe2480ce6c0 2023-10-16T11:20:38.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2480b1f70 con 0x7fe2480ce6c0 2023-10-16T11:20:38.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75047020 0 0) 0x7fe2340035b0 con 0x7fe24815f150 2023-10-16T11:20:38.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 msgr2=0x7fe248166aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:38.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 0x7fe248166aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 >> v1:172.21.15.190:6789/0 conn(0x7fe2480ce6c0 legacy=0x7fe2480cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:38.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2480b2dc0 con 0x7fe24815f150 2023-10-16T11:20:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fe238001670 con 0x7fe24815f150 2023-10-16T11:20:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 >> v1:172.21.15.181:6789/0 conn(0x7fe24815f150 legacy=0x7fe2480cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe234003bc0 0x7fe234003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 0x7fe234004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234004d90 0x7fe234005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480cd880 con 0x7fe234004d90 2023-10-16T11:20:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23400bac0 con 0x7fe234003bc0 2023-10-16T11:20:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.281+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23400bc30 con 0x7fe2481646b0 2023-10-16T11:20:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d90a700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234004d90 0x7fe234005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d109700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe234003bc0 0x7fe234003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24c908700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 0x7fe234004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d90a700 1 -- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 msgr2=0x7fe234004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:38.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d90a700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 0x7fe234004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d90a700 1 -- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe234003bc0 msgr2=0x7fe234003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:38.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d90a700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe234003bc0 0x7fe234003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d90a700 1 -- 172.21.15.181:0/3547091528 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2340035b0 con 0x7fe234004d90 2023-10-16T11:20:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d109700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe234003bc0 0x7fe234003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24c908700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2481646b0 0x7fe234004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24d90a700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234004d90 0x7fe234005210 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fe240009f90 tx=0x7fe234005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4a229807238049 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000ec30 con 0x7fe234004d90 2023-10-16T11:20:38.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24000f040 con 0x7fe234004d90 2023-10-16T11:20:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe23ffff700 1 -- 172.21.15.181:0/3547091528 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240014480 con 0x7fe234004d90 2023-10-16T11:20:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.282+0000 7fe24f36d700 1 -- 172.21.15.181:0/3547091528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234004d90 msgr2=0x7fe234005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.283+0000 7fe24f36d700 1 --2- 172.21.15.181:0/3547091528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe234004d90 0x7fe234005210 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fe240009f90 tx=0x7fe234005880 comp rx=0 tx=0).stop 2023-10-16T11:20:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.283+0000 7fe24d90a700 1 -- 172.21.15.181:0/3547091528 reap_dead start 2023-10-16T11:20:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.283+0000 7fe24f36d700 1 -- 172.21.15.181:0/3547091528 shutdown_connections 2023-10-16T11:20:38.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.283+0000 7fe24f36d700 1 -- 172.21.15.181:0/3547091528 >> 172.21.15.181:0/3547091528 conn(0x7fe24809a930 msgr2=0x7fe2480d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.283+0000 7fe24f36d700 1 -- 172.21.15.181:0/3547091528 shutdown_connections 2023-10-16T11:20:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.283+0000 7fe24f36d700 1 -- 172.21.15.181:0/3547091528 wait complete. 2023-10-16T11:20:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.284+0000 7fe24f36d700 1 Processor -- start 2023-10-16T11:20:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.284+0000 7fe24f36d700 1 -- start start 2023-10-16T11:20:38.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24f36d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2480c6d20 0x7fe2480c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24f36d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480af7f0 0x7fe2480afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24f36d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248163940 0x7fe248163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24f36d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480c7d30 con 0x7fe2480c6d20 2023-10-16T11:20:38.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24f36d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24806c620 con 0x7fe2480af7f0 2023-10-16T11:20:38.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24f36d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2480b01b0 con 0x7fe248163940 2023-10-16T11:20:38.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2480c6d20 0x7fe2480c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2480c6d20 0x7fe2480c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58052/0 (socket says 172.21.15.181:58052) 2023-10-16T11:20:38.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 -- 172.21.15.181:0/3924447430 learned_addr learned my addr 172.21.15.181:0/3924447430 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:38.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24c908700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480af7f0 0x7fe2480afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d90a700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248163940 0x7fe248163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 -- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248163940 msgr2=0x7fe248163dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:38.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248163940 0x7fe248163dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 -- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480af7f0 msgr2=0x7fe2480afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:38.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480af7f0 0x7fe2480afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:38.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 -- 172.21.15.181:0/3924447430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe248164300 con 0x7fe2480c6d20 2023-10-16T11:20:38.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24c908700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480af7f0 0x7fe2480afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:38.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24d109700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2480c6d20 0x7fe2480c5130 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fe23800e780 tx=0x7fe23800eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe23dffb700 1 -- 172.21.15.181:0/3924447430 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012590 con 0x7fe2480c6d20 2023-10-16T11:20:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe23dffb700 1 -- 172.21.15.181:0/3924447430 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2380126f0 con 0x7fe2480c6d20 2023-10-16T11:20:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe23dffb700 1 -- 172.21.15.181:0/3924447430 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012a60 con 0x7fe2480c6d20 2023-10-16T11:20:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.285+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24806ba50 con 0x7fe2480c6d20 2023-10-16T11:20:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.286+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24806bfd0 con 0x7fe2480c6d20 2023-10-16T11:20:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.286+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248067d20 con 0x7fe2480c6d20 2023-10-16T11:20:38.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.287+0000 7fe23dffb700 1 -- 172.21.15.181:0/3924447430 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe238012bc0 con 0x7fe2480c6d20 2023-10-16T11:20:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.287+0000 7fe23dffb700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe22c06b140 0x7fe22c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.287+0000 7fe23dffb700 1 -- 172.21.15.181:0/3924447430 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7019+0+0 (secure 0 0 0) 0x7fe238089ca0 con 0x7fe2480c6d20 2023-10-16T11:20:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.287+0000 7fe24c908700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe22c06b140 0x7fe22c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.288+0000 7fe24c908700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe22c06b140 0x7fe22c06d600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe248156ba0 tx=0x7fe244009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:38.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.292+0000 7fe23dffb700 1 -- 172.21.15.181:0/3924447430 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe2380545b0 con 0x7fe2480c6d20 2023-10-16T11:20:38.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:38.531+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7fe2480cbac0 con 0x7fe2480c6d20 2023-10-16T11:20:39.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.131+0000 7fe23dffb700 1 -- 172.21.15.181:0/3924447430 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 32 and data pool 33 v130) v1 ==== 183+0+0 (secure 0 0 0) 0x7fe238057bd0 con 0x7fe2480c6d20 2023-10-16T11:20:39.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.135+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe22c06b140 msgr2=0x7fe22c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.135+0000 7fe24f36d700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe22c06b140 0x7fe22c06d600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe248156ba0 tx=0x7fe244009040 comp rx=0 tx=0).stop 2023-10-16T11:20:39.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.135+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2480c6d20 msgr2=0x7fe2480c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.135+0000 7fe24f36d700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2480c6d20 0x7fe2480c5130 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fe23800e780 tx=0x7fe23800eb40 comp rx=0 tx=0).stop 2023-10-16T11:20:39.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.136+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 shutdown_connections 2023-10-16T11:20:39.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.136+0000 7fe24f36d700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe22c06b140 0x7fe22c06d600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.136+0000 7fe24f36d700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe248163940 0x7fe248163dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.136+0000 7fe24f36d700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2480c6d20 0x7fe2480c5130 secure :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fe23800e780 tx=0x7fe23800eb40 comp rx=0 tx=0).stop 2023-10-16T11:20:39.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.136+0000 7fe24f36d700 1 --2- 172.21.15.181:0/3924447430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2480af7f0 0x7fe2480afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.136+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 >> 172.21.15.181:0/3924447430 conn(0x7fe24809a930 msgr2=0x7fe2480cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:39.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.136+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 shutdown_connections 2023-10-16T11:20:39.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.137+0000 7fe24f36d700 1 -- 172.21.15.181:0/3924447430 wait complete. 2023-10-16T11:20:39.147 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 32 and data pool 33 2023-10-16T11:20:39.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.298+0000 7f15c6bf3700 1 Processor -- start 2023-10-16T11:20:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.298+0000 7f15c6bf3700 1 -- start start 2023-10-16T11:20:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15c6bf3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15c00d0370 0x7f15c00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15c6bf3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15c6bf3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00cc510 con 0x7f15c015b990 2023-10-16T11:20:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15c6bf3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00d24f0 con 0x7f15c00d0370 2023-10-16T11:20:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15c6bf3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00cc7f0 con 0x7f15c00cd2d0 2023-10-16T11:20:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15bffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15c00d0370 0x7f15c00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15bffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15c00d0370 0x7f15c00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58062/0 (socket says 172.21.15.181:58062) 2023-10-16T11:20:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.299+0000 7f15bffff700 1 -- 172.21.15.181:0/528466631 learned_addr learned my addr 172.21.15.181:0/528466631 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15c5190700 1 --2- 172.21.15.181:0/528466631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bffff700 1 -- 172.21.15.181:0/528466631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 msgr2=0x7f15c00d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bffff700 1 --2- 172.21.15.181:0/528466631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c00d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bffff700 1 -- 172.21.15.181:0/528466631 >> v1:172.21.15.190:6789/0 conn(0x7f15c015b990 legacy=0x7f15c00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bffff700 1 -- 172.21.15.181:0/528466631 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00ccad0 con 0x7f15c00d0370 2023-10-16T11:20:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15c5190700 1 --2- 172.21.15.181:0/528466631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c00d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bffff700 1 --2- 172.21.15.181:0/528466631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c00d0370 0x7f15c00ce8f0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0098f0 tx=0x7f15ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efce6bf8f2ef6918 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bf7fe700 1 -- 172.21.15.181:0/528466631 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00b8a0 con 0x7f15c00d0370 2023-10-16T11:20:39.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bf7fe700 1 -- 172.21.15.181:0/528466631 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac00ba00 con 0x7f15c00d0370 2023-10-16T11:20:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15bf7fe700 1 -- 172.21.15.181:0/528466631 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00bd90 con 0x7f15c00d0370 2023-10-16T11:20:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15c6bf3700 1 -- 172.21.15.181:0/528466631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c00d0370 msgr2=0x7f15c00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.300+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/528466631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c00d0370 0x7f15c00ce8f0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0098f0 tx=0x7f15ac009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.301+0000 7f15c6bf3700 1 -- 172.21.15.181:0/528466631 shutdown_connections 2023-10-16T11:20:39.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.301+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/528466631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c00d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.301+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/528466631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c00d0370 0x7f15c00ce8f0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.301+0000 7f15c6bf3700 1 -- 172.21.15.181:0/528466631 >> 172.21.15.181:0/528466631 conn(0x7f15c009a9a0 msgr2=0x7f15c00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.301+0000 7f15c6bf3700 1 -- 172.21.15.181:0/528466631 shutdown_connections 2023-10-16T11:20:39.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.301+0000 7f15c6bf3700 1 -- 172.21.15.181:0/528466631 wait complete. 2023-10-16T11:20:39.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.302+0000 7f15c6bf3700 1 Processor -- start 2023-10-16T11:20:39.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.302+0000 7f15c6bf3700 1 -- start start 2023-10-16T11:20:39.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c00d0370 0x7f15c00c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c015b990 0x7f15c00c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00b6b80 con 0x7f15c015b990 2023-10-16T11:20:39.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00b5f70 con 0x7f15c00d0370 2023-10-16T11:20:39.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00b6250 con 0x7f15c00cd2d0 2023-10-16T11:20:39.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c015b990 0x7f15c00c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c015b990 0x7f15c00c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58070/0 (socket says 172.21.15.181:58070) 2023-10-16T11:20:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 -- 172.21.15.181:0/350489840 learned_addr learned my addr 172.21.15.181:0/350489840 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c498f700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15bffff700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c00d0370 0x7f15c00c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 -- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 msgr2=0x7f15c0151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c0151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 -- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c00d0370 msgr2=0x7f15c00c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c00d0370 0x7f15c00c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 -- 172.21.15.181:0/350489840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac009550 con 0x7f15c015b990 2023-10-16T11:20:39.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c498f700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c0151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:39.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15bffff700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c00d0370 0x7f15c00c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:39.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c5190700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c015b990 0x7f15c00c8c90 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f15b8007e70 tx=0x7f15b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:39.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15bd7fa700 1 -- 172.21.15.181:0/350489840 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800de40 con 0x7f15c015b990 2023-10-16T11:20:39.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15bd7fa700 1 -- 172.21.15.181:0/350489840 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b800f040 con 0x7f15c015b990 2023-10-16T11:20:39.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c00b6530 con 0x7f15c015b990 2023-10-16T11:20:39.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15bd7fa700 1 -- 172.21.15.181:0/350489840 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8003c60 con 0x7f15c015b990 2023-10-16T11:20:39.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.303+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c01520f0 con 0x7f15c015b990 2023-10-16T11:20:39.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.304+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c00b6d30 con 0x7f15c015b990 2023-10-16T11:20:39.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.305+0000 7f15bd7fa700 1 -- 172.21.15.181:0/350489840 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f15b8004030 con 0x7f15c015b990 2023-10-16T11:20:39.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.305+0000 7f15bd7fa700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f15a406b140 0x7f15a406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.305+0000 7f15bd7fa700 1 -- 172.21.15.181:0/350489840 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f15b808a1f0 con 0x7f15c015b990 2023-10-16T11:20:39.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.305+0000 7f15c498f700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f15a406b140 0x7f15a406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.306+0000 7f15c498f700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f15a406b140 0x7f15a406d600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15c00d2060 tx=0x7f15b4009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:39.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.310+0000 7f15bd7fa700 1 -- 172.21.15.181:0/350489840 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f15c00b6d30 con 0x7f15c015b990 2023-10-16T11:20:39.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.504+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15c00b6d30 con 0x7f15c015b990 2023-10-16T11:20:39.508 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.506+0000 7f15bd7fa700 1 -- 172.21.15.181:0/350489840 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+21087 (secure 0 0 0) 0x7f15c00b6d30 con 0x7f15c015b990 2023-10-16T11:20:39.508 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:39.509 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":239,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:38.533078+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:39.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f15a406b140 msgr2=0x7f15a406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f15a406b140 0x7f15a406d600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15c00d2060 tx=0x7f15b4009040 comp rx=0 tx=0).stop 2023-10-16T11:20:39.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c015b990 msgr2=0x7f15c00c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.510 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c015b990 0x7f15c00c8c90 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f15b8007e70 tx=0x7f15b800d3b0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 shutdown_connections 2023-10-16T11:20:39.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f15a406b140 0x7f15a406d600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15c00cd2d0 0x7f15c0151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15c015b990 0x7f15c00c8c90 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 --2- 172.21.15.181:0/350489840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c00d0370 0x7f15c00c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 >> 172.21.15.181:0/350489840 conn(0x7f15c009a9a0 msgr2=0x7f15c00cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:39.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 shutdown_connections 2023-10-16T11:20:39.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.508+0000 7f15c6bf3700 1 -- 172.21.15.181:0/350489840 wait complete. 2023-10-16T11:20:39.522 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:39.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.640+0000 7fe1d5441700 1 Processor -- start 2023-10-16T11:20:39.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.640+0000 7fe1d5441700 1 -- start start 2023-10-16T11:20:39.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.641+0000 7fe1d5441700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe1d015e6d0 0x7fe1d00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.641+0000 7fe1d5441700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d006ed00 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.641+0000 7fe1d5441700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d00d02d0 con 0x7fe1d015eab0 2023-10-16T11:20:39.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.641+0000 7fe1d5441700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d006e0f0 con 0x7fe1d015e6d0 2023-10-16T11:20:39.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.641+0000 7fe1cf7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe1d00cb0c0 0x7fe1d015a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40924/0 (socket says 172.21.15.181:40924) 2023-10-16T11:20:39.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.641+0000 7fe1cf7fe700 1 -- 172.21.15.181:0/566175638 learned_addr learned my addr 172.21.15.181:0/566175638 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:39.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.641+0000 7fe1ceffd700 1 --2- 172.21.15.181:0/566175638 >> v2:172.21.15.190:3300/0 conn(0x7fe1d015e6d0 0x7fe1d00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1cdffb700 1 -- 172.21.15.181:0/566175638 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167481619 0 0) 0x7fe1d006ed00 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1cdffb700 1 -- 172.21.15.181:0/566175638 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b80035b0 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1cdffb700 1 -- 172.21.15.181:0/566175638 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521976848 0 0) 0x7fe1d00d02d0 con 0x7fe1d015eab0 2023-10-16T11:20:39.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1cdffb700 1 -- 172.21.15.181:0/566175638 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d006ed00 con 0x7fe1d015eab0 2023-10-16T11:20:39.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1ceffd700 1 -- 172.21.15.181:0/566175638 >> v1:172.21.15.190:6789/0 conn(0x7fe1d015eab0 legacy=0x7fe1d015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1ceffd700 1 -- 172.21.15.181:0/566175638 >> v1:172.21.15.181:6789/0 conn(0x7fe1d00cb0c0 legacy=0x7fe1d015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1ceffd700 1 -- 172.21.15.181:0/566175638 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d006e3d0 con 0x7fe1d015e6d0 2023-10-16T11:20:39.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1ceffd700 1 --2- 172.21.15.181:0/566175638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 0x7fe1d00d0fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009820 tx=0x7fe1c4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ed926496976a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:39.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1cdffb700 1 -- 172.21.15.181:0/566175638 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4005e30 con 0x7fe1d015e6d0 2023-10-16T11:20:39.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1cdffb700 1 -- 172.21.15.181:0/566175638 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c4006030 con 0x7fe1d015e6d0 2023-10-16T11:20:39.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.642+0000 7fe1cdffb700 1 -- 172.21.15.181:0/566175638 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40063c0 con 0x7fe1d015e6d0 2023-10-16T11:20:39.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.643+0000 7fe1d5441700 1 -- 172.21.15.181:0/566175638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 msgr2=0x7fe1d00d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.643+0000 7fe1d5441700 1 --2- 172.21.15.181:0/566175638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 0x7fe1d00d0fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009820 tx=0x7fe1c4009b30 comp rx=0 tx=0).stop 2023-10-16T11:20:39.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.643+0000 7fe1d5441700 1 -- 172.21.15.181:0/566175638 shutdown_connections 2023-10-16T11:20:39.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.643+0000 7fe1d5441700 1 --2- 172.21.15.181:0/566175638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 0x7fe1d00d0fc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.643+0000 7fe1d5441700 1 -- 172.21.15.181:0/566175638 >> 172.21.15.181:0/566175638 conn(0x7fe1d00d04a0 msgr2=0x7fe1d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:39.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.643+0000 7fe1d5441700 1 -- 172.21.15.181:0/566175638 shutdown_connections 2023-10-16T11:20:39.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.643+0000 7fe1d5441700 1 -- 172.21.15.181:0/566175638 wait complete. 2023-10-16T11:20:39.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.644+0000 7fe1d5441700 1 Processor -- start 2023-10-16T11:20:39.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.644+0000 7fe1d5441700 1 -- start start 2023-10-16T11:20:39.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1d5441700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00cb0c0 0x7fe1d00c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1d5441700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 0x7fe1d00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1d5441700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d015eab0 0x7fe1d00b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1d5441700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d01f7a20 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1d5441700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00b5cc0 con 0x7fe1d015e6d0 2023-10-16T11:20:39.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1d5441700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00b5fa0 con 0x7fe1d015eab0 2023-10-16T11:20:39.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00cb0c0 0x7fe1d00c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00cb0c0 0x7fe1d00c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58080/0 (socket says 172.21.15.181:58080) 2023-10-16T11:20:39.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 -- 172.21.15.181:0/1085794784 learned_addr learned my addr 172.21.15.181:0/1085794784 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:39.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1cf7fe700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d015eab0 0x7fe1d00b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ce7fc700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 0x7fe1d00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 -- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d015eab0 msgr2=0x7fe1d00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d015eab0 0x7fe1d00b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 -- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 msgr2=0x7fe1d00b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 0x7fe1d00b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 -- 172.21.15.181:0/1085794784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4009430 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1ceffd700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00cb0c0 0x7fe1d00c35b0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4013040 tx=0x7fe1c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:39.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.645+0000 7fe1b77fe700 1 -- 172.21.15.181:0/1085794784 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40068f0 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.646+0000 7fe1b77fe700 1 -- 172.21.15.181:0/1085794784 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c4020070 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.646+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d01f7c10 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.646+0000 7fe1b77fe700 1 -- 172.21.15.181:0/1085794784 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4025670 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.646+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d00b5940 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.647+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d01f8130 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.647+0000 7fe1b77fe700 1 -- 172.21.15.181:0/1085794784 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe1c4022030 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.647+0000 7fe1b77fe700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe1bc06b140 0x7fe1bc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.648+0000 7fe1b77fe700 1 -- 172.21.15.181:0/1085794784 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fe1c408b3f0 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.648+0000 7fe1ce7fc700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe1bc06b140 0x7fe1bc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.648+0000 7fe1ce7fc700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe1bc06b140 0x7fe1bc06d600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00b2e90 tx=0x7fe1c0009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:39.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.652+0000 7fe1b77fe700 1 -- 172.21.15.181:0/1085794784 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe1d01f8130 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.847+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1d01f8130 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.848+0000 7fe1b77fe700 1 -- 172.21.15.181:0/1085794784 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+21087 (secure 0 0 0) 0x7fe1d01f8130 con 0x7fe1d00cb0c0 2023-10-16T11:20:39.851 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:39.851 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":239,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:38.533078+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:39.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe1bc06b140 msgr2=0x7fe1bc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe1bc06b140 0x7fe1bc06d600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00b2e90 tx=0x7fe1c0009040 comp rx=0 tx=0).stop 2023-10-16T11:20:39.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00cb0c0 msgr2=0x7fe1d00c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00cb0c0 0x7fe1d00c35b0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4013040 tx=0x7fe1c4004750 comp rx=0 tx=0).stop 2023-10-16T11:20:39.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 shutdown_connections 2023-10-16T11:20:39.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fe1bc06b140 0x7fe1bc06d600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1d015eab0 0x7fe1d00b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d00cb0c0 0x7fe1d00c35b0 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 --2- 172.21.15.181:0/1085794784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1d015e6d0 0x7fe1d00b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.850+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 >> 172.21.15.181:0/1085794784 conn(0x7fe1d00d04a0 msgr2=0x7fe1d00c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.851+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 shutdown_connections 2023-10-16T11:20:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.851+0000 7fe1d5441700 1 -- 172.21.15.181:0/1085794784 wait complete. 2023-10-16T11:20:39.865 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-16T11:20:39.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf90e6700 1 Processor -- start 2023-10-16T11:20:39.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf90e6700 1 -- start start 2023-10-16T11:20:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf90e6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40ce6c0 0x7fcbf40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf90e6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf40b1f70 con 0x7fcbf415f150 2023-10-16T11:20:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf90e6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf40b2ae0 con 0x7fcbf40cc6a0 2023-10-16T11:20:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf90e6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40b2dc0 con 0x7fcbf40ce6c0 2023-10-16T11:20:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf2d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcbf415f150 0x7fcbf40cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40934/0 (socket says 172.21.15.181:40934) 2023-10-16T11:20:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf2d9d700 1 -- 172.21.15.181:0/3573150790 learned_addr learned my addr 172.21.15.181:0/3573150790 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:39.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf259c700 1 --2- 172.21.15.181:0/3573150790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40ce6c0 0x7fcbf40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf1d9b700 1 -- 172.21.15.181:0/3573150790 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190551769 0 0) 0x7fcbf40b1f70 con 0x7fcbf415f150 2023-10-16T11:20:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf1d9b700 1 -- 172.21.15.181:0/3573150790 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbdc0035b0 con 0x7fcbf415f150 2023-10-16T11:20:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf259c700 1 -- 172.21.15.181:0/3573150790 >> v1:172.21.15.190:6789/0 conn(0x7fcbf40cc6a0 legacy=0x7fcbf40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf1d9b700 1 -- 172.21.15.181:0/3573150790 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389626684 0 0) 0x7fcbf40b2ae0 con 0x7fcbf40cc6a0 2023-10-16T11:20:39.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf259c700 1 -- 172.21.15.181:0/3573150790 >> v1:172.21.15.181:6789/0 conn(0x7fcbf415f150 legacy=0x7fcbf40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf259c700 1 -- 172.21.15.181:0/3573150790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf40cf9d0 con 0x7fcbf40ce6c0 2023-10-16T11:20:39.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf259c700 1 --2- 172.21.15.181:0/3573150790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40ce6c0 0x7fcbf40cc160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17a585e88993dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:39.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf1d9b700 1 -- 172.21.15.181:0/3573150790 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8011830 con 0x7fcbf40ce6c0 2023-10-16T11:20:39.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf1d9b700 1 -- 172.21.15.181:0/3573150790 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8011990 con 0x7fcbf40ce6c0 2023-10-16T11:20:39.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.988+0000 7fcbf1d9b700 1 -- 172.21.15.181:0/3573150790 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8011d20 con 0x7fcbf40ce6c0 2023-10-16T11:20:39.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.989+0000 7fcbf90e6700 1 -- 172.21.15.181:0/3573150790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40ce6c0 msgr2=0x7fcbf40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:39.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.989+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/3573150790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40ce6c0 0x7fcbf40cc160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe8009b80 comp rx=0 tx=0).stop 2023-10-16T11:20:39.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.989+0000 7fcbf90e6700 1 -- 172.21.15.181:0/3573150790 shutdown_connections 2023-10-16T11:20:39.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.989+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/3573150790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40ce6c0 0x7fcbf40cc160 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:39.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.989+0000 7fcbf90e6700 1 -- 172.21.15.181:0/3573150790 >> 172.21.15.181:0/3573150790 conn(0x7fcbf409a930 msgr2=0x7fcbf40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.989+0000 7fcbf90e6700 1 -- 172.21.15.181:0/3573150790 shutdown_connections 2023-10-16T11:20:39.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.989+0000 7fcbf90e6700 1 -- 172.21.15.181:0/3573150790 wait complete. 2023-10-16T11:20:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.990+0000 7fcbf90e6700 1 Processor -- start 2023-10-16T11:20:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.990+0000 7fcbf90e6700 1 -- start start 2023-10-16T11:20:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf90e6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40cc6a0 0x7fcbf415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf90e6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf40ce6c0 0x7fcbf406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf90e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415f150 0x7fcbf4159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf90e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40b6f70 con 0x7fcbf415f150 2023-10-16T11:20:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf90e6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c8ec0 con 0x7fcbf40ce6c0 2023-10-16T11:20:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf90e6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40b6c90 con 0x7fcbf40cc6a0 2023-10-16T11:20:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415f150 0x7fcbf4159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415f150 0x7fcbf4159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58096/0 (socket says 172.21.15.181:58096) 2023-10-16T11:20:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf359e700 1 -- 172.21.15.181:0/1374712213 learned_addr learned my addr 172.21.15.181:0/1374712213 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf2d9d700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40cc6a0 0x7fcbf415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.991+0000 7fcbf259c700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf40ce6c0 0x7fcbf406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf359e700 1 -- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40cc6a0 msgr2=0x7fcbf415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf359e700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40cc6a0 0x7fcbf415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf359e700 1 -- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf40ce6c0 msgr2=0x7fcbf406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf359e700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf40ce6c0 0x7fcbf406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf359e700 1 -- 172.21.15.181:0/1374712213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80094a0 con 0x7fcbf415f150 2023-10-16T11:20:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf359e700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415f150 0x7fcbf4159230 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7fcbec001560 tx=0x7fcbec00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbdb7fe700 1 -- 172.21.15.181:0/1374712213 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec003e30 con 0x7fcbf415f150 2023-10-16T11:20:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbdb7fe700 1 -- 172.21.15.181:0/1374712213 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec005070 con 0x7fcbf415f150 2023-10-16T11:20:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf40c8870 con 0x7fcbf415f150 2023-10-16T11:20:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbdb7fe700 1 -- 172.21.15.181:0/1374712213 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec005420 con 0x7fcbf415f150 2023-10-16T11:20:40.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.992+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf415a230 con 0x7fcbf415f150 2023-10-16T11:20:40.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.993+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf40c86d0 con 0x7fcbf415f150 2023-10-16T11:20:40.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.993+0000 7fcbdb7fe700 1 -- 172.21.15.181:0/1374712213 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcbec005580 con 0x7fcbf415f150 2023-10-16T11:20:40.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.994+0000 7fcbdb7fe700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcbe006b140 0x7fcbe006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:40.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.994+0000 7fcbdb7fe700 1 -- 172.21.15.181:0/1374712213 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fcbec0896b0 con 0x7fcbf415f150 2023-10-16T11:20:40.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.994+0000 7fcbf2d9d700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcbe006b140 0x7fcbe006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:40.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.994+0000 7fcbf2d9d700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcbe006b140 0x7fcbe006d600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4007fd0 tx=0x7fcbe4009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:40.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:39.998+0000 7fcbdb7fe700 1 -- 172.21.15.181:0/1374712213 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcbf40c86d0 con 0x7fcbf415f150 2023-10-16T11:20:40.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:40.240+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcbf40c86d0 con 0x7fcbf415f150 2023-10-16T11:20:41.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.156+0000 7fcbdb7fe700 1 -- 172.21.15.181:0/1374712213 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v132) v1 ==== 121+0+0 (secure 0 0 0) 0x7fcbf40c86d0 con 0x7fcbf415f150 2023-10-16T11:20:41.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcbe006b140 msgr2=0x7fcbe006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:41.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcbe006b140 0x7fcbe006d600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4007fd0 tx=0x7fcbe4009040 comp rx=0 tx=0).stop 2023-10-16T11:20:41.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415f150 msgr2=0x7fcbf4159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:41.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415f150 0x7fcbf4159230 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7fcbec001560 tx=0x7fcbec00cb50 comp rx=0 tx=0).stop 2023-10-16T11:20:41.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 shutdown_connections 2023-10-16T11:20:41.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fcbe006b140 0x7fcbe006d600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcbf40cc6a0 0x7fcbf415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415f150 0x7fcbf4159230 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 --2- 172.21.15.181:0/1374712213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcbf40ce6c0 0x7fcbf406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 >> 172.21.15.181:0/1374712213 conn(0x7fcbf409a930 msgr2=0x7fcbf40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:41.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 shutdown_connections 2023-10-16T11:20:41.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.159+0000 7fcbf90e6700 1 -- 172.21.15.181:0/1374712213 wait complete. 2023-10-16T11:20:41.175 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:20:41.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-16T11:20:41.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.304+0000 7f5ab9518700 1 Processor -- start 2023-10-16T11:20:41.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.304+0000 7f5ab9518700 1 -- start start 2023-10-16T11:20:41.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab9518700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5ab40d01d0 0x7f5ab40ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:41.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab9518700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5ab40d1500 0x7f5ab40d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:41.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab9518700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab40cf580 con 0x7f5ab415ea40 2023-10-16T11:20:41.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab9518700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab40d4290 con 0x7f5ab40d1500 2023-10-16T11:20:41.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab9518700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab40ce970 con 0x7f5ab40d01d0 2023-10-16T11:20:41.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5ab40d1500 0x7f5ab40d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:41.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5ab40d1500 0x7f5ab40d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58106/0 (socket says 172.21.15.181:58106) 2023-10-16T11:20:41.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 -- 172.21.15.181:0/1532632686 learned_addr learned my addr 172.21.15.181:0/1532632686 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:41.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab2ffd700 1 --2- 172.21.15.181:0/1532632686 >> v2:172.21.15.190:3300/0 conn(0x7f5ab40d01d0 0x7f5ab40ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:41.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 -- 172.21.15.181:0/1532632686 >> v2:172.21.15.190:3300/0 conn(0x7f5ab40d01d0 msgr2=0x7f5ab40ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:41.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 --2- 172.21.15.181:0/1532632686 >> v2:172.21.15.190:3300/0 conn(0x7f5ab40d01d0 0x7f5ab40ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 -- 172.21.15.181:0/1532632686 >> v1:172.21.15.190:6789/0 conn(0x7f5ab415ea40 legacy=0x7f5ab40d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:41.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 -- 172.21.15.181:0/1532632686 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40cec50 con 0x7f5ab40d1500 2023-10-16T11:20:41.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab2ffd700 1 --2- 172.21.15.181:0/1532632686 >> v2:172.21.15.190:3300/0 conn(0x7f5ab40d01d0 0x7f5ab40ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:41.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab37fe700 1 --2- 172.21.15.181:0/1532632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40d1990 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f5ab408e050 tx=0x7f5aac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9746104f149b8a8c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:41.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab1ffb700 1 -- 172.21.15.181:0/1532632686 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00e070 con 0x7f5ab40d1500 2023-10-16T11:20:41.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab1ffb700 1 -- 172.21.15.181:0/1532632686 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac003a20 con 0x7f5ab40d1500 2023-10-16T11:20:41.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.305+0000 7f5ab1ffb700 1 -- 172.21.15.181:0/1532632686 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac003dd0 con 0x7f5ab40d1500 2023-10-16T11:20:41.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 -- 172.21.15.181:0/1532632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 msgr2=0x7f5ab40d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:41.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 --2- 172.21.15.181:0/1532632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40d1990 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f5ab408e050 tx=0x7f5aac00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:41.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 -- 172.21.15.181:0/1532632686 shutdown_connections 2023-10-16T11:20:41.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 --2- 172.21.15.181:0/1532632686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40d1990 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 --2- 172.21.15.181:0/1532632686 >> v2:172.21.15.190:3300/0 conn(0x7f5ab40d01d0 0x7f5ab40ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 -- 172.21.15.181:0/1532632686 >> 172.21.15.181:0/1532632686 conn(0x7f5ab409cb60 msgr2=0x7f5ab40d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:41.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 -- 172.21.15.181:0/1532632686 shutdown_connections 2023-10-16T11:20:41.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.306+0000 7f5ab9518700 1 -- 172.21.15.181:0/1532632686 wait complete. 2023-10-16T11:20:41.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.307+0000 7f5ab9518700 1 Processor -- start 2023-10-16T11:20:41.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.307+0000 7f5ab9518700 1 -- start start 2023-10-16T11:20:41.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab9518700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ab40d01d0 0x7f5ab41f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:41.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab9518700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:41.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab9518700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ab415ea40 0x7f5ab41f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:41.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab9518700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40b1820 con 0x7f5ab40d1500 2023-10-16T11:20:41.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab9518700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40b0990 con 0x7f5ab415ea40 2023-10-16T11:20:41.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab9518700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab41f3100 con 0x7f5ab40d01d0 2023-10-16T11:20:41.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:41.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58110/0 (socket says 172.21.15.181:58110) 2023-10-16T11:20:41.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab27fc700 1 -- 172.21.15.181:0/3498403932 learned_addr learned my addr 172.21.15.181:0/3498403932 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:41.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab2ffd700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ab40d01d0 0x7f5ab41f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:41.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.308+0000 7f5ab37fe700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ab415ea40 0x7f5ab41f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:41.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab27fc700 1 -- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ab40d01d0 msgr2=0x7f5ab41f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:41.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab27fc700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ab40d01d0 0x7f5ab41f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab27fc700 1 -- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ab415ea40 msgr2=0x7f5ab41f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:41.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab27fc700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ab415ea40 0x7f5ab41f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:41.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab27fc700 1 -- 172.21.15.181:0/3498403932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac009b30 con 0x7f5ab40d1500 2023-10-16T11:20:41.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab27fc700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40b0450 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c500 tx=0x7f5aa400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:41.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5a9b7fe700 1 -- 172.21.15.181:0/3498403932 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4011790 con 0x7f5ab40d1500 2023-10-16T11:20:41.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5a9b7fe700 1 -- 172.21.15.181:0/3498403932 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa40118f0 con 0x7f5ab40d1500 2023-10-16T11:20:41.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab40b0c10 con 0x7f5ab40d1500 2023-10-16T11:20:41.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5a9b7fe700 1 -- 172.21.15.181:0/3498403932 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4011ca0 con 0x7f5ab40d1500 2023-10-16T11:20:41.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.309+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab40b11d0 con 0x7f5ab40d1500 2023-10-16T11:20:41.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.310+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab40b19d0 con 0x7f5ab40d1500 2023-10-16T11:20:41.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.310+0000 7f5a9b7fe700 1 -- 172.21.15.181:0/3498403932 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5aa401c460 con 0x7f5ab40d1500 2023-10-16T11:20:41.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.311+0000 7f5a9b7fe700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5aa006b140 0x7f5aa006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:41.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.311+0000 7f5a9b7fe700 1 -- 172.21.15.181:0/3498403932 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f5aa4089d50 con 0x7f5ab40d1500 2023-10-16T11:20:41.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.311+0000 7f5ab2ffd700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5aa006b140 0x7f5aa006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:41.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.311+0000 7f5ab2ffd700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5aa006b140 0x7f5aa006d600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006fd0 tx=0x7f5aa8008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:41.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.315+0000 7f5a9b7fe700 1 -- 172.21.15.181:0/3498403932 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5ab40b19d0 con 0x7f5ab40d1500 2023-10-16T11:20:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:41.553+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5ab40b19d0 con 0x7f5ab40d1500 2023-10-16T11:20:42.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.157+0000 7f5a9b7fe700 1 -- 172.21.15.181:0/3498403932 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v133) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5ab40b19d0 con 0x7f5ab40d1500 2023-10-16T11:20:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5aa006b140 msgr2=0x7f5aa006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5aa006b140 0x7f5aa006d600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006fd0 tx=0x7f5aa8008040 comp rx=0 tx=0).stop 2023-10-16T11:20:42.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 msgr2=0x7f5ab40b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40b0450 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c500 tx=0x7f5aa400c8c0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 shutdown_connections 2023-10-16T11:20:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f5aa006b140 0x7f5aa006d600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ab40d01d0 0x7f5ab41f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ab415ea40 0x7f5ab41f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 --2- 172.21.15.181:0/3498403932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ab40d1500 0x7f5ab40b0450 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 >> 172.21.15.181:0/3498403932 conn(0x7f5ab409cb60 msgr2=0x7f5ab415ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 shutdown_connections 2023-10-16T11:20:42.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.160+0000 7f5ab9518700 1 -- 172.21.15.181:0/3498403932 wait complete. 2023-10-16T11:20:42.176 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-16T11:20:42.177 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:42.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.299+0000 7ff5916e0700 1 Processor -- start 2023-10-16T11:20:42.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.299+0000 7ff5916e0700 1 -- start start 2023-10-16T11:20:42.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff5916e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 0x7ff58c0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff5916e0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c0d1640 con 0x7ff58c0cf690 2023-10-16T11:20:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff5916e0700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c0d1920 con 0x7ff58c15d9b0 2023-10-16T11:20:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff5916e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0cfb10 con 0x7ff58c1621c0 2023-10-16T11:20:42.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff58b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff58c0cf690 0x7ff58c0cbcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40966/0 (socket says 172.21.15.181:40966) 2023-10-16T11:20:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff58b7fe700 1 -- 172.21.15.181:0/2991328551 learned_addr learned my addr 172.21.15.181:0/2991328551 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff58a7fc700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 0x7ff58c0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953163139 0 0) 0x7ff58c0d1640 con 0x7ff58c0cf690 2023-10-16T11:20:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5740035b0 con 0x7ff58c0cf690 2023-10-16T11:20:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652374925 0 0) 0x7ff58c0d1920 con 0x7ff58c15d9b0 2023-10-16T11:20:42.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff58c0d1640 con 0x7ff58c15d9b0 2023-10-16T11:20:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 381313383 0 0) 0x7ff5740035b0 con 0x7ff58c0cf690 2023-10-16T11:20:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 msgr2=0x7ff58c0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 0x7ff58c0cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 >> v1:172.21.15.190:6789/0 conn(0x7ff58c15d9b0 legacy=0x7ff58c0b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0d1c00 con 0x7ff58c0cf690 2023-10-16T11:20:42.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff5840019f0 con 0x7ff58c0cf690 2023-10-16T11:20:42.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 >> v1:172.21.15.181:6789/0 conn(0x7ff58c0cf690 legacy=0x7ff58c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff574003bc0 0x7ff574003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 0x7ff574004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574004d90 0x7ff574005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0cfb10 con 0x7ff574003bc0 2023-10-16T11:20:42.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57400bac0 con 0x7ff574004d90 2023-10-16T11:20:42.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57400bc80 con 0x7ff58c1621c0 2023-10-16T11:20:42.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.300+0000 7ff58affd700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff574003bc0 0x7ff574003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58a7fc700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 0x7ff574004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58b7fe700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574004d90 0x7ff574005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58affd700 1 -- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 msgr2=0x7ff574004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:42.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58affd700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff58c1621c0 0x7ff574004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58affd700 1 -- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574004d90 msgr2=0x7ff574005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58affd700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574004d90 0x7ff574005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58affd700 1 -- 172.21.15.181:0/2991328551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740035b0 con 0x7ff574003bc0 2023-10-16T11:20:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58b7fe700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff574004d90 0x7ff574005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58affd700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff574003bc0 0x7ff574003fc0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7ff580001940 tx=0x7ff574005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d5c3e83f0e14e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:42.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580001430 con 0x7ff574003bc0 2023-10-16T11:20:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580001d70 con 0x7ff574003bc0 2023-10-16T11:20:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff589ffb700 1 -- 172.21.15.181:0/2991328551 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580005510 con 0x7ff574003bc0 2023-10-16T11:20:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff5916e0700 1 -- 172.21.15.181:0/2991328551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff574003bc0 msgr2=0x7ff574003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff5916e0700 1 --2- 172.21.15.181:0/2991328551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff574003bc0 0x7ff574003fc0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7ff580001940 tx=0x7ff574005880 comp rx=0 tx=0).stop 2023-10-16T11:20:42.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.301+0000 7ff58b7fe700 1 -- 172.21.15.181:0/2991328551 reap_dead start 2023-10-16T11:20:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.302+0000 7ff5916e0700 1 -- 172.21.15.181:0/2991328551 shutdown_connections 2023-10-16T11:20:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.302+0000 7ff5916e0700 1 -- 172.21.15.181:0/2991328551 >> 172.21.15.181:0/2991328551 conn(0x7ff58c0665a0 msgr2=0x7ff58c0669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.302+0000 7ff5916e0700 1 -- 172.21.15.181:0/2991328551 shutdown_connections 2023-10-16T11:20:42.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.302+0000 7ff5916e0700 1 -- 172.21.15.181:0/2991328551 wait complete. 2023-10-16T11:20:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.303+0000 7ff5916e0700 1 Processor -- start 2023-10-16T11:20:42.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.303+0000 7ff5916e0700 1 -- start start 2023-10-16T11:20:42.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5916e0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574003bc0 0x7ff58c1f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5916e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff58c0c8ed0 0x7ff58c1f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5916e0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58c06c380 0x7ff58c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5916e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c06cd70 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5916e0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1f7410 con 0x7ff58c06c380 2023-10-16T11:20:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5916e0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0ca010 con 0x7ff574003bc0 2023-10-16T11:20:42.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff58c0c8ed0 0x7ff58c1f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff58c0c8ed0 0x7ff58c1f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58130/0 (socket says 172.21.15.181:58130) 2023-10-16T11:20:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 -- 172.21.15.181:0/601981822 learned_addr learned my addr 172.21.15.181:0/601981822 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58b7fe700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58c06c380 0x7ff58c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58affd700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574003bc0 0x7ff58c1f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 -- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574003bc0 msgr2=0x7ff58c1f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574003bc0 0x7ff58c1f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 -- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58c06c380 msgr2=0x7ff58c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58c06c380 0x7ff58c06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 -- 172.21.15.181:0/601981822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0c9490 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58affd700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574003bc0 0x7ff58c1f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:42.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58b7fe700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58c06c380 0x7ff58c06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:42.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff58a7fc700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff58c0c8ed0 0x7ff58c1f2690 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7ff57c011aa0 tx=0x7ff57c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:42.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5737fe700 1 -- 172.21.15.181:0/601981822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00f980 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5737fe700 1 -- 172.21.15.181:0/601981822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c00fae0 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c0c9770 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.304+0000 7ff5737fe700 1 -- 172.21.15.181:0/601981822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00fe90 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.305+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c0c9d30 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.305+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c04ebb0 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.306+0000 7ff5737fe700 1 -- 172.21.15.181:0/601981822 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff57c014440 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.306+0000 7ff5737fe700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff57806b140 0x7ff57806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.306+0000 7ff5737fe700 1 -- 172.21.15.181:0/601981822 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7ff57c08a700 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.306+0000 7ff58affd700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff57806b140 0x7ff57806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.307+0000 7ff58affd700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff57806b140 0x7ff57806d600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff580015040 tx=0x7ff58000a000 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:42.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.311+0000 7ff5737fe700 1 -- 172.21.15.181:0/601981822 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff57c055010 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.547+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff58c0b5880 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.547+0000 7ff5737fe700 1 -- 172.21.15.181:0/601981822 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+6164 (secure 0 0 0) 0x7ff57c058630 con 0x7ff58c0c8ed0 2023-10-16T11:20:42.549 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:42.549 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:20:42.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff57806b140 msgr2=0x7ff57806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff57806b140 0x7ff57806d600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff580015040 tx=0x7ff58000a000 comp rx=0 tx=0).stop 2023-10-16T11:20:42.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff58c0c8ed0 msgr2=0x7ff58c1f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff58c0c8ed0 0x7ff58c1f2690 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7ff57c011aa0 tx=0x7ff57c011e60 comp rx=0 tx=0).stop 2023-10-16T11:20:42.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 shutdown_connections 2023-10-16T11:20:42.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7ff57806b140 0x7ff57806d600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff574003bc0 0x7ff58c1f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58c06c380 0x7ff58c06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 --2- 172.21.15.181:0/601981822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff58c0c8ed0 0x7ff58c1f2690 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 >> 172.21.15.181:0/601981822 conn(0x7ff58c0665a0 msgr2=0x7ff58c15cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:42.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 shutdown_connections 2023-10-16T11:20:42.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.550+0000 7ff5916e0700 1 -- 172.21.15.181:0/601981822 wait complete. 2023-10-16T11:20:42.556 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 133 2023-10-16T11:20:42.563 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:42.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.682+0000 7fd601d56700 1 Processor -- start 2023-10-16T11:20:42.684 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.682+0000 7fd601d56700 1 -- start start 2023-10-16T11:20:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd601d56700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5fc15e6d0 0x7fd5fc0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd601d56700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5fc0cb0c0 0x7fd5fc15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd601d56700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc06ed00 con 0x7fd5fc15eab0 2023-10-16T11:20:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd601d56700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0d02d0 con 0x7fd5fc0cb0c0 2023-10-16T11:20:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd601d56700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc15ad40 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd5fbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5fc0cb0c0 0x7fd5fc15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd5fbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5fc0cb0c0 0x7fd5fc15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58140/0 (socket says 172.21.15.181:58140) 2023-10-16T11:20:42.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd5faffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd5fc15eab0 0x7fd5fc15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40982/0 (socket says 172.21.15.181:40982) 2023-10-16T11:20:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd5fbfff700 1 -- 172.21.15.181:0/3280724883 learned_addr learned my addr 172.21.15.181:0/3280724883 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.683+0000 7fd5fb7fe700 1 --2- 172.21.15.181:0/3280724883 >> v2:172.21.15.190:3300/0 conn(0x7fd5fc15e6d0 0x7fd5fc0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fbfff700 1 -- 172.21.15.181:0/3280724883 >> v2:172.21.15.190:3300/0 conn(0x7fd5fc15e6d0 msgr2=0x7fd5fc0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fbfff700 1 --2- 172.21.15.181:0/3280724883 >> v2:172.21.15.190:3300/0 conn(0x7fd5fc15e6d0 0x7fd5fc0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fbfff700 1 -- 172.21.15.181:0/3280724883 >> v1:172.21.15.181:6789/0 conn(0x7fd5fc15eab0 legacy=0x7fd5fc15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fa7fc700 1 -- 172.21.15.181:0/3280724883 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337900117 0 0) 0x7fd5fc06ed00 con 0x7fd5fc15eab0 2023-10-16T11:20:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fbfff700 1 -- 172.21.15.181:0/3280724883 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc06e0f0 con 0x7fd5fc0cb0c0 2023-10-16T11:20:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fbfff700 1 --2- 172.21.15.181:0/3280724883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a800 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0717b0 tx=0x7fd5f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2737e90d726bf1ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fa7fc700 1 -- 172.21.15.181:0/3280724883 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400e070 con 0x7fd5fc0cb0c0 2023-10-16T11:20:42.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fa7fc700 1 -- 172.21.15.181:0/3280724883 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f400b9c0 con 0x7fd5fc0cb0c0 2023-10-16T11:20:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd5fa7fc700 1 -- 172.21.15.181:0/3280724883 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400bd70 con 0x7fd5fc0cb0c0 2023-10-16T11:20:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd601d56700 1 -- 172.21.15.181:0/3280724883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc0cb0c0 msgr2=0x7fd5fc15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd601d56700 1 --2- 172.21.15.181:0/3280724883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a800 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0717b0 tx=0x7fd5f400b330 comp rx=0 tx=0).stop 2023-10-16T11:20:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd601d56700 1 -- 172.21.15.181:0/3280724883 shutdown_connections 2023-10-16T11:20:42.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd601d56700 1 --2- 172.21.15.181:0/3280724883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a800 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd601d56700 1 --2- 172.21.15.181:0/3280724883 >> v2:172.21.15.190:3300/0 conn(0x7fd5fc15e6d0 0x7fd5fc0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.684+0000 7fd601d56700 1 -- 172.21.15.181:0/3280724883 >> 172.21.15.181:0/3280724883 conn(0x7fd5fc0d04a0 msgr2=0x7fd5fc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.685+0000 7fd601d56700 1 -- 172.21.15.181:0/3280724883 shutdown_connections 2023-10-16T11:20:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.685+0000 7fd601d56700 1 -- 172.21.15.181:0/3280724883 wait complete. 2023-10-16T11:20:42.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.686+0000 7fd601d56700 1 Processor -- start 2023-10-16T11:20:42.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.686+0000 7fd601d56700 1 -- start start 2023-10-16T11:20:42.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.686+0000 7fd601d56700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.686+0000 7fd601d56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc15e6d0 0x7fd5fc1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.686+0000 7fd601d56700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5fc15eab0 0x7fd5fc14ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd601d56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc1546b0 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd601d56700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0c3fb0 con 0x7fd5fc0cb0c0 2023-10-16T11:20:42.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd601d56700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc153e10 con 0x7fd5fc15eab0 2023-10-16T11:20:42.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc15e6d0 0x7fd5fc1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc15e6d0 0x7fd5fc1533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58142/0 (socket says 172.21.15.181:58142) 2023-10-16T11:20:42.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 -- 172.21.15.181:0/2593376664 learned_addr learned my addr 172.21.15.181:0/2593376664 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:42.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5fb7fe700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5fbfff700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5fc15eab0 0x7fd5fc14ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 -- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5fc15eab0 msgr2=0x7fd5fc14ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5fc15eab0 0x7fd5fc14ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 -- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5fc0cb0c0 msgr2=0x7fd5fc15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 -- 172.21.15.181:0/2593376664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4009b30 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5fb7fe700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:42.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5fbfff700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5fc15eab0 0x7fd5fc14ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:42.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5faffd700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc15e6d0 0x7fd5fc1533a0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0011e0 tx=0x7fd5ec00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:42.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5e3fff700 1 -- 172.21.15.181:0/2593376664 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec004c80 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5e3fff700 1 -- 172.21.15.181:0/2593376664 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec004e00 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc1540f0 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd5e3fff700 1 -- 172.21.15.181:0/2593376664 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec007310 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.687+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0c41e0 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.688+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc0c4400 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.689+0000 7fd5e3fff700 1 -- 172.21.15.181:0/2593376664 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd5ec007470 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.689+0000 7fd5e3fff700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e806b140 0x7fd5e806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:42.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.689+0000 7fd5e3fff700 1 -- 172.21.15.181:0/2593376664 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fd5ec089db0 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.689+0000 7fd5fb7fe700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e806b140 0x7fd5e806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:42.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.689+0000 7fd5fb7fe700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e806b140 0x7fd5e806d600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009820 tx=0x7fd5f0008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:42.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.694+0000 7fd5e3fff700 1 -- 172.21.15.181:0/2593376664 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd5ec054570 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.886+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5fc06e680 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.888+0000 7fd5e3fff700 1 -- 172.21.15.181:0/2593376664 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v240) v1 ==== 74+0+21074 (secure 0 0 0) 0x7fd5ec089020 con 0x7fd5fc15e6d0 2023-10-16T11:20:42.889 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:42.889 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":240,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:40.147469+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e806b140 msgr2=0x7fd5e806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e806b140 0x7fd5e806d600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009820 tx=0x7fd5f0008040 comp rx=0 tx=0).stop 2023-10-16T11:20:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc15e6d0 msgr2=0x7fd5fc1533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc15e6d0 0x7fd5fc1533a0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0011e0 tx=0x7fd5ec00cae0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 shutdown_connections 2023-10-16T11:20:42.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e806b140 0x7fd5e806d600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5fc15eab0 0x7fd5fc14ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5fc0cb0c0 0x7fd5fc15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 --2- 172.21.15.181:0/2593376664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5fc15e6d0 0x7fd5fc1533a0 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:42.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 >> 172.21.15.181:0/2593376664 conn(0x7fd5fc0d04a0 msgr2=0x7fd5fc0adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:42.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 shutdown_connections 2023-10-16T11:20:42.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:42.890+0000 7fd601d56700 1 -- 172.21.15.181:0/2593376664 wait complete. 2023-10-16T11:20:42.903 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:20:42.904 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:20:43.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.024+0000 7f65f6721700 1 Processor -- start 2023-10-16T11:20:43.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.024+0000 7f65f6721700 1 -- start start 2023-10-16T11:20:43.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65f6721700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f65f00c26b0 0x7f65f00c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65f6721700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00b2710 con 0x7f65f015e020 2023-10-16T11:20:43.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65f6721700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00c18f0 con 0x7f65f015dc40 2023-10-16T11:20:43.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65f6721700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f015ebc0 con 0x7f65f00c26b0 2023-10-16T11:20:43.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65f4cbe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f65f015e020 0x7f65f015e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40996/0 (socket says 172.21.15.181:40996) 2023-10-16T11:20:43.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65f4cbe700 1 -- 172.21.15.181:0/1679017255 learned_addr learned my addr 172.21.15.181:0/1679017255 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:43.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65ef7fe700 1 --2- 172.21.15.181:0/1679017255 >> v2:172.21.15.190:3300/0 conn(0x7f65f00c26b0 0x7f65f00c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65eeffd700 1 -- 172.21.15.181:0/1679017255 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706348276 0 0) 0x7f65e80014e0 con 0x7f65f015e020 2023-10-16T11:20:43.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.025+0000 7f65eeffd700 1 -- 172.21.15.181:0/1679017255 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d80046c0 con 0x7f65f015e020 2023-10-16T11:20:43.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65eeffd700 1 -- 172.21.15.181:0/1679017255 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1991680463 0 0) 0x7f65d80046c0 con 0x7f65f015e020 2023-10-16T11:20:43.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65ef7fe700 1 -- 172.21.15.181:0/1679017255 >> v1:172.21.15.190:6789/0 conn(0x7f65f015dc40 legacy=0x7f65f006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65ef7fe700 1 -- 172.21.15.181:0/1679017255 >> v1:172.21.15.181:6789/0 conn(0x7f65f015e020 legacy=0x7f65f015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65ef7fe700 1 -- 172.21.15.181:0/1679017255 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00c1bd0 con 0x7f65f00c26b0 2023-10-16T11:20:43.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65ef7fe700 1 --2- 172.21.15.181:0/1679017255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f00c26b0 0x7f65f00c2b10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009840 tx=0x7f65e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d354c6ca783e7f4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65eeffd700 1 -- 172.21.15.181:0/1679017255 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8001bd0 con 0x7f65f00c26b0 2023-10-16T11:20:43.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65eeffd700 1 -- 172.21.15.181:0/1679017255 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e0004030 con 0x7f65f00c26b0 2023-10-16T11:20:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65eeffd700 1 -- 172.21.15.181:0/1679017255 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e00043a0 con 0x7f65f00c26b0 2023-10-16T11:20:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65f6721700 1 -- 172.21.15.181:0/1679017255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f00c26b0 msgr2=0x7f65f00c2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65f6721700 1 --2- 172.21.15.181:0/1679017255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f00c26b0 0x7f65f00c2b10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009840 tx=0x7f65e0009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65f6721700 1 -- 172.21.15.181:0/1679017255 shutdown_connections 2023-10-16T11:20:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65f6721700 1 --2- 172.21.15.181:0/1679017255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f00c26b0 0x7f65f00c2b10 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.026+0000 7f65f6721700 1 -- 172.21.15.181:0/1679017255 >> 172.21.15.181:0/1679017255 conn(0x7f65f00b1780 msgr2=0x7f65f00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:43.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.027+0000 7f65f6721700 1 -- 172.21.15.181:0/1679017255 shutdown_connections 2023-10-16T11:20:43.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.027+0000 7f65f6721700 1 -- 172.21.15.181:0/1679017255 wait complete. 2023-10-16T11:20:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.028+0000 7f65f6721700 1 Processor -- start 2023-10-16T11:20:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.028+0000 7f65f6721700 1 -- start start 2023-10-16T11:20:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65f00c26b0 0x7f65f00c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f015dc40 0x7f65f00a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f015e020 0x7f65f00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015a4f0 con 0x7f65f015e020 2023-10-16T11:20:43.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f009ea70 con 0x7f65f015dc40 2023-10-16T11:20:43.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015a790 con 0x7f65f00c26b0 2023-10-16T11:20:43.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f015e020 0x7f65f00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f015e020 0x7f65f00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58156/0 (socket says 172.21.15.181:58156) 2023-10-16T11:20:43.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 -- 172.21.15.181:0/1435548362 learned_addr learned my addr 172.21.15.181:0/1435548362 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:43.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65effff700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65f00c26b0 0x7f65f00c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65ef7fe700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f015dc40 0x7f65f00a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 -- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65f00c26b0 msgr2=0x7f65f00c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65f00c26b0 0x7f65f00c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 -- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f015dc40 msgr2=0x7f65f00a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f015dc40 0x7f65f00a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 -- 172.21.15.181:0/1435548362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e00094a0 con 0x7f65f015e020 2023-10-16T11:20:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f4cbe700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f015e020 0x7f65f00afb90 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f65e800cc00 tx=0x7f65e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65ecff9700 1 -- 172.21.15.181:0/1435548362 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e800ab20 con 0x7f65f015e020 2023-10-16T11:20:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65ecff9700 1 -- 172.21.15.181:0/1435548362 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e800aca0 con 0x7f65f015e020 2023-10-16T11:20:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f015aa70 con 0x7f65f015e020 2023-10-16T11:20:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65ecff9700 1 -- 172.21.15.181:0/1435548362 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8007310 con 0x7f65f015e020 2023-10-16T11:20:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.029+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f009ed50 con 0x7f65f015e020 2023-10-16T11:20:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.030+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f015ac90 con 0x7f65f015e020 2023-10-16T11:20:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.031+0000 7f65ecff9700 1 -- 172.21.15.181:0/1435548362 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f65e8007470 con 0x7f65f015e020 2023-10-16T11:20:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.031+0000 7f65ecff9700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f65dc06b140 0x7f65dc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.031+0000 7f65ecff9700 1 -- 172.21.15.181:0/1435548362 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f65e8089bc0 con 0x7f65f015e020 2023-10-16T11:20:43.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.031+0000 7f65effff700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f65dc06b140 0x7f65dc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.032+0000 7f65effff700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f65dc06b140 0x7f65dc06d600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65e40016c0 tx=0x7f65e4009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.035+0000 7f65ecff9700 1 -- 172.21.15.181:0/1435548362 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f65e8054410 con 0x7f65f015e020 2023-10-16T11:20:43.053 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-10-16T11:20:43.051+0000 7fb991f98700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:20:43.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.225+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 --> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65f00c2190 con 0x7f65dc06b140 2023-10-16T11:20:43.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.239+0000 7f65ecff9700 1 -- 172.21.15.181:0/1435548362 <== mgr.11814 v2:172.21.15.181:6832/3273985848 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529066 (secure 0 0 0) 0x7f65e40050d0 con 0x7f65dc06b140 2023-10-16T11:20:43.246 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f65dc06b140 msgr2=0x7f65dc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f65dc06b140 0x7f65dc06d600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65e40016c0 tx=0x7f65e4009040 comp rx=0 tx=0).stop 2023-10-16T11:20:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f015e020 msgr2=0x7f65f00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f015e020 0x7f65f00afb90 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f65e800cc00 tx=0x7f65e800a060 comp rx=0 tx=0).stop 2023-10-16T11:20:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 shutdown_connections 2023-10-16T11:20:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f65dc06b140 0x7f65dc06d600 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65f00c26b0 0x7f65f00c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f015e020 0x7f65f00afb90 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 --2- 172.21.15.181:0/1435548362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65f015dc40 0x7f65f00a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 >> 172.21.15.181:0/1435548362 conn(0x7f65f00b1780 msgr2=0x7f65f006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.243+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 shutdown_connections 2023-10-16T11:20:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.244+0000 7f65f6721700 1 -- 172.21.15.181:0/1435548362 wait complete. 2023-10-16T11:20:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:20:43.257 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-16T11:20:41.757610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596435,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":710,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":449,"ondisk_log_size":449,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":273,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465660,"kb_used_data":24692,"kb_used_omap":12,"kb_used_meta":440819,"kb_avail":754509060,"statfs":{"total":773094113280,"available":772617277440,"internally_reserved":0,"allocated":25284608,"data_stored":13609862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":451399224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.631342"},"pg_stats":[{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161677+0000","last_change":"2023-10-16T11:20:39.160588+0000","last_active":"2023-10-16T11:20:39.161677+0000","last_peered":"2023-10-16T11:20:39.161677+0000","last_clean":"2023-10-16T11:20:39.161677+0000","last_became_active":"2023-10-16T11:20:39.160284+0000","last_became_peered":"2023-10-16T11:20:39.160284+0000","last_unstale":"2023-10-16T11:20:39.161677+0000","last_undegraded":"2023-10-16T11:20:39.161677+0000","last_fullsized":"2023-10-16T11:20:39.161677+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:29.209829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.142502+0000","last_change":"2023-10-16T11:20:38.151690+0000","last_active":"2023-10-16T11:20:39.142502+0000","last_peered":"2023-10-16T11:20:39.142502+0000","last_clean":"2023-10-16T11:20:39.142502+0000","last_became_active":"2023-10-16T11:20:38.151420+0000","last_became_peered":"2023-10-16T11:20:38.151420+0000","last_unstale":"2023-10-16T11:20:39.142502+0000","last_undegraded":"2023-10-16T11:20:39.142502+0000","last_fullsized":"2023-10-16T11:20:39.142502+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:26:27.292365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138584+0000","last_change":"2023-10-16T11:20:26.092187+0000","last_active":"2023-10-16T11:20:38.138584+0000","last_peered":"2023-10-16T11:20:38.138584+0000","last_clean":"2023-10-16T11:20:38.138584+0000","last_became_active":"2023-10-16T11:20:26.091884+0000","last_became_peered":"2023-10-16T11:20:26.091884+0000","last_unstale":"2023-10-16T11:20:38.138584+0000","last_undegraded":"2023-10-16T11:20:38.138584+0000","last_fullsized":"2023-10-16T11:20:38.138584+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:16.851198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164076+0000","last_change":"2023-10-16T11:20:39.158941+0000","last_active":"2023-10-16T11:20:39.164076+0000","last_peered":"2023-10-16T11:20:39.164076+0000","last_clean":"2023-10-16T11:20:39.164076+0000","last_became_active":"2023-10-16T11:20:39.158259+0000","last_became_peered":"2023-10-16T11:20:39.158259+0000","last_unstale":"2023-10-16T11:20:39.164076+0000","last_undegraded":"2023-10-16T11:20:39.164076+0000","last_fullsized":"2023-10-16T11:20:39.164076+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:19.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.146698+0000","last_change":"2023-10-16T11:20:38.147489+0000","last_active":"2023-10-16T11:20:39.146698+0000","last_peered":"2023-10-16T11:20:39.146698+0000","last_clean":"2023-10-16T11:20:39.146698+0000","last_became_active":"2023-10-16T11:20:38.147047+0000","last_became_peered":"2023-10-16T11:20:38.147047+0000","last_unstale":"2023-10-16T11:20:39.146698+0000","last_undegraded":"2023-10-16T11:20:39.146698+0000","last_fullsized":"2023-10-16T11:20:39.146698+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:38.993484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137790+0000","last_change":"2023-10-16T11:20:26.085301+0000","last_active":"2023-10-16T11:20:38.137790+0000","last_peered":"2023-10-16T11:20:38.137790+0000","last_clean":"2023-10-16T11:20:38.137790+0000","last_became_active":"2023-10-16T11:20:26.084922+0000","last_became_peered":"2023-10-16T11:20:26.084922+0000","last_unstale":"2023-10-16T11:20:38.137790+0000","last_undegraded":"2023-10-16T11:20:38.137790+0000","last_fullsized":"2023-10-16T11:20:38.137790+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:36.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.154275+0000","last_change":"2023-10-16T11:20:39.148752+0000","last_active":"2023-10-16T11:20:39.154275+0000","last_peered":"2023-10-16T11:20:39.154275+0000","last_clean":"2023-10-16T11:20:39.154275+0000","last_became_active":"2023-10-16T11:20:39.148478+0000","last_became_peered":"2023-10-16T11:20:39.148478+0000","last_unstale":"2023-10-16T11:20:39.154275+0000","last_undegraded":"2023-10-16T11:20:39.154275+0000","last_fullsized":"2023-10-16T11:20:39.154275+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:17.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.136470+0000","last_change":"2023-10-16T11:20:38.143454+0000","last_active":"2023-10-16T11:20:39.136470+0000","last_peered":"2023-10-16T11:20:39.136470+0000","last_clean":"2023-10-16T11:20:39.136470+0000","last_became_active":"2023-10-16T11:20:38.143171+0000","last_became_peered":"2023-10-16T11:20:38.143171+0000","last_unstale":"2023-10-16T11:20:39.136470+0000","last_undegraded":"2023-10-16T11:20:39.136470+0000","last_fullsized":"2023-10-16T11:20:39.136470+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:08.172420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139386+0000","last_change":"2023-10-16T11:20:26.083255+0000","last_active":"2023-10-16T11:20:38.139386+0000","last_peered":"2023-10-16T11:20:38.139386+0000","last_clean":"2023-10-16T11:20:38.139386+0000","last_became_active":"2023-10-16T11:20:26.082915+0000","last_became_peered":"2023-10-16T11:20:26.082915+0000","last_unstale":"2023-10-16T11:20:38.139386+0000","last_undegraded":"2023-10-16T11:20:38.139386+0000","last_fullsized":"2023-10-16T11:20:38.139386+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:02.635859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155981+0000","last_change":"2023-10-16T11:20:39.147597+0000","last_active":"2023-10-16T11:20:39.155981+0000","last_peered":"2023-10-16T11:20:39.155981+0000","last_clean":"2023-10-16T11:20:39.155981+0000","last_became_active":"2023-10-16T11:20:39.147315+0000","last_became_peered":"2023-10-16T11:20:39.147315+0000","last_unstale":"2023-10-16T11:20:39.155981+0000","last_undegraded":"2023-10-16T11:20:39.155981+0000","last_fullsized":"2023-10-16T11:20:39.155981+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:07.186956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.135958+0000","last_change":"2023-10-16T11:20:38.149234+0000","last_active":"2023-10-16T11:20:39.135958+0000","last_peered":"2023-10-16T11:20:39.135958+0000","last_clean":"2023-10-16T11:20:39.135958+0000","last_became_active":"2023-10-16T11:20:38.148942+0000","last_became_peered":"2023-10-16T11:20:38.148942+0000","last_unstale":"2023-10-16T11:20:39.135958+0000","last_undegraded":"2023-10-16T11:20:39.135958+0000","last_fullsized":"2023-10-16T11:20:39.135958+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:52.920256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145086+0000","last_change":"2023-10-16T11:20:26.084494+0000","last_active":"2023-10-16T11:20:38.145086+0000","last_peered":"2023-10-16T11:20:38.145086+0000","last_clean":"2023-10-16T11:20:38.145086+0000","last_became_active":"2023-10-16T11:20:26.084169+0000","last_became_peered":"2023-10-16T11:20:26.084169+0000","last_unstale":"2023-10-16T11:20:38.145086+0000","last_undegraded":"2023-10-16T11:20:38.145086+0000","last_fullsized":"2023-10-16T11:20:38.145086+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156251+0000","last_change":"2023-10-16T11:20:39.151439+0000","last_active":"2023-10-16T11:20:39.156251+0000","last_peered":"2023-10-16T11:20:39.156251+0000","last_clean":"2023-10-16T11:20:39.156251+0000","last_became_active":"2023-10-16T11:20:39.151122+0000","last_became_peered":"2023-10-16T11:20:39.151122+0000","last_unstale":"2023-10-16T11:20:39.156251+0000","last_undegraded":"2023-10-16T11:20:39.156251+0000","last_fullsized":"2023-10-16T11:20:39.156251+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.142220+0000","last_change":"2023-10-16T11:20:38.146065+0000","last_active":"2023-10-16T11:20:39.142220+0000","last_peered":"2023-10-16T11:20:39.142220+0000","last_clean":"2023-10-16T11:20:39.142220+0000","last_became_active":"2023-10-16T11:20:38.145791+0000","last_became_peered":"2023-10-16T11:20:38.145791+0000","last_unstale":"2023-10-16T11:20:39.142220+0000","last_undegraded":"2023-10-16T11:20:39.142220+0000","last_fullsized":"2023-10-16T11:20:39.142220+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:50.313696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140695+0000","last_change":"2023-10-16T11:20:26.080175+0000","last_active":"2023-10-16T11:20:38.140695+0000","last_peered":"2023-10-16T11:20:38.140695+0000","last_clean":"2023-10-16T11:20:38.140695+0000","last_became_active":"2023-10-16T11:20:26.079884+0000","last_became_peered":"2023-10-16T11:20:26.079884+0000","last_unstale":"2023-10-16T11:20:38.140695+0000","last_undegraded":"2023-10-16T11:20:38.140695+0000","last_fullsized":"2023-10-16T11:20:38.140695+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:58.169049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.147776+0000","last_change":"2023-10-16T11:20:39.143971+0000","last_active":"2023-10-16T11:20:39.147776+0000","last_peered":"2023-10-16T11:20:39.147776+0000","last_clean":"2023-10-16T11:20:39.147776+0000","last_became_active":"2023-10-16T11:20:39.143672+0000","last_became_peered":"2023-10-16T11:20:39.143672+0000","last_unstale":"2023-10-16T11:20:39.147776+0000","last_undegraded":"2023-10-16T11:20:39.147776+0000","last_fullsized":"2023-10-16T11:20:39.147776+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:01.290963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155136+0000","last_change":"2023-10-16T11:20:38.148647+0000","last_active":"2023-10-16T11:20:39.155136+0000","last_peered":"2023-10-16T11:20:39.155136+0000","last_clean":"2023-10-16T11:20:39.155136+0000","last_became_active":"2023-10-16T11:20:38.148311+0000","last_became_peered":"2023-10-16T11:20:38.148311+0000","last_unstale":"2023-10-16T11:20:39.155136+0000","last_undegraded":"2023-10-16T11:20:39.155136+0000","last_fullsized":"2023-10-16T11:20:39.155136+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:55:02.128796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143834+0000","last_change":"2023-10-16T11:20:26.099739+0000","last_active":"2023-10-16T11:20:38.143834+0000","last_peered":"2023-10-16T11:20:38.143834+0000","last_clean":"2023-10-16T11:20:38.143834+0000","last_became_active":"2023-10-16T11:20:26.099459+0000","last_became_peered":"2023-10-16T11:20:26.099459+0000","last_unstale":"2023-10-16T11:20:38.143834+0000","last_undegraded":"2023-10-16T11:20:38.143834+0000","last_fullsized":"2023-10-16T11:20:38.143834+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:14.530550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164978+0000","last_change":"2023-10-16T11:20:39.160298+0000","last_active":"2023-10-16T11:20:39.164978+0000","last_peered":"2023-10-16T11:20:39.164978+0000","last_clean":"2023-10-16T11:20:39.164978+0000","last_became_active":"2023-10-16T11:20:39.159862+0000","last_became_peered":"2023-10-16T11:20:39.159862+0000","last_unstale":"2023-10-16T11:20:39.164978+0000","last_undegraded":"2023-10-16T11:20:39.164978+0000","last_fullsized":"2023-10-16T11:20:39.164978+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:57.413653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.140110+0000","last_change":"2023-10-16T11:20:38.155852+0000","last_active":"2023-10-16T11:20:39.140110+0000","last_peered":"2023-10-16T11:20:39.140110+0000","last_clean":"2023-10-16T11:20:39.140110+0000","last_became_active":"2023-10-16T11:20:38.155607+0000","last_became_peered":"2023-10-16T11:20:38.155607+0000","last_unstale":"2023-10-16T11:20:39.140110+0000","last_undegraded":"2023-10-16T11:20:39.140110+0000","last_fullsized":"2023-10-16T11:20:39.140110+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:07.405065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137992+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:38.137992+0000","last_peered":"2023-10-16T11:20:38.137992+0000","last_clean":"2023-10-16T11:20:38.137992+0000","last_became_active":"2023-10-16T11:20:26.098864+0000","last_became_peered":"2023-10-16T11:20:26.098864+0000","last_unstale":"2023-10-16T11:20:38.137992+0000","last_undegraded":"2023-10-16T11:20:38.137992+0000","last_fullsized":"2023-10-16T11:20:38.137992+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:04.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159032+0000","last_change":"2023-10-16T11:20:39.150940+0000","last_active":"2023-10-16T11:20:39.159032+0000","last_peered":"2023-10-16T11:20:39.159032+0000","last_clean":"2023-10-16T11:20:39.159032+0000","last_became_active":"2023-10-16T11:20:39.150672+0000","last_became_peered":"2023-10-16T11:20:39.150672+0000","last_unstale":"2023-10-16T11:20:39.159032+0000","last_undegraded":"2023-10-16T11:20:39.159032+0000","last_fullsized":"2023-10-16T11:20:39.159032+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:37.473407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164777+0000","last_change":"2023-10-16T11:20:38.147703+0000","last_active":"2023-10-16T11:20:39.164777+0000","last_peered":"2023-10-16T11:20:39.164777+0000","last_clean":"2023-10-16T11:20:39.164777+0000","last_became_active":"2023-10-16T11:20:38.147409+0000","last_became_peered":"2023-10-16T11:20:38.147409+0000","last_unstale":"2023-10-16T11:20:39.164777+0000","last_undegraded":"2023-10-16T11:20:39.164777+0000","last_fullsized":"2023-10-16T11:20:39.164777+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:42.287590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.146935+0000","last_change":"2023-10-16T11:20:26.084562+0000","last_active":"2023-10-16T11:20:38.146935+0000","last_peered":"2023-10-16T11:20:38.146935+0000","last_clean":"2023-10-16T11:20:38.146935+0000","last_became_active":"2023-10-16T11:20:26.083055+0000","last_became_peered":"2023-10-16T11:20:26.083055+0000","last_unstale":"2023-10-16T11:20:38.146935+0000","last_undegraded":"2023-10-16T11:20:38.146935+0000","last_fullsized":"2023-10-16T11:20:38.146935+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.947551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.154018+0000","last_change":"2023-10-16T11:20:39.153110+0000","last_active":"2023-10-16T11:20:39.154018+0000","last_peered":"2023-10-16T11:20:39.154018+0000","last_clean":"2023-10-16T11:20:39.154018+0000","last_became_active":"2023-10-16T11:20:39.152840+0000","last_became_peered":"2023-10-16T11:20:39.152840+0000","last_unstale":"2023-10-16T11:20:39.154018+0000","last_undegraded":"2023-10-16T11:20:39.154018+0000","last_fullsized":"2023-10-16T11:20:39.154018+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:49.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.148219+0000","last_change":"2023-10-16T11:20:38.149235+0000","last_active":"2023-10-16T11:20:39.148219+0000","last_peered":"2023-10-16T11:20:39.148219+0000","last_clean":"2023-10-16T11:20:39.148219+0000","last_became_active":"2023-10-16T11:20:38.148954+0000","last_became_peered":"2023-10-16T11:20:38.148954+0000","last_unstale":"2023-10-16T11:20:39.148219+0000","last_undegraded":"2023-10-16T11:20:39.148219+0000","last_fullsized":"2023-10-16T11:20:39.148219+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:29.223643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140645+0000","last_change":"2023-10-16T11:20:26.077403+0000","last_active":"2023-10-16T11:20:38.140645+0000","last_peered":"2023-10-16T11:20:38.140645+0000","last_clean":"2023-10-16T11:20:38.140645+0000","last_became_active":"2023-10-16T11:20:26.077133+0000","last_became_peered":"2023-10-16T11:20:26.077133+0000","last_unstale":"2023-10-16T11:20:38.140645+0000","last_undegraded":"2023-10-16T11:20:38.140645+0000","last_fullsized":"2023-10-16T11:20:38.140645+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:15.067913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164765+0000","last_change":"2023-10-16T11:20:39.161088+0000","last_active":"2023-10-16T11:20:39.164765+0000","last_peered":"2023-10-16T11:20:39.164765+0000","last_clean":"2023-10-16T11:20:39.164765+0000","last_became_active":"2023-10-16T11:20:39.160727+0000","last_became_peered":"2023-10-16T11:20:39.160727+0000","last_unstale":"2023-10-16T11:20:39.164765+0000","last_undegraded":"2023-10-16T11:20:39.164765+0000","last_fullsized":"2023-10-16T11:20:39.164765+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:49.867073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149404+0000","last_change":"2023-10-16T11:20:38.153279+0000","last_active":"2023-10-16T11:20:39.149404+0000","last_peered":"2023-10-16T11:20:39.149404+0000","last_clean":"2023-10-16T11:20:39.149404+0000","last_became_active":"2023-10-16T11:20:38.152980+0000","last_became_peered":"2023-10-16T11:20:38.152980+0000","last_unstale":"2023-10-16T11:20:39.149404+0000","last_undegraded":"2023-10-16T11:20:39.149404+0000","last_fullsized":"2023-10-16T11:20:39.149404+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:29:50.661320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138945+0000","last_change":"2023-10-16T11:20:26.092685+0000","last_active":"2023-10-16T11:20:38.138945+0000","last_peered":"2023-10-16T11:20:38.138945+0000","last_clean":"2023-10-16T11:20:38.138945+0000","last_became_active":"2023-10-16T11:20:26.092413+0000","last_became_peered":"2023-10-16T11:20:26.092413+0000","last_unstale":"2023-10-16T11:20:38.138945+0000","last_undegraded":"2023-10-16T11:20:38.138945+0000","last_fullsized":"2023-10-16T11:20:38.138945+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:40.852817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166731+0000","last_change":"2023-10-16T11:20:39.163622+0000","last_active":"2023-10-16T11:20:39.166731+0000","last_peered":"2023-10-16T11:20:39.166731+0000","last_clean":"2023-10-16T11:20:39.166731+0000","last_became_active":"2023-10-16T11:20:39.162568+0000","last_became_peered":"2023-10-16T11:20:39.162568+0000","last_unstale":"2023-10-16T11:20:39.166731+0000","last_undegraded":"2023-10-16T11:20:39.166731+0000","last_fullsized":"2023-10-16T11:20:39.166731+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:28.140801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149047+0000","last_change":"2023-10-16T11:20:38.155283+0000","last_active":"2023-10-16T11:20:39.149047+0000","last_peered":"2023-10-16T11:20:39.149047+0000","last_clean":"2023-10-16T11:20:39.149047+0000","last_became_active":"2023-10-16T11:20:38.155034+0000","last_became_peered":"2023-10-16T11:20:38.155034+0000","last_unstale":"2023-10-16T11:20:39.149047+0000","last_undegraded":"2023-10-16T11:20:39.149047+0000","last_fullsized":"2023-10-16T11:20:39.149047+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:53:09.383768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135013+0000","last_change":"2023-10-16T11:20:26.094837+0000","last_active":"2023-10-16T11:20:38.135013+0000","last_peered":"2023-10-16T11:20:38.135013+0000","last_clean":"2023-10-16T11:20:38.135013+0000","last_became_active":"2023-10-16T11:20:26.094247+0000","last_became_peered":"2023-10-16T11:20:26.094247+0000","last_unstale":"2023-10-16T11:20:38.135013+0000","last_undegraded":"2023-10-16T11:20:38.135013+0000","last_fullsized":"2023-10-16T11:20:38.135013+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:16.136940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164583+0000","last_change":"2023-10-16T11:20:39.161219+0000","last_active":"2023-10-16T11:20:39.164583+0000","last_peered":"2023-10-16T11:20:39.164583+0000","last_clean":"2023-10-16T11:20:39.164583+0000","last_became_active":"2023-10-16T11:20:39.160957+0000","last_became_peered":"2023-10-16T11:20:39.160957+0000","last_unstale":"2023-10-16T11:20:39.164583+0000","last_undegraded":"2023-10-16T11:20:39.164583+0000","last_fullsized":"2023-10-16T11:20:39.164583+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:55:02.646050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.153113+0000","last_change":"2023-10-16T11:20:38.143859+0000","last_active":"2023-10-16T11:20:39.153113+0000","last_peered":"2023-10-16T11:20:39.153113+0000","last_clean":"2023-10-16T11:20:39.153113+0000","last_became_active":"2023-10-16T11:20:38.143299+0000","last_became_peered":"2023-10-16T11:20:38.143299+0000","last_unstale":"2023-10-16T11:20:39.153113+0000","last_undegraded":"2023-10-16T11:20:39.153113+0000","last_fullsized":"2023-10-16T11:20:39.153113+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:42.910732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147559+0000","last_change":"2023-10-16T11:20:26.091607+0000","last_active":"2023-10-16T11:20:38.147559+0000","last_peered":"2023-10-16T11:20:38.147559+0000","last_clean":"2023-10-16T11:20:38.147559+0000","last_became_active":"2023-10-16T11:20:26.091336+0000","last_became_peered":"2023-10-16T11:20:26.091336+0000","last_unstale":"2023-10-16T11:20:38.147559+0000","last_undegraded":"2023-10-16T11:20:38.147559+0000","last_fullsized":"2023-10-16T11:20:38.147559+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:47.664336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.151327+0000","last_change":"2023-10-16T11:20:39.145395+0000","last_active":"2023-10-16T11:20:39.151327+0000","last_peered":"2023-10-16T11:20:39.151327+0000","last_clean":"2023-10-16T11:20:39.151327+0000","last_became_active":"2023-10-16T11:20:39.144591+0000","last_became_peered":"2023-10-16T11:20:39.144591+0000","last_unstale":"2023-10-16T11:20:39.151327+0000","last_undegraded":"2023-10-16T11:20:39.151327+0000","last_fullsized":"2023-10-16T11:20:39.151327+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:37.473407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.146176+0000","last_change":"2023-10-16T11:20:38.146394+0000","last_active":"2023-10-16T11:20:39.146176+0000","last_peered":"2023-10-16T11:20:39.146176+0000","last_clean":"2023-10-16T11:20:39.146176+0000","last_became_active":"2023-10-16T11:20:38.146137+0000","last_became_peered":"2023-10-16T11:20:38.146137+0000","last_unstale":"2023-10-16T11:20:39.146176+0000","last_undegraded":"2023-10-16T11:20:39.146176+0000","last_fullsized":"2023-10-16T11:20:39.146176+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:38:09.171546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138016+0000","last_change":"2023-10-16T11:20:26.101354+0000","last_active":"2023-10-16T11:20:38.138016+0000","last_peered":"2023-10-16T11:20:38.138016+0000","last_clean":"2023-10-16T11:20:38.138016+0000","last_became_active":"2023-10-16T11:20:26.101082+0000","last_became_peered":"2023-10-16T11:20:26.101082+0000","last_unstale":"2023-10-16T11:20:38.138016+0000","last_undegraded":"2023-10-16T11:20:38.138016+0000","last_fullsized":"2023-10-16T11:20:38.138016+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:48.559039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155223+0000","last_change":"2023-10-16T11:20:39.152171+0000","last_active":"2023-10-16T11:20:39.155223+0000","last_peered":"2023-10-16T11:20:39.155223+0000","last_clean":"2023-10-16T11:20:39.155223+0000","last_became_active":"2023-10-16T11:20:39.151905+0000","last_became_peered":"2023-10-16T11:20:39.151905+0000","last_unstale":"2023-10-16T11:20:39.155223+0000","last_undegraded":"2023-10-16T11:20:39.155223+0000","last_fullsized":"2023-10-16T11:20:39.155223+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:56.439651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167209+0000","last_change":"2023-10-16T11:20:38.153302+0000","last_active":"2023-10-16T11:20:39.167209+0000","last_peered":"2023-10-16T11:20:39.167209+0000","last_clean":"2023-10-16T11:20:39.167209+0000","last_became_active":"2023-10-16T11:20:38.153051+0000","last_became_peered":"2023-10-16T11:20:38.153051+0000","last_unstale":"2023-10-16T11:20:39.167209+0000","last_undegraded":"2023-10-16T11:20:39.167209+0000","last_fullsized":"2023-10-16T11:20:39.167209+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:11.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138179+0000","last_change":"2023-10-16T11:20:26.087929+0000","last_active":"2023-10-16T11:20:38.138179+0000","last_peered":"2023-10-16T11:20:38.138179+0000","last_clean":"2023-10-16T11:20:38.138179+0000","last_became_active":"2023-10-16T11:20:26.087611+0000","last_became_peered":"2023-10-16T11:20:26.087611+0000","last_unstale":"2023-10-16T11:20:38.138179+0000","last_undegraded":"2023-10-16T11:20:38.138179+0000","last_fullsized":"2023-10-16T11:20:38.138179+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:47.826756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159557+0000","last_change":"2023-10-16T11:20:39.155764+0000","last_active":"2023-10-16T11:20:39.159557+0000","last_peered":"2023-10-16T11:20:39.159557+0000","last_clean":"2023-10-16T11:20:39.159557+0000","last_became_active":"2023-10-16T11:20:39.155482+0000","last_became_peered":"2023-10-16T11:20:39.155482+0000","last_unstale":"2023-10-16T11:20:39.159557+0000","last_undegraded":"2023-10-16T11:20:39.159557+0000","last_fullsized":"2023-10-16T11:20:39.159557+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:20.055333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166250+0000","last_change":"2023-10-16T11:20:38.152264+0000","last_active":"2023-10-16T11:20:39.166250+0000","last_peered":"2023-10-16T11:20:39.166250+0000","last_clean":"2023-10-16T11:20:39.166250+0000","last_became_active":"2023-10-16T11:20:38.150890+0000","last_became_peered":"2023-10-16T11:20:38.150890+0000","last_unstale":"2023-10-16T11:20:39.166250+0000","last_undegraded":"2023-10-16T11:20:39.166250+0000","last_fullsized":"2023-10-16T11:20:39.166250+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:19.049883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140855+0000","last_change":"2023-10-16T11:20:26.076097+0000","last_active":"2023-10-16T11:20:38.140855+0000","last_peered":"2023-10-16T11:20:38.140855+0000","last_clean":"2023-10-16T11:20:38.140855+0000","last_became_active":"2023-10-16T11:20:26.075864+0000","last_became_peered":"2023-10-16T11:20:26.075864+0000","last_unstale":"2023-10-16T11:20:38.140855+0000","last_undegraded":"2023-10-16T11:20:38.140855+0000","last_fullsized":"2023-10-16T11:20:38.140855+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:36.478137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159058+0000","last_change":"2023-10-16T11:20:39.151346+0000","last_active":"2023-10-16T11:20:39.159058+0000","last_peered":"2023-10-16T11:20:39.159058+0000","last_clean":"2023-10-16T11:20:39.159058+0000","last_became_active":"2023-10-16T11:20:39.151087+0000","last_became_peered":"2023-10-16T11:20:39.151087+0000","last_unstale":"2023-10-16T11:20:39.159058+0000","last_undegraded":"2023-10-16T11:20:39.159058+0000","last_fullsized":"2023-10-16T11:20:39.159058+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.410514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162062+0000","last_change":"2023-10-16T11:20:38.150490+0000","last_active":"2023-10-16T11:20:39.162062+0000","last_peered":"2023-10-16T11:20:39.162062+0000","last_clean":"2023-10-16T11:20:39.162062+0000","last_became_active":"2023-10-16T11:20:38.149916+0000","last_became_peered":"2023-10-16T11:20:38.149916+0000","last_unstale":"2023-10-16T11:20:39.162062+0000","last_undegraded":"2023-10-16T11:20:39.162062+0000","last_fullsized":"2023-10-16T11:20:39.162062+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:26.373477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137015+0000","last_change":"2023-10-16T11:20:26.084690+0000","last_active":"2023-10-16T11:20:38.137015+0000","last_peered":"2023-10-16T11:20:38.137015+0000","last_clean":"2023-10-16T11:20:38.137015+0000","last_became_active":"2023-10-16T11:20:26.084390+0000","last_became_peered":"2023-10-16T11:20:26.084390+0000","last_unstale":"2023-10-16T11:20:38.137015+0000","last_undegraded":"2023-10-16T11:20:38.137015+0000","last_fullsized":"2023-10-16T11:20:38.137015+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:14.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156380+0000","last_change":"2023-10-16T11:20:39.149712+0000","last_active":"2023-10-16T11:20:39.156380+0000","last_peered":"2023-10-16T11:20:39.156380+0000","last_clean":"2023-10-16T11:20:39.156380+0000","last_became_active":"2023-10-16T11:20:39.149442+0000","last_became_peered":"2023-10-16T11:20:39.149442+0000","last_unstale":"2023-10-16T11:20:39.156380+0000","last_undegraded":"2023-10-16T11:20:39.156380+0000","last_fullsized":"2023-10-16T11:20:39.156380+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.279291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"239'8","reported_seq":19,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.168267+0000","last_change":"2023-10-16T11:20:38.151976+0000","last_active":"2023-10-16T11:20:39.168267+0000","last_peered":"2023-10-16T11:20:39.168267+0000","last_clean":"2023-10-16T11:20:39.168267+0000","last_became_active":"2023-10-16T11:20:38.151146+0000","last_became_peered":"2023-10-16T11:20:38.151146+0000","last_unstale":"2023-10-16T11:20:39.168267+0000","last_undegraded":"2023-10-16T11:20:39.168267+0000","last_fullsized":"2023-10-16T11:20:39.168267+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:20.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.150259+0000","last_change":"2023-10-16T11:20:26.089392+0000","last_active":"2023-10-16T11:20:38.150259+0000","last_peered":"2023-10-16T11:20:38.150259+0000","last_clean":"2023-10-16T11:20:38.150259+0000","last_became_active":"2023-10-16T11:20:26.089083+0000","last_became_peered":"2023-10-16T11:20:26.089083+0000","last_unstale":"2023-10-16T11:20:38.150259+0000","last_undegraded":"2023-10-16T11:20:38.150259+0000","last_fullsized":"2023-10-16T11:20:38.150259+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:16.986811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162394+0000","last_change":"2023-10-16T11:20:39.157175+0000","last_active":"2023-10-16T11:20:39.162394+0000","last_peered":"2023-10-16T11:20:39.162394+0000","last_clean":"2023-10-16T11:20:39.162394+0000","last_became_active":"2023-10-16T11:20:39.155597+0000","last_became_peered":"2023-10-16T11:20:39.155597+0000","last_unstale":"2023-10-16T11:20:39.162394+0000","last_undegraded":"2023-10-16T11:20:39.162394+0000","last_fullsized":"2023-10-16T11:20:39.162394+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149643+0000","last_change":"2023-10-16T11:20:38.144446+0000","last_active":"2023-10-16T11:20:39.149643+0000","last_peered":"2023-10-16T11:20:39.149643+0000","last_clean":"2023-10-16T11:20:39.149643+0000","last_became_active":"2023-10-16T11:20:38.144073+0000","last_became_peered":"2023-10-16T11:20:38.144073+0000","last_unstale":"2023-10-16T11:20:39.149643+0000","last_undegraded":"2023-10-16T11:20:39.149643+0000","last_fullsized":"2023-10-16T11:20:39.149643+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:02.781626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137878+0000","last_change":"2023-10-16T11:20:26.084395+0000","last_active":"2023-10-16T11:20:38.137878+0000","last_peered":"2023-10-16T11:20:38.137878+0000","last_clean":"2023-10-16T11:20:38.137878+0000","last_became_active":"2023-10-16T11:20:26.084119+0000","last_became_peered":"2023-10-16T11:20:26.084119+0000","last_unstale":"2023-10-16T11:20:38.137878+0000","last_undegraded":"2023-10-16T11:20:38.137878+0000","last_fullsized":"2023-10-16T11:20:38.137878+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.234510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166630+0000","last_change":"2023-10-16T11:20:39.160984+0000","last_active":"2023-10-16T11:20:39.166630+0000","last_peered":"2023-10-16T11:20:39.166630+0000","last_clean":"2023-10-16T11:20:39.166630+0000","last_became_active":"2023-10-16T11:20:39.160711+0000","last_became_peered":"2023-10-16T11:20:39.160711+0000","last_unstale":"2023-10-16T11:20:39.166630+0000","last_undegraded":"2023-10-16T11:20:39.166630+0000","last_fullsized":"2023-10-16T11:20:39.166630+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:52.110161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.136317+0000","last_change":"2023-10-16T11:20:38.149377+0000","last_active":"2023-10-16T11:20:39.136317+0000","last_peered":"2023-10-16T11:20:39.136317+0000","last_clean":"2023-10-16T11:20:39.136317+0000","last_became_active":"2023-10-16T11:20:38.149090+0000","last_became_peered":"2023-10-16T11:20:38.149090+0000","last_unstale":"2023-10-16T11:20:39.136317+0000","last_undegraded":"2023-10-16T11:20:39.136317+0000","last_fullsized":"2023-10-16T11:20:39.136317+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:55.434201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148505+0000","last_change":"2023-10-16T11:20:26.095722+0000","last_active":"2023-10-16T11:20:38.148505+0000","last_peered":"2023-10-16T11:20:38.148505+0000","last_clean":"2023-10-16T11:20:38.148505+0000","last_became_active":"2023-10-16T11:20:26.095489+0000","last_became_peered":"2023-10-16T11:20:26.095489+0000","last_unstale":"2023-10-16T11:20:38.148505+0000","last_undegraded":"2023-10-16T11:20:38.148505+0000","last_fullsized":"2023-10-16T11:20:38.148505+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:32.623008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.148235+0000","last_change":"2023-10-16T11:20:39.146671+0000","last_active":"2023-10-16T11:20:39.148235+0000","last_peered":"2023-10-16T11:20:39.148235+0000","last_clean":"2023-10-16T11:20:39.148235+0000","last_became_active":"2023-10-16T11:20:39.146403+0000","last_became_peered":"2023-10-16T11:20:39.146403+0000","last_unstale":"2023-10-16T11:20:39.148235+0000","last_undegraded":"2023-10-16T11:20:39.148235+0000","last_fullsized":"2023-10-16T11:20:39.148235+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:47.362989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166905+0000","last_change":"2023-10-16T11:20:38.156486+0000","last_active":"2023-10-16T11:20:39.166905+0000","last_peered":"2023-10-16T11:20:39.166905+0000","last_clean":"2023-10-16T11:20:39.166905+0000","last_became_active":"2023-10-16T11:20:38.156213+0000","last_became_peered":"2023-10-16T11:20:38.156213+0000","last_unstale":"2023-10-16T11:20:39.166905+0000","last_undegraded":"2023-10-16T11:20:39.166905+0000","last_fullsized":"2023-10-16T11:20:39.166905+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:51.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140187+0000","last_change":"2023-10-16T11:20:26.072060+0000","last_active":"2023-10-16T11:20:38.140187+0000","last_peered":"2023-10-16T11:20:38.140187+0000","last_clean":"2023-10-16T11:20:38.140187+0000","last_became_active":"2023-10-16T11:20:26.071739+0000","last_became_peered":"2023-10-16T11:20:26.071739+0000","last_unstale":"2023-10-16T11:20:38.140187+0000","last_undegraded":"2023-10-16T11:20:38.140187+0000","last_fullsized":"2023-10-16T11:20:38.140187+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:29.585136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162172+0000","last_change":"2023-10-16T11:20:39.158804+0000","last_active":"2023-10-16T11:20:39.162172+0000","last_peered":"2023-10-16T11:20:39.162172+0000","last_clean":"2023-10-16T11:20:39.162172+0000","last_became_active":"2023-10-16T11:20:39.158534+0000","last_became_peered":"2023-10-16T11:20:39.158534+0000","last_unstale":"2023-10-16T11:20:39.162172+0000","last_undegraded":"2023-10-16T11:20:39.162172+0000","last_fullsized":"2023-10-16T11:20:39.162172+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:36:19.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"239'6","reported_seq":17,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.165955+0000","last_change":"2023-10-16T11:20:38.149465+0000","last_active":"2023-10-16T11:20:39.165955+0000","last_peered":"2023-10-16T11:20:39.165955+0000","last_clean":"2023-10-16T11:20:39.165955+0000","last_became_active":"2023-10-16T11:20:38.149180+0000","last_became_peered":"2023-10-16T11:20:38.149180+0000","last_unstale":"2023-10-16T11:20:39.165955+0000","last_undegraded":"2023-10-16T11:20:39.165955+0000","last_fullsized":"2023-10-16T11:20:39.165955+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:19.685275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147587+0000","last_change":"2023-10-16T11:20:26.083156+0000","last_active":"2023-10-16T11:20:38.147587+0000","last_peered":"2023-10-16T11:20:38.147587+0000","last_clean":"2023-10-16T11:20:38.147587+0000","last_became_active":"2023-10-16T11:20:26.082813+0000","last_became_peered":"2023-10-16T11:20:26.082813+0000","last_unstale":"2023-10-16T11:20:38.147587+0000","last_undegraded":"2023-10-16T11:20:38.147587+0000","last_fullsized":"2023-10-16T11:20:38.147587+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:26.407654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164617+0000","last_change":"2023-10-16T11:20:39.162139+0000","last_active":"2023-10-16T11:20:39.164617+0000","last_peered":"2023-10-16T11:20:39.164617+0000","last_clean":"2023-10-16T11:20:39.164617+0000","last_became_active":"2023-10-16T11:20:39.161358+0000","last_became_peered":"2023-10-16T11:20:39.161358+0000","last_unstale":"2023-10-16T11:20:39.164617+0000","last_undegraded":"2023-10-16T11:20:39.164617+0000","last_fullsized":"2023-10-16T11:20:39.164617+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:25.286014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"239'3","reported_seq":14,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167678+0000","last_change":"2023-10-16T11:20:38.154552+0000","last_active":"2023-10-16T11:20:39.167678+0000","last_peered":"2023-10-16T11:20:39.167678+0000","last_clean":"2023-10-16T11:20:39.167678+0000","last_became_active":"2023-10-16T11:20:38.154006+0000","last_became_peered":"2023-10-16T11:20:38.154006+0000","last_unstale":"2023-10-16T11:20:39.167678+0000","last_undegraded":"2023-10-16T11:20:39.167678+0000","last_fullsized":"2023-10-16T11:20:39.167678+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:11.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138345+0000","last_change":"2023-10-16T11:20:26.097008+0000","last_active":"2023-10-16T11:20:38.138345+0000","last_peered":"2023-10-16T11:20:38.138345+0000","last_clean":"2023-10-16T11:20:38.138345+0000","last_became_active":"2023-10-16T11:20:26.096754+0000","last_became_peered":"2023-10-16T11:20:26.096754+0000","last_unstale":"2023-10-16T11:20:38.138345+0000","last_undegraded":"2023-10-16T11:20:38.138345+0000","last_fullsized":"2023-10-16T11:20:38.138345+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:24.400518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164959+0000","last_change":"2023-10-16T11:20:39.160131+0000","last_active":"2023-10-16T11:20:39.164959+0000","last_peered":"2023-10-16T11:20:39.164959+0000","last_clean":"2023-10-16T11:20:39.164959+0000","last_became_active":"2023-10-16T11:20:39.159491+0000","last_became_peered":"2023-10-16T11:20:39.159491+0000","last_unstale":"2023-10-16T11:20:39.164959+0000","last_undegraded":"2023-10-16T11:20:39.164959+0000","last_fullsized":"2023-10-16T11:20:39.164959+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:04.137178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149765+0000","last_change":"2023-10-16T11:20:38.152108+0000","last_active":"2023-10-16T11:20:39.149765+0000","last_peered":"2023-10-16T11:20:39.149765+0000","last_clean":"2023-10-16T11:20:39.149765+0000","last_became_active":"2023-10-16T11:20:38.151725+0000","last_became_peered":"2023-10-16T11:20:38.151725+0000","last_unstale":"2023-10-16T11:20:39.149765+0000","last_undegraded":"2023-10-16T11:20:39.149765+0000","last_fullsized":"2023-10-16T11:20:39.149765+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:35.137888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147312+0000","last_change":"2023-10-16T11:20:26.093039+0000","last_active":"2023-10-16T11:20:38.147312+0000","last_peered":"2023-10-16T11:20:38.147312+0000","last_clean":"2023-10-16T11:20:38.147312+0000","last_became_active":"2023-10-16T11:20:26.092725+0000","last_became_peered":"2023-10-16T11:20:26.092725+0000","last_unstale":"2023-10-16T11:20:38.147312+0000","last_undegraded":"2023-10-16T11:20:38.147312+0000","last_fullsized":"2023-10-16T11:20:38.147312+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:37.794560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161746+0000","last_change":"2023-10-16T11:20:39.157268+0000","last_active":"2023-10-16T11:20:39.161746+0000","last_peered":"2023-10-16T11:20:39.161746+0000","last_clean":"2023-10-16T11:20:39.161746+0000","last_became_active":"2023-10-16T11:20:39.155468+0000","last_became_peered":"2023-10-16T11:20:39.155468+0000","last_unstale":"2023-10-16T11:20:39.161746+0000","last_undegraded":"2023-10-16T11:20:39.161746+0000","last_fullsized":"2023-10-16T11:20:39.161746+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:14.690224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.144741+0000","last_change":"2023-10-16T11:20:38.150702+0000","last_active":"2023-10-16T11:20:39.144741+0000","last_peered":"2023-10-16T11:20:39.144741+0000","last_clean":"2023-10-16T11:20:39.144741+0000","last_became_active":"2023-10-16T11:20:38.150460+0000","last_became_peered":"2023-10-16T11:20:38.150460+0000","last_unstale":"2023-10-16T11:20:39.144741+0000","last_undegraded":"2023-10-16T11:20:39.144741+0000","last_fullsized":"2023-10-16T11:20:39.144741+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:48.920755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137738+0000","last_change":"2023-10-16T11:20:26.091655+0000","last_active":"2023-10-16T11:20:38.137738+0000","last_peered":"2023-10-16T11:20:38.137738+0000","last_clean":"2023-10-16T11:20:38.137738+0000","last_became_active":"2023-10-16T11:20:26.091402+0000","last_became_peered":"2023-10-16T11:20:26.091402+0000","last_unstale":"2023-10-16T11:20:38.137738+0000","last_undegraded":"2023-10-16T11:20:38.137738+0000","last_fullsized":"2023-10-16T11:20:38.137738+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:25.104508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161802+0000","last_change":"2023-10-16T11:20:39.157173+0000","last_active":"2023-10-16T11:20:39.161802+0000","last_peered":"2023-10-16T11:20:39.161802+0000","last_clean":"2023-10-16T11:20:39.161802+0000","last_became_active":"2023-10-16T11:20:39.154819+0000","last_became_peered":"2023-10-16T11:20:39.154819+0000","last_unstale":"2023-10-16T11:20:39.161802+0000","last_undegraded":"2023-10-16T11:20:39.161802+0000","last_fullsized":"2023-10-16T11:20:39.161802+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:43.293040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.147567+0000","last_change":"2023-10-16T11:20:38.143615+0000","last_active":"2023-10-16T11:20:39.147567+0000","last_peered":"2023-10-16T11:20:39.147567+0000","last_clean":"2023-10-16T11:20:39.147567+0000","last_became_active":"2023-10-16T11:20:38.143225+0000","last_became_peered":"2023-10-16T11:20:38.143225+0000","last_unstale":"2023-10-16T11:20:39.147567+0000","last_undegraded":"2023-10-16T11:20:39.147567+0000","last_fullsized":"2023-10-16T11:20:39.147567+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:04.671147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145217+0000","last_change":"2023-10-16T11:20:26.088880+0000","last_active":"2023-10-16T11:20:38.145217+0000","last_peered":"2023-10-16T11:20:38.145217+0000","last_clean":"2023-10-16T11:20:38.145217+0000","last_became_active":"2023-10-16T11:20:26.088427+0000","last_became_peered":"2023-10-16T11:20:26.088427+0000","last_unstale":"2023-10-16T11:20:38.145217+0000","last_undegraded":"2023-10-16T11:20:38.145217+0000","last_fullsized":"2023-10-16T11:20:38.145217+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:30.068691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167431+0000","last_change":"2023-10-16T11:20:39.162512+0000","last_active":"2023-10-16T11:20:39.167431+0000","last_peered":"2023-10-16T11:20:39.167431+0000","last_clean":"2023-10-16T11:20:39.167431+0000","last_became_active":"2023-10-16T11:20:39.161697+0000","last_became_peered":"2023-10-16T11:20:39.161697+0000","last_unstale":"2023-10-16T11:20:39.167431+0000","last_undegraded":"2023-10-16T11:20:39.167431+0000","last_fullsized":"2023-10-16T11:20:39.167431+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:21:44.297465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.138713+0000","last_change":"2023-10-16T11:20:38.155830+0000","last_active":"2023-10-16T11:20:39.138713+0000","last_peered":"2023-10-16T11:20:39.138713+0000","last_clean":"2023-10-16T11:20:39.138713+0000","last_became_active":"2023-10-16T11:20:38.155569+0000","last_became_peered":"2023-10-16T11:20:38.155569+0000","last_unstale":"2023-10-16T11:20:39.138713+0000","last_undegraded":"2023-10-16T11:20:39.138713+0000","last_fullsized":"2023-10-16T11:20:39.138713+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:17.423235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138862+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:38.138862+0000","last_peered":"2023-10-16T11:20:38.138862+0000","last_clean":"2023-10-16T11:20:38.138862+0000","last_became_active":"2023-10-16T11:20:26.098861+0000","last_became_peered":"2023-10-16T11:20:26.098861+0000","last_unstale":"2023-10-16T11:20:38.138862+0000","last_undegraded":"2023-10-16T11:20:38.138862+0000","last_fullsized":"2023-10-16T11:20:38.138862+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:03.247595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.163764+0000","last_change":"2023-10-16T11:20:39.157612+0000","last_active":"2023-10-16T11:20:39.163764+0000","last_peered":"2023-10-16T11:20:39.163764+0000","last_clean":"2023-10-16T11:20:39.163764+0000","last_became_active":"2023-10-16T11:20:39.156968+0000","last_became_peered":"2023-10-16T11:20:39.156968+0000","last_unstale":"2023-10-16T11:20:39.163764+0000","last_undegraded":"2023-10-16T11:20:39.163764+0000","last_fullsized":"2023-10-16T11:20:39.163764+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:53.149153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161976+0000","last_change":"2023-10-16T11:20:38.146077+0000","last_active":"2023-10-16T11:20:39.161976+0000","last_peered":"2023-10-16T11:20:39.161976+0000","last_clean":"2023-10-16T11:20:39.161976+0000","last_became_active":"2023-10-16T11:20:38.141808+0000","last_became_peered":"2023-10-16T11:20:38.141808+0000","last_unstale":"2023-10-16T11:20:39.161976+0000","last_undegraded":"2023-10-16T11:20:39.161976+0000","last_fullsized":"2023-10-16T11:20:39.161976+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:17.840209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135121+0000","last_change":"2023-10-16T11:20:26.098406+0000","last_active":"2023-10-16T11:20:38.135121+0000","last_peered":"2023-10-16T11:20:38.135121+0000","last_clean":"2023-10-16T11:20:38.135121+0000","last_became_active":"2023-10-16T11:20:26.098082+0000","last_became_peered":"2023-10-16T11:20:26.098082+0000","last_unstale":"2023-10-16T11:20:38.135121+0000","last_undegraded":"2023-10-16T11:20:38.135121+0000","last_fullsized":"2023-10-16T11:20:38.135121+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:44.140416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161132+0000","last_change":"2023-10-16T11:20:39.154978+0000","last_active":"2023-10-16T11:20:39.161132+0000","last_peered":"2023-10-16T11:20:39.161132+0000","last_clean":"2023-10-16T11:20:39.161132+0000","last_became_active":"2023-10-16T11:20:39.154413+0000","last_became_peered":"2023-10-16T11:20:39.154413+0000","last_unstale":"2023-10-16T11:20:39.161132+0000","last_undegraded":"2023-10-16T11:20:39.161132+0000","last_fullsized":"2023-10-16T11:20:39.161132+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:07.186956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162262+0000","last_change":"2023-10-16T11:20:38.148730+0000","last_active":"2023-10-16T11:20:39.162262+0000","last_peered":"2023-10-16T11:20:39.162262+0000","last_clean":"2023-10-16T11:20:39.162262+0000","last_became_active":"2023-10-16T11:20:38.148449+0000","last_became_peered":"2023-10-16T11:20:38.148449+0000","last_unstale":"2023-10-16T11:20:39.162262+0000","last_undegraded":"2023-10-16T11:20:39.162262+0000","last_fullsized":"2023-10-16T11:20:39.162262+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:26.584595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139246+0000","last_change":"2023-10-16T11:20:26.082815+0000","last_active":"2023-10-16T11:20:38.139246+0000","last_peered":"2023-10-16T11:20:38.139246+0000","last_clean":"2023-10-16T11:20:38.139246+0000","last_became_active":"2023-10-16T11:20:26.082524+0000","last_became_peered":"2023-10-16T11:20:26.082524+0000","last_unstale":"2023-10-16T11:20:38.139246+0000","last_undegraded":"2023-10-16T11:20:38.139246+0000","last_fullsized":"2023-10-16T11:20:38.139246+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:16.774780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.160408+0000","last_change":"2023-10-16T11:20:39.157899+0000","last_active":"2023-10-16T11:20:39.160408+0000","last_peered":"2023-10-16T11:20:39.160408+0000","last_clean":"2023-10-16T11:20:39.160408+0000","last_became_active":"2023-10-16T11:20:39.157736+0000","last_became_peered":"2023-10-16T11:20:39.157736+0000","last_unstale":"2023-10-16T11:20:39.160408+0000","last_undegraded":"2023-10-16T11:20:39.160408+0000","last_fullsized":"2023-10-16T11:20:39.160408+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:03:57.268779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166398+0000","last_change":"2023-10-16T11:20:38.153358+0000","last_active":"2023-10-16T11:20:39.166398+0000","last_peered":"2023-10-16T11:20:39.166398+0000","last_clean":"2023-10-16T11:20:39.166398+0000","last_became_active":"2023-10-16T11:20:38.153067+0000","last_became_peered":"2023-10-16T11:20:38.153067+0000","last_unstale":"2023-10-16T11:20:39.166398+0000","last_undegraded":"2023-10-16T11:20:39.166398+0000","last_fullsized":"2023-10-16T11:20:39.166398+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:16.499508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134733+0000","last_change":"2023-10-16T11:20:26.100460+0000","last_active":"2023-10-16T11:20:38.134733+0000","last_peered":"2023-10-16T11:20:38.134733+0000","last_clean":"2023-10-16T11:20:38.134733+0000","last_became_active":"2023-10-16T11:20:26.100088+0000","last_became_peered":"2023-10-16T11:20:26.100088+0000","last_unstale":"2023-10-16T11:20:38.134733+0000","last_undegraded":"2023-10-16T11:20:38.134733+0000","last_fullsized":"2023-10-16T11:20:38.134733+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:38.907350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161011+0000","last_change":"2023-10-16T11:20:39.154070+0000","last_active":"2023-10-16T11:20:39.161011+0000","last_peered":"2023-10-16T11:20:39.161011+0000","last_clean":"2023-10-16T11:20:39.161011+0000","last_became_active":"2023-10-16T11:20:39.153638+0000","last_became_peered":"2023-10-16T11:20:39.153638+0000","last_unstale":"2023-10-16T11:20:39.161011+0000","last_undegraded":"2023-10-16T11:20:39.161011+0000","last_fullsized":"2023-10-16T11:20:39.161011+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:45.452553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167470+0000","last_change":"2023-10-16T11:20:38.154607+0000","last_active":"2023-10-16T11:20:39.167470+0000","last_peered":"2023-10-16T11:20:39.167470+0000","last_clean":"2023-10-16T11:20:39.167470+0000","last_became_active":"2023-10-16T11:20:38.154195+0000","last_became_peered":"2023-10-16T11:20:38.154195+0000","last_unstale":"2023-10-16T11:20:39.167470+0000","last_undegraded":"2023-10-16T11:20:39.167470+0000","last_fullsized":"2023-10-16T11:20:39.167470+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:20.923075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140478+0000","last_change":"2023-10-16T11:20:26.070717+0000","last_active":"2023-10-16T11:20:38.140478+0000","last_peered":"2023-10-16T11:20:38.140478+0000","last_clean":"2023-10-16T11:20:38.140478+0000","last_became_active":"2023-10-16T11:20:26.070394+0000","last_became_peered":"2023-10-16T11:20:26.070394+0000","last_unstale":"2023-10-16T11:20:38.140478+0000","last_undegraded":"2023-10-16T11:20:38.140478+0000","last_fullsized":"2023-10-16T11:20:38.140478+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.158924+0000","last_change":"2023-10-16T11:20:39.154353+0000","last_active":"2023-10-16T11:20:39.158924+0000","last_peered":"2023-10-16T11:20:39.158924+0000","last_clean":"2023-10-16T11:20:39.158924+0000","last_became_active":"2023-10-16T11:20:39.154048+0000","last_became_peered":"2023-10-16T11:20:39.154048+0000","last_unstale":"2023-10-16T11:20:39.158924+0000","last_undegraded":"2023-10-16T11:20:39.158924+0000","last_fullsized":"2023-10-16T11:20:39.158924+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:49.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.163231+0000","last_change":"2023-10-16T11:20:38.152029+0000","last_active":"2023-10-16T11:20:39.163231+0000","last_peered":"2023-10-16T11:20:39.163231+0000","last_clean":"2023-10-16T11:20:39.163231+0000","last_became_active":"2023-10-16T11:20:38.151751+0000","last_became_peered":"2023-10-16T11:20:38.151751+0000","last_unstale":"2023-10-16T11:20:39.163231+0000","last_undegraded":"2023-10-16T11:20:39.163231+0000","last_fullsized":"2023-10-16T11:20:39.163231+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:28.415965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.146679+0000","last_change":"2023-10-16T11:20:26.086782+0000","last_active":"2023-10-16T11:20:38.146679+0000","last_peered":"2023-10-16T11:20:38.146679+0000","last_clean":"2023-10-16T11:20:38.146679+0000","last_became_active":"2023-10-16T11:20:26.086510+0000","last_became_peered":"2023-10-16T11:20:26.086510+0000","last_unstale":"2023-10-16T11:20:38.146679+0000","last_undegraded":"2023-10-16T11:20:38.146679+0000","last_fullsized":"2023-10-16T11:20:38.146679+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:26:24.730788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161759+0000","last_change":"2023-10-16T11:20:39.157450+0000","last_active":"2023-10-16T11:20:39.161759+0000","last_peered":"2023-10-16T11:20:39.161759+0000","last_clean":"2023-10-16T11:20:39.161759+0000","last_became_active":"2023-10-16T11:20:39.156969+0000","last_became_peered":"2023-10-16T11:20:39.156969+0000","last_unstale":"2023-10-16T11:20:39.161759+0000","last_undegraded":"2023-10-16T11:20:39.161759+0000","last_fullsized":"2023-10-16T11:20:39.161759+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:46.589205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"239'4","reported_seq":15,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.165200+0000","last_change":"2023-10-16T11:20:38.157572+0000","last_active":"2023-10-16T11:20:39.165200+0000","last_peered":"2023-10-16T11:20:39.165200+0000","last_clean":"2023-10-16T11:20:39.165200+0000","last_became_active":"2023-10-16T11:20:38.157341+0000","last_became_peered":"2023-10-16T11:20:38.157341+0000","last_unstale":"2023-10-16T11:20:39.165200+0000","last_undegraded":"2023-10-16T11:20:39.165200+0000","last_fullsized":"2023-10-16T11:20:39.165200+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:48.659423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137829+0000","last_change":"2023-10-16T11:20:26.083094+0000","last_active":"2023-10-16T11:20:38.137829+0000","last_peered":"2023-10-16T11:20:38.137829+0000","last_clean":"2023-10-16T11:20:38.137829+0000","last_became_active":"2023-10-16T11:20:26.080031+0000","last_became_peered":"2023-10-16T11:20:26.080031+0000","last_unstale":"2023-10-16T11:20:38.137829+0000","last_undegraded":"2023-10-16T11:20:38.137829+0000","last_fullsized":"2023-10-16T11:20:38.137829+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:06:55.285592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157624+0000","last_change":"2023-10-16T11:20:39.157223+0000","last_active":"2023-10-16T11:20:39.157624+0000","last_peered":"2023-10-16T11:20:39.157624+0000","last_clean":"2023-10-16T11:20:39.157624+0000","last_became_active":"2023-10-16T11:20:39.156903+0000","last_became_peered":"2023-10-16T11:20:39.156903+0000","last_unstale":"2023-10-16T11:20:39.157624+0000","last_undegraded":"2023-10-16T11:20:39.157624+0000","last_fullsized":"2023-10-16T11:20:39.157624+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:40.015528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148371+0000","last_change":"2023-10-16T11:20:26.096214+0000","last_active":"2023-10-16T11:20:38.148371+0000","last_peered":"2023-10-16T11:20:38.148371+0000","last_clean":"2023-10-16T11:20:38.148371+0000","last_became_active":"2023-10-16T11:20:26.093850+0000","last_became_peered":"2023-10-16T11:20:26.093850+0000","last_unstale":"2023-10-16T11:20:38.148371+0000","last_undegraded":"2023-10-16T11:20:38.148371+0000","last_fullsized":"2023-10-16T11:20:38.148371+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:48.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138671+0000","last_change":"2023-10-16T11:20:25.084166+0000","last_active":"2023-10-16T11:20:38.138671+0000","last_peered":"2023-10-16T11:20:38.138671+0000","last_clean":"2023-10-16T11:20:38.138671+0000","last_became_active":"2023-10-16T11:20:25.083857+0000","last_became_peered":"2023-10-16T11:20:25.083857+0000","last_unstale":"2023-10-16T11:20:38.138671+0000","last_undegraded":"2023-10-16T11:20:38.138671+0000","last_fullsized":"2023-10-16T11:20:38.138671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:47.210086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164122+0000","last_change":"2023-10-16T11:20:39.160255+0000","last_active":"2023-10-16T11:20:39.164122+0000","last_peered":"2023-10-16T11:20:39.164122+0000","last_clean":"2023-10-16T11:20:39.164122+0000","last_became_active":"2023-10-16T11:20:39.158836+0000","last_became_peered":"2023-10-16T11:20:39.158836+0000","last_unstale":"2023-10-16T11:20:39.164122+0000","last_undegraded":"2023-10-16T11:20:39.164122+0000","last_fullsized":"2023-10-16T11:20:39.164122+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:16.320484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134779+0000","last_change":"2023-10-16T11:20:26.094539+0000","last_active":"2023-10-16T11:20:38.134779+0000","last_peered":"2023-10-16T11:20:38.134779+0000","last_clean":"2023-10-16T11:20:38.134779+0000","last_became_active":"2023-10-16T11:20:26.094220+0000","last_became_peered":"2023-10-16T11:20:26.094220+0000","last_unstale":"2023-10-16T11:20:38.134779+0000","last_undegraded":"2023-10-16T11:20:38.134779+0000","last_fullsized":"2023-10-16T11:20:38.134779+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:05.280523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139278+0000","last_change":"2023-10-16T11:20:25.080334+0000","last_active":"2023-10-16T11:20:38.139278+0000","last_peered":"2023-10-16T11:20:38.139278+0000","last_clean":"2023-10-16T11:20:38.139278+0000","last_became_active":"2023-10-16T11:20:25.080084+0000","last_became_peered":"2023-10-16T11:20:25.080084+0000","last_unstale":"2023-10-16T11:20:38.139278+0000","last_undegraded":"2023-10-16T11:20:38.139278+0000","last_fullsized":"2023-10-16T11:20:38.139278+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:19.309219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162266+0000","last_change":"2023-10-16T11:20:39.159896+0000","last_active":"2023-10-16T11:20:39.162266+0000","last_peered":"2023-10-16T11:20:39.162266+0000","last_clean":"2023-10-16T11:20:39.162266+0000","last_became_active":"2023-10-16T11:20:39.159635+0000","last_became_peered":"2023-10-16T11:20:39.159635+0000","last_unstale":"2023-10-16T11:20:39.162266+0000","last_undegraded":"2023-10-16T11:20:39.162266+0000","last_fullsized":"2023-10-16T11:20:39.162266+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:15.954886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148596+0000","last_change":"2023-10-16T11:20:26.086141+0000","last_active":"2023-10-16T11:20:38.148596+0000","last_peered":"2023-10-16T11:20:38.148596+0000","last_clean":"2023-10-16T11:20:38.148596+0000","last_became_active":"2023-10-16T11:20:26.085865+0000","last_became_peered":"2023-10-16T11:20:26.085865+0000","last_unstale":"2023-10-16T11:20:38.148596+0000","last_undegraded":"2023-10-16T11:20:38.148596+0000","last_fullsized":"2023-10-16T11:20:38.148596+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:31.177316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140439+0000","last_change":"2023-10-16T11:20:25.087325+0000","last_active":"2023-10-16T11:20:38.140439+0000","last_peered":"2023-10-16T11:20:38.140439+0000","last_clean":"2023-10-16T11:20:38.140439+0000","last_became_active":"2023-10-16T11:20:25.087033+0000","last_became_peered":"2023-10-16T11:20:25.087033+0000","last_unstale":"2023-10-16T11:20:38.140439+0000","last_undegraded":"2023-10-16T11:20:38.140439+0000","last_fullsized":"2023-10-16T11:20:38.140439+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:17:22.652116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.165332+0000","last_change":"2023-10-16T11:20:39.162377+0000","last_active":"2023-10-16T11:20:39.165332+0000","last_peered":"2023-10-16T11:20:39.165332+0000","last_clean":"2023-10-16T11:20:39.165332+0000","last_became_active":"2023-10-16T11:20:39.161718+0000","last_became_peered":"2023-10-16T11:20:39.161718+0000","last_unstale":"2023-10-16T11:20:39.165332+0000","last_undegraded":"2023-10-16T11:20:39.165332+0000","last_fullsized":"2023-10-16T11:20:39.165332+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:09.177870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134545+0000","last_change":"2023-10-16T11:20:26.090198+0000","last_active":"2023-10-16T11:20:38.134545+0000","last_peered":"2023-10-16T11:20:38.134545+0000","last_clean":"2023-10-16T11:20:38.134545+0000","last_became_active":"2023-10-16T11:20:26.089545+0000","last_became_peered":"2023-10-16T11:20:26.089545+0000","last_unstale":"2023-10-16T11:20:38.134545+0000","last_undegraded":"2023-10-16T11:20:38.134545+0000","last_fullsized":"2023-10-16T11:20:38.134545+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:03.204915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143339+0000","last_change":"2023-10-16T11:20:25.084321+0000","last_active":"2023-10-16T11:20:38.143339+0000","last_peered":"2023-10-16T11:20:38.143339+0000","last_clean":"2023-10-16T11:20:38.143339+0000","last_became_active":"2023-10-16T11:20:25.083847+0000","last_became_peered":"2023-10-16T11:20:25.083847+0000","last_unstale":"2023-10-16T11:20:38.143339+0000","last_undegraded":"2023-10-16T11:20:38.143339+0000","last_fullsized":"2023-10-16T11:20:38.143339+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:16.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161437+0000","last_change":"2023-10-16T11:20:39.155651+0000","last_active":"2023-10-16T11:20:39.161437+0000","last_peered":"2023-10-16T11:20:39.161437+0000","last_clean":"2023-10-16T11:20:39.161437+0000","last_became_active":"2023-10-16T11:20:39.155383+0000","last_became_peered":"2023-10-16T11:20:39.155383+0000","last_unstale":"2023-10-16T11:20:39.161437+0000","last_undegraded":"2023-10-16T11:20:39.161437+0000","last_fullsized":"2023-10-16T11:20:39.161437+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:58.804201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140784+0000","last_change":"2023-10-16T11:20:26.089495+0000","last_active":"2023-10-16T11:20:38.140784+0000","last_peered":"2023-10-16T11:20:38.140784+0000","last_clean":"2023-10-16T11:20:38.140784+0000","last_became_active":"2023-10-16T11:20:26.089203+0000","last_became_peered":"2023-10-16T11:20:26.089203+0000","last_unstale":"2023-10-16T11:20:38.140784+0000","last_undegraded":"2023-10-16T11:20:38.140784+0000","last_fullsized":"2023-10-16T11:20:38.140784+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:59.154547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138269+0000","last_change":"2023-10-16T11:20:25.083810+0000","last_active":"2023-10-16T11:20:38.138269+0000","last_peered":"2023-10-16T11:20:38.138269+0000","last_clean":"2023-10-16T11:20:38.138269+0000","last_became_active":"2023-10-16T11:20:25.083251+0000","last_became_peered":"2023-10-16T11:20:25.083251+0000","last_unstale":"2023-10-16T11:20:38.138269+0000","last_undegraded":"2023-10-16T11:20:38.138269+0000","last_fullsized":"2023-10-16T11:20:38.138269+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:44.723324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157058+0000","last_change":"2023-10-16T11:20:39.155271+0000","last_active":"2023-10-16T11:20:39.157058+0000","last_peered":"2023-10-16T11:20:39.157058+0000","last_clean":"2023-10-16T11:20:39.157058+0000","last_became_active":"2023-10-16T11:20:39.154993+0000","last_became_peered":"2023-10-16T11:20:39.154993+0000","last_unstale":"2023-10-16T11:20:39.157058+0000","last_undegraded":"2023-10-16T11:20:39.157058+0000","last_fullsized":"2023-10-16T11:20:39.157058+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:19.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136843+0000","last_change":"2023-10-16T11:20:26.071683+0000","last_active":"2023-10-16T11:20:38.136843+0000","last_peered":"2023-10-16T11:20:38.136843+0000","last_clean":"2023-10-16T11:20:38.136843+0000","last_became_active":"2023-10-16T11:20:26.071396+0000","last_became_peered":"2023-10-16T11:20:26.071396+0000","last_unstale":"2023-10-16T11:20:38.136843+0000","last_undegraded":"2023-10-16T11:20:38.136843+0000","last_fullsized":"2023-10-16T11:20:38.136843+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:15.600869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138188+0000","last_change":"2023-10-16T11:20:25.085339+0000","last_active":"2023-10-16T11:20:38.138188+0000","last_peered":"2023-10-16T11:20:38.138188+0000","last_clean":"2023-10-16T11:20:38.138188+0000","last_became_active":"2023-10-16T11:20:25.085073+0000","last_became_peered":"2023-10-16T11:20:25.085073+0000","last_unstale":"2023-10-16T11:20:38.138188+0000","last_undegraded":"2023-10-16T11:20:38.138188+0000","last_fullsized":"2023-10-16T11:20:38.138188+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:23.090533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161391+0000","last_change":"2023-10-16T11:20:39.159106+0000","last_active":"2023-10-16T11:20:39.161391+0000","last_peered":"2023-10-16T11:20:39.161391+0000","last_clean":"2023-10-16T11:20:39.161391+0000","last_became_active":"2023-10-16T11:20:39.158709+0000","last_became_peered":"2023-10-16T11:20:39.158709+0000","last_unstale":"2023-10-16T11:20:39.161391+0000","last_undegraded":"2023-10-16T11:20:39.161391+0000","last_fullsized":"2023-10-16T11:20:39.161391+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:53.925705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137998+0000","last_change":"2023-10-16T11:20:26.085542+0000","last_active":"2023-10-16T11:20:38.137998+0000","last_peered":"2023-10-16T11:20:38.137998+0000","last_clean":"2023-10-16T11:20:38.137998+0000","last_became_active":"2023-10-16T11:20:26.083529+0000","last_became_peered":"2023-10-16T11:20:26.083529+0000","last_unstale":"2023-10-16T11:20:38.137998+0000","last_undegraded":"2023-10-16T11:20:38.137998+0000","last_fullsized":"2023-10-16T11:20:38.137998+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136934+0000","last_change":"2023-10-16T11:20:25.088288+0000","last_active":"2023-10-16T11:20:38.136934+0000","last_peered":"2023-10-16T11:20:38.136934+0000","last_clean":"2023-10-16T11:20:38.136934+0000","last_became_active":"2023-10-16T11:20:25.087144+0000","last_became_peered":"2023-10-16T11:20:25.087144+0000","last_unstale":"2023-10-16T11:20:38.136934+0000","last_undegraded":"2023-10-16T11:20:38.136934+0000","last_fullsized":"2023-10-16T11:20:38.136934+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:16:49.706199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157304+0000","last_change":"2023-10-16T11:20:39.151272+0000","last_active":"2023-10-16T11:20:39.157304+0000","last_peered":"2023-10-16T11:20:39.157304+0000","last_clean":"2023-10-16T11:20:39.157304+0000","last_became_active":"2023-10-16T11:20:39.151019+0000","last_became_peered":"2023-10-16T11:20:39.151019+0000","last_unstale":"2023-10-16T11:20:39.157304+0000","last_undegraded":"2023-10-16T11:20:39.157304+0000","last_fullsized":"2023-10-16T11:20:39.157304+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:02.270086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138027+0000","last_change":"2023-10-16T11:20:26.085425+0000","last_active":"2023-10-16T11:20:38.138027+0000","last_peered":"2023-10-16T11:20:38.138027+0000","last_clean":"2023-10-16T11:20:38.138027+0000","last_became_active":"2023-10-16T11:20:26.085150+0000","last_became_peered":"2023-10-16T11:20:26.085150+0000","last_unstale":"2023-10-16T11:20:38.138027+0000","last_undegraded":"2023-10-16T11:20:38.138027+0000","last_fullsized":"2023-10-16T11:20:38.138027+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:14.978178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.150373+0000","last_change":"2023-10-16T11:20:25.078062+0000","last_active":"2023-10-16T11:20:38.150373+0000","last_peered":"2023-10-16T11:20:38.150373+0000","last_clean":"2023-10-16T11:20:38.150373+0000","last_became_active":"2023-10-16T11:20:25.077777+0000","last_became_peered":"2023-10-16T11:20:25.077777+0000","last_unstale":"2023-10-16T11:20:38.150373+0000","last_undegraded":"2023-10-16T11:20:38.150373+0000","last_fullsized":"2023-10-16T11:20:38.150373+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:13.108385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161818+0000","last_change":"2023-10-16T11:20:39.157413+0000","last_active":"2023-10-16T11:20:39.161818+0000","last_peered":"2023-10-16T11:20:39.161818+0000","last_clean":"2023-10-16T11:20:39.161818+0000","last_became_active":"2023-10-16T11:20:39.156903+0000","last_became_peered":"2023-10-16T11:20:39.156903+0000","last_unstale":"2023-10-16T11:20:39.161818+0000","last_undegraded":"2023-10-16T11:20:39.161818+0000","last_fullsized":"2023-10-16T11:20:39.161818+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:15:21.969270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135165+0000","last_change":"2023-10-16T11:20:26.098504+0000","last_active":"2023-10-16T11:20:38.135165+0000","last_peered":"2023-10-16T11:20:38.135165+0000","last_clean":"2023-10-16T11:20:38.135165+0000","last_became_active":"2023-10-16T11:20:26.098021+0000","last_became_peered":"2023-10-16T11:20:26.098021+0000","last_unstale":"2023-10-16T11:20:38.135165+0000","last_undegraded":"2023-10-16T11:20:38.135165+0000","last_fullsized":"2023-10-16T11:20:38.135165+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:06.218021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.141329+0000","last_change":"2023-10-16T11:20:25.088083+0000","last_active":"2023-10-16T11:20:38.141329+0000","last_peered":"2023-10-16T11:20:38.141329+0000","last_clean":"2023-10-16T11:20:38.141329+0000","last_became_active":"2023-10-16T11:20:25.087792+0000","last_became_peered":"2023-10-16T11:20:25.087792+0000","last_unstale":"2023-10-16T11:20:38.141329+0000","last_undegraded":"2023-10-16T11:20:38.141329+0000","last_fullsized":"2023-10-16T11:20:38.141329+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:22.682758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161110+0000","last_change":"2023-10-16T11:20:39.150282+0000","last_active":"2023-10-16T11:20:39.161110+0000","last_peered":"2023-10-16T11:20:39.161110+0000","last_clean":"2023-10-16T11:20:39.161110+0000","last_became_active":"2023-10-16T11:20:39.149971+0000","last_became_peered":"2023-10-16T11:20:39.149971+0000","last_unstale":"2023-10-16T11:20:39.161110+0000","last_undegraded":"2023-10-16T11:20:39.161110+0000","last_fullsized":"2023-10-16T11:20:39.161110+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:39.998933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140270+0000","last_change":"2023-10-16T11:20:26.090580+0000","last_active":"2023-10-16T11:20:38.140270+0000","last_peered":"2023-10-16T11:20:38.140270+0000","last_clean":"2023-10-16T11:20:38.140270+0000","last_became_active":"2023-10-16T11:20:26.090296+0000","last_became_peered":"2023-10-16T11:20:26.090296+0000","last_unstale":"2023-10-16T11:20:38.140270+0000","last_undegraded":"2023-10-16T11:20:38.140270+0000","last_fullsized":"2023-10-16T11:20:38.140270+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:29.302528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"235'3","reported_seq":18,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145056+0000","last_change":"2023-10-16T11:20:25.083562+0000","last_active":"2023-10-16T11:20:38.145056+0000","last_peered":"2023-10-16T11:20:38.145056+0000","last_clean":"2023-10-16T11:20:38.145056+0000","last_became_active":"2023-10-16T11:20:25.083270+0000","last_became_peered":"2023-10-16T11:20:25.083270+0000","last_unstale":"2023-10-16T11:20:38.145056+0000","last_undegraded":"2023-10-16T11:20:38.145056+0000","last_fullsized":"2023-10-16T11:20:38.145056+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:34.227829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.153724+0000","last_change":"2023-10-16T11:20:39.145474+0000","last_active":"2023-10-16T11:20:39.153724+0000","last_peered":"2023-10-16T11:20:39.153724+0000","last_clean":"2023-10-16T11:20:39.153724+0000","last_became_active":"2023-10-16T11:20:39.145204+0000","last_became_peered":"2023-10-16T11:20:39.145204+0000","last_unstale":"2023-10-16T11:20:39.153724+0000","last_undegraded":"2023-10-16T11:20:39.153724+0000","last_fullsized":"2023-10-16T11:20:39.153724+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:38:10.176996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139388+0000","last_change":"2023-10-16T11:20:26.079886+0000","last_active":"2023-10-16T11:20:38.139388+0000","last_peered":"2023-10-16T11:20:38.139388+0000","last_clean":"2023-10-16T11:20:38.139388+0000","last_became_active":"2023-10-16T11:20:26.079622+0000","last_became_peered":"2023-10-16T11:20:26.079622+0000","last_unstale":"2023-10-16T11:20:38.139388+0000","last_undegraded":"2023-10-16T11:20:38.139388+0000","last_fullsized":"2023-10-16T11:20:38.139388+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:30.843293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"236'8","reported_seq":23,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139257+0000","last_change":"2023-10-16T11:20:25.083500+0000","last_active":"2023-10-16T11:20:38.139257+0000","last_peered":"2023-10-16T11:20:38.139257+0000","last_clean":"2023-10-16T11:20:38.139257+0000","last_became_active":"2023-10-16T11:20:25.083213+0000","last_became_peered":"2023-10-16T11:20:25.083213+0000","last_unstale":"2023-10-16T11:20:38.139257+0000","last_undegraded":"2023-10-16T11:20:38.139257+0000","last_fullsized":"2023-10-16T11:20:38.139257+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:04.495826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2859,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161544+0000","last_change":"2023-10-16T11:20:39.154135+0000","last_active":"2023-10-16T11:20:39.161544+0000","last_peered":"2023-10-16T11:20:39.161544+0000","last_clean":"2023-10-16T11:20:39.161544+0000","last_became_active":"2023-10-16T11:20:39.153840+0000","last_became_peered":"2023-10-16T11:20:39.153840+0000","last_unstale":"2023-10-16T11:20:39.161544+0000","last_undegraded":"2023-10-16T11:20:39.161544+0000","last_fullsized":"2023-10-16T11:20:39.161544+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:23.727389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138562+0000","last_change":"2023-10-16T11:20:26.083135+0000","last_active":"2023-10-16T11:20:38.138562+0000","last_peered":"2023-10-16T11:20:38.138562+0000","last_clean":"2023-10-16T11:20:38.138562+0000","last_became_active":"2023-10-16T11:20:26.082824+0000","last_became_peered":"2023-10-16T11:20:26.082824+0000","last_unstale":"2023-10-16T11:20:38.138562+0000","last_undegraded":"2023-10-16T11:20:38.138562+0000","last_fullsized":"2023-10-16T11:20:38.138562+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:55.337626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138471+0000","last_change":"2023-10-16T11:20:25.083057+0000","last_active":"2023-10-16T11:20:38.138471+0000","last_peered":"2023-10-16T11:20:38.138471+0000","last_clean":"2023-10-16T11:20:38.138471+0000","last_became_active":"2023-10-16T11:20:25.082601+0000","last_became_peered":"2023-10-16T11:20:25.082601+0000","last_unstale":"2023-10-16T11:20:38.138471+0000","last_undegraded":"2023-10-16T11:20:38.138471+0000","last_fullsized":"2023-10-16T11:20:38.138471+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.609347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159434+0000","last_change":"2023-10-16T11:20:39.153616+0000","last_active":"2023-10-16T11:20:39.159434+0000","last_peered":"2023-10-16T11:20:39.159434+0000","last_clean":"2023-10-16T11:20:39.159434+0000","last_became_active":"2023-10-16T11:20:39.153341+0000","last_became_peered":"2023-10-16T11:20:39.153341+0000","last_unstale":"2023-10-16T11:20:39.159434+0000","last_undegraded":"2023-10-16T11:20:39.159434+0000","last_fullsized":"2023-10-16T11:20:39.159434+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:18.576703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140892+0000","last_change":"2023-10-16T11:20:26.073157+0000","last_active":"2023-10-16T11:20:38.140892+0000","last_peered":"2023-10-16T11:20:38.140892+0000","last_clean":"2023-10-16T11:20:38.140892+0000","last_became_active":"2023-10-16T11:20:26.072892+0000","last_became_peered":"2023-10-16T11:20:26.072892+0000","last_unstale":"2023-10-16T11:20:38.140892+0000","last_undegraded":"2023-10-16T11:20:38.140892+0000","last_fullsized":"2023-10-16T11:20:38.140892+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:21:31.224577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136738+0000","last_change":"2023-10-16T11:20:25.079093+0000","last_active":"2023-10-16T11:20:38.136738+0000","last_peered":"2023-10-16T11:20:38.136738+0000","last_clean":"2023-10-16T11:20:38.136738+0000","last_became_active":"2023-10-16T11:20:25.078740+0000","last_became_peered":"2023-10-16T11:20:25.078740+0000","last_unstale":"2023-10-16T11:20:38.136738+0000","last_undegraded":"2023-10-16T11:20:38.136738+0000","last_fullsized":"2023-10-16T11:20:38.136738+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:27.511026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161727+0000","last_change":"2023-10-16T11:20:39.155879+0000","last_active":"2023-10-16T11:20:39.161727+0000","last_peered":"2023-10-16T11:20:39.161727+0000","last_clean":"2023-10-16T11:20:39.161727+0000","last_became_active":"2023-10-16T11:20:39.153704+0000","last_became_peered":"2023-10-16T11:20:39.153704+0000","last_unstale":"2023-10-16T11:20:39.161727+0000","last_undegraded":"2023-10-16T11:20:39.161727+0000","last_fullsized":"2023-10-16T11:20:39.161727+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:28.544528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139058+0000","last_change":"2023-10-16T11:20:26.084841+0000","last_active":"2023-10-16T11:20:38.139058+0000","last_peered":"2023-10-16T11:20:38.139058+0000","last_clean":"2023-10-16T11:20:38.139058+0000","last_became_active":"2023-10-16T11:20:26.084153+0000","last_became_peered":"2023-10-16T11:20:26.084153+0000","last_unstale":"2023-10-16T11:20:38.139058+0000","last_undegraded":"2023-10-16T11:20:38.139058+0000","last_fullsized":"2023-10-16T11:20:38.139058+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:30.506645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138285+0000","last_change":"2023-10-16T11:20:25.084386+0000","last_active":"2023-10-16T11:20:38.138285+0000","last_peered":"2023-10-16T11:20:38.138285+0000","last_clean":"2023-10-16T11:20:38.138285+0000","last_became_active":"2023-10-16T11:20:25.083974+0000","last_became_peered":"2023-10-16T11:20:25.083974+0000","last_unstale":"2023-10-16T11:20:38.138285+0000","last_undegraded":"2023-10-16T11:20:38.138285+0000","last_fullsized":"2023-10-16T11:20:38.138285+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:05.210033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156404+0000","last_change":"2023-10-16T11:20:39.149432+0000","last_active":"2023-10-16T11:20:39.156404+0000","last_peered":"2023-10-16T11:20:39.156404+0000","last_clean":"2023-10-16T11:20:39.156404+0000","last_became_active":"2023-10-16T11:20:39.149146+0000","last_became_peered":"2023-10-16T11:20:39.149146+0000","last_unstale":"2023-10-16T11:20:39.156404+0000","last_undegraded":"2023-10-16T11:20:39.156404+0000","last_fullsized":"2023-10-16T11:20:39.156404+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:50.453477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138082+0000","last_change":"2023-10-16T11:20:26.096969+0000","last_active":"2023-10-16T11:20:38.138082+0000","last_peered":"2023-10-16T11:20:38.138082+0000","last_clean":"2023-10-16T11:20:38.138082+0000","last_became_active":"2023-10-16T11:20:26.096691+0000","last_became_peered":"2023-10-16T11:20:26.096691+0000","last_unstale":"2023-10-16T11:20:38.138082+0000","last_undegraded":"2023-10-16T11:20:38.138082+0000","last_fullsized":"2023-10-16T11:20:38.138082+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:05.355796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"235'8","reported_seq":23,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134121+0000","last_change":"2023-10-16T11:20:25.082367+0000","last_active":"2023-10-16T11:20:38.134121+0000","last_peered":"2023-10-16T11:20:38.134121+0000","last_clean":"2023-10-16T11:20:38.134121+0000","last_became_active":"2023-10-16T11:20:25.082053+0000","last_became_peered":"2023-10-16T11:20:25.082053+0000","last_unstale":"2023-10-16T11:20:38.134121+0000","last_undegraded":"2023-10-16T11:20:38.134121+0000","last_fullsized":"2023-10-16T11:20:38.134121+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:29.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167450+0000","last_change":"2023-10-16T11:20:39.162687+0000","last_active":"2023-10-16T11:20:39.167450+0000","last_peered":"2023-10-16T11:20:39.167450+0000","last_clean":"2023-10-16T11:20:39.167450+0000","last_became_active":"2023-10-16T11:20:39.162202+0000","last_became_peered":"2023-10-16T11:20:39.162202+0000","last_unstale":"2023-10-16T11:20:39.167450+0000","last_undegraded":"2023-10-16T11:20:39.167450+0000","last_fullsized":"2023-10-16T11:20:39.167450+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:49.551026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143568+0000","last_change":"2023-10-16T11:20:26.085864+0000","last_active":"2023-10-16T11:20:38.143568+0000","last_peered":"2023-10-16T11:20:38.143568+0000","last_clean":"2023-10-16T11:20:38.143568+0000","last_became_active":"2023-10-16T11:20:26.085644+0000","last_became_peered":"2023-10-16T11:20:26.085644+0000","last_unstale":"2023-10-16T11:20:38.143568+0000","last_undegraded":"2023-10-16T11:20:38.143568+0000","last_fullsized":"2023-10-16T11:20:38.143568+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:16.138806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138671+0000","last_change":"2023-10-16T11:20:25.084626+0000","last_active":"2023-10-16T11:20:38.138671+0000","last_peered":"2023-10-16T11:20:38.138671+0000","last_clean":"2023-10-16T11:20:38.138671+0000","last_became_active":"2023-10-16T11:20:25.084276+0000","last_became_peered":"2023-10-16T11:20:25.084276+0000","last_unstale":"2023-10-16T11:20:38.138671+0000","last_undegraded":"2023-10-16T11:20:38.138671+0000","last_fullsized":"2023-10-16T11:20:38.138671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.241621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155332+0000","last_change":"2023-10-16T11:20:39.151680+0000","last_active":"2023-10-16T11:20:39.155332+0000","last_peered":"2023-10-16T11:20:39.155332+0000","last_clean":"2023-10-16T11:20:39.155332+0000","last_became_active":"2023-10-16T11:20:39.151404+0000","last_became_peered":"2023-10-16T11:20:39.151404+0000","last_unstale":"2023-10-16T11:20:39.155332+0000","last_undegraded":"2023-10-16T11:20:39.155332+0000","last_fullsized":"2023-10-16T11:20:39.155332+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:43.916182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148396+0000","last_change":"2023-10-16T11:20:26.084908+0000","last_active":"2023-10-16T11:20:38.148396+0000","last_peered":"2023-10-16T11:20:38.148396+0000","last_clean":"2023-10-16T11:20:38.148396+0000","last_became_active":"2023-10-16T11:20:26.084599+0000","last_became_peered":"2023-10-16T11:20:26.084599+0000","last_unstale":"2023-10-16T11:20:38.148396+0000","last_undegraded":"2023-10-16T11:20:38.148396+0000","last_fullsized":"2023-10-16T11:20:38.148396+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:13.336265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.142877+0000","last_change":"2023-10-16T11:20:25.087228+0000","last_active":"2023-10-16T11:20:38.142877+0000","last_peered":"2023-10-16T11:20:38.142877+0000","last_clean":"2023-10-16T11:20:38.142877+0000","last_became_active":"2023-10-16T11:20:25.086965+0000","last_became_peered":"2023-10-16T11:20:25.086965+0000","last_unstale":"2023-10-16T11:20:38.142877+0000","last_undegraded":"2023-10-16T11:20:38.142877+0000","last_fullsized":"2023-10-16T11:20:38.142877+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:08.265203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164101+0000","last_change":"2023-10-16T11:20:39.159811+0000","last_active":"2023-10-16T11:20:39.164101+0000","last_peered":"2023-10-16T11:20:39.164101+0000","last_clean":"2023-10-16T11:20:39.164101+0000","last_became_active":"2023-10-16T11:20:39.159524+0000","last_became_peered":"2023-10-16T11:20:39.159524+0000","last_unstale":"2023-10-16T11:20:39.164101+0000","last_undegraded":"2023-10-16T11:20:39.164101+0000","last_fullsized":"2023-10-16T11:20:39.164101+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:01.631928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140557+0000","last_change":"2023-10-16T11:20:26.077503+0000","last_active":"2023-10-16T11:20:38.140557+0000","last_peered":"2023-10-16T11:20:38.140557+0000","last_clean":"2023-10-16T11:20:38.140557+0000","last_became_active":"2023-10-16T11:20:26.077247+0000","last_became_peered":"2023-10-16T11:20:26.077247+0000","last_unstale":"2023-10-16T11:20:38.140557+0000","last_undegraded":"2023-10-16T11:20:38.140557+0000","last_fullsized":"2023-10-16T11:20:38.140557+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:49.573161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140761+0000","last_change":"2023-10-16T11:20:25.076671+0000","last_active":"2023-10-16T11:20:38.140761+0000","last_peered":"2023-10-16T11:20:38.140761+0000","last_clean":"2023-10-16T11:20:38.140761+0000","last_became_active":"2023-10-16T11:20:25.076356+0000","last_became_peered":"2023-10-16T11:20:25.076356+0000","last_unstale":"2023-10-16T11:20:38.140761+0000","last_undegraded":"2023-10-16T11:20:38.140761+0000","last_fullsized":"2023-10-16T11:20:38.140761+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:21.582269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.154261+0000","last_change":"2023-10-16T11:20:39.147832+0000","last_active":"2023-10-16T11:20:39.154261+0000","last_peered":"2023-10-16T11:20:39.154261+0000","last_clean":"2023-10-16T11:20:39.154261+0000","last_became_active":"2023-10-16T11:20:39.147549+0000","last_became_peered":"2023-10-16T11:20:39.147549+0000","last_unstale":"2023-10-16T11:20:39.154261+0000","last_undegraded":"2023-10-16T11:20:39.154261+0000","last_fullsized":"2023-10-16T11:20:39.154261+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:18.428685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148690+0000","last_change":"2023-10-16T11:20:26.086294+0000","last_active":"2023-10-16T11:20:38.148690+0000","last_peered":"2023-10-16T11:20:38.148690+0000","last_clean":"2023-10-16T11:20:38.148690+0000","last_became_active":"2023-10-16T11:20:26.085985+0000","last_became_peered":"2023-10-16T11:20:26.085985+0000","last_unstale":"2023-10-16T11:20:38.148690+0000","last_undegraded":"2023-10-16T11:20:38.148690+0000","last_fullsized":"2023-10-16T11:20:38.148690+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:58.159985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134968+0000","last_change":"2023-10-16T11:20:25.085222+0000","last_active":"2023-10-16T11:20:38.134968+0000","last_peered":"2023-10-16T11:20:38.134968+0000","last_clean":"2023-10-16T11:20:38.134968+0000","last_became_active":"2023-10-16T11:20:25.084958+0000","last_became_peered":"2023-10-16T11:20:25.084958+0000","last_unstale":"2023-10-16T11:20:38.134968+0000","last_undegraded":"2023-10-16T11:20:38.134968+0000","last_fullsized":"2023-10-16T11:20:38.134968+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:15.130817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161885+0000","last_change":"2023-10-16T11:20:39.160066+0000","last_active":"2023-10-16T11:20:39.161885+0000","last_peered":"2023-10-16T11:20:39.161885+0000","last_clean":"2023-10-16T11:20:39.161885+0000","last_became_active":"2023-10-16T11:20:39.158692+0000","last_became_peered":"2023-10-16T11:20:39.158692+0000","last_unstale":"2023-10-16T11:20:39.161885+0000","last_undegraded":"2023-10-16T11:20:39.161885+0000","last_fullsized":"2023-10-16T11:20:39.161885+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:48.505360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148032+0000","last_change":"2023-10-16T11:20:26.091707+0000","last_active":"2023-10-16T11:20:38.148032+0000","last_peered":"2023-10-16T11:20:38.148032+0000","last_clean":"2023-10-16T11:20:38.148032+0000","last_became_active":"2023-10-16T11:20:26.091470+0000","last_became_peered":"2023-10-16T11:20:26.091470+0000","last_unstale":"2023-10-16T11:20:38.148032+0000","last_undegraded":"2023-10-16T11:20:38.148032+0000","last_fullsized":"2023-10-16T11:20:38.148032+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137172+0000","last_change":"2023-10-16T11:20:25.086865+0000","last_active":"2023-10-16T11:20:38.137172+0000","last_peered":"2023-10-16T11:20:38.137172+0000","last_clean":"2023-10-16T11:20:38.137172+0000","last_became_active":"2023-10-16T11:20:25.086528+0000","last_became_peered":"2023-10-16T11:20:25.086528+0000","last_unstale":"2023-10-16T11:20:38.137172+0000","last_undegraded":"2023-10-16T11:20:38.137172+0000","last_fullsized":"2023-10-16T11:20:38.137172+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:17.050518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161507+0000","last_change":"2023-10-16T11:20:39.159722+0000","last_active":"2023-10-16T11:20:39.161507+0000","last_peered":"2023-10-16T11:20:39.161507+0000","last_clean":"2023-10-16T11:20:39.161507+0000","last_became_active":"2023-10-16T11:20:39.159417+0000","last_became_peered":"2023-10-16T11:20:39.159417+0000","last_unstale":"2023-10-16T11:20:39.161507+0000","last_undegraded":"2023-10-16T11:20:39.161507+0000","last_fullsized":"2023-10-16T11:20:39.161507+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:36.538334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138379+0000","last_change":"2023-10-16T11:20:26.080221+0000","last_active":"2023-10-16T11:20:38.138379+0000","last_peered":"2023-10-16T11:20:38.138379+0000","last_clean":"2023-10-16T11:20:38.138379+0000","last_became_active":"2023-10-16T11:20:26.079927+0000","last_became_peered":"2023-10-16T11:20:26.079927+0000","last_unstale":"2023-10-16T11:20:38.138379+0000","last_undegraded":"2023-10-16T11:20:38.138379+0000","last_fullsized":"2023-10-16T11:20:38.138379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:27.603420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134588+0000","last_change":"2023-10-16T11:20:25.084862+0000","last_active":"2023-10-16T11:20:38.134588+0000","last_peered":"2023-10-16T11:20:38.134588+0000","last_clean":"2023-10-16T11:20:38.134588+0000","last_became_active":"2023-10-16T11:20:25.084538+0000","last_became_peered":"2023-10-16T11:20:25.084538+0000","last_unstale":"2023-10-16T11:20:38.134588+0000","last_undegraded":"2023-10-16T11:20:38.134588+0000","last_fullsized":"2023-10-16T11:20:38.134588+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:11:13.297732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.160194+0000","last_change":"2023-10-16T11:20:39.154016+0000","last_active":"2023-10-16T11:20:39.160194+0000","last_peered":"2023-10-16T11:20:39.160194+0000","last_clean":"2023-10-16T11:20:39.160194+0000","last_became_active":"2023-10-16T11:20:39.152666+0000","last_became_peered":"2023-10-16T11:20:39.152666+0000","last_unstale":"2023-10-16T11:20:39.160194+0000","last_undegraded":"2023-10-16T11:20:39.160194+0000","last_fullsized":"2023-10-16T11:20:39.160194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:55.505270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134055+0000","last_change":"2023-10-16T11:20:26.101781+0000","last_active":"2023-10-16T11:20:38.134055+0000","last_peered":"2023-10-16T11:20:38.134055+0000","last_clean":"2023-10-16T11:20:38.134055+0000","last_became_active":"2023-10-16T11:20:26.101470+0000","last_became_peered":"2023-10-16T11:20:26.101470+0000","last_unstale":"2023-10-16T11:20:38.134055+0000","last_undegraded":"2023-10-16T11:20:38.134055+0000","last_fullsized":"2023-10-16T11:20:38.134055+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140950+0000","last_change":"2023-10-16T11:20:25.081336+0000","last_active":"2023-10-16T11:20:38.140950+0000","last_peered":"2023-10-16T11:20:38.140950+0000","last_clean":"2023-10-16T11:20:38.140950+0000","last_became_active":"2023-10-16T11:20:25.081017+0000","last_became_peered":"2023-10-16T11:20:25.081017+0000","last_unstale":"2023-10-16T11:20:38.140950+0000","last_undegraded":"2023-10-16T11:20:38.140950+0000","last_fullsized":"2023-10-16T11:20:38.140950+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:38.114268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.160792+0000","last_change":"2023-10-16T11:20:39.157949+0000","last_active":"2023-10-16T11:20:39.160792+0000","last_peered":"2023-10-16T11:20:39.160792+0000","last_clean":"2023-10-16T11:20:39.160792+0000","last_became_active":"2023-10-16T11:20:39.157626+0000","last_became_peered":"2023-10-16T11:20:39.157626+0000","last_unstale":"2023-10-16T11:20:39.160792+0000","last_undegraded":"2023-10-16T11:20:39.160792+0000","last_fullsized":"2023-10-16T11:20:39.160792+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:25.286014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147415+0000","last_change":"2023-10-16T11:20:26.085568+0000","last_active":"2023-10-16T11:20:38.147415+0000","last_peered":"2023-10-16T11:20:38.147415+0000","last_clean":"2023-10-16T11:20:38.147415+0000","last_became_active":"2023-10-16T11:20:26.085298+0000","last_became_peered":"2023-10-16T11:20:26.085298+0000","last_unstale":"2023-10-16T11:20:38.147415+0000","last_undegraded":"2023-10-16T11:20:38.147415+0000","last_fullsized":"2023-10-16T11:20:38.147415+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:23:15.106414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137842+0000","last_change":"2023-10-16T11:20:25.084776+0000","last_active":"2023-10-16T11:20:38.137842+0000","last_peered":"2023-10-16T11:20:38.137842+0000","last_clean":"2023-10-16T11:20:38.137842+0000","last_became_active":"2023-10-16T11:20:25.084415+0000","last_became_peered":"2023-10-16T11:20:25.084415+0000","last_unstale":"2023-10-16T11:20:38.137842+0000","last_undegraded":"2023-10-16T11:20:38.137842+0000","last_fullsized":"2023-10-16T11:20:38.137842+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:43.132428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162131+0000","last_change":"2023-10-16T11:20:39.160768+0000","last_active":"2023-10-16T11:20:39.162131+0000","last_peered":"2023-10-16T11:20:39.162131+0000","last_clean":"2023-10-16T11:20:39.162131+0000","last_became_active":"2023-10-16T11:20:39.160483+0000","last_became_peered":"2023-10-16T11:20:39.160483+0000","last_unstale":"2023-10-16T11:20:39.162131+0000","last_undegraded":"2023-10-16T11:20:39.162131+0000","last_fullsized":"2023-10-16T11:20:39.162131+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:00:21.564025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138763+0000","last_change":"2023-10-16T11:20:26.080094+0000","last_active":"2023-10-16T11:20:38.138763+0000","last_peered":"2023-10-16T11:20:38.138763+0000","last_clean":"2023-10-16T11:20:38.138763+0000","last_became_active":"2023-10-16T11:20:26.079844+0000","last_became_peered":"2023-10-16T11:20:26.079844+0000","last_unstale":"2023-10-16T11:20:38.138763+0000","last_undegraded":"2023-10-16T11:20:38.138763+0000","last_fullsized":"2023-10-16T11:20:38.138763+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:54.114067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145447+0000","last_change":"2023-10-16T11:20:25.084135+0000","last_active":"2023-10-16T11:20:38.145447+0000","last_peered":"2023-10-16T11:20:38.145447+0000","last_clean":"2023-10-16T11:20:38.145447+0000","last_became_active":"2023-10-16T11:20:25.083794+0000","last_became_peered":"2023-10-16T11:20:25.083794+0000","last_unstale":"2023-10-16T11:20:38.145447+0000","last_undegraded":"2023-10-16T11:20:38.145447+0000","last_fullsized":"2023-10-16T11:20:38.145447+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:57.161061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161089+0000","last_change":"2023-10-16T11:20:39.154294+0000","last_active":"2023-10-16T11:20:39.161089+0000","last_peered":"2023-10-16T11:20:39.161089+0000","last_clean":"2023-10-16T11:20:39.161089+0000","last_became_active":"2023-10-16T11:20:39.153885+0000","last_became_peered":"2023-10-16T11:20:39.153885+0000","last_unstale":"2023-10-16T11:20:39.161089+0000","last_undegraded":"2023-10-16T11:20:39.161089+0000","last_fullsized":"2023-10-16T11:20:39.161089+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:48.308706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138681+0000","last_change":"2023-10-16T11:20:26.083428+0000","last_active":"2023-10-16T11:20:38.138681+0000","last_peered":"2023-10-16T11:20:38.138681+0000","last_clean":"2023-10-16T11:20:38.138681+0000","last_became_active":"2023-10-16T11:20:26.082473+0000","last_became_peered":"2023-10-16T11:20:26.082473+0000","last_unstale":"2023-10-16T11:20:38.138681+0000","last_undegraded":"2023-10-16T11:20:38.138681+0000","last_fullsized":"2023-10-16T11:20:38.138681+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:39.122257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"236'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.142679+0000","last_change":"2023-10-16T11:20:25.085721+0000","last_active":"2023-10-16T11:20:38.142679+0000","last_peered":"2023-10-16T11:20:38.142679+0000","last_clean":"2023-10-16T11:20:38.142679+0000","last_became_active":"2023-10-16T11:20:25.085421+0000","last_became_peered":"2023-10-16T11:20:25.085421+0000","last_unstale":"2023-10-16T11:20:38.142679+0000","last_undegraded":"2023-10-16T11:20:38.142679+0000","last_fullsized":"2023-10-16T11:20:38.142679+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:08.936039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.163645+0000","last_change":"2023-10-16T11:20:39.159812+0000","last_active":"2023-10-16T11:20:39.163645+0000","last_peered":"2023-10-16T11:20:39.163645+0000","last_clean":"2023-10-16T11:20:39.163645+0000","last_became_active":"2023-10-16T11:20:39.159542+0000","last_became_peered":"2023-10-16T11:20:39.159542+0000","last_unstale":"2023-10-16T11:20:39.163645+0000","last_undegraded":"2023-10-16T11:20:39.163645+0000","last_fullsized":"2023-10-16T11:20:39.163645+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:17.101688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134870+0000","last_change":"2023-10-16T11:20:26.098738+0000","last_active":"2023-10-16T11:20:38.134870+0000","last_peered":"2023-10-16T11:20:38.134870+0000","last_clean":"2023-10-16T11:20:38.134870+0000","last_became_active":"2023-10-16T11:20:26.098226+0000","last_became_peered":"2023-10-16T11:20:26.098226+0000","last_unstale":"2023-10-16T11:20:38.134870+0000","last_undegraded":"2023-10-16T11:20:38.134870+0000","last_fullsized":"2023-10-16T11:20:38.134870+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:48.218074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140917+0000","last_change":"2023-10-16T11:20:25.081249+0000","last_active":"2023-10-16T11:20:38.140917+0000","last_peered":"2023-10-16T11:20:38.140917+0000","last_clean":"2023-10-16T11:20:38.140917+0000","last_became_active":"2023-10-16T11:20:25.080938+0000","last_became_peered":"2023-10-16T11:20:25.080938+0000","last_unstale":"2023-10-16T11:20:38.140917+0000","last_undegraded":"2023-10-16T11:20:38.140917+0000","last_fullsized":"2023-10-16T11:20:38.140917+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:09.538322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.151134+0000","last_change":"2023-10-16T11:20:39.147468+0000","last_active":"2023-10-16T11:20:39.151134+0000","last_peered":"2023-10-16T11:20:39.151134+0000","last_clean":"2023-10-16T11:20:39.151134+0000","last_became_active":"2023-10-16T11:20:39.147172+0000","last_became_peered":"2023-10-16T11:20:39.147172+0000","last_unstale":"2023-10-16T11:20:39.151134+0000","last_undegraded":"2023-10-16T11:20:39.151134+0000","last_fullsized":"2023-10-16T11:20:39.151134+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:51.980238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135227+0000","last_change":"2023-10-16T11:20:26.094947+0000","last_active":"2023-10-16T11:20:38.135227+0000","last_peered":"2023-10-16T11:20:38.135227+0000","last_clean":"2023-10-16T11:20:38.135227+0000","last_became_active":"2023-10-16T11:20:26.094613+0000","last_became_peered":"2023-10-16T11:20:26.094613+0000","last_unstale":"2023-10-16T11:20:38.135227+0000","last_undegraded":"2023-10-16T11:20:38.135227+0000","last_fullsized":"2023-10-16T11:20:38.135227+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:24.098521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135083+0000","last_change":"2023-10-16T11:20:25.082461+0000","last_active":"2023-10-16T11:20:38.135083+0000","last_peered":"2023-10-16T11:20:38.135083+0000","last_clean":"2023-10-16T11:20:38.135083+0000","last_became_active":"2023-10-16T11:20:25.082161+0000","last_became_peered":"2023-10-16T11:20:25.082161+0000","last_unstale":"2023-10-16T11:20:38.135083+0000","last_undegraded":"2023-10-16T11:20:38.135083+0000","last_fullsized":"2023-10-16T11:20:38.135083+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:13.522561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157468+0000","last_change":"2023-10-16T11:20:39.156073+0000","last_active":"2023-10-16T11:20:39.157468+0000","last_peered":"2023-10-16T11:20:39.157468+0000","last_clean":"2023-10-16T11:20:39.157468+0000","last_became_active":"2023-10-16T11:20:39.155796+0000","last_became_peered":"2023-10-16T11:20:39.155796+0000","last_unstale":"2023-10-16T11:20:39.157468+0000","last_undegraded":"2023-10-16T11:20:39.157468+0000","last_fullsized":"2023-10-16T11:20:39.157468+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:57.213305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138452+0000","last_change":"2023-10-16T11:20:26.083434+0000","last_active":"2023-10-16T11:20:38.138452+0000","last_peered":"2023-10-16T11:20:38.138452+0000","last_clean":"2023-10-16T11:20:38.138452+0000","last_became_active":"2023-10-16T11:20:26.083146+0000","last_became_peered":"2023-10-16T11:20:26.083146+0000","last_unstale":"2023-10-16T11:20:38.138452+0000","last_undegraded":"2023-10-16T11:20:38.138452+0000","last_fullsized":"2023-10-16T11:20:38.138452+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:36.022948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138103+0000","last_change":"2023-10-16T11:20:25.084296+0000","last_active":"2023-10-16T11:20:38.138103+0000","last_peered":"2023-10-16T11:20:38.138103+0000","last_clean":"2023-10-16T11:20:38.138103+0000","last_became_active":"2023-10-16T11:20:25.083831+0000","last_became_peered":"2023-10-16T11:20:25.083831+0000","last_unstale":"2023-10-16T11:20:38.138103+0000","last_undegraded":"2023-10-16T11:20:38.138103+0000","last_fullsized":"2023-10-16T11:20:38.138103+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:15.128952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156567+0000","last_change":"2023-10-16T11:20:39.152971+0000","last_active":"2023-10-16T11:20:39.156567+0000","last_peered":"2023-10-16T11:20:39.156567+0000","last_clean":"2023-10-16T11:20:39.156567+0000","last_became_active":"2023-10-16T11:20:39.152647+0000","last_became_peered":"2023-10-16T11:20:39.152647+0000","last_unstale":"2023-10-16T11:20:39.156567+0000","last_undegraded":"2023-10-16T11:20:39.156567+0000","last_fullsized":"2023-10-16T11:20:39.156567+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:17.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147685+0000","last_change":"2023-10-16T11:20:26.097299+0000","last_active":"2023-10-16T11:20:38.147685+0000","last_peered":"2023-10-16T11:20:38.147685+0000","last_clean":"2023-10-16T11:20:38.147685+0000","last_became_active":"2023-10-16T11:20:26.096992+0000","last_became_peered":"2023-10-16T11:20:26.096992+0000","last_unstale":"2023-10-16T11:20:38.147685+0000","last_undegraded":"2023-10-16T11:20:38.147685+0000","last_fullsized":"2023-10-16T11:20:38.147685+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:01.203662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143932+0000","last_change":"2023-10-16T11:20:25.085431+0000","last_active":"2023-10-16T11:20:38.143932+0000","last_peered":"2023-10-16T11:20:38.143932+0000","last_clean":"2023-10-16T11:20:38.143932+0000","last_became_active":"2023-10-16T11:20:25.084034+0000","last_became_peered":"2023-10-16T11:20:25.084034+0000","last_unstale":"2023-10-16T11:20:38.143932+0000","last_undegraded":"2023-10-16T11:20:38.143932+0000","last_fullsized":"2023-10-16T11:20:38.143932+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:44.288602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162716+0000","last_change":"2023-10-16T11:20:39.160977+0000","last_active":"2023-10-16T11:20:39.162716+0000","last_peered":"2023-10-16T11:20:39.162716+0000","last_clean":"2023-10-16T11:20:39.162716+0000","last_became_active":"2023-10-16T11:20:39.160680+0000","last_became_peered":"2023-10-16T11:20:39.160680+0000","last_unstale":"2023-10-16T11:20:39.162716+0000","last_undegraded":"2023-10-16T11:20:39.162716+0000","last_fullsized":"2023-10-16T11:20:39.162716+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.949930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147036+0000","last_change":"2023-10-16T11:20:26.082969+0000","last_active":"2023-10-16T11:20:38.147036+0000","last_peered":"2023-10-16T11:20:38.147036+0000","last_clean":"2023-10-16T11:20:38.147036+0000","last_became_active":"2023-10-16T11:20:26.082661+0000","last_became_peered":"2023-10-16T11:20:26.082661+0000","last_unstale":"2023-10-16T11:20:38.147036+0000","last_undegraded":"2023-10-16T11:20:38.147036+0000","last_fullsized":"2023-10-16T11:20:38.147036+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:53.296236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136888+0000","last_change":"2023-10-16T11:20:25.080407+0000","last_active":"2023-10-16T11:20:38.136888+0000","last_peered":"2023-10-16T11:20:38.136888+0000","last_clean":"2023-10-16T11:20:38.136888+0000","last_became_active":"2023-10-16T11:20:25.079543+0000","last_became_peered":"2023-10-16T11:20:25.079543+0000","last_unstale":"2023-10-16T11:20:38.136888+0000","last_undegraded":"2023-10-16T11:20:38.136888+0000","last_fullsized":"2023-10-16T11:20:38.136888+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:12.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162068+0000","last_change":"2023-10-16T11:20:39.159286+0000","last_active":"2023-10-16T11:20:39.162068+0000","last_peered":"2023-10-16T11:20:39.162068+0000","last_clean":"2023-10-16T11:20:39.162068+0000","last_became_active":"2023-10-16T11:20:39.159009+0000","last_became_peered":"2023-10-16T11:20:39.159009+0000","last_unstale":"2023-10-16T11:20:39.162068+0000","last_undegraded":"2023-10-16T11:20:39.162068+0000","last_fullsized":"2023-10-16T11:20:39.162068+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:54.326272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138081+0000","last_change":"2023-10-16T11:20:26.086262+0000","last_active":"2023-10-16T11:20:38.138081+0000","last_peered":"2023-10-16T11:20:38.138081+0000","last_clean":"2023-10-16T11:20:38.138081+0000","last_became_active":"2023-10-16T11:20:26.085901+0000","last_became_peered":"2023-10-16T11:20:26.085901+0000","last_unstale":"2023-10-16T11:20:38.138081+0000","last_undegraded":"2023-10-16T11:20:38.138081+0000","last_fullsized":"2023-10-16T11:20:38.138081+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:56.664462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137098+0000","last_change":"2023-10-16T11:20:25.086844+0000","last_active":"2023-10-16T11:20:38.137098+0000","last_peered":"2023-10-16T11:20:38.137098+0000","last_clean":"2023-10-16T11:20:38.137098+0000","last_became_active":"2023-10-16T11:20:25.086570+0000","last_became_peered":"2023-10-16T11:20:25.086570+0000","last_unstale":"2023-10-16T11:20:38.137098+0000","last_undegraded":"2023-10-16T11:20:38.137098+0000","last_fullsized":"2023-10-16T11:20:38.137098+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:17.084881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161016+0000","last_change":"2023-10-16T11:20:39.155334+0000","last_active":"2023-10-16T11:20:39.161016+0000","last_peered":"2023-10-16T11:20:39.161016+0000","last_clean":"2023-10-16T11:20:39.161016+0000","last_became_active":"2023-10-16T11:20:39.155077+0000","last_became_peered":"2023-10-16T11:20:39.155077+0000","last_unstale":"2023-10-16T11:20:39.161016+0000","last_undegraded":"2023-10-16T11:20:39.161016+0000","last_fullsized":"2023-10-16T11:20:39.161016+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:29.099178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.142552+0000","last_change":"2023-10-16T11:20:26.078189+0000","last_active":"2023-10-16T11:20:38.142552+0000","last_peered":"2023-10-16T11:20:38.142552+0000","last_clean":"2023-10-16T11:20:38.142552+0000","last_became_active":"2023-10-16T11:20:26.077754+0000","last_became_peered":"2023-10-16T11:20:26.077754+0000","last_unstale":"2023-10-16T11:20:38.142552+0000","last_undegraded":"2023-10-16T11:20:38.142552+0000","last_fullsized":"2023-10-16T11:20:38.142552+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:45.296590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"235'4","reported_seq":19,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148521+0000","last_change":"2023-10-16T11:20:25.087446+0000","last_active":"2023-10-16T11:20:38.148521+0000","last_peered":"2023-10-16T11:20:38.148521+0000","last_clean":"2023-10-16T11:20:38.148521+0000","last_became_active":"2023-10-16T11:20:25.086617+0000","last_became_peered":"2023-10-16T11:20:25.086617+0000","last_unstale":"2023-10-16T11:20:38.148521+0000","last_undegraded":"2023-10-16T11:20:38.148521+0000","last_fullsized":"2023-10-16T11:20:38.148521+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:50.072765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'370","reported_seq":1311,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140831+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:20:38.140831+0000","last_peered":"2023-10-16T11:20:38.140831+0000","last_clean":"2023-10-16T11:20:38.140831+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:20:38.140831+0000","last_undegraded":"2023-10-16T11:20:38.140831+0000","last_fullsized":"2023-10-16T11:20:38.140831+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":3657,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543292,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52760,"kb_used_data":3524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319080,"statfs":{"total":96636764160,"available":96582737920,"internally_reserved":0,"allocated":3608576,"data_stored":2147036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.85699999999999998,"5min":0.878,"15min":0.878},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.27800000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.33300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.83399999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69799999999999995}]},{"osd":3,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":1,"5min":1,"15min":1.002},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.33100000000000002},"max":{"1min":0.93500000000000005,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.71499999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67000000000000004}]}]},{"osd":6,"up_from":14,"seq":60129543291,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56340,"kb_used_data":2952,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315500,"statfs":{"total":96636764160,"available":96579072000,"internally_reserved":0,"allocated":3022848,"data_stored":1557244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29899999999999999},"max":{"1min":0.66700000000000004,"5min":0.81699999999999995,"15min":0.84499999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83099999999999996,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.48899999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.42599999999999999,"5min":0.40799999999999997,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68799999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.371},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76300000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.41699999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.373,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.502},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51600000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54300000000000004}]}]},{"osd":1,"up_from":14,"seq":60129543291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60496,"kb_used_data":2944,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311344,"statfs":{"total":96636764160,"available":96574816256,"internally_reserved":0,"allocated":3014656,"data_stored":1551314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39700000000000002,"15min":0.40600000000000003},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51800000000000002,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.40600000000000003,"15min":0.43099999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.49199999999999999,"5min":0.62,"15min":0.91700000000000004},"last":0.371}]},{"osd":2,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40200000000000002,"15min":0.41699999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45000000000000001,"5min":0.58599999999999997,"15min":0.98899999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.441,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63200000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.34999999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46400000000000002,"15min":0.49199999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.78300000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.57999999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77200000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85199999999999998,"5min":1.0289999999999999,"15min":1.097},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85499999999999998,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.90100000000000002,"15min":1.147},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.98699999999999999,"15min":1.0509999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.70299999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":1.0900000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":1.004,"15min":1.0600000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129543292,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61060,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310780,"statfs":{"total":96636764160,"available":96574238720,"internally_reserved":0,"allocated":3592192,"data_stored":2138733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.22,"15min":0.214},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.39200000000000002,"15min":0.41099999999999998},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.65100000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.70299999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.42399999999999999,"15min":0.44400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.374}]},{"osd":3,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42499999999999999,"15min":0.44800000000000001},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.47399999999999998,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.84599999999999997},"last":0.441}]},{"osd":4,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66000000000000003,"15min":0.66700000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.621},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":1.008},"last":0.501}]},{"osd":5,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79900000000000004,"5min":1.004,"15min":1.008},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81999999999999995,"5min":0.94199999999999995,"15min":1.1419999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65100000000000002,"15min":0.68100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79000000000000004,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65600000000000003,"15min":0.68400000000000005},"min":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74099999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52500000000000002}]}]},{"osd":2,"up_from":14,"seq":60129543291,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48000,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323840,"statfs":{"total":96636764160,"available":96587612160,"internally_reserved":0,"allocated":2998272,"data_stored":1548421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.40500000000000003},"min":{"1min":0.24399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.28499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.436},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.4929999999999999,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.28199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.754}]},{"osd":5,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.6779999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.6549999999999998,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.69499999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.7320000000000002,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.59899999999999998}]}]},{"osd":3,"up_from":14,"seq":60129543292,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48032,"kb_used_data":2956,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323808,"statfs":{"total":96636764160,"available":96587579392,"internally_reserved":0,"allocated":3026944,"data_stored":1559304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.59799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.52600000000000002,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.308}]},{"osd":1,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.40600000000000003,"15min":0.42499999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58099999999999996,"5min":0.87,"15min":0.87},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54800000000000004,"5min":0.65100000000000002,"15min":0.76300000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.44400000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.49099999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.46400000000000002,"15min":0.48299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56100000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.65300000000000002},"min":{"1min":0.39900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.59199999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65100000000000002,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.66400000000000003},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.623},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96399999999999997,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.63500000000000001}]}]},{"osd":4,"up_from":14,"seq":60129543291,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78480,"kb_used_data":2944,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94293360,"statfs":{"total":96636764160,"available":96556400640,"internally_reserved":0,"allocated":3014656,"data_stored":1551282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.74399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60399999999999998,"15min":0.61899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71899999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52100000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.83399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.628,"15min":0.63400000000000001},"min":{"1min":0.5,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":0.745,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.879},"last":0.55100000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.621},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.57499999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.875,"15min":0.875},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.73499999999999999,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.66900000000000004}]}]},{"osd":5,"up_from":14,"seq":60129543292,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3006464,"data_stored":1556528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.437},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":1.873,"15min":1.873},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.41999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.373,"15min":0.373},"max":{"1min":0.745,"5min":1.024,"15min":1.024},"last":0.376}]},{"osd":3,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.626,"15min":0.628},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73999999999999999,"5min":1.821,"15min":1.821},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79900000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.503}]},{"osd":4,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.40200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":1.919,"15min":1.919},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.61799999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94299999999999995,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.96899999999999997,"5min":2.02,"15min":2.02},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:20:43.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:20:43.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.415+0000 7fd607137700 1 Processor -- start 2023-10-16T11:20:43.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd607137700 1 -- start start 2023-10-16T11:20:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd607137700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd6000ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd607137700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6000cd2d0 0x7fd6000d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd607137700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6000cfc60 0x7fd6000d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd607137700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6000d0630 con 0x7fd6000cfc60 2023-10-16T11:20:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd607137700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6000d24f0 con 0x7fd6000cd2d0 2023-10-16T11:20:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd607137700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000cc510 con 0x7fd60015b990 2023-10-16T11:20:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.416+0000 7fd6056d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6000cfc60 0x7fd6000d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6000cfc60 0x7fd6000d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58164/0 (socket says 172.21.15.181:58164) 2023-10-16T11:20:43.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 -- 172.21.15.181:0/1809578337 learned_addr learned my addr 172.21.15.181:0/1809578337 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:43.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd604ed3700 1 --2- 172.21.15.181:0/1809578337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd6000ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd5fffff700 1 --2- 172.21.15.181:0/1809578337 >> v2:172.21.15.190:3300/0 conn(0x7fd6000cd2d0 0x7fd6000d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 -- 172.21.15.181:0/1809578337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 msgr2=0x7fd6000ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:43.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 --2- 172.21.15.181:0/1809578337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd6000ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 -- 172.21.15.181:0/1809578337 >> v2:172.21.15.190:3300/0 conn(0x7fd6000cd2d0 msgr2=0x7fd6000d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:43.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 --2- 172.21.15.181:0/1809578337 >> v2:172.21.15.190:3300/0 conn(0x7fd6000cd2d0 0x7fd6000d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 -- 172.21.15.181:0/1809578337 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6000cc7f0 con 0x7fd6000cfc60 2023-10-16T11:20:43.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd5fffff700 1 --2- 172.21.15.181:0/1809578337 >> v2:172.21.15.190:3300/0 conn(0x7fd6000cd2d0 0x7fd6000d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:43.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd6056d4700 1 --2- 172.21.15.181:0/1809578337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cfc60 0x7fd6000d00f0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fd60008bec0 tx=0x7fd5f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72935203818f3d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd5ff7fe700 1 -- 172.21.15.181:0/1809578337 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800e070 con 0x7fd6000cfc60 2023-10-16T11:20:43.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd5ff7fe700 1 -- 172.21.15.181:0/1809578337 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f800b9c0 con 0x7fd6000cfc60 2023-10-16T11:20:43.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.417+0000 7fd5ff7fe700 1 -- 172.21.15.181:0/1809578337 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800bd70 con 0x7fd6000cfc60 2023-10-16T11:20:43.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 -- 172.21.15.181:0/1809578337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cfc60 msgr2=0x7fd6000d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 --2- 172.21.15.181:0/1809578337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cfc60 0x7fd6000d00f0 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fd60008bec0 tx=0x7fd5f800b330 comp rx=0 tx=0).stop 2023-10-16T11:20:43.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 -- 172.21.15.181:0/1809578337 shutdown_connections 2023-10-16T11:20:43.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 --2- 172.21.15.181:0/1809578337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cfc60 0x7fd6000d00f0 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 --2- 172.21.15.181:0/1809578337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd6000ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 --2- 172.21.15.181:0/1809578337 >> v2:172.21.15.190:3300/0 conn(0x7fd6000cd2d0 0x7fd6000d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 -- 172.21.15.181:0/1809578337 >> 172.21.15.181:0/1809578337 conn(0x7fd60009a9a0 msgr2=0x7fd6000d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:43.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 -- 172.21.15.181:0/1809578337 shutdown_connections 2023-10-16T11:20:43.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.418+0000 7fd607137700 1 -- 172.21.15.181:0/1809578337 wait complete. 2023-10-16T11:20:43.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.419+0000 7fd607137700 1 Processor -- start 2023-10-16T11:20:43.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.419+0000 7fd607137700 1 -- start start 2023-10-16T11:20:43.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd607137700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cd2d0 0x7fd6000c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd607137700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6000cfc60 0x7fd60015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd607137700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd60015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd607137700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001f45c0 con 0x7fd6000cd2d0 2023-10-16T11:20:43.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd607137700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60015d370 con 0x7fd6000cfc60 2023-10-16T11:20:43.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd607137700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60006d080 con 0x7fd60015b990 2023-10-16T11:20:43.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cd2d0 0x7fd6000c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cd2d0 0x7fd6000c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58166/0 (socket says 172.21.15.181:58166) 2023-10-16T11:20:43.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 -- 172.21.15.181:0/2216112051 learned_addr learned my addr 172.21.15.181:0/2216112051 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:43.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 -- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 msgr2=0x7fd60015acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd5fffff700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6000cfc60 0x7fd60015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd6056d4700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd60015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd60015acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 -- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6000cfc60 msgr2=0x7fd60015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6000cfc60 0x7fd60015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 -- 172.21.15.181:0/2216112051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8009b30 con 0x7fd6000cd2d0 2023-10-16T11:20:43.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd5fffff700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6000cfc60 0x7fd60015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:43.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd604ed3700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cd2d0 0x7fd6000c9470 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400d7d0 tx=0x7fd5f400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd5fd7fa700 1 -- 172.21.15.181:0/2216112051 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40125f0 con 0x7fd6000cd2d0 2023-10-16T11:20:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd5fd7fa700 1 -- 172.21.15.181:0/2216112051 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4012750 con 0x7fd6000cd2d0 2023-10-16T11:20:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60015d610 con 0x7fd6000cd2d0 2023-10-16T11:20:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.420+0000 7fd5fd7fa700 1 -- 172.21.15.181:0/2216112051 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4012b00 con 0x7fd6000cd2d0 2023-10-16T11:20:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.421+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60006d360 con 0x7fd6000cd2d0 2023-10-16T11:20:43.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.421+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60015db30 con 0x7fd6000cd2d0 2023-10-16T11:20:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.422+0000 7fd5fd7fa700 1 -- 172.21.15.181:0/2216112051 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd5f4012c60 con 0x7fd6000cd2d0 2023-10-16T11:20:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.422+0000 7fd5fd7fa700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e406b140 0x7fd5e406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.423+0000 7fd5fffff700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e406b140 0x7fd5e406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.423+0000 7fd5fd7fa700 1 -- 172.21.15.181:0/2216112051 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fd5f408a000 con 0x7fd6000cd2d0 2023-10-16T11:20:43.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.423+0000 7fd5fffff700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e406b140 0x7fd5e406d600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd60009f5b0 tx=0x7fd5ec008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.427+0000 7fd5fd7fa700 1 -- 172.21.15.181:0/2216112051 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd60015db30 con 0x7fd6000cd2d0 2023-10-16T11:20:43.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.615+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 --> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6000ccdb0 con 0x7fd5e406b140 2023-10-16T11:20:43.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.628+0000 7fd5fd7fa700 1 -- 172.21.15.181:0/2216112051 <== mgr.11814 v2:172.21.15.181:6832/3273985848 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529066 (secure 0 0 0) 0x7fd5ec00c040 con 0x7fd5e406b140 2023-10-16T11:20:43.634 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:43.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e406b140 msgr2=0x7fd5e406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e406b140 0x7fd5e406d600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd60009f5b0 tx=0x7fd5ec008040 comp rx=0 tx=0).stop 2023-10-16T11:20:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cd2d0 msgr2=0x7fd6000c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cd2d0 0x7fd6000c9470 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400d7d0 tx=0x7fd5f400db90 comp rx=0 tx=0).stop 2023-10-16T11:20:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 shutdown_connections 2023-10-16T11:20:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd5e406b140 0x7fd5e406d600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd60015b990 0x7fd60015acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6000cd2d0 0x7fd6000c9470 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 --2- 172.21.15.181:0/2216112051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6000cfc60 0x7fd60015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 >> 172.21.15.181:0/2216112051 conn(0x7fd60009a9a0 msgr2=0x7fd60006fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:43.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 shutdown_connections 2023-10-16T11:20:43.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.632+0000 7fd607137700 1 -- 172.21.15.181:0/2216112051 wait complete. 2023-10-16T11:20:43.642 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:20:43.645 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-16T11:20:41.757610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596435,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":710,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":449,"ondisk_log_size":449,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":273,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465660,"kb_used_data":24692,"kb_used_omap":12,"kb_used_meta":440819,"kb_avail":754509060,"statfs":{"total":773094113280,"available":772617277440,"internally_reserved":0,"allocated":25284608,"data_stored":13609862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":451399224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.631342"},"pg_stats":[{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161677+0000","last_change":"2023-10-16T11:20:39.160588+0000","last_active":"2023-10-16T11:20:39.161677+0000","last_peered":"2023-10-16T11:20:39.161677+0000","last_clean":"2023-10-16T11:20:39.161677+0000","last_became_active":"2023-10-16T11:20:39.160284+0000","last_became_peered":"2023-10-16T11:20:39.160284+0000","last_unstale":"2023-10-16T11:20:39.161677+0000","last_undegraded":"2023-10-16T11:20:39.161677+0000","last_fullsized":"2023-10-16T11:20:39.161677+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:29.209829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.142502+0000","last_change":"2023-10-16T11:20:38.151690+0000","last_active":"2023-10-16T11:20:39.142502+0000","last_peered":"2023-10-16T11:20:39.142502+0000","last_clean":"2023-10-16T11:20:39.142502+0000","last_became_active":"2023-10-16T11:20:38.151420+0000","last_became_peered":"2023-10-16T11:20:38.151420+0000","last_unstale":"2023-10-16T11:20:39.142502+0000","last_undegraded":"2023-10-16T11:20:39.142502+0000","last_fullsized":"2023-10-16T11:20:39.142502+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:26:27.292365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138584+0000","last_change":"2023-10-16T11:20:26.092187+0000","last_active":"2023-10-16T11:20:38.138584+0000","last_peered":"2023-10-16T11:20:38.138584+0000","last_clean":"2023-10-16T11:20:38.138584+0000","last_became_active":"2023-10-16T11:20:26.091884+0000","last_became_peered":"2023-10-16T11:20:26.091884+0000","last_unstale":"2023-10-16T11:20:38.138584+0000","last_undegraded":"2023-10-16T11:20:38.138584+0000","last_fullsized":"2023-10-16T11:20:38.138584+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:07:16.851198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164076+0000","last_change":"2023-10-16T11:20:39.158941+0000","last_active":"2023-10-16T11:20:39.164076+0000","last_peered":"2023-10-16T11:20:39.164076+0000","last_clean":"2023-10-16T11:20:39.164076+0000","last_became_active":"2023-10-16T11:20:39.158259+0000","last_became_peered":"2023-10-16T11:20:39.158259+0000","last_unstale":"2023-10-16T11:20:39.164076+0000","last_undegraded":"2023-10-16T11:20:39.164076+0000","last_fullsized":"2023-10-16T11:20:39.164076+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:19.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.146698+0000","last_change":"2023-10-16T11:20:38.147489+0000","last_active":"2023-10-16T11:20:39.146698+0000","last_peered":"2023-10-16T11:20:39.146698+0000","last_clean":"2023-10-16T11:20:39.146698+0000","last_became_active":"2023-10-16T11:20:38.147047+0000","last_became_peered":"2023-10-16T11:20:38.147047+0000","last_unstale":"2023-10-16T11:20:39.146698+0000","last_undegraded":"2023-10-16T11:20:39.146698+0000","last_fullsized":"2023-10-16T11:20:39.146698+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:38.993484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137790+0000","last_change":"2023-10-16T11:20:26.085301+0000","last_active":"2023-10-16T11:20:38.137790+0000","last_peered":"2023-10-16T11:20:38.137790+0000","last_clean":"2023-10-16T11:20:38.137790+0000","last_became_active":"2023-10-16T11:20:26.084922+0000","last_became_peered":"2023-10-16T11:20:26.084922+0000","last_unstale":"2023-10-16T11:20:38.137790+0000","last_undegraded":"2023-10-16T11:20:38.137790+0000","last_fullsized":"2023-10-16T11:20:38.137790+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:36.074430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.154275+0000","last_change":"2023-10-16T11:20:39.148752+0000","last_active":"2023-10-16T11:20:39.154275+0000","last_peered":"2023-10-16T11:20:39.154275+0000","last_clean":"2023-10-16T11:20:39.154275+0000","last_became_active":"2023-10-16T11:20:39.148478+0000","last_became_peered":"2023-10-16T11:20:39.148478+0000","last_unstale":"2023-10-16T11:20:39.154275+0000","last_undegraded":"2023-10-16T11:20:39.154275+0000","last_fullsized":"2023-10-16T11:20:39.154275+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:17.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.136470+0000","last_change":"2023-10-16T11:20:38.143454+0000","last_active":"2023-10-16T11:20:39.136470+0000","last_peered":"2023-10-16T11:20:39.136470+0000","last_clean":"2023-10-16T11:20:39.136470+0000","last_became_active":"2023-10-16T11:20:38.143171+0000","last_became_peered":"2023-10-16T11:20:38.143171+0000","last_unstale":"2023-10-16T11:20:39.136470+0000","last_undegraded":"2023-10-16T11:20:39.136470+0000","last_fullsized":"2023-10-16T11:20:39.136470+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:08.172420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139386+0000","last_change":"2023-10-16T11:20:26.083255+0000","last_active":"2023-10-16T11:20:38.139386+0000","last_peered":"2023-10-16T11:20:38.139386+0000","last_clean":"2023-10-16T11:20:38.139386+0000","last_became_active":"2023-10-16T11:20:26.082915+0000","last_became_peered":"2023-10-16T11:20:26.082915+0000","last_unstale":"2023-10-16T11:20:38.139386+0000","last_undegraded":"2023-10-16T11:20:38.139386+0000","last_fullsized":"2023-10-16T11:20:38.139386+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:43:02.635859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155981+0000","last_change":"2023-10-16T11:20:39.147597+0000","last_active":"2023-10-16T11:20:39.155981+0000","last_peered":"2023-10-16T11:20:39.155981+0000","last_clean":"2023-10-16T11:20:39.155981+0000","last_became_active":"2023-10-16T11:20:39.147315+0000","last_became_peered":"2023-10-16T11:20:39.147315+0000","last_unstale":"2023-10-16T11:20:39.155981+0000","last_undegraded":"2023-10-16T11:20:39.155981+0000","last_fullsized":"2023-10-16T11:20:39.155981+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:07.186956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.135958+0000","last_change":"2023-10-16T11:20:38.149234+0000","last_active":"2023-10-16T11:20:39.135958+0000","last_peered":"2023-10-16T11:20:39.135958+0000","last_clean":"2023-10-16T11:20:39.135958+0000","last_became_active":"2023-10-16T11:20:38.148942+0000","last_became_peered":"2023-10-16T11:20:38.148942+0000","last_unstale":"2023-10-16T11:20:39.135958+0000","last_undegraded":"2023-10-16T11:20:39.135958+0000","last_fullsized":"2023-10-16T11:20:39.135958+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:52.920256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145086+0000","last_change":"2023-10-16T11:20:26.084494+0000","last_active":"2023-10-16T11:20:38.145086+0000","last_peered":"2023-10-16T11:20:38.145086+0000","last_clean":"2023-10-16T11:20:38.145086+0000","last_became_active":"2023-10-16T11:20:26.084169+0000","last_became_peered":"2023-10-16T11:20:26.084169+0000","last_unstale":"2023-10-16T11:20:38.145086+0000","last_undegraded":"2023-10-16T11:20:38.145086+0000","last_fullsized":"2023-10-16T11:20:38.145086+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156251+0000","last_change":"2023-10-16T11:20:39.151439+0000","last_active":"2023-10-16T11:20:39.156251+0000","last_peered":"2023-10-16T11:20:39.156251+0000","last_clean":"2023-10-16T11:20:39.156251+0000","last_became_active":"2023-10-16T11:20:39.151122+0000","last_became_peered":"2023-10-16T11:20:39.151122+0000","last_unstale":"2023-10-16T11:20:39.156251+0000","last_undegraded":"2023-10-16T11:20:39.156251+0000","last_fullsized":"2023-10-16T11:20:39.156251+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.142220+0000","last_change":"2023-10-16T11:20:38.146065+0000","last_active":"2023-10-16T11:20:39.142220+0000","last_peered":"2023-10-16T11:20:39.142220+0000","last_clean":"2023-10-16T11:20:39.142220+0000","last_became_active":"2023-10-16T11:20:38.145791+0000","last_became_peered":"2023-10-16T11:20:38.145791+0000","last_unstale":"2023-10-16T11:20:39.142220+0000","last_undegraded":"2023-10-16T11:20:39.142220+0000","last_fullsized":"2023-10-16T11:20:39.142220+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:50.313696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140695+0000","last_change":"2023-10-16T11:20:26.080175+0000","last_active":"2023-10-16T11:20:38.140695+0000","last_peered":"2023-10-16T11:20:38.140695+0000","last_clean":"2023-10-16T11:20:38.140695+0000","last_became_active":"2023-10-16T11:20:26.079884+0000","last_became_peered":"2023-10-16T11:20:26.079884+0000","last_unstale":"2023-10-16T11:20:38.140695+0000","last_undegraded":"2023-10-16T11:20:38.140695+0000","last_fullsized":"2023-10-16T11:20:38.140695+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:58.169049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.147776+0000","last_change":"2023-10-16T11:20:39.143971+0000","last_active":"2023-10-16T11:20:39.147776+0000","last_peered":"2023-10-16T11:20:39.147776+0000","last_clean":"2023-10-16T11:20:39.147776+0000","last_became_active":"2023-10-16T11:20:39.143672+0000","last_became_peered":"2023-10-16T11:20:39.143672+0000","last_unstale":"2023-10-16T11:20:39.147776+0000","last_undegraded":"2023-10-16T11:20:39.147776+0000","last_fullsized":"2023-10-16T11:20:39.147776+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:01.290963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155136+0000","last_change":"2023-10-16T11:20:38.148647+0000","last_active":"2023-10-16T11:20:39.155136+0000","last_peered":"2023-10-16T11:20:39.155136+0000","last_clean":"2023-10-16T11:20:39.155136+0000","last_became_active":"2023-10-16T11:20:38.148311+0000","last_became_peered":"2023-10-16T11:20:38.148311+0000","last_unstale":"2023-10-16T11:20:39.155136+0000","last_undegraded":"2023-10-16T11:20:39.155136+0000","last_fullsized":"2023-10-16T11:20:39.155136+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:55:02.128796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143834+0000","last_change":"2023-10-16T11:20:26.099739+0000","last_active":"2023-10-16T11:20:38.143834+0000","last_peered":"2023-10-16T11:20:38.143834+0000","last_clean":"2023-10-16T11:20:38.143834+0000","last_became_active":"2023-10-16T11:20:26.099459+0000","last_became_peered":"2023-10-16T11:20:26.099459+0000","last_unstale":"2023-10-16T11:20:38.143834+0000","last_undegraded":"2023-10-16T11:20:38.143834+0000","last_fullsized":"2023-10-16T11:20:38.143834+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:14.530550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164978+0000","last_change":"2023-10-16T11:20:39.160298+0000","last_active":"2023-10-16T11:20:39.164978+0000","last_peered":"2023-10-16T11:20:39.164978+0000","last_clean":"2023-10-16T11:20:39.164978+0000","last_became_active":"2023-10-16T11:20:39.159862+0000","last_became_peered":"2023-10-16T11:20:39.159862+0000","last_unstale":"2023-10-16T11:20:39.164978+0000","last_undegraded":"2023-10-16T11:20:39.164978+0000","last_fullsized":"2023-10-16T11:20:39.164978+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:26:57.413653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.140110+0000","last_change":"2023-10-16T11:20:38.155852+0000","last_active":"2023-10-16T11:20:39.140110+0000","last_peered":"2023-10-16T11:20:39.140110+0000","last_clean":"2023-10-16T11:20:39.140110+0000","last_became_active":"2023-10-16T11:20:38.155607+0000","last_became_peered":"2023-10-16T11:20:38.155607+0000","last_unstale":"2023-10-16T11:20:39.140110+0000","last_undegraded":"2023-10-16T11:20:39.140110+0000","last_fullsized":"2023-10-16T11:20:39.140110+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:07.405065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137992+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:38.137992+0000","last_peered":"2023-10-16T11:20:38.137992+0000","last_clean":"2023-10-16T11:20:38.137992+0000","last_became_active":"2023-10-16T11:20:26.098864+0000","last_became_peered":"2023-10-16T11:20:26.098864+0000","last_unstale":"2023-10-16T11:20:38.137992+0000","last_undegraded":"2023-10-16T11:20:38.137992+0000","last_fullsized":"2023-10-16T11:20:38.137992+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:04.128044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159032+0000","last_change":"2023-10-16T11:20:39.150940+0000","last_active":"2023-10-16T11:20:39.159032+0000","last_peered":"2023-10-16T11:20:39.159032+0000","last_clean":"2023-10-16T11:20:39.159032+0000","last_became_active":"2023-10-16T11:20:39.150672+0000","last_became_peered":"2023-10-16T11:20:39.150672+0000","last_unstale":"2023-10-16T11:20:39.159032+0000","last_undegraded":"2023-10-16T11:20:39.159032+0000","last_fullsized":"2023-10-16T11:20:39.159032+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:37.473407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164777+0000","last_change":"2023-10-16T11:20:38.147703+0000","last_active":"2023-10-16T11:20:39.164777+0000","last_peered":"2023-10-16T11:20:39.164777+0000","last_clean":"2023-10-16T11:20:39.164777+0000","last_became_active":"2023-10-16T11:20:38.147409+0000","last_became_peered":"2023-10-16T11:20:38.147409+0000","last_unstale":"2023-10-16T11:20:39.164777+0000","last_undegraded":"2023-10-16T11:20:39.164777+0000","last_fullsized":"2023-10-16T11:20:39.164777+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:42.287590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.146935+0000","last_change":"2023-10-16T11:20:26.084562+0000","last_active":"2023-10-16T11:20:38.146935+0000","last_peered":"2023-10-16T11:20:38.146935+0000","last_clean":"2023-10-16T11:20:38.146935+0000","last_became_active":"2023-10-16T11:20:26.083055+0000","last_became_peered":"2023-10-16T11:20:26.083055+0000","last_unstale":"2023-10-16T11:20:38.146935+0000","last_undegraded":"2023-10-16T11:20:38.146935+0000","last_fullsized":"2023-10-16T11:20:38.146935+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:49.947551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.154018+0000","last_change":"2023-10-16T11:20:39.153110+0000","last_active":"2023-10-16T11:20:39.154018+0000","last_peered":"2023-10-16T11:20:39.154018+0000","last_clean":"2023-10-16T11:20:39.154018+0000","last_became_active":"2023-10-16T11:20:39.152840+0000","last_became_peered":"2023-10-16T11:20:39.152840+0000","last_unstale":"2023-10-16T11:20:39.154018+0000","last_undegraded":"2023-10-16T11:20:39.154018+0000","last_fullsized":"2023-10-16T11:20:39.154018+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:49.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.148219+0000","last_change":"2023-10-16T11:20:38.149235+0000","last_active":"2023-10-16T11:20:39.148219+0000","last_peered":"2023-10-16T11:20:39.148219+0000","last_clean":"2023-10-16T11:20:39.148219+0000","last_became_active":"2023-10-16T11:20:38.148954+0000","last_became_peered":"2023-10-16T11:20:38.148954+0000","last_unstale":"2023-10-16T11:20:39.148219+0000","last_undegraded":"2023-10-16T11:20:39.148219+0000","last_fullsized":"2023-10-16T11:20:39.148219+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:16:29.223643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140645+0000","last_change":"2023-10-16T11:20:26.077403+0000","last_active":"2023-10-16T11:20:38.140645+0000","last_peered":"2023-10-16T11:20:38.140645+0000","last_clean":"2023-10-16T11:20:38.140645+0000","last_became_active":"2023-10-16T11:20:26.077133+0000","last_became_peered":"2023-10-16T11:20:26.077133+0000","last_unstale":"2023-10-16T11:20:38.140645+0000","last_undegraded":"2023-10-16T11:20:38.140645+0000","last_fullsized":"2023-10-16T11:20:38.140645+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:15.067913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164765+0000","last_change":"2023-10-16T11:20:39.161088+0000","last_active":"2023-10-16T11:20:39.164765+0000","last_peered":"2023-10-16T11:20:39.164765+0000","last_clean":"2023-10-16T11:20:39.164765+0000","last_became_active":"2023-10-16T11:20:39.160727+0000","last_became_peered":"2023-10-16T11:20:39.160727+0000","last_unstale":"2023-10-16T11:20:39.164765+0000","last_undegraded":"2023-10-16T11:20:39.164765+0000","last_fullsized":"2023-10-16T11:20:39.164765+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:49.867073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149404+0000","last_change":"2023-10-16T11:20:38.153279+0000","last_active":"2023-10-16T11:20:39.149404+0000","last_peered":"2023-10-16T11:20:39.149404+0000","last_clean":"2023-10-16T11:20:39.149404+0000","last_became_active":"2023-10-16T11:20:38.152980+0000","last_became_peered":"2023-10-16T11:20:38.152980+0000","last_unstale":"2023-10-16T11:20:39.149404+0000","last_undegraded":"2023-10-16T11:20:39.149404+0000","last_fullsized":"2023-10-16T11:20:39.149404+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:29:50.661320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138945+0000","last_change":"2023-10-16T11:20:26.092685+0000","last_active":"2023-10-16T11:20:38.138945+0000","last_peered":"2023-10-16T11:20:38.138945+0000","last_clean":"2023-10-16T11:20:38.138945+0000","last_became_active":"2023-10-16T11:20:26.092413+0000","last_became_peered":"2023-10-16T11:20:26.092413+0000","last_unstale":"2023-10-16T11:20:38.138945+0000","last_undegraded":"2023-10-16T11:20:38.138945+0000","last_fullsized":"2023-10-16T11:20:38.138945+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:40.852817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166731+0000","last_change":"2023-10-16T11:20:39.163622+0000","last_active":"2023-10-16T11:20:39.166731+0000","last_peered":"2023-10-16T11:20:39.166731+0000","last_clean":"2023-10-16T11:20:39.166731+0000","last_became_active":"2023-10-16T11:20:39.162568+0000","last_became_peered":"2023-10-16T11:20:39.162568+0000","last_unstale":"2023-10-16T11:20:39.166731+0000","last_undegraded":"2023-10-16T11:20:39.166731+0000","last_fullsized":"2023-10-16T11:20:39.166731+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:28.140801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149047+0000","last_change":"2023-10-16T11:20:38.155283+0000","last_active":"2023-10-16T11:20:39.149047+0000","last_peered":"2023-10-16T11:20:39.149047+0000","last_clean":"2023-10-16T11:20:39.149047+0000","last_became_active":"2023-10-16T11:20:38.155034+0000","last_became_peered":"2023-10-16T11:20:38.155034+0000","last_unstale":"2023-10-16T11:20:39.149047+0000","last_undegraded":"2023-10-16T11:20:39.149047+0000","last_fullsized":"2023-10-16T11:20:39.149047+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:53:09.383768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135013+0000","last_change":"2023-10-16T11:20:26.094837+0000","last_active":"2023-10-16T11:20:38.135013+0000","last_peered":"2023-10-16T11:20:38.135013+0000","last_clean":"2023-10-16T11:20:38.135013+0000","last_became_active":"2023-10-16T11:20:26.094247+0000","last_became_peered":"2023-10-16T11:20:26.094247+0000","last_unstale":"2023-10-16T11:20:38.135013+0000","last_undegraded":"2023-10-16T11:20:38.135013+0000","last_fullsized":"2023-10-16T11:20:38.135013+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:16.136940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164583+0000","last_change":"2023-10-16T11:20:39.161219+0000","last_active":"2023-10-16T11:20:39.164583+0000","last_peered":"2023-10-16T11:20:39.164583+0000","last_clean":"2023-10-16T11:20:39.164583+0000","last_became_active":"2023-10-16T11:20:39.160957+0000","last_became_peered":"2023-10-16T11:20:39.160957+0000","last_unstale":"2023-10-16T11:20:39.164583+0000","last_undegraded":"2023-10-16T11:20:39.164583+0000","last_fullsized":"2023-10-16T11:20:39.164583+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:55:02.646050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.153113+0000","last_change":"2023-10-16T11:20:38.143859+0000","last_active":"2023-10-16T11:20:39.153113+0000","last_peered":"2023-10-16T11:20:39.153113+0000","last_clean":"2023-10-16T11:20:39.153113+0000","last_became_active":"2023-10-16T11:20:38.143299+0000","last_became_peered":"2023-10-16T11:20:38.143299+0000","last_unstale":"2023-10-16T11:20:39.153113+0000","last_undegraded":"2023-10-16T11:20:39.153113+0000","last_fullsized":"2023-10-16T11:20:39.153113+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:42.910732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147559+0000","last_change":"2023-10-16T11:20:26.091607+0000","last_active":"2023-10-16T11:20:38.147559+0000","last_peered":"2023-10-16T11:20:38.147559+0000","last_clean":"2023-10-16T11:20:38.147559+0000","last_became_active":"2023-10-16T11:20:26.091336+0000","last_became_peered":"2023-10-16T11:20:26.091336+0000","last_unstale":"2023-10-16T11:20:38.147559+0000","last_undegraded":"2023-10-16T11:20:38.147559+0000","last_fullsized":"2023-10-16T11:20:38.147559+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:47.664336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.151327+0000","last_change":"2023-10-16T11:20:39.145395+0000","last_active":"2023-10-16T11:20:39.151327+0000","last_peered":"2023-10-16T11:20:39.151327+0000","last_clean":"2023-10-16T11:20:39.151327+0000","last_became_active":"2023-10-16T11:20:39.144591+0000","last_became_peered":"2023-10-16T11:20:39.144591+0000","last_unstale":"2023-10-16T11:20:39.151327+0000","last_undegraded":"2023-10-16T11:20:39.151327+0000","last_fullsized":"2023-10-16T11:20:39.151327+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:37.473407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.146176+0000","last_change":"2023-10-16T11:20:38.146394+0000","last_active":"2023-10-16T11:20:39.146176+0000","last_peered":"2023-10-16T11:20:39.146176+0000","last_clean":"2023-10-16T11:20:39.146176+0000","last_became_active":"2023-10-16T11:20:38.146137+0000","last_became_peered":"2023-10-16T11:20:38.146137+0000","last_unstale":"2023-10-16T11:20:39.146176+0000","last_undegraded":"2023-10-16T11:20:39.146176+0000","last_fullsized":"2023-10-16T11:20:39.146176+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:38:09.171546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138016+0000","last_change":"2023-10-16T11:20:26.101354+0000","last_active":"2023-10-16T11:20:38.138016+0000","last_peered":"2023-10-16T11:20:38.138016+0000","last_clean":"2023-10-16T11:20:38.138016+0000","last_became_active":"2023-10-16T11:20:26.101082+0000","last_became_peered":"2023-10-16T11:20:26.101082+0000","last_unstale":"2023-10-16T11:20:38.138016+0000","last_undegraded":"2023-10-16T11:20:38.138016+0000","last_fullsized":"2023-10-16T11:20:38.138016+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:48.559039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155223+0000","last_change":"2023-10-16T11:20:39.152171+0000","last_active":"2023-10-16T11:20:39.155223+0000","last_peered":"2023-10-16T11:20:39.155223+0000","last_clean":"2023-10-16T11:20:39.155223+0000","last_became_active":"2023-10-16T11:20:39.151905+0000","last_became_peered":"2023-10-16T11:20:39.151905+0000","last_unstale":"2023-10-16T11:20:39.155223+0000","last_undegraded":"2023-10-16T11:20:39.155223+0000","last_fullsized":"2023-10-16T11:20:39.155223+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:56.439651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167209+0000","last_change":"2023-10-16T11:20:38.153302+0000","last_active":"2023-10-16T11:20:39.167209+0000","last_peered":"2023-10-16T11:20:39.167209+0000","last_clean":"2023-10-16T11:20:39.167209+0000","last_became_active":"2023-10-16T11:20:38.153051+0000","last_became_peered":"2023-10-16T11:20:38.153051+0000","last_unstale":"2023-10-16T11:20:39.167209+0000","last_undegraded":"2023-10-16T11:20:39.167209+0000","last_fullsized":"2023-10-16T11:20:39.167209+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:11.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138179+0000","last_change":"2023-10-16T11:20:26.087929+0000","last_active":"2023-10-16T11:20:38.138179+0000","last_peered":"2023-10-16T11:20:38.138179+0000","last_clean":"2023-10-16T11:20:38.138179+0000","last_became_active":"2023-10-16T11:20:26.087611+0000","last_became_peered":"2023-10-16T11:20:26.087611+0000","last_unstale":"2023-10-16T11:20:38.138179+0000","last_undegraded":"2023-10-16T11:20:38.138179+0000","last_fullsized":"2023-10-16T11:20:38.138179+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:47.826756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159557+0000","last_change":"2023-10-16T11:20:39.155764+0000","last_active":"2023-10-16T11:20:39.159557+0000","last_peered":"2023-10-16T11:20:39.159557+0000","last_clean":"2023-10-16T11:20:39.159557+0000","last_became_active":"2023-10-16T11:20:39.155482+0000","last_became_peered":"2023-10-16T11:20:39.155482+0000","last_unstale":"2023-10-16T11:20:39.159557+0000","last_undegraded":"2023-10-16T11:20:39.159557+0000","last_fullsized":"2023-10-16T11:20:39.159557+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:20.055333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166250+0000","last_change":"2023-10-16T11:20:38.152264+0000","last_active":"2023-10-16T11:20:39.166250+0000","last_peered":"2023-10-16T11:20:39.166250+0000","last_clean":"2023-10-16T11:20:39.166250+0000","last_became_active":"2023-10-16T11:20:38.150890+0000","last_became_peered":"2023-10-16T11:20:38.150890+0000","last_unstale":"2023-10-16T11:20:39.166250+0000","last_undegraded":"2023-10-16T11:20:39.166250+0000","last_fullsized":"2023-10-16T11:20:39.166250+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:19.049883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140855+0000","last_change":"2023-10-16T11:20:26.076097+0000","last_active":"2023-10-16T11:20:38.140855+0000","last_peered":"2023-10-16T11:20:38.140855+0000","last_clean":"2023-10-16T11:20:38.140855+0000","last_became_active":"2023-10-16T11:20:26.075864+0000","last_became_peered":"2023-10-16T11:20:26.075864+0000","last_unstale":"2023-10-16T11:20:38.140855+0000","last_undegraded":"2023-10-16T11:20:38.140855+0000","last_fullsized":"2023-10-16T11:20:38.140855+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:36.478137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159058+0000","last_change":"2023-10-16T11:20:39.151346+0000","last_active":"2023-10-16T11:20:39.159058+0000","last_peered":"2023-10-16T11:20:39.159058+0000","last_clean":"2023-10-16T11:20:39.159058+0000","last_became_active":"2023-10-16T11:20:39.151087+0000","last_became_peered":"2023-10-16T11:20:39.151087+0000","last_unstale":"2023-10-16T11:20:39.159058+0000","last_undegraded":"2023-10-16T11:20:39.159058+0000","last_fullsized":"2023-10-16T11:20:39.159058+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:08.410514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162062+0000","last_change":"2023-10-16T11:20:38.150490+0000","last_active":"2023-10-16T11:20:39.162062+0000","last_peered":"2023-10-16T11:20:39.162062+0000","last_clean":"2023-10-16T11:20:39.162062+0000","last_became_active":"2023-10-16T11:20:38.149916+0000","last_became_peered":"2023-10-16T11:20:38.149916+0000","last_unstale":"2023-10-16T11:20:39.162062+0000","last_undegraded":"2023-10-16T11:20:39.162062+0000","last_fullsized":"2023-10-16T11:20:39.162062+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:26.373477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137015+0000","last_change":"2023-10-16T11:20:26.084690+0000","last_active":"2023-10-16T11:20:38.137015+0000","last_peered":"2023-10-16T11:20:38.137015+0000","last_clean":"2023-10-16T11:20:38.137015+0000","last_became_active":"2023-10-16T11:20:26.084390+0000","last_became_peered":"2023-10-16T11:20:26.084390+0000","last_unstale":"2023-10-16T11:20:38.137015+0000","last_undegraded":"2023-10-16T11:20:38.137015+0000","last_fullsized":"2023-10-16T11:20:38.137015+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:56:14.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156380+0000","last_change":"2023-10-16T11:20:39.149712+0000","last_active":"2023-10-16T11:20:39.156380+0000","last_peered":"2023-10-16T11:20:39.156380+0000","last_clean":"2023-10-16T11:20:39.156380+0000","last_became_active":"2023-10-16T11:20:39.149442+0000","last_became_peered":"2023-10-16T11:20:39.149442+0000","last_unstale":"2023-10-16T11:20:39.156380+0000","last_undegraded":"2023-10-16T11:20:39.156380+0000","last_fullsized":"2023-10-16T11:20:39.156380+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:21.279291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"239'8","reported_seq":19,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.168267+0000","last_change":"2023-10-16T11:20:38.151976+0000","last_active":"2023-10-16T11:20:39.168267+0000","last_peered":"2023-10-16T11:20:39.168267+0000","last_clean":"2023-10-16T11:20:39.168267+0000","last_became_active":"2023-10-16T11:20:38.151146+0000","last_became_peered":"2023-10-16T11:20:38.151146+0000","last_unstale":"2023-10-16T11:20:39.168267+0000","last_undegraded":"2023-10-16T11:20:39.168267+0000","last_fullsized":"2023-10-16T11:20:39.168267+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:20.871613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.150259+0000","last_change":"2023-10-16T11:20:26.089392+0000","last_active":"2023-10-16T11:20:38.150259+0000","last_peered":"2023-10-16T11:20:38.150259+0000","last_clean":"2023-10-16T11:20:38.150259+0000","last_became_active":"2023-10-16T11:20:26.089083+0000","last_became_peered":"2023-10-16T11:20:26.089083+0000","last_unstale":"2023-10-16T11:20:38.150259+0000","last_undegraded":"2023-10-16T11:20:38.150259+0000","last_fullsized":"2023-10-16T11:20:38.150259+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:36:16.986811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162394+0000","last_change":"2023-10-16T11:20:39.157175+0000","last_active":"2023-10-16T11:20:39.162394+0000","last_peered":"2023-10-16T11:20:39.162394+0000","last_clean":"2023-10-16T11:20:39.162394+0000","last_became_active":"2023-10-16T11:20:39.155597+0000","last_became_peered":"2023-10-16T11:20:39.155597+0000","last_unstale":"2023-10-16T11:20:39.162394+0000","last_undegraded":"2023-10-16T11:20:39.162394+0000","last_fullsized":"2023-10-16T11:20:39.162394+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:51:10.343880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149643+0000","last_change":"2023-10-16T11:20:38.144446+0000","last_active":"2023-10-16T11:20:39.149643+0000","last_peered":"2023-10-16T11:20:39.149643+0000","last_clean":"2023-10-16T11:20:39.149643+0000","last_became_active":"2023-10-16T11:20:38.144073+0000","last_became_peered":"2023-10-16T11:20:38.144073+0000","last_unstale":"2023-10-16T11:20:39.149643+0000","last_undegraded":"2023-10-16T11:20:39.149643+0000","last_fullsized":"2023-10-16T11:20:39.149643+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:17:02.781626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137878+0000","last_change":"2023-10-16T11:20:26.084395+0000","last_active":"2023-10-16T11:20:38.137878+0000","last_peered":"2023-10-16T11:20:38.137878+0000","last_clean":"2023-10-16T11:20:38.137878+0000","last_became_active":"2023-10-16T11:20:26.084119+0000","last_became_peered":"2023-10-16T11:20:26.084119+0000","last_unstale":"2023-10-16T11:20:38.137878+0000","last_undegraded":"2023-10-16T11:20:38.137878+0000","last_fullsized":"2023-10-16T11:20:38.137878+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:49:23.234510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166630+0000","last_change":"2023-10-16T11:20:39.160984+0000","last_active":"2023-10-16T11:20:39.166630+0000","last_peered":"2023-10-16T11:20:39.166630+0000","last_clean":"2023-10-16T11:20:39.166630+0000","last_became_active":"2023-10-16T11:20:39.160711+0000","last_became_peered":"2023-10-16T11:20:39.160711+0000","last_unstale":"2023-10-16T11:20:39.166630+0000","last_undegraded":"2023-10-16T11:20:39.166630+0000","last_fullsized":"2023-10-16T11:20:39.166630+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:52.110161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.136317+0000","last_change":"2023-10-16T11:20:38.149377+0000","last_active":"2023-10-16T11:20:39.136317+0000","last_peered":"2023-10-16T11:20:39.136317+0000","last_clean":"2023-10-16T11:20:39.136317+0000","last_became_active":"2023-10-16T11:20:38.149090+0000","last_became_peered":"2023-10-16T11:20:38.149090+0000","last_unstale":"2023-10-16T11:20:39.136317+0000","last_undegraded":"2023-10-16T11:20:39.136317+0000","last_fullsized":"2023-10-16T11:20:39.136317+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:55.434201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148505+0000","last_change":"2023-10-16T11:20:26.095722+0000","last_active":"2023-10-16T11:20:38.148505+0000","last_peered":"2023-10-16T11:20:38.148505+0000","last_clean":"2023-10-16T11:20:38.148505+0000","last_became_active":"2023-10-16T11:20:26.095489+0000","last_became_peered":"2023-10-16T11:20:26.095489+0000","last_unstale":"2023-10-16T11:20:38.148505+0000","last_undegraded":"2023-10-16T11:20:38.148505+0000","last_fullsized":"2023-10-16T11:20:38.148505+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:32.623008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.148235+0000","last_change":"2023-10-16T11:20:39.146671+0000","last_active":"2023-10-16T11:20:39.148235+0000","last_peered":"2023-10-16T11:20:39.148235+0000","last_clean":"2023-10-16T11:20:39.148235+0000","last_became_active":"2023-10-16T11:20:39.146403+0000","last_became_peered":"2023-10-16T11:20:39.146403+0000","last_unstale":"2023-10-16T11:20:39.148235+0000","last_undegraded":"2023-10-16T11:20:39.148235+0000","last_fullsized":"2023-10-16T11:20:39.148235+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:38:47.362989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166905+0000","last_change":"2023-10-16T11:20:38.156486+0000","last_active":"2023-10-16T11:20:39.166905+0000","last_peered":"2023-10-16T11:20:39.166905+0000","last_clean":"2023-10-16T11:20:39.166905+0000","last_became_active":"2023-10-16T11:20:38.156213+0000","last_became_peered":"2023-10-16T11:20:38.156213+0000","last_unstale":"2023-10-16T11:20:39.166905+0000","last_undegraded":"2023-10-16T11:20:39.166905+0000","last_fullsized":"2023-10-16T11:20:39.166905+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:57:51.842239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140187+0000","last_change":"2023-10-16T11:20:26.072060+0000","last_active":"2023-10-16T11:20:38.140187+0000","last_peered":"2023-10-16T11:20:38.140187+0000","last_clean":"2023-10-16T11:20:38.140187+0000","last_became_active":"2023-10-16T11:20:26.071739+0000","last_became_peered":"2023-10-16T11:20:26.071739+0000","last_unstale":"2023-10-16T11:20:38.140187+0000","last_undegraded":"2023-10-16T11:20:38.140187+0000","last_fullsized":"2023-10-16T11:20:38.140187+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:06:29.585136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162172+0000","last_change":"2023-10-16T11:20:39.158804+0000","last_active":"2023-10-16T11:20:39.162172+0000","last_peered":"2023-10-16T11:20:39.162172+0000","last_clean":"2023-10-16T11:20:39.162172+0000","last_became_active":"2023-10-16T11:20:39.158534+0000","last_became_peered":"2023-10-16T11:20:39.158534+0000","last_unstale":"2023-10-16T11:20:39.162172+0000","last_undegraded":"2023-10-16T11:20:39.162172+0000","last_fullsized":"2023-10-16T11:20:39.162172+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:36:19.917012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"239'6","reported_seq":17,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.165955+0000","last_change":"2023-10-16T11:20:38.149465+0000","last_active":"2023-10-16T11:20:39.165955+0000","last_peered":"2023-10-16T11:20:39.165955+0000","last_clean":"2023-10-16T11:20:39.165955+0000","last_became_active":"2023-10-16T11:20:38.149180+0000","last_became_peered":"2023-10-16T11:20:38.149180+0000","last_unstale":"2023-10-16T11:20:39.165955+0000","last_undegraded":"2023-10-16T11:20:39.165955+0000","last_fullsized":"2023-10-16T11:20:39.165955+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:19.685275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147587+0000","last_change":"2023-10-16T11:20:26.083156+0000","last_active":"2023-10-16T11:20:38.147587+0000","last_peered":"2023-10-16T11:20:38.147587+0000","last_clean":"2023-10-16T11:20:38.147587+0000","last_became_active":"2023-10-16T11:20:26.082813+0000","last_became_peered":"2023-10-16T11:20:26.082813+0000","last_unstale":"2023-10-16T11:20:38.147587+0000","last_undegraded":"2023-10-16T11:20:38.147587+0000","last_fullsized":"2023-10-16T11:20:38.147587+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:56:26.407654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164617+0000","last_change":"2023-10-16T11:20:39.162139+0000","last_active":"2023-10-16T11:20:39.164617+0000","last_peered":"2023-10-16T11:20:39.164617+0000","last_clean":"2023-10-16T11:20:39.164617+0000","last_became_active":"2023-10-16T11:20:39.161358+0000","last_became_peered":"2023-10-16T11:20:39.161358+0000","last_unstale":"2023-10-16T11:20:39.164617+0000","last_undegraded":"2023-10-16T11:20:39.164617+0000","last_fullsized":"2023-10-16T11:20:39.164617+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:25.286014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"239'3","reported_seq":14,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167678+0000","last_change":"2023-10-16T11:20:38.154552+0000","last_active":"2023-10-16T11:20:39.167678+0000","last_peered":"2023-10-16T11:20:39.167678+0000","last_clean":"2023-10-16T11:20:39.167678+0000","last_became_active":"2023-10-16T11:20:38.154006+0000","last_became_peered":"2023-10-16T11:20:38.154006+0000","last_unstale":"2023-10-16T11:20:39.167678+0000","last_undegraded":"2023-10-16T11:20:39.167678+0000","last_fullsized":"2023-10-16T11:20:39.167678+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:11.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138345+0000","last_change":"2023-10-16T11:20:26.097008+0000","last_active":"2023-10-16T11:20:38.138345+0000","last_peered":"2023-10-16T11:20:38.138345+0000","last_clean":"2023-10-16T11:20:38.138345+0000","last_became_active":"2023-10-16T11:20:26.096754+0000","last_became_peered":"2023-10-16T11:20:26.096754+0000","last_unstale":"2023-10-16T11:20:38.138345+0000","last_undegraded":"2023-10-16T11:20:38.138345+0000","last_fullsized":"2023-10-16T11:20:38.138345+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:08:24.400518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164959+0000","last_change":"2023-10-16T11:20:39.160131+0000","last_active":"2023-10-16T11:20:39.164959+0000","last_peered":"2023-10-16T11:20:39.164959+0000","last_clean":"2023-10-16T11:20:39.164959+0000","last_became_active":"2023-10-16T11:20:39.159491+0000","last_became_peered":"2023-10-16T11:20:39.159491+0000","last_unstale":"2023-10-16T11:20:39.164959+0000","last_undegraded":"2023-10-16T11:20:39.164959+0000","last_fullsized":"2023-10-16T11:20:39.164959+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:04.137178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.149765+0000","last_change":"2023-10-16T11:20:38.152108+0000","last_active":"2023-10-16T11:20:39.149765+0000","last_peered":"2023-10-16T11:20:39.149765+0000","last_clean":"2023-10-16T11:20:39.149765+0000","last_became_active":"2023-10-16T11:20:38.151725+0000","last_became_peered":"2023-10-16T11:20:38.151725+0000","last_unstale":"2023-10-16T11:20:39.149765+0000","last_undegraded":"2023-10-16T11:20:39.149765+0000","last_fullsized":"2023-10-16T11:20:39.149765+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:35.137888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147312+0000","last_change":"2023-10-16T11:20:26.093039+0000","last_active":"2023-10-16T11:20:38.147312+0000","last_peered":"2023-10-16T11:20:38.147312+0000","last_clean":"2023-10-16T11:20:38.147312+0000","last_became_active":"2023-10-16T11:20:26.092725+0000","last_became_peered":"2023-10-16T11:20:26.092725+0000","last_unstale":"2023-10-16T11:20:38.147312+0000","last_undegraded":"2023-10-16T11:20:38.147312+0000","last_fullsized":"2023-10-16T11:20:38.147312+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:23:37.794560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161746+0000","last_change":"2023-10-16T11:20:39.157268+0000","last_active":"2023-10-16T11:20:39.161746+0000","last_peered":"2023-10-16T11:20:39.161746+0000","last_clean":"2023-10-16T11:20:39.161746+0000","last_became_active":"2023-10-16T11:20:39.155468+0000","last_became_peered":"2023-10-16T11:20:39.155468+0000","last_unstale":"2023-10-16T11:20:39.161746+0000","last_undegraded":"2023-10-16T11:20:39.161746+0000","last_fullsized":"2023-10-16T11:20:39.161746+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:14.690224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.144741+0000","last_change":"2023-10-16T11:20:38.150702+0000","last_active":"2023-10-16T11:20:39.144741+0000","last_peered":"2023-10-16T11:20:39.144741+0000","last_clean":"2023-10-16T11:20:39.144741+0000","last_became_active":"2023-10-16T11:20:38.150460+0000","last_became_peered":"2023-10-16T11:20:38.150460+0000","last_unstale":"2023-10-16T11:20:39.144741+0000","last_undegraded":"2023-10-16T11:20:39.144741+0000","last_fullsized":"2023-10-16T11:20:39.144741+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:48.920755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137738+0000","last_change":"2023-10-16T11:20:26.091655+0000","last_active":"2023-10-16T11:20:38.137738+0000","last_peered":"2023-10-16T11:20:38.137738+0000","last_clean":"2023-10-16T11:20:38.137738+0000","last_became_active":"2023-10-16T11:20:26.091402+0000","last_became_peered":"2023-10-16T11:20:26.091402+0000","last_unstale":"2023-10-16T11:20:38.137738+0000","last_undegraded":"2023-10-16T11:20:38.137738+0000","last_fullsized":"2023-10-16T11:20:38.137738+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:25.104508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161802+0000","last_change":"2023-10-16T11:20:39.157173+0000","last_active":"2023-10-16T11:20:39.161802+0000","last_peered":"2023-10-16T11:20:39.161802+0000","last_clean":"2023-10-16T11:20:39.161802+0000","last_became_active":"2023-10-16T11:20:39.154819+0000","last_became_peered":"2023-10-16T11:20:39.154819+0000","last_unstale":"2023-10-16T11:20:39.161802+0000","last_undegraded":"2023-10-16T11:20:39.161802+0000","last_fullsized":"2023-10-16T11:20:39.161802+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:57:43.293040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.147567+0000","last_change":"2023-10-16T11:20:38.143615+0000","last_active":"2023-10-16T11:20:39.147567+0000","last_peered":"2023-10-16T11:20:39.147567+0000","last_clean":"2023-10-16T11:20:39.147567+0000","last_became_active":"2023-10-16T11:20:38.143225+0000","last_became_peered":"2023-10-16T11:20:38.143225+0000","last_unstale":"2023-10-16T11:20:39.147567+0000","last_undegraded":"2023-10-16T11:20:39.147567+0000","last_fullsized":"2023-10-16T11:20:39.147567+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:04.671147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145217+0000","last_change":"2023-10-16T11:20:26.088880+0000","last_active":"2023-10-16T11:20:38.145217+0000","last_peered":"2023-10-16T11:20:38.145217+0000","last_clean":"2023-10-16T11:20:38.145217+0000","last_became_active":"2023-10-16T11:20:26.088427+0000","last_became_peered":"2023-10-16T11:20:26.088427+0000","last_unstale":"2023-10-16T11:20:38.145217+0000","last_undegraded":"2023-10-16T11:20:38.145217+0000","last_fullsized":"2023-10-16T11:20:38.145217+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:30.068691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167431+0000","last_change":"2023-10-16T11:20:39.162512+0000","last_active":"2023-10-16T11:20:39.167431+0000","last_peered":"2023-10-16T11:20:39.167431+0000","last_clean":"2023-10-16T11:20:39.167431+0000","last_became_active":"2023-10-16T11:20:39.161697+0000","last_became_peered":"2023-10-16T11:20:39.161697+0000","last_unstale":"2023-10-16T11:20:39.167431+0000","last_undegraded":"2023-10-16T11:20:39.167431+0000","last_fullsized":"2023-10-16T11:20:39.167431+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:21:44.297465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.138713+0000","last_change":"2023-10-16T11:20:38.155830+0000","last_active":"2023-10-16T11:20:39.138713+0000","last_peered":"2023-10-16T11:20:39.138713+0000","last_clean":"2023-10-16T11:20:39.138713+0000","last_became_active":"2023-10-16T11:20:38.155569+0000","last_became_peered":"2023-10-16T11:20:38.155569+0000","last_unstale":"2023-10-16T11:20:39.138713+0000","last_undegraded":"2023-10-16T11:20:39.138713+0000","last_fullsized":"2023-10-16T11:20:39.138713+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:17.423235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138862+0000","last_change":"2023-10-16T11:20:26.099172+0000","last_active":"2023-10-16T11:20:38.138862+0000","last_peered":"2023-10-16T11:20:38.138862+0000","last_clean":"2023-10-16T11:20:38.138862+0000","last_became_active":"2023-10-16T11:20:26.098861+0000","last_became_peered":"2023-10-16T11:20:26.098861+0000","last_unstale":"2023-10-16T11:20:38.138862+0000","last_undegraded":"2023-10-16T11:20:38.138862+0000","last_fullsized":"2023-10-16T11:20:38.138862+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:03.247595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.163764+0000","last_change":"2023-10-16T11:20:39.157612+0000","last_active":"2023-10-16T11:20:39.163764+0000","last_peered":"2023-10-16T11:20:39.163764+0000","last_clean":"2023-10-16T11:20:39.163764+0000","last_became_active":"2023-10-16T11:20:39.156968+0000","last_became_peered":"2023-10-16T11:20:39.156968+0000","last_unstale":"2023-10-16T11:20:39.163764+0000","last_undegraded":"2023-10-16T11:20:39.163764+0000","last_fullsized":"2023-10-16T11:20:39.163764+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:53.149153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161976+0000","last_change":"2023-10-16T11:20:38.146077+0000","last_active":"2023-10-16T11:20:39.161976+0000","last_peered":"2023-10-16T11:20:39.161976+0000","last_clean":"2023-10-16T11:20:39.161976+0000","last_became_active":"2023-10-16T11:20:38.141808+0000","last_became_peered":"2023-10-16T11:20:38.141808+0000","last_unstale":"2023-10-16T11:20:39.161976+0000","last_undegraded":"2023-10-16T11:20:39.161976+0000","last_fullsized":"2023-10-16T11:20:39.161976+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:17.840209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135121+0000","last_change":"2023-10-16T11:20:26.098406+0000","last_active":"2023-10-16T11:20:38.135121+0000","last_peered":"2023-10-16T11:20:38.135121+0000","last_clean":"2023-10-16T11:20:38.135121+0000","last_became_active":"2023-10-16T11:20:26.098082+0000","last_became_peered":"2023-10-16T11:20:26.098082+0000","last_unstale":"2023-10-16T11:20:38.135121+0000","last_undegraded":"2023-10-16T11:20:38.135121+0000","last_fullsized":"2023-10-16T11:20:38.135121+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:44.140416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161132+0000","last_change":"2023-10-16T11:20:39.154978+0000","last_active":"2023-10-16T11:20:39.161132+0000","last_peered":"2023-10-16T11:20:39.161132+0000","last_clean":"2023-10-16T11:20:39.161132+0000","last_became_active":"2023-10-16T11:20:39.154413+0000","last_became_peered":"2023-10-16T11:20:39.154413+0000","last_unstale":"2023-10-16T11:20:39.161132+0000","last_undegraded":"2023-10-16T11:20:39.161132+0000","last_fullsized":"2023-10-16T11:20:39.161132+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:33:07.186956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162262+0000","last_change":"2023-10-16T11:20:38.148730+0000","last_active":"2023-10-16T11:20:39.162262+0000","last_peered":"2023-10-16T11:20:39.162262+0000","last_clean":"2023-10-16T11:20:39.162262+0000","last_became_active":"2023-10-16T11:20:38.148449+0000","last_became_peered":"2023-10-16T11:20:38.148449+0000","last_unstale":"2023-10-16T11:20:39.162262+0000","last_undegraded":"2023-10-16T11:20:39.162262+0000","last_fullsized":"2023-10-16T11:20:39.162262+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:19:26.584595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139246+0000","last_change":"2023-10-16T11:20:26.082815+0000","last_active":"2023-10-16T11:20:38.139246+0000","last_peered":"2023-10-16T11:20:38.139246+0000","last_clean":"2023-10-16T11:20:38.139246+0000","last_became_active":"2023-10-16T11:20:26.082524+0000","last_became_peered":"2023-10-16T11:20:26.082524+0000","last_unstale":"2023-10-16T11:20:38.139246+0000","last_undegraded":"2023-10-16T11:20:38.139246+0000","last_fullsized":"2023-10-16T11:20:38.139246+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:18:16.774780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.160408+0000","last_change":"2023-10-16T11:20:39.157899+0000","last_active":"2023-10-16T11:20:39.160408+0000","last_peered":"2023-10-16T11:20:39.160408+0000","last_clean":"2023-10-16T11:20:39.160408+0000","last_became_active":"2023-10-16T11:20:39.157736+0000","last_became_peered":"2023-10-16T11:20:39.157736+0000","last_unstale":"2023-10-16T11:20:39.160408+0000","last_undegraded":"2023-10-16T11:20:39.160408+0000","last_fullsized":"2023-10-16T11:20:39.160408+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:03:57.268779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.166398+0000","last_change":"2023-10-16T11:20:38.153358+0000","last_active":"2023-10-16T11:20:39.166398+0000","last_peered":"2023-10-16T11:20:39.166398+0000","last_clean":"2023-10-16T11:20:39.166398+0000","last_became_active":"2023-10-16T11:20:38.153067+0000","last_became_peered":"2023-10-16T11:20:38.153067+0000","last_unstale":"2023-10-16T11:20:39.166398+0000","last_undegraded":"2023-10-16T11:20:39.166398+0000","last_fullsized":"2023-10-16T11:20:39.166398+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:16.499508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134733+0000","last_change":"2023-10-16T11:20:26.100460+0000","last_active":"2023-10-16T11:20:38.134733+0000","last_peered":"2023-10-16T11:20:38.134733+0000","last_clean":"2023-10-16T11:20:38.134733+0000","last_became_active":"2023-10-16T11:20:26.100088+0000","last_became_peered":"2023-10-16T11:20:26.100088+0000","last_unstale":"2023-10-16T11:20:38.134733+0000","last_undegraded":"2023-10-16T11:20:38.134733+0000","last_fullsized":"2023-10-16T11:20:38.134733+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:38.907350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161011+0000","last_change":"2023-10-16T11:20:39.154070+0000","last_active":"2023-10-16T11:20:39.161011+0000","last_peered":"2023-10-16T11:20:39.161011+0000","last_clean":"2023-10-16T11:20:39.161011+0000","last_became_active":"2023-10-16T11:20:39.153638+0000","last_became_peered":"2023-10-16T11:20:39.153638+0000","last_unstale":"2023-10-16T11:20:39.161011+0000","last_undegraded":"2023-10-16T11:20:39.161011+0000","last_fullsized":"2023-10-16T11:20:39.161011+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:45.452553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"239'2","reported_seq":13,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167470+0000","last_change":"2023-10-16T11:20:38.154607+0000","last_active":"2023-10-16T11:20:39.167470+0000","last_peered":"2023-10-16T11:20:39.167470+0000","last_clean":"2023-10-16T11:20:39.167470+0000","last_became_active":"2023-10-16T11:20:38.154195+0000","last_became_peered":"2023-10-16T11:20:38.154195+0000","last_unstale":"2023-10-16T11:20:39.167470+0000","last_undegraded":"2023-10-16T11:20:39.167470+0000","last_fullsized":"2023-10-16T11:20:39.167470+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:01:20.923075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140478+0000","last_change":"2023-10-16T11:20:26.070717+0000","last_active":"2023-10-16T11:20:38.140478+0000","last_peered":"2023-10-16T11:20:38.140478+0000","last_clean":"2023-10-16T11:20:38.140478+0000","last_became_active":"2023-10-16T11:20:26.070394+0000","last_became_peered":"2023-10-16T11:20:26.070394+0000","last_unstale":"2023-10-16T11:20:38.140478+0000","last_undegraded":"2023-10-16T11:20:38.140478+0000","last_fullsized":"2023-10-16T11:20:38.140478+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:46:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.158924+0000","last_change":"2023-10-16T11:20:39.154353+0000","last_active":"2023-10-16T11:20:39.158924+0000","last_peered":"2023-10-16T11:20:39.158924+0000","last_clean":"2023-10-16T11:20:39.158924+0000","last_became_active":"2023-10-16T11:20:39.154048+0000","last_became_peered":"2023-10-16T11:20:39.154048+0000","last_unstale":"2023-10-16T11:20:39.158924+0000","last_undegraded":"2023-10-16T11:20:39.158924+0000","last_fullsized":"2023-10-16T11:20:39.158924+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:49.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"239'1","reported_seq":12,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.163231+0000","last_change":"2023-10-16T11:20:38.152029+0000","last_active":"2023-10-16T11:20:39.163231+0000","last_peered":"2023-10-16T11:20:39.163231+0000","last_clean":"2023-10-16T11:20:39.163231+0000","last_became_active":"2023-10-16T11:20:38.151751+0000","last_became_peered":"2023-10-16T11:20:38.151751+0000","last_unstale":"2023-10-16T11:20:39.163231+0000","last_undegraded":"2023-10-16T11:20:39.163231+0000","last_fullsized":"2023-10-16T11:20:39.163231+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:28.415965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.146679+0000","last_change":"2023-10-16T11:20:26.086782+0000","last_active":"2023-10-16T11:20:38.146679+0000","last_peered":"2023-10-16T11:20:38.146679+0000","last_clean":"2023-10-16T11:20:38.146679+0000","last_became_active":"2023-10-16T11:20:26.086510+0000","last_became_peered":"2023-10-16T11:20:26.086510+0000","last_unstale":"2023-10-16T11:20:38.146679+0000","last_undegraded":"2023-10-16T11:20:38.146679+0000","last_fullsized":"2023-10-16T11:20:38.146679+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:26:24.730788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161759+0000","last_change":"2023-10-16T11:20:39.157450+0000","last_active":"2023-10-16T11:20:39.161759+0000","last_peered":"2023-10-16T11:20:39.161759+0000","last_clean":"2023-10-16T11:20:39.161759+0000","last_became_active":"2023-10-16T11:20:39.156969+0000","last_became_peered":"2023-10-16T11:20:39.156969+0000","last_unstale":"2023-10-16T11:20:39.161759+0000","last_undegraded":"2023-10-16T11:20:39.161759+0000","last_fullsized":"2023-10-16T11:20:39.161759+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:22:46.589205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"239'4","reported_seq":15,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.165200+0000","last_change":"2023-10-16T11:20:38.157572+0000","last_active":"2023-10-16T11:20:39.165200+0000","last_peered":"2023-10-16T11:20:39.165200+0000","last_clean":"2023-10-16T11:20:39.165200+0000","last_became_active":"2023-10-16T11:20:38.157341+0000","last_became_peered":"2023-10-16T11:20:38.157341+0000","last_unstale":"2023-10-16T11:20:39.165200+0000","last_undegraded":"2023-10-16T11:20:39.165200+0000","last_fullsized":"2023-10-16T11:20:39.165200+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:37.120464+0000","last_clean_scrub_stamp":"2023-10-16T11:20:37.120464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:45:48.659423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137829+0000","last_change":"2023-10-16T11:20:26.083094+0000","last_active":"2023-10-16T11:20:38.137829+0000","last_peered":"2023-10-16T11:20:38.137829+0000","last_clean":"2023-10-16T11:20:38.137829+0000","last_became_active":"2023-10-16T11:20:26.080031+0000","last_became_peered":"2023-10-16T11:20:26.080031+0000","last_unstale":"2023-10-16T11:20:38.137829+0000","last_undegraded":"2023-10-16T11:20:38.137829+0000","last_fullsized":"2023-10-16T11:20:38.137829+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:06:55.285592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157624+0000","last_change":"2023-10-16T11:20:39.157223+0000","last_active":"2023-10-16T11:20:39.157624+0000","last_peered":"2023-10-16T11:20:39.157624+0000","last_clean":"2023-10-16T11:20:39.157624+0000","last_became_active":"2023-10-16T11:20:39.156903+0000","last_became_peered":"2023-10-16T11:20:39.156903+0000","last_unstale":"2023-10-16T11:20:39.157624+0000","last_undegraded":"2023-10-16T11:20:39.157624+0000","last_fullsized":"2023-10-16T11:20:39.157624+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:40.015528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148371+0000","last_change":"2023-10-16T11:20:26.096214+0000","last_active":"2023-10-16T11:20:38.148371+0000","last_peered":"2023-10-16T11:20:38.148371+0000","last_clean":"2023-10-16T11:20:38.148371+0000","last_became_active":"2023-10-16T11:20:26.093850+0000","last_became_peered":"2023-10-16T11:20:26.093850+0000","last_unstale":"2023-10-16T11:20:38.148371+0000","last_undegraded":"2023-10-16T11:20:38.148371+0000","last_fullsized":"2023-10-16T11:20:38.148371+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:03:48.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138671+0000","last_change":"2023-10-16T11:20:25.084166+0000","last_active":"2023-10-16T11:20:38.138671+0000","last_peered":"2023-10-16T11:20:38.138671+0000","last_clean":"2023-10-16T11:20:38.138671+0000","last_became_active":"2023-10-16T11:20:25.083857+0000","last_became_peered":"2023-10-16T11:20:25.083857+0000","last_unstale":"2023-10-16T11:20:38.138671+0000","last_undegraded":"2023-10-16T11:20:38.138671+0000","last_fullsized":"2023-10-16T11:20:38.138671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:47.210086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164122+0000","last_change":"2023-10-16T11:20:39.160255+0000","last_active":"2023-10-16T11:20:39.164122+0000","last_peered":"2023-10-16T11:20:39.164122+0000","last_clean":"2023-10-16T11:20:39.164122+0000","last_became_active":"2023-10-16T11:20:39.158836+0000","last_became_peered":"2023-10-16T11:20:39.158836+0000","last_unstale":"2023-10-16T11:20:39.164122+0000","last_undegraded":"2023-10-16T11:20:39.164122+0000","last_fullsized":"2023-10-16T11:20:39.164122+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:00:16.320484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134779+0000","last_change":"2023-10-16T11:20:26.094539+0000","last_active":"2023-10-16T11:20:38.134779+0000","last_peered":"2023-10-16T11:20:38.134779+0000","last_clean":"2023-10-16T11:20:38.134779+0000","last_became_active":"2023-10-16T11:20:26.094220+0000","last_became_peered":"2023-10-16T11:20:26.094220+0000","last_unstale":"2023-10-16T11:20:38.134779+0000","last_undegraded":"2023-10-16T11:20:38.134779+0000","last_fullsized":"2023-10-16T11:20:38.134779+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:05.280523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139278+0000","last_change":"2023-10-16T11:20:25.080334+0000","last_active":"2023-10-16T11:20:38.139278+0000","last_peered":"2023-10-16T11:20:38.139278+0000","last_clean":"2023-10-16T11:20:38.139278+0000","last_became_active":"2023-10-16T11:20:25.080084+0000","last_became_peered":"2023-10-16T11:20:25.080084+0000","last_unstale":"2023-10-16T11:20:38.139278+0000","last_undegraded":"2023-10-16T11:20:38.139278+0000","last_fullsized":"2023-10-16T11:20:38.139278+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:34:19.309219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162266+0000","last_change":"2023-10-16T11:20:39.159896+0000","last_active":"2023-10-16T11:20:39.162266+0000","last_peered":"2023-10-16T11:20:39.162266+0000","last_clean":"2023-10-16T11:20:39.162266+0000","last_became_active":"2023-10-16T11:20:39.159635+0000","last_became_peered":"2023-10-16T11:20:39.159635+0000","last_unstale":"2023-10-16T11:20:39.162266+0000","last_undegraded":"2023-10-16T11:20:39.162266+0000","last_fullsized":"2023-10-16T11:20:39.162266+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:15.954886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148596+0000","last_change":"2023-10-16T11:20:26.086141+0000","last_active":"2023-10-16T11:20:38.148596+0000","last_peered":"2023-10-16T11:20:38.148596+0000","last_clean":"2023-10-16T11:20:38.148596+0000","last_became_active":"2023-10-16T11:20:26.085865+0000","last_became_peered":"2023-10-16T11:20:26.085865+0000","last_unstale":"2023-10-16T11:20:38.148596+0000","last_undegraded":"2023-10-16T11:20:38.148596+0000","last_fullsized":"2023-10-16T11:20:38.148596+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:31.177316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140439+0000","last_change":"2023-10-16T11:20:25.087325+0000","last_active":"2023-10-16T11:20:38.140439+0000","last_peered":"2023-10-16T11:20:38.140439+0000","last_clean":"2023-10-16T11:20:38.140439+0000","last_became_active":"2023-10-16T11:20:25.087033+0000","last_became_peered":"2023-10-16T11:20:25.087033+0000","last_unstale":"2023-10-16T11:20:38.140439+0000","last_undegraded":"2023-10-16T11:20:38.140439+0000","last_fullsized":"2023-10-16T11:20:38.140439+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:17:22.652116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.165332+0000","last_change":"2023-10-16T11:20:39.162377+0000","last_active":"2023-10-16T11:20:39.165332+0000","last_peered":"2023-10-16T11:20:39.165332+0000","last_clean":"2023-10-16T11:20:39.165332+0000","last_became_active":"2023-10-16T11:20:39.161718+0000","last_became_peered":"2023-10-16T11:20:39.161718+0000","last_unstale":"2023-10-16T11:20:39.165332+0000","last_undegraded":"2023-10-16T11:20:39.165332+0000","last_fullsized":"2023-10-16T11:20:39.165332+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:09.177870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134545+0000","last_change":"2023-10-16T11:20:26.090198+0000","last_active":"2023-10-16T11:20:38.134545+0000","last_peered":"2023-10-16T11:20:38.134545+0000","last_clean":"2023-10-16T11:20:38.134545+0000","last_became_active":"2023-10-16T11:20:26.089545+0000","last_became_peered":"2023-10-16T11:20:26.089545+0000","last_unstale":"2023-10-16T11:20:38.134545+0000","last_undegraded":"2023-10-16T11:20:38.134545+0000","last_fullsized":"2023-10-16T11:20:38.134545+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:15:03.204915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143339+0000","last_change":"2023-10-16T11:20:25.084321+0000","last_active":"2023-10-16T11:20:38.143339+0000","last_peered":"2023-10-16T11:20:38.143339+0000","last_clean":"2023-10-16T11:20:38.143339+0000","last_became_active":"2023-10-16T11:20:25.083847+0000","last_became_peered":"2023-10-16T11:20:25.083847+0000","last_unstale":"2023-10-16T11:20:38.143339+0000","last_undegraded":"2023-10-16T11:20:38.143339+0000","last_fullsized":"2023-10-16T11:20:38.143339+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:45:16.338794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161437+0000","last_change":"2023-10-16T11:20:39.155651+0000","last_active":"2023-10-16T11:20:39.161437+0000","last_peered":"2023-10-16T11:20:39.161437+0000","last_clean":"2023-10-16T11:20:39.161437+0000","last_became_active":"2023-10-16T11:20:39.155383+0000","last_became_peered":"2023-10-16T11:20:39.155383+0000","last_unstale":"2023-10-16T11:20:39.161437+0000","last_undegraded":"2023-10-16T11:20:39.161437+0000","last_fullsized":"2023-10-16T11:20:39.161437+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:58.804201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140784+0000","last_change":"2023-10-16T11:20:26.089495+0000","last_active":"2023-10-16T11:20:38.140784+0000","last_peered":"2023-10-16T11:20:38.140784+0000","last_clean":"2023-10-16T11:20:38.140784+0000","last_became_active":"2023-10-16T11:20:26.089203+0000","last_became_peered":"2023-10-16T11:20:26.089203+0000","last_unstale":"2023-10-16T11:20:38.140784+0000","last_undegraded":"2023-10-16T11:20:38.140784+0000","last_fullsized":"2023-10-16T11:20:38.140784+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:01:59.154547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138269+0000","last_change":"2023-10-16T11:20:25.083810+0000","last_active":"2023-10-16T11:20:38.138269+0000","last_peered":"2023-10-16T11:20:38.138269+0000","last_clean":"2023-10-16T11:20:38.138269+0000","last_became_active":"2023-10-16T11:20:25.083251+0000","last_became_peered":"2023-10-16T11:20:25.083251+0000","last_unstale":"2023-10-16T11:20:38.138269+0000","last_undegraded":"2023-10-16T11:20:38.138269+0000","last_fullsized":"2023-10-16T11:20:38.138269+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:15:44.723324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157058+0000","last_change":"2023-10-16T11:20:39.155271+0000","last_active":"2023-10-16T11:20:39.157058+0000","last_peered":"2023-10-16T11:20:39.157058+0000","last_clean":"2023-10-16T11:20:39.157058+0000","last_became_active":"2023-10-16T11:20:39.154993+0000","last_became_peered":"2023-10-16T11:20:39.154993+0000","last_unstale":"2023-10-16T11:20:39.157058+0000","last_undegraded":"2023-10-16T11:20:39.157058+0000","last_fullsized":"2023-10-16T11:20:39.157058+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:19.290910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136843+0000","last_change":"2023-10-16T11:20:26.071683+0000","last_active":"2023-10-16T11:20:38.136843+0000","last_peered":"2023-10-16T11:20:38.136843+0000","last_clean":"2023-10-16T11:20:38.136843+0000","last_became_active":"2023-10-16T11:20:26.071396+0000","last_became_peered":"2023-10-16T11:20:26.071396+0000","last_unstale":"2023-10-16T11:20:38.136843+0000","last_undegraded":"2023-10-16T11:20:38.136843+0000","last_fullsized":"2023-10-16T11:20:38.136843+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:15.600869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138188+0000","last_change":"2023-10-16T11:20:25.085339+0000","last_active":"2023-10-16T11:20:38.138188+0000","last_peered":"2023-10-16T11:20:38.138188+0000","last_clean":"2023-10-16T11:20:38.138188+0000","last_became_active":"2023-10-16T11:20:25.085073+0000","last_became_peered":"2023-10-16T11:20:25.085073+0000","last_unstale":"2023-10-16T11:20:38.138188+0000","last_undegraded":"2023-10-16T11:20:38.138188+0000","last_fullsized":"2023-10-16T11:20:38.138188+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:23.090533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161391+0000","last_change":"2023-10-16T11:20:39.159106+0000","last_active":"2023-10-16T11:20:39.161391+0000","last_peered":"2023-10-16T11:20:39.161391+0000","last_clean":"2023-10-16T11:20:39.161391+0000","last_became_active":"2023-10-16T11:20:39.158709+0000","last_became_peered":"2023-10-16T11:20:39.158709+0000","last_unstale":"2023-10-16T11:20:39.161391+0000","last_undegraded":"2023-10-16T11:20:39.161391+0000","last_fullsized":"2023-10-16T11:20:39.161391+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:14:53.925705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137998+0000","last_change":"2023-10-16T11:20:26.085542+0000","last_active":"2023-10-16T11:20:38.137998+0000","last_peered":"2023-10-16T11:20:38.137998+0000","last_clean":"2023-10-16T11:20:38.137998+0000","last_became_active":"2023-10-16T11:20:26.083529+0000","last_became_peered":"2023-10-16T11:20:26.083529+0000","last_unstale":"2023-10-16T11:20:38.137998+0000","last_undegraded":"2023-10-16T11:20:38.137998+0000","last_fullsized":"2023-10-16T11:20:38.137998+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:00.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136934+0000","last_change":"2023-10-16T11:20:25.088288+0000","last_active":"2023-10-16T11:20:38.136934+0000","last_peered":"2023-10-16T11:20:38.136934+0000","last_clean":"2023-10-16T11:20:38.136934+0000","last_became_active":"2023-10-16T11:20:25.087144+0000","last_became_peered":"2023-10-16T11:20:25.087144+0000","last_unstale":"2023-10-16T11:20:38.136934+0000","last_undegraded":"2023-10-16T11:20:38.136934+0000","last_fullsized":"2023-10-16T11:20:38.136934+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:16:49.706199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157304+0000","last_change":"2023-10-16T11:20:39.151272+0000","last_active":"2023-10-16T11:20:39.157304+0000","last_peered":"2023-10-16T11:20:39.157304+0000","last_clean":"2023-10-16T11:20:39.157304+0000","last_became_active":"2023-10-16T11:20:39.151019+0000","last_became_peered":"2023-10-16T11:20:39.151019+0000","last_unstale":"2023-10-16T11:20:39.157304+0000","last_undegraded":"2023-10-16T11:20:39.157304+0000","last_fullsized":"2023-10-16T11:20:39.157304+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:48:02.270086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138027+0000","last_change":"2023-10-16T11:20:26.085425+0000","last_active":"2023-10-16T11:20:38.138027+0000","last_peered":"2023-10-16T11:20:38.138027+0000","last_clean":"2023-10-16T11:20:38.138027+0000","last_became_active":"2023-10-16T11:20:26.085150+0000","last_became_peered":"2023-10-16T11:20:26.085150+0000","last_unstale":"2023-10-16T11:20:38.138027+0000","last_undegraded":"2023-10-16T11:20:38.138027+0000","last_fullsized":"2023-10-16T11:20:38.138027+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:38:14.978178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.150373+0000","last_change":"2023-10-16T11:20:25.078062+0000","last_active":"2023-10-16T11:20:38.150373+0000","last_peered":"2023-10-16T11:20:38.150373+0000","last_clean":"2023-10-16T11:20:38.150373+0000","last_became_active":"2023-10-16T11:20:25.077777+0000","last_became_peered":"2023-10-16T11:20:25.077777+0000","last_unstale":"2023-10-16T11:20:38.150373+0000","last_undegraded":"2023-10-16T11:20:38.150373+0000","last_fullsized":"2023-10-16T11:20:38.150373+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:13.108385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161818+0000","last_change":"2023-10-16T11:20:39.157413+0000","last_active":"2023-10-16T11:20:39.161818+0000","last_peered":"2023-10-16T11:20:39.161818+0000","last_clean":"2023-10-16T11:20:39.161818+0000","last_became_active":"2023-10-16T11:20:39.156903+0000","last_became_peered":"2023-10-16T11:20:39.156903+0000","last_unstale":"2023-10-16T11:20:39.161818+0000","last_undegraded":"2023-10-16T11:20:39.161818+0000","last_fullsized":"2023-10-16T11:20:39.161818+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:15:21.969270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135165+0000","last_change":"2023-10-16T11:20:26.098504+0000","last_active":"2023-10-16T11:20:38.135165+0000","last_peered":"2023-10-16T11:20:38.135165+0000","last_clean":"2023-10-16T11:20:38.135165+0000","last_became_active":"2023-10-16T11:20:26.098021+0000","last_became_peered":"2023-10-16T11:20:26.098021+0000","last_unstale":"2023-10-16T11:20:38.135165+0000","last_undegraded":"2023-10-16T11:20:38.135165+0000","last_fullsized":"2023-10-16T11:20:38.135165+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:06.218021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.141329+0000","last_change":"2023-10-16T11:20:25.088083+0000","last_active":"2023-10-16T11:20:38.141329+0000","last_peered":"2023-10-16T11:20:38.141329+0000","last_clean":"2023-10-16T11:20:38.141329+0000","last_became_active":"2023-10-16T11:20:25.087792+0000","last_became_peered":"2023-10-16T11:20:25.087792+0000","last_unstale":"2023-10-16T11:20:38.141329+0000","last_undegraded":"2023-10-16T11:20:38.141329+0000","last_fullsized":"2023-10-16T11:20:38.141329+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:37:22.682758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161110+0000","last_change":"2023-10-16T11:20:39.150282+0000","last_active":"2023-10-16T11:20:39.161110+0000","last_peered":"2023-10-16T11:20:39.161110+0000","last_clean":"2023-10-16T11:20:39.161110+0000","last_became_active":"2023-10-16T11:20:39.149971+0000","last_became_peered":"2023-10-16T11:20:39.149971+0000","last_unstale":"2023-10-16T11:20:39.161110+0000","last_undegraded":"2023-10-16T11:20:39.161110+0000","last_fullsized":"2023-10-16T11:20:39.161110+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:39.998933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140270+0000","last_change":"2023-10-16T11:20:26.090580+0000","last_active":"2023-10-16T11:20:38.140270+0000","last_peered":"2023-10-16T11:20:38.140270+0000","last_clean":"2023-10-16T11:20:38.140270+0000","last_became_active":"2023-10-16T11:20:26.090296+0000","last_became_peered":"2023-10-16T11:20:26.090296+0000","last_unstale":"2023-10-16T11:20:38.140270+0000","last_undegraded":"2023-10-16T11:20:38.140270+0000","last_fullsized":"2023-10-16T11:20:38.140270+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:26:29.302528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"235'3","reported_seq":18,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145056+0000","last_change":"2023-10-16T11:20:25.083562+0000","last_active":"2023-10-16T11:20:38.145056+0000","last_peered":"2023-10-16T11:20:38.145056+0000","last_clean":"2023-10-16T11:20:38.145056+0000","last_became_active":"2023-10-16T11:20:25.083270+0000","last_became_peered":"2023-10-16T11:20:25.083270+0000","last_unstale":"2023-10-16T11:20:38.145056+0000","last_undegraded":"2023-10-16T11:20:38.145056+0000","last_fullsized":"2023-10-16T11:20:38.145056+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:34.227829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.153724+0000","last_change":"2023-10-16T11:20:39.145474+0000","last_active":"2023-10-16T11:20:39.153724+0000","last_peered":"2023-10-16T11:20:39.153724+0000","last_clean":"2023-10-16T11:20:39.153724+0000","last_became_active":"2023-10-16T11:20:39.145204+0000","last_became_peered":"2023-10-16T11:20:39.145204+0000","last_unstale":"2023-10-16T11:20:39.153724+0000","last_undegraded":"2023-10-16T11:20:39.153724+0000","last_fullsized":"2023-10-16T11:20:39.153724+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:38:10.176996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139388+0000","last_change":"2023-10-16T11:20:26.079886+0000","last_active":"2023-10-16T11:20:38.139388+0000","last_peered":"2023-10-16T11:20:38.139388+0000","last_clean":"2023-10-16T11:20:38.139388+0000","last_became_active":"2023-10-16T11:20:26.079622+0000","last_became_peered":"2023-10-16T11:20:26.079622+0000","last_unstale":"2023-10-16T11:20:38.139388+0000","last_undegraded":"2023-10-16T11:20:38.139388+0000","last_fullsized":"2023-10-16T11:20:38.139388+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:30.843293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"236'8","reported_seq":23,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139257+0000","last_change":"2023-10-16T11:20:25.083500+0000","last_active":"2023-10-16T11:20:38.139257+0000","last_peered":"2023-10-16T11:20:38.139257+0000","last_clean":"2023-10-16T11:20:38.139257+0000","last_became_active":"2023-10-16T11:20:25.083213+0000","last_became_peered":"2023-10-16T11:20:25.083213+0000","last_unstale":"2023-10-16T11:20:38.139257+0000","last_undegraded":"2023-10-16T11:20:38.139257+0000","last_fullsized":"2023-10-16T11:20:38.139257+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:04.495826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2859,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161544+0000","last_change":"2023-10-16T11:20:39.154135+0000","last_active":"2023-10-16T11:20:39.161544+0000","last_peered":"2023-10-16T11:20:39.161544+0000","last_clean":"2023-10-16T11:20:39.161544+0000","last_became_active":"2023-10-16T11:20:39.153840+0000","last_became_peered":"2023-10-16T11:20:39.153840+0000","last_unstale":"2023-10-16T11:20:39.161544+0000","last_undegraded":"2023-10-16T11:20:39.161544+0000","last_fullsized":"2023-10-16T11:20:39.161544+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:41:23.727389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138562+0000","last_change":"2023-10-16T11:20:26.083135+0000","last_active":"2023-10-16T11:20:38.138562+0000","last_peered":"2023-10-16T11:20:38.138562+0000","last_clean":"2023-10-16T11:20:38.138562+0000","last_became_active":"2023-10-16T11:20:26.082824+0000","last_became_peered":"2023-10-16T11:20:26.082824+0000","last_unstale":"2023-10-16T11:20:38.138562+0000","last_undegraded":"2023-10-16T11:20:38.138562+0000","last_fullsized":"2023-10-16T11:20:38.138562+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:55.337626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138471+0000","last_change":"2023-10-16T11:20:25.083057+0000","last_active":"2023-10-16T11:20:38.138471+0000","last_peered":"2023-10-16T11:20:38.138471+0000","last_clean":"2023-10-16T11:20:38.138471+0000","last_became_active":"2023-10-16T11:20:25.082601+0000","last_became_peered":"2023-10-16T11:20:25.082601+0000","last_unstale":"2023-10-16T11:20:38.138471+0000","last_undegraded":"2023-10-16T11:20:38.138471+0000","last_fullsized":"2023-10-16T11:20:38.138471+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:52:00.609347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.159434+0000","last_change":"2023-10-16T11:20:39.153616+0000","last_active":"2023-10-16T11:20:39.159434+0000","last_peered":"2023-10-16T11:20:39.159434+0000","last_clean":"2023-10-16T11:20:39.159434+0000","last_became_active":"2023-10-16T11:20:39.153341+0000","last_became_peered":"2023-10-16T11:20:39.153341+0000","last_unstale":"2023-10-16T11:20:39.159434+0000","last_undegraded":"2023-10-16T11:20:39.159434+0000","last_fullsized":"2023-10-16T11:20:39.159434+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:08:18.576703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140892+0000","last_change":"2023-10-16T11:20:26.073157+0000","last_active":"2023-10-16T11:20:38.140892+0000","last_peered":"2023-10-16T11:20:38.140892+0000","last_clean":"2023-10-16T11:20:38.140892+0000","last_became_active":"2023-10-16T11:20:26.072892+0000","last_became_peered":"2023-10-16T11:20:26.072892+0000","last_unstale":"2023-10-16T11:20:38.140892+0000","last_undegraded":"2023-10-16T11:20:38.140892+0000","last_fullsized":"2023-10-16T11:20:38.140892+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:21:31.224577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136738+0000","last_change":"2023-10-16T11:20:25.079093+0000","last_active":"2023-10-16T11:20:38.136738+0000","last_peered":"2023-10-16T11:20:38.136738+0000","last_clean":"2023-10-16T11:20:38.136738+0000","last_became_active":"2023-10-16T11:20:25.078740+0000","last_became_peered":"2023-10-16T11:20:25.078740+0000","last_unstale":"2023-10-16T11:20:38.136738+0000","last_undegraded":"2023-10-16T11:20:38.136738+0000","last_fullsized":"2023-10-16T11:20:38.136738+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:47:27.511026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161727+0000","last_change":"2023-10-16T11:20:39.155879+0000","last_active":"2023-10-16T11:20:39.161727+0000","last_peered":"2023-10-16T11:20:39.161727+0000","last_clean":"2023-10-16T11:20:39.161727+0000","last_became_active":"2023-10-16T11:20:39.153704+0000","last_became_peered":"2023-10-16T11:20:39.153704+0000","last_unstale":"2023-10-16T11:20:39.161727+0000","last_undegraded":"2023-10-16T11:20:39.161727+0000","last_fullsized":"2023-10-16T11:20:39.161727+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:28.544528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.139058+0000","last_change":"2023-10-16T11:20:26.084841+0000","last_active":"2023-10-16T11:20:38.139058+0000","last_peered":"2023-10-16T11:20:38.139058+0000","last_clean":"2023-10-16T11:20:38.139058+0000","last_became_active":"2023-10-16T11:20:26.084153+0000","last_became_peered":"2023-10-16T11:20:26.084153+0000","last_unstale":"2023-10-16T11:20:38.139058+0000","last_undegraded":"2023-10-16T11:20:38.139058+0000","last_fullsized":"2023-10-16T11:20:38.139058+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:30.506645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138285+0000","last_change":"2023-10-16T11:20:25.084386+0000","last_active":"2023-10-16T11:20:38.138285+0000","last_peered":"2023-10-16T11:20:38.138285+0000","last_clean":"2023-10-16T11:20:38.138285+0000","last_became_active":"2023-10-16T11:20:25.083974+0000","last_became_peered":"2023-10-16T11:20:25.083974+0000","last_unstale":"2023-10-16T11:20:38.138285+0000","last_undegraded":"2023-10-16T11:20:38.138285+0000","last_fullsized":"2023-10-16T11:20:38.138285+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:05.210033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156404+0000","last_change":"2023-10-16T11:20:39.149432+0000","last_active":"2023-10-16T11:20:39.156404+0000","last_peered":"2023-10-16T11:20:39.156404+0000","last_clean":"2023-10-16T11:20:39.156404+0000","last_became_active":"2023-10-16T11:20:39.149146+0000","last_became_peered":"2023-10-16T11:20:39.149146+0000","last_unstale":"2023-10-16T11:20:39.156404+0000","last_undegraded":"2023-10-16T11:20:39.156404+0000","last_fullsized":"2023-10-16T11:20:39.156404+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:05:50.453477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138082+0000","last_change":"2023-10-16T11:20:26.096969+0000","last_active":"2023-10-16T11:20:38.138082+0000","last_peered":"2023-10-16T11:20:38.138082+0000","last_clean":"2023-10-16T11:20:38.138082+0000","last_became_active":"2023-10-16T11:20:26.096691+0000","last_became_peered":"2023-10-16T11:20:26.096691+0000","last_unstale":"2023-10-16T11:20:38.138082+0000","last_undegraded":"2023-10-16T11:20:38.138082+0000","last_fullsized":"2023-10-16T11:20:38.138082+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:05.355796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"235'8","reported_seq":23,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134121+0000","last_change":"2023-10-16T11:20:25.082367+0000","last_active":"2023-10-16T11:20:38.134121+0000","last_peered":"2023-10-16T11:20:38.134121+0000","last_clean":"2023-10-16T11:20:38.134121+0000","last_became_active":"2023-10-16T11:20:25.082053+0000","last_became_peered":"2023-10-16T11:20:25.082053+0000","last_unstale":"2023-10-16T11:20:38.134121+0000","last_undegraded":"2023-10-16T11:20:38.134121+0000","last_fullsized":"2023-10-16T11:20:38.134121+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:33:29.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.167450+0000","last_change":"2023-10-16T11:20:39.162687+0000","last_active":"2023-10-16T11:20:39.167450+0000","last_peered":"2023-10-16T11:20:39.167450+0000","last_clean":"2023-10-16T11:20:39.167450+0000","last_became_active":"2023-10-16T11:20:39.162202+0000","last_became_peered":"2023-10-16T11:20:39.162202+0000","last_unstale":"2023-10-16T11:20:39.167450+0000","last_undegraded":"2023-10-16T11:20:39.167450+0000","last_fullsized":"2023-10-16T11:20:39.167450+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:49.551026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143568+0000","last_change":"2023-10-16T11:20:26.085864+0000","last_active":"2023-10-16T11:20:38.143568+0000","last_peered":"2023-10-16T11:20:38.143568+0000","last_clean":"2023-10-16T11:20:38.143568+0000","last_became_active":"2023-10-16T11:20:26.085644+0000","last_became_peered":"2023-10-16T11:20:26.085644+0000","last_unstale":"2023-10-16T11:20:38.143568+0000","last_undegraded":"2023-10-16T11:20:38.143568+0000","last_fullsized":"2023-10-16T11:20:38.143568+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:16.138806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138671+0000","last_change":"2023-10-16T11:20:25.084626+0000","last_active":"2023-10-16T11:20:38.138671+0000","last_peered":"2023-10-16T11:20:38.138671+0000","last_clean":"2023-10-16T11:20:38.138671+0000","last_became_active":"2023-10-16T11:20:25.084276+0000","last_became_peered":"2023-10-16T11:20:25.084276+0000","last_unstale":"2023-10-16T11:20:38.138671+0000","last_undegraded":"2023-10-16T11:20:38.138671+0000","last_fullsized":"2023-10-16T11:20:38.138671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:31.241621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.155332+0000","last_change":"2023-10-16T11:20:39.151680+0000","last_active":"2023-10-16T11:20:39.155332+0000","last_peered":"2023-10-16T11:20:39.155332+0000","last_clean":"2023-10-16T11:20:39.155332+0000","last_became_active":"2023-10-16T11:20:39.151404+0000","last_became_peered":"2023-10-16T11:20:39.151404+0000","last_unstale":"2023-10-16T11:20:39.155332+0000","last_undegraded":"2023-10-16T11:20:39.155332+0000","last_fullsized":"2023-10-16T11:20:39.155332+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:43.916182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148396+0000","last_change":"2023-10-16T11:20:26.084908+0000","last_active":"2023-10-16T11:20:38.148396+0000","last_peered":"2023-10-16T11:20:38.148396+0000","last_clean":"2023-10-16T11:20:38.148396+0000","last_became_active":"2023-10-16T11:20:26.084599+0000","last_became_peered":"2023-10-16T11:20:26.084599+0000","last_unstale":"2023-10-16T11:20:38.148396+0000","last_undegraded":"2023-10-16T11:20:38.148396+0000","last_fullsized":"2023-10-16T11:20:38.148396+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:00:13.336265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.142877+0000","last_change":"2023-10-16T11:20:25.087228+0000","last_active":"2023-10-16T11:20:38.142877+0000","last_peered":"2023-10-16T11:20:38.142877+0000","last_clean":"2023-10-16T11:20:38.142877+0000","last_became_active":"2023-10-16T11:20:25.086965+0000","last_became_peered":"2023-10-16T11:20:25.086965+0000","last_unstale":"2023-10-16T11:20:38.142877+0000","last_undegraded":"2023-10-16T11:20:38.142877+0000","last_fullsized":"2023-10-16T11:20:38.142877+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:08.265203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.164101+0000","last_change":"2023-10-16T11:20:39.159811+0000","last_active":"2023-10-16T11:20:39.164101+0000","last_peered":"2023-10-16T11:20:39.164101+0000","last_clean":"2023-10-16T11:20:39.164101+0000","last_became_active":"2023-10-16T11:20:39.159524+0000","last_became_peered":"2023-10-16T11:20:39.159524+0000","last_unstale":"2023-10-16T11:20:39.164101+0000","last_undegraded":"2023-10-16T11:20:39.164101+0000","last_fullsized":"2023-10-16T11:20:39.164101+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:01.631928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140557+0000","last_change":"2023-10-16T11:20:26.077503+0000","last_active":"2023-10-16T11:20:38.140557+0000","last_peered":"2023-10-16T11:20:38.140557+0000","last_clean":"2023-10-16T11:20:38.140557+0000","last_became_active":"2023-10-16T11:20:26.077247+0000","last_became_peered":"2023-10-16T11:20:26.077247+0000","last_unstale":"2023-10-16T11:20:38.140557+0000","last_undegraded":"2023-10-16T11:20:38.140557+0000","last_fullsized":"2023-10-16T11:20:38.140557+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:54:49.573161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"235'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140761+0000","last_change":"2023-10-16T11:20:25.076671+0000","last_active":"2023-10-16T11:20:38.140761+0000","last_peered":"2023-10-16T11:20:38.140761+0000","last_clean":"2023-10-16T11:20:38.140761+0000","last_became_active":"2023-10-16T11:20:25.076356+0000","last_became_peered":"2023-10-16T11:20:25.076356+0000","last_unstale":"2023-10-16T11:20:38.140761+0000","last_undegraded":"2023-10-16T11:20:38.140761+0000","last_fullsized":"2023-10-16T11:20:38.140761+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:30:21.582269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.154261+0000","last_change":"2023-10-16T11:20:39.147832+0000","last_active":"2023-10-16T11:20:39.154261+0000","last_peered":"2023-10-16T11:20:39.154261+0000","last_clean":"2023-10-16T11:20:39.154261+0000","last_became_active":"2023-10-16T11:20:39.147549+0000","last_became_peered":"2023-10-16T11:20:39.147549+0000","last_unstale":"2023-10-16T11:20:39.154261+0000","last_undegraded":"2023-10-16T11:20:39.154261+0000","last_fullsized":"2023-10-16T11:20:39.154261+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:18.428685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148690+0000","last_change":"2023-10-16T11:20:26.086294+0000","last_active":"2023-10-16T11:20:38.148690+0000","last_peered":"2023-10-16T11:20:38.148690+0000","last_clean":"2023-10-16T11:20:38.148690+0000","last_became_active":"2023-10-16T11:20:26.085985+0000","last_became_peered":"2023-10-16T11:20:26.085985+0000","last_unstale":"2023-10-16T11:20:38.148690+0000","last_undegraded":"2023-10-16T11:20:38.148690+0000","last_fullsized":"2023-10-16T11:20:38.148690+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:58.159985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134968+0000","last_change":"2023-10-16T11:20:25.085222+0000","last_active":"2023-10-16T11:20:38.134968+0000","last_peered":"2023-10-16T11:20:38.134968+0000","last_clean":"2023-10-16T11:20:38.134968+0000","last_became_active":"2023-10-16T11:20:25.084958+0000","last_became_peered":"2023-10-16T11:20:25.084958+0000","last_unstale":"2023-10-16T11:20:38.134968+0000","last_undegraded":"2023-10-16T11:20:38.134968+0000","last_fullsized":"2023-10-16T11:20:38.134968+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:11:15.130817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161885+0000","last_change":"2023-10-16T11:20:39.160066+0000","last_active":"2023-10-16T11:20:39.161885+0000","last_peered":"2023-10-16T11:20:39.161885+0000","last_clean":"2023-10-16T11:20:39.161885+0000","last_became_active":"2023-10-16T11:20:39.158692+0000","last_became_peered":"2023-10-16T11:20:39.158692+0000","last_unstale":"2023-10-16T11:20:39.161885+0000","last_undegraded":"2023-10-16T11:20:39.161885+0000","last_fullsized":"2023-10-16T11:20:39.161885+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:48.505360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148032+0000","last_change":"2023-10-16T11:20:26.091707+0000","last_active":"2023-10-16T11:20:38.148032+0000","last_peered":"2023-10-16T11:20:38.148032+0000","last_clean":"2023-10-16T11:20:38.148032+0000","last_became_active":"2023-10-16T11:20:26.091470+0000","last_became_peered":"2023-10-16T11:20:26.091470+0000","last_unstale":"2023-10-16T11:20:38.148032+0000","last_undegraded":"2023-10-16T11:20:38.148032+0000","last_fullsized":"2023-10-16T11:20:38.148032+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137172+0000","last_change":"2023-10-16T11:20:25.086865+0000","last_active":"2023-10-16T11:20:38.137172+0000","last_peered":"2023-10-16T11:20:38.137172+0000","last_clean":"2023-10-16T11:20:38.137172+0000","last_became_active":"2023-10-16T11:20:25.086528+0000","last_became_peered":"2023-10-16T11:20:25.086528+0000","last_unstale":"2023-10-16T11:20:38.137172+0000","last_undegraded":"2023-10-16T11:20:38.137172+0000","last_fullsized":"2023-10-16T11:20:38.137172+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:27:17.050518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161507+0000","last_change":"2023-10-16T11:20:39.159722+0000","last_active":"2023-10-16T11:20:39.161507+0000","last_peered":"2023-10-16T11:20:39.161507+0000","last_clean":"2023-10-16T11:20:39.161507+0000","last_became_active":"2023-10-16T11:20:39.159417+0000","last_became_peered":"2023-10-16T11:20:39.159417+0000","last_unstale":"2023-10-16T11:20:39.161507+0000","last_undegraded":"2023-10-16T11:20:39.161507+0000","last_fullsized":"2023-10-16T11:20:39.161507+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:57:36.538334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138379+0000","last_change":"2023-10-16T11:20:26.080221+0000","last_active":"2023-10-16T11:20:38.138379+0000","last_peered":"2023-10-16T11:20:38.138379+0000","last_clean":"2023-10-16T11:20:38.138379+0000","last_became_active":"2023-10-16T11:20:26.079927+0000","last_became_peered":"2023-10-16T11:20:26.079927+0000","last_unstale":"2023-10-16T11:20:38.138379+0000","last_undegraded":"2023-10-16T11:20:38.138379+0000","last_fullsized":"2023-10-16T11:20:38.138379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:27.603420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134588+0000","last_change":"2023-10-16T11:20:25.084862+0000","last_active":"2023-10-16T11:20:38.134588+0000","last_peered":"2023-10-16T11:20:38.134588+0000","last_clean":"2023-10-16T11:20:38.134588+0000","last_became_active":"2023-10-16T11:20:25.084538+0000","last_became_peered":"2023-10-16T11:20:25.084538+0000","last_unstale":"2023-10-16T11:20:38.134588+0000","last_undegraded":"2023-10-16T11:20:38.134588+0000","last_fullsized":"2023-10-16T11:20:38.134588+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:11:13.297732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.160194+0000","last_change":"2023-10-16T11:20:39.154016+0000","last_active":"2023-10-16T11:20:39.160194+0000","last_peered":"2023-10-16T11:20:39.160194+0000","last_clean":"2023-10-16T11:20:39.160194+0000","last_became_active":"2023-10-16T11:20:39.152666+0000","last_became_peered":"2023-10-16T11:20:39.152666+0000","last_unstale":"2023-10-16T11:20:39.160194+0000","last_undegraded":"2023-10-16T11:20:39.160194+0000","last_fullsized":"2023-10-16T11:20:39.160194+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:55.505270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134055+0000","last_change":"2023-10-16T11:20:26.101781+0000","last_active":"2023-10-16T11:20:38.134055+0000","last_peered":"2023-10-16T11:20:38.134055+0000","last_clean":"2023-10-16T11:20:38.134055+0000","last_became_active":"2023-10-16T11:20:26.101470+0000","last_became_peered":"2023-10-16T11:20:26.101470+0000","last_unstale":"2023-10-16T11:20:38.134055+0000","last_undegraded":"2023-10-16T11:20:38.134055+0000","last_fullsized":"2023-10-16T11:20:38.134055+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:58:32.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140950+0000","last_change":"2023-10-16T11:20:25.081336+0000","last_active":"2023-10-16T11:20:38.140950+0000","last_peered":"2023-10-16T11:20:38.140950+0000","last_clean":"2023-10-16T11:20:38.140950+0000","last_became_active":"2023-10-16T11:20:25.081017+0000","last_became_peered":"2023-10-16T11:20:25.081017+0000","last_unstale":"2023-10-16T11:20:38.140950+0000","last_undegraded":"2023-10-16T11:20:38.140950+0000","last_fullsized":"2023-10-16T11:20:38.140950+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:38.114268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.160792+0000","last_change":"2023-10-16T11:20:39.157949+0000","last_active":"2023-10-16T11:20:39.160792+0000","last_peered":"2023-10-16T11:20:39.160792+0000","last_clean":"2023-10-16T11:20:39.160792+0000","last_became_active":"2023-10-16T11:20:39.157626+0000","last_became_peered":"2023-10-16T11:20:39.157626+0000","last_unstale":"2023-10-16T11:20:39.160792+0000","last_undegraded":"2023-10-16T11:20:39.160792+0000","last_fullsized":"2023-10-16T11:20:39.160792+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:52:25.286014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147415+0000","last_change":"2023-10-16T11:20:26.085568+0000","last_active":"2023-10-16T11:20:38.147415+0000","last_peered":"2023-10-16T11:20:38.147415+0000","last_clean":"2023-10-16T11:20:38.147415+0000","last_became_active":"2023-10-16T11:20:26.085298+0000","last_became_peered":"2023-10-16T11:20:26.085298+0000","last_unstale":"2023-10-16T11:20:38.147415+0000","last_undegraded":"2023-10-16T11:20:38.147415+0000","last_fullsized":"2023-10-16T11:20:38.147415+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:23:15.106414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137842+0000","last_change":"2023-10-16T11:20:25.084776+0000","last_active":"2023-10-16T11:20:38.137842+0000","last_peered":"2023-10-16T11:20:38.137842+0000","last_clean":"2023-10-16T11:20:38.137842+0000","last_became_active":"2023-10-16T11:20:25.084415+0000","last_became_peered":"2023-10-16T11:20:25.084415+0000","last_unstale":"2023-10-16T11:20:38.137842+0000","last_undegraded":"2023-10-16T11:20:38.137842+0000","last_fullsized":"2023-10-16T11:20:38.137842+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:43.132428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162131+0000","last_change":"2023-10-16T11:20:39.160768+0000","last_active":"2023-10-16T11:20:39.162131+0000","last_peered":"2023-10-16T11:20:39.162131+0000","last_clean":"2023-10-16T11:20:39.162131+0000","last_became_active":"2023-10-16T11:20:39.160483+0000","last_became_peered":"2023-10-16T11:20:39.160483+0000","last_unstale":"2023-10-16T11:20:39.162131+0000","last_undegraded":"2023-10-16T11:20:39.162131+0000","last_fullsized":"2023-10-16T11:20:39.162131+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:00:21.564025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138763+0000","last_change":"2023-10-16T11:20:26.080094+0000","last_active":"2023-10-16T11:20:38.138763+0000","last_peered":"2023-10-16T11:20:38.138763+0000","last_clean":"2023-10-16T11:20:38.138763+0000","last_became_active":"2023-10-16T11:20:26.079844+0000","last_became_peered":"2023-10-16T11:20:26.079844+0000","last_unstale":"2023-10-16T11:20:38.138763+0000","last_undegraded":"2023-10-16T11:20:38.138763+0000","last_fullsized":"2023-10-16T11:20:38.138763+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:32:54.114067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"235'2","reported_seq":17,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.145447+0000","last_change":"2023-10-16T11:20:25.084135+0000","last_active":"2023-10-16T11:20:38.145447+0000","last_peered":"2023-10-16T11:20:38.145447+0000","last_clean":"2023-10-16T11:20:38.145447+0000","last_became_active":"2023-10-16T11:20:25.083794+0000","last_became_peered":"2023-10-16T11:20:25.083794+0000","last_unstale":"2023-10-16T11:20:38.145447+0000","last_undegraded":"2023-10-16T11:20:38.145447+0000","last_fullsized":"2023-10-16T11:20:38.145447+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:57.161061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161089+0000","last_change":"2023-10-16T11:20:39.154294+0000","last_active":"2023-10-16T11:20:39.161089+0000","last_peered":"2023-10-16T11:20:39.161089+0000","last_clean":"2023-10-16T11:20:39.161089+0000","last_became_active":"2023-10-16T11:20:39.153885+0000","last_became_peered":"2023-10-16T11:20:39.153885+0000","last_unstale":"2023-10-16T11:20:39.161089+0000","last_undegraded":"2023-10-16T11:20:39.161089+0000","last_fullsized":"2023-10-16T11:20:39.161089+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:10:48.308706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138681+0000","last_change":"2023-10-16T11:20:26.083428+0000","last_active":"2023-10-16T11:20:38.138681+0000","last_peered":"2023-10-16T11:20:38.138681+0000","last_clean":"2023-10-16T11:20:38.138681+0000","last_became_active":"2023-10-16T11:20:26.082473+0000","last_became_peered":"2023-10-16T11:20:26.082473+0000","last_unstale":"2023-10-16T11:20:38.138681+0000","last_undegraded":"2023-10-16T11:20:38.138681+0000","last_fullsized":"2023-10-16T11:20:38.138681+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:50:39.122257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"236'1","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.142679+0000","last_change":"2023-10-16T11:20:25.085721+0000","last_active":"2023-10-16T11:20:38.142679+0000","last_peered":"2023-10-16T11:20:38.142679+0000","last_clean":"2023-10-16T11:20:38.142679+0000","last_became_active":"2023-10-16T11:20:25.085421+0000","last_became_peered":"2023-10-16T11:20:25.085421+0000","last_unstale":"2023-10-16T11:20:38.142679+0000","last_undegraded":"2023-10-16T11:20:38.142679+0000","last_fullsized":"2023-10-16T11:20:38.142679+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:06:08.936039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.163645+0000","last_change":"2023-10-16T11:20:39.159812+0000","last_active":"2023-10-16T11:20:39.163645+0000","last_peered":"2023-10-16T11:20:39.163645+0000","last_clean":"2023-10-16T11:20:39.163645+0000","last_became_active":"2023-10-16T11:20:39.159542+0000","last_became_peered":"2023-10-16T11:20:39.159542+0000","last_unstale":"2023-10-16T11:20:39.163645+0000","last_undegraded":"2023-10-16T11:20:39.163645+0000","last_fullsized":"2023-10-16T11:20:39.163645+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:17.101688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.134870+0000","last_change":"2023-10-16T11:20:26.098738+0000","last_active":"2023-10-16T11:20:38.134870+0000","last_peered":"2023-10-16T11:20:38.134870+0000","last_clean":"2023-10-16T11:20:38.134870+0000","last_became_active":"2023-10-16T11:20:26.098226+0000","last_became_peered":"2023-10-16T11:20:26.098226+0000","last_unstale":"2023-10-16T11:20:38.134870+0000","last_undegraded":"2023-10-16T11:20:38.134870+0000","last_fullsized":"2023-10-16T11:20:38.134870+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:19:48.218074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140917+0000","last_change":"2023-10-16T11:20:25.081249+0000","last_active":"2023-10-16T11:20:38.140917+0000","last_peered":"2023-10-16T11:20:38.140917+0000","last_clean":"2023-10-16T11:20:38.140917+0000","last_became_active":"2023-10-16T11:20:25.080938+0000","last_became_peered":"2023-10-16T11:20:25.080938+0000","last_unstale":"2023-10-16T11:20:38.140917+0000","last_undegraded":"2023-10-16T11:20:38.140917+0000","last_fullsized":"2023-10-16T11:20:38.140917+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:26:09.538322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.151134+0000","last_change":"2023-10-16T11:20:39.147468+0000","last_active":"2023-10-16T11:20:39.151134+0000","last_peered":"2023-10-16T11:20:39.151134+0000","last_clean":"2023-10-16T11:20:39.151134+0000","last_became_active":"2023-10-16T11:20:39.147172+0000","last_became_peered":"2023-10-16T11:20:39.147172+0000","last_unstale":"2023-10-16T11:20:39.151134+0000","last_undegraded":"2023-10-16T11:20:39.151134+0000","last_fullsized":"2023-10-16T11:20:39.151134+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:41:51.980238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135227+0000","last_change":"2023-10-16T11:20:26.094947+0000","last_active":"2023-10-16T11:20:38.135227+0000","last_peered":"2023-10-16T11:20:38.135227+0000","last_clean":"2023-10-16T11:20:38.135227+0000","last_became_active":"2023-10-16T11:20:26.094613+0000","last_became_peered":"2023-10-16T11:20:26.094613+0000","last_unstale":"2023-10-16T11:20:38.135227+0000","last_undegraded":"2023-10-16T11:20:38.135227+0000","last_fullsized":"2023-10-16T11:20:38.135227+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:26:24.098521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.135083+0000","last_change":"2023-10-16T11:20:25.082461+0000","last_active":"2023-10-16T11:20:38.135083+0000","last_peered":"2023-10-16T11:20:38.135083+0000","last_clean":"2023-10-16T11:20:38.135083+0000","last_became_active":"2023-10-16T11:20:25.082161+0000","last_became_peered":"2023-10-16T11:20:25.082161+0000","last_unstale":"2023-10-16T11:20:38.135083+0000","last_undegraded":"2023-10-16T11:20:38.135083+0000","last_fullsized":"2023-10-16T11:20:38.135083+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:20:13.522561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.157468+0000","last_change":"2023-10-16T11:20:39.156073+0000","last_active":"2023-10-16T11:20:39.157468+0000","last_peered":"2023-10-16T11:20:39.157468+0000","last_clean":"2023-10-16T11:20:39.157468+0000","last_became_active":"2023-10-16T11:20:39.155796+0000","last_became_peered":"2023-10-16T11:20:39.155796+0000","last_unstale":"2023-10-16T11:20:39.157468+0000","last_undegraded":"2023-10-16T11:20:39.157468+0000","last_fullsized":"2023-10-16T11:20:39.157468+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:59:57.213305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138452+0000","last_change":"2023-10-16T11:20:26.083434+0000","last_active":"2023-10-16T11:20:38.138452+0000","last_peered":"2023-10-16T11:20:38.138452+0000","last_clean":"2023-10-16T11:20:38.138452+0000","last_became_active":"2023-10-16T11:20:26.083146+0000","last_became_peered":"2023-10-16T11:20:26.083146+0000","last_unstale":"2023-10-16T11:20:38.138452+0000","last_undegraded":"2023-10-16T11:20:38.138452+0000","last_fullsized":"2023-10-16T11:20:38.138452+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:36.022948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138103+0000","last_change":"2023-10-16T11:20:25.084296+0000","last_active":"2023-10-16T11:20:38.138103+0000","last_peered":"2023-10-16T11:20:38.138103+0000","last_clean":"2023-10-16T11:20:38.138103+0000","last_became_active":"2023-10-16T11:20:25.083831+0000","last_became_peered":"2023-10-16T11:20:25.083831+0000","last_unstale":"2023-10-16T11:20:38.138103+0000","last_undegraded":"2023-10-16T11:20:38.138103+0000","last_fullsized":"2023-10-16T11:20:38.138103+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:54:15.128952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.156567+0000","last_change":"2023-10-16T11:20:39.152971+0000","last_active":"2023-10-16T11:20:39.156567+0000","last_peered":"2023-10-16T11:20:39.156567+0000","last_clean":"2023-10-16T11:20:39.156567+0000","last_became_active":"2023-10-16T11:20:39.152647+0000","last_became_peered":"2023-10-16T11:20:39.152647+0000","last_unstale":"2023-10-16T11:20:39.156567+0000","last_undegraded":"2023-10-16T11:20:39.156567+0000","last_fullsized":"2023-10-16T11:20:39.156567+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:34:17.200932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147685+0000","last_change":"2023-10-16T11:20:26.097299+0000","last_active":"2023-10-16T11:20:38.147685+0000","last_peered":"2023-10-16T11:20:38.147685+0000","last_clean":"2023-10-16T11:20:38.147685+0000","last_became_active":"2023-10-16T11:20:26.096992+0000","last_became_peered":"2023-10-16T11:20:26.096992+0000","last_unstale":"2023-10-16T11:20:38.147685+0000","last_undegraded":"2023-10-16T11:20:38.147685+0000","last_fullsized":"2023-10-16T11:20:38.147685+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:48:01.203662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.143932+0000","last_change":"2023-10-16T11:20:25.085431+0000","last_active":"2023-10-16T11:20:38.143932+0000","last_peered":"2023-10-16T11:20:38.143932+0000","last_clean":"2023-10-16T11:20:38.143932+0000","last_became_active":"2023-10-16T11:20:25.084034+0000","last_became_peered":"2023-10-16T11:20:25.084034+0000","last_unstale":"2023-10-16T11:20:38.143932+0000","last_undegraded":"2023-10-16T11:20:38.143932+0000","last_fullsized":"2023-10-16T11:20:38.143932+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:44.288602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162716+0000","last_change":"2023-10-16T11:20:39.160977+0000","last_active":"2023-10-16T11:20:39.162716+0000","last_peered":"2023-10-16T11:20:39.162716+0000","last_clean":"2023-10-16T11:20:39.162716+0000","last_became_active":"2023-10-16T11:20:39.160680+0000","last_became_peered":"2023-10-16T11:20:39.160680+0000","last_unstale":"2023-10-16T11:20:39.162716+0000","last_undegraded":"2023-10-16T11:20:39.162716+0000","last_fullsized":"2023-10-16T11:20:39.162716+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:58:32.949930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.147036+0000","last_change":"2023-10-16T11:20:26.082969+0000","last_active":"2023-10-16T11:20:38.147036+0000","last_peered":"2023-10-16T11:20:38.147036+0000","last_clean":"2023-10-16T11:20:38.147036+0000","last_became_active":"2023-10-16T11:20:26.082661+0000","last_became_peered":"2023-10-16T11:20:26.082661+0000","last_unstale":"2023-10-16T11:20:38.147036+0000","last_undegraded":"2023-10-16T11:20:38.147036+0000","last_fullsized":"2023-10-16T11:20:38.147036+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:49:53.296236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.136888+0000","last_change":"2023-10-16T11:20:25.080407+0000","last_active":"2023-10-16T11:20:38.136888+0000","last_peered":"2023-10-16T11:20:38.136888+0000","last_clean":"2023-10-16T11:20:38.136888+0000","last_became_active":"2023-10-16T11:20:25.079543+0000","last_became_peered":"2023-10-16T11:20:25.079543+0000","last_unstale":"2023-10-16T11:20:38.136888+0000","last_undegraded":"2023-10-16T11:20:38.136888+0000","last_fullsized":"2023-10-16T11:20:38.136888+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:05:12.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.162068+0000","last_change":"2023-10-16T11:20:39.159286+0000","last_active":"2023-10-16T11:20:39.162068+0000","last_peered":"2023-10-16T11:20:39.162068+0000","last_clean":"2023-10-16T11:20:39.162068+0000","last_became_active":"2023-10-16T11:20:39.159009+0000","last_became_peered":"2023-10-16T11:20:39.159009+0000","last_unstale":"2023-10-16T11:20:39.162068+0000","last_undegraded":"2023-10-16T11:20:39.162068+0000","last_fullsized":"2023-10-16T11:20:39.162068+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:54.326272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.138081+0000","last_change":"2023-10-16T11:20:26.086262+0000","last_active":"2023-10-16T11:20:38.138081+0000","last_peered":"2023-10-16T11:20:38.138081+0000","last_clean":"2023-10-16T11:20:38.138081+0000","last_became_active":"2023-10-16T11:20:26.085901+0000","last_became_peered":"2023-10-16T11:20:26.085901+0000","last_unstale":"2023-10-16T11:20:38.138081+0000","last_undegraded":"2023-10-16T11:20:38.138081+0000","last_fullsized":"2023-10-16T11:20:38.138081+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:56.664462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":15,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.137098+0000","last_change":"2023-10-16T11:20:25.086844+0000","last_active":"2023-10-16T11:20:38.137098+0000","last_peered":"2023-10-16T11:20:38.137098+0000","last_clean":"2023-10-16T11:20:38.137098+0000","last_became_active":"2023-10-16T11:20:25.086570+0000","last_became_peered":"2023-10-16T11:20:25.086570+0000","last_unstale":"2023-10-16T11:20:38.137098+0000","last_undegraded":"2023-10-16T11:20:38.137098+0000","last_fullsized":"2023-10-16T11:20:38.137098+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:17.084881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":239,"state":"active+clean","last_fresh":"2023-10-16T11:20:39.161016+0000","last_change":"2023-10-16T11:20:39.155334+0000","last_active":"2023-10-16T11:20:39.161016+0000","last_peered":"2023-10-16T11:20:39.161016+0000","last_clean":"2023-10-16T11:20:39.161016+0000","last_became_active":"2023-10-16T11:20:39.155077+0000","last_became_peered":"2023-10-16T11:20:39.155077+0000","last_unstale":"2023-10-16T11:20:39.161016+0000","last_undegraded":"2023-10-16T11:20:39.161016+0000","last_fullsized":"2023-10-16T11:20:39.161016+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":238,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:38.125914+0000","last_clean_scrub_stamp":"2023-10-16T11:20:38.125914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:08:29.099178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.142552+0000","last_change":"2023-10-16T11:20:26.078189+0000","last_active":"2023-10-16T11:20:38.142552+0000","last_peered":"2023-10-16T11:20:38.142552+0000","last_clean":"2023-10-16T11:20:38.142552+0000","last_became_active":"2023-10-16T11:20:26.077754+0000","last_became_peered":"2023-10-16T11:20:26.077754+0000","last_unstale":"2023-10-16T11:20:38.142552+0000","last_undegraded":"2023-10-16T11:20:38.142552+0000","last_fullsized":"2023-10-16T11:20:38.142552+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:25.053025+0000","last_clean_scrub_stamp":"2023-10-16T11:20:25.053025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:11:45.296590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"235'4","reported_seq":19,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.148521+0000","last_change":"2023-10-16T11:20:25.087446+0000","last_active":"2023-10-16T11:20:38.148521+0000","last_peered":"2023-10-16T11:20:38.148521+0000","last_clean":"2023-10-16T11:20:38.148521+0000","last_became_active":"2023-10-16T11:20:25.086617+0000","last_became_peered":"2023-10-16T11:20:25.086617+0000","last_unstale":"2023-10-16T11:20:38.148521+0000","last_undegraded":"2023-10-16T11:20:38.148521+0000","last_fullsized":"2023-10-16T11:20:38.148521+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:20:24.045037+0000","last_clean_scrub_stamp":"2023-10-16T11:20:24.045037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:58:50.072765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'370","reported_seq":1311,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-16T11:20:38.140831+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:20:38.140831+0000","last_peered":"2023-10-16T11:20:38.140831+0000","last_clean":"2023-10-16T11:20:38.140831+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:20:38.140831+0000","last_undegraded":"2023-10-16T11:20:38.140831+0000","last_fullsized":"2023-10-16T11:20:38.140831+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":3657,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543292,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52760,"kb_used_data":3524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319080,"statfs":{"total":96636764160,"available":96582737920,"internally_reserved":0,"allocated":3608576,"data_stored":2147036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.85699999999999998,"5min":0.878,"15min":0.878},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.27800000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.33300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.83399999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69799999999999995}]},{"osd":3,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":1,"5min":1,"15min":1.002},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.33100000000000002},"max":{"1min":0.93500000000000005,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63300000000000001,"15min":0.63},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.71499999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67000000000000004}]}]},{"osd":6,"up_from":14,"seq":60129543291,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56340,"kb_used_data":2952,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315500,"statfs":{"total":96636764160,"available":96579072000,"internally_reserved":0,"allocated":3022848,"data_stored":1557244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29899999999999999},"max":{"1min":0.66700000000000004,"5min":0.81699999999999995,"15min":0.84499999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59599999999999997,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83099999999999996,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.48899999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.42599999999999999,"5min":0.40799999999999997,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.82999999999999996,"15min":0.86499999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68799999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.371},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76300000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.41699999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.29699999999999999},"max":{"1min":0.82999999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.373,"5min":0.34399999999999997,"15min":0.28100000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.502},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51600000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.872,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54300000000000004}]}]},{"osd":1,"up_from":14,"seq":60129543291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60496,"kb_used_data":2944,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311344,"statfs":{"total":96636764160,"available":96574816256,"internally_reserved":0,"allocated":3014656,"data_stored":1551314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.39700000000000002,"15min":0.40600000000000003},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51800000000000002,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.40600000000000003,"15min":0.43099999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.49199999999999999,"5min":0.62,"15min":0.91700000000000004},"last":0.371}]},{"osd":2,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40200000000000002,"15min":0.41699999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45000000000000001,"5min":0.58599999999999997,"15min":0.98899999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.441,"15min":0.46500000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63200000000000001,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.34999999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.46400000000000002,"15min":0.49199999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.78300000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45900000000000002,"15min":0.47299999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.57999999999999996,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77200000000000002,"5min":0.94999999999999996,"15min":0.96899999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85199999999999998,"5min":1.0289999999999999,"15min":1.097},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85499999999999998,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83099999999999996,"5min":0.90100000000000002,"15min":1.147},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.98699999999999999,"15min":1.0509999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.70299999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":1.0900000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Mon Oct 16 11:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":1.004,"15min":1.0600000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.58499999999999996}]}]},{"osd":0,"up_from":14,"seq":60129543292,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61060,"kb_used_data":3508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310780,"statfs":{"total":96636764160,"available":96574238720,"internally_reserved":0,"allocated":3592192,"data_stored":2138733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.22,"15min":0.214},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.39200000000000002,"15min":0.41099999999999998},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.65100000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32300000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.70299999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.42399999999999999,"15min":0.44400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.374}]},{"osd":3,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42499999999999999,"15min":0.44800000000000001},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.47399999999999998,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.84599999999999997},"last":0.441}]},{"osd":4,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66000000000000003,"15min":0.66700000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.621},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":1.008},"last":0.501}]},{"osd":5,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79900000000000004,"5min":1.004,"15min":1.008},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81999999999999995,"5min":0.94199999999999995,"15min":1.1419999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65100000000000002,"15min":0.68100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79000000000000004,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65600000000000003,"15min":0.68400000000000005},"min":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74099999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52500000000000002}]}]},{"osd":2,"up_from":14,"seq":60129543291,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48000,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323840,"statfs":{"total":96636764160,"available":96587612160,"internally_reserved":0,"allocated":2998272,"data_stored":1548421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.40500000000000003},"min":{"1min":0.24399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.432,"15min":0.42599999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.2509999999999999,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.46600000000000003},"min":{"1min":0.28499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.438,"15min":0.436},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":3.4929999999999999,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.28199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.593,"5min":3.593,"15min":3.593},"last":0.754}]},{"osd":5,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.6779999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.6549999999999998,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.69499999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.7320000000000002,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.59899999999999998}]}]},{"osd":3,"up_from":14,"seq":60129543292,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48032,"kb_used_data":2956,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323808,"statfs":{"total":96636764160,"available":96587579392,"internally_reserved":0,"allocated":3026944,"data_stored":1559304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.59799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.437,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.52600000000000002,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.308}]},{"osd":1,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.40600000000000003,"15min":0.42499999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58099999999999996,"5min":0.87,"15min":0.87},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.42699999999999999,"15min":0.45200000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54800000000000004,"5min":0.65100000000000002,"15min":0.76300000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.44400000000000001,"15min":0.46800000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.49099999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.46400000000000002,"15min":0.48299999999999998},"min":{"1min":0.28899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56100000000000005,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.38500000000000001,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61799999999999999,"15min":0.65300000000000002},"min":{"1min":0.39900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.625,"15min":0.64700000000000002},"min":{"1min":0.437,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80500000000000005,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.59199999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65100000000000002,"15min":0.67300000000000004},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.65800000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.92300000000000004,"15min":0.94399999999999995},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.66400000000000003},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.623},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65700000000000003,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96399999999999997,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.63500000000000001}]}]},{"osd":4,"up_from":14,"seq":60129543291,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78480,"kb_used_data":2944,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94293360,"statfs":{"total":96636764160,"available":96556400640,"internally_reserved":0,"allocated":3014656,"data_stored":1551282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59199999999999997,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.74399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60199999999999998,"15min":0.61899999999999999},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60399999999999998,"15min":0.61899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71899999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52100000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.83399999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.628,"15min":0.63400000000000001},"min":{"1min":0.5,"5min":0.441,"15min":0.39800000000000002},"max":{"1min":0.745,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63300000000000001,"15min":0.64600000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.879},"last":0.55100000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":0.66800000000000004,"5min":0.81299999999999994,"15min":0.84599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.621},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.57499999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.875,"15min":0.875},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.73499999999999999,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.66900000000000004}]}]},{"osd":5,"up_from":14,"seq":60129543292,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":2936,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3006464,"data_stored":1556528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59899999999999998,"15min":0.60899999999999999},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.437},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.78000000000000003,"5min":1.873,"15min":1.873},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.41999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61199999999999999,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.373,"15min":0.373},"max":{"1min":0.745,"5min":1.024,"15min":1.024},"last":0.376}]},{"osd":3,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.626,"15min":0.628},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73999999999999999,"5min":1.821,"15min":1.821},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.79900000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.503}]},{"osd":4,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.40200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":1.919,"15min":1.919},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.84199999999999997},"last":0.61799999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94299999999999995,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.36199999999999999},"max":{"1min":0.96899999999999997,"5min":2.02,"15min":2.02},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:20:43.655 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:20:43.656 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:43.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.805+0000 7f4d592f9700 1 Processor -- start 2023-10-16T11:20:43.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.805+0000 7f4d592f9700 1 -- start start 2023-10-16T11:20:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d592f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d5415d9b0 0x7f4d540b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d592f9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d540cf860 0x7f4d540cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d592f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d540d1640 con 0x7f4d541621c0 2023-10-16T11:20:43.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d592f9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d540d1920 con 0x7f4d540cf860 2023-10-16T11:20:43.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d592f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d540d1c00 con 0x7f4d5415d9b0 2023-10-16T11:20:43.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d537fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d540cf860 0x7f4d540cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d527fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4d541621c0 0x7f4d540cf150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41020/0 (socket says 172.21.15.181:41020) 2023-10-16T11:20:43.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d527fc700 1 -- 172.21.15.181:0/3954573767 learned_addr learned my addr 172.21.15.181:0/3954573767 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:43.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d52ffd700 1 --2- 172.21.15.181:0/3954573767 >> v2:172.21.15.190:3300/0 conn(0x7f4d5415d9b0 0x7f4d540b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d537fe700 1 -- 172.21.15.181:0/3954573767 >> v2:172.21.15.190:3300/0 conn(0x7f4d5415d9b0 msgr2=0x7f4d540b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:43.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d537fe700 1 --2- 172.21.15.181:0/3954573767 >> v2:172.21.15.190:3300/0 conn(0x7f4d5415d9b0 0x7f4d540b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d537fe700 1 -- 172.21.15.181:0/3954573767 >> v1:172.21.15.181:6789/0 conn(0x7f4d541621c0 legacy=0x7f4d540cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.806+0000 7f4d537fe700 1 -- 172.21.15.181:0/3954573767 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540d1ee0 con 0x7f4d540cf860 2023-10-16T11:20:43.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d537fe700 1 --2- 172.21.15.181:0/3954573767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d540cbcd0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f4d540b3790 tx=0x7f4d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cbca5249711dd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d51ffb700 1 -- 172.21.15.181:0/3954573767 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00e070 con 0x7f4d540cf860 2023-10-16T11:20:43.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d51ffb700 1 -- 172.21.15.181:0/3954573767 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c00bbd0 con 0x7f4d540cf860 2023-10-16T11:20:43.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d51ffb700 1 -- 172.21.15.181:0/3954573767 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c012430 con 0x7f4d540cf860 2023-10-16T11:20:43.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 -- 172.21.15.181:0/3954573767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 msgr2=0x7f4d540cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 --2- 172.21.15.181:0/3954573767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d540cbcd0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f4d540b3790 tx=0x7f4d4c00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:43.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 -- 172.21.15.181:0/3954573767 shutdown_connections 2023-10-16T11:20:43.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 --2- 172.21.15.181:0/3954573767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d540cbcd0 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 --2- 172.21.15.181:0/3954573767 >> v2:172.21.15.190:3300/0 conn(0x7f4d5415d9b0 0x7f4d540b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 -- 172.21.15.181:0/3954573767 >> 172.21.15.181:0/3954573767 conn(0x7f4d540665a0 msgr2=0x7f4d540669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:43.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 -- 172.21.15.181:0/3954573767 shutdown_connections 2023-10-16T11:20:43.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.807+0000 7f4d592f9700 1 -- 172.21.15.181:0/3954573767 wait complete. 2023-10-16T11:20:43.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.809+0000 7f4d592f9700 1 Processor -- start 2023-10-16T11:20:43.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.809+0000 7f4d592f9700 1 -- start start 2023-10-16T11:20:43.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d592f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d541f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d592f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d5415d9b0 0x7f4d540c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d592f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d541621c0 0x7f4d54160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d592f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540c80a0 con 0x7f4d540cf860 2023-10-16T11:20:43.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d592f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540c9860 con 0x7f4d5415d9b0 2023-10-16T11:20:43.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d592f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d541f40b0 con 0x7f4d541621c0 2023-10-16T11:20:43.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d541f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d541f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58180/0 (socket says 172.21.15.181:58180) 2023-10-16T11:20:43.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 -- 172.21.15.181:0/2198983380 learned_addr learned my addr 172.21.15.181:0/2198983380 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:43.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d537fe700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d541621c0 0x7f4d54160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d527fc700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d5415d9b0 0x7f4d540c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 -- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d541621c0 msgr2=0x7f4d54160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d541621c0 0x7f4d54160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 -- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d5415d9b0 msgr2=0x7f4d540c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:43.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d5415d9b0 0x7f4d540c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:43.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.810+0000 7f4d52ffd700 1 -- 172.21.15.181:0/2198983380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c009a70 con 0x7f4d540cf860 2023-10-16T11:20:43.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.811+0000 7f4d52ffd700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d541f3110 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e810 tx=0x7f4d4400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.811+0000 7f4d3b7fe700 1 -- 172.21.15.181:0/2198983380 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44013610 con 0x7f4d540cf860 2023-10-16T11:20:43.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.811+0000 7f4d3b7fe700 1 -- 172.21.15.181:0/2198983380 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d44013770 con 0x7f4d540cf860 2023-10-16T11:20:43.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.811+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d541f3810 con 0x7f4d540cf860 2023-10-16T11:20:43.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.811+0000 7f4d3b7fe700 1 -- 172.21.15.181:0/2198983380 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44013b20 con 0x7f4d540cf860 2023-10-16T11:20:43.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.811+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d541f3dd0 con 0x7f4d540cf860 2023-10-16T11:20:43.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.812+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d540c9b40 con 0x7f4d540cf860 2023-10-16T11:20:43.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.812+0000 7f4d3b7fe700 1 -- 172.21.15.181:0/2198983380 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4d44013c80 con 0x7f4d540cf860 2023-10-16T11:20:43.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.812+0000 7f4d3b7fe700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4d4006b140 0x7f4d4006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:43.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.813+0000 7f4d527fc700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4d4006b140 0x7f4d4006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:43.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.813+0000 7f4d3b7fe700 1 -- 172.21.15.181:0/2198983380 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f4d44089d20 con 0x7f4d540cf860 2023-10-16T11:20:43.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.813+0000 7f4d527fc700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4d4006b140 0x7f4d4006d600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4d540c4af0 tx=0x7f4d48009040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:43.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:43.817+0000 7f4d3b7fe700 1 -- 172.21.15.181:0/2198983380 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4d440543e0 con 0x7f4d540cf860 2023-10-16T11:20:44.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.050+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d5404ebb0 con 0x7f4d540cf860 2023-10-16T11:20:44.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.051+0000 7f4d3b7fe700 1 -- 172.21.15.181:0/2198983380 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+6164 (secure 0 0 0) 0x7f4d44057a00 con 0x7f4d540cf860 2023-10-16T11:20:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:44.053 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:20:44.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4d4006b140 msgr2=0x7f4d4006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4d4006b140 0x7f4d4006d600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4d540c4af0 tx=0x7f4d48009040 comp rx=0 tx=0).stop 2023-10-16T11:20:44.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 msgr2=0x7f4d541f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d541f3110 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e810 tx=0x7f4d4400ebd0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 shutdown_connections 2023-10-16T11:20:44.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f4d4006b140 0x7f4d4006d600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d541621c0 0x7f4d54160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d540cf860 0x7f4d541f3110 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 --2- 172.21.15.181:0/2198983380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d5415d9b0 0x7f4d540c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 >> 172.21.15.181:0/2198983380 conn(0x7f4d540665a0 msgr2=0x7f4d540cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 shutdown_connections 2023-10-16T11:20:44.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.053+0000 7f4d592f9700 1 -- 172.21.15.181:0/2198983380 wait complete. 2023-10-16T11:20:44.057 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 133 2023-10-16T11:20:44.469 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 133, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:20:38.533416+0000', 'modified': '2023-10-16T11:20:41.554525+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10690}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10690': {'gid': 10690, 'name': 'b', 'rank': 0, 'incarnation': 130, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.190:6834/1302040487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 1302040487}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 1302040487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:20:44.469 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:20:44.470 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T11:20:44.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.591+0000 7f07a6d9c700 1 Processor -- start 2023-10-16T11:20:44.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.591+0000 7f07a6d9c700 1 -- start start 2023-10-16T11:20:44.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f07a6d9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 0x7f07a00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f07a6d9c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00cddf0 con 0x7f07a006a8a0 2023-10-16T11:20:44.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f07a6d9c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00ccf50 con 0x7f07a015df60 2023-10-16T11:20:44.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f07a6d9c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00af9c0 con 0x7f07a00d0fc0 2023-10-16T11:20:44.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f07a5339700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f07a006a8a0 0x7f07a006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41038/0 (socket says 172.21.15.181:41038) 2023-10-16T11:20:44.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f07a5339700 1 -- 172.21.15.181:0/2862711388 learned_addr learned my addr 172.21.15.181:0/2862711388 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:44.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079ffff700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 0x7f07a00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848720669 0 0) 0x7f07a00cddf0 con 0x7f07a006a8a0 2023-10-16T11:20:44.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07900035b0 con 0x7f07a006a8a0 2023-10-16T11:20:44.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2205978373 0 0) 0x7f07900035b0 con 0x7f07a006a8a0 2023-10-16T11:20:44.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 msgr2=0x7f07a00cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 0x7f07a00cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 >> v1:172.21.15.190:6789/0 conn(0x7f07a015df60 legacy=0x7f07a00cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00cc600 con 0x7f07a006a8a0 2023-10-16T11:20:44.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f07980019f0 con 0x7f07a006a8a0 2023-10-16T11:20:44.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 >> v1:172.21.15.181:6789/0 conn(0x7f07a006a8a0 legacy=0x7f07a006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.592+0000 7f079f7fe700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0790003a30 0x7f0790003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 0x7f0790004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0790004d90 0x7f0790005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079000bac0 con 0x7f0790004d90 2023-10-16T11:20:44.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079000bc00 con 0x7f0790003a30 2023-10-16T11:20:44.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079000bdc0 con 0x7f07a00d0fc0 2023-10-16T11:20:44.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a5339700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0790004d90 0x7f0790005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a4b38700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0790003a30 0x7f0790003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079ffff700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 0x7f0790004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a5339700 1 -- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 msgr2=0x7f0790004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a5339700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00d0fc0 0x7f0790004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a5339700 1 -- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0790003a30 msgr2=0x7f0790003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a5339700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0790003a30 0x7f0790003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a5339700 1 -- 172.21.15.181:0/2862711388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900035b0 con 0x7f0790004d90 2023-10-16T11:20:44.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f07a5339700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0790004d90 0x7f0790005210 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f079800a800 tx=0x7f0790005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c012deb9ad70a3a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800a2f0 con 0x7f0790004d90 2023-10-16T11:20:44.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0798012620 con 0x7f0790004d90 2023-10-16T11:20:44.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.593+0000 7f079f7fe700 1 -- 172.21.15.181:0/2862711388 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980129d0 con 0x7f0790004d90 2023-10-16T11:20:44.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.594+0000 7f07a6d9c700 1 -- 172.21.15.181:0/2862711388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0790004d90 msgr2=0x7f0790005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.594+0000 7f07a6d9c700 1 --2- 172.21.15.181:0/2862711388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0790004d90 0x7f0790005210 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f079800a800 tx=0x7f0790005880 comp rx=0 tx=0).stop 2023-10-16T11:20:44.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.594+0000 7f07a5339700 1 -- 172.21.15.181:0/2862711388 reap_dead start 2023-10-16T11:20:44.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.594+0000 7f07a6d9c700 1 -- 172.21.15.181:0/2862711388 shutdown_connections 2023-10-16T11:20:44.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.594+0000 7f07a6d9c700 1 -- 172.21.15.181:0/2862711388 >> 172.21.15.181:0/2862711388 conn(0x7f07a00d04a0 msgr2=0x7f07a00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.594+0000 7f07a6d9c700 1 -- 172.21.15.181:0/2862711388 shutdown_connections 2023-10-16T11:20:44.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.594+0000 7f07a6d9c700 1 -- 172.21.15.181:0/2862711388 wait complete. 2023-10-16T11:20:44.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.595+0000 7f07a6d9c700 1 Processor -- start 2023-10-16T11:20:44.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.595+0000 7f07a6d9c700 1 -- start start 2023-10-16T11:20:44.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a6d9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0153a70 0x7f07a00c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a6d9c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00aff00 0x7f07a00b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a6d9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a00b1b30 0x7f07a00c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a6d9c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b1fe0 con 0x7f07a0153a70 2023-10-16T11:20:44.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a6d9c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b39f0 con 0x7f07a00b1b30 2023-10-16T11:20:44.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a6d9c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b0390 con 0x7f07a00aff00 2023-10-16T11:20:44.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0153a70 0x7f07a00c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0153a70 0x7f07a00c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58200/0 (socket says 172.21.15.181:58200) 2023-10-16T11:20:44.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 -- 172.21.15.181:0/841584934 learned_addr learned my addr 172.21.15.181:0/841584934 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:44.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f079ffff700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00aff00 0x7f07a00b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a5339700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a00b1b30 0x7f07a00c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 -- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00aff00 msgr2=0x7f07a00b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00aff00 0x7f07a00b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 -- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a00b1b30 msgr2=0x7f07a00c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a00b1b30 0x7f07a00c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 -- 172.21.15.181:0/841584934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00b0670 con 0x7f07a0153a70 2023-10-16T11:20:44.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f079ffff700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00aff00 0x7f07a00b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:44.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a5339700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a00b1b30 0x7f07a00c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:44.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f07a4b38700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0153a70 0x7f07a00c5f00 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f079400f820 tx=0x7f079400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.596+0000 7f079d7fa700 1 -- 172.21.15.181:0/841584934 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07940052e0 con 0x7f07a0153a70 2023-10-16T11:20:44.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.597+0000 7f079d7fa700 1 -- 172.21.15.181:0/841584934 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0794005440 con 0x7f07a0153a70 2023-10-16T11:20:44.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.597+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a00b0950 con 0x7f07a0153a70 2023-10-16T11:20:44.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.597+0000 7f079d7fa700 1 -- 172.21.15.181:0/841584934 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794003910 con 0x7f07a0153a70 2023-10-16T11:20:44.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.597+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a0153e50 con 0x7f07a0153a70 2023-10-16T11:20:44.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.598+0000 7f079d7fa700 1 -- 172.21.15.181:0/841584934 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0794003a70 con 0x7f07a0153a70 2023-10-16T11:20:44.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.598+0000 7f079d7fa700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f078406b140 0x7f078406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.598+0000 7f079d7fa700 1 -- 172.21.15.181:0/841584934 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f0794089e30 con 0x7f07a0153a70 2023-10-16T11:20:44.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.599+0000 7f079ffff700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f078406b140 0x7f078406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.599+0000 7f079ffff700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f078406b140 0x7f078406d600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07a006ea10 tx=0x7f078c008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.600+0000 7f07b2f86b80 1 Processor -- start 2023-10-16T11:20:44.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.600+0000 7f07b2f86b80 1 -- start start 2023-10-16T11:20:44.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb420fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb420ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb4159c20 con 0x55cbb41e3010 2023-10-16T11:20:44.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb413ec10 con 0x55cbb420d630 2023-10-16T11:20:44.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbb413c530 con 0x55cbb4208220 2023-10-16T11:20:44.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07a5339700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07a5339700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41e36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58212/0 (socket says 172.21.15.181:58212) 2023-10-16T11:20:44.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07a5339700 1 -- 172.21.15.181:0/865040736 learned_addr learned my addr 172.21.15.181:0/865040736 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:44.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f07a4b38700 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb420fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.601+0000 7f079ffff700 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb420ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07a5339700 1 -- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 msgr2=0x55cbb420fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07a5339700 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb420fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07a5339700 1 -- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 msgr2=0x55cbb420ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07a5339700 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb420ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07a5339700 1 -- 172.21.15.181:0/865040736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbb40f53a0 con 0x55cbb41e3010 2023-10-16T11:20:44.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f079ffff700 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb420ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:44.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07a5339700 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41e36c0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f0798012000 tx=0x7f0798010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f1a7b5deb00ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f078a7fc700 1 -- 172.21.15.181:0/865040736 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798001df0 con 0x55cbb41e3010 2023-10-16T11:20:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f078a7fc700 1 -- 172.21.15.181:0/865040736 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079800a210 con 0x55cbb41e3010 2023-10-16T11:20:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f078a7fc700 1 -- 172.21.15.181:0/865040736 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980017b0 con 0x55cbb41e3010 2023-10-16T11:20:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07b2f86b80 1 -- 172.21.15.181:0/865040736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 msgr2=0x55cbb41e36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41e36c0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f0798012000 tx=0x7f0798010000 comp rx=0 tx=0).stop 2023-10-16T11:20:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07b2f86b80 1 -- 172.21.15.181:0/865040736 shutdown_connections 2023-10-16T11:20:44.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb420fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41e36c0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.602+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/865040736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb420ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.603+0000 7f07b2f86b80 1 -- 172.21.15.181:0/865040736 >> 172.21.15.181:0/865040736 conn(0x55cbb4112d80 msgr2=0x55cbb422dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.603+0000 7f07b2f86b80 1 -- 172.21.15.181:0/865040736 shutdown_connections 2023-10-16T11:20:44.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.603+0000 7f07b2f86b80 1 -- 172.21.15.181:0/865040736 wait complete. 2023-10-16T11:20:44.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.603+0000 7f07b2f86b80 1 -- start start 2023-10-16T11:20:44.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.603+0000 7f07b2f86b80 1 Processor -- start 2023-10-16T11:20:44.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb41e3010 0x55cbb4208cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4207280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4205d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb4137e20 con 0x55cbb4208220 2023-10-16T11:20:44.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb40d46c0 con 0x55cbb420d630 2023-10-16T11:20:44.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbb40d4360 con 0x55cbb41e3010 2023-10-16T11:20:44.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07b2f86b80 10 client.0 _ll_register_callbacks cb 0x55cbb3f317b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:20:44.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f079ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4207280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f079ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4207280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58216/0 (socket says 172.21.15.181:58216) 2023-10-16T11:20:44.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f079ffff700 1 -- 172.21.15.181:0/460075806 learned_addr learned my addr 172.21.15.181:0/460075806 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:44.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07a4b38700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb41e3010 0x55cbb4208cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.604+0000 7f07a5339700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4205d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f079ffff700 1 -- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb41e3010 msgr2=0x55cbb4208cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f079ffff700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb41e3010 0x55cbb4208cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f079ffff700 1 -- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 msgr2=0x55cbb4205d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f079ffff700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4205d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f079ffff700 1 -- 172.21.15.181:0/460075806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07900035b0 con 0x55cbb4208220 2023-10-16T11:20:44.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07a5339700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4205d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:44.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f079ffff700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4207280 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f078c016780 tx=0x7f078c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9337a69e0886bcd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f077f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:20:44.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f077f7fe700 1 -- 172.21.15.181:0/460075806 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c024700 con 0x55cbb4208220 2023-10-16T11:20:44.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbb41d85b0 con 0x55cbb4208220 2023-10-16T11:20:44.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f077f7fe700 1 -- 172.21.15.181:0/460075806 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f078c024860 con 0x55cbb4208220 2023-10-16T11:20:44.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f077f7fe700 1 -- 172.21.15.181:0/460075806 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078c024a20 con 0x55cbb4208220 2023-10-16T11:20:44.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 10 client.12456 fetch_fsmap learned FSMap version 133 2023-10-16T11:20:44.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x55cbb3f62d90 con 0x55cbb4208220 2023-10-16T11:20:44.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f077f7fe700 1 -- 172.21.15.181:0/460075806 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbb41d85b0 con 0x55cbb4208220 2023-10-16T11:20:44.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f077f7fe700 1 -- 172.21.15.181:0/460075806 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 133) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f078c02a020 con 0x55cbb4208220 2023-10-16T11:20:44.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 10 client.12456 fetch_fsmap finished waiting for FSMap version 133 2023-10-16T11:20:44.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 10 client.12456 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:20:44.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 20 client.12456 populate_metadata read hostname 'smithi181' 2023-10-16T11:20:44.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb422cf40 0x55cbb420bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 4 client.12456 mds_command: new command op to 10690 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:20:44.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.605+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 --> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbb4130e00 con 0x55cbb422cf40 2023-10-16T11:20:44.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.606+0000 7f07a4b38700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb422cf40 0x55cbb420bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.606+0000 7f07a4b38700 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb422cf40 0x55cbb420bb40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e40ec0917a8757 server_cookie=a38213ef53cfc4fe in_seq=0 out_seq=0 2023-10-16T11:20:44.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.606+0000 7f077f7fe700 10 client.12456 ms_handle_connect on v2:172.21.15.190:6832/1302040487 2023-10-16T11:20:44.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f077f7fe700 1 -- 172.21.15.181:0/460075806 <== mds.0 v2:172.21.15.190:6832/1302040487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cbb4130e00 con 0x55cbb422cf40 2023-10-16T11:20:44.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f077f7fe700 10 client.12456 handle_command_reply: tid=0 2023-10-16T11:20:44.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 client.12456 shutdown 2023-10-16T11:20:44.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 msgr2=0x55cbb4207280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4207280 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f078c016780 tx=0x7f078c0047c0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 shutdown_connections 2023-10-16T11:20:44.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb422cf40 0x55cbb420bb40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb41e3010 0x55cbb4208cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4205d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/460075806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4207280 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f077f7fe700 0 client.12456 ms_handle_reset on v2:172.21.15.190:6832/1302040487 2023-10-16T11:20:44.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 >> 172.21.15.181:0/460075806 conn(0x55cbb4112d80 msgr2=0x55cbb42247c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 shutdown_connections 2023-10-16T11:20:44.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 1 -- 172.21.15.181:0/460075806 wait complete. 2023-10-16T11:20:44.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.607+0000 7f07b2f86b80 20 client.12456 trim_cache size 0 max 16384 2023-10-16T11:20:44.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.624+0000 7f07b2f86b80 1 Processor -- start 2023-10-16T11:20:44.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f07b2f86b80 1 -- start start 2023-10-16T11:20:44.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb41e3010 0x55cbb4213310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4208720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb420d630 0x55cbb4239190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb3fb88d0 con 0x55cbb4208220 2023-10-16T11:20:44.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb4205680 con 0x55cbb41e3010 2023-10-16T11:20:44.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbb4213bc0 con 0x55cbb420d630 2023-10-16T11:20:44.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f079ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4208720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f079ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4208720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58224/0 (socket says 172.21.15.181:58224) 2023-10-16T11:20:44.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.625+0000 7f079ffff700 1 -- 172.21.15.181:0/3970145350 learned_addr learned my addr 172.21.15.181:0/3970145350 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:44.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07a5339700 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb420d630 0x55cbb4239190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07a4b38700 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb41e3010 0x55cbb4213310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f079ffff700 1 -- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb420d630 msgr2=0x55cbb4239190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f079ffff700 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb420d630 0x55cbb4239190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f079ffff700 1 -- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb41e3010 msgr2=0x55cbb4213310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f079ffff700 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb41e3010 0x55cbb4213310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f079ffff700 1 -- 172.21.15.181:0/3970145350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbb3f4bc10 con 0x55cbb4208220 2023-10-16T11:20:44.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07a5339700 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb420d630 0x55cbb4239190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:20:44.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f079ffff700 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4208720 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f078c024bf0 tx=0x7f078c025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959a7faa4dff8bc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f078affd700 1 -- 172.21.15.181:0/3970145350 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c024900 con 0x55cbb4208220 2023-10-16T11:20:44.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f078affd700 1 -- 172.21.15.181:0/3970145350 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c005a20 con 0x55cbb4208220 2023-10-16T11:20:44.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f078affd700 1 -- 172.21.15.181:0/3970145350 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c005d90 con 0x55cbb4208220 2023-10-16T11:20:44.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07b2f86b80 1 -- 172.21.15.181:0/3970145350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 msgr2=0x55cbb4208720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4208720 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f078c024bf0 tx=0x7f078c025000 comp rx=0 tx=0).stop 2023-10-16T11:20:44.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07b2f86b80 1 -- 172.21.15.181:0/3970145350 shutdown_connections 2023-10-16T11:20:44.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb420d630 0x55cbb4239190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb41e3010 0x55cbb4213310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/3970145350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb4208220 0x55cbb4208720 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.626+0000 7f07b2f86b80 1 -- 172.21.15.181:0/3970145350 >> 172.21.15.181:0/3970145350 conn(0x55cbb4112d80 msgr2=0x55cbb424c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.627+0000 7f07b2f86b80 1 -- 172.21.15.181:0/3970145350 shutdown_connections 2023-10-16T11:20:44.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.627+0000 7f07b2f86b80 1 -- 172.21.15.181:0/3970145350 wait complete. 2023-10-16T11:20:44.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.627+0000 7f07b2f86b80 1 -- start start 2023-10-16T11:20:44.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.627+0000 7f07b2f86b80 1 Processor -- start 2023-10-16T11:20:44.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb41da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07b2f86b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4294c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb4227d10 con 0x55cbb41e3010 2023-10-16T11:20:44.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb41e3480 con 0x55cbb420d630 2023-10-16T11:20:44.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07b2f86b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbb41e4250 con 0x55cbb4208220 2023-10-16T11:20:44.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07b2f86b80 10 client.0 _ll_register_callbacks cb 0x7f078c014cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:20:44.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41d9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58228/0 (socket says 172.21.15.181:58228) 2023-10-16T11:20:44.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 -- 172.21.15.181:0/2225232048 learned_addr learned my addr 172.21.15.181:0/2225232048 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:44.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a5339700 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4294c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f079ffff700 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb41da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 -- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 msgr2=0x55cbb41da480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb41da480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 -- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 msgr2=0x55cbb4294c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4294c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.628+0000 7f07a4b38700 1 -- 172.21.15.181:0/2225232048 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbb3f62d90 con 0x55cbb41e3010 2023-10-16T11:20:44.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07a4b38700 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41d9f40 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f0794057fc0 tx=0x7f0794000fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d57d778eea793ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f077f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:20:44.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f077f7fe700 1 -- 172.21.15.181:0/2225232048 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079405c410 con 0x55cbb41e3010 2023-10-16T11:20:44.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f077f7fe700 1 -- 172.21.15.181:0/2225232048 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f07940605c0 con 0x55cbb41e3010 2023-10-16T11:20:44.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f079800a210 con 0x55cbb41e3010 2023-10-16T11:20:44.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f077f7fe700 1 -- 172.21.15.181:0/2225232048 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07940549d0 con 0x55cbb41e3010 2023-10-16T11:20:44.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 10 client.12462 fetch_fsmap learned FSMap version 133 2023-10-16T11:20:44.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) v3 -- 0x55cbb3f89200 con 0x55cbb41e3010 2023-10-16T11:20:44.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f077f7fe700 1 -- 172.21.15.181:0/2225232048 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079800a210 con 0x55cbb41e3010 2023-10-16T11:20:44.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f077f7fe700 1 -- 172.21.15.181:0/2225232048 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 133) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f0794058520 con 0x55cbb41e3010 2023-10-16T11:20:44.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 10 client.12462 fetch_fsmap finished waiting for FSMap version 133 2023-10-16T11:20:44.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 10 client.12462 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:20:44.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 20 client.12462 populate_metadata read hostname 'smithi181' 2023-10-16T11:20:44.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb4295190 0x55cbb41e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 4 client.12462 mds_command: new command op to 10690 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:20:44.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 --> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cbb4130b20 con 0x55cbb4295190 2023-10-16T11:20:44.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.629+0000 7f079ffff700 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb4295190 0x55cbb41e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f077f7fe700 10 client.12462 ms_handle_connect on v2:172.21.15.190:6832/1302040487 2023-10-16T11:20:44.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f079ffff700 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb4295190 0x55cbb41e4a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142baf28af42b18c server_cookie=5ceb5cb3cc7a4df1 in_seq=0 out_seq=0 2023-10-16T11:20:44.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f077f7fe700 1 -- 172.21.15.181:0/2225232048 <== mds.0 v2:172.21.15.190:6832/1302040487 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55cbb4130b20 con 0x55cbb4295190 2023-10-16T11:20:44.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f077f7fe700 10 client.12462 handle_command_reply: tid=0 2023-10-16T11:20:44.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 client.12462 shutdown 2023-10-16T11:20:44.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 msgr2=0x55cbb41d9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41d9f40 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f0794057fc0 tx=0x7f0794000fc0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 shutdown_connections 2023-10-16T11:20:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:6832/1302040487,v1:172.21.15.190:6834/1302040487] conn(0x55cbb4295190 0x55cbb41e4a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cbb420d630 0x55cbb4294c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f077f7fe700 0 client.12462 ms_handle_reset on v2:172.21.15.190:6832/1302040487 2023-10-16T11:20:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbb41e3010 0x55cbb41d9f40 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 --2- 172.21.15.181:0/2225232048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cbb4208220 0x55cbb41da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 >> 172.21.15.181:0/2225232048 conn(0x55cbb4112d80 msgr2=0x55cbb4208cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 shutdown_connections 2023-10-16T11:20:44.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 1 -- 172.21.15.181:0/2225232048 wait complete. 2023-10-16T11:20:44.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.630+0000 7f07b2f86b80 20 client.12462 trim_cache size 0 max 16384 2023-10-16T11:20:44.660 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:20:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:20:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:20:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 10690, 2023-10-16T11:20:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:20:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:20:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "backup_fs", 2023-10-16T11:20:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.4966941120000001, 2023-10-16T11:20:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 133, 2023-10-16T11:20:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 240, 2023-10-16T11:20:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 240, 2023-10-16T11:20:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 23.470560313 2023-10-16T11:20:44.662 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:20:44.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f078406b140 msgr2=0x7f078406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f078406b140 0x7f078406d600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07a006ea10 tx=0x7f078c008040 comp rx=0 tx=0).stop 2023-10-16T11:20:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0153a70 msgr2=0x7f07a00c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0153a70 0x7f07a00c5f00 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f079400f820 tx=0x7f079400fbe0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 shutdown_connections 2023-10-16T11:20:44.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f078406b140 0x7f078406d600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07a00b1b30 0x7f07a00c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0153a70 0x7f07a00c5f00 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 --2- 172.21.15.181:0/841584934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07a00aff00 0x7f07a00b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 >> 172.21.15.181:0/841584934 conn(0x7f07a00d04a0 msgr2=0x7f07a006f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 shutdown_connections 2023-10-16T11:20:44.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.631+0000 7f07a6d9c700 1 -- 172.21.15.181:0/841584934 wait complete. 2023-10-16T11:20:44.667 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:20:44.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.789+0000 7f728f8fc700 1 Processor -- start 2023-10-16T11:20:44.791 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.789+0000 7f728f8fc700 1 -- start start 2023-10-16T11:20:44.793 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728f8fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f728815b7b0 0x7f72880cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728f8fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f72880665a0 0x7f728806e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728f8fc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728815f590 con 0x7f72880b2310 2023-10-16T11:20:44.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728f8fc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7288066a30 con 0x7f728815b7b0 2023-10-16T11:20:44.794 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728f8fc700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728815f850 con 0x7f72880665a0 2023-10-16T11:20:44.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728d698700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f728815b7b0 0x7f72880cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728de99700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f72880b2310 0x7f728806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41086/0 (socket says 172.21.15.181:41086) 2023-10-16T11:20:44.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728de99700 1 -- 172.21.15.181:0/2386823009 learned_addr learned my addr 172.21.15.181:0/2386823009 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:44.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728ce97700 1 --2- 172.21.15.181:0/2386823009 >> v2:172.21.15.190:3300/0 conn(0x7f72880665a0 0x7f728806e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.795 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728d698700 1 -- 172.21.15.181:0/2386823009 >> v2:172.21.15.190:3300/0 conn(0x7f72880665a0 msgr2=0x7f728806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:44.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.790+0000 7f728d698700 1 --2- 172.21.15.181:0/2386823009 >> v2:172.21.15.190:3300/0 conn(0x7f72880665a0 0x7f728806e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728d698700 1 -- 172.21.15.181:0/2386823009 >> v1:172.21.15.181:6789/0 conn(0x7f72880b2310 legacy=0x7f728806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f727ffff700 1 -- 172.21.15.181:0/2386823009 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321714294 0 0) 0x7f728815f590 con 0x7f72880b2310 2023-10-16T11:20:44.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728d698700 1 -- 172.21.15.181:0/2386823009 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728815fb30 con 0x7f728815b7b0 2023-10-16T11:20:44.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728d698700 1 --2- 172.21.15.181:0/2386823009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f728815b7b0 0x7f72880cc1d0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f72880cb230 tx=0x7f7278009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662a60cd37cad50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f727ffff700 1 -- 172.21.15.181:0/2386823009 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278011830 con 0x7f728815b7b0 2023-10-16T11:20:44.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f727ffff700 1 -- 172.21.15.181:0/2386823009 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278011990 con 0x7f728815b7b0 2023-10-16T11:20:44.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f727ffff700 1 -- 172.21.15.181:0/2386823009 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278011d20 con 0x7f728815b7b0 2023-10-16T11:20:44.797 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728f8fc700 1 -- 172.21.15.181:0/2386823009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f728815b7b0 msgr2=0x7f72880cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728f8fc700 1 --2- 172.21.15.181:0/2386823009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f728815b7b0 0x7f72880cc1d0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f72880cb230 tx=0x7f7278009b20 comp rx=0 tx=0).stop 2023-10-16T11:20:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728f8fc700 1 -- 172.21.15.181:0/2386823009 shutdown_connections 2023-10-16T11:20:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728f8fc700 1 --2- 172.21.15.181:0/2386823009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f728815b7b0 0x7f72880cc1d0 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728f8fc700 1 --2- 172.21.15.181:0/2386823009 >> v2:172.21.15.190:3300/0 conn(0x7f72880665a0 0x7f728806e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.798 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728f8fc700 1 -- 172.21.15.181:0/2386823009 >> 172.21.15.181:0/2386823009 conn(0x7f72880b1780 msgr2=0x7f72880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.791+0000 7f728f8fc700 1 -- 172.21.15.181:0/2386823009 shutdown_connections 2023-10-16T11:20:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.792+0000 7f728f8fc700 1 -- 172.21.15.181:0/2386823009 wait complete. 2023-10-16T11:20:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 Processor -- start 2023-10-16T11:20:44.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 -- start start 2023-10-16T11:20:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72880b2310 0x7f72880c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f728815b7b0 0x7f72880b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72880b2710 0x7f728809de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72880a07b0 con 0x7f72880b2310 2023-10-16T11:20:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72880b0f60 con 0x7f728815b7b0 2023-10-16T11:20:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.793+0000 7f728f8fc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72880cecf0 con 0x7f72880b2710 2023-10-16T11:20:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72880b2310 0x7f72880c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72880b2310 0x7f72880c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58242/0 (socket says 172.21.15.181:58242) 2023-10-16T11:20:44.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 -- 172.21.15.181:0/438799118 learned_addr learned my addr 172.21.15.181:0/438799118 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:44.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728ce97700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f728815b7b0 0x7f72880b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728de99700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72880b2710 0x7f728809de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 -- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72880b2710 msgr2=0x7f728809de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72880b2710 0x7f728809de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 -- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f728815b7b0 msgr2=0x7f72880b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:44.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f728815b7b0 0x7f72880b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:44.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 -- 172.21.15.181:0/438799118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72780094a0 con 0x7f72880b2310 2023-10-16T11:20:44.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728ce97700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f728815b7b0 0x7f72880b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:44.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728d698700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72880b2310 0x7f72880c4d50 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fd0 tx=0x7f7278004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f727dffb700 1 -- 172.21.15.181:0/438799118 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780036a0 con 0x7f72880b2310 2023-10-16T11:20:44.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f727dffb700 1 -- 172.21.15.181:0/438799118 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727801e070 con 0x7f72880b2310 2023-10-16T11:20:44.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72880cea10 con 0x7f72880b2310 2023-10-16T11:20:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f727dffb700 1 -- 172.21.15.181:0/438799118 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278003a60 con 0x7f72880b2310 2023-10-16T11:20:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.794+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72880a0440 con 0x7f72880b2310 2023-10-16T11:20:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.795+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728809ffc0 con 0x7f72880b2310 2023-10-16T11:20:44.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.796+0000 7f727dffb700 1 -- 172.21.15.181:0/438799118 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7278003bc0 con 0x7f72880b2310 2023-10-16T11:20:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.796+0000 7f727dffb700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f726c06b140 0x7f726c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.796+0000 7f727dffb700 1 -- 172.21.15.181:0/438799118 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f727808afd0 con 0x7f72880b2310 2023-10-16T11:20:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.796+0000 7f728ce97700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f726c06b140 0x7f726c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.796+0000 7f728ce97700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f726c06b140 0x7f726c06d600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f72880c9f90 tx=0x7f7284008040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:44.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:44.801+0000 7f727dffb700 1 -- 172.21.15.181:0/438799118 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f728809ffc0 con 0x7f72880b2310 2023-10-16T11:20:45.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.019+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 --> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f72880b5cc0 con 0x7f726c06b140 2023-10-16T11:20:45.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.020+0000 7f727dffb700 1 -- 172.21.15.181:0/438799118 <== mgr.11814 v2:172.21.15.181:6832/3273985848 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f72880b5cc0 con 0x7f726c06b140 2023-10-16T11:20:45.022 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:20:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T11:20:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:20:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:20:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "backup_fs" 2023-10-16T11:20:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:20:45.023 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:20:45.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f726c06b140 msgr2=0x7f726c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f726c06b140 0x7f726c06d600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f72880c9f90 tx=0x7f7284008040 comp rx=0 tx=0).stop 2023-10-16T11:20:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72880b2310 msgr2=0x7f72880c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72880b2310 0x7f72880c4d50 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fd0 tx=0x7f7278004750 comp rx=0 tx=0).stop 2023-10-16T11:20:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 shutdown_connections 2023-10-16T11:20:45.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f726c06b140 0x7f726c06d600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72880b2710 0x7f728809de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72880b2310 0x7f72880c4d50 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 --2- 172.21.15.181:0/438799118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f728815b7b0 0x7f72880b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 >> 172.21.15.181:0/438799118 conn(0x7f72880b1780 msgr2=0x7f72881595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:45.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 shutdown_connections 2023-10-16T11:20:45.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.023+0000 7f728f8fc700 1 -- 172.21.15.181:0/438799118 wait complete. 2023-10-16T11:20:45.036 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:20:45.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.158+0000 7f943ac8f700 1 Processor -- start 2023-10-16T11:20:45.160 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.159+0000 7f943ac8f700 1 -- start start 2023-10-16T11:20:45.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.159+0000 7f943ac8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 0x7f94340cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.159+0000 7f943ac8f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94340b2ae0 con 0x7f94340cd170 2023-10-16T11:20:45.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.159+0000 7f943ac8f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94340b2dc0 con 0x7f94340ce4f0 2023-10-16T11:20:45.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.159+0000 7f943ac8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94340ce970 con 0x7f943415f150 2023-10-16T11:20:45.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f943922c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f94340cd170 0x7f94340b1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41100/0 (socket says 172.21.15.181:41100) 2023-10-16T11:20:45.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f943922c700 1 -- 172.21.15.181:0/4030523584 learned_addr learned my addr 172.21.15.181:0/4030523584 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:45.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9438a2b700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 0x7f94340cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883592367 0 0) 0x7f94340b2ae0 con 0x7f94340cd170 2023-10-16T11:20:45.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94240035b0 con 0x7f94340cd170 2023-10-16T11:20:45.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636568750 0 0) 0x7f94340b2dc0 con 0x7f94340ce4f0 2023-10-16T11:20:45.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94340b2ae0 con 0x7f94340ce4f0 2023-10-16T11:20:45.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3973821927 0 0) 0x7f94240035b0 con 0x7f94340cd170 2023-10-16T11:20:45.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 msgr2=0x7f94340cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:45.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 0x7f94340cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 >> v1:172.21.15.190:6789/0 conn(0x7f94340ce4f0 legacy=0x7f94340b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:45.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94340cf9d0 con 0x7f94340cd170 2023-10-16T11:20:45.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f942c0019f0 con 0x7f94340cd170 2023-10-16T11:20:45.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9438a2b700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 0x7f94340cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:45.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 >> v1:172.21.15.181:6789/0 conn(0x7f94340cd170 legacy=0x7f94340b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:45.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9424003bc0 0x7f9424003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9424004850 0x7f9424004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 0x7f9424005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94340ce970 con 0x7f9424003bc0 2023-10-16T11:20:45.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942400ba40 con 0x7f9424004850 2023-10-16T11:20:45.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942400bc00 con 0x7f943415f150 2023-10-16T11:20:45.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9438a2b700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9424003bc0 0x7f9424003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9433fff700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9424004850 0x7f9424004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f943922c700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 0x7f9424005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9438a2b700 1 -- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 msgr2=0x7f9424005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:45.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9438a2b700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f943415f150 0x7f9424005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9438a2b700 1 -- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9424004850 msgr2=0x7f9424004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:45.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.160+0000 7f9438a2b700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9424004850 0x7f9424004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f9438a2b700 1 -- 172.21.15.181:0/4030523584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280094a0 con 0x7f9424003bc0 2023-10-16T11:20:45.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f9433fff700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9424004850 0x7f9424004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:45.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f9438a2b700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9424003bc0 0x7f9424003fc0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f9428009d00 tx=0x7f9424005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5570fa8c82dfb05a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:45.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428009e50 con 0x7f9424003bc0 2023-10-16T11:20:45.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942800e9a0 con 0x7f9424003bc0 2023-10-16T11:20:45.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f94337fe700 1 -- 172.21.15.181:0/4030523584 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942800ed30 con 0x7f9424003bc0 2023-10-16T11:20:45.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f943ac8f700 1 -- 172.21.15.181:0/4030523584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9424003bc0 msgr2=0x7f9424003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:45.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f943ac8f700 1 --2- 172.21.15.181:0/4030523584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9424003bc0 0x7f9424003fc0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f9428009d00 tx=0x7f9424005880 comp rx=0 tx=0).stop 2023-10-16T11:20:45.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f943922c700 1 -- 172.21.15.181:0/4030523584 reap_dead start 2023-10-16T11:20:45.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f943ac8f700 1 -- 172.21.15.181:0/4030523584 shutdown_connections 2023-10-16T11:20:45.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f943ac8f700 1 -- 172.21.15.181:0/4030523584 >> 172.21.15.181:0/4030523584 conn(0x7f943409a930 msgr2=0x7f94340d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:45.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f943ac8f700 1 -- 172.21.15.181:0/4030523584 shutdown_connections 2023-10-16T11:20:45.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.161+0000 7f943ac8f700 1 -- 172.21.15.181:0/4030523584 wait complete. 2023-10-16T11:20:45.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 Processor -- start 2023-10-16T11:20:45.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 -- start start 2023-10-16T11:20:45.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9424003bc0 0x7f94341f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94341e5db0 0x7f943415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341f6fd0 0x7f943406fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94341f3800 con 0x7f94341f6fd0 2023-10-16T11:20:45.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9434150330 con 0x7f94341e5db0 2023-10-16T11:20:45.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.163+0000 7f943ac8f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94341f6860 con 0x7f9424003bc0 2023-10-16T11:20:45.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341f6fd0 0x7f943406fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341f6fd0 0x7f943406fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58266/0 (socket says 172.21.15.181:58266) 2023-10-16T11:20:45.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 -- 172.21.15.181:0/77704776 learned_addr learned my addr 172.21.15.181:0/77704776 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:45.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f9438a2b700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9424003bc0 0x7f94341f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f9433fff700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94341e5db0 0x7f943415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 -- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9424003bc0 msgr2=0x7f94341f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:45.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9424003bc0 0x7f94341f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 -- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94341e5db0 msgr2=0x7f943415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:45.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94341e5db0 0x7f943415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:45.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 -- 172.21.15.181:0/77704776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943415d430 con 0x7f94341f6fd0 2023-10-16T11:20:45.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f9433fff700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94341e5db0 0x7f943415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943922c700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341f6fd0 0x7f943406fd30 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f942c001330 tx=0x7f942c007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00ec80 con 0x7f94341f6fd0 2023-10-16T11:20:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00ede0 con 0x7f94341f6fd0 2023-10-16T11:20:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94341f3a00 con 0x7f94341f6fd0 2023-10-16T11:20:45.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c013660 con 0x7f94341f6fd0 2023-10-16T11:20:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.164+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94341f3fc0 con 0x7f94341f6fd0 2023-10-16T11:20:45.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.165+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9434067d20 con 0x7f94341f6fd0 2023-10-16T11:20:45.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.166+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 149) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f942c0137c0 con 0x7f94341f6fd0 2023-10-16T11:20:45.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.166+0000 7f94317fa700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f941806b140 0x7f941806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:45.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.166+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f942c08a460 con 0x7f94341f6fd0 2023-10-16T11:20:45.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.166+0000 7f9438a2b700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f941806b140 0x7f941806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:45.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.166+0000 7f9438a2b700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f941806b140 0x7f941806d600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f94280090b0 tx=0x7f942801f040 comp rx=0 tx=0).ready entity=mgr.11814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:45.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.171+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f942c054d00 con 0x7f94341f6fd0 2023-10-16T11:20:45.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:45.422+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f943406ac10 con 0x7f94341f6fd0 2023-10-16T11:20:46.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.180+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v150) v1 ==== 92+0+0 (secure 0 0 0) 0x7f942c058320 con 0x7f94341f6fd0 2023-10-16T11:20:46.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.185+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f941806b140 msgr2=0x7f941806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.185+0000 7f943ac8f700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f941806b140 0x7f941806d600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f94280090b0 tx=0x7f942801f040 comp rx=0 tx=0).stop 2023-10-16T11:20:46.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.185+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341f6fd0 msgr2=0x7f943406fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f94317fa700 1 -- 172.21.15.181:0/77704776 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 150) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f942c0550d0 con 0x7f94341f6fd0 2023-10-16T11:20:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341f6fd0 0x7f943406fd30 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f942c001330 tx=0x7f942c007c90 comp rx=0 tx=0).stop 2023-10-16T11:20:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 shutdown_connections 2023-10-16T11:20:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f941806b140 0x7f941806d600 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9424003bc0 0x7f94341f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341f6fd0 0x7f943406fd30 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94341e5db0 0x7f943415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 >> 172.21.15.181:0/77704776 conn(0x7f943409a930 msgr2=0x7f9434151de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f94317fa700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9418036700 0x7f941806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 shutdown_connections 2023-10-16T11:20:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.186+0000 7f943ac8f700 1 --2- 172.21.15.181:0/77704776 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f9418036700 0x7f941806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.187+0000 7f943ac8f700 1 -- 172.21.15.181:0/77704776 wait complete. 2023-10-16T11:20:46.207 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:20:46.323 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:46.321+0000 7f2ae673f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:20:46.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.339+0000 7fca95061700 1 Processor -- start 2023-10-16T11:20:46.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.339+0000 7fca95061700 1 -- start start 2023-10-16T11:20:46.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca95061700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fca900cc6e0 0x7fca90071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca95061700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca900ce1f0 con 0x7fca900b2010 2023-10-16T11:20:46.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca95061700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca900ccda0 con 0x7fca900ce680 2023-10-16T11:20:46.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca95061700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca900ccb90 con 0x7fca900cc6e0 2023-10-16T11:20:46.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca8e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fca900cc6e0 0x7fca90071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca8e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fca900cc6e0 0x7fca90071be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58300/0 (socket says 172.21.15.181:58300) 2023-10-16T11:20:46.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca8e59c700 1 -- 172.21.15.181:0/3427745816 learned_addr learned my addr 172.21.15.181:0/3427745816 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:46.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca8e59c700 1 -- 172.21.15.181:0/3427745816 >> v1:172.21.15.190:6789/0 conn(0x7fca900ce680 legacy=0x7fca900ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca8e59c700 1 -- 172.21.15.181:0/3427745816 >> v1:172.21.15.181:6789/0 conn(0x7fca900b2010 legacy=0x7fca900b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca8e59c700 1 -- 172.21.15.181:0/3427745816 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca900727e0 con 0x7fca900cc6e0 2023-10-16T11:20:46.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.340+0000 7fca8e59c700 1 --2- 172.21.15.181:0/3427745816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900cc6e0 0x7fca90071be0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fca800098f0 tx=0x7fca80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7861ff82f25375e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:46.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca8dd9b700 1 -- 172.21.15.181:0/3427745816 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000b8a0 con 0x7fca900cc6e0 2023-10-16T11:20:46.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca8dd9b700 1 -- 172.21.15.181:0/3427745816 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8000ba00 con 0x7fca900cc6e0 2023-10-16T11:20:46.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca8dd9b700 1 -- 172.21.15.181:0/3427745816 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000bd90 con 0x7fca900cc6e0 2023-10-16T11:20:46.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca95061700 1 -- 172.21.15.181:0/3427745816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900cc6e0 msgr2=0x7fca90071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca95061700 1 --2- 172.21.15.181:0/3427745816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900cc6e0 0x7fca90071be0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fca800098f0 tx=0x7fca80009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:46.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca95061700 1 -- 172.21.15.181:0/3427745816 shutdown_connections 2023-10-16T11:20:46.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca95061700 1 --2- 172.21.15.181:0/3427745816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900cc6e0 0x7fca90071be0 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca95061700 1 -- 172.21.15.181:0/3427745816 >> 172.21.15.181:0/3427745816 conn(0x7fca900cd010 msgr2=0x7fca900cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:46.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca95061700 1 -- 172.21.15.181:0/3427745816 shutdown_connections 2023-10-16T11:20:46.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.341+0000 7fca95061700 1 -- 172.21.15.181:0/3427745816 wait complete. 2023-10-16T11:20:46.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 Processor -- start 2023-10-16T11:20:46.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 -- start start 2023-10-16T11:20:46.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900b2010 0x7fca9015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900cc6e0 0x7fca900ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900ce680 0x7fca900aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9015b840 con 0x7fca900b2010 2023-10-16T11:20:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9015bb20 con 0x7fca900ce680 2023-10-16T11:20:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.343+0000 7fca95061700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9015be00 con 0x7fca900cc6e0 2023-10-16T11:20:46.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900b2010 0x7fca9015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900b2010 0x7fca9015d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58304/0 (socket says 172.21.15.181:58304) 2023-10-16T11:20:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 -- 172.21.15.181:0/836871249 learned_addr learned my addr 172.21.15.181:0/836871249 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8e59c700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900cc6e0 0x7fca900ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8f59e700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900ce680 0x7fca900aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 -- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900cc6e0 msgr2=0x7fca900ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900cc6e0 0x7fca900ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 -- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900ce680 msgr2=0x7fca900aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900ce680 0x7fca900aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 -- 172.21.15.181:0/836871249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca80009550 con 0x7fca900b2010 2023-10-16T11:20:46.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca8ed9d700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900b2010 0x7fca9015d030 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c450 tx=0x7fca8400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:46.353 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fdff0bb6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:20:46.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca777fe700 1 -- 172.21.15.181:0/836871249 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840052d0 con 0x7fca900b2010 2023-10-16T11:20:46.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca777fe700 1 -- 172.21.15.181:0/836871249 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca84005430 con 0x7fca900b2010 2023-10-16T11:20:46.354 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9015c0e0 con 0x7fca900b2010 2023-10-16T11:20:46.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca777fe700 1 -- 172.21.15.181:0/836871249 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840038c0 con 0x7fca900b2010 2023-10-16T11:20:46.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.344+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9015d570 con 0x7fca900b2010 2023-10-16T11:20:46.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.345+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca901f0520 con 0x7fca900b2010 2023-10-16T11:20:46.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.346+0000 7fca777fe700 1 -- 172.21.15.181:0/836871249 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 150) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fca84003a20 con 0x7fca900b2010 2023-10-16T11:20:46.355 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.347+0000 7fca777fe700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 0x7fca7c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.347+0000 7fca8e59c700 1 -- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 msgr2=0x7fca7c06d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:46.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.347+0000 7fca777fe700 1 -- 172.21.15.181:0/836871249 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fca84089620 con 0x7fca900b2010 2023-10-16T11:20:46.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.347+0000 7fca8e59c700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 0x7fca7c06d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:46.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.351+0000 7fca777fe700 1 -- 172.21.15.181:0/836871249 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fca84053f20 con 0x7fca900b2010 2023-10-16T11:20:46.390 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:46.389+0000 7f2ae673f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:46.414 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:46.413+0000 7fdff0bb6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:46.462 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:46.461+0000 7f2ae673f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:20:46.489 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:46.487+0000 7fdff0bb6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:20:46.539 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:46.538+0000 7f2ae673f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:20:46.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.548+0000 7fca8e59c700 1 -- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 msgr2=0x7fca7c06d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:46.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.548+0000 7fca8e59c700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 0x7fca7c06d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:46.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.553+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fca900b6510 con 0x7fca900b2010 2023-10-16T11:20:46.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.556+0000 7fca777fe700 1 -- 172.21.15.181:0/836871249 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v83)=0 v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fca84017020 con 0x7fca900b2010 2023-10-16T11:20:46.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 msgr2=0x7fca7c06d650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:46.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 0x7fca7c06d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900b2010 msgr2=0x7fca9015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900b2010 0x7fca9015d030 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c450 tx=0x7fca8400c760 comp rx=0 tx=0).stop 2023-10-16T11:20:46.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 shutdown_connections 2023-10-16T11:20:46.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fca7c06b190 0x7fca7c06d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900ce680 0x7fca900aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca900b2010 0x7fca9015d030 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 --2- 172.21.15.181:0/836871249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900cc6e0 0x7fca900ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 >> 172.21.15.181:0/836871249 conn(0x7fca900cd010 msgr2=0x7fca900b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:46.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 shutdown_connections 2023-10-16T11:20:46.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.559+0000 7fca95061700 1 -- 172.21.15.181:0/836871249 wait complete. 2023-10-16T11:20:46.570 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:46.569+0000 7fdff0bb6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:20:46.572 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:20:46.681 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:46.679+0000 7f2ae673f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:20:46.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.698+0000 7fd360714700 1 Processor -- start 2023-10-16T11:20:46.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.698+0000 7fd360714700 1 -- start start 2023-10-16T11:20:46.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.699+0000 7fd360714700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3580b2010 0x7fd3580cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.699+0000 7fd360714700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd3580ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.699+0000 7fd360714700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3580ce1f0 con 0x7fd358071be0 2023-10-16T11:20:46.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.699+0000 7fd360714700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd358072090 con 0x7fd3580b2010 2023-10-16T11:20:46.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.699+0000 7fd360714700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580ccda0 con 0x7fd3580ce680 2023-10-16T11:20:46.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35e4b0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3580b2010 0x7fd3580cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35ecb1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd3580ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35dcaf700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd358071be0 0x7fd3580b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:53380/0 (socket says 172.21.15.181:53380) 2023-10-16T11:20:46.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35dcaf700 1 -- 172.21.15.181:0/2823457443 learned_addr learned my addr 172.21.15.181:0/2823457443 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:46.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35e4b0700 1 -- 172.21.15.181:0/2823457443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 msgr2=0x7fd3580ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:46.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35e4b0700 1 --2- 172.21.15.181:0/2823457443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd3580ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35e4b0700 1 -- 172.21.15.181:0/2823457443 >> v1:172.21.15.190:6789/0 conn(0x7fd358071be0 legacy=0x7fd3580b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35e4b0700 1 -- 172.21.15.181:0/2823457443 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580727e0 con 0x7fd3580b2010 2023-10-16T11:20:46.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35d4ae700 1 -- 172.21.15.181:0/2823457443 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211105320 0 0) 0x7fd3580ce1f0 con 0x7fd358071be0 2023-10-16T11:20:46.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35e4b0700 1 --2- 172.21.15.181:0/2823457443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 0x7fd3580cc6e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd35400bae0 tx=0x7fd35400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d76fea4d8b42d060 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:46.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35d4ae700 1 -- 172.21.15.181:0/2823457443 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35400dc70 con 0x7fd3580b2010 2023-10-16T11:20:46.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35d4ae700 1 -- 172.21.15.181:0/2823457443 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35400ddd0 con 0x7fd3580b2010 2023-10-16T11:20:46.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.700+0000 7fd35d4ae700 1 -- 172.21.15.181:0/2823457443 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354010690 con 0x7fd3580b2010 2023-10-16T11:20:46.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 -- 172.21.15.181:0/2823457443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 msgr2=0x7fd3580cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 --2- 172.21.15.181:0/2823457443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 0x7fd3580cc6e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd35400bae0 tx=0x7fd35400bdf0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 -- 172.21.15.181:0/2823457443 shutdown_connections 2023-10-16T11:20:46.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 --2- 172.21.15.181:0/2823457443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd3580ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 --2- 172.21.15.181:0/2823457443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 0x7fd3580cc6e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 -- 172.21.15.181:0/2823457443 >> 172.21.15.181:0/2823457443 conn(0x7fd3580cd010 msgr2=0x7fd3580cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:46.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 -- 172.21.15.181:0/2823457443 shutdown_connections 2023-10-16T11:20:46.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.701+0000 7fd360714700 1 -- 172.21.15.181:0/2823457443 wait complete. 2023-10-16T11:20:46.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.702+0000 7fd360714700 1 Processor -- start 2023-10-16T11:20:46.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.702+0000 7fd360714700 1 -- start start 2023-10-16T11:20:46.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd360714700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd358071be0 0x7fd3581f0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd360714700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 0x7fd358156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd360714700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd358158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd360714700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581565b0 con 0x7fd358071be0 2023-10-16T11:20:46.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd360714700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd358157d70 con 0x7fd3580b2010 2023-10-16T11:20:46.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd360714700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581f0bd0 con 0x7fd3580ce680 2023-10-16T11:20:46.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd358071be0 0x7fd3581f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd358071be0 0x7fd3581f0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58364/0 (socket says 172.21.15.181:58364) 2023-10-16T11:20:46.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 -- 172.21.15.181:0/3145484579 learned_addr learned my addr 172.21.15.181:0/3145484579 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:46.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35dcaf700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 0x7fd358156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35ecb1700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd358158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 -- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 msgr2=0x7fd358158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd358158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 -- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 msgr2=0x7fd358156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 0x7fd358156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.703+0000 7fd35e4b0700 1 -- 172.21.15.181:0/3145484579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35400b740 con 0x7fd358071be0 2023-10-16T11:20:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.704+0000 7fd35e4b0700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd358071be0 0x7fd3581f0690 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7fd358070460 tx=0x7fd354004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.704+0000 7fd34effd700 1 -- 172.21.15.181:0/3145484579 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354003b70 con 0x7fd358071be0 2023-10-16T11:20:46.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.704+0000 7fd34effd700 1 -- 172.21.15.181:0/3145484579 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd354003cd0 con 0x7fd358071be0 2023-10-16T11:20:46.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.704+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3581f3e20 con 0x7fd358071be0 2023-10-16T11:20:46.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.704+0000 7fd34effd700 1 -- 172.21.15.181:0/3145484579 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354010690 con 0x7fd358071be0 2023-10-16T11:20:46.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.704+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd358155f60 con 0x7fd358071be0 2023-10-16T11:20:46.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.705+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd358156890 con 0x7fd358071be0 2023-10-16T11:20:46.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.705+0000 7fd34effd700 1 -- 172.21.15.181:0/3145484579 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 150) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fd35402c430 con 0x7fd358071be0 2023-10-16T11:20:46.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.706+0000 7fd34effd700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 0x7fd33c06d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:46.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.706+0000 7fd35dcaf700 1 -- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 msgr2=0x7fd33c06d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:46.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.706+0000 7fd34effd700 1 -- 172.21.15.181:0/3145484579 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fd35408bc30 con 0x7fd358071be0 2023-10-16T11:20:46.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.706+0000 7fd35dcaf700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 0x7fd33c06d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:46.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.710+0000 7fd34effd700 1 -- 172.21.15.181:0/3145484579 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd354056450 con 0x7fd358071be0 2023-10-16T11:20:46.727 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:46.720+0000 7fdff0bb6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:20:46.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.907+0000 7fd35dcaf700 1 -- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 msgr2=0x7fd33c06d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:46.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.907+0000 7fd35dcaf700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 0x7fd33c06d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:46.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.913+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fd3580b6510 con 0x7fd358071be0 2023-10-16T11:20:46.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.916+0000 7fd34effd700 1 -- 172.21.15.181:0/3145484579 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v84)=0 v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd354059a70 con 0x7fd358071be0 2023-10-16T11:20:46.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 msgr2=0x7fd33c06d650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:46.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 0x7fd33c06d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd358071be0 msgr2=0x7fd3581f0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:46.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd358071be0 0x7fd3581f0690 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7fd358070460 tx=0x7fd354004750 comp rx=0 tx=0).stop 2023-10-16T11:20:46.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 shutdown_connections 2023-10-16T11:20:46.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7fd33c06b190 0x7fd33c06d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3580ce680 0x7fd358158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd358071be0 0x7fd3581f0690 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 --2- 172.21.15.181:0/3145484579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3580b2010 0x7fd358156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:46.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 >> 172.21.15.181:0/3145484579 conn(0x7fd3580cd010 msgr2=0x7fd3580b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:46.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 shutdown_connections 2023-10-16T11:20:46.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:46.919+0000 7fd360714700 1 -- 172.21.15.181:0/3145484579 wait complete. 2023-10-16T11:20:46.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:46.938 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-16T11:20:46.959 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-10-16T11:20:47.004 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:47.003+0000 7f2ae673f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:20:47.066 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:47.066+0000 7fdff0bb6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:20:47.121 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:47.119+0000 7f2ae673f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:20:47.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.123+0000 7f10a9070700 1 Processor -- start 2023-10-16T11:20:47.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.123+0000 7f10a9070700 1 -- start start 2023-10-16T11:20:47.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a9070700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10a415da70 0x7f10a40ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:47.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a9070700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40cef00 con 0x7f10a40ce6a0 2023-10-16T11:20:47.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a9070700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40cec20 con 0x7f10a406afb0 2023-10-16T11:20:47.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a9070700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a4071c80 con 0x7f10a415da70 2023-10-16T11:20:47.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f10a40ce6a0 0x7f10a406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41264/0 (socket says 172.21.15.181:41264) 2023-10-16T11:20:47.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a259c700 1 -- 172.21.15.181:0/247761975 learned_addr learned my addr 172.21.15.181:0/247761975 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:47.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a2d9d700 1 --2- 172.21.15.181:0/247761975 >> v2:172.21.15.190:3300/0 conn(0x7f10a415da70 0x7f10a40ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:47.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a1d9b700 1 -- 172.21.15.181:0/247761975 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130793011 0 0) 0x7f10a40cef00 con 0x7f10a40ce6a0 2023-10-16T11:20:47.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.124+0000 7f10a1d9b700 1 -- 172.21.15.181:0/247761975 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c0035b0 con 0x7f10a40ce6a0 2023-10-16T11:20:47.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a2d9d700 1 -- 172.21.15.181:0/247761975 >> v1:172.21.15.190:6789/0 conn(0x7f10a406afb0 legacy=0x7f10a41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:47.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a1d9b700 1 -- 172.21.15.181:0/247761975 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677642522 0 0) 0x7f10a40cec20 con 0x7f10a406afb0 2023-10-16T11:20:47.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a2d9d700 1 -- 172.21.15.181:0/247761975 >> v1:172.21.15.181:6789/0 conn(0x7f10a40ce6a0 legacy=0x7f10a406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:47.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a2d9d700 1 -- 172.21.15.181:0/247761975 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a4071f60 con 0x7f10a415da70 2023-10-16T11:20:47.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a2d9d700 1 --2- 172.21.15.181:0/247761975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a415da70 0x7f10a40ce160 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1094009840 tx=0x7f1094009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d721833655d83e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:47.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a1d9b700 1 -- 172.21.15.181:0/247761975 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094011830 con 0x7f10a415da70 2023-10-16T11:20:47.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a1d9b700 1 -- 172.21.15.181:0/247761975 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1094011990 con 0x7f10a415da70 2023-10-16T11:20:47.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a1d9b700 1 -- 172.21.15.181:0/247761975 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094011d20 con 0x7f10a415da70 2023-10-16T11:20:47.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a9070700 1 -- 172.21.15.181:0/247761975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a415da70 msgr2=0x7f10a40ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:47.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.125+0000 7f10a9070700 1 --2- 172.21.15.181:0/247761975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a415da70 0x7f10a40ce160 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1094009840 tx=0x7f1094009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:47.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.126+0000 7f10a9070700 1 -- 172.21.15.181:0/247761975 shutdown_connections 2023-10-16T11:20:47.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.126+0000 7f10a9070700 1 --2- 172.21.15.181:0/247761975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a415da70 0x7f10a40ce160 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:47.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.126+0000 7f10a9070700 1 -- 172.21.15.181:0/247761975 >> 172.21.15.181:0/247761975 conn(0x7f10a40cd010 msgr2=0x7f10a40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:47.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.126+0000 7f10a9070700 1 -- 172.21.15.181:0/247761975 shutdown_connections 2023-10-16T11:20:47.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.126+0000 7f10a9070700 1 -- 172.21.15.181:0/247761975 wait complete. 2023-10-16T11:20:47.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.127+0000 7f10a9070700 1 Processor -- start 2023-10-16T11:20:47.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.127+0000 7f10a9070700 1 -- start start 2023-10-16T11:20:47.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a9070700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a406afb0 0x7f10a4159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:47.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a9070700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10a40ce6a0 0x7f10a415dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:47.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a9070700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a415da70 0x7f10a40b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:47.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a9070700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40cb610 con 0x7f10a415da70 2023-10-16T11:20:47.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a9070700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40af5c0 con 0x7f10a406afb0 2023-10-16T11:20:47.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a9070700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40cac70 con 0x7f10a40ce6a0 2023-10-16T11:20:47.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a415da70 0x7f10a40b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a406afb0 0x7f10a4159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a406afb0 0x7f10a4159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:35240/0 (socket says 172.21.15.181:35240) 2023-10-16T11:20:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a415da70 0x7f10a40b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58426/0 (socket says 172.21.15.181:58426) 2023-10-16T11:20:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 -- 172.21.15.181:0/315905299 learned_addr learned my addr 172.21.15.181:0/315905299 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:47.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10a40ce6a0 0x7f10a415dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10a40ce6a0 msgr2=0x7f10a415dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10a40ce6a0 0x7f10a415dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a415da70 msgr2=0x7f10a40b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a415da70 0x7f10a40b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:47.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a2d9d700 1 -- 172.21.15.181:0/315905299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940094a0 con 0x7f10a406afb0 2023-10-16T11:20:47.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.128+0000 7f10a359e700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a415da70 0x7f10a40b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:47.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.129+0000 7f10a2d9d700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10a406afb0 0x7f10a4159990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f10940090b0 tx=0x7f1094004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:47.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.129+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109401a7e0 con 0x7f10a406afb0 2023-10-16T11:20:47.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.129+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109401a940 con 0x7f10a406afb0 2023-10-16T11:20:47.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.129+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109401acd0 con 0x7f10a406afb0 2023-10-16T11:20:47.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.129+0000 7f10a9070700 1 -- 172.21.15.181:0/315905299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a40caf50 con 0x7f10a406afb0 2023-10-16T11:20:47.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.129+0000 7f10a9070700 1 -- 172.21.15.181:0/315905299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a415e510 con 0x7f10a406afb0 2023-10-16T11:20:47.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.130+0000 7f10a9070700 1 -- 172.21.15.181:0/315905299 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a406f6c0 con 0x7f10a406afb0 2023-10-16T11:20:47.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.130+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 150) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f1094022070 con 0x7f10a406afb0 2023-10-16T11:20:47.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.130+0000 7f108b7fe700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 0x7f109006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:47.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.131+0000 7f10a259c700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 msgr2=0x7f109006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:47.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.131+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f109408ad60 con 0x7f10a406afb0 2023-10-16T11:20:47.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.131+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 0x7f109006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:47.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.135+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1094055590 con 0x7f10a406afb0 2023-10-16T11:20:47.191 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:47.190+0000 7fdff0bb6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:20:47.198 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:47.197+0000 7f2ae673f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:20:47.267 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:47.266+0000 7fdff0bb6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:20:47.277 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:47.276+0000 7f2ae673f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:20:47.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.332+0000 7f10a259c700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 msgr2=0x7f109006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:47.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.332+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 0x7f109006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:47.350 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:47.349+0000 7fdff0bb6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:20:47.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.366+0000 7f10a9070700 1 -- 172.21.15.181:0/315905299 --> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f10a415de50 con 0x7f109006b190 2023-10-16T11:20:47.645 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:47.643+0000 7f2ae673f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:20:47.720 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:47.719+0000 7fdff0bb6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:20:47.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.733+0000 7f10a259c700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 msgr2=0x7f109006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:47.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:47.733+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 0x7f109006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:20:48.214 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:48.212+0000 7f2ae673f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:20:48.283 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:48.282+0000 7f2ae673f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:20:48.290 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:48.290+0000 7fdff0bb6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:20:48.361 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:48.360+0000 7fdff0bb6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:20:48.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:48.534+0000 7f10a259c700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 msgr2=0x7f109006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:48.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:48.534+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 0x7f109006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:20:48.911 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:48.910+0000 7f2ae673f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:48.988 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:48.987+0000 7f2ae673f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:20:48.993 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:48.992+0000 7fdff0bb6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:49.071 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:49.070+0000 7fdff0bb6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:20:49.134 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:49.132+0000 7f2ae673f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:49.218 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:49.218+0000 7fdff0bb6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:49.363 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:49.362+0000 7f2ae673f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:20:49.437 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:49.436+0000 7f2ae673f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:20:49.448 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:49.448+0000 7fdff0bb6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:20:49.523 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:49.522+0000 7fdff0bb6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:20:49.703 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:49.702+0000 7f2ae673f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:20:49.791 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:49.790+0000 7fdff0bb6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:20:49.926 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:49.924+0000 7f2ae673f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:20:50.015 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:50.014+0000 7fdff0bb6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:20:50.078 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:50.077+0000 7f2ae673f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:20:50.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:50.135+0000 7f10a259c700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 msgr2=0x7f109006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3273985848 2023-10-16T11:20:50.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:50.135+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 0x7f109006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:20:50.171 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:50.170+0000 7fdff0bb6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:20:50.195 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:50.194+0000 7f2ae673f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:20:50.285 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:50.284+0000 7fdff0bb6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:20:50.353 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:50.351+0000 7f2ae673f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:20:50.443 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:50.442+0000 7fdff0bb6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:20:50.616 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:50.615+0000 7f2ae673f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:50.708 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:50.707+0000 7fdff0bb6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:50.776 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:50.775+0000 7f2ae673f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:20:50.871 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:50.871+0000 7fdff0bb6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:20:51.302 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:51.300+0000 7f2ae673f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:20:51.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:51.371+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 151) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f1094055960 con 0x7f10a406afb0 2023-10-16T11:20:51.407 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.406+0000 7fdff0bb6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:20:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:51.430+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mgrmap(e 152) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f10940610c0 con 0x7f10a406afb0 2023-10-16T11:20:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:51.430+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 msgr2=0x7f109006d650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:51.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:51.430+0000 7f108b7fe700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/3273985848,v1:172.21.15.181:6833/3273985848] conn(0x7f109006b190 0x7f109006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:51.657 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.656+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.658 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.656+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.658 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.656+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.658 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.656+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.659 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.656+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.659 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.657+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.659 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.657+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.659 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.657+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.660 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.657+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.660 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.657+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.668 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.669 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.669 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.669 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.670 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf63390700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.670 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.670 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.671 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.671 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:51.671 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:51.667+0000 7fdf66396700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:52.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.431+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mon.1 v2:172.21.15.190:3300/0 9 ==== mgrmap(e 153) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f109402fd70 con 0x7f10a406afb0 2023-10-16T11:20:52.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.432+0000 7f108b7fe700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f109001d000 0x7f109001d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.432+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 --> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1090071b60 con 0x7f109001d000 2023-10-16T11:20:52.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.433+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f109001d000 0x7f109001d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:52.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.435+0000 7f10a259c700 1 --2- 172.21.15.181:0/315905299 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f109001d000 0x7f109001d3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10a41f6690 tx=0x7f1098009040 comp rx=0 tx=0).ready entity=mgr.10909 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:52.439 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:52.439+0000 7fdf6d3a4700 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-10-16T11:20:52.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.439+0000 7f108b7fe700 1 -- 172.21.15.181:0/315905299 <== mgr.10909 v2:172.21.15.181:6832/1878395836 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7f1090071b60 con 0x7f109001d000 2023-10-16T11:20:52.442 INFO:teuthology.orchestra.run.smithi181.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-10-16T11:20:52.446 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-16T11:20:52.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:52.448 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-16T11:20:52.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:20:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.633+0000 7fb303fe9700 1 Processor -- start 2023-10-16T11:20:52.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.633+0000 7fb303fe9700 1 -- start start 2023-10-16T11:20:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb303fe9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb2fc15f150 0x7fb2fc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:52.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb303fe9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb303fe9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc0b2ae0 con 0x7fb2fc0b14a0 2023-10-16T11:20:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb303fe9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0ce970 con 0x7fb2fc15f150 2023-10-16T11:20:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb303fe9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0b2dc0 con 0x7fb2fc0ce4f0 2023-10-16T11:20:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb301d85700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb2fc15f150 0x7fb2fc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:52.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb301584700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb301d85700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb2fc15f150 0x7fb2fc0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:35464/0 (socket says 172.21.15.181:35464) 2023-10-16T11:20:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.634+0000 7fb301d85700 1 -- 172.21.15.181:0/773720333 learned_addr learned my addr 172.21.15.181:0/773720333 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb301d85700 1 -- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 msgr2=0x7fb2fc0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb301d85700 1 --2- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:52.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb301d85700 1 -- 172.21.15.181:0/773720333 >> v1:172.21.15.190:6789/0 conn(0x7fb2fc0b14a0 legacy=0x7fb2fc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb301d85700 1 -- 172.21.15.181:0/773720333 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc0cf9d0 con 0x7fb2fc15f150 2023-10-16T11:20:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb301584700 1 --2- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb301d85700 1 --2- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0cdfb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80098f0 tx=0x7fb2f8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2062e4e7f7f482c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:52.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb300d83700 1 -- 172.21.15.181:0/773720333 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8011830 con 0x7fb2fc15f150 2023-10-16T11:20:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb300d83700 1 -- 172.21.15.181:0/773720333 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f8011990 con 0x7fb2fc15f150 2023-10-16T11:20:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.635+0000 7fb300d83700 1 -- 172.21.15.181:0/773720333 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8011d20 con 0x7fb2fc15f150 2023-10-16T11:20:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 -- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc15f150 msgr2=0x7fb2fc0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 --2- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0cdfb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80098f0 tx=0x7fb2f8009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:52.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 -- 172.21.15.181:0/773720333 shutdown_connections 2023-10-16T11:20:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 --2- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0cdfb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 --2- 172.21.15.181:0/773720333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 -- 172.21.15.181:0/773720333 >> 172.21.15.181:0/773720333 conn(0x7fb2fc09a930 msgr2=0x7fb2fc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 -- 172.21.15.181:0/773720333 shutdown_connections 2023-10-16T11:20:52.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.636+0000 7fb303fe9700 1 -- 172.21.15.181:0/773720333 wait complete. 2023-10-16T11:20:52.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.637+0000 7fb303fe9700 1 Processor -- start 2023-10-16T11:20:52.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.637+0000 7fb303fe9700 1 -- start start 2023-10-16T11:20:52.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb303fe9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc0b14a0 0x7fb2fc158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:52.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb303fe9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:52.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb303fe9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:52.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb303fe9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc158db0 con 0x7fb2fc15f150 2023-10-16T11:20:52.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb303fe9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0c70b0 con 0x7fb2fc0b14a0 2023-10-16T11:20:52.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb303fe9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0c72e0 con 0x7fb2fc0ce4f0 2023-10-16T11:20:52.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:52.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58656/0 (socket says 172.21.15.181:58656) 2023-10-16T11:20:52.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 -- 172.21.15.181:0/3709320146 learned_addr learned my addr 172.21.15.181:0/3709320146 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:52.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb301584700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:52.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb301d85700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc0b14a0 0x7fb2fc158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:52.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 -- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 msgr2=0x7fb2fc06d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:52.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc06d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:52.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 -- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc0b14a0 msgr2=0x7fb2fc158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc0b14a0 0x7fb2fc158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 -- 172.21.15.181:0/3709320146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0094a0 con 0x7fb2fc15f150 2023-10-16T11:20:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb302586700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0ae2e0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400cb50 tx=0x7fb2f400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4003c00 con 0x7fb2fc15f150 2023-10-16T11:20:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f4003d60 con 0x7fb2fc15f150 2023-10-16T11:20:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f8009550 con 0x7fb2fc15f150 2023-10-16T11:20:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.638+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400ab70 con 0x7fb2fc15f150 2023-10-16T11:20:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.639+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc0c78a0 con 0x7fb2fc15f150 2023-10-16T11:20:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.639+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc067d20 con 0x7fb2fc15f150 2023-10-16T11:20:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.640+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 153) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fb2f400acd0 con 0x7fb2fc15f150 2023-10-16T11:20:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.640+0000 7fb2f27fc700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb2e006ada0 0x7fb2e006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.640+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fb2f4089a00 con 0x7fb2fc15f150 2023-10-16T11:20:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.640+0000 7fb301d85700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb2e006ada0 0x7fb2e006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.641+0000 7fb301d85700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb2e006ada0 0x7fb2e006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000c00 tx=0x7fb2f8008030 comp rx=0 tx=0).ready entity=mgr.10909 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:52.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.645+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb2f40546b0 con 0x7fb2fc15f150 2023-10-16T11:20:52.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:52.893+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb2fc06ea40 con 0x7fb2fc15f150 2023-10-16T11:20:53.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.436+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v154) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb2f4057cd0 con 0x7fb2fc15f150 2023-10-16T11:20:53.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb2f27fc700 1 -- 172.21.15.181:0/3709320146 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 154) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb2f4010dc0 con 0x7fb2fc15f150 2023-10-16T11:20:53.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb2e006ada0 msgr2=0x7fb2e006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:53.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb2e006ada0 0x7fb2e006d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000c00 tx=0x7fb2f8008030 comp rx=0 tx=0).stop 2023-10-16T11:20:53.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2fc15f150 msgr2=0x7fb2fc0ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:53.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0ae2e0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400cb50 tx=0x7fb2f400cf10 comp rx=0 tx=0).stop 2023-10-16T11:20:53.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 shutdown_connections 2023-10-16T11:20:53.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2fc15f150 0x7fb2fc0ae2e0 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2fc0b14a0 0x7fb2fc158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb2e006ada0 0x7fb2e006d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 --2- 172.21.15.181:0/3709320146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2fc0ce4f0 0x7fb2fc06d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 >> 172.21.15.181:0/3709320146 conn(0x7fb2fc09a930 msgr2=0x7fb2fc15aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:53.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 shutdown_connections 2023-10-16T11:20:53.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.441+0000 7fb303fe9700 1 -- 172.21.15.181:0/3709320146 wait complete. 2023-10-16T11:20:53.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:20:53.456 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:20:53.498 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:20:53.537 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:20:53.537 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:20:53.538 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:53.555 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:20:53.556 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:53.581 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:53.579+0000 7fce16c49200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:20:53.586 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:53.585+0000 7f8a4cb6b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:20:53.603 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:53.646 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:53.644+0000 7fce16c49200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:53.654 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:53.653+0000 7f8a4cb6b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:53.719 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:53.717+0000 7fce16c49200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:20:53.730 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:53.729+0000 7f8a4cb6b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:20:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.767+0000 7f70d961f700 1 Processor -- start 2023-10-16T11:20:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.767+0000 7f70d961f700 1 -- start start 2023-10-16T11:20:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d961f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70d406d5f0 0x7f70d40d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d961f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70d4160d20 0x7f70d41611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d961f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d41616f0 con 0x7f70d4161b60 2023-10-16T11:20:53.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d961f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d4162800 con 0x7f70d406d5f0 2023-10-16T11:20:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d961f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d40d43e0 con 0x7f70d4160d20 2023-10-16T11:20:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70d406d5f0 0x7f70d40d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70d406d5f0 0x7f70d40d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58704/0 (socket says 172.21.15.181:58704) 2023-10-16T11:20:53.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 -- 172.21.15.181:0/1008535994 learned_addr learned my addr 172.21.15.181:0/1008535994 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:53.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d37fe700 1 --2- 172.21.15.181:0/1008535994 >> v2:172.21.15.190:3300/0 conn(0x7f70d4160d20 0x7f70d41611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:53.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 -- 172.21.15.181:0/1008535994 >> v2:172.21.15.190:3300/0 conn(0x7f70d4160d20 msgr2=0x7f70d41611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:53.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 --2- 172.21.15.181:0/1008535994 >> v2:172.21.15.190:3300/0 conn(0x7f70d4160d20 0x7f70d41611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 -- 172.21.15.181:0/1008535994 >> v1:172.21.15.190:6789/0 conn(0x7f70d4161b60 legacy=0x7f70d4161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 -- 172.21.15.181:0/1008535994 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d40d2560 con 0x7f70d406d5f0 2023-10-16T11:20:53.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.768+0000 7f70d27fc700 1 --2- 172.21.15.181:0/1008535994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d406d5f0 0x7f70d40d2020 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f70d40d48d0 tx=0x7f70c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccec15c40b7b072 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d1ffb700 1 -- 172.21.15.181:0/1008535994 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c800b870 con 0x7f70d406d5f0 2023-10-16T11:20:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d1ffb700 1 -- 172.21.15.181:0/1008535994 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c800b9d0 con 0x7f70d406d5f0 2023-10-16T11:20:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d1ffb700 1 -- 172.21.15.181:0/1008535994 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c800bd60 con 0x7f70d406d5f0 2023-10-16T11:20:53.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 -- 172.21.15.181:0/1008535994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d406d5f0 msgr2=0x7f70d40d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:53.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 --2- 172.21.15.181:0/1008535994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d406d5f0 0x7f70d40d2020 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f70d40d48d0 tx=0x7f70c8009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 -- 172.21.15.181:0/1008535994 shutdown_connections 2023-10-16T11:20:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 --2- 172.21.15.181:0/1008535994 >> v2:172.21.15.190:3300/0 conn(0x7f70d4160d20 0x7f70d41611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 --2- 172.21.15.181:0/1008535994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d406d5f0 0x7f70d40d2020 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 -- 172.21.15.181:0/1008535994 >> 172.21.15.181:0/1008535994 conn(0x7f70d409cba0 msgr2=0x7f70d40d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:53.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 -- 172.21.15.181:0/1008535994 shutdown_connections 2023-10-16T11:20:53.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.769+0000 7f70d961f700 1 -- 172.21.15.181:0/1008535994 wait complete. 2023-10-16T11:20:53.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 Processor -- start 2023-10-16T11:20:53.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 -- start start 2023-10-16T11:20:53.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70d406d5f0 0x7f70d40b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:53.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d4160d20 0x7f70d409f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:53.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70d4161b60 0x7f70d415def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:53.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d415edc0 con 0x7f70d4160d20 2023-10-16T11:20:53.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40b65a0 con 0x7f70d4161b60 2023-10-16T11:20:53.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.771+0000 7f70d961f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d409e400 con 0x7f70d406d5f0 2023-10-16T11:20:53.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d4160d20 0x7f70d409f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:53.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d4160d20 0x7f70d409f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58710/0 (socket says 172.21.15.181:58710) 2023-10-16T11:20:53.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 -- 172.21.15.181:0/291209258 learned_addr learned my addr 172.21.15.181:0/291209258 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:53.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d2ffd700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70d406d5f0 0x7f70d40b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:53.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d37fe700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70d4161b60 0x7f70d415def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:53.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 -- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70d406d5f0 msgr2=0x7f70d40b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:53.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70d406d5f0 0x7f70d40b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 -- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70d4161b60 msgr2=0x7f70d415def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:53.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70d4161b60 0x7f70d415def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:53.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 -- 172.21.15.181:0/291209258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8009550 con 0x7f70d4160d20 2023-10-16T11:20:53.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d27fc700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d4160d20 0x7f70d409f740 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f70c8000c00 tx=0x7f70c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:53.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70bb7fe700 1 -- 172.21.15.181:0/291209258 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8011830 con 0x7f70d4160d20 2023-10-16T11:20:53.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70bb7fe700 1 -- 172.21.15.181:0/291209258 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c801e070 con 0x7f70d4160d20 2023-10-16T11:20:53.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70bb7fe700 1 -- 172.21.15.181:0/291209258 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8011bd0 con 0x7f70d4160d20 2023-10-16T11:20:53.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d409e6e0 con 0x7f70d4160d20 2023-10-16T11:20:53.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.772+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d40b95c0 con 0x7f70d4160d20 2023-10-16T11:20:53.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.773+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d40b6aa0 con 0x7f70d4160d20 2023-10-16T11:20:53.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.774+0000 7f70bb7fe700 1 -- 172.21.15.181:0/291209258 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 154) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f70c8011d30 con 0x7f70d4160d20 2023-10-16T11:20:53.789 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.774+0000 7f70bb7fe700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 0x7f70c006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:53.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.774+0000 7f70d2ffd700 1 -- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 msgr2=0x7f70c006d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:53.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.774+0000 7f70bb7fe700 1 -- 172.21.15.181:0/291209258 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7f70c8020070 con 0x7f70d4160d20 2023-10-16T11:20:53.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.774+0000 7f70d2ffd700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 0x7f70c006d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:53.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.779+0000 7f70bb7fe700 1 -- 172.21.15.181:0/291209258 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f70d40b6aa0 con 0x7f70d4160d20 2023-10-16T11:20:53.797 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:53.795+0000 7fce16c49200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:20:53.813 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:53.813+0000 7f8a4cb6b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:20:53.939 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:53.937+0000 7fce16c49200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:20:53.969 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:53.968+0000 7f8a4cb6b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:20:53.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.975+0000 7f70d2ffd700 1 -- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 msgr2=0x7f70c006d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:53.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.975+0000 7f70d2ffd700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 0x7f70c006d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:53.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.995+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70d40b6aa0 con 0x7f70d4160d20 2023-10-16T11:20:53.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.997+0000 7f70bb7fe700 1 -- 172.21.15.181:0/291209258 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+21709 (secure 0 0 0) 0x7f70d40b6aa0 con 0x7f70d4160d20 2023-10-16T11:20:53.998 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:53.998 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":241,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:51.419932+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:54.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 msgr2=0x7f70c006d210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:54.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 0x7f70c006d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d4160d20 msgr2=0x7f70d409f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d4160d20 0x7f70d409f740 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f70c8000c00 tx=0x7f70c8004750 comp rx=0 tx=0).stop 2023-10-16T11:20:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 shutdown_connections 2023-10-16T11:20:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f70c006ad50 0x7f70c006d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70d406d5f0 0x7f70d40b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70d4161b60 0x7f70d415def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 --2- 172.21.15.181:0/291209258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d4160d20 0x7f70d409f740 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 >> 172.21.15.181:0/291209258 conn(0x7f70d409cba0 msgr2=0x7f70d415f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 shutdown_connections 2023-10-16T11:20:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:53.999+0000 7f70d961f700 1 -- 172.21.15.181:0/291209258 wait complete. 2023-10-16T11:20:54.013 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:20:23.899617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '235', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:20:24.428675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:20:36.190463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '239', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:20:37.509605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '240', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:20:54.014 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:20:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.179+0000 7fcd1d4f5700 1 Processor -- start 2023-10-16T11:20:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.179+0000 7fcd1d4f5700 1 -- start start 2023-10-16T11:20:54.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd1d4f5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcd180d01d0 0x7fcd180ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd1d4f5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd180cf580 con 0x7fcd180d1500 2023-10-16T11:20:54.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd1d4f5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd180ce970 con 0x7fcd1815ea40 2023-10-16T11:20:54.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd1d4f5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd180d4290 con 0x7fcd180d01d0 2023-10-16T11:20:54.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd177fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcd180d1500 0x7fcd180d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41602/0 (socket says 172.21.15.181:41602) 2023-10-16T11:20:54.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd177fe700 1 -- 172.21.15.181:0/1733736963 learned_addr learned my addr 172.21.15.181:0/1733736963 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:54.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd16ffd700 1 --2- 172.21.15.181:0/1733736963 >> v2:172.21.15.190:3300/0 conn(0x7fcd180d01d0 0x7fcd180ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1733736963 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220925205 0 0) 0x7fcd180cf580 con 0x7fcd180d1500 2023-10-16T11:20:54.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1733736963 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd000035b0 con 0x7fcd180d1500 2023-10-16T11:20:54.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd16ffd700 1 -- 172.21.15.181:0/1733736963 >> v1:172.21.15.190:6789/0 conn(0x7fcd1815ea40 legacy=0x7fcd180d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1733736963 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423494983 0 0) 0x7fcd180ce970 con 0x7fcd1815ea40 2023-10-16T11:20:54.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd16ffd700 1 -- 172.21.15.181:0/1733736963 >> v1:172.21.15.181:6789/0 conn(0x7fcd180d1500 legacy=0x7fcd180d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.180+0000 7fcd16ffd700 1 -- 172.21.15.181:0/1733736963 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180cec50 con 0x7fcd180d01d0 2023-10-16T11:20:54.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd16ffd700 1 --2- 172.21.15.181:0/1733736963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd180d01d0 0x7fcd180ce2d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009840 tx=0x7fcd08009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c086aabe104feae5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:54.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1733736963 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08011830 con 0x7fcd180d01d0 2023-10-16T11:20:54.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1733736963 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08011990 con 0x7fcd180d01d0 2023-10-16T11:20:54.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd15ffb700 1 -- 172.21.15.181:0/1733736963 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08011d20 con 0x7fcd180d01d0 2023-10-16T11:20:54.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/1733736963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd180d01d0 msgr2=0x7fcd180ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/1733736963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd180d01d0 0x7fcd180ce2d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009840 tx=0x7fcd08009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:54.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/1733736963 shutdown_connections 2023-10-16T11:20:54.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/1733736963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd180d01d0 0x7fcd180ce2d0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/1733736963 >> 172.21.15.181:0/1733736963 conn(0x7fcd1809cb60 msgr2=0x7fcd180d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:54.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.181+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/1733736963 shutdown_connections 2023-10-16T11:20:54.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.182+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/1733736963 wait complete. 2023-10-16T11:20:54.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.183+0000 7fcd1d4f5700 1 Processor -- start 2023-10-16T11:20:54.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.183+0000 7fcd1d4f5700 1 -- start start 2023-10-16T11:20:54.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd1d4f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd180d01d0 0x7fcd18154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd1d4f5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd180d1500 0x7fcd180cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd1d4f5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1815ea40 0x7fcd180cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd1d4f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180cb640 con 0x7fcd180d01d0 2023-10-16T11:20:54.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd1d4f5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180cb920 con 0x7fcd1815ea40 2023-10-16T11:20:54.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd1d4f5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180cbc00 con 0x7fcd180d1500 2023-10-16T11:20:54.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd16ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd180d01d0 0x7fcd18154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd16ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd180d01d0 0x7fcd18154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58758/0 (socket says 172.21.15.181:58758) 2023-10-16T11:20:54.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd16ffd700 1 -- 172.21.15.181:0/38317037 learned_addr learned my addr 172.21.15.181:0/38317037 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:54.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd177fe700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1815ea40 0x7fcd180cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd167fc700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd180d1500 0x7fcd180cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd177fe700 1 -- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd180d1500 msgr2=0x7fcd180cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd177fe700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd180d1500 0x7fcd180cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd177fe700 1 -- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd180d01d0 msgr2=0x7fcd18154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd177fe700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd180d01d0 0x7fcd18154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd177fe700 1 -- 172.21.15.181:0/38317037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080094a0 con 0x7fcd1815ea40 2023-10-16T11:20:54.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd16ffd700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd180d01d0 0x7fcd18154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:54.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd177fe700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1815ea40 0x7fcd180cd380 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ca80 tx=0x7fcd1000cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:54.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fccff7fe700 1 -- 172.21.15.181:0/38317037 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000a9e0 con 0x7fcd1815ea40 2023-10-16T11:20:54.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fccff7fe700 1 -- 172.21.15.181:0/38317037 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000ab40 con 0x7fcd1815ea40 2023-10-16T11:20:54.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd180cbee0 con 0x7fcd1815ea40 2023-10-16T11:20:54.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.184+0000 7fccff7fe700 1 -- 172.21.15.181:0/38317037 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10012400 con 0x7fcd1815ea40 2023-10-16T11:20:54.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.185+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd180cb270 con 0x7fcd1815ea40 2023-10-16T11:20:54.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.185+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd180c8830 con 0x7fcd1815ea40 2023-10-16T11:20:54.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.186+0000 7fccff7fe700 1 -- 172.21.15.181:0/38317037 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 154) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcd1001b3b0 con 0x7fcd1815ea40 2023-10-16T11:20:54.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.186+0000 7fccff7fe700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 0x7fcd0406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.186+0000 7fcd16ffd700 1 -- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 msgr2=0x7fcd0406d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:54.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.186+0000 7fccff7fe700 1 -- 172.21.15.181:0/38317037 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fcd10089390 con 0x7fcd1815ea40 2023-10-16T11:20:54.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.186+0000 7fcd16ffd700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 0x7fcd0406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:54.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.191+0000 7fccff7fe700 1 -- 172.21.15.181:0/38317037 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcd180c8830 con 0x7fcd1815ea40 2023-10-16T11:20:54.266 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:54.265+0000 7fce16c49200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:20:54.312 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:54.311+0000 7f8a4cb6b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:20:54.383 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:54.382+0000 7fce16c49200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:20:54.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.387+0000 7fcd16ffd700 1 -- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 msgr2=0x7fcd0406d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:54.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.387+0000 7fcd16ffd700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 0x7fcd0406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:54.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.387+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd180c8830 con 0x7fcd1815ea40 2023-10-16T11:20:54.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.388+0000 7fccff7fe700 1 -- 172.21.15.181:0/38317037 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v241) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd180c8830 con 0x7fcd1815ea40 2023-10-16T11:20:54.389 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:20:54.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 msgr2=0x7fcd0406d210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:54.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 0x7fcd0406d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1815ea40 msgr2=0x7fcd180cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1815ea40 0x7fcd180cd380 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ca80 tx=0x7fcd1000cd90 comp rx=0 tx=0).stop 2023-10-16T11:20:54.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 shutdown_connections 2023-10-16T11:20:54.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1815ea40 0x7fcd180cd380 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd180d01d0 0x7fcd18154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcd0406ad50 0x7fcd0406d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 --2- 172.21.15.181:0/38317037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd180d1500 0x7fcd180cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 >> 172.21.15.181:0/38317037 conn(0x7fcd1809cb60 msgr2=0x7fcd1815dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:54.393 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.390+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 shutdown_connections 2023-10-16T11:20:54.394 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.391+0000 7fcd1d4f5700 1 -- 172.21.15.181:0/38317037 wait complete. 2023-10-16T11:20:54.405 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:20:54.437 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:54.436+0000 7f8a4cb6b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:20:54.459 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:54.458+0000 7fce16c49200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:20:54.516 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:54.515+0000 7f8a4cb6b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:20:54.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.533+0000 7fd810328700 1 Processor -- start 2023-10-16T11:20:54.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.533+0000 7fd810328700 1 -- start start 2023-10-16T11:20:54.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.534+0000 7fd810328700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd80815f150 0x7fd8080cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.534+0000 7fd810328700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8080cd170 0x7fd8080b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.534+0000 7fd810328700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080b2ae0 con 0x7fd8080ce4f0 2023-10-16T11:20:54.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.534+0000 7fd810328700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080ce970 con 0x7fd8080cd170 2023-10-16T11:20:54.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.534+0000 7fd810328700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b2dc0 con 0x7fd80815f150 2023-10-16T11:20:54.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e8c5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8080cd170 0x7fd8080b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e0c4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd80815f150 0x7fd8080cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80d8c3700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd8080ce4f0 0x7fd8080b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:53790/0 (socket says 172.21.15.181:53790) 2023-10-16T11:20:54.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80d8c3700 1 -- 172.21.15.181:0/1404686311 learned_addr learned my addr 172.21.15.181:0/1404686311 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:54.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e0c4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd80815f150 0x7fd8080cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:55794/0 (socket says 172.21.15.181:55794) 2023-10-16T11:20:54.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e8c5700 1 -- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd80815f150 msgr2=0x7fd8080cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:54.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e8c5700 1 --2- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd80815f150 0x7fd8080cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e8c5700 1 -- 172.21.15.181:0/1404686311 >> v1:172.21.15.190:6789/0 conn(0x7fd8080ce4f0 legacy=0x7fd8080b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80d0c2700 1 -- 172.21.15.181:0/1404686311 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987796875 0 0) 0x7fd8080b2ae0 con 0x7fd8080ce4f0 2023-10-16T11:20:54.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e8c5700 1 -- 172.21.15.181:0/1404686311 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080cf9d0 con 0x7fd8080cd170 2023-10-16T11:20:54.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e0c4700 1 --2- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd80815f150 0x7fd8080cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:54.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.535+0000 7fd80e8c5700 1 --2- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080cd170 0x7fd8080b1670 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd800009d90 tx=0x7fd80000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81022e26e8415a80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:54.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd80d0c2700 1 -- 172.21.15.181:0/1404686311 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80000e070 con 0x7fd8080cd170 2023-10-16T11:20:54.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd80d0c2700 1 -- 172.21.15.181:0/1404686311 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80000bb20 con 0x7fd8080cd170 2023-10-16T11:20:54.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd80d0c2700 1 -- 172.21.15.181:0/1404686311 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800012430 con 0x7fd8080cd170 2023-10-16T11:20:54.541 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:54.538+0000 7fce16c49200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:20:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 -- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080cd170 msgr2=0x7fd8080b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 --2- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080cd170 0x7fd8080b1670 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd800009d90 tx=0x7fd80000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 -- 172.21.15.181:0/1404686311 shutdown_connections 2023-10-16T11:20:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 --2- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080cd170 0x7fd8080b1670 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 --2- 172.21.15.181:0/1404686311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd80815f150 0x7fd8080cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 -- 172.21.15.181:0/1404686311 >> 172.21.15.181:0/1404686311 conn(0x7fd80809a930 msgr2=0x7fd8080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:54.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 -- 172.21.15.181:0/1404686311 shutdown_connections 2023-10-16T11:20:54.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.536+0000 7fd810328700 1 -- 172.21.15.181:0/1404686311 wait complete. 2023-10-16T11:20:54.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.538+0000 7fd810328700 1 Processor -- start 2023-10-16T11:20:54.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.538+0000 7fd810328700 1 -- start start 2023-10-16T11:20:54.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd810328700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8080cd170 0x7fd808159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd810328700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080ce4f0 0x7fd808158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd810328700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd80815f150 0x7fd8080c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd810328700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c89e0 con 0x7fd80815f150 2023-10-16T11:20:54.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd810328700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c86d0 con 0x7fd8080ce4f0 2023-10-16T11:20:54.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd810328700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd808159be0 con 0x7fd8080cd170 2023-10-16T11:20:54.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e0c4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8080cd170 0x7fd808159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e8c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd80815f150 0x7fd8080c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e0c4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8080cd170 0x7fd808159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:55800/0 (socket says 172.21.15.181:55800) 2023-10-16T11:20:54.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80d8c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080ce4f0 0x7fd808158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e0c4700 1 -- 172.21.15.181:0/3877624781 learned_addr learned my addr 172.21.15.181:0/3877624781 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e8c5700 1 -- 172.21.15.181:0/3877624781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8080cd170 msgr2=0x7fd808159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e8c5700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8080cd170 0x7fd808159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e8c5700 1 -- 172.21.15.181:0/3877624781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080ce4f0 msgr2=0x7fd808158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e8c5700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080ce4f0 0x7fd808158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80e8c5700 1 -- 172.21.15.181:0/3877624781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040094a0 con 0x7fd80815f150 2023-10-16T11:20:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.539+0000 7fd80d8c3700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080ce4f0 0x7fd808158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.540+0000 7fd80e8c5700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd80815f150 0x7fd8080c7e00 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fd800016000 tx=0x7fd800004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.540+0000 7fd7feffd700 1 -- 172.21.15.181:0/3877624781 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800012430 con 0x7fd80815f150 2023-10-16T11:20:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.540+0000 7fd7feffd700 1 -- 172.21.15.181:0/3877624781 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd800020070 con 0x7fd80815f150 2023-10-16T11:20:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.540+0000 7fd7feffd700 1 -- 172.21.15.181:0/3877624781 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800004280 con 0x7fd80815f150 2023-10-16T11:20:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.540+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd800009a70 con 0x7fd80815f150 2023-10-16T11:20:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.540+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80815a1a0 con 0x7fd80815f150 2023-10-16T11:20:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.541+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8080c8b20 con 0x7fd80815f150 2023-10-16T11:20:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.542+0000 7fd7feffd700 1 -- 172.21.15.181:0/3877624781 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fd8000043e0 con 0x7fd80815f150 2023-10-16T11:20:54.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.542+0000 7fd7feffd700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 0x7fd7ec06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.542+0000 7fd80e0c4700 1 -- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 msgr2=0x7fd7ec06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:54.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.542+0000 7fd7feffd700 1 -- 172.21.15.181:0/3877624781 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fd800022070 con 0x7fd80815f150 2023-10-16T11:20:54.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.542+0000 7fd80e0c4700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 0x7fd7ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:54.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.546+0000 7fd7feffd700 1 -- 172.21.15.181:0/3877624781 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd800056340 con 0x7fd80815f150 2023-10-16T11:20:54.601 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:54.600+0000 7f8a4cb6b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:20:54.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.743+0000 7fd80e0c4700 1 -- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 msgr2=0x7fd7ec06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:54.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.743+0000 7fd80e0c4700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 0x7fd7ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:54.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.747+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd808067d20 con 0x7fd80815f150 2023-10-16T11:20:54.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.747+0000 7fd7feffd700 1 -- 172.21.15.181:0/3877624781 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v241) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd800059960 con 0x7fd80815f150 2023-10-16T11:20:54.749 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 msgr2=0x7fd7ec06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 0x7fd7ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd80815f150 msgr2=0x7fd8080c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd80815f150 0x7fd8080c7e00 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fd800016000 tx=0x7fd800004ad0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 shutdown_connections 2023-10-16T11:20:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd7ec06b140 0x7fd7ec06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8080cd170 0x7fd808159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd80815f150 0x7fd8080c7e00 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 --2- 172.21.15.181:0/3877624781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8080ce4f0 0x7fd808158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 >> 172.21.15.181:0/3877624781 conn(0x7fd80809a930 msgr2=0x7fd8080ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:54.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 shutdown_connections 2023-10-16T11:20:54.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.750+0000 7fd810328700 1 -- 172.21.15.181:0/3877624781 wait complete. 2023-10-16T11:20:54.764 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:20:54.910 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:54.909+0000 7fce16c49200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:20:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.930+0000 7fd568779700 1 Processor -- start 2023-10-16T11:20:54.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.930+0000 7fd568779700 1 -- start start 2023-10-16T11:20:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.931+0000 7fd568779700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5600d24e0 0x7fd5600d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.931+0000 7fd568779700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5600d0a50 con 0x7fd5600d0600 2023-10-16T11:20:54.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.931+0000 7fd568779700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5600d2960 con 0x7fd56015d9e0 2023-10-16T11:20:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.931+0000 7fd568779700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600d20c0 con 0x7fd5600d24e0 2023-10-16T11:20:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.931+0000 7fd566d16700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd5600d0600 0x7fd5600b6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41660/0 (socket says 172.21.15.181:41660) 2023-10-16T11:20:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.931+0000 7fd566d16700 1 -- 172.21.15.181:0/3109203829 learned_addr learned my addr 172.21.15.181:0/3109203829 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565d14700 1 --2- 172.21.15.181:0/3109203829 >> v2:172.21.15.190:3300/0 conn(0x7fd5600d24e0 0x7fd5600d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565513700 1 -- 172.21.15.181:0/3109203829 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467448522 0 0) 0x7fd558001490 con 0x7fd5600d0600 2023-10-16T11:20:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565513700 1 -- 172.21.15.181:0/3109203829 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd54c0035b0 con 0x7fd5600d0600 2023-10-16T11:20:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565d14700 1 -- 172.21.15.181:0/3109203829 >> v1:172.21.15.190:6789/0 conn(0x7fd56015d9e0 legacy=0x7fd5600cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565d14700 1 -- 172.21.15.181:0/3109203829 >> v1:172.21.15.181:6789/0 conn(0x7fd5600d0600 legacy=0x7fd5600b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565d14700 1 -- 172.21.15.181:0/3109203829 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600d4180 con 0x7fd5600d24e0 2023-10-16T11:20:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565d14700 1 --2- 172.21.15.181:0/3109203829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d24e0 0x7fd5600d00c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd550009840 tx=0x7fd550009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37cd8a6dcf410a56 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:54.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565513700 1 -- 172.21.15.181:0/3109203829 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550011830 con 0x7fd5600d24e0 2023-10-16T11:20:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565513700 1 -- 172.21.15.181:0/3109203829 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550011990 con 0x7fd5600d24e0 2023-10-16T11:20:54.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.932+0000 7fd565513700 1 -- 172.21.15.181:0/3109203829 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550011d20 con 0x7fd5600d24e0 2023-10-16T11:20:54.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.933+0000 7fd568779700 1 -- 172.21.15.181:0/3109203829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d24e0 msgr2=0x7fd5600d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.933+0000 7fd568779700 1 --2- 172.21.15.181:0/3109203829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d24e0 0x7fd5600d00c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd550009840 tx=0x7fd550009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:54.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.933+0000 7fd568779700 1 -- 172.21.15.181:0/3109203829 shutdown_connections 2023-10-16T11:20:54.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.933+0000 7fd568779700 1 --2- 172.21.15.181:0/3109203829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d24e0 0x7fd5600d00c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.933+0000 7fd568779700 1 -- 172.21.15.181:0/3109203829 >> 172.21.15.181:0/3109203829 conn(0x7fd56009cb40 msgr2=0x7fd5600d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.933+0000 7fd568779700 1 -- 172.21.15.181:0/3109203829 shutdown_connections 2023-10-16T11:20:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.933+0000 7fd568779700 1 -- 172.21.15.181:0/3109203829 wait complete. 2023-10-16T11:20:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.934+0000 7fd568779700 1 Processor -- start 2023-10-16T11:20:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.934+0000 7fd568779700 1 -- start start 2023-10-16T11:20:54.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd568779700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d0600 0x7fd5600b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd568779700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d24e0 0x7fd5600b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd568779700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd56015d9e0 0x7fd5600b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd568779700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600b7370 con 0x7fd56015d9e0 2023-10-16T11:20:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd568779700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600cb4d0 con 0x7fd5600d0600 2023-10-16T11:20:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd568779700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5601f4440 con 0x7fd5600d24e0 2023-10-16T11:20:54.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd56015d9e0 0x7fd5600b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd56015d9e0 0x7fd5600b7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58820/0 (socket says 172.21.15.181:58820) 2023-10-16T11:20:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 -- 172.21.15.181:0/851691251 learned_addr learned my addr 172.21.15.181:0/851691251 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd565d14700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d24e0 0x7fd5600b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566515700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d0600 0x7fd5600b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:54.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 -- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d24e0 msgr2=0x7fd5600b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d24e0 0x7fd5600b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 -- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d0600 msgr2=0x7fd5600b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d0600 0x7fd5600b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.935+0000 7fd566d16700 1 -- 172.21.15.181:0/851691251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500094a0 con 0x7fd56015d9e0 2023-10-16T11:20:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.936+0000 7fd566d16700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd56015d9e0 0x7fd5600b7ae0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7fd55800cae0 tx=0x7fd55800cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:54.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.936+0000 7fd556ffd700 1 -- 172.21.15.181:0/851691251 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800a940 con 0x7fd56015d9e0 2023-10-16T11:20:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.936+0000 7fd556ffd700 1 -- 172.21.15.181:0/851691251 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800aaa0 con 0x7fd56015d9e0 2023-10-16T11:20:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.936+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5600b6db0 con 0x7fd56015d9e0 2023-10-16T11:20:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.936+0000 7fd556ffd700 1 -- 172.21.15.181:0/851691251 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800ae30 con 0x7fd56015d9e0 2023-10-16T11:20:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.936+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5600cb160 con 0x7fd56015d9e0 2023-10-16T11:20:54.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.937+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5600cafd0 con 0x7fd56015d9e0 2023-10-16T11:20:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.938+0000 7fd556ffd700 1 -- 172.21.15.181:0/851691251 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fd55801b3b0 con 0x7fd56015d9e0 2023-10-16T11:20:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.938+0000 7fd556ffd700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 0x7fd54406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.938+0000 7fd566515700 1 -- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 msgr2=0x7fd54406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.938+0000 7fd556ffd700 1 -- 172.21.15.181:0/851691251 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fd558089ef0 con 0x7fd56015d9e0 2023-10-16T11:20:54.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.938+0000 7fd566515700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 0x7fd54406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:54.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:54.941+0000 7fd556ffd700 1 -- 172.21.15.181:0/851691251 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd558058070 con 0x7fd56015d9e0 2023-10-16T11:20:54.991 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:54.991+0000 7f8a4cb6b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:20:55.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.139+0000 7fd566515700 1 -- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 msgr2=0x7fd54406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:55.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.139+0000 7fd566515700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 0x7fd54406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:55.141 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.141+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5600d0a50 con 0x7fd56015d9e0 2023-10-16T11:20:55.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.141+0000 7fd556ffd700 1 -- 172.21.15.181:0/851691251 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v241) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5600d0a50 con 0x7fd56015d9e0 2023-10-16T11:20:55.142 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 msgr2=0x7fd54406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 0x7fd54406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd56015d9e0 msgr2=0x7fd5600b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd56015d9e0 0x7fd5600b7ae0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7fd55800cae0 tx=0x7fd55800cdf0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 shutdown_connections 2023-10-16T11:20:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd56015d9e0 0x7fd5600b7ae0 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d0600 0x7fd5600b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd54406b140 0x7fd54406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 --2- 172.21.15.181:0/851691251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d24e0 0x7fd5600b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 >> 172.21.15.181:0/851691251 conn(0x7fd56009cb40 msgr2=0x7fd56015efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 shutdown_connections 2023-10-16T11:20:55.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.144+0000 7fd568779700 1 -- 172.21.15.181:0/851691251 wait complete. 2023-10-16T11:20:55.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:20:55.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.282+0000 7fcb099b9700 1 Processor -- start 2023-10-16T11:20:55.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.282+0000 7fcb099b9700 1 -- start start 2023-10-16T11:20:55.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb099b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb0415dbd0 0x7fcb040d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb099b9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb040d0610 0x7fcb040cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb099b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040ce2c0 0x7fcb04161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb099b9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040ce740 con 0x7fcb0415dbd0 2023-10-16T11:20:55.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb099b9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0415dfb0 con 0x7fcb040d0610 2023-10-16T11:20:55.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb099b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040cd3a0 con 0x7fcb040ce2c0 2023-10-16T11:20:55.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb0415dbd0 0x7fcb040d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb0415dbd0 0x7fcb040d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58826/0 (socket says 172.21.15.181:58826) 2023-10-16T11:20:55.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 -- 172.21.15.181:0/2815733033 learned_addr learned my addr 172.21.15.181:0/2815733033 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:55.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb027fc700 1 --2- 172.21.15.181:0/2815733033 >> v2:172.21.15.190:3300/0 conn(0x7fcb040d0610 0x7fcb040cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb037fe700 1 --2- 172.21.15.181:0/2815733033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040ce2c0 0x7fcb04161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 -- 172.21.15.181:0/2815733033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040ce2c0 msgr2=0x7fcb04161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:55.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 --2- 172.21.15.181:0/2815733033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040ce2c0 0x7fcb04161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 -- 172.21.15.181:0/2815733033 >> v2:172.21.15.190:3300/0 conn(0x7fcb040d0610 msgr2=0x7fcb040cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:55.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 --2- 172.21.15.181:0/2815733033 >> v2:172.21.15.190:3300/0 conn(0x7fcb040d0610 0x7fcb040cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.283+0000 7fcb02ffd700 1 -- 172.21.15.181:0/2815733033 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040cd830 con 0x7fcb0415dbd0 2023-10-16T11:20:55.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb02ffd700 1 --2- 172.21.15.181:0/2815733033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0415dbd0 0x7fcb040d00d0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fcb040cb990 tx=0x7fcaf4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7b4628268b7ca6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:55.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb01ffb700 1 -- 172.21.15.181:0/2815733033 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400b870 con 0x7fcb0415dbd0 2023-10-16T11:20:55.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb01ffb700 1 -- 172.21.15.181:0/2815733033 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf400b9d0 con 0x7fcb0415dbd0 2023-10-16T11:20:55.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb01ffb700 1 -- 172.21.15.181:0/2815733033 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400bd60 con 0x7fcb0415dbd0 2023-10-16T11:20:55.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 -- 172.21.15.181:0/2815733033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0415dbd0 msgr2=0x7fcb040d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 --2- 172.21.15.181:0/2815733033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0415dbd0 0x7fcb040d00d0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fcb040cb990 tx=0x7fcaf4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 -- 172.21.15.181:0/2815733033 shutdown_connections 2023-10-16T11:20:55.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 --2- 172.21.15.181:0/2815733033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb0415dbd0 0x7fcb040d00d0 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 --2- 172.21.15.181:0/2815733033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040ce2c0 0x7fcb04161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 --2- 172.21.15.181:0/2815733033 >> v2:172.21.15.190:3300/0 conn(0x7fcb040d0610 0x7fcb040cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 -- 172.21.15.181:0/2815733033 >> 172.21.15.181:0/2815733033 conn(0x7fcb0409a930 msgr2=0x7fcb040d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:55.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 -- 172.21.15.181:0/2815733033 shutdown_connections 2023-10-16T11:20:55.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.284+0000 7fcb099b9700 1 -- 172.21.15.181:0/2815733033 wait complete. 2023-10-16T11:20:55.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 Processor -- start 2023-10-16T11:20:55.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 -- start start 2023-10-16T11:20:55.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb040ce2c0 0x7fcb041f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040d0610 0x7fcb040b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0415dbd0 0x7fcb040b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041f3780 con 0x7fcb040ce2c0 2023-10-16T11:20:55.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041f7220 con 0x7fcb0415dbd0 2023-10-16T11:20:55.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.286+0000 7fcb099b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041f7500 con 0x7fcb040d0610 2023-10-16T11:20:55.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb040ce2c0 0x7fcb041f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb040ce2c0 0x7fcb041f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58832/0 (socket says 172.21.15.181:58832) 2023-10-16T11:20:55.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 -- 172.21.15.181:0/3051915081 learned_addr learned my addr 172.21.15.181:0/3051915081 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:55.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb027fc700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040d0610 0x7fcb040b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb037fe700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0415dbd0 0x7fcb040b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 -- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040d0610 msgr2=0x7fcb040b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040d0610 0x7fcb040b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 -- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0415dbd0 msgr2=0x7fcb040b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0415dbd0 0x7fcb040b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 -- 172.21.15.181:0/3051915081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4009550 con 0x7fcb040ce2c0 2023-10-16T11:20:55.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb02ffd700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb040ce2c0 0x7fcb041f6bd0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4000c00 tx=0x7fcaf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:55.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcaebfff700 1 -- 172.21.15.181:0/3051915081 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4004250 con 0x7fcb040ce2c0 2023-10-16T11:20:55.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcaebfff700 1 -- 172.21.15.181:0/3051915081 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf401f070 con 0x7fcb040ce2c0 2023-10-16T11:20:55.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb041f77e0 con 0x7fcb040ce2c0 2023-10-16T11:20:55.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcaebfff700 1 -- 172.21.15.181:0/3051915081 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40045d0 con 0x7fcb040ce2c0 2023-10-16T11:20:55.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.287+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb040caee0 con 0x7fcb040ce2c0 2023-10-16T11:20:55.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.288+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb04067d20 con 0x7fcb040ce2c0 2023-10-16T11:20:55.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.289+0000 7fcaebfff700 1 -- 172.21.15.181:0/3051915081 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fcaf4021030 con 0x7fcb040ce2c0 2023-10-16T11:20:55.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.289+0000 7fcaebfff700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 0x7fcaf006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.289+0000 7fcb027fc700 1 -- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 msgr2=0x7fcaf006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:55.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.289+0000 7fcaebfff700 1 -- 172.21.15.181:0/3051915081 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fcaf408b170 con 0x7fcb040ce2c0 2023-10-16T11:20:55.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.289+0000 7fcb027fc700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 0x7fcaf006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:55.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.294+0000 7fcaebfff700 1 -- 172.21.15.181:0/3051915081 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcaf40558e0 con 0x7fcb040ce2c0 2023-10-16T11:20:55.487 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:55.486+0000 7fce16c49200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:20:55.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.490+0000 7fcb027fc700 1 -- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 msgr2=0x7fcaf006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:55.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.490+0000 7fcb027fc700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 0x7fcaf006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:55.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.515+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb040cebb0 con 0x7fcb040ce2c0 2023-10-16T11:20:55.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.516+0000 7fcaebfff700 1 -- 172.21.15.181:0/3051915081 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v241) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcaf40150b0 con 0x7fcb040ce2c0 2023-10-16T11:20:55.517 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 msgr2=0x7fcaf006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:55.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 0x7fcaf006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb040ce2c0 msgr2=0x7fcb041f6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb040ce2c0 0x7fcb041f6bd0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4000c00 tx=0x7fcaf4004750 comp rx=0 tx=0).stop 2023-10-16T11:20:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 shutdown_connections 2023-10-16T11:20:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb0415dbd0 0x7fcb040b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb040ce2c0 0x7fcb041f6bd0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fcaf006b140 0x7fcaf006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 --2- 172.21.15.181:0/3051915081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb040d0610 0x7fcb040b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 >> 172.21.15.181:0/3051915081 conn(0x7fcb0409a930 msgr2=0x7fcb04161de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:55.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 shutdown_connections 2023-10-16T11:20:55.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.518+0000 7fcb099b9700 1 -- 172.21.15.181:0/3051915081 wait complete. 2023-10-16T11:20:55.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:20:55.557 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:55.555+0000 7fce16c49200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:20:55.592 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:55.591+0000 7f8a4cb6b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:20:55.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.660+0000 7ff12fca5700 1 Processor -- start 2023-10-16T11:20:55.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.660+0000 7ff12fca5700 1 -- start start 2023-10-16T11:20:55.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12fca5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff128071be0 0x7ff1280ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12fca5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280ce1f0 con 0x7ff1280b2010 2023-10-16T11:20:55.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12fca5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280ccda0 con 0x7ff1280cc6e0 2023-10-16T11:20:55.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12fca5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1280ccb90 con 0x7ff128071be0 2023-10-16T11:20:55.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12e242700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff128071be0 0x7ff1280ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12da41700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff1280b2010 0x7ff1280b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41688/0 (socket says 172.21.15.181:41688) 2023-10-16T11:20:55.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12da41700 1 -- 172.21.15.181:0/3200306599 learned_addr learned my addr 172.21.15.181:0/3200306599 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:55.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12e242700 1 -- 172.21.15.181:0/3200306599 >> v1:172.21.15.190:6789/0 conn(0x7ff1280cc6e0 legacy=0x7ff1280ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12e242700 1 -- 172.21.15.181:0/3200306599 >> v1:172.21.15.181:6789/0 conn(0x7ff1280b2010 legacy=0x7ff1280b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12ca3f700 1 -- 172.21.15.181:0/3200306599 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350682544 0 0) 0x7ff1280ce1f0 con 0x7ff1280b2010 2023-10-16T11:20:55.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.661+0000 7ff12e242700 1 -- 172.21.15.181:0/3200306599 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280727e0 con 0x7ff128071be0 2023-10-16T11:20:55.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12e242700 1 --2- 172.21.15.181:0/3200306599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff128071be0 0x7ff1280ced90 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7ff120009d90 tx=0x7ff12000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e069cedd03749b36 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:55.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12ca3f700 1 -- 172.21.15.181:0/3200306599 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000e070 con 0x7ff128071be0 2023-10-16T11:20:55.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12ca3f700 1 -- 172.21.15.181:0/3200306599 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12000bbd0 con 0x7ff128071be0 2023-10-16T11:20:55.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12ca3f700 1 -- 172.21.15.181:0/3200306599 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120012430 con 0x7ff128071be0 2023-10-16T11:20:55.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12fca5700 1 -- 172.21.15.181:0/3200306599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff128071be0 msgr2=0x7ff1280ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12fca5700 1 --2- 172.21.15.181:0/3200306599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff128071be0 0x7ff1280ced90 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7ff120009d90 tx=0x7ff12000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:55.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12fca5700 1 -- 172.21.15.181:0/3200306599 shutdown_connections 2023-10-16T11:20:55.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12fca5700 1 --2- 172.21.15.181:0/3200306599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff128071be0 0x7ff1280ced90 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12fca5700 1 -- 172.21.15.181:0/3200306599 >> 172.21.15.181:0/3200306599 conn(0x7ff1280cd010 msgr2=0x7ff1280cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:55.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.662+0000 7ff12fca5700 1 -- 172.21.15.181:0/3200306599 shutdown_connections 2023-10-16T11:20:55.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.663+0000 7ff12fca5700 1 -- 172.21.15.181:0/3200306599 wait complete. 2023-10-16T11:20:55.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 Processor -- start 2023-10-16T11:20:55.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 -- start start 2023-10-16T11:20:55.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff128071be0 0x7ff1281eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1280b2010 0x7ff1280ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1280cc6e0 0x7ff12814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12815ad90 con 0x7ff1280b2010 2023-10-16T11:20:55.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280cfa70 con 0x7ff1280cc6e0 2023-10-16T11:20:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12fca5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280cebc0 con 0x7ff128071be0 2023-10-16T11:20:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.664+0000 7ff12d240700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1280b2010 0x7ff1280ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1280b2010 0x7ff1280ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58850/0 (socket says 172.21.15.181:58850) 2023-10-16T11:20:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 -- 172.21.15.181:0/1799300779 learned_addr learned my addr 172.21.15.181:0/1799300779 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:55.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12da41700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff128071be0 0x7ff1281eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12e242700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1280cc6e0 0x7ff12814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 -- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff128071be0 msgr2=0x7ff1281eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff128071be0 0x7ff1281eecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 -- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1280cc6e0 msgr2=0x7ff12814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1280cc6e0 0x7ff12814e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 -- 172.21.15.181:0/1799300779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120009a70 con 0x7ff1280b2010 2023-10-16T11:20:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12da41700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff128071be0 0x7ff1281eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12d240700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1280b2010 0x7ff1280ce680 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c4e0 tx=0x7ff12400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff11e7fc700 1 -- 172.21.15.181:0/1799300779 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1240052d0 con 0x7ff1280b2010 2023-10-16T11:20:55.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff11e7fc700 1 -- 172.21.15.181:0/1799300779 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff124005430 con 0x7ff1280b2010 2023-10-16T11:20:55.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1280cee60 con 0x7ff1280b2010 2023-10-16T11:20:55.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff11e7fc700 1 -- 172.21.15.181:0/1799300779 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1240038c0 con 0x7ff1280b2010 2023-10-16T11:20:55.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.665+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1280cf420 con 0x7ff1280b2010 2023-10-16T11:20:55.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.666+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1280b6510 con 0x7ff1280b2010 2023-10-16T11:20:55.675 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.667+0000 7ff11e7fc700 1 -- 172.21.15.181:0/1799300779 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7ff124003a20 con 0x7ff1280b2010 2023-10-16T11:20:55.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.667+0000 7ff11e7fc700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 0x7ff10c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:55.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.667+0000 7ff12da41700 1 -- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 msgr2=0x7ff10c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:55.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.667+0000 7ff11e7fc700 1 -- 172.21.15.181:0/1799300779 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7ff124089400 con 0x7ff1280b2010 2023-10-16T11:20:55.676 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:55.667+0000 7f8a4cb6b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:20:55.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.667+0000 7ff12da41700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 0x7ff10c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:55.677 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.671+0000 7ff11e7fc700 1 -- 172.21.15.181:0/1799300779 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff124053b70 con 0x7ff1280b2010 2023-10-16T11:20:55.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.868+0000 7ff12da41700 1 -- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 msgr2=0x7ff10c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:55.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.868+0000 7ff12da41700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 0x7ff10c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:55.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.868+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7ff128072230 con 0x7ff1280b2010 2023-10-16T11:20:55.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.869+0000 7ff11e7fc700 1 -- 172.21.15.181:0/1799300779 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v241) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff124057190 con 0x7ff1280b2010 2023-10-16T11:20:55.870 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:55.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 msgr2=0x7ff10c06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:55.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 0x7ff10c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1280b2010 msgr2=0x7ff1280ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:55.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1280b2010 0x7ff1280ce680 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7ff12400c4e0 tx=0x7ff12400c7f0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 shutdown_connections 2023-10-16T11:20:55.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7ff10c06b140 0x7ff10c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff128071be0 0x7ff1281eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1280cc6e0 0x7ff12814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 --2- 172.21.15.181:0/1799300779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1280b2010 0x7ff1280ce680 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:55.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 >> 172.21.15.181:0/1799300779 conn(0x7ff1280cd010 msgr2=0x7ff1280b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 shutdown_connections 2023-10-16T11:20:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:55.871+0000 7ff12fca5700 1 -- 172.21.15.181:0/1799300779 wait complete. 2023-10-16T11:20:55.885 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:20:56.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.006+0000 7f9f6b5de700 1 Processor -- start 2023-10-16T11:20:56.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.006+0000 7f9f6b5de700 1 -- start start 2023-10-16T11:20:56.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f6b5de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 0x7f9f640d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f6b5de700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f640d0940 con 0x7f9f6415dda0 2023-10-16T11:20:56.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f6b5de700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f640d0c20 con 0x7f9f640d26b0 2023-10-16T11:20:56.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f6b5de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640d2b20 con 0x7f9f640cfa60 2023-10-16T11:20:56.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f6937a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9f6415dda0 0x7f9f640d0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41704/0 (socket says 172.21.15.181:41704) 2023-10-16T11:20:56.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f6937a700 1 -- 172.21.15.181:0/3844414222 learned_addr learned my addr 172.21.15.181:0/3844414222 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:56.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f68b79700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 0x7f9f640d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255415172 0 0) 0x7f9f640d0940 con 0x7f9f6415dda0 2023-10-16T11:20:56.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f500035b0 con 0x7f9f6415dda0 2023-10-16T11:20:56.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278115109 0 0) 0x7f9f640d0c20 con 0x7f9f640d26b0 2023-10-16T11:20:56.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f640d0940 con 0x7f9f640d26b0 2023-10-16T11:20:56.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413183993 0 0) 0x7f9f500035b0 con 0x7f9f6415dda0 2023-10-16T11:20:56.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 msgr2=0x7f9f640d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:56.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 0x7f9f640d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 >> v1:172.21.15.190:6789/0 conn(0x7f9f640d26b0 legacy=0x7f9f640b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640b2c20 con 0x7f9f6415dda0 2023-10-16T11:20:56.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f9f600017f0 con 0x7f9f6415dda0 2023-10-16T11:20:56.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f68b79700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 0x7f9f640d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:56.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 >> v1:172.21.15.181:6789/0 conn(0x7f9f6415dda0 legacy=0x7f9f640d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f50003bc0 0x7f9f50003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f50004850 0x7f9f50004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 0x7f9f50005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640d2b20 con 0x7f9f50003bc0 2023-10-16T11:20:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5000ba70 con 0x7f9f50004850 2023-10-16T11:20:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.007+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5000bc30 con 0x7f9f640cfa60 2023-10-16T11:20:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f6937a700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f50003bc0 0x7f9f50003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f68b79700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f50004850 0x7f9f50004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f69b7b700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 0x7f9f50005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f6937a700 1 -- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 msgr2=0x7f9f50005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f6937a700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f640cfa60 0x7f9f50005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f6937a700 1 -- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f50004850 msgr2=0x7f9f50004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f6937a700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f50004850 0x7f9f50004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f6937a700 1 -- 172.21.15.181:0/3844414222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f540094a0 con 0x7f9f50003bc0 2023-10-16T11:20:56.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f6937a700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f50003bc0 0x7f9f50003fc0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c210 tx=0x7f9f50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2de6704e3400c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:56.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6000cd50 con 0x7f9f50003bc0 2023-10-16T11:20:56.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f60007b50 con 0x7f9f50003bc0 2023-10-16T11:20:56.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.008+0000 7f9f5bfff700 1 -- 172.21.15.181:0/3844414222 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60007310 con 0x7f9f50003bc0 2023-10-16T11:20:56.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.009+0000 7f9f6b5de700 1 -- 172.21.15.181:0/3844414222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f50003bc0 msgr2=0x7f9f50003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.009+0000 7f9f6b5de700 1 --2- 172.21.15.181:0/3844414222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f50003bc0 0x7f9f50003fc0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c210 tx=0x7f9f50005880 comp rx=0 tx=0).stop 2023-10-16T11:20:56.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.009+0000 7f9f69b7b700 1 -- 172.21.15.181:0/3844414222 reap_dead start 2023-10-16T11:20:56.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.009+0000 7f9f6b5de700 1 -- 172.21.15.181:0/3844414222 shutdown_connections 2023-10-16T11:20:56.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.009+0000 7f9f6b5de700 1 -- 172.21.15.181:0/3844414222 >> 172.21.15.181:0/3844414222 conn(0x7f9f6409cb10 msgr2=0x7f9f640d3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:56.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.009+0000 7f9f6b5de700 1 -- 172.21.15.181:0/3844414222 shutdown_connections 2023-10-16T11:20:56.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.009+0000 7f9f6b5de700 1 -- 172.21.15.181:0/3844414222 wait complete. 2023-10-16T11:20:56.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.011+0000 7f9f6b5de700 1 Processor -- start 2023-10-16T11:20:56.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.011+0000 7f9f6b5de700 1 -- start start 2023-10-16T11:20:56.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f6b5de700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f50003bc0 0x7f9f641f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f6b5de700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641f6be0 0x7f9f64155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f6b5de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f64160470 0x7f9f64160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f6b5de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64160e40 con 0x7f9f64160470 2023-10-16T11:20:56.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f6b5de700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641f4560 con 0x7f9f641f6be0 2023-10-16T11:20:56.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f6b5de700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64155ec0 con 0x7f9f50003bc0 2023-10-16T11:20:56.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f69b7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f64160470 0x7f9f64160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f69b7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f64160470 0x7f9f64160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58874/0 (socket says 172.21.15.181:58874) 2023-10-16T11:20:56.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f69b7b700 1 -- 172.21.15.181:0/4133244850 learned_addr learned my addr 172.21.15.181:0/4133244850 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:56.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f6937a700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f50003bc0 0x7f9f641f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.012+0000 7f9f68b79700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641f6be0 0x7f9f64155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f69b7b700 1 -- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f50003bc0 msgr2=0x7f9f641f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f69b7b700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f50003bc0 0x7f9f641f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f69b7b700 1 -- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641f6be0 msgr2=0x7f9f64155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f69b7b700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641f6be0 0x7f9f64155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f69b7b700 1 -- 172.21.15.181:0/4133244850 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f64155550 con 0x7f9f64160470 2023-10-16T11:20:56.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f68b79700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641f6be0 0x7f9f64155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:56.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f69b7b700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f64160470 0x7f9f64160900 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f9f6408ded0 tx=0x7f9f5c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:56.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f59ffb700 1 -- 172.21.15.181:0/4133244850 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c003c10 con 0x7f9f64160470 2023-10-16T11:20:56.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f59ffb700 1 -- 172.21.15.181:0/4133244850 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c003d70 con 0x7f9f64160470 2023-10-16T11:20:56.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f641f70c0 con 0x7f9f64160470 2023-10-16T11:20:56.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f59ffb700 1 -- 172.21.15.181:0/4133244850 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00db20 con 0x7f9f64160470 2023-10-16T11:20:56.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.013+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641f7680 con 0x7f9f64160470 2023-10-16T11:20:56.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.014+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6404ebb0 con 0x7f9f64160470 2023-10-16T11:20:56.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.015+0000 7f9f59ffb700 1 -- 172.21.15.181:0/4133244850 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f9f5c00dc80 con 0x7f9f64160470 2023-10-16T11:20:56.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.015+0000 7f9f59ffb700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 0x7f9f4806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.015+0000 7f9f6937a700 1 -- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 msgr2=0x7f9f4806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:56.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.015+0000 7f9f59ffb700 1 -- 172.21.15.181:0/4133244850 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7f9f5c08a2b0 con 0x7f9f64160470 2023-10-16T11:20:56.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.015+0000 7f9f6937a700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 0x7f9f4806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:56.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.019+0000 7f9f59ffb700 1 -- 172.21.15.181:0/4133244850 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9f5c0548c0 con 0x7f9f64160470 2023-10-16T11:20:56.188 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:56.187+0000 7fce16c49200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:56.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.215+0000 7f9f6937a700 1 -- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 msgr2=0x7f9f4806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:56.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.215+0000 7f9f6937a700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 0x7f9f4806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:56.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.233+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9f640d3320 con 0x7f9f64160470 2023-10-16T11:20:56.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.233+0000 7f9f59ffb700 1 -- 172.21.15.181:0/4133244850 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v241) v1 ==== 81+0+596 (secure 0 0 0) 0x7f9f5c057ee0 con 0x7f9f64160470 2023-10-16T11:20:56.236 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1783725009 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.236 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3959972769 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.236 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2468932000 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.236 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2390273904 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.237 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2847184738 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.237 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2797066080 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.237 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3273985848 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.237 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3532323147 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.237 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3273985848 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.237 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3731618096 2023-10-17T11:20:51.419893+0000 2023-10-16T11:20:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 msgr2=0x7f9f4806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 0x7f9f4806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f64160470 msgr2=0x7f9f64160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f64160470 0x7f9f64160900 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f9f6408ded0 tx=0x7f9f5c00d430 comp rx=0 tx=0).stop 2023-10-16T11:20:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 shutdown_connections 2023-10-16T11:20:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f9f4806b140 0x7f9f4806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f50003bc0 0x7f9f641f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f64160470 0x7f9f64160900 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 --2- 172.21.15.181:0/4133244850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f641f6be0 0x7f9f64155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 >> 172.21.15.181:0/4133244850 conn(0x7f9f6409cb10 msgr2=0x7f9f6406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 shutdown_connections 2023-10-16T11:20:56.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.236+0000 7f9f6b5de700 1 -- 172.21.15.181:0/4133244850 wait complete. 2023-10-16T11:20:56.241 INFO:teuthology.orchestra.run.smithi181.stderr:listed 10 entries 2023-10-16T11:20:56.251 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:20:56.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:56.251 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:20:56.265 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:56.264+0000 7fce16c49200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:20:56.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:20:56.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:20:56.296 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:20:56.313 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:20:56.313 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:56.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:20:56.330 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:20:56.331 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:56.333 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:56.331+0000 7f8a4cb6b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:56.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:20:56.351 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:20:56.351 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:20:56.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:20:56.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:56.414 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:56.413+0000 7fce16c49200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:56.415 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:56.413+0000 7f8a4cb6b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:20:56.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.512+0000 7fbeaeaf1700 1 Processor -- start 2023-10-16T11:20:56.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.512+0000 7fbeaeaf1700 1 -- start start 2023-10-16T11:20:56.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbeaeaf1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbea806d5f0 0x7fbea80d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbeaeaf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea8160d20 0x7fbea81611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbeaeaf1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea81616f0 con 0x7fbea8161b60 2023-10-16T11:20:56.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbeaeaf1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea8162800 con 0x7fbea806d5f0 2023-10-16T11:20:56.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbeaeaf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80d43e0 con 0x7fbea8160d20 2023-10-16T11:20:56.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbeac88d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbea8161b60 0x7fbea8161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41740/0 (socket says 172.21.15.181:41740) 2023-10-16T11:20:56.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbeac88d700 1 -- 172.21.15.181:0/3464937899 learned_addr learned my addr 172.21.15.181:0/3464937899 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:56.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbea7fff700 1 --2- 172.21.15.181:0/3464937899 >> v2:172.21.15.190:3300/0 conn(0x7fbea806d5f0 0x7fbea80d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbead08e700 1 --2- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea8160d20 0x7fbea81611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbea77fe700 1 -- 172.21.15.181:0/3464937899 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28786245 0 0) 0x7fbea81616f0 con 0x7fbea8161b60 2023-10-16T11:20:56.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbea77fe700 1 -- 172.21.15.181:0/3464937899 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe980035b0 con 0x7fbea8161b60 2023-10-16T11:20:56.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbea7fff700 1 -- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea8160d20 msgr2=0x7fbea81611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:56.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbea7fff700 1 --2- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea8160d20 0x7fbea81611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbea7fff700 1 -- 172.21.15.181:0/3464937899 >> v1:172.21.15.181:6789/0 conn(0x7fbea8161b60 legacy=0x7fbea8161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.513+0000 7fbea7fff700 1 -- 172.21.15.181:0/3464937899 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80d2560 con 0x7fbea806d5f0 2023-10-16T11:20:56.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbea7fff700 1 --2- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea806d5f0 0x7fbea80d2020 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009840 tx=0x7fbe94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f330c1a840bfeb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:56.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbea77fe700 1 -- 172.21.15.181:0/3464937899 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94011830 con 0x7fbea806d5f0 2023-10-16T11:20:56.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbea77fe700 1 -- 172.21.15.181:0/3464937899 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe94011990 con 0x7fbea806d5f0 2023-10-16T11:20:56.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbea77fe700 1 -- 172.21.15.181:0/3464937899 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94011d00 con 0x7fbea806d5f0 2023-10-16T11:20:56.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea806d5f0 msgr2=0x7fbea80d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea806d5f0 0x7fbea80d2020 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009840 tx=0x7fbe94009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:56.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/3464937899 shutdown_connections 2023-10-16T11:20:56.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea8160d20 0x7fbea81611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/3464937899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea806d5f0 0x7fbea80d2020 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/3464937899 >> 172.21.15.181:0/3464937899 conn(0x7fbea809cba0 msgr2=0x7fbea80d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:56.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/3464937899 shutdown_connections 2023-10-16T11:20:56.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.514+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/3464937899 wait complete. 2023-10-16T11:20:56.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.516+0000 7fbeaeaf1700 1 Processor -- start 2023-10-16T11:20:56.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.516+0000 7fbeaeaf1700 1 -- start start 2023-10-16T11:20:56.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.516+0000 7fbeaeaf1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea806d5f0 0x7fbea8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.516+0000 7fbeaeaf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea8160d20 0x7fbea80c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.516+0000 7fbeaeaf1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbea8161b60 0x7fbea80b8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.516+0000 7fbeaeaf1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80c94d0 con 0x7fbea8161b60 2023-10-16T11:20:56.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.516+0000 7fbeaeaf1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea8153170 con 0x7fbea8160d20 2023-10-16T11:20:56.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbeaeaf1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80b6fd0 con 0x7fbea806d5f0 2023-10-16T11:20:56.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbea8161b60 0x7fbea80b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbea8161b60 0x7fbea80b8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58904/0 (socket says 172.21.15.181:58904) 2023-10-16T11:20:56.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 -- 172.21.15.181:0/2364266752 learned_addr learned my addr 172.21.15.181:0/2364266752 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:56.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbeac88d700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea806d5f0 0x7fbea8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbea7fff700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea8160d20 0x7fbea80c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 -- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea806d5f0 msgr2=0x7fbea8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea806d5f0 0x7fbea8152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 -- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea8160d20 msgr2=0x7fbea80c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea8160d20 0x7fbea80c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 -- 172.21.15.181:0/2364266752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe940094a0 con 0x7fbea8161b60 2023-10-16T11:20:56.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbeac88d700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea806d5f0 0x7fbea8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:56.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbead08e700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbea8161b60 0x7fbea80b8290 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fbea000ec10 tx=0x7fbea000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:56.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbea57fa700 1 -- 172.21.15.181:0/2364266752 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea000ca00 con 0x7fbea8161b60 2023-10-16T11:20:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbea57fa700 1 -- 172.21.15.181:0/2364266752 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea000f040 con 0x7fbea8161b60 2023-10-16T11:20:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea80b72b0 con 0x7fbea8161b60 2023-10-16T11:20:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbea57fa700 1 -- 172.21.15.181:0/2364266752 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea000cda0 con 0x7fbea8161b60 2023-10-16T11:20:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.517+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea80b7870 con 0x7fbea8161b60 2023-10-16T11:20:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.518+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea804ebb0 con 0x7fbea8161b60 2023-10-16T11:20:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.519+0000 7fbea57fa700 1 -- 172.21.15.181:0/2364266752 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fbea0014480 con 0x7fbea8161b60 2023-10-16T11:20:56.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.519+0000 7fbea57fa700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 0x7fbe8c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.519+0000 7fbeac88d700 1 -- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 msgr2=0x7fbe8c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:56.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.519+0000 7fbea57fa700 1 -- 172.21.15.181:0/2364266752 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fbea008a850 con 0x7fbea8161b60 2023-10-16T11:20:56.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.519+0000 7fbeac88d700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 0x7fbe8c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:56.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.524+0000 7fbea57fa700 1 -- 172.21.15.181:0/2364266752 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbea0054de0 con 0x7fbea8161b60 2023-10-16T11:20:56.570 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:56.569+0000 7f8a4cb6b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:20:56.644 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:56.643+0000 7fce16c49200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:20:56.718 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:56.717+0000 7fce16c49200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:20:56.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.720+0000 7fbeac88d700 1 -- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 msgr2=0x7fbe8c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:56.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.720+0000 7fbeac88d700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 0x7fbe8c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:56.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.767+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbea80d0db0 con 0x7fbea8161b60 2023-10-16T11:20:56.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.768+0000 7fbea57fa700 1 -- 172.21.15.181:0/2364266752 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+6164 (secure 0 0 0) 0x7fbea0018020 con 0x7fbea8161b60 2023-10-16T11:20:56.770 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:56.770 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:20:56.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 msgr2=0x7fbe8c06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:56.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 0x7fbe8c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbea8161b60 msgr2=0x7fbea80b8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbea8161b60 0x7fbea80b8290 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fbea000ec10 tx=0x7fbea000ef20 comp rx=0 tx=0).stop 2023-10-16T11:20:56.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 shutdown_connections 2023-10-16T11:20:56.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbe8c06b140 0x7fbe8c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbea806d5f0 0x7fbea8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbea8161b60 0x7fbea80b8290 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 --2- 172.21.15.181:0/2364266752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbea8160d20 0x7fbea80c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 >> 172.21.15.181:0/2364266752 conn(0x7fbea809cba0 msgr2=0x7fbea81601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:56.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 shutdown_connections 2023-10-16T11:20:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.770+0000 7fbeaeaf1700 1 -- 172.21.15.181:0/2364266752 wait complete. 2023-10-16T11:20:56.775 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 133 2023-10-16T11:20:56.785 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:56.807 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:56.805+0000 7f8a4cb6b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:20:56.887 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:56.885+0000 7f8a4cb6b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:20:56.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.928+0000 7fd84b492700 1 Processor -- start 2023-10-16T11:20:56.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.928+0000 7fd84b492700 1 -- start start 2023-10-16T11:20:56.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84b492700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd844161b60 0x7fd84406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84b492700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84406ce20 con 0x7fd844160d20 2023-10-16T11:20:56.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84b492700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8440d45f0 con 0x7fd844161f40 2023-10-16T11:20:56.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84b492700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd844162800 con 0x7fd844161b60 2023-10-16T11:20:56.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd849a2f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd844160d20 0x7fd8440b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41760/0 (socket says 172.21.15.181:41760) 2023-10-16T11:20:56.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd849a2f700 1 -- 172.21.15.181:0/3878086017 learned_addr learned my addr 172.21.15.181:0/3878086017 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:56.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84922e700 1 --2- 172.21.15.181:0/3878086017 >> v2:172.21.15.190:3300/0 conn(0x7fd844161b60 0x7fd84406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd83bfff700 1 -- 172.21.15.181:0/3878086017 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347718457 0 0) 0x7fd84406ce20 con 0x7fd844160d20 2023-10-16T11:20:56.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd83bfff700 1 -- 172.21.15.181:0/3878086017 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8300035b0 con 0x7fd844160d20 2023-10-16T11:20:56.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84922e700 1 -- 172.21.15.181:0/3878086017 >> v1:172.21.15.190:6789/0 conn(0x7fd844161f40 legacy=0x7fd8440d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84922e700 1 -- 172.21.15.181:0/3878086017 >> v1:172.21.15.181:6789/0 conn(0x7fd844160d20 legacy=0x7fd8440b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84922e700 1 -- 172.21.15.181:0/3878086017 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8441623c0 con 0x7fd844161b60 2023-10-16T11:20:56.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd84922e700 1 --2- 172.21.15.181:0/3878086017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844161b60 0x7fd84406d5f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd840009820 tx=0x7fd840009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67c2c09ca70a144 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:56.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.929+0000 7fd83bfff700 1 -- 172.21.15.181:0/3878086017 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840005e30 con 0x7fd844161b60 2023-10-16T11:20:56.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd83bfff700 1 -- 172.21.15.181:0/3878086017 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840006030 con 0x7fd844161b60 2023-10-16T11:20:56.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd83bfff700 1 -- 172.21.15.181:0/3878086017 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400063c0 con 0x7fd844161b60 2023-10-16T11:20:56.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd84b492700 1 -- 172.21.15.181:0/3878086017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844161b60 msgr2=0x7fd84406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd84b492700 1 --2- 172.21.15.181:0/3878086017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844161b60 0x7fd84406d5f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd840009820 tx=0x7fd840009b30 comp rx=0 tx=0).stop 2023-10-16T11:20:56.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd84b492700 1 -- 172.21.15.181:0/3878086017 shutdown_connections 2023-10-16T11:20:56.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd84b492700 1 --2- 172.21.15.181:0/3878086017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844161b60 0x7fd84406d5f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd84b492700 1 -- 172.21.15.181:0/3878086017 >> 172.21.15.181:0/3878086017 conn(0x7fd84409cba0 msgr2=0x7fd8440d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd84b492700 1 -- 172.21.15.181:0/3878086017 shutdown_connections 2023-10-16T11:20:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.930+0000 7fd84b492700 1 -- 172.21.15.181:0/3878086017 wait complete. 2023-10-16T11:20:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.931+0000 7fd84b492700 1 Processor -- start 2023-10-16T11:20:56.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84b492700 1 -- start start 2023-10-16T11:20:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84b492700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd844156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84b492700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844161b60 0x7fd84415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84b492700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161f40 0x7fd8440b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84b492700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b75a0 con 0x7fd844161b60 2023-10-16T11:20:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84b492700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441574a0 con 0x7fd844160d20 2023-10-16T11:20:56.940 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84b492700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b77b0 con 0x7fd844161f40 2023-10-16T11:20:56.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd848a2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844161b60 0x7fd84415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84922e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd844156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84922e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd844156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:35734/0 (socket says 172.21.15.181:35734) 2023-10-16T11:20:56.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd848a2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844161b60 0x7fd84415d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58918/0 (socket says 172.21.15.181:58918) 2023-10-16T11:20:56.941 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd84922e700 1 -- 172.21.15.181:0/2236085122 learned_addr learned my addr 172.21.15.181:0/2236085122 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:56.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.932+0000 7fd849a2f700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161f40 0x7fd8440b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:56.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd848a2d700 1 -- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161f40 msgr2=0x7fd8440b8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd848a2d700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161f40 0x7fd8440b8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd848a2d700 1 -- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 msgr2=0x7fd844156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:56.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd848a2d700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd844156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:56.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd848a2d700 1 -- 172.21.15.181:0/2236085122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840009430 con 0x7fd844161b60 2023-10-16T11:20:56.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd84922e700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd844156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:56.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd848a2d700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844161b60 0x7fd84415d240 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fd834000f60 tx=0x7fd83400c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:56.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd839ffb700 1 -- 172.21.15.181:0/2236085122 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8340052d0 con 0x7fd844161b60 2023-10-16T11:20:56.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd839ffb700 1 -- 172.21.15.181:0/2236085122 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834005430 con 0x7fd844161b60 2023-10-16T11:20:56.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8440b7a90 con 0x7fd844161b60 2023-10-16T11:20:56.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd839ffb700 1 -- 172.21.15.181:0/2236085122 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8340038c0 con 0x7fd844161b60 2023-10-16T11:20:56.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.933+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8440b8050 con 0x7fd844161b60 2023-10-16T11:20:56.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.934+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84404ebb0 con 0x7fd844161b60 2023-10-16T11:20:56.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.935+0000 7fd839ffb700 1 -- 172.21.15.181:0/2236085122 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fd834003c30 con 0x7fd844161b60 2023-10-16T11:20:56.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.935+0000 7fd839ffb700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 0x7fd82806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:56.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.935+0000 7fd84922e700 1 -- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 msgr2=0x7fd82806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:56.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.935+0000 7fd839ffb700 1 -- 172.21.15.181:0/2236085122 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fd8340894c0 con 0x7fd844161b60 2023-10-16T11:20:56.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.935+0000 7fd84922e700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 0x7fd82806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:56.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:56.940+0000 7fd839ffb700 1 -- 172.21.15.181:0/2236085122 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd834053ba0 con 0x7fd844161b60 2023-10-16T11:20:56.984 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:56.983+0000 7fce16c49200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:20:57.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.136+0000 7fd84922e700 1 -- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 msgr2=0x7fd82806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:57.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.136+0000 7fd84922e700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 0x7fd82806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:57.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.140+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8440b20b0 con 0x7fd844161b60 2023-10-16T11:20:57.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.142+0000 7fd839ffb700 1 -- 172.21.15.181:0/2236085122 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+21709 (secure 0 0 0) 0x7fd8340571c0 con 0x7fd844161b60 2023-10-16T11:20:57.144 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:57.145 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":241,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:51.419932+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:57.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 msgr2=0x7fd82806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:57.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 0x7fd82806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844161b60 msgr2=0x7fd84415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844161b60 0x7fd84415d240 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fd834000f60 tx=0x7fd83400c6e0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 shutdown_connections 2023-10-16T11:20:57.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fd82806b140 0x7fd82806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161f40 0x7fd8440b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd844156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 --2- 172.21.15.181:0/2236085122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd844161b60 0x7fd84415d240 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 >> 172.21.15.181:0/2236085122 conn(0x7fd84409cba0 msgr2=0x7fd8440d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:57.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 shutdown_connections 2023-10-16T11:20:57.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.144+0000 7fd84b492700 1 -- 172.21.15.181:0/2236085122 wait complete. 2023-10-16T11:20:57.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:20:23.899617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '235', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:20:24.428675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:20:36.190463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '239', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:20:37.509605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '240', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:20:57.161 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:20:57.167 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:57.166+0000 7f8a4cb6b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:20:57.210 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:57.209+0000 7fce16c49200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:20:57.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.285+0000 7fbcab9f8700 1 Processor -- start 2023-10-16T11:20:57.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.285+0000 7fbcab9f8700 1 -- start start 2023-10-16T11:20:57.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbcab9f8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbca40d07d0 0x7fbca40b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbcab9f8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca40d2960 con 0x7fbca415d9e0 2023-10-16T11:20:57.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbcab9f8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca40d4180 con 0x7fbca40d24e0 2023-10-16T11:20:57.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbcab9f8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca40d20c0 con 0x7fbca40d07d0 2023-10-16T11:20:57.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbca9794700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbca415d9e0 0x7fbca40cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41774/0 (socket says 172.21.15.181:41774) 2023-10-16T11:20:57.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbca9794700 1 -- 172.21.15.181:0/3269764800 learned_addr learned my addr 172.21.15.181:0/3269764800 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:57.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbca9f95700 1 --2- 172.21.15.181:0/3269764800 >> v2:172.21.15.190:3300/0 conn(0x7fbca40d07d0 0x7fbca40b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbc9bfff700 1 -- 172.21.15.181:0/3269764800 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514021525 0 0) 0x7fbca40d2960 con 0x7fbca415d9e0 2023-10-16T11:20:57.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.286+0000 7fbc9bfff700 1 -- 172.21.15.181:0/3269764800 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc900035b0 con 0x7fbca415d9e0 2023-10-16T11:20:57.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.287+0000 7fbca9f95700 1 -- 172.21.15.181:0/3269764800 >> v1:172.21.15.190:6789/0 conn(0x7fbca40d24e0 legacy=0x7fbca40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.287+0000 7fbca9f95700 1 -- 172.21.15.181:0/3269764800 >> v1:172.21.15.181:6789/0 conn(0x7fbca415d9e0 legacy=0x7fbca40cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.287+0000 7fbca9f95700 1 -- 172.21.15.181:0/3269764800 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40d1440 con 0x7fbca40d07d0 2023-10-16T11:20:57.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.287+0000 7fbca9f95700 1 --2- 172.21.15.181:0/3269764800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40b6620 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009d90 tx=0x7fbc9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27a8060bf1aa8f72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:57.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.287+0000 7fbc9bfff700 1 -- 172.21.15.181:0/3269764800 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c00e070 con 0x7fbca40d07d0 2023-10-16T11:20:57.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.287+0000 7fbc9bfff700 1 -- 172.21.15.181:0/3269764800 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c00bb20 con 0x7fbca40d07d0 2023-10-16T11:20:57.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.287+0000 7fbc9bfff700 1 -- 172.21.15.181:0/3269764800 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c012430 con 0x7fbca40d07d0 2023-10-16T11:20:57.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.288+0000 7fbcab9f8700 1 -- 172.21.15.181:0/3269764800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 msgr2=0x7fbca40b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.288+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/3269764800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40b6620 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009d90 tx=0x7fbc9c00b330 comp rx=0 tx=0).stop 2023-10-16T11:20:57.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.288+0000 7fbcab9f8700 1 -- 172.21.15.181:0/3269764800 shutdown_connections 2023-10-16T11:20:57.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.288+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/3269764800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40b6620 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.288+0000 7fbcab9f8700 1 -- 172.21.15.181:0/3269764800 >> 172.21.15.181:0/3269764800 conn(0x7fbca409cb40 msgr2=0x7fbca40d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:57.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.288+0000 7fbcab9f8700 1 -- 172.21.15.181:0/3269764800 shutdown_connections 2023-10-16T11:20:57.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.288+0000 7fbcab9f8700 1 -- 172.21.15.181:0/3269764800 wait complete. 2023-10-16T11:20:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.289+0000 7fbcab9f8700 1 Processor -- start 2023-10-16T11:20:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.289+0000 7fbcab9f8700 1 -- start start 2023-10-16T11:20:57.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbcab9f8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbcab9f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca40d24e0 0x7fbca415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbcab9f8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca415d9e0 0x7fbca415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbcab9f8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40cc320 con 0x7fbca40d24e0 2023-10-16T11:20:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbcab9f8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40c91e0 con 0x7fbca40d07d0 2023-10-16T11:20:57.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbcab9f8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40c85d0 con 0x7fbca415d9e0 2023-10-16T11:20:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbca8f93700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca40d24e0 0x7fbca415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbca9794700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbca8f93700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca40d24e0 0x7fbca415b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58936/0 (socket says 172.21.15.181:58936) 2023-10-16T11:20:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.290+0000 7fbca8f93700 1 -- 172.21.15.181:0/475487476 learned_addr learned my addr 172.21.15.181:0/475487476 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:57.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca9f95700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca415d9e0 0x7fbca415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca8f93700 1 -- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca415d9e0 msgr2=0x7fbca415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca8f93700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca415d9e0 0x7fbca415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca8f93700 1 -- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 msgr2=0x7fbca40cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca8f93700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca8f93700 1 -- 172.21.15.181:0/475487476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c009a70 con 0x7fbca40d24e0 2023-10-16T11:20:57.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca9794700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbca8f93700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca40d24e0 0x7fbca415b5e0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c700 tx=0x7fbc9400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbc99ffb700 1 -- 172.21.15.181:0/475487476 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940052d0 con 0x7fbca40d24e0 2023-10-16T11:20:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbc99ffb700 1 -- 172.21.15.181:0/475487476 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc94005430 con 0x7fbca40d24e0 2023-10-16T11:20:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca40c88b0 con 0x7fbca40d24e0 2023-10-16T11:20:57.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbc99ffb700 1 -- 172.21.15.181:0/475487476 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94003880 con 0x7fbca40d24e0 2023-10-16T11:20:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.291+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca40c8e70 con 0x7fbca40d24e0 2023-10-16T11:20:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.292+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca4067d20 con 0x7fbca40d24e0 2023-10-16T11:20:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.293+0000 7fbc99ffb700 1 -- 172.21.15.181:0/475487476 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fbc940039e0 con 0x7fbca40d24e0 2023-10-16T11:20:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.293+0000 7fbc99ffb700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 0x7fbc8806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.293+0000 7fbca9794700 1 -- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 msgr2=0x7fbc8806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.293+0000 7fbc99ffb700 1 -- 172.21.15.181:0/475487476 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fbc94089320 con 0x7fbca40d24e0 2023-10-16T11:20:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.293+0000 7fbca9794700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 0x7fbc8806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:57.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.298+0000 7fbc99ffb700 1 -- 172.21.15.181:0/475487476 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc94053ab0 con 0x7fbca40d24e0 2023-10-16T11:20:57.363 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:57.362+0000 7fce16c49200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:20:57.406 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:57.405+0000 7f8a4cb6b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:20:57.479 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:57.477+0000 7fce16c49200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:20:57.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.494+0000 7fbca9794700 1 -- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 msgr2=0x7fbc8806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:57.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.494+0000 7fbca9794700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 0x7fbc8806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:57.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.529+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbca40d0e50 con 0x7fbca40d24e0 2023-10-16T11:20:57.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.530+0000 7fbc99ffb700 1 -- 172.21.15.181:0/475487476 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v241) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc940570d0 con 0x7fbca40d24e0 2023-10-16T11:20:57.532 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:20:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 msgr2=0x7fbc8806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 0x7fbc8806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca40d24e0 msgr2=0x7fbca415b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca40d24e0 0x7fbca415b5e0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c700 tx=0x7fbc9400ca10 comp rx=0 tx=0).stop 2023-10-16T11:20:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 shutdown_connections 2023-10-16T11:20:57.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fbc8806b140 0x7fbc8806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbca415d9e0 0x7fbca415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbca40d07d0 0x7fbca40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 --2- 172.21.15.181:0/475487476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbca40d24e0 0x7fbca415b5e0 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 >> 172.21.15.181:0/475487476 conn(0x7fbca409cb40 msgr2=0x7fbca415f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:57.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 shutdown_connections 2023-10-16T11:20:57.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.532+0000 7fbcab9f8700 1 -- 172.21.15.181:0/475487476 wait complete. 2023-10-16T11:20:57.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:20:57.587 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:57.585+0000 7f8a4cb6b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:20:57.636 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:57.634+0000 7fce16c49200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:20:57.709 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:57.707+0000 7f8a4cb6b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:20:57.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.711+0000 7f9459d1a700 1 Processor -- start 2023-10-16T11:20:57.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.711+0000 7f9459d1a700 1 -- start start 2023-10-16T11:20:57.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9459d1a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94540d01d0 0x7f94540ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9459d1a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94540d1500 0x7f94540d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9459d1a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94540cf580 con 0x7f945415ea40 2023-10-16T11:20:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9459d1a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94540d4290 con 0x7f94540d01d0 2023-10-16T11:20:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9459d1a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540ce970 con 0x7f94540d1500 2023-10-16T11:20:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f94537fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94540d01d0 0x7f94540ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9452ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f945415ea40 0x7f94540d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41790/0 (socket says 172.21.15.181:41790) 2023-10-16T11:20:57.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9452ffd700 1 -- 172.21.15.181:0/826316257 learned_addr learned my addr 172.21.15.181:0/826316257 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:57.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f9453fff700 1 --2- 172.21.15.181:0/826316257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94540d1500 0x7f94540d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f94537fe700 1 -- 172.21.15.181:0/826316257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94540d1500 msgr2=0x7f94540d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:57.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f94537fe700 1 --2- 172.21.15.181:0/826316257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94540d1500 0x7f94540d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f94537fe700 1 -- 172.21.15.181:0/826316257 >> v1:172.21.15.181:6789/0 conn(0x7f945415ea40 legacy=0x7f94540d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.712+0000 7f94537fe700 1 -- 172.21.15.181:0/826316257 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540cec50 con 0x7f94540d01d0 2023-10-16T11:20:57.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f94537fe700 1 --2- 172.21.15.181:0/826316257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94540ce2d0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f9444009840 tx=0x7f9444009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc73dc79c352f7c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:57.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f94527fc700 1 -- 172.21.15.181:0/826316257 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444011830 con 0x7f94540d01d0 2023-10-16T11:20:57.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f94527fc700 1 -- 172.21.15.181:0/826316257 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444011990 con 0x7f94540d01d0 2023-10-16T11:20:57.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f94527fc700 1 -- 172.21.15.181:0/826316257 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444011d20 con 0x7f94540d01d0 2023-10-16T11:20:57.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 -- 172.21.15.181:0/826316257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 msgr2=0x7f94540ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 --2- 172.21.15.181:0/826316257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94540ce2d0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f9444009840 tx=0x7f9444009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:57.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 -- 172.21.15.181:0/826316257 shutdown_connections 2023-10-16T11:20:57.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 --2- 172.21.15.181:0/826316257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94540d1500 0x7f94540d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 --2- 172.21.15.181:0/826316257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94540ce2d0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 -- 172.21.15.181:0/826316257 >> 172.21.15.181:0/826316257 conn(0x7f945409cb60 msgr2=0x7f94540d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:57.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 -- 172.21.15.181:0/826316257 shutdown_connections 2023-10-16T11:20:57.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.713+0000 7f9459d1a700 1 -- 172.21.15.181:0/826316257 wait complete. 2023-10-16T11:20:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.715+0000 7f9459d1a700 1 Processor -- start 2023-10-16T11:20:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.715+0000 7f9459d1a700 1 -- start start 2023-10-16T11:20:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9459d1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94541f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9459d1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94540d1500 0x7f94540b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9459d1a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945415ea40 0x7f94540b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9459d1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541f4460 con 0x7f94540d01d0 2023-10-16T11:20:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9459d1a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541f3a90 con 0x7f94540d1500 2023-10-16T11:20:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9459d1a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94541f3460 con 0x7f945415ea40 2023-10-16T11:20:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94541f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94541f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58950/0 (socket says 172.21.15.181:58950) 2023-10-16T11:20:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 -- 172.21.15.181:0/2014096570 learned_addr learned my addr 172.21.15.181:0/2014096570 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9453fff700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945415ea40 0x7f94540b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9452ffd700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94540d1500 0x7f94540b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 -- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945415ea40 msgr2=0x7f94540b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945415ea40 0x7f94540b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 -- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94540d1500 msgr2=0x7f94540b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94540d1500 0x7f94540b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f94537fe700 1 -- 172.21.15.181:0/2014096570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440094a0 con 0x7f94540d01d0 2023-10-16T11:20:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9453fff700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945415ea40 0x7f94540b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:57.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.716+0000 7f9452ffd700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94540d1500 0x7f94540b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:57.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.717+0000 7f94537fe700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94541f3f20 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f9444009b90 tx=0x7f9444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:57.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.717+0000 7f943bfff700 1 -- 172.21.15.181:0/2014096570 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94440036a0 con 0x7f94540d01d0 2023-10-16T11:20:57.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.717+0000 7f943bfff700 1 -- 172.21.15.181:0/2014096570 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944401f070 con 0x7f94540d01d0 2023-10-16T11:20:57.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.717+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94540b3a20 con 0x7f94540d01d0 2023-10-16T11:20:57.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.717+0000 7f943bfff700 1 -- 172.21.15.181:0/2014096570 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444003a40 con 0x7f94540d01d0 2023-10-16T11:20:57.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.717+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94541f3720 con 0x7f94540d01d0 2023-10-16T11:20:57.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.718+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94541f3d70 con 0x7f94540d01d0 2023-10-16T11:20:57.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.719+0000 7f943bfff700 1 -- 172.21.15.181:0/2014096570 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f9444003ba0 con 0x7f94540d01d0 2023-10-16T11:20:57.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.719+0000 7f943bfff700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 0x7f944006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:57.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.719+0000 7f9452ffd700 1 -- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 msgr2=0x7f944006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:57.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.719+0000 7f943bfff700 1 -- 172.21.15.181:0/2014096570 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7f9444021070 con 0x7f94540d01d0 2023-10-16T11:20:57.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.719+0000 7f9452ffd700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 0x7f944006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:57.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.723+0000 7f943bfff700 1 -- 172.21.15.181:0/2014096570 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f94440561f0 con 0x7f94540d01d0 2023-10-16T11:20:57.734 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:57.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:20:57.875 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:57.873+0000 7f8a4cb6b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:20:57.901 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:57.900+0000 7fce16c49200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:57.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.920+0000 7f9452ffd700 1 -- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 msgr2=0x7f944006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:57.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.920+0000 7f9452ffd700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 0x7f944006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:57.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.928+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94540b3f40 con 0x7f94540d01d0 2023-10-16T11:20:57.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.929+0000 7f943bfff700 1 -- 172.21.15.181:0/2014096570 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v241) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9444059810 con 0x7f94540d01d0 2023-10-16T11:20:57.931 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:57.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 msgr2=0x7f944006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:57.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 0x7f944006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 msgr2=0x7f94541f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:57.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94541f3f20 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f9444009b90 tx=0x7f9444004750 comp rx=0 tx=0).stop 2023-10-16T11:20:57.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 shutdown_connections 2023-10-16T11:20:57.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f944006b140 0x7f944006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945415ea40 0x7f94540b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94540d01d0 0x7f94541f3f20 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 --2- 172.21.15.181:0/2014096570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94540d1500 0x7f94540b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:57.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 >> 172.21.15.181:0/2014096570 conn(0x7f945409cb60 msgr2=0x7f945415f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:57.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 shutdown_connections 2023-10-16T11:20:57.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:57.931+0000 7f9459d1a700 1 -- 172.21.15.181:0/2014096570 wait complete. 2023-10-16T11:20:57.947 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:20:58.063 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:58.061+0000 7fce16c49200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:20:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.074+0000 7f1a5805d700 1 Processor -- start 2023-10-16T11:20:58.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.074+0000 7f1a5805d700 1 -- start start 2023-10-16T11:20:58.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a5805d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d24e0 0x7f1a500d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a5805d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1a500d0600 0x7f1a500b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a5805d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a500d2960 con 0x7f1a5015d9e0 2023-10-16T11:20:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a5805d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a500d20c0 con 0x7f1a500d0600 2023-10-16T11:20:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a5805d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500d0a90 con 0x7f1a500d24e0 2023-10-16T11:20:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a555f8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d24e0 0x7f1a500d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a55df9700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1a5015d9e0 0x7f1a500cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:53980/0 (socket says 172.21.15.181:53980) 2023-10-16T11:20:58.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a55df9700 1 -- 172.21.15.181:0/2123597849 learned_addr learned my addr 172.21.15.181:0/2123597849 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a565fa700 1 --2- 172.21.15.181:0/2123597849 >> v2:172.21.15.190:3300/0 conn(0x7f1a500d0600 0x7f1a500b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a54df7700 1 -- 172.21.15.181:0/2123597849 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460684766 0 0) 0x7f1a500d2960 con 0x7f1a5015d9e0 2023-10-16T11:20:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.075+0000 7f1a54df7700 1 -- 172.21.15.181:0/2123597849 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a3c0035b0 con 0x7f1a5015d9e0 2023-10-16T11:20:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a565fa700 1 -- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d24e0 msgr2=0x7f1a500d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a565fa700 1 --2- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d24e0 0x7f1a500d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a565fa700 1 -- 172.21.15.181:0/2123597849 >> v1:172.21.15.190:6789/0 conn(0x7f1a5015d9e0 legacy=0x7f1a500cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a565fa700 1 -- 172.21.15.181:0/2123597849 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a500d4180 con 0x7f1a500d0600 2023-10-16T11:20:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a555f8700 1 --2- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d24e0 0x7f1a500d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a565fa700 1 --2- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d0600 0x7f1a500b6620 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009d90 tx=0x7f1a4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3082b7fd69f09cd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a54df7700 1 -- 172.21.15.181:0/2123597849 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800e070 con 0x7f1a500d0600 2023-10-16T11:20:58.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a54df7700 1 -- 172.21.15.181:0/2123597849 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4800bb20 con 0x7f1a500d0600 2023-10-16T11:20:58.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.076+0000 7f1a54df7700 1 -- 172.21.15.181:0/2123597849 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48012430 con 0x7f1a500d0600 2023-10-16T11:20:58.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 -- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d0600 msgr2=0x7f1a500b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d0600 0x7f1a500b6620 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009d90 tx=0x7f1a4800b330 comp rx=0 tx=0).stop 2023-10-16T11:20:58.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 -- 172.21.15.181:0/2123597849 shutdown_connections 2023-10-16T11:20:58.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d0600 0x7f1a500b6620 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2123597849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d24e0 0x7f1a500d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 -- 172.21.15.181:0/2123597849 >> 172.21.15.181:0/2123597849 conn(0x7f1a5009cb40 msgr2=0x7f1a500d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:58.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 -- 172.21.15.181:0/2123597849 shutdown_connections 2023-10-16T11:20:58.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.077+0000 7f1a5805d700 1 -- 172.21.15.181:0/2123597849 wait complete. 2023-10-16T11:20:58.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.078+0000 7f1a5805d700 1 Processor -- start 2023-10-16T11:20:58.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.078+0000 7f1a5805d700 1 -- start start 2023-10-16T11:20:58.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a5805d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d0600 0x7f1a500c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a5805d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d24e0 0x7f1a500b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a5805d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5015d9e0 0x7f1a50159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a5805d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500cc540 con 0x7f1a5015d9e0 2023-10-16T11:20:58.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a5805d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500cbc30 con 0x7f1a500d24e0 2023-10-16T11:20:58.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a5805d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a500b41d0 con 0x7f1a500d0600 2023-10-16T11:20:58.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d24e0 0x7f1a500b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d24e0 0x7f1a500b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:35786/0 (socket says 172.21.15.181:35786) 2023-10-16T11:20:58.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 -- 172.21.15.181:0/2995984244 learned_addr learned my addr 172.21.15.181:0/2995984244 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:58.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a55df9700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d0600 0x7f1a500c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a565fa700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5015d9e0 0x7f1a50159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 -- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d0600 msgr2=0x7f1a500c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d0600 0x7f1a500c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 -- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5015d9e0 msgr2=0x7f1a50159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5015d9e0 0x7f1a50159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 -- 172.21.15.181:0/2995984244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400094a0 con 0x7f1a500d24e0 2023-10-16T11:20:58.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a565fa700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5015d9e0 0x7f1a50159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:58.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a55df9700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d0600 0x7f1a500c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:58.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.079+0000 7f1a555f8700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d24e0 0x7f1a500b3710 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000ebe0 tx=0x7f1a4000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:58.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.080+0000 7f1a467fc700 1 -- 172.21.15.181:0/2995984244 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000c850 con 0x7f1a500d24e0 2023-10-16T11:20:58.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.080+0000 7f1a467fc700 1 -- 172.21.15.181:0/2995984244 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4000c9b0 con 0x7f1a500d24e0 2023-10-16T11:20:58.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.080+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a48009a70 con 0x7f1a500d24e0 2023-10-16T11:20:58.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.080+0000 7f1a467fc700 1 -- 172.21.15.181:0/2995984244 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000cd40 con 0x7f1a500d24e0 2023-10-16T11:20:58.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.080+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a500cc160 con 0x7f1a500d24e0 2023-10-16T11:20:58.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.081+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a500cc680 con 0x7f1a500d24e0 2023-10-16T11:20:58.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.081+0000 7f1a467fc700 1 -- 172.21.15.181:0/2995984244 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f1a40013470 con 0x7f1a500d24e0 2023-10-16T11:20:58.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.081+0000 7f1a467fc700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 0x7f1a3406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.081+0000 7f1a55df9700 1 -- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 msgr2=0x7f1a3406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:58.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.081+0000 7f1a467fc700 1 -- 172.21.15.181:0/2995984244 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7f1a4008a120 con 0x7f1a500d24e0 2023-10-16T11:20:58.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.081+0000 7f1a55df9700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 0x7f1a3406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:58.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.087+0000 7f1a467fc700 1 -- 172.21.15.181:0/2995984244 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a500cc680 con 0x7f1a500d24e0 2023-10-16T11:20:58.158 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.157+0000 7f8a4cb6b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:20:58.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.282+0000 7f1a55df9700 1 -- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 msgr2=0x7f1a3406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:58.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.282+0000 7f1a55df9700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 0x7f1a3406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:58.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.287+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a500cc680 con 0x7f1a500d24e0 2023-10-16T11:20:58.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.288+0000 7f1a467fc700 1 -- 172.21.15.181:0/2995984244 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v241) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a500cc680 con 0x7f1a500d24e0 2023-10-16T11:20:58.290 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:58.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 msgr2=0x7f1a3406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:58.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 0x7f1a3406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d24e0 msgr2=0x7f1a500b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d24e0 0x7f1a500b3710 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000ebe0 tx=0x7f1a4000eef0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 shutdown_connections 2023-10-16T11:20:58.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7f1a3406b140 0x7f1a3406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a500d0600 0x7f1a500c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5015d9e0 0x7f1a50159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 --2- 172.21.15.181:0/2995984244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a500d24e0 0x7f1a500b3710 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 >> 172.21.15.181:0/2995984244 conn(0x7f1a5009cb40 msgr2=0x7f1a501613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:58.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 shutdown_connections 2023-10-16T11:20:58.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.290+0000 7f1a5805d700 1 -- 172.21.15.181:0/2995984244 wait complete. 2023-10-16T11:20:58.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:20:58.327 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.325+0000 7f8a4cb6b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:20:58.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.428+0000 7fb595daf700 1 Processor -- start 2023-10-16T11:20:58.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.428+0000 7fb595daf700 1 -- start start 2023-10-16T11:20:58.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.429+0000 7fb595daf700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5900d01d0 0x7fb5900ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb595daf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d1500 0x7fb5900d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb595daf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5900cf580 con 0x7fb59015ea40 2023-10-16T11:20:58.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb595daf700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5900d4290 con 0x7fb5900d01d0 2023-10-16T11:20:58.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb595daf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900ce970 con 0x7fb5900d1500 2023-10-16T11:20:58.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb59015ea40 0x7fb5900d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41824/0 (socket says 172.21.15.181:41824) 2023-10-16T11:20:58.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58effd700 1 -- 172.21.15.181:0/803650094 learned_addr learned my addr 172.21.15.181:0/803650094 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:58.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58ffff700 1 --2- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d1500 0x7fb5900d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58f7fe700 1 --2- 172.21.15.181:0/803650094 >> v2:172.21.15.190:3300/0 conn(0x7fb5900d01d0 0x7fb5900ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58e7fc700 1 -- 172.21.15.181:0/803650094 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344823677 0 0) 0x7fb5900cf580 con 0x7fb59015ea40 2023-10-16T11:20:58.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58e7fc700 1 -- 172.21.15.181:0/803650094 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb57c0035b0 con 0x7fb59015ea40 2023-10-16T11:20:58.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58f7fe700 1 -- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d1500 msgr2=0x7fb5900d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:58.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58f7fe700 1 --2- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d1500 0x7fb5900d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58f7fe700 1 -- 172.21.15.181:0/803650094 >> v1:172.21.15.181:6789/0 conn(0x7fb59015ea40 legacy=0x7fb5900d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58f7fe700 1 -- 172.21.15.181:0/803650094 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5900cec50 con 0x7fb5900d01d0 2023-10-16T11:20:58.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58ffff700 1 --2- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d1500 0x7fb5900d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:20:58.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.430+0000 7fb58f7fe700 1 --2- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5900d01d0 0x7fb5900ce2d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb580009d90 tx=0x7fb58000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15703c97a17361ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:58.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb58e7fc700 1 -- 172.21.15.181:0/803650094 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58000e070 con 0x7fb5900d01d0 2023-10-16T11:20:58.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb58e7fc700 1 -- 172.21.15.181:0/803650094 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58000bb20 con 0x7fb5900d01d0 2023-10-16T11:20:58.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb58e7fc700 1 -- 172.21.15.181:0/803650094 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580012430 con 0x7fb5900d01d0 2023-10-16T11:20:58.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb595daf700 1 -- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5900d01d0 msgr2=0x7fb5900ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb595daf700 1 --2- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5900d01d0 0x7fb5900ce2d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb580009d90 tx=0x7fb58000b330 comp rx=0 tx=0).stop 2023-10-16T11:20:58.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb595daf700 1 -- 172.21.15.181:0/803650094 shutdown_connections 2023-10-16T11:20:58.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb595daf700 1 --2- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d1500 0x7fb5900d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb595daf700 1 --2- 172.21.15.181:0/803650094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5900d01d0 0x7fb5900ce2d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.431+0000 7fb595daf700 1 -- 172.21.15.181:0/803650094 >> 172.21.15.181:0/803650094 conn(0x7fb59009cb60 msgr2=0x7fb5900d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:58.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.432+0000 7fb595daf700 1 -- 172.21.15.181:0/803650094 shutdown_connections 2023-10-16T11:20:58.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.432+0000 7fb595daf700 1 -- 172.21.15.181:0/803650094 wait complete. 2023-10-16T11:20:58.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.433+0000 7fb595daf700 1 Processor -- start 2023-10-16T11:20:58.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.433+0000 7fb595daf700 1 -- start start 2023-10-16T11:20:58.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d01d0 0x7fb5900b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5900d1500 0x7fb5900c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb59015ea40 0x7fb5900b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900b17d0 con 0x7fb5900d1500 2023-10-16T11:20:58.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900b08f0 con 0x7fb59015ea40 2023-10-16T11:20:58.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900b0b50 con 0x7fb5900d01d0 2023-10-16T11:20:58.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5900d1500 0x7fb5900c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5900d1500 0x7fb5900c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:58984/0 (socket says 172.21.15.181:58984) 2023-10-16T11:20:58.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 -- 172.21.15.181:0/4026228849 learned_addr learned my addr 172.21.15.181:0/4026228849 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:58.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58f7fe700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d01d0 0x7fb5900b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58ffff700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb59015ea40 0x7fb5900b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 -- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d01d0 msgr2=0x7fb5900b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d01d0 0x7fb5900b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 -- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb59015ea40 msgr2=0x7fb5900b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb59015ea40 0x7fb5900b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 -- 172.21.15.181:0/4026228849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5880094a0 con 0x7fb5900d1500 2023-10-16T11:20:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb58effd700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5900d1500 0x7fb5900c9cc0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7fb57800c700 tx=0x7fb57800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb587fff700 1 -- 172.21.15.181:0/4026228849 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578005d70 con 0x7fb5900d1500 2023-10-16T11:20:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb587fff700 1 -- 172.21.15.181:0/4026228849 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5780070d0 con 0x7fb5900d1500 2023-10-16T11:20:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb587fff700 1 -- 172.21.15.181:0/4026228849 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578007460 con 0x7fb5900d1500 2023-10-16T11:20:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb580009a70 con 0x7fb5900d1500 2023-10-16T11:20:58.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.434+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5900b1110 con 0x7fb5900d1500 2023-10-16T11:20:58.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.435+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5900b1910 con 0x7fb5900d1500 2023-10-16T11:20:58.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.436+0000 7fb587fff700 1 -- 172.21.15.181:0/4026228849 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fb5780075c0 con 0x7fb5900d1500 2023-10-16T11:20:58.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.436+0000 7fb587fff700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 0x7fb57006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.437+0000 7fb58f7fe700 1 -- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 msgr2=0x7fb57006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:58.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.437+0000 7fb587fff700 1 -- 172.21.15.181:0/4026228849 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fb5780894e0 con 0x7fb5900d1500 2023-10-16T11:20:58.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.437+0000 7fb58f7fe700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 0x7fb57006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:58.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.441+0000 7fb587fff700 1 -- 172.21.15.181:0/4026228849 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb5900b1910 con 0x7fb5900d1500 2023-10-16T11:20:58.599 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:20:58.597+0000 7fce16c49200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:20:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.637+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5900b1910 con 0x7fb5900d1500 2023-10-16T11:20:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.638+0000 7fb58f7fe700 1 -- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 msgr2=0x7fb57006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:58.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.638+0000 7fb58f7fe700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 0x7fb57006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:20:58.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.638+0000 7fb587fff700 1 -- 172.21.15.181:0/4026228849 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v241) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb5900b1910 con 0x7fb5900d1500 2023-10-16T11:20:58.641 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:20:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 msgr2=0x7fb57006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 0x7fb57006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5900d1500 msgr2=0x7fb5900c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5900d1500 0x7fb5900c9cc0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7fb57800c700 tx=0x7fb57800ca10 comp rx=0 tx=0).stop 2023-10-16T11:20:58.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 shutdown_connections 2023-10-16T11:20:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fb57006b140 0x7fb57006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb5900d01d0 0x7fb5900b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb59015ea40 0x7fb5900b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 --2- 172.21.15.181:0/4026228849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5900d1500 0x7fb5900c9cc0 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.640+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 >> 172.21.15.181:0/4026228849 conn(0x7fb59009cb60 msgr2=0x7fb5900d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:58.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.641+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 shutdown_connections 2023-10-16T11:20:58.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.641+0000 7fb595daf700 1 -- 172.21.15.181:0/4026228849 wait complete. 2023-10-16T11:20:58.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:20:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.825+0000 7fa5b99b2700 1 Processor -- start 2023-10-16T11:20:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.825+0000 7fa5b99b2700 1 -- start start 2023-10-16T11:20:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b99b2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa5b415dbd0 0x7fa5b40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b99b2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b99b2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b415dfb0 con 0x7fa5b40ce2c0 2023-10-16T11:20:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b99b2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b40ce740 con 0x7fa5b415dbd0 2023-10-16T11:20:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b99b2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40cd3a0 con 0x7fa5b40d0610 2023-10-16T11:20:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b2ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa5b415dbd0 0x7fa5b40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa5b40ce2c0 0x7fa5b4161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41842/0 (socket says 172.21.15.181:41842) 2023-10-16T11:20:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b37fe700 1 -- 172.21.15.181:0/4210386142 learned_addr learned my addr 172.21.15.181:0/4210386142 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.826+0000 7fa5b27fc700 1 --2- 172.21.15.181:0/4210386142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b2ffd700 1 -- 172.21.15.181:0/4210386142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 msgr2=0x7fa5b40cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:58.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b2ffd700 1 --2- 172.21.15.181:0/4210386142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b2ffd700 1 -- 172.21.15.181:0/4210386142 >> v1:172.21.15.181:6789/0 conn(0x7fa5b40ce2c0 legacy=0x7fa5b4161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b1ffb700 1 -- 172.21.15.181:0/4210386142 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365150058 0 0) 0x7fa5ac001490 con 0x7fa5b40ce2c0 2023-10-16T11:20:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b2ffd700 1 -- 172.21.15.181:0/4210386142 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40cd830 con 0x7fa5b415dbd0 2023-10-16T11:20:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b2ffd700 1 --2- 172.21.15.181:0/4210386142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b40d00d0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009840 tx=0x7fa5a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbad55d09307e12e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b1ffb700 1 -- 172.21.15.181:0/4210386142 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4011830 con 0x7fa5b415dbd0 2023-10-16T11:20:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b1ffb700 1 -- 172.21.15.181:0/4210386142 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a4011990 con 0x7fa5b415dbd0 2023-10-16T11:20:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b1ffb700 1 -- 172.21.15.181:0/4210386142 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4011d20 con 0x7fa5b415dbd0 2023-10-16T11:20:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b99b2700 1 -- 172.21.15.181:0/4210386142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 msgr2=0x7fa5b40d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.827+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/4210386142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b40d00d0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009840 tx=0x7fa5a4009b50 comp rx=0 tx=0).stop 2023-10-16T11:20:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.828+0000 7fa5b99b2700 1 -- 172.21.15.181:0/4210386142 shutdown_connections 2023-10-16T11:20:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.828+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/4210386142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b40d00d0 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.828+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/4210386142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.828+0000 7fa5b99b2700 1 -- 172.21.15.181:0/4210386142 >> 172.21.15.181:0/4210386142 conn(0x7fa5b409a930 msgr2=0x7fa5b40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.828+0000 7fa5b99b2700 1 -- 172.21.15.181:0/4210386142 shutdown_connections 2023-10-16T11:20:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.828+0000 7fa5b99b2700 1 -- 172.21.15.181:0/4210386142 wait complete. 2023-10-16T11:20:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.829+0000 7fa5b99b2700 1 Processor -- start 2023-10-16T11:20:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.829+0000 7fa5b99b2700 1 -- start start 2023-10-16T11:20:58.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b40ce2c0 0x7fa5b4158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4160450 con 0x7fa5b415dbd0 2023-10-16T11:20:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40c45d0 con 0x7fa5b40ce2c0 2023-10-16T11:20:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4160680 con 0x7fa5b40d0610 2023-10-16T11:20:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b415ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59002/0 (socket says 172.21.15.181:59002) 2023-10-16T11:20:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 -- 172.21.15.181:0/267569927 learned_addr learned my addr 172.21.15.181:0/267569927 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b27fc700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b2ffd700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b40ce2c0 0x7fa5b4158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 -- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 msgr2=0x7fa5b40b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 -- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b40ce2c0 msgr2=0x7fa5b4158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b40ce2c0 0x7fa5b4158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 -- 172.21.15.181:0/267569927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a40094a0 con 0x7fa5b415dbd0 2023-10-16T11:20:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b2ffd700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b40ce2c0 0x7fa5b4158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b37fe700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b415ff10 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00cb30 tx=0x7fa5ac00ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00aaa0 con 0x7fa5b415dbd0 2023-10-16T11:20:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00ac00 con 0x7fa5b415dbd0 2023-10-16T11:20:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b4160960 con 0x7fa5b415dbd0 2023-10-16T11:20:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac0070d0 con 0x7fa5b415dbd0 2023-10-16T11:20:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.830+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b4160f20 con 0x7fa5b415dbd0 2023-10-16T11:20:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.831+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b40c4780 con 0x7fa5b415dbd0 2023-10-16T11:20:58.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.832+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 155) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fa5ac007230 con 0x7fa5b415dbd0 2023-10-16T11:20:58.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.833+0000 7fa59bfff700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fa5a006b140 0x7fa5a006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:58.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.833+0000 7fa5b2ffd700 1 -- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fa5a006b140 msgr2=0x7fa5a006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1878395836 2023-10-16T11:20:58.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.833+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7555+0+0 (secure 0 0 0) 0x7fa5ac089c40 con 0x7fa5b415dbd0 2023-10-16T11:20:58.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.833+0000 7fa5b2ffd700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fa5a006b140 0x7fa5a006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:20:58.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.837+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa5b40c4780 con 0x7fa5b415dbd0 2023-10-16T11:20:58.900 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.898+0000 7f8a4cb6b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:20:58.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.925+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 156) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa5ac0546b0 con 0x7fa5b415dbd0 2023-10-16T11:20:58.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.926+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fa5a006b140 msgr2=0x7fa5a006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:20:58.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:58.926+0000 7fa59bfff700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fa5a006b140 0x7fa5a006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:58.991 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.991 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.992 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.992 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.992 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.992 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.992 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.993 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.993 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:58.993 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.989+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.001 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.001 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.001 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.001 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.001 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c8357700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.002 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.002 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.002 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.003 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.003 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:20:58.999+0000 7f89c5351700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:20:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.058+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5b415dfb0 con 0x7fa5b415dbd0 2023-10-16T11:20:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.059+0000 7fa59bfff700 1 -- 172.21.15.181:0/267569927 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v242) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa5b415dfb0 con 0x7fa5b415dbd0 2023-10-16T11:20:59.061 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:20:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 msgr2=0x7fa5b415ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b415ff10 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00cb30 tx=0x7fa5ac00ce40 comp rx=0 tx=0).stop 2023-10-16T11:20:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 shutdown_connections 2023-10-16T11:20:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b415dbd0 0x7fa5b415ff10 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b40ce2c0 0x7fa5b4158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.181:6832/1878395836,v1:172.21.15.181:6833/1878395836] conn(0x7fa5a006b140 0x7fa5a006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 --2- 172.21.15.181:0/267569927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5b40d0610 0x7fa5b40b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 >> 172.21.15.181:0/267569927 conn(0x7fa5b409a930 msgr2=0x7fa5b415b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 shutdown_connections 2023-10-16T11:20:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.061+0000 7fa5b99b2700 1 -- 172.21.15.181:0/267569927 wait complete. 2023-10-16T11:20:59.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:59.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.195+0000 7f47fdf5f700 1 Processor -- start 2023-10-16T11:20:59.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.195+0000 7f47fdf5f700 1 -- start start 2023-10-16T11:20:59.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.195+0000 7f47fdf5f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47f815dfd0 0x7f47f80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.195+0000 7f47fdf5f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80af450 0x7f47f806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.195+0000 7f47fdf5f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f806f870 con 0x7f47f806db80 2023-10-16T11:20:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.195+0000 7f47fdf5f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80d1160 con 0x7f47f815dfd0 2023-10-16T11:20:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.195+0000 7f47fdf5f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80cc8e0 con 0x7f47f80af450 2023-10-16T11:20:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f7fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f47f806db80 0x7f47f80cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41956/0 (socket says 172.21.15.181:41956) 2023-10-16T11:20:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f7fff700 1 -- 172.21.15.181:0/2738150941 learned_addr learned my addr 172.21.15.181:0/2738150941 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f77fe700 1 --2- 172.21.15.181:0/2738150941 >> v2:172.21.15.190:3300/0 conn(0x7f47f815dfd0 0x7f47f80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f6ffd700 1 --2- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80af450 0x7f47f806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f67fc700 1 -- 172.21.15.181:0/2738150941 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350705080 0 0) 0x7f47f806f870 con 0x7f47f806db80 2023-10-16T11:20:59.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f67fc700 1 -- 172.21.15.181:0/2738150941 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e00035b0 con 0x7f47f806db80 2023-10-16T11:20:59.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f67fc700 1 -- 172.21.15.181:0/2738150941 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2760790028 0 0) 0x7f47e00035b0 con 0x7f47f806db80 2023-10-16T11:20:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f77fe700 1 -- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80af450 msgr2=0x7f47f806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f77fe700 1 --2- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80af450 0x7f47f806d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f77fe700 1 -- 172.21.15.181:0/2738150941 >> v1:172.21.15.181:6789/0 conn(0x7f47f806db80 legacy=0x7f47f80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f77fe700 1 -- 172.21.15.181:0/2738150941 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80ccbc0 con 0x7f47f815dfd0 2023-10-16T11:20:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f77fe700 1 --2- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f815dfd0 0x7f47f80af9a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009820 tx=0x7f47e8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5346a3cdb83f1f91 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f67fc700 1 -- 172.21.15.181:0/2738150941 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00019f0 con 0x7f47f815dfd0 2023-10-16T11:20:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f67fc700 1 -- 172.21.15.181:0/2738150941 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e8005c30 con 0x7f47f815dfd0 2023-10-16T11:20:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.196+0000 7f47f67fc700 1 -- 172.21.15.181:0/2738150941 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8004030 con 0x7f47f815dfd0 2023-10-16T11:20:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 -- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f815dfd0 msgr2=0x7f47f80af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 --2- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f815dfd0 0x7f47f80af9a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009820 tx=0x7f47e8009b30 comp rx=0 tx=0).stop 2023-10-16T11:20:59.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 -- 172.21.15.181:0/2738150941 shutdown_connections 2023-10-16T11:20:59.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 --2- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f815dfd0 0x7f47f80af9a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 --2- 172.21.15.181:0/2738150941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f80af450 0x7f47f806d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 -- 172.21.15.181:0/2738150941 >> 172.21.15.181:0/2738150941 conn(0x7f47f80d04a0 msgr2=0x7f47f80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 -- 172.21.15.181:0/2738150941 shutdown_connections 2023-10-16T11:20:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.197+0000 7f47fdf5f700 1 -- 172.21.15.181:0/2738150941 wait complete. 2023-10-16T11:20:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.198+0000 7f47fdf5f700 1 Processor -- start 2023-10-16T11:20:59.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.198+0000 7f47fdf5f700 1 -- start start 2023-10-16T11:20:59.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47fdf5f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47f806db80 0x7f47f80b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47fdf5f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f80af450 0x7f47f815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47fdf5f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f815dfd0 0x7f47f815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47fdf5f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80ca400 con 0x7f47f806db80 2023-10-16T11:20:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47fdf5f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f81472f0 con 0x7f47f80af450 2023-10-16T11:20:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47fdf5f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8147b90 con 0x7f47f815dfd0 2023-10-16T11:20:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47f77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47f806db80 0x7f47f80b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47f77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47f806db80 0x7f47f80b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59114/0 (socket says 172.21.15.181:59114) 2023-10-16T11:20:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47f77fe700 1 -- 172.21.15.181:0/3706819746 learned_addr learned my addr 172.21.15.181:0/3706819746 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47f6ffd700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f80af450 0x7f47f815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.199+0000 7f47f7fff700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f815dfd0 0x7f47f815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47f77fe700 1 -- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f815dfd0 msgr2=0x7f47f815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47f77fe700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f815dfd0 0x7f47f815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47f77fe700 1 -- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f80af450 msgr2=0x7f47f815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47f77fe700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f80af450 0x7f47f815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47f77fe700 1 -- 172.21.15.181:0/3706819746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e8009430 con 0x7f47f806db80 2023-10-16T11:20:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47f77fe700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47f806db80 0x7f47f80b56c0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009770 tx=0x7f47e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47dffff700 1 -- 172.21.15.181:0/3706819746 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8003ab0 con 0x7f47f806db80 2023-10-16T11:20:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f80c3fd0 con 0x7f47f806db80 2023-10-16T11:20:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47dffff700 1 -- 172.21.15.181:0/3706819746 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e8003c30 con 0x7f47f806db80 2023-10-16T11:20:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47dffff700 1 -- 172.21.15.181:0/3706819746 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e801d430 con 0x7f47f806db80 2023-10-16T11:20:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.200+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f8147540 con 0x7f47f806db80 2023-10-16T11:20:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.201+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f8067d20 con 0x7f47f806db80 2023-10-16T11:20:59.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.201+0000 7f47dffff700 1 -- 172.21.15.181:0/3706819746 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 156) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f47e801d590 con 0x7f47f806db80 2023-10-16T11:20:59.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.202+0000 7f47dffff700 1 -- 172.21.15.181:0/3706819746 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7985+0+0 (secure 0 0 0) 0x7f47e808aa40 con 0x7f47f806db80 2023-10-16T11:20:59.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.206+0000 7f47dffff700 1 -- 172.21.15.181:0/3706819746 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f47e80556f0 con 0x7f47f806db80 2023-10-16T11:20:59.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.440+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47f80ad870 con 0x7f47f806db80 2023-10-16T11:20:59.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.440+0000 7f47dffff700 1 -- 172.21.15.181:0/3706819746 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+6164 (secure 0 0 0) 0x7f47e802b020 con 0x7f47f806db80 2023-10-16T11:20:59.443 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:20:59.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47f806db80 msgr2=0x7f47f80b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47f806db80 0x7f47f80b56c0 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009770 tx=0x7f47e8004750 comp rx=0 tx=0).stop 2023-10-16T11:20:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 shutdown_connections 2023-10-16T11:20:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47f815dfd0 0x7f47f815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47f806db80 0x7f47f80b56c0 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 --2- 172.21.15.181:0/3706819746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47f80af450 0x7f47f815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 >> 172.21.15.181:0/3706819746 conn(0x7f47f80d04a0 msgr2=0x7f47f80adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:59.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.442+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 shutdown_connections 2023-10-16T11:20:59.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.443+0000 7f47fdf5f700 1 -- 172.21.15.181:0/3706819746 wait complete. 2023-10-16T11:20:59.447 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 133 2023-10-16T11:20:59.457 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:20:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.576+0000 7f227fc1d700 1 Processor -- start 2023-10-16T11:20:59.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.576+0000 7f227fc1d700 1 -- start start 2023-10-16T11:20:59.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227fc1d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f227815db70 0x7f22780665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227fc1d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780ca2e0 0x7f22780ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227fc1d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2278066c60 con 0x7f22780cedd0 2023-10-16T11:20:59.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227fc1d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22780cda10 con 0x7f227815db70 2023-10-16T11:20:59.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227fc1d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227806c1a0 con 0x7f22780ca2e0 2023-10-16T11:20:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f227815db70 0x7f22780665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f227815db70 0x7f22780665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59118/0 (socket says 172.21.15.181:59118) 2023-10-16T11:20:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 -- 172.21.15.181:0/460464146 learned_addr learned my addr 172.21.15.181:0/460464146 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227e1ba700 1 --2- 172.21.15.181:0/460464146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780ca2e0 0x7f22780ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 -- 172.21.15.181:0/460464146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780ca2e0 msgr2=0x7f22780ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 --2- 172.21.15.181:0/460464146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780ca2e0 0x7f22780ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 -- 172.21.15.181:0/460464146 >> v1:172.21.15.190:6789/0 conn(0x7f22780cedd0 legacy=0x7f22780cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 -- 172.21.15.181:0/460464146 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227806b3e0 con 0x7f227815db70 2023-10-16T11:20:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227d9b9700 1 --2- 172.21.15.181:0/460464146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780665a0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f22740098f0 tx=0x7f2274009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311d3142a2a5fa66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:59.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.577+0000 7f227c9b7700 1 -- 172.21.15.181:0/460464146 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274004230 con 0x7f227815db70 2023-10-16T11:20:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227c9b7700 1 -- 172.21.15.181:0/460464146 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274004390 con 0x7f227815db70 2023-10-16T11:20:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227c9b7700 1 -- 172.21.15.181:0/460464146 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740036a0 con 0x7f227815db70 2023-10-16T11:20:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 -- 172.21.15.181:0/460464146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 msgr2=0x7f22780665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 --2- 172.21.15.181:0/460464146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780665a0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f22740098f0 tx=0x7f2274009c00 comp rx=0 tx=0).stop 2023-10-16T11:20:59.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 -- 172.21.15.181:0/460464146 shutdown_connections 2023-10-16T11:20:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 --2- 172.21.15.181:0/460464146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780ca2e0 0x7f22780ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 --2- 172.21.15.181:0/460464146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780665a0 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 -- 172.21.15.181:0/460464146 >> 172.21.15.181:0/460464146 conn(0x7f22780b1780 msgr2=0x7f22780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 -- 172.21.15.181:0/460464146 shutdown_connections 2023-10-16T11:20:59.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.578+0000 7f227fc1d700 1 -- 172.21.15.181:0/460464146 wait complete. 2023-10-16T11:20:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.579+0000 7f227fc1d700 1 Processor -- start 2023-10-16T11:20:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.579+0000 7f227fc1d700 1 -- start start 2023-10-16T11:20:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227fc1d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22780ca2e0 0x7f227815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227fc1d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780cedd0 0x7f22780c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227fc1d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227fc1d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22780c85a0 con 0x7f227815db70 2023-10-16T11:20:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227fc1d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227815b1a0 con 0x7f22780ca2e0 2023-10-16T11:20:59.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227fc1d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22780c7920 con 0x7f22780cedd0 2023-10-16T11:20:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59126/0 (socket says 172.21.15.181:59126) 2023-10-16T11:20:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 -- 172.21.15.181:0/1759038886 learned_addr learned my addr 172.21.15.181:0/1759038886 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227d9b9700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22780ca2e0 0x7f227815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227d1b8700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780cedd0 0x7f22780c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 -- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780cedd0 msgr2=0x7f22780c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780cedd0 0x7f22780c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 -- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22780ca2e0 msgr2=0x7f227815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22780ca2e0 0x7f227815ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227e1ba700 1 -- 172.21.15.181:0/1759038886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274009550 con 0x7f227815db70 2023-10-16T11:20:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.580+0000 7f227d1b8700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780cedd0 0x7f22780c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:20:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.581+0000 7f227e1ba700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780c72a0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f2270007e40 tx=0x7f227000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.581+0000 7f226e7fc700 1 -- 172.21.15.181:0/1759038886 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227000ed80 con 0x7f227815db70 2023-10-16T11:20:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.581+0000 7f226e7fc700 1 -- 172.21.15.181:0/1759038886 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2270014420 con 0x7f227815db70 2023-10-16T11:20:59.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.581+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22780c7c00 con 0x7f227815db70 2023-10-16T11:20:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.581+0000 7f226e7fc700 1 -- 172.21.15.181:0/1759038886 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700147d0 con 0x7f227815db70 2023-10-16T11:20:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.581+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22780c81c0 con 0x7f227815db70 2023-10-16T11:20:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.582+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22780c86e0 con 0x7f227815db70 2023-10-16T11:20:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.582+0000 7f226e7fc700 1 -- 172.21.15.181:0/1759038886 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 156) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f2270014930 con 0x7f227815db70 2023-10-16T11:20:59.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.583+0000 7f226e7fc700 1 -- 172.21.15.181:0/1759038886 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7985+0+0 (secure 0 0 0) 0x7f227008a040 con 0x7f227815db70 2023-10-16T11:20:59.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.587+0000 7f226e7fc700 1 -- 172.21.15.181:0/1759038886 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f22780c86e0 con 0x7f227815db70 2023-10-16T11:20:59.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.779+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22780c86e0 con 0x7f227815db70 2023-10-16T11:20:59.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.780+0000 7f226e7fc700 1 -- 172.21.15.181:0/1759038886 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+22345 (secure 0 0 0) 0x7f22780c86e0 con 0x7f227815db70 2023-10-16T11:20:59.783 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:20:59.783 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":242,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:58.912345+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/4174526666":"2023-10-17T11:20:58.912264+0000","172.21.15.181:6832/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2905113763":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1596741027":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3516831361":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2396727780":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2869414246":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3445652558":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3765547571":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:20:59.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 msgr2=0x7f22780c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780c72a0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f2270007e40 tx=0x7f227000e3c0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 shutdown_connections 2023-10-16T11:20:59.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227815db70 0x7f22780c72a0 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22780ca2e0 0x7f227815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 --2- 172.21.15.181:0/1759038886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f22780cedd0 0x7f22780c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 >> 172.21.15.181:0/1759038886 conn(0x7f22780b1780 msgr2=0x7f22780c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:59.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 shutdown_connections 2023-10-16T11:20:59.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.783+0000 7f227fc1d700 1 -- 172.21.15.181:0/1759038886 wait complete. 2023-10-16T11:20:59.798 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:20:59.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:20:59.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.917+0000 7fc062761700 1 Processor -- start 2023-10-16T11:20:59.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.917+0000 7fc062761700 1 -- start start 2023-10-16T11:20:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.917+0000 7fc062761700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 0x7fc05c06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.917+0000 7fc062761700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc05c0d0350 0x7fc05c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.917+0000 7fc062761700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05c0cc470 con 0x7fc05c0d2390 2023-10-16T11:20:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.917+0000 7fc062761700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05c06ca30 con 0x7fc05c0d0350 2023-10-16T11:20:59.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.917+0000 7fc062761700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c06cd10 con 0x7fc05c0ce7f0 2023-10-16T11:20:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc05c0d2390 0x7fc05c0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41978/0 (socket says 172.21.15.181:41978) 2023-10-16T11:20:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05b7fe700 1 -- 172.21.15.181:0/1964072034 learned_addr learned my addr 172.21.15.181:0/1964072034 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:20:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05bfff700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 0x7fc05c06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc060cfe700 1 --2- 172.21.15.181:0/1964072034 >> v2:172.21.15.190:3300/0 conn(0x7fc05c0d0350 0x7fc05c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803848266 0 0) 0x7fc05c0cc470 con 0x7fc05c0d2390 2023-10-16T11:20:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0440035b0 con 0x7fc05c0d2390 2023-10-16T11:20:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454875700 0 0) 0x7fc0440035b0 con 0x7fc05c0d2390 2023-10-16T11:20:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 msgr2=0x7fc05c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:59.925 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 0x7fc05c06c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 >> v2:172.21.15.190:3300/0 conn(0x7fc05c0d0350 msgr2=0x7fc05c0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 --2- 172.21.15.181:0/1964072034 >> v2:172.21.15.190:3300/0 conn(0x7fc05c0d0350 0x7fc05c0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c06cff0 con 0x7fc05c0d2390 2023-10-16T11:20:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fc050001670 con 0x7fc05c0d2390 2023-10-16T11:20:59.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 >> v1:172.21.15.181:6789/0 conn(0x7fc05c0d2390 legacy=0x7fc05c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc044003a00 0x7fc044003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 0x7fc044004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc044004d90 0x7fc044005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c06ca30 con 0x7fc044004d90 2023-10-16T11:20:59.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c06cd10 con 0x7fc044003a00 2023-10-16T11:20:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04400ba10 con 0x7fc05c0ce7f0 2023-10-16T11:20:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc060cfe700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc044004d90 0x7fc044005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05b7fe700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 0x7fc044004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.918+0000 7fc05bfff700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc044003a00 0x7fc044003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc060cfe700 1 -- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 msgr2=0x7fc044004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:59.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc060cfe700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c0ce7f0 0x7fc044004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc060cfe700 1 -- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc044003a00 msgr2=0x7fc044003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:20:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc060cfe700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc044003a00 0x7fc044003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc060cfe700 1 -- 172.21.15.181:0/1964072034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440035b0 con 0x7fc044004d90 2023-10-16T11:20:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc060cfe700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc044004d90 0x7fc044005210 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fc054009f90 tx=0x7fc044005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f9b786211c5fd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:59.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400ec30 con 0x7fc044004d90 2023-10-16T11:20:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05400f040 con 0x7fc044004d90 2023-10-16T11:20:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.919+0000 7fc05affd700 1 -- 172.21.15.181:0/1964072034 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054014480 con 0x7fc044004d90 2023-10-16T11:20:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.920+0000 7fc062761700 1 -- 172.21.15.181:0/1964072034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc044004d90 msgr2=0x7fc044005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.920+0000 7fc062761700 1 --2- 172.21.15.181:0/1964072034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc044004d90 0x7fc044005210 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fc054009f90 tx=0x7fc044005880 comp rx=0 tx=0).stop 2023-10-16T11:20:59.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.920+0000 7fc060cfe700 1 -- 172.21.15.181:0/1964072034 reap_dead start 2023-10-16T11:20:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.920+0000 7fc062761700 1 -- 172.21.15.181:0/1964072034 shutdown_connections 2023-10-16T11:20:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.920+0000 7fc062761700 1 -- 172.21.15.181:0/1964072034 >> 172.21.15.181:0/1964072034 conn(0x7fc05c09a9a0 msgr2=0x7fc05c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:20:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.920+0000 7fc062761700 1 -- 172.21.15.181:0/1964072034 shutdown_connections 2023-10-16T11:20:59.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.920+0000 7fc062761700 1 -- 172.21.15.181:0/1964072034 wait complete. 2023-10-16T11:20:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.921+0000 7fc062761700 1 Processor -- start 2023-10-16T11:20:59.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.922+0000 7fc062761700 1 -- start start 2023-10-16T11:20:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.923+0000 7fc062761700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc05c15f650 0x7fc05c152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.923+0000 7fc062761700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc05c15fa30 0x7fc05c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.923+0000 7fc062761700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c09e5f0 0x7fc05c09ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.923+0000 7fc062761700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0b4bb0 con 0x7fc05c15fa30 2023-10-16T11:20:59.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.923+0000 7fc062761700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c09f320 con 0x7fc05c15f650 2023-10-16T11:20:59.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.923+0000 7fc062761700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0b4280 con 0x7fc05c09e5f0 2023-10-16T11:20:59.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc05c15f650 0x7fc05c152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc05c15f650 0x7fc05c152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:35966/0 (socket says 172.21.15.181:35966) 2023-10-16T11:20:59.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 -- 172.21.15.181:0/3240061133 learned_addr learned my addr 172.21.15.181:0/3240061133 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:20:59.935 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05b7fe700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc05c15fa30 0x7fc05c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 -- 172.21.15.181:0/3240061133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c09e5f0 msgr2=0x7fc05c09ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c09e5f0 0x7fc05c09ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 -- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc05c15fa30 msgr2=0x7fc05c152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:20:59.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc05c15fa30 0x7fc05c152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:20:59.936 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.924+0000 7fc05bfff700 1 -- 172.21.15.181:0/3240061133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c0b4560 con 0x7fc05c15f650 2023-10-16T11:20:59.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.925+0000 7fc05bfff700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc05c15f650 0x7fc05c152700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009fd0 tx=0x7fc04c011c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:59.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.927+0000 7fc058ff9700 1 -- 172.21.15.181:0/3240061133 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c011ec0 con 0x7fc05c15f650 2023-10-16T11:20:59.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.927+0000 7fc058ff9700 1 -- 172.21.15.181:0/3240061133 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04c005c30 con 0x7fc05c15f650 2023-10-16T11:20:59.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.927+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c0b4840 con 0x7fc05c15f650 2023-10-16T11:20:59.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.927+0000 7fc058ff9700 1 -- 172.21.15.181:0/3240061133 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c00f730 con 0x7fc05c15f650 2023-10-16T11:20:59.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.927+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c1f6690 con 0x7fc05c15f650 2023-10-16T11:20:59.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.929+0000 7fc058ff9700 1 -- 172.21.15.181:0/3240061133 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 157) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc04c00f890 con 0x7fc05c15f650 2023-10-16T11:20:59.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.930+0000 7fc058ff9700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc04806ad50 0x7fc04806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:20:59.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.930+0000 7fc058ff9700 1 -- 172.21.15.181:0/3240061133 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7985+0+0 (secure 0 0 0) 0x7fc04c08d170 con 0x7fc05c15f650 2023-10-16T11:20:59.938 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.930+0000 7fc05b7fe700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc04806ad50 0x7fc04806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:20:59.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.931+0000 7fc05b7fe700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc04806ad50 0x7fc04806d210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0500048d0 tx=0x7fc050007980 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:20:59.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.931+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c067d20 con 0x7fc05c15f650 2023-10-16T11:20:59.939 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:20:59.937+0000 7fc058ff9700 1 -- 172.21.15.181:0/3240061133 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc04c08e060 con 0x7fc05c15f650 2023-10-16T11:21:00.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.172+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc05c15e150 con 0x7fc05c15f650 2023-10-16T11:21:00.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.173+0000 7fc058ff9700 1 -- 172.21.15.181:0/3240061133 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+6164 (secure 0 0 0) 0x7fc04c054200 con 0x7fc05c15f650 2023-10-16T11:21:00.175 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:00.175 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:21:00.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc04806ad50 msgr2=0x7fc04806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc04806ad50 0x7fc04806d210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0500048d0 tx=0x7fc050007980 comp rx=0 tx=0).stop 2023-10-16T11:21:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc05c15f650 msgr2=0x7fc05c152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc05c15f650 0x7fc05c152700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009fd0 tx=0x7fc04c011c80 comp rx=0 tx=0).stop 2023-10-16T11:21:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 shutdown_connections 2023-10-16T11:21:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc05c09e5f0 0x7fc05c09ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc04806ad50 0x7fc04806d210 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc05c15f650 0x7fc05c152700 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 --2- 172.21.15.181:0/3240061133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc05c15fa30 0x7fc05c152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 >> 172.21.15.181:0/3240061133 conn(0x7fc05c09a9a0 msgr2=0x7fc05c06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 shutdown_connections 2023-10-16T11:21:00.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.175+0000 7fc062761700 1 -- 172.21.15.181:0/3240061133 wait complete. 2023-10-16T11:21:00.180 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 133 2023-10-16T11:21:00.189 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:00.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.307+0000 7f7c2e84d700 1 Processor -- start 2023-10-16T11:21:00.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.307+0000 7f7c2e84d700 1 -- start start 2023-10-16T11:21:00.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.308+0000 7f7c2e84d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 0x7f7c280d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.308+0000 7f7c2e84d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c280cb0c0 0x7f7c2815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.308+0000 7f7c2e84d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2806ed00 con 0x7f7c2815e6d0 2023-10-16T11:21:00.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.308+0000 7f7c2e84d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c280d02d0 con 0x7f7c280cb0c0 2023-10-16T11:21:00.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.308+0000 7f7c2e84d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2815ad40 con 0x7f7c2815eab0 2023-10-16T11:21:00.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.308+0000 7f7c27fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7c2815e6d0 0x7f7c2815a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42018/0 (socket says 172.21.15.181:42018) 2023-10-16T11:21:00.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.308+0000 7f7c27fff700 1 -- 172.21.15.181:0/2688630715 learned_addr learned my addr 172.21.15.181:0/2688630715 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:00.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719395283 0 0) 0x7f7c2806ed00 con 0x7f7c2815e6d0 2023-10-16T11:21:00.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c277fe700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 0x7f7c280d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c140035b0 con 0x7f7c2815e6d0 2023-10-16T11:21:00.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c2cdea700 1 --2- 172.21.15.181:0/2688630715 >> v2:172.21.15.190:3300/0 conn(0x7f7c280cb0c0 0x7f7c2815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413313844 0 0) 0x7f7c140035b0 con 0x7f7c2815e6d0 2023-10-16T11:21:00.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 msgr2=0x7f7c280d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:00.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 0x7f7c280d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 >> v2:172.21.15.190:3300/0 conn(0x7f7c280cb0c0 msgr2=0x7f7c2815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:00.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 --2- 172.21.15.181:0/2688630715 >> v2:172.21.15.190:3300/0 conn(0x7f7c280cb0c0 0x7f7c2815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2806e0f0 con 0x7f7c2815e6d0 2023-10-16T11:21:00.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7c100017f0 con 0x7f7c2815e6d0 2023-10-16T11:21:00.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 >> v1:172.21.15.181:6789/0 conn(0x7f7c2815e6d0 legacy=0x7f7c2815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 0x7f7c14003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c14003f80 0x7f7c14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c14004d90 0x7f7c14005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280d02d0 con 0x7f7c14003f80 2023-10-16T11:21:00.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2815ad40 con 0x7f7c14004d90 2023-10-16T11:21:00.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1400ba10 con 0x7f7c2815eab0 2023-10-16T11:21:00.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c277fe700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c14003f80 0x7f7c14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c27fff700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 0x7f7c14003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c2cdea700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c14004d90 0x7f7c14005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c277fe700 1 -- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 msgr2=0x7f7c14003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c277fe700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c2815eab0 0x7f7c14003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c277fe700 1 -- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c14004d90 msgr2=0x7f7c14005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:00.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c277fe700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c14004d90 0x7f7c14005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.309+0000 7f7c277fe700 1 -- 172.21.15.181:0/2688630715 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c140035b0 con 0x7f7c14003f80 2023-10-16T11:21:00.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c277fe700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c14003f80 0x7f7c14004850 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009b60 tx=0x7f7c14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f076e11a58ddc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:00.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800e660 con 0x7f7c14003f80 2023-10-16T11:21:00.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1800e7c0 con 0x7f7c14003f80 2023-10-16T11:21:00.320 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c26ffd700 1 -- 172.21.15.181:0/2688630715 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800eb30 con 0x7f7c14003f80 2023-10-16T11:21:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2688630715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c14003f80 msgr2=0x7f7c14004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c2e84d700 1 --2- 172.21.15.181:0/2688630715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c14003f80 0x7f7c14004850 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009b60 tx=0x7f7c14005880 comp rx=0 tx=0).stop 2023-10-16T11:21:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c2cdea700 1 -- 172.21.15.181:0/2688630715 reap_dead start 2023-10-16T11:21:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2688630715 shutdown_connections 2023-10-16T11:21:00.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2688630715 >> 172.21.15.181:0/2688630715 conn(0x7f7c280d04a0 msgr2=0x7f7c280d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:00.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2688630715 shutdown_connections 2023-10-16T11:21:00.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.310+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2688630715 wait complete. 2023-10-16T11:21:00.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.312+0000 7f7c2e84d700 1 Processor -- start 2023-10-16T11:21:00.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.312+0000 7f7c2e84d700 1 -- start start 2023-10-16T11:21:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.312+0000 7f7c2e84d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c14003f80 0x7f7c280b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.312+0000 7f7c2e84d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2814fcc0 0x7f7c28158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.312+0000 7f7c2e84d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c28158bd0 0x7f7c280b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2e84d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280b4bc0 con 0x7f7c28158bd0 2023-10-16T11:21:00.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2e84d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28159060 con 0x7f7c2814fcc0 2023-10-16T11:21:00.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2e84d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2815b280 con 0x7f7c14003f80 2023-10-16T11:21:00.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c28158bd0 0x7f7c280b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c28158bd0 0x7f7c280b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59188/0 (socket says 172.21.15.181:59188) 2023-10-16T11:21:00.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 -- 172.21.15.181:0/2490709326 learned_addr learned my addr 172.21.15.181:0/2490709326 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:00.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c277fe700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2814fcc0 0x7f7c28158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c27fff700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c14003f80 0x7f7c280b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 -- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c14003f80 msgr2=0x7f7c280b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c14003f80 0x7f7c280b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 -- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2814fcc0 msgr2=0x7f7c28158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2814fcc0 0x7f7c28158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 -- 172.21.15.181:0/2490709326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280b3f40 con 0x7f7c28158bd0 2023-10-16T11:21:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c27fff700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c14003f80 0x7f7c280b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2cdea700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c28158bd0 0x7f7c280b32f0 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f7c2815ae80 tx=0x7f7c20011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:00.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c24ff9700 1 -- 172.21.15.181:0/2490709326 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000e040 con 0x7f7c28158bd0 2023-10-16T11:21:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c24ff9700 1 -- 172.21.15.181:0/2490709326 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2000f040 con 0x7f7c28158bd0 2023-10-16T11:21:00.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c280b4220 con 0x7f7c28158bd0 2023-10-16T11:21:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c24ff9700 1 -- 172.21.15.181:0/2490709326 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000fca0 con 0x7f7c28158bd0 2023-10-16T11:21:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.313+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c280b47e0 con 0x7f7c28158bd0 2023-10-16T11:21:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.314+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c280b49a0 con 0x7f7c28158bd0 2023-10-16T11:21:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.315+0000 7f7c24ff9700 1 -- 172.21.15.181:0/2490709326 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 157) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7c20022460 con 0x7f7c28158bd0 2023-10-16T11:21:00.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.315+0000 7f7c24ff9700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7c0806ad50 0x7f7c0806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.315+0000 7f7c24ff9700 1 -- 172.21.15.181:0/2490709326 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7985+0+0 (secure 0 0 0) 0x7f7c20089f40 con 0x7f7c28158bd0 2023-10-16T11:21:00.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.315+0000 7f7c27fff700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7c0806ad50 0x7f7c0806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.316+0000 7f7c27fff700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7c0806ad50 0x7f7c0806d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7c10000c00 tx=0x7f7c10009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:00.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.320+0000 7f7c24ff9700 1 -- 172.21.15.181:0/2490709326 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7c20054a40 con 0x7f7c28158bd0 2023-10-16T11:21:00.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.555+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c280b4d00 con 0x7f7c28158bd0 2023-10-16T11:21:00.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.555+0000 7f7c24ff9700 1 -- 172.21.15.181:0/2490709326 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+6164 (secure 0 0 0) 0x7f7c280b4d00 con 0x7f7c28158bd0 2023-10-16T11:21:00.558 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:00.558 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":133,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:25.468288+0000","modified":"2023-10-16T11:20:28.484587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12171":{"gid":12171,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/3483656577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3483656577},{"type":"v1","addr":"172.21.15.181:6836","nonce":3483656577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:21:00.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7c0806ad50 msgr2=0x7f7c0806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7c0806ad50 0x7f7c0806d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7c10000c00 tx=0x7f7c10009040 comp rx=0 tx=0).stop 2023-10-16T11:21:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c28158bd0 msgr2=0x7f7c280b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c28158bd0 0x7f7c280b32f0 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f7c2815ae80 tx=0x7f7c20011ed0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 shutdown_connections 2023-10-16T11:21:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c14003f80 0x7f7c280b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7c0806ad50 0x7f7c0806d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c28158bd0 0x7f7c280b32f0 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 --2- 172.21.15.181:0/2490709326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c2814fcc0 0x7f7c28158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.557+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 >> 172.21.15.181:0/2490709326 conn(0x7f7c280d04a0 msgr2=0x7f7c280cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.558+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 shutdown_connections 2023-10-16T11:21:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.558+0000 7f7c2e84d700 1 -- 172.21.15.181:0/2490709326 wait complete. 2023-10-16T11:21:00.562 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 133 2023-10-16T11:21:00.572 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:00.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.692+0000 7fbc7e7f4700 1 Processor -- start 2023-10-16T11:21:00.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.692+0000 7fbc7e7f4700 1 -- start start 2023-10-16T11:21:00.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.692+0000 7fbc7e7f4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc7815a7b0 0x7fbc780cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.692+0000 7fbc7e7f4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc780c9a50 0x7fbc780c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc7e7f4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc780ca880 con 0x7fbc780ca410 2023-10-16T11:21:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc7e7f4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc780cf2b0 con 0x7fbc780c9a50 2023-10-16T11:21:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc7e7f4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780b5300 con 0x7fbc7815a7b0 2023-10-16T11:21:00.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc780c9a50 0x7fbc780c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc780c9a50 0x7fbc780c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59194/0 (socket says 172.21.15.181:59194) 2023-10-16T11:21:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc7cd91700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbc780ca410 0x7fbc780ccfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42042/0 (socket says 172.21.15.181:42042) 2023-10-16T11:21:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 -- 172.21.15.181:0/2340721013 learned_addr learned my addr 172.21.15.181:0/2340721013 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc77fff700 1 --2- 172.21.15.181:0/2340721013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc7815a7b0 0x7fbc780cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 -- 172.21.15.181:0/2340721013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc7815a7b0 msgr2=0x7fbc780cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 --2- 172.21.15.181:0/2340721013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc7815a7b0 0x7fbc780cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 -- 172.21.15.181:0/2340721013 >> v1:172.21.15.181:6789/0 conn(0x7fbc780ca410 legacy=0x7fbc780ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 -- 172.21.15.181:0/2340721013 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7815e390 con 0x7fbc780c9a50 2023-10-16T11:21:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc777fe700 1 --2- 172.21.15.181:0/2340721013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc780c9ed0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0098f0 tx=0x7fbc6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd2926833cda441 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:00.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc76ffd700 1 -- 172.21.15.181:0/2340721013 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00b8a0 con 0x7fbc780c9a50 2023-10-16T11:21:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc76ffd700 1 -- 172.21.15.181:0/2340721013 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c00ba00 con 0x7fbc780c9a50 2023-10-16T11:21:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.693+0000 7fbc76ffd700 1 -- 172.21.15.181:0/2340721013 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00bd90 con 0x7fbc780c9a50 2023-10-16T11:21:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/2340721013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 msgr2=0x7fbc780c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/2340721013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc780c9ed0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0098f0 tx=0x7fbc6c009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:00.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/2340721013 shutdown_connections 2023-10-16T11:21:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/2340721013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc7815a7b0 0x7fbc780cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/2340721013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc780c9ed0 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/2340721013 >> 172.21.15.181:0/2340721013 conn(0x7fbc7806aed0 msgr2=0x7fbc780b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/2340721013 shutdown_connections 2023-10-16T11:21:00.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.694+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/2340721013 wait complete. 2023-10-16T11:21:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.695+0000 7fbc7e7f4700 1 Processor -- start 2023-10-16T11:21:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.695+0000 7fbc7e7f4700 1 -- start start 2023-10-16T11:21:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc7e7f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc781f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc7e7f4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc780ca410 0x7fbc780c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc7e7f4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc7815a7b0 0x7fbc780c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc7e7f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780c7550 con 0x7fbc780c9a50 2023-10-16T11:21:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc7e7f4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780c82c0 con 0x7fbc7815a7b0 2023-10-16T11:21:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc7e7f4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780c6f90 con 0x7fbc780ca410 2023-10-16T11:21:00.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc781f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc781f0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59198/0 (socket says 172.21.15.181:59198) 2023-10-16T11:21:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 -- 172.21.15.181:0/1948438338 learned_addr learned my addr 172.21.15.181:0/1948438338 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 -- 172.21.15.181:0/1948438338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc780ca410 msgr2=0x7fbc780c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc780ca410 0x7fbc780c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 -- 172.21.15.181:0/1948438338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc7815a7b0 msgr2=0x7fbc780c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc7815a7b0 0x7fbc780c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 -- 172.21.15.181:0/1948438338 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009550 con 0x7fbc780c9a50 2023-10-16T11:21:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.696+0000 7fbc77fff700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc781f0a30 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d800 tx=0x7fbc6800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.697+0000 7fbc74ff9700 1 -- 172.21.15.181:0/1948438338 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680125f0 con 0x7fbc780c9a50 2023-10-16T11:21:00.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.697+0000 7fbc74ff9700 1 -- 172.21.15.181:0/1948438338 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68012750 con 0x7fbc780c9a50 2023-10-16T11:21:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.697+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc780c7270 con 0x7fbc780c9a50 2023-10-16T11:21:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.697+0000 7fbc74ff9700 1 -- 172.21.15.181:0/1948438338 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68012b00 con 0x7fbc780c9a50 2023-10-16T11:21:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.697+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc780c91f0 con 0x7fbc780c9a50 2023-10-16T11:21:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.698+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc780c7db0 con 0x7fbc780c9a50 2023-10-16T11:21:00.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.698+0000 7fbc74ff9700 1 -- 172.21.15.181:0/1948438338 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 157) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbc68012c60 con 0x7fbc780c9a50 2023-10-16T11:21:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.698+0000 7fbc74ff9700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbc6406ad50 0x7fbc6406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.699+0000 7fbc74ff9700 1 -- 172.21.15.181:0/1948438338 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7985+0+0 (secure 0 0 0) 0x7fbc68089e90 con 0x7fbc780c9a50 2023-10-16T11:21:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.699+0000 7fbc777fe700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbc6406ad50 0x7fbc6406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:00.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.699+0000 7fbc777fe700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbc6406ad50 0x7fbc6406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc78159270 tx=0x7fbc6c010000 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:00.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.703+0000 7fbc74ff9700 1 -- 172.21.15.181:0/1948438338 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbc780c7db0 con 0x7fbc780c9a50 2023-10-16T11:21:00.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.896+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc780c7db0 con 0x7fbc780c9a50 2023-10-16T11:21:00.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.897+0000 7fbc74ff9700 1 -- 172.21.15.181:0/1948438338 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+22345 (secure 0 0 0) 0x7fbc68076760 con 0x7fbc780c9a50 2023-10-16T11:21:00.900 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:00.900 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":242,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:20:58.912345+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:20:23.899617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_data","create_time":"2023-10-16T11:20:24.428675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/4174526666":"2023-10-17T11:20:58.912264+0000","172.21.15.181:6832/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2905113763":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1596741027":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3516831361":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2396727780":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2869414246":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3445652558":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3765547571":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:00.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbc6406ad50 msgr2=0x7fbc6406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbc6406ad50 0x7fbc6406d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc78159270 tx=0x7fbc6c010000 comp rx=0 tx=0).stop 2023-10-16T11:21:00.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 msgr2=0x7fbc781f0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:00.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc781f0a30 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d800 tx=0x7fbc6800dbc0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 shutdown_connections 2023-10-16T11:21:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbc6406ad50 0x7fbc6406d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc7815a7b0 0x7fbc780c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc780c9a50 0x7fbc781f0a30 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 --2- 172.21.15.181:0/1948438338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc780ca410 0x7fbc780c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 >> 172.21.15.181:0/1948438338 conn(0x7fbc7806aed0 msgr2=0x7fbc780af750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 shutdown_connections 2023-10-16T11:21:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:00.900+0000 7fbc7e7f4700 1 -- 172.21.15.181:0/1948438338 wait complete. 2023-10-16T11:21:00.915 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:21:01.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.041+0000 7f6f9b0a6700 1 Processor -- start 2023-10-16T11:21:01.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.041+0000 7f6f9b0a6700 1 -- start start 2023-10-16T11:21:01.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f9b0a6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f9415a890 0x7f6f940ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f9b0a6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f9406cb40 0x7f6f9406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f9b0a6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9415f350 con 0x7f6f940cd370 2023-10-16T11:21:01.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f9b0a6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9406d510 con 0x7f6f9406cb40 2023-10-16T11:21:01.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f9b0a6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940cdae0 con 0x7f6f9415a890 2023-10-16T11:21:01.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f99643700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f9406cb40 0x7f6f9406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f99643700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f9406cb40 0x7f6f9406cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59228/0 (socket says 172.21.15.181:59228) 2023-10-16T11:21:01.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f99643700 1 -- 172.21.15.181:0/3165810275 learned_addr learned my addr 172.21.15.181:0/3165810275 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:01.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.042+0000 7f6f98e42700 1 --2- 172.21.15.181:0/3165810275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f9415a890 0x7f6f940ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f99643700 1 -- 172.21.15.181:0/3165810275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f9415a890 msgr2=0x7f6f940ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:01.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f99643700 1 --2- 172.21.15.181:0/3165810275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f9415a890 0x7f6f940ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f99643700 1 -- 172.21.15.181:0/3165810275 >> v1:172.21.15.190:6789/0 conn(0x7f6f940cd370 legacy=0x7f6f9415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f99643700 1 -- 172.21.15.181:0/3165810275 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f940cddc0 con 0x7f6f9406cb40 2023-10-16T11:21:01.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f98e42700 1 --2- 172.21.15.181:0/3165810275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f9415a890 0x7f6f940ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:01.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f99643700 1 --2- 172.21.15.181:0/3165810275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f9406cfd0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00d150 tx=0x7f6f8c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10d36ad90cfed2a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:01.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f937fe700 1 -- 172.21.15.181:0/3165810275 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c010070 con 0x7f6f9406cb40 2023-10-16T11:21:01.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f937fe700 1 -- 172.21.15.181:0/3165810275 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c00ddc0 con 0x7f6f9406cb40 2023-10-16T11:21:01.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f937fe700 1 -- 172.21.15.181:0/3165810275 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00b310 con 0x7f6f9406cb40 2023-10-16T11:21:01.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3165810275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 msgr2=0x7f6f9406cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3165810275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f9406cfd0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00d150 tx=0x7f6f8c00d460 comp rx=0 tx=0).stop 2023-10-16T11:21:01.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3165810275 shutdown_connections 2023-10-16T11:21:01.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3165810275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f9406cfd0 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3165810275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f9415a890 0x7f6f940ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.043+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3165810275 >> 172.21.15.181:0/3165810275 conn(0x7f6f9406aed0 msgr2=0x7f6f940cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:01.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.044+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3165810275 shutdown_connections 2023-10-16T11:21:01.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.044+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3165810275 wait complete. 2023-10-16T11:21:01.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.045+0000 7f6f9b0a6700 1 Processor -- start 2023-10-16T11:21:01.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.045+0000 7f6f9b0a6700 1 -- start start 2023-10-16T11:21:01.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f940c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f940cd370 0x7f6f94071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9415a890 0x7f6f94072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940c5610 con 0x7f6f9406cb40 2023-10-16T11:21:01.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940a9a40 con 0x7f6f9415a890 2023-10-16T11:21:01.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f940a91a0 con 0x7f6f940cd370 2023-10-16T11:21:01.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f940c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f940c50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59230/0 (socket says 172.21.15.181:59230) 2023-10-16T11:21:01.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 -- 172.21.15.181:0/3399264280 learned_addr learned my addr 172.21.15.181:0/3399264280 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:01.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f99643700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9415a890 0x7f6f94072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f93fff700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f940cd370 0x7f6f94071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 -- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f940cd370 msgr2=0x7f6f94071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f940cd370 0x7f6f94071cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 -- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9415a890 msgr2=0x7f6f94072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9415a890 0x7f6f94072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 -- 172.21.15.181:0/3399264280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c00bd40 con 0x7f6f9406cb40 2023-10-16T11:21:01.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f99643700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9415a890 0x7f6f94072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:01.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f98e42700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f940c50d0 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e810 tx=0x7f6f8800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:01.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f917fa700 1 -- 172.21.15.181:0/3399264280 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800c020 con 0x7f6f9406cb40 2023-10-16T11:21:01.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f917fa700 1 -- 172.21.15.181:0/3399264280 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f88004250 con 0x7f6f9406cb40 2023-10-16T11:21:01.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f940a9480 con 0x7f6f9406cb40 2023-10-16T11:21:01.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f917fa700 1 -- 172.21.15.181:0/3399264280 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880036a0 con 0x7f6f9406cb40 2023-10-16T11:21:01.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.046+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f940c9110 con 0x7f6f9406cb40 2023-10-16T11:21:01.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.047+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f940c92d0 con 0x7f6f9406cb40 2023-10-16T11:21:01.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.048+0000 7f6f917fa700 1 -- 172.21.15.181:0/3399264280 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 157) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6f88003800 con 0x7f6f9406cb40 2023-10-16T11:21:01.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.048+0000 7f6f917fa700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f6f7806ad50 0x7f6f7806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.048+0000 7f6f917fa700 1 -- 172.21.15.181:0/3399264280 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7985+0+0 (secure 0 0 0) 0x7f6f88089890 con 0x7f6f9406cb40 2023-10-16T11:21:01.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.048+0000 7f6f93fff700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f6f7806ad50 0x7f6f7806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.048+0000 7f6f93fff700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f6f7806ad50 0x7f6f7806d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f94155100 tx=0x7f6f80009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:01.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.052+0000 7f6f917fa700 1 -- 172.21.15.181:0/3399264280 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6f88054390 con 0x7f6f9406cb40 2023-10-16T11:21:01.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.291+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6f940c8f70 con 0x7f6f9406cb40 2023-10-16T11:21:01.294 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:21:01.292+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:21:01.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.293+0000 7f6f917fa700 1 -- 172.21.15.181:0/3399264280 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v134) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6f940c8f70 con 0x7f6f9406cb40 2023-10-16T11:21:01.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f6f7806ad50 msgr2=0x7f6f7806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f6f7806ad50 0x7f6f7806d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f94155100 tx=0x7f6f80009040 comp rx=0 tx=0).stop 2023-10-16T11:21:01.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 msgr2=0x7f6f940c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f940c50d0 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e810 tx=0x7f6f8800ebd0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 shutdown_connections 2023-10-16T11:21:01.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f6f7806ad50 0x7f6f7806d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f9415a890 0x7f6f94072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9406cb40 0x7f6f940c50d0 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 --2- 172.21.15.181:0/3399264280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f940cd370 0x7f6f94071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 >> 172.21.15.181:0/3399264280 conn(0x7f6f9406aed0 msgr2=0x7f6f9415cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:01.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 shutdown_connections 2023-10-16T11:21:01.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.297+0000 7f6f9b0a6700 1 -- 172.21.15.181:0/3399264280 wait complete. 2023-10-16T11:21:01.302 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:21:01.303 INFO:tasks.ceph.mds.a.smithi181.stderr: -5279> 2023-10-16T11:20:32.048+0000 7f3a4cec7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:21:01.312 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:21:01.330 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:21:01.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.433+0000 7fa4b0d1b700 1 Processor -- start 2023-10-16T11:21:01.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.433+0000 7fa4b0d1b700 1 -- start start 2023-10-16T11:21:01.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4b0d1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4ac15fb30 0x7fa4ac06f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4b0d1b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac070590 con 0x7fa4ac0cb2b0 2023-10-16T11:21:01.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4b0d1b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac156330 con 0x7fa4ac0cb690 2023-10-16T11:21:01.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4b0d1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac06fcf0 con 0x7fa4ac15fb30 2023-10-16T11:21:01.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4aa59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa4ac0cb2b0 0x7fa4ac0b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42096/0 (socket says 172.21.15.181:42096) 2023-10-16T11:21:01.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4aa59c700 1 -- 172.21.15.181:0/874450944 learned_addr learned my addr 172.21.15.181:0/874450944 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:01.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4aad9d700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4ac15fb30 0x7fa4ac06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492187256 0 0) 0x7fa4ac070590 con 0x7fa4ac0cb2b0 2023-10-16T11:21:01.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4940035b0 con 0x7fa4ac0cb2b0 2023-10-16T11:21:01.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2680243347 0 0) 0x7fa4940035b0 con 0x7fa4ac0cb2b0 2023-10-16T11:21:01.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4ac15fb30 msgr2=0x7fa4ac06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:01.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4ac15fb30 0x7fa4ac06f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 >> v1:172.21.15.190:6789/0 conn(0x7fa4ac0cb690 legacy=0x7fa4ac15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac06ffd0 con 0x7fa4ac0cb2b0 2023-10-16T11:21:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa49c001670 con 0x7fa4ac0cb2b0 2023-10-16T11:21:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 >> v1:172.21.15.181:6789/0 conn(0x7fa4ac0cb2b0 legacy=0x7fa4ac0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa494003a30 0x7fa494003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4940043e0 0x7fa494004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa494004fd0 0x7fa49400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac06fcf0 con 0x7fa494004fd0 2023-10-16T11:21:01.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49400bf50 con 0x7fa494003a30 2023-10-16T11:21:01.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.434+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49400c090 con 0x7fa4940043e0 2023-10-16T11:21:01.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aad9d700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa494004fd0 0x7fa49400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aa59c700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa494003a30 0x7fa494003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4a9d9b700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4940043e0 0x7fa494004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aad9d700 1 -- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4940043e0 msgr2=0x7fa494004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:01.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aad9d700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4940043e0 0x7fa494004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aad9d700 1 -- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa494003a30 msgr2=0x7fa494003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:01.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aad9d700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa494003a30 0x7fa494003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aad9d700 1 -- 172.21.15.181:0/874450944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4940035b0 con 0x7fa494004fd0 2023-10-16T11:21:01.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4aad9d700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa494004fd0 0x7fa49400ba10 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009f90 tx=0x7fa494005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9f21c9c84d973c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:01.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400ec30 con 0x7fa494004fd0 2023-10-16T11:21:01.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a400f040 con 0x7fa494004fd0 2023-10-16T11:21:01.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.435+0000 7fa4a959a700 1 -- 172.21.15.181:0/874450944 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4014480 con 0x7fa494004fd0 2023-10-16T11:21:01.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.436+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/874450944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa494004fd0 msgr2=0x7fa49400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.436+0000 7fa4b0d1b700 1 --2- 172.21.15.181:0/874450944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa494004fd0 0x7fa49400ba10 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009f90 tx=0x7fa494005880 comp rx=0 tx=0).stop 2023-10-16T11:21:01.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.436+0000 7fa4aad9d700 1 -- 172.21.15.181:0/874450944 reap_dead start 2023-10-16T11:21:01.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.436+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/874450944 shutdown_connections 2023-10-16T11:21:01.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.436+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/874450944 >> 172.21.15.181:0/874450944 conn(0x7fa4ac0cd010 msgr2=0x7fa4ac0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:01.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.436+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/874450944 shutdown_connections 2023-10-16T11:21:01.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.436+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/874450944 wait complete. 2023-10-16T11:21:01.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.437+0000 7fa4b0d1b700 1 Processor -- start 2023-10-16T11:21:01.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.437+0000 7fa4b0d1b700 1 -- start start 2023-10-16T11:21:01.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4b0d1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa494004fd0 0x7fa4ac145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4b0d1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac0c7660 0x7fa4ac0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4b0d1b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0c5a00 0x7fa4ac1f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4b0d1b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0c7bb0 con 0x7fa4ac0c7660 2023-10-16T11:21:01.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4b0d1b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0c7e90 con 0x7fa4ac0c5a00 2023-10-16T11:21:01.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4b0d1b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0c8170 con 0x7fa494004fd0 2023-10-16T11:21:01.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac0c7660 0x7fa4ac0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac0c7660 0x7fa4ac0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59264/0 (socket says 172.21.15.181:59264) 2023-10-16T11:21:01.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 -- 172.21.15.181:0/3139624906 learned_addr learned my addr 172.21.15.181:0/3139624906 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:01.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4aa59c700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa494004fd0 0x7fa4ac145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4aad9d700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0c5a00 0x7fa4ac1f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 -- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa494004fd0 msgr2=0x7fa4ac145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa494004fd0 0x7fa4ac145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 -- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0c5a00 msgr2=0x7fa4ac1f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:01.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0c5a00 0x7fa4ac1f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:01.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 -- 172.21.15.181:0/3139624906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac1f3ff0 con 0x7fa4ac0c7660 2023-10-16T11:21:01.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4a9d9b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac0c7660 0x7fa4ac0ca700 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000f780 tx=0x7fa4a000fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:01.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0017070 con 0x7fa4ac0c7660 2023-10-16T11:21:01.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a00052e0 con 0x7fa4ac0c7660 2023-10-16T11:21:01.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.438+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac0c8450 con 0x7fa4ac0c7660 2023-10-16T11:21:01.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.439+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0005470 con 0x7fa4ac0c7660 2023-10-16T11:21:01.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.439+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac0cac40 con 0x7fa4ac0c7660 2023-10-16T11:21:01.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.440+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ac06f4f0 con 0x7fa4ac0c7660 2023-10-16T11:21:01.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.441+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 157) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa4a0006860 con 0x7fa4ac0c7660 2023-10-16T11:21:01.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.441+0000 7fa49affd700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa48c06ad50 0x7fa48c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:01.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.441+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 8071+0+0 (secure 0 0 0) 0x7fa4a0089c10 con 0x7fa4ac0c7660 2023-10-16T11:21:01.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.441+0000 7fa4aa59c700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa48c06ad50 0x7fa48c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:01.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.441+0000 7fa4aa59c700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa48c06ad50 0x7fa48c06d210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa49c001eb0 tx=0x7fa49c000f80 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:01.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.445+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa4a0054780 con 0x7fa4ac0c7660 2023-10-16T11:21:01.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.684+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa4ac15e120 con 0x7fa4ac0c7660 2023-10-16T11:21:01.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:01.930+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 158) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa4a0054b50 con 0x7fa4ac0c7660 2023-10-16T11:21:02.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.296+0000 7fa49affd700 1 -- 172.21.15.181:0/3139624906 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v135) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa4a003f810 con 0x7fa4ac0c7660 2023-10-16T11:21:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.298+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa48c06ad50 msgr2=0x7fa48c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.298+0000 7fa4b0d1b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa48c06ad50 0x7fa48c06d210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa49c001eb0 tx=0x7fa49c000f80 comp rx=0 tx=0).stop 2023-10-16T11:21:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac0c7660 msgr2=0x7fa4ac0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:02.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac0c7660 0x7fa4ac0ca700 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000f780 tx=0x7fa4a000fa90 comp rx=0 tx=0).stop 2023-10-16T11:21:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 shutdown_connections 2023-10-16T11:21:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa494004fd0 0x7fa4ac145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa48c06ad50 0x7fa48c06d210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4ac0c5a00 0x7fa4ac1f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 --2- 172.21.15.181:0/3139624906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac0c7660 0x7fa4ac0ca700 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 >> 172.21.15.181:0/3139624906 conn(0x7fa4ac0cd010 msgr2=0x7fa4ac0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 shutdown_connections 2023-10-16T11:21:02.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.299+0000 7fa4b0d1b700 1 -- 172.21.15.181:0/3139624906 wait complete. 2023-10-16T11:21:02.315 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:21:02.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.481+0000 7ff423cfa700 1 Processor -- start 2023-10-16T11:21:02.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.481+0000 7ff423cfa700 1 -- start start 2023-10-16T11:21:02.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff423cfa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 0x7ff41c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff423cfa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c06b270 con 0x7ff41c0cc230 2023-10-16T11:21:02.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff423cfa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c15a630 con 0x7ff41c06e7a0 2023-10-16T11:21:02.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff423cfa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c06a940 con 0x7ff41c0665a0 2023-10-16T11:21:02.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff421a96700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff41c0cc230 0x7ff41c06d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42118/0 (socket says 172.21.15.181:42118) 2023-10-16T11:21:02.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff421a96700 1 -- 172.21.15.181:0/3399564798 learned_addr learned my addr 172.21.15.181:0/3399564798 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:02.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff422297700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 0x7ff41c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075692545 0 0) 0x7ff41c06b270 con 0x7ff41c0cc230 2023-10-16T11:21:02.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4080035b0 con 0x7ff41c0cc230 2023-10-16T11:21:02.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986822898 0 0) 0x7ff41c15a630 con 0x7ff41c06e7a0 2023-10-16T11:21:02.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff41c06b270 con 0x7ff41c06e7a0 2023-10-16T11:21:02.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3489442661 0 0) 0x7ff4080035b0 con 0x7ff41c0cc230 2023-10-16T11:21:02.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 msgr2=0x7ff41c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:02.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 0x7ff41c0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 >> v1:172.21.15.190:6789/0 conn(0x7ff41c06e7a0 legacy=0x7ff41c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:02.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c06ac20 con 0x7ff41c0cc230 2023-10-16T11:21:02.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff40c001670 con 0x7ff41c0cc230 2023-10-16T11:21:02.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 >> v1:172.21.15.181:6789/0 conn(0x7ff41c0cc230 legacy=0x7ff41c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:02.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 0x7ff408003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff408004170 0x7ff408004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004d90 0x7ff408005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c06a940 con 0x7ff408004170 2023-10-16T11:21:02.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40800bbd0 con 0x7ff408004d90 2023-10-16T11:21:02.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40800bd90 con 0x7ff41c0665a0 2023-10-16T11:21:02.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.482+0000 7ff421295700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff408004170 0x7ff408004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff422297700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004d90 0x7ff408005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff421a96700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 0x7ff408003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff421295700 1 -- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 msgr2=0x7ff408003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:02.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff421295700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0665a0 0x7ff408003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff421295700 1 -- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004d90 msgr2=0x7ff408005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:02.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff421295700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004d90 0x7ff408005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff421295700 1 -- 172.21.15.181:0/3399564798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080035b0 con 0x7ff408004170 2023-10-16T11:21:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff421295700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff408004170 0x7ff408004850 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ff418001980 tx=0x7ff408005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3078f7c9e9c4b369 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418001470 con 0x7ff408004170 2023-10-16T11:21:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180052d0 con 0x7ff408004170 2023-10-16T11:21:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff420a94700 1 -- 172.21.15.181:0/3399564798 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418003680 con 0x7ff408004170 2023-10-16T11:21:02.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff423cfa700 1 -- 172.21.15.181:0/3399564798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff408004170 msgr2=0x7ff408004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff423cfa700 1 --2- 172.21.15.181:0/3399564798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff408004170 0x7ff408004850 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ff418001980 tx=0x7ff408005880 comp rx=0 tx=0).stop 2023-10-16T11:21:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.483+0000 7ff422297700 1 -- 172.21.15.181:0/3399564798 reap_dead start 2023-10-16T11:21:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.484+0000 7ff423cfa700 1 -- 172.21.15.181:0/3399564798 shutdown_connections 2023-10-16T11:21:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.484+0000 7ff423cfa700 1 -- 172.21.15.181:0/3399564798 >> 172.21.15.181:0/3399564798 conn(0x7ff41c0b1780 msgr2=0x7ff41c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:02.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.484+0000 7ff423cfa700 1 -- 172.21.15.181:0/3399564798 shutdown_connections 2023-10-16T11:21:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.484+0000 7ff423cfa700 1 -- 172.21.15.181:0/3399564798 wait complete. 2023-10-16T11:21:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.485+0000 7ff423cfa700 1 Processor -- start 2023-10-16T11:21:02.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.485+0000 7ff423cfa700 1 -- start start 2023-10-16T11:21:02.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff423cfa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004170 0x7ff41c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff423cfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c15c180 0x7ff41c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff423cfa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c1f35b0 0x7ff41c1ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff423cfa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c146da0 con 0x7ff41c15c180 2023-10-16T11:21:02.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff423cfa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b5490 con 0x7ff408004170 2023-10-16T11:21:02.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff423cfa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c1461d0 con 0x7ff41c1f35b0 2023-10-16T11:21:02.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c15c180 0x7ff41c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c15c180 0x7ff41c0b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59286/0 (socket says 172.21.15.181:59286) 2023-10-16T11:21:02.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 -- 172.21.15.181:0/224945562 learned_addr learned my addr 172.21.15.181:0/224945562 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:02.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff422297700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c1f35b0 0x7ff41c1ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421a96700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004170 0x7ff41c15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 -- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c1f35b0 msgr2=0x7ff41c1ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:02.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c1f35b0 0x7ff41c1ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 -- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004170 msgr2=0x7ff41c15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:02.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004170 0x7ff41c15a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:02.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.486+0000 7ff421295700 1 -- 172.21.15.181:0/224945562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c15c600 con 0x7ff41c15c180 2023-10-16T11:21:02.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.487+0000 7ff422297700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c1f35b0 0x7ff41c1ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:02.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.487+0000 7ff421295700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c15c180 0x7ff41c0b4f50 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7ff41800c3c0 tx=0x7ff418001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:02.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.487+0000 7ff4127fc700 1 -- 172.21.15.181:0/224945562 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418001410 con 0x7ff41c15c180 2023-10-16T11:21:02.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.487+0000 7ff4127fc700 1 -- 172.21.15.181:0/224945562 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff418007040 con 0x7ff41c15c180 2023-10-16T11:21:02.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.487+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c146da0 con 0x7ff41c15c180 2023-10-16T11:21:02.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.487+0000 7ff4127fc700 1 -- 172.21.15.181:0/224945562 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180073f0 con 0x7ff41c15c180 2023-10-16T11:21:02.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.487+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c146750 con 0x7ff41c15c180 2023-10-16T11:21:02.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.488+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c06fd40 con 0x7ff41c15c180 2023-10-16T11:21:02.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.488+0000 7ff4127fc700 1 -- 172.21.15.181:0/224945562 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 158) v1 ==== 89377+0+0 (secure 0 0 0) 0x7ff418007550 con 0x7ff41c15c180 2023-10-16T11:21:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.489+0000 7ff4127fc700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff40006b0b0 0x7ff40006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.489+0000 7ff421a96700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff40006b0b0 0x7ff40006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:02.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.489+0000 7ff4127fc700 1 -- 172.21.15.181:0/224945562 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 8071+0+0 (secure 0 0 0) 0x7ff41808b2c0 con 0x7ff41c15c180 2023-10-16T11:21:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.489+0000 7ff421a96700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff40006b0b0 0x7ff40006d570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff40c000c00 tx=0x7ff40c009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:02.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.493+0000 7ff4127fc700 1 -- 172.21.15.181:0/224945562 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff418055840 con 0x7ff41c15c180 2023-10-16T11:21:02.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:02.684+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff41c067d20 con 0x7ff41c15c180 2023-10-16T11:21:03.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.460+0000 7ff4127fc700 1 -- 172.21.15.181:0/224945562 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v244) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff418058e60 con 0x7ff41c15c180 2023-10-16T11:21:03.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.468+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff40006b0b0 msgr2=0x7ff40006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:03.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.468+0000 7ff423cfa700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff40006b0b0 0x7ff40006d570 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff40c000c00 tx=0x7ff40c009040 comp rx=0 tx=0).stop 2023-10-16T11:21:03.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.468+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c15c180 msgr2=0x7ff41c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:03.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.468+0000 7ff423cfa700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c15c180 0x7ff41c0b4f50 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7ff41800c3c0 tx=0x7ff418001310 comp rx=0 tx=0).stop 2023-10-16T11:21:03.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 shutdown_connections 2023-10-16T11:21:03.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c1f35b0 0x7ff41c1ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff40006b0b0 0x7ff40006d570 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff408004170 0x7ff41c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 --2- 172.21.15.181:0/224945562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c15c180 0x7ff41c0b4f50 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 >> 172.21.15.181:0/224945562 conn(0x7ff41c0b1780 msgr2=0x7ff41c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:03.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 shutdown_connections 2023-10-16T11:21:03.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.469+0000 7ff423cfa700 1 -- 172.21.15.181:0/224945562 wait complete. 2023-10-16T11:21:03.476 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:21:03.490 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:21:03.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.621+0000 7f266b4e9700 1 Processor -- start 2023-10-16T11:21:03.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.622+0000 7f266b4e9700 1 -- start start 2023-10-16T11:21:03.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.622+0000 7f266b4e9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f266415eea0 0x7f26640b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:03.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.622+0000 7f266b4e9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:03.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.622+0000 7f266b4e9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26640d23c0 con 0x7f26640ce640 2023-10-16T11:21:03.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.622+0000 7f266b4e9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640cd990 con 0x7f266415eea0 2023-10-16T11:21:03.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.622+0000 7f266b4e9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640cdc70 con 0x7f26640b0f70 2023-10-16T11:21:03.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f266415eea0 0x7f26640b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:03.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2668a84700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:03.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f266415eea0 0x7f26640b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:36110/0 (socket says 172.21.15.181:36110) 2023-10-16T11:21:03.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2668a84700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:56318/0 (socket says 172.21.15.181:56318) 2023-10-16T11:21:03.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 -- 172.21.15.181:0/3152205970 learned_addr learned my addr 172.21.15.181:0/3152205970 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:03.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 -- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 msgr2=0x7f26640ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:03.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 --2- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 -- 172.21.15.181:0/3152205970 >> v1:172.21.15.190:6789/0 conn(0x7f26640ce640 legacy=0x7f26640cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:03.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 -- 172.21.15.181:0/3152205970 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640cce60 con 0x7f266415eea0 2023-10-16T11:21:03.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2668a84700 1 --2- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:03.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f2669285700 1 --2- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415eea0 0x7f26640b03e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f26540098f0 tx=0x7f2654009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f489ff58cd9227b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:03.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.623+0000 7f265bfff700 1 -- 172.21.15.181:0/3152205970 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654011830 con 0x7f266415eea0 2023-10-16T11:21:03.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f265bfff700 1 -- 172.21.15.181:0/3152205970 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654011990 con 0x7f266415eea0 2023-10-16T11:21:03.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f265bfff700 1 -- 172.21.15.181:0/3152205970 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654011d20 con 0x7f266415eea0 2023-10-16T11:21:03.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 -- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415eea0 msgr2=0x7f26640b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:03.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 --2- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415eea0 0x7f26640b03e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f26540098f0 tx=0x7f2654009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:03.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 -- 172.21.15.181:0/3152205970 shutdown_connections 2023-10-16T11:21:03.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 --2- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f266415eea0 0x7f26640b03e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 --2- 172.21.15.181:0/3152205970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 -- 172.21.15.181:0/3152205970 >> 172.21.15.181:0/3152205970 conn(0x7f26640d1610 msgr2=0x7f26640d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:03.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 -- 172.21.15.181:0/3152205970 shutdown_connections 2023-10-16T11:21:03.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.624+0000 7f266b4e9700 1 -- 172.21.15.181:0/3152205970 wait complete. 2023-10-16T11:21:03.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.625+0000 7f266b4e9700 1 Processor -- start 2023-10-16T11:21:03.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.625+0000 7f266b4e9700 1 -- start start 2023-10-16T11:21:03.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f266b4e9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:03.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f266b4e9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26640ce640 0x7f266406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:03.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f266b4e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266415eea0 0x7f26641514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:03.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f266b4e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640c51b0 con 0x7f266415eea0 2023-10-16T11:21:03.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f266b4e9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641f6ec0 con 0x7f26640ce640 2023-10-16T11:21:03.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f266b4e9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26641f71a0 con 0x7f26640b0f70 2023-10-16T11:21:03.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669a86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266415eea0 0x7f26641514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:03.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669a86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266415eea0 0x7f26641514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59302/0 (socket says 172.21.15.181:59302) 2023-10-16T11:21:03.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669a86700 1 -- 172.21.15.181:0/543050116 learned_addr learned my addr 172.21.15.181:0/543050116 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:03.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2668a84700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26640ce640 0x7f266406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:03.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669285700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:03.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669a86700 1 -- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 msgr2=0x7f26640c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:03.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669a86700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669a86700 1 -- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26640ce640 msgr2=0x7f266406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:03.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.626+0000 7f2669a86700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26640ce640 0x7f266406fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:03.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f2669a86700 1 -- 172.21.15.181:0/543050116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26600094a0 con 0x7f266415eea0 2023-10-16T11:21:03.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f2669285700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:03.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f2668a84700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26640ce640 0x7f266406fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:03.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f2669a86700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266415eea0 0x7f26641514f0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f265c00cc50 tx=0x7f265c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:03.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f2659ffb700 1 -- 172.21.15.181:0/543050116 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c00ad80 con 0x7f266415eea0 2023-10-16T11:21:03.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f2659ffb700 1 -- 172.21.15.181:0/543050116 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c00ce50 con 0x7f266415eea0 2023-10-16T11:21:03.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f2659ffb700 1 -- 172.21.15.181:0/543050116 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c003c40 con 0x7f266415eea0 2023-10-16T11:21:03.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2654009550 con 0x7f266415eea0 2023-10-16T11:21:03.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.627+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26641f7760 con 0x7f266415eea0 2023-10-16T11:21:03.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.628+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2664067d20 con 0x7f266415eea0 2023-10-16T11:21:03.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.628+0000 7f2659ffb700 1 -- 172.21.15.181:0/543050116 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f265c0050d0 con 0x7f266415eea0 2023-10-16T11:21:03.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.629+0000 7f2659ffb700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f264806b140 0x7f264806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.629+0000 7f2659ffb700 1 -- 172.21.15.181:0/543050116 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 7633+0+0 (secure 0 0 0) 0x7f265c08a1e0 con 0x7f266415eea0 2023-10-16T11:21:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.629+0000 7f2669285700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f264806b140 0x7f264806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.629+0000 7f2669285700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f264806b140 0x7f264806d600 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2654014040 tx=0x7f2654008030 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:03.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.633+0000 7f2659ffb700 1 -- 172.21.15.181:0/543050116 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f265c054830 con 0x7f266415eea0 2023-10-16T11:21:03.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:03.824+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f26640ccb70 con 0x7f266415eea0 2023-10-16T11:21:04.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.467+0000 7f2659ffb700 1 -- 172.21.15.181:0/543050116 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v245) v1 ==== 169+0+0 (secure 0 0 0) 0x7f265c057e50 con 0x7f266415eea0 2023-10-16T11:21:04.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f264806b140 msgr2=0x7f264806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f264806b140 0x7f264806d600 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2654014040 tx=0x7f2654008030 comp rx=0 tx=0).stop 2023-10-16T11:21:04.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266415eea0 msgr2=0x7f26641514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266415eea0 0x7f26641514f0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f265c00cc50 tx=0x7f265c00a340 comp rx=0 tx=0).stop 2023-10-16T11:21:04.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 shutdown_connections 2023-10-16T11:21:04.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26640b0f70 0x7f26640c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f264806b140 0x7f264806d600 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f266415eea0 0x7f26641514f0 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 --2- 172.21.15.181:0/543050116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26640ce640 0x7f266406fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 >> 172.21.15.181:0/543050116 conn(0x7f26640d1610 msgr2=0x7f26640d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:04.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 shutdown_connections 2023-10-16T11:21:04.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.470+0000 7f266b4e9700 1 -- 172.21.15.181:0/543050116 wait complete. 2023-10-16T11:21:04.475 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:21:04.491 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:04.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.615+0000 7f7271707700 1 Processor -- start 2023-10-16T11:21:04.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.615+0000 7f7271707700 1 -- start start 2023-10-16T11:21:04.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f7271707700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f726c0cb000 0x7f726c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f7271707700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f726c0cc4d0 0x7f726c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f7271707700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726c15e210 con 0x7f726c0cac20 2023-10-16T11:21:04.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f7271707700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0cc0b0 con 0x7f726c0cc4d0 2023-10-16T11:21:04.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f7271707700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c15e4f0 con 0x7f726c0cb000 2023-10-16T11:21:04.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f726c0cc4d0 0x7f726c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f726c0cc4d0 0x7f726c0cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59310/0 (socket says 172.21.15.181:59310) 2023-10-16T11:21:04.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 -- 172.21.15.181:0/4248269718 learned_addr learned my addr 172.21.15.181:0/4248269718 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:04.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726a7fc700 1 --2- 172.21.15.181:0/4248269718 >> v2:172.21.15.190:3300/0 conn(0x7f726c0cb000 0x7f726c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 -- 172.21.15.181:0/4248269718 >> v2:172.21.15.190:3300/0 conn(0x7f726c0cb000 msgr2=0x7f726c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:04.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 --2- 172.21.15.181:0/4248269718 >> v2:172.21.15.190:3300/0 conn(0x7f726c0cb000 0x7f726c15af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 -- 172.21.15.181:0/4248269718 >> v1:172.21.15.190:6789/0 conn(0x7f726c0cac20 legacy=0x7f726c15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:04.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 -- 172.21.15.181:0/4248269718 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c15e7d0 con 0x7f726c0cc4d0 2023-10-16T11:21:04.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f726b7fe700 1 --2- 172.21.15.181:0/4248269718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cc4d0 0x7f726c0cf090 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f726c0b0f20 tx=0x7f726400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386ce5c8091080fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:04.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.616+0000 7f7269ffb700 1 -- 172.21.15.181:0/4248269718 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726400f070 con 0x7f726c0cc4d0 2023-10-16T11:21:04.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7269ffb700 1 -- 172.21.15.181:0/4248269718 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726400a2e0 con 0x7f726c0cc4d0 2023-10-16T11:21:04.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7269ffb700 1 -- 172.21.15.181:0/4248269718 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726400cc80 con 0x7f726c0cc4d0 2023-10-16T11:21:04.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 -- 172.21.15.181:0/4248269718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cc4d0 msgr2=0x7f726c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 --2- 172.21.15.181:0/4248269718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cc4d0 0x7f726c0cf090 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f726c0b0f20 tx=0x7f726400c330 comp rx=0 tx=0).stop 2023-10-16T11:21:04.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 -- 172.21.15.181:0/4248269718 shutdown_connections 2023-10-16T11:21:04.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 --2- 172.21.15.181:0/4248269718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cc4d0 0x7f726c0cf090 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 --2- 172.21.15.181:0/4248269718 >> v2:172.21.15.190:3300/0 conn(0x7f726c0cb000 0x7f726c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 -- 172.21.15.181:0/4248269718 >> 172.21.15.181:0/4248269718 conn(0x7f726c06aed0 msgr2=0x7f726c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:04.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 -- 172.21.15.181:0/4248269718 shutdown_connections 2023-10-16T11:21:04.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.617+0000 7f7271707700 1 -- 172.21.15.181:0/4248269718 wait complete. 2023-10-16T11:21:04.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.618+0000 7f7271707700 1 Processor -- start 2023-10-16T11:21:04.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.618+0000 7f7271707700 1 -- start start 2023-10-16T11:21:04.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f7271707700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f726c0cac20 0x7f726c152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f7271707700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cb000 0x7f726c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f7271707700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f726c0cc4d0 0x7f726c15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f7271707700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c1e4ff0 con 0x7f726c0cb000 2023-10-16T11:21:04.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f7271707700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c15d530 con 0x7f726c0cc4d0 2023-10-16T11:21:04.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f7271707700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0c75c0 con 0x7f726c0cac20 2023-10-16T11:21:04.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cb000 0x7f726c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cb000 0x7f726c15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59314/0 (socket says 172.21.15.181:59314) 2023-10-16T11:21:04.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726a7fc700 1 -- 172.21.15.181:0/3956381020 learned_addr learned my addr 172.21.15.181:0/3956381020 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:04.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726a7fc700 1 -- 172.21.15.181:0/3956381020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f726c0cac20 msgr2=0x7f726c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726b7fe700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f726c0cc4d0 0x7f726c15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726a7fc700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f726c0cac20 0x7f726c152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726a7fc700 1 -- 172.21.15.181:0/3956381020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f726c0cc4d0 msgr2=0x7f726c15cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.619+0000 7f726a7fc700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f726c0cc4d0 0x7f726c15cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.620+0000 7f726a7fc700 1 -- 172.21.15.181:0/3956381020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726400ac40 con 0x7f726c0cb000 2023-10-16T11:21:04.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.620+0000 7f726a7fc700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cb000 0x7f726c15c760 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c850 tx=0x7f725c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:04.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.620+0000 7f72537fe700 1 -- 172.21.15.181:0/3956381020 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c004220 con 0x7f726c0cb000 2023-10-16T11:21:04.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.620+0000 7f72537fe700 1 -- 172.21.15.181:0/3956381020 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c004380 con 0x7f726c0cb000 2023-10-16T11:21:04.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.620+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c0cbd80 con 0x7f726c0cb000 2023-10-16T11:21:04.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.620+0000 7f72537fe700 1 -- 172.21.15.181:0/3956381020 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0036a0 con 0x7f726c0cb000 2023-10-16T11:21:04.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.620+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c153390 con 0x7f726c0cb000 2023-10-16T11:21:04.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.621+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c04ebb0 con 0x7f726c0cb000 2023-10-16T11:21:04.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.621+0000 7f72537fe700 1 -- 172.21.15.181:0/3956381020 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f725c003800 con 0x7f726c0cb000 2023-10-16T11:21:04.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.622+0000 7f72537fe700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f725806b140 0x7f725806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.622+0000 7f72537fe700 1 -- 172.21.15.181:0/3956381020 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f725c089c80 con 0x7f726c0cb000 2023-10-16T11:21:04.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.622+0000 7f726affd700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f725806b140 0x7f725806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.622+0000 7f726affd700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f725806b140 0x7f725806d600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7260001600 tx=0x7f7260007a20 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:04.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.626+0000 7f72537fe700 1 -- 172.21.15.181:0/3956381020 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f725c054360 con 0x7f726c0cb000 2023-10-16T11:21:04.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.819+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f726c0cd700 con 0x7f726c0cb000 2023-10-16T11:21:04.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.820+0000 7f72537fe700 1 -- 172.21.15.181:0/3956381020 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v245) v1 ==== 74+0+18825 (secure 0 0 0) 0x7f725c057980 con 0x7f726c0cb000 2023-10-16T11:21:04.822 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:04.823 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":245,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:04.461998+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:6834/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:0/4174526666":"2023-10-17T11:20:58.912264+0000","172.21.15.181:6832/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2905113763":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1596741027":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3516831361":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2396727780":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2869414246":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3445652558":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3765547571":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:04.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f725806b140 msgr2=0x7f725806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f725806b140 0x7f725806d600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7260001600 tx=0x7f7260007a20 comp rx=0 tx=0).stop 2023-10-16T11:21:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cb000 msgr2=0x7f726c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cb000 0x7f726c15c760 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c850 tx=0x7f725c00cc10 comp rx=0 tx=0).stop 2023-10-16T11:21:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 shutdown_connections 2023-10-16T11:21:04.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f726c0cac20 0x7f726c152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f725806b140 0x7f725806d600 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f726c0cc4d0 0x7f726c15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 --2- 172.21.15.181:0/3956381020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f726c0cb000 0x7f726c15c760 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 >> 172.21.15.181:0/3956381020 conn(0x7f726c06aed0 msgr2=0x7f726c153a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:04.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 shutdown_connections 2023-10-16T11:21:04.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.822+0000 7f7271707700 1 -- 172.21.15.181:0/3956381020 wait complete. 2023-10-16T11:21:04.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-16T11:20:36.190463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '239', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-16T11:20:37.509605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '240', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:21:04.838 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:21:04.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a64c8700 1 Processor -- start 2023-10-16T11:21:04.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a64c8700 1 -- start start 2023-10-16T11:21:04.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a64c8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60a00d01d0 0x7f60a00ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a64c8700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f60a015ea40 0x7f60a00d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a64c8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a00cf580 con 0x7f60a0161530 2023-10-16T11:21:04.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a64c8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a00d4290 con 0x7f60a00d01d0 2023-10-16T11:21:04.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a64c8700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a00d0af0 con 0x7f60a015ea40 2023-10-16T11:21:04.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f609ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60a00d01d0 0x7f60a00ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a4a65700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f60a0161530 0x7f60a01619c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42170/0 (socket says 172.21.15.181:42170) 2023-10-16T11:21:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.961+0000 7f60a4a65700 1 -- 172.21.15.181:0/2807046226 learned_addr learned my addr 172.21.15.181:0/2807046226 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609f7fe700 1 --2- 172.21.15.181:0/2807046226 >> v2:172.21.15.190:3300/0 conn(0x7f60a015ea40 0x7f60a00d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609ffff700 1 -- 172.21.15.181:0/2807046226 >> v2:172.21.15.190:3300/0 conn(0x7f60a015ea40 msgr2=0x7f60a00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:04.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609ffff700 1 --2- 172.21.15.181:0/2807046226 >> v2:172.21.15.190:3300/0 conn(0x7f60a015ea40 0x7f60a00d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609ffff700 1 -- 172.21.15.181:0/2807046226 >> v1:172.21.15.181:6789/0 conn(0x7f60a0161530 legacy=0x7f60a01619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609ffff700 1 -- 172.21.15.181:0/2807046226 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a00ce970 con 0x7f60a00d01d0 2023-10-16T11:21:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609effd700 1 -- 172.21.15.181:0/2807046226 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624640296 0 0) 0x7f60a00cf580 con 0x7f60a0161530 2023-10-16T11:21:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609f7fe700 1 --2- 172.21.15.181:0/2807046226 >> v2:172.21.15.190:3300/0 conn(0x7f60a015ea40 0x7f60a00d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:04.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609ffff700 1 --2- 172.21.15.181:0/2807046226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a00d01d0 0x7f60a00ce2d0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f60a00d12c0 tx=0x7f6094009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def8d078469d6189 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609effd700 1 -- 172.21.15.181:0/2807046226 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094011830 con 0x7f60a00d01d0 2023-10-16T11:21:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609effd700 1 -- 172.21.15.181:0/2807046226 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6094011990 con 0x7f60a00d01d0 2023-10-16T11:21:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.962+0000 7f609effd700 1 -- 172.21.15.181:0/2807046226 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094011d20 con 0x7f60a00d01d0 2023-10-16T11:21:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 -- 172.21.15.181:0/2807046226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a00d01d0 msgr2=0x7f60a00ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 --2- 172.21.15.181:0/2807046226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a00d01d0 0x7f60a00ce2d0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f60a00d12c0 tx=0x7f6094009b20 comp rx=0 tx=0).stop 2023-10-16T11:21:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 -- 172.21.15.181:0/2807046226 shutdown_connections 2023-10-16T11:21:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 --2- 172.21.15.181:0/2807046226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a00d01d0 0x7f60a00ce2d0 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 --2- 172.21.15.181:0/2807046226 >> v2:172.21.15.190:3300/0 conn(0x7f60a015ea40 0x7f60a00d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 -- 172.21.15.181:0/2807046226 >> 172.21.15.181:0/2807046226 conn(0x7f60a009cb60 msgr2=0x7f60a00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:04.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 -- 172.21.15.181:0/2807046226 shutdown_connections 2023-10-16T11:21:04.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.963+0000 7f60a64c8700 1 -- 172.21.15.181:0/2807046226 wait complete. 2023-10-16T11:21:04.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.964+0000 7f60a64c8700 1 Processor -- start 2023-10-16T11:21:04.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.964+0000 7f60a64c8700 1 -- start start 2023-10-16T11:21:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a64c8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60a00d01d0 0x7f60a015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a64c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60a015ea40 0x7f60a006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a64c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a0161530 0x7f60a015f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a64c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00b6590 con 0x7f60a0161530 2023-10-16T11:21:04.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a64c8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00b8410 con 0x7f60a00d01d0 2023-10-16T11:21:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a64c8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a015b2a0 con 0x7f60a015ea40 2023-10-16T11:21:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a0161530 0x7f60a015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a0161530 0x7f60a015f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59330/0 (socket says 172.21.15.181:59330) 2023-10-16T11:21:04.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 -- 172.21.15.181:0/981461953 learned_addr learned my addr 172.21.15.181:0/981461953 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f609ffff700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60a00d01d0 0x7f60a015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 -- 172.21.15.181:0/981461953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60a015ea40 msgr2=0x7f60a006ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60a015ea40 0x7f60a006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 -- 172.21.15.181:0/981461953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60a00d01d0 msgr2=0x7f60a015ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:04.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60a00d01d0 0x7f60a015ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 -- 172.21.15.181:0/981461953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940094a0 con 0x7f60a0161530 2023-10-16T11:21:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a4a65700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a0161530 0x7f60a015f470 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f609800cbd0 tx=0x7f609800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f609cff9700 1 -- 172.21.15.181:0/981461953 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800ab70 con 0x7f60a0161530 2023-10-16T11:21:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f609cff9700 1 -- 172.21.15.181:0/981461953 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609800acf0 con 0x7f60a0161530 2023-10-16T11:21:04.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a01f4560 con 0x7f60a0161530 2023-10-16T11:21:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.965+0000 7f609cff9700 1 -- 172.21.15.181:0/981461953 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098007310 con 0x7f60a0161530 2023-10-16T11:21:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.966+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a00b88e0 con 0x7f60a0161530 2023-10-16T11:21:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.966+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a004ebb0 con 0x7f60a0161530 2023-10-16T11:21:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.967+0000 7f609cff9700 1 -- 172.21.15.181:0/981461953 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6098007470 con 0x7f60a0161530 2023-10-16T11:21:04.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.967+0000 7f609cff9700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f608c06b140 0x7f608c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.967+0000 7f609cff9700 1 -- 172.21.15.181:0/981461953 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f6098089e60 con 0x7f60a0161530 2023-10-16T11:21:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.967+0000 7f609ffff700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f608c06b140 0x7f608c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.968+0000 7f609ffff700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f608c06b140 0x7f608c06d600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f6094008030 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:04.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:04.972+0000 7f609cff9700 1 -- 172.21.15.181:0/981461953 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f60980545c0 con 0x7f60a0161530 2023-10-16T11:21:05.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.165+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60a00d0e00 con 0x7f60a0161530 2023-10-16T11:21:05.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.165+0000 7f609cff9700 1 -- 172.21.15.181:0/981461953 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v245) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6098018020 con 0x7f60a0161530 2023-10-16T11:21:05.168 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:21:05.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f608c06b140 msgr2=0x7f608c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f608c06b140 0x7f608c06d600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f6094008030 comp rx=0 tx=0).stop 2023-10-16T11:21:05.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a0161530 msgr2=0x7f60a015f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a0161530 0x7f60a015f470 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f609800cbd0 tx=0x7f609800cf90 comp rx=0 tx=0).stop 2023-10-16T11:21:05.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 shutdown_connections 2023-10-16T11:21:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f608c06b140 0x7f608c06d600 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a0161530 0x7f60a015f470 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60a00d01d0 0x7f60a015ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 --2- 172.21.15.181:0/981461953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60a015ea40 0x7f60a006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.167+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 >> 172.21.15.181:0/981461953 conn(0x7f60a009cb60 msgr2=0x7f60a00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.168+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 shutdown_connections 2023-10-16T11:21:05.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.168+0000 7f60a64c8700 1 -- 172.21.15.181:0/981461953 wait complete. 2023-10-16T11:21:05.182 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-16T11:21:05.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.310+0000 7fb3d7e0d700 1 Processor -- start 2023-10-16T11:21:05.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.310+0000 7fb3d7e0d700 1 -- start start 2023-10-16T11:21:05.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d7e0d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3d00666a0 0x7fb3d006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d7e0d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3d00ce0e0 0x7fb3d006afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d7e0d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d00ca300 con 0x7fb3d00cdd00 2023-10-16T11:21:05.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d7e0d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d006e5e0 con 0x7fb3d00ce0e0 2023-10-16T11:21:05.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d7e0d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00ca580 con 0x7fb3d00666a0 2023-10-16T11:21:05.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d5ba9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb3d00cdd00 0x7fb3d006a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42180/0 (socket says 172.21.15.181:42180) 2023-10-16T11:21:05.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d5ba9700 1 -- 172.21.15.181:0/1855042296 learned_addr learned my addr 172.21.15.181:0/1855042296 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:05.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d63aa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3d00ce0e0 0x7fb3d006afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d53a8700 1 --2- 172.21.15.181:0/1855042296 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00666a0 0x7fb3d006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d63aa700 1 -- 172.21.15.181:0/1855042296 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00666a0 msgr2=0x7fb3d006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:05.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d63aa700 1 --2- 172.21.15.181:0/1855042296 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00666a0 0x7fb3d006f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d63aa700 1 -- 172.21.15.181:0/1855042296 >> v1:172.21.15.181:6789/0 conn(0x7fb3d00cdd00 legacy=0x7fb3d006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d63aa700 1 -- 172.21.15.181:0/1855042296 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00ca860 con 0x7fb3d00ce0e0 2023-10-16T11:21:05.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d4ba7700 1 -- 172.21.15.181:0/1855042296 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651411024 0 0) 0x7fb3d00ca300 con 0x7fb3d00cdd00 2023-10-16T11:21:05.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.311+0000 7fb3d63aa700 1 --2- 172.21.15.181:0/1855042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00ce0e0 0x7fb3d006afb0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00cc6e0 tx=0x7fb3c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489c0a2fd652914e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:05.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d4ba7700 1 -- 172.21.15.181:0/1855042296 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800e070 con 0x7fb3d00ce0e0 2023-10-16T11:21:05.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d4ba7700 1 -- 172.21.15.181:0/1855042296 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c800bbd0 con 0x7fb3d00ce0e0 2023-10-16T11:21:05.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d4ba7700 1 -- 172.21.15.181:0/1855042296 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8012430 con 0x7fb3d00ce0e0 2023-10-16T11:21:05.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/1855042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00ce0e0 msgr2=0x7fb3d006afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/1855042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00ce0e0 0x7fb3d006afb0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00cc6e0 tx=0x7fb3c800b330 comp rx=0 tx=0).stop 2023-10-16T11:21:05.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/1855042296 shutdown_connections 2023-10-16T11:21:05.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/1855042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00ce0e0 0x7fb3d006afb0 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/1855042296 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00666a0 0x7fb3d006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/1855042296 >> 172.21.15.181:0/1855042296 conn(0x7fb3d00cc730 msgr2=0x7fb3d00ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/1855042296 shutdown_connections 2023-10-16T11:21:05.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.312+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/1855042296 wait complete. 2023-10-16T11:21:05.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.313+0000 7fb3d7e0d700 1 Processor -- start 2023-10-16T11:21:05.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.313+0000 7fb3d7e0d700 1 -- start start 2023-10-16T11:21:05.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d7e0d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00666a0 0x7fb3d01502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d7e0d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d00cdd00 0x7fb3d00c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d7e0d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00ce0e0 0x7fb3d00ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d7e0d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00c8150 con 0x7fb3d00666a0 2023-10-16T11:21:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d7e0d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00c8430 con 0x7fb3d00cdd00 2023-10-16T11:21:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d7e0d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00c8710 con 0x7fb3d00ce0e0 2023-10-16T11:21:05.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d5ba9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00666a0 0x7fb3d01502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d5ba9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00666a0 0x7fb3d01502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59340/0 (socket says 172.21.15.181:59340) 2023-10-16T11:21:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.314+0000 7fb3d5ba9700 1 -- 172.21.15.181:0/3050316636 learned_addr learned my addr 172.21.15.181:0/3050316636 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d53a8700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d00cdd00 0x7fb3d00c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d5ba9700 1 -- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00ce0e0 msgr2=0x7fb3d00ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d63aa700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00ce0e0 0x7fb3d00ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d5ba9700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00ce0e0 0x7fb3d00ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d5ba9700 1 -- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d00cdd00 msgr2=0x7fb3d00c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d5ba9700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d00cdd00 0x7fb3d00c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d5ba9700 1 -- 172.21.15.181:0/3050316636 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8009a70 con 0x7fb3d00666a0 2023-10-16T11:21:05.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d5ba9700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00666a0 0x7fb3d01502e0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0011e0 tx=0x7fb3cc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:05.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3c67fc700 1 -- 172.21.15.181:0/3050316636 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc004c80 con 0x7fb3d00666a0 2023-10-16T11:21:05.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3c67fc700 1 -- 172.21.15.181:0/3050316636 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc004e00 con 0x7fb3d00666a0 2023-10-16T11:21:05.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d00c89f0 con 0x7fb3d00666a0 2023-10-16T11:21:05.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3c67fc700 1 -- 172.21.15.181:0/3050316636 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc007310 con 0x7fb3d00666a0 2023-10-16T11:21:05.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.315+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d015d7a0 con 0x7fb3d00666a0 2023-10-16T11:21:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.316+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d00c3d50 con 0x7fb3d00666a0 2023-10-16T11:21:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.317+0000 7fb3c67fc700 1 -- 172.21.15.181:0/3050316636 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb3cc007470 con 0x7fb3d00666a0 2023-10-16T11:21:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.317+0000 7fb3c67fc700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fb3b406b140 0x7fb3b406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.317+0000 7fb3c67fc700 1 -- 172.21.15.181:0/3050316636 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7fb3cc089d40 con 0x7fb3d00666a0 2023-10-16T11:21:05.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.317+0000 7fb3d53a8700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fb3b406b140 0x7fb3b406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.317+0000 7fb3d53a8700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fb3b406b140 0x7fb3b406d600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb3d015db50 tx=0x7fb3c0008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.321+0000 7fb3c67fc700 1 -- 172.21.15.181:0/3050316636 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb3cc054530 con 0x7fb3d00666a0 2023-10-16T11:21:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.515+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3d0160740 con 0x7fb3d00666a0 2023-10-16T11:21:05.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.515+0000 7fb3c67fc700 1 -- 172.21.15.181:0/3050316636 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v245) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb3cc057b50 con 0x7fb3d00666a0 2023-10-16T11:21:05.518 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:21:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fb3b406b140 msgr2=0x7fb3b406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fb3b406b140 0x7fb3b406d600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb3d015db50 tx=0x7fb3c0008040 comp rx=0 tx=0).stop 2023-10-16T11:21:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00666a0 msgr2=0x7fb3d01502e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00666a0 0x7fb3d01502e0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0011e0 tx=0x7fb3cc00cae0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 shutdown_connections 2023-10-16T11:21:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d00ce0e0 0x7fb3d00ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fb3b406b140 0x7fb3b406d600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3d00666a0 0x7fb3d01502e0 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 --2- 172.21.15.181:0/3050316636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d00cdd00 0x7fb3d00c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 >> 172.21.15.181:0/3050316636 conn(0x7fb3d00cc730 msgr2=0x7fb3d006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 shutdown_connections 2023-10-16T11:21:05.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.518+0000 7fb3d7e0d700 1 -- 172.21.15.181:0/3050316636 wait complete. 2023-10-16T11:21:05.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-16T11:21:05.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.652+0000 7fba900b3700 1 Processor -- start 2023-10-16T11:21:05.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.653+0000 7fba900b3700 1 -- start start 2023-10-16T11:21:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.653+0000 7fba900b3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba880665a0 0x7fba8815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.653+0000 7fba900b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba8806f1e0 0x7fba8806f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.653+0000 7fba900b3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba88160420 con 0x7fba8815b850 2023-10-16T11:21:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.653+0000 7fba900b3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8806e4b0 con 0x7fba8806f1e0 2023-10-16T11:21:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.653+0000 7fba900b3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8806ed50 con 0x7fba880665a0 2023-10-16T11:21:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba8806f1e0 0x7fba8806f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba8806f1e0 0x7fba8806f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59352/0 (socket says 172.21.15.181:59352) 2023-10-16T11:21:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 -- 172.21.15.181:0/3886459381 learned_addr learned my addr 172.21.15.181:0/3886459381 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8d64e700 1 --2- 172.21.15.181:0/3886459381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba880665a0 0x7fba8815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 -- 172.21.15.181:0/3886459381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba880665a0 msgr2=0x7fba8815fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 --2- 172.21.15.181:0/3886459381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba880665a0 0x7fba8815fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 -- 172.21.15.181:0/3886459381 >> v1:172.21.15.190:6789/0 conn(0x7fba8815b850 legacy=0x7fba880c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 -- 172.21.15.181:0/3886459381 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880cb0f0 con 0x7fba8806f1e0 2023-10-16T11:21:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8d64e700 1 --2- 172.21.15.181:0/3886459381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba880665a0 0x7fba8815fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8e650700 1 --2- 172.21.15.181:0/3886459381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8806f1e0 0x7fba8806f640 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7fba80009e50 tx=0x7fba8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8644951feb203c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8ce4d700 1 -- 172.21.15.181:0/3886459381 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8000e070 con 0x7fba8806f1e0 2023-10-16T11:21:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8ce4d700 1 -- 172.21.15.181:0/3886459381 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8000b9c0 con 0x7fba8806f1e0 2023-10-16T11:21:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.654+0000 7fba8ce4d700 1 -- 172.21.15.181:0/3886459381 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8000bd70 con 0x7fba8806f1e0 2023-10-16T11:21:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 -- 172.21.15.181:0/3886459381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8806f1e0 msgr2=0x7fba8806f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 --2- 172.21.15.181:0/3886459381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8806f1e0 0x7fba8806f640 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7fba80009e50 tx=0x7fba8000b330 comp rx=0 tx=0).stop 2023-10-16T11:21:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 -- 172.21.15.181:0/3886459381 shutdown_connections 2023-10-16T11:21:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 --2- 172.21.15.181:0/3886459381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8806f1e0 0x7fba8806f640 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 --2- 172.21.15.181:0/3886459381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba880665a0 0x7fba8815fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 -- 172.21.15.181:0/3886459381 >> 172.21.15.181:0/3886459381 conn(0x7fba880cc730 msgr2=0x7fba880ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 -- 172.21.15.181:0/3886459381 shutdown_connections 2023-10-16T11:21:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.655+0000 7fba900b3700 1 -- 172.21.15.181:0/3886459381 wait complete. 2023-10-16T11:21:05.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.656+0000 7fba900b3700 1 Processor -- start 2023-10-16T11:21:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.656+0000 7fba900b3700 1 -- start start 2023-10-16T11:21:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba880665a0 0x7fba8814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8806f1e0 0x7fba88155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8815b850 0x7fba88156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba880b56e0 con 0x7fba880665a0 2023-10-16T11:21:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba880b4af0 con 0x7fba8806f1e0 2023-10-16T11:21:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880b4d40 con 0x7fba8815b850 2023-10-16T11:21:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba880665a0 0x7fba8814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba880665a0 0x7fba8814b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59354/0 (socket says 172.21.15.181:59354) 2023-10-16T11:21:05.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 -- 172.21.15.181:0/3149834212 learned_addr learned my addr 172.21.15.181:0/3149834212 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8d64e700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8806f1e0 0x7fba88155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8e650700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8815b850 0x7fba88156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 -- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8815b850 msgr2=0x7fba88156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8815b850 0x7fba88156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 -- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8806f1e0 msgr2=0x7fba88155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8806f1e0 0x7fba88155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 -- 172.21.15.181:0/3149834212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80009b30 con 0x7fba880665a0 2023-10-16T11:21:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8d64e700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8806f1e0 0x7fba88155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8e650700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8815b850 0x7fba88156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba8de4f700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba880665a0 0x7fba8814b290 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c450 tx=0x7fba7800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba7e7fc700 1 -- 172.21.15.181:0/3149834212 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780052d0 con 0x7fba880665a0 2023-10-16T11:21:05.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba7e7fc700 1 -- 172.21.15.181:0/3149834212 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba78005430 con 0x7fba880665a0 2023-10-16T11:21:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba7e7fc700 1 -- 172.21.15.181:0/3149834212 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780038c0 con 0x7fba880665a0 2023-10-16T11:21:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba880b5020 con 0x7fba880665a0 2023-10-16T11:21:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.657+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8814b7d0 con 0x7fba880665a0 2023-10-16T11:21:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.658+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba880b5a00 con 0x7fba880665a0 2023-10-16T11:21:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.659+0000 7fba7e7fc700 1 -- 172.21.15.181:0/3149834212 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fba78003a20 con 0x7fba880665a0 2023-10-16T11:21:05.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.659+0000 7fba7e7fc700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fba6c06b170 0x7fba6c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.659+0000 7fba7e7fc700 1 -- 172.21.15.181:0/3149834212 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7fba780890e0 con 0x7fba880665a0 2023-10-16T11:21:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.659+0000 7fba8d64e700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fba6c06b170 0x7fba6c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.660+0000 7fba8d64e700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fba6c06b170 0x7fba6c06d630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fba8806bf50 tx=0x7fba84008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:05.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.664+0000 7fba7e7fc700 1 -- 172.21.15.181:0/3149834212 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fba78053910 con 0x7fba880665a0 2023-10-16T11:21:05.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.853+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fba88160420 con 0x7fba880665a0 2023-10-16T11:21:05.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.854+0000 7fba7e7fc700 1 -- 172.21.15.181:0/3149834212 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v245) v1 ==== 103+0+11 (secure 0 0 0) 0x7fba88160420 con 0x7fba880665a0 2023-10-16T11:21:05.856 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:21:05.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fba6c06b170 msgr2=0x7fba6c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fba6c06b170 0x7fba6c06d630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fba8806bf50 tx=0x7fba84008040 comp rx=0 tx=0).stop 2023-10-16T11:21:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba880665a0 msgr2=0x7fba8814b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba880665a0 0x7fba8814b290 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c450 tx=0x7fba7800c810 comp rx=0 tx=0).stop 2023-10-16T11:21:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 shutdown_connections 2023-10-16T11:21:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba8815b850 0x7fba88156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fba6c06b170 0x7fba6c06d630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba880665a0 0x7fba8814b290 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 --2- 172.21.15.181:0/3149834212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba8806f1e0 0x7fba88155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 >> 172.21.15.181:0/3149834212 conn(0x7fba880cc730 msgr2=0x7fba880c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 shutdown_connections 2023-10-16T11:21:05.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.856+0000 7fba900b3700 1 -- 172.21.15.181:0/3149834212 wait complete. 2023-10-16T11:21:05.870 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:05.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.989+0000 7f8ac82db700 1 Processor -- start 2023-10-16T11:21:05.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.989+0000 7f8ac82db700 1 -- start start 2023-10-16T11:21:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac82db700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8ac015e0e0 0x7f8ac01574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac82db700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ac0157a30 0x7f8ac00c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac82db700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac0157ec0 con 0x7f8ac015dd00 2023-10-16T11:21:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac82db700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac0071df0 con 0x7f8ac0157a30 2023-10-16T11:21:05.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac82db700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac00720d0 con 0x7f8ac015e0e0 2023-10-16T11:21:05.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac6077700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8ac015dd00 0x7f8ac0071330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42208/0 (socket says 172.21.15.181:42208) 2023-10-16T11:21:05.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac6077700 1 -- 172.21.15.181:0/247221890 learned_addr learned my addr 172.21.15.181:0/247221890 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:05.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac6878700 1 --2- 172.21.15.181:0/247221890 >> v2:172.21.15.181:3300/0 conn(0x7f8ac0157a30 0x7f8ac00c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac5075700 1 -- 172.21.15.181:0/247221890 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536938074 0 0) 0x7f8ac0157ec0 con 0x7f8ac015dd00 2023-10-16T11:21:05.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac5876700 1 --2- 172.21.15.181:0/247221890 >> v2:172.21.15.190:3300/0 conn(0x7f8ac015e0e0 0x7f8ac01574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:05.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac5075700 1 -- 172.21.15.181:0/247221890 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa80035b0 con 0x7f8ac015dd00 2023-10-16T11:21:05.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac6878700 1 -- 172.21.15.181:0/247221890 >> v2:172.21.15.190:3300/0 conn(0x7f8ac015e0e0 msgr2=0x7f8ac01574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:05.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac6878700 1 --2- 172.21.15.181:0/247221890 >> v2:172.21.15.190:3300/0 conn(0x7f8ac015e0e0 0x7f8ac01574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac6878700 1 -- 172.21.15.181:0/247221890 >> v1:172.21.15.181:6789/0 conn(0x7f8ac015dd00 legacy=0x7f8ac0071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac6878700 1 -- 172.21.15.181:0/247221890 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00723b0 con 0x7f8ac0157a30 2023-10-16T11:21:05.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.990+0000 7f8ac5876700 1 --2- 172.21.15.181:0/247221890 >> v2:172.21.15.190:3300/0 conn(0x7f8ac015e0e0 0x7f8ac01574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:05.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac6878700 1 --2- 172.21.15.181:0/247221890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac0157a30 0x7f8ac00c9b30 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f8ac006ebd0 tx=0x7f8ab000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e49e66e5248f13a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:05.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac5075700 1 -- 172.21.15.181:0/247221890 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000e070 con 0x7f8ac0157a30 2023-10-16T11:21:05.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac5075700 1 -- 172.21.15.181:0/247221890 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab000bb20 con 0x7f8ac0157a30 2023-10-16T11:21:05.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac5075700 1 -- 172.21.15.181:0/247221890 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0012430 con 0x7f8ac0157a30 2023-10-16T11:21:05.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 -- 172.21.15.181:0/247221890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac0157a30 msgr2=0x7f8ac00c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:05.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 --2- 172.21.15.181:0/247221890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac0157a30 0x7f8ac00c9b30 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f8ac006ebd0 tx=0x7f8ab000b330 comp rx=0 tx=0).stop 2023-10-16T11:21:05.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 -- 172.21.15.181:0/247221890 shutdown_connections 2023-10-16T11:21:05.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 --2- 172.21.15.181:0/247221890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac0157a30 0x7f8ac00c9b30 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 --2- 172.21.15.181:0/247221890 >> v2:172.21.15.190:3300/0 conn(0x7f8ac015e0e0 0x7f8ac01574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:05.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 -- 172.21.15.181:0/247221890 >> 172.21.15.181:0/247221890 conn(0x7f8ac006aed0 msgr2=0x7f8ac00b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:05.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 -- 172.21.15.181:0/247221890 shutdown_connections 2023-10-16T11:21:05.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.991+0000 7f8ac82db700 1 -- 172.21.15.181:0/247221890 wait complete. 2023-10-16T11:21:06.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 Processor -- start 2023-10-16T11:21:06.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 -- start start 2023-10-16T11:21:06.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac0157a30 0x7f8ac0150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac015dd00 0x7f8ac00c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac015e0e0 0x7f8ac014fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0150c60 con 0x7f8ac015e0e0 2023-10-16T11:21:06.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00c7b20 con 0x7f8ac0157a30 2023-10-16T11:21:06.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac82db700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00c7e00 con 0x7f8ac015dd00 2023-10-16T11:21:06.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac6878700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac015e0e0 0x7f8ac014fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac6878700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac015e0e0 0x7f8ac014fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59372/0 (socket says 172.21.15.181:59372) 2023-10-16T11:21:06.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.993+0000 7f8ac6878700 1 -- 172.21.15.181:0/673199908 learned_addr learned my addr 172.21.15.181:0/673199908 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:06.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6077700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac0157a30 0x7f8ac0150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac5876700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac015dd00 0x7f8ac00c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6878700 1 -- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac015dd00 msgr2=0x7f8ac00c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6878700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac015dd00 0x7f8ac00c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6878700 1 -- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac0157a30 msgr2=0x7f8ac0150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6878700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac0157a30 0x7f8ac0150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6878700 1 -- 172.21.15.181:0/673199908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0009a70 con 0x7f8ac015e0e0 2023-10-16T11:21:06.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6077700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac0157a30 0x7f8ac0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:06.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac6878700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac015e0e0 0x7f8ac014fb50 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0014040 tx=0x7f8ab0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8abeffd700 1 -- 172.21.15.181:0/673199908 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0012430 con 0x7f8ac015e0e0 2023-10-16T11:21:06.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8abeffd700 1 -- 172.21.15.181:0/673199908 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab0020070 con 0x7f8ac015e0e0 2023-10-16T11:21:06.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac00c80e0 con 0x7f8ac015e0e0 2023-10-16T11:21:06.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8abeffd700 1 -- 172.21.15.181:0/673199908 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0004280 con 0x7f8ac015e0e0 2023-10-16T11:21:06.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.994+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac00c86a0 con 0x7f8ac015e0e0 2023-10-16T11:21:06.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.995+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac004ebb0 con 0x7f8ac015e0e0 2023-10-16T11:21:06.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.995+0000 7f8abeffd700 1 -- 172.21.15.181:0/673199908 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8ab00043e0 con 0x7f8ac015e0e0 2023-10-16T11:21:06.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.996+0000 7f8abeffd700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f8aac06b140 0x7f8aac06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.996+0000 7f8ac6077700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f8aac06b140 0x7f8aac06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.996+0000 7f8abeffd700 1 -- 172.21.15.181:0/673199908 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f8ab0022070 con 0x7f8ac015e0e0 2023-10-16T11:21:06.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:05.996+0000 7f8ac6077700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f8aac06b140 0x7f8aac06d600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80011e0 tx=0x7f8ab8009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.001+0000 7f8abeffd700 1 -- 172.21.15.181:0/673199908 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8ab0056870 con 0x7f8ac015e0e0 2023-10-16T11:21:06.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.238+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ac015b290 con 0x7f8ac015e0e0 2023-10-16T11:21:06.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.238+0000 7f8abeffd700 1 -- 172.21.15.181:0/673199908 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4942 (secure 0 0 0) 0x7f8ab0059e90 con 0x7f8ac015e0e0 2023-10-16T11:21:06.241 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:06.241 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":135,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":12747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2505407408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2505407408},{"type":"v1","addr":"172.21.15.181:6836","nonce":2505407408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:21:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f8aac06b140 msgr2=0x7f8aac06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f8aac06b140 0x7f8aac06d600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80011e0 tx=0x7f8ab8009040 comp rx=0 tx=0).stop 2023-10-16T11:21:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac015e0e0 msgr2=0x7f8ac014fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac015e0e0 0x7f8ac014fb50 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0014040 tx=0x7f8ab0004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 shutdown_connections 2023-10-16T11:21:06.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f8aac06b140 0x7f8aac06d600 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ac015e0e0 0x7f8ac014fb50 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac0157a30 0x7f8ac0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 --2- 172.21.15.181:0/673199908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac015dd00 0x7f8ac00c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.240+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 >> 172.21.15.181:0/673199908 conn(0x7f8ac006aed0 msgr2=0x7f8ac0151140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:06.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.241+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 shutdown_connections 2023-10-16T11:21:06.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.241+0000 7f8ac82db700 1 -- 172.21.15.181:0/673199908 wait complete. 2023-10-16T11:21:06.245 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 135 2023-10-16T11:21:06.255 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:06.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.373+0000 7f0d3db3c700 1 Processor -- start 2023-10-16T11:21:06.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.374+0000 7f0d3db3c700 1 -- start start 2023-10-16T11:21:06.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.374+0000 7f0d3db3c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815a680 0x7f0d380cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.374+0000 7f0d3db3c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d380d0c80 0x7f0d380ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.374+0000 7f0d3db3c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d380cb3b0 con 0x7f0d3815aa60 2023-10-16T11:21:06.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.374+0000 7f0d3db3c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380cbd50 con 0x7f0d380d0c80 2023-10-16T11:21:06.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.374+0000 7f0d3db3c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380cb690 con 0x7f0d3815a680 2023-10-16T11:21:06.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d36ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d380d0c80 0x7f0d380ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d377fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815a680 0x7f0d380cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d36ffd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d380d0c80 0x7f0d380ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:36196/0 (socket says 172.21.15.181:36196) 2023-10-16T11:21:06.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d36ffd700 1 -- 172.21.15.181:0/622486536 learned_addr learned my addr 172.21.15.181:0/622486536 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:06.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d377fe700 1 -- 172.21.15.181:0/622486536 >> v2:172.21.15.190:3300/0 conn(0x7f0d380d0c80 msgr2=0x7f0d380ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:06.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d377fe700 1 --2- 172.21.15.181:0/622486536 >> v2:172.21.15.190:3300/0 conn(0x7f0d380d0c80 0x7f0d380ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d377fe700 1 -- 172.21.15.181:0/622486536 >> v1:172.21.15.190:6789/0 conn(0x7f0d3815aa60 legacy=0x7f0d3815e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d377fe700 1 -- 172.21.15.181:0/622486536 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d380cb970 con 0x7f0d3815a680 2023-10-16T11:21:06.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d36ffd700 1 --2- 172.21.15.181:0/622486536 >> v2:172.21.15.190:3300/0 conn(0x7f0d380d0c80 0x7f0d380ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:06.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d377fe700 1 --2- 172.21.15.181:0/622486536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815a680 0x7f0d380cecc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009820 tx=0x7f0d2c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3931ce4d072aec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d367fc700 1 -- 172.21.15.181:0/622486536 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c005e30 con 0x7f0d3815a680 2023-10-16T11:21:06.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d367fc700 1 -- 172.21.15.181:0/622486536 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c006030 con 0x7f0d3815a680 2023-10-16T11:21:06.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.375+0000 7f0d367fc700 1 -- 172.21.15.181:0/622486536 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0063c0 con 0x7f0d3815a680 2023-10-16T11:21:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 -- 172.21.15.181:0/622486536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815a680 msgr2=0x7f0d380cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/622486536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815a680 0x7f0d380cecc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009820 tx=0x7f0d2c009b30 comp rx=0 tx=0).stop 2023-10-16T11:21:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 -- 172.21.15.181:0/622486536 shutdown_connections 2023-10-16T11:21:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/622486536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815a680 0x7f0d380cecc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/622486536 >> v2:172.21.15.190:3300/0 conn(0x7f0d380d0c80 0x7f0d380ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 -- 172.21.15.181:0/622486536 >> 172.21.15.181:0/622486536 conn(0x7f0d38099a00 msgr2=0x7f0d380d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 -- 172.21.15.181:0/622486536 shutdown_connections 2023-10-16T11:21:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.376+0000 7f0d3db3c700 1 -- 172.21.15.181:0/622486536 wait complete. 2023-10-16T11:21:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.377+0000 7f0d3db3c700 1 Processor -- start 2023-10-16T11:21:06.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.377+0000 7f0d3db3c700 1 -- start start 2023-10-16T11:21:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d3db3c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d380d0c80 0x7f0d38152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d3db3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3815a680 0x7f0d3806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d3db3c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815aa60 0x7f0d3806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d3db3c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3815bfd0 con 0x7f0d3815a680 2023-10-16T11:21:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d3db3c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3815c2b0 con 0x7f0d380d0c80 2023-10-16T11:21:06.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d3db3c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3815c590 con 0x7f0d3815aa60 2023-10-16T11:21:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3815a680 0x7f0d3806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d36ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3815a680 0x7f0d3806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59384/0 (socket says 172.21.15.181:59384) 2023-10-16T11:21:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d36ffd700 1 -- 172.21.15.181:0/2234020655 learned_addr learned my addr 172.21.15.181:0/2234020655 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d37fff700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815aa60 0x7f0d3806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.378+0000 7f0d377fe700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d380d0c80 0x7f0d38152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d36ffd700 1 -- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815aa60 msgr2=0x7f0d3806ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d36ffd700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815aa60 0x7f0d3806ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d36ffd700 1 -- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d380d0c80 msgr2=0x7f0d38152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d36ffd700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d380d0c80 0x7f0d38152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d36ffd700 1 -- 172.21.15.181:0/2234020655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009550 con 0x7f0d3815a680 2023-10-16T11:21:06.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d36ffd700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3815a680 0x7f0d3806a8a0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e980 tx=0x7f0d2800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d1ffff700 1 -- 172.21.15.181:0/2234020655 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800c850 con 0x7f0d3815a680 2023-10-16T11:21:06.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d1ffff700 1 -- 172.21.15.181:0/2234020655 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2800c9b0 con 0x7f0d3815a680 2023-10-16T11:21:06.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c009430 con 0x7f0d3815a680 2023-10-16T11:21:06.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d1ffff700 1 -- 172.21.15.181:0/2234020655 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800cd40 con 0x7f0d3815a680 2023-10-16T11:21:06.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.379+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3815cb50 con 0x7f0d3815a680 2023-10-16T11:21:06.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.380+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3815cd10 con 0x7f0d3815a680 2023-10-16T11:21:06.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.380+0000 7f0d1ffff700 1 -- 172.21.15.181:0/2234020655 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0d28013470 con 0x7f0d3815a680 2023-10-16T11:21:06.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.381+0000 7f0d1ffff700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0d2406b140 0x7f0d2406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.381+0000 7f0d377fe700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0d2406b140 0x7f0d2406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.381+0000 7f0d1ffff700 1 -- 172.21.15.181:0/2234020655 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f0d2808a0f0 con 0x7f0d3815a680 2023-10-16T11:21:06.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.381+0000 7f0d377fe700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0d2406b140 0x7f0d2406d600 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c008070 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.385+0000 7f0d1ffff700 1 -- 172.21.15.181:0/2234020655 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0d28054850 con 0x7f0d3815a680 2023-10-16T11:21:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.577+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d3815dd80 con 0x7f0d3815a680 2023-10-16T11:21:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.578+0000 7f0d1ffff700 1 -- 172.21.15.181:0/2234020655 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v245) v1 ==== 74+0+18825 (secure 0 0 0) 0x7f0d28057e70 con 0x7f0d3815a680 2023-10-16T11:21:06.581 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:06.581 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":245,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:04.461998+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:6834/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:0/4174526666":"2023-10-17T11:20:58.912264+0000","172.21.15.181:6832/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2905113763":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1596741027":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3516831361":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2396727780":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2869414246":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3445652558":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3765547571":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0d2406b140 msgr2=0x7f0d2406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0d2406b140 0x7f0d2406d600 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c008070 comp rx=0 tx=0).stop 2023-10-16T11:21:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3815a680 msgr2=0x7f0d3806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3815a680 0x7f0d3806a8a0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800e980 tx=0x7f0d2800ed40 comp rx=0 tx=0).stop 2023-10-16T11:21:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 shutdown_connections 2023-10-16T11:21:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d3815aa60 0x7f0d3806ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0d2406b140 0x7f0d2406d600 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.580+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d380d0c80 0x7f0d38152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.581+0000 7f0d3db3c700 1 --2- 172.21.15.181:0/2234020655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3815a680 0x7f0d3806a8a0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.581+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 >> 172.21.15.181:0/2234020655 conn(0x7f0d38099a00 msgr2=0x7f0d381598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:06.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.581+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 shutdown_connections 2023-10-16T11:21:06.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.581+0000 7f0d3db3c700 1 -- 172.21.15.181:0/2234020655 wait complete. 2023-10-16T11:21:06.595 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-16T11:21:06.595 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:06.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.713+0000 7f0de87bc700 1 Processor -- start 2023-10-16T11:21:06.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.713+0000 7f0de87bc700 1 -- start start 2023-10-16T11:21:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de87bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0de015eab0 0x7f0de00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de87bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de87bc700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de006ed00 con 0x7f0de015e6d0 2023-10-16T11:21:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de87bc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de00d02d0 con 0x7f0de015eab0 2023-10-16T11:21:06.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de87bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de015ad40 con 0x7f0de00cb0c0 2023-10-16T11:21:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de5d57700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0de015eab0 0x7f0de00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de5d57700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0de015eab0 0x7f0de00d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59392/0 (socket says 172.21.15.181:59392) 2023-10-16T11:21:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de5d57700 1 -- 172.21.15.181:0/3123378006 learned_addr learned my addr 172.21.15.181:0/3123378006 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:06.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de5d57700 1 -- 172.21.15.181:0/3123378006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 msgr2=0x7f0de015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:06.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de5d57700 1 --2- 172.21.15.181:0/3123378006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de5d57700 1 -- 172.21.15.181:0/3123378006 >> v1:172.21.15.190:6789/0 conn(0x7f0de015e6d0 legacy=0x7f0de015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:06.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.714+0000 7f0de5d57700 1 -- 172.21.15.181:0/3123378006 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de006e0f0 con 0x7f0de015eab0 2023-10-16T11:21:06.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de5d57700 1 --2- 172.21.15.181:0/3123378006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de00d0fc0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00bb90 tx=0x7f0ddc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6756c5a2c638fab7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de5556700 1 -- 172.21.15.181:0/3123378006 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00dd10 con 0x7f0de015eab0 2023-10-16T11:21:06.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de5556700 1 -- 172.21.15.181:0/3123378006 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc00de70 con 0x7f0de015eab0 2023-10-16T11:21:06.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de5556700 1 -- 172.21.15.181:0/3123378006 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc004230 con 0x7f0de015eab0 2023-10-16T11:21:06.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 -- 172.21.15.181:0/3123378006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 msgr2=0x7f0de00d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 --2- 172.21.15.181:0/3123378006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de00d0fc0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00bb90 tx=0x7f0ddc00bea0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 -- 172.21.15.181:0/3123378006 shutdown_connections 2023-10-16T11:21:06.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 --2- 172.21.15.181:0/3123378006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 --2- 172.21.15.181:0/3123378006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de00d0fc0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 -- 172.21.15.181:0/3123378006 >> 172.21.15.181:0/3123378006 conn(0x7f0de00d04a0 msgr2=0x7f0de00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:06.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 -- 172.21.15.181:0/3123378006 shutdown_connections 2023-10-16T11:21:06.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.715+0000 7f0de87bc700 1 -- 172.21.15.181:0/3123378006 wait complete. 2023-10-16T11:21:06.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 Processor -- start 2023-10-16T11:21:06.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 -- start start 2023-10-16T11:21:06.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0de015e6d0 0x7f0de00c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de014d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de00c3fb0 con 0x7f0de015eab0 2023-10-16T11:21:06.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de00c4570 con 0x7f0de015e6d0 2023-10-16T11:21:06.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de87bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de01568c0 con 0x7f0de00cb0c0 2023-10-16T11:21:06.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.717+0000 7f0de6d59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de014d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de014d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59398/0 (socket says 172.21.15.181:59398) 2023-10-16T11:21:06.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 -- 172.21.15.181:0/4256389719 learned_addr learned my addr 172.21.15.181:0/4256389719 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:06.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6558700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de5d57700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0de015e6d0 0x7f0de00c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 -- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 msgr2=0x7f0de015d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 -- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0de015e6d0 msgr2=0x7f0de00c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0de015e6d0 0x7f0de00c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 -- 172.21.15.181:0/4256389719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc00b7f0 con 0x7f0de015eab0 2023-10-16T11:21:06.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de5d57700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0de015e6d0 0x7f0de00c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:06.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6558700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:06.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de6d59700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de014d610 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800b970 tx=0x7f0dd800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0dd6ffd700 1 -- 172.21.15.181:0/4256389719 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800c9b0 con 0x7f0de015eab0 2023-10-16T11:21:06.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0dd6ffd700 1 -- 172.21.15.181:0/4256389719 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd800cb30 con 0x7f0de015eab0 2023-10-16T11:21:06.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de0156ba0 con 0x7f0de015eab0 2023-10-16T11:21:06.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0dd6ffd700 1 -- 172.21.15.181:0/4256389719 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8014440 con 0x7f0de015eab0 2023-10-16T11:21:06.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.718+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de015d790 con 0x7f0de015eab0 2023-10-16T11:21:06.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.719+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de00c46e0 con 0x7f0de015eab0 2023-10-16T11:21:06.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.719+0000 7f0dd6ffd700 1 -- 172.21.15.181:0/4256389719 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0dd801d3b0 con 0x7f0de015eab0 2023-10-16T11:21:06.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.720+0000 7f0dd6ffd700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0dc406b140 0x7f0dc406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:06.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.720+0000 7f0dd6ffd700 1 -- 172.21.15.181:0/4256389719 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f0dd8089d50 con 0x7f0de015eab0 2023-10-16T11:21:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.720+0000 7f0de6558700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0dc406b140 0x7f0dc406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.720+0000 7f0de6558700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0dc406b140 0x7f0dc406d600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000a8b0 tx=0x7f0dd0008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:06.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.724+0000 7f0dd6ffd700 1 -- 172.21.15.181:0/4256389719 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0dd808d030 con 0x7f0de015eab0 2023-10-16T11:21:06.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.958+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0de0067d20 con 0x7f0de015eab0 2023-10-16T11:21:06.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.959+0000 7f0dd6ffd700 1 -- 172.21.15.181:0/4256389719 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4942 (secure 0 0 0) 0x7f0dd8019020 con 0x7f0de015eab0 2023-10-16T11:21:06.961 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:06.961 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":135,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":12747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2505407408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2505407408},{"type":"v1","addr":"172.21.15.181:6836","nonce":2505407408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:21:06.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0dc406b140 msgr2=0x7f0dc406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0dc406b140 0x7f0dc406d600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000a8b0 tx=0x7f0dd0008040 comp rx=0 tx=0).stop 2023-10-16T11:21:06.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 msgr2=0x7f0de014d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:06.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de014d610 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800b970 tx=0x7f0dd800bd30 comp rx=0 tx=0).stop 2023-10-16T11:21:06.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 shutdown_connections 2023-10-16T11:21:06.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0de00cb0c0 0x7f0de015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f0dc406b140 0x7f0dc406d600 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0de015eab0 0x7f0de014d610 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 --2- 172.21.15.181:0/4256389719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0de015e6d0 0x7f0de00c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:06.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 >> 172.21.15.181:0/4256389719 conn(0x7f0de00d04a0 msgr2=0x7f0de006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:06.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 shutdown_connections 2023-10-16T11:21:06.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:06.961+0000 7f0de87bc700 1 -- 172.21.15.181:0/4256389719 wait complete. 2023-10-16T11:21:06.966 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 135 2023-10-16T11:21:06.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:07.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.094+0000 7ff0d90aa700 1 Processor -- start 2023-10-16T11:21:07.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.094+0000 7ff0d90aa700 1 -- start start 2023-10-16T11:21:07.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d90aa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0d40ce7f0 0x7ff0d406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d90aa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d90aa700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d40cc470 con 0x7ff0d40d0180 2023-10-16T11:21:07.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d90aa700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d40d0610 con 0x7ff0d40ce7f0 2023-10-16T11:21:07.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d90aa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d406ca30 con 0x7ff0d40d2390 2023-10-16T11:21:07.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d2d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0d40ce7f0 0x7ff0d406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d2d9d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0d40ce7f0 0x7ff0d406c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:36222/0 (socket says 172.21.15.181:36222) 2023-10-16T11:21:07.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.095+0000 7ff0d259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:56432/0 (socket says 172.21.15.181:56432) 2023-10-16T11:21:07.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d2d9d700 1 -- 172.21.15.181:0/1116182307 learned_addr learned my addr 172.21.15.181:0/1116182307 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:07.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d259c700 1 -- 172.21.15.181:0/1116182307 >> v2:172.21.15.190:3300/0 conn(0x7ff0d40ce7f0 msgr2=0x7ff0d406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:07.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d259c700 1 --2- 172.21.15.181:0/1116182307 >> v2:172.21.15.190:3300/0 conn(0x7ff0d40ce7f0 0x7ff0d406c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d259c700 1 -- 172.21.15.181:0/1116182307 >> v1:172.21.15.190:6789/0 conn(0x7ff0d40d0180 legacy=0x7ff0d4161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d259c700 1 -- 172.21.15.181:0/1116182307 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d406cd10 con 0x7ff0d40d2390 2023-10-16T11:21:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d2d9d700 1 --2- 172.21.15.181:0/1116182307 >> v2:172.21.15.190:3300/0 conn(0x7ff0d40ce7f0 0x7ff0d406c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d259c700 1 --2- 172.21.15.181:0/1116182307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40cfc40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009840 tx=0x7ff0c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315c7cdc18a16186 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d1d9b700 1 -- 172.21.15.181:0/1116182307 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011830 con 0x7ff0d40d2390 2023-10-16T11:21:07.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d1d9b700 1 -- 172.21.15.181:0/1116182307 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4011990 con 0x7ff0d40d2390 2023-10-16T11:21:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.096+0000 7ff0d1d9b700 1 -- 172.21.15.181:0/1116182307 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011d20 con 0x7ff0d40d2390 2023-10-16T11:21:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 -- 172.21.15.181:0/1116182307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 msgr2=0x7ff0d40cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/1116182307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40cfc40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009840 tx=0x7ff0c4009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 -- 172.21.15.181:0/1116182307 shutdown_connections 2023-10-16T11:21:07.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/1116182307 >> v2:172.21.15.190:3300/0 conn(0x7ff0d40ce7f0 0x7ff0d406c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/1116182307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40cfc40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 -- 172.21.15.181:0/1116182307 >> 172.21.15.181:0/1116182307 conn(0x7ff0d409a9a0 msgr2=0x7ff0d40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 -- 172.21.15.181:0/1116182307 shutdown_connections 2023-10-16T11:21:07.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.097+0000 7ff0d90aa700 1 -- 172.21.15.181:0/1116182307 wait complete. 2023-10-16T11:21:07.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.098+0000 7ff0d90aa700 1 Processor -- start 2023-10-16T11:21:07.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.098+0000 7ff0d90aa700 1 -- start start 2023-10-16T11:21:07.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d90aa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40ce7f0 0x7ff0d409b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d90aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40d0180 0x7ff0d409b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d90aa700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d90aa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d415e1a0 con 0x7ff0d40d0180 2023-10-16T11:21:07.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d90aa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4151480 con 0x7ff0d40ce7f0 2023-10-16T11:21:07.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d90aa700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4151760 con 0x7ff0d40d2390 2023-10-16T11:21:07.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40d0180 0x7ff0d409b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40d0180 0x7ff0d409b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59412/0 (socket says 172.21.15.181:59412) 2023-10-16T11:21:07.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 -- 172.21.15.181:0/2625852099 learned_addr learned my addr 172.21.15.181:0/2625852099 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:07.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d2d9d700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40ce7f0 0x7ff0d409b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d359e700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 -- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 msgr2=0x7ff0d40b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 -- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40ce7f0 msgr2=0x7ff0d409b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40ce7f0 0x7ff0d409b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d259c700 1 -- 172.21.15.181:0/2625852099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8009500 con 0x7ff0d40d0180 2023-10-16T11:21:07.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.099+0000 7ff0d2d9d700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40ce7f0 0x7ff0d409b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:07.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.100+0000 7ff0d259c700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40d0180 0x7ff0d409b850 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009fd0 tx=0x7ff0c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.100+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2625852099 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40036a0 con 0x7ff0d40d0180 2023-10-16T11:21:07.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.100+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2625852099 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c401f070 con 0x7ff0d40d0180 2023-10-16T11:21:07.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.100+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c40094a0 con 0x7ff0d40d0180 2023-10-16T11:21:07.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.100+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2625852099 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4003a60 con 0x7ff0d40d0180 2023-10-16T11:21:07.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.100+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d409d780 con 0x7ff0d40d0180 2023-10-16T11:21:07.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.101+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d409d300 con 0x7ff0d40d0180 2023-10-16T11:21:07.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.101+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2625852099 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff0c4003bc0 con 0x7ff0d40d0180 2023-10-16T11:21:07.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.101+0000 7ff0bb7fe700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff0c006b140 0x7ff0c006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.102+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2625852099 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7ff0c4021070 con 0x7ff0d40d0180 2023-10-16T11:21:07.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.102+0000 7ff0d2d9d700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff0c006b140 0x7ff0c006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.102+0000 7ff0d2d9d700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff0c006b140 0x7ff0c006d600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40b4060 tx=0x7ff0c8008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.106+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2625852099 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff0d409d300 con 0x7ff0d40d0180 2023-10-16T11:21:07.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.344+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0d409d300 con 0x7ff0d40d0180 2023-10-16T11:21:07.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.344+0000 7ff0bb7fe700 1 -- 172.21.15.181:0/2625852099 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4942 (secure 0 0 0) 0x7ff0d409d300 con 0x7ff0d40d0180 2023-10-16T11:21:07.347 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:07.347 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":135,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":12747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2505407408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2505407408},{"type":"v1","addr":"172.21.15.181:6836","nonce":2505407408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:20:38.533416+0000","modified":"2023-10-16T11:20:41.554525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10690":{"gid":10690,"name":"b","rank":0,"incarnation":130,"state":"up:active","state_seq":6,"addr":"172.21.15.190:6834/1302040487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1302040487},{"type":"v1","addr":"172.21.15.190:6834","nonce":1302040487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-16T11:21:07.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff0c006b140 msgr2=0x7ff0c006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff0c006b140 0x7ff0c006d600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40b4060 tx=0x7ff0c8008040 comp rx=0 tx=0).stop 2023-10-16T11:21:07.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40d0180 msgr2=0x7ff0d409b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40d0180 0x7ff0d409b850 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009fd0 tx=0x7ff0c4004750 comp rx=0 tx=0).stop 2023-10-16T11:21:07.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 shutdown_connections 2023-10-16T11:21:07.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0d40d2390 0x7ff0d40b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff0c006b140 0x7ff0c006d600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0d40ce7f0 0x7ff0d409b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 --2- 172.21.15.181:0/2625852099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0d40d0180 0x7ff0d409b850 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 >> 172.21.15.181:0/2625852099 conn(0x7ff0d409a9a0 msgr2=0x7ff0d40c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:07.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 shutdown_connections 2023-10-16T11:21:07.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.347+0000 7ff0d90aa700 1 -- 172.21.15.181:0/2625852099 wait complete. 2023-10-16T11:21:07.351 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 135 2023-10-16T11:21:07.361 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:07.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.479+0000 7f4c2800f700 1 Processor -- start 2023-10-16T11:21:07.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.479+0000 7f4c2800f700 1 -- start start 2023-10-16T11:21:07.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c2800f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c2006d5f0 0x7f4c200d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c2800f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c20160d20 0x7f4c201611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c2800f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c201616f0 con 0x7f4c20161b60 2023-10-16T11:21:07.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c2800f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c20162800 con 0x7f4c20160d20 2023-10-16T11:21:07.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c2800f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200d43e0 con 0x7f4c2006d5f0 2023-10-16T11:21:07.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c20160d20 0x7f4c201611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c20160d20 0x7f4c201611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59422/0 (socket says 172.21.15.181:59422) 2023-10-16T11:21:07.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 -- 172.21.15.181:0/304372552 learned_addr learned my addr 172.21.15.181:0/304372552 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:07.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 -- 172.21.15.181:0/304372552 >> v2:172.21.15.190:3300/0 conn(0x7f4c2006d5f0 msgr2=0x7f4c200d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:07.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c255aa700 1 --2- 172.21.15.181:0/304372552 >> v2:172.21.15.190:3300/0 conn(0x7f4c2006d5f0 0x7f4c200d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 --2- 172.21.15.181:0/304372552 >> v2:172.21.15.190:3300/0 conn(0x7f4c2006d5f0 0x7f4c200d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 -- 172.21.15.181:0/304372552 >> v1:172.21.15.190:6789/0 conn(0x7f4c20161b60 legacy=0x7f4c20161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:07.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 -- 172.21.15.181:0/304372552 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200d2560 con 0x7f4c20160d20 2023-10-16T11:21:07.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c255aa700 1 --2- 172.21.15.181:0/304372552 >> v2:172.21.15.190:3300/0 conn(0x7f4c2006d5f0 0x7f4c200d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:07.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c265ac700 1 --2- 172.21.15.181:0/304372552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c201611b0 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f4c2008ddd0 tx=0x7f4c1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c06bec5ad92e60c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c24da9700 1 -- 172.21.15.181:0/304372552 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800e070 con 0x7f4c20160d20 2023-10-16T11:21:07.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c24da9700 1 -- 172.21.15.181:0/304372552 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c18003a20 con 0x7f4c20160d20 2023-10-16T11:21:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.480+0000 7f4c24da9700 1 -- 172.21.15.181:0/304372552 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18003dd0 con 0x7f4c20160d20 2023-10-16T11:21:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 -- 172.21.15.181:0/304372552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 msgr2=0x7f4c201611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 --2- 172.21.15.181:0/304372552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c201611b0 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f4c2008ddd0 tx=0x7f4c1800b330 comp rx=0 tx=0).stop 2023-10-16T11:21:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 -- 172.21.15.181:0/304372552 shutdown_connections 2023-10-16T11:21:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 --2- 172.21.15.181:0/304372552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c201611b0 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 --2- 172.21.15.181:0/304372552 >> v2:172.21.15.190:3300/0 conn(0x7f4c2006d5f0 0x7f4c200d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 -- 172.21.15.181:0/304372552 >> 172.21.15.181:0/304372552 conn(0x7f4c2009cba0 msgr2=0x7f4c200d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 -- 172.21.15.181:0/304372552 shutdown_connections 2023-10-16T11:21:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.481+0000 7f4c2800f700 1 -- 172.21.15.181:0/304372552 wait complete. 2023-10-16T11:21:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 Processor -- start 2023-10-16T11:21:07.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 -- start start 2023-10-16T11:21:07.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2006d5f0 0x7f4c200cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c2015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c20161b60 0x7f4c2009e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200cc220 con 0x7f4c20160d20 2023-10-16T11:21:07.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200cc500 con 0x7f4c20161b60 2023-10-16T11:21:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c2800f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200cc7e0 con 0x7f4c2006d5f0 2023-10-16T11:21:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.483+0000 7f4c255aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c2015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c2015bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59426/0 (socket says 172.21.15.181:59426) 2023-10-16T11:21:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 -- 172.21.15.181:0/4149039882 learned_addr learned my addr 172.21.15.181:0/4149039882 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c265ac700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c20161b60 0x7f4c2009e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c25dab700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2006d5f0 0x7f4c200cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 -- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2006d5f0 msgr2=0x7f4c200cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2006d5f0 0x7f4c200cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 -- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c20161b60 msgr2=0x7f4c2009e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c20161b60 0x7f4c2009e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 -- 172.21.15.181:0/4149039882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c18009b30 con 0x7f4c20160d20 2023-10-16T11:21:07.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.484+0000 7f4c255aa700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c2015bdb0 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000c7f0 tx=0x7f4c1000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.488+0000 7f4c167fc700 1 -- 172.21.15.181:0/4149039882 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10011800 con 0x7f4c20160d20 2023-10-16T11:21:07.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.488+0000 7f4c167fc700 1 -- 172.21.15.181:0/4149039882 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c10011960 con 0x7f4c20160d20 2023-10-16T11:21:07.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.488+0000 7f4c167fc700 1 -- 172.21.15.181:0/4149039882 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10011cf0 con 0x7f4c20160d20 2023-10-16T11:21:07.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.488+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c200b6420 con 0x7f4c20160d20 2023-10-16T11:21:07.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.488+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c200cbe60 con 0x7f4c20160d20 2023-10-16T11:21:07.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.489+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c200b6a70 con 0x7f4c20160d20 2023-10-16T11:21:07.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.489+0000 7f4c167fc700 1 -- 172.21.15.181:0/4149039882 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4c10018360 con 0x7f4c20160d20 2023-10-16T11:21:07.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.490+0000 7f4c167fc700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f4c0406b140 0x7f4c0406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.490+0000 7f4c167fc700 1 -- 172.21.15.181:0/4149039882 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f4c10089ff0 con 0x7f4c20160d20 2023-10-16T11:21:07.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.490+0000 7f4c25dab700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f4c0406b140 0x7f4c0406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.490+0000 7f4c25dab700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f4c0406b140 0x7f4c0406d600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c006fd0 tx=0x7f4c1c008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.494+0000 7f4c167fc700 1 -- 172.21.15.181:0/4149039882 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c10054650 con 0x7f4c20160d20 2023-10-16T11:21:07.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.688+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c2004ebb0 con 0x7f4c20160d20 2023-10-16T11:21:07.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.689+0000 7f4c167fc700 1 -- 172.21.15.181:0/4149039882 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v245) v1 ==== 74+0+18825 (secure 0 0 0) 0x7f4c10057c70 con 0x7f4c20160d20 2023-10-16T11:21:07.692 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:07.692 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":245,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:04.461998+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"backup_fs_metadata","create_time":"2023-10-16T11:20:36.190463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_data","create_time":"2023-10-16T11:20:37.509605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"240","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:6834/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:0/4174526666":"2023-10-17T11:20:58.912264+0000","172.21.15.181:6832/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2905113763":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1596741027":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3516831361":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2396727780":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2869414246":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3445652558":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3765547571":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:07.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f4c0406b140 msgr2=0x7f4c0406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f4c0406b140 0x7f4c0406d600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c006fd0 tx=0x7f4c1c008040 comp rx=0 tx=0).stop 2023-10-16T11:21:07.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 msgr2=0x7f4c2015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c2015bdb0 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000c7f0 tx=0x7f4c1000cb00 comp rx=0 tx=0).stop 2023-10-16T11:21:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 shutdown_connections 2023-10-16T11:21:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c2006d5f0 0x7f4c200cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f4c0406b140 0x7f4c0406d600 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c20161b60 0x7f4c2009e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 --2- 172.21.15.181:0/4149039882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20160d20 0x7f4c2015bdb0 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 >> 172.21.15.181:0/4149039882 conn(0x7f4c2009cba0 msgr2=0x7f4c20161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 shutdown_connections 2023-10-16T11:21:07.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.692+0000 7f4c2800f700 1 -- 172.21.15.181:0/4149039882 wait complete. 2023-10-16T11:21:07.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-16T11:21:07.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.825+0000 7f87d4ead700 1 Processor -- start 2023-10-16T11:21:07.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.825+0000 7f87d4ead700 1 -- start start 2023-10-16T11:21:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87d4ead700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87d015df60 0x7f87d00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87d4ead700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 0x7f87d006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87d4ead700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d00cddf0 con 0x7f87d00d0fc0 2023-10-16T11:21:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87d4ead700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d006b290 con 0x7f87d015df60 2023-10-16T11:21:07.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87d4ead700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00af9c0 con 0x7f87d006a8a0 2023-10-16T11:21:07.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87ce59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87d015df60 0x7f87d00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87cdd9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f87d00d0fc0 0x7f87d00cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42278/0 (socket says 172.21.15.181:42278) 2023-10-16T11:21:07.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.826+0000 7f87cdd9b700 1 -- 172.21.15.181:0/3592752026 learned_addr learned my addr 172.21.15.181:0/3592752026 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:07.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87ce59c700 1 -- 172.21.15.181:0/3592752026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 msgr2=0x7f87d006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:07.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87ce59c700 1 --2- 172.21.15.181:0/3592752026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 0x7f87d006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87ce59c700 1 -- 172.21.15.181:0/3592752026 >> v1:172.21.15.181:6789/0 conn(0x7f87d00d0fc0 legacy=0x7f87d00cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87ce59c700 1 -- 172.21.15.181:0/3592752026 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00ccf50 con 0x7f87d015df60 2023-10-16T11:21:07.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87cd59a700 1 -- 172.21.15.181:0/3592752026 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377751041 0 0) 0x7f87d00cddf0 con 0x7f87d00d0fc0 2023-10-16T11:21:07.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87ce59c700 1 --2- 172.21.15.181:0/3592752026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d015df60 0x7f87d00cae20 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009850 tx=0x7f87c4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60da8bf1d5052bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87cd59a700 1 -- 172.21.15.181:0/3592752026 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4005e30 con 0x7f87d015df60 2023-10-16T11:21:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87cd59a700 1 -- 172.21.15.181:0/3592752026 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4006030 con 0x7f87d015df60 2023-10-16T11:21:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87cd59a700 1 -- 172.21.15.181:0/3592752026 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c40063c0 con 0x7f87d015df60 2023-10-16T11:21:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87d4ead700 1 -- 172.21.15.181:0/3592752026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d015df60 msgr2=0x7f87d00cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87d4ead700 1 --2- 172.21.15.181:0/3592752026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d015df60 0x7f87d00cae20 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009850 tx=0x7f87c4009b60 comp rx=0 tx=0).stop 2023-10-16T11:21:07.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87d4ead700 1 -- 172.21.15.181:0/3592752026 shutdown_connections 2023-10-16T11:21:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87d4ead700 1 --2- 172.21.15.181:0/3592752026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 0x7f87d006ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87d4ead700 1 --2- 172.21.15.181:0/3592752026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d015df60 0x7f87d00cae20 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.827+0000 7f87d4ead700 1 -- 172.21.15.181:0/3592752026 >> 172.21.15.181:0/3592752026 conn(0x7f87d00d04a0 msgr2=0x7f87d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.828+0000 7f87d4ead700 1 -- 172.21.15.181:0/3592752026 shutdown_connections 2023-10-16T11:21:07.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.828+0000 7f87d4ead700 1 -- 172.21.15.181:0/3592752026 wait complete. 2023-10-16T11:21:07.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.829+0000 7f87d4ead700 1 Processor -- start 2023-10-16T11:21:07.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.829+0000 7f87d4ead700 1 -- start start 2023-10-16T11:21:07.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87d4ead700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 0x7f87d0151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87d4ead700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00d0fc0 0x7f87d015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87d4ead700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d015df60 0x7f87d015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87d4ead700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00c9300 con 0x7f87d00d0fc0 2023-10-16T11:21:07.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87d4ead700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00c3f00 con 0x7f87d015df60 2023-10-16T11:21:07.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87d4ead700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d0152180 con 0x7f87d006a8a0 2023-10-16T11:21:07.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87cdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00d0fc0 0x7f87d015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87cdd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00d0fc0 0x7f87d015d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59438/0 (socket says 172.21.15.181:59438) 2023-10-16T11:21:07.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.830+0000 7f87cdd9b700 1 -- 172.21.15.181:0/4092615139 learned_addr learned my addr 172.21.15.181:0/4092615139 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:07.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87ce59c700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 0x7f87d0151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87ced9d700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d015df60 0x7f87d015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87cdd9b700 1 -- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 msgr2=0x7f87d0151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87cdd9b700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 0x7f87d0151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87cdd9b700 1 -- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d015df60 msgr2=0x7f87d015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:07.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87cdd9b700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d015df60 0x7f87d015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:07.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87cdd9b700 1 -- 172.21.15.181:0/4092615139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c4009430 con 0x7f87d00d0fc0 2023-10-16T11:21:07.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87cdd9b700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00d0fc0 0x7f87d015d230 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f87c00011e0 tx=0x7f87c000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87beffd700 1 -- 172.21.15.181:0/4092615139 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0004c80 con 0x7f87d00d0fc0 2023-10-16T11:21:07.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87beffd700 1 -- 172.21.15.181:0/4092615139 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0004e00 con 0x7f87d00d0fc0 2023-10-16T11:21:07.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d00b0fb0 con 0x7f87d00d0fc0 2023-10-16T11:21:07.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87beffd700 1 -- 172.21.15.181:0/4092615139 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0007310 con 0x7f87d00d0fc0 2023-10-16T11:21:07.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.831+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d00c79e0 con 0x7f87d00d0fc0 2023-10-16T11:21:07.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.832+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d0067d20 con 0x7f87d00d0fc0 2023-10-16T11:21:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.832+0000 7f87beffd700 1 -- 172.21.15.181:0/4092615139 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f87c0007470 con 0x7f87d00d0fc0 2023-10-16T11:21:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.833+0000 7f87beffd700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f87b006b140 0x7f87b006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.833+0000 7f87beffd700 1 -- 172.21.15.181:0/4092615139 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f87c0089d80 con 0x7f87d00d0fc0 2023-10-16T11:21:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.833+0000 7f87ce59c700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f87b006b140 0x7f87b006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:07.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.833+0000 7f87ce59c700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f87b006b140 0x7f87b006d600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87c4013040 tx=0x7f87c4008070 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:07.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:07.837+0000 7f87beffd700 1 -- 172.21.15.181:0/4092615139 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f87c0054570 con 0x7f87d00d0fc0 2023-10-16T11:21:08.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.073+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f87d00b1140 con 0x7f87d00d0fc0 2023-10-16T11:21:08.076 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:21:08.074+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:21:08.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.075+0000 7f87beffd700 1 -- 172.21.15.181:0/4092615139 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v136) v1 ==== 167+0+0 (secure 0 0 0) 0x7f87c0057b90 con 0x7f87d00d0fc0 2023-10-16T11:21:08.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f87b006b140 msgr2=0x7f87b006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:08.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f87b006b140 0x7f87b006d600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87c4013040 tx=0x7f87c4008070 comp rx=0 tx=0).stop 2023-10-16T11:21:08.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00d0fc0 msgr2=0x7f87d015d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00d0fc0 0x7f87d015d230 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f87c00011e0 tx=0x7f87c000cae0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 shutdown_connections 2023-10-16T11:21:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d006a8a0 0x7f87d0151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f87b006b140 0x7f87b006d600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d015df60 0x7f87d015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 --2- 172.21.15.181:0/4092615139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00d0fc0 0x7f87d015d230 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 >> 172.21.15.181:0/4092615139 conn(0x7f87d00d04a0 msgr2=0x7f87d0153b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 shutdown_connections 2023-10-16T11:21:08.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.079+0000 7f87d4ead700 1 -- 172.21.15.181:0/4092615139 wait complete. 2023-10-16T11:21:08.084 INFO:tasks.ceph.mds.b.smithi190.stderr: -1575> 2023-10-16T11:20:43.051+0000 7fb991f98700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:21:08.084 INFO:teuthology.orchestra.run.smithi181.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:21:08.093 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-16T11:21:08.109 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:21:08.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.214+0000 7fc19809c700 1 Processor -- start 2023-10-16T11:21:08.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.214+0000 7fc19809c700 1 -- start start 2023-10-16T11:21:08.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc19809c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc19015fb30 0x7fc19006f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:08.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc19809c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc190070590 con 0x7fc1900cb2b0 2023-10-16T11:21:08.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc19809c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc190156330 con 0x7fc1900cb690 2023-10-16T11:21:08.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc19809c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19006fcf0 con 0x7fc19015fb30 2023-10-16T11:21:08.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc195e38700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc1900cb2b0 0x7fc1900b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42290/0 (socket says 172.21.15.181:42290) 2023-10-16T11:21:08.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc195e38700 1 -- 172.21.15.181:0/3731153408 learned_addr learned my addr 172.21.15.181:0/3731153408 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:08.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc196639700 1 --2- 172.21.15.181:0/3731153408 >> v2:172.21.15.190:3300/0 conn(0x7fc19015fb30 0x7fc19006f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:08.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228815326 0 0) 0x7fc190070590 con 0x7fc1900cb2b0 2023-10-16T11:21:08.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc17c0035b0 con 0x7fc1900cb2b0 2023-10-16T11:21:08.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367036468 0 0) 0x7fc190156330 con 0x7fc1900cb690 2023-10-16T11:21:08.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc190070590 con 0x7fc1900cb690 2023-10-16T11:21:08.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc196639700 1 -- 172.21.15.181:0/3731153408 >> v1:172.21.15.190:6789/0 conn(0x7fc1900cb690 legacy=0x7fc19015f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:08.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc196639700 1 -- 172.21.15.181:0/3731153408 >> v1:172.21.15.181:6789/0 conn(0x7fc1900cb2b0 legacy=0x7fc1900b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:08.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857531224 0 0) 0x7fc17c0035b0 con 0x7fc1900cb2b0 2023-10-16T11:21:08.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.215+0000 7fc196639700 1 -- 172.21.15.181:0/3731153408 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19006ffd0 con 0x7fc19015fb30 2023-10-16T11:21:08.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc196639700 1 --2- 172.21.15.181:0/3731153408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc19015fb30 0x7fc19006f710 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d90 tx=0x7fc18800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caee93013e3361ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:08.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18800e070 con 0x7fc19015fb30 2023-10-16T11:21:08.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188003a20 con 0x7fc19015fb30 2023-10-16T11:21:08.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc194e36700 1 -- 172.21.15.181:0/3731153408 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c001670 con 0x7fc19015fb30 2023-10-16T11:21:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc19809c700 1 -- 172.21.15.181:0/3731153408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc19015fb30 msgr2=0x7fc19006f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc19809c700 1 --2- 172.21.15.181:0/3731153408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc19015fb30 0x7fc19006f710 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d90 tx=0x7fc18800b330 comp rx=0 tx=0).stop 2023-10-16T11:21:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc19809c700 1 -- 172.21.15.181:0/3731153408 shutdown_connections 2023-10-16T11:21:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc19809c700 1 --2- 172.21.15.181:0/3731153408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc19015fb30 0x7fc19006f710 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc19809c700 1 -- 172.21.15.181:0/3731153408 >> 172.21.15.181:0/3731153408 conn(0x7fc1900cd010 msgr2=0x7fc1900cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:08.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc19809c700 1 -- 172.21.15.181:0/3731153408 shutdown_connections 2023-10-16T11:21:08.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.216+0000 7fc19809c700 1 -- 172.21.15.181:0/3731153408 wait complete. 2023-10-16T11:21:08.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.217+0000 7fc19809c700 1 Processor -- start 2023-10-16T11:21:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc19809c700 1 -- start start 2023-10-16T11:21:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc19809c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1900cb2b0 0x7fc1901f1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc19809c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1900cb690 0x7fc1900c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc19809c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1900c9190 0x7fc1900c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:08.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc19809c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c4140 con 0x7fc1900c9190 2023-10-16T11:21:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc19809c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1901f16a0 con 0x7fc1900cb2b0 2023-10-16T11:21:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc19809c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c9b40 con 0x7fc1900cb690 2023-10-16T11:21:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc196639700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1900c9190 0x7fc1900c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc195e38700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1900cb2b0 0x7fc1901f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:08.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc196639700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1900c9190 0x7fc1900c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59454/0 (socket says 172.21.15.181:59454) 2023-10-16T11:21:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc195e38700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1900cb2b0 0x7fc1901f1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:36268/0 (socket says 172.21.15.181:36268) 2023-10-16T11:21:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc196639700 1 -- 172.21.15.181:0/1978371711 learned_addr learned my addr 172.21.15.181:0/1978371711 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.218+0000 7fc195637700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1900cb690 0x7fc1900c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc195e38700 1 -- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1900cb690 msgr2=0x7fc1900c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:08.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc195e38700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1900cb690 0x7fc1900c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc195e38700 1 -- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1900c9190 msgr2=0x7fc1900c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc195e38700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1900c9190 0x7fc1900c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc195e38700 1 -- 172.21.15.181:0/1978371711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188009a70 con 0x7fc1900cb2b0 2023-10-16T11:21:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc196639700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1900c9190 0x7fc1900c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:08.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc195e38700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1900cb2b0 0x7fc1901f1160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c520 tx=0x7fc18c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc1867fc700 1 -- 172.21.15.181:0/1978371711 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004c80 con 0x7fc1900cb2b0 2023-10-16T11:21:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc1867fc700 1 -- 172.21.15.181:0/1978371711 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c004e00 con 0x7fc1900cb2b0 2023-10-16T11:21:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1900c43e0 con 0x7fc1900cb2b0 2023-10-16T11:21:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc1867fc700 1 -- 172.21.15.181:0/1978371711 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c007310 con 0x7fc1900cb2b0 2023-10-16T11:21:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.219+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1900c49a0 con 0x7fc1900cb2b0 2023-10-16T11:21:08.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.220+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19006f4f0 con 0x7fc1900cb2b0 2023-10-16T11:21:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.221+0000 7fc1867fc700 1 -- 172.21.15.181:0/1978371711 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc18c007470 con 0x7fc1900cb2b0 2023-10-16T11:21:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.221+0000 7fc1867fc700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc17406b140 0x7fc17406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.221+0000 7fc1867fc700 1 -- 172.21.15.181:0/1978371711 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fc18c089c10 con 0x7fc1900cb2b0 2023-10-16T11:21:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.221+0000 7fc195637700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc17406b140 0x7fc17406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:08.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.222+0000 7fc195637700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc17406b140 0x7fc17406d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc1800011e0 tx=0x7fc180009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:08.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.226+0000 7fc1867fc700 1 -- 172.21.15.181:0/1978371711 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc18c0543b0 con 0x7fc1900cb2b0 2023-10-16T11:21:08.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:08.462+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc190066910 con 0x7fc1900cb2b0 2023-10-16T11:21:09.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.080+0000 7fc1867fc700 1 -- 172.21.15.181:0/1978371711 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v137) v1 ==== 107+0+0 (secure 0 0 0) 0x7fc18c0579d0 con 0x7fc1900cb2b0 2023-10-16T11:21:09.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc17406b140 msgr2=0x7fc17406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:09.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc17406b140 0x7fc17406d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc1800011e0 tx=0x7fc180009040 comp rx=0 tx=0).stop 2023-10-16T11:21:09.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1900cb2b0 msgr2=0x7fc1901f1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:09.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1900cb2b0 0x7fc1901f1160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c520 tx=0x7fc18c00c830 comp rx=0 tx=0).stop 2023-10-16T11:21:09.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 shutdown_connections 2023-10-16T11:21:09.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc17406b140 0x7fc17406d600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1900c9190 0x7fc1900c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1900cb2b0 0x7fc1901f1160 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 --2- 172.21.15.181:0/1978371711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1900cb690 0x7fc1900c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 >> 172.21.15.181:0/1978371711 conn(0x7fc1900cd010 msgr2=0x7fc19015cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:09.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.083+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 shutdown_connections 2023-10-16T11:21:09.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.084+0000 7fc19809c700 1 -- 172.21.15.181:0/1978371711 wait complete. 2023-10-16T11:21:09.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-16T11:21:09.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.228+0000 7fcae0d69700 1 Processor -- start 2023-10-16T11:21:09.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.228+0000 7fcae0d69700 1 -- start start 2023-10-16T11:21:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.229+0000 7fcae0d69700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcadc166b20 0x7fcadc168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.229+0000 7fcae0d69700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc06ce60 con 0x7fcadc1608e0 2023-10-16T11:21:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.229+0000 7fcae0d69700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc06d2f0 con 0x7fcadc160cc0 2023-10-16T11:21:09.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.229+0000 7fcae0d69700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcadc0d1020 con 0x7fcadc166b20 2023-10-16T11:21:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcada59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcadc1608e0 0x7fcadc0cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42304/0 (socket says 172.21.15.181:42304) 2023-10-16T11:21:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcada59c700 1 -- 172.21.15.181:0/1925356565 learned_addr learned my addr 172.21.15.181:0/1925356565 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcadad9d700 1 --2- 172.21.15.181:0/1925356565 >> v2:172.21.15.190:3300/0 conn(0x7fcadc166b20 0x7fcadc168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946624685 0 0) 0x7fcadc06ce60 con 0x7fcadc1608e0 2023-10-16T11:21:09.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac40035b0 con 0x7fcadc1608e0 2023-10-16T11:21:09.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14470319 0 0) 0x7fcac40035b0 con 0x7fcadc1608e0 2023-10-16T11:21:09.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 >> v2:172.21.15.190:3300/0 conn(0x7fcadc166b20 msgr2=0x7fcadc168f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:09.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 --2- 172.21.15.181:0/1925356565 >> v2:172.21.15.190:3300/0 conn(0x7fcadc166b20 0x7fcadc168f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 >> v1:172.21.15.190:6789/0 conn(0x7fcadc160cc0 legacy=0x7fcadc0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:09.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0b2f20 con 0x7fcadc1608e0 2023-10-16T11:21:09.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fcad0001670 con 0x7fcadc1608e0 2023-10-16T11:21:09.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 >> v1:172.21.15.181:6789/0 conn(0x7fcadc1608e0 legacy=0x7fcadc0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:09.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4003a00 0x7fcac4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcac4004850 0x7fcac4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac4005210 0x7fcac400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac400c080 con 0x7fcac4005210 2023-10-16T11:21:09.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac400c1f0 con 0x7fcac4004850 2023-10-16T11:21:09.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac400c3b0 con 0x7fcac4003a00 2023-10-16T11:21:09.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcadad9d700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac4005210 0x7fcac400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:09.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcada59c700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4003a00 0x7fcac4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:09.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcad9d9b700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcac4004850 0x7fcac4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:09.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.230+0000 7fcadad9d700 1 -- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4003a00 msgr2=0x7fcac4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:09.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcadad9d700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4003a00 0x7fcac4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcadad9d700 1 -- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcac4004850 msgr2=0x7fcac4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:09.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcadad9d700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcac4004850 0x7fcac4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcadad9d700 1 -- 172.21.15.181:0/1925356565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac40035b0 con 0x7fcac4005210 2023-10-16T11:21:09.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcadad9d700 1 -- 172.21.15.181:0/1925356565 reap_dead start 2023-10-16T11:21:09.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcadad9d700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac4005210 0x7fcac400ba10 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009f90 tx=0x7fcac4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a24689b9030fbc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:09.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4003a20 con 0x7fcac4005210 2023-10-16T11:21:09.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad400f040 con 0x7fcac4005210 2023-10-16T11:21:09.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcad959a700 1 -- 172.21.15.181:0/1925356565 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4003da0 con 0x7fcac4005210 2023-10-16T11:21:09.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcae0d69700 1 -- 172.21.15.181:0/1925356565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac4005210 msgr2=0x7fcac400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:09.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcae0d69700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac4005210 0x7fcac400ba10 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009f90 tx=0x7fcac4005880 comp rx=0 tx=0).stop 2023-10-16T11:21:09.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcae0d69700 1 -- 172.21.15.181:0/1925356565 shutdown_connections 2023-10-16T11:21:09.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcae0d69700 1 --2- 172.21.15.181:0/1925356565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcac4005210 0x7fcac400ba10 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcae0d69700 1 -- 172.21.15.181:0/1925356565 >> 172.21.15.181:0/1925356565 conn(0x7fcadc0d1610 msgr2=0x7fcadc0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:09.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcae0d69700 1 -- 172.21.15.181:0/1925356565 shutdown_connections 2023-10-16T11:21:09.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.231+0000 7fcae0d69700 1 -- 172.21.15.181:0/1925356565 wait complete. 2023-10-16T11:21:09.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.233+0000 7fcae0d69700 1 Processor -- start 2023-10-16T11:21:09.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.233+0000 7fcae0d69700 1 -- start start 2023-10-16T11:21:09.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.233+0000 7fcae0d69700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4005210 0x7fcadc0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcae0d69700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcadc0cadc0 0x7fcadc070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcae0d69700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcadc0cb240 0x7fcadc070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcae0d69700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0c6b90 con 0x7fcadc0cb240 2023-10-16T11:21:09.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcae0d69700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0c5830 con 0x7fcadc0cadc0 2023-10-16T11:21:09.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcae0d69700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0c5b10 con 0x7fcac4005210 2023-10-16T11:21:09.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcadc0cb240 0x7fcadc070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:09.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcadc0cb240 0x7fcadc070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59474/0 (socket says 172.21.15.181:59474) 2023-10-16T11:21:09.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 -- 172.21.15.181:0/698982071 learned_addr learned my addr 172.21.15.181:0/698982071 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:09.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcad9d9b700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcadc0cadc0 0x7fcadc070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:09.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 -- 172.21.15.181:0/698982071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4005210 msgr2=0x7fcadc0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:09.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4005210 0x7fcadc0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 -- 172.21.15.181:0/698982071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcadc0cadc0 msgr2=0x7fcadc070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:09.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcadc0cadc0 0x7fcadc070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:09.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 -- 172.21.15.181:0/698982071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0c5df0 con 0x7fcadc0cb240 2023-10-16T11:21:09.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcadad9d700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcadc0cb240 0x7fcadc070640 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0717c0 tx=0x7fcad4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:09.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcacaffd700 1 -- 172.21.15.181:0/698982071 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4007000 con 0x7fcadc0cb240 2023-10-16T11:21:09.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcacaffd700 1 -- 172.21.15.181:0/698982071 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4003c30 con 0x7fcadc0cb240 2023-10-16T11:21:09.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc0c60d0 con 0x7fcadc0cb240 2023-10-16T11:21:09.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcacaffd700 1 -- 172.21.15.181:0/698982071 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4015930 con 0x7fcadc0cb240 2023-10-16T11:21:09.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.234+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc0c6690 con 0x7fcadc0cb240 2023-10-16T11:21:09.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.235+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc0c6850 con 0x7fcadc0cb240 2023-10-16T11:21:09.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.236+0000 7fcacaffd700 1 -- 172.21.15.181:0/698982071 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcad4015a90 con 0x7fcadc0cb240 2023-10-16T11:21:09.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.236+0000 7fcacaffd700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fcabc06b140 0x7fcabc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:09.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.236+0000 7fcacaffd700 1 -- 172.21.15.181:0/698982071 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fcad408b2c0 con 0x7fcadc0cb240 2023-10-16T11:21:09.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.236+0000 7fcada59c700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fcabc06b140 0x7fcabc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:09.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.236+0000 7fcada59c700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fcabc06b140 0x7fcabc06d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcad00011e0 tx=0x7fcad0001110 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:09.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.240+0000 7fcacaffd700 1 -- 172.21.15.181:0/698982071 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcad4055a70 con 0x7fcadc0cb240 2023-10-16T11:21:09.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:09.435+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcadc067d20 con 0x7fcadc0cb240 2023-10-16T11:21:10.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.080+0000 7fcacaffd700 1 -- 172.21.15.181:0/698982071 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v247) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcad4015d40 con 0x7fcadc0cb240 2023-10-16T11:21:10.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fcabc06b140 msgr2=0x7fcabc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:10.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fcabc06b140 0x7fcabc06d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcad00011e0 tx=0x7fcad0001110 comp rx=0 tx=0).stop 2023-10-16T11:21:10.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcadc0cb240 msgr2=0x7fcadc070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:10.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcadc0cb240 0x7fcadc070640 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0717c0 tx=0x7fcad4004b40 comp rx=0 tx=0).stop 2023-10-16T11:21:10.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 shutdown_connections 2023-10-16T11:21:10.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcac4005210 0x7fcadc0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:10.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fcabc06b140 0x7fcabc06d600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:10.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcadc0cb240 0x7fcadc070640 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:10.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 --2- 172.21.15.181:0/698982071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcadc0cadc0 0x7fcadc070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:10.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 >> 172.21.15.181:0/698982071 conn(0x7fcadc0d1610 msgr2=0x7fcadc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:10.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 shutdown_connections 2023-10-16T11:21:10.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.084+0000 7fcae0d69700 1 -- 172.21.15.181:0/698982071 wait complete. 2023-10-16T11:21:10.089 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_metadata' removed 2023-10-16T11:21:10.101 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-16T11:21:10.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.227+0000 7ff8268b9700 1 Processor -- start 2023-10-16T11:21:10.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.228+0000 7ff8268b9700 1 -- start start 2023-10-16T11:21:10.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.228+0000 7ff8268b9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff82006afb0 0x7ff82006d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:10.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.228+0000 7ff8268b9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff820162650 con 0x7ff8200ce610 2023-10-16T11:21:10.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.228+0000 7ff8268b9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200d0230 con 0x7ff8200d0fc0 2023-10-16T11:21:10.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.228+0000 7ff8268b9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff820162190 con 0x7ff82006afb0 2023-10-16T11:21:10.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff824e56700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff82006afb0 0x7ff82006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:10.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff81ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff8200ce610 0x7ff8200cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42324/0 (socket says 172.21.15.181:42324) 2023-10-16T11:21:10.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff81ffff700 1 -- 172.21.15.181:0/1741706908 learned_addr learned my addr 172.21.15.181:0/1741706908 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:10.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff824e56700 1 -- 172.21.15.181:0/1741706908 >> v1:172.21.15.190:6789/0 conn(0x7ff8200d0fc0 legacy=0x7ff82006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:10.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff824e56700 1 -- 172.21.15.181:0/1741706908 >> v1:172.21.15.181:6789/0 conn(0x7ff8200ce610 legacy=0x7ff8200cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:10.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff824e56700 1 -- 172.21.15.181:0/1741706908 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff820162370 con 0x7ff82006afb0 2023-10-16T11:21:10.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff81effd700 1 -- 172.21.15.181:0/1741706908 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703182228 0 0) 0x7ff820162650 con 0x7ff8200ce610 2023-10-16T11:21:10.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff824e56700 1 --2- 172.21.15.181:0/1741706908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82006afb0 0x7ff82006d660 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7ff818009d90 tx=0x7ff81800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbb47d7d9e8b7b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:10.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff81effd700 1 -- 172.21.15.181:0/1741706908 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800e070 con 0x7ff82006afb0 2023-10-16T11:21:10.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff81effd700 1 -- 172.21.15.181:0/1741706908 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81800bbd0 con 0x7ff82006afb0 2023-10-16T11:21:10.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.229+0000 7ff81effd700 1 -- 172.21.15.181:0/1741706908 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818012430 con 0x7ff82006afb0 2023-10-16T11:21:10.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.230+0000 7ff8268b9700 1 -- 172.21.15.181:0/1741706908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82006afb0 msgr2=0x7ff82006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:10.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.230+0000 7ff8268b9700 1 --2- 172.21.15.181:0/1741706908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82006afb0 0x7ff82006d660 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7ff818009d90 tx=0x7ff81800b330 comp rx=0 tx=0).stop 2023-10-16T11:21:10.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.230+0000 7ff8268b9700 1 -- 172.21.15.181:0/1741706908 shutdown_connections 2023-10-16T11:21:10.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.230+0000 7ff8268b9700 1 --2- 172.21.15.181:0/1741706908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82006afb0 0x7ff82006d660 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:10.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.230+0000 7ff8268b9700 1 -- 172.21.15.181:0/1741706908 >> 172.21.15.181:0/1741706908 conn(0x7ff8200d04a0 msgr2=0x7ff8200d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:10.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.230+0000 7ff8268b9700 1 -- 172.21.15.181:0/1741706908 shutdown_connections 2023-10-16T11:21:10.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.230+0000 7ff8268b9700 1 -- 172.21.15.181:0/1741706908 wait complete. 2023-10-16T11:21:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.231+0000 7ff8268b9700 1 Processor -- start 2023-10-16T11:21:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.231+0000 7ff8268b9700 1 -- start start 2023-10-16T11:21:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff8268b9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82006afb0 0x7ff820157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:10.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff8268b9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8200ce610 0x7ff820157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff8268b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8200d0fc0 0x7ff820154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff8268b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82015fea0 con 0x7ff8200d0fc0 2023-10-16T11:21:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff8268b9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201492a0 con 0x7ff8200ce610 2023-10-16T11:21:10.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff8268b9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff820157940 con 0x7ff82006afb0 2023-10-16T11:21:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8200d0fc0 0x7ff820154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8200d0fc0 0x7ff820154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59484/0 (socket says 172.21.15.181:59484) 2023-10-16T11:21:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 -- 172.21.15.181:0/2360981847 learned_addr learned my addr 172.21.15.181:0/2360981847 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff81f7fe700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8200ce610 0x7ff820157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:10.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff81ffff700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82006afb0 0x7ff820157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:10.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 -- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82006afb0 msgr2=0x7ff820157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:10.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82006afb0 0x7ff820157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:10.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 -- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8200ce610 msgr2=0x7ff820157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:10.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8200ce610 0x7ff820157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:10.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 -- 172.21.15.181:0/2360981847 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818009a70 con 0x7ff8200d0fc0 2023-10-16T11:21:10.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff81f7fe700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8200ce610 0x7ff820157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:10.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff81ffff700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82006afb0 0x7ff820157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:10.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.232+0000 7ff824e56700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8200d0fc0 0x7ff820154970 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7ff818014040 tx=0x7ff818004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:10.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.233+0000 7ff81cff9700 1 -- 172.21.15.181:0/2360981847 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81801b810 con 0x7ff8200d0fc0 2023-10-16T11:21:10.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.233+0000 7ff81cff9700 1 -- 172.21.15.181:0/2360981847 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818020070 con 0x7ff8200d0fc0 2023-10-16T11:21:10.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.233+0000 7ff81cff9700 1 -- 172.21.15.181:0/2360981847 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81801bbb0 con 0x7ff8200d0fc0 2023-10-16T11:21:10.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.233+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82015f290 con 0x7ff8200d0fc0 2023-10-16T11:21:10.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.233+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82015f850 con 0x7ff8200d0fc0 2023-10-16T11:21:10.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.233+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff820160050 con 0x7ff8200d0fc0 2023-10-16T11:21:10.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.234+0000 7ff81cff9700 1 -- 172.21.15.181:0/2360981847 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff81801bd10 con 0x7ff8200d0fc0 2023-10-16T11:21:10.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.234+0000 7ff81cff9700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff80c06b140 0x7ff80c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:10.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.235+0000 7ff81cff9700 1 -- 172.21.15.181:0/2360981847 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 6861+0+0 (secure 0 0 0) 0x7ff818022070 con 0x7ff8200d0fc0 2023-10-16T11:21:10.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.235+0000 7ff81ffff700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff80c06b140 0x7ff80c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:10.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.235+0000 7ff81ffff700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff80c06b140 0x7ff80c06d600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff82009de50 tx=0x7ff814009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:10.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.239+0000 7ff81cff9700 1 -- 172.21.15.181:0/2360981847 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff820160050 con 0x7ff8200d0fc0 2023-10-16T11:21:10.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:10.429+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff82015ffe0 con 0x7ff8200d0fc0 2023-10-16T11:21:11.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.082+0000 7ff81cff9700 1 -- 172.21.15.181:0/2360981847 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v248) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff81805a4f0 con 0x7ff8200d0fc0 2023-10-16T11:21:11.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff80c06b140 msgr2=0x7ff80c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff80c06b140 0x7ff80c06d600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff82009de50 tx=0x7ff814009040 comp rx=0 tx=0).stop 2023-10-16T11:21:11.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8200d0fc0 msgr2=0x7ff820154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.089 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8200d0fc0 0x7ff820154970 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7ff818014040 tx=0x7ff818004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 shutdown_connections 2023-10-16T11:21:11.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff82006afb0 0x7ff820157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff80c06b140 0x7ff80c06d600 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8200d0fc0 0x7ff820154970 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 --2- 172.21.15.181:0/2360981847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8200ce610 0x7ff820157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 >> 172.21.15.181:0/2360981847 conn(0x7ff8200d04a0 msgr2=0x7ff8200cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:11.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 shutdown_connections 2023-10-16T11:21:11.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.087+0000 7ff8268b9700 1 -- 172.21.15.181:0/2360981847 wait complete. 2023-10-16T11:21:11.092 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'backup_fs_data' removed 2023-10-16T11:21:11.105 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:21:11.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.234+0000 7f44fc767700 1 Processor -- start 2023-10-16T11:21:11.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.234+0000 7f44fc767700 1 -- start start 2023-10-16T11:21:11.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fc767700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44f415ac20 0x7f44f406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fc767700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44f40ceb60 0x7f44f40cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fc767700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f40cf4f0 con 0x7f44f4158360 2023-10-16T11:21:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fc767700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f40cb340 con 0x7f44f40ceb60 2023-10-16T11:21:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fc767700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f415b0d0 con 0x7f44f415ac20 2023-10-16T11:21:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44f40ceb60 0x7f44f40cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44f40ceb60 0x7f44f40cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59494/0 (socket says 172.21.15.181:59494) 2023-10-16T11:21:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 -- 172.21.15.181:0/3676055432 learned_addr learned my addr 172.21.15.181:0/3676055432 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:11.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 -- 172.21.15.181:0/3676055432 >> v2:172.21.15.190:3300/0 conn(0x7f44f415ac20 msgr2=0x7f44f406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 --2- 172.21.15.181:0/3676055432 >> v2:172.21.15.190:3300/0 conn(0x7f44f415ac20 0x7f44f406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 -- 172.21.15.181:0/3676055432 >> v1:172.21.15.190:6789/0 conn(0x7f44f4158360 legacy=0x7f44f415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 -- 172.21.15.181:0/3676055432 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f415b3b0 con 0x7f44f40ceb60 2023-10-16T11:21:11.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44fad04700 1 --2- 172.21.15.181:0/3676055432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40cefb0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f44f40730d0 tx=0x7f44ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957d856d7191ff09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44f9501700 1 -- 172.21.15.181:0/3676055432 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00e070 con 0x7f44f40ceb60 2023-10-16T11:21:11.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44f9501700 1 -- 172.21.15.181:0/3676055432 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec003a20 con 0x7f44f40ceb60 2023-10-16T11:21:11.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.235+0000 7f44f9501700 1 -- 172.21.15.181:0/3676055432 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec003dd0 con 0x7f44f40ceb60 2023-10-16T11:21:11.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 -- 172.21.15.181:0/3676055432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 msgr2=0x7f44f40cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 --2- 172.21.15.181:0/3676055432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40cefb0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f44f40730d0 tx=0x7f44ec00b330 comp rx=0 tx=0).stop 2023-10-16T11:21:11.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 -- 172.21.15.181:0/3676055432 shutdown_connections 2023-10-16T11:21:11.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 --2- 172.21.15.181:0/3676055432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40cefb0 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 --2- 172.21.15.181:0/3676055432 >> v2:172.21.15.190:3300/0 conn(0x7f44f415ac20 0x7f44f406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 -- 172.21.15.181:0/3676055432 >> 172.21.15.181:0/3676055432 conn(0x7f44f40cc730 msgr2=0x7f44f40ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:11.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 -- 172.21.15.181:0/3676055432 shutdown_connections 2023-10-16T11:21:11.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.236+0000 7f44fc767700 1 -- 172.21.15.181:0/3676055432 wait complete. 2023-10-16T11:21:11.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.237+0000 7f44fc767700 1 Processor -- start 2023-10-16T11:21:11.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.237+0000 7f44fc767700 1 -- start start 2023-10-16T11:21:11.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fc767700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fc767700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f4158360 0x7f44f41e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fc767700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f415ac20 0x7f44f41f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fc767700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40c4f40 con 0x7f44f40ceb60 2023-10-16T11:21:11.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fc767700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40c4060 con 0x7f44f4158360 2023-10-16T11:21:11.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fc767700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40aeeb0 con 0x7f44f415ac20 2023-10-16T11:21:11.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59496/0 (socket says 172.21.15.181:59496) 2023-10-16T11:21:11.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 -- 172.21.15.181:0/3943523218 learned_addr learned my addr 172.21.15.181:0/3943523218 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 -- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f415ac20 msgr2=0x7f44f41f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f415ac20 0x7f44f41f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 -- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f4158360 msgr2=0x7f44f41e48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44f9d02700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f4158360 0x7f44f41e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f4158360 0x7f44f41e48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 -- 172.21.15.181:0/3943523218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009b30 con 0x7f44f40ceb60 2023-10-16T11:21:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44f9d02700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f4158360 0x7f44f41e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.238+0000 7f44fa503700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40ae970 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c4d0 tx=0x7f44e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.239+0000 7f44eaffd700 1 -- 172.21.15.181:0/3943523218 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40054e0 con 0x7f44f40ceb60 2023-10-16T11:21:11.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.239+0000 7f44eaffd700 1 -- 172.21.15.181:0/3943523218 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4003680 con 0x7f44f40ceb60 2023-10-16T11:21:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.239+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f41e5250 con 0x7f44f40ceb60 2023-10-16T11:21:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.239+0000 7f44eaffd700 1 -- 172.21.15.181:0/3943523218 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003a30 con 0x7f44f40ceb60 2023-10-16T11:21:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.239+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f41e5810 con 0x7f44f40ceb60 2023-10-16T11:21:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.240+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f415e400 con 0x7f44f40ceb60 2023-10-16T11:21:11.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.240+0000 7f44eaffd700 1 -- 172.21.15.181:0/3943523218 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f44e4003b90 con 0x7f44f40ceb60 2023-10-16T11:21:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.241+0000 7f44eaffd700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f44d806b140 0x7f44d806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.241+0000 7f44eaffd700 1 -- 172.21.15.181:0/3943523218 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f44e4089170 con 0x7f44f40ceb60 2023-10-16T11:21:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.241+0000 7f44f9d02700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f44d806b140 0x7f44d806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.241+0000 7f44f9d02700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f44d806b140 0x7f44d806d600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44f41e6540 tx=0x7f44f0008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.245+0000 7f44eaffd700 1 -- 172.21.15.181:0/3943523218 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f44e4053c70 con 0x7f44f40ceb60 2023-10-16T11:21:11.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.444+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f44f4072090 con 0x7f44f40ceb60 2023-10-16T11:21:11.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.447+0000 7f44eaffd700 1 -- 172.21.15.181:0/3943523218 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v85) v1 ==== 115+0+0 (secure 0 0 0) 0x7f44e4057290 con 0x7f44f40ceb60 2023-10-16T11:21:11.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f44d806b140 msgr2=0x7f44d806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.452 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f44d806b140 0x7f44d806d600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44f41e6540 tx=0x7f44f0008040 comp rx=0 tx=0).stop 2023-10-16T11:21:11.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 msgr2=0x7f44f40ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40ae970 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c4d0 tx=0x7f44e400c890 comp rx=0 tx=0).stop 2023-10-16T11:21:11.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 shutdown_connections 2023-10-16T11:21:11.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44f415ac20 0x7f44f41f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f44d806b140 0x7f44d806d600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f40ceb60 0x7f44f40ae970 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 --2- 172.21.15.181:0/3943523218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44f4158360 0x7f44f41e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 >> 172.21.15.181:0/3943523218 conn(0x7f44f40cc730 msgr2=0x7f44f415dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:11.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 shutdown_connections 2023-10-16T11:21:11.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.450+0000 7f44fc767700 1 -- 172.21.15.181:0/3943523218 wait complete. 2023-10-16T11:21:11.465 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:21:11.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.585+0000 7f7ac3503700 1 Processor -- start 2023-10-16T11:21:11.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.586+0000 7f7ac3503700 1 -- start start 2023-10-16T11:21:11.591 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.586+0000 7f7ac3503700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 0x7f7abc06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.586+0000 7f7ac3503700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc06b230 con 0x7f7abc06ade0 2023-10-16T11:21:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.586+0000 7f7ac3503700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc0cef00 con 0x7f7abc15da70 2023-10-16T11:21:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.586+0000 7f7ac3503700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0cec20 con 0x7f7abc072760 2023-10-16T11:21:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac1aa0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7abc06ade0 0x7f7abc1615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42354/0 (socket says 172.21.15.181:42354) 2023-10-16T11:21:11.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac1aa0700 1 -- 172.21.15.181:0/2167800689 learned_addr learned my addr 172.21.15.181:0/2167800689 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac0a9e700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 0x7f7abc06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521239569 0 0) 0x7f7ab4001490 con 0x7f7abc06ade0 2023-10-16T11:21:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa80035b0 con 0x7f7abc06ade0 2023-10-16T11:21:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221682516 0 0) 0x7f7aa80035b0 con 0x7f7abc06ade0 2023-10-16T11:21:11.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 msgr2=0x7f7abc06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 0x7f7abc06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 >> v1:172.21.15.190:6789/0 conn(0x7f7abc15da70 legacy=0x7f7abc0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc071c80 con 0x7f7abc06ade0 2023-10-16T11:21:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f7ab4001b80 con 0x7f7abc06ade0 2023-10-16T11:21:11.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 >> v1:172.21.15.181:6789/0 conn(0x7f7abc06ade0 legacy=0x7f7abc1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8003a30 0x7f7aa8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa8004850 0x7f7aa8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.595 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 0x7f7aa8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0cec20 con 0x7f7aa8004850 2023-10-16T11:21:11.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa800bb70 con 0x7f7aa8003a30 2023-10-16T11:21:11.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa800bd30 con 0x7f7abc072760 2023-10-16T11:21:11.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac0a9e700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa8004850 0x7f7aa8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac1aa0700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 0x7f7aa8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac129f700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8003a30 0x7f7aa8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac0a9e700 1 -- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 msgr2=0x7f7aa8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac0a9e700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 0x7f7aa8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac0a9e700 1 -- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8003a30 msgr2=0x7f7aa8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac0a9e700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8003a30 0x7f7aa8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac0a9e700 1 -- 172.21.15.181:0/2167800689 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa80035b0 con 0x7f7aa8004850 2023-10-16T11:21:11.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac1aa0700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc072760 0x7f7aa8005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.587+0000 7f7ac129f700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8003a30 0x7f7aa8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac0a9e700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa8004850 0x7f7aa8004cd0 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009b60 tx=0x7f7aa8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752b53428a22d927 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00e6c0 con 0x7f7aa8004850 2023-10-16T11:21:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac00e820 con 0x7f7aa8004850 2023-10-16T11:21:11.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ab3fff700 1 -- 172.21.15.181:0/2167800689 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00eb90 con 0x7f7aa8004850 2023-10-16T11:21:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac3503700 1 -- 172.21.15.181:0/2167800689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa8004850 msgr2=0x7f7aa8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac3503700 1 --2- 172.21.15.181:0/2167800689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7aa8004850 0x7f7aa8004cd0 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009b60 tx=0x7f7aa8005880 comp rx=0 tx=0).stop 2023-10-16T11:21:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac1aa0700 1 -- 172.21.15.181:0/2167800689 reap_dead start 2023-10-16T11:21:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac3503700 1 -- 172.21.15.181:0/2167800689 shutdown_connections 2023-10-16T11:21:11.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac3503700 1 -- 172.21.15.181:0/2167800689 >> 172.21.15.181:0/2167800689 conn(0x7f7abc0cd010 msgr2=0x7f7abc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac3503700 1 -- 172.21.15.181:0/2167800689 shutdown_connections 2023-10-16T11:21:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.588+0000 7f7ac3503700 1 -- 172.21.15.181:0/2167800689 wait complete. 2023-10-16T11:21:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.590+0000 7f7ac3503700 1 Processor -- start 2023-10-16T11:21:11.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.590+0000 7f7ac3503700 1 -- start start 2023-10-16T11:21:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.590+0000 7f7ac3503700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8004850 0x7f7abc0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac3503700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc1f2e20 0x7f7abc159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac3503700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7abc0aea60 0x7f7abc0aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac3503700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1f3e10 con 0x7f7abc0aea60 2023-10-16T11:21:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac3503700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0b3fc0 con 0x7f7aa8004850 2023-10-16T11:21:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac3503700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0af420 con 0x7f7abc1f2e20 2023-10-16T11:21:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7abc0aea60 0x7f7abc0aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7abc0aea60 0x7f7abc0aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59520/0 (socket says 172.21.15.181:59520) 2023-10-16T11:21:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 -- 172.21.15.181:0/1192388961 learned_addr learned my addr 172.21.15.181:0/1192388961 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac129f700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8004850 0x7f7abc0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac0a9e700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc1f2e20 0x7f7abc159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 -- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc1f2e20 msgr2=0x7f7abc159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc1f2e20 0x7f7abc159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 -- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8004850 msgr2=0x7f7abc0b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8004850 0x7f7abc0b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 -- 172.21.15.181:0/1192388961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc0c4e80 con 0x7f7abc0aea60 2023-10-16T11:21:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac1aa0700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7abc0aea60 0x7f7abc0aeee0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f7abc06f170 tx=0x7f7ab4007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ab1ffb700 1 -- 172.21.15.181:0/1192388961 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400ea30 con 0x7f7abc0aea60 2023-10-16T11:21:11.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ab1ffb700 1 -- 172.21.15.181:0/1192388961 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab400eb90 con 0x7f7abc0aea60 2023-10-16T11:21:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc1f32a0 con 0x7f7abc0aea60 2023-10-16T11:21:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ab1ffb700 1 -- 172.21.15.181:0/1192388961 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4013420 con 0x7f7abc0aea60 2023-10-16T11:21:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.591+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1f3850 con 0x7f7abc0aea60 2023-10-16T11:21:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.592+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc0c50a0 con 0x7f7abc0aea60 2023-10-16T11:21:11.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.593+0000 7f7ab1ffb700 1 -- 172.21.15.181:0/1192388961 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7ab4013580 con 0x7f7abc0aea60 2023-10-16T11:21:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.594+0000 7f7ab1ffb700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7aa006b140 0x7f7aa006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.594+0000 7f7ab1ffb700 1 -- 172.21.15.181:0/1192388961 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7ab408a950 con 0x7f7abc0aea60 2023-10-16T11:21:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.594+0000 7f7ac129f700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7aa006b140 0x7f7aa006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.594+0000 7f7ac129f700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7aa006b140 0x7f7aa006d600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000c00 tx=0x7f7ab8009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.597+0000 7f7ab1ffb700 1 -- 172.21.15.181:0/1192388961 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7ab40553a0 con 0x7f7abc0aea60 2023-10-16T11:21:11.790 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.789+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f7abc0c50a0 con 0x7f7abc0aea60 2023-10-16T11:21:11.796 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.794+0000 7f7ab1ffb700 1 -- 172.21.15.181:0/1192388961 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v86) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7abc0c50a0 con 0x7f7abc0aea60 2023-10-16T11:21:11.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7aa006b140 msgr2=0x7f7aa006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7aa006b140 0x7f7aa006d600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000c00 tx=0x7f7ab8009040 comp rx=0 tx=0).stop 2023-10-16T11:21:11.799 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7abc0aea60 msgr2=0x7f7abc0aeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7abc0aea60 0x7f7abc0aeee0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f7abc06f170 tx=0x7f7ab4007c90 comp rx=0 tx=0).stop 2023-10-16T11:21:11.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 shutdown_connections 2023-10-16T11:21:11.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7aa006b140 0x7f7aa006d600 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7abc0aea60 0x7f7abc0aeee0 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa8004850 0x7f7abc0b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 --2- 172.21.15.181:0/1192388961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7abc1f2e20 0x7f7abc159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 >> 172.21.15.181:0/1192388961 conn(0x7f7abc0cd010 msgr2=0x7f7abc0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:11.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 shutdown_connections 2023-10-16T11:21:11.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.797+0000 7f7ac3503700 1 -- 172.21.15.181:0/1192388961 wait complete. 2023-10-16T11:21:11.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:21:11.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.932+0000 7ff37f6ff700 1 Processor -- start 2023-10-16T11:21:11.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.932+0000 7ff37f6ff700 1 -- start start 2023-10-16T11:21:11.942 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff37f6ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff37815da70 0x7ff3780ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff37f6ff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3780cef00 con 0x7ff3780ce6a0 2023-10-16T11:21:11.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff37f6ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3780cec20 con 0x7ff37806afb0 2023-10-16T11:21:11.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff37f6ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff378071c80 con 0x7ff37815da70 2023-10-16T11:21:11.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff37cc9a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff3780ce6a0 0x7ff37806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42370/0 (socket says 172.21.15.181:42370) 2023-10-16T11:21:11.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3030386572 learned_addr learned my addr 172.21.15.181:0/3030386572 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:11.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff37d49b700 1 --2- 172.21.15.181:0/3030386572 >> v2:172.21.15.190:3300/0 conn(0x7ff37815da70 0x7ff3780ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512866515 0 0) 0x7ff3780cef00 con 0x7ff3780ce6a0 2023-10-16T11:21:11.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3640035b0 con 0x7ff3780ce6a0 2023-10-16T11:21:11.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2543312271 0 0) 0x7ff3640035b0 con 0x7ff3780ce6a0 2023-10-16T11:21:11.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 >> v2:172.21.15.190:3300/0 conn(0x7ff37815da70 msgr2=0x7ff3780ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.933+0000 7ff36ffff700 1 --2- 172.21.15.181:0/3030386572 >> v2:172.21.15.190:3300/0 conn(0x7ff37815da70 0x7ff3780ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 >> v1:172.21.15.190:6789/0 conn(0x7ff37806afb0 legacy=0x7ff3781615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378071f60 con 0x7ff3780ce6a0 2023-10-16T11:21:11.945 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff374001670 con 0x7ff3780ce6a0 2023-10-16T11:21:11.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 >> v1:172.21.15.181:6789/0 conn(0x7ff3780ce6a0 legacy=0x7ff37806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364003a00 0x7ff364003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364004850 0x7ff364004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.946 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff364005330 0x7ff36400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff378071c80 con 0x7ff364004850 2023-10-16T11:21:11.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36400c200 con 0x7ff364005330 2023-10-16T11:21:11.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36400c3c0 con 0x7ff364003a00 2023-10-16T11:21:11.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37cc9a700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364004850 0x7ff364004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.947 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37d49b700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364003a00 0x7ff364003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37dc9c700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff364005330 0x7ff36400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364003a00 msgr2=0x7ff364003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37cc9a700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364003a00 0x7ff364003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff364005330 msgr2=0x7ff36400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:11.948 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37cc9a700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff364005330 0x7ff36400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3030386572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3640035b0 con 0x7ff364004850 2023-10-16T11:21:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37dc9c700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff364005330 0x7ff36400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37d49b700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364003a00 0x7ff364003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37dc9c700 1 -- 172.21.15.181:0/3030386572 reap_dead start 2023-10-16T11:21:11.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff37cc9a700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364004850 0x7ff364004cd0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c210 tx=0x7ff364005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09ad40ac746b51e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374007c10 con 0x7ff364004850 2023-10-16T11:21:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37400cdb0 con 0x7ff364004850 2023-10-16T11:21:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.934+0000 7ff36ffff700 1 -- 172.21.15.181:0/3030386572 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374005ba0 con 0x7ff364004850 2023-10-16T11:21:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.935+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3030386572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364004850 msgr2=0x7ff364004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.935+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364004850 0x7ff364004cd0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c210 tx=0x7ff364005880 comp rx=0 tx=0).stop 2023-10-16T11:21:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.935+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3030386572 shutdown_connections 2023-10-16T11:21:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.935+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3030386572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364004850 0x7ff364004cd0 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.935+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3030386572 >> 172.21.15.181:0/3030386572 conn(0x7ff3780cd010 msgr2=0x7ff3780cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.935+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3030386572 shutdown_connections 2023-10-16T11:21:11.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.935+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3030386572 wait complete. 2023-10-16T11:21:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.936+0000 7ff37f6ff700 1 Processor -- start 2023-10-16T11:21:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.936+0000 7ff37f6ff700 1 -- start start 2023-10-16T11:21:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37f6ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364004850 0x7ff378151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37f6ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3781542b0 0x7ff378154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37f6ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b1060 0x7ff378154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37f6ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff378153e90 con 0x7ff3781542b0 2023-10-16T11:21:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37f6ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780b1540 con 0x7ff3780b1060 2023-10-16T11:21:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37f6ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3781537d0 con 0x7ff364004850 2023-10-16T11:21:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3781542b0 0x7ff378154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3781542b0 0x7ff378154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59538/0 (socket says 172.21.15.181:59538) 2023-10-16T11:21:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3436728931 learned_addr learned my addr 172.21.15.181:0/3436728931 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37dc9c700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b1060 0x7ff378154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37d49b700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364004850 0x7ff378151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364004850 msgr2=0x7ff378151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364004850 0x7ff378151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b1060 msgr2=0x7ff378154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b1060 0x7ff378154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 -- 172.21.15.181:0/3436728931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378153ab0 con 0x7ff3781542b0 2023-10-16T11:21:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37d49b700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364004850 0x7ff378151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:11.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff37cc9a700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3781542b0 0x7ff378154910 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c1d0 tx=0x7ff374001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff36dffb700 1 -- 172.21.15.181:0/3436728931 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3740070d0 con 0x7ff3781542b0 2023-10-16T11:21:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.937+0000 7ff36dffb700 1 -- 172.21.15.181:0/3436728931 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff374007230 con 0x7ff3781542b0 2023-10-16T11:21:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.938+0000 7ff36dffb700 1 -- 172.21.15.181:0/3436728931 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3740075a0 con 0x7ff3781542b0 2023-10-16T11:21:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.938+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff378151f80 con 0x7ff3781542b0 2023-10-16T11:21:11.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.938+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff378155d30 con 0x7ff3781542b0 2023-10-16T11:21:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.939+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37806f6c0 con 0x7ff3781542b0 2023-10-16T11:21:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.939+0000 7ff36dffb700 1 -- 172.21.15.181:0/3436728931 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff37401e430 con 0x7ff3781542b0 2023-10-16T11:21:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.939+0000 7ff36dffb700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff35c06b140 0x7ff35c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.940+0000 7ff36dffb700 1 -- 172.21.15.181:0/3436728931 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7ff37408ac20 con 0x7ff3781542b0 2023-10-16T11:21:11.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.940+0000 7ff37d49b700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff35c06b140 0x7ff35c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:11.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.940+0000 7ff37d49b700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff35c06b140 0x7ff35c06d600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff3780697f0 tx=0x7ff368008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:11.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:11.944+0000 7ff36dffb700 1 -- 172.21.15.181:0/3436728931 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff374055720 con 0x7ff3781542b0 2023-10-16T11:21:12.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.140+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff378067ca0 con 0x7ff3781542b0 2023-10-16T11:21:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.145+0000 7ff36dffb700 1 -- 172.21.15.181:0/3436728931 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff374058d40 con 0x7ff3781542b0 2023-10-16T11:21:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff35c06b140 msgr2=0x7ff35c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff35c06b140 0x7ff35c06d600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff3780697f0 tx=0x7ff368008040 comp rx=0 tx=0).stop 2023-10-16T11:21:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3781542b0 msgr2=0x7ff378154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:12.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3781542b0 0x7ff378154910 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c1d0 tx=0x7ff374001a00 comp rx=0 tx=0).stop 2023-10-16T11:21:12.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 shutdown_connections 2023-10-16T11:21:12.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff364004850 0x7ff378151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff35c06b140 0x7ff35c06d600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3780b1060 0x7ff378154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 --2- 172.21.15.181:0/3436728931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3781542b0 0x7ff378154910 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 >> 172.21.15.181:0/3436728931 conn(0x7ff3780cd010 msgr2=0x7ff37815b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:12.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 shutdown_connections 2023-10-16T11:21:12.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.147+0000 7ff37f6ff700 1 -- 172.21.15.181:0/3436728931 wait complete. 2023-10-16T11:21:12.162 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-16T11:21:12.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.281+0000 7f52606b7700 1 Processor -- start 2023-10-16T11:21:12.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.281+0000 7f52606b7700 1 -- start start 2023-10-16T11:21:12.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f52606b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52580af450 0x7f52580af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:12.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f52606b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52580cd470 0x7f52580cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:12.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f52606b7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525806f870 con 0x7f525815dfd0 2023-10-16T11:21:12.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f52606b7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580d1160 con 0x7f52580af450 2023-10-16T11:21:12.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f52606b7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580cc8e0 con 0x7f52580cd470 2023-10-16T11:21:12.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52580af450 0x7f52580af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:12.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52580af450 0x7f52580af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59546/0 (socket says 172.21.15.181:59546) 2023-10-16T11:21:12.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 -- 172.21.15.181:0/2587015600 learned_addr learned my addr 172.21.15.181:0/2587015600 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:12.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525ec54700 1 --2- 172.21.15.181:0/2587015600 >> v2:172.21.15.190:3300/0 conn(0x7f52580cd470 0x7f52580cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:12.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 -- 172.21.15.181:0/2587015600 >> v2:172.21.15.190:3300/0 conn(0x7f52580cd470 msgr2=0x7f52580cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:12.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 --2- 172.21.15.181:0/2587015600 >> v2:172.21.15.190:3300/0 conn(0x7f52580cd470 0x7f52580cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 -- 172.21.15.181:0/2587015600 >> v1:172.21.15.190:6789/0 conn(0x7f525815dfd0 legacy=0x7f525806d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:12.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 -- 172.21.15.181:0/2587015600 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580ccbc0 con 0x7f52580af450 2023-10-16T11:21:12.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525dc52700 1 --2- 172.21.15.181:0/2587015600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52580af9a0 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f52580af1c0 tx=0x7f5248009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f50e11970959a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:12.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.282+0000 7f525d451700 1 -- 172.21.15.181:0/2587015600 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248004230 con 0x7f52580af450 2023-10-16T11:21:12.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f525d451700 1 -- 172.21.15.181:0/2587015600 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5248004390 con 0x7f52580af450 2023-10-16T11:21:12.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f525d451700 1 -- 172.21.15.181:0/2587015600 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480036a0 con 0x7f52580af450 2023-10-16T11:21:12.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 -- 172.21.15.181:0/2587015600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 msgr2=0x7f52580af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:12.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 --2- 172.21.15.181:0/2587015600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52580af9a0 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f52580af1c0 tx=0x7f5248009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 -- 172.21.15.181:0/2587015600 shutdown_connections 2023-10-16T11:21:12.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 --2- 172.21.15.181:0/2587015600 >> v2:172.21.15.190:3300/0 conn(0x7f52580cd470 0x7f52580cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 --2- 172.21.15.181:0/2587015600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52580af9a0 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 -- 172.21.15.181:0/2587015600 >> 172.21.15.181:0/2587015600 conn(0x7f52580d04a0 msgr2=0x7f52580d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:12.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 -- 172.21.15.181:0/2587015600 shutdown_connections 2023-10-16T11:21:12.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.283+0000 7f52606b7700 1 -- 172.21.15.181:0/2587015600 wait complete. 2023-10-16T11:21:12.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.285+0000 7f52606b7700 1 Processor -- start 2023-10-16T11:21:12.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.285+0000 7f52606b7700 1 -- start start 2023-10-16T11:21:12.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52581f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:12.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52580cd470 0x7f525814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:12.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f525815dfd0 0x7f52580c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:12.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580c44c0 con 0x7f52580af450 2023-10-16T11:21:12.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525814fb00 con 0x7f52580cd470 2023-10-16T11:21:12.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525814fde0 con 0x7f525815dfd0 2023-10-16T11:21:12.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52581f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:12.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52581f2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59550/0 (socket says 172.21.15.181:59550) 2023-10-16T11:21:12.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 -- 172.21.15.181:0/1395689698 learned_addr learned my addr 172.21.15.181:0/1395689698 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:12.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 -- 172.21.15.181:0/1395689698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f525815dfd0 msgr2=0x7f52580c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:12.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f525815dfd0 0x7f52580c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 -- 172.21.15.181:0/1395689698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52580cd470 msgr2=0x7f525814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:12.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52580cd470 0x7f525814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:12.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 -- 172.21.15.181:0/1395689698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248009550 con 0x7f52580af450 2023-10-16T11:21:12.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f525e453700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52581f2900 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f525400b450 tx=0x7f525400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:12.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f524effd700 1 -- 172.21.15.181:0/1395689698 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254004020 con 0x7f52580af450 2023-10-16T11:21:12.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f524effd700 1 -- 172.21.15.181:0/1395689698 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5254004180 con 0x7f52580af450 2023-10-16T11:21:12.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52580c4150 con 0x7f52580af450 2023-10-16T11:21:12.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f524effd700 1 -- 172.21.15.181:0/1395689698 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254004530 con 0x7f52580af450 2023-10-16T11:21:12.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.286+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52581f2e40 con 0x7f52580af450 2023-10-16T11:21:12.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.287+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5258150000 con 0x7f52580af450 2023-10-16T11:21:12.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.288+0000 7f524effd700 1 -- 172.21.15.181:0/1395689698 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f52540036d0 con 0x7f52580af450 2023-10-16T11:21:12.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.288+0000 7f524effd700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f523c06b170 0x7f523c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:12.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.288+0000 7f525dc52700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f523c06b170 0x7f523c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:12.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.288+0000 7f524effd700 1 -- 172.21.15.181:0/1395689698 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f5254089480 con 0x7f52580af450 2023-10-16T11:21:12.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.289+0000 7f525dc52700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f523c06b170 0x7f523c06d630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52580b62f0 tx=0x7f5248010000 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:12.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.293+0000 7f524effd700 1 -- 172.21.15.181:0/1395689698 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5254053f80 con 0x7f52580af450 2023-10-16T11:21:12.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:12.567+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f5258067d20 con 0x7f52580af450 2023-10-16T11:21:13.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.086+0000 7f524effd700 1 -- 172.21.15.181:0/1395689698 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3201) v1 ==== 187+0+0 (secure 0 0 0) 0x7f5254015100 con 0x7f52580af450 2023-10-16T11:21:13.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f523c06b170 msgr2=0x7f523c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f523c06b170 0x7f523c06d630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52580b62f0 tx=0x7f5248010000 comp rx=0 tx=0).stop 2023-10-16T11:21:13.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 msgr2=0x7f52581f2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52581f2900 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f525400b450 tx=0x7f525400b810 comp rx=0 tx=0).stop 2023-10-16T11:21:13.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 shutdown_connections 2023-10-16T11:21:13.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f525815dfd0 0x7f52580c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f523c06b170 0x7f523c06d630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52580af450 0x7f52581f2900 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 --2- 172.21.15.181:0/1395689698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52580cd470 0x7f525814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 >> 172.21.15.181:0/1395689698 conn(0x7f52580d04a0 msgr2=0x7f52580cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 shutdown_connections 2023-10-16T11:21:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.089+0000 7f52606b7700 1 -- 172.21.15.181:0/1395689698 wait complete. 2023-10-16T11:21:13.105 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-10-16T11:21:13.105 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-16T11:21:13.106 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-16T11:21:13.106 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:13.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.228+0000 7fefb9a48700 1 Processor -- start 2023-10-16T11:21:13.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.228+0000 7fefb9a48700 1 -- start start 2023-10-16T11:21:13.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb9a48700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fefb415dd00 0x7fefb4071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb9a48700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fefb4157c00 0x7fefb40c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb9a48700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb4071df0 con 0x7fefb4071870 2023-10-16T11:21:13.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb9a48700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40720d0 con 0x7fefb4157c00 2023-10-16T11:21:13.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb9a48700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40723b0 con 0x7fefb415dd00 2023-10-16T11:21:13.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fefb4157c00 0x7fefb40c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fefb4157c00 0x7fefb40c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59562/0 (socket says 172.21.15.181:59562) 2023-10-16T11:21:13.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 -- 172.21.15.181:0/2811336537 learned_addr learned my addr 172.21.15.181:0/2811336537 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:13.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb2ffd700 1 --2- 172.21.15.181:0/2811336537 >> v2:172.21.15.190:3300/0 conn(0x7fefb415dd00 0x7fefb4071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 -- 172.21.15.181:0/2811336537 >> v2:172.21.15.190:3300/0 conn(0x7fefb415dd00 msgr2=0x7fefb4071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:13.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 --2- 172.21.15.181:0/2811336537 >> v2:172.21.15.190:3300/0 conn(0x7fefb415dd00 0x7fefb4071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 -- 172.21.15.181:0/2811336537 >> v1:172.21.15.190:6789/0 conn(0x7fefb4071870 legacy=0x7fefb41574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:13.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 -- 172.21.15.181:0/2811336537 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4072690 con 0x7fefb4157c00 2023-10-16T11:21:13.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb37fe700 1 --2- 172.21.15.181:0/2811336537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40c9b30 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fefb40d04c0 tx=0x7fefac00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46544764f4e6a0ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb1ffb700 1 -- 172.21.15.181:0/2811336537 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac010070 con 0x7fefb4157c00 2023-10-16T11:21:13.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb1ffb700 1 -- 172.21.15.181:0/2811336537 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac00dd50 con 0x7fefb4157c00 2023-10-16T11:21:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.229+0000 7fefb1ffb700 1 -- 172.21.15.181:0/2811336537 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00b310 con 0x7fefb4157c00 2023-10-16T11:21:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 -- 172.21.15.181:0/2811336537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 msgr2=0x7fefb40c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 --2- 172.21.15.181:0/2811336537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40c9b30 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fefb40d04c0 tx=0x7fefac00d430 comp rx=0 tx=0).stop 2023-10-16T11:21:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 -- 172.21.15.181:0/2811336537 shutdown_connections 2023-10-16T11:21:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 --2- 172.21.15.181:0/2811336537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40c9b30 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 --2- 172.21.15.181:0/2811336537 >> v2:172.21.15.190:3300/0 conn(0x7fefb415dd00 0x7fefb4071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 -- 172.21.15.181:0/2811336537 >> 172.21.15.181:0/2811336537 conn(0x7fefb406aed0 msgr2=0x7fefb40b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:13.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 -- 172.21.15.181:0/2811336537 shutdown_connections 2023-10-16T11:21:13.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.230+0000 7fefb9a48700 1 -- 172.21.15.181:0/2811336537 wait complete. 2023-10-16T11:21:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.231+0000 7fefb9a48700 1 Processor -- start 2023-10-16T11:21:13.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.231+0000 7fefb9a48700 1 -- start start 2023-10-16T11:21:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb9a48700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefb4071870 0x7fefb40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb9a48700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb9a48700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefb415dd00 0x7fefb40af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb9a48700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40ad0b0 con 0x7fefb4157c00 2023-10-16T11:21:13.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb9a48700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40ad390 con 0x7fefb4071870 2023-10-16T11:21:13.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb9a48700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40ad670 con 0x7fefb415dd00 2023-10-16T11:21:13.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59568/0 (socket says 172.21.15.181:59568) 2023-10-16T11:21:13.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb27fc700 1 -- 172.21.15.181:0/1083349933 learned_addr learned my addr 172.21.15.181:0/1083349933 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:13.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb2ffd700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefb4071870 0x7fefb40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb37fe700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefb415dd00 0x7fefb40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb27fc700 1 -- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefb415dd00 msgr2=0x7fefb40af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb27fc700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefb415dd00 0x7fefb40af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.232+0000 7fefb27fc700 1 -- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefb4071870 msgr2=0x7fefb40ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fefb27fc700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefb4071870 0x7fefb40ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fefb27fc700 1 -- 172.21.15.181:0/1083349933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac00bd40 con 0x7fefb4157c00 2023-10-16T11:21:13.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fefb2ffd700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefb4071870 0x7fefb40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:13.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fefb27fc700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40aede0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fefa800b470 tx=0x7fefa800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fef9bfff700 1 -- 172.21.15.181:0/1083349933 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8011830 con 0x7fefb4157c00 2023-10-16T11:21:13.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fef9bfff700 1 -- 172.21.15.181:0/1083349933 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa8011990 con 0x7fefb4157c00 2023-10-16T11:21:13.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fef9bfff700 1 -- 172.21.15.181:0/1083349933 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8011d40 con 0x7fefb4157c00 2023-10-16T11:21:13.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb40ad950 con 0x7fefb4157c00 2023-10-16T11:21:13.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.233+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb40acdc0 con 0x7fefb4157c00 2023-10-16T11:21:13.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.234+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb404ebb0 con 0x7fefb4157c00 2023-10-16T11:21:13.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.234+0000 7fef9bfff700 1 -- 172.21.15.181:0/1083349933 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fefa801a400 con 0x7fefb4157c00 2023-10-16T11:21:13.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.235+0000 7fef9bfff700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fefa006b140 0x7fefa006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.235+0000 7fef9bfff700 1 -- 172.21.15.181:0/1083349933 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fefa8089d90 con 0x7fefb4157c00 2023-10-16T11:21:13.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.235+0000 7fefb2ffd700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fefa006b140 0x7fefa006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.235+0000 7fefb2ffd700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fefa006b140 0x7fefa006d600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fefa400aa50 tx=0x7fefa4008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.239+0000 7fef9bfff700 1 -- 172.21.15.181:0/1083349933 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fefa8054800 con 0x7fefb4157c00 2023-10-16T11:21:13.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.432+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb40cd700 con 0x7fefb4157c00 2023-10-16T11:21:13.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.433+0000 7fef9bfff700 1 -- 172.21.15.181:0/1083349933 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v248) v1 ==== 74+0+15289 (secure 0 0 0) 0x7fefa8015020 con 0x7fefb4157c00 2023-10-16T11:21:13.435 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:13.436 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":248,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:11.081958+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6834/1302040487":"2023-10-17T11:21:08.075349+0000","172.21.15.190:6832/1302040487":"2023-10-17T11:21:08.075349+0000","172.21.15.181:6836/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:6834/3483656577":"2023-10-17T11:21:01.293282+0000","172.21.15.181:0/4174526666":"2023-10-17T11:20:58.912264+0000","172.21.15.181:6832/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2905113763":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1596741027":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3516831361":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2396727780":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2869414246":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/3445652558":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/1783725009":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3959972769":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/1878395836":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2468932000":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3765547571":"2023-10-17T11:20:58.912264+0000","172.21.15.181:0/2390273904":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2847184738":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/2797066080":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6833/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3532323147":"2023-10-17T11:20:51.419893+0000","172.21.15.181:6832/3273985848":"2023-10-17T11:20:51.419893+0000","172.21.15.181:0/3731618096":"2023-10-17T11:20:51.419893+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:13.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fefa006b140 msgr2=0x7fefa006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fefa006b140 0x7fefa006d600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fefa400aa50 tx=0x7fefa4008040 comp rx=0 tx=0).stop 2023-10-16T11:21:13.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 msgr2=0x7fefb40aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40aede0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fefa800b470 tx=0x7fefa800b830 comp rx=0 tx=0).stop 2023-10-16T11:21:13.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 shutdown_connections 2023-10-16T11:21:13.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefb415dd00 0x7fefb40af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fefa006b140 0x7fefa006d600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefb4071870 0x7fefb40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 --2- 172.21.15.181:0/1083349933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefb4157c00 0x7fefb40aede0 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 >> 172.21.15.181:0/1083349933 conn(0x7fefb406aed0 msgr2=0x7fefb40cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:13.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 shutdown_connections 2023-10-16T11:21:13.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.435+0000 7fefb9a48700 1 -- 172.21.15.181:0/1083349933 wait complete. 2023-10-16T11:21:13.450 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:21:13.450 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:21:13.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.572+0000 7f9e0d81b700 1 Processor -- start 2023-10-16T11:21:13.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e0d81b700 1 -- start start 2023-10-16T11:21:13.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e0d81b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e0815dbd0 0x7f9e080d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e0d81b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e080ce490 0x7f9e0806b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e0d81b700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0815dfb0 con 0x7f9e080d0610 2023-10-16T11:21:13.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e0d81b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e080cd3a0 con 0x7f9e080ce490 2023-10-16T11:21:13.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e0d81b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e080cd830 con 0x7f9e0815dbd0 2023-10-16T11:21:13.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e077fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e080ce490 0x7f9e0806b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e077fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e080ce490 0x7f9e0806b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59576/0 (socket says 172.21.15.181:59576) 2023-10-16T11:21:13.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.573+0000 7f9e077fe700 1 -- 172.21.15.181:0/2392243126 learned_addr learned my addr 172.21.15.181:0/2392243126 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:13.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e077fe700 1 -- 172.21.15.181:0/2392243126 >> v2:172.21.15.190:3300/0 conn(0x7f9e0815dbd0 msgr2=0x7f9e080d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:13.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e077fe700 1 --2- 172.21.15.181:0/2392243126 >> v2:172.21.15.190:3300/0 conn(0x7f9e0815dbd0 0x7f9e080d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e077fe700 1 -- 172.21.15.181:0/2392243126 >> v1:172.21.15.190:6789/0 conn(0x7f9e080d0610 legacy=0x7f9e080cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:13.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e077fe700 1 -- 172.21.15.181:0/2392243126 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080d1fc0 con 0x7f9e080ce490 2023-10-16T11:21:13.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e077fe700 1 --2- 172.21.15.181:0/2392243126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e080ce490 0x7f9e0806b6f0 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f9e080729b0 tx=0x7f9e0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2853c9108ac17f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e05ffb700 1 -- 172.21.15.181:0/2392243126 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000e070 con 0x7f9e080ce490 2023-10-16T11:21:13.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e05ffb700 1 -- 172.21.15.181:0/2392243126 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e00003a20 con 0x7f9e080ce490 2023-10-16T11:21:13.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e05ffb700 1 -- 172.21.15.181:0/2392243126 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00003dd0 con 0x7f9e080ce490 2023-10-16T11:21:13.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e0d81b700 1 -- 172.21.15.181:0/2392243126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e080ce490 msgr2=0x7f9e0806b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/2392243126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e080ce490 0x7f9e0806b6f0 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f9e080729b0 tx=0x7f9e0000b330 comp rx=0 tx=0).stop 2023-10-16T11:21:13.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e0d81b700 1 -- 172.21.15.181:0/2392243126 shutdown_connections 2023-10-16T11:21:13.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/2392243126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e080ce490 0x7f9e0806b6f0 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.574+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/2392243126 >> v2:172.21.15.190:3300/0 conn(0x7f9e0815dbd0 0x7f9e080d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.575+0000 7f9e0d81b700 1 -- 172.21.15.181:0/2392243126 >> 172.21.15.181:0/2392243126 conn(0x7f9e0809a930 msgr2=0x7f9e080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:13.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.575+0000 7f9e0d81b700 1 -- 172.21.15.181:0/2392243126 shutdown_connections 2023-10-16T11:21:13.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.575+0000 7f9e0d81b700 1 -- 172.21.15.181:0/2392243126 wait complete. 2023-10-16T11:21:13.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.576+0000 7f9e0d81b700 1 Processor -- start 2023-10-16T11:21:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.576+0000 7f9e0d81b700 1 -- start start 2023-10-16T11:21:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e0d81b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e080ce490 0x7f9e0814af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e0d81b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e080d0610 0x7f9e081604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e0d81b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e0815dbd0 0x7f9e081f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e0d81b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080b7460 con 0x7f9e0815dbd0 2023-10-16T11:21:13.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e0d81b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0814b4c0 con 0x7f9e080ce490 2023-10-16T11:21:13.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e0d81b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0815b3d0 con 0x7f9e080d0610 2023-10-16T11:21:13.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e0815dbd0 0x7f9e081f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e0815dbd0 0x7f9e081f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59582/0 (socket says 172.21.15.181:59582) 2023-10-16T11:21:13.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 -- 172.21.15.181:0/883923520 learned_addr learned my addr 172.21.15.181:0/883923520 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:13.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e06ffd700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e080ce490 0x7f9e0814af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e067fc700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e080d0610 0x7f9e081604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 -- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e080d0610 msgr2=0x7f9e081604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e080d0610 0x7f9e081604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 -- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e080ce490 msgr2=0x7f9e0814af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e080ce490 0x7f9e0814af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 -- 172.21.15.181:0/883923520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e00009b30 con 0x7f9e0815dbd0 2023-10-16T11:21:13.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e06ffd700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e080ce490 0x7f9e0814af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:13.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e077fe700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e0815dbd0 0x7f9e081f6bf0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f9e00015040 tx=0x7f9e00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9deffff700 1 -- 172.21.15.181:0/883923520 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00004240 con 0x7f9e0815dbd0 2023-10-16T11:21:13.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9deffff700 1 -- 172.21.15.181:0/883923520 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e000043a0 con 0x7f9e0815dbd0 2023-10-16T11:21:13.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0815b6b0 con 0x7f9e0815dbd0 2023-10-16T11:21:13.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.577+0000 7f9deffff700 1 -- 172.21.15.181:0/883923520 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00004750 con 0x7f9e0815dbd0 2023-10-16T11:21:13.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.578+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0815bc70 con 0x7f9e0815dbd0 2023-10-16T11:21:13.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.578+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0806c410 con 0x7f9e0815dbd0 2023-10-16T11:21:13.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.579+0000 7f9deffff700 1 -- 172.21.15.181:0/883923520 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9e00019070 con 0x7f9e0815dbd0 2023-10-16T11:21:13.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.579+0000 7f9deffff700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f9df406b140 0x7f9df406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.580+0000 7f9deffff700 1 -- 172.21.15.181:0/883923520 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f9e0008ac80 con 0x7f9e0815dbd0 2023-10-16T11:21:13.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.580+0000 7f9e06ffd700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f9df406b140 0x7f9df406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.580+0000 7f9e06ffd700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f9df406b140 0x7f9df406d600 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc006fd0 tx=0x7f9dfc008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.584+0000 7f9deffff700 1 -- 172.21.15.181:0/883923520 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9e000555f0 con 0x7f9e0815dbd0 2023-10-16T11:21:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.777+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e0815dfb0 con 0x7f9e0815dbd0 2023-10-16T11:21:13.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.777+0000 7f9deffff700 1 -- 172.21.15.181:0/883923520 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v248) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e00017020 con 0x7f9e0815dbd0 2023-10-16T11:21:13.780 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:21:13.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f9df406b140 msgr2=0x7f9df406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f9df406b140 0x7f9df406d600 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc006fd0 tx=0x7f9dfc008040 comp rx=0 tx=0).stop 2023-10-16T11:21:13.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e0815dbd0 msgr2=0x7f9e081f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e0815dbd0 0x7f9e081f6bf0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f9e00015040 tx=0x7f9e00004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 shutdown_connections 2023-10-16T11:21:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f9df406b140 0x7f9df406d600 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e0815dbd0 0x7f9e081f6bf0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e080ce490 0x7f9e0814af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 --2- 172.21.15.181:0/883923520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e080d0610 0x7f9e081604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.779+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 >> 172.21.15.181:0/883923520 conn(0x7f9e0809a930 msgr2=0x7f9e080ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.780+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 shutdown_connections 2023-10-16T11:21:13.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.780+0000 7f9e0d81b700 1 -- 172.21.15.181:0/883923520 wait complete. 2023-10-16T11:21:13.794 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-16T11:21:13.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.912+0000 7fbd4385e700 1 Processor -- start 2023-10-16T11:21:13.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.913+0000 7fbd4385e700 1 -- start start 2023-10-16T11:21:13.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.913+0000 7fbd4385e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd3c15dc20 0x7fbd3c0d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.913+0000 7fbd4385e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c0d1e60 0x7fbd3c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.913+0000 7fbd4385e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3c0b2f00 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.913+0000 7fbd4385e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c0b3b80 con 0x7fbd3c15dc20 2023-10-16T11:21:13.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.913+0000 7fbd4385e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0d2820 con 0x7fbd3c0d1e60 2023-10-16T11:21:13.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd40df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c0d1e60 0x7fbd3c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd415fa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd3c15dc20 0x7fbd3c0d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd41dfb700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbd3c0ce4b0 0x7fbd3c0ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:54610/0 (socket says 172.21.15.181:54610) 2023-10-16T11:21:13.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd40df9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c0d1e60 0x7fbd3c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:56614/0 (socket says 172.21.15.181:56614) 2023-10-16T11:21:13.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd415fa700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd3c15dc20 0x7fbd3c0d41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:36404/0 (socket says 172.21.15.181:36404) 2023-10-16T11:21:13.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd41dfb700 1 -- 172.21.15.181:0/426694120 learned_addr learned my addr 172.21.15.181:0/426694120 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:13.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd40df9700 1 -- 172.21.15.181:0/426694120 >> v2:172.21.15.190:3300/0 conn(0x7fbd3c15dc20 msgr2=0x7fbd3c0d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:13.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd40df9700 1 --2- 172.21.15.181:0/426694120 >> v2:172.21.15.190:3300/0 conn(0x7fbd3c15dc20 0x7fbd3c0d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd40df9700 1 -- 172.21.15.181:0/426694120 >> v1:172.21.15.190:6789/0 conn(0x7fbd3c0ce4b0 legacy=0x7fbd3c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd40df9700 1 -- 172.21.15.181:0/426694120 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0b31e0 con 0x7fbd3c0d1e60 2023-10-16T11:21:13.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd415fa700 1 --2- 172.21.15.181:0/426694120 >> v2:172.21.15.190:3300/0 conn(0x7fbd3c15dc20 0x7fbd3c0d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:13.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.914+0000 7fbd40df9700 1 --2- 172.21.15.181:0/426694120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c0d1e60 0x7fbd3c0d22e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009840 tx=0x7fbd38009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674b3f93dd4a8d55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd33fff700 1 -- 172.21.15.181:0/426694120 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38011830 con 0x7fbd3c0d1e60 2023-10-16T11:21:13.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd33fff700 1 -- 172.21.15.181:0/426694120 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd38011990 con 0x7fbd3c0d1e60 2023-10-16T11:21:13.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd33fff700 1 -- 172.21.15.181:0/426694120 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38011d20 con 0x7fbd3c0d1e60 2023-10-16T11:21:13.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 -- 172.21.15.181:0/426694120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c0d1e60 msgr2=0x7fbd3c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 --2- 172.21.15.181:0/426694120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c0d1e60 0x7fbd3c0d22e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009840 tx=0x7fbd38009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:13.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 -- 172.21.15.181:0/426694120 shutdown_connections 2023-10-16T11:21:13.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 --2- 172.21.15.181:0/426694120 >> v2:172.21.15.190:3300/0 conn(0x7fbd3c15dc20 0x7fbd3c0d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 --2- 172.21.15.181:0/426694120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c0d1e60 0x7fbd3c0d22e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 -- 172.21.15.181:0/426694120 >> 172.21.15.181:0/426694120 conn(0x7fbd3c08af90 msgr2=0x7fbd3c0d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:13.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 -- 172.21.15.181:0/426694120 shutdown_connections 2023-10-16T11:21:13.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.915+0000 7fbd4385e700 1 -- 172.21.15.181:0/426694120 wait complete. 2023-10-16T11:21:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.916+0000 7fbd4385e700 1 Processor -- start 2023-10-16T11:21:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.916+0000 7fbd4385e700 1 -- start start 2023-10-16T11:21:13.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd4385e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c0ce4b0 0x7fbd3c1f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd4385e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd3c0d1e60 0x7fbd3c1f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd4385e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c15dc20 0x7fbd3c09f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd4385e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1f44d0 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.928 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd4385e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1f3f10 con 0x7fbd3c0d1e60 2023-10-16T11:21:13.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd4385e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1f41f0 con 0x7fbd3c15dc20 2023-10-16T11:21:13.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd415fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c0ce4b0 0x7fbd3c1f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd415fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c0ce4b0 0x7fbd3c1f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59592/0 (socket says 172.21.15.181:59592) 2023-10-16T11:21:13.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd415fa700 1 -- 172.21.15.181:0/3194799063 learned_addr learned my addr 172.21.15.181:0/3194799063 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:13.929 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd41dfb700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c15dc20 0x7fbd3c09f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.917+0000 7fbd40df9700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd3c0d1e60 0x7fbd3c1f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd415fa700 1 -- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c15dc20 msgr2=0x7fbd3c09f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd415fa700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c15dc20 0x7fbd3c09f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.930 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd415fa700 1 -- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd3c0d1e60 msgr2=0x7fbd3c1f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:13.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd415fa700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd3c0d1e60 0x7fbd3c1f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:13.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd415fa700 1 -- 172.21.15.181:0/3194799063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c009430 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd40df9700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd3c0d1e60 0x7fbd3c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:13.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd415fa700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c0ce4b0 0x7fbd3c1f0810 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00ebe0 tx=0x7fbd2c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.931 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd31ffb700 1 -- 172.21.15.181:0/3194799063 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c005a20 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd31ffb700 1 -- 172.21.15.181:0/3194799063 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c005b80 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd31ffb700 1 -- 172.21.15.181:0/3194799063 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00c850 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd380094a0 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.918+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c09eef0 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.932 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.919+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c0b23c0 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.919+0000 7fbd31ffb700 1 -- 172.21.15.181:0/3194799063 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbd2c00c9b0 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.920+0000 7fbd31ffb700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbd2006b140 0x7fbd2006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.920+0000 7fbd31ffb700 1 -- 172.21.15.181:0/3194799063 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fbd2c089f40 con 0x7fbd3c0ce4b0 2023-10-16T11:21:13.933 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.920+0000 7fbd40df9700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbd2006b140 0x7fbd2006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:13.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.920+0000 7fbd40df9700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbd2006b140 0x7fbd2006d600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c00 tx=0x7fbd38008030 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:13.934 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:13.925+0000 7fbd31ffb700 1 -- 172.21.15.181:0/3194799063 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbd2c0549b0 con 0x7fbd3c0ce4b0 2023-10-16T11:21:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:14.199+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fbd3c0d1aa0 con 0x7fbd3c0ce4b0 2023-10-16T11:21:15.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.096+0000 7fbd31ffb700 1 -- 172.21.15.181:0/3194799063 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3203) v1 ==== 158+0+0 (secure 0 0 0) 0x7fbd2c020020 con 0x7fbd3c0ce4b0 2023-10-16T11:21:15.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbd2006b140 msgr2=0x7fbd2006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbd2006b140 0x7fbd2006d600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c00 tx=0x7fbd38008030 comp rx=0 tx=0).stop 2023-10-16T11:21:15.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c0ce4b0 msgr2=0x7fbd3c1f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c0ce4b0 0x7fbd3c1f0810 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00ebe0 tx=0x7fbd2c00eef0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 shutdown_connections 2023-10-16T11:21:15.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd3c15dc20 0x7fbd3c09f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbd2006b140 0x7fbd2006d600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd3c0ce4b0 0x7fbd3c1f0810 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 --2- 172.21.15.181:0/3194799063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd3c0d1e60 0x7fbd3c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 >> 172.21.15.181:0/3194799063 conn(0x7fbd3c08af90 msgr2=0x7fbd3c0ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:15.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 shutdown_connections 2023-10-16T11:21:15.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.099+0000 7fbd4385e700 1 -- 172.21.15.181:0/3194799063 wait complete. 2023-10-16T11:21:15.117 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:21:15.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.244+0000 7f1f5f12f700 1 Processor -- start 2023-10-16T11:21:15.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.244+0000 7f1f5f12f700 1 -- start start 2023-10-16T11:21:15.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5f12f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f58159e10 0x7f1f580ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5f12f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f5815a1f0 0x7f1f5815db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5f12f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5806e4b0 con 0x7f1f5806bef0 2023-10-16T11:21:15.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5f12f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5806ed50 con 0x7f1f5815a1f0 2023-10-16T11:21:15.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5f12f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f580ceea0 con 0x7f1f58159e10 2023-10-16T11:21:15.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f5815a1f0 0x7f1f5815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f5815a1f0 0x7f1f5815db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59602/0 (socket says 172.21.15.181:59602) 2023-10-16T11:21:15.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 -- 172.21.15.181:0/4052760327 learned_addr learned my addr 172.21.15.181:0/4052760327 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:15.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5cecb700 1 --2- 172.21.15.181:0/4052760327 >> v2:172.21.15.190:3300/0 conn(0x7f1f58159e10 0x7f1f580ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 -- 172.21.15.181:0/4052760327 >> v2:172.21.15.190:3300/0 conn(0x7f1f58159e10 msgr2=0x7f1f580ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:15.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 --2- 172.21.15.181:0/4052760327 >> v2:172.21.15.190:3300/0 conn(0x7f1f58159e10 0x7f1f580ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 -- 172.21.15.181:0/4052760327 >> v1:172.21.15.190:6789/0 conn(0x7f1f5806bef0 legacy=0x7f1f580caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 -- 172.21.15.181:0/4052760327 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5806e700 con 0x7f1f5815a1f0 2023-10-16T11:21:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f5d6cc700 1 --2- 172.21.15.181:0/4052760327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5815a1f0 0x7f1f5815db90 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f1f58067360 tx=0x7f1f5000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0e61b5182a17c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f577fe700 1 -- 172.21.15.181:0/4052760327 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50010070 con 0x7f1f5815a1f0 2023-10-16T11:21:15.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f577fe700 1 -- 172.21.15.181:0/4052760327 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000dd50 con 0x7f1f5815a1f0 2023-10-16T11:21:15.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.245+0000 7f1f577fe700 1 -- 172.21.15.181:0/4052760327 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000b310 con 0x7f1f5815a1f0 2023-10-16T11:21:15.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 -- 172.21.15.181:0/4052760327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5815a1f0 msgr2=0x7f1f5815db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/4052760327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5815a1f0 0x7f1f5815db90 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f1f58067360 tx=0x7f1f5000d430 comp rx=0 tx=0).stop 2023-10-16T11:21:15.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 -- 172.21.15.181:0/4052760327 shutdown_connections 2023-10-16T11:21:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/4052760327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5815a1f0 0x7f1f5815db90 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/4052760327 >> v2:172.21.15.190:3300/0 conn(0x7f1f58159e10 0x7f1f580ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 -- 172.21.15.181:0/4052760327 >> 172.21.15.181:0/4052760327 conn(0x7f1f580cc730 msgr2=0x7f1f580ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 -- 172.21.15.181:0/4052760327 shutdown_connections 2023-10-16T11:21:15.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.246+0000 7f1f5f12f700 1 -- 172.21.15.181:0/4052760327 wait complete. 2023-10-16T11:21:15.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.247+0000 7f1f5f12f700 1 Processor -- start 2023-10-16T11:21:15.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.247+0000 7f1f5f12f700 1 -- start start 2023-10-16T11:21:15.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5f12f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5806bef0 0x7f1f580c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5f12f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58159e10 0x7f1f5815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5f12f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f5815a1f0 0x7f1f5815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5f12f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580c6270 con 0x7f1f5806bef0 2023-10-16T11:21:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5f12f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5814bb70 con 0x7f1f5815a1f0 2023-10-16T11:21:15.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5f12f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5814be50 con 0x7f1f58159e10 2023-10-16T11:21:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5806bef0 0x7f1f580c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5806bef0 0x7f1f580c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59604/0 (socket says 172.21.15.181:59604) 2023-10-16T11:21:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 -- 172.21.15.181:0/2948164269 learned_addr learned my addr 172.21.15.181:0/2948164269 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5d6cc700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f5815a1f0 0x7f1f5815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f57fff700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58159e10 0x7f1f5815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 -- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58159e10 msgr2=0x7f1f5815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58159e10 0x7f1f5815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 -- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f5815a1f0 msgr2=0x7f1f5815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f5815a1f0 0x7f1f5815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f5cecb700 1 -- 172.21.15.181:0/2948164269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5000bd40 con 0x7f1f5806bef0 2023-10-16T11:21:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.248+0000 7f1f57fff700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58159e10 0x7f1f5815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.249+0000 7f1f5cecb700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5806bef0 0x7f1f580c5d30 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00e810 tx=0x7f1f4c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.249+0000 7f1f557fa700 1 -- 172.21.15.181:0/2948164269 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c013610 con 0x7f1f5806bef0 2023-10-16T11:21:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.249+0000 7f1f557fa700 1 -- 172.21.15.181:0/2948164269 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c013770 con 0x7f1f5806bef0 2023-10-16T11:21:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.249+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5814c130 con 0x7f1f5806bef0 2023-10-16T11:21:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.249+0000 7f1f557fa700 1 -- 172.21.15.181:0/2948164269 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c013b20 con 0x7f1f5806bef0 2023-10-16T11:21:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.249+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5814c6f0 con 0x7f1f5806bef0 2023-10-16T11:21:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.250+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f580cd5d0 con 0x7f1f5806bef0 2023-10-16T11:21:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.250+0000 7f1f557fa700 1 -- 172.21.15.181:0/2948164269 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1f4c013c80 con 0x7f1f5806bef0 2023-10-16T11:21:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.251+0000 7f1f557fa700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f3c06b140 0x7f1f3c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.251+0000 7f1f57fff700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f3c06b140 0x7f1f3c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.251+0000 7f1f557fa700 1 -- 172.21.15.181:0/2948164269 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1f4c089a40 con 0x7f1f5806bef0 2023-10-16T11:21:15.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.251+0000 7f1f57fff700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f3c06b140 0x7f1f3c06d600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1f5814aa10 tx=0x7f1f44009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:15.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.255+0000 7f1f557fa700 1 -- 172.21.15.181:0/2948164269 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1f4c08be60 con 0x7f1f5806bef0 2023-10-16T11:21:15.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.449+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1f5814ca60 con 0x7f1f5806bef0 2023-10-16T11:21:15.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.453+0000 7f1f557fa700 1 -- 172.21.15.181:0/2948164269 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v88)=0 v88) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f4c0543b0 con 0x7f1f5806bef0 2023-10-16T11:21:15.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f3c06b140 msgr2=0x7f1f3c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f3c06b140 0x7f1f3c06d600 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1f5814aa10 tx=0x7f1f44009040 comp rx=0 tx=0).stop 2023-10-16T11:21:15.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5806bef0 msgr2=0x7f1f580c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5806bef0 0x7f1f580c5d30 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00e810 tx=0x7f1f4c00eb20 comp rx=0 tx=0).stop 2023-10-16T11:21:15.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 shutdown_connections 2023-10-16T11:21:15.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f3c06b140 0x7f1f3c06d600 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f5815a1f0 0x7f1f5815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5806bef0 0x7f1f580c5d30 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 --2- 172.21.15.181:0/2948164269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58159e10 0x7f1f5815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 >> 172.21.15.181:0/2948164269 conn(0x7f1f580cc730 msgr2=0x7f1f5806abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:15.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 shutdown_connections 2023-10-16T11:21:15.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.455+0000 7f1f5f12f700 1 -- 172.21.15.181:0/2948164269 wait complete. 2023-10-16T11:21:15.469 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:21:15.470 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:21:15.514 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:21:15.565 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:21:15.566 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:15.566 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:15.584 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:21:15.584 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:21:15.585 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:15.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:21:15.605 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:15.605 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:15.641 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:21:15.642 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:21:15.642 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:15.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:21:15.665 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:21:15.666 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:15.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.799+0000 7f1d32d7a700 1 Processor -- start 2023-10-16T11:21:15.802 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.799+0000 7f1d32d7a700 1 -- start start 2023-10-16T11:21:15.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.800+0000 7f1d32d7a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 0x7f1d2c1611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.800+0000 7f1d32d7a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c06ce20 con 0x7f1d2c161b60 2023-10-16T11:21:15.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.800+0000 7f1d32d7a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c0d45f0 con 0x7f1d2c06d5f0 2023-10-16T11:21:15.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.800+0000 7f1d32d7a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c162800 con 0x7f1d2c160d20 2023-10-16T11:21:15.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.800+0000 7f1d30b16700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1d2c161b60 0x7f1d2c161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42458/0 (socket says 172.21.15.181:42458) 2023-10-16T11:21:15.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.800+0000 7f1d30b16700 1 -- 172.21.15.181:0/1612475616 learned_addr learned my addr 172.21.15.181:0/1612475616 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:15.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789000878 0 0) 0x7f1d2c06ce20 con 0x7f1d2c161b60 2023-10-16T11:21:15.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d1c0035b0 con 0x7f1d2c161b60 2023-10-16T11:21:15.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 0x7f1d2c1611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858861308 0 0) 0x7f1d1c0035b0 con 0x7f1d2c161b60 2023-10-16T11:21:15.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 msgr2=0x7f1d2c1611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:15.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 0x7f1d2c1611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 >> v1:172.21.15.190:6789/0 conn(0x7f1d2c06d5f0 legacy=0x7f1d2c0d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c06d2d0 con 0x7f1d2c161b60 2023-10-16T11:21:15.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f1d200017f0 con 0x7f1d2c161b60 2023-10-16T11:21:15.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 >> v1:172.21.15.181:6789/0 conn(0x7f1d2c161b60 legacy=0x7f1d2c161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c003a30 0x7f1d1c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 0x7f1d1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c004d90 0x7f1d1c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c162800 con 0x7f1d1c004d90 2023-10-16T11:21:15.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c00bc20 con 0x7f1d1c003a30 2023-10-16T11:21:15.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c00bde0 con 0x7f1d2c160d20 2023-10-16T11:21:15.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c004d90 0x7f1d1c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d30b16700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c003a30 0x7f1d1c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2bfff700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 0x7f1d1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 -- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 msgr2=0x7f1d1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:15.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 0x7f1d1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 -- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c003a30 msgr2=0x7f1d1c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:15.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c003a30 0x7f1d1c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 -- 172.21.15.181:0/1612475616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c0035b0 con 0x7f1d1c004d90 2023-10-16T11:21:15.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d2bfff700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c160d20 0x7f1d1c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:15.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.801+0000 7f1d31317700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c004d90 0x7f1d1c005210 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f1d24007360 tx=0x7f1d1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88f9c9476a69ea4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:15.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24007e80 con 0x7f1d1c004d90 2023-10-16T11:21:15.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d24014400 con 0x7f1d1c004d90 2023-10-16T11:21:15.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d2b7fe700 1 -- 172.21.15.181:0/1612475616 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24014790 con 0x7f1d1c004d90 2023-10-16T11:21:15.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d32d7a700 1 -- 172.21.15.181:0/1612475616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c004d90 msgr2=0x7f1d1c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d32d7a700 1 --2- 172.21.15.181:0/1612475616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c004d90 0x7f1d1c005210 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f1d24007360 tx=0x7f1d1c005880 comp rx=0 tx=0).stop 2023-10-16T11:21:15.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d31317700 1 -- 172.21.15.181:0/1612475616 reap_dead start 2023-10-16T11:21:15.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d32d7a700 1 -- 172.21.15.181:0/1612475616 shutdown_connections 2023-10-16T11:21:15.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d32d7a700 1 -- 172.21.15.181:0/1612475616 >> 172.21.15.181:0/1612475616 conn(0x7f1d2c09cba0 msgr2=0x7f1d2c0d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:15.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d32d7a700 1 -- 172.21.15.181:0/1612475616 shutdown_connections 2023-10-16T11:21:15.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.802+0000 7f1d32d7a700 1 -- 172.21.15.181:0/1612475616 wait complete. 2023-10-16T11:21:15.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.803+0000 7f1d32d7a700 1 Processor -- start 2023-10-16T11:21:15.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.803+0000 7f1d32d7a700 1 -- start start 2023-10-16T11:21:15.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d32d7a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c004d90 0x7f1d2c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d32d7a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c0c91b0 0x7f1d2c0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d32d7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2c0b4ce0 0x7f1d2c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d32d7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b0ed0 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d32d7a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b56d0 con 0x7f1d1c004d90 2023-10-16T11:21:15.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d32d7a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b12f0 con 0x7f1d2c0c91b0 2023-10-16T11:21:15.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d31317700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2c0b4ce0 0x7f1d2c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d31317700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2c0b4ce0 0x7f1d2c0b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59626/0 (socket says 172.21.15.181:59626) 2023-10-16T11:21:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.804+0000 7f1d31317700 1 -- 172.21.15.181:0/3920623836 learned_addr learned my addr 172.21.15.181:0/3920623836 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d30b16700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c004d90 0x7f1d2c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d2bfff700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c0c91b0 0x7f1d2c0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d31317700 1 -- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c0c91b0 msgr2=0x7f1d2c0c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d31317700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c0c91b0 0x7f1d2c0c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d31317700 1 -- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c004d90 msgr2=0x7f1d2c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d31317700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c004d90 0x7f1d2c15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d31317700 1 -- 172.21.15.181:0/3920623836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c15cf40 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d2bfff700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c0c91b0 0x7f1d2c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:15.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d31317700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2c0b4ce0 0x7f1d2c0b5190 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f1d24007950 tx=0x7f1d24004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:15.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d297fa700 1 -- 172.21.15.181:0/3920623836 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24007000 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d297fa700 1 -- 172.21.15.181:0/3920623836 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2401e410 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c0b0530 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d297fa700 1 -- 172.21.15.181:0/3920623836 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24009410 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.805+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c0b0af0 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.806+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c04ebb0 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.806+0000 7f1d297fa700 1 -- 172.21.15.181:0/3920623836 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1d2401b050 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.807+0000 7f1d297fa700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1d1006b140 0x7f1d1006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:15.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.807+0000 7f1d30b16700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1d1006b140 0x7f1d1006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:15.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.807+0000 7f1d297fa700 1 -- 172.21.15.181:0/3920623836 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1d2408af80 con 0x7f1d2c0b4ce0 2023-10-16T11:21:15.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.807+0000 7f1d30b16700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1d1006b140 0x7f1d1006d600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0b1c30 tx=0x7f1d20009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:15.823 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:15.812+0000 7f1d297fa700 1 -- 172.21.15.181:0/3920623836 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1d240bdfe0 con 0x7f1d2c0b4ce0 2023-10-16T11:21:16.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.051+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d2c0b5e80 con 0x7f1d2c0b4ce0 2023-10-16T11:21:16.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.051+0000 7f1d297fa700 1 -- 172.21.15.181:0/3920623836 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+3718 (secure 0 0 0) 0x7f1d24019090 con 0x7f1d2c0b4ce0 2023-10-16T11:21:16.054 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:16.054 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":137,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3346063539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3346063539},{"type":"v1","addr":"172.21.15.181:6837","nonce":3346063539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9377,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3663068493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3663068493},{"type":"v1","addr":"172.21.15.190:6835","nonce":3663068493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/2478528976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2478528976},{"type":"v1","addr":"172.21.15.190:6834","nonce":2478528976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":137},{"gid":12747,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2505407408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2505407408},{"type":"v1","addr":"172.21.15.181:6836","nonce":2505407408}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[]} 2023-10-16T11:21:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.053+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1d1006b140 msgr2=0x7f1d1006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.053+0000 7f1d32d7a700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1d1006b140 0x7f1d1006d600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0b1c30 tx=0x7f1d20009040 comp rx=0 tx=0).stop 2023-10-16T11:21:16.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.053+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2c0b4ce0 msgr2=0x7f1d2c0b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.053+0000 7f1d32d7a700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2c0b4ce0 0x7f1d2c0b5190 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f1d24007950 tx=0x7f1d24004b40 comp rx=0 tx=0).stop 2023-10-16T11:21:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 shutdown_connections 2023-10-16T11:21:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1d1006b140 0x7f1d1006d600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2c0b4ce0 0x7f1d2c0b5190 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d1c004d90 0x7f1d2c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 --2- 172.21.15.181:0/3920623836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1d2c0c91b0 0x7f1d2c0c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 >> 172.21.15.181:0/3920623836 conn(0x7f1d2c09cba0 msgr2=0x7f1d2c15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 shutdown_connections 2023-10-16T11:21:16.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.054+0000 7f1d32d7a700 1 -- 172.21.15.181:0/3920623836 wait complete. 2023-10-16T11:21:16.059 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 137 2023-10-16T11:21:16.068 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:21:16.068 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:21:16.069 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:21:16.069 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:16.069 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:21:16.070 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:21:16.070 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:21:16.070 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:16.070 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:21:16.071 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:21:16.071 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:21:16.071 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:16.072 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:21:16.072 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:21:16.072 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:21:16.072 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:16.077 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:21:16.078 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:21:16.079 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:21:16.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:21:16.082 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:21:16.082 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:21:16.083 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:21:16.084 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:21:16.121 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:21:16.121 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:16.122 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:21:16.122 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:16.125 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:21:16.125 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:16.126 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:21:16.126 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:16.127 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:21:16.200 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:21:16.201 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:21:16.208 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:21:16.209 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:21:16.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.253+0000 7f18a1e63700 1 Processor -- start 2023-10-16T11:21:16.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.253+0000 7f18a1e63700 1 -- start start 2023-10-16T11:21:16.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f18a1e63700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f189c15b9f0 0x7f189c0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:16.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f18a1e63700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c0cc400 0x7f189c0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f18a1e63700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f189c0cf980 0x7f189c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f18a1e63700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c06b800 con 0x7f189c15b9f0 2023-10-16T11:21:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f18a1e63700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c06b370 con 0x7f189c0cf980 2023-10-16T11:21:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f18a1e63700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0cfdc0 con 0x7f189c0cc400 2023-10-16T11:21:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f189c15b9f0 0x7f189c0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f189c15b9f0 0x7f189c0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59654/0 (socket says 172.21.15.181:59654) 2023-10-16T11:21:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 -- 172.21.15.181:0/1708682668 learned_addr learned my addr 172.21.15.181:0/1708682668 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189bfff700 1 --2- 172.21.15.181:0/1708682668 >> v2:172.21.15.190:3300/0 conn(0x7f189c0cf980 0x7f189c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189affd700 1 --2- 172.21.15.181:0/1708682668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c0cc400 0x7f189c0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 -- 172.21.15.181:0/1708682668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c0cc400 msgr2=0x7f189c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 --2- 172.21.15.181:0/1708682668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c0cc400 0x7f189c0cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 -- 172.21.15.181:0/1708682668 >> v2:172.21.15.190:3300/0 conn(0x7f189c0cf980 msgr2=0x7f189c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 --2- 172.21.15.181:0/1708682668 >> v2:172.21.15.190:3300/0 conn(0x7f189c0cf980 0x7f189c0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 -- 172.21.15.181:0/1708682668 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c06aa40 con 0x7f189c15b9f0 2023-10-16T11:21:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189bfff700 1 --2- 172.21.15.181:0/1708682668 >> v2:172.21.15.190:3300/0 conn(0x7f189c0cf980 0x7f189c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189b7fe700 1 --2- 172.21.15.181:0/1708682668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c15b9f0 0x7f189c0cd950 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f189c071f30 tx=0x7f188c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5887147936877907 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189a7fc700 1 -- 172.21.15.181:0/1708682668 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c005e30 con 0x7f189c15b9f0 2023-10-16T11:21:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189a7fc700 1 -- 172.21.15.181:0/1708682668 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c004030 con 0x7f189c15b9f0 2023-10-16T11:21:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.254+0000 7f189a7fc700 1 -- 172.21.15.181:0/1708682668 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0043e0 con 0x7f189c15b9f0 2023-10-16T11:21:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 -- 172.21.15.181:0/1708682668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c15b9f0 msgr2=0x7f189c0cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1708682668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c15b9f0 0x7f189c0cd950 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f189c071f30 tx=0x7f188c009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 -- 172.21.15.181:0/1708682668 shutdown_connections 2023-10-16T11:21:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1708682668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c15b9f0 0x7f189c0cd950 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1708682668 >> v2:172.21.15.190:3300/0 conn(0x7f189c0cf980 0x7f189c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1708682668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c0cc400 0x7f189c0cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 -- 172.21.15.181:0/1708682668 >> 172.21.15.181:0/1708682668 conn(0x7f189c0d04a0 msgr2=0x7f189c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 -- 172.21.15.181:0/1708682668 shutdown_connections 2023-10-16T11:21:16.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.255+0000 7f18a1e63700 1 -- 172.21.15.181:0/1708682668 wait complete. 2023-10-16T11:21:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.257+0000 7f18a1e63700 1 Processor -- start 2023-10-16T11:21:16.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.257+0000 7f18a1e63700 1 -- start start 2023-10-16T11:21:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.257+0000 7f18a1e63700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f189c0cc400 0x7f189c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.257+0000 7f18a1e63700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c0cf980 0x7f189c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.257+0000 7f18a1e63700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c15b9f0 0x7f189c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.257+0000 7f18a1e63700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0b62a0 con 0x7f189c0cf980 2023-10-16T11:21:16.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f18a1e63700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0b53c0 con 0x7f189c0cc400 2023-10-16T11:21:16.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f18a1e63700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0b5690 con 0x7f189c15b9f0 2023-10-16T11:21:16.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c0cf980 0x7f189c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:16.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c0cf980 0x7f189c0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59662/0 (socket says 172.21.15.181:59662) 2023-10-16T11:21:16.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 -- 172.21.15.181:0/1179245893 learned_addr learned my addr 172.21.15.181:0/1179245893 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:16.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189b7fe700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f189c0cc400 0x7f189c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189bfff700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c15b9f0 0x7f189c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 -- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c15b9f0 msgr2=0x7f189c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c15b9f0 0x7f189c1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 -- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f189c0cc400 msgr2=0x7f189c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f189c0cc400 0x7f189c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 -- 172.21.15.181:0/1179245893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c009500 con 0x7f189c0cf980 2023-10-16T11:21:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189b7fe700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f189c0cc400 0x7f189c0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f189affd700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c0cf980 0x7f189c0c9df0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f189000e810 tx=0x7f189000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f1883fff700 1 -- 172.21.15.181:0/1179245893 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890013610 con 0x7f189c0cf980 2023-10-16T11:21:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f1883fff700 1 -- 172.21.15.181:0/1179245893 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1890013770 con 0x7f189c0cf980 2023-10-16T11:21:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f1883fff700 1 -- 172.21.15.181:0/1179245893 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890013b00 con 0x7f189c0cf980 2023-10-16T11:21:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c0b5970 con 0x7f189c0cf980 2023-10-16T11:21:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.258+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c0b5f30 con 0x7f189c0cf980 2023-10-16T11:21:16.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.259+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c067d20 con 0x7f189c0cf980 2023-10-16T11:21:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.260+0000 7f1883fff700 1 -- 172.21.15.181:0/1179245893 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1890013c60 con 0x7f189c0cf980 2023-10-16T11:21:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.260+0000 7f1883fff700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f188806b140 0x7f188806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.260+0000 7f1883fff700 1 -- 172.21.15.181:0/1179245893 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1890089af0 con 0x7f189c0cf980 2023-10-16T11:21:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.260+0000 7f189b7fe700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f188806b140 0x7f188806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:16.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.260+0000 7f189b7fe700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f188806b140 0x7f188806d600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f189c071f30 tx=0x7f188c008070 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:16.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.264+0000 7f1883fff700 1 -- 172.21.15.181:0/1179245893 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f18900543c0 con 0x7f189c0cf980 2023-10-16T11:21:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:16.472+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f189c0b63e0 con 0x7f189c0cf980 2023-10-16T11:21:17.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.112+0000 7f1883fff700 1 -- 172.21.15.181:0/1179245893 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v250) v1 ==== 97+0+0 (secure 0 0 0) 0x7f189c0b63e0 con 0x7f189c0cf980 2023-10-16T11:21:17.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f188806b140 msgr2=0x7f188806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f188806b140 0x7f188806d600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f189c071f30 tx=0x7f188c008070 comp rx=0 tx=0).stop 2023-10-16T11:21:17.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c0cf980 msgr2=0x7f189c0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c0cf980 0x7f189c0c9df0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f189000e810 tx=0x7f189000ebd0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 shutdown_connections 2023-10-16T11:21:17.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189c15b9f0 0x7f189c1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f188806b140 0x7f188806d600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f189c0cc400 0x7f189c0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 --2- 172.21.15.181:0/1179245893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f189c0cf980 0x7f189c0c9df0 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 >> 172.21.15.181:0/1179245893 conn(0x7f189c0d04a0 msgr2=0x7f189c15ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:17.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 shutdown_connections 2023-10-16T11:21:17.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.115+0000 7f18a1e63700 1 -- 172.21.15.181:0/1179245893 wait complete. 2023-10-16T11:21:17.120 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T11:21:17.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:21:17.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.256+0000 7ff1e7487700 1 Processor -- start 2023-10-16T11:21:17.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.257+0000 7ff1e7487700 1 -- start start 2023-10-16T11:21:17.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.257+0000 7ff1e7487700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1e00cdb80 0x7ff1e00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.257+0000 7ff1e7487700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e006f870 con 0x7ff1e015dfd0 2023-10-16T11:21:17.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.257+0000 7ff1e7487700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e00d1160 con 0x7ff1e00af450 2023-10-16T11:21:17.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.257+0000 7ff1e7487700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e00cc8e0 con 0x7ff1e00cdb80 2023-10-16T11:21:17.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1e5223700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff1e015dfd0 0x7ff1e006d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42516/0 (socket says 172.21.15.181:42516) 2023-10-16T11:21:17.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1e5223700 1 -- 172.21.15.181:0/417480089 learned_addr learned my addr 172.21.15.181:0/417480089 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:17.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1e5a24700 1 --2- 172.21.15.181:0/417480089 >> v2:172.21.15.190:3300/0 conn(0x7ff1e00cdb80 0x7ff1e00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703685670 0 0) 0x7ff1e006f870 con 0x7ff1e015dfd0 2023-10-16T11:21:17.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1cc0035b0 con 0x7ff1e015dfd0 2023-10-16T11:21:17.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2222575984 0 0) 0x7ff1cc0035b0 con 0x7ff1e015dfd0 2023-10-16T11:21:17.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 >> v2:172.21.15.190:3300/0 conn(0x7ff1e00cdb80 msgr2=0x7ff1e00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:17.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 --2- 172.21.15.181:0/417480089 >> v2:172.21.15.190:3300/0 conn(0x7ff1e00cdb80 0x7ff1e00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 >> v1:172.21.15.190:6789/0 conn(0x7ff1e00af450 legacy=0x7ff1e00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00ccbc0 con 0x7ff1e015dfd0 2023-10-16T11:21:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7ff1d00017f0 con 0x7ff1e015dfd0 2023-10-16T11:21:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 >> v1:172.21.15.181:6789/0 conn(0x7ff1e015dfd0 legacy=0x7ff1e006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc003a00 0x7ff1cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1cc004400 0x7ff1cc0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc005170 0x7ff1cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00cc8e0 con 0x7ff1cc004400 2023-10-16T11:21:17.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc00c0b0 con 0x7ff1cc003a00 2023-10-16T11:21:17.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc00c270 con 0x7ff1cc005170 2023-10-16T11:21:17.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1e4a22700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1cc004400 0x7ff1cc0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1e5223700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc003a00 0x7ff1cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.258+0000 7ff1e5a24700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc005170 0x7ff1cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e4a22700 1 -- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc005170 msgr2=0x7ff1cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:17.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e4a22700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc005170 0x7ff1cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e4a22700 1 -- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc003a00 msgr2=0x7ff1cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:17.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e4a22700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc003a00 0x7ff1cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e5a24700 1 -- 172.21.15.181:0/417480089 reap_dead start 2023-10-16T11:21:17.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e4a22700 1 -- 172.21.15.181:0/417480089 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0035b0 con 0x7ff1cc004400 2023-10-16T11:21:17.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e4a22700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1cc004400 0x7ff1cc0048d0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0018d0 tx=0x7ff1cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab21a83fb10f908 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:17.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0052d0 con 0x7ff1cc004400 2023-10-16T11:21:17.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc005430 con 0x7ff1cc004400 2023-10-16T11:21:17.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1d7fff700 1 -- 172.21.15.181:0/417480089 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0038a0 con 0x7ff1cc004400 2023-10-16T11:21:17.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e7487700 1 -- 172.21.15.181:0/417480089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1cc004400 msgr2=0x7ff1cc0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e7487700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1cc004400 0x7ff1cc0048d0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0018d0 tx=0x7ff1cc005880 comp rx=0 tx=0).stop 2023-10-16T11:21:17.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e7487700 1 -- 172.21.15.181:0/417480089 shutdown_connections 2023-10-16T11:21:17.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e7487700 1 --2- 172.21.15.181:0/417480089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1cc004400 0x7ff1cc0048d0 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e7487700 1 -- 172.21.15.181:0/417480089 >> 172.21.15.181:0/417480089 conn(0x7ff1e00d04a0 msgr2=0x7ff1e00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:17.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.259+0000 7ff1e7487700 1 -- 172.21.15.181:0/417480089 shutdown_connections 2023-10-16T11:21:17.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.260+0000 7ff1e7487700 1 -- 172.21.15.181:0/417480089 wait complete. 2023-10-16T11:21:17.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 Processor -- start 2023-10-16T11:21:17.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 -- start start 2023-10-16T11:21:17.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc004400 0x7ff1e015a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e015f990 0x7ff1e015fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e0158a70 0x7ff1e0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0160450 con 0x7ff1e015f990 2023-10-16T11:21:17.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0160730 con 0x7ff1e0158a70 2023-10-16T11:21:17.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e7487700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0160a10 con 0x7ff1cc004400 2023-10-16T11:21:17.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e4a22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e015f990 0x7ff1e015fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e4a22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e015f990 0x7ff1e015fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59682/0 (socket says 172.21.15.181:59682) 2023-10-16T11:21:17.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.261+0000 7ff1e4a22700 1 -- 172.21.15.181:0/3209442477 learned_addr learned my addr 172.21.15.181:0/3209442477 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:17.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e5223700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc004400 0x7ff1e015a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e5a24700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e0158a70 0x7ff1e0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e4a22700 1 -- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc004400 msgr2=0x7ff1e015a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e4a22700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc004400 0x7ff1e015a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e4a22700 1 -- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e0158a70 msgr2=0x7ff1e0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e4a22700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e0158a70 0x7ff1e0158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e4a22700 1 -- 172.21.15.181:0/3209442477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0160cf0 con 0x7ff1e015f990 2023-10-16T11:21:17.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e5a24700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e0158a70 0x7ff1e0158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e4a22700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e015f990 0x7ff1e015fe10 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c0e0 tx=0x7ff1dc001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1d5ffb700 1 -- 172.21.15.181:0/3209442477 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc006a60 con 0x7ff1e015f990 2023-10-16T11:21:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1d5ffb700 1 -- 172.21.15.181:0/3209442477 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc006be0 con 0x7ff1e015f990 2023-10-16T11:21:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1d5ffb700 1 -- 172.21.15.181:0/3209442477 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc000ed0 con 0x7ff1e015f990 2023-10-16T11:21:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e0160fd0 con 0x7ff1e015f990 2023-10-16T11:21:17.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.262+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e00c80f0 con 0x7ff1e015f990 2023-10-16T11:21:17.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.263+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e0067d20 con 0x7ff1e015f990 2023-10-16T11:21:17.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.264+0000 7ff1d5ffb700 1 -- 172.21.15.181:0/3209442477 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff1dc007030 con 0x7ff1e015f990 2023-10-16T11:21:17.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.264+0000 7ff1d5ffb700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff1c406b140 0x7ff1c406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.264+0000 7ff1d5ffb700 1 -- 172.21.15.181:0/3209442477 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 5409+0+0 (secure 0 0 0) 0x7ff1dc08a7d0 con 0x7ff1e015f990 2023-10-16T11:21:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.264+0000 7ff1e5223700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff1c406b140 0x7ff1c406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.265+0000 7ff1e5223700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff1c406b140 0x7ff1c406d600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0001eb0 tx=0x7ff1d0009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.268+0000 7ff1d5ffb700 1 -- 172.21.15.181:0/3209442477 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff1dc0556d0 con 0x7ff1e015f990 2023-10-16T11:21:17.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.543+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff1e00aee00 con 0x7ff1e015f990 2023-10-16T11:21:17.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.544+0000 7ff1d5ffb700 1 -- 172.21.15.181:0/3209442477 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v53) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff1dc017090 con 0x7ff1e015f990 2023-10-16T11:21:17.546 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:17.547 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:21:17.547 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T11:21:17.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.547 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T11:21:17.548 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T11:21:17.548 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.548 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.548 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:17.549 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:17.549 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.549 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.549 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.550 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T11:21:17.550 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T11:21:17.550 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.550 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.550 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:17.551 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:17.551 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.551 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.552 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T11:21:17.552 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T11:21:17.552 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.552 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.552 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:17.552 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:17.553 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.553 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.553 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T11:21:17.554 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T11:21:17.554 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.554 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.554 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:17.554 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:17.555 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.555 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.555 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T11:21:17.555 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T11:21:17.556 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.556 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.556 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.556 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.556 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.557 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T11:21:17.557 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T11:21:17.557 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.558 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.558 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.558 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.558 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.559 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.559 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T11:21:17.559 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T11:21:17.559 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.559 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.560 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.560 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.560 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.561 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T11:21:17.561 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T11:21:17.561 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.561 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.561 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.561 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.562 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.562 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.562 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T11:21:17.562 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T11:21:17.563 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.563 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.563 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.563 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.564 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.564 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T11:21:17.564 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T11:21:17.565 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.565 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.565 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.565 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.565 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.566 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T11:21:17.566 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T11:21:17.566 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.567 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.567 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.567 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.567 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.568 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T11:21:17.568 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T11:21:17.568 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.568 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:17.568 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:17.569 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.569 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.569 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T11:21:17.569 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T11:21:17.570 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.570 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.570 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T11:21:17.570 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:21:17.570 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.571 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T11:21:17.571 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T11:21:17.571 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.571 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.572 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:21:17.572 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:21:17.572 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:21:17.572 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.573 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T11:21:17.573 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T11:21:17.573 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.573 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.574 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:21:17.574 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:21:17.574 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:21:17.574 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.575 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T11:21:17.575 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T11:21:17.575 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.575 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:17.575 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:21:17.576 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:21:17.576 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:21:17.576 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.576 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T11:21:17.577 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T11:21:17.577 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.577 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:21:17.577 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:17.577 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:17.577 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.578 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.578 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T11:21:17.578 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T11:21:17.579 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.579 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:21:17.579 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:21:17.579 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T11:21:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T11:21:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:21:17.580 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:21:17.581 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:17.581 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.581 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:17.581 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T11:21:17.581 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:21:17.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff1c406b140 msgr2=0x7ff1c406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff1c406b140 0x7ff1c406d600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0001eb0 tx=0x7ff1d0009040 comp rx=0 tx=0).stop 2023-10-16T11:21:17.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e015f990 msgr2=0x7ff1e015fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e015f990 0x7ff1e015fe10 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c0e0 tx=0x7ff1dc001230 comp rx=0 tx=0).stop 2023-10-16T11:21:17.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 shutdown_connections 2023-10-16T11:21:17.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1cc004400 0x7ff1e015a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff1c406b140 0x7ff1c406d600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1e0158a70 0x7ff1e0158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 --2- 172.21.15.181:0/3209442477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1e015f990 0x7ff1e015fe10 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 >> 172.21.15.181:0/3209442477 conn(0x7ff1e00d04a0 msgr2=0x7ff1e00cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:17.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 shutdown_connections 2023-10-16T11:21:17.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.546+0000 7ff1e7487700 1 -- 172.21.15.181:0/3209442477 wait complete. 2023-10-16T11:21:17.586 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:17.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.705+0000 7f7dd4120700 1 Processor -- start 2023-10-16T11:21:17.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.705+0000 7f7dd4120700 1 -- start start 2023-10-16T11:21:17.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.705+0000 7f7dd4120700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc15b8b0 0x7f7dcc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd4120700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7dcc0b22a0 0x7f7dcc15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd4120700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dcc06e010 0x7f7dcc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd4120700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc066c90 con 0x7f7dcc06e010 2023-10-16T11:21:17.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd4120700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc06e980 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd4120700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc066740 con 0x7f7dcc15b8b0 2023-10-16T11:21:17.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dcc06e010 0x7f7dcc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dcc06e010 0x7f7dcc06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59692/0 (socket says 172.21.15.181:59692) 2023-10-16T11:21:17.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 -- 172.21.15.181:0/593056141 learned_addr learned my addr 172.21.15.181:0/593056141 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:17.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd16bb700 1 --2- 172.21.15.181:0/593056141 >> v2:172.21.15.190:3300/0 conn(0x7f7dcc0b22a0 0x7f7dcc15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd1ebc700 1 --2- 172.21.15.181:0/593056141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc15b8b0 0x7f7dcc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 -- 172.21.15.181:0/593056141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc15b8b0 msgr2=0x7f7dcc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 --2- 172.21.15.181:0/593056141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc15b8b0 0x7f7dcc0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 -- 172.21.15.181:0/593056141 >> v2:172.21.15.190:3300/0 conn(0x7f7dcc0b22a0 msgr2=0x7f7dcc15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 --2- 172.21.15.181:0/593056141 >> v2:172.21.15.190:3300/0 conn(0x7f7dcc0b22a0 0x7f7dcc15feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 -- 172.21.15.181:0/593056141 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc1603f0 con 0x7f7dcc06e010 2023-10-16T11:21:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd26bd700 1 --2- 172.21.15.181:0/593056141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc06e010 0x7f7dcc06e440 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc06c780 tx=0x7f7dc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22658279631c488e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd0eba700 1 -- 172.21.15.181:0/593056141 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400e070 con 0x7f7dcc06e010 2023-10-16T11:21:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd0eba700 1 -- 172.21.15.181:0/593056141 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc400b9c0 con 0x7f7dcc06e010 2023-10-16T11:21:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.706+0000 7f7dd0eba700 1 -- 172.21.15.181:0/593056141 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400bd70 con 0x7f7dcc06e010 2023-10-16T11:21:17.713 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 -- 172.21.15.181:0/593056141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc06e010 msgr2=0x7f7dcc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 --2- 172.21.15.181:0/593056141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc06e010 0x7f7dcc06e440 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc06c780 tx=0x7f7dc400b330 comp rx=0 tx=0).stop 2023-10-16T11:21:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 -- 172.21.15.181:0/593056141 shutdown_connections 2023-10-16T11:21:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 --2- 172.21.15.181:0/593056141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc06e010 0x7f7dcc06e440 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 --2- 172.21.15.181:0/593056141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc15b8b0 0x7f7dcc0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 --2- 172.21.15.181:0/593056141 >> v2:172.21.15.190:3300/0 conn(0x7f7dcc0b22a0 0x7f7dcc15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 -- 172.21.15.181:0/593056141 >> 172.21.15.181:0/593056141 conn(0x7f7dcc0b1780 msgr2=0x7f7dcc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 -- 172.21.15.181:0/593056141 shutdown_connections 2023-10-16T11:21:17.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.707+0000 7f7dd4120700 1 -- 172.21.15.181:0/593056141 wait complete. 2023-10-16T11:21:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.708+0000 7f7dd4120700 1 Processor -- start 2023-10-16T11:21:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.708+0000 7f7dd4120700 1 -- start start 2023-10-16T11:21:17.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd4120700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc06e010 0x7f7dcc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd4120700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc0b22a0 0x7f7dcc0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd4120700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc15b8b0 0x7f7dcc06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd4120700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0c7db0 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd4120700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc09fd90 con 0x7f7dcc15b8b0 2023-10-16T11:21:17.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd4120700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0a0030 con 0x7f7dcc06e010 2023-10-16T11:21:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc0b22a0 0x7f7dcc0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc0b22a0 0x7f7dcc0c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59696/0 (socket says 172.21.15.181:59696) 2023-10-16T11:21:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 -- 172.21.15.181:0/4174697118 learned_addr learned my addr 172.21.15.181:0/4174697118 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd1ebc700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc06e010 0x7f7dcc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd26bd700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc15b8b0 0x7f7dcc06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 -- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc06e010 msgr2=0x7f7dcc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc06e010 0x7f7dcc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 -- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc15b8b0 msgr2=0x7f7dcc06d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc15b8b0 0x7f7dcc06d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 -- 172.21.15.181:0/4174697118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4009b30 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dd16bb700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc0b22a0 0x7f7dcc0c94f0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800e810 tx=0x7f7dc800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.709+0000 7f7dc27fc700 1 -- 172.21.15.181:0/4174697118 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800c070 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.710+0000 7f7dc27fc700 1 -- 172.21.15.181:0/4174697118 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc8013610 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.710+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc0a0310 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.710+0000 7f7dc27fc700 1 -- 172.21.15.181:0/4174697118 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80139a0 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.710+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc0a08d0 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.711+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc067d20 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.711+0000 7f7dc27fc700 1 -- 172.21.15.181:0/4174697118 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7dc8013b00 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.712+0000 7f7dc27fc700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7db006b140 0x7f7db006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:17.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.712+0000 7f7dd1ebc700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7db006b140 0x7f7db006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.712+0000 7f7dc27fc700 1 -- 172.21.15.181:0/4174697118 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f7dc80896a0 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.712+0000 7f7dd1ebc700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7db006b140 0x7f7db006d600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc006b20 tx=0x7f7dbc008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:17.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.716+0000 7f7dc27fc700 1 -- 172.21.15.181:0/4174697118 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7dc8054400 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.909+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dcc06e3f0 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.910+0000 7f7dc27fc700 1 -- 172.21.15.181:0/4174697118 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v252) v1 ==== 74+0+13754 (secure 0 0 0) 0x7f7dc8057a20 con 0x7f7dcc0b22a0 2023-10-16T11:21:17.912 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:17.912 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":252,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:17.126888+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":238,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:17.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7db006b140 msgr2=0x7f7db006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7db006b140 0x7f7db006d600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc006b20 tx=0x7f7dbc008040 comp rx=0 tx=0).stop 2023-10-16T11:21:17.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc0b22a0 msgr2=0x7f7dcc0c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:17.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc0b22a0 0x7f7dcc0c94f0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800e810 tx=0x7f7dc800eb20 comp rx=0 tx=0).stop 2023-10-16T11:21:17.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 shutdown_connections 2023-10-16T11:21:17.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7dcc06e010 0x7f7dcc0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f7db006b140 0x7f7db006d600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7dcc15b8b0 0x7f7dcc06d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 --2- 172.21.15.181:0/4174697118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dcc0b22a0 0x7f7dcc0c94f0 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:17.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 >> 172.21.15.181:0/4174697118 conn(0x7f7dcc0b1780 msgr2=0x7f7dcc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:17.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 shutdown_connections 2023-10-16T11:21:17.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:17.912+0000 7f7dd4120700 1 -- 172.21.15.181:0/4174697118 wait complete. 2023-10-16T11:21:17.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:21:17.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:21:18.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 Processor -- start 2023-10-16T11:21:18.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 -- start start 2023-10-16T11:21:18.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe44415f150 0x7fe4440cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4440ce4f0 0x7fe4440b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b14a0 0x7fe4440b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440ce9a0 con 0x7fe4440ce4f0 2023-10-16T11:21:18.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440b2ae0 con 0x7fe44415f150 2023-10-16T11:21:18.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.050+0000 7fe44a1eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b2dc0 con 0x7fe4440b14a0 2023-10-16T11:21:18.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4440ce4f0 0x7fe4440b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4440ce4f0 0x7fe4440b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59704/0 (socket says 172.21.15.181:59704) 2023-10-16T11:21:18.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 -- 172.21.15.181:0/3022963593 learned_addr learned my addr 172.21.15.181:0/3022963593 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe4437fe700 1 --2- 172.21.15.181:0/3022963593 >> v2:172.21.15.190:3300/0 conn(0x7fe44415f150 0x7fe4440cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe443fff700 1 --2- 172.21.15.181:0/3022963593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b14a0 0x7fe4440b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 -- 172.21.15.181:0/3022963593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b14a0 msgr2=0x7fe4440b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:18.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 --2- 172.21.15.181:0/3022963593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b14a0 0x7fe4440b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 -- 172.21.15.181:0/3022963593 >> v2:172.21.15.190:3300/0 conn(0x7fe44415f150 msgr2=0x7fe4440cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 --2- 172.21.15.181:0/3022963593 >> v2:172.21.15.190:3300/0 conn(0x7fe44415f150 0x7fe4440cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 -- 172.21.15.181:0/3022963593 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440cf9d0 con 0x7fe4440ce4f0 2023-10-16T11:21:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe442ffd700 1 --2- 172.21.15.181:0/3022963593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440ce4f0 0x7fe4440b0f60 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098f0 tx=0x7fe438009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bd844eb97e6bd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe4427fc700 1 -- 172.21.15.181:0/3022963593 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43800b860 con 0x7fe4440ce4f0 2023-10-16T11:21:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe4427fc700 1 -- 172.21.15.181:0/3022963593 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43800b9c0 con 0x7fe4440ce4f0 2023-10-16T11:21:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.051+0000 7fe4427fc700 1 -- 172.21.15.181:0/3022963593 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43800bd50 con 0x7fe4440ce4f0 2023-10-16T11:21:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 -- 172.21.15.181:0/3022963593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440ce4f0 msgr2=0x7fe4440b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/3022963593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440ce4f0 0x7fe4440b0f60 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098f0 tx=0x7fe438009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:18.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 -- 172.21.15.181:0/3022963593 shutdown_connections 2023-10-16T11:21:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/3022963593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b14a0 0x7fe4440b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/3022963593 >> v2:172.21.15.190:3300/0 conn(0x7fe44415f150 0x7fe4440cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/3022963593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440ce4f0 0x7fe4440b0f60 unknown :-1 s=CLOSED pgs=2938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 -- 172.21.15.181:0/3022963593 >> 172.21.15.181:0/3022963593 conn(0x7fe44409a930 msgr2=0x7fe4440d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:18.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 -- 172.21.15.181:0/3022963593 shutdown_connections 2023-10-16T11:21:18.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.052+0000 7fe44a1eb700 1 -- 172.21.15.181:0/3022963593 wait complete. 2023-10-16T11:21:18.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.053+0000 7fe44a1eb700 1 Processor -- start 2023-10-16T11:21:18.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.053+0000 7fe44a1eb700 1 -- start start 2023-10-16T11:21:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe44a1eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440b14a0 0x7fe444158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe44a1eb700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440ce4f0 0x7fe4440c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe44a1eb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44415f150 0x7fe4440c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe44a1eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44409eec0 con 0x7fe4440b14a0 2023-10-16T11:21:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe44a1eb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440c5540 con 0x7fe44415f150 2023-10-16T11:21:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe44a1eb700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440c9320 con 0x7fe4440ce4f0 2023-10-16T11:21:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440b14a0 0x7fe444158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440b14a0 0x7fe444158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59710/0 (socket says 172.21.15.181:59710) 2023-10-16T11:21:18.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 -- 172.21.15.181:0/871458160 learned_addr learned my addr 172.21.15.181:0/871458160 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe443fff700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44415f150 0x7fe4440c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 -- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440ce4f0 msgr2=0x7fe4440c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe442ffd700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440ce4f0 0x7fe4440c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440ce4f0 0x7fe4440c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 -- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44415f150 msgr2=0x7fe4440c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44415f150 0x7fe4440c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 -- 172.21.15.181:0/871458160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe438009550 con 0x7fe4440b14a0 2023-10-16T11:21:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe443fff700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44415f150 0x7fe4440c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.054+0000 7fe4437fe700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440b14a0 0x7fe444158a80 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c820 tx=0x7fe43400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.055+0000 7fe42bfff700 1 -- 172.21.15.181:0/871458160 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434011800 con 0x7fe4440b14a0 2023-10-16T11:21:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.055+0000 7fe42bfff700 1 -- 172.21.15.181:0/871458160 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe434011960 con 0x7fe4440b14a0 2023-10-16T11:21:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.055+0000 7fe42bfff700 1 -- 172.21.15.181:0/871458160 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434011d10 con 0x7fe4440b14a0 2023-10-16T11:21:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.055+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4440c9600 con 0x7fe4440b14a0 2023-10-16T11:21:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.055+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4440c51d0 con 0x7fe4440b14a0 2023-10-16T11:21:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.056+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe444067d20 con 0x7fe4440b14a0 2023-10-16T11:21:18.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.056+0000 7fe42bfff700 1 -- 172.21.15.181:0/871458160 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe434018360 con 0x7fe4440b14a0 2023-10-16T11:21:18.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.056+0000 7fe42bfff700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fe43006b140 0x7fe43006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.057+0000 7fe42bfff700 1 -- 172.21.15.181:0/871458160 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 5409+0+0 (secure 0 0 0) 0x7fe434089ab0 con 0x7fe4440b14a0 2023-10-16T11:21:18.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.057+0000 7fe442ffd700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fe43006b140 0x7fe43006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.057+0000 7fe442ffd700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fe43006b140 0x7fe43006d600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4440d07c0 tx=0x7fe438008030 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.061+0000 7fe42bfff700 1 -- 172.21.15.181:0/871458160 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe434054770 con 0x7fe4440b14a0 2023-10-16T11:21:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.258+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4440ccb10 con 0x7fe4440b14a0 2023-10-16T11:21:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.259+0000 7fe42bfff700 1 -- 172.21.15.181:0/871458160 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v252) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe43401f0a0 con 0x7fe4440b14a0 2023-10-16T11:21:18.261 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:21:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fe43006b140 msgr2=0x7fe43006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fe43006b140 0x7fe43006d600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4440d07c0 tx=0x7fe438008030 comp rx=0 tx=0).stop 2023-10-16T11:21:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440b14a0 msgr2=0x7fe444158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440b14a0 0x7fe444158a80 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c820 tx=0x7fe43400cbe0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 shutdown_connections 2023-10-16T11:21:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fe43006b140 0x7fe43006d600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe44415f150 0x7fe4440c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4440b14a0 0x7fe444158a80 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 --2- 172.21.15.181:0/871458160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440ce4f0 0x7fe4440c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 >> 172.21.15.181:0/871458160 conn(0x7fe44409a930 msgr2=0x7fe44415ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:18.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 shutdown_connections 2023-10-16T11:21:18.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.261+0000 7fe44a1eb700 1 -- 172.21.15.181:0/871458160 wait complete. 2023-10-16T11:21:18.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:21:18.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.394+0000 7fa8fe8a2700 1 Processor -- start 2023-10-16T11:21:18.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.394+0000 7fa8fe8a2700 1 -- start start 2023-10-16T11:21:18.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.394+0000 7fa8fe8a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 0x7fa8f80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8fe8a2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f806ed00 con 0x7fa8f80cb0c0 2023-10-16T11:21:18.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8fe8a2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f80d02d0 con 0x7fa8f815e6d0 2023-10-16T11:21:18.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8fe8a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f806e0f0 con 0x7fa8f815eab0 2023-10-16T11:21:18.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8fce3f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa8f80cb0c0 0x7fa8f815a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42568/0 (socket says 172.21.15.181:42568) 2023-10-16T11:21:18.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8fce3f700 1 -- 172.21.15.181:0/1228785900 learned_addr learned my addr 172.21.15.181:0/1228785900 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:18.403 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 0x7fa8f80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134488521 0 0) 0x7fa8f806ed00 con 0x7fa8f80cb0c0 2023-10-16T11:21:18.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8e00035b0 con 0x7fa8f80cb0c0 2023-10-16T11:21:18.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406210018 0 0) 0x7fa8f80d02d0 con 0x7fa8f815e6d0 2023-10-16T11:21:18.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f806ed00 con 0x7fa8f815e6d0 2023-10-16T11:21:18.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 480266595 0 0) 0x7fa8e00035b0 con 0x7fa8f80cb0c0 2023-10-16T11:21:18.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 msgr2=0x7fa8f80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:18.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 0x7fa8f80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 >> v1:172.21.15.190:6789/0 conn(0x7fa8f815e6d0 legacy=0x7fa8f815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f806e3d0 con 0x7fa8f80cb0c0 2023-10-16T11:21:18.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa8f00019f0 con 0x7fa8f80cb0c0 2023-10-16T11:21:18.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 >> v1:172.21.15.181:6789/0 conn(0x7fa8f80cb0c0 legacy=0x7fa8f815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8e0003bc0 0x7fa8e0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.406 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0004850 0x7fa8e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 0x7fa8e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f806e0f0 con 0x7fa8e0004850 2023-10-16T11:21:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e000bac0 con 0x7fa8e0003bc0 2023-10-16T11:21:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e000bc80 con 0x7fa8f815eab0 2023-10-16T11:21:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0004850 0x7fa8e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8f7fff700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8e0003bc0 0x7fa8e0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.395+0000 7fa8fce3f700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 0x7fa8e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f77fe700 1 -- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 msgr2=0x7fa8e0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8f815eab0 0x7fa8e0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f77fe700 1 -- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8e0003bc0 msgr2=0x7fa8e0003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8e0003bc0 0x7fa8e0003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f77fe700 1 -- 172.21.15.181:0/1228785900 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e00035b0 con 0x7fa8e0004850 2023-10-16T11:21:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0004850 0x7fa8e0004cd0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009b60 tx=0x7fa8e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371bd8ec2e0e75b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00e6c0 con 0x7fa8e0004850 2023-10-16T11:21:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec00e820 con 0x7fa8e0004850 2023-10-16T11:21:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8f6ffd700 1 -- 172.21.15.181:0/1228785900 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00eb90 con 0x7fa8e0004850 2023-10-16T11:21:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/1228785900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0004850 msgr2=0x7fa8e0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8fe8a2700 1 --2- 172.21.15.181:0/1228785900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0004850 0x7fa8e0004cd0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009b60 tx=0x7fa8e0005880 comp rx=0 tx=0).stop 2023-10-16T11:21:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8fce3f700 1 -- 172.21.15.181:0/1228785900 reap_dead start 2023-10-16T11:21:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/1228785900 shutdown_connections 2023-10-16T11:21:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/1228785900 >> 172.21.15.181:0/1228785900 conn(0x7fa8f80d04a0 msgr2=0x7fa8f80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.396+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/1228785900 shutdown_connections 2023-10-16T11:21:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.397+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/1228785900 wait complete. 2023-10-16T11:21:18.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.398+0000 7fa8fe8a2700 1 Processor -- start 2023-10-16T11:21:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.398+0000 7fa8fe8a2700 1 -- start start 2023-10-16T11:21:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8e0004850 0x7fa8f814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f80b3fe0 0x7fa8f815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f80c6420 0x7fa8f80c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80c6de0 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f81f47e0 con 0x7fa8f80c6420 2023-10-16T11:21:18.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f81f4ac0 con 0x7fa8e0004850 2023-10-16T11:21:18.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f80b3fe0 0x7fa8f815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f80b3fe0 0x7fa8f815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59732/0 (socket says 172.21.15.181:59732) 2023-10-16T11:21:18.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 -- 172.21.15.181:0/2791169046 learned_addr learned my addr 172.21.15.181:0/2791169046 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:18.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fce3f700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f80c6420 0x7fa8f80c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f7fff700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8e0004850 0x7fa8f814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 -- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8e0004850 msgr2=0x7fa8f814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8e0004850 0x7fa8f814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 -- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f80c6420 msgr2=0x7fa8f80c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f80c6420 0x7fa8f80c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 -- 172.21.15.181:0/2791169046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f81f4da0 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fce3f700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f80c6420 0x7fa8f80c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:18.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f77fe700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f80b3fe0 0x7fa8f815af60 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0005830 tx=0x7fa8ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f4ff9700 1 -- 172.21.15.181:0/2791169046 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec021070 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f4ff9700 1 -- 172.21.15.181:0/2791169046 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec0045d0 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f81f5080 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8f4ff9700 1 -- 172.21.15.181:0/2791169046 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00e6c0 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.399+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f8150220 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.400+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f8067d20 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.401+0000 7fa8f4ff9700 1 -- 172.21.15.181:0/2791169046 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa8ec025070 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.402+0000 7fa8f4ff9700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa8e406b140 0x7fa8e406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.402+0000 7fa8f4ff9700 1 -- 172.21.15.181:0/2791169046 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 5409+0+0 (secure 0 0 0) 0x7fa8ec08aea0 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.402+0000 7fa8f7fff700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa8e406b140 0x7fa8e406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.402+0000 7fa8f7fff700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa8e406b140 0x7fa8e406d600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8000c00 tx=0x7fa8e8009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.405+0000 7fa8f4ff9700 1 -- 172.21.15.181:0/2791169046 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa8ec055ea0 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.598+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa8f80cb610 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.601 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.599+0000 7fa8f4ff9700 1 -- 172.21.15.181:0/2791169046 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v252) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa8ec017090 con 0x7fa8f80b3fe0 2023-10-16T11:21:18.601 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:18.602 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:21:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:21:18.602 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:21:18.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:18.603 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:21:18.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa8e406b140 msgr2=0x7fa8e406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa8e406b140 0x7fa8e406d600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8000c00 tx=0x7fa8e8009040 comp rx=0 tx=0).stop 2023-10-16T11:21:18.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f80b3fe0 msgr2=0x7fa8f815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f80b3fe0 0x7fa8f815af60 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0005830 tx=0x7fa8ec0047c0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 shutdown_connections 2023-10-16T11:21:18.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8e0004850 0x7fa8f814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fa8e406b140 0x7fa8e406d600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8f80c6420 0x7fa8f80c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 --2- 172.21.15.181:0/2791169046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8f80b3fe0 0x7fa8f815af60 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 >> 172.21.15.181:0/2791169046 conn(0x7fa8f80d04a0 msgr2=0x7fa8f80c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:18.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 shutdown_connections 2023-10-16T11:21:18.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.601+0000 7fa8fe8a2700 1 -- 172.21.15.181:0/2791169046 wait complete. 2023-10-16T11:21:18.615 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:21:18.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:21:18.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.738+0000 7f7a0cd23700 1 Processor -- start 2023-10-16T11:21:18.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.738+0000 7f7a0cd23700 1 -- start start 2023-10-16T11:21:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a0cd23700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a080cf5c0 0x7f7a080caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a0cd23700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080cc4a0 0x7f7a0815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a0cd23700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a08160240 con 0x7f7a0806a8a0 2023-10-16T11:21:18.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a0cd23700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0815bd80 con 0x7f7a080cf5c0 2023-10-16T11:21:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a0cd23700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0806b6d0 con 0x7f7a080cc4a0 2023-10-16T11:21:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a06d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080cc4a0 0x7f7a0815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a05d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a080cf5c0 0x7f7a080caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a06d9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080cc4a0 0x7f7a0815c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:56768/0 (socket says 172.21.15.181:56768) 2023-10-16T11:21:18.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a06d9d700 1 -- 172.21.15.181:0/2691723567 learned_addr learned my addr 172.21.15.181:0/2691723567 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.739+0000 7f7a0659c700 1 --1- 172.21.15.181:0/2691723567 >> v1:172.21.15.190:6789/0 conn(0x7f7a0806a8a0 0x7f7a0806ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:54758/0 (socket says 172.21.15.181:54758) 2023-10-16T11:21:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a06d9d700 1 -- 172.21.15.181:0/2691723567 >> v2:172.21.15.190:3300/0 conn(0x7f7a080cf5c0 msgr2=0x7f7a080caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a06d9d700 1 --2- 172.21.15.181:0/2691723567 >> v2:172.21.15.190:3300/0 conn(0x7f7a080cf5c0 0x7f7a080caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a06d9d700 1 -- 172.21.15.181:0/2691723567 >> v1:172.21.15.190:6789/0 conn(0x7f7a0806a8a0 legacy=0x7f7a0806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a06d9d700 1 -- 172.21.15.181:0/2691723567 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0815ff60 con 0x7f7a080cc4a0 2023-10-16T11:21:18.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a05d9b700 1 --2- 172.21.15.181:0/2691723567 >> v2:172.21.15.190:3300/0 conn(0x7f7a080cf5c0 0x7f7a080caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:18.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a06d9d700 1 --2- 172.21.15.181:0/2691723567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080cc4a0 0x7f7a0815c1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009e50 tx=0x7f7a0000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b00dcb4ff32737 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0559a700 1 -- 172.21.15.181:0/2691723567 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000e070 con 0x7f7a080cc4a0 2023-10-16T11:21:18.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0559a700 1 -- 172.21.15.181:0/2691723567 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0000b910 con 0x7f7a080cc4a0 2023-10-16T11:21:18.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0559a700 1 -- 172.21.15.181:0/2691723567 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000bcc0 con 0x7f7a080cc4a0 2023-10-16T11:21:18.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0cd23700 1 -- 172.21.15.181:0/2691723567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080cc4a0 msgr2=0x7f7a0815c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/2691723567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080cc4a0 0x7f7a0815c1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009e50 tx=0x7f7a0000b330 comp rx=0 tx=0).stop 2023-10-16T11:21:18.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0cd23700 1 -- 172.21.15.181:0/2691723567 shutdown_connections 2023-10-16T11:21:18.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/2691723567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080cc4a0 0x7f7a0815c1a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.740+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/2691723567 >> v2:172.21.15.190:3300/0 conn(0x7f7a080cf5c0 0x7f7a080caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.741+0000 7f7a0cd23700 1 -- 172.21.15.181:0/2691723567 >> 172.21.15.181:0/2691723567 conn(0x7f7a080d04a0 msgr2=0x7f7a080d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:18.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.741+0000 7f7a0cd23700 1 -- 172.21.15.181:0/2691723567 shutdown_connections 2023-10-16T11:21:18.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.741+0000 7f7a0cd23700 1 -- 172.21.15.181:0/2691723567 wait complete. 2023-10-16T11:21:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.742+0000 7f7a0cd23700 1 Processor -- start 2023-10-16T11:21:18.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.742+0000 7f7a0cd23700 1 -- start start 2023-10-16T11:21:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0cd23700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806a8a0 0x7f7a08156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0cd23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a080cc4a0 0x7f7a08153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0cd23700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080cf5c0 0x7f7a0806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0cd23700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08154330 con 0x7f7a080cc4a0 2023-10-16T11:21:18.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0cd23700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08155020 con 0x7f7a080cf5c0 2023-10-16T11:21:18.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0cd23700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08154140 con 0x7f7a0806a8a0 2023-10-16T11:21:18.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a080cc4a0 0x7f7a08153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a080cc4a0 0x7f7a08153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59746/0 (socket says 172.21.15.181:59746) 2023-10-16T11:21:18.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 -- 172.21.15.181:0/3798725231 learned_addr learned my addr 172.21.15.181:0/3798725231 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a06d9d700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080cf5c0 0x7f7a0806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0659c700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806a8a0 0x7f7a08156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 -- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806a8a0 msgr2=0x7f7a08156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806a8a0 0x7f7a08156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 -- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080cf5c0 msgr2=0x7f7a0806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080cf5c0 0x7f7a0806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 -- 172.21.15.181:0/3798725231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0094a0 con 0x7f7a080cc4a0 2023-10-16T11:21:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a0659c700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806a8a0 0x7f7a08156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:18.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f7a05d9b700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a080cc4a0 0x7f7a08153c00 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00ec10 tx=0x7f79fc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f79f6ffd700 1 -- 172.21.15.181:0/3798725231 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00c850 con 0x7f7a080cc4a0 2023-10-16T11:21:18.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.743+0000 7f79f6ffd700 1 -- 172.21.15.181:0/3798725231 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc00c9b0 con 0x7f7a080cc4a0 2023-10-16T11:21:18.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.744+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a00009b30 con 0x7f7a080cc4a0 2023-10-16T11:21:18.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.744+0000 7f79f6ffd700 1 -- 172.21.15.181:0/3798725231 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00cd40 con 0x7f7a080cc4a0 2023-10-16T11:21:18.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.744+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a08154960 con 0x7f7a080cc4a0 2023-10-16T11:21:18.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.744+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a08155160 con 0x7f7a080cc4a0 2023-10-16T11:21:18.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.745+0000 7f79f6ffd700 1 -- 172.21.15.181:0/3798725231 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f79fc013470 con 0x7f7a080cc4a0 2023-10-16T11:21:18.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.745+0000 7f79f6ffd700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e806b170 0x7f79e806d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:18.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.745+0000 7f79f6ffd700 1 -- 172.21.15.181:0/3798725231 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f79fc0899a0 con 0x7f7a080cc4a0 2023-10-16T11:21:18.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.745+0000 7f7a0659c700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e806b170 0x7f79e806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:18.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.746+0000 7f7a0659c700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e806b170 0x7f79e806d630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f8007fd0 tx=0x7f79f8007d80 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:18.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.750+0000 7f79f6ffd700 1 -- 172.21.15.181:0/3798725231 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7a08155160 con 0x7f7a080cc4a0 2023-10-16T11:21:18.943 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:18.941+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7a08155160 con 0x7f7a080cc4a0 2023-10-16T11:21:19.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.112+0000 7f79f6ffd700 1 -- 172.21.15.181:0/3798725231 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v253) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a08155160 con 0x7f7a080cc4a0 2023-10-16T11:21:19.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.114+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e806b170 msgr2=0x7f79e806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:19.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e806b170 0x7f79e806d630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f8007fd0 tx=0x7f79f8007d80 comp rx=0 tx=0).stop 2023-10-16T11:21:19.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a080cc4a0 msgr2=0x7f7a08153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:19.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a080cc4a0 0x7f7a08153c00 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00ec10 tx=0x7f79fc00ef20 comp rx=0 tx=0).stop 2023-10-16T11:21:19.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 shutdown_connections 2023-10-16T11:21:19.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806a8a0 0x7f7a08156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e806b170 0x7f79e806d630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080cf5c0 0x7f7a0806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 --2- 172.21.15.181:0/3798725231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a080cc4a0 0x7f7a08153c00 secure :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00ec10 tx=0x7f79fc00ef20 comp rx=0 tx=0).stop 2023-10-16T11:21:19.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 >> 172.21.15.181:0/3798725231 conn(0x7f7a080d04a0 msgr2=0x7f7a08159580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:19.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 shutdown_connections 2023-10-16T11:21:19.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.115+0000 7f7a0cd23700 1 -- 172.21.15.181:0/3798725231 wait complete. 2023-10-16T11:21:19.119 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T11:21:19.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:21:19.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.258+0000 7fbbbb70c700 1 Processor -- start 2023-10-16T11:21:19.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.259+0000 7fbbbb70c700 1 -- start start 2023-10-16T11:21:19.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbbb70c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbb40cb2b0 0x7fbbb406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:19.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbbb70c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:19.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbbb70c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb4070590 con 0x7fbbb40cb690 2023-10-16T11:21:19.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbbb70c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb4156330 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbbb70c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb406fcf0 con 0x7fbbb40b5fd0 2023-10-16T11:21:19.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbb40cb2b0 0x7fbbb406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:19.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbb40cb2b0 0x7fbbb406f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59752/0 (socket says 172.21.15.181:59752) 2023-10-16T11:21:19.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 -- 172.21.15.181:0/3660918772 learned_addr learned my addr 172.21.15.181:0/3660918772 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:19.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb9ca9700 1 --2- 172.21.15.181:0/3660918772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:19.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 -- 172.21.15.181:0/3660918772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 msgr2=0x7fbbb415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:19.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 --2- 172.21.15.181:0/3660918772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 -- 172.21.15.181:0/3660918772 >> v1:172.21.15.190:6789/0 conn(0x7fbbb40cb690 legacy=0x7fbbb40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:19.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 -- 172.21.15.181:0/3660918772 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb406ffd0 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbb94a8700 1 --2- 172.21.15.181:0/3660918772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb406f710 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098f0 tx=0x7fbba4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1ad17dcef5cef1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:19.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbabfff700 1 -- 172.21.15.181:0/3660918772 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4004230 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbabfff700 1 -- 172.21.15.181:0/3660918772 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba4004390 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.260+0000 7fbbabfff700 1 -- 172.21.15.181:0/3660918772 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba40036a0 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 -- 172.21.15.181:0/3660918772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 msgr2=0x7fbbb406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:19.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/3660918772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb406f710 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7fbba40098f0 tx=0x7fbba4009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:19.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 -- 172.21.15.181:0/3660918772 shutdown_connections 2023-10-16T11:21:19.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/3660918772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/3660918772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb406f710 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 -- 172.21.15.181:0/3660918772 >> 172.21.15.181:0/3660918772 conn(0x7fbbb40cd010 msgr2=0x7fbbb40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:19.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 -- 172.21.15.181:0/3660918772 shutdown_connections 2023-10-16T11:21:19.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.261+0000 7fbbbb70c700 1 -- 172.21.15.181:0/3660918772 wait complete. 2023-10-16T11:21:19.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.262+0000 7fbbbb70c700 1 Processor -- start 2023-10-16T11:21:19.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.263+0000 7fbbbb70c700 1 -- start start 2023-10-16T11:21:19.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.263+0000 7fbbbb70c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:19.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.263+0000 7fbbbb70c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb4158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:19.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.263+0000 7fbbbb70c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbb40cb690 0x7fbbb41585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:19.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.263+0000 7fbbbb70c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41f2440 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.263+0000 7fbbbb70c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb40c40a0 con 0x7fbbb40cb690 2023-10-16T11:21:19.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.263+0000 7fbbbb70c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41f26e0 con 0x7fbbb40b5fd0 2023-10-16T11:21:19.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb4158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:19.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb4158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59760/0 (socket says 172.21.15.181:59760) 2023-10-16T11:21:19.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 -- 172.21.15.181:0/2269057955 learned_addr learned my addr 172.21.15.181:0/2269057955 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:19.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb94a8700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:19.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb9ca9700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbb40cb690 0x7fbbb41585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:19.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 -- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 msgr2=0x7fbbb415e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:19.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 -- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbb40cb690 msgr2=0x7fbbb41585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:19.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbb40cb690 0x7fbbb41585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:19.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 -- 172.21.15.181:0/2269057955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba4009550 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbb8ca7700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb4158090 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b450 tx=0x7fbbb000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:19.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbba9ffb700 1 -- 172.21.15.181:0/2269057955 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0011830 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbba9ffb700 1 -- 172.21.15.181:0/2269057955 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb0011990 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb41f29c0 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbba9ffb700 1 -- 172.21.15.181:0/2269057955 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0011d40 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.264+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41f2f80 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.265+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb406f4f0 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.266+0000 7fbba9ffb700 1 -- 172.21.15.181:0/2269057955 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbbb001a400 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.266+0000 7fbba9ffb700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb9806b140 0x7fbb9806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:19.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.266+0000 7fbba9ffb700 1 -- 172.21.15.181:0/2269057955 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 5779+0+0 (secure 0 0 0) 0x7fbbb0089760 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.266+0000 7fbbb94a8700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb9806b140 0x7fbb9806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:19.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.267+0000 7fbbb94a8700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb9806b140 0x7fbb9806d600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009130 tx=0x7fbba4008030 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:19.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.271+0000 7fbba9ffb700 1 -- 172.21.15.181:0/2269057955 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbbb0054460 con 0x7fbbb40cb2b0 2023-10-16T11:21:19.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:19.462+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbbb4158b70 con 0x7fbbb40cb2b0 2023-10-16T11:21:20.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.117+0000 7fbba9ffb700 1 -- 172.21.15.181:0/2269057955 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v254) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbbb0057a80 con 0x7fbbb40cb2b0 2023-10-16T11:21:20.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.121+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb9806b140 msgr2=0x7fbb9806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:20.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.121+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb9806b140 0x7fbb9806d600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009130 tx=0x7fbba4008030 comp rx=0 tx=0).stop 2023-10-16T11:21:20.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.121+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 msgr2=0x7fbbb4158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:20.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.121+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb4158090 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b450 tx=0x7fbbb000b810 comp rx=0 tx=0).stop 2023-10-16T11:21:20.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.124+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 shutdown_connections 2023-10-16T11:21:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.124+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbb40b5fd0 0x7fbbb415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.124+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb9806b140 0x7fbb9806d600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.124+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbb40cb690 0x7fbbb41585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.124+0000 7fbbbb70c700 1 --2- 172.21.15.181:0/2269057955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb40cb2b0 0x7fbbb4158090 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.124+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 >> 172.21.15.181:0/2269057955 conn(0x7fbbb40cd010 msgr2=0x7fbbb40b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:20.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.124+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 shutdown_connections 2023-10-16T11:21:20.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.125+0000 7fbbbb70c700 1 -- 172.21.15.181:0/2269057955 wait complete. 2023-10-16T11:21:20.131 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T11:21:20.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:21:20.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.276+0000 7f915d337700 1 Processor -- start 2023-10-16T11:21:20.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.276+0000 7f915d337700 1 -- start start 2023-10-16T11:21:20.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f915d337700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f91580d01d0 0x7f91580ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:20.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f915d337700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91580d1500 0x7f91580d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:20.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f915d337700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580cf580 con 0x7f915815ea40 2023-10-16T11:21:20.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f915d337700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91580d4290 con 0x7f91580d1500 2023-10-16T11:21:20.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f915d337700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91580ce970 con 0x7f91580d01d0 2023-10-16T11:21:20.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91580d1500 0x7f91580d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:20.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91580d1500 0x7f91580d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59770/0 (socket says 172.21.15.181:59770) 2023-10-16T11:21:20.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 -- 172.21.15.181:0/1147846863 learned_addr learned my addr 172.21.15.181:0/1147846863 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f9156ffd700 1 --2- 172.21.15.181:0/1147846863 >> v2:172.21.15.190:3300/0 conn(0x7f91580d01d0 0x7f91580ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 -- 172.21.15.181:0/1147846863 >> v2:172.21.15.190:3300/0 conn(0x7f91580d01d0 msgr2=0x7f91580ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 --2- 172.21.15.181:0/1147846863 >> v2:172.21.15.190:3300/0 conn(0x7f91580d01d0 0x7f91580ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 -- 172.21.15.181:0/1147846863 >> v1:172.21.15.190:6789/0 conn(0x7f915815ea40 legacy=0x7f91580d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:20.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 -- 172.21.15.181:0/1147846863 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580cec50 con 0x7f91580d1500 2023-10-16T11:21:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f9156ffd700 1 --2- 172.21.15.181:0/1147846863 >> v2:172.21.15.190:3300/0 conn(0x7f91580d01d0 0x7f91580ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.277+0000 7f91577fe700 1 --2- 172.21.15.181:0/1147846863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580d1990 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f915808e050 tx=0x7f915000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c872afc0a3fb74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f9155ffb700 1 -- 172.21.15.181:0/1147846863 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915000e070 con 0x7f91580d1500 2023-10-16T11:21:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f9155ffb700 1 -- 172.21.15.181:0/1147846863 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9150003a20 con 0x7f91580d1500 2023-10-16T11:21:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f9155ffb700 1 -- 172.21.15.181:0/1147846863 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150003dd0 con 0x7f91580d1500 2023-10-16T11:21:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 -- 172.21.15.181:0/1147846863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 msgr2=0x7f91580d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 --2- 172.21.15.181:0/1147846863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580d1990 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f915808e050 tx=0x7f915000b330 comp rx=0 tx=0).stop 2023-10-16T11:21:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 -- 172.21.15.181:0/1147846863 shutdown_connections 2023-10-16T11:21:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 --2- 172.21.15.181:0/1147846863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580d1990 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 --2- 172.21.15.181:0/1147846863 >> v2:172.21.15.190:3300/0 conn(0x7f91580d01d0 0x7f91580ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 -- 172.21.15.181:0/1147846863 >> 172.21.15.181:0/1147846863 conn(0x7f915809cb60 msgr2=0x7f91580d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 -- 172.21.15.181:0/1147846863 shutdown_connections 2023-10-16T11:21:20.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.278+0000 7f915d337700 1 -- 172.21.15.181:0/1147846863 wait complete. 2023-10-16T11:21:20.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 Processor -- start 2023-10-16T11:21:20.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 -- start start 2023-10-16T11:21:20.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91580d01d0 0x7f91581f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:20.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:20.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f915815ea40 0x7f91581f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580b1820 con 0x7f91580d1500 2023-10-16T11:21:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580b0990 con 0x7f915815ea40 2023-10-16T11:21:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f915d337700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91581f3100 con 0x7f91580d01d0 2023-10-16T11:21:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f91567fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f91567fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59774/0 (socket says 172.21.15.181:59774) 2023-10-16T11:21:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.280+0000 7f91567fc700 1 -- 172.21.15.181:0/2558984346 learned_addr learned my addr 172.21.15.181:0/2558984346 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f9156ffd700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91580d01d0 0x7f91581f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f91577fe700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f915815ea40 0x7f91581f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f91567fc700 1 -- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91580d01d0 msgr2=0x7f91581f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f91567fc700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91580d01d0 0x7f91581f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f91567fc700 1 -- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f915815ea40 msgr2=0x7f91581f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f91567fc700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f915815ea40 0x7f91581f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f91567fc700 1 -- 172.21.15.181:0/2558984346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150009b30 con 0x7f91580d1500 2023-10-16T11:21:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f91567fc700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580b0450 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f914800c500 tx=0x7f914800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f913f7fe700 1 -- 172.21.15.181:0/2558984346 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148011790 con 0x7f91580d1500 2023-10-16T11:21:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f913f7fe700 1 -- 172.21.15.181:0/2558984346 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91480118f0 con 0x7f91580d1500 2023-10-16T11:21:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f913f7fe700 1 -- 172.21.15.181:0/2558984346 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148011ca0 con 0x7f91580d1500 2023-10-16T11:21:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91580b0c10 con 0x7f91580d1500 2023-10-16T11:21:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.281+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91580b11d0 con 0x7f91580d1500 2023-10-16T11:21:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.282+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91580b19d0 con 0x7f91580d1500 2023-10-16T11:21:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.283+0000 7f913f7fe700 1 -- 172.21.15.181:0/2558984346 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f914801c460 con 0x7f91580d1500 2023-10-16T11:21:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.283+0000 7f913f7fe700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f914406b140 0x7f914406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.283+0000 7f9156ffd700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f914406b140 0x7f914406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.283+0000 7f913f7fe700 1 -- 172.21.15.181:0/2558984346 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f9148089820 con 0x7f91580d1500 2023-10-16T11:21:20.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.284+0000 7f9156ffd700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f914406b140 0x7f914406d600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f914c006fd0 tx=0x7f914c008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:20.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.288+0000 7f913f7fe700 1 -- 172.21.15.181:0/2558984346 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f91580b19d0 con 0x7f91580d1500 2023-10-16T11:21:20.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:20.524+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91580b19d0 con 0x7f91580d1500 2023-10-16T11:21:21.140 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.138+0000 7f913f7fe700 1 -- 172.21.15.181:0/2558984346 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 34 and data pool 35 v140) v1 ==== 174+0+0 (secure 0 0 0) 0x7f91580b19d0 con 0x7f91580d1500 2023-10-16T11:21:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f914406b140 msgr2=0x7f914406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f914406b140 0x7f914406d600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f914c006fd0 tx=0x7f914c008040 comp rx=0 tx=0).stop 2023-10-16T11:21:21.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 msgr2=0x7f91580b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580b0450 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f914800c500 tx=0x7f914800c810 comp rx=0 tx=0).stop 2023-10-16T11:21:21.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 shutdown_connections 2023-10-16T11:21:21.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91580d01d0 0x7f91581f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f914406b140 0x7f914406d600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f915815ea40 0x7f91581f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 --2- 172.21.15.181:0/2558984346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91580d1500 0x7f91580b0450 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.146+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 >> 172.21.15.181:0/2558984346 conn(0x7f915809cb60 msgr2=0x7f915815ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.147+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 shutdown_connections 2023-10-16T11:21:21.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.147+0000 7f915d337700 1 -- 172.21.15.181:0/2558984346 wait complete. 2023-10-16T11:21:21.153 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 34 and data pool 35 2023-10-16T11:21:21.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:21.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.291+0000 7f11076fd700 1 Processor -- start 2023-10-16T11:21:21.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.291+0000 7f11076fd700 1 -- start start 2023-10-16T11:21:21.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.291+0000 7f11076fd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f110015dfd0 0x7f11000af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.291+0000 7f11076fd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 0x7f110006d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f11076fd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110006f870 con 0x7f110006db80 2023-10-16T11:21:21.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f11076fd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11000d1160 con 0x7f110015dfd0 2023-10-16T11:21:21.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f11076fd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000cc8e0 con 0x7f11000af450 2023-10-16T11:21:21.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f1105c9a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f110006db80 0x7f11000cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42626/0 (socket says 172.21.15.181:42626) 2023-10-16T11:21:21.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f1105c9a700 1 -- 172.21.15.181:0/3280197170 learned_addr learned my addr 172.21.15.181:0/3280197170 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:21.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f1105499700 1 --2- 172.21.15.181:0/3280197170 >> v2:172.21.15.190:3300/0 conn(0x7f110015dfd0 0x7f11000af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300505378 0 0) 0x7f110006f870 con 0x7f110006db80 2023-10-16T11:21:21.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f1104c98700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 0x7f110006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10ec0035b0 con 0x7f110006db80 2023-10-16T11:21:21.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1506251850 0 0) 0x7f10ec0035b0 con 0x7f110006db80 2023-10-16T11:21:21.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 msgr2=0x7f110006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:21.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 0x7f110006d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 >> v2:172.21.15.190:3300/0 conn(0x7f110015dfd0 msgr2=0x7f11000af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:21.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 --2- 172.21.15.181:0/3280197170 >> v2:172.21.15.190:3300/0 conn(0x7f110015dfd0 0x7f11000af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000ccbc0 con 0x7f110006db80 2023-10-16T11:21:21.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f10f80019f0 con 0x7f110006db80 2023-10-16T11:21:21.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 >> v1:172.21.15.181:6789/0 conn(0x7f110006db80 legacy=0x7f11000cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 0x7f10ec003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec003f80 0x7f10ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec004d90 0x7f10ec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000d1160 con 0x7f10ec003f80 2023-10-16T11:21:21.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000cc8e0 con 0x7f10ec004d90 2023-10-16T11:21:21.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec00ba60 con 0x7f11000af450 2023-10-16T11:21:21.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f1104c98700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec003f80 0x7f10ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f1105c9a700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec004d90 0x7f10ec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.292+0000 7f1105499700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 0x7f10ec003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f1104c98700 1 -- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 msgr2=0x7f10ec003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:21.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f1104c98700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000af450 0x7f10ec003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f1104c98700 1 -- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec004d90 msgr2=0x7f10ec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:21.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f1104c98700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec004d90 0x7f10ec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f1104c98700 1 -- 172.21.15.181:0/3280197170 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ec0035b0 con 0x7f10ec003f80 2023-10-16T11:21:21.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f1104c98700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec003f80 0x7f10ec004850 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009b90 tx=0x7f10ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a9096422ae3e25 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00e6c0 con 0x7f10ec003f80 2023-10-16T11:21:21.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00e820 con 0x7f10ec003f80 2023-10-16T11:21:21.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f10f7fff700 1 -- 172.21.15.181:0/3280197170 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00eb90 con 0x7f10ec003f80 2023-10-16T11:21:21.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f11076fd700 1 -- 172.21.15.181:0/3280197170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec003f80 msgr2=0x7f10ec004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f11076fd700 1 --2- 172.21.15.181:0/3280197170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10ec003f80 0x7f10ec004850 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009b90 tx=0x7f10ec005880 comp rx=0 tx=0).stop 2023-10-16T11:21:21.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f1105c9a700 1 -- 172.21.15.181:0/3280197170 reap_dead start 2023-10-16T11:21:21.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f11076fd700 1 -- 172.21.15.181:0/3280197170 shutdown_connections 2023-10-16T11:21:21.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.293+0000 7f11076fd700 1 -- 172.21.15.181:0/3280197170 >> 172.21.15.181:0/3280197170 conn(0x7f11000d04a0 msgr2=0x7f11000d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:21.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.294+0000 7f11076fd700 1 -- 172.21.15.181:0/3280197170 shutdown_connections 2023-10-16T11:21:21.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.294+0000 7f11076fd700 1 -- 172.21.15.181:0/3280197170 wait complete. 2023-10-16T11:21:21.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 Processor -- start 2023-10-16T11:21:21.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 -- start start 2023-10-16T11:21:21.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec003f80 0x7f1100154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11000c4c70 0x7f110015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000b5630 0x7f11000c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000b4340 con 0x7f11000c4c70 2023-10-16T11:21:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000b4590 con 0x7f10ec003f80 2023-10-16T11:21:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.295+0000 7f11076fd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000b4870 con 0x7f11000b5630 2023-10-16T11:21:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11000c4c70 0x7f110015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11000c4c70 0x7f110015edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59794/0 (socket says 172.21.15.181:59794) 2023-10-16T11:21:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 -- 172.21.15.181:0/2077592177 learned_addr learned my addr 172.21.15.181:0/2077592177 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1105499700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec003f80 0x7f1100154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1105c9a700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000b5630 0x7f11000c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 -- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000b5630 msgr2=0x7f11000c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000b5630 0x7f11000c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 -- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec003f80 msgr2=0x7f1100154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec003f80 0x7f1100154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 -- 172.21.15.181:0/2077592177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000b4b50 con 0x7f11000c4c70 2023-10-16T11:21:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1105499700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec003f80 0x7f1100154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f1104c98700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11000c4c70 0x7f110015edf0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009a80 tx=0x7f10fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f10f5ffb700 1 -- 172.21.15.181:0/2077592177 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00e6c0 con 0x7f11000c4c70 2023-10-16T11:21:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f10f5ffb700 1 -- 172.21.15.181:0/2077592177 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc003ad0 con 0x7f11000c4c70 2023-10-16T11:21:21.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11000b4e30 con 0x7f11000c4c70 2023-10-16T11:21:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f10f5ffb700 1 -- 172.21.15.181:0/2077592177 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0045d0 con 0x7f11000c4c70 2023-10-16T11:21:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.296+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11001550e0 con 0x7f11000c4c70 2023-10-16T11:21:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.297+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1100067d20 con 0x7f11000c4c70 2023-10-16T11:21:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.298+0000 7f10f5ffb700 1 -- 172.21.15.181:0/2077592177 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f10fc013470 con 0x7f11000c4c70 2023-10-16T11:21:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.298+0000 7f10f5ffb700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f10e406b140 0x7f10e406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.298+0000 7f10f5ffb700 1 -- 172.21.15.181:0/2077592177 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f10fc08b530 con 0x7f11000c4c70 2023-10-16T11:21:21.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.298+0000 7f1105499700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f10e406b140 0x7f10e406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.299+0000 7f1105499700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f10e406b140 0x7f10e406d600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009cc0 tx=0x7f10f0008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.303+0000 7f10f5ffb700 1 -- 172.21.15.181:0/2077592177 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f10fc056160 con 0x7f11000c4c70 2023-10-16T11:21:21.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.497+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11000b4340 con 0x7f11000c4c70 2023-10-16T11:21:21.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.499+0000 7f10f5ffb700 1 -- 172.21.15.181:0/2077592177 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v255) v1 ==== 74+0+17427 (secure 0 0 0) 0x7f10fc0273b0 con 0x7f11000c4c70 2023-10-16T11:21:21.501 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:21.502 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":255,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:20.526515+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:18.964173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:19.485121+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:21.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f10e406b140 msgr2=0x7f10e406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f10e406b140 0x7f10e406d600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009cc0 tx=0x7f10f0008040 comp rx=0 tx=0).stop 2023-10-16T11:21:21.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11000c4c70 msgr2=0x7f110015edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11000c4c70 0x7f110015edf0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009a80 tx=0x7f10fc0047c0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 shutdown_connections 2023-10-16T11:21:21.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000b5630 0x7f11000c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f10e406b140 0x7f10e406d600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec003f80 0x7f1100154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 --2- 172.21.15.181:0/2077592177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11000c4c70 0x7f110015edf0 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 >> 172.21.15.181:0/2077592177 conn(0x7f11000d04a0 msgr2=0x7f110015ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:21.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 shutdown_connections 2023-10-16T11:21:21.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.501+0000 7f11076fd700 1 -- 172.21.15.181:0/2077592177 wait complete. 2023-10-16T11:21:21.515 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:21.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.633+0000 7fd67e170700 1 Processor -- start 2023-10-16T11:21:21.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.634+0000 7fd67e170700 1 -- start start 2023-10-16T11:21:21.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.634+0000 7fd67e170700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.634+0000 7fd67e170700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6780cb0c0 0x7fd67815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.634+0000 7fd67e170700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67806ed00 con 0x7fd67815eab0 2023-10-16T11:21:21.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.634+0000 7fd67e170700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780d02d0 con 0x7fd6780cb0c0 2023-10-16T11:21:21.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.634+0000 7fd67e170700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67815ad40 con 0x7fd67815e6d0 2023-10-16T11:21:21.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.634+0000 7fd677fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6780cb0c0 0x7fd67815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd677fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6780cb0c0 0x7fd67815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59804/0 (socket says 172.21.15.181:59804) 2023-10-16T11:21:21.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd677fff700 1 -- 172.21.15.181:0/2264616461 learned_addr learned my addr 172.21.15.181:0/2264616461 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:21.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd6777fe700 1 --2- 172.21.15.181:0/2264616461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd677fff700 1 -- 172.21.15.181:0/2264616461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 msgr2=0x7fd6780d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:21.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd677fff700 1 --2- 172.21.15.181:0/2264616461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd677fff700 1 -- 172.21.15.181:0/2264616461 >> v1:172.21.15.190:6789/0 conn(0x7fd67815eab0 legacy=0x7fd67815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd677fff700 1 -- 172.21.15.181:0/2264616461 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67806e0f0 con 0x7fd6780cb0c0 2023-10-16T11:21:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd677fff700 1 --2- 172.21.15.181:0/2264616461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6780cb0c0 0x7fd67815a800 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fd670009e50 tx=0x7fd67000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df622c65cf263bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd6767fc700 1 -- 172.21.15.181:0/2264616461 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000e070 con 0x7fd6780cb0c0 2023-10-16T11:21:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd6767fc700 1 -- 172.21.15.181:0/2264616461 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67000b9c0 con 0x7fd6780cb0c0 2023-10-16T11:21:21.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.635+0000 7fd6767fc700 1 -- 172.21.15.181:0/2264616461 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000bd70 con 0x7fd6780cb0c0 2023-10-16T11:21:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 -- 172.21.15.181:0/2264616461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6780cb0c0 msgr2=0x7fd67815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 --2- 172.21.15.181:0/2264616461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6780cb0c0 0x7fd67815a800 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fd670009e50 tx=0x7fd67000b330 comp rx=0 tx=0).stop 2023-10-16T11:21:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 -- 172.21.15.181:0/2264616461 shutdown_connections 2023-10-16T11:21:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 --2- 172.21.15.181:0/2264616461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6780cb0c0 0x7fd67815a800 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 --2- 172.21.15.181:0/2264616461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 -- 172.21.15.181:0/2264616461 >> 172.21.15.181:0/2264616461 conn(0x7fd6780d04a0 msgr2=0x7fd6780d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 -- 172.21.15.181:0/2264616461 shutdown_connections 2023-10-16T11:21:21.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.636+0000 7fd67e170700 1 -- 172.21.15.181:0/2264616461 wait complete. 2023-10-16T11:21:21.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.637+0000 7fd67e170700 1 Processor -- start 2023-10-16T11:21:21.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.637+0000 7fd67e170700 1 -- start start 2023-10-16T11:21:21.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd67e170700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cb0c0 0x7fd6781f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd67e170700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd67e170700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd67815eab0 0x7fd67815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd67e170700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781f4710 con 0x7fd67815eab0 2023-10-16T11:21:21.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd67e170700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67815b440 con 0x7fd6780cb0c0 2023-10-16T11:21:21.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd67e170700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67815b720 con 0x7fd67815e6d0 2023-10-16T11:21:21.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd67815eab0 0x7fd67815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd67815eab0 0x7fd67815c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59810/0 (socket says 172.21.15.181:59810) 2023-10-16T11:21:21.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 -- 172.21.15.181:0/227692708 learned_addr learned my addr 172.21.15.181:0/227692708 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:21.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd6777fe700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cb0c0 0x7fd6781f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd676ffd700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 -- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 msgr2=0x7fd6780c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 -- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cb0c0 msgr2=0x7fd6781f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cb0c0 0x7fd6781f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 -- 172.21.15.181:0/227692708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670009b30 con 0x7fd67815eab0 2023-10-16T11:21:21.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd676ffd700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:21.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.638+0000 7fd677fff700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd67815eab0 0x7fd67815c4e0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7fd678071580 tx=0x7fd670004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.639+0000 7fd66ffff700 1 -- 172.21.15.181:0/227692708 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6700048b0 con 0x7fd67815eab0 2023-10-16T11:21:21.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.639+0000 7fd66ffff700 1 -- 172.21.15.181:0/227692708 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6700099b0 con 0x7fd67815eab0 2023-10-16T11:21:21.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.639+0000 7fd66ffff700 1 -- 172.21.15.181:0/227692708 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67001b670 con 0x7fd67815eab0 2023-10-16T11:21:21.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.639+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67815ba00 con 0x7fd67815eab0 2023-10-16T11:21:21.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.639+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67815bfc0 con 0x7fd67815eab0 2023-10-16T11:21:21.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.639+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd678067d20 con 0x7fd67815eab0 2023-10-16T11:21:21.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.640+0000 7fd66ffff700 1 -- 172.21.15.181:0/227692708 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd67001b7d0 con 0x7fd67815eab0 2023-10-16T11:21:21.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.640+0000 7fd66ffff700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fd65806b140 0x7fd65806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.640+0000 7fd66ffff700 1 -- 172.21.15.181:0/227692708 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd67008b5f0 con 0x7fd67815eab0 2023-10-16T11:21:21.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.640+0000 7fd6777fe700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fd65806b140 0x7fd65806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.641+0000 7fd6777fe700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fd65806b140 0x7fd65806d600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd660006fd0 tx=0x7fd660008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.645+0000 7fd66ffff700 1 -- 172.21.15.181:0/227692708 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd670056090 con 0x7fd67815eab0 2023-10-16T11:21:21.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.835+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6780ca7f0 con 0x7fd67815eab0 2023-10-16T11:21:21.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.837+0000 7fd66ffff700 1 -- 172.21.15.181:0/227692708 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v255) v1 ==== 74+0+17427 (secure 0 0 0) 0x7fd6700596b0 con 0x7fd67815eab0 2023-10-16T11:21:21.839 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:21.839 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":255,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:20.526515+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:18.964173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:19.485121+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:21.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fd65806b140 msgr2=0x7fd65806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fd65806b140 0x7fd65806d600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd660006fd0 tx=0x7fd660008040 comp rx=0 tx=0).stop 2023-10-16T11:21:21.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd67815eab0 msgr2=0x7fd67815c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd67815eab0 0x7fd67815c4e0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7fd678071580 tx=0x7fd670004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 shutdown_connections 2023-10-16T11:21:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fd65806b140 0x7fd65806d600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd67815eab0 0x7fd67815c4e0 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6780cb0c0 0x7fd6781f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 --2- 172.21.15.181:0/227692708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd67815e6d0 0x7fd6780c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 >> 172.21.15.181:0/227692708 conn(0x7fd6780d04a0 msgr2=0x7fd6780ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 shutdown_connections 2023-10-16T11:21:21.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.839+0000 7fd67e170700 1 -- 172.21.15.181:0/227692708 wait complete. 2023-10-16T11:21:21.853 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:21:21.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.974+0000 7fc734b9f700 1 Processor -- start 2023-10-16T11:21:21.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.974+0000 7fc734b9f700 1 -- start start 2023-10-16T11:21:21.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.975+0000 7fc734b9f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc7300b2010 0x7fc7300cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.975+0000 7fc734b9f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc730071be0 0x7fc7300b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.975+0000 7fc734b9f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7300ce1f0 con 0x7fc73006f3b0 2023-10-16T11:21:21.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.975+0000 7fc734b9f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc730072090 con 0x7fc730071be0 2023-10-16T11:21:21.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.975+0000 7fc734b9f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7300b2930 con 0x7fc7300b2010 2023-10-16T11:21:21.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc730071be0 0x7fc7300b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc730071be0 0x7fc7300b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59814/0 (socket says 172.21.15.181:59814) 2023-10-16T11:21:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 -- 172.21.15.181:0/1996268146 learned_addr learned my addr 172.21.15.181:0/1996268146 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72e59c700 1 --2- 172.21.15.181:0/1996268146 >> v2:172.21.15.190:3300/0 conn(0x7fc7300b2010 0x7fc7300cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 -- 172.21.15.181:0/1996268146 >> v2:172.21.15.190:3300/0 conn(0x7fc7300b2010 msgr2=0x7fc7300cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:21.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 --2- 172.21.15.181:0/1996268146 >> v2:172.21.15.190:3300/0 conn(0x7fc7300b2010 0x7fc7300cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 -- 172.21.15.181:0/1996268146 >> v1:172.21.15.190:6789/0 conn(0x7fc73006f3b0 legacy=0x7fc73006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 -- 172.21.15.181:0/1996268146 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300ccda0 con 0x7fc730071be0 2023-10-16T11:21:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72e59c700 1 --2- 172.21.15.181:0/1996268146 >> v2:172.21.15.190:3300/0 conn(0x7fc7300b2010 0x7fc7300cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72dd9b700 1 --2- 172.21.15.181:0/1996268146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc730071be0 0x7fc7300b23f0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7fc730070430 tx=0x7fc720009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30708b2d97c138a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72d59a700 1 -- 172.21.15.181:0/1996268146 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000b870 con 0x7fc730071be0 2023-10-16T11:21:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72d59a700 1 -- 172.21.15.181:0/1996268146 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000b9d0 con 0x7fc730071be0 2023-10-16T11:21:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.976+0000 7fc72d59a700 1 -- 172.21.15.181:0/1996268146 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000bd60 con 0x7fc730071be0 2023-10-16T11:21:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 -- 172.21.15.181:0/1996268146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc730071be0 msgr2=0x7fc7300b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 --2- 172.21.15.181:0/1996268146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc730071be0 0x7fc7300b23f0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7fc730070430 tx=0x7fc720009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 -- 172.21.15.181:0/1996268146 shutdown_connections 2023-10-16T11:21:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 --2- 172.21.15.181:0/1996268146 >> v2:172.21.15.190:3300/0 conn(0x7fc7300b2010 0x7fc7300cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 --2- 172.21.15.181:0/1996268146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc730071be0 0x7fc7300b23f0 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 -- 172.21.15.181:0/1996268146 >> 172.21.15.181:0/1996268146 conn(0x7fc7300cd010 msgr2=0x7fc7300cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 -- 172.21.15.181:0/1996268146 shutdown_connections 2023-10-16T11:21:21.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.977+0000 7fc734b9f700 1 -- 172.21.15.181:0/1996268146 wait complete. 2023-10-16T11:21:21.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.978+0000 7fc734b9f700 1 Processor -- start 2023-10-16T11:21:21.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.978+0000 7fc734b9f700 1 -- start start 2023-10-16T11:21:21.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc734b9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73006f3b0 0x7fc7300c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc734b9f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730071be0 0x7fc7300c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc734b9f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300b2010 0x7fc7300ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc734b9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c73a0 con 0x7fc73006f3b0 2023-10-16T11:21:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc734b9f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73015b790 con 0x7fc730071be0 2023-10-16T11:21:21.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc734b9f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73015ba70 con 0x7fc7300b2010 2023-10-16T11:21:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73006f3b0 0x7fc7300c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73006f3b0 0x7fc7300c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59820/0 (socket says 172.21.15.181:59820) 2023-10-16T11:21:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 -- 172.21.15.181:0/2340667869 learned_addr learned my addr 172.21.15.181:0/2340667869 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 -- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300b2010 msgr2=0x7fc7300ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72dd9b700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730071be0 0x7fc7300c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72ed9d700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300b2010 0x7fc7300ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300b2010 0x7fc7300ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 -- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730071be0 msgr2=0x7fc7300c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:21.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730071be0 0x7fc7300c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 -- 172.21.15.181:0/2340667869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720009550 con 0x7fc73006f3b0 2023-10-16T11:21:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72dd9b700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730071be0 0x7fc7300c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc72e59c700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73006f3b0 0x7fc7300c6e60 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7fc72400e810 tx=0x7fc72400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.979+0000 7fc71effd700 1 -- 172.21.15.181:0/2340667869 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724004230 con 0x7fc73006f3b0 2023-10-16T11:21:21.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.980+0000 7fc71effd700 1 -- 172.21.15.181:0/2340667869 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc724004390 con 0x7fc73006f3b0 2023-10-16T11:21:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.980+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73015bd50 con 0x7fc73006f3b0 2023-10-16T11:21:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.980+0000 7fc71effd700 1 -- 172.21.15.181:0/2340667869 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7240036a0 con 0x7fc73006f3b0 2023-10-16T11:21:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.980+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73015c310 con 0x7fc73006f3b0 2023-10-16T11:21:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.980+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73015c680 con 0x7fc73006f3b0 2023-10-16T11:21:21.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.981+0000 7fc71effd700 1 -- 172.21.15.181:0/2340667869 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc724003800 con 0x7fc73006f3b0 2023-10-16T11:21:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.982+0000 7fc71effd700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc71006b140 0x7fc71006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.982+0000 7fc71effd700 1 -- 172.21.15.181:0/2340667869 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc724089b20 con 0x7fc73006f3b0 2023-10-16T11:21:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.982+0000 7fc72dd9b700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc71006b140 0x7fc71006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.982+0000 7fc72dd9b700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc71006b140 0x7fc71006d600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc73015dd30 tx=0x7fc720008030 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:21.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:21.985+0000 7fc71effd700 1 -- 172.21.15.181:0/2340667869 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc7240546a0 con 0x7fc73006f3b0 2023-10-16T11:21:22.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:22.227+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc7300b6510 con 0x7fc73006f3b0 2023-10-16T11:21:23.115 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-10-16T11:21:23.113+0000 7fd434197700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:21:23.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.157+0000 7fc71effd700 1 -- 172.21.15.181:0/2340667869 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v143) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc724057cc0 con 0x7fc73006f3b0 2023-10-16T11:21:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc71006b140 msgr2=0x7fc71006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc71006b140 0x7fc71006d600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc73015dd30 tx=0x7fc720008030 comp rx=0 tx=0).stop 2023-10-16T11:21:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73006f3b0 msgr2=0x7fc7300c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73006f3b0 0x7fc7300c6e60 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7fc72400e810 tx=0x7fc72400ebd0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 shutdown_connections 2023-10-16T11:21:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7300b2010 0x7fc7300ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fc71006b140 0x7fc71006d600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73006f3b0 0x7fc7300c6e60 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 --2- 172.21.15.181:0/2340667869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc730071be0 0x7fc7300c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.160+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 >> 172.21.15.181:0/2340667869 conn(0x7fc7300cd010 msgr2=0x7fc7300afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:23.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.161+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 shutdown_connections 2023-10-16T11:21:23.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.161+0000 7fc734b9f700 1 -- 172.21.15.181:0/2340667869 wait complete. 2023-10-16T11:21:23.176 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:21:23.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:21:23.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.299+0000 7f1f94b7d700 1 Processor -- start 2023-10-16T11:21:23.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.299+0000 7f1f94b7d700 1 -- start start 2023-10-16T11:21:23.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.300+0000 7f1f94b7d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f9015dbd0 0x7f1f900d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:23.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.300+0000 7f1f94b7d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f900d0610 0x7f1f900cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:23.304 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.300+0000 7f1f94b7d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9015dfb0 con 0x7f1f900ce2c0 2023-10-16T11:21:23.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.300+0000 7f1f94b7d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900ce770 con 0x7f1f9015dbd0 2023-10-16T11:21:23.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.300+0000 7f1f94b7d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900cd3a0 con 0x7f1f900d0610 2023-10-16T11:21:23.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.300+0000 7f1f8e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f9015dbd0 0x7f1f900d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:23.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1f900ce2c0 0x7f1f90161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42678/0 (socket says 172.21.15.181:42678) 2023-10-16T11:21:23.305 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8ed9d700 1 -- 172.21.15.181:0/1215113218 learned_addr learned my addr 172.21.15.181:0/1215113218 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:23.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8dd9b700 1 --2- 172.21.15.181:0/1215113218 >> v2:172.21.15.190:3300/0 conn(0x7f1f900d0610 0x7f1f900cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:23.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8e59c700 1 -- 172.21.15.181:0/1215113218 >> v2:172.21.15.190:3300/0 conn(0x7f1f900d0610 msgr2=0x7f1f900cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:23.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8e59c700 1 --2- 172.21.15.181:0/1215113218 >> v2:172.21.15.190:3300/0 conn(0x7f1f900d0610 0x7f1f900cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.306 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8e59c700 1 -- 172.21.15.181:0/1215113218 >> v1:172.21.15.181:6789/0 conn(0x7f1f900ce2c0 legacy=0x7f1f90161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:23.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8d59a700 1 -- 172.21.15.181:0/1215113218 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172758781 0 0) 0x7f1f88001490 con 0x7f1f900ce2c0 2023-10-16T11:21:23.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8e59c700 1 -- 172.21.15.181:0/1215113218 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900cd830 con 0x7f1f9015dbd0 2023-10-16T11:21:23.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8e59c700 1 --2- 172.21.15.181:0/1215113218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9015dbd0 0x7f1f900d00d0 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f1f900cb990 tx=0x7f1f84009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d48b145c9d1117 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:23.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8d59a700 1 -- 172.21.15.181:0/1215113218 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84011830 con 0x7f1f9015dbd0 2023-10-16T11:21:23.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8d59a700 1 -- 172.21.15.181:0/1215113218 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f84011990 con 0x7f1f9015dbd0 2023-10-16T11:21:23.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.301+0000 7f1f8d59a700 1 -- 172.21.15.181:0/1215113218 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84011d20 con 0x7f1f9015dbd0 2023-10-16T11:21:23.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 -- 172.21.15.181:0/1215113218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9015dbd0 msgr2=0x7f1f900d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:23.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/1215113218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9015dbd0 0x7f1f900d00d0 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f1f900cb990 tx=0x7f1f84009b20 comp rx=0 tx=0).stop 2023-10-16T11:21:23.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 -- 172.21.15.181:0/1215113218 shutdown_connections 2023-10-16T11:21:23.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/1215113218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f9015dbd0 0x7f1f900d00d0 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/1215113218 >> v2:172.21.15.190:3300/0 conn(0x7f1f900d0610 0x7f1f900cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 -- 172.21.15.181:0/1215113218 >> 172.21.15.181:0/1215113218 conn(0x7f1f9009a930 msgr2=0x7f1f900d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:23.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 -- 172.21.15.181:0/1215113218 shutdown_connections 2023-10-16T11:21:23.309 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.302+0000 7f1f94b7d700 1 -- 172.21.15.181:0/1215113218 wait complete. 2023-10-16T11:21:23.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.303+0000 7f1f94b7d700 1 Processor -- start 2023-10-16T11:21:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.304+0000 7f1f94b7d700 1 -- start start 2023-10-16T11:21:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.304+0000 7f1f94b7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f900ce2c0 0x7f1f900b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.304+0000 7f1f94b7d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f900d0610 0x7f1f9015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:23.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.304+0000 7f1f94b7d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9015dbd0 0x7f1f9015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f94b7d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b55e0 con 0x7f1f900ce2c0 2023-10-16T11:21:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f94b7d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90160280 con 0x7f1f900d0610 2023-10-16T11:21:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f94b7d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b4c40 con 0x7f1f9015dbd0 2023-10-16T11:21:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f900ce2c0 0x7f1f900b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:23.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f900ce2c0 0x7f1f900b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59834/0 (socket says 172.21.15.181:59834) 2023-10-16T11:21:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 -- 172.21.15.181:0/2269249668 learned_addr learned my addr 172.21.15.181:0/2269249668 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 -- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9015dbd0 msgr2=0x7f1f9015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9015dbd0 0x7f1f9015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 -- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f900d0610 msgr2=0x7f1f9015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:23.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8dd9b700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f900d0610 0x7f1f9015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:23.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f900d0610 0x7f1f9015fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:23.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 -- 172.21.15.181:0/2269249668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840094a0 con 0x7f1f900ce2c0 2023-10-16T11:21:23.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8dd9b700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f900d0610 0x7f1f9015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:23.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f8e59c700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f900ce2c0 0x7f1f900b1d40 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f1f90072890 tx=0x7f1f84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:23.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f7effd700 1 -- 172.21.15.181:0/2269249668 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840038a0 con 0x7f1f900ce2c0 2023-10-16T11:21:23.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f7effd700 1 -- 172.21.15.181:0/2269249668 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8401f070 con 0x7f1f900ce2c0 2023-10-16T11:21:23.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900b4f20 con 0x7f1f900ce2c0 2023-10-16T11:21:23.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f7effd700 1 -- 172.21.15.181:0/2269249668 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84003c40 con 0x7f1f900ce2c0 2023-10-16T11:21:23.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.305+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9015af40 con 0x7f1f900ce2c0 2023-10-16T11:21:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.306+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900b5720 con 0x7f1f900ce2c0 2023-10-16T11:21:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.307+0000 7f1f7effd700 1 -- 172.21.15.181:0/2269249668 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1f84026480 con 0x7f1f900ce2c0 2023-10-16T11:21:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.307+0000 7f1f7effd700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f7006b140 0x7f1f7006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.307+0000 7f1f7effd700 1 -- 172.21.15.181:0/2269249668 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1f84021070 con 0x7f1f900ce2c0 2023-10-16T11:21:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.307+0000 7f1f8dd9b700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f7006b140 0x7f1f7006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.307+0000 7f1f8dd9b700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f7006b140 0x7f1f7006d600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1f900b4b70 tx=0x7f1f80008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:23.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.311+0000 7f1f7effd700 1 -- 172.21.15.181:0/2269249668 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1f900b5720 con 0x7f1f900ce2c0 2023-10-16T11:21:23.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:23.548+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1f900b5720 con 0x7f1f900ce2c0 2023-10-16T11:21:24.161 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.160+0000 7f1f7effd700 1 -- 172.21.15.181:0/2269249668 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v144) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f900b5720 con 0x7f1f900ce2c0 2023-10-16T11:21:24.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f7006b140 msgr2=0x7f1f7006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f7006b140 0x7f1f7006d600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1f900b4b70 tx=0x7f1f80008040 comp rx=0 tx=0).stop 2023-10-16T11:21:24.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f900ce2c0 msgr2=0x7f1f900b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f900ce2c0 0x7f1f900b1d40 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f1f90072890 tx=0x7f1f84004750 comp rx=0 tx=0).stop 2023-10-16T11:21:24.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 shutdown_connections 2023-10-16T11:21:24.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9015dbd0 0x7f1f9015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f7006b140 0x7f1f7006d600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f900ce2c0 0x7f1f900b1d40 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 --2- 172.21.15.181:0/2269249668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f900d0610 0x7f1f9015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 >> 172.21.15.181:0/2269249668 conn(0x7f1f9009a930 msgr2=0x7f1f90070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:24.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 shutdown_connections 2023-10-16T11:21:24.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.163+0000 7f1f94b7d700 1 -- 172.21.15.181:0/2269249668 wait complete. 2023-10-16T11:21:24.179 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:21:24.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:24.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.305+0000 7f81ea79e700 1 Processor -- start 2023-10-16T11:21:24.307 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.305+0000 7f81ea79e700 1 -- start start 2023-10-16T11:21:24.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81ea79e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 0x7f81e40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81ea79e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e40cee70 con 0x7f81e4159730 2023-10-16T11:21:24.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81ea79e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e4159b10 con 0x7f81e40cd3f0 2023-10-16T11:21:24.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81ea79e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40cd880 con 0x7f81e415cd20 2023-10-16T11:21:24.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f81e4159730 0x7f81e40cc400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42688/0 (socket says 172.21.15.181:42688) 2023-10-16T11:21:24.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e3fff700 1 -- 172.21.15.181:0/2831370475 learned_addr learned my addr 172.21.15.181:0/2831370475 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:24.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162285922 0 0) 0x7f81e40cee70 con 0x7f81e4159730 2023-10-16T11:21:24.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e37fe700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 0x7f81e40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d00035b0 con 0x7f81e4159730 2023-10-16T11:21:24.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649838224 0 0) 0x7f81d00035b0 con 0x7f81e4159730 2023-10-16T11:21:24.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 msgr2=0x7f81e40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:24.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 0x7f81e40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 >> v1:172.21.15.190:6789/0 conn(0x7f81e40cd3f0 legacy=0x7f81e40cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40ce540 con 0x7f81e4159730 2023-10-16T11:21:24.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.306+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f81cc0017f0 con 0x7f81e4159730 2023-10-16T11:21:24.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 >> v1:172.21.15.181:6789/0 conn(0x7f81e4159730 legacy=0x7f81e40cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 0x7f81d0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81d0003fb0 0x7f81d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0004d90 0x7f81d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40cd880 con 0x7f81d0003fb0 2023-10-16T11:21:24.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d000baf0 con 0x7f81d0004d90 2023-10-16T11:21:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d000bcb0 con 0x7f81e415cd20 2023-10-16T11:21:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e37fe700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81d0003fb0 0x7f81d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e3fff700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 0x7f81d0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e8d3b700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0004d90 0x7f81d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e37fe700 1 -- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 msgr2=0x7f81d0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e37fe700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e415cd20 0x7f81d0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e37fe700 1 -- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0004d90 msgr2=0x7f81d0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e37fe700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0004d90 0x7f81d0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e37fe700 1 -- 172.21.15.181:0/2831370475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00035b0 con 0x7f81d0003fb0 2023-10-16T11:21:24.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e37fe700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81d0003fb0 0x7f81d0004850 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009b60 tx=0x7f81d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84b34f8c1db8535 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:24.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d400e6c0 con 0x7f81d0003fb0 2023-10-16T11:21:24.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d400e820 con 0x7f81d0003fb0 2023-10-16T11:21:24.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.307+0000 7f81e2ffd700 1 -- 172.21.15.181:0/2831370475 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d400eb90 con 0x7f81d0003fb0 2023-10-16T11:21:24.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.308+0000 7f81ea79e700 1 -- 172.21.15.181:0/2831370475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81d0003fb0 msgr2=0x7f81d0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.308+0000 7f81ea79e700 1 --2- 172.21.15.181:0/2831370475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81d0003fb0 0x7f81d0004850 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009b60 tx=0x7f81d0005880 comp rx=0 tx=0).stop 2023-10-16T11:21:24.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.308+0000 7f81e8d3b700 1 -- 172.21.15.181:0/2831370475 reap_dead start 2023-10-16T11:21:24.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.308+0000 7f81ea79e700 1 -- 172.21.15.181:0/2831370475 shutdown_connections 2023-10-16T11:21:24.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.308+0000 7f81ea79e700 1 -- 172.21.15.181:0/2831370475 >> 172.21.15.181:0/2831370475 conn(0x7f81e40d04a0 msgr2=0x7f81e40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:24.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.308+0000 7f81ea79e700 1 -- 172.21.15.181:0/2831370475 shutdown_connections 2023-10-16T11:21:24.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.308+0000 7f81ea79e700 1 -- 172.21.15.181:0/2831370475 wait complete. 2023-10-16T11:21:24.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.309+0000 7f81ea79e700 1 Processor -- start 2023-10-16T11:21:24.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.309+0000 7f81ea79e700 1 -- start start 2023-10-16T11:21:24.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81ea79e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0003fb0 0x7f81e40b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81ea79e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e4156d30 0x7f81e40c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81ea79e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e40c9ac0 0x7f81e406d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81ea79e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40b5620 con 0x7f81e40c9ac0 2023-10-16T11:21:24.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81ea79e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40c9f70 con 0x7f81d0003fb0 2023-10-16T11:21:24.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81ea79e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40b4240 con 0x7f81e4156d30 2023-10-16T11:21:24.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e40c9ac0 0x7f81e406d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e40c9ac0 0x7f81e406d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59858/0 (socket says 172.21.15.181:59858) 2023-10-16T11:21:24.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 -- 172.21.15.181:0/1464153154 learned_addr learned my addr 172.21.15.181:0/1464153154 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:24.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e37fe700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e4156d30 0x7f81e40c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e3fff700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0003fb0 0x7f81e40b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 -- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e4156d30 msgr2=0x7f81e40c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e4156d30 0x7f81e40c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 -- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0003fb0 msgr2=0x7f81e40b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0003fb0 0x7f81e40b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e8d3b700 1 -- 172.21.15.181:0/1464153154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40b49a0 con 0x7f81e40c9ac0 2023-10-16T11:21:24.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.310+0000 7f81e3fff700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0003fb0 0x7f81e40b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:24.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.311+0000 7f81e8d3b700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e40c9ac0 0x7f81e406d640 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f81e40cc1f0 tx=0x7f81dc00d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:24.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.311+0000 7f81e0ff9700 1 -- 172.21.15.181:0/1464153154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc003c10 con 0x7f81e40c9ac0 2023-10-16T11:21:24.326 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.311+0000 7f81e0ff9700 1 -- 172.21.15.181:0/1464153154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc003d90 con 0x7f81e40c9ac0 2023-10-16T11:21:24.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.311+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e40b4c80 con 0x7f81e40c9ac0 2023-10-16T11:21:24.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.311+0000 7f81e0ff9700 1 -- 172.21.15.181:0/1464153154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00da10 con 0x7f81e40c9ac0 2023-10-16T11:21:24.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.311+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e40b5240 con 0x7f81e40c9ac0 2023-10-16T11:21:24.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.312+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e4067d20 con 0x7f81e40c9ac0 2023-10-16T11:21:24.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.312+0000 7f81e0ff9700 1 -- 172.21.15.181:0/1464153154 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f81dc00db70 con 0x7f81e40c9ac0 2023-10-16T11:21:24.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.313+0000 7f81e0ff9700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f81c406b140 0x7f81c406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.313+0000 7f81e0ff9700 1 -- 172.21.15.181:0/1464153154 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f81dc089da0 con 0x7f81e40c9ac0 2023-10-16T11:21:24.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.313+0000 7f81e3fff700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f81c406b140 0x7f81c406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.313+0000 7f81e3fff700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f81c406b140 0x7f81c406d600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81e40b63f0 tx=0x7f81cc009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:24.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.317+0000 7f81e0ff9700 1 -- 172.21.15.181:0/1464153154 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f81dc054950 con 0x7f81e40c9ac0 2023-10-16T11:21:24.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.552+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81e40b2db0 con 0x7f81e40c9ac0 2023-10-16T11:21:24.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.553+0000 7f81e0ff9700 1 -- 172.21.15.181:0/1464153154 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+4941 (secure 0 0 0) 0x7f81dc057f70 con 0x7f81e40c9ac0 2023-10-16T11:21:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:24.555 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3049755410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3049755410},{"type":"v1","addr":"172.21.15.190:6834","nonce":3049755410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2185474337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2185474337},{"type":"v1","addr":"172.21.15.181:6837","nonce":2185474337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2846112501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2846112501},{"type":"v1","addr":"172.21.15.181:6836","nonce":2846112501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:20.526852+0000","modified":"2023-10-16T11:21:23.550181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12930":{"gid":12930,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3258038365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3258038365},{"type":"v1","addr":"172.21.15.190:6835","nonce":3258038365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:21:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f81c406b140 msgr2=0x7f81c406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f81c406b140 0x7f81c406d600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f81e40b63f0 tx=0x7f81cc009040 comp rx=0 tx=0).stop 2023-10-16T11:21:24.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e40c9ac0 msgr2=0x7f81e406d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e40c9ac0 0x7f81e406d640 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f81e40cc1f0 tx=0x7f81dc00d150 comp rx=0 tx=0).stop 2023-10-16T11:21:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 shutdown_connections 2023-10-16T11:21:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f81c406b140 0x7f81c406d600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81e40c9ac0 0x7f81e406d640 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81d0003fb0 0x7f81e40b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 --2- 172.21.15.181:0/1464153154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e4156d30 0x7f81e40c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 >> 172.21.15.181:0/1464153154 conn(0x7f81e40d04a0 msgr2=0x7f81e4158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 shutdown_connections 2023-10-16T11:21:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.555+0000 7f81ea79e700 1 -- 172.21.15.181:0/1464153154 wait complete. 2023-10-16T11:21:24.560 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 144 2023-10-16T11:21:24.569 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:24.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.689+0000 7f47d6ed1700 1 Processor -- start 2023-10-16T11:21:24.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.689+0000 7f47d6ed1700 1 -- start start 2023-10-16T11:21:24.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.689+0000 7f47d6ed1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d006d5f0 0x7f47d00d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.689+0000 7f47d6ed1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47d0160d20 0x7f47d01611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d6ed1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d01616f0 con 0x7f47d0161b60 2023-10-16T11:21:24.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d6ed1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d0162800 con 0x7f47d0160d20 2023-10-16T11:21:24.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d6ed1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00d43e0 con 0x7f47d006d5f0 2023-10-16T11:21:24.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47d0160d20 0x7f47d01611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47d0160d20 0x7f47d01611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59866/0 (socket says 172.21.15.181:59866) 2023-10-16T11:21:24.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 -- 172.21.15.181:0/254016351 learned_addr learned my addr 172.21.15.181:0/254016351 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:24.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47cffff700 1 --2- 172.21.15.181:0/254016351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d006d5f0 0x7f47d00d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 -- 172.21.15.181:0/254016351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d006d5f0 msgr2=0x7f47d00d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:24.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 --2- 172.21.15.181:0/254016351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d006d5f0 0x7f47d00d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 -- 172.21.15.181:0/254016351 >> v1:172.21.15.190:6789/0 conn(0x7f47d0161b60 legacy=0x7f47d0161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:24.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 -- 172.21.15.181:0/254016351 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00d2560 con 0x7f47d0160d20 2023-10-16T11:21:24.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47d546e700 1 --2- 172.21.15.181:0/254016351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d01611b0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009e50 tx=0x7f47c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ca82e70f89747b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:24.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47cf7fe700 1 -- 172.21.15.181:0/254016351 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800e070 con 0x7f47d0160d20 2023-10-16T11:21:24.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47cf7fe700 1 -- 172.21.15.181:0/254016351 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8003a20 con 0x7f47d0160d20 2023-10-16T11:21:24.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.690+0000 7f47cf7fe700 1 -- 172.21.15.181:0/254016351 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8003dd0 con 0x7f47d0160d20 2023-10-16T11:21:24.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 -- 172.21.15.181:0/254016351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 msgr2=0x7f47d01611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/254016351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d01611b0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009e50 tx=0x7f47c800b330 comp rx=0 tx=0).stop 2023-10-16T11:21:24.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 -- 172.21.15.181:0/254016351 shutdown_connections 2023-10-16T11:21:24.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/254016351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d01611b0 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/254016351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d006d5f0 0x7f47d00d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 -- 172.21.15.181:0/254016351 >> 172.21.15.181:0/254016351 conn(0x7f47d009cba0 msgr2=0x7f47d00d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:24.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 -- 172.21.15.181:0/254016351 shutdown_connections 2023-10-16T11:21:24.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.691+0000 7f47d6ed1700 1 -- 172.21.15.181:0/254016351 wait complete. 2023-10-16T11:21:24.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.692+0000 7f47d6ed1700 1 Processor -- start 2023-10-16T11:21:24.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.692+0000 7f47d6ed1700 1 -- start start 2023-10-16T11:21:24.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d6ed1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d006d5f0 0x7f47d015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d6ed1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d00cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d6ed1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0161b60 0x7f47d00ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d6ed1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00b74e0 con 0x7f47d0160d20 2023-10-16T11:21:24.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d6ed1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00c9490 con 0x7f47d006d5f0 2023-10-16T11:21:24.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d6ed1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00c9770 con 0x7f47d0161b60 2023-10-16T11:21:24.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d00cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d00cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59870/0 (socket says 172.21.15.181:59870) 2023-10-16T11:21:24.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 -- 172.21.15.181:0/3251838783 learned_addr learned my addr 172.21.15.181:0/3251838783 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:24.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d4c6d700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d006d5f0 0x7f47d015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47d546e700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0161b60 0x7f47d00ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 -- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0161b60 msgr2=0x7f47d00ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0161b60 0x7f47d00ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 -- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d006d5f0 msgr2=0x7f47d015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d006d5f0 0x7f47d015af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.693+0000 7f47cffff700 1 -- 172.21.15.181:0/3251838783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8009b30 con 0x7f47d0160d20 2023-10-16T11:21:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.694+0000 7f47cffff700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d00cca10 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c7f0 tx=0x7f47bc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:24.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.694+0000 7f47cd7fa700 1 -- 172.21.15.181:0/3251838783 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc011800 con 0x7f47d0160d20 2023-10-16T11:21:24.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.694+0000 7f47cd7fa700 1 -- 172.21.15.181:0/3251838783 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc011960 con 0x7f47d0160d20 2023-10-16T11:21:24.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.694+0000 7f47cd7fa700 1 -- 172.21.15.181:0/3251838783 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc011cf0 con 0x7f47d0160d20 2023-10-16T11:21:24.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.694+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d00c9a50 con 0x7f47d0160d20 2023-10-16T11:21:24.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.694+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d00c9140 con 0x7f47d0160d20 2023-10-16T11:21:24.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.695+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d00c9f70 con 0x7f47d0160d20 2023-10-16T11:21:24.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.696+0000 7f47cd7fa700 1 -- 172.21.15.181:0/3251838783 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f47bc024480 con 0x7f47d0160d20 2023-10-16T11:21:24.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.696+0000 7f47cd7fa700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f47b406b140 0x7f47b406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:24.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.696+0000 7f47cd7fa700 1 -- 172.21.15.181:0/3251838783 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f47bc089ff0 con 0x7f47d0160d20 2023-10-16T11:21:24.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.696+0000 7f47d4c6d700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f47b406b140 0x7f47b406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:24.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.696+0000 7f47d4c6d700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f47b406b140 0x7f47b406d600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47c4006fd0 tx=0x7f47c4008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:24.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.700+0000 7f47cd7fa700 1 -- 172.21.15.181:0/3251838783 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f47bc054b00 con 0x7f47d0160d20 2023-10-16T11:21:24.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.894+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47d006d2a0 con 0x7f47d0160d20 2023-10-16T11:21:24.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.895+0000 7f47cd7fa700 1 -- 172.21.15.181:0/3251838783 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v256) v1 ==== 74+0+17414 (secure 0 0 0) 0x7f47bc01f090 con 0x7f47d0160d20 2023-10-16T11:21:24.898 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:24.898 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":256,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:22.139167+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:18.964173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:19.485121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:24.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f47b406b140 msgr2=0x7f47b406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f47b406b140 0x7f47b406d600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47c4006fd0 tx=0x7f47c4008040 comp rx=0 tx=0).stop 2023-10-16T11:21:24.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 msgr2=0x7f47d00cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:24.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d00cca10 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c7f0 tx=0x7f47bc00cbb0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 shutdown_connections 2023-10-16T11:21:24.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47d0161b60 0x7f47d00ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f47b406b140 0x7f47b406d600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47d006d5f0 0x7f47d015af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 --2- 172.21.15.181:0/3251838783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47d0160d20 0x7f47d00cca10 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:24.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 >> 172.21.15.181:0/3251838783 conn(0x7f47d009cba0 msgr2=0x7f47d0073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:24.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 shutdown_connections 2023-10-16T11:21:24.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:24.898+0000 7f47d6ed1700 1 -- 172.21.15.181:0/3251838783 wait complete. 2023-10-16T11:21:24.912 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:21:24.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:21:25.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.033+0000 7f5d89b59700 1 Processor -- start 2023-10-16T11:21:25.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.033+0000 7f5d89b59700 1 -- start start 2023-10-16T11:21:25.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.033+0000 7f5d89b59700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d840cdfb0 0x7f5d8415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.033+0000 7f5d89b59700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d840cabd0 con 0x7f5d8406cdf0 2023-10-16T11:21:25.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.033+0000 7f5d89b59700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d840ca5c0 con 0x7f5d840cc0c0 2023-10-16T11:21:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.033+0000 7f5d89b59700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d84066740 con 0x7f5d840cdfb0 2023-10-16T11:21:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.033+0000 7f5d837fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d840cdfb0 0x7f5d8415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d83fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5d8406cdf0 0x7f5d840cb060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42722/0 (socket says 172.21.15.181:42722) 2023-10-16T11:21:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d83fff700 1 -- 172.21.15.181:0/3289359815 learned_addr learned my addr 172.21.15.181:0/3289359815 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:25.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d837fe700 1 -- 172.21.15.181:0/3289359815 >> v1:172.21.15.190:6789/0 conn(0x7f5d840cc0c0 legacy=0x7f5d8415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d837fe700 1 -- 172.21.15.181:0/3289359815 >> v1:172.21.15.181:6789/0 conn(0x7f5d8406cdf0 legacy=0x7f5d840cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d837fe700 1 -- 172.21.15.181:0/3289359815 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84066a20 con 0x7f5d840cdfb0 2023-10-16T11:21:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d837fe700 1 --2- 172.21.15.181:0/3289359815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d840cdfb0 0x7f5d8415d8b0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009840 tx=0x7f5d74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e19a1bd512d19ce server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d827fc700 1 -- 172.21.15.181:0/3289359815 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011830 con 0x7f5d840cdfb0 2023-10-16T11:21:25.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d827fc700 1 -- 172.21.15.181:0/3289359815 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d74011990 con 0x7f5d840cdfb0 2023-10-16T11:21:25.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.034+0000 7f5d827fc700 1 -- 172.21.15.181:0/3289359815 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011d20 con 0x7f5d840cdfb0 2023-10-16T11:21:25.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.035+0000 7f5d89b59700 1 -- 172.21.15.181:0/3289359815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d840cdfb0 msgr2=0x7f5d8415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.035+0000 7f5d89b59700 1 --2- 172.21.15.181:0/3289359815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d840cdfb0 0x7f5d8415d8b0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009840 tx=0x7f5d74009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:25.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.035+0000 7f5d89b59700 1 -- 172.21.15.181:0/3289359815 shutdown_connections 2023-10-16T11:21:25.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.035+0000 7f5d89b59700 1 --2- 172.21.15.181:0/3289359815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d840cdfb0 0x7f5d8415d8b0 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.035+0000 7f5d89b59700 1 -- 172.21.15.181:0/3289359815 >> 172.21.15.181:0/3289359815 conn(0x7f5d840b1780 msgr2=0x7f5d840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:25.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.035+0000 7f5d89b59700 1 -- 172.21.15.181:0/3289359815 shutdown_connections 2023-10-16T11:21:25.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.035+0000 7f5d89b59700 1 -- 172.21.15.181:0/3289359815 wait complete. 2023-10-16T11:21:25.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.036+0000 7f5d89b59700 1 Processor -- start 2023-10-16T11:21:25.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.036+0000 7f5d89b59700 1 -- start start 2023-10-16T11:21:25.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d89b59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8406cdf0 0x7f5d841573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d89b59700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d840cc0c0 0x7f5d84158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d89b59700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d840cdfb0 0x7f5d84159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d89b59700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84159760 con 0x7f5d8406cdf0 2023-10-16T11:21:25.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d89b59700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84158850 con 0x7f5d840cc0c0 2023-10-16T11:21:25.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d89b59700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84157c40 con 0x7f5d840cdfb0 2023-10-16T11:21:25.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8406cdf0 0x7f5d841573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d837fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8406cdf0 0x7f5d841573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59880/0 (socket says 172.21.15.181:59880) 2023-10-16T11:21:25.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d837fe700 1 -- 172.21.15.181:0/681309056 learned_addr learned my addr 172.21.15.181:0/681309056 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:25.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d82ffd700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d840cc0c0 0x7f5d84158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d83fff700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d840cdfb0 0x7f5d84159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d837fe700 1 -- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d840cdfb0 msgr2=0x7f5d84159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d837fe700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d840cdfb0 0x7f5d84159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d837fe700 1 -- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d840cc0c0 msgr2=0x7f5d84158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.037+0000 7f5d837fe700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d840cc0c0 0x7f5d84158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.038+0000 7f5d837fe700 1 -- 172.21.15.181:0/681309056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740094a0 con 0x7f5d8406cdf0 2023-10-16T11:21:25.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.038+0000 7f5d837fe700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8406cdf0 0x7f5d841573f0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f5d740090b0 tx=0x7f5d74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.038+0000 7f5d6bfff700 1 -- 172.21.15.181:0/681309056 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d740038a0 con 0x7f5d8406cdf0 2023-10-16T11:21:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.038+0000 7f5d6bfff700 1 -- 172.21.15.181:0/681309056 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7401f070 con 0x7f5d8406cdf0 2023-10-16T11:21:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.038+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d84157f20 con 0x7f5d8406cdf0 2023-10-16T11:21:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.038+0000 7f5d6bfff700 1 -- 172.21.15.181:0/681309056 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74003c60 con 0x7f5d8406cdf0 2023-10-16T11:21:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.038+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d841584e0 con 0x7f5d8406cdf0 2023-10-16T11:21:25.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.039+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d84158b30 con 0x7f5d8406cdf0 2023-10-16T11:21:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.039+0000 7f5d6bfff700 1 -- 172.21.15.181:0/681309056 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5d7401a430 con 0x7f5d8406cdf0 2023-10-16T11:21:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.040+0000 7f5d6bfff700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f5d7006b140 0x7f5d7006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.040+0000 7f5d6bfff700 1 -- 172.21.15.181:0/681309056 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f5d74021070 con 0x7f5d8406cdf0 2023-10-16T11:21:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.040+0000 7f5d82ffd700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f5d7006b140 0x7f5d7006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.040+0000 7f5d82ffd700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f5d7006b140 0x7f5d7006d600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d840a05b0 tx=0x7f5d78009040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.044+0000 7f5d6bfff700 1 -- 172.21.15.181:0/681309056 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5d74056200 con 0x7f5d8406cdf0 2023-10-16T11:21:25.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.235+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 --> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d840ca860 con 0x7f5d7006b140 2023-10-16T11:21:25.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.243+0000 7f5d6bfff700 1 -- 172.21.15.181:0/681309056 <== mgr.12564 v2:172.21.15.181:6832/3850286714 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290188 (secure 0 0 0) 0x7f5d78001490 con 0x7f5d7006b140 2023-10-16T11:21:25.249 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:25.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f5d7006b140 msgr2=0x7f5d7006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f5d7006b140 0x7f5d7006d600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d840a05b0 tx=0x7f5d78009040 comp rx=0 tx=0).stop 2023-10-16T11:21:25.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8406cdf0 msgr2=0x7f5d841573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8406cdf0 0x7f5d841573f0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f5d740090b0 tx=0x7f5d74004750 comp rx=0 tx=0).stop 2023-10-16T11:21:25.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 shutdown_connections 2023-10-16T11:21:25.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d840cdfb0 0x7f5d84159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f5d7006b140 0x7f5d7006d600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d8406cdf0 0x7f5d841573f0 unknown :-1 s=CLOSED pgs=2965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 --2- 172.21.15.181:0/681309056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d840cc0c0 0x7f5d84158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 >> 172.21.15.181:0/681309056 conn(0x7f5d840b1780 msgr2=0x7f5d8406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:25.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 shutdown_connections 2023-10-16T11:21:25.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.247+0000 7f5d89b59700 1 -- 172.21.15.181:0/681309056 wait complete. 2023-10-16T11:21:25.255 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:21:25.261 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-16T11:21:24.938827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":683,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":489620,"kb_used_data":26472,"kb_used_omap":12,"kb_used_meta":463027,"kb_avail":754485100,"statfs":{"total":773094113280,"available":772592742400,"internally_reserved":0,"allocated":27107328,"data_stored":14620596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":474140216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.820267"},"pg_stats":[{"pgid":"35.3f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393693+0000","last_change":"2023-10-16T11:21:21.149392+0000","last_active":"2023-10-16T11:21:23.393693+0000","last_peered":"2023-10-16T11:21:23.393693+0000","last_clean":"2023-10-16T11:21:23.393693+0000","last_became_active":"2023-10-16T11:21:21.149103+0000","last_became_peered":"2023-10-16T11:21:21.149103+0000","last_unstale":"2023-10-16T11:21:23.393693+0000","last_undegraded":"2023-10-16T11:21:23.393693+0000","last_fullsized":"2023-10-16T11:21:23.393693+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:55:44.633526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.3e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.150120+0000","last_change":"2023-10-16T11:21:21.148324+0000","last_active":"2023-10-16T11:21:22.150120+0000","last_peered":"2023-10-16T11:21:22.150120+0000","last_clean":"2023-10-16T11:21:22.150120+0000","last_became_active":"2023-10-16T11:21:21.148049+0000","last_became_peered":"2023-10-16T11:21:21.148049+0000","last_unstale":"2023-10-16T11:21:22.150120+0000","last_undegraded":"2023-10-16T11:21:22.150120+0000","last_fullsized":"2023-10-16T11:21:22.150120+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:43.435571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149959+0000","last_change":"2023-10-16T11:21:21.144352+0000","last_active":"2023-10-16T11:21:22.149959+0000","last_peered":"2023-10-16T11:21:22.149959+0000","last_clean":"2023-10-16T11:21:22.149959+0000","last_became_active":"2023-10-16T11:21:21.143694+0000","last_became_peered":"2023-10-16T11:21:21.143694+0000","last_unstale":"2023-10-16T11:21:22.149959+0000","last_undegraded":"2023-10-16T11:21:22.149959+0000","last_fullsized":"2023-10-16T11:21:22.149959+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:32.490879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393456+0000","last_change":"2023-10-16T11:21:21.149568+0000","last_active":"2023-10-16T11:21:23.393456+0000","last_peered":"2023-10-16T11:21:23.393456+0000","last_clean":"2023-10-16T11:21:23.393456+0000","last_became_active":"2023-10-16T11:21:21.149291+0000","last_became_peered":"2023-10-16T11:21:21.149291+0000","last_unstale":"2023-10-16T11:21:23.393456+0000","last_undegraded":"2023-10-16T11:21:23.393456+0000","last_fullsized":"2023-10-16T11:21:23.393456+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:58.394396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.3b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149926+0000","last_change":"2023-10-16T11:21:21.148349+0000","last_active":"2023-10-16T11:21:22.149926+0000","last_peered":"2023-10-16T11:21:22.149926+0000","last_clean":"2023-10-16T11:21:22.149926+0000","last_became_active":"2023-10-16T11:21:21.148069+0000","last_became_peered":"2023-10-16T11:21:21.148069+0000","last_unstale":"2023-10-16T11:21:22.149926+0000","last_undegraded":"2023-10-16T11:21:22.149926+0000","last_fullsized":"2023-10-16T11:21:22.149926+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:05.441449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149440+0000","last_change":"2023-10-16T11:21:21.146179+0000","last_active":"2023-10-16T11:21:22.149440+0000","last_peered":"2023-10-16T11:21:22.149440+0000","last_clean":"2023-10-16T11:21:22.149440+0000","last_became_active":"2023-10-16T11:21:21.145933+0000","last_became_peered":"2023-10-16T11:21:21.145933+0000","last_unstale":"2023-10-16T11:21:22.149440+0000","last_undegraded":"2023-10-16T11:21:22.149440+0000","last_fullsized":"2023-10-16T11:21:22.149440+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:22.850812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.39","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083952+0000","last_change":"2023-10-16T11:21:21.157892+0000","last_active":"2023-10-16T11:21:23.083952+0000","last_peered":"2023-10-16T11:21:23.083952+0000","last_clean":"2023-10-16T11:21:23.083952+0000","last_became_active":"2023-10-16T11:21:21.157634+0000","last_became_peered":"2023-10-16T11:21:21.157634+0000","last_unstale":"2023-10-16T11:21:23.083952+0000","last_undegraded":"2023-10-16T11:21:23.083952+0000","last_fullsized":"2023-10-16T11:21:23.083952+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:08.548262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.38","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083864+0000","last_change":"2023-10-16T11:21:21.146440+0000","last_active":"2023-10-16T11:21:23.083864+0000","last_peered":"2023-10-16T11:21:23.083864+0000","last_clean":"2023-10-16T11:21:23.083864+0000","last_became_active":"2023-10-16T11:21:21.146179+0000","last_became_peered":"2023-10-16T11:21:21.146179+0000","last_unstale":"2023-10-16T11:21:23.083864+0000","last_undegraded":"2023-10-16T11:21:23.083864+0000","last_fullsized":"2023-10-16T11:21:23.083864+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:15.377571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.37","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393287+0000","last_change":"2023-10-16T11:21:21.160261+0000","last_active":"2023-10-16T11:21:23.393287+0000","last_peered":"2023-10-16T11:21:23.393287+0000","last_clean":"2023-10-16T11:21:23.393287+0000","last_became_active":"2023-10-16T11:21:21.159989+0000","last_became_peered":"2023-10-16T11:21:21.159989+0000","last_unstale":"2023-10-16T11:21:23.393287+0000","last_undegraded":"2023-10-16T11:21:23.393287+0000","last_fullsized":"2023-10-16T11:21:23.393287+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:59.089164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.36","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393670+0000","last_change":"2023-10-16T11:21:21.143017+0000","last_active":"2023-10-16T11:21:23.393670+0000","last_peered":"2023-10-16T11:21:23.393670+0000","last_clean":"2023-10-16T11:21:23.393670+0000","last_became_active":"2023-10-16T11:21:21.142709+0000","last_became_peered":"2023-10-16T11:21:21.142709+0000","last_unstale":"2023-10-16T11:21:23.393670+0000","last_undegraded":"2023-10-16T11:21:23.393670+0000","last_fullsized":"2023-10-16T11:21:23.393670+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:30.129457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.35","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148987+0000","last_change":"2023-10-16T11:21:21.148407+0000","last_active":"2023-10-16T11:21:22.148987+0000","last_peered":"2023-10-16T11:21:22.148987+0000","last_clean":"2023-10-16T11:21:22.148987+0000","last_became_active":"2023-10-16T11:21:21.148146+0000","last_became_peered":"2023-10-16T11:21:21.148146+0000","last_unstale":"2023-10-16T11:21:22.148987+0000","last_undegraded":"2023-10-16T11:21:22.148987+0000","last_fullsized":"2023-10-16T11:21:22.148987+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:40.028718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.34","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392998+0000","last_change":"2023-10-16T11:21:21.144160+0000","last_active":"2023-10-16T11:21:23.392998+0000","last_peered":"2023-10-16T11:21:23.392998+0000","last_clean":"2023-10-16T11:21:23.392998+0000","last_became_active":"2023-10-16T11:21:21.143889+0000","last_became_peered":"2023-10-16T11:21:21.143889+0000","last_unstale":"2023-10-16T11:21:23.392998+0000","last_undegraded":"2023-10-16T11:21:23.392998+0000","last_fullsized":"2023-10-16T11:21:23.392998+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:17.650622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.33","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393179+0000","last_change":"2023-10-16T11:21:21.144120+0000","last_active":"2023-10-16T11:21:23.393179+0000","last_peered":"2023-10-16T11:21:23.393179+0000","last_clean":"2023-10-16T11:21:23.393179+0000","last_became_active":"2023-10-16T11:21:21.143867+0000","last_became_peered":"2023-10-16T11:21:21.143867+0000","last_unstale":"2023-10-16T11:21:23.393179+0000","last_undegraded":"2023-10-16T11:21:23.393179+0000","last_fullsized":"2023-10-16T11:21:23.393179+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:27:05.606674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.32","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393433+0000","last_change":"2023-10-16T11:21:21.148316+0000","last_active":"2023-10-16T11:21:23.393433+0000","last_peered":"2023-10-16T11:21:23.393433+0000","last_clean":"2023-10-16T11:21:23.393433+0000","last_became_active":"2023-10-16T11:21:21.148007+0000","last_became_peered":"2023-10-16T11:21:21.148007+0000","last_unstale":"2023-10-16T11:21:23.393433+0000","last_undegraded":"2023-10-16T11:21:23.393433+0000","last_fullsized":"2023-10-16T11:21:23.393433+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:29.400971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.31","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083869+0000","last_change":"2023-10-16T11:21:21.146254+0000","last_active":"2023-10-16T11:21:23.083869+0000","last_peered":"2023-10-16T11:21:23.083869+0000","last_clean":"2023-10-16T11:21:23.083869+0000","last_became_active":"2023-10-16T11:21:21.144477+0000","last_became_peered":"2023-10-16T11:21:21.144477+0000","last_unstale":"2023-10-16T11:21:23.083869+0000","last_undegraded":"2023-10-16T11:21:23.083869+0000","last_fullsized":"2023-10-16T11:21:23.083869+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:43.351438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.30","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083867+0000","last_change":"2023-10-16T11:21:21.148636+0000","last_active":"2023-10-16T11:21:23.083867+0000","last_peered":"2023-10-16T11:21:23.083867+0000","last_clean":"2023-10-16T11:21:23.083867+0000","last_became_active":"2023-10-16T11:21:21.148023+0000","last_became_peered":"2023-10-16T11:21:21.148023+0000","last_unstale":"2023-10-16T11:21:23.083867+0000","last_undegraded":"2023-10-16T11:21:23.083867+0000","last_fullsized":"2023-10-16T11:21:23.083867+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:23.579378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393400+0000","last_change":"2023-10-16T11:21:21.162130+0000","last_active":"2023-10-16T11:21:23.393400+0000","last_peered":"2023-10-16T11:21:23.393400+0000","last_clean":"2023-10-16T11:21:23.393400+0000","last_became_active":"2023-10-16T11:21:21.161853+0000","last_became_peered":"2023-10-16T11:21:21.161853+0000","last_unstale":"2023-10-16T11:21:23.393400+0000","last_undegraded":"2023-10-16T11:21:23.393400+0000","last_fullsized":"2023-10-16T11:21:23.393400+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:34.097636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149924+0000","last_change":"2023-10-16T11:21:21.155001+0000","last_active":"2023-10-16T11:21:22.149924+0000","last_peered":"2023-10-16T11:21:22.149924+0000","last_clean":"2023-10-16T11:21:22.149924+0000","last_became_active":"2023-10-16T11:21:21.153648+0000","last_became_peered":"2023-10-16T11:21:21.153648+0000","last_unstale":"2023-10-16T11:21:22.149924+0000","last_undegraded":"2023-10-16T11:21:22.149924+0000","last_fullsized":"2023-10-16T11:21:22.149924+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:55.513062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.2d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084309+0000","last_change":"2023-10-16T11:21:21.144577+0000","last_active":"2023-10-16T11:21:23.084309+0000","last_peered":"2023-10-16T11:21:23.084309+0000","last_clean":"2023-10-16T11:21:23.084309+0000","last_became_active":"2023-10-16T11:21:21.144057+0000","last_became_peered":"2023-10-16T11:21:21.144057+0000","last_unstale":"2023-10-16T11:21:23.084309+0000","last_undegraded":"2023-10-16T11:21:23.084309+0000","last_fullsized":"2023-10-16T11:21:23.084309+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:19.324785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084129+0000","last_change":"2023-10-16T11:21:21.143333+0000","last_active":"2023-10-16T11:21:23.084129+0000","last_peered":"2023-10-16T11:21:23.084129+0000","last_clean":"2023-10-16T11:21:23.084129+0000","last_became_active":"2023-10-16T11:21:21.141836+0000","last_became_peered":"2023-10-16T11:21:21.141836+0000","last_unstale":"2023-10-16T11:21:23.084129+0000","last_undegraded":"2023-10-16T11:21:23.084129+0000","last_fullsized":"2023-10-16T11:21:23.084129+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:04.333555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393303+0000","last_change":"2023-10-16T11:21:21.145062+0000","last_active":"2023-10-16T11:21:23.393303+0000","last_peered":"2023-10-16T11:21:23.393303+0000","last_clean":"2023-10-16T11:21:23.393303+0000","last_became_active":"2023-10-16T11:21:21.144757+0000","last_became_peered":"2023-10-16T11:21:21.144757+0000","last_unstale":"2023-10-16T11:21:23.393303+0000","last_undegraded":"2023-10-16T11:21:23.393303+0000","last_fullsized":"2023-10-16T11:21:23.393303+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:23.075210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.2a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149523+0000","last_change":"2023-10-16T11:21:21.163737+0000","last_active":"2023-10-16T11:21:22.149523+0000","last_peered":"2023-10-16T11:21:22.149523+0000","last_clean":"2023-10-16T11:21:22.149523+0000","last_became_active":"2023-10-16T11:21:21.163442+0000","last_became_peered":"2023-10-16T11:21:21.163442+0000","last_unstale":"2023-10-16T11:21:22.149523+0000","last_undegraded":"2023-10-16T11:21:22.149523+0000","last_fullsized":"2023-10-16T11:21:22.149523+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:50.445572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.29","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392755+0000","last_change":"2023-10-16T11:21:21.145490+0000","last_active":"2023-10-16T11:21:23.392755+0000","last_peered":"2023-10-16T11:21:23.392755+0000","last_clean":"2023-10-16T11:21:23.392755+0000","last_became_active":"2023-10-16T11:21:21.145238+0000","last_became_peered":"2023-10-16T11:21:21.145238+0000","last_unstale":"2023-10-16T11:21:23.392755+0000","last_undegraded":"2023-10-16T11:21:23.392755+0000","last_fullsized":"2023-10-16T11:21:23.392755+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:02.678200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.28","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149276+0000","last_change":"2023-10-16T11:21:21.150825+0000","last_active":"2023-10-16T11:21:22.149276+0000","last_peered":"2023-10-16T11:21:22.149276+0000","last_clean":"2023-10-16T11:21:22.149276+0000","last_became_active":"2023-10-16T11:21:21.150487+0000","last_became_peered":"2023-10-16T11:21:21.150487+0000","last_unstale":"2023-10-16T11:21:22.149276+0000","last_undegraded":"2023-10-16T11:21:22.149276+0000","last_fullsized":"2023-10-16T11:21:22.149276+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:15.882753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.27","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393242+0000","last_change":"2023-10-16T11:21:21.150391+0000","last_active":"2023-10-16T11:21:23.393242+0000","last_peered":"2023-10-16T11:21:23.393242+0000","last_clean":"2023-10-16T11:21:23.393242+0000","last_became_active":"2023-10-16T11:21:21.150111+0000","last_became_peered":"2023-10-16T11:21:21.150111+0000","last_unstale":"2023-10-16T11:21:23.393242+0000","last_undegraded":"2023-10-16T11:21:23.393242+0000","last_fullsized":"2023-10-16T11:21:23.393242+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:57.942362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.26","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393600+0000","last_change":"2023-10-16T11:21:21.147732+0000","last_active":"2023-10-16T11:21:23.393600+0000","last_peered":"2023-10-16T11:21:23.393600+0000","last_clean":"2023-10-16T11:21:23.393600+0000","last_became_active":"2023-10-16T11:21:21.147428+0000","last_became_peered":"2023-10-16T11:21:21.147428+0000","last_unstale":"2023-10-16T11:21:23.393600+0000","last_undegraded":"2023-10-16T11:21:23.393600+0000","last_fullsized":"2023-10-16T11:21:23.393600+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:26.284941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.25","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084183+0000","last_change":"2023-10-16T11:21:21.155577+0000","last_active":"2023-10-16T11:21:23.084183+0000","last_peered":"2023-10-16T11:21:23.084183+0000","last_clean":"2023-10-16T11:21:23.084183+0000","last_became_active":"2023-10-16T11:21:21.154302+0000","last_became_peered":"2023-10-16T11:21:21.154302+0000","last_unstale":"2023-10-16T11:21:23.084183+0000","last_undegraded":"2023-10-16T11:21:23.084183+0000","last_fullsized":"2023-10-16T11:21:23.084183+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:18:12.039616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.24","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393597+0000","last_change":"2023-10-16T11:21:21.150105+0000","last_active":"2023-10-16T11:21:23.393597+0000","last_peered":"2023-10-16T11:21:23.393597+0000","last_clean":"2023-10-16T11:21:23.393597+0000","last_became_active":"2023-10-16T11:21:21.149841+0000","last_became_peered":"2023-10-16T11:21:21.149841+0000","last_unstale":"2023-10-16T11:21:23.393597+0000","last_undegraded":"2023-10-16T11:21:23.393597+0000","last_fullsized":"2023-10-16T11:21:23.393597+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:49.072960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.23","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083917+0000","last_change":"2023-10-16T11:21:21.144111+0000","last_active":"2023-10-16T11:21:23.083917+0000","last_peered":"2023-10-16T11:21:23.083917+0000","last_clean":"2023-10-16T11:21:23.083917+0000","last_became_active":"2023-10-16T11:21:21.143779+0000","last_became_peered":"2023-10-16T11:21:21.143779+0000","last_unstale":"2023-10-16T11:21:23.083917+0000","last_undegraded":"2023-10-16T11:21:23.083917+0000","last_fullsized":"2023-10-16T11:21:23.083917+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.617473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.21","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084063+0000","last_change":"2023-10-16T11:21:21.144235+0000","last_active":"2023-10-16T11:21:23.084063+0000","last_peered":"2023-10-16T11:21:23.084063+0000","last_clean":"2023-10-16T11:21:23.084063+0000","last_became_active":"2023-10-16T11:21:21.143613+0000","last_became_peered":"2023-10-16T11:21:21.143613+0000","last_unstale":"2023-10-16T11:21:23.084063+0000","last_undegraded":"2023-10-16T11:21:23.084063+0000","last_fullsized":"2023-10-16T11:21:23.084063+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:27.659969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.20","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392836+0000","last_change":"2023-10-16T11:21:21.158661+0000","last_active":"2023-10-16T11:21:23.392836+0000","last_peered":"2023-10-16T11:21:23.392836+0000","last_clean":"2023-10-16T11:21:23.392836+0000","last_became_active":"2023-10-16T11:21:21.158356+0000","last_became_peered":"2023-10-16T11:21:21.158356+0000","last_unstale":"2023-10-16T11:21:23.392836+0000","last_undegraded":"2023-10-16T11:21:23.392836+0000","last_fullsized":"2023-10-16T11:21:23.392836+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:26.284941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393672+0000","last_change":"2023-10-16T11:21:21.156993+0000","last_active":"2023-10-16T11:21:23.393672+0000","last_peered":"2023-10-16T11:21:23.393672+0000","last_clean":"2023-10-16T11:21:23.393672+0000","last_became_active":"2023-10-16T11:21:21.156687+0000","last_became_peered":"2023-10-16T11:21:21.156687+0000","last_unstale":"2023-10-16T11:21:23.393672+0000","last_undegraded":"2023-10-16T11:21:23.393672+0000","last_fullsized":"2023-10-16T11:21:23.393672+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:05:31.538501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1f","version":"255'4","reported_seq":16,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.164073+0000","last_change":"2023-10-16T11:21:20.134648+0000","last_active":"2023-10-16T11:21:21.164073+0000","last_peered":"2023-10-16T11:21:21.164073+0000","last_clean":"2023-10-16T11:21:21.164073+0000","last_became_active":"2023-10-16T11:21:20.134317+0000","last_became_peered":"2023-10-16T11:21:20.134317+0000","last_unstale":"2023-10-16T11:21:21.164073+0000","last_undegraded":"2023-10-16T11:21:21.164073+0000","last_fullsized":"2023-10-16T11:21:21.164073+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:04.691534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149066+0000","last_change":"2023-10-16T11:21:21.156017+0000","last_active":"2023-10-16T11:21:22.149066+0000","last_peered":"2023-10-16T11:21:22.149066+0000","last_clean":"2023-10-16T11:21:22.149066+0000","last_became_active":"2023-10-16T11:21:21.155761+0000","last_became_peered":"2023-10-16T11:21:21.155761+0000","last_unstale":"2023-10-16T11:21:22.149066+0000","last_undegraded":"2023-10-16T11:21:22.149066+0000","last_fullsized":"2023-10-16T11:21:22.149066+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:02.678200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.1e","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.162089+0000","last_change":"2023-10-16T11:21:20.140722+0000","last_active":"2023-10-16T11:21:21.162089+0000","last_peered":"2023-10-16T11:21:21.162089+0000","last_clean":"2023-10-16T11:21:21.162089+0000","last_became_active":"2023-10-16T11:21:20.140377+0000","last_became_peered":"2023-10-16T11:21:20.140377+0000","last_unstale":"2023-10-16T11:21:21.162089+0000","last_undegraded":"2023-10-16T11:21:21.162089+0000","last_fullsized":"2023-10-16T11:21:21.162089+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:22.070911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149523+0000","last_change":"2023-10-16T11:21:21.145767+0000","last_active":"2023-10-16T11:21:22.149523+0000","last_peered":"2023-10-16T11:21:22.149523+0000","last_clean":"2023-10-16T11:21:22.149523+0000","last_became_active":"2023-10-16T11:21:21.145299+0000","last_became_peered":"2023-10-16T11:21:21.145299+0000","last_unstale":"2023-10-16T11:21:22.149523+0000","last_undegraded":"2023-10-16T11:21:22.149523+0000","last_fullsized":"2023-10-16T11:21:22.149523+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:51.747625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1d","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.157335+0000","last_change":"2023-10-16T11:21:20.140601+0000","last_active":"2023-10-16T11:21:21.157335+0000","last_peered":"2023-10-16T11:21:21.157335+0000","last_clean":"2023-10-16T11:21:21.157335+0000","last_became_active":"2023-10-16T11:21:20.140319+0000","last_became_peered":"2023-10-16T11:21:20.140319+0000","last_unstale":"2023-10-16T11:21:21.157335+0000","last_undegraded":"2023-10-16T11:21:21.157335+0000","last_fullsized":"2023-10-16T11:21:21.157335+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:33.178321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149610+0000","last_change":"2023-10-16T11:21:21.144827+0000","last_active":"2023-10-16T11:21:22.149610+0000","last_peered":"2023-10-16T11:21:22.149610+0000","last_clean":"2023-10-16T11:21:22.149610+0000","last_became_active":"2023-10-16T11:21:21.144086+0000","last_became_peered":"2023-10-16T11:21:21.144086+0000","last_unstale":"2023-10-16T11:21:22.149610+0000","last_undegraded":"2023-10-16T11:21:22.149610+0000","last_fullsized":"2023-10-16T11:21:22.149610+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:45:21.800256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.161466+0000","last_change":"2023-10-16T11:21:20.133950+0000","last_active":"2023-10-16T11:21:21.161466+0000","last_peered":"2023-10-16T11:21:21.161466+0000","last_clean":"2023-10-16T11:21:21.161466+0000","last_became_active":"2023-10-16T11:21:20.133490+0000","last_became_peered":"2023-10-16T11:21:20.133490+0000","last_unstale":"2023-10-16T11:21:21.161466+0000","last_undegraded":"2023-10-16T11:21:21.161466+0000","last_fullsized":"2023-10-16T11:21:21.161466+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:17.126514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392753+0000","last_change":"2023-10-16T11:21:21.151705+0000","last_active":"2023-10-16T11:21:23.392753+0000","last_peered":"2023-10-16T11:21:23.392753+0000","last_clean":"2023-10-16T11:21:23.392753+0000","last_became_active":"2023-10-16T11:21:21.150058+0000","last_became_peered":"2023-10-16T11:21:21.150058+0000","last_unstale":"2023-10-16T11:21:23.392753+0000","last_undegraded":"2023-10-16T11:21:23.392753+0000","last_fullsized":"2023-10-16T11:21:23.392753+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:29.350464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1b","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.163348+0000","last_change":"2023-10-16T11:21:20.133303+0000","last_active":"2023-10-16T11:21:21.163348+0000","last_peered":"2023-10-16T11:21:21.163348+0000","last_clean":"2023-10-16T11:21:21.163348+0000","last_became_active":"2023-10-16T11:21:20.133032+0000","last_became_peered":"2023-10-16T11:21:20.133032+0000","last_unstale":"2023-10-16T11:21:21.163348+0000","last_undegraded":"2023-10-16T11:21:21.163348+0000","last_fullsized":"2023-10-16T11:21:21.163348+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:56.995189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083968+0000","last_change":"2023-10-16T11:21:21.146548+0000","last_active":"2023-10-16T11:21:23.083968+0000","last_peered":"2023-10-16T11:21:23.083968+0000","last_clean":"2023-10-16T11:21:23.083968+0000","last_became_active":"2023-10-16T11:21:21.146286+0000","last_became_peered":"2023-10-16T11:21:21.146286+0000","last_unstale":"2023-10-16T11:21:23.083968+0000","last_undegraded":"2023-10-16T11:21:23.083968+0000","last_fullsized":"2023-10-16T11:21:23.083968+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:38:18.751110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1a","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.164607+0000","last_change":"2023-10-16T11:21:20.135318+0000","last_active":"2023-10-16T11:21:21.164607+0000","last_peered":"2023-10-16T11:21:21.164607+0000","last_clean":"2023-10-16T11:21:21.164607+0000","last_became_active":"2023-10-16T11:21:20.135057+0000","last_became_peered":"2023-10-16T11:21:20.135057+0000","last_unstale":"2023-10-16T11:21:21.164607+0000","last_undegraded":"2023-10-16T11:21:21.164607+0000","last_fullsized":"2023-10-16T11:21:21.164607+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:55.031053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392957+0000","last_change":"2023-10-16T11:21:21.159119+0000","last_active":"2023-10-16T11:21:23.392957+0000","last_peered":"2023-10-16T11:21:23.392957+0000","last_clean":"2023-10-16T11:21:23.392957+0000","last_became_active":"2023-10-16T11:21:21.158849+0000","last_became_peered":"2023-10-16T11:21:21.158849+0000","last_unstale":"2023-10-16T11:21:23.392957+0000","last_undegraded":"2023-10-16T11:21:23.392957+0000","last_fullsized":"2023-10-16T11:21:23.392957+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:05:31.538501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137343+0000","last_change":"2023-10-16T11:21:20.136910+0000","last_active":"2023-10-16T11:21:21.137343+0000","last_peered":"2023-10-16T11:21:21.137343+0000","last_clean":"2023-10-16T11:21:21.137343+0000","last_became_active":"2023-10-16T11:21:20.136132+0000","last_became_peered":"2023-10-16T11:21:20.136132+0000","last_unstale":"2023-10-16T11:21:21.137343+0000","last_undegraded":"2023-10-16T11:21:21.137343+0000","last_fullsized":"2023-10-16T11:21:21.137343+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:49.341656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149422+0000","last_change":"2023-10-16T11:21:21.156347+0000","last_active":"2023-10-16T11:21:22.149422+0000","last_peered":"2023-10-16T11:21:22.149422+0000","last_clean":"2023-10-16T11:21:22.149422+0000","last_became_active":"2023-10-16T11:21:21.156062+0000","last_became_peered":"2023-10-16T11:21:21.156062+0000","last_unstale":"2023-10-16T11:21:22.149422+0000","last_undegraded":"2023-10-16T11:21:22.149422+0000","last_fullsized":"2023-10-16T11:21:22.149422+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:51.724826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137545+0000","last_change":"2023-10-16T11:21:20.136880+0000","last_active":"2023-10-16T11:21:21.137545+0000","last_peered":"2023-10-16T11:21:21.137545+0000","last_clean":"2023-10-16T11:21:21.137545+0000","last_became_active":"2023-10-16T11:21:20.136549+0000","last_became_peered":"2023-10-16T11:21:20.136549+0000","last_unstale":"2023-10-16T11:21:21.137545+0000","last_undegraded":"2023-10-16T11:21:21.137545+0000","last_fullsized":"2023-10-16T11:21:21.137545+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:33.178321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149724+0000","last_change":"2023-10-16T11:21:21.158168+0000","last_active":"2023-10-16T11:21:22.149724+0000","last_peered":"2023-10-16T11:21:22.149724+0000","last_clean":"2023-10-16T11:21:22.149724+0000","last_became_active":"2023-10-16T11:21:21.157925+0000","last_became_peered":"2023-10-16T11:21:21.157925+0000","last_unstale":"2023-10-16T11:21:22.149724+0000","last_undegraded":"2023-10-16T11:21:22.149724+0000","last_fullsized":"2023-10-16T11:21:22.149724+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:10.038543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.136497+0000","last_change":"2023-10-16T11:21:20.136649+0000","last_active":"2023-10-16T11:21:21.136497+0000","last_peered":"2023-10-16T11:21:21.136497+0000","last_clean":"2023-10-16T11:21:21.136497+0000","last_became_active":"2023-10-16T11:21:20.135908+0000","last_became_peered":"2023-10-16T11:21:20.135908+0000","last_unstale":"2023-10-16T11:21:21.136497+0000","last_undegraded":"2023-10-16T11:21:21.136497+0000","last_fullsized":"2023-10-16T11:21:21.136497+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:43.263648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393840+0000","last_change":"2023-10-16T11:21:21.152790+0000","last_active":"2023-10-16T11:21:23.393840+0000","last_peered":"2023-10-16T11:21:23.393840+0000","last_clean":"2023-10-16T11:21:23.393840+0000","last_became_active":"2023-10-16T11:21:21.152524+0000","last_became_peered":"2023-10-16T11:21:21.152524+0000","last_unstale":"2023-10-16T11:21:23.393840+0000","last_undegraded":"2023-10-16T11:21:23.393840+0000","last_fullsized":"2023-10-16T11:21:23.393840+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:44.086547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.142862+0000","last_change":"2023-10-16T11:21:20.134788+0000","last_active":"2023-10-16T11:21:21.142862+0000","last_peered":"2023-10-16T11:21:21.142862+0000","last_clean":"2023-10-16T11:21:21.142862+0000","last_became_active":"2023-10-16T11:21:20.134497+0000","last_became_peered":"2023-10-16T11:21:20.134497+0000","last_unstale":"2023-10-16T11:21:21.142862+0000","last_undegraded":"2023-10-16T11:21:21.142862+0000","last_fullsized":"2023-10-16T11:21:21.142862+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:42.293378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148749+0000","last_change":"2023-10-16T11:21:21.159556+0000","last_active":"2023-10-16T11:21:22.148749+0000","last_peered":"2023-10-16T11:21:22.148749+0000","last_clean":"2023-10-16T11:21:22.148749+0000","last_became_active":"2023-10-16T11:21:21.159290+0000","last_became_peered":"2023-10-16T11:21:21.159290+0000","last_unstale":"2023-10-16T11:21:22.148749+0000","last_undegraded":"2023-10-16T11:21:22.148749+0000","last_fullsized":"2023-10-16T11:21:22.148749+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:54:40.915612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.15","version":"255'3","reported_seq":15,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.160075+0000","last_change":"2023-10-16T11:21:20.131549+0000","last_active":"2023-10-16T11:21:21.160075+0000","last_peered":"2023-10-16T11:21:21.160075+0000","last_clean":"2023-10-16T11:21:21.160075+0000","last_became_active":"2023-10-16T11:21:20.131029+0000","last_became_peered":"2023-10-16T11:21:20.131029+0000","last_unstale":"2023-10-16T11:21:21.160075+0000","last_undegraded":"2023-10-16T11:21:21.160075+0000","last_fullsized":"2023-10-16T11:21:21.160075+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:54.508762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393724+0000","last_change":"2023-10-16T11:21:21.139139+0000","last_active":"2023-10-16T11:21:23.393724+0000","last_peered":"2023-10-16T11:21:23.393724+0000","last_clean":"2023-10-16T11:21:23.393724+0000","last_became_active":"2023-10-16T11:21:21.138819+0000","last_became_peered":"2023-10-16T11:21:21.138819+0000","last_unstale":"2023-10-16T11:21:23.393724+0000","last_undegraded":"2023-10-16T11:21:23.393724+0000","last_fullsized":"2023-10-16T11:21:23.393724+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:50.021261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.14","version":"255'6","reported_seq":18,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.165605+0000","last_change":"2023-10-16T11:21:20.127077+0000","last_active":"2023-10-16T11:21:21.165605+0000","last_peered":"2023-10-16T11:21:21.165605+0000","last_clean":"2023-10-16T11:21:21.165605+0000","last_became_active":"2023-10-16T11:21:20.126774+0000","last_became_peered":"2023-10-16T11:21:20.126774+0000","last_unstale":"2023-10-16T11:21:21.165605+0000","last_undegraded":"2023-10-16T11:21:21.165605+0000","last_fullsized":"2023-10-16T11:21:21.165605+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:18.733921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083980+0000","last_change":"2023-10-16T11:21:21.143022+0000","last_active":"2023-10-16T11:21:23.083980+0000","last_peered":"2023-10-16T11:21:23.083980+0000","last_clean":"2023-10-16T11:21:23.083980+0000","last_became_active":"2023-10-16T11:21:21.142711+0000","last_became_peered":"2023-10-16T11:21:21.142711+0000","last_unstale":"2023-10-16T11:21:23.083980+0000","last_undegraded":"2023-10-16T11:21:23.083980+0000","last_fullsized":"2023-10-16T11:21:23.083980+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:14.502383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.13","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.156589+0000","last_change":"2023-10-16T11:21:20.137401+0000","last_active":"2023-10-16T11:21:21.156589+0000","last_peered":"2023-10-16T11:21:21.156589+0000","last_clean":"2023-10-16T11:21:21.156589+0000","last_became_active":"2023-10-16T11:21:20.137108+0000","last_became_peered":"2023-10-16T11:21:20.137108+0000","last_unstale":"2023-10-16T11:21:21.156589+0000","last_undegraded":"2023-10-16T11:21:21.156589+0000","last_fullsized":"2023-10-16T11:21:21.156589+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149584+0000","last_change":"2023-10-16T11:21:21.155157+0000","last_active":"2023-10-16T11:21:22.149584+0000","last_peered":"2023-10-16T11:21:22.149584+0000","last_clean":"2023-10-16T11:21:22.149584+0000","last_became_active":"2023-10-16T11:21:21.154777+0000","last_became_peered":"2023-10-16T11:21:21.154777+0000","last_unstale":"2023-10-16T11:21:22.149584+0000","last_undegraded":"2023-10-16T11:21:22.149584+0000","last_fullsized":"2023-10-16T11:21:22.149584+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:18.294874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.141510+0000","last_change":"2023-10-16T11:21:20.135577+0000","last_active":"2023-10-16T11:21:21.141510+0000","last_peered":"2023-10-16T11:21:21.141510+0000","last_clean":"2023-10-16T11:21:21.141510+0000","last_became_active":"2023-10-16T11:21:20.135290+0000","last_became_peered":"2023-10-16T11:21:20.135290+0000","last_unstale":"2023-10-16T11:21:21.141510+0000","last_undegraded":"2023-10-16T11:21:21.141510+0000","last_fullsized":"2023-10-16T11:21:21.141510+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:39.024418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148159+0000","last_change":"2023-10-16T11:21:21.156830+0000","last_active":"2023-10-16T11:21:22.148159+0000","last_peered":"2023-10-16T11:21:22.148159+0000","last_clean":"2023-10-16T11:21:22.148159+0000","last_became_active":"2023-10-16T11:21:21.156545+0000","last_became_peered":"2023-10-16T11:21:21.156545+0000","last_unstale":"2023-10-16T11:21:22.148159+0000","last_undegraded":"2023-10-16T11:21:22.148159+0000","last_fullsized":"2023-10-16T11:21:22.148159+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:47.023237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.133566+0000","last_change":"2023-10-16T11:21:20.137041+0000","last_active":"2023-10-16T11:21:21.133566+0000","last_peered":"2023-10-16T11:21:21.133566+0000","last_clean":"2023-10-16T11:21:21.133566+0000","last_became_active":"2023-10-16T11:21:20.136734+0000","last_became_peered":"2023-10-16T11:21:20.136734+0000","last_unstale":"2023-10-16T11:21:21.133566+0000","last_undegraded":"2023-10-16T11:21:21.133566+0000","last_fullsized":"2023-10-16T11:21:21.133566+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:33.259877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084252+0000","last_change":"2023-10-16T11:21:21.139823+0000","last_active":"2023-10-16T11:21:23.084252+0000","last_peered":"2023-10-16T11:21:23.084252+0000","last_clean":"2023-10-16T11:21:23.084252+0000","last_became_active":"2023-10-16T11:21:21.139509+0000","last_became_peered":"2023-10-16T11:21:21.139509+0000","last_unstale":"2023-10-16T11:21:23.084252+0000","last_undegraded":"2023-10-16T11:21:23.084252+0000","last_fullsized":"2023-10-16T11:21:23.084252+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:59:46.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.10","version":"255'8","reported_seq":20,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.161026+0000","last_change":"2023-10-16T11:21:20.131005+0000","last_active":"2023-10-16T11:21:21.161026+0000","last_peered":"2023-10-16T11:21:21.161026+0000","last_clean":"2023-10-16T11:21:21.161026+0000","last_became_active":"2023-10-16T11:21:20.130698+0000","last_became_peered":"2023-10-16T11:21:20.130698+0000","last_unstale":"2023-10-16T11:21:21.161026+0000","last_undegraded":"2023-10-16T11:21:21.161026+0000","last_fullsized":"2023-10-16T11:21:21.161026+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:44.667056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393039+0000","last_change":"2023-10-16T11:21:21.144905+0000","last_active":"2023-10-16T11:21:23.393039+0000","last_peered":"2023-10-16T11:21:23.393039+0000","last_clean":"2023-10-16T11:21:23.393039+0000","last_became_active":"2023-10-16T11:21:21.144575+0000","last_became_peered":"2023-10-16T11:21:21.144575+0000","last_unstale":"2023-10-16T11:21:23.393039+0000","last_undegraded":"2023-10-16T11:21:23.393039+0000","last_fullsized":"2023-10-16T11:21:23.393039+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:03.266767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.158868+0000","last_change":"2023-10-16T11:21:20.137849+0000","last_active":"2023-10-16T11:21:21.158868+0000","last_peered":"2023-10-16T11:21:21.158868+0000","last_clean":"2023-10-16T11:21:21.158868+0000","last_became_active":"2023-10-16T11:21:20.137585+0000","last_became_peered":"2023-10-16T11:21:20.137585+0000","last_unstale":"2023-10-16T11:21:21.158868+0000","last_undegraded":"2023-10-16T11:21:21.158868+0000","last_fullsized":"2023-10-16T11:21:21.158868+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:03.818822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.150178+0000","last_change":"2023-10-16T11:21:21.156832+0000","last_active":"2023-10-16T11:21:22.150178+0000","last_peered":"2023-10-16T11:21:22.150178+0000","last_clean":"2023-10-16T11:21:22.150178+0000","last_became_active":"2023-10-16T11:21:21.156578+0000","last_became_peered":"2023-10-16T11:21:21.156578+0000","last_unstale":"2023-10-16T11:21:22.150178+0000","last_undegraded":"2023-10-16T11:21:22.150178+0000","last_fullsized":"2023-10-16T11:21:22.150178+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:02.469221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.e","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.158981+0000","last_change":"2023-10-16T11:21:20.135672+0000","last_active":"2023-10-16T11:21:21.158981+0000","last_peered":"2023-10-16T11:21:21.158981+0000","last_clean":"2023-10-16T11:21:21.158981+0000","last_became_active":"2023-10-16T11:21:20.135413+0000","last_became_peered":"2023-10-16T11:21:20.135413+0000","last_unstale":"2023-10-16T11:21:21.158981+0000","last_undegraded":"2023-10-16T11:21:21.158981+0000","last_fullsized":"2023-10-16T11:21:21.158981+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:03.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149780+0000","last_change":"2023-10-16T11:21:21.163368+0000","last_active":"2023-10-16T11:21:22.149780+0000","last_peered":"2023-10-16T11:21:22.149780+0000","last_clean":"2023-10-16T11:21:22.149780+0000","last_became_active":"2023-10-16T11:21:21.163079+0000","last_became_peered":"2023-10-16T11:21:21.163079+0000","last_unstale":"2023-10-16T11:21:22.149780+0000","last_undegraded":"2023-10-16T11:21:22.149780+0000","last_fullsized":"2023-10-16T11:21:22.149780+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:50.345956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.d","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.163211+0000","last_change":"2023-10-16T11:21:20.139111+0000","last_active":"2023-10-16T11:21:21.163211+0000","last_peered":"2023-10-16T11:21:21.163211+0000","last_clean":"2023-10-16T11:21:21.163211+0000","last_became_active":"2023-10-16T11:21:20.138713+0000","last_became_peered":"2023-10-16T11:21:20.138713+0000","last_unstale":"2023-10-16T11:21:21.163211+0000","last_undegraded":"2023-10-16T11:21:21.163211+0000","last_fullsized":"2023-10-16T11:21:21.163211+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:27.847025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149823+0000","last_change":"2023-10-16T11:21:21.146872+0000","last_active":"2023-10-16T11:21:22.149823+0000","last_peered":"2023-10-16T11:21:22.149823+0000","last_clean":"2023-10-16T11:21:22.149823+0000","last_became_active":"2023-10-16T11:21:21.146581+0000","last_became_peered":"2023-10-16T11:21:21.146581+0000","last_unstale":"2023-10-16T11:21:22.149823+0000","last_undegraded":"2023-10-16T11:21:22.149823+0000","last_fullsized":"2023-10-16T11:21:22.149823+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:13:26.045032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.134555+0000","last_change":"2023-10-16T11:21:20.134928+0000","last_active":"2023-10-16T11:21:21.134555+0000","last_peered":"2023-10-16T11:21:21.134555+0000","last_clean":"2023-10-16T11:21:21.134555+0000","last_became_active":"2023-10-16T11:21:20.134219+0000","last_became_peered":"2023-10-16T11:21:20.134219+0000","last_unstale":"2023-10-16T11:21:21.134555+0000","last_undegraded":"2023-10-16T11:21:21.134555+0000","last_fullsized":"2023-10-16T11:21:21.134555+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:09.781648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392977+0000","last_change":"2023-10-16T11:21:21.158073+0000","last_active":"2023-10-16T11:21:23.392977+0000","last_peered":"2023-10-16T11:21:23.392977+0000","last_clean":"2023-10-16T11:21:23.392977+0000","last_became_active":"2023-10-16T11:21:21.157732+0000","last_became_peered":"2023-10-16T11:21:21.157732+0000","last_unstale":"2023-10-16T11:21:23.392977+0000","last_undegraded":"2023-10-16T11:21:23.392977+0000","last_fullsized":"2023-10-16T11:21:23.392977+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:55:44.633526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.b","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.157618+0000","last_change":"2023-10-16T11:21:20.134884+0000","last_active":"2023-10-16T11:21:21.157618+0000","last_peered":"2023-10-16T11:21:21.157618+0000","last_clean":"2023-10-16T11:21:21.157618+0000","last_became_active":"2023-10-16T11:21:20.134581+0000","last_became_peered":"2023-10-16T11:21:20.134581+0000","last_unstale":"2023-10-16T11:21:21.157618+0000","last_undegraded":"2023-10-16T11:21:21.157618+0000","last_fullsized":"2023-10-16T11:21:21.157618+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:39:24.151565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149786+0000","last_change":"2023-10-16T11:21:21.154639+0000","last_active":"2023-10-16T11:21:22.149786+0000","last_peered":"2023-10-16T11:21:22.149786+0000","last_clean":"2023-10-16T11:21:22.149786+0000","last_became_active":"2023-10-16T11:21:21.154244+0000","last_became_peered":"2023-10-16T11:21:21.154244+0000","last_unstale":"2023-10-16T11:21:22.149786+0000","last_undegraded":"2023-10-16T11:21:22.149786+0000","last_fullsized":"2023-10-16T11:21:22.149786+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:42.887120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137141+0000","last_change":"2023-10-16T11:21:20.127897+0000","last_active":"2023-10-16T11:21:21.137141+0000","last_peered":"2023-10-16T11:21:21.137141+0000","last_clean":"2023-10-16T11:21:21.137141+0000","last_became_active":"2023-10-16T11:21:20.127589+0000","last_became_peered":"2023-10-16T11:21:20.127589+0000","last_unstale":"2023-10-16T11:21:21.137141+0000","last_undegraded":"2023-10-16T11:21:21.137141+0000","last_fullsized":"2023-10-16T11:21:21.137141+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:21.578413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149350+0000","last_change":"2023-10-16T11:21:21.155072+0000","last_active":"2023-10-16T11:21:22.149350+0000","last_peered":"2023-10-16T11:21:22.149350+0000","last_clean":"2023-10-16T11:21:22.149350+0000","last_became_active":"2023-10-16T11:21:21.154630+0000","last_became_peered":"2023-10-16T11:21:21.154630+0000","last_unstale":"2023-10-16T11:21:22.149350+0000","last_undegraded":"2023-10-16T11:21:22.149350+0000","last_fullsized":"2023-10-16T11:21:22.149350+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:31.134795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.133715+0000","last_change":"2023-10-16T11:21:20.137514+0000","last_active":"2023-10-16T11:21:21.133715+0000","last_peered":"2023-10-16T11:21:21.133715+0000","last_clean":"2023-10-16T11:21:21.133715+0000","last_became_active":"2023-10-16T11:21:20.137300+0000","last_became_peered":"2023-10-16T11:21:20.137300+0000","last_unstale":"2023-10-16T11:21:21.133715+0000","last_undegraded":"2023-10-16T11:21:21.133715+0000","last_fullsized":"2023-10-16T11:21:21.133715+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:38.226471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393409+0000","last_change":"2023-10-16T11:21:21.155156+0000","last_active":"2023-10-16T11:21:23.393409+0000","last_peered":"2023-10-16T11:21:23.393409+0000","last_clean":"2023-10-16T11:21:23.393409+0000","last_became_active":"2023-10-16T11:21:21.154816+0000","last_became_peered":"2023-10-16T11:21:21.154816+0000","last_unstale":"2023-10-16T11:21:23.393409+0000","last_undegraded":"2023-10-16T11:21:23.393409+0000","last_fullsized":"2023-10-16T11:21:23.393409+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:54.214911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135698+0000","last_change":"2023-10-16T11:21:20.139486+0000","last_active":"2023-10-16T11:21:21.135698+0000","last_peered":"2023-10-16T11:21:21.135698+0000","last_clean":"2023-10-16T11:21:21.135698+0000","last_became_active":"2023-10-16T11:21:20.138918+0000","last_became_peered":"2023-10-16T11:21:20.138918+0000","last_unstale":"2023-10-16T11:21:21.135698+0000","last_undegraded":"2023-10-16T11:21:21.135698+0000","last_fullsized":"2023-10-16T11:21:21.135698+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:17.290575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084241+0000","last_change":"2023-10-16T11:21:21.146671+0000","last_active":"2023-10-16T11:21:23.084241+0000","last_peered":"2023-10-16T11:21:23.084241+0000","last_clean":"2023-10-16T11:21:23.084241+0000","last_became_active":"2023-10-16T11:21:21.146386+0000","last_became_peered":"2023-10-16T11:21:21.146386+0000","last_unstale":"2023-10-16T11:21:23.084241+0000","last_undegraded":"2023-10-16T11:21:23.084241+0000","last_fullsized":"2023-10-16T11:21:23.084241+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:13.153234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.7","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.161508+0000","last_change":"2023-10-16T11:21:20.135967+0000","last_active":"2023-10-16T11:21:21.161508+0000","last_peered":"2023-10-16T11:21:21.161508+0000","last_clean":"2023-10-16T11:21:21.161508+0000","last_became_active":"2023-10-16T11:21:20.135622+0000","last_became_peered":"2023-10-16T11:21:20.135622+0000","last_unstale":"2023-10-16T11:21:21.161508+0000","last_undegraded":"2023-10-16T11:21:21.161508+0000","last_fullsized":"2023-10-16T11:21:21.161508+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:23.358592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149789+0000","last_change":"2023-10-16T11:21:21.161249+0000","last_active":"2023-10-16T11:21:22.149789+0000","last_peered":"2023-10-16T11:21:22.149789+0000","last_clean":"2023-10-16T11:21:22.149789+0000","last_became_active":"2023-10-16T11:21:21.158669+0000","last_became_peered":"2023-10-16T11:21:21.158669+0000","last_unstale":"2023-10-16T11:21:22.149789+0000","last_undegraded":"2023-10-16T11:21:22.149789+0000","last_fullsized":"2023-10-16T11:21:22.149789+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:11.911562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.152367+0000","last_change":"2023-10-16T11:21:20.131488+0000","last_active":"2023-10-16T11:21:21.152367+0000","last_peered":"2023-10-16T11:21:21.152367+0000","last_clean":"2023-10-16T11:21:21.152367+0000","last_became_active":"2023-10-16T11:21:20.131214+0000","last_became_peered":"2023-10-16T11:21:20.131214+0000","last_unstale":"2023-10-16T11:21:21.152367+0000","last_undegraded":"2023-10-16T11:21:21.152367+0000","last_fullsized":"2023-10-16T11:21:21.152367+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:23:21.659484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084243+0000","last_change":"2023-10-16T11:21:21.147831+0000","last_active":"2023-10-16T11:21:23.084243+0000","last_peered":"2023-10-16T11:21:23.084243+0000","last_clean":"2023-10-16T11:21:23.084243+0000","last_became_active":"2023-10-16T11:21:21.147568+0000","last_became_peered":"2023-10-16T11:21:21.147568+0000","last_unstale":"2023-10-16T11:21:23.084243+0000","last_undegraded":"2023-10-16T11:21:23.084243+0000","last_fullsized":"2023-10-16T11:21:23.084243+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:34:07.295356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.142597+0000","last_change":"2023-10-16T11:21:20.136215+0000","last_active":"2023-10-16T11:21:21.142597+0000","last_peered":"2023-10-16T11:21:21.142597+0000","last_clean":"2023-10-16T11:21:21.142597+0000","last_became_active":"2023-10-16T11:21:20.135897+0000","last_became_peered":"2023-10-16T11:21:20.135897+0000","last_unstale":"2023-10-16T11:21:21.142597+0000","last_undegraded":"2023-10-16T11:21:21.142597+0000","last_fullsized":"2023-10-16T11:21:21.142597+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:23.641201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.150123+0000","last_change":"2023-10-16T11:21:21.145015+0000","last_active":"2023-10-16T11:21:22.150123+0000","last_peered":"2023-10-16T11:21:22.150123+0000","last_clean":"2023-10-16T11:21:22.150123+0000","last_became_active":"2023-10-16T11:21:21.144545+0000","last_became_peered":"2023-10-16T11:21:21.144545+0000","last_unstale":"2023-10-16T11:21:22.150123+0000","last_undegraded":"2023-10-16T11:21:22.150123+0000","last_fullsized":"2023-10-16T11:21:22.150123+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:51.353240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.144404+0000","last_change":"2023-10-16T11:21:20.127796+0000","last_active":"2023-10-16T11:21:21.144404+0000","last_peered":"2023-10-16T11:21:21.144404+0000","last_clean":"2023-10-16T11:21:21.144404+0000","last_became_active":"2023-10-16T11:21:20.127496+0000","last_became_peered":"2023-10-16T11:21:20.127496+0000","last_unstale":"2023-10-16T11:21:21.144404+0000","last_undegraded":"2023-10-16T11:21:21.144404+0000","last_fullsized":"2023-10-16T11:21:21.144404+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:07.392329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393134+0000","last_change":"2023-10-16T11:21:21.157926+0000","last_active":"2023-10-16T11:21:23.393134+0000","last_peered":"2023-10-16T11:21:23.393134+0000","last_clean":"2023-10-16T11:21:23.393134+0000","last_became_active":"2023-10-16T11:21:21.157376+0000","last_became_peered":"2023-10-16T11:21:21.157376+0000","last_unstale":"2023-10-16T11:21:23.393134+0000","last_undegraded":"2023-10-16T11:21:23.393134+0000","last_fullsized":"2023-10-16T11:21:23.393134+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:10.128277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135642+0000","last_change":"2023-10-16T11:21:20.140881+0000","last_active":"2023-10-16T11:21:21.135642+0000","last_peered":"2023-10-16T11:21:21.135642+0000","last_clean":"2023-10-16T11:21:21.135642+0000","last_became_active":"2023-10-16T11:21:20.140500+0000","last_became_peered":"2023-10-16T11:21:20.140500+0000","last_unstale":"2023-10-16T11:21:21.135642+0000","last_undegraded":"2023-10-16T11:21:21.135642+0000","last_fullsized":"2023-10-16T11:21:21.135642+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:01.038508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149816+0000","last_change":"2023-10-16T11:21:21.161571+0000","last_active":"2023-10-16T11:21:22.149816+0000","last_peered":"2023-10-16T11:21:22.149816+0000","last_clean":"2023-10-16T11:21:22.149816+0000","last_became_active":"2023-10-16T11:21:21.158952+0000","last_became_peered":"2023-10-16T11:21:21.158952+0000","last_unstale":"2023-10-16T11:21:22.149816+0000","last_undegraded":"2023-10-16T11:21:22.149816+0000","last_fullsized":"2023-10-16T11:21:22.149816+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:34.182621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135578+0000","last_change":"2023-10-16T11:21:20.139168+0000","last_active":"2023-10-16T11:21:21.135578+0000","last_peered":"2023-10-16T11:21:21.135578+0000","last_clean":"2023-10-16T11:21:21.135578+0000","last_became_active":"2023-10-16T11:21:20.138825+0000","last_became_peered":"2023-10-16T11:21:20.138825+0000","last_unstale":"2023-10-16T11:21:21.135578+0000","last_undegraded":"2023-10-16T11:21:21.135578+0000","last_fullsized":"2023-10-16T11:21:21.135578+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:30.130495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149496+0000","last_change":"2023-10-16T11:21:21.154880+0000","last_active":"2023-10-16T11:21:22.149496+0000","last_peered":"2023-10-16T11:21:22.149496+0000","last_clean":"2023-10-16T11:21:22.149496+0000","last_became_active":"2023-10-16T11:21:21.154247+0000","last_became_peered":"2023-10-16T11:21:21.154247+0000","last_unstale":"2023-10-16T11:21:22.149496+0000","last_undegraded":"2023-10-16T11:21:22.149496+0000","last_fullsized":"2023-10-16T11:21:22.149496+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:19:11.835144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137434+0000","last_change":"2023-10-16T11:21:20.139225+0000","last_active":"2023-10-16T11:21:21.137434+0000","last_peered":"2023-10-16T11:21:21.137434+0000","last_clean":"2023-10-16T11:21:21.137434+0000","last_became_active":"2023-10-16T11:21:20.138671+0000","last_became_peered":"2023-10-16T11:21:20.138671+0000","last_unstale":"2023-10-16T11:21:21.137434+0000","last_undegraded":"2023-10-16T11:21:21.137434+0000","last_fullsized":"2023-10-16T11:21:21.137434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:10.907262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393297+0000","last_change":"2023-10-16T11:21:21.150780+0000","last_active":"2023-10-16T11:21:23.393297+0000","last_peered":"2023-10-16T11:21:23.393297+0000","last_clean":"2023-10-16T11:21:23.393297+0000","last_became_active":"2023-10-16T11:21:21.150505+0000","last_became_peered":"2023-10-16T11:21:21.150505+0000","last_unstale":"2023-10-16T11:21:23.393297+0000","last_undegraded":"2023-10-16T11:21:23.393297+0000","last_fullsized":"2023-10-16T11:21:23.393297+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:45.993140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135545+0000","last_change":"2023-10-16T11:21:20.129727+0000","last_active":"2023-10-16T11:21:21.135545+0000","last_peered":"2023-10-16T11:21:21.135545+0000","last_clean":"2023-10-16T11:21:21.135545+0000","last_became_active":"2023-10-16T11:21:20.129419+0000","last_became_peered":"2023-10-16T11:21:20.129419+0000","last_unstale":"2023-10-16T11:21:21.135545+0000","last_undegraded":"2023-10-16T11:21:21.135545+0000","last_fullsized":"2023-10-16T11:21:21.135545+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:00.056698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.22","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148490+0000","last_change":"2023-10-16T11:21:21.162583+0000","last_active":"2023-10-16T11:21:22.148490+0000","last_peered":"2023-10-16T11:21:22.148490+0000","last_clean":"2023-10-16T11:21:22.148490+0000","last_became_active":"2023-10-16T11:21:21.162198+0000","last_became_peered":"2023-10-16T11:21:21.162198+0000","last_unstale":"2023-10-16T11:21:22.148490+0000","last_undegraded":"2023-10-16T11:21:22.148490+0000","last_fullsized":"2023-10-16T11:21:22.148490+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:59.212211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"242'381","reported_seq":1367,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.142151+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:21:21.142151+0000","last_peered":"2023-10-16T11:21:21.142151+0000","last_clean":"2023-10-16T11:21:21.142151+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:21:21.142151+0000","last_undegraded":"2023-10-16T11:21:21.142151+0000","last_fullsized":"2023-10-16T11:21:21.142151+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":35,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":34,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543300,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52964,"kb_used_data":3736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318876,"statfs":{"total":96636764160,"available":96582529024,"internally_reserved":0,"allocated":3825664,"data_stored":2269748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.878,"15min":0.878},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27800000000000002},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64900000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.503}]},{"osd":2,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55100000000000005,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.31900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69799999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.628}]},{"osd":3,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":1,"15min":1.002},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68400000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60399999999999998,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84299999999999997,"5min":1.052,"15min":1.052},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68300000000000005}]}]},{"osd":6,"up_from":14,"seq":60129543299,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60712,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311128,"statfs":{"total":96636764160,"available":96574595072,"internally_reserved":0,"allocated":3235840,"data_stored":1679470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54500000000000004,"15min":0.58199999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.84499999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.78100000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.63400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57699999999999996,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.68500000000000005,"5min":0.77200000000000002,"15min":0.86499999999999999},"last":0.879}]},{"osd":2,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.57299999999999995,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.66300000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78800000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28100000000000003},"max":{"1min":0.747,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.496,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.73799999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59799999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.81399999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81299999999999994}]}]},{"osd":1,"up_from":14,"seq":60129543299,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60708,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311132,"statfs":{"total":96636764160,"available":96574599168,"internally_reserved":0,"allocated":3235840,"data_stored":1679414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.38300000000000001,"15min":0.40100000000000002},"min":{"1min":0.23400000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.52600000000000002,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.39100000000000001,"15min":0.42599999999999999},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":0.62,"15min":0.91700000000000004},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.38400000000000001,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.44900000000000001,"5min":0.56999999999999995,"15min":0.98899999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.42199999999999999,"15min":0.45700000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47599999999999998,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.434}]},{"osd":3,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.443,"15min":0.48399999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.51300000000000001,"5min":0.69699999999999995,"15min":0.78300000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.435,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.43099999999999999,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81799999999999995,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.66200000000000003,"15min":0.68000000000000005},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.73799999999999999,"5min":1.0289999999999999,"15min":1.097},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.66900000000000004},"min":{"1min":0.41899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72499999999999998,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64800000000000002,"15min":0.68500000000000005},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76900000000000002,"5min":0.88200000000000001,"15min":1.147},"last":0.505}]},{"osd":6,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80600000000000005,"5min":0.98699999999999999,"15min":1.0509999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.436,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82899999999999996,"5min":1.004,"15min":1.0600000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.67200000000000004,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76800000000000002,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.55100000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543300,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61300,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310540,"statfs":{"total":96636764160,"available":96573992960,"internally_reserved":0,"allocated":3837952,"data_stored":2270386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.22,"15min":0.214},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.39200000000000002,"15min":0.41099999999999998},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.65100000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.70299999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.42399999999999999,"15min":0.44400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.377}]},{"osd":3,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42499999999999999,"15min":0.44800000000000001},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.47399999999999998,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.84599999999999997},"last":0.39300000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66000000000000003,"15min":0.66700000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":1.008},"last":0.443}]},{"osd":5,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79900000000000004,"5min":1.004,"15min":1.008},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81999999999999995,"5min":0.94199999999999995,"15min":1.1419999999999999},"last":0.42299999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65100000000000002,"15min":0.68100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79000000000000004,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.5}]},{"osd":7,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65600000000000003,"15min":0.68400000000000005},"min":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74099999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.49199999999999999}]}]},{"osd":2,"up_from":14,"seq":60129543299,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48232,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323608,"statfs":{"total":96636764160,"available":96587374592,"internally_reserved":0,"allocated":3235840,"data_stored":1679470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.433,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.57299999999999995,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.436,"15min":0.441},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.441,"15min":0.45800000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.44800000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.163,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69899999999999995,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":3.593,"15min":3.593},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.64700000000000002,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73499999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67700000000000005,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67500000000000004,"15min":0.68999999999999995},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.73599999999999999,"5min":3.633,"15min":3.633},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.67200000000000004,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.70799999999999996,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66300000000000003,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.70699999999999996,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.67400000000000004}]}]},{"osd":3,"up_from":14,"seq":60129543300,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48236,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323604,"statfs":{"total":96636764160,"available":96587370496,"internally_reserved":0,"allocated":3244032,"data_stored":1680992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40200000000000002,"15min":0.41099999999999998},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.52600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.41799999999999998,"15min":0.44400000000000001},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.40899999999999997,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.38800000000000001,"15min":0.41499999999999998},"min":{"1min":0.24299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.42899999999999999,"5min":0.87,"15min":0.87},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40699999999999997,"15min":0.441},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.41199999999999998,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.42899999999999999,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.48099999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.44800000000000001,"15min":0.47499999999999998},"min":{"1min":0.33600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.503}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.73699999999999999,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71299999999999997,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.626,"15min":0.66200000000000003},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63300000000000001,"15min":0.66400000000000003},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.623,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.377,"15min":0.377},"max":{"1min":0.73499999999999999,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63700000000000001,"15min":0.67700000000000005},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.81799999999999995}]}]},{"osd":4,"up_from":14,"seq":60129543299,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96744,"kb_used_data":3164,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94275096,"statfs":{"total":96636764160,"available":96537698304,"internally_reserved":0,"allocated":3239936,"data_stored":1679472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.62,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":0.84399999999999997},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58699999999999997,"15min":0.61299999999999999},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68300000000000005,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.436},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.502}]},{"osd":2,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66000000000000003,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.68799999999999994,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.58599999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.65200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.64200000000000002},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.70099999999999996,"5min":0.81299999999999994,"15min":0.879},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.72399999999999998,"5min":0.81299999999999994,"15min":0.84099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.63800000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.29399999999999998},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.71399999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.875,"15min":0.875},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.72899999999999998}]}]},{"osd":5,"up_from":14,"seq":60129543300,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60724,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311116,"statfs":{"total":96636764160,"available":96574582784,"internally_reserved":0,"allocated":3252224,"data_stored":1681644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.504},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":1.873,"15min":1.873},"last":0.61099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.024,"15min":1.024},"last":0.55100000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88700000000000001,"5min":1.821,"15min":1.821},"last":0.437},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63100000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.65500000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.54300000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.878,"15min":0.878},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40799999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80000000000000004,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86099999999999999,"5min":2.02,"15min":2.02},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:21:25.268 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:21:25.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.420+0000 7efd396bf700 1 Processor -- start 2023-10-16T11:21:25.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.420+0000 7efd396bf700 1 -- start start 2023-10-16T11:21:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.421+0000 7efd396bf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 0x7efd34162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.421+0000 7efd396bf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd340703b0 con 0x7efd3406a8a0 2023-10-16T11:21:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.421+0000 7efd396bf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3415eeb0 con 0x7efd3415e700 2023-10-16T11:21:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.421+0000 7efd396bf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd341584d0 con 0x7efd340ce9a0 2023-10-16T11:21:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.421+0000 7efd327fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efd3406a8a0 0x7efd3415dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42734/0 (socket says 172.21.15.181:42734) 2023-10-16T11:21:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.421+0000 7efd327fc700 1 -- 172.21.15.181:0/3274704529 learned_addr learned my addr 172.21.15.181:0/3274704529 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd32ffd700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 0x7efd34162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259678180 0 0) 0x7efd340703b0 con 0x7efd3406a8a0 2023-10-16T11:21:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd1c0035b0 con 0x7efd3406a8a0 2023-10-16T11:21:25.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208085448 0 0) 0x7efd1c0035b0 con 0x7efd3406a8a0 2023-10-16T11:21:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 msgr2=0x7efd34162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 0x7efd34162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 >> v1:172.21.15.190:6789/0 conn(0x7efd3415e700 legacy=0x7efd34071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3415ebd0 con 0x7efd3406a8a0 2023-10-16T11:21:25.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7efd24001670 con 0x7efd3406a8a0 2023-10-16T11:21:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 >> v1:172.21.15.181:6789/0 conn(0x7efd3406a8a0 legacy=0x7efd3415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c003a30 0x7efd1c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 0x7efd1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00bac0 con 0x7efd1c003a30 2023-10-16T11:21:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00bc00 con 0x7efd1c004d90 2023-10-16T11:21:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00bdc0 con 0x7efd340ce9a0 2023-10-16T11:21:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd32ffd700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c003a30 0x7efd1c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd327fc700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 0x7efd1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd337fe700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd32ffd700 1 -- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 msgr2=0x7efd1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd32ffd700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd340ce9a0 0x7efd1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd32ffd700 1 -- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c004d90 msgr2=0x7efd1c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd32ffd700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd32ffd700 1 -- 172.21.15.181:0/3274704529 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c0035b0 con 0x7efd1c003a30 2023-10-16T11:21:25.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.422+0000 7efd337fe700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd32ffd700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c003a30 0x7efd1c003e50 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7efd28009b30 tx=0x7efd1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef49cc8def0f0fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800e6c0 con 0x7efd1c003a30 2023-10-16T11:21:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2800e820 con 0x7efd1c003a30 2023-10-16T11:21:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd31ffb700 1 -- 172.21.15.181:0/3274704529 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800eb90 con 0x7efd1c003a30 2023-10-16T11:21:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd396bf700 1 -- 172.21.15.181:0/3274704529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c003a30 msgr2=0x7efd1c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd396bf700 1 --2- 172.21.15.181:0/3274704529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd1c003a30 0x7efd1c003e50 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7efd28009b30 tx=0x7efd1c005880 comp rx=0 tx=0).stop 2023-10-16T11:21:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd337fe700 1 -- 172.21.15.181:0/3274704529 reap_dead start 2023-10-16T11:21:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd396bf700 1 -- 172.21.15.181:0/3274704529 shutdown_connections 2023-10-16T11:21:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd396bf700 1 -- 172.21.15.181:0/3274704529 >> 172.21.15.181:0/3274704529 conn(0x7efd340ccf80 msgr2=0x7efd340cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd396bf700 1 -- 172.21.15.181:0/3274704529 shutdown_connections 2023-10-16T11:21:25.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.423+0000 7efd396bf700 1 -- 172.21.15.181:0/3274704529 wait complete. 2023-10-16T11:21:25.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 Processor -- start 2023-10-16T11:21:25.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 -- start start 2023-10-16T11:21:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c003a30 0x7efd340b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd341f2fd0 0x7efd340aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd340ad5e0 0x7efd34154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340b3080 con 0x7efd341f2fd0 2023-10-16T11:21:25.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340b4ca0 con 0x7efd340ad5e0 2023-10-16T11:21:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd396bf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd341f3540 con 0x7efd1c003a30 2023-10-16T11:21:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd327fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd341f2fd0 0x7efd340aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd327fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd341f2fd0 0x7efd340aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59902/0 (socket says 172.21.15.181:59902) 2023-10-16T11:21:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd327fc700 1 -- 172.21.15.181:0/943480361 learned_addr learned my addr 172.21.15.181:0/943480361 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:25.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.425+0000 7efd337fe700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd340ad5e0 0x7efd34154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd32ffd700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c003a30 0x7efd340b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd327fc700 1 -- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c003a30 msgr2=0x7efd340b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd327fc700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c003a30 0x7efd340b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd327fc700 1 -- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd340ad5e0 msgr2=0x7efd34154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd327fc700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd340ad5e0 0x7efd34154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd327fc700 1 -- 172.21.15.181:0/943480361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd341f3820 con 0x7efd341f2fd0 2023-10-16T11:21:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd32ffd700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c003a30 0x7efd340b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd327fc700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd341f2fd0 0x7efd340aca40 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7efd240072f0 tx=0x7efd24007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd1b7fe700 1 -- 172.21.15.181:0/943480361 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2400f6c0 con 0x7efd341f2fd0 2023-10-16T11:21:25.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd1b7fe700 1 -- 172.21.15.181:0/943480361 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2400f820 con 0x7efd341f2fd0 2023-10-16T11:21:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd341f3b00 con 0x7efd341f2fd0 2023-10-16T11:21:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd1b7fe700 1 -- 172.21.15.181:0/943480361 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2400fbd0 con 0x7efd341f2fd0 2023-10-16T11:21:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.426+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd340acf80 con 0x7efd341f2fd0 2023-10-16T11:21:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.427+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd340b31f0 con 0x7efd341f2fd0 2023-10-16T11:21:25.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.427+0000 7efd1b7fe700 1 -- 172.21.15.181:0/943480361 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7efd2400fd30 con 0x7efd341f2fd0 2023-10-16T11:21:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.428+0000 7efd1b7fe700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7efd2006b140 0x7efd2006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.428+0000 7efd1b7fe700 1 -- 172.21.15.181:0/943480361 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7efd24089b60 con 0x7efd341f2fd0 2023-10-16T11:21:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.428+0000 7efd32ffd700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7efd2006b140 0x7efd2006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.428+0000 7efd32ffd700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7efd2006b140 0x7efd2006d600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd280090b0 tx=0x7efd28008030 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.432+0000 7efd1b7fe700 1 -- 172.21.15.181:0/943480361 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efd340b31f0 con 0x7efd341f2fd0 2023-10-16T11:21:25.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.620+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 --> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd34161960 con 0x7efd2006b140 2023-10-16T11:21:25.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.627+0000 7efd1b7fe700 1 -- 172.21.15.181:0/943480361 <== mgr.12564 v2:172.21.15.181:6832/3850286714 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290188 (secure 0 0 0) 0x7efd28022040 con 0x7efd2006b140 2023-10-16T11:21:25.633 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:25.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7efd2006b140 msgr2=0x7efd2006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7efd2006b140 0x7efd2006d600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd280090b0 tx=0x7efd28008030 comp rx=0 tx=0).stop 2023-10-16T11:21:25.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd341f2fd0 msgr2=0x7efd340aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd341f2fd0 0x7efd340aca40 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7efd240072f0 tx=0x7efd24007600 comp rx=0 tx=0).stop 2023-10-16T11:21:25.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 shutdown_connections 2023-10-16T11:21:25.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1c003a30 0x7efd340b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7efd2006b140 0x7efd2006d600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd340ad5e0 0x7efd34154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 --2- 172.21.15.181:0/943480361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd341f2fd0 0x7efd340aca40 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 >> 172.21.15.181:0/943480361 conn(0x7efd340ccf80 msgr2=0x7efd3406acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 shutdown_connections 2023-10-16T11:21:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.631+0000 7efd396bf700 1 -- 172.21.15.181:0/943480361 wait complete. 2023-10-16T11:21:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:21:25.646 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-16T11:21:24.938827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":683,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419,"ondisk_log_size":419,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":489620,"kb_used_data":26472,"kb_used_omap":12,"kb_used_meta":463027,"kb_avail":754485100,"statfs":{"total":773094113280,"available":772592742400,"internally_reserved":0,"allocated":27107328,"data_stored":14620596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":474140216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.820267"},"pg_stats":[{"pgid":"35.3f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393693+0000","last_change":"2023-10-16T11:21:21.149392+0000","last_active":"2023-10-16T11:21:23.393693+0000","last_peered":"2023-10-16T11:21:23.393693+0000","last_clean":"2023-10-16T11:21:23.393693+0000","last_became_active":"2023-10-16T11:21:21.149103+0000","last_became_peered":"2023-10-16T11:21:21.149103+0000","last_unstale":"2023-10-16T11:21:23.393693+0000","last_undegraded":"2023-10-16T11:21:23.393693+0000","last_fullsized":"2023-10-16T11:21:23.393693+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:55:44.633526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.3e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.150120+0000","last_change":"2023-10-16T11:21:21.148324+0000","last_active":"2023-10-16T11:21:22.150120+0000","last_peered":"2023-10-16T11:21:22.150120+0000","last_clean":"2023-10-16T11:21:22.150120+0000","last_became_active":"2023-10-16T11:21:21.148049+0000","last_became_peered":"2023-10-16T11:21:21.148049+0000","last_unstale":"2023-10-16T11:21:22.150120+0000","last_undegraded":"2023-10-16T11:21:22.150120+0000","last_fullsized":"2023-10-16T11:21:22.150120+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:36:43.435571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149959+0000","last_change":"2023-10-16T11:21:21.144352+0000","last_active":"2023-10-16T11:21:22.149959+0000","last_peered":"2023-10-16T11:21:22.149959+0000","last_clean":"2023-10-16T11:21:22.149959+0000","last_became_active":"2023-10-16T11:21:21.143694+0000","last_became_peered":"2023-10-16T11:21:21.143694+0000","last_unstale":"2023-10-16T11:21:22.149959+0000","last_undegraded":"2023-10-16T11:21:22.149959+0000","last_fullsized":"2023-10-16T11:21:22.149959+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:27:32.490879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393456+0000","last_change":"2023-10-16T11:21:21.149568+0000","last_active":"2023-10-16T11:21:23.393456+0000","last_peered":"2023-10-16T11:21:23.393456+0000","last_clean":"2023-10-16T11:21:23.393456+0000","last_became_active":"2023-10-16T11:21:21.149291+0000","last_became_peered":"2023-10-16T11:21:21.149291+0000","last_unstale":"2023-10-16T11:21:23.393456+0000","last_undegraded":"2023-10-16T11:21:23.393456+0000","last_fullsized":"2023-10-16T11:21:23.393456+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:55:58.394396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.3b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149926+0000","last_change":"2023-10-16T11:21:21.148349+0000","last_active":"2023-10-16T11:21:22.149926+0000","last_peered":"2023-10-16T11:21:22.149926+0000","last_clean":"2023-10-16T11:21:22.149926+0000","last_became_active":"2023-10-16T11:21:21.148069+0000","last_became_peered":"2023-10-16T11:21:21.148069+0000","last_unstale":"2023-10-16T11:21:22.149926+0000","last_undegraded":"2023-10-16T11:21:22.149926+0000","last_fullsized":"2023-10-16T11:21:22.149926+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:22:05.441449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149440+0000","last_change":"2023-10-16T11:21:21.146179+0000","last_active":"2023-10-16T11:21:22.149440+0000","last_peered":"2023-10-16T11:21:22.149440+0000","last_clean":"2023-10-16T11:21:22.149440+0000","last_became_active":"2023-10-16T11:21:21.145933+0000","last_became_peered":"2023-10-16T11:21:21.145933+0000","last_unstale":"2023-10-16T11:21:22.149440+0000","last_undegraded":"2023-10-16T11:21:22.149440+0000","last_fullsized":"2023-10-16T11:21:22.149440+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:36:22.850812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.39","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083952+0000","last_change":"2023-10-16T11:21:21.157892+0000","last_active":"2023-10-16T11:21:23.083952+0000","last_peered":"2023-10-16T11:21:23.083952+0000","last_clean":"2023-10-16T11:21:23.083952+0000","last_became_active":"2023-10-16T11:21:21.157634+0000","last_became_peered":"2023-10-16T11:21:21.157634+0000","last_unstale":"2023-10-16T11:21:23.083952+0000","last_undegraded":"2023-10-16T11:21:23.083952+0000","last_fullsized":"2023-10-16T11:21:23.083952+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:08.548262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.38","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083864+0000","last_change":"2023-10-16T11:21:21.146440+0000","last_active":"2023-10-16T11:21:23.083864+0000","last_peered":"2023-10-16T11:21:23.083864+0000","last_clean":"2023-10-16T11:21:23.083864+0000","last_became_active":"2023-10-16T11:21:21.146179+0000","last_became_peered":"2023-10-16T11:21:21.146179+0000","last_unstale":"2023-10-16T11:21:23.083864+0000","last_undegraded":"2023-10-16T11:21:23.083864+0000","last_fullsized":"2023-10-16T11:21:23.083864+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:15.377571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.37","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393287+0000","last_change":"2023-10-16T11:21:21.160261+0000","last_active":"2023-10-16T11:21:23.393287+0000","last_peered":"2023-10-16T11:21:23.393287+0000","last_clean":"2023-10-16T11:21:23.393287+0000","last_became_active":"2023-10-16T11:21:21.159989+0000","last_became_peered":"2023-10-16T11:21:21.159989+0000","last_unstale":"2023-10-16T11:21:23.393287+0000","last_undegraded":"2023-10-16T11:21:23.393287+0000","last_fullsized":"2023-10-16T11:21:23.393287+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:46:59.089164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.36","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393670+0000","last_change":"2023-10-16T11:21:21.143017+0000","last_active":"2023-10-16T11:21:23.393670+0000","last_peered":"2023-10-16T11:21:23.393670+0000","last_clean":"2023-10-16T11:21:23.393670+0000","last_became_active":"2023-10-16T11:21:21.142709+0000","last_became_peered":"2023-10-16T11:21:21.142709+0000","last_unstale":"2023-10-16T11:21:23.393670+0000","last_undegraded":"2023-10-16T11:21:23.393670+0000","last_fullsized":"2023-10-16T11:21:23.393670+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:30.129457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.35","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148987+0000","last_change":"2023-10-16T11:21:21.148407+0000","last_active":"2023-10-16T11:21:22.148987+0000","last_peered":"2023-10-16T11:21:22.148987+0000","last_clean":"2023-10-16T11:21:22.148987+0000","last_became_active":"2023-10-16T11:21:21.148146+0000","last_became_peered":"2023-10-16T11:21:21.148146+0000","last_unstale":"2023-10-16T11:21:22.148987+0000","last_undegraded":"2023-10-16T11:21:22.148987+0000","last_fullsized":"2023-10-16T11:21:22.148987+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:40.028718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.34","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392998+0000","last_change":"2023-10-16T11:21:21.144160+0000","last_active":"2023-10-16T11:21:23.392998+0000","last_peered":"2023-10-16T11:21:23.392998+0000","last_clean":"2023-10-16T11:21:23.392998+0000","last_became_active":"2023-10-16T11:21:21.143889+0000","last_became_peered":"2023-10-16T11:21:21.143889+0000","last_unstale":"2023-10-16T11:21:23.392998+0000","last_undegraded":"2023-10-16T11:21:23.392998+0000","last_fullsized":"2023-10-16T11:21:23.392998+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:17.650622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.33","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393179+0000","last_change":"2023-10-16T11:21:21.144120+0000","last_active":"2023-10-16T11:21:23.393179+0000","last_peered":"2023-10-16T11:21:23.393179+0000","last_clean":"2023-10-16T11:21:23.393179+0000","last_became_active":"2023-10-16T11:21:21.143867+0000","last_became_peered":"2023-10-16T11:21:21.143867+0000","last_unstale":"2023-10-16T11:21:23.393179+0000","last_undegraded":"2023-10-16T11:21:23.393179+0000","last_fullsized":"2023-10-16T11:21:23.393179+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:27:05.606674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.32","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393433+0000","last_change":"2023-10-16T11:21:21.148316+0000","last_active":"2023-10-16T11:21:23.393433+0000","last_peered":"2023-10-16T11:21:23.393433+0000","last_clean":"2023-10-16T11:21:23.393433+0000","last_became_active":"2023-10-16T11:21:21.148007+0000","last_became_peered":"2023-10-16T11:21:21.148007+0000","last_unstale":"2023-10-16T11:21:23.393433+0000","last_undegraded":"2023-10-16T11:21:23.393433+0000","last_fullsized":"2023-10-16T11:21:23.393433+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:36:29.400971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.31","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083869+0000","last_change":"2023-10-16T11:21:21.146254+0000","last_active":"2023-10-16T11:21:23.083869+0000","last_peered":"2023-10-16T11:21:23.083869+0000","last_clean":"2023-10-16T11:21:23.083869+0000","last_became_active":"2023-10-16T11:21:21.144477+0000","last_became_peered":"2023-10-16T11:21:21.144477+0000","last_unstale":"2023-10-16T11:21:23.083869+0000","last_undegraded":"2023-10-16T11:21:23.083869+0000","last_fullsized":"2023-10-16T11:21:23.083869+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:14:43.351438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.30","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083867+0000","last_change":"2023-10-16T11:21:21.148636+0000","last_active":"2023-10-16T11:21:23.083867+0000","last_peered":"2023-10-16T11:21:23.083867+0000","last_clean":"2023-10-16T11:21:23.083867+0000","last_became_active":"2023-10-16T11:21:21.148023+0000","last_became_peered":"2023-10-16T11:21:21.148023+0000","last_unstale":"2023-10-16T11:21:23.083867+0000","last_undegraded":"2023-10-16T11:21:23.083867+0000","last_fullsized":"2023-10-16T11:21:23.083867+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:23.579378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393400+0000","last_change":"2023-10-16T11:21:21.162130+0000","last_active":"2023-10-16T11:21:23.393400+0000","last_peered":"2023-10-16T11:21:23.393400+0000","last_clean":"2023-10-16T11:21:23.393400+0000","last_became_active":"2023-10-16T11:21:21.161853+0000","last_became_peered":"2023-10-16T11:21:21.161853+0000","last_unstale":"2023-10-16T11:21:23.393400+0000","last_undegraded":"2023-10-16T11:21:23.393400+0000","last_fullsized":"2023-10-16T11:21:23.393400+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:54:34.097636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149924+0000","last_change":"2023-10-16T11:21:21.155001+0000","last_active":"2023-10-16T11:21:22.149924+0000","last_peered":"2023-10-16T11:21:22.149924+0000","last_clean":"2023-10-16T11:21:22.149924+0000","last_became_active":"2023-10-16T11:21:21.153648+0000","last_became_peered":"2023-10-16T11:21:21.153648+0000","last_unstale":"2023-10-16T11:21:22.149924+0000","last_undegraded":"2023-10-16T11:21:22.149924+0000","last_fullsized":"2023-10-16T11:21:22.149924+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:55.513062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.2d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084309+0000","last_change":"2023-10-16T11:21:21.144577+0000","last_active":"2023-10-16T11:21:23.084309+0000","last_peered":"2023-10-16T11:21:23.084309+0000","last_clean":"2023-10-16T11:21:23.084309+0000","last_became_active":"2023-10-16T11:21:21.144057+0000","last_became_peered":"2023-10-16T11:21:21.144057+0000","last_unstale":"2023-10-16T11:21:23.084309+0000","last_undegraded":"2023-10-16T11:21:23.084309+0000","last_fullsized":"2023-10-16T11:21:23.084309+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:19.324785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084129+0000","last_change":"2023-10-16T11:21:21.143333+0000","last_active":"2023-10-16T11:21:23.084129+0000","last_peered":"2023-10-16T11:21:23.084129+0000","last_clean":"2023-10-16T11:21:23.084129+0000","last_became_active":"2023-10-16T11:21:21.141836+0000","last_became_peered":"2023-10-16T11:21:21.141836+0000","last_unstale":"2023-10-16T11:21:23.084129+0000","last_undegraded":"2023-10-16T11:21:23.084129+0000","last_fullsized":"2023-10-16T11:21:23.084129+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:52:04.333555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393303+0000","last_change":"2023-10-16T11:21:21.145062+0000","last_active":"2023-10-16T11:21:23.393303+0000","last_peered":"2023-10-16T11:21:23.393303+0000","last_clean":"2023-10-16T11:21:23.393303+0000","last_became_active":"2023-10-16T11:21:21.144757+0000","last_became_peered":"2023-10-16T11:21:21.144757+0000","last_unstale":"2023-10-16T11:21:23.393303+0000","last_undegraded":"2023-10-16T11:21:23.393303+0000","last_fullsized":"2023-10-16T11:21:23.393303+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:23.075210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.2a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149523+0000","last_change":"2023-10-16T11:21:21.163737+0000","last_active":"2023-10-16T11:21:22.149523+0000","last_peered":"2023-10-16T11:21:22.149523+0000","last_clean":"2023-10-16T11:21:22.149523+0000","last_became_active":"2023-10-16T11:21:21.163442+0000","last_became_peered":"2023-10-16T11:21:21.163442+0000","last_unstale":"2023-10-16T11:21:22.149523+0000","last_undegraded":"2023-10-16T11:21:22.149523+0000","last_fullsized":"2023-10-16T11:21:22.149523+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:05:50.445572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.29","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392755+0000","last_change":"2023-10-16T11:21:21.145490+0000","last_active":"2023-10-16T11:21:23.392755+0000","last_peered":"2023-10-16T11:21:23.392755+0000","last_clean":"2023-10-16T11:21:23.392755+0000","last_became_active":"2023-10-16T11:21:21.145238+0000","last_became_peered":"2023-10-16T11:21:21.145238+0000","last_unstale":"2023-10-16T11:21:23.392755+0000","last_undegraded":"2023-10-16T11:21:23.392755+0000","last_fullsized":"2023-10-16T11:21:23.392755+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:02.678200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.28","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149276+0000","last_change":"2023-10-16T11:21:21.150825+0000","last_active":"2023-10-16T11:21:22.149276+0000","last_peered":"2023-10-16T11:21:22.149276+0000","last_clean":"2023-10-16T11:21:22.149276+0000","last_became_active":"2023-10-16T11:21:21.150487+0000","last_became_peered":"2023-10-16T11:21:21.150487+0000","last_unstale":"2023-10-16T11:21:22.149276+0000","last_undegraded":"2023-10-16T11:21:22.149276+0000","last_fullsized":"2023-10-16T11:21:22.149276+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:35:15.882753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.27","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393242+0000","last_change":"2023-10-16T11:21:21.150391+0000","last_active":"2023-10-16T11:21:23.393242+0000","last_peered":"2023-10-16T11:21:23.393242+0000","last_clean":"2023-10-16T11:21:23.393242+0000","last_became_active":"2023-10-16T11:21:21.150111+0000","last_became_peered":"2023-10-16T11:21:21.150111+0000","last_unstale":"2023-10-16T11:21:23.393242+0000","last_undegraded":"2023-10-16T11:21:23.393242+0000","last_fullsized":"2023-10-16T11:21:23.393242+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:04:57.942362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.26","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393600+0000","last_change":"2023-10-16T11:21:21.147732+0000","last_active":"2023-10-16T11:21:23.393600+0000","last_peered":"2023-10-16T11:21:23.393600+0000","last_clean":"2023-10-16T11:21:23.393600+0000","last_became_active":"2023-10-16T11:21:21.147428+0000","last_became_peered":"2023-10-16T11:21:21.147428+0000","last_unstale":"2023-10-16T11:21:23.393600+0000","last_undegraded":"2023-10-16T11:21:23.393600+0000","last_fullsized":"2023-10-16T11:21:23.393600+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:26.284941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.25","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084183+0000","last_change":"2023-10-16T11:21:21.155577+0000","last_active":"2023-10-16T11:21:23.084183+0000","last_peered":"2023-10-16T11:21:23.084183+0000","last_clean":"2023-10-16T11:21:23.084183+0000","last_became_active":"2023-10-16T11:21:21.154302+0000","last_became_peered":"2023-10-16T11:21:21.154302+0000","last_unstale":"2023-10-16T11:21:23.084183+0000","last_undegraded":"2023-10-16T11:21:23.084183+0000","last_fullsized":"2023-10-16T11:21:23.084183+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:18:12.039616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.24","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393597+0000","last_change":"2023-10-16T11:21:21.150105+0000","last_active":"2023-10-16T11:21:23.393597+0000","last_peered":"2023-10-16T11:21:23.393597+0000","last_clean":"2023-10-16T11:21:23.393597+0000","last_became_active":"2023-10-16T11:21:21.149841+0000","last_became_peered":"2023-10-16T11:21:21.149841+0000","last_unstale":"2023-10-16T11:21:23.393597+0000","last_undegraded":"2023-10-16T11:21:23.393597+0000","last_fullsized":"2023-10-16T11:21:23.393597+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:46:49.072960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.23","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083917+0000","last_change":"2023-10-16T11:21:21.144111+0000","last_active":"2023-10-16T11:21:23.083917+0000","last_peered":"2023-10-16T11:21:23.083917+0000","last_clean":"2023-10-16T11:21:23.083917+0000","last_became_active":"2023-10-16T11:21:21.143779+0000","last_became_peered":"2023-10-16T11:21:21.143779+0000","last_unstale":"2023-10-16T11:21:23.083917+0000","last_undegraded":"2023-10-16T11:21:23.083917+0000","last_fullsized":"2023-10-16T11:21:23.083917+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:36:22.617473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.21","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084063+0000","last_change":"2023-10-16T11:21:21.144235+0000","last_active":"2023-10-16T11:21:23.084063+0000","last_peered":"2023-10-16T11:21:23.084063+0000","last_clean":"2023-10-16T11:21:23.084063+0000","last_became_active":"2023-10-16T11:21:21.143613+0000","last_became_peered":"2023-10-16T11:21:21.143613+0000","last_unstale":"2023-10-16T11:21:23.084063+0000","last_undegraded":"2023-10-16T11:21:23.084063+0000","last_fullsized":"2023-10-16T11:21:23.084063+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:54:27.659969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.20","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392836+0000","last_change":"2023-10-16T11:21:21.158661+0000","last_active":"2023-10-16T11:21:23.392836+0000","last_peered":"2023-10-16T11:21:23.392836+0000","last_clean":"2023-10-16T11:21:23.392836+0000","last_became_active":"2023-10-16T11:21:21.158356+0000","last_became_peered":"2023-10-16T11:21:21.158356+0000","last_unstale":"2023-10-16T11:21:23.392836+0000","last_undegraded":"2023-10-16T11:21:23.392836+0000","last_fullsized":"2023-10-16T11:21:23.392836+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:22:26.284941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393672+0000","last_change":"2023-10-16T11:21:21.156993+0000","last_active":"2023-10-16T11:21:23.393672+0000","last_peered":"2023-10-16T11:21:23.393672+0000","last_clean":"2023-10-16T11:21:23.393672+0000","last_became_active":"2023-10-16T11:21:21.156687+0000","last_became_peered":"2023-10-16T11:21:21.156687+0000","last_unstale":"2023-10-16T11:21:23.393672+0000","last_undegraded":"2023-10-16T11:21:23.393672+0000","last_fullsized":"2023-10-16T11:21:23.393672+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:05:31.538501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1f","version":"255'4","reported_seq":16,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.164073+0000","last_change":"2023-10-16T11:21:20.134648+0000","last_active":"2023-10-16T11:21:21.164073+0000","last_peered":"2023-10-16T11:21:21.164073+0000","last_clean":"2023-10-16T11:21:21.164073+0000","last_became_active":"2023-10-16T11:21:20.134317+0000","last_became_peered":"2023-10-16T11:21:20.134317+0000","last_unstale":"2023-10-16T11:21:21.164073+0000","last_undegraded":"2023-10-16T11:21:21.164073+0000","last_fullsized":"2023-10-16T11:21:21.164073+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:04.691534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149066+0000","last_change":"2023-10-16T11:21:21.156017+0000","last_active":"2023-10-16T11:21:22.149066+0000","last_peered":"2023-10-16T11:21:22.149066+0000","last_clean":"2023-10-16T11:21:22.149066+0000","last_became_active":"2023-10-16T11:21:21.155761+0000","last_became_peered":"2023-10-16T11:21:21.155761+0000","last_unstale":"2023-10-16T11:21:22.149066+0000","last_undegraded":"2023-10-16T11:21:22.149066+0000","last_fullsized":"2023-10-16T11:21:22.149066+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:02.678200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.1e","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.162089+0000","last_change":"2023-10-16T11:21:20.140722+0000","last_active":"2023-10-16T11:21:21.162089+0000","last_peered":"2023-10-16T11:21:21.162089+0000","last_clean":"2023-10-16T11:21:21.162089+0000","last_became_active":"2023-10-16T11:21:20.140377+0000","last_became_peered":"2023-10-16T11:21:20.140377+0000","last_unstale":"2023-10-16T11:21:21.162089+0000","last_undegraded":"2023-10-16T11:21:21.162089+0000","last_fullsized":"2023-10-16T11:21:21.162089+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:22.070911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149523+0000","last_change":"2023-10-16T11:21:21.145767+0000","last_active":"2023-10-16T11:21:22.149523+0000","last_peered":"2023-10-16T11:21:22.149523+0000","last_clean":"2023-10-16T11:21:22.149523+0000","last_became_active":"2023-10-16T11:21:21.145299+0000","last_became_peered":"2023-10-16T11:21:21.145299+0000","last_unstale":"2023-10-16T11:21:22.149523+0000","last_undegraded":"2023-10-16T11:21:22.149523+0000","last_fullsized":"2023-10-16T11:21:22.149523+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:51.747625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1d","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.157335+0000","last_change":"2023-10-16T11:21:20.140601+0000","last_active":"2023-10-16T11:21:21.157335+0000","last_peered":"2023-10-16T11:21:21.157335+0000","last_clean":"2023-10-16T11:21:21.157335+0000","last_became_active":"2023-10-16T11:21:20.140319+0000","last_became_peered":"2023-10-16T11:21:20.140319+0000","last_unstale":"2023-10-16T11:21:21.157335+0000","last_undegraded":"2023-10-16T11:21:21.157335+0000","last_fullsized":"2023-10-16T11:21:21.157335+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:33.178321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149610+0000","last_change":"2023-10-16T11:21:21.144827+0000","last_active":"2023-10-16T11:21:22.149610+0000","last_peered":"2023-10-16T11:21:22.149610+0000","last_clean":"2023-10-16T11:21:22.149610+0000","last_became_active":"2023-10-16T11:21:21.144086+0000","last_became_peered":"2023-10-16T11:21:21.144086+0000","last_unstale":"2023-10-16T11:21:22.149610+0000","last_undegraded":"2023-10-16T11:21:22.149610+0000","last_fullsized":"2023-10-16T11:21:22.149610+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:45:21.800256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.161466+0000","last_change":"2023-10-16T11:21:20.133950+0000","last_active":"2023-10-16T11:21:21.161466+0000","last_peered":"2023-10-16T11:21:21.161466+0000","last_clean":"2023-10-16T11:21:21.161466+0000","last_became_active":"2023-10-16T11:21:20.133490+0000","last_became_peered":"2023-10-16T11:21:20.133490+0000","last_unstale":"2023-10-16T11:21:21.161466+0000","last_undegraded":"2023-10-16T11:21:21.161466+0000","last_fullsized":"2023-10-16T11:21:21.161466+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:17.126514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392753+0000","last_change":"2023-10-16T11:21:21.151705+0000","last_active":"2023-10-16T11:21:23.392753+0000","last_peered":"2023-10-16T11:21:23.392753+0000","last_clean":"2023-10-16T11:21:23.392753+0000","last_became_active":"2023-10-16T11:21:21.150058+0000","last_became_peered":"2023-10-16T11:21:21.150058+0000","last_unstale":"2023-10-16T11:21:23.392753+0000","last_undegraded":"2023-10-16T11:21:23.392753+0000","last_fullsized":"2023-10-16T11:21:23.392753+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:39:29.350464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1b","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.163348+0000","last_change":"2023-10-16T11:21:20.133303+0000","last_active":"2023-10-16T11:21:21.163348+0000","last_peered":"2023-10-16T11:21:21.163348+0000","last_clean":"2023-10-16T11:21:21.163348+0000","last_became_active":"2023-10-16T11:21:20.133032+0000","last_became_peered":"2023-10-16T11:21:20.133032+0000","last_unstale":"2023-10-16T11:21:21.163348+0000","last_undegraded":"2023-10-16T11:21:21.163348+0000","last_fullsized":"2023-10-16T11:21:21.163348+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:56:56.995189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083968+0000","last_change":"2023-10-16T11:21:21.146548+0000","last_active":"2023-10-16T11:21:23.083968+0000","last_peered":"2023-10-16T11:21:23.083968+0000","last_clean":"2023-10-16T11:21:23.083968+0000","last_became_active":"2023-10-16T11:21:21.146286+0000","last_became_peered":"2023-10-16T11:21:21.146286+0000","last_unstale":"2023-10-16T11:21:23.083968+0000","last_undegraded":"2023-10-16T11:21:23.083968+0000","last_fullsized":"2023-10-16T11:21:23.083968+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:38:18.751110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1a","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.164607+0000","last_change":"2023-10-16T11:21:20.135318+0000","last_active":"2023-10-16T11:21:21.164607+0000","last_peered":"2023-10-16T11:21:21.164607+0000","last_clean":"2023-10-16T11:21:21.164607+0000","last_became_active":"2023-10-16T11:21:20.135057+0000","last_became_peered":"2023-10-16T11:21:20.135057+0000","last_unstale":"2023-10-16T11:21:21.164607+0000","last_undegraded":"2023-10-16T11:21:21.164607+0000","last_fullsized":"2023-10-16T11:21:21.164607+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:24:55.031053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392957+0000","last_change":"2023-10-16T11:21:21.159119+0000","last_active":"2023-10-16T11:21:23.392957+0000","last_peered":"2023-10-16T11:21:23.392957+0000","last_clean":"2023-10-16T11:21:23.392957+0000","last_became_active":"2023-10-16T11:21:21.158849+0000","last_became_peered":"2023-10-16T11:21:21.158849+0000","last_unstale":"2023-10-16T11:21:23.392957+0000","last_undegraded":"2023-10-16T11:21:23.392957+0000","last_fullsized":"2023-10-16T11:21:23.392957+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:05:31.538501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137343+0000","last_change":"2023-10-16T11:21:20.136910+0000","last_active":"2023-10-16T11:21:21.137343+0000","last_peered":"2023-10-16T11:21:21.137343+0000","last_clean":"2023-10-16T11:21:21.137343+0000","last_became_active":"2023-10-16T11:21:20.136132+0000","last_became_peered":"2023-10-16T11:21:20.136132+0000","last_unstale":"2023-10-16T11:21:21.137343+0000","last_undegraded":"2023-10-16T11:21:21.137343+0000","last_fullsized":"2023-10-16T11:21:21.137343+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:49.341656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149422+0000","last_change":"2023-10-16T11:21:21.156347+0000","last_active":"2023-10-16T11:21:22.149422+0000","last_peered":"2023-10-16T11:21:22.149422+0000","last_clean":"2023-10-16T11:21:22.149422+0000","last_became_active":"2023-10-16T11:21:21.156062+0000","last_became_peered":"2023-10-16T11:21:21.156062+0000","last_unstale":"2023-10-16T11:21:22.149422+0000","last_undegraded":"2023-10-16T11:21:22.149422+0000","last_fullsized":"2023-10-16T11:21:22.149422+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:35:51.724826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137545+0000","last_change":"2023-10-16T11:21:20.136880+0000","last_active":"2023-10-16T11:21:21.137545+0000","last_peered":"2023-10-16T11:21:21.137545+0000","last_clean":"2023-10-16T11:21:21.137545+0000","last_became_active":"2023-10-16T11:21:20.136549+0000","last_became_peered":"2023-10-16T11:21:20.136549+0000","last_unstale":"2023-10-16T11:21:21.137545+0000","last_undegraded":"2023-10-16T11:21:21.137545+0000","last_fullsized":"2023-10-16T11:21:21.137545+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:33.178321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149724+0000","last_change":"2023-10-16T11:21:21.158168+0000","last_active":"2023-10-16T11:21:22.149724+0000","last_peered":"2023-10-16T11:21:22.149724+0000","last_clean":"2023-10-16T11:21:22.149724+0000","last_became_active":"2023-10-16T11:21:21.157925+0000","last_became_peered":"2023-10-16T11:21:21.157925+0000","last_unstale":"2023-10-16T11:21:22.149724+0000","last_undegraded":"2023-10-16T11:21:22.149724+0000","last_fullsized":"2023-10-16T11:21:22.149724+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:39:10.038543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.136497+0000","last_change":"2023-10-16T11:21:20.136649+0000","last_active":"2023-10-16T11:21:21.136497+0000","last_peered":"2023-10-16T11:21:21.136497+0000","last_clean":"2023-10-16T11:21:21.136497+0000","last_became_active":"2023-10-16T11:21:20.135908+0000","last_became_peered":"2023-10-16T11:21:20.135908+0000","last_unstale":"2023-10-16T11:21:21.136497+0000","last_undegraded":"2023-10-16T11:21:21.136497+0000","last_fullsized":"2023-10-16T11:21:21.136497+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:24:43.263648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393840+0000","last_change":"2023-10-16T11:21:21.152790+0000","last_active":"2023-10-16T11:21:23.393840+0000","last_peered":"2023-10-16T11:21:23.393840+0000","last_clean":"2023-10-16T11:21:23.393840+0000","last_became_active":"2023-10-16T11:21:21.152524+0000","last_became_peered":"2023-10-16T11:21:21.152524+0000","last_unstale":"2023-10-16T11:21:23.393840+0000","last_undegraded":"2023-10-16T11:21:23.393840+0000","last_fullsized":"2023-10-16T11:21:23.393840+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:44.086547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.142862+0000","last_change":"2023-10-16T11:21:20.134788+0000","last_active":"2023-10-16T11:21:21.142862+0000","last_peered":"2023-10-16T11:21:21.142862+0000","last_clean":"2023-10-16T11:21:21.142862+0000","last_became_active":"2023-10-16T11:21:20.134497+0000","last_became_peered":"2023-10-16T11:21:20.134497+0000","last_unstale":"2023-10-16T11:21:21.142862+0000","last_undegraded":"2023-10-16T11:21:21.142862+0000","last_fullsized":"2023-10-16T11:21:21.142862+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:04:42.293378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148749+0000","last_change":"2023-10-16T11:21:21.159556+0000","last_active":"2023-10-16T11:21:22.148749+0000","last_peered":"2023-10-16T11:21:22.148749+0000","last_clean":"2023-10-16T11:21:22.148749+0000","last_became_active":"2023-10-16T11:21:21.159290+0000","last_became_peered":"2023-10-16T11:21:21.159290+0000","last_unstale":"2023-10-16T11:21:22.148749+0000","last_undegraded":"2023-10-16T11:21:22.148749+0000","last_fullsized":"2023-10-16T11:21:22.148749+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:54:40.915612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.15","version":"255'3","reported_seq":15,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.160075+0000","last_change":"2023-10-16T11:21:20.131549+0000","last_active":"2023-10-16T11:21:21.160075+0000","last_peered":"2023-10-16T11:21:21.160075+0000","last_clean":"2023-10-16T11:21:21.160075+0000","last_became_active":"2023-10-16T11:21:20.131029+0000","last_became_peered":"2023-10-16T11:21:20.131029+0000","last_unstale":"2023-10-16T11:21:21.160075+0000","last_undegraded":"2023-10-16T11:21:21.160075+0000","last_fullsized":"2023-10-16T11:21:21.160075+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:44:54.508762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393724+0000","last_change":"2023-10-16T11:21:21.139139+0000","last_active":"2023-10-16T11:21:23.393724+0000","last_peered":"2023-10-16T11:21:23.393724+0000","last_clean":"2023-10-16T11:21:23.393724+0000","last_became_active":"2023-10-16T11:21:21.138819+0000","last_became_peered":"2023-10-16T11:21:21.138819+0000","last_unstale":"2023-10-16T11:21:23.393724+0000","last_undegraded":"2023-10-16T11:21:23.393724+0000","last_fullsized":"2023-10-16T11:21:23.393724+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:50.021261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.14","version":"255'6","reported_seq":18,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.165605+0000","last_change":"2023-10-16T11:21:20.127077+0000","last_active":"2023-10-16T11:21:21.165605+0000","last_peered":"2023-10-16T11:21:21.165605+0000","last_clean":"2023-10-16T11:21:21.165605+0000","last_became_active":"2023-10-16T11:21:20.126774+0000","last_became_peered":"2023-10-16T11:21:20.126774+0000","last_unstale":"2023-10-16T11:21:21.165605+0000","last_undegraded":"2023-10-16T11:21:21.165605+0000","last_fullsized":"2023-10-16T11:21:21.165605+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:28:18.733921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.083980+0000","last_change":"2023-10-16T11:21:21.143022+0000","last_active":"2023-10-16T11:21:23.083980+0000","last_peered":"2023-10-16T11:21:23.083980+0000","last_clean":"2023-10-16T11:21:23.083980+0000","last_became_active":"2023-10-16T11:21:21.142711+0000","last_became_peered":"2023-10-16T11:21:21.142711+0000","last_unstale":"2023-10-16T11:21:23.083980+0000","last_undegraded":"2023-10-16T11:21:23.083980+0000","last_fullsized":"2023-10-16T11:21:23.083980+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:14.502383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.13","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.156589+0000","last_change":"2023-10-16T11:21:20.137401+0000","last_active":"2023-10-16T11:21:21.156589+0000","last_peered":"2023-10-16T11:21:21.156589+0000","last_clean":"2023-10-16T11:21:21.156589+0000","last_became_active":"2023-10-16T11:21:20.137108+0000","last_became_peered":"2023-10-16T11:21:20.137108+0000","last_unstale":"2023-10-16T11:21:21.156589+0000","last_undegraded":"2023-10-16T11:21:21.156589+0000","last_fullsized":"2023-10-16T11:21:21.156589+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:14.250170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149584+0000","last_change":"2023-10-16T11:21:21.155157+0000","last_active":"2023-10-16T11:21:22.149584+0000","last_peered":"2023-10-16T11:21:22.149584+0000","last_clean":"2023-10-16T11:21:22.149584+0000","last_became_active":"2023-10-16T11:21:21.154777+0000","last_became_peered":"2023-10-16T11:21:21.154777+0000","last_unstale":"2023-10-16T11:21:22.149584+0000","last_undegraded":"2023-10-16T11:21:22.149584+0000","last_fullsized":"2023-10-16T11:21:22.149584+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:18.294874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.141510+0000","last_change":"2023-10-16T11:21:20.135577+0000","last_active":"2023-10-16T11:21:21.141510+0000","last_peered":"2023-10-16T11:21:21.141510+0000","last_clean":"2023-10-16T11:21:21.141510+0000","last_became_active":"2023-10-16T11:21:20.135290+0000","last_became_peered":"2023-10-16T11:21:20.135290+0000","last_unstale":"2023-10-16T11:21:21.141510+0000","last_undegraded":"2023-10-16T11:21:21.141510+0000","last_fullsized":"2023-10-16T11:21:21.141510+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:26:39.024418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148159+0000","last_change":"2023-10-16T11:21:21.156830+0000","last_active":"2023-10-16T11:21:22.148159+0000","last_peered":"2023-10-16T11:21:22.148159+0000","last_clean":"2023-10-16T11:21:22.148159+0000","last_became_active":"2023-10-16T11:21:21.156545+0000","last_became_peered":"2023-10-16T11:21:21.156545+0000","last_unstale":"2023-10-16T11:21:22.148159+0000","last_undegraded":"2023-10-16T11:21:22.148159+0000","last_fullsized":"2023-10-16T11:21:22.148159+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:19:47.023237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.133566+0000","last_change":"2023-10-16T11:21:20.137041+0000","last_active":"2023-10-16T11:21:21.133566+0000","last_peered":"2023-10-16T11:21:21.133566+0000","last_clean":"2023-10-16T11:21:21.133566+0000","last_became_active":"2023-10-16T11:21:20.136734+0000","last_became_peered":"2023-10-16T11:21:20.136734+0000","last_unstale":"2023-10-16T11:21:21.133566+0000","last_undegraded":"2023-10-16T11:21:21.133566+0000","last_fullsized":"2023-10-16T11:21:21.133566+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:33.259877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084252+0000","last_change":"2023-10-16T11:21:21.139823+0000","last_active":"2023-10-16T11:21:23.084252+0000","last_peered":"2023-10-16T11:21:23.084252+0000","last_clean":"2023-10-16T11:21:23.084252+0000","last_became_active":"2023-10-16T11:21:21.139509+0000","last_became_peered":"2023-10-16T11:21:21.139509+0000","last_unstale":"2023-10-16T11:21:23.084252+0000","last_undegraded":"2023-10-16T11:21:23.084252+0000","last_fullsized":"2023-10-16T11:21:23.084252+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:59:46.141117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.10","version":"255'8","reported_seq":20,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.161026+0000","last_change":"2023-10-16T11:21:20.131005+0000","last_active":"2023-10-16T11:21:21.161026+0000","last_peered":"2023-10-16T11:21:21.161026+0000","last_clean":"2023-10-16T11:21:21.161026+0000","last_became_active":"2023-10-16T11:21:20.130698+0000","last_became_peered":"2023-10-16T11:21:20.130698+0000","last_unstale":"2023-10-16T11:21:21.161026+0000","last_undegraded":"2023-10-16T11:21:21.161026+0000","last_fullsized":"2023-10-16T11:21:21.161026+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:14:44.667056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393039+0000","last_change":"2023-10-16T11:21:21.144905+0000","last_active":"2023-10-16T11:21:23.393039+0000","last_peered":"2023-10-16T11:21:23.393039+0000","last_clean":"2023-10-16T11:21:23.393039+0000","last_became_active":"2023-10-16T11:21:21.144575+0000","last_became_peered":"2023-10-16T11:21:21.144575+0000","last_unstale":"2023-10-16T11:21:23.393039+0000","last_undegraded":"2023-10-16T11:21:23.393039+0000","last_fullsized":"2023-10-16T11:21:23.393039+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:03.266767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.158868+0000","last_change":"2023-10-16T11:21:20.137849+0000","last_active":"2023-10-16T11:21:21.158868+0000","last_peered":"2023-10-16T11:21:21.158868+0000","last_clean":"2023-10-16T11:21:21.158868+0000","last_became_active":"2023-10-16T11:21:20.137585+0000","last_became_peered":"2023-10-16T11:21:20.137585+0000","last_unstale":"2023-10-16T11:21:21.158868+0000","last_undegraded":"2023-10-16T11:21:21.158868+0000","last_fullsized":"2023-10-16T11:21:21.158868+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:56:03.818822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.150178+0000","last_change":"2023-10-16T11:21:21.156832+0000","last_active":"2023-10-16T11:21:22.150178+0000","last_peered":"2023-10-16T11:21:22.150178+0000","last_clean":"2023-10-16T11:21:22.150178+0000","last_became_active":"2023-10-16T11:21:21.156578+0000","last_became_peered":"2023-10-16T11:21:21.156578+0000","last_unstale":"2023-10-16T11:21:22.150178+0000","last_undegraded":"2023-10-16T11:21:22.150178+0000","last_fullsized":"2023-10-16T11:21:22.150178+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:02.469221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.e","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.158981+0000","last_change":"2023-10-16T11:21:20.135672+0000","last_active":"2023-10-16T11:21:21.158981+0000","last_peered":"2023-10-16T11:21:21.158981+0000","last_clean":"2023-10-16T11:21:21.158981+0000","last_became_active":"2023-10-16T11:21:20.135413+0000","last_became_peered":"2023-10-16T11:21:20.135413+0000","last_unstale":"2023-10-16T11:21:21.158981+0000","last_undegraded":"2023-10-16T11:21:21.158981+0000","last_fullsized":"2023-10-16T11:21:21.158981+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:03.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149780+0000","last_change":"2023-10-16T11:21:21.163368+0000","last_active":"2023-10-16T11:21:22.149780+0000","last_peered":"2023-10-16T11:21:22.149780+0000","last_clean":"2023-10-16T11:21:22.149780+0000","last_became_active":"2023-10-16T11:21:21.163079+0000","last_became_peered":"2023-10-16T11:21:21.163079+0000","last_unstale":"2023-10-16T11:21:22.149780+0000","last_undegraded":"2023-10-16T11:21:22.149780+0000","last_fullsized":"2023-10-16T11:21:22.149780+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:07:50.345956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.d","version":"255'1","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.163211+0000","last_change":"2023-10-16T11:21:20.139111+0000","last_active":"2023-10-16T11:21:21.163211+0000","last_peered":"2023-10-16T11:21:21.163211+0000","last_clean":"2023-10-16T11:21:21.163211+0000","last_became_active":"2023-10-16T11:21:20.138713+0000","last_became_peered":"2023-10-16T11:21:20.138713+0000","last_unstale":"2023-10-16T11:21:21.163211+0000","last_undegraded":"2023-10-16T11:21:21.163211+0000","last_fullsized":"2023-10-16T11:21:21.163211+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:16:27.847025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149823+0000","last_change":"2023-10-16T11:21:21.146872+0000","last_active":"2023-10-16T11:21:22.149823+0000","last_peered":"2023-10-16T11:21:22.149823+0000","last_clean":"2023-10-16T11:21:22.149823+0000","last_became_active":"2023-10-16T11:21:21.146581+0000","last_became_peered":"2023-10-16T11:21:21.146581+0000","last_unstale":"2023-10-16T11:21:22.149823+0000","last_undegraded":"2023-10-16T11:21:22.149823+0000","last_fullsized":"2023-10-16T11:21:22.149823+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:13:26.045032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.134555+0000","last_change":"2023-10-16T11:21:20.134928+0000","last_active":"2023-10-16T11:21:21.134555+0000","last_peered":"2023-10-16T11:21:21.134555+0000","last_clean":"2023-10-16T11:21:21.134555+0000","last_became_active":"2023-10-16T11:21:20.134219+0000","last_became_peered":"2023-10-16T11:21:20.134219+0000","last_unstale":"2023-10-16T11:21:21.134555+0000","last_undegraded":"2023-10-16T11:21:21.134555+0000","last_fullsized":"2023-10-16T11:21:21.134555+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:01:09.781648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.392977+0000","last_change":"2023-10-16T11:21:21.158073+0000","last_active":"2023-10-16T11:21:23.392977+0000","last_peered":"2023-10-16T11:21:23.392977+0000","last_clean":"2023-10-16T11:21:23.392977+0000","last_became_active":"2023-10-16T11:21:21.157732+0000","last_became_peered":"2023-10-16T11:21:21.157732+0000","last_unstale":"2023-10-16T11:21:23.392977+0000","last_undegraded":"2023-10-16T11:21:23.392977+0000","last_fullsized":"2023-10-16T11:21:23.392977+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:55:44.633526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.b","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.157618+0000","last_change":"2023-10-16T11:21:20.134884+0000","last_active":"2023-10-16T11:21:21.157618+0000","last_peered":"2023-10-16T11:21:21.157618+0000","last_clean":"2023-10-16T11:21:21.157618+0000","last_became_active":"2023-10-16T11:21:20.134581+0000","last_became_peered":"2023-10-16T11:21:20.134581+0000","last_unstale":"2023-10-16T11:21:21.157618+0000","last_undegraded":"2023-10-16T11:21:21.157618+0000","last_fullsized":"2023-10-16T11:21:21.157618+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:39:24.151565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149786+0000","last_change":"2023-10-16T11:21:21.154639+0000","last_active":"2023-10-16T11:21:22.149786+0000","last_peered":"2023-10-16T11:21:22.149786+0000","last_clean":"2023-10-16T11:21:22.149786+0000","last_became_active":"2023-10-16T11:21:21.154244+0000","last_became_peered":"2023-10-16T11:21:21.154244+0000","last_unstale":"2023-10-16T11:21:22.149786+0000","last_undegraded":"2023-10-16T11:21:22.149786+0000","last_fullsized":"2023-10-16T11:21:22.149786+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:09:42.887120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137141+0000","last_change":"2023-10-16T11:21:20.127897+0000","last_active":"2023-10-16T11:21:21.137141+0000","last_peered":"2023-10-16T11:21:21.137141+0000","last_clean":"2023-10-16T11:21:21.137141+0000","last_became_active":"2023-10-16T11:21:20.127589+0000","last_became_peered":"2023-10-16T11:21:20.127589+0000","last_unstale":"2023-10-16T11:21:21.137141+0000","last_undegraded":"2023-10-16T11:21:21.137141+0000","last_fullsized":"2023-10-16T11:21:21.137141+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:33:21.578413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149350+0000","last_change":"2023-10-16T11:21:21.155072+0000","last_active":"2023-10-16T11:21:22.149350+0000","last_peered":"2023-10-16T11:21:22.149350+0000","last_clean":"2023-10-16T11:21:22.149350+0000","last_became_active":"2023-10-16T11:21:21.154630+0000","last_became_peered":"2023-10-16T11:21:21.154630+0000","last_unstale":"2023-10-16T11:21:22.149350+0000","last_undegraded":"2023-10-16T11:21:22.149350+0000","last_fullsized":"2023-10-16T11:21:22.149350+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:31.134795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.133715+0000","last_change":"2023-10-16T11:21:20.137514+0000","last_active":"2023-10-16T11:21:21.133715+0000","last_peered":"2023-10-16T11:21:21.133715+0000","last_clean":"2023-10-16T11:21:21.133715+0000","last_became_active":"2023-10-16T11:21:20.137300+0000","last_became_peered":"2023-10-16T11:21:20.137300+0000","last_unstale":"2023-10-16T11:21:21.133715+0000","last_undegraded":"2023-10-16T11:21:21.133715+0000","last_fullsized":"2023-10-16T11:21:21.133715+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:08:38.226471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393409+0000","last_change":"2023-10-16T11:21:21.155156+0000","last_active":"2023-10-16T11:21:23.393409+0000","last_peered":"2023-10-16T11:21:23.393409+0000","last_clean":"2023-10-16T11:21:23.393409+0000","last_became_active":"2023-10-16T11:21:21.154816+0000","last_became_peered":"2023-10-16T11:21:21.154816+0000","last_unstale":"2023-10-16T11:21:23.393409+0000","last_undegraded":"2023-10-16T11:21:23.393409+0000","last_fullsized":"2023-10-16T11:21:23.393409+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:54.214911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135698+0000","last_change":"2023-10-16T11:21:20.139486+0000","last_active":"2023-10-16T11:21:21.135698+0000","last_peered":"2023-10-16T11:21:21.135698+0000","last_clean":"2023-10-16T11:21:21.135698+0000","last_became_active":"2023-10-16T11:21:20.138918+0000","last_became_peered":"2023-10-16T11:21:20.138918+0000","last_unstale":"2023-10-16T11:21:21.135698+0000","last_undegraded":"2023-10-16T11:21:21.135698+0000","last_fullsized":"2023-10-16T11:21:21.135698+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:50:17.290575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084241+0000","last_change":"2023-10-16T11:21:21.146671+0000","last_active":"2023-10-16T11:21:23.084241+0000","last_peered":"2023-10-16T11:21:23.084241+0000","last_clean":"2023-10-16T11:21:23.084241+0000","last_became_active":"2023-10-16T11:21:21.146386+0000","last_became_peered":"2023-10-16T11:21:21.146386+0000","last_unstale":"2023-10-16T11:21:23.084241+0000","last_undegraded":"2023-10-16T11:21:23.084241+0000","last_fullsized":"2023-10-16T11:21:23.084241+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:13.153234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.7","version":"255'2","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.161508+0000","last_change":"2023-10-16T11:21:20.135967+0000","last_active":"2023-10-16T11:21:21.161508+0000","last_peered":"2023-10-16T11:21:21.161508+0000","last_clean":"2023-10-16T11:21:21.161508+0000","last_became_active":"2023-10-16T11:21:20.135622+0000","last_became_peered":"2023-10-16T11:21:20.135622+0000","last_unstale":"2023-10-16T11:21:21.161508+0000","last_undegraded":"2023-10-16T11:21:21.161508+0000","last_fullsized":"2023-10-16T11:21:21.161508+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:23.358592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149789+0000","last_change":"2023-10-16T11:21:21.161249+0000","last_active":"2023-10-16T11:21:22.149789+0000","last_peered":"2023-10-16T11:21:22.149789+0000","last_clean":"2023-10-16T11:21:22.149789+0000","last_became_active":"2023-10-16T11:21:21.158669+0000","last_became_peered":"2023-10-16T11:21:21.158669+0000","last_unstale":"2023-10-16T11:21:22.149789+0000","last_undegraded":"2023-10-16T11:21:22.149789+0000","last_fullsized":"2023-10-16T11:21:22.149789+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:11.911562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.152367+0000","last_change":"2023-10-16T11:21:20.131488+0000","last_active":"2023-10-16T11:21:21.152367+0000","last_peered":"2023-10-16T11:21:21.152367+0000","last_clean":"2023-10-16T11:21:21.152367+0000","last_became_active":"2023-10-16T11:21:20.131214+0000","last_became_peered":"2023-10-16T11:21:20.131214+0000","last_unstale":"2023-10-16T11:21:21.152367+0000","last_undegraded":"2023-10-16T11:21:21.152367+0000","last_fullsized":"2023-10-16T11:21:21.152367+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:23:21.659484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.084243+0000","last_change":"2023-10-16T11:21:21.147831+0000","last_active":"2023-10-16T11:21:23.084243+0000","last_peered":"2023-10-16T11:21:23.084243+0000","last_clean":"2023-10-16T11:21:23.084243+0000","last_became_active":"2023-10-16T11:21:21.147568+0000","last_became_peered":"2023-10-16T11:21:21.147568+0000","last_unstale":"2023-10-16T11:21:23.084243+0000","last_undegraded":"2023-10-16T11:21:23.084243+0000","last_fullsized":"2023-10-16T11:21:23.084243+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:34:07.295356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.142597+0000","last_change":"2023-10-16T11:21:20.136215+0000","last_active":"2023-10-16T11:21:21.142597+0000","last_peered":"2023-10-16T11:21:21.142597+0000","last_clean":"2023-10-16T11:21:21.142597+0000","last_became_active":"2023-10-16T11:21:20.135897+0000","last_became_peered":"2023-10-16T11:21:20.135897+0000","last_unstale":"2023-10-16T11:21:21.142597+0000","last_undegraded":"2023-10-16T11:21:21.142597+0000","last_fullsized":"2023-10-16T11:21:21.142597+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:23.641201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.150123+0000","last_change":"2023-10-16T11:21:21.145015+0000","last_active":"2023-10-16T11:21:22.150123+0000","last_peered":"2023-10-16T11:21:22.150123+0000","last_clean":"2023-10-16T11:21:22.150123+0000","last_became_active":"2023-10-16T11:21:21.144545+0000","last_became_peered":"2023-10-16T11:21:21.144545+0000","last_unstale":"2023-10-16T11:21:22.150123+0000","last_undegraded":"2023-10-16T11:21:22.150123+0000","last_fullsized":"2023-10-16T11:21:22.150123+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:07:51.353240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.144404+0000","last_change":"2023-10-16T11:21:20.127796+0000","last_active":"2023-10-16T11:21:21.144404+0000","last_peered":"2023-10-16T11:21:21.144404+0000","last_clean":"2023-10-16T11:21:21.144404+0000","last_became_active":"2023-10-16T11:21:20.127496+0000","last_became_peered":"2023-10-16T11:21:20.127496+0000","last_unstale":"2023-10-16T11:21:21.144404+0000","last_undegraded":"2023-10-16T11:21:21.144404+0000","last_fullsized":"2023-10-16T11:21:21.144404+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:07.392329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393134+0000","last_change":"2023-10-16T11:21:21.157926+0000","last_active":"2023-10-16T11:21:23.393134+0000","last_peered":"2023-10-16T11:21:23.393134+0000","last_clean":"2023-10-16T11:21:23.393134+0000","last_became_active":"2023-10-16T11:21:21.157376+0000","last_became_peered":"2023-10-16T11:21:21.157376+0000","last_unstale":"2023-10-16T11:21:23.393134+0000","last_undegraded":"2023-10-16T11:21:23.393134+0000","last_fullsized":"2023-10-16T11:21:23.393134+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:10.128277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135642+0000","last_change":"2023-10-16T11:21:20.140881+0000","last_active":"2023-10-16T11:21:21.135642+0000","last_peered":"2023-10-16T11:21:21.135642+0000","last_clean":"2023-10-16T11:21:21.135642+0000","last_became_active":"2023-10-16T11:21:20.140500+0000","last_became_peered":"2023-10-16T11:21:20.140500+0000","last_unstale":"2023-10-16T11:21:21.135642+0000","last_undegraded":"2023-10-16T11:21:21.135642+0000","last_fullsized":"2023-10-16T11:21:21.135642+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:01.038508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149816+0000","last_change":"2023-10-16T11:21:21.161571+0000","last_active":"2023-10-16T11:21:22.149816+0000","last_peered":"2023-10-16T11:21:22.149816+0000","last_clean":"2023-10-16T11:21:22.149816+0000","last_became_active":"2023-10-16T11:21:21.158952+0000","last_became_peered":"2023-10-16T11:21:21.158952+0000","last_unstale":"2023-10-16T11:21:22.149816+0000","last_undegraded":"2023-10-16T11:21:22.149816+0000","last_fullsized":"2023-10-16T11:21:22.149816+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:51:34.182621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135578+0000","last_change":"2023-10-16T11:21:20.139168+0000","last_active":"2023-10-16T11:21:21.135578+0000","last_peered":"2023-10-16T11:21:21.135578+0000","last_clean":"2023-10-16T11:21:21.135578+0000","last_became_active":"2023-10-16T11:21:20.138825+0000","last_became_peered":"2023-10-16T11:21:20.138825+0000","last_unstale":"2023-10-16T11:21:21.135578+0000","last_undegraded":"2023-10-16T11:21:21.135578+0000","last_fullsized":"2023-10-16T11:21:21.135578+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:18:30.130495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.149496+0000","last_change":"2023-10-16T11:21:21.154880+0000","last_active":"2023-10-16T11:21:22.149496+0000","last_peered":"2023-10-16T11:21:22.149496+0000","last_clean":"2023-10-16T11:21:22.149496+0000","last_became_active":"2023-10-16T11:21:21.154247+0000","last_became_peered":"2023-10-16T11:21:21.154247+0000","last_unstale":"2023-10-16T11:21:22.149496+0000","last_undegraded":"2023-10-16T11:21:22.149496+0000","last_fullsized":"2023-10-16T11:21:22.149496+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:19:11.835144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.137434+0000","last_change":"2023-10-16T11:21:20.139225+0000","last_active":"2023-10-16T11:21:21.137434+0000","last_peered":"2023-10-16T11:21:21.137434+0000","last_clean":"2023-10-16T11:21:21.137434+0000","last_became_active":"2023-10-16T11:21:20.138671+0000","last_became_peered":"2023-10-16T11:21:20.138671+0000","last_unstale":"2023-10-16T11:21:21.137434+0000","last_undegraded":"2023-10-16T11:21:21.137434+0000","last_fullsized":"2023-10-16T11:21:21.137434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:08:10.907262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:23.393297+0000","last_change":"2023-10-16T11:21:21.150780+0000","last_active":"2023-10-16T11:21:23.393297+0000","last_peered":"2023-10-16T11:21:23.393297+0000","last_clean":"2023-10-16T11:21:23.393297+0000","last_became_active":"2023-10-16T11:21:21.150505+0000","last_became_peered":"2023-10-16T11:21:21.150505+0000","last_unstale":"2023-10-16T11:21:23.393297+0000","last_undegraded":"2023-10-16T11:21:23.393297+0000","last_fullsized":"2023-10-16T11:21:23.393297+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:17:45.993140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.135545+0000","last_change":"2023-10-16T11:21:20.129727+0000","last_active":"2023-10-16T11:21:21.135545+0000","last_peered":"2023-10-16T11:21:21.135545+0000","last_clean":"2023-10-16T11:21:21.135545+0000","last_became_active":"2023-10-16T11:21:20.129419+0000","last_became_peered":"2023-10-16T11:21:20.129419+0000","last_unstale":"2023-10-16T11:21:21.135545+0000","last_undegraded":"2023-10-16T11:21:21.135545+0000","last_fullsized":"2023-10-16T11:21:21.135545+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:19.109089+0000","last_clean_scrub_stamp":"2023-10-16T11:21:19.109089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:00.056698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.22","version":"0'0","reported_seq":11,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-10-16T11:21:22.148490+0000","last_change":"2023-10-16T11:21:21.162583+0000","last_active":"2023-10-16T11:21:22.148490+0000","last_peered":"2023-10-16T11:21:22.148490+0000","last_clean":"2023-10-16T11:21:22.148490+0000","last_became_active":"2023-10-16T11:21:21.162198+0000","last_became_peered":"2023-10-16T11:21:21.162198+0000","last_unstale":"2023-10-16T11:21:22.148490+0000","last_undegraded":"2023-10-16T11:21:22.148490+0000","last_fullsized":"2023-10-16T11:21:22.148490+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:20.113389+0000","last_clean_scrub_stamp":"2023-10-16T11:21:20.113389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:59.212211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"242'381","reported_seq":1367,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-10-16T11:21:21.142151+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:21:21.142151+0000","last_peered":"2023-10-16T11:21:21.142151+0000","last_clean":"2023-10-16T11:21:21.142151+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:21:21.142151+0000","last_undegraded":"2023-10-16T11:21:21.142151+0000","last_fullsized":"2023-10-16T11:21:21.142151+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":35,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":34,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543300,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52964,"kb_used_data":3736,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318876,"statfs":{"total":96636764160,"available":96582529024,"internally_reserved":0,"allocated":3825664,"data_stored":2269748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.878,"15min":0.878},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27800000000000002},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64900000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.503}]},{"osd":2,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55100000000000005,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.31900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69799999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.628}]},{"osd":3,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":1,"15min":1.002},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68400000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60399999999999998,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84299999999999997,"5min":1.052,"15min":1.052},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68300000000000005}]}]},{"osd":6,"up_from":14,"seq":60129543299,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60712,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311128,"statfs":{"total":96636764160,"available":96574595072,"internally_reserved":0,"allocated":3235840,"data_stored":1679470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54500000000000004,"15min":0.58199999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.84499999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.78100000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.63400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57699999999999996,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.68500000000000005,"5min":0.77200000000000002,"15min":0.86499999999999999},"last":0.879}]},{"osd":2,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.57299999999999995,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.66300000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78800000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.89300000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28100000000000003},"max":{"1min":0.747,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.496,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.73799999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59799999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.81399999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81299999999999994}]}]},{"osd":1,"up_from":14,"seq":60129543299,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60708,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311132,"statfs":{"total":96636764160,"available":96574599168,"internally_reserved":0,"allocated":3235840,"data_stored":1679414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.38300000000000001,"15min":0.40100000000000002},"min":{"1min":0.23400000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.52600000000000002,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.39100000000000001,"15min":0.42599999999999999},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":0.62,"15min":0.91700000000000004},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.38400000000000001,"15min":0.40999999999999998},"min":{"1min":0.252,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.44900000000000001,"5min":0.56999999999999995,"15min":0.98899999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.42199999999999999,"15min":0.45700000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47599999999999998,"5min":0.73599999999999999,"15min":0.83099999999999996},"last":0.434}]},{"osd":3,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.443,"15min":0.48399999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.51300000000000001,"5min":0.69699999999999995,"15min":0.78300000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.435,"15min":0.46500000000000002},"min":{"1min":0.27900000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.43099999999999999,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81799999999999995,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.66200000000000003,"15min":0.68000000000000005},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.73799999999999999,"5min":1.0289999999999999,"15min":1.097},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.66900000000000004},"min":{"1min":0.41899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72499999999999998,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64800000000000002,"15min":0.68500000000000005},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76900000000000002,"5min":0.88200000000000001,"15min":1.147},"last":0.505}]},{"osd":6,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80600000000000005,"5min":0.98699999999999999,"15min":1.0509999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.436,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.45300000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82899999999999996,"5min":1.004,"15min":1.0600000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.67200000000000004,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76800000000000002,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.55100000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543300,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61300,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310540,"statfs":{"total":96636764160,"available":96573992960,"internally_reserved":0,"allocated":3837952,"data_stored":2270386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.22,"15min":0.214},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.39200000000000002,"15min":0.41099999999999998},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.65100000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40799999999999997,"15min":0.42099999999999999},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.70299999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.42399999999999999,"15min":0.44400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.72199999999999998,"15min":0.86799999999999999},"last":0.377}]},{"osd":3,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.42499999999999999,"15min":0.44800000000000001},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.47399999999999998,"15min":0.49399999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.84599999999999997},"last":0.39300000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66000000000000003,"15min":0.66700000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84399999999999997,"5min":0.91900000000000004,"15min":1.008},"last":0.443}]},{"osd":5,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.64300000000000002,"15min":0.65600000000000003},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79900000000000004,"5min":1.004,"15min":1.008},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81999999999999995,"5min":0.94199999999999995,"15min":1.1419999999999999},"last":0.42299999999999999}]},{"osd":6,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63900000000000001,"15min":0.66100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65100000000000002,"15min":0.68100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79000000000000004,"5min":0.97099999999999997,"15min":1.0449999999999999},"last":0.5}]},{"osd":7,"last update":"Mon Oct 16 11:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65600000000000003,"15min":0.68400000000000005},"min":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74099999999999999,"5min":0.98499999999999999,"15min":1.077},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80500000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.49199999999999999}]}]},{"osd":2,"up_from":14,"seq":60129543299,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48232,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323608,"statfs":{"total":96636764160,"available":96587374592,"internally_reserved":0,"allocated":3235840,"data_stored":1679470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.433,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.57299999999999995,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.436,"15min":0.441},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.441,"15min":0.45800000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.44800000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.163,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69899999999999995,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":3.593,"15min":3.593},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.64700000000000002,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73499999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67700000000000005,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67500000000000004,"15min":0.68999999999999995},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.73599999999999999,"5min":3.633,"15min":3.633},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.67200000000000004,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.70799999999999996,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66300000000000003,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.70699999999999996,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.67400000000000004}]}]},{"osd":3,"up_from":14,"seq":60129543300,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48236,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323604,"statfs":{"total":96636764160,"available":96587370496,"internally_reserved":0,"allocated":3244032,"data_stored":1680992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40200000000000002,"15min":0.41099999999999998},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.52600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.41799999999999998,"15min":0.44400000000000001},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.40899999999999997,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.38800000000000001,"15min":0.41499999999999998},"min":{"1min":0.24299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.42899999999999999,"5min":0.87,"15min":0.87},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40699999999999997,"15min":0.441},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.41199999999999998,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.42899999999999999,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.48099999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.432},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.44800000000000001,"15min":0.47499999999999998},"min":{"1min":0.33600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.503}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.73699999999999999,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71299999999999997,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.626,"15min":0.66200000000000003},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63300000000000001,"15min":0.66400000000000003},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.623,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.377,"15min":0.377},"max":{"1min":0.73499999999999999,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63700000000000001,"15min":0.67700000000000005},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.81799999999999995}]}]},{"osd":4,"up_from":14,"seq":60129543299,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96744,"kb_used_data":3164,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94275096,"statfs":{"total":96636764160,"available":96537698304,"internally_reserved":0,"allocated":3239936,"data_stored":1679472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.62,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":0.84399999999999997},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58699999999999997,"15min":0.61299999999999999},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68300000000000005,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.436},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.502}]},{"osd":2,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66000000000000003,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.68799999999999994,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.58599999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.65200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.64200000000000002},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.70099999999999996,"5min":0.81299999999999994,"15min":0.879},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.72399999999999998,"5min":0.81299999999999994,"15min":0.84099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.63800000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.29399999999999998},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.71399999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.875,"15min":0.875},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.72899999999999998}]}]},{"osd":5,"up_from":14,"seq":60129543300,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60724,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311116,"statfs":{"total":96636764160,"available":96574582784,"internally_reserved":0,"allocated":3252224,"data_stored":1681644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.504},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":1.873,"15min":1.873},"last":0.61099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.024,"15min":1.024},"last":0.55100000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88700000000000001,"5min":1.821,"15min":1.821},"last":0.437},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63100000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.65500000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.54300000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.878,"15min":0.878},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40799999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80000000000000004,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86099999999999999,"5min":2.02,"15min":2.02},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:21:25.652 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:21:25.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:21:25.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.811+0000 7f83872ab700 1 Processor -- start 2023-10-16T11:21:25.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.811+0000 7f83872ab700 1 -- start start 2023-10-16T11:21:25.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f83872ab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f838015da70 0x7f83800ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f83872ab700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83800ce6a0 0x7f838006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f83872ab700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8380072760 0x7f838006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f83872ab700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83800cef00 con 0x7f838015da70 2023-10-16T11:21:25.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f83872ab700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83800cec20 con 0x7f8380072760 2023-10-16T11:21:25.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f83872ab700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8380071c80 con 0x7f83800ce6a0 2023-10-16T11:21:25.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f8385047700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f838015da70 0x7f83800ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f8385047700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f838015da70 0x7f83800ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59908/0 (socket says 172.21.15.181:59908) 2023-10-16T11:21:25.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f8385047700 1 -- 172.21.15.181:0/1324290938 learned_addr learned my addr 172.21.15.181:0/1324290938 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:25.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.812+0000 7f8385848700 1 --2- 172.21.15.181:0/1324290938 >> v2:172.21.15.190:3300/0 conn(0x7f8380072760 0x7f838006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8384846700 1 --2- 172.21.15.181:0/1324290938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83800ce6a0 0x7f838006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8385047700 1 -- 172.21.15.181:0/1324290938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83800ce6a0 msgr2=0x7f838006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:25.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8385047700 1 --2- 172.21.15.181:0/1324290938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83800ce6a0 0x7f838006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8385047700 1 -- 172.21.15.181:0/1324290938 >> v2:172.21.15.190:3300/0 conn(0x7f8380072760 msgr2=0x7f838006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:25.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8385047700 1 --2- 172.21.15.181:0/1324290938 >> v2:172.21.15.190:3300/0 conn(0x7f8380072760 0x7f838006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8385047700 1 -- 172.21.15.181:0/1324290938 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8380071f60 con 0x7f838015da70 2023-10-16T11:21:25.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8385848700 1 --2- 172.21.15.181:0/1324290938 >> v2:172.21.15.190:3300/0 conn(0x7f8380072760 0x7f838006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:25.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8385047700 1 --2- 172.21.15.181:0/1324290938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838015da70 0x7f83800ce160 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f8380161530 tx=0x7f8370009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2915e046121753ad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8377fff700 1 -- 172.21.15.181:0/1324290938 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000b870 con 0x7f838015da70 2023-10-16T11:21:25.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8377fff700 1 -- 172.21.15.181:0/1324290938 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837000b9d0 con 0x7f838015da70 2023-10-16T11:21:25.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f8377fff700 1 -- 172.21.15.181:0/1324290938 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000bd60 con 0x7f838015da70 2023-10-16T11:21:25.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f83872ab700 1 -- 172.21.15.181:0/1324290938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838015da70 msgr2=0x7f83800ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.813+0000 7f83872ab700 1 --2- 172.21.15.181:0/1324290938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838015da70 0x7f83800ce160 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f8380161530 tx=0x7f8370009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.814+0000 7f83872ab700 1 -- 172.21.15.181:0/1324290938 shutdown_connections 2023-10-16T11:21:25.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.814+0000 7f83872ab700 1 --2- 172.21.15.181:0/1324290938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f838015da70 0x7f83800ce160 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.814+0000 7f83872ab700 1 --2- 172.21.15.181:0/1324290938 >> v2:172.21.15.190:3300/0 conn(0x7f8380072760 0x7f838006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.814+0000 7f83872ab700 1 --2- 172.21.15.181:0/1324290938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83800ce6a0 0x7f838006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.814+0000 7f83872ab700 1 -- 172.21.15.181:0/1324290938 >> 172.21.15.181:0/1324290938 conn(0x7f83800cd010 msgr2=0x7f83800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:25.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.814+0000 7f83872ab700 1 -- 172.21.15.181:0/1324290938 shutdown_connections 2023-10-16T11:21:25.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.814+0000 7f83872ab700 1 -- 172.21.15.181:0/1324290938 wait complete. 2023-10-16T11:21:25.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.815+0000 7f83872ab700 1 Processor -- start 2023-10-16T11:21:25.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.815+0000 7f83872ab700 1 -- start start 2023-10-16T11:21:25.824 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.815+0000 7f83872ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380072760 0x7f83800ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f83872ab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83800ce6a0 0x7f83800c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f83872ab700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f838015da70 0x7f83801597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f83872ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800b4260 con 0x7f8380072760 2023-10-16T11:21:25.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f83872ab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800b4520 con 0x7f83800ce6a0 2023-10-16T11:21:25.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f83872ab700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83800b4800 con 0x7f838015da70 2023-10-16T11:21:25.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380072760 0x7f83800ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380072760 0x7f83800ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59914/0 (socket says 172.21.15.181:59914) 2023-10-16T11:21:25.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 -- 172.21.15.181:0/186841541 learned_addr learned my addr 172.21.15.181:0/186841541 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:25.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 -- 172.21.15.181:0/186841541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f838015da70 msgr2=0x7f83801597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f838015da70 0x7f83801597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 -- 172.21.15.181:0/186841541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83800ce6a0 msgr2=0x7f83800c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83800ce6a0 0x7f83800c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 -- 172.21.15.181:0/186841541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8370009550 con 0x7f8380072760 2023-10-16T11:21:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8385047700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380072760 0x7f83800ac880 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f8370004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8375ffb700 1 -- 172.21.15.181:0/186841541 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370004540 con 0x7f8380072760 2023-10-16T11:21:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8375ffb700 1 -- 172.21.15.181:0/186841541 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83700036a0 con 0x7f8380072760 2023-10-16T11:21:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83800b4ae0 con 0x7f8380072760 2023-10-16T11:21:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f8375ffb700 1 -- 172.21.15.181:0/186841541 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370003a50 con 0x7f8380072760 2023-10-16T11:21:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.816+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83800c9e20 con 0x7f8380072760 2023-10-16T11:21:25.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.817+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83800b5000 con 0x7f8380072760 2023-10-16T11:21:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.818+0000 7f8375ffb700 1 -- 172.21.15.181:0/186841541 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8370021070 con 0x7f8380072760 2023-10-16T11:21:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.818+0000 7f8375ffb700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f836406b140 0x7f836406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.818+0000 7f8375ffb700 1 -- 172.21.15.181:0/186841541 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f837008b2f0 con 0x7f8380072760 2023-10-16T11:21:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.818+0000 7f8384846700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f836406b140 0x7f836406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:25.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.819+0000 7f8384846700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f836406b140 0x7f836406d600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f838006b270 tx=0x7f837c008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:25.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:25.823+0000 7f8375ffb700 1 -- 172.21.15.181:0/186841541 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f83800b5000 con 0x7f8380072760 2023-10-16T11:21:26.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.094+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f83800b5000 con 0x7f8380072760 2023-10-16T11:21:26.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.096+0000 7f8375ffb700 1 -- 172.21.15.181:0/186841541 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v54) v1 ==== 190+0+147 (secure 0 0 0) 0x7f83700150b0 con 0x7f8380072760 2023-10-16T11:21:26.099 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T11:21:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T11:21:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T11:21:26.099 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T11:21:26.100 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:21:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.098+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f836406b140 msgr2=0x7f836406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f836406b140 0x7f836406d600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f838006b270 tx=0x7f837c008040 comp rx=0 tx=0).stop 2023-10-16T11:21:26.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380072760 msgr2=0x7f83800ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380072760 0x7f83800ac880 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f8370004750 comp rx=0 tx=0).stop 2023-10-16T11:21:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 shutdown_connections 2023-10-16T11:21:26.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f838015da70 0x7f83801597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f836406b140 0x7f836406d600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8380072760 0x7f83800ac880 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 --2- 172.21.15.181:0/186841541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83800ce6a0 0x7f83800c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 >> 172.21.15.181:0/186841541 conn(0x7f83800cd010 msgr2=0x7f83800cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:26.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 shutdown_connections 2023-10-16T11:21:26.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.099+0000 7f83872ab700 1 -- 172.21.15.181:0/186841541 wait complete. 2023-10-16T11:21:26.105 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T11:21:26.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:26.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.233+0000 7f1ebff5c700 1 Processor -- start 2023-10-16T11:21:26.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.233+0000 7f1ebff5c700 1 -- start start 2023-10-16T11:21:26.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebff5c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1eb815dfd0 0x7f1eb80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:26.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebff5c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1eb80af450 0x7f1eb806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:26.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebff5c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb806f870 con 0x7f1eb806db80 2023-10-16T11:21:26.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebff5c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb80d1160 con 0x7f1eb80af450 2023-10-16T11:21:26.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebff5c700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb80cc8e0 con 0x7f1eb815dfd0 2023-10-16T11:21:26.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebd4f7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1eb80af450 0x7f1eb806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:26.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebe4f9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1eb806db80 0x7f1eb80cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42772/0 (socket says 172.21.15.181:42772) 2023-10-16T11:21:26.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebe4f9700 1 -- 172.21.15.181:0/3142030553 learned_addr learned my addr 172.21.15.181:0/3142030553 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:26.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebdcf8700 1 --2- 172.21.15.181:0/3142030553 >> v2:172.21.15.190:3300/0 conn(0x7f1eb815dfd0 0x7f1eb80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:26.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebd4f7700 1 -- 172.21.15.181:0/3142030553 >> v2:172.21.15.190:3300/0 conn(0x7f1eb815dfd0 msgr2=0x7f1eb80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:26.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebd4f7700 1 --2- 172.21.15.181:0/3142030553 >> v2:172.21.15.190:3300/0 conn(0x7f1eb815dfd0 0x7f1eb80af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebd4f7700 1 -- 172.21.15.181:0/3142030553 >> v1:172.21.15.181:6789/0 conn(0x7f1eb806db80 legacy=0x7f1eb80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebccf6700 1 -- 172.21.15.181:0/3142030553 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947012456 0 0) 0x7f1eb806f870 con 0x7f1eb806db80 2023-10-16T11:21:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.234+0000 7f1ebd4f7700 1 -- 172.21.15.181:0/3142030553 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80ccbc0 con 0x7f1eb80af450 2023-10-16T11:21:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebd4f7700 1 --2- 172.21.15.181:0/3142030553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb80af450 0x7f1eb806d640 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8070340 tx=0x7f1eb4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac74a7d70b9e473c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebccf6700 1 -- 172.21.15.181:0/3142030553 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4011830 con 0x7f1eb80af450 2023-10-16T11:21:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebccf6700 1 -- 172.21.15.181:0/3142030553 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb4011990 con 0x7f1eb80af450 2023-10-16T11:21:26.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebccf6700 1 -- 172.21.15.181:0/3142030553 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4011d20 con 0x7f1eb80af450 2023-10-16T11:21:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 -- 172.21.15.181:0/3142030553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb80af450 msgr2=0x7f1eb806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/3142030553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb80af450 0x7f1eb806d640 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8070340 tx=0x7f1eb4009b20 comp rx=0 tx=0).stop 2023-10-16T11:21:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 -- 172.21.15.181:0/3142030553 shutdown_connections 2023-10-16T11:21:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/3142030553 >> v2:172.21.15.190:3300/0 conn(0x7f1eb815dfd0 0x7f1eb80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/3142030553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb80af450 0x7f1eb806d640 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 -- 172.21.15.181:0/3142030553 >> 172.21.15.181:0/3142030553 conn(0x7f1eb80d04a0 msgr2=0x7f1eb80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 -- 172.21.15.181:0/3142030553 shutdown_connections 2023-10-16T11:21:26.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.235+0000 7f1ebff5c700 1 -- 172.21.15.181:0/3142030553 wait complete. 2023-10-16T11:21:26.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 Processor -- start 2023-10-16T11:21:26.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 -- start start 2023-10-16T11:21:26.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1eb806db80 0x7f1eb80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:26.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1eb80af450 0x7f1eb81518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:26.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb815dfd0 0x7f1eb80ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:26.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80ade20 con 0x7f1eb815dfd0 2023-10-16T11:21:26.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80b02e0 con 0x7f1eb80af450 2023-10-16T11:21:26.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebff5c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80b05c0 con 0x7f1eb806db80 2023-10-16T11:21:26.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebe4f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb815dfd0 0x7f1eb80ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:26.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebe4f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb815dfd0 0x7f1eb80ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59932/0 (socket says 172.21.15.181:59932) 2023-10-16T11:21:26.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.237+0000 7f1ebe4f9700 1 -- 172.21.15.181:0/2317966926 learned_addr learned my addr 172.21.15.181:0/2317966926 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:26.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebd4f7700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1eb80af450 0x7f1eb81518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:26.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebdcf8700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1eb806db80 0x7f1eb80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:26.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebe4f9700 1 -- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1eb806db80 msgr2=0x7f1eb80ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebe4f9700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1eb806db80 0x7f1eb80ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebe4f9700 1 -- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1eb80af450 msgr2=0x7f1eb81518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.247 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebe4f9700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1eb80af450 0x7f1eb81518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebe4f9700 1 -- 172.21.15.181:0/2317966926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40094a0 con 0x7f1eb815dfd0 2023-10-16T11:21:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebd4f7700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1eb80af450 0x7f1eb81518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebe4f9700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb815dfd0 0x7f1eb80ce3a0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cca0 tx=0x7f1eb000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1eae7fc700 1 -- 172.21.15.181:0/2317966926 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000cea0 con 0x7f1eb815dfd0 2023-10-16T11:21:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1eae7fc700 1 -- 172.21.15.181:0/2317966926 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000ade0 con 0x7f1eb815dfd0 2023-10-16T11:21:26.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb80b08a0 con 0x7f1eb815dfd0 2023-10-16T11:21:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1eae7fc700 1 -- 172.21.15.181:0/2317966926 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0007310 con 0x7f1eb815dfd0 2023-10-16T11:21:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.238+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb80b0e60 con 0x7f1eb815dfd0 2023-10-16T11:21:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.239+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb80b1080 con 0x7f1eb815dfd0 2023-10-16T11:21:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.240+0000 7f1eae7fc700 1 -- 172.21.15.181:0/2317966926 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1eb0007470 con 0x7f1eb815dfd0 2023-10-16T11:21:26.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.240+0000 7f1eae7fc700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1e9c06b140 0x7f1e9c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.240+0000 7f1ebdcf8700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1e9c06b140 0x7f1e9c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.240+0000 7f1eae7fc700 1 -- 172.21.15.181:0/2317966926 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1eb0089910 con 0x7f1eb815dfd0 2023-10-16T11:21:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.240+0000 7f1ebdcf8700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1e9c06b140 0x7f1e9c06d600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006fd0 tx=0x7f1ea8008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:26.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.245+0000 7f1eae7fc700 1 -- 172.21.15.181:0/2317966926 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1eb0054430 con 0x7f1eb815dfd0 2023-10-16T11:21:26.481 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.479+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1eb8067d20 con 0x7f1eb815dfd0 2023-10-16T11:21:26.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.480+0000 7f1eae7fc700 1 -- 172.21.15.181:0/2317966926 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+4941 (secure 0 0 0) 0x7f1eb0018020 con 0x7f1eb815dfd0 2023-10-16T11:21:26.482 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:26.482 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3049755410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3049755410},{"type":"v1","addr":"172.21.15.190:6834","nonce":3049755410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2185474337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2185474337},{"type":"v1","addr":"172.21.15.181:6837","nonce":2185474337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2846112501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2846112501},{"type":"v1","addr":"172.21.15.181:6836","nonce":2846112501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:20.526852+0000","modified":"2023-10-16T11:21:23.550181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12930":{"gid":12930,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3258038365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3258038365},{"type":"v1","addr":"172.21.15.190:6835","nonce":3258038365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:21:26.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1e9c06b140 msgr2=0x7f1e9c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.484 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1e9c06b140 0x7f1e9c06d600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006fd0 tx=0x7f1ea8008040 comp rx=0 tx=0).stop 2023-10-16T11:21:26.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb815dfd0 msgr2=0x7f1eb80ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:26.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb815dfd0 0x7f1eb80ce3a0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cca0 tx=0x7f1eb000a340 comp rx=0 tx=0).stop 2023-10-16T11:21:26.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 shutdown_connections 2023-10-16T11:21:26.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1eb806db80 0x7f1eb80ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1e9c06b140 0x7f1e9c06d600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1eb815dfd0 0x7f1eb80ce3a0 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 --2- 172.21.15.181:0/2317966926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1eb80af450 0x7f1eb81518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:26.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 >> 172.21.15.181:0/2317966926 conn(0x7f1eb80d04a0 msgr2=0x7f1eb80cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:26.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 shutdown_connections 2023-10-16T11:21:26.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:26.482+0000 7f1ebff5c700 1 -- 172.21.15.181:0/2317966926 wait complete. 2023-10-16T11:21:26.487 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 144 2023-10-16T11:21:26.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 144, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:21:20.526852+0000', 'modified': '2023-10-16T11:21:23.550181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12930}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12930': {'gid': 12930, 'name': 'b', 'rank': 0, 'incarnation': 140, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/3258038365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 3258038365}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 3258038365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [35], 'metadata_pool': 34, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:21:26.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:21:26.900 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T11:21:27.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 Processor -- start 2023-10-16T11:21:27.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 -- start start 2023-10-16T11:21:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92d015df60 0x7f92d00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f92d00d0fc0 0x7f92d00cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d006a8a0 0x7f92d006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d006b290 con 0x7f92d015df60 2023-10-16T11:21:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d00af9c0 con 0x7f92d00d0fc0 2023-10-16T11:21:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92d68d0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00ccf50 con 0x7f92d006a8a0 2023-10-16T11:21:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.021+0000 7f92cffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92d015df60 0x7f92d00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92d015df60 0x7f92d00cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59936/0 (socket says 172.21.15.181:59936) 2023-10-16T11:21:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 -- 172.21.15.181:0/869255416 learned_addr learned my addr 172.21.15.181:0/869255416 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/869255416 >> v2:172.21.15.190:3300/0 conn(0x7f92d00d0fc0 0x7f92d00cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 -- 172.21.15.181:0/869255416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d006a8a0 msgr2=0x7f92d006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 --2- 172.21.15.181:0/869255416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d006a8a0 0x7f92d006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 -- 172.21.15.181:0/869255416 >> v2:172.21.15.190:3300/0 conn(0x7f92d00d0fc0 msgr2=0x7f92d00cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 --2- 172.21.15.181:0/869255416 >> v2:172.21.15.190:3300/0 conn(0x7f92d00d0fc0 0x7f92d00cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 -- 172.21.15.181:0/869255416 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d00cc600 con 0x7f92d015df60 2023-10-16T11:21:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92cffff700 1 --2- 172.21.15.181:0/869255416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d00cae20 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f92d006c990 tx=0x7f92c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d942d58a013a49 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92ceffd700 1 -- 172.21.15.181:0/869255416 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4005e30 con 0x7f92d015df60 2023-10-16T11:21:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92ceffd700 1 -- 172.21.15.181:0/869255416 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4004030 con 0x7f92d015df60 2023-10-16T11:21:27.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.022+0000 7f92ceffd700 1 -- 172.21.15.181:0/869255416 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40043e0 con 0x7f92d015df60 2023-10-16T11:21:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 -- 172.21.15.181:0/869255416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 msgr2=0x7f92d00cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 --2- 172.21.15.181:0/869255416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d00cae20 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f92d006c990 tx=0x7f92c4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 -- 172.21.15.181:0/869255416 shutdown_connections 2023-10-16T11:21:27.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 --2- 172.21.15.181:0/869255416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d00cae20 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 --2- 172.21.15.181:0/869255416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d006a8a0 0x7f92d006ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 --2- 172.21.15.181:0/869255416 >> v2:172.21.15.190:3300/0 conn(0x7f92d00d0fc0 0x7f92d00cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 -- 172.21.15.181:0/869255416 >> 172.21.15.181:0/869255416 conn(0x7f92d00d04a0 msgr2=0x7f92d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 -- 172.21.15.181:0/869255416 shutdown_connections 2023-10-16T11:21:27.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.023+0000 7f92d68d0700 1 -- 172.21.15.181:0/869255416 wait complete. 2023-10-16T11:21:27.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.024+0000 7f92d68d0700 1 Processor -- start 2023-10-16T11:21:27.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.024+0000 7f92d68d0700 1 -- start start 2023-10-16T11:21:27.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d68d0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d006a8a0 0x7f92d00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d68d0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d00d0fc0 0x7f92d00c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d68d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d68d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d006f010 con 0x7f92d015df60 2023-10-16T11:21:27.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d68d0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d006e390 con 0x7f92d006a8a0 2023-10-16T11:21:27.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d68d0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d006e670 con 0x7f92d00d0fc0 2023-10-16T11:21:27.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d006d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59946/0 (socket says 172.21.15.181:59946) 2023-10-16T11:21:27.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 -- 172.21.15.181:0/2454199521 learned_addr learned my addr 172.21.15.181:0/2454199521 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:27.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92cffff700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d006a8a0 0x7f92d00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d00d0fc0 0x7f92d00c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 -- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d00d0fc0 msgr2=0x7f92d00c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d00d0fc0 0x7f92d00c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 -- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d006a8a0 msgr2=0x7f92d00c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d006a8a0 0x7f92d00c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 -- 172.21.15.181:0/2454199521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c4009500 con 0x7f92d015df60 2023-10-16T11:21:27.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d006d6d0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f92c800b970 tx=0x7f92c800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92ccff9700 1 -- 172.21.15.181:0/2454199521 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800d860 con 0x7f92d015df60 2023-10-16T11:21:27.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92ccff9700 1 -- 172.21.15.181:0/2454199521 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c800d9e0 con 0x7f92d015df60 2023-10-16T11:21:27.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.025+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d006e950 con 0x7f92d015df60 2023-10-16T11:21:27.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.026+0000 7f92ccff9700 1 -- 172.21.15.181:0/2454199521 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800dd90 con 0x7f92d015df60 2023-10-16T11:21:27.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.026+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d014fef0 con 0x7f92d015df60 2023-10-16T11:21:27.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.027+0000 7f92ccff9700 1 -- 172.21.15.181:0/2454199521 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f92c800e430 con 0x7f92d015df60 2023-10-16T11:21:27.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.027+0000 7f92ccff9700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f92bc06b140 0x7f92bc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.028+0000 7f92ccff9700 1 -- 172.21.15.181:0/2454199521 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f92c8006e40 con 0x7f92d015df60 2023-10-16T11:21:27.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.028+0000 7f92cffff700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f92bc06b140 0x7f92bc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.028+0000 7f92cffff700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f92bc06b140 0x7f92bc06d600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92c4000c00 tx=0x7f92c4008070 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.029+0000 7f92e2abab80 1 Processor -- start 2023-10-16T11:21:27.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.029+0000 7f92e2abab80 1 -- start start 2023-10-16T11:21:27.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x5603269c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92e2abab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032690dc20 con 0x5603269bc220 2023-10-16T11:21:27.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603268f2c10 con 0x5603269c1630 2023-10-16T11:21:27.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603268f0530 con 0x560326997010 2023-10-16T11:21:27.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x5603269c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x5603269c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59950/0 (socket says 172.21.15.181:59950) 2023-10-16T11:21:27.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 -- 172.21.15.181:0/4013730906 learned_addr learned my addr 172.21.15.181:0/4013730906 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:27.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 -- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 msgr2=0x5603269976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 -- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 msgr2=0x5603269c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.030+0000 7f92cffff700 1 -- 172.21.15.181:0/4013730906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603268a93a0 con 0x5603269bc220 2023-10-16T11:21:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92cffff700 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x5603269c3a70 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f92c4014040 tx=0x7f92c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1774383342ef8a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92b5ffb700 1 -- 172.21.15.181:0/4013730906 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4028640 con 0x5603269bc220 2023-10-16T11:21:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92b5ffb700 1 -- 172.21.15.181:0/4013730906 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4020070 con 0x5603269bc220 2023-10-16T11:21:27.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92b5ffb700 1 -- 172.21.15.181:0/4013730906 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40289c0 con 0x5603269bc220 2023-10-16T11:21:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 -- 172.21.15.181:0/4013730906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 msgr2=0x5603269c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x5603269c3a70 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f92c4014040 tx=0x7f92c4010000 comp rx=0 tx=0).stop 2023-10-16T11:21:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 -- 172.21.15.181:0/4013730906 shutdown_connections 2023-10-16T11:21:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x5603269c3a70 unknown :-1 s=CLOSED pgs=2976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 --2- 172.21.15.181:0/4013730906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 -- 172.21.15.181:0/4013730906 >> 172.21.15.181:0/4013730906 conn(0x5603268c6d80 msgr2=0x5603269e1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 -- 172.21.15.181:0/4013730906 shutdown_connections 2023-10-16T11:21:27.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.031+0000 7f92e2abab80 1 -- 172.21.15.181:0/4013730906 wait complete. 2023-10-16T11:21:27.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.032+0000 7f92e2abab80 1 -- start start 2023-10-16T11:21:27.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.032+0000 7f92e2abab80 1 Processor -- start 2023-10-16T11:21:27.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92e2abab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603268ebe20 con 0x560326997010 2023-10-16T11:21:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603268886c0 con 0x5603269c1630 2023-10-16T11:21:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560326888360 con 0x5603269bc220 2023-10-16T11:21:27.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92e2abab80 10 client.0 _ll_register_callbacks cb 0x5603266e57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:21:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269ee8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59956/0 (socket says 172.21.15.181:59956) 2023-10-16T11:21:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 -- 172.21.15.181:0/2979526296 learned_addr learned my addr 172.21.15.181:0/2979526296 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:27.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 -- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 msgr2=0x5603269bd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269bd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 -- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 msgr2=0x5603269c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cffff700 1 -- 172.21.15.181:0/2979526296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d00cc600 con 0x560326997010 2023-10-16T11:21:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.033+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269bd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92cffff700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269ee8a0 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f92c401e040 tx=0x7f92c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67537f9a9870a464 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:21:27.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92aeffd700 1 -- 172.21.15.181:0/2979526296 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4028e20 con 0x560326997010 2023-10-16T11:21:27.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92aeffd700 1 -- 172.21.15.181:0/2979526296 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f92c4037070 con 0x560326997010 2023-10-16T11:21:27.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56032698c5b0 con 0x560326997010 2023-10-16T11:21:27.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92aeffd700 1 -- 172.21.15.181:0/2979526296 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92c403c480 con 0x560326997010 2023-10-16T11:21:27.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 10 client.13053 fetch_fsmap learned FSMap version 144 2023-10-16T11:21:27.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=144,monmap=2+}) v3 -- 0x560326716d90 con 0x560326997010 2023-10-16T11:21:27.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92aeffd700 1 -- 172.21.15.181:0/2979526296 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56032698c5b0 con 0x560326997010 2023-10-16T11:21:27.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92aeffd700 1 -- 172.21.15.181:0/2979526296 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 144) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92c4025050 con 0x560326997010 2023-10-16T11:21:27.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 10 client.13053 fetch_fsmap finished waiting for FSMap version 144 2023-10-16T11:21:27.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 10 client.13053 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:21:27.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 20 client.13053 populate_metadata read hostname 'smithi181' 2023-10-16T11:21:27.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x5603269bfae0 0x560326999570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 4 client.13053 mds_command: new command op to 12930 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:21:27.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 --> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603268e4e00 con 0x5603269bfae0 2023-10-16T11:21:27.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.034+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x5603269bfae0 0x560326999570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.035+0000 7f92aeffd700 10 client.13053 ms_handle_connect on v2:172.21.15.190:6833/3258038365 2023-10-16T11:21:27.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.035+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x5603269bfae0 0x560326999570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ee3687f0c40792 server_cookie=51fa4fc2c92ebc0 in_seq=0 out_seq=0 2023-10-16T11:21:27.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92aeffd700 1 -- 172.21.15.181:0/2979526296 <== mds.0 v2:172.21.15.190:6833/3258038365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603268e4e00 con 0x5603269bfae0 2023-10-16T11:21:27.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92aeffd700 10 client.13053 handle_command_reply: tid=0 2023-10-16T11:21:27.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 client.13053 shutdown 2023-10-16T11:21:27.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 msgr2=0x5603269ee8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269ee8a0 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f92c401e040 tx=0x7f92c40047c0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 shutdown_connections 2023-10-16T11:21:27.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x5603269bfae0 0x560326999570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269ee8a0 unknown :-1 s=CLOSED pgs=2977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 --2- 172.21.15.181:0/2979526296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269bd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92aeffd700 0 client.13053 ms_handle_reset on v2:172.21.15.190:6833/3258038365 2023-10-16T11:21:27.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 >> 172.21.15.181:0/2979526296 conn(0x5603268c6d80 msgr2=0x5603269e1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 shutdown_connections 2023-10-16T11:21:27.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 1 -- 172.21.15.181:0/2979526296 wait complete. 2023-10-16T11:21:27.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.036+0000 7f92e2abab80 20 client.13053 trim_cache size 0 max 16384 2023-10-16T11:21:27.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.053+0000 7f92e2abab80 1 Processor -- start 2023-10-16T11:21:27.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92e2abab80 1 -- start start 2023-10-16T11:21:27.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269be130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92e2abab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032676c8d0 con 0x560326997010 2023-10-16T11:21:27.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032696f3c0 con 0x5603269c1630 2023-10-16T11:21:27.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603269bc8f0 con 0x5603269bc220 2023-10-16T11:21:27.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269be130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92cffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269be130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59964/0 (socket says 172.21.15.181:59964) 2023-10-16T11:21:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92cffff700 1 -- 172.21.15.181:0/1215465865 learned_addr learned my addr 172.21.15.181:0/1215465865 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.054+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92cffff700 1 -- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 msgr2=0x5603269ecec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92cffff700 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269ecec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92cffff700 1 -- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 msgr2=0x5603269ed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92cffff700 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269ed400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92cffff700 1 -- 172.21.15.181:0/1215465865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603266ffc10 con 0x560326997010 2023-10-16T11:21:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92cffff700 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269be130 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7f92c4004630 tx=0x7f92c4033070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfeb1ff00b52c332 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92b67fc700 1 -- 172.21.15.181:0/1215465865 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4028e20 con 0x560326997010 2023-10-16T11:21:27.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92b67fc700 1 -- 172.21.15.181:0/1215465865 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4035070 con 0x560326997010 2023-10-16T11:21:27.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92b67fc700 1 -- 172.21.15.181:0/1215465865 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c403cd10 con 0x560326997010 2023-10-16T11:21:27.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 -- 172.21.15.181:0/1215465865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 msgr2=0x5603269be130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269be130 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7f92c4004630 tx=0x7f92c4033070 comp rx=0 tx=0).stop 2023-10-16T11:21:27.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 -- 172.21.15.181:0/1215465865 shutdown_connections 2023-10-16T11:21:27.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560326997010 0x5603269be130 unknown :-1 s=CLOSED pgs=2978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 --2- 172.21.15.181:0/1215465865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5603269bc220 0x5603269ecec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 -- 172.21.15.181:0/1215465865 >> 172.21.15.181:0/1215465865 conn(0x5603268c6d80 msgr2=0x5603269c4580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 -- 172.21.15.181:0/1215465865 shutdown_connections 2023-10-16T11:21:27.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.055+0000 7f92e2abab80 1 -- 172.21.15.181:0/1215465865 wait complete. 2023-10-16T11:21:27.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.056+0000 7f92e2abab80 1 -- start start 2023-10-16T11:21:27.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.056+0000 7f92e2abab80 1 Processor -- start 2023-10-16T11:21:27.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x560326a44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92e2abab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269dea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92e2abab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560326a45580 con 0x5603269bc220 2023-10-16T11:21:27.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603269baf10 con 0x5603269c1630 2023-10-16T11:21:27.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92e2abab80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603269bb160 con 0x560326997010 2023-10-16T11:21:27.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92e2abab80 10 client.0 _ll_register_callbacks cb 0x7f92c40036a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:21:27.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x560326a44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x560326a44a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59972/0 (socket says 172.21.15.181:59972) 2023-10-16T11:21:27.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 -- 172.21.15.181:0/14512333 learned_addr learned my addr 172.21.15.181:0/14512333 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:27.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cffff700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 -- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 msgr2=0x5603269d8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269d8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 -- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 msgr2=0x5603269dea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269dea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 -- 172.21.15.181:0/14512333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560326716d90 con 0x5603269bc220 2023-10-16T11:21:27.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92d4e6d700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269dea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:27.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92cf7fe700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x560326a44a80 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f92d009e460 tx=0x7f92c001b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ed6cd88956f986 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.057+0000 7f92aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:21:27.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92aeffd700 1 -- 172.21.15.181:0/14512333 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004030 con 0x5603269bc220 2023-10-16T11:21:27.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92aeffd700 1 -- 172.21.15.181:0/14512333 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f92c0004190 con 0x5603269bc220 2023-10-16T11:21:27.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92c4020070 con 0x5603269bc220 2023-10-16T11:21:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92aeffd700 1 -- 172.21.15.181:0/14512333 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=144) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92c0004350 con 0x5603269bc220 2023-10-16T11:21:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 10 client.13059 fetch_fsmap learned FSMap version 144 2023-10-16T11:21:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=144,monmap=2+}) v3 -- 0x56032673d200 con 0x5603269bc220 2023-10-16T11:21:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92aeffd700 1 -- 172.21.15.181:0/14512333 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4020070 con 0x5603269bc220 2023-10-16T11:21:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92aeffd700 1 -- 172.21.15.181:0/14512333 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 144) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92c00044b0 con 0x5603269bc220 2023-10-16T11:21:27.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 10 client.13059 fetch_fsmap finished waiting for FSMap version 144 2023-10-16T11:21:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 10 client.13059 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:21:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 20 client.13059 populate_metadata read hostname 'smithi181' 2023-10-16T11:21:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x560326997ba0 0x560326997f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 4 client.13059 mds_command: new command op to 12930 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:21:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 --> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603268e4b20 con 0x560326997ba0 2023-10-16T11:21:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.058+0000 7f92cffff700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x560326997ba0 0x560326997f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92aeffd700 10 client.13059 ms_handle_connect on v2:172.21.15.190:6833/3258038365 2023-10-16T11:21:27.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92cffff700 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x560326997ba0 0x560326997f80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94881b2995407a65 server_cookie=ff0b15e7656da32a in_seq=0 out_seq=0 2023-10-16T11:21:27.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92aeffd700 1 -- 172.21.15.181:0/14512333 <== mds.0 v2:172.21.15.190:6833/3258038365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x5603268e4b20 con 0x560326997ba0 2023-10-16T11:21:27.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92aeffd700 10 client.13059 handle_command_reply: tid=0 2023-10-16T11:21:27.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 client.13059 shutdown 2023-10-16T11:21:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 msgr2=0x560326a44a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x560326a44a80 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f92d009e460 tx=0x7f92c001b3b0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 shutdown_connections 2023-10-16T11:21:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:6833/3258038365,v1:172.21.15.190:6835/3258038365] conn(0x560326997ba0 0x560326997f80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560326997010 0x5603269d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92aeffd700 0 client.13059 ms_handle_reset on v2:172.21.15.190:6833/3258038365 2023-10-16T11:21:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5603269c1630 0x5603269dea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 --2- 172.21.15.181:0/14512333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5603269bc220 0x560326a44a80 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 >> 172.21.15.181:0/14512333 conn(0x5603268c6d80 msgr2=0x560326995850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 shutdown_connections 2023-10-16T11:21:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 1 -- 172.21.15.181:0/14512333 wait complete. 2023-10-16T11:21:27.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.059+0000 7f92e2abab80 20 client.13059 trim_cache size 0 max 16384 2023-10-16T11:21:27.088 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:21:27.088 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:21:27.089 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:21:27.089 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 12930, 2023-10-16T11:21:27.089 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:21:27.089 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:21:27.089 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T11:21:27.090 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9202817489999999, 2023-10-16T11:21:27.090 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 144, 2023-10-16T11:21:27.090 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 256, 2023-10-16T11:21:27.090 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 256, 2023-10-16T11:21:27.090 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.850385753999999 2023-10-16T11:21:27.090 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:21:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f92bc06b140 msgr2=0x7f92bc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f92bc06b140 0x7f92bc06d600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92c4000c00 tx=0x7f92c4008070 comp rx=0 tx=0).stop 2023-10-16T11:21:27.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 msgr2=0x7f92d006d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d006d6d0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f92c800b970 tx=0x7f92c800bc80 comp rx=0 tx=0).stop 2023-10-16T11:21:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 shutdown_connections 2023-10-16T11:21:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f92bc06b140 0x7f92bc06d600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92d015df60 0x7f92d006d6d0 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92d006a8a0 0x7f92d00c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 --2- 172.21.15.181:0/2454199521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92d00d0fc0 0x7f92d00c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 >> 172.21.15.181:0/2454199521 conn(0x7f92d00d04a0 msgr2=0x7f92d015a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 shutdown_connections 2023-10-16T11:21:27.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.060+0000 7f92d68d0700 1 -- 172.21.15.181:0/2454199521 wait complete. 2023-10-16T11:21:27.095 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:21:27.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.215+0000 7f3c88bd5700 1 Processor -- start 2023-10-16T11:21:27.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.215+0000 7f3c88bd5700 1 -- start start 2023-10-16T11:21:27.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c88bd5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c8415eab0 0x7f3c840d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c88bd5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8406ed00 con 0x7f3c8415e6d0 2023-10-16T11:21:27.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c88bd5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c840d02d0 con 0x7f3c840cb0c0 2023-10-16T11:21:27.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c88bd5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8406e0f0 con 0x7f3c8415eab0 2023-10-16T11:21:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3c8415e6d0 0x7f3c8415a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42824/0 (socket says 172.21.15.181:42824) 2023-10-16T11:21:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8259c700 1 -- 172.21.15.181:0/1356243911 learned_addr learned my addr 172.21.15.181:0/1356243911 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c81d9b700 1 --2- 172.21.15.181:0/1356243911 >> v2:172.21.15.190:3300/0 conn(0x7f3c8415eab0 0x7f3c840d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453563556 0 0) 0x7f3c8406ed00 con 0x7f3c8415e6d0 2023-10-16T11:21:27.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c6c0035b0 con 0x7f3c8415e6d0 2023-10-16T11:21:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362785109 0 0) 0x7f3c6c0035b0 con 0x7f3c8415e6d0 2023-10-16T11:21:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 >> v2:172.21.15.190:3300/0 conn(0x7f3c8415eab0 msgr2=0x7f3c840d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 --2- 172.21.15.181:0/1356243911 >> v2:172.21.15.190:3300/0 conn(0x7f3c8415eab0 0x7f3c840d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 >> v1:172.21.15.190:6789/0 conn(0x7f3c840cb0c0 legacy=0x7f3c8415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8406e3d0 con 0x7f3c8415e6d0 2023-10-16T11:21:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f3c740017f0 con 0x7f3c8415e6d0 2023-10-16T11:21:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c81d9b700 1 --2- 172.21.15.181:0/1356243911 >> v2:172.21.15.190:3300/0 conn(0x7f3c8415eab0 0x7f3c840d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 >> v1:172.21.15.181:6789/0 conn(0x7f3c8415e6d0 legacy=0x7f3c8415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c6c004850 0x7f3c6c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6c005210 0x7f3c6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8406e0f0 con 0x7f3c6c003a00 2023-10-16T11:21:27.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c00c0b0 con 0x7f3c6c005210 2023-10-16T11:21:27.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.216+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c00c270 con 0x7f3c6c004850 2023-10-16T11:21:27.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8259c700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c81d9b700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c6c004850 0x7f3c6c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c82d9d700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6c005210 0x7f3c6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8259c700 1 -- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c6c004850 msgr2=0x7f3c6c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8259c700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c6c004850 0x7f3c6c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8259c700 1 -- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6c005210 msgr2=0x7f3c6c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:27.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8259c700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6c005210 0x7f3c6c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8259c700 1 -- 172.21.15.181:0/1356243911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c0035b0 con 0x7f3c6c003a00 2023-10-16T11:21:27.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c82d9d700 1 -- 172.21.15.181:0/1356243911 reap_dead start 2023-10-16T11:21:27.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8259c700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c6c003e20 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c240 tx=0x7f3c6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b83b1dab822e27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7400cde0 con 0x7f3c6c003a00 2023-10-16T11:21:27.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74007b80 con 0x7f3c6c003a00 2023-10-16T11:21:27.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.217+0000 7f3c8159a700 1 -- 172.21.15.181:0/1356243911 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740072f0 con 0x7f3c6c003a00 2023-10-16T11:21:27.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.218+0000 7f3c88bd5700 1 -- 172.21.15.181:0/1356243911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 msgr2=0x7f3c6c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.218+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c6c003e20 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c240 tx=0x7f3c6c005880 comp rx=0 tx=0).stop 2023-10-16T11:21:27.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.218+0000 7f3c88bd5700 1 -- 172.21.15.181:0/1356243911 shutdown_connections 2023-10-16T11:21:27.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.218+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/1356243911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c6c003e20 unknown :-1 s=CLOSED pgs=2981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.218+0000 7f3c88bd5700 1 -- 172.21.15.181:0/1356243911 >> 172.21.15.181:0/1356243911 conn(0x7f3c840d04a0 msgr2=0x7f3c840d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.218+0000 7f3c88bd5700 1 -- 172.21.15.181:0/1356243911 shutdown_connections 2023-10-16T11:21:27.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.218+0000 7f3c88bd5700 1 -- 172.21.15.181:0/1356243911 wait complete. 2023-10-16T11:21:27.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.219+0000 7f3c88bd5700 1 Processor -- start 2023-10-16T11:21:27.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.219+0000 7f3c88bd5700 1 -- start start 2023-10-16T11:21:27.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c88bd5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c84157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c88bd5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c84158eb0 0x7f3c84158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c88bd5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c840b4660 0x7f3c840b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c88bd5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b4240 con 0x7f3c6c003a00 2023-10-16T11:21:27.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c88bd5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b3930 con 0x7f3c840b4660 2023-10-16T11:21:27.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c88bd5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841589a0 con 0x7f3c84158eb0 2023-10-16T11:21:27.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c84157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c84157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59990/0 (socket says 172.21.15.181:59990) 2023-10-16T11:21:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 -- 172.21.15.181:0/170490892 learned_addr learned my addr 172.21.15.181:0/170490892 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c82d9d700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c840b4660 0x7f3c840b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c81d9b700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c84158eb0 0x7f3c84158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 -- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c84158eb0 msgr2=0x7f3c84158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c84158eb0 0x7f3c84158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 -- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c840b4660 msgr2=0x7f3c840b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c840b4660 0x7f3c840b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 -- 172.21.15.181:0/170490892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8415d900 con 0x7f3c6c003a00 2023-10-16T11:21:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c82d9d700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c840b4660 0x7f3c840b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c8259c700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c84157f20 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7f3c74000c00 tx=0x7f3c74001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c72ffd700 1 -- 172.21.15.181:0/170490892 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74000f70 con 0x7f3c6c003a00 2023-10-16T11:21:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.220+0000 7f3c72ffd700 1 -- 172.21.15.181:0/170490892 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74005830 con 0x7f3c6c003a00 2023-10-16T11:21:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.221+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c840b3b80 con 0x7f3c6c003a00 2023-10-16T11:21:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.221+0000 7f3c72ffd700 1 -- 172.21.15.181:0/170490892 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74005be0 con 0x7f3c6c003a00 2023-10-16T11:21:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.221+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8414ea10 con 0x7f3c6c003a00 2023-10-16T11:21:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.221+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c840b44b0 con 0x7f3c6c003a00 2023-10-16T11:21:27.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.222+0000 7f3c72ffd700 1 -- 172.21.15.181:0/170490892 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3c74005d40 con 0x7f3c6c003a00 2023-10-16T11:21:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.222+0000 7f3c72ffd700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f3c6406b140 0x7f3c6406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.222+0000 7f3c81d9b700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f3c6406b140 0x7f3c6406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.223+0000 7f3c72ffd700 1 -- 172.21.15.181:0/170490892 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f3c7408a780 con 0x7f3c6c003a00 2023-10-16T11:21:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.223+0000 7f3c81d9b700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f3c6406b140 0x7f3c6406d600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cfc00 tx=0x7f3c78008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:27.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.227+0000 7f3c72ffd700 1 -- 172.21.15.181:0/170490892 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3c74055330 con 0x7f3c6c003a00 2023-10-16T11:21:27.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.464+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c8414ebd0 con 0x7f3c6c003a00 2023-10-16T11:21:27.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.465+0000 7f3c72ffd700 1 -- 172.21.15.181:0/170490892 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v144) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3c8414ebd0 con 0x7f3c6c003a00 2023-10-16T11:21:27.467 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:21:27.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f3c6406b140 msgr2=0x7f3c6406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f3c6406b140 0x7f3c6406d600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cfc00 tx=0x7f3c78008040 comp rx=0 tx=0).stop 2023-10-16T11:21:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 msgr2=0x7f3c84157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c84157f20 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7f3c74000c00 tx=0x7f3c74001b80 comp rx=0 tx=0).stop 2023-10-16T11:21:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 shutdown_connections 2023-10-16T11:21:27.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f3c6406b140 0x7f3c6406d600 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c840b4660 0x7f3c840b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c6c003a00 0x7f3c84157f20 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 --2- 172.21.15.181:0/170490892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c84158eb0 0x7f3c84158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 >> 172.21.15.181:0/170490892 conn(0x7f3c840d04a0 msgr2=0x7f3c8406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:27.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 shutdown_connections 2023-10-16T11:21:27.472 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:27.467+0000 7f3c88bd5700 1 -- 172.21.15.181:0/170490892 wait complete. 2023-10-16T11:21:27.481 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:21:27.482 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:21:27.482 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T11:21:27.482 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:21:27.483 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:21:27.483 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:21:27.483 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:21:27.484 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:21:27.484 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:21:27.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:27.485 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T11:21:27.503 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:21:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:21:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:21:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:21:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T11:21:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:21:27.504 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:21:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:21:27.504 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:21:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:21:27.505 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:21:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T11:21:27.505 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:21:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T11:21:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:21:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 36582sec preferred_lft 36582sec 2023-10-16T11:21:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T11:21:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:21:27.507 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:21:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:21:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:21:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:21:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T11:21:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:21:27.508 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:21:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:21:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T11:21:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:21:27.509 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:21:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:21:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T11:21:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:21:27.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:27.510 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:21:27.568 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T11:21:27.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:27.569 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T11:21:27.629 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T11:21:27.629 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:21:27.629 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:21:27.630 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:21:27.630 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:21:27.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:27.631 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T11:21:27.631 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T11:21:27.631 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:21:27.632 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:21:27.632 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:21:27.633 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T11:21:27.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:27.734 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:21:27.751 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:21:27.751 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:21:27.751 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:21:27.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:27.752 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T11:21:27.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:21:27.818 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:21:27.863 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:27.874 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:21:27.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:21:27.875 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:21:27.916 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:27.931 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T11:21:27.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:27.932 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:21:28.016 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:21:28.017 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:21:28.017 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:21:28.025 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:21:28.025 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:21:28.025 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:21:28.025 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:21:28.026 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:21:28.100 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:21:28.101 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:21:28.101 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:21:28.101 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:21:28.101 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:21:28.101 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T11:21:28.102 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T11:21:28.102 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T11:21:28.103 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:21:28.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:28.104 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:21:28.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:28.171 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:21:28.257 INFO:teuthology.orchestra.run.smithi181.stdout:client11410 2023-10-16T11:21:28.257 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:28.258 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:28.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:21:28.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.434+0000 7f907ad00700 1 Processor -- start 2023-10-16T11:21:28.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.434+0000 7f907ad00700 1 -- start start 2023-10-16T11:21:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.435+0000 7f907ad00700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f907415df60 0x7f90740cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.435+0000 7f907ad00700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90740cddf0 con 0x7f90740d0fc0 2023-10-16T11:21:28.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.435+0000 7f907ad00700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90740ccf50 con 0x7f907406a8a0 2023-10-16T11:21:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.435+0000 7f907ad00700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740af9c0 con 0x7f907415df60 2023-10-16T11:21:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.435+0000 7f9078a9c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f907415df60 0x7f90740cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.435+0000 7f9073fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f90740d0fc0 0x7f90740cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42850/0 (socket says 172.21.15.181:42850) 2023-10-16T11:21:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.435+0000 7f9073fff700 1 -- 172.21.15.181:0/3914740151 learned_addr learned my addr 172.21.15.181:0/3914740151 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f9078a9c700 1 -- 172.21.15.181:0/3914740151 >> v1:172.21.15.190:6789/0 conn(0x7f907406a8a0 legacy=0x7f907406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:28.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f9078a9c700 1 -- 172.21.15.181:0/3914740151 >> v1:172.21.15.181:6789/0 conn(0x7f90740d0fc0 legacy=0x7f90740cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f90737fe700 1 -- 172.21.15.181:0/3914740151 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585758372 0 0) 0x7f90740cddf0 con 0x7f90740d0fc0 2023-10-16T11:21:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f9078a9c700 1 -- 172.21.15.181:0/3914740151 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740cc600 con 0x7f907415df60 2023-10-16T11:21:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f9078a9c700 1 --2- 172.21.15.181:0/3914740151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907415df60 0x7f90740cae20 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f9068009820 tx=0x7f9068009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a058e870d8a773af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f90737fe700 1 -- 172.21.15.181:0/3914740151 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068005e30 con 0x7f907415df60 2023-10-16T11:21:28.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f90737fe700 1 -- 172.21.15.181:0/3914740151 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9068004030 con 0x7f907415df60 2023-10-16T11:21:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f90737fe700 1 -- 172.21.15.181:0/3914740151 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680043c0 con 0x7f907415df60 2023-10-16T11:21:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f907ad00700 1 -- 172.21.15.181:0/3914740151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907415df60 msgr2=0x7f90740cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.436+0000 7f907ad00700 1 --2- 172.21.15.181:0/3914740151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907415df60 0x7f90740cae20 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f9068009820 tx=0x7f9068009b30 comp rx=0 tx=0).stop 2023-10-16T11:21:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.437+0000 7f907ad00700 1 -- 172.21.15.181:0/3914740151 shutdown_connections 2023-10-16T11:21:28.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.437+0000 7f907ad00700 1 --2- 172.21.15.181:0/3914740151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907415df60 0x7f90740cae20 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.437+0000 7f907ad00700 1 -- 172.21.15.181:0/3914740151 >> 172.21.15.181:0/3914740151 conn(0x7f90740d04a0 msgr2=0x7f90740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.437+0000 7f907ad00700 1 -- 172.21.15.181:0/3914740151 shutdown_connections 2023-10-16T11:21:28.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.437+0000 7f907ad00700 1 -- 172.21.15.181:0/3914740151 wait complete. 2023-10-16T11:21:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.438+0000 7f907ad00700 1 Processor -- start 2023-10-16T11:21:28.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.438+0000 7f907ad00700 1 -- start start 2023-10-16T11:21:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907ad00700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907406a8a0 0x7f90741597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907ad00700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90740d0fc0 0x7f90740c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907ad00700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907415df60 0x7f90740b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907ad00700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740b3390 con 0x7f90740d0fc0 2023-10-16T11:21:28.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907ad00700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740b4eb0 con 0x7f907406a8a0 2023-10-16T11:21:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907ad00700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90740b3630 con 0x7f907415df60 2023-10-16T11:21:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90740d0fc0 0x7f90740c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90740d0fc0 0x7f90740c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60010/0 (socket says 172.21.15.181:60010) 2023-10-16T11:21:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 -- 172.21.15.181:0/2279222008 learned_addr learned my addr 172.21.15.181:0/2279222008 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:28.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907929d700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907415df60 0x7f90740b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9078a9c700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907406a8a0 0x7f90741597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 -- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907415df60 msgr2=0x7f90740b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907415df60 0x7f90740b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 -- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907406a8a0 msgr2=0x7f90741597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907406a8a0 0x7f90741597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 -- 172.21.15.181:0/2279222008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9068009430 con 0x7f90740d0fc0 2023-10-16T11:21:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f9073fff700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90740d0fc0 0x7f90740c9dc0 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7f906000c520 tx=0x7f906000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f90717fa700 1 -- 172.21.15.181:0/2279222008 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060004c80 con 0x7f90740d0fc0 2023-10-16T11:21:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f90717fa700 1 -- 172.21.15.181:0/2279222008 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9060004e00 con 0x7f90740d0fc0 2023-10-16T11:21:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90740b3910 con 0x7f90740d0fc0 2023-10-16T11:21:28.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.439+0000 7f90717fa700 1 -- 172.21.15.181:0/2279222008 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060007310 con 0x7f90740d0fc0 2023-10-16T11:21:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.440+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9074159ec0 con 0x7f90740d0fc0 2023-10-16T11:21:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.441+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90740b3b00 con 0x7f90740d0fc0 2023-10-16T11:21:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.441+0000 7f90717fa700 1 -- 172.21.15.181:0/2279222008 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9060007470 con 0x7f90740d0fc0 2023-10-16T11:21:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.441+0000 7f90717fa700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f905806b140 0x7f905806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.441+0000 7f90717fa700 1 -- 172.21.15.181:0/2279222008 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9060089990 con 0x7f90740d0fc0 2023-10-16T11:21:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.441+0000 7f9078a9c700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f905806b140 0x7f905806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.442+0000 7f9078a9c700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f905806b140 0x7f905806d600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9068009b70 tx=0x7f9068008070 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.446+0000 7f90717fa700 1 -- 172.21.15.181:0/2279222008 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f90600544a0 con 0x7f90740d0fc0 2023-10-16T11:21:28.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.665+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 --> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f90740cd1c0 con 0x7f905806b140 2023-10-16T11:21:28.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.666+0000 7f90717fa700 1 -- 172.21.15.181:0/2279222008 <== mgr.12564 v2:172.21.15.181:6832/3850286714 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9068021070 con 0x7f905806b140 2023-10-16T11:21:28.668 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:21:28.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:28.669 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T11:21:28.669 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:28.669 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:21:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.668+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f905806b140 msgr2=0x7f905806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.668+0000 7f907ad00700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f905806b140 0x7f905806d600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9068009b70 tx=0x7f9068008070 comp rx=0 tx=0).stop 2023-10-16T11:21:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.668+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90740d0fc0 msgr2=0x7f90740c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.668+0000 7f907ad00700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90740d0fc0 0x7f90740c9dc0 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7f906000c520 tx=0x7f906000c8e0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 shutdown_connections 2023-10-16T11:21:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907415df60 0x7f90740b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f905806b140 0x7f905806d600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907406a8a0 0x7f90741597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 --2- 172.21.15.181:0/2279222008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90740d0fc0 0x7f90740c9dc0 unknown :-1 s=CLOSED pgs=2985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 >> 172.21.15.181:0/2279222008 conn(0x7f90740d04a0 msgr2=0x7f907415a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:28.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 shutdown_connections 2023-10-16T11:21:28.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.669+0000 7f907ad00700 1 -- 172.21.15.181:0/2279222008 wait complete. 2023-10-16T11:21:28.683 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:21:28.806 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.804+0000 7f1fb7715700 1 Processor -- start 2023-10-16T11:21:28.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb7715700 1 -- start start 2023-10-16T11:21:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb7715700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1fb015dbd0 0x7f1fb00d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb7715700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1fb00d0610 0x7f1fb00cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb7715700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb015dfb0 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb7715700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb00ce770 con 0x7f1fb015dbd0 2023-10-16T11:21:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb7715700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb00cd3a0 con 0x7f1fb00d0610 2023-10-16T11:21:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb54b1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1fb015dbd0 0x7f1fb00d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb5cb2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1fb00ce2c0 0x7f1fb0161650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42866/0 (socket says 172.21.15.181:42866) 2023-10-16T11:21:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb54b1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1fb015dbd0 0x7f1fb00d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60016/0 (socket says 172.21.15.181:60016) 2023-10-16T11:21:28.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.805+0000 7f1fb5cb2700 1 -- 172.21.15.181:0/1552145848 learned_addr learned my addr 172.21.15.181:0/1552145848 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:28.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fb54b1700 1 -- 172.21.15.181:0/1552145848 >> v2:172.21.15.190:3300/0 conn(0x7f1fb00d0610 msgr2=0x7f1fb00cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:28.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fb54b1700 1 --2- 172.21.15.181:0/1552145848 >> v2:172.21.15.190:3300/0 conn(0x7f1fb00d0610 0x7f1fb00cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fb54b1700 1 -- 172.21.15.181:0/1552145848 >> v1:172.21.15.181:6789/0 conn(0x7f1fb00ce2c0 legacy=0x7f1fb0161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fb54b1700 1 -- 172.21.15.181:0/1552145848 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00cd830 con 0x7f1fb015dbd0 2023-10-16T11:21:28.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fa7fff700 1 -- 172.21.15.181:0/1552145848 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403534604 0 0) 0x7f1fa8001490 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fb54b1700 1 --2- 172.21.15.181:0/1552145848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb015dbd0 0x7f1fb00d00d0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00cb990 tx=0x7f1fa0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e119a2a32fbb6e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:28.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fa7fff700 1 -- 172.21.15.181:0/1552145848 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0011830 con 0x7f1fb015dbd0 2023-10-16T11:21:28.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fa7fff700 1 -- 172.21.15.181:0/1552145848 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0011990 con 0x7f1fb015dbd0 2023-10-16T11:21:28.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fa7fff700 1 -- 172.21.15.181:0/1552145848 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0011d20 con 0x7f1fb015dbd0 2023-10-16T11:21:28.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.806+0000 7f1fb7715700 1 -- 172.21.15.181:0/1552145848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb015dbd0 msgr2=0x7f1fb00d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.812 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.807+0000 7f1fb7715700 1 --2- 172.21.15.181:0/1552145848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb015dbd0 0x7f1fb00d00d0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00cb990 tx=0x7f1fa0009b20 comp rx=0 tx=0).stop 2023-10-16T11:21:28.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.807+0000 7f1fb7715700 1 -- 172.21.15.181:0/1552145848 shutdown_connections 2023-10-16T11:21:28.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.807+0000 7f1fb7715700 1 --2- 172.21.15.181:0/1552145848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb015dbd0 0x7f1fb00d00d0 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.807+0000 7f1fb7715700 1 --2- 172.21.15.181:0/1552145848 >> v2:172.21.15.190:3300/0 conn(0x7f1fb00d0610 0x7f1fb00cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.807+0000 7f1fb7715700 1 -- 172.21.15.181:0/1552145848 >> 172.21.15.181:0/1552145848 conn(0x7f1fb009a930 msgr2=0x7f1fb00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:28.813 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.807+0000 7f1fb7715700 1 -- 172.21.15.181:0/1552145848 shutdown_connections 2023-10-16T11:21:28.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.807+0000 7f1fb7715700 1 -- 172.21.15.181:0/1552145848 wait complete. 2023-10-16T11:21:28.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.808+0000 7f1fb7715700 1 Processor -- start 2023-10-16T11:21:28.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.808+0000 7f1fb7715700 1 -- start start 2023-10-16T11:21:28.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.808+0000 7f1fb7715700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb00ce2c0 0x7f1fb015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.808+0000 7f1fb7715700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1fb00d0610 0x7f1fb015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.808+0000 7f1fb7715700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fb015dbd0 0x7f1fb01606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb7715700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb015f890 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb7715700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb015ec10 con 0x7f1fb015dbd0 2023-10-16T11:21:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb7715700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb015eef0 con 0x7f1fb00d0610 2023-10-16T11:21:28.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb00ce2c0 0x7f1fb015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb00ce2c0 0x7f1fb015a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60022/0 (socket says 172.21.15.181:60022) 2023-10-16T11:21:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 -- 172.21.15.181:0/3900083171 learned_addr learned my addr 172.21.15.181:0/3900083171 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb5cb2700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fb015dbd0 0x7f1fb01606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb4cb0700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1fb00d0610 0x7f1fb015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 -- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1fb00d0610 msgr2=0x7f1fb015e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1fb00d0610 0x7f1fb015e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 -- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fb015dbd0 msgr2=0x7f1fb01606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fb015dbd0 0x7f1fb01606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 -- 172.21.15.181:0/3900083171 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00094a0 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb54b1700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb00ce2c0 0x7f1fb015a630 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00090b0 tx=0x7f1fa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:28.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fa5ffb700 1 -- 172.21.15.181:0/3900083171 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00038a0 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fa5ffb700 1 -- 172.21.15.181:0/3900083171 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa001f070 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb015f1d0 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fa5ffb700 1 -- 172.21.15.181:0/3900083171 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0003c40 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.809+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb015ab70 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.810+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb015f9d0 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.811+0000 7f1fa5ffb700 1 -- 172.21.15.181:0/3900083171 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 159) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1fa0026480 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.811+0000 7f1fa5ffb700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f9406b140 0x7f1f9406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.811+0000 7f1fb4cb0700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f9406b140 0x7f1f9406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.811+0000 7f1fa5ffb700 1 -- 172.21.15.181:0/3900083171 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1fa0021070 con 0x7f1fb00ce2c0 2023-10-16T11:21:28.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.812+0000 7f1fb4cb0700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f9406b140 0x7f1f9406d600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006fd0 tx=0x7f1fac008040 comp rx=0 tx=0).ready entity=mgr.12564 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:28.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:28.816+0000 7f1fa5ffb700 1 -- 172.21.15.181:0/3900083171 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1fa0056290 con 0x7f1fb00ce2c0 2023-10-16T11:21:29.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.064+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1fb0067d20 con 0x7f1fb00ce2c0 2023-10-16T11:21:29.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.177+0000 7f1fa5ffb700 1 -- 172.21.15.181:0/3900083171 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v160) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1fa00598b0 con 0x7f1fb00ce2c0 2023-10-16T11:21:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f9406b140 msgr2=0x7f1f9406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f9406b140 0x7f1f9406d600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006fd0 tx=0x7f1fac008040 comp rx=0 tx=0).stop 2023-10-16T11:21:29.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb00ce2c0 msgr2=0x7f1fb015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb00ce2c0 0x7f1fb015a630 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00090b0 tx=0x7f1fa0004750 comp rx=0 tx=0).stop 2023-10-16T11:21:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 shutdown_connections 2023-10-16T11:21:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f1f9406b140 0x7f1f9406d600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fb015dbd0 0x7f1fb01606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fb00ce2c0 0x7f1fb015a630 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 --2- 172.21.15.181:0/3900083171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1fb00d0610 0x7f1fb015e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.181+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 >> 172.21.15.181:0/3900083171 conn(0x7f1fb009a930 msgr2=0x7f1fb00cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:29.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.183+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 shutdown_connections 2023-10-16T11:21:29.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.183+0000 7f1fb7715700 1 -- 172.21.15.181:0/3900083171 wait complete. 2023-10-16T11:21:29.198 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:21:29.327 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:29.325+0000 7efc70780200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:21:29.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.325+0000 7ff81ae3e700 1 Processor -- start 2023-10-16T11:21:29.327 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.325+0000 7ff81ae3e700 1 -- start start 2023-10-16T11:21:29.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff81ae3e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff8140cb2b0 0x7ff81406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff81ae3e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff8140cb690 0x7ff8140b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff81ae3e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff814070590 con 0x7ff8140b2ba0 2023-10-16T11:21:29.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff81ae3e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff814156330 con 0x7ff8140cb690 2023-10-16T11:21:29.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff81ae3e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81406fcf0 con 0x7ff8140cb2b0 2023-10-16T11:21:29.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff813fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff8140cb690 0x7ff8140b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff813fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff8140cb690 0x7ff8140b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60060/0 (socket says 172.21.15.181:60060) 2023-10-16T11:21:29.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff813fff700 1 -- 172.21.15.181:0/3613022846 learned_addr learned my addr 172.21.15.181:0/3613022846 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:29.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.326+0000 7ff818bda700 1 --2- 172.21.15.181:0/3613022846 >> v2:172.21.15.190:3300/0 conn(0x7ff8140cb2b0 0x7ff81406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff813fff700 1 -- 172.21.15.181:0/3613022846 >> v2:172.21.15.190:3300/0 conn(0x7ff8140cb2b0 msgr2=0x7ff81406f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:29.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff813fff700 1 --2- 172.21.15.181:0/3613022846 >> v2:172.21.15.190:3300/0 conn(0x7ff8140cb2b0 0x7ff81406f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff813fff700 1 -- 172.21.15.181:0/3613022846 >> v1:172.21.15.190:6789/0 conn(0x7ff8140b2ba0 legacy=0x7ff81415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff813fff700 1 -- 172.21.15.181:0/3613022846 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81406ffd0 con 0x7ff8140cb690 2023-10-16T11:21:29.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff813fff700 1 --2- 172.21.15.181:0/3613022846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8140b2660 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7ff8140d1600 tx=0x7ff800009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7133e5f447ae528a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:29.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff8137fe700 1 -- 172.21.15.181:0/3613022846 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800011830 con 0x7ff8140cb690 2023-10-16T11:21:29.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff8137fe700 1 -- 172.21.15.181:0/3613022846 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800011990 con 0x7ff8140cb690 2023-10-16T11:21:29.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.327+0000 7ff8137fe700 1 -- 172.21.15.181:0/3613022846 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800011d20 con 0x7ff8140cb690 2023-10-16T11:21:29.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 -- 172.21.15.181:0/3613022846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 msgr2=0x7ff8140b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/3613022846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8140b2660 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7ff8140d1600 tx=0x7ff800009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 -- 172.21.15.181:0/3613022846 shutdown_connections 2023-10-16T11:21:29.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/3613022846 >> v2:172.21.15.190:3300/0 conn(0x7ff8140cb2b0 0x7ff81406f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/3613022846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8140b2660 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 -- 172.21.15.181:0/3613022846 >> 172.21.15.181:0/3613022846 conn(0x7ff8140cd010 msgr2=0x7ff8140cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:29.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 -- 172.21.15.181:0/3613022846 shutdown_connections 2023-10-16T11:21:29.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.328+0000 7ff81ae3e700 1 -- 172.21.15.181:0/3613022846 wait complete. 2023-10-16T11:21:29.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.329+0000 7ff81ae3e700 1 Processor -- start 2023-10-16T11:21:29.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.329+0000 7ff81ae3e700 1 -- start start 2023-10-16T11:21:29.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff81ae3e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8140b2ba0 0x7ff8141552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff81ae3e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8140cb2b0 0x7ff81415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff81ae3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8141f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff81ae3e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140ce750 con 0x7ff8140cb690 2023-10-16T11:21:29.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff81ae3e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81415bf50 con 0x7ff8140cb2b0 2023-10-16T11:21:29.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff81ae3e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81415c230 con 0x7ff8140b2ba0 2023-10-16T11:21:29.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8141f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.337 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8141f6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60068/0 (socket says 172.21.15.181:60068) 2023-10-16T11:21:29.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 -- 172.21.15.181:0/1251085434 learned_addr learned my addr 172.21.15.181:0/1251085434 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:29.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff818bda700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8140b2ba0 0x7ff8141552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff813fff700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8140cb2b0 0x7ff81415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 -- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8140b2ba0 msgr2=0x7ff8141552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8140b2ba0 0x7ff8141552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 -- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8140cb2b0 msgr2=0x7ff81415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8140cb2b0 0x7ff81415b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.330+0000 7ff8193db700 1 -- 172.21.15.181:0/1251085434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009550 con 0x7ff8140cb690 2023-10-16T11:21:29.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.331+0000 7ff8193db700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8141f6470 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00cc50 tx=0x7ff80c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:29.339 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:29.331+0000 7efcaafac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:21:29.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.331+0000 7ff8117fa700 1 -- 172.21.15.181:0/1251085434 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c003c00 con 0x7ff8140cb690 2023-10-16T11:21:29.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.331+0000 7ff8117fa700 1 -- 172.21.15.181:0/1251085434 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c003d60 con 0x7ff8140cb690 2023-10-16T11:21:29.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.331+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8140ce960 con 0x7ff8140cb690 2023-10-16T11:21:29.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.331+0000 7ff8117fa700 1 -- 172.21.15.181:0/1251085434 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00aab0 con 0x7ff8140cb690 2023-10-16T11:21:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.331+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8141f7260 con 0x7ff8140cb690 2023-10-16T11:21:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.332+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8140cee80 con 0x7ff8140cb690 2023-10-16T11:21:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.332+0000 7ff8117fa700 1 -- 172.21.15.181:0/1251085434 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 160) v1 ==== 89450+0+0 (secure 0 0 0) 0x7ff80c00ac10 con 0x7ff8140cb690 2023-10-16T11:21:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.333+0000 7ff8117fa700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 0x7ff7f806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.333+0000 7ff818bda700 1 -- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 msgr2=0x7ff7f806d680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.333+0000 7ff8117fa700 1 -- 172.21.15.181:0/1251085434 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ff80c089c70 con 0x7ff8140cb690 2023-10-16T11:21:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.333+0000 7ff818bda700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 0x7ff7f806d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:29.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.337+0000 7ff8117fa700 1 -- 172.21.15.181:0/1251085434 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff8140cee80 con 0x7ff8140cb690 2023-10-16T11:21:29.393 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:29.392+0000 7efc70780200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:29.401 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:29.399+0000 7efcaafac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:29.466 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:29.464+0000 7efc70780200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:21:29.474 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:29.473+0000 7efcaafac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:21:29.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.534+0000 7ff818bda700 1 -- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 msgr2=0x7ff7f806d680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:29.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.534+0000 7ff818bda700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 0x7ff7f806d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:29.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.537+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff8140cee80 con 0x7ff8140cb690 2023-10-16T11:21:29.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.540+0000 7ff8117fa700 1 -- 172.21.15.181:0/1251085434 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v89)=0 v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff8140cee80 con 0x7ff8140cb690 2023-10-16T11:21:29.543 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:29.541+0000 7efc70780200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:21:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.542+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 msgr2=0x7ff7f806d680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.542+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 0x7ff7f806d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.542+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 msgr2=0x7ff8141f6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.542+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8141f6470 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00cc50 tx=0x7ff80c00cf60 comp rx=0 tx=0).stop 2023-10-16T11:21:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 shutdown_connections 2023-10-16T11:21:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8140b2ba0 0x7ff8141552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7ff7f806b1c0 0x7ff7f806d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8140cb690 0x7ff8141f6470 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 --2- 172.21.15.181:0/1251085434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8140cb2b0 0x7ff81415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 >> 172.21.15.181:0/1251085434 conn(0x7ff8140cd010 msgr2=0x7ff8140b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 shutdown_connections 2023-10-16T11:21:29.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.543+0000 7ff81ae3e700 1 -- 172.21.15.181:0/1251085434 wait complete. 2023-10-16T11:21:29.554 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:29.552+0000 7efcaafac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:21:29.557 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:21:29.685 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:29.683+0000 7efc70780200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:21:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.685+0000 7fbb9d0f1700 1 Processor -- start 2023-10-16T11:21:29.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.685+0000 7fbb9d0f1700 1 -- start start 2023-10-16T11:21:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.686+0000 7fbb9d0f1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb980b2010 0x7fbb980cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.686+0000 7fbb9d0f1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb980ce1f0 con 0x7fbb98071be0 2023-10-16T11:21:29.689 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.686+0000 7fbb9d0f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb98072090 con 0x7fbb980ce680 2023-10-16T11:21:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.686+0000 7fbb9d0f1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980ccda0 con 0x7fbb980b2010 2023-10-16T11:21:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.686+0000 7fbb96d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb980b2010 0x7fbb980cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.686+0000 7fbb96d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb980b2010 0x7fbb980cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60132/0 (socket says 172.21.15.181:60132) 2023-10-16T11:21:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.686+0000 7fbb96d9d700 1 -- 172.21.15.181:0/2043831672 learned_addr learned my addr 172.21.15.181:0/2043831672 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:29.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.687+0000 7fbb96d9d700 1 -- 172.21.15.181:0/2043831672 >> v1:172.21.15.190:6789/0 conn(0x7fbb980ce680 legacy=0x7fbb980ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.687+0000 7fbb96d9d700 1 -- 172.21.15.181:0/2043831672 >> v1:172.21.15.181:6789/0 conn(0x7fbb98071be0 legacy=0x7fbb980b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.687+0000 7fbb96d9d700 1 -- 172.21.15.181:0/2043831672 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980727e0 con 0x7fbb980b2010 2023-10-16T11:21:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.687+0000 7fbb96d9d700 1 --2- 172.21.15.181:0/2043831672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980b2010 0x7fbb980cc6e0 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7fbb880098f0 tx=0x7fbb88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4156ac183d5acf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.687+0000 7fbb95d9b700 1 -- 172.21.15.181:0/2043831672 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800f8a0 con 0x7fbb980b2010 2023-10-16T11:21:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.687+0000 7fbb95d9b700 1 -- 172.21.15.181:0/2043831672 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8800fa00 con 0x7fbb980b2010 2023-10-16T11:21:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.687+0000 7fbb95d9b700 1 -- 172.21.15.181:0/2043831672 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800fd90 con 0x7fbb980b2010 2023-10-16T11:21:29.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.688+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/2043831672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980b2010 msgr2=0x7fbb980cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.688+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/2043831672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980b2010 0x7fbb980cc6e0 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7fbb880098f0 tx=0x7fbb88009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.688+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/2043831672 shutdown_connections 2023-10-16T11:21:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.688+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/2043831672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980b2010 0x7fbb980cc6e0 unknown :-1 s=CLOSED pgs=2994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.688+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/2043831672 >> 172.21.15.181:0/2043831672 conn(0x7fbb980cd010 msgr2=0x7fbb980cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.688+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/2043831672 shutdown_connections 2023-10-16T11:21:29.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.688+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/2043831672 wait complete. 2023-10-16T11:21:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 Processor -- start 2023-10-16T11:21:29.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 -- start start 2023-10-16T11:21:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb98071be0 0x7fbb98155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb980b2010 0x7fbb9815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980ce680 0x7fbb98158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb981560c0 con 0x7fbb980ce680 2023-10-16T11:21:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980c8610 con 0x7fbb98071be0 2023-10-16T11:21:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.691+0000 7fbb9d0f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980c7a00 con 0x7fbb980b2010 2023-10-16T11:21:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980ce680 0x7fbb98158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980ce680 0x7fbb98158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60140/0 (socket says 172.21.15.181:60140) 2023-10-16T11:21:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 -- 172.21.15.181:0/1122624635 learned_addr learned my addr 172.21.15.181:0/1122624635 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb96d9d700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb98071be0 0x7fbb98155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9659c700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb980b2010 0x7fbb9815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 -- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb980b2010 msgr2=0x7fbb9815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb980b2010 0x7fbb9815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 -- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb98071be0 msgr2=0x7fbb98155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb98071be0 0x7fbb98155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 -- 172.21.15.181:0/1122624635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88009550 con 0x7fbb980ce680 2023-10-16T11:21:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9759e700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980ce680 0x7fbb98158780 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cc80 tx=0x7fbb9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb7f7fe700 1 -- 172.21.15.181:0/1122624635 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90003c00 con 0x7fbb980ce680 2023-10-16T11:21:29.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb7f7fe700 1 -- 172.21.15.181:0/1122624635 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb90003d60 con 0x7fbb980ce680 2023-10-16T11:21:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb980c7ce0 con 0x7fbb980ce680 2023-10-16T11:21:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb7f7fe700 1 -- 172.21.15.181:0/1122624635 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000aaa0 con 0x7fbb980ce680 2023-10-16T11:21:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.692+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb980c82a0 con 0x7fbb980ce680 2023-10-16T11:21:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.693+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb980b6510 con 0x7fbb980ce680 2023-10-16T11:21:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.694+0000 7fbb7f7fe700 1 -- 172.21.15.181:0/1122624635 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 160) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fbb9000ac00 con 0x7fbb980ce680 2023-10-16T11:21:29.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.694+0000 7fbb7f7fe700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 0x7fbb8406d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.694+0000 7fbb96d9d700 1 -- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 msgr2=0x7fbb8406d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.694+0000 7fbb7f7fe700 1 -- 172.21.15.181:0/1122624635 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fbb9008a5a0 con 0x7fbb980ce680 2023-10-16T11:21:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.695+0000 7fbb96d9d700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 0x7fbb8406d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:29.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.699+0000 7fbb7f7fe700 1 -- 172.21.15.181:0/1122624635 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbb90055230 con 0x7fbb980ce680 2023-10-16T11:21:29.710 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:29.708+0000 7efcaafac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:21:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.896+0000 7fbb96d9d700 1 -- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 msgr2=0x7fbb8406d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:29.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.896+0000 7fbb96d9d700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 0x7fbb8406d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:29.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.897+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fbb980cc370 con 0x7fbb980ce680 2023-10-16T11:21:29.902 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.900+0000 7fbb7f7fe700 1 -- 172.21.15.181:0/1122624635 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v90)=0 v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbb90058850 con 0x7fbb980ce680 2023-10-16T11:21:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 msgr2=0x7fbb8406d650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:29.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 0x7fbb8406d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980ce680 msgr2=0x7fbb98158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:29.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980ce680 0x7fbb98158780 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cc80 tx=0x7fbb9000cf90 comp rx=0 tx=0).stop 2023-10-16T11:21:29.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 shutdown_connections 2023-10-16T11:21:29.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7fbb8406b190 0x7fbb8406d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb980ce680 0x7fbb98158780 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb98071be0 0x7fbb98155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 --2- 172.21.15.181:0/1122624635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb980b2010 0x7fbb9815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 >> 172.21.15.181:0/1122624635 conn(0x7fbb980cd010 msgr2=0x7fbb980cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:29.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 shutdown_connections 2023-10-16T11:21:29.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:29.903+0000 7fbb9d0f1700 1 -- 172.21.15.181:0/1122624635 wait complete. 2023-10-16T11:21:29.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:29.918 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-16T11:21:29.939 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-16T11:21:30.009 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:30.008+0000 7efc70780200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:21:30.047 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:30.045+0000 7efcaafac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:21:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.104+0000 7f7a037c3700 1 Processor -- start 2023-10-16T11:21:30.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.104+0000 7f7a037c3700 1 -- start start 2023-10-16T11:21:30.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a037c3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f79fc15f150 0x7f79fc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a037c3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79fc0ce4f0 0x7f79fc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a037c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 0x7f79fc0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a037c3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0ce9a0 con 0x7f79fc0ce4f0 2023-10-16T11:21:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a037c3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0b2ae0 con 0x7f79fc15f150 2023-10-16T11:21:30.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a037c3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0b2dc0 con 0x7f79fc0b14a0 2023-10-16T11:21:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79fc0ce4f0 0x7f79fc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79fc0ce4f0 0x7f79fc0b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60190/0 (socket says 172.21.15.181:60190) 2023-10-16T11:21:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 -- 172.21.15.181:0/2115151268 learned_addr learned my addr 172.21.15.181:0/2115151268 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:30.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a0155f700 1 --2- 172.21.15.181:0/2115151268 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15f150 0x7f79fc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a01d60700 1 --2- 172.21.15.181:0/2115151268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 0x7f79fc0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 -- 172.21.15.181:0/2115151268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 msgr2=0x7f79fc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 --2- 172.21.15.181:0/2115151268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 0x7f79fc0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 -- 172.21.15.181:0/2115151268 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15f150 msgr2=0x7f79fc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 --2- 172.21.15.181:0/2115151268 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15f150 0x7f79fc0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:30.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 -- 172.21.15.181:0/2115151268 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0cf9d0 con 0x7f79fc0ce4f0 2023-10-16T11:21:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a0155f700 1 --2- 172.21.15.181:0/2115151268 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15f150 0x7f79fc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f7a00d5e700 1 --2- 172.21.15.181:0/2115151268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc0ce4f0 0x7f79fc0b0f60 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098f0 tx=0x7f79ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633fdfa1dfe7f73b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.105+0000 7f79f3fff700 1 -- 172.21.15.181:0/2115151268 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00b860 con 0x7f79fc0ce4f0 2023-10-16T11:21:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f79f3fff700 1 -- 172.21.15.181:0/2115151268 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec00b9c0 con 0x7f79fc0ce4f0 2023-10-16T11:21:30.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f79f3fff700 1 -- 172.21.15.181:0/2115151268 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00bd50 con 0x7f79fc0ce4f0 2023-10-16T11:21:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 -- 172.21.15.181:0/2115151268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc0ce4f0 msgr2=0x7f79fc0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 --2- 172.21.15.181:0/2115151268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc0ce4f0 0x7f79fc0b0f60 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0098f0 tx=0x7f79ec009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 -- 172.21.15.181:0/2115151268 shutdown_connections 2023-10-16T11:21:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 --2- 172.21.15.181:0/2115151268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 0x7f79fc0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:30.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 --2- 172.21.15.181:0/2115151268 >> v2:172.21.15.190:3300/0 conn(0x7f79fc15f150 0x7f79fc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:30.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 --2- 172.21.15.181:0/2115151268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc0ce4f0 0x7f79fc0b0f60 unknown :-1 s=CLOSED pgs=2996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:30.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 -- 172.21.15.181:0/2115151268 >> 172.21.15.181:0/2115151268 conn(0x7f79fc09a930 msgr2=0x7f79fc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:30.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 -- 172.21.15.181:0/2115151268 shutdown_connections 2023-10-16T11:21:30.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.106+0000 7f7a037c3700 1 -- 172.21.15.181:0/2115151268 wait complete. 2023-10-16T11:21:30.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 Processor -- start 2023-10-16T11:21:30.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 -- start start 2023-10-16T11:21:30.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 0x7f79fc1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:30.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc0ce4f0 0x7f79fc0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc15f150 0x7f79fc0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0c9270 con 0x7f79fc15f150 2023-10-16T11:21:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0c7070 con 0x7f79fc0ce4f0 2023-10-16T11:21:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a037c3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0c94a0 con 0x7f79fc0b14a0 2023-10-16T11:21:30.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a01d60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc15f150 0x7f79fc0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a01d60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc15f150 0x7f79fc0c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60198/0 (socket says 172.21.15.181:60198) 2023-10-16T11:21:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a01d60700 1 -- 172.21.15.181:0/3316905440 learned_addr learned my addr 172.21.15.181:0/3316905440 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 0x7f79fc1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.108+0000 7f7a00d5e700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc0ce4f0 0x7f79fc0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:30.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a01d60700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 msgr2=0x7f79fc1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a01d60700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79fc0b14a0 0x7f79fc1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a01d60700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc0ce4f0 msgr2=0x7f79fc0c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a01d60700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc0ce4f0 0x7f79fc0c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a01d60700 1 -- 172.21.15.181:0/3316905440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec009550 con 0x7f79fc15f150 2023-10-16T11:21:30.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a00d5e700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79fc0ce4f0 0x7f79fc0c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:30.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a01d60700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc15f150 0x7f79fc0c8d30 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f79f4007d20 tx=0x7f79f400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:30.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400ed30 con 0x7f79fc15f150 2023-10-16T11:21:30.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f4007e70 con 0x7f79fc15f150 2023-10-16T11:21:30.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4014640 con 0x7f79fc15f150 2023-10-16T11:21:30.121 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc0c9780 con 0x7f79fc15f150 2023-10-16T11:21:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.109+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc0c72c0 con 0x7f79fc15f150 2023-10-16T11:21:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.111+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc067d20 con 0x7f79fc15f150 2023-10-16T11:21:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.111+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 160) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f79f40147a0 con 0x7f79fc15f150 2023-10-16T11:21:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.111+0000 7f79f1ffb700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 0x7f79e006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:30.122 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.111+0000 7f7a0155f700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 msgr2=0x7f79e006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:30.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.111+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f79f408a120 con 0x7f79fc15f150 2023-10-16T11:21:30.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.111+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 0x7f79e006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:30.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.116+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f79f4054d20 con 0x7f79fc15f150 2023-10-16T11:21:30.126 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:30.125+0000 7efc70780200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:21:30.168 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:30.166+0000 7efcaafac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:21:30.201 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:30.200+0000 7efc70780200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:21:30.244 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:30.243+0000 7efcaafac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:21:30.281 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:30.280+0000 7efc70780200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:21:30.313 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.312+0000 7f7a0155f700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 msgr2=0x7f79e006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:30.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.312+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 0x7f79e006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:30.326 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:30.325+0000 7efcaafac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:21:30.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.344+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 --> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f79fc0cff90 con 0x7f79e006b190 2023-10-16T11:21:30.652 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:30.650+0000 7efc70780200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:21:30.699 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:30.698+0000 7efcaafac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:21:30.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.713+0000 7f7a0155f700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 msgr2=0x7f79e006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:30.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:30.713+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 0x7f79e006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:21:31.222 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:31.220+0000 7efc70780200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:21:31.270 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:31.269+0000 7efcaafac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:21:31.291 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:31.290+0000 7efc70780200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:21:31.340 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:31.339+0000 7efcaafac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:21:31.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:31.514+0000 7f7a0155f700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 msgr2=0x7f79e006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:31.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:31.514+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 0x7f79e006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:21:31.922 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:31.920+0000 7efc70780200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:31.972 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:31.971+0000 7efcaafac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:31.999 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:31.998+0000 7efc70780200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:21:32.050 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:32.049+0000 7efcaafac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:21:32.144 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:32.142+0000 7efc70780200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:32.195 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:32.194+0000 7efcaafac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:32.372 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:32.370+0000 7efc70780200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:21:32.425 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:32.424+0000 7efcaafac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:21:32.445 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:32.444+0000 7efc70780200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:21:32.499 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:32.498+0000 7efcaafac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:21:32.712 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:32.711+0000 7efc70780200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:21:32.767 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:32.766+0000 7efcaafac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:21:32.936 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:32.934+0000 7efc70780200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:21:32.990 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:32.989+0000 7efcaafac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:21:33.087 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:33.085+0000 7efc70780200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:21:33.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:33.115+0000 7f7a0155f700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 msgr2=0x7f79e006d650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3850286714 2023-10-16T11:21:33.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:33.115+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 0x7f79e006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:21:33.143 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:33.142+0000 7efcaafac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:21:33.202 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:33.200+0000 7efc70780200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:21:33.257 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:33.256+0000 7efcaafac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:21:33.358 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:33.357+0000 7efc70780200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:21:33.413 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:33.412+0000 7efcaafac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:21:33.622 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:33.620+0000 7efc70780200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:33.679 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:33.678+0000 7efcaafac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:33.783 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:33.782+0000 7efc70780200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:21:33.839 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:33.839+0000 7efcaafac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:21:34.313 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:34.312+0000 7efc70780200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:21:34.370 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.369+0000 7efcaafac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:21:34.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:34.382+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 161) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f79f4012dc0 con 0x7f79fc15f150 2023-10-16T11:21:34.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:34.394+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 162) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f79f40550f0 con 0x7f79fc15f150 2023-10-16T11:21:34.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:34.394+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 msgr2=0x7f79e006d650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:34.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:34.394+0000 7f79f1ffb700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3850286714,v1:172.21.15.181:6833/3850286714] conn(0x7f79e006b190 0x7f79e006d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:34.540 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc20f8d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.541 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc20f8d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.541 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc20f8d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.541 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc20f8d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.541 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc20f8d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.542 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc1df87700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.542 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc1df87700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.542 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc1df87700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.542 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc1df87700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:34.542 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:34.539+0000 7efc1df87700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:35.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.402+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mon.0 v2:172.21.15.181:3300/0 9 ==== mgrmap(e 163) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f79f4074dc0 con 0x7f79fc15f150 2023-10-16T11:21:35.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.403+0000 7f79f1ffb700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f79e001ccd0 0x7f79e001d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:35.404 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.403+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 --> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f79e0071190 con 0x7f79e001ccd0 2023-10-16T11:21:35.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.403+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f79e001ccd0 0x7f79e001d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:35.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.406+0000 7f7a0155f700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f79e001ccd0 0x7f79e001d0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f79f8008010 tx=0x7f79f8008040 comp rx=0 tx=0).ready entity=mgr.13098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:35.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.470+0000 7f79f1ffb700 1 -- 172.21.15.181:0/3316905440 <== mgr.13098 v2:172.21.15.181:6832/3503716662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f79e0071190 con 0x7f79e001ccd0 2023-10-16T11:21:35.471 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule set for path snap_test_dir1 2023-10-16T11:21:35.473 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.472+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f79e001ccd0 msgr2=0x7f79e001d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.472+0000 7f7a037c3700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f79e001ccd0 0x7f79e001d0b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f79f8008010 tx=0x7f79f8008040 comp rx=0 tx=0).stop 2023-10-16T11:21:35.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.472+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc15f150 msgr2=0x7f79fc0c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.472+0000 7f7a037c3700 1 --2- 172.21.15.181:0/3316905440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc15f150 0x7f79fc0c8d30 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f79f4007d20 tx=0x7f79f400e3c0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.472+0000 7f7a01d60700 1 -- 172.21.15.181:0/3316905440 reap_dead start 2023-10-16T11:21:35.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.472+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 shutdown_connections 2023-10-16T11:21:35.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.472+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 >> 172.21.15.181:0/3316905440 conn(0x7f79fc09a930 msgr2=0x7f79fc0cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:35.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.473+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 shutdown_connections 2023-10-16T11:21:35.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.473+0000 7f7a037c3700 1 -- 172.21.15.181:0/3316905440 wait complete. 2023-10-16T11:21:35.487 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-16T11:21:35.488 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-16T11:21:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.613+0000 7f0205492700 1 Processor -- start 2023-10-16T11:21:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.613+0000 7f0205492700 1 -- start start 2023-10-16T11:21:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.613+0000 7f0205492700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f02000cea80 0x7f0200070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.613+0000 7f0205492700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020006dc50 con 0x7f0200159ff0 2023-10-16T11:21:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.613+0000 7f0205492700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02000aea30 con 0x7f02000cbc80 2023-10-16T11:21:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.613+0000 7f0205492700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0200071040 con 0x7f02000cea80 2023-10-16T11:21:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01feffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0200159ff0 0x7f02000cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43218/0 (socket says 172.21.15.181:43218) 2023-10-16T11:21:35.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01feffd700 1 -- 172.21.15.181:0/3461965380 learned_addr learned my addr 172.21.15.181:0/3461965380 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:35.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fe7fc700 1 --2- 172.21.15.181:0/3461965380 >> v2:172.21.15.190:3300/0 conn(0x7f02000cea80 0x7f0200070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:35.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fdffb700 1 -- 172.21.15.181:0/3461965380 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701873847 0 0) 0x7f020006dc50 con 0x7f0200159ff0 2023-10-16T11:21:35.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fdffb700 1 -- 172.21.15.181:0/3461965380 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e80035b0 con 0x7f0200159ff0 2023-10-16T11:21:35.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fdffb700 1 -- 172.21.15.181:0/3461965380 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244323506 0 0) 0x7f02000aea30 con 0x7f02000cbc80 2023-10-16T11:21:35.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fdffb700 1 -- 172.21.15.181:0/3461965380 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f020006dc50 con 0x7f02000cbc80 2023-10-16T11:21:35.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fe7fc700 1 -- 172.21.15.181:0/3461965380 >> v1:172.21.15.190:6789/0 conn(0x7f02000cbc80 legacy=0x7f02000ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fe7fc700 1 -- 172.21.15.181:0/3461965380 >> v1:172.21.15.181:6789/0 conn(0x7f0200159ff0 legacy=0x7f02000cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fe7fc700 1 -- 172.21.15.181:0/3461965380 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000cf0f0 con 0x7f02000cea80 2023-10-16T11:21:35.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fe7fc700 1 --2- 172.21.15.181:0/3461965380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02000cea80 0x7f0200070b00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009840 tx=0x7f01f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a0c8349ef086a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:35.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fdffb700 1 -- 172.21.15.181:0/3461965380 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011830 con 0x7f02000cea80 2023-10-16T11:21:35.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fdffb700 1 -- 172.21.15.181:0/3461965380 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f4011990 con 0x7f02000cea80 2023-10-16T11:21:35.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.614+0000 7f01fdffb700 1 -- 172.21.15.181:0/3461965380 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011d20 con 0x7f02000cea80 2023-10-16T11:21:35.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.615+0000 7f0205492700 1 -- 172.21.15.181:0/3461965380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02000cea80 msgr2=0x7f0200070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.615+0000 7f0205492700 1 --2- 172.21.15.181:0/3461965380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02000cea80 0x7f0200070b00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009840 tx=0x7f01f4009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:35.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.615+0000 7f0205492700 1 -- 172.21.15.181:0/3461965380 shutdown_connections 2023-10-16T11:21:35.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.615+0000 7f0205492700 1 --2- 172.21.15.181:0/3461965380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02000cea80 0x7f0200070b00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.615+0000 7f0205492700 1 -- 172.21.15.181:0/3461965380 >> 172.21.15.181:0/3461965380 conn(0x7f020006aa80 msgr2=0x7f020006ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:35.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.615+0000 7f0205492700 1 -- 172.21.15.181:0/3461965380 shutdown_connections 2023-10-16T11:21:35.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.615+0000 7f0205492700 1 -- 172.21.15.181:0/3461965380 wait complete. 2023-10-16T11:21:35.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.616+0000 7f0205492700 1 Processor -- start 2023-10-16T11:21:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.616+0000 7f0205492700 1 -- start start 2023-10-16T11:21:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f0205492700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02000cbc80 0x7f0200098930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f0205492700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02000cea80 0x7f020015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f0205492700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0200159ff0 0x7f020015cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:35.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f0205492700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000b3440 con 0x7f02000cea80 2023-10-16T11:21:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f0205492700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c32c0 con 0x7f0200159ff0 2023-10-16T11:21:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f0205492700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0200098e70 con 0x7f02000cbc80 2023-10-16T11:21:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02000cea80 0x7f020015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02000cea80 0x7f020015c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60380/0 (socket says 172.21.15.181:60380) 2023-10-16T11:21:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 -- 172.21.15.181:0/249397320 learned_addr learned my addr 172.21.15.181:0/249397320 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 -- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02000cbc80 msgr2=0x7f0200098930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01feffd700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02000cbc80 0x7f0200098930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01ff7fe700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0200159ff0 0x7f020015cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02000cbc80 0x7f0200098930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 -- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0200159ff0 msgr2=0x7f020015cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0200159ff0 0x7f020015cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.617+0000 7f01fe7fc700 1 -- 172.21.15.181:0/249397320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40094a0 con 0x7f02000cea80 2023-10-16T11:21:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.618+0000 7f01fe7fc700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02000cea80 0x7f020015c7f0 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b90 tx=0x7f01f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.618+0000 7f01e77fe700 1 -- 172.21.15.181:0/249397320 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f401a860 con 0x7f02000cea80 2023-10-16T11:21:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.618+0000 7f01e77fe700 1 -- 172.21.15.181:0/249397320 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f401a9c0 con 0x7f02000cea80 2023-10-16T11:21:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.618+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02000c7bd0 con 0x7f02000cea80 2023-10-16T11:21:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.618+0000 7f01e77fe700 1 -- 172.21.15.181:0/249397320 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f401ad50 con 0x7f02000cea80 2023-10-16T11:21:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.618+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02000c8190 con 0x7f02000cea80 2023-10-16T11:21:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.619+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02000c3120 con 0x7f02000cea80 2023-10-16T11:21:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.619+0000 7f01e77fe700 1 -- 172.21.15.181:0/249397320 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 163) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f01f4022070 con 0x7f02000cea80 2023-10-16T11:21:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.619+0000 7f01e77fe700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f01ec06add0 0x7f01ec06d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.620+0000 7f01e77fe700 1 -- 172.21.15.181:0/249397320 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f01f408a460 con 0x7f02000cea80 2023-10-16T11:21:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.620+0000 7f01feffd700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f01ec06add0 0x7f01ec06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:35.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.620+0000 7f01feffd700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f01ec06add0 0x7f01ec06d290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01f00011e0 tx=0x7f01f0009040 comp rx=0 tx=0).ready entity=mgr.13098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:35.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.624+0000 7f01e77fe700 1 -- 172.21.15.181:0/249397320 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f02000c3120 con 0x7f02000cea80 2023-10-16T11:21:35.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.844+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 --> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f020006e180 con 0x7f01ec06add0 2023-10-16T11:21:35.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.903+0000 7f01e77fe700 1 -- 172.21.15.181:0/249397320 <== mgr.13098 v2:172.21.15.181:6832/3503716662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f01f0001350 con 0x7f01ec06add0 2023-10-16T11:21:35.905 INFO:teuthology.orchestra.run.smithi181.stdout:Retention added to path snap_test_dir1 2023-10-16T11:21:35.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.905+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f01ec06add0 msgr2=0x7f01ec06d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.905+0000 7f0205492700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f01ec06add0 0x7f01ec06d290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01f00011e0 tx=0x7f01f0009040 comp rx=0 tx=0).stop 2023-10-16T11:21:35.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.905+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02000cea80 msgr2=0x7f020015c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:35.907 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.905+0000 7f0205492700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02000cea80 0x7f020015c7f0 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b90 tx=0x7f01f4004750 comp rx=0 tx=0).stop 2023-10-16T11:21:35.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 shutdown_connections 2023-10-16T11:21:35.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f01ec06add0 0x7f01ec06d290 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02000cbc80 0x7f0200098930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0200159ff0 0x7f020015cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.908 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 --2- 172.21.15.181:0/249397320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02000cea80 0x7f020015c7f0 unknown :-1 s=CLOSED pgs=3007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:35.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 >> 172.21.15.181:0/249397320 conn(0x7f020006aa80 msgr2=0x7f02000ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:35.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 shutdown_connections 2023-10-16T11:21:35.909 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:35.906+0000 7f0205492700 1 -- 172.21.15.181:0/249397320 wait complete. 2023-10-16T11:21:35.919 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-16T11:21:35.919 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-16-11_22_00 in ~25s... 2023-10-16T11:21:35.920 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-16T11:21:35.920 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-16T11:21:36.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.041+0000 7fe143457700 1 Processor -- start 2023-10-16T11:21:36.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.042+0000 7fe143457700 1 -- start start 2023-10-16T11:21:36.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.042+0000 7fe143457700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe13c0d24e0 0x7fe13c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.042+0000 7fe143457700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d0600 0x7fe13c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.042+0000 7fe143457700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c0d2960 con 0x7fe13c15d9e0 2023-10-16T11:21:36.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.042+0000 7fe143457700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13c0d20c0 con 0x7fe13c0d24e0 2023-10-16T11:21:36.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.042+0000 7fe143457700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0d0a90 con 0x7fe13c0d0600 2023-10-16T11:21:36.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1419f4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d0600 0x7fe13c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1409f2700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe13c0d24e0 0x7fe13c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1411f3700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe13c15d9e0 0x7fe13c0cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:55406/0 (socket says 172.21.15.181:55406) 2023-10-16T11:21:36.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1411f3700 1 -- 172.21.15.181:0/2897364435 learned_addr learned my addr 172.21.15.181:0/2897364435 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:36.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1419f4700 1 -- 172.21.15.181:0/2897364435 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0d24e0 msgr2=0x7fe13c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:36.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1419f4700 1 --2- 172.21.15.181:0/2897364435 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0d24e0 0x7fe13c0d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1419f4700 1 -- 172.21.15.181:0/2897364435 >> v1:172.21.15.190:6789/0 conn(0x7fe13c15d9e0 legacy=0x7fe13c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1419f4700 1 -- 172.21.15.181:0/2897364435 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0d4180 con 0x7fe13c0d0600 2023-10-16T11:21:36.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1409f2700 1 --2- 172.21.15.181:0/2897364435 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0d24e0 0x7fe13c0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:36.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.043+0000 7fe1419f4700 1 --2- 172.21.15.181:0/2897364435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d0600 0x7fe13c0b6620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe134009d90 tx=0x7fe13400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d845cd91bc629089 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:36.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe12ffff700 1 -- 172.21.15.181:0/2897364435 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13400e070 con 0x7fe13c0d0600 2023-10-16T11:21:36.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe12ffff700 1 -- 172.21.15.181:0/2897364435 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13400bb20 con 0x7fe13c0d0600 2023-10-16T11:21:36.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe12ffff700 1 -- 172.21.15.181:0/2897364435 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134012430 con 0x7fe13c0d0600 2023-10-16T11:21:36.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 -- 172.21.15.181:0/2897364435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d0600 msgr2=0x7fe13c0b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 --2- 172.21.15.181:0/2897364435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d0600 0x7fe13c0b6620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe134009d90 tx=0x7fe13400b330 comp rx=0 tx=0).stop 2023-10-16T11:21:36.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 -- 172.21.15.181:0/2897364435 shutdown_connections 2023-10-16T11:21:36.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 --2- 172.21.15.181:0/2897364435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d0600 0x7fe13c0b6620 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 --2- 172.21.15.181:0/2897364435 >> v2:172.21.15.190:3300/0 conn(0x7fe13c0d24e0 0x7fe13c0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 -- 172.21.15.181:0/2897364435 >> 172.21.15.181:0/2897364435 conn(0x7fe13c09cb40 msgr2=0x7fe13c0d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:36.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 -- 172.21.15.181:0/2897364435 shutdown_connections 2023-10-16T11:21:36.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.044+0000 7fe143457700 1 -- 172.21.15.181:0/2897364435 wait complete. 2023-10-16T11:21:36.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.045+0000 7fe143457700 1 Processor -- start 2023-10-16T11:21:36.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.045+0000 7fe143457700 1 -- start start 2023-10-16T11:21:36.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe143457700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0600 0x7fe13c15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe143457700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d24e0 0x7fe13c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe143457700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c15d9e0 0x7fe13c160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe143457700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c15c3c0 con 0x7fe13c0d0600 2023-10-16T11:21:36.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe143457700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0b2b20 con 0x7fe13c15d9e0 2023-10-16T11:21:36.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe143457700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1f3240 con 0x7fe13c0d24e0 2023-10-16T11:21:36.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe1411f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0600 0x7fe13c15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe1411f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0600 0x7fe13c15be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60392/0 (socket says 172.21.15.181:60392) 2023-10-16T11:21:36.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe1411f3700 1 -- 172.21.15.181:0/2438184552 learned_addr learned my addr 172.21.15.181:0/2438184552 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:36.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe1409f2700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d24e0 0x7fe13c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.046+0000 7fe1419f4700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c15d9e0 0x7fe13c160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe1411f3700 1 -- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d24e0 msgr2=0x7fe13c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe1411f3700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d24e0 0x7fe13c0b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe1411f3700 1 -- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c15d9e0 msgr2=0x7fe13c160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe1411f3700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c15d9e0 0x7fe13c160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe1411f3700 1 -- 172.21.15.181:0/2438184552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380094a0 con 0x7fe13c0d0600 2023-10-16T11:21:36.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe1411f3700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0600 0x7fe13c15be80 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c480 tx=0x7fe13000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:36.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe12dffb700 1 -- 172.21.15.181:0/2438184552 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130005d70 con 0x7fe13c0d0600 2023-10-16T11:21:36.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe12dffb700 1 -- 172.21.15.181:0/2438184552 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13000ce90 con 0x7fe13c0d0600 2023-10-16T11:21:36.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe134009a70 con 0x7fe13c0d0600 2023-10-16T11:21:36.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe12dffb700 1 -- 172.21.15.181:0/2438184552 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300072d0 con 0x7fe13c0d0600 2023-10-16T11:21:36.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.047+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c1f3800 con 0x7fe13c0d0600 2023-10-16T11:21:36.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.048+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c0b2c60 con 0x7fe13c0d0600 2023-10-16T11:21:36.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.048+0000 7fe12dffb700 1 -- 172.21.15.181:0/2438184552 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 163) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fe130007430 con 0x7fe13c0d0600 2023-10-16T11:21:36.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.049+0000 7fe12dffb700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fe12006ada0 0x7fe12006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.049+0000 7fe12dffb700 1 -- 172.21.15.181:0/2438184552 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fe130088840 con 0x7fe13c0d0600 2023-10-16T11:21:36.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.049+0000 7fe1409f2700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fe12006ada0 0x7fe12006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.049+0000 7fe1409f2700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fe12006ada0 0x7fe12006d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe138009d70 tx=0x7fe138009210 comp rx=0 tx=0).ready entity=mgr.13098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:36.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.053+0000 7fe12dffb700 1 -- 172.21.15.181:0/2438184552 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe130053900 con 0x7fe13c0d0600 2023-10-16T11:21:36.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.279+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 --> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe13c0d1720 con 0x7fe12006ada0 2023-10-16T11:21:36.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.288+0000 7fe12dffb700 1 -- 172.21.15.181:0/2438184552 <== mgr.13098 v2:172.21.15.181:6832/3503716662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fe1380099c0 con 0x7fe12006ada0 2023-10-16T11:21:36.289 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:36.290 INFO:teuthology.orchestra.run.smithi181.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:21:36.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.290+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fe12006ada0 msgr2=0x7fe12006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.290+0000 7fe143457700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fe12006ada0 0x7fe12006d260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe138009d70 tx=0x7fe138009210 comp rx=0 tx=0).stop 2023-10-16T11:21:36.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.290+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0600 msgr2=0x7fe13c15be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.290+0000 7fe143457700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0600 0x7fe13c15be80 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c480 tx=0x7fe13000c790 comp rx=0 tx=0).stop 2023-10-16T11:21:36.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 shutdown_connections 2023-10-16T11:21:36.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fe12006ada0 0x7fe12006d260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe13c15d9e0 0x7fe13c160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c0d0600 0x7fe13c15be80 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 --2- 172.21.15.181:0/2438184552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe13c0d24e0 0x7fe13c0b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 >> 172.21.15.181:0/2438184552 conn(0x7fe13c09cb40 msgr2=0x7fe13c15ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:36.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 shutdown_connections 2023-10-16T11:21:36.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.291+0000 7fe143457700 1 -- 172.21.15.181:0/2438184552 wait complete. 2023-10-16T11:21:36.304 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-16T11:21:36.304 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-16T11:21:36.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-16T11:21:36.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.433+0000 7f130a25c700 1 Processor -- start 2023-10-16T11:21:36.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.433+0000 7f130a25c700 1 -- start start 2023-10-16T11:21:36.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f130a25c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13040d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f130a25c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13040d0600 0x7f13040b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f130a25c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13040d2960 con 0x7f130415d9e0 2023-10-16T11:21:36.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f130a25c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13040d20c0 con 0x7f13040d0600 2023-10-16T11:21:36.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f130a25c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13040d0a90 con 0x7f13040d24e0 2023-10-16T11:21:36.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f1303fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13040d0600 0x7f13040b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f1303fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f13040d0600 0x7f13040b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60406/0 (socket says 172.21.15.181:60406) 2023-10-16T11:21:36.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f1303fff700 1 -- 172.21.15.181:0/281241678 learned_addr learned my addr 172.21.15.181:0/281241678 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:36.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.434+0000 7f1302ffd700 1 --2- 172.21.15.181:0/281241678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13040d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f1303fff700 1 -- 172.21.15.181:0/281241678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 msgr2=0x7f13040d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:36.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f1303fff700 1 --2- 172.21.15.181:0/281241678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13040d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f1303fff700 1 -- 172.21.15.181:0/281241678 >> v1:172.21.15.190:6789/0 conn(0x7f130415d9e0 legacy=0x7f13040cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f1303fff700 1 -- 172.21.15.181:0/281241678 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040d4180 con 0x7f13040d0600 2023-10-16T11:21:36.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f1303fff700 1 --2- 172.21.15.181:0/281241678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13040d0600 0x7f13040b6620 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009e50 tx=0x7f12fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10352c9d6533d04e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:36.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f13027fc700 1 -- 172.21.15.181:0/281241678 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc00e070 con 0x7f13040d0600 2023-10-16T11:21:36.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f13027fc700 1 -- 172.21.15.181:0/281241678 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc00b910 con 0x7f13040d0600 2023-10-16T11:21:36.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f13027fc700 1 -- 172.21.15.181:0/281241678 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc00bcc0 con 0x7f13040d0600 2023-10-16T11:21:36.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f130a25c700 1 -- 172.21.15.181:0/281241678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13040d0600 msgr2=0x7f13040b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f130a25c700 1 --2- 172.21.15.181:0/281241678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13040d0600 0x7f13040b6620 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009e50 tx=0x7f12fc00b330 comp rx=0 tx=0).stop 2023-10-16T11:21:36.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.435+0000 7f130a25c700 1 -- 172.21.15.181:0/281241678 shutdown_connections 2023-10-16T11:21:36.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.436+0000 7f130a25c700 1 --2- 172.21.15.181:0/281241678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13040d0600 0x7f13040b6620 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.436+0000 7f130a25c700 1 --2- 172.21.15.181:0/281241678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13040d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.436+0000 7f130a25c700 1 -- 172.21.15.181:0/281241678 >> 172.21.15.181:0/281241678 conn(0x7f130409cb40 msgr2=0x7f13040d37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:36.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.436+0000 7f130a25c700 1 -- 172.21.15.181:0/281241678 shutdown_connections 2023-10-16T11:21:36.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.436+0000 7f130a25c700 1 -- 172.21.15.181:0/281241678 wait complete. 2023-10-16T11:21:36.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.437+0000 7f130a25c700 1 Processor -- start 2023-10-16T11:21:36.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.437+0000 7f130a25c700 1 -- start start 2023-10-16T11:21:36.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f130a25c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13040d0600 0x7f13040c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f130a25c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13041f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f130a25c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130415d9e0 0x7f13040b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f130a25c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13041f3e30 con 0x7f130415d9e0 2023-10-16T11:21:36.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f130a25c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13040b0b80 con 0x7f13040d0600 2023-10-16T11:21:36.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f130a25c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13040b19e0 con 0x7f13040d24e0 2023-10-16T11:21:36.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130415d9e0 0x7f13040b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1302ffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13041f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130415d9e0 0x7f13040b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60412/0 (socket says 172.21.15.181:60412) 2023-10-16T11:21:36.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f13037fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13040d0600 0x7f13040c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 -- 172.21.15.181:0/351776222 learned_addr learned my addr 172.21.15.181:0/351776222 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:36.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 -- 172.21.15.181:0/351776222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 msgr2=0x7f13041f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13041f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 -- 172.21.15.181:0/351776222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13040d0600 msgr2=0x7f13040c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13040d0600 0x7f13040c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 -- 172.21.15.181:0/351776222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc009b30 con 0x7f130415d9e0 2023-10-16T11:21:36.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f1303fff700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130415d9e0 0x7f13040b2a60 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00b680 tx=0x7f12fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:36.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f12ebfff700 1 -- 172.21.15.181:0/351776222 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0047b0 con 0x7f130415d9e0 2023-10-16T11:21:36.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f12ebfff700 1 -- 172.21.15.181:0/351776222 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc004930 con 0x7f130415d9e0 2023-10-16T11:21:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.438+0000 7f12ebfff700 1 -- 172.21.15.181:0/351776222 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc01c680 con 0x7f130415d9e0 2023-10-16T11:21:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.439+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13040b1e70 con 0x7f130415d9e0 2023-10-16T11:21:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.439+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13040b10b0 con 0x7f130415d9e0 2023-10-16T11:21:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.440+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1304067d20 con 0x7f130415d9e0 2023-10-16T11:21:36.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.440+0000 7f12ebfff700 1 -- 172.21.15.181:0/351776222 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 164) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f12fc01c7e0 con 0x7f130415d9e0 2023-10-16T11:21:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.440+0000 7f12ebfff700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f12f006b070 0x7f12f006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.441+0000 7f12ebfff700 1 -- 172.21.15.181:0/351776222 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f12fc08b850 con 0x7f130415d9e0 2023-10-16T11:21:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.441+0000 7f13037fe700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f12f006b070 0x7f12f006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.441+0000 7f13037fe700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f12f006b070 0x7f12f006d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1304160790 tx=0x7f12f4009040 comp rx=0 tx=0).ready entity=mgr.13098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:36.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.446+0000 7f12ebfff700 1 -- 172.21.15.181:0/351776222 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f12fc056310 con 0x7f130415d9e0 2023-10-16T11:21:36.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.666+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 --> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f13040d1720 con 0x7f12f006b070 2023-10-16T11:21:36.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.735+0000 7f12ebfff700 1 -- 172.21.15.181:0/351776222 <== mgr.13098 v2:172.21.15.181:6832/3503716662 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f12f4001580 con 0x7f12f006b070 2023-10-16T11:21:36.736 INFO:teuthology.orchestra.run.smithi181.stdout:Schedule removed for path snap_test_dir1 2023-10-16T11:21:36.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f12f006b070 msgr2=0x7f12f006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f12f006b070 0x7f12f006d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1304160790 tx=0x7f12f4009040 comp rx=0 tx=0).stop 2023-10-16T11:21:36.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130415d9e0 msgr2=0x7f13040b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:36.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130415d9e0 0x7f13040b2a60 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00b680 tx=0x7f12fc004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 shutdown_connections 2023-10-16T11:21:36.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f12f006b070 0x7f12f006d530 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f130415d9e0 0x7f13040b2a60 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13040d0600 0x7f13040c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 --2- 172.21.15.181:0/351776222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13040d24e0 0x7f13041f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:36.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 >> 172.21.15.181:0/351776222 conn(0x7f130409cb40 msgr2=0x7f13040b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:36.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 shutdown_connections 2023-10-16T11:21:36.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.737+0000 7f130a25c700 1 -- 172.21.15.181:0/351776222 wait complete. 2023-10-16T11:21:36.750 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-16T11:21:36.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:36.751 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-16T11:21:36.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:36.769 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-16T11:21:36.829 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:21:36.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.991+0000 7f1115a18700 1 Processor -- start 2023-10-16T11:21:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.991+0000 7f1115a18700 1 -- start start 2023-10-16T11:21:36.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f1115a18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 0x7f111006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f1115a18700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f111006ade0 0x7f11101615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:36.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f1115a18700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111006b230 con 0x7f111015da70 2023-10-16T11:21:36.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f1115a18700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11100cef00 con 0x7f111006ade0 2023-10-16T11:21:36.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f1115a18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11100cec20 con 0x7f1110072760 2023-10-16T11:21:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f110effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f111015da70 0x7f11100ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43278/0 (socket says 172.21.15.181:43278) 2023-10-16T11:21:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f110effd700 1 -- 172.21.15.181:0/535066822 learned_addr learned my addr 172.21.15.181:0/535066822 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.992+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893129205 0 0) 0x7f1104001110 con 0x7f111015da70 2023-10-16T11:21:36.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f80035b0 con 0x7f111015da70 2023-10-16T11:21:36.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110e7fc700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 0x7f111006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 --2- 172.21.15.181:0/535066822 >> v2:172.21.15.190:3300/0 conn(0x7f111006ade0 0x7f11101615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:36.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 118038954 0 0) 0x7f10f80035b0 con 0x7f111015da70 2023-10-16T11:21:36.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 msgr2=0x7f111006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:36.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 0x7f111006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 >> v2:172.21.15.190:3300/0 conn(0x7f111006ade0 msgr2=0x7f11101615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 --2- 172.21.15.181:0/535066822 >> v2:172.21.15.190:3300/0 conn(0x7f111006ade0 0x7f11101615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110071c80 con 0x7f111015da70 2023-10-16T11:21:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 --2- 172.21.15.181:0/535066822 >> v2:172.21.15.190:3300/0 conn(0x7f111006ade0 0x7f11101615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:37.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f1104001800 con 0x7f111015da70 2023-10-16T11:21:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 >> v1:172.21.15.181:6789/0 conn(0x7f111015da70 legacy=0x7f11100ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8003a00 0x7f10f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 0x7f10f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f8004d90 0x7f10f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100cef00 con 0x7f10f8004d90 2023-10-16T11:21:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100cec20 con 0x7f10f8003a00 2023-10-16T11:21:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f800bc50 con 0x7f1110072760 2023-10-16T11:21:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f8004d90 0x7f10f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110e7fc700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 0x7f10f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110effd700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8003a00 0x7f10f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 -- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 msgr2=0x7f10f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1110072760 0x7f10f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 -- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8003a00 msgr2=0x7f10f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8003a00 0x7f10f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 -- 172.21.15.181:0/535066822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80035b0 con 0x7f10f8004d90 2023-10-16T11:21:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.993+0000 7f110f7fe700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f8004d90 0x7f10f8005210 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7f11080073e0 tx=0x7f10f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accf72641b82fb5b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108014400 con 0x7f10f8004d90 2023-10-16T11:21:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1108014560 con 0x7f10f8004d90 2023-10-16T11:21:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f110dffb700 1 -- 172.21.15.181:0/535066822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11080148d0 con 0x7f10f8004d90 2023-10-16T11:21:37.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f1115a18700 1 -- 172.21.15.181:0/535066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f8004d90 msgr2=0x7f10f8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f1115a18700 1 --2- 172.21.15.181:0/535066822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10f8004d90 0x7f10f8005210 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7f11080073e0 tx=0x7f10f8005880 comp rx=0 tx=0).stop 2023-10-16T11:21:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f110f7fe700 1 -- 172.21.15.181:0/535066822 reap_dead start 2023-10-16T11:21:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f1115a18700 1 -- 172.21.15.181:0/535066822 shutdown_connections 2023-10-16T11:21:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f1115a18700 1 -- 172.21.15.181:0/535066822 >> 172.21.15.181:0/535066822 conn(0x7f11100cd010 msgr2=0x7f11100cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f1115a18700 1 -- 172.21.15.181:0/535066822 shutdown_connections 2023-10-16T11:21:37.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.994+0000 7f1115a18700 1 -- 172.21.15.181:0/535066822 wait complete. 2023-10-16T11:21:37.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.995+0000 7f1115a18700 1 Processor -- start 2023-10-16T11:21:37.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f1115a18700 1 -- start start 2023-10-16T11:21:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f1115a18700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8004d90 0x7f11101597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f1115a18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11100c9f00 0x7f11100c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f1115a18700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11100c8970 0x7f11100cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f1115a18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100c8e60 con 0x7f11100c9f00 2023-10-16T11:21:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f1115a18700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11100c9140 con 0x7f10f8004d90 2023-10-16T11:21:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f1115a18700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11100c9420 con 0x7f11100c8970 2023-10-16T11:21:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f110e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11100c9f00 0x7f11100c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f110e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11100c9f00 0x7f11100c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60446/0 (socket says 172.21.15.181:60446) 2023-10-16T11:21:37.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.996+0000 7f110e7fc700 1 -- 172.21.15.181:0/1931770120 learned_addr learned my addr 172.21.15.181:0/1931770120 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110f7fe700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11100c8970 0x7f11100cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110effd700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8004d90 0x7f11101597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110e7fc700 1 -- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11100c8970 msgr2=0x7f11100cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110e7fc700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11100c8970 0x7f11100cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110e7fc700 1 -- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8004d90 msgr2=0x7f11101597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110e7fc700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8004d90 0x7f11101597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110e7fc700 1 -- 172.21.15.181:0/1931770120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11100c9700 con 0x7f11100c9f00 2023-10-16T11:21:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110f7fe700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11100c8970 0x7f11100cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110effd700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8004d90 0x7f11101597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:37.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f110e7fc700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11100c9f00 0x7f11100c8430 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f11000116c0 tx=0x7f1100011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f10f7fff700 1 -- 172.21.15.181:0/1931770120 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110000c070 con 0x7f11100c9f00 2023-10-16T11:21:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f10f7fff700 1 -- 172.21.15.181:0/1931770120 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1100015870 con 0x7f11100c9f00 2023-10-16T11:21:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f10f7fff700 1 -- 172.21.15.181:0/1931770120 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100015c20 con 0x7f11100c9f00 2023-10-16T11:21:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11100c99e0 con 0x7f11100c9f00 2023-10-16T11:21:37.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.997+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1110159d20 con 0x7f11100c9f00 2023-10-16T11:21:37.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.998+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11100c9d50 con 0x7f11100c9f00 2023-10-16T11:21:37.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.998+0000 7f10f7fff700 1 -- 172.21.15.181:0/1931770120 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 164) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f1100013400 con 0x7f11100c9f00 2023-10-16T11:21:37.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.999+0000 7f10f7fff700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f10fc06b070 0x7f10fc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.999+0000 7f10f7fff700 1 -- 172.21.15.181:0/1931770120 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f1100089f90 con 0x7f11100c9f00 2023-10-16T11:21:37.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.999+0000 7f110effd700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f10fc06b070 0x7f10fc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:36.999+0000 7f110effd700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f10fc06b070 0x7f10fc06d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1104000c00 tx=0x7f1104009040 comp rx=0 tx=0).ready entity=mgr.13098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:37.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.003+0000 7f10f7fff700 1 -- 172.21.15.181:0/1931770120 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1100054a40 con 0x7f11100c9f00 2023-10-16T11:21:37.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.255+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f111006f6c0 con 0x7f11100c9f00 2023-10-16T11:21:37.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.408+0000 7f10f7fff700 1 -- 172.21.15.181:0/1931770120 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v165) v1 ==== 93+0+0 (secure 0 0 0) 0x7f11000230a0 con 0x7f11100c9f00 2023-10-16T11:21:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.412+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f10fc06b070 msgr2=0x7f10fc06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.412+0000 7f1115a18700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f10fc06b070 0x7f10fc06d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1104000c00 tx=0x7f1104009040 comp rx=0 tx=0).stop 2023-10-16T11:21:37.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.412+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11100c9f00 msgr2=0x7f11100c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.412+0000 7f1115a18700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11100c9f00 0x7f11100c8430 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f11000116c0 tx=0x7f1100011a80 comp rx=0 tx=0).stop 2023-10-16T11:21:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.413+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 shutdown_connections 2023-10-16T11:21:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.413+0000 7f1115a18700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f10fc06b070 0x7f10fc06d530 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.413+0000 7f1115a18700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11100c8970 0x7f11100cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.413+0000 7f1115a18700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f8004d90 0x7f11101597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.413+0000 7f1115a18700 1 --2- 172.21.15.181:0/1931770120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11100c9f00 0x7f11100c8430 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.413+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 >> 172.21.15.181:0/1931770120 conn(0x7f11100cd010 msgr2=0x7f11100b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.413+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 shutdown_connections 2023-10-16T11:21:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.414+0000 7f1115a18700 1 -- 172.21.15.181:0/1931770120 wait complete. 2023-10-16T11:21:37.428 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:21:37.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:21:37.471 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:21:37.517 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:21:37.518 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:37.518 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:37.535 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:37.535 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:37.554 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:37.553+0000 7f43536ed200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:21:37.563 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:37.562+0000 7f0c7f014200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:21:37.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:37.620 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:37.619+0000 7f43536ed200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:37.630 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:37.630+0000 7f0c7f014200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:37.692 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:37.690+0000 7f43536ed200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:21:37.705 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:37.705+0000 7f0c7f014200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:21:37.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.740+0000 7f7adfb36700 1 Processor -- start 2023-10-16T11:21:37.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.741+0000 7f7adfb36700 1 -- start start 2023-10-16T11:21:37.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.741+0000 7f7adfb36700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ad80ce7f0 0x7f7ad806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.741+0000 7f7adfb36700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ad80d0350 0x7f7ad80cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.741+0000 7f7adfb36700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad80cc470 con 0x7f7ad80d2390 2023-10-16T11:21:37.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.741+0000 7f7adfb36700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad806ca30 con 0x7f7ad80d0350 2023-10-16T11:21:37.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.741+0000 7f7adfb36700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad806cd10 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ad80d0350 0x7f7ad80cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ad80d0350 0x7f7ad80cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60512/0 (socket says 172.21.15.181:60512) 2023-10-16T11:21:37.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 -- 172.21.15.181:0/478117228 learned_addr learned my addr 172.21.15.181:0/478117228 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:37.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7add8d2700 1 --2- 172.21.15.181:0/478117228 >> v2:172.21.15.190:3300/0 conn(0x7f7ad80ce7f0 0x7f7ad806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 -- 172.21.15.181:0/478117228 >> v2:172.21.15.190:3300/0 conn(0x7f7ad80ce7f0 msgr2=0x7f7ad806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:37.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 --2- 172.21.15.181:0/478117228 >> v2:172.21.15.190:3300/0 conn(0x7f7ad80ce7f0 0x7f7ad806c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 -- 172.21.15.181:0/478117228 >> v1:172.21.15.190:6789/0 conn(0x7f7ad80d2390 legacy=0x7f7ad80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 -- 172.21.15.181:0/478117228 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad806cff0 con 0x7f7ad80d0350 2023-10-16T11:21:37.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7ade0d3700 1 --2- 172.21.15.181:0/478117228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80d0350 0x7f7ad80cc930 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f7ad809af40 tx=0x7f7ad000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8834f6caf99b8ef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:37.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7adc8d0700 1 -- 172.21.15.181:0/478117228 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000e070 con 0x7f7ad80d0350 2023-10-16T11:21:37.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7adc8d0700 1 -- 172.21.15.181:0/478117228 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad000b910 con 0x7f7ad80d0350 2023-10-16T11:21:37.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.742+0000 7f7adc8d0700 1 -- 172.21.15.181:0/478117228 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000bcc0 con 0x7f7ad80d0350 2023-10-16T11:21:37.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 -- 172.21.15.181:0/478117228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80d0350 msgr2=0x7f7ad80cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 --2- 172.21.15.181:0/478117228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80d0350 0x7f7ad80cc930 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f7ad809af40 tx=0x7f7ad000b330 comp rx=0 tx=0).stop 2023-10-16T11:21:37.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 -- 172.21.15.181:0/478117228 shutdown_connections 2023-10-16T11:21:37.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 --2- 172.21.15.181:0/478117228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80d0350 0x7f7ad80cc930 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 --2- 172.21.15.181:0/478117228 >> v2:172.21.15.190:3300/0 conn(0x7f7ad80ce7f0 0x7f7ad806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 -- 172.21.15.181:0/478117228 >> 172.21.15.181:0/478117228 conn(0x7f7ad809a9a0 msgr2=0x7f7ad80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:37.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 -- 172.21.15.181:0/478117228 shutdown_connections 2023-10-16T11:21:37.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.743+0000 7f7adfb36700 1 -- 172.21.15.181:0/478117228 wait complete. 2023-10-16T11:21:37.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.744+0000 7f7adfb36700 1 Processor -- start 2023-10-16T11:21:37.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.744+0000 7f7adfb36700 1 -- start start 2023-10-16T11:21:37.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7adfb36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80ce7f0 0x7f7ad809ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7adfb36700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad80d0350 0x7f7ad81f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7adfb36700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad80d2390 0x7f7ad809c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7adfb36700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad809b310 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7adfb36700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad809b560 con 0x7f7ad80d0350 2023-10-16T11:21:37.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7adfb36700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad809b840 con 0x7f7ad80d2390 2023-10-16T11:21:37.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80ce7f0 0x7f7ad809ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80ce7f0 0x7f7ad809ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60514/0 (socket says 172.21.15.181:60514) 2023-10-16T11:21:37.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 -- 172.21.15.181:0/1237603913 learned_addr learned my addr 172.21.15.181:0/1237603913 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:37.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7ade0d3700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad80d2390 0x7f7ad809c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add0d1700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad80d0350 0x7f7ad81f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:37.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 -- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad80d2390 msgr2=0x7f7ad809c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad80d2390 0x7f7ad809c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 -- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad80d0350 msgr2=0x7f7ad81f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad80d0350 0x7f7ad81f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 -- 172.21.15.181:0/1237603913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad0009b30 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7ade0d3700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad80d2390 0x7f7ad809c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:37.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add0d1700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad80d0350 0x7f7ad81f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:37.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7add8d2700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80ce7f0 0x7f7ad809ef30 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e900 tx=0x7f7ad400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:37.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.745+0000 7f7ace7fc700 1 -- 172.21.15.181:0/1237603913 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4009650 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.746+0000 7f7ace7fc700 1 -- 172.21.15.181:0/1237603913 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad400c850 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.746+0000 7f7ace7fc700 1 -- 172.21.15.181:0/1237603913 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400cbc0 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.746+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad809bb20 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.746+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad80cd040 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.746+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad809c170 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.747+0000 7f7ace7fc700 1 -- 172.21.15.181:0/1237603913 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f7ad400cd20 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.747+0000 7f7ace7fc700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 0x7f7abc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:37.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.747+0000 7f7add0d1700 1 -- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 msgr2=0x7f7abc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:37.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.748+0000 7f7ace7fc700 1 -- 172.21.15.181:0/1237603913 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f7ad4089eb0 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.748+0000 7f7add0d1700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 0x7f7abc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:37.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.756+0000 7f7ace7fc700 1 -- 172.21.15.181:0/1237603913 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7ad4054b00 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.769 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:37.767+0000 7f43536ed200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:21:37.792 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:37.792+0000 7f0c7f014200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:21:37.909 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:37.908+0000 7f43536ed200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:21:37.937 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:37.936+0000 7f0c7f014200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:21:37.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.949+0000 7f7add0d1700 1 -- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 msgr2=0x7f7abc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:37.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.949+0000 7f7add0d1700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 0x7f7abc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:37.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.955+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ad8067d20 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.957+0000 7f7ace7fc700 1 -- 172.21.15.181:0/1237603913 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+18047 (secure 0 0 0) 0x7f7ad4021090 con 0x7f7ad80ce7f0 2023-10-16T11:21:37.959 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:37.959 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":257,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:34.384677+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:18.964173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:19.485121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3637292004":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3476229854":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/1366795220":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2569889467":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6833/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6832/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2946054209":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/649394230":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/507502116":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3665147670":"2023-10-17T11:21:34.384612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:37.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 msgr2=0x7f7abc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:37.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 0x7f7abc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80ce7f0 msgr2=0x7f7ad809ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:37.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80ce7f0 0x7f7ad809ef30 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e900 tx=0x7f7ad400ecc0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 shutdown_connections 2023-10-16T11:21:37.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f7abc06b020 0x7f7abc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ad80d2390 0x7f7ad809c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad80ce7f0 0x7f7ad809ef30 unknown :-1 s=CLOSED pgs=3021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 --2- 172.21.15.181:0/1237603913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ad80d0350 0x7f7ad81f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:37.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 >> 172.21.15.181:0/1237603913 conn(0x7f7ad809a9a0 msgr2=0x7f7ad80d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:37.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 shutdown_connections 2023-10-16T11:21:37.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:37.960+0000 7f7adfb36700 1 -- 172.21.15.181:0/1237603913 wait complete. 2023-10-16T11:21:37.974 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:21:18.964173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '255', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:21:19.485121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '256', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:21:37.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:21:38.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.142+0000 7f07454b4700 1 Processor -- start 2023-10-16T11:21:38.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.142+0000 7f07454b4700 1 -- start start 2023-10-16T11:21:38.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f07454b4700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f074015a7d0 0x7f07400b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f07454b4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074015abb0 con 0x7f07400cf1a0 2023-10-16T11:21:38.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f07454b4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07400b1870 con 0x7f07400d0d60 2023-10-16T11:21:38.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f07454b4700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07400afe50 con 0x7f074015a7d0 2023-10-16T11:21:38.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f073e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f07400cf1a0 0x7f07400d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43406/0 (socket says 172.21.15.181:43406) 2023-10-16T11:21:38.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f073e7fc700 1 -- 172.21.15.181:0/2030639333 learned_addr learned my addr 172.21.15.181:0/2030639333 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:38.147 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f073effd700 1 --2- 172.21.15.181:0/2030639333 >> v2:172.21.15.190:3300/0 conn(0x7f074015a7d0 0x7f07400b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f073dffb700 1 -- 172.21.15.181:0/2030639333 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322630167 0 0) 0x7f074015abb0 con 0x7f07400cf1a0 2023-10-16T11:21:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.143+0000 7f073dffb700 1 -- 172.21.15.181:0/2030639333 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07280035b0 con 0x7f07400cf1a0 2023-10-16T11:21:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073effd700 1 -- 172.21.15.181:0/2030639333 >> v1:172.21.15.190:6789/0 conn(0x7f07400d0d60 legacy=0x7f07400cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073effd700 1 -- 172.21.15.181:0/2030639333 >> v1:172.21.15.181:6789/0 conn(0x7f07400cf1a0 legacy=0x7f07400d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073dffb700 1 -- 172.21.15.181:0/2030639333 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495690838 0 0) 0x7f07400b1870 con 0x7f07400d0d60 2023-10-16T11:21:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073effd700 1 -- 172.21.15.181:0/2030639333 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400b1b50 con 0x7f074015a7d0 2023-10-16T11:21:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073effd700 1 --2- 172.21.15.181:0/2030639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f07400b0f30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0730009840 tx=0x7f0730009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29564a1a1add6737 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073dffb700 1 -- 172.21.15.181:0/2030639333 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730011830 con 0x7f074015a7d0 2023-10-16T11:21:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073dffb700 1 -- 172.21.15.181:0/2030639333 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0730011990 con 0x7f074015a7d0 2023-10-16T11:21:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.144+0000 7f073dffb700 1 -- 172.21.15.181:0/2030639333 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730011d00 con 0x7f074015a7d0 2023-10-16T11:21:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.145+0000 7f07454b4700 1 -- 172.21.15.181:0/2030639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 msgr2=0x7f07400b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.145+0000 7f07454b4700 1 --2- 172.21.15.181:0/2030639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f07400b0f30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0730009840 tx=0x7f0730009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.145+0000 7f07454b4700 1 -- 172.21.15.181:0/2030639333 shutdown_connections 2023-10-16T11:21:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.145+0000 7f07454b4700 1 --2- 172.21.15.181:0/2030639333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f07400b0f30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.145+0000 7f07454b4700 1 -- 172.21.15.181:0/2030639333 >> 172.21.15.181:0/2030639333 conn(0x7f074006c430 msgr2=0x7f0740099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:38.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.145+0000 7f07454b4700 1 -- 172.21.15.181:0/2030639333 shutdown_connections 2023-10-16T11:21:38.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.145+0000 7f07454b4700 1 -- 172.21.15.181:0/2030639333 wait complete. 2023-10-16T11:21:38.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.146+0000 7f07454b4700 1 Processor -- start 2023-10-16T11:21:38.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.146+0000 7f07454b4700 1 -- start start 2023-10-16T11:21:38.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07454b4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07400cf1a0 0x7f0740153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07454b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400d0d60 0x7f07400c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07454b4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f074006e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07454b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400aebd0 con 0x7f07400d0d60 2023-10-16T11:21:38.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07454b4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0740154210 con 0x7f074015a7d0 2023-10-16T11:21:38.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07454b4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07400c5990 con 0x7f07400cf1a0 2023-10-16T11:21:38.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400d0d60 0x7f07400c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400d0d60 0x7f07400c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60566/0 (socket says 172.21.15.181:60566) 2023-10-16T11:21:38.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073e7fc700 1 -- 172.21.15.181:0/4069262174 learned_addr learned my addr 172.21.15.181:0/4069262174 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:38.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073f7fe700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f074006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073effd700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07400cf1a0 0x7f0740153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.155 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073f7fe700 1 -- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07400cf1a0 msgr2=0x7f0740153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073f7fe700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07400cf1a0 0x7f0740153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073f7fe700 1 -- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400d0d60 msgr2=0x7f07400c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073f7fe700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400d0d60 0x7f07400c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073f7fe700 1 -- 172.21.15.181:0/4069262174 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300094a0 con 0x7f074015a7d0 2023-10-16T11:21:38.156 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f073f7fe700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f074006e720 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07400d04e0 tx=0x7f073800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:38.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07277fe700 1 -- 172.21.15.181:0/4069262174 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738003e30 con 0x7f074015a7d0 2023-10-16T11:21:38.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.147+0000 7f07277fe700 1 -- 172.21.15.181:0/4069262174 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07380050d0 con 0x7f074015a7d0 2023-10-16T11:21:38.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.148+0000 7f07277fe700 1 -- 172.21.15.181:0/4069262174 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738005480 con 0x7f074015a7d0 2023-10-16T11:21:38.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.148+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07400ae580 con 0x7f074015a7d0 2023-10-16T11:21:38.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.148+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074006f510 con 0x7f074015a7d0 2023-10-16T11:21:38.157 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.149+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07400aed80 con 0x7f074015a7d0 2023-10-16T11:21:38.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.149+0000 7f07277fe700 1 -- 172.21.15.181:0/4069262174 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f07380055e0 con 0x7f074015a7d0 2023-10-16T11:21:38.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.149+0000 7f07277fe700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 0x7f072c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.149+0000 7f073effd700 1 -- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 msgr2=0x7f072c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:38.158 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.149+0000 7f07277fe700 1 -- 172.21.15.181:0/4069262174 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f0738089020 con 0x7f074015a7d0 2023-10-16T11:21:38.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.149+0000 7f073effd700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 0x7f072c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:38.159 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.154+0000 7f07277fe700 1 -- 172.21.15.181:0/4069262174 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f07400aed80 con 0x7f074015a7d0 2023-10-16T11:21:38.233 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:38.231+0000 7f43536ed200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:21:38.284 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:38.283+0000 7f0c7f014200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:21:38.347 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:38.345+0000 7f43536ed200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:21:38.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.350+0000 7f073effd700 1 -- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 msgr2=0x7f072c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:38.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.350+0000 7f073effd700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 0x7f072c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:38.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.358+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07400aed40 con 0x7f074015a7d0 2023-10-16T11:21:38.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.358+0000 7f07277fe700 1 -- 172.21.15.181:0/4069262174 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v257) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07380170a0 con 0x7f074015a7d0 2023-10-16T11:21:38.359 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:21:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 msgr2=0x7f072c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 0x7f072c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 msgr2=0x7f074006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f074006e720 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07400d04e0 tx=0x7f073800ca10 comp rx=0 tx=0).stop 2023-10-16T11:21:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 shutdown_connections 2023-10-16T11:21:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f072c06b020 0x7f072c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07400cf1a0 0x7f0740153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f074015a7d0 0x7f074006e720 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 --2- 172.21.15.181:0/4069262174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07400d0d60 0x7f07400c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 >> 172.21.15.181:0/4069262174 conn(0x7f074006c430 msgr2=0x7f07400b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 shutdown_connections 2023-10-16T11:21:38.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.361+0000 7f07454b4700 1 -- 172.21.15.181:0/4069262174 wait complete. 2023-10-16T11:21:38.376 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:21:38.406 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:38.405+0000 7f0c7f014200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:21:38.421 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:38.419+0000 7f43536ed200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:21:38.483 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:38.482+0000 7f0c7f014200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:21:38.513 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:38.499+0000 7f43536ed200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:21:38.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.502+0000 7f3dec5fe700 1 Processor -- start 2023-10-16T11:21:38.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.503+0000 7f3dec5fe700 1 -- start start 2023-10-16T11:21:38.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dec5fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3de415f150 0x7f3de40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dec5fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de40b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dec5fe700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3de40b14a0 0x7f3de40b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dec5fe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40ce970 con 0x7f3de415f150 2023-10-16T11:21:38.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dec5fe700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40b2ae0 con 0x7f3de40b14a0 2023-10-16T11:21:38.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dec5fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40b2dc0 con 0x7f3de40ce4f0 2023-10-16T11:21:38.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dea39a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3de415f150 0x7f3de40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dea39a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3de415f150 0x7f3de40cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60574/0 (socket says 172.21.15.181:60574) 2023-10-16T11:21:38.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dea39a700 1 -- 172.21.15.181:0/822174141 learned_addr learned my addr 172.21.15.181:0/822174141 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:38.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3deab9b700 1 --2- 172.21.15.181:0/822174141 >> v2:172.21.15.190:3300/0 conn(0x7f3de40b14a0 0x7f3de40b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3de9b99700 1 --2- 172.21.15.181:0/822174141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de40b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dea39a700 1 -- 172.21.15.181:0/822174141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 msgr2=0x7f3de40b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:38.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dea39a700 1 --2- 172.21.15.181:0/822174141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de40b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dea39a700 1 -- 172.21.15.181:0/822174141 >> v2:172.21.15.190:3300/0 conn(0x7f3de40b14a0 msgr2=0x7f3de40b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:38.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.504+0000 7f3dea39a700 1 --2- 172.21.15.181:0/822174141 >> v2:172.21.15.190:3300/0 conn(0x7f3de40b14a0 0x7f3de40b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.505+0000 7f3dea39a700 1 -- 172.21.15.181:0/822174141 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40cf9d0 con 0x7f3de415f150 2023-10-16T11:21:38.521 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.505+0000 7f3deab9b700 1 --2- 172.21.15.181:0/822174141 >> v2:172.21.15.190:3300/0 conn(0x7f3de40b14a0 0x7f3de40b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:38.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.505+0000 7f3dea39a700 1 --2- 172.21.15.181:0/822174141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de40cdfb0 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7f3de40ceb70 tx=0x7f3de0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7494c39b531add47 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:38.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.505+0000 7f3de9398700 1 -- 172.21.15.181:0/822174141 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0011830 con 0x7f3de415f150 2023-10-16T11:21:38.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.505+0000 7f3de9398700 1 -- 172.21.15.181:0/822174141 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de0011990 con 0x7f3de415f150 2023-10-16T11:21:38.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.505+0000 7f3de9398700 1 -- 172.21.15.181:0/822174141 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0011d20 con 0x7f3de415f150 2023-10-16T11:21:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 -- 172.21.15.181:0/822174141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 msgr2=0x7f3de40cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/822174141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de40cdfb0 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7f3de40ceb70 tx=0x7f3de0009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 -- 172.21.15.181:0/822174141 shutdown_connections 2023-10-16T11:21:38.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/822174141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de40cdfb0 unknown :-1 s=CLOSED pgs=3023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/822174141 >> v2:172.21.15.190:3300/0 conn(0x7f3de40b14a0 0x7f3de40b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/822174141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de40b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 -- 172.21.15.181:0/822174141 >> 172.21.15.181:0/822174141 conn(0x7f3de409a930 msgr2=0x7f3de40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:38.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 -- 172.21.15.181:0/822174141 shutdown_connections 2023-10-16T11:21:38.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.506+0000 7f3dec5fe700 1 -- 172.21.15.181:0/822174141 wait complete. 2023-10-16T11:21:38.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.508+0000 7f3dec5fe700 1 Processor -- start 2023-10-16T11:21:38.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.508+0000 7f3dec5fe700 1 -- start start 2023-10-16T11:21:38.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3dec5fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3de40b14a0 0x7f3de409b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3dec5fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de409cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3dec5fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de41f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3dec5fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c8fd0 con 0x7f3de415f150 2023-10-16T11:21:38.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3dec5fe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c92b0 con 0x7f3de40b14a0 2023-10-16T11:21:38.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3dec5fe700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c9590 con 0x7f3de40ce4f0 2023-10-16T11:21:38.526 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3deab9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de41f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3deab9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de41f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60584/0 (socket says 172.21.15.181:60584) 2023-10-16T11:21:38.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3deab9b700 1 -- 172.21.15.181:0/380314517 learned_addr learned my addr 172.21.15.181:0/380314517 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:38.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3de9b99700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de409cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3dea39a700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3de40b14a0 0x7f3de409b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3deab9b700 1 -- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 msgr2=0x7f3de409cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.509+0000 7f3deab9b700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de409cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3deab9b700 1 -- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3de40b14a0 msgr2=0x7f3de409b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3deab9b700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3de40b14a0 0x7f3de409b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3deab9b700 1 -- 172.21.15.181:0/380314517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0009550 con 0x7f3de415f150 2023-10-16T11:21:38.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3deab9b700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de41f4440 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc007e10 tx=0x7f3ddc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:38.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3ddaffd700 1 -- 172.21.15.181:0/380314517 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00dd20 con 0x7f3de415f150 2023-10-16T11:21:38.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3ddaffd700 1 -- 172.21.15.181:0/380314517 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc00f040 con 0x7f3de415f150 2023-10-16T11:21:38.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de40c9870 con 0x7f3de415f150 2023-10-16T11:21:38.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3ddaffd700 1 -- 172.21.15.181:0/380314517 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc003c60 con 0x7f3de415f150 2023-10-16T11:21:38.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.510+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de415a340 con 0x7f3de415f150 2023-10-16T11:21:38.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.511+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de40c9d90 con 0x7f3de415f150 2023-10-16T11:21:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.512+0000 7f3ddaffd700 1 -- 172.21.15.181:0/380314517 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3ddc004030 con 0x7f3de415f150 2023-10-16T11:21:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.512+0000 7f3ddaffd700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 0x7f3dc806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.512+0000 7f3dea39a700 1 -- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 msgr2=0x7f3dc806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.512+0000 7f3ddaffd700 1 -- 172.21.15.181:0/380314517 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f3ddc089c70 con 0x7f3de415f150 2023-10-16T11:21:38.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.512+0000 7f3dea39a700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 0x7f3dc806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:38.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.516+0000 7f3ddaffd700 1 -- 172.21.15.181:0/380314517 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3ddc054720 con 0x7f3de415f150 2023-10-16T11:21:38.570 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:38.570+0000 7f0c7f014200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:21:38.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.713+0000 7f3dea39a700 1 -- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 msgr2=0x7f3dc806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:38.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.713+0000 7f3dea39a700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 0x7f3dc806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:38.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.716+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3de4067d20 con 0x7f3de415f150 2023-10-16T11:21:38.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.716+0000 7f3ddaffd700 1 -- 172.21.15.181:0/380314517 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v257) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ddc01a020 con 0x7f3de415f150 2023-10-16T11:21:38.718 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:21:38.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 msgr2=0x7f3dc806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:38.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 0x7f3dc806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 msgr2=0x7f3de41f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de41f4440 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc007e10 tx=0x7f3ddc00d3b0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 shutdown_connections 2023-10-16T11:21:38.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f3dc806b020 0x7f3dc806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de415f150 0x7f3de41f4440 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3de40b14a0 0x7f3de409b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 --2- 172.21.15.181:0/380314517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3de40ce4f0 0x7f3de409cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 >> 172.21.15.181:0/380314517 conn(0x7f3de409a930 msgr2=0x7f3de40ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:38.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 shutdown_connections 2023-10-16T11:21:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.719+0000 7f3dec5fe700 1 -- 172.21.15.181:0/380314517 wait complete. 2023-10-16T11:21:38.733 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:21:38.869 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:38.868+0000 7f43536ed200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:21:38.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.899+0000 7f282bcd1700 1 Processor -- start 2023-10-16T11:21:38.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.900+0000 7f282bcd1700 1 -- start start 2023-10-16T11:21:38.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f282bcd1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28240b2010 0x7f28240cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.903 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f282bcd1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f28240ce680 0x7f28240ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f282bcd1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28240ce1f0 con 0x7f2824071be0 2023-10-16T11:21:38.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f282bcd1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2824072090 con 0x7f28240b2010 2023-10-16T11:21:38.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f282bcd1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28240ccda0 con 0x7f28240ce680 2023-10-16T11:21:38.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f2829a6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28240b2010 0x7f28240cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f2829a6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f28240b2010 0x7f28240cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60616/0 (socket says 172.21.15.181:60616) 2023-10-16T11:21:38.904 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f2829a6d700 1 -- 172.21.15.181:0/2695336298 learned_addr learned my addr 172.21.15.181:0/2695336298 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:38.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f282a26e700 1 --2- 172.21.15.181:0/2695336298 >> v2:172.21.15.190:3300/0 conn(0x7f28240ce680 0x7f28240ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f2829a6d700 1 -- 172.21.15.181:0/2695336298 >> v2:172.21.15.190:3300/0 conn(0x7f28240ce680 msgr2=0x7f28240ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:38.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f2829a6d700 1 --2- 172.21.15.181:0/2695336298 >> v2:172.21.15.190:3300/0 conn(0x7f28240ce680 0x7f28240ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f2829a6d700 1 -- 172.21.15.181:0/2695336298 >> v1:172.21.15.190:6789/0 conn(0x7f2824071be0 legacy=0x7f28240b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.905 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.901+0000 7f2829a6d700 1 -- 172.21.15.181:0/2695336298 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28240727e0 con 0x7f28240b2010 2023-10-16T11:21:38.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.902+0000 7f2829a6d700 1 --2- 172.21.15.181:0/2695336298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f28240cc6e0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f2824071a30 tx=0x7f2820009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aed7d632fee770 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:38.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.902+0000 7f2828a6b700 1 -- 172.21.15.181:0/2695336298 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282000b870 con 0x7f28240b2010 2023-10-16T11:21:38.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.902+0000 7f2828a6b700 1 -- 172.21.15.181:0/2695336298 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282000b9d0 con 0x7f28240b2010 2023-10-16T11:21:38.906 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.902+0000 7f2828a6b700 1 -- 172.21.15.181:0/2695336298 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282000bd60 con 0x7f28240b2010 2023-10-16T11:21:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 -- 172.21.15.181:0/2695336298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 msgr2=0x7f28240cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2695336298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f28240cc6e0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f2824071a30 tx=0x7f2820009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 -- 172.21.15.181:0/2695336298 shutdown_connections 2023-10-16T11:21:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2695336298 >> v2:172.21.15.190:3300/0 conn(0x7f28240ce680 0x7f28240ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.910 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2695336298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f28240cc6e0 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 -- 172.21.15.181:0/2695336298 >> 172.21.15.181:0/2695336298 conn(0x7f28240cd010 msgr2=0x7f28240cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 -- 172.21.15.181:0/2695336298 shutdown_connections 2023-10-16T11:21:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.903+0000 7f282bcd1700 1 -- 172.21.15.181:0/2695336298 wait complete. 2023-10-16T11:21:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.905+0000 7f282bcd1700 1 Processor -- start 2023-10-16T11:21:38.911 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.905+0000 7f282bcd1700 1 -- start start 2023-10-16T11:21:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282bcd1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2824071be0 0x7f28241491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282bcd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f2824148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282bcd1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28240ce680 0x7f282415d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282bcd1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282415ba60 con 0x7f28240b2010 2023-10-16T11:21:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282bcd1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282415bd40 con 0x7f28240ce680 2023-10-16T11:21:38.912 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282bcd1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28240c8f80 con 0x7f2824071be0 2023-10-16T11:21:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282926c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f2824148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282926c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f2824148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60624/0 (socket says 172.21.15.181:60624) 2023-10-16T11:21:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282926c700 1 -- 172.21.15.181:0/2782674059 learned_addr learned my addr 172.21.15.181:0/2782674059 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282a26e700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28240ce680 0x7f282415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f2829a6d700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2824071be0 0x7f28241491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.906+0000 7f282926c700 1 -- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2824071be0 msgr2=0x7f28241491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f282926c700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2824071be0 0x7f28241491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f282926c700 1 -- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28240ce680 msgr2=0x7f282415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f282926c700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28240ce680 0x7f282415d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:38.914 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f282926c700 1 -- 172.21.15.181:0/2782674059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820009550 con 0x7f28240b2010 2023-10-16T11:21:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f282926c700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f2824148920 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f281400e710 tx=0x7f281400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f281a7fc700 1 -- 172.21.15.181:0/2782674059 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140052d0 con 0x7f28240b2010 2023-10-16T11:21:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f281a7fc700 1 -- 172.21.15.181:0/2782674059 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2814005430 con 0x7f28240b2010 2023-10-16T11:21:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28240c9260 con 0x7f28240b2010 2023-10-16T11:21:38.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f281a7fc700 1 -- 172.21.15.181:0/2782674059 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140108e0 con 0x7f28240b2010 2023-10-16T11:21:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.907+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2824148e60 con 0x7f28240b2010 2023-10-16T11:21:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.908+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282415bf60 con 0x7f28240b2010 2023-10-16T11:21:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.908+0000 7f281a7fc700 1 -- 172.21.15.181:0/2782674059 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2814010a40 con 0x7f28240b2010 2023-10-16T11:21:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.909+0000 7f281a7fc700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 0x7f280806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:38.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.909+0000 7f2829a6d700 1 -- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 msgr2=0x7f280806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.909+0000 7f281a7fc700 1 -- 172.21.15.181:0/2782674059 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f281408baf0 con 0x7f28240b2010 2023-10-16T11:21:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.909+0000 7f2829a6d700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 0x7f280806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:38.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:38.913+0000 7f281a7fc700 1 -- 172.21.15.181:0/2782674059 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f282415bf60 con 0x7f28240b2010 2023-10-16T11:21:38.960 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:38.960+0000 7f0c7f014200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:21:39.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.110+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f282415bf00 con 0x7f28240b2010 2023-10-16T11:21:39.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.110+0000 7f2829a6d700 1 -- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 msgr2=0x7f280806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:39.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.110+0000 7f2829a6d700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 0x7f280806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:39.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.110+0000 7f281a7fc700 1 -- 172.21.15.181:0/2782674059 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v257) v1 ==== 100+0+11 (secure 0 0 0) 0x7f281401f020 con 0x7f28240b2010 2023-10-16T11:21:39.112 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:21:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 msgr2=0x7f280806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 0x7f280806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 msgr2=0x7f2824148920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f2824148920 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f281400e710 tx=0x7f281400ea20 comp rx=0 tx=0).stop 2023-10-16T11:21:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 shutdown_connections 2023-10-16T11:21:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f280806b020 0x7f280806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2824071be0 0x7f28241491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28240ce680 0x7f282415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 --2- 172.21.15.181:0/2782674059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28240b2010 0x7f2824148920 unknown :-1 s=CLOSED pgs=3026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 >> 172.21.15.181:0/2782674059 conn(0x7f28240cd010 msgr2=0x7f2824159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:39.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 shutdown_connections 2023-10-16T11:21:39.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.112+0000 7f282bcd1700 1 -- 172.21.15.181:0/2782674059 wait complete. 2023-10-16T11:21:39.126 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:21:39.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.248+0000 7f58f86d4700 1 Processor -- start 2023-10-16T11:21:39.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.248+0000 7f58f86d4700 1 -- start start 2023-10-16T11:21:39.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f86d4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f86d4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00b0410 con 0x7f58f00d1e80 2023-10-16T11:21:39.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f86d4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00b2ca0 con 0x7f58f00d2300 2023-10-16T11:21:39.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f86d4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00b3c00 con 0x7f58f015dd50 2023-10-16T11:21:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f5c6f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f58f00d1e80 0x7f58f00ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43478/0 (socket says 172.21.15.181:43478) 2023-10-16T11:21:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f5c6f700 1 -- 172.21.15.181:0/402931747 learned_addr learned my addr 172.21.15.181:0/402931747 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f6470700 1 --2- 172.21.15.181:0/402931747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f546e700 1 -- 172.21.15.181:0/402931747 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487837347 0 0) 0x7f58f00b0410 con 0x7f58f00d1e80 2023-10-16T11:21:39.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f546e700 1 -- 172.21.15.181:0/402931747 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58dc0035b0 con 0x7f58f00d1e80 2023-10-16T11:21:39.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f546e700 1 -- 172.21.15.181:0/402931747 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742927408 0 0) 0x7f58f00b2ca0 con 0x7f58f00d2300 2023-10-16T11:21:39.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f546e700 1 -- 172.21.15.181:0/402931747 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f00b0410 con 0x7f58f00d2300 2023-10-16T11:21:39.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f6470700 1 -- 172.21.15.181:0/402931747 >> v1:172.21.15.190:6789/0 conn(0x7f58f00d2300 legacy=0x7f58f00ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f6470700 1 -- 172.21.15.181:0/402931747 >> v1:172.21.15.181:6789/0 conn(0x7f58f00d1e80 legacy=0x7f58f00ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.250+0000 7f58f6470700 1 -- 172.21.15.181:0/402931747 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00b2f80 con 0x7f58f015dd50 2023-10-16T11:21:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.251+0000 7f58f6470700 1 --2- 172.21.15.181:0/402931747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00d41c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009850 tx=0x7f58ec009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b791aaf71a9a99e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:39.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.251+0000 7f58f546e700 1 -- 172.21.15.181:0/402931747 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec005e30 con 0x7f58f015dd50 2023-10-16T11:21:39.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.251+0000 7f58f546e700 1 -- 172.21.15.181:0/402931747 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec006030 con 0x7f58f015dd50 2023-10-16T11:21:39.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.251+0000 7f58f546e700 1 -- 172.21.15.181:0/402931747 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0063e0 con 0x7f58f015dd50 2023-10-16T11:21:39.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.251+0000 7f58f86d4700 1 -- 172.21.15.181:0/402931747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 msgr2=0x7f58f00d41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.251+0000 7f58f86d4700 1 --2- 172.21.15.181:0/402931747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00d41c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009850 tx=0x7f58ec009b60 comp rx=0 tx=0).stop 2023-10-16T11:21:39.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.252+0000 7f58f86d4700 1 -- 172.21.15.181:0/402931747 shutdown_connections 2023-10-16T11:21:39.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.252+0000 7f58f86d4700 1 --2- 172.21.15.181:0/402931747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00d41c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.252+0000 7f58f86d4700 1 -- 172.21.15.181:0/402931747 >> 172.21.15.181:0/402931747 conn(0x7f58f008af90 msgr2=0x7f58f00d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:39.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.252+0000 7f58f86d4700 1 -- 172.21.15.181:0/402931747 shutdown_connections 2023-10-16T11:21:39.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.252+0000 7f58f86d4700 1 -- 172.21.15.181:0/402931747 wait complete. 2023-10-16T11:21:39.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.253+0000 7f58f86d4700 1 Processor -- start 2023-10-16T11:21:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.254+0000 7f58f86d4700 1 -- start start 2023-10-16T11:21:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f86d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00d1e80 0x7f58f015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f86d4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f00d2300 0x7f58f00cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f86d4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f86d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00cc240 con 0x7f58f00d1e80 2023-10-16T11:21:39.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f86d4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00cc520 con 0x7f58f00d2300 2023-10-16T11:21:39.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f86d4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00cc800 con 0x7f58f015dd50 2023-10-16T11:21:39.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00d1e80 0x7f58f015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00d1e80 0x7f58f015c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60636/0 (socket says 172.21.15.181:60636) 2023-10-16T11:21:39.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 -- 172.21.15.181:0/2058340308 learned_addr learned my addr 172.21.15.181:0/2058340308 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:39.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f5c6f700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f00d2300 0x7f58f00cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6c71700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 -- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 msgr2=0x7f58f00b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 -- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f00d2300 msgr2=0x7f58f00cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f00d2300 0x7f58f00cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 -- 172.21.15.181:0/2058340308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec009430 con 0x7f58f00d1e80 2023-10-16T11:21:39.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.255+0000 7f58f6470700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00d1e80 0x7f58f015c050 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7f58ec000c00 tx=0x7f58ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:39.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.256+0000 7f58e6ffd700 1 -- 172.21.15.181:0/2058340308 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec006790 con 0x7f58f00d1e80 2023-10-16T11:21:39.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.256+0000 7f58e6ffd700 1 -- 172.21.15.181:0/2058340308 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec020070 con 0x7f58f00d1e80 2023-10-16T11:21:39.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.256+0000 7f58e6ffd700 1 -- 172.21.15.181:0/2058340308 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec025670 con 0x7f58f00d1e80 2023-10-16T11:21:39.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.256+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f00ccae0 con 0x7f58f00d1e80 2023-10-16T11:21:39.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.256+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f00b94b0 con 0x7f58f00d1e80 2023-10-16T11:21:39.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.257+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f00cc380 con 0x7f58f00d1e80 2023-10-16T11:21:39.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.257+0000 7f58e6ffd700 1 -- 172.21.15.181:0/2058340308 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f58ec022030 con 0x7f58f00d1e80 2023-10-16T11:21:39.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.258+0000 7f58e6ffd700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 0x7f58d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.258+0000 7f58f5c6f700 1 -- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 msgr2=0x7f58d406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:39.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.258+0000 7f58e6ffd700 1 -- 172.21.15.181:0/2058340308 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f58ec08ae20 con 0x7f58f00d1e80 2023-10-16T11:21:39.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.258+0000 7f58f5c6f700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 0x7f58d406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:39.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.262+0000 7f58e6ffd700 1 -- 172.21.15.181:0/2058340308 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f58f00cc380 con 0x7f58f00d1e80 2023-10-16T11:21:39.438 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:39.435+0000 7f43536ed200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:21:39.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.459+0000 7f58f5c6f700 1 -- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 msgr2=0x7f58d406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:39.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.459+0000 7f58f5c6f700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 0x7f58d406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:39.507 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:39.505+0000 7f43536ed200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:21:39.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.513+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f58f00cc380 con 0x7f58f00d1e80 2023-10-16T11:21:39.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.513+0000 7f58e6ffd700 1 -- 172.21.15.181:0/2058340308 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v257) v1 ==== 81+0+594 (secure 0 0 0) 0x7f58f00cc380 con 0x7f58f00d1e80 2023-10-16T11:21:39.515 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3637292004 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.515 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3476229854 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.515 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/1366795220 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.515 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2569889467 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.516 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/3850286714 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.516 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/3850286714 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.516 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2946054209 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.516 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/649394230 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.516 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/507502116 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.517 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3665147670 2023-10-17T11:21:34.384612+0000 2023-10-16T11:21:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 msgr2=0x7f58d406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 0x7f58d406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00d1e80 msgr2=0x7f58f015c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00d1e80 0x7f58f015c050 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7f58ec000c00 tx=0x7f58ec004750 comp rx=0 tx=0).stop 2023-10-16T11:21:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 shutdown_connections 2023-10-16T11:21:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f58d406b020 0x7f58d406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58f015dd50 0x7f58f00b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00d1e80 0x7f58f015c050 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 --2- 172.21.15.181:0/2058340308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f00d2300 0x7f58f00cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 >> 172.21.15.181:0/2058340308 conn(0x7f58f008af90 msgr2=0x7f58f00b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:39.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 shutdown_connections 2023-10-16T11:21:39.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.516+0000 7f58f86d4700 1 -- 172.21.15.181:0/2058340308 wait complete. 2023-10-16T11:21:39.520 INFO:teuthology.orchestra.run.smithi181.stderr:listed 10 entries 2023-10-16T11:21:39.531 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:21:39.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:39.531 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:21:39.553 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:39.553+0000 7f0c7f014200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:21:39.572 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:21:39.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:21:39.573 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:21:39.628 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:39.627+0000 7f0c7f014200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:21:39.629 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:21:39.629 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:39.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:21:39.650 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:39.651 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:39.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:21:39.687 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:21:39.687 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:39.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:21:39.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:39.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.850+0000 7f4ca175b700 1 Processor -- start 2023-10-16T11:21:39.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.850+0000 7f4ca175b700 1 -- start start 2023-10-16T11:21:39.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4ca175b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c9c15b990 0x7f4c9c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4ca175b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4ca175b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cfc60 0x7f4c9c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4ca175b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c0d0630 con 0x7f4c9c15b990 2023-10-16T11:21:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4ca175b700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c0d24f0 con 0x7f4c9c0cfc60 2023-10-16T11:21:39.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4ca175b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0cc510 con 0x7f4c9c0cd2d0 2023-10-16T11:21:39.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4c9affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c9c15b990 0x7f4c9c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4c9affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c9c15b990 0x7f4c9c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60646/0 (socket says 172.21.15.181:60646) 2023-10-16T11:21:39.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4c9affd700 1 -- 172.21.15.181:0/499199690 learned_addr learned my addr 172.21.15.181:0/499199690 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:39.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4c9a7fc700 1 --2- 172.21.15.181:0/499199690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.851+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/499199690 >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cfc60 0x7f4c9c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c9affd700 1 -- 172.21.15.181:0/499199690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 msgr2=0x7f4c9c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:39.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c9affd700 1 --2- 172.21.15.181:0/499199690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c9affd700 1 -- 172.21.15.181:0/499199690 >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cfc60 msgr2=0x7f4c9c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c9affd700 1 --2- 172.21.15.181:0/499199690 >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cfc60 0x7f4c9c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c9affd700 1 -- 172.21.15.181:0/499199690 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0cc7f0 con 0x7f4c9c15b990 2023-10-16T11:21:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c9affd700 1 --2- 172.21.15.181:0/499199690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0ce8f0 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0cbc60 tx=0x7f4c90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1acc7ee066e8e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c99ffb700 1 -- 172.21.15.181:0/499199690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90005e30 con 0x7f4c9c15b990 2023-10-16T11:21:39.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c99ffb700 1 -- 172.21.15.181:0/499199690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90004030 con 0x7f4c9c15b990 2023-10-16T11:21:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.852+0000 7f4c99ffb700 1 -- 172.21.15.181:0/499199690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900043e0 con 0x7f4c9c15b990 2023-10-16T11:21:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 -- 172.21.15.181:0/499199690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 msgr2=0x7f4c9c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 --2- 172.21.15.181:0/499199690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0ce8f0 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0cbc60 tx=0x7f4c90009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 -- 172.21.15.181:0/499199690 shutdown_connections 2023-10-16T11:21:39.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 --2- 172.21.15.181:0/499199690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0ce8f0 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 --2- 172.21.15.181:0/499199690 >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cfc60 0x7f4c9c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 --2- 172.21.15.181:0/499199690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 -- 172.21.15.181:0/499199690 >> 172.21.15.181:0/499199690 conn(0x7f4c9c09a9a0 msgr2=0x7f4c9c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:39.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 -- 172.21.15.181:0/499199690 shutdown_connections 2023-10-16T11:21:39.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.853+0000 7f4ca175b700 1 -- 172.21.15.181:0/499199690 wait complete. 2023-10-16T11:21:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.854+0000 7f4ca175b700 1 Processor -- start 2023-10-16T11:21:39.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.854+0000 7f4ca175b700 1 -- start start 2023-10-16T11:21:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.855+0000 7f4ca175b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c06c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.855+0000 7f4ca175b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c9c0cfc60 0x7f4c9c1f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.855+0000 7f4ca175b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.864 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.855+0000 7f4ca175b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b42a0 con 0x7f4c9c15b990 2023-10-16T11:21:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.855+0000 7f4ca175b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b4e20 con 0x7f4c9c0cfc60 2023-10-16T11:21:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.855+0000 7f4ca175b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0b44f0 con 0x7f4c9c0cd2d0 2023-10-16T11:21:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.855+0000 7f4c9b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60656/0 (socket says 172.21.15.181:60656) 2023-10-16T11:21:39.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/2246391344 learned_addr learned my addr 172.21.15.181:0/2246391344 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:39.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9affd700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9a7fc700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c9c0cfc60 0x7f4c9c1f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:39.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 msgr2=0x7f4c9c06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c06c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c9c0cfc60 msgr2=0x7f4c9c1f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c9c0cfc60 0x7f4c9c1f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/2246391344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90009500 con 0x7f4c9c15b990 2023-10-16T11:21:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0af2e0 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400efc0 tx=0x7f4c9400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c837fe700 1 -- 172.21.15.181:0/2246391344 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9400cad0 con 0x7f4c9c15b990 2023-10-16T11:21:39.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c837fe700 1 -- 172.21.15.181:0/2246391344 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9400f040 con 0x7f4c9c15b990 2023-10-16T11:21:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4c837fe700 1 -- 172.21.15.181:0/2246391344 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9400ce70 con 0x7f4c9c15b990 2023-10-16T11:21:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c0b47d0 con 0x7f4c9c15b990 2023-10-16T11:21:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.856+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c06cdd0 con 0x7f4c9c15b990 2023-10-16T11:21:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.857+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c074460 con 0x7f4c9c15b990 2023-10-16T11:21:39.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.858+0000 7f4c837fe700 1 -- 172.21.15.181:0/2246391344 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4c94014480 con 0x7f4c9c15b990 2023-10-16T11:21:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.858+0000 7f4c837fe700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 0x7f4c8806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.858+0000 7f4c9affd700 1 -- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 msgr2=0x7f4c8806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.858+0000 7f4c837fe700 1 -- 172.21.15.181:0/2246391344 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f4c9408a550 con 0x7f4c9c15b990 2023-10-16T11:21:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.858+0000 7f4c9affd700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 0x7f4c8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:39.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:39.862+0000 7f4c837fe700 1 -- 172.21.15.181:0/2246391344 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c94055090 con 0x7f4c9c15b990 2023-10-16T11:21:40.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.059+0000 7f4c9affd700 1 -- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 msgr2=0x7f4c8806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:40.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.059+0000 7f4c9affd700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 0x7f4c8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:40.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.108+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c9c0b4f90 con 0x7f4c9c15b990 2023-10-16T11:21:40.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.108+0000 7f4c837fe700 1 -- 172.21.15.181:0/2246391344 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+4941 (secure 0 0 0) 0x7f4c940586b0 con 0x7f4c9c15b990 2023-10-16T11:21:40.109 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:40.109 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3049755410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3049755410},{"type":"v1","addr":"172.21.15.190:6834","nonce":3049755410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2185474337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2185474337},{"type":"v1","addr":"172.21.15.181:6837","nonce":2185474337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2846112501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2846112501},{"type":"v1","addr":"172.21.15.181:6836","nonce":2846112501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:20.526852+0000","modified":"2023-10-16T11:21:23.550181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12930":{"gid":12930,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3258038365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3258038365},{"type":"v1","addr":"172.21.15.190:6835","nonce":3258038365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:21:40.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 msgr2=0x7f4c8806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:40.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 0x7f4c8806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 msgr2=0x7f4c9c0af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0af2e0 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400efc0 tx=0x7f4c9400c0c0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 shutdown_connections 2023-10-16T11:21:40.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4c8806b020 0x7f4c8806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd2d0 0x7f4c9c06c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c15b990 0x7f4c9c0af2e0 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 --2- 172.21.15.181:0/2246391344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c9c0cfc60 0x7f4c9c1f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.110+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 >> 172.21.15.181:0/2246391344 conn(0x7f4c9c09a9a0 msgr2=0x7f4c9c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:40.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.111+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 shutdown_connections 2023-10-16T11:21:40.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.111+0000 7f4ca175b700 1 -- 172.21.15.181:0/2246391344 wait complete. 2023-10-16T11:21:40.114 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 144 2023-10-16T11:21:40.124 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:40.144 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:40.142+0000 7f43536ed200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:40.223 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:40.220+0000 7f43536ed200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:21:40.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.252+0000 7fc377f81700 1 Processor -- start 2023-10-16T11:21:40.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.252+0000 7fc377f81700 1 -- start start 2023-10-16T11:21:40.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc377f81700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc37015e6d0 0x7fc3700d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc377f81700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37006ed00 con 0x7fc3700cb0c0 2023-10-16T11:21:40.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc377f81700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3700d02d0 con 0x7fc37015eab0 2023-10-16T11:21:40.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc377f81700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37006e0f0 con 0x7fc37015e6d0 2023-10-16T11:21:40.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc37651e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc3700cb0c0 0x7fc37015a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43512/0 (socket says 172.21.15.181:43512) 2023-10-16T11:21:40.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc37651e700 1 -- 172.21.15.181:0/4182157840 learned_addr learned my addr 172.21.15.181:0/4182157840 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:40.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc375d1d700 1 --2- 172.21.15.181:0/4182157840 >> v2:172.21.15.190:3300/0 conn(0x7fc37015e6d0 0x7fc3700d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc374d1b700 1 -- 172.21.15.181:0/4182157840 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011150599 0 0) 0x7fc37006ed00 con 0x7fc3700cb0c0 2023-10-16T11:21:40.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc374d1b700 1 -- 172.21.15.181:0/4182157840 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc35c0035b0 con 0x7fc3700cb0c0 2023-10-16T11:21:40.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc375d1d700 1 -- 172.21.15.181:0/4182157840 >> v1:172.21.15.190:6789/0 conn(0x7fc37015eab0 legacy=0x7fc37015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc375d1d700 1 -- 172.21.15.181:0/4182157840 >> v1:172.21.15.181:6789/0 conn(0x7fc3700cb0c0 legacy=0x7fc37015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc375d1d700 1 -- 172.21.15.181:0/4182157840 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37006e3d0 con 0x7fc37015e6d0 2023-10-16T11:21:40.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.253+0000 7fc375d1d700 1 --2- 172.21.15.181:0/4182157840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 0x7fc3700d0fc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009820 tx=0x7fc36c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30813f0ba89fd0e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:40.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc374d1b700 1 -- 172.21.15.181:0/4182157840 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c005e30 con 0x7fc37015e6d0 2023-10-16T11:21:40.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc374d1b700 1 -- 172.21.15.181:0/4182157840 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c006030 con 0x7fc37015e6d0 2023-10-16T11:21:40.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc374d1b700 1 -- 172.21.15.181:0/4182157840 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0063a0 con 0x7fc37015e6d0 2023-10-16T11:21:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc377f81700 1 -- 172.21.15.181:0/4182157840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 msgr2=0x7fc3700d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc377f81700 1 --2- 172.21.15.181:0/4182157840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 0x7fc3700d0fc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009820 tx=0x7fc36c009b30 comp rx=0 tx=0).stop 2023-10-16T11:21:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc377f81700 1 -- 172.21.15.181:0/4182157840 shutdown_connections 2023-10-16T11:21:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc377f81700 1 --2- 172.21.15.181:0/4182157840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 0x7fc3700d0fc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc377f81700 1 -- 172.21.15.181:0/4182157840 >> 172.21.15.181:0/4182157840 conn(0x7fc3700d04a0 msgr2=0x7fc3700d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc377f81700 1 -- 172.21.15.181:0/4182157840 shutdown_connections 2023-10-16T11:21:40.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.254+0000 7fc377f81700 1 -- 172.21.15.181:0/4182157840 wait complete. 2023-10-16T11:21:40.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 Processor -- start 2023-10-16T11:21:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 -- start start 2023-10-16T11:21:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3700cb0c0 0x7fc37015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 0x7fc37006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37015eab0 0x7fc37006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37015cf20 con 0x7fc37015eab0 2023-10-16T11:21:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700c76a0 con 0x7fc37015e6d0 2023-10-16T11:21:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc377f81700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700c7980 con 0x7fc3700cb0c0 2023-10-16T11:21:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc37651e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37015eab0 0x7fc37006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc37651e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37015eab0 0x7fc37006ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60672/0 (socket says 172.21.15.181:60672) 2023-10-16T11:21:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc37651e700 1 -- 172.21.15.181:0/207061142 learned_addr learned my addr 172.21.15.181:0/207061142 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:40.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.256+0000 7fc37551c700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 0x7fc37006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc375d1d700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3700cb0c0 0x7fc37015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc37651e700 1 -- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3700cb0c0 msgr2=0x7fc37015d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc37651e700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3700cb0c0 0x7fc37015d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc37651e700 1 -- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 msgr2=0x7fc37006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc37651e700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 0x7fc37006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc37651e700 1 -- 172.21.15.181:0/207061142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c009430 con 0x7fc37015eab0 2023-10-16T11:21:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc37651e700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37015eab0 0x7fc37006ade0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7fc36800caa0 tx=0x7fc36800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc3667fc700 1 -- 172.21.15.181:0/207061142 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36800a9e0 con 0x7fc37015eab0 2023-10-16T11:21:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc3667fc700 1 -- 172.21.15.181:0/207061142 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36800ab40 con 0x7fc37015eab0 2023-10-16T11:21:40.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3700c7c60 con 0x7fc37015eab0 2023-10-16T11:21:40.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc3667fc700 1 -- 172.21.15.181:0/207061142 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368012400 con 0x7fc37015eab0 2023-10-16T11:21:40.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.257+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3700c8220 con 0x7fc37015eab0 2023-10-16T11:21:40.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.258+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3700c8440 con 0x7fc37015eab0 2023-10-16T11:21:40.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.258+0000 7fc3667fc700 1 -- 172.21.15.181:0/207061142 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fc36801b3b0 con 0x7fc37015eab0 2023-10-16T11:21:40.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.259+0000 7fc3667fc700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 0x7fc35406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.259+0000 7fc375d1d700 1 -- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 msgr2=0x7fc35406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.259+0000 7fc3667fc700 1 -- 172.21.15.181:0/207061142 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fc3680896a0 con 0x7fc37015eab0 2023-10-16T11:21:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.259+0000 7fc375d1d700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 0x7fc35406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:40.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.263+0000 7fc3667fc700 1 -- 172.21.15.181:0/207061142 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc3680543b0 con 0x7fc37015eab0 2023-10-16T11:21:40.302 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:40.301+0000 7f0c7f014200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:40.368 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:40.365+0000 7f43536ed200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:40.383 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:40.383+0000 7f0c7f014200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:21:40.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.460+0000 7fc375d1d700 1 -- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 msgr2=0x7fc35406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:40.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.460+0000 7fc375d1d700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 0x7fc35406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:40.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.464+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc370067d20 con 0x7fc37015eab0 2023-10-16T11:21:40.466 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.465+0000 7fc3667fc700 1 -- 172.21.15.181:0/207061142 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+18047 (secure 0 0 0) 0x7fc3680579d0 con 0x7fc37015eab0 2023-10-16T11:21:40.467 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:40.467 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":257,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:34.384677+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:18.964173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:19.485121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3637292004":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3476229854":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/1366795220":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2569889467":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6833/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6832/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2946054209":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/649394230":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/507502116":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3665147670":"2023-10-17T11:21:34.384612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:40.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.467+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 msgr2=0x7fc35406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:40.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.467+0000 7fc377f81700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 0x7fc35406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.467+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37015eab0 msgr2=0x7fc37006ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37015eab0 0x7fc37006ade0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7fc36800caa0 tx=0x7fc36800cdb0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 shutdown_connections 2023-10-16T11:21:40.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fc35406b020 0x7fc35406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3700cb0c0 0x7fc37015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37015eab0 0x7fc37006ade0 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 --2- 172.21.15.181:0/207061142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc37015e6d0 0x7fc37006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 >> 172.21.15.181:0/207061142 conn(0x7fc3700d04a0 msgr2=0x7fc3700ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 shutdown_connections 2023-10-16T11:21:40.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.468+0000 7fc377f81700 1 -- 172.21.15.181:0/207061142 wait complete. 2023-10-16T11:21:40.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:21:18.964173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '255', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:21:19.485121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '256', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:21:40.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:21:40.541 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:40.540+0000 7f0c7f014200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:21:40.596 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:40.595+0000 7f43536ed200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:21:40.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.608+0000 7f2bc43f0700 1 Processor -- start 2023-10-16T11:21:40.609 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.608+0000 7f2bc43f0700 1 -- start start 2023-10-16T11:21:40.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc43f0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2bbc15f150 0x7f2bbc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc43f0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0ce4f0 0x7f2bbc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc43f0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0b2ae0 con 0x7f2bbc0b14a0 2023-10-16T11:21:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc43f0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc0ce970 con 0x7f2bbc15f150 2023-10-16T11:21:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc43f0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0b2dc0 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc218c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2bbc15f150 0x7f2bbc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc218c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2bbc15f150 0x7f2bbc0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60706/0 (socket says 172.21.15.181:60706) 2023-10-16T11:21:40.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc218c700 1 -- 172.21.15.181:0/3129696867 learned_addr learned my addr 172.21.15.181:0/3129696867 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.609+0000 7f2bc198b700 1 --2- 172.21.15.181:0/3129696867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0ce4f0 0x7f2bbc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc218c700 1 -- 172.21.15.181:0/3129696867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0ce4f0 msgr2=0x7f2bbc0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc218c700 1 --2- 172.21.15.181:0/3129696867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0ce4f0 0x7f2bbc0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc218c700 1 -- 172.21.15.181:0/3129696867 >> v1:172.21.15.181:6789/0 conn(0x7f2bbc0b14a0 legacy=0x7f2bbc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc218c700 1 -- 172.21.15.181:0/3129696867 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0cf9d0 con 0x7f2bbc15f150 2023-10-16T11:21:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc218c700 1 --2- 172.21.15.181:0/3129696867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc15f150 0x7f2bbc0cdfb0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80098f0 tx=0x7f2bb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95de67d7216f6e6b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc118a700 1 -- 172.21.15.181:0/3129696867 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8011830 con 0x7f2bbc15f150 2023-10-16T11:21:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc118a700 1 -- 172.21.15.181:0/3129696867 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8011990 con 0x7f2bbc15f150 2023-10-16T11:21:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc118a700 1 -- 172.21.15.181:0/3129696867 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8011d20 con 0x7f2bbc15f150 2023-10-16T11:21:40.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc43f0700 1 -- 172.21.15.181:0/3129696867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc15f150 msgr2=0x7f2bbc0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.610+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/3129696867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc15f150 0x7f2bbc0cdfb0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80098f0 tx=0x7f2bb8009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.611+0000 7f2bc43f0700 1 -- 172.21.15.181:0/3129696867 shutdown_connections 2023-10-16T11:21:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.611+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/3129696867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc15f150 0x7f2bbc0cdfb0 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.611+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/3129696867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0ce4f0 0x7f2bbc0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.611+0000 7f2bc43f0700 1 -- 172.21.15.181:0/3129696867 >> 172.21.15.181:0/3129696867 conn(0x7f2bbc09a930 msgr2=0x7f2bbc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:40.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.611+0000 7f2bc43f0700 1 -- 172.21.15.181:0/3129696867 shutdown_connections 2023-10-16T11:21:40.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.611+0000 7f2bc43f0700 1 -- 172.21.15.181:0/3129696867 wait complete. 2023-10-16T11:21:40.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.612+0000 7f2bc43f0700 1 Processor -- start 2023-10-16T11:21:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.612+0000 7f2bc43f0700 1 -- start start 2023-10-16T11:21:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0b14a0 0x7f2bbc06f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc0ce4f0 0x7f2bbc153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bbc15f150 0x7f2bbc0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc1f3550 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0c7b80 con 0x7f2bbc15f150 2023-10-16T11:21:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0c6f70 con 0x7f2bbc0b14a0 2023-10-16T11:21:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc0ce4f0 0x7f2bbc153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc0ce4f0 0x7f2bbc153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60714/0 (socket says 172.21.15.181:60714) 2023-10-16T11:21:40.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 -- 172.21.15.181:0/1063127226 learned_addr learned my addr 172.21.15.181:0/1063127226 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc218c700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0b14a0 0x7f2bbc06f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc298d700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bbc15f150 0x7f2bbc0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 -- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0b14a0 msgr2=0x7f2bbc06f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0b14a0 0x7f2bbc06f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.622 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 -- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bbc15f150 msgr2=0x7f2bbc0c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bbc15f150 0x7f2bbc0c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 -- 172.21.15.181:0/1063127226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8009550 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc198b700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc0ce4f0 0x7f2bbc153a30 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9b0 tx=0x7f2bac00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bb2ffd700 1 -- 172.21.15.181:0/1063127226 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac009710 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.623 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bb2ffd700 1 -- 172.21.15.181:0/1063127226 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac00ee50 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bb2ffd700 1 -- 172.21.15.181:0/1063127226 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00ca50 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc0c7250 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.613+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc0c7810 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.614+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc067d20 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.616+0000 7f2bb2ffd700 1 -- 172.21.15.181:0/1063127226 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2bac00cbb0 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.616+0000 7f2bb2ffd700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 0x7f2ba006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.616+0000 7f2bc218c700 1 -- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 msgr2=0x7f2ba006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.616+0000 7f2bb2ffd700 1 -- 172.21.15.181:0/1063127226 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f2bac089a10 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.616+0000 7f2bc218c700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 0x7f2ba006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:40.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.621+0000 7f2bb2ffd700 1 -- 172.21.15.181:0/1063127226 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2bac054660 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.671 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:40.670+0000 7f43536ed200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:21:40.782 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:40.781+0000 7f0c7f014200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:21:40.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.817+0000 7f2bc218c700 1 -- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 msgr2=0x7f2ba006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:40.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.817+0000 7f2bc218c700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 0x7f2ba006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:40.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.821+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bbc0cbac0 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.822 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.821+0000 7f2bb2ffd700 1 -- 172.21.15.181:0/1063127226 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v257) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2bac00ce60 con 0x7f2bbc0ce4f0 2023-10-16T11:21:40.822 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:21:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.823+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 msgr2=0x7f2ba006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.823+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 0x7f2ba006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc0ce4f0 msgr2=0x7f2bbc153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc0ce4f0 0x7f2bbc153a30 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9b0 tx=0x7f2bac00ecc0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 shutdown_connections 2023-10-16T11:21:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f2ba006b020 0x7f2ba006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc0b14a0 0x7f2bbc06f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.826 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bbc15f150 0x7f2bbc0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 --2- 172.21.15.181:0/1063127226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bbc0ce4f0 0x7f2bbc153a30 unknown :-1 s=CLOSED pgs=3035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 >> 172.21.15.181:0/1063127226 conn(0x7f2bbc09a930 msgr2=0x7f2bbc0ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:40.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 shutdown_connections 2023-10-16T11:21:40.827 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.824+0000 7f2bc43f0700 1 -- 172.21.15.181:0/1063127226 wait complete. 2023-10-16T11:21:40.837 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:21:40.863 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:40.863+0000 7f0c7f014200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:21:40.938 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:40.937+0000 7f43536ed200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:21:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.966+0000 7fab828b5700 1 Processor -- start 2023-10-16T11:21:40.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.966+0000 7fab828b5700 1 -- start start 2023-10-16T11:21:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.967+0000 7fab828b5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab7c1646b0 0x7fab7c166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.967+0000 7fab828b5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0b1f70 con 0x7fab7c15f150 2023-10-16T11:21:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.967+0000 7fab828b5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c0b2ae0 con 0x7fab7c0ce6c0 2023-10-16T11:21:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.967+0000 7fab828b5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7c0cd880 con 0x7fab7c1646b0 2023-10-16T11:21:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fab7c15f150 0x7fab7c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43568/0 (socket says 172.21.15.181:43568) 2023-10-16T11:21:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7bfff700 1 -- 172.21.15.181:0/42203130 learned_addr learned my addr 172.21.15.181:0/42203130 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab80e52700 1 --2- 172.21.15.181:0/42203130 >> v2:172.21.15.190:3300/0 conn(0x7fab7c1646b0 0x7fab7c166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7affd700 1 -- 172.21.15.181:0/42203130 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383042687 0 0) 0x7fab7c0b1f70 con 0x7fab7c15f150 2023-10-16T11:21:40.971 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7affd700 1 -- 172.21.15.181:0/42203130 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab640035b0 con 0x7fab7c15f150 2023-10-16T11:21:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab80e52700 1 -- 172.21.15.181:0/42203130 >> v1:172.21.15.190:6789/0 conn(0x7fab7c0ce6c0 legacy=0x7fab7c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7affd700 1 -- 172.21.15.181:0/42203130 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663145541 0 0) 0x7fab7c0b2ae0 con 0x7fab7c0ce6c0 2023-10-16T11:21:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab80e52700 1 -- 172.21.15.181:0/42203130 >> v1:172.21.15.181:6789/0 conn(0x7fab7c15f150 legacy=0x7fab7c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab80e52700 1 -- 172.21.15.181:0/42203130 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0b2dc0 con 0x7fab7c1646b0 2023-10-16T11:21:40.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab80e52700 1 --2- 172.21.15.181:0/42203130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c166aa0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fab74009d90 tx=0x7fab7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed4357898b466fc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7affd700 1 -- 172.21.15.181:0/42203130 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7400e070 con 0x7fab7c1646b0 2023-10-16T11:21:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7affd700 1 -- 172.21.15.181:0/42203130 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7400bb20 con 0x7fab7c1646b0 2023-10-16T11:21:40.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.968+0000 7fab7affd700 1 -- 172.21.15.181:0/42203130 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7400bed0 con 0x7fab7c1646b0 2023-10-16T11:21:40.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.969+0000 7fab828b5700 1 -- 172.21.15.181:0/42203130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 msgr2=0x7fab7c166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.969+0000 7fab828b5700 1 --2- 172.21.15.181:0/42203130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c166aa0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fab74009d90 tx=0x7fab7400b330 comp rx=0 tx=0).stop 2023-10-16T11:21:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.969+0000 7fab828b5700 1 -- 172.21.15.181:0/42203130 shutdown_connections 2023-10-16T11:21:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.969+0000 7fab828b5700 1 --2- 172.21.15.181:0/42203130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c166aa0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.969+0000 7fab828b5700 1 -- 172.21.15.181:0/42203130 >> 172.21.15.181:0/42203130 conn(0x7fab7c09a930 msgr2=0x7fab7c0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.969+0000 7fab828b5700 1 -- 172.21.15.181:0/42203130 shutdown_connections 2023-10-16T11:21:40.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.969+0000 7fab828b5700 1 -- 172.21.15.181:0/42203130 wait complete. 2023-10-16T11:21:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.970+0000 7fab828b5700 1 Processor -- start 2023-10-16T11:21:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.971+0000 7fab828b5700 1 -- start start 2023-10-16T11:21:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.971+0000 7fab828b5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab7c0ce6c0 0x7fab7c154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.971+0000 7fab828b5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab7c15f150 0x7fab7c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.971+0000 7fab828b5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.971+0000 7fab828b5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c155110 con 0x7fab7c0ce6c0 2023-10-16T11:21:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.971+0000 7fab828b5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c06c3c0 con 0x7fab7c1646b0 2023-10-16T11:21:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.971+0000 7fab828b5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c06d420 con 0x7fab7c15f150 2023-10-16T11:21:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab7bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab7c0ce6c0 0x7fab7c154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:37550/0 (socket says 172.21.15.181:37550) 2023-10-16T11:21:40.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab7b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab7c15f150 0x7fab7c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 -- 172.21.15.181:0/134892817 learned_addr learned my addr 172.21.15.181:0/134892817 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 -- 172.21.15.181:0/134892817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab7c15f150 msgr2=0x7fab7c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab7c15f150 0x7fab7c0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 -- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab7c0ce6c0 msgr2=0x7fab7c154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:40.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab7c0ce6c0 0x7fab7c154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:40.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 -- 172.21.15.181:0/134892817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74009a70 con 0x7fab7c1646b0 2023-10-16T11:21:40.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab7bfff700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab7c0ce6c0 0x7fab7c154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:40.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab80e52700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c0b1890 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fab74014040 tx=0x7fab74004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:40.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab78ff9700 1 -- 172.21.15.181:0/134892817 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7401b810 con 0x7fab7c1646b0 2023-10-16T11:21:40.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab78ff9700 1 -- 172.21.15.181:0/134892817 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab74020070 con 0x7fab7c1646b0 2023-10-16T11:21:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab78ff9700 1 -- 172.21.15.181:0/134892817 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7401bbb0 con 0x7fab7c1646b0 2023-10-16T11:21:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c06c660 con 0x7fab7c1646b0 2023-10-16T11:21:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.972+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c06cc20 con 0x7fab7c1646b0 2023-10-16T11:21:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.973+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7c067d20 con 0x7fab7c1646b0 2023-10-16T11:21:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.974+0000 7fab78ff9700 1 -- 172.21.15.181:0/134892817 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fab7401bd10 con 0x7fab7c1646b0 2023-10-16T11:21:40.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.974+0000 7fab78ff9700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 0x7fab6806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.974+0000 7fab7bfff700 1 -- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 msgr2=0x7fab6806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.974+0000 7fab78ff9700 1 -- 172.21.15.181:0/134892817 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fab74022070 con 0x7fab7c1646b0 2023-10-16T11:21:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.974+0000 7fab7bfff700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 0x7fab6806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:40.983 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:40.979+0000 7fab78ff9700 1 -- 172.21.15.181:0/134892817 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fab740566d0 con 0x7fab7c1646b0 2023-10-16T11:21:41.142 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:41.141+0000 7f0c7f014200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:21:41.159 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:41.158+0000 7f43536ed200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:21:41.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.175+0000 7fab7bfff700 1 -- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 msgr2=0x7fab6806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:41.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.175+0000 7fab7bfff700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 0x7fab6806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:41.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.177+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab7c0cff60 con 0x7fab7c1646b0 2023-10-16T11:21:41.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.177+0000 7fab78ff9700 1 -- 172.21.15.181:0/134892817 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v257) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab7402c0a0 con 0x7fab7c1646b0 2023-10-16T11:21:41.178 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:21:41.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 msgr2=0x7fab6806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:41.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 0x7fab6806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 msgr2=0x7fab7c0b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c0b1890 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fab74014040 tx=0x7fab74004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 shutdown_connections 2023-10-16T11:21:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fab6806b020 0x7fab6806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab7c1646b0 0x7fab7c0b1890 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab7c0ce6c0 0x7fab7c154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 --2- 172.21.15.181:0/134892817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab7c15f150 0x7fab7c0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 >> 172.21.15.181:0/134892817 conn(0x7fab7c09a930 msgr2=0x7fab7c15e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 shutdown_connections 2023-10-16T11:21:41.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.180+0000 7fab828b5700 1 -- 172.21.15.181:0/134892817 wait complete. 2023-10-16T11:21:41.193 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:21:41.310 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:41.308+0000 7f43536ed200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:21:41.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.367+0000 7f4cf7249700 1 Processor -- start 2023-10-16T11:21:41.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.367+0000 7f4cf7249700 1 -- start start 2023-10-16T11:21:41.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.368+0000 7f4cf7249700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4cf00b2010 0x7f4cf00cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.368+0000 7f4cf7249700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4cf00ce680 0x7f4cf00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.368+0000 7f4cf7249700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf00ce1f0 con 0x7f4cf0071be0 2023-10-16T11:21:41.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf7249700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf0072090 con 0x7f4cf00b2010 2023-10-16T11:21:41.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf7249700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf00ccda0 con 0x7f4cf00ce680 2023-10-16T11:21:41.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4cf00b2010 0x7f4cf00cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4cf00b2010 0x7f4cf00cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60738/0 (socket says 172.21.15.181:60738) 2023-10-16T11:21:41.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3275273133 learned_addr learned my addr 172.21.15.181:0/3275273133 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:41.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf57e6700 1 --2- 172.21.15.181:0/3275273133 >> v2:172.21.15.190:3300/0 conn(0x7f4cf00ce680 0x7f4cf00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3275273133 >> v2:172.21.15.190:3300/0 conn(0x7f4cf00ce680 msgr2=0x7f4cf00ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:41.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 --2- 172.21.15.181:0/3275273133 >> v2:172.21.15.190:3300/0 conn(0x7f4cf00ce680 0x7f4cf00ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3275273133 >> v1:172.21.15.190:6789/0 conn(0x7f4cf0071be0 legacy=0x7f4cf00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3275273133 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00727e0 con 0x7f4cf00b2010 2023-10-16T11:21:41.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cf4fe5700 1 --2- 172.21.15.181:0/3275273133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf00b2010 0x7f4cf00cc6e0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0071a30 tx=0x7f4ce4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac724e67d72ca024 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:41.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cef7fe700 1 -- 172.21.15.181:0/3275273133 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011830 con 0x7f4cf00b2010 2023-10-16T11:21:41.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cef7fe700 1 -- 172.21.15.181:0/3275273133 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4011990 con 0x7f4cf00b2010 2023-10-16T11:21:41.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.369+0000 7f4cef7fe700 1 -- 172.21.15.181:0/3275273133 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011d20 con 0x7f4cf00b2010 2023-10-16T11:21:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 -- 172.21.15.181:0/3275273133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf00b2010 msgr2=0x7f4cf00cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3275273133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf00b2010 0x7f4cf00cc6e0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0071a30 tx=0x7f4ce4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 -- 172.21.15.181:0/3275273133 shutdown_connections 2023-10-16T11:21:41.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3275273133 >> v2:172.21.15.190:3300/0 conn(0x7f4cf00ce680 0x7f4cf00ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3275273133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf00b2010 0x7f4cf00cc6e0 unknown :-1 s=CLOSED pgs=3037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 -- 172.21.15.181:0/3275273133 >> 172.21.15.181:0/3275273133 conn(0x7f4cf00cd010 msgr2=0x7f4cf00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:41.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 -- 172.21.15.181:0/3275273133 shutdown_connections 2023-10-16T11:21:41.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.370+0000 7f4cf7249700 1 -- 172.21.15.181:0/3275273133 wait complete. 2023-10-16T11:21:41.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.372+0000 7f4cf7249700 1 Processor -- start 2023-10-16T11:21:41.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.372+0000 7f4cf7249700 1 -- start start 2023-10-16T11:21:41.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0071be0 0x7f4cf00b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cf00b2010 0x7f4cf0150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cf00ce680 0x7f4cf0150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00ada10 con 0x7f4cf0071be0 2023-10-16T11:21:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00ac9b0 con 0x7f4cf00b2010 2023-10-16T11:21:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00acc50 con 0x7f4cf00ce680 2023-10-16T11:21:41.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0071be0 0x7f4cf00b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0071be0 0x7f4cf00b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60744/0 (socket says 172.21.15.181:60744) 2023-10-16T11:21:41.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3490825667 learned_addr learned my addr 172.21.15.181:0/3490825667 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:41.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4ceffff700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cf00b2010 0x7f4cf0150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf57e6700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cf00ce680 0x7f4cf0150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cf00ce680 msgr2=0x7f4cf0150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cf00ce680 0x7f4cf0150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cf00b2010 msgr2=0x7f4cf0150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cf00b2010 0x7f4cf0150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 -- 172.21.15.181:0/3490825667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce4009550 con 0x7f4cf0071be0 2023-10-16T11:21:41.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf4fe5700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0071be0 0x7f4cf00b09b0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009c10 tx=0x7f4ce4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:41.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4ced7fa700 1 -- 172.21.15.181:0/3490825667 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4004510 con 0x7f4cf0071be0 2023-10-16T11:21:41.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4ced7fa700 1 -- 172.21.15.181:0/3490825667 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce401e070 con 0x7f4cf0071be0 2023-10-16T11:21:41.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4ced7fa700 1 -- 172.21.15.181:0/3490825667 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40038e0 con 0x7f4cf0071be0 2023-10-16T11:21:41.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf00acf30 con 0x7f4cf0071be0 2023-10-16T11:21:41.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.374+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf00ad4f0 con 0x7f4cf0071be0 2023-10-16T11:21:41.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.376+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf00b6510 con 0x7f4cf0071be0 2023-10-16T11:21:41.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.376+0000 7f4ced7fa700 1 -- 172.21.15.181:0/3490825667 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4ce4003a40 con 0x7f4cf0071be0 2023-10-16T11:21:41.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.376+0000 7f4ced7fa700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 0x7f4cd406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.376+0000 7f4ceffff700 1 -- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 msgr2=0x7f4cd406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:41.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.376+0000 7f4ced7fa700 1 -- 172.21.15.181:0/3490825667 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f4ce408aa30 con 0x7f4cf0071be0 2023-10-16T11:21:41.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.376+0000 7f4ceffff700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 0x7f4cd406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:41.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.381+0000 7f4ced7fa700 1 -- 172.21.15.181:0/3490825667 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4ce4055600 con 0x7f4cf0071be0 2023-10-16T11:21:41.393 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:41.393+0000 7f0c7f014200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:21:41.424 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:41.423+0000 7f43536ed200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:21:41.553 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:41.552+0000 7f0c7f014200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:21:41.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.577+0000 7f4ceffff700 1 -- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 msgr2=0x7f4cd406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:41.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.577+0000 7f4ceffff700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 0x7f4cd406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:41.580 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:41.579+0000 7f43536ed200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:21:41.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.588+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cf00b4240 con 0x7f4cf0071be0 2023-10-16T11:21:41.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.588+0000 7f4ced7fa700 1 -- 172.21.15.181:0/3490825667 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v257) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ce4058c20 con 0x7f4cf0071be0 2023-10-16T11:21:41.590 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:21:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 msgr2=0x7f4cd406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 0x7f4cd406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0071be0 msgr2=0x7f4cf00b09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0071be0 0x7f4cf00b09b0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009c10 tx=0x7f4ce4004750 comp rx=0 tx=0).stop 2023-10-16T11:21:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 shutdown_connections 2023-10-16T11:21:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f4cd406b020 0x7f4cd406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cf00ce680 0x7f4cf0150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cf0071be0 0x7f4cf00b09b0 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 --2- 172.21.15.181:0/3490825667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cf00b2010 0x7f4cf0150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 >> 172.21.15.181:0/3490825667 conn(0x7f4cf00cd010 msgr2=0x7f4cf00b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:41.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 shutdown_connections 2023-10-16T11:21:41.594 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.591+0000 7f4cf7249700 1 -- 172.21.15.181:0/3490825667 wait complete. 2023-10-16T11:21:41.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:41.676 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:41.675+0000 7f0c7f014200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:21:41.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.727+0000 7fcdd65cf700 1 Processor -- start 2023-10-16T11:21:41.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.727+0000 7fcdd65cf700 1 -- start start 2023-10-16T11:21:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdd65cf700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcdd015eab0 0x7fcdd00d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdd65cf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd006ed00 con 0x7fcdd015e6d0 2023-10-16T11:21:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdd65cf700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd00d02d0 con 0x7fcdd00cb0c0 2023-10-16T11:21:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdd65cf700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd006e0f0 con 0x7fcdd015eab0 2023-10-16T11:21:41.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdcffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcdd015e6d0 0x7fcdd015a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43600/0 (socket says 172.21.15.181:43600) 2023-10-16T11:21:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdcffff700 1 -- 172.21.15.181:0/1250030771 learned_addr learned my addr 172.21.15.181:0/1250030771 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdcf7fe700 1 --2- 172.21.15.181:0/1250030771 >> v2:172.21.15.190:3300/0 conn(0x7fcdd015eab0 0x7fcdd00d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdceffd700 1 -- 172.21.15.181:0/1250030771 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904914872 0 0) 0x7fcdd006ed00 con 0x7fcdd015e6d0 2023-10-16T11:21:41.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.728+0000 7fcdceffd700 1 -- 172.21.15.181:0/1250030771 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdbc0035b0 con 0x7fcdd015e6d0 2023-10-16T11:21:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdceffd700 1 -- 172.21.15.181:0/1250030771 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885355605 0 0) 0x7fcdd00d02d0 con 0x7fcdd00cb0c0 2023-10-16T11:21:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdceffd700 1 -- 172.21.15.181:0/1250030771 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd006ed00 con 0x7fcdd00cb0c0 2023-10-16T11:21:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdcf7fe700 1 -- 172.21.15.181:0/1250030771 >> v1:172.21.15.190:6789/0 conn(0x7fcdd00cb0c0 legacy=0x7fcdd015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdcf7fe700 1 -- 172.21.15.181:0/1250030771 >> v1:172.21.15.181:6789/0 conn(0x7fcdd015e6d0 legacy=0x7fcdd015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdcf7fe700 1 -- 172.21.15.181:0/1250030771 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd006e3d0 con 0x7fcdd015eab0 2023-10-16T11:21:41.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdcf7fe700 1 --2- 172.21.15.181:0/1250030771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd015eab0 0x7fcdd00d0fc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009840 tx=0x7fcdb8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb61f177490d107 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:41.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdceffd700 1 -- 172.21.15.181:0/1250030771 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8011830 con 0x7fcdd015eab0 2023-10-16T11:21:41.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdceffd700 1 -- 172.21.15.181:0/1250030771 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8011990 con 0x7fcdd015eab0 2023-10-16T11:21:41.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.729+0000 7fcdceffd700 1 -- 172.21.15.181:0/1250030771 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8011d20 con 0x7fcdd015eab0 2023-10-16T11:21:41.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.730+0000 7fcdd65cf700 1 -- 172.21.15.181:0/1250030771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd015eab0 msgr2=0x7fcdd00d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.730+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/1250030771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd015eab0 0x7fcdd00d0fc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009840 tx=0x7fcdb8009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.730+0000 7fcdd65cf700 1 -- 172.21.15.181:0/1250030771 shutdown_connections 2023-10-16T11:21:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.730+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/1250030771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd015eab0 0x7fcdd00d0fc0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.730+0000 7fcdd65cf700 1 -- 172.21.15.181:0/1250030771 >> 172.21.15.181:0/1250030771 conn(0x7fcdd00d04a0 msgr2=0x7fcdd00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:41.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.730+0000 7fcdd65cf700 1 -- 172.21.15.181:0/1250030771 shutdown_connections 2023-10-16T11:21:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.730+0000 7fcdd65cf700 1 -- 172.21.15.181:0/1250030771 wait complete. 2023-10-16T11:21:41.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.731+0000 7fcdd65cf700 1 Processor -- start 2023-10-16T11:21:41.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.731+0000 7fcdd65cf700 1 -- start start 2023-10-16T11:21:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdd65cf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd00cb0c0 0x7fcdd014eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdd65cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd015e6d0 0x7fcdd014d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdd65cf700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd015eab0 0x7fcdd00c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdd65cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd014e710 con 0x7fcdd015e6d0 2023-10-16T11:21:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdd65cf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0151ed0 con 0x7fcdd00cb0c0 2023-10-16T11:21:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdd65cf700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd015aba0 con 0x7fcdd015eab0 2023-10-16T11:21:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd015e6d0 0x7fcdd014d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd015e6d0 0x7fcdd014d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60762/0 (socket says 172.21.15.181:60762) 2023-10-16T11:21:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 -- 172.21.15.181:0/2021917853 learned_addr learned my addr 172.21.15.181:0/2021917853 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:41.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcffff700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd00cb0c0 0x7fcdd014eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdd4b6c700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd015eab0 0x7fcdd00c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:41.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 -- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd015eab0 msgr2=0x7fcdd00c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd015eab0 0x7fcdd00c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 -- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd00cb0c0 msgr2=0x7fcdd014eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.741 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd00cb0c0 0x7fcdd014eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.732+0000 7fcdcf7fe700 1 -- 172.21.15.181:0/2021917853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb80094a0 con 0x7fcdd015e6d0 2023-10-16T11:21:41.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.733+0000 7fcdcf7fe700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd015e6d0 0x7fcdd014d6d0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009b90 tx=0x7fcdb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:41.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.733+0000 7fcdccff9700 1 -- 172.21.15.181:0/2021917853 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb801a6e0 con 0x7fcdd015e6d0 2023-10-16T11:21:41.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.733+0000 7fcdccff9700 1 -- 172.21.15.181:0/2021917853 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb801a840 con 0x7fcdd015e6d0 2023-10-16T11:21:41.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.733+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd014e0c0 con 0x7fcdd015e6d0 2023-10-16T11:21:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.733+0000 7fcdccff9700 1 -- 172.21.15.181:0/2021917853 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb801abf0 con 0x7fcdd015e6d0 2023-10-16T11:21:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.733+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd014f0e0 con 0x7fcdd015e6d0 2023-10-16T11:21:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.734+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd0067d20 con 0x7fcdd015e6d0 2023-10-16T11:21:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.734+0000 7fcdccff9700 1 -- 172.21.15.181:0/2021917853 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fcdb8022070 con 0x7fcdd015e6d0 2023-10-16T11:21:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.735+0000 7fcdccff9700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 0x7fcdb006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:41.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.735+0000 7fcdcffff700 1 -- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 msgr2=0x7fcdb006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:41.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.735+0000 7fcdccff9700 1 -- 172.21.15.181:0/2021917853 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fcdb808aa30 con 0x7fcdd015e6d0 2023-10-16T11:21:41.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.735+0000 7fcdcffff700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 0x7fcdb006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:41.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.740+0000 7fcdccff9700 1 -- 172.21.15.181:0/2021917853 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcdb8055680 con 0x7fcdd015e6d0 2023-10-16T11:21:41.843 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:41.842+0000 7f0c7f014200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:21:41.844 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:41.842+0000 7f43536ed200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:41.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.936+0000 7fcdcffff700 1 -- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 msgr2=0x7fcdb006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:41.937 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.936+0000 7fcdcffff700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 0x7fcdb006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:41.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.983+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdd014e850 con 0x7fcdd015e6d0 2023-10-16T11:21:41.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.983+0000 7fcdccff9700 1 -- 172.21.15.181:0/2021917853 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+4941 (secure 0 0 0) 0x7fcdb8058ca0 con 0x7fcdd015e6d0 2023-10-16T11:21:41.985 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:41.985 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3049755410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3049755410},{"type":"v1","addr":"172.21.15.190:6834","nonce":3049755410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2185474337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2185474337},{"type":"v1","addr":"172.21.15.181:6837","nonce":2185474337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2846112501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2846112501},{"type":"v1","addr":"172.21.15.181:6836","nonce":2846112501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:20.526852+0000","modified":"2023-10-16T11:21:23.550181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12930":{"gid":12930,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3258038365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3258038365},{"type":"v1","addr":"172.21.15.190:6835","nonce":3258038365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:21:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 msgr2=0x7fcdb006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 0x7fcdb006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd015e6d0 msgr2=0x7fcdd014d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd015e6d0 0x7fcdd014d6d0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009b90 tx=0x7fcdb8004750 comp rx=0 tx=0).stop 2023-10-16T11:21:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 shutdown_connections 2023-10-16T11:21:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7fcdb006b020 0x7fcdb006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdd015eab0 0x7fcdd00c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdd00cb0c0 0x7fcdd014eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 --2- 172.21.15.181:0/2021917853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd015e6d0 0x7fcdd014d6d0 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 >> 172.21.15.181:0/2021917853 conn(0x7fcdd00d04a0 msgr2=0x7fcdd0152b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 shutdown_connections 2023-10-16T11:21:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:41.986+0000 7fcdd65cf700 1 -- 172.21.15.181:0/2021917853 wait complete. 2023-10-16T11:21:41.989 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 144 2023-10-16T11:21:42.000 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:42.006 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:42.004+0000 7f43536ed200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:21:42.120 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.119+0000 7f0c7f014200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:21:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.122+0000 7f3581476700 1 Processor -- start 2023-10-16T11:21:42.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.122+0000 7f3581476700 1 -- start start 2023-10-16T11:21:42.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f3581476700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f357c15eab0 0x7f357c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f3581476700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f357c0cb0c0 0x7f357c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f3581476700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f357c06ed00 con 0x7f357c15e6d0 2023-10-16T11:21:42.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f3581476700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357c0d02d0 con 0x7f357c0cb0c0 2023-10-16T11:21:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f3581476700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357c15ad40 con 0x7f357c15eab0 2023-10-16T11:21:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f357b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f357c0cb0c0 0x7f357c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f357b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f357c0cb0c0 0x7f357c15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60774/0 (socket says 172.21.15.181:60774) 2023-10-16T11:21:42.127 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f357b7fe700 1 -- 172.21.15.181:0/4201893734 learned_addr learned my addr 172.21.15.181:0/4201893734 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f357affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f357c15e6d0 0x7f357c15a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43616/0 (socket says 172.21.15.181:43616) 2023-10-16T11:21:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.123+0000 7f357a7fc700 1 --2- 172.21.15.181:0/4201893734 >> v2:172.21.15.190:3300/0 conn(0x7f357c15eab0 0x7f357c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f357b7fe700 1 -- 172.21.15.181:0/4201893734 >> v2:172.21.15.190:3300/0 conn(0x7f357c15eab0 msgr2=0x7f357c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f357b7fe700 1 --2- 172.21.15.181:0/4201893734 >> v2:172.21.15.190:3300/0 conn(0x7f357c15eab0 0x7f357c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.128 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f357b7fe700 1 -- 172.21.15.181:0/4201893734 >> v1:172.21.15.181:6789/0 conn(0x7f357c15e6d0 legacy=0x7f357c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f357b7fe700 1 -- 172.21.15.181:0/4201893734 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c06e0f0 con 0x7f357c0cb0c0 2023-10-16T11:21:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f357b7fe700 1 --2- 172.21.15.181:0/4201893734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c0cb0c0 0x7f357c15a800 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f357c0714b0 tx=0x7f357400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769fcdcd1083262e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f3579ffb700 1 -- 172.21.15.181:0/4201893734 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400e070 con 0x7f357c0cb0c0 2023-10-16T11:21:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f3579ffb700 1 -- 172.21.15.181:0/4201893734 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357400b910 con 0x7f357c0cb0c0 2023-10-16T11:21:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f3579ffb700 1 -- 172.21.15.181:0/4201893734 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400bcc0 con 0x7f357c0cb0c0 2023-10-16T11:21:42.129 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f3581476700 1 -- 172.21.15.181:0/4201893734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c0cb0c0 msgr2=0x7f357c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.124+0000 7f3581476700 1 --2- 172.21.15.181:0/4201893734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c0cb0c0 0x7f357c15a800 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f357c0714b0 tx=0x7f357400b330 comp rx=0 tx=0).stop 2023-10-16T11:21:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.125+0000 7f3581476700 1 -- 172.21.15.181:0/4201893734 shutdown_connections 2023-10-16T11:21:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.125+0000 7f3581476700 1 --2- 172.21.15.181:0/4201893734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c0cb0c0 0x7f357c15a800 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.125+0000 7f3581476700 1 --2- 172.21.15.181:0/4201893734 >> v2:172.21.15.190:3300/0 conn(0x7f357c15eab0 0x7f357c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.130 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.125+0000 7f3581476700 1 -- 172.21.15.181:0/4201893734 >> 172.21.15.181:0/4201893734 conn(0x7f357c0d04a0 msgr2=0x7f357c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.125+0000 7f3581476700 1 -- 172.21.15.181:0/4201893734 shutdown_connections 2023-10-16T11:21:42.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.125+0000 7f3581476700 1 -- 172.21.15.181:0/4201893734 wait complete. 2023-10-16T11:21:42.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.126+0000 7f3581476700 1 Processor -- start 2023-10-16T11:21:42.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.126+0000 7f3581476700 1 -- start start 2023-10-16T11:21:42.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f3581476700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357c0cb0c0 0x7f357c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f3581476700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357c15e6d0 0x7f357c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f3581476700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c15eab0 0x7f357c0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f3581476700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c0b2d70 con 0x7f357c15eab0 2023-10-16T11:21:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f3581476700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c0b2a90 con 0x7f357c0cb0c0 2023-10-16T11:21:42.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f3581476700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357c157df0 con 0x7f357c15e6d0 2023-10-16T11:21:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c15eab0 0x7f357c0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c15eab0 0x7f357c0ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60780/0 (socket says 172.21.15.181:60780) 2023-10-16T11:21:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 -- 172.21.15.181:0/724496986 learned_addr learned my addr 172.21.15.181:0/724496986 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357a7fc700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357c15e6d0 0x7f357c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357affd700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357c0cb0c0 0x7f357c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 -- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357c15e6d0 msgr2=0x7f357c0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357c15e6d0 0x7f357c0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 -- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357c0cb0c0 msgr2=0x7f357c0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357c0cb0c0 0x7f357c0b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 -- 172.21.15.181:0/724496986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574009b30 con 0x7f357c15eab0 2023-10-16T11:21:42.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f357b7fe700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c15eab0 0x7f357c0ade20 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7f357400b370 tx=0x7f3574004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:42.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f35637fe700 1 -- 172.21.15.181:0/724496986 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574004880 con 0x7f357c15eab0 2023-10-16T11:21:42.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f35637fe700 1 -- 172.21.15.181:0/724496986 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35740099b0 con 0x7f357c15eab0 2023-10-16T11:21:42.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f35637fe700 1 -- 172.21.15.181:0/724496986 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357401b670 con 0x7f357c15eab0 2023-10-16T11:21:42.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.127+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357c1580d0 con 0x7f357c15eab0 2023-10-16T11:21:42.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.128+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c15a480 con 0x7f357c15eab0 2023-10-16T11:21:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.128+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357c157c50 con 0x7f357c15eab0 2023-10-16T11:21:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.129+0000 7f35637fe700 1 -- 172.21.15.181:0/724496986 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f357401b7d0 con 0x7f357c15eab0 2023-10-16T11:21:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.130+0000 7f35637fe700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 0x7f356806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.130+0000 7f357affd700 1 -- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 msgr2=0x7f356806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:42.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.130+0000 7f35637fe700 1 -- 172.21.15.181:0/724496986 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f357408b100 con 0x7f357c15eab0 2023-10-16T11:21:42.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.130+0000 7f357affd700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 0x7f356806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:42.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.134+0000 7f35637fe700 1 -- 172.21.15.181:0/724496986 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f357c157c50 con 0x7f357c15eab0 2023-10-16T11:21:42.292 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.291+0000 7f0c7f014200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:21:42.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.331+0000 7f357affd700 1 -- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 msgr2=0x7f356806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:42.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.331+0000 7f357affd700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 0x7f356806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:42.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.339+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f357c157c50 con 0x7f357c15eab0 2023-10-16T11:21:42.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.341+0000 7f35637fe700 1 -- 172.21.15.181:0/724496986 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+18047 (secure 0 0 0) 0x7f357c157c50 con 0x7f357c15eab0 2023-10-16T11:21:42.342 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:42.342 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":257,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:34.384677+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:18.964173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:19.485121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3637292004":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3476229854":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/1366795220":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2569889467":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6833/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6832/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2946054209":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/649394230":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/507502116":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3665147670":"2023-10-17T11:21:34.384612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:42.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 msgr2=0x7f356806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:42.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 0x7f356806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c15eab0 msgr2=0x7f357c0ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c15eab0 0x7f357c0ade20 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7f357400b370 tx=0x7f3574004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 shutdown_connections 2023-10-16T11:21:42.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f356806b020 0x7f356806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f357c15eab0 0x7f357c0ade20 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357c0cb0c0 0x7f357c0b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 --2- 172.21.15.181:0/724496986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357c15e6d0 0x7f357c0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 >> 172.21.15.181:0/724496986 conn(0x7f357c0d04a0 msgr2=0x7f357c0aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:42.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 shutdown_connections 2023-10-16T11:21:42.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.343+0000 7f3581476700 1 -- 172.21.15.181:0/724496986 wait complete. 2023-10-16T11:21:42.358 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:21:42.358 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:42.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.482+0000 7f8f1240f700 1 Processor -- start 2023-10-16T11:21:42.483 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.482+0000 7f8f1240f700 1 -- start start 2023-10-16T11:21:42.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f1240f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c1621c0 0x7f8f0c0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f1240f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f0c0cf690 0x7f8f0c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f1240f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0d1640 con 0x7f8f0c15d9b0 2023-10-16T11:21:42.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f1240f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0cfb10 con 0x7f8f0c0cf690 2023-10-16T11:21:42.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f1240f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0d1920 con 0x7f8f0c1621c0 2023-10-16T11:21:42.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f109ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f0c0cf690 0x7f8f0c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f109ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f0c0cf690 0x7f8f0c0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60790/0 (socket says 172.21.15.181:60790) 2023-10-16T11:21:42.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f0b7fe700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c1621c0 0x7f8f0c0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f109ac700 1 -- 172.21.15.181:0/722881043 learned_addr learned my addr 172.21.15.181:0/722881043 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:42.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f109ac700 1 -- 172.21.15.181:0/722881043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c1621c0 msgr2=0x7f8f0c0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:42.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f109ac700 1 --2- 172.21.15.181:0/722881043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c1621c0 0x7f8f0c0cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f0affd700 1 -- 172.21.15.181:0/722881043 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761911457 0 0) 0x7f8f0c0d1640 con 0x7f8f0c15d9b0 2023-10-16T11:21:42.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f109ac700 1 -- 172.21.15.181:0/722881043 >> v1:172.21.15.190:6789/0 conn(0x7f8f0c15d9b0 legacy=0x7f8f0c0b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.483+0000 7f8f109ac700 1 -- 172.21.15.181:0/722881043 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0d1c00 con 0x7f8f0c0cf690 2023-10-16T11:21:42.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f109ac700 1 --2- 172.21.15.181:0/722881043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c0cf690 0x7f8f0c0cbcd0 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009e50 tx=0x7f8f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9196872e1f43c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:42.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f0affd700 1 -- 172.21.15.181:0/722881043 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400e070 con 0x7f8f0c0cf690 2023-10-16T11:21:42.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f0affd700 1 -- 172.21.15.181:0/722881043 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0400b910 con 0x7f8f0c0cf690 2023-10-16T11:21:42.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f0affd700 1 -- 172.21.15.181:0/722881043 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400bcc0 con 0x7f8f0c0cf690 2023-10-16T11:21:42.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 -- 172.21.15.181:0/722881043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c0cf690 msgr2=0x7f8f0c0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 --2- 172.21.15.181:0/722881043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c0cf690 0x7f8f0c0cbcd0 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009e50 tx=0x7f8f0400b330 comp rx=0 tx=0).stop 2023-10-16T11:21:42.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 -- 172.21.15.181:0/722881043 shutdown_connections 2023-10-16T11:21:42.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 --2- 172.21.15.181:0/722881043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c0cf690 0x7f8f0c0cbcd0 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 --2- 172.21.15.181:0/722881043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c1621c0 0x7f8f0c0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 -- 172.21.15.181:0/722881043 >> 172.21.15.181:0/722881043 conn(0x7f8f0c0665a0 msgr2=0x7f8f0c0669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 -- 172.21.15.181:0/722881043 shutdown_connections 2023-10-16T11:21:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.484+0000 7f8f1240f700 1 -- 172.21.15.181:0/722881043 wait complete. 2023-10-16T11:21:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 Processor -- start 2023-10-16T11:21:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 -- start start 2023-10-16T11:21:42.496 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c0cf690 0x7f8f0c1f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0c15d9b0 0x7f8f0c160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c1621c0 0x7f8f0c0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c50d0 con 0x7f8f0c1621c0 2023-10-16T11:21:42.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c53b0 con 0x7f8f0c15d9b0 2023-10-16T11:21:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.486+0000 7f8f1240f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c5690 con 0x7f8f0c0cf690 2023-10-16T11:21:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c1621c0 0x7f8f0c0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c1621c0 0x7f8f0c0ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60796/0 (socket says 172.21.15.181:60796) 2023-10-16T11:21:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 -- 172.21.15.181:0/1249465005 learned_addr learned my addr 172.21.15.181:0/1249465005 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f0bfff700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c0cf690 0x7f8f0c1f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f0b7fe700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0c15d9b0 0x7f8f0c160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 -- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c0cf690 msgr2=0x7f8f0c1f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c0cf690 0x7f8f0c1f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 -- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0c15d9b0 msgr2=0x7f8f0c160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0c15d9b0 0x7f8f0c160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 -- 172.21.15.181:0/1249465005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04009b30 con 0x7f8f0c1621c0 2023-10-16T11:21:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f0bfff700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c0cf690 0x7f8f0c1f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f109ac700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c1621c0 0x7f8f0c0ca1f0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0cedb0 tx=0x7f8f04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f08ff9700 1 -- 172.21.15.181:0/1249465005 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04003ea0 con 0x7f8f0c1621c0 2023-10-16T11:21:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f08ff9700 1 -- 172.21.15.181:0/1249465005 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f040048e0 con 0x7f8f0c1621c0 2023-10-16T11:21:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f08ff9700 1 -- 172.21.15.181:0/1249465005 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0401c680 con 0x7f8f0c1621c0 2023-10-16T11:21:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c0c5970 con 0x7f8f0c1621c0 2023-10-16T11:21:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.487+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c0ca730 con 0x7f8f0c1621c0 2023-10-16T11:21:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.488+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c0c5b90 con 0x7f8f0c1621c0 2023-10-16T11:21:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.489+0000 7f8f08ff9700 1 -- 172.21.15.181:0/1249465005 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 165) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8f0401c7e0 con 0x7f8f0c1621c0 2023-10-16T11:21:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.490+0000 7f8f08ff9700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 0x7f8ef806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.490+0000 7f8f0bfff700 1 -- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 msgr2=0x7f8ef806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.490+0000 7f8f08ff9700 1 -- 172.21.15.181:0/1249465005 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f8f0408b7c0 con 0x7f8f0c1621c0 2023-10-16T11:21:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.490+0000 7f8f0bfff700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 0x7f8ef806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:21:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.494+0000 7f8f08ff9700 1 -- 172.21.15.181:0/1249465005 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f8f04056300 con 0x7f8f0c1621c0 2023-10-16T11:21:42.537 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:21:42.536+0000 7f43536ed200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:21:42.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.691+0000 7f8f0bfff700 1 -- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 msgr2=0x7f8ef806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3503716662 2023-10-16T11:21:42.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.691+0000 7f8f0bfff700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 0x7f8ef806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:21:42.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.741+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f0c0c5210 con 0x7f8f0c1621c0 2023-10-16T11:21:42.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.742+0000 7f8f08ff9700 1 -- 172.21.15.181:0/1249465005 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+4941 (secure 0 0 0) 0x7f8f0c0c5210 con 0x7f8f0c1621c0 2023-10-16T11:21:42.743 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:42.743 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3049755410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3049755410},{"type":"v1","addr":"172.21.15.190:6834","nonce":3049755410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2185474337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2185474337},{"type":"v1","addr":"172.21.15.181:6837","nonce":2185474337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2846112501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2846112501},{"type":"v1","addr":"172.21.15.181:6836","nonce":2846112501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:20.526852+0000","modified":"2023-10-16T11:21:23.550181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12930":{"gid":12930,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3258038365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3258038365},{"type":"v1","addr":"172.21.15.190:6835","nonce":3258038365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:21:42.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 msgr2=0x7f8ef806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:21:42.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 0x7f8ef806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c1621c0 msgr2=0x7f8f0c0ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c1621c0 0x7f8f0c0ca1f0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0cedb0 tx=0x7f8f04004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 shutdown_connections 2023-10-16T11:21:42.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:6832/3503716662,v1:172.21.15.181:6833/3503716662] conn(0x7f8ef806b020 0x7f8ef806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f0c0cf690 0x7f8f0c1f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f0c1621c0 0x7f8f0c0ca1f0 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 --2- 172.21.15.181:0/1249465005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f0c15d9b0 0x7f8f0c160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 >> 172.21.15.181:0/1249465005 conn(0x7f8f0c0665a0 msgr2=0x7f8f0c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:42.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 shutdown_connections 2023-10-16T11:21:42.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.744+0000 7f8f1240f700 1 -- 172.21.15.181:0/1249465005 wait complete. 2023-10-16T11:21:42.748 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 144 2023-10-16T11:21:42.758 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:42.855 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.855+0000 7f0c7f014200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:21:42.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.885+0000 7f1f1cc10700 1 Processor -- start 2023-10-16T11:21:42.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.885+0000 7f1f1cc10700 1 -- start start 2023-10-16T11:21:42.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.886+0000 7f1f1cc10700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f1815db70 0x7f1f180665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.886+0000 7f1f1cc10700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180ca2e0 0x7f1f180ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.886+0000 7f1f1cc10700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f18066c60 con 0x7f1f180cedd0 2023-10-16T11:21:42.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.886+0000 7f1f1cc10700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f180cda10 con 0x7f1f1815db70 2023-10-16T11:21:42.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.886+0000 7f1f1cc10700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1806c1a0 con 0x7f1f180ca2e0 2023-10-16T11:21:42.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f1815db70 0x7f1f180665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f1815db70 0x7f1f180665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60816/0 (socket says 172.21.15.181:60816) 2023-10-16T11:21:42.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 -- 172.21.15.181:0/3304333411 learned_addr learned my addr 172.21.15.181:0/3304333411 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:42.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f16d9d700 1 --2- 172.21.15.181:0/3304333411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180ca2e0 0x7f1f180ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 -- 172.21.15.181:0/3304333411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180ca2e0 msgr2=0x7f1f180ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:42.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 --2- 172.21.15.181:0/3304333411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180ca2e0 0x7f1f180ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 -- 172.21.15.181:0/3304333411 >> v1:172.21.15.181:6789/0 conn(0x7f1f180cedd0 legacy=0x7f1f180cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 -- 172.21.15.181:0/3304333411 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1806b3e0 con 0x7f1f1815db70 2023-10-16T11:21:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1659c700 1 --2- 172.21.15.181:0/3304333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f1815db70 0x7f1f180665a0 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098f0 tx=0x7f1f08009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a866badd53a60 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1559a700 1 -- 172.21.15.181:0/3304333411 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800b8a0 con 0x7f1f1815db70 2023-10-16T11:21:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1559a700 1 -- 172.21.15.181:0/3304333411 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0800ba00 con 0x7f1f1815db70 2023-10-16T11:21:42.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.887+0000 7f1f1559a700 1 -- 172.21.15.181:0/3304333411 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800bd90 con 0x7f1f1815db70 2023-10-16T11:21:42.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3304333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f1815db70 msgr2=0x7f1f180665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 --2- 172.21.15.181:0/3304333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f1815db70 0x7f1f180665a0 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098f0 tx=0x7f1f08009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:42.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3304333411 shutdown_connections 2023-10-16T11:21:42.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 --2- 172.21.15.181:0/3304333411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180ca2e0 0x7f1f180ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 --2- 172.21.15.181:0/3304333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f1815db70 0x7f1f180665a0 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3304333411 >> 172.21.15.181:0/3304333411 conn(0x7f1f180b1780 msgr2=0x7f1f180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:42.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3304333411 shutdown_connections 2023-10-16T11:21:42.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.888+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3304333411 wait complete. 2023-10-16T11:21:42.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.890+0000 7f1f1cc10700 1 Processor -- start 2023-10-16T11:21:42.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.890+0000 7f1f1cc10700 1 -- start start 2023-10-16T11:21:42.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1cc10700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180ca2e0 0x7f1f18160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1cc10700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180cedd0 0x7f1f1815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1cc10700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f1815db70 0x7f1f1815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:42.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1cc10700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1815a330 con 0x7f1f180ca2e0 2023-10-16T11:21:42.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1cc10700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180c7930 con 0x7f1f1815db70 2023-10-16T11:21:42.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1cc10700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180c6d60 con 0x7f1f180cedd0 2023-10-16T11:21:42.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180ca2e0 0x7f1f18160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180ca2e0 0x7f1f18160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60828/0 (socket says 172.21.15.181:60828) 2023-10-16T11:21:42.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 -- 172.21.15.181:0/3417085196 learned_addr learned my addr 172.21.15.181:0/3417085196 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:42.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f15d9b700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180cedd0 0x7f1f1815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f16d9d700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f1815db70 0x7f1f1815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:42.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 -- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180cedd0 msgr2=0x7f1f1815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180cedd0 0x7f1f1815df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 -- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f1815db70 msgr2=0x7f1f1815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:42.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f1815db70 0x7f1f1815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:42.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 -- 172.21.15.181:0/3417085196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08009550 con 0x7f1f180ca2e0 2023-10-16T11:21:42.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1659c700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180ca2e0 0x7f1f18160730 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009c40 tx=0x7f1f08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:42.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f06ffd700 1 -- 172.21.15.181:0/3417085196 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004510 con 0x7f1f180ca2e0 2023-10-16T11:21:42.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f06ffd700 1 -- 172.21.15.181:0/3417085196 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0801e070 con 0x7f1f180ca2e0 2023-10-16T11:21:42.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f180c7000 con 0x7f1f180ca2e0 2023-10-16T11:21:42.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.891+0000 7f1f06ffd700 1 -- 172.21.15.181:0/3417085196 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080038e0 con 0x7f1f180ca2e0 2023-10-16T11:21:42.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.892+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f180c75c0 con 0x7f1f180ca2e0 2023-10-16T11:21:42.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.892+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f18067d20 con 0x7f1f180ca2e0 2023-10-16T11:21:42.900 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.893+0000 7f1f06ffd700 1 -- 172.21.15.181:0/3417085196 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 166) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f1f08003a40 con 0x7f1f180ca2e0 2023-10-16T11:21:42.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.893+0000 7f1f06ffd700 1 -- 172.21.15.181:0/3417085196 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7035+0+0 (secure 0 0 0) 0x7f1f08020070 con 0x7f1f180ca2e0 2023-10-16T11:21:42.901 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:42.898+0000 7f1f06ffd700 1 -- 172.21.15.181:0/3417085196 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1f08055950 con 0x7f1f180ca2e0 2023-10-16T11:21:42.941 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bf6ff9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.941 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bf6ff9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.941 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bf6ff9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.942 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bf6ff9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.942 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bf6ff9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.942 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bfa800700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.942 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bfa800700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.942 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bfa800700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.943 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bfa800700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:42.943 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:21:42.940+0000 7f0bfa800700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:21:43.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.149+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f18072510 con 0x7f1f180ca2e0 2023-10-16T11:21:43.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.149+0000 7f1f06ffd700 1 -- 172.21.15.181:0/3417085196 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+4941 (secure 0 0 0) 0x7f1f08058f70 con 0x7f1f180ca2e0 2023-10-16T11:21:43.151 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:43.151 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":144,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3049755410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3049755410},{"type":"v1","addr":"172.21.15.190:6834","nonce":3049755410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2185474337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2185474337},{"type":"v1","addr":"172.21.15.181:6837","nonce":2185474337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2846112501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2846112501},{"type":"v1","addr":"172.21.15.181:6836","nonce":2846112501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:20.526852+0000","modified":"2023-10-16T11:21:23.550181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12930":{"gid":12930,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3258038365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3258038365},{"type":"v1","addr":"172.21.15.190:6835","nonce":3258038365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-16T11:21:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180ca2e0 msgr2=0x7f1f18160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.152 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180ca2e0 0x7f1f18160730 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009c40 tx=0x7f1f08004750 comp rx=0 tx=0).stop 2023-10-16T11:21:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 shutdown_connections 2023-10-16T11:21:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f1815db70 0x7f1f1815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180ca2e0 0x7f1f18160730 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 --2- 172.21.15.181:0/3417085196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f180cedd0 0x7f1f1815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.153 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 >> 172.21.15.181:0/3417085196 conn(0x7f1f180b1780 msgr2=0x7f1f180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:43.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 shutdown_connections 2023-10-16T11:21:43.154 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.151+0000 7f1f1cc10700 1 -- 172.21.15.181:0/3417085196 wait complete. 2023-10-16T11:21:43.154 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 144 2023-10-16T11:21:43.165 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:43.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.284+0000 7f9214411700 1 Processor -- start 2023-10-16T11:21:43.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.284+0000 7f9214411700 1 -- start start 2023-10-16T11:21:43.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f9214411700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c15e6d0 0x7f920c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f9214411700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f920c15eab0 0x7f920c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f9214411700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c06ed00 con 0x7f920c0cb0c0 2023-10-16T11:21:43.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f9214411700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c0d02d0 con 0x7f920c15eab0 2023-10-16T11:21:43.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f9214411700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c15ad40 con 0x7f920c15e6d0 2023-10-16T11:21:43.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f92129ae700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f920c0cb0c0 0x7f920c15a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43732/0 (socket says 172.21.15.181:43732) 2023-10-16T11:21:43.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f92129ae700 1 -- 172.21.15.181:0/2151190694 learned_addr learned my addr 172.21.15.181:0/2151190694 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:43.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f92119ac700 1 --2- 172.21.15.181:0/2151190694 >> v2:172.21.15.190:3300/0 conn(0x7f920c15eab0 0x7f920c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.285+0000 7f92121ad700 1 --2- 172.21.15.181:0/2151190694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c15e6d0 0x7f920c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92111ab700 1 -- 172.21.15.181:0/2151190694 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245427735 0 0) 0x7f920c06ed00 con 0x7f920c0cb0c0 2023-10-16T11:21:43.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92111ab700 1 -- 172.21.15.181:0/2151190694 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f80035b0 con 0x7f920c0cb0c0 2023-10-16T11:21:43.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92121ad700 1 -- 172.21.15.181:0/2151190694 >> v2:172.21.15.190:3300/0 conn(0x7f920c15eab0 msgr2=0x7f920c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:43.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92121ad700 1 --2- 172.21.15.181:0/2151190694 >> v2:172.21.15.190:3300/0 conn(0x7f920c15eab0 0x7f920c15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92121ad700 1 -- 172.21.15.181:0/2151190694 >> v1:172.21.15.181:6789/0 conn(0x7f920c0cb0c0 legacy=0x7f920c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92121ad700 1 -- 172.21.15.181:0/2151190694 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c06e0f0 con 0x7f920c15e6d0 2023-10-16T11:21:43.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92119ac700 1 --2- 172.21.15.181:0/2151190694 >> v2:172.21.15.190:3300/0 conn(0x7f920c15eab0 0x7f920c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:43.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92121ad700 1 --2- 172.21.15.181:0/2151190694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c15e6d0 0x7f920c0d0fc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009820 tx=0x7f91fc009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389c91c1fbd92572 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:43.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92111ab700 1 -- 172.21.15.181:0/2151190694 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc005e30 con 0x7f920c15e6d0 2023-10-16T11:21:43.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92111ab700 1 -- 172.21.15.181:0/2151190694 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc006030 con 0x7f920c15e6d0 2023-10-16T11:21:43.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.286+0000 7f92111ab700 1 -- 172.21.15.181:0/2151190694 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc0063a0 con 0x7f920c15e6d0 2023-10-16T11:21:43.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 -- 172.21.15.181:0/2151190694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c15e6d0 msgr2=0x7f920c0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 --2- 172.21.15.181:0/2151190694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c15e6d0 0x7f920c0d0fc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009820 tx=0x7f91fc009b30 comp rx=0 tx=0).stop 2023-10-16T11:21:43.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 -- 172.21.15.181:0/2151190694 shutdown_connections 2023-10-16T11:21:43.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 --2- 172.21.15.181:0/2151190694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c15e6d0 0x7f920c0d0fc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 --2- 172.21.15.181:0/2151190694 >> v2:172.21.15.190:3300/0 conn(0x7f920c15eab0 0x7f920c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 -- 172.21.15.181:0/2151190694 >> 172.21.15.181:0/2151190694 conn(0x7f920c0d04a0 msgr2=0x7f920c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:43.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 -- 172.21.15.181:0/2151190694 shutdown_connections 2023-10-16T11:21:43.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.287+0000 7f9214411700 1 -- 172.21.15.181:0/2151190694 wait complete. 2023-10-16T11:21:43.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.288+0000 7f9214411700 1 Processor -- start 2023-10-16T11:21:43.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.288+0000 7f9214411700 1 -- start start 2023-10-16T11:21:43.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f9214411700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c0cb0c0 0x7f920c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f9214411700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f920c15e6d0 0x7f920c0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.296 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f9214411700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c15eab0 0x7f920c0c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f9214411700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c156490 con 0x7f920c15eab0 2023-10-16T11:21:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f9214411700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0afee0 con 0x7f920c15e6d0 2023-10-16T11:21:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f9214411700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0af960 con 0x7f920c0cb0c0 2023-10-16T11:21:43.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f92129ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c15eab0 0x7f920c0c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f92129ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c15eab0 0x7f920c0c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60896/0 (socket says 172.21.15.181:60896) 2023-10-16T11:21:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f92129ae700 1 -- 172.21.15.181:0/1856183124 learned_addr learned my addr 172.21.15.181:0/1856183124 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f92121ad700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c0cb0c0 0x7f920c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.289+0000 7f92119ac700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f920c15e6d0 0x7f920c0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f92129ae700 1 -- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c0cb0c0 msgr2=0x7f920c0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f92129ae700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c0cb0c0 0x7f920c0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f92129ae700 1 -- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f920c15e6d0 msgr2=0x7f920c0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f92129ae700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f920c15e6d0 0x7f920c0c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f92129ae700 1 -- 172.21.15.181:0/1856183124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080094a0 con 0x7f920c15eab0 2023-10-16T11:21:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f92129ae700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c15eab0 0x7f920c0c2f70 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f920400caa0 tx=0x7f920400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:43.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f9202ffd700 1 -- 172.21.15.181:0/1856183124 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400a9e0 con 0x7f920c15eab0 2023-10-16T11:21:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f9202ffd700 1 -- 172.21.15.181:0/1856183124 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920400ab40 con 0x7f920c15eab0 2023-10-16T11:21:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc009430 con 0x7f920c15eab0 2023-10-16T11:21:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f9202ffd700 1 -- 172.21.15.181:0/1856183124 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204012400 con 0x7f920c15eab0 2023-10-16T11:21:43.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.290+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c0afc00 con 0x7f920c15eab0 2023-10-16T11:21:43.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.291+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920c067d20 con 0x7f920c15eab0 2023-10-16T11:21:43.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.291+0000 7f9202ffd700 1 -- 172.21.15.181:0/1856183124 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 166) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f920401b3b0 con 0x7f920c15eab0 2023-10-16T11:21:43.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.292+0000 7f9202ffd700 1 -- 172.21.15.181:0/1856183124 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7035+0+0 (secure 0 0 0) 0x7f9204089180 con 0x7f920c15eab0 2023-10-16T11:21:43.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.296+0000 7f9202ffd700 1 -- 172.21.15.181:0/1856183124 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9204054210 con 0x7f920c15eab0 2023-10-16T11:21:43.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.489+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f920c0cb610 con 0x7f920c15eab0 2023-10-16T11:21:43.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.490+0000 7f9202ffd700 1 -- 172.21.15.181:0/1856183124 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+18554 (secure 0 0 0) 0x7f9204057830 con 0x7f920c15eab0 2023-10-16T11:21:43.491 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:43.492 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":258,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:42.868350+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:18.964173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:19.485121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3430602932":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/452036432":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/3825373001":"2023-10-17T11:21:42.868291+0000","172.21.15.181:6833/3503716662":"2023-10-17T11:21:42.868291+0000","172.21.15.181:6832/3503716662":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/1323931653":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/3637292004":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3476229854":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/1366795220":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2569889467":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6833/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6832/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/58286368":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/2946054209":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/1975124632":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/649394230":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/507502116":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3665147670":"2023-10-17T11:21:34.384612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:43.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c15eab0 msgr2=0x7f920c0c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c15eab0 0x7f920c0c2f70 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f920400caa0 tx=0x7f920400cdb0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 shutdown_connections 2023-10-16T11:21:43.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f920c0cb0c0 0x7f920c0b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f920c15eab0 0x7f920c0c2f70 unknown :-1 s=CLOSED pgs=3056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 --2- 172.21.15.181:0/1856183124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f920c15e6d0 0x7f920c0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 >> 172.21.15.181:0/1856183124 conn(0x7f920c0d04a0 msgr2=0x7f920c0ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:43.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 shutdown_connections 2023-10-16T11:21:43.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.492+0000 7f9214411700 1 -- 172.21.15.181:0/1856183124 wait complete. 2023-10-16T11:21:43.506 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:21:43.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.626+0000 7f4ca24fc700 1 Processor -- start 2023-10-16T11:21:43.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.626+0000 7f4ca24fc700 1 -- start start 2023-10-16T11:21:43.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4ca24fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 0x7f4c9c0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4ca24fc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cf980 0x7f4c9c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4ca24fc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c06b370 con 0x7f4c9c15b9f0 2023-10-16T11:21:43.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4ca24fc700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c06b800 con 0x7f4c9c0cf980 2023-10-16T11:21:43.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4ca24fc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0cddd0 con 0x7f4c9c0cd950 2023-10-16T11:21:43.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4c9bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4c9c15b9f0 0x7f4c9c0cc400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43744/0 (socket says 172.21.15.181:43744) 2023-10-16T11:21:43.632 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4c9bfff700 1 -- 172.21.15.181:0/3365509935 learned_addr learned my addr 172.21.15.181:0/3365509935 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:43.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027566628 0 0) 0x7f4c9c06b370 con 0x7f4c9c15b9f0 2023-10-16T11:21:43.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4ca0a99700 1 --2- 172.21.15.181:0/3365509935 >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cf980 0x7f4c9c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 0x7f4c9c0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.627+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c840035b0 con 0x7f4c9c15b9f0 2023-10-16T11:21:43.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044093384 0 0) 0x7f4c840035b0 con 0x7f4c9c15b9f0 2023-10-16T11:21:43.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 msgr2=0x7f4c9c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:43.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 0x7f4c9c0cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cf980 msgr2=0x7f4c9c0ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:43.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 --2- 172.21.15.181:0/3365509935 >> v2:172.21.15.190:3300/0 conn(0x7f4c9c0cf980 0x7f4c9c0ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0cfdc0 con 0x7f4c9c15b9f0 2023-10-16T11:21:43.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4c900017f0 con 0x7f4c9c15b9f0 2023-10-16T11:21:43.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 >> v1:172.21.15.181:6789/0 conn(0x7f4c9c15b9f0 legacy=0x7f4c9c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 0x7f4c84003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c84003f80 0x7f4c84004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84004d90 0x7f4c84005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.635 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c06b800 con 0x7f4c84003f80 2023-10-16T11:21:43.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0cddd0 con 0x7f4c84004d90 2023-10-16T11:21:43.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8400ba10 con 0x7f4c9c0cd950 2023-10-16T11:21:43.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c84003f80 0x7f4c84004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4ca0a99700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84004d90 0x7f4c84005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.636 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9bfff700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 0x7f4c84003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 msgr2=0x7f4c84003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:43.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c0cd950 0x7f4c84003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84004d90 msgr2=0x7f4c84005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:43.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84004d90 0x7f4c84005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/3365509935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c840035b0 con 0x7f4c84003f80 2023-10-16T11:21:43.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c84003f80 0x7f4c84004850 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009b60 tx=0x7f4c84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e47d5d0a34339c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:43.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00e660 con 0x7f4c84003f80 2023-10-16T11:21:43.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00e7c0 con 0x7f4c84003f80 2023-10-16T11:21:43.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.628+0000 7f4c9affd700 1 -- 172.21.15.181:0/3365509935 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00eb30 con 0x7f4c84003f80 2023-10-16T11:21:43.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.629+0000 7f4ca24fc700 1 -- 172.21.15.181:0/3365509935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c84003f80 msgr2=0x7f4c84004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.629+0000 7f4ca24fc700 1 --2- 172.21.15.181:0/3365509935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c84003f80 0x7f4c84004850 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009b60 tx=0x7f4c84005880 comp rx=0 tx=0).stop 2023-10-16T11:21:43.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.629+0000 7f4ca0a99700 1 -- 172.21.15.181:0/3365509935 reap_dead start 2023-10-16T11:21:43.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.629+0000 7f4ca24fc700 1 -- 172.21.15.181:0/3365509935 shutdown_connections 2023-10-16T11:21:43.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.629+0000 7f4ca24fc700 1 -- 172.21.15.181:0/3365509935 >> 172.21.15.181:0/3365509935 conn(0x7f4c9c0d04a0 msgr2=0x7f4c9c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.629+0000 7f4ca24fc700 1 -- 172.21.15.181:0/3365509935 shutdown_connections 2023-10-16T11:21:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.629+0000 7f4ca24fc700 1 -- 172.21.15.181:0/3365509935 wait complete. 2023-10-16T11:21:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.630+0000 7f4ca24fc700 1 Processor -- start 2023-10-16T11:21:43.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.630+0000 7f4ca24fc700 1 -- start start 2023-10-16T11:21:43.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4ca24fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84003f80 0x7f4c9c154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4ca24fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c154e40 0x7f4c9c1f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4ca24fc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c1f0e80 0x7f4c9c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4ca24fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1552c0 con 0x7f4c9c154e40 2023-10-16T11:21:43.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4ca24fc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1f12e0 con 0x7f4c84003f80 2023-10-16T11:21:43.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4ca24fc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c153df0 con 0x7f4c9c1f0e80 2023-10-16T11:21:43.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4c9b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c154e40 0x7f4c9c1f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4c9bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84003f80 0x7f4c9c154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4c9b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c154e40 0x7f4c9c1f0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60912/0 (socket says 172.21.15.181:60912) 2023-10-16T11:21:43.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/1769072433 learned_addr learned my addr 172.21.15.181:0/1769072433 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:43.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.631+0000 7f4ca0a99700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c1f0e80 0x7f4c9c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/1769072433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c1f0e80 msgr2=0x7f4c9c152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c1f0e80 0x7f4c9c152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/1769072433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84003f80 msgr2=0x7f4c9c154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84003f80 0x7f4c9c154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c9b7fe700 1 -- 172.21.15.181:0/1769072433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c153270 con 0x7f4c9c154e40 2023-10-16T11:21:43.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c9b7fe700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c154e40 0x7f4c9c1f0940 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7f4c840046f0 tx=0x7f4c8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:43.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00e660 con 0x7f4c9c154e40 2023-10-16T11:21:43.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0038a0 con 0x7f4c9c154e40 2023-10-16T11:21:43.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c153550 con 0x7f4c9c154e40 2023-10-16T11:21:43.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003c50 con 0x7f4c9c154e40 2023-10-16T11:21:43.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.632+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c153b10 con 0x7f4c9c154e40 2023-10-16T11:21:43.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.633+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c067d20 con 0x7f4c9c154e40 2023-10-16T11:21:43.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.633+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 166) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4c8c013470 con 0x7f4c9c154e40 2023-10-16T11:21:43.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.634+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7035+0+0 (secure 0 0 0) 0x7f4c8c08ade0 con 0x7f4c9c154e40 2023-10-16T11:21:43.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.638+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c8c055ed0 con 0x7f4c9c154e40 2023-10-16T11:21:43.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.874+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4c9c074160 con 0x7f4c9c154e40 2023-10-16T11:21:43.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.880+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 167) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4c8c055980 con 0x7f4c9c154e40 2023-10-16T11:21:43.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.880+0000 7f4c98ff9700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f4c88070ed0 0x7f4c880732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:43.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.881+0000 7f4c9bfff700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f4c88070ed0 0x7f4c880732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:43.882 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:21:43.881+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:21:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.882+0000 7f4c9bfff700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f4c88070ed0 0x7f4c880732c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c90000c00 tx=0x7f4c90009040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.883+0000 7f4c98ff9700 1 -- 172.21.15.181:0/1769072433 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v145) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4c8c05cf80 con 0x7f4c9c154e40 2023-10-16T11:21:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f4c88070ed0 msgr2=0x7f4c880732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f4c88070ed0 0x7f4c880732c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c90000c00 tx=0x7f4c90009040 comp rx=0 tx=0).stop 2023-10-16T11:21:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c154e40 msgr2=0x7f4c9c1f0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c154e40 0x7f4c9c1f0940 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7f4c840046f0 tx=0x7f4c8c0047c0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 shutdown_connections 2023-10-16T11:21:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f4c88070ed0 0x7f4c880732c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c9c1f0e80 0x7f4c9c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c84003f80 0x7f4c9c154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 --2- 172.21.15.181:0/1769072433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9c154e40 0x7f4c9c1f0940 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 >> 172.21.15.181:0/1769072433 conn(0x7f4c9c0d04a0 msgr2=0x7f4c9c0ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 shutdown_connections 2023-10-16T11:21:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:43.887+0000 7f4ca24fc700 1 -- 172.21.15.181:0/1769072433 wait complete. 2023-10-16T11:21:43.891 INFO:tasks.ceph.mds.b.smithi190.stderr: -5182> 2023-10-16T11:21:23.113+0000 7fd434197700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:21:43.891 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:21:43.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:21:43.921 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:21:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.022+0000 7fecbef0d700 1 Processor -- start 2023-10-16T11:21:44.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.022+0000 7fecbef0d700 1 -- start start 2023-10-16T11:21:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbef0d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fecb815da70 0x7fecb80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbef0d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fecb80ce6a0 0x7fecb806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbef0d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb80cef00 con 0x7fecb8072760 2023-10-16T11:21:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbef0d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb8072b40 con 0x7fecb815da70 2023-10-16T11:21:44.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbef0d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb80cec20 con 0x7fecb80ce6a0 2023-10-16T11:21:44.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbcca9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fecb815da70 0x7fecb80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:44.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbd4aa700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fecb8072760 0x7fecb81615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43788/0 (socket says 172.21.15.181:43788) 2023-10-16T11:21:44.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbd4aa700 1 -- 172.21.15.181:0/3257053710 learned_addr learned my addr 172.21.15.181:0/3257053710 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:44.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbcca9700 1 --2- 172.21.15.181:0/3257053710 >> v2:172.21.15.181:3300/0 conn(0x7fecb815da70 0x7fecb80ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60940/0 (socket says 172.21.15.181:60940) 2023-10-16T11:21:44.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecb7fff700 1 --2- 172.21.15.181:0/3257053710 >> v2:172.21.15.190:3300/0 conn(0x7fecb80ce6a0 0x7fecb806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:44.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbcca9700 1 -- 172.21.15.181:0/3257053710 >> v2:172.21.15.190:3300/0 conn(0x7fecb80ce6a0 msgr2=0x7fecb806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:44.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbcca9700 1 --2- 172.21.15.181:0/3257053710 >> v2:172.21.15.190:3300/0 conn(0x7fecb80ce6a0 0x7fecb806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbcca9700 1 -- 172.21.15.181:0/3257053710 >> v1:172.21.15.181:6789/0 conn(0x7fecb8072760 legacy=0x7fecb81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:44.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecbcca9700 1 -- 172.21.15.181:0/3257053710 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb8071c80 con 0x7fecb815da70 2023-10-16T11:21:44.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.023+0000 7fecb77fe700 1 -- 172.21.15.181:0/3257053710 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830732436 0 0) 0x7fecb80cef00 con 0x7fecb8072760 2023-10-16T11:21:44.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbcca9700 1 --2- 172.21.15.181:0/3257053710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb815da70 0x7fecb80ce160 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7fecb806ef60 tx=0x7fecac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516c3cd86c6dba33 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:44.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecb77fe700 1 -- 172.21.15.181:0/3257053710 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac011830 con 0x7fecb815da70 2023-10-16T11:21:44.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecb77fe700 1 -- 172.21.15.181:0/3257053710 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac011990 con 0x7fecb815da70 2023-10-16T11:21:44.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecb77fe700 1 -- 172.21.15.181:0/3257053710 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac011d20 con 0x7fecb815da70 2023-10-16T11:21:44.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 -- 172.21.15.181:0/3257053710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb815da70 msgr2=0x7fecb80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:44.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3257053710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb815da70 0x7fecb80ce160 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7fecb806ef60 tx=0x7fecac009b20 comp rx=0 tx=0).stop 2023-10-16T11:21:44.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 -- 172.21.15.181:0/3257053710 shutdown_connections 2023-10-16T11:21:44.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3257053710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb815da70 0x7fecb80ce160 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3257053710 >> v2:172.21.15.190:3300/0 conn(0x7fecb80ce6a0 0x7fecb806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 -- 172.21.15.181:0/3257053710 >> 172.21.15.181:0/3257053710 conn(0x7fecb80cd010 msgr2=0x7fecb80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:44.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 -- 172.21.15.181:0/3257053710 shutdown_connections 2023-10-16T11:21:44.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.024+0000 7fecbef0d700 1 -- 172.21.15.181:0/3257053710 wait complete. 2023-10-16T11:21:44.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.025+0000 7fecbef0d700 1 Processor -- start 2023-10-16T11:21:44.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecbef0d700 1 -- start start 2023-10-16T11:21:44.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecbef0d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecb8072760 0x7fecb8151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:44.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecbef0d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce6a0 0x7fecb80c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:44.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecbef0d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb815da70 0x7fecb80cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:44.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecbef0d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb8152080 con 0x7fecb80ce6a0 2023-10-16T11:21:44.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecbef0d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80c50b0 con 0x7fecb8072760 2023-10-16T11:21:44.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecbef0d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80c44a0 con 0x7fecb815da70 2023-10-16T11:21:44.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.026+0000 7fecb7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce6a0 0x7fecb80c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:44.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce6a0 0x7fecb80c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60946/0 (socket says 172.21.15.181:60946) 2023-10-16T11:21:44.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 -- 172.21.15.181:0/3568776509 learned_addr learned my addr 172.21.15.181:0/3568776509 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:44.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecbcca9700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecb8072760 0x7fecb8151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:44.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecbd4aa700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb815da70 0x7fecb80cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:44.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 -- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb815da70 msgr2=0x7fecb80cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:44.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb815da70 0x7fecb80cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 -- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecb8072760 msgr2=0x7fecb8151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:44.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecb8072760 0x7fecb8151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 -- 172.21.15.181:0/3568776509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0094a0 con 0x7fecb80ce6a0 2023-10-16T11:21:44.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb7fff700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce6a0 0x7fecb80c67e0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7feca400e810 tx=0x7feca400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:44.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb57fa700 1 -- 172.21.15.181:0/3568776509 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca400c070 con 0x7fecb80ce6a0 2023-10-16T11:21:44.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb57fa700 1 -- 172.21.15.181:0/3568776509 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca4013610 con 0x7fecb80ce6a0 2023-10-16T11:21:44.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb80c4780 con 0x7fecb80ce6a0 2023-10-16T11:21:44.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecb57fa700 1 -- 172.21.15.181:0/3568776509 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40139a0 con 0x7fecb80ce6a0 2023-10-16T11:21:44.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.027+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb80c4d40 con 0x7fecb80ce6a0 2023-10-16T11:21:44.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.028+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb80c51f0 con 0x7fecb80ce6a0 2023-10-16T11:21:44.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.028+0000 7fecb57fa700 1 -- 172.21.15.181:0/3568776509 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 167) v1 ==== 89067+0+0 (secure 0 0 0) 0x7feca4013b00 con 0x7fecb80ce6a0 2023-10-16T11:21:44.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.029+0000 7fecb57fa700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fec9c06ad50 0x7fec9c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:44.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.029+0000 7fecb57fa700 1 -- 172.21.15.181:0/3568776509 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7121+0+0 (secure 0 0 0) 0x7feca4089530 con 0x7fecb80ce6a0 2023-10-16T11:21:44.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.029+0000 7fecbcca9700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fec9c06ad50 0x7fec9c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:44.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.029+0000 7fecbcca9700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fec9c06ad50 0x7fec9c06d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fecac0090b0 tx=0x7fecac008030 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:44.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.033+0000 7fecb57fa700 1 -- 172.21.15.181:0/3568776509 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fecb80c51f0 con 0x7fecb80ce6a0 2023-10-16T11:21:44.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.278+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fecb80c51f0 con 0x7fecb80ce6a0 2023-10-16T11:21:44.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.887+0000 7fecb57fa700 1 -- 172.21.15.181:0/3568776509 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v146) v1 ==== 104+0+0 (secure 0 0 0) 0x7fecb80c51f0 con 0x7fecb80ce6a0 2023-10-16T11:21:44.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fec9c06ad50 msgr2=0x7fec9c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:44.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fec9c06ad50 0x7fec9c06d210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fecac0090b0 tx=0x7fecac008030 comp rx=0 tx=0).stop 2023-10-16T11:21:44.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce6a0 msgr2=0x7fecb80c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:44.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce6a0 0x7fecb80c67e0 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7feca400e810 tx=0x7feca400eb20 comp rx=0 tx=0).stop 2023-10-16T11:21:44.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 shutdown_connections 2023-10-16T11:21:44.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fec9c06ad50 0x7fec9c06d210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb815da70 0x7fecb80cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecb8072760 0x7fecb8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 --2- 172.21.15.181:0/3568776509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce6a0 0x7fecb80c67e0 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:44.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 >> 172.21.15.181:0/3568776509 conn(0x7fecb80cd010 msgr2=0x7fecb80cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:44.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 shutdown_connections 2023-10-16T11:21:44.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:44.890+0000 7fecbef0d700 1 -- 172.21.15.181:0/3568776509 wait complete. 2023-10-16T11:21:44.905 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:21:45.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.036+0000 7fb1b43f9700 1 Processor -- start 2023-10-16T11:21:45.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.036+0000 7fb1b43f9700 1 -- start start 2023-10-16T11:21:45.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b43f9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0afa10 0x7fb1ac0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:45.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b43f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:45.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b43f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac0cd850 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b43f9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac15b540 con 0x7fb1ac0afa10 2023-10-16T11:21:45.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b43f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac15b820 con 0x7fb1ac06d870 2023-10-16T11:21:45.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b2996700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb1ac0cd3f0 0x7fb1ac0cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43820/0 (socket says 172.21.15.181:43820) 2023-10-16T11:21:45.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b2996700 1 -- 172.21.15.181:0/694525383 learned_addr learned my addr 172.21.15.181:0/694525383 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:45.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b1994700 1 --2- 172.21.15.181:0/694525383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:45.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b2195700 1 --2- 172.21.15.181:0/694525383 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0afa10 0x7fb1ac0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:45.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b1994700 1 -- 172.21.15.181:0/694525383 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0afa10 msgr2=0x7fb1ac0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:45.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b1994700 1 --2- 172.21.15.181:0/694525383 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0afa10 0x7fb1ac0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:45.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b1994700 1 -- 172.21.15.181:0/694525383 >> v1:172.21.15.181:6789/0 conn(0x7fb1ac0cd3f0 legacy=0x7fb1ac0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:45.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b1994700 1 -- 172.21.15.181:0/694525383 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac15bb00 con 0x7fb1ac06d870 2023-10-16T11:21:45.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.037+0000 7fb1b2195700 1 --2- 172.21.15.181:0/694525383 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0afa10 0x7fb1ac0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:45.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.038+0000 7fb1b1994700 1 --2- 172.21.15.181:0/694525383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0cceb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009840 tx=0x7fb19c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab6cd71a3f8c3f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:45.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.039+0000 7fb1b1193700 1 -- 172.21.15.181:0/694525383 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c011830 con 0x7fb1ac06d870 2023-10-16T11:21:45.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.039+0000 7fb1b1193700 1 -- 172.21.15.181:0/694525383 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c011990 con 0x7fb1ac06d870 2023-10-16T11:21:45.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.039+0000 7fb1b1193700 1 -- 172.21.15.181:0/694525383 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c011d20 con 0x7fb1ac06d870 2023-10-16T11:21:45.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 -- 172.21.15.181:0/694525383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 msgr2=0x7fb1ac0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:45.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/694525383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0cceb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009840 tx=0x7fb19c009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:45.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 -- 172.21.15.181:0/694525383 shutdown_connections 2023-10-16T11:21:45.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/694525383 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0afa10 0x7fb1ac0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:45.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/694525383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0cceb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:45.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 -- 172.21.15.181:0/694525383 >> 172.21.15.181:0/694525383 conn(0x7fb1ac0d04a0 msgr2=0x7fb1ac0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:45.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 -- 172.21.15.181:0/694525383 shutdown_connections 2023-10-16T11:21:45.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.040+0000 7fb1b43f9700 1 -- 172.21.15.181:0/694525383 wait complete. 2023-10-16T11:21:45.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.041+0000 7fb1b43f9700 1 Processor -- start 2023-10-16T11:21:45.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.041+0000 7fb1b43f9700 1 -- start start 2023-10-16T11:21:45.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b43f9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:45.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b43f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1ac0afa10 0x7fb1ac15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:45.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b43f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ac0cd3f0 0x7fb1ac0c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:45.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b43f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac15e790 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b43f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac15d8e0 con 0x7fb1ac0afa10 2023-10-16T11:21:45.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b43f9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0b5f20 con 0x7fb1ac06d870 2023-10-16T11:21:45.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b2996700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ac0cd3f0 0x7fb1ac0c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:45.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b2996700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ac0cd3f0 0x7fb1ac0c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60980/0 (socket says 172.21.15.181:60980) 2023-10-16T11:21:45.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b2996700 1 -- 172.21.15.181:0/4281858965 learned_addr learned my addr 172.21.15.181:0/4281858965 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:45.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b2195700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:45.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.042+0000 7fb1b1994700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1ac0afa10 0x7fb1ac15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:45.049 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b2996700 1 -- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 msgr2=0x7fb1ac0b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:45.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b2996700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:45.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b2996700 1 -- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1ac0afa10 msgr2=0x7fb1ac15ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:45.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b2996700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1ac0afa10 0x7fb1ac15ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:45.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b2996700 1 -- 172.21.15.181:0/4281858965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009430 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b2996700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ac0cd3f0 0x7fb1ac0c6040 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400cbc0 tx=0x7fb1a400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:45.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a400a9e0 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a400ab40 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c0094a0 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4012400 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.043+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac15e0d0 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.044+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac067300 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.044+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 167) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb1a401b3b0 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.045+0000 7fb1a2ffd700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb19006ad50 0x7fb19006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:45.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.045+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7121+0+0 (secure 0 0 0) 0x7fb1a4089360 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.045+0000 7fb1b2195700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb19006ad50 0x7fb19006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:45.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.045+0000 7fb1b2195700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb19006ad50 0x7fb19006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009d50 tx=0x7fb1a80091c0 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:45.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.049+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb1a40541a0 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.243+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1ac0ccd20 con 0x7fb1ac0cd3f0 2023-10-16T11:21:45.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:45.427+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 168) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb1a4054570 con 0x7fb1ac0cd3f0 2023-10-16T11:21:46.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.037+0000 7fb1a2ffd700 1 -- 172.21.15.181:0/4281858965 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v260) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb1a4074790 con 0x7fb1ac0cd3f0 2023-10-16T11:21:46.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb19006ad50 msgr2=0x7fb19006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:46.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb19006ad50 0x7fb19006d210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009d50 tx=0x7fb1a80091c0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ac0cd3f0 msgr2=0x7fb1ac0c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:46.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ac0cd3f0 0x7fb1ac0c6040 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400cbc0 tx=0x7fb1a400ced0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 shutdown_connections 2023-10-16T11:21:46.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb19006ad50 0x7fb19006d210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac06d870 0x7fb1ac0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ac0cd3f0 0x7fb1ac0c6040 unknown :-1 s=CLOSED pgs=3066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 --2- 172.21.15.181:0/4281858965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1ac0afa10 0x7fb1ac15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.041+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 >> 172.21.15.181:0/4281858965 conn(0x7fb1ac0d04a0 msgr2=0x7fb1ac0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:46.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.042+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 shutdown_connections 2023-10-16T11:21:46.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.042+0000 7fb1b43f9700 1 -- 172.21.15.181:0/4281858965 wait complete. 2023-10-16T11:21:46.046 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:21:46.060 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:21:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.188+0000 7f193f30c700 1 Processor -- start 2023-10-16T11:21:46.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.188+0000 7f193f30c700 1 -- start start 2023-10-16T11:21:46.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193f30c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19380d0fc0 0x7f193806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:46.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193f30c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380ce9f0 0x7f193806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:46.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193f30c700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1938162650 con 0x7f19380ce610 2023-10-16T11:21:46.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193f30c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1938162190 con 0x7f19380d0fc0 2023-10-16T11:21:46.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193f30c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380d0230 con 0x7f19380ce9f0 2023-10-16T11:21:46.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19380d0fc0 0x7f193806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:46.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19380d0fc0 0x7f193806d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60986/0 (socket says 172.21.15.181:60986) 2023-10-16T11:21:46.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 -- 172.21.15.181:0/3400525886 learned_addr learned my addr 172.21.15.181:0/3400525886 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:46.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 -- 172.21.15.181:0/3400525886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380ce9f0 msgr2=0x7f193806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:46.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193d8a9700 1 --2- 172.21.15.181:0/3400525886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380ce9f0 0x7f193806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:46.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 --2- 172.21.15.181:0/3400525886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380ce9f0 0x7f193806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 -- 172.21.15.181:0/3400525886 >> v1:172.21.15.190:6789/0 conn(0x7f19380ce610 legacy=0x7f19380cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:46.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 -- 172.21.15.181:0/3400525886 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1938162370 con 0x7f19380d0fc0 2023-10-16T11:21:46.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193d8a9700 1 --2- 172.21.15.181:0/3400525886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380ce9f0 0x7f193806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:46.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f193c8a7700 1 --2- 172.21.15.181:0/3400525886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380d0fc0 0x7f193806d660 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f19340098f0 tx=0x7f1934009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d96d738bdbdc813 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:46.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.189+0000 7f192ffff700 1 -- 172.21.15.181:0/3400525886 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400b8a0 con 0x7f19380d0fc0 2023-10-16T11:21:46.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f192ffff700 1 -- 172.21.15.181:0/3400525886 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193400ba00 con 0x7f19380d0fc0 2023-10-16T11:21:46.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f192ffff700 1 -- 172.21.15.181:0/3400525886 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400bd90 con 0x7f19380d0fc0 2023-10-16T11:21:46.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 -- 172.21.15.181:0/3400525886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380d0fc0 msgr2=0x7f193806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:46.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 --2- 172.21.15.181:0/3400525886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380d0fc0 0x7f193806d660 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f19340098f0 tx=0x7f1934009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:46.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 -- 172.21.15.181:0/3400525886 shutdown_connections 2023-10-16T11:21:46.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 --2- 172.21.15.181:0/3400525886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380ce9f0 0x7f193806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 --2- 172.21.15.181:0/3400525886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380d0fc0 0x7f193806d660 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 -- 172.21.15.181:0/3400525886 >> 172.21.15.181:0/3400525886 conn(0x7f19380d04a0 msgr2=0x7f19380d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:46.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 -- 172.21.15.181:0/3400525886 shutdown_connections 2023-10-16T11:21:46.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.190+0000 7f193f30c700 1 -- 172.21.15.181:0/3400525886 wait complete. 2023-10-16T11:21:46.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.191+0000 7f193f30c700 1 Processor -- start 2023-10-16T11:21:46.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.191+0000 7f193f30c700 1 -- start start 2023-10-16T11:21:46.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193f30c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19380ce610 0x7f193815f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:46.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193f30c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380ce9f0 0x7f1938160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:46.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193f30c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380d0fc0 0x7f19381f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:46.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193f30c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19380cba70 con 0x7f19380ce9f0 2023-10-16T11:21:46.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193f30c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193815edf0 con 0x7f19380ce610 2023-10-16T11:21:46.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193f30c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193815f0d0 con 0x7f19380d0fc0 2023-10-16T11:21:46.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193c8a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380ce9f0 0x7f1938160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:46.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193c8a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380ce9f0 0x7f1938160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60992/0 (socket says 172.21.15.181:60992) 2023-10-16T11:21:46.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193c8a7700 1 -- 172.21.15.181:0/980895083 learned_addr learned my addr 172.21.15.181:0/980895083 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:46.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193d0a8700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19380ce610 0x7f193815f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:46.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.192+0000 7f193d8a9700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380d0fc0 0x7f19381f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:46.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193c8a7700 1 -- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380d0fc0 msgr2=0x7f19381f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:46.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193c8a7700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380d0fc0 0x7f19381f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193c8a7700 1 -- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19380ce610 msgr2=0x7f193815f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:46.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193c8a7700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19380ce610 0x7f193815f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:46.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193c8a7700 1 -- 172.21.15.181:0/980895083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009550 con 0x7f19380ce9f0 2023-10-16T11:21:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193c8a7700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380ce9f0 0x7f1938160350 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f1934009c40 tx=0x7f1934004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f192dffb700 1 -- 172.21.15.181:0/980895083 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934004400 con 0x7f19380ce9f0 2023-10-16T11:21:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f192dffb700 1 -- 172.21.15.181:0/980895083 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193401f070 con 0x7f19380ce9f0 2023-10-16T11:21:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f192dffb700 1 -- 172.21.15.181:0/980895083 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19340038e0 con 0x7f19380ce9f0 2023-10-16T11:21:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193815f3b0 con 0x7f19380ce9f0 2023-10-16T11:21:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.193+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19380cb020 con 0x7f19380ce9f0 2023-10-16T11:21:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.194+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193815ec50 con 0x7f19380ce9f0 2023-10-16T11:21:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.194+0000 7f192dffb700 1 -- 172.21.15.181:0/980895083 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 168) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1934021030 con 0x7f19380ce9f0 2023-10-16T11:21:46.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.195+0000 7f192dffb700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f191c06af90 0x7f191c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:46.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.195+0000 7f192dffb700 1 -- 172.21.15.181:0/980895083 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 6683+0+0 (secure 0 0 0) 0x7f193408ad50 con 0x7f19380ce9f0 2023-10-16T11:21:46.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.195+0000 7f193d0a8700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f191c06af90 0x7f191c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:46.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.195+0000 7f193d0a8700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f191c06af90 0x7f191c06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1928000fb0 tx=0x7f1928009040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:46.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.199+0000 7f192dffb700 1 -- 172.21.15.181:0/980895083 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f193815ec50 con 0x7f19380ce9f0 2023-10-16T11:21:46.399 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:46.397+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f193815f5f0 con 0x7f19380ce9f0 2023-10-16T11:21:47.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.039+0000 7f192dffb700 1 -- 172.21.15.181:0/980895083 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v261) v1 ==== 169+0+0 (secure 0 0 0) 0x7f193815f5f0 con 0x7f19380ce9f0 2023-10-16T11:21:47.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f191c06af90 msgr2=0x7f191c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f191c06af90 0x7f191c06d450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1928000fb0 tx=0x7f1928009040 comp rx=0 tx=0).stop 2023-10-16T11:21:47.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380ce9f0 msgr2=0x7f1938160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380ce9f0 0x7f1938160350 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f1934009c40 tx=0x7f1934004750 comp rx=0 tx=0).stop 2023-10-16T11:21:47.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 shutdown_connections 2023-10-16T11:21:47.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f191c06af90 0x7f191c06d450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19380d0fc0 0x7f19381f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19380ce610 0x7f193815f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 --2- 172.21.15.181:0/980895083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19380ce9f0 0x7f1938160350 unknown :-1 s=CLOSED pgs=3068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 >> 172.21.15.181:0/980895083 conn(0x7f19380d04a0 msgr2=0x7f19380d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:47.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 shutdown_connections 2023-10-16T11:21:47.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.041+0000 7f193f30c700 1 -- 172.21.15.181:0/980895083 wait complete. 2023-10-16T11:21:47.046 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:21:47.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:21:47.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.181+0000 7f4951a09700 1 Processor -- start 2023-10-16T11:21:47.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.181+0000 7f4951a09700 1 -- start start 2023-10-16T11:21:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4951a09700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f494c072760 0x7f494c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4951a09700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 0x7f494c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4951a09700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494c06b210 con 0x7f494c15da70 2023-10-16T11:21:47.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4951a09700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494c0cef00 con 0x7f494c072760 2023-10-16T11:21:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4951a09700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0cec20 con 0x7f494c06ade0 2023-10-16T11:21:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f494affd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f494c15da70 0x7f494c0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43844/0 (socket says 172.21.15.181:43844) 2023-10-16T11:21:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f494affd700 1 -- 172.21.15.181:0/1332557105 learned_addr learned my addr 172.21.15.181:0/1332557105 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f494a7fc700 1 --2- 172.21.15.181:0/1332557105 >> v2:172.21.15.190:3300/0 conn(0x7f494c072760 0x7f494c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f494b7fe700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 0x7f494c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850071700 0 0) 0x7f494c06b210 con 0x7f494c15da70 2023-10-16T11:21:47.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49340035b0 con 0x7f494c15da70 2023-10-16T11:21:47.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4287615441 0 0) 0x7f49340035b0 con 0x7f494c15da70 2023-10-16T11:21:47.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 msgr2=0x7f494c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:47.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 0x7f494c1615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 >> v2:172.21.15.190:3300/0 conn(0x7f494c072760 msgr2=0x7f494c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 --2- 172.21.15.181:0/1332557105 >> v2:172.21.15.190:3300/0 conn(0x7f494c072760 0x7f494c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c071c80 con 0x7f494c15da70 2023-10-16T11:21:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f493c001670 con 0x7f494c15da70 2023-10-16T11:21:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 >> v1:172.21.15.181:6789/0 conn(0x7f494c15da70 legacy=0x7f494c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4934003a00 0x7f4934003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934004850 0x7f4934004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 0x7f4934005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0cef00 con 0x7f4934003a00 2023-10-16T11:21:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0cec20 con 0x7f4934004850 2023-10-16T11:21:47.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.182+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493400bac0 con 0x7f494c06ade0 2023-10-16T11:21:47.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494affd700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4934003a00 0x7f4934003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494a7fc700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934004850 0x7f4934004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494b7fe700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 0x7f4934005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494affd700 1 -- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 msgr2=0x7f4934005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:47.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494affd700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c06ade0 0x7f4934005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494affd700 1 -- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934004850 msgr2=0x7f4934004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:47.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494affd700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934004850 0x7f4934004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494affd700 1 -- 172.21.15.181:0/1332557105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340035b0 con 0x7f4934003a00 2023-10-16T11:21:47.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f494affd700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4934003a00 0x7f4934003e20 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c370 tx=0x7f4934005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9423c93f2255ccb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c007c60 con 0x7f4934003a00 2023-10-16T11:21:47.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c005960 con 0x7f4934003a00 2023-10-16T11:21:47.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.183+0000 7f4949ffb700 1 -- 172.21.15.181:0/1332557105 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c005d10 con 0x7f4934003a00 2023-10-16T11:21:47.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.184+0000 7f4951a09700 1 -- 172.21.15.181:0/1332557105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4934003a00 msgr2=0x7f4934003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.184+0000 7f4951a09700 1 --2- 172.21.15.181:0/1332557105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4934003a00 0x7f4934003e20 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c370 tx=0x7f4934005880 comp rx=0 tx=0).stop 2023-10-16T11:21:47.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.184+0000 7f494b7fe700 1 -- 172.21.15.181:0/1332557105 reap_dead start 2023-10-16T11:21:47.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.184+0000 7f4951a09700 1 -- 172.21.15.181:0/1332557105 shutdown_connections 2023-10-16T11:21:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.184+0000 7f4951a09700 1 -- 172.21.15.181:0/1332557105 >> 172.21.15.181:0/1332557105 conn(0x7f494c0cd010 msgr2=0x7f494c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.184+0000 7f4951a09700 1 -- 172.21.15.181:0/1332557105 shutdown_connections 2023-10-16T11:21:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.184+0000 7f4951a09700 1 -- 172.21.15.181:0/1332557105 wait complete. 2023-10-16T11:21:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.185+0000 7f4951a09700 1 Processor -- start 2023-10-16T11:21:47.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.185+0000 7f4951a09700 1 -- start start 2023-10-16T11:21:47.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4951a09700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934003a00 0x7f494c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4951a09700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f494c15b550 0x7f494c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4951a09700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c0c7660 0x7f494c0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4951a09700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c14c020 con 0x7f494c15b550 2023-10-16T11:21:47.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4951a09700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0c8020 con 0x7f4934003a00 2023-10-16T11:21:47.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4951a09700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c15e560 con 0x7f494c0c7660 2023-10-16T11:21:47.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f494c15b550 0x7f494c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f494c15b550 0x7f494c15b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32780/0 (socket says 172.21.15.181:32780) 2023-10-16T11:21:47.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 -- 172.21.15.181:0/2862007069 learned_addr learned my addr 172.21.15.181:0/2862007069 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:47.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494affd700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934003a00 0x7f494c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494b7fe700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c0c7660 0x7f494c0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 -- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c0c7660 msgr2=0x7f494c0c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c0c7660 0x7f494c0c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 -- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934003a00 msgr2=0x7f494c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934003a00 0x7f494c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 -- 172.21.15.181:0/2862007069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c15c5d0 con 0x7f494c15b550 2023-10-16T11:21:47.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f494a7fc700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f494c15b550 0x7f494c15b9d0 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7f4940011aa0 tx=0x7f4940011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4933fff700 1 -- 172.21.15.181:0/2862007069 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940011ee0 con 0x7f494c15b550 2023-10-16T11:21:47.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4933fff700 1 -- 172.21.15.181:0/2862007069 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494000f940 con 0x7f494c15b550 2023-10-16T11:21:47.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c15c010 con 0x7f494c15b550 2023-10-16T11:21:47.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.186+0000 7f4933fff700 1 -- 172.21.15.181:0/2862007069 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494000fd10 con 0x7f494c15b550 2023-10-16T11:21:47.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.187+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c15e1f0 con 0x7f494c15b550 2023-10-16T11:21:47.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.187+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c06f6c0 con 0x7f494c15b550 2023-10-16T11:21:47.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.188+0000 7f4933fff700 1 -- 172.21.15.181:0/2862007069 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 168) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4940014440 con 0x7f494c15b550 2023-10-16T11:21:47.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.188+0000 7f4933fff700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f493806af90 0x7f493806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.188+0000 7f494affd700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f493806af90 0x7f493806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.189+0000 7f4933fff700 1 -- 172.21.15.181:0/2862007069 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f4940089cf0 con 0x7f494c15b550 2023-10-16T11:21:47.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.189+0000 7f494affd700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f493806af90 0x7f493806d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f493c015040 tx=0x7f493c00a000 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.193+0000 7f4933fff700 1 -- 172.21.15.181:0/2862007069 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4940054cd0 con 0x7f494c15b550 2023-10-16T11:21:47.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.382+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f494c0cd670 con 0x7f494c15b550 2023-10-16T11:21:47.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.385+0000 7f4933fff700 1 -- 172.21.15.181:0/2862007069 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v91) v1 ==== 115+0+0 (secure 0 0 0) 0x7f4940022020 con 0x7f494c15b550 2023-10-16T11:21:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f493806af90 msgr2=0x7f493806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f493806af90 0x7f493806d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f493c015040 tx=0x7f493c00a000 comp rx=0 tx=0).stop 2023-10-16T11:21:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f494c15b550 msgr2=0x7f494c15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f494c15b550 0x7f494c15b9d0 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7f4940011aa0 tx=0x7f4940011db0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 shutdown_connections 2023-10-16T11:21:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f493806af90 0x7f493806d450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f494c0c7660 0x7f494c0c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4934003a00 0x7f494c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 --2- 172.21.15.181:0/2862007069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f494c15b550 0x7f494c15b9d0 unknown :-1 s=CLOSED pgs=3071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 >> 172.21.15.181:0/2862007069 conn(0x7f494c0cd010 msgr2=0x7f494c161560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 shutdown_connections 2023-10-16T11:21:47.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.388+0000 7f4951a09700 1 -- 172.21.15.181:0/2862007069 wait complete. 2023-10-16T11:21:47.403 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:21:47.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.526+0000 7fed42b40700 1 Processor -- start 2023-10-16T11:21:47.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.526+0000 7fed42b40700 1 -- start start 2023-10-16T11:21:47.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.526+0000 7fed42b40700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed3c15fee0 0x7fed3c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.526+0000 7fed42b40700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c06e4b0 con 0x7fed3c15b850 2023-10-16T11:21:47.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.526+0000 7fed42b40700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c06ed50 con 0x7fed3c0665a0 2023-10-16T11:21:47.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.526+0000 7fed42b40700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3c0cb0f0 con 0x7fed3c15fee0 2023-10-16T11:21:47.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed410dd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed3c15fee0 0x7fed3c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed408dc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fed3c15b850 0x7fed3c0c3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43864/0 (socket says 172.21.15.181:43864) 2023-10-16T11:21:47.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed408dc700 1 -- 172.21.15.181:0/2533307985 learned_addr learned my addr 172.21.15.181:0/2533307985 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:47.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed410dd700 1 -- 172.21.15.181:0/2533307985 >> v1:172.21.15.190:6789/0 conn(0x7fed3c0665a0 legacy=0x7fed3c06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:47.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed3b7fe700 1 -- 172.21.15.181:0/2533307985 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412097303 0 0) 0x7fed3c06e4b0 con 0x7fed3c15b850 2023-10-16T11:21:47.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed410dd700 1 -- 172.21.15.181:0/2533307985 >> v1:172.21.15.181:6789/0 conn(0x7fed3c15b850 legacy=0x7fed3c0c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed410dd700 1 -- 172.21.15.181:0/2533307985 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c066b80 con 0x7fed3c15fee0 2023-10-16T11:21:47.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed410dd700 1 --2- 172.21.15.181:0/2533307985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c0ca300 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7fed34009d90 tx=0x7fed3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0305050e7799e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed3b7fe700 1 -- 172.21.15.181:0/2533307985 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400e070 con 0x7fed3c15fee0 2023-10-16T11:21:47.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed3b7fe700 1 -- 172.21.15.181:0/2533307985 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3400bbd0 con 0x7fed3c15fee0 2023-10-16T11:21:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.527+0000 7fed3b7fe700 1 -- 172.21.15.181:0/2533307985 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34012430 con 0x7fed3c15fee0 2023-10-16T11:21:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.528+0000 7fed42b40700 1 -- 172.21.15.181:0/2533307985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 msgr2=0x7fed3c0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.528+0000 7fed42b40700 1 --2- 172.21.15.181:0/2533307985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c0ca300 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7fed34009d90 tx=0x7fed3400b330 comp rx=0 tx=0).stop 2023-10-16T11:21:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.528+0000 7fed42b40700 1 -- 172.21.15.181:0/2533307985 shutdown_connections 2023-10-16T11:21:47.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.528+0000 7fed42b40700 1 --2- 172.21.15.181:0/2533307985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c0ca300 unknown :-1 s=CLOSED pgs=3073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.528+0000 7fed42b40700 1 -- 172.21.15.181:0/2533307985 >> 172.21.15.181:0/2533307985 conn(0x7fed3c0cc730 msgr2=0x7fed3c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.528+0000 7fed42b40700 1 -- 172.21.15.181:0/2533307985 shutdown_connections 2023-10-16T11:21:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.528+0000 7fed42b40700 1 -- 172.21.15.181:0/2533307985 wait complete. 2023-10-16T11:21:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.529+0000 7fed42b40700 1 Processor -- start 2023-10-16T11:21:47.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.529+0000 7fed42b40700 1 -- start start 2023-10-16T11:21:47.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed42b40700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed3c0665a0 0x7fed3c150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed42b40700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed3c15b850 0x7fed3c1f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed42b40700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c1f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed42b40700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c149f30 con 0x7fed3c15fee0 2023-10-16T11:21:47.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed42b40700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c1f15f0 con 0x7fed3c0665a0 2023-10-16T11:21:47.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed42b40700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c149be0 con 0x7fed3c15b850 2023-10-16T11:21:47.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c1f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c1f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32796/0 (socket says 172.21.15.181:32796) 2023-10-16T11:21:47.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 -- 172.21.15.181:0/3287929004 learned_addr learned my addr 172.21.15.181:0/3287929004 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:47.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed408dc700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed3c0665a0 0x7fed3c150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed3bfff700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed3c15b850 0x7fed3c1f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 -- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed3c15b850 msgr2=0x7fed3c1f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed3c15b850 0x7fed3c1f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 -- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed3c0665a0 msgr2=0x7fed3c150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed3c0665a0 0x7fed3c150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 -- 172.21.15.181:0/3287929004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009a70 con 0x7fed3c15fee0 2023-10-16T11:21:47.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.530+0000 7fed410dd700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c1f1dc0 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7fed3c070de0 tx=0x7fed34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.531+0000 7fed397fa700 1 -- 172.21.15.181:0/3287929004 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3401b810 con 0x7fed3c15fee0 2023-10-16T11:21:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.531+0000 7fed397fa700 1 -- 172.21.15.181:0/3287929004 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed34020070 con 0x7fed3c15fee0 2023-10-16T11:21:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.531+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c152cd0 con 0x7fed3c15fee0 2023-10-16T11:21:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.531+0000 7fed397fa700 1 -- 172.21.15.181:0/3287929004 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3401bbf0 con 0x7fed3c15fee0 2023-10-16T11:21:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.531+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c153290 con 0x7fed3c15fee0 2023-10-16T11:21:47.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.532+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c153450 con 0x7fed3c15fee0 2023-10-16T11:21:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.532+0000 7fed397fa700 1 -- 172.21.15.181:0/3287929004 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fed3401bd50 con 0x7fed3c15fee0 2023-10-16T11:21:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.532+0000 7fed397fa700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fed2006b020 0x7fed2006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.532+0000 7fed397fa700 1 -- 172.21.15.181:0/3287929004 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7fed34022070 con 0x7fed3c15fee0 2023-10-16T11:21:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.533+0000 7fed408dc700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fed2006b020 0x7fed2006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.533+0000 7fed408dc700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fed2006b020 0x7fed2006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed30007b10 tx=0x7fed30007a60 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.537+0000 7fed397fa700 1 -- 172.21.15.181:0/3287929004 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fed3405aa50 con 0x7fed3c15fee0 2023-10-16T11:21:47.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.738+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fed3c15bc30 con 0x7fed3c15fee0 2023-10-16T11:21:47.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.741+0000 7fed397fa700 1 -- 172.21.15.181:0/3287929004 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v92) v1 ==== 120+0+0 (secure 0 0 0) 0x7fed3405a5e0 con 0x7fed3c15fee0 2023-10-16T11:21:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.743+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fed2006b020 msgr2=0x7fed2006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.743+0000 7fed42b40700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fed2006b020 0x7fed2006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed30007b10 tx=0x7fed30007a60 comp rx=0 tx=0).stop 2023-10-16T11:21:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.743+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 msgr2=0x7fed3c1f1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.743+0000 7fed42b40700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c1f1dc0 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 crypto rx=0x7fed3c070de0 tx=0x7fed34004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 shutdown_connections 2023-10-16T11:21:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3c15fee0 0x7fed3c1f1dc0 unknown :-1 s=CLOSED pgs=3074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed3c0665a0 0x7fed3c150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fed2006b020 0x7fed2006d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 --2- 172.21.15.181:0/3287929004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed3c15b850 0x7fed3c1f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 >> 172.21.15.181:0/3287929004 conn(0x7fed3c0cc730 msgr2=0x7fed3c15c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:47.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 shutdown_connections 2023-10-16T11:21:47.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.744+0000 7fed42b40700 1 -- 172.21.15.181:0/3287929004 wait complete. 2023-10-16T11:21:47.758 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:21:47.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.879+0000 7f81b1849700 1 Processor -- start 2023-10-16T11:21:47.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.879+0000 7f81b1849700 1 -- start start 2023-10-16T11:21:47.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81b1849700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f81ac159e50 0x7f81ac071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81b1849700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81ac15db30 0x7f81ac0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81b1849700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0cbed0 0x7f81ac0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81b1849700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac15dfe0 con 0x7f81ac159e50 2023-10-16T11:21:47.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81b1849700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac0cb390 con 0x7f81ac15db30 2023-10-16T11:21:47.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81b1849700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0cc880 con 0x7f81ac0cbed0 2023-10-16T11:21:47.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f81ac159e50 0x7f81ac071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f81ac159e50 0x7f81ac071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32800/0 (socket says 172.21.15.181:32800) 2023-10-16T11:21:47.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 -- 172.21.15.181:0/2235945516 learned_addr learned my addr 172.21.15.181:0/2235945516 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:47.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aa7fc700 1 --2- 172.21.15.181:0/2235945516 >> v2:172.21.15.190:3300/0 conn(0x7f81ac15db30 0x7f81ac0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81ab7fe700 1 --2- 172.21.15.181:0/2235945516 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0cbed0 0x7f81ac0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 -- 172.21.15.181:0/2235945516 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0cbed0 msgr2=0x7f81ac0cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:47.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 --2- 172.21.15.181:0/2235945516 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0cbed0 0x7f81ac0cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 -- 172.21.15.181:0/2235945516 >> v2:172.21.15.190:3300/0 conn(0x7f81ac15db30 msgr2=0x7f81ac0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:47.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 --2- 172.21.15.181:0/2235945516 >> v2:172.21.15.190:3300/0 conn(0x7f81ac15db30 0x7f81ac0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 -- 172.21.15.181:0/2235945516 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac15ea60 con 0x7f81ac159e50 2023-10-16T11:21:47.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.880+0000 7f81aaffd700 1 --2- 172.21.15.181:0/2235945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac159e50 0x7f81ac071460 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0cccc0 tx=0x7f81a000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b1c08858ea4b99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81a9ffb700 1 -- 172.21.15.181:0/2235945516 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a000dc70 con 0x7f81ac159e50 2023-10-16T11:21:47.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81a9ffb700 1 -- 172.21.15.181:0/2235945516 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a000ddd0 con 0x7f81ac159e50 2023-10-16T11:21:47.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81a9ffb700 1 -- 172.21.15.181:0/2235945516 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0010690 con 0x7f81ac159e50 2023-10-16T11:21:47.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 -- 172.21.15.181:0/2235945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac159e50 msgr2=0x7f81ac071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 --2- 172.21.15.181:0/2235945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac159e50 0x7f81ac071460 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0cccc0 tx=0x7f81a000be70 comp rx=0 tx=0).stop 2023-10-16T11:21:47.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 -- 172.21.15.181:0/2235945516 shutdown_connections 2023-10-16T11:21:47.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 --2- 172.21.15.181:0/2235945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac159e50 0x7f81ac071460 unknown :-1 s=CLOSED pgs=3075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 --2- 172.21.15.181:0/2235945516 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac0cbed0 0x7f81ac0cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 --2- 172.21.15.181:0/2235945516 >> v2:172.21.15.190:3300/0 conn(0x7f81ac15db30 0x7f81ac0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 -- 172.21.15.181:0/2235945516 >> 172.21.15.181:0/2235945516 conn(0x7f81ac06aed0 msgr2=0x7f81ac06cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:47.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 -- 172.21.15.181:0/2235945516 shutdown_connections 2023-10-16T11:21:47.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.881+0000 7f81b1849700 1 -- 172.21.15.181:0/2235945516 wait complete. 2023-10-16T11:21:47.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.882+0000 7f81b1849700 1 Processor -- start 2023-10-16T11:21:47.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81b1849700 1 -- start start 2023-10-16T11:21:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81b1849700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac0cbed0 0x7f81ac1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81b1849700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac159e50 0x7f81ac154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81b1849700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac15db30 0x7f81ac15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81b1849700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0c8070 con 0x7f81ac0cbed0 2023-10-16T11:21:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81b1849700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac15a770 con 0x7f81ac159e50 2023-10-16T11:21:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81b1849700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac155e40 con 0x7f81ac15db30 2023-10-16T11:21:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81aaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac0cbed0 0x7f81ac1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81aaffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac0cbed0 0x7f81ac1555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32806/0 (socket says 172.21.15.181:32806) 2023-10-16T11:21:47.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81aaffd700 1 -- 172.21.15.181:0/3938649407 learned_addr learned my addr 172.21.15.181:0/3938649407 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81aa7fc700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac159e50 0x7f81ac154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.883+0000 7f81ab7fe700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac15db30 0x7f81ac15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81aaffd700 1 -- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac15db30 msgr2=0x7f81ac15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81aaffd700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac15db30 0x7f81ac15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81aaffd700 1 -- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac159e50 msgr2=0x7f81ac154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:47.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81aaffd700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac159e50 0x7f81ac154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:47.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81aaffd700 1 -- 172.21.15.181:0/3938649407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a000b7f0 con 0x7f81ac0cbed0 2023-10-16T11:21:47.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81aaffd700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac0cbed0 0x7f81ac1555c0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0b20e0 tx=0x7f81a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f8193fff700 1 -- 172.21.15.181:0/3938649407 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a001f070 con 0x7f81ac0cbed0 2023-10-16T11:21:47.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f8193fff700 1 -- 172.21.15.181:0/3938649407 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a00038b0 con 0x7f81ac0cbed0 2023-10-16T11:21:47.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac0c7d00 con 0x7f81ac0cbed0 2023-10-16T11:21:47.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f8193fff700 1 -- 172.21.15.181:0/3938649407 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0003c40 con 0x7f81ac0cbed0 2023-10-16T11:21:47.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.884+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac15ac60 con 0x7f81ac0cbed0 2023-10-16T11:21:47.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.885+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac04ebb0 con 0x7f81ac0cbed0 2023-10-16T11:21:47.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.886+0000 7f8193fff700 1 -- 172.21.15.181:0/3938649407 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f81a0019400 con 0x7f81ac0cbed0 2023-10-16T11:21:47.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.886+0000 7f8193fff700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f819806b020 0x7f819806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:47.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.886+0000 7f8193fff700 1 -- 172.21.15.181:0/3938649407 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f81a008cc20 con 0x7f81ac0cbed0 2023-10-16T11:21:47.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.886+0000 7f81aa7fc700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f819806b020 0x7f819806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:47.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.886+0000 7f81aa7fc700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f819806b020 0x7f819806d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f81ac1596b0 tx=0x7f819c009210 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:47.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:47.891+0000 7f8193fff700 1 -- 172.21.15.181:0/3938649407 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f81a0057890 con 0x7f81ac0cbed0 2023-10-16T11:21:48.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.085+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f81ac0cd8f0 con 0x7f81ac0cbed0 2023-10-16T11:21:48.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.091+0000 7f8193fff700 1 -- 172.21.15.181:0/3938649407 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81a005aeb0 con 0x7f81ac0cbed0 2023-10-16T11:21:48.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f819806b020 msgr2=0x7f819806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:48.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f819806b020 0x7f819806d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f81ac1596b0 tx=0x7f819c009210 comp rx=0 tx=0).stop 2023-10-16T11:21:48.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac0cbed0 msgr2=0x7f81ac1555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:48.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac0cbed0 0x7f81ac1555c0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0b20e0 tx=0x7f81a0004750 comp rx=0 tx=0).stop 2023-10-16T11:21:48.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 shutdown_connections 2023-10-16T11:21:48.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f819806b020 0x7f819806d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81ac15db30 0x7f81ac15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81ac0cbed0 0x7f81ac1555c0 unknown :-1 s=CLOSED pgs=3076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 --2- 172.21.15.181:0/3938649407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81ac159e50 0x7f81ac154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 >> 172.21.15.181:0/3938649407 conn(0x7f81ac06aed0 msgr2=0x7f81ac0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:48.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 shutdown_connections 2023-10-16T11:21:48.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.093+0000 7f81b1849700 1 -- 172.21.15.181:0/3938649407 wait complete. 2023-10-16T11:21:48.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-16T11:21:48.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.226+0000 7f4728e22700 1 Processor -- start 2023-10-16T11:21:48.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.226+0000 7f4728e22700 1 -- start start 2023-10-16T11:21:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4728e22700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415e6d0 0x7f47240d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4728e22700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f472415eab0 0x7f472415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:48.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4728e22700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472406ed00 con 0x7f47240cb0c0 2023-10-16T11:21:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4728e22700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47240d02d0 con 0x7f472415eab0 2023-10-16T11:21:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4728e22700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472415ad40 con 0x7f472415e6d0 2023-10-16T11:21:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f472259c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415e6d0 0x7f47240d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4721d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f472415eab0 0x7f472415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:48.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4721d9b700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f472415eab0 0x7f472415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:37864/0 (socket says 172.21.15.181:37864) 2023-10-16T11:21:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.227+0000 7f4721d9b700 1 -- 172.21.15.181:0/1242618624 learned_addr learned my addr 172.21.15.181:0/1242618624 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f4721d9b700 1 -- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415e6d0 msgr2=0x7f47240d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f4721d9b700 1 --2- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415e6d0 0x7f47240d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f4721d9b700 1 -- 172.21.15.181:0/1242618624 >> v1:172.21.15.190:6789/0 conn(0x7f47240cb0c0 legacy=0x7f472415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:48.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f4721d9b700 1 -- 172.21.15.181:0/1242618624 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472406e0f0 con 0x7f472415eab0 2023-10-16T11:21:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f472159a700 1 -- 172.21.15.181:0/1242618624 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860495836 0 0) 0x7f472406ed00 con 0x7f47240cb0c0 2023-10-16T11:21:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f472259c700 1 --2- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415e6d0 0x7f47240d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f4721d9b700 1 --2- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472415eab0 0x7f472415a0f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f47140098f0 tx=0x7f4714009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b44b2c8695894ab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f472159a700 1 -- 172.21.15.181:0/1242618624 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714011830 con 0x7f472415eab0 2023-10-16T11:21:48.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f472159a700 1 -- 172.21.15.181:0/1242618624 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714011990 con 0x7f472415eab0 2023-10-16T11:21:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.228+0000 7f472159a700 1 -- 172.21.15.181:0/1242618624 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714011d20 con 0x7f472415eab0 2023-10-16T11:21:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 -- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472415eab0 msgr2=0x7f472415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 --2- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472415eab0 0x7f472415a0f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f47140098f0 tx=0x7f4714009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 -- 172.21.15.181:0/1242618624 shutdown_connections 2023-10-16T11:21:48.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 --2- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415e6d0 0x7f47240d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 --2- 172.21.15.181:0/1242618624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f472415eab0 0x7f472415a0f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 -- 172.21.15.181:0/1242618624 >> 172.21.15.181:0/1242618624 conn(0x7f47240d04a0 msgr2=0x7f47240d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:48.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 -- 172.21.15.181:0/1242618624 shutdown_connections 2023-10-16T11:21:48.236 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.229+0000 7f4728e22700 1 -- 172.21.15.181:0/1242618624 wait complete. 2023-10-16T11:21:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.230+0000 7f4728e22700 1 Processor -- start 2023-10-16T11:21:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.230+0000 7f4728e22700 1 -- start start 2023-10-16T11:21:48.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4728e22700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47240cb0c0 0x7f47240c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4728e22700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f472415e6d0 0x7f4724157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4728e22700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415eab0 0x7f47240b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4728e22700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240b5080 con 0x7f472415e6d0 2023-10-16T11:21:48.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4728e22700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240b5360 con 0x7f47240cb0c0 2023-10-16T11:21:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4728e22700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c5fc0 con 0x7f472415eab0 2023-10-16T11:21:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f472415e6d0 0x7f4724157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f472415e6d0 0x7f4724157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32820/0 (socket says 172.21.15.181:32820) 2023-10-16T11:21:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 -- 172.21.15.181:0/2522422135 learned_addr learned my addr 172.21.15.181:0/2522422135 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:48.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f472259c700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47240cb0c0 0x7f47240c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4722d9d700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415eab0 0x7f47240b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 -- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415eab0 msgr2=0x7f47240b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415eab0 0x7f47240b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 -- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47240cb0c0 msgr2=0x7f47240c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:48.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47240cb0c0 0x7f47240c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 -- 172.21.15.181:0/2522422135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718009430 con 0x7f472415e6d0 2023-10-16T11:21:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f472259c700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47240cb0c0 0x7f47240c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.231+0000 7f4721d9b700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f472415e6d0 0x7f4724157460 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 crypto rx=0x7f4714009c40 tx=0x7f4714004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.232+0000 7f4712ffd700 1 -- 172.21.15.181:0/2522422135 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714004550 con 0x7f472415e6d0 2023-10-16T11:21:48.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.232+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4714009550 con 0x7f472415e6d0 2023-10-16T11:21:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.232+0000 7f4712ffd700 1 -- 172.21.15.181:0/2522422135 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471401f070 con 0x7f472415e6d0 2023-10-16T11:21:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.232+0000 7f4712ffd700 1 -- 172.21.15.181:0/2522422135 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140038e0 con 0x7f472415e6d0 2023-10-16T11:21:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.232+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4724150cb0 con 0x7f472415e6d0 2023-10-16T11:21:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.233+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47240c5e20 con 0x7f472415e6d0 2023-10-16T11:21:48.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.233+0000 7f4712ffd700 1 -- 172.21.15.181:0/2522422135 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4714003a40 con 0x7f472415e6d0 2023-10-16T11:21:48.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.233+0000 7f4712ffd700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f470406b020 0x7f470406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:48.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.234+0000 7f4712ffd700 1 -- 172.21.15.181:0/2522422135 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f4714021070 con 0x7f472415e6d0 2023-10-16T11:21:48.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.234+0000 7f472259c700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f470406b020 0x7f470406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:48.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.234+0000 7f472259c700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f470406b020 0x7f470406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f472415e060 tx=0x7f47180091c0 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:48.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.238+0000 7f4712ffd700 1 -- 172.21.15.181:0/2522422135 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f47240c5e20 con 0x7f472415e6d0 2023-10-16T11:21:48.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:48.510+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f47240c64e0 con 0x7f472415e6d0 2023-10-16T11:21:49.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.041+0000 7f4712ffd700 1 -- 172.21.15.181:0/2522422135 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3233) v1 ==== 155+0+0 (secure 0 0 0) 0x7f47240c64e0 con 0x7f472415e6d0 2023-10-16T11:21:49.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f470406b020 msgr2=0x7f470406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f470406b020 0x7f470406d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f472415e060 tx=0x7f47180091c0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f472415e6d0 msgr2=0x7f4724157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f472415e6d0 0x7f4724157460 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 crypto rx=0x7f4714009c40 tx=0x7f4714004750 comp rx=0 tx=0).stop 2023-10-16T11:21:49.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 shutdown_connections 2023-10-16T11:21:49.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f470406b020 0x7f470406d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f472415eab0 0x7f47240b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47240cb0c0 0x7f47240c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 --2- 172.21.15.181:0/2522422135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f472415e6d0 0x7f4724157460 unknown :-1 s=CLOSED pgs=3077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 >> 172.21.15.181:0/2522422135 conn(0x7f47240d04a0 msgr2=0x7f47240cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:49.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 shutdown_connections 2023-10-16T11:21:49.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.044+0000 7f4728e22700 1 -- 172.21.15.181:0/2522422135 wait complete. 2023-10-16T11:21:49.059 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-10-16T11:21:49.059 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) Test subvol and group arguments to snap-schedule cli. 2023-10-16T11:21:49.060 INFO:tasks.cephfs_test_runner:test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:21:49.060 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:49.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.180+0000 7fd4e2b80700 1 Processor -- start 2023-10-16T11:21:49.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.180+0000 7fd4e2b80700 1 -- start start 2023-10-16T11:21:49.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e2b80700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd4dc0ce7f0 0x7fd4dc06c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e2b80700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0d2390 0x7fd4dc0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e2b80700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc0cc470 con 0x7fd4dc0d0180 2023-10-16T11:21:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e2b80700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc0d0610 con 0x7fd4dc0ce7f0 2023-10-16T11:21:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e2b80700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc06ca30 con 0x7fd4dc0d2390 2023-10-16T11:21:49.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e091c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd4dc0ce7f0 0x7fd4dc06c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e111d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd4dc0d0180 0x7fd4dc161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43906/0 (socket says 172.21.15.181:43906) 2023-10-16T11:21:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e111d700 1 -- 172.21.15.181:0/2028880566 learned_addr learned my addr 172.21.15.181:0/2028880566 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4dbfff700 1 --2- 172.21.15.181:0/2028880566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0d2390 0x7fd4dc0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e091c700 1 -- 172.21.15.181:0/2028880566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0d2390 msgr2=0x7fd4dc0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:49.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e091c700 1 --2- 172.21.15.181:0/2028880566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0d2390 0x7fd4dc0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e091c700 1 -- 172.21.15.181:0/2028880566 >> v1:172.21.15.181:6789/0 conn(0x7fd4dc0d0180 legacy=0x7fd4dc161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4db7fe700 1 -- 172.21.15.181:0/2028880566 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140223266 0 0) 0x7fd4dc0cc470 con 0x7fd4dc0d0180 2023-10-16T11:21:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e091c700 1 -- 172.21.15.181:0/2028880566 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc06cd10 con 0x7fd4dc0ce7f0 2023-10-16T11:21:49.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4dbfff700 1 --2- 172.21.15.181:0/2028880566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0d2390 0x7fd4dc0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4e091c700 1 --2- 172.21.15.181:0/2028880566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0ce7f0 0x7fd4dc06c100 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009820 tx=0x7fd4d0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77664779da60103 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4db7fe700 1 -- 172.21.15.181:0/2028880566 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0005e30 con 0x7fd4dc0ce7f0 2023-10-16T11:21:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4db7fe700 1 -- 172.21.15.181:0/2028880566 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d0006030 con 0x7fd4dc0ce7f0 2023-10-16T11:21:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.181+0000 7fd4db7fe700 1 -- 172.21.15.181:0/2028880566 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00063e0 con 0x7fd4dc0ce7f0 2023-10-16T11:21:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 -- 172.21.15.181:0/2028880566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0ce7f0 msgr2=0x7fd4dc06c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/2028880566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0ce7f0 0x7fd4dc06c100 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009820 tx=0x7fd4d0009b30 comp rx=0 tx=0).stop 2023-10-16T11:21:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 -- 172.21.15.181:0/2028880566 shutdown_connections 2023-10-16T11:21:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/2028880566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0ce7f0 0x7fd4dc06c100 unknown :-1 s=CLOSED pgs=3079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/2028880566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0d2390 0x7fd4dc0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 -- 172.21.15.181:0/2028880566 >> 172.21.15.181:0/2028880566 conn(0x7fd4dc09a9a0 msgr2=0x7fd4dc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:49.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 -- 172.21.15.181:0/2028880566 shutdown_connections 2023-10-16T11:21:49.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.182+0000 7fd4e2b80700 1 -- 172.21.15.181:0/2028880566 wait complete. 2023-10-16T11:21:49.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.183+0000 7fd4e2b80700 1 Processor -- start 2023-10-16T11:21:49.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.183+0000 7fd4e2b80700 1 -- start start 2023-10-16T11:21:49.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e2b80700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0ce7f0 0x7fd4dc0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e2b80700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc0d0180 0x7fd4dc0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e2b80700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0d2390 0x7fd4dc155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e2b80700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0b2270 con 0x7fd4dc0d2390 2023-10-16T11:21:49.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e2b80700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0b1310 con 0x7fd4dc0d0180 2023-10-16T11:21:49.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e2b80700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0b15f0 con 0x7fd4dc0ce7f0 2023-10-16T11:21:49.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e111d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0d2390 0x7fd4dc155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e111d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0d2390 0x7fd4dc155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32834/0 (socket says 172.21.15.181:32834) 2023-10-16T11:21:49.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e111d700 1 -- 172.21.15.181:0/260970351 learned_addr learned my addr 172.21.15.181:0/260970351 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:49.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4e091c700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0ce7f0 0x7fd4dc0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.184+0000 7fd4dbfff700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc0d0180 0x7fd4dc0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e111d700 1 -- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0ce7f0 msgr2=0x7fd4dc0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e111d700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0ce7f0 0x7fd4dc0ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e111d700 1 -- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc0d0180 msgr2=0x7fd4dc0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e111d700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc0d0180 0x7fd4dc0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e111d700 1 -- 172.21.15.181:0/260970351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0009430 con 0x7fd4dc0d2390 2023-10-16T11:21:49.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e091c700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0ce7f0 0x7fd4dc0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:49.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e111d700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0d2390 0x7fd4dc155650 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400cb60 tx=0x7fd4d400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4d97fa700 1 -- 172.21.15.181:0/260970351 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400ab50 con 0x7fd4dc0d2390 2023-10-16T11:21:49.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4d97fa700 1 -- 172.21.15.181:0/260970351 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d400acb0 con 0x7fd4dc0d2390 2023-10-16T11:21:49.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc0b18d0 con 0x7fd4dc0d2390 2023-10-16T11:21:49.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4d97fa700 1 -- 172.21.15.181:0/260970351 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4007310 con 0x7fd4dc0d2390 2023-10-16T11:21:49.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.185+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc0b1e90 con 0x7fd4dc0d2390 2023-10-16T11:21:49.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.186+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc067d20 con 0x7fd4dc0d2390 2023-10-16T11:21:49.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.186+0000 7fd4d97fa700 1 -- 172.21.15.181:0/260970351 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd4d4007470 con 0x7fd4dc0d2390 2023-10-16T11:21:49.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.187+0000 7fd4d97fa700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fd4c006b020 0x7fd4c006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.187+0000 7fd4d97fa700 1 -- 172.21.15.181:0/260970351 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7fd4d4089760 con 0x7fd4dc0d2390 2023-10-16T11:21:49.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.187+0000 7fd4e091c700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fd4c006b020 0x7fd4c006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.187+0000 7fd4e091c700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fd4c006b020 0x7fd4c006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0013040 tx=0x7fd4d0008070 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.191+0000 7fd4d97fa700 1 -- 172.21.15.181:0/260970351 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd4d40544d0 con 0x7fd4dc0d2390 2023-10-16T11:21:49.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.383+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4dc0c4ed0 con 0x7fd4dc0d2390 2023-10-16T11:21:49.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.384+0000 7fd4d97fa700 1 -- 172.21.15.181:0/260970351 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v261) v1 ==== 74+0+15026 (secure 0 0 0) 0x7fd4d4057af0 con 0x7fd4dc0d2390 2023-10-16T11:21:49.387 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:49.387 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":261,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:47.038916+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6833/3258038365":"2023-10-17T11:21:43.881713+0000","172.21.15.181:0/3430602932":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/452036432":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/3825373001":"2023-10-17T11:21:42.868291+0000","172.21.15.181:6833/3503716662":"2023-10-17T11:21:42.868291+0000","172.21.15.181:6832/3503716662":"2023-10-17T11:21:42.868291+0000","172.21.15.190:6835/3258038365":"2023-10-17T11:21:43.881713+0000","172.21.15.181:0/1323931653":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/3637292004":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3476229854":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/1366795220":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/2569889467":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6833/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:6832/3850286714":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/58286368":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/2946054209":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/1975124632":"2023-10-17T11:21:42.868291+0000","172.21.15.181:0/649394230":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/507502116":"2023-10-17T11:21:34.384612+0000","172.21.15.181:0/3665147670":"2023-10-17T11:21:34.384612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:49.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.386+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fd4c006b020 msgr2=0x7fd4c006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.386+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fd4c006b020 0x7fd4c006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0013040 tx=0x7fd4d0008070 comp rx=0 tx=0).stop 2023-10-16T11:21:49.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.386+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0d2390 msgr2=0x7fd4dc155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.386+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0d2390 0x7fd4dc155650 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400cb60 tx=0x7fd4d400cf20 comp rx=0 tx=0).stop 2023-10-16T11:21:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 shutdown_connections 2023-10-16T11:21:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fd4c006b020 0x7fd4c006d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0ce7f0 0x7fd4dc0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc0d2390 0x7fd4dc155650 unknown :-1 s=CLOSED pgs=3080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 --2- 172.21.15.181:0/260970351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc0d0180 0x7fd4dc0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 >> 172.21.15.181:0/260970351 conn(0x7fd4dc09a9a0 msgr2=0x7fd4dc0c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 shutdown_connections 2023-10-16T11:21:49.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.387+0000 7fd4e2b80700 1 -- 172.21.15.181:0/260970351 wait complete. 2023-10-16T11:21:49.401 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:21:49.402 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:21:49.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.523+0000 7f6ceb406700 1 Processor -- start 2023-10-16T11:21:49.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.523+0000 7f6ceb406700 1 -- start start 2023-10-16T11:21:49.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ceb406700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce415f150 0x7f6ce40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ceb406700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6ce40cd170 0x7f6ce40b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ceb406700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce40b2ae0 con 0x7f6ce40ce4f0 2023-10-16T11:21:49.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ceb406700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce40ce970 con 0x7f6ce40cd170 2023-10-16T11:21:49.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ceb406700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40b2dc0 con 0x7f6ce415f150 2023-10-16T11:21:49.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce99a3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6ce40cd170 0x7f6ce40b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce91a2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce415f150 0x7f6ce40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce89a1700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f6ce40ce4f0 0x7f6ce40b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:56090/0 (socket says 172.21.15.181:56090) 2023-10-16T11:21:49.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce89a1700 1 -- 172.21.15.181:0/1166242684 learned_addr learned my addr 172.21.15.181:0/1166242684 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:49.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce99a3700 1 -- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce415f150 msgr2=0x7f6ce40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:49.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce99a3700 1 --2- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce415f150 0x7f6ce40cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce99a3700 1 -- 172.21.15.181:0/1166242684 >> v1:172.21.15.190:6789/0 conn(0x7f6ce40ce4f0 legacy=0x7f6ce40b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce99a3700 1 -- 172.21.15.181:0/1166242684 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40cf9d0 con 0x7f6ce40cd170 2023-10-16T11:21:49.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6cdbfff700 1 -- 172.21.15.181:0/1166242684 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886867552 0 0) 0x7f6ce40b2ae0 con 0x7f6ce40ce4f0 2023-10-16T11:21:49.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.524+0000 7f6ce91a2700 1 --2- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce415f150 0x7f6ce40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:49.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ce99a3700 1 --2- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40cd170 0x7f6ce40b1670 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009d90 tx=0x7f6cdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533fabe7a5608368 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6cdbfff700 1 -- 172.21.15.181:0/1166242684 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00e070 con 0x7f6ce40cd170 2023-10-16T11:21:49.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6cdbfff700 1 -- 172.21.15.181:0/1166242684 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00bb20 con 0x7f6ce40cd170 2023-10-16T11:21:49.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6cdbfff700 1 -- 172.21.15.181:0/1166242684 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc012430 con 0x7f6ce40cd170 2023-10-16T11:21:49.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 -- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40cd170 msgr2=0x7f6ce40b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 --2- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40cd170 0x7f6ce40b1670 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009d90 tx=0x7f6cdc00b330 comp rx=0 tx=0).stop 2023-10-16T11:21:49.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 -- 172.21.15.181:0/1166242684 shutdown_connections 2023-10-16T11:21:49.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 --2- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40cd170 0x7f6ce40b1670 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 --2- 172.21.15.181:0/1166242684 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce415f150 0x7f6ce40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 -- 172.21.15.181:0/1166242684 >> 172.21.15.181:0/1166242684 conn(0x7f6ce409a930 msgr2=0x7f6ce40d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:49.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 -- 172.21.15.181:0/1166242684 shutdown_connections 2023-10-16T11:21:49.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.525+0000 7f6ceb406700 1 -- 172.21.15.181:0/1166242684 wait complete. 2023-10-16T11:21:49.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 Processor -- start 2023-10-16T11:21:49.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 -- start start 2023-10-16T11:21:49.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce40cd170 0x7f6ce40c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40ce4f0 0x7f6ce41f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce415f150 0x7f6ce415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40c5820 con 0x7f6ce415f150 2023-10-16T11:21:49.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce41f5550 con 0x7f6ce40ce4f0 2023-10-16T11:21:49.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ceb406700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40c81b0 con 0x7f6ce40cd170 2023-10-16T11:21:49.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ce99a3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce415f150 0x7f6ce415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ce99a3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce415f150 0x7f6ce415d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32848/0 (socket says 172.21.15.181:32848) 2023-10-16T11:21:49.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.527+0000 7f6ce99a3700 1 -- 172.21.15.181:0/527883706 learned_addr learned my addr 172.21.15.181:0/527883706 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:49.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce89a1700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40ce4f0 0x7f6ce41f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce91a2700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce40cd170 0x7f6ce40c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce99a3700 1 -- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce40cd170 msgr2=0x7f6ce40c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce99a3700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce40cd170 0x7f6ce40c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce99a3700 1 -- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40ce4f0 msgr2=0x7f6ce41f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce99a3700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40ce4f0 0x7f6ce41f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce99a3700 1 -- 172.21.15.181:0/527883706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00094a0 con 0x7f6ce415f150 2023-10-16T11:21:49.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ce99a3700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce415f150 0x7f6ce415d7a0 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc014040 tx=0x7f6cdc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6cd9ffb700 1 -- 172.21.15.181:0/527883706 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc012430 con 0x7f6ce415f150 2023-10-16T11:21:49.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6cd9ffb700 1 -- 172.21.15.181:0/527883706 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc020070 con 0x7f6ce415f150 2023-10-16T11:21:49.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc009a70 con 0x7f6ce415f150 2023-10-16T11:21:49.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6cd9ffb700 1 -- 172.21.15.181:0/527883706 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc004280 con 0x7f6ce415f150 2023-10-16T11:21:49.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.528+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce40c70d0 con 0x7f6ce415f150 2023-10-16T11:21:49.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.529+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce4067d20 con 0x7f6ce415f150 2023-10-16T11:21:49.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.530+0000 7f6cd9ffb700 1 -- 172.21.15.181:0/527883706 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6cdc0043e0 con 0x7f6ce415f150 2023-10-16T11:21:49.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.530+0000 7f6cd9ffb700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6cc806b020 0x7f6cc806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.530+0000 7f6cd9ffb700 1 -- 172.21.15.181:0/527883706 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f6cdc022070 con 0x7f6ce415f150 2023-10-16T11:21:49.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.530+0000 7f6ce91a2700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6cc806b020 0x7f6cc806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.531+0000 7f6ce91a2700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6cc806b020 0x7f6cc806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6ce41f4560 tx=0x7f6ce0009210 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.535+0000 7f6cd9ffb700 1 -- 172.21.15.181:0/527883706 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6cdc056360 con 0x7f6ce415f150 2023-10-16T11:21:49.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.728+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ce40c5960 con 0x7f6ce415f150 2023-10-16T11:21:49.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.728+0000 7f6cd9ffb700 1 -- 172.21.15.181:0/527883706 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v261) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6cdc059980 con 0x7f6ce415f150 2023-10-16T11:21:49.731 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:21:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6cc806b020 msgr2=0x7f6cc806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6cc806b020 0x7f6cc806d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6ce41f4560 tx=0x7f6ce0009210 comp rx=0 tx=0).stop 2023-10-16T11:21:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce415f150 msgr2=0x7f6ce415d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce415f150 0x7f6ce415d7a0 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc014040 tx=0x7f6cdc004ad0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 shutdown_connections 2023-10-16T11:21:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6cc806b020 0x7f6cc806d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ce40cd170 0x7f6ce40c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ce415f150 0x7f6ce415d7a0 unknown :-1 s=CLOSED pgs=3081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 --2- 172.21.15.181:0/527883706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ce40ce4f0 0x7f6ce41f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 >> 172.21.15.181:0/527883706 conn(0x7f6ce409a930 msgr2=0x7f6ce406ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:49.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 shutdown_connections 2023-10-16T11:21:49.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.731+0000 7f6ceb406700 1 -- 172.21.15.181:0/527883706 wait complete. 2023-10-16T11:21:49.745 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments' 2023-10-16T11:21:49.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.864+0000 7f7f9d907700 1 Processor -- start 2023-10-16T11:21:49.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.864+0000 7f7f9d907700 1 -- start start 2023-10-16T11:21:49.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f9d907700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f981621c0 0x7f7f980cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f9d907700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f980d1640 con 0x7f7f980cf690 2023-10-16T11:21:49.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f9d907700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f980d1920 con 0x7f7f9815d9b0 2023-10-16T11:21:49.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f9d907700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f980cfb40 con 0x7f7f981621c0 2023-10-16T11:21:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f967fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f981621c0 0x7f7f980cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f977fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7f980cf690 0x7f7f980cbcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43934/0 (socket says 172.21.15.181:43934) 2023-10-16T11:21:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f977fe700 1 -- 172.21.15.181:0/1910551690 learned_addr learned my addr 172.21.15.181:0/1910551690 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f967fc700 1 -- 172.21.15.181:0/1910551690 >> v1:172.21.15.190:6789/0 conn(0x7f7f9815d9b0 legacy=0x7f7f980b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f967fc700 1 -- 172.21.15.181:0/1910551690 >> v1:172.21.15.181:6789/0 conn(0x7f7f980cf690 legacy=0x7f7f980cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f967fc700 1 -- 172.21.15.181:0/1910551690 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f980d1c00 con 0x7f7f981621c0 2023-10-16T11:21:49.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f95ffb700 1 -- 172.21.15.181:0/1910551690 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271989393 0 0) 0x7f7f980d1640 con 0x7f7f980cf690 2023-10-16T11:21:49.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.865+0000 7f7f967fc700 1 --2- 172.21.15.181:0/1910551690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f981621c0 0x7f7f980cf150 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009840 tx=0x7f7f88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b33dee988109b9c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.871 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f95ffb700 1 -- 172.21.15.181:0/1910551690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88011830 con 0x7f7f981621c0 2023-10-16T11:21:49.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f95ffb700 1 -- 172.21.15.181:0/1910551690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88011990 con 0x7f7f981621c0 2023-10-16T11:21:49.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f95ffb700 1 -- 172.21.15.181:0/1910551690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88011d20 con 0x7f7f981621c0 2023-10-16T11:21:49.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f9d907700 1 -- 172.21.15.181:0/1910551690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f981621c0 msgr2=0x7f7f980cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f9d907700 1 --2- 172.21.15.181:0/1910551690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f981621c0 0x7f7f980cf150 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009840 tx=0x7f7f88009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:49.872 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f9d907700 1 -- 172.21.15.181:0/1910551690 shutdown_connections 2023-10-16T11:21:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f9d907700 1 --2- 172.21.15.181:0/1910551690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f981621c0 0x7f7f980cf150 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f9d907700 1 -- 172.21.15.181:0/1910551690 >> 172.21.15.181:0/1910551690 conn(0x7f7f980665a0 msgr2=0x7f7f980669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f9d907700 1 -- 172.21.15.181:0/1910551690 shutdown_connections 2023-10-16T11:21:49.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.866+0000 7f7f9d907700 1 -- 172.21.15.181:0/1910551690 wait complete. 2023-10-16T11:21:49.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.867+0000 7f7f9d907700 1 Processor -- start 2023-10-16T11:21:49.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f9d907700 1 -- start start 2023-10-16T11:21:49.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f9d907700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f980cf690 0x7f7f980b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f9d907700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9815d9b0 0x7f7f980b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f9d907700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f981621c0 0x7f7f980c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f9d907700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980c9be0 con 0x7f7f980cf690 2023-10-16T11:21:49.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f9d907700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980ca480 con 0x7f7f9815d9b0 2023-10-16T11:21:49.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f9d907700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980b71e0 con 0x7f7f981621c0 2023-10-16T11:21:49.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f96ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f980cf690 0x7f7f980b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f96ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f980cf690 0x7f7f980b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32860/0 (socket says 172.21.15.181:32860) 2023-10-16T11:21:49.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f96ffd700 1 -- 172.21.15.181:0/3192833075 learned_addr learned my addr 172.21.15.181:0/3192833075 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:49.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f967fc700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9815d9b0 0x7f7f980b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.868+0000 7f7f977fe700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f981621c0 0x7f7f980c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f96ffd700 1 -- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f981621c0 msgr2=0x7f7f980c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f96ffd700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f981621c0 0x7f7f980c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f96ffd700 1 -- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9815d9b0 msgr2=0x7f7f980b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:49.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f96ffd700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9815d9b0 0x7f7f980b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:49.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f96ffd700 1 -- 172.21.15.181:0/3192833075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880094a0 con 0x7f7f980cf690 2023-10-16T11:21:49.879 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f96ffd700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f980cf690 0x7f7f980b41a0 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c4e0 tx=0x7f7f8c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f7ffff700 1 -- 172.21.15.181:0/3192833075 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0052d0 con 0x7f7f980cf690 2023-10-16T11:21:49.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f7ffff700 1 -- 172.21.15.181:0/3192833075 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c005430 con 0x7f7f980cf690 2023-10-16T11:21:49.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f980b46e0 con 0x7f7f980cf690 2023-10-16T11:21:49.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f7ffff700 1 -- 172.21.15.181:0/3192833075 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0038c0 con 0x7f7f980cf690 2023-10-16T11:21:49.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.869+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f980c9e30 con 0x7f7f980cf690 2023-10-16T11:21:49.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.870+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f980c7f20 con 0x7f7f980cf690 2023-10-16T11:21:49.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.871+0000 7f7f7ffff700 1 -- 172.21.15.181:0/3192833075 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7f8c003a20 con 0x7f7f980cf690 2023-10-16T11:21:49.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.871+0000 7f7f7ffff700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f7f8406b020 0x7f7f8406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:49.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.871+0000 7f7f7ffff700 1 -- 172.21.15.181:0/3192833075 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f7f8c088c80 con 0x7f7f980cf690 2023-10-16T11:21:49.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.871+0000 7f7f967fc700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f7f8406b020 0x7f7f8406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:49.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.871+0000 7f7f967fc700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f7f8406b020 0x7f7f8406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7f880090b0 tx=0x7f7f8801c040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:49.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:49.875+0000 7f7f7ffff700 1 -- 172.21.15.181:0/3192833075 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f7f8c053a80 con 0x7f7f980cf690 2023-10-16T11:21:50.151 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:50.149+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]} v 0) v1 -- 0x7f7f9804ebb0 con 0x7f7f980cf690 2023-10-16T11:21:51.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.048+0000 7f7f7ffff700 1 -- 172.21.15.181:0/3192833075 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]}]=0 v3235) v1 ==== 201+0+0 (secure 0 0 0) 0x7f7f8c017110 con 0x7f7f980cf690 2023-10-16T11:21:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f7f8406b020 msgr2=0x7f7f8406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f7f8406b020 0x7f7f8406d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7f880090b0 tx=0x7f7f8801c040 comp rx=0 tx=0).stop 2023-10-16T11:21:51.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f980cf690 msgr2=0x7f7f980b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f980cf690 0x7f7f980b41a0 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c4e0 tx=0x7f7f8c00c7f0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 shutdown_connections 2023-10-16T11:21:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f7f8406b020 0x7f7f8406d4e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f981621c0 0x7f7f980c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f980cf690 0x7f7f980b41a0 unknown :-1 s=CLOSED pgs=3084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 --2- 172.21.15.181:0/3192833075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f9815d9b0 0x7f7f980b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 >> 172.21.15.181:0/3192833075 conn(0x7f7f980665a0 msgr2=0x7f7f980cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 shutdown_connections 2023-10-16T11:21:51.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.051+0000 7f7f9d907700 1 -- 172.21.15.181:0/3192833075 wait complete. 2023-10-16T11:21:51.068 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-16T11:21:51.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.196+0000 7fb7a6053700 1 Processor -- start 2023-10-16T11:21:51.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.196+0000 7fb7a6053700 1 -- start start 2023-10-16T11:21:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb7a6053700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7a015f150 0x7fb7a00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb7a6053700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a00b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb7a6053700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a00b2ae0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb7a6053700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a00ce970 con 0x7fb7a015f150 2023-10-16T11:21:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb7a6053700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00b2dc0 con 0x7fb7a00ce4f0 2023-10-16T11:21:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7a015f150 0x7fb7a00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb7a00b14a0 0x7fb7a00b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43948/0 (socket says 172.21.15.181:43948) 2023-10-16T11:21:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7a015f150 0x7fb7a00cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32866/0 (socket says 172.21.15.181:32866) 2023-10-16T11:21:51.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79ffff700 1 -- 172.21.15.181:0/2345037686 learned_addr learned my addr 172.21.15.181:0/2345037686 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79effd700 1 --2- 172.21.15.181:0/2345037686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a00b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79f7fe700 1 -- 172.21.15.181:0/2345037686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 msgr2=0x7fb7a00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79f7fe700 1 --2- 172.21.15.181:0/2345037686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a00b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79f7fe700 1 -- 172.21.15.181:0/2345037686 >> v1:172.21.15.181:6789/0 conn(0x7fb7a00b14a0 legacy=0x7fb7a00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79f7fe700 1 -- 172.21.15.181:0/2345037686 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a00cf9d0 con 0x7fb7a015f150 2023-10-16T11:21:51.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79e7fc700 1 -- 172.21.15.181:0/2345037686 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158235587 0 0) 0x7fb7a00b2ae0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79f7fe700 1 --2- 172.21.15.181:0/2345037686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a015f150 0x7fb7a00cdfb0 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7fb790009840 tx=0x7fb790009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257726d2cd4f9a13 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.197+0000 7fb79e7fc700 1 -- 172.21.15.181:0/2345037686 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790011830 con 0x7fb7a015f150 2023-10-16T11:21:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb79e7fc700 1 -- 172.21.15.181:0/2345037686 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790011990 con 0x7fb7a015f150 2023-10-16T11:21:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb79e7fc700 1 -- 172.21.15.181:0/2345037686 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790011d20 con 0x7fb7a015f150 2023-10-16T11:21:51.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 -- 172.21.15.181:0/2345037686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a015f150 msgr2=0x7fb7a00cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 --2- 172.21.15.181:0/2345037686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a015f150 0x7fb7a00cdfb0 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7fb790009840 tx=0x7fb790009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:51.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 -- 172.21.15.181:0/2345037686 shutdown_connections 2023-10-16T11:21:51.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 --2- 172.21.15.181:0/2345037686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a015f150 0x7fb7a00cdfb0 unknown :-1 s=CLOSED pgs=3086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 --2- 172.21.15.181:0/2345037686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a00b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 -- 172.21.15.181:0/2345037686 >> 172.21.15.181:0/2345037686 conn(0x7fb7a009a930 msgr2=0x7fb7a00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:51.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 -- 172.21.15.181:0/2345037686 shutdown_connections 2023-10-16T11:21:51.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.198+0000 7fb7a6053700 1 -- 172.21.15.181:0/2345037686 wait complete. 2023-10-16T11:21:51.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.199+0000 7fb7a6053700 1 Processor -- start 2023-10-16T11:21:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb7a6053700 1 -- start start 2023-10-16T11:21:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb7a6053700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a00b14a0 0x7fb7a006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb7a6053700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb7a6053700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7a015f150 0x7fb7a00c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb7a6053700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a006c8f0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb7a6053700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a006d860 con 0x7fb7a015f150 2023-10-16T11:21:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb7a6053700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a01f2c20 con 0x7fb7a00ce4f0 2023-10-16T11:21:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb79f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a00b14a0 0x7fb7a006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb79f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a00b14a0 0x7fb7a006c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32872/0 (socket says 172.21.15.181:32872) 2023-10-16T11:21:51.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.200+0000 7fb79f7fe700 1 -- 172.21.15.181:0/3770752006 learned_addr learned my addr 172.21.15.181:0/3770752006 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:51.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79effd700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79ffff700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7a015f150 0x7fb7a00c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79f7fe700 1 -- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 msgr2=0x7fb7a006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79f7fe700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a006d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79f7fe700 1 -- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7a015f150 msgr2=0x7fb7a00c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79f7fe700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7a015f150 0x7fb7a00c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79f7fe700 1 -- 172.21.15.181:0/3770752006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7900094a0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79effd700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a006d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb79f7fe700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a00b14a0 0x7fb7a006c3b0 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7fb790000c00 tx=0x7fb790004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb787fff700 1 -- 172.21.15.181:0/3770752006 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900038a0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb787fff700 1 -- 172.21.15.181:0/3770752006 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79001e070 con 0x7fb7a00b14a0 2023-10-16T11:21:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a01f2f00 con 0x7fb7a00b14a0 2023-10-16T11:21:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb787fff700 1 -- 172.21.15.181:0/3770752006 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790003c20 con 0x7fb7a00b14a0 2023-10-16T11:21:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.201+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a01f34c0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.202+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a0067d20 con 0x7fb7a00b14a0 2023-10-16T11:21:51.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.202+0000 7fb787fff700 1 -- 172.21.15.181:0/3770752006 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb79002d450 con 0x7fb7a00b14a0 2023-10-16T11:21:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.203+0000 7fb787fff700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb78c06b020 0x7fb78c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.203+0000 7fb787fff700 1 -- 172.21.15.181:0/3770752006 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7fb79008aab0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.203+0000 7fb79effd700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb78c06b020 0x7fb78c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.203+0000 7fb79effd700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb78c06b020 0x7fb78c06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb7a01f5e20 tx=0x7fb794008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:51.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.207+0000 7fb787fff700 1 -- 172.21.15.181:0/3770752006 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb7900558b0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.402 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.400+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7a006dce0 con 0x7fb7a00b14a0 2023-10-16T11:21:51.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.403+0000 7fb787fff700 1 -- 172.21.15.181:0/3770752006 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v94)=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb790015020 con 0x7fb7a00b14a0 2023-10-16T11:21:51.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb78c06b020 msgr2=0x7fb78c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb78c06b020 0x7fb78c06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb7a01f5e20 tx=0x7fb794008040 comp rx=0 tx=0).stop 2023-10-16T11:21:51.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a00b14a0 msgr2=0x7fb7a006c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a00b14a0 0x7fb7a006c3b0 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7fb790000c00 tx=0x7fb790004750 comp rx=0 tx=0).stop 2023-10-16T11:21:51.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 shutdown_connections 2023-10-16T11:21:51.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7a015f150 0x7fb7a00c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7a00b14a0 0x7fb7a006c3b0 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fb78c06b020 0x7fb78c06d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 --2- 172.21.15.181:0/3770752006 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7a00ce4f0 0x7fb7a006d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.405+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 >> 172.21.15.181:0/3770752006 conn(0x7fb7a009a930 msgr2=0x7fb7a00ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:51.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.406+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 shutdown_connections 2023-10-16T11:21:51.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.406+0000 7fb7a6053700 1 -- 172.21.15.181:0/3770752006 wait complete. 2023-10-16T11:21:51.420 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:21:51.420 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:21:51.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:21:51.520 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:21:51.521 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:51.521 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:51.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:21:51.539 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:21:51.539 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:51.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:21:51.559 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:21:51.560 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:51.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:21:51.596 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:21:51.597 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:21:51.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:21:51.618 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-16T11:21:51.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:21:51.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.755+0000 7f60397f0700 1 Processor -- start 2023-10-16T11:21:51.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.755+0000 7f60397f0700 1 -- start start 2023-10-16T11:21:51.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.755+0000 7f60397f0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6034161b60 0x7f603406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.755+0000 7f60397f0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 0x7f60341611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.755+0000 7f60397f0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603406ce20 con 0x7f6034161f40 2023-10-16T11:21:51.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.755+0000 7f60397f0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6034162800 con 0x7f6034161b60 2023-10-16T11:21:51.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.755+0000 7f60397f0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340d43e0 con 0x7f6034160d20 2023-10-16T11:21:51.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f60327fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6034161f40 0x7f60340d2020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:43958/0 (socket says 172.21.15.181:43958) 2023-10-16T11:21:51.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f60327fc700 1 -- 172.21.15.181:0/489384746 learned_addr learned my addr 172.21.15.181:0/489384746 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:51.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f60337fe700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 0x7f60341611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6032ffd700 1 --2- 172.21.15.181:0/489384746 >> v2:172.21.15.190:3300/0 conn(0x7f6034161b60 0x7f603406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401538931 0 0) 0x7f603406ce20 con 0x7f6034161f40 2023-10-16T11:21:51.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60200035b0 con 0x7f6034161f40 2023-10-16T11:21:51.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 479104165 0 0) 0x7f60200035b0 con 0x7f6034161f40 2023-10-16T11:21:51.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 msgr2=0x7f60341611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:51.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 0x7f60341611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 >> v2:172.21.15.190:3300/0 conn(0x7f6034161b60 msgr2=0x7f603406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:51.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 --2- 172.21.15.181:0/489384746 >> v2:172.21.15.190:3300/0 conn(0x7f6034161b60 0x7f603406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60340d45f0 con 0x7f6034161f40 2023-10-16T11:21:51.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f601c001670 con 0x7f6034161f40 2023-10-16T11:21:51.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 >> v1:172.21.15.181:6789/0 conn(0x7f6034161f40 legacy=0x7f60340d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6020003a00 0x7f6020003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 0x7f6020004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020004d90 0x7f6020005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6034162800 con 0x7f6020003a00 2023-10-16T11:21:51.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340d43e0 con 0x7f6020004d90 2023-10-16T11:21:51.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602000ba40 con 0x7f6034160d20 2023-10-16T11:21:51.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f6032ffd700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6020003a00 0x7f6020003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f60327fc700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 0x7f6020004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.756+0000 7f60337fe700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020004d90 0x7f6020005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6032ffd700 1 -- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 msgr2=0x7f6020004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:51.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6032ffd700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6034160d20 0x7f6020004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6032ffd700 1 -- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020004d90 msgr2=0x7f6020005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6032ffd700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020004d90 0x7f6020005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6032ffd700 1 -- 172.21.15.181:0/489384746 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60200035b0 con 0x7f6020003a00 2023-10-16T11:21:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6032ffd700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6020003a00 0x7f6020003e20 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f6024009b40 tx=0x7f6020005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ba0c58281951cb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:51.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024005a20 con 0x7f6020003a00 2023-10-16T11:21:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6024005b80 con 0x7f6020003a00 2023-10-16T11:21:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f6031ffb700 1 -- 172.21.15.181:0/489384746 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602400e6c0 con 0x7f6020003a00 2023-10-16T11:21:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f60397f0700 1 -- 172.21.15.181:0/489384746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6020003a00 msgr2=0x7f6020003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f60397f0700 1 --2- 172.21.15.181:0/489384746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6020003a00 0x7f6020003e20 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f6024009b40 tx=0x7f6020005880 comp rx=0 tx=0).stop 2023-10-16T11:21:51.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f60337fe700 1 -- 172.21.15.181:0/489384746 reap_dead start 2023-10-16T11:21:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f60397f0700 1 -- 172.21.15.181:0/489384746 shutdown_connections 2023-10-16T11:21:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f60397f0700 1 -- 172.21.15.181:0/489384746 >> 172.21.15.181:0/489384746 conn(0x7f603409cba0 msgr2=0x7f60340d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f60397f0700 1 -- 172.21.15.181:0/489384746 shutdown_connections 2023-10-16T11:21:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.757+0000 7f60397f0700 1 -- 172.21.15.181:0/489384746 wait complete. 2023-10-16T11:21:51.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.759+0000 7f60397f0700 1 Processor -- start 2023-10-16T11:21:51.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.759+0000 7f60397f0700 1 -- start start 2023-10-16T11:21:51.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.759+0000 7f60397f0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020003a00 0x7f603414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.759+0000 7f60397f0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f603414f470 0x7f60340b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.759+0000 7f60397f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603414f8f0 0x7f60340d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60397f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340c93b0 con 0x7f603414f8f0 2023-10-16T11:21:51.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60397f0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340c8ac0 con 0x7f6020003a00 2023-10-16T11:21:51.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60397f0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340b7d50 con 0x7f603414f470 2023-10-16T11:21:51.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603414f8f0 0x7f60340d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603414f8f0 0x7f60340d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32894/0 (socket says 172.21.15.181:32894) 2023-10-16T11:21:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 -- 172.21.15.181:0/4081588798 learned_addr learned my addr 172.21.15.181:0/4081588798 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 -- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f603414f470 msgr2=0x7f60340b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f603414f470 0x7f60340b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f6032ffd700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020003a00 0x7f603414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 -- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020003a00 msgr2=0x7f603414ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020003a00 0x7f603414ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 -- 172.21.15.181:0/4081588798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60340c8d60 con 0x7f603414f8f0 2023-10-16T11:21:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f6032ffd700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020003a00 0x7f603414ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:51.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60337fe700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603414f8f0 0x7f60340d00a0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7f60340d42b0 tx=0x7f602c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f602b7fe700 1 -- 172.21.15.181:0/4081588798 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00e040 con 0x7f603414f8f0 2023-10-16T11:21:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f602b7fe700 1 -- 172.21.15.181:0/4081588798 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c00f040 con 0x7f603414f8f0 2023-10-16T11:21:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60340c9040 con 0x7f603414f8f0 2023-10-16T11:21:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f602b7fe700 1 -- 172.21.15.181:0/4081588798 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00fd00 con 0x7f603414f8f0 2023-10-16T11:21:51.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.760+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60341521c0 con 0x7f603414f8f0 2023-10-16T11:21:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.761+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60340c91d0 con 0x7f603414f8f0 2023-10-16T11:21:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.762+0000 7f602b7fe700 1 -- 172.21.15.181:0/4081588798 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f602c015430 con 0x7f603414f8f0 2023-10-16T11:21:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.762+0000 7f602b7fe700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f601406b020 0x7f601406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.762+0000 7f602b7fe700 1 -- 172.21.15.181:0/4081588798 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f602c08a260 con 0x7f603414f8f0 2023-10-16T11:21:51.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.762+0000 7f6032ffd700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f601406b020 0x7f601406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:51.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.762+0000 7f6032ffd700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f601406b020 0x7f601406d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f602400a010 tx=0x7f6024008030 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:51.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:51.766+0000 7f602b7fe700 1 -- 172.21.15.181:0/4081588798 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f602c054f50 con 0x7f603414f8f0 2023-10-16T11:21:52.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.003+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f603404ebb0 con 0x7f603414f8f0 2023-10-16T11:21:52.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.004+0000 7f602b7fe700 1 -- 172.21.15.181:0/4081588798 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+3720 (secure 0 0 0) 0x7f602c058570 con 0x7f603414f8f0 2023-10-16T11:21:52.007 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:52.007 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":146,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11314,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3049755410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3049755410},{"type":"v1","addr":"172.21.15.190:6834","nonce":3049755410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":11662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1125502501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":1125502501},{"type":"v1","addr":"172.21.15.190:6835","nonce":1125502501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":146},{"gid":12915,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2185474337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2185474337},{"type":"v1","addr":"172.21.15.181:6837","nonce":2185474337}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2846112501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2846112501},{"type":"v1","addr":"172.21.15.181:6836","nonce":2846112501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[]} 2023-10-16T11:21:52.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f601406b020 msgr2=0x7f601406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:52.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f601406b020 0x7f601406d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f602400a010 tx=0x7f6024008030 comp rx=0 tx=0).stop 2023-10-16T11:21:52.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603414f8f0 msgr2=0x7f60340d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:52.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603414f8f0 0x7f60340d00a0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7f60340d42b0 tx=0x7f602c011ed0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 shutdown_connections 2023-10-16T11:21:52.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f603414f8f0 0x7f60340d00a0 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6020003a00 0x7f603414ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f601406b020 0x7f601406d4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 --2- 172.21.15.181:0/4081588798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f603414f470 0x7f60340b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 >> 172.21.15.181:0/4081588798 conn(0x7f603409cba0 msgr2=0x7f603415db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:52.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 shutdown_connections 2023-10-16T11:21:52.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.006+0000 7f60397f0700 1 -- 172.21.15.181:0/4081588798 wait complete. 2023-10-16T11:21:52.011 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 146 2023-10-16T11:21:52.020 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T11:21:52.021 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-16T11:21:52.021 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:21:52.021 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:52.022 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T11:21:52.022 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-16T11:21:52.022 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:21:52.023 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:52.023 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T11:21:52.023 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-16T11:21:52.023 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:21:52.024 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:52.024 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T11:21:52.024 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-16T11:21:52.025 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:21:52.025 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:21:52.030 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:21:52.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T11:21:52.031 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:21:52.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T11:21:52.034 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:21:52.034 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T11:21:52.035 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:21:52.036 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T11:21:52.073 INFO:tasks.ceph.mds.a:Started 2023-10-16T11:21:52.074 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:52.075 INFO:tasks.ceph.mds.c:Started 2023-10-16T11:21:52.075 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:52.077 INFO:tasks.ceph.mds.b:Started 2023-10-16T11:21:52.078 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:52.079 INFO:tasks.ceph.mds.d:Started 2023-10-16T11:21:52.079 DEBUG:teuthology.parallel:result is None 2023-10-16T11:21:52.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-16T11:21:52.152 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2023-10-16T11:21:52.153 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2023-10-16T11:21:52.163 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:21:52.163 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-10-16T11:21:52.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.209+0000 7f2e1e46d700 1 Processor -- start 2023-10-16T11:21:52.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.209+0000 7f2e1e46d700 1 -- start start 2023-10-16T11:21:52.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e1e46d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 0x7f2e1806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e1e46d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2e1815be30 0x7f2e1815fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e1e46d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1815f7c0 con 0x7f2e180b0ef0 2023-10-16T11:21:52.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e1e46d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1815f4c0 con 0x7f2e1815be30 2023-10-16T11:21:52.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e1e46d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1815c6d0 con 0x7f2e18159560 2023-10-16T11:21:52.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e1ca0a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2e180b0ef0 0x7f2e180cd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44006/0 (socket says 172.21.15.181:44006) 2023-10-16T11:21:52.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e1ca0a700 1 -- 172.21.15.181:0/521231346 learned_addr learned my addr 172.21.15.181:0/521231346 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:52.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e177fe700 1 --2- 172.21.15.181:0/521231346 >> v2:172.21.15.190:3300/0 conn(0x7f2e1815be30 0x7f2e1815fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017848117 0 0) 0x7f2e1815f7c0 con 0x7f2e180b0ef0 2023-10-16T11:21:52.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e000035b0 con 0x7f2e180b0ef0 2023-10-16T11:21:52.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e17fff700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 0x7f2e1806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490173537 0 0) 0x7f2e000035b0 con 0x7f2e180b0ef0 2023-10-16T11:21:52.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 msgr2=0x7f2e1806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:52.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 0x7f2e1806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 >> v2:172.21.15.190:3300/0 conn(0x7f2e1815be30 msgr2=0x7f2e1815fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:52.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 --2- 172.21.15.181:0/521231346 >> v2:172.21.15.190:3300/0 conn(0x7f2e1815be30 0x7f2e1815fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1815c9b0 con 0x7f2e180b0ef0 2023-10-16T11:21:52.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e177fe700 1 --2- 172.21.15.181:0/521231346 >> v2:172.21.15.190:3300/0 conn(0x7f2e1815be30 0x7f2e1815fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:52.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f2e100019f0 con 0x7f2e180b0ef0 2023-10-16T11:21:52.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 >> v1:172.21.15.181:6789/0 conn(0x7f2e180b0ef0 legacy=0x7f2e180cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:52.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e00003a00 0x7f2e00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 0x7f2e00004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00004d90 0x7f2e00005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1815f4c0 con 0x7f2e00004d90 2023-10-16T11:21:52.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1815c6d0 con 0x7f2e00003a00 2023-10-16T11:21:52.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.210+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0000ba10 con 0x7f2e18159560 2023-10-16T11:21:52.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e17fff700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e00003a00 0x7f2e00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e1ca0a700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00004d90 0x7f2e00005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e177fe700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 0x7f2e00004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e17fff700 1 -- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 msgr2=0x7f2e00004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e17fff700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e18159560 0x7f2e00004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e17fff700 1 -- 172.21.15.181:0/521231346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00004d90 msgr2=0x7f2e00005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:52.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e17fff700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00004d90 0x7f2e00005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e17fff700 1 -- 172.21.15.181:0/521231346 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e000035b0 con 0x7f2e00003a00 2023-10-16T11:21:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e1ca0a700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00004d90 0x7f2e00005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:21:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e17fff700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e00003a00 0x7f2e00003e20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009b60 tx=0x7f2e00005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9e8a067c789abb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:52.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00e6c0 con 0x7f2e00003a00 2023-10-16T11:21:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c00e820 con 0x7f2e00003a00 2023-10-16T11:21:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.211+0000 7f2e16ffd700 1 -- 172.21.15.181:0/521231346 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00eb90 con 0x7f2e00003a00 2023-10-16T11:21:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.212+0000 7f2e1e46d700 1 -- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e00003a00 msgr2=0x7f2e00003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.212+0000 7f2e1e46d700 1 --2- 172.21.15.181:0/521231346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e00003a00 0x7f2e00003e20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009b60 tx=0x7f2e00005880 comp rx=0 tx=0).stop 2023-10-16T11:21:52.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.212+0000 7f2e1ca0a700 1 -- 172.21.15.181:0/521231346 reap_dead start 2023-10-16T11:21:52.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.212+0000 7f2e1e46d700 1 -- 172.21.15.181:0/521231346 shutdown_connections 2023-10-16T11:21:52.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.212+0000 7f2e1e46d700 1 -- 172.21.15.181:0/521231346 >> 172.21.15.181:0/521231346 conn(0x7f2e1806aed0 msgr2=0x7f2e180cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:52.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.212+0000 7f2e1e46d700 1 -- 172.21.15.181:0/521231346 shutdown_connections 2023-10-16T11:21:52.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.212+0000 7f2e1e46d700 1 -- 172.21.15.181:0/521231346 wait complete. 2023-10-16T11:21:52.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.213+0000 7f2e1e46d700 1 Processor -- start 2023-10-16T11:21:52.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.213+0000 7f2e1e46d700 1 -- start start 2023-10-16T11:21:52.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e1e46d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00003a00 0x7f2e1814a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e1e46d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e181f5120 0x7f2e180ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e1e46d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e180c5700 0x7f2e180cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e1e46d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1814dca0 con 0x7f2e00003a00 2023-10-16T11:21:52.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e1e46d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1814a7d0 con 0x7f2e181f5120 2023-10-16T11:21:52.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e1e46d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180c5ca0 con 0x7f2e180c5700 2023-10-16T11:21:52.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00003a00 0x7f2e1814a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00003a00 0x7f2e1814a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32936/0 (socket says 172.21.15.181:32936) 2023-10-16T11:21:52.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 -- 172.21.15.181:0/713035323 learned_addr learned my addr 172.21.15.181:0/713035323 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:52.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e177fe700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e181f5120 0x7f2e180ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e1ca0a700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e180c5700 0x7f2e180cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 -- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e180c5700 msgr2=0x7f2e180cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:52.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e180c5700 0x7f2e180cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 -- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e181f5120 msgr2=0x7f2e180ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:52.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e181f5120 0x7f2e180ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:52.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 -- 172.21.15.181:0/713035323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e181f4640 con 0x7f2e00003a00 2023-10-16T11:21:52.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e17fff700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00003a00 0x7f2e1814a290 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0090b0 tx=0x7f2e0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:52.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.214+0000 7f2e14ff9700 1 -- 172.21.15.181:0/713035323 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00e6c0 con 0x7f2e00003a00 2023-10-16T11:21:52.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.215+0000 7f2e14ff9700 1 -- 172.21.15.181:0/713035323 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c003ab0 con 0x7f2e00003a00 2023-10-16T11:21:52.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.215+0000 7f2e14ff9700 1 -- 172.21.15.181:0/713035323 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c003e60 con 0x7f2e00003a00 2023-10-16T11:21:52.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.215+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e181f4920 con 0x7f2e00003a00 2023-10-16T11:21:52.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.215+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1814cd50 con 0x7f2e00003a00 2023-10-16T11:21:52.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.215+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e181f4f70 con 0x7f2e00003a00 2023-10-16T11:21:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.216+0000 7f2e14ff9700 1 -- 172.21.15.181:0/713035323 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2e0c013470 con 0x7f2e00003a00 2023-10-16T11:21:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.216+0000 7f2e14ff9700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f2e0406b020 0x7f2e0406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.216+0000 7f2e14ff9700 1 -- 172.21.15.181:0/713035323 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 6269+0+0 (secure 0 0 0) 0x7f2e0c08b230 con 0x7f2e00003a00 2023-10-16T11:21:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.216+0000 7f2e177fe700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f2e0406b020 0x7f2e0406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.217+0000 7f2e177fe700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f2e0406b020 0x7f2e0406d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e181f55a0 tx=0x7f2e08008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.221+0000 7f2e14ff9700 1 -- 172.21.15.181:0/713035323 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f2e0c056030 con 0x7f2e00003a00 2023-10-16T11:21:52.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:52.430+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2e181f44a0 con 0x7f2e00003a00 2023-10-16T11:21:53.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.065+0000 7f2e14ff9700 1 -- 172.21.15.181:0/713035323 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v263) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2e181f44a0 con 0x7f2e00003a00 2023-10-16T11:21:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f2e0406b020 msgr2=0x7f2e0406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f2e0406b020 0x7f2e0406d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e181f55a0 tx=0x7f2e08008040 comp rx=0 tx=0).stop 2023-10-16T11:21:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00003a00 msgr2=0x7f2e1814a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00003a00 0x7f2e1814a290 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0090b0 tx=0x7f2e0c0047c0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 shutdown_connections 2023-10-16T11:21:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f2e0406b020 0x7f2e0406d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e180c5700 0x7f2e180cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e00003a00 0x7f2e1814a290 unknown :-1 s=CLOSED pgs=3100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 --2- 172.21.15.181:0/713035323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e181f5120 0x7f2e180ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 >> 172.21.15.181:0/713035323 conn(0x7f2e1806aed0 msgr2=0x7f2e1815b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 shutdown_connections 2023-10-16T11:21:53.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.068+0000 7f2e1e46d700 1 -- 172.21.15.181:0/713035323 wait complete. 2023-10-16T11:21:53.072 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2023-10-16T11:21:53.085 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-16T11:21:53.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1915f700 1 Processor -- start 2023-10-16T11:21:53.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1915f700 1 -- start start 2023-10-16T11:21:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1915f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d140d2390 0x7f6d140cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1915f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d140cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1915f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d140cc470 con 0x7f6d140ce7f0 2023-10-16T11:21:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1915f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140d0610 con 0x7f6d140d2390 2023-10-16T11:21:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1915f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1406c810 con 0x7f6d140d0180 2023-10-16T11:21:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.207+0000 7f6d1259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d140d2390 0x7f6d140cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d140d2390 0x7f6d140cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32946/0 (socket says 172.21.15.181:32946) 2023-10-16T11:21:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1259c700 1 -- 172.21.15.181:0/3039224601 learned_addr learned my addr 172.21.15.181:0/3039224601 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1359e700 1 --2- 172.21.15.181:0/3039224601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d140cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1259c700 1 -- 172.21.15.181:0/3039224601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 msgr2=0x7f6d140cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1259c700 1 --2- 172.21.15.181:0/3039224601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d140cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1259c700 1 -- 172.21.15.181:0/3039224601 >> v1:172.21.15.190:6789/0 conn(0x7f6d140ce7f0 legacy=0x7f6d1406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1259c700 1 -- 172.21.15.181:0/3039224601 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1406ca30 con 0x7f6d140d2390 2023-10-16T11:21:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d1259c700 1 --2- 172.21.15.181:0/3039224601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140cfc40 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009920 tx=0x7f6d08009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6c3a030efc21e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d11d9b700 1 -- 172.21.15.181:0/3039224601 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800ba20 con 0x7f6d140d2390 2023-10-16T11:21:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d11d9b700 1 -- 172.21.15.181:0/3039224601 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0800bb80 con 0x7f6d140d2390 2023-10-16T11:21:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.208+0000 7f6d11d9b700 1 -- 172.21.15.181:0/3039224601 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08009e70 con 0x7f6d140d2390 2023-10-16T11:21:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 -- 172.21.15.181:0/3039224601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 msgr2=0x7f6d140cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 --2- 172.21.15.181:0/3039224601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140cfc40 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009920 tx=0x7f6d08009c30 comp rx=0 tx=0).stop 2023-10-16T11:21:53.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 -- 172.21.15.181:0/3039224601 shutdown_connections 2023-10-16T11:21:53.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 --2- 172.21.15.181:0/3039224601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d140cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 --2- 172.21.15.181:0/3039224601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140cfc40 unknown :-1 s=CLOSED pgs=3101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 -- 172.21.15.181:0/3039224601 >> 172.21.15.181:0/3039224601 conn(0x7f6d1409a9a0 msgr2=0x7f6d140d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 -- 172.21.15.181:0/3039224601 shutdown_connections 2023-10-16T11:21:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.209+0000 7f6d1915f700 1 -- 172.21.15.181:0/3039224601 wait complete. 2023-10-16T11:21:53.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.210+0000 7f6d1915f700 1 Processor -- start 2023-10-16T11:21:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.210+0000 7f6d1915f700 1 -- start start 2023-10-16T11:21:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1915f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d140ce7f0 0x7f6d1409ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1915f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d1414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1915f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1915f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1414e6e0 con 0x7f6d140d2390 2023-10-16T11:21:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1915f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1414f260 con 0x7f6d140ce7f0 2023-10-16T11:21:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1915f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1414e930 con 0x7f6d140d0180 2023-10-16T11:21:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32954/0 (socket says 172.21.15.181:32954) 2023-10-16T11:21:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 -- 172.21.15.181:0/617904865 learned_addr learned my addr 172.21.15.181:0/617904865 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1259c700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d1414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d12d9d700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d140ce7f0 0x7f6d1409ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 -- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 msgr2=0x7f6d1414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d1414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 -- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d140ce7f0 msgr2=0x7f6d1409ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d140ce7f0 0x7f6d1409ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 -- 172.21.15.181:0/617904865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08009550 con 0x7f6d140d2390 2023-10-16T11:21:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d12d9d700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d140ce7f0 0x7f6d1409ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.211+0000 7f6d1359e700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140c5bc0 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c007e60 tx=0x7f6d0c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.212+0000 7f6cfb7fe700 1 -- 172.21.15.181:0/617904865 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00ede0 con 0x7f6d140d2390 2023-10-16T11:21:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.212+0000 7f6cfb7fe700 1 -- 172.21.15.181:0/617904865 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c014420 con 0x7f6d140d2390 2023-10-16T11:21:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.212+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1414ec10 con 0x7f6d140d2390 2023-10-16T11:21:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.212+0000 7f6cfb7fe700 1 -- 172.21.15.181:0/617904865 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0147d0 con 0x7f6d140d2390 2023-10-16T11:21:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.212+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d140c6100 con 0x7f6d140d2390 2023-10-16T11:21:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.213+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1414f3d0 con 0x7f6d140d2390 2023-10-16T11:21:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.213+0000 7f6cfb7fe700 1 -- 172.21.15.181:0/617904865 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6d0c014930 con 0x7f6d140d2390 2023-10-16T11:21:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.214+0000 7f6cfb7fe700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6d0006b020 0x7f6d0006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.214+0000 7f6cfb7fe700 1 -- 172.21.15.181:0/617904865 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f6d0c08abb0 con 0x7f6d140d2390 2023-10-16T11:21:53.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.214+0000 7f6d12d9d700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6d0006b020 0x7f6d0006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.214+0000 7f6d12d9d700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6d0006b020 0x7f6d0006d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6d04000fb0 tx=0x7f6d04009040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:53.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.218+0000 7f6cfb7fe700 1 -- 172.21.15.181:0/617904865 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6d0c055c60 con 0x7f6d140d2390 2023-10-16T11:21:53.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.488+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6d14067d20 con 0x7f6d140d2390 2023-10-16T11:21:53.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.489+0000 7f6cfb7fe700 1 -- 172.21.15.181:0/617904865 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v54) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6d0c059280 con 0x7f6d140d2390 2023-10-16T11:21:53.491 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:53.491 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:21:53.491 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2023-10-16T11:21:53.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.492 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2023-10-16T11:21:53.492 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lywJICRAAnUrj56rgB94eUNns1Vh/rQ==", 2023-10-16T11:21:53.492 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.492 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.492 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:53.493 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:53.493 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.493 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.494 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2023-10-16T11:21:53.494 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1w5/FRAAcuomjWYCPsHdfADEqIyA6A==", 2023-10-16T11:21:53.494 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.494 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.494 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:53.495 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:53.495 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.495 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.496 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2023-10-16T11:21:53.496 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lE5aNEBAA+KkWwRO9cvaBcHJiGVs2AQ==", 2023-10-16T11:21:53.496 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.496 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.497 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:53.497 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:53.497 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.498 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2023-10-16T11:21:53.498 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l1aLQHBAAyxrCpbbHzpyWtn7q9RoZMw==", 2023-10-16T11:21:53.498 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.498 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.499 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:53.499 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:53.499 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.499 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.499 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.500 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2023-10-16T11:21:53.500 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCxBS1lgfXTOBAAni+r5BjNKQyzOZ7OqiL+gA==", 2023-10-16T11:21:53.500 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.500 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.500 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.501 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.501 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.501 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2023-10-16T11:21:53.502 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC0BS1lJN0LOBAAtt8L6D8+mJD8KiKTW9SEyg==", 2023-10-16T11:21:53.502 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.502 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.502 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.502 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.503 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.503 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2023-10-16T11:21:53.503 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC3BS1leTT2NxAANNSIVH3/dd0GNBz/y6dFtQ==", 2023-10-16T11:21:53.503 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.504 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.504 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.504 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.504 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.505 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2023-10-16T11:21:53.505 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC6BS1lDByDNxAAaQtiFskO3ZSQKusB66DzXw==", 2023-10-16T11:21:53.505 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.505 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.506 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.506 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.506 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.507 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2023-10-16T11:21:53.507 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDBBS1lm19KBBAArPEMyR2Oy9nakoC4czg90w==", 2023-10-16T11:21:53.507 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.507 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.507 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.508 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.508 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.508 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2023-10-16T11:21:53.509 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDEBS1lVdRHAxAA1i2f97UtlJXjC4nWYTL/Sw==", 2023-10-16T11:21:53.509 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.509 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.509 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.509 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.510 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2023-10-16T11:21:53.510 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDHBS1lNqDoAhAAHNImw/2bVk89n/uJ6+EX9Q==", 2023-10-16T11:21:53.511 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.511 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.511 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.512 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.512 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2023-10-16T11:21:53.512 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDKBS1lwgN5AhAAWTl6U1nr7ZXRFcci3OaefA==", 2023-10-16T11:21:53.512 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.512 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2023-10-16T11:21:53.513 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2023-10-16T11:21:53.513 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.513 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.514 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2023-10-16T11:21:53.514 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w==", 2023-10-16T11:21:53.514 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.514 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.514 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow r", 2023-10-16T11:21:53.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-16T11:21:53.515 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.515 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2023-10-16T11:21:53.516 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lBvUPKhAA4RYF0+SxVacL/DCetgVVAA==", 2023-10-16T11:21:53.516 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.516 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.516 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:21:53.516 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:21:53.517 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:21:53.517 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.517 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2023-10-16T11:21:53.517 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1l8jTBJhAAObYEq31EFZC20Gfre3lqkQ==", 2023-10-16T11:21:53.518 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.518 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.518 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:21:53.518 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:21:53.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:21:53.519 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.519 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2023-10-16T11:21:53.519 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGAvBLxAASr+3SZnvaeZ0O7uOGKZKag==", 2023-10-16T11:21:53.520 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.520 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2023-10-16T11:21:53.520 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2023-10-16T11:21:53.520 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2023-10-16T11:21:53.520 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2023-10-16T11:21:53.520 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.521 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2023-10-16T11:21:53.521 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtBS1lLwt1FxAA55CM/yfhNpsKlJMjM/7FxA==", 2023-10-16T11:21:53.521 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.522 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:21:53.522 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2023-10-16T11:21:53.522 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2023-10-16T11:21:53.522 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.522 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.523 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2023-10-16T11:21:53.523 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lGF3kBRAAB9p2UbyxuthpnlT8F6J1Ag==", 2023-10-16T11:21:53.523 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.524 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:21:53.524 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:21:53.524 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.524 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2023-10-16T11:21:53.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:53.525 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2023-10-16T11:21:53.525 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCuBS1lKuXTABAASwAyjq7nuJobBvRzqcAnyg==", 2023-10-16T11:21:53.525 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2023-10-16T11:21:53.525 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2023-10-16T11:21:53.526 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2023-10-16T11:21:53.526 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2023-10-16T11:21:53.526 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.526 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:53.526 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-10-16T11:21:53.527 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:21:53.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.490+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6d0006b020 msgr2=0x7f6d0006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6d0006b020 0x7f6d0006d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6d04000fb0 tx=0x7f6d04009040 comp rx=0 tx=0).stop 2023-10-16T11:21:53.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 msgr2=0x7f6d140c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140c5bc0 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c007e60 tx=0x7f6d0c00e3c0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 shutdown_connections 2023-10-16T11:21:53.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140d2390 0x7f6d140c5bc0 unknown :-1 s=CLOSED pgs=3102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d140ce7f0 0x7f6d1409ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6d0006b020 0x7f6d0006d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 --2- 172.21.15.181:0/617904865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d140d0180 0x7f6d1414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 >> 172.21.15.181:0/617904865 conn(0x7f6d1409a9a0 msgr2=0x7f6d1406edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:53.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 shutdown_connections 2023-10-16T11:21:53.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.491+0000 7f6d1915f700 1 -- 172.21.15.181:0/617904865 wait complete. 2023-10-16T11:21:53.531 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:53.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.650+0000 7f7a13480700 1 Processor -- start 2023-10-16T11:21:53.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.650+0000 7f7a13480700 1 -- start start 2023-10-16T11:21:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a13480700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a0c15e6d0 0x7f7a0c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a13480700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c15eab0 0x7f7a0c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a13480700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c06ed00 con 0x7f7a0c0cb0c0 2023-10-16T11:21:53.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a13480700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0d02d0 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a13480700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15ad40 con 0x7f7a0c15eab0 2023-10-16T11:21:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a10a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c15eab0 0x7f7a0c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a1121c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a0c15e6d0 0x7f7a0c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a10a1b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c15eab0 0x7f7a0c15a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:58216/0 (socket says 172.21.15.181:58216) 2023-10-16T11:21:53.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a10a1b700 1 -- 172.21.15.181:0/950942986 learned_addr learned my addr 172.21.15.181:0/950942986 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a10a1b700 1 -- 172.21.15.181:0/950942986 >> v2:172.21.15.190:3300/0 conn(0x7f7a0c15e6d0 msgr2=0x7f7a0c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a10a1b700 1 --2- 172.21.15.181:0/950942986 >> v2:172.21.15.190:3300/0 conn(0x7f7a0c15e6d0 0x7f7a0c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a10a1b700 1 -- 172.21.15.181:0/950942986 >> v1:172.21.15.190:6789/0 conn(0x7f7a0c0cb0c0 legacy=0x7f7a0c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.651+0000 7f7a10a1b700 1 -- 172.21.15.181:0/950942986 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c06e0f0 con 0x7f7a0c15eab0 2023-10-16T11:21:53.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a10a1b700 1 --2- 172.21.15.181:0/950942986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c15eab0 0x7f7a0c15a0f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0098f0 tx=0x7f79fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d35985cbeaedf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:53.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a03fff700 1 -- 172.21.15.181:0/950942986 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc011830 con 0x7f7a0c15eab0 2023-10-16T11:21:53.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a03fff700 1 -- 172.21.15.181:0/950942986 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc011990 con 0x7f7a0c15eab0 2023-10-16T11:21:53.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a03fff700 1 -- 172.21.15.181:0/950942986 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc011d20 con 0x7f7a0c15eab0 2023-10-16T11:21:53.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 -- 172.21.15.181:0/950942986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c15eab0 msgr2=0x7f7a0c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 --2- 172.21.15.181:0/950942986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c15eab0 0x7f7a0c15a0f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0098f0 tx=0x7f79fc009c00 comp rx=0 tx=0).stop 2023-10-16T11:21:53.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 -- 172.21.15.181:0/950942986 shutdown_connections 2023-10-16T11:21:53.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 --2- 172.21.15.181:0/950942986 >> v2:172.21.15.190:3300/0 conn(0x7f7a0c15e6d0 0x7f7a0c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 --2- 172.21.15.181:0/950942986 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c15eab0 0x7f7a0c15a0f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 -- 172.21.15.181:0/950942986 >> 172.21.15.181:0/950942986 conn(0x7f7a0c0d04a0 msgr2=0x7f7a0c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:53.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 -- 172.21.15.181:0/950942986 shutdown_connections 2023-10-16T11:21:53.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.652+0000 7f7a13480700 1 -- 172.21.15.181:0/950942986 wait complete. 2023-10-16T11:21:53.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 Processor -- start 2023-10-16T11:21:53.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 -- start start 2023-10-16T11:21:53.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c0cb0c0 0x7f7a0c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0c15e6d0 0x7f7a0c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a0c15eab0 0x7f7a0c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0c52a0 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0c5580 con 0x7f7a0c15eab0 2023-10-16T11:21:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a13480700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0c5860 con 0x7f7a0c0cb0c0 2023-10-16T11:21:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a10a1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0c15e6d0 0x7f7a0c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a10a1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0c15e6d0 0x7f7a0c153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32966/0 (socket says 172.21.15.181:32966) 2023-10-16T11:21:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a10a1b700 1 -- 172.21.15.181:0/3800152166 learned_addr learned my addr 172.21.15.181:0/3800152166 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.654+0000 7f7a1121c700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c0cb0c0 0x7f7a0c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a11a1d700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a0c15eab0 0x7f7a0c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a10a1b700 1 -- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c0cb0c0 msgr2=0x7f7a0c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a10a1b700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c0cb0c0 0x7f7a0c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a10a1b700 1 -- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a0c15eab0 msgr2=0x7f7a0c14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a10a1b700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a0c15eab0 0x7f7a0c14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a10a1b700 1 -- 172.21.15.181:0/3800152166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc009550 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a10a1b700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0c15e6d0 0x7f7a0c153fc0 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009c40 tx=0x7f79fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:53.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a01ffb700 1 -- 172.21.15.181:0/3800152166 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc004510 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a01ffb700 1 -- 172.21.15.181:0/3800152166 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc0036a0 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c0c5b40 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a01ffb700 1 -- 172.21.15.181:0/3800152166 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc003a50 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.655+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c154500 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.656+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c0c5d50 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.657+0000 7f7a01ffb700 1 -- 172.21.15.181:0/3800152166 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f79fc003bb0 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.657+0000 7f7a01ffb700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f79f006b020 0x7f79f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:53.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.657+0000 7f7a01ffb700 1 -- 172.21.15.181:0/3800152166 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f79fc08a7c0 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.657+0000 7f7a1121c700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f79f006b020 0x7f79f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:53.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.657+0000 7f7a1121c700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f79f006b020 0x7f79f006d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009b20 tx=0x7f7a080091c0 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:53.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.662+0000 7f7a01ffb700 1 -- 172.21.15.181:0/3800152166 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f79fc055890 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.855+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a0c067d20 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.856+0000 7f7a01ffb700 1 -- 172.21.15.181:0/3800152166 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) v1 ==== 74+0+13754 (secure 0 0 0) 0x7f79fc003e60 con 0x7f7a0c15e6d0 2023-10-16T11:21:53.858 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:53.858 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":265,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:53.074732+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f79f006b020 msgr2=0x7f79f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f79f006b020 0x7f79f006d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009b20 tx=0x7f7a080091c0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0c15e6d0 msgr2=0x7f7a0c153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:53.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0c15e6d0 0x7f7a0c153fc0 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009c40 tx=0x7f79fc004750 comp rx=0 tx=0).stop 2023-10-16T11:21:53.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 shutdown_connections 2023-10-16T11:21:53.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f79f006b020 0x7f79f006d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0c0cb0c0 0x7f7a0c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a0c15eab0 0x7f7a0c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 --2- 172.21.15.181:0/3800152166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0c15e6d0 0x7f7a0c153fc0 unknown :-1 s=CLOSED pgs=3103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 >> 172.21.15.181:0/3800152166 conn(0x7f7a0c0d04a0 msgr2=0x7f7a0c0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 shutdown_connections 2023-10-16T11:21:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.858+0000 7f7a13480700 1 -- 172.21.15.181:0/3800152166 wait complete. 2023-10-16T11:21:53.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:21:53.873 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:21:53.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.995+0000 7f6ecafd3700 1 Processor -- start 2023-10-16T11:21:53.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.995+0000 7f6ecafd3700 1 -- start start 2023-10-16T11:21:54.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ecafd3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40666a0 0x7f6ec406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ecafd3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ec40ce0e0 0x7f6ec406afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ecafd3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec40ca300 con 0x7f6ec40cdd00 2023-10-16T11:21:54.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ecafd3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec406e5e0 con 0x7f6ec40ce0e0 2023-10-16T11:21:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ecafd3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40ca580 con 0x7f6ec40666a0 2023-10-16T11:21:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ec8d6f700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6ec40cdd00 0x7f6ec406a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44050/0 (socket says 172.21.15.181:44050) 2023-10-16T11:21:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ec8d6f700 1 -- 172.21.15.181:0/2063521001 learned_addr learned my addr 172.21.15.181:0/2063521001 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ec9570700 1 --2- 172.21.15.181:0/2063521001 >> v2:172.21.15.181:3300/0 conn(0x7f6ec40ce0e0 0x7f6ec406afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.996+0000 7f6ec3fff700 1 --2- 172.21.15.181:0/2063521001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40666a0 0x7f6ec406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec37fe700 1 -- 172.21.15.181:0/2063521001 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100800715 0 0) 0x7f6ec40ca300 con 0x7f6ec40cdd00 2023-10-16T11:21:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec37fe700 1 -- 172.21.15.181:0/2063521001 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eb40035b0 con 0x7f6ec40cdd00 2023-10-16T11:21:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec9570700 1 -- 172.21.15.181:0/2063521001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40666a0 msgr2=0x7f6ec406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:54.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec9570700 1 --2- 172.21.15.181:0/2063521001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40666a0 0x7f6ec406f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec9570700 1 -- 172.21.15.181:0/2063521001 >> v1:172.21.15.181:6789/0 conn(0x7f6ec40cdd00 legacy=0x7f6ec406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec9570700 1 -- 172.21.15.181:0/2063521001 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40ca860 con 0x7f6ec40ce0e0 2023-10-16T11:21:54.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec9570700 1 --2- 172.21.15.181:0/2063521001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40ce0e0 0x7f6ec406afb0 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009d90 tx=0x7f6ebc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265f5b14e32c7c15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec37fe700 1 -- 172.21.15.181:0/2063521001 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00e070 con 0x7f6ec40ce0e0 2023-10-16T11:21:54.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec37fe700 1 -- 172.21.15.181:0/2063521001 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc00bb20 con 0x7f6ec40ce0e0 2023-10-16T11:21:54.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.997+0000 7f6ec37fe700 1 -- 172.21.15.181:0/2063521001 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc012430 con 0x7f6ec40ce0e0 2023-10-16T11:21:54.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2063521001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40ce0e0 msgr2=0x7f6ec406afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2063521001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40ce0e0 0x7f6ec406afb0 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009d90 tx=0x7f6ebc00b330 comp rx=0 tx=0).stop 2023-10-16T11:21:54.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2063521001 shutdown_connections 2023-10-16T11:21:54.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2063521001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40ce0e0 0x7f6ec406afb0 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2063521001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40666a0 0x7f6ec406f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2063521001 >> 172.21.15.181:0/2063521001 conn(0x7f6ec40cc730 msgr2=0x7f6ec40ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:54.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2063521001 shutdown_connections 2023-10-16T11:21:54.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.998+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2063521001 wait complete. 2023-10-16T11:21:54.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.999+0000 7f6ecafd3700 1 Processor -- start 2023-10-16T11:21:54.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:53.999+0000 7f6ecafd3700 1 -- start start 2023-10-16T11:21:54.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ecafd3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ec40666a0 0x7f6ec415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ecafd3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40cdd00 0x7f6ec41f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ecafd3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40ce0e0 0x7f6ec41e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ecafd3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415b1c0 con 0x7f6ec40cdd00 2023-10-16T11:21:54.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ecafd3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415aee0 con 0x7f6ec40666a0 2023-10-16T11:21:54.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ecafd3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415f100 con 0x7f6ec40ce0e0 2023-10-16T11:21:54.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40cdd00 0x7f6ec41f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40cdd00 0x7f6ec41f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32980/0 (socket says 172.21.15.181:32980) 2023-10-16T11:21:54.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 -- 172.21.15.181:0/2659332034 learned_addr learned my addr 172.21.15.181:0/2659332034 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:54.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec9570700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40ce0e0 0x7f6ec41e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec8d6f700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ec40666a0 0x7f6ec415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 -- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40ce0e0 msgr2=0x7f6ec41e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40ce0e0 0x7f6ec41e7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 -- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ec40666a0 msgr2=0x7f6ec415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ec40666a0 0x7f6ec415a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec3fff700 1 -- 172.21.15.181:0/2659332034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc009a70 con 0x7f6ec40cdd00 2023-10-16T11:21:54.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.000+0000 7f6ec8d6f700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ec40666a0 0x7f6ec415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:54.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.001+0000 7f6ec3fff700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40cdd00 0x7f6ec41f2ad0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e840 tx=0x7f6eb000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.001+0000 7f6ec17fa700 1 -- 172.21.15.181:0/2659332034 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0009590 con 0x7f6ec40cdd00 2023-10-16T11:21:54.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.001+0000 7f6ec17fa700 1 -- 172.21.15.181:0/2659332034 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb0013680 con 0x7f6ec40cdd00 2023-10-16T11:21:54.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.001+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec415f3e0 con 0x7f6ec40cdd00 2023-10-16T11:21:54.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.001+0000 7f6ec17fa700 1 -- 172.21.15.181:0/2659332034 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0013a10 con 0x7f6ec40cdd00 2023-10-16T11:21:54.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.001+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec415ab10 con 0x7f6ec40cdd00 2023-10-16T11:21:54.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.002+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec406bf60 con 0x7f6ec40cdd00 2023-10-16T11:21:54.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.002+0000 7f6ec17fa700 1 -- 172.21.15.181:0/2659332034 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6eb0013b70 con 0x7f6ec40cdd00 2023-10-16T11:21:54.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.003+0000 7f6ec17fa700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6ea806b020 0x7f6ea806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.003+0000 7f6ec17fa700 1 -- 172.21.15.181:0/2659332034 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f6eb00895d0 con 0x7f6ec40cdd00 2023-10-16T11:21:54.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.003+0000 7f6ec8d6f700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6ea806b020 0x7f6ea806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.003+0000 7f6ec8d6f700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6ea806b020 0x7f6ea806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80011e0 tx=0x7f6eb8009040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.007+0000 7f6ec17fa700 1 -- 172.21.15.181:0/2659332034 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6eb0054690 con 0x7f6ec40cdd00 2023-10-16T11:21:54.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.202+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ec406b230 con 0x7f6ec40cdd00 2023-10-16T11:21:54.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.202+0000 7f6ec17fa700 1 -- 172.21.15.181:0/2659332034 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v265) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6eb0057cb0 con 0x7f6ec40cdd00 2023-10-16T11:21:54.205 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:21:54.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.204+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6ea806b020 msgr2=0x7f6ea806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6ea806b020 0x7f6ea806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80011e0 tx=0x7f6eb8009040 comp rx=0 tx=0).stop 2023-10-16T11:21:54.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40cdd00 msgr2=0x7f6ec41f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40cdd00 0x7f6ec41f2ad0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e840 tx=0x7f6eb000ec00 comp rx=0 tx=0).stop 2023-10-16T11:21:54.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 shutdown_connections 2023-10-16T11:21:54.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f6ea806b020 0x7f6ea806d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ec40ce0e0 0x7f6ec41e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ec40666a0 0x7f6ec415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 --2- 172.21.15.181:0/2659332034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ec40cdd00 0x7f6ec41f2ad0 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 >> 172.21.15.181:0/2659332034 conn(0x7f6ec40cc730 msgr2=0x7f6ec40ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 shutdown_connections 2023-10-16T11:21:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.205+0000 7f6ecafd3700 1 -- 172.21.15.181:0/2659332034 wait complete. 2023-10-16T11:21:54.219 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T11:21:54.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.338+0000 7f3b42eff700 1 Processor -- start 2023-10-16T11:21:54.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.338+0000 7f3b42eff700 1 -- start start 2023-10-16T11:21:54.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b42eff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 0x7f3b3c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b42eff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b3c0af450 0x7f3b3c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b42eff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c06f870 con 0x7f3b3c06db80 2023-10-16T11:21:54.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b42eff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3c0d1160 con 0x7f3b3c0af450 2023-10-16T11:21:54.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b42eff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0cc8e0 con 0x7f3b3c15dfd0 2023-10-16T11:21:54.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b4149c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3b3c06db80 0x7f3b3c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44070/0 (socket says 172.21.15.181:44070) 2023-10-16T11:21:54.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b4149c700 1 -- 172.21.15.181:0/1519066311 learned_addr learned my addr 172.21.15.181:0/1519066311 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:54.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 0x7f3b3c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3bfff700 1 --2- 172.21.15.181:0/1519066311 >> v2:172.21.15.190:3300/0 conn(0x7f3b3c0af450 0x7f3b3c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134609224 0 0) 0x7f3b3c06f870 con 0x7f3b3c06db80 2023-10-16T11:21:54.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b2c0035b0 con 0x7f3b3c06db80 2023-10-16T11:21:54.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3414545558 0 0) 0x7f3b2c0035b0 con 0x7f3b3c06db80 2023-10-16T11:21:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 msgr2=0x7f3b3c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 0x7f3b3c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 >> v2:172.21.15.190:3300/0 conn(0x7f3b3c0af450 msgr2=0x7f3b3c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:54.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 --2- 172.21.15.181:0/1519066311 >> v2:172.21.15.190:3300/0 conn(0x7f3b3c0af450 0x7f3b3c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0ccbc0 con 0x7f3b3c06db80 2023-10-16T11:21:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f3b340019f0 con 0x7f3b3c06db80 2023-10-16T11:21:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 >> v1:172.21.15.181:6789/0 conn(0x7f3b3c06db80 legacy=0x7f3b3c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c003a00 0x7f3b2c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c004850 0x7f3b2c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 0x7f3b2c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0d1160 con 0x7f3b2c003a00 2023-10-16T11:21:54.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.339+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0cc8e0 con 0x7f3b2c004850 2023-10-16T11:21:54.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c00ba10 con 0x7f3b3c15dfd0 2023-10-16T11:21:54.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c003a00 0x7f3b2c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b3bfff700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c004850 0x7f3b2c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b4149c700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 0x7f3b2c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b40c9b700 1 -- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 msgr2=0x7f3b2c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:54.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c15dfd0 0x7f3b2c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b40c9b700 1 -- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c004850 msgr2=0x7f3b2c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c004850 0x7f3b2c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b40c9b700 1 -- 172.21.15.181:0/1519066311 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0035b0 con 0x7f3b2c003a00 2023-10-16T11:21:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c003a00 0x7f3b2c003e20 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b70 tx=0x7f3b2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea95e49c8353ed80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30005a20 con 0x7f3b2c003a00 2023-10-16T11:21:54.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b30005b80 con 0x7f3b2c003a00 2023-10-16T11:21:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.340+0000 7f3b3b7fe700 1 -- 172.21.15.181:0/1519066311 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000e6c0 con 0x7f3b2c003a00 2023-10-16T11:21:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.341+0000 7f3b42eff700 1 -- 172.21.15.181:0/1519066311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c003a00 msgr2=0x7f3b2c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.341+0000 7f3b42eff700 1 --2- 172.21.15.181:0/1519066311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b2c003a00 0x7f3b2c003e20 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b70 tx=0x7f3b2c005880 comp rx=0 tx=0).stop 2023-10-16T11:21:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.341+0000 7f3b4149c700 1 -- 172.21.15.181:0/1519066311 reap_dead start 2023-10-16T11:21:54.352 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.341+0000 7f3b42eff700 1 -- 172.21.15.181:0/1519066311 shutdown_connections 2023-10-16T11:21:54.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.341+0000 7f3b42eff700 1 -- 172.21.15.181:0/1519066311 >> 172.21.15.181:0/1519066311 conn(0x7f3b3c0d04a0 msgr2=0x7f3b3c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:54.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.341+0000 7f3b42eff700 1 -- 172.21.15.181:0/1519066311 shutdown_connections 2023-10-16T11:21:54.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.341+0000 7f3b42eff700 1 -- 172.21.15.181:0/1519066311 wait complete. 2023-10-16T11:21:54.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.342+0000 7f3b42eff700 1 Processor -- start 2023-10-16T11:21:54.353 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.342+0000 7f3b42eff700 1 -- start start 2023-10-16T11:21:54.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b42eff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c003a00 0x7f3b3c0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b42eff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c148650 0x7f3b3c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.356 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b42eff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c0ae730 0x7f3b3c0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b42eff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0aebb0 con 0x7f3b3c0ae730 2023-10-16T11:21:54.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b42eff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0c9680 con 0x7f3b2c003a00 2023-10-16T11:21:54.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b42eff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c148160 con 0x7f3b3c148650 2023-10-16T11:21:54.357 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c0ae730 0x7f3b3c0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c0ae730 0x7f3b3c0c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33006/0 (socket says 172.21.15.181:33006) 2023-10-16T11:21:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 -- 172.21.15.181:0/1070786385 learned_addr learned my addr 172.21.15.181:0/1070786385 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b3bfff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c148650 0x7f3b3c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c003a00 0x7f3b3c0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 -- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c148650 msgr2=0x7f3b3c0ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c148650 0x7f3b3c0ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 -- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c003a00 msgr2=0x7f3b3c0b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c003a00 0x7f3b3c0b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 -- 172.21.15.181:0/1070786385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0c5790 con 0x7f3b3c0ae730 2023-10-16T11:21:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b3bfff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c148650 0x7f3b3c0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c003a00 0x7f3b3c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b4149c700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c0ae730 0x7f3b3c0c8630 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f3b34001330 tx=0x7f3b34007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b397fa700 1 -- 172.21.15.181:0/1070786385 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400ed90 con 0x7f3b3c0ae730 2023-10-16T11:21:54.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b397fa700 1 -- 172.21.15.181:0/1070786385 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b34013420 con 0x7f3b3c0ae730 2023-10-16T11:21:54.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c0c93a0 con 0x7f3b3c0ae730 2023-10-16T11:21:54.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.343+0000 7f3b397fa700 1 -- 172.21.15.181:0/1070786385 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340137d0 con 0x7f3b3c0ae730 2023-10-16T11:21:54.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.344+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c0c5d10 con 0x7f3b3c0ae730 2023-10-16T11:21:54.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.344+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c0c55f0 con 0x7f3b3c0ae730 2023-10-16T11:21:54.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.345+0000 7f3b397fa700 1 -- 172.21.15.181:0/1070786385 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b34013930 con 0x7f3b3c0ae730 2023-10-16T11:21:54.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.345+0000 7f3b397fa700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b2006b020 0x7f3b2006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.345+0000 7f3b397fa700 1 -- 172.21.15.181:0/1070786385 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 5409+0+0 (secure 0 0 0) 0x7f3b34089e20 con 0x7f3b3c0ae730 2023-10-16T11:21:54.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.346+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b2006b020 0x7f3b2006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.346+0000 7f3b40c9b700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b2006b020 0x7f3b2006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b10 tx=0x7f3b30008030 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.350+0000 7f3b397fa700 1 -- 172.21.15.181:0/1070786385 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b3c0c55f0 con 0x7f3b3c0ae730 2023-10-16T11:21:54.545 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.543+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3b3c0c55f0 con 0x7f3b3c0ae730 2023-10-16T11:21:54.546 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.543+0000 7f3b397fa700 1 -- 172.21.15.181:0/1070786385 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v265) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3b3c0c55f0 con 0x7f3b3c0ae730 2023-10-16T11:21:54.546 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:54.546 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:21:54.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:21:54.547 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2023-10-16T11:21:54.547 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2023-10-16T11:21:54.547 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:21:54.547 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:21:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b2006b020 msgr2=0x7f3b2006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b2006b020 0x7f3b2006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b10 tx=0x7f3b30008030 comp rx=0 tx=0).stop 2023-10-16T11:21:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c0ae730 msgr2=0x7f3b3c0c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c0ae730 0x7f3b3c0c8630 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f3b34001330 tx=0x7f3b34007c90 comp rx=0 tx=0).stop 2023-10-16T11:21:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 shutdown_connections 2023-10-16T11:21:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b3c0ae730 0x7f3b3c0c8630 unknown :-1 s=CLOSED pgs=3109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b2c003a00 0x7f3b3c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b2006b020 0x7f3b2006d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 --2- 172.21.15.181:0/1070786385 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b3c148650 0x7f3b3c0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 >> 172.21.15.181:0/1070786385 conn(0x7f3b3c0d04a0 msgr2=0x7f3b3c0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:54.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 shutdown_connections 2023-10-16T11:21:54.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.546+0000 7f3b42eff700 1 -- 172.21.15.181:0/1070786385 wait complete. 2023-10-16T11:21:54.561 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T11:21:54.562 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T11:21:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.685+0000 7fe3694ff700 1 Processor -- start 2023-10-16T11:21:54.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.685+0000 7fe3694ff700 1 -- start start 2023-10-16T11:21:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe3694ff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe36415d880 0x7fe3640cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe3694ff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe36415dc60 0x7fe3640d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.690 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe3694ff700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3640d22a0 con 0x7fe3640cf510 2023-10-16T11:21:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe3694ff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3640d1690 con 0x7fe36415d880 2023-10-16T11:21:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe3694ff700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3640d1970 con 0x7fe36415dc60 2023-10-16T11:21:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe362ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe36415d880 0x7fe3640cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe362ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe36415d880 0x7fe3640cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33010/0 (socket says 172.21.15.181:33010) 2023-10-16T11:21:54.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe362ffd700 1 -- 172.21.15.181:0/2183832478 learned_addr learned my addr 172.21.15.181:0/2183832478 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:54.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe3627fc700 1 --2- 172.21.15.181:0/2183832478 >> v2:172.21.15.190:3300/0 conn(0x7fe36415dc60 0x7fe3640d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe362ffd700 1 -- 172.21.15.181:0/2183832478 >> v2:172.21.15.190:3300/0 conn(0x7fe36415dc60 msgr2=0x7fe3640d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:54.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe362ffd700 1 --2- 172.21.15.181:0/2183832478 >> v2:172.21.15.190:3300/0 conn(0x7fe36415dc60 0x7fe3640d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe362ffd700 1 -- 172.21.15.181:0/2183832478 >> v1:172.21.15.190:6789/0 conn(0x7fe3640cf510 legacy=0x7fe3640b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.686+0000 7fe362ffd700 1 -- 172.21.15.181:0/2183832478 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3640d1c50 con 0x7fe36415d880 2023-10-16T11:21:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe362ffd700 1 --2- 172.21.15.181:0/2183832478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36415d880 0x7fe3640cefd0 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 crypto rx=0x7fe36408da90 tx=0x7fe358009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b661677a101e1f88 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe361ffb700 1 -- 172.21.15.181:0/2183832478 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358005e30 con 0x7fe36415d880 2023-10-16T11:21:54.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe361ffb700 1 -- 172.21.15.181:0/2183832478 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe358006030 con 0x7fe36415d880 2023-10-16T11:21:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe361ffb700 1 -- 172.21.15.181:0/2183832478 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580063e0 con 0x7fe36415d880 2023-10-16T11:21:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 -- 172.21.15.181:0/2183832478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36415d880 msgr2=0x7fe3640cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 --2- 172.21.15.181:0/2183832478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36415d880 0x7fe3640cefd0 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 crypto rx=0x7fe36408da90 tx=0x7fe358009bd0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 -- 172.21.15.181:0/2183832478 shutdown_connections 2023-10-16T11:21:54.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 --2- 172.21.15.181:0/2183832478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe36415d880 0x7fe3640cefd0 unknown :-1 s=CLOSED pgs=3110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 --2- 172.21.15.181:0/2183832478 >> v2:172.21.15.190:3300/0 conn(0x7fe36415dc60 0x7fe3640d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 -- 172.21.15.181:0/2183832478 >> 172.21.15.181:0/2183832478 conn(0x7fe36409cb10 msgr2=0x7fe3640d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:54.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 -- 172.21.15.181:0/2183832478 shutdown_connections 2023-10-16T11:21:54.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.687+0000 7fe3694ff700 1 -- 172.21.15.181:0/2183832478 wait complete. 2023-10-16T11:21:54.698 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 Processor -- start 2023-10-16T11:21:54.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 -- start start 2023-10-16T11:21:54.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3640cf510 0x7fe3641f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.699 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36415d880 0x7fe3641f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe36415dc60 0x7fe3640c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641f0e80 con 0x7fe3640cf510 2023-10-16T11:21:54.700 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641f4330 con 0x7fe36415d880 2023-10-16T11:21:54.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe3694ff700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36409d820 con 0x7fe36415dc60 2023-10-16T11:21:54.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe362ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3640cf510 0x7fe3641f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.701 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe362ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3640cf510 0x7fe3641f0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33014/0 (socket says 172.21.15.181:33014) 2023-10-16T11:21:54.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.689+0000 7fe362ffd700 1 -- 172.21.15.181:0/1343328081 learned_addr learned my addr 172.21.15.181:0/1343328081 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:54.702 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe3627fc700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36415d880 0x7fe3641f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe3637fe700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe36415dc60 0x7fe3640c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe362ffd700 1 -- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe36415dc60 msgr2=0x7fe3640c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.703 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe362ffd700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe36415dc60 0x7fe3640c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe362ffd700 1 -- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36415d880 msgr2=0x7fe3641f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:54.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe362ffd700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36415d880 0x7fe3641f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:54.704 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe362ffd700 1 -- 172.21.15.181:0/1343328081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358009500 con 0x7fe3640cf510 2023-10-16T11:21:54.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe3637fe700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe36415dc60 0x7fe3640c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:54.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe362ffd700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3640cf510 0x7fe3641f0940 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7fe358006000 tx=0x7fe358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.705 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe34b7fe700 1 -- 172.21.15.181:0/1343328081 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358004250 con 0x7fe3640cf510 2023-10-16T11:21:54.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe34b7fe700 1 -- 172.21.15.181:0/1343328081 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580043b0 con 0x7fe3640cf510 2023-10-16T11:21:54.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36409db00 con 0x7fe3640cf510 2023-10-16T11:21:54.706 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe34b7fe700 1 -- 172.21.15.181:0/1343328081 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358006950 con 0x7fe3640cf510 2023-10-16T11:21:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.690+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36409e0c0 con 0x7fe3640cf510 2023-10-16T11:21:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.691+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36415f080 con 0x7fe3640cf510 2023-10-16T11:21:54.707 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.691+0000 7fe34b7fe700 1 -- 172.21.15.181:0/1343328081 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe358022070 con 0x7fe3640cf510 2023-10-16T11:21:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.692+0000 7fe34b7fe700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fe35006b020 0x7fe35006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.692+0000 7fe34b7fe700 1 -- 172.21.15.181:0/1343328081 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 5409+0+0 (secure 0 0 0) 0x7fe35808ad70 con 0x7fe3640cf510 2023-10-16T11:21:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.692+0000 7fe3627fc700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fe35006b020 0x7fe35006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.692+0000 7fe3627fc700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fe35006b020 0x7fe35006d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ceaa0 tx=0x7fe354008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:54.709 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.696+0000 7fe34b7fe700 1 -- 172.21.15.181:0/1343328081 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe358055ec0 con 0x7fe3640cf510 2023-10-16T11:21:54.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:54.887+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe364160fd0 con 0x7fe3640cf510 2023-10-16T11:21:55.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.069+0000 7fe34b7fe700 1 -- 172.21.15.181:0/1343328081 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v266) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe3580594e0 con 0x7fe3640cf510 2023-10-16T11:21:55.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.072+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fe35006b020 msgr2=0x7fe35006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:55.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.072+0000 7fe3694ff700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fe35006b020 0x7fe35006d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ceaa0 tx=0x7fe354008040 comp rx=0 tx=0).stop 2023-10-16T11:21:55.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.072+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3640cf510 msgr2=0x7fe3641f0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:55.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.072+0000 7fe3694ff700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3640cf510 0x7fe3641f0940 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7fe358006000 tx=0x7fe358004750 comp rx=0 tx=0).stop 2023-10-16T11:21:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.073+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 shutdown_connections 2023-10-16T11:21:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.073+0000 7fe3694ff700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fe35006b020 0x7fe35006d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.073+0000 7fe3694ff700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe36415dc60 0x7fe3640c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.073+0000 7fe3694ff700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3640cf510 0x7fe3641f0940 unknown :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.073+0000 7fe3694ff700 1 --2- 172.21.15.181:0/1343328081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe36415d880 0x7fe3641f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.073+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 >> 172.21.15.181:0/1343328081 conn(0x7fe36409cb10 msgr2=0x7fe364160130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.074+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 shutdown_connections 2023-10-16T11:21:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.074+0000 7fe3694ff700 1 -- 172.21.15.181:0/1343328081 wait complete. 2023-10-16T11:21:55.077 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2023-10-16T11:21:55.090 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T11:21:55.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.216+0000 7f06f5b35700 1 Processor -- start 2023-10-16T11:21:55.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.216+0000 7f06f5b35700 1 -- start start 2023-10-16T11:21:55.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06f5b35700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06f0159e10 0x7f06f00ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:55.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06f5b35700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f06f015a1f0 0x7f06f015db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:55.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06f5b35700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f006e4b0 con 0x7f06f006bef0 2023-10-16T11:21:55.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06f5b35700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f006ed50 con 0x7f06f0159e10 2023-10-16T11:21:55.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06f5b35700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f00ceea0 con 0x7f06f015a1f0 2023-10-16T11:21:55.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06ef7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06f0159e10 0x7f06f00ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:55.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06eeffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f06f006bef0 0x7f06f00caba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44100/0 (socket says 172.21.15.181:44100) 2023-10-16T11:21:55.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06eeffd700 1 -- 172.21.15.181:0/1553567732 learned_addr learned my addr 172.21.15.181:0/1553567732 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:55.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06effff700 1 --2- 172.21.15.181:0/1553567732 >> v2:172.21.15.190:3300/0 conn(0x7f06f015a1f0 0x7f06f015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:55.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06ef7fe700 1 -- 172.21.15.181:0/1553567732 >> v2:172.21.15.190:3300/0 conn(0x7f06f015a1f0 msgr2=0x7f06f015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:55.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06ef7fe700 1 --2- 172.21.15.181:0/1553567732 >> v2:172.21.15.190:3300/0 conn(0x7f06f015a1f0 0x7f06f015db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06ef7fe700 1 -- 172.21.15.181:0/1553567732 >> v1:172.21.15.181:6789/0 conn(0x7f06f006bef0 legacy=0x7f06f00caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:55.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06ee7fc700 1 -- 172.21.15.181:0/1553567732 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241591658 0 0) 0x7f06f006e4b0 con 0x7f06f006bef0 2023-10-16T11:21:55.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06ef7fe700 1 -- 172.21.15.181:0/1553567732 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f006e700 con 0x7f06f0159e10 2023-10-16T11:21:55.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.217+0000 7f06ef7fe700 1 --2- 172.21.15.181:0/1553567732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f0159e10 0x7f06f00ce5a0 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f06f0072c10 tx=0x7f06d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0588af2e8b59762 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:55.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06ee7fc700 1 -- 172.21.15.181:0/1553567732 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8011830 con 0x7f06f0159e10 2023-10-16T11:21:55.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06ee7fc700 1 -- 172.21.15.181:0/1553567732 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d8011990 con 0x7f06f0159e10 2023-10-16T11:21:55.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06ee7fc700 1 -- 172.21.15.181:0/1553567732 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8011d20 con 0x7f06f0159e10 2023-10-16T11:21:55.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 -- 172.21.15.181:0/1553567732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f0159e10 msgr2=0x7f06f00ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:55.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 --2- 172.21.15.181:0/1553567732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f0159e10 0x7f06f00ce5a0 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f06f0072c10 tx=0x7f06d8009b20 comp rx=0 tx=0).stop 2023-10-16T11:21:55.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 -- 172.21.15.181:0/1553567732 shutdown_connections 2023-10-16T11:21:55.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 --2- 172.21.15.181:0/1553567732 >> v2:172.21.15.190:3300/0 conn(0x7f06f015a1f0 0x7f06f015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 --2- 172.21.15.181:0/1553567732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f0159e10 0x7f06f00ce5a0 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 -- 172.21.15.181:0/1553567732 >> 172.21.15.181:0/1553567732 conn(0x7f06f00cc730 msgr2=0x7f06f00ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:55.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 -- 172.21.15.181:0/1553567732 shutdown_connections 2023-10-16T11:21:55.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.218+0000 7f06f5b35700 1 -- 172.21.15.181:0/1553567732 wait complete. 2023-10-16T11:21:55.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.220+0000 7f06f5b35700 1 Processor -- start 2023-10-16T11:21:55.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.220+0000 7f06f5b35700 1 -- start start 2023-10-16T11:21:55.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06f5b35700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f006bef0 0x7f06f00c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:55.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06f5b35700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f0159e10 0x7f06f00c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:55.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06f5b35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f015a1f0 0x7f06f014bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:55.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06f5b35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00c66c0 con 0x7f06f015a1f0 2023-10-16T11:21:55.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06f5b35700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00c8b40 con 0x7f06f0159e10 2023-10-16T11:21:55.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06f5b35700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00c9ba0 con 0x7f06f006bef0 2023-10-16T11:21:55.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f015a1f0 0x7f06f014bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:55.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f015a1f0 0x7f06f014bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33030/0 (socket says 172.21.15.181:33030) 2023-10-16T11:21:55.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 -- 172.21.15.181:0/383232180 learned_addr learned my addr 172.21.15.181:0/383232180 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:55.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06eeffd700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f0159e10 0x7f06f00c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:55.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 -- 172.21.15.181:0/383232180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f006bef0 msgr2=0x7f06f00c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:55.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f006bef0 0x7f06f00c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 -- 172.21.15.181:0/383232180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f0159e10 msgr2=0x7f06f00c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:55.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f0159e10 0x7f06f00c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:55.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 -- 172.21.15.181:0/383232180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80094a0 con 0x7f06f015a1f0 2023-10-16T11:21:55.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06effff700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f015a1f0 0x7f06f014bf20 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7f06e800cc60 tx=0x7f06e800a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:55.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.221+0000 7f06e7fff700 1 -- 172.21.15.181:0/383232180 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e800ce60 con 0x7f06f015a1f0 2023-10-16T11:21:55.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.222+0000 7f06e7fff700 1 -- 172.21.15.181:0/383232180 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e800acf0 con 0x7f06f015a1f0 2023-10-16T11:21:55.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.222+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f014ae80 con 0x7f06f015a1f0 2023-10-16T11:21:55.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.222+0000 7f06e7fff700 1 -- 172.21.15.181:0/383232180 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8003c40 con 0x7f06f015a1f0 2023-10-16T11:21:55.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.222+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f014b210 con 0x7f06f015a1f0 2023-10-16T11:21:55.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.223+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f00cbf00 con 0x7f06f015a1f0 2023-10-16T11:21:55.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.223+0000 7f06e7fff700 1 -- 172.21.15.181:0/383232180 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f06e800f3f0 con 0x7f06f015a1f0 2023-10-16T11:21:55.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.224+0000 7f06e7fff700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f06d006b020 0x7f06d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:55.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.224+0000 7f06ef7fe700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f06d006b020 0x7f06d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:55.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.224+0000 7f06e7fff700 1 -- 172.21.15.181:0/383232180 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 5779+0+0 (secure 0 0 0) 0x7f06e8089a80 con 0x7f06f015a1f0 2023-10-16T11:21:55.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.224+0000 7f06ef7fe700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f06d006b020 0x7f06d006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06f00666c0 tx=0x7f06d8010000 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:55.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.228+0000 7f06e7fff700 1 -- 172.21.15.181:0/383232180 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f06e80548c0 con 0x7f06f015a1f0 2023-10-16T11:21:55.422 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:55.420+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f06f00cbc20 con 0x7f06f015a1f0 2023-10-16T11:21:56.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.071+0000 7f06e7fff700 1 -- 172.21.15.181:0/383232180 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v267) v1 ==== 170+0+0 (secure 0 0 0) 0x7f06e801d070 con 0x7f06f015a1f0 2023-10-16T11:21:56.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f06d006b020 msgr2=0x7f06d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:56.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f06d006b020 0x7f06d006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06f00666c0 tx=0x7f06d8010000 comp rx=0 tx=0).stop 2023-10-16T11:21:56.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f015a1f0 msgr2=0x7f06f014bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:56.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f015a1f0 0x7f06f014bf20 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7f06e800cc60 tx=0x7f06e800a4d0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 shutdown_connections 2023-10-16T11:21:56.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f06d006b020 0x7f06d006d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06f006bef0 0x7f06f00c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06f015a1f0 0x7f06f014bf20 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 --2- 172.21.15.181:0/383232180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06f0159e10 0x7f06f00c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 >> 172.21.15.181:0/383232180 conn(0x7f06f00cc730 msgr2=0x7f06f015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:56.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 shutdown_connections 2023-10-16T11:21:56.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.075+0000 7f06f5b35700 1 -- 172.21.15.181:0/383232180 wait complete. 2023-10-16T11:21:56.080 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2023-10-16T11:21:56.103 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T11:21:56.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.230+0000 7f3b75619700 1 Processor -- start 2023-10-16T11:21:56.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.231+0000 7f3b75619700 1 -- start start 2023-10-16T11:21:56.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.231+0000 7f3b75619700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b70159e50 0x7f3b70071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.231+0000 7f3b75619700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7015ea60 con 0x7f3b700cc5e0 2023-10-16T11:21:56.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.231+0000 7f3b75619700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7015e1c0 con 0x7f3b7015db30 2023-10-16T11:21:56.237 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.231+0000 7f3b75619700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7015dfe0 con 0x7f3b70159e50 2023-10-16T11:21:56.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.231+0000 7f3b6f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3b700cc5e0 0x7f3b700ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44116/0 (socket says 172.21.15.181:44116) 2023-10-16T11:21:56.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.231+0000 7f3b6f7fe700 1 -- 172.21.15.181:0/1931073374 learned_addr learned my addr 172.21.15.181:0/1931073374 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:56.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6effd700 1 --2- 172.21.15.181:0/1931073374 >> v2:172.21.15.190:3300/0 conn(0x7f3b70159e50 0x7f3b70071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822115230 0 0) 0x7f3b7015ea60 con 0x7f3b700cc5e0 2023-10-16T11:21:56.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b580035b0 con 0x7f3b700cc5e0 2023-10-16T11:21:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 222158727 0 0) 0x7f3b580035b0 con 0x7f3b700cc5e0 2023-10-16T11:21:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 >> v2:172.21.15.190:3300/0 conn(0x7f3b70159e50 msgr2=0x7f3b70071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 --2- 172.21.15.181:0/1931073374 >> v2:172.21.15.190:3300/0 conn(0x7f3b70159e50 0x7f3b70071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 >> v1:172.21.15.190:6789/0 conn(0x7f3b7015db30 legacy=0x7f3b700cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:56.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7015e4a0 con 0x7f3b700cc5e0 2023-10-16T11:21:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f3b680019f0 con 0x7f3b700cc5e0 2023-10-16T11:21:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 >> v1:172.21.15.181:6789/0 conn(0x7f3b700cc5e0 legacy=0x7f3b700ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.240 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58004850 0x7f3b58004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b58005210 0x7f3b5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7015dfe0 con 0x7f3b58003a00 2023-10-16T11:21:56.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5800c1b0 con 0x7f3b58005210 2023-10-16T11:21:56.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5800e580 con 0x7f3b58004850 2023-10-16T11:21:56.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6effd700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6f7fe700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b58005210 0x7f3b5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6e7fc700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58004850 0x7f3b58004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6effd700 1 -- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58004850 msgr2=0x7f3b58004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:56.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6effd700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58004850 0x7f3b58004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6effd700 1 -- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b58005210 msgr2=0x7f3b5800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:56.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6effd700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b58005210 0x7f3b5800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6effd700 1 -- 172.21.15.181:0/1931073374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b580035b0 con 0x7f3b58003a00 2023-10-16T11:21:56.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6f7fe700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b58005210 0x7f3b5800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:56.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.232+0000 7f3b6f7fe700 1 -- 172.21.15.181:0/1931073374 reap_dead start 2023-10-16T11:21:56.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b6effd700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009b60 tx=0x7f3b58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4db1135c1493f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:56.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400e6c0 con 0x7f3b58003a00 2023-10-16T11:21:56.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6400e820 con 0x7f3b58003a00 2023-10-16T11:21:56.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b6dffb700 1 -- 172.21.15.181:0/1931073374 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400eb90 con 0x7f3b58003a00 2023-10-16T11:21:56.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b75619700 1 -- 172.21.15.181:0/1931073374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b58003a00 msgr2=0x7f3b58003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:56.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b75619700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009b60 tx=0x7f3b58005880 comp rx=0 tx=0).stop 2023-10-16T11:21:56.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b75619700 1 -- 172.21.15.181:0/1931073374 shutdown_connections 2023-10-16T11:21:56.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b75619700 1 --2- 172.21.15.181:0/1931073374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b75619700 1 -- 172.21.15.181:0/1931073374 >> 172.21.15.181:0/1931073374 conn(0x7f3b7006aed0 msgr2=0x7f3b7006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:56.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b75619700 1 -- 172.21.15.181:0/1931073374 shutdown_connections 2023-10-16T11:21:56.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.233+0000 7f3b75619700 1 -- 172.21.15.181:0/1931073374 wait complete. 2023-10-16T11:21:56.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 Processor -- start 2023-10-16T11:21:56.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 -- start start 2023-10-16T11:21:56.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58003a00 0x7f3b701f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b700adad0 0x7f3b700c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b700b2110 0x7f3b700b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700c79c0 con 0x7f3b700adad0 2023-10-16T11:21:56.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700c58b0 con 0x7f3b700b2110 2023-10-16T11:21:56.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.235+0000 7f3b75619700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b700c5b90 con 0x7f3b58003a00 2023-10-16T11:21:56.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b700adad0 0x7f3b700c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b700adad0 0x7f3b700c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33048/0 (socket says 172.21.15.181:33048) 2023-10-16T11:21:56.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 -- 172.21.15.181:0/880227709 learned_addr learned my addr 172.21.15.181:0/880227709 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:56.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6effd700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58003a00 0x7f3b701f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6f7fe700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b700b2110 0x7f3b700b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 -- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58003a00 msgr2=0x7f3b701f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:56.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58003a00 0x7f3b701f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 -- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b700b2110 msgr2=0x7f3b700b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:56.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b700b2110 0x7f3b700b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:56.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 -- 172.21.15.181:0/880227709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b700c5e70 con 0x7f3b700adad0 2023-10-16T11:21:56.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6f7fe700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b700b2110 0x7f3b700b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:56.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b6e7fc700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b700adad0 0x7f3b700c4f60 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000e550 tx=0x7f3b6000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:56.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b577fe700 1 -- 172.21.15.181:0/880227709 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b600052e0 con 0x7f3b700adad0 2023-10-16T11:21:56.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b577fe700 1 -- 172.21.15.181:0/880227709 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b60005440 con 0x7f3b700adad0 2023-10-16T11:21:56.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b701f34e0 con 0x7f3b700adad0 2023-10-16T11:21:56.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b577fe700 1 -- 172.21.15.181:0/880227709 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b600038a0 con 0x7f3b700adad0 2023-10-16T11:21:56.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.236+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b700c5680 con 0x7f3b700adad0 2023-10-16T11:21:56.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.237+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b700c7b00 con 0x7f3b700adad0 2023-10-16T11:21:56.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.238+0000 7f3b577fe700 1 -- 172.21.15.181:0/880227709 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b60003a00 con 0x7f3b700adad0 2023-10-16T11:21:56.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.238+0000 7f3b577fe700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b5c06b020 0x7f3b5c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:56.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.238+0000 7f3b577fe700 1 -- 172.21.15.181:0/880227709 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(267..267 src has 1..267) v4 ==== 6161+0+0 (secure 0 0 0) 0x7f3b60088f70 con 0x7f3b700adad0 2023-10-16T11:21:56.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.238+0000 7f3b6effd700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b5c06b020 0x7f3b5c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:56.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.239+0000 7f3b6effd700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b5c06b020 0x7f3b5c06d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b640090b0 tx=0x7f3b6401e040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:56.254 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.243+0000 7f3b577fe700 1 -- 172.21.15.181:0/880227709 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3b600bbfe0 con 0x7f3b700adad0 2023-10-16T11:21:56.485 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:56.483+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b7004ebb0 con 0x7f3b700adad0 2023-10-16T11:21:57.104 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.102+0000 7f3b577fe700 1 -- 172.21.15.181:0/880227709 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 36 and data pool 37 v149) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3b60019020 con 0x7f3b700adad0 2023-10-16T11:21:57.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.112+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b5c06b020 msgr2=0x7f3b5c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.112+0000 7f3b75619700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b5c06b020 0x7f3b5c06d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b640090b0 tx=0x7f3b6401e040 comp rx=0 tx=0).stop 2023-10-16T11:21:57.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.112+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b700adad0 msgr2=0x7f3b700c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.112+0000 7f3b75619700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b700adad0 0x7f3b700c4f60 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000e550 tx=0x7f3b6000e860 comp rx=0 tx=0).stop 2023-10-16T11:21:57.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.113+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 shutdown_connections 2023-10-16T11:21:57.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.113+0000 7f3b75619700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f3b5c06b020 0x7f3b5c06d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.113+0000 7f3b75619700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b58003a00 0x7f3b701f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.113+0000 7f3b75619700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b700b2110 0x7f3b700b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.113+0000 7f3b75619700 1 --2- 172.21.15.181:0/880227709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b700adad0 0x7f3b700c4f60 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.113+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 >> 172.21.15.181:0/880227709 conn(0x7f3b7006aed0 msgr2=0x7f3b700725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:57.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.116+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 shutdown_connections 2023-10-16T11:21:57.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.116+0000 7f3b75619700 1 -- 172.21.15.181:0/880227709 wait complete. 2023-10-16T11:21:57.120 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 36 and data pool 37 2023-10-16T11:21:57.134 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:57.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.256+0000 7f9270069700 1 Processor -- start 2023-10-16T11:21:57.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.256+0000 7f9270069700 1 -- start start 2023-10-16T11:21:57.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f9270069700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f926815e6d0 0x7f92680d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f9270069700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92680cb0c0 0x7f926815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f9270069700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926806ed00 con 0x7f926815eab0 2023-10-16T11:21:57.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f9270069700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92680d02d0 con 0x7f926815e6d0 2023-10-16T11:21:57.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f9270069700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926815ad40 con 0x7f92680cb0c0 2023-10-16T11:21:57.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f926de05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f926815e6d0 0x7f92680d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f926de05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f926815e6d0 0x7f92680d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33056/0 (socket says 172.21.15.181:33056) 2023-10-16T11:21:57.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f926de05700 1 -- 172.21.15.181:0/1024452398 learned_addr learned my addr 172.21.15.181:0/1024452398 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:57.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.257+0000 7f926e606700 1 --2- 172.21.15.181:0/1024452398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92680cb0c0 0x7f926815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926de05700 1 -- 172.21.15.181:0/1024452398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92680cb0c0 msgr2=0x7f926815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:57.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926de05700 1 --2- 172.21.15.181:0/1024452398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92680cb0c0 0x7f926815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926de05700 1 -- 172.21.15.181:0/1024452398 >> v1:172.21.15.181:6789/0 conn(0x7f926815eab0 legacy=0x7f926815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926de05700 1 -- 172.21.15.181:0/1024452398 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926806e0f0 con 0x7f926815e6d0 2023-10-16T11:21:57.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926ce03700 1 -- 172.21.15.181:0/1024452398 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386206860 0 0) 0x7f926806ed00 con 0x7f926815eab0 2023-10-16T11:21:57.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926e606700 1 --2- 172.21.15.181:0/1024452398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92680cb0c0 0x7f926815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:57.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926de05700 1 --2- 172.21.15.181:0/1024452398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815e6d0 0x7f92680d0fc0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f9258006fd0 tx=0x7f925800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd1f9049d20ffa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926ce03700 1 -- 172.21.15.181:0/1024452398 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258005e30 con 0x7f926815e6d0 2023-10-16T11:21:57.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926ce03700 1 -- 172.21.15.181:0/1024452398 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9258004030 con 0x7f926815e6d0 2023-10-16T11:21:57.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f926ce03700 1 -- 172.21.15.181:0/1024452398 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92580043e0 con 0x7f926815e6d0 2023-10-16T11:21:57.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f9270069700 1 -- 172.21.15.181:0/1024452398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815e6d0 msgr2=0x7f92680d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.258+0000 7f9270069700 1 --2- 172.21.15.181:0/1024452398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815e6d0 0x7f92680d0fc0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f9258006fd0 tx=0x7f925800be70 comp rx=0 tx=0).stop 2023-10-16T11:21:57.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.259+0000 7f9270069700 1 -- 172.21.15.181:0/1024452398 shutdown_connections 2023-10-16T11:21:57.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.259+0000 7f9270069700 1 --2- 172.21.15.181:0/1024452398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92680cb0c0 0x7f926815a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.259+0000 7f9270069700 1 --2- 172.21.15.181:0/1024452398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815e6d0 0x7f92680d0fc0 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.259+0000 7f9270069700 1 -- 172.21.15.181:0/1024452398 >> 172.21.15.181:0/1024452398 conn(0x7f92680d04a0 msgr2=0x7f92680d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:57.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.259+0000 7f9270069700 1 -- 172.21.15.181:0/1024452398 shutdown_connections 2023-10-16T11:21:57.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.259+0000 7f9270069700 1 -- 172.21.15.181:0/1024452398 wait complete. 2023-10-16T11:21:57.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.260+0000 7f9270069700 1 Processor -- start 2023-10-16T11:21:57.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.260+0000 7f9270069700 1 -- start start 2023-10-16T11:21:57.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cb0c0 0x7f92680c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926815e6d0 0x7f9268159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815eab0 0x7f9268159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926815da90 con 0x7f926815eab0 2023-10-16T11:21:57.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926815df20 con 0x7f92680cb0c0 2023-10-16T11:21:57.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92680c9490 con 0x7f926815e6d0 2023-10-16T11:21:57.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815eab0 0x7f9268159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815eab0 0x7f9268159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33064/0 (socket says 172.21.15.181:33064) 2023-10-16T11:21:57.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 -- 172.21.15.181:0/3764654309 learned_addr learned my addr 172.21.15.181:0/3764654309 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:57.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926d604700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926815e6d0 0x7f9268159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926de05700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cb0c0 0x7f92680c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 -- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926815e6d0 msgr2=0x7f9268159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926815e6d0 0x7f9268159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 -- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cb0c0 msgr2=0x7f92680c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cb0c0 0x7f92680c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 -- 172.21.15.181:0/3764654309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925800b7a0 con 0x7f926815eab0 2023-10-16T11:21:57.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926de05700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cb0c0 0x7f92680c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:57.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f926e606700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815eab0 0x7f9268159e50 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7f9260007e70 tx=0x7f926000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f925e7fc700 1 -- 172.21.15.181:0/3764654309 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926000ddd0 con 0x7f926815eab0 2023-10-16T11:21:57.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f925e7fc700 1 -- 172.21.15.181:0/3764654309 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926000f040 con 0x7f926815eab0 2023-10-16T11:21:57.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f925e7fc700 1 -- 172.21.15.181:0/3764654309 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260003c60 con 0x7f926815eab0 2023-10-16T11:21:57.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926815ce80 con 0x7f926815eab0 2023-10-16T11:21:57.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.261+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926815d440 con 0x7f926815eab0 2023-10-16T11:21:57.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.262+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9268067d20 con 0x7f926815eab0 2023-10-16T11:21:57.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.263+0000 7f925e7fc700 1 -- 172.21.15.181:0/3764654309 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9260004030 con 0x7f926815eab0 2023-10-16T11:21:57.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.263+0000 7f925e7fc700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f924c06b020 0x7f924c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.263+0000 7f925e7fc700 1 -- 172.21.15.181:0/3764654309 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9260089b50 con 0x7f926815eab0 2023-10-16T11:21:57.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.263+0000 7f926de05700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f924c06b020 0x7f924c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.264+0000 7f926de05700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f924c06b020 0x7f924c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f925800b340 tx=0x7f9258012000 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.268+0000 7f925e7fc700 1 -- 172.21.15.181:0/3764654309 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f92600547a0 con 0x7f926815eab0 2023-10-16T11:21:57.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.461+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92680cb610 con 0x7f926815eab0 2023-10-16T11:21:57.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.463+0000 7f925e7fc700 1 -- 172.21.15.181:0/3764654309 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v268) v1 ==== 74+0+17427 (secure 0 0 0) 0x7f926001a090 con 0x7f926815eab0 2023-10-16T11:21:57.465 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:57.465 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":268,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:56.485017+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:54.910438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:55.443301+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:57.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f924c06b020 msgr2=0x7f924c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f924c06b020 0x7f924c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f925800b340 tx=0x7f9258012000 comp rx=0 tx=0).stop 2023-10-16T11:21:57.467 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815eab0 msgr2=0x7f9268159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815eab0 0x7f9268159e50 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7f9260007e70 tx=0x7f926000d3b0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 shutdown_connections 2023-10-16T11:21:57.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f926815eab0 0x7f9268159e50 unknown :-1 s=CLOSED pgs=3120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92680cb0c0 0x7f92680c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.468 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f924c06b020 0x7f924c06d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 --2- 172.21.15.181:0/3764654309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f926815e6d0 0x7f9268159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 >> 172.21.15.181:0/3764654309 conn(0x7f92680d04a0 msgr2=0x7f92680ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:57.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 shutdown_connections 2023-10-16T11:21:57.469 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.465+0000 7f9270069700 1 -- 172.21.15.181:0/3764654309 wait complete. 2023-10-16T11:21:57.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:21:57.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.597+0000 7f0f1353b700 1 Processor -- start 2023-10-16T11:21:57.600 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.598+0000 7f0f1353b700 1 -- start start 2023-10-16T11:21:57.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.598+0000 7f0f1353b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.598+0000 7f0f1353b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f0c15eab0 0x7f0f0c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.602 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.598+0000 7f0f1353b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c06ed00 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.598+0000 7f0f1353b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c0d02d0 con 0x7f0f0c15eab0 2023-10-16T11:21:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.598+0000 7f0f1353b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c15ad40 con 0x7f0f0c15e6d0 2023-10-16T11:21:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f10ad6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f0c15eab0 0x7f0f0c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f11ad8700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0f0c0cb0c0 0x7f0f0c15a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44150/0 (socket says 172.21.15.181:44150) 2023-10-16T11:21:57.603 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f11ad8700 1 -- 172.21.15.181:0/4228741617 learned_addr learned my addr 172.21.15.181:0/4228741617 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f112d7700 1 --2- 172.21.15.181:0/4228741617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f10ad6700 1 -- 172.21.15.181:0/4228741617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 msgr2=0x7f0f0c0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f10ad6700 1 --2- 172.21.15.181:0/4228741617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f10ad6700 1 -- 172.21.15.181:0/4228741617 >> v1:172.21.15.181:6789/0 conn(0x7f0f0c0cb0c0 legacy=0x7f0f0c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f10ad6700 1 -- 172.21.15.181:0/4228741617 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c06e0f0 con 0x7f0f0c15eab0 2023-10-16T11:21:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f03fff700 1 -- 172.21.15.181:0/4228741617 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172499304 0 0) 0x7f0f0c06ed00 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f10ad6700 1 --2- 172.21.15.181:0/4228741617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c15eab0 0x7f0f0c15a0f0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800bae0 tx=0x7f0f0800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d1802b6125aa2f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f03fff700 1 -- 172.21.15.181:0/4228741617 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0800dd30 con 0x7f0f0c15eab0 2023-10-16T11:21:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f03fff700 1 -- 172.21.15.181:0/4228741617 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0800de90 con 0x7f0f0c15eab0 2023-10-16T11:21:57.605 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.599+0000 7f0f03fff700 1 -- 172.21.15.181:0/4228741617 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08010690 con 0x7f0f0c15eab0 2023-10-16T11:21:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 -- 172.21.15.181:0/4228741617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c15eab0 msgr2=0x7f0f0c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 --2- 172.21.15.181:0/4228741617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c15eab0 0x7f0f0c15a0f0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800bae0 tx=0x7f0f0800bdf0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 -- 172.21.15.181:0/4228741617 shutdown_connections 2023-10-16T11:21:57.606 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 --2- 172.21.15.181:0/4228741617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 --2- 172.21.15.181:0/4228741617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c15eab0 0x7f0f0c15a0f0 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 -- 172.21.15.181:0/4228741617 >> 172.21.15.181:0/4228741617 conn(0x7f0f0c0d04a0 msgr2=0x7f0f0c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 -- 172.21.15.181:0/4228741617 shutdown_connections 2023-10-16T11:21:57.607 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.600+0000 7f0f1353b700 1 -- 172.21.15.181:0/4228741617 wait complete. 2023-10-16T11:21:57.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.601+0000 7f0f1353b700 1 Processor -- start 2023-10-16T11:21:57.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.601+0000 7f0f1353b700 1 -- start start 2023-10-16T11:21:57.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f1353b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c0cb0c0 0x7f0f0c14d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f1353b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f1353b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f0c15eab0 0x7f0f0c0ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f1353b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c14e940 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f1353b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c15d0a0 con 0x7f0f0c15eab0 2023-10-16T11:21:57.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f1353b700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c14dee0 con 0x7f0f0c15e6d0 2023-10-16T11:21:57.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c0cb0c0 0x7f0f0c14d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c0cb0c0 0x7f0f0c14d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33074/0 (socket says 172.21.15.181:33074) 2023-10-16T11:21:57.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 -- 172.21.15.181:0/1510955637 learned_addr learned my addr 172.21.15.181:0/1510955637 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:57.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f10ad6700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 -- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 msgr2=0x7f0f0c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 -- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f0c15eab0 msgr2=0x7f0f0c0ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f0c15eab0 0x7f0f0c0ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 -- 172.21.15.181:0/1510955637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0800b740 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f10ad6700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:21:57.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f112d7700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c0cb0c0 0x7f0f0c14d9a0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00d7d0 tx=0x7f0efc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f01ffb700 1 -- 172.21.15.181:0/1510955637 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0125f0 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f01ffb700 1 -- 172.21.15.181:0/1510955637 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0efc012750 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f01ffb700 1 -- 172.21.15.181:0/1510955637 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc012b00 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.602+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c0afa20 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.603+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c15d5d0 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.604+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c0b0000 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.604+0000 7f0f01ffb700 1 -- 172.21.15.181:0/1510955637 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0efc012c60 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.604+0000 7f0f01ffb700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0ef006b020 0x7f0ef006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.604+0000 7f0f01ffb700 1 -- 172.21.15.181:0/1510955637 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f0efc089a60 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.604+0000 7f0f10ad6700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0ef006b020 0x7f0ef006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.605+0000 7f0f10ad6700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0ef006b020 0x7f0ef006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0c39e0 tx=0x7f0f08013000 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.609+0000 7f0f01ffb700 1 -- 172.21.15.181:0/1510955637 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0efc0547d0 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.801+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f0c067d20 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.803+0000 7f0f01ffb700 1 -- 172.21.15.181:0/1510955637 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v268) v1 ==== 74+0+17427 (secure 0 0 0) 0x7f0efc057df0 con 0x7f0f0c0cb0c0 2023-10-16T11:21:57.805 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:21:57.805 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":268,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:56.485017+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:54.910438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:55.443301+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:21:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0ef006b020 msgr2=0x7f0ef006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0ef006b020 0x7f0ef006d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0c39e0 tx=0x7f0f08013000 comp rx=0 tx=0).stop 2023-10-16T11:21:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c0cb0c0 msgr2=0x7f0f0c14d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c0cb0c0 0x7f0f0c14d9a0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00d7d0 tx=0x7f0efc00db90 comp rx=0 tx=0).stop 2023-10-16T11:21:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 shutdown_connections 2023-10-16T11:21:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f0c15eab0 0x7f0f0c0ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f0c0cb0c0 0x7f0f0c14d9a0 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0ef006b020 0x7f0ef006d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 --2- 172.21.15.181:0/1510955637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f0c15e6d0 0x7f0f0c0ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 >> 172.21.15.181:0/1510955637 conn(0x7f0f0c0d04a0 msgr2=0x7f0f0c0cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 shutdown_connections 2023-10-16T11:21:57.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.805+0000 7f0f1353b700 1 -- 172.21.15.181:0/1510955637 wait complete. 2023-10-16T11:21:57.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T11:21:57.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.942+0000 7fc041f42700 1 Processor -- start 2023-10-16T11:21:57.944 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.942+0000 7fc041f42700 1 -- start start 2023-10-16T11:21:57.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc041f42700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc03c15da70 0x7fc03c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc041f42700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c0ce6a0 0x7fc03c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc041f42700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c0cef00 con 0x7fc03c072760 2023-10-16T11:21:57.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc041f42700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c0cec20 con 0x7fc03c15da70 2023-10-16T11:21:57.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc041f42700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c071c80 con 0x7fc03c0ce6a0 2023-10-16T11:21:57.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc03c15da70 0x7fc03c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc03c072760 0x7fc03c1615f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44164/0 (socket says 172.21.15.181:44164) 2023-10-16T11:21:57.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03bfff700 1 -- 172.21.15.181:0/948335763 learned_addr learned my addr 172.21.15.181:0/948335763 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:57.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03affd700 1 --2- 172.21.15.181:0/948335763 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c0ce6a0 0x7fc03c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03b7fe700 1 -- 172.21.15.181:0/948335763 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c0ce6a0 msgr2=0x7fc03c06a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:57.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03b7fe700 1 --2- 172.21.15.181:0/948335763 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c0ce6a0 0x7fc03c06a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03b7fe700 1 -- 172.21.15.181:0/948335763 >> v1:172.21.15.181:6789/0 conn(0x7fc03c072760 legacy=0x7fc03c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03b7fe700 1 -- 172.21.15.181:0/948335763 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c071f60 con 0x7fc03c15da70 2023-10-16T11:21:57.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03a7fc700 1 -- 172.21.15.181:0/948335763 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219941201 0 0) 0x7fc03c0cef00 con 0x7fc03c072760 2023-10-16T11:21:57.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03b7fe700 1 --2- 172.21.15.181:0/948335763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0ce160 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009840 tx=0x7fc02c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69b1e7c0972fbee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03a7fc700 1 -- 172.21.15.181:0/948335763 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011830 con 0x7fc03c15da70 2023-10-16T11:21:57.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03a7fc700 1 -- 172.21.15.181:0/948335763 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c011990 con 0x7fc03c15da70 2023-10-16T11:21:57.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.943+0000 7fc03a7fc700 1 -- 172.21.15.181:0/948335763 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c011d20 con 0x7fc03c15da70 2023-10-16T11:21:57.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 -- 172.21.15.181:0/948335763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 msgr2=0x7fc03c0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 --2- 172.21.15.181:0/948335763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0ce160 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009840 tx=0x7fc02c009b50 comp rx=0 tx=0).stop 2023-10-16T11:21:57.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 -- 172.21.15.181:0/948335763 shutdown_connections 2023-10-16T11:21:57.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 --2- 172.21.15.181:0/948335763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0ce160 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 --2- 172.21.15.181:0/948335763 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c0ce6a0 0x7fc03c06a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 -- 172.21.15.181:0/948335763 >> 172.21.15.181:0/948335763 conn(0x7fc03c0cd010 msgr2=0x7fc03c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:57.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 -- 172.21.15.181:0/948335763 shutdown_connections 2023-10-16T11:21:57.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.944+0000 7fc041f42700 1 -- 172.21.15.181:0/948335763 wait complete. 2023-10-16T11:21:57.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.945+0000 7fc041f42700 1 Processor -- start 2023-10-16T11:21:57.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.945+0000 7fc041f42700 1 -- start start 2023-10-16T11:21:57.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc041f42700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c072760 0x7fc03c0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc041f42700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc03c0ce6a0 0x7fc03c160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc041f42700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc041f42700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c1f2ff0 con 0x7fc03c15da70 2023-10-16T11:21:57.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc041f42700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c15fbb0 con 0x7fc03c0ce6a0 2023-10-16T11:21:57.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc041f42700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c15fdc0 con 0x7fc03c072760 2023-10-16T11:21:57.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33092/0 (socket says 172.21.15.181:33092) 2023-10-16T11:21:57.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03bfff700 1 -- 172.21.15.181:0/2073958665 learned_addr learned my addr 172.21.15.181:0/2073958665 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:57.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03b7fe700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c072760 0x7fc03c0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03affd700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc03c0ce6a0 0x7fc03c160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03bfff700 1 -- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c072760 msgr2=0x7fc03c0b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03bfff700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c072760 0x7fc03c0b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03bfff700 1 -- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc03c0ce6a0 msgr2=0x7fc03c160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:57.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.946+0000 7fc03bfff700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc03c0ce6a0 0x7fc03c160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:57.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc03bfff700 1 -- 172.21.15.181:0/2073958665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0094a0 con 0x7fc03c15da70 2023-10-16T11:21:57.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc03affd700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc03c0ce6a0 0x7fc03c160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:57.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc03bfff700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0b54b0 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7fc03400cba0 tx=0x7fc03400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc023fff700 1 -- 172.21.15.181:0/2073958665 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400aad0 con 0x7fc03c15da70 2023-10-16T11:21:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc023fff700 1 -- 172.21.15.181:0/2073958665 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400ac50 con 0x7fc03c15da70 2023-10-16T11:21:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c1600a0 con 0x7fc03c15da70 2023-10-16T11:21:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc023fff700 1 -- 172.21.15.181:0/2073958665 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340070d0 con 0x7fc03c15da70 2023-10-16T11:21:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.947+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c0c7660 con 0x7fc03c15da70 2023-10-16T11:21:57.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.948+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c0ce1f0 con 0x7fc03c15da70 2023-10-16T11:21:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.948+0000 7fc023fff700 1 -- 172.21.15.181:0/2073958665 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc034007230 con 0x7fc03c15da70 2023-10-16T11:21:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.949+0000 7fc023fff700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fc02806b020 0x7fc02806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.949+0000 7fc03b7fe700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fc02806b020 0x7fc02806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.949+0000 7fc023fff700 1 -- 172.21.15.181:0/2073958665 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(268..268 src has 1..268) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc0340896b0 con 0x7fc03c15da70 2023-10-16T11:21:57.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.949+0000 7fc03b7fe700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fc02806b020 0x7fc02806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0090b0 tx=0x7fc02c01c040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:57.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:57.953+0000 7fc023fff700 1 -- 172.21.15.181:0/2073958665 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fc0340544b0 con 0x7fc03c15da70 2023-10-16T11:21:57.986 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:57.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:21:58.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:58.203+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc03c06f6c0 con 0x7fc03c15da70 2023-10-16T11:21:59.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.111+0000 7fc023fff700 1 -- 172.21.15.181:0/2073958665 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v152) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc034057ad0 con 0x7fc03c15da70 2023-10-16T11:21:59.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.114+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fc02806b020 msgr2=0x7fc02806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:59.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.114+0000 7fc041f42700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fc02806b020 0x7fc02806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0090b0 tx=0x7fc02c01c040 comp rx=0 tx=0).stop 2023-10-16T11:21:59.117 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.114+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 msgr2=0x7fc03c0b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:59.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.114+0000 7fc041f42700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0b54b0 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7fc03400cba0 tx=0x7fc03400ceb0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.114+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 shutdown_connections 2023-10-16T11:21:59.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.114+0000 7fc041f42700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fc02806b020 0x7fc02806d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.115+0000 7fc041f42700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc03c072760 0x7fc03c0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.118 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.115+0000 7fc041f42700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc03c15da70 0x7fc03c0b54b0 unknown :-1 s=CLOSED pgs=3126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.115+0000 7fc041f42700 1 --2- 172.21.15.181:0/2073958665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc03c0ce6a0 0x7fc03c160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.115+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 >> 172.21.15.181:0/2073958665 conn(0x7fc03c0cd010 msgr2=0x7fc03c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:59.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.115+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 shutdown_connections 2023-10-16T11:21:59.119 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.115+0000 7fc041f42700 1 -- 172.21.15.181:0/2073958665 wait complete. 2023-10-16T11:21:59.132 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T11:21:59.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T11:21:59.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.262+0000 7f972ccec700 1 Processor -- start 2023-10-16T11:21:59.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.262+0000 7f972ccec700 1 -- start start 2023-10-16T11:21:59.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.263+0000 7f972ccec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 0x7f97280ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.263+0000 7f972ccec700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972815dfb0 con 0x7f972815dbd0 2023-10-16T11:21:59.271 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.263+0000 7f972ccec700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97280cd3a0 con 0x7f972806b6f0 2023-10-16T11:21:59.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.263+0000 7f972ccec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97280ce740 con 0x7f97280cdd80 2023-10-16T11:21:59.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.263+0000 7f972659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f972815dbd0 0x7f97280d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44174/0 (socket says 172.21.15.181:44174) 2023-10-16T11:21:59.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.263+0000 7f972659c700 1 -- 172.21.15.181:0/1123441756 learned_addr learned my addr 172.21.15.181:0/1123441756 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:21:59.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714375138 0 0) 0x7f971c001110 con 0x7f972815dbd0 2023-10-16T11:21:59.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97100035b0 con 0x7f972815dbd0 2023-10-16T11:21:59.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f9725d9b700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 0x7f97280ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 368731987 0 0) 0x7f97100035b0 con 0x7f972815dbd0 2023-10-16T11:21:59.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 msgr2=0x7f97280ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:59.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 0x7f97280ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 >> v1:172.21.15.190:6789/0 conn(0x7f972806b6f0 legacy=0x7f972806bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97280cd830 con 0x7f972815dbd0 2023-10-16T11:21:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f971c001800 con 0x7f972815dbd0 2023-10-16T11:21:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 >> v1:172.21.15.181:6789/0 conn(0x7f972815dbd0 legacy=0x7f97280d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f9710003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 0x7f9710004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9710004d90 0x7f9710005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97280ce740 con 0x7f9710003a30 2023-10-16T11:21:59.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971000bba0 con 0x7f9710004d90 2023-10-16T11:21:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971000bd30 con 0x7f97280cdd80 2023-10-16T11:21:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972659c700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f9710003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f9725d9b700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 0x7f9710004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f9726d9d700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9710004d90 0x7f9710005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972659c700 1 -- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 msgr2=0x7f9710004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972659c700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 0x7f9710004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972659c700 1 -- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9710004d90 msgr2=0x7f9710005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:21:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972659c700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9710004d90 0x7f9710005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972659c700 1 -- 172.21.15.181:0/1123441756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97100035b0 con 0x7f9710003a30 2023-10-16T11:21:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f9725d9b700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97280cdd80 0x7f9710004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:21:59.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972659c700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f9710003e50 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7f971c00c270 tx=0x7f9710005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badac0f8f3fdffd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:59.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.264+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c007db0 con 0x7f9710003a30 2023-10-16T11:21:59.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c00cdf0 con 0x7f9710003a30 2023-10-16T11:21:59.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972559a700 1 -- 172.21.15.181:0/1123441756 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c005ba0 con 0x7f9710003a30 2023-10-16T11:21:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972ccec700 1 -- 172.21.15.181:0/1123441756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 msgr2=0x7f9710003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972ccec700 1 --2- 172.21.15.181:0/1123441756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f9710003e50 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7f971c00c270 tx=0x7f9710005880 comp rx=0 tx=0).stop 2023-10-16T11:21:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f9726d9d700 1 -- 172.21.15.181:0/1123441756 reap_dead start 2023-10-16T11:21:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972ccec700 1 -- 172.21.15.181:0/1123441756 shutdown_connections 2023-10-16T11:21:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972ccec700 1 -- 172.21.15.181:0/1123441756 >> 172.21.15.181:0/1123441756 conn(0x7f972809a930 msgr2=0x7f97280d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:21:59.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972ccec700 1 -- 172.21.15.181:0/1123441756 shutdown_connections 2023-10-16T11:21:59.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.265+0000 7f972ccec700 1 -- 172.21.15.181:0/1123441756 wait complete. 2023-10-16T11:21:59.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.266+0000 7f972ccec700 1 Processor -- start 2023-10-16T11:21:59.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972ccec700 1 -- start start 2023-10-16T11:21:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972ccec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f972806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972ccec700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97280c86d0 0x7f97280c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972ccec700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9728152480 0x7f9728152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972ccec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97280c9170 con 0x7f9710003a30 2023-10-16T11:21:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972ccec700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97281f6a40 con 0x7f97280c86d0 2023-10-16T11:21:59.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972ccec700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9728152e40 con 0x7f9728152480 2023-10-16T11:21:59.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f972806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f972806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33108/0 (socket says 172.21.15.181:33108) 2023-10-16T11:21:59.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f972659c700 1 -- 172.21.15.181:0/1547254775 learned_addr learned my addr 172.21.15.181:0/1547254775 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:21:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f9725d9b700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97280c86d0 0x7f97280c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.267+0000 7f9726d9d700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9728152480 0x7f9728152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972659c700 1 -- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9728152480 msgr2=0x7f9728152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972659c700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9728152480 0x7f9728152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972659c700 1 -- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97280c86d0 msgr2=0x7f97280c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:21:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972659c700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97280c86d0 0x7f97280c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:21:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972659c700 1 -- 172.21.15.181:0/1547254775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97281f6c50 con 0x7f9710003a30 2023-10-16T11:21:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972659c700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f972806bdf0 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f971c001ac0 tx=0x7f971c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f9716ffd700 1 -- 172.21.15.181:0/1547254775 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0017c0 con 0x7f9710003a30 2023-10-16T11:21:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f9716ffd700 1 -- 172.21.15.181:0/1547254775 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c001c00 con 0x7f9710003a30 2023-10-16T11:21:59.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97281f6f30 con 0x7f9710003a30 2023-10-16T11:21:59.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f9716ffd700 1 -- 172.21.15.181:0/1547254775 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c007310 con 0x7f9710003a30 2023-10-16T11:21:59.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.268+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97281f74f0 con 0x7f9710003a30 2023-10-16T11:21:59.285 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.269+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97281f7680 con 0x7f9710003a30 2023-10-16T11:21:59.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.269+0000 7f9716ffd700 1 -- 172.21.15.181:0/1547254775 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f971c019030 con 0x7f9710003a30 2023-10-16T11:21:59.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.270+0000 7f9716ffd700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f970806b020 0x7f970806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:21:59.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.270+0000 7f9716ffd700 1 -- 172.21.15.181:0/1547254775 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f971c08ad90 con 0x7f9710003a30 2023-10-16T11:21:59.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.270+0000 7f9725d9b700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f970806b020 0x7f970806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:21:59.287 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.270+0000 7f9725d9b700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f970806b020 0x7f970806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97280b2d90 tx=0x7f9718008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:21:59.288 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.274+0000 7f9716ffd700 1 -- 172.21.15.181:0/1547254775 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f971c055c10 con 0x7f9710003a30 2023-10-16T11:21:59.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:21:59.513+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9728067d20 con 0x7f9710003a30 2023-10-16T11:22:00.120 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.118+0000 7f9716ffd700 1 -- 172.21.15.181:0/1547254775 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v153) v1 ==== 122+0+0 (secure 0 0 0) 0x7f971c0170c0 con 0x7f9710003a30 2023-10-16T11:22:00.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f970806b020 msgr2=0x7f970806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.123 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f970806b020 0x7f970806d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97280b2d90 tx=0x7f9718008040 comp rx=0 tx=0).stop 2023-10-16T11:22:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 msgr2=0x7f972806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f972806bdf0 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f971c001ac0 tx=0x7f971c001b90 comp rx=0 tx=0).stop 2023-10-16T11:22:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 shutdown_connections 2023-10-16T11:22:00.124 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f970806b020 0x7f970806d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9728152480 0x7f9728152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9710003a30 0x7f972806bdf0 unknown :-1 s=CLOSED pgs=3129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 --2- 172.21.15.181:0/1547254775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97280c86d0 0x7f97280c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 >> 172.21.15.181:0/1547254775 conn(0x7f972809a930 msgr2=0x7f972815b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:00.125 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 shutdown_connections 2023-10-16T11:22:00.126 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.121+0000 7f972ccec700 1 -- 172.21.15.181:0/1547254775 wait complete. 2023-10-16T11:22:00.138 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T11:22:00.138 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:00.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.260+0000 7efdc29ee700 1 Processor -- start 2023-10-16T11:22:00.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.260+0000 7efdc29ee700 1 -- start start 2023-10-16T11:22:00.264 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdc29ee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdbc15dfd0 0x7efdbc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdc29ee700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdbc0af450 0x7efdbc06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdc29ee700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc06f870 con 0x7efdbc06db80 2023-10-16T11:22:00.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdc29ee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc0d1160 con 0x7efdbc15dfd0 2023-10-16T11:22:00.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdc29ee700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc0cc8e0 con 0x7efdbc0af450 2023-10-16T11:22:00.265 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdbbfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdbc15dfd0 0x7efdbc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdc0f8b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efdbc06db80 0x7efdbc0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44198/0 (socket says 172.21.15.181:44198) 2023-10-16T11:22:00.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdc0f8b700 1 -- 172.21.15.181:0/2725220319 learned_addr learned my addr 172.21.15.181:0/2725220319 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:00.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdbbfff700 1 -- 172.21.15.181:0/2725220319 >> v2:172.21.15.190:3300/0 conn(0x7efdbc0af450 msgr2=0x7efdbc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:00.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdbbfff700 1 --2- 172.21.15.181:0/2725220319 >> v2:172.21.15.190:3300/0 conn(0x7efdbc0af450 0x7efdbc06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdbbfff700 1 -- 172.21.15.181:0/2725220319 >> v1:172.21.15.181:6789/0 conn(0x7efdbc06db80 legacy=0x7efdbc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdbbfff700 1 -- 172.21.15.181:0/2725220319 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0ccbc0 con 0x7efdbc15dfd0 2023-10-16T11:22:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdbaffd700 1 -- 172.21.15.181:0/2725220319 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203936764 0 0) 0x7efdbc06f870 con 0x7efdbc06db80 2023-10-16T11:22:00.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.261+0000 7efdbbfff700 1 --2- 172.21.15.181:0/2725220319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc15dfd0 0x7efdbc0af9a0 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0b3070 tx=0x7efdb0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4d75c825ac75da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdbaffd700 1 -- 172.21.15.181:0/2725220319 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0005e30 con 0x7efdbc15dfd0 2023-10-16T11:22:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdbaffd700 1 -- 172.21.15.181:0/2725220319 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb0006030 con 0x7efdbc15dfd0 2023-10-16T11:22:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdbaffd700 1 -- 172.21.15.181:0/2725220319 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb00063c0 con 0x7efdbc15dfd0 2023-10-16T11:22:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 -- 172.21.15.181:0/2725220319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc15dfd0 msgr2=0x7efdbc0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 --2- 172.21.15.181:0/2725220319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc15dfd0 0x7efdbc0af9a0 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0b3070 tx=0x7efdb0009b00 comp rx=0 tx=0).stop 2023-10-16T11:22:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 -- 172.21.15.181:0/2725220319 shutdown_connections 2023-10-16T11:22:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 --2- 172.21.15.181:0/2725220319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc15dfd0 0x7efdbc0af9a0 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 --2- 172.21.15.181:0/2725220319 >> v2:172.21.15.190:3300/0 conn(0x7efdbc0af450 0x7efdbc06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 -- 172.21.15.181:0/2725220319 >> 172.21.15.181:0/2725220319 conn(0x7efdbc0d04a0 msgr2=0x7efdbc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:00.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 -- 172.21.15.181:0/2725220319 shutdown_connections 2023-10-16T11:22:00.270 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.262+0000 7efdc29ee700 1 -- 172.21.15.181:0/2725220319 wait complete. 2023-10-16T11:22:00.272 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.263+0000 7efdc29ee700 1 Processor -- start 2023-10-16T11:22:00.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdc29ee700 1 -- start start 2023-10-16T11:22:00.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdc29ee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdbc06db80 0x7efdbc0b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdc29ee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc0af450 0x7efdbc15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdc29ee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdbc15dfd0 0x7efdbc0ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.273 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdc29ee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0c3fd0 con 0x7efdbc0af450 2023-10-16T11:22:00.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdc29ee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0c4590 con 0x7efdbc06db80 2023-10-16T11:22:00.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdc29ee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc1e8310 con 0x7efdbc15dfd0 2023-10-16T11:22:00.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdbb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc0af450 0x7efdbc15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdbb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc0af450 0x7efdbc15a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33124/0 (socket says 172.21.15.181:33124) 2023-10-16T11:22:00.274 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.264+0000 7efdbb7fe700 1 -- 172.21.15.181:0/4082815197 learned_addr learned my addr 172.21.15.181:0/4082815197 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:00.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdc0f8b700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdbc15dfd0 0x7efdbc0ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbbfff700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdbc06db80 0x7efdbc0b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbb7fe700 1 -- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdbc15dfd0 msgr2=0x7efdbc0ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbb7fe700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdbc15dfd0 0x7efdbc0ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbb7fe700 1 -- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdbc06db80 msgr2=0x7efdbc0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbb7fe700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdbc06db80 0x7efdbc0b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbb7fe700 1 -- 172.21.15.181:0/4082815197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb0009430 con 0x7efdbc0af450 2023-10-16T11:22:00.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbbfff700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdbc06db80 0x7efdbc0b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:00.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdbb7fe700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc0af450 0x7efdbc15a590 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7efdac00b560 tx=0x7efdac00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:00.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdb8ff9700 1 -- 172.21.15.181:0/4082815197 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac011830 con 0x7efdbc0af450 2023-10-16T11:22:00.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdb8ff9700 1 -- 172.21.15.181:0/4082815197 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac011990 con 0x7efdbc0af450 2023-10-16T11:22:00.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc1e85f0 con 0x7efdbc0af450 2023-10-16T11:22:00.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdb8ff9700 1 -- 172.21.15.181:0/4082815197 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac011d40 con 0x7efdbc0af450 2023-10-16T11:22:00.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.265+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc15b0f0 con 0x7efdbc0af450 2023-10-16T11:22:00.277 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.266+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc0c4720 con 0x7efdbc0af450 2023-10-16T11:22:00.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.266+0000 7efdb8ff9700 1 -- 172.21.15.181:0/4082815197 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efdac01a400 con 0x7efdbc0af450 2023-10-16T11:22:00.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.267+0000 7efdb8ff9700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.267+0000 7efdb8ff9700 1 -- 172.21.15.181:0/4082815197 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7efdac089950 con 0x7efdbc0af450 2023-10-16T11:22:00.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.267+0000 7efdbbfff700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.267+0000 7efdbbfff700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7efda806b020 0x7efda806d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdb0013040 tx=0x7efdb0008070 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:00.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.271+0000 7efdb8ff9700 1 -- 172.21.15.181:0/4082815197 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7efdac054750 con 0x7efdbc0af450 2023-10-16T11:22:00.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.509+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdbc0cdb00 con 0x7efdbc0af450 2023-10-16T11:22:00.512 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.510+0000 7efdb8ff9700 1 -- 172.21.15.181:0/4082815197 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+4932 (secure 0 0 0) 0x7efdac015020 con 0x7efdbc0af450 2023-10-16T11:22:00.512 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:00.513 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":153,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/984505218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":984505218},{"type":"v1","addr":"172.21.15.190:6835","nonce":984505218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/350629849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":350629849},{"type":"v1","addr":"172.21.15.181:6835","nonce":350629849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13440,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1354411415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1354411415},{"type":"v1","addr":"172.21.15.181:6837","nonce":1354411415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:56.485392+0000","modified":"2023-10-16T11:21:59.515101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13452":{"gid":13452,"name":"b","rank":0,"incarnation":149,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/284709222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":284709222},{"type":"v1","addr":"172.21.15.190:6834","nonce":284709222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:22:00.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7efda806b020 msgr2=0x7efda806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7efda806b020 0x7efda806d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdb0013040 tx=0x7efdb0008070 comp rx=0 tx=0).stop 2023-10-16T11:22:00.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc0af450 msgr2=0x7efdbc15a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc0af450 0x7efdbc15a590 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7efdac00b560 tx=0x7efdac00b920 comp rx=0 tx=0).stop 2023-10-16T11:22:00.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 shutdown_connections 2023-10-16T11:22:00.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7efda806b020 0x7efda806d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdbc15dfd0 0x7efdbc0ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdbc06db80 0x7efdbc0b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 --2- 172.21.15.181:0/4082815197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdbc0af450 0x7efdbc15a590 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 >> 172.21.15.181:0/4082815197 conn(0x7efdbc0d04a0 msgr2=0x7efdbc15ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:00.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 shutdown_connections 2023-10-16T11:22:00.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.512+0000 7efdc29ee700 1 -- 172.21.15.181:0/4082815197 wait complete. 2023-10-16T11:22:00.517 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 153 2023-10-16T11:22:00.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:00.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.645+0000 7f85dda1c700 1 Processor -- start 2023-10-16T11:22:00.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.645+0000 7f85dda1c700 1 -- start start 2023-10-16T11:22:00.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85dda1c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d815e6d0 0x7f85d80d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85dda1c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85d815eab0 0x7f85d815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85dda1c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d806ed00 con 0x7f85d80cb0c0 2023-10-16T11:22:00.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85dda1c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d80d02d0 con 0x7f85d815eab0 2023-10-16T11:22:00.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85dda1c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d815ad40 con 0x7f85d815e6d0 2023-10-16T11:22:00.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85d67fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85d815eab0 0x7f85d815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85d77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f85d80cb0c0 0x7f85d815a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44210/0 (socket says 172.21.15.181:44210) 2023-10-16T11:22:00.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85d67fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85d815eab0 0x7f85d815a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33130/0 (socket says 172.21.15.181:33130) 2023-10-16T11:22:00.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85d77fe700 1 -- 172.21.15.181:0/98484709 learned_addr learned my addr 172.21.15.181:0/98484709 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:00.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.646+0000 7f85d6ffd700 1 --2- 172.21.15.181:0/98484709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d815e6d0 0x7f85d80d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d67fc700 1 -- 172.21.15.181:0/98484709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d815e6d0 msgr2=0x7f85d80d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:00.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d67fc700 1 --2- 172.21.15.181:0/98484709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d815e6d0 0x7f85d80d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d67fc700 1 -- 172.21.15.181:0/98484709 >> v1:172.21.15.181:6789/0 conn(0x7f85d80cb0c0 legacy=0x7f85d815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d67fc700 1 -- 172.21.15.181:0/98484709 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d806e0f0 con 0x7f85d815eab0 2023-10-16T11:22:00.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d5ffb700 1 -- 172.21.15.181:0/98484709 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6395766 0 0) 0x7f85d806ed00 con 0x7f85d80cb0c0 2023-10-16T11:22:00.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d67fc700 1 --2- 172.21.15.181:0/98484709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815eab0 0x7f85d815a0f0 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009840 tx=0x7f85c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f888125578bade48 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:00.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d5ffb700 1 -- 172.21.15.181:0/98484709 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8011830 con 0x7f85d815eab0 2023-10-16T11:22:00.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d5ffb700 1 -- 172.21.15.181:0/98484709 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c8011990 con 0x7f85d815eab0 2023-10-16T11:22:00.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85d5ffb700 1 -- 172.21.15.181:0/98484709 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8011d20 con 0x7f85d815eab0 2023-10-16T11:22:00.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85dda1c700 1 -- 172.21.15.181:0/98484709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815eab0 msgr2=0x7f85d815a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.647+0000 7f85dda1c700 1 --2- 172.21.15.181:0/98484709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815eab0 0x7f85d815a0f0 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009840 tx=0x7f85c8009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:00.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.648+0000 7f85dda1c700 1 -- 172.21.15.181:0/98484709 shutdown_connections 2023-10-16T11:22:00.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.648+0000 7f85dda1c700 1 --2- 172.21.15.181:0/98484709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d815e6d0 0x7f85d80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.648+0000 7f85dda1c700 1 --2- 172.21.15.181:0/98484709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815eab0 0x7f85d815a0f0 unknown :-1 s=CLOSED pgs=3134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.648+0000 7f85dda1c700 1 -- 172.21.15.181:0/98484709 >> 172.21.15.181:0/98484709 conn(0x7f85d80d04a0 msgr2=0x7f85d80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:00.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.648+0000 7f85dda1c700 1 -- 172.21.15.181:0/98484709 shutdown_connections 2023-10-16T11:22:00.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.648+0000 7f85dda1c700 1 -- 172.21.15.181:0/98484709 wait complete. 2023-10-16T11:22:00.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.649+0000 7f85dda1c700 1 Processor -- start 2023-10-16T11:22:00.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.649+0000 7f85dda1c700 1 -- start start 2023-10-16T11:22:00.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d80cb0c0 0x7f85d8158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815e6d0 0x7f85d80cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d815eab0 0x7f85d80cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d815d5f0 con 0x7f85d815e6d0 2023-10-16T11:22:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80c9fd0 con 0x7f85d815eab0 2023-10-16T11:22:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d815c9e0 con 0x7f85d80cb0c0 2023-10-16T11:22:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815e6d0 0x7f85d80cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815e6d0 0x7f85d80cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33136/0 (socket says 172.21.15.181:33136) 2023-10-16T11:22:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 -- 172.21.15.181:0/3740051414 learned_addr learned my addr 172.21.15.181:0/3740051414 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d6ffd700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d80cb0c0 0x7f85d8158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d77fe700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d815eab0 0x7f85d80cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 -- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d80cb0c0 msgr2=0x7f85d8158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d80cb0c0 0x7f85d8158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 -- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d815eab0 msgr2=0x7f85d80cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d815eab0 0x7f85d80cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 -- 172.21.15.181:0/3740051414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80094a0 con 0x7f85d815e6d0 2023-10-16T11:22:00.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85d67fc700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815e6d0 0x7f85d80cf200 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009fd0 tx=0x7f85c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:00.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85bffff700 1 -- 172.21.15.181:0/3740051414 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80036a0 con 0x7f85d815e6d0 2023-10-16T11:22:00.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85bffff700 1 -- 172.21.15.181:0/3740051414 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c8003800 con 0x7f85d815e6d0 2023-10-16T11:22:00.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d815ccc0 con 0x7f85d815e6d0 2023-10-16T11:22:00.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85bffff700 1 -- 172.21.15.181:0/3740051414 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8003b90 con 0x7f85d815e6d0 2023-10-16T11:22:00.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.650+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d815d280 con 0x7f85d815e6d0 2023-10-16T11:22:00.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.651+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d8067d20 con 0x7f85d815e6d0 2023-10-16T11:22:00.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.652+0000 7f85bffff700 1 -- 172.21.15.181:0/3740051414 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f85c8003cf0 con 0x7f85d815e6d0 2023-10-16T11:22:00.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.653+0000 7f85bffff700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f85c406b020 0x7f85c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.653+0000 7f85bffff700 1 -- 172.21.15.181:0/3740051414 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f85c808cef0 con 0x7f85d815e6d0 2023-10-16T11:22:00.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.653+0000 7f85d6ffd700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f85c406b020 0x7f85c406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.653+0000 7f85d6ffd700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f85c406b020 0x7f85c406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009b20 tx=0x7f85cc0091c0 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:00.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.657+0000 7f85bffff700 1 -- 172.21.15.181:0/3740051414 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f85c8057cd0 con 0x7f85d815e6d0 2023-10-16T11:22:00.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.851+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85d815d730 con 0x7f85d815e6d0 2023-10-16T11:22:00.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.852+0000 7f85bffff700 1 -- 172.21.15.181:0/3740051414 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v269) v1 ==== 74+0+17414 (secure 0 0 0) 0x7f85c805b2f0 con 0x7f85d815e6d0 2023-10-16T11:22:00.854 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:00.854 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":269,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:21:58.101125+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:54.910438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:55.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:00.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f85c406b020 msgr2=0x7f85c406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f85c406b020 0x7f85c406d4e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009b20 tx=0x7f85cc0091c0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815e6d0 msgr2=0x7f85d80cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815e6d0 0x7f85d80cf200 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009fd0 tx=0x7f85c8004750 comp rx=0 tx=0).stop 2023-10-16T11:22:00.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 shutdown_connections 2023-10-16T11:22:00.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f85c406b020 0x7f85c406d4e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d80cb0c0 0x7f85d8158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d815eab0 0x7f85d80cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 --2- 172.21.15.181:0/3740051414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85d815e6d0 0x7f85d80cf200 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 >> 172.21.15.181:0/3740051414 conn(0x7f85d80d04a0 msgr2=0x7f85d80ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:00.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 shutdown_connections 2023-10-16T11:22:00.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.854+0000 7f85dda1c700 1 -- 172.21.15.181:0/3740051414 wait complete. 2023-10-16T11:22:00.869 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T11:22:00.869 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:00.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.990+0000 7faf59259700 1 Processor -- start 2023-10-16T11:22:00.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.990+0000 7faf59259700 1 -- start start 2023-10-16T11:22:00.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf59259700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf540d0370 0x7faf540ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf59259700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf540cd2d0 0x7faf540d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:00.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf59259700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf540cc510 con 0x7faf5415b990 2023-10-16T11:22:00.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf59259700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf540d24f0 con 0x7faf540cd2d0 2023-10-16T11:22:00.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf59259700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf540cc7f0 con 0x7faf540d0370 2023-10-16T11:22:00.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf5359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf540cd2d0 0x7faf540d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:00.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf52d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faf5415b990 0x7faf540cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44220/0 (socket says 172.21.15.181:44220) 2023-10-16T11:22:00.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf52d9d700 1 -- 172.21.15.181:0/871340768 learned_addr learned my addr 172.21.15.181:0/871340768 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:00.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf5359e700 1 -- 172.21.15.181:0/871340768 >> v2:172.21.15.190:3300/0 conn(0x7faf540d0370 msgr2=0x7faf540ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:00.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf5359e700 1 --2- 172.21.15.181:0/871340768 >> v2:172.21.15.190:3300/0 conn(0x7faf540d0370 0x7faf540ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:00.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf5359e700 1 -- 172.21.15.181:0/871340768 >> v1:172.21.15.181:6789/0 conn(0x7faf5415b990 legacy=0x7faf540cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf5359e700 1 -- 172.21.15.181:0/871340768 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540ccad0 con 0x7faf540cd2d0 2023-10-16T11:22:00.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf51d9b700 1 -- 172.21.15.181:0/871340768 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582671938 0 0) 0x7faf540cc510 con 0x7faf5415b990 2023-10-16T11:22:00.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf5359e700 1 --2- 172.21.15.181:0/871340768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf540d1fb0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7faf540b4230 tx=0x7faf4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed5e0bf7bd5a932 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:00.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.991+0000 7faf51d9b700 1 -- 172.21.15.181:0/871340768 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00e070 con 0x7faf540cd2d0 2023-10-16T11:22:00.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf51d9b700 1 -- 172.21.15.181:0/871340768 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c00bbd0 con 0x7faf540cd2d0 2023-10-16T11:22:00.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf51d9b700 1 -- 172.21.15.181:0/871340768 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c012430 con 0x7faf540cd2d0 2023-10-16T11:22:00.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 -- 172.21.15.181:0/871340768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 msgr2=0x7faf540d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:00.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 --2- 172.21.15.181:0/871340768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf540d1fb0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7faf540b4230 tx=0x7faf4c00b330 comp rx=0 tx=0).stop 2023-10-16T11:22:00.999 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 -- 172.21.15.181:0/871340768 shutdown_connections 2023-10-16T11:22:01.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 --2- 172.21.15.181:0/871340768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf540d1fb0 unknown :-1 s=CLOSED pgs=3137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 --2- 172.21.15.181:0/871340768 >> v2:172.21.15.190:3300/0 conn(0x7faf540d0370 0x7faf540ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 -- 172.21.15.181:0/871340768 >> 172.21.15.181:0/871340768 conn(0x7faf5409a9a0 msgr2=0x7faf540d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:01.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 -- 172.21.15.181:0/871340768 shutdown_connections 2023-10-16T11:22:01.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.992+0000 7faf59259700 1 -- 172.21.15.181:0/871340768 wait complete. 2023-10-16T11:22:01.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 Processor -- start 2023-10-16T11:22:01.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 -- start start 2023-10-16T11:22:01.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf541f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf540d0370 0x7faf5414f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf5415b990 0x7faf5415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541598e0 con 0x7faf540cd2d0 2023-10-16T11:22:01.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf540c9b10 con 0x7faf540d0370 2023-10-16T11:22:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf59259700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf541f1f10 con 0x7faf5415b990 2023-10-16T11:22:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf52d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf541f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf52d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf541f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33148/0 (socket says 172.21.15.181:33148) 2023-10-16T11:22:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.994+0000 7faf52d9d700 1 -- 172.21.15.181:0/3750415666 learned_addr learned my addr 172.21.15.181:0/3750415666 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:01.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf5359e700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf5415b990 0x7faf5415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf5259c700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf540d0370 0x7faf5414f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf52d9d700 1 -- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf5415b990 msgr2=0x7faf5415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf52d9d700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf5415b990 0x7faf5415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf52d9d700 1 -- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf540d0370 msgr2=0x7faf5414f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf52d9d700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf540d0370 0x7faf5414f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf52d9d700 1 -- 172.21.15.181:0/3750415666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c009a70 con 0x7faf540cd2d0 2023-10-16T11:22:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf52d9d700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf541f19d0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7faf480010e0 tx=0x7faf4800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf3b7fe700 1 -- 172.21.15.181:0/3750415666 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480074e0 con 0x7faf540cd2d0 2023-10-16T11:22:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf3b7fe700 1 -- 172.21.15.181:0/3750415666 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48007640 con 0x7faf540cd2d0 2023-10-16T11:22:01.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf540c8cc0 con 0x7faf540cd2d0 2023-10-16T11:22:01.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf3b7fe700 1 -- 172.21.15.181:0/3750415666 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48005a70 con 0x7faf540cd2d0 2023-10-16T11:22:01.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.995+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf540c9280 con 0x7faf540cd2d0 2023-10-16T11:22:01.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.996+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf540c9c80 con 0x7faf540cd2d0 2023-10-16T11:22:01.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.997+0000 7faf3b7fe700 1 -- 172.21.15.181:0/3750415666 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faf48005bd0 con 0x7faf540cd2d0 2023-10-16T11:22:01.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.997+0000 7faf3b7fe700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7faf4006b020 0x7faf4006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.997+0000 7faf3b7fe700 1 -- 172.21.15.181:0/3750415666 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7faf48088e00 con 0x7faf540cd2d0 2023-10-16T11:22:01.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.997+0000 7faf5259c700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7faf4006b020 0x7faf4006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:00.998+0000 7faf5259c700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7faf4006b020 0x7faf4006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf54159be0 tx=0x7faf44009110 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.002+0000 7faf3b7fe700 1 -- 172.21.15.181:0/3750415666 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7faf48053b70 con 0x7faf540cd2d0 2023-10-16T11:22:01.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.189+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 --> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf540cc200 con 0x7faf4006b020 2023-10-16T11:22:01.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.197+0000 7faf3b7fe700 1 -- 172.21.15.181:0/3750415666 <== mgr.13197 v2:172.21.15.181:6832/2291437477 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290159 (secure 0 0 0) 0x7faf44009e10 con 0x7faf4006b020 2023-10-16T11:22:01.202 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:01.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7faf4006b020 msgr2=0x7faf4006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7faf4006b020 0x7faf4006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faf54159be0 tx=0x7faf44009110 comp rx=0 tx=0).stop 2023-10-16T11:22:01.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 msgr2=0x7faf541f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf541f19d0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7faf480010e0 tx=0x7faf4800ca30 comp rx=0 tx=0).stop 2023-10-16T11:22:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 shutdown_connections 2023-10-16T11:22:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7faf4006b020 0x7faf4006d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf5415b990 0x7faf5415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf540cd2d0 0x7faf541f19d0 unknown :-1 s=CLOSED pgs=3138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 --2- 172.21.15.181:0/3750415666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf540d0370 0x7faf5414f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.200+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 >> 172.21.15.181:0/3750415666 conn(0x7faf5409a9a0 msgr2=0x7faf540d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.201+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 shutdown_connections 2023-10-16T11:22:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.201+0000 7faf59259700 1 -- 172.21.15.181:0/3750415666 wait complete. 2023-10-16T11:22:01.208 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:22:01.215 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T11:22:00.887972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":723,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":441,"ondisk_log_size":441,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":482564,"kb_used_data":27712,"kb_used_omap":12,"kb_used_meta":454707,"kb_avail":754492156,"statfs":{"total":773094113280,"available":772599967744,"internally_reserved":0,"allocated":28377088,"data_stored":15282540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":465620536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815763"},"pg_stats":[{"pgid":"37.3f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110444+0000","last_change":"2023-10-16T11:21:57.103408+0000","last_active":"2023-10-16T11:21:58.110444+0000","last_peered":"2023-10-16T11:21:58.110444+0000","last_clean":"2023-10-16T11:21:58.110444+0000","last_became_active":"2023-10-16T11:21:57.103039+0000","last_became_peered":"2023-10-16T11:21:57.103039+0000","last_unstale":"2023-10-16T11:21:58.110444+0000","last_undegraded":"2023-10-16T11:21:58.110444+0000","last_fullsized":"2023-10-16T11:21:58.110444+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:59.536665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.3e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599838+0000","last_change":"2023-10-16T11:21:57.104067+0000","last_active":"2023-10-16T11:21:58.599838+0000","last_peered":"2023-10-16T11:21:58.599838+0000","last_clean":"2023-10-16T11:21:58.599838+0000","last_became_active":"2023-10-16T11:21:57.103794+0000","last_became_peered":"2023-10-16T11:21:57.103794+0000","last_unstale":"2023-10-16T11:21:58.599838+0000","last_undegraded":"2023-10-16T11:21:58.599838+0000","last_fullsized":"2023-10-16T11:21:58.599838+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:13.049091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.3d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110113+0000","last_change":"2023-10-16T11:21:57.108404+0000","last_active":"2023-10-16T11:21:58.110113+0000","last_peered":"2023-10-16T11:21:58.110113+0000","last_clean":"2023-10-16T11:21:58.110113+0000","last_became_active":"2023-10-16T11:21:57.104988+0000","last_became_peered":"2023-10-16T11:21:57.104988+0000","last_unstale":"2023-10-16T11:21:58.110113+0000","last_undegraded":"2023-10-16T11:21:58.110113+0000","last_fullsized":"2023-10-16T11:21:58.110113+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.986005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110263+0000","last_change":"2023-10-16T11:21:57.104802+0000","last_active":"2023-10-16T11:21:58.110263+0000","last_peered":"2023-10-16T11:21:58.110263+0000","last_clean":"2023-10-16T11:21:58.110263+0000","last_became_active":"2023-10-16T11:21:57.104462+0000","last_became_peered":"2023-10-16T11:21:57.104462+0000","last_unstale":"2023-10-16T11:21:58.110263+0000","last_undegraded":"2023-10-16T11:21:58.110263+0000","last_fullsized":"2023-10-16T11:21:58.110263+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:05.307751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110334+0000","last_change":"2023-10-16T11:21:57.108578+0000","last_active":"2023-10-16T11:21:58.110334+0000","last_peered":"2023-10-16T11:21:58.110334+0000","last_clean":"2023-10-16T11:21:58.110334+0000","last_became_active":"2023-10-16T11:21:57.104875+0000","last_became_peered":"2023-10-16T11:21:57.104875+0000","last_unstale":"2023-10-16T11:21:58.110334+0000","last_undegraded":"2023-10-16T11:21:58.110334+0000","last_fullsized":"2023-10-16T11:21:58.110334+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:39.224053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108886+0000","last_change":"2023-10-16T11:21:57.100994+0000","last_active":"2023-10-16T11:21:58.108886+0000","last_peered":"2023-10-16T11:21:58.108886+0000","last_clean":"2023-10-16T11:21:58.108886+0000","last_became_active":"2023-10-16T11:21:57.100540+0000","last_became_peered":"2023-10-16T11:21:57.100540+0000","last_unstale":"2023-10-16T11:21:58.108886+0000","last_undegraded":"2023-10-16T11:21:58.108886+0000","last_fullsized":"2023-10-16T11:21:58.108886+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:46.618520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.39","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109944+0000","last_change":"2023-10-16T11:21:57.101674+0000","last_active":"2023-10-16T11:21:58.109944+0000","last_peered":"2023-10-16T11:21:58.109944+0000","last_clean":"2023-10-16T11:21:58.109944+0000","last_became_active":"2023-10-16T11:21:57.101318+0000","last_became_peered":"2023-10-16T11:21:57.101318+0000","last_unstale":"2023-10-16T11:21:58.109944+0000","last_undegraded":"2023-10-16T11:21:58.109944+0000","last_fullsized":"2023-10-16T11:21:58.109944+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:02:39.873287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.38","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599674+0000","last_change":"2023-10-16T11:21:57.103737+0000","last_active":"2023-10-16T11:21:58.599674+0000","last_peered":"2023-10-16T11:21:58.599674+0000","last_clean":"2023-10-16T11:21:58.599674+0000","last_became_active":"2023-10-16T11:21:57.103353+0000","last_became_peered":"2023-10-16T11:21:57.103353+0000","last_unstale":"2023-10-16T11:21:58.599674+0000","last_undegraded":"2023-10-16T11:21:58.599674+0000","last_fullsized":"2023-10-16T11:21:58.599674+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:13.461214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.37","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600379+0000","last_change":"2023-10-16T11:21:57.106474+0000","last_active":"2023-10-16T11:21:58.600379+0000","last_peered":"2023-10-16T11:21:58.600379+0000","last_clean":"2023-10-16T11:21:58.600379+0000","last_became_active":"2023-10-16T11:21:57.106174+0000","last_became_peered":"2023-10-16T11:21:57.106174+0000","last_unstale":"2023-10-16T11:21:58.600379+0000","last_undegraded":"2023-10-16T11:21:58.600379+0000","last_fullsized":"2023-10-16T11:21:58.600379+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:20:26.953180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.36","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110348+0000","last_change":"2023-10-16T11:21:57.110568+0000","last_active":"2023-10-16T11:21:58.110348+0000","last_peered":"2023-10-16T11:21:58.110348+0000","last_clean":"2023-10-16T11:21:58.110348+0000","last_became_active":"2023-10-16T11:21:57.110283+0000","last_became_peered":"2023-10-16T11:21:57.110283+0000","last_unstale":"2023-10-16T11:21:58.110348+0000","last_undegraded":"2023-10-16T11:21:58.110348+0000","last_fullsized":"2023-10-16T11:21:58.110348+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:09.611531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.35","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599963+0000","last_change":"2023-10-16T11:21:57.107717+0000","last_active":"2023-10-16T11:21:58.599963+0000","last_peered":"2023-10-16T11:21:58.599963+0000","last_clean":"2023-10-16T11:21:58.599963+0000","last_became_active":"2023-10-16T11:21:57.107452+0000","last_became_peered":"2023-10-16T11:21:57.107452+0000","last_unstale":"2023-10-16T11:21:58.599963+0000","last_undegraded":"2023-10-16T11:21:58.599963+0000","last_fullsized":"2023-10-16T11:21:58.599963+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:41.982466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.34","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600753+0000","last_change":"2023-10-16T11:21:57.102771+0000","last_active":"2023-10-16T11:21:58.600753+0000","last_peered":"2023-10-16T11:21:58.600753+0000","last_clean":"2023-10-16T11:21:58.600753+0000","last_became_active":"2023-10-16T11:21:57.102499+0000","last_became_peered":"2023-10-16T11:21:57.102499+0000","last_unstale":"2023-10-16T11:21:58.600753+0000","last_undegraded":"2023-10-16T11:21:58.600753+0000","last_fullsized":"2023-10-16T11:21:58.600753+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:09.764520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.33","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600040+0000","last_change":"2023-10-16T11:21:57.113283+0000","last_active":"2023-10-16T11:21:58.600040+0000","last_peered":"2023-10-16T11:21:58.600040+0000","last_clean":"2023-10-16T11:21:58.600040+0000","last_became_active":"2023-10-16T11:21:57.110770+0000","last_became_peered":"2023-10-16T11:21:57.110770+0000","last_unstale":"2023-10-16T11:21:58.600040+0000","last_undegraded":"2023-10-16T11:21:58.600040+0000","last_fullsized":"2023-10-16T11:21:58.600040+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:15.358414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.32","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600825+0000","last_change":"2023-10-16T11:21:57.110696+0000","last_active":"2023-10-16T11:21:58.600825+0000","last_peered":"2023-10-16T11:21:58.600825+0000","last_clean":"2023-10-16T11:21:58.600825+0000","last_became_active":"2023-10-16T11:21:57.110420+0000","last_became_peered":"2023-10-16T11:21:57.110420+0000","last_unstale":"2023-10-16T11:21:58.600825+0000","last_undegraded":"2023-10-16T11:21:58.600825+0000","last_fullsized":"2023-10-16T11:21:58.600825+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:57.596577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.31","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110004+0000","last_change":"2023-10-16T11:21:57.119288+0000","last_active":"2023-10-16T11:21:58.110004+0000","last_peered":"2023-10-16T11:21:58.110004+0000","last_clean":"2023-10-16T11:21:58.110004+0000","last_became_active":"2023-10-16T11:21:57.117885+0000","last_became_peered":"2023-10-16T11:21:57.117885+0000","last_unstale":"2023-10-16T11:21:58.110004+0000","last_undegraded":"2023-10-16T11:21:58.110004+0000","last_fullsized":"2023-10-16T11:21:58.110004+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:55:16.872898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.30","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600580+0000","last_change":"2023-10-16T11:21:57.113483+0000","last_active":"2023-10-16T11:21:58.600580+0000","last_peered":"2023-10-16T11:21:58.600580+0000","last_clean":"2023-10-16T11:21:58.600580+0000","last_became_active":"2023-10-16T11:21:57.113216+0000","last_became_peered":"2023-10-16T11:21:57.113216+0000","last_unstale":"2023-10-16T11:21:58.600580+0000","last_undegraded":"2023-10-16T11:21:58.600580+0000","last_fullsized":"2023-10-16T11:21:58.600580+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:31.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600845+0000","last_change":"2023-10-16T11:21:57.112197+0000","last_active":"2023-10-16T11:21:58.600845+0000","last_peered":"2023-10-16T11:21:58.600845+0000","last_clean":"2023-10-16T11:21:58.600845+0000","last_became_active":"2023-10-16T11:21:57.110851+0000","last_became_peered":"2023-10-16T11:21:57.110851+0000","last_unstale":"2023-10-16T11:21:58.600845+0000","last_undegraded":"2023-10-16T11:21:58.600845+0000","last_fullsized":"2023-10-16T11:21:58.600845+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:09.263907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600733+0000","last_change":"2023-10-16T11:21:57.112831+0000","last_active":"2023-10-16T11:21:58.600733+0000","last_peered":"2023-10-16T11:21:58.600733+0000","last_clean":"2023-10-16T11:21:58.600733+0000","last_became_active":"2023-10-16T11:21:57.111893+0000","last_became_peered":"2023-10-16T11:21:57.111893+0000","last_unstale":"2023-10-16T11:21:58.600733+0000","last_undegraded":"2023-10-16T11:21:58.600733+0000","last_fullsized":"2023-10-16T11:21:58.600733+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:08.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600870+0000","last_change":"2023-10-16T11:21:57.109112+0000","last_active":"2023-10-16T11:21:58.600870+0000","last_peered":"2023-10-16T11:21:58.600870+0000","last_clean":"2023-10-16T11:21:58.600870+0000","last_became_active":"2023-10-16T11:21:57.108466+0000","last_became_peered":"2023-10-16T11:21:57.108466+0000","last_unstale":"2023-10-16T11:21:58.600870+0000","last_undegraded":"2023-10-16T11:21:58.600870+0000","last_fullsized":"2023-10-16T11:21:58.600870+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:40.013407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.2c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599932+0000","last_change":"2023-10-16T11:21:57.103066+0000","last_active":"2023-10-16T11:21:58.599932+0000","last_peered":"2023-10-16T11:21:58.599932+0000","last_clean":"2023-10-16T11:21:58.599932+0000","last_became_active":"2023-10-16T11:21:57.102752+0000","last_became_peered":"2023-10-16T11:21:57.102752+0000","last_unstale":"2023-10-16T11:21:58.599932+0000","last_undegraded":"2023-10-16T11:21:58.599932+0000","last_fullsized":"2023-10-16T11:21:58.599932+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:27.717350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.2b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600017+0000","last_change":"2023-10-16T11:21:57.107889+0000","last_active":"2023-10-16T11:21:58.600017+0000","last_peered":"2023-10-16T11:21:58.600017+0000","last_clean":"2023-10-16T11:21:58.600017+0000","last_became_active":"2023-10-16T11:21:57.107079+0000","last_became_peered":"2023-10-16T11:21:57.107079+0000","last_unstale":"2023-10-16T11:21:58.600017+0000","last_undegraded":"2023-10-16T11:21:58.600017+0000","last_fullsized":"2023-10-16T11:21:58.600017+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:10:26.049018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.2a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109930+0000","last_change":"2023-10-16T11:21:57.112519+0000","last_active":"2023-10-16T11:21:58.109930+0000","last_peered":"2023-10-16T11:21:58.109930+0000","last_clean":"2023-10-16T11:21:58.109930+0000","last_became_active":"2023-10-16T11:21:57.112184+0000","last_became_peered":"2023-10-16T11:21:57.112184+0000","last_unstale":"2023-10-16T11:21:58.109930+0000","last_undegraded":"2023-10-16T11:21:58.109930+0000","last_fullsized":"2023-10-16T11:21:58.109930+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:35.169498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.29","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601266+0000","last_change":"2023-10-16T11:21:57.110605+0000","last_active":"2023-10-16T11:21:58.601266+0000","last_peered":"2023-10-16T11:21:58.601266+0000","last_clean":"2023-10-16T11:21:58.601266+0000","last_became_active":"2023-10-16T11:21:57.110315+0000","last_became_peered":"2023-10-16T11:21:57.110315+0000","last_unstale":"2023-10-16T11:21:58.601266+0000","last_undegraded":"2023-10-16T11:21:58.601266+0000","last_fullsized":"2023-10-16T11:21:58.601266+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:07.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.28","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601086+0000","last_change":"2023-10-16T11:21:57.108878+0000","last_active":"2023-10-16T11:21:58.601086+0000","last_peered":"2023-10-16T11:21:58.601086+0000","last_clean":"2023-10-16T11:21:58.601086+0000","last_became_active":"2023-10-16T11:21:57.108573+0000","last_became_peered":"2023-10-16T11:21:57.108573+0000","last_unstale":"2023-10-16T11:21:58.601086+0000","last_undegraded":"2023-10-16T11:21:58.601086+0000","last_fullsized":"2023-10-16T11:21:58.601086+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:38.421119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.27","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600992+0000","last_change":"2023-10-16T11:21:57.114221+0000","last_active":"2023-10-16T11:21:58.600992+0000","last_peered":"2023-10-16T11:21:58.600992+0000","last_clean":"2023-10-16T11:21:58.600992+0000","last_became_active":"2023-10-16T11:21:57.113963+0000","last_became_peered":"2023-10-16T11:21:57.113963+0000","last_unstale":"2023-10-16T11:21:58.600992+0000","last_undegraded":"2023-10-16T11:21:58.600992+0000","last_fullsized":"2023-10-16T11:21:58.600992+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:46.242576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.26","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110380+0000","last_change":"2023-10-16T11:21:57.109269+0000","last_active":"2023-10-16T11:21:58.110380+0000","last_peered":"2023-10-16T11:21:58.110380+0000","last_clean":"2023-10-16T11:21:58.110380+0000","last_became_active":"2023-10-16T11:21:57.108973+0000","last_became_peered":"2023-10-16T11:21:57.108973+0000","last_unstale":"2023-10-16T11:21:58.110380+0000","last_undegraded":"2023-10-16T11:21:58.110380+0000","last_fullsized":"2023-10-16T11:21:58.110380+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:49.076157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.25","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600932+0000","last_change":"2023-10-16T11:21:57.106793+0000","last_active":"2023-10-16T11:21:58.600932+0000","last_peered":"2023-10-16T11:21:58.600932+0000","last_clean":"2023-10-16T11:21:58.600932+0000","last_became_active":"2023-10-16T11:21:57.106546+0000","last_became_peered":"2023-10-16T11:21:57.106546+0000","last_unstale":"2023-10-16T11:21:58.600932+0000","last_undegraded":"2023-10-16T11:21:58.600932+0000","last_fullsized":"2023-10-16T11:21:58.600932+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:06.761299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.23","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108899+0000","last_change":"2023-10-16T11:21:57.104108+0000","last_active":"2023-10-16T11:21:58.108899+0000","last_peered":"2023-10-16T11:21:58.108899+0000","last_clean":"2023-10-16T11:21:58.108899+0000","last_became_active":"2023-10-16T11:21:57.103804+0000","last_became_peered":"2023-10-16T11:21:57.103804+0000","last_unstale":"2023-10-16T11:21:58.108899+0000","last_undegraded":"2023-10-16T11:21:58.108899+0000","last_fullsized":"2023-10-16T11:21:58.108899+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:33.653509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.22","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600553+0000","last_change":"2023-10-16T11:21:57.114747+0000","last_active":"2023-10-16T11:21:58.600553+0000","last_peered":"2023-10-16T11:21:58.600553+0000","last_clean":"2023-10-16T11:21:58.600553+0000","last_became_active":"2023-10-16T11:21:57.114485+0000","last_became_peered":"2023-10-16T11:21:57.114485+0000","last_unstale":"2023-10-16T11:21:58.600553+0000","last_undegraded":"2023-10-16T11:21:58.600553+0000","last_fullsized":"2023-10-16T11:21:58.600553+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:20.214848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.21","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599894+0000","last_change":"2023-10-16T11:21:57.107795+0000","last_active":"2023-10-16T11:21:58.599894+0000","last_peered":"2023-10-16T11:21:58.599894+0000","last_clean":"2023-10-16T11:21:58.599894+0000","last_became_active":"2023-10-16T11:21:57.107300+0000","last_became_peered":"2023-10-16T11:21:57.107300+0000","last_unstale":"2023-10-16T11:21:58.599894+0000","last_undegraded":"2023-10-16T11:21:58.599894+0000","last_fullsized":"2023-10-16T11:21:58.599894+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:19.235355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.20","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109919+0000","last_change":"2023-10-16T11:21:57.101954+0000","last_active":"2023-10-16T11:21:58.109919+0000","last_peered":"2023-10-16T11:21:58.109919+0000","last_clean":"2023-10-16T11:21:58.109919+0000","last_became_active":"2023-10-16T11:21:57.101510+0000","last_became_peered":"2023-10-16T11:21:57.101510+0000","last_unstale":"2023-10-16T11:21:58.109919+0000","last_undegraded":"2023-10-16T11:21:58.109919+0000","last_fullsized":"2023-10-16T11:21:58.109919+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:22.980523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600524+0000","last_change":"2023-10-16T11:21:57.101258+0000","last_active":"2023-10-16T11:21:58.600524+0000","last_peered":"2023-10-16T11:21:58.600524+0000","last_clean":"2023-10-16T11:21:58.600524+0000","last_became_active":"2023-10-16T11:21:57.100961+0000","last_became_peered":"2023-10-16T11:21:57.100961+0000","last_unstale":"2023-10-16T11:21:58.600524+0000","last_undegraded":"2023-10-16T11:21:58.600524+0000","last_fullsized":"2023-10-16T11:21:58.600524+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:55.020140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.1f","version":"268'4","reported_seq":16,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601175+0000","last_change":"2023-10-16T11:21:56.085945+0000","last_active":"2023-10-16T11:21:58.601175+0000","last_peered":"2023-10-16T11:21:58.601175+0000","last_clean":"2023-10-16T11:21:58.601175+0000","last_became_active":"2023-10-16T11:21:56.085707+0000","last_became_peered":"2023-10-16T11:21:56.085707+0000","last_unstale":"2023-10-16T11:21:58.601175+0000","last_undegraded":"2023-10-16T11:21:58.601175+0000","last_fullsized":"2023-10-16T11:21:58.601175+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:34:39.723203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601065+0000","last_change":"2023-10-16T11:21:57.122061+0000","last_active":"2023-10-16T11:21:58.601065+0000","last_peered":"2023-10-16T11:21:58.601065+0000","last_clean":"2023-10-16T11:21:58.601065+0000","last_became_active":"2023-10-16T11:21:57.121634+0000","last_became_peered":"2023-10-16T11:21:57.121634+0000","last_unstale":"2023-10-16T11:21:58.601065+0000","last_undegraded":"2023-10-16T11:21:58.601065+0000","last_fullsized":"2023-10-16T11:21:58.601065+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:08.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.1e","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108963+0000","last_change":"2023-10-16T11:21:56.095843+0000","last_active":"2023-10-16T11:21:58.108963+0000","last_peered":"2023-10-16T11:21:58.108963+0000","last_clean":"2023-10-16T11:21:58.108963+0000","last_became_active":"2023-10-16T11:21:56.095553+0000","last_became_peered":"2023-10-16T11:21:56.095553+0000","last_unstale":"2023-10-16T11:21:58.108963+0000","last_undegraded":"2023-10-16T11:21:58.108963+0000","last_fullsized":"2023-10-16T11:21:58.108963+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:50.330309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109116+0000","last_change":"2023-10-16T11:21:57.107916+0000","last_active":"2023-10-16T11:21:58.109116+0000","last_peered":"2023-10-16T11:21:58.109116+0000","last_clean":"2023-10-16T11:21:58.109116+0000","last_became_active":"2023-10-16T11:21:57.107628+0000","last_became_peered":"2023-10-16T11:21:57.107628+0000","last_unstale":"2023-10-16T11:21:58.109116+0000","last_undegraded":"2023-10-16T11:21:58.109116+0000","last_fullsized":"2023-10-16T11:21:58.109116+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:44.353916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1d","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109284+0000","last_change":"2023-10-16T11:21:56.092059+0000","last_active":"2023-10-16T11:21:58.109284+0000","last_peered":"2023-10-16T11:21:58.109284+0000","last_clean":"2023-10-16T11:21:58.109284+0000","last_became_active":"2023-10-16T11:21:56.091479+0000","last_became_peered":"2023-10-16T11:21:56.091479+0000","last_unstale":"2023-10-16T11:21:58.109284+0000","last_undegraded":"2023-10-16T11:21:58.109284+0000","last_fullsized":"2023-10-16T11:21:58.109284+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:00:21.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600260+0000","last_change":"2023-10-16T11:21:57.110071+0000","last_active":"2023-10-16T11:21:58.600260+0000","last_peered":"2023-10-16T11:21:58.600260+0000","last_clean":"2023-10-16T11:21:58.600260+0000","last_became_active":"2023-10-16T11:21:57.109760+0000","last_became_peered":"2023-10-16T11:21:57.109760+0000","last_unstale":"2023-10-16T11:21:58.600260+0000","last_undegraded":"2023-10-16T11:21:58.600260+0000","last_fullsized":"2023-10-16T11:21:58.600260+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:38.376580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1c","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110507+0000","last_change":"2023-10-16T11:21:56.093566+0000","last_active":"2023-10-16T11:21:58.110507+0000","last_peered":"2023-10-16T11:21:58.110507+0000","last_clean":"2023-10-16T11:21:58.110507+0000","last_became_active":"2023-10-16T11:21:56.093272+0000","last_became_peered":"2023-10-16T11:21:56.093272+0000","last_unstale":"2023-10-16T11:21:58.110507+0000","last_undegraded":"2023-10-16T11:21:58.110507+0000","last_fullsized":"2023-10-16T11:21:58.110507+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:43.349367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600283+0000","last_change":"2023-10-16T11:21:57.104361+0000","last_active":"2023-10-16T11:21:58.600283+0000","last_peered":"2023-10-16T11:21:58.600283+0000","last_clean":"2023-10-16T11:21:58.600283+0000","last_became_active":"2023-10-16T11:21:57.104091+0000","last_became_peered":"2023-10-16T11:21:57.104091+0000","last_unstale":"2023-10-16T11:21:58.600283+0000","last_undegraded":"2023-10-16T11:21:58.600283+0000","last_fullsized":"2023-10-16T11:21:58.600283+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:52.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1b","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600901+0000","last_change":"2023-10-16T11:21:56.093241+0000","last_active":"2023-10-16T11:21:58.600901+0000","last_peered":"2023-10-16T11:21:58.600901+0000","last_clean":"2023-10-16T11:21:58.600901+0000","last_became_active":"2023-10-16T11:21:56.092977+0000","last_became_peered":"2023-10-16T11:21:56.092977+0000","last_unstale":"2023-10-16T11:21:58.600901+0000","last_undegraded":"2023-10-16T11:21:58.600901+0000","last_fullsized":"2023-10-16T11:21:58.600901+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:23.976293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599966+0000","last_change":"2023-10-16T11:21:57.106549+0000","last_active":"2023-10-16T11:21:58.599966+0000","last_peered":"2023-10-16T11:21:58.599966+0000","last_clean":"2023-10-16T11:21:58.599966+0000","last_became_active":"2023-10-16T11:21:57.106260+0000","last_became_peered":"2023-10-16T11:21:57.106260+0000","last_unstale":"2023-10-16T11:21:58.599966+0000","last_undegraded":"2023-10-16T11:21:58.599966+0000","last_fullsized":"2023-10-16T11:21:58.599966+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:55.318964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1a","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600648+0000","last_change":"2023-10-16T11:21:56.094876+0000","last_active":"2023-10-16T11:21:58.600648+0000","last_peered":"2023-10-16T11:21:58.600648+0000","last_clean":"2023-10-16T11:21:58.600648+0000","last_became_active":"2023-10-16T11:21:56.094625+0000","last_became_peered":"2023-10-16T11:21:56.094625+0000","last_unstale":"2023-10-16T11:21:58.600648+0000","last_undegraded":"2023-10-16T11:21:58.600648+0000","last_fullsized":"2023-10-16T11:21:58.600648+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:36.857225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600173+0000","last_change":"2023-10-16T11:21:57.110831+0000","last_active":"2023-10-16T11:21:58.600173+0000","last_peered":"2023-10-16T11:21:58.600173+0000","last_clean":"2023-10-16T11:21:58.600173+0000","last_became_active":"2023-10-16T11:21:57.110541+0000","last_became_peered":"2023-10-16T11:21:57.110541+0000","last_unstale":"2023-10-16T11:21:58.600173+0000","last_undegraded":"2023-10-16T11:21:58.600173+0000","last_fullsized":"2023-10-16T11:21:58.600173+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:22.081940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600511+0000","last_change":"2023-10-16T11:21:56.089461+0000","last_active":"2023-10-16T11:21:58.600511+0000","last_peered":"2023-10-16T11:21:58.600511+0000","last_clean":"2023-10-16T11:21:58.600511+0000","last_became_active":"2023-10-16T11:21:56.089171+0000","last_became_peered":"2023-10-16T11:21:56.089171+0000","last_unstale":"2023-10-16T11:21:58.600511+0000","last_undegraded":"2023-10-16T11:21:58.600511+0000","last_fullsized":"2023-10-16T11:21:58.600511+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:40.317963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108930+0000","last_change":"2023-10-16T11:21:57.101482+0000","last_active":"2023-10-16T11:21:58.108930+0000","last_peered":"2023-10-16T11:21:58.108930+0000","last_clean":"2023-10-16T11:21:58.108930+0000","last_became_active":"2023-10-16T11:21:57.101135+0000","last_became_peered":"2023-10-16T11:21:57.101135+0000","last_unstale":"2023-10-16T11:21:58.108930+0000","last_undegraded":"2023-10-16T11:21:58.108930+0000","last_fullsized":"2023-10-16T11:21:58.108930+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:45.296959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108649+0000","last_change":"2023-10-16T11:21:56.093686+0000","last_active":"2023-10-16T11:21:58.108649+0000","last_peered":"2023-10-16T11:21:58.108649+0000","last_clean":"2023-10-16T11:21:58.108649+0000","last_became_active":"2023-10-16T11:21:56.093424+0000","last_became_peered":"2023-10-16T11:21:56.093424+0000","last_unstale":"2023-10-16T11:21:58.108649+0000","last_undegraded":"2023-10-16T11:21:58.108649+0000","last_fullsized":"2023-10-16T11:21:58.108649+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:43.501000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108972+0000","last_change":"2023-10-16T11:21:57.101779+0000","last_active":"2023-10-16T11:21:58.108972+0000","last_peered":"2023-10-16T11:21:58.108972+0000","last_clean":"2023-10-16T11:21:58.108972+0000","last_became_active":"2023-10-16T11:21:57.101506+0000","last_became_peered":"2023-10-16T11:21:57.101506+0000","last_unstale":"2023-10-16T11:21:58.108972+0000","last_undegraded":"2023-10-16T11:21:58.108972+0000","last_fullsized":"2023-10-16T11:21:58.108972+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:00.602787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601266+0000","last_change":"2023-10-16T11:21:56.095906+0000","last_active":"2023-10-16T11:21:58.601266+0000","last_peered":"2023-10-16T11:21:58.601266+0000","last_clean":"2023-10-16T11:21:58.601266+0000","last_became_active":"2023-10-16T11:21:56.095582+0000","last_became_peered":"2023-10-16T11:21:56.095582+0000","last_unstale":"2023-10-16T11:21:58.601266+0000","last_undegraded":"2023-10-16T11:21:58.601266+0000","last_fullsized":"2023-10-16T11:21:58.601266+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:28.440079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601075+0000","last_change":"2023-10-16T11:21:57.109634+0000","last_active":"2023-10-16T11:21:58.601075+0000","last_peered":"2023-10-16T11:21:58.601075+0000","last_clean":"2023-10-16T11:21:58.601075+0000","last_became_active":"2023-10-16T11:21:57.108191+0000","last_became_peered":"2023-10-16T11:21:57.108191+0000","last_unstale":"2023-10-16T11:21:58.601075+0000","last_undegraded":"2023-10-16T11:21:58.601075+0000","last_fullsized":"2023-10-16T11:21:58.601075+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:01.750572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600479+0000","last_change":"2023-10-16T11:21:56.086243+0000","last_active":"2023-10-16T11:21:58.600479+0000","last_peered":"2023-10-16T11:21:58.600479+0000","last_clean":"2023-10-16T11:21:58.600479+0000","last_became_active":"2023-10-16T11:21:56.085981+0000","last_became_peered":"2023-10-16T11:21:56.085981+0000","last_unstale":"2023-10-16T11:21:58.600479+0000","last_undegraded":"2023-10-16T11:21:58.600479+0000","last_fullsized":"2023-10-16T11:21:58.600479+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:11:29.707155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.14","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110446+0000","last_change":"2023-10-16T11:21:57.111164+0000","last_active":"2023-10-16T11:21:58.110446+0000","last_peered":"2023-10-16T11:21:58.110446+0000","last_clean":"2023-10-16T11:21:58.110446+0000","last_became_active":"2023-10-16T11:21:57.110893+0000","last_became_peered":"2023-10-16T11:21:57.110893+0000","last_unstale":"2023-10-16T11:21:58.110446+0000","last_undegraded":"2023-10-16T11:21:58.110446+0000","last_fullsized":"2023-10-16T11:21:58.110446+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:37.861774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.15","version":"268'3","reported_seq":15,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110112+0000","last_change":"2023-10-16T11:21:56.097616+0000","last_active":"2023-10-16T11:21:58.110112+0000","last_peered":"2023-10-16T11:21:58.110112+0000","last_clean":"2023-10-16T11:21:58.110112+0000","last_became_active":"2023-10-16T11:21:56.096477+0000","last_became_peered":"2023-10-16T11:21:56.096477+0000","last_unstale":"2023-10-16T11:21:58.110112+0000","last_undegraded":"2023-10-16T11:21:58.110112+0000","last_fullsized":"2023-10-16T11:21:58.110112+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:32.648961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.15","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109238+0000","last_change":"2023-10-16T11:21:57.109495+0000","last_active":"2023-10-16T11:21:58.109238+0000","last_peered":"2023-10-16T11:21:58.109238+0000","last_clean":"2023-10-16T11:21:58.109238+0000","last_became_active":"2023-10-16T11:21:57.109223+0000","last_became_peered":"2023-10-16T11:21:57.109223+0000","last_unstale":"2023-10-16T11:21:58.109238+0000","last_undegraded":"2023-10-16T11:21:58.109238+0000","last_fullsized":"2023-10-16T11:21:58.109238+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:39.699223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.14","version":"268'6","reported_seq":18,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600292+0000","last_change":"2023-10-16T11:21:56.086852+0000","last_active":"2023-10-16T11:21:58.600292+0000","last_peered":"2023-10-16T11:21:58.600292+0000","last_clean":"2023-10-16T11:21:58.600292+0000","last_became_active":"2023-10-16T11:21:56.085203+0000","last_became_peered":"2023-10-16T11:21:56.085203+0000","last_unstale":"2023-10-16T11:21:58.600292+0000","last_undegraded":"2023-10-16T11:21:58.600292+0000","last_fullsized":"2023-10-16T11:21:58.600292+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:32.895099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600451+0000","last_change":"2023-10-16T11:21:57.111512+0000","last_active":"2023-10-16T11:21:58.600451+0000","last_peered":"2023-10-16T11:21:58.600451+0000","last_clean":"2023-10-16T11:21:58.600451+0000","last_became_active":"2023-10-16T11:21:57.111192+0000","last_became_peered":"2023-10-16T11:21:57.111192+0000","last_unstale":"2023-10-16T11:21:58.600451+0000","last_undegraded":"2023-10-16T11:21:58.600451+0000","last_fullsized":"2023-10-16T11:21:58.600451+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:00.833411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600110+0000","last_change":"2023-10-16T11:21:56.094709+0000","last_active":"2023-10-16T11:21:58.600110+0000","last_peered":"2023-10-16T11:21:58.600110+0000","last_clean":"2023-10-16T11:21:58.600110+0000","last_became_active":"2023-10-16T11:21:56.094426+0000","last_became_peered":"2023-10-16T11:21:56.094426+0000","last_unstale":"2023-10-16T11:21:58.600110+0000","last_undegraded":"2023-10-16T11:21:58.600110+0000","last_fullsized":"2023-10-16T11:21:58.600110+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:08.759971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.13","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110070+0000","last_change":"2023-10-16T11:21:57.103895+0000","last_active":"2023-10-16T11:21:58.110070+0000","last_peered":"2023-10-16T11:21:58.110070+0000","last_clean":"2023-10-16T11:21:58.110070+0000","last_became_active":"2023-10-16T11:21:57.103586+0000","last_became_peered":"2023-10-16T11:21:57.103586+0000","last_unstale":"2023-10-16T11:21:58.110070+0000","last_undegraded":"2023-10-16T11:21:58.110070+0000","last_fullsized":"2023-10-16T11:21:58.110070+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:38.000095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109960+0000","last_change":"2023-10-16T11:21:56.095924+0000","last_active":"2023-10-16T11:21:58.109960+0000","last_peered":"2023-10-16T11:21:58.109960+0000","last_clean":"2023-10-16T11:21:58.109960+0000","last_became_active":"2023-10-16T11:21:56.095389+0000","last_became_peered":"2023-10-16T11:21:56.095389+0000","last_unstale":"2023-10-16T11:21:58.109960+0000","last_undegraded":"2023-10-16T11:21:58.109960+0000","last_fullsized":"2023-10-16T11:21:58.109960+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:20.624093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.10","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600201+0000","last_change":"2023-10-16T11:21:57.103683+0000","last_active":"2023-10-16T11:21:58.600201+0000","last_peered":"2023-10-16T11:21:58.600201+0000","last_clean":"2023-10-16T11:21:58.600201+0000","last_became_active":"2023-10-16T11:21:57.103349+0000","last_became_peered":"2023-10-16T11:21:57.103349+0000","last_unstale":"2023-10-16T11:21:58.600201+0000","last_undegraded":"2023-10-16T11:21:58.600201+0000","last_fullsized":"2023-10-16T11:21:58.600201+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:48.516759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601371+0000","last_change":"2023-10-16T11:21:56.089325+0000","last_active":"2023-10-16T11:21:58.601371+0000","last_peered":"2023-10-16T11:21:58.601371+0000","last_clean":"2023-10-16T11:21:58.601371+0000","last_became_active":"2023-10-16T11:21:56.088953+0000","last_became_peered":"2023-10-16T11:21:56.088953+0000","last_unstale":"2023-10-16T11:21:58.601371+0000","last_undegraded":"2023-10-16T11:21:58.601371+0000","last_fullsized":"2023-10-16T11:21:58.601371+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:06.807286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110122+0000","last_change":"2023-10-16T11:21:57.108137+0000","last_active":"2023-10-16T11:21:58.110122+0000","last_peered":"2023-10-16T11:21:58.110122+0000","last_clean":"2023-10-16T11:21:58.110122+0000","last_became_active":"2023-10-16T11:21:57.107836+0000","last_became_peered":"2023-10-16T11:21:57.107836+0000","last_unstale":"2023-10-16T11:21:58.110122+0000","last_undegraded":"2023-10-16T11:21:58.110122+0000","last_fullsized":"2023-10-16T11:21:58.110122+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:20:30.010063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.10","version":"268'8","reported_seq":20,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110397+0000","last_change":"2023-10-16T11:21:56.097420+0000","last_active":"2023-10-16T11:21:58.110397+0000","last_peered":"2023-10-16T11:21:58.110397+0000","last_clean":"2023-10-16T11:21:58.110397+0000","last_became_active":"2023-10-16T11:21:56.095598+0000","last_became_peered":"2023-10-16T11:21:56.095598+0000","last_unstale":"2023-10-16T11:21:58.110397+0000","last_undegraded":"2023-10-16T11:21:58.110397+0000","last_fullsized":"2023-10-16T11:21:58.110397+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:59.598238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110380+0000","last_change":"2023-10-16T11:21:57.118137+0000","last_active":"2023-10-16T11:21:58.110380+0000","last_peered":"2023-10-16T11:21:58.110380+0000","last_clean":"2023-10-16T11:21:58.110380+0000","last_became_active":"2023-10-16T11:21:57.117670+0000","last_became_peered":"2023-10-16T11:21:57.117670+0000","last_unstale":"2023-10-16T11:21:58.110380+0000","last_undegraded":"2023-10-16T11:21:58.110380+0000","last_fullsized":"2023-10-16T11:21:58.110380+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:33.899648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.f","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599906+0000","last_change":"2023-10-16T11:21:56.091998+0000","last_active":"2023-10-16T11:21:58.599906+0000","last_peered":"2023-10-16T11:21:58.599906+0000","last_clean":"2023-10-16T11:21:58.599906+0000","last_became_active":"2023-10-16T11:21:56.091723+0000","last_became_peered":"2023-10-16T11:21:56.091723+0000","last_unstale":"2023-10-16T11:21:58.599906+0000","last_undegraded":"2023-10-16T11:21:58.599906+0000","last_fullsized":"2023-10-16T11:21:58.599906+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:44.778516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601397+0000","last_change":"2023-10-16T11:21:57.117970+0000","last_active":"2023-10-16T11:21:58.601397+0000","last_peered":"2023-10-16T11:21:58.601397+0000","last_clean":"2023-10-16T11:21:58.601397+0000","last_became_active":"2023-10-16T11:21:57.117695+0000","last_became_peered":"2023-10-16T11:21:57.117695+0000","last_unstale":"2023-10-16T11:21:58.601397+0000","last_undegraded":"2023-10-16T11:21:58.601397+0000","last_fullsized":"2023-10-16T11:21:58.601397+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:13.461214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.e","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600578+0000","last_change":"2023-10-16T11:21:56.085894+0000","last_active":"2023-10-16T11:21:58.600578+0000","last_peered":"2023-10-16T11:21:58.600578+0000","last_clean":"2023-10-16T11:21:58.600578+0000","last_became_active":"2023-10-16T11:21:56.085641+0000","last_became_peered":"2023-10-16T11:21:56.085641+0000","last_unstale":"2023-10-16T11:21:58.600578+0000","last_undegraded":"2023-10-16T11:21:58.600578+0000","last_fullsized":"2023-10-16T11:21:58.600578+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:58.532208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109047+0000","last_change":"2023-10-16T11:21:57.104245+0000","last_active":"2023-10-16T11:21:58.109047+0000","last_peered":"2023-10-16T11:21:58.109047+0000","last_clean":"2023-10-16T11:21:58.109047+0000","last_became_active":"2023-10-16T11:21:57.103988+0000","last_became_peered":"2023-10-16T11:21:57.103988+0000","last_unstale":"2023-10-16T11:21:58.109047+0000","last_undegraded":"2023-10-16T11:21:58.109047+0000","last_fullsized":"2023-10-16T11:21:58.109047+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:19.254965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.d","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109352+0000","last_change":"2023-10-16T11:21:56.092253+0000","last_active":"2023-10-16T11:21:58.109352+0000","last_peered":"2023-10-16T11:21:58.109352+0000","last_clean":"2023-10-16T11:21:58.109352+0000","last_became_active":"2023-10-16T11:21:56.091785+0000","last_became_peered":"2023-10-16T11:21:56.091785+0000","last_unstale":"2023-10-16T11:21:58.109352+0000","last_undegraded":"2023-10-16T11:21:58.109352+0000","last_fullsized":"2023-10-16T11:21:58.109352+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:54.277523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600910+0000","last_change":"2023-10-16T11:21:57.108911+0000","last_active":"2023-10-16T11:21:58.600910+0000","last_peered":"2023-10-16T11:21:58.600910+0000","last_clean":"2023-10-16T11:21:58.600910+0000","last_became_active":"2023-10-16T11:21:57.108115+0000","last_became_peered":"2023-10-16T11:21:57.108115+0000","last_unstale":"2023-10-16T11:21:58.600910+0000","last_undegraded":"2023-10-16T11:21:58.600910+0000","last_fullsized":"2023-10-16T11:21:58.600910+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:12:09.288671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109159+0000","last_change":"2023-10-16T11:21:56.091047+0000","last_active":"2023-10-16T11:21:58.109159+0000","last_peered":"2023-10-16T11:21:58.109159+0000","last_clean":"2023-10-16T11:21:58.109159+0000","last_became_active":"2023-10-16T11:21:56.090754+0000","last_became_peered":"2023-10-16T11:21:56.090754+0000","last_unstale":"2023-10-16T11:21:58.109159+0000","last_undegraded":"2023-10-16T11:21:58.109159+0000","last_fullsized":"2023-10-16T11:21:58.109159+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:48.105971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600259+0000","last_change":"2023-10-16T11:21:57.106835+0000","last_active":"2023-10-16T11:21:58.600259+0000","last_peered":"2023-10-16T11:21:58.600259+0000","last_clean":"2023-10-16T11:21:58.600259+0000","last_became_active":"2023-10-16T11:21:57.106502+0000","last_became_peered":"2023-10-16T11:21:57.106502+0000","last_unstale":"2023-10-16T11:21:58.600259+0000","last_undegraded":"2023-10-16T11:21:58.600259+0000","last_fullsized":"2023-10-16T11:21:58.600259+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:53:31.935527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.b","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110343+0000","last_change":"2023-10-16T11:21:56.095583+0000","last_active":"2023-10-16T11:21:58.110343+0000","last_peered":"2023-10-16T11:21:58.110343+0000","last_clean":"2023-10-16T11:21:58.110343+0000","last_became_active":"2023-10-16T11:21:56.095041+0000","last_became_peered":"2023-10-16T11:21:56.095041+0000","last_unstale":"2023-10-16T11:21:58.110343+0000","last_undegraded":"2023-10-16T11:21:58.110343+0000","last_fullsized":"2023-10-16T11:21:58.110343+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:38.694674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600106+0000","last_change":"2023-10-16T11:21:57.109272+0000","last_active":"2023-10-16T11:21:58.600106+0000","last_peered":"2023-10-16T11:21:58.600106+0000","last_clean":"2023-10-16T11:21:58.600106+0000","last_became_active":"2023-10-16T11:21:57.109013+0000","last_became_peered":"2023-10-16T11:21:57.109013+0000","last_unstale":"2023-10-16T11:21:58.600106+0000","last_undegraded":"2023-10-16T11:21:58.600106+0000","last_fullsized":"2023-10-16T11:21:58.600106+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:15.786493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600201+0000","last_change":"2023-10-16T11:21:56.086890+0000","last_active":"2023-10-16T11:21:58.600201+0000","last_peered":"2023-10-16T11:21:58.600201+0000","last_clean":"2023-10-16T11:21:58.600201+0000","last_became_active":"2023-10-16T11:21:56.086617+0000","last_became_peered":"2023-10-16T11:21:56.086617+0000","last_unstale":"2023-10-16T11:21:58.600201+0000","last_undegraded":"2023-10-16T11:21:58.600201+0000","last_fullsized":"2023-10-16T11:21:58.600201+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:30.665567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600195+0000","last_change":"2023-10-16T11:21:57.102277+0000","last_active":"2023-10-16T11:21:58.600195+0000","last_peered":"2023-10-16T11:21:58.600195+0000","last_clean":"2023-10-16T11:21:58.600195+0000","last_became_active":"2023-10-16T11:21:57.101966+0000","last_became_peered":"2023-10-16T11:21:57.101966+0000","last_unstale":"2023-10-16T11:21:58.600195+0000","last_undegraded":"2023-10-16T11:21:58.600195+0000","last_fullsized":"2023-10-16T11:21:58.600195+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:06.126659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108993+0000","last_change":"2023-10-16T11:21:56.091691+0000","last_active":"2023-10-16T11:21:58.108993+0000","last_peered":"2023-10-16T11:21:58.108993+0000","last_clean":"2023-10-16T11:21:58.108993+0000","last_became_active":"2023-10-16T11:21:56.091414+0000","last_became_peered":"2023-10-16T11:21:56.091414+0000","last_unstale":"2023-10-16T11:21:58.108993+0000","last_undegraded":"2023-10-16T11:21:58.108993+0000","last_fullsized":"2023-10-16T11:21:58.108993+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:49.455121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600027+0000","last_change":"2023-10-16T11:21:57.116051+0000","last_active":"2023-10-16T11:21:58.600027+0000","last_peered":"2023-10-16T11:21:58.600027+0000","last_clean":"2023-10-16T11:21:58.600027+0000","last_became_active":"2023-10-16T11:21:57.115776+0000","last_became_peered":"2023-10-16T11:21:57.115776+0000","last_unstale":"2023-10-16T11:21:58.600027+0000","last_undegraded":"2023-10-16T11:21:58.600027+0000","last_fullsized":"2023-10-16T11:21:58.600027+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:39.472479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600250+0000","last_change":"2023-10-16T11:21:56.087100+0000","last_active":"2023-10-16T11:21:58.600250+0000","last_peered":"2023-10-16T11:21:58.600250+0000","last_clean":"2023-10-16T11:21:58.600250+0000","last_became_active":"2023-10-16T11:21:56.086599+0000","last_became_peered":"2023-10-16T11:21:56.086599+0000","last_unstale":"2023-10-16T11:21:58.600250+0000","last_undegraded":"2023-10-16T11:21:58.600250+0000","last_fullsized":"2023-10-16T11:21:58.600250+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:47.512210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.6","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600289+0000","last_change":"2023-10-16T11:21:57.103368+0000","last_active":"2023-10-16T11:21:58.600289+0000","last_peered":"2023-10-16T11:21:58.600289+0000","last_clean":"2023-10-16T11:21:58.600289+0000","last_became_active":"2023-10-16T11:21:57.102680+0000","last_became_peered":"2023-10-16T11:21:57.102680+0000","last_unstale":"2023-10-16T11:21:58.600289+0000","last_undegraded":"2023-10-16T11:21:58.600289+0000","last_fullsized":"2023-10-16T11:21:58.600289+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:58.049533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.7","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110210+0000","last_change":"2023-10-16T11:21:56.100889+0000","last_active":"2023-10-16T11:21:58.110210+0000","last_peered":"2023-10-16T11:21:58.110210+0000","last_clean":"2023-10-16T11:21:58.110210+0000","last_became_active":"2023-10-16T11:21:56.100600+0000","last_became_peered":"2023-10-16T11:21:56.100600+0000","last_unstale":"2023-10-16T11:21:58.110210+0000","last_undegraded":"2023-10-16T11:21:58.110210+0000","last_fullsized":"2023-10-16T11:21:58.110210+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:59.315630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.7","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601092+0000","last_change":"2023-10-16T11:21:57.117900+0000","last_active":"2023-10-16T11:21:58.601092+0000","last_peered":"2023-10-16T11:21:58.601092+0000","last_clean":"2023-10-16T11:21:58.601092+0000","last_became_active":"2023-10-16T11:21:57.117480+0000","last_became_peered":"2023-10-16T11:21:57.117480+0000","last_unstale":"2023-10-16T11:21:58.601092+0000","last_undegraded":"2023-10-16T11:21:58.601092+0000","last_fullsized":"2023-10-16T11:21:58.601092+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:07.609635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600216+0000","last_change":"2023-10-16T11:21:56.089475+0000","last_active":"2023-10-16T11:21:58.600216+0000","last_peered":"2023-10-16T11:21:58.600216+0000","last_clean":"2023-10-16T11:21:58.600216+0000","last_became_active":"2023-10-16T11:21:56.089135+0000","last_became_peered":"2023-10-16T11:21:56.089135+0000","last_unstale":"2023-10-16T11:21:58.600216+0000","last_undegraded":"2023-10-16T11:21:58.600216+0000","last_fullsized":"2023-10-16T11:21:58.600216+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:23:48.366188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600423+0000","last_change":"2023-10-16T11:21:57.106821+0000","last_active":"2023-10-16T11:21:58.600423+0000","last_peered":"2023-10-16T11:21:58.600423+0000","last_clean":"2023-10-16T11:21:58.600423+0000","last_became_active":"2023-10-16T11:21:57.106547+0000","last_became_peered":"2023-10-16T11:21:57.106547+0000","last_unstale":"2023-10-16T11:21:58.600423+0000","last_undegraded":"2023-10-16T11:21:58.600423+0000","last_fullsized":"2023-10-16T11:21:58.600423+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:24.086344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110207+0000","last_change":"2023-10-16T11:21:56.093546+0000","last_active":"2023-10-16T11:21:58.110207+0000","last_peered":"2023-10-16T11:21:58.110207+0000","last_clean":"2023-10-16T11:21:58.110207+0000","last_became_active":"2023-10-16T11:21:56.093249+0000","last_became_peered":"2023-10-16T11:21:56.093249+0000","last_unstale":"2023-10-16T11:21:58.110207+0000","last_undegraded":"2023-10-16T11:21:58.110207+0000","last_fullsized":"2023-10-16T11:21:58.110207+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:18.250416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110178+0000","last_change":"2023-10-16T11:21:57.102695+0000","last_active":"2023-10-16T11:21:58.110178+0000","last_peered":"2023-10-16T11:21:58.110178+0000","last_clean":"2023-10-16T11:21:58.110178+0000","last_became_active":"2023-10-16T11:21:57.102419+0000","last_became_peered":"2023-10-16T11:21:57.102419+0000","last_unstale":"2023-10-16T11:21:58.110178+0000","last_undegraded":"2023-10-16T11:21:58.110178+0000","last_fullsized":"2023-10-16T11:21:58.110178+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:50.894692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600849+0000","last_change":"2023-10-16T11:21:56.088856+0000","last_active":"2023-10-16T11:21:58.600849+0000","last_peered":"2023-10-16T11:21:58.600849+0000","last_clean":"2023-10-16T11:21:58.600849+0000","last_became_active":"2023-10-16T11:21:56.088495+0000","last_became_peered":"2023-10-16T11:21:56.088495+0000","last_unstale":"2023-10-16T11:21:58.600849+0000","last_undegraded":"2023-10-16T11:21:58.600849+0000","last_fullsized":"2023-10-16T11:21:58.600849+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:28.835213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108977+0000","last_change":"2023-10-16T11:21:57.100999+0000","last_active":"2023-10-16T11:21:58.108977+0000","last_peered":"2023-10-16T11:21:58.108977+0000","last_clean":"2023-10-16T11:21:58.108977+0000","last_became_active":"2023-10-16T11:21:57.100031+0000","last_became_peered":"2023-10-16T11:21:57.100031+0000","last_unstale":"2023-10-16T11:21:58.108977+0000","last_undegraded":"2023-10-16T11:21:58.108977+0000","last_fullsized":"2023-10-16T11:21:58.108977+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:00.320179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601005+0000","last_change":"2023-10-16T11:21:56.085086+0000","last_active":"2023-10-16T11:21:58.601005+0000","last_peered":"2023-10-16T11:21:58.601005+0000","last_clean":"2023-10-16T11:21:58.601005+0000","last_became_active":"2023-10-16T11:21:56.084749+0000","last_became_peered":"2023-10-16T11:21:56.084749+0000","last_unstale":"2023-10-16T11:21:58.601005+0000","last_undegraded":"2023-10-16T11:21:58.601005+0000","last_fullsized":"2023-10-16T11:21:58.601005+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:27.596861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110237+0000","last_change":"2023-10-16T11:21:57.108386+0000","last_active":"2023-10-16T11:21:58.110237+0000","last_peered":"2023-10-16T11:21:58.110237+0000","last_clean":"2023-10-16T11:21:58.110237+0000","last_became_active":"2023-10-16T11:21:57.106808+0000","last_became_peered":"2023-10-16T11:21:57.106808+0000","last_unstale":"2023-10-16T11:21:58.110237+0000","last_undegraded":"2023-10-16T11:21:58.110237+0000","last_fullsized":"2023-10-16T11:21:58.110237+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:38.635487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110015+0000","last_change":"2023-10-16T11:21:56.097915+0000","last_active":"2023-10-16T11:21:58.110015+0000","last_peered":"2023-10-16T11:21:58.110015+0000","last_clean":"2023-10-16T11:21:58.110015+0000","last_became_active":"2023-10-16T11:21:56.097197+0000","last_became_peered":"2023-10-16T11:21:56.097197+0000","last_unstale":"2023-10-16T11:21:58.110015+0000","last_undegraded":"2023-10-16T11:21:58.110015+0000","last_fullsized":"2023-10-16T11:21:58.110015+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:50.207208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600331+0000","last_change":"2023-10-16T11:21:57.106260+0000","last_active":"2023-10-16T11:21:58.600331+0000","last_peered":"2023-10-16T11:21:58.600331+0000","last_clean":"2023-10-16T11:21:58.600331+0000","last_became_active":"2023-10-16T11:21:57.105987+0000","last_became_peered":"2023-10-16T11:21:57.105987+0000","last_unstale":"2023-10-16T11:21:58.600331+0000","last_undegraded":"2023-10-16T11:21:58.600331+0000","last_fullsized":"2023-10-16T11:21:58.600331+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:56.298539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600078+0000","last_change":"2023-10-16T11:21:56.084734+0000","last_active":"2023-10-16T11:21:58.600078+0000","last_peered":"2023-10-16T11:21:58.600078+0000","last_clean":"2023-10-16T11:21:58.600078+0000","last_became_active":"2023-10-16T11:21:56.084451+0000","last_became_peered":"2023-10-16T11:21:56.084451+0000","last_unstale":"2023-10-16T11:21:58.600078+0000","last_undegraded":"2023-10-16T11:21:58.600078+0000","last_fullsized":"2023-10-16T11:21:58.600078+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:54.257628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110356+0000","last_change":"2023-10-16T11:21:57.108052+0000","last_active":"2023-10-16T11:21:58.110356+0000","last_peered":"2023-10-16T11:21:58.110356+0000","last_clean":"2023-10-16T11:21:58.110356+0000","last_became_active":"2023-10-16T11:21:57.107734+0000","last_became_peered":"2023-10-16T11:21:57.107734+0000","last_unstale":"2023-10-16T11:21:58.110356+0000","last_undegraded":"2023-10-16T11:21:58.110356+0000","last_fullsized":"2023-10-16T11:21:58.110356+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:21.731838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110532+0000","last_change":"2023-10-16T11:21:56.096004+0000","last_active":"2023-10-16T11:21:58.110532+0000","last_peered":"2023-10-16T11:21:58.110532+0000","last_clean":"2023-10-16T11:21:58.110532+0000","last_became_active":"2023-10-16T11:21:56.095163+0000","last_became_peered":"2023-10-16T11:21:56.095163+0000","last_unstale":"2023-10-16T11:21:58.110532+0000","last_undegraded":"2023-10-16T11:21:58.110532+0000","last_fullsized":"2023-10-16T11:21:58.110532+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:16.267840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.24","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110397+0000","last_change":"2023-10-16T11:21:57.101165+0000","last_active":"2023-10-16T11:21:58.110397+0000","last_peered":"2023-10-16T11:21:58.110397+0000","last_clean":"2023-10-16T11:21:58.110397+0000","last_became_active":"2023-10-16T11:21:57.100790+0000","last_became_peered":"2023-10-16T11:21:57.100790+0000","last_unstale":"2023-10-16T11:21:58.110397+0000","last_undegraded":"2023-10-16T11:21:58.110397+0000","last_fullsized":"2023-10-16T11:21:58.110397+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:07.811835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"259'403","reported_seq":1455,"reported_epoch":268,"state":"active+clean","last_fresh":"2023-10-16T11:21:57.095189+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:21:57.095189+0000","last_peered":"2023-10-16T11:21:57.095189+0000","last_clean":"2023-10-16T11:21:57.095189+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:21:57.095189+0000","last_undegraded":"2023-10-16T11:21:57.095189+0000","last_fullsized":"2023-10-16T11:21:57.095189+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":679,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":37,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":36,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":679,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":403,"ondisk_log_size":403,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57280,"kb_used_data":3888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314560,"statfs":{"total":96636764160,"available":96578109440,"internally_reserved":0,"allocated":3981312,"data_stored":2350778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.878,"15min":0.878},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27800000000000002},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.5}]},{"osd":2,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55100000000000005,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.31900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69799999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61599999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":1,"15min":1.002},"last":0.67600000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68400000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60399999999999998,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.55200000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84299999999999997,"5min":1.052,"15min":1.052},"last":0.69799999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65400000000000003}]}]},{"osd":6,"up_from":14,"seq":60129543304,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60868,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310972,"statfs":{"total":96636764160,"available":96574435328,"internally_reserved":0,"allocated":3399680,"data_stored":1762022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54500000000000004,"15min":0.58199999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.84499999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.622}]},{"osd":1,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.63400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.436},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57699999999999996,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.68500000000000005,"5min":0.77200000000000002,"15min":0.86499999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.57299999999999995,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.66300000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78800000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28100000000000003},"max":{"1min":0.747,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.496,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.73799999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59799999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.81399999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.746},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68999999999999995}]}]},{"osd":1,"up_from":14,"seq":60129543304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60876,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310964,"statfs":{"total":96636764160,"available":96574427136,"internally_reserved":0,"allocated":3407872,"data_stored":1767318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.39700000000000002},"min":{"1min":0.23499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.377,"15min":0.41799999999999998},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.44400000000000001,"5min":0.62,"15min":0.66900000000000004},"last":0.48699999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45200000000000001,"5min":0.56999999999999995,"15min":0.61799999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.40799999999999997,"15min":0.44900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.497,"5min":0.69699999999999995,"15min":0.755},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67300000000000004,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.52800000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":0.92600000000000005,"15min":1.097},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76600000000000001,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":1.147},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.0509999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.0600000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.68200000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543305,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61452,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310388,"statfs":{"total":96636764160,"available":96573837312,"internally_reserved":0,"allocated":3989504,"data_stored":2350870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36899999999999999,"15min":0.38200000000000001},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.214},"max":{"1min":0.46200000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38200000000000001,"15min":0.40699999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.314}]},{"osd":2,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39700000000000002,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.67300000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.439},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55200000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41699999999999998,"15min":0.443},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47199999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84399999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.998},"last":0.70799999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":1.008},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.627,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":1.034},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.65400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63300000000000001,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.95999999999999996,"15min":1.0449999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.67600000000000005},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65400000000000003}]}]},{"osd":3,"up_from":14,"seq":60129543305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48384,"kb_used_data":3312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323456,"statfs":{"total":96636764160,"available":96587218944,"internally_reserved":0,"allocated":3391488,"data_stored":1760992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40200000000000002,"15min":0.41099999999999998},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.52600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.372},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.41799999999999998,"15min":0.44400000000000001},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.40899999999999997,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.38800000000000001,"15min":0.41499999999999998},"min":{"1min":0.24299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.42899999999999999,"5min":0.87,"15min":0.87},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40699999999999997,"15min":0.441},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.41199999999999998,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.29699999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.42899999999999999,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.48099999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.44800000000000001,"15min":0.47499999999999998},"min":{"1min":0.33600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.39900000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.73699999999999999,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71299999999999997,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.626,"15min":0.66200000000000003},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63300000000000001,"15min":0.66400000000000003},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.53500000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.623,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.377,"15min":0.377},"max":{"1min":0.73499999999999999,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63700000000000001,"15min":0.67700000000000005},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.52400000000000002}]}]},{"osd":2,"up_from":14,"seq":60129543304,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48392,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323448,"statfs":{"total":96636764160,"available":96587210752,"internally_reserved":0,"allocated":3395584,"data_stored":1760534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.433,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.57299999999999995,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.504}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.436,"15min":0.441},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.441,"15min":0.45800000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.44800000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.163,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.434},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69899999999999995,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":3.593,"15min":3.593},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.64700000000000002,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73499999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67700000000000005,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67500000000000004,"15min":0.68999999999999995},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.73599999999999999,"5min":3.633,"15min":3.633},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.67200000000000004,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.70799999999999996,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66300000000000003,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.70699999999999996,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.61199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543304,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96900,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94274940,"statfs":{"total":96636764160,"available":96537538560,"internally_reserved":0,"allocated":3395584,"data_stored":1761062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.62,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":0.84399999999999997},"last":0.79000000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58699999999999997,"15min":0.61299999999999999},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68300000000000005,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66000000000000003,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.68799999999999994,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.57699999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.65200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.64200000000000002},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.70099999999999996,"5min":0.81299999999999994,"15min":0.879},"last":0.47799999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.72399999999999998,"5min":0.81299999999999994,"15min":0.84099999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.29399999999999998},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.74099999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.875,"15min":0.875},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.69999999999999996}]}]},{"osd":5,"up_from":14,"seq":60129543305,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48412,"kb_used_data":3336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323428,"statfs":{"total":96636764160,"available":96587190272,"internally_reserved":0,"allocated":3416064,"data_stored":1768964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":1.873,"15min":1.873},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.024,"15min":1.024},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88700000000000001,"5min":1.821,"15min":1.821},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63600000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.62},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.54300000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.878,"15min":0.878},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72099999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80000000000000004,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.748},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86099999999999999,"5min":2.02,"15min":2.02},"last":0.70599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:01.221 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:01.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.373+0000 7f0e1ec68700 1 Processor -- start 2023-10-16T11:22:01.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.374+0000 7f0e1ec68700 1 -- start start 2023-10-16T11:22:01.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1ec68700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e1815dc20 0x7f0e180d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1ec68700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e180d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1ec68700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e180b2f00 con 0x7f0e180ce4b0 2023-10-16T11:22:01.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1ec68700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e180b3b80 con 0x7f0e1815dc20 2023-10-16T11:22:01.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1ec68700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180d2820 con 0x7f0e180d1e60 2023-10-16T11:22:01.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e17fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e180d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1d205700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0e180ce4b0 0x7f0e180ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:56412/0 (socket says 172.21.15.181:56412) 2023-10-16T11:22:01.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e17fff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e180d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:58416/0 (socket says 172.21.15.181:58416) 2023-10-16T11:22:01.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1d205700 1 -- 172.21.15.181:0/3284204058 learned_addr learned my addr 172.21.15.181:0/3284204058 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:01.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e1ca04700 1 --2- 172.21.15.181:0/3284204058 >> v2:172.21.15.190:3300/0 conn(0x7f0e1815dc20 0x7f0e180d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e177fe700 1 -- 172.21.15.181:0/3284204058 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062918033 0 0) 0x7f0e180b2f00 con 0x7f0e180ce4b0 2023-10-16T11:22:01.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e17fff700 1 -- 172.21.15.181:0/3284204058 >> v2:172.21.15.190:3300/0 conn(0x7f0e1815dc20 msgr2=0x7f0e180d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:01.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e17fff700 1 --2- 172.21.15.181:0/3284204058 >> v2:172.21.15.190:3300/0 conn(0x7f0e1815dc20 0x7f0e180d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e17fff700 1 -- 172.21.15.181:0/3284204058 >> v1:172.21.15.190:6789/0 conn(0x7f0e180ce4b0 legacy=0x7f0e180ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.375+0000 7f0e17fff700 1 -- 172.21.15.181:0/3284204058 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180b31e0 con 0x7f0e180d1e60 2023-10-16T11:22:01.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e17fff700 1 --2- 172.21.15.181:0/3284204058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e180d22e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009840 tx=0x7f0e04009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310c67698681969b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e177fe700 1 -- 172.21.15.181:0/3284204058 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04011830 con 0x7f0e180d1e60 2023-10-16T11:22:01.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e177fe700 1 -- 172.21.15.181:0/3284204058 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e04011990 con 0x7f0e180d1e60 2023-10-16T11:22:01.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e177fe700 1 -- 172.21.15.181:0/3284204058 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04011d20 con 0x7f0e180d1e60 2023-10-16T11:22:01.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 -- 172.21.15.181:0/3284204058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 msgr2=0x7f0e180d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/3284204058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e180d22e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009840 tx=0x7f0e04009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:01.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 -- 172.21.15.181:0/3284204058 shutdown_connections 2023-10-16T11:22:01.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/3284204058 >> v2:172.21.15.190:3300/0 conn(0x7f0e1815dc20 0x7f0e180d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/3284204058 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e180d22e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 -- 172.21.15.181:0/3284204058 >> 172.21.15.181:0/3284204058 conn(0x7f0e1808af90 msgr2=0x7f0e180d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:01.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 -- 172.21.15.181:0/3284204058 shutdown_connections 2023-10-16T11:22:01.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.376+0000 7f0e1ec68700 1 -- 172.21.15.181:0/3284204058 wait complete. 2023-10-16T11:22:01.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.378+0000 7f0e1ec68700 1 Processor -- start 2023-10-16T11:22:01.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.378+0000 7f0e1ec68700 1 -- start start 2023-10-16T11:22:01.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1ec68700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e180ce4b0 0x7f0e18153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1ec68700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e1815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1ec68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e1815dc20 0x7f0e180d3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1ec68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e18154400 con 0x7f0e1815dc20 2023-10-16T11:22:01.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1ec68700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180cb890 con 0x7f0e180ce4b0 2023-10-16T11:22:01.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1ec68700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e18153ad0 con 0x7f0e180d1e60 2023-10-16T11:22:01.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1d205700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e1815dc20 0x7f0e180d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1d205700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e1815dc20 0x7f0e180d3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33164/0 (socket says 172.21.15.181:33164) 2023-10-16T11:22:01.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1d205700 1 -- 172.21.15.181:0/2468222412 learned_addr learned my addr 172.21.15.181:0/2468222412 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:01.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e1ca04700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e180ce4b0 0x7f0e18153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.379+0000 7f0e17fff700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e1815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1d205700 1 -- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 msgr2=0x7f0e1815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1d205700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e1815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1d205700 1 -- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e180ce4b0 msgr2=0x7f0e18153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1d205700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e180ce4b0 0x7f0e18153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1d205700 1 -- 172.21.15.181:0/2468222412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e040094a0 con 0x7f0e1815dc20 2023-10-16T11:22:01.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1d205700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e1815dc20 0x7f0e180d3790 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000cba0 tx=0x7f0e1000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e157fa700 1 -- 172.21.15.181:0/2468222412 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000ab80 con 0x7f0e1815dc20 2023-10-16T11:22:01.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e157fa700 1 -- 172.21.15.181:0/2468222412 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1000ad00 con 0x7f0e1815dc20 2023-10-16T11:22:01.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e18153db0 con 0x7f0e1815dc20 2023-10-16T11:22:01.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e157fa700 1 -- 172.21.15.181:0/2468222412 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10007310 con 0x7f0e1815dc20 2023-10-16T11:22:01.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.380+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e180d3cd0 con 0x7f0e1815dc20 2023-10-16T11:22:01.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.381+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e18156740 con 0x7f0e1815dc20 2023-10-16T11:22:01.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.382+0000 7f0e157fa700 1 -- 172.21.15.181:0/2468222412 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0e10007470 con 0x7f0e1815dc20 2023-10-16T11:22:01.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.382+0000 7f0e157fa700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0dfc06b020 0x7f0dfc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.382+0000 7f0e157fa700 1 -- 172.21.15.181:0/2468222412 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f0e10089840 con 0x7f0e1815dc20 2023-10-16T11:22:01.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.382+0000 7f0e1ca04700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0dfc06b020 0x7f0dfc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.382+0000 7f0e1ca04700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0dfc06b020 0x7f0dfc06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009b50 tx=0x7f0e0c008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.392 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.386+0000 7f0e157fa700 1 -- 172.21.15.181:0/2468222412 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f0e100545b0 con 0x7f0e1815dc20 2023-10-16T11:22:01.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.573+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 --> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e180b37a0 con 0x7f0dfc06b020 2023-10-16T11:22:01.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.581+0000 7f0e157fa700 1 -- 172.21.15.181:0/2468222412 <== mgr.13197 v2:172.21.15.181:6832/2291437477 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290159 (secure 0 0 0) 0x7f0e0c0117b0 con 0x7f0dfc06b020 2023-10-16T11:22:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:01.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0dfc06b020 msgr2=0x7f0dfc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0dfc06b020 0x7f0dfc06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009b50 tx=0x7f0e0c008040 comp rx=0 tx=0).stop 2023-10-16T11:22:01.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e1815dc20 msgr2=0x7f0e180d3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e1815dc20 0x7f0e180d3790 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000cba0 tx=0x7f0e1000cf60 comp rx=0 tx=0).stop 2023-10-16T11:22:01.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 shutdown_connections 2023-10-16T11:22:01.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e1815dc20 0x7f0e180d3790 unknown :-1 s=CLOSED pgs=3139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e180ce4b0 0x7f0e18153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f0dfc06b020 0x7f0dfc06d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 --2- 172.21.15.181:0/2468222412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e180d1e60 0x7f0e1815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 >> 172.21.15.181:0/2468222412 conn(0x7f0e1808af90 msgr2=0x7f0e18156030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:01.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 shutdown_connections 2023-10-16T11:22:01.590 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.584+0000 7f0e1ec68700 1 -- 172.21.15.181:0/2468222412 wait complete. 2023-10-16T11:22:01.591 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:22:01.599 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T11:22:00.887972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":723,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":441,"ondisk_log_size":441,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":482564,"kb_used_data":27712,"kb_used_omap":12,"kb_used_meta":454707,"kb_avail":754492156,"statfs":{"total":773094113280,"available":772599967744,"internally_reserved":0,"allocated":28377088,"data_stored":15282540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":465620536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815763"},"pg_stats":[{"pgid":"37.3f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110444+0000","last_change":"2023-10-16T11:21:57.103408+0000","last_active":"2023-10-16T11:21:58.110444+0000","last_peered":"2023-10-16T11:21:58.110444+0000","last_clean":"2023-10-16T11:21:58.110444+0000","last_became_active":"2023-10-16T11:21:57.103039+0000","last_became_peered":"2023-10-16T11:21:57.103039+0000","last_unstale":"2023-10-16T11:21:58.110444+0000","last_undegraded":"2023-10-16T11:21:58.110444+0000","last_fullsized":"2023-10-16T11:21:58.110444+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:48:59.536665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.3e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599838+0000","last_change":"2023-10-16T11:21:57.104067+0000","last_active":"2023-10-16T11:21:58.599838+0000","last_peered":"2023-10-16T11:21:58.599838+0000","last_clean":"2023-10-16T11:21:58.599838+0000","last_became_active":"2023-10-16T11:21:57.103794+0000","last_became_peered":"2023-10-16T11:21:57.103794+0000","last_unstale":"2023-10-16T11:21:58.599838+0000","last_undegraded":"2023-10-16T11:21:58.599838+0000","last_fullsized":"2023-10-16T11:21:58.599838+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:22:13.049091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.3d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110113+0000","last_change":"2023-10-16T11:21:57.108404+0000","last_active":"2023-10-16T11:21:58.110113+0000","last_peered":"2023-10-16T11:21:58.110113+0000","last_clean":"2023-10-16T11:21:58.110113+0000","last_became_active":"2023-10-16T11:21:57.104988+0000","last_became_peered":"2023-10-16T11:21:57.104988+0000","last_unstale":"2023-10-16T11:21:58.110113+0000","last_undegraded":"2023-10-16T11:21:58.110113+0000","last_fullsized":"2023-10-16T11:21:58.110113+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:27:15.986005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110263+0000","last_change":"2023-10-16T11:21:57.104802+0000","last_active":"2023-10-16T11:21:58.110263+0000","last_peered":"2023-10-16T11:21:58.110263+0000","last_clean":"2023-10-16T11:21:58.110263+0000","last_became_active":"2023-10-16T11:21:57.104462+0000","last_became_peered":"2023-10-16T11:21:57.104462+0000","last_unstale":"2023-10-16T11:21:58.110263+0000","last_undegraded":"2023-10-16T11:21:58.110263+0000","last_fullsized":"2023-10-16T11:21:58.110263+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:05.307751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110334+0000","last_change":"2023-10-16T11:21:57.108578+0000","last_active":"2023-10-16T11:21:58.110334+0000","last_peered":"2023-10-16T11:21:58.110334+0000","last_clean":"2023-10-16T11:21:58.110334+0000","last_became_active":"2023-10-16T11:21:57.104875+0000","last_became_peered":"2023-10-16T11:21:57.104875+0000","last_unstale":"2023-10-16T11:21:58.110334+0000","last_undegraded":"2023-10-16T11:21:58.110334+0000","last_fullsized":"2023-10-16T11:21:58.110334+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:20:39.224053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.3a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108886+0000","last_change":"2023-10-16T11:21:57.100994+0000","last_active":"2023-10-16T11:21:58.108886+0000","last_peered":"2023-10-16T11:21:58.108886+0000","last_clean":"2023-10-16T11:21:58.108886+0000","last_became_active":"2023-10-16T11:21:57.100540+0000","last_became_peered":"2023-10-16T11:21:57.100540+0000","last_unstale":"2023-10-16T11:21:58.108886+0000","last_undegraded":"2023-10-16T11:21:58.108886+0000","last_fullsized":"2023-10-16T11:21:58.108886+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:46.618520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.39","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109944+0000","last_change":"2023-10-16T11:21:57.101674+0000","last_active":"2023-10-16T11:21:58.109944+0000","last_peered":"2023-10-16T11:21:58.109944+0000","last_clean":"2023-10-16T11:21:58.109944+0000","last_became_active":"2023-10-16T11:21:57.101318+0000","last_became_peered":"2023-10-16T11:21:57.101318+0000","last_unstale":"2023-10-16T11:21:58.109944+0000","last_undegraded":"2023-10-16T11:21:58.109944+0000","last_fullsized":"2023-10-16T11:21:58.109944+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:02:39.873287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.38","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599674+0000","last_change":"2023-10-16T11:21:57.103737+0000","last_active":"2023-10-16T11:21:58.599674+0000","last_peered":"2023-10-16T11:21:58.599674+0000","last_clean":"2023-10-16T11:21:58.599674+0000","last_became_active":"2023-10-16T11:21:57.103353+0000","last_became_peered":"2023-10-16T11:21:57.103353+0000","last_unstale":"2023-10-16T11:21:58.599674+0000","last_undegraded":"2023-10-16T11:21:58.599674+0000","last_fullsized":"2023-10-16T11:21:58.599674+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:13.461214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.37","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600379+0000","last_change":"2023-10-16T11:21:57.106474+0000","last_active":"2023-10-16T11:21:58.600379+0000","last_peered":"2023-10-16T11:21:58.600379+0000","last_clean":"2023-10-16T11:21:58.600379+0000","last_became_active":"2023-10-16T11:21:57.106174+0000","last_became_peered":"2023-10-16T11:21:57.106174+0000","last_unstale":"2023-10-16T11:21:58.600379+0000","last_undegraded":"2023-10-16T11:21:58.600379+0000","last_fullsized":"2023-10-16T11:21:58.600379+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:20:26.953180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.36","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110348+0000","last_change":"2023-10-16T11:21:57.110568+0000","last_active":"2023-10-16T11:21:58.110348+0000","last_peered":"2023-10-16T11:21:58.110348+0000","last_clean":"2023-10-16T11:21:58.110348+0000","last_became_active":"2023-10-16T11:21:57.110283+0000","last_became_peered":"2023-10-16T11:21:57.110283+0000","last_unstale":"2023-10-16T11:21:58.110348+0000","last_undegraded":"2023-10-16T11:21:58.110348+0000","last_fullsized":"2023-10-16T11:21:58.110348+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:31:09.611531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.35","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599963+0000","last_change":"2023-10-16T11:21:57.107717+0000","last_active":"2023-10-16T11:21:58.599963+0000","last_peered":"2023-10-16T11:21:58.599963+0000","last_clean":"2023-10-16T11:21:58.599963+0000","last_became_active":"2023-10-16T11:21:57.107452+0000","last_became_peered":"2023-10-16T11:21:57.107452+0000","last_unstale":"2023-10-16T11:21:58.599963+0000","last_undegraded":"2023-10-16T11:21:58.599963+0000","last_fullsized":"2023-10-16T11:21:58.599963+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:41.982466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.34","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600753+0000","last_change":"2023-10-16T11:21:57.102771+0000","last_active":"2023-10-16T11:21:58.600753+0000","last_peered":"2023-10-16T11:21:58.600753+0000","last_clean":"2023-10-16T11:21:58.600753+0000","last_became_active":"2023-10-16T11:21:57.102499+0000","last_became_peered":"2023-10-16T11:21:57.102499+0000","last_unstale":"2023-10-16T11:21:58.600753+0000","last_undegraded":"2023-10-16T11:21:58.600753+0000","last_fullsized":"2023-10-16T11:21:58.600753+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:09.764520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.33","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600040+0000","last_change":"2023-10-16T11:21:57.113283+0000","last_active":"2023-10-16T11:21:58.600040+0000","last_peered":"2023-10-16T11:21:58.600040+0000","last_clean":"2023-10-16T11:21:58.600040+0000","last_became_active":"2023-10-16T11:21:57.110770+0000","last_became_peered":"2023-10-16T11:21:57.110770+0000","last_unstale":"2023-10-16T11:21:58.600040+0000","last_undegraded":"2023-10-16T11:21:58.600040+0000","last_fullsized":"2023-10-16T11:21:58.600040+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:28:15.358414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.32","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600825+0000","last_change":"2023-10-16T11:21:57.110696+0000","last_active":"2023-10-16T11:21:58.600825+0000","last_peered":"2023-10-16T11:21:58.600825+0000","last_clean":"2023-10-16T11:21:58.600825+0000","last_became_active":"2023-10-16T11:21:57.110420+0000","last_became_peered":"2023-10-16T11:21:57.110420+0000","last_unstale":"2023-10-16T11:21:58.600825+0000","last_undegraded":"2023-10-16T11:21:58.600825+0000","last_fullsized":"2023-10-16T11:21:58.600825+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:25:57.596577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.31","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110004+0000","last_change":"2023-10-16T11:21:57.119288+0000","last_active":"2023-10-16T11:21:58.110004+0000","last_peered":"2023-10-16T11:21:58.110004+0000","last_clean":"2023-10-16T11:21:58.110004+0000","last_became_active":"2023-10-16T11:21:57.117885+0000","last_became_peered":"2023-10-16T11:21:57.117885+0000","last_unstale":"2023-10-16T11:21:58.110004+0000","last_undegraded":"2023-10-16T11:21:58.110004+0000","last_fullsized":"2023-10-16T11:21:58.110004+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:55:16.872898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.30","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600580+0000","last_change":"2023-10-16T11:21:57.113483+0000","last_active":"2023-10-16T11:21:58.600580+0000","last_peered":"2023-10-16T11:21:58.600580+0000","last_clean":"2023-10-16T11:21:58.600580+0000","last_became_active":"2023-10-16T11:21:57.113216+0000","last_became_peered":"2023-10-16T11:21:57.113216+0000","last_unstale":"2023-10-16T11:21:58.600580+0000","last_undegraded":"2023-10-16T11:21:58.600580+0000","last_fullsized":"2023-10-16T11:21:58.600580+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:31.670116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600845+0000","last_change":"2023-10-16T11:21:57.112197+0000","last_active":"2023-10-16T11:21:58.600845+0000","last_peered":"2023-10-16T11:21:58.600845+0000","last_clean":"2023-10-16T11:21:58.600845+0000","last_became_active":"2023-10-16T11:21:57.110851+0000","last_became_peered":"2023-10-16T11:21:57.110851+0000","last_unstale":"2023-10-16T11:21:58.600845+0000","last_undegraded":"2023-10-16T11:21:58.600845+0000","last_fullsized":"2023-10-16T11:21:58.600845+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:09.263907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600733+0000","last_change":"2023-10-16T11:21:57.112831+0000","last_active":"2023-10-16T11:21:58.600733+0000","last_peered":"2023-10-16T11:21:58.600733+0000","last_clean":"2023-10-16T11:21:58.600733+0000","last_became_active":"2023-10-16T11:21:57.111893+0000","last_became_peered":"2023-10-16T11:21:57.111893+0000","last_unstale":"2023-10-16T11:21:58.600733+0000","last_undegraded":"2023-10-16T11:21:58.600733+0000","last_fullsized":"2023-10-16T11:21:58.600733+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:07:08.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.2d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600870+0000","last_change":"2023-10-16T11:21:57.109112+0000","last_active":"2023-10-16T11:21:58.600870+0000","last_peered":"2023-10-16T11:21:58.600870+0000","last_clean":"2023-10-16T11:21:58.600870+0000","last_became_active":"2023-10-16T11:21:57.108466+0000","last_became_peered":"2023-10-16T11:21:57.108466+0000","last_unstale":"2023-10-16T11:21:58.600870+0000","last_undegraded":"2023-10-16T11:21:58.600870+0000","last_fullsized":"2023-10-16T11:21:58.600870+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:45:40.013407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.2c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599932+0000","last_change":"2023-10-16T11:21:57.103066+0000","last_active":"2023-10-16T11:21:58.599932+0000","last_peered":"2023-10-16T11:21:58.599932+0000","last_clean":"2023-10-16T11:21:58.599932+0000","last_became_active":"2023-10-16T11:21:57.102752+0000","last_became_peered":"2023-10-16T11:21:57.102752+0000","last_unstale":"2023-10-16T11:21:58.599932+0000","last_undegraded":"2023-10-16T11:21:58.599932+0000","last_fullsized":"2023-10-16T11:21:58.599932+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:35:27.717350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"37.2b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600017+0000","last_change":"2023-10-16T11:21:57.107889+0000","last_active":"2023-10-16T11:21:58.600017+0000","last_peered":"2023-10-16T11:21:58.600017+0000","last_clean":"2023-10-16T11:21:58.600017+0000","last_became_active":"2023-10-16T11:21:57.107079+0000","last_became_peered":"2023-10-16T11:21:57.107079+0000","last_unstale":"2023-10-16T11:21:58.600017+0000","last_undegraded":"2023-10-16T11:21:58.600017+0000","last_fullsized":"2023-10-16T11:21:58.600017+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:10:26.049018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.2a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109930+0000","last_change":"2023-10-16T11:21:57.112519+0000","last_active":"2023-10-16T11:21:58.109930+0000","last_peered":"2023-10-16T11:21:58.109930+0000","last_clean":"2023-10-16T11:21:58.109930+0000","last_became_active":"2023-10-16T11:21:57.112184+0000","last_became_peered":"2023-10-16T11:21:57.112184+0000","last_unstale":"2023-10-16T11:21:58.109930+0000","last_undegraded":"2023-10-16T11:21:58.109930+0000","last_fullsized":"2023-10-16T11:21:58.109930+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:35.169498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.29","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601266+0000","last_change":"2023-10-16T11:21:57.110605+0000","last_active":"2023-10-16T11:21:58.601266+0000","last_peered":"2023-10-16T11:21:58.601266+0000","last_clean":"2023-10-16T11:21:58.601266+0000","last_became_active":"2023-10-16T11:21:57.110315+0000","last_became_peered":"2023-10-16T11:21:57.110315+0000","last_unstale":"2023-10-16T11:21:58.601266+0000","last_undegraded":"2023-10-16T11:21:58.601266+0000","last_fullsized":"2023-10-16T11:21:58.601266+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:18:07.870966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.28","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601086+0000","last_change":"2023-10-16T11:21:57.108878+0000","last_active":"2023-10-16T11:21:58.601086+0000","last_peered":"2023-10-16T11:21:58.601086+0000","last_clean":"2023-10-16T11:21:58.601086+0000","last_became_active":"2023-10-16T11:21:57.108573+0000","last_became_peered":"2023-10-16T11:21:57.108573+0000","last_unstale":"2023-10-16T11:21:58.601086+0000","last_undegraded":"2023-10-16T11:21:58.601086+0000","last_fullsized":"2023-10-16T11:21:58.601086+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:34:38.421119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.27","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600992+0000","last_change":"2023-10-16T11:21:57.114221+0000","last_active":"2023-10-16T11:21:58.600992+0000","last_peered":"2023-10-16T11:21:58.600992+0000","last_clean":"2023-10-16T11:21:58.600992+0000","last_became_active":"2023-10-16T11:21:57.113963+0000","last_became_peered":"2023-10-16T11:21:57.113963+0000","last_unstale":"2023-10-16T11:21:58.600992+0000","last_undegraded":"2023-10-16T11:21:58.600992+0000","last_fullsized":"2023-10-16T11:21:58.600992+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:46.242576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.26","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110380+0000","last_change":"2023-10-16T11:21:57.109269+0000","last_active":"2023-10-16T11:21:58.110380+0000","last_peered":"2023-10-16T11:21:58.110380+0000","last_clean":"2023-10-16T11:21:58.110380+0000","last_became_active":"2023-10-16T11:21:57.108973+0000","last_became_peered":"2023-10-16T11:21:57.108973+0000","last_unstale":"2023-10-16T11:21:58.110380+0000","last_undegraded":"2023-10-16T11:21:58.110380+0000","last_fullsized":"2023-10-16T11:21:58.110380+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:28:49.076157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"37.25","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600932+0000","last_change":"2023-10-16T11:21:57.106793+0000","last_active":"2023-10-16T11:21:58.600932+0000","last_peered":"2023-10-16T11:21:58.600932+0000","last_clean":"2023-10-16T11:21:58.600932+0000","last_became_active":"2023-10-16T11:21:57.106546+0000","last_became_peered":"2023-10-16T11:21:57.106546+0000","last_unstale":"2023-10-16T11:21:58.600932+0000","last_undegraded":"2023-10-16T11:21:58.600932+0000","last_fullsized":"2023-10-16T11:21:58.600932+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:04:06.761299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.23","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108899+0000","last_change":"2023-10-16T11:21:57.104108+0000","last_active":"2023-10-16T11:21:58.108899+0000","last_peered":"2023-10-16T11:21:58.108899+0000","last_clean":"2023-10-16T11:21:58.108899+0000","last_became_active":"2023-10-16T11:21:57.103804+0000","last_became_peered":"2023-10-16T11:21:57.103804+0000","last_unstale":"2023-10-16T11:21:58.108899+0000","last_undegraded":"2023-10-16T11:21:58.108899+0000","last_fullsized":"2023-10-16T11:21:58.108899+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:33.653509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.22","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600553+0000","last_change":"2023-10-16T11:21:57.114747+0000","last_active":"2023-10-16T11:21:58.600553+0000","last_peered":"2023-10-16T11:21:58.600553+0000","last_clean":"2023-10-16T11:21:58.600553+0000","last_became_active":"2023-10-16T11:21:57.114485+0000","last_became_peered":"2023-10-16T11:21:57.114485+0000","last_unstale":"2023-10-16T11:21:58.600553+0000","last_undegraded":"2023-10-16T11:21:58.600553+0000","last_fullsized":"2023-10-16T11:21:58.600553+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:20.214848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.21","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599894+0000","last_change":"2023-10-16T11:21:57.107795+0000","last_active":"2023-10-16T11:21:58.599894+0000","last_peered":"2023-10-16T11:21:58.599894+0000","last_clean":"2023-10-16T11:21:58.599894+0000","last_became_active":"2023-10-16T11:21:57.107300+0000","last_became_peered":"2023-10-16T11:21:57.107300+0000","last_unstale":"2023-10-16T11:21:58.599894+0000","last_undegraded":"2023-10-16T11:21:58.599894+0000","last_fullsized":"2023-10-16T11:21:58.599894+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:19.235355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.20","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109919+0000","last_change":"2023-10-16T11:21:57.101954+0000","last_active":"2023-10-16T11:21:58.109919+0000","last_peered":"2023-10-16T11:21:58.109919+0000","last_clean":"2023-10-16T11:21:58.109919+0000","last_became_active":"2023-10-16T11:21:57.101510+0000","last_became_peered":"2023-10-16T11:21:57.101510+0000","last_unstale":"2023-10-16T11:21:58.109919+0000","last_undegraded":"2023-10-16T11:21:58.109919+0000","last_fullsized":"2023-10-16T11:21:58.109919+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:20:22.980523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600524+0000","last_change":"2023-10-16T11:21:57.101258+0000","last_active":"2023-10-16T11:21:58.600524+0000","last_peered":"2023-10-16T11:21:58.600524+0000","last_clean":"2023-10-16T11:21:58.600524+0000","last_became_active":"2023-10-16T11:21:57.100961+0000","last_became_peered":"2023-10-16T11:21:57.100961+0000","last_unstale":"2023-10-16T11:21:58.600524+0000","last_undegraded":"2023-10-16T11:21:58.600524+0000","last_fullsized":"2023-10-16T11:21:58.600524+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:30:55.020140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.1f","version":"268'4","reported_seq":16,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601175+0000","last_change":"2023-10-16T11:21:56.085945+0000","last_active":"2023-10-16T11:21:58.601175+0000","last_peered":"2023-10-16T11:21:58.601175+0000","last_clean":"2023-10-16T11:21:58.601175+0000","last_became_active":"2023-10-16T11:21:56.085707+0000","last_became_peered":"2023-10-16T11:21:56.085707+0000","last_unstale":"2023-10-16T11:21:58.601175+0000","last_undegraded":"2023-10-16T11:21:58.601175+0000","last_fullsized":"2023-10-16T11:21:58.601175+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:34:39.723203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601065+0000","last_change":"2023-10-16T11:21:57.122061+0000","last_active":"2023-10-16T11:21:58.601065+0000","last_peered":"2023-10-16T11:21:58.601065+0000","last_clean":"2023-10-16T11:21:58.601065+0000","last_became_active":"2023-10-16T11:21:57.121634+0000","last_became_peered":"2023-10-16T11:21:57.121634+0000","last_unstale":"2023-10-16T11:21:58.601065+0000","last_undegraded":"2023-10-16T11:21:58.601065+0000","last_fullsized":"2023-10-16T11:21:58.601065+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:05:08.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.1e","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108963+0000","last_change":"2023-10-16T11:21:56.095843+0000","last_active":"2023-10-16T11:21:58.108963+0000","last_peered":"2023-10-16T11:21:58.108963+0000","last_clean":"2023-10-16T11:21:58.108963+0000","last_became_active":"2023-10-16T11:21:56.095553+0000","last_became_peered":"2023-10-16T11:21:56.095553+0000","last_unstale":"2023-10-16T11:21:58.108963+0000","last_undegraded":"2023-10-16T11:21:58.108963+0000","last_fullsized":"2023-10-16T11:21:58.108963+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:35:50.330309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109116+0000","last_change":"2023-10-16T11:21:57.107916+0000","last_active":"2023-10-16T11:21:58.109116+0000","last_peered":"2023-10-16T11:21:58.109116+0000","last_clean":"2023-10-16T11:21:58.109116+0000","last_became_active":"2023-10-16T11:21:57.107628+0000","last_became_peered":"2023-10-16T11:21:57.107628+0000","last_unstale":"2023-10-16T11:21:58.109116+0000","last_undegraded":"2023-10-16T11:21:58.109116+0000","last_fullsized":"2023-10-16T11:21:58.109116+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:44.353916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1d","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109284+0000","last_change":"2023-10-16T11:21:56.092059+0000","last_active":"2023-10-16T11:21:58.109284+0000","last_peered":"2023-10-16T11:21:58.109284+0000","last_clean":"2023-10-16T11:21:58.109284+0000","last_became_active":"2023-10-16T11:21:56.091479+0000","last_became_peered":"2023-10-16T11:21:56.091479+0000","last_unstale":"2023-10-16T11:21:58.109284+0000","last_undegraded":"2023-10-16T11:21:58.109284+0000","last_fullsized":"2023-10-16T11:21:58.109284+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:00:21.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.1d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600260+0000","last_change":"2023-10-16T11:21:57.110071+0000","last_active":"2023-10-16T11:21:58.600260+0000","last_peered":"2023-10-16T11:21:58.600260+0000","last_clean":"2023-10-16T11:21:58.600260+0000","last_became_active":"2023-10-16T11:21:57.109760+0000","last_became_peered":"2023-10-16T11:21:57.109760+0000","last_unstale":"2023-10-16T11:21:58.600260+0000","last_undegraded":"2023-10-16T11:21:58.600260+0000","last_fullsized":"2023-10-16T11:21:58.600260+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:38.376580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1c","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110507+0000","last_change":"2023-10-16T11:21:56.093566+0000","last_active":"2023-10-16T11:21:58.110507+0000","last_peered":"2023-10-16T11:21:58.110507+0000","last_clean":"2023-10-16T11:21:58.110507+0000","last_became_active":"2023-10-16T11:21:56.093272+0000","last_became_peered":"2023-10-16T11:21:56.093272+0000","last_unstale":"2023-10-16T11:21:58.110507+0000","last_undegraded":"2023-10-16T11:21:58.110507+0000","last_fullsized":"2023-10-16T11:21:58.110507+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:01:43.349367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.1a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600283+0000","last_change":"2023-10-16T11:21:57.104361+0000","last_active":"2023-10-16T11:21:58.600283+0000","last_peered":"2023-10-16T11:21:58.600283+0000","last_clean":"2023-10-16T11:21:58.600283+0000","last_became_active":"2023-10-16T11:21:57.104091+0000","last_became_peered":"2023-10-16T11:21:57.104091+0000","last_unstale":"2023-10-16T11:21:58.600283+0000","last_undegraded":"2023-10-16T11:21:58.600283+0000","last_fullsized":"2023-10-16T11:21:58.600283+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:30:52.665690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1b","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600901+0000","last_change":"2023-10-16T11:21:56.093241+0000","last_active":"2023-10-16T11:21:58.600901+0000","last_peered":"2023-10-16T11:21:58.600901+0000","last_clean":"2023-10-16T11:21:58.600901+0000","last_became_active":"2023-10-16T11:21:56.092977+0000","last_became_peered":"2023-10-16T11:21:56.092977+0000","last_unstale":"2023-10-16T11:21:58.600901+0000","last_undegraded":"2023-10-16T11:21:58.600901+0000","last_fullsized":"2023-10-16T11:21:58.600901+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:39:23.976293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.1b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599966+0000","last_change":"2023-10-16T11:21:57.106549+0000","last_active":"2023-10-16T11:21:58.599966+0000","last_peered":"2023-10-16T11:21:58.599966+0000","last_clean":"2023-10-16T11:21:58.599966+0000","last_became_active":"2023-10-16T11:21:57.106260+0000","last_became_peered":"2023-10-16T11:21:57.106260+0000","last_unstale":"2023-10-16T11:21:58.599966+0000","last_undegraded":"2023-10-16T11:21:58.599966+0000","last_fullsized":"2023-10-16T11:21:58.599966+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:34:55.318964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1a","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600648+0000","last_change":"2023-10-16T11:21:56.094876+0000","last_active":"2023-10-16T11:21:58.600648+0000","last_peered":"2023-10-16T11:21:58.600648+0000","last_clean":"2023-10-16T11:21:58.600648+0000","last_became_active":"2023-10-16T11:21:56.094625+0000","last_became_peered":"2023-10-16T11:21:56.094625+0000","last_unstale":"2023-10-16T11:21:58.600648+0000","last_undegraded":"2023-10-16T11:21:58.600648+0000","last_fullsized":"2023-10-16T11:21:58.600648+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:36.857225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.18","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600173+0000","last_change":"2023-10-16T11:21:57.110831+0000","last_active":"2023-10-16T11:21:58.600173+0000","last_peered":"2023-10-16T11:21:58.600173+0000","last_clean":"2023-10-16T11:21:58.600173+0000","last_became_active":"2023-10-16T11:21:57.110541+0000","last_became_peered":"2023-10-16T11:21:57.110541+0000","last_unstale":"2023-10-16T11:21:58.600173+0000","last_undegraded":"2023-10-16T11:21:58.600173+0000","last_fullsized":"2023-10-16T11:21:58.600173+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:22.081940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600511+0000","last_change":"2023-10-16T11:21:56.089461+0000","last_active":"2023-10-16T11:21:58.600511+0000","last_peered":"2023-10-16T11:21:58.600511+0000","last_clean":"2023-10-16T11:21:58.600511+0000","last_became_active":"2023-10-16T11:21:56.089171+0000","last_became_peered":"2023-10-16T11:21:56.089171+0000","last_unstale":"2023-10-16T11:21:58.600511+0000","last_undegraded":"2023-10-16T11:21:58.600511+0000","last_fullsized":"2023-10-16T11:21:58.600511+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:26:40.317963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.19","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108930+0000","last_change":"2023-10-16T11:21:57.101482+0000","last_active":"2023-10-16T11:21:58.108930+0000","last_peered":"2023-10-16T11:21:58.108930+0000","last_clean":"2023-10-16T11:21:58.108930+0000","last_became_active":"2023-10-16T11:21:57.101135+0000","last_became_peered":"2023-10-16T11:21:57.101135+0000","last_unstale":"2023-10-16T11:21:58.108930+0000","last_undegraded":"2023-10-16T11:21:58.108930+0000","last_fullsized":"2023-10-16T11:21:58.108930+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:57:45.296959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108649+0000","last_change":"2023-10-16T11:21:56.093686+0000","last_active":"2023-10-16T11:21:58.108649+0000","last_peered":"2023-10-16T11:21:58.108649+0000","last_clean":"2023-10-16T11:21:58.108649+0000","last_became_active":"2023-10-16T11:21:56.093424+0000","last_became_peered":"2023-10-16T11:21:56.093424+0000","last_unstale":"2023-10-16T11:21:58.108649+0000","last_undegraded":"2023-10-16T11:21:58.108649+0000","last_fullsized":"2023-10-16T11:21:58.108649+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:06:43.501000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.16","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108972+0000","last_change":"2023-10-16T11:21:57.101779+0000","last_active":"2023-10-16T11:21:58.108972+0000","last_peered":"2023-10-16T11:21:58.108972+0000","last_clean":"2023-10-16T11:21:58.108972+0000","last_became_active":"2023-10-16T11:21:57.101506+0000","last_became_peered":"2023-10-16T11:21:57.101506+0000","last_unstale":"2023-10-16T11:21:58.108972+0000","last_undegraded":"2023-10-16T11:21:58.108972+0000","last_fullsized":"2023-10-16T11:21:58.108972+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:08:00.602787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601266+0000","last_change":"2023-10-16T11:21:56.095906+0000","last_active":"2023-10-16T11:21:58.601266+0000","last_peered":"2023-10-16T11:21:58.601266+0000","last_clean":"2023-10-16T11:21:58.601266+0000","last_became_active":"2023-10-16T11:21:56.095582+0000","last_became_peered":"2023-10-16T11:21:56.095582+0000","last_unstale":"2023-10-16T11:21:58.601266+0000","last_undegraded":"2023-10-16T11:21:58.601266+0000","last_fullsized":"2023-10-16T11:21:58.601266+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:22:28.440079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.17","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601075+0000","last_change":"2023-10-16T11:21:57.109634+0000","last_active":"2023-10-16T11:21:58.601075+0000","last_peered":"2023-10-16T11:21:58.601075+0000","last_clean":"2023-10-16T11:21:58.601075+0000","last_became_active":"2023-10-16T11:21:57.108191+0000","last_became_peered":"2023-10-16T11:21:57.108191+0000","last_unstale":"2023-10-16T11:21:58.601075+0000","last_undegraded":"2023-10-16T11:21:58.601075+0000","last_fullsized":"2023-10-16T11:21:58.601075+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:01.750572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600479+0000","last_change":"2023-10-16T11:21:56.086243+0000","last_active":"2023-10-16T11:21:58.600479+0000","last_peered":"2023-10-16T11:21:58.600479+0000","last_clean":"2023-10-16T11:21:58.600479+0000","last_became_active":"2023-10-16T11:21:56.085981+0000","last_became_peered":"2023-10-16T11:21:56.085981+0000","last_unstale":"2023-10-16T11:21:58.600479+0000","last_undegraded":"2023-10-16T11:21:58.600479+0000","last_fullsized":"2023-10-16T11:21:58.600479+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:11:29.707155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.14","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110446+0000","last_change":"2023-10-16T11:21:57.111164+0000","last_active":"2023-10-16T11:21:58.110446+0000","last_peered":"2023-10-16T11:21:58.110446+0000","last_clean":"2023-10-16T11:21:58.110446+0000","last_became_active":"2023-10-16T11:21:57.110893+0000","last_became_peered":"2023-10-16T11:21:57.110893+0000","last_unstale":"2023-10-16T11:21:58.110446+0000","last_undegraded":"2023-10-16T11:21:58.110446+0000","last_fullsized":"2023-10-16T11:21:58.110446+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:37:37.861774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.15","version":"268'3","reported_seq":15,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110112+0000","last_change":"2023-10-16T11:21:56.097616+0000","last_active":"2023-10-16T11:21:58.110112+0000","last_peered":"2023-10-16T11:21:58.110112+0000","last_clean":"2023-10-16T11:21:58.110112+0000","last_became_active":"2023-10-16T11:21:56.096477+0000","last_became_peered":"2023-10-16T11:21:56.096477+0000","last_unstale":"2023-10-16T11:21:58.110112+0000","last_undegraded":"2023-10-16T11:21:58.110112+0000","last_fullsized":"2023-10-16T11:21:58.110112+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:44:32.648961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.15","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109238+0000","last_change":"2023-10-16T11:21:57.109495+0000","last_active":"2023-10-16T11:21:58.109238+0000","last_peered":"2023-10-16T11:21:58.109238+0000","last_clean":"2023-10-16T11:21:58.109238+0000","last_became_active":"2023-10-16T11:21:57.109223+0000","last_became_peered":"2023-10-16T11:21:57.109223+0000","last_unstale":"2023-10-16T11:21:58.109238+0000","last_undegraded":"2023-10-16T11:21:58.109238+0000","last_fullsized":"2023-10-16T11:21:58.109238+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:39.699223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.14","version":"268'6","reported_seq":18,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600292+0000","last_change":"2023-10-16T11:21:56.086852+0000","last_active":"2023-10-16T11:21:58.600292+0000","last_peered":"2023-10-16T11:21:58.600292+0000","last_clean":"2023-10-16T11:21:58.600292+0000","last_became_active":"2023-10-16T11:21:56.085203+0000","last_became_peered":"2023-10-16T11:21:56.085203+0000","last_unstale":"2023-10-16T11:21:58.600292+0000","last_undegraded":"2023-10-16T11:21:58.600292+0000","last_fullsized":"2023-10-16T11:21:58.600292+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:32.895099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.12","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600451+0000","last_change":"2023-10-16T11:21:57.111512+0000","last_active":"2023-10-16T11:21:58.600451+0000","last_peered":"2023-10-16T11:21:58.600451+0000","last_clean":"2023-10-16T11:21:58.600451+0000","last_became_active":"2023-10-16T11:21:57.111192+0000","last_became_peered":"2023-10-16T11:21:57.111192+0000","last_unstale":"2023-10-16T11:21:58.600451+0000","last_undegraded":"2023-10-16T11:21:58.600451+0000","last_fullsized":"2023-10-16T11:21:58.600451+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:53:00.833411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600110+0000","last_change":"2023-10-16T11:21:56.094709+0000","last_active":"2023-10-16T11:21:58.600110+0000","last_peered":"2023-10-16T11:21:58.600110+0000","last_clean":"2023-10-16T11:21:58.600110+0000","last_became_active":"2023-10-16T11:21:56.094426+0000","last_became_peered":"2023-10-16T11:21:56.094426+0000","last_unstale":"2023-10-16T11:21:58.600110+0000","last_undegraded":"2023-10-16T11:21:58.600110+0000","last_fullsized":"2023-10-16T11:21:58.600110+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:37:08.759971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.13","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110070+0000","last_change":"2023-10-16T11:21:57.103895+0000","last_active":"2023-10-16T11:21:58.110070+0000","last_peered":"2023-10-16T11:21:58.110070+0000","last_clean":"2023-10-16T11:21:58.110070+0000","last_became_active":"2023-10-16T11:21:57.103586+0000","last_became_peered":"2023-10-16T11:21:57.103586+0000","last_unstale":"2023-10-16T11:21:58.110070+0000","last_undegraded":"2023-10-16T11:21:58.110070+0000","last_fullsized":"2023-10-16T11:21:58.110070+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:38.000095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109960+0000","last_change":"2023-10-16T11:21:56.095924+0000","last_active":"2023-10-16T11:21:58.109960+0000","last_peered":"2023-10-16T11:21:58.109960+0000","last_clean":"2023-10-16T11:21:58.109960+0000","last_became_active":"2023-10-16T11:21:56.095389+0000","last_became_peered":"2023-10-16T11:21:56.095389+0000","last_unstale":"2023-10-16T11:21:58.109960+0000","last_undegraded":"2023-10-16T11:21:58.109960+0000","last_fullsized":"2023-10-16T11:21:58.109960+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:15:20.624093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.10","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600201+0000","last_change":"2023-10-16T11:21:57.103683+0000","last_active":"2023-10-16T11:21:58.600201+0000","last_peered":"2023-10-16T11:21:58.600201+0000","last_clean":"2023-10-16T11:21:58.600201+0000","last_became_active":"2023-10-16T11:21:57.103349+0000","last_became_peered":"2023-10-16T11:21:57.103349+0000","last_unstale":"2023-10-16T11:21:58.600201+0000","last_undegraded":"2023-10-16T11:21:58.600201+0000","last_fullsized":"2023-10-16T11:21:58.600201+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:48.516759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601371+0000","last_change":"2023-10-16T11:21:56.089325+0000","last_active":"2023-10-16T11:21:58.601371+0000","last_peered":"2023-10-16T11:21:58.601371+0000","last_clean":"2023-10-16T11:21:58.601371+0000","last_became_active":"2023-10-16T11:21:56.088953+0000","last_became_peered":"2023-10-16T11:21:56.088953+0000","last_unstale":"2023-10-16T11:21:58.601371+0000","last_undegraded":"2023-10-16T11:21:58.601371+0000","last_fullsized":"2023-10-16T11:21:58.601371+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:06.807286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"37.11","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110122+0000","last_change":"2023-10-16T11:21:57.108137+0000","last_active":"2023-10-16T11:21:58.110122+0000","last_peered":"2023-10-16T11:21:58.110122+0000","last_clean":"2023-10-16T11:21:58.110122+0000","last_became_active":"2023-10-16T11:21:57.107836+0000","last_became_peered":"2023-10-16T11:21:57.107836+0000","last_unstale":"2023-10-16T11:21:58.110122+0000","last_undegraded":"2023-10-16T11:21:58.110122+0000","last_fullsized":"2023-10-16T11:21:58.110122+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:20:30.010063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.10","version":"268'8","reported_seq":20,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110397+0000","last_change":"2023-10-16T11:21:56.097420+0000","last_active":"2023-10-16T11:21:58.110397+0000","last_peered":"2023-10-16T11:21:58.110397+0000","last_clean":"2023-10-16T11:21:58.110397+0000","last_became_active":"2023-10-16T11:21:56.095598+0000","last_became_peered":"2023-10-16T11:21:56.095598+0000","last_unstale":"2023-10-16T11:21:58.110397+0000","last_undegraded":"2023-10-16T11:21:58.110397+0000","last_fullsized":"2023-10-16T11:21:58.110397+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:07:59.598238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.e","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110380+0000","last_change":"2023-10-16T11:21:57.118137+0000","last_active":"2023-10-16T11:21:58.110380+0000","last_peered":"2023-10-16T11:21:58.110380+0000","last_clean":"2023-10-16T11:21:58.110380+0000","last_became_active":"2023-10-16T11:21:57.117670+0000","last_became_peered":"2023-10-16T11:21:57.117670+0000","last_unstale":"2023-10-16T11:21:58.110380+0000","last_undegraded":"2023-10-16T11:21:58.110380+0000","last_fullsized":"2023-10-16T11:21:58.110380+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:05:33.899648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.f","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.599906+0000","last_change":"2023-10-16T11:21:56.091998+0000","last_active":"2023-10-16T11:21:58.599906+0000","last_peered":"2023-10-16T11:21:58.599906+0000","last_clean":"2023-10-16T11:21:58.599906+0000","last_became_active":"2023-10-16T11:21:56.091723+0000","last_became_peered":"2023-10-16T11:21:56.091723+0000","last_unstale":"2023-10-16T11:21:58.599906+0000","last_undegraded":"2023-10-16T11:21:58.599906+0000","last_fullsized":"2023-10-16T11:21:58.599906+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:44.778516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.f","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601397+0000","last_change":"2023-10-16T11:21:57.117970+0000","last_active":"2023-10-16T11:21:58.601397+0000","last_peered":"2023-10-16T11:21:58.601397+0000","last_clean":"2023-10-16T11:21:58.601397+0000","last_became_active":"2023-10-16T11:21:57.117695+0000","last_became_peered":"2023-10-16T11:21:57.117695+0000","last_unstale":"2023-10-16T11:21:58.601397+0000","last_undegraded":"2023-10-16T11:21:58.601397+0000","last_fullsized":"2023-10-16T11:21:58.601397+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:45:13.461214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.e","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600578+0000","last_change":"2023-10-16T11:21:56.085894+0000","last_active":"2023-10-16T11:21:58.600578+0000","last_peered":"2023-10-16T11:21:58.600578+0000","last_clean":"2023-10-16T11:21:58.600578+0000","last_became_active":"2023-10-16T11:21:56.085641+0000","last_became_peered":"2023-10-16T11:21:56.085641+0000","last_unstale":"2023-10-16T11:21:58.600578+0000","last_undegraded":"2023-10-16T11:21:58.600578+0000","last_fullsized":"2023-10-16T11:21:58.600578+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:58.532208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.c","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109047+0000","last_change":"2023-10-16T11:21:57.104245+0000","last_active":"2023-10-16T11:21:58.109047+0000","last_peered":"2023-10-16T11:21:58.109047+0000","last_clean":"2023-10-16T11:21:58.109047+0000","last_became_active":"2023-10-16T11:21:57.103988+0000","last_became_peered":"2023-10-16T11:21:57.103988+0000","last_unstale":"2023-10-16T11:21:58.109047+0000","last_undegraded":"2023-10-16T11:21:58.109047+0000","last_fullsized":"2023-10-16T11:21:58.109047+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:19.254965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.d","version":"268'1","reported_seq":13,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109352+0000","last_change":"2023-10-16T11:21:56.092253+0000","last_active":"2023-10-16T11:21:58.109352+0000","last_peered":"2023-10-16T11:21:58.109352+0000","last_clean":"2023-10-16T11:21:58.109352+0000","last_became_active":"2023-10-16T11:21:56.091785+0000","last_became_peered":"2023-10-16T11:21:56.091785+0000","last_unstale":"2023-10-16T11:21:58.109352+0000","last_undegraded":"2023-10-16T11:21:58.109352+0000","last_fullsized":"2023-10-16T11:21:58.109352+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:01:54.277523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.d","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600910+0000","last_change":"2023-10-16T11:21:57.108911+0000","last_active":"2023-10-16T11:21:58.600910+0000","last_peered":"2023-10-16T11:21:58.600910+0000","last_clean":"2023-10-16T11:21:58.600910+0000","last_became_active":"2023-10-16T11:21:57.108115+0000","last_became_peered":"2023-10-16T11:21:57.108115+0000","last_unstale":"2023-10-16T11:21:58.600910+0000","last_undegraded":"2023-10-16T11:21:58.600910+0000","last_fullsized":"2023-10-16T11:21:58.600910+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:12:09.288671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.109159+0000","last_change":"2023-10-16T11:21:56.091047+0000","last_active":"2023-10-16T11:21:58.109159+0000","last_peered":"2023-10-16T11:21:58.109159+0000","last_clean":"2023-10-16T11:21:58.109159+0000","last_became_active":"2023-10-16T11:21:56.090754+0000","last_became_peered":"2023-10-16T11:21:56.090754+0000","last_unstale":"2023-10-16T11:21:58.109159+0000","last_undegraded":"2023-10-16T11:21:58.109159+0000","last_fullsized":"2023-10-16T11:21:58.109159+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:00:48.105971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.a","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600259+0000","last_change":"2023-10-16T11:21:57.106835+0000","last_active":"2023-10-16T11:21:58.600259+0000","last_peered":"2023-10-16T11:21:58.600259+0000","last_clean":"2023-10-16T11:21:58.600259+0000","last_became_active":"2023-10-16T11:21:57.106502+0000","last_became_peered":"2023-10-16T11:21:57.106502+0000","last_unstale":"2023-10-16T11:21:58.600259+0000","last_undegraded":"2023-10-16T11:21:58.600259+0000","last_fullsized":"2023-10-16T11:21:58.600259+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:53:31.935527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.b","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110343+0000","last_change":"2023-10-16T11:21:56.095583+0000","last_active":"2023-10-16T11:21:58.110343+0000","last_peered":"2023-10-16T11:21:58.110343+0000","last_clean":"2023-10-16T11:21:58.110343+0000","last_became_active":"2023-10-16T11:21:56.095041+0000","last_became_peered":"2023-10-16T11:21:56.095041+0000","last_unstale":"2023-10-16T11:21:58.110343+0000","last_undegraded":"2023-10-16T11:21:58.110343+0000","last_fullsized":"2023-10-16T11:21:58.110343+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:20:38.694674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.b","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600106+0000","last_change":"2023-10-16T11:21:57.109272+0000","last_active":"2023-10-16T11:21:58.600106+0000","last_peered":"2023-10-16T11:21:58.600106+0000","last_clean":"2023-10-16T11:21:58.600106+0000","last_became_active":"2023-10-16T11:21:57.109013+0000","last_became_peered":"2023-10-16T11:21:57.109013+0000","last_unstale":"2023-10-16T11:21:58.600106+0000","last_undegraded":"2023-10-16T11:21:58.600106+0000","last_fullsized":"2023-10-16T11:21:58.600106+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:37:15.786493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600201+0000","last_change":"2023-10-16T11:21:56.086890+0000","last_active":"2023-10-16T11:21:58.600201+0000","last_peered":"2023-10-16T11:21:58.600201+0000","last_clean":"2023-10-16T11:21:58.600201+0000","last_became_active":"2023-10-16T11:21:56.086617+0000","last_became_peered":"2023-10-16T11:21:56.086617+0000","last_unstale":"2023-10-16T11:21:58.600201+0000","last_undegraded":"2023-10-16T11:21:58.600201+0000","last_fullsized":"2023-10-16T11:21:58.600201+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:29:30.665567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.8","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600195+0000","last_change":"2023-10-16T11:21:57.102277+0000","last_active":"2023-10-16T11:21:58.600195+0000","last_peered":"2023-10-16T11:21:58.600195+0000","last_clean":"2023-10-16T11:21:58.600195+0000","last_became_active":"2023-10-16T11:21:57.101966+0000","last_became_peered":"2023-10-16T11:21:57.101966+0000","last_unstale":"2023-10-16T11:21:58.600195+0000","last_undegraded":"2023-10-16T11:21:58.600195+0000","last_fullsized":"2023-10-16T11:21:58.600195+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:06.126659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108993+0000","last_change":"2023-10-16T11:21:56.091691+0000","last_active":"2023-10-16T11:21:58.108993+0000","last_peered":"2023-10-16T11:21:58.108993+0000","last_clean":"2023-10-16T11:21:58.108993+0000","last_became_active":"2023-10-16T11:21:56.091414+0000","last_became_peered":"2023-10-16T11:21:56.091414+0000","last_unstale":"2023-10-16T11:21:58.108993+0000","last_undegraded":"2023-10-16T11:21:58.108993+0000","last_fullsized":"2023-10-16T11:21:58.108993+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:08:49.455121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"37.9","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600027+0000","last_change":"2023-10-16T11:21:57.116051+0000","last_active":"2023-10-16T11:21:58.600027+0000","last_peered":"2023-10-16T11:21:58.600027+0000","last_clean":"2023-10-16T11:21:58.600027+0000","last_became_active":"2023-10-16T11:21:57.115776+0000","last_became_peered":"2023-10-16T11:21:57.115776+0000","last_unstale":"2023-10-16T11:21:58.600027+0000","last_undegraded":"2023-10-16T11:21:58.600027+0000","last_fullsized":"2023-10-16T11:21:58.600027+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:31:39.472479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600250+0000","last_change":"2023-10-16T11:21:56.087100+0000","last_active":"2023-10-16T11:21:58.600250+0000","last_peered":"2023-10-16T11:21:58.600250+0000","last_clean":"2023-10-16T11:21:58.600250+0000","last_became_active":"2023-10-16T11:21:56.086599+0000","last_became_peered":"2023-10-16T11:21:56.086599+0000","last_unstale":"2023-10-16T11:21:58.600250+0000","last_undegraded":"2023-10-16T11:21:58.600250+0000","last_fullsized":"2023-10-16T11:21:58.600250+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:30:47.512210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.6","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600289+0000","last_change":"2023-10-16T11:21:57.103368+0000","last_active":"2023-10-16T11:21:58.600289+0000","last_peered":"2023-10-16T11:21:58.600289+0000","last_clean":"2023-10-16T11:21:58.600289+0000","last_became_active":"2023-10-16T11:21:57.102680+0000","last_became_peered":"2023-10-16T11:21:57.102680+0000","last_unstale":"2023-10-16T11:21:58.600289+0000","last_undegraded":"2023-10-16T11:21:58.600289+0000","last_fullsized":"2023-10-16T11:21:58.600289+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:19:58.049533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.7","version":"268'2","reported_seq":14,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110210+0000","last_change":"2023-10-16T11:21:56.100889+0000","last_active":"2023-10-16T11:21:58.110210+0000","last_peered":"2023-10-16T11:21:58.110210+0000","last_clean":"2023-10-16T11:21:58.110210+0000","last_became_active":"2023-10-16T11:21:56.100600+0000","last_became_peered":"2023-10-16T11:21:56.100600+0000","last_unstale":"2023-10-16T11:21:58.110210+0000","last_undegraded":"2023-10-16T11:21:58.110210+0000","last_fullsized":"2023-10-16T11:21:58.110210+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:59.315630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.7","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601092+0000","last_change":"2023-10-16T11:21:57.117900+0000","last_active":"2023-10-16T11:21:58.601092+0000","last_peered":"2023-10-16T11:21:58.601092+0000","last_clean":"2023-10-16T11:21:58.601092+0000","last_became_active":"2023-10-16T11:21:57.117480+0000","last_became_peered":"2023-10-16T11:21:57.117480+0000","last_unstale":"2023-10-16T11:21:58.601092+0000","last_undegraded":"2023-10-16T11:21:58.601092+0000","last_fullsized":"2023-10-16T11:21:58.601092+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:07.609635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600216+0000","last_change":"2023-10-16T11:21:56.089475+0000","last_active":"2023-10-16T11:21:58.600216+0000","last_peered":"2023-10-16T11:21:58.600216+0000","last_clean":"2023-10-16T11:21:58.600216+0000","last_became_active":"2023-10-16T11:21:56.089135+0000","last_became_peered":"2023-10-16T11:21:56.089135+0000","last_unstale":"2023-10-16T11:21:58.600216+0000","last_undegraded":"2023-10-16T11:21:58.600216+0000","last_fullsized":"2023-10-16T11:21:58.600216+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:23:48.366188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"37.4","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600423+0000","last_change":"2023-10-16T11:21:57.106821+0000","last_active":"2023-10-16T11:21:58.600423+0000","last_peered":"2023-10-16T11:21:58.600423+0000","last_clean":"2023-10-16T11:21:58.600423+0000","last_became_active":"2023-10-16T11:21:57.106547+0000","last_became_peered":"2023-10-16T11:21:57.106547+0000","last_unstale":"2023-10-16T11:21:58.600423+0000","last_undegraded":"2023-10-16T11:21:58.600423+0000","last_fullsized":"2023-10-16T11:21:58.600423+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:54:24.086344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110207+0000","last_change":"2023-10-16T11:21:56.093546+0000","last_active":"2023-10-16T11:21:58.110207+0000","last_peered":"2023-10-16T11:21:58.110207+0000","last_clean":"2023-10-16T11:21:58.110207+0000","last_became_active":"2023-10-16T11:21:56.093249+0000","last_became_peered":"2023-10-16T11:21:56.093249+0000","last_unstale":"2023-10-16T11:21:58.110207+0000","last_undegraded":"2023-10-16T11:21:58.110207+0000","last_fullsized":"2023-10-16T11:21:58.110207+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:05:18.250416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.5","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110178+0000","last_change":"2023-10-16T11:21:57.102695+0000","last_active":"2023-10-16T11:21:58.110178+0000","last_peered":"2023-10-16T11:21:58.110178+0000","last_clean":"2023-10-16T11:21:58.110178+0000","last_became_active":"2023-10-16T11:21:57.102419+0000","last_became_peered":"2023-10-16T11:21:57.102419+0000","last_unstale":"2023-10-16T11:21:58.110178+0000","last_undegraded":"2023-10-16T11:21:58.110178+0000","last_fullsized":"2023-10-16T11:21:58.110178+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:59:50.894692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600849+0000","last_change":"2023-10-16T11:21:56.088856+0000","last_active":"2023-10-16T11:21:58.600849+0000","last_peered":"2023-10-16T11:21:58.600849+0000","last_clean":"2023-10-16T11:21:58.600849+0000","last_became_active":"2023-10-16T11:21:56.088495+0000","last_became_peered":"2023-10-16T11:21:56.088495+0000","last_unstale":"2023-10-16T11:21:58.600849+0000","last_undegraded":"2023-10-16T11:21:58.600849+0000","last_fullsized":"2023-10-16T11:21:58.600849+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:02:28.835213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.2","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.108977+0000","last_change":"2023-10-16T11:21:57.100999+0000","last_active":"2023-10-16T11:21:58.108977+0000","last_peered":"2023-10-16T11:21:58.108977+0000","last_clean":"2023-10-16T11:21:58.108977+0000","last_became_active":"2023-10-16T11:21:57.100031+0000","last_became_peered":"2023-10-16T11:21:57.100031+0000","last_unstale":"2023-10-16T11:21:58.108977+0000","last_undegraded":"2023-10-16T11:21:58.108977+0000","last_fullsized":"2023-10-16T11:21:58.108977+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:28:00.320179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.601005+0000","last_change":"2023-10-16T11:21:56.085086+0000","last_active":"2023-10-16T11:21:58.601005+0000","last_peered":"2023-10-16T11:21:58.601005+0000","last_clean":"2023-10-16T11:21:58.601005+0000","last_became_active":"2023-10-16T11:21:56.084749+0000","last_became_peered":"2023-10-16T11:21:56.084749+0000","last_unstale":"2023-10-16T11:21:58.601005+0000","last_undegraded":"2023-10-16T11:21:58.601005+0000","last_fullsized":"2023-10-16T11:21:58.601005+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:25:27.596861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"37.3","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110237+0000","last_change":"2023-10-16T11:21:57.108386+0000","last_active":"2023-10-16T11:21:58.110237+0000","last_peered":"2023-10-16T11:21:58.110237+0000","last_clean":"2023-10-16T11:21:58.110237+0000","last_became_active":"2023-10-16T11:21:57.106808+0000","last_became_peered":"2023-10-16T11:21:57.106808+0000","last_unstale":"2023-10-16T11:21:58.110237+0000","last_undegraded":"2023-10-16T11:21:58.110237+0000","last_fullsized":"2023-10-16T11:21:58.110237+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:51:38.635487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110015+0000","last_change":"2023-10-16T11:21:56.097915+0000","last_active":"2023-10-16T11:21:58.110015+0000","last_peered":"2023-10-16T11:21:58.110015+0000","last_clean":"2023-10-16T11:21:58.110015+0000","last_became_active":"2023-10-16T11:21:56.097197+0000","last_became_peered":"2023-10-16T11:21:56.097197+0000","last_unstale":"2023-10-16T11:21:58.110015+0000","last_undegraded":"2023-10-16T11:21:58.110015+0000","last_fullsized":"2023-10-16T11:21:58.110015+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:50.207208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.0","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600331+0000","last_change":"2023-10-16T11:21:57.106260+0000","last_active":"2023-10-16T11:21:58.600331+0000","last_peered":"2023-10-16T11:21:58.600331+0000","last_clean":"2023-10-16T11:21:58.600331+0000","last_became_active":"2023-10-16T11:21:57.105987+0000","last_became_peered":"2023-10-16T11:21:57.105987+0000","last_unstale":"2023-10-16T11:21:58.600331+0000","last_undegraded":"2023-10-16T11:21:58.600331+0000","last_fullsized":"2023-10-16T11:21:58.600331+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:56.298539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.600078+0000","last_change":"2023-10-16T11:21:56.084734+0000","last_active":"2023-10-16T11:21:58.600078+0000","last_peered":"2023-10-16T11:21:58.600078+0000","last_clean":"2023-10-16T11:21:58.600078+0000","last_became_active":"2023-10-16T11:21:56.084451+0000","last_became_peered":"2023-10-16T11:21:56.084451+0000","last_unstale":"2023-10-16T11:21:58.600078+0000","last_undegraded":"2023-10-16T11:21:58.600078+0000","last_fullsized":"2023-10-16T11:21:58.600078+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:50:54.257628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"37.1","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110356+0000","last_change":"2023-10-16T11:21:57.108052+0000","last_active":"2023-10-16T11:21:58.110356+0000","last_peered":"2023-10-16T11:21:58.110356+0000","last_clean":"2023-10-16T11:21:58.110356+0000","last_became_active":"2023-10-16T11:21:57.107734+0000","last_became_peered":"2023-10-16T11:21:57.107734+0000","last_unstale":"2023-10-16T11:21:58.110356+0000","last_undegraded":"2023-10-16T11:21:58.110356+0000","last_fullsized":"2023-10-16T11:21:58.110356+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:21.731838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":12,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110532+0000","last_change":"2023-10-16T11:21:56.096004+0000","last_active":"2023-10-16T11:21:58.110532+0000","last_peered":"2023-10-16T11:21:58.110532+0000","last_clean":"2023-10-16T11:21:58.110532+0000","last_became_active":"2023-10-16T11:21:56.095163+0000","last_became_peered":"2023-10-16T11:21:56.095163+0000","last_unstale":"2023-10-16T11:21:58.110532+0000","last_undegraded":"2023-10-16T11:21:58.110532+0000","last_fullsized":"2023-10-16T11:21:58.110532+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":266,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:55.066127+0000","last_clean_scrub_stamp":"2023-10-16T11:21:55.066127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:16.267840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"37.24","version":"0'0","reported_seq":11,"reported_epoch":269,"state":"active+clean","last_fresh":"2023-10-16T11:21:58.110397+0000","last_change":"2023-10-16T11:21:57.101165+0000","last_active":"2023-10-16T11:21:58.110397+0000","last_peered":"2023-10-16T11:21:58.110397+0000","last_clean":"2023-10-16T11:21:58.110397+0000","last_became_active":"2023-10-16T11:21:57.100790+0000","last_became_peered":"2023-10-16T11:21:57.100790+0000","last_unstale":"2023-10-16T11:21:58.110397+0000","last_undegraded":"2023-10-16T11:21:58.110397+0000","last_fullsized":"2023-10-16T11:21:58.110397+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":267,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T11:21:56.070676+0000","last_clean_scrub_stamp":"2023-10-16T11:21:56.070676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:55:07.811835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"259'403","reported_seq":1455,"reported_epoch":268,"state":"active+clean","last_fresh":"2023-10-16T11:21:57.095189+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:21:57.095189+0000","last_peered":"2023-10-16T11:21:57.095189+0000","last_clean":"2023-10-16T11:21:57.095189+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:21:57.095189+0000","last_undegraded":"2023-10-16T11:21:57.095189+0000","last_fullsized":"2023-10-16T11:21:57.095189+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":679,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":37,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":36,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":679,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":403,"ondisk_log_size":403,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57280,"kb_used_data":3888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314560,"statfs":{"total":96636764160,"available":96578109440,"internally_reserved":0,"allocated":3981312,"data_stored":2350778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.878,"15min":0.878},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27800000000000002},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.5}]},{"osd":2,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55100000000000005,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.31900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69799999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61599999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":1,"15min":1.002},"last":0.67600000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68400000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60399999999999998,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.55200000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84299999999999997,"5min":1.052,"15min":1.052},"last":0.69799999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65400000000000003}]}]},{"osd":6,"up_from":14,"seq":60129543304,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60868,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310972,"statfs":{"total":96636764160,"available":96574435328,"internally_reserved":0,"allocated":3399680,"data_stored":1762022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54500000000000004,"15min":0.58199999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.84499999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.622}]},{"osd":1,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.56000000000000005,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.63400000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.436},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.57699999999999996,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.68500000000000005,"5min":0.77200000000000002,"15min":0.86499999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.57299999999999995,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59499999999999997,"15min":0.60499999999999998},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.39100000000000001},"max":{"1min":0.66300000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60299999999999998,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78800000000000003,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51800000000000002,"15min":0.499},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.69499999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.28100000000000003},"max":{"1min":0.747,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.496,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.73799999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59799999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.32100000000000001},"max":{"1min":0.81399999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.746},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68999999999999995}]}]},{"osd":1,"up_from":14,"seq":60129543304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60876,"kb_used_data":3328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310964,"statfs":{"total":96636764160,"available":96574427136,"internally_reserved":0,"allocated":3407872,"data_stored":1767318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.39700000000000002},"min":{"1min":0.23499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.377,"15min":0.41799999999999998},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.44400000000000001,"5min":0.62,"15min":0.66900000000000004},"last":0.48699999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45200000000000001,"5min":0.56999999999999995,"15min":0.61799999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.379,"5min":0.40799999999999997,"15min":0.44900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.497,"5min":0.69699999999999995,"15min":0.755},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67300000000000004,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.52800000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":0.92600000000000005,"15min":1.097},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76600000000000001,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":1.147},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.0509999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.0600000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.68200000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543305,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61452,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310388,"statfs":{"total":96636764160,"available":96573837312,"internally_reserved":0,"allocated":3989504,"data_stored":2350870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36899999999999999,"15min":0.38200000000000001},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.214},"max":{"1min":0.46200000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38200000000000001,"15min":0.40699999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.314}]},{"osd":2,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39700000000000002,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.67300000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.439},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55200000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41699999999999998,"15min":0.443},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47199999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84399999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.998},"last":0.70799999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":1.008},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.627,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":1.034},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.65400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63300000000000001,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.95999999999999996,"15min":1.0449999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.67600000000000005},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65400000000000003}]}]},{"osd":3,"up_from":14,"seq":60129543305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48384,"kb_used_data":3312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323456,"statfs":{"total":96636764160,"available":96587218944,"internally_reserved":0,"allocated":3391488,"data_stored":1760992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40200000000000002,"15min":0.41099999999999998},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.52600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.372},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.41799999999999998,"15min":0.44400000000000001},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.40899999999999997,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.38800000000000001,"15min":0.41499999999999998},"min":{"1min":0.24299999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.42899999999999999,"5min":0.87,"15min":0.87},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40699999999999997,"15min":0.441},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.41199999999999998,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.29699999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.42899999999999999,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.48099999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.44800000000000001,"15min":0.47499999999999998},"min":{"1min":0.33600000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.39900000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60399999999999998,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.73699999999999999,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71299999999999997,"5min":1.0129999999999999,"15min":1.1140000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.626,"15min":0.66200000000000003},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.63300000000000001,"15min":0.66400000000000003},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.98799999999999999,"15min":1.0649999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61199999999999999,"15min":0.64900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79100000000000004,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.53500000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.623,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.377,"15min":0.377},"max":{"1min":0.73499999999999999,"5min":0.90900000000000003,"15min":1.0229999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63700000000000001,"15min":0.67700000000000005},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.52400000000000002}]}]},{"osd":2,"up_from":14,"seq":60129543304,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48392,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323448,"statfs":{"total":96636764160,"available":96587210752,"internally_reserved":0,"allocated":3395584,"data_stored":1760534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.433,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.57299999999999995,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.504}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.436,"15min":0.441},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.441,"15min":0.45800000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.44800000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.163,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.434},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69899999999999995,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":3.593,"15min":3.593},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.64700000000000002,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73499999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67700000000000005,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67500000000000004,"15min":0.68999999999999995},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.73599999999999999,"5min":3.633,"15min":3.633},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.67200000000000004,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.70799999999999996,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66300000000000003,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.70699999999999996,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.61199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129543304,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96900,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94274940,"statfs":{"total":96636764160,"available":96537538560,"internally_reserved":0,"allocated":3395584,"data_stored":1761062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.60299999999999998},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.62,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64600000000000002,"5min":0.77100000000000002,"15min":0.84399999999999997},"last":0.79000000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58699999999999997,"15min":0.61299999999999999},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68300000000000005,"5min":0.82899999999999996,"15min":0.89100000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68400000000000005,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66000000000000003,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.68799999999999994,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.57699999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.65200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.64200000000000002},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.36299999999999999},"max":{"1min":0.70099999999999996,"5min":0.81299999999999994,"15min":0.879},"last":0.47799999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.72399999999999998,"5min":0.81299999999999994,"15min":0.84099999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.77300000000000002,"15min":0.80200000000000005},"last":0.71999999999999997}]},{"osd":6,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.29399999999999998},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.91300000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.74099999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.875,"15min":0.875},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.89200000000000002,"15min":0.95299999999999996},"last":0.69999999999999996}]}]},{"osd":5,"up_from":14,"seq":60129543305,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48412,"kb_used_data":3336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323428,"statfs":{"total":96636764160,"available":96587190272,"internally_reserved":0,"allocated":3416064,"data_stored":1768964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57999999999999996,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.84099999999999997,"5min":1.873,"15min":1.873},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59099999999999997,"15min":0.60399999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.84299999999999997,"5min":1.024,"15min":1.024},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88700000000000001,"5min":1.821,"15min":1.821},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63600000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.62},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.54300000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.71899999999999997,"5min":0.878,"15min":0.878},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72099999999999997}]},{"osd":7,"last update":"Mon Oct 16 11:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80000000000000004,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.748},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86099999999999999,"5min":2.02,"15min":2.02},"last":0.70599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":37,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:01.604 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T11:22:01.605 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-16T11:22:01.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.764+0000 7f53a8b9f700 1 Processor -- start 2023-10-16T11:22:01.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.764+0000 7f53a8b9f700 1 -- start start 2023-10-16T11:22:01.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a8b9f700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53a415da70 0x7f53a40ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a8b9f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a40ce6a0 0x7f53a406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a8b9f700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a40cef00 con 0x7f53a4072760 2023-10-16T11:22:01.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a8b9f700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a40cec20 con 0x7f53a415da70 2023-10-16T11:22:01.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a8b9f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4071c80 con 0x7f53a40ce6a0 2023-10-16T11:22:01.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a1d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a40ce6a0 0x7f53a406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a1d9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a40ce6a0 0x7f53a406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:58428/0 (socket says 172.21.15.181:58428) 2023-10-16T11:22:01.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a259c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53a415da70 0x7f53a40ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a1d9b700 1 -- 172.21.15.181:0/1932052589 learned_addr learned my addr 172.21.15.181:0/1932052589 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:01.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a1d9b700 1 -- 172.21.15.181:0/1932052589 >> v2:172.21.15.190:3300/0 conn(0x7f53a415da70 msgr2=0x7f53a40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:01.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a1d9b700 1 --2- 172.21.15.181:0/1932052589 >> v2:172.21.15.190:3300/0 conn(0x7f53a415da70 0x7f53a40ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a1d9b700 1 -- 172.21.15.181:0/1932052589 >> v1:172.21.15.190:6789/0 conn(0x7f53a4072760 legacy=0x7f53a41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.765+0000 7f53a1d9b700 1 -- 172.21.15.181:0/1932052589 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a4071f60 con 0x7f53a40ce6a0 2023-10-16T11:22:01.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a1d9b700 1 --2- 172.21.15.181:0/1932052589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a40ce6a0 0x7f53a406a8a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f53940098f0 tx=0x7f5394009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db08704c21eb0cc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a159a700 1 -- 172.21.15.181:0/1932052589 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394011830 con 0x7f53a40ce6a0 2023-10-16T11:22:01.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a159a700 1 -- 172.21.15.181:0/1932052589 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5394011990 con 0x7f53a40ce6a0 2023-10-16T11:22:01.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a159a700 1 -- 172.21.15.181:0/1932052589 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394011d20 con 0x7f53a40ce6a0 2023-10-16T11:22:01.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 -- 172.21.15.181:0/1932052589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a40ce6a0 msgr2=0x7f53a406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/1932052589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a40ce6a0 0x7f53a406a8a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f53940098f0 tx=0x7f5394009c00 comp rx=0 tx=0).stop 2023-10-16T11:22:01.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 -- 172.21.15.181:0/1932052589 shutdown_connections 2023-10-16T11:22:01.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/1932052589 >> v2:172.21.15.190:3300/0 conn(0x7f53a415da70 0x7f53a40ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/1932052589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a40ce6a0 0x7f53a406a8a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 -- 172.21.15.181:0/1932052589 >> 172.21.15.181:0/1932052589 conn(0x7f53a40cd010 msgr2=0x7f53a40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 -- 172.21.15.181:0/1932052589 shutdown_connections 2023-10-16T11:22:01.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.766+0000 7f53a8b9f700 1 -- 172.21.15.181:0/1932052589 wait complete. 2023-10-16T11:22:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.768+0000 7f53a8b9f700 1 Processor -- start 2023-10-16T11:22:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.768+0000 7f53a8b9f700 1 -- start start 2023-10-16T11:22:01.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53a4072760 0x7f53a41f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a40ce6a0 0x7f53a40ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a415da70 0x7f53a40adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a415fb10 con 0x7f53a40ce6a0 2023-10-16T11:22:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a415f830 con 0x7f53a4072760 2023-10-16T11:22:01.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40b3250 con 0x7f53a415da70 2023-10-16T11:22:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a40ce6a0 0x7f53a40ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a40ce6a0 0x7f53a40ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33178/0 (socket says 172.21.15.181:33178) 2023-10-16T11:22:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 -- 172.21.15.181:0/924746406 learned_addr learned my addr 172.21.15.181:0/924746406 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:01.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a2d9d700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a415da70 0x7f53a40adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a259c700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53a4072760 0x7f53a41f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 -- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a415da70 msgr2=0x7f53a40adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a415da70 0x7f53a40adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 -- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53a4072760 msgr2=0x7f53a41f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:01.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53a4072760 0x7f53a41f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:01.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 -- 172.21.15.181:0/924746406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5394009550 con 0x7f53a40ce6a0 2023-10-16T11:22:01.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a2d9d700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a415da70 0x7f53a40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:01.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a1d9b700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a40ce6a0 0x7f53a40ad6e0 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7f5394000c00 tx=0x7f5394004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f5392ffd700 1 -- 172.21.15.181:0/924746406 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394004570 con 0x7f53a40ce6a0 2023-10-16T11:22:01.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f5392ffd700 1 -- 172.21.15.181:0/924746406 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539401f070 con 0x7f53a40ce6a0 2023-10-16T11:22:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f5392ffd700 1 -- 172.21.15.181:0/924746406 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53940038c0 con 0x7f53a40ce6a0 2023-10-16T11:22:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a40b3530 con 0x7f53a40ce6a0 2023-10-16T11:22:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.769+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a40ae160 con 0x7f53a40ce6a0 2023-10-16T11:22:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.770+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a40b3a50 con 0x7f53a40ce6a0 2023-10-16T11:22:01.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.771+0000 7f5392ffd700 1 -- 172.21.15.181:0/924746406 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5394021030 con 0x7f53a40ce6a0 2023-10-16T11:22:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.771+0000 7f5392ffd700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f538406b020 0x7f538406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.771+0000 7f5392ffd700 1 -- 172.21.15.181:0/924746406 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f539408adf0 con 0x7f53a40ce6a0 2023-10-16T11:22:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.771+0000 7f53a259c700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f538406b020 0x7f538406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:01.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.772+0000 7f53a259c700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f538406b020 0x7f538406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5398009b70 tx=0x7f5398008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:01.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:01.776+0000 7f5392ffd700 1 -- 172.21.15.181:0/924746406 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f53a40b3a50 con 0x7f53a40ce6a0 2023-10-16T11:22:02.048 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.046+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f53a40b3a50 con 0x7f53a40ce6a0 2023-10-16T11:22:02.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.049+0000 7f5392ffd700 1 -- 172.21.15.181:0/924746406 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v55) v1 ==== 190+0+147 (secure 0 0 0) 0x7f53a40b3a50 con 0x7f53a40ce6a0 2023-10-16T11:22:02.051 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-16T11:22:02.051 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCuBS1lHSkSIRAApWv71PwIgyrGg3tCkA1/4w== 2023-10-16T11:22:02.051 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2023-10-16T11:22:02.052 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2023-10-16T11:22:02.052 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-16T11:22:02.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f538406b020 msgr2=0x7f538406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f538406b020 0x7f538406d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5398009b70 tx=0x7f5398008040 comp rx=0 tx=0).stop 2023-10-16T11:22:02.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a40ce6a0 msgr2=0x7f53a40ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a40ce6a0 0x7f53a40ad6e0 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7f5394000c00 tx=0x7f5394004750 comp rx=0 tx=0).stop 2023-10-16T11:22:02.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 shutdown_connections 2023-10-16T11:22:02.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f538406b020 0x7f538406d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53a415da70 0x7f53a40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53a4072760 0x7f53a41f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 --2- 172.21.15.181:0/924746406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53a40ce6a0 0x7f53a40ad6e0 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 >> 172.21.15.181:0/924746406 conn(0x7f53a40cd010 msgr2=0x7f53a40cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:02.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 shutdown_connections 2023-10-16T11:22:02.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.051+0000 7f53a8b9f700 1 -- 172.21.15.181:0/924746406 wait complete. 2023-10-16T11:22:02.056 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2023-10-16T11:22:02.065 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:02.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe6e13700 1 Processor -- start 2023-10-16T11:22:02.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe6e13700 1 -- start start 2023-10-16T11:22:02.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe6e13700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfe00af450 0x7fbfe00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe6e13700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfe00cd470 0x7fbfe00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe6e13700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe006f870 con 0x7fbfe015dfd0 2023-10-16T11:22:02.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe6e13700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00d1160 con 0x7fbfe00cd470 2023-10-16T11:22:02.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe6e13700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00cc8e0 con 0x7fbfe00af450 2023-10-16T11:22:02.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.186+0000 7fbfe53b0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfe00cd470 0x7fbfe00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfe53b0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfe00cd470 0x7fbfe00cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33188/0 (socket says 172.21.15.181:33188) 2023-10-16T11:22:02.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfe53b0700 1 -- 172.21.15.181:0/1828445002 learned_addr learned my addr 172.21.15.181:0/1828445002 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:02.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfdffff700 1 --2- 172.21.15.181:0/1828445002 >> v2:172.21.15.190:3300/0 conn(0x7fbfe00af450 0x7fbfe00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfe53b0700 1 -- 172.21.15.181:0/1828445002 >> v2:172.21.15.190:3300/0 conn(0x7fbfe00af450 msgr2=0x7fbfe00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:02.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfe53b0700 1 --2- 172.21.15.181:0/1828445002 >> v2:172.21.15.190:3300/0 conn(0x7fbfe00af450 0x7fbfe00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfe53b0700 1 -- 172.21.15.181:0/1828445002 >> v1:172.21.15.190:6789/0 conn(0x7fbfe015dfd0 legacy=0x7fbfe006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:02.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfe53b0700 1 -- 172.21.15.181:0/1828445002 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00ccbc0 con 0x7fbfe00cd470 2023-10-16T11:22:02.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfe53b0700 1 --2- 172.21.15.181:0/1828445002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe00cd470 0x7fbfe00cd920 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b3070 tx=0x7fbfd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a47f39d6a786dd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:02.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfdf7fe700 1 -- 172.21.15.181:0/1828445002 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800e070 con 0x7fbfe00cd470 2023-10-16T11:22:02.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfdf7fe700 1 -- 172.21.15.181:0/1828445002 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8003a20 con 0x7fbfe00cd470 2023-10-16T11:22:02.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.187+0000 7fbfdf7fe700 1 -- 172.21.15.181:0/1828445002 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8003dd0 con 0x7fbfe00cd470 2023-10-16T11:22:02.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 -- 172.21.15.181:0/1828445002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe00cd470 msgr2=0x7fbfe00cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/1828445002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe00cd470 0x7fbfe00cd920 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b3070 tx=0x7fbfd800b330 comp rx=0 tx=0).stop 2023-10-16T11:22:02.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 -- 172.21.15.181:0/1828445002 shutdown_connections 2023-10-16T11:22:02.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/1828445002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe00cd470 0x7fbfe00cd920 unknown :-1 s=CLOSED pgs=3141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/1828445002 >> v2:172.21.15.190:3300/0 conn(0x7fbfe00af450 0x7fbfe00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 -- 172.21.15.181:0/1828445002 >> 172.21.15.181:0/1828445002 conn(0x7fbfe00d04a0 msgr2=0x7fbfe00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:02.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 -- 172.21.15.181:0/1828445002 shutdown_connections 2023-10-16T11:22:02.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.188+0000 7fbfe6e13700 1 -- 172.21.15.181:0/1828445002 wait complete. 2023-10-16T11:22:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.189+0000 7fbfe6e13700 1 Processor -- start 2023-10-16T11:22:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.189+0000 7fbfe6e13700 1 -- start start 2023-10-16T11:22:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe6e13700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe00af450 0x7fbfe00b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe6e13700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfe00cd470 0x7fbfe00c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe6e13700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe015dfd0 0x7fbfe00c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe6e13700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00c62a0 con 0x7fbfe015dfd0 2023-10-16T11:22:02.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe6e13700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00c4170 con 0x7fbfe00af450 2023-10-16T11:22:02.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe6e13700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00c4450 con 0x7fbfe00cd470 2023-10-16T11:22:02.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe015dfd0 0x7fbfe00c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe015dfd0 0x7fbfe00c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33194/0 (socket says 172.21.15.181:33194) 2023-10-16T11:22:02.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 -- 172.21.15.181:0/2783256816 learned_addr learned my addr 172.21.15.181:0/2783256816 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:02.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe4baf700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe00af450 0x7fbfe00b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 -- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfe00cd470 msgr2=0x7fbfe00c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfdffff700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfe00cd470 0x7fbfe00c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfe00cd470 0x7fbfe00c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 -- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe00af450 msgr2=0x7fbfe00b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe00af450 0x7fbfe00b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 -- 172.21.15.181:0/2783256816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd8009b30 con 0x7fbfe015dfd0 2023-10-16T11:22:02.205 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe4baf700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe00af450 0x7fbfe00b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:02.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfdffff700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfe00cd470 0x7fbfe00c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:02.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.190+0000 7fbfe53b0700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe015dfd0 0x7fbfe00c5d60 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b370 tx=0x7fbfd8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:02.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.191+0000 7fbfdd7fa700 1 -- 172.21.15.181:0/2783256816 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8004440 con 0x7fbfe015dfd0 2023-10-16T11:22:02.206 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.191+0000 7fbfdd7fa700 1 -- 172.21.15.181:0/2783256816 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd80045a0 con 0x7fbfe015dfd0 2023-10-16T11:22:02.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.191+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe00c4730 con 0x7fbfe015dfd0 2023-10-16T11:22:02.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.191+0000 7fbfdd7fa700 1 -- 172.21.15.181:0/2783256816 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8004950 con 0x7fbfe015dfd0 2023-10-16T11:22:02.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.191+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe00c4cf0 con 0x7fbfe015dfd0 2023-10-16T11:22:02.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.192+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0067d20 con 0x7fbfe015dfd0 2023-10-16T11:22:02.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.192+0000 7fbfdd7fa700 1 -- 172.21.15.181:0/2783256816 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbfd8019030 con 0x7fbfe015dfd0 2023-10-16T11:22:02.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.192+0000 7fbfdd7fa700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fbfc406b020 0x7fbfc406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.192+0000 7fbfdd7fa700 1 -- 172.21.15.181:0/2783256816 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fbfd808acd0 con 0x7fbfe015dfd0 2023-10-16T11:22:02.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.192+0000 7fbfe4baf700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fbfc406b020 0x7fbfc406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.193+0000 7fbfe4baf700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fbfc406b020 0x7fbfc406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400a8b0 tx=0x7fbfd4008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:02.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.197+0000 7fbfdd7fa700 1 -- 172.21.15.181:0/2783256816 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fbfd8059450 con 0x7fbfe015dfd0 2023-10-16T11:22:02.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.432+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfe006d920 con 0x7fbfe015dfd0 2023-10-16T11:22:02.435 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.433+0000 7fbfdd7fa700 1 -- 172.21.15.181:0/2783256816 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+4932 (secure 0 0 0) 0x7fbfd8017090 con 0x7fbfe015dfd0 2023-10-16T11:22:02.435 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:02.435 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":153,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/984505218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":984505218},{"type":"v1","addr":"172.21.15.190:6835","nonce":984505218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/350629849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":350629849},{"type":"v1","addr":"172.21.15.181:6835","nonce":350629849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13440,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1354411415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1354411415},{"type":"v1","addr":"172.21.15.181:6837","nonce":1354411415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:56.485392+0000","modified":"2023-10-16T11:21:59.515101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13452":{"gid":13452,"name":"b","rank":0,"incarnation":149,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/284709222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":284709222},{"type":"v1","addr":"172.21.15.190:6834","nonce":284709222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:22:02.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fbfc406b020 msgr2=0x7fbfc406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fbfc406b020 0x7fbfc406d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400a8b0 tx=0x7fbfd4008040 comp rx=0 tx=0).stop 2023-10-16T11:22:02.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe015dfd0 msgr2=0x7fbfe00c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe015dfd0 0x7fbfe00c5d60 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b370 tx=0x7fbfd8004ad0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 shutdown_connections 2023-10-16T11:22:02.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe015dfd0 0x7fbfe00c5d60 unknown :-1 s=CLOSED pgs=3142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfe00af450 0x7fbfe00b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fbfc406b020 0x7fbfc406d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 --2- 172.21.15.181:0/2783256816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfe00cd470 0x7fbfe00c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 >> 172.21.15.181:0/2783256816 conn(0x7fbfe00d04a0 msgr2=0x7fbfe0073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:02.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 shutdown_connections 2023-10-16T11:22:02.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.435+0000 7fbfe6e13700 1 -- 172.21.15.181:0/2783256816 wait complete. 2023-10-16T11:22:02.440 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 153 2023-10-16T11:22:02.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 153, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T11:21:56.485392+0000', 'modified': '2023-10-16T11:21:59.515101+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 13452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_13452': {'gid': 13452, 'name': 'b', 'rank': 0, 'incarnation': 149, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6834/284709222', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 284709222}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 284709222}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [37], 'metadata_pool': 36, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T11:22:02.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T11:22:02.853 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T11:22:02.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.975+0000 7f44a47df700 1 Processor -- start 2023-10-16T11:22:02.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.975+0000 7f44a47df700 1 -- start start 2023-10-16T11:22:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.976+0000 7f44a47df700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f449c15df60 0x7f449c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.976+0000 7f44a47df700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c0cddf0 con 0x7f449c0d0fc0 2023-10-16T11:22:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.976+0000 7f44a47df700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c0ccf50 con 0x7f449c06a8a0 2023-10-16T11:22:02.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.976+0000 7f44a47df700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449c0af9c0 con 0x7f449c15df60 2023-10-16T11:22:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1d7a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f449c0d0fc0 0x7f449c0cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44276/0 (socket says 172.21.15.181:44276) 2023-10-16T11:22:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1d7a700 1 -- 172.21.15.181:0/3490321249 learned_addr learned my addr 172.21.15.181:0/3490321249 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a257b700 1 --2- 172.21.15.181:0/3490321249 >> v2:172.21.15.190:3300/0 conn(0x7f449c15df60 0x7f449c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801962965 0 0) 0x7f449c0cddf0 con 0x7f449c0d0fc0 2023-10-16T11:22:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44840035b0 con 0x7f449c0d0fc0 2023-10-16T11:22:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3554358568 0 0) 0x7f44840035b0 con 0x7f449c0d0fc0 2023-10-16T11:22:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 >> v2:172.21.15.190:3300/0 conn(0x7f449c15df60 msgr2=0x7f449c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:02.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 --2- 172.21.15.181:0/3490321249 >> v2:172.21.15.190:3300/0 conn(0x7f449c15df60 0x7f449c0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 >> v1:172.21.15.190:6789/0 conn(0x7f449c06a8a0 legacy=0x7f449c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c0cc600 con 0x7f449c0d0fc0 2023-10-16T11:22:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4490001670 con 0x7f449c0d0fc0 2023-10-16T11:22:02.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 >> v1:172.21.15.181:6789/0 conn(0x7f449c0d0fc0 legacy=0x7f449c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484003a00 0x7f4484003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4484004850 0x7f4484004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4484005210 0x7f448400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448400c080 con 0x7f4484005210 2023-10-16T11:22:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448400c1f0 con 0x7f4484003a00 2023-10-16T11:22:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448400c3b0 con 0x7f4484004850 2023-10-16T11:22:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4484005210 0x7f448400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a257b700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484003a00 0x7f4484003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a1d7a700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4484004850 0x7f4484004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a2d7c700 1 -- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4484004850 msgr2=0x7f4484004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4484004850 0x7f4484004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a2d7c700 1 -- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484003a00 msgr2=0x7f4484003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:02.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484003a00 0x7f4484003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.977+0000 7f44a2d7c700 1 -- 172.21.15.181:0/3490321249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840035b0 con 0x7f4484005210 2023-10-16T11:22:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a2d7c700 1 -- 172.21.15.181:0/3490321249 reap_dead start 2023-10-16T11:22:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4484005210 0x7f448400ba10 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7f448c0017c0 tx=0x7f4484005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67d7589f36d66dd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c0052d0 con 0x7f4484005210 2023-10-16T11:22:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c005430 con 0x7f4484005210 2023-10-16T11:22:02.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a1579700 1 -- 172.21.15.181:0/3490321249 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c0038c0 con 0x7f4484005210 2023-10-16T11:22:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a47df700 1 -- 172.21.15.181:0/3490321249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4484005210 msgr2=0x7f448400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a47df700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4484005210 0x7f448400ba10 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7f448c0017c0 tx=0x7f4484005880 comp rx=0 tx=0).stop 2023-10-16T11:22:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a47df700 1 -- 172.21.15.181:0/3490321249 shutdown_connections 2023-10-16T11:22:02.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a47df700 1 --2- 172.21.15.181:0/3490321249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4484005210 0x7f448400ba10 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a47df700 1 -- 172.21.15.181:0/3490321249 >> 172.21.15.181:0/3490321249 conn(0x7f449c0d04a0 msgr2=0x7f449c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.978+0000 7f44a47df700 1 -- 172.21.15.181:0/3490321249 shutdown_connections 2023-10-16T11:22:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.979+0000 7f44a47df700 1 -- 172.21.15.181:0/3490321249 wait complete. 2023-10-16T11:22:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 Processor -- start 2023-10-16T11:22:02.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 -- start start 2023-10-16T11:22:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484005210 0x7f449c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f449c0c6290 0x7f449c159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f449c0c6710 0x7f449c15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c159d30 con 0x7f449c0c6710 2023-10-16T11:22:02.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c06f1c0 con 0x7f4484005210 2023-10-16T11:22:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.980+0000 7f44a47df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c06f4a0 con 0x7f449c0c6290 2023-10-16T11:22:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f449c0c6710 0x7f449c15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f449c0c6710 0x7f449c15a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33214/0 (socket says 172.21.15.181:33214) 2023-10-16T11:22:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4106891313 learned_addr learned my addr 172.21.15.181:0/4106891313 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:02.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a257b700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484005210 0x7f449c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a1d7a700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f449c0c6290 0x7f449c159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:02.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f449c0c6290 msgr2=0x7f449c159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f449c0c6290 0x7f449c159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484005210 msgr2=0x7f449c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:02.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484005210 0x7f449c1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:02.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4106891313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c06f780 con 0x7f449c0c6710 2023-10-16T11:22:02.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a257b700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484005210 0x7f449c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:02.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f449c0c6710 0x7f449c15a4a0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f449c071410 tx=0x7f448c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:02.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f449affd700 1 -- 172.21.15.181:0/4106891313 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c020070 con 0x7f449c0c6710 2023-10-16T11:22:02.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f449affd700 1 -- 172.21.15.181:0/4106891313 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c001290 con 0x7f449c0c6710 2023-10-16T11:22:02.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c0c5a90 con 0x7f449c0c6710 2023-10-16T11:22:02.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f449affd700 1 -- 172.21.15.181:0/4106891313 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c01c420 con 0x7f449c0c6710 2023-10-16T11:22:02.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.981+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c0b5d90 con 0x7f449c0c6710 2023-10-16T11:22:03.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.983+0000 7f449affd700 1 -- 172.21.15.181:0/4106891313 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f448c01c580 con 0x7f449c0c6710 2023-10-16T11:22:03.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.983+0000 7f449affd700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f448806b020 0x7f448806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.983+0000 7f449affd700 1 -- 172.21.15.181:0/4106891313 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f448c0892a0 con 0x7f449c0c6710 2023-10-16T11:22:03.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.983+0000 7f44a257b700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f448806b020 0x7f448806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.983+0000 7f44a257b700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f448806b020 0x7f448806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4494007520 tx=0x7f449400b040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.985+0000 7f44b09c9b80 1 Processor -- start 2023-10-16T11:22:03.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.985+0000 7f44b09c9b80 1 -- start start 2023-10-16T11:22:03.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20bf220 0x557ab20c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20c4630 0x557ab20c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab209a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab2010c20 con 0x557ab20bf220 2023-10-16T11:22:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab1ff5c10 con 0x557ab20c4630 2023-10-16T11:22:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ab1ff3530 con 0x557ab209a010 2023-10-16T11:22:03.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20bf220 0x557ab20c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20bf220 0x557ab20c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33218/0 (socket says 172.21.15.181:33218) 2023-10-16T11:22:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 -- 172.21.15.181:0/2290851609 learned_addr learned my addr 172.21.15.181:0/2290851609 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a1d7a700 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20c4630 0x557ab20c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 -- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 msgr2=0x557ab209a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab209a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab209a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 -- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20c4630 msgr2=0x557ab20c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20c4630 0x557ab20c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 -- 172.21.15.181:0/2290851609 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ab1fac3a0 con 0x557ab20bf220 2023-10-16T11:22:03.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a1d7a700 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20c4630 0x557ab20c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.986+0000 7f44a257b700 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20bf220 0x557ab20c6a70 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f4494016430 tx=0x7f4494016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f140e5c7725b4bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f4483fff700 1 -- 172.21.15.181:0/2290851609 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400e040 con 0x557ab20bf220 2023-10-16T11:22:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f4483fff700 1 -- 172.21.15.181:0/2290851609 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449400f040 con 0x557ab20bf220 2023-10-16T11:22:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f4483fff700 1 -- 172.21.15.181:0/2290851609 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494023640 con 0x557ab20bf220 2023-10-16T11:22:03.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 -- 172.21.15.181:0/2290851609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20bf220 msgr2=0x557ab20c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20bf220 0x557ab20c6a70 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f4494016430 tx=0x7f4494016000 comp rx=0 tx=0).stop 2023-10-16T11:22:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 -- 172.21.15.181:0/2290851609 shutdown_connections 2023-10-16T11:22:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab209a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20bf220 0x557ab20c6a70 unknown :-1 s=CLOSED pgs=3146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/2290851609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20c4630 0x557ab20c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 -- 172.21.15.181:0/2290851609 >> 172.21.15.181:0/2290851609 conn(0x557ab1fc9d80 msgr2=0x557ab20e4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 -- 172.21.15.181:0/2290851609 shutdown_connections 2023-10-16T11:22:03.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.987+0000 7f44b09c9b80 1 -- 172.21.15.181:0/2290851609 wait complete. 2023-10-16T11:22:03.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.988+0000 7f44b09c9b80 1 -- start start 2023-10-16T11:22:03.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.988+0000 7f44b09c9b80 1 Processor -- start 2023-10-16T11:22:03.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab20c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab20c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab209c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab1feee20 con 0x557ab20c4630 2023-10-16T11:22:03.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab1f8b6c0 con 0x557ab209a010 2023-10-16T11:22:03.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ab1f8b360 con 0x557ab20bf220 2023-10-16T11:22:03.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44b09c9b80 10 client.0 _ll_register_callbacks cb 0x557ab1de87b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:22:03.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab209c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab209c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33228/0 (socket says 172.21.15.181:33228) 2023-10-16T11:22:03.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 -- 172.21.15.181:0/1306195424 learned_addr learned my addr 172.21.15.181:0/1306195424 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:03.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a257b700 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab20c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a1d7a700 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab20c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 -- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 msgr2=0x557ab20c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab20c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 -- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 msgr2=0x557ab20c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab20c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 -- 172.21.15.181:0/1306195424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4494007d10 con 0x557ab20c4630 2023-10-16T11:22:03.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab209c9b0 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7f448c0017c0 tx=0x7f448c057740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a78b9f3ce1037d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.989+0000 7f44817fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:22:03.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44817fa700 1 -- 172.21.15.181:0/1306195424 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c06f070 con 0x557ab20c4630 2023-10-16T11:22:03.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44817fa700 1 -- 172.21.15.181:0/1306195424 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f448c057d60 con 0x557ab20c4630 2023-10-16T11:22:03.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ab208f5b0 con 0x557ab20c4630 2023-10-16T11:22:03.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44817fa700 1 -- 172.21.15.181:0/1306195424 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448c05b030 con 0x557ab20c4630 2023-10-16T11:22:03.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 10 client.13587 fetch_fsmap learned FSMap version 153 2023-10-16T11:22:03.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=153,monmap=2+}) v3 -- 0x557ab1e19d90 con 0x557ab20c4630 2023-10-16T11:22:03.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44817fa700 1 -- 172.21.15.181:0/1306195424 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ab208f5b0 con 0x557ab20c4630 2023-10-16T11:22:03.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44817fa700 1 -- 172.21.15.181:0/1306195424 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 153) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f448c01a2e0 con 0x557ab20c4630 2023-10-16T11:22:03.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 10 client.13587 fetch_fsmap finished waiting for FSMap version 153 2023-10-16T11:22:03.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 10 client.13587 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:22:03.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 20 client.13587 populate_metadata read hostname 'smithi181' 2023-10-16T11:22:03.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab20bfe70 0x557ab20c0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 4 client.13587 mds_command: new command op to 13452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T11:22:03.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 --> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ab1fe7e00 con 0x557ab20bfe70 2023-10-16T11:22:03.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.990+0000 7f44a257b700 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab20bfe70 0x557ab20c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.991+0000 7f44817fa700 10 client.13587 ms_handle_connect on v2:172.21.15.190:6832/284709222 2023-10-16T11:22:03.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.991+0000 7f44a257b700 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab20bfe70 0x557ab20c0250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420537d141135fbb server_cookie=c3498a3b15017edc in_seq=0 out_seq=0 2023-10-16T11:22:03.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.991+0000 7f44817fa700 1 -- 172.21.15.181:0/1306195424 <== mds.0 v2:172.21.15.190:6832/284709222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ab1fe7e00 con 0x557ab20bfe70 2023-10-16T11:22:03.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.991+0000 7f44817fa700 10 client.13587 handle_command_reply: tid=0 2023-10-16T11:22:03.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.991+0000 7f44b09c9b80 1 client.13587 shutdown 2023-10-16T11:22:03.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.991+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 msgr2=0x557ab209c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.991+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab209c9b0 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7f448c0017c0 tx=0x7f448c057740 comp rx=0 tx=0).stop 2023-10-16T11:22:03.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 shutdown_connections 2023-10-16T11:22:03.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab20bfe70 0x557ab20c0250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab209c9b0 unknown :-1 s=CLOSED pgs=3147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab20c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44817fa700 0 client.13587 ms_handle_reset on v2:172.21.15.190:6832/284709222 2023-10-16T11:22:03.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/1306195424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab20c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 >> 172.21.15.181:0/1306195424 conn(0x557ab1fc9d80 msgr2=0x557ab20e1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:03.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 shutdown_connections 2023-10-16T11:22:03.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 1 -- 172.21.15.181:0/1306195424 wait complete. 2023-10-16T11:22:03.024 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:02.992+0000 7f44b09c9b80 20 client.13587 trim_cache size 0 max 16384 2023-10-16T11:22:03.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.009+0000 7f44b09c9b80 1 Processor -- start 2023-10-16T11:22:03.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.009+0000 7f44b09c9b80 1 -- start start 2023-10-16T11:22:03.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab2098b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab209ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab1e6f8d0 con 0x557ab20c4630 2023-10-16T11:22:03.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab20c1a30 con 0x557ab209a010 2023-10-16T11:22:03.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ab20c15b0 con 0x557ab20bf220 2023-10-16T11:22:03.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33236/0 (socket says 172.21.15.181:33236) 2023-10-16T11:22:03.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4009605822 learned_addr learned my addr 172.21.15.181:0/4009605822 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:03.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a1d7a700 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab209ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a257b700 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab2098b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 msgr2=0x557ab209ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab209ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 msgr2=0x557ab2098b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab2098b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4009605822 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ab1e02c10 con 0x557ab20c4630 2023-10-16T11:22:03.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20c3970 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f448c054260 tx=0x7f448c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296d04cb42f7f178 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f4498ff9700 1 -- 172.21.15.181:0/4009605822 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c05b070 con 0x557ab20c4630 2023-10-16T11:22:03.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f4498ff9700 1 -- 172.21.15.181:0/4009605822 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c0576c0 con 0x557ab20c4630 2023-10-16T11:22:03.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.010+0000 7f4498ff9700 1 -- 172.21.15.181:0/4009605822 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c06f040 con 0x557ab20c4630 2023-10-16T11:22:03.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4009605822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 msgr2=0x557ab20c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20c3970 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f448c054260 tx=0x7f448c026000 comp rx=0 tx=0).stop 2023-10-16T11:22:03.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4009605822 shutdown_connections 2023-10-16T11:22:03.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20c3970 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab209a010 0x557ab2098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4009605822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab20bf220 0x557ab209ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4009605822 >> 172.21.15.181:0/4009605822 conn(0x557ab1fc9d80 msgr2=0x557ab20bf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:03.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4009605822 shutdown_connections 2023-10-16T11:22:03.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.011+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4009605822 wait complete. 2023-10-16T11:22:03.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.012+0000 7f44b09c9b80 1 -- start start 2023-10-16T11:22:03.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.012+0000 7f44b09c9b80 1 Processor -- start 2023-10-16T11:22:03.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.012+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab20bc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.012+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20bf220 0x557ab2149200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.037 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.012+0000 7f44b09c9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab20bd0e0 con 0x557ab20c4630 2023-10-16T11:22:03.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab20bf820 con 0x557ab20bf220 2023-10-16T11:22:03.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ab2149d60 con 0x557ab209a010 2023-10-16T11:22:03.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 10 client.0 _ll_register_callbacks cb 0x7f448c057b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T11:22:03.038 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33242/0 (socket says 172.21.15.181:33242) 2023-10-16T11:22:03.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4166391410 learned_addr learned my addr 172.21.15.181:0/4166391410 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:03.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a257b700 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab20bc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a1d7a700 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20bf220 0x557ab2149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 msgr2=0x557ab20bc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab20bc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20bf220 msgr2=0x557ab2149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20bf220 0x557ab2149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 -- 172.21.15.181:0/4166391410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ab1e19d90 con 0x557ab20c4630 2023-10-16T11:22:03.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44a2d7c700 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20ca030 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7f448c057cf0 tx=0x7f448c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df14de4a29e1a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44817fa700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2023-10-16T11:22:03.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44817fa700 1 -- 172.21.15.181:0/4166391410 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c05b070 con 0x557ab20c4630 2023-10-16T11:22:03.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44817fa700 1 -- 172.21.15.181:0/4166391410 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f448c057d60 con 0x557ab20c4630 2023-10-16T11:22:03.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f449400f040 con 0x557ab20c4630 2023-10-16T11:22:03.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44817fa700 1 -- 172.21.15.181:0/4166391410 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448c06f040 con 0x557ab20c4630 2023-10-16T11:22:03.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 10 client.13593 fetch_fsmap learned FSMap version 153 2023-10-16T11:22:03.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=153,monmap=2+}) v3 -- 0x557ab1e40200 con 0x557ab20c4630 2023-10-16T11:22:03.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44817fa700 1 -- 172.21.15.181:0/4166391410 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449400f040 con 0x557ab20c4630 2023-10-16T11:22:03.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44817fa700 1 -- 172.21.15.181:0/4166391410 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 153) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f448c087120 con 0x557ab20c4630 2023-10-16T11:22:03.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 10 client.13593 fetch_fsmap finished waiting for FSMap version 153 2023-10-16T11:22:03.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.013+0000 7f44b09c9b80 10 client.13593 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T11:22:03.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44b09c9b80 20 client.13593 populate_metadata read hostname 'smithi181' 2023-10-16T11:22:03.043 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab2148400 0x557ab20ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44b09c9b80 4 client.13593 mds_command: new command op to 13452 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T11:22:03.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 --> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557ab1fe7b20 con 0x557ab2148400 2023-10-16T11:22:03.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44a257b700 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab2148400 0x557ab20ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44a257b700 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab2148400 0x557ab20ca570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774fc200cd5659eb server_cookie=d98892c3c4982848 in_seq=0 out_seq=0 2023-10-16T11:22:03.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44817fa700 10 client.13593 ms_handle_connect on v2:172.21.15.190:6832/284709222 2023-10-16T11:22:03.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44817fa700 1 -- 172.21.15.181:0/4166391410 <== mds.0 v2:172.21.15.190:6832/284709222 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x557ab1fe7b20 con 0x557ab2148400 2023-10-16T11:22:03.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44817fa700 10 client.13593 handle_command_reply: tid=0 2023-10-16T11:22:03.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44b09c9b80 1 client.13593 shutdown 2023-10-16T11:22:03.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 msgr2=0x557ab20ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.045 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.014+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20ca030 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7f448c057cf0 tx=0x7f448c014000 comp rx=0 tx=0).stop 2023-10-16T11:22:03.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 shutdown_connections 2023-10-16T11:22:03.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:6832/284709222,v1:172.21.15.190:6834/284709222] conn(0x557ab2148400 0x557ab20ca570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557ab209a010 0x557ab20bc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557ab20c4630 0x557ab20ca030 unknown :-1 s=CLOSED pgs=3149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 --2- 172.21.15.181:0/4166391410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ab20bf220 0x557ab2149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44817fa700 0 client.13593 ms_handle_reset on v2:172.21.15.190:6832/284709222 2023-10-16T11:22:03.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 >> 172.21.15.181:0/4166391410 conn(0x557ab1fc9d80 msgr2=0x557ab2096d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:03.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 shutdown_connections 2023-10-16T11:22:03.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 1 -- 172.21.15.181:0/4166391410 wait complete. 2023-10-16T11:22:03.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44b09c9b80 20 client.13593 trim_cache size 0 max 16384 2023-10-16T11:22:03.048 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:03.048 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "c0cf369f-f4d9-40c8-9248-54aac3970136", 2023-10-16T11:22:03.048 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2023-10-16T11:22:03.049 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 13452, 2023-10-16T11:22:03.049 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2023-10-16T11:22:03.049 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2023-10-16T11:22:03.049 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2023-10-16T11:22:03.049 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 5.9109742580000004, 2023-10-16T11:22:03.050 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 153, 2023-10-16T11:22:03.050 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 269, 2023-10-16T11:22:03.050 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 269, 2023-10-16T11:22:03.050 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 10.852202482999999 2023-10-16T11:22:03.050 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f448806b020 msgr2=0x7f448806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f448806b020 0x7f448806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4494007520 tx=0x7f449400b040 comp rx=0 tx=0).stop 2023-10-16T11:22:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f449c0c6710 msgr2=0x7f449c15a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f449c0c6710 0x7f449c15a4a0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f449c071410 tx=0x7f448c001120 comp rx=0 tx=0).stop 2023-10-16T11:22:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 shutdown_connections 2023-10-16T11:22:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f449c0c6710 0x7f449c15a4a0 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4484005210 0x7f449c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f448806b020 0x7f448806d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 --2- 172.21.15.181:0/4106891313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f449c0c6290 0x7f449c159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 >> 172.21.15.181:0/4106891313 conn(0x7f449c0d04a0 msgr2=0x7f449c06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:03.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 shutdown_connections 2023-10-16T11:22:03.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.015+0000 7f44a47df700 1 -- 172.21.15.181:0/4106891313 wait complete. 2023-10-16T11:22:03.055 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T11:22:03.068 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-10-16T11:22:03.067+0000 7f1dd2fe5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:22:03.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.175+0000 7fa6a6cba700 1 Processor -- start 2023-10-16T11:22:03.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.175+0000 7fa6a6cba700 1 -- start start 2023-10-16T11:22:03.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.176+0000 7fa6a6cba700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa6a015b990 0x7fa6a00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.176+0000 7fa6a6cba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 0x7fa6a00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.176+0000 7fa6a6cba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00cc510 con 0x7fa6a00cd2d0 2023-10-16T11:22:03.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.176+0000 7fa6a6cba700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a00d24f0 con 0x7fa6a015b990 2023-10-16T11:22:03.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.176+0000 7fa6a6cba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00cc7f0 con 0x7fa6a00d0370 2023-10-16T11:22:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.176+0000 7fa69ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa6a00cd2d0 0x7fa6a00cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44326/0 (socket says 172.21.15.181:44326) 2023-10-16T11:22:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.176+0000 7fa69ffff700 1 -- 172.21.15.181:0/1793754555 learned_addr learned my addr 172.21.15.181:0/1793754555 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa6a5257700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 0x7fa6a00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa6a4a56700 1 --2- 172.21.15.181:0/1793754555 >> v2:172.21.15.190:3300/0 conn(0x7fa6a015b990 0x7fa6a00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857822913 0 0) 0x7fa6a00cc510 con 0x7fa6a00cd2d0 2023-10-16T11:22:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6900035b0 con 0x7fa6a00cd2d0 2023-10-16T11:22:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892636512 0 0) 0x7fa6900035b0 con 0x7fa6a00cd2d0 2023-10-16T11:22:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 msgr2=0x7fa6a00d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 0x7fa6a00d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 >> v2:172.21.15.190:3300/0 conn(0x7fa6a015b990 msgr2=0x7fa6a00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 --2- 172.21.15.181:0/1793754555 >> v2:172.21.15.190:3300/0 conn(0x7fa6a015b990 0x7fa6a00ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00ccad0 con 0x7fa6a00cd2d0 2023-10-16T11:22:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fa68c001670 con 0x7fa6a00cd2d0 2023-10-16T11:22:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 >> v1:172.21.15.181:6789/0 conn(0x7fa6a00cd2d0 legacy=0x7fa6a00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 0x7fa690003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa690004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa690004d90 0x7fa690005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00d24f0 con 0x7fa690003f80 2023-10-16T11:22:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00cc7f0 con 0x7fa690004d90 2023-10-16T11:22:03.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69000ba40 con 0x7fa6a00d0370 2023-10-16T11:22:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69ffff700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa690004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa6a4a56700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 0x7fa690003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa6a5257700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa690004d90 0x7fa690005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69ffff700 1 -- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 msgr2=0x7fa690003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69ffff700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00d0370 0x7fa690003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69ffff700 1 -- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa690004d90 msgr2=0x7fa690005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:03.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69ffff700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa690004d90 0x7fa690005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.177+0000 7fa69ffff700 1 -- 172.21.15.181:0/1793754555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6900035b0 con 0x7fa690003f80 2023-10-16T11:22:03.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa69ffff700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa690004850 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00c370 tx=0x7fa690005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f201321693ba60 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c007c60 con 0x7fa690003f80 2023-10-16T11:22:03.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c005960 con 0x7fa690003f80 2023-10-16T11:22:03.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa69f7fe700 1 -- 172.21.15.181:0/1793754555 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c005d10 con 0x7fa690003f80 2023-10-16T11:22:03.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa6a6cba700 1 -- 172.21.15.181:0/1793754555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 msgr2=0x7fa690004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa6a6cba700 1 --2- 172.21.15.181:0/1793754555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa690004850 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00c370 tx=0x7fa690005880 comp rx=0 tx=0).stop 2023-10-16T11:22:03.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa6a5257700 1 -- 172.21.15.181:0/1793754555 reap_dead start 2023-10-16T11:22:03.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa6a6cba700 1 -- 172.21.15.181:0/1793754555 shutdown_connections 2023-10-16T11:22:03.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa6a6cba700 1 -- 172.21.15.181:0/1793754555 >> 172.21.15.181:0/1793754555 conn(0x7fa6a009a9a0 msgr2=0x7fa6a00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:03.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa6a6cba700 1 -- 172.21.15.181:0/1793754555 shutdown_connections 2023-10-16T11:22:03.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.178+0000 7fa6a6cba700 1 -- 172.21.15.181:0/1793754555 wait complete. 2023-10-16T11:22:03.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 Processor -- start 2023-10-16T11:22:03.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 -- start start 2023-10-16T11:22:03.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa6a00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a00b42a0 0x7fa6a00cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00cb1f0 0x7fa6a00b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a0149790 con 0x7fa690003f80 2023-10-16T11:22:03.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006c460 con 0x7fa6a00b42a0 2023-10-16T11:22:03.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.180+0000 7fa6a6cba700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006c740 con 0x7fa6a00cb1f0 2023-10-16T11:22:03.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa6a00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa6a00ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33258/0 (socket says 172.21.15.181:33258) 2023-10-16T11:22:03.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 -- 172.21.15.181:0/339141166 learned_addr learned my addr 172.21.15.181:0/339141166 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:03.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa69ffff700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a00b42a0 0x7fa6a00cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a5257700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00cb1f0 0x7fa6a00b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 -- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00cb1f0 msgr2=0x7fa6a00b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00cb1f0 0x7fa6a00b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 -- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a00b42a0 msgr2=0x7fa6a00cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a00b42a0 0x7fa6a00cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 -- 172.21.15.181:0/339141166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a006ca20 con 0x7fa690003f80 2023-10-16T11:22:03.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a5257700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00cb1f0 0x7fa6a00b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:03.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa69ffff700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a00b42a0 0x7fa6a00cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:03.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a4a56700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa6a00ca770 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7fa694011a70 tx=0x7fa694011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa69d7fa700 1 -- 172.21.15.181:0/339141166 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69400fa00 con 0x7fa690003f80 2023-10-16T11:22:03.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa69d7fa700 1 -- 172.21.15.181:0/339141166 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69400fb80 con 0x7fa690003f80 2023-10-16T11:22:03.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a006cd00 con 0x7fa690003f80 2023-10-16T11:22:03.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa69d7fa700 1 -- 172.21.15.181:0/339141166 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694014440 con 0x7fa690003f80 2023-10-16T11:22:03.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.181+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a006d220 con 0x7fa690003f80 2023-10-16T11:22:03.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.182+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a006cef0 con 0x7fa690003f80 2023-10-16T11:22:03.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.183+0000 7fa69d7fa700 1 -- 172.21.15.181:0/339141166 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa6940145a0 con 0x7fa690003f80 2023-10-16T11:22:03.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.183+0000 7fa69d7fa700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fa68406b020 0x7fa68406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:03.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.183+0000 7fa69ffff700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fa68406b020 0x7fa68406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:03.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.183+0000 7fa69d7fa700 1 -- 172.21.15.181:0/339141166 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fa69408a1b0 con 0x7fa690003f80 2023-10-16T11:22:03.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.184+0000 7fa69ffff700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fa68406b020 0x7fa68406d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa68c015040 tx=0x7fa68c00a000 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:03.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.188+0000 7fa69d7fa700 1 -- 172.21.15.181:0/339141166 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa694054e90 con 0x7fa690003f80 2023-10-16T11:22:03.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.425+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6a0067d20 con 0x7fa690003f80 2023-10-16T11:22:03.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.426+0000 7fa69d7fa700 1 -- 172.21.15.181:0/339141166 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v153) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa6940584b0 con 0x7fa690003f80 2023-10-16T11:22:03.428 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T11:22:03.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fa68406b020 msgr2=0x7fa68406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fa68406b020 0x7fa68406d4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa68c015040 tx=0x7fa68c00a000 comp rx=0 tx=0).stop 2023-10-16T11:22:03.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 msgr2=0x7fa6a00ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa6a00ca770 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7fa694011a70 tx=0x7fa694011e30 comp rx=0 tx=0).stop 2023-10-16T11:22:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 shutdown_connections 2023-10-16T11:22:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fa68406b020 0x7fa68406d4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6a00cb1f0 0x7fa6a00b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa690003f80 0x7fa6a00ca770 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 --2- 172.21.15.181:0/339141166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a00b42a0 0x7fa6a00cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:03.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 >> 172.21.15.181:0/339141166 conn(0x7fa6a009a9a0 msgr2=0x7fa6a015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:03.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 shutdown_connections 2023-10-16T11:22:03.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:03.428+0000 7fa6a6cba700 1 -- 172.21.15.181:0/339141166 wait complete. 2023-10-16T11:22:03.442 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T11:22:03.442 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T11:22:03.443 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-16T11:22:03.443 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T11:22:03.443 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:03.444 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T11:22:03.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-16T11:22:03.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T11:22:03.445 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:22:03.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:03.445 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-16T11:22:03.463 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T11:22:03.463 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T11:22:03.464 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T11:22:03.464 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:03.464 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-16T11:22:03.464 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:03.464 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:22:03.465 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:03.465 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:22:03.465 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:03.465 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T11:22:03.465 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:03.466 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T11:22:03.466 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:03.466 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T11:22:03.466 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 36546sec preferred_lft 36546sec 2023-10-16T11:22:03.466 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f543:5035:7b25:ba68/64 scope link noprefixroute 2023-10-16T11:22:03.467 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:03.467 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T11:22:03.467 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff 2023-10-16T11:22:03.467 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T11:22:03.467 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:03.468 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::869:a4ff:fe8f:e7ed/64 scope link 2023-10-16T11:22:03.468 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:03.468 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:22:03.468 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7e:b8:57:ac:26:38 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:22:03.468 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7cb8:57ff:feac:2638/64 scope link 2023-10-16T11:22:03.468 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:03.469 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T11:22:03.469 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether da:bb:6f:2e:14:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:22:03.469 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::d8bb:6fff:fe2e:14c8/64 scope link 2023-10-16T11:22:03.469 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-16T11:22:03.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:03.470 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T11:22:03.529 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-16T11:22:03.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:03.531 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-16T11:22:03.589 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-16T11:22:03.590 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T11:22:03.590 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T11:22:03.590 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T11:22:03.590 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T11:22:03.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:03.591 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-16T11:22:03.592 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-16T11:22:03.592 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T11:22:03.592 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T11:22:03.593 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T11:22:03.593 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-16T11:22:03.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:03.695 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:22:03.711 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:22:03.712 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:22:03.712 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:22:03.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:03.713 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-16T11:22:03.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:22:03.781 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:03.835 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T11:22:03.836 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T11:22:03.836 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:03.892 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-16T11:22:03.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:03.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:22:03.977 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:22:03.977 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:22:03.977 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:22:03.986 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:22:03.986 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:22:03.986 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:22:03.986 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:22:03.987 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:22:04.060 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-16T11:22:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-16T11:22:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:invalid new device string format 2023-10-16T11:22:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.190:3300,172.21.15.190:3301" 2023-10-16T11:22:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-16T11:22:04.061 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790" 2023-10-16T11:22:04.062 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.181:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2023-10-16T11:22:04.062 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c0cf369f-f4d9-40c8-9248-54aac3970136" will pass to kernel 2023-10-16T11:22:04.063 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T11:22:04.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:04.064 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:04.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:04.091 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T11:22:04.181 INFO:teuthology.orchestra.run.smithi181.stdout:client11854 2023-10-16T11:22:04.181 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:22:04.182 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:04.199 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-16T11:22:04.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.358+0000 7f5bfbaa2700 1 Processor -- start 2023-10-16T11:22:04.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.359+0000 7f5bfbaa2700 1 -- start start 2023-10-16T11:22:04.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.359+0000 7f5bfbaa2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bf415b9f0 0x7f5bf40cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.359+0000 7f5bfbaa2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf40cfb50 0x7f5bf40ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.359+0000 7f5bfbaa2700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf406b370 con 0x7f5bf40cc400 2023-10-16T11:22:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.359+0000 7f5bfbaa2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf406b800 con 0x7f5bf415b9f0 2023-10-16T11:22:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.359+0000 7f5bfbaa2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf406aa40 con 0x7f5bf40cfb50 2023-10-16T11:22:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.359+0000 7f5bf983e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bf415b9f0 0x7f5bf40cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf983e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bf415b9f0 0x7f5bf40cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33270/0 (socket says 172.21.15.181:33270) 2023-10-16T11:22:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf983e700 1 -- 172.21.15.181:0/1374429909 learned_addr learned my addr 172.21.15.181:0/1374429909 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bfa03f700 1 --2- 172.21.15.181:0/1374429909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf40cfb50 0x7f5bf40ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf983e700 1 -- 172.21.15.181:0/1374429909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf40cfb50 msgr2=0x7f5bf40ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf983e700 1 --2- 172.21.15.181:0/1374429909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf40cfb50 0x7f5bf40ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf983e700 1 -- 172.21.15.181:0/1374429909 >> v1:172.21.15.190:6789/0 conn(0x7f5bf40cc400 legacy=0x7f5bf40cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf983e700 1 -- 172.21.15.181:0/1374429909 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf406ad20 con 0x7f5bf415b9f0 2023-10-16T11:22:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bfa03f700 1 --2- 172.21.15.181:0/1374429909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf40cfb50 0x7f5bf40ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf983e700 1 --2- 172.21.15.181:0/1374429909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf415b9f0 0x7f5bf40cd950 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 crypto rx=0x7f5be4006fd0 tx=0x7f5be4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493a621a8b153cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:04.366 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf883c700 1 -- 172.21.15.181:0/1374429909 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4005e30 con 0x7f5bf415b9f0 2023-10-16T11:22:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf883c700 1 -- 172.21.15.181:0/1374429909 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4004030 con 0x7f5bf415b9f0 2023-10-16T11:22:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.360+0000 7f5bf883c700 1 -- 172.21.15.181:0/1374429909 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40043e0 con 0x7f5bf415b9f0 2023-10-16T11:22:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/1374429909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf415b9f0 msgr2=0x7f5bf40cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/1374429909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf415b9f0 0x7f5bf40cd950 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 crypto rx=0x7f5be4006fd0 tx=0x7f5be4009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/1374429909 shutdown_connections 2023-10-16T11:22:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/1374429909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf40cfb50 0x7f5bf40ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/1374429909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf415b9f0 0x7f5bf40cd950 unknown :-1 s=CLOSED pgs=3153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/1374429909 >> 172.21.15.181:0/1374429909 conn(0x7f5bf40d04a0 msgr2=0x7f5bf40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/1374429909 shutdown_connections 2023-10-16T11:22:04.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.361+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/1374429909 wait complete. 2023-10-16T11:22:04.371 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.362+0000 7f5bfbaa2700 1 Processor -- start 2023-10-16T11:22:04.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.362+0000 7f5bfbaa2700 1 -- start start 2023-10-16T11:22:04.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40cc400 0x7f5bf41e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40cfb50 0x7f5bf41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf415b9f0 0x7f5bf40b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.372 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf406e4a0 con 0x7f5bf40cc400 2023-10-16T11:22:04.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf406e780 con 0x7f5bf40cfb50 2023-10-16T11:22:04.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf41e4370 con 0x7f5bf415b9f0 2023-10-16T11:22:04.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40cc400 0x7f5bf41e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40cc400 0x7f5bf41e1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33274/0 (socket says 172.21.15.181:33274) 2023-10-16T11:22:04.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 -- 172.21.15.181:0/3088050811 learned_addr learned my addr 172.21.15.181:0/3088050811 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:04.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfa03f700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf415b9f0 0x7f5bf40b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf903d700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40cfb50 0x7f5bf41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 -- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf415b9f0 msgr2=0x7f5bf40b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.374 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf415b9f0 0x7f5bf40b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 -- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40cfb50 msgr2=0x7f5bf41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40cfb50 0x7f5bf41589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 -- 172.21.15.181:0/3088050811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4009500 con 0x7f5bf40cc400 2023-10-16T11:22:04.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf903d700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40cfb50 0x7f5bf41589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:04.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bf983e700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40cc400 0x7f5bf41e1f60 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009c10 tx=0x7f5be4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:04.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bea7fc700 1 -- 172.21.15.181:0/3088050811 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4003d90 con 0x7f5bf40cc400 2023-10-16T11:22:04.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bea7fc700 1 -- 172.21.15.181:0/3088050811 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4011420 con 0x7f5bf40cc400 2023-10-16T11:22:04.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bea7fc700 1 -- 172.21.15.181:0/3088050811 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40117d0 con 0x7f5bf40cc400 2023-10-16T11:22:04.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf41e4650 con 0x7f5bf40cc400 2023-10-16T11:22:04.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.363+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf41e24a0 con 0x7f5bf40cc400 2023-10-16T11:22:04.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.364+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf41e41d0 con 0x7f5bf40cc400 2023-10-16T11:22:04.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.365+0000 7f5bea7fc700 1 -- 172.21.15.181:0/3088050811 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5be4011930 con 0x7f5bf40cc400 2023-10-16T11:22:04.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.365+0000 7f5bea7fc700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f5bd806b050 0x7f5bd806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.365+0000 7f5bea7fc700 1 -- 172.21.15.181:0/3088050811 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f5be408abb0 con 0x7f5bf40cc400 2023-10-16T11:22:04.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.365+0000 7f5bf903d700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f5bd806b050 0x7f5bd806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.366+0000 7f5bf903d700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f5bd806b050 0x7f5bd806d510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4073da0 tx=0x7f5bf0009040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:04.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.370+0000 7f5bea7fc700 1 -- 172.21.15.181:0/3088050811 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5bf41e41d0 con 0x7f5bf40cc400 2023-10-16T11:22:04.592 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.590+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 --> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bf406b000 con 0x7f5bd806b050 2023-10-16T11:22:04.593 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.592+0000 7f5bea7fc700 1 -- 172.21.15.181:0/3088050811 <== mgr.13197 v2:172.21.15.181:6832/2291437477 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5bf0006860 con 0x7f5bd806b050 2023-10-16T11:22:04.594 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2023-10-16T11:22:04.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2023-10-16T11:22:04.594 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "cephfs" 2023-10-16T11:22:04.594 INFO:teuthology.orchestra.run.smithi181.stdout: } 2023-10-16T11:22:04.595 INFO:teuthology.orchestra.run.smithi181.stdout:] 2023-10-16T11:22:04.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f5bd806b050 msgr2=0x7f5bd806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.596 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f5bd806b050 0x7f5bd806d510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4073da0 tx=0x7f5bf0009040 comp rx=0 tx=0).stop 2023-10-16T11:22:04.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40cc400 msgr2=0x7f5bf41e1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40cc400 0x7f5bf41e1f60 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009c10 tx=0x7f5be4004750 comp rx=0 tx=0).stop 2023-10-16T11:22:04.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 shutdown_connections 2023-10-16T11:22:04.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f5bd806b050 0x7f5bd806d510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bf415b9f0 0x7f5bf40b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40cc400 0x7f5bf41e1f60 unknown :-1 s=CLOSED pgs=3154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 --2- 172.21.15.181:0/3088050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bf40cfb50 0x7f5bf41589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 >> 172.21.15.181:0/3088050811 conn(0x7f5bf40d04a0 msgr2=0x7f5bf40af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 shutdown_connections 2023-10-16T11:22:04.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.594+0000 7f5bfbaa2700 1 -- 172.21.15.181:0/3088050811 wait complete. 2023-10-16T11:22:04.608 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-16T11:22:04.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.729+0000 7f40ecf26700 1 Processor -- start 2023-10-16T11:22:04.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.729+0000 7f40ecf26700 1 -- start start 2023-10-16T11:22:04.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40ecf26700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40ecf26700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f40e8071be0 0x7f40e80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40ecf26700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40e806f3b0 0x7f40e806f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40ecf26700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e80ce1f0 con 0x7f40e806f3b0 2023-10-16T11:22:04.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40ecf26700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e80b2930 con 0x7f40e8071be0 2023-10-16T11:22:04.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40ecf26700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80ccda0 con 0x7f40e80b2010 2023-10-16T11:22:04.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40e806f3b0 0x7f40e806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40e806f3b0 0x7f40e806f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33286/0 (socket says 172.21.15.181:33286) 2023-10-16T11:22:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 -- 172.21.15.181:0/3190404916 learned_addr learned my addr 172.21.15.181:0/3190404916 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e659c700 1 --2- 172.21.15.181:0/3190404916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e5d9b700 1 --2- 172.21.15.181:0/3190404916 >> v2:172.21.15.190:3300/0 conn(0x7f40e8071be0 0x7f40e80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 -- 172.21.15.181:0/3190404916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 msgr2=0x7f40e80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:04.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 --2- 172.21.15.181:0/3190404916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 -- 172.21.15.181:0/3190404916 >> v2:172.21.15.190:3300/0 conn(0x7f40e8071be0 msgr2=0x7f40e80b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 --2- 172.21.15.181:0/3190404916 >> v2:172.21.15.190:3300/0 conn(0x7f40e8071be0 0x7f40e80b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.730+0000 7f40e6d9d700 1 -- 172.21.15.181:0/3190404916 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80727e0 con 0x7f40e806f3b0 2023-10-16T11:22:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40e6d9d700 1 --2- 172.21.15.181:0/3190404916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e806f3b0 0x7f40e806f840 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 crypto rx=0x7f40e8067600 tx=0x7f40e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71daa49fcb9d000 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:04.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40e559a700 1 -- 172.21.15.181:0/3190404916 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000e070 con 0x7f40e806f3b0 2023-10-16T11:22:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40e559a700 1 -- 172.21.15.181:0/3190404916 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e000b9c0 con 0x7f40e806f3b0 2023-10-16T11:22:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40e559a700 1 -- 172.21.15.181:0/3190404916 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000bd70 con 0x7f40e806f3b0 2023-10-16T11:22:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 -- 172.21.15.181:0/3190404916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e806f3b0 msgr2=0x7f40e806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 --2- 172.21.15.181:0/3190404916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e806f3b0 0x7f40e806f840 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 crypto rx=0x7f40e8067600 tx=0x7f40e000b330 comp rx=0 tx=0).stop 2023-10-16T11:22:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 -- 172.21.15.181:0/3190404916 shutdown_connections 2023-10-16T11:22:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 --2- 172.21.15.181:0/3190404916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e806f3b0 0x7f40e806f840 unknown :-1 s=CLOSED pgs=3155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 --2- 172.21.15.181:0/3190404916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 --2- 172.21.15.181:0/3190404916 >> v2:172.21.15.190:3300/0 conn(0x7f40e8071be0 0x7f40e80b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 -- 172.21.15.181:0/3190404916 >> 172.21.15.181:0/3190404916 conn(0x7f40e80cd010 msgr2=0x7f40e80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:04.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 -- 172.21.15.181:0/3190404916 shutdown_connections 2023-10-16T11:22:04.740 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.731+0000 7f40ecf26700 1 -- 172.21.15.181:0/3190404916 wait complete. 2023-10-16T11:22:04.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.733+0000 7f40ecf26700 1 Processor -- start 2023-10-16T11:22:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.733+0000 7f40ecf26700 1 -- start start 2023-10-16T11:22:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.733+0000 7f40ecf26700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40e806f3b0 0x7f40e8148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40ecf26700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8071be0 0x7f40e80c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40ecf26700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40ecf26700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80c6e60 con 0x7f40e8071be0 2023-10-16T11:22:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40ecf26700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80b5540 con 0x7f40e806f3b0 2023-10-16T11:22:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40ecf26700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80c6b80 con 0x7f40e80b2010 2023-10-16T11:22:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8071be0 0x7f40e80c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8071be0 0x7f40e80c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33290/0 (socket says 172.21.15.181:33290) 2023-10-16T11:22:04.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 -- 172.21.15.181:0/376108953 learned_addr learned my addr 172.21.15.181:0/376108953 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e6d9d700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e659c700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40e806f3b0 0x7f40e8148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 -- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 msgr2=0x7f40e80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 -- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40e806f3b0 msgr2=0x7f40e8148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40e806f3b0 0x7f40e8148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 -- 172.21.15.181:0/376108953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0009b30 con 0x7f40e8071be0 2023-10-16T11:22:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40e5d9b700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8071be0 0x7f40e80c62a0 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f40d800d800 tx=0x7f40d800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40d6ffd700 1 -- 172.21.15.181:0/376108953 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80125f0 con 0x7f40e8071be0 2023-10-16T11:22:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40d6ffd700 1 -- 172.21.15.181:0/376108953 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d8012750 con 0x7f40e8071be0 2023-10-16T11:22:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e80b4ef0 con 0x7f40e8071be0 2023-10-16T11:22:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40d6ffd700 1 -- 172.21.15.181:0/376108953 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8012ae0 con 0x7f40e8071be0 2023-10-16T11:22:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.734+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e80c67e0 con 0x7f40e8071be0 2023-10-16T11:22:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.735+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e80b4d50 con 0x7f40e8071be0 2023-10-16T11:22:04.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.736+0000 7f40d6ffd700 1 -- 172.21.15.181:0/376108953 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 169) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f40d8012c40 con 0x7f40e8071be0 2023-10-16T11:22:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.736+0000 7f40d6ffd700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f40c806b020 0x7f40c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.736+0000 7f40d6ffd700 1 -- 172.21.15.181:0/376108953 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f40d80898d0 con 0x7f40e8071be0 2023-10-16T11:22:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.736+0000 7f40e659c700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f40c806b020 0x7f40c806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.737+0000 7f40e659c700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f40c806b020 0x7f40c806d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f40e80c9f00 tx=0x7f40dc008040 comp rx=0 tx=0).ready entity=mgr.13197 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:04.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.741+0000 7f40d6ffd700 1 -- 172.21.15.181:0/376108953 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f40e80b4d50 con 0x7f40e8071be0 2023-10-16T11:22:04.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:04.991+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f40e80b4d50 con 0x7f40e8071be0 2023-10-16T11:22:05.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.137+0000 7f40d6ffd700 1 -- 172.21.15.181:0/376108953 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v170) v1 ==== 92+0+0 (secure 0 0 0) 0x7f40e80b4d50 con 0x7f40e8071be0 2023-10-16T11:22:05.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.140+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f40c806b020 msgr2=0x7f40c806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.140+0000 7f40ecf26700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f40c806b020 0x7f40c806d4e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f40e80c9f00 tx=0x7f40dc008040 comp rx=0 tx=0).stop 2023-10-16T11:22:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.140+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8071be0 msgr2=0x7f40e80c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.140+0000 7f40ecf26700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8071be0 0x7f40e80c62a0 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f40d800d800 tx=0x7f40d800db10 comp rx=0 tx=0).stop 2023-10-16T11:22:05.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 shutdown_connections 2023-10-16T11:22:05.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f40c806b020 0x7f40c806d4e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40e80b2010 0x7f40e80c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40e806f3b0 0x7f40e8148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 --2- 172.21.15.181:0/376108953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40e8071be0 0x7f40e80c62a0 unknown :-1 s=CLOSED pgs=3156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 >> 172.21.15.181:0/376108953 conn(0x7f40e80cd010 msgr2=0x7f40e80b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 shutdown_connections 2023-10-16T11:22:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.142+0000 7f40ecf26700 1 -- 172.21.15.181:0/376108953 wait complete. 2023-10-16T11:22:05.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-16T11:22:05.285 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:05.284+0000 7fa3cc297200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:05.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.284+0000 7ff3427a5700 1 Processor -- start 2023-10-16T11:22:05.286 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.284+0000 7ff3427a5700 1 -- start start 2023-10-16T11:22:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff3427a5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff33c072760 0x7ff33c06a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff3427a5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c06b210 con 0x7ff33c15da70 2023-10-16T11:22:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff3427a5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c0cef00 con 0x7ff33c06ade0 2023-10-16T11:22:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff3427a5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c0cec20 con 0x7ff33c072760 2023-10-16T11:22:05.289 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff33b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff33c072760 0x7ff33c06a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff33bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff33c15da70 0x7ff33c0ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44400/0 (socket says 172.21.15.181:44400) 2023-10-16T11:22:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff33bfff700 1 -- 172.21.15.181:0/3780429114 learned_addr learned my addr 172.21.15.181:0/3780429114 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff33affd700 1 -- 172.21.15.181:0/3780429114 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648566590 0 0) 0x7ff33c06b210 con 0x7ff33c15da70 2023-10-16T11:22:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.285+0000 7ff33affd700 1 -- 172.21.15.181:0/3780429114 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3280035b0 con 0x7ff33c15da70 2023-10-16T11:22:05.290 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33affd700 1 -- 172.21.15.181:0/3780429114 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115263003 0 0) 0x7ff33c0cef00 con 0x7ff33c06ade0 2023-10-16T11:22:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33b7fe700 1 -- 172.21.15.181:0/3780429114 >> v1:172.21.15.190:6789/0 conn(0x7ff33c06ade0 legacy=0x7ff33c1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33b7fe700 1 -- 172.21.15.181:0/3780429114 >> v1:172.21.15.181:6789/0 conn(0x7ff33c15da70 legacy=0x7ff33c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33b7fe700 1 -- 172.21.15.181:0/3780429114 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c071c80 con 0x7ff33c072760 2023-10-16T11:22:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33b7fe700 1 --2- 172.21.15.181:0/3780429114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c072760 0x7ff33c06a8a0 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009840 tx=0x7ff32c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc81bf7fe282a98d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:05.291 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33affd700 1 -- 172.21.15.181:0/3780429114 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c011830 con 0x7ff33c072760 2023-10-16T11:22:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33affd700 1 -- 172.21.15.181:0/3780429114 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c011990 con 0x7ff33c072760 2023-10-16T11:22:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff33affd700 1 -- 172.21.15.181:0/3780429114 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c011d20 con 0x7ff33c072760 2023-10-16T11:22:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff3427a5700 1 -- 172.21.15.181:0/3780429114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c072760 msgr2=0x7ff33c06a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff3427a5700 1 --2- 172.21.15.181:0/3780429114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c072760 0x7ff33c06a8a0 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009840 tx=0x7ff32c009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:05.292 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff3427a5700 1 -- 172.21.15.181:0/3780429114 shutdown_connections 2023-10-16T11:22:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff3427a5700 1 --2- 172.21.15.181:0/3780429114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c072760 0x7ff33c06a8a0 unknown :-1 s=CLOSED pgs=3163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.286+0000 7ff3427a5700 1 -- 172.21.15.181:0/3780429114 >> 172.21.15.181:0/3780429114 conn(0x7ff33c0cd010 msgr2=0x7ff33c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.287+0000 7ff3427a5700 1 -- 172.21.15.181:0/3780429114 shutdown_connections 2023-10-16T11:22:05.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.287+0000 7ff3427a5700 1 -- 172.21.15.181:0/3780429114 wait complete. 2023-10-16T11:22:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.288+0000 7ff3427a5700 1 Processor -- start 2023-10-16T11:22:05.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.288+0000 7ff3427a5700 1 -- start start 2023-10-16T11:22:05.294 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:05.288+0000 7fa7d928a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c06ade0 0x7ff33c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff33c072760 0x7ff33c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff33c15da70 0x7ff33c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0cbbd0 con 0x7ff33c06ade0 2023-10-16T11:22:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c1580e0 con 0x7ff33c072760 2023-10-16T11:22:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0b46a0 con 0x7ff33c15da70 2023-10-16T11:22:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c06ade0 0x7ff33c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c06ade0 0x7ff33c157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33324/0 (socket says 172.21.15.181:33324) 2023-10-16T11:22:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 -- 172.21.15.181:0/1402448494 learned_addr learned my addr 172.21.15.181:0/1402448494 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33b7fe700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff33c072760 0x7ff33c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff340d42700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff33c15da70 0x7ff33c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 -- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff33c15da70 msgr2=0x7ff33c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff33c15da70 0x7ff33c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 -- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff33c072760 msgr2=0x7ff33c0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff33c072760 0x7ff33c0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 -- 172.21.15.181:0/1402448494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0094a0 con 0x7ff33c06ade0 2023-10-16T11:22:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff33bfff700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c06ade0 0x7ff33c157ba0 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c500 tx=0x7ff32400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:05.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff338ff9700 1 -- 172.21.15.181:0/1402448494 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324005d70 con 0x7ff33c06ade0 2023-10-16T11:22:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff338ff9700 1 -- 172.21.15.181:0/1402448494 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32400ce70 con 0x7ff33c06ade0 2023-10-16T11:22:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c0b43c0 con 0x7ff33c06ade0 2023-10-16T11:22:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff338ff9700 1 -- 172.21.15.181:0/1402448494 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240072d0 con 0x7ff33c06ade0 2023-10-16T11:22:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.289+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c0cb3d0 con 0x7ff33c06ade0 2023-10-16T11:22:05.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.291+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c0caf50 con 0x7ff33c06ade0 2023-10-16T11:22:05.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.291+0000 7ff338ff9700 1 -- 172.21.15.181:0/1402448494 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 170) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff324007430 con 0x7ff33c06ade0 2023-10-16T11:22:05.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.292+0000 7ff338ff9700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 0x7ff31c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.292+0000 7ff33b7fe700 1 -- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 msgr2=0x7ff31c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:05.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.292+0000 7ff338ff9700 1 -- 172.21.15.181:0/1402448494 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ff324089000 con 0x7ff33c06ade0 2023-10-16T11:22:05.302 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.292+0000 7ff33b7fe700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 0x7ff31c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:05.303 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.296+0000 7ff338ff9700 1 -- 172.21.15.181:0/1402448494 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff324053e00 con 0x7ff33c06ade0 2023-10-16T11:22:05.352 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:05.351+0000 7fa3cc297200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:05.359 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:05.357+0000 7fa7d928a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:05.425 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:05.423+0000 7fa3cc297200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:05.433 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:05.431+0000 7fa7d928a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:05.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.493+0000 7ff33b7fe700 1 -- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 msgr2=0x7ff31c06d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:05.495 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.493+0000 7ff33b7fe700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 0x7ff31c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:05.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.497+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff33c0caf50 con 0x7ff33c06ade0 2023-10-16T11:22:05.501 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:05.500+0000 7fa3cc297200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.501+0000 7ff338ff9700 1 -- 172.21.15.181:0/1402448494 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v95)=0 v95) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff33c0caf50 con 0x7ff33c06ade0 2023-10-16T11:22:05.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 msgr2=0x7ff31c06d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:05.505 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 0x7ff31c06d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c06ade0 msgr2=0x7ff33c157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c06ade0 0x7ff33c157ba0 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c500 tx=0x7ff32400c810 comp rx=0 tx=0).stop 2023-10-16T11:22:05.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 shutdown_connections 2023-10-16T11:22:05.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7ff31c06b070 0x7ff31c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.506 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff33c15da70 0x7ff33c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c06ade0 0x7ff33c157ba0 unknown :-1 s=CLOSED pgs=3164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 --2- 172.21.15.181:0/1402448494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff33c072760 0x7ff33c0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 >> 172.21.15.181:0/1402448494 conn(0x7ff33c0cd010 msgr2=0x7ff33c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:05.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 shutdown_connections 2023-10-16T11:22:05.507 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.503+0000 7ff3427a5700 1 -- 172.21.15.181:0/1402448494 wait complete. 2023-10-16T11:22:05.513 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:05.511+0000 7fa7d928a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:05.518 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-16T11:22:05.642 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:05.641+0000 7fa3cc297200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:05.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.644+0000 7f51128df700 1 Processor -- start 2023-10-16T11:22:05.646 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.644+0000 7f51128df700 1 -- start start 2023-10-16T11:22:05.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f51128df700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f510c0b2010 0x7f510c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f51128df700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f510c071be0 0x7f510c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f51128df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c06f3b0 0x7f510c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f51128df700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c0ce1f0 con 0x7f510c071be0 2023-10-16T11:22:05.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f51128df700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c072090 con 0x7f510c0b2010 2023-10-16T11:22:05.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f51128df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0b2930 con 0x7f510c06f3b0 2023-10-16T11:22:05.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f510c071be0 0x7f510c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f510c071be0 0x7f510c0b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33368/0 (socket says 172.21.15.181:33368) 2023-10-16T11:22:05.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 -- 172.21.15.181:0/2495851139 learned_addr learned my addr 172.21.15.181:0/2495851139 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:05.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510bfff700 1 --2- 172.21.15.181:0/2495851139 >> v2:172.21.15.190:3300/0 conn(0x7f510c0b2010 0x7f510c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f5110e7c700 1 --2- 172.21.15.181:0/2495851139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c06f3b0 0x7f510c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 -- 172.21.15.181:0/2495851139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c06f3b0 msgr2=0x7f510c06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:05.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 --2- 172.21.15.181:0/2495851139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c06f3b0 0x7f510c06f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 -- 172.21.15.181:0/2495851139 >> v2:172.21.15.190:3300/0 conn(0x7f510c0b2010 msgr2=0x7f510c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:05.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 --2- 172.21.15.181:0/2495851139 >> v2:172.21.15.190:3300/0 conn(0x7f510c0b2010 0x7f510c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 -- 172.21.15.181:0/2495851139 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c0ccda0 con 0x7f510c071be0 2023-10-16T11:22:05.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510b7fe700 1 --2- 172.21.15.181:0/2495851139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0b23f0 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7f510c070430 tx=0x7f50f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1142b2a81ef0335 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:05.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.645+0000 7f510affd700 1 -- 172.21.15.181:0/2495851139 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8011830 con 0x7f510c071be0 2023-10-16T11:22:05.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f510affd700 1 -- 172.21.15.181:0/2495851139 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f8011990 con 0x7f510c071be0 2023-10-16T11:22:05.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f510affd700 1 -- 172.21.15.181:0/2495851139 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8011d20 con 0x7f510c071be0 2023-10-16T11:22:05.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 -- 172.21.15.181:0/2495851139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 msgr2=0x7f510c0b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 --2- 172.21.15.181:0/2495851139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0b23f0 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7f510c070430 tx=0x7f50f8009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 -- 172.21.15.181:0/2495851139 shutdown_connections 2023-10-16T11:22:05.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 --2- 172.21.15.181:0/2495851139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c06f3b0 0x7f510c06f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 --2- 172.21.15.181:0/2495851139 >> v2:172.21.15.190:3300/0 conn(0x7f510c0b2010 0x7f510c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 --2- 172.21.15.181:0/2495851139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0b23f0 unknown :-1 s=CLOSED pgs=3165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 -- 172.21.15.181:0/2495851139 >> 172.21.15.181:0/2495851139 conn(0x7f510c0cd010 msgr2=0x7f510c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:05.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 -- 172.21.15.181:0/2495851139 shutdown_connections 2023-10-16T11:22:05.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.646+0000 7f51128df700 1 -- 172.21.15.181:0/2495851139 wait complete. 2023-10-16T11:22:05.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.648+0000 7f51128df700 1 Processor -- start 2023-10-16T11:22:05.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.648+0000 7f51128df700 1 -- start start 2023-10-16T11:22:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f51128df700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f510c06f3b0 0x7f510c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f51128df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f51128df700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c0b2010 0x7f510c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f51128df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c15b390 con 0x7f510c071be0 2023-10-16T11:22:05.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f51128df700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c15cf90 con 0x7f510c06f3b0 2023-10-16T11:22:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f51128df700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c146cc0 con 0x7f510c0b2010 2023-10-16T11:22:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33372/0 (socket says 172.21.15.181:33372) 2023-10-16T11:22:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 -- 172.21.15.181:0/3797008140 learned_addr learned my addr 172.21.15.181:0/3797008140 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:05.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f5110e7c700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c0b2010 0x7f510c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 -- 172.21.15.181:0/3797008140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c0b2010 msgr2=0x7f510c0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c0b2010 0x7f510c0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 -- 172.21.15.181:0/3797008140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f510c06f3b0 msgr2=0x7f510c15ca50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T11:22:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f510c06f3b0 0x7f510c15ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 -- 172.21.15.181:0/3797008140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f8009550 con 0x7f510c071be0 2023-10-16T11:22:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f510b7fe700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0c9670 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 crypto rx=0x7f50f8014040 tx=0x7f50f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.649+0000 7f5108ff9700 1 -- 172.21.15.181:0/3797008140 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8004450 con 0x7f510c071be0 2023-10-16T11:22:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.650+0000 7f5108ff9700 1 -- 172.21.15.181:0/3797008140 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f801e070 con 0x7f510c071be0 2023-10-16T11:22:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.650+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c0ca1e0 con 0x7f510c071be0 2023-10-16T11:22:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.650+0000 7f5108ff9700 1 -- 172.21.15.181:0/3797008140 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80038e0 con 0x7f510c071be0 2023-10-16T11:22:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.650+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c15ad40 con 0x7f510c071be0 2023-10-16T11:22:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.651+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c0b6510 con 0x7f510c071be0 2023-10-16T11:22:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.651+0000 7f5108ff9700 1 -- 172.21.15.181:0/3797008140 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 170) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f50f8003a40 con 0x7f510c071be0 2023-10-16T11:22:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.651+0000 7f5108ff9700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 0x7f50f006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:05.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.651+0000 7f510bfff700 1 -- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 msgr2=0x7f50f006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.651+0000 7f5108ff9700 1 -- 172.21.15.181:0/3797008140 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f50f808a760 con 0x7f510c071be0 2023-10-16T11:22:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.651+0000 7f510bfff700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 0x7f50f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:05.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.656+0000 7f5108ff9700 1 -- 172.21.15.181:0/3797008140 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f50f80554e0 con 0x7f510c071be0 2023-10-16T11:22:05.665 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:05.663+0000 7fa7d928a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.852+0000 7f510bfff700 1 -- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 msgr2=0x7f50f006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:05.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.852+0000 7f510bfff700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 0x7f50f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:05.859 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.858+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f510c0cf2e0 con 0x7f510c071be0 2023-10-16T11:22:05.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.861+0000 7f5108ff9700 1 -- 172.21.15.181:0/3797008140 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v96)=0 v96) v1 ==== 133+0+0 (secure 0 0 0) 0x7f50f8058b00 con 0x7f510c071be0 2023-10-16T11:22:05.865 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 msgr2=0x7f50f006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 0x7f50f006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 msgr2=0x7f510c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0c9670 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 crypto rx=0x7f50f8014040 tx=0x7f50f8004750 comp rx=0 tx=0).stop 2023-10-16T11:22:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 shutdown_connections 2023-10-16T11:22:05.866 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7f50f006b070 0x7f50f006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f510c0b2010 0x7f510c0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f510c06f3b0 0x7f510c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 --2- 172.21.15.181:0/3797008140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f510c071be0 0x7f510c0c9670 unknown :-1 s=CLOSED pgs=3166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:05.867 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 >> 172.21.15.181:0/3797008140 conn(0x7f510c0cd010 msgr2=0x7f510c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 shutdown_connections 2023-10-16T11:22:05.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:05.863+0000 7f51128df700 1 -- 172.21.15.181:0/3797008140 wait complete. 2023-10-16T11:22:05.878 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create a sv1 2023-10-16T11:22:05.965 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:05.964+0000 7fa3cc297200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:06.003 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:06.001+0000 7fa7d928a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:06.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.002+0000 7fcaa48f1700 1 Processor -- start 2023-10-16T11:22:06.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.002+0000 7fcaa48f1700 1 -- start start 2023-10-16T11:22:06.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.002+0000 7fcaa48f1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcaa015f150 0x7fcaa00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:06.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fcaa48f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa00cd170 0x7fcaa00b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:06.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fcaa48f1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa00b2ae0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fcaa48f1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa00ce9a0 con 0x7fcaa015f150 2023-10-16T11:22:06.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fcaa48f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa00b2dc0 con 0x7fcaa00cd170 2023-10-16T11:22:06.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9e59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcaa015f150 0x7fcaa00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:06.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa00cd170 0x7fcaa00b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:06.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9dd9b700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcaa00ce4f0 0x7fcaa00b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:56680/0 (socket says 172.21.15.181:56680) 2023-10-16T11:22:06.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9dd9b700 1 -- 172.21.15.181:0/296879248 learned_addr learned my addr 172.21.15.181:0/296879248 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:06.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9ed9d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa00cd170 0x7fcaa00b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:58688/0 (socket says 172.21.15.181:58688) 2023-10-16T11:22:06.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9e59c700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcaa015f150 0x7fcaa00cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:38476/0 (socket says 172.21.15.181:38476) 2023-10-16T11:22:06.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9d59a700 1 -- 172.21.15.181:0/296879248 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420874795 0 0) 0x7fcaa00b2ae0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9d59a700 1 -- 172.21.15.181:0/296879248 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca880035b0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9e59c700 1 -- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa00cd170 msgr2=0x7fcaa00b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:06.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9e59c700 1 --2- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa00cd170 0x7fcaa00b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:06.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9e59c700 1 -- 172.21.15.181:0/296879248 >> v1:172.21.15.190:6789/0 conn(0x7fcaa00ce4f0 legacy=0x7fcaa00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:06.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9e59c700 1 -- 172.21.15.181:0/296879248 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00cf9d0 con 0x7fcaa015f150 2023-10-16T11:22:06.010 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9ed9d700 1 --2- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa00cd170 0x7fcaa00b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:06.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9e59c700 1 --2- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa015f150 0x7fcaa00cdfb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca90009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=100e51e07809de84 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:06.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.003+0000 7fca9d59a700 1 -- 172.21.15.181:0/296879248 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011830 con 0x7fcaa015f150 2023-10-16T11:22:06.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fca9d59a700 1 -- 172.21.15.181:0/296879248 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90011990 con 0x7fcaa015f150 2023-10-16T11:22:06.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fca9d59a700 1 -- 172.21.15.181:0/296879248 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011d20 con 0x7fcaa015f150 2023-10-16T11:22:06.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 -- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa015f150 msgr2=0x7fcaa00cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:06.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 --2- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa015f150 0x7fcaa00cdfb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca90009b80 comp rx=0 tx=0).stop 2023-10-16T11:22:06.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 -- 172.21.15.181:0/296879248 shutdown_connections 2023-10-16T11:22:06.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 --2- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa00cd170 0x7fcaa00b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:06.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 --2- 172.21.15.181:0/296879248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa015f150 0x7fcaa00cdfb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:06.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 -- 172.21.15.181:0/296879248 >> 172.21.15.181:0/296879248 conn(0x7fcaa009a930 msgr2=0x7fcaa00d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:06.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 -- 172.21.15.181:0/296879248 shutdown_connections 2023-10-16T11:22:06.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.004+0000 7fcaa48f1700 1 -- 172.21.15.181:0/296879248 wait complete. 2023-10-16T11:22:06.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.006+0000 7fcaa48f1700 1 Processor -- start 2023-10-16T11:22:06.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.006+0000 7fcaa48f1700 1 -- start start 2023-10-16T11:22:06.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fcaa48f1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa00cd170 0x7fcaa00c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:06.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fcaa48f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa00ce4f0 0x7fcaa01537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:06.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fcaa48f1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa015f150 0x7fcaa00af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:06.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fcaa48f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0155110 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fcaa48f1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0154070 con 0x7fcaa00cd170 2023-10-16T11:22:06.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fcaa48f1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0154350 con 0x7fcaa015f150 2023-10-16T11:22:06.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fca9dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa00ce4f0 0x7fcaa01537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:06.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fca9dd9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa00ce4f0 0x7fcaa01537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33434/0 (socket says 172.21.15.181:33434) 2023-10-16T11:22:06.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fca9dd9b700 1 -- 172.21.15.181:0/1018529786 learned_addr learned my addr 172.21.15.181:0/1018529786 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:06.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fca9ed9d700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa015f150 0x7fcaa00af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:06.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.007+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa00cd170 0x7fcaa00c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:06.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9dd9b700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa015f150 msgr2=0x7fcaa00af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:06.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9dd9b700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa015f150 0x7fcaa00af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:06.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9dd9b700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa00cd170 msgr2=0x7fcaa00c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:06.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9dd9b700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa00cd170 0x7fcaa00c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:06.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9dd9b700 1 -- 172.21.15.181:0/1018529786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98009a70 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa00cd170 0x7fcaa00c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:06.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9ed9d700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaa015f150 0x7fcaa00af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:06.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca9dd9b700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcaa00ce4f0 0x7fcaa01537e0 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 crypto rx=0x7fca9400c700 tx=0x7fca9400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:06.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94005d70 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca940070d0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fcaa48f1700 1 -- 172.21.15.181:0/1018529786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca900094a0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94007460 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.008+0000 7fcaa48f1700 1 -- 172.21.15.181:0/1018529786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa0154910 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.009+0000 7fcaa48f1700 1 -- 172.21.15.181:0/1018529786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa0067d20 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.010+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 170) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fca940075c0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.010+0000 7fca8effd700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 0x7fca8006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:06.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.010+0000 7fca9e59c700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 msgr2=0x7fca8006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:06.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.010+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(269..269 src has 1..269) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fca94088dd0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.010+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 0x7fca8006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:06.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.014+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fca94053bd0 con 0x7fcaa00ce4f0 2023-10-16T11:22:06.081 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:06.079+0000 7fa3cc297200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:06.124 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:06.123+0000 7fa7d928a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:06.153 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:06.152+0000 7fa3cc297200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:06.200 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:06.198+0000 7fa7d928a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:06.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.211+0000 7fca9e59c700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 msgr2=0x7fca8006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:06.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.211+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 0x7fca8006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:06.233 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:06.232+0000 7fa3cc297200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:06.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.246+0000 7fcaa48f1700 1 -- 172.21.15.181:0/1018529786 --> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "a", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaa00cff90 con 0x7fca8006b070 2023-10-16T11:22:06.281 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:06.280+0000 7fa7d928a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:06.600 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:06.599+0000 7fa3cc297200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:06.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.612+0000 7fca9e59c700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 msgr2=0x7fca8006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:06.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:06.612+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 0x7fca8006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T11:22:06.653 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:06.650+0000 7fa7d928a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:07.167 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:07.165+0000 7fa3cc297200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:07.218 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:07.216+0000 7fa7d928a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:07.237 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:07.235+0000 7fa3cc297200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:07.288 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:07.286+0000 7fa7d928a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:07.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:07.413+0000 7fca9e59c700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 msgr2=0x7fca8006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:07.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:07.413+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 0x7fca8006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T11:22:07.866 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:07.864+0000 7fa3cc297200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:07.916 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:07.914+0000 7fa7d928a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:07.943 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:07.942+0000 7fa3cc297200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:07.994 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:07.992+0000 7fa7d928a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:08.086 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:08.085+0000 7fa3cc297200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:08.137 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:08.135+0000 7fa7d928a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:08.315 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:08.313+0000 7fa3cc297200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:08.368 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:08.366+0000 7fa7d928a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:08.389 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:08.387+0000 7fa3cc297200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:08.442 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:08.440+0000 7fa7d928a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:08.653 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:08.651+0000 7fa3cc297200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:08.710 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:08.708+0000 7fa7d928a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:08.874 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:08.873+0000 7fa3cc297200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:08.935 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:08.933+0000 7fa7d928a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:09.014+0000 7fca9e59c700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 msgr2=0x7fca8006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291437477 2023-10-16T11:22:09.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:09.014+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 0x7fca8006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T11:22:09.024 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:09.022+0000 7fa3cc297200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:09.086 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:09.084+0000 7fa7d928a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:09.138 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:09.136+0000 7fa3cc297200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:09.201 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:09.199+0000 7fa7d928a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:09.293 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:09.292+0000 7fa3cc297200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:09.358 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:09.356+0000 7fa7d928a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:09.554 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:09.552+0000 7fa3cc297200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:09.622 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:09.620+0000 7fa7d928a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:09.714 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:09.712+0000 7fa3cc297200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:09.785 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:09.783+0000 7fa7d928a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:10.242 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:10.240+0000 7fa3cc297200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:10.319 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.317+0000 7fa7d928a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:10.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:10.340+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 171) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fca94053fa0 con 0x7fcaa00ce4f0 2023-10-16T11:22:10.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:10.340+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 msgr2=0x7fca8006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:10.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:10.340+0000 7fca8effd700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2291437477,v1:172.21.15.181:6833/2291437477] conn(0x7fca8006b070 0x7fca8006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:10.484 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74f26b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.485 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74f26b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.485 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74f26b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.485 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74f26b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.485 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74f26b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.485 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74c265700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.486 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74c265700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.486 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74c265700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.486 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74c265700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:10.486 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:10.482+0000 7fa74c265700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:11.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.342+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 172) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fca94087090 con 0x7fcaa00ce4f0 2023-10-16T11:22:11.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.342+0000 7fca8effd700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fca80035910 0x7fca800756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.342+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 --> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "a", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fca80071070 con 0x7fca80035910 2023-10-16T11:22:11.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.343+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fca80035910 0x7fca800756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.345+0000 7fca9e59c700 1 --2- 172.21.15.181:0/1018529786 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fca80035910 0x7fca800756e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca900090b0 tx=0x7fca90008030 comp rx=0 tx=0).ready entity=mgr.13629 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:11.350 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:11.348+0000 7fa74925f700 -1 mgr.server reply reply (2) No such file or directory FS 'a' not found 2023-10-16T11:22:11.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.348+0000 7fca8effd700 1 -- 172.21.15.181:0/1018529786 <== mgr.13629 v2:172.21.15.181:6832/2902800927 1 ==== mgr_command_reply(tid 0: -2 FS 'a' not found) v1 ==== 24+0+0 (secure 0 0 0) 0x7fca80071070 con 0x7fca80035910 2023-10-16T11:22:11.352 INFO:teuthology.orchestra.run.smithi181.stderr:Error ENOENT: FS 'a' not found 2023-10-16T11:22:11.356 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-16T11:22:11.357 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-16T11:22:11.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.480+0000 7f4fc080e700 1 Processor -- start 2023-10-16T11:22:11.482 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.480+0000 7f4fc080e700 1 -- start start 2023-10-16T11:22:11.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fc080e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4fbc0cb690 0x7f4fbc06f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fc080e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc070590 con 0x7f4fbc0b5fd0 2023-10-16T11:22:11.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fc080e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc156330 con 0x7f4fbc0cb2b0 2023-10-16T11:22:11.486 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fc080e700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc06fcf0 con 0x7f4fbc0cb690 2023-10-16T11:22:11.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fbad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4fbc0b5fd0 0x7f4fbc15f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44718/0 (socket says 172.21.15.181:44718) 2023-10-16T11:22:11.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fbad9d700 1 -- 172.21.15.181:0/553820243 learned_addr learned my addr 172.21.15.181:0/553820243 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:11.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763028553 0 0) 0x7f4fbc070590 con 0x7f4fbc0b5fd0 2023-10-16T11:22:11.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa40035b0 con 0x7f4fbc0b5fd0 2023-10-16T11:22:11.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.481+0000 7f4fb9d9b700 1 --2- 172.21.15.181:0/553820243 >> v2:172.21.15.190:3300/0 conn(0x7f4fbc0cb690 0x7f4fbc06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255454939 0 0) 0x7f4fa40035b0 con 0x7f4fbc0b5fd0 2023-10-16T11:22:11.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 >> v2:172.21.15.190:3300/0 conn(0x7f4fbc0cb690 msgr2=0x7f4fbc06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:11.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 --2- 172.21.15.181:0/553820243 >> v2:172.21.15.190:3300/0 conn(0x7f4fbc0cb690 0x7f4fbc06f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:11.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 >> v1:172.21.15.190:6789/0 conn(0x7f4fbc0cb2b0 legacy=0x7f4fbc0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:11.488 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc06ffd0 con 0x7f4fbc0b5fd0 2023-10-16T11:22:11.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f4fb40019f0 con 0x7f4fbc0b5fd0 2023-10-16T11:22:11.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb9d9b700 1 --2- 172.21.15.181:0/553820243 >> v2:172.21.15.190:3300/0 conn(0x7f4fbc0cb690 0x7f4fbc06f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:11.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 >> v1:172.21.15.181:6789/0 conn(0x7f4fbc0b5fd0 legacy=0x7f4fbc15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:11.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.489 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa4004850 0x7f4fa4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa4005330 0x7f4fa400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc06fcf0 con 0x7f4fa4003a00 2023-10-16T11:22:11.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa400c280 con 0x7f4fa4005330 2023-10-16T11:22:11.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa400c440 con 0x7f4fa4004850 2023-10-16T11:22:11.490 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fba59c700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb9d9b700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa4004850 0x7f4fa4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fbad9d700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa4005330 0x7f4fa400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fba59c700 1 -- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa4004850 msgr2=0x7f4fa4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:11.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fba59c700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fa4004850 0x7f4fa4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:11.491 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fba59c700 1 -- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa4005330 msgr2=0x7f4fa400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fba59c700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fa4005330 0x7f4fa400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fba59c700 1 -- 172.21.15.181:0/553820243 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40035b0 con 0x7f4fa4003a00 2023-10-16T11:22:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fbad9d700 1 -- 172.21.15.181:0/553820243 reap_dead start 2023-10-16T11:22:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fba59c700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e00 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0007fd0 tx=0x7f4fa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9da339441b726e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:11.492 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0016070 con 0x7f4fa4003a00 2023-10-16T11:22:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0006860 con 0x7f4fa4003a00 2023-10-16T11:22:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.482+0000 7f4fb959a700 1 -- 172.21.15.181:0/553820243 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003950 con 0x7f4fa4003a00 2023-10-16T11:22:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.483+0000 7f4fc080e700 1 -- 172.21.15.181:0/553820243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 msgr2=0x7f4fa4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.483+0000 7f4fc080e700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e00 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0007fd0 tx=0x7f4fa4005880 comp rx=0 tx=0).stop 2023-10-16T11:22:11.493 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.483+0000 7f4fc080e700 1 -- 172.21.15.181:0/553820243 shutdown_connections 2023-10-16T11:22:11.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.483+0000 7f4fc080e700 1 --2- 172.21.15.181:0/553820243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e00 unknown :-1 s=CLOSED pgs=3176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:11.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.483+0000 7f4fc080e700 1 -- 172.21.15.181:0/553820243 >> 172.21.15.181:0/553820243 conn(0x7f4fbc0cd010 msgr2=0x7f4fbc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:11.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.483+0000 7f4fc080e700 1 -- 172.21.15.181:0/553820243 shutdown_connections 2023-10-16T11:22:11.494 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.483+0000 7f4fc080e700 1 -- 172.21.15.181:0/553820243 wait complete. 2023-10-16T11:22:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.484+0000 7f4fc080e700 1 Processor -- start 2023-10-16T11:22:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.484+0000 7f4fc080e700 1 -- start start 2023-10-16T11:22:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fc080e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fbc15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fc080e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fbc153680 0x7f4fbc152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.497 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fc080e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fbc15de80 0x7f4fbc15e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fc080e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc159dc0 con 0x7f4fa4003a00 2023-10-16T11:22:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fc080e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15e850 con 0x7f4fbc153680 2023-10-16T11:22:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fc080e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15c900 con 0x7f4fbc15de80 2023-10-16T11:22:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fbc15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.498 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fbc15b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33652/0 (socket says 172.21.15.181:33652) 2023-10-16T11:22:11.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 -- 172.21.15.181:0/1480567206 learned_addr learned my addr 172.21.15.181:0/1480567206 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:11.499 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fbad9d700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fbc15de80 0x7f4fbc15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fb9d9b700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fbc153680 0x7f4fbc152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 -- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fbc15de80 msgr2=0x7f4fbc15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fbc15de80 0x7f4fbc15e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 -- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fbc153680 msgr2=0x7f4fbc152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:11.500 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fbc153680 0x7f4fbc152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.485+0000 7f4fba59c700 1 -- 172.21.15.181:0/1480567206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc153450 con 0x7f4fa4003a00 2023-10-16T11:22:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.486+0000 7f4fbad9d700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fbc15de80 0x7f4fbc15e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.486+0000 7f4fba59c700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fbc15b180 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c250 tx=0x7f4fb0003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:11.501 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.486+0000 7f4faaffd700 1 -- 172.21.15.181:0/1480567206 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0016040 con 0x7f4fa4003a00 2023-10-16T11:22:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.486+0000 7f4faaffd700 1 -- 172.21.15.181:0/1480567206 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0007530 con 0x7f4fa4003a00 2023-10-16T11:22:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.486+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc15b860 con 0x7f4fa4003a00 2023-10-16T11:22:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.486+0000 7f4faaffd700 1 -- 172.21.15.181:0/1480567206 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0012420 con 0x7f4fa4003a00 2023-10-16T11:22:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.486+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc15be20 con 0x7f4fa4003a00 2023-10-16T11:22:11.502 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.487+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc0b28b0 con 0x7f4fa4003a00 2023-10-16T11:22:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.487+0000 7f4faaffd700 1 -- 172.21.15.181:0/1480567206 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 172) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f4fb0012580 con 0x7f4fa4003a00 2023-10-16T11:22:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.488+0000 7f4faaffd700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4f9c06ada0 0x7f4f9c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.488+0000 7f4fb9d9b700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4f9c06ada0 0x7f4f9c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.488+0000 7f4faaffd700 1 -- 172.21.15.181:0/1480567206 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f4fb008a330 con 0x7f4fa4003a00 2023-10-16T11:22:11.503 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.488+0000 7f4fb9d9b700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4f9c06ada0 0x7f4f9c06d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0cbaa0 tx=0x7f4fac008040 comp rx=0 tx=0).ready entity=mgr.13629 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:11.504 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.492+0000 7f4faaffd700 1 -- 172.21.15.181:0/1480567206 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4fb00554c0 con 0x7f4fa4003a00 2023-10-16T11:22:11.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:11.740+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f4fbc153e60 con 0x7f4fa4003a00 2023-10-16T11:22:12.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.343+0000 7f4faaffd700 1 -- 172.21.15.181:0/1480567206 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v173) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4fb0058ae0 con 0x7f4fa4003a00 2023-10-16T11:22:12.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4f9c06ada0 msgr2=0x7f4f9c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:12.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4f9c06ada0 0x7f4f9c06d260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0cbaa0 tx=0x7f4fac008040 comp rx=0 tx=0).stop 2023-10-16T11:22:12.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 msgr2=0x7f4fbc15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:12.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fbc15b180 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c250 tx=0x7f4fb0003780 comp rx=0 tx=0).stop 2023-10-16T11:22:12.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 shutdown_connections 2023-10-16T11:22:12.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4f9c06ada0 0x7f4f9c06d260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fbc15de80 0x7f4fbc15e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003a00 0x7f4fbc15b180 unknown :-1 s=CLOSED pgs=3177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 --2- 172.21.15.181:0/1480567206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fbc153680 0x7f4fbc152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 >> 172.21.15.181:0/1480567206 conn(0x7f4fbc0cd010 msgr2=0x7f4fbc155870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.347+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 shutdown_connections 2023-10-16T11:22:12.351 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.348+0000 7f4fc080e700 1 -- 172.21.15.181:0/1480567206 wait complete. 2023-10-16T11:22:12.365 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-16T11:22:12.365 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:22:12.408 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-16T11:22:12.445 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-16T11:22:12.445 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:22:12.446 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:12.464 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:22:12.465 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:12.495 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:12.493+0000 7fb3a4504200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:12.496 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:12.494+0000 7f26b8b55200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T11:22:12.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:12.560 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:12.559+0000 7f26b8b55200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:12.563 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:12.561+0000 7fb3a4504200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:12.633 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:12.631+0000 7f26b8b55200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:12.638 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:12.636+0000 7fb3a4504200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T11:22:12.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.674+0000 7f977b664700 1 Processor -- start 2023-10-16T11:22:12.676 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.674+0000 7f977b664700 1 -- start start 2023-10-16T11:22:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f977b664700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977415dc20 0x7f97740d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f977b664700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97740d1e60 0x7f97740d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f977b664700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f97740ce4b0 0x7f97740ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f977b664700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97740b3b80 con 0x7f97740ce4b0 2023-10-16T11:22:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f977b664700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97740d2820 con 0x7f97740d1e60 2023-10-16T11:22:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f977b664700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740b2f00 con 0x7f977415dc20 2023-10-16T11:22:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f9778bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97740d1e60 0x7f97740d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f9779400700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977415dc20 0x7f97740d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:12.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f9779400700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977415dc20 0x7f97740d41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:59000/0 (socket says 172.21.15.181:59000) 2023-10-16T11:22:12.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f9778bff700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97740d1e60 0x7f97740d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:38792/0 (socket says 172.21.15.181:38792) 2023-10-16T11:22:12.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f9779400700 1 -- 172.21.15.181:0/2979196792 learned_addr learned my addr 172.21.15.181:0/2979196792 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:12.682 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.675+0000 7f9779c01700 1 --2- 172.21.15.181:0/2979196792 >> v2:172.21.15.181:3300/0 conn(0x7f97740ce4b0 0x7f97740ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f9778bff700 1 -- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977415dc20 msgr2=0x7f97740d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f9778bff700 1 --2- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977415dc20 0x7f97740d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f9778bff700 1 -- 172.21.15.181:0/2979196792 >> v2:172.21.15.181:3300/0 conn(0x7f97740ce4b0 msgr2=0x7f97740ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f9778bff700 1 --2- 172.21.15.181:0/2979196792 >> v2:172.21.15.181:3300/0 conn(0x7f97740ce4b0 0x7f97740ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f9778bff700 1 -- 172.21.15.181:0/2979196792 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740b31e0 con 0x7f97740d1e60 2023-10-16T11:22:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f9779400700 1 --2- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977415dc20 0x7f97740d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:12.685 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f9778bff700 1 --2- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97740d22e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f977406b790 tx=0x7f9764009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd224b76788ae712 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f976bfff700 1 -- 172.21.15.181:0/2979196792 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764011830 con 0x7f97740d1e60 2023-10-16T11:22:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f976bfff700 1 -- 172.21.15.181:0/2979196792 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9764011990 con 0x7f97740d1e60 2023-10-16T11:22:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.676+0000 7f976bfff700 1 -- 172.21.15.181:0/2979196792 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764011d20 con 0x7f97740d1e60 2023-10-16T11:22:12.686 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 -- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 msgr2=0x7f97740d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 --2- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97740d22e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f977406b790 tx=0x7f9764009b20 comp rx=0 tx=0).stop 2023-10-16T11:22:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 -- 172.21.15.181:0/2979196792 shutdown_connections 2023-10-16T11:22:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 --2- 172.21.15.181:0/2979196792 >> v2:172.21.15.181:3300/0 conn(0x7f97740ce4b0 0x7f97740ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 --2- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f977415dc20 0x7f97740d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.687 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 --2- 172.21.15.181:0/2979196792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97740d22e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 -- 172.21.15.181:0/2979196792 >> 172.21.15.181:0/2979196792 conn(0x7f977408af90 msgr2=0x7f97740d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 -- 172.21.15.181:0/2979196792 shutdown_connections 2023-10-16T11:22:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.677+0000 7f977b664700 1 -- 172.21.15.181:0/2979196792 wait complete. 2023-10-16T11:22:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.678+0000 7f977b664700 1 Processor -- start 2023-10-16T11:22:12.688 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.678+0000 7f977b664700 1 -- start start 2023-10-16T11:22:12.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f977b664700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97740ce4b0 0x7f97740c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:12.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f977b664700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97741609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:12.691 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f977b664700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f977415dc20 0x7f977409d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f977b664700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97740c99b0 con 0x7f977415dc20 2023-10-16T11:22:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f977b664700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977415e000 con 0x7f97740d1e60 2023-10-16T11:22:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f977b664700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977415e2e0 con 0x7f97740ce4b0 2023-10-16T11:22:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f977415dc20 0x7f977409d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:12.692 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f977415dc20 0x7f977409d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33752/0 (socket says 172.21.15.181:33752) 2023-10-16T11:22:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779400700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97740ce4b0 0x7f97740c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 -- 172.21.15.181:0/2550845663 learned_addr learned my addr 172.21.15.181:0/2550845663 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9778bff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97741609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 -- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97740ce4b0 msgr2=0x7f97740c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:12.693 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97740ce4b0 0x7f97740c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 -- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 msgr2=0x7f97741609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97741609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 -- 172.21.15.181:0/2550845663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770009500 con 0x7f977415dc20 2023-10-16T11:22:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779400700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97740ce4b0 0x7f97740c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9778bff700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97741609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:12.694 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.679+0000 7f9779c01700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f977415dc20 0x7f977409d590 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7f976c00eed0 tx=0x7f976c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.680+0000 7f9769ffb700 1 -- 172.21.15.181:0/2550845663 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c010040 con 0x7f977415dc20 2023-10-16T11:22:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.680+0000 7f9769ffb700 1 -- 172.21.15.181:0/2550845663 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c00b9f0 con 0x7f977415dc20 2023-10-16T11:22:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.680+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97640094a0 con 0x7f977415dc20 2023-10-16T11:22:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.680+0000 7f9769ffb700 1 -- 172.21.15.181:0/2550845663 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00bda0 con 0x7f977415dc20 2023-10-16T11:22:12.695 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.680+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977415e8a0 con 0x7f977415dc20 2023-10-16T11:22:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.681+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97740008d0 con 0x7f977415dc20 2023-10-16T11:22:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.681+0000 7f9769ffb700 1 -- 172.21.15.181:0/2550845663 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 173) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f976c014400 con 0x7f977415dc20 2023-10-16T11:22:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.682+0000 7f9769ffb700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 0x7f975806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.682+0000 7f9779400700 1 -- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 msgr2=0x7f975806d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.682+0000 7f9769ffb700 1 -- 172.21.15.181:0/2550845663 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f976c0898b0 con 0x7f977415dc20 2023-10-16T11:22:12.696 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.682+0000 7f9779400700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 0x7f975806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:12.697 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.686+0000 7f9769ffb700 1 -- 172.21.15.181:0/2550845663 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f976c0548a0 con 0x7f977415dc20 2023-10-16T11:22:12.711 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:12.709+0000 7f26b8b55200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:12.722 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:12.720+0000 7fb3a4504200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T11:22:12.851 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:12.849+0000 7f26b8b55200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:12.867 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:12.865+0000 7fb3a4504200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T11:22:12.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.883+0000 7f9779400700 1 -- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 msgr2=0x7f975806d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:12.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.883+0000 7f9779400700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 0x7f975806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:12.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.887+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f977406a9c0 con 0x7f977415dc20 2023-10-16T11:22:12.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.888+0000 7f9769ffb700 1 -- 172.21.15.181:0/2550845663 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v270) v1 ==== 74+0+17919 (secure 0 0 0) 0x7f976c057ec0 con 0x7f977415dc20 2023-10-16T11:22:12.890 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:12.891 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":270,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:22:10.332242+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:54.910438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:55.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2735196384":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6833/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6832/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/691655073":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/3571285885":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/171534192":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/714342180":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/432126216":"2023-10-17T11:22:10.332184+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:12.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 msgr2=0x7f975806d210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:12.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 0x7f975806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f977415dc20 msgr2=0x7f977409d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:12.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f977415dc20 0x7f977409d590 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7f976c00eed0 tx=0x7f976c00ef00 comp rx=0 tx=0).stop 2023-10-16T11:22:12.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 shutdown_connections 2023-10-16T11:22:12.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f975806ad50 0x7f975806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97740ce4b0 0x7f97740c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f977415dc20 0x7f977409d590 unknown :-1 s=CLOSED pgs=3182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 --2- 172.21.15.181:0/2550845663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97740d1e60 0x7f97741609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:12.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 >> 172.21.15.181:0/2550845663 conn(0x7f977408af90 msgr2=0x7f97740b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:12.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 shutdown_connections 2023-10-16T11:22:12.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:12.890+0000 7f977b664700 1 -- 172.21.15.181:0/2550845663 wait complete. 2023-10-16T11:22:12.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:21:54.910438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '268', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 37, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:21:55.443301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '269', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:22:12.906 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:22:13.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.070+0000 7ff68b1c8700 1 Processor -- start 2023-10-16T11:22:13.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.070+0000 7ff68b1c8700 1 -- start start 2023-10-16T11:22:13.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.071+0000 7ff68b1c8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff684071be0 0x7ff6840ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.071+0000 7ff68b1c8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840ce1f0 con 0x7ff6840b2010 2023-10-16T11:22:13.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.071+0000 7ff68b1c8700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840ccda0 con 0x7ff6840cc6e0 2023-10-16T11:22:13.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.071+0000 7ff68b1c8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6840ccb90 con 0x7ff684071be0 2023-10-16T11:22:13.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.071+0000 7ff688f64700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff6840b2010 0x7ff6840b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44864/0 (socket says 172.21.15.181:44864) 2023-10-16T11:22:13.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.071+0000 7ff688f64700 1 -- 172.21.15.181:0/613493435 learned_addr learned my addr 172.21.15.181:0/613493435 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:13.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.071+0000 7ff689765700 1 --2- 172.21.15.181:0/613493435 >> v2:172.21.15.181:3300/0 conn(0x7ff684071be0 0x7ff6840ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff689765700 1 -- 172.21.15.181:0/613493435 >> v1:172.21.15.190:6789/0 conn(0x7ff6840cc6e0 legacy=0x7ff6840ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff689765700 1 -- 172.21.15.181:0/613493435 >> v1:172.21.15.181:6789/0 conn(0x7ff6840b2010 legacy=0x7ff6840b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff6837fe700 1 -- 172.21.15.181:0/613493435 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350823416 0 0) 0x7ff6840ccda0 con 0x7ff6840cc6e0 2023-10-16T11:22:13.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff689765700 1 -- 172.21.15.181:0/613493435 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840727e0 con 0x7ff684071be0 2023-10-16T11:22:13.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff689765700 1 --2- 172.21.15.181:0/613493435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff684071be0 0x7ff6840ced90 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009d90 tx=0x7ff67c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec6da59e738f64d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:13.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff6837fe700 1 -- 172.21.15.181:0/613493435 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00e070 con 0x7ff684071be0 2023-10-16T11:22:13.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff6837fe700 1 -- 172.21.15.181:0/613493435 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c00bad0 con 0x7ff684071be0 2023-10-16T11:22:13.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.072+0000 7ff6837fe700 1 -- 172.21.15.181:0/613493435 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00be80 con 0x7ff684071be0 2023-10-16T11:22:13.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.073+0000 7ff68b1c8700 1 -- 172.21.15.181:0/613493435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff684071be0 msgr2=0x7ff6840ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.073+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/613493435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff684071be0 0x7ff6840ced90 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009d90 tx=0x7ff67c00b330 comp rx=0 tx=0).stop 2023-10-16T11:22:13.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.073+0000 7ff68b1c8700 1 -- 172.21.15.181:0/613493435 shutdown_connections 2023-10-16T11:22:13.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.073+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/613493435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff684071be0 0x7ff6840ced90 unknown :-1 s=CLOSED pgs=3184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.073+0000 7ff68b1c8700 1 -- 172.21.15.181:0/613493435 >> 172.21.15.181:0/613493435 conn(0x7ff6840cd010 msgr2=0x7ff6840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:13.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.073+0000 7ff68b1c8700 1 -- 172.21.15.181:0/613493435 shutdown_connections 2023-10-16T11:22:13.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.073+0000 7ff68b1c8700 1 -- 172.21.15.181:0/613493435 wait complete. 2023-10-16T11:22:13.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.075+0000 7ff68b1c8700 1 Processor -- start 2023-10-16T11:22:13.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.075+0000 7ff68b1c8700 1 -- start start 2023-10-16T11:22:13.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.076+0000 7ff68b1c8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff684071be0 0x7ff6841f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.076+0000 7ff68b1c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6840b2010 0x7ff6841483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.076+0000 7ff68b1c8700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6840cc6e0 0x7ff6840c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff68b1c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6841f1e30 con 0x7ff6840b2010 2023-10-16T11:22:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff68b1c8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68414a440 con 0x7ff684071be0 2023-10-16T11:22:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff68b1c8700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6841f48d0 con 0x7ff6840cc6e0 2023-10-16T11:22:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6840b2010 0x7ff6841483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6840b2010 0x7ff6841483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33794/0 (socket says 172.21.15.181:33794) 2023-10-16T11:22:13.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 -- 172.21.15.181:0/3456327949 learned_addr learned my addr 172.21.15.181:0/3456327949 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:13.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff688f64700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff684071be0 0x7ff6841f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff689765700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6840cc6e0 0x7ff6840c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 -- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6840cc6e0 msgr2=0x7ff6840c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6840cc6e0 0x7ff6840c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 -- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff684071be0 msgr2=0x7ff6841f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff684071be0 0x7ff6841f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 -- 172.21.15.181:0/3456327949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c009a70 con 0x7ff6840b2010 2023-10-16T11:22:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff683fff700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6840b2010 0x7ff6841483f0 secure :-1 s=READY pgs=3185 cs=0 l=1 rev1=1 crypto rx=0x7ff67000c520 tx=0x7ff67000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff6817fa700 1 -- 172.21.15.181:0/3456327949 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700054e0 con 0x7ff6840b2010 2023-10-16T11:22:13.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff6817fa700 1 -- 172.21.15.181:0/3456327949 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670003680 con 0x7ff6840b2010 2023-10-16T11:22:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6841f4bb0 con 0x7ff6840b2010 2023-10-16T11:22:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff6817fa700 1 -- 172.21.15.181:0/3456327949 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700039f0 con 0x7ff6840b2010 2023-10-16T11:22:13.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.077+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6841f5170 con 0x7ff6840b2010 2023-10-16T11:22:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.079+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68414a5d0 con 0x7ff6840b2010 2023-10-16T11:22:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.079+0000 7ff6817fa700 1 -- 172.21.15.181:0/3456327949 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 173) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff670003b50 con 0x7ff6840b2010 2023-10-16T11:22:13.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.079+0000 7ff6817fa700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 0x7ff66806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.079+0000 7ff688f64700 1 -- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 msgr2=0x7ff66806d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:13.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.079+0000 7ff6817fa700 1 -- 172.21.15.181:0/3456327949 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7ff6700888b0 con 0x7ff6840b2010 2023-10-16T11:22:13.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.079+0000 7ff688f64700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 0x7ff66806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:13.088 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.084+0000 7ff6817fa700 1 -- 172.21.15.181:0/3456327949 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff670053950 con 0x7ff6840b2010 2023-10-16T11:22:13.179 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:13.176+0000 7f26b8b55200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:13.221 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:13.219+0000 7fb3a4504200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T11:22:13.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.280+0000 7ff688f64700 1 -- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 msgr2=0x7ff66806d210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:13.282 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.280+0000 7ff688f64700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 0x7ff66806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:13.294 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.292+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6840b6510 con 0x7ff6840b2010 2023-10-16T11:22:13.295 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:13.292+0000 7f26b8b55200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:13.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.293+0000 7ff6817fa700 1 -- 172.21.15.181:0/3456327949 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v270) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff670056f70 con 0x7ff6840b2010 2023-10-16T11:22:13.295 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:22:13.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 msgr2=0x7ff66806d210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:13.297 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 0x7ff66806d210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6840b2010 msgr2=0x7ff6841483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6840b2010 0x7ff6841483f0 secure :-1 s=READY pgs=3185 cs=0 l=1 rev1=1 crypto rx=0x7ff67000c520 tx=0x7ff67000c830 comp rx=0 tx=0).stop 2023-10-16T11:22:13.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 shutdown_connections 2023-10-16T11:22:13.298 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff66806ad50 0x7ff66806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6840cc6e0 0x7ff6840c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff684071be0 0x7ff6841f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 --2- 172.21.15.181:0/3456327949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6840b2010 0x7ff6841483f0 unknown :-1 s=CLOSED pgs=3185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 >> 172.21.15.181:0/3456327949 conn(0x7ff6840cd010 msgr2=0x7ff6840b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:13.299 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 shutdown_connections 2023-10-16T11:22:13.300 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.295+0000 7ff68b1c8700 1 -- 172.21.15.181:0/3456327949 wait complete. 2023-10-16T11:22:13.311 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:22:13.342 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:13.340+0000 7fb3a4504200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T11:22:13.370 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:13.367+0000 7f26b8b55200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:13.421 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:13.419+0000 7fb3a4504200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T11:22:13.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.437+0000 7f385bffd700 1 Processor -- start 2023-10-16T11:22:13.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.438+0000 7f385bffd700 1 -- start start 2023-10-16T11:22:13.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f385bffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38540cb690 0x7f385406f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.442 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f385bffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540b5fd0 0x7f385415f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f385bffd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3854070590 con 0x7f38540cb2b0 2023-10-16T11:22:13.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f385bffd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3854156330 con 0x7f38540cb690 2023-10-16T11:22:13.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f385bffd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385406fcf0 con 0x7f38540b5fd0 2023-10-16T11:22:13.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859598700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38540cb690 0x7f385406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859d99700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f38540cb2b0 0x7f38540b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44880/0 (socket says 172.21.15.181:44880) 2023-10-16T11:22:13.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859d99700 1 -- 172.21.15.181:0/2830965173 learned_addr learned my addr 172.21.15.181:0/2830965173 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:13.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f385a59a700 1 --2- 172.21.15.181:0/2830965173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540b5fd0 0x7f385415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859598700 1 -- 172.21.15.181:0/2830965173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540b5fd0 msgr2=0x7f385415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:13.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859598700 1 --2- 172.21.15.181:0/2830965173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540b5fd0 0x7f385415f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859598700 1 -- 172.21.15.181:0/2830965173 >> v1:172.21.15.181:6789/0 conn(0x7f38540cb2b0 legacy=0x7f38540b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859598700 1 -- 172.21.15.181:0/2830965173 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385406ffd0 con 0x7f38540cb690 2023-10-16T11:22:13.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3858d97700 1 -- 172.21.15.181:0/2830965173 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618202604 0 0) 0x7f3854070590 con 0x7f38540cb2b0 2023-10-16T11:22:13.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.439+0000 7f3859598700 1 --2- 172.21.15.181:0/2830965173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385406f710 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f3844009840 tx=0x7f3844009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0bd677162c465c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:13.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f3858d97700 1 -- 172.21.15.181:0/2830965173 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844011830 con 0x7f38540cb690 2023-10-16T11:22:13.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f3858d97700 1 -- 172.21.15.181:0/2830965173 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844011990 con 0x7f38540cb690 2023-10-16T11:22:13.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f3858d97700 1 -- 172.21.15.181:0/2830965173 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844011d20 con 0x7f38540cb690 2023-10-16T11:22:13.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 -- 172.21.15.181:0/2830965173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 msgr2=0x7f385406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 --2- 172.21.15.181:0/2830965173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385406f710 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f3844009840 tx=0x7f3844009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:13.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 -- 172.21.15.181:0/2830965173 shutdown_connections 2023-10-16T11:22:13.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 --2- 172.21.15.181:0/2830965173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540b5fd0 0x7f385415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 --2- 172.21.15.181:0/2830965173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385406f710 unknown :-1 s=CLOSED pgs=3187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 -- 172.21.15.181:0/2830965173 >> 172.21.15.181:0/2830965173 conn(0x7f38540cd010 msgr2=0x7f38540cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:13.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 -- 172.21.15.181:0/2830965173 shutdown_connections 2023-10-16T11:22:13.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.440+0000 7f385bffd700 1 -- 172.21.15.181:0/2830965173 wait complete. 2023-10-16T11:22:13.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.441+0000 7f385bffd700 1 Processor -- start 2023-10-16T11:22:13.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385bffd700 1 -- start start 2023-10-16T11:22:13.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385bffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38540b5fd0 0x7f3854157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385bffd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540cb2b0 0x7f385415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385bffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385bffd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385415b6a0 con 0x7f38540cb690 2023-10-16T11:22:13.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385bffd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385415aa20 con 0x7f38540b5fd0 2023-10-16T11:22:13.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385bffd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385415ad00 con 0x7f38540cb2b0 2023-10-16T11:22:13.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385a59a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f3859598700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540cb2b0 0x7f385415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385a59a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385415dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33812/0 (socket says 172.21.15.181:33812) 2023-10-16T11:22:13.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f385a59a700 1 -- 172.21.15.181:0/3388509790 learned_addr learned my addr 172.21.15.181:0/3388509790 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:13.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.442+0000 7f3859d99700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38540b5fd0 0x7f3854157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f3859598700 1 -- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38540b5fd0 msgr2=0x7f3854157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f3859598700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38540b5fd0 0x7f3854157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f3859598700 1 -- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 msgr2=0x7f385415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f3859598700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385415dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f3859598700 1 -- 172.21.15.181:0/3388509790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440094a0 con 0x7f38540cb2b0 2023-10-16T11:22:13.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f3859598700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540cb2b0 0x7f385415d7e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3844016000 tx=0x7f3844004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:13.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f384a7fc700 1 -- 172.21.15.181:0/3388509790 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844004550 con 0x7f38540cb2b0 2023-10-16T11:22:13.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f384a7fc700 1 -- 172.21.15.181:0/3388509790 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384401e070 con 0x7f38540cb2b0 2023-10-16T11:22:13.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385415afe0 con 0x7f38540cb2b0 2023-10-16T11:22:13.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f384a7fc700 1 -- 172.21.15.181:0/3388509790 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38440038e0 con 0x7f38540cb2b0 2023-10-16T11:22:13.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.443+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854158480 con 0x7f38540cb2b0 2023-10-16T11:22:13.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.444+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385406f4f0 con 0x7f38540cb2b0 2023-10-16T11:22:13.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.444+0000 7f384a7fc700 1 -- 172.21.15.181:0/3388509790 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3844003a40 con 0x7f38540cb2b0 2023-10-16T11:22:13.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.445+0000 7f384a7fc700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 0x7f383806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.445+0000 7f3859d99700 1 -- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 msgr2=0x7f383806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.445+0000 7f384a7fc700 1 -- 172.21.15.181:0/3388509790 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f384408a920 con 0x7f38540cb2b0 2023-10-16T11:22:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.445+0000 7f3859d99700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 0x7f383806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:13.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.449+0000 7f384a7fc700 1 -- 172.21.15.181:0/3388509790 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f38440555d0 con 0x7f38540cb2b0 2023-10-16T11:22:13.463 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:13.447+0000 7f26b8b55200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:13.506 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:13.504+0000 7fb3a4504200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T11:22:13.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.646+0000 7f3859d99700 1 -- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 msgr2=0x7f383806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:13.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.646+0000 7f3859d99700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 0x7f383806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.647+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3854070280 con 0x7f38540cb2b0 2023-10-16T11:22:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.648+0000 7f384a7fc700 1 -- 172.21.15.181:0/3388509790 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v270) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3844058bf0 con 0x7f38540cb2b0 2023-10-16T11:22:13.650 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:22:13.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 msgr2=0x7f383806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:13.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 0x7f383806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540cb2b0 msgr2=0x7f385415d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540cb2b0 0x7f385415d7e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3844016000 tx=0x7f3844004750 comp rx=0 tx=0).stop 2023-10-16T11:22:13.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 shutdown_connections 2023-10-16T11:22:13.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f383806b020 0x7f383806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38540cb690 0x7f385415dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38540b5fd0 0x7f3854157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 --2- 172.21.15.181:0/3388509790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38540cb2b0 0x7f385415d7e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 >> 172.21.15.181:0/3388509790 conn(0x7f38540cd010 msgr2=0x7f3854072940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:13.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 shutdown_connections 2023-10-16T11:22:13.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.650+0000 7f385bffd700 1 -- 172.21.15.181:0/3388509790 wait complete. 2023-10-16T11:22:13.665 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:22:13.819 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:13.816+0000 7f26b8b55200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:13.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.832+0000 7fa1c739e700 1 Processor -- start 2023-10-16T11:22:13.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.832+0000 7fa1c739e700 1 -- start start 2023-10-16T11:22:13.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.833+0000 7fa1c739e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.833+0000 7fa1c739e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1c006d8c0 0x7fa1c006abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.833+0000 7fa1c739e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c006dd50 con 0x7fa1c0159a00 2023-10-16T11:22:13.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.833+0000 7fa1c739e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00cb080 con 0x7fa1c006d8c0 2023-10-16T11:22:13.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.833+0000 7fa1c739e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00cbbb0 con 0x7fa1c0159de0 2023-10-16T11:22:13.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c6b9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1c006d8c0 0x7fa1c006abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c5b9b700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c639c700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa1c0159a00 0x7fa1c00cb4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:57098/0 (socket says 172.21.15.181:57098) 2023-10-16T11:22:13.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c6b9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1c006d8c0 0x7fa1c006abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33850/0 (socket says 172.21.15.181:33850) 2023-10-16T11:22:13.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c639c700 1 -- 172.21.15.181:0/548076672 learned_addr learned my addr 172.21.15.181:0/548076672 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:13.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c539a700 1 -- 172.21.15.181:0/548076672 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98681754 0 0) 0x7fa1bc001110 con 0x7fa1c0159a00 2023-10-16T11:22:13.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c5b9b700 1 -- 172.21.15.181:0/548076672 >> v2:172.21.15.181:3300/0 conn(0x7fa1c006d8c0 msgr2=0x7fa1c006abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:13.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c5b9b700 1 --2- 172.21.15.181:0/548076672 >> v2:172.21.15.181:3300/0 conn(0x7fa1c006d8c0 0x7fa1c006abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c5b9b700 1 -- 172.21.15.181:0/548076672 >> v1:172.21.15.190:6789/0 conn(0x7fa1c0159a00 legacy=0x7fa1c00cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c5b9b700 1 -- 172.21.15.181:0/548076672 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00ce910 con 0x7fa1c0159de0 2023-10-16T11:22:13.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c6b9d700 1 --2- 172.21.15.181:0/548076672 >> v2:172.21.15.181:3300/0 conn(0x7fa1c006d8c0 0x7fa1c006abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:13.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c5b9b700 1 --2- 172.21.15.181:0/548076672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c006d380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009840 tx=0x7fa1b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4113b15d6375253b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:13.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c539a700 1 -- 172.21.15.181:0/548076672 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0011830 con 0x7fa1c0159de0 2023-10-16T11:22:13.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c539a700 1 -- 172.21.15.181:0/548076672 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0011990 con 0x7fa1c0159de0 2023-10-16T11:22:13.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.834+0000 7fa1c539a700 1 -- 172.21.15.181:0/548076672 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0011d20 con 0x7fa1c0159de0 2023-10-16T11:22:13.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 -- 172.21.15.181:0/548076672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 msgr2=0x7fa1c006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 --2- 172.21.15.181:0/548076672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c006d380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009840 tx=0x7fa1b0009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:13.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 -- 172.21.15.181:0/548076672 shutdown_connections 2023-10-16T11:22:13.844 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 --2- 172.21.15.181:0/548076672 >> v2:172.21.15.181:3300/0 conn(0x7fa1c006d8c0 0x7fa1c006abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 --2- 172.21.15.181:0/548076672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c006d380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 -- 172.21.15.181:0/548076672 >> 172.21.15.181:0/548076672 conn(0x7fa1c00cfb60 msgr2=0x7fa1c00cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:13.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 -- 172.21.15.181:0/548076672 shutdown_connections 2023-10-16T11:22:13.845 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.835+0000 7fa1c739e700 1 -- 172.21.15.181:0/548076672 wait complete. 2023-10-16T11:22:13.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.836+0000 7fa1c739e700 1 Processor -- start 2023-10-16T11:22:13.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.836+0000 7fa1c739e700 1 -- start start 2023-10-16T11:22:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c739e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c006d8c0 0x7fa1c0146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c739e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1c0159a00 0x7fa1c01e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c739e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c01e3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c739e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00c3a00 con 0x7fa1c0159a00 2023-10-16T11:22:13.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c739e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00c3ce0 con 0x7fa1c006d8c0 2023-10-16T11:22:13.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c739e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00c3fc0 con 0x7fa1c0159de0 2023-10-16T11:22:13.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c5b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1c0159a00 0x7fa1c01e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c5b9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1c0159a00 0x7fa1c01e33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33854/0 (socket says 172.21.15.181:33854) 2023-10-16T11:22:13.851 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c5b9b700 1 -- 172.21.15.181:0/144580591 learned_addr learned my addr 172.21.15.181:0/144580591 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:13.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c639c700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c006d8c0 0x7fa1c0146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.837+0000 7fa1c6b9d700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c01e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:13.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c5b9b700 1 -- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 msgr2=0x7fa1c01e3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c5b9b700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c01e3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c5b9b700 1 -- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c006d8c0 msgr2=0x7fa1c0146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:13.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c5b9b700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c006d8c0 0x7fa1c0146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:13.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c5b9b700 1 -- 172.21.15.181:0/144580591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b800ab80 con 0x7fa1c0159a00 2023-10-16T11:22:13.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c5b9b700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1c0159a00 0x7fa1c01e33c0 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009b90 tx=0x7fa1b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:13.853 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1b6ffd700 1 -- 172.21.15.181:0/144580591 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00036a0 con 0x7fa1c0159a00 2023-10-16T11:22:13.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1b6ffd700 1 -- 172.21.15.181:0/144580591 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b001e070 con 0x7fa1c0159a00 2023-10-16T11:22:13.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b00094a0 con 0x7fa1c0159a00 2023-10-16T11:22:13.854 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1b6ffd700 1 -- 172.21.15.181:0/144580591 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0003a60 con 0x7fa1c0159a00 2023-10-16T11:22:13.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.838+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c00c4580 con 0x7fa1c0159a00 2023-10-16T11:22:13.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.839+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c00c47a0 con 0x7fa1c0159a00 2023-10-16T11:22:13.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.840+0000 7fa1b6ffd700 1 -- 172.21.15.181:0/144580591 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa1b0003bc0 con 0x7fa1c0159a00 2023-10-16T11:22:13.855 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.841+0000 7fa1b6ffd700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 0x7fa1a406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:13.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.841+0000 7fa1c639c700 1 -- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 msgr2=0x7fa1a406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:13.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.841+0000 7fa1b6ffd700 1 -- 172.21.15.181:0/144580591 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fa1b008ae40 con 0x7fa1c0159a00 2023-10-16T11:22:13.856 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.841+0000 7fa1c639c700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 0x7fa1a406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:13.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:13.844+0000 7fa1b6ffd700 1 -- 172.21.15.181:0/144580591 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa1b0055a50 con 0x7fa1c0159a00 2023-10-16T11:22:13.895 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:13.893+0000 7fb3a4504200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T11:22:14.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.042+0000 7fa1c639c700 1 -- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 msgr2=0x7fa1a406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:14.044 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.042+0000 7fa1c639c700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 0x7fa1a406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:14.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.045+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa1c004f400 con 0x7fa1c0159a00 2023-10-16T11:22:14.047 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.045+0000 7fa1b6ffd700 1 -- 172.21.15.181:0/144580591 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v270) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa1b0059070 con 0x7fa1c0159a00 2023-10-16T11:22:14.047 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:22:14.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.047+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 msgr2=0x7fa1a406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:14.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.047+0000 7fa1c739e700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 0x7fa1a406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.047+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1c0159a00 msgr2=0x7fa1c01e33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.047+0000 7fa1c739e700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1c0159a00 0x7fa1c01e33c0 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009b90 tx=0x7fa1b0004750 comp rx=0 tx=0).stop 2023-10-16T11:22:14.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 shutdown_connections 2023-10-16T11:22:14.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fa1a406b020 0x7fa1a406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1c0159de0 0x7fa1c01e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1c006d8c0 0x7fa1c0146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 --2- 172.21.15.181:0/144580591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1c0159a00 0x7fa1c01e33c0 unknown :-1 s=CLOSED pgs=3188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 >> 172.21.15.181:0/144580591 conn(0x7fa1c00cfb60 msgr2=0x7fa1c00cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:14.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 shutdown_connections 2023-10-16T11:22:14.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.048+0000 7fa1c739e700 1 -- 172.21.15.181:0/144580591 wait complete. 2023-10-16T11:22:14.062 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T11:22:14.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.185+0000 7ff041869700 1 Processor -- start 2023-10-16T11:22:14.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.185+0000 7ff041869700 1 -- start start 2023-10-16T11:22:14.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff041869700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c15dd50 0x7ff03c0d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff041869700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff03c0d1e80 0x7ff03c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff041869700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03c0b0410 con 0x7ff03c0ce440 2023-10-16T11:22:14.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff041869700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c0b3c00 con 0x7ff03c0d1e80 2023-10-16T11:22:14.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff041869700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c15e130 con 0x7ff03c15dd50 2023-10-16T11:22:14.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03a7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff03c0d1e80 0x7ff03c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03b7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff03c0ce440 0x7ff03c0ce8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:44944/0 (socket says 172.21.15.181:44944) 2023-10-16T11:22:14.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03b7fe700 1 -- 172.21.15.181:0/1741455195 learned_addr learned my addr 172.21.15.181:0/1741455195 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:14.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03affd700 1 --2- 172.21.15.181:0/1741455195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c15dd50 0x7ff03c0d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03a7fc700 1 -- 172.21.15.181:0/1741455195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c15dd50 msgr2=0x7ff03c0d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:14.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03a7fc700 1 --2- 172.21.15.181:0/1741455195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c15dd50 0x7ff03c0d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03a7fc700 1 -- 172.21.15.181:0/1741455195 >> v1:172.21.15.181:6789/0 conn(0x7ff03c0ce440 legacy=0x7ff03c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff039ffb700 1 -- 172.21.15.181:0/1741455195 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532616491 0 0) 0x7ff03c0b0410 con 0x7ff03c0ce440 2023-10-16T11:22:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.186+0000 7ff03a7fc700 1 -- 172.21.15.181:0/1741455195 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c0b2ca0 con 0x7ff03c0d1e80 2023-10-16T11:22:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff03a7fc700 1 --2- 172.21.15.181:0/1741455195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0d2300 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 crypto rx=0x7ff030009840 tx=0x7ff030009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ff531f4359ed7b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:14.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff039ffb700 1 -- 172.21.15.181:0/1741455195 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030011830 con 0x7ff03c0d1e80 2023-10-16T11:22:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff039ffb700 1 -- 172.21.15.181:0/1741455195 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff030011990 con 0x7ff03c0d1e80 2023-10-16T11:22:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff039ffb700 1 -- 172.21.15.181:0/1741455195 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030011d20 con 0x7ff03c0d1e80 2023-10-16T11:22:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 -- 172.21.15.181:0/1741455195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 msgr2=0x7ff03c0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 --2- 172.21.15.181:0/1741455195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0d2300 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 crypto rx=0x7ff030009840 tx=0x7ff030009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 -- 172.21.15.181:0/1741455195 shutdown_connections 2023-10-16T11:22:14.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 --2- 172.21.15.181:0/1741455195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c15dd50 0x7ff03c0d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 --2- 172.21.15.181:0/1741455195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0d2300 unknown :-1 s=CLOSED pgs=3190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 -- 172.21.15.181:0/1741455195 >> 172.21.15.181:0/1741455195 conn(0x7ff03c08af90 msgr2=0x7ff03c0d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 -- 172.21.15.181:0/1741455195 shutdown_connections 2023-10-16T11:22:14.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.187+0000 7ff041869700 1 -- 172.21.15.181:0/1741455195 wait complete. 2023-10-16T11:22:14.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.189+0000 7ff041869700 1 Processor -- start 2023-10-16T11:22:14.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.189+0000 7ff041869700 1 -- start start 2023-10-16T11:22:14.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.189+0000 7ff041869700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c0ce440 0x7ff03c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.189+0000 7ff041869700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.189+0000 7ff041869700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c15dd50 0x7ff03c0b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.189+0000 7ff041869700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0cb8d0 con 0x7ff03c0d1e80 2023-10-16T11:22:14.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.189+0000 7ff041869700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c157660 con 0x7ff03c15dd50 2023-10-16T11:22:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff041869700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0cab10 con 0x7ff03c0ce440 2023-10-16T11:22:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c0ce440 0x7ff03c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c0ce440 0x7ff03c157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:59126/0 (socket says 172.21.15.181:59126) 2023-10-16T11:22:14.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 -- 172.21.15.181:0/980929377 learned_addr learned my addr 172.21.15.181:0/980929377 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33870/0 (socket says 172.21.15.181:33870) 2023-10-16T11:22:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03b7fe700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c15dd50 0x7ff03c0b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 -- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c15dd50 msgr2=0x7ff03c0b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c15dd50 0x7ff03c0b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 -- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 msgr2=0x7ff03c0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 -- 172.21.15.181:0/980929377 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0300094a0 con 0x7ff03c0ce440 2023-10-16T11:22:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03a7fc700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff03affd700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c0ce440 0x7ff03c157120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00e930 tx=0x7ff02c00ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:14.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff023fff700 1 -- 172.21.15.181:0/980929377 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c009650 con 0x7ff03c0ce440 2023-10-16T11:22:14.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.190+0000 7ff023fff700 1 -- 172.21.15.181:0/980929377 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c013680 con 0x7ff03c0ce440 2023-10-16T11:22:14.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.191+0000 7ff023fff700 1 -- 172.21.15.181:0/980929377 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0139f0 con 0x7ff03c0ce440 2023-10-16T11:22:14.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.191+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c0cadf0 con 0x7ff03c0ce440 2023-10-16T11:22:14.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.191+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c0cb3b0 con 0x7ff03c0ce440 2023-10-16T11:22:14.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.192+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03c0cb720 con 0x7ff03c0ce440 2023-10-16T11:22:14.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.192+0000 7ff023fff700 1 -- 172.21.15.181:0/980929377 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7ff02c013b50 con 0x7ff03c0ce440 2023-10-16T11:22:14.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.193+0000 7ff023fff700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 0x7ff02806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.193+0000 7ff03a7fc700 1 -- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 msgr2=0x7ff02806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:14.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.193+0000 7ff023fff700 1 -- 172.21.15.181:0/980929377 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7ff02c089d50 con 0x7ff03c0ce440 2023-10-16T11:22:14.203 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.193+0000 7ff03a7fc700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 0x7ff02806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:14.204 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.201+0000 7ff023fff700 1 -- 172.21.15.181:0/980929377 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7ff02c08d030 con 0x7ff03c0ce440 2023-10-16T11:22:14.390 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:14.387+0000 7f26b8b55200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.394+0000 7ff03a7fc700 1 -- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 msgr2=0x7ff02806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:14.396 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.394+0000 7ff03a7fc700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 0x7ff02806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:14.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.447+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff03c0b3510 con 0x7ff03c0ce440 2023-10-16T11:22:14.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.448+0000 7ff023fff700 1 -- 172.21.15.181:0/980929377 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v270) v1 ==== 80+0+474 (secure 0 0 0) 0x7ff02c0211f0 con 0x7ff03c0ce440 2023-10-16T11:22:14.450 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/2735196384 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.451 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6833/2291437477 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.451 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:6832/2291437477 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.451 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/691655073 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.451 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/3571285885 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.451 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/171534192 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.452 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/714342180 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.452 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181:0/432126216 2023-10-17T11:22:10.332184+0000 2023-10-16T11:22:14.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 msgr2=0x7ff02806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:14.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 0x7ff02806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c0ce440 msgr2=0x7ff03c157120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c0ce440 0x7ff03c157120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00e930 tx=0x7ff02c00ec40 comp rx=0 tx=0).stop 2023-10-16T11:22:14.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 shutdown_connections 2023-10-16T11:22:14.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7ff02806b020 0x7ff02806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c0ce440 0x7ff03c157120 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c15dd50 0x7ff03c0b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 --2- 172.21.15.181:0/980929377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff03c0d1e80 0x7ff03c0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.450+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 >> 172.21.15.181:0/980929377 conn(0x7ff03c08af90 msgr2=0x7ff03c0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:14.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.451+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 shutdown_connections 2023-10-16T11:22:14.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.451+0000 7ff041869700 1 -- 172.21.15.181:0/980929377 wait complete. 2023-10-16T11:22:14.456 INFO:teuthology.orchestra.run.smithi181.stderr:listed 8 entries 2023-10-16T11:22:14.459 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:14.457+0000 7f26b8b55200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:14.466 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T11:22:14.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:14.467 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T11:22:14.492 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:14.491+0000 7fb3a4504200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T11:22:14.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:22:14.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:14.508 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T11:22:14.564 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:22:14.564 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:14.566 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:14.564+0000 7fb3a4504200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T11:22:14.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:22:14.582 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:22:14.582 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:14.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:22:14.599 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:22:14.600 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:14.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:22:14.640 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:14.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.762+0000 7f9690476700 1 Processor -- start 2023-10-16T11:22:14.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.762+0000 7f9690476700 1 -- start start 2023-10-16T11:22:14.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f9690476700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968815dc20 0x7f96880d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f9690476700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96880d1e60 0x7f96880d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f9690476700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f96880ce4b0 0x7f96880ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f9690476700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96880b3b80 con 0x7f96880d1e60 2023-10-16T11:22:14.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f9690476700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96880d2820 con 0x7f96880ce4b0 2023-10-16T11:22:14.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f9690476700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880b2f00 con 0x7f968815dc20 2023-10-16T11:22:14.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96880d1e60 0x7f96880d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96880d1e60 0x7f96880d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33878/0 (socket says 172.21.15.181:33878) 2023-10-16T11:22:14.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 -- 172.21.15.181:0/194090238 learned_addr learned my addr 172.21.15.181:0/194090238 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:14.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968ea13700 1 --2- 172.21.15.181:0/194090238 >> v2:172.21.15.190:3300/0 conn(0x7f96880ce4b0 0x7f96880ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968e212700 1 --2- 172.21.15.181:0/194090238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968815dc20 0x7f96880d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 -- 172.21.15.181:0/194090238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968815dc20 msgr2=0x7f96880d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:14.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 --2- 172.21.15.181:0/194090238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968815dc20 0x7f96880d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 -- 172.21.15.181:0/194090238 >> v2:172.21.15.190:3300/0 conn(0x7f96880ce4b0 msgr2=0x7f96880ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:14.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 --2- 172.21.15.181:0/194090238 >> v2:172.21.15.190:3300/0 conn(0x7f96880ce4b0 0x7f96880ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 -- 172.21.15.181:0/194090238 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880b31e0 con 0x7f96880d1e60 2023-10-16T11:22:14.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968ea13700 1 --2- 172.21.15.181:0/194090238 >> v2:172.21.15.190:3300/0 conn(0x7f96880ce4b0 0x7f96880ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:14.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968e212700 1 --2- 172.21.15.181:0/194090238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968815dc20 0x7f96880d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:14.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.763+0000 7f968da11700 1 --2- 172.21.15.181:0/194090238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880d1e60 0x7f96880d22e0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f96880cde80 tx=0x7f9678009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed4eacd7db4c1d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:14.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f968d210700 1 -- 172.21.15.181:0/194090238 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967800b870 con 0x7f96880d1e60 2023-10-16T11:22:14.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f968d210700 1 -- 172.21.15.181:0/194090238 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967800b9d0 con 0x7f96880d1e60 2023-10-16T11:22:14.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f968d210700 1 -- 172.21.15.181:0/194090238 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967800bd60 con 0x7f96880d1e60 2023-10-16T11:22:14.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 -- 172.21.15.181:0/194090238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880d1e60 msgr2=0x7f96880d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 --2- 172.21.15.181:0/194090238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880d1e60 0x7f96880d22e0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f96880cde80 tx=0x7f9678009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 -- 172.21.15.181:0/194090238 shutdown_connections 2023-10-16T11:22:14.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 --2- 172.21.15.181:0/194090238 >> v2:172.21.15.190:3300/0 conn(0x7f96880ce4b0 0x7f96880ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 --2- 172.21.15.181:0/194090238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968815dc20 0x7f96880d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 --2- 172.21.15.181:0/194090238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96880d1e60 0x7f96880d22e0 unknown :-1 s=CLOSED pgs=3191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 -- 172.21.15.181:0/194090238 >> 172.21.15.181:0/194090238 conn(0x7f968808af90 msgr2=0x7f96880d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:14.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 -- 172.21.15.181:0/194090238 shutdown_connections 2023-10-16T11:22:14.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.764+0000 7f9690476700 1 -- 172.21.15.181:0/194090238 wait complete. 2023-10-16T11:22:14.775 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.766+0000 7f9690476700 1 Processor -- start 2023-10-16T11:22:14.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.766+0000 7f9690476700 1 -- start start 2023-10-16T11:22:14.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f9690476700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96880ce4b0 0x7f968814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f9690476700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96880d1e60 0x7f968809d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f9690476700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968815dc20 0x7f968809dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.776 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f9690476700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968814f5c0 con 0x7f968815dc20 2023-10-16T11:22:14.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f9690476700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968814e9b0 con 0x7f96880d1e60 2023-10-16T11:22:14.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f9690476700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968814ec90 con 0x7f96880ce4b0 2023-10-16T11:22:14.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f968ea13700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968815dc20 0x7f968809dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.777 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f968e212700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96880ce4b0 0x7f968814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f968e212700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96880ce4b0 0x7f968814d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:59140/0 (socket says 172.21.15.181:59140) 2023-10-16T11:22:14.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f968e212700 1 -- 172.21.15.181:0/87776399 learned_addr learned my addr 172.21.15.181:0/87776399 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:14.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.767+0000 7f968da11700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96880d1e60 0x7f968809d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:14.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968ea13700 1 -- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96880ce4b0 msgr2=0x7f968814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.778 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968ea13700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96880ce4b0 0x7f968814d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968ea13700 1 -- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96880d1e60 msgr2=0x7f968809d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:14.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968ea13700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96880d1e60 0x7f968809d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:14.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968ea13700 1 -- 172.21.15.181:0/87776399 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009550 con 0x7f968815dc20 2023-10-16T11:22:14.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968e212700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96880ce4b0 0x7f968814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:14.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968da11700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96880d1e60 0x7f968809d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:14.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f968ea13700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968815dc20 0x7f968809dfb0 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f9680007d40 tx=0x7f968000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:14.780 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f967effd700 1 -- 172.21.15.181:0/87776399 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968000dd20 con 0x7f968815dc20 2023-10-16T11:22:14.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f967effd700 1 -- 172.21.15.181:0/87776399 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968000f040 con 0x7f968815dc20 2023-10-16T11:22:14.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968814ef70 con 0x7f968815dc20 2023-10-16T11:22:14.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f967effd700 1 -- 172.21.15.181:0/87776399 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680003c60 con 0x7f968815dc20 2023-10-16T11:22:14.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.768+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968809db40 con 0x7f968815dc20 2023-10-16T11:22:14.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.769+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96880ce0f0 con 0x7f968815dc20 2023-10-16T11:22:14.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.769+0000 7f967effd700 1 -- 172.21.15.181:0/87776399 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f9680004030 con 0x7f968815dc20 2023-10-16T11:22:14.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.770+0000 7f967effd700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 0x7f966c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:14.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.770+0000 7f968e212700 1 -- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 msgr2=0x7f966c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:14.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.770+0000 7f967effd700 1 -- 172.21.15.181:0/87776399 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f9680089a30 con 0x7f968815dc20 2023-10-16T11:22:14.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.770+0000 7f968e212700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 0x7f966c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:14.783 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.776+0000 7f967effd700 1 -- 172.21.15.181:0/87776399 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9680054540 con 0x7f968815dc20 2023-10-16T11:22:14.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.971+0000 7f968e212700 1 -- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 msgr2=0x7f966c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:14.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:14.971+0000 7f968e212700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 0x7f966c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.020+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96880d3a10 con 0x7f968815dc20 2023-10-16T11:22:15.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.020+0000 7f967effd700 1 -- 172.21.15.181:0/87776399 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f9680057b60 con 0x7f968815dc20 2023-10-16T11:22:15.023 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:15.023 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":153,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/984505218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":984505218},{"type":"v1","addr":"172.21.15.190:6835","nonce":984505218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/350629849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":350629849},{"type":"v1","addr":"172.21.15.181:6835","nonce":350629849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13440,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1354411415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1354411415},{"type":"v1","addr":"172.21.15.181:6837","nonce":1354411415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:56.485392+0000","modified":"2023-10-16T11:21:59.515101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13452":{"gid":13452,"name":"b","rank":0,"incarnation":149,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/284709222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":284709222},{"type":"v1","addr":"172.21.15.190:6834","nonce":284709222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:22:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 msgr2=0x7f966c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 0x7f966c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968815dc20 msgr2=0x7f968809dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968815dc20 0x7f968809dfb0 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f9680007d40 tx=0x7f968000d3b0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 shutdown_connections 2023-10-16T11:22:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f966c06b020 0x7f966c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96880ce4b0 0x7f968814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f968815dc20 0x7f968809dfb0 unknown :-1 s=CLOSED pgs=3192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 --2- 172.21.15.181:0/87776399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96880d1e60 0x7f968809d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.022+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 >> 172.21.15.181:0/87776399 conn(0x7f968808af90 msgr2=0x7f968806ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:15.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.023+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 shutdown_connections 2023-10-16T11:22:15.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.023+0000 7f9690476700 1 -- 172.21.15.181:0/87776399 wait complete. 2023-10-16T11:22:15.027 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 153 2023-10-16T11:22:15.037 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:15.107 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:15.106+0000 7f26b8b55200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:15.164 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.162+0000 7fe730bb5700 1 Processor -- start 2023-10-16T11:22:15.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.162+0000 7fe730bb5700 1 -- start start 2023-10-16T11:22:15.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe730bb5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe72c0d2390 0x7fe72c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe730bb5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe72c0d0180 0x7fe72c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe730bb5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c0cc470 con 0x7fe72c0ce7f0 2023-10-16T11:22:15.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe730bb5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c06c810 con 0x7fe72c0d0180 2023-10-16T11:22:15.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe730bb5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c0d05f0 con 0x7fe72c0d2390 2023-10-16T11:22:15.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe72ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe72c0d0180 0x7fe72c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe72ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe72c0d0180 0x7fe72c0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33896/0 (socket says 172.21.15.181:33896) 2023-10-16T11:22:15.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.163+0000 7fe72ad9d700 1 -- 172.21.15.181:0/1751491636 learned_addr learned my addr 172.21.15.181:0/1751491636 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:15.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe729d9b700 1 --2- 172.21.15.181:0/1751491636 >> v2:172.21.15.190:3300/0 conn(0x7fe72c0d2390 0x7fe72c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72ad9d700 1 -- 172.21.15.181:0/1751491636 >> v2:172.21.15.190:3300/0 conn(0x7fe72c0d2390 msgr2=0x7fe72c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:15.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72ad9d700 1 --2- 172.21.15.181:0/1751491636 >> v2:172.21.15.190:3300/0 conn(0x7fe72c0d2390 0x7fe72c0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72ad9d700 1 -- 172.21.15.181:0/1751491636 >> v1:172.21.15.190:6789/0 conn(0x7fe72c0ce7f0 legacy=0x7fe72c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72ad9d700 1 -- 172.21.15.181:0/1751491636 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c06ca30 con 0x7fe72c0d0180 2023-10-16T11:22:15.169 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72ad9d700 1 --2- 172.21.15.181:0/1751491636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0cc930 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0cc1e0 tx=0x7fe72400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea25e08790dbfffb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:15.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72959a700 1 -- 172.21.15.181:0/1751491636 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72400e070 con 0x7fe72c0d0180 2023-10-16T11:22:15.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72959a700 1 -- 172.21.15.181:0/1751491636 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72400b910 con 0x7fe72c0d0180 2023-10-16T11:22:15.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.164+0000 7fe72959a700 1 -- 172.21.15.181:0/1751491636 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72400bcc0 con 0x7fe72c0d0180 2023-10-16T11:22:15.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 -- 172.21.15.181:0/1751491636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 msgr2=0x7fe72c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 --2- 172.21.15.181:0/1751491636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0cc930 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0cc1e0 tx=0x7fe72400b330 comp rx=0 tx=0).stop 2023-10-16T11:22:15.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 -- 172.21.15.181:0/1751491636 shutdown_connections 2023-10-16T11:22:15.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 --2- 172.21.15.181:0/1751491636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0cc930 unknown :-1 s=CLOSED pgs=3193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 --2- 172.21.15.181:0/1751491636 >> v2:172.21.15.190:3300/0 conn(0x7fe72c0d2390 0x7fe72c0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 -- 172.21.15.181:0/1751491636 >> 172.21.15.181:0/1751491636 conn(0x7fe72c09a9a0 msgr2=0x7fe72c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:15.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 -- 172.21.15.181:0/1751491636 shutdown_connections 2023-10-16T11:22:15.171 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.165+0000 7fe730bb5700 1 -- 172.21.15.181:0/1751491636 wait complete. 2023-10-16T11:22:15.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.166+0000 7fe730bb5700 1 Processor -- start 2023-10-16T11:22:15.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.166+0000 7fe730bb5700 1 -- start start 2023-10-16T11:22:15.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe730bb5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe72c0ce7f0 0x7fe72c09ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe730bb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe730bb5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe72c0d2390 0x7fe72c1f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe730bb5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c09f3d0 con 0x7fe72c0d0180 2023-10-16T11:22:15.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe730bb5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0c67a0 con 0x7fe72c0ce7f0 2023-10-16T11:22:15.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe730bb5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0c5700 con 0x7fe72c0d2390 2023-10-16T11:22:15.175 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe729d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe729d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33900/0 (socket says 172.21.15.181:33900) 2023-10-16T11:22:15.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe729d9b700 1 -- 172.21.15.181:0/828139782 learned_addr learned my addr 172.21.15.181:0/828139782 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:15.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe72a59c700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe72c0ce7f0 0x7fe72c09ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.167+0000 7fe72ad9d700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe72c0d2390 0x7fe72c1f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe729d9b700 1 -- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe72c0d2390 msgr2=0x7fe72c1f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe729d9b700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe72c0d2390 0x7fe72c1f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe729d9b700 1 -- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe72c0ce7f0 msgr2=0x7fe72c09ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe729d9b700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe72c0ce7f0 0x7fe72c09ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe729d9b700 1 -- 172.21.15.181:0/828139782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724009b30 con 0x7fe72c0d0180 2023-10-16T11:22:15.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe729d9b700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0c7f20 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00e9b0 tx=0x7fe71c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:15.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe71affd700 1 -- 172.21.15.181:0/828139782 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c009710 con 0x7fe72c0d0180 2023-10-16T11:22:15.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe71affd700 1 -- 172.21.15.181:0/828139782 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c00ee50 con 0x7fe72c0d0180 2023-10-16T11:22:15.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c0c59e0 con 0x7fe72c0d0180 2023-10-16T11:22:15.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe71affd700 1 -- 172.21.15.181:0/828139782 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00ca50 con 0x7fe72c0d0180 2023-10-16T11:22:15.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.168+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c0c5fa0 con 0x7fe72c0d0180 2023-10-16T11:22:15.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.169+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c067d20 con 0x7fe72c0d0180 2023-10-16T11:22:15.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.169+0000 7fe71affd700 1 -- 172.21.15.181:0/828139782 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fe71c00cbb0 con 0x7fe72c0d0180 2023-10-16T11:22:15.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.170+0000 7fe71affd700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 0x7fe70c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.170+0000 7fe72a59c700 1 -- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 msgr2=0x7fe70c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:15.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.170+0000 7fe71affd700 1 -- 172.21.15.181:0/828139782 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fe71c089af0 con 0x7fe72c0d0180 2023-10-16T11:22:15.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.170+0000 7fe72a59c700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 0x7fe70c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:15.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.174+0000 7fe71affd700 1 -- 172.21.15.181:0/828139782 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe71c054710 con 0x7fe72c0d0180 2023-10-16T11:22:15.185 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:15.184+0000 7f26b8b55200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:15.236 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:15.234+0000 7fb3a4504200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:15.317 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:15.315+0000 7fb3a4504200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T11:22:15.329 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:15.328+0000 7f26b8b55200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:15.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.371+0000 7fe72a59c700 1 -- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 msgr2=0x7fe70c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:15.373 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.371+0000 7fe72a59c700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 0x7fe70c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:15.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.379+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe72c0cef40 con 0x7fe72c0d0180 2023-10-16T11:22:15.382 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.381+0000 7fe71affd700 1 -- 172.21.15.181:0/828139782 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v270) v1 ==== 74+0+17919 (secure 0 0 0) 0x7fe71c021090 con 0x7fe72c0d0180 2023-10-16T11:22:15.383 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:15.383 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":270,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:22:10.332242+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:54.910438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:55.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2735196384":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6833/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6832/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/691655073":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/3571285885":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/171534192":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/714342180":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/432126216":"2023-10-17T11:22:10.332184+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:15.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 msgr2=0x7fe70c06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:15.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 0x7fe70c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 msgr2=0x7fe72c0c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0c7f20 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00e9b0 tx=0x7fe71c00ecc0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 shutdown_connections 2023-10-16T11:22:15.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe70c06b050 0x7fe70c06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe72c0d2390 0x7fe72c1f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe72c0ce7f0 0x7fe72c09ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 --2- 172.21.15.181:0/828139782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe72c0d0180 0x7fe72c0c7f20 unknown :-1 s=CLOSED pgs=3194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 >> 172.21.15.181:0/828139782 conn(0x7fe72c09a9a0 msgr2=0x7fe72c0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:15.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 shutdown_connections 2023-10-16T11:22:15.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.383+0000 7fe730bb5700 1 -- 172.21.15.181:0/828139782 wait complete. 2023-10-16T11:22:15.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T09:43:55.719041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T11:21:54.910438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '268', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 37, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T11:21:55.443301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '269', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T11:22:15.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T11:22:15.473 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:15.471+0000 7fb3a4504200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T11:22:15.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.522+0000 7fe60fc33700 1 Processor -- start 2023-10-16T11:22:15.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.523+0000 7fe60fc33700 1 -- start start 2023-10-16T11:22:15.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.523+0000 7fe60fc33700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6081646b0 0x7fe608166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.523+0000 7fe60fc33700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6080b1f70 con 0x7fe60815f150 2023-10-16T11:22:15.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.523+0000 7fe60fc33700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6080b2ae0 con 0x7fe6080ce6c0 2023-10-16T11:22:15.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.523+0000 7fe60fc33700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6080cd880 con 0x7fe6081646b0 2023-10-16T11:22:15.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60e1d0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6081646b0 0x7fe608166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60d9cf700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe60815f150 0x7fe6080cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45004/0 (socket says 172.21.15.181:45004) 2023-10-16T11:22:15.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60d9cf700 1 -- 172.21.15.181:0/2792560105 learned_addr learned my addr 172.21.15.181:0/2792560105 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:15.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60e1d0700 1 -- 172.21.15.181:0/2792560105 >> v1:172.21.15.190:6789/0 conn(0x7fe6080ce6c0 legacy=0x7fe6080cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.528 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60e1d0700 1 -- 172.21.15.181:0/2792560105 >> v1:172.21.15.181:6789/0 conn(0x7fe60815f150 legacy=0x7fe6080cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60e1d0700 1 -- 172.21.15.181:0/2792560105 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080b2dc0 con 0x7fe6081646b0 2023-10-16T11:22:15.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60e1d0700 1 --2- 172.21.15.181:0/2792560105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6081646b0 0x7fe608166aa0 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7fe600009d90 tx=0x7fe60000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e5c83949e3605d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:15.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60c9cd700 1 -- 172.21.15.181:0/2792560105 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60000e070 con 0x7fe6081646b0 2023-10-16T11:22:15.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60c9cd700 1 -- 172.21.15.181:0/2792560105 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60000bbd0 con 0x7fe6081646b0 2023-10-16T11:22:15.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.524+0000 7fe60c9cd700 1 -- 172.21.15.181:0/2792560105 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600012430 con 0x7fe6081646b0 2023-10-16T11:22:15.529 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.525+0000 7fe60fc33700 1 -- 172.21.15.181:0/2792560105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6081646b0 msgr2=0x7fe608166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.525+0000 7fe60fc33700 1 --2- 172.21.15.181:0/2792560105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6081646b0 0x7fe608166aa0 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7fe600009d90 tx=0x7fe60000b330 comp rx=0 tx=0).stop 2023-10-16T11:22:15.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.525+0000 7fe60fc33700 1 -- 172.21.15.181:0/2792560105 shutdown_connections 2023-10-16T11:22:15.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.525+0000 7fe60fc33700 1 --2- 172.21.15.181:0/2792560105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6081646b0 0x7fe608166aa0 unknown :-1 s=CLOSED pgs=3196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.525+0000 7fe60fc33700 1 -- 172.21.15.181:0/2792560105 >> 172.21.15.181:0/2792560105 conn(0x7fe60809a930 msgr2=0x7fe6080d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:15.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.525+0000 7fe60fc33700 1 -- 172.21.15.181:0/2792560105 shutdown_connections 2023-10-16T11:22:15.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.525+0000 7fe60fc33700 1 -- 172.21.15.181:0/2792560105 wait complete. 2023-10-16T11:22:15.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.526+0000 7fe60fc33700 1 Processor -- start 2023-10-16T11:22:15.533 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.526+0000 7fe60fc33700 1 -- start start 2023-10-16T11:22:15.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60fc33700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6080ce6c0 0x7fe608156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60fc33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815f150 0x7fe608163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60fc33700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6081646b0 0x7fe608163e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.534 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60fc33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080c6f40 con 0x7fe60815f150 2023-10-16T11:22:15.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60fc33700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe608156bc0 con 0x7fe6080ce6c0 2023-10-16T11:22:15.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60fc33700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60806c9a0 con 0x7fe6081646b0 2023-10-16T11:22:15.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60d1ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815f150 0x7fe608163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60d1ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815f150 0x7fe608163940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33934/0 (socket says 172.21.15.181:33934) 2023-10-16T11:22:15.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60d1ce700 1 -- 172.21.15.181:0/3432287821 learned_addr learned my addr 172.21.15.181:0/3432287821 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:15.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60e1d0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6081646b0 0x7fe608163e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.527+0000 7fe60d9cf700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6080ce6c0 0x7fe608156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60d1ce700 1 -- 172.21.15.181:0/3432287821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6081646b0 msgr2=0x7fe608163e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60d1ce700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6081646b0 0x7fe608163e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60d1ce700 1 -- 172.21.15.181:0/3432287821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6080ce6c0 msgr2=0x7fe608156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60d1ce700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6080ce6c0 0x7fe608156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60d1ce700 1 -- 172.21.15.181:0/3432287821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600009a70 con 0x7fe60815f150 2023-10-16T11:22:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60d1ce700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815f150 0x7fe608163940 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c450 tx=0x7fe60400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe5fe7fc700 1 -- 172.21.15.181:0/3432287821 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040052d0 con 0x7fe60815f150 2023-10-16T11:22:15.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe5fe7fc700 1 -- 172.21.15.181:0/3432287821 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604005430 con 0x7fe60815f150 2023-10-16T11:22:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6080c65a0 con 0x7fe60815f150 2023-10-16T11:22:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe5fe7fc700 1 -- 172.21.15.181:0/3432287821 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040038c0 con 0x7fe60815f150 2023-10-16T11:22:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.528+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6080c6b60 con 0x7fe60815f150 2023-10-16T11:22:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.529+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6080c6cf0 con 0x7fe60815f150 2023-10-16T11:22:15.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.530+0000 7fe5fe7fc700 1 -- 172.21.15.181:0/3432287821 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fe604003a20 con 0x7fe60815f150 2023-10-16T11:22:15.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.530+0000 7fe5fe7fc700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 0x7fe5ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.530+0000 7fe60d9cf700 1 -- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 msgr2=0x7fe5ec06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:15.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.530+0000 7fe5fe7fc700 1 -- 172.21.15.181:0/3432287821 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fe604088d60 con 0x7fe60815f150 2023-10-16T11:22:15.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.530+0000 7fe60d9cf700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 0x7fe5ec06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:15.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.534+0000 7fe5fe7fc700 1 -- 172.21.15.181:0/3432287821 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe604053a10 con 0x7fe60815f150 2023-10-16T11:22:15.559 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:15.557+0000 7f26b8b55200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:15.632 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:15.631+0000 7f26b8b55200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:15.714 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:15.712+0000 7fb3a4504200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T11:22:15.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.731+0000 7fe60d9cf700 1 -- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 msgr2=0x7fe5ec06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:15.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.731+0000 7fe60d9cf700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 0x7fe5ec06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:15.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.732+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6080c7080 con 0x7fe60815f150 2023-10-16T11:22:15.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.732+0000 7fe5fe7fc700 1 -- 172.21.15.181:0/3432287821 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v270) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6080c7080 con 0x7fe60815f150 2023-10-16T11:22:15.734 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2023-10-16T11:22:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.734+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 msgr2=0x7fe5ec06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.734+0000 7fe60fc33700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 0x7fe5ec06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.734+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815f150 msgr2=0x7fe608163940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.734+0000 7fe60fc33700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815f150 0x7fe608163940 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c450 tx=0x7fe60400c760 comp rx=0 tx=0).stop 2023-10-16T11:22:15.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 shutdown_connections 2023-10-16T11:22:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fe5ec06b020 0x7fe5ec06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6081646b0 0x7fe608163e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6080ce6c0 0x7fe608156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 --2- 172.21.15.181:0/3432287821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815f150 0x7fe608163940 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 >> 172.21.15.181:0/3432287821 conn(0x7fe60809a930 msgr2=0x7fe60815d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:15.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 shutdown_connections 2023-10-16T11:22:15.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.735+0000 7fe60fc33700 1 -- 172.21.15.181:0/3432287821 wait complete. 2023-10-16T11:22:15.749 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T11:22:15.796 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:15.794+0000 7fb3a4504200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T11:22:15.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.876+0000 7f584d338700 1 Processor -- start 2023-10-16T11:22:15.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.876+0000 7f584d338700 1 -- start start 2023-10-16T11:22:15.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f584d338700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58480cc6e0 0x7f5848071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f584d338700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480ce1f0 con 0x7f58480ce680 2023-10-16T11:22:15.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f584d338700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480ccda0 con 0x7f58480b2010 2023-10-16T11:22:15.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f584d338700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58480ccb90 con 0x7f58480cc6e0 2023-10-16T11:22:15.880 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f58467fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f58480cc6e0 0x7f5848071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f58477fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f58480ce680 0x7f58480ceaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45024/0 (socket says 172.21.15.181:45024) 2023-10-16T11:22:15.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f58477fe700 1 -- 172.21.15.181:0/1000500954 learned_addr learned my addr 172.21.15.181:0/1000500954 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:15.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f58467fc700 1 -- 172.21.15.181:0/1000500954 >> v1:172.21.15.190:6789/0 conn(0x7f58480b2010 legacy=0x7f58480b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f58467fc700 1 -- 172.21.15.181:0/1000500954 >> v1:172.21.15.181:6789/0 conn(0x7f58480ce680 legacy=0x7f58480ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f58467fc700 1 -- 172.21.15.181:0/1000500954 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480727e0 con 0x7f58480cc6e0 2023-10-16T11:22:15.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.877+0000 7f58467fc700 1 --2- 172.21.15.181:0/1000500954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480cc6e0 0x7f5848071be0 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 crypto rx=0x7f583c009840 tx=0x7f583c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7055923842cadde server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:15.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f5845ffb700 1 -- 172.21.15.181:0/1000500954 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c011830 con 0x7f58480cc6e0 2023-10-16T11:22:15.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f5845ffb700 1 -- 172.21.15.181:0/1000500954 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c011990 con 0x7f58480cc6e0 2023-10-16T11:22:15.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f5845ffb700 1 -- 172.21.15.181:0/1000500954 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c011d20 con 0x7f58480cc6e0 2023-10-16T11:22:15.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f584d338700 1 -- 172.21.15.181:0/1000500954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480cc6e0 msgr2=0x7f5848071be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f584d338700 1 --2- 172.21.15.181:0/1000500954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480cc6e0 0x7f5848071be0 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 crypto rx=0x7f583c009840 tx=0x7f583c009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:15.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f584d338700 1 -- 172.21.15.181:0/1000500954 shutdown_connections 2023-10-16T11:22:15.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f584d338700 1 --2- 172.21.15.181:0/1000500954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480cc6e0 0x7f5848071be0 unknown :-1 s=CLOSED pgs=3199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f584d338700 1 -- 172.21.15.181:0/1000500954 >> 172.21.15.181:0/1000500954 conn(0x7f58480cd010 msgr2=0x7f58480cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:15.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f584d338700 1 -- 172.21.15.181:0/1000500954 shutdown_connections 2023-10-16T11:22:15.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.878+0000 7f584d338700 1 -- 172.21.15.181:0/1000500954 wait complete. 2023-10-16T11:22:15.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.880+0000 7f584d338700 1 Processor -- start 2023-10-16T11:22:15.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.880+0000 7f584d338700 1 -- start start 2023-10-16T11:22:15.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.880+0000 7f584d338700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58480b2010 0x7f5848146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f584d338700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58480cc6e0 0x7f5848155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f584d338700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480ce680 0x7f5848155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f584d338700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c9e70 con 0x7f58480ce680 2023-10-16T11:22:15.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f584d338700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c9260 con 0x7f58480cc6e0 2023-10-16T11:22:15.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f584d338700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c9540 con 0x7f58480b2010 2023-10-16T11:22:15.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480ce680 0x7f5848155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480ce680 0x7f5848155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33952/0 (socket says 172.21.15.181:33952) 2023-10-16T11:22:15.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58467fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58480cc6e0 0x7f5848155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 -- 172.21.15.181:0/316871740 learned_addr learned my addr 172.21.15.181:0/316871740 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:15.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f5846ffd700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58480b2010 0x7f5848146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:15.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 -- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58480b2010 msgr2=0x7f5848146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58480b2010 0x7f5848146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 -- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58480cc6e0 msgr2=0x7f5848155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:15.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58480cc6e0 0x7f5848155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:15.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 -- 172.21.15.181:0/316871740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0094a0 con 0x7f58480ce680 2023-10-16T11:22:15.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58467fc700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58480cc6e0 0x7f5848155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:15.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f58477fe700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480ce680 0x7f5848155a70 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7f584000cbc0 tx=0x7f584000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:15.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f582f7fe700 1 -- 172.21.15.181:0/316871740 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584000a9e0 con 0x7f58480ce680 2023-10-16T11:22:15.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f582f7fe700 1 -- 172.21.15.181:0/316871740 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584000ab40 con 0x7f58480ce680 2023-10-16T11:22:15.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58480c9820 con 0x7f58480ce680 2023-10-16T11:22:15.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f582f7fe700 1 -- 172.21.15.181:0/316871740 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840012400 con 0x7f58480ce680 2023-10-16T11:22:15.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.881+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5848155fb0 con 0x7f58480ce680 2023-10-16T11:22:15.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.883+0000 7f582f7fe700 1 -- 172.21.15.181:0/316871740 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f584001b3b0 con 0x7f58480ce680 2023-10-16T11:22:15.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.883+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58480c9fe0 con 0x7f58480ce680 2023-10-16T11:22:15.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.883+0000 7f582f7fe700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 0x7f583406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:15.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.883+0000 7f5846ffd700 1 -- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 msgr2=0x7f583406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:15.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.884+0000 7f582f7fe700 1 -- 172.21.15.181:0/316871740 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f5840089590 con 0x7f58480ce680 2023-10-16T11:22:15.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.884+0000 7f5846ffd700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 0x7f583406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:15.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:15.888+0000 7f582f7fe700 1 -- 172.21.15.181:0/316871740 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5840054260 con 0x7f58480ce680 2023-10-16T11:22:15.899 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:15.898+0000 7f26b8b55200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:16.085 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:16.083+0000 7fb3a4504200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T11:22:16.086 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.085+0000 7f5846ffd700 1 -- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 msgr2=0x7f583406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:16.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.085+0000 7f5846ffd700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 0x7f583406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:16.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.100+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58480b6510 con 0x7f58480ce680 2023-10-16T11:22:16.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.101+0000 7f582f7fe700 1 -- 172.21.15.181:0/316871740 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v270) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5840057880 con 0x7f58480ce680 2023-10-16T11:22:16.103 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2023-10-16T11:22:16.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 msgr2=0x7f583406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:16.105 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 0x7f583406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480ce680 msgr2=0x7f5848155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480ce680 0x7f5848155a70 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7f584000cbc0 tx=0x7f584000ced0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 shutdown_connections 2023-10-16T11:22:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f583406b020 0x7f583406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58480b2010 0x7f5848146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58480ce680 0x7f5848155a70 unknown :-1 s=CLOSED pgs=3200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 --2- 172.21.15.181:0/316871740 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58480cc6e0 0x7f5848155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 >> 172.21.15.181:0/316871740 conn(0x7f58480cd010 msgr2=0x7f5848158810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 shutdown_connections 2023-10-16T11:22:16.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.103+0000 7f584d338700 1 -- 172.21.15.181:0/316871740 wait complete. 2023-10-16T11:22:16.118 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T11:22:16.121 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:16.120+0000 7f26b8b55200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:16.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.243+0000 7fb61b91a700 1 Processor -- start 2023-10-16T11:22:16.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.243+0000 7fb61b91a700 1 -- start start 2023-10-16T11:22:16.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb61b91a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6140d01d0 0x7fb6140ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.248 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb61b91a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb61415ea40 0x7fb6140d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb61b91a700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6140cf580 con 0x7fb614161530 2023-10-16T11:22:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb61b91a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6140d4290 con 0x7fb6140d01d0 2023-10-16T11:22:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb61b91a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140d0af0 con 0x7fb61415ea40 2023-10-16T11:22:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6140d01d0 0x7fb6140ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb6140d01d0 0x7fb6140ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33958/0 (socket says 172.21.15.181:33958) 2023-10-16T11:22:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 -- 172.21.15.181:0/4060593668 learned_addr learned my addr 172.21.15.181:0/4060593668 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb618eb5700 1 --2- 172.21.15.181:0/4060593668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb61415ea40 0x7fb6140d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 -- 172.21.15.181:0/4060593668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb61415ea40 msgr2=0x7fb6140d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 --2- 172.21.15.181:0/4060593668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb61415ea40 0x7fb6140d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 -- 172.21.15.181:0/4060593668 >> v1:172.21.15.190:6789/0 conn(0x7fb614161530 legacy=0x7fb6141619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 -- 172.21.15.181:0/4060593668 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6140ce970 con 0x7fb6140d01d0 2023-10-16T11:22:16.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb6196b6700 1 --2- 172.21.15.181:0/4060593668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6140d01d0 0x7fb6140ce2d0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098f0 tx=0x7fb610009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92ed709ff9818a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:16.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb60bfff700 1 -- 172.21.15.181:0/4060593668 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610011830 con 0x7fb6140d01d0 2023-10-16T11:22:16.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb60bfff700 1 -- 172.21.15.181:0/4060593668 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb610011990 con 0x7fb6140d01d0 2023-10-16T11:22:16.251 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.244+0000 7fb60bfff700 1 -- 172.21.15.181:0/4060593668 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610011d20 con 0x7fb6140d01d0 2023-10-16T11:22:16.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 -- 172.21.15.181:0/4060593668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6140d01d0 msgr2=0x7fb6140ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 --2- 172.21.15.181:0/4060593668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6140d01d0 0x7fb6140ce2d0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098f0 tx=0x7fb610009c00 comp rx=0 tx=0).stop 2023-10-16T11:22:16.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 -- 172.21.15.181:0/4060593668 shutdown_connections 2023-10-16T11:22:16.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 --2- 172.21.15.181:0/4060593668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6140d01d0 0x7fb6140ce2d0 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.252 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 --2- 172.21.15.181:0/4060593668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb61415ea40 0x7fb6140d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 -- 172.21.15.181:0/4060593668 >> 172.21.15.181:0/4060593668 conn(0x7fb61409cb60 msgr2=0x7fb6140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:16.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 -- 172.21.15.181:0/4060593668 shutdown_connections 2023-10-16T11:22:16.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.245+0000 7fb61b91a700 1 -- 172.21.15.181:0/4060593668 wait complete. 2023-10-16T11:22:16.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.246+0000 7fb61b91a700 1 Processor -- start 2023-10-16T11:22:16.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.246+0000 7fb61b91a700 1 -- start start 2023-10-16T11:22:16.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb61b91a700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6140d01d0 0x7fb614158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb61b91a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb61415ea40 0x7fb6140b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.256 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb61b91a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb614161530 0x7fb6140b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb61b91a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140b3660 con 0x7fb614161530 2023-10-16T11:22:16.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb61b91a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140c7500 con 0x7fb61415ea40 2023-10-16T11:22:16.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb61b91a700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140cb6f0 con 0x7fb6140d01d0 2023-10-16T11:22:16.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb614161530 0x7fb6140b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.257 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb6196b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6140d01d0 0x7fb614158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb614161530 0x7fb6140b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33968/0 (socket says 172.21.15.181:33968) 2023-10-16T11:22:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 -- 172.21.15.181:0/1551700031 learned_addr learned my addr 172.21.15.181:0/1551700031 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb618eb5700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb61415ea40 0x7fb6140b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 -- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6140d01d0 msgr2=0x7fb614158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6140d01d0 0x7fb614158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 -- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb61415ea40 msgr2=0x7fb6140b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb61415ea40 0x7fb6140b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb619eb7700 1 -- 172.21.15.181:0/1551700031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610009550 con 0x7fb614161530 2023-10-16T11:22:16.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.247+0000 7fb6196b6700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6140d01d0 0x7fb614158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.248+0000 7fb619eb7700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb614161530 0x7fb6140b14a0 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00cc50 tx=0x7fb60c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.248+0000 7fb609ffb700 1 -- 172.21.15.181:0/1551700031 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c003c00 con 0x7fb614161530 2023-10-16T11:22:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.248+0000 7fb609ffb700 1 -- 172.21.15.181:0/1551700031 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c003d60 con 0x7fb614161530 2023-10-16T11:22:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.248+0000 7fb609ffb700 1 -- 172.21.15.181:0/1551700031 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00aab0 con 0x7fb614161530 2023-10-16T11:22:16.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.248+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6140cae50 con 0x7fb614161530 2023-10-16T11:22:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.248+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6140cb410 con 0x7fb614161530 2023-10-16T11:22:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.249+0000 7fb609ffb700 1 -- 172.21.15.181:0/1551700031 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb60c00ac10 con 0x7fb614161530 2023-10-16T11:22:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.249+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6140c76b0 con 0x7fb614161530 2023-10-16T11:22:16.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.250+0000 7fb609ffb700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 0x7fb5f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.250+0000 7fb6196b6700 1 -- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 msgr2=0x7fb5f806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.250+0000 7fb609ffb700 1 -- 172.21.15.181:0/1551700031 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fb60c089ae0 con 0x7fb614161530 2023-10-16T11:22:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.250+0000 7fb6196b6700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 0x7fb5f806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:16.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.254+0000 7fb609ffb700 1 -- 172.21.15.181:0/1551700031 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb6140c76b0 con 0x7fb614161530 2023-10-16T11:22:16.273 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:16.271+0000 7f26b8b55200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:16.336 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:16.334+0000 7fb3a4504200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T11:22:16.387 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:16.386+0000 7f26b8b55200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:16.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.451+0000 7fb6196b6700 1 -- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 msgr2=0x7fb5f806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:16.453 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.451+0000 7fb6196b6700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 0x7fb5f806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:16.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.469+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6140c7670 con 0x7fb614161530 2023-10-16T11:22:16.471 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.469+0000 7fb609ffb700 1 -- 172.21.15.181:0/1551700031 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v270) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb60c054660 con 0x7fb614161530 2023-10-16T11:22:16.472 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2023-10-16T11:22:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 msgr2=0x7fb5f806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 0x7fb5f806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb614161530 msgr2=0x7fb6140b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb614161530 0x7fb6140b14a0 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00cc50 tx=0x7fb60c00cf60 comp rx=0 tx=0).stop 2023-10-16T11:22:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 shutdown_connections 2023-10-16T11:22:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7fb5f806b020 0x7fb5f806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6140d01d0 0x7fb614158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb614161530 0x7fb6140b14a0 unknown :-1 s=CLOSED pgs=3202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.475 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 --2- 172.21.15.181:0/1551700031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb61415ea40 0x7fb6140b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 >> 172.21.15.181:0/1551700031 conn(0x7fb61409cb60 msgr2=0x7fb61415eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:16.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 shutdown_connections 2023-10-16T11:22:16.476 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.472+0000 7fb61b91a700 1 -- 172.21.15.181:0/1551700031 wait complete. 2023-10-16T11:22:16.486 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:16.497 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:16.495+0000 7fb3a4504200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T11:22:16.544 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:16.543+0000 7f26b8b55200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:16.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.610+0000 7f699c5be700 1 Processor -- start 2023-10-16T11:22:16.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.610+0000 7f699c5be700 1 -- start start 2023-10-16T11:22:16.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.611+0000 7f699c5be700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f699415dfd0 0x7f69940af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.611+0000 7f699c5be700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699406f870 con 0x7f69940af450 2023-10-16T11:22:16.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.611+0000 7f699c5be700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69940d1160 con 0x7f69940cd470 2023-10-16T11:22:16.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.611+0000 7f699c5be700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69940cc8e0 con 0x7f699415dfd0 2023-10-16T11:22:16.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.611+0000 7f6999b59700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f69940af450 0x7f699406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45052/0 (socket says 172.21.15.181:45052) 2023-10-16T11:22:16.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.611+0000 7f6999b59700 1 -- 172.21.15.181:0/1129720718 learned_addr learned my addr 172.21.15.181:0/1129720718 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:16.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f699a35a700 1 --2- 172.21.15.181:0/1129720718 >> v2:172.21.15.190:3300/0 conn(0x7f699415dfd0 0x7f69940af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376096823 0 0) 0x7f699406f870 con 0x7f69940af450 2023-10-16T11:22:16.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69800035b0 con 0x7f69940af450 2023-10-16T11:22:16.617 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415036429 0 0) 0x7f69940d1160 con 0x7f69940cd470 2023-10-16T11:22:16.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699406f870 con 0x7f69940cd470 2023-10-16T11:22:16.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f699a35a700 1 -- 172.21.15.181:0/1129720718 >> v1:172.21.15.190:6789/0 conn(0x7f69940cd470 legacy=0x7f699406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f699a35a700 1 -- 172.21.15.181:0/1129720718 >> v1:172.21.15.181:6789/0 conn(0x7f69940af450 legacy=0x7f699406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 678170714 0 0) 0x7f69800035b0 con 0x7f69940af450 2023-10-16T11:22:16.618 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f699a35a700 1 -- 172.21.15.181:0/1129720718 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940ccbc0 con 0x7f699415dfd0 2023-10-16T11:22:16.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f699a35a700 1 --2- 172.21.15.181:0/1129720718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 0x7f69940af9a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6990009820 tx=0x7f6990009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5e6430348dc9f3a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:16.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984001670 con 0x7f699415dfd0 2023-10-16T11:22:16.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6990005c30 con 0x7f699415dfd0 2023-10-16T11:22:16.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.612+0000 7f6999358700 1 -- 172.21.15.181:0/1129720718 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990004030 con 0x7f699415dfd0 2023-10-16T11:22:16.619 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.613+0000 7f699c5be700 1 -- 172.21.15.181:0/1129720718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 msgr2=0x7f69940af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.613+0000 7f699c5be700 1 --2- 172.21.15.181:0/1129720718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 0x7f69940af9a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6990009820 tx=0x7f6990009b30 comp rx=0 tx=0).stop 2023-10-16T11:22:16.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.613+0000 7f699c5be700 1 -- 172.21.15.181:0/1129720718 shutdown_connections 2023-10-16T11:22:16.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.613+0000 7f699c5be700 1 --2- 172.21.15.181:0/1129720718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 0x7f69940af9a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.620 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.613+0000 7f699c5be700 1 -- 172.21.15.181:0/1129720718 >> 172.21.15.181:0/1129720718 conn(0x7f69940d04a0 msgr2=0x7f69940d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:16.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.613+0000 7f699c5be700 1 -- 172.21.15.181:0/1129720718 shutdown_connections 2023-10-16T11:22:16.621 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.613+0000 7f699c5be700 1 -- 172.21.15.181:0/1129720718 wait complete. 2023-10-16T11:22:16.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.614+0000 7f699c5be700 1 Processor -- start 2023-10-16T11:22:16.624 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.614+0000 7f699c5be700 1 -- start start 2023-10-16T11:22:16.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699c5be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69940af450 0x7f69941f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699c5be700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69940cd470 0x7f69940c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699c5be700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 0x7f69941f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.625 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699c5be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b4b50 con 0x7f69940af450 2023-10-16T11:22:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699c5be700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940c6d90 con 0x7f699415dfd0 2023-10-16T11:22:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699c5be700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b4500 con 0x7f69940cd470 2023-10-16T11:22:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69940af450 0x7f69941f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69940af450 0x7f69941f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33980/0 (socket says 172.21.15.181:33980) 2023-10-16T11:22:16.626 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 -- 172.21.15.181:0/1296185979 learned_addr learned my addr 172.21.15.181:0/1296185979 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f6999b59700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69940cd470 0x7f69940c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699ab5b700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 0x7f69941f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 -- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69940cd470 msgr2=0x7f69940c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69940cd470 0x7f69940c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.627 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 -- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 msgr2=0x7f69941f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 0x7f69941f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.615+0000 7f699a35a700 1 -- 172.21.15.181:0/1296185979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990009430 con 0x7f69940af450 2023-10-16T11:22:16.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.616+0000 7f699a35a700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69940af450 0x7f69941f4000 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7f6990009fd0 tx=0x7f6990004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:16.628 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.616+0000 7f698affd700 1 -- 172.21.15.181:0/1296185979 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990003ab0 con 0x7f69940af450 2023-10-16T11:22:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.616+0000 7f698affd700 1 -- 172.21.15.181:0/1296185979 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6990003c10 con 0x7f69940af450 2023-10-16T11:22:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.616+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69940b47e0 con 0x7f69940af450 2023-10-16T11:22:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.616+0000 7f698affd700 1 -- 172.21.15.181:0/1296185979 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699001d430 con 0x7f69940af450 2023-10-16T11:22:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.616+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6994154d20 con 0x7f69940af450 2023-10-16T11:22:16.629 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.617+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69940b4340 con 0x7f69940af450 2023-10-16T11:22:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.617+0000 7f698affd700 1 -- 172.21.15.181:0/1296185979 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f699001d590 con 0x7f69940af450 2023-10-16T11:22:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.618+0000 7f698affd700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 0x7f697806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.618+0000 7f6999b59700 1 -- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 msgr2=0x7f697806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.618+0000 7f698affd700 1 -- 172.21.15.181:0/1296185979 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f699008aec0 con 0x7f69940af450 2023-10-16T11:22:16.630 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.618+0000 7f6999b59700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 0x7f697806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:16.631 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:16.621+0000 7fb3a4504200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T11:22:16.631 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.622+0000 7f698affd700 1 -- 172.21.15.181:0/1296185979 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f6990055c90 con 0x7f69940af450 2023-10-16T11:22:16.787 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:16.786+0000 7fb3a4504200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T11:22:16.808 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:16.807+0000 7f26b8b55200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:16.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.819+0000 7f6999b59700 1 -- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 msgr2=0x7f697806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:16.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.819+0000 7f6999b59700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 0x7f697806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:16.869 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.867+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6994067d20 con 0x7f69940af450 2023-10-16T11:22:16.870 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.868+0000 7f698affd700 1 -- 172.21.15.181:0/1296185979 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f699002b090 con 0x7f69940af450 2023-10-16T11:22:16.871 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:16.871 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":153,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/984505218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":984505218},{"type":"v1","addr":"172.21.15.190:6835","nonce":984505218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/350629849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":350629849},{"type":"v1","addr":"172.21.15.181:6835","nonce":350629849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13440,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1354411415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1354411415},{"type":"v1","addr":"172.21.15.181:6837","nonce":1354411415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:56.485392+0000","modified":"2023-10-16T11:21:59.515101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13452":{"gid":13452,"name":"b","rank":0,"incarnation":149,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/284709222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":284709222},{"type":"v1","addr":"172.21.15.190:6834","nonce":284709222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:22:16.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 msgr2=0x7f697806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:16.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 0x7f697806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69940af450 msgr2=0x7f69941f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:16.873 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69940af450 0x7f69941f4000 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7f6990009fd0 tx=0x7f6990004750 comp rx=0 tx=0).stop 2023-10-16T11:22:16.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 shutdown_connections 2023-10-16T11:22:16.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f697806b020 0x7f697806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f699415dfd0 0x7f69941f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69940af450 0x7f69941f4000 unknown :-1 s=CLOSED pgs=3204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.874 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 --2- 172.21.15.181:0/1296185979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69940cd470 0x7f69940c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:16.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 >> 172.21.15.181:0/1296185979 conn(0x7f69940d04a0 msgr2=0x7f699415d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:16.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 shutdown_connections 2023-10-16T11:22:16.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:16.870+0000 7f699c5be700 1 -- 172.21.15.181:0/1296185979 wait complete. 2023-10-16T11:22:16.876 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 153 2023-10-16T11:22:16.887 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:16.969 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:16.967+0000 7f26b8b55200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:17.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.048+0000 7f53d7511700 1 Processor -- start 2023-10-16T11:22:17.051 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.049+0000 7f53d7511700 1 -- start start 2023-10-16T11:22:17.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.049+0000 7f53d7511700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53d00d2390 0x7f53d00cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.049+0000 7f53d7511700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d00cc470 con 0x7f53d00d0180 2023-10-16T11:22:17.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.049+0000 7f53d7511700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d006ca30 con 0x7f53d00ce7f0 2023-10-16T11:22:17.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.049+0000 7f53d7511700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d006c810 con 0x7f53d00d2390 2023-10-16T11:22:17.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.049+0000 7f53d5aae700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f53d00d0180 0x7f53d00cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45072/0 (socket says 172.21.15.181:45072) 2023-10-16T11:22:17.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53d5aae700 1 -- 172.21.15.181:0/553160076 learned_addr learned my addr 172.21.15.181:0/553160076 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:17.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53d4aac700 1 --2- 172.21.15.181:0/553160076 >> v2:172.21.15.190:3300/0 conn(0x7f53d00d2390 0x7f53d00cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53c7fff700 1 -- 172.21.15.181:0/553160076 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280553511 0 0) 0x7f53d00cc470 con 0x7f53d00d0180 2023-10-16T11:22:17.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53c7fff700 1 -- 172.21.15.181:0/553160076 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53bc0035b0 con 0x7f53d00d0180 2023-10-16T11:22:17.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53d4aac700 1 -- 172.21.15.181:0/553160076 >> v1:172.21.15.190:6789/0 conn(0x7f53d00ce7f0 legacy=0x7f53d006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53c7fff700 1 -- 172.21.15.181:0/553160076 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750244969 0 0) 0x7f53bc0035b0 con 0x7f53d00d0180 2023-10-16T11:22:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53d4aac700 1 -- 172.21.15.181:0/553160076 >> v1:172.21.15.181:6789/0 conn(0x7f53d00d0180 legacy=0x7f53d00cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53d4aac700 1 -- 172.21.15.181:0/553160076 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d006cd10 con 0x7f53d00d2390 2023-10-16T11:22:17.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53d4aac700 1 --2- 172.21.15.181:0/553160076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00d2390 0x7f53d00cfc40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009840 tx=0x7f53c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbbfd492af3e1d64 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53c7fff700 1 -- 172.21.15.181:0/553160076 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80019f0 con 0x7f53d00d2390 2023-10-16T11:22:17.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53c7fff700 1 -- 172.21.15.181:0/553160076 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c0004030 con 0x7f53d00d2390 2023-10-16T11:22:17.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.050+0000 7f53c7fff700 1 -- 172.21.15.181:0/553160076 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00043c0 con 0x7f53d00d2390 2023-10-16T11:22:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.051+0000 7f53d7511700 1 -- 172.21.15.181:0/553160076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00d2390 msgr2=0x7f53d00cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.051+0000 7f53d7511700 1 --2- 172.21.15.181:0/553160076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00d2390 0x7f53d00cfc40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009840 tx=0x7f53c0009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:17.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.051+0000 7f53d7511700 1 -- 172.21.15.181:0/553160076 shutdown_connections 2023-10-16T11:22:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.051+0000 7f53d7511700 1 --2- 172.21.15.181:0/553160076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00d2390 0x7f53d00cfc40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.051+0000 7f53d7511700 1 -- 172.21.15.181:0/553160076 >> 172.21.15.181:0/553160076 conn(0x7f53d009a9a0 msgr2=0x7f53d00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.051+0000 7f53d7511700 1 -- 172.21.15.181:0/553160076 shutdown_connections 2023-10-16T11:22:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.051+0000 7f53d7511700 1 -- 172.21.15.181:0/553160076 wait complete. 2023-10-16T11:22:17.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.052+0000 7f53d7511700 1 Processor -- start 2023-10-16T11:22:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.052+0000 7f53d7511700 1 -- start start 2023-10-16T11:22:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d7511700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00ce7f0 0x7f53d009f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d7511700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d00d0180 0x7f53d014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d7511700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d00d2390 0x7f53d014ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d7511700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d014f2e0 con 0x7f53d00d2390 2023-10-16T11:22:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d7511700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d014f5c0 con 0x7f53d00ce7f0 2023-10-16T11:22:17.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d7511700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d014f8a0 con 0x7f53d00d0180 2023-10-16T11:22:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d00d2390 0x7f53d014ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d00d2390 0x7f53d014ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34000/0 (socket says 172.21.15.181:34000) 2023-10-16T11:22:17.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 -- 172.21.15.181:0/1922762038 learned_addr learned my addr 172.21.15.181:0/1922762038 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:17.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d52ad700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00ce7f0 0x7f53d009f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d4aac700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d00d0180 0x7f53d014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 -- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d00d0180 msgr2=0x7f53d014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d00d0180 0x7f53d014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 -- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00ce7f0 msgr2=0x7f53d009f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00ce7f0 0x7f53d009f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.053+0000 7f53d5aae700 1 -- 172.21.15.181:0/1922762038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00094a0 con 0x7f53d00d2390 2023-10-16T11:22:17.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.054+0000 7f53d5aae700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d00d2390 0x7f53d014ecb0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f53c8001560 tx=0x7f53c800cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.054+0000 7f53c5ffb700 1 -- 172.21.15.181:0/1922762038 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800aaa0 con 0x7f53d00d2390 2023-10-16T11:22:17.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.054+0000 7f53c5ffb700 1 -- 172.21.15.181:0/1922762038 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c800ac20 con 0x7f53d00d2390 2023-10-16T11:22:17.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.054+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d014fb80 con 0x7f53d00d2390 2023-10-16T11:22:17.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.054+0000 7f53c5ffb700 1 -- 172.21.15.181:0/1922762038 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80070d0 con 0x7f53d00d2390 2023-10-16T11:22:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.054+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d0154ad0 con 0x7f53d00d2390 2023-10-16T11:22:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.055+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d0067d20 con 0x7f53d00d2390 2023-10-16T11:22:17.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.056+0000 7f53c5ffb700 1 -- 172.21.15.181:0/1922762038 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f53c8007230 con 0x7f53d00d2390 2023-10-16T11:22:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.056+0000 7f53c5ffb700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 0x7f53b406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.056+0000 7f53d52ad700 1 -- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 msgr2=0x7f53b406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.056+0000 7f53c5ffb700 1 -- 172.21.15.181:0/1922762038 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f53c8089810 con 0x7f53d00d2390 2023-10-16T11:22:17.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.056+0000 7f53d52ad700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 0x7f53b406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:17.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.060+0000 7f53c5ffb700 1 -- 172.21.15.181:0/1922762038 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f53c8054430 con 0x7f53d00d2390 2023-10-16T11:22:17.078 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.066+0000 7fb3a4504200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T11:22:17.238 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.236+0000 7fb3a4504200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T11:22:17.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.257+0000 7f53d52ad700 1 -- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 msgr2=0x7f53b406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:17.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.257+0000 7f53d52ad700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 0x7f53b406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:17.262 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.260+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53d00cef40 con 0x7f53d00d2390 2023-10-16T11:22:17.263 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.261+0000 7f53c5ffb700 1 -- 172.21.15.181:0/1922762038 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v270) v1 ==== 74+0+17919 (secure 0 0 0) 0x7f53c8057a50 con 0x7f53d00d2390 2023-10-16T11:22:17.264 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:17.264 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":270,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:22:10.332242+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:54.910438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:55.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2735196384":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6833/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6832/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/691655073":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/3571285885":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/171534192":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/714342180":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/432126216":"2023-10-17T11:22:10.332184+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:17.266 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 msgr2=0x7f53b406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:17.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 0x7f53b406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d00d2390 msgr2=0x7f53d014ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.267 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d00d2390 0x7f53d014ecb0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f53c8001560 tx=0x7f53c800cec0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 shutdown_connections 2023-10-16T11:22:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f53b406b020 0x7f53b406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d00d2390 0x7f53d014ecb0 unknown :-1 s=CLOSED pgs=3206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d00ce7f0 0x7f53d009f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.268 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 --2- 172.21.15.181:0/1922762038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d00d0180 0x7f53d014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 >> 172.21.15.181:0/1922762038 conn(0x7f53d009a9a0 msgr2=0x7f53d006edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 shutdown_connections 2023-10-16T11:22:17.269 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.264+0000 7f53d7511700 1 -- 172.21.15.181:0/1922762038 wait complete. 2023-10-16T11:22:17.280 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:22:17.280 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:17.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.403+0000 7f4c4ac33700 1 Processor -- start 2023-10-16T11:22:17.405 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.403+0000 7f4c4ac33700 1 -- start start 2023-10-16T11:22:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.404+0000 7f4c4ac33700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c4406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.404+0000 7f4c4ac33700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c440c2dc0 0x7f4c4415e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.404+0000 7f4c4ac33700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c440b2710 con 0x7f4c4406ade0 2023-10-16T11:22:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.404+0000 7f4c4ac33700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440c18f0 con 0x7f4c440c2dc0 2023-10-16T11:22:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.404+0000 7f4c4ac33700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c1bd0 con 0x7f4c4415dc40 2023-10-16T11:22:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.404+0000 7f4c43fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4c4406ade0 0x7f4c440c26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45086/0 (socket says 172.21.15.181:45086) 2023-10-16T11:22:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.404+0000 7f4c43fff700 1 -- 172.21.15.181:0/4224067959 learned_addr learned my addr 172.21.15.181:0/4224067959 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c489cf700 1 --2- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c4406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c491d0700 1 --2- 172.21.15.181:0/4224067959 >> v2:172.21.15.190:3300/0 conn(0x7f4c440c2dc0 0x7f4c4415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c437fe700 1 -- 172.21.15.181:0/4224067959 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803484378 0 0) 0x7f4c30001110 con 0x7f4c4406ade0 2023-10-16T11:22:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c437fe700 1 -- 172.21.15.181:0/4224067959 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c340035b0 con 0x7f4c4406ade0 2023-10-16T11:22:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c491d0700 1 -- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 msgr2=0x7f4c4406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c491d0700 1 --2- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c4406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c491d0700 1 -- 172.21.15.181:0/4224067959 >> v1:172.21.15.181:6789/0 conn(0x7f4c4406ade0 legacy=0x7f4c440c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c491d0700 1 -- 172.21.15.181:0/4224067959 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440c1eb0 con 0x7f4c440c2dc0 2023-10-16T11:22:17.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c489cf700 1 --2- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c4406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:17.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c491d0700 1 --2- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 0x7f4c4415e020 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009d90 tx=0x7f4c3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7bdd445c0a6206 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c437fe700 1 -- 172.21.15.181:0/4224067959 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00e070 con 0x7f4c440c2dc0 2023-10-16T11:22:17.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c437fe700 1 -- 172.21.15.181:0/4224067959 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00bb20 con 0x7f4c440c2dc0 2023-10-16T11:22:17.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.405+0000 7f4c437fe700 1 -- 172.21.15.181:0/4224067959 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c012430 con 0x7f4c440c2dc0 2023-10-16T11:22:17.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 -- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 msgr2=0x7f4c4415e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 0x7f4c4415e020 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009d90 tx=0x7f4c3c00b330 comp rx=0 tx=0).stop 2023-10-16T11:22:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 -- 172.21.15.181:0/4224067959 shutdown_connections 2023-10-16T11:22:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 0x7f4c4415e020 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/4224067959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c4406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 -- 172.21.15.181:0/4224067959 >> 172.21.15.181:0/4224067959 conn(0x7f4c440b1780 msgr2=0x7f4c440b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 -- 172.21.15.181:0/4224067959 shutdown_connections 2023-10-16T11:22:17.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.406+0000 7f4c4ac33700 1 -- 172.21.15.181:0/4224067959 wait complete. 2023-10-16T11:22:17.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.408+0000 7f4c4ac33700 1 Processor -- start 2023-10-16T11:22:17.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.408+0000 7f4c4ac33700 1 -- start start 2023-10-16T11:22:17.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.409+0000 7f4c4ac33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c4406ade0 0x7f4c4406e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.409+0000 7f4c4ac33700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 0x7f4c4406cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.409+0000 7f4c4ac33700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c440c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.409+0000 7f4c4ac33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4406bc80 con 0x7f4c4406ade0 2023-10-16T11:22:17.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.409+0000 7f4c4ac33700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44166e50 con 0x7f4c440c2dc0 2023-10-16T11:22:17.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.409+0000 7f4c4ac33700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4406d4f0 con 0x7f4c4415dc40 2023-10-16T11:22:17.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c4406ade0 0x7f4c4406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c4406ade0 0x7f4c4406e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34012/0 (socket says 172.21.15.181:34012) 2023-10-16T11:22:17.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 -- 172.21.15.181:0/3865108877 learned_addr learned my addr 172.21.15.181:0/3865108877 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:17.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c43fff700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 0x7f4c4406cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c491d0700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c440c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 -- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 msgr2=0x7f4c440c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c440c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 -- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 msgr2=0x7f4c4406cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 0x7f4c4406cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 -- 172.21.15.181:0/3865108877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380094a0 con 0x7f4c4406ade0 2023-10-16T11:22:17.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c489cf700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c4406ade0 0x7f4c4406e590 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800ebe0 tx=0x7f4c3800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c417fa700 1 -- 172.21.15.181:0/3865108877 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800c850 con 0x7f4c4406ade0 2023-10-16T11:22:17.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c417fa700 1 -- 172.21.15.181:0/3865108877 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3800c9b0 con 0x7f4c4406ade0 2023-10-16T11:22:17.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c009a70 con 0x7f4c4406ade0 2023-10-16T11:22:17.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c417fa700 1 -- 172.21.15.181:0/3865108877 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800cd40 con 0x7f4c4406ade0 2023-10-16T11:22:17.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.410+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4406dab0 con 0x7f4c4406ade0 2023-10-16T11:22:17.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.411+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c44067d20 con 0x7f4c4406ade0 2023-10-16T11:22:17.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.412+0000 7f4c417fa700 1 -- 172.21.15.181:0/3865108877 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4c38013470 con 0x7f4c4406ade0 2023-10-16T11:22:17.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.412+0000 7f4c417fa700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 0x7f4c2806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.412+0000 7f4c43fff700 1 -- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 msgr2=0x7f4c2806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:17.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.412+0000 7f4c417fa700 1 -- 172.21.15.181:0/3865108877 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f4c38089bd0 con 0x7f4c4406ade0 2023-10-16T11:22:17.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.412+0000 7f4c43fff700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 0x7f4c2806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:17.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.417+0000 7f4c417fa700 1 -- 172.21.15.181:0/3865108877 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4c380547f0 con 0x7f4c4406ade0 2023-10-16T11:22:17.502 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-10-16T11:22:17.500+0000 7f26b8b55200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:17.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.613+0000 7f4c43fff700 1 -- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 msgr2=0x7f4c2806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:17.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.613+0000 7f4c43fff700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 0x7f4c2806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T11:22:17.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.663+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c440c7e90 con 0x7f4c4406ade0 2023-10-16T11:22:17.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.663+0000 7f4c417fa700 1 -- 172.21.15.181:0/3865108877 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f4c38021090 con 0x7f4c4406ade0 2023-10-16T11:22:17.666 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:17.666 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":153,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/984505218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":984505218},{"type":"v1","addr":"172.21.15.190:6835","nonce":984505218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/350629849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":350629849},{"type":"v1","addr":"172.21.15.181:6835","nonce":350629849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13440,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1354411415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1354411415},{"type":"v1","addr":"172.21.15.181:6837","nonce":1354411415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:56.485392+0000","modified":"2023-10-16T11:21:59.515101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13452":{"gid":13452,"name":"b","rank":0,"incarnation":149,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/284709222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":284709222},{"type":"v1","addr":"172.21.15.190:6834","nonce":284709222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:22:17.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 msgr2=0x7f4c2806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 0x7f4c2806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c4406ade0 msgr2=0x7f4c4406e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c4406ade0 0x7f4c4406e590 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800ebe0 tx=0x7f4c3800eef0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 shutdown_connections 2023-10-16T11:22:17.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f4c2806b020 0x7f4c2806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c4415dc40 0x7f4c440c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c4406ade0 0x7f4c4406e590 unknown :-1 s=CLOSED pgs=3208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 --2- 172.21.15.181:0/3865108877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c2dc0 0x7f4c4406cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 >> 172.21.15.181:0/3865108877 conn(0x7f4c440b1780 msgr2=0x7f4c4406b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 shutdown_connections 2023-10-16T11:22:17.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.666+0000 7f4c4ac33700 1 -- 172.21.15.181:0/3865108877 wait complete. 2023-10-16T11:22:17.671 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 153 2023-10-16T11:22:17.681 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:17.804 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.802+0000 7f563a6a6700 1 Processor -- start 2023-10-16T11:22:17.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.803+0000 7f563a6a6700 1 -- start start 2023-10-16T11:22:17.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.803+0000 7f563a6a6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56340d22e0 0x7f56340b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.803+0000 7f563a6a6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340b2f00 con 0x7f563415dc20 2023-10-16T11:22:17.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.803+0000 7f563a6a6700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340b31e0 con 0x7f56340d1e60 2023-10-16T11:22:17.807 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.803+0000 7f563a6a6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56340b3b80 con 0x7f56340d22e0 2023-10-16T11:22:17.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.803+0000 7f5633fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f563415dc20 0x7f56340d41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45100/0 (socket says 172.21.15.181:45100) 2023-10-16T11:22:17.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5633fff700 1 -- 172.21.15.181:0/1319929584 learned_addr learned my addr 172.21.15.181:0/1319929584 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:17.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5638c43700 1 --2- 172.21.15.181:0/1319929584 >> v2:172.21.15.190:3300/0 conn(0x7f56340d22e0 0x7f56340b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5632ffd700 1 -- 172.21.15.181:0/1319929584 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321827556 0 0) 0x7f56340b2f00 con 0x7f563415dc20 2023-10-16T11:22:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5632ffd700 1 -- 172.21.15.181:0/1319929584 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f561c0035b0 con 0x7f563415dc20 2023-10-16T11:22:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5632ffd700 1 -- 172.21.15.181:0/1319929584 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3040018620 0 0) 0x7f561c0035b0 con 0x7f563415dc20 2023-10-16T11:22:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5638c43700 1 -- 172.21.15.181:0/1319929584 >> v1:172.21.15.190:6789/0 conn(0x7f56340d1e60 legacy=0x7f56340b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5638c43700 1 -- 172.21.15.181:0/1319929584 >> v1:172.21.15.181:6789/0 conn(0x7f563415dc20 legacy=0x7f56340d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5638c43700 1 -- 172.21.15.181:0/1319929584 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340b34c0 con 0x7f56340d22e0 2023-10-16T11:22:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5638c43700 1 --2- 172.21.15.181:0/1319929584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d22e0 0x7f56340b46b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f562c009d90 tx=0x7f562c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d63e89cdeea56a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5632ffd700 1 -- 172.21.15.181:0/1319929584 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c00e070 con 0x7f56340d22e0 2023-10-16T11:22:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5632ffd700 1 -- 172.21.15.181:0/1319929584 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c003a20 con 0x7f56340d22e0 2023-10-16T11:22:17.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.804+0000 7f5632ffd700 1 -- 172.21.15.181:0/1319929584 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56240017f0 con 0x7f56340d22e0 2023-10-16T11:22:17.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.805+0000 7f563a6a6700 1 -- 172.21.15.181:0/1319929584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d22e0 msgr2=0x7f56340b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.811 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.805+0000 7f563a6a6700 1 --2- 172.21.15.181:0/1319929584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d22e0 0x7f56340b46b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f562c009d90 tx=0x7f562c00b330 comp rx=0 tx=0).stop 2023-10-16T11:22:17.811 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.805+0000 7fb3a4504200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T11:22:17.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.805+0000 7f563a6a6700 1 -- 172.21.15.181:0/1319929584 shutdown_connections 2023-10-16T11:22:17.814 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.805+0000 7f563a6a6700 1 --2- 172.21.15.181:0/1319929584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d22e0 0x7f56340b46b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.805+0000 7f563a6a6700 1 -- 172.21.15.181:0/1319929584 >> 172.21.15.181:0/1319929584 conn(0x7f563408af90 msgr2=0x7f56340d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:17.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.805+0000 7f563a6a6700 1 -- 172.21.15.181:0/1319929584 shutdown_connections 2023-10-16T11:22:17.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.805+0000 7f563a6a6700 1 -- 172.21.15.181:0/1319929584 wait complete. 2023-10-16T11:22:17.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.806+0000 7f563a6a6700 1 Processor -- start 2023-10-16T11:22:17.815 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.806+0000 7f563a6a6700 1 -- start start 2023-10-16T11:22:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f563a6a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d1e60 0x7f56341f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f563a6a6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56340d22e0 0x7f5634153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f563a6a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563415dc20 0x7f563415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f563a6a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56340cbaa0 con 0x7f563415dc20 2023-10-16T11:22:17.816 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f563a6a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341f2ca0 con 0x7f56340d1e60 2023-10-16T11:22:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f563a6a6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56341546b0 con 0x7f56340d22e0 2023-10-16T11:22:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f5638c43700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563415dc20 0x7f563415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f5638c43700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563415dc20 0x7f563415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34032/0 (socket says 172.21.15.181:34032) 2023-10-16T11:22:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f5638c43700 1 -- 172.21.15.181:0/3945938617 learned_addr learned my addr 172.21.15.181:0/3945938617 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:17.817 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f5633fff700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d1e60 0x7f56341f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.807+0000 7f56337fe700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56340d22e0 0x7f5634153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5638c43700 1 -- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56340d22e0 msgr2=0x7f5634153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5638c43700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56340d22e0 0x7f5634153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5638c43700 1 -- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d1e60 msgr2=0x7f56341f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:17.818 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5638c43700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d1e60 0x7f56341f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5638c43700 1 -- 172.21.15.181:0/3945938617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c009a70 con 0x7f563415dc20 2023-10-16T11:22:17.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5638c43700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563415dc20 0x7f563415f800 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 crypto rx=0x7f563408d0d0 tx=0x7f562c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:17.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c009450 con 0x7f563415dc20 2023-10-16T11:22:17.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c004610 con 0x7f563415dc20 2023-10-16T11:22:17.819 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5634154990 con 0x7f563415dc20 2023-10-16T11:22:17.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c012450 con 0x7f563415dc20 2023-10-16T11:22:17.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.808+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56340cbfd0 con 0x7f563415dc20 2023-10-16T11:22:17.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.809+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563406c610 con 0x7f563415dc20 2023-10-16T11:22:17.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.810+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 174) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f562c0125b0 con 0x7f563415dc20 2023-10-16T11:22:17.820 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.810+0000 7f5630ff9700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f562006b020 0x7f562006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:17.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.810+0000 7f5633fff700 1 -- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f562006b020 msgr2=0x7f562006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2902800927 2023-10-16T11:22:17.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.810+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(270..270 src has 1..270) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f562c08bbc0 con 0x7f563415dc20 2023-10-16T11:22:17.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.810+0000 7f5633fff700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f562006b020 0x7f562006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T11:22:17.821 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.814+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f562c056760 con 0x7f563415dc20 2023-10-16T11:22:17.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.829+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 175) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f562c089d60 con 0x7f563415dc20 2023-10-16T11:22:17.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.829+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f562006b020 msgr2=0x7f562006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T11:22:17.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:17.829+0000 7f5630ff9700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f562006b020 0x7f562006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:17.893 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31ccea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.893 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31ccea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.893 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31ccea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.894 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31ccea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.894 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31ccea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.894 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31fd30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.894 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31fd30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.894 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31fd30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.895 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31fd30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:17.895 INFO:tasks.ceph.mgr.y.smithi181.stderr:2023-10-16T11:22:17.891+0000 7fb31fd30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T11:22:18.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.068+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56340cf140 con 0x7f563415dc20 2023-10-16T11:22:18.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.069+0000 7f5630ff9700 1 -- 172.21.15.181:0/3945938617 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+4932 (secure 0 0 0) 0x7f562c071560 con 0x7f563415dc20 2023-10-16T11:22:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:18.071 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":153,"default_fscid":18,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/984505218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":984505218},{"type":"v1","addr":"172.21.15.190:6835","nonce":984505218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/350629849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":350629849},{"type":"v1","addr":"172.21.15.181:6835","nonce":350629849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13440,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1354411415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1354411415},{"type":"v1","addr":"172.21.15.181:6837","nonce":1354411415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T11:21:56.485392+0000","modified":"2023-10-16T11:21:59.515101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":13452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_13452":{"gid":13452,"name":"b","rank":0,"incarnation":149,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/284709222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":284709222},{"type":"v1","addr":"172.21.15.190:6834","nonce":284709222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[37],"metadata_pool":36,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":18}]} 2023-10-16T11:22:18.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563415dc20 msgr2=0x7f563415f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563415dc20 0x7f563415f800 secure :-1 s=READY pgs=3210 cs=0 l=1 rev1=1 crypto rx=0x7f563408d0d0 tx=0x7f562c004ad0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 shutdown_connections 2023-10-16T11:22:18.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:6832/2902800927,v1:172.21.15.181:6833/2902800927] conn(0x7f562006b020 0x7f562006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f563415dc20 0x7f563415f800 unknown :-1 s=CLOSED pgs=3210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56340d1e60 0x7f56341f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 --2- 172.21.15.181:0/3945938617 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56340d22e0 0x7f5634153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 >> 172.21.15.181:0/3945938617 conn(0x7f563408af90 msgr2=0x7f563406ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 shutdown_connections 2023-10-16T11:22:18.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.071+0000 7f563a6a6700 1 -- 172.21.15.181:0/3945938617 wait complete. 2023-10-16T11:22:18.075 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 153 2023-10-16T11:22:18.085 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:18.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.205+0000 7fccb1465700 1 Processor -- start 2023-10-16T11:22:18.208 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.205+0000 7fccb1465700 1 -- start start 2023-10-16T11:22:18.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.206+0000 7fccb1465700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccac0ce9a0 0x7fccac162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.206+0000 7fccb1465700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac0703b0 con 0x7fccac06a8a0 2023-10-16T11:22:18.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.206+0000 7fccb1465700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac15eeb0 con 0x7fccac15e700 2023-10-16T11:22:18.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.206+0000 7fccb1465700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac1584d0 con 0x7fccac0ce9a0 2023-10-16T11:22:18.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.206+0000 7fccaaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccac0ce9a0 0x7fccac162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.206+0000 7fccaaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccac0ce9a0 0x7fccac162140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34108/0 (socket says 172.21.15.181:34108) 2023-10-16T11:22:18.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.206+0000 7fccaaffd700 1 -- 172.21.15.181:0/1876063555 learned_addr learned my addr 172.21.15.181:0/1876063555 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:18.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fccaaffd700 1 -- 172.21.15.181:0/1876063555 >> v1:172.21.15.190:6789/0 conn(0x7fccac15e700 legacy=0x7fccac071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fccaaffd700 1 -- 172.21.15.181:0/1876063555 >> v1:172.21.15.181:6789/0 conn(0x7fccac06a8a0 legacy=0x7fccac15dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fccaaffd700 1 -- 172.21.15.181:0/1876063555 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac15ebd0 con 0x7fccac0ce9a0 2023-10-16T11:22:18.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fccaaffd700 1 --2- 172.21.15.181:0/1876063555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac162140 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7fcca00098f0 tx=0x7fcca0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0f3b13dba5b3a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fcca9ffb700 1 -- 172.21.15.181:0/1876063555 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca000b8a0 con 0x7fccac0ce9a0 2023-10-16T11:22:18.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fcca9ffb700 1 -- 172.21.15.181:0/1876063555 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca000ba00 con 0x7fccac0ce9a0 2023-10-16T11:22:18.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fcca9ffb700 1 -- 172.21.15.181:0/1876063555 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca000bd90 con 0x7fccac0ce9a0 2023-10-16T11:22:18.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fccb1465700 1 -- 172.21.15.181:0/1876063555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 msgr2=0x7fccac162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.207+0000 7fccb1465700 1 --2- 172.21.15.181:0/1876063555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac162140 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7fcca00098f0 tx=0x7fcca0009c00 comp rx=0 tx=0).stop 2023-10-16T11:22:18.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.208+0000 7fccb1465700 1 -- 172.21.15.181:0/1876063555 shutdown_connections 2023-10-16T11:22:18.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.208+0000 7fccb1465700 1 --2- 172.21.15.181:0/1876063555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac162140 unknown :-1 s=CLOSED pgs=3220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.208+0000 7fccb1465700 1 -- 172.21.15.181:0/1876063555 >> 172.21.15.181:0/1876063555 conn(0x7fccac0ccf80 msgr2=0x7fccac0cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.208+0000 7fccb1465700 1 -- 172.21.15.181:0/1876063555 shutdown_connections 2023-10-16T11:22:18.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.208+0000 7fccb1465700 1 -- 172.21.15.181:0/1876063555 wait complete. 2023-10-16T11:22:18.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.209+0000 7fccb1465700 1 Processor -- start 2023-10-16T11:22:18.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.209+0000 7fccb1465700 1 -- start start 2023-10-16T11:22:18.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac06a8a0 0x7fccac0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac15e700 0x7fccac0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac15af10 con 0x7fccac0ce9a0 2023-10-16T11:22:18.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac15d610 con 0x7fccac15e700 2023-10-16T11:22:18.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac15d8f0 con 0x7fccac06a8a0 2023-10-16T11:22:18.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac15cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34114/0 (socket says 172.21.15.181:34114) 2023-10-16T11:22:18.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 -- 172.21.15.181:0/3484240456 learned_addr learned my addr 172.21.15.181:0/3484240456 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:18.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaaffd700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac06a8a0 0x7fccac0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccab7fe700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac15e700 0x7fccac0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 -- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac06a8a0 msgr2=0x7fccac0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac06a8a0 0x7fccac0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 -- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac15e700 msgr2=0x7fccac0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac15e700 0x7fccac0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 -- 172.21.15.181:0/3484240456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0009550 con 0x7fccac0ce9a0 2023-10-16T11:22:18.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccaa7fc700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac15cd90 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c4e0 tx=0x7fcc9c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fcc937fe700 1 -- 172.21.15.181:0/3484240456 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0052d0 con 0x7fccac0ce9a0 2023-10-16T11:22:18.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fcc937fe700 1 -- 172.21.15.181:0/3484240456 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c005430 con 0x7fccac0ce9a0 2023-10-16T11:22:18.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac1586c0 con 0x7fccac0ce9a0 2023-10-16T11:22:18.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fcc937fe700 1 -- 172.21.15.181:0/3484240456 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0038c0 con 0x7fccac0ce9a0 2023-10-16T11:22:18.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.210+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac158c80 con 0x7fccac0ce9a0 2023-10-16T11:22:18.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.211+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac04ebb0 con 0x7fccac0ce9a0 2023-10-16T11:22:18.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.212+0000 7fcc937fe700 1 -- 172.21.15.181:0/3484240456 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 175) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fcc9c003a20 con 0x7fccac0ce9a0 2023-10-16T11:22:18.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.212+0000 7fcc937fe700 1 -- 172.21.15.181:0/3484240456 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fcc9c088b90 con 0x7fccac0ce9a0 2023-10-16T11:22:18.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.217+0000 7fcc937fe700 1 -- 172.21.15.181:0/3484240456 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fcc9c053a40 con 0x7fccac0ce9a0 2023-10-16T11:22:18.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.411+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccac06f650 con 0x7fccac0ce9a0 2023-10-16T11:22:18.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.412+0000 7fcc937fe700 1 -- 172.21.15.181:0/3484240456 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+18428 (secure 0 0 0) 0x7fcc9c057060 con 0x7fccac0ce9a0 2023-10-16T11:22:18.415 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:18.415 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":271,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:22:17.819462+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_metadata","create_time":"2023-10-16T11:21:54.910438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"268","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":37,"pool_name":"cephfs_data","create_time":"2023-10-16T11:21:55.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"269","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1173792749":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/2911320793":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/290359218":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/3577989439":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/1668082750":"2023-10-17T11:22:17.819401+0000","172.21.15.181:6833/2902800927":"2023-10-17T11:22:17.819401+0000","172.21.15.181:6832/2902800927":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/3788072243":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/2735196384":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6833/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6832/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/691655073":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/3571285885":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/171534192":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/714342180":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/432126216":"2023-10-17T11:22:10.332184+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:18.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 msgr2=0x7fccac15cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac15cd90 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c4e0 tx=0x7fcc9c00c8a0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 shutdown_connections 2023-10-16T11:22:18.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccac06a8a0 0x7fccac0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccac15e700 0x7fccac0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 --2- 172.21.15.181:0/3484240456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccac0ce9a0 0x7fccac15cd90 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 >> 172.21.15.181:0/3484240456 conn(0x7fccac0ccf80 msgr2=0x7fccac15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 shutdown_connections 2023-10-16T11:22:18.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.414+0000 7fccb1465700 1 -- 172.21.15.181:0/3484240456 wait complete. 2023-10-16T11:22:18.431 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T11:22:18.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.549+0000 7fa76a091700 1 Processor -- start 2023-10-16T11:22:18.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.549+0000 7fa76a091700 1 -- start start 2023-10-16T11:22:18.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa76a091700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa76415df60 0x7fa7640cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa76a091700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa7640d0fc0 0x7fa7640cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa76a091700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7640ccf50 con 0x7fa76406a8a0 2023-10-16T11:22:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa76a091700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7640af9c0 con 0x7fa76415df60 2023-10-16T11:22:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa76a091700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7640cc600 con 0x7fa7640d0fc0 2023-10-16T11:22:18.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa7637fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa76415df60 0x7fa7640cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa7637fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa76415df60 0x7fa7640cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34118/0 (socket says 172.21.15.181:34118) 2023-10-16T11:22:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.550+0000 7fa7637fe700 1 -- 172.21.15.181:0/244551647 learned_addr learned my addr 172.21.15.181:0/244551647 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa762ffd700 1 --2- 172.21.15.181:0/244551647 >> v2:172.21.15.190:3300/0 conn(0x7fa7640d0fc0 0x7fa7640cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7637fe700 1 -- 172.21.15.181:0/244551647 >> v2:172.21.15.190:3300/0 conn(0x7fa7640d0fc0 msgr2=0x7fa7640cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:18.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7637fe700 1 --2- 172.21.15.181:0/244551647 >> v2:172.21.15.190:3300/0 conn(0x7fa7640d0fc0 0x7fa7640cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7637fe700 1 -- 172.21.15.181:0/244551647 >> v1:172.21.15.181:6789/0 conn(0x7fa76406a8a0 legacy=0x7fa76406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7637fe700 1 -- 172.21.15.181:0/244551647 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640cc8a0 con 0x7fa76415df60 2023-10-16T11:22:18.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7637fe700 1 --2- 172.21.15.181:0/244551647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76415df60 0x7fa7640cae20 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7fa76406c990 tx=0x7fa758009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e491843808bc9cc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7627fc700 1 -- 172.21.15.181:0/244551647 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758005e30 con 0x7fa76415df60 2023-10-16T11:22:18.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7627fc700 1 -- 172.21.15.181:0/244551647 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa758004030 con 0x7fa76415df60 2023-10-16T11:22:18.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.551+0000 7fa7627fc700 1 -- 172.21.15.181:0/244551647 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580043e0 con 0x7fa76415df60 2023-10-16T11:22:18.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 -- 172.21.15.181:0/244551647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76415df60 msgr2=0x7fa7640cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 --2- 172.21.15.181:0/244551647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76415df60 0x7fa7640cae20 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7fa76406c990 tx=0x7fa758009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 -- 172.21.15.181:0/244551647 shutdown_connections 2023-10-16T11:22:18.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 --2- 172.21.15.181:0/244551647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76415df60 0x7fa7640cae20 unknown :-1 s=CLOSED pgs=3223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 --2- 172.21.15.181:0/244551647 >> v2:172.21.15.190:3300/0 conn(0x7fa7640d0fc0 0x7fa7640cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 -- 172.21.15.181:0/244551647 >> 172.21.15.181:0/244551647 conn(0x7fa7640d04a0 msgr2=0x7fa7640d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 -- 172.21.15.181:0/244551647 shutdown_connections 2023-10-16T11:22:18.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.552+0000 7fa76a091700 1 -- 172.21.15.181:0/244551647 wait complete. 2023-10-16T11:22:18.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.553+0000 7fa76a091700 1 Processor -- start 2023-10-16T11:22:18.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.553+0000 7fa76a091700 1 -- start start 2023-10-16T11:22:18.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa76a091700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76406a8a0 0x7fa7640c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa76a091700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7640d0fc0 0x7fa7640b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa76a091700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa76415df60 0x7fa7641f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa76a091700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c8c10 con 0x7fa76406a8a0 2023-10-16T11:22:18.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa76a091700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640b5ec0 con 0x7fa76415df60 2023-10-16T11:22:18.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa76a091700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641f4020 con 0x7fa7640d0fc0 2023-10-16T11:22:18.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76406a8a0 0x7fa7640c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76406a8a0 0x7fa7640c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34124/0 (socket says 172.21.15.181:34124) 2023-10-16T11:22:18.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 -- 172.21.15.181:0/668407568 learned_addr learned my addr 172.21.15.181:0/668407568 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:18.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa763fff700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa76415df60 0x7fa7641f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa762ffd700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7640d0fc0 0x7fa7640b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 -- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7640d0fc0 msgr2=0x7fa7640b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7640d0fc0 0x7fa7640b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 -- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa76415df60 msgr2=0x7fa7641f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa76415df60 0x7fa7641f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 -- 172.21.15.181:0/668407568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa758009500 con 0x7fa76406a8a0 2023-10-16T11:22:18.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa7637fe700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76406a8a0 0x7fa7640c7570 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7fa7640716a0 tx=0x7fa758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.554+0000 7fa74bfff700 1 -- 172.21.15.181:0/668407568 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758003e10 con 0x7fa76406a8a0 2023-10-16T11:22:18.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.555+0000 7fa74bfff700 1 -- 172.21.15.181:0/668407568 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75801d430 con 0x7fa76406a8a0 2023-10-16T11:22:18.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.555+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640c7b70 con 0x7fa76406a8a0 2023-10-16T11:22:18.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.555+0000 7fa74bfff700 1 -- 172.21.15.181:0/668407568 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75801d7e0 con 0x7fa76406a8a0 2023-10-16T11:22:18.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.555+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640c8130 con 0x7fa76406a8a0 2023-10-16T11:22:18.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.555+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa764067d20 con 0x7fa76406a8a0 2023-10-16T11:22:18.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.556+0000 7fa74bfff700 1 -- 172.21.15.181:0/668407568 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 175) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa75801d940 con 0x7fa76406a8a0 2023-10-16T11:22:18.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.556+0000 7fa74bfff700 1 -- 172.21.15.181:0/668407568 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 6949+0+0 (secure 0 0 0) 0x7fa75808a910 con 0x7fa76406a8a0 2023-10-16T11:22:18.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.561+0000 7fa74bfff700 1 -- 172.21.15.181:0/668407568 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa758055930 con 0x7fa76406a8a0 2023-10-16T11:22:18.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.800+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa764066fb0 con 0x7fa76406a8a0 2023-10-16T11:22:18.802 INFO:tasks.ceph.mon.a.smithi181.stderr:2023-10-16T11:22:18.800+0000 7f2a35b2a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T11:22:18.805 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.804+0000 7fa74bfff700 1 -- 172.21.15.181:0/668407568 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v154) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa758058f50 con 0x7fa76406a8a0 2023-10-16T11:22:18.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76406a8a0 msgr2=0x7fa7640c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.808 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76406a8a0 0x7fa7640c7570 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7fa7640716a0 tx=0x7fa758004750 comp rx=0 tx=0).stop 2023-10-16T11:22:18.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 shutdown_connections 2023-10-16T11:22:18.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa76415df60 0x7fa7641f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa76406a8a0 0x7fa7640c7570 unknown :-1 s=CLOSED pgs=3224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 --2- 172.21.15.181:0/668407568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7640d0fc0 0x7fa7640b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.809 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 >> 172.21.15.181:0/668407568 conn(0x7fa7640d04a0 msgr2=0x7fa76415cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.806+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 shutdown_connections 2023-10-16T11:22:18.810 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.807+0000 7fa76a091700 1 -- 172.21.15.181:0/668407568 wait complete. 2023-10-16T11:22:18.811 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T11:22:18.812 INFO:tasks.ceph.mds.b.smithi190.stderr: -3628> 2023-10-16T11:22:03.067+0000 7f1dd2fe5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T11:22:18.822 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T11:22:18.846 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-10-16T11:22:18.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9aa1274700 1 Processor -- start 2023-10-16T11:22:18.950 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9aa1274700 1 -- start start 2023-10-16T11:22:18.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9aa1274700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9a9c0d01d0 0x7f9a9c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9aa1274700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c0d1500 0x7f9a9c0d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9aa1274700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c0cf580 con 0x7f9a9c15ea40 2023-10-16T11:22:18.953 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9aa1274700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0d4290 con 0x7f9a9c0d01d0 2023-10-16T11:22:18.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9aa1274700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0ce970 con 0x7f9a9c0d1500 2023-10-16T11:22:18.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9a9c0d01d0 0x7f9a9c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9a7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9a9c15ea40 0x7f9a9c0d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45230/0 (socket says 172.21.15.181:45230) 2023-10-16T11:22:18.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9a7fc700 1 -- 172.21.15.181:0/1120637597 learned_addr learned my addr 172.21.15.181:0/1120637597 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:18.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9b7fe700 1 --2- 172.21.15.181:0/1120637597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c0d1500 0x7f9a9c0d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9affd700 1 -- 172.21.15.181:0/1120637597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c0d1500 msgr2=0x7f9a9c0d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:18.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9affd700 1 --2- 172.21.15.181:0/1120637597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c0d1500 0x7f9a9c0d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.955 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9affd700 1 -- 172.21.15.181:0/1120637597 >> v1:172.21.15.181:6789/0 conn(0x7f9a9c15ea40 legacy=0x7f9a9c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9affd700 1 -- 172.21.15.181:0/1120637597 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c0cec50 con 0x7f9a9c0d01d0 2023-10-16T11:22:18.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a99ffb700 1 -- 172.21.15.181:0/1120637597 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900945964 0 0) 0x7f9a9c0cf580 con 0x7f9a9c15ea40 2023-10-16T11:22:18.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a9affd700 1 --2- 172.21.15.181:0/1120637597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c0ce2d0 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009840 tx=0x7f9a90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7abf70cf723c9f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.956 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a99ffb700 1 -- 172.21.15.181:0/1120637597 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90011830 con 0x7f9a9c0d01d0 2023-10-16T11:22:18.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a99ffb700 1 -- 172.21.15.181:0/1120637597 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a90011990 con 0x7f9a9c0d01d0 2023-10-16T11:22:18.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.949+0000 7f9a99ffb700 1 -- 172.21.15.181:0/1120637597 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90011d20 con 0x7f9a9c0d01d0 2023-10-16T11:22:18.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 -- 172.21.15.181:0/1120637597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d01d0 msgr2=0x7f9a9c0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 --2- 172.21.15.181:0/1120637597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c0ce2d0 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009840 tx=0x7f9a90009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:18.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 -- 172.21.15.181:0/1120637597 shutdown_connections 2023-10-16T11:22:18.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 --2- 172.21.15.181:0/1120637597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c0d1500 0x7f9a9c0d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 --2- 172.21.15.181:0/1120637597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c0ce2d0 unknown :-1 s=CLOSED pgs=3227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 -- 172.21.15.181:0/1120637597 >> 172.21.15.181:0/1120637597 conn(0x7f9a9c09cb60 msgr2=0x7f9a9c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:18.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 -- 172.21.15.181:0/1120637597 shutdown_connections 2023-10-16T11:22:18.959 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.950+0000 7f9aa1274700 1 -- 172.21.15.181:0/1120637597 wait complete. 2023-10-16T11:22:18.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.951+0000 7f9aa1274700 1 Processor -- start 2023-10-16T11:22:18.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.951+0000 7f9aa1274700 1 -- start start 2023-10-16T11:22:18.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9aa1274700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c14d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9aa1274700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d1500 0x7f9a9c149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9aa1274700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c15ea40 0x7f9a9c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9aa1274700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0b5ac0 con 0x7f9a9c0d1500 2023-10-16T11:22:18.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9aa1274700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c14d780 con 0x7f9a9c0d01d0 2023-10-16T11:22:18.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9aa1274700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0b4e40 con 0x7f9a9c15ea40 2023-10-16T11:22:18.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d1500 0x7f9a9c149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d1500 0x7f9a9c149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34160/0 (socket says 172.21.15.181:34160) 2023-10-16T11:22:18.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 -- 172.21.15.181:0/2172009788 learned_addr learned my addr 172.21.15.181:0/2172009788 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:18.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9b7fe700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c15ea40 0x7f9a9c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9affd700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.964 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 -- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c15ea40 msgr2=0x7f9a9c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c15ea40 0x7f9a9c15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 -- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9c0d01d0 msgr2=0x7f9a9c14d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:18.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c14d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:18.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9a7fc700 1 -- 172.21.15.181:0/2172009788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900094a0 con 0x7f9a9c0d1500 2023-10-16T11:22:18.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.952+0000 7f9a9affd700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c14d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:18.965 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.953+0000 7f9a9a7fc700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d1500 0x7f9a9c149ee0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0011e0 tx=0x7f9a8c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.953+0000 7f9a837fe700 1 -- 172.21.15.181:0/2172009788 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c004c80 con 0x7f9a9c0d1500 2023-10-16T11:22:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.953+0000 7f9a837fe700 1 -- 172.21.15.181:0/2172009788 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c004e00 con 0x7f9a9c0d1500 2023-10-16T11:22:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.953+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c0b5120 con 0x7f9a9c0d1500 2023-10-16T11:22:18.966 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.953+0000 7f9a837fe700 1 -- 172.21.15.181:0/2172009788 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c007310 con 0x7f9a9c0d1500 2023-10-16T11:22:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.953+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c0b56e0 con 0x7f9a9c0d1500 2023-10-16T11:22:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.954+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c0b5c00 con 0x7f9a9c0d1500 2023-10-16T11:22:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.954+0000 7f9a837fe700 1 -- 172.21.15.181:0/2172009788 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 176) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f9a8c007470 con 0x7f9a9c0d1500 2023-10-16T11:22:18.967 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.955+0000 7f9a837fe700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9a8806ad50 0x7f9a8806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:18.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.955+0000 7f9a837fe700 1 -- 172.21.15.181:0/2172009788 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 7035+0+0 (secure 0 0 0) 0x7f9a8c089500 con 0x7f9a9c0d1500 2023-10-16T11:22:18.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.955+0000 7f9a9affd700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9a8806ad50 0x7f9a8806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:18.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.955+0000 7f9a9affd700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9a8806ad50 0x7f9a8806d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a900090b0 tx=0x7f9a90008030 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:18.968 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:18.959+0000 7f9a837fe700 1 -- 172.21.15.181:0/2172009788 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9a9c0b5c00 con 0x7f9a9c0d1500 2023-10-16T11:22:19.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.196+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9a9c0b5c00 con 0x7f9a9c0d1500 2023-10-16T11:22:19.833 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.832+0000 7f9a837fe700 1 -- 172.21.15.181:0/2172009788 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v155) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9a9c0b5c00 con 0x7f9a9c0d1500 2023-10-16T11:22:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9a8806ad50 msgr2=0x7f9a8806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:19.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9a8806ad50 0x7f9a8806d210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a900090b0 tx=0x7f9a90008030 comp rx=0 tx=0).stop 2023-10-16T11:22:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d1500 msgr2=0x7f9a9c149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d1500 0x7f9a9c149ee0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0011e0 tx=0x7f9a8c00cae0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 shutdown_connections 2023-10-16T11:22:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9a8806ad50 0x7f9a8806d210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9c15ea40 0x7f9a9c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9c0d01d0 0x7f9a9c14d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 --2- 172.21.15.181:0/2172009788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a9c0d1500 0x7f9a9c149ee0 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 >> 172.21.15.181:0/2172009788 conn(0x7f9a9c09cb60 msgr2=0x7f9a9c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 shutdown_connections 2023-10-16T11:22:19.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.836+0000 7f9aa1274700 1 -- 172.21.15.181:0/2172009788 wait complete. 2023-10-16T11:22:19.853 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T11:22:19.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.986+0000 7f1a7c7b6700 1 Processor -- start 2023-10-16T11:22:19.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.986+0000 7f1a7c7b6700 1 -- start start 2023-10-16T11:22:19.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7c7b6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a740ce610 0x7f1a7406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:19.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7c7b6700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1a740d0fc0 0x7f1a740cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:19.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7c7b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740d0060 0x7f1a7406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:19.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7c7b6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a74162190 con 0x7f1a740ce610 2023-10-16T11:22:19.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7c7b6700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a74162650 con 0x7f1a740d0fc0 2023-10-16T11:22:19.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7c7b6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74162370 con 0x7f1a740d0060 2023-10-16T11:22:19.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7a552700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a740ce610 0x7f1a7406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:19.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7a552700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a740ce610 0x7f1a7406d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34184/0 (socket says 172.21.15.181:34184) 2023-10-16T11:22:19.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.987+0000 7f1a7a552700 1 -- 172.21.15.181:0/2542115866 learned_addr learned my addr 172.21.15.181:0/2542115866 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:19.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a79d51700 1 --2- 172.21.15.181:0/2542115866 >> v2:172.21.15.190:3300/0 conn(0x7f1a740d0fc0 0x7f1a740cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:19.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7ad53700 1 --2- 172.21.15.181:0/2542115866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740d0060 0x7f1a7406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:19.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7a552700 1 -- 172.21.15.181:0/2542115866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740d0060 msgr2=0x7f1a7406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:19.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7a552700 1 --2- 172.21.15.181:0/2542115866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740d0060 0x7f1a7406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7a552700 1 -- 172.21.15.181:0/2542115866 >> v2:172.21.15.190:3300/0 conn(0x7f1a740d0fc0 msgr2=0x7f1a740cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:19.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7a552700 1 --2- 172.21.15.181:0/2542115866 >> v2:172.21.15.190:3300/0 conn(0x7f1a740d0fc0 0x7f1a740cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7a552700 1 -- 172.21.15.181:0/2542115866 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7406f0f0 con 0x7f1a740ce610 2023-10-16T11:22:19.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7ad53700 1 --2- 172.21.15.181:0/2542115866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740d0060 0x7f1a7406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:19.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7a552700 1 --2- 172.21.15.181:0/2542115866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740ce610 0x7f1a7406d660 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f1a740c7520 tx=0x7f1a6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c546c9053f57054a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:19.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a79550700 1 -- 172.21.15.181:0/2542115866 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00e070 con 0x7f1a740ce610 2023-10-16T11:22:19.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a79550700 1 -- 172.21.15.181:0/2542115866 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c00b9c0 con 0x7f1a740ce610 2023-10-16T11:22:19.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a79550700 1 -- 172.21.15.181:0/2542115866 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00bd70 con 0x7f1a740ce610 2023-10-16T11:22:19.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/2542115866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740ce610 msgr2=0x7f1a7406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:19.995 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.988+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/2542115866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740ce610 0x7f1a7406d660 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f1a740c7520 tx=0x7f1a6c00b330 comp rx=0 tx=0).stop 2023-10-16T11:22:19.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.989+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/2542115866 shutdown_connections 2023-10-16T11:22:19.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.989+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/2542115866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740ce610 0x7f1a7406d660 unknown :-1 s=CLOSED pgs=3231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.989+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/2542115866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740d0060 0x7f1a7406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.989+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/2542115866 >> v2:172.21.15.190:3300/0 conn(0x7f1a740d0fc0 0x7f1a740cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:19.996 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.989+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/2542115866 >> 172.21.15.181:0/2542115866 conn(0x7f1a740d04a0 msgr2=0x7f1a740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:19.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.989+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/2542115866 shutdown_connections 2023-10-16T11:22:19.997 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.989+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/2542115866 wait complete. 2023-10-16T11:22:20.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.990+0000 7f1a7c7b6700 1 Processor -- start 2023-10-16T11:22:20.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.990+0000 7f1a7c7b6700 1 -- start start 2023-10-16T11:22:20.000 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7c7b6700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740ce610 0x7f1a7415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7c7b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740d0fc0 0x7f1a741f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7c7b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a741f4af0 0x7f1a7409ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7c7b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7415bfc0 con 0x7f1a740d0fc0 2023-10-16T11:22:20.001 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7c7b6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7409d450 con 0x7f1a741f4af0 2023-10-16T11:22:20.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7c7b6700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7409d170 con 0x7f1a740ce610 2023-10-16T11:22:20.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a79d51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740d0fc0 0x7f1a741f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a79d51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740d0fc0 0x7f1a741f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34194/0 (socket says 172.21.15.181:34194) 2023-10-16T11:22:20.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a79d51700 1 -- 172.21.15.181:0/4109984630 learned_addr learned my addr 172.21.15.181:0/4109984630 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:20.002 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7a552700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740ce610 0x7f1a7415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.991+0000 7f1a7ad53700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a741f4af0 0x7f1a7409ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a79d51700 1 -- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740ce610 msgr2=0x7f1a7415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a79d51700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740ce610 0x7f1a7415ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a79d51700 1 -- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a741f4af0 msgr2=0x7f1a7409ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:20.003 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a79d51700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a741f4af0 0x7f1a7409ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:20.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a79d51700 1 -- 172.21.15.181:0/4109984630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c009b30 con 0x7f1a740d0fc0 2023-10-16T11:22:20.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a79d51700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740d0fc0 0x7f1a741f45b0 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d800 tx=0x7f1a6800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:20.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680126b0 con 0x7f1a740d0fc0 2023-10-16T11:22:20.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a68012810 con 0x7f1a740d0fc0 2023-10-16T11:22:20.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7409a320 con 0x7f1a740d0fc0 2023-10-16T11:22:20.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68012ba0 con 0x7f1a740d0fc0 2023-10-16T11:22:20.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.992+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7409a8e0 con 0x7f1a740d0fc0 2023-10-16T11:22:20.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.993+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7409ac50 con 0x7f1a740d0fc0 2023-10-16T11:22:20.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.993+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 176) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f1a68012d00 con 0x7f1a740d0fc0 2023-10-16T11:22:20.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.994+0000 7f1a72ffd700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f1a6006ad50 0x7f1a6006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:20.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.994+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 7035+0+0 (secure 0 0 0) 0x7f1a68089890 con 0x7f1a740d0fc0 2023-10-16T11:22:20.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.994+0000 7f1a7a552700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f1a6006ad50 0x7f1a6006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:20.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.994+0000 7f1a7a552700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f1a6006ad50 0x7f1a6006d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c000f80 tx=0x7f1a6c009000 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:20.006 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:19.998+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1a68054680 con 0x7f1a740d0fc0 2023-10-16T11:22:20.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:20.198+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1a7409a180 con 0x7f1a740d0fc0 2023-10-16T11:22:20.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:20.835+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 177) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1a68054a50 con 0x7f1a740d0fc0 2023-10-16T11:22:21.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.051+0000 7f1a72ffd700 1 -- 172.21.15.181:0/4109984630 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v273) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1a680736b0 con 0x7f1a740d0fc0 2023-10-16T11:22:21.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.056+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f1a6006ad50 msgr2=0x7f1a6006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.056+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f1a6006ad50 0x7f1a6006d210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c000f80 tx=0x7f1a6c009000 comp rx=0 tx=0).stop 2023-10-16T11:22:21.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.056+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740d0fc0 msgr2=0x7f1a741f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.056+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740d0fc0 0x7f1a741f45b0 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d800 tx=0x7f1a6800db10 comp rx=0 tx=0).stop 2023-10-16T11:22:21.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.057+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 shutdown_connections 2023-10-16T11:22:21.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.057+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f1a6006ad50 0x7f1a6006d210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.057+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a740ce610 0x7f1a7415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.057+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a741f4af0 0x7f1a7409ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.057+0000 7f1a7c7b6700 1 --2- 172.21.15.181:0/4109984630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a740d0fc0 0x7f1a741f45b0 secure :-1 s=CLOSED pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d800 tx=0x7f1a6800db10 comp rx=0 tx=0).stop 2023-10-16T11:22:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.057+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 >> 172.21.15.181:0/4109984630 conn(0x7f1a740d04a0 msgr2=0x7f1a740b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:21.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.058+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 shutdown_connections 2023-10-16T11:22:21.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.059+0000 7f1a7c7b6700 1 -- 172.21.15.181:0/4109984630 wait complete. 2023-10-16T11:22:21.062 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2023-10-16T11:22:21.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T11:22:21.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.205+0000 7f912ebdd700 1 Processor -- start 2023-10-16T11:22:21.207 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.206+0000 7f912ebdd700 1 -- start start 2023-10-16T11:22:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.206+0000 7f912ebdd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91280d0fc0 0x7f912806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.206+0000 7f912ebdd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.206+0000 7f912ebdd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9128162650 con 0x7f91280ce610 2023-10-16T11:22:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.206+0000 7f912ebdd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9128162190 con 0x7f91280d0fc0 2023-10-16T11:22:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.206+0000 7f912ebdd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280d0230 con 0x7f91280ce9f0 2023-10-16T11:22:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.206+0000 7f9127fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91280d0fc0 0x7f912806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f9127fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91280d0fc0 0x7f912806d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34200/0 (socket says 172.21.15.181:34200) 2023-10-16T11:22:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f9127fff700 1 -- 172.21.15.181:0/3242759916 learned_addr learned my addr 172.21.15.181:0/3242759916 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f9127fff700 1 -- 172.21.15.181:0/3242759916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 msgr2=0x7f912806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f912d17a700 1 --2- 172.21.15.181:0/3242759916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f9127fff700 1 --2- 172.21.15.181:0/3242759916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f9127fff700 1 -- 172.21.15.181:0/3242759916 >> v1:172.21.15.190:6789/0 conn(0x7f91280ce610 legacy=0x7f91280cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f9127fff700 1 -- 172.21.15.181:0/3242759916 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128162370 con 0x7f91280d0fc0 2023-10-16T11:22:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f9127fff700 1 --2- 172.21.15.181:0/3242759916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280d0fc0 0x7f912806d660 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7f91140098f0 tx=0x7f9114009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a149cd01637f95 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f91277fe700 1 -- 172.21.15.181:0/3242759916 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114004230 con 0x7f91280d0fc0 2023-10-16T11:22:21.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f91277fe700 1 -- 172.21.15.181:0/3242759916 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114004390 con 0x7f91280d0fc0 2023-10-16T11:22:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.207+0000 7f91277fe700 1 -- 172.21.15.181:0/3242759916 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140036a0 con 0x7f91280d0fc0 2023-10-16T11:22:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 -- 172.21.15.181:0/3242759916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280d0fc0 msgr2=0x7f912806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 --2- 172.21.15.181:0/3242759916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280d0fc0 0x7f912806d660 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7f91140098f0 tx=0x7f9114009c00 comp rx=0 tx=0).stop 2023-10-16T11:22:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 -- 172.21.15.181:0/3242759916 shutdown_connections 2023-10-16T11:22:21.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 --2- 172.21.15.181:0/3242759916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 --2- 172.21.15.181:0/3242759916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280d0fc0 0x7f912806d660 unknown :-1 s=CLOSED pgs=3233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 -- 172.21.15.181:0/3242759916 >> 172.21.15.181:0/3242759916 conn(0x7f91280d04a0 msgr2=0x7f91280d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 -- 172.21.15.181:0/3242759916 shutdown_connections 2023-10-16T11:22:21.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.208+0000 7f912ebdd700 1 -- 172.21.15.181:0/3242759916 wait complete. 2023-10-16T11:22:21.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.209+0000 7f912ebdd700 1 Processor -- start 2023-10-16T11:22:21.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.209+0000 7f912ebdd700 1 -- start start 2023-10-16T11:22:21.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912ebdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280ce610 0x7f912809ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912ebdd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912ebdd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91280d0fc0 0x7f91281572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912ebdd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815e8c0 con 0x7f91280ce610 2023-10-16T11:22:21.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912ebdd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912809e0f0 con 0x7f91280d0fc0 2023-10-16T11:22:21.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912ebdd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912815e240 con 0x7f91280ce9f0 2023-10-16T11:22:21.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280ce610 0x7f912809ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280ce610 0x7f912809ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34204/0 (socket says 172.21.15.181:34204) 2023-10-16T11:22:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 -- 172.21.15.181:0/1709910343 learned_addr learned my addr 172.21.15.181:0/1709910343 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912d17a700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91280d0fc0 0x7f91281572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f9127fff700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 -- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 msgr2=0x7f912809dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912809dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 -- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91280d0fc0 msgr2=0x7f91281572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91280d0fc0 0x7f91281572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.210+0000 7f912c979700 1 -- 172.21.15.181:0/1709910343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9114009550 con 0x7f91280ce610 2023-10-16T11:22:21.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.211+0000 7f912c979700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280ce610 0x7f912809ab90 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c4b0 tx=0x7f911c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.211+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c011790 con 0x7f91280ce610 2023-10-16T11:22:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.211+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0118f0 con 0x7f91280ce610 2023-10-16T11:22:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.211+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912809b290 con 0x7f91280ce610 2023-10-16T11:22:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.211+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c011ca0 con 0x7f91280ce610 2023-10-16T11:22:21.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.211+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912815e4e0 con 0x7f91280ce610 2023-10-16T11:22:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.212+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912815ea00 con 0x7f91280ce610 2023-10-16T11:22:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.212+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 177) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f911c01c460 con 0x7f91280ce610 2023-10-16T11:22:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.213+0000 7f91257fa700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 0x7f910c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.213+0000 7f9127fff700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 0x7f910c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:21.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.213+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(273..273 src has 1..273) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f911c089750 con 0x7f91280ce610 2023-10-16T11:22:21.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.213+0000 7f9127fff700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 0x7f910c06d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91280d13f0 tx=0x7f9114008030 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:21.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.217+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f911c054520 con 0x7f91280ce610 2023-10-16T11:22:21.412 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:21.412+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f91280b24f0 con 0x7f91280ce610 2023-10-16T11:22:22.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.374+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v274) v1 ==== 169+0+0 (secure 0 0 0) 0x7f911c057b40 con 0x7f91280ce610 2023-10-16T11:22:22.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.386+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 msgr2=0x7f910c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.386+0000 7f912ebdd700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 0x7f910c06d450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91280d13f0 tx=0x7f9114008030 comp rx=0 tx=0).stop 2023-10-16T11:22:22.387 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.386+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280ce610 msgr2=0x7f912809ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.387+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 <== mon.0 v2:172.21.15.181:3300/0 8 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f911c0548f0 con 0x7f91280ce610 2023-10-16T11:22:22.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.387+0000 7f91257fa700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 0x7f910c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.388 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.387+0000 7f912ebdd700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280ce610 0x7f912809ab90 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c4b0 tx=0x7f911c00c870 comp rx=0 tx=0).stop 2023-10-16T11:22:22.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 shutdown_connections 2023-10-16T11:22:22.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 0x7f910c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91280d0fc0 0x7f91281572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.389 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91280ce610 0x7f912809ab90 unknown :-1 s=CLOSED pgs=3234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91280ce9f0 0x7f912809dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 >> 172.21.15.181:0/1709910343 conn(0x7f91280d04a0 msgr2=0x7f912806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f91257fa700 1 -- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 msgr2=0x7f910c06d3e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-10-16T11:22:22.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f91257fa700 1 --2- 172.21.15.181:0/1709910343 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f910c06af90 0x7f910c06d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.390 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 shutdown_connections 2023-10-16T11:22:22.391 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.388+0000 7f912ebdd700 1 -- 172.21.15.181:0/1709910343 wait complete. 2023-10-16T11:22:22.391 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2023-10-16T11:22:22.406 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-16T11:22:22.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.530+0000 7f31dea2d700 1 Processor -- start 2023-10-16T11:22:22.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.530+0000 7f31dea2d700 1 -- start start 2023-10-16T11:22:22.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31dea2d700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31d815da70 0x7f31d80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31dea2d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80cef00 con 0x7f31d80ce6a0 2023-10-16T11:22:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31dea2d700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80cec20 con 0x7f31d806afb0 2023-10-16T11:22:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31dea2d700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d8071c80 con 0x7f31d815da70 2023-10-16T11:22:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f31d80ce6a0 0x7f31d806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45292/0 (socket says 172.21.15.181:45292) 2023-10-16T11:22:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d77fe700 1 -- 172.21.15.181:0/485977179 learned_addr learned my addr 172.21.15.181:0/485977179 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750377345 0 0) 0x7f31d80cef00 con 0x7f31d80ce6a0 2023-10-16T11:22:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31c00035b0 con 0x7f31d80ce6a0 2023-10-16T11:22:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d7fff700 1 --2- 172.21.15.181:0/485977179 >> v2:172.21.15.190:3300/0 conn(0x7f31d815da70 0x7f31d80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3065591230 0 0) 0x7f31c00035b0 con 0x7f31d80ce6a0 2023-10-16T11:22:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 >> v2:172.21.15.190:3300/0 conn(0x7f31d815da70 msgr2=0x7f31d80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.537 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 --2- 172.21.15.181:0/485977179 >> v2:172.21.15.190:3300/0 conn(0x7f31d815da70 0x7f31d80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 >> v1:172.21.15.190:6789/0 conn(0x7f31d806afb0 legacy=0x7f31d81615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8071f60 con 0x7f31d80ce6a0 2023-10-16T11:22:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f31cc001670 con 0x7f31d80ce6a0 2023-10-16T11:22:22.538 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 >> v1:172.21.15.181:6789/0 conn(0x7f31d80ce6a0 legacy=0x7f31d806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c0004850 0x7f31c0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c0005410 0x7f31c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8071c80 con 0x7f31c0003a00 2023-10-16T11:22:22.539 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c000c2a0 con 0x7f31c0005410 2023-10-16T11:22:22.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.531+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c000c460 con 0x7f31c0004850 2023-10-16T11:22:22.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d7fff700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31dcfca700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c0005410 0x7f31c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d77fe700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c0004850 0x7f31c0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.540 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d7fff700 1 -- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c0004850 msgr2=0x7f31c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d7fff700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31c0004850 0x7f31c0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d7fff700 1 -- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c0005410 msgr2=0x7f31c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d7fff700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c0005410 0x7f31c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d7fff700 1 -- 172.21.15.181:0/485977179 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00035b0 con 0x7f31c0003a00 2023-10-16T11:22:22.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31dcfca700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31c0005410 0x7f31c000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:22.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31dcfca700 1 -- 172.21.15.181:0/485977179 reap_dead start 2023-10-16T11:22:22.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d7fff700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31c0003e20 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009890 tx=0x7f31c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d222257fb7127d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c800c070 con 0x7f31c0003a00 2023-10-16T11:22:22.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c8013610 con 0x7f31c0003a00 2023-10-16T11:22:22.542 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.532+0000 7f31d6ffd700 1 -- 172.21.15.181:0/485977179 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80139a0 con 0x7f31c0003a00 2023-10-16T11:22:22.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.533+0000 7f31dea2d700 1 -- 172.21.15.181:0/485977179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 msgr2=0x7f31c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.533+0000 7f31dea2d700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31c0003e20 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009890 tx=0x7f31c0005880 comp rx=0 tx=0).stop 2023-10-16T11:22:22.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.533+0000 7f31dea2d700 1 -- 172.21.15.181:0/485977179 shutdown_connections 2023-10-16T11:22:22.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.533+0000 7f31dea2d700 1 --2- 172.21.15.181:0/485977179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31c0003e20 unknown :-1 s=CLOSED pgs=3236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.533+0000 7f31dea2d700 1 -- 172.21.15.181:0/485977179 >> 172.21.15.181:0/485977179 conn(0x7f31d80cd010 msgr2=0x7f31d80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.533+0000 7f31dea2d700 1 -- 172.21.15.181:0/485977179 shutdown_connections 2023-10-16T11:22:22.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.533+0000 7f31dea2d700 1 -- 172.21.15.181:0/485977179 wait complete. 2023-10-16T11:22:22.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.534+0000 7f31dea2d700 1 Processor -- start 2023-10-16T11:22:22.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.534+0000 7f31dea2d700 1 -- start start 2023-10-16T11:22:22.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dea2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31d8155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dea2d700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d81f6ea0 0x7f31d81f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.547 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dea2d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d80b0da0 0x7f31d80c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dea2d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8155d90 con 0x7f31c0003a00 2023-10-16T11:22:22.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dea2d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d81f7380 con 0x7f31d80b0da0 2023-10-16T11:22:22.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dea2d700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80c5d70 con 0x7f31d81f6ea0 2023-10-16T11:22:22.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31d8155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.548 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31d8155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34226/0 (socket says 172.21.15.181:34226) 2023-10-16T11:22:22.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 -- 172.21.15.181:0/2495640174 learned_addr learned my addr 172.21.15.181:0/2495640174 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:22.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d77fe700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d81f6ea0 0x7f31d81f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dcfca700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d80b0da0 0x7f31d80c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 -- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d81f6ea0 msgr2=0x7f31d81f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.549 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d81f6ea0 0x7f31d81f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 -- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d80b0da0 msgr2=0x7f31d80c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d80b0da0 0x7f31d80c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 -- 172.21.15.181:0/2495640174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d81f2fb0 con 0x7f31c0003a00 2023-10-16T11:22:22.550 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dcfca700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d80b0da0 0x7f31d80c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:22.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d77fe700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d81f6ea0 0x7f31d81f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:22.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d7fff700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31d8155850 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7f31c000c270 tx=0x7f31c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d4ff9700 1 -- 172.21.15.181:0/2495640174 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8013c50 con 0x7f31c0003a00 2023-10-16T11:22:22.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d4ff9700 1 -- 172.21.15.181:0/2495640174 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c8013db0 con 0x7f31c0003a00 2023-10-16T11:22:22.551 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31d4ff9700 1 -- 172.21.15.181:0/2495640174 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80038a0 con 0x7f31c0003a00 2023-10-16T11:22:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.535+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d81f3290 con 0x7f31c0003a00 2023-10-16T11:22:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.536+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d81f3cf0 con 0x7f31c0003a00 2023-10-16T11:22:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.536+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d806fe90 con 0x7f31c0003a00 2023-10-16T11:22:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.537+0000 7f31d4ff9700 1 -- 172.21.15.181:0/2495640174 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31c8003a00 con 0x7f31c0003a00 2023-10-16T11:22:22.552 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.537+0000 7f31d4ff9700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f31c406b020 0x7f31c406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.537+0000 7f31d4ff9700 1 -- 172.21.15.181:0/2495640174 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f31c808ade0 con 0x7f31c0003a00 2023-10-16T11:22:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.537+0000 7f31d77fe700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f31c406b020 0x7f31c406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.538+0000 7f31d77fe700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f31c406b020 0x7f31c406d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31cc001eb0 tx=0x7f31cc001d80 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.542+0000 7f31d4ff9700 1 -- 172.21.15.181:0/2495640174 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f31c8055b20 con 0x7f31c0003a00 2023-10-16T11:22:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.735+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f31d80b2ab0 con 0x7f31c0003a00 2023-10-16T11:22:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.739+0000 7f31d4ff9700 1 -- 172.21.15.181:0/2495640174 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v97) v1 ==== 115+0+0 (secure 0 0 0) 0x7f31c8003cb0 con 0x7f31c0003a00 2023-10-16T11:22:22.742 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f31c406b020 msgr2=0x7f31c406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f31c406b020 0x7f31c406d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31cc001eb0 tx=0x7f31cc001d80 comp rx=0 tx=0).stop 2023-10-16T11:22:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 msgr2=0x7f31d8155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31d8155850 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7f31c000c270 tx=0x7f31c80047c0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 shutdown_connections 2023-10-16T11:22:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f31c406b020 0x7f31c406d4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d80b0da0 0x7f31d80c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31c0003a00 0x7f31d8155850 unknown :-1 s=CLOSED pgs=3237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 --2- 172.21.15.181:0/2495640174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d81f6ea0 0x7f31d81f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 >> 172.21.15.181:0/2495640174 conn(0x7f31d80cd010 msgr2=0x7f31d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 shutdown_connections 2023-10-16T11:22:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.741+0000 7f31dea2d700 1 -- 172.21.15.181:0/2495640174 wait complete. 2023-10-16T11:22:22.755 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-16T11:22:22.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.876+0000 7f375ef15700 1 Processor -- start 2023-10-16T11:22:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.876+0000 7f375ef15700 1 -- start start 2023-10-16T11:22:22.881 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375ef15700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f37580ce6c0 0x7f37580cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375ef15700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 0x7f37580ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375ef15700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37580b1f70 con 0x7f375815f150 2023-10-16T11:22:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375ef15700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37580b2ae0 con 0x7f37580ce6c0 2023-10-16T11:22:22.882 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375ef15700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37580b2dc0 con 0x7f37580cc6a0 2023-10-16T11:22:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375ccb1700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f375815f150 0x7f37580cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45312/0 (socket says 172.21.15.181:45312) 2023-10-16T11:22:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375ccb1700 1 -- 172.21.15.181:0/4046286522 learned_addr learned my addr 172.21.15.181:0/4046286522 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f375d4b2700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 0x7f37580ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.877+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585443028 0 0) 0x7f37580b1f70 con 0x7f375815f150 2023-10-16T11:22:22.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 --2- 172.21.15.181:0/4046286522 >> v2:172.21.15.190:3300/0 conn(0x7f37580ce6c0 0x7f37580cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37480035b0 con 0x7f375815f150 2023-10-16T11:22:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3241607096 0 0) 0x7f37480035b0 con 0x7f375815f150 2023-10-16T11:22:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 msgr2=0x7f37580ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 0x7f37580ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 >> v2:172.21.15.190:3300/0 conn(0x7f37580ce6c0 msgr2=0x7f37580cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.884 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 --2- 172.21.15.181:0/4046286522 >> v2:172.21.15.190:3300/0 conn(0x7f37580ce6c0 0x7f37580cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37580cf9d0 con 0x7f375815f150 2023-10-16T11:22:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f374c001670 con 0x7f375815f150 2023-10-16T11:22:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 >> v1:172.21.15.181:6789/0 conn(0x7f375815f150 legacy=0x7f37580cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748003a00 0x7f3748003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.885 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3748004850 0x7f3748004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 0x7f3748005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580b2ae0 con 0x7f3748004850 2023-10-16T11:22:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37580b2dc0 con 0x7f3748003a00 2023-10-16T11:22:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374800ba10 con 0x7f37580cc6a0 2023-10-16T11:22:22.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3748004850 0x7f3748004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f375ccb1700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748003a00 0x7f3748003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f375d4b2700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 0x7f3748005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 -- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 msgr2=0x7f3748005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 0x7f3748005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 -- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748003a00 msgr2=0x7f3748003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:22.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748003a00 0x7f3748003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 -- 172.21.15.181:0/4046286522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480035b0 con 0x7f3748004850 2023-10-16T11:22:22.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f375d4b2700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37580cc6a0 0x7f3748005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:22.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f375ccb1700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748003a00 0x7f3748003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:22.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f3757fff700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3748004850 0x7f3748004cd0 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f3744009b60 tx=0x7f3748005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2856321bc79258 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.878+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400e6c0 con 0x7f3748004850 2023-10-16T11:22:22.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400e820 con 0x7f3748004850 2023-10-16T11:22:22.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f37577fe700 1 -- 172.21.15.181:0/4046286522 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400eb90 con 0x7f3748004850 2023-10-16T11:22:22.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f375ef15700 1 -- 172.21.15.181:0/4046286522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3748004850 msgr2=0x7f3748004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.889 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f375ef15700 1 --2- 172.21.15.181:0/4046286522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3748004850 0x7f3748004cd0 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f3744009b60 tx=0x7f3748005880 comp rx=0 tx=0).stop 2023-10-16T11:22:22.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f375d4b2700 1 -- 172.21.15.181:0/4046286522 reap_dead start 2023-10-16T11:22:22.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f375ef15700 1 -- 172.21.15.181:0/4046286522 shutdown_connections 2023-10-16T11:22:22.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f375ef15700 1 -- 172.21.15.181:0/4046286522 >> 172.21.15.181:0/4046286522 conn(0x7f375809a930 msgr2=0x7f37580d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:22.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f375ef15700 1 -- 172.21.15.181:0/4046286522 shutdown_connections 2023-10-16T11:22:22.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.879+0000 7f375ef15700 1 -- 172.21.15.181:0/4046286522 wait complete. 2023-10-16T11:22:22.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.880+0000 7f375ef15700 1 Processor -- start 2023-10-16T11:22:22.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.880+0000 7f375ef15700 1 -- start start 2023-10-16T11:22:22.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375ef15700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748004850 0x7f375815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375ef15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f375814f630 0x7f37580c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375ef15700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375815caa0 0x7f37580c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375ef15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375815dbf0 con 0x7f375814f630 2023-10-16T11:22:22.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375ef15700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37581f13f0 con 0x7f3748004850 2023-10-16T11:22:22.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375ef15700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375815cfe0 con 0x7f375815caa0 2023-10-16T11:22:22.894 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f3757fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f375814f630 0x7f37580c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f3757fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f375814f630 0x7f37580c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34248/0 (socket says 172.21.15.181:34248) 2023-10-16T11:22:22.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f3757fff700 1 -- 172.21.15.181:0/176021806 learned_addr learned my addr 172.21.15.181:0/176021806 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:22.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375ccb1700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748004850 0x7f375815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.881+0000 7f375d4b2700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375815caa0 0x7f37580c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.895 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f3757fff700 1 -- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375815caa0 msgr2=0x7f37580c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f3757fff700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375815caa0 0x7f37580c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f3757fff700 1 -- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748004850 msgr2=0x7f375815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:22.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f3757fff700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748004850 0x7f375815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:22.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f3757fff700 1 -- 172.21.15.181:0/176021806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375815d2c0 con 0x7f375814f630 2023-10-16T11:22:22.896 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f375d4b2700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375815caa0 0x7f37580c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:22.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f3757fff700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f375814f630 0x7f37580c8b00 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f37440090b0 tx=0x7f37440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f37557fa700 1 -- 172.21.15.181:0/176021806 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400e6c0 con 0x7f375814f630 2023-10-16T11:22:22.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f37557fa700 1 -- 172.21.15.181:0/176021806 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744003ad0 con 0x7f375814f630 2023-10-16T11:22:22.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375815d5a0 con 0x7f375814f630 2023-10-16T11:22:22.897 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f37557fa700 1 -- 172.21.15.181:0/176021806 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744003e80 con 0x7f375814f630 2023-10-16T11:22:22.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.882+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375815a900 con 0x7f375814f630 2023-10-16T11:22:22.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.883+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375815ded0 con 0x7f375814f630 2023-10-16T11:22:22.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.884+0000 7f37557fa700 1 -- 172.21.15.181:0/176021806 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3744013470 con 0x7f375814f630 2023-10-16T11:22:22.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.884+0000 7f37557fa700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f373c06b020 0x7f373c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:22.898 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.884+0000 7f37557fa700 1 -- 172.21.15.181:0/176021806 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f374408b300 con 0x7f375814f630 2023-10-16T11:22:22.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.884+0000 7f375ccb1700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f373c06b020 0x7f373c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:22.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.884+0000 7f375ccb1700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f373c06b020 0x7f373c06d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f374c001eb0 tx=0x7f374c007040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:22.899 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:22.888+0000 7f37557fa700 1 -- 172.21.15.181:0/176021806 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f3744056150 con 0x7f375814f630 2023-10-16T11:22:23.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.081+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f37580cfc80 con 0x7f375814f630 2023-10-16T11:22:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.086+0000 7f37557fa700 1 -- 172.21.15.181:0/176021806 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v98) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3744059770 con 0x7f375814f630 2023-10-16T11:22:23.090 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f373c06b020 msgr2=0x7f373c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f373c06b020 0x7f373c06d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f374c001eb0 tx=0x7f374c007040 comp rx=0 tx=0).stop 2023-10-16T11:22:23.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f375814f630 msgr2=0x7f37580c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f375814f630 0x7f37580c8b00 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f37440090b0 tx=0x7f37440047c0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 shutdown_connections 2023-10-16T11:22:23.091 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f373c06b020 0x7f373c06d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375815caa0 0x7f37580c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3748004850 0x7f375815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 --2- 172.21.15.181:0/176021806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f375814f630 0x7f37580c8b00 unknown :-1 s=CLOSED pgs=3240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 >> 172.21.15.181:0/176021806 conn(0x7f375809a930 msgr2=0x7f375806f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:23.092 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 shutdown_connections 2023-10-16T11:22:23.093 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.089+0000 7f375ef15700 1 -- 172.21.15.181:0/176021806 wait complete. 2023-10-16T11:22:23.103 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-16T11:22:23.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.226+0000 7fe08f615700 1 Processor -- start 2023-10-16T11:22:23.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.226+0000 7fe08f615700 1 -- start start 2023-10-16T11:22:23.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08f615700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe08815f150 0x7fe0880cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08f615700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0880cd170 0x7fe0880b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08f615700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0880b2ae0 con 0x7fe0880ce4f0 2023-10-16T11:22:23.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08f615700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0880ce9a0 con 0x7fe0880cd170 2023-10-16T11:22:23.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08f615700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0880b2dc0 con 0x7fe08815f150 2023-10-16T11:22:23.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08dbb2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0880cd170 0x7fe0880b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08cbb0700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe0880ce4f0 0x7fe0880b0f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45332/0 (socket says 172.21.15.181:45332) 2023-10-16T11:22:23.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08cbb0700 1 -- 172.21.15.181:0/265820321 learned_addr learned my addr 172.21.15.181:0/265820321 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:23.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08d3b1700 1 --2- 172.21.15.181:0/265820321 >> v2:172.21.15.190:3300/0 conn(0x7fe08815f150 0x7fe0880cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08dbb2700 1 -- 172.21.15.181:0/265820321 >> v2:172.21.15.190:3300/0 conn(0x7fe08815f150 msgr2=0x7fe0880cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:23.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08dbb2700 1 --2- 172.21.15.181:0/265820321 >> v2:172.21.15.190:3300/0 conn(0x7fe08815f150 0x7fe0880cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08dbb2700 1 -- 172.21.15.181:0/265820321 >> v1:172.21.15.181:6789/0 conn(0x7fe0880ce4f0 legacy=0x7fe0880b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe07ffff700 1 -- 172.21.15.181:0/265820321 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673544769 0 0) 0x7fe0880b2ae0 con 0x7fe0880ce4f0 2023-10-16T11:22:23.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08dbb2700 1 -- 172.21.15.181:0/265820321 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0880cf9d0 con 0x7fe0880cd170 2023-10-16T11:22:23.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.227+0000 7fe08d3b1700 1 --2- 172.21.15.181:0/265820321 >> v2:172.21.15.190:3300/0 conn(0x7fe08815f150 0x7fe0880cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:23.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08dbb2700 1 --2- 172.21.15.181:0/265820321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880cd170 0x7fe0880b1670 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 crypto rx=0x7fe080009d90 tx=0x7fe08000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d072b4aa739edf17 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe07ffff700 1 -- 172.21.15.181:0/265820321 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08000e070 con 0x7fe0880cd170 2023-10-16T11:22:23.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe07ffff700 1 -- 172.21.15.181:0/265820321 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08000bb90 con 0x7fe0880cd170 2023-10-16T11:22:23.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe07ffff700 1 -- 172.21.15.181:0/265820321 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080012430 con 0x7fe0880cd170 2023-10-16T11:22:23.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 -- 172.21.15.181:0/265820321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880cd170 msgr2=0x7fe0880b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 --2- 172.21.15.181:0/265820321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880cd170 0x7fe0880b1670 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 crypto rx=0x7fe080009d90 tx=0x7fe08000b330 comp rx=0 tx=0).stop 2023-10-16T11:22:23.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 -- 172.21.15.181:0/265820321 shutdown_connections 2023-10-16T11:22:23.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 --2- 172.21.15.181:0/265820321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880cd170 0x7fe0880b1670 unknown :-1 s=CLOSED pgs=3242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 --2- 172.21.15.181:0/265820321 >> v2:172.21.15.190:3300/0 conn(0x7fe08815f150 0x7fe0880cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 -- 172.21.15.181:0/265820321 >> 172.21.15.181:0/265820321 conn(0x7fe08809a930 msgr2=0x7fe0880d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:23.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 -- 172.21.15.181:0/265820321 shutdown_connections 2023-10-16T11:22:23.235 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.228+0000 7fe08f615700 1 -- 172.21.15.181:0/265820321 wait complete. 2023-10-16T11:22:23.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 Processor -- start 2023-10-16T11:22:23.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 -- start start 2023-10-16T11:22:23.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0880cd170 0x7fe0880c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.238 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880ce4f0 0x7fe0881e5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08815f150 0x7fe0881e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880cb430 con 0x7fe0880ce4f0 2023-10-16T11:22:23.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880cafa0 con 0x7fe08815f150 2023-10-16T11:22:23.239 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08f615700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08815d180 con 0x7fe0880cd170 2023-10-16T11:22:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08cbb0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880ce4f0 0x7fe0881e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08cbb0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880ce4f0 0x7fe0881e5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34262/0 (socket says 172.21.15.181:34262) 2023-10-16T11:22:23.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.230+0000 7fe08cbb0700 1 -- 172.21.15.181:0/1110604558 learned_addr learned my addr 172.21.15.181:0/1110604558 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08d3b1700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0880cd170 0x7fe0880c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08dbb2700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08815f150 0x7fe0881e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08cbb0700 1 -- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0880cd170 msgr2=0x7fe0880c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08cbb0700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0880cd170 0x7fe0880c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.242 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08cbb0700 1 -- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08815f150 msgr2=0x7fe0881e57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08cbb0700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08815f150 0x7fe0881e57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08cbb0700 1 -- 172.21.15.181:0/1110604558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe080009a70 con 0x7fe0880ce4f0 2023-10-16T11:22:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08dbb2700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08815f150 0x7fe0881e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08cbb0700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880ce4f0 0x7fe0881e5290 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 crypto rx=0x7fe0840011e0 tx=0x7fe08400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe07dffb700 1 -- 172.21.15.181:0/1110604558 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084004c80 con 0x7fe0880ce4f0 2023-10-16T11:22:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe07dffb700 1 -- 172.21.15.181:0/1110604558 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe084004e00 con 0x7fe0880ce4f0 2023-10-16T11:22:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe07dffb700 1 -- 172.21.15.181:0/1110604558 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084007310 con 0x7fe0880ce4f0 2023-10-16T11:22:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0880ca390 con 0x7fe0880ce4f0 2023-10-16T11:22:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.231+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0880ca950 con 0x7fe0880ce4f0 2023-10-16T11:22:23.244 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.232+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0880cb130 con 0x7fe0880ce4f0 2023-10-16T11:22:23.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.233+0000 7fe07dffb700 1 -- 172.21.15.181:0/1110604558 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe084007470 con 0x7fe0880ce4f0 2023-10-16T11:22:23.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.233+0000 7fe07dffb700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe06c06b020 0x7fe06c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.233+0000 7fe07dffb700 1 -- 172.21.15.181:0/1110604558 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fe084089770 con 0x7fe0880ce4f0 2023-10-16T11:22:23.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.233+0000 7fe08d3b1700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe06c06b020 0x7fe06c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.245 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.233+0000 7fe08d3b1700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe06c06b020 0x7fe06c06d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe078006fd0 tx=0x7fe078008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.246 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.238+0000 7fe07dffb700 1 -- 172.21.15.181:0/1110604558 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe084054530 con 0x7fe0880ce4f0 2023-10-16T11:22:23.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.430+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe088067d20 con 0x7fe0880ce4f0 2023-10-16T11:22:23.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.435+0000 7fe07dffb700 1 -- 172.21.15.181:0/1110604558 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v99) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe084057b50 con 0x7fe0880ce4f0 2023-10-16T11:22:23.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe06c06b020 msgr2=0x7fe06c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe06c06b020 0x7fe06c06d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe078006fd0 tx=0x7fe078008040 comp rx=0 tx=0).stop 2023-10-16T11:22:23.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880ce4f0 msgr2=0x7fe0881e5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880ce4f0 0x7fe0881e5290 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 crypto rx=0x7fe0840011e0 tx=0x7fe08400cae0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 shutdown_connections 2023-10-16T11:22:23.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe06c06b020 0x7fe06c06d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0880cd170 0x7fe0880c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe08815f150 0x7fe0881e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 --2- 172.21.15.181:0/1110604558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0880ce4f0 0x7fe0881e5290 unknown :-1 s=CLOSED pgs=3243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 >> 172.21.15.181:0/1110604558 conn(0x7fe08809a930 msgr2=0x7fe08815a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:23.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 shutdown_connections 2023-10-16T11:22:23.441 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.438+0000 7fe08f615700 1 -- 172.21.15.181:0/1110604558 wait complete. 2023-10-16T11:22:23.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments' 2023-10-16T11:22:23.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.571+0000 7f64b44e7700 1 Processor -- start 2023-10-16T11:22:23.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.571+0000 7f64b44e7700 1 -- start start 2023-10-16T11:22:23.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.572+0000 7f64b44e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64ac15eab0 0x7f64ac0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.572+0000 7f64b44e7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64ac0cb0c0 0x7f64ac15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.572+0000 7f64b44e7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac06ed00 con 0x7f64ac15e6d0 2023-10-16T11:22:23.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.572+0000 7f64b44e7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0d02d0 con 0x7f64ac15eab0 2023-10-16T11:22:23.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.572+0000 7f64b44e7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac15ad40 con 0x7f64ac0cb0c0 2023-10-16T11:22:23.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.572+0000 7f64b1a82700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64ac15eab0 0x7f64ac0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.572+0000 7f64b1a82700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64ac15eab0 0x7f64ac0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34270/0 (socket says 172.21.15.181:34270) 2023-10-16T11:22:23.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1a82700 1 -- 172.21.15.181:0/3117256065 learned_addr learned my addr 172.21.15.181:0/3117256065 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:23.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1a82700 1 -- 172.21.15.181:0/3117256065 >> v2:172.21.15.190:3300/0 conn(0x7f64ac0cb0c0 msgr2=0x7f64ac15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:23.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1a82700 1 --2- 172.21.15.181:0/3117256065 >> v2:172.21.15.190:3300/0 conn(0x7f64ac0cb0c0 0x7f64ac15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1a82700 1 -- 172.21.15.181:0/3117256065 >> v1:172.21.15.190:6789/0 conn(0x7f64ac15e6d0 legacy=0x7f64ac15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:23.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1a82700 1 -- 172.21.15.181:0/3117256065 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac06e0f0 con 0x7f64ac15eab0 2023-10-16T11:22:23.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1a82700 1 --2- 172.21.15.181:0/3117256065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15eab0 0x7f64ac0d0fc0 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7f64ac071650 tx=0x7f64a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420c643f6ee2818e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1281700 1 -- 172.21.15.181:0/3117256065 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8004230 con 0x7f64ac15eab0 2023-10-16T11:22:23.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1281700 1 -- 172.21.15.181:0/3117256065 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a8004390 con 0x7f64ac15eab0 2023-10-16T11:22:23.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.573+0000 7f64b1281700 1 -- 172.21.15.181:0/3117256065 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80036a0 con 0x7f64ac15eab0 2023-10-16T11:22:23.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 -- 172.21.15.181:0/3117256065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15eab0 msgr2=0x7f64ac0d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 --2- 172.21.15.181:0/3117256065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15eab0 0x7f64ac0d0fc0 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7f64ac071650 tx=0x7f64a8009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 -- 172.21.15.181:0/3117256065 shutdown_connections 2023-10-16T11:22:23.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 --2- 172.21.15.181:0/3117256065 >> v2:172.21.15.190:3300/0 conn(0x7f64ac0cb0c0 0x7f64ac15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 --2- 172.21.15.181:0/3117256065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15eab0 0x7f64ac0d0fc0 unknown :-1 s=CLOSED pgs=3244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 -- 172.21.15.181:0/3117256065 >> 172.21.15.181:0/3117256065 conn(0x7f64ac0d04a0 msgr2=0x7f64ac0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:23.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 -- 172.21.15.181:0/3117256065 shutdown_connections 2023-10-16T11:22:23.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.574+0000 7f64b44e7700 1 -- 172.21.15.181:0/3117256065 wait complete. 2023-10-16T11:22:23.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.575+0000 7f64b44e7700 1 Processor -- start 2023-10-16T11:22:23.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.575+0000 7f64b44e7700 1 -- start start 2023-10-16T11:22:23.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b44e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac0cb0c0 0x7f64ac0ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b44e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15e6d0 0x7f64ac1f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b44e7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15eab0 0x7f64ac1f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b44e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0aeed0 con 0x7f64ac15e6d0 2023-10-16T11:22:23.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b44e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0ae2c0 con 0x7f64ac0cb0c0 2023-10-16T11:22:23.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b44e7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0ae5a0 con 0x7f64ac15eab0 2023-10-16T11:22:23.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15e6d0 0x7f64ac1f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15e6d0 0x7f64ac1f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34276/0 (socket says 172.21.15.181:34276) 2023-10-16T11:22:23.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 -- 172.21.15.181:0/1424152234 learned_addr learned my addr 172.21.15.181:0/1424152234 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:23.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 -- 172.21.15.181:0/1424152234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15eab0 msgr2=0x7f64ac1f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15eab0 0x7f64ac1f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 -- 172.21.15.181:0/1424152234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac0cb0c0 msgr2=0x7f64ac0ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:23.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac0cb0c0 0x7f64ac0ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:23.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 -- 172.21.15.181:0/1424152234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009550 con 0x7f64ac15e6d0 2023-10-16T11:22:23.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64b1a82700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15e6d0 0x7f64ac1f3680 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009130 tx=0x7f64a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.576+0000 7f64a2ffd700 1 -- 172.21.15.181:0/1424152234 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8003dc0 con 0x7f64ac15e6d0 2023-10-16T11:22:23.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.577+0000 7f64a2ffd700 1 -- 172.21.15.181:0/1424152234 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a801f070 con 0x7f64ac15e6d0 2023-10-16T11:22:23.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.577+0000 7f64a2ffd700 1 -- 172.21.15.181:0/1424152234 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8011870 con 0x7f64ac15e6d0 2023-10-16T11:22:23.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.577+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac0ae880 con 0x7f64ac15e6d0 2023-10-16T11:22:23.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.577+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac0b0990 con 0x7f64ac15e6d0 2023-10-16T11:22:23.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.577+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac067d20 con 0x7f64ac15e6d0 2023-10-16T11:22:23.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.578+0000 7f64a2ffd700 1 -- 172.21.15.181:0/1424152234 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64a8021030 con 0x7f64ac15e6d0 2023-10-16T11:22:23.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.578+0000 7f64a2ffd700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f649006b020 0x7f649006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:23.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.578+0000 7f64a2ffd700 1 -- 172.21.15.181:0/1424152234 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f64a808b0a0 con 0x7f64ac15e6d0 2023-10-16T11:22:23.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.578+0000 7f64b2283700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f649006b020 0x7f649006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:23.588 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.579+0000 7f64b2283700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f649006b020 0x7f649006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f64ac06d890 tx=0x7f649c006c20 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:23.589 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.583+0000 7f64a2ffd700 1 -- 172.21.15.181:0/1424152234 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f64a8056010 con 0x7f64ac15e6d0 2023-10-16T11:22:23.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:23.856+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]} v 0) v1 -- 0x7f64ac0cb610 con 0x7f64ac15e6d0 2023-10-16T11:22:24.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.379+0000 7f64a2ffd700 1 -- 172.21.15.181:0/1424152234 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments.test_snap_schedule_subvol_and_group_arguments"]}]=0 v3264) v1 ==== 198+0+0 (secure 0 0 0) 0x7f64a8059630 con 0x7f64ac15e6d0 2023-10-16T11:22:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f649006b020 msgr2=0x7f649006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f649006b020 0x7f649006d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f64ac06d890 tx=0x7f649c006c20 comp rx=0 tx=0).stop 2023-10-16T11:22:24.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15e6d0 msgr2=0x7f64ac1f3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:24.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15e6d0 0x7f64ac1f3680 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009130 tx=0x7f64a8004750 comp rx=0 tx=0).stop 2023-10-16T11:22:24.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 shutdown_connections 2023-10-16T11:22:24.384 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f649006b020 0x7f649006d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15eab0 0x7f64ac1f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac0cb0c0 0x7f64ac0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.385 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 --2- 172.21.15.181:0/1424152234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64ac15e6d0 0x7f64ac1f3680 unknown :-1 s=CLOSED pgs=3245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:24.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.381+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 >> 172.21.15.181:0/1424152234 conn(0x7f64ac0d04a0 msgr2=0x7f64ac157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:24.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.382+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 shutdown_connections 2023-10-16T11:22:24.386 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:24.382+0000 7f64b44e7700 1 -- 172.21.15.181:0/1424152234 wait complete. 2023-10-16T11:22:24.398 INFO:tasks.cephfs_test_runner:Test subvol and group arguments to snap-schedule cli. ... ERROR 2023-10-16T11:22:24.399 INFO:tasks.cephfs_test_runner: 2023-10-16T11:22:24.399 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T11:22:24.400 INFO:tasks.cephfs_test_runner:ERROR: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:22:24.400 INFO:tasks.cephfs_test_runner:Test subvol and group arguments to snap-schedule cli. 2023-10-16T11:22:24.400 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T11:22:24.401 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T11:22:24.401 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 585, in test_snap_schedule_subvol_and_group_arguments 2023-10-16T11:22:24.402 INFO:tasks.cephfs_test_runner: self._fs_cmd('subvolume', 'create', 'a', 'sv1') 2023-10-16T11:22:24.402 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 39, in _fs_cmd 2023-10-16T11:22:24.402 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2023-10-16T11:22:24.403 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-10-16T11:22:24.403 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-10-16T11:22:24.404 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-10-16T11:22:24.404 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-10-16T11:22:24.404 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd 2023-10-16T11:22:24.405 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-16T11:22:24.405 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run 2023-10-16T11:22:24.405 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-16T11:22:24.406 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run 2023-10-16T11:22:24.406 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-16T11:22:24.407 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T11:22:24.407 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T11:22:24.407 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T11:22:24.408 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T11:22:24.408 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi181 with status 2: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create a sv1' 2023-10-16T11:22:24.409 INFO:tasks.cephfs_test_runner: 2023-10-16T11:22:24.409 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T11:22:24.410 INFO:tasks.cephfs_test_runner:Ran 15 tests in 5873.968s 2023-10-16T11:22:24.410 INFO:tasks.cephfs_test_runner: 2023-10-16T11:22:24.410 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-16T11:22:24.411 INFO:tasks.cephfs_test_runner: 2023-10-16T11:22:24.411 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-16T11:22:24.412 INFO:tasks.cephfs_test_runner:ERROR: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:22:24.412 INFO:tasks.cephfs_test_runner:Test subvol and group arguments to snap-schedule cli. 2023-10-16T11:22:24.413 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-16T11:22:24.413 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-16T11:22:24.413 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 585, in test_snap_schedule_subvol_and_group_arguments 2023-10-16T11:22:24.414 INFO:tasks.cephfs_test_runner: self._fs_cmd('subvolume', 'create', 'a', 'sv1') 2023-10-16T11:22:24.414 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs/test_snap_schedules.py", line 39, in _fs_cmd 2023-10-16T11:22:24.415 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2023-10-16T11:22:24.415 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2023-10-16T11:22:24.415 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2023-10-16T11:22:24.416 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2023-10-16T11:22:24.416 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2023-10-16T11:22:24.417 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd 2023-10-16T11:22:24.417 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-16T11:22:24.418 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run 2023-10-16T11:22:24.418 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-16T11:22:24.418 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run 2023-10-16T11:22:24.419 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-16T11:22:24.419 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait 2023-10-16T11:22:24.420 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-16T11:22:24.420 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-16T11:22:24.420 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-16T11:22:24.421 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi181 with status 2: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create a sv1' 2023-10-16T11:22:24.421 INFO:tasks.cephfs_test_runner: 2023-10-16T11:22:24.422 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:22:24.494 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7e1be5092f8641caa7cee2892beef95b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_7b3d12460841f3cbe9400ba1b68e3e20957ff531/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) 2023-10-16T11:22:24.496 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-16T11:22:24.507 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-16T11:22:24.514 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-16T11:22:24.601 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:22:24.602 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:24.632 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:22:24.633 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-16T11:22:24.633 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:24.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=393 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-10-16T11:22:24.660 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:22:24.660 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:24.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:22:24.686 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-10-16T11:22:24.686 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T11:22:24.747 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284284k,nr_inodes=4071071,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18570 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-10-16T11:22:24.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:24.748 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:22:24.770 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T11:22:24.770 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T11:22:24.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:24.772 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T11:22:24.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:24.857 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T11:22:24.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:24.953 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-16T11:22:25.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.032 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-16T11:22:25.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.051 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-16T11:22:25.122 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-16T11:22:25.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:22:25.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.125 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-10-16T11:22:25.147 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-16T11:22:25.148 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T11:22:25.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.150 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T11:22:25.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.233 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T11:22:25.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.316 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-10-16T11:22:25.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.405 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-10-16T11:22:25.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T11:22:25.424 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-10-16T11:22:25.493 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-10-16T11:22:25.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:22:25.495 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-16T11:22:25.510 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T11:22:25.511 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:25.633 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.632+0000 7fd313ed3700 1 Processor -- start 2023-10-16T11:22:25.634 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.632+0000 7fd313ed3700 1 -- start start 2023-10-16T11:22:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.633+0000 7fd313ed3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd30c0cebb0 0x7fd30c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.633+0000 7fd313ed3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c0cb680 0x7fd30c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.633+0000 7fd313ed3700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0cf060 con 0x7fd30c15a7b0 2023-10-16T11:22:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.633+0000 7fd313ed3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c0cf2b0 con 0x7fd30c0cebb0 2023-10-16T11:22:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.633+0000 7fd313ed3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0b5300 con 0x7fd30c0cb680 2023-10-16T11:22:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd31146e700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd30c0cebb0 0x7fd30c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd312470700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c0cb680 0x7fd30c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd311c6f700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd30c15a7b0 0x7fd30c0c9a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:57534/0 (socket says 172.21.15.181:57534) 2023-10-16T11:22:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd311c6f700 1 -- 172.21.15.181:0/18192958 learned_addr learned my addr 172.21.15.181:0/18192958 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd310c6d700 1 -- 172.21.15.181:0/18192958 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257314334 0 0) 0x7fd30c0cf060 con 0x7fd30c15a7b0 2023-10-16T11:22:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd310c6d700 1 -- 172.21.15.181:0/18192958 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f80035b0 con 0x7fd30c15a7b0 2023-10-16T11:22:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd31146e700 1 -- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c0cb680 msgr2=0x7fd30c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd31146e700 1 --2- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c0cb680 0x7fd30c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd31146e700 1 -- 172.21.15.181:0/18192958 >> v1:172.21.15.190:6789/0 conn(0x7fd30c15a7b0 legacy=0x7fd30c0c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd31146e700 1 -- 172.21.15.181:0/18192958 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c15e390 con 0x7fd30c0cebb0 2023-10-16T11:22:25.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd312470700 1 --2- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c0cb680 0x7fd30c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:25.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd31146e700 1 --2- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0ca160 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd308009840 tx=0x7fd308009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836b7217787353a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd310c6d700 1 -- 172.21.15.181:0/18192958 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308011830 con 0x7fd30c0cebb0 2023-10-16T11:22:25.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd310c6d700 1 -- 172.21.15.181:0/18192958 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd308011990 con 0x7fd30c0cebb0 2023-10-16T11:22:25.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.634+0000 7fd310c6d700 1 -- 172.21.15.181:0/18192958 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308011d20 con 0x7fd30c0cebb0 2023-10-16T11:22:25.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 -- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cebb0 msgr2=0x7fd30c0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.642 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 --2- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0ca160 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd308009840 tx=0x7fd308009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:25.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 -- 172.21.15.181:0/18192958 shutdown_connections 2023-10-16T11:22:25.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 --2- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c0cb680 0x7fd30c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 --2- 172.21.15.181:0/18192958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0ca160 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 -- 172.21.15.181:0/18192958 >> 172.21.15.181:0/18192958 conn(0x7fd30c06aed0 msgr2=0x7fd30c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 -- 172.21.15.181:0/18192958 shutdown_connections 2023-10-16T11:22:25.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.635+0000 7fd313ed3700 1 -- 172.21.15.181:0/18192958 wait complete. 2023-10-16T11:22:25.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.636+0000 7fd313ed3700 1 Processor -- start 2023-10-16T11:22:25.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.636+0000 7fd313ed3700 1 -- start start 2023-10-16T11:22:25.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd313ed3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cb680 0x7fd30c0b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd313ed3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd313ed3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c15a7b0 0x7fd30c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd313ed3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0b3580 con 0x7fd30c0cebb0 2023-10-16T11:22:25.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd313ed3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0b32a0 con 0x7fd30c0cb680 2023-10-16T11:22:25.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd313ed3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0c3a50 con 0x7fd30c15a7b0 2023-10-16T11:22:25.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34290/0 (socket says 172.21.15.181:34290) 2023-10-16T11:22:25.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 -- 172.21.15.181:0/2698249009 learned_addr learned my addr 172.21.15.181:0/2698249009 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:25.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd311c6f700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cb680 0x7fd30c0b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd312470700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c15a7b0 0x7fd30c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 -- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c15a7b0 msgr2=0x7fd30c153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.651 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c15a7b0 0x7fd30c153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 -- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cb680 msgr2=0x7fd30c0b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cb680 0x7fd30c0b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd31146e700 1 -- 172.21.15.181:0/2698249009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304009a70 con 0x7fd30c0cebb0 2023-10-16T11:22:25.652 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.637+0000 7fd311c6f700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cb680 0x7fd30c0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:25.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.638+0000 7fd31146e700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0c4530 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7fd3080090b0 tx=0x7fd308004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.638+0000 7fd3027fc700 1 -- 172.21.15.181:0/2698249009 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308004550 con 0x7fd30c0cebb0 2023-10-16T11:22:25.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.638+0000 7fd3027fc700 1 -- 172.21.15.181:0/2698249009 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30801f070 con 0x7fd30c0cebb0 2023-10-16T11:22:25.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.638+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3080094a0 con 0x7fd30c0cebb0 2023-10-16T11:22:25.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.638+0000 7fd3027fc700 1 -- 172.21.15.181:0/2698249009 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3080038e0 con 0x7fd30c0cebb0 2023-10-16T11:22:25.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.638+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0c4010 con 0x7fd30c0cebb0 2023-10-16T11:22:25.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.639+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c04ebb0 con 0x7fd30c0cebb0 2023-10-16T11:22:25.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.639+0000 7fd3027fc700 1 -- 172.21.15.181:0/2698249009 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd308003a40 con 0x7fd30c0cebb0 2023-10-16T11:22:25.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.640+0000 7fd3027fc700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd2f006b020 0x7fd2f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.655 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.640+0000 7fd3027fc700 1 -- 172.21.15.181:0/2698249009 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fd308021070 con 0x7fd30c0cebb0 2023-10-16T11:22:25.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.640+0000 7fd311c6f700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd2f006b020 0x7fd2f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.640+0000 7fd311c6f700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd2f006b020 0x7fd2f006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007fd0 tx=0x7fd2fc007dc0 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.644+0000 7fd3027fc700 1 -- 172.21.15.181:0/2698249009 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd308055e90 con 0x7fd30c0cebb0 2023-10-16T11:22:25.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.831+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd30c15e950 con 0x7fd2f006b020 2023-10-16T11:22:25.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.833+0000 7fd3027fc700 1 -- 172.21.15.181:0/2698249009 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50928 (secure 0 0 0) 0x7fd2fc007120 con 0x7fd2f006b020 2023-10-16T11:22:25.835 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.835+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd2f006b020 msgr2=0x7fd2f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.835+0000 7fd313ed3700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd2f006b020 0x7fd2f006d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007fd0 tx=0x7fd2fc007dc0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.837 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.835+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd30c0cebb0 msgr2=0x7fd30c0c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.835+0000 7fd313ed3700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0c4530 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7fd3080090b0 tx=0x7fd308004750 comp rx=0 tx=0).stop 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 shutdown_connections 2023-10-16T11:22:25.838 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd2f006b020 0x7fd2f006d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd30c15a7b0 0x7fd30c153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30c0cb680 0x7fd30c0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 --2- 172.21.15.181:0/2698249009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd30c0cebb0 0x7fd30c0c4530 unknown :-1 s=CLOSED pgs=3246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 >> 172.21.15.181:0/2698249009 conn(0x7fd30c06aed0 msgr2=0x7fd30c154550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 shutdown_connections 2023-10-16T11:22:25.840 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.836+0000 7fd313ed3700 1 -- 172.21.15.181:0/2698249009 wait complete. 2023-10-16T11:22:25.841 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:22:25.849 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-16T11:22:23.837595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":679,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":403,"ondisk_log_size":403,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":501432,"kb_used_data":28556,"kb_used_omap":12,"kb_used_meta":472755,"kb_avail":754473288,"statfs":{"total":773094113280,"available":772580646912,"internally_reserved":0,"allocated":29241344,"data_stored":15950029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":484101688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-530476,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-86,"num_read_kb":-73,"num_write":-163,"num_write_kb":-1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.999734"},"pg_stats":[{"pgid":"1.0","version":"259'403","reported_seq":1457,"reported_epoch":270,"state":"active+clean","last_fresh":"2023-10-16T11:22:10.388802+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:22:10.388802+0000","last_peered":"2023-10-16T11:22:10.388802+0000","last_clean":"2023-10-16T11:22:10.388802+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:22:10.388802+0000","last_undegraded":"2023-10-16T11:22:10.388802+0000","last_fullsized":"2023-10-16T11:22:10.388802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":679,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2612,"num_read_kb":5456,"num_write":679,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":403,"ondisk_log_size":403,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543308,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57696,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314144,"statfs":{"total":96636764160,"available":96577683456,"internally_reserved":0,"allocated":4411392,"data_stored":2765488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.878,"15min":0.878},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27800000000000002},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39500000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.50900000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55100000000000005,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.31900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.497},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69799999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":1,"15min":1.002},"last":0.42899999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68400000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.314},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60399999999999998,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.35599999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84299999999999997,"5min":1.052,"15min":1.052},"last":0.47399999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.439},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41399999999999998}]}]},{"osd":6,"up_from":14,"seq":60129543308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60940,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310900,"statfs":{"total":96636764160,"available":96574361600,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.60099999999999998},"min":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54000000000000004,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.59599999999999997},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.47199999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.745,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66700000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.29699999999999999},"max":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.98999999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.90700000000000003},"last":0.754}]},{"osd":5,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69699999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129543308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78984,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292856,"statfs":{"total":96636764160,"available":96555884544,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.39700000000000002},"min":{"1min":0.23499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.377,"15min":0.41799999999999998},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.44400000000000001,"5min":0.62,"15min":0.66900000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45200000000000001,"5min":0.56999999999999995,"15min":0.61799999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.379,"5min":0.40799999999999997,"15min":0.44900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.497,"5min":0.69699999999999995,"15min":0.755},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67300000000000004,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.36199999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":0.92600000000000005,"15min":1.097},"last":0.52200000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76600000000000001,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":1.147},"last":0.499}]},{"osd":6,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.0509999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.0600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.54800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543309,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61520,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310320,"statfs":{"total":96636764160,"available":96573767680,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36899999999999999,"15min":0.38200000000000001},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.214},"max":{"1min":0.46200000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38200000000000001,"15min":0.40699999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39700000000000002,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.67300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.439},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55200000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41699999999999998,"15min":0.443},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84399999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.998},"last":0.501}]},{"osd":5,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":1.008},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.627,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":1.034},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.65400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63300000000000001,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.95999999999999996,"15min":1.0449999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.67600000000000005},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52900000000000003}]}]},{"osd":2,"up_from":14,"seq":60129543307,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48408,"kb_used_data":3340,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323432,"statfs":{"total":96636764160,"available":96587194368,"internally_reserved":0,"allocated":3420160,"data_stored":1774535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.433,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.40799999999999997,"15min":0.41799999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.57299999999999995,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.436,"15min":0.441},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.441,"15min":0.45800000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.44800000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42399999999999999,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.163,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59699999999999998,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.69899999999999995,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.754,"5min":3.593,"15min":3.593},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.64700000000000002,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.73499999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74299999999999999,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.47199999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67700000000000005,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67500000000000004,"15min":0.68999999999999995},"min":{"1min":0.46700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.73599999999999999,"5min":3.633,"15min":3.633},"last":0.625}]},{"osd":7,"last update":"Mon Oct 16 11:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.67200000000000004,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.70799999999999996,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66300000000000003,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.70699999999999996,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.53200000000000003}]}]},{"osd":3,"up_from":14,"seq":60129543309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23899999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.48699999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.40100000000000002,"15min":0.437},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.41699999999999998,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.374,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.437,"5min":0.87,"15min":0.87},"last":0.313},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.437},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40899999999999997,"15min":0.44800000000000001},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.46600000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56200000000000006,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.68000000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.628},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60399999999999998,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71799999999999997,"5min":0.98799999999999999,"15min":1.0489999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59199999999999997,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.81299999999999994}]},{"osd":7,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.60599999999999998,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.70399999999999996}]}]},{"osd":4,"up_from":14,"seq":60129543308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96972,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94274868,"statfs":{"total":96636764160,"available":96537464832,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73199999999999998,"5min":0.78900000000000003,"15min":0.85099999999999998},"last":0.502},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55200000000000005,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55000000000000004,"15min":0.59999999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59199999999999997,"15min":0.625},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73599999999999999,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.879},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.748}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.91300000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.95299999999999996},"last":0.73399999999999999}]}]},{"osd":5,"up_from":14,"seq":60129543309,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.622},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":1.873,"15min":1.873},"last":0.44700000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69699999999999995,"5min":1.024,"15min":1.024},"last":0.54000000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":1.821,"15min":1.821},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58999999999999997,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.872,"15min":0.872},"last":0.754},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.745,"5min":1.919,"15min":1.919},"last":0.65700000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76200000000000001,"5min":2.02,"15min":2.02},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:25.851 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:25.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.971+0000 7f08f4146700 1 Processor -- start 2023-10-16T11:22:25.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.971+0000 7f08f4146700 1 -- start start 2023-10-16T11:22:25.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f4146700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0af450 0x7f08ec0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f4146700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f08ec0cd470 0x7f08ec0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f4146700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec06f870 con 0x7f08ec15dfd0 2023-10-16T11:22:25.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f4146700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec0d1160 con 0x7f08ec0cd470 2023-10-16T11:22:25.975 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f4146700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0cc8e0 con 0x7f08ec0af450 2023-10-16T11:22:25.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f08ec0cd470 0x7f08ec0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f08ec0cd470 0x7f08ec0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34300/0 (socket says 172.21.15.181:34300) 2023-10-16T11:22:25.976 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 -- 172.21.15.181:0/3676648897 learned_addr learned my addr 172.21.15.181:0/3676648897 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f16e1700 1 --2- 172.21.15.181:0/3676648897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0af450 0x7f08ec0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 -- 172.21.15.181:0/3676648897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0af450 msgr2=0x7f08ec0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 --2- 172.21.15.181:0/3676648897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0af450 0x7f08ec0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 -- 172.21.15.181:0/3676648897 >> v1:172.21.15.190:6789/0 conn(0x7f08ec15dfd0 legacy=0x7f08ec06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 -- 172.21.15.181:0/3676648897 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0ccbc0 con 0x7f08ec0cd470 2023-10-16T11:22:25.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f16e1700 1 --2- 172.21.15.181:0/3676648897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0af450 0x7f08ec0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:25.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f26e3700 1 --2- 172.21.15.181:0/3676648897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0cd470 0x7f08ec0cd920 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009e50 tx=0x7f08e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b37b9d7c26c2cc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.978 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.972+0000 7f08f0ee0700 1 -- 172.21.15.181:0/3676648897 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400e070 con 0x7f08ec0cd470 2023-10-16T11:22:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f0ee0700 1 -- 172.21.15.181:0/3676648897 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400b9c0 con 0x7f08ec0cd470 2023-10-16T11:22:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f0ee0700 1 -- 172.21.15.181:0/3676648897 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400bd70 con 0x7f08ec0cd470 2023-10-16T11:22:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 -- 172.21.15.181:0/3676648897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0cd470 msgr2=0x7f08ec0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 --2- 172.21.15.181:0/3676648897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0cd470 0x7f08ec0cd920 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009e50 tx=0x7f08e400b330 comp rx=0 tx=0).stop 2023-10-16T11:22:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 -- 172.21.15.181:0/3676648897 shutdown_connections 2023-10-16T11:22:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 --2- 172.21.15.181:0/3676648897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0cd470 0x7f08ec0cd920 unknown :-1 s=CLOSED pgs=3247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 --2- 172.21.15.181:0/3676648897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0af450 0x7f08ec0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 -- 172.21.15.181:0/3676648897 >> 172.21.15.181:0/3676648897 conn(0x7f08ec0d04a0 msgr2=0x7f08ec0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:25.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 -- 172.21.15.181:0/3676648897 shutdown_connections 2023-10-16T11:22:25.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.973+0000 7f08f4146700 1 -- 172.21.15.181:0/3676648897 wait complete. 2023-10-16T11:22:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.974+0000 7f08f4146700 1 Processor -- start 2023-10-16T11:22:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.974+0000 7f08f4146700 1 -- start start 2023-10-16T11:22:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f4146700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0af450 0x7f08ec0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f4146700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0cd470 0x7f08ec1f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f4146700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08ec15dfd0 0x7f08ec0c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f4146700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c5f80 con 0x7f08ec0af450 2023-10-16T11:22:25.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f4146700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c7230 con 0x7f08ec15dfd0 2023-10-16T11:22:25.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f4146700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c6190 con 0x7f08ec0cd470 2023-10-16T11:22:25.986 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0af450 0x7f08ec0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0af450 0x7f08ec0b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34302/0 (socket says 172.21.15.181:34302) 2023-10-16T11:22:25.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 -- 172.21.15.181:0/2419057185 learned_addr learned my addr 172.21.15.181:0/2419057185 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:25.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f26e3700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08ec15dfd0 0x7f08ec0c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.987 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f16e1700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0cd470 0x7f08ec1f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 -- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0cd470 msgr2=0x7f08ec1f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0cd470 0x7f08ec1f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.988 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 -- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08ec15dfd0 msgr2=0x7f08ec0c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:25.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08ec15dfd0 0x7f08ec0c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:25.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.975+0000 7f08f1ee2700 1 -- 172.21.15.181:0/2419057185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4009b30 con 0x7f08ec0af450 2023-10-16T11:22:25.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.976+0000 7f08f1ee2700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0af450 0x7f08ec0b5ae0 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c500 tx=0x7f08e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.989 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.976+0000 7f08e27fc700 1 -- 172.21.15.181:0/2419057185 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e80052d0 con 0x7f08ec0af450 2023-10-16T11:22:25.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.976+0000 7f08e27fc700 1 -- 172.21.15.181:0/2419057185 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e8005430 con 0x7f08ec0af450 2023-10-16T11:22:25.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.976+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec0c6470 con 0x7f08ec0af450 2023-10-16T11:22:25.990 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.976+0000 7f08e27fc700 1 -- 172.21.15.181:0/2419057185 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e80038f0 con 0x7f08ec0af450 2023-10-16T11:22:25.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.976+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0c6a30 con 0x7f08ec0af450 2023-10-16T11:22:25.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.977+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec067d20 con 0x7f08ec0af450 2023-10-16T11:22:25.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.977+0000 7f08e27fc700 1 -- 172.21.15.181:0/2419057185 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f08e8003a50 con 0x7f08ec0af450 2023-10-16T11:22:25.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.978+0000 7f08e27fc700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f08d006b020 0x7f08d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:25.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.978+0000 7f08e27fc700 1 -- 172.21.15.181:0/2419057185 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f08e8088d30 con 0x7f08ec0af450 2023-10-16T11:22:25.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.978+0000 7f08f16e1700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f08d006b020 0x7f08d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:25.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.978+0000 7f08f16e1700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f08d006b020 0x7f08d006d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08ec14b7e0 tx=0x7f08dc008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:25.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:25.982+0000 7f08e27fc700 1 -- 172.21.15.181:0/2419057185 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f08e8053b80 con 0x7f08ec0af450 2023-10-16T11:22:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.171+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08ec0cc590 con 0x7f08d006b020 2023-10-16T11:22:26.174 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.173+0000 7f08e27fc700 1 -- 172.21.15.181:0/2419057185 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50986 (secure 0 0 0) 0x7f08dc009dd0 con 0x7f08d006b020 2023-10-16T11:22:26.175 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:26.176 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f08d006b020 msgr2=0x7f08d006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f08d006b020 0x7f08d006d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f08ec14b7e0 tx=0x7f08dc008040 comp rx=0 tx=0).stop 2023-10-16T11:22:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0af450 msgr2=0x7f08ec0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.177 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0af450 0x7f08ec0b5ae0 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 crypto rx=0x7f08e800c500 tx=0x7f08e800c8c0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 shutdown_connections 2023-10-16T11:22:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f08d006b020 0x7f08d006d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08ec15dfd0 0x7f08ec0c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.178 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ec0af450 0x7f08ec0b5ae0 unknown :-1 s=CLOSED pgs=3248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 --2- 172.21.15.181:0/2419057185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08ec0cd470 0x7f08ec1f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 >> 172.21.15.181:0/2419057185 conn(0x7f08ec0d04a0 msgr2=0x7f08ec0adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.179 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 shutdown_connections 2023-10-16T11:22:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.175+0000 7f08f4146700 1 -- 172.21.15.181:0/2419057185 wait complete. 2023-10-16T11:22:26.180 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:22:26.188 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-10-16T11:22:25.837945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":501140,"kb_used_data":28264,"kb_used_omap":12,"kb_used_meta":472755,"kb_avail":754473580,"statfs":{"total":773094113280,"available":772580945920,"internally_reserved":0,"allocated":28942336,"data_stored":15606920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":484101688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-530476,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":10,"num_read_kb":133,"num_write":-143,"num_write_kb":-1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.000083"},"pg_stats":[{"pgid":"1.0","version":"272'414","reported_seq":1499,"reported_epoch":273,"state":"active+clean","last_fresh":"2023-10-16T11:22:21.337591+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:22:21.337591+0000","last_peered":"2023-10-16T11:22:21.337591+0000","last_clean":"2023-10-16T11:22:21.337591+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:22:21.337591+0000","last_undegraded":"2023-10-16T11:22:21.337591+0000","last_fullsized":"2023-10-16T11:22:21.337591+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543309,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57356,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314484,"statfs":{"total":96636764160,"available":96578031616,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.878,"15min":0.878},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27800000000000002},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55100000000000005,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.31900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69799999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.41299999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":1,"15min":1.002},"last":0.63300000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68400000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60399999999999998,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84299999999999997,"5min":1.052,"15min":1.052},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59099999999999997}]}]},{"osd":6,"up_from":14,"seq":60129543308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60940,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310900,"statfs":{"total":96636764160,"available":96574361600,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.60099999999999998},"min":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54000000000000004,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.59599999999999997},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.47199999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.745,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66700000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.29699999999999999},"max":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.98999999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.90700000000000003},"last":0.754}]},{"osd":5,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69699999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129543308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78984,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292856,"statfs":{"total":96636764160,"available":96555884544,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.39700000000000002},"min":{"1min":0.23499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.377,"15min":0.41799999999999998},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.44400000000000001,"5min":0.62,"15min":0.66900000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45200000000000001,"5min":0.56999999999999995,"15min":0.61799999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.379,"5min":0.40799999999999997,"15min":0.44900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.497,"5min":0.69699999999999995,"15min":0.755},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67300000000000004,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.36199999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":0.92600000000000005,"15min":1.097},"last":0.52200000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76600000000000001,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":1.147},"last":0.499}]},{"osd":6,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.0509999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.0600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.54800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543309,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61520,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310320,"statfs":{"total":96636764160,"available":96573767680,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36899999999999999,"15min":0.38200000000000001},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.214},"max":{"1min":0.46200000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38200000000000001,"15min":0.40699999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39700000000000002,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.67300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.439},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55200000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41699999999999998,"15min":0.443},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84399999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.998},"last":0.501}]},{"osd":5,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":1.008},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.627,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":1.034},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.65400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63300000000000001,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.95999999999999996,"15min":1.0449999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.67600000000000005},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52900000000000003}]}]},{"osd":2,"up_from":14,"seq":60129543308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.39500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48699999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.39100000000000001,"15min":0.41299999999999998},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.51400000000000001,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.42099999999999999,"15min":0.436},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48099999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.42399999999999999,"15min":0.45100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.311}]},{"osd":3,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.40799999999999997,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.503,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46999999999999997,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.53400000000000003,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.28199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.60499999999999998,"15min":0.63700000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67600000000000005,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.113,"5min":3.593,"15min":3.593},"last":0.39600000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.628,"15min":0.65500000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1910000000000001,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.505}]},{"osd":6,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":3.633,"15min":3.633},"last":0.443}]},{"osd":7,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.68200000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.151,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.75700000000000001,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.55600000000000005}]}]},{"osd":3,"up_from":14,"seq":60129543309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23899999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.48699999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.40100000000000002,"15min":0.437},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.41699999999999998,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.374,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.437,"5min":0.87,"15min":0.87},"last":0.313},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.437},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40899999999999997,"15min":0.44800000000000001},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.46600000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56200000000000006,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.68000000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.628},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60399999999999998,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71799999999999997,"5min":0.98799999999999999,"15min":1.0489999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59199999999999997,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.81299999999999994}]},{"osd":7,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.60599999999999998,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.70399999999999996}]}]},{"osd":4,"up_from":14,"seq":60129543308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96972,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94274868,"statfs":{"total":96636764160,"available":96537464832,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73199999999999998,"5min":0.78900000000000003,"15min":0.85099999999999998},"last":0.502},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55200000000000005,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55000000000000004,"15min":0.59999999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59199999999999997,"15min":0.625},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73599999999999999,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.879},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.748}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.91300000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.95299999999999996},"last":0.73399999999999999}]}]},{"osd":5,"up_from":14,"seq":60129543309,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.622},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":1.873,"15min":1.873},"last":0.44700000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69699999999999995,"5min":1.024,"15min":1.024},"last":0.54000000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":1.821,"15min":1.821},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58999999999999997,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.872,"15min":0.872},"last":0.754},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.745,"5min":1.919,"15min":1.919},"last":0.65700000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76200000000000001,"5min":2.02,"15min":2.02},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:26.190 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T11:22:26.190 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:26.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.310+0000 7f1169fc9700 1 Processor -- start 2023-10-16T11:22:26.312 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.310+0000 7f1169fc9700 1 -- start start 2023-10-16T11:22:26.314 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1169fc9700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11640af450 0x7f11640af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.315 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1169fc9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116406f870 con 0x7f116415dfd0 2023-10-16T11:22:26.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1169fc9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11640d1160 con 0x7f11640cd470 2023-10-16T11:22:26.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1169fc9700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11640cc8e0 con 0x7f11640af450 2023-10-16T11:22:26.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f11637fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f116415dfd0 0x7f116406d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45388/0 (socket says 172.21.15.181:45388) 2023-10-16T11:22:26.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f11637fe700 1 -- 172.21.15.181:0/3993244362 learned_addr learned my addr 172.21.15.181:0/3993244362 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:26.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1162ffd700 1 --2- 172.21.15.181:0/3993244362 >> v2:172.21.15.190:3300/0 conn(0x7f11640af450 0x7f11640af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f11627fc700 1 -- 172.21.15.181:0/3993244362 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593607863 0 0) 0x7f116406f870 con 0x7f116415dfd0 2023-10-16T11:22:26.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f11627fc700 1 -- 172.21.15.181:0/3993244362 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f114c0035b0 con 0x7f116415dfd0 2023-10-16T11:22:26.318 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1162ffd700 1 -- 172.21.15.181:0/3993244362 >> v1:172.21.15.190:6789/0 conn(0x7f11640cd470 legacy=0x7f116406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.319 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f11627fc700 1 -- 172.21.15.181:0/3993244362 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621921585 0 0) 0x7f11640d1160 con 0x7f11640cd470 2023-10-16T11:22:26.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1162ffd700 1 -- 172.21.15.181:0/3993244362 >> v1:172.21.15.181:6789/0 conn(0x7f116415dfd0 legacy=0x7f116406d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.321 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.311+0000 7f1162ffd700 1 -- 172.21.15.181:0/3993244362 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640ccbc0 con 0x7f11640af450 2023-10-16T11:22:26.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1162ffd700 1 --2- 172.21.15.181:0/3993244362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 0x7f11640af9a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1154009840 tx=0x7f1154009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286f45c020dc1646 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f11627fc700 1 -- 172.21.15.181:0/3993244362 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580017f0 con 0x7f11640af450 2023-10-16T11:22:26.322 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f11627fc700 1 -- 172.21.15.181:0/3993244362 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1154004030 con 0x7f11640af450 2023-10-16T11:22:26.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f11627fc700 1 -- 172.21.15.181:0/3993244362 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540043c0 con 0x7f11640af450 2023-10-16T11:22:26.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1169fc9700 1 -- 172.21.15.181:0/3993244362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 msgr2=0x7f11640af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.323 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1169fc9700 1 --2- 172.21.15.181:0/3993244362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 0x7f11640af9a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1154009840 tx=0x7f1154009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1169fc9700 1 -- 172.21.15.181:0/3993244362 shutdown_connections 2023-10-16T11:22:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1169fc9700 1 --2- 172.21.15.181:0/3993244362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 0x7f11640af9a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1169fc9700 1 -- 172.21.15.181:0/3993244362 >> 172.21.15.181:0/3993244362 conn(0x7f11640d04a0 msgr2=0x7f11640d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.324 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1169fc9700 1 -- 172.21.15.181:0/3993244362 shutdown_connections 2023-10-16T11:22:26.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.312+0000 7f1169fc9700 1 -- 172.21.15.181:0/3993244362 wait complete. 2023-10-16T11:22:26.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.313+0000 7f1169fc9700 1 Processor -- start 2023-10-16T11:22:26.325 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1169fc9700 1 -- start start 2023-10-16T11:22:26.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1169fc9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 0x7f11640c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1169fc9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11640cd470 0x7f11640b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.328 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1169fc9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f116415dfd0 0x7f1164158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1169fc9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164159a40 con 0x7f11640cd470 2023-10-16T11:22:26.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1169fc9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640b4a40 con 0x7f11640af450 2023-10-16T11:22:26.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1169fc9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11641594a0 con 0x7f116415dfd0 2023-10-16T11:22:26.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1162ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11640cd470 0x7f11640b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1162ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11640cd470 0x7f11640b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34318/0 (socket says 172.21.15.181:34318) 2023-10-16T11:22:26.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.314+0000 7f1162ffd700 1 -- 172.21.15.181:0/946721090 learned_addr learned my addr 172.21.15.181:0/946721090 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:26.330 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f11637fe700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 0x7f11640c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1163fff700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f116415dfd0 0x7f1164158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1162ffd700 1 -- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f116415dfd0 msgr2=0x7f1164158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.331 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1162ffd700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f116415dfd0 0x7f1164158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1162ffd700 1 -- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 msgr2=0x7f11640c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1162ffd700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 0x7f11640c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1162ffd700 1 -- 172.21.15.181:0/946721090 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540094a0 con 0x7f11640cd470 2023-10-16T11:22:26.332 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1162ffd700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11640cd470 0x7f11640b29a0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f11640716a0 tx=0x7f1154004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f114bfff700 1 -- 172.21.15.181:0/946721090 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540045b0 con 0x7f11640cd470 2023-10-16T11:22:26.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11640c51e0 con 0x7f11640cd470 2023-10-16T11:22:26.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f114bfff700 1 -- 172.21.15.181:0/946721090 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1154010590 con 0x7f11640cd470 2023-10-16T11:22:26.333 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f114bfff700 1 -- 172.21.15.181:0/946721090 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154010920 con 0x7f11640cd470 2023-10-16T11:22:26.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.315+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640b46d0 con 0x7f11640cd470 2023-10-16T11:22:26.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.316+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11640672c0 con 0x7f11640cd470 2023-10-16T11:22:26.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.316+0000 7f114bfff700 1 -- 172.21.15.181:0/946721090 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1154010a80 con 0x7f11640cd470 2023-10-16T11:22:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.317+0000 7f114bfff700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f115006b020 0x7f115006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.317+0000 7f114bfff700 1 -- 172.21.15.181:0/946721090 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f115408b100 con 0x7f11640cd470 2023-10-16T11:22:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.317+0000 7f11637fe700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f115006b020 0x7f115006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.335 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.317+0000 7f11637fe700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f115006b020 0x7f115006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1158001eb0 tx=0x7f11580012f0 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.321+0000 7f114bfff700 1 -- 172.21.15.181:0/946721090 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f1154055ed0 con 0x7f11640cd470 2023-10-16T11:22:26.511 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.510+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11640cc590 con 0x7f115006b020 2023-10-16T11:22:26.513 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.512+0000 7f114bfff700 1 -- 172.21.15.181:0/946721090 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50986 (secure 0 0 0) 0x7f11580058e0 con 0x7f115006b020 2023-10-16T11:22:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:26.515 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f115006b020 msgr2=0x7f115006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f115006b020 0x7f115006d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1158001eb0 tx=0x7f11580012f0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11640cd470 msgr2=0x7f11640b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11640cd470 0x7f11640b29a0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f11640716a0 tx=0x7f1154004750 comp rx=0 tx=0).stop 2023-10-16T11:22:26.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 shutdown_connections 2023-10-16T11:22:26.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f115006b020 0x7f115006d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f116415dfd0 0x7f1164158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640af450 0x7f11640c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 --2- 172.21.15.181:0/946721090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11640cd470 0x7f11640b29a0 unknown :-1 s=CLOSED pgs=3250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 >> 172.21.15.181:0/946721090 conn(0x7f11640d04a0 msgr2=0x7f116406d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.518 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 shutdown_connections 2023-10-16T11:22:26.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.514+0000 7f1169fc9700 1 -- 172.21.15.181:0/946721090 wait complete. 2023-10-16T11:22:26.519 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:22:26.530 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-10-16T11:22:25.837945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":501140,"kb_used_data":28264,"kb_used_omap":12,"kb_used_meta":472755,"kb_avail":754473580,"statfs":{"total":773094113280,"available":772580945920,"internally_reserved":0,"allocated":28942336,"data_stored":15606920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":484101688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-530476,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":10,"num_read_kb":133,"num_write":-143,"num_write_kb":-1330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.000083"},"pg_stats":[{"pgid":"1.0","version":"272'414","reported_seq":1499,"reported_epoch":273,"state":"active+clean","last_fresh":"2023-10-16T11:22:21.337591+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:22:21.337591+0000","last_peered":"2023-10-16T11:22:21.337591+0000","last_clean":"2023-10-16T11:22:21.337591+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:22:21.337591+0000","last_undegraded":"2023-10-16T11:22:21.337591+0000","last_fullsized":"2023-10-16T11:22:21.337591+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543309,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57356,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314484,"statfs":{"total":96636764160,"available":96578031616,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.36699999999999999,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.878,"15min":0.878},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.27800000000000002},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55500000000000005,"15min":0.59199999999999997},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55100000000000005,"15min":0.57799999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.31900000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58899999999999997,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69799999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.41299999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57099999999999995,"15min":0.59299999999999997},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.46100000000000002,"5min":0.39100000000000001,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":1,"15min":1.002},"last":0.63300000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.497,"15min":0.48299999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68400000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.60399999999999998,"15min":0.54500000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76900000000000002,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.52900000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84299999999999997,"5min":1.052,"15min":1.052},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.37,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59099999999999997}]}]},{"osd":6,"up_from":14,"seq":60129543308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60940,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310900,"statfs":{"total":96636764160,"available":96574361600,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.60099999999999998},"min":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54000000000000004,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.59599999999999997},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.47199999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.745,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66700000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.29699999999999999},"max":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.98999999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.90700000000000003},"last":0.754}]},{"osd":5,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69699999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.71299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129543308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78984,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292856,"statfs":{"total":96636764160,"available":96555884544,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.39700000000000002},"min":{"1min":0.23499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.377,"15min":0.41799999999999998},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.44400000000000001,"5min":0.62,"15min":0.66900000000000004},"last":0.42599999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45200000000000001,"5min":0.56999999999999995,"15min":0.61799999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.379,"5min":0.40799999999999997,"15min":0.44900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.497,"5min":0.69699999999999995,"15min":0.755},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67300000000000004,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.36199999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":0.92600000000000005,"15min":1.097},"last":0.52200000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76600000000000001,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":1.147},"last":0.499}]},{"osd":6,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.0509999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.0600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.54800000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543309,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61520,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310320,"statfs":{"total":96636764160,"available":96573767680,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36899999999999999,"15min":0.38200000000000001},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.214},"max":{"1min":0.46200000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38200000000000001,"15min":0.40699999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39700000000000002,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.67300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.439},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55200000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41699999999999998,"15min":0.443},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84399999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.998},"last":0.501}]},{"osd":5,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":1.008},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.627,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":1.034},"last":0.63}]},{"osd":6,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.65400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63300000000000001,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.95999999999999996,"15min":1.0449999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.67600000000000005},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52900000000000003}]}]},{"osd":2,"up_from":14,"seq":60129543308,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.39500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48699999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.39100000000000001,"15min":0.41299999999999998},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.51400000000000001,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.42099999999999999,"15min":0.436},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48099999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.42399999999999999,"15min":0.45100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.311}]},{"osd":3,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.40799999999999997,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.503,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46999999999999997,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.53400000000000003,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.28199999999999997}]},{"osd":4,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.60499999999999998,"15min":0.63700000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67600000000000005,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.113,"5min":3.593,"15min":3.593},"last":0.39600000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.628,"15min":0.65500000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1910000000000001,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.505}]},{"osd":6,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":3.633,"15min":3.633},"last":0.443}]},{"osd":7,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.68200000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.151,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.75700000000000001,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.55600000000000005}]}]},{"osd":3,"up_from":14,"seq":60129543309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23899999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.48699999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.40100000000000002,"15min":0.437},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.41699999999999998,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.374,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.437,"5min":0.87,"15min":0.87},"last":0.313},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.437},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40899999999999997,"15min":0.44800000000000001},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.46600000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56200000000000006,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.68000000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.628},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60399999999999998,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71799999999999997,"5min":0.98799999999999999,"15min":1.0489999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59199999999999997,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.81299999999999994}]},{"osd":7,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.60599999999999998,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.70399999999999996}]}]},{"osd":4,"up_from":14,"seq":60129543308,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96972,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94274868,"statfs":{"total":96636764160,"available":96537464832,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73199999999999998,"5min":0.78900000000000003,"15min":0.85099999999999998},"last":0.502},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55200000000000005,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65800000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55000000000000004,"15min":0.59999999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59199999999999997,"15min":0.625},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73599999999999999,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.879},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.748}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.91300000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.95299999999999996},"last":0.73399999999999999}]}]},{"osd":5,"up_from":14,"seq":60129543309,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.622},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":1.873,"15min":1.873},"last":0.44700000000000001}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69699999999999995,"5min":1.024,"15min":1.024},"last":0.54000000000000004}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":1.821,"15min":1.821},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58999999999999997,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.872,"15min":0.872},"last":0.754},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.745,"5min":1.919,"15min":1.919},"last":0.65700000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76200000000000001,"5min":2.02,"15min":2.02},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:26.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T11:22:26.653 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.652+0000 7fd06f4b4700 1 Processor -- start 2023-10-16T11:22:26.654 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.652+0000 7fd06f4b4700 1 -- start start 2023-10-16T11:22:26.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06f4b4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06815dfd0 0x7fd0680af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.656 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06f4b4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0680af450 0x7fd06806d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06f4b4700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06806f870 con 0x7fd06806db80 2023-10-16T11:22:26.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06f4b4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0680d1160 con 0x7fd0680af450 2023-10-16T11:22:26.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06f4b4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680cc8e0 con 0x7fd06815dfd0 2023-10-16T11:22:26.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06ca4f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0680af450 0x7fd06806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06ca4f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0680af450 0x7fd06806d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34324/0 (socket says 172.21.15.181:34324) 2023-10-16T11:22:26.658 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06ca4f700 1 -- 172.21.15.181:0/3864978387 learned_addr learned my addr 172.21.15.181:0/3864978387 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:26.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06d250700 1 --2- 172.21.15.181:0/3864978387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06815dfd0 0x7fd0680af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06ca4f700 1 -- 172.21.15.181:0/3864978387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06815dfd0 msgr2=0x7fd0680af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:26.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06ca4f700 1 --2- 172.21.15.181:0/3864978387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06815dfd0 0x7fd0680af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06ca4f700 1 -- 172.21.15.181:0/3864978387 >> v1:172.21.15.190:6789/0 conn(0x7fd06806db80 legacy=0x7fd0680cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06ca4f700 1 -- 172.21.15.181:0/3864978387 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680ccbc0 con 0x7fd0680af450 2023-10-16T11:22:26.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.653+0000 7fd06d250700 1 --2- 172.21.15.181:0/3864978387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06815dfd0 0x7fd0680af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:26.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06ca4f700 1 --2- 172.21.15.181:0/3864978387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0680af450 0x7fd06806d640 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7fd0640098f0 tx=0x7fd064009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31947e625199a621 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd05ffff700 1 -- 172.21.15.181:0/3864978387 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400b8a0 con 0x7fd0680af450 2023-10-16T11:22:26.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd05ffff700 1 -- 172.21.15.181:0/3864978387 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06400ba00 con 0x7fd0680af450 2023-10-16T11:22:26.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd05ffff700 1 -- 172.21.15.181:0/3864978387 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400bd90 con 0x7fd0680af450 2023-10-16T11:22:26.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 -- 172.21.15.181:0/3864978387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0680af450 msgr2=0x7fd06806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/3864978387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0680af450 0x7fd06806d640 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7fd0640098f0 tx=0x7fd064009c00 comp rx=0 tx=0).stop 2023-10-16T11:22:26.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 -- 172.21.15.181:0/3864978387 shutdown_connections 2023-10-16T11:22:26.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/3864978387 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06815dfd0 0x7fd0680af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/3864978387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0680af450 0x7fd06806d640 unknown :-1 s=CLOSED pgs=3251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 -- 172.21.15.181:0/3864978387 >> 172.21.15.181:0/3864978387 conn(0x7fd0680d04a0 msgr2=0x7fd0680d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 -- 172.21.15.181:0/3864978387 shutdown_connections 2023-10-16T11:22:26.663 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.654+0000 7fd06f4b4700 1 -- 172.21.15.181:0/3864978387 wait complete. 2023-10-16T11:22:26.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 Processor -- start 2023-10-16T11:22:26.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 -- start start 2023-10-16T11:22:26.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06806db80 0x7fd0680b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0680af450 0x7fd06814c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06815dfd0 0x7fd0680c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680b2460 con 0x7fd06815dfd0 2023-10-16T11:22:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680b0ff0 con 0x7fd0680af450 2023-10-16T11:22:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06f4b4700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680b12d0 con 0x7fd06806db80 2023-10-16T11:22:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06da51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06815dfd0 0x7fd0680c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.668 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06da51700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06815dfd0 0x7fd0680c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34334/0 (socket says 172.21.15.181:34334) 2023-10-16T11:22:26.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06da51700 1 -- 172.21.15.181:0/207742356 learned_addr learned my addr 172.21.15.181:0/207742356 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:26.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06d250700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06806db80 0x7fd0680b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.669 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.656+0000 7fd06ca4f700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0680af450 0x7fd06814c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06da51700 1 -- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06806db80 msgr2=0x7fd0680b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06da51700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06806db80 0x7fd0680b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06da51700 1 -- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0680af450 msgr2=0x7fd06814c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.670 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06da51700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0680af450 0x7fd06814c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06da51700 1 -- 172.21.15.181:0/207742356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064009550 con 0x7fd06815dfd0 2023-10-16T11:22:26.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06da51700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06815dfd0 0x7fd0680c57f0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7fd06000cc50 tx=0x7fd06000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.671 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd05dffb700 1 -- 172.21.15.181:0/207742356 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060003c00 con 0x7fd06815dfd0 2023-10-16T11:22:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd05dffb700 1 -- 172.21.15.181:0/207742356 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060003d60 con 0x7fd06815dfd0 2023-10-16T11:22:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0680b15b0 con 0x7fd06815dfd0 2023-10-16T11:22:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd05dffb700 1 -- 172.21.15.181:0/207742356 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000aab0 con 0x7fd06815dfd0 2023-10-16T11:22:26.672 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.657+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0680c7ca0 con 0x7fd06815dfd0 2023-10-16T11:22:26.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.658+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0680b1ad0 con 0x7fd06815dfd0 2023-10-16T11:22:26.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.659+0000 7fd05dffb700 1 -- 172.21.15.181:0/207742356 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd06000ac10 con 0x7fd06815dfd0 2023-10-16T11:22:26.673 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.659+0000 7fd05dffb700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd04c06b020 0x7fd04c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:26.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.659+0000 7fd05dffb700 1 -- 172.21.15.181:0/207742356 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fd060089920 con 0x7fd06815dfd0 2023-10-16T11:22:26.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.659+0000 7fd06d250700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd04c06b020 0x7fd04c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:26.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.659+0000 7fd06d250700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd04c06b020 0x7fd04c06d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c960 tx=0x7fd058008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:26.674 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.663+0000 7fd05dffb700 1 -- 172.21.15.181:0/207742356 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fd0680b1ad0 con 0x7fd06815dfd0 2023-10-16T11:22:26.857 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.856+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0680b1ad0 con 0x7fd06815dfd0 2023-10-16T11:22:26.858 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.856+0000 7fd05dffb700 1 -- 172.21.15.181:0/207742356 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v274) v1 ==== 74+0+14898 (secure 0 0 0) 0x7fd0680b1ad0 con 0x7fd06815dfd0 2023-10-16T11:22:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:26.858 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":274,"fsid":"c0cf369f-f4d9-40c8-9248-54aac3970136","created":"2023-10-16T09:43:44.024148+0000","modified":"2023-10-16T11:22:22.366585+0000","last_up_change":"2023-10-16T09:43:54.540378+0000","last_in_change":"2023-10-16T09:43:46.520078+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":37,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T09:43:55.719041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b92cf7de-b5f2-4270-83c3-2aaf2f896a5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6818","nonce":2908478956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6822","nonce":2908478956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6830","nonce":2908478956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2908478956},{"type":"v1","addr":"172.21.15.181:6826","nonce":2908478956}]},"public_addr":"172.21.15.181:6818/2908478956","cluster_addr":"172.21.15.181:6822/2908478956","heartbeat_back_addr":"172.21.15.181:6830/2908478956","heartbeat_front_addr":"172.21.15.181:6826/2908478956","state":["exists","up"]},{"osd":1,"uuid":"620b29ff-a66f-46b7-bb2f-949b053468cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6809","nonce":3991385762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6811","nonce":3991385762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6815","nonce":3991385762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3991385762},{"type":"v1","addr":"172.21.15.181:6813","nonce":3991385762}]},"public_addr":"172.21.15.181:6809/3991385762","cluster_addr":"172.21.15.181:6811/3991385762","heartbeat_back_addr":"172.21.15.181:6815/3991385762","heartbeat_front_addr":"172.21.15.181:6813/3991385762","state":["exists","up"]},{"osd":2,"uuid":"151b692d-6678-4237-b515-29e7c9df96f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6801","nonce":4285977707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6803","nonce":4285977707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6807","nonce":4285977707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4285977707},{"type":"v1","addr":"172.21.15.181:6805","nonce":4285977707}]},"public_addr":"172.21.15.181:6801/4285977707","cluster_addr":"172.21.15.181:6803/4285977707","heartbeat_back_addr":"172.21.15.181:6807/4285977707","heartbeat_front_addr":"172.21.15.181:6805/4285977707","state":["exists","up"]},{"osd":3,"uuid":"a8e30fbf-6d07-4029-91b3-a42e22ba92e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6819","nonce":2943330843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6823","nonce":2943330843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6829","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6831","nonce":2943330843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6825","nonce":2943330843},{"type":"v1","addr":"172.21.15.181:6827","nonce":2943330843}]},"public_addr":"172.21.15.181:6819/2943330843","cluster_addr":"172.21.15.181:6823/2943330843","heartbeat_back_addr":"172.21.15.181:6831/2943330843","heartbeat_front_addr":"172.21.15.181:6827/2943330843","state":["exists","up"]},{"osd":4,"uuid":"f35da5fd-caf3-44aa-b5c4-cc642216e1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6817","nonce":4008105976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6819","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6821","nonce":4008105976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6827","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6829","nonce":4008105976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6823","nonce":4008105976},{"type":"v1","addr":"172.21.15.190:6825","nonce":4008105976}]},"public_addr":"172.21.15.190:6817/4008105976","cluster_addr":"172.21.15.190:6821/4008105976","heartbeat_back_addr":"172.21.15.190:6829/4008105976","heartbeat_front_addr":"172.21.15.190:6825/4008105976","state":["exists","up"]},{"osd":5,"uuid":"fa387ca5-fce1-4570-a767-5ee46549a491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6809","nonce":646354310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6811","nonce":646354310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6815","nonce":646354310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":646354310},{"type":"v1","addr":"172.21.15.190:6813","nonce":646354310}]},"public_addr":"172.21.15.190:6809/646354310","cluster_addr":"172.21.15.190:6811/646354310","heartbeat_back_addr":"172.21.15.190:6815/646354310","heartbeat_front_addr":"172.21.15.190:6813/646354310","state":["exists","up"]},{"osd":6,"uuid":"792bf2d5-f768-42cc-af93-fe4b434c1cbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6801","nonce":487839335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6803","nonce":487839335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6807","nonce":487839335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":487839335},{"type":"v1","addr":"172.21.15.190:6805","nonce":487839335}]},"public_addr":"172.21.15.190:6801/487839335","cluster_addr":"172.21.15.190:6803/487839335","heartbeat_back_addr":"172.21.15.190:6807/487839335","heartbeat_front_addr":"172.21.15.190:6805/487839335","state":["exists","up"]},{"osd":7,"uuid":"13d6e7b6-bb6a-4843-9cba-8fb3454fccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":267,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6820","nonce":4234176418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6824","nonce":4234176418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6831","nonce":4234176418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":4234176418},{"type":"v1","addr":"172.21.15.190:6828","nonce":4234176418}]},"public_addr":"172.21.15.190:6820/4234176418","cluster_addr":"172.21.15.190:6824/4234176418","heartbeat_back_addr":"172.21.15.190:6831/4234176418","heartbeat_front_addr":"172.21.15.190:6828/4234176418","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.732326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.638219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.677685+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.692060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.761260+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.713044+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:51.655135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T09:43:53.537415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6832/284709222":"2023-10-17T11:22:18.801728+0000","172.21.15.181:0/1173792749":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/2911320793":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/290359218":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/3577989439":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/1668082750":"2023-10-17T11:22:17.819401+0000","172.21.15.181:6833/2902800927":"2023-10-17T11:22:17.819401+0000","172.21.15.181:6832/2902800927":"2023-10-17T11:22:17.819401+0000","172.21.15.190:6834/284709222":"2023-10-17T11:22:18.801728+0000","172.21.15.181:0/3788072243":"2023-10-17T11:22:17.819401+0000","172.21.15.181:0/2735196384":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6833/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:6832/2291437477":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/691655073":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/3571285885":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/171534192":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/714342180":"2023-10-17T11:22:10.332184+0000","172.21.15.181:0/432126216":"2023-10-17T11:22:10.332184+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":10,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":28,"length":1},{"begin":32,"length":1},{"begin":38,"length":1},{"begin":43,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":88,"length":1},{"begin":92,"length":1},{"begin":96,"length":1},{"begin":98,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":197,"length":25}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T11:22:26.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd04c06b020 msgr2=0x7fd04c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd04c06b020 0x7fd04c06d4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c960 tx=0x7fd058008040 comp rx=0 tx=0).stop 2023-10-16T11:22:26.860 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06815dfd0 msgr2=0x7fd0680c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:26.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06815dfd0 0x7fd0680c57f0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7fd06000cc50 tx=0x7fd06000cf60 comp rx=0 tx=0).stop 2023-10-16T11:22:26.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 shutdown_connections 2023-10-16T11:22:26.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fd04c06b020 0x7fd04c06d4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.861 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd06806db80 0x7fd0680b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06815dfd0 0x7fd0680c57f0 unknown :-1 s=CLOSED pgs=3252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 --2- 172.21.15.181:0/207742356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0680af450 0x7fd06814c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:26.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 >> 172.21.15.181:0/207742356 conn(0x7fd0680d04a0 msgr2=0x7fd06815cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:26.862 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 shutdown_connections 2023-10-16T11:22:26.863 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:26.859+0000 7fd06f4b4700 1 -- 172.21.15.181:0/207742356 wait complete. 2023-10-16T11:22:27.873 INFO:tasks.ceph:Scrubbing osd.0 2023-10-16T11:22:27.874 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:28.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.011+0000 7f2423b49700 1 Processor -- start 2023-10-16T11:22:28.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.011+0000 7f2423b49700 1 -- start start 2023-10-16T11:22:28.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f2423b49700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f241c0ce3a0 0x7f241c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f2423b49700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f241c0cd3f0 0x7f241c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f2423b49700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c0ce7e0 con 0x7f241c0afa10 2023-10-16T11:22:28.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f2423b49700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c0cd880 con 0x7f241c0ce3a0 2023-10-16T11:22:28.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f2423b49700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c15b540 con 0x7f241c0cd3f0 2023-10-16T11:22:28.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f241c0ce3a0 0x7f241c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f241c0ce3a0 0x7f241c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34340/0 (socket says 172.21.15.181:34340) 2023-10-16T11:22:28.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 -- 172.21.15.181:0/3215652109 learned_addr learned my addr 172.21.15.181:0/3215652109 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:28.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24220e6700 1 --2- 172.21.15.181:0/3215652109 >> v2:172.21.15.190:3300/0 conn(0x7f241c0cd3f0 0x7f241c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 -- 172.21.15.181:0/3215652109 >> v2:172.21.15.190:3300/0 conn(0x7f241c0cd3f0 msgr2=0x7f241c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 --2- 172.21.15.181:0/3215652109 >> v2:172.21.15.190:3300/0 conn(0x7f241c0cd3f0 0x7f241c0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 -- 172.21.15.181:0/3215652109 >> v1:172.21.15.190:6789/0 conn(0x7f241c0afa10 legacy=0x7f241c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 -- 172.21.15.181:0/3215652109 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c15b820 con 0x7f241c0ce3a0 2023-10-16T11:22:28.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24220e6700 1 --2- 172.21.15.181:0/3215652109 >> v2:172.21.15.190:3300/0 conn(0x7f241c0cd3f0 0x7f241c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:28.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.012+0000 7f24210e4700 1 --2- 172.21.15.181:0/3215652109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0cceb0 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 crypto rx=0x7f241c0703d0 tx=0x7f240c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf40e8af800c749 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f24208e3700 1 -- 172.21.15.181:0/3215652109 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c00dd30 con 0x7f241c0ce3a0 2023-10-16T11:22:28.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f24208e3700 1 -- 172.21.15.181:0/3215652109 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c00de90 con 0x7f241c0ce3a0 2023-10-16T11:22:28.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f24208e3700 1 -- 172.21.15.181:0/3215652109 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c010690 con 0x7f241c0ce3a0 2023-10-16T11:22:28.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 -- 172.21.15.181:0/3215652109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 msgr2=0x7f241c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.020 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 --2- 172.21.15.181:0/3215652109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0cceb0 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 crypto rx=0x7f241c0703d0 tx=0x7f240c00be70 comp rx=0 tx=0).stop 2023-10-16T11:22:28.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 -- 172.21.15.181:0/3215652109 shutdown_connections 2023-10-16T11:22:28.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 --2- 172.21.15.181:0/3215652109 >> v2:172.21.15.190:3300/0 conn(0x7f241c0cd3f0 0x7f241c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 --2- 172.21.15.181:0/3215652109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0cceb0 unknown :-1 s=CLOSED pgs=3253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.021 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 -- 172.21.15.181:0/3215652109 >> 172.21.15.181:0/3215652109 conn(0x7f241c0d04a0 msgr2=0x7f241c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 -- 172.21.15.181:0/3215652109 shutdown_connections 2023-10-16T11:22:28.022 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.013+0000 7f2423b49700 1 -- 172.21.15.181:0/3215652109 wait complete. 2023-10-16T11:22:28.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.014+0000 7f2423b49700 1 Processor -- start 2023-10-16T11:22:28.025 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.014+0000 7f2423b49700 1 -- start start 2023-10-16T11:22:28.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f2423b49700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f241c0afa10 0x7f241c157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f2423b49700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f241c0cd3f0 0x7f241c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.026 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f2423b49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f2423b49700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c1573d0 con 0x7f241c0ce3a0 2023-10-16T11:22:28.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f2423b49700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c154960 con 0x7f241c0cd3f0 2023-10-16T11:22:28.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f2423b49700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0c7ce0 con 0x7f241c0afa10 2023-10-16T11:22:28.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f24220e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f24220e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34348/0 (socket says 172.21.15.181:34348) 2023-10-16T11:22:28.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f24220e6700 1 -- 172.21.15.181:0/3385345979 learned_addr learned my addr 172.21.15.181:0/3385345979 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:28.028 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f24210e4700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f241c0cd3f0 0x7f241c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.015+0000 7f24218e5700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f241c0afa10 0x7f241c157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24220e6700 1 -- 172.21.15.181:0/3385345979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f241c0afa10 msgr2=0x7f241c157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.029 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24220e6700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f241c0afa10 0x7f241c157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24220e6700 1 -- 172.21.15.181:0/3385345979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f241c0cd3f0 msgr2=0x7f241c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24220e6700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f241c0cd3f0 0x7f241c0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24220e6700 1 -- 172.21.15.181:0/3385345979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c00b7f0 con 0x7f241c0ce3a0 2023-10-16T11:22:28.030 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24220e6700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0b3380 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 crypto rx=0x7f2414007ef0 tx=0x7f241400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400dd80 con 0x7f241c0ce3a0 2023-10-16T11:22:28.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400f040 con 0x7f241c0ce3a0 2023-10-16T11:22:28.031 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c0c7a00 con 0x7f241c0ce3a0 2023-10-16T11:22:28.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414003c60 con 0x7f241c0ce3a0 2023-10-16T11:22:28.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.016+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c157060 con 0x7f241c0ce3a0 2023-10-16T11:22:28.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.017+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f241c15bde0 con 0x7f241c0ce3a0 2023-10-16T11:22:28.032 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.017+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2414004030 con 0x7f241c0ce3a0 2023-10-16T11:22:28.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24127fc700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f240006b020 0x7f240006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f241408a210 con 0x7f241c0ce3a0 2023-10-16T11:22:28.033 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24218e5700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f240006b020 0x7f240006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24127fc700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f2400071a30 0x7f2400073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 --> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24000744e0 con 0x7f2400071a30 2023-10-16T11:22:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241408a580 con 0x7f241c0ce3a0 2023-10-16T11:22:28.034 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24210e4700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f2400071a30 0x7f2400073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24218e5700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f240006b020 0x7f240006d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2418000fb0 tx=0x7f2418009040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.018+0000 7f24210e4700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f2400071a30 0x7f2400073eb0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.035 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.019+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== osd.0 v2:172.21.15.181:6816/2908478956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f24000744e0 con 0x7f2400071a30 2023-10-16T11:22:28.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 --> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f241c15bb00 con 0x7f2400071a30 2023-10-16T11:22:28.036 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f24127fc700 1 -- 172.21.15.181:0/3385345979 <== osd.0 v2:172.21.15.181:6816/2908478956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f240c020040 con 0x7f2400071a30 2023-10-16T11:22:28.037 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:28.037 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:28.038 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:28.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f2400071a30 msgr2=0x7f2400073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6816/2908478956,v1:172.21.15.181:6818/2908478956] conn(0x7f2400071a30 0x7f2400073eb0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.039 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f240006b020 msgr2=0x7f240006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f240006b020 0x7f240006d4e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2418000fb0 tx=0x7f2418009040 comp rx=0 tx=0).stop 2023-10-16T11:22:28.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 msgr2=0x7f241c0b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 --2- 172.21.15.181:0/3385345979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f241c0ce3a0 0x7f241c0b3380 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 crypto rx=0x7f2414007ef0 tx=0x7f241400d3b0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.040 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f24220e6700 1 -- 172.21.15.181:0/3385345979 reap_dead start 2023-10-16T11:22:28.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 shutdown_connections 2023-10-16T11:22:28.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 >> 172.21.15.181:0/3385345979 conn(0x7f241c0d04a0 msgr2=0x7f241c15a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.041 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 shutdown_connections 2023-10-16T11:22:28.042 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.034+0000 7f2423b49700 1 -- 172.21.15.181:0/3385345979 wait complete. 2023-10-16T11:22:28.047 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-16T11:22:28.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:28.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.171+0000 7fae15ce1700 1 Processor -- start 2023-10-16T11:22:28.173 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.171+0000 7fae15ce1700 1 -- start start 2023-10-16T11:22:28.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae15ce1700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fae100d0fc0 0x7fae100cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae15ce1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae100cddf0 con 0x7fae1015df60 2023-10-16T11:22:28.180 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae15ce1700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae100ccf50 con 0x7fae1006a8a0 2023-10-16T11:22:28.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae15ce1700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae100af9c0 con 0x7fae100d0fc0 2023-10-16T11:22:28.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fae1015df60 0x7fae100cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45432/0 (socket says 172.21.15.181:45432) 2023-10-16T11:22:28.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0f7fe700 1 -- 172.21.15.181:0/2736743593 learned_addr learned my addr 172.21.15.181:0/2736743593 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:28.181 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0effd700 1 --2- 172.21.15.181:0/2736743593 >> v2:172.21.15.190:3300/0 conn(0x7fae100d0fc0 0x7fae100cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569432139 0 0) 0x7fae100cddf0 con 0x7fae1015df60 2023-10-16T11:22:28.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf80035b0 con 0x7fae1015df60 2023-10-16T11:22:28.182 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2314571215 0 0) 0x7fadf80035b0 con 0x7fae1015df60 2023-10-16T11:22:28.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 >> v2:172.21.15.190:3300/0 conn(0x7fae100d0fc0 msgr2=0x7fae100cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 --2- 172.21.15.181:0/2736743593 >> v2:172.21.15.190:3300/0 conn(0x7fae100d0fc0 0x7fae100cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.183 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 >> v1:172.21.15.190:6789/0 conn(0x7fae1006a8a0 legacy=0x7fae1006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100cc600 con 0x7fae1015df60 2023-10-16T11:22:28.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7fae040017f0 con 0x7fae1015df60 2023-10-16T11:22:28.184 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 >> v1:172.21.15.181:6789/0 conn(0x7fae1015df60 legacy=0x7fae100cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadf8003a00 0x7fadf8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadf8004850 0x7fadf8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.185 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fadf800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf800c080 con 0x7fadf8005210 2023-10-16T11:22:28.186 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf800c1f0 con 0x7fadf8004850 2023-10-16T11:22:28.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.172+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf800c3b0 con 0x7fadf8003a00 2023-10-16T11:22:28.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fadf800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0effd700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadf8004850 0x7fadf8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.187 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0f7fe700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadf8003a00 0x7fadf8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 -- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadf8003a00 msgr2=0x7fadf8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fadf8003a00 0x7fadf8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.188 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 -- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadf8004850 msgr2=0x7fadf8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadf8004850 0x7fadf8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 -- 172.21.15.181:0/2736743593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80035b0 con 0x7fadf8005210 2023-10-16T11:22:28.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0effd700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fadf8004850 0x7fadf8004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:28.189 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 -- 172.21.15.181:0/2736743593 reap_dead start 2023-10-16T11:22:28.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0ffff700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fadf800ba10 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c190 tx=0x7fadf8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91e6231af0b1598 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08001db0 con 0x7fadf8005210 2023-10-16T11:22:28.190 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800ccf0 con 0x7fadf8005210 2023-10-16T11:22:28.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.173+0000 7fae0e7fc700 1 -- 172.21.15.181:0/2736743593 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08003c40 con 0x7fadf8005210 2023-10-16T11:22:28.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.174+0000 7fae15ce1700 1 -- 172.21.15.181:0/2736743593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 msgr2=0x7fadf800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.174+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fadf800ba10 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c190 tx=0x7fadf8005880 comp rx=0 tx=0).stop 2023-10-16T11:22:28.191 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.174+0000 7fae15ce1700 1 -- 172.21.15.181:0/2736743593 shutdown_connections 2023-10-16T11:22:28.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.174+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2736743593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fadf800ba10 unknown :-1 s=CLOSED pgs=3256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.174+0000 7fae15ce1700 1 -- 172.21.15.181:0/2736743593 >> 172.21.15.181:0/2736743593 conn(0x7fae100d04a0 msgr2=0x7fae100d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.192 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.174+0000 7fae15ce1700 1 -- 172.21.15.181:0/2736743593 shutdown_connections 2023-10-16T11:22:28.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.174+0000 7fae15ce1700 1 -- 172.21.15.181:0/2736743593 wait complete. 2023-10-16T11:22:28.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.175+0000 7fae15ce1700 1 Processor -- start 2023-10-16T11:22:28.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.175+0000 7fae15ce1700 1 -- start start 2023-10-16T11:22:28.193 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae15ce1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fae1015d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae15ce1700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae101504f0 0x7fae1006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae15ce1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae1006f040 0x7fae1015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae15ce1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1006f4d0 con 0x7fadf8005210 2023-10-16T11:22:28.194 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae15ce1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae10160470 con 0x7fae1006f040 2023-10-16T11:22:28.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae15ce1700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae100b0fb0 con 0x7fae101504f0 2023-10-16T11:22:28.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fae1015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fae1015d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34366/0 (socket says 172.21.15.181:34366) 2023-10-16T11:22:28.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 -- 172.21.15.181:0/2238320788 learned_addr learned my addr 172.21.15.181:0/2238320788 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:28.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0ffff700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae1006f040 0x7fae1015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0effd700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae101504f0 0x7fae1006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.196 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 -- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae101504f0 msgr2=0x7fae1006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae101504f0 0x7fae1006eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 -- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae1006f040 msgr2=0x7fae1015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.197 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae1006f040 0x7fae1015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 -- 172.21.15.181:0/2238320788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100b1290 con 0x7fadf8005210 2023-10-16T11:22:28.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0ffff700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae1006f040 0x7fae1015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:28.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0effd700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae101504f0 0x7fae1006eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:28.198 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae0f7fe700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fae1015d160 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 crypto rx=0x7fae040011d0 tx=0x7fae0400fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fadf7fff700 1 -- 172.21.15.181:0/2238320788 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040072e0 con 0x7fadf8005210 2023-10-16T11:22:28.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fadf7fff700 1 -- 172.21.15.181:0/2238320788 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae04007440 con 0x7fadf8005210 2023-10-16T11:22:28.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fadf7fff700 1 -- 172.21.15.181:0/2238320788 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04005830 con 0x7fadf8005210 2023-10-16T11:22:28.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.176+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae100b1570 con 0x7fadf8005210 2023-10-16T11:22:28.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.177+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae100b1b30 con 0x7fadf8005210 2023-10-16T11:22:28.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.177+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae10067d20 con 0x7fadf8005210 2023-10-16T11:22:28.200 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.178+0000 7fadf7fff700 1 -- 172.21.15.181:0/2238320788 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fae04005990 con 0x7fadf8005210 2023-10-16T11:22:28.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.178+0000 7fadf7fff700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fadfc06b020 0x7fadfc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.178+0000 7fae0effd700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fadfc06b020 0x7fadfc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.178+0000 7fadf7fff700 1 -- 172.21.15.181:0/2238320788 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fae04089a00 con 0x7fadf8005210 2023-10-16T11:22:28.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.179+0000 7fae0effd700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fadfc06b020 0x7fadfc06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fae00009b50 tx=0x7fae00008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.202 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.183+0000 7fadf7fff700 1 -- 172.21.15.181:0/2238320788 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fae04054720 con 0x7fadf8005210 2023-10-16T11:22:28.375 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.374+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fae100cd1c0 con 0x7fadfc06b020 2023-10-16T11:22:28.376 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.374+0000 7fadf7fff700 1 -- 172.21.15.181:0/2238320788 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fae00004020 con 0x7fadfc06b020 2023-10-16T11:22:28.377 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fadfc06b020 msgr2=0x7fadfc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fadfc06b020 0x7fadfc06d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fae00009b50 tx=0x7fae00008040 comp rx=0 tx=0).stop 2023-10-16T11:22:28.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 msgr2=0x7fae1015d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.378 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fae1015d160 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 crypto rx=0x7fae040011d0 tx=0x7fae0400fe50 comp rx=0 tx=0).stop 2023-10-16T11:22:28.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 shutdown_connections 2023-10-16T11:22:28.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fadfc06b020 0x7fadfc06d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.379 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae1006f040 0x7fae1015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf8005210 0x7fae1015d160 unknown :-1 s=CLOSED pgs=3257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 --2- 172.21.15.181:0/2238320788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae101504f0 0x7fae1006eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 >> 172.21.15.181:0/2238320788 conn(0x7fae100d04a0 msgr2=0x7fae1006dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.380 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 shutdown_connections 2023-10-16T11:22:28.381 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.376+0000 7fae15ce1700 1 -- 172.21.15.181:0/2238320788 wait complete. 2023-10-16T11:22:28.381 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 0 to deep-scrub 2023-10-16T11:22:28.390 INFO:tasks.ceph:Scrubbing osd.1 2023-10-16T11:22:28.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:28.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.553+0000 7fdcb2182700 1 Processor -- start 2023-10-16T11:22:28.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.553+0000 7fdcb2182700 1 -- start start 2023-10-16T11:22:28.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcb2182700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac1608e0 0x7fdcac160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcb2182700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdcac0cff20 0x7fdcac0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcb2182700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcac06ce60 con 0x7fdcac0d03a0 2023-10-16T11:22:28.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcb2182700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcac0d1020 con 0x7fdcac0cff20 2023-10-16T11:22:28.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcb2182700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac06d2f0 con 0x7fdcac1608e0 2023-10-16T11:22:28.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcabfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdcac0d03a0 0x7fdcac161800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45454/0 (socket says 172.21.15.181:45454) 2023-10-16T11:22:28.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcabfff700 1 -- 172.21.15.181:0/263046431 learned_addr learned my addr 172.21.15.181:0/263046431 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:28.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcab7fe700 1 --2- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac1608e0 0x7fdcac160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcaaffd700 1 --2- 172.21.15.181:0/263046431 >> v2:172.21.15.190:3300/0 conn(0x7fdcac0cff20 0x7fdcac0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.554+0000 7fdcaa7fc700 1 -- 172.21.15.181:0/263046431 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283554839 0 0) 0x7fdcac06ce60 con 0x7fdcac0d03a0 2023-10-16T11:22:28.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaa7fc700 1 -- 172.21.15.181:0/263046431 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc940035b0 con 0x7fdcac0d03a0 2023-10-16T11:22:28.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaaffd700 1 -- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac1608e0 msgr2=0x7fdcac160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaaffd700 1 --2- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac1608e0 0x7fdcac160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaaffd700 1 -- 172.21.15.181:0/263046431 >> v1:172.21.15.181:6789/0 conn(0x7fdcac0d03a0 legacy=0x7fdcac161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaaffd700 1 -- 172.21.15.181:0/263046431 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac06c250 con 0x7fdcac0cff20 2023-10-16T11:22:28.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaaffd700 1 --2- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac0cff20 0x7fdcac0b2810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009d90 tx=0x7fdca400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c964651b96c5ab84 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaa7fc700 1 -- 172.21.15.181:0/263046431 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca400e070 con 0x7fdcac0cff20 2023-10-16T11:22:28.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaa7fc700 1 -- 172.21.15.181:0/263046431 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca400bb20 con 0x7fdcac0cff20 2023-10-16T11:22:28.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.555+0000 7fdcaa7fc700 1 -- 172.21.15.181:0/263046431 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca400be90 con 0x7fdcac0cff20 2023-10-16T11:22:28.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 -- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac0cff20 msgr2=0x7fdcac0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 --2- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac0cff20 0x7fdcac0b2810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009d90 tx=0x7fdca400b330 comp rx=0 tx=0).stop 2023-10-16T11:22:28.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 -- 172.21.15.181:0/263046431 shutdown_connections 2023-10-16T11:22:28.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 --2- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac1608e0 0x7fdcac160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 --2- 172.21.15.181:0/263046431 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac0cff20 0x7fdcac0b2810 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 -- 172.21.15.181:0/263046431 >> 172.21.15.181:0/263046431 conn(0x7fdcac0d1610 msgr2=0x7fdcac0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 -- 172.21.15.181:0/263046431 shutdown_connections 2023-10-16T11:22:28.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.556+0000 7fdcb2182700 1 -- 172.21.15.181:0/263046431 wait complete. 2023-10-16T11:22:28.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.557+0000 7fdcb2182700 1 Processor -- start 2023-10-16T11:22:28.566 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.557+0000 7fdcb2182700 1 -- start start 2023-10-16T11:22:28.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac0cff20 0x7fdcac1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac1608e0 0x7fdcac0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcac0b2dc0 0x7fdcac09baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac0b3300 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1f4010 con 0x7fdcac1608e0 2023-10-16T11:22:28.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1f3470 con 0x7fdcac0cff20 2023-10-16T11:22:28.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcac0b2dc0 0x7fdcac09baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcac0b2dc0 0x7fdcac09baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34382/0 (socket says 172.21.15.181:34382) 2023-10-16T11:22:28.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 -- 172.21.15.181:0/3631748336 learned_addr learned my addr 172.21.15.181:0/3631748336 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:28.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcaaffd700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac1608e0 0x7fdcac0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcab7fe700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac0cff20 0x7fdcac1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 -- 172.21.15.181:0/3631748336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac0cff20 msgr2=0x7fdcac1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdcac0cff20 0x7fdcac1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 -- 172.21.15.181:0/3631748336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac1608e0 msgr2=0x7fdcac0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdcac1608e0 0x7fdcac0b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 -- 172.21.15.181:0/3631748336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca4009a70 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcabfff700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcac0b2dc0 0x7fdcac09baa0 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c500 tx=0x7fdca000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0005d70 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca00070d0 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac1f36e0 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0007440 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.558+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac1f3ca0 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.559+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdcac06c810 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.560+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdca00075a0 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.560+0000 7fdc93fff700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fdc9806b020 0x7fdc9806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.560+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fdca00894f0 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.560+0000 7fdcab7fe700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fdc9806b020 0x7fdc9806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.560+0000 7fdc93fff700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fdc98071a30 0x7fdc98073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.560+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 --> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc980744e0 con 0x7fdc98071a30 2023-10-16T11:22:28.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.560+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdca0089940 con 0x7fdcac0b2dc0 2023-10-16T11:22:28.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.561+0000 7fdcaaffd700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fdc98071a30 0x7fdc98073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.561+0000 7fdcab7fe700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fdc9806b020 0x7fdc9806d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009b40 tx=0x7fdc9c009210 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.561+0000 7fdcaaffd700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fdc98071a30 0x7fdc98073eb0 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.561+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== osd.1 v2:172.21.15.181:6808/3991385762 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdc980744e0 con 0x7fdc98071a30 2023-10-16T11:22:28.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.576+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 --> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdcac06c530 con 0x7fdc98071a30 2023-10-16T11:22:28.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.576+0000 7fdc93fff700 1 -- 172.21.15.181:0/3631748336 <== osd.1 v2:172.21.15.181:6808/3991385762 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdca4003a20 con 0x7fdc98071a30 2023-10-16T11:22:28.582 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:28.583 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:28.583 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:28.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.576+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fdc98071a30 msgr2=0x7fdc98073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.576+0000 7fdcb2182700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6808/3991385762,v1:172.21.15.181:6809/3991385762] conn(0x7fdc98071a30 0x7fdc98073eb0 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fdc9806b020 msgr2=0x7fdc9806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fdc9806b020 0x7fdc9806d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009b40 tx=0x7fdc9c009210 comp rx=0 tx=0).stop 2023-10-16T11:22:28.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcac0b2dc0 msgr2=0x7fdcac09baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 --2- 172.21.15.181:0/3631748336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcac0b2dc0 0x7fdcac09baa0 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c500 tx=0x7fdca000c810 comp rx=0 tx=0).stop 2023-10-16T11:22:28.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcabfff700 1 -- 172.21.15.181:0/3631748336 reap_dead start 2023-10-16T11:22:28.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 shutdown_connections 2023-10-16T11:22:28.586 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 >> 172.21.15.181:0/3631748336 conn(0x7fdcac0d1610 msgr2=0x7fdcac0ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 shutdown_connections 2023-10-16T11:22:28.587 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.577+0000 7fdcb2182700 1 -- 172.21.15.181:0/3631748336 wait complete. 2023-10-16T11:22:28.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-16T11:22:28.710 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.709+0000 7f86ccda9700 1 Processor -- start 2023-10-16T11:22:28.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.710+0000 7f86ccda9700 1 -- start start 2023-10-16T11:22:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.710+0000 7f86ccda9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 0x7f86c80cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.710+0000 7f86ccda9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c806c870 con 0x7f86c80b3890 2023-10-16T11:22:28.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.710+0000 7f86ccda9700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c80cc3d0 con 0x7f86c815dea0 2023-10-16T11:22:28.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.710+0000 7f86ccda9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c806fa00 con 0x7f86c815dac0 2023-10-16T11:22:28.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c6d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f86c80b3890 0x7f86c806ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45470/0 (socket says 172.21.15.181:45470) 2023-10-16T11:22:28.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c6d9d700 1 -- 172.21.15.181:0/1863402397 learned_addr learned my addr 172.21.15.181:0/1863402397 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c659c700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 0x7f86c80cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280782783 0 0) 0x7f86c806c870 con 0x7f86c80b3890 2023-10-16T11:22:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b00035b0 con 0x7f86c80b3890 2023-10-16T11:22:28.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625258467 0 0) 0x7f86c80cc3d0 con 0x7f86c815dea0 2023-10-16T11:22:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c806c870 con 0x7f86c815dea0 2023-10-16T11:22:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3772255352 0 0) 0x7f86b00035b0 con 0x7f86c80b3890 2023-10-16T11:22:28.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 msgr2=0x7f86c80cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 0x7f86c80cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 >> v1:172.21.15.190:6789/0 conn(0x7f86c815dea0 legacy=0x7f86c80b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c806bf40 con 0x7f86c80b3890 2023-10-16T11:22:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f86c00019f0 con 0x7f86c80b3890 2023-10-16T11:22:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c659c700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 0x7f86c80cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 >> v1:172.21.15.181:6789/0 conn(0x7f86c80b3890 legacy=0x7f86c806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 0x7f86b0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b0004100 0x7f86b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c806fa00 con 0x7f86b0004d90 2023-10-16T11:22:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b000bac0 con 0x7f86b0004100 2023-10-16T11:22:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b000bc80 con 0x7f86c815dac0 2023-10-16T11:22:28.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c6d9d700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c659c700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 0x7f86b0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.711+0000 7f86c5d9b700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b0004100 0x7f86b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c6d9d700 1 -- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 msgr2=0x7f86b0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c6d9d700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c815dac0 0x7f86b0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c6d9d700 1 -- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b0004100 msgr2=0x7f86b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c6d9d700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86b0004100 0x7f86b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c6d9d700 1 -- 172.21.15.181:0/1863402397 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0094a0 con 0x7f86b0004d90 2023-10-16T11:22:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c6d9d700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86b0005210 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 crypto rx=0x7f86c806c820 tx=0x7f86b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8c797202c2ccaa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00072d0 con 0x7f86b0004d90 2023-10-16T11:22:28.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0007450 con 0x7f86b0004d90 2023-10-16T11:22:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c559a700 1 -- 172.21.15.181:0/1863402397 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0007800 con 0x7f86b0004d90 2023-10-16T11:22:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86ccda9700 1 -- 172.21.15.181:0/1863402397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 msgr2=0x7f86b0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86ccda9700 1 --2- 172.21.15.181:0/1863402397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86b0005210 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 crypto rx=0x7f86c806c820 tx=0x7f86b0005880 comp rx=0 tx=0).stop 2023-10-16T11:22:28.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86c6d9d700 1 -- 172.21.15.181:0/1863402397 reap_dead start 2023-10-16T11:22:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86ccda9700 1 -- 172.21.15.181:0/1863402397 shutdown_connections 2023-10-16T11:22:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86ccda9700 1 -- 172.21.15.181:0/1863402397 >> 172.21.15.181:0/1863402397 conn(0x7f86c80ce180 msgr2=0x7f86c80ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.712+0000 7f86ccda9700 1 -- 172.21.15.181:0/1863402397 shutdown_connections 2023-10-16T11:22:28.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.713+0000 7f86ccda9700 1 -- 172.21.15.181:0/1863402397 wait complete. 2023-10-16T11:22:28.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.714+0000 7f86ccda9700 1 Processor -- start 2023-10-16T11:22:28.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.714+0000 7f86ccda9700 1 -- start start 2023-10-16T11:22:28.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86c8152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c81601d0 0x7f86c80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c80cd200 0x7f86c80cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80cdbd0 con 0x7f86b0004d90 2023-10-16T11:22:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c8160650 con 0x7f86c81601d0 2023-10-16T11:22:28.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80b4860 con 0x7f86c80cd200 2023-10-16T11:22:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86c8152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86c8152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34400/0 (socket says 172.21.15.181:34400) 2023-10-16T11:22:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 -- 172.21.15.181:0/2689774120 learned_addr learned my addr 172.21.15.181:0/2689774120 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:28.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c6d9d700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c80cd200 0x7f86c80cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c5d9b700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c81601d0 0x7f86c80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 -- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c80cd200 msgr2=0x7f86c80cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c80cd200 0x7f86c80cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 -- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c81601d0 msgr2=0x7f86c80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c81601d0 0x7f86c80b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 -- 172.21.15.181:0/2689774120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c80b4ae0 con 0x7f86b0004d90 2023-10-16T11:22:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86c659c700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86c8152d40 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 crypto rx=0x7f86bc011cd0 tx=0x7f86bc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86b6ffd700 1 -- 172.21.15.181:0/2689774120 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc00fa00 con 0x7f86b0004d90 2023-10-16T11:22:28.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86b6ffd700 1 -- 172.21.15.181:0/2689774120 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00fb60 con 0x7f86b0004d90 2023-10-16T11:22:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c80b4dc0 con 0x7f86b0004d90 2023-10-16T11:22:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86b6ffd700 1 -- 172.21.15.181:0/2689774120 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc014440 con 0x7f86b0004d90 2023-10-16T11:22:28.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.715+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c80b5380 con 0x7f86b0004d90 2023-10-16T11:22:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.716+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c815c4e0 con 0x7f86b0004d90 2023-10-16T11:22:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.717+0000 7f86b6ffd700 1 -- 172.21.15.181:0/2689774120 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f86bc0145a0 con 0x7f86b0004d90 2023-10-16T11:22:28.738 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.717+0000 7f86b6ffd700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86a806b020 0x7f86a806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:28.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.717+0000 7f86b6ffd700 1 -- 172.21.15.181:0/2689774120 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f86bc08a0d0 con 0x7f86b0004d90 2023-10-16T11:22:28.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.717+0000 7f86c5d9b700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86a806b020 0x7f86a806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:28.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.717+0000 7f86c5d9b700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86a806b020 0x7f86a806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86b8007fd0 tx=0x7f86b8009040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:28.739 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.721+0000 7f86b6ffd700 1 -- 172.21.15.181:0/2689774120 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f86bc054f20 con 0x7f86b0004d90 2023-10-16T11:22:28.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.914+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f86c806c500 con 0x7f86a806b020 2023-10-16T11:22:28.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.914+0000 7f86b6ffd700 1 -- 172.21.15.181:0/2689774120 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f86b8006860 con 0x7f86a806b020 2023-10-16T11:22:28.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.916+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86a806b020 msgr2=0x7f86a806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.916+0000 7f86ccda9700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86a806b020 0x7f86a806d4e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86b8007fd0 tx=0x7f86b8009040 comp rx=0 tx=0).stop 2023-10-16T11:22:28.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.916+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 msgr2=0x7f86c8152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:28.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.916+0000 7f86ccda9700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86c8152d40 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 crypto rx=0x7f86bc011cd0 tx=0x7f86bc009ed0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.916+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 shutdown_connections 2023-10-16T11:22:28.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.916+0000 7f86ccda9700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86a806b020 0x7f86a806d4e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.916+0000 7f86ccda9700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86c80cd200 0x7f86c80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.917+0000 7f86ccda9700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b0004d90 0x7f86c8152d40 unknown :-1 s=CLOSED pgs=3262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.917+0000 7f86ccda9700 1 --2- 172.21.15.181:0/2689774120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86c81601d0 0x7f86c80b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:28.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.917+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 >> 172.21.15.181:0/2689774120 conn(0x7f86c80ce180 msgr2=0x7f86c80b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:28.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.917+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 shutdown_connections 2023-10-16T11:22:28.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:28.917+0000 7f86ccda9700 1 -- 172.21.15.181:0/2689774120 wait complete. 2023-10-16T11:22:28.922 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 1 to deep-scrub 2023-10-16T11:22:28.930 INFO:tasks.ceph:Scrubbing osd.2 2023-10-16T11:22:28.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:29.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.053+0000 7fccf1e05700 1 Processor -- start 2023-10-16T11:22:29.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fccf1e05700 1 -- start start 2023-10-16T11:22:29.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fccf1e05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccec15dc60 0x7fccec0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fccf1e05700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fccec0d06a0 0x7fccec0b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fccf1e05700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec0d22a0 con 0x7fccec15d880 2023-10-16T11:22:29.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fccf1e05700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0d0ae0 con 0x7fccec15dc60 2023-10-16T11:22:29.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fccf1e05700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0d1690 con 0x7fccec0d06a0 2023-10-16T11:22:29.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fcceaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccec15dc60 0x7fccec0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fcceaffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccec15dc60 0x7fccec0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34410/0 (socket says 172.21.15.181:34410) 2023-10-16T11:22:29.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.054+0000 7fcceaffd700 1 -- 172.21.15.181:0/1510752122 learned_addr learned my addr 172.21.15.181:0/1510752122 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:29.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fccebfff700 1 --2- 172.21.15.181:0/1510752122 >> v2:172.21.15.190:3300/0 conn(0x7fccec0d06a0 0x7fccec0b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fcceaffd700 1 -- 172.21.15.181:0/1510752122 >> v2:172.21.15.190:3300/0 conn(0x7fccec0d06a0 msgr2=0x7fccec0b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:29.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fcceaffd700 1 --2- 172.21.15.181:0/1510752122 >> v2:172.21.15.190:3300/0 conn(0x7fccec0d06a0 0x7fccec0b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fcceaffd700 1 -- 172.21.15.181:0/1510752122 >> v1:172.21.15.190:6789/0 conn(0x7fccec15d880 legacy=0x7fccec0cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fcceaffd700 1 -- 172.21.15.181:0/1510752122 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0d1970 con 0x7fccec15dc60 2023-10-16T11:22:29.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fcceaffd700 1 --2- 172.21.15.181:0/1510752122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec0d0160 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 crypto rx=0x7fccec0d4860 tx=0x7fccdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a0dd163da42adb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fccea7fc700 1 -- 172.21.15.181:0/1510752122 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00b870 con 0x7fccec15dc60 2023-10-16T11:22:29.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fccea7fc700 1 -- 172.21.15.181:0/1510752122 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc00b9d0 con 0x7fccec15dc60 2023-10-16T11:22:29.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fccea7fc700 1 -- 172.21.15.181:0/1510752122 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00bd60 con 0x7fccec15dc60 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fccf1e05700 1 -- 172.21.15.181:0/1510752122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 msgr2=0x7fccec0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.055+0000 7fccf1e05700 1 --2- 172.21.15.181:0/1510752122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec0d0160 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 crypto rx=0x7fccec0d4860 tx=0x7fccdc009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.056+0000 7fccf1e05700 1 -- 172.21.15.181:0/1510752122 shutdown_connections 2023-10-16T11:22:29.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.056+0000 7fccf1e05700 1 --2- 172.21.15.181:0/1510752122 >> v2:172.21.15.190:3300/0 conn(0x7fccec0d06a0 0x7fccec0b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.056+0000 7fccf1e05700 1 --2- 172.21.15.181:0/1510752122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec0d0160 unknown :-1 s=CLOSED pgs=3263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.056+0000 7fccf1e05700 1 -- 172.21.15.181:0/1510752122 >> 172.21.15.181:0/1510752122 conn(0x7fccec09cb10 msgr2=0x7fccec0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.056+0000 7fccf1e05700 1 -- 172.21.15.181:0/1510752122 shutdown_connections 2023-10-16T11:22:29.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.056+0000 7fccf1e05700 1 -- 172.21.15.181:0/1510752122 wait complete. 2023-10-16T11:22:29.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.057+0000 7fccf1e05700 1 Processor -- start 2023-10-16T11:22:29.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.057+0000 7fccf1e05700 1 -- start start 2023-10-16T11:22:29.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccec0d06a0 0x7fccec1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccec15d880 0x7fccec1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec155d40 con 0x7fccec15dc60 2023-10-16T11:22:29.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec155530 con 0x7fccec0d06a0 2023-10-16T11:22:29.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec157040 con 0x7fccec15d880 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34418/0 (socket says 172.21.15.181:34418) 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 -- 172.21.15.181:0/549560433 learned_addr learned my addr 172.21.15.181:0/549560433 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:29.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fcceb7fe700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccec0d06a0 0x7fccec1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fcceaffd700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccec15d880 0x7fccec1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 -- 172.21.15.181:0/549560433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccec15d880 msgr2=0x7fccec1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccec15d880 0x7fccec1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 -- 172.21.15.181:0/549560433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccec0d06a0 msgr2=0x7fccec1560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccec0d06a0 0x7fccec1560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 -- 172.21.15.181:0/549560433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc009550 con 0x7fccec15dc60 2023-10-16T11:22:29.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccebfff700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec149a60 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7fcce4007e70 tx=0x7fcce400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400ed30 con 0x7fccec15dc60 2023-10-16T11:22:29.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce400eeb0 con 0x7fccec15dc60 2023-10-16T11:22:29.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4014640 con 0x7fccec15dc60 2023-10-16T11:22:29.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec155780 con 0x7fccec15dc60 2023-10-16T11:22:29.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.058+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1547e0 con 0x7fccec15dc60 2023-10-16T11:22:29.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.059+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fccec0d1f30 con 0x7fccec15dc60 2023-10-16T11:22:29.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.060+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcce40147a0 con 0x7fccec15dc60 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.060+0000 7fcce3fff700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fcccc06b020 0x7fcccc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.060+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fcce408ae90 con 0x7fccec15dc60 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.060+0000 7fcceb7fe700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fcccc06b020 0x7fcccc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.060+0000 7fcce3fff700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fcccc071a30 0x7fcccc073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.060+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 --> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcccc0744e0 con 0x7fcccc071a30 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.060+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcce408b200 con 0x7fccec15dc60 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.061+0000 7fcceaffd700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fcccc071a30 0x7fcccc073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.061+0000 7fcceb7fe700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fcccc06b020 0x7fcccc06d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccd4000fb0 tx=0x7fccd4009040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.061+0000 7fcceaffd700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fcccc071a30 0x7fcccc073eb0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.061+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== osd.2 v2:172.21.15.181:6800/4285977707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcccc0744e0 con 0x7fcccc071a30 2023-10-16T11:22:29.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.075+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 --> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fccec0d1c50 con 0x7fcccc071a30 2023-10-16T11:22:29.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fcce3fff700 1 -- 172.21.15.181:0/549560433 <== osd.2 v2:172.21.15.181:6800/4285977707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fccdc021040 con 0x7fcccc071a30 2023-10-16T11:22:29.079 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:29.079 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:29.079 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:29.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fcccc071a30 msgr2=0x7fcccc073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6800/4285977707,v1:172.21.15.181:6801/4285977707] conn(0x7fcccc071a30 0x7fcccc073eb0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fcccc06b020 msgr2=0x7fcccc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fcccc06b020 0x7fcccc06d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccd4000fb0 tx=0x7fccd4009040 comp rx=0 tx=0).stop 2023-10-16T11:22:29.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 msgr2=0x7fccec149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 --2- 172.21.15.181:0/549560433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccec15dc60 0x7fccec149a60 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7fcce4007e70 tx=0x7fcce400e3c0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccebfff700 1 -- 172.21.15.181:0/549560433 reap_dead start 2023-10-16T11:22:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 shutdown_connections 2023-10-16T11:22:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 >> 172.21.15.181:0/549560433 conn(0x7fccec09cb10 msgr2=0x7fccec15ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 shutdown_connections 2023-10-16T11:22:29.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.076+0000 7fccf1e05700 1 -- 172.21.15.181:0/549560433 wait complete. 2023-10-16T11:22:29.089 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-16T11:22:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.210+0000 7f5a9d7bc700 1 Processor -- start 2023-10-16T11:22:29.211 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.210+0000 7f5a9d7bc700 1 -- start start 2023-10-16T11:22:29.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.210+0000 7f5a9d7bc700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5a9815a890 0x7f5a980ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.210+0000 7f5a9d7bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5a980cd370 0x7f5a9815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.210+0000 7f5a9d7bc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a980cdae0 con 0x7f5a9806cb40 2023-10-16T11:22:29.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.210+0000 7f5a9d7bc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980cddc0 con 0x7f5a980cd370 2023-10-16T11:22:29.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.210+0000 7f5a9d7bc700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980ce0a0 con 0x7f5a9815a890 2023-10-16T11:22:29.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a967fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5a980cd370 0x7f5a9815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a977fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5a9806cb40 0x7f5a98071cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45506/0 (socket says 172.21.15.181:45506) 2023-10-16T11:22:29.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a977fe700 1 -- 172.21.15.181:0/4145723305 learned_addr learned my addr 172.21.15.181:0/4145723305 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:29.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a96ffd700 1 --2- 172.21.15.181:0/4145723305 >> v2:172.21.15.190:3300/0 conn(0x7f5a9815a890 0x7f5a980ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a967fc700 1 -- 172.21.15.181:0/4145723305 >> v2:172.21.15.190:3300/0 conn(0x7f5a9815a890 msgr2=0x7f5a980ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:29.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a967fc700 1 --2- 172.21.15.181:0/4145723305 >> v2:172.21.15.190:3300/0 conn(0x7f5a9815a890 0x7f5a980ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a967fc700 1 -- 172.21.15.181:0/4145723305 >> v1:172.21.15.181:6789/0 conn(0x7f5a9806cb40 legacy=0x7f5a98071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a967fc700 1 -- 172.21.15.181:0/4145723305 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9815e440 con 0x7f5a980cd370 2023-10-16T11:22:29.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a95ffb700 1 -- 172.21.15.181:0/4145723305 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471545871 0 0) 0x7f5a980cdae0 con 0x7f5a9806cb40 2023-10-16T11:22:29.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a967fc700 1 --2- 172.21.15.181:0/4145723305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a980cd370 0x7f5a9815ec40 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f5a9806ec20 tx=0x7f5a8c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c09df9e686052f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a95ffb700 1 -- 172.21.15.181:0/4145723305 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c011830 con 0x7f5a980cd370 2023-10-16T11:22:29.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a95ffb700 1 -- 172.21.15.181:0/4145723305 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c011990 con 0x7f5a980cd370 2023-10-16T11:22:29.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.211+0000 7f5a95ffb700 1 -- 172.21.15.181:0/4145723305 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c011d20 con 0x7f5a980cd370 2023-10-16T11:22:29.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/4145723305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a980cd370 msgr2=0x7f5a9815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/4145723305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a980cd370 0x7f5a9815ec40 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f5a9806ec20 tx=0x7f5a8c009b20 comp rx=0 tx=0).stop 2023-10-16T11:22:29.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/4145723305 shutdown_connections 2023-10-16T11:22:29.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/4145723305 >> v2:172.21.15.190:3300/0 conn(0x7f5a9815a890 0x7f5a980ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/4145723305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a980cd370 0x7f5a9815ec40 unknown :-1 s=CLOSED pgs=3266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/4145723305 >> 172.21.15.181:0/4145723305 conn(0x7f5a9806aed0 msgr2=0x7f5a980cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/4145723305 shutdown_connections 2023-10-16T11:22:29.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.212+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/4145723305 wait complete. 2023-10-16T11:22:29.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.213+0000 7f5a9d7bc700 1 Processor -- start 2023-10-16T11:22:29.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.213+0000 7f5a9d7bc700 1 -- start start 2023-10-16T11:22:29.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a9d7bc700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9806cb40 0x7f5a981f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a9d7bc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a980cd370 0x7f5a980b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a9d7bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a9815a890 0x7f5a9815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a9d7bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981f36f0 con 0x7f5a9815a890 2023-10-16T11:22:29.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a9d7bc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980b3b70 con 0x7f5a980cd370 2023-10-16T11:22:29.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a9d7bc700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980b3e50 con 0x7f5a9806cb40 2023-10-16T11:22:29.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a9815a890 0x7f5a9815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a9815a890 0x7f5a9815c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34434/0 (socket says 172.21.15.181:34434) 2023-10-16T11:22:29.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 -- 172.21.15.181:0/728355683 learned_addr learned my addr 172.21.15.181:0/728355683 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:29.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a96ffd700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9806cb40 0x7f5a981f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a967fc700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a980cd370 0x7f5a980b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 -- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9806cb40 msgr2=0x7f5a981f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9806cb40 0x7f5a981f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 -- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a980cd370 msgr2=0x7f5a980b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a980cd370 0x7f5a980b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 -- 172.21.15.181:0/728355683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0094a0 con 0x7f5a9815a890 2023-10-16T11:22:29.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.214+0000 7f5a977fe700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a9815a890 0x7f5a9815c600 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 crypto rx=0x7f5a90002230 tx=0x7f5a9000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.215+0000 7f5a7f7fe700 1 -- 172.21.15.181:0/728355683 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000aac0 con 0x7f5a9815a890 2023-10-16T11:22:29.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.215+0000 7f5a7f7fe700 1 -- 172.21.15.181:0/728355683 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000ac40 con 0x7f5a9815a890 2023-10-16T11:22:29.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.215+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a980b4130 con 0x7f5a9815a890 2023-10-16T11:22:29.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.215+0000 7f5a7f7fe700 1 -- 172.21.15.181:0/728355683 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a900070d0 con 0x7f5a9815a890 2023-10-16T11:22:29.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.215+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a980b46f0 con 0x7f5a9815a890 2023-10-16T11:22:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.216+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a980b48e0 con 0x7f5a9815a890 2023-10-16T11:22:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.216+0000 7f5a7f7fe700 1 -- 172.21.15.181:0/728355683 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5a90007250 con 0x7f5a9815a890 2023-10-16T11:22:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.216+0000 7f5a7f7fe700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f5a8406b020 0x7f5a8406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.216+0000 7f5a7f7fe700 1 -- 172.21.15.181:0/728355683 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f5a900896f0 con 0x7f5a9815a890 2023-10-16T11:22:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.216+0000 7f5a96ffd700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f5a8406b020 0x7f5a8406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.217+0000 7f5a96ffd700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f5a8406b020 0x7f5a8406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009b40 tx=0x7f5a88009210 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.221+0000 7f5a7f7fe700 1 -- 172.21.15.181:0/728355683 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f5a900544b0 con 0x7f5a9815a890 2023-10-16T11:22:29.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.413+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a980cd7f0 con 0x7f5a8406b020 2023-10-16T11:22:29.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.413+0000 7f5a7f7fe700 1 -- 172.21.15.181:0/728355683 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5a88004020 con 0x7f5a8406b020 2023-10-16T11:22:29.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f5a8406b020 msgr2=0x7f5a8406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f5a8406b020 0x7f5a8406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009b40 tx=0x7f5a88009210 comp rx=0 tx=0).stop 2023-10-16T11:22:29.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a9815a890 msgr2=0x7f5a9815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a9815a890 0x7f5a9815c600 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 crypto rx=0x7f5a90002230 tx=0x7f5a9000cf50 comp rx=0 tx=0).stop 2023-10-16T11:22:29.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 shutdown_connections 2023-10-16T11:22:29.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f5a8406b020 0x7f5a8406d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a9806cb40 0x7f5a981f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a9815a890 0x7f5a9815c600 unknown :-1 s=CLOSED pgs=3267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 --2- 172.21.15.181:0/728355683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a980cd370 0x7f5a980b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 >> 172.21.15.181:0/728355683 conn(0x7f5a9806aed0 msgr2=0x7f5a98159930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 shutdown_connections 2023-10-16T11:22:29.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.415+0000 7f5a9d7bc700 1 -- 172.21.15.181:0/728355683 wait complete. 2023-10-16T11:22:29.420 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 2 to deep-scrub 2023-10-16T11:22:29.428 INFO:tasks.ceph:Scrubbing osd.3 2023-10-16T11:22:29.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:29.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.552+0000 7efd78cc7700 1 Processor -- start 2023-10-16T11:22:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.553+0000 7efd78cc7700 1 -- start start 2023-10-16T11:22:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.553+0000 7efd78cc7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd740d0fc0 0x7efd7406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.553+0000 7efd78cc7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efd740ce9f0 0x7efd7406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.553+0000 7efd78cc7700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd74162650 con 0x7efd740ce610 2023-10-16T11:22:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.553+0000 7efd78cc7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd74162190 con 0x7efd740d0fc0 2023-10-16T11:22:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.553+0000 7efd78cc7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd740d0230 con 0x7efd740ce9f0 2023-10-16T11:22:29.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd740d0fc0 0x7efd7406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd740d0fc0 0x7efd7406d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34440/0 (socket says 172.21.15.181:34440) 2023-10-16T11:22:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 -- 172.21.15.181:0/2386998750 learned_addr learned my addr 172.21.15.181:0/2386998750 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd72d9d700 1 --2- 172.21.15.181:0/2386998750 >> v2:172.21.15.190:3300/0 conn(0x7efd740ce9f0 0x7efd7406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 -- 172.21.15.181:0/2386998750 >> v2:172.21.15.190:3300/0 conn(0x7efd740ce9f0 msgr2=0x7efd7406a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 --2- 172.21.15.181:0/2386998750 >> v2:172.21.15.190:3300/0 conn(0x7efd740ce9f0 0x7efd7406a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 -- 172.21.15.181:0/2386998750 >> v1:172.21.15.190:6789/0 conn(0x7efd740ce610 legacy=0x7efd740cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 -- 172.21.15.181:0/2386998750 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74162370 con 0x7efd740d0fc0 2023-10-16T11:22:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd72d9d700 1 --2- 172.21.15.181:0/2386998750 >> v2:172.21.15.190:3300/0 conn(0x7efd740ce9f0 0x7efd7406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd71d9b700 1 --2- 172.21.15.181:0/2386998750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740d0fc0 0x7efd7406d660 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 crypto rx=0x7efd7406e7e0 tx=0x7efd64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763ca39d3542a58e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd7159a700 1 -- 172.21.15.181:0/2386998750 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6400b870 con 0x7efd740d0fc0 2023-10-16T11:22:29.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd7159a700 1 -- 172.21.15.181:0/2386998750 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6400b9d0 con 0x7efd740d0fc0 2023-10-16T11:22:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.554+0000 7efd7159a700 1 -- 172.21.15.181:0/2386998750 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6400bd60 con 0x7efd740d0fc0 2023-10-16T11:22:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 -- 172.21.15.181:0/2386998750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740d0fc0 msgr2=0x7efd7406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 --2- 172.21.15.181:0/2386998750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740d0fc0 0x7efd7406d660 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 crypto rx=0x7efd7406e7e0 tx=0x7efd64009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 -- 172.21.15.181:0/2386998750 shutdown_connections 2023-10-16T11:22:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 --2- 172.21.15.181:0/2386998750 >> v2:172.21.15.190:3300/0 conn(0x7efd740ce9f0 0x7efd7406a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 --2- 172.21.15.181:0/2386998750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740d0fc0 0x7efd7406d660 unknown :-1 s=CLOSED pgs=3268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 -- 172.21.15.181:0/2386998750 >> 172.21.15.181:0/2386998750 conn(0x7efd740d04a0 msgr2=0x7efd740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 -- 172.21.15.181:0/2386998750 shutdown_connections 2023-10-16T11:22:29.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.555+0000 7efd78cc7700 1 -- 172.21.15.181:0/2386998750 wait complete. 2023-10-16T11:22:29.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.556+0000 7efd78cc7700 1 Processor -- start 2023-10-16T11:22:29.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.556+0000 7efd78cc7700 1 -- start start 2023-10-16T11:22:29.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd78cc7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740ce610 0x7efd7409c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd78cc7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd740ce9f0 0x7efd741604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd78cc7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd740d0fc0 0x7efd741f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd78cc7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd74160020 con 0x7efd740ce610 2023-10-16T11:22:29.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd78cc7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7415f410 con 0x7efd740ce9f0 2023-10-16T11:22:29.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd78cc7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7415f6f0 con 0x7efd740d0fc0 2023-10-16T11:22:29.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740ce610 0x7efd7409c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740ce610 0x7efd7409c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34444/0 (socket says 172.21.15.181:34444) 2023-10-16T11:22:29.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 -- 172.21.15.181:0/3763019724 learned_addr learned my addr 172.21.15.181:0/3763019724 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:29.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd71d9b700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd740ce9f0 0x7efd741604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd72d9d700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd740d0fc0 0x7efd741f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 -- 172.21.15.181:0/3763019724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd740d0fc0 msgr2=0x7efd741f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd740d0fc0 0x7efd741f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 -- 172.21.15.181:0/3763019724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd740ce9f0 msgr2=0x7efd741604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd740ce9f0 0x7efd741604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 -- 172.21.15.181:0/3763019724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd64009550 con 0x7efd740ce610 2023-10-16T11:22:29.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd72d9d700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd740d0fc0 0x7efd741f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:29.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.557+0000 7efd7259c700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740ce610 0x7efd7409c590 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c4d0 tx=0x7efd6800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.558+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680052d0 con 0x7efd740ce610 2023-10-16T11:22:29.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.558+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7415f9d0 con 0x7efd740ce610 2023-10-16T11:22:29.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.558+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd68005430 con 0x7efd740ce610 2023-10-16T11:22:29.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.558+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680038f0 con 0x7efd740ce610 2023-10-16T11:22:29.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.558+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7409cad0 con 0x7efd740ce610 2023-10-16T11:22:29.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.559+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd7406f3d0 con 0x7efd740ce610 2023-10-16T11:22:29.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.559+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efd68003a50 con 0x7efd740ce610 2023-10-16T11:22:29.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.559+0000 7efd62ffd700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7efd5406b020 0x7efd5406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.559+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7efd68088db0 con 0x7efd740ce610 2023-10-16T11:22:29.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.559+0000 7efd71d9b700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7efd5406b020 0x7efd5406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.560+0000 7efd62ffd700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd54071a30 0x7efd54073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.560+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 --> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd540744e0 con 0x7efd54071a30 2023-10-16T11:22:29.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.560+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd6808b470 con 0x7efd740ce610 2023-10-16T11:22:29.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.560+0000 7efd72d9d700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd54071a30 0x7efd54073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.560+0000 7efd71d9b700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7efd5406b020 0x7efd5406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd740d13f0 tx=0x7efd64008030 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.560+0000 7efd72d9d700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd54071a30 0x7efd54073eb0 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.561+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== osd.3 v2:172.21.15.181:6817/2943330843 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7efd540744e0 con 0x7efd54071a30 2023-10-16T11:22:29.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 --> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efd7406f0f0 con 0x7efd54071a30 2023-10-16T11:22:29.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd62ffd700 1 -- 172.21.15.181:0/3763019724 <== osd.3 v2:172.21.15.181:6817/2943330843 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efd6c009bd0 con 0x7efd54071a30 2023-10-16T11:22:29.579 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:29.579 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:29.580 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd54071a30 msgr2=0x7efd54073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6817/2943330843,v1:172.21.15.181:6819/2943330843] conn(0x7efd54071a30 0x7efd54073eb0 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7efd5406b020 msgr2=0x7efd5406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7efd5406b020 0x7efd5406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd740d13f0 tx=0x7efd64008030 comp rx=0 tx=0).stop 2023-10-16T11:22:29.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740ce610 msgr2=0x7efd7409c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 --2- 172.21.15.181:0/3763019724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd740ce610 0x7efd7409c590 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c4d0 tx=0x7efd6800c7e0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd72d9d700 1 -- 172.21.15.181:0/3763019724 reap_dead start 2023-10-16T11:22:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 shutdown_connections 2023-10-16T11:22:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 >> 172.21.15.181:0/3763019724 conn(0x7efd740d04a0 msgr2=0x7efd740b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 shutdown_connections 2023-10-16T11:22:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.575+0000 7efd78cc7700 1 -- 172.21.15.181:0/3763019724 wait complete. 2023-10-16T11:22:29.588 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-16T11:22:29.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.708+0000 7f446bd31700 1 Processor -- start 2023-10-16T11:22:29.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.708+0000 7f446bd31700 1 -- start start 2023-10-16T11:22:29.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.709+0000 7f446bd31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f446415bd50 0x7f44640d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.709+0000 7f446bd31700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44640d24f0 con 0x7f44640b05d0 2023-10-16T11:22:29.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.709+0000 7f446bd31700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446415c320 con 0x7f44640cfc60 2023-10-16T11:22:29.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.709+0000 7f446bd31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446415ccc0 con 0x7f446415bd50 2023-10-16T11:22:29.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.709+0000 7f446a2ce700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f44640b05d0 0x7f44640cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45536/0 (socket says 172.21.15.181:45536) 2023-10-16T11:22:29.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.709+0000 7f446a2ce700 1 -- 172.21.15.181:0/3613271062 learned_addr learned my addr 172.21.15.181:0/3613271062 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:29.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.709+0000 7f44692cc700 1 --2- 172.21.15.181:0/3613271062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f446415bd50 0x7f44640d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f4468acb700 1 -- 172.21.15.181:0/3613271062 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204851467 0 0) 0x7f44640d24f0 con 0x7f44640b05d0 2023-10-16T11:22:29.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f4468acb700 1 -- 172.21.15.181:0/3613271062 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44500035b0 con 0x7f44640b05d0 2023-10-16T11:22:29.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f4468acb700 1 -- 172.21.15.181:0/3613271062 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447746089 0 0) 0x7f446415c320 con 0x7f44640cfc60 2023-10-16T11:22:29.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f4468acb700 1 -- 172.21.15.181:0/3613271062 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44640d24f0 con 0x7f44640cfc60 2023-10-16T11:22:29.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f44692cc700 1 -- 172.21.15.181:0/3613271062 >> v1:172.21.15.190:6789/0 conn(0x7f44640cfc60 legacy=0x7f44640d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f44692cc700 1 -- 172.21.15.181:0/3613271062 >> v1:172.21.15.181:6789/0 conn(0x7f44640b05d0 legacy=0x7f44640cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f44692cc700 1 -- 172.21.15.181:0/3613271062 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446415c600 con 0x7f446415bd50 2023-10-16T11:22:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f44692cc700 1 --2- 172.21.15.181:0/3613271062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f446415bd50 0x7f44640d1fb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4460009870 tx=0x7f4460009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ad744dceab08bb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f4468acb700 1 -- 172.21.15.181:0/3613271062 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460011830 con 0x7f446415bd50 2023-10-16T11:22:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f4468acb700 1 -- 172.21.15.181:0/3613271062 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4460011990 con 0x7f446415bd50 2023-10-16T11:22:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.710+0000 7f4468acb700 1 -- 172.21.15.181:0/3613271062 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460011d20 con 0x7f446415bd50 2023-10-16T11:22:29.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.711+0000 7f446bd31700 1 -- 172.21.15.181:0/3613271062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f446415bd50 msgr2=0x7f44640d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.711+0000 7f446bd31700 1 --2- 172.21.15.181:0/3613271062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f446415bd50 0x7f44640d1fb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4460009870 tx=0x7f4460009b80 comp rx=0 tx=0).stop 2023-10-16T11:22:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.711+0000 7f446bd31700 1 -- 172.21.15.181:0/3613271062 shutdown_connections 2023-10-16T11:22:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.711+0000 7f446bd31700 1 --2- 172.21.15.181:0/3613271062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f446415bd50 0x7f44640d1fb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.711+0000 7f446bd31700 1 -- 172.21.15.181:0/3613271062 >> 172.21.15.181:0/3613271062 conn(0x7f446409a9a0 msgr2=0x7f44640d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.711+0000 7f446bd31700 1 -- 172.21.15.181:0/3613271062 shutdown_connections 2023-10-16T11:22:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.711+0000 7f446bd31700 1 -- 172.21.15.181:0/3613271062 wait complete. 2023-10-16T11:22:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.712+0000 7f446bd31700 1 Processor -- start 2023-10-16T11:22:29.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.712+0000 7f446bd31700 1 -- start start 2023-10-16T11:22:29.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44640b05d0 0x7f446415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44640cfc60 0x7f44640ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f446415bd50 0x7f44640cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446409b5b0 con 0x7f44640b05d0 2023-10-16T11:22:29.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446415b060 con 0x7f446415bd50 2023-10-16T11:22:29.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b4360 con 0x7f44640cfc60 2023-10-16T11:22:29.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44640b05d0 0x7f446415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44640b05d0 0x7f446415ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34460/0 (socket says 172.21.15.181:34460) 2023-10-16T11:22:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 -- 172.21.15.181:0/3058975919 learned_addr learned my addr 172.21.15.181:0/3058975919 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:29.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446a2ce700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f446415bd50 0x7f44640cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f44692cc700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44640cfc60 0x7f44640ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 -- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44640cfc60 msgr2=0x7f44640ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44640cfc60 0x7f44640ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 -- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f446415bd50 msgr2=0x7f44640cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f446415bd50 0x7f44640cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 -- 172.21.15.181:0/3058975919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44600094a0 con 0x7f44640b05d0 2023-10-16T11:22:29.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f44692cc700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44640cfc60 0x7f44640ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f4469acd700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44640b05d0 0x7f446415ab20 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7f445400c520 tx=0x7f445400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f445a7fc700 1 -- 172.21.15.181:0/3058975919 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44540052d0 con 0x7f44640b05d0 2023-10-16T11:22:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f445a7fc700 1 -- 172.21.15.181:0/3058975919 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4454005430 con 0x7f44640b05d0 2023-10-16T11:22:29.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44640b4640 con 0x7f44640b05d0 2023-10-16T11:22:29.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f445a7fc700 1 -- 172.21.15.181:0/3058975919 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44540038a0 con 0x7f44640b05d0 2023-10-16T11:22:29.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.713+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44640b4c00 con 0x7f44640b05d0 2023-10-16T11:22:29.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.714+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4464067d20 con 0x7f44640b05d0 2023-10-16T11:22:29.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.715+0000 7f445a7fc700 1 -- 172.21.15.181:0/3058975919 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4454003a00 con 0x7f44640b05d0 2023-10-16T11:22:29.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.715+0000 7f445a7fc700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f444806b020 0x7f444806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:29.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.715+0000 7f44692cc700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f444806b020 0x7f444806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:29.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.715+0000 7f445a7fc700 1 -- 172.21.15.181:0/3058975919 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f4454088e60 con 0x7f44640b05d0 2023-10-16T11:22:29.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.716+0000 7f44692cc700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f444806b020 0x7f444806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f44600090b0 tx=0x7f446001c040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:29.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.720+0000 7f445a7fc700 1 -- 172.21.15.181:0/3058975919 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f4454053c20 con 0x7f44640b05d0 2023-10-16T11:22:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.911+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f446415c8e0 con 0x7f444806b020 2023-10-16T11:22:29.913 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.912+0000 7f445a7fc700 1 -- 172.21.15.181:0/3058975919 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4460011b80 con 0x7f444806b020 2023-10-16T11:22:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f444806b020 msgr2=0x7f444806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.915 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f444806b020 0x7f444806d4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f44600090b0 tx=0x7f446001c040 comp rx=0 tx=0).stop 2023-10-16T11:22:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44640b05d0 msgr2=0x7f446415ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44640b05d0 0x7f446415ab20 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7f445400c520 tx=0x7f445400c830 comp rx=0 tx=0).stop 2023-10-16T11:22:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 shutdown_connections 2023-10-16T11:22:29.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f444806b020 0x7f444806d4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f446415bd50 0x7f44640cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44640b05d0 0x7f446415ab20 unknown :-1 s=CLOSED pgs=3271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 --2- 172.21.15.181:0/3058975919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44640cfc60 0x7f44640ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:29.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 >> 172.21.15.181:0/3058975919 conn(0x7f446409a9a0 msgr2=0x7f44640af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 shutdown_connections 2023-10-16T11:22:29.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:29.914+0000 7f446bd31700 1 -- 172.21.15.181:0/3058975919 wait complete. 2023-10-16T11:22:29.919 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 3 to deep-scrub 2023-10-16T11:22:29.927 INFO:tasks.ceph:Scrubbing osd.4 2023-10-16T11:22:29.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:30.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.052+0000 7fafd474e700 1 Processor -- start 2023-10-16T11:22:30.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.052+0000 7fafd474e700 1 -- start start 2023-10-16T11:22:30.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd474e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fafcc0ce3a0 0x7fafcc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd474e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc0ce7e0 con 0x7fafcc0afa10 2023-10-16T11:22:30.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd474e700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc15b540 con 0x7fafcc0cd3f0 2023-10-16T11:22:30.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd474e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc0cd880 con 0x7fafcc0ce3a0 2023-10-16T11:22:30.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd1ce9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fafcc0ce3a0 0x7fafcc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd24ea700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fafcc0afa10 0x7fafcc06d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45546/0 (socket says 172.21.15.181:45546) 2023-10-16T11:22:30.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd24ea700 1 -- 172.21.15.181:0/1354962949 learned_addr learned my addr 172.21.15.181:0/1354962949 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:30.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd1ce9700 1 -- 172.21.15.181:0/1354962949 >> v1:172.21.15.190:6789/0 conn(0x7fafcc0cd3f0 legacy=0x7fafcc0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd1ce9700 1 -- 172.21.15.181:0/1354962949 >> v1:172.21.15.181:6789/0 conn(0x7fafcc0afa10 legacy=0x7fafcc06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd1ce9700 1 -- 172.21.15.181:0/1354962949 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc15b820 con 0x7fafcc0ce3a0 2023-10-16T11:22:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd1ce9700 1 --2- 172.21.15.181:0/1354962949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0ce3a0 0x7fafcc0cceb0 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009840 tx=0x7fafbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af36c3d12c0b4e54 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd14e8700 1 -- 172.21.15.181:0/1354962949 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc011830 con 0x7fafcc0ce3a0 2023-10-16T11:22:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd14e8700 1 -- 172.21.15.181:0/1354962949 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc011990 con 0x7fafcc0ce3a0 2023-10-16T11:22:30.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.053+0000 7fafd14e8700 1 -- 172.21.15.181:0/1354962949 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc011d20 con 0x7fafcc0ce3a0 2023-10-16T11:22:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.054+0000 7fafd474e700 1 -- 172.21.15.181:0/1354962949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0ce3a0 msgr2=0x7fafcc0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.054+0000 7fafd474e700 1 --2- 172.21.15.181:0/1354962949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0ce3a0 0x7fafcc0cceb0 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009840 tx=0x7fafbc009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:30.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.054+0000 7fafd474e700 1 -- 172.21.15.181:0/1354962949 shutdown_connections 2023-10-16T11:22:30.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.054+0000 7fafd474e700 1 --2- 172.21.15.181:0/1354962949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0ce3a0 0x7fafcc0cceb0 unknown :-1 s=CLOSED pgs=3273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.054+0000 7fafd474e700 1 -- 172.21.15.181:0/1354962949 >> 172.21.15.181:0/1354962949 conn(0x7fafcc0d04a0 msgr2=0x7fafcc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.054+0000 7fafd474e700 1 -- 172.21.15.181:0/1354962949 shutdown_connections 2023-10-16T11:22:30.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.054+0000 7fafd474e700 1 -- 172.21.15.181:0/1354962949 wait complete. 2023-10-16T11:22:30.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.055+0000 7fafd474e700 1 Processor -- start 2023-10-16T11:22:30.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.055+0000 7fafd474e700 1 -- start start 2023-10-16T11:22:30.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd474e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0afa10 0x7fafcc156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd474e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafcc0cd3f0 0x7fafcc15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd474e700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafcc0ce3a0 0x7fafcc0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd474e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c7570 con 0x7fafcc0afa10 2023-10-16T11:22:30.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd474e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c77c0 con 0x7fafcc0cd3f0 2023-10-16T11:22:30.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd474e700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c7aa0 con 0x7fafcc0ce3a0 2023-10-16T11:22:30.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0afa10 0x7fafcc156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0afa10 0x7fafcc156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34474/0 (socket says 172.21.15.181:34474) 2023-10-16T11:22:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 -- 172.21.15.181:0/2807614851 learned_addr learned my addr 172.21.15.181:0/2807614851 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd1ce9700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafcc0cd3f0 0x7fafcc15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd2ceb700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafcc0ce3a0 0x7fafcc0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 -- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafcc0ce3a0 msgr2=0x7fafcc0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafcc0ce3a0 0x7fafcc0b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 -- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafcc0cd3f0 msgr2=0x7fafcc15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafcc0cd3f0 0x7fafcc15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 -- 172.21.15.181:0/2807614851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0094a0 con 0x7fafcc0afa10 2023-10-16T11:22:30.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.056+0000 7fafd24ea700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0afa10 0x7fafcc156aa0 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c700 tx=0x7fafc800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.057+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80074e0 con 0x7fafcc0afa10 2023-10-16T11:22:30.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.057+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc8007640 con 0x7fafcc0afa10 2023-10-16T11:22:30.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.057+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc0c7d80 con 0x7fafcc0afa10 2023-10-16T11:22:30.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.057+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8005a30 con 0x7fafcc0afa10 2023-10-16T11:22:30.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.057+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc0c8340 con 0x7fafcc0afa10 2023-10-16T11:22:30.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.057+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fafcc15bde0 con 0x7fafcc0afa10 2023-10-16T11:22:30.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.058+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fafc8005b90 con 0x7fafcc0afa10 2023-10-16T11:22:30.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.058+0000 7fafc2ffd700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fafb006b020 0x7fafb006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fafc8089330 con 0x7fafcc0afa10 2023-10-16T11:22:30.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafd1ce9700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fafb006b020 0x7fafb006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafc2ffd700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7fafb0071a30 0x7fafb0073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 --> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafb00744e0 con 0x7fafb0071a30 2023-10-16T11:22:30.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc8089780 con 0x7fafcc0afa10 2023-10-16T11:22:30.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafd2ceb700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7fafb0071a30 0x7fafb0073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafd1ce9700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fafb006b020 0x7fafb006d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0090b0 tx=0x7fafbc01c040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.059+0000 7fafd2ceb700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7fafb0071a30 0x7fafb0073eb0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.060+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== osd.4 v2:172.21.15.190:6816/4008105976 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fafb00744e0 con 0x7fafb0071a30 2023-10-16T11:22:30.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.075+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 --> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fafcc15bb00 con 0x7fafb0071a30 2023-10-16T11:22:30.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.075+0000 7fafc2ffd700 1 -- 172.21.15.181:0/2807614851 <== osd.4 v2:172.21.15.190:6816/4008105976 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fafc4001fc0 con 0x7fafb0071a30 2023-10-16T11:22:30.078 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:30.078 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:30.078 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7fafb0071a30 msgr2=0x7fafb0073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.190:6816/4008105976,v1:172.21.15.190:6817/4008105976] conn(0x7fafb0071a30 0x7fafb0073eb0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fafb006b020 msgr2=0x7fafb006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fafb006b020 0x7fafb006d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0090b0 tx=0x7fafbc01c040 comp rx=0 tx=0).stop 2023-10-16T11:22:30.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0afa10 msgr2=0x7fafcc156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.081 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 --2- 172.21.15.181:0/2807614851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafcc0afa10 0x7fafcc156aa0 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c700 tx=0x7fafc800ca10 comp rx=0 tx=0).stop 2023-10-16T11:22:30.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd2ceb700 1 -- 172.21.15.181:0/2807614851 reap_dead start 2023-10-16T11:22:30.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 shutdown_connections 2023-10-16T11:22:30.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 >> 172.21.15.181:0/2807614851 conn(0x7fafcc0d04a0 msgr2=0x7fafcc0ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 shutdown_connections 2023-10-16T11:22:30.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.076+0000 7fafd474e700 1 -- 172.21.15.181:0/2807614851 wait complete. 2023-10-16T11:22:30.089 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-16T11:22:30.209 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.208+0000 7fa261ddd700 1 Processor -- start 2023-10-16T11:22:30.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.208+0000 7fa261ddd700 1 -- start start 2023-10-16T11:22:30.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa261ddd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa25c0ce4d0 0x7fa25c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa261ddd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa261ddd700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0cf460 con 0x7fa25c06d210 2023-10-16T11:22:30.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa261ddd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c0d0380 con 0x7fa25c0ce4d0 2023-10-16T11:22:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa261ddd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c06c700 con 0x7fa25c0cc4c0 2023-10-16T11:22:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa25affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa25c0ce4d0 0x7fa25c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.209+0000 7fa25b7fe700 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa25c06d210 0x7fa25c0cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:57736/0 (socket says 172.21.15.181:57736) 2023-10-16T11:22:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25affd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa25c0ce4d0 0x7fa25c0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:39536/0 (socket says 172.21.15.181:39536) 2023-10-16T11:22:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c15d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:59746/0 (socket says 172.21.15.181:59746) 2023-10-16T11:22:30.215 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25b7fe700 1 -- 172.21.15.181:0/5366087 learned_addr learned my addr 172.21.15.181:0/5366087 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25bfff700 1 -- 172.21.15.181:0/5366087 >> v2:172.21.15.190:3300/0 conn(0x7fa25c0ce4d0 msgr2=0x7fa25c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25bfff700 1 --2- 172.21.15.181:0/5366087 >> v2:172.21.15.190:3300/0 conn(0x7fa25c0ce4d0 0x7fa25c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25bfff700 1 -- 172.21.15.181:0/5366087 >> v1:172.21.15.190:6789/0 conn(0x7fa25c06d210 legacy=0x7fa25c0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25bfff700 1 -- 172.21.15.181:0/5366087 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c06d5f0 con 0x7fa25c0cc4c0 2023-10-16T11:22:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25a7fc700 1 -- 172.21.15.181:0/5366087 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325960787 0 0) 0x7fa25c0cf460 con 0x7fa25c06d210 2023-10-16T11:22:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25affd700 1 --2- 172.21.15.181:0/5366087 >> v2:172.21.15.190:3300/0 conn(0x7fa25c0ce4d0 0x7fa25c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25bfff700 1 --2- 172.21.15.181:0/5366087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c15d2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa254009d90 tx=0x7fa25400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32732c5a84fdd8b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25a7fc700 1 -- 172.21.15.181:0/5366087 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400e070 con 0x7fa25c0cc4c0 2023-10-16T11:22:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25a7fc700 1 -- 172.21.15.181:0/5366087 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400bb20 con 0x7fa25c0cc4c0 2023-10-16T11:22:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.210+0000 7fa25a7fc700 1 -- 172.21.15.181:0/5366087 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254012430 con 0x7fa25c0cc4c0 2023-10-16T11:22:30.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 -- 172.21.15.181:0/5366087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 msgr2=0x7fa25c15d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 --2- 172.21.15.181:0/5366087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c15d2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa254009d90 tx=0x7fa25400b330 comp rx=0 tx=0).stop 2023-10-16T11:22:30.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 -- 172.21.15.181:0/5366087 shutdown_connections 2023-10-16T11:22:30.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 --2- 172.21.15.181:0/5366087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c15d2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 --2- 172.21.15.181:0/5366087 >> v2:172.21.15.190:3300/0 conn(0x7fa25c0ce4d0 0x7fa25c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 -- 172.21.15.181:0/5366087 >> 172.21.15.181:0/5366087 conn(0x7fa25c09a9a0 msgr2=0x7fa25c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 -- 172.21.15.181:0/5366087 shutdown_connections 2023-10-16T11:22:30.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.211+0000 7fa261ddd700 1 -- 172.21.15.181:0/5366087 wait complete. 2023-10-16T11:22:30.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.212+0000 7fa261ddd700 1 Processor -- start 2023-10-16T11:22:30.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.212+0000 7fa261ddd700 1 -- start start 2023-10-16T11:22:30.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa261ddd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25c06d210 0x7fa25c15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa261ddd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa261ddd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa25c0ce4d0 0x7fa25c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa261ddd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c15c170 con 0x7fa25c06d210 2023-10-16T11:22:30.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa261ddd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0ca430 con 0x7fa25c0ce4d0 2023-10-16T11:22:30.225 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa261ddd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0c9b90 con 0x7fa25c0cc4c0 2023-10-16T11:22:30.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25c06d210 0x7fa25c15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa25c0ce4d0 0x7fa25c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25c06d210 0x7fa25c15bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34490/0 (socket says 172.21.15.181:34490) 2023-10-16T11:22:30.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25affd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:59750/0 (socket says 172.21.15.181:59750) 2023-10-16T11:22:30.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa25c0ce4d0 0x7fa25c0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:39544/0 (socket says 172.21.15.181:39544) 2023-10-16T11:22:30.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 -- 172.21.15.181:0/2450184911 learned_addr learned my addr 172.21.15.181:0/2450184911 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:30.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 -- 172.21.15.181:0/2450184911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 msgr2=0x7fa25c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 -- 172.21.15.181:0/2450184911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa25c0ce4d0 msgr2=0x7fa25c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa25c0ce4d0 0x7fa25c0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.213+0000 7fa25b7fe700 1 -- 172.21.15.181:0/2450184911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c0094a0 con 0x7fa25c06d210 2023-10-16T11:22:30.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.214+0000 7fa25b7fe700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25c06d210 0x7fa25c15bc30 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 crypto rx=0x7fa25000c550 tx=0x7fa25000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.214+0000 7fa243fff700 1 -- 172.21.15.181:0/2450184911 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250004c80 con 0x7fa25c06d210 2023-10-16T11:22:30.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.214+0000 7fa243fff700 1 -- 172.21.15.181:0/2450184911 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250004e00 con 0x7fa25c06d210 2023-10-16T11:22:30.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.214+0000 7fa243fff700 1 -- 172.21.15.181:0/2450184911 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250007310 con 0x7fa25c06d210 2023-10-16T11:22:30.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.214+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa254009a70 con 0x7fa25c06d210 2023-10-16T11:22:30.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.214+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c0ca150 con 0x7fa25c06d210 2023-10-16T11:22:30.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.215+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c067d20 con 0x7fa25c06d210 2023-10-16T11:22:30.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.215+0000 7fa243fff700 1 -- 172.21.15.181:0/2450184911 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa250007470 con 0x7fa25c06d210 2023-10-16T11:22:30.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.215+0000 7fa243fff700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fa24806b020 0x7fa24806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.216+0000 7fa243fff700 1 -- 172.21.15.181:0/2450184911 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fa2500894d0 con 0x7fa25c06d210 2023-10-16T11:22:30.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.216+0000 7fa25affd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fa24806b020 0x7fa24806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.216+0000 7fa25affd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fa24806b020 0x7fa24806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa25c06ba40 tx=0x7fa24c009210 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.220+0000 7fa243fff700 1 -- 172.21.15.181:0/2450184911 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fa250054320 con 0x7fa25c06d210 2023-10-16T11:22:30.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.413+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa25c0cf9e0 con 0x7fa24806b020 2023-10-16T11:22:30.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.413+0000 7fa243fff700 1 -- 172.21.15.181:0/2450184911 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa24c0099c0 con 0x7fa24806b020 2023-10-16T11:22:30.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.415+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fa24806b020 msgr2=0x7fa24806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.415+0000 7fa261ddd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fa24806b020 0x7fa24806d4e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa25c06ba40 tx=0x7fa24c009210 comp rx=0 tx=0).stop 2023-10-16T11:22:30.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.415+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25c06d210 msgr2=0x7fa25c15bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.415+0000 7fa261ddd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25c06d210 0x7fa25c15bc30 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 crypto rx=0x7fa25000c550 tx=0x7fa25000c860 comp rx=0 tx=0).stop 2023-10-16T11:22:30.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.415+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 shutdown_connections 2023-10-16T11:22:30.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.415+0000 7fa261ddd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fa24806b020 0x7fa24806d4e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.415+0000 7fa261ddd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa25c0ce4d0 0x7fa25c0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.416+0000 7fa261ddd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa25c06d210 0x7fa25c15bc30 unknown :-1 s=CLOSED pgs=3275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.416+0000 7fa261ddd700 1 --2- 172.21.15.181:0/2450184911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa25c0cc4c0 0x7fa25c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.416+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 >> 172.21.15.181:0/2450184911 conn(0x7fa25c09a9a0 msgr2=0x7fa25c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.416+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 shutdown_connections 2023-10-16T11:22:30.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.416+0000 7fa261ddd700 1 -- 172.21.15.181:0/2450184911 wait complete. 2023-10-16T11:22:30.421 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 4 to deep-scrub 2023-10-16T11:22:30.429 INFO:tasks.ceph:Scrubbing osd.5 2023-10-16T11:22:30.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.553+0000 7fb053105700 1 Processor -- start 2023-10-16T11:22:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.553+0000 7fb053105700 1 -- start start 2023-10-16T11:22:30.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb053105700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c0cc230 0x7fb04c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb053105700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb04c06e7a0 0x7fb04c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb053105700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c06b270 con 0x7fb04c0cd860 2023-10-16T11:22:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb053105700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c15a630 con 0x7fb04c06e7a0 2023-10-16T11:22:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb053105700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c06a940 con 0x7fb04c0cc230 2023-10-16T11:22:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb04bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb04c06e7a0 0x7fb04c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb04bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb04c06e7a0 0x7fb04c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34500/0 (socket says 172.21.15.181:34500) 2023-10-16T11:22:30.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb04bfff700 1 -- 172.21.15.181:0/433018395 learned_addr learned my addr 172.21.15.181:0/433018395 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:30.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb050ea1700 1 --2- 172.21.15.181:0/433018395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c0cc230 0x7fb04c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb04bfff700 1 -- 172.21.15.181:0/433018395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c0cc230 msgr2=0x7fb04c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:30.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb04bfff700 1 --2- 172.21.15.181:0/433018395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c0cc230 0x7fb04c06d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb04bfff700 1 -- 172.21.15.181:0/433018395 >> v1:172.21.15.190:6789/0 conn(0x7fb04c0cd860 legacy=0x7fb04c06b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.554+0000 7fb04bfff700 1 -- 172.21.15.181:0/433018395 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c06ac20 con 0x7fb04c06e7a0 2023-10-16T11:22:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb04bfff700 1 --2- 172.21.15.181:0/433018395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c06e7a0 0x7fb04c0ca2e0 secure :-1 s=READY pgs=3276 cs=0 l=1 rev1=1 crypto rx=0x7fb0380098f0 tx=0x7fb038009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f8739be6fbf0f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb04b7fe700 1 -- 172.21.15.181:0/433018395 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03800b8a0 con 0x7fb04c06e7a0 2023-10-16T11:22:30.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb04b7fe700 1 -- 172.21.15.181:0/433018395 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03800ba00 con 0x7fb04c06e7a0 2023-10-16T11:22:30.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb04b7fe700 1 -- 172.21.15.181:0/433018395 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03800bd90 con 0x7fb04c06e7a0 2023-10-16T11:22:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 -- 172.21.15.181:0/433018395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c06e7a0 msgr2=0x7fb04c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.567 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 --2- 172.21.15.181:0/433018395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c06e7a0 0x7fb04c0ca2e0 secure :-1 s=READY pgs=3276 cs=0 l=1 rev1=1 crypto rx=0x7fb0380098f0 tx=0x7fb038009c00 comp rx=0 tx=0).stop 2023-10-16T11:22:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 -- 172.21.15.181:0/433018395 shutdown_connections 2023-10-16T11:22:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 --2- 172.21.15.181:0/433018395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c0cc230 0x7fb04c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 --2- 172.21.15.181:0/433018395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c06e7a0 0x7fb04c0ca2e0 unknown :-1 s=CLOSED pgs=3276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 -- 172.21.15.181:0/433018395 >> 172.21.15.181:0/433018395 conn(0x7fb04c0b1780 msgr2=0x7fb04c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 -- 172.21.15.181:0/433018395 shutdown_connections 2023-10-16T11:22:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.555+0000 7fb053105700 1 -- 172.21.15.181:0/433018395 wait complete. 2023-10-16T11:22:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.556+0000 7fb053105700 1 Processor -- start 2023-10-16T11:22:30.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.556+0000 7fb053105700 1 -- start start 2023-10-16T11:22:30.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.557+0000 7fb053105700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c06e7a0 0x7fb04c154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.557+0000 7fb053105700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c0cc230 0x7fb04c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.557+0000 7fb053105700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04c0cd860 0x7fb04c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.557+0000 7fb053105700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c155600 con 0x7fb04c0cc230 2023-10-16T11:22:30.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.557+0000 7fb053105700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c156180 con 0x7fb04c0cd860 2023-10-16T11:22:30.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.557+0000 7fb053105700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c155850 con 0x7fb04c06e7a0 2023-10-16T11:22:30.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.557+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c0cc230 0x7fb04c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c0cc230 0x7fb04c0c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34506/0 (socket says 172.21.15.181:34506) 2023-10-16T11:22:30.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 -- 172.21.15.181:0/3425253068 learned_addr learned my addr 172.21.15.181:0/3425253068 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:30.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb050ea1700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c06e7a0 0x7fb04c154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb0516a2700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04c0cd860 0x7fb04c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 -- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c06e7a0 msgr2=0x7fb04c154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb04c06e7a0 0x7fb04c154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 -- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04c0cd860 msgr2=0x7fb04c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04c0cd860 0x7fb04c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 -- 172.21.15.181:0/3425253068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb038009550 con 0x7fb04c0cc230 2023-10-16T11:22:30.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb04bfff700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c0cc230 0x7fb04c0c9050 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 crypto rx=0x7fb038000c00 tx=0x7fb038004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038004510 con 0x7fb04c0cc230 2023-10-16T11:22:30.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03801e070 con 0x7fb04c0cc230 2023-10-16T11:22:30.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c155b30 con 0x7fb04c0cc230 2023-10-16T11:22:30.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0380038e0 con 0x7fb04c0cc230 2023-10-16T11:22:30.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.558+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c156610 con 0x7fb04c0cc230 2023-10-16T11:22:30.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.559+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb04c06dd60 con 0x7fb04c0cc230 2023-10-16T11:22:30.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb038003a40 con 0x7fb04c0cc230 2023-10-16T11:22:30.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb0497fa700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb03006b020 0x7fb03006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fb038020070 con 0x7fb04c0cc230 2023-10-16T11:22:30.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb050ea1700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb03006b020 0x7fb03006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb0497fa700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7fb030071a30 0x7fb030073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 --> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0300744e0 con 0x7fb030071a30 2023-10-16T11:22:30.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb03808f2c0 con 0x7fb04c0cc230 2023-10-16T11:22:30.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb0516a2700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7fb030071a30 0x7fb030073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.560+0000 7fb050ea1700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb03006b020 0x7fb03006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb040006b20 tx=0x7fb040008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.561+0000 7fb0516a2700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7fb030071a30 0x7fb030073eb0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.561+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== osd.5 v2:172.21.15.190:6808/646354310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb0300744e0 con 0x7fb030071a30 2023-10-16T11:22:30.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.576+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 --> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb04c06af00 con 0x7fb030071a30 2023-10-16T11:22:30.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb0497fa700 1 -- 172.21.15.181:0/3425253068 <== osd.5 v2:172.21.15.190:6808/646354310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb044001f20 con 0x7fb030071a30 2023-10-16T11:22:30.580 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:30.580 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:30.581 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:30.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7fb030071a30 msgr2=0x7fb030073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.190:6808/646354310,v1:172.21.15.190:6809/646354310] conn(0x7fb030071a30 0x7fb030073eb0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb03006b020 msgr2=0x7fb03006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb03006b020 0x7fb03006d4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb040006b20 tx=0x7fb040008040 comp rx=0 tx=0).stop 2023-10-16T11:22:30.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c0cc230 msgr2=0x7fb04c0c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 --2- 172.21.15.181:0/3425253068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb04c0cc230 0x7fb04c0c9050 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 crypto rx=0x7fb038000c00 tx=0x7fb038004750 comp rx=0 tx=0).stop 2023-10-16T11:22:30.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb0516a2700 1 -- 172.21.15.181:0/3425253068 reap_dead start 2023-10-16T11:22:30.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 shutdown_connections 2023-10-16T11:22:30.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 >> 172.21.15.181:0/3425253068 conn(0x7fb04c0b1780 msgr2=0x7fb04c06ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 shutdown_connections 2023-10-16T11:22:30.585 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.577+0000 7fb053105700 1 -- 172.21.15.181:0/3425253068 wait complete. 2023-10-16T11:22:30.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-16T11:22:30.711 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.710+0000 7f87169d0700 1 Processor -- start 2023-10-16T11:22:30.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.710+0000 7f87169d0700 1 -- start start 2023-10-16T11:22:30.714 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f87169d0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871006d210 0x7f87100cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f87169d0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87100cc4c0 0x7f87100cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f87169d0700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f87100ce300 0x7f871015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f87169d0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87100d0380 con 0x7f87100cc4c0 2023-10-16T11:22:30.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f87169d0700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87100cc940 con 0x7f87100ce300 2023-10-16T11:22:30.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f87169d0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87100ce790 con 0x7f871006d210 2023-10-16T11:22:30.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87100cc4c0 0x7f87100cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87100cc4c0 0x7f87100cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34514/0 (socket says 172.21.15.181:34514) 2023-10-16T11:22:30.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 -- 172.21.15.181:0/2863255961 learned_addr learned my addr 172.21.15.181:0/2863255961 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:30.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f8714f6d700 1 --2- 172.21.15.181:0/2863255961 >> v2:172.21.15.190:3300/0 conn(0x7f87100ce300 0x7f871015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870ffff700 1 --2- 172.21.15.181:0/2863255961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871006d210 0x7f87100cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 -- 172.21.15.181:0/2863255961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871006d210 msgr2=0x7f87100cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:30.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 --2- 172.21.15.181:0/2863255961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871006d210 0x7f87100cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 -- 172.21.15.181:0/2863255961 >> v2:172.21.15.190:3300/0 conn(0x7f87100ce300 msgr2=0x7f871015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:30.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 --2- 172.21.15.181:0/2863255961 >> v2:172.21.15.190:3300/0 conn(0x7f87100ce300 0x7f871015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 -- 172.21.15.181:0/2863255961 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100cf460 con 0x7f87100cc4c0 2023-10-16T11:22:30.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f8714f6d700 1 --2- 172.21.15.181:0/2863255961 >> v2:172.21.15.190:3300/0 conn(0x7f87100ce300 0x7f871015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:30.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.711+0000 7f870f7fe700 1 --2- 172.21.15.181:0/2863255961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100cc4c0 0x7f87100cddc0 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 crypto rx=0x7f871006dad0 tx=0x7f8700009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864a984b7d80fd9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f870effd700 1 -- 172.21.15.181:0/2863255961 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000b870 con 0x7f87100cc4c0 2023-10-16T11:22:30.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f870effd700 1 -- 172.21.15.181:0/2863255961 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000b9d0 con 0x7f87100cc4c0 2023-10-16T11:22:30.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f870effd700 1 -- 172.21.15.181:0/2863255961 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000bd60 con 0x7f87100cc4c0 2023-10-16T11:22:30.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 -- 172.21.15.181:0/2863255961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100cc4c0 msgr2=0x7f87100cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 --2- 172.21.15.181:0/2863255961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100cc4c0 0x7f87100cddc0 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 crypto rx=0x7f871006dad0 tx=0x7f8700009bd0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 -- 172.21.15.181:0/2863255961 shutdown_connections 2023-10-16T11:22:30.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 --2- 172.21.15.181:0/2863255961 >> v2:172.21.15.190:3300/0 conn(0x7f87100ce300 0x7f871015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 --2- 172.21.15.181:0/2863255961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f871006d210 0x7f87100cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 --2- 172.21.15.181:0/2863255961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100cc4c0 0x7f87100cddc0 unknown :-1 s=CLOSED pgs=3278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 -- 172.21.15.181:0/2863255961 >> 172.21.15.181:0/2863255961 conn(0x7f871009a9a0 msgr2=0x7f87100d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 -- 172.21.15.181:0/2863255961 shutdown_connections 2023-10-16T11:22:30.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.712+0000 7f87169d0700 1 -- 172.21.15.181:0/2863255961 wait complete. 2023-10-16T11:22:30.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.713+0000 7f87169d0700 1 Processor -- start 2023-10-16T11:22:30.724 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.714+0000 7f87169d0700 1 -- start start 2023-10-16T11:22:30.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.714+0000 7f87169d0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871006d210 0x7f871014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.714+0000 7f87169d0700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87100cc4c0 0x7f87100b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.714+0000 7f87169d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100ce300 0x7f87100ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.714+0000 7f87169d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b5d30 con 0x7f87100ce300 2023-10-16T11:22:30.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.714+0000 7f87169d0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b5a50 con 0x7f871006d210 2023-10-16T11:22:30.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.714+0000 7f87169d0700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87100c72f0 con 0x7f87100cc4c0 2023-10-16T11:22:30.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100ce300 0x7f87100ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100ce300 0x7f87100ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34522/0 (socket says 172.21.15.181:34522) 2023-10-16T11:22:30.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 -- 172.21.15.181:0/3752929588 learned_addr learned my addr 172.21.15.181:0/3752929588 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:30.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f870ffff700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871006d210 0x7f871014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f870f7fe700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87100cc4c0 0x7f87100b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 -- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87100cc4c0 msgr2=0x7f87100b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87100cc4c0 0x7f87100b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 -- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871006d210 msgr2=0x7f871014f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871006d210 0x7f871014f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 -- 172.21.15.181:0/3752929588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700009550 con 0x7f87100ce300 2023-10-16T11:22:30.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f8714f6d700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100ce300 0x7f87100ca700 secure :-1 s=READY pgs=3279 cs=0 l=1 rev1=1 crypto rx=0x7f870800cef0 tx=0x7f870800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f870cff9700 1 -- 172.21.15.181:0/3752929588 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870800ae50 con 0x7f87100ce300 2023-10-16T11:22:30.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f870cff9700 1 -- 172.21.15.181:0/3752929588 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708015440 con 0x7f87100ce300 2023-10-16T11:22:30.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100c75d0 con 0x7f87100ce300 2023-10-16T11:22:30.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f870cff9700 1 -- 172.21.15.181:0/3752929588 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87080157f0 con 0x7f87100ce300 2023-10-16T11:22:30.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.715+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871014f9a0 con 0x7f87100ce300 2023-10-16T11:22:30.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.716+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87100c7150 con 0x7f87100ce300 2023-10-16T11:22:30.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.717+0000 7f870cff9700 1 -- 172.21.15.181:0/3752929588 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f870800d430 con 0x7f87100ce300 2023-10-16T11:22:30.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.717+0000 7f870cff9700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86fc06b020 0x7f86fc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:30.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.717+0000 7f870cff9700 1 -- 172.21.15.181:0/3752929588 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f8708089fb0 con 0x7f87100ce300 2023-10-16T11:22:30.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.717+0000 7f870ffff700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86fc06b020 0x7f86fc06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:30.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.718+0000 7f870ffff700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86fc06b020 0x7f86fc06d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f870400a8d0 tx=0x7f8704008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:30.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.722+0000 7f870cff9700 1 -- 172.21.15.181:0/3752929588 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f87100c7150 con 0x7f87100ce300 2023-10-16T11:22:30.916 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.915+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f87100cf700 con 0x7f86fc06b020 2023-10-16T11:22:30.917 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.915+0000 7f870cff9700 1 -- 172.21.15.181:0/3752929588 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f870400c050 con 0x7f86fc06b020 2023-10-16T11:22:30.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86fc06b020 msgr2=0x7f86fc06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86fc06b020 0x7f86fc06d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f870400a8d0 tx=0x7f8704008040 comp rx=0 tx=0).stop 2023-10-16T11:22:30.919 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100ce300 msgr2=0x7f87100ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:30.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100ce300 0x7f87100ca700 secure :-1 s=READY pgs=3279 cs=0 l=1 rev1=1 crypto rx=0x7f870800cef0 tx=0x7f870800a340 comp rx=0 tx=0).stop 2023-10-16T11:22:30.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 shutdown_connections 2023-10-16T11:22:30.920 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f86fc06b020 0x7f86fc06d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87100ce300 0x7f87100ca700 unknown :-1 s=CLOSED pgs=3279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f871006d210 0x7f871014f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 --2- 172.21.15.181:0/3752929588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87100cc4c0 0x7f87100b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:30.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 >> 172.21.15.181:0/3752929588 conn(0x7f871009a9a0 msgr2=0x7f871006b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:30.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 shutdown_connections 2023-10-16T11:22:30.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:30.918+0000 7f87169d0700 1 -- 172.21.15.181:0/3752929588 wait complete. 2023-10-16T11:22:30.922 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 5 to deep-scrub 2023-10-16T11:22:30.931 INFO:tasks.ceph:Scrubbing osd.6 2023-10-16T11:22:30.931 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:31.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.054+0000 7f7407aee700 1 Processor -- start 2023-10-16T11:22:31.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.055+0000 7f7407aee700 1 -- start start 2023-10-16T11:22:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.055+0000 7f7407aee700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74000d0fc0 0x7f740006d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.055+0000 7f7407aee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74000ce9f0 0x7f740006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.055+0000 7f7407aee700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7400162650 con 0x7f74000ce610 2023-10-16T11:22:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.055+0000 7f7407aee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7400162190 con 0x7f74000ce9f0 2023-10-16T11:22:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.055+0000 7f7407aee700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74000d0230 con 0x7f74000d0fc0 2023-10-16T11:22:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74000ce9f0 0x7f740006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f74000ce9f0 0x7f740006a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34530/0 (socket says 172.21.15.181:34530) 2023-10-16T11:22:31.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 -- 172.21.15.181:0/2442074766 learned_addr learned my addr 172.21.15.181:0/2442074766 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:31.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f7405089700 1 --2- 172.21.15.181:0/2442074766 >> v2:172.21.15.190:3300/0 conn(0x7f74000d0fc0 0x7f740006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 -- 172.21.15.181:0/2442074766 >> v2:172.21.15.190:3300/0 conn(0x7f74000d0fc0 msgr2=0x7f740006d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 --2- 172.21.15.181:0/2442074766 >> v2:172.21.15.190:3300/0 conn(0x7f74000d0fc0 0x7f740006d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 -- 172.21.15.181:0/2442074766 >> v1:172.21.15.190:6789/0 conn(0x7f74000ce610 legacy=0x7f74000cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 -- 172.21.15.181:0/2442074766 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400162370 con 0x7f74000ce9f0 2023-10-16T11:22:31.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f7405089700 1 --2- 172.21.15.181:0/2442074766 >> v2:172.21.15.190:3300/0 conn(0x7f74000d0fc0 0x7f740006d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:31.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f740608b700 1 --2- 172.21.15.181:0/2442074766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000ce9f0 0x7f740006a8a0 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 crypto rx=0x7f74000c7520 tx=0x7f73f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa231733a306cb18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f7404888700 1 -- 172.21.15.181:0/2442074766 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f800e070 con 0x7f74000ce9f0 2023-10-16T11:22:31.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f7404888700 1 -- 172.21.15.181:0/2442074766 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f800b9c0 con 0x7f74000ce9f0 2023-10-16T11:22:31.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.056+0000 7f7404888700 1 -- 172.21.15.181:0/2442074766 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f800bd70 con 0x7f74000ce9f0 2023-10-16T11:22:31.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 -- 172.21.15.181:0/2442074766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000ce9f0 msgr2=0x7f740006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 --2- 172.21.15.181:0/2442074766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000ce9f0 0x7f740006a8a0 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 crypto rx=0x7f74000c7520 tx=0x7f73f800b330 comp rx=0 tx=0).stop 2023-10-16T11:22:31.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 -- 172.21.15.181:0/2442074766 shutdown_connections 2023-10-16T11:22:31.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 --2- 172.21.15.181:0/2442074766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000ce9f0 0x7f740006a8a0 unknown :-1 s=CLOSED pgs=3280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 --2- 172.21.15.181:0/2442074766 >> v2:172.21.15.190:3300/0 conn(0x7f74000d0fc0 0x7f740006d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 -- 172.21.15.181:0/2442074766 >> 172.21.15.181:0/2442074766 conn(0x7f74000d04a0 msgr2=0x7f74000d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 -- 172.21.15.181:0/2442074766 shutdown_connections 2023-10-16T11:22:31.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.057+0000 7f7407aee700 1 -- 172.21.15.181:0/2442074766 wait complete. 2023-10-16T11:22:31.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.058+0000 7f7407aee700 1 Processor -- start 2023-10-16T11:22:31.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.058+0000 7f7407aee700 1 -- start start 2023-10-16T11:22:31.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f7407aee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74000ce610 0x7f740015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f7407aee700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74000ce9f0 0x7f74001523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f7407aee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000d0fc0 0x7f74001608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f7407aee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74001613a0 con 0x7f74000d0fc0 2023-10-16T11:22:31.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f7407aee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7400160df0 con 0x7f74000ce610 2023-10-16T11:22:31.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f7407aee700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74001610c0 con 0x7f74000ce9f0 2023-10-16T11:22:31.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000d0fc0 0x7f74001608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000d0fc0 0x7f74001608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34536/0 (socket says 172.21.15.181:34536) 2023-10-16T11:22:31.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 -- 172.21.15.181:0/2262826172 learned_addr learned my addr 172.21.15.181:0/2262826172 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:31.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f7405089700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74000ce9f0 0x7f74001523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740588a700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74000ce610 0x7f740015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 -- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74000ce9f0 msgr2=0x7f74001523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.073 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74000ce9f0 0x7f74001523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 -- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74000ce610 msgr2=0x7f740015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74000ce610 0x7f740015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 -- 172.21.15.181:0/2262826172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f8009b30 con 0x7f74000d0fc0 2023-10-16T11:22:31.074 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f740608b700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000d0fc0 0x7f74001608b0 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f73f8012000 tx=0x7f73f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.059+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80099b0 con 0x7f74000d0fc0 2023-10-16T11:22:31.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.060+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f80092e0 con 0x7f74000d0fc0 2023-10-16T11:22:31.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.060+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740015e3e0 con 0x7f74000d0fc0 2023-10-16T11:22:31.075 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.060+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f801c640 con 0x7f74000d0fc0 2023-10-16T11:22:31.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.060+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740015e9a0 con 0x7f74000d0fc0 2023-10-16T11:22:31.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.060+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f740006f3d0 con 0x7f74000d0fc0 2023-10-16T11:22:31.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.061+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f73f801c7a0 con 0x7f74000d0fc0 2023-10-16T11:22:31.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.061+0000 7f73f67fc700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f73e406b020 0x7f73e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.061+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f73f808b3c0 con 0x7f74000d0fc0 2023-10-16T11:22:31.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.061+0000 7f740588a700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f73e406b020 0x7f73e406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.077 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.062+0000 7f73f67fc700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7f73e4071a30 0x7f73e4073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.062+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 --> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73e40744e0 con 0x7f73e4071a30 2023-10-16T11:22:31.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.062+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73f808da80 con 0x7f74000d0fc0 2023-10-16T11:22:31.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.062+0000 7f740588a700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f73e406b020 0x7f73e406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73fc000fb0 tx=0x7f73fc009040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.062+0000 7f7405089700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7f73e4071a30 0x7f73e4073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.062+0000 7f7405089700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7f73e4071a30 0x7f73e4073eb0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.063+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== osd.6 v2:172.21.15.190:6800/487839335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f73e40744e0 con 0x7f73e4071a30 2023-10-16T11:22:31.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.077+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 --> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f740006f0f0 con 0x7f73e4071a30 2023-10-16T11:22:31.080 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f73f67fc700 1 -- 172.21.15.181:0/2262826172 <== osd.6 v2:172.21.15.190:6800/487839335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f73f000c530 con 0x7f73e4071a30 2023-10-16T11:22:31.080 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:31.081 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:31.081 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:31.082 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7f73e4071a30 msgr2=0x7f73e4073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.190:6800/487839335,v1:172.21.15.190:6801/487839335] conn(0x7f73e4071a30 0x7f73e4073eb0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f73e406b020 msgr2=0x7f73e406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.083 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f73e406b020 0x7f73e406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73fc000fb0 tx=0x7f73fc009040 comp rx=0 tx=0).stop 2023-10-16T11:22:31.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000d0fc0 msgr2=0x7f74001608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 --2- 172.21.15.181:0/2262826172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74000d0fc0 0x7f74001608b0 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f73f8012000 tx=0x7f73f8004ad0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f740608b700 1 -- 172.21.15.181:0/2262826172 reap_dead start 2023-10-16T11:22:31.084 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 shutdown_connections 2023-10-16T11:22:31.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 >> 172.21.15.181:0/2262826172 conn(0x7f74000d04a0 msgr2=0x7f74000b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 shutdown_connections 2023-10-16T11:22:31.085 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.078+0000 7f7407aee700 1 -- 172.21.15.181:0/2262826172 wait complete. 2023-10-16T11:22:31.091 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-16T11:22:31.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.212+0000 7fedf9806700 1 Processor -- start 2023-10-16T11:22:31.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.212+0000 7fedf9806700 1 -- start start 2023-10-16T11:22:31.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.212+0000 7fedf9806700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fedf415bd50 0x7fedf40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.212+0000 7fedf9806700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.212+0000 7fedf9806700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf40d24f0 con 0x7fedf40cfc60 2023-10-16T11:22:31.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.212+0000 7fedf9806700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf415ccc0 con 0x7fedf415bd50 2023-10-16T11:22:31.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.212+0000 7fedf9806700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf415c320 con 0x7fedf40b05d0 2023-10-16T11:22:31.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf27fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fedf415bd50 0x7fedf40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf2ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fedf40cfc60 0x7fedf40d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45620/0 (socket says 172.21.15.181:45620) 2023-10-16T11:22:31.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf2ffd700 1 -- 172.21.15.181:0/891084677 learned_addr learned my addr 172.21.15.181:0/891084677 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:31.218 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf37fe700 1 --2- 172.21.15.181:0/891084677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf27fc700 1 -- 172.21.15.181:0/891084677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 msgr2=0x7fedf40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf27fc700 1 --2- 172.21.15.181:0/891084677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf40cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf27fc700 1 -- 172.21.15.181:0/891084677 >> v1:172.21.15.181:6789/0 conn(0x7fedf40cfc60 legacy=0x7fedf40d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf27fc700 1 -- 172.21.15.181:0/891084677 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf415c600 con 0x7fedf415bd50 2023-10-16T11:22:31.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf1ffb700 1 -- 172.21.15.181:0/891084677 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657509355 0 0) 0x7fedf40d24f0 con 0x7fedf40cfc60 2023-10-16T11:22:31.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf27fc700 1 --2- 172.21.15.181:0/891084677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40d1fb0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 crypto rx=0x7fede8009840 tx=0x7fede8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8c7825601950b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf1ffb700 1 -- 172.21.15.181:0/891084677 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8011830 con 0x7fedf415bd50 2023-10-16T11:22:31.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf1ffb700 1 -- 172.21.15.181:0/891084677 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede8011990 con 0x7fedf415bd50 2023-10-16T11:22:31.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.213+0000 7fedf1ffb700 1 -- 172.21.15.181:0/891084677 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8011d20 con 0x7fedf415bd50 2023-10-16T11:22:31.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 -- 172.21.15.181:0/891084677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 msgr2=0x7fedf40d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 --2- 172.21.15.181:0/891084677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40d1fb0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 crypto rx=0x7fede8009840 tx=0x7fede8009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:31.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 -- 172.21.15.181:0/891084677 shutdown_connections 2023-10-16T11:22:31.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 --2- 172.21.15.181:0/891084677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf40cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 --2- 172.21.15.181:0/891084677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40d1fb0 unknown :-1 s=CLOSED pgs=3283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 -- 172.21.15.181:0/891084677 >> 172.21.15.181:0/891084677 conn(0x7fedf409a9a0 msgr2=0x7fedf40d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 -- 172.21.15.181:0/891084677 shutdown_connections 2023-10-16T11:22:31.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.214+0000 7fedf9806700 1 -- 172.21.15.181:0/891084677 wait complete. 2023-10-16T11:22:31.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.215+0000 7fedf9806700 1 Processor -- start 2023-10-16T11:22:31.224 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.215+0000 7fedf9806700 1 -- start start 2023-10-16T11:22:31.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf9806700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf41f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.226 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf9806700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf40cfc60 0x7fedf4158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf9806700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf9806700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c95d0 con 0x7fedf415bd50 2023-10-16T11:22:31.227 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf9806700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf41f68e0 con 0x7fedf40cfc60 2023-10-16T11:22:31.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf9806700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c9010 con 0x7fedf40b05d0 2023-10-16T11:22:31.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34552/0 (socket says 172.21.15.181:34552) 2023-10-16T11:22:31.228 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 -- 172.21.15.181:0/2005771593 learned_addr learned my addr 172.21.15.181:0/2005771593 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:31.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf2ffd700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf41f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf27fc700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf40cfc60 0x7fedf4158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 -- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 msgr2=0x7fedf41f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf41f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 -- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf40cfc60 msgr2=0x7fedf4158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf40cfc60 0x7fedf4158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.216+0000 7fedf37fe700 1 -- 172.21.15.181:0/2005771593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80094a0 con 0x7fedf415bd50 2023-10-16T11:22:31.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.217+0000 7fedf37fe700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40c8750 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 crypto rx=0x7fedec00efc0 tx=0x7fedec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.217+0000 7feddbfff700 1 -- 172.21.15.181:0/2005771593 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00cd40 con 0x7fedf415bd50 2023-10-16T11:22:31.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.217+0000 7feddbfff700 1 -- 172.21.15.181:0/2005771593 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec00f040 con 0x7fedf415bd50 2023-10-16T11:22:31.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.217+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf40c92f0 con 0x7fedf415bd50 2023-10-16T11:22:31.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.217+0000 7feddbfff700 1 -- 172.21.15.181:0/2005771593 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0146c0 con 0x7fedf415bd50 2023-10-16T11:22:31.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.217+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf41f6e10 con 0x7fedf415bd50 2023-10-16T11:22:31.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.218+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf41f7000 con 0x7fedf415bd50 2023-10-16T11:22:31.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.218+0000 7feddbfff700 1 -- 172.21.15.181:0/2005771593 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fedec014820 con 0x7fedf415bd50 2023-10-16T11:22:31.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.219+0000 7feddbfff700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fede006b020 0x7fede006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.219+0000 7feddbfff700 1 -- 172.21.15.181:0/2005771593 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fedec08ab50 con 0x7fedf415bd50 2023-10-16T11:22:31.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.219+0000 7fedf2ffd700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fede006b020 0x7fede006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.219+0000 7fedf2ffd700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fede006b020 0x7fede006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fede4000f60 tx=0x7fede4007be0 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.234 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.223+0000 7feddbfff700 1 -- 172.21.15.181:0/2005771593 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fedec055800 con 0x7fedf415bd50 2023-10-16T11:22:31.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.414+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fedf415c8e0 con 0x7fede006b020 2023-10-16T11:22:31.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.414+0000 7feddbfff700 1 -- 172.21.15.181:0/2005771593 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fede4013420 con 0x7fede006b020 2023-10-16T11:22:31.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fede006b020 msgr2=0x7fede006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fede006b020 0x7fede006d4e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fede4000f60 tx=0x7fede4007be0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 msgr2=0x7fedf40c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40c8750 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 crypto rx=0x7fedec00efc0 tx=0x7fedec00c3a0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 shutdown_connections 2023-10-16T11:22:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fede006b020 0x7fede006d4e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedf40b05d0 0x7fedf41f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedf415bd50 0x7fedf40c8750 unknown :-1 s=CLOSED pgs=3284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 --2- 172.21.15.181:0/2005771593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedf40cfc60 0x7fedf4158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.416+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 >> 172.21.15.181:0/2005771593 conn(0x7fedf409a9a0 msgr2=0x7fedf415fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.417+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 shutdown_connections 2023-10-16T11:22:31.421 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.417+0000 7fedf9806700 1 -- 172.21.15.181:0/2005771593 wait complete. 2023-10-16T11:22:31.421 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 6 to deep-scrub 2023-10-16T11:22:31.429 INFO:tasks.ceph:Scrubbing osd.7 2023-10-16T11:22:31.430 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-16T11:22:31.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.554+0000 7fe55c276700 1 Processor -- start 2023-10-16T11:22:31.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.554+0000 7fe55c276700 1 -- start start 2023-10-16T11:22:31.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe55c276700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5540caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe55c276700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5540cf5c0 0x7fe55406ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe55c276700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe554160240 con 0x7fe5540cc4a0 2023-10-16T11:22:31.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe55c276700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55415bd80 con 0x7fe5540cf5c0 2023-10-16T11:22:31.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe55c276700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55406b6d0 con 0x7fe55406a8a0 2023-10-16T11:22:31.559 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5540cf5c0 0x7fe55406ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5540cf5c0 0x7fe55406ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34558/0 (socket says 172.21.15.181:34558) 2023-10-16T11:22:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 -- 172.21.15.181:0/2088922719 learned_addr learned my addr 172.21.15.181:0/2088922719 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 -- 172.21.15.181:0/2088922719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 msgr2=0x7fe5540caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe55a012700 1 --2- 172.21.15.181:0/2088922719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5540caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 --2- 172.21.15.181:0/2088922719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5540caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 -- 172.21.15.181:0/2088922719 >> v1:172.21.15.190:6789/0 conn(0x7fe5540cc4a0 legacy=0x7fe55415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 -- 172.21.15.181:0/2088922719 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55406b1c0 con 0x7fe5540cf5c0 2023-10-16T11:22:31.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe55a012700 1 --2- 172.21.15.181:0/2088922719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5540caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.555+0000 7fe559811700 1 --2- 172.21.15.181:0/2088922719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe55406ac80 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 crypto rx=0x7fe5440098f0 tx=0x7fe544009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91e1d20cb9fce26 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe559010700 1 -- 172.21.15.181:0/2088922719 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544004230 con 0x7fe5540cf5c0 2023-10-16T11:22:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe559010700 1 -- 172.21.15.181:0/2088922719 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe544004390 con 0x7fe5540cf5c0 2023-10-16T11:22:31.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe559010700 1 -- 172.21.15.181:0/2088922719 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5440036a0 con 0x7fe5540cf5c0 2023-10-16T11:22:31.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 -- 172.21.15.181:0/2088922719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 msgr2=0x7fe55406ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 --2- 172.21.15.181:0/2088922719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe55406ac80 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 crypto rx=0x7fe5440098f0 tx=0x7fe544009c00 comp rx=0 tx=0).stop 2023-10-16T11:22:31.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 -- 172.21.15.181:0/2088922719 shutdown_connections 2023-10-16T11:22:31.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 --2- 172.21.15.181:0/2088922719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5540caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 --2- 172.21.15.181:0/2088922719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe55406ac80 unknown :-1 s=CLOSED pgs=3285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.564 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 -- 172.21.15.181:0/2088922719 >> 172.21.15.181:0/2088922719 conn(0x7fe5540d04a0 msgr2=0x7fe5540d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 -- 172.21.15.181:0/2088922719 shutdown_connections 2023-10-16T11:22:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.556+0000 7fe55c276700 1 -- 172.21.15.181:0/2088922719 wait complete. 2023-10-16T11:22:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.557+0000 7fe55c276700 1 Processor -- start 2023-10-16T11:22:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55c276700 1 -- start start 2023-10-16T11:22:31.568 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55c276700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5541f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55c276700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5540cc4a0 0x7fe55406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55c276700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe554155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.569 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55c276700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55406f730 con 0x7fe5540cf5c0 2023-10-16T11:22:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55c276700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5541f2eb0 con 0x7fe5540cc4a0 2023-10-16T11:22:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55c276700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55406f500 con 0x7fe55406a8a0 2023-10-16T11:22:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55a813700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe554155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55a012700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5541f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55a813700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe554155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34566/0 (socket says 172.21.15.181:34566) 2023-10-16T11:22:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe55a813700 1 -- 172.21.15.181:0/4002273640 learned_addr learned my addr 172.21.15.181:0/4002273640 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:31.571 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.558+0000 7fe559811700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5540cc4a0 0x7fe55406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55a813700 1 -- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 msgr2=0x7fe5541f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55a813700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe55406a8a0 0x7fe5541f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55a813700 1 -- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5540cc4a0 msgr2=0x7fe55406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.572 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55a813700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5540cc4a0 0x7fe55406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55a813700 1 -- 172.21.15.181:0/4002273640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544009550 con 0x7fe5540cf5c0 2023-10-16T11:22:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55a813700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe554155880 secure :-1 s=READY pgs=3286 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00cc10 tx=0x7fe54c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00aa60 con 0x7fe5540cf5c0 2023-10-16T11:22:31.573 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c00abc0 con 0x7fe5540cf5c0 2023-10-16T11:22:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55406d800 con 0x7fe5540cf5c0 2023-10-16T11:22:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c012420 con 0x7fe5540cf5c0 2023-10-16T11:22:31.574 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.559+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55406ddc0 con 0x7fe5540cf5c0 2023-10-16T11:22:31.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.560+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe55415ff60 con 0x7fe5540cf5c0 2023-10-16T11:22:31.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.560+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe54c012580 con 0x7fe5540cf5c0 2023-10-16T11:22:31.575 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe54affd700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe53806b020 0x7fe53806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe55a012700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe53806b020 0x7fe53806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fe54c089b30 con 0x7fe5540cf5c0 2023-10-16T11:22:31.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe54affd700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fe538071a30 0x7fe538073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.576 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 --> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5380744e0 con 0x7fe538071a30 2023-10-16T11:22:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe54c08e2c0 con 0x7fe5540cf5c0 2023-10-16T11:22:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe55a012700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe53806b020 0x7fe53806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe55000abe0 tx=0x7fe550008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.561+0000 7fe559811700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fe538071a30 0x7fe538073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.577 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.562+0000 7fe559811700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fe538071a30 0x7fe538073eb0 crc :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.562+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== osd.7 v2:172.21.15.190:6818/4234176418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe5380744e0 con 0x7fe538071a30 2023-10-16T11:22:31.578 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.577+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 --> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe5540d0290 con 0x7fe538071a30 2023-10-16T11:22:31.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe54affd700 1 -- 172.21.15.181:0/4002273640 <== osd.7 v2:172.21.15.190:6818/4234176418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe5440045e0 con 0x7fe538071a30 2023-10-16T11:22:31.579 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-10-16T11:22:31.580 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T11:22:31.580 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-10-16T11:22:31.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fe538071a30 msgr2=0x7fe538073eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.581 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.190:6818/4234176418,v1:172.21.15.190:6820/4234176418] conn(0x7fe538071a30 0x7fe538073eb0 crc :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe53806b020 msgr2=0x7fe53806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe53806b020 0x7fe53806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe55000abe0 tx=0x7fe550008040 comp rx=0 tx=0).stop 2023-10-16T11:22:31.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 msgr2=0x7fe554155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 --2- 172.21.15.181:0/4002273640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5540cf5c0 0x7fe554155880 secure :-1 s=READY pgs=3286 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00cc10 tx=0x7fe54c00cf20 comp rx=0 tx=0).stop 2023-10-16T11:22:31.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55a813700 1 -- 172.21.15.181:0/4002273640 reap_dead start 2023-10-16T11:22:31.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 shutdown_connections 2023-10-16T11:22:31.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 >> 172.21.15.181:0/4002273640 conn(0x7fe5540d04a0 msgr2=0x7fe554158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 shutdown_connections 2023-10-16T11:22:31.584 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.578+0000 7fe55c276700 1 -- 172.21.15.181:0/4002273640 wait complete. 2023-10-16T11:22:31.591 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-16T11:22:31.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.711+0000 7f82d00e5700 1 Processor -- start 2023-10-16T11:22:31.712 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.711+0000 7f82d00e5700 1 -- start start 2023-10-16T11:22:31.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82d00e5700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82c80ce4d0 0x7f82c80cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82d00e5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80cc4c0 0x7f82c815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.715 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82d00e5700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c80cf460 con 0x7f82c806d210 2023-10-16T11:22:31.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82d00e5700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c80d0380 con 0x7f82c80ce4d0 2023-10-16T11:22:31.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82d00e5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c806c700 con 0x7f82c80cc4c0 2023-10-16T11:22:31.716 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cd680700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82c80ce4d0 0x7f82c80cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82ce682700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80cc4c0 0x7f82c815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82ce682700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80cc4c0 0x7f82c815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.181:59834/0 (socket says 172.21.15.181:59834) 2023-10-16T11:22:31.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cd680700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82c80ce4d0 0x7f82c80cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.181:39622/0 (socket says 172.21.15.181:39622) 2023-10-16T11:22:31.717 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82ce682700 1 -- 172.21.15.181:0/887137152 learned_addr learned my addr 172.21.15.181:0/887137152 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:31.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cde81700 1 --1- 172.21.15.181:0/887137152 >> v1:172.21.15.190:6789/0 conn(0x7f82c806d210 0x7f82c80cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.181:57824/0 (socket says 172.21.15.181:57824) 2023-10-16T11:22:31.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cd680700 1 -- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80cc4c0 msgr2=0x7f82c815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:31.718 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cd680700 1 --2- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80cc4c0 0x7f82c815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cd680700 1 -- 172.21.15.181:0/887137152 >> v1:172.21.15.190:6789/0 conn(0x7f82c806d210 legacy=0x7f82c80cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cd680700 1 -- 172.21.15.181:0/887137152 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c806d5f0 con 0x7f82c80ce4d0 2023-10-16T11:22:31.719 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82ce682700 1 --2- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80cc4c0 0x7f82c815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:31.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.712+0000 7f82cd680700 1 --2- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c80ce4d0 0x7f82c80cd4b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009840 tx=0x7f82c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=273d82f7d1cd2d97 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82cce7f700 1 -- 172.21.15.181:0/887137152 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4011830 con 0x7f82c80ce4d0 2023-10-16T11:22:31.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82cce7f700 1 -- 172.21.15.181:0/887137152 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4011990 con 0x7f82c80ce4d0 2023-10-16T11:22:31.720 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82cce7f700 1 -- 172.21.15.181:0/887137152 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4011d20 con 0x7f82c80ce4d0 2023-10-16T11:22:31.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 -- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c80ce4d0 msgr2=0x7f82c80cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 --2- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c80ce4d0 0x7f82c80cd4b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009840 tx=0x7f82c4009b50 comp rx=0 tx=0).stop 2023-10-16T11:22:31.721 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 -- 172.21.15.181:0/887137152 shutdown_connections 2023-10-16T11:22:31.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 --2- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80cc4c0 0x7f82c815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 --2- 172.21.15.181:0/887137152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c80ce4d0 0x7f82c80cd4b0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 -- 172.21.15.181:0/887137152 >> 172.21.15.181:0/887137152 conn(0x7f82c809a9a0 msgr2=0x7f82c80d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.722 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 -- 172.21.15.181:0/887137152 shutdown_connections 2023-10-16T11:22:31.723 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.713+0000 7f82d00e5700 1 -- 172.21.15.181:0/887137152 wait complete. 2023-10-16T11:22:31.725 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.714+0000 7f82d00e5700 1 Processor -- start 2023-10-16T11:22:31.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.714+0000 7f82d00e5700 1 -- start start 2023-10-16T11:22:31.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82d00e5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c806d210 0x7f82c814eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.726 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82d00e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80cc4c0 0x7f82c80ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82d00e5700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80ce4d0 0x7f82c80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82d00e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c81525a0 con 0x7f82c80cc4c0 2023-10-16T11:22:31.727 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82d00e5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80c5350 con 0x7f82c806d210 2023-10-16T11:22:31.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82d00e5700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80c5630 con 0x7f82c80ce4d0 2023-10-16T11:22:31.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82cd680700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80cc4c0 0x7f82c80ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82cd680700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80cc4c0 0x7f82c80ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34580/0 (socket says 172.21.15.181:34580) 2023-10-16T11:22:31.728 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82cd680700 1 -- 172.21.15.181:0/25965114 learned_addr learned my addr 172.21.15.181:0/25965114 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:31.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82cde81700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c806d210 0x7f82c814eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.715+0000 7f82ce682700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80ce4d0 0x7f82c80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.729 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82cd680700 1 -- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80ce4d0 msgr2=0x7f82c80ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82cd680700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80ce4d0 0x7f82c80ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82cd680700 1 -- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c806d210 msgr2=0x7f82c814eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.730 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82cd680700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c806d210 0x7f82c814eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82cd680700 1 -- 172.21.15.181:0/25965114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0009b30 con 0x7f82c80cc4c0 2023-10-16T11:22:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82cde81700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c806d210 0x7f82c814eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T11:22:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82cd680700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80cc4c0 0x7f82c80ae290 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009b90 tx=0x7f82c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.731 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82be7fc700 1 -- 172.21.15.181:0/25965114 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004590 con 0x7f82c80cc4c0 2023-10-16T11:22:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82be7fc700 1 -- 172.21.15.181:0/25965114 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c40036a0 con 0x7f82c80cc4c0 2023-10-16T11:22:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c40094a0 con 0x7f82c80cc4c0 2023-10-16T11:22:31.732 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82be7fc700 1 -- 172.21.15.181:0/25965114 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4003a30 con 0x7f82c80cc4c0 2023-10-16T11:22:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.716+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c80c5bf0 con 0x7f82c80cc4c0 2023-10-16T11:22:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.717+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c80c50a0 con 0x7f82c80cc4c0 2023-10-16T11:22:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.717+0000 7f82be7fc700 1 -- 172.21.15.181:0/25965114 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f82c4003b90 con 0x7f82c80cc4c0 2023-10-16T11:22:31.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.718+0000 7f82be7fc700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f82ac06b020 0x7f82ac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:31.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.718+0000 7f82be7fc700 1 -- 172.21.15.181:0/25965114 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f82c408ac60 con 0x7f82c80cc4c0 2023-10-16T11:22:31.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.718+0000 7f82cde81700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f82ac06b020 0x7f82ac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:31.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.718+0000 7f82cde81700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f82ac06b020 0x7f82ac06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82b8000f60 tx=0x7f82b8007ed0 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:31.735 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.722+0000 7f82be7fc700 1 -- 172.21.15.181:0/25965114 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f82c4055b50 con 0x7f82c80cc4c0 2023-10-16T11:22:31.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.917+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f82c80c54c0 con 0x7f82ac06b020 2023-10-16T11:22:31.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.917+0000 7f82be7fc700 1 -- 172.21.15.181:0/25965114 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f82c80c54c0 con 0x7f82ac06b020 2023-10-16T11:22:31.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f82ac06b020 msgr2=0x7f82ac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f82ac06b020 0x7f82ac06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82b8000f60 tx=0x7f82b8007ed0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80cc4c0 msgr2=0x7f82c80ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:31.921 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80cc4c0 0x7f82c80ae290 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009b90 tx=0x7f82c4004750 comp rx=0 tx=0).stop 2023-10-16T11:22:31.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 shutdown_connections 2023-10-16T11:22:31.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f82ac06b020 0x7f82ac06d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.922 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82c80ce4d0 0x7f82c80ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c806d210 0x7f82c814eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 --2- 172.21.15.181:0/25965114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c80cc4c0 0x7f82c80ae290 unknown :-1 s=CLOSED pgs=3287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:31.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 >> 172.21.15.181:0/25965114 conn(0x7f82c809a9a0 msgr2=0x7f82c80706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:31.923 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 shutdown_connections 2023-10-16T11:22:31.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:31.919+0000 7f82d00e5700 1 -- 172.21.15.181:0/25965114 wait complete. 2023-10-16T11:22:31.924 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 7 to deep-scrub 2023-10-16T11:22:31.932 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:32.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.051+0000 7fe0b6544700 1 Processor -- start 2023-10-16T11:22:32.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0b6544700 1 -- start start 2023-10-16T11:22:32.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0b6544700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0b00cd2d0 0x7fe0b00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0b6544700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00cc510 con 0x7fe0b015b990 2023-10-16T11:22:32.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0b6544700 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00cc7f0 con 0x7fe0b00d0370 2023-10-16T11:22:32.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0b6544700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b00ccad0 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0b4ae1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0b00cd2d0 0x7fe0b00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0affff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe0b015b990 0x7fe0b00cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45664/0 (socket says 172.21.15.181:45664) 2023-10-16T11:22:32.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.052+0000 7fe0affff700 1 -- 172.21.15.181:0/4096359910 learned_addr learned my addr 172.21.15.181:0/4096359910 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:32.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0b4ae1700 1 -- 172.21.15.181:0/4096359910 >> v1:172.21.15.190:6789/0 conn(0x7fe0b00d0370 legacy=0x7fe0b00ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:32.057 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0b4ae1700 1 -- 172.21.15.181:0/4096359910 >> v1:172.21.15.181:6789/0 conn(0x7fe0b015b990 legacy=0x7fe0b00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0b4ae1700 1 -- 172.21.15.181:0/4096359910 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00ccdb0 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0b4ae1700 1 --2- 172.21.15.181:0/4096359910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b00ce8f0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009d90 tx=0x7fe0a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38634eeb1e19db5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0aeffd700 1 -- 172.21.15.181:0/4096359910 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a800e070 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0aeffd700 1 -- 172.21.15.181:0/4096359910 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a800bb90 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0aeffd700 1 -- 172.21.15.181:0/4096359910 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8003a20 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0b6544700 1 -- 172.21.15.181:0/4096359910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 msgr2=0x7fe0b00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.053+0000 7fe0b6544700 1 --2- 172.21.15.181:0/4096359910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b00ce8f0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009d90 tx=0x7fe0a800b330 comp rx=0 tx=0).stop 2023-10-16T11:22:32.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.054+0000 7fe0b6544700 1 -- 172.21.15.181:0/4096359910 shutdown_connections 2023-10-16T11:22:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.054+0000 7fe0b6544700 1 --2- 172.21.15.181:0/4096359910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b00ce8f0 unknown :-1 s=CLOSED pgs=3289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.054+0000 7fe0b6544700 1 -- 172.21.15.181:0/4096359910 >> 172.21.15.181:0/4096359910 conn(0x7fe0b009a9a0 msgr2=0x7fe0b00d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.054+0000 7fe0b6544700 1 -- 172.21.15.181:0/4096359910 shutdown_connections 2023-10-16T11:22:32.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.054+0000 7fe0b6544700 1 -- 172.21.15.181:0/4096359910 wait complete. 2023-10-16T11:22:32.063 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.055+0000 7fe0b6544700 1 Processor -- start 2023-10-16T11:22:32.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.055+0000 7fe0b6544700 1 -- start start 2023-10-16T11:22:32.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b0157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.064 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0b00d0370 0x7fe0b0155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0b015b990 0x7fe0b0155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0158870 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0157fd0 con 0x7fe0b00d0370 2023-10-16T11:22:32.065 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b01582b0 con 0x7fe0b015b990 2023-10-16T11:22:32.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b0157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b0157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34592/0 (socket says 172.21.15.181:34592) 2023-10-16T11:22:32.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 -- 172.21.15.181:0/2766576306 learned_addr learned my addr 172.21.15.181:0/2766576306 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:32.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b4ae1700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0b015b990 0x7fe0b0155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0af7fe700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0b00d0370 0x7fe0b0155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 -- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0b015b990 msgr2=0x7fe0b0155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.067 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0b015b990 0x7fe0b0155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 -- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0b00d0370 msgr2=0x7fe0b0155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0b00d0370 0x7fe0b0155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.068 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 -- 172.21.15.181:0/2766576306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a8009a70 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b4ae1700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0b015b990 0x7fe0b0155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:32.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0af7fe700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0b00d0370 0x7fe0b0155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T11:22:32.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0affff700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b0157680 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c4d0 tx=0x7fe0a000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0acff9700 1 -- 172.21.15.181:0/2766576306 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00074e0 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0acff9700 1 -- 172.21.15.181:0/2766576306 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a0007640 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0acff9700 1 -- 172.21.15.181:0/2766576306 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0005a30 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b0158590 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.070 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.056+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b0157bc0 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.057+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b00ae600 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.058+0000 7fe0acff9700 1 -- 172.21.15.181:0/2766576306 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe0a0005b90 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.071 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.058+0000 7fe0acff9700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe09c06b020 0x7fe09c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:32.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.059+0000 7fe0acff9700 1 -- 172.21.15.181:0/2766576306 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fe0a0088f60 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.059+0000 7fe0af7fe700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe09c06b020 0x7fe09c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:32.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.059+0000 7fe0af7fe700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe09c06b020 0x7fe09c06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4006fd0 tx=0x7fe0a4008040 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:32.072 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.062+0000 7fe0acff9700 1 -- 172.21.15.181:0/2766576306 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fe0a0053e40 con 0x7fe0b00cd2d0 2023-10-16T11:22:32.253 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.252+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0b00cc200 con 0x7fe09c06b020 2023-10-16T11:22:32.255 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.255+0000 7fe0acff9700 1 -- 172.21.15.181:0/2766576306 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50918 (secure 0 0 0) 0x7fe0a400ad30 con 0x7fe09c06b020 2023-10-16T11:22:32.257 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:32.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe09c06b020 msgr2=0x7fe09c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.258 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe09c06b020 0x7fe09c06d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4006fd0 tx=0x7fe0a4008040 comp rx=0 tx=0).stop 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 msgr2=0x7fe0b0157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b0157680 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c4d0 tx=0x7fe0a000c890 comp rx=0 tx=0).stop 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 shutdown_connections 2023-10-16T11:22:32.259 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fe09c06b020 0x7fe09c06d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0b015b990 0x7fe0b0155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0b00cd2d0 0x7fe0b0157680 unknown :-1 s=CLOSED pgs=3290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.260 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 --2- 172.21.15.181:0/2766576306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0b00d0370 0x7fe0b0155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 >> 172.21.15.181:0/2766576306 conn(0x7fe0b009a9a0 msgr2=0x7fe0b00ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 shutdown_connections 2023-10-16T11:22:32.261 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:32.257+0000 7fe0b6544700 1 -- 172.21.15.181:0/2766576306 wait complete. 2023-10-16T11:22:32.262 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:22:32.270 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-10-16T11:22:31.838975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":501140,"kb_used_data":28264,"kb_used_omap":12,"kb_used_meta":472755,"kb_avail":754473580,"statfs":{"total":773094113280,"available":772580945920,"internally_reserved":0,"allocated":28942336,"data_stored":15606920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":484101688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001776"},"pg_stats":[{"pgid":"1.0","version":"272'414","reported_seq":1500,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-16T11:22:22.403405+0000","last_change":"2023-10-16T09:43:57.598064+0000","last_active":"2023-10-16T11:22:22.403405+0000","last_peered":"2023-10-16T11:22:22.403405+0000","last_clean":"2023-10-16T11:22:22.403405+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:22:22.403405+0000","last_undegraded":"2023-10-16T11:22:22.403405+0000","last_fullsized":"2023-10-16T11:22:22.403405+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T09:43:56.552266+0000","last_clean_scrub_stamp":"2023-10-16T09:43:56.552266+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:41.390255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":414,"ondisk_log_size":414,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543310,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57356,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314484,"statfs":{"total":96636764160,"available":96578031616,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.54400000000000004,"15min":0.57299999999999995},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.71599999999999997,"5min":0.878,"15min":0.878},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53400000000000003,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.59199999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54100000000000004}]},{"osd":1,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54000000000000004,"15min":0.58399999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.30099999999999999},"max":{"1min":0.56699999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53800000000000003,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.439}]},{"osd":2,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53300000000000003,"15min":0.57099999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.58299999999999996,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.60399999999999998},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.61599999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67400000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67600000000000005,"5min":1,"15min":1.002},"last":0.66500000000000004}]},{"osd":4,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.59199999999999997,"15min":0.54300000000000004},"min":{"1min":0.33200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65400000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.63700000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":1.052,"15min":1.052},"last":0.628}]},{"osd":6,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57299999999999995,"15min":0.58799999999999997},"min":{"1min":0.32900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69899999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.69699999999999995}]}]},{"osd":6,"up_from":14,"seq":60129543309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60940,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310900,"statfs":{"total":96636764160,"available":96574361600,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.60099999999999998},"min":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54000000000000004,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.59599999999999997},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.745,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.56100000000000005}]},{"osd":3,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.29699999999999999},"max":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.98999999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.90700000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.78700000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67100000000000004}]}]},{"osd":1,"up_from":14,"seq":60129543309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78984,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292856,"statfs":{"total":96636764160,"available":96555884544,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.39700000000000002},"min":{"1min":0.23499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.377,"15min":0.41799999999999998},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.44400000000000001,"5min":0.62,"15min":0.66900000000000004},"last":0.30099999999999999}]},{"osd":2,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45200000000000001,"5min":0.56999999999999995,"15min":0.61799999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.40799999999999997,"15min":0.44900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.34499999999999997}]},{"osd":3,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.497,"5min":0.69699999999999995,"15min":0.755},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67300000000000004,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.438},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":0.92600000000000005,"15min":1.097},"last":0.46700000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76600000000000001,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":1.147},"last":0.52900000000000003}]},{"osd":6,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.0509999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.0600000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.55500000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543310,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61520,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310320,"statfs":{"total":96636764160,"available":96573767680,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36899999999999999,"15min":0.38200000000000001},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.214},"max":{"1min":0.46200000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.38200000000000001,"15min":0.40699999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60299999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.53800000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39700000000000002,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.499,"5min":0.66300000000000003,"15min":0.67300000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.439},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55200000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41699999999999998,"15min":0.443},"min":{"1min":0.28899999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46600000000000003,"15min":0.48899999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.73299999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84399999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61699999999999999,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.879,"5min":0.91400000000000003,"15min":0.998},"last":0.70999999999999996}]},{"osd":5,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77900000000000003,"5min":0.91600000000000004,"15min":1.008},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.627,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75600000000000001,"5min":0.94199999999999995,"15min":1.034},"last":0.68200000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.621,"15min":0.65400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63300000000000001,"15min":0.67000000000000004},"min":{"1min":0.439,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.95999999999999996,"15min":1.0449999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.67600000000000005},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":0.628},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63600000000000001,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.753}]}]},{"osd":2,"up_from":14,"seq":60129543309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.39500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48699999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.39100000000000001,"15min":0.41299999999999998},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.51400000000000001,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.35899999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.42099999999999999,"15min":0.436},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48099999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.42399999999999999,"15min":0.45100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.40799999999999997,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.503,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46999999999999997,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.53400000000000003,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.34699999999999998}]},{"osd":4,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.60499999999999998,"15min":0.63700000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67600000000000005,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.113,"5min":3.593,"15min":3.593},"last":0.44700000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.628,"15min":0.65500000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1910000000000001,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.45700000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":3.633,"15min":3.633},"last":0.48799999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.68200000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.151,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.498},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.75700000000000001,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.55600000000000005}]}]},{"osd":3,"up_from":14,"seq":60129543310,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23899999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.48699999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.379},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.40100000000000002,"15min":0.437},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.41699999999999998,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.36099999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.374,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.437,"5min":0.87,"15min":0.87},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.437},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.41599999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40899999999999997,"15min":0.44800000000000001},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.46600000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56200000000000006,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.312}]},{"osd":4,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.68000000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.628},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60399999999999998,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71799999999999997,"5min":0.98799999999999999,"15min":1.0489999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59199999999999997,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.54800000000000004}]},{"osd":7,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.60599999999999998,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.68100000000000005}]}]},{"osd":4,"up_from":14,"seq":60129543309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96972,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94274868,"statfs":{"total":96636764160,"available":96537464832,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73199999999999998,"5min":0.78900000000000003,"15min":0.85099999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55200000000000005,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55000000000000004,"15min":0.59999999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59199999999999997,"15min":0.625},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73599999999999999,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.498}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.879},"last":0.54400000000000004}]},{"osd":5,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45300000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.91300000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.56799999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.95299999999999996},"last":0.57699999999999996}]}]},{"osd":5,"up_from":14,"seq":60129543310,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.622}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":1.873,"15min":1.873},"last":0.60699999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69699999999999995,"5min":1.024,"15min":1.024},"last":0.75600000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":1.821,"15min":1.821},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58999999999999997,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70599999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.872,"15min":0.872},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.745,"5min":1.919,"15min":1.919},"last":0.46400000000000002}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.438},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.495}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76200000000000001,"5min":2.02,"15min":2.02},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:32.274 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-16T09:43:56.552266+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=9, tm_min=43, tm_sec=56, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=11, tm_min=22, tm_sec=26, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T11:22:32.274 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-16T11:22:52.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T11:22:52.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.409+0000 7f38e20b7700 1 Processor -- start 2023-10-16T11:22:52.411 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.409+0000 7f38e20b7700 1 -- start start 2023-10-16T11:22:52.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.409+0000 7f38e20b7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38dc0af450 0x7f38dc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.409+0000 7f38e20b7700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f38dc0cd470 0x7f38dc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.409+0000 7f38e20b7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc06f870 con 0x7f38dc15dfd0 2023-10-16T11:22:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.409+0000 7f38e20b7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc0d1160 con 0x7f38dc0af450 2023-10-16T11:22:52.414 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.409+0000 7f38e20b7700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc0cc8e0 con 0x7f38dc0cd470 2023-10-16T11:22:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38daffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38dc0af450 0x7f38dc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38db7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f38dc15dfd0 0x7f38dc06d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45678/0 (socket says 172.21.15.181:45678) 2023-10-16T11:22:52.415 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38db7fe700 1 -- 172.21.15.181:0/3991053805 learned_addr learned my addr 172.21.15.181:0/3991053805 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38dbfff700 1 --2- 172.21.15.181:0/3991053805 >> v2:172.21.15.190:3300/0 conn(0x7f38dc0cd470 0x7f38dc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38daffd700 1 -- 172.21.15.181:0/3991053805 >> v2:172.21.15.190:3300/0 conn(0x7f38dc0cd470 msgr2=0x7f38dc0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38daffd700 1 --2- 172.21.15.181:0/3991053805 >> v2:172.21.15.190:3300/0 conn(0x7f38dc0cd470 0x7f38dc0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.416 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38daffd700 1 -- 172.21.15.181:0/3991053805 >> v1:172.21.15.181:6789/0 conn(0x7f38dc15dfd0 legacy=0x7f38dc06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38da7fc700 1 -- 172.21.15.181:0/3991053805 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597628174 0 0) 0x7f38dc06f870 con 0x7f38dc15dfd0 2023-10-16T11:22:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38daffd700 1 -- 172.21.15.181:0/3991053805 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0ccbc0 con 0x7f38dc0af450 2023-10-16T11:22:52.417 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38daffd700 1 --2- 172.21.15.181:0/3991053805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc0af9a0 secure :-1 s=READY pgs=3292 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0af1c0 tx=0x7f38d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33257bf8f7720db server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:52.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38da7fc700 1 -- 172.21.15.181:0/3991053805 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011830 con 0x7f38dc0af450 2023-10-16T11:22:52.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38da7fc700 1 -- 172.21.15.181:0/3991053805 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0011990 con 0x7f38dc0af450 2023-10-16T11:22:52.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.410+0000 7f38da7fc700 1 -- 172.21.15.181:0/3991053805 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011d20 con 0x7f38dc0af450 2023-10-16T11:22:52.418 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 -- 172.21.15.181:0/3991053805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 msgr2=0x7f38dc0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 --2- 172.21.15.181:0/3991053805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc0af9a0 secure :-1 s=READY pgs=3292 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0af1c0 tx=0x7f38d0009b20 comp rx=0 tx=0).stop 2023-10-16T11:22:52.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 -- 172.21.15.181:0/3991053805 shutdown_connections 2023-10-16T11:22:52.419 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 --2- 172.21.15.181:0/3991053805 >> v2:172.21.15.190:3300/0 conn(0x7f38dc0cd470 0x7f38dc0cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 --2- 172.21.15.181:0/3991053805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc0af9a0 unknown :-1 s=CLOSED pgs=3292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 -- 172.21.15.181:0/3991053805 >> 172.21.15.181:0/3991053805 conn(0x7f38dc0d04a0 msgr2=0x7f38dc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:52.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 -- 172.21.15.181:0/3991053805 shutdown_connections 2023-10-16T11:22:52.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.411+0000 7f38e20b7700 1 -- 172.21.15.181:0/3991053805 wait complete. 2023-10-16T11:22:52.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.412+0000 7f38e20b7700 1 Processor -- start 2023-10-16T11:22:52.423 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.412+0000 7f38e20b7700 1 -- start start 2023-10-16T11:22:52.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38e20b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38e20b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38dc0cd470 0x7f38dc1e44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38e20b7700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38dc15dfd0 0x7f38dc1e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38e20b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc155dd0 con 0x7f38dc0af450 2023-10-16T11:22:52.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38e20b7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc156390 con 0x7f38dc0cd470 2023-10-16T11:22:52.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38e20b7700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc14f080 con 0x7f38dc15dfd0 2023-10-16T11:22:52.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38db7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc14eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34606/0 (socket says 172.21.15.181:34606) 2023-10-16T11:22:52.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38db7fe700 1 -- 172.21.15.181:0/1135805234 learned_addr learned my addr 172.21.15.181:0/1135805234 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:52.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38dbfff700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38dc15dfd0 0x7f38dc1e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38daffd700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38dc0cd470 0x7f38dc1e44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.413+0000 7f38db7fe700 1 -- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38dc15dfd0 msgr2=0x7f38dc1e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38db7fe700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38dc15dfd0 0x7f38dc1e8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.427 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38db7fe700 1 -- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38dc0cd470 msgr2=0x7f38dc1e44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38db7fe700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38dc0cd470 0x7f38dc1e44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38db7fe700 1 -- 172.21.15.181:0/1135805234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00094a0 con 0x7f38dc0af450 2023-10-16T11:22:52.428 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38daffd700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38dc0cd470 0x7f38dc1e44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:52.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38db7fe700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc14eb40 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c4a0 tx=0x7f38cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:52.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38c3fff700 1 -- 172.21.15.181:0/1135805234 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0074e0 con 0x7f38dc0af450 2023-10-16T11:22:52.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38c3fff700 1 -- 172.21.15.181:0/1135805234 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc007640 con 0x7f38dc0af450 2023-10-16T11:22:52.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc0c64b0 con 0x7f38dc0af450 2023-10-16T11:22:52.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38c3fff700 1 -- 172.21.15.181:0/1135805234 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc005a70 con 0x7f38dc0af450 2023-10-16T11:22:52.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.414+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc156020 con 0x7f38dc0af450 2023-10-16T11:22:52.430 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.415+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc067d20 con 0x7f38dc0af450 2023-10-16T11:22:52.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.415+0000 7f38c3fff700 1 -- 172.21.15.181:0/1135805234 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f38cc005bd0 con 0x7f38dc0af450 2023-10-16T11:22:52.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.416+0000 7f38c3fff700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f38c806b020 0x7f38c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.416+0000 7f38c3fff700 1 -- 172.21.15.181:0/1135805234 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f38cc088dc0 con 0x7f38dc0af450 2023-10-16T11:22:52.431 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.416+0000 7f38daffd700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f38c806b020 0x7f38c806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.416+0000 7f38daffd700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f38c806b020 0x7f38c806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38d00090b0 tx=0x7f38d0008030 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:52.432 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.420+0000 7f38c3fff700 1 -- 172.21.15.181:0/1135805234 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f38cc053c10 con 0x7f38dc0af450 2023-10-16T11:22:52.608 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.606+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 --> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38dc0cc590 con 0x7f38c806b020 2023-10-16T11:22:52.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.608+0000 7f38c3fff700 1 -- 172.21.15.181:0/1135805234 <== mgr.13704 v2:172.21.15.181:6832/2766566872 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51036 (secure 0 0 0) 0x7f38d00117d0 con 0x7f38c806b020 2023-10-16T11:22:52.611 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:52.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f38c806b020 msgr2=0x7f38c806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f38c806b020 0x7f38c806d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38d00090b0 tx=0x7f38d0008030 comp rx=0 tx=0).stop 2023-10-16T11:22:52.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 msgr2=0x7f38dc14eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc14eb40 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c4a0 tx=0x7f38cc00c860 comp rx=0 tx=0).stop 2023-10-16T11:22:52.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 shutdown_connections 2023-10-16T11:22:52.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f38c806b020 0x7f38c806d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.614 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38dc15dfd0 0x7f38dc1e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38dc0af450 0x7f38dc14eb40 unknown :-1 s=CLOSED pgs=3293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 --2- 172.21.15.181:0/1135805234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38dc0cd470 0x7f38dc1e44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 >> 172.21.15.181:0/1135805234 conn(0x7f38dc0d04a0 msgr2=0x7f38dc15cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:52.615 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.610+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 shutdown_connections 2023-10-16T11:22:52.616 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.611+0000 7f38e20b7700 1 -- 172.21.15.181:0/1135805234 wait complete. 2023-10-16T11:22:52.616 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2023-10-16T11:22:52.624 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T11:22:51.843394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":415,"ondisk_log_size":415,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":501128,"kb_used_data":28264,"kb_used_omap":12,"kb_used_meta":472755,"kb_avail":754473592,"statfs":{"total":773094113280,"available":772580958208,"internally_reserved":0,"allocated":28942336,"data_stored":15606920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":484101688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002842"},"pg_stats":[{"pgid":"1.0","version":"274'415","reported_seq":1510,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-16T11:22:33.332189+0000","last_change":"2023-10-16T11:22:33.332253+0000","last_active":"2023-10-16T11:22:33.332189+0000","last_peered":"2023-10-16T11:22:33.332189+0000","last_clean":"2023-10-16T11:22:33.332189+0000","last_became_active":"2023-10-16T09:43:57.596912+0000","last_became_peered":"2023-10-16T09:43:57.596912+0000","last_unstale":"2023-10-16T11:22:33.332189+0000","last_undegraded":"2023-10-16T11:22:33.332189+0000","last_fullsized":"2023-10-16T11:22:33.332189+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'415","last_scrub_stamp":"2023-10-16T11:22:33.332090+0000","last_deep_scrub":"274'415","last_deep_scrub_stamp":"2023-10-16T11:22:33.332090+0000","last_clean_scrub_stamp":"2023-10-16T11:22:33.332090+0000","objects_scrubbed":2,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:22:27.942175+0000","scrub_duration":0.0060827659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2708,"num_read_kb":5662,"num_write":699,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":415,"ondisk_log_size":415,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129543314,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57352,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314488,"statfs":{"total":96636764160,"available":96578035712,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.54400000000000004,"15min":0.57299999999999995},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.23799999999999999},"max":{"1min":0.71599999999999997,"5min":0.878,"15min":0.878},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53400000000000003,"15min":0.57399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.27800000000000002},"max":{"1min":0.59199999999999997,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65600000000000003}]},{"osd":1,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54000000000000004,"15min":0.58399999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.30099999999999999},"max":{"1min":0.56699999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.53800000000000003,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53300000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53300000000000003,"15min":0.57099999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.31900000000000001},"max":{"1min":0.58299999999999996,"5min":0.81599999999999995,"15min":0.83399999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.60399999999999998},"min":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.61599999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.48699999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55200000000000005,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67400000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.59299999999999997,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67600000000000005,"5min":1,"15min":1.002},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48699999999999999,"15min":0.48399999999999999},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.59199999999999997,"15min":0.54300000000000004},"min":{"1min":0.33200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65400000000000003,"5min":8.1950000000000003,"15min":8.1950000000000003},"last":0.73199999999999998}]},{"osd":5,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.74299999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.40100000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72699999999999998,"5min":1.052,"15min":1.052},"last":0.76000000000000001}]},{"osd":6,"last update":"Mon Oct 16 11:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57299999999999995,"15min":0.58799999999999997},"min":{"1min":0.32900000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69899999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.77800000000000002}]}]},{"osd":6,"up_from":14,"seq":60129543313,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60940,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310900,"statfs":{"total":96636764160,"available":96574361600,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54400000000000004,"15min":0.57799999999999996},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56699999999999995,"15min":0.60099999999999998},"min":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.85999999999999999,"15min":0.88600000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54000000000000004,"15min":0.58599999999999997},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.629},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55900000000000005,"15min":0.59599999999999997},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.46000000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56299999999999994,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.745,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.79600000000000004,"15min":0.81399999999999995},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57799999999999996,"15min":0.60299999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.68000000000000005}]},{"osd":4,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.503,"15min":0.5},"min":{"1min":0.315,"5min":0.315,"15min":0.29699999999999999},"max":{"1min":0.73399999999999999,"5min":0.82999999999999996,"15min":0.98999999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":0.90700000000000003},"last":0.51800000000000002}]},{"osd":5,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89700000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70399999999999996}]},{"osd":7,"last update":"Mon Oct 16 11:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59999999999999998}]}]},{"osd":1,"up_from":14,"seq":60129543313,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":78984,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292856,"statfs":{"total":96636764160,"available":96555884544,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.39700000000000002},"min":{"1min":0.23499999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.68700000000000006},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.377,"15min":0.41799999999999998},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.44400000000000001,"5min":0.62,"15min":0.66900000000000004},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36899999999999999,"15min":0.40300000000000002},"min":{"1min":0.25,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45200000000000001,"5min":0.56999999999999995,"15min":0.61799999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.40799999999999997,"15min":0.44900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52900000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42499999999999999,"15min":0.47599999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.497,"5min":0.69699999999999995,"15min":0.755},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67300000000000004,"5min":0.68100000000000005,"15min":0.70699999999999996},"last":0.56799999999999995}]},{"osd":4,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":0.94999999999999996,"15min":0.95899999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":0.92600000000000005,"15min":1.097},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76600000000000001,"5min":0.94799999999999995,"15min":1.0289999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.67800000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":1.147},"last":0.627}]},{"osd":6,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.65000000000000002,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.0509999999999999},"last":0.749},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63800000000000001,"15min":0.68700000000000006},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Mon Oct 16 11:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.70499999999999996},"min":{"1min":0.46300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.0600000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.96599999999999997,"15min":1.1060000000000001},"last":0.78000000000000003}]}]},{"osd":0,"up_from":14,"seq":60129543314,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61512,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310328,"statfs":{"total":96636764160,"available":96573775872,"internally_reserved":0,"allocated":4063232,"data_stored":2393641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 11:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53000000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.37,"15min":0.40400000000000003},"min":{"1min":0.28499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53800000000000003,"5min":0.65100000000000002,"15min":0.68400000000000005},"last":0.33400000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38700000000000001,"15min":0.41399999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51900000000000002,"5min":0.66300000000000003,"15min":0.67300000000000004},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40200000000000002,"15min":0.435},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Mon Oct 16 11:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40300000000000002,"15min":0.438},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.66000000000000003,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45400000000000001,"15min":0.48499999999999999},"min":{"1min":0.32900000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.503}]},{"osd":4,"last update":"Mon Oct 16 11:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.626,"15min":0.65400000000000003},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78300000000000003,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58699999999999997,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88,"5min":0.89900000000000002,"15min":0.97199999999999998},"last":0.64000000000000001}]},{"osd":5,"last update":"Mon Oct 16 11:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86799999999999999,"5min":0.91600000000000004,"15min":1.008},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59899999999999998,"15min":0.66200000000000003},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73399999999999999,"5min":0.94199999999999995,"15min":1.0229999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":0.93100000000000005,"15min":0.95399999999999996},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60899999999999999,"15min":0.66400000000000003},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74199999999999999,"5min":0.95999999999999996,"15min":1.0449999999999999},"last":0.60799999999999998}]},{"osd":7,"last update":"Mon Oct 16 11:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60799999999999998,"15min":0.66900000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":0.624},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60999999999999999,"15min":0.67000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82199999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.70899999999999996}]}]},{"osd":2,"up_from":14,"seq":60129543313,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38200000000000001,"15min":0.39500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48699999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.39100000000000001,"15min":0.41299999999999998},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.51400000000000001,"5min":3.2509999999999999,"15min":3.2509999999999999},"last":0.436}]},{"osd":1,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.42099999999999999,"15min":0.436},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48099999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.42399999999999999,"15min":0.45100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.58399999999999996}]},{"osd":3,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.40799999999999997,"15min":0.42699999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.503,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46999999999999997,"15min":0.49199999999999999},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.53400000000000003,"5min":3.4929999999999999,"15min":3.4929999999999999},"last":0.61299999999999999}]},{"osd":4,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.60499999999999998,"15min":0.63700000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67600000000000005,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65300000000000002},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.113,"5min":3.593,"15min":3.593},"last":0.88}]},{"osd":5,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.628,"15min":0.65500000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1910000000000001,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65700000000000003},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72399999999999998,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.92300000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":3.6549999999999998,"15min":3.6549999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":3.633,"15min":3.633},"last":0.90400000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.68200000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.151,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64300000000000002,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.75700000000000001,"5min":3.7320000000000002,"15min":3.7320000000000002},"last":0.999}]}]},{"osd":3,"up_from":14,"seq":60129543314,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23899999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.48699999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.40100000000000002,"15min":0.437},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.41699999999999998,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.48999999999999999}]},{"osd":1,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.374,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.437,"5min":0.87,"15min":0.87},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.437},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.76300000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.40899999999999997,"15min":0.44800000000000001},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.46600000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.438,"15min":0.46800000000000003},"min":{"1min":0.28000000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56200000000000006,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.61799999999999999},"min":{"1min":0.41699999999999998,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.68000000000000005,"5min":0.92900000000000005,"15min":1.0740000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56499999999999995,"15min":0.628},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":1.0649999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.63100000000000001},"min":{"1min":0.39700000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60399999999999998,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76500000000000001,"5min":0.94799999999999995,"15min":1.1020000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60899999999999999,"15min":0.65200000000000002},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.71799999999999997,"5min":0.98799999999999999,"15min":1.0489999999999999},"last":0.624},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59199999999999997,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.94399999999999995},"last":0.65600000000000003}]},{"osd":7,"last update":"Mon Oct 16 11:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.60599999999999998,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":0.97599999999999998,"15min":1.0489999999999999},"last":0.68799999999999994}]}]},{"osd":4,"up_from":14,"seq":60129543313,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96972,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94274868,"statfs":{"total":96636764160,"available":96537464832,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95812038},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.73199999999999998,"5min":0.78900000000000003,"15min":0.85099999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58199999999999996}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55200000000000005,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66400000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47199999999999998}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55000000000000004,"15min":0.59999999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69199999999999995,"5min":0.78600000000000003,"15min":0.83399999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59199999999999997,"15min":0.625},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73599999999999999,"5min":0.82599999999999996,"15min":0.89600000000000002},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34399999999999997},"max":{"1min":0.71699999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.433},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.63900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.879},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53000000000000003,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55200000000000005}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29399999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.91300000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89700000000000002},"last":0.49199999999999999}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.95299999999999996},"last":0.60599999999999998}]}]},{"osd":5,"up_from":14,"seq":60129543314,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48456,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323384,"statfs":{"total":96636764160,"available":96587145216,"internally_reserved":0,"allocated":3469312,"data_stored":1803273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.31,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.59999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55800000000000005,"15min":0.59299999999999997},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65900000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58099999999999996,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.32900000000000001},"max":{"1min":0.71899999999999997,"5min":1.873,"15min":1.873},"last":0.53400000000000003}]},{"osd":2,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56899999999999995,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69899999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.69699999999999995,"5min":1.024,"15min":1.024},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59599999999999997,"15min":0.61899999999999999},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75900000000000001,"5min":1.821,"15min":1.821},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58999999999999997,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73799999999999999,"5min":0.872,"15min":0.872},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.745,"5min":1.919,"15min":1.919},"last":0.72899999999999998}]},{"osd":6,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.76200000000000001,"5min":0.878,"15min":0.878},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81799999999999995}]},{"osd":7,"last update":"Mon Oct 16 11:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76200000000000001,"5min":2.02,"15min":2.02},"last":0.86099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T11:22:52.627 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-16T11:22:52.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.745+0000 7f9b5f0d3700 1 Processor -- start 2023-10-16T11:22:52.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.745+0000 7f9b5f0d3700 1 -- start start 2023-10-16T11:22:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b5f0d3700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b580cdd00 0x7f9b5806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b5f0d3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 0x7f9b5806afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b5f0d3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b580ca300 con 0x7f9b580666a0 2023-10-16T11:22:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b5f0d3700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5806e5e0 con 0x7f9b580cdd00 2023-10-16T11:22:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b5f0d3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580ca580 con 0x7f9b580ce0e0 2023-10-16T11:22:52.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b57fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9b580666a0 0x7f9b5806a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45694/0 (socket says 172.21.15.181:45694) 2023-10-16T11:22:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b57fff700 1 -- 172.21.15.181:0/1590330328 learned_addr learned my addr 172.21.15.181:0/1590330328 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b5d670700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 0x7f9b5806afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770455913 0 0) 0x7f9b580ca300 con 0x7f9b580666a0 2023-10-16T11:22:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b480035b0 con 0x7f9b580666a0 2023-10-16T11:22:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.746+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/1590330328 >> v2:172.21.15.190:3300/0 conn(0x7f9b580cdd00 0x7f9b5806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729202516 0 0) 0x7f9b480035b0 con 0x7f9b580666a0 2023-10-16T11:22:52.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 msgr2=0x7f9b5806afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 0x7f9b5806afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 >> v2:172.21.15.190:3300/0 conn(0x7f9b580cdd00 msgr2=0x7f9b5806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:52.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 --2- 172.21.15.181:0/1590330328 >> v2:172.21.15.190:3300/0 conn(0x7f9b580cdd00 0x7f9b5806f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580ca860 con 0x7f9b580666a0 2023-10-16T11:22:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1691745811 0 0) 0x7f9b44001670 con 0x7f9b580666a0 2023-10-16T11:22:52.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 >> v1:172.21.15.181:6789/0 conn(0x7f9b580666a0 legacy=0x7f9b5806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b48003a00 0x7f9b48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48004850 0x7f9b48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 0x7f9b48005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5806e5e0 con 0x7f9b48003a00 2023-10-16T11:22:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580ca580 con 0x7f9b48004850 2023-10-16T11:22:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4800ba90 con 0x7f9b580ce0e0 2023-10-16T11:22:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b48003a00 0x7f9b48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.760 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b57fff700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48004850 0x7f9b48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5d670700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 0x7f9b48005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5ce6f700 1 -- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 msgr2=0x7f9b48005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b580ce0e0 0x7f9b48005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.761 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5ce6f700 1 -- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48004850 msgr2=0x7f9b48004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48004850 0x7f9b48004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5ce6f700 1 -- 172.21.15.181:0/1590330328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b480035b0 con 0x7f9b48003a00 2023-10-16T11:22:52.762 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b57fff700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48004850 0x7f9b48004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T11:22:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b48003a00 0x7f9b48003e20 secure :-1 s=READY pgs=3295 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009b60 tx=0x7f9b48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6576b25911c8547 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00e6c0 con 0x7f9b48003a00 2023-10-16T11:22:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c00e820 con 0x7f9b48003a00 2023-10-16T11:22:52.763 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.747+0000 7f9b577fe700 1 -- 172.21.15.181:0/1590330328 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00eb90 con 0x7f9b48003a00 2023-10-16T11:22:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.748+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/1590330328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b48003a00 msgr2=0x7f9b48003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.748+0000 7f9b5f0d3700 1 --2- 172.21.15.181:0/1590330328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b48003a00 0x7f9b48003e20 secure :-1 s=READY pgs=3295 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009b60 tx=0x7f9b48005880 comp rx=0 tx=0).stop 2023-10-16T11:22:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.748+0000 7f9b5d670700 1 -- 172.21.15.181:0/1590330328 reap_dead start 2023-10-16T11:22:52.764 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.748+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/1590330328 shutdown_connections 2023-10-16T11:22:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.748+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/1590330328 >> 172.21.15.181:0/1590330328 conn(0x7f9b580cc730 msgr2=0x7f9b580ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.748+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/1590330328 shutdown_connections 2023-10-16T11:22:52.765 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.748+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/1590330328 wait complete. 2023-10-16T11:22:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.749+0000 7f9b5f0d3700 1 Processor -- start 2023-10-16T11:22:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.749+0000 7f9b5f0d3700 1 -- start start 2023-10-16T11:22:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5f0d3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48003a00 0x7f9b5814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.766 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5f0d3700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b5815db60 0x7f9b581e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5f0d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b5815b500 0x7f9b5815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5f0d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5814d6d0 con 0x7f9b5815b500 2023-10-16T11:22:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5f0d3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5815b070 con 0x7f9b48003a00 2023-10-16T11:22:52.767 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5f0d3700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b581e9400 con 0x7f9b5815db60 2023-10-16T11:22:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b5815b500 0x7f9b5815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b5815b500 0x7f9b5815b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34630/0 (socket says 172.21.15.181:34630) 2023-10-16T11:22:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 -- 172.21.15.181:0/3330517447 learned_addr learned my addr 172.21.15.181:0/3330517447 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:52.768 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48003a00 0x7f9b5814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b57fff700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b5815db60 0x7f9b581e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 -- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b5815db60 msgr2=0x7f9b581e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.769 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b5815db60 0x7f9b581e8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 -- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48003a00 msgr2=0x7f9b5814d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48003a00 0x7f9b5814d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 -- 172.21.15.181:0/3330517447 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b581e8b60 con 0x7f9b5815b500 2023-10-16T11:22:52.770 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.750+0000 7f9b5d670700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b5815b500 0x7f9b5815b990 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 crypto rx=0x7f9b58070b00 tx=0x7f9b500114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:52.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.751+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5000e070 con 0x7f9b5815b500 2023-10-16T11:22:52.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.751+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50003cf0 con 0x7f9b5815b500 2023-10-16T11:22:52.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.751+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b581e8e40 con 0x7f9b5815b500 2023-10-16T11:22:52.771 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.751+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50003ea0 con 0x7f9b5815b500 2023-10-16T11:22:52.772 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.751+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b581e87a0 con 0x7f9b5815b500 2023-10-16T11:22:52.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.752+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5815b220 con 0x7f9b5815b500 2023-10-16T11:22:52.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.752+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9b50011a10 con 0x7f9b5815b500 2023-10-16T11:22:52.773 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.752+0000 7f9b557fa700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9b3c06b020 0x7f9b3c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:52.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.753+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f9b5008a0b0 con 0x7f9b5815b500 2023-10-16T11:22:52.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.753+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9b3c06b020 0x7f9b3c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:52.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.753+0000 7f9b5ce6f700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9b3c06b020 0x7f9b3c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b580b1460 tx=0x7f9b4c008030 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:52.774 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.757+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7f9b5815b220 con 0x7f9b5815b500 2023-10-16T11:22:52.954 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.952+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f9b5815b1e0 con 0x7f9b5815b500 2023-10-16T11:22:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.955+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v100)=0 v100) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9b50054d60 con 0x7f9b5815b500 2023-10-16T11:22:52.958 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.956+0000 7f9b557fa700 1 -- 172.21.15.181:0/3330517447 <== mon.0 v2:172.21.15.181:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9b50054d60 con 0x7f9b5815b500 2023-10-16T11:22:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9b3c06b020 msgr2=0x7f9b3c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9b3c06b020 0x7f9b3c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b580b1460 tx=0x7f9b4c008030 comp rx=0 tx=0).stop 2023-10-16T11:22:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b5815b500 msgr2=0x7f9b5815b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:52.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b5815b500 0x7f9b5815b990 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 crypto rx=0x7f9b58070b00 tx=0x7f9b500114b0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 shutdown_connections 2023-10-16T11:22:52.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7f9b3c06b020 0x7f9b3c06d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b5815b500 0x7f9b5815b990 unknown :-1 s=CLOSED pgs=3296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b48003a00 0x7f9b5814d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 --2- 172.21.15.181:0/3330517447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b5815db60 0x7f9b581e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:52.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 >> 172.21.15.181:0/3330517447 conn(0x7f9b580cc730 msgr2=0x7f9b580af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:52.962 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 shutdown_connections 2023-10-16T11:22:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:52.958+0000 7f9b5f0d3700 1 -- 172.21.15.181:0/3330517447 wait complete. 2023-10-16T11:22:52.972 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T11:22:52.973 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T11:22:53.094 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.092+0000 7fb113cbd700 1 Processor -- start 2023-10-16T11:22:53.095 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.092+0000 7fb113cbd700 1 -- start start 2023-10-16T11:22:53.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb113cbd700 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb10c15dfd0 0x7fb10c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb113cbd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb10c0af450 0x7fb10c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb113cbd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c06f870 con 0x7fb10c06db80 2023-10-16T11:22:53.097 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb113cbd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0d1160 con 0x7fb10c0af450 2023-10-16T11:22:53.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb113cbd700 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0cc8e0 con 0x7fb10c15dfd0 2023-10-16T11:22:53.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb111258700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb10c0af450 0x7fb10c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.098 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb11225a700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb10c06db80 0x7fb10c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:45714/0 (socket says 172.21.15.181:45714) 2023-10-16T11:22:53.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb11225a700 1 -- 172.21.15.181:0/3539876381 learned_addr learned my addr 172.21.15.181:0/3539876381 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-16T11:22:53.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb111a59700 1 --2- 172.21.15.181:0/3539876381 >> v2:172.21.15.190:3300/0 conn(0x7fb10c15dfd0 0x7fb10c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb111258700 1 -- 172.21.15.181:0/3539876381 >> v2:172.21.15.190:3300/0 conn(0x7fb10c15dfd0 msgr2=0x7fb10c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T11:22:53.099 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb111258700 1 --2- 172.21.15.181:0/3539876381 >> v2:172.21.15.190:3300/0 conn(0x7fb10c15dfd0 0x7fb10c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb111258700 1 -- 172.21.15.181:0/3539876381 >> v1:172.21.15.181:6789/0 conn(0x7fb10c06db80 legacy=0x7fb10c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb110a57700 1 -- 172.21.15.181:0/3539876381 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566901617 0 0) 0x7fb10c06f870 con 0x7fb10c06db80 2023-10-16T11:22:53.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb111258700 1 -- 172.21.15.181:0/3539876381 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0ccbc0 con 0x7fb10c0af450 2023-10-16T11:22:53.100 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.093+0000 7fb111258700 1 --2- 172.21.15.181:0/3539876381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c0af450 0x7fb10c06d640 secure :-1 s=READY pgs=3298 cs=0 l=1 rev1=1 crypto rx=0x7fb10c070340 tx=0x7fb0fc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc250530945f0578 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb110a57700 1 -- 172.21.15.181:0/3539876381 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc011830 con 0x7fb10c0af450 2023-10-16T11:22:53.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb110a57700 1 -- 172.21.15.181:0/3539876381 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb0fc011990 con 0x7fb10c0af450 2023-10-16T11:22:53.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb110a57700 1 -- 172.21.15.181:0/3539876381 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc011d40 con 0x7fb10c0af450 2023-10-16T11:22:53.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 -- 172.21.15.181:0/3539876381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c0af450 msgr2=0x7fb10c06d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3539876381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c0af450 0x7fb10c06d640 secure :-1 s=READY pgs=3298 cs=0 l=1 rev1=1 crypto rx=0x7fb10c070340 tx=0x7fb0fc009b20 comp rx=0 tx=0).stop 2023-10-16T11:22:53.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 -- 172.21.15.181:0/3539876381 shutdown_connections 2023-10-16T11:22:53.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3539876381 >> v2:172.21.15.190:3300/0 conn(0x7fb10c15dfd0 0x7fb10c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3539876381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c0af450 0x7fb10c06d640 unknown :-1 s=CLOSED pgs=3298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 -- 172.21.15.181:0/3539876381 >> 172.21.15.181:0/3539876381 conn(0x7fb10c0d04a0 msgr2=0x7fb10c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:53.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 -- 172.21.15.181:0/3539876381 shutdown_connections 2023-10-16T11:22:53.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.094+0000 7fb113cbd700 1 -- 172.21.15.181:0/3539876381 wait complete. 2023-10-16T11:22:53.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.095+0000 7fb113cbd700 1 Processor -- start 2023-10-16T11:22:53.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.095+0000 7fb113cbd700 1 -- start start 2023-10-16T11:22:53.106 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb113cbd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c06db80 0x7fb10c154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb113cbd700 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0af450 0x7fb10c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb113cbd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb10c15dfd0 0x7fb10c0c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb113cbd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c158e20 con 0x7fb10c06db80 2023-10-16T11:22:53.107 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb113cbd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1599a0 con 0x7fb10c15dfd0 2023-10-16T11:22:53.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb113cbd700 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c159070 con 0x7fb10c0af450 2023-10-16T11:22:53.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb111a59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c06db80 0x7fb10c154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb111a59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c06db80 0x7fb10c154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34640/0 (socket says 172.21.15.181:34640) 2023-10-16T11:22:53.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb111a59700 1 -- 172.21.15.181:0/3903306408 learned_addr learned my addr 172.21.15.181:0/3903306408 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-16T11:22:53.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb111258700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0af450 0x7fb10c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.096+0000 7fb11225a700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb10c15dfd0 0x7fb10c0c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb111a59700 1 -- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0af450 msgr2=0x7fb10c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb111a59700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0af450 0x7fb10c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb111a59700 1 -- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb10c15dfd0 msgr2=0x7fb10c0c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb111a59700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb10c15dfd0 0x7fb10c0c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb111a59700 1 -- 172.21.15.181:0/3903306408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc0094a0 con 0x7fb10c06db80 2023-10-16T11:22:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb111a59700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c06db80 0x7fb10c154ee0 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7fb10800d7d0 tx=0x7fb10800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb1027fc700 1 -- 172.21.15.181:0/3903306408 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108017070 con 0x7fb10c06db80 2023-10-16T11:22:53.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb1027fc700 1 -- 172.21.15.181:0/3903306408 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb108005a20 con 0x7fb10c06db80 2023-10-16T11:22:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c159350 con 0x7fb10c06db80 2023-10-16T11:22:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb1027fc700 1 -- 172.21.15.181:0/3903306408 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108005bf0 con 0x7fb10c06db80 2023-10-16T11:22:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.097+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c155420 con 0x7fb10c06db80 2023-10-16T11:22:53.112 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.098+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c067d20 con 0x7fb10c06db80 2023-10-16T11:22:53.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.098+0000 7fb1027fc700 1 -- 172.21.15.181:0/3903306408 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 178) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb108005d50 con 0x7fb10c06db80 2023-10-16T11:22:53.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.099+0000 7fb1027fc700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb0f006b020 0x7fb0f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T11:22:53.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.099+0000 7fb111258700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb0f006b020 0x7fb0f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T11:22:53.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.099+0000 7fb1027fc700 1 -- 172.21.15.181:0/3903306408 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fb108089410 con 0x7fb10c06db80 2023-10-16T11:22:53.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.099+0000 7fb111258700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb0f006b020 0x7fb0f006d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0090b0 tx=0x7fb0fc008030 comp rx=0 tx=0).ready entity=mgr.13704 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T11:22:53.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.103+0000 7fb1027fc700 1 -- 172.21.15.181:0/3903306408 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180165 (secure 0 0 0) 0x7fb108054140 con 0x7fb10c06db80 2023-10-16T11:22:53.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.341+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb10c0d0170 con 0x7fb10c06db80 2023-10-16T11:22:53.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.342+0000 7fb1027fc700 1 -- 172.21.15.181:0/3903306408 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) v1 ==== 95+0+3714 (secure 0 0 0) 0x7fb108057760 con 0x7fb10c06db80 2023-10-16T11:22:53.345 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-16T11:22:53.345 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":155,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/984505218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":984505218},{"type":"v1","addr":"172.21.15.190:6835","nonce":984505218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":12076,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3506240211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3506240211},{"type":"v1","addr":"172.21.15.190:6834","nonce":3506240211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":13437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/350629849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":350629849},{"type":"v1","addr":"172.21.15.181:6835","nonce":350629849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":13440,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1354411415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1354411415},{"type":"v1","addr":"172.21.15.181:6837","nonce":1354411415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[]} 2023-10-16T11:22:53.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb0f006b020 msgr2=0x7fb0f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb0f006b020 0x7fb0f006d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0090b0 tx=0x7fb0fc008030 comp rx=0 tx=0).stop 2023-10-16T11:22:53.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c06db80 msgr2=0x7fb10c154ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T11:22:53.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c06db80 0x7fb10c154ee0 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7fb10800d7d0 tx=0x7fb10800dae0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 shutdown_connections 2023-10-16T11:22:53.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:6832/2766566872,v1:172.21.15.181:6833/2766566872] conn(0x7fb0f006b020 0x7fb0f006d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb10c15dfd0 0x7fb10c0c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb10c06db80 0x7fb10c154ee0 unknown :-1 s=CLOSED pgs=3299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 --2- 172.21.15.181:0/3903306408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb10c0af450 0x7fb10c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T11:22:53.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 >> 172.21.15.181:0/3903306408 conn(0x7fb10c0d04a0 msgr2=0x7fb10c0cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T11:22:53.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 shutdown_connections 2023-10-16T11:22:53.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-16T11:22:53.344+0000 7fb113cbd700 1 -- 172.21.15.181:0/3903306408 wait complete. 2023-10-16T11:22:53.350 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 155 2023-10-16T11:22:53.358 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-16T11:22:53.358 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-16T11:22:53.359 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T11:22:53.359 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.367 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T11:22:53.367 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T11:22:53.368 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.415 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T11:22:53.416 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T11:22:53.416 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.427 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T11:22:53.427 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T11:22:53.427 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.479 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T11:22:53.479 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-16T11:22:53.480 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-16T11:22:53.480 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.523 INFO:tasks.ceph.osd.0:Stopped 2023-10-16T11:22:53.523 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-16T11:22:53.524 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.567 INFO:tasks.ceph.osd.1:Stopped 2023-10-16T11:22:53.568 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-16T11:22:53.568 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.611 INFO:tasks.ceph.osd.2:Stopped 2023-10-16T11:22:53.612 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-16T11:22:53.612 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.648 INFO:tasks.ceph.osd.3:Stopped 2023-10-16T11:22:53.649 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-16T11:22:53.649 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.687 INFO:tasks.ceph.osd.4:Stopped 2023-10-16T11:22:53.687 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-16T11:22:53.687 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.782 INFO:tasks.ceph.osd.5:Stopped 2023-10-16T11:22:53.783 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-16T11:22:53.783 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.824 INFO:tasks.ceph.osd.6:Stopped 2023-10-16T11:22:53.825 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-16T11:22:53.825 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.876 INFO:tasks.ceph.osd.7:Stopped 2023-10-16T11:22:53.877 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-16T11:22:53.877 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-16T11:22:53.878 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.908 INFO:tasks.ceph.mgr.y:Stopped 2023-10-16T11:22:53.908 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-16T11:22:53.909 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.936 INFO:tasks.ceph.mgr.x:Stopped 2023-10-16T11:22:53.936 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-16T11:22:53.937 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-16T11:22:53.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.964 INFO:tasks.ceph.mon.a:Stopped 2023-10-16T11:22:53.965 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-16T11:22:53.965 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:53.991 INFO:tasks.ceph.mon.b:Stopped 2023-10-16T11:22:53.992 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-16T11:22:53.992 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T11:22:54.047 INFO:tasks.ceph.mon.c:Stopped 2023-10-16T11:22:54.047 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-16T11:22:54.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-16T11:22:54.080 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:22:54.080 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-16T11:22:54.356 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:22:54.357 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-16T11:22:54.468 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:22:54.468 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-16T11:22:54.547 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:22:54.548 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-16T11:22:54.622 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T11:22:54.622 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-16T11:22:54.855 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T11:22:54.856 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-16T11:22:54.952 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T11:22:54.953 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-16T11:22:55.032 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T11:22:55.033 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-16T11:22:55.115 INFO:tasks.ceph:Archiving mon data... 2023-10-16T11:22:55.117 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/data/mon.a.tgz 2023-10-16T11:22:55.118 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-10-16T11:22:55.137 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.BM5xsIKvKB 2023-10-16T11:22:55.138 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.BM5xsIKvKB 2023-10-16T11:22:55.846 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.BM5xsIKvKB 2023-10-16T11:22:55.932 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.BM5xsIKvKB is 4MB 2023-10-16T11:22:56.073 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.BM5xsIKvKB 2023-10-16T11:22:56.091 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/data/mon.b.tgz 2023-10-16T11:22:56.091 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-10-16T11:22:56.112 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.Hwj32IzpNQ 2023-10-16T11:22:56.112 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Hwj32IzpNQ 2023-10-16T11:22:56.841 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.Hwj32IzpNQ 2023-10-16T11:22:56.927 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.Hwj32IzpNQ is 4MB 2023-10-16T11:22:57.087 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.Hwj32IzpNQ 2023-10-16T11:22:57.105 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/data/mon.c.tgz 2023-10-16T11:22:57.106 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-10-16T11:22:57.162 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.PKFKxTzO78 2023-10-16T11:22:57.162 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PKFKxTzO78 2023-10-16T11:22:57.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.PKFKxTzO78 2023-10-16T11:22:57.953 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.PKFKxTzO78 is 4MB 2023-10-16T11:22:58.112 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.PKFKxTzO78 2023-10-16T11:22:58.130 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-16T11:22:58.130 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T11:22:58.133 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T11:22:58.196 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-16T11:22:58.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T11:22:58.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T11:22:58.206 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:58.244 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T11:22:58.246 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:58.273 INFO:teuthology.orchestra.run.smithi190.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T11:22:58.278 INFO:tasks.ceph:Archiving crash dumps... 2023-10-16T11:22:58.281 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/remote/smithi181/crash 2023-10-16T11:22:58.282 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T11:22:58.319 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/remote/smithi190/crash 2023-10-16T11:22:58.320 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T11:22:58.387 INFO:tasks.ceph:Shutting down logrotate 2023-10-16T11:22:58.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:58.391 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:58.488 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:58.517 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T11:22:58.543 INFO:tasks.ceph:Compressing logs... 2023-10-16T11:22:58.544 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T11:22:58.559 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T11:22:58.623 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104106.log 2023-10-16T11:22:58.624 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-16T11:22:58.624 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104106.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T11:22:58.625 INFO:teuthology.orchestra.run.smithi190.stderr:gzip 87.3%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.104106.log.gz-- 2023-10-16T11:22:58.625 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104014.log 2023-10-16T11:22:58.625 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-16T11:22:58.625 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110919.log 2023-10-16T11:22:58.626 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.6.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104014.log.gz 2023-10-16T11:22:58.626 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104151.log 2023-10-16T11:22:58.626 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.3.110919.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-16T11:22:58.627 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110919.log.gz 2023-10-16T11:22:58.627 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104151.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T11:22:58.628 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-16T11:22:58.629 INFO:teuthology.orchestra.run.smithi190.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104151.log.gz 2023-10-16T11:22:58.629 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110308.log 2023-10-16T11:22:58.629 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.1.110308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110308.log.gz 2023-10-16T11:22:58.630 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-16T11:22:58.630 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110300.log 2023-10-16T11:22:58.630 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.1.110300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110300.log.gz 2023-10-16T11:22:58.630 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-16T11:22:58.631 INFO:teuthology.orchestra.run.smithi190.stderr: 94.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T11:22:58.637 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.c.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T11:22:58.638 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103967.log 2023-10-16T11:22:58.638 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.103967.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103967.log.gz 2023-10-16T11:22:58.638 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104196.log 2023-10-16T11:22:58.639 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104196.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104196.log.gz 2023-10-16T11:22:58.639 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104294.log 2023-10-16T11:22:58.639 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104294.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104294.log.gz 2023-10-16T11:22:58.639 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-16T11:22:58.640 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104241.log 2023-10-16T11:22:58.642 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104241.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104241.log.gz 2023-10-16T11:22:58.643 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110915.log 2023-10-16T11:22:58.643 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.3.110915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110915.log.gz 2023-10-16T11:22:58.643 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-16T11:22:58.648 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-16T11:22:58.678 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104059.log 2023-10-16T11:22:58.691 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.104059.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104059.log.gz 2023-10-16T11:22:58.705 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112454.log 2023-10-16T11:22:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189812.log 2023-10-16T11:22:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131274.log 2023-10-16T11:22:58.706 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183740.log 2023-10-16T11:22:58.707 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112454.log.gz 2023-10-16T11:22:58.707 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189812.log.gz 2023-10-16T11:22:58.707 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164816.log 2023-10-16T11:22:58.708 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.131274.log.gz 2023-10-16T11:22:58.708 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132723.log 2023-10-16T11:22:58.708 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135172.log 2023-10-16T11:22:58.708 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.183740.log.gz 2023-10-16T11:22:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132723.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164816.log.gz 2023-10-16T11:22:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136736.log 2023-10-16T11:22:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135172.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132723.log.gz 2023-10-16T11:22:58.709 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174637.log 2023-10-16T11:22:58.710 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135172.log.gz 2023-10-16T11:22:58.710 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134583.log 2023-10-16T11:22:58.710 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136736.log: /var/log/ceph/ceph-client.admin.174637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176921.log 2023-10-16T11:22:58.711 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134583.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174637.log.gz 2023-10-16T11:22:58.711 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165652.log 2023-10-16T11:22:58.711 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136736.log.gz 2023-10-16T11:22:58.711 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142974.log 2023-10-16T11:22:58.712 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134583.log.gz 2023-10-16T11:22:58.712 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.165652.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.176921.log.gz 2023-10-16T11:22:58.712 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177853.log 2023-10-16T11:22:58.713 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.165652.log.gz 2023-10-16T11:22:58.713 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118537.log 2023-10-16T11:22:58.713 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142974.log.gz 2023-10-16T11:22:58.713 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148482.log 2023-10-16T11:22:58.714 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118537.log: gzip -5 88.8%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.177853.log.gz/var/log/ceph/ceph-client.admin.168017.log 2023-10-16T11:22:58.714 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:22:58.714 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148482.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118537.log.gz 2023-10-16T11:22:58.714 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154515.log 2023-10-16T11:22:58.715 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148482.log.gz 2023-10-16T11:22:58.715 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193377.log 2023-10-16T11:22:58.715 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.154515.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168017.log.gz 2023-10-16T11:22:58.716 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150593.log 2023-10-16T11:22:58.716 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193377.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154515.log.gz 2023-10-16T11:22:58.716 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162030.log 2023-10-16T11:22:58.716 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.193377.log.gz 2023-10-16T11:22:58.717 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118283.log 2023-10-16T11:22:58.717 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150593.log: /var/log/ceph/ceph-client.admin.162030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146726.log 2023-10-16T11:22:58.717 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150593.log.gz 2023-10-16T11:22:58.717 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117249.log 2023-10-16T11:22:58.718 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118283.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162030.log.gz 2023-10-16T11:22:58.718 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.146726.log: 87.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.118283.log.gz--verbose 2023-10-16T11:22:58.718 INFO:teuthology.orchestra.run.smithi181.stderr:-- /var/log/ceph/ceph-client.admin.186333.log 2023-10-16T11:22:58.719 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146726.log.gz 2023-10-16T11:22:58.719 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 /var/log/ceph/ceph-client.admin.117249.log: --verbose -- /var/log/ceph/ceph-client.admin.183437.log 2023-10-16T11:22:58.719 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177656.log 2023-10-16T11:22:58.719 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117249.log.gz 2023-10-16T11:22:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110223.log 2023-10-16T11:22:58.720 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.186333.log.gz 2023-10-16T11:22:58.720 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111649.log 2023-10-16T11:22:58.720 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.183437.log.gz 2023-10-16T11:22:58.721 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110223.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177656.log.gz 2023-10-16T11:22:58.721 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189070.log 2023-10-16T11:22:58.721 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110223.log.gz 2023-10-16T11:22:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126059.log 2023-10-16T11:22:58.722 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5%/var/log/ceph/ceph-client.admin.189070.log: -- replaced with /var/log/ceph/ceph-client.admin.111649.log.gz 2023-10-16T11:22:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159568.log 2023-10-16T11:22:58.722 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126059.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.189070.log.gz 2023-10-16T11:22:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119876.log 2023-10-16T11:22:58.723 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126059.log.gz 2023-10-16T11:22:58.723 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122886.log 2023-10-16T11:22:58.723 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5%/var/log/ceph/ceph-client.admin.119876.log: -- replaced with /var/log/ceph/ceph-client.admin.159568.log.gz 2023-10-16T11:22:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133284.log 2023-10-16T11:22:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.122886.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119876.log.gz 2023-10-16T11:22:58.724 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134239.log 2023-10-16T11:22:58.725 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122886.log.gz 2023-10-16T11:22:58.725 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120366.log 2023-10-16T11:22:58.725 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.134239.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133284.log.gz 2023-10-16T11:22:58.725 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143624.log 2023-10-16T11:22:58.726 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.134239.log.gz 2023-10-16T11:22:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133060.log 2023-10-16T11:22:58.726 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120366.log.gz 2023-10-16T11:22:58.726 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115082.log 2023-10-16T11:22:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143624.log: /var/log/ceph/ceph-client.admin.133060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175093.log 2023-10-16T11:22:58.727 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143624.log.gz 2023-10-16T11:22:58.727 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147245.log 2023-10-16T11:22:58.727 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133060.log.gz 2023-10-16T11:22:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175093.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115082.log.gz 2023-10-16T11:22:58.728 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138863.log 2023-10-16T11:22:58.728 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175093.log.gz 2023-10-16T11:22:58.729 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132390.log 2023-10-16T11:22:58.729 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147245.log: /var/log/ceph/ceph-client.admin.138863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121338.log 2023-10-16T11:22:58.729 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147245.log.gz 2023-10-16T11:22:58.729 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138863.log.gz 2023-10-16T11:22:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150934.log 2023-10-16T11:22:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110639.log 2023-10-16T11:22:58.730 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132390.log.gz 2023-10-16T11:22:58.730 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150934.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121338.log.gz 2023-10-16T11:22:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175574.log 2023-10-16T11:22:58.731 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150934.log.gz 2023-10-16T11:22:58.731 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163817.log 2023-10-16T11:22:58.732 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110639.log: /var/log/ceph/ceph-client.admin.175574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177366.log 2023-10-16T11:22:58.732 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110639.log.gz 2023-10-16T11:22:58.732 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132677.log 2023-10-16T11:22:58.732 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175574.log.gz 2023-10-16T11:22:58.733 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177366.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.163817.log.gz 2023-10-16T11:22:58.733 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182505.log 2023-10-16T11:22:58.733 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132677.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177366.log.gz 2023-10-16T11:22:58.733 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143824.log 2023-10-16T11:22:58.734 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182505.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132677.log.gz 2023-10-16T11:22:58.734 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163955.log 2023-10-16T11:22:58.734 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143824.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.182505.log.gz 2023-10-16T11:22:58.734 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164001.log 2023-10-16T11:22:58.735 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143824.log.gz 2023-10-16T11:22:58.735 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121476.log 2023-10-16T11:22:58.735 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164001.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163955.log.gz 2023-10-16T11:22:58.736 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115128.log 2023-10-16T11:22:58.736 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1%/var/log/ceph/ceph-client.admin.121476.log: -- replaced with /var/log/ceph/ceph-client.admin.164001.log.gz 2023-10-16T11:22:58.736 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190048.log 2023-10-16T11:22:58.736 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115128.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121476.log.gz 2023-10-16T11:22:58.737 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153349.log 2023-10-16T11:22:58.737 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190048.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115128.log.gz 2023-10-16T11:22:58.737 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121898.log 2023-10-16T11:22:58.737 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153349.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.190048.log.gz 2023-10-16T11:22:58.738 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119990.log 2023-10-16T11:22:58.738 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121898.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153349.log.gz 2023-10-16T11:22:58.738 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151983.log 2023-10-16T11:22:58.739 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119990.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121898.log.gz 2023-10-16T11:22:58.739 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178172.log 2023-10-16T11:22:58.739 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151983.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119990.log.gz 2023-10-16T11:22:58.739 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188474.log 2023-10-16T11:22:58.740 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151983.log.gz 2023-10-16T11:22:58.740 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144232.log 2023-10-16T11:22:58.740 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188474.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178172.log.gz 2023-10-16T11:22:58.740 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164724.log 2023-10-16T11:22:58.741 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144232.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188474.log.gz 2023-10-16T11:22:58.741 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189254.log 2023-10-16T11:22:58.741 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144232.log.gz 2023-10-16T11:22:58.742 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116383.log 2023-10-16T11:22:58.742 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189254.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164724.log.gz 2023-10-16T11:22:58.742 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185217.log 2023-10-16T11:22:58.742 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.189254.log.gz 2023-10-16T11:22:58.743 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182133.log 2023-10-16T11:22:58.743 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116383.log.gz 2023-10-16T11:22:58.743 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127653.log 2023-10-16T11:22:58.744 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185217.log.gz 2023-10-16T11:22:58.744 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121200.log 2023-10-16T11:22:58.744 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127653.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.182133.log.gz 2023-10-16T11:22:58.744 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128556.log 2023-10-16T11:22:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121200.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127653.log.gz 2023-10-16T11:22:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147764.log 2023-10-16T11:22:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128556.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121200.log.gz 2023-10-16T11:22:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119608.log 2023-10-16T11:22:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147764.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128556.log.gz 2023-10-16T11:22:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128692.log 2023-10-16T11:22:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119608.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147764.log.gz 2023-10-16T11:22:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153119.log 2023-10-16T11:22:58.747 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2%/var/log/ceph/ceph-client.admin.128692.log: -- replaced with /var/log/ceph/ceph-client.admin.119608.log.gz 2023-10-16T11:22:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169988.log 2023-10-16T11:22:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153119.log: gzip -5 --verbose 88.6%-- /var/log/ceph/ceph-client.admin.167263.log -- replaced with /var/log/ceph/ceph-client.admin.128692.log.gz 2023-10-16T11:22:58.748 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:22:58.748 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.153119.log.gz 2023-10-16T11:22:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127745.log 2023-10-16T11:22:58.749 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.167263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169988.log.gz 2023-10-16T11:22:58.749 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116567.log 2023-10-16T11:22:58.749 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127745.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167263.log.gz 2023-10-16T11:22:58.749 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151583.log 2023-10-16T11:22:58.750 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127745.log.gz 2023-10-16T11:22:58.750 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191933.log 2023-10-16T11:22:58.750 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116567.log.gz 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134953.log 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191933.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151583.log.gz 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.175923.log 2023-10-16T11:22:58.751 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191933.log.gz 2023-10-16T11:22:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174997.log 2023-10-16T11:22:58.752 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3%/var/log/ceph/ceph-client.0.175923.log: -- replaced with /var/log/ceph/ceph-client.admin.134953.log.gz 2023-10-16T11:22:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117203.log 2023-10-16T11:22:58.752 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.175923.log.gz 2023-10-16T11:22:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162696.log 2023-10-16T11:22:58.753 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.174997.log.gz 2023-10-16T11:22:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131459.log 2023-10-16T11:22:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162696.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117203.log.gz 2023-10-16T11:22:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161184.log 2023-10-16T11:22:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131459.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162696.log.gz 2023-10-16T11:22:58.754 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162386.log 2023-10-16T11:22:58.755 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131459.log.gz 2023-10-16T11:22:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149794.log 2023-10-16T11:22:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162386.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161184.log.gz 2023-10-16T11:22:58.755 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139115.log 2023-10-16T11:22:58.756 INFO:teuthology.orchestra.run.smithi181.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.162386.log.gz 2023-10-16T11:22:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143348.log 2023-10-16T11:22:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139115.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149794.log.gz 2023-10-16T11:22:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-16T11:22:58.757 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143348.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139115.log.gz 2023-10-16T11:22:58.757 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183537.log 2023-10-16T11:22:58.757 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.2.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143348.log.gz 2023-10-16T11:22:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135441.log 2023-10-16T11:22:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151218.log 2023-10-16T11:22:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135441.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.183537.log.gz 2023-10-16T11:22:58.758 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133564.log 2023-10-16T11:22:58.759 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135441.log.gz 2023-10-16T11:22:58.759 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169568.log 2023-10-16T11:22:58.759 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133564.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.151218.log.gz 2023-10-16T11:22:58.759 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.165445.log 2023-10-16T11:22:58.760 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133564.log.gz 2023-10-16T11:22:58.760 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185539.log 2023-10-16T11:22:58.760 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.169568.log.gz 2023-10-16T11:22:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.165445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126108.log 2023-10-16T11:22:58.761 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.165445.log.gz 2023-10-16T11:22:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161276.log 2023-10-16T11:22:58.761 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126108.log: gzip 88.2%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.185539.log.gz/var/log/ceph/ceph-client.admin.145685.log 2023-10-16T11:22:58.762 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:22:58.762 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161276.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126108.log.gz 2023-10-16T11:22:58.762 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173594.log 2023-10-16T11:22:58.762 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7%/var/log/ceph/ceph-client.admin.145685.log: -- replaced with /var/log/ceph/ceph-client.admin.161276.log.gz 2023-10-16T11:22:58.763 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147642.log 2023-10-16T11:22:58.763 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173594.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145685.log.gz 2023-10-16T11:22:58.763 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177609.log 2023-10-16T11:22:58.764 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147642.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.173594.log.gz 2023-10-16T11:22:58.764 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135992.log 2023-10-16T11:22:58.764 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177609.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.147642.log.gz 2023-10-16T11:22:58.764 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143294.log 2023-10-16T11:22:58.765 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180764.log 2023-10-16T11:22:58.765 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177609.log.gz 2023-10-16T11:22:58.765 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143294.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135992.log.gz 2023-10-16T11:22:58.765 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118947.log 2023-10-16T11:22:58.766 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180764.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143294.log.gz 2023-10-16T11:22:58.766 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191834.log 2023-10-16T11:22:58.766 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115220.log 2023-10-16T11:22:58.767 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.180764.log.gz 2023-10-16T11:22:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191834.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118947.log.gz 2023-10-16T11:22:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132493.log 2023-10-16T11:22:58.767 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115220.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.191834.log.gz 2023-10-16T11:22:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188723.log 2023-10-16T11:22:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132493.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115220.log.gz 2023-10-16T11:22:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159368.log 2023-10-16T11:22:58.768 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188723.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132493.log.gz 2023-10-16T11:22:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175323.log 2023-10-16T11:22:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159368.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188723.log.gz 2023-10-16T11:22:58.769 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149177.log 2023-10-16T11:22:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175323.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159368.log.gz 2023-10-16T11:22:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111356.log 2023-10-16T11:22:58.770 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175323.log.gz 2023-10-16T11:22:58.770 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160755.log 2023-10-16T11:22:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169838.log 2023-10-16T11:22:58.771 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149177.log.gz 2023-10-16T11:22:58.771 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111356.log.gz 2023-10-16T11:22:58.771 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117711.log 2023-10-16T11:22:58.772 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160755.log.gz 2023-10-16T11:22:58.772 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141556.log 2023-10-16T11:22:58.772 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117711.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.169838.log.gz 2023-10-16T11:22:58.773 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137312.log 2023-10-16T11:22:58.773 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117711.log.gz 2023-10-16T11:22:58.773 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.141556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152076.log 2023-10-16T11:22:58.773 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137312.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141556.log.gz 2023-10-16T11:22:58.774 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184711.log 2023-10-16T11:22:58.774 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185493.log 2023-10-16T11:22:58.774 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.137312.log.gz 2023-10-16T11:22:58.774 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.152076.log.gz 2023-10-16T11:22:58.775 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175620.log 2023-10-16T11:22:58.775 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185493.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.184711.log.gz 2023-10-16T11:22:58.775 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137404.log 2023-10-16T11:22:58.776 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160056.log 2023-10-16T11:22:58.776 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.185493.log.gz 2023-10-16T11:22:58.776 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184589.log 2023-10-16T11:22:58.776 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160056.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.175620.log.gz 2023-10-16T11:22:58.777 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170128.log 2023-10-16T11:22:58.777 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.137404.log.gz 2023-10-16T11:22:58.777 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160056.log.gz 2023-10-16T11:22:58.777 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193653.log 2023-10-16T11:22:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170128.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.184589.log.gzgzip 2023-10-16T11:22:58.778 INFO:teuthology.orchestra.run.smithi181.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.164515.log 2023-10-16T11:22:58.778 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0%/var/log/ceph/ceph-client.admin.193653.log: -- replaced with /var/log/ceph/ceph-client.admin.170128.log.gz 2023-10-16T11:22:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.161738.log 2023-10-16T11:22:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164515.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.193653.log.gz 2023-10-16T11:22:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168878.log 2023-10-16T11:22:58.779 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.161738.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164515.log.gz 2023-10-16T11:22:58.780 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131606.log 2023-10-16T11:22:58.780 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.161738.log.gz 2023-10-16T11:22:58.780 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151384.log 2023-10-16T11:22:58.780 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131606.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168878.log.gz 2023-10-16T11:22:58.781 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111371.log 2023-10-16T11:22:58.781 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131606.log.gz 2023-10-16T11:22:58.781 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134446.log 2023-10-16T11:22:58.782 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151384.log.gz 2023-10-16T11:22:58.782 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143236.log 2023-10-16T11:22:58.782 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134446.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111371.log.gz 2023-10-16T11:22:58.783 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174858.log 2023-10-16T11:22:58.783 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143236.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.134446.log.gz 2023-10-16T11:22:58.783 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151068.log 2023-10-16T11:22:58.783 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191042.log 2023-10-16T11:22:58.784 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143236.log.gz 2023-10-16T11:22:58.784 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151068.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.174858.log.gz 2023-10-16T11:22:58.784 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140278.log 2023-10-16T11:22:58.785 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191042.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151068.log.gz 2023-10-16T11:22:58.785 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T11:22:58.785 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7%/var/log/ceph/ceph-client.admin.140278.log: -- replaced with /var/log/ceph/ceph-client.admin.191042.log.gz 2023-10-16T11:22:58.785 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120319.log 2023-10-16T11:22:58.786 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140278.log.gz 2023-10-16T11:22:58.786 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147092.log 2023-10-16T11:22:58.786 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162794.log 2023-10-16T11:22:58.787 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147092.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120319.log.gz 2023-10-16T11:22:58.787 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181168.log 2023-10-16T11:22:58.787 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147092.log.gz 2023-10-16T11:22:58.787 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181007.log 2023-10-16T11:22:58.788 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162794.log.gz 2023-10-16T11:22:58.788 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126406.log 2023-10-16T11:22:58.788 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181168.log: /var/log/ceph/ceph-client.admin.181007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111001.log 2023-10-16T11:22:58.789 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.181168.log.gz 2023-10-16T11:22:58.789 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181007.log.gz 2023-10-16T11:22:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121154.log 2023-10-16T11:22:58.789 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117341.log 2023-10-16T11:22:58.790 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126406.log.gz 2023-10-16T11:22:58.790 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121568.log 2023-10-16T11:22:58.790 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111001.log.gz 2023-10-16T11:22:58.790 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121154.log.gz 2023-10-16T11:22:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136406.log 2023-10-16T11:22:58.791 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117341.log: /var/log/ceph/ceph-client.admin.121568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119158.log 2023-10-16T11:22:58.791 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117341.log.gz 2023-10-16T11:22:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143394.log 2023-10-16T11:22:58.792 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121568.log.gz 2023-10-16T11:22:58.792 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136406.log.gz 2023-10-16T11:22:58.792 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119158.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.97607.log 2023-10-16T11:22:58.793 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119158.log.gz 2023-10-16T11:22:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143670.log 2023-10-16T11:22:58.793 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143394.log: /var/log/ceph/ceph.tmp-client.admin.97607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121292.log 2023-10-16T11:22:58.794 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.97607.log.gz 2023-10-16T11:22:58.794 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.143394.log.gz 2023-10-16T11:22:58.794 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138814.log 2023-10-16T11:22:58.794 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121292.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143670.log.gz 2023-10-16T11:22:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144725.log 2023-10-16T11:22:58.795 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121292.log.gz 2023-10-16T11:22:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152500.log 2023-10-16T11:22:58.795 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138814.log: /var/log/ceph/ceph-client.admin.144725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181903.log 2023-10-16T11:22:58.796 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.138814.log.gz 2023-10-16T11:22:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152500.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144725.log.gz 2023-10-16T11:22:58.796 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116889.log 2023-10-16T11:22:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123804.log 2023-10-16T11:22:58.797 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152500.log.gz 2023-10-16T11:22:58.797 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181903.log: /var/log/ceph/ceph-client.admin.116889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137799.log 2023-10-16T11:22:58.797 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.181903.log.gz 2023-10-16T11:22:58.798 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116889.log.gz 2023-10-16T11:22:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.123804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168282.log 2023-10-16T11:22:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144587.log 2023-10-16T11:22:58.798 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.137799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123804.log.gz 2023-10-16T11:22:58.799 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.137799.log.gz 2023-10-16T11:22:58.799 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169792.log 2023-10-16T11:22:58.799 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168282.log.gz 2023-10-16T11:22:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148947.log 2023-10-16T11:22:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182041.log 2023-10-16T11:22:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169792.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.144587.log.gz 2023-10-16T11:22:58.800 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135935.log 2023-10-16T11:22:58.801 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.169792.log.gz 2023-10-16T11:22:58.801 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148947.log.gz 2023-10-16T11:22:58.801 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143172.log 2023-10-16T11:22:58.801 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120225.log 2023-10-16T11:22:58.802 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.182041.log.gz 2023-10-16T11:22:58.802 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173947.log 2023-10-16T11:22:58.802 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135935.log.gz 2023-10-16T11:22:58.802 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120225.log: gzip -5 88.7%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.143172.log.gz/var/log/ceph/ceph-client.admin.175277.log 2023-10-16T11:22:58.803 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:22:58.803 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T11:22:58.803 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173947.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120225.log.gz 2023-10-16T11:22:58.804 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183690.log 2023-10-16T11:22:58.804 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175277.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173947.log.gz 2023-10-16T11:22:58.804 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168385.log 2023-10-16T11:22:58.804 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183690.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175277.log.gz 2023-10-16T11:22:58.805 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132026.log 2023-10-16T11:22:58.805 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.183690.log.gz 2023-10-16T11:22:58.805 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.170753.log 2023-10-16T11:22:58.805 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132026.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168385.log.gz 2023-10-16T11:22:58.806 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174951.log 2023-10-16T11:22:58.806 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.170753.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132026.log.gz 2023-10-16T11:22:58.806 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186287.log 2023-10-16T11:22:58.807 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.170753.log.gz 2023-10-16T11:22:58.807 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147344.log 2023-10-16T11:22:58.807 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186287.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.174951.log.gz 2023-10-16T11:22:58.807 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163679.log 2023-10-16T11:22:58.808 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.186287.log.gz 2023-10-16T11:22:58.808 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175231.log 2023-10-16T11:22:58.808 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163679.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147344.log.gz 2023-10-16T11:22:58.808 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176505.log 2023-10-16T11:22:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175231.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163679.log.gz 2023-10-16T11:22:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115393.log 2023-10-16T11:22:58.809 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175231.log.gz 2023-10-16T11:22:58.809 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170450.log 2023-10-16T11:22:58.810 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.176505.log.gz 2023-10-16T11:22:58.810 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193423.log 2023-10-16T11:22:58.810 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170450.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115393.log.gz 2023-10-16T11:22:58.811 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141120.log 2023-10-16T11:22:58.811 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180497.log 2023-10-16T11:22:58.811 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.193423.log.gz 2023-10-16T11:22:58.811 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.170450.log.gz 2023-10-16T11:22:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.141120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124722.log 2023-10-16T11:22:58.812 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1%/var/log/ceph/ceph-client.admin.180497.log: -- replaced with /var/log/ceph/ceph-client.admin.141120.log.gz 2023-10-16T11:22:58.812 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118191.log 2023-10-16T11:22:58.813 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.180497.log.gz 2023-10-16T11:22:58.813 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.124722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131825.log 2023-10-16T11:22:58.813 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118191.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124722.log.gzgzip 2023-10-16T11:22:58.813 INFO:teuthology.orchestra.run.smithi181.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.147969.log 2023-10-16T11:22:58.814 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115347.log 2023-10-16T11:22:58.814 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118191.log.gz 2023-10-16T11:22:58.814 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131825.log.gz 2023-10-16T11:22:58.814 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118145.log 2023-10-16T11:22:58.815 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147969.log.gz 2023-10-16T11:22:58.815 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110021.log 2023-10-16T11:22:58.815 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181490.log 2023-10-16T11:22:58.816 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115347.log.gz 2023-10-16T11:22:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110021.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118145.log.gz 2023-10-16T11:22:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190207.log 2023-10-16T11:22:58.816 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191986.log 2023-10-16T11:22:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181490.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110021.log.gz 2023-10-16T11:22:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190253.log 2023-10-16T11:22:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191986.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.181490.log.gz 2023-10-16T11:22:58.817 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147295.log 2023-10-16T11:22:58.818 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.190207.log.gz 2023-10-16T11:22:58.818 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.191986.log.gz 2023-10-16T11:22:58.818 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132970.log 2023-10-16T11:22:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117433.log 2023-10-16T11:22:58.819 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.190253.log.gz 2023-10-16T11:22:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132970.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147295.log.gz 2023-10-16T11:22:58.819 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140707.log 2023-10-16T11:22:58.820 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132970.log.gz 2023-10-16T11:22:58.820 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117433.log: /var/log/ceph/ceph-client.admin.149682.log 2023-10-16T11:22:58.820 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181053.log 2023-10-16T11:22:58.820 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117433.log.gz 2023-10-16T11:22:58.821 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149682.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140707.log.gz 2023-10-16T11:22:58.821 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131876.log 2023-10-16T11:22:58.821 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.149682.log.gz 2023-10-16T11:22:58.822 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139903.log 2023-10-16T11:22:58.822 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170174.log 2023-10-16T11:22:58.822 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.181053.log.gz 2023-10-16T11:22:58.822 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139903.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131876.log.gz 2023-10-16T11:22:58.823 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153487.log 2023-10-16T11:22:58.823 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139903.log.gz 2023-10-16T11:22:58.823 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185447.log 2023-10-16T11:22:58.823 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170174.log: /var/log/ceph/ceph-client.admin.153487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181352.log 2023-10-16T11:22:58.824 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170174.log.gz 2023-10-16T11:22:58.824 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185447.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.153487.log.gz 2023-10-16T11:22:58.824 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140613.log 2023-10-16T11:22:58.825 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182971.log 2023-10-16T11:22:58.825 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181352.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.185447.log.gz 2023-10-16T11:22:58.825 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140613.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.181352.log.gz 2023-10-16T11:22:58.825 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118099.log 2023-10-16T11:22:58.826 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140613.log.gz 2023-10-16T11:22:58.826 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153649.log 2023-10-16T11:22:58.826 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118099.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.182971.log.gz 2023-10-16T11:22:58.826 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142725.log 2023-10-16T11:22:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186701.log 2023-10-16T11:22:58.827 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118099.log.gz 2023-10-16T11:22:58.827 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142725.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.153649.log.gz 2023-10-16T11:22:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153257.log 2023-10-16T11:22:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186701.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142725.log.gz 2023-10-16T11:22:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179496.log 2023-10-16T11:22:58.828 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186701.log.gz 2023-10-16T11:22:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131773.log 2023-10-16T11:22:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179496.log: 88.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.153257.log.gz--verbose 2023-10-16T11:22:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:-- /var/log/ceph/ceph-client.admin.173078.log 2023-10-16T11:22:58.829 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131773.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179496.log.gz 2023-10-16T11:22:58.830 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168332.log 2023-10-16T11:22:58.830 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131773.log.gz 2023-10-16T11:22:58.830 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144679.log 2023-10-16T11:22:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168332.log: 87.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.173078.log.gz--verbose 2023-10-16T11:22:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:-- /var/log/ceph/ceph-client.admin.127883.log 2023-10-16T11:22:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144679.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168332.log.gz 2023-10-16T11:22:58.831 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135541.log 2023-10-16T11:22:58.832 INFO:teuthology.orchestra.run.smithi181.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.144679.log.gz 2023-10-16T11:22:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164047.log 2023-10-16T11:22:58.832 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132631.log 2023-10-16T11:22:58.833 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127883.log.gz 2023-10-16T11:22:58.833 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135541.log.gz 2023-10-16T11:22:58.833 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192988.log 2023-10-16T11:22:58.833 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.164047.log.gz 2023-10-16T11:22:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180389.log 2023-10-16T11:22:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143716.log 2023-10-16T11:22:58.834 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132631.log.gz 2023-10-16T11:22:58.834 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180389.log: 88.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.192988.log.gz--verbose 2023-10-16T11:22:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:-- /var/log/ceph/ceph-client.admin.119675.log 2023-10-16T11:22:58.835 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.180389.log.gz 2023-10-16T11:22:58.835 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149269.log 2023-10-16T11:22:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119675.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143716.log.gz 2023-10-16T11:22:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162340.log 2023-10-16T11:22:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149269.log: gzip 88.8%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.119675.log.gz-- 2023-10-16T11:22:58.836 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.0.log 2023-10-16T11:22:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145330.log 2023-10-16T11:22:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.0.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162340.log.gz 2023-10-16T11:22:58.837 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.149269.log.gz 2023-10-16T11:22:58.837 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128434.log 2023-10-16T11:22:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.145330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.165441.log 2023-10-16T11:22:58.838 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.145330.log.gz 2023-10-16T11:22:58.838 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.141357.log 2023-10-16T11:22:58.839 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128434.log.gz 2023-10-16T11:22:58.839 INFO:teuthology.orchestra.run.smithi181.stderr:gzip /var/log/ceph/ceph-client.0.165441.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.114437.log 2023-10-16T11:22:58.839 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.165441.log.gz 2023-10-16T11:22:58.839 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.141357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186563.log 2023-10-16T11:22:58.840 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.141357.log.gz 2023-10-16T11:22:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140891.log 2023-10-16T11:22:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186563.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114437.log.gz 2023-10-16T11:22:58.840 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136176.log 2023-10-16T11:22:58.841 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.186563.log.gz 2023-10-16T11:22:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133840.log 2023-10-16T11:22:58.841 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140891.log: /var/log/ceph/ceph-client.admin.136176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137955.log 2023-10-16T11:22:58.842 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140891.log.gz 2023-10-16T11:22:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136176.log.gz 2023-10-16T11:22:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152931.log 2023-10-16T11:22:58.842 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137955.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133840.log.gz 2023-10-16T11:22:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165208.log 2023-10-16T11:22:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152931.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137955.log.gz 2023-10-16T11:22:58.843 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161044.log 2023-10-16T11:22:58.844 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152931.log.gz 2023-10-16T11:22:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.165208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127515.log 2023-10-16T11:22:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161044.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165208.log.gz 2023-10-16T11:22:58.844 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118007.log 2023-10-16T11:22:58.845 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161044.log.gz 2023-10-16T11:22:58.845 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185125.log 2023-10-16T11:22:58.845 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127515.log.gz 2023-10-16T11:22:58.846 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193745.log 2023-10-16T11:22:58.846 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118007.log: /var/log/ceph/ceph-client.admin.185125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173695.log 2023-10-16T11:22:58.846 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118007.log.gz 2023-10-16T11:22:58.846 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.185125.log.gz 2023-10-16T11:22:58.847 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161505.log 2023-10-16T11:22:58.847 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110455.log 2023-10-16T11:22:58.847 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173695.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.193745.log.gz 2023-10-16T11:22:58.847 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161505.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.173695.log.gz-5 2023-10-16T11:22:58.848 INFO:teuthology.orchestra.run.smithi181.stderr:--verbose -- /var/log/ceph/ceph-client.admin.190537.log 2023-10-16T11:22:58.848 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161505.log.gz 2023-10-16T11:22:58.848 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170864.log 2023-10-16T11:22:58.849 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184397.log 2023-10-16T11:22:58.849 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110455.log.gz 2023-10-16T11:22:58.849 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164678.log 2023-10-16T11:22:58.849 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.170864.log.gz 2023-10-16T11:22:58.850 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184397.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.190537.log.gz 2023-10-16T11:22:58.850 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.184397.log.gz 2023-10-16T11:22:58.850 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164678.log.gz 2023-10-16T11:22:58.850 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164155.log 2023-10-16T11:22:58.851 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148667.log 2023-10-16T11:22:58.851 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128021.log 2023-10-16T11:22:58.851 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148667.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164155.log.gz 2023-10-16T11:22:58.852 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154180.log 2023-10-16T11:22:58.852 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148667.log.gz 2023-10-16T11:22:58.852 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189392.log 2023-10-16T11:22:58.852 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128021.log: /var/log/ceph/ceph-client.admin.154180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137126.log 2023-10-16T11:22:58.853 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189392.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154180.log.gz 2023-10-16T11:22:58.853 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.128021.log.gz 2023-10-16T11:22:58.853 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.189392.log.gz 2023-10-16T11:22:58.853 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151433.log 2023-10-16T11:22:58.854 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137126.log.gz 2023-10-16T11:22:58.854 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160801.log 2023-10-16T11:22:58.854 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132769.log 2023-10-16T11:22:58.854 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160801.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151433.log.gz 2023-10-16T11:22:58.855 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168115.log 2023-10-16T11:22:58.855 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160801.log.gz 2023-10-16T11:22:58.855 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158745.log 2023-10-16T11:22:58.856 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132769.log: /var/log/ceph/ceph-client.admin.168115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110593.log 2023-10-16T11:22:58.856 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132769.log.gz 2023-10-16T11:22:58.856 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.158745.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168115.log.gz 2023-10-16T11:22:58.856 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113504.log 2023-10-16T11:22:58.857 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.158745.log.gz 2023-10-16T11:22:58.857 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176551.log 2023-10-16T11:22:58.857 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110593.log.gz 2023-10-16T11:22:58.857 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164954.log 2023-10-16T11:22:58.858 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.113504.log: /var/log/ceph/ceph-client.admin.176551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174207.log 2023-10-16T11:22:58.858 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113504.log.gz 2023-10-16T11:22:58.858 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164954.log: gzip -5 88.2%--verbose -- /var/log/ceph/ceph-client.admin.111740.log -- replaced with /var/log/ceph/ceph-client.admin.176551.log.gz 2023-10-16T11:22:58.859 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:22:58.859 INFO:teuthology.orchestra.run.smithi181.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164954.log.gz 2023-10-16T11:22:58.859 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174207.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-16T11:22:58.859 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174207.log.gz 2023-10-16T11:22:58.860 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192674.log 2023-10-16T11:22:58.860 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111740.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160102.log 2023-10-16T11:22:58.860 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111740.log.gz 2023-10-16T11:22:58.860 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186191.log 2023-10-16T11:22:58.861 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.192674.log.gz 2023-10-16T11:22:58.861 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160102.log: /var/log/ceph/ceph-client.admin.186191.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160102.log.gz 2023-10-16T11:22:58.861 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.186191.log.gz 2023-10-16T11:22:58.861 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142407.log 2023-10-16T11:22:58.862 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114529.log 2023-10-16T11:22:58.862 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126208.log 2023-10-16T11:22:58.862 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114529.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142407.log.gz 2023-10-16T11:22:58.863 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114529.log.gz 2023-10-16T11:22:58.863 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144280.log 2023-10-16T11:22:58.863 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126208.log.gz 2023-10-16T11:22:58.863 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135339.log 2023-10-16T11:22:58.864 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165717.log 2023-10-16T11:22:58.864 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135339.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144280.log.gz 2023-10-16T11:22:58.864 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135339.log.gz 2023-10-16T11:22:58.864 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.165717.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.165717.log.gz 2023-10-16T11:22:58.865 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126159.log 2023-10-16T11:22:58.865 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183587.log 2023-10-16T11:22:58.865 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163465.log 2023-10-16T11:22:58.866 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183587.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126159.log.gz 2023-10-16T11:22:58.866 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163465.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.183587.log.gz 2023-10-16T11:22:58.866 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163465.log.gz 2023-10-16T11:22:58.866 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192387.log 2023-10-16T11:22:58.867 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186052.log 2023-10-16T11:22:58.867 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157559.log 2023-10-16T11:22:58.867 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192387.log.gz 2023-10-16T11:22:58.867 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177221.log 2023-10-16T11:22:58.868 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.157559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119742.log 2023-10-16T11:22:58.868 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.186052.log.gz 2023-10-16T11:22:58.868 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.157559.log.gz/var/log/ceph/ceph-client.admin.177221.log: 2023-10-16T11:22:58.869 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182689.log 2023-10-16T11:22:58.869 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119742.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177221.log.gzgzip 2023-10-16T11:22:58.869 INFO:teuthology.orchestra.run.smithi181.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.114161.log 2023-10-16T11:22:58.869 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182689.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119742.log.gz 2023-10-16T11:22:58.870 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.182689.log.gz 2023-10-16T11:22:58.870 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114161.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114161.log.gz 2023-10-16T11:22:58.870 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152270.log 2023-10-16T11:22:58.870 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152977.log 2023-10-16T11:22:58.871 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177059.log 2023-10-16T11:22:58.871 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152977.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152270.log.gz 2023-10-16T11:22:58.871 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111372.log 2023-10-16T11:22:58.872 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177059.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152977.log.gz 2023-10-16T11:22:58.872 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170080.log 2023-10-16T11:22:58.872 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166665.log 2023-10-16T11:22:58.872 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111372.log.gz 2023-10-16T11:22:58.873 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.177059.log.gz 2023-10-16T11:22:58.873 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190399.log 2023-10-16T11:22:58.873 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170080.log: /var/log/ceph/ceph-client.admin.166665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.187121.log 2023-10-16T11:22:58.873 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.170080.log.gz 2023-10-16T11:22:58.874 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190399.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.166665.log.gz 2023-10-16T11:22:58.874 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.187121.log: 0.0% 87.9% -- replaced with /var/log/ceph/ceph-client.0.187121.log.gz -- replaced with /var/log/ceph/ceph-client.admin.190399.log.gz 2023-10-16T11:22:58.874 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:22:58.875 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112638.log 2023-10-16T11:22:58.875 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119541.log 2023-10-16T11:22:58.875 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187682.log 2023-10-16T11:22:58.875 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119541.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.112638.log.gz 2023-10-16T11:22:58.876 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.187682.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119541.log.gz 2023-10-16T11:22:58.876 INFO:teuthology.orchestra.run.smithi181.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.187682.log.gz 2023-10-16T11:22:58.876 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179634.log 2023-10-16T11:22:58.876 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164469.log 2023-10-16T11:22:58.877 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184489.log 2023-10-16T11:22:58.877 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179634.log.gz 2023-10-16T11:22:58.877 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133610.log 2023-10-16T11:22:58.878 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177965.log 2023-10-16T11:22:58.878 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184489.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164469.log.gz 2023-10-16T11:22:58.878 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133610.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.184489.log.gz 2023-10-16T11:22:58.878 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177965.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133610.log.gz 2023-10-16T11:22:58.879 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177965.log.gz 2023-10-16T11:22:58.879 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184175.log 2023-10-16T11:22:58.879 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175436.log 2023-10-16T11:22:58.879 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179726.log 2023-10-16T11:22:58.880 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.184175.log.gz 2023-10-16T11:22:58.880 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152454.log 2023-10-16T11:22:58.880 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111367.log 2023-10-16T11:22:58.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179726.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175436.log.gz 2023-10-16T11:22:58.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181995.log 2023-10-16T11:22:58.881 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179726.log.gz 2023-10-16T11:22:58.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111367.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152454.log.gz 2023-10-16T11:22:58.882 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163863.log 2023-10-16T11:22:58.882 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111367.log.gz 2023-10-16T11:22:58.882 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181560.log 2023-10-16T11:22:58.882 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145276.log 2023-10-16T11:22:58.883 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.181995.log.gz 2023-10-16T11:22:58.883 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181560.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163863.log.gz 2023-10-16T11:22:58.883 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.181560.log.gz 2023-10-16T11:22:58.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.145276.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145276.log.gz 2023-10-16T11:22:58.884 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165138.log 2023-10-16T11:22:58.884 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161964.log 2023-10-16T11:22:58.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.165138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179209.log 2023-10-16T11:22:58.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161964.log: /var/log/ceph/ceph-client.admin.179209.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161964.log.gz 2023-10-16T11:22:58.885 INFO:teuthology.orchestra.run.smithi181.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.165138.log.gz 2023-10-16T11:22:58.885 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.179209.log.gz 2023-10-16T11:22:58.885 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134392.log 2023-10-16T11:22:58.886 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110271.log 2023-10-16T11:22:58.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.141353.log 2023-10-16T11:22:58.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110271.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134392.log.gz 2023-10-16T11:22:58.887 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110271.log.gz 2023-10-16T11:22:58.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.141353.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.0.141353.log.gz--verbose 2023-10-16T11:22:58.887 INFO:teuthology.orchestra.run.smithi181.stderr:-- /var/log/ceph/ceph-client.admin.142031.log 2023-10-16T11:22:58.887 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136970.log 2023-10-16T11:22:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116245.log 2023-10-16T11:22:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136970.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142031.log.gz 2023-10-16T11:22:58.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116245.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136970.log.gz 2023-10-16T11:22:58.888 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116245.log.gz 2023-10-16T11:22:58.889 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140567.log 2023-10-16T11:22:58.889 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176033.log 2023-10-16T11:22:58.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127607.log 2023-10-16T11:22:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176033.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140567.log.gz 2023-10-16T11:22:58.890 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176033.log.gz 2023-10-16T11:22:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127607.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.127607.log.gz 2023-10-16T11:22:58.890 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186239.log 2023-10-16T11:22:58.891 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183851.log 2023-10-16T11:22:58.891 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190491.log 2023-10-16T11:22:58.891 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183851.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.186239.log.gz 2023-10-16T11:22:58.891 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183851.log.gz 2023-10-16T11:22:58.892 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190491.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.190491.log.gz 2023-10-16T11:22:58.892 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134512.log 2023-10-16T11:22:58.892 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169278.log 2023-10-16T11:22:58.893 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111648.log 2023-10-16T11:22:58.893 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134512.log.gz 2023-10-16T11:22:58.893 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169278.log: /var/log/ceph/ceph-client.admin.111648.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.169278.log.gz 2023-10-16T11:22:58.893 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111648.log.gz 2023-10-16T11:22:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133378.log 2023-10-16T11:22:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-16T11:22:58.894 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109883.log 2023-10-16T11:22:58.894 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133378.log.gz 2023-10-16T11:22:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184664.log 2023-10-16T11:22:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138915.log 2023-10-16T11:22:58.895 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.109883.log: /var/log/ceph/ceph-client.admin.184664.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109883.log.gz 2023-10-16T11:22:58.896 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138915.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.184664.log.gz 2023-10-16T11:22:58.896 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138915.log.gz 2023-10-16T11:22:58.896 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137450.log 2023-10-16T11:22:58.896 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174904.log 2023-10-16T11:22:58.897 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164422.log 2023-10-16T11:22:58.897 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137450.log.gz 2023-10-16T11:22:58.897 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174904.log: /var/log/ceph/ceph-client.admin.164422.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174904.log.gz 2023-10-16T11:22:58.897 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.164422.log.gz 2023-10-16T11:22:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160709.log 2023-10-16T11:22:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116751.log 2023-10-16T11:22:58.898 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188422.log 2023-10-16T11:22:58.898 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160709.log.gz 2023-10-16T11:22:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125590.log 2023-10-16T11:22:58.899 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116751.log.gz 2023-10-16T11:22:58.899 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159914.log 2023-10-16T11:22:58.900 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.188422.log.gz 2023-10-16T11:22:58.900 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.125590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112500.log 2023-10-16T11:22:58.900 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125590.log.gz 2023-10-16T11:22:58.900 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117905.log 2023-10-16T11:22:58.901 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159914.log.gz/var/log/ceph/ceph-client.admin.112500.log: 2023-10-16T11:22:58.901 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112500.log.gz 2023-10-16T11:22:58.901 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117905.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117905.log.gz 2023-10-16T11:22:58.901 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117813.log 2023-10-16T11:22:58.902 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125049.log 2023-10-16T11:22:58.902 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117813.log: /var/log/ceph/ceph-client.admin.125049.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117813.log.gz 2023-10-16T11:22:58.902 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125049.log.gz 2023-10-16T11:22:58.902 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151483.log 2023-10-16T11:22:58.903 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182805.log 2023-10-16T11:22:58.903 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187393.log 2023-10-16T11:22:58.903 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151483.log.gz 2023-10-16T11:22:58.904 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182805.log: /var/log/ceph/ceph-client.admin.187393.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182805.log.gz 2023-10-16T11:22:58.904 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.187393.log.gz 2023-10-16T11:22:58.904 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185309.log 2023-10-16T11:22:58.904 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111743.log 2023-10-16T11:22:58.905 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166026.log 2023-10-16T11:22:58.905 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111743.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.185309.log.gz 2023-10-16T11:22:58.905 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111743.log.gz 2023-10-16T11:22:58.905 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113935.log 2023-10-16T11:22:58.906 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.166026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142674.log 2023-10-16T11:22:58.906 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.2.113935.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166026.log.gz 2023-10-16T11:22:58.906 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113935.log.gz 2023-10-16T11:22:58.906 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181214.log 2023-10-16T11:22:58.907 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121634.log 2023-10-16T11:22:58.907 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181214.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.142674.log.gz 2023-10-16T11:22:58.907 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127791.log 2023-10-16T11:22:58.908 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.181214.log.gz 2023-10-16T11:22:58.908 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121634.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121634.log.gz 2023-10-16T11:22:58.908 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188623.log 2023-10-16T11:22:58.908 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117525.log 2023-10-16T11:22:58.909 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188623.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127791.log.gz 2023-10-16T11:22:58.909 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.188623.log.gz 2023-10-16T11:22:58.909 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117525.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117525.log.gz 2023-10-16T11:22:58.909 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149131.log 2023-10-16T11:22:58.910 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115174.log 2023-10-16T11:22:58.910 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149131.log: /var/log/ceph/ceph-client.admin.115174.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149131.log.gz 2023-10-16T11:22:58.910 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115174.log.gz 2023-10-16T11:22:58.911 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189906.log 2023-10-16T11:22:58.911 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169094.log 2023-10-16T11:22:58.911 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122274.log 2023-10-16T11:22:58.911 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.189906.log.gz 2023-10-16T11:22:58.912 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169094.log: /var/log/ceph/ceph-client.admin.122274.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169094.log.gz 2023-10-16T11:22:58.912 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122274.log.gz 2023-10-16T11:22:58.912 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188204.log 2023-10-16T11:22:58.912 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113939.log 2023-10-16T11:22:58.913 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188204.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.188204.log.gz 2023-10-16T11:22:58.913 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.2.113939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113939.log.gz 2023-10-16T11:22:58.913 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179542.log 2023-10-16T11:22:58.913 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151937.log 2023-10-16T11:22:58.915 INFO:teuthology.orchestra.run.smithi190.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-16T11:22:58.976 INFO:teuthology.orchestra.run.smithi190.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-16T11:22:59.017 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179542.log: /var/log/ceph/ceph-client.admin.151937.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.179542.log.gz 2023-10-16T11:22:59.018 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151937.log.gz 2023-10-16T11:22:59.018 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118237.log 2023-10-16T11:22:59.018 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150157.log 2023-10-16T11:22:59.019 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144184.log 2023-10-16T11:22:59.019 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118237.log.gz 2023-10-16T11:22:59.019 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150157.log: /var/log/ceph/ceph-client.admin.144184.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150157.log.gz 2023-10-16T11:22:59.019 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144184.log.gz 2023-10-16T11:22:59.020 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.145120.log 2023-10-16T11:22:59.020 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150704.log 2023-10-16T11:22:59.020 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.145120.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.188156.log 2023-10-16T11:22:59.020 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.0.145120.log.gz 2023-10-16T11:22:59.021 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150704.log: /var/log/ceph/ceph-client.admin.188156.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.150704.log.gz 2023-10-16T11:22:59.021 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.188156.log.gz 2023-10-16T11:22:59.021 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139627.log 2023-10-16T11:22:59.022 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174812.log 2023-10-16T11:22:59.022 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118467.log 2023-10-16T11:22:59.022 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.139627.log.gz 2023-10-16T11:22:59.022 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125355.log 2023-10-16T11:22:59.023 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174812.log.gz 2023-10-16T11:22:59.023 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161435.log 2023-10-16T11:22:59.023 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.125355.log: /var/log/ceph/ceph-client.admin.161435.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125355.log.gz 2023-10-16T11:22:59.024 INFO:teuthology.orchestra.run.smithi181.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.118467.log.gz 2023-10-16T11:22:59.024 INFO:teuthology.orchestra.run.smithi181.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.161435.log.gz 2023-10-16T11:22:59.024 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132585.log 2023-10-16T11:22:59.024 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177703.log 2023-10-16T11:22:59.025 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132585.log: /var/log/ceph/ceph-client.admin.177703.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.132585.log.gz 2023-10-16T11:22:59.025 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138498.log 2023-10-16T11:22:59.025 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.177703.log.gz 2023-10-16T11:22:59.025 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112408.log 2023-10-16T11:22:59.026 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126896.log 2023-10-16T11:22:59.026 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138498.log: /var/log/ceph/ceph-client.admin.112408.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.138498.log.gz 2023-10-16T11:22:59.026 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126896.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112408.log.gz 2023-10-16T11:22:59.027 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126896.log.gz 2023-10-16T11:22:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114760.log 2023-10-16T11:22:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116199.log 2023-10-16T11:22:59.027 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.170757.log 2023-10-16T11:22:59.028 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116199.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114760.log.gz 2023-10-16T11:22:59.028 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.170757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.170757.log.gz 2023-10-16T11:22:59.028 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116199.log.gz 2023-10-16T11:22:59.029 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178435.log 2023-10-16T11:22:59.029 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115973.log 2023-10-16T11:22:59.029 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149728.log 2023-10-16T11:22:59.029 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.178435.log.gz/var/log/ceph/ceph-client.admin.115973.log: 2023-10-16T11:22:59.030 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149728.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115973.log.gz 2023-10-16T11:22:59.030 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149728.log.gz 2023-10-16T11:22:59.030 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191383.log 2023-10-16T11:22:59.030 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175482.log 2023-10-16T11:22:59.031 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160332.log 2023-10-16T11:22:59.031 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175482.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.191383.log.gz 2023-10-16T11:22:59.031 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173276.log 2023-10-16T11:22:59.031 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160332.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175482.log.gz 2023-10-16T11:22:59.032 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160332.log.gz 2023-10-16T11:22:59.032 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173276.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173276.log.gz 2023-10-16T11:22:59.032 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160194.log 2023-10-16T11:22:59.033 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182367.log 2023-10-16T11:22:59.033 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137566.log 2023-10-16T11:22:59.033 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160194.log.gz 2023-10-16T11:22:59.033 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182367.log: /var/log/ceph/ceph-client.admin.137566.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.182367.log.gz 2023-10-16T11:22:59.034 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182735.log 2023-10-16T11:22:59.034 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.137566.log.gz 2023-10-16T11:22:59.034 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147877.log 2023-10-16T11:22:59.034 INFO:teuthology.orchestra.run.smithi181.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.182735.log.gz 2023-10-16T11:22:59.035 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161138.log 2023-10-16T11:22:59.035 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117953.log 2023-10-16T11:22:59.035 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147877.log.gz 2023-10-16T11:22:59.036 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150638.log 2023-10-16T11:22:59.036 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.161138.log.gz 2023-10-16T11:22:59.036 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117953.log: /var/log/ceph/ceph-client.admin.150638.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117953.log.gz 2023-10-16T11:22:59.036 INFO:teuthology.orchestra.run.smithi181.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.150638.log.gz 2023-10-16T11:22:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151735.log 2023-10-16T11:22:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112178.log 2023-10-16T11:22:59.037 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119067.log 2023-10-16T11:22:59.037 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151735.log.gz 2023-10-16T11:22:59.038 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140983.log 2023-10-16T11:22:59.038 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112178.log.gz 2023-10-16T11:22:59.038 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119067.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119067.log.gz 2023-10-16T11:22:59.039 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170312.log 2023-10-16T11:22:59.039 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180113.log 2023-10-16T11:22:59.039 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170312.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140983.log.gz 2023-10-16T11:22:59.039 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147395.log 2023-10-16T11:22:59.040 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170312.log.gz 2023-10-16T11:22:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180113.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.180113.log.gz 2023-10-16T11:22:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.192135.log 2023-10-16T11:22:59.040 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147395.log: /var/log/ceph/ceph-client.admin.192135.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147395.log.gz 2023-10-16T11:22:59.041 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.192135.log.gz 2023-10-16T11:22:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150224.log 2023-10-16T11:22:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.149576.log 2023-10-16T11:22:59.041 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163213.log 2023-10-16T11:22:59.042 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150224.log.gz 2023-10-16T11:22:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.149576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.149576.log.gz 2023-10-16T11:22:59.042 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136690.log 2023-10-16T11:22:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185079.log 2023-10-16T11:22:59.043 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163213.log.gz 2023-10-16T11:22:59.043 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165597.log 2023-10-16T11:22:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140371.log 2023-10-16T11:22:59.044 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136690.log.gz 2023-10-16T11:22:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185079.log: /var/log/ceph/ceph-client.admin.165597.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.185079.log.gz 2023-10-16T11:22:59.044 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140371.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.165597.log.gz 2023-10-16T11:22:59.045 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.140371.log.gz 2023-10-16T11:22:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131724.log 2023-10-16T11:22:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184803.log 2023-10-16T11:22:59.045 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150503.log 2023-10-16T11:22:59.046 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184803.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131724.log.gz 2023-10-16T11:22:59.046 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145467.log 2023-10-16T11:22:59.046 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176599.log 2023-10-16T11:22:59.047 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.184803.log.gz 2023-10-16T11:22:59.047 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150503.log: /var/log/ceph/ceph-client.admin.145467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112224.log 2023-10-16T11:22:59.047 INFO:teuthology.orchestra.run.smithi181.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.150503.log.gz 2023-10-16T11:22:59.047 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176599.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145467.log.gz 2023-10-16T11:22:59.048 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.176599.log.gz 2023-10-16T11:22:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112224.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112224.log.gz 2023-10-16T11:22:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164563.log 2023-10-16T11:22:59.048 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160617.log 2023-10-16T11:22:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164563.log: /var/log/ceph/ceph-client.admin.160617.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164563.log.gz 2023-10-16T11:22:59.049 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.160617.log.gz 2023-10-16T11:22:59.049 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141464.log 2023-10-16T11:22:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116705.log 2023-10-16T11:22:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.141464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175528.log 2023-10-16T11:22:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116705.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141464.log.gz 2023-10-16T11:22:59.050 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175528.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116705.log.gz 2023-10-16T11:22:59.051 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175528.log.gz 2023-10-16T11:22:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136604.log 2023-10-16T11:22:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185585.log 2023-10-16T11:22:59.051 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176783.log 2023-10-16T11:22:59.052 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136604.log.gz 2023-10-16T11:22:59.052 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179117.log 2023-10-16T11:22:59.052 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.185585.log.gz 2023-10-16T11:22:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185033.log 2023-10-16T11:22:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179117.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176783.log.gz 2023-10-16T11:22:59.053 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.179117.log.gz 2023-10-16T11:22:59.053 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136922.log 2023-10-16T11:22:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160571.log 2023-10-16T11:22:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136922.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.185033.log.gz 2023-10-16T11:22:59.054 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.136922.log.gz 2023-10-16T11:22:59.054 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160571.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160571.log.gz 2023-10-16T11:22:59.055 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164632.log 2023-10-16T11:22:59.055 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193883.log 2023-10-16T11:22:59.055 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164632.log: /var/log/ceph/ceph-client.admin.193883.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164632.log.gz 2023-10-16T11:22:59.056 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.193883.log.gz 2023-10-16T11:22:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180953.log 2023-10-16T11:22:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131558.log 2023-10-16T11:22:59.056 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140937.log 2023-10-16T11:22:59.057 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.180953.log.gz 2023-10-16T11:22:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131558.log: /var/log/ceph/ceph-client.admin.140937.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131558.log.gz 2023-10-16T11:22:59.057 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150369.log 2023-10-16T11:22:59.058 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140937.log.gz 2023-10-16T11:22:59.058 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145230.log 2023-10-16T11:22:59.058 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109798.log 2023-10-16T11:22:59.058 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150369.log.gz 2023-10-16T11:22:59.059 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.145230.log: /var/log/ceph/ceph-client.admin.109798.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145230.log.gz 2023-10-16T11:22:59.059 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109798.log.gz 2023-10-16T11:22:59.059 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168433.log 2023-10-16T11:22:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159815.log 2023-10-16T11:22:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125490.log 2023-10-16T11:22:59.060 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168433.log.gz 2023-10-16T11:22:59.060 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134002.log 2023-10-16T11:22:59.061 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159815.log.gz 2023-10-16T11:22:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.125490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146777.log 2023-10-16T11:22:59.061 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134002.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125490.log.gz 2023-10-16T11:22:59.061 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.134002.log.gz 2023-10-16T11:22:59.062 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.146777.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146777.log.gz 2023-10-16T11:22:59.062 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147596.log 2023-10-16T11:22:59.062 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148621.log 2023-10-16T11:22:59.062 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147596.log: /var/log/ceph/ceph-client.admin.148621.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.147596.log.gz 2023-10-16T11:22:59.063 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148621.log.gz 2023-10-16T11:22:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178800.log 2023-10-16T11:22:59.063 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112592.log 2023-10-16T11:22:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182227.log 2023-10-16T11:22:59.064 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5%/var/log/ceph/ceph-client.admin.112592.log: -- replaced with /var/log/ceph/ceph-client.admin.178800.log.gz 2023-10-16T11:22:59.064 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182227.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112592.log.gz 2023-10-16T11:22:59.064 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.182227.log.gz 2023-10-16T11:22:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152132.log 2023-10-16T11:22:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133656.log 2023-10-16T11:22:59.065 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104332.log 2023-10-16T11:22:59.065 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.152132.log.gz 2023-10-16T11:22:59.066 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133656.log: /var/log/ceph/ceph-client.admin.104332.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133656.log.gz 2023-10-16T11:22:59.066 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104332.log.gz 2023-10-16T11:22:59.066 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164908.log 2023-10-16T11:22:59.067 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158879.log 2023-10-16T11:22:59.067 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164376.log 2023-10-16T11:22:59.067 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164908.log.gz 2023-10-16T11:22:59.067 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.158879.log: /var/log/ceph/ceph-client.admin.164376.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.158879.log.gz 2023-10-16T11:22:59.068 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164376.log.gz 2023-10-16T11:22:59.068 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148715.log 2023-10-16T11:22:59.068 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139673.log 2023-10-16T11:22:59.068 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140753.log 2023-10-16T11:22:59.069 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148715.log.gz 2023-10-16T11:22:59.069 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120131.log 2023-10-16T11:22:59.069 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139673.log.gz 2023-10-16T11:22:59.070 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140753.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140753.log.gz 2023-10-16T11:22:59.070 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-16T11:22:59.070 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136268.log 2023-10-16T11:22:59.070 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.c.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120131.log.gz 2023-10-16T11:22:59.071 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136268.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136268.log.gz 2023-10-16T11:22:59.071 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114345.log 2023-10-16T11:22:59.071 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188674.log 2023-10-16T11:22:59.071 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114345.log: /var/log/ceph/ceph-client.admin.188674.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114345.log.gz 2023-10-16T11:22:59.072 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.188674.log.gz 2023-10-16T11:22:59.072 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126620.log 2023-10-16T11:22:59.072 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185263.log 2023-10-16T11:22:59.073 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126620.log: /var/log/ceph/ceph-client.admin.185263.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126620.log.gz 2023-10-16T11:22:59.073 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185263.log.gz 2023-10-16T11:22:59.073 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137909.log 2023-10-16T11:22:59.073 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174299.log 2023-10-16T11:22:59.074 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137909.log: /var/log/ceph/ceph-client.admin.174299.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.137909.log.gz 2023-10-16T11:22:59.074 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174299.log.gz 2023-10-16T11:22:59.074 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192034.log 2023-10-16T11:22:59.075 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126528.log 2023-10-16T11:22:59.075 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192034.log: /var/log/ceph/ceph-client.admin.126528.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.192034.log.gz 2023-10-16T11:22:59.075 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126528.log.gz 2023-10-16T11:22:59.075 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173646.log 2023-10-16T11:22:59.076 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184125.log 2023-10-16T11:22:59.076 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173646.log: /var/log/ceph/ceph-client.admin.184125.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173646.log.gz 2023-10-16T11:22:59.076 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.184125.log.gz 2023-10-16T11:22:59.076 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182597.log 2023-10-16T11:22:59.077 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110996.log 2023-10-16T11:22:59.077 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182597.log: /var/log/ceph/ceph-client.admin.110996.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.182597.log.gz 2023-10-16T11:22:59.077 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110996.log.gz 2023-10-16T11:22:59.077 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114714.log 2023-10-16T11:22:59.078 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159861.log 2023-10-16T11:22:59.078 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114714.log: /var/log/ceph/ceph-client.admin.159861.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114714.log.gz 2023-10-16T11:22:59.078 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159861.log.gz 2023-10-16T11:22:59.079 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125806.log 2023-10-16T11:22:59.079 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135591.log 2023-10-16T11:22:59.079 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.125806.log: /var/log/ceph/ceph-client.admin.135591.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125806.log.gz 2023-10-16T11:22:59.079 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.135591.log.gz 2023-10-16T11:22:59.080 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179818.log 2023-10-16T11:22:59.080 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136130.log 2023-10-16T11:22:59.080 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179818.log: /var/log/ceph/ceph-client.admin.136130.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.179818.log.gz 2023-10-16T11:22:59.081 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136130.log.gz 2023-10-16T11:22:59.081 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148901.log 2023-10-16T11:22:59.081 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142455.log 2023-10-16T11:22:59.081 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148901.log: /var/log/ceph/ceph-client.admin.142455.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.148901.log.gz 2023-10-16T11:22:59.082 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142455.log.gz 2023-10-16T11:22:59.082 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162102.log 2023-10-16T11:22:59.082 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139535.log 2023-10-16T11:22:59.082 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162102.log: /var/log/ceph/ceph-client.admin.139535.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162102.log.gz 2023-10-16T11:22:59.083 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139535.log.gz 2023-10-16T11:22:59.083 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176737.log 2023-10-16T11:22:59.083 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126804.log 2023-10-16T11:22:59.083 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176737.log: /var/log/ceph/ceph-client.admin.126804.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.176737.log.gz 2023-10-16T11:22:59.084 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.126804.log.gz 2023-10-16T11:22:59.084 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152876.log 2023-10-16T11:22:59.084 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127419.log 2023-10-16T11:22:59.085 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189438.log 2023-10-16T11:22:59.085 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127419.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152876.log.gz 2023-10-16T11:22:59.085 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189438.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127419.log.gz 2023-10-16T11:22:59.085 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191569.log 2023-10-16T11:22:59.086 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.189438.log.gz 2023-10-16T11:22:59.086 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190996.log 2023-10-16T11:22:59.086 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191569.log: /var/log/ceph/ceph-client.admin.190996.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.191569.log.gz 2023-10-16T11:22:59.086 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.190996.log.gz 2023-10-16T11:22:59.087 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146876.log 2023-10-16T11:22:59.087 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121384.log 2023-10-16T11:22:59.087 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.146876.log: 88.0%/var/log/ceph/ceph-client.admin.121384.log: -- replaced with /var/log/ceph/ceph-client.admin.146876.log.gz 2023-10-16T11:22:59.088 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121384.log.gz 2023-10-16T11:22:59.088 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143999.log 2023-10-16T11:22:59.088 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171030.log 2023-10-16T11:22:59.088 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143999.log: /var/log/ceph/ceph-client.admin.171030.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143999.log.gz 2023-10-16T11:22:59.089 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171030.log.gz 2023-10-16T11:22:59.089 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183635.log 2023-10-16T11:22:59.089 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115439.log 2023-10-16T11:22:59.090 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183635.log: /var/log/ceph/ceph-client.admin.115439.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.183635.log.gz 2023-10-16T11:22:59.090 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115439.log.gz 2023-10-16T11:22:59.090 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115665.log 2023-10-16T11:22:59.090 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139719.log 2023-10-16T11:22:59.091 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115665.log: /var/log/ceph/ceph-client.admin.139719.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115665.log.gz 2023-10-16T11:22:59.091 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139719.log.gz 2023-10-16T11:22:59.091 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168233.log 2023-10-16T11:22:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174437.log 2023-10-16T11:22:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168233.log: /var/log/ceph/ceph-client.admin.174437.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168233.log.gz 2023-10-16T11:22:59.092 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.174437.log.gz 2023-10-16T11:22:59.092 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119943.log 2023-10-16T11:22:59.093 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111741.log 2023-10-16T11:22:59.093 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119943.log: /var/log/ceph/ceph-client.admin.111741.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119943.log.gz 2023-10-16T11:22:59.093 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111741.log.gz 2023-10-16T11:22:59.094 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121832.log 2023-10-16T11:22:59.094 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180159.log 2023-10-16T11:22:59.094 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121832.log: /var/log/ceph/ceph-client.admin.180159.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121832.log.gz 2023-10-16T11:22:59.094 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.180159.log.gz 2023-10-16T11:22:59.095 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112792.log 2023-10-16T11:22:59.095 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118421.log 2023-10-16T11:22:59.095 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112792.log: /var/log/ceph/ceph-client.admin.118421.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112792.log.gz 2023-10-16T11:22:59.095 INFO:teuthology.orchestra.run.smithi181.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118421.log.gz 2023-10-16T11:22:59.096 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186471.log 2023-10-16T11:22:59.096 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118791.log 2023-10-16T11:22:59.096 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186471.log: /var/log/ceph/ceph-client.0.118791.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186471.log.gz 2023-10-16T11:22:59.097 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118791.log.gz 2023-10-16T11:22:59.097 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126574.log 2023-10-16T11:22:59.097 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142622.log 2023-10-16T11:22:59.097 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126574.log: /var/log/ceph/ceph-client.admin.142622.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126574.log.gz 2023-10-16T11:22:59.098 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142622.log.gz 2023-10-16T11:22:59.098 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121246.log 2023-10-16T11:22:59.098 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132151.log 2023-10-16T11:22:59.098 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121246.log: /var/log/ceph/ceph-client.admin.132151.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121246.log.gz 2023-10-16T11:22:59.099 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132151.log.gz 2023-10-16T11:22:59.099 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176125.log 2023-10-16T11:22:59.099 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159517.log 2023-10-16T11:22:59.100 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176125.log: /var/log/ceph/ceph-client.admin.159517.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.176125.log.gz 2023-10-16T11:22:59.100 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159517.log.gz 2023-10-16T11:22:59.100 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169186.log 2023-10-16T11:22:59.100 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150302.log 2023-10-16T11:22:59.101 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169186.log: /var/log/ceph/ceph-client.admin.150302.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169186.log.gz 2023-10-16T11:22:59.101 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.150302.log.gz 2023-10-16T11:22:59.101 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143914.log 2023-10-16T11:22:59.101 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139413.log 2023-10-16T11:22:59.102 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143914.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143914.log.gz 2023-10-16T11:22:59.102 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110121.log 2023-10-16T11:22:59.102 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184941.log 2023-10-16T11:22:59.103 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139413.log.gz 2023-10-16T11:22:59.103 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110121.log: /var/log/ceph/ceph-client.admin.184941.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110121.log.gz 2023-10-16T11:22:59.103 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.184941.log.gz 2023-10-16T11:22:59.103 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139949.log 2023-10-16T11:22:59.104 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187872.log 2023-10-16T11:22:59.104 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139949.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139949.log.gz 2023-10-16T11:22:59.104 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188006.log 2023-10-16T11:22:59.104 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.187872.log: /var/log/ceph/ceph-client.admin.188006.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.187872.log.gz 2023-10-16T11:22:59.105 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.188006.log.gz 2023-10-16T11:22:59.105 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179312.log 2023-10-16T11:22:59.105 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135691.log 2023-10-16T11:22:59.106 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179312.log: /var/log/ceph/ceph-client.admin.135691.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179312.log.gz 2023-10-16T11:22:59.106 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135691.log.gz 2023-10-16T11:22:59.106 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170404.log 2023-10-16T11:22:59.106 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187273.log 2023-10-16T11:22:59.107 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170404.log: /var/log/ceph/ceph-client.admin.187273.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170404.log.gz 2023-10-16T11:22:59.107 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187273.log.gz 2023-10-16T11:22:59.107 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186655.log 2023-10-16T11:22:59.108 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191167.log 2023-10-16T11:22:59.108 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186655.log: /var/log/ceph/ceph-client.admin.191167.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.186655.log.gz 2023-10-16T11:22:59.108 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.191167.log.gz 2023-10-16T11:22:59.108 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115803.log 2023-10-16T11:22:59.109 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139581.log 2023-10-16T11:22:59.109 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115803.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115803.log.gz 2023-10-16T11:22:59.109 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139581.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.139581.log.gz 2023-10-16T11:22:59.109 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151635.log 2023-10-16T11:22:59.110 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135004.log 2023-10-16T11:22:59.110 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151635.log: /var/log/ceph/ceph-client.admin.135004.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151635.log.gz 2023-10-16T11:22:59.110 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135004.log.gz 2023-10-16T11:22:59.111 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118795.log 2023-10-16T11:22:59.111 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160441.log 2023-10-16T11:22:59.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.118795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118795.log.gz 2023-10-16T11:22:59.111 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160441.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160441.log.gz 2023-10-16T11:22:59.112 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170520.log 2023-10-16T11:22:59.112 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148855.log 2023-10-16T11:22:59.112 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170520.log: /var/log/ceph/ceph-client.admin.148855.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.170520.log.gz 2023-10-16T11:22:59.112 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148855.log.gz 2023-10-16T11:22:59.113 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157513.log 2023-10-16T11:22:59.113 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169140.log 2023-10-16T11:22:59.113 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.157513.log: /var/log/ceph/ceph-client.admin.169140.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157513.log.gz 2023-10-16T11:22:59.113 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.169140.log.gz 2023-10-16T11:22:59.114 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144091.log 2023-10-16T11:22:59.114 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140799.log 2023-10-16T11:22:59.114 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144091.log: /var/log/ceph/ceph-client.admin.140799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144091.log.gz 2023-10-16T11:22:59.115 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140799.log.gz 2023-10-16T11:22:59.115 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138648.log 2023-10-16T11:22:59.115 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184757.log 2023-10-16T11:22:59.115 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138648.log: /var/log/ceph/ceph-client.admin.184757.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138648.log.gz 2023-10-16T11:22:59.116 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.184757.log.gz 2023-10-16T11:22:59.116 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178749.log 2023-10-16T11:22:59.116 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191518.log 2023-10-16T11:22:59.116 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178749.log: /var/log/ceph/ceph-client.admin.191518.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.178749.log.gz 2023-10-16T11:22:59.117 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.191518.log.gz 2023-10-16T11:22:59.117 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165551.log 2023-10-16T11:22:59.117 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127141.log 2023-10-16T11:22:59.117 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.165551.log: /var/log/ceph/ceph-client.admin.127141.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165551.log.gz 2023-10-16T11:22:59.118 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184075.log 2023-10-16T11:22:59.118 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127141.log.gz 2023-10-16T11:22:59.118 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147195.log 2023-10-16T11:22:59.119 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184075.log: /var/log/ceph/ceph-client.admin.147195.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.184075.log.gz 2023-10-16T11:22:59.119 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147195.log.gz 2023-10-16T11:22:59.119 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116475.log 2023-10-16T11:22:59.120 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176171.log 2023-10-16T11:22:59.120 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116475.log: /var/log/ceph/ceph-client.admin.176171.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116475.log.gz 2023-10-16T11:22:59.120 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.176171.log.gz 2023-10-16T11:22:59.120 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163909.log 2023-10-16T11:22:59.121 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159098.log 2023-10-16T11:22:59.121 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163909.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163909.log.gz 2023-10-16T11:22:59.121 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159098.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159098.log.gz 2023-10-16T11:22:59.121 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125855.log 2023-10-16T11:22:59.122 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.190840.log 2023-10-16T11:22:59.122 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.125855.log: /var/log/ceph/ceph-client.0.190840.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125855.log.gz 2023-10-16T11:22:59.122 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.190840.log.gz 2023-10-16T11:22:59.123 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161092.log 2023-10-16T11:22:59.123 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114253.log 2023-10-16T11:22:59.123 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161092.log: /var/log/ceph/ceph-client.admin.114253.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161092.log.gz 2023-10-16T11:22:59.123 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114253.log.gz 2023-10-16T11:22:59.124 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122580.log 2023-10-16T11:22:59.124 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193285.log 2023-10-16T11:22:59.124 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.122580.log: /var/log/ceph/ceph-client.admin.193285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122580.log.gz 2023-10-16T11:22:59.125 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193285.log.gz 2023-10-16T11:22:59.125 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154468.log 2023-10-16T11:22:59.125 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115711.log 2023-10-16T11:22:59.125 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.154468.log: /var/log/ceph/ceph-client.admin.115711.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154468.log.gz 2023-10-16T11:22:59.126 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115711.log.gz 2023-10-16T11:22:59.126 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149919.log 2023-10-16T11:22:59.126 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153533.log 2023-10-16T11:22:59.126 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149919.log: /var/log/ceph/ceph-client.admin.153533.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149919.log.gz 2023-10-16T11:22:59.127 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153533.log.gz 2023-10-16T11:22:59.127 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173178.log 2023-10-16T11:22:59.127 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.161742.log 2023-10-16T11:22:59.129 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173178.log: /var/log/ceph/ceph-client.0.161742.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.173178.log.gz 2023-10-16T11:22:59.129 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.161742.log.gz 2023-10-16T11:22:59.129 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192085.log 2023-10-16T11:22:59.130 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184351.log 2023-10-16T11:22:59.130 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192085.log: /var/log/ceph/ceph-client.admin.184351.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.192085.log.gz 2023-10-16T11:22:59.130 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.184351.log.gz 2023-10-16T11:22:59.131 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153025.log 2023-10-16T11:22:59.131 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149339.log 2023-10-16T11:22:59.131 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153025.log: /var/log/ceph/ceph-client.admin.149339.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153025.log.gz 2023-10-16T11:22:59.131 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.149339.log.gz 2023-10-16T11:22:59.132 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114575.log 2023-10-16T11:22:59.132 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183303.log 2023-10-16T11:22:59.132 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114575.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114575.log.gz 2023-10-16T11:22:59.133 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183303.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.183303.log.gz 2023-10-16T11:22:59.133 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184443.log 2023-10-16T11:22:59.133 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182179.log 2023-10-16T11:22:59.133 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184443.log: /var/log/ceph/ceph-client.admin.182179.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.184443.log.gz 2023-10-16T11:22:59.134 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.182179.log.gz 2023-10-16T11:22:59.134 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135124.log 2023-10-16T11:22:59.134 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148015.log 2023-10-16T11:22:59.135 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135124.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135124.log.gz 2023-10-16T11:22:59.135 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148015.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148015.log.gz 2023-10-16T11:22:59.135 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115927.log 2023-10-16T11:22:59.135 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116429.log 2023-10-16T11:22:59.136 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115927.log: /var/log/ceph/ceph-client.admin.116429.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115927.log.gz 2023-10-16T11:22:59.136 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116429.log.gz 2023-10-16T11:22:59.136 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159000.log 2023-10-16T11:22:59.136 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119463.log 2023-10-16T11:22:59.137 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159000.log: /var/log/ceph/ceph-client.admin.119463.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159000.log.gz 2023-10-16T11:22:59.137 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119463.log.gz 2023-10-16T11:22:59.137 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179680.log 2023-10-16T11:22:59.138 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144045.log 2023-10-16T11:22:59.138 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179680.log: /var/log/ceph/ceph-client.admin.144045.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.179680.log.gz 2023-10-16T11:22:59.138 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.144045.log.gz 2023-10-16T11:22:59.138 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146590.log 2023-10-16T11:22:59.139 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192479.log 2023-10-16T11:22:59.139 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.146590.log: /var/log/ceph/ceph-client.admin.192479.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146590.log.gz 2023-10-16T11:22:59.139 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.192479.log.gz 2023-10-16T11:22:59.139 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140659.log 2023-10-16T11:22:59.140 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173394.log 2023-10-16T11:22:59.140 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140659.log: /var/log/ceph/ceph-client.admin.173394.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140659.log.gz 2023-10-16T11:22:59.140 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173394.log.gz 2023-10-16T11:22:59.141 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142775.log 2023-10-16T11:22:59.141 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142573.log 2023-10-16T11:22:59.141 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142775.log: /var/log/ceph/ceph-client.admin.142573.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142775.log.gz 2023-10-16T11:22:59.141 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142573.log.gz 2023-10-16T11:22:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176263.log 2023-10-16T11:22:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177129.log 2023-10-16T11:22:59.142 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176263.log: /var/log/ceph/ceph-client.admin.177129.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176263.log.gz 2023-10-16T11:22:59.142 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.177129.log.gz 2023-10-16T11:22:59.143 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139477.log 2023-10-16T11:22:59.143 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T11:22:59.143 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139477.log: /var/log/ceph/ceph.audit.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139477.log.gz 2023-10-16T11:22:59.144 INFO:teuthology.orchestra.run.smithi181.stderr: 89.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T11:22:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192582.log 2023-10-16T11:22:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138075.log 2023-10-16T11:22:59.144 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192582.log: /var/log/ceph/ceph-client.admin.138075.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.192582.log.gz 2023-10-16T11:22:59.145 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138075.log.gz 2023-10-16T11:22:59.145 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173993.log 2023-10-16T11:22:59.145 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182897.log 2023-10-16T11:22:59.146 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173993.log: /var/log/ceph/ceph-client.admin.182897.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173993.log.gz 2023-10-16T11:22:59.146 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182897.log.gz 2023-10-16T11:22:59.146 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112270.log 2023-10-16T11:22:59.146 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110363.log 2023-10-16T11:22:59.147 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112270.log: /var/log/ceph/ceph-client.admin.110363.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112270.log.gz 2023-10-16T11:22:59.147 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110363.log.gz 2023-10-16T11:22:59.147 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191116.log 2023-10-16T11:22:59.147 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177806.log 2023-10-16T11:22:59.148 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191116.log: /var/log/ceph/ceph-client.admin.177806.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.191116.log.gz 2023-10-16T11:22:59.148 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177806.log.gz 2023-10-16T11:22:59.148 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188967.log 2023-10-16T11:22:59.149 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136314.log 2023-10-16T11:22:59.149 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188967.log: /var/log/ceph/ceph-client.admin.136314.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.188967.log.gz 2023-10-16T11:22:59.149 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136314.log.gz 2023-10-16T11:22:59.149 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116659.log 2023-10-16T11:22:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162746.log 2023-10-16T11:22:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116659.log: /var/log/ceph/ceph-client.admin.162746.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116659.log.gz 2023-10-16T11:22:59.150 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162746.log.gz 2023-10-16T11:22:59.150 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186098.log 2023-10-16T11:22:59.151 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184987.log 2023-10-16T11:22:59.151 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186098.log: /var/log/ceph/ceph-client.admin.184987.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.186098.log.gz 2023-10-16T11:22:59.151 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.184987.log.gz 2023-10-16T11:22:59.152 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179025.log 2023-10-16T11:22:59.152 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111373.log 2023-10-16T11:22:59.152 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179025.log: /var/log/ceph/ceph-client.admin.111373.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.179025.log.gz 2023-10-16T11:22:59.152 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111373.log.gz 2023-10-16T11:22:59.153 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115757.log 2023-10-16T11:22:59.153 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135489.log 2023-10-16T11:22:59.153 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115757.log: /var/log/ceph/ceph-client.admin.135489.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115757.log.gz 2023-10-16T11:22:59.153 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.135489.log.gz 2023-10-16T11:22:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148153.log 2023-10-16T11:22:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193515.log 2023-10-16T11:22:59.154 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148153.log: /var/log/ceph/ceph-client.admin.193515.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148153.log.gz 2023-10-16T11:22:59.155 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.193515.log.gz 2023-10-16T11:22:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148199.log 2023-10-16T11:22:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186379.log 2023-10-16T11:22:59.155 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148199.log: /var/log/ceph/ceph-client.admin.186379.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148199.log.gz 2023-10-16T11:22:59.156 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.186379.log.gz 2023-10-16T11:22:59.156 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174345.log 2023-10-16T11:22:59.156 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116111.log 2023-10-16T11:22:59.157 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174345.log: /var/log/ceph/ceph-client.admin.116111.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174345.log.gz 2023-10-16T11:22:59.157 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116111.log.gz 2023-10-16T11:22:59.157 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172837.log 2023-10-16T11:22:59.157 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154084.log 2023-10-16T11:22:59.158 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.172837.log: /var/log/ceph/ceph-client.admin.154084.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172837.log.gz 2023-10-16T11:22:59.158 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154084.log.gz 2023-10-16T11:22:59.158 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126712.log 2023-10-16T11:22:59.159 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122228.log 2023-10-16T11:22:59.159 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126712.log: /var/log/ceph/ceph-client.admin.122228.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126712.log.gz 2023-10-16T11:22:59.159 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122228.log.gz 2023-10-16T11:22:59.159 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140513.log 2023-10-16T11:22:59.160 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135389.log 2023-10-16T11:22:59.160 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140513.log: /var/log/ceph/ceph-client.admin.135389.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140513.log.gz 2023-10-16T11:22:59.160 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135389.log.gz 2023-10-16T11:22:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184849.log 2023-10-16T11:22:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163633.log 2023-10-16T11:22:59.161 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184849.log: /var/log/ceph/ceph-client.admin.163633.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184849.log.gz 2023-10-16T11:22:59.161 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163633.log.gz 2023-10-16T11:22:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104199.log 2023-10-16T11:22:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138696.log 2023-10-16T11:22:59.162 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.104199.log: /var/log/ceph/ceph-client.admin.138696.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104199.log.gz 2023-10-16T11:22:59.162 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138696.log.gz 2023-10-16T11:22:59.163 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193699.log 2023-10-16T11:22:59.163 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180905.log 2023-10-16T11:22:59.163 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193699.log: /var/log/ceph/ceph-client.admin.180905.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.193699.log.gz 2023-10-16T11:22:59.164 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.180905.log.gz 2023-10-16T11:22:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179928.log 2023-10-16T11:22:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144401.log 2023-10-16T11:22:59.164 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179928.log: /var/log/ceph/ceph-client.admin.144401.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179928.log.gz 2023-10-16T11:22:59.165 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.144401.log.gz 2023-10-16T11:22:59.165 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128328.log 2023-10-16T11:22:59.165 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122096.log 2023-10-16T11:22:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.128328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.128328.log.gz 2023-10-16T11:22:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.122096.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122096.log.gz 2023-10-16T11:22:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178121.log 2023-10-16T11:22:59.166 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192186.log 2023-10-16T11:22:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178121.log: /var/log/ceph/ceph-client.admin.192186.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.178121.log.gz 2023-10-16T11:22:59.167 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192186.log.gz 2023-10-16T11:22:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169886.log 2023-10-16T11:22:59.167 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136828.log 2023-10-16T11:22:59.168 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169886.log: /var/log/ceph/ceph-client.admin.136828.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169886.log.gz 2023-10-16T11:22:59.168 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.136828.log.gz 2023-10-16T11:22:59.168 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111003.log 2023-10-16T11:22:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152316.log 2023-10-16T11:22:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111003.log: /var/log/ceph/ceph-client.admin.152316.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111003.log.gz 2023-10-16T11:22:59.169 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152316.log.gz 2023-10-16T11:22:59.169 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189162.log 2023-10-16T11:22:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185677.log 2023-10-16T11:22:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189162.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.189162.log.gz 2023-10-16T11:22:59.170 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185677.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.185677.log.gz 2023-10-16T11:22:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142257.log 2023-10-16T11:22:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182321.log 2023-10-16T11:22:59.171 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142257.log: /var/log/ceph/ceph-client.admin.182321.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142257.log.gz 2023-10-16T11:22:59.171 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.182321.log.gz 2023-10-16T11:22:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147923.log 2023-10-16T11:22:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190445.log 2023-10-16T11:22:59.172 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147923.log: /var/log/ceph/ceph-client.admin.190445.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147923.log.gz 2023-10-16T11:22:59.172 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.190445.log.gz 2023-10-16T11:22:59.173 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169324.log 2023-10-16T11:22:59.173 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186747.log 2023-10-16T11:22:59.173 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117479.log 2023-10-16T11:22:59.174 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169324.log.gz 2023-10-16T11:22:59.174 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186747.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186747.log.gz 2023-10-16T11:22:59.174 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183904.log 2023-10-16T11:22:59.174 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117479.log: /var/log/ceph/ceph-client.admin.183904.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117479.log.gz 2023-10-16T11:22:59.175 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183904.log.gz 2023-10-16T11:22:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134819.log 2023-10-16T11:22:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147706.log 2023-10-16T11:22:59.175 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134819.log: /var/log/ceph/ceph-client.admin.147706.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.134819.log.gz 2023-10-16T11:22:59.176 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147706.log.gz 2023-10-16T11:22:59.176 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144495.log 2023-10-16T11:22:59.176 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110409.log 2023-10-16T11:22:59.177 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144495.log: /var/log/ceph/ceph-client.admin.110409.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144495.log.gz 2023-10-16T11:22:59.177 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110409.log.gz 2023-10-16T11:22:59.177 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111361.log 2023-10-16T11:22:59.177 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178849.log 2023-10-16T11:22:59.178 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111361.log: /var/log/ceph/ceph-client.admin.178849.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111361.log.gz 2023-10-16T11:22:59.178 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178849.log.gz 2023-10-16T11:22:59.178 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193218.log 2023-10-16T11:22:59.178 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139765.log 2023-10-16T11:22:59.179 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193218.log: /var/log/ceph/ceph-client.admin.139765.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.193218.log.gz 2023-10-16T11:22:59.179 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139765.log.gz 2023-10-16T11:22:59.179 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187319.log 2023-10-16T11:22:59.180 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180857.log 2023-10-16T11:22:59.180 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.187319.log: /var/log/ceph/ceph-client.admin.180857.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.187319.log.gz 2023-10-16T11:22:59.180 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.180857.log.gz 2023-10-16T11:22:59.180 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114391.log 2023-10-16T11:22:59.181 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148061.log 2023-10-16T11:22:59.181 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114391.log: /var/log/ceph/ceph-client.admin.148061.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114391.log.gz 2023-10-16T11:22:59.181 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148061.log.gz 2023-10-16T11:22:59.181 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160898.log 2023-10-16T11:22:59.182 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179450.log 2023-10-16T11:22:59.182 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160898.log: /var/log/ceph/ceph-client.admin.179450.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160898.log.gz 2023-10-16T11:22:59.182 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135889.log 2023-10-16T11:22:59.183 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.179450.log.gz 2023-10-16T11:22:59.183 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161322.log 2023-10-16T11:22:59.183 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135889.log: /var/log/ceph/ceph-client.admin.161322.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135889.log.gz 2023-10-16T11:22:59.183 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.161322.log.gz 2023-10-16T11:22:59.184 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.149572.log 2023-10-16T11:22:59.184 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116935.log 2023-10-16T11:22:59.184 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.149572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.149572.log.gz 2023-10-16T11:22:59.185 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162912.log 2023-10-16T11:22:59.185 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116935.log.gz 2023-10-16T11:22:59.185 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185723.log 2023-10-16T11:22:59.185 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162912.log: /var/log/ceph/ceph-client.admin.185723.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162912.log.gz 2023-10-16T11:22:59.186 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.185723.log.gz 2023-10-16T11:22:59.186 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188574.log 2023-10-16T11:22:59.186 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147042.log 2023-10-16T11:22:59.187 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188574.log: /var/log/ceph/ceph-client.admin.147042.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188574.log.gz 2023-10-16T11:22:59.187 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147042.log.gz 2023-10-16T11:22:59.187 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163771.log 2023-10-16T11:22:59.187 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185921.log 2023-10-16T11:22:59.188 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163771.log: /var/log/ceph/ceph-client.admin.185921.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163771.log.gz 2023-10-16T11:22:59.188 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.185921.log.gz 2023-10-16T11:22:59.188 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111738.log 2023-10-16T11:22:59.189 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173495.log 2023-10-16T11:22:59.189 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111738.log: /var/log/ceph/ceph-client.admin.173495.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111738.log.gz 2023-10-16T11:22:59.189 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.173495.log.gz 2023-10-16T11:22:59.189 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148307.log 2023-10-16T11:22:59.190 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137266.log 2023-10-16T11:22:59.190 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148307.log: /var/log/ceph/ceph-client.admin.137266.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148307.log.gz 2023-10-16T11:22:59.190 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137266.log.gz 2023-10-16T11:22:59.191 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192628.log 2023-10-16T11:22:59.191 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179975.log 2023-10-16T11:22:59.191 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192628.log: /var/log/ceph/ceph-client.admin.179975.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.192628.log.gz 2023-10-16T11:22:59.191 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.179975.log.gz 2023-10-16T11:22:59.192 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138363.log 2023-10-16T11:22:59.192 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139811.log 2023-10-16T11:22:59.192 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138363.log: /var/log/ceph/ceph-client.admin.139811.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.138363.log.gz 2023-10-16T11:22:59.193 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139811.log.gz 2023-10-16T11:22:59.193 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111603.log 2023-10-16T11:22:59.193 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153073.log 2023-10-16T11:22:59.193 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111603.log: /var/log/ceph/ceph-client.admin.153073.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111603.log.gz 2023-10-16T11:22:59.194 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153073.log.gz 2023-10-16T11:22:59.194 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160998.log 2023-10-16T11:22:59.194 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174529.log 2023-10-16T11:22:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160998.log: /var/log/ceph/ceph-client.admin.174529.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.160998.log.gz 2023-10-16T11:22:59.195 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174529.log.gz 2023-10-16T11:22:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.181797.log 2023-10-16T11:22:59.195 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189766.log 2023-10-16T11:22:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.181797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.181797.log.gz 2023-10-16T11:22:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189766.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.189766.log.gz 2023-10-16T11:22:59.196 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186793.log 2023-10-16T11:22:59.197 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180587.log 2023-10-16T11:22:59.197 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186793.log: /var/log/ceph/ceph-client.admin.180587.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.180587.log.gz 2023-10-16T11:22:59.197 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.186793.log.gz 2023-10-16T11:22:59.197 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110685.log 2023-10-16T11:22:59.198 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120084.log 2023-10-16T11:22:59.198 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110685.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110685.log.gz 2023-10-16T11:22:59.198 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120084.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120084.log.gz 2023-10-16T11:22:59.198 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110317.log 2023-10-16T11:22:59.199 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148107.log 2023-10-16T11:22:59.199 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110317.log: /var/log/ceph/ceph-client.admin.148107.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110317.log.gz 2023-10-16T11:22:59.199 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148107.log.gz 2023-10-16T11:22:59.200 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120272.log 2023-10-16T11:22:59.200 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180205.log 2023-10-16T11:22:59.200 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120272.log: /var/log/ceph/ceph-client.admin.180205.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120272.log.gz 2023-10-16T11:22:59.200 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.180205.log.gz 2023-10-16T11:22:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135843.log 2023-10-16T11:22:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175369.log 2023-10-16T11:22:59.201 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135843.log: /var/log/ceph/ceph-client.admin.175369.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135843.log.gz 2023-10-16T11:22:59.202 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.175369.log.gz 2023-10-16T11:22:59.202 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110993.log 2023-10-16T11:22:59.202 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160663.log 2023-10-16T11:22:59.202 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110993.log: /var/log/ceph/ceph-client.admin.160663.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110993.log.gz 2023-10-16T11:22:59.203 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.160663.log.gz 2023-10-16T11:22:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169654.log 2023-10-16T11:22:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185631.log 2023-10-16T11:22:59.203 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169654.log: /var/log/ceph/ceph-client.admin.185631.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169654.log.gz 2023-10-16T11:22:59.204 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.185631.log.gz 2023-10-16T11:22:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121964.log 2023-10-16T11:22:59.204 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117617.log 2023-10-16T11:22:59.205 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121964.log: /var/log/ceph/ceph-client.admin.117617.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121964.log.gz 2023-10-16T11:22:59.205 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117617.log.gz 2023-10-16T11:22:59.205 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191668.log 2023-10-16T11:22:59.205 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162596.log 2023-10-16T11:22:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191668.log: /var/log/ceph/ceph-client.admin.162596.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.191668.log.gz 2023-10-16T11:22:59.206 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162596.log.gz 2023-10-16T11:22:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144633.log 2023-10-16T11:22:59.206 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179163.log 2023-10-16T11:22:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144633.log: /var/log/ceph/ceph-client.admin.179163.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144633.log.gz 2023-10-16T11:22:59.207 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.179163.log.gz 2023-10-16T11:22:59.207 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117387.log 2023-10-16T11:22:59.208 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136084.log 2023-10-16T11:22:59.208 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117387.log: /var/log/ceph/ceph-client.admin.136084.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117387.log.gz 2023-10-16T11:22:59.208 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136084.log.gz 2023-10-16T11:22:59.208 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127467.log 2023-10-16T11:22:59.209 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139857.log 2023-10-16T11:22:59.209 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127467.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127467.log.gz 2023-10-16T11:22:59.209 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187464.log 2023-10-16T11:22:59.209 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139857.log.gz 2023-10-16T11:22:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163064.log 2023-10-16T11:22:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.187464.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.187464.log.gz 2023-10-16T11:22:59.210 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117759.log 2023-10-16T11:22:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163064.log: /var/log/ceph/ceph-client.admin.117759.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163064.log.gz 2023-10-16T11:22:59.211 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117759.log.gz 2023-10-16T11:22:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111005.log 2023-10-16T11:22:59.211 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136222.log 2023-10-16T11:22:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111005.log: /var/log/ceph/ceph-client.admin.136222.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111005.log.gz 2023-10-16T11:22:59.212 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136222.log.gz 2023-10-16T11:22:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168832.log 2023-10-16T11:22:59.212 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136038.log 2023-10-16T11:22:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168832.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168832.log.gz 2023-10-16T11:22:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136038.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136038.log.gz 2023-10-16T11:22:59.213 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133330.log 2023-10-16T11:22:59.214 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133192.log 2023-10-16T11:22:59.214 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133330.log: /var/log/ceph/ceph-client.admin.133192.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.133330.log.gz 2023-10-16T11:22:59.214 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133192.log.gz 2023-10-16T11:22:59.214 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193607.log 2023-10-16T11:22:59.215 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166711.log 2023-10-16T11:22:59.215 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193607.log: /var/log/ceph/ceph-client.admin.166711.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.193607.log.gz 2023-10-16T11:22:59.215 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166711.log.gz 2023-10-16T11:22:59.215 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140147.log 2023-10-16T11:22:59.216 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110075.log 2023-10-16T11:22:59.216 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140147.log: /var/log/ceph/ceph-client.admin.110075.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140147.log.gz 2023-10-16T11:22:59.216 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110075.log.gz 2023-10-16T11:22:59.217 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147444.log 2023-10-16T11:22:59.217 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172943.log 2023-10-16T11:22:59.217 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147444.log: /var/log/ceph/ceph-client.admin.172943.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147444.log.gz 2023-10-16T11:22:59.217 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.172943.log.gz 2023-10-16T11:22:59.218 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182851.log 2023-10-16T11:22:59.218 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190094.log 2023-10-16T11:22:59.218 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182851.log: /var/log/ceph/ceph-client.admin.190094.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.182851.log.gz 2023-10-16T11:22:59.218 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.190094.log.gz 2023-10-16T11:22:59.219 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147831.log 2023-10-16T11:22:59.219 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146484.log 2023-10-16T11:22:59.219 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.147831.log: /var/log/ceph/ceph-client.admin.146484.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147831.log.gz 2023-10-16T11:22:59.219 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146484.log.gz 2023-10-16T11:22:59.220 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138966.log 2023-10-16T11:22:59.220 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186145.log 2023-10-16T11:22:59.220 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138966.log: /var/log/ceph/ceph-client.admin.186145.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138966.log.gz 2023-10-16T11:22:59.221 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.186145.log.gz 2023-10-16T11:22:59.221 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163313.log 2023-10-16T11:22:59.221 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114944.log 2023-10-16T11:22:59.221 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163313.log: /var/log/ceph/ceph-client.admin.114944.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163313.log.gz 2023-10-16T11:22:59.222 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114944.log.gz 2023-10-16T11:22:59.222 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178321.log 2023-10-16T11:22:59.222 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174161.log 2023-10-16T11:22:59.223 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178321.log: /var/log/ceph/ceph-client.admin.174161.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178321.log.gz 2023-10-16T11:22:59.223 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174161.log.gz 2023-10-16T11:22:59.223 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143126.log 2023-10-16T11:22:59.223 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169746.log 2023-10-16T11:22:59.224 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143126.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143126.log.gz 2023-10-16T11:22:59.224 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169746.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169746.log.gz 2023-10-16T11:22:59.224 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122162.log 2023-10-16T11:22:59.224 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142077.log 2023-10-16T11:22:59.225 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.122162.log: /var/log/ceph/ceph-client.admin.142077.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122162.log.gz 2023-10-16T11:22:59.225 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142077.log.gz 2023-10-16T11:22:59.225 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118053.log 2023-10-16T11:22:59.226 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179256.log 2023-10-16T11:22:59.226 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118053.log: /var/log/ceph/ceph-client.admin.179256.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118053.log.gz 2023-10-16T11:22:59.226 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179256.log.gz 2023-10-16T11:22:59.226 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110777.log 2023-10-16T11:22:59.227 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176691.log 2023-10-16T11:22:59.227 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110777.log: /var/log/ceph/ceph-client.admin.176691.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110777.log.gz 2023-10-16T11:22:59.227 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.176691.log.gz 2023-10-16T11:22:59.228 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131148.log 2023-10-16T11:22:59.228 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170962.log 2023-10-16T11:22:59.228 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131148.log: /var/log/ceph/ceph-client.admin.170962.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131148.log.gz 2023-10-16T11:22:59.228 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170962.log.gz 2023-10-16T11:22:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114898.log 2023-10-16T11:22:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148993.log 2023-10-16T11:22:59.229 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114898.log: /var/log/ceph/ceph-client.admin.148993.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114898.log.gz 2023-10-16T11:22:59.229 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148993.log.gz 2023-10-16T11:22:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183042.log 2023-10-16T11:22:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136782.log 2023-10-16T11:22:59.230 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183042.log: /var/log/ceph/ceph-client.admin.136782.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183042.log.gz 2023-10-16T11:22:59.231 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136782.log.gz 2023-10-16T11:22:59.231 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189954.log 2023-10-16T11:22:59.231 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137358.log 2023-10-16T11:22:59.231 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189954.log: /var/log/ceph/ceph-client.admin.137358.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.189954.log.gz 2023-10-16T11:22:59.232 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137358.log.gz 2023-10-16T11:22:59.232 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128622.log 2023-10-16T11:22:59.232 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189636.log 2023-10-16T11:22:59.232 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128622.log: /var/log/ceph/ceph-client.admin.189636.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128622.log.gz 2023-10-16T11:22:59.233 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.189636.log.gz 2023-10-16T11:22:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154038.log 2023-10-16T11:22:59.233 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109947.log 2023-10-16T11:22:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.154038.log: /var/log/ceph/ceph-client.admin.109947.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154038.log.gz 2023-10-16T11:22:59.234 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109947.log.gz 2023-10-16T11:22:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176459.log 2023-10-16T11:22:59.234 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135074.log 2023-10-16T11:22:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176459.log: /var/log/ceph/ceph-client.admin.135074.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.176459.log.gz 2023-10-16T11:22:59.235 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135074.log.gz 2023-10-16T11:22:59.235 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178377.log 2023-10-16T11:22:59.236 INFO:teuthology.orchestra.run.smithi181.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-16T11:22:59.236 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111366.log 2023-10-16T11:22:59.236 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141630.log 2023-10-16T11:22:59.236 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.178377.log.gz 2023-10-16T11:22:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112040.log 2023-10-16T11:22:59.237 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111366.log.gz 2023-10-16T11:22:59.237 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.141630.log: /var/log/ceph/ceph-client.admin.112040.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141630.log.gz 2023-10-16T11:22:59.237 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112040.log.gz 2023-10-16T11:22:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151685.log 2023-10-16T11:22:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140324.log 2023-10-16T11:22:59.238 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184895.log 2023-10-16T11:22:59.239 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151685.log.gz 2023-10-16T11:22:59.239 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140324.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140324.log.gz 2023-10-16T11:22:59.239 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124416.log 2023-10-16T11:22:59.239 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152408.log 2023-10-16T11:22:59.240 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.184895.log.gz 2023-10-16T11:22:59.240 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.124416.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124416.log.gz 2023-10-16T11:22:59.240 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178700.log 2023-10-16T11:22:59.240 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173795.log 2023-10-16T11:22:59.241 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152408.log.gz 2023-10-16T11:22:59.241 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178700.log: 87.8%/var/log/ceph/ceph-client.admin.173795.log: -- replaced with /var/log/ceph/ceph-client.admin.178700.log.gz 2023-10-16T11:22:59.241 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.173795.log.gz 2023-10-16T11:22:59.242 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115531.log 2023-10-16T11:22:59.242 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180251.log 2023-10-16T11:22:59.242 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177175.log 2023-10-16T11:22:59.242 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180251.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115531.log.gz 2023-10-16T11:22:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177175.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.180251.log.gz 2023-10-16T11:22:59.243 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177175.log.gz 2023-10-16T11:22:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179404.log 2023-10-16T11:22:59.243 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144541.log 2023-10-16T11:22:59.244 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179404.log: /var/log/ceph/ceph-client.admin.144541.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.179404.log.gz 2023-10-16T11:22:59.244 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144541.log.gz 2023-10-16T11:22:59.244 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160849.log 2023-10-16T11:22:59.244 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186863.log 2023-10-16T11:22:59.245 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160849.log: /var/log/ceph/ceph-client.admin.186863.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160849.log.gz 2023-10-16T11:22:59.245 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.186863.log.gz 2023-10-16T11:22:59.245 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115619.log 2023-10-16T11:22:59.246 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120178.log 2023-10-16T11:22:59.246 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111368.log 2023-10-16T11:22:59.246 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115619.log.gz 2023-10-16T11:22:59.246 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120178.log: /var/log/ceph/ceph-client.admin.111368.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120178.log.gz 2023-10-16T11:22:59.247 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111368.log.gz 2023-10-16T11:22:59.247 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.175927.log 2023-10-16T11:22:59.247 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133702.log 2023-10-16T11:22:59.248 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.175927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.175927.log.gz 2023-10-16T11:22:59.248 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133702.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133702.log.gz 2023-10-16T11:22:59.248 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135642.log 2023-10-16T11:22:59.248 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-16T11:22:59.249 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.135642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190140.log 2023-10-16T11:22:59.249 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.1.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135642.log.gz 2023-10-16T11:22:59.249 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190140.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.190140.log.gz 2023-10-16T11:22:59.250 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183488.log 2023-10-16T11:22:59.250 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180718.log 2023-10-16T11:22:59.250 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183488.log: /var/log/ceph/ceph-client.admin.180718.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183488.log.gz 2023-10-16T11:22:59.250 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.180718.log.gz 2023-10-16T11:22:59.251 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142874.log 2023-10-16T11:22:59.251 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188371.log 2023-10-16T11:22:59.251 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142874.log: /var/log/ceph/ceph-client.admin.188371.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142874.log.gz 2023-10-16T11:22:59.251 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188371.log.gz 2023-10-16T11:22:59.252 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191883.log 2023-10-16T11:22:59.252 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190950.log 2023-10-16T11:22:59.252 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191883.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.191883.log.gz 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190950.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.190950.log.gz 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127226.log 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140417.log 2023-10-16T11:22:59.253 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127226.log: /var/log/ceph/ceph-client.admin.140417.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127226.log.gz 2023-10-16T11:22:59.254 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140417.log.gz 2023-10-16T11:22:59.254 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181949.log 2023-10-16T11:22:59.254 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144817.log 2023-10-16T11:22:59.255 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181949.log: /var/log/ceph/ceph-client.admin.144817.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.181949.log.gz 2023-10-16T11:22:59.255 INFO:teuthology.orchestra.run.smithi181.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.144817.log.gz 2023-10-16T11:22:59.255 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174727.log 2023-10-16T11:22:59.255 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114207.log 2023-10-16T11:22:59.256 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174727.log: /var/log/ceph/ceph-client.admin.114207.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.174727.log.gz 2023-10-16T11:22:59.256 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114207.log.gz 2023-10-16T11:22:59.256 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151535.log 2023-10-16T11:22:59.256 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186609.log 2023-10-16T11:22:59.257 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151535.log: /var/log/ceph/ceph-client.admin.186609.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151535.log.gz 2023-10-16T11:22:59.257 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.186609.log.gz 2023-10-16T11:22:59.257 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170034.log 2023-10-16T11:22:59.257 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191716.log 2023-10-16T11:22:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170034.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170034.log.gz 2023-10-16T11:22:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.191716.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.191716.log.gz 2023-10-16T11:22:59.258 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148397.log 2023-10-16T11:22:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151119.log 2023-10-16T11:22:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148397.log: /var/log/ceph/ceph-client.admin.151119.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.148397.log.gz 2023-10-16T11:22:59.259 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151119.log.gz 2023-10-16T11:22:59.259 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111004.log 2023-10-16T11:22:59.260 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189346.log 2023-10-16T11:22:59.260 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111004.log: /var/log/ceph/ceph-client.admin.189346.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111004.log.gz 2023-10-16T11:22:59.260 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189346.log.gz 2023-10-16T11:22:59.260 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181444.log 2023-10-16T11:22:59.261 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127975.log 2023-10-16T11:22:59.261 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181444.log: /var/log/ceph/ceph-client.admin.127975.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.181444.log.gz 2023-10-16T11:22:59.261 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127975.log.gz 2023-10-16T11:22:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173228.log 2023-10-16T11:22:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132344.log 2023-10-16T11:22:59.262 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173228.log: /var/log/ceph/ceph-client.admin.132344.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.173228.log.gz 2023-10-16T11:22:59.262 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132344.log.gz 2023-10-16T11:22:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119396.log 2023-10-16T11:22:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121766.log 2023-10-16T11:22:59.263 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119396.log: /var/log/ceph/ceph-client.admin.121766.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119396.log.gz 2023-10-16T11:22:59.263 INFO:teuthology.orchestra.run.smithi181.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.121766.log.gz 2023-10-16T11:22:59.264 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117663.log 2023-10-16T11:22:59.264 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125905.log 2023-10-16T11:22:59.264 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117663.log: /var/log/ceph/ceph-client.admin.125905.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117663.log.gz 2023-10-16T11:22:59.264 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125905.log.gz 2023-10-16T11:22:59.265 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127561.log 2023-10-16T11:22:59.265 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124110.log 2023-10-16T11:22:59.265 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127561.log: /var/log/ceph/ceph-client.admin.124110.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127561.log.gz 2023-10-16T11:22:59.266 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124110.log.gz 2023-10-16T11:22:59.266 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133794.log 2023-10-16T11:22:59.266 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126007.log 2023-10-16T11:22:59.266 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133794.log: /var/log/ceph/ceph-client.admin.126007.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133794.log.gz 2023-10-16T11:22:59.267 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126007.log.gz 2023-10-16T11:22:59.267 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168484.log 2023-10-16T11:22:59.267 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114990.log 2023-10-16T11:22:59.268 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168484.log: /var/log/ceph/ceph-client.admin.114990.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168484.log.gz 2023-10-16T11:22:59.268 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114990.log.gz 2023-10-16T11:22:59.268 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128910.log 2023-10-16T11:22:59.268 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138001.log 2023-10-16T11:22:59.269 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128910.log: /var/log/ceph/ceph-client.admin.138001.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128910.log.gz 2023-10-16T11:22:59.269 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138001.log.gz 2023-10-16T11:22:59.269 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188522.log 2023-10-16T11:22:59.269 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132539.log 2023-10-16T11:22:59.270 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188522.log: /var/log/ceph/ceph-client.admin.132539.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.188522.log.gz 2023-10-16T11:22:59.270 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132539.log.gz 2023-10-16T11:22:59.270 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113282.log 2023-10-16T11:22:59.271 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127373.log 2023-10-16T11:22:59.271 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.113282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113282.log.gz 2023-10-16T11:22:59.271 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127373.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127373.log.gz 2023-10-16T11:22:59.271 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184004.log 2023-10-16T11:22:59.272 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193080.log 2023-10-16T11:22:59.272 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.184004.log: /var/log/ceph/ceph-client.admin.193080.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.184004.log.gz 2023-10-16T11:22:59.272 INFO:teuthology.orchestra.run.smithi181.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.193080.log.gz 2023-10-16T11:22:59.273 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167968.log 2023-10-16T11:22:59.273 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169048.log 2023-10-16T11:22:59.273 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.167968.log: /var/log/ceph/ceph-client.admin.169048.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167968.log.gz 2023-10-16T11:22:59.273 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.169048.log.gz 2023-10-16T11:22:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119809.log 2023-10-16T11:22:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170266.log 2023-10-16T11:22:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.119809.log: /var/log/ceph/ceph-client.admin.170266.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119809.log.gz 2023-10-16T11:22:59.274 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169934.log 2023-10-16T11:22:59.275 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170266.log.gz 2023-10-16T11:22:59.275 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192235.log 2023-10-16T11:22:59.275 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169934.log: /var/log/ceph/ceph-client.admin.192235.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169934.log.gz 2023-10-16T11:22:59.276 INFO:teuthology.orchestra.run.smithi181.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.192235.log.gz 2023-10-16T11:22:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169232.log 2023-10-16T11:22:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168955.log 2023-10-16T11:22:59.276 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169232.log: /var/log/ceph/ceph-client.admin.168955.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169232.log.gz 2023-10-16T11:22:59.277 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168955.log.gz 2023-10-16T11:22:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132815.log 2023-10-16T11:22:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165046.log 2023-10-16T11:22:59.277 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132815.log: /var/log/ceph/ceph-client.admin.165046.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132815.log.gz 2023-10-16T11:22:59.278 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165046.log.gz 2023-10-16T11:22:59.278 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133426.log 2023-10-16T11:22:59.278 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133472.log 2023-10-16T11:22:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133426.log: /var/log/ceph/ceph-client.admin.133472.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133426.log.gz 2023-10-16T11:22:59.279 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133472.log.gz 2023-10-16T11:22:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178271.log 2023-10-16T11:22:59.279 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160952.log 2023-10-16T11:22:59.280 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178271.log: /var/log/ceph/ceph-client.admin.160952.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178271.log.gz 2023-10-16T11:22:59.280 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160952.log.gz 2023-10-16T11:22:59.280 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116981.log 2023-10-16T11:22:59.280 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193977.log 2023-10-16T11:22:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116981.log: /var/log/ceph/ceph-client.admin.193977.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116981.log.gz 2023-10-16T11:22:59.281 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193977.log.gz 2023-10-16T11:22:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176357.log 2023-10-16T11:22:59.281 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153165.log 2023-10-16T11:22:59.282 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176357.log: /var/log/ceph/ceph-client.admin.153165.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.176357.log.gz 2023-10-16T11:22:59.282 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153165.log.gz 2023-10-16T11:22:59.282 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153441.log 2023-10-16T11:22:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182643.log 2023-10-16T11:22:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153441.log: /var/log/ceph/ceph-client.admin.182643.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153441.log.gz 2023-10-16T11:22:59.283 INFO:teuthology.orchestra.run.smithi181.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.182643.log.gz 2023-10-16T11:22:59.283 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173746.log 2023-10-16T11:22:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117295.log 2023-10-16T11:22:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173746.log: /var/log/ceph/ceph-client.admin.117295.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173746.log.gz 2023-10-16T11:22:59.284 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117295.log.gz 2023-10-16T11:22:59.284 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150548.log 2023-10-16T11:22:59.285 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163011.log 2023-10-16T11:22:59.285 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150548.log: /var/log/ceph/ceph-client.admin.163011.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.150548.log.gz 2023-10-16T11:22:59.285 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163011.log.gz 2023-10-16T11:22:59.286 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133518.log 2023-10-16T11:22:59.286 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153992.log 2023-10-16T11:22:59.286 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133518.log: /var/log/ceph/ceph-client.admin.153992.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133518.log.gz 2023-10-16T11:22:59.286 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153992.log.gz 2023-10-16T11:22:59.287 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112086.log 2023-10-16T11:22:59.287 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164330.log 2023-10-16T11:22:59.287 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112086.log: /var/log/ceph/ceph-client.admin.164330.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112086.log.gz 2023-10-16T11:22:59.287 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164330.log.gz 2023-10-16T11:22:59.288 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131408.log 2023-10-16T11:22:59.288 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112362.log 2023-10-16T11:22:59.288 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131408.log: /var/log/ceph/ceph-client.admin.112362.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131408.log.gz 2023-10-16T11:22:59.289 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112362.log.gz 2023-10-16T11:22:59.289 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170358.log 2023-10-16T11:22:59.289 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126470.log 2023-10-16T11:22:59.289 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170358.log: /var/log/ceph/ceph-client.admin.126470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.170358.log.gz 2023-10-16T11:22:59.290 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126470.log.gz 2023-10-16T11:22:59.290 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192766.log 2023-10-16T11:22:59.290 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169700.log 2023-10-16T11:22:59.291 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192766.log: /var/log/ceph/ceph-client.admin.169700.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192766.log.gz 2023-10-16T11:22:59.291 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169700.log.gz 2023-10-16T11:22:59.291 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125640.log 2023-10-16T11:22:59.291 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182413.log 2023-10-16T11:22:59.292 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.125640.log: /var/log/ceph/ceph-client.admin.182413.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125640.log.gz 2023-10-16T11:22:59.292 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.182413.log.gz 2023-10-16T11:22:59.292 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131508.log 2023-10-16T11:22:59.292 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161898.log 2023-10-16T11:22:59.293 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131508.log: /var/log/ceph/ceph-client.admin.161898.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131508.log.gz 2023-10-16T11:22:59.293 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161898.log.gz 2023-10-16T11:22:59.293 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153211.log 2023-10-16T11:22:59.294 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137174.log 2023-10-16T11:22:59.294 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153211.log: /var/log/ceph/ceph-client.admin.137174.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153211.log.gz 2023-10-16T11:22:59.294 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137174.log.gz 2023-10-16T11:22:59.294 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180672.log 2023-10-16T11:22:59.295 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186517.log 2023-10-16T11:22:59.295 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180672.log: /var/log/ceph/ceph-client.admin.186517.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.180672.log.gz 2023-10-16T11:22:59.295 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.186517.log.gz 2023-10-16T11:22:59.295 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168786.log 2023-10-16T11:22:59.296 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125541.log 2023-10-16T11:22:59.296 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168786.log: /var/log/ceph/ceph-client.admin.125541.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168786.log.gz 2023-10-16T11:22:59.296 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125541.log.gz 2023-10-16T11:22:59.297 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169002.log 2023-10-16T11:22:59.297 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148763.log 2023-10-16T11:22:59.297 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169002.log: /var/log/ceph/ceph-client.admin.148763.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.169002.log.gz 2023-10-16T11:22:59.297 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148763.log.gz 2023-10-16T11:22:59.298 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160010.log 2023-10-16T11:22:59.298 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176217.log 2023-10-16T11:22:59.298 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160010.log: /var/log/ceph/ceph-client.admin.176217.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.160010.log.gz 2023-10-16T11:22:59.298 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176217.log.gz 2023-10-16T11:22:59.299 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133748.log 2023-10-16T11:22:59.299 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193034.log 2023-10-16T11:22:59.299 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133748.log: /var/log/ceph/ceph-client.admin.193034.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.133748.log.gz 2023-10-16T11:22:59.300 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.193034.log.gz 2023-10-16T11:22:59.300 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192526.log 2023-10-16T11:22:59.300 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117859.log 2023-10-16T11:22:59.300 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192526.log: /var/log/ceph/ceph-client.admin.117859.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.192526.log.gz 2023-10-16T11:22:59.301 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117859.log.gz 2023-10-16T11:22:59.301 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185401.log 2023-10-16T11:22:59.301 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178221.log 2023-10-16T11:22:59.301 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185401.log: /var/log/ceph/ceph-client.admin.178221.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.185401.log.gz 2023-10-16T11:22:59.302 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.178221.log.gz 2023-10-16T11:22:59.302 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193561.log 2023-10-16T11:22:59.302 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162961.log 2023-10-16T11:22:59.303 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193561.log: /var/log/ceph/ceph-client.admin.162961.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.193561.log.gz 2023-10-16T11:22:59.303 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162961.log.gz 2023-10-16T11:22:59.303 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141681.log 2023-10-16T11:22:59.303 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161230.log 2023-10-16T11:22:59.304 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.141681.log: /var/log/ceph/ceph-client.admin.161230.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141681.log.gz 2023-10-16T11:22:59.304 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161230.log.gz 2023-10-16T11:22:59.304 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.187117.log 2023-10-16T11:22:59.304 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111602.log 2023-10-16T11:22:59.305 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.187117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.187117.log.gz 2023-10-16T11:22:59.305 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111602.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111602.log.gz 2023-10-16T11:22:59.305 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128957.log 2023-10-16T11:22:59.306 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116337.log 2023-10-16T11:22:59.306 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128957.log: /var/log/ceph/ceph-client.admin.116337.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128957.log.gz 2023-10-16T11:22:59.306 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116337.log.gz 2023-10-16T11:22:59.306 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142308.log 2023-10-16T11:22:59.307 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116521.log 2023-10-16T11:22:59.307 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142308.log: /var/log/ceph/ceph-client.admin.116521.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142308.log.gz 2023-10-16T11:22:59.307 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116521.log.gz 2023-10-16T11:22:59.308 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123192.log 2023-10-16T11:22:59.308 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153579.log 2023-10-16T11:22:59.308 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.123192.log: /var/log/ceph/ceph-client.admin.153579.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123192.log.gz 2023-10-16T11:22:59.308 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.153579.log.gz 2023-10-16T11:22:59.309 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180811.log 2023-10-16T11:22:59.309 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173546.log 2023-10-16T11:22:59.309 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180811.log: /var/log/ceph/ceph-client.admin.173546.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.180811.log.gz 2023-10-16T11:22:59.309 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173546.log.gz 2023-10-16T11:22:59.310 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175690.log 2023-10-16T11:22:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139066.log 2023-10-16T11:22:59.351 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175690.log: /var/log/ceph/ceph-client.admin.139066.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175690.log.gz 2023-10-16T11:22:59.351 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139066.log.gz 2023-10-16T11:22:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146924.log 2023-10-16T11:22:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110997.log 2023-10-16T11:22:59.352 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.146924.log: /var/log/ceph/ceph-client.admin.110997.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146924.log.gz 2023-10-16T11:22:59.353 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110997.log.gz 2023-10-16T11:22:59.353 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127051.log 2023-10-16T11:22:59.353 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125958.log 2023-10-16T11:22:59.353 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127051.log: /var/log/ceph/ceph-client.admin.125958.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127051.log.gz 2023-10-16T11:22:59.354 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125958.log.gz 2023-10-16T11:22:59.354 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163511.log 2023-10-16T11:22:59.354 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151168.log 2023-10-16T11:22:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163511.log: 87.7%/var/log/ceph/ceph-client.admin.151168.log: -- replaced with /var/log/ceph/ceph-client.admin.163511.log.gz 2023-10-16T11:22:59.355 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151168.log.gz 2023-10-16T11:22:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176829.log 2023-10-16T11:22:59.355 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180021.log 2023-10-16T11:22:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176829.log: /var/log/ceph/ceph-client.admin.180021.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.176829.log.gz 2023-10-16T11:22:59.356 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.180021.log.gz 2023-10-16T11:22:59.356 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167781.log 2023-10-16T11:22:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177899.log 2023-10-16T11:22:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.167781.log: /var/log/ceph/ceph-client.admin.177899.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167781.log.gz 2023-10-16T11:22:59.357 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177899.log.gz 2023-10-16T11:22:59.357 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152362.log 2023-10-16T11:22:59.358 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185831.log 2023-10-16T11:22:59.358 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152362.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152362.log.gz 2023-10-16T11:22:59.358 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182087.log 2023-10-16T11:22:59.359 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.185831.log.gz 2023-10-16T11:22:59.359 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123498.log 2023-10-16T11:22:59.359 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182087.log: /var/log/ceph/ceph-client.admin.123498.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.182087.log.gz 2023-10-16T11:22:59.359 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123498.log.gz 2023-10-16T11:22:59.360 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163163.log 2023-10-16T11:22:59.360 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167917.log 2023-10-16T11:22:59.360 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163163.log: /var/log/ceph/ceph-client.admin.167917.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163163.log.gz 2023-10-16T11:22:59.360 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.167917.log.gz 2023-10-16T11:22:59.361 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175045.log 2023-10-16T11:22:59.361 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116065.log 2023-10-16T11:22:59.361 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175045.log: /var/log/ceph/ceph-client.admin.116065.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175045.log.gz 2023-10-16T11:22:59.362 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116065.log.gz 2023-10-16T11:22:59.362 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112132.log 2023-10-16T11:22:59.362 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144355.log 2023-10-16T11:22:59.362 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112132.log: /var/log/ceph/ceph-client.admin.144355.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112132.log.gz 2023-10-16T11:22:59.363 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144355.log.gz 2023-10-16T11:22:59.363 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137220.log 2023-10-16T11:22:59.363 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191618.log 2023-10-16T11:22:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137220.log: /var/log/ceph/ceph-client.admin.191618.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137220.log.gz 2023-10-16T11:22:59.364 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191618.log.gz 2023-10-16T11:22:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152699.log 2023-10-16T11:22:59.364 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187729.log 2023-10-16T11:22:59.365 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152699.log: /var/log/ceph/ceph-client.admin.187729.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.152699.log.gz 2023-10-16T11:22:59.365 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.187729.log.gz 2023-10-16T11:22:59.365 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126758.log 2023-10-16T11:22:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145396.log 2023-10-16T11:22:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126758.log: /var/log/ceph/ceph-client.admin.145396.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.126758.log.gz 2023-10-16T11:22:59.366 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145396.log.gz 2023-10-16T11:22:59.366 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180067.log 2023-10-16T11:22:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173443.log 2023-10-16T11:22:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180067.log: /var/log/ceph/ceph-client.admin.173443.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.180067.log.gz 2023-10-16T11:22:59.367 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173443.log.gz 2023-10-16T11:22:59.367 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170910.log 2023-10-16T11:22:59.368 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142357.log 2023-10-16T11:22:59.368 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.170910.log: /var/log/ceph/ceph-client.admin.142357.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170910.log.gz 2023-10-16T11:22:59.368 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132076.log 2023-10-16T11:22:59.369 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142357.log.gz 2023-10-16T11:22:59.369 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177013.log 2023-10-16T11:22:59.369 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132076.log: /var/log/ceph/ceph-client.admin.177013.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132076.log.gz 2023-10-16T11:22:59.369 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177013.log.gz 2023-10-16T11:22:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138549.log 2023-10-16T11:22:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163112.log 2023-10-16T11:22:59.370 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138549.log: /var/log/ceph/ceph-client.admin.163112.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138549.log.gz 2023-10-16T11:22:59.370 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.163112.log.gz 2023-10-16T11:22:59.371 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127272.log 2023-10-16T11:22:59.371 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137803.log 2023-10-16T11:22:59.371 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127272.log: /var/log/ceph/ceph-client.0.137803.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127272.log.gz 2023-10-16T11:22:59.372 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.137803.log.gz 2023-10-16T11:22:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151266.log 2023-10-16T11:22:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194067.log 2023-10-16T11:22:59.372 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151266.log: /var/log/ceph/ceph-client.admin.194067.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151266.log.gz 2023-10-16T11:22:59.373 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.194067.log.gz 2023-10-16T11:22:59.373 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174115.log 2023-10-16T11:22:59.373 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169370.log 2023-10-16T11:22:59.373 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174115.log: /var/log/ceph/ceph-client.admin.169370.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.174115.log.gz 2023-10-16T11:22:59.374 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169370.log.gz 2023-10-16T11:22:59.374 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182459.log 2023-10-16T11:22:59.374 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171320.log 2023-10-16T11:22:59.375 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182459.log: /var/log/ceph/ceph-client.admin.171320.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.182459.log.gz 2023-10-16T11:22:59.375 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171320.log.gz 2023-10-16T11:22:59.375 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178650.log 2023-10-16T11:22:59.375 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193929.log 2023-10-16T11:22:59.376 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178650.log: /var/log/ceph/ceph-client.admin.193929.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178650.log.gz 2023-10-16T11:22:59.376 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193929.log.gz 2023-10-16T11:22:59.376 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174391.log 2023-10-16T11:22:59.376 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116613.log 2023-10-16T11:22:59.377 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174391.log: /var/log/ceph/ceph-client.admin.116613.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174391.log.gz 2023-10-16T11:22:59.377 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116613.log.gz 2023-10-16T11:22:59.377 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188057.log 2023-10-16T11:22:59.378 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125688.log 2023-10-16T11:22:59.378 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188057.log: /var/log/ceph/ceph-client.admin.125688.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188057.log.gz 2023-10-16T11:22:59.378 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125688.log.gz 2023-10-16T11:22:59.378 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114622.log 2023-10-16T11:22:59.379 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170220.log 2023-10-16T11:22:59.379 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114622.log: /var/log/ceph/ceph-client.admin.170220.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114622.log.gz 2023-10-16T11:22:59.379 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170220.log.gz 2023-10-16T11:22:59.379 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.145124.log 2023-10-16T11:22:59.380 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182275.log 2023-10-16T11:22:59.380 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.145124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.145124.log.gz 2023-10-16T11:22:59.380 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182275.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.182275.log.gz 2023-10-16T11:22:59.381 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149085.log 2023-10-16T11:22:59.381 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147144.log 2023-10-16T11:22:59.381 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.149085.log: /var/log/ceph/ceph-client.admin.147144.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149085.log.gz 2023-10-16T11:22:59.381 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147144.log.gz 2023-10-16T11:22:59.382 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160148.log 2023-10-16T11:22:59.382 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190353.log 2023-10-16T11:22:59.382 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160148.log: /var/log/ceph/ceph-client.admin.190353.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160148.log.gz 2023-10-16T11:22:59.383 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190353.log.gz 2023-10-16T11:22:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140057.log 2023-10-16T11:22:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176079.log 2023-10-16T11:22:59.383 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140057.log: /var/log/ceph/ceph-client.admin.176079.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140057.log.gz 2023-10-16T11:22:59.384 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.176079.log.gz 2023-10-16T11:22:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181099.log 2023-10-16T11:22:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114852.log 2023-10-16T11:22:59.384 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181099.log: /var/log/ceph/ceph-client.admin.114852.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181099.log.gz 2023-10-16T11:22:59.385 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114852.log.gz 2023-10-16T11:22:59.385 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118901.log 2023-10-16T11:22:59.385 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117117.log 2023-10-16T11:22:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118901.log: /var/log/ceph/ceph-client.admin.117117.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118901.log.gz 2023-10-16T11:22:59.386 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117117.log.gz 2023-10-16T11:22:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178600.log 2023-10-16T11:22:59.386 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143532.log 2023-10-16T11:22:59.387 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.178600.log: /var/log/ceph/ceph-client.admin.143532.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178600.log.gz 2023-10-16T11:22:59.387 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143532.log.gz 2023-10-16T11:22:59.387 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164862.log 2023-10-16T11:22:59.388 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171367.log 2023-10-16T11:22:59.388 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164862.log: /var/log/ceph/ceph-client.admin.171367.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164862.log.gz 2023-10-16T11:22:59.388 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177760.log 2023-10-16T11:22:59.388 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171367.log.gz 2023-10-16T11:22:59.389 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144887.log 2023-10-16T11:22:59.389 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177760.log: /var/log/ceph/ceph-client.admin.144887.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177760.log.gz 2023-10-16T11:22:59.389 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144887.log.gz 2023-10-16T11:22:59.389 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188106.log 2023-10-16T11:22:59.390 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165808.log 2023-10-16T11:22:59.390 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188106.log: /var/log/ceph/ceph-client.admin.165808.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.188106.log.gz 2023-10-16T11:22:59.390 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165808.log.gz 2023-10-16T11:22:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164770.log 2023-10-16T11:22:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138598.log 2023-10-16T11:22:59.391 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164770.log: /var/log/ceph/ceph-client.admin.138598.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.164770.log.gz 2023-10-16T11:22:59.391 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138598.log.gz 2023-10-16T11:22:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143486.log 2023-10-16T11:22:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149848.log 2023-10-16T11:22:59.392 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.143486.log: /var/log/ceph/ceph-client.admin.149848.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143486.log.gz 2023-10-16T11:22:59.392 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149848.log.gz 2023-10-16T11:22:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165000.log 2023-10-16T11:22:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153395.log 2023-10-16T11:22:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.165000.log: /var/log/ceph/ceph-client.admin.153395.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.165000.log.gz 2023-10-16T11:22:59.394 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153395.log.gz 2023-10-16T11:22:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183952.log 2023-10-16T11:22:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159416.log 2023-10-16T11:22:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.183952.log: /var/log/ceph/ceph-client.admin.159416.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.183952.log.gz 2023-10-16T11:22:59.395 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159416.log.gz 2023-10-16T11:22:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174057.log 2023-10-16T11:22:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131926.log 2023-10-16T11:22:59.395 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174057.log: /var/log/ceph/ceph-client.admin.131926.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174057.log.gz 2023-10-16T11:22:59.396 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.131926.log.gz 2023-10-16T11:22:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193172.log 2023-10-16T11:22:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116843.log 2023-10-16T11:22:59.396 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193172.log: /var/log/ceph/ceph-client.admin.116843.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.193172.log.gz 2023-10-16T11:22:59.397 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116843.log.gz 2023-10-16T11:22:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139016.log 2023-10-16T11:22:59.397 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148574.log 2023-10-16T11:22:59.398 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139016.log: /var/log/ceph/ceph-client.admin.148574.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139016.log.gz 2023-10-16T11:22:59.398 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148574.log.gz 2023-10-16T11:22:59.398 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160286.log 2023-10-16T11:22:59.398 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190002.log 2023-10-16T11:22:59.399 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160286.log: /var/log/ceph/ceph-client.admin.190002.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160286.log.gz 2023-10-16T11:22:59.399 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.190002.log.gz 2023-10-16T11:22:59.399 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176309.log 2023-10-16T11:22:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119017.log 2023-10-16T11:22:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176309.log: /var/log/ceph/ceph-client.admin.119017.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176309.log.gz 2023-10-16T11:22:59.400 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119017.log.gz 2023-10-16T11:22:59.400 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185355.log 2023-10-16T11:22:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110547.log 2023-10-16T11:22:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185355.log: /var/log/ceph/ceph-client.admin.110547.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185355.log.gz 2023-10-16T11:22:59.401 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110547.log.gz 2023-10-16T11:22:59.401 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140845.log 2023-10-16T11:22:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120037.log 2023-10-16T11:22:59.402 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140845.log: /var/log/ceph/ceph-client.admin.120037.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140845.log.gz 2023-10-16T11:22:59.402 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120037.log.gz 2023-10-16T11:22:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173129.log 2023-10-16T11:22:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132861.log 2023-10-16T11:22:59.403 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.173129.log: /var/log/ceph/ceph-client.admin.132861.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173129.log.gz 2023-10-16T11:22:59.403 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132861.log.gz 2023-10-16T11:22:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122030.log 2023-10-16T11:22:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163725.log 2023-10-16T11:22:59.404 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.122030.log: /var/log/ceph/ceph-client.admin.163725.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122030.log.gz 2023-10-16T11:22:59.404 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163725.log.gz 2023-10-16T11:22:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132447.log 2023-10-16T11:22:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179071.log 2023-10-16T11:22:59.405 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.132447.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132447.log.gz 2023-10-16T11:22:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139166.log 2023-10-16T11:22:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179071.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179071.log.gz 2023-10-16T11:22:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139166.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139166.log.gz 2023-10-16T11:22:59.406 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134235.log 2023-10-16T11:22:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136360.log 2023-10-16T11:22:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.134235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.134235.log.gz 2023-10-16T11:22:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136360.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136360.log.gz 2023-10-16T11:22:59.407 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167329.log 2023-10-16T11:22:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159964.log 2023-10-16T11:22:59.408 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.167329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162451.log 2023-10-16T11:22:59.408 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167329.log.gz 2023-10-16T11:22:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159964.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159964.log.gz 2023-10-16T11:22:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137496.log 2023-10-16T11:22:59.409 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162451.log: /var/log/ceph/ceph-client.admin.137496.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162451.log.gz 2023-10-16T11:22:59.409 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.137496.log.gz 2023-10-16T11:22:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188921.log 2023-10-16T11:22:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126942.log 2023-10-16T11:22:59.410 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188921.log: /var/log/ceph/ceph-client.admin.126942.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.188921.log.gz 2023-10-16T11:22:59.410 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126942.log.gz 2023-10-16T11:22:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174253.log 2023-10-16T11:22:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.190844.log 2023-10-16T11:22:59.411 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.174253.log: /var/log/ceph/ceph-client.0.190844.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174253.log.gz 2023-10-16T11:22:59.412 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.190844.log.gz 2023-10-16T11:22:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180297.log 2023-10-16T11:22:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133238.log 2023-10-16T11:22:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180297.log: /var/log/ceph/ceph-client.admin.133238.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.180297.log.gz 2023-10-16T11:22:59.413 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133238.log.gz 2023-10-16T11:22:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171101.log 2023-10-16T11:22:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174483.log 2023-10-16T11:22:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.171101.log: /var/log/ceph/ceph-client.admin.174483.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.171101.log.gz 2023-10-16T11:22:59.414 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.174483.log.gz 2023-10-16T11:22:59.414 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141899.log 2023-10-16T11:22:59.414 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116291.log 2023-10-16T11:22:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.141899.log: /var/log/ceph/ceph-client.admin.116291.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141899.log.gz 2023-10-16T11:22:59.415 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116291.log.gz 2023-10-16T11:22:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110172.log 2023-10-16T11:22:59.415 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159468.log 2023-10-16T11:22:59.416 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110172.log: /var/log/ceph/ceph-client.admin.159468.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110172.log.gz 2023-10-16T11:22:59.416 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159468.log.gz 2023-10-16T11:22:59.416 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159216.log 2023-10-16T11:22:59.416 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193791.log 2023-10-16T11:22:59.417 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159216.log: /var/log/ceph/ceph-client.admin.193791.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159216.log.gz 2023-10-16T11:22:59.417 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.193791.log.gz 2023-10-16T11:22:59.417 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190607.log 2023-10-16T11:22:59.418 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127318.log 2023-10-16T11:22:59.418 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190607.log: /var/log/ceph/ceph-client.admin.127318.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.190607.log.gz 2023-10-16T11:22:59.418 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127318.log.gz 2023-10-16T11:22:59.418 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168634.log 2023-10-16T11:22:59.419 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128324.log 2023-10-16T11:22:59.419 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168634.log: /var/log/ceph/ceph-client.0.128324.log: 87.8% 0.0% -- replaced with /var/log/ceph/ceph-client.0.128324.log.gz -- replaced with /var/log/ceph/ceph-client.admin.168634.log.gz 2023-10-16T11:22:59.419 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:22:59.419 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176967.log 2023-10-16T11:22:59.420 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179588.log 2023-10-16T11:22:59.420 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176967.log: /var/log/ceph/ceph-client.admin.179588.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176967.log.gz 2023-10-16T11:22:59.420 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.179588.log.gz 2023-10-16T11:22:59.421 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142925.log 2023-10-16T11:22:59.421 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149223.log 2023-10-16T11:22:59.421 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142925.log: /var/log/ceph/ceph-client.admin.149223.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142925.log.gz 2023-10-16T11:22:59.421 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149223.log.gz 2023-10-16T11:22:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179772.log 2023-10-16T11:22:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178549.log 2023-10-16T11:22:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179772.log: /var/log/ceph/ceph-client.admin.178549.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.179772.log.gz 2023-10-16T11:22:59.422 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.178549.log.gz 2023-10-16T11:22:59.423 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150436.log 2023-10-16T11:22:59.423 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193469.log 2023-10-16T11:22:59.423 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150436.log: /var/log/ceph/ceph-client.admin.193469.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150436.log.gz 2023-10-16T11:22:59.423 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.193469.log.gz 2023-10-16T11:22:59.424 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110999.log 2023-10-16T11:22:59.424 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149039.log 2023-10-16T11:22:59.424 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110999.log: /var/log/ceph/ceph-client.admin.149039.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110999.log.gz 2023-10-16T11:22:59.425 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149039.log.gz 2023-10-16T11:22:59.425 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109689.log 2023-10-16T11:22:59.425 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159315.log 2023-10-16T11:22:59.425 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.109689.log: /var/log/ceph/ceph-client.admin.159315.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109689.log.gz 2023-10-16T11:22:59.426 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159315.log.gz 2023-10-16T11:22:59.426 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180343.log 2023-10-16T11:22:59.426 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184535.log 2023-10-16T11:22:59.426 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.180343.log: /var/log/ceph/ceph-client.admin.184535.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180343.log.gz 2023-10-16T11:22:59.427 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.184535.log.gz 2023-10-16T11:22:59.427 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190299.log 2023-10-16T11:22:59.427 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189720.log 2023-10-16T11:22:59.428 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.190299.log: /var/log/ceph/ceph-client.admin.189720.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.190299.log.gz 2023-10-16T11:22:59.428 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.189720.log.gz 2023-10-16T11:22:59.428 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175185.log 2023-10-16T11:22:59.428 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189016.log 2023-10-16T11:22:59.429 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175185.log: /var/log/ceph/ceph-client.admin.189016.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175185.log.gz 2023-10-16T11:22:59.429 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.189016.log.gz 2023-10-16T11:22:59.429 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137034.log 2023-10-16T11:22:59.429 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186006.log 2023-10-16T11:22:59.430 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.137034.log: /var/log/ceph/ceph-client.admin.186006.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137034.log.gz 2023-10-16T11:22:59.430 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186006.log.gz 2023-10-16T11:22:59.430 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161389.log 2023-10-16T11:22:59.431 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116797.log 2023-10-16T11:22:59.431 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161389.log: /var/log/ceph/ceph-client.admin.116797.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161389.log.gz 2023-10-16T11:22:59.431 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116797.log.gz 2023-10-16T11:22:59.431 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176645.log 2023-10-16T11:22:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143578.log 2023-10-16T11:22:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176645.log: /var/log/ceph/ceph-client.admin.143578.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176645.log.gz 2023-10-16T11:22:59.432 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.143578.log.gz 2023-10-16T11:22:59.432 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187775.log 2023-10-16T11:22:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138126.log 2023-10-16T11:22:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.187775.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.187775.log.gz 2023-10-16T11:22:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.138126.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138126.log.gz 2023-10-16T11:22:59.433 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192720.log 2023-10-16T11:22:59.434 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189300.log 2023-10-16T11:22:59.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192720.log: /var/log/ceph/ceph-client.admin.189300.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.192720.log.gz 2023-10-16T11:22:59.434 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.189300.log.gz 2023-10-16T11:22:59.435 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133886.log 2023-10-16T11:22:59.435 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159050.log 2023-10-16T11:22:59.435 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133886.log: /var/log/ceph/ceph-client.admin.159050.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.133886.log.gz 2023-10-16T11:22:59.435 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159050.log.gz 2023-10-16T11:22:59.436 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151785.log 2023-10-16T11:22:59.436 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152830.log 2023-10-16T11:22:59.436 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.151785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181398.log 2023-10-16T11:22:59.437 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151785.log.gz 2023-10-16T11:22:59.437 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152830.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152830.log.gz 2023-10-16T11:22:59.437 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114299.log 2023-10-16T11:22:59.437 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181398.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.181398.log.gz 2023-10-16T11:22:59.438 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177295.log 2023-10-16T11:22:59.438 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114299.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114299.log.gz 2023-10-16T11:22:59.438 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163264.log 2023-10-16T11:22:59.439 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.177295.log: /var/log/ceph/ceph-client.admin.163264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177295.log.gz 2023-10-16T11:22:59.439 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163264.log.gz 2023-10-16T11:22:59.439 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128091.log 2023-10-16T11:22:59.439 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141050.log 2023-10-16T11:22:59.440 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128091.log: /var/log/ceph/ceph-client.admin.141050.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128091.log.gz 2023-10-16T11:22:59.440 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.141050.log.gz 2023-10-16T11:22:59.440 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159769.log 2023-10-16T11:22:59.440 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111948.log 2023-10-16T11:22:59.441 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159769.log: /var/log/ceph/ceph-client.admin.111948.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.159769.log.gz 2023-10-16T11:22:59.441 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111948.log.gz 2023-10-16T11:22:59.441 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116019.log 2023-10-16T11:22:59.441 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117027.log 2023-10-16T11:22:59.442 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.116019.log: /var/log/ceph/ceph-client.admin.117027.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116019.log.gz 2023-10-16T11:22:59.442 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117027.log.gz 2023-10-16T11:22:59.442 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110501.log 2023-10-16T11:22:59.443 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137080.log 2023-10-16T11:22:59.443 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110501.log: /var/log/ceph/ceph-client.admin.137080.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110501.log.gz 2023-10-16T11:22:59.443 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.137080.log.gz 2023-10-16T11:22:59.443 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168067.log 2023-10-16T11:22:59.444 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127929.log 2023-10-16T11:22:59.444 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168067.log: /var/log/ceph/ceph-client.admin.127929.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.168067.log.gz 2023-10-16T11:22:59.444 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127929.log.gz 2023-10-16T11:22:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175139.log 2023-10-16T11:22:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141510.log 2023-10-16T11:22:59.445 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.175139.log: /var/log/ceph/ceph-client.admin.141510.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175139.log.gz 2023-10-16T11:22:59.445 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141510.log.gz 2023-10-16T11:22:59.446 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112316.log 2023-10-16T11:22:59.446 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143440.log 2023-10-16T11:22:59.446 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112316.log: /var/log/ceph/ceph-client.admin.143440.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112316.log.gz 2023-10-16T11:22:59.446 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143440.log.gz 2023-10-16T11:22:59.447 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192433.log 2023-10-16T11:22:59.447 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126666.log 2023-10-16T11:22:59.447 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.192433.log: /var/log/ceph/ceph-client.admin.126666.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192433.log.gz 2023-10-16T11:22:59.447 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126666.log.gz 2023-10-16T11:22:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144138.log 2023-10-16T11:22:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113274.log 2023-10-16T11:22:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144138.log: /var/log/ceph/ceph-client.0.113274.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144138.log.gz 2023-10-16T11:22:59.449 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113274.log.gz 2023-10-16T11:22:59.449 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153303.log 2023-10-16T11:22:59.449 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115485.log 2023-10-16T11:22:59.449 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.153303.log: /var/log/ceph/ceph-client.admin.115485.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153303.log.gz 2023-10-16T11:22:59.450 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115485.log.gz 2023-10-16T11:22:59.450 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134346.log 2023-10-16T11:22:59.450 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135290.log 2023-10-16T11:22:59.450 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.134346.log: /var/log/ceph/ceph-client.admin.135290.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134346.log.gz 2023-10-16T11:22:59.451 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135290.log.gz 2023-10-16T11:22:59.451 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127837.log 2023-10-16T11:22:59.451 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118329.log 2023-10-16T11:22:59.452 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127837.log: /var/log/ceph/ceph-client.admin.118329.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127837.log.gz 2023-10-16T11:22:59.452 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118329.log.gz 2023-10-16T11:22:59.452 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193126.log 2023-10-16T11:22:59.452 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152224.log 2023-10-16T11:22:59.453 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193126.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.193126.log.gz 2023-10-16T11:22:59.453 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152224.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152224.log.gz 2023-10-16T11:22:59.453 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189546.log 2023-10-16T11:22:59.454 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152029.log 2023-10-16T11:22:59.454 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189546.log: /var/log/ceph/ceph-client.admin.152029.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.189546.log.gz 2023-10-16T11:22:59.454 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152029.log.gz 2023-10-16T11:22:59.454 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168534.log 2023-10-16T11:22:59.455 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103891.log 2023-10-16T11:22:59.455 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168534.log: /var/log/ceph/ceph-client.admin.103891.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168534.log.gz 2023-10-16T11:22:59.455 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103891.log.gz 2023-10-16T11:22:59.455 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152784.log 2023-10-16T11:22:59.456 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187227.log 2023-10-16T11:22:59.456 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152784.log: /var/log/ceph/ceph-client.admin.187227.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152784.log.gz 2023-10-16T11:22:59.456 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.187227.log.gz 2023-10-16T11:22:59.457 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181306.log 2023-10-16T11:22:59.457 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114806.log 2023-10-16T11:22:59.457 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181306.log: /var/log/ceph/ceph-client.admin.114806.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181306.log.gz 2023-10-16T11:22:59.457 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114806.log.gz 2023-10-16T11:22:59.458 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163575.log 2023-10-16T11:22:59.458 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112546.log 2023-10-16T11:22:59.458 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.163575.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163575.log.gz 2023-10-16T11:22:59.458 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.112546.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.112546.log.gz 2023-10-16T11:22:59.459 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111994.log 2023-10-16T11:22:59.459 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167375.log 2023-10-16T11:22:59.459 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.111994.log: /var/log/ceph/ceph-client.admin.167375.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111994.log.gz 2023-10-16T11:22:59.459 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167375.log.gz 2023-10-16T11:22:59.460 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188875.log 2023-10-16T11:22:59.460 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139215.log 2023-10-16T11:22:59.460 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188875.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.188875.log.gz 2023-10-16T11:22:59.461 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139215.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139215.log.gz 2023-10-16T11:22:59.461 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159265.log 2023-10-16T11:22:59.461 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176875.log 2023-10-16T11:22:59.461 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159265.log: /var/log/ceph/ceph-client.admin.176875.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159265.log.gz 2023-10-16T11:22:59.462 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.176875.log.gz 2023-10-16T11:22:59.462 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189860.log 2023-10-16T11:22:59.462 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179864.log 2023-10-16T11:22:59.463 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189860.log: /var/log/ceph/ceph-client.admin.179864.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.189860.log.gz 2023-10-16T11:22:59.463 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179864.log.gz 2023-10-16T11:22:59.463 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194021.log 2023-10-16T11:22:59.463 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152178.log 2023-10-16T11:22:59.464 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.194021.log: /var/log/ceph/ceph-client.admin.152178.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.194021.log.gz 2023-10-16T11:22:59.464 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152178.log.gz 2023-10-16T11:22:59.464 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188322.log 2023-10-16T11:22:59.464 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127699.log 2023-10-16T11:22:59.465 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.188322.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.188322.log.gz 2023-10-16T11:22:59.465 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.127699.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.127699.log.gz 2023-10-16T11:22:59.465 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-16T11:22:59.466 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132298.log 2023-10-16T11:22:59.466 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.132298.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132298.log.gz 2023-10-16T11:22:59.466 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186425.log 2023-10-16T11:22:59.466 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.186425.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.186425.log.gz 2023-10-16T11:22:59.467 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140465.log 2023-10-16T11:22:59.467 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140465.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140465.log.gz 2023-10-16T11:22:59.467 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148809.log 2023-10-16T11:22:59.468 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148809.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148809.log.gz 2023-10-16T11:22:59.468 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121700.log 2023-10-16T11:22:59.468 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121700.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121700.log.gz 2023-10-16T11:22:59.468 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179358.log 2023-10-16T11:22:59.469 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.179358.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.179358.log.gz 2023-10-16T11:22:59.469 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144447.log 2023-10-16T11:22:59.469 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144447.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144447.log.gz 2023-10-16T11:22:59.469 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136514.log 2023-10-16T11:22:59.470 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136514.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136514.log.gz 2023-10-16T11:22:59.470 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185171.log 2023-10-16T11:22:59.470 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.185171.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.185171.log.gz 2023-10-16T11:22:59.471 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126360.log 2023-10-16T11:22:59.471 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126360.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126360.log.gz 2023-10-16T11:22:59.471 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114483.log 2023-10-16T11:22:59.471 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114483.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114483.log.gz 2023-10-16T11:22:59.472 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120413.log 2023-10-16T11:22:59.472 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.120413.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120413.log.gz 2023-10-16T11:22:59.472 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152609.log 2023-10-16T11:22:59.472 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.152609.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152609.log.gz 2023-10-16T11:22:59.473 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140232.log 2023-10-16T11:22:59.473 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.140232.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.140232.log.gz 2023-10-16T11:22:59.473 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.153886.log 2023-10-16T11:22:59.474 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.153886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.153886.log.gz 2023-10-16T11:22:59.474 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133932.log 2023-10-16T11:22:59.474 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133932.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.133932.log.gz 2023-10-16T11:22:59.474 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144771.log 2023-10-16T11:22:59.475 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.144771.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144771.log.gz 2023-10-16T11:22:59.475 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142123.log 2023-10-16T11:22:59.475 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142123.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142123.log.gz 2023-10-16T11:22:59.475 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121522.log 2023-10-16T11:22:59.476 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121522.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121522.log.gz 2023-10-16T11:22:59.476 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165092.log 2023-10-16T11:22:59.476 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.165092.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165092.log.gz 2023-10-16T11:22:59.477 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114668.log 2023-10-16T11:22:59.477 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.114668.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114668.log.gz 2023-10-16T11:22:59.477 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136874.log 2023-10-16T11:22:59.477 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.136874.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136874.log.gz 2023-10-16T11:22:59.478 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118375.log 2023-10-16T11:22:59.478 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.118375.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118375.log.gz 2023-10-16T11:22:59.478 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159617.log 2023-10-16T11:22:59.479 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.159617.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.159617.log.gz 2023-10-16T11:22:59.479 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.153882.log 2023-10-16T11:22:59.479 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.153882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.153882.log.gz 2023-10-16T11:22:59.479 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133146.log 2023-10-16T11:22:59.480 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.133146.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.133146.log.gz 2023-10-16T11:22:59.480 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193837.log 2023-10-16T11:22:59.480 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193837.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.193837.log.gz 2023-10-16T11:22:59.481 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189116.log 2023-10-16T11:22:59.481 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189116.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.189116.log.gz 2023-10-16T11:22:59.481 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117571.log 2023-10-16T11:22:59.481 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.117571.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117571.log.gz 2023-10-16T11:22:59.482 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176405.log 2023-10-16T11:22:59.482 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.176405.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176405.log.gz 2023-10-16T11:22:59.482 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-16T11:22:59.482 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150749.log 2023-10-16T11:22:59.483 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.150749.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.150749.log.gz 2023-10-16T11:22:59.483 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139367.log 2023-10-16T11:22:59.483 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.139367.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139367.log.gz 2023-10-16T11:22:59.484 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169478.log 2023-10-16T11:22:59.484 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.169478.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169478.log.gz 2023-10-16T11:22:59.484 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126850.log 2023-10-16T11:22:59.484 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.126850.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126850.log.gz 2023-10-16T11:22:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.181793.log 2023-10-16T11:22:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.181793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.181793.log.gz 2023-10-16T11:22:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158930.log 2023-10-16T11:22:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.158930.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158930.log.gz 2023-10-16T11:22:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189208.log 2023-10-16T11:22:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.189208.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.189208.log.gz 2023-10-16T11:22:59.486 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182551.log 2023-10-16T11:22:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.182551.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.182551.log.gz 2023-10-16T11:22:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115036.log 2023-10-16T11:22:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.115036.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.115036.log.gz 2023-10-16T11:22:59.487 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193331.log 2023-10-16T11:22:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.193331.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.193331.log.gz 2023-10-16T11:22:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142825.log 2023-10-16T11:22:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.142825.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142825.log.gz 2023-10-16T11:22:59.488 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148528.log 2023-10-16T11:22:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.148528.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148528.log.gz 2023-10-16T11:22:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128501.log 2023-10-16T11:22:59.489 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.128501.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.128501.log.gz 2023-10-16T11:22:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154250.log 2023-10-16T11:22:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.154250.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154250.log.gz 2023-10-16T11:22:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131976.log 2023-10-16T11:22:59.490 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.131976.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131976.log.gz 2023-10-16T11:22:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110731.log 2023-10-16T11:22:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.110731.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110731.log.gz 2023-10-16T11:22:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160240.log 2023-10-16T11:22:59.491 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.160240.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160240.log.gz 2023-10-16T11:22:59.492 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146826.log 2023-10-16T11:22:59.492 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.146826.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146826.log.gz 2023-10-16T11:22:59.492 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162647.log 2023-10-16T11:22:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.162647.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162647.log.gz 2023-10-16T11:22:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121430.log 2023-10-16T11:22:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.121430.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121430.log.gz 2023-10-16T11:22:59.493 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181260.log 2023-10-16T11:22:59.494 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.181260.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181260.log.gz 2023-10-16T11:22:59.494 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164245.log 2023-10-16T11:22:59.494 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.164245.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164245.log.gz 2023-10-16T11:22:59.495 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168585.log 2023-10-16T11:22:59.495 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.168585.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.168585.log.gz 2023-10-16T11:22:59.495 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161850.log 2023-10-16T11:22:59.495 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.161850.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161850.log.gz 2023-10-16T11:22:59.509 INFO:teuthology.orchestra.run.smithi181.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-16T11:22:59.857 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-16T11:23:00.634 INFO:teuthology.orchestra.run.smithi190.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-16T11:23:00.646 INFO:teuthology.orchestra.run.smithi190.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-16T11:23:01.295 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-16T11:23:01.336 INFO:teuthology.orchestra.run.smithi181.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-16T11:23:01.497 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-16T11:23:01.799 INFO:teuthology.orchestra.run.smithi190.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-16T11:23:01.953 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-16T11:23:02.203 INFO:teuthology.orchestra.run.smithi181.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-16T11:23:02.336 INFO:teuthology.orchestra.run.smithi181.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-16T11:23:02.430 INFO:teuthology.orchestra.run.smithi190.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-16T11:23:02.431 INFO:teuthology.orchestra.run.smithi190.stderr: 2023-10-16T11:23:02.432 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m3.826s 2023-10-16T11:23:02.432 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m18.172s 2023-10-16T11:23:02.432 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.644s 2023-10-16T11:23:02.534 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-16T11:23:02.944 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-16T11:23:02.946 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-16T11:23:02.946 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m4.373s 2023-10-16T11:23:02.946 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m20.301s 2023-10-16T11:23:02.946 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m1.214s 2023-10-16T11:23:02.947 INFO:tasks.ceph:Archiving logs... 2023-10-16T11:23:02.947 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/remote/smithi181/log 2023-10-16T11:23:02.949 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T11:23:06.372 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/remote/smithi190/log 2023-10-16T11:23:06.373 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T11:23:08.602 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-16T11:23:08.611 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-16T11:23:08.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T11:23:08.615 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T11:23:08.706 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-16T11:23:08.716 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-16T11:23:08.716 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T11:23:08.719 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T11:23:08.734 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-10-16T11:23:08.737 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-10-16T11:23:08.739 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T11:23:08.739 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-16T11:23:08.740 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 +1121us[+1121us] +/- 90ms 2023-10-16T11:23:08.740 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -692us[ -728us] +/- 72ms 2023-10-16T11:23:08.740 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 120 -210us[ -253us] +/- 77ms 2023-10-16T11:23:08.740 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T11:23:08.743 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T11:23:08.743 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-10-16T11:23:08.743 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 57 +1131us[+1131us] +/- 90ms 2023-10-16T11:23:08.744 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -719us[ -720us] +/- 72ms 2023-10-16T11:23:08.744 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 123 -251us[ -254us] +/- 77ms 2023-10-16T11:23:08.744 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T11:23:08.745 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-16T11:23:08.756 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-16T11:23:08.756 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-16T11:23:08.765 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-16T11:23:08.843 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-16T11:23:08.950 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T11:23:08.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:23:08.980 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T11:23:09.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:23:09.015 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-16T11:23:09.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-16T11:23:09.039 INFO:teuthology.task.internal:Duration was 6671.111021 seconds 2023-10-16T11:23:09.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-16T11:23:09.048 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-16T11:23:09.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T11:23:09.051 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T11:23:09.095 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T11:23:09.102 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T11:23:09.420 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-16T11:23:09.421 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-10-16T11:23:09.421 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T11:23:09.441 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2023-10-16T11:23:09.442 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T11:23:09.464 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-16T11:23:09.464 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T11:23:09.484 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T11:23:09.548 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-16T11:23:09.549 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T11:23:09.551 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T11:23:09.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-16T11:23:09.682 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-16T11:23:09.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T11:23:09.710 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T11:23:09.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-16T11:23:09.748 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:23:09.753 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:23:09.777 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-10-16T11:23:09.803 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2023-10-16T11:23:09.817 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:23:09.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:23:09.847 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T11:23:09.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T11:23:09.874 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-16T11:23:09.885 INFO:teuthology.task.internal:Transferring archived files... 2023-10-16T11:23:09.886 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/remote/smithi181 2023-10-16T11:23:09.886 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T11:23:09.923 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969/remote/smithi190 2023-10-16T11:23:09.924 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T11:23:09.962 INFO:teuthology.task.internal:Removing archive directory... 2023-10-16T11:23:09.963 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T11:23:09.965 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T11:23:10.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-16T11:23:10.025 INFO:teuthology.task.internal:Not uploading archives. 2023-10-16T11:23:10.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-16T11:23:10.037 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-16T11:23:10.037 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T11:23:10.040 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T11:23:10.053 INFO:teuthology.orchestra.run.smithi181.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 11:23 /home/ubuntu/cephtest 2023-10-16T11:23:10.069 INFO:teuthology.orchestra.run.smithi190.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 11:23 /home/ubuntu/cephtest 2023-10-16T11:23:10.070 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-16T11:23:10.081 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-16T11:23:10.103 INFO:teuthology.nuke:Checking targets against current locks 2023-10-16T11:23:10.133 DEBUG:teuthology.nuke:shortname: smithi181 2023-10-16T11:23:10.134 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T11:23:10.158 DEBUG:teuthology.nuke:shortname: smithi190 2023-10-16T11:23:10.159 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T11:23:10.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:53.871675', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T11:23:10.187 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_09:20:41-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428969', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 09:26:53.869712', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T11:23:10.214 INFO:teuthology.orchestra.console.smithi181:Power off 2023-10-16T11:23:10.215 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T11:23:10.239 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-10-16T11:23:10.240 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:23:10.256 INFO:teuthology.orchestra.console.smithi190:Power off 2023-10-16T11:23:10.257 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T11:23:10.268 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-16T11:23:10.279 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-10-16T11:23:10.280 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:23:10.304 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-10-16T11:23:14.269 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:23:14.306 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:23:14.384 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-16T11:23:14.416 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-10-16T11:23:18.389 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:23:18.417 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:23:18.499 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-10-16T11:23:18.527 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-10-16T11:23:18.600 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-10-16T11:23:22.530 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T11:23:22.641 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-10-16T11:23:22.742 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-10-16T11:23:22.848 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6671.11102104187 failure_reason: 'Test failure: test_snap_schedule_subvol_and_group_arguments (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)' flavor: default owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7e1be5092f8641caa7cee2892beef95b status: fail success: false 2023-10-16T11:23:22.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T11:23:22.937 INFO:teuthology.run:FAIL